2024-04-25T08:47:34.692 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T08:47:34.693 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_neesingh@teuthology --description fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/scrub} --verbose --name neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi --archive /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673004 -- /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673004/orig.config.yaml 2024-04-25T08:47:34.735 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T08:47:34.817 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673004 branch: wip-neesingh-testing-240416 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/scrub} email: neesingh@redhat.com first_in_suite: false job_id: '7673004' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-neesingh-testing-240416 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds cache max size: 1000 mds debug frag: true mds debug scatterstat: true mds log max segments: 1 mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - Replacing daemon mds - Scrub error on inode - Behind on trimming - Metadata damage detected - bad backtrace on inode - overall HEALTH_ - \(MDS_TRIM\) sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-62537 sha1: 5787d42d10024ba410e822a8aba474331cb9af02 owner: scheduled_neesingh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 5869 sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec sleep_before_teardown: 0 suite: fs:functional suite_branch: wip-62537 suite_path: /home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa suite_relpath: qa suite_repo: https://github.com/neesingh-rh/ceph suite_sha1: 5787d42d10024ba410e822a8aba474331cb9af02 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi043.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAlXJxRzvrFjsp6Th37vUptyCRLd/bRZfLL7KIsn9/jD51iOJAApGr4gLlNVDsheg40xYpAol3+nyQCi2TiVVpk= smithi193.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPGxWHh2OQKftzsvA8ZjW49N0cBd2B0i2ghi1EO18C1qyJQn5z1JrwJ7tqcpAbfHh2zLyZr2NhN8MBszz3zzeIc= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_scrub_checks - tasks.cephfs.test_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_08:15:18 tube: smithi user: neesingh verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T08:47:34.818 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa; will attempt to use it 2024-04-25T08:47:34.818 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa/tasks 2024-04-25T08:47:34.818 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T08:47:34.820 INFO:teuthology.task.internal:Checking packages... 2024-04-25T08:47:34.845 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec' 2024-04-25T08:47:34.846 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T08:47:34.846 INFO:teuthology.packaging:ref: None 2024-04-25T08:47:34.846 INFO:teuthology.packaging:tag: None 2024-04-25T08:47:34.846 INFO:teuthology.packaging:branch: wip-neesingh-testing-240416 2024-04-25T08:47:34.846 INFO:teuthology.packaging:sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T08:47:34.846 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-neesingh-testing-240416 2024-04-25T08:47:34.985 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1628.g97ae6cc9 2024-04-25T08:47:34.987 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T08:47:34.994 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T08:47:34.994 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T08:47:35.002 INFO:teuthology.task.internal:Saving configuration 2024-04-25T08:47:35.018 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T08:47:35.024 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T08:47:35.044 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi043.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673004', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 08:42:42.018161', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAlXJxRzvrFjsp6Th37vUptyCRLd/bRZfLL7KIsn9/jD51iOJAApGr4gLlNVDsheg40xYpAol3+nyQCi2TiVVpk='} 2024-04-25T08:47:35.065 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi193.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673004', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 08:42:42.016287', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPGxWHh2OQKftzsvA8ZjW49N0cBd2B0i2ghi1EO18C1qyJQn5z1JrwJ7tqcpAbfHh2zLyZr2NhN8MBszz3zzeIc='} 2024-04-25T08:47:35.065 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T08:47:35.071 INFO:teuthology.task.internal:roles: ubuntu@smithi043.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-25T08:47:35.071 INFO:teuthology.task.internal:roles: ubuntu@smithi193.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-25T08:47:35.072 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T08:47:35.138 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f5baf595160>, signals=[15]) 2024-04-25T08:47:35.139 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T08:47:35.145 INFO:teuthology.task.internal:Opening connections... 2024-04-25T08:47:35.145 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi043.front.sepia.ceph.com 2024-04-25T08:47:35.148 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:47:35.223 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi193.front.sepia.ceph.com 2024-04-25T08:47:35.223 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:47:35.293 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T08:47:35.299 DEBUG:teuthology.orchestra.run.smithi043:> uname -m 2024-04-25T08:47:35.317 INFO:teuthology.orchestra.run.smithi043.stdout:x86_64 2024-04-25T08:47:35.317 DEBUG:teuthology.orchestra.run.smithi043:> cat /etc/os-release 2024-04-25T08:47:35.374 INFO:teuthology.orchestra.run.smithi043.stdout:NAME="CentOS Stream" 2024-04-25T08:47:35.374 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION="9" 2024-04-25T08:47:35.375 INFO:teuthology.orchestra.run.smithi043.stdout:ID="centos" 2024-04-25T08:47:35.375 INFO:teuthology.orchestra.run.smithi043.stdout:ID_LIKE="rhel fedora" 2024-04-25T08:47:35.375 INFO:teuthology.orchestra.run.smithi043.stdout:VERSION_ID="9" 2024-04-25T08:47:35.375 INFO:teuthology.orchestra.run.smithi043.stdout:PLATFORM_ID="platform:el9" 2024-04-25T08:47:35.375 INFO:teuthology.orchestra.run.smithi043.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T08:47:35.375 INFO:teuthology.orchestra.run.smithi043.stdout:ANSI_COLOR="0;31" 2024-04-25T08:47:35.375 INFO:teuthology.orchestra.run.smithi043.stdout:LOGO="fedora-logo-icon" 2024-04-25T08:47:35.375 INFO:teuthology.orchestra.run.smithi043.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T08:47:35.375 INFO:teuthology.orchestra.run.smithi043.stdout:HOME_URL="https://centos.org/" 2024-04-25T08:47:35.375 INFO:teuthology.orchestra.run.smithi043.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T08:47:35.375 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T08:47:35.375 INFO:teuthology.orchestra.run.smithi043.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T08:47:35.376 INFO:teuthology.lock.ops:Updating smithi043.front.sepia.ceph.com on lock server 2024-04-25T08:47:35.400 DEBUG:teuthology.orchestra.run.smithi193:> uname -m 2024-04-25T08:47:35.417 INFO:teuthology.orchestra.run.smithi193.stdout:x86_64 2024-04-25T08:47:35.417 DEBUG:teuthology.orchestra.run.smithi193:> cat /etc/os-release 2024-04-25T08:47:35.474 INFO:teuthology.orchestra.run.smithi193.stdout:NAME="CentOS Stream" 2024-04-25T08:47:35.474 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION="9" 2024-04-25T08:47:35.474 INFO:teuthology.orchestra.run.smithi193.stdout:ID="centos" 2024-04-25T08:47:35.474 INFO:teuthology.orchestra.run.smithi193.stdout:ID_LIKE="rhel fedora" 2024-04-25T08:47:35.474 INFO:teuthology.orchestra.run.smithi193.stdout:VERSION_ID="9" 2024-04-25T08:47:35.474 INFO:teuthology.orchestra.run.smithi193.stdout:PLATFORM_ID="platform:el9" 2024-04-25T08:47:35.474 INFO:teuthology.orchestra.run.smithi193.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T08:47:35.475 INFO:teuthology.orchestra.run.smithi193.stdout:ANSI_COLOR="0;31" 2024-04-25T08:47:35.475 INFO:teuthology.orchestra.run.smithi193.stdout:LOGO="fedora-logo-icon" 2024-04-25T08:47:35.475 INFO:teuthology.orchestra.run.smithi193.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T08:47:35.475 INFO:teuthology.orchestra.run.smithi193.stdout:HOME_URL="https://centos.org/" 2024-04-25T08:47:35.475 INFO:teuthology.orchestra.run.smithi193.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T08:47:35.475 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T08:47:35.475 INFO:teuthology.orchestra.run.smithi193.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T08:47:35.475 INFO:teuthology.lock.ops:Updating smithi193.front.sepia.ceph.com on lock server 2024-04-25T08:47:35.491 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T08:47:35.499 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T08:47:35.503 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T08:47:35.503 DEBUG:teuthology.orchestra.run.smithi043:> test '!' -e /home/ubuntu/cephtest 2024-04-25T08:47:35.505 DEBUG:teuthology.orchestra.run.smithi193:> test '!' -e /home/ubuntu/cephtest 2024-04-25T08:47:35.530 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T08:47:35.535 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T08:47:35.535 DEBUG:teuthology.orchestra.run.smithi043:> test -z $(ls -A /var/lib/ceph) 2024-04-25T08:47:35.562 DEBUG:teuthology.orchestra.run.smithi193:> test -z $(ls -A /var/lib/ceph) 2024-04-25T08:47:35.595 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T08:47:35.639 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T08:47:35.652 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-04-25T08:47:35.653 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-04-25T08:47:35.653 DEBUG:teuthology.orchestra.run.smithi043:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T08:47:35.653 DEBUG:teuthology.orchestra.run.smithi193:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T08:47:35.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:47:35.670 DEBUG:teuthology.orchestra.run.smithi193:> uname -r 2024-04-25T08:47:35.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:47:35.671 DEBUG:teuthology.orchestra.run.smithi043:> uname -r 2024-04-25T08:47:35.726 INFO:teuthology.orchestra.run.smithi193.stdout:5.14.0-437.el9.x86_64 2024-04-25T08:47:35.727 INFO:teuthology.task.kernel:Running kernel on smithi193: 5.14.0-437.el9.x86_64 2024-04-25T08:47:35.727 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum install -y kernel 2024-04-25T08:47:35.728 INFO:teuthology.orchestra.run.smithi043.stdout:5.14.0-437.el9.x86_64 2024-04-25T08:47:35.728 INFO:teuthology.task.kernel:Running kernel on smithi043: 5.14.0-437.el9.x86_64 2024-04-25T08:47:35.728 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum install -y kernel 2024-04-25T08:47:38.887 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - BaseOS 5.3 MB/s | 8.0 MB 00:01 2024-04-25T08:47:38.974 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - BaseOS 4.9 MB/s | 8.0 MB 00:01 2024-04-25T08:47:40.941 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-04-25T08:47:41.296 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - AppStream 14 MB/s | 19 MB 00:01 2024-04-25T08:47:45.592 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - CRB 6.4 MB/s | 6.2 MB 00:00 2024-04-25T08:47:45.671 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - CRB 7.5 MB/s | 6.2 MB 00:00 2024-04-25T08:47:47.322 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-04-25T08:47:47.334 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - Extras packages 32 kB/s | 16 kB 00:00 2024-04-25T08:47:49.414 INFO:teuthology.orchestra.run.smithi043.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-25T08:47:50.132 INFO:teuthology.orchestra.run.smithi193.stdout:Extra Packages for Enterprise Linux 7.8 MB/s | 21 MB 00:02 2024-04-25T08:47:54.652 INFO:teuthology.orchestra.run.smithi043.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-25T08:47:55.198 INFO:teuthology.orchestra.run.smithi193.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-25T08:47:56.609 INFO:teuthology.orchestra.run.smithi043.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T08:47:56.609 INFO:teuthology.orchestra.run.smithi043.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T08:47:56.707 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-04-25T08:47:56.708 INFO:teuthology.orchestra.run.smithi043.stdout:Nothing to do. 2024-04-25T08:47:56.708 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-04-25T08:47:56.820 DEBUG:teuthology.orchestra.run.smithi043:> echo no | sudo yum reinstall kernel || true 2024-04-25T08:47:57.095 INFO:teuthology.orchestra.run.smithi193.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-25T08:47:57.096 INFO:teuthology.orchestra.run.smithi193.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-25T08:47:57.186 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-04-25T08:47:57.187 INFO:teuthology.orchestra.run.smithi193.stdout:Nothing to do. 2024-04-25T08:47:57.188 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-04-25T08:47:57.292 DEBUG:teuthology.orchestra.run.smithi193:> echo no | sudo yum reinstall kernel || true 2024-04-25T08:47:57.344 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 08:47:54 AM UTC. 2024-04-25T08:47:57.478 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-04-25T08:47:57.491 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-04-25T08:47:57.491 INFO:teuthology.orchestra.run.smithi043.stdout: Package Architecture Version Repository Size 2024-04-25T08:47:57.491 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-04-25T08:47:57.492 INFO:teuthology.orchestra.run.smithi043.stdout:Reinstalling: 2024-04-25T08:47:57.492 INFO:teuthology.orchestra.run.smithi043.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T08:47:57.492 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:47:57.492 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-04-25T08:47:57.492 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-04-25T08:47:57.492 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:47:57.492 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 255 k 2024-04-25T08:47:57.492 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 0 2024-04-25T08:47:57.493 INFO:teuthology.orchestra.run.smithi043.stderr:Operation aborted. 2024-04-25T08:47:57.549 INFO:teuthology.orchestra.run.smithi043.stdout:Is this ok [y/N]: 2024-04-25T08:47:57.550 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum reinstall -y kernel || true 2024-04-25T08:47:57.833 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:00:02 ago on Thu 25 Apr 2024 08:47:55 AM UTC. 2024-04-25T08:47:57.953 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-04-25T08:47:57.972 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T08:47:57.972 INFO:teuthology.orchestra.run.smithi193.stdout: Package Architecture Version Repository Size 2024-04-25T08:47:57.972 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T08:47:57.972 INFO:teuthology.orchestra.run.smithi193.stdout:Reinstalling: 2024-04-25T08:47:57.972 INFO:teuthology.orchestra.run.smithi193.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T08:47:57.972 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:47:57.973 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-04-25T08:47:57.973 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T08:47:57.973 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:47:57.973 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 255 k 2024-04-25T08:47:57.973 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 0 2024-04-25T08:47:57.973 INFO:teuthology.orchestra.run.smithi193.stderr:Operation aborted. 2024-04-25T08:47:58.034 INFO:teuthology.orchestra.run.smithi193.stdout:Is this ok [y/N]: 2024-04-25T08:47:58.035 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum reinstall -y kernel || true 2024-04-25T08:47:58.068 INFO:teuthology.orchestra.run.smithi043.stdout:Last metadata expiration check: 0:00:04 ago on Thu 25 Apr 2024 08:47:54 AM UTC. 2024-04-25T08:47:58.189 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-04-25T08:47:58.189 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-04-25T08:47:58.189 INFO:teuthology.orchestra.run.smithi043.stdout: Package Architecture Version Repository Size 2024-04-25T08:47:58.189 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-04-25T08:47:58.189 INFO:teuthology.orchestra.run.smithi043.stdout:Reinstalling: 2024-04-25T08:47:58.189 INFO:teuthology.orchestra.run.smithi043.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T08:47:58.190 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:47:58.190 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-04-25T08:47:58.190 INFO:teuthology.orchestra.run.smithi043.stdout:================================================================================ 2024-04-25T08:47:58.190 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:47:58.190 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 255 k 2024-04-25T08:47:58.190 INFO:teuthology.orchestra.run.smithi043.stdout:Installed size: 0 2024-04-25T08:47:58.190 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-04-25T08:47:58.530 INFO:teuthology.orchestra.run.smithi193.stdout:Last metadata expiration check: 0:00:03 ago on Thu 25 Apr 2024 08:47:55 AM UTC. 2024-04-25T08:47:58.605 INFO:teuthology.orchestra.run.smithi043.stdout:kernel-5.14.0-437.el9.x86_64.rpm 860 kB/s | 255 kB 00:00 2024-04-25T08:47:58.605 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-04-25T08:47:58.605 INFO:teuthology.orchestra.run.smithi043.stdout:Total 615 kB/s | 255 kB 00:00 2024-04-25T08:47:58.641 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-04-25T08:47:58.646 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-04-25T08:47:58.647 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T08:47:58.647 INFO:teuthology.orchestra.run.smithi193.stdout: Package Architecture Version Repository Size 2024-04-25T08:47:58.647 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T08:47:58.647 INFO:teuthology.orchestra.run.smithi193.stdout:Reinstalling: 2024-04-25T08:47:58.647 INFO:teuthology.orchestra.run.smithi193.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-25T08:47:58.647 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:47:58.648 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-04-25T08:47:58.648 INFO:teuthology.orchestra.run.smithi193.stdout:================================================================================ 2024-04-25T08:47:58.648 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:47:58.648 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 255 k 2024-04-25T08:47:58.648 INFO:teuthology.orchestra.run.smithi193.stdout:Installed size: 0 2024-04-25T08:47:58.648 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-04-25T08:47:58.822 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-04-25T08:47:58.822 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-04-25T08:47:58.827 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-04-25T08:47:58.828 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-04-25T08:47:58.959 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-04-25T08:47:59.039 INFO:teuthology.orchestra.run.smithi043.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T08:47:59.039 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T08:47:59.125 INFO:teuthology.orchestra.run.smithi193.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.3 MB/s | 255 kB 00:00 2024-04-25T08:47:59.125 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-04-25T08:47:59.125 INFO:teuthology.orchestra.run.smithi193.stdout:Total 534 kB/s | 255 kB 00:00 2024-04-25T08:47:59.140 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-04-25T08:47:59.315 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-04-25T08:47:59.315 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-04-25T08:47:59.320 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-04-25T08:47:59.321 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-04-25T08:47:59.460 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-04-25T08:47:59.551 INFO:teuthology.orchestra.run.smithi193.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T08:47:59.551 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T08:47:59.759 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T08:47:59.759 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T08:48:00.285 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T08:48:00.285 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-25T08:48:00.498 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T08:48:00.498 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:48:00.498 INFO:teuthology.orchestra.run.smithi043.stdout:Reinstalled: 2024-04-25T08:48:00.499 INFO:teuthology.orchestra.run.smithi043.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T08:48:00.499 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:48:00.499 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-04-25T08:48:00.659 DEBUG:teuthology.orchestra.run.smithi043:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T08:48:00.706 INFO:teuthology.orchestra.run.smithi043.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T08:48:00.707 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T08:48:00.707 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T08:48:00.707 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-25T08:48:00.707 DEBUG:teuthology.orchestra.run.smithi043:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T08:48:00.750 INFO:teuthology.orchestra.run.smithi043.stdout:ttyS1 2024-04-25T08:48:00.768 DEBUG:teuthology.parallel:result is None 2024-04-25T08:48:00.942 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-25T08:48:00.943 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:48:00.943 INFO:teuthology.orchestra.run.smithi193.stdout:Reinstalled: 2024-04-25T08:48:00.943 INFO:teuthology.orchestra.run.smithi193.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-25T08:48:00.943 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:48:00.943 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-04-25T08:48:01.136 DEBUG:teuthology.orchestra.run.smithi193:> rpm -q kernel | sort -rV | head -n 1 2024-04-25T08:48:01.183 INFO:teuthology.orchestra.run.smithi193.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-25T08:48:01.183 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-25T08:48:01.183 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-25T08:48:01.183 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-25T08:48:01.183 DEBUG:teuthology.orchestra.run.smithi193:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-25T08:48:01.218 INFO:teuthology.orchestra.run.smithi193.stdout:ttyS1 2024-04-25T08:48:01.240 DEBUG:teuthology.parallel:result is None 2024-04-25T08:48:01.240 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T08:48:01.247 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T08:48:01.247 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T08:48:01.250 DEBUG:teuthology.orchestra.run.smithi193:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T08:48:01.295 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T08:48:01.302 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T08:48:01.308 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T08:48:01.309 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T08:48:01.311 DEBUG:teuthology.orchestra.run.smithi193:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T08:48:01.365 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T08:48:01.372 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T08:48:01.372 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T08:48:01.379 DEBUG:teuthology.orchestra.run.smithi193:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T08:48:01.408 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T08:48:01.418 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T08:48:01.432 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T08:48:01.441 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T08:48:01.442 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T08:48:01.450 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T08:48:01.450 DEBUG:teuthology.orchestra.run.smithi043:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T08:48:01.462 DEBUG:teuthology.orchestra.run.smithi193:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T08:48:01.505 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T08:48:01.514 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T08:48:01.514 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T08:48:01.529 DEBUG:teuthology.orchestra.run.smithi193:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T08:48:01.560 DEBUG:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T08:48:01.621 DEBUG:teuthology.orchestra.run.smithi043:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T08:48:01.685 DEBUG:teuthology.orchestra.run.smithi043:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T08:48:01.727 DEBUG:teuthology.orchestra.run.smithi043:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T08:48:01.768 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:48:01.768 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T08:48:01.834 DEBUG:teuthology.orchestra.run.smithi193:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T08:48:01.874 DEBUG:teuthology.orchestra.run.smithi193:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T08:48:01.918 DEBUG:teuthology.orchestra.run.smithi193:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T08:48:01.999 DEBUG:teuthology.orchestra.run.smithi193:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T08:48:02.039 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T08:48:02.039 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T08:48:02.102 DEBUG:teuthology.orchestra.run.smithi043:> sudo service rsyslog restart 2024-04-25T08:48:02.105 DEBUG:teuthology.orchestra.run.smithi193:> sudo service rsyslog restart 2024-04-25T08:48:02.150 INFO:teuthology.orchestra.run.smithi043.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T08:48:02.173 INFO:teuthology.orchestra.run.smithi193.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T08:48:02.573 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T08:48:02.582 INFO:teuthology.task.internal:Starting timer... 2024-04-25T08:48:02.583 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T08:48:02.597 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T08:48:02.606 DEBUG:teuthology.orchestra.run.smithi043:> sudo service auditd rotate 2024-04-25T08:48:02.678 INFO:teuthology.orchestra.run.smithi043.stdout:Rotating logs: 2024-04-25T08:48:02.680 DEBUG:teuthology.orchestra.run.smithi193:> sudo service auditd rotate 2024-04-25T08:48:02.729 INFO:teuthology.orchestra.run.smithi193.stdout:Rotating logs: 2024-04-25T08:48:02.730 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T08:48:02.730 DEBUG:teuthology.orchestra.run.smithi043:> /usr/sbin/getenforce 2024-04-25T08:48:02.754 INFO:teuthology.orchestra.run.smithi043.stdout:Permissive 2024-04-25T08:48:02.755 DEBUG:teuthology.orchestra.run.smithi193:> /usr/sbin/getenforce 2024-04-25T08:48:02.776 INFO:teuthology.orchestra.run.smithi193.stdout:Permissive 2024-04-25T08:48:02.777 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi043.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi193.front.sepia.ceph.com': 'permissive'} 2024-04-25T08:48:02.777 DEBUG:teuthology.orchestra.run.smithi043:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T08:48:02.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:48:02.820 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T08:48:02.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:48:02.844 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T08:48:02.845 DEBUG:teuthology.orchestra.run.smithi043:> sudo /usr/sbin/setenforce permissive 2024-04-25T08:48:02.885 DEBUG:teuthology.orchestra.run.smithi193:> sudo /usr/sbin/setenforce permissive 2024-04-25T08:48:02.908 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T08:48:02.918 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T08:48:02.933 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T08:48:02.935 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi043.front.sepia.ceph.com,smithi193.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T08:51:31.803 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi043.front.sepia.ceph.com'), Remote(name='ubuntu@smithi193.front.sepia.ceph.com')] 2024-04-25T08:51:31.805 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-04-25T08:51:31.806 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi043.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:51:31.881 DEBUG:teuthology.orchestra.run.smithi043:> true 2024-04-25T08:51:31.958 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi043.front.sepia.ceph.com' 2024-04-25T08:51:31.958 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-04-25T08:51:31.959 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi193.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T08:51:32.030 DEBUG:teuthology.orchestra.run.smithi193:> true 2024-04-25T08:51:32.104 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi193.front.sepia.ceph.com' 2024-04-25T08:51:32.105 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T08:51:32.116 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T08:51:32.116 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T08:51:32.117 DEBUG:teuthology.orchestra.run.smithi043:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T08:51:32.119 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T08:51:32.119 DEBUG:teuthology.orchestra.run.smithi193:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T08:51:32.159 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T08:51:32.172 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T08:51:32.175 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T08:51:32.185 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T08:51:32.201 INFO:teuthology.orchestra.run.smithi043.stderr:sudo: ntpd: command not found 2024-04-25T08:51:32.207 INFO:teuthology.orchestra.run.smithi193.stderr:sudo: ntpd: command not found 2024-04-25T08:51:32.212 INFO:teuthology.orchestra.run.smithi043.stdout:506 Cannot talk to daemon 2024-04-25T08:51:32.218 INFO:teuthology.orchestra.run.smithi193.stdout:506 Cannot talk to daemon 2024-04-25T08:51:32.227 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T08:51:32.232 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T08:51:32.241 INFO:teuthology.orchestra.run.smithi043.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T08:51:32.245 INFO:teuthology.orchestra.run.smithi193.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T08:51:32.296 INFO:teuthology.orchestra.run.smithi193.stderr:bash: line 1: ntpq: command not found 2024-04-25T08:51:32.297 INFO:teuthology.orchestra.run.smithi043.stderr:bash: line 1: ntpq: command not found 2024-04-25T08:51:32.300 INFO:teuthology.orchestra.run.smithi193.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T08:51:32.300 INFO:teuthology.orchestra.run.smithi193.stdout:=============================================================================== 2024-04-25T08:51:32.300 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T08:51:32.300 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T08:51:32.300 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T08:51:32.300 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T08:51:32.301 INFO:teuthology.orchestra.run.smithi043.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T08:51:32.301 INFO:teuthology.orchestra.run.smithi043.stdout:=============================================================================== 2024-04-25T08:51:32.301 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T08:51:32.301 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T08:51:32.302 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T08:51:32.302 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T08:51:32.302 INFO:teuthology.run_tasks:Running task install... 2024-04-25T08:51:32.311 DEBUG:teuthology.task.install:project ceph 2024-04-25T08:51:32.311 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}} 2024-04-25T08:51:32.312 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'} 2024-04-25T08:51:32.312 INFO:teuthology.task.install:Using flavor: default 2024-04-25T08:51:32.317 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T08:51:32.317 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T08:51:32.318 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'tag': None, 'wait_for_package': False} 2024-04-25T08:51:32.318 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T08:51:32.320 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'tag': None, 'wait_for_package': False} 2024-04-25T08:51:32.320 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T08:51:32.460 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/ 2024-04-25T08:51:32.460 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1628.g97ae6cc9 2024-04-25T08:51:32.492 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/ 2024-04-25T08:51:32.492 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1628.g97ae6cc9 2024-04-25T08:51:32.569 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T08:51:32.569 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:51:32.569 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T08:51:32.594 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T08:51:32.594 DEBUG:teuthology.orchestra.run.smithi043:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T08:51:32.602 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T08:51:32.602 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T08:51:32.602 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T08:51:32.627 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T08:51:32.627 DEBUG:teuthology.orchestra.run.smithi193:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T08:51:32.659 DEBUG:teuthology.orchestra.run.smithi043:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T08:51:32.691 DEBUG:teuthology.orchestra.run.smithi193:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T08:51:32.734 DEBUG:teuthology.orchestra.run.smithi043:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T08:51:32.765 DEBUG:teuthology.orchestra.run.smithi193:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T08:51:32.798 INFO:teuthology.orchestra.run.smithi043.stdout:check_obsoletes = 1 2024-04-25T08:51:32.799 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum clean all 2024-04-25T08:51:32.828 INFO:teuthology.orchestra.run.smithi193.stdout:check_obsoletes = 1 2024-04-25T08:51:32.830 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum clean all 2024-04-25T08:51:33.034 INFO:teuthology.orchestra.run.smithi043.stdout:48 files removed 2024-04-25T08:51:33.060 INFO:teuthology.orchestra.run.smithi193.stdout:48 files removed 2024-04-25T08:51:33.062 DEBUG:teuthology.orchestra.run.smithi043:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T08:51:33.089 DEBUG:teuthology.orchestra.run.smithi193:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T08:51:33.679 INFO:teuthology.orchestra.run.smithi043.stdout:ceph packages for x86_64 241 kB/s | 82 kB 00:00 2024-04-25T08:51:33.702 INFO:teuthology.orchestra.run.smithi193.stdout:ceph packages for x86_64 240 kB/s | 82 kB 00:00 2024-04-25T08:51:34.021 INFO:teuthology.orchestra.run.smithi043.stdout:ceph noarch packages 34 kB/s | 10 kB 00:00 2024-04-25T08:51:34.078 INFO:teuthology.orchestra.run.smithi193.stdout:ceph noarch packages 30 kB/s | 10 kB 00:00 2024-04-25T08:51:34.380 INFO:teuthology.orchestra.run.smithi043.stdout:ceph source packages 5.8 kB/s | 1.9 kB 00:00 2024-04-25T08:51:34.420 INFO:teuthology.orchestra.run.smithi193.stdout:ceph source packages 6.2 kB/s | 1.9 kB 00:00 2024-04-25T08:51:35.389 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - BaseOS 8.3 MB/s | 8.0 MB 00:00 2024-04-25T08:51:35.636 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - BaseOS 6.8 MB/s | 8.0 MB 00:01 2024-04-25T08:51:38.071 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - AppStream 14 MB/s | 19 MB 00:01 2024-04-25T08:51:38.857 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - AppStream 8.1 MB/s | 19 MB 00:02 2024-04-25T08:51:42.296 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - CRB 9.2 MB/s | 6.2 MB 00:00 2024-04-25T08:51:43.366 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - CRB 7.5 MB/s | 6.2 MB 00:00 2024-04-25T08:51:43.697 INFO:teuthology.orchestra.run.smithi193.stdout:CentOS Stream 9 - Extras packages 62 kB/s | 16 kB 00:00 2024-04-25T08:51:44.817 INFO:teuthology.orchestra.run.smithi043.stdout:CentOS Stream 9 - Extras packages 62 kB/s | 16 kB 00:00 2024-04-25T08:51:45.184 INFO:teuthology.orchestra.run.smithi193.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-04-25T08:51:47.985 INFO:teuthology.orchestra.run.smithi043.stdout:Extra Packages for Enterprise Linux 6.9 MB/s | 21 MB 00:03 2024-04-25T08:51:50.182 INFO:teuthology.orchestra.run.smithi193.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-04-25T08:51:52.317 INFO:teuthology.orchestra.run.smithi193.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T08:51:52.317 INFO:teuthology.orchestra.run.smithi193.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T08:51:52.328 INFO:teuthology.orchestra.run.smithi193.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T08:51:52.329 INFO:teuthology.orchestra.run.smithi193.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T08:51:52.329 INFO:teuthology.orchestra.run.smithi193.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T08:51:52.330 INFO:teuthology.orchestra.run.smithi193.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T08:51:52.332 INFO:teuthology.orchestra.run.smithi193.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T08:51:52.334 INFO:teuthology.orchestra.run.smithi193.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T08:51:52.334 INFO:teuthology.orchestra.run.smithi193.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T08:51:52.394 INFO:teuthology.orchestra.run.smithi193.stdout:Dependencies resolved. 2024-04-25T08:51:52.403 INFO:teuthology.orchestra.run.smithi193.stdout:======================================================================================= 2024-04-25T08:51:52.403 INFO:teuthology.orchestra.run.smithi193.stdout: Package Arch Version Repository Size 2024-04-25T08:51:52.404 INFO:teuthology.orchestra.run.smithi193.stdout:======================================================================================= 2024-04-25T08:51:52.404 INFO:teuthology.orchestra.run.smithi193.stdout:Installing: 2024-04-25T08:51:52.404 INFO:teuthology.orchestra.run.smithi193.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T08:51:52.404 INFO:teuthology.orchestra.run.smithi193.stdout: ceph x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 6.6 k 2024-04-25T08:51:52.404 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-base x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.3 M 2024-04-25T08:51:52.404 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 887 k 2024-04-25T08:51:52.404 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 145 k 2024-04-25T08:51:52.404 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 1.5 M 2024-04-25T08:51:52.404 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 138 k 2024-04-25T08:51:52.404 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 1.8 M 2024-04-25T08:51:52.404 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 7.4 M 2024-04-25T08:51:52.404 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-rook noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 49 k 2024-04-25T08:51:52.404 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-radosgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 9.8 M 2024-04-25T08:51:52.405 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-test x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 47 M 2024-04-25T08:51:52.405 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-volume noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 264 k 2024-04-25T08:51:52.405 INFO:teuthology.orchestra.run.smithi193.stdout: cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 748 k 2024-04-25T08:51:52.405 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 220 k 2024-04-25T08:51:52.405 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-shell x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 27 k 2024-04-25T08:51:52.405 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-top noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 18 k 2024-04-25T08:51:52.405 INFO:teuthology.orchestra.run.smithi193.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T08:51:52.405 INFO:teuthology.orchestra.run.smithi193.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T08:51:52.405 INFO:teuthology.orchestra.run.smithi193.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T08:51:52.405 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 31 k 2024-04-25T08:51:52.405 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 744 k 2024-04-25T08:51:52.405 INFO:teuthology.orchestra.run.smithi193.stdout: librados-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 127 k 2024-04-25T08:51:52.405 INFO:teuthology.orchestra.run.smithi193.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T08:51:52.405 INFO:teuthology.orchestra.run.smithi193.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T08:51:52.405 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T08:51:52.406 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T08:51:52.406 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T08:51:52.406 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T08:51:52.406 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cephfs x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 162 k 2024-04-25T08:51:52.406 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rados x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 321 k 2024-04-25T08:51:52.406 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 298 k 2024-04-25T08:51:52.406 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 99 k 2024-04-25T08:51:52.406 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 85 k 2024-04-25T08:51:52.406 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T08:51:52.406 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-nbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 170 k 2024-04-25T08:51:52.406 INFO:teuthology.orchestra.run.smithi193.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T08:51:52.406 INFO:teuthology.orchestra.run.smithi193.stdout:Upgrading: 2024-04-25T08:51:52.406 INFO:teuthology.orchestra.run.smithi193.stdout: librados2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.4 M 2024-04-25T08:51:52.407 INFO:teuthology.orchestra.run.smithi193.stdout: librbd1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T08:51:52.407 INFO:teuthology.orchestra.run.smithi193.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T08:51:52.407 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T08:51:52.407 INFO:teuthology.orchestra.run.smithi193.stdout:Installing dependencies: 2024-04-25T08:51:52.407 INFO:teuthology.orchestra.run.smithi193.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T08:51:52.407 INFO:teuthology.orchestra.run.smithi193.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T08:51:52.407 INFO:teuthology.orchestra.run.smithi193.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T08:51:52.407 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 20 M 2024-04-25T08:51:52.407 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 31 k 2024-04-25T08:51:52.407 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mds x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 2.1 M 2024-04-25T08:51:52.407 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 248 k 2024-04-25T08:51:52.407 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mon x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 4.7 M 2024-04-25T08:51:52.407 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-osd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 17 M 2024-04-25T08:51:52.408 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 15 k 2024-04-25T08:51:52.408 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-selinux x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 25 k 2024-04-25T08:51:52.408 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T08:51:52.408 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T08:51:52.408 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T08:51:52.408 INFO:teuthology.orchestra.run.smithi193.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T08:51:52.408 INFO:teuthology.orchestra.run.smithi193.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T08:51:52.408 INFO:teuthology.orchestra.run.smithi193.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T08:51:52.408 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T08:51:52.408 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T08:51:52.408 INFO:teuthology.orchestra.run.smithi193.stdout: libcephsqlite x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 163 k 2024-04-25T08:51:52.408 INFO:teuthology.orchestra.run.smithi193.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T08:51:52.408 INFO:teuthology.orchestra.run.smithi193.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T08:51:52.408 INFO:teuthology.orchestra.run.smithi193.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T08:51:52.409 INFO:teuthology.orchestra.run.smithi193.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T08:51:52.409 INFO:teuthology.orchestra.run.smithi193.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T08:51:52.409 INFO:teuthology.orchestra.run.smithi193.stdout: libradosstriper1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 501 k 2024-04-25T08:51:52.409 INFO:teuthology.orchestra.run.smithi193.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T08:51:52.409 INFO:teuthology.orchestra.run.smithi193.stdout: librgw2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.1 M 2024-04-25T08:51:52.409 INFO:teuthology.orchestra.run.smithi193.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T08:51:52.409 INFO:teuthology.orchestra.run.smithi193.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T08:51:52.409 INFO:teuthology.orchestra.run.smithi193.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T08:51:52.409 INFO:teuthology.orchestra.run.smithi193.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T08:51:52.409 INFO:teuthology.orchestra.run.smithi193.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T08:51:52.409 INFO:teuthology.orchestra.run.smithi193.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T08:51:52.409 INFO:teuthology.orchestra.run.smithi193.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T08:51:52.409 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T08:51:52.409 INFO:teuthology.orchestra.run.smithi193.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T08:51:52.409 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T08:51:52.410 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T08:51:52.410 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T08:51:52.410 INFO:teuthology.orchestra.run.smithi193.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T08:51:52.410 INFO:teuthology.orchestra.run.smithi193.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T08:51:52.410 INFO:teuthology.orchestra.run.smithi193.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T08:51:52.410 INFO:teuthology.orchestra.run.smithi193.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T08:51:52.410 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T08:51:52.410 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-argparse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 45 k 2024-04-25T08:51:52.410 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 131 k 2024-04-25T08:51:52.410 INFO:teuthology.orchestra.run.smithi193.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T08:51:52.410 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T08:51:52.410 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T08:51:52.410 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T08:51:52.411 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T08:51:52.411 INFO:teuthology.orchestra.run.smithi193.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T08:51:52.411 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T08:51:52.411 INFO:teuthology.orchestra.run.smithi193.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T08:51:52.411 INFO:teuthology.orchestra.run.smithi193.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T08:51:52.411 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T08:51:52.411 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T08:51:52.411 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T08:51:52.411 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T08:51:52.411 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T08:51:52.411 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T08:51:52.411 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T08:51:52.411 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T08:51:52.411 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T08:51:52.411 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T08:51:52.412 INFO:teuthology.orchestra.run.smithi193.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T08:51:52.412 INFO:teuthology.orchestra.run.smithi193.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T08:51:52.412 INFO:teuthology.orchestra.run.smithi193.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T08:51:52.412 INFO:teuthology.orchestra.run.smithi193.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T08:51:52.412 INFO:teuthology.orchestra.run.smithi193.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T08:51:52.412 INFO:teuthology.orchestra.run.smithi193.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T08:51:52.412 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T08:51:52.412 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T08:51:52.412 INFO:teuthology.orchestra.run.smithi193.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T08:51:52.412 INFO:teuthology.orchestra.run.smithi193.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T08:51:52.412 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T08:51:52.412 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T08:51:52.412 INFO:teuthology.orchestra.run.smithi193.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T08:51:52.412 INFO:teuthology.orchestra.run.smithi193.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T08:51:52.413 INFO:teuthology.orchestra.run.smithi193.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T08:51:52.413 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T08:51:52.413 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T08:51:52.413 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T08:51:52.413 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T08:51:52.413 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T08:51:52.413 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T08:51:52.413 INFO:teuthology.orchestra.run.smithi193.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T08:51:52.413 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T08:51:52.413 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T08:51:52.413 INFO:teuthology.orchestra.run.smithi193.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T08:51:52.413 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T08:51:52.413 INFO:teuthology.orchestra.run.smithi193.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T08:51:52.413 INFO:teuthology.orchestra.run.smithi193.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T08:51:52.414 INFO:teuthology.orchestra.run.smithi193.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T08:51:52.414 INFO:teuthology.orchestra.run.smithi193.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T08:51:52.414 INFO:teuthology.orchestra.run.smithi193.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T08:51:52.414 INFO:teuthology.orchestra.run.smithi193.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T08:51:52.414 INFO:teuthology.orchestra.run.smithi193.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T08:51:52.414 INFO:teuthology.orchestra.run.smithi193.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T08:51:52.414 INFO:teuthology.orchestra.run.smithi193.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T08:51:52.414 INFO:teuthology.orchestra.run.smithi193.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T08:51:52.414 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T08:51:52.414 INFO:teuthology.orchestra.run.smithi193.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T08:51:52.414 INFO:teuthology.orchestra.run.smithi193.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T08:51:52.414 INFO:teuthology.orchestra.run.smithi193.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T08:51:52.414 INFO:teuthology.orchestra.run.smithi193.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T08:51:52.414 INFO:teuthology.orchestra.run.smithi193.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T08:51:52.415 INFO:teuthology.orchestra.run.smithi193.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T08:51:52.415 INFO:teuthology.orchestra.run.smithi193.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T08:51:52.415 INFO:teuthology.orchestra.run.smithi193.stdout:Installing weak dependencies: 2024-04-25T08:51:52.415 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T08:51:52.415 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:51:52.415 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction Summary 2024-04-25T08:51:52.415 INFO:teuthology.orchestra.run.smithi193.stdout:======================================================================================= 2024-04-25T08:51:52.415 INFO:teuthology.orchestra.run.smithi193.stdout:Install 148 Packages 2024-04-25T08:51:52.415 INFO:teuthology.orchestra.run.smithi193.stdout:Upgrade 4 Packages 2024-04-25T08:51:52.415 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:51:52.416 INFO:teuthology.orchestra.run.smithi193.stdout:Total download size: 215 M 2024-04-25T08:51:52.416 INFO:teuthology.orchestra.run.smithi193.stdout:Downloading Packages: 2024-04-25T08:51:53.293 INFO:teuthology.orchestra.run.smithi043.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-04-25T08:51:53.533 INFO:teuthology.orchestra.run.smithi193.stdout:(1/152): ceph-19.0.0-1628.g97ae6cc9.el9.x86_64. 31 kB/s | 6.6 kB 00:00 2024-04-25T08:51:53.741 INFO:teuthology.orchestra.run.smithi193.stdout:(2/152): ceph-fuse-19.0.0-1628.g97ae6cc9.el9.x8 4.2 MB/s | 887 kB 00:00 2024-04-25T08:51:53.808 INFO:teuthology.orchestra.run.smithi193.stdout:(3/152): ceph-immutable-object-cache-19.0.0-162 2.1 MB/s | 145 kB 00:00 2024-04-25T08:51:54.134 INFO:teuthology.orchestra.run.smithi193.stdout:(4/152): ceph-base-19.0.0-1628.g97ae6cc9.el9.x8 6.6 MB/s | 5.3 MB 00:00 2024-04-25T08:51:54.184 INFO:teuthology.orchestra.run.smithi193.stdout:(5/152): ceph-mds-19.0.0-1628.g97ae6cc9.el9.x86 5.7 MB/s | 2.1 MB 00:00 2024-04-25T08:51:54.317 INFO:teuthology.orchestra.run.smithi193.stdout:(6/152): ceph-mgr-19.0.0-1628.g97ae6cc9.el9.x86 8.1 MB/s | 1.5 MB 00:00 2024-04-25T08:51:54.901 INFO:teuthology.orchestra.run.smithi193.stdout:(7/152): ceph-mon-19.0.0-1628.g97ae6cc9.el9.x86 6.5 MB/s | 4.7 MB 00:00 2024-04-25T08:51:55.459 INFO:teuthology.orchestra.run.smithi193.stdout:(8/152): ceph-common-19.0.0-1628.g97ae6cc9.el9. 9.3 MB/s | 20 MB 00:02 2024-04-25T08:51:55.503 INFO:teuthology.orchestra.run.smithi043.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T08:51:55.504 INFO:teuthology.orchestra.run.smithi043.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T08:51:55.515 INFO:teuthology.orchestra.run.smithi043.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T08:51:55.516 INFO:teuthology.orchestra.run.smithi043.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T08:51:55.516 INFO:teuthology.orchestra.run.smithi043.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T08:51:55.517 INFO:teuthology.orchestra.run.smithi043.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T08:51:55.519 INFO:teuthology.orchestra.run.smithi043.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T08:51:55.521 INFO:teuthology.orchestra.run.smithi043.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T08:51:55.522 INFO:teuthology.orchestra.run.smithi043.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T08:51:55.526 INFO:teuthology.orchestra.run.smithi193.stdout:(9/152): ceph-selinux-19.0.0-1628.g97ae6cc9.el9 378 kB/s | 25 kB 00:00 2024-04-25T08:51:55.586 INFO:teuthology.orchestra.run.smithi043.stdout:Dependencies resolved. 2024-04-25T08:51:55.595 INFO:teuthology.orchestra.run.smithi043.stdout:======================================================================================= 2024-04-25T08:51:55.595 INFO:teuthology.orchestra.run.smithi043.stdout: Package Arch Version Repository Size 2024-04-25T08:51:55.595 INFO:teuthology.orchestra.run.smithi043.stdout:======================================================================================= 2024-04-25T08:51:55.595 INFO:teuthology.orchestra.run.smithi043.stdout:Installing: 2024-04-25T08:51:55.595 INFO:teuthology.orchestra.run.smithi043.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T08:51:55.595 INFO:teuthology.orchestra.run.smithi043.stdout: ceph x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 6.6 k 2024-04-25T08:51:55.595 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-base x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.3 M 2024-04-25T08:51:55.596 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 887 k 2024-04-25T08:51:55.596 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 145 k 2024-04-25T08:51:55.596 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 1.5 M 2024-04-25T08:51:55.596 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 138 k 2024-04-25T08:51:55.596 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 1.8 M 2024-04-25T08:51:55.596 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 7.4 M 2024-04-25T08:51:55.596 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-rook noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 49 k 2024-04-25T08:51:55.596 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-radosgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 9.8 M 2024-04-25T08:51:55.596 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-test x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 47 M 2024-04-25T08:51:55.596 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-volume noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 264 k 2024-04-25T08:51:55.596 INFO:teuthology.orchestra.run.smithi043.stdout: cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 748 k 2024-04-25T08:51:55.596 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 220 k 2024-04-25T08:51:55.596 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-shell x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 27 k 2024-04-25T08:51:55.596 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-top noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 18 k 2024-04-25T08:51:55.596 INFO:teuthology.orchestra.run.smithi043.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T08:51:55.597 INFO:teuthology.orchestra.run.smithi043.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T08:51:55.597 INFO:teuthology.orchestra.run.smithi043.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T08:51:55.597 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 31 k 2024-04-25T08:51:55.597 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 744 k 2024-04-25T08:51:55.597 INFO:teuthology.orchestra.run.smithi043.stdout: librados-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 127 k 2024-04-25T08:51:55.597 INFO:teuthology.orchestra.run.smithi043.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T08:51:55.597 INFO:teuthology.orchestra.run.smithi043.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T08:51:55.597 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T08:51:55.597 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T08:51:55.597 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T08:51:55.597 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T08:51:55.597 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cephfs x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 162 k 2024-04-25T08:51:55.597 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rados x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 321 k 2024-04-25T08:51:55.598 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 298 k 2024-04-25T08:51:55.598 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 99 k 2024-04-25T08:51:55.598 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 85 k 2024-04-25T08:51:55.598 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T08:51:55.598 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-nbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 170 k 2024-04-25T08:51:55.598 INFO:teuthology.orchestra.run.smithi043.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T08:51:55.598 INFO:teuthology.orchestra.run.smithi043.stdout:Upgrading: 2024-04-25T08:51:55.598 INFO:teuthology.orchestra.run.smithi043.stdout: librados2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.4 M 2024-04-25T08:51:55.598 INFO:teuthology.orchestra.run.smithi043.stdout: librbd1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T08:51:55.598 INFO:teuthology.orchestra.run.smithi043.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T08:51:55.598 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T08:51:55.598 INFO:teuthology.orchestra.run.smithi043.stdout:Installing dependencies: 2024-04-25T08:51:55.598 INFO:teuthology.orchestra.run.smithi043.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T08:51:55.598 INFO:teuthology.orchestra.run.smithi043.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T08:51:55.599 INFO:teuthology.orchestra.run.smithi043.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T08:51:55.599 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 20 M 2024-04-25T08:51:55.599 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 31 k 2024-04-25T08:51:55.599 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mds x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 2.1 M 2024-04-25T08:51:55.599 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 248 k 2024-04-25T08:51:55.599 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mon x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 4.7 M 2024-04-25T08:51:55.599 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-osd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 17 M 2024-04-25T08:51:55.599 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 15 k 2024-04-25T08:51:55.599 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-selinux x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 25 k 2024-04-25T08:51:55.599 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T08:51:55.599 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T08:51:55.599 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T08:51:55.599 INFO:teuthology.orchestra.run.smithi043.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T08:51:55.599 INFO:teuthology.orchestra.run.smithi043.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T08:51:55.600 INFO:teuthology.orchestra.run.smithi043.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T08:51:55.600 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T08:51:55.600 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T08:51:55.600 INFO:teuthology.orchestra.run.smithi043.stdout: libcephsqlite x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 163 k 2024-04-25T08:51:55.600 INFO:teuthology.orchestra.run.smithi043.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T08:51:55.600 INFO:teuthology.orchestra.run.smithi043.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T08:51:55.600 INFO:teuthology.orchestra.run.smithi043.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T08:51:55.600 INFO:teuthology.orchestra.run.smithi043.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T08:51:55.600 INFO:teuthology.orchestra.run.smithi043.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T08:51:55.600 INFO:teuthology.orchestra.run.smithi043.stdout: libradosstriper1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 501 k 2024-04-25T08:51:55.600 INFO:teuthology.orchestra.run.smithi043.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T08:51:55.600 INFO:teuthology.orchestra.run.smithi043.stdout: librgw2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.1 M 2024-04-25T08:51:55.600 INFO:teuthology.orchestra.run.smithi043.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T08:51:55.600 INFO:teuthology.orchestra.run.smithi043.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T08:51:55.600 INFO:teuthology.orchestra.run.smithi043.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T08:51:55.601 INFO:teuthology.orchestra.run.smithi043.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T08:51:55.601 INFO:teuthology.orchestra.run.smithi043.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T08:51:55.601 INFO:teuthology.orchestra.run.smithi043.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T08:51:55.601 INFO:teuthology.orchestra.run.smithi043.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T08:51:55.601 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T08:51:55.601 INFO:teuthology.orchestra.run.smithi043.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T08:51:55.601 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T08:51:55.601 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T08:51:55.601 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T08:51:55.601 INFO:teuthology.orchestra.run.smithi043.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T08:51:55.601 INFO:teuthology.orchestra.run.smithi043.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T08:51:55.601 INFO:teuthology.orchestra.run.smithi043.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T08:51:55.601 INFO:teuthology.orchestra.run.smithi043.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T08:51:55.601 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T08:51:55.602 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-argparse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 45 k 2024-04-25T08:51:55.602 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 131 k 2024-04-25T08:51:55.602 INFO:teuthology.orchestra.run.smithi043.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T08:51:55.602 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T08:51:55.602 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T08:51:55.602 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T08:51:55.602 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T08:51:55.602 INFO:teuthology.orchestra.run.smithi043.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T08:51:55.602 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T08:51:55.602 INFO:teuthology.orchestra.run.smithi043.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T08:51:55.602 INFO:teuthology.orchestra.run.smithi043.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T08:51:55.602 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T08:51:55.602 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T08:51:55.602 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T08:51:55.603 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T08:51:55.603 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T08:51:55.603 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T08:51:55.603 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T08:51:55.603 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T08:51:55.603 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T08:51:55.603 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T08:51:55.603 INFO:teuthology.orchestra.run.smithi043.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T08:51:55.603 INFO:teuthology.orchestra.run.smithi043.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T08:51:55.603 INFO:teuthology.orchestra.run.smithi043.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T08:51:55.603 INFO:teuthology.orchestra.run.smithi043.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T08:51:55.603 INFO:teuthology.orchestra.run.smithi043.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T08:51:55.603 INFO:teuthology.orchestra.run.smithi043.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T08:51:55.603 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T08:51:55.603 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T08:51:55.604 INFO:teuthology.orchestra.run.smithi043.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T08:51:55.604 INFO:teuthology.orchestra.run.smithi043.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T08:51:55.604 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T08:51:55.604 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T08:51:55.604 INFO:teuthology.orchestra.run.smithi043.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T08:51:55.604 INFO:teuthology.orchestra.run.smithi043.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T08:51:55.604 INFO:teuthology.orchestra.run.smithi043.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T08:51:55.604 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T08:51:55.604 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T08:51:55.604 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T08:51:55.604 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T08:51:55.604 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T08:51:55.604 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T08:51:55.604 INFO:teuthology.orchestra.run.smithi043.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T08:51:55.605 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T08:51:55.605 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T08:51:55.605 INFO:teuthology.orchestra.run.smithi043.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T08:51:55.605 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T08:51:55.605 INFO:teuthology.orchestra.run.smithi043.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T08:51:55.605 INFO:teuthology.orchestra.run.smithi043.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T08:51:55.605 INFO:teuthology.orchestra.run.smithi043.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T08:51:55.605 INFO:teuthology.orchestra.run.smithi043.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T08:51:55.605 INFO:teuthology.orchestra.run.smithi043.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T08:51:55.605 INFO:teuthology.orchestra.run.smithi043.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T08:51:55.605 INFO:teuthology.orchestra.run.smithi043.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T08:51:55.605 INFO:teuthology.orchestra.run.smithi043.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T08:51:55.605 INFO:teuthology.orchestra.run.smithi043.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T08:51:55.605 INFO:teuthology.orchestra.run.smithi043.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T08:51:55.606 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T08:51:55.606 INFO:teuthology.orchestra.run.smithi043.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T08:51:55.606 INFO:teuthology.orchestra.run.smithi043.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T08:51:55.606 INFO:teuthology.orchestra.run.smithi043.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T08:51:55.606 INFO:teuthology.orchestra.run.smithi043.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T08:51:55.606 INFO:teuthology.orchestra.run.smithi043.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T08:51:55.606 INFO:teuthology.orchestra.run.smithi043.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T08:51:55.606 INFO:teuthology.orchestra.run.smithi043.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T08:51:55.606 INFO:teuthology.orchestra.run.smithi043.stdout:Installing weak dependencies: 2024-04-25T08:51:55.606 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T08:51:55.606 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:51:55.606 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction Summary 2024-04-25T08:51:55.606 INFO:teuthology.orchestra.run.smithi043.stdout:======================================================================================= 2024-04-25T08:51:55.606 INFO:teuthology.orchestra.run.smithi043.stdout:Install 148 Packages 2024-04-25T08:51:55.606 INFO:teuthology.orchestra.run.smithi043.stdout:Upgrade 4 Packages 2024-04-25T08:51:55.607 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:51:55.607 INFO:teuthology.orchestra.run.smithi043.stdout:Total download size: 215 M 2024-04-25T08:51:55.607 INFO:teuthology.orchestra.run.smithi043.stdout:Downloading Packages: 2024-04-25T08:51:56.093 INFO:teuthology.orchestra.run.smithi193.stdout:(10/152): ceph-osd-19.0.0-1628.g97ae6cc9.el9.x8 9.5 MB/s | 17 MB 00:01 2024-04-25T08:51:56.218 INFO:teuthology.orchestra.run.smithi193.stdout:(11/152): ceph-radosgw-19.0.0-1628.g97ae6cc9.el 7.4 MB/s | 9.8 MB 00:01 2024-04-25T08:51:56.243 INFO:teuthology.orchestra.run.smithi193.stdout:(12/152): cephfs-mirror-19.0.0-1628.g97ae6cc9.e 1.4 MB/s | 220 kB 00:00 2024-04-25T08:51:56.269 INFO:teuthology.orchestra.run.smithi193.stdout:(13/152): cephfs-shell-19.0.0-1628.g97ae6cc9.el 540 kB/s | 27 kB 00:00 2024-04-25T08:51:56.311 INFO:teuthology.orchestra.run.smithi193.stdout:(14/152): libcephfs-devel-19.0.0-1628.g97ae6cc9 465 kB/s | 31 kB 00:00 2024-04-25T08:51:56.386 INFO:teuthology.orchestra.run.smithi193.stdout:(15/152): libcephfs2-19.0.0-1628.g97ae6cc9.el9. 6.2 MB/s | 744 kB 00:00 2024-04-25T08:51:56.411 INFO:teuthology.orchestra.run.smithi193.stdout:(16/152): libcephsqlite-19.0.0-1628.g97ae6cc9.e 1.6 MB/s | 163 kB 00:00 2024-04-25T08:51:56.453 INFO:teuthology.orchestra.run.smithi193.stdout:(17/152): librados-devel-19.0.0-1628.g97ae6cc9. 1.9 MB/s | 127 kB 00:00 2024-04-25T08:51:56.487 INFO:teuthology.orchestra.run.smithi193.stdout:(18/152): libradosstriper1-19.0.0-1628.g97ae6cc 6.5 MB/s | 501 kB 00:00 2024-04-25T08:51:56.503 INFO:teuthology.orchestra.run.smithi043.stdout:(1/152): ceph-19.0.0-1628.g97ae6cc9.el9.x86_64. 26 kB/s | 6.6 kB 00:00 2024-04-25T08:51:56.562 INFO:teuthology.orchestra.run.smithi193.stdout:(19/152): python3-ceph-argparse-19.0.0-1628.g97 600 kB/s | 45 kB 00:00 2024-04-25T08:51:56.637 INFO:teuthology.orchestra.run.smithi193.stdout:(20/152): python3-ceph-common-19.0.0-1628.g97ae 1.7 MB/s | 131 kB 00:00 2024-04-25T08:51:56.712 INFO:teuthology.orchestra.run.smithi193.stdout:(21/152): python3-cephfs-19.0.0-1628.g97ae6cc9. 2.1 MB/s | 162 kB 00:00 2024-04-25T08:51:56.805 INFO:teuthology.orchestra.run.smithi193.stdout:(22/152): python3-rados-19.0.0-1628.g97ae6cc9.e 3.4 MB/s | 321 kB 00:00 2024-04-25T08:51:56.897 INFO:teuthology.orchestra.run.smithi193.stdout:(23/152): python3-rbd-19.0.0-1628.g97ae6cc9.el9 3.2 MB/s | 298 kB 00:00 2024-04-25T08:51:56.972 INFO:teuthology.orchestra.run.smithi193.stdout:(24/152): python3-rgw-19.0.0-1628.g97ae6cc9.el9 1.3 MB/s | 99 kB 00:00 2024-04-25T08:51:57.047 INFO:teuthology.orchestra.run.smithi193.stdout:(25/152): rbd-fuse-19.0.0-1628.g97ae6cc9.el9.x8 1.1 MB/s | 85 kB 00:00 2024-04-25T08:51:57.122 INFO:teuthology.orchestra.run.smithi193.stdout:(26/152): librgw2-19.0.0-1628.g97ae6cc9.el9.x86 7.7 MB/s | 5.1 MB 00:00 2024-04-25T08:51:57.214 INFO:teuthology.orchestra.run.smithi193.stdout:(27/152): rbd-nbd-19.0.0-1628.g97ae6cc9.el9.x86 1.8 MB/s | 170 kB 00:00 2024-04-25T08:51:57.294 INFO:teuthology.orchestra.run.smithi043.stdout:(2/152): ceph-fuse-19.0.0-1628.g97ae6cc9.el9.x8 1.1 MB/s | 887 kB 00:00 2024-04-25T08:51:57.298 INFO:teuthology.orchestra.run.smithi193.stdout:(28/152): ceph-grafana-dashboards-19.0.0-1628.g 376 kB/s | 31 kB 00:00 2024-04-25T08:51:57.398 INFO:teuthology.orchestra.run.smithi193.stdout:(29/152): ceph-mgr-cephadm-19.0.0-1628.g97ae6cc 1.3 MB/s | 138 kB 00:00 2024-04-25T08:51:57.436 INFO:teuthology.orchestra.run.smithi043.stdout:(3/152): ceph-immutable-object-cache-19.0.0-162 1.0 MB/s | 145 kB 00:00 2024-04-25T08:51:57.474 INFO:teuthology.orchestra.run.smithi193.stdout:(30/152): rbd-mirror-19.0.0-1628.g97ae6cc9.el9. 7.2 MB/s | 3.1 MB 00:00 2024-04-25T08:51:57.725 INFO:teuthology.orchestra.run.smithi193.stdout:(31/152): ceph-mgr-dashboard-19.0.0-1628.g97ae6 5.5 MB/s | 1.8 MB 00:00 2024-04-25T08:51:57.800 INFO:teuthology.orchestra.run.smithi193.stdout:(32/152): ceph-mgr-modules-core-19.0.0-1628.g97 3.2 MB/s | 248 kB 00:00 2024-04-25T08:51:57.892 INFO:teuthology.orchestra.run.smithi193.stdout:(33/152): ceph-mgr-rook-19.0.0-1628.g97ae6cc9.e 539 kB/s | 49 kB 00:00 2024-04-25T08:51:57.967 INFO:teuthology.orchestra.run.smithi193.stdout:(34/152): ceph-prometheus-alerts-19.0.0-1628.g9 204 kB/s | 15 kB 00:00 2024-04-25T08:51:58.050 INFO:teuthology.orchestra.run.smithi193.stdout:(35/152): ceph-volume-19.0.0-1628.g97ae6cc9.el9 3.1 MB/s | 264 kB 00:00 2024-04-25T08:51:58.209 INFO:teuthology.orchestra.run.smithi193.stdout:(36/152): cephadm-19.0.0-1628.g97ae6cc9.el9.noa 4.6 MB/s | 748 kB 00:00 2024-04-25T08:51:58.284 INFO:teuthology.orchestra.run.smithi193.stdout:(37/152): cephfs-top-19.0.0-1628.g97ae6cc9.el9. 243 kB/s | 18 kB 00:00 2024-04-25T08:51:58.460 INFO:teuthology.orchestra.run.smithi193.stdout:(38/152): ceph-mgr-diskprediction-local-19.0.0- 7.5 MB/s | 7.4 MB 00:00 2024-04-25T08:51:58.485 INFO:teuthology.orchestra.run.smithi193.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 401 kB/s | 80 kB 00:00 2024-04-25T08:51:58.570 INFO:teuthology.orchestra.run.smithi043.stdout:(4/152): ceph-mds-19.0.0-1628.g97ae6cc9.el9.x86 1.9 MB/s | 2.1 MB 00:01 2024-04-25T08:51:58.610 INFO:teuthology.orchestra.run.smithi193.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.5 MB/s | 195 kB 00:00 2024-04-25T08:51:58.702 INFO:teuthology.orchestra.run.smithi193.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-04-25T08:51:58.735 INFO:teuthology.orchestra.run.smithi193.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.8 MB/s | 803 kB 00:00 2024-04-25T08:51:58.970 INFO:teuthology.orchestra.run.smithi043.stdout:(5/152): ceph-base-19.0.0-1628.g97ae6cc9.el9.x8 2.0 MB/s | 5.3 MB 00:02 2024-04-25T08:51:59.029 INFO:teuthology.orchestra.run.smithi043.stdout:(6/152): ceph-mgr-19.0.0-1628.g97ae6cc9.el9.x86 3.3 MB/s | 1.5 MB 00:00 2024-04-25T08:51:59.144 INFO:teuthology.orchestra.run.smithi193.stdout:(43/152): ceph-test-19.0.0-1628.g97ae6cc9.el9.x 13 MB/s | 47 MB 00:03 2024-04-25T08:51:59.169 INFO:teuthology.orchestra.run.smithi193.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 245 kB/s | 106 kB 00:00 2024-04-25T08:51:59.211 INFO:teuthology.orchestra.run.smithi193.stdout:(45/152): python3-cryptography-36.0.1-4.el9.x86 2.4 MB/s | 1.2 MB 00:00 2024-04-25T08:51:59.236 INFO:teuthology.orchestra.run.smithi193.stdout:(46/152): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-04-25T08:51:59.270 INFO:teuthology.orchestra.run.smithi193.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-04-25T08:51:59.395 INFO:teuthology.orchestra.run.smithi193.stdout:(48/152): python3-pycparser-2.20-6.el9.noarch.r 539 kB/s | 135 kB 00:00 2024-04-25T08:51:59.937 INFO:teuthology.orchestra.run.smithi193.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 197 kB/s | 107 kB 00:00 2024-04-25T08:51:59.946 INFO:teuthology.orchestra.run.smithi043.stdout:(7/152): ceph-mon-19.0.0-1628.g97ae6cc9.el9.x86 4.8 MB/s | 4.7 MB 00:00 2024-04-25T08:52:00.213 INFO:teuthology.orchestra.run.smithi193.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 78 kB/s | 21 kB 00:00 2024-04-25T08:52:00.346 INFO:teuthology.orchestra.run.smithi193.stdout:(51/152): automake-1.16.2-8.el9.noarch.rpm 621 kB/s | 689 kB 00:01 2024-04-25T08:52:00.613 INFO:teuthology.orchestra.run.smithi193.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 123 kB/s | 33 kB 00:00 2024-04-25T08:52:00.897 INFO:teuthology.orchestra.run.smithi193.stdout:(53/152): flex-2.6.4-9.el9.x86_64.rpm 459 kB/s | 313 kB 00:00 2024-04-25T08:52:00.972 INFO:teuthology.orchestra.run.smithi193.stdout:(54/152): bison-3.7.4-5.el9.x86_64.rpm 554 kB/s | 943 kB 00:01 2024-04-25T08:52:01.164 INFO:teuthology.orchestra.run.smithi193.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 58 kB/s | 11 kB 00:00 2024-04-25T08:52:01.191 INFO:teuthology.orchestra.run.smithi193.stdout:(56/152): flexiblas-openblas-openmp-3.0.4-8.el9 62 kB/s | 18 kB 00:00 2024-04-25T08:52:01.462 INFO:teuthology.orchestra.run.smithi043.stdout:(8/152): ceph-common-19.0.0-1628.g97ae6cc9.el9. 3.8 MB/s | 20 MB 00:05 2024-04-25T08:52:01.542 INFO:teuthology.orchestra.run.smithi193.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 129 kB/s | 45 kB 00:00 2024-04-25T08:52:01.587 INFO:teuthology.orchestra.run.smithi043.stdout:(9/152): ceph-radosgw-19.0.0-1628.g97ae6cc9.el9 5.9 MB/s | 9.8 MB 00:01 2024-04-25T08:52:01.604 INFO:teuthology.orchestra.run.smithi043.stdout:(10/152): ceph-selinux-19.0.0-1628.g97ae6cc9.el 178 kB/s | 25 kB 00:00 2024-04-25T08:52:01.608 INFO:teuthology.orchestra.run.smithi193.stdout:(58/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 361 kB/s | 160 kB 00:00 2024-04-25T08:52:01.680 INFO:teuthology.orchestra.run.smithi043.stdout:(11/152): cephfs-mirror-19.0.0-1628.g97ae6cc9.e 2.9 MB/s | 220 kB 00:00 2024-04-25T08:52:01.722 INFO:teuthology.orchestra.run.smithi043.stdout:(12/152): cephfs-shell-19.0.0-1628.g97ae6cc9.el 653 kB/s | 27 kB 00:00 2024-04-25T08:52:01.772 INFO:teuthology.orchestra.run.smithi043.stdout:(13/152): libcephfs-devel-19.0.0-1628.g97ae6cc9 626 kB/s | 31 kB 00:00 2024-04-25T08:52:01.914 INFO:teuthology.orchestra.run.smithi043.stdout:(14/152): libcephfs2-19.0.0-1628.g97ae6cc9.el9. 5.1 MB/s | 744 kB 00:00 2024-04-25T08:52:01.973 INFO:teuthology.orchestra.run.smithi043.stdout:(15/152): libcephsqlite-19.0.0-1628.g97ae6cc9.e 2.7 MB/s | 163 kB 00:00 2024-04-25T08:52:02.031 INFO:teuthology.orchestra.run.smithi043.stdout:(16/152): librados-devel-19.0.0-1628.g97ae6cc9. 2.1 MB/s | 127 kB 00:00 2024-04-25T08:52:02.215 INFO:teuthology.orchestra.run.smithi043.stdout:(17/152): ceph-osd-19.0.0-1628.g97ae6cc9.el9.x8 5.3 MB/s | 17 MB 00:03 2024-04-25T08:52:02.217 INFO:teuthology.orchestra.run.smithi193.stdout:(59/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 1.9 MB/s | 3.0 MB 00:01 2024-04-25T08:52:02.248 INFO:teuthology.orchestra.run.smithi043.stdout:(18/152): libradosstriper1-19.0.0-1628.g97ae6cc 2.3 MB/s | 501 kB 00:00 2024-04-25T08:52:02.307 INFO:teuthology.orchestra.run.smithi043.stdout:(19/152): python3-ceph-argparse-19.0.0-1628.g97 776 kB/s | 45 kB 00:00 2024-04-25T08:52:02.309 INFO:teuthology.orchestra.run.smithi193.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 831 kB/s | 582 kB 00:00 2024-04-25T08:52:02.366 INFO:teuthology.orchestra.run.smithi043.stdout:(20/152): python3-ceph-common-19.0.0-1628.g97ae 2.2 MB/s | 131 kB 00:00 2024-04-25T08:52:02.424 INFO:teuthology.orchestra.run.smithi043.stdout:(21/152): python3-cephfs-19.0.0-1628.g97ae6cc9. 2.7 MB/s | 162 kB 00:00 2024-04-25T08:52:02.492 INFO:teuthology.orchestra.run.smithi193.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 92 kB/s | 25 kB 00:00 2024-04-25T08:52:02.533 INFO:teuthology.orchestra.run.smithi043.stdout:(22/152): python3-rados-19.0.0-1628.g97ae6cc9.e 2.9 MB/s | 321 kB 00:00 2024-04-25T08:52:02.559 INFO:teuthology.orchestra.run.smithi193.stdout:(62/152): librdkafka-1.6.1-102.el9.x86_64.rpm 651 kB/s | 662 kB 00:01 2024-04-25T08:52:02.641 INFO:teuthology.orchestra.run.smithi043.stdout:(23/152): python3-rbd-19.0.0-1628.g97ae6cc9.el9 2.7 MB/s | 298 kB 00:00 2024-04-25T08:52:02.651 INFO:teuthology.orchestra.run.smithi193.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 139 kB/s | 47 kB 00:00 2024-04-25T08:52:02.708 INFO:teuthology.orchestra.run.smithi043.stdout:(24/152): python3-rgw-19.0.0-1628.g97ae6cc9.el9 1.5 MB/s | 99 kB 00:00 2024-04-25T08:52:02.768 INFO:teuthology.orchestra.run.smithi043.stdout:(25/152): rbd-fuse-19.0.0-1628.g97ae6cc9.el9.x8 1.4 MB/s | 85 kB 00:00 2024-04-25T08:52:02.918 INFO:teuthology.orchestra.run.smithi193.stdout:(64/152): openblas-0.3.21-2.el9.x86_64.rpm 132 kB/s | 35 kB 00:00 2024-04-25T08:52:03.026 INFO:teuthology.orchestra.run.smithi193.stdout:(65/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 548 kB/s | 292 kB 00:00 2024-04-25T08:52:03.076 INFO:teuthology.orchestra.run.smithi043.stdout:(26/152): librgw2-19.0.0-1628.g97ae6cc9.el9.x86 5.9 MB/s | 5.1 MB 00:00 2024-04-25T08:52:03.127 INFO:teuthology.orchestra.run.smithi043.stdout:(27/152): rbd-nbd-19.0.0-1628.g97ae6cc9.el9.x86 3.3 MB/s | 170 kB 00:00 2024-04-25T08:52:03.152 INFO:teuthology.orchestra.run.smithi193.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 318 kB/s | 188 kB 00:00 2024-04-25T08:52:03.177 INFO:teuthology.orchestra.run.smithi043.stdout:(28/152): ceph-grafana-dashboards-19.0.0-1628.g 633 kB/s | 31 kB 00:00 2024-04-25T08:52:03.235 INFO:teuthology.orchestra.run.smithi043.stdout:(29/152): ceph-mgr-cephadm-19.0.0-1628.g97ae6cc 2.3 MB/s | 138 kB 00:00 2024-04-25T08:52:03.319 INFO:teuthology.orchestra.run.smithi043.stdout:(30/152): rbd-mirror-19.0.0-1628.g97ae6cc9.el9. 5.6 MB/s | 3.1 MB 00:00 2024-04-25T08:52:03.511 INFO:teuthology.orchestra.run.smithi043.stdout:(31/152): ceph-mgr-dashboard-19.0.0-1628.g97ae6 6.5 MB/s | 1.8 MB 00:00 2024-04-25T08:52:03.586 INFO:teuthology.orchestra.run.smithi043.stdout:(32/152): ceph-mgr-modules-core-19.0.0-1628.g97 3.2 MB/s | 248 kB 00:00 2024-04-25T08:52:03.637 INFO:teuthology.orchestra.run.smithi043.stdout:(33/152): ceph-mgr-rook-19.0.0-1628.g97ae6cc9.e 987 kB/s | 49 kB 00:00 2024-04-25T08:52:03.687 INFO:teuthology.orchestra.run.smithi043.stdout:(34/152): ceph-prometheus-alerts-19.0.0-1628.g9 308 kB/s | 15 kB 00:00 2024-04-25T08:52:03.771 INFO:teuthology.orchestra.run.smithi043.stdout:(35/152): ceph-volume-19.0.0-1628.g97ae6cc9.el9 3.1 MB/s | 264 kB 00:00 2024-04-25T08:52:03.904 INFO:teuthology.orchestra.run.smithi043.stdout:(36/152): cephadm-19.0.0-1628.g97ae6cc9.el9.noa 5.5 MB/s | 748 kB 00:00 2024-04-25T08:52:03.946 INFO:teuthology.orchestra.run.smithi043.stdout:(37/152): cephfs-top-19.0.0-1628.g97ae6cc9.el9. 438 kB/s | 18 kB 00:00 2024-04-25T08:52:04.055 INFO:teuthology.orchestra.run.smithi043.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 743 kB/s | 80 kB 00:00 2024-04-25T08:52:04.163 INFO:teuthology.orchestra.run.smithi043.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 7.3 MB/s | 803 kB 00:00 2024-04-25T08:52:04.214 INFO:teuthology.orchestra.run.smithi043.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 195 kB 00:00 2024-04-25T08:52:04.264 INFO:teuthology.orchestra.run.smithi043.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-04-25T08:52:04.339 INFO:teuthology.orchestra.run.smithi043.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-04-25T08:52:04.381 INFO:teuthology.orchestra.run.smithi043.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-04-25T08:52:04.423 INFO:teuthology.orchestra.run.smithi043.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-04-25T08:52:04.465 INFO:teuthology.orchestra.run.smithi043.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-04-25T08:52:04.469 INFO:teuthology.orchestra.run.smithi193.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 1.2 MB/s | 1.6 MB 00:01 2024-04-25T08:52:04.524 INFO:teuthology.orchestra.run.smithi043.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-04-25T08:52:04.616 INFO:teuthology.orchestra.run.smithi043.stdout:(47/152): ceph-mgr-diskprediction-local-19.0.0- 5.7 MB/s | 7.4 MB 00:01 2024-04-25T08:52:04.649 INFO:teuthology.orchestra.run.smithi043.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 5.4 MB/s | 689 kB 00:00 2024-04-25T08:52:04.691 INFO:teuthology.orchestra.run.smithi043.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-04-25T08:52:04.735 INFO:teuthology.orchestra.run.smithi043.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 493 kB/s | 21 kB 00:00 2024-04-25T08:52:04.786 INFO:teuthology.orchestra.run.smithi043.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-04-25T08:52:04.828 INFO:teuthology.orchestra.run.smithi043.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 790 kB/s | 33 kB 00:00 2024-04-25T08:52:04.861 INFO:teuthology.orchestra.run.smithi043.stdout:(53/152): bison-3.7.4-5.el9.x86_64.rpm 3.8 MB/s | 943 kB 00:00 2024-04-25T08:52:04.953 INFO:teuthology.orchestra.run.smithi043.stdout:(54/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-04-25T08:52:04.977 INFO:teuthology.orchestra.run.smithi193.stdout:(68/152): openblas-openmp-0.3.21-2.el9.x86_64.r 2.3 MB/s | 4.7 MB 00:02 2024-04-25T08:52:04.978 INFO:teuthology.orchestra.run.smithi043.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 156 kB/s | 18 kB 00:00 2024-04-25T08:52:04.995 INFO:teuthology.orchestra.run.smithi043.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 267 kB/s | 11 kB 00:00 2024-04-25T08:52:05.021 INFO:teuthology.orchestra.run.smithi043.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 160 kB 00:00 2024-04-25T08:52:05.036 INFO:teuthology.orchestra.run.smithi193.stdout:(69/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 2.2 MB/s | 4.4 MB 00:02 2024-04-25T08:52:05.046 INFO:teuthology.orchestra.run.smithi043.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 899 kB/s | 45 kB 00:00 2024-04-25T08:52:05.079 INFO:teuthology.orchestra.run.smithi043.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-04-25T08:52:05.113 INFO:teuthology.orchestra.run.smithi043.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 582 kB 00:00 2024-04-25T08:52:05.130 INFO:teuthology.orchestra.run.smithi043.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 507 kB/s | 25 kB 00:00 2024-04-25T08:52:05.155 INFO:teuthology.orchestra.run.smithi043.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-04-25T08:52:05.189 INFO:teuthology.orchestra.run.smithi043.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.9 MB/s | 292 kB 00:00 2024-04-25T08:52:05.214 INFO:teuthology.orchestra.run.smithi043.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-04-25T08:52:05.231 INFO:teuthology.orchestra.run.smithi043.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 840 kB/s | 35 kB 00:00 2024-04-25T08:52:05.386 INFO:teuthology.orchestra.run.smithi193.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 994 kB/s | 911 kB 00:00 2024-04-25T08:52:05.390 INFO:teuthology.orchestra.run.smithi043.stdout:(66/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 28 MB/s | 4.4 MB 00:00 2024-04-25T08:52:05.411 INFO:teuthology.orchestra.run.smithi193.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 307 kB/s | 133 kB 00:00 2024-04-25T08:52:05.465 INFO:teuthology.orchestra.run.smithi043.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 19 MB/s | 4.7 MB 00:00 2024-04-25T08:52:05.515 INFO:teuthology.orchestra.run.smithi043.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-04-25T08:52:05.549 INFO:teuthology.orchestra.run.smithi043.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 11 MB/s | 911 kB 00:00 2024-04-25T08:52:05.574 INFO:teuthology.orchestra.run.smithi043.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-04-25T08:52:05.641 INFO:teuthology.orchestra.run.smithi043.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-04-25T08:52:05.691 INFO:teuthology.orchestra.run.smithi043.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-04-25T08:52:05.761 INFO:teuthology.orchestra.run.smithi193.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 250 kB/s | 87 kB 00:00 2024-04-25T08:52:05.767 INFO:teuthology.orchestra.run.smithi043.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 27 MB/s | 5.8 MB 00:00 2024-04-25T08:52:05.818 INFO:teuthology.orchestra.run.smithi043.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-04-25T08:52:05.893 INFO:teuthology.orchestra.run.smithi043.stdout:(75/152): python3-babel-2.9.1-2.el9.noarch.rpm 30 MB/s | 6.0 MB 00:00 2024-04-25T08:52:05.918 INFO:teuthology.orchestra.run.smithi043.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-04-25T08:52:05.943 INFO:teuthology.orchestra.run.smithi043.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-04-25T08:52:05.968 INFO:teuthology.orchestra.run.smithi043.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 380 kB/s | 19 kB 00:00 2024-04-25T08:52:06.002 INFO:teuthology.orchestra.run.smithi043.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-04-25T08:52:06.027 INFO:teuthology.orchestra.run.smithi043.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 591 kB/s | 35 kB 00:00 2024-04-25T08:52:06.103 INFO:teuthology.orchestra.run.smithi043.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-04-25T08:52:06.170 INFO:teuthology.orchestra.run.smithi043.stdout:(82/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 31 MB/s | 5.1 MB 00:00 2024-04-25T08:52:06.195 INFO:teuthology.orchestra.run.smithi043.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-04-25T08:52:06.220 INFO:teuthology.orchestra.run.smithi043.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-04-25T08:52:06.246 INFO:teuthology.orchestra.run.smithi043.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 842 kB/s | 42 kB 00:00 2024-04-25T08:52:06.272 INFO:teuthology.orchestra.run.smithi043.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-04-25T08:52:06.296 INFO:teuthology.orchestra.run.smithi043.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-25T08:52:06.321 INFO:teuthology.orchestra.run.smithi043.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-04-25T08:52:06.346 INFO:teuthology.orchestra.run.smithi043.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-25T08:52:06.372 INFO:teuthology.orchestra.run.smithi043.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-04-25T08:52:06.423 INFO:teuthology.orchestra.run.smithi043.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 832 kB/s | 42 kB 00:00 2024-04-25T08:52:06.464 INFO:teuthology.orchestra.run.smithi043.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-04-25T08:52:06.478 INFO:teuthology.orchestra.run.smithi193.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 946 kB/s | 1.0 MB 00:01 2024-04-25T08:52:06.515 INFO:teuthology.orchestra.run.smithi043.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-25T08:52:06.557 INFO:teuthology.orchestra.run.smithi043.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 861 kB/s | 36 kB 00:00 2024-04-25T08:52:06.599 INFO:teuthology.orchestra.run.smithi043.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-04-25T08:52:06.649 INFO:teuthology.orchestra.run.smithi043.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-04-25T08:52:06.683 INFO:teuthology.orchestra.run.smithi043.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 1.7 MB/s | 58 kB 00:00 2024-04-25T08:52:06.725 INFO:teuthology.orchestra.run.smithi043.stdout:(98/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 2.2 MB/s | 94 kB 00:00 2024-04-25T08:52:06.766 INFO:teuthology.orchestra.run.smithi043.stdout:(99/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-04-25T08:52:06.925 INFO:teuthology.orchestra.run.smithi043.stdout:(100/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-04-25T08:52:06.942 INFO:teuthology.orchestra.run.smithi043.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 255 kB/s | 45 kB 00:00 2024-04-25T08:52:07.012 INFO:teuthology.orchestra.run.smithi193.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 2.9 MB/s | 5.8 MB 00:01 2024-04-25T08:52:07.037 INFO:teuthology.orchestra.run.smithi193.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 437 kB/s | 244 kB 00:00 2024-04-25T08:52:07.067 INFO:teuthology.orchestra.run.smithi043.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 157 kB/s | 22 kB 00:00 2024-04-25T08:52:07.218 INFO:teuthology.orchestra.run.smithi043.stdout:(103/152): abseil-cpp-20211102.0-3.el9.x86_64.r 3.6 MB/s | 548 kB 00:00 2024-04-25T08:52:07.251 INFO:teuthology.orchestra.run.smithi043.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.7 MB/s | 863 kB 00:00 2024-04-25T08:52:07.276 INFO:teuthology.orchestra.run.smithi043.stdout:(105/152): gperftools-libs-2.9.1-2.el9.x86_64.r 5.2 MB/s | 309 kB 00:00 2024-04-25T08:52:07.321 INFO:teuthology.orchestra.run.smithi193.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 90 kB/s | 26 kB 00:00 2024-04-25T08:52:07.335 INFO:teuthology.orchestra.run.smithi043.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 233 kB/s | 19 kB 00:00 2024-04-25T08:52:07.377 INFO:teuthology.orchestra.run.smithi043.stdout:(107/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 618 kB/s | 26 kB 00:00 2024-04-25T08:52:07.419 INFO:teuthology.orchestra.run.smithi043.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 1.1 MB/s | 49 kB 00:00 2024-04-25T08:52:07.461 INFO:teuthology.orchestra.run.smithi043.stdout:(109/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2024-04-25T08:52:07.521 INFO:teuthology.orchestra.run.smithi193.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 487 kB/s | 248 kB 00:00 2024-04-25T08:52:07.529 INFO:teuthology.orchestra.run.smithi043.stdout:(110/152): libarrow-9.0.0-10.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-04-25T08:52:07.554 INFO:teuthology.orchestra.run.smithi043.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-04-25T08:52:07.588 INFO:teuthology.orchestra.run.smithi193.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 72 kB/s | 19 kB 00:00 2024-04-25T08:52:07.595 INFO:teuthology.orchestra.run.smithi043.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 12 MB/s | 838 kB 00:00 2024-04-25T08:52:07.620 INFO:teuthology.orchestra.run.smithi043.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 8.0 MB/s | 548 kB 00:00 2024-04-25T08:52:07.646 INFO:teuthology.orchestra.run.smithi043.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 863 kB/s | 43 kB 00:00 2024-04-25T08:52:07.671 INFO:teuthology.orchestra.run.smithi043.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 642 kB/s | 32 kB 00:00 2024-04-25T08:52:07.696 INFO:teuthology.orchestra.run.smithi043.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 299 kB/s | 15 kB 00:00 2024-04-25T08:52:07.721 INFO:teuthology.orchestra.run.smithi043.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-04-25T08:52:07.747 INFO:teuthology.orchestra.run.smithi043.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 7.0 MB/s | 358 kB 00:00 2024-04-25T08:52:07.772 INFO:teuthology.orchestra.run.smithi043.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-04-25T08:52:07.797 INFO:teuthology.orchestra.run.smithi043.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 702 kB/s | 35 kB 00:00 2024-04-25T08:52:07.823 INFO:teuthology.orchestra.run.smithi043.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 4.0 MB/s | 206 kB 00:00 2024-04-25T08:52:07.854 INFO:teuthology.orchestra.run.smithi193.stdout:(79/152): python3-babel-2.9.1-2.el9.noarch.rpm 2.8 MB/s | 6.0 MB 00:02 2024-04-25T08:52:07.864 INFO:teuthology.orchestra.run.smithi043.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 3.4 MB/s | 144 kB 00:00 2024-04-25T08:52:07.907 INFO:teuthology.orchestra.run.smithi043.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 257 kB/s | 11 kB 00:00 2024-04-25T08:52:07.965 INFO:teuthology.orchestra.run.smithi043.stdout:(124/152): python3-grpcio-1.46.7-10.el9.x86_64. 12 MB/s | 2.0 MB 00:00 2024-04-25T08:52:07.971 INFO:teuthology.orchestra.run.smithi193.stdout:(80/152): python3-mako-1.1.4-6.el9.noarch.rpm 382 kB/s | 172 kB 00:00 2024-04-25T08:52:07.990 INFO:teuthology.orchestra.run.smithi043.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 212 kB/s | 18 kB 00:00 2024-04-25T08:52:07.997 INFO:teuthology.orchestra.run.smithi193.stdout:(81/152): python3-markupsafe-1.1.1-12.el9.x86_6 85 kB/s | 35 kB 00:00 2024-04-25T08:52:08.016 INFO:teuthology.orchestra.run.smithi043.stdout:(126/152): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-04-25T08:52:08.041 INFO:teuthology.orchestra.run.smithi043.stdout:(127/152): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-04-25T08:52:08.066 INFO:teuthology.orchestra.run.smithi043.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-04-25T08:52:08.108 INFO:teuthology.orchestra.run.smithi043.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 1.1 MB/s | 46 kB 00:00 2024-04-25T08:52:08.162 INFO:teuthology.orchestra.run.smithi043.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 8.5 MB/s | 1.0 MB 00:00 2024-04-25T08:52:08.192 INFO:teuthology.orchestra.run.smithi043.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 943 kB/s | 79 kB 00:00 2024-04-25T08:52:08.217 INFO:teuthology.orchestra.run.smithi043.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 1.0 MB/s | 58 kB 00:00 2024-04-25T08:52:08.259 INFO:teuthology.orchestra.run.smithi043.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-04-25T08:52:08.285 INFO:teuthology.orchestra.run.smithi043.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 246 kB/s | 16 kB 00:00 2024-04-25T08:52:08.310 INFO:teuthology.orchestra.run.smithi043.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-04-25T08:52:08.335 INFO:teuthology.orchestra.run.smithi043.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-04-25T08:52:08.361 INFO:teuthology.orchestra.run.smithi043.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 611 kB/s | 31 kB 00:00 2024-04-25T08:52:08.386 INFO:teuthology.orchestra.run.smithi043.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 3.6 MB/s | 188 kB 00:00 2024-04-25T08:52:08.411 INFO:teuthology.orchestra.run.smithi043.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-04-25T08:52:08.436 INFO:teuthology.orchestra.run.smithi043.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 714 kB/s | 36 kB 00:00 2024-04-25T08:52:08.462 INFO:teuthology.orchestra.run.smithi043.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-04-25T08:52:08.574 INFO:teuthology.orchestra.run.smithi193.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 416 kB/s | 250 kB 00:00 2024-04-25T08:52:08.606 INFO:teuthology.orchestra.run.smithi193.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 365 kB/s | 222 kB 00:00 2024-04-25T08:52:08.879 INFO:teuthology.orchestra.run.smithi043.stdout:(142/152): ceph-test-19.0.0-1628.g97ae6cc9.el9. 6.5 MB/s | 47 MB 00:07 2024-04-25T08:52:08.904 INFO:teuthology.orchestra.run.smithi043.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 203 kB/s | 90 kB 00:00 2024-04-25T08:52:08.929 INFO:teuthology.orchestra.run.smithi043.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 466 kB/s | 230 kB 00:00 2024-04-25T08:52:08.954 INFO:teuthology.orchestra.run.smithi043.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 399 kB/s | 20 kB 00:00 2024-04-25T08:52:08.956 INFO:teuthology.orchestra.run.smithi193.stdout:(84/152): python3-prettytable-0.7.2-27.el9.noar 121 kB/s | 42 kB 00:00 2024-04-25T08:52:08.980 INFO:teuthology.orchestra.run.smithi043.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-04-25T08:52:08.989 INFO:teuthology.orchestra.run.smithi193.stdout:(85/152): python3-packaging-20.9-5.el9.noarch.r 186 kB/s | 77 kB 00:00 2024-04-25T08:52:09.055 INFO:teuthology.orchestra.run.smithi043.stdout:(147/152): python3-werkzeug-2.0.3-3.el9.1.noarc 2.4 MB/s | 427 kB 00:00 2024-04-25T08:52:09.097 INFO:teuthology.orchestra.run.smithi043.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-25T08:52:09.331 INFO:teuthology.orchestra.run.smithi043.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 5.8 MB/s | 1.3 MB 00:00 2024-04-25T08:52:09.448 INFO:teuthology.orchestra.run.smithi043.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 21 MB/s | 2.4 MB 00:00 2024-04-25T08:52:09.490 INFO:teuthology.orchestra.run.smithi193.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 505 kB/s | 269 kB 00:00 2024-04-25T08:52:09.532 INFO:teuthology.orchestra.run.smithi193.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 293 kB/s | 159 kB 00:00 2024-04-25T08:52:09.614 INFO:teuthology.orchestra.run.smithi043.stdout:(151/152): librbd1-19.0.0-1628.g97ae6cc9.el9.x8 5.6 MB/s | 3.1 MB 00:00 2024-04-25T08:52:09.632 INFO:teuthology.orchestra.run.smithi193.stdout:(88/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.9 MB/s | 5.1 MB 00:01 2024-04-25T08:52:09.764 INFO:teuthology.orchestra.run.smithi043.stdout:(152/152): librados2-19.0.0-1628.g97ae6cc9.el9. 4.3 MB/s | 3.4 MB 00:00 2024-04-25T08:52:09.767 INFO:teuthology.orchestra.run.smithi043.stdout:-------------------------------------------------------------------------------- 2024-04-25T08:52:09.767 INFO:teuthology.orchestra.run.smithi043.stdout:Total 15 MB/s | 215 MB 00:14 2024-04-25T08:52:09.899 INFO:teuthology.orchestra.run.smithi193.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 139 kB/s | 51 kB 00:00 2024-04-25T08:52:09.999 INFO:teuthology.orchestra.run.smithi193.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 146 kB/s | 54 kB 00:00 2024-04-25T08:52:10.024 INFO:teuthology.orchestra.run.smithi193.stdout:(91/152): python3-pyasn1-modules-0.4.8-6.el9.no 522 kB/s | 279 kB 00:00 2024-04-25T08:52:10.266 INFO:teuthology.orchestra.run.smithi193.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 156 kB/s | 42 kB 00:00 2024-04-25T08:52:10.366 INFO:teuthology.orchestra.run.smithi193.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 129 kB/s | 44 kB 00:00 2024-04-25T08:52:10.633 INFO:teuthology.orchestra.run.smithi193.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 135 kB/s | 36 kB 00:00 2024-04-25T08:52:10.783 INFO:teuthology.orchestra.run.smithi193.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 427 kB/s | 221 kB 00:00 2024-04-25T08:52:10.866 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction check 2024-04-25T08:52:11.024 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction check succeeded. 2024-04-25T08:52:11.024 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction test 2024-04-25T08:52:11.392 INFO:teuthology.orchestra.run.smithi193.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 62 kB/s | 47 kB 00:00 2024-04-25T08:52:11.426 INFO:teuthology.orchestra.run.smithi193.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 475 kB/s | 305 kB 00:00 2024-04-25T08:52:11.751 INFO:teuthology.orchestra.run.smithi193.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 161 kB/s | 58 kB 00:00 2024-04-25T08:52:11.876 INFO:teuthology.orchestra.run.smithi193.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 208 kB/s | 94 kB 00:00 2024-04-25T08:52:12.176 INFO:teuthology.orchestra.run.smithi193.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 150 kB/s | 64 kB 00:00 2024-04-25T08:52:12.218 INFO:teuthology.orchestra.run.smithi193.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 131 kB/s | 45 kB 00:00 2024-04-25T08:52:12.669 INFO:teuthology.orchestra.run.smithi193.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 45 kB/s | 22 kB 00:00 2024-04-25T08:52:12.857 INFO:teuthology.orchestra.run.smithi043.stdout:Transaction test succeeded. 2024-04-25T08:52:12.857 INFO:teuthology.orchestra.run.smithi043.stdout:Running transaction 2024-04-25T08:52:12.860 INFO:teuthology.orchestra.run.smithi193.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.3 MB/s | 863 kB 00:00 2024-04-25T08:52:12.886 INFO:teuthology.orchestra.run.smithi193.stdout:(104/152): abseil-cpp-20211102.0-3.el9.x86_64.r 2.5 MB/s | 548 kB 00:00 2024-04-25T08:52:12.936 INFO:teuthology.orchestra.run.smithi193.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 389 kB/s | 19 kB 00:00 2024-04-25T08:52:13.103 INFO:teuthology.orchestra.run.smithi193.stdout:(106/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 5.1 MB/s | 16 MB 00:03 2024-04-25T08:52:13.203 INFO:teuthology.orchestra.run.smithi193.stdout:(107/152): libarrow-9.0.0-10.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-04-25T08:52:13.228 INFO:teuthology.orchestra.run.smithi193.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 208 kB/s | 26 kB 00:00 2024-04-25T08:52:13.262 INFO:teuthology.orchestra.run.smithi193.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 834 kB/s | 49 kB 00:00 2024-04-25T08:52:13.312 INFO:teuthology.orchestra.run.smithi193.stdout:(110/152): gperftools-libs-2.9.1-2.el9.x86_64.r 684 kB/s | 309 kB 00:00 2024-04-25T08:52:13.337 INFO:teuthology.orchestra.run.smithi193.stdout:(111/152): libunwind-1.6.2-1.el9.x86_64.rpm 619 kB/s | 67 kB 00:00 2024-04-25T08:52:13.363 INFO:teuthology.orchestra.run.smithi193.stdout:(112/152): luarocks-3.9.2-1.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-04-25T08:52:13.421 INFO:teuthology.orchestra.run.smithi193.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 747 kB/s | 43 kB 00:00 2024-04-25T08:52:13.472 INFO:teuthology.orchestra.run.smithi193.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 4.0 MB/s | 548 kB 00:00 2024-04-25T08:52:13.497 INFO:teuthology.orchestra.run.smithi193.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 428 kB/s | 32 kB 00:00 2024-04-25T08:52:13.530 INFO:teuthology.orchestra.run.smithi193.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 257 kB/s | 15 kB 00:00 2024-04-25T08:52:13.556 INFO:teuthology.orchestra.run.smithi193.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-04-25T08:52:13.673 INFO:teuthology.orchestra.run.smithi193.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 2.5 MB/s | 358 kB 00:00 2024-04-25T08:52:13.832 INFO:teuthology.orchestra.run.smithi193.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.1 MB/s | 303 kB 00:00 2024-04-25T08:52:13.857 INFO:teuthology.orchestra.run.smithi193.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 192 kB/s | 35 kB 00:00 2024-04-25T08:52:13.882 INFO:teuthology.orchestra.run.smithi193.stdout:(121/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.4 MB/s | 838 kB 00:00 2024-04-25T08:52:13.907 INFO:teuthology.orchestra.run.smithi193.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-04-25T08:52:13.957 INFO:teuthology.orchestra.run.smithi193.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 214 kB/s | 11 kB 00:00 2024-04-25T08:52:14.000 INFO:teuthology.orchestra.run.smithi193.stdout:(124/152): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-04-25T08:52:14.025 INFO:teuthology.orchestra.run.smithi193.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 264 kB/s | 18 kB 00:00 2024-04-25T08:52:14.050 INFO:teuthology.orchestra.run.smithi193.stdout:(126/152): python3-grpcio-tools-1.46.7-10.el9.x 862 kB/s | 144 kB 00:00 2024-04-25T08:52:14.075 INFO:teuthology.orchestra.run.smithi193.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 307 kB/s | 23 kB 00:00 2024-04-25T08:52:14.101 INFO:teuthology.orchestra.run.smithi193.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 257 kB/s | 19 kB 00:00 2024-04-25T08:52:14.126 INFO:teuthology.orchestra.run.smithi193.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 261 kB/s | 20 kB 00:00 2024-04-25T08:52:14.168 INFO:teuthology.orchestra.run.smithi193.stdout:(130/152): python3-kubernetes-26.1.0-2.el9.noar 11 MB/s | 1.0 MB 00:00 2024-04-25T08:52:14.193 INFO:teuthology.orchestra.run.smithi193.stdout:(131/152): python3-logutils-0.3.5-21.el9.noarch 502 kB/s | 46 kB 00:00 2024-04-25T08:52:14.210 INFO:teuthology.orchestra.run.smithi193.stdout:(132/152): python3-more-itertools-8.12.0-2.el9. 941 kB/s | 79 kB 00:00 2024-04-25T08:52:14.235 INFO:teuthology.orchestra.run.smithi193.stdout:(133/152): python3-natsort-7.1.1-5.el9.noarch.r 862 kB/s | 58 kB 00:00 2024-04-25T08:52:14.261 INFO:teuthology.orchestra.run.smithi193.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 326 kB/s | 16 kB 00:00 2024-04-25T08:52:14.294 INFO:teuthology.orchestra.run.smithi193.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-04-25T08:52:14.319 INFO:teuthology.orchestra.run.smithi193.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 484 kB/s | 28 kB 00:00 2024-04-25T08:52:14.344 INFO:teuthology.orchestra.run.smithi193.stdout:(137/152): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-04-25T08:52:14.361 INFO:teuthology.orchestra.run.smithi193.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 460 kB/s | 31 kB 00:00 2024-04-25T08:52:14.387 INFO:teuthology.orchestra.run.smithi193.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 2.7 MB/s | 188 kB 00:00 2024-04-25T08:52:14.404 INFO:teuthology.orchestra.run.smithi193.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-04-25T08:52:14.421 INFO:teuthology.orchestra.run.smithi193.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 608 kB/s | 36 kB 00:00 2024-04-25T08:52:14.438 INFO:teuthology.orchestra.run.smithi193.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-04-25T08:52:14.479 INFO:teuthology.orchestra.run.smithi193.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-04-25T08:52:14.505 INFO:teuthology.orchestra.run.smithi193.stdout:(144/152): python3-werkzeug-2.0.3-3.el9.1.noarc 6.2 MB/s | 427 kB 00:00 2024-04-25T08:52:14.530 INFO:teuthology.orchestra.run.smithi193.stdout:(145/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-04-25T08:52:14.547 INFO:teuthology.orchestra.run.smithi193.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 298 kB/s | 20 kB 00:00 2024-04-25T08:52:14.572 INFO:teuthology.orchestra.run.smithi193.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-04-25T08:52:14.689 INFO:teuthology.orchestra.run.smithi193.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-25T08:52:14.972 INFO:teuthology.orchestra.run.smithi193.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 4.8 MB/s | 1.3 MB 00:00 2024-04-25T08:52:15.190 INFO:teuthology.orchestra.run.smithi193.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 11 MB/s | 2.4 MB 00:00 2024-04-25T08:52:15.881 INFO:teuthology.orchestra.run.smithi193.stdout:(151/152): librbd1-19.0.0-1628.g97ae6cc9.el9.x8 2.4 MB/s | 3.1 MB 00:01 2024-04-25T08:52:16.206 INFO:teuthology.orchestra.run.smithi193.stdout:(152/152): librados2-19.0.0-1628.g97ae6cc9.el9. 2.0 MB/s | 3.4 MB 00:01 2024-04-25T08:52:16.209 INFO:teuthology.orchestra.run.smithi193.stdout:-------------------------------------------------------------------------------- 2024-04-25T08:52:16.209 INFO:teuthology.orchestra.run.smithi193.stdout:Total 9.1 MB/s | 215 MB 00:23 2024-04-25T08:52:17.296 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction check 2024-04-25T08:52:17.437 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction check succeeded. 2024-04-25T08:52:17.437 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction test 2024-04-25T08:52:19.105 INFO:teuthology.orchestra.run.smithi043.stdout: Preparing : 1/1 2024-04-25T08:52:19.168 INFO:teuthology.orchestra.run.smithi193.stdout:Transaction test succeeded. 2024-04-25T08:52:19.168 INFO:teuthology.orchestra.run.smithi193.stdout:Running transaction 2024-04-25T08:52:19.218 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T08:52:19.265 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T08:52:19.319 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T08:52:19.362 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T08:52:19.386 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T08:52:19.431 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T08:52:19.772 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T08:52:19.887 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T08:52:20.012 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T08:52:20.038 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T08:52:20.131 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T08:52:20.159 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T08:52:20.315 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T08:52:20.366 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T08:52:20.393 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T08:52:20.490 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T08:52:20.519 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T08:52:20.582 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T08:52:20.603 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T08:52:20.627 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T08:52:20.830 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T08:52:20.870 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T08:52:20.983 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T08:52:21.077 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T08:52:21.119 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T08:52:21.155 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T08:52:21.230 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T08:52:21.278 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T08:52:21.338 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T08:52:21.371 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T08:52:21.399 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T08:52:21.471 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T08:52:21.531 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T08:52:21.560 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T08:52:21.596 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T08:52:21.674 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T08:52:21.715 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 31/156 2024-04-25T08:52:21.758 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 32/156 2024-04-25T08:52:21.786 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 33/156 2024-04-25T08:52:21.859 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T08:52:21.902 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T08:52:21.988 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T08:52:22.132 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T08:52:22.197 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T08:52:22.754 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T08:52:22.844 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T08:52:22.879 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T08:52:22.922 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T08:52:22.960 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 43/156 2024-04-25T08:52:23.003 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T08:52:23.171 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T08:52:23.221 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T08:52:23.278 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T08:52:23.352 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T08:52:23.397 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T08:52:23.433 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T08:52:23.481 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T08:52:23.536 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T08:52:23.717 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T08:52:23.836 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T08:52:23.987 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T08:52:24.034 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T08:52:24.062 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T08:52:24.104 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T08:52:24.203 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T08:52:24.237 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T08:52:24.277 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T08:52:24.302 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T08:52:24.327 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T08:52:24.404 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T08:52:24.534 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T08:52:24.568 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T08:52:24.596 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T08:52:24.630 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T08:52:24.670 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T08:52:24.709 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T08:52:24.739 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T08:52:24.774 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T08:52:24.807 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T08:52:24.850 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T08:52:24.883 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T08:52:24.973 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T08:52:25.023 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T08:52:25.061 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T08:52:25.109 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T08:52:25.221 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T08:52:25.273 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T08:52:25.296 INFO:teuthology.orchestra.run.smithi193.stdout: Preparing : 1/1 2024-04-25T08:52:25.385 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T08:52:25.423 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T08:52:25.484 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T08:52:25.518 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T08:52:25.544 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T08:52:25.579 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T08:52:25.903 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T08:52:26.012 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T08:52:26.030 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T08:52:26.088 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T08:52:26.119 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T08:52:26.134 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T08:52:26.163 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T08:52:26.213 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T08:52:26.255 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T08:52:26.270 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T08:52:26.284 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T08:52:26.360 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T08:52:26.414 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T08:52:26.435 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T08:52:26.598 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T08:52:26.622 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T08:52:26.626 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T08:52:26.650 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T08:52:26.707 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T08:52:26.736 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T08:52:26.760 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T08:52:26.814 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T08:52:26.835 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T08:52:26.856 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T08:52:27.053 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T08:52:27.085 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T08:52:27.194 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T08:52:27.290 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T08:52:27.326 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T08:52:27.362 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T08:52:27.379 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T08:52:27.438 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T08:52:27.485 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T08:52:27.495 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T08:52:27.529 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T08:52:27.561 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T08:52:27.589 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T08:52:27.673 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T08:52:27.754 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T08:52:27.786 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T08:52:27.837 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T08:52:27.952 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T08:52:27.992 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 31/156 2024-04-25T08:52:28.040 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 32/156 2024-04-25T08:52:28.070 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 33/156 2024-04-25T08:52:28.183 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T08:52:28.244 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T08:52:28.325 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T08:52:28.459 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T08:52:28.519 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T08:52:28.793 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T08:52:28.865 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T08:52:28.893 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T08:52:28.926 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T08:52:29.051 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T08:52:29.139 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T08:52:29.168 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T08:52:29.210 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T08:52:29.249 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 43/156 2024-04-25T08:52:29.254 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T08:52:29.288 INFO:teuthology.orchestra.run.smithi043.stdout: Upgrading : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T08:52:29.290 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T08:52:29.380 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T08:52:29.410 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 100/156 2024-04-25T08:52:29.447 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T08:52:29.462 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T08:52:29.518 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T08:52:29.573 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T08:52:29.646 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T08:52:29.677 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T08:52:29.713 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T08:52:29.760 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T08:52:29.796 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T08:52:29.839 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T08:52:29.890 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T08:52:29.926 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T08:52:29.950 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T08:52:30.021 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T08:52:30.025 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T08:52:30.068 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 104/156 2024-04-25T08:52:30.137 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T08:52:30.166 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T08:52:30.208 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T08:52:30.302 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T08:52:30.349 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T08:52:30.389 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T08:52:30.415 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T08:52:30.439 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T08:52:30.514 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T08:52:30.649 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T08:52:30.680 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T08:52:30.708 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T08:52:30.742 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T08:52:30.781 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T08:52:30.813 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T08:52:30.843 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T08:52:30.877 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T08:52:30.911 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T08:52:30.952 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T08:52:30.987 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T08:52:31.073 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T08:52:31.117 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T08:52:31.156 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T08:52:31.204 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T08:52:31.301 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T08:52:31.349 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T08:52:32.078 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T08:52:32.123 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T08:52:32.156 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T08:52:32.257 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T08:52:32.330 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T08:52:32.339 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T08:52:32.364 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T08:52:32.428 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T08:52:32.475 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T08:52:32.651 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T08:52:32.672 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T08:52:32.750 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T08:52:32.931 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T08:52:32.968 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T08:52:33.067 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T08:52:33.067 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T08:52:33.067 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T08:52:33.067 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:52:33.094 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T08:52:33.397 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T08:52:33.513 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T08:52:34.768 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T08:52:34.850 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T08:52:34.887 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T08:52:34.920 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T08:52:35.231 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T08:52:35.275 INFO:teuthology.orchestra.run.smithi193.stdout: Upgrading : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T08:52:35.365 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T08:52:35.396 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 100/156 2024-04-25T08:52:35.432 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T08:52:35.885 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T08:52:35.910 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T08:52:35.981 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T08:52:36.020 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 104/156 2024-04-25T08:52:38.243 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T08:52:38.283 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T08:52:38.825 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T08:52:38.870 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T08:52:38.960 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T08:52:38.960 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T08:52:38.960 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T08:52:38.960 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:52:38.988 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T08:52:45.058 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T08:52:45.059 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /sys 2024-04-25T08:52:45.059 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /proc 2024-04-25T08:52:45.059 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /mnt 2024-04-25T08:52:45.059 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /var/tmp 2024-04-25T08:52:45.059 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /home 2024-04-25T08:52:45.059 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /root 2024-04-25T08:52:45.059 INFO:teuthology.orchestra.run.smithi043.stdout:skipping the directory /tmp 2024-04-25T08:52:45.059 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:52:45.233 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T08:52:45.249 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T08:52:45.249 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:52:45.249 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T08:52:45.249 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:52:46.777 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T08:52:46.804 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T08:52:46.804 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:52:46.804 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T08:52:46.805 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T08:52:46.805 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T08:52:46.805 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:52:47.028 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T08:52:47.052 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T08:52:47.052 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:52:47.052 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T08:52:47.052 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T08:52:47.052 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T08:52:47.052 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:52:47.512 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T08:52:47.536 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T08:52:47.536 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:52:47.536 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T08:52:47.536 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T08:52:47.536 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T08:52:47.536 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:52:47.643 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T08:52:47.701 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T08:52:47.765 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T08:52:47.959 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T08:52:48.014 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T08:52:48.094 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T08:52:48.185 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T08:52:48.325 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T08:52:48.363 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T08:52:48.395 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T08:52:48.422 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T08:52:48.785 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T08:52:48.822 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T08:52:48.854 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 124/156 2024-04-25T08:52:48.917 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 125/156 2024-04-25T08:52:48.945 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T08:52:49.186 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T08:52:49.220 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T08:52:50.234 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T08:52:50.329 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T08:52:50.437 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T08:52:50.565 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T08:52:50.566 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /sys 2024-04-25T08:52:50.566 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /proc 2024-04-25T08:52:50.566 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /mnt 2024-04-25T08:52:50.566 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /var/tmp 2024-04-25T08:52:50.566 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /home 2024-04-25T08:52:50.566 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /root 2024-04-25T08:52:50.567 INFO:teuthology.orchestra.run.smithi193.stdout:skipping the directory /tmp 2024-04-25T08:52:50.567 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:52:50.599 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 129/156 2024-04-25T08:52:50.623 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T08:52:50.647 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T08:52:50.647 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:52:50.647 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T08:52:50.647 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T08:52:50.647 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T08:52:50.647 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:52:50.699 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T08:52:50.710 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T08:52:50.723 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T08:52:50.738 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T08:52:50.738 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:52:50.738 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T08:52:50.738 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:52:50.802 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 132/156 2024-04-25T08:52:50.856 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 133/156 2024-04-25T08:52:51.758 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T08:52:51.785 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T08:52:51.808 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T08:52:51.808 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:52:51.808 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T08:52:51.809 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T08:52:51.809 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T08:52:51.809 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:52:51.852 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T08:52:51.875 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T08:52:51.875 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:52:51.875 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T08:52:51.875 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:52:51.947 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T08:52:51.971 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T08:52:51.971 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:52:51.971 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T08:52:51.971 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T08:52:51.971 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T08:52:51.971 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:52:52.226 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T08:52:52.252 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T08:52:52.252 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:52:52.252 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T08:52:52.252 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T08:52:52.252 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T08:52:52.252 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:52:52.295 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T08:52:52.318 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T08:52:52.319 INFO:teuthology.orchestra.run.smithi043.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:52:52.319 INFO:teuthology.orchestra.run.smithi043.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T08:52:52.319 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T08:52:52.319 INFO:teuthology.orchestra.run.smithi043.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T08:52:52.319 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:52:52.452 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T08:52:52.474 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T08:52:52.474 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:52:52.474 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T08:52:52.474 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T08:52:52.474 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T08:52:52.475 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:52:52.911 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T08:52:52.933 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T08:52:52.933 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:52:52.933 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T08:52:52.933 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T08:52:52.934 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T08:52:52.934 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:52:53.041 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T08:52:53.098 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T08:52:53.154 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T08:52:53.348 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T08:52:53.395 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T08:52:53.473 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T08:52:53.562 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T08:52:53.711 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T08:52:53.753 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T08:52:53.785 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T08:52:53.812 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T08:52:54.280 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T08:52:54.319 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T08:52:54.351 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 124/156 2024-04-25T08:52:54.411 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 125/156 2024-04-25T08:52:54.434 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T08:52:54.661 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T08:52:54.693 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T08:52:55.650 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T08:52:55.685 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T08:52:55.786 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T08:52:55.947 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 129/156 2024-04-25T08:52:55.979 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T08:52:56.002 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T08:52:56.002 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:52:56.002 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T08:52:56.002 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T08:52:56.002 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T08:52:56.002 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:52:56.046 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T08:52:56.057 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T08:52:56.154 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 132/156 2024-04-25T08:52:56.203 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 133/156 2024-04-25T08:52:56.990 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 139/156 2024-04-25T08:52:57.040 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 140/156 2024-04-25T08:52:57.054 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T08:52:57.081 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 141/156 2024-04-25T08:52:57.082 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T08:52:57.104 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T08:52:57.105 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:52:57.105 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T08:52:57.105 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T08:52:57.105 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T08:52:57.105 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:52:57.113 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T08:52:57.140 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T08:52:57.149 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T08:52:57.171 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T08:52:57.171 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:52:57.171 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T08:52:57.171 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:52:57.218 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T08:52:57.239 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T08:52:57.239 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:52:57.239 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T08:52:57.239 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T08:52:57.239 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T08:52:57.240 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:52:57.341 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 144/156 2024-04-25T08:52:57.386 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T08:52:57.416 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 146/156 2024-04-25T08:52:57.542 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T08:52:57.564 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T08:52:57.564 INFO:teuthology.orchestra.run.smithi193.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T08:52:57.564 INFO:teuthology.orchestra.run.smithi193.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T08:52:57.565 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T08:52:57.565 INFO:teuthology.orchestra.run.smithi193.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T08:52:57.565 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:52:58.036 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T08:52:58.072 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T08:52:58.092 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T08:52:59.010 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 148/156 2024-04-25T08:52:59.102 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T08:52:59.144 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T08:52:59.228 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T08:52:59.265 INFO:teuthology.orchestra.run.smithi043.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T08:52:59.265 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T08:52:59.349 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T08:52:59.382 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T08:52:59.382 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T08:52:59.476 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T08:52:59.550 INFO:teuthology.orchestra.run.smithi043.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T08:53:01.954 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 139/156 2024-04-25T08:53:01.995 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 140/156 2024-04-25T08:53:02.036 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 141/156 2024-04-25T08:53:02.068 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T08:53:02.095 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T08:53:02.285 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 144/156 2024-04-25T08:53:02.324 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T08:53:02.354 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 146/156 2024-04-25T08:53:02.953 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T08:53:02.993 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T08:53:03.012 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T08:53:03.837 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 148/156 2024-04-25T08:53:03.942 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T08:53:03.980 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T08:53:04.072 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T08:53:04.109 INFO:teuthology.orchestra.run.smithi193.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T08:53:04.110 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T08:53:04.177 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T08:53:04.201 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T08:53:04.202 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T08:53:04.282 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T08:53:04.350 INFO:teuthology.orchestra.run.smithi193.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T08:53:04.595 INFO:teuthology.orchestra.run.smithi043.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T08:53:04.595 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 1/156 2024-04-25T08:53:04.595 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2/156 2024-04-25T08:53:04.595 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 3/156 2024-04-25T08:53:04.596 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 4/156 2024-04-25T08:53:04.596 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 5/156 2024-04-25T08:53:04.596 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 6/156 2024-04-25T08:53:04.596 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 7/156 2024-04-25T08:53:04.596 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T08:53:04.596 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T08:53:04.596 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T08:53:04.596 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 11/156 2024-04-25T08:53:04.596 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 12/156 2024-04-25T08:53:04.596 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T08:53:04.596 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 14/156 2024-04-25T08:53:04.596 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 15/156 2024-04-25T08:53:04.596 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 16/156 2024-04-25T08:53:04.597 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 17/156 2024-04-25T08:53:04.597 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 18/156 2024-04-25T08:53:04.597 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 19/156 2024-04-25T08:53:04.597 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 20/156 2024-04-25T08:53:04.597 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 21/156 2024-04-25T08:53:04.597 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 22/156 2024-04-25T08:53:04.597 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 23/156 2024-04-25T08:53:04.597 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 24/156 2024-04-25T08:53:04.597 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 25/156 2024-04-25T08:53:04.597 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 26/156 2024-04-25T08:53:04.597 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 27/156 2024-04-25T08:53:04.597 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 28/156 2024-04-25T08:53:04.597 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 29/156 2024-04-25T08:53:04.597 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 30/156 2024-04-25T08:53:04.598 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 31/156 2024-04-25T08:53:04.598 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 32/156 2024-04-25T08:53:04.598 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 33/156 2024-04-25T08:53:04.598 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 34/156 2024-04-25T08:53:04.598 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 35/156 2024-04-25T08:53:04.598 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 36/156 2024-04-25T08:53:04.598 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 37/156 2024-04-25T08:53:04.598 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 38/156 2024-04-25T08:53:04.598 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 39/156 2024-04-25T08:53:04.598 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T08:53:04.598 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T08:53:04.598 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T08:53:04.598 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T08:53:04.599 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T08:53:04.599 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T08:53:04.599 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T08:53:04.599 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T08:53:04.599 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T08:53:04.599 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T08:53:04.599 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T08:53:04.599 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T08:53:04.599 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T08:53:04.599 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T08:53:04.599 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T08:53:04.599 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T08:53:04.599 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T08:53:04.599 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T08:53:04.600 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T08:53:04.600 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T08:53:04.600 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T08:53:04.600 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T08:53:04.600 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T08:53:04.600 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T08:53:04.600 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T08:53:04.600 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T08:53:04.600 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T08:53:04.600 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T08:53:04.600 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T08:53:04.600 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T08:53:04.600 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T08:53:04.601 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T08:53:04.601 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T08:53:04.601 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T08:53:04.601 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T08:53:04.601 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T08:53:04.601 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T08:53:04.601 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T08:53:04.601 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T08:53:04.601 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T08:53:04.601 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T08:53:04.601 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T08:53:04.601 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T08:53:04.601 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T08:53:04.601 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T08:53:04.602 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T08:53:04.602 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T08:53:04.602 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T08:53:04.602 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T08:53:04.602 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T08:53:04.602 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T08:53:04.602 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T08:53:04.602 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T08:53:04.602 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T08:53:04.602 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T08:53:04.602 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T08:53:04.602 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T08:53:04.602 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T08:53:04.603 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T08:53:04.603 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T08:53:04.603 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T08:53:04.603 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T08:53:04.603 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T08:53:04.603 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T08:53:04.603 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T08:53:04.603 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T08:53:04.603 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T08:53:04.603 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T08:53:04.603 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T08:53:04.603 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T08:53:04.603 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T08:53:04.603 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T08:53:04.604 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T08:53:04.604 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T08:53:04.604 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T08:53:04.604 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T08:53:04.604 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T08:53:04.604 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T08:53:04.604 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T08:53:04.604 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T08:53:04.604 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T08:53:04.604 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T08:53:04.604 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T08:53:04.604 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T08:53:04.604 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T08:53:04.604 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T08:53:04.605 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T08:53:04.605 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T08:53:04.605 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T08:53:04.605 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T08:53:04.605 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T08:53:04.605 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T08:53:04.605 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T08:53:04.605 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T08:53:04.605 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T08:53:04.605 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T08:53:04.605 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T08:53:04.605 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T08:53:04.605 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T08:53:04.605 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T08:53:04.606 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T08:53:04.606 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T08:53:04.606 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T08:53:04.606 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T08:53:04.606 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T08:53:04.606 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T08:53:04.606 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T08:53:04.606 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T08:53:04.606 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T08:53:04.606 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 149/156 2024-04-25T08:53:04.606 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T08:53:04.606 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 151/156 2024-04-25T08:53:04.606 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T08:53:04.606 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T08:53:04.607 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T08:53:04.607 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T08:53:06.668 INFO:teuthology.orchestra.run.smithi043.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T08:53:06.668 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:53:06.668 INFO:teuthology.orchestra.run.smithi043.stdout:Upgraded: 2024-04-25T08:53:06.668 INFO:teuthology.orchestra.run.smithi043.stdout: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:06.668 INFO:teuthology.orchestra.run.smithi043.stdout: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:06.669 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T08:53:06.669 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T08:53:06.669 INFO:teuthology.orchestra.run.smithi043.stdout:Installed: 2024-04-25T08:53:06.669 INFO:teuthology.orchestra.run.smithi043.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T08:53:06.669 INFO:teuthology.orchestra.run.smithi043.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T08:53:06.669 INFO:teuthology.orchestra.run.smithi043.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T08:53:06.669 INFO:teuthology.orchestra.run.smithi043.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T08:53:06.669 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:06.669 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:06.669 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:06.669 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:06.669 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:53:06.669 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:06.669 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:06.670 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:06.670 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:53:06.670 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:53:06.670 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:53:06.670 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:53:06.670 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:53:06.670 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:06.670 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:06.670 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:53:06.670 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:06.670 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:06.670 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:06.670 INFO:teuthology.orchestra.run.smithi043.stdout: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:53:06.670 INFO:teuthology.orchestra.run.smithi043.stdout: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:53:06.671 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:06.671 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:06.671 INFO:teuthology.orchestra.run.smithi043.stdout: cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:53:06.671 INFO:teuthology.orchestra.run.smithi043.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T08:53:06.671 INFO:teuthology.orchestra.run.smithi043.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T08:53:06.671 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T08:53:06.671 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T08:53:06.671 INFO:teuthology.orchestra.run.smithi043.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T08:53:06.671 INFO:teuthology.orchestra.run.smithi043.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T08:53:06.671 INFO:teuthology.orchestra.run.smithi043.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T08:53:06.671 INFO:teuthology.orchestra.run.smithi043.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T08:53:06.671 INFO:teuthology.orchestra.run.smithi043.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T08:53:06.671 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T08:53:06.671 INFO:teuthology.orchestra.run.smithi043.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T08:53:06.672 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:06.672 INFO:teuthology.orchestra.run.smithi043.stdout: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:06.672 INFO:teuthology.orchestra.run.smithi043.stdout: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:06.672 INFO:teuthology.orchestra.run.smithi043.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T08:53:06.672 INFO:teuthology.orchestra.run.smithi043.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T08:53:06.672 INFO:teuthology.orchestra.run.smithi043.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T08:53:06.672 INFO:teuthology.orchestra.run.smithi043.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T08:53:06.672 INFO:teuthology.orchestra.run.smithi043.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T08:53:06.672 INFO:teuthology.orchestra.run.smithi043.stdout: librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:06.672 INFO:teuthology.orchestra.run.smithi043.stdout: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:06.672 INFO:teuthology.orchestra.run.smithi043.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T08:53:06.672 INFO:teuthology.orchestra.run.smithi043.stdout: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:06.672 INFO:teuthology.orchestra.run.smithi043.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T08:53:06.672 INFO:teuthology.orchestra.run.smithi043.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T08:53:06.673 INFO:teuthology.orchestra.run.smithi043.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T08:53:06.673 INFO:teuthology.orchestra.run.smithi043.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T08:53:06.673 INFO:teuthology.orchestra.run.smithi043.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T08:53:06.673 INFO:teuthology.orchestra.run.smithi043.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T08:53:06.673 INFO:teuthology.orchestra.run.smithi043.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T08:53:06.673 INFO:teuthology.orchestra.run.smithi043.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T08:53:06.673 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T08:53:06.673 INFO:teuthology.orchestra.run.smithi043.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T08:53:06.673 INFO:teuthology.orchestra.run.smithi043.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T08:53:06.673 INFO:teuthology.orchestra.run.smithi043.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T08:53:06.673 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T08:53:06.673 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T08:53:06.673 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T08:53:06.673 INFO:teuthology.orchestra.run.smithi043.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T08:53:06.674 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T08:53:06.674 INFO:teuthology.orchestra.run.smithi043.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T08:53:06.674 INFO:teuthology.orchestra.run.smithi043.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T08:53:06.674 INFO:teuthology.orchestra.run.smithi043.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T08:53:06.674 INFO:teuthology.orchestra.run.smithi043.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T08:53:06.674 INFO:teuthology.orchestra.run.smithi043.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T08:53:06.674 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T08:53:06.674 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:06.674 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:06.674 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:06.674 INFO:teuthology.orchestra.run.smithi043.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T08:53:06.674 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T08:53:06.674 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T08:53:06.674 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T08:53:06.675 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T08:53:06.675 INFO:teuthology.orchestra.run.smithi043.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T08:53:06.675 INFO:teuthology.orchestra.run.smithi043.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T08:53:06.675 INFO:teuthology.orchestra.run.smithi043.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T08:53:06.675 INFO:teuthology.orchestra.run.smithi043.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T08:53:06.675 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T08:53:06.675 INFO:teuthology.orchestra.run.smithi043.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T08:53:06.675 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T08:53:06.675 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T08:53:06.675 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T08:53:06.675 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T08:53:06.675 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T08:53:06.675 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T08:53:06.675 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T08:53:06.675 INFO:teuthology.orchestra.run.smithi043.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T08:53:06.676 INFO:teuthology.orchestra.run.smithi043.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T08:53:06.676 INFO:teuthology.orchestra.run.smithi043.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T08:53:06.676 INFO:teuthology.orchestra.run.smithi043.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T08:53:06.676 INFO:teuthology.orchestra.run.smithi043.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T08:53:06.676 INFO:teuthology.orchestra.run.smithi043.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T08:53:06.676 INFO:teuthology.orchestra.run.smithi043.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T08:53:06.676 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T08:53:06.676 INFO:teuthology.orchestra.run.smithi043.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T08:53:06.676 INFO:teuthology.orchestra.run.smithi043.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T08:53:06.676 INFO:teuthology.orchestra.run.smithi043.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T08:53:06.676 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T08:53:06.676 INFO:teuthology.orchestra.run.smithi043.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T08:53:06.676 INFO:teuthology.orchestra.run.smithi043.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T08:53:06.676 INFO:teuthology.orchestra.run.smithi043.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T08:53:06.676 INFO:teuthology.orchestra.run.smithi043.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T08:53:06.677 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T08:53:06.677 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T08:53:06.677 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T08:53:06.677 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T08:53:06.677 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T08:53:06.677 INFO:teuthology.orchestra.run.smithi043.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T08:53:06.677 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:06.677 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:06.677 INFO:teuthology.orchestra.run.smithi043.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T08:53:06.677 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T08:53:06.677 INFO:teuthology.orchestra.run.smithi043.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T08:53:06.677 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:06.677 INFO:teuthology.orchestra.run.smithi043.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T08:53:06.677 INFO:teuthology.orchestra.run.smithi043.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T08:53:06.677 INFO:teuthology.orchestra.run.smithi043.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T08:53:06.678 INFO:teuthology.orchestra.run.smithi043.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T08:53:06.678 INFO:teuthology.orchestra.run.smithi043.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T08:53:06.678 INFO:teuthology.orchestra.run.smithi043.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T08:53:06.678 INFO:teuthology.orchestra.run.smithi043.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T08:53:06.678 INFO:teuthology.orchestra.run.smithi043.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T08:53:06.678 INFO:teuthology.orchestra.run.smithi043.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T08:53:06.678 INFO:teuthology.orchestra.run.smithi043.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T08:53:06.678 INFO:teuthology.orchestra.run.smithi043.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T08:53:06.678 INFO:teuthology.orchestra.run.smithi043.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T08:53:06.678 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T08:53:06.678 INFO:teuthology.orchestra.run.smithi043.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T08:53:06.678 INFO:teuthology.orchestra.run.smithi043.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T08:53:06.678 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:06.678 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:06.679 INFO:teuthology.orchestra.run.smithi043.stdout: rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:06.679 INFO:teuthology.orchestra.run.smithi043.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T08:53:06.679 INFO:teuthology.orchestra.run.smithi043.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T08:53:06.679 INFO:teuthology.orchestra.run.smithi043.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T08:53:06.679 INFO:teuthology.orchestra.run.smithi043.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T08:53:06.679 INFO:teuthology.orchestra.run.smithi043.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T08:53:06.679 INFO:teuthology.orchestra.run.smithi043.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T08:53:06.679 INFO:teuthology.orchestra.run.smithi043.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T08:53:06.679 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:53:06.679 INFO:teuthology.orchestra.run.smithi043.stdout:Complete! 2024-04-25T08:53:07.106 DEBUG:teuthology.parallel:result is None 2024-04-25T08:53:09.378 INFO:teuthology.orchestra.run.smithi193.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T08:53:09.378 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 1/156 2024-04-25T08:53:09.378 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2/156 2024-04-25T08:53:09.378 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 3/156 2024-04-25T08:53:09.378 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 4/156 2024-04-25T08:53:09.378 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 5/156 2024-04-25T08:53:09.379 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 6/156 2024-04-25T08:53:09.379 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 7/156 2024-04-25T08:53:09.379 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T08:53:09.379 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T08:53:09.379 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T08:53:09.379 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 11/156 2024-04-25T08:53:09.379 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 12/156 2024-04-25T08:53:09.379 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T08:53:09.379 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 14/156 2024-04-25T08:53:09.379 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 15/156 2024-04-25T08:53:09.379 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 16/156 2024-04-25T08:53:09.379 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 17/156 2024-04-25T08:53:09.379 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 18/156 2024-04-25T08:53:09.379 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 19/156 2024-04-25T08:53:09.380 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 20/156 2024-04-25T08:53:09.380 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 21/156 2024-04-25T08:53:09.380 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 22/156 2024-04-25T08:53:09.380 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 23/156 2024-04-25T08:53:09.380 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 24/156 2024-04-25T08:53:09.380 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 25/156 2024-04-25T08:53:09.380 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 26/156 2024-04-25T08:53:09.380 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 27/156 2024-04-25T08:53:09.380 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 28/156 2024-04-25T08:53:09.380 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 29/156 2024-04-25T08:53:09.380 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 30/156 2024-04-25T08:53:09.380 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 31/156 2024-04-25T08:53:09.380 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 32/156 2024-04-25T08:53:09.380 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 33/156 2024-04-25T08:53:09.381 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 34/156 2024-04-25T08:53:09.381 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 35/156 2024-04-25T08:53:09.381 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 36/156 2024-04-25T08:53:09.381 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 37/156 2024-04-25T08:53:09.381 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 38/156 2024-04-25T08:53:09.381 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 39/156 2024-04-25T08:53:09.381 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T08:53:09.381 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T08:53:09.381 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T08:53:09.381 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T08:53:09.381 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T08:53:09.381 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T08:53:09.381 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T08:53:09.382 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T08:53:09.382 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T08:53:09.382 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T08:53:09.382 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T08:53:09.382 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T08:53:09.382 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T08:53:09.382 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T08:53:09.382 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T08:53:09.382 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T08:53:09.382 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T08:53:09.382 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T08:53:09.382 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T08:53:09.382 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T08:53:09.383 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T08:53:09.383 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T08:53:09.383 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T08:53:09.383 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T08:53:09.383 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T08:53:09.383 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T08:53:09.383 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T08:53:09.383 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T08:53:09.383 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T08:53:09.383 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T08:53:09.383 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T08:53:09.383 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T08:53:09.383 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T08:53:09.384 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T08:53:09.384 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T08:53:09.384 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T08:53:09.384 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T08:53:09.384 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T08:53:09.384 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T08:53:09.384 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T08:53:09.384 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T08:53:09.384 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T08:53:09.384 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T08:53:09.384 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T08:53:09.384 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T08:53:09.384 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T08:53:09.384 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T08:53:09.384 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T08:53:09.385 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T08:53:09.385 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T08:53:09.385 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T08:53:09.385 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T08:53:09.385 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T08:53:09.385 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T08:53:09.385 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T08:53:09.385 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T08:53:09.385 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T08:53:09.385 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T08:53:09.385 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T08:53:09.385 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T08:53:09.386 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T08:53:09.386 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T08:53:09.386 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T08:53:09.386 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T08:53:09.386 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T08:53:09.386 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T08:53:09.386 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T08:53:09.386 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T08:53:09.386 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T08:53:09.386 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T08:53:09.386 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T08:53:09.386 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T08:53:09.386 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T08:53:09.387 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T08:53:09.387 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T08:53:09.387 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T08:53:09.387 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T08:53:09.387 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T08:53:09.387 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T08:53:09.387 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T08:53:09.387 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T08:53:09.387 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T08:53:09.387 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T08:53:09.387 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T08:53:09.387 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T08:53:09.387 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T08:53:09.387 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T08:53:09.387 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T08:53:09.388 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T08:53:09.388 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T08:53:09.388 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T08:53:09.388 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T08:53:09.388 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T08:53:09.388 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T08:53:09.388 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T08:53:09.388 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T08:53:09.388 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T08:53:09.388 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T08:53:09.388 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T08:53:09.388 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T08:53:09.388 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T08:53:09.388 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T08:53:09.388 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T08:53:09.389 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T08:53:09.389 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T08:53:09.389 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T08:53:09.389 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T08:53:09.389 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T08:53:09.389 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T08:53:09.389 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 149/156 2024-04-25T08:53:09.389 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T08:53:09.389 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 151/156 2024-04-25T08:53:09.389 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T08:53:09.389 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T08:53:09.389 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T08:53:09.389 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T08:53:11.511 INFO:teuthology.orchestra.run.smithi193.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T08:53:11.511 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:53:11.511 INFO:teuthology.orchestra.run.smithi193.stdout:Upgraded: 2024-04-25T08:53:11.511 INFO:teuthology.orchestra.run.smithi193.stdout: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:11.511 INFO:teuthology.orchestra.run.smithi193.stdout: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:11.511 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T08:53:11.511 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T08:53:11.511 INFO:teuthology.orchestra.run.smithi193.stdout:Installed: 2024-04-25T08:53:11.511 INFO:teuthology.orchestra.run.smithi193.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T08:53:11.511 INFO:teuthology.orchestra.run.smithi193.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T08:53:11.511 INFO:teuthology.orchestra.run.smithi193.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T08:53:11.512 INFO:teuthology.orchestra.run.smithi193.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T08:53:11.512 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:11.512 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:11.512 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:11.512 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:11.512 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:53:11.512 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:11.512 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:11.512 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:11.512 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:53:11.512 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:53:11.512 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:53:11.512 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:53:11.512 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:53:11.513 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:11.513 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:11.513 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:53:11.513 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:11.513 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:11.513 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:11.513 INFO:teuthology.orchestra.run.smithi193.stdout: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:53:11.513 INFO:teuthology.orchestra.run.smithi193.stdout: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:53:11.513 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:11.513 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:11.513 INFO:teuthology.orchestra.run.smithi193.stdout: cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T08:53:11.513 INFO:teuthology.orchestra.run.smithi193.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T08:53:11.513 INFO:teuthology.orchestra.run.smithi193.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T08:53:11.514 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T08:53:11.514 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T08:53:11.514 INFO:teuthology.orchestra.run.smithi193.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T08:53:11.514 INFO:teuthology.orchestra.run.smithi193.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T08:53:11.514 INFO:teuthology.orchestra.run.smithi193.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T08:53:11.514 INFO:teuthology.orchestra.run.smithi193.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T08:53:11.514 INFO:teuthology.orchestra.run.smithi193.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T08:53:11.514 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T08:53:11.514 INFO:teuthology.orchestra.run.smithi193.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T08:53:11.514 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:11.514 INFO:teuthology.orchestra.run.smithi193.stdout: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:11.514 INFO:teuthology.orchestra.run.smithi193.stdout: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:11.514 INFO:teuthology.orchestra.run.smithi193.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T08:53:11.515 INFO:teuthology.orchestra.run.smithi193.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T08:53:11.515 INFO:teuthology.orchestra.run.smithi193.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T08:53:11.515 INFO:teuthology.orchestra.run.smithi193.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T08:53:11.515 INFO:teuthology.orchestra.run.smithi193.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T08:53:11.515 INFO:teuthology.orchestra.run.smithi193.stdout: librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:11.515 INFO:teuthology.orchestra.run.smithi193.stdout: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:11.515 INFO:teuthology.orchestra.run.smithi193.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T08:53:11.515 INFO:teuthology.orchestra.run.smithi193.stdout: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:11.516 INFO:teuthology.orchestra.run.smithi193.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T08:53:11.516 INFO:teuthology.orchestra.run.smithi193.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T08:53:11.516 INFO:teuthology.orchestra.run.smithi193.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T08:53:11.516 INFO:teuthology.orchestra.run.smithi193.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T08:53:11.516 INFO:teuthology.orchestra.run.smithi193.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T08:53:11.516 INFO:teuthology.orchestra.run.smithi193.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T08:53:11.516 INFO:teuthology.orchestra.run.smithi193.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T08:53:11.516 INFO:teuthology.orchestra.run.smithi193.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T08:53:11.516 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T08:53:11.516 INFO:teuthology.orchestra.run.smithi193.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T08:53:11.516 INFO:teuthology.orchestra.run.smithi193.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T08:53:11.516 INFO:teuthology.orchestra.run.smithi193.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T08:53:11.516 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T08:53:11.516 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T08:53:11.517 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T08:53:11.517 INFO:teuthology.orchestra.run.smithi193.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T08:53:11.517 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T08:53:11.517 INFO:teuthology.orchestra.run.smithi193.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T08:53:11.517 INFO:teuthology.orchestra.run.smithi193.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T08:53:11.517 INFO:teuthology.orchestra.run.smithi193.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T08:53:11.517 INFO:teuthology.orchestra.run.smithi193.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T08:53:11.517 INFO:teuthology.orchestra.run.smithi193.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T08:53:11.517 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T08:53:11.517 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:11.517 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:11.517 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:11.517 INFO:teuthology.orchestra.run.smithi193.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T08:53:11.518 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T08:53:11.518 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T08:53:11.518 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T08:53:11.518 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T08:53:11.518 INFO:teuthology.orchestra.run.smithi193.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T08:53:11.518 INFO:teuthology.orchestra.run.smithi193.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T08:53:11.518 INFO:teuthology.orchestra.run.smithi193.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T08:53:11.518 INFO:teuthology.orchestra.run.smithi193.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T08:53:11.518 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T08:53:11.518 INFO:teuthology.orchestra.run.smithi193.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T08:53:11.518 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T08:53:11.518 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T08:53:11.518 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T08:53:11.519 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T08:53:11.519 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T08:53:11.519 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T08:53:11.519 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T08:53:11.519 INFO:teuthology.orchestra.run.smithi193.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T08:53:11.519 INFO:teuthology.orchestra.run.smithi193.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T08:53:11.519 INFO:teuthology.orchestra.run.smithi193.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T08:53:11.519 INFO:teuthology.orchestra.run.smithi193.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T08:53:11.519 INFO:teuthology.orchestra.run.smithi193.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T08:53:11.519 INFO:teuthology.orchestra.run.smithi193.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T08:53:11.519 INFO:teuthology.orchestra.run.smithi193.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T08:53:11.519 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T08:53:11.519 INFO:teuthology.orchestra.run.smithi193.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T08:53:11.519 INFO:teuthology.orchestra.run.smithi193.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T08:53:11.520 INFO:teuthology.orchestra.run.smithi193.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T08:53:11.520 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T08:53:11.520 INFO:teuthology.orchestra.run.smithi193.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T08:53:11.520 INFO:teuthology.orchestra.run.smithi193.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T08:53:11.520 INFO:teuthology.orchestra.run.smithi193.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T08:53:11.520 INFO:teuthology.orchestra.run.smithi193.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T08:53:11.520 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T08:53:11.520 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T08:53:11.520 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T08:53:11.520 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T08:53:11.520 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T08:53:11.520 INFO:teuthology.orchestra.run.smithi193.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T08:53:11.520 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:11.520 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:11.521 INFO:teuthology.orchestra.run.smithi193.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T08:53:11.521 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T08:53:11.521 INFO:teuthology.orchestra.run.smithi193.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T08:53:11.521 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:11.521 INFO:teuthology.orchestra.run.smithi193.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T08:53:11.521 INFO:teuthology.orchestra.run.smithi193.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T08:53:11.521 INFO:teuthology.orchestra.run.smithi193.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T08:53:11.521 INFO:teuthology.orchestra.run.smithi193.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T08:53:11.521 INFO:teuthology.orchestra.run.smithi193.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T08:53:11.521 INFO:teuthology.orchestra.run.smithi193.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T08:53:11.521 INFO:teuthology.orchestra.run.smithi193.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T08:53:11.521 INFO:teuthology.orchestra.run.smithi193.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T08:53:11.521 INFO:teuthology.orchestra.run.smithi193.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T08:53:11.521 INFO:teuthology.orchestra.run.smithi193.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T08:53:11.522 INFO:teuthology.orchestra.run.smithi193.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T08:53:11.522 INFO:teuthology.orchestra.run.smithi193.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T08:53:11.522 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T08:53:11.522 INFO:teuthology.orchestra.run.smithi193.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T08:53:11.522 INFO:teuthology.orchestra.run.smithi193.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T08:53:11.522 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:11.522 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:11.522 INFO:teuthology.orchestra.run.smithi193.stdout: rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T08:53:11.522 INFO:teuthology.orchestra.run.smithi193.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T08:53:11.522 INFO:teuthology.orchestra.run.smithi193.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T08:53:11.522 INFO:teuthology.orchestra.run.smithi193.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T08:53:11.522 INFO:teuthology.orchestra.run.smithi193.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T08:53:11.522 INFO:teuthology.orchestra.run.smithi193.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T08:53:11.523 INFO:teuthology.orchestra.run.smithi193.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T08:53:11.523 INFO:teuthology.orchestra.run.smithi193.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T08:53:11.523 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:53:11.523 INFO:teuthology.orchestra.run.smithi193.stdout:Complete! 2024-04-25T08:53:11.925 DEBUG:teuthology.parallel:result is None 2024-04-25T08:53:11.925 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T08:53:12.091 DEBUG:teuthology.orchestra.run.smithi043:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T08:53:12.118 INFO:teuthology.orchestra.run.smithi043.stdout:19.0.0-1628.g97ae6cc9.el9 2024-04-25T08:53:12.118 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1628.g97ae6cc9.el9 2024-04-25T08:53:12.119 INFO:teuthology.task.install:The correct ceph version 19.0.0-1628.g97ae6cc9 is installed. 2024-04-25T08:53:12.120 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T08:53:12.283 DEBUG:teuthology.orchestra.run.smithi193:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T08:53:12.304 INFO:teuthology.orchestra.run.smithi193.stdout:19.0.0-1628.g97ae6cc9.el9 2024-04-25T08:53:12.304 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1628.g97ae6cc9.el9 2024-04-25T08:53:12.304 INFO:teuthology.task.install:The correct ceph version 19.0.0-1628.g97ae6cc9 is installed. 2024-04-25T08:53:12.306 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T08:53:12.306 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:53:12.306 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T08:53:12.338 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T08:53:12.338 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T08:53:12.369 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T08:53:12.369 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:53:12.369 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T08:53:12.406 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T08:53:12.474 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T08:53:12.474 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T08:53:12.497 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T08:53:12.562 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T08:53:12.563 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:53:12.563 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T08:53:12.589 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T08:53:12.657 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T08:53:12.657 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T08:53:12.680 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T08:53:12.745 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T08:53:12.745 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:53:12.746 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T08:53:12.772 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T08:53:12.840 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T08:53:12.840 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T08:53:12.864 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T08:53:12.928 INFO:teuthology.run_tasks:Running task ceph... 2024-04-25T08:53:13.013 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-25T08:53:13.013 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 777 /var/log/ceph 2024-04-25T08:53:13.018 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod 777 /var/log/ceph 2024-04-25T08:53:13.042 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-25T08:53:13.042 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T08:53:13.085 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T08:53:13.110 INFO:tasks.ceph:Creating extra log directories... 2024-04-25T08:53:13.110 DEBUG:teuthology.orchestra.run.smithi043:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T08:53:13.154 DEBUG:teuthology.orchestra.run.smithi193:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T08:53:13.186 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-25T08:53:13.186 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-25T08:53:13.186 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-25T08:53:13.186 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:53:13.186 DEBUG:teuthology.orchestra.run.smithi043:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T08:53:13.238 DEBUG:teuthology.orchestra.run.smithi043:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T08:53:13.351 DEBUG:teuthology.orchestra.run.smithi043:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T08:53:13.378 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T08:53:13.378 DEBUG:teuthology.orchestra.run.smithi193:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T08:53:13.393 DEBUG:teuthology.orchestra.run.smithi193:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T08:53:13.499 DEBUG:teuthology.orchestra.run.smithi193:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T08:53:13.545 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-25T08:53:13.545 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-25T08:53:13.545 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673004', 'branch': 'wip-neesingh-testing-240416', 'description': 'fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/scrub}', 'email': 'neesingh@redhat.com', 'first_in_suite': False, 'job_id': '7673004', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-neesingh-testing-240416'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-62537', 'sha1': '5787d42d10024ba410e822a8aba474331cb9af02'}}, 'owner': 'scheduled_neesingh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 5869, 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'sleep_before_teardown': 0, 'suite': 'fs:functional', 'suite_branch': 'wip-62537', 'suite_path': '/home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/neesingh-rh/ceph', 'suite_sha1': '5787d42d10024ba410e822a8aba474331cb9af02', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi043.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAlXJxRzvrFjsp6Th37vUptyCRLd/bRZfLL7KIsn9/jD51iOJAApGr4gLlNVDsheg40xYpAol3+nyQCi2TiVVpk=', 'smithi193.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPGxWHh2OQKftzsvA8ZjW49N0cBd2B0i2ghi1EO18C1qyJQn5z1JrwJ7tqcpAbfHh2zLyZr2NhN8MBszz3zzeIc='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_scrub_checks', 'tasks.cephfs.test_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-25_08:15:18', 'tube': 'smithi', 'user': 'neesingh', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-25T08:53:13.546 DEBUG:teuthology.orchestra.run.smithi043:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T08:53:13.548 DEBUG:teuthology.orchestra.run.smithi193:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T08:53:13.595 DEBUG:teuthology.orchestra.run.smithi043:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T08:53:13.604 DEBUG:teuthology.orchestra.run.smithi193:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T08:53:13.660 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:53:13.661 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T08:53:13.686 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T08:53:13.686 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_1 2024-04-25T08:53:13.742 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T08:53:13.742 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:53:13.742 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-25T08:53:13.742 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:53:13.742 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T08:53:13.743 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-04-25 08:52:59.937537415 +0000 2024-04-25T08:53:13.743 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-04-25 08:51:13.705078671 +0000 2024-04-25T08:53:13.743 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-04-25 08:51:13.705078671 +0000 2024-04-25T08:53:13.743 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-04-25 08:51:13.705078671 +0000 2024-04-25T08:53:13.743 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T08:53:13.807 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T08:53:13.807 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T08:53:13.807 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 9.7893e-05 s, 5.2 MB/s 2024-04-25T08:53:13.808 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T08:53:13.865 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_2 2024-04-25T08:53:13.921 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T08:53:13.921 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:53:13.921 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-04-25T08:53:13.921 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:53:13.921 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T08:53:13.921 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-04-25 08:52:59.938537403 +0000 2024-04-25T08:53:13.921 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-04-25 08:51:14.043074607 +0000 2024-04-25T08:53:13.921 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-04-25 08:51:14.043074607 +0000 2024-04-25T08:53:13.921 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-04-25 08:51:14.043074607 +0000 2024-04-25T08:53:13.922 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T08:53:13.984 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T08:53:13.984 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T08:53:13.984 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000102823 s, 5.0 MB/s 2024-04-25T08:53:13.985 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T08:53:14.042 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_3 2024-04-25T08:53:14.098 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T08:53:14.098 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:53:14.098 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-25T08:53:14.098 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:53:14.099 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T08:53:14.099 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-04-25 08:52:59.938537403 +0000 2024-04-25T08:53:14.099 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-04-25 08:51:14.393070399 +0000 2024-04-25T08:53:14.099 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-04-25 08:51:14.393070399 +0000 2024-04-25T08:53:14.099 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-04-25 08:51:14.393070399 +0000 2024-04-25T08:53:14.099 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T08:53:14.162 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T08:53:14.162 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T08:53:14.162 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 8.6484e-05 s, 5.9 MB/s 2024-04-25T08:53:14.163 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T08:53:14.220 DEBUG:teuthology.orchestra.run.smithi043:> stat /dev/vg_nvme/lv_4 2024-04-25T08:53:14.275 INFO:teuthology.orchestra.run.smithi043.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T08:53:14.275 INFO:teuthology.orchestra.run.smithi043.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:53:14.275 INFO:teuthology.orchestra.run.smithi043.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-04-25T08:53:14.275 INFO:teuthology.orchestra.run.smithi043.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:53:14.276 INFO:teuthology.orchestra.run.smithi043.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T08:53:14.276 INFO:teuthology.orchestra.run.smithi043.stdout:Access: 2024-04-25 08:52:59.938537403 +0000 2024-04-25T08:53:14.276 INFO:teuthology.orchestra.run.smithi043.stdout:Modify: 2024-04-25 08:51:14.721066455 +0000 2024-04-25T08:53:14.276 INFO:teuthology.orchestra.run.smithi043.stdout:Change: 2024-04-25 08:51:14.721066455 +0000 2024-04-25T08:53:14.276 INFO:teuthology.orchestra.run.smithi043.stdout: Birth: 2024-04-25 08:51:14.721066455 +0000 2024-04-25T08:53:14.276 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T08:53:14.339 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T08:53:14.339 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T08:53:14.339 INFO:teuthology.orchestra.run.smithi043.stderr:512 bytes copied, 0.000103663 s, 4.9 MB/s 2024-04-25T08:53:14.340 DEBUG:teuthology.orchestra.run.smithi043:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T08:53:14.396 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T08:53:14.396 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T08:53:14.396 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T08:53:14.411 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T08:53:14.411 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_1 2024-04-25T08:53:14.467 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T08:53:14.467 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:53:14.467 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-25T08:53:14.467 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:53:14.467 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T08:53:14.467 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-04-25 08:53:04.713968120 +0000 2024-04-25T08:53:14.468 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-04-25 08:51:09.593822168 +0000 2024-04-25T08:53:14.468 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-04-25 08:51:09.593822168 +0000 2024-04-25T08:53:14.468 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-04-25 08:51:09.593822168 +0000 2024-04-25T08:53:14.468 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T08:53:14.530 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-04-25T08:53:14.530 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-04-25T08:53:14.530 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000206006 s, 2.5 MB/s 2024-04-25T08:53:14.531 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T08:53:14.588 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_2 2024-04-25T08:53:14.643 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T08:53:14.643 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:53:14.643 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-25T08:53:14.643 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:53:14.643 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T08:53:14.643 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-04-25 08:53:04.713968120 +0000 2024-04-25T08:53:14.643 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-04-25 08:51:09.945816445 +0000 2024-04-25T08:53:14.643 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-04-25 08:51:09.945816445 +0000 2024-04-25T08:53:14.643 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-04-25 08:51:09.945816445 +0000 2024-04-25T08:53:14.643 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T08:53:14.705 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-04-25T08:53:14.705 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-04-25T08:53:14.705 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.00019497 s, 2.6 MB/s 2024-04-25T08:53:14.706 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T08:53:14.762 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_3 2024-04-25T08:53:14.817 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T08:53:14.818 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:53:14.818 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-25T08:53:14.818 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:53:14.818 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T08:53:14.818 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-04-25 08:53:04.713968120 +0000 2024-04-25T08:53:14.818 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-04-25 08:51:10.267811210 +0000 2024-04-25T08:53:14.818 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-04-25 08:51:10.267811210 +0000 2024-04-25T08:53:14.818 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-04-25 08:51:10.267811210 +0000 2024-04-25T08:53:14.818 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T08:53:14.879 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-04-25T08:53:14.880 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-04-25T08:53:14.880 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000207544 s, 2.5 MB/s 2024-04-25T08:53:14.881 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T08:53:14.936 DEBUG:teuthology.orchestra.run.smithi193:> stat /dev/vg_nvme/lv_4 2024-04-25T08:53:14.991 INFO:teuthology.orchestra.run.smithi193.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T08:53:14.991 INFO:teuthology.orchestra.run.smithi193.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T08:53:14.991 INFO:teuthology.orchestra.run.smithi193.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-25T08:53:14.991 INFO:teuthology.orchestra.run.smithi193.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T08:53:14.991 INFO:teuthology.orchestra.run.smithi193.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T08:53:14.991 INFO:teuthology.orchestra.run.smithi193.stdout:Access: 2024-04-25 08:53:04.714968105 +0000 2024-04-25T08:53:14.991 INFO:teuthology.orchestra.run.smithi193.stdout:Modify: 2024-04-25 08:51:10.770803032 +0000 2024-04-25T08:53:14.991 INFO:teuthology.orchestra.run.smithi193.stdout:Change: 2024-04-25 08:51:10.770803032 +0000 2024-04-25T08:53:14.991 INFO:teuthology.orchestra.run.smithi193.stdout: Birth: 2024-04-25 08:51:10.770803032 +0000 2024-04-25T08:53:14.992 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T08:53:15.052 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records in 2024-04-25T08:53:15.053 INFO:teuthology.orchestra.run.smithi193.stderr:1+0 records out 2024-04-25T08:53:15.053 INFO:teuthology.orchestra.run.smithi193.stderr:512 bytes copied, 0.000201102 s, 2.5 MB/s 2024-04-25T08:53:15.054 DEBUG:teuthology.orchestra.run.smithi193:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T08:53:15.109 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T08:53:15.109 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi043.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi193.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T08:53:15.109 INFO:tasks.ceph:Generating config... 2024-04-25T08:53:15.111 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-25T08:53:15.111 INFO:tasks.ceph:[client] debug client = 20 2024-04-25T08:53:15.111 INFO:tasks.ceph:[client] debug ms = 1 2024-04-25T08:53:15.111 INFO:tasks.ceph:[client] pid file = 2024-04-25T08:53:15.111 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-25T08:53:15.111 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-25T08:53:15.111 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-25T08:53:15.111 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-25T08:53:15.111 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-25T08:53:15.111 INFO:tasks.ceph:[mds] mds cache max size = 1000 2024-04-25T08:53:15.111 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-25T08:53:15.112 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-25T08:53:15.112 INFO:tasks.ceph:[mds] mds log max segments = 1 2024-04-25T08:53:15.112 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-25T08:53:15.112 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-25T08:53:15.112 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-25T08:53:15.112 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-25T08:53:15.112 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-25T08:53:15.112 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-25T08:53:15.112 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-25T08:53:15.112 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-25T08:53:15.112 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-25T08:53:15.112 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-25T08:53:15.112 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-25T08:53:15.113 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-25T08:53:15.113 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-25T08:53:15.113 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-25T08:53:15.113 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-25T08:53:15.113 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-25T08:53:15.113 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-25T08:53:15.113 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-25T08:53:15.113 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-25T08:53:15.113 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-25T08:53:15.113 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-25T08:53:15.113 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-25T08:53:15.113 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-25T08:53:15.114 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-25T08:53:15.114 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-25T08:53:15.114 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-25T08:53:15.114 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-25T08:53:15.114 INFO:tasks.ceph:Setting up mon.a... 2024-04-25T08:53:15.114 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-25T08:53:15.164 INFO:teuthology.orchestra.run.smithi043.stdout:creating /etc/ceph/ceph.keyring 2024-04-25T08:53:15.166 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-25T08:53:15.210 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T08:53:15.276 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.43'), ('mon.b', '172.21.15.193'), ('mon.c', '[v2:172.21.15.193:3301,v1:172.21.15.193:6790]')] 2024-04-25T08:53:15.276 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.43,172.21.15.193,[v2:172.21.15.193:3301,v1:172.21.15.193:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-25T08:53:15.277 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:53:15.277 DEBUG:teuthology.orchestra.run.smithi043:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T08:53:15.332 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.43 --add b 172.21.15.193 --addv c '[v2:172.21.15.193:3301,v1:172.21.15.193:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:53:15.417 INFO:teuthology.orchestra.run.smithi043.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:53:15.418 INFO:teuthology.orchestra.run.smithi043.stdout:monmaptool: generated fsid 6c4bd50e-aa00-4a7f-8d09-352344ef5a4b 2024-04-25T08:53:15.418 INFO:teuthology.orchestra.run.smithi043.stdout:setting min_mon_release = quincy 2024-04-25T08:53:15.418 INFO:teuthology.orchestra.run.smithi043.stdout:epoch 0 2024-04-25T08:53:15.418 INFO:teuthology.orchestra.run.smithi043.stdout:fsid 6c4bd50e-aa00-4a7f-8d09-352344ef5a4b 2024-04-25T08:53:15.418 INFO:teuthology.orchestra.run.smithi043.stdout:last_changed 2024-04-25T08:53:15.428799+0000 2024-04-25T08:53:15.418 INFO:teuthology.orchestra.run.smithi043.stdout:created 2024-04-25T08:53:15.428799+0000 2024-04-25T08:53:15.418 INFO:teuthology.orchestra.run.smithi043.stdout:min_mon_release 17 (quincy) 2024-04-25T08:53:15.418 INFO:teuthology.orchestra.run.smithi043.stdout:election_strategy: 1 2024-04-25T08:53:15.418 INFO:teuthology.orchestra.run.smithi043.stdout:0: [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] mon.a 2024-04-25T08:53:15.418 INFO:teuthology.orchestra.run.smithi043.stdout:1: [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] mon.b 2024-04-25T08:53:15.418 INFO:teuthology.orchestra.run.smithi043.stdout:2: [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] mon.c 2024-04-25T08:53:15.418 INFO:teuthology.orchestra.run.smithi043.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-25T08:53:15.420 DEBUG:teuthology.orchestra.run.smithi043:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T08:53:15.434 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 6c4bd50e-aa00-4a7f-8d09-352344ef5a4b... 2024-04-25T08:53:15.435 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T08:53:15.477 DEBUG:teuthology.orchestra.run.smithi193:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T08:53:15.517 INFO:teuthology.orchestra.run.smithi043.stdout:[global] 2024-04-25T08:53:15.517 INFO:teuthology.orchestra.run.smithi043.stdout: chdir = "" 2024-04-25T08:53:15.517 INFO:teuthology.orchestra.run.smithi043.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T08:53:15.517 INFO:teuthology.orchestra.run.smithi043.stdout: auth supported = cephx 2024-04-25T08:53:15.517 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:53:15.517 INFO:teuthology.orchestra.run.smithi043.stdout: filestore xattr use omap = true 2024-04-25T08:53:15.517 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:53:15.517 INFO:teuthology.orchestra.run.smithi043.stdout: mon clock drift allowed = 1.000 2024-04-25T08:53:15.517 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:53:15.518 INFO:teuthology.orchestra.run.smithi043.stdout: osd crush chooseleaf type = 0 2024-04-25T08:53:15.518 INFO:teuthology.orchestra.run.smithi043.stdout: auth debug = true 2024-04-25T08:53:15.518 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:53:15.518 INFO:teuthology.orchestra.run.smithi043.stdout: ms die on old message = true 2024-04-25T08:53:15.518 INFO:teuthology.orchestra.run.smithi043.stdout: ms die on bug = true 2024-04-25T08:53:15.518 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:53:15.518 INFO:teuthology.orchestra.run.smithi043.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T08:53:15.518 INFO:teuthology.orchestra.run.smithi043.stdout: mon pg warn max object skew = 0 2024-04-25T08:53:15.518 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:53:15.518 INFO:teuthology.orchestra.run.smithi043.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T08:53:15.518 INFO:teuthology.orchestra.run.smithi043.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T08:53:15.518 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:53:15.518 INFO:teuthology.orchestra.run.smithi043.stdout: osd pool default size = 2 2024-04-25T08:53:15.518 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:53:15.519 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd allow primary affinity = true 2024-04-25T08:53:15.519 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd allow pg remap = true 2024-04-25T08:53:15.519 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on legacy crush tunables = false 2024-04-25T08:53:15.519 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on crush straw calc version zero = false 2024-04-25T08:53:15.519 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on no sortbitwise = false 2024-04-25T08:53:15.519 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on osd down out interval zero = false 2024-04-25T08:53:15.519 INFO:teuthology.orchestra.run.smithi043.stdout: mon warn on too few osds = false 2024-04-25T08:53:15.519 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T08:53:15.519 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T08:53:15.519 INFO:teuthology.orchestra.run.smithi043.stdout: mon_allow_pool_size_one = true 2024-04-25T08:53:15.519 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:53:15.519 INFO:teuthology.orchestra.run.smithi043.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T08:53:15.520 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:53:15.520 INFO:teuthology.orchestra.run.smithi043.stdout: osd default data pool replay window = 5 2024-04-25T08:53:15.520 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:53:15.520 INFO:teuthology.orchestra.run.smithi043.stdout: mon allow pool delete = true 2024-04-25T08:53:15.520 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:53:15.520 INFO:teuthology.orchestra.run.smithi043.stdout: mon cluster log file level = debug 2024-04-25T08:53:15.520 INFO:teuthology.orchestra.run.smithi043.stdout: debug asserts on shutdown = true 2024-04-25T08:53:15.520 INFO:teuthology.orchestra.run.smithi043.stdout: mon health detail to clog = false 2024-04-25T08:53:15.521 INFO:teuthology.orchestra.run.smithi043.stdout: mon host = "172.21.15.43,172.21.15.193,[v2:172.21.15.193:3301,v1:172.21.15.193:6790]" 2024-04-25T08:53:15.521 INFO:teuthology.orchestra.run.smithi043.stdout: fsid = 6c4bd50e-aa00-4a7f-8d09-352344ef5a4b 2024-04-25T08:53:15.521 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:53:15.521 INFO:teuthology.orchestra.run.smithi043.stdout:[osd] 2024-04-25T08:53:15.521 INFO:teuthology.orchestra.run.smithi043.stdout: osd journal size = 100 2024-04-25T08:53:15.522 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:53:15.522 INFO:teuthology.orchestra.run.smithi043.stdout: osd scrub load threshold = 5.0 2024-04-25T08:53:15.522 INFO:teuthology.orchestra.run.smithi043.stdout: osd scrub max interval = 600 2024-04-25T08:53:15.522 INFO:teuthology.orchestra.run.smithi043.stdout: osd mclock profile = high_recovery_ops 2024-04-25T08:53:15.522 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:53:15.522 INFO:teuthology.orchestra.run.smithi043.stdout: osd recover clone overlap = true 2024-04-25T08:53:15.523 INFO:teuthology.orchestra.run.smithi043.stdout: osd recovery max chunk = 1048576 2024-04-25T08:53:15.523 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:53:15.523 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug shutdown = true 2024-04-25T08:53:15.523 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug op order = true 2024-04-25T08:53:15.523 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug verify stray on activate = true 2024-04-25T08:53:15.523 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug trim objects = true 2024-04-25T08:53:15.523 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:53:15.523 INFO:teuthology.orchestra.run.smithi043.stdout: osd open classes on start = true 2024-04-25T08:53:15.523 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug pg log writeout = true 2024-04-25T08:53:15.523 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:53:15.523 INFO:teuthology.orchestra.run.smithi043.stdout: osd deep scrub update digest min age = 30 2024-04-25T08:53:15.523 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:53:15.523 INFO:teuthology.orchestra.run.smithi043.stdout: osd map max advance = 10 2024-04-25T08:53:15.523 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:53:15.523 INFO:teuthology.orchestra.run.smithi043.stdout: journal zero on create = true 2024-04-25T08:53:15.524 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:53:15.524 INFO:teuthology.orchestra.run.smithi043.stdout: filestore ondisk finisher threads = 3 2024-04-25T08:53:15.524 INFO:teuthology.orchestra.run.smithi043.stdout: filestore apply finisher threads = 3 2024-04-25T08:53:15.524 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:53:15.524 INFO:teuthology.orchestra.run.smithi043.stdout: bdev debug aio = true 2024-04-25T08:53:15.524 INFO:teuthology.orchestra.run.smithi043.stdout: osd debug misdirected ops = true 2024-04-25T08:53:15.524 INFO:teuthology.orchestra.run.smithi043.stdout: bdev async discard = True 2024-04-25T08:53:15.524 INFO:teuthology.orchestra.run.smithi043.stdout: bdev enable discard = True 2024-04-25T08:53:15.524 INFO:teuthology.orchestra.run.smithi043.stdout: bluestore allocator = bitmap 2024-04-25T08:53:15.524 INFO:teuthology.orchestra.run.smithi043.stdout: bluestore block size = 96636764160 2024-04-25T08:53:15.524 INFO:teuthology.orchestra.run.smithi043.stdout: bluestore fsck on mount = True 2024-04-25T08:53:15.524 INFO:teuthology.orchestra.run.smithi043.stdout: debug bluefs = 1/20 2024-04-25T08:53:15.524 INFO:teuthology.orchestra.run.smithi043.stdout: debug bluestore = 1/20 2024-04-25T08:53:15.524 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2024-04-25T08:53:15.524 INFO:teuthology.orchestra.run.smithi043.stdout: debug osd = 20 2024-04-25T08:53:15.525 INFO:teuthology.orchestra.run.smithi043.stdout: debug rocksdb = 4/10 2024-04-25T08:53:15.525 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T08:53:15.525 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd full ratio = 0.9 2024-04-25T08:53:15.525 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd nearfull ratio = 0.8 2024-04-25T08:53:15.525 INFO:teuthology.orchestra.run.smithi043.stdout: osd failsafe full ratio = 0.95 2024-04-25T08:53:15.525 INFO:teuthology.orchestra.run.smithi043.stdout: osd objectstore = bluestore 2024-04-25T08:53:15.525 INFO:teuthology.orchestra.run.smithi043.stdout: osd op complaint time = 180 2024-04-25T08:53:15.525 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:53:15.525 INFO:teuthology.orchestra.run.smithi043.stdout:[mgr] 2024-04-25T08:53:15.525 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2024-04-25T08:53:15.525 INFO:teuthology.orchestra.run.smithi043.stdout: debug mgr = 20 2024-04-25T08:53:15.525 INFO:teuthology.orchestra.run.smithi043.stdout: debug mon = 20 2024-04-25T08:53:15.525 INFO:teuthology.orchestra.run.smithi043.stdout: debug auth = 20 2024-04-25T08:53:15.525 INFO:teuthology.orchestra.run.smithi043.stdout: mon reweight min pgs per osd = 4 2024-04-25T08:53:15.525 INFO:teuthology.orchestra.run.smithi043.stdout: mon reweight min bytes per osd = 10 2024-04-25T08:53:15.525 INFO:teuthology.orchestra.run.smithi043.stdout: mgr/telemetry/nag = false 2024-04-25T08:53:15.526 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:53:15.526 INFO:teuthology.orchestra.run.smithi043.stdout:[mon] 2024-04-25T08:53:15.526 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2024-04-25T08:53:15.526 INFO:teuthology.orchestra.run.smithi043.stdout: debug mon = 20 2024-04-25T08:53:15.526 INFO:teuthology.orchestra.run.smithi043.stdout: debug paxos = 20 2024-04-25T08:53:15.526 INFO:teuthology.orchestra.run.smithi043.stdout: debug auth = 20 2024-04-25T08:53:15.526 INFO:teuthology.orchestra.run.smithi043.stdout: mon data avail warn = 5 2024-04-25T08:53:15.526 INFO:teuthology.orchestra.run.smithi043.stdout: mon mgr mkfs grace = 240 2024-04-25T08:53:15.526 INFO:teuthology.orchestra.run.smithi043.stdout: mon reweight min pgs per osd = 4 2024-04-25T08:53:15.526 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd reporter subtree level = osd 2024-04-25T08:53:15.526 INFO:teuthology.orchestra.run.smithi043.stdout: mon osd prime pg temp = true 2024-04-25T08:53:15.526 INFO:teuthology.orchestra.run.smithi043.stdout: mon reweight min bytes per osd = 10 2024-04-25T08:53:15.526 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:53:15.527 INFO:teuthology.orchestra.run.smithi043.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T08:53:15.527 INFO:teuthology.orchestra.run.smithi043.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T08:53:15.527 INFO:teuthology.orchestra.run.smithi043.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T08:53:15.527 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:53:15.527 INFO:teuthology.orchestra.run.smithi043.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T08:53:15.527 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T08:53:15.527 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T08:53:15.527 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:53:15.527 INFO:teuthology.orchestra.run.smithi043.stdout: # 1m isn't quite enough 2024-04-25T08:53:15.527 INFO:teuthology.orchestra.run.smithi043.stdout: mon_down_mkfs_grace = 2m 2024-04-25T08:53:15.527 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:53:15.527 INFO:teuthology.orchestra.run.smithi043.stdout: mon_warn_on_filestore_osds = false 2024-04-25T08:53:15.527 INFO:teuthology.orchestra.run.smithi043.stdout: mon op complaint time = 120 2024-04-25T08:53:15.527 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:53:15.527 INFO:teuthology.orchestra.run.smithi043.stdout:[client] 2024-04-25T08:53:15.528 INFO:teuthology.orchestra.run.smithi043.stdout: rgw cache enabled = true 2024-04-25T08:53:15.528 INFO:teuthology.orchestra.run.smithi043.stdout: rgw enable ops log = true 2024-04-25T08:53:15.528 INFO:teuthology.orchestra.run.smithi043.stdout: rgw enable usage log = true 2024-04-25T08:53:15.528 INFO:teuthology.orchestra.run.smithi043.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T08:53:15.528 INFO:teuthology.orchestra.run.smithi043.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T08:53:15.528 INFO:teuthology.orchestra.run.smithi043.stdout: client mount timeout = 600 2024-04-25T08:53:15.528 INFO:teuthology.orchestra.run.smithi043.stdout: debug client = 20 2024-04-25T08:53:15.528 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2024-04-25T08:53:15.528 INFO:teuthology.orchestra.run.smithi043.stdout: pid file = "" 2024-04-25T08:53:15.528 INFO:teuthology.orchestra.run.smithi043.stdout: rados mon op timeout = 900 2024-04-25T08:53:15.528 INFO:teuthology.orchestra.run.smithi043.stdout: rados osd op timeout = 900 2024-04-25T08:53:15.528 INFO:teuthology.orchestra.run.smithi043.stdout:[mon.a] 2024-04-25T08:53:15.528 INFO:teuthology.orchestra.run.smithi043.stdout:[mon.b] 2024-04-25T08:53:15.528 INFO:teuthology.orchestra.run.smithi043.stdout:[mon.c] 2024-04-25T08:53:15.529 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.a] 2024-04-25T08:53:15.529 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.c] 2024-04-25T08:53:15.529 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.b] 2024-04-25T08:53:15.529 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.d] 2024-04-25T08:53:15.529 INFO:teuthology.orchestra.run.smithi043.stdout:[mds] 2024-04-25T08:53:15.529 INFO:teuthology.orchestra.run.smithi043.stdout: debug mds = 20 2024-04-25T08:53:15.529 INFO:teuthology.orchestra.run.smithi043.stdout: debug mds balancer = 20 2024-04-25T08:53:15.529 INFO:teuthology.orchestra.run.smithi043.stdout: debug ms = 1 2024-04-25T08:53:15.529 INFO:teuthology.orchestra.run.smithi043.stdout: mds cache max size = 1000 2024-04-25T08:53:15.529 INFO:teuthology.orchestra.run.smithi043.stdout: mds debug frag = True 2024-04-25T08:53:15.529 INFO:teuthology.orchestra.run.smithi043.stdout: mds debug scatterstat = True 2024-04-25T08:53:15.529 INFO:teuthology.orchestra.run.smithi043.stdout: mds log max segments = 1 2024-04-25T08:53:15.529 INFO:teuthology.orchestra.run.smithi043.stdout: mds op complaint time = 180 2024-04-25T08:53:15.529 INFO:teuthology.orchestra.run.smithi043.stdout: mds verify scatter = True 2024-04-25T08:53:15.529 INFO:teuthology.orchestra.run.smithi043.stdout: osd op complaint time = 180 2024-04-25T08:53:15.530 INFO:teuthology.orchestra.run.smithi043.stdout: rados mon op timeout = 900 2024-04-25T08:53:15.530 INFO:teuthology.orchestra.run.smithi043.stdout: rados osd op timeout = 900 2024-04-25T08:53:15.547 INFO:teuthology.orchestra.run.smithi193.stdout:[global] 2024-04-25T08:53:15.547 INFO:teuthology.orchestra.run.smithi193.stdout: chdir = "" 2024-04-25T08:53:15.547 INFO:teuthology.orchestra.run.smithi193.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T08:53:15.547 INFO:teuthology.orchestra.run.smithi193.stdout: auth supported = cephx 2024-04-25T08:53:15.547 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:53:15.547 INFO:teuthology.orchestra.run.smithi193.stdout: filestore xattr use omap = true 2024-04-25T08:53:15.547 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:53:15.547 INFO:teuthology.orchestra.run.smithi193.stdout: mon clock drift allowed = 1.000 2024-04-25T08:53:15.547 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:53:15.547 INFO:teuthology.orchestra.run.smithi193.stdout: osd crush chooseleaf type = 0 2024-04-25T08:53:15.547 INFO:teuthology.orchestra.run.smithi193.stdout: auth debug = true 2024-04-25T08:53:15.547 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:53:15.548 INFO:teuthology.orchestra.run.smithi193.stdout: ms die on old message = true 2024-04-25T08:53:15.548 INFO:teuthology.orchestra.run.smithi193.stdout: ms die on bug = true 2024-04-25T08:53:15.548 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:53:15.548 INFO:teuthology.orchestra.run.smithi193.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T08:53:15.548 INFO:teuthology.orchestra.run.smithi193.stdout: mon pg warn max object skew = 0 2024-04-25T08:53:15.548 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:53:15.548 INFO:teuthology.orchestra.run.smithi193.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T08:53:15.548 INFO:teuthology.orchestra.run.smithi193.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T08:53:15.548 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:53:15.548 INFO:teuthology.orchestra.run.smithi193.stdout: osd pool default size = 2 2024-04-25T08:53:15.548 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:53:15.548 INFO:teuthology.orchestra.run.smithi193.stdout: mon osd allow primary affinity = true 2024-04-25T08:53:15.548 INFO:teuthology.orchestra.run.smithi193.stdout: mon osd allow pg remap = true 2024-04-25T08:53:15.548 INFO:teuthology.orchestra.run.smithi193.stdout: mon warn on legacy crush tunables = false 2024-04-25T08:53:15.548 INFO:teuthology.orchestra.run.smithi193.stdout: mon warn on crush straw calc version zero = false 2024-04-25T08:53:15.549 INFO:teuthology.orchestra.run.smithi193.stdout: mon warn on no sortbitwise = false 2024-04-25T08:53:15.549 INFO:teuthology.orchestra.run.smithi193.stdout: mon warn on osd down out interval zero = false 2024-04-25T08:53:15.549 INFO:teuthology.orchestra.run.smithi193.stdout: mon warn on too few osds = false 2024-04-25T08:53:15.549 INFO:teuthology.orchestra.run.smithi193.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T08:53:15.549 INFO:teuthology.orchestra.run.smithi193.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T08:53:15.549 INFO:teuthology.orchestra.run.smithi193.stdout: mon_allow_pool_size_one = true 2024-04-25T08:53:15.549 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:53:15.549 INFO:teuthology.orchestra.run.smithi193.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T08:53:15.549 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:53:15.549 INFO:teuthology.orchestra.run.smithi193.stdout: osd default data pool replay window = 5 2024-04-25T08:53:15.549 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:53:15.549 INFO:teuthology.orchestra.run.smithi193.stdout: mon allow pool delete = true 2024-04-25T08:53:15.549 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:53:15.549 INFO:teuthology.orchestra.run.smithi193.stdout: mon cluster log file level = debug 2024-04-25T08:53:15.549 INFO:teuthology.orchestra.run.smithi193.stdout: debug asserts on shutdown = true 2024-04-25T08:53:15.550 INFO:teuthology.orchestra.run.smithi193.stdout: mon health detail to clog = false 2024-04-25T08:53:15.550 INFO:teuthology.orchestra.run.smithi193.stdout: mon host = "172.21.15.43,172.21.15.193,[v2:172.21.15.193:3301,v1:172.21.15.193:6790]" 2024-04-25T08:53:15.550 INFO:teuthology.orchestra.run.smithi193.stdout: fsid = 6c4bd50e-aa00-4a7f-8d09-352344ef5a4b 2024-04-25T08:53:15.550 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:53:15.550 INFO:teuthology.orchestra.run.smithi193.stdout:[osd] 2024-04-25T08:53:15.550 INFO:teuthology.orchestra.run.smithi193.stdout: osd journal size = 100 2024-04-25T08:53:15.550 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:53:15.550 INFO:teuthology.orchestra.run.smithi193.stdout: osd scrub load threshold = 5.0 2024-04-25T08:53:15.550 INFO:teuthology.orchestra.run.smithi193.stdout: osd scrub max interval = 600 2024-04-25T08:53:15.550 INFO:teuthology.orchestra.run.smithi193.stdout: osd mclock profile = high_recovery_ops 2024-04-25T08:53:15.550 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:53:15.550 INFO:teuthology.orchestra.run.smithi193.stdout: osd recover clone overlap = true 2024-04-25T08:53:15.550 INFO:teuthology.orchestra.run.smithi193.stdout: osd recovery max chunk = 1048576 2024-04-25T08:53:15.550 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:53:15.551 INFO:teuthology.orchestra.run.smithi193.stdout: osd debug shutdown = true 2024-04-25T08:53:15.551 INFO:teuthology.orchestra.run.smithi193.stdout: osd debug op order = true 2024-04-25T08:53:15.551 INFO:teuthology.orchestra.run.smithi193.stdout: osd debug verify stray on activate = true 2024-04-25T08:53:15.551 INFO:teuthology.orchestra.run.smithi193.stdout: osd debug trim objects = true 2024-04-25T08:53:15.551 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:53:15.551 INFO:teuthology.orchestra.run.smithi193.stdout: osd open classes on start = true 2024-04-25T08:53:15.551 INFO:teuthology.orchestra.run.smithi193.stdout: osd debug pg log writeout = true 2024-04-25T08:53:15.551 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:53:15.551 INFO:teuthology.orchestra.run.smithi193.stdout: osd deep scrub update digest min age = 30 2024-04-25T08:53:15.551 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:53:15.551 INFO:teuthology.orchestra.run.smithi193.stdout: osd map max advance = 10 2024-04-25T08:53:15.551 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:53:15.551 INFO:teuthology.orchestra.run.smithi193.stdout: journal zero on create = true 2024-04-25T08:53:15.551 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:53:15.551 INFO:teuthology.orchestra.run.smithi193.stdout: filestore ondisk finisher threads = 3 2024-04-25T08:53:15.552 INFO:teuthology.orchestra.run.smithi193.stdout: filestore apply finisher threads = 3 2024-04-25T08:53:15.552 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:53:15.552 INFO:teuthology.orchestra.run.smithi193.stdout: bdev debug aio = true 2024-04-25T08:53:15.552 INFO:teuthology.orchestra.run.smithi193.stdout: osd debug misdirected ops = true 2024-04-25T08:53:15.552 INFO:teuthology.orchestra.run.smithi193.stdout: bdev async discard = True 2024-04-25T08:53:15.552 INFO:teuthology.orchestra.run.smithi193.stdout: bdev enable discard = True 2024-04-25T08:53:15.552 INFO:teuthology.orchestra.run.smithi193.stdout: bluestore allocator = bitmap 2024-04-25T08:53:15.552 INFO:teuthology.orchestra.run.smithi193.stdout: bluestore block size = 96636764160 2024-04-25T08:53:15.552 INFO:teuthology.orchestra.run.smithi193.stdout: bluestore fsck on mount = True 2024-04-25T08:53:15.552 INFO:teuthology.orchestra.run.smithi193.stdout: debug bluefs = 1/20 2024-04-25T08:53:15.552 INFO:teuthology.orchestra.run.smithi193.stdout: debug bluestore = 1/20 2024-04-25T08:53:15.552 INFO:teuthology.orchestra.run.smithi193.stdout: debug ms = 1 2024-04-25T08:53:15.552 INFO:teuthology.orchestra.run.smithi193.stdout: debug osd = 20 2024-04-25T08:53:15.552 INFO:teuthology.orchestra.run.smithi193.stdout: debug rocksdb = 4/10 2024-04-25T08:53:15.552 INFO:teuthology.orchestra.run.smithi193.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T08:53:15.553 INFO:teuthology.orchestra.run.smithi193.stdout: mon osd full ratio = 0.9 2024-04-25T08:53:15.553 INFO:teuthology.orchestra.run.smithi193.stdout: mon osd nearfull ratio = 0.8 2024-04-25T08:53:15.553 INFO:teuthology.orchestra.run.smithi193.stdout: osd failsafe full ratio = 0.95 2024-04-25T08:53:15.553 INFO:teuthology.orchestra.run.smithi193.stdout: osd objectstore = bluestore 2024-04-25T08:53:15.553 INFO:teuthology.orchestra.run.smithi193.stdout: osd op complaint time = 180 2024-04-25T08:53:15.553 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:53:15.553 INFO:teuthology.orchestra.run.smithi193.stdout:[mgr] 2024-04-25T08:53:15.553 INFO:teuthology.orchestra.run.smithi193.stdout: debug ms = 1 2024-04-25T08:53:15.553 INFO:teuthology.orchestra.run.smithi193.stdout: debug mgr = 20 2024-04-25T08:53:15.553 INFO:teuthology.orchestra.run.smithi193.stdout: debug mon = 20 2024-04-25T08:53:15.553 INFO:teuthology.orchestra.run.smithi193.stdout: debug auth = 20 2024-04-25T08:53:15.553 INFO:teuthology.orchestra.run.smithi193.stdout: mon reweight min pgs per osd = 4 2024-04-25T08:53:15.553 INFO:teuthology.orchestra.run.smithi193.stdout: mon reweight min bytes per osd = 10 2024-04-25T08:53:15.553 INFO:teuthology.orchestra.run.smithi193.stdout: mgr/telemetry/nag = false 2024-04-25T08:53:15.553 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:53:15.554 INFO:teuthology.orchestra.run.smithi193.stdout:[mon] 2024-04-25T08:53:15.554 INFO:teuthology.orchestra.run.smithi193.stdout: debug ms = 1 2024-04-25T08:53:15.554 INFO:teuthology.orchestra.run.smithi193.stdout: debug mon = 20 2024-04-25T08:53:15.554 INFO:teuthology.orchestra.run.smithi193.stdout: debug paxos = 20 2024-04-25T08:53:15.554 INFO:teuthology.orchestra.run.smithi193.stdout: debug auth = 20 2024-04-25T08:53:15.554 INFO:teuthology.orchestra.run.smithi193.stdout: mon data avail warn = 5 2024-04-25T08:53:15.554 INFO:teuthology.orchestra.run.smithi193.stdout: mon mgr mkfs grace = 240 2024-04-25T08:53:15.554 INFO:teuthology.orchestra.run.smithi193.stdout: mon reweight min pgs per osd = 4 2024-04-25T08:53:15.554 INFO:teuthology.orchestra.run.smithi193.stdout: mon osd reporter subtree level = osd 2024-04-25T08:53:15.554 INFO:teuthology.orchestra.run.smithi193.stdout: mon osd prime pg temp = true 2024-04-25T08:53:15.554 INFO:teuthology.orchestra.run.smithi193.stdout: mon reweight min bytes per osd = 10 2024-04-25T08:53:15.554 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:53:15.554 INFO:teuthology.orchestra.run.smithi193.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T08:53:15.554 INFO:teuthology.orchestra.run.smithi193.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T08:53:15.554 INFO:teuthology.orchestra.run.smithi193.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T08:53:15.555 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:53:15.555 INFO:teuthology.orchestra.run.smithi193.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T08:53:15.555 INFO:teuthology.orchestra.run.smithi193.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T08:53:15.555 INFO:teuthology.orchestra.run.smithi193.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T08:53:15.555 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:53:15.555 INFO:teuthology.orchestra.run.smithi193.stdout: # 1m isn't quite enough 2024-04-25T08:53:15.555 INFO:teuthology.orchestra.run.smithi193.stdout: mon_down_mkfs_grace = 2m 2024-04-25T08:53:15.555 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:53:15.555 INFO:teuthology.orchestra.run.smithi193.stdout: mon_warn_on_filestore_osds = false 2024-04-25T08:53:15.555 INFO:teuthology.orchestra.run.smithi193.stdout: mon op complaint time = 120 2024-04-25T08:53:15.555 INFO:teuthology.orchestra.run.smithi193.stdout: 2024-04-25T08:53:15.555 INFO:teuthology.orchestra.run.smithi193.stdout:[client] 2024-04-25T08:53:15.555 INFO:teuthology.orchestra.run.smithi193.stdout: rgw cache enabled = true 2024-04-25T08:53:15.555 INFO:teuthology.orchestra.run.smithi193.stdout: rgw enable ops log = true 2024-04-25T08:53:15.555 INFO:teuthology.orchestra.run.smithi193.stdout: rgw enable usage log = true 2024-04-25T08:53:15.556 INFO:teuthology.orchestra.run.smithi193.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T08:53:15.556 INFO:teuthology.orchestra.run.smithi193.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T08:53:15.556 INFO:teuthology.orchestra.run.smithi193.stdout: client mount timeout = 600 2024-04-25T08:53:15.556 INFO:teuthology.orchestra.run.smithi193.stdout: debug client = 20 2024-04-25T08:53:15.556 INFO:teuthology.orchestra.run.smithi193.stdout: debug ms = 1 2024-04-25T08:53:15.556 INFO:teuthology.orchestra.run.smithi193.stdout: pid file = "" 2024-04-25T08:53:15.556 INFO:teuthology.orchestra.run.smithi193.stdout: rados mon op timeout = 900 2024-04-25T08:53:15.556 INFO:teuthology.orchestra.run.smithi193.stdout: rados osd op timeout = 900 2024-04-25T08:53:15.556 INFO:teuthology.orchestra.run.smithi193.stdout:[mon.a] 2024-04-25T08:53:15.556 INFO:teuthology.orchestra.run.smithi193.stdout:[mon.b] 2024-04-25T08:53:15.556 INFO:teuthology.orchestra.run.smithi193.stdout:[mon.c] 2024-04-25T08:53:15.556 INFO:teuthology.orchestra.run.smithi193.stdout:[mds.a] 2024-04-25T08:53:15.556 INFO:teuthology.orchestra.run.smithi193.stdout:[mds.c] 2024-04-25T08:53:15.556 INFO:teuthology.orchestra.run.smithi193.stdout:[mds.b] 2024-04-25T08:53:15.556 INFO:teuthology.orchestra.run.smithi193.stdout:[mds.d] 2024-04-25T08:53:15.556 INFO:teuthology.orchestra.run.smithi193.stdout:[mds] 2024-04-25T08:53:15.557 INFO:teuthology.orchestra.run.smithi193.stdout: debug mds = 20 2024-04-25T08:53:15.557 INFO:teuthology.orchestra.run.smithi193.stdout: debug mds balancer = 20 2024-04-25T08:53:15.557 INFO:teuthology.orchestra.run.smithi193.stdout: debug ms = 1 2024-04-25T08:53:15.557 INFO:teuthology.orchestra.run.smithi193.stdout: mds cache max size = 1000 2024-04-25T08:53:15.557 INFO:teuthology.orchestra.run.smithi193.stdout: mds debug frag = True 2024-04-25T08:53:15.557 INFO:teuthology.orchestra.run.smithi193.stdout: mds debug scatterstat = True 2024-04-25T08:53:15.557 INFO:teuthology.orchestra.run.smithi193.stdout: mds log max segments = 1 2024-04-25T08:53:15.557 INFO:teuthology.orchestra.run.smithi193.stdout: mds op complaint time = 180 2024-04-25T08:53:15.557 INFO:teuthology.orchestra.run.smithi193.stdout: mds verify scatter = True 2024-04-25T08:53:15.557 INFO:teuthology.orchestra.run.smithi193.stdout: osd op complaint time = 180 2024-04-25T08:53:15.557 INFO:teuthology.orchestra.run.smithi193.stdout: rados mon op timeout = 900 2024-04-25T08:53:15.557 INFO:teuthology.orchestra.run.smithi193.stdout: rados osd op timeout = 900 2024-04-25T08:53:15.558 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-25T08:53:15.558 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-25T08:53:15.624 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-25T08:53:15.625 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:53:15.625 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-25T08:53:15.640 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:53:15.640 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-25T08:53:15.696 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi043.front.sepia.ceph.com 2024-04-25T08:53:15.696 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:53:15.696 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T08:53:15.696 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T08:53:15.769 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:53:15.769 DEBUG:teuthology.orchestra.run.smithi043:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:53:15.825 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi193.front.sepia.ceph.com 2024-04-25T08:53:15.825 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T08:53:15.825 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T08:53:15.825 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T08:53:15.859 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T08:53:15.859 DEBUG:teuthology.orchestra.run.smithi193:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:53:15.914 INFO:tasks.ceph:Setting up mon nodes... 2024-04-25T08:53:15.914 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-25T08:53:15.914 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T08:53:15.964 INFO:teuthology.orchestra.run.smithi043.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T08:53:15.967 DEBUG:teuthology.orchestra.run.smithi193:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T08:53:16.033 INFO:teuthology.orchestra.run.smithi193.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T08:53:16.035 INFO:tasks.ceph:Setting up mds nodes... 2024-04-25T08:53:16.036 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T08:53:16.086 INFO:teuthology.orchestra.run.smithi043.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T08:53:16.088 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-25T08:53:16.112 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T08:53:16.201 INFO:teuthology.orchestra.run.smithi043.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T08:53:16.204 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-25T08:53:16.228 DEBUG:teuthology.orchestra.run.smithi193:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T08:53:16.276 INFO:teuthology.orchestra.run.smithi193.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T08:53:16.279 DEBUG:teuthology.orchestra.run.smithi193:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-25T08:53:16.302 DEBUG:teuthology.orchestra.run.smithi193:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T08:53:16.390 INFO:teuthology.orchestra.run.smithi193.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T08:53:16.392 DEBUG:teuthology.orchestra.run.smithi193:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-25T08:53:16.415 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-25T08:53:16.415 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T08:53:16.456 INFO:teuthology.orchestra.run.smithi043.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-25T08:53:16.468 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-25T08:53:16.548 INFO:teuthology.orchestra.run.smithi043.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-25T08:53:16.560 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-25T08:53:16.598 INFO:teuthology.orchestra.run.smithi193.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-25T08:53:16.610 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-25T08:53:16.689 INFO:teuthology.orchestra.run.smithi193.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-25T08:53:16.701 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-25T08:53:16.701 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi043.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi193.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T08:53:16.701 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-25T08:53:16.725 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T08:53:16.726 INFO:tasks.ceph:role: osd.0 2024-04-25T08:53:16.726 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi043.front.sepia.ceph.com 2024-04-25T08:53:16.726 DEBUG:teuthology.orchestra.run.smithi043:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T08:53:16.789 INFO:teuthology.orchestra.run.smithi043.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:53:16.789 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:53:16.790 INFO:teuthology.orchestra.run.smithi043.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:53:16.790 INFO:teuthology.orchestra.run.smithi043.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T08:53:16.790 INFO:teuthology.orchestra.run.smithi043.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:53:16.790 INFO:teuthology.orchestra.run.smithi043.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:53:16.790 INFO:teuthology.orchestra.run.smithi043.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:53:16.790 INFO:teuthology.orchestra.run.smithi043.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T08:53:16.790 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:53:16.790 INFO:teuthology.orchestra.run.smithi043.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:53:16.822 INFO:teuthology.orchestra.run.smithi043.stdout:Discarding blocks...Done. 2024-04-25T08:53:16.823 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi043.front.sepia.ceph.com -o noatime 2024-04-25T08:53:16.824 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-25T08:53:16.892 DEBUG:teuthology.orchestra.run.smithi043:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-25T08:53:16.961 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-25T08:53:17.026 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T08:53:17.026 INFO:tasks.ceph:role: osd.1 2024-04-25T08:53:17.026 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi043.front.sepia.ceph.com 2024-04-25T08:53:17.027 DEBUG:teuthology.orchestra.run.smithi043:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T08:53:17.091 INFO:teuthology.orchestra.run.smithi043.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:53:17.091 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:53:17.092 INFO:teuthology.orchestra.run.smithi043.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:53:17.092 INFO:teuthology.orchestra.run.smithi043.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T08:53:17.092 INFO:teuthology.orchestra.run.smithi043.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:53:17.092 INFO:teuthology.orchestra.run.smithi043.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:53:17.092 INFO:teuthology.orchestra.run.smithi043.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:53:17.092 INFO:teuthology.orchestra.run.smithi043.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T08:53:17.092 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:53:17.092 INFO:teuthology.orchestra.run.smithi043.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:53:17.124 INFO:teuthology.orchestra.run.smithi043.stdout:Discarding blocks...Done. 2024-04-25T08:53:17.126 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi043.front.sepia.ceph.com -o noatime 2024-04-25T08:53:17.126 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-25T08:53:17.194 DEBUG:teuthology.orchestra.run.smithi043:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-25T08:53:17.262 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-25T08:53:17.327 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T08:53:17.327 INFO:tasks.ceph:role: osd.2 2024-04-25T08:53:17.327 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi043.front.sepia.ceph.com 2024-04-25T08:53:17.327 DEBUG:teuthology.orchestra.run.smithi043:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T08:53:17.392 INFO:teuthology.orchestra.run.smithi043.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:53:17.392 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:53:17.392 INFO:teuthology.orchestra.run.smithi043.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:53:17.392 INFO:teuthology.orchestra.run.smithi043.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T08:53:17.392 INFO:teuthology.orchestra.run.smithi043.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:53:17.392 INFO:teuthology.orchestra.run.smithi043.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:53:17.392 INFO:teuthology.orchestra.run.smithi043.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:53:17.393 INFO:teuthology.orchestra.run.smithi043.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T08:53:17.393 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:53:17.393 INFO:teuthology.orchestra.run.smithi043.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:53:17.425 INFO:teuthology.orchestra.run.smithi043.stdout:Discarding blocks...Done. 2024-04-25T08:53:17.427 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi043.front.sepia.ceph.com -o noatime 2024-04-25T08:53:17.427 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-25T08:53:17.496 DEBUG:teuthology.orchestra.run.smithi043:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-25T08:53:17.566 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-25T08:53:17.631 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T08:53:17.631 INFO:tasks.ceph:role: osd.3 2024-04-25T08:53:17.631 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi043.front.sepia.ceph.com 2024-04-25T08:53:17.631 DEBUG:teuthology.orchestra.run.smithi043:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T08:53:17.695 INFO:teuthology.orchestra.run.smithi043.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:53:17.695 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:53:17.695 INFO:teuthology.orchestra.run.smithi043.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:53:17.695 INFO:teuthology.orchestra.run.smithi043.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T08:53:17.695 INFO:teuthology.orchestra.run.smithi043.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:53:17.695 INFO:teuthology.orchestra.run.smithi043.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:53:17.695 INFO:teuthology.orchestra.run.smithi043.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:53:17.695 INFO:teuthology.orchestra.run.smithi043.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T08:53:17.696 INFO:teuthology.orchestra.run.smithi043.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:53:17.696 INFO:teuthology.orchestra.run.smithi043.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:53:17.727 INFO:teuthology.orchestra.run.smithi043.stdout:Discarding blocks...Done. 2024-04-25T08:53:17.729 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi043.front.sepia.ceph.com -o noatime 2024-04-25T08:53:17.729 DEBUG:teuthology.orchestra.run.smithi043:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-25T08:53:17.797 DEBUG:teuthology.orchestra.run.smithi043:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-25T08:53:17.865 DEBUG:teuthology.orchestra.run.smithi043:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:53:17.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:17.966+0000 7fb5f8e16640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-25T08:53:17.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:17.966+0000 7fb5f8e16640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-25T08:53:17.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:17.966+0000 7fb5f8e16640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T08:53:17.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:17.966+0000 7fb5f8e16640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T08:53:17.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:17.966+0000 7fb5f8e16640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T08:53:17.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:17.966+0000 7fb5f8e16640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-25T08:53:20.808 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T08:53:20.837 DEBUG:teuthology.orchestra.run.smithi043:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:53:20.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:20.937+0000 7fb59aba7640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-25T08:53:20.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:20.938+0000 7fb59aba7640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-25T08:53:20.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:20.938+0000 7fb59aba7640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T08:53:20.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:20.938+0000 7fb59aba7640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T08:53:20.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:20.938+0000 7fb59aba7640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T08:53:20.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:20.938+0000 7fb59aba7640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-25T08:53:23.777 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T08:53:23.806 DEBUG:teuthology.orchestra.run.smithi043:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:53:23.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:23.908+0000 7eff5100c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-25T08:53:23.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:23.908+0000 7eff5100c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-25T08:53:23.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:23.908+0000 7eff5100c640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T08:53:23.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:23.908+0000 7eff5100c640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T08:53:23.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:23.908+0000 7eff5100c640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T08:53:23.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:23.908+0000 7eff5100c640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-25T08:53:26.743 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T08:53:26.772 DEBUG:teuthology.orchestra.run.smithi043:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:53:26.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:26.873+0000 7f4bd120f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-25T08:53:26.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:26.873+0000 7f4bd120f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-25T08:53:26.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:26.873+0000 7f4bd120f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T08:53:26.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:26.873+0000 7f4bd120f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T08:53:26.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:26.873+0000 7f4bd120f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T08:53:26.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:26.873+0000 7f4bd120f640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-25T08:53:29.725 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T08:53:29.754 DEBUG:teuthology.orchestra.run.smithi193:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-25T08:53:29.782 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T08:53:29.782 INFO:tasks.ceph:role: osd.4 2024-04-25T08:53:29.782 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi193.front.sepia.ceph.com 2024-04-25T08:53:29.782 DEBUG:teuthology.orchestra.run.smithi193:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T08:53:29.855 INFO:teuthology.orchestra.run.smithi193.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:53:29.856 INFO:teuthology.orchestra.run.smithi193.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:53:29.856 INFO:teuthology.orchestra.run.smithi193.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:53:29.856 INFO:teuthology.orchestra.run.smithi193.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T08:53:29.856 INFO:teuthology.orchestra.run.smithi193.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:53:29.856 INFO:teuthology.orchestra.run.smithi193.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:53:29.856 INFO:teuthology.orchestra.run.smithi193.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:53:29.856 INFO:teuthology.orchestra.run.smithi193.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T08:53:29.856 INFO:teuthology.orchestra.run.smithi193.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:53:29.856 INFO:teuthology.orchestra.run.smithi193.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:53:30.360 INFO:teuthology.orchestra.run.smithi193.stdout:Discarding blocks...Done. 2024-04-25T08:53:30.363 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi193.front.sepia.ceph.com -o noatime 2024-04-25T08:53:30.363 DEBUG:teuthology.orchestra.run.smithi193:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-25T08:53:30.478 DEBUG:teuthology.orchestra.run.smithi193:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-25T08:53:30.508 DEBUG:teuthology.orchestra.run.smithi193:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-25T08:53:30.573 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T08:53:30.573 INFO:tasks.ceph:role: osd.5 2024-04-25T08:53:30.573 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi193.front.sepia.ceph.com 2024-04-25T08:53:30.573 DEBUG:teuthology.orchestra.run.smithi193:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T08:53:30.646 INFO:teuthology.orchestra.run.smithi193.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:53:30.646 INFO:teuthology.orchestra.run.smithi193.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:53:30.646 INFO:teuthology.orchestra.run.smithi193.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:53:30.646 INFO:teuthology.orchestra.run.smithi193.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T08:53:30.646 INFO:teuthology.orchestra.run.smithi193.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:53:30.647 INFO:teuthology.orchestra.run.smithi193.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:53:30.647 INFO:teuthology.orchestra.run.smithi193.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:53:30.647 INFO:teuthology.orchestra.run.smithi193.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T08:53:30.647 INFO:teuthology.orchestra.run.smithi193.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:53:30.647 INFO:teuthology.orchestra.run.smithi193.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:53:31.111 INFO:teuthology.orchestra.run.smithi193.stdout:Discarding blocks...Done. 2024-04-25T08:53:31.114 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi193.front.sepia.ceph.com -o noatime 2024-04-25T08:53:31.114 DEBUG:teuthology.orchestra.run.smithi193:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-25T08:53:31.205 DEBUG:teuthology.orchestra.run.smithi193:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-25T08:53:31.233 DEBUG:teuthology.orchestra.run.smithi193:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-25T08:53:31.297 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T08:53:31.297 INFO:tasks.ceph:role: osd.6 2024-04-25T08:53:31.297 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi193.front.sepia.ceph.com 2024-04-25T08:53:31.297 DEBUG:teuthology.orchestra.run.smithi193:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T08:53:31.368 INFO:teuthology.orchestra.run.smithi193.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:53:31.368 INFO:teuthology.orchestra.run.smithi193.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:53:31.368 INFO:teuthology.orchestra.run.smithi193.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:53:31.368 INFO:teuthology.orchestra.run.smithi193.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T08:53:31.368 INFO:teuthology.orchestra.run.smithi193.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:53:31.368 INFO:teuthology.orchestra.run.smithi193.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:53:31.368 INFO:teuthology.orchestra.run.smithi193.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:53:31.368 INFO:teuthology.orchestra.run.smithi193.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T08:53:31.369 INFO:teuthology.orchestra.run.smithi193.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:53:31.369 INFO:teuthology.orchestra.run.smithi193.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:53:31.854 INFO:teuthology.orchestra.run.smithi193.stdout:Discarding blocks...Done. 2024-04-25T08:53:31.856 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi193.front.sepia.ceph.com -o noatime 2024-04-25T08:53:31.856 DEBUG:teuthology.orchestra.run.smithi193:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-25T08:53:32.007 DEBUG:teuthology.orchestra.run.smithi193:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-25T08:53:32.037 DEBUG:teuthology.orchestra.run.smithi193:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-25T08:53:32.102 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T08:53:32.102 INFO:tasks.ceph:role: osd.7 2024-04-25T08:53:32.103 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi193.front.sepia.ceph.com 2024-04-25T08:53:32.103 DEBUG:teuthology.orchestra.run.smithi193:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T08:53:32.174 INFO:teuthology.orchestra.run.smithi193.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T08:53:32.174 INFO:teuthology.orchestra.run.smithi193.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T08:53:32.174 INFO:teuthology.orchestra.run.smithi193.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T08:53:32.174 INFO:teuthology.orchestra.run.smithi193.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T08:53:32.174 INFO:teuthology.orchestra.run.smithi193.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T08:53:32.174 INFO:teuthology.orchestra.run.smithi193.stdout: = sunit=0 swidth=0 blks 2024-04-25T08:53:32.174 INFO:teuthology.orchestra.run.smithi193.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T08:53:32.174 INFO:teuthology.orchestra.run.smithi193.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T08:53:32.174 INFO:teuthology.orchestra.run.smithi193.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T08:53:32.175 INFO:teuthology.orchestra.run.smithi193.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T08:53:32.658 INFO:teuthology.orchestra.run.smithi193.stdout:Discarding blocks...Done. 2024-04-25T08:53:32.660 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi193.front.sepia.ceph.com -o noatime 2024-04-25T08:53:32.660 DEBUG:teuthology.orchestra.run.smithi193:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-25T08:53:32.803 DEBUG:teuthology.orchestra.run.smithi193:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-25T08:53:32.834 DEBUG:teuthology.orchestra.run.smithi193:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:53:32.925 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:32.934+0000 7f331a4b3640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-25T08:53:32.925 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:32.935+0000 7f331a4b3640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-25T08:53:32.925 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:32.935+0000 7f331a4b3640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T08:53:32.925 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:32.935+0000 7f331a4b3640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T08:53:32.925 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:32.935+0000 7f331a4b3640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T08:53:32.926 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:32.935+0000 7f331a4b3640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-25T08:53:35.775 DEBUG:teuthology.orchestra.run.smithi193:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T08:53:35.803 DEBUG:teuthology.orchestra.run.smithi193:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:53:35.892 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:35.901+0000 7fb6885a1640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-25T08:53:35.893 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:35.901+0000 7fb6885a1640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-25T08:53:35.893 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:35.901+0000 7fb6885a1640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T08:53:35.893 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:35.901+0000 7fb6885a1640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T08:53:35.893 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:35.901+0000 7fb6885a1640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T08:53:35.893 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:35.902+0000 7fb6885a1640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-25T08:53:38.738 DEBUG:teuthology.orchestra.run.smithi193:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T08:53:38.766 DEBUG:teuthology.orchestra.run.smithi193:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:53:38.855 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:38.864+0000 7f5d70516640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-25T08:53:38.855 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:38.865+0000 7f5d70516640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-25T08:53:38.855 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:38.865+0000 7f5d70516640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T08:53:38.855 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:38.865+0000 7f5d70516640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T08:53:38.855 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:38.865+0000 7f5d70516640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T08:53:38.855 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:38.865+0000 7f5d70516640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-25T08:53:41.705 DEBUG:teuthology.orchestra.run.smithi193:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-25T08:53:41.730 DEBUG:teuthology.orchestra.run.smithi193:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:53:41.815 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:41.824+0000 7f8736e0d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-25T08:53:41.815 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:41.825+0000 7f8736e0d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-25T08:53:41.815 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:41.825+0000 7f8736e0d640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T08:53:41.815 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:41.825+0000 7f8736e0d640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T08:53:41.815 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:41.825+0000 7f8736e0d640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T08:53:41.815 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:41.825+0000 7f8736e0d640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-25T08:53:43.549 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:53:43.552 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:53:44.668 DEBUG:teuthology.orchestra.run.smithi193:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-25T08:53:44.695 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-25T08:53:44.695 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:53:44.695 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-25T08:53:44.724 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:53:44.724 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-25T08:53:44.793 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:53:44.793 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-25T08:53:44.860 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:53:44.861 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-25T08:53:44.927 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:53:44.928 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-25T08:53:44.994 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:53:44.994 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-25T08:53:45.060 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:53:45.060 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-25T08:53:45.125 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T08:53:45.126 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-25T08:53:45.154 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T08:53:45.154 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-25T08:53:45.221 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T08:53:45.221 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-25T08:53:45.288 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T08:53:45.289 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-25T08:53:45.354 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T08:53:45.355 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-25T08:53:45.419 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T08:53:45.419 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-25T08:53:45.484 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T08:53:45.484 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-25T08:53:45.549 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:53:45.549 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-25T08:53:45.567 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:53:45.567 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-25T08:53:45.626 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T08:53:45.626 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-25T08:53:45.641 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T08:53:45.642 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-25T08:53:45.696 INFO:tasks.ceph:Adding keys to all mons... 2024-04-25T08:53:45.697 DEBUG:teuthology.orchestra.run.smithi043:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T08:53:45.699 DEBUG:teuthology.orchestra.run.smithi193:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T08:53:45.740 INFO:teuthology.orchestra.run.smithi043.stdout:[mgr.y] 2024-04-25T08:53:45.740 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQD7GSpmPwwpOhAALeiQ/LKM2zgjE7yu16RQOg== 2024-04-25T08:53:45.740 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.a] 2024-04-25T08:53:45.740 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQD8GSpmIjzEBRAAs2L10XrFO3mThjZevna5wA== 2024-04-25T08:53:45.740 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.c] 2024-04-25T08:53:45.740 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQD8GSpm1OOtDBAAlUYbCh1mBEEka9CWRnUz9A== 2024-04-25T08:53:45.740 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.0] 2024-04-25T08:53:45.740 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQD9GSpmi7qkORAAwSNgwzcVMitrPPf3nDyqvQ== 2024-04-25T08:53:45.740 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.1] 2024-04-25T08:53:45.740 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAAGipmJDT3NxAAcWBCd9nt7Au+Xftjqcuc5w== 2024-04-25T08:53:45.741 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.2] 2024-04-25T08:53:45.741 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQADGipm68kzNhAA0ZOnlr4edWDjx6RfoEHYIw== 2024-04-25T08:53:45.741 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.3] 2024-04-25T08:53:45.741 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAGGipmT1AeNBAAoNPiEtkznSxxhVJ1EmZ1ZA== 2024-04-25T08:53:45.741 INFO:teuthology.orchestra.run.smithi043.stdout:[mgr.x] 2024-04-25T08:53:45.741 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQD8GSpm/uGhAhAAr0FaJ6SwZ8Su1MEqyZwNRA== 2024-04-25T08:53:45.741 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.b] 2024-04-25T08:53:45.741 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQD8GSpm51cmERAAij4vy8xyihh8u/4dkmcllw== 2024-04-25T08:53:45.741 INFO:teuthology.orchestra.run.smithi043.stdout:[mds.d] 2024-04-25T08:53:45.741 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQD8GSpmpEvsFxAAXX1h37UWdFkyJ3Iw3xd8kw== 2024-04-25T08:53:45.741 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.4] 2024-04-25T08:53:45.741 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAMGipmBjHHNxAAH/SCzKHus8fKxxI3UN4wWQ== 2024-04-25T08:53:45.741 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.5] 2024-04-25T08:53:45.741 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAPGipm3FXKNRAAL97YYWONg4QHr4ajC5D7fA== 2024-04-25T08:53:45.742 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.6] 2024-04-25T08:53:45.742 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQASGipmCwKaMxAAcM0p9Q2sbNQviGkwSLg8AA== 2024-04-25T08:53:45.742 INFO:teuthology.orchestra.run.smithi043.stdout:[osd.7] 2024-04-25T08:53:45.742 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQAVGipmP7A2MRAAxnFJFosMcbZ6VDbgH8r6wA== 2024-04-25T08:53:45.742 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2024-04-25T08:53:45.742 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQD8GSpmVOjWGxAAFNmxeGi0h2D//pa6Ne3eQw== 2024-04-25T08:53:45.742 INFO:teuthology.orchestra.run.smithi043.stdout:[client.2] 2024-04-25T08:53:45.742 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQD8GSpm8b5UIRAACcOwDL630oJrXfp7s+fNTA== 2024-04-25T08:53:45.742 INFO:teuthology.orchestra.run.smithi043.stdout:[client.1] 2024-04-25T08:53:45.742 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQD8GSpm24laJBAA9ijbDlFDTmF/ZZ4IhLfBVQ== 2024-04-25T08:53:45.742 INFO:teuthology.orchestra.run.smithi043.stdout:[client.3] 2024-04-25T08:53:45.742 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQD8GSpmLsbBKRAAIKWtRdpuyCaW1LnKMWTMiw== 2024-04-25T08:53:45.758 INFO:teuthology.orchestra.run.smithi193.stdout:[mgr.y] 2024-04-25T08:53:45.758 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQD7GSpmPwwpOhAALeiQ/LKM2zgjE7yu16RQOg== 2024-04-25T08:53:45.758 INFO:teuthology.orchestra.run.smithi193.stdout:[mds.a] 2024-04-25T08:53:45.758 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQD8GSpmIjzEBRAAs2L10XrFO3mThjZevna5wA== 2024-04-25T08:53:45.759 INFO:teuthology.orchestra.run.smithi193.stdout:[mds.c] 2024-04-25T08:53:45.759 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQD8GSpm1OOtDBAAlUYbCh1mBEEka9CWRnUz9A== 2024-04-25T08:53:45.759 INFO:teuthology.orchestra.run.smithi193.stdout:[osd.0] 2024-04-25T08:53:45.759 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQD9GSpmi7qkORAAwSNgwzcVMitrPPf3nDyqvQ== 2024-04-25T08:53:45.759 INFO:teuthology.orchestra.run.smithi193.stdout:[osd.1] 2024-04-25T08:53:45.759 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQAAGipmJDT3NxAAcWBCd9nt7Au+Xftjqcuc5w== 2024-04-25T08:53:45.759 INFO:teuthology.orchestra.run.smithi193.stdout:[osd.2] 2024-04-25T08:53:45.759 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQADGipm68kzNhAA0ZOnlr4edWDjx6RfoEHYIw== 2024-04-25T08:53:45.759 INFO:teuthology.orchestra.run.smithi193.stdout:[osd.3] 2024-04-25T08:53:45.759 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQAGGipmT1AeNBAAoNPiEtkznSxxhVJ1EmZ1ZA== 2024-04-25T08:53:45.759 INFO:teuthology.orchestra.run.smithi193.stdout:[mgr.x] 2024-04-25T08:53:45.759 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQD8GSpm/uGhAhAAr0FaJ6SwZ8Su1MEqyZwNRA== 2024-04-25T08:53:45.759 INFO:teuthology.orchestra.run.smithi193.stdout:[mds.b] 2024-04-25T08:53:45.760 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQD8GSpm51cmERAAij4vy8xyihh8u/4dkmcllw== 2024-04-25T08:53:45.760 INFO:teuthology.orchestra.run.smithi193.stdout:[mds.d] 2024-04-25T08:53:45.760 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQD8GSpmpEvsFxAAXX1h37UWdFkyJ3Iw3xd8kw== 2024-04-25T08:53:45.760 INFO:teuthology.orchestra.run.smithi193.stdout:[osd.4] 2024-04-25T08:53:45.760 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQAMGipmBjHHNxAAH/SCzKHus8fKxxI3UN4wWQ== 2024-04-25T08:53:45.760 INFO:teuthology.orchestra.run.smithi193.stdout:[osd.5] 2024-04-25T08:53:45.760 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQAPGipm3FXKNRAAL97YYWONg4QHr4ajC5D7fA== 2024-04-25T08:53:45.760 INFO:teuthology.orchestra.run.smithi193.stdout:[osd.6] 2024-04-25T08:53:45.760 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQASGipmCwKaMxAAcM0p9Q2sbNQviGkwSLg8AA== 2024-04-25T08:53:45.760 INFO:teuthology.orchestra.run.smithi193.stdout:[osd.7] 2024-04-25T08:53:45.760 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQAVGipmP7A2MRAAxnFJFosMcbZ6VDbgH8r6wA== 2024-04-25T08:53:45.760 INFO:teuthology.orchestra.run.smithi193.stdout:[client.0] 2024-04-25T08:53:45.760 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQD8GSpmVOjWGxAAFNmxeGi0h2D//pa6Ne3eQw== 2024-04-25T08:53:45.760 INFO:teuthology.orchestra.run.smithi193.stdout:[client.2] 2024-04-25T08:53:45.760 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQD8GSpm8b5UIRAACcOwDL630oJrXfp7s+fNTA== 2024-04-25T08:53:45.761 INFO:teuthology.orchestra.run.smithi193.stdout:[client.1] 2024-04-25T08:53:45.761 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQD8GSpm24laJBAA9ijbDlFDTmF/ZZ4IhLfBVQ== 2024-04-25T08:53:45.761 INFO:teuthology.orchestra.run.smithi193.stdout:[client.3] 2024-04-25T08:53:45.761 INFO:teuthology.orchestra.run.smithi193.stdout: key = AQD8GSpmLsbBKRAAIKWtRdpuyCaW1LnKMWTMiw== 2024-04-25T08:53:45.762 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T08:53:45.786 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T08:53:45.850 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:53:45.853 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:53:45.906 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:53:45.908 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:53:45.960 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:53:45.962 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:53:46.010 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:53:46.013 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:53:46.061 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:53:46.064 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:53:46.112 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:53:46.115 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:53:46.163 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T08:53:46.165 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T08:53:46.214 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:53:46.216 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:53:46.267 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:53:46.270 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T08:53:46.321 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:53:46.323 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:53:46.372 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:53:46.374 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:53:46.422 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:53:46.424 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:53:46.473 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:53:46.475 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T08:53:46.524 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:53:46.526 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:53:46.577 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:53:46.579 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:53:46.633 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:53:46.635 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:53:46.686 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:53:46.688 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T08:53:46.740 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-25T08:53:46.740 DEBUG:teuthology.orchestra.run.smithi043:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-25T08:53:46.764 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T08:53:46.860 DEBUG:teuthology.orchestra.run.smithi043:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-25T08:53:46.884 DEBUG:teuthology.orchestra.run.smithi193:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-25T08:53:46.908 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T08:53:47.001 DEBUG:teuthology.orchestra.run.smithi193:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-25T08:53:47.024 DEBUG:teuthology.orchestra.run.smithi193:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-25T08:53:47.088 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T08:53:47.182 DEBUG:teuthology.orchestra.run.smithi193:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-25T08:53:47.206 DEBUG:teuthology.orchestra.run.smithi043:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:53:47.209 DEBUG:teuthology.orchestra.run.smithi193:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T08:53:47.260 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-25T08:53:47.260 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T08:53:47.260 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T08:53:47.265 INFO:tasks.ceph.mon.a:Started 2024-04-25T08:53:47.265 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T08:53:47.265 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T08:53:47.302 INFO:tasks.ceph.mon.b:Started 2024-04-25T08:53:47.302 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T08:53:47.302 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T08:53:47.304 INFO:tasks.ceph.mon.c:Started 2024-04-25T08:53:47.304 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-25T08:53:47.305 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T08:53:47.305 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-25T08:53:47.307 INFO:tasks.ceph.mgr.y:Started 2024-04-25T08:53:47.307 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T08:53:47.307 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-25T08:53:47.309 INFO:tasks.ceph.mgr.x:Started 2024-04-25T08:53:47.310 INFO:tasks.ceph:Setting crush tunables to default 2024-04-25T08:53:47.310 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph osd crush tunables default 2024-04-25T08:53:47.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.422+0000 7f0a275c4640 1 Processor -- start 2024-04-25T08:53:47.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.422+0000 7f0a275c4640 1 -- start start 2024-04-25T08:53:47.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.423+0000 7f0a275c4640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0a20155df0 0x7f0a200ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:47.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.423+0000 7f0a275c4640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a2015f8b0 con 0x7f0a2006ea80 2024-04-25T08:53:47.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.423+0000 7f0a275c4640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a2015fb90 con 0x7f0a2006b240 2024-04-25T08:53:47.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.423+0000 7f0a275c4640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2015fe70 con 0x7f0a20155df0 2024-04-25T08:53:47.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.423+0000 7f0a24b5f640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f0a2006ea80 0x7f0a2006ab30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58146/0 (socket says 172.21.15.43:58146) 2024-04-25T08:53:47.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.423+0000 7f0a24b5f640 1 -- 172.21.15.43:0/1636026218 learned_addr learned my addr 172.21.15.43:0/1636026218 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:53:47.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.423+0000 7f0a25360640 1 --2- 172.21.15.43:0/1636026218 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0a20155df0 0x7f0a200ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:47.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.424+0000 7f0a25360640 1 -- 172.21.15.43:0/1636026218 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0a20155df0 msgr2=0x7f0a200ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-25T08:53:47.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.424+0000 7f0a25360640 1 -- 172.21.15.43:0/1636026218 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0a20155df0 msgr2=0x7f0a200ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T08:53:47.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.424+0000 7f0a25360640 1 --2- 172.21.15.43:0/1636026218 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0a20155df0 0x7f0a200ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T08:53:47.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.424+0000 7f0a25360640 1 --2- 172.21.15.43:0/1636026218 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0a20155df0 0x7f0a200ce800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T08:53:47.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.581+0000 7f0a17fff640 1 -- 172.21.15.43:0/1636026218 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 841054960 0 0) 0x7f0a2015f8b0 con 0x7f0a2006ea80 2024-04-25T08:53:47.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.581+0000 7f0a17fff640 1 -- 172.21.15.43:0/1636026218 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a0c003520 con 0x7f0a2006ea80 2024-04-25T08:53:47.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.581+0000 7f0a17fff640 1 -- 172.21.15.43:0/1636026218 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4234271455 0 0) 0x7f0a2015fb90 con 0x7f0a2006b240 2024-04-25T08:53:47.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.581+0000 7f0a17fff640 1 -- 172.21.15.43:0/1636026218 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a2015f8b0 con 0x7f0a2006b240 2024-04-25T08:53:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.582+0000 7f0a17fff640 1 -- 172.21.15.43:0/1636026218 <== mon.1 v1:172.21.15.193:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1795747528 0 0) 0x7f0a2015f8b0 con 0x7f0a2006b240 2024-04-25T08:53:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.582+0000 7f0a17fff640 1 -- 172.21.15.43:0/1636026218 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0a20155df0 msgr2=0x7f0a200ce800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T08:53:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.582+0000 7f0a17fff640 1 --2- 172.21.15.43:0/1636026218 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0a20155df0 0x7f0a200ce800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.582+0000 7f0a17fff640 1 -- 172.21.15.43:0/1636026218 >> v1:172.21.15.43:6789/0 conn(0x7f0a2006ea80 legacy=0x7f0a2006ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.582+0000 7f0a17fff640 1 -- 172.21.15.43:0/1636026218 --> v1:172.21.15.193:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a20160150 con 0x7f0a2006b240 2024-04-25T08:53:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.582+0000 7f0a17fff640 1 -- 172.21.15.43:0/1636026218 <== mon.1 v1:172.21.15.193:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f0a180019f0 con 0x7f0a2006b240 2024-04-25T08:53:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.582+0000 7f0a17fff640 1 -- 172.21.15.43:0/1636026218 >> v1:172.21.15.193:6789/0 conn(0x7f0a2006b240 legacy=0x7f0a2015c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.582+0000 7f0a17fff640 1 --2- 172.21.15.43:0/1636026218 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a0c003b40 0x7f0a0c003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.582+0000 7f0a17fff640 1 --2- 172.21.15.43:0/1636026218 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0a20155df0 0x7f0a0c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.582+0000 7f0a17fff640 1 --2- 172.21.15.43:0/1636026218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a0c004d40 0x7f0a0c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.582+0000 7f0a17fff640 1 -- 172.21.15.43:0/1636026218 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2015fe70 con 0x7f0a0c004d40 2024-04-25T08:53:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.582+0000 7f0a17fff640 1 -- 172.21.15.43:0/1636026218 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c00bb00 con 0x7f0a0c003b40 2024-04-25T08:53:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.582+0000 7f0a17fff640 1 -- 172.21.15.43:0/1636026218 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c00bd00 con 0x7f0a20155df0 2024-04-25T08:53:47.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.582+0000 7f0a25b61640 1 --2- 172.21.15.43:0/1636026218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a0c004d40 0x7f0a0c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:47.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.582+0000 7f0a25360640 1 --2- 172.21.15.43:0/1636026218 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a0c003b40 0x7f0a0c003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:47.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.582+0000 7f0a24b5f640 1 --2- 172.21.15.43:0/1636026218 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0a20155df0 0x7f0a0c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:47.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.582+0000 7f0a25b61640 1 -- 172.21.15.43:0/1636026218 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0a20155df0 msgr2=0x7f0a0c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:47.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.582+0000 7f0a25b61640 1 --2- 172.21.15.43:0/1636026218 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0a20155df0 0x7f0a0c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:47.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.582+0000 7f0a25b61640 1 -- 172.21.15.43:0/1636026218 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a0c003b40 msgr2=0x7f0a0c003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:47.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.582+0000 7f0a25b61640 1 --2- 172.21.15.43:0/1636026218 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a0c003b40 0x7f0a0c003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:47.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.582+0000 7f0a25b61640 1 -- 172.21.15.43:0/1636026218 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a2015f8b0 con 0x7f0a0c004d40 2024-04-25T08:53:47.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.584+0000 7f0a25b61640 1 --2- 172.21.15.43:0/1636026218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a0c004d40 0x7f0a0c0051c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0a2006e920 tx=0x7f0a0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57ea9c6e77df2438 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:47.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.584+0000 7f0a17fff640 1 -- 172.21.15.43:0/1636026218 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a18033070 con 0x7f0a0c004d40 2024-04-25T08:53:47.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.584+0000 7f0a17fff640 1 -- 172.21.15.43:0/1636026218 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a1802f640 con 0x7f0a0c004d40 2024-04-25T08:53:47.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.584+0000 7f0a17fff640 1 -- 172.21.15.43:0/1636026218 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a1800c0e0 con 0x7f0a0c004d40 2024-04-25T08:53:47.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.585+0000 7f0a275c4640 1 -- 172.21.15.43:0/1636026218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a0c004d40 msgr2=0x7f0a0c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:47.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.585+0000 7f0a275c4640 1 --2- 172.21.15.43:0/1636026218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a0c004d40 0x7f0a0c0051c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0a2006e920 tx=0x7f0a0c005830 comp rx=0 tx=0).stop 2024-04-25T08:53:47.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.585+0000 7f0a25b61640 1 -- 172.21.15.43:0/1636026218 reap_dead start 2024-04-25T08:53:47.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.585+0000 7f0a275c4640 1 -- 172.21.15.43:0/1636026218 shutdown_connections 2024-04-25T08:53:47.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.585+0000 7f0a275c4640 1 -- 172.21.15.43:0/1636026218 >> 172.21.15.43:0/1636026218 conn(0x7f0a20066d10 msgr2=0x7f0a2006bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:47.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.585+0000 7f0a275c4640 1 -- 172.21.15.43:0/1636026218 shutdown_connections 2024-04-25T08:53:47.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.585+0000 7f0a275c4640 1 -- 172.21.15.43:0/1636026218 wait complete. 2024-04-25T08:53:47.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.586+0000 7f0a275c4640 1 Processor -- start 2024-04-25T08:53:47.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.587+0000 7f0a275c4640 1 -- start start 2024-04-25T08:53:47.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.587+0000 7f0a275c4640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0a0c004d40 0x7f0a2014bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:47.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.587+0000 7f0a275c4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a2014b770 0x7f0a200b2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:47.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.587+0000 7f0a275c4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a200b2e70 0x7f0a2015a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:47.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.587+0000 7f0a275c4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a20156350 con 0x7f0a2014b770 2024-04-25T08:53:47.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.587+0000 7f0a275c4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a201561d0 con 0x7f0a200b2e70 2024-04-25T08:53:47.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.587+0000 7f0a275c4640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2006eee0 con 0x7f0a0c004d40 2024-04-25T08:53:47.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.588+0000 7f0a25360640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0a0c004d40 0x7f0a2014bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:47.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.588+0000 7f0a24b5f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a2014b770 0x7f0a200b2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:47.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.588+0000 7f0a25360640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0a0c004d40 0x7f0a2014bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:39250/0 (socket says 172.21.15.43:39250) 2024-04-25T08:53:47.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.588+0000 7f0a24b5f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a2014b770 0x7f0a200b2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:53182/0 (socket says 172.21.15.43:53182) 2024-04-25T08:53:47.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.588+0000 7f0a24b5f640 1 -- 172.21.15.43:0/2801597948 learned_addr learned my addr 172.21.15.43:0/2801597948 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:53:47.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.588+0000 7f0a25b61640 1 --2- 172.21.15.43:0/2801597948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a200b2e70 0x7f0a2015a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:47.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.588+0000 7f0a24b5f640 1 -- 172.21.15.43:0/2801597948 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0a0c004d40 msgr2=0x7f0a2014bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:47.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.588+0000 7f0a24b5f640 1 --2- 172.21.15.43:0/2801597948 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0a0c004d40 0x7f0a2014bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:47.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.588+0000 7f0a24b5f640 1 -- 172.21.15.43:0/2801597948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a200b2e70 msgr2=0x7f0a2015a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:47.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.588+0000 7f0a24b5f640 1 --2- 172.21.15.43:0/2801597948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a200b2e70 0x7f0a2015a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:47.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.588+0000 7f0a24b5f640 1 -- 172.21.15.43:0/2801597948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a2014f5d0 con 0x7f0a2014b770 2024-04-25T08:53:47.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.588+0000 7f0a24b5f640 1 --2- 172.21.15.43:0/2801597948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a2014b770 0x7f0a200b2930 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c001780 tx=0x7f0a1c001c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:47.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.589+0000 7f0a15ffb640 1 -- 172.21.15.43:0/2801597948 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c007740 con 0x7f0a2014b770 2024-04-25T08:53:47.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.589+0000 7f0a15ffb640 1 -- 172.21.15.43:0/2801597948 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a1c018070 con 0x7f0a2014b770 2024-04-25T08:53:47.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.589+0000 7f0a275c4640 1 -- 172.21.15.43:0/2801597948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a2014f8b0 con 0x7f0a2014b770 2024-04-25T08:53:47.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.589+0000 7f0a15ffb640 1 -- 172.21.15.43:0/2801597948 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a1c00fba0 con 0x7f0a2014b770 2024-04-25T08:53:47.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.589+0000 7f0a275c4640 1 -- 172.21.15.43:0/2801597948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a2014fe70 con 0x7f0a2014b770 2024-04-25T08:53:47.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.589+0000 7f0a15ffb640 1 -- 172.21.15.43:0/2801597948 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f0a1c00fd40 con 0x7f0a2014b770 2024-04-25T08:53:47.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.589+0000 7f0a15ffb640 1 -- 172.21.15.43:0/2801597948 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0a1c013570 con 0x7f0a2014b770 2024-04-25T08:53:47.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.590+0000 7f0a275c4640 1 -- 172.21.15.43:0/2801597948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a20074f60 con 0x7f0a2014b770 2024-04-25T08:53:47.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.594+0000 7f0a15ffb640 1 -- 172.21.15.43:0/2801597948 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f0a1c0137c0 con 0x7f0a2014b770 2024-04-25T08:53:47.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.661+0000 7f0a275c4640 1 -- 172.21.15.43:0/2801597948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f0a2014efd0 con 0x7f0a2014b770 2024-04-25T08:53:47.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.663+0000 7f0a15ffb640 1 -- 172.21.15.43:0/2801597948 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f0a1c013a00 con 0x7f0a2014b770 2024-04-25T08:53:47.653 INFO:teuthology.orchestra.run.smithi043.stderr:adjusted tunables profile to default 2024-04-25T08:53:47.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.664+0000 7f0a275c4640 1 -- 172.21.15.43:0/2801597948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a2014b770 msgr2=0x7f0a200b2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:47.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.664+0000 7f0a275c4640 1 --2- 172.21.15.43:0/2801597948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a2014b770 0x7f0a200b2930 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c001780 tx=0x7f0a1c001c40 comp rx=0 tx=0).stop 2024-04-25T08:53:47.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.664+0000 7f0a275c4640 1 -- 172.21.15.43:0/2801597948 shutdown_connections 2024-04-25T08:53:47.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.664+0000 7f0a275c4640 1 --2- 172.21.15.43:0/2801597948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a200b2e70 0x7f0a2015a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:47.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.664+0000 7f0a275c4640 1 --2- 172.21.15.43:0/2801597948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a2014b770 0x7f0a200b2930 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:47.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.664+0000 7f0a275c4640 1 --2- 172.21.15.43:0/2801597948 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0a0c004d40 0x7f0a2014bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:47.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.664+0000 7f0a275c4640 1 -- 172.21.15.43:0/2801597948 >> 172.21.15.43:0/2801597948 conn(0x7f0a20066d10 msgr2=0x7f0a200cffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:47.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.664+0000 7f0a275c4640 1 -- 172.21.15.43:0/2801597948 shutdown_connections 2024-04-25T08:53:47.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:47.664+0000 7f0a275c4640 1 -- 172.21.15.43:0/2801597948 wait complete. 2024-04-25T08:53:47.665 INFO:tasks.ceph:check_enable_crimson: False 2024-04-25T08:53:47.665 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-25T08:53:47.665 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:53:47.665 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-25T08:53:47.682 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-25T08:53:47.693+0000 7facf6922040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T08:53:47.685 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-25T08:53:47.695+0000 7f78d0b23040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T08:53:47.690 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:53:47.690 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-25T08:53:47.754 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:53:47.754 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-25T08:53:47.818 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:53:47.818 DEBUG:teuthology.orchestra.run.smithi043:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-25T08:53:47.835 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-25T08:53:47.846+0000 7facf6922040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T08:53:47.836 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-25T08:53:47.847+0000 7f78d0b23040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T08:53:47.863 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T08:53:47.863 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-25T08:53:47.887 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T08:53:47.887 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-25T08:53:47.890 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-25T08:53:47.900+0000 7facf6922040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T08:53:47.894 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-25T08:53:47.904+0000 7f78d0b23040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T08:53:47.913 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T08:53:47.913 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-25T08:53:47.977 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T08:53:47.977 DEBUG:teuthology.orchestra.run.smithi193:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-25T08:53:48.041 DEBUG:teuthology.orchestra.run.smithi193:> sudo ceph --cluster ceph osd new a7d2992b-c326-4e19-9b74-72f017cc2613 0 2024-04-25T08:53:48.145 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-25T08:53:48.155+0000 7facf6922040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T08:53:48.154 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-25T08:53:48.164+0000 7f78d0b23040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T08:53:48.180 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.191+0000 7eff1156a640 1 Processor -- start 2024-04-25T08:53:48.181 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.191+0000 7eff1156a640 1 -- start start 2024-04-25T08:53:48.182 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.191+0000 7eff1156a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7eff0c15e090 0x7eff0c161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:48.182 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.191+0000 7eff1156a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff0c161fe0 0x7eff0c0b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:48.183 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.191+0000 7eff1156a640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff0c162610 con 0x7eff0c0b5aa0 2024-04-25T08:53:48.183 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.191+0000 7eff1156a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff0c0b5f00 con 0x7eff0c15e090 2024-04-25T08:53:48.183 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.191+0000 7eff1156a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff0c0d1da0 con 0x7eff0c161fe0 2024-04-25T08:53:48.183 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.192+0000 7eff0a7fc640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff0c161fe0 0x7eff0c0b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:48.183 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.192+0000 7eff0b7fe640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7eff0c0b5aa0 0x7eff0c0ce560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.193:40238/0 (socket says 172.21.15.193:40238) 2024-04-25T08:53:48.183 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.192+0000 7eff0b7fe640 1 -- 172.21.15.193:0/1333064316 learned_addr learned my addr 172.21.15.193:0/1333064316 (peer_addr_for_me v1:172.21.15.193:0/0) 2024-04-25T08:53:48.183 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.192+0000 7eff0affd640 1 --2- 172.21.15.193:0/1333064316 >> v2:172.21.15.43:3300/0 conn(0x7eff0c15e090 0x7eff0c161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:48.183 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.192+0000 7eff0a7fc640 1 -- 172.21.15.193:0/1333064316 >> v2:172.21.15.43:3300/0 conn(0x7eff0c15e090 msgr2=0x7eff0c161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:48.183 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.192+0000 7eff0a7fc640 1 --2- 172.21.15.193:0/1333064316 >> v2:172.21.15.43:3300/0 conn(0x7eff0c15e090 0x7eff0c161aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.183 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.192+0000 7eff09ffb640 1 -- 172.21.15.193:0/1333064316 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3289374614 0 0) 0x7eff0c162610 con 0x7eff0c0b5aa0 2024-04-25T08:53:48.183 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.192+0000 7eff0a7fc640 1 -- 172.21.15.193:0/1333064316 >> v1:172.21.15.193:6789/0 conn(0x7eff0c0b5aa0 legacy=0x7eff0c0ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:48.183 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.192+0000 7eff0a7fc640 1 -- 172.21.15.193:0/1333064316 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff0c0d1750 con 0x7eff0c161fe0 2024-04-25T08:53:48.184 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.192+0000 7eff0affd640 1 --2- 172.21.15.193:0/1333064316 >> v2:172.21.15.43:3300/0 conn(0x7eff0c15e090 0x7eff0c161aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:48.184 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.193+0000 7eff0a7fc640 1 --2- 172.21.15.193:0/1333064316 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff0c161fe0 0x7eff0c0b5560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7eff000097a0 tx=0x7eff0002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99248efa79bdec8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:48.184 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.193+0000 7eff09ffb640 1 -- 172.21.15.193:0/1333064316 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff00037070 con 0x7eff0c161fe0 2024-04-25T08:53:48.184 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.193+0000 7eff09ffb640 1 -- 172.21.15.193:0/1333064316 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff00032440 con 0x7eff0c161fe0 2024-04-25T08:53:48.184 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.193+0000 7eff09ffb640 1 -- 172.21.15.193:0/1333064316 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff00032810 con 0x7eff0c161fe0 2024-04-25T08:53:48.187 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.193+0000 7eff1156a640 1 -- 172.21.15.193:0/1333064316 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff0c161fe0 msgr2=0x7eff0c0b5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:48.188 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.193+0000 7eff1156a640 1 --2- 172.21.15.193:0/1333064316 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff0c161fe0 0x7eff0c0b5560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7eff000097a0 tx=0x7eff0002a210 comp rx=0 tx=0).stop 2024-04-25T08:53:48.188 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.193+0000 7eff1156a640 1 -- 172.21.15.193:0/1333064316 shutdown_connections 2024-04-25T08:53:48.188 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.193+0000 7eff1156a640 1 --2- 172.21.15.193:0/1333064316 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff0c161fe0 0x7eff0c0b5560 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.188 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.193+0000 7eff1156a640 1 --2- 172.21.15.193:0/1333064316 >> v2:172.21.15.43:3300/0 conn(0x7eff0c15e090 0x7eff0c161aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.188 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.193+0000 7eff1156a640 1 -- 172.21.15.193:0/1333064316 >> 172.21.15.193:0/1333064316 conn(0x7eff0c0d0560 msgr2=0x7eff0c0d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:48.188 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.193+0000 7eff1156a640 1 -- 172.21.15.193:0/1333064316 shutdown_connections 2024-04-25T08:53:48.188 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.193+0000 7eff1156a640 1 -- 172.21.15.193:0/1333064316 wait complete. 2024-04-25T08:53:48.188 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.195+0000 7eff1156a640 1 Processor -- start 2024-04-25T08:53:48.188 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.195+0000 7eff1156a640 1 -- start start 2024-04-25T08:53:48.188 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.195+0000 7eff1156a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff0c0b5aa0 0x7eff0c1567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:48.188 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.195+0000 7eff1156a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff0c15e090 0x7eff0c1581e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:48.188 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.195+0000 7eff1156a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff0c161fe0 0x7eff0c158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:48.189 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.195+0000 7eff1156a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0c0b3280 con 0x7eff0c0b5aa0 2024-04-25T08:53:48.189 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.195+0000 7eff1156a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0c1628f0 con 0x7eff0c15e090 2024-04-25T08:53:48.189 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.195+0000 7eff1156a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff0c0d3240 con 0x7eff0c161fe0 2024-04-25T08:53:48.189 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.195+0000 7eff0a7fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff0c15e090 0x7eff0c1581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:48.189 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.195+0000 7eff0b7fe640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff0c161fe0 0x7eff0c158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:48.189 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.196+0000 7eff0a7fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff0c15e090 0x7eff0c1581e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:55158/0 (socket says 172.21.15.193:55158) 2024-04-25T08:53:48.189 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.196+0000 7eff0a7fc640 1 -- 172.21.15.193:0/1045117899 learned_addr learned my addr 172.21.15.193:0/1045117899 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-25T08:53:48.189 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.196+0000 7eff0affd640 1 --2- 172.21.15.193:0/1045117899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff0c0b5aa0 0x7eff0c1567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:48.189 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.196+0000 7eff0b7fe640 1 -- 172.21.15.193:0/1045117899 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff0c15e090 msgr2=0x7eff0c1581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:48.189 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.196+0000 7eff0b7fe640 1 --2- 172.21.15.193:0/1045117899 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff0c15e090 0x7eff0c1581e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.189 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.196+0000 7eff0b7fe640 1 -- 172.21.15.193:0/1045117899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff0c0b5aa0 msgr2=0x7eff0c1567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:48.189 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.196+0000 7eff0b7fe640 1 --2- 172.21.15.193:0/1045117899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff0c0b5aa0 0x7eff0c1567a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.190 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.196+0000 7eff0b7fe640 1 -- 172.21.15.193:0/1045117899 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff0c160e00 con 0x7eff0c161fe0 2024-04-25T08:53:48.190 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.196+0000 7eff0a7fc640 1 --2- 172.21.15.193:0/1045117899 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff0c15e090 0x7eff0c1581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:53:48.190 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.196+0000 7eff0b7fe640 1 --2- 172.21.15.193:0/1045117899 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff0c161fe0 0x7eff0c158720 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7eff0400c6d0 tx=0x7eff0400cb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:48.190 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.196+0000 7efef37fe640 1 -- 172.21.15.193:0/1045117899 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff04018070 con 0x7eff0c161fe0 2024-04-25T08:53:48.190 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.196+0000 7efef37fe640 1 -- 172.21.15.193:0/1045117899 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff04004d60 con 0x7eff0c161fe0 2024-04-25T08:53:48.190 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.196+0000 7eff1156a640 1 -- 172.21.15.193:0/1045117899 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff0c1534f0 con 0x7eff0c161fe0 2024-04-25T08:53:48.190 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.196+0000 7efef37fe640 1 -- 172.21.15.193:0/1045117899 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0400a880 con 0x7eff0c161fe0 2024-04-25T08:53:48.190 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.196+0000 7eff1156a640 1 -- 172.21.15.193:0/1045117899 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff0c153ab0 con 0x7eff0c161fe0 2024-04-25T08:53:48.190 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.196+0000 7efef37fe640 1 -- 172.21.15.193:0/1045117899 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7eff0401b070 con 0x7eff0c161fe0 2024-04-25T08:53:48.190 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.196+0000 7efef37fe640 1 -- 172.21.15.193:0/1045117899 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7eff0401d780 con 0x7eff0c161fe0 2024-04-25T08:53:48.190 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.197+0000 7eff1156a640 1 -- 172.21.15.193:0/1045117899 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff0c06bf00 con 0x7eff0c161fe0 2024-04-25T08:53:48.191 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.200+0000 7efef37fe640 1 -- 172.21.15.193:0/1045117899 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7eff0c06bf00 con 0x7eff0c161fe0 2024-04-25T08:53:48.194 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-25T08:53:48.204+0000 7facf6922040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T08:53:48.201 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-25T08:53:48.211+0000 7f78d0b23040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T08:53:48.261 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.271+0000 7eff1156a640 1 -- 172.21.15.193:0/1045117899 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "osd new", "uuid": "a7d2992b-c326-4e19-9b74-72f017cc2613", "id": 0} v 0) v1 -- 0x7eff0c074f60 con 0x7eff0c161fe0 2024-04-25T08:53:48.267 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.277+0000 7efef37fe640 1 -- 172.21.15.193:0/1045117899 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "a7d2992b-c326-4e19-9b74-72f017cc2613", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7eff0400ac80 con 0x7eff0c161fe0 2024-04-25T08:53:48.267 INFO:teuthology.orchestra.run.smithi193.stdout:0 2024-04-25T08:53:48.268 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.278+0000 7eff1156a640 1 -- 172.21.15.193:0/1045117899 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff0c161fe0 msgr2=0x7eff0c158720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:48.268 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.278+0000 7eff1156a640 1 --2- 172.21.15.193:0/1045117899 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff0c161fe0 0x7eff0c158720 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7eff0400c6d0 tx=0x7eff0400cb90 comp rx=0 tx=0).stop 2024-04-25T08:53:48.268 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.278+0000 7eff1156a640 1 -- 172.21.15.193:0/1045117899 shutdown_connections 2024-04-25T08:53:48.268 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.278+0000 7eff1156a640 1 --2- 172.21.15.193:0/1045117899 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff0c161fe0 0x7eff0c158720 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.268 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.278+0000 7eff1156a640 1 --2- 172.21.15.193:0/1045117899 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff0c15e090 0x7eff0c1581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.269 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.278+0000 7eff1156a640 1 --2- 172.21.15.193:0/1045117899 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff0c0b5aa0 0x7eff0c1567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.269 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.278+0000 7eff1156a640 1 -- 172.21.15.193:0/1045117899 >> 172.21.15.193:0/1045117899 conn(0x7eff0c0d0560 msgr2=0x7eff0c0b8230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:48.269 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.278+0000 7eff1156a640 1 -- 172.21.15.193:0/1045117899 shutdown_connections 2024-04-25T08:53:48.269 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.278+0000 7eff1156a640 1 -- 172.21.15.193:0/1045117899 wait complete. 2024-04-25T08:53:48.279 DEBUG:teuthology.orchestra.run.smithi193:> sudo ceph --cluster ceph osd new 88734834-c723-40a7-914b-1090e7fe47ac 1 2024-04-25T08:53:48.376 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.386+0000 7fc82439e640 1 Processor -- start 2024-04-25T08:53:48.376 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.386+0000 7fc82439e640 1 -- start start 2024-04-25T08:53:48.377 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.387+0000 7fc82439e640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fc81c070b80 0x7fc81c15bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:48.377 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.387+0000 7fc82439e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc81c0aea20 0x7fc81c0aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:48.377 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.387+0000 7fc82439e640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc81c15ce90 con 0x7fc81c15cab0 2024-04-25T08:53:48.377 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.387+0000 7fc82439e640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc81c0ad7e0 con 0x7fc81c070b80 2024-04-25T08:53:48.377 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.387+0000 7fc82439e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c0adac0 con 0x7fc81c0aea20 2024-04-25T08:53:48.377 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.387+0000 7fc821939640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fc81c070b80 0x7fc81c15bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:48.378 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.387+0000 7fc82293b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc81c0aea20 0x7fc81c0aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:48.378 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.387+0000 7fc82213a640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7fc81c15cab0 0x7fc81c0acfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.193:40242/0 (socket says 172.21.15.193:40242) 2024-04-25T08:53:48.378 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.387+0000 7fc82213a640 1 -- 172.21.15.193:0/2034761248 learned_addr learned my addr 172.21.15.193:0/2034761248 (peer_addr_for_me v1:172.21.15.193:0/0) 2024-04-25T08:53:48.378 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.387+0000 7fc821939640 1 -- 172.21.15.193:0/2034761248 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc81c0aea20 msgr2=0x7fc81c0aeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:48.378 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.387+0000 7fc821939640 1 --2- 172.21.15.193:0/2034761248 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc81c0aea20 0x7fc81c0aeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.378 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.387+0000 7fc821939640 1 -- 172.21.15.193:0/2034761248 >> v1:172.21.15.193:6789/0 conn(0x7fc81c15cab0 legacy=0x7fc81c0acfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:48.378 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.387+0000 7fc821939640 1 -- 172.21.15.193:0/2034761248 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc81c0adda0 con 0x7fc81c070b80 2024-04-25T08:53:48.378 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.387+0000 7fc821138640 1 -- 172.21.15.193:0/2034761248 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 359846425 0 0) 0x7fc81c15ce90 con 0x7fc81c15cab0 2024-04-25T08:53:48.378 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.387+0000 7fc82293b640 1 --2- 172.21.15.193:0/2034761248 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc81c0aea20 0x7fc81c0aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:53:48.379 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.388+0000 7fc821939640 1 --2- 172.21.15.193:0/2034761248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc81c070b80 0x7fc81c15bfa0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc80c0097a0 tx=0x7fc80c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0e80dcba2268619 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:48.380 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.388+0000 7fc821138640 1 -- 172.21.15.193:0/2034761248 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c037070 con 0x7fc81c070b80 2024-04-25T08:53:48.380 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.388+0000 7fc821138640 1 -- 172.21.15.193:0/2034761248 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc80c032440 con 0x7fc81c070b80 2024-04-25T08:53:48.380 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.388+0000 7fc821138640 1 -- 172.21.15.193:0/2034761248 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c032810 con 0x7fc81c070b80 2024-04-25T08:53:48.380 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.389+0000 7fc82439e640 1 -- 172.21.15.193:0/2034761248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc81c070b80 msgr2=0x7fc81c15bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:48.380 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.389+0000 7fc82439e640 1 --2- 172.21.15.193:0/2034761248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc81c070b80 0x7fc81c15bfa0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc80c0097a0 tx=0x7fc80c02a180 comp rx=0 tx=0).stop 2024-04-25T08:53:48.380 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.389+0000 7fc82439e640 1 -- 172.21.15.193:0/2034761248 shutdown_connections 2024-04-25T08:53:48.380 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.389+0000 7fc82439e640 1 --2- 172.21.15.193:0/2034761248 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc81c0aea20 0x7fc81c0aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.380 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.389+0000 7fc82439e640 1 --2- 172.21.15.193:0/2034761248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc81c070b80 0x7fc81c15bfa0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.380 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.389+0000 7fc82439e640 1 -- 172.21.15.193:0/2034761248 >> 172.21.15.193:0/2034761248 conn(0x7fc81c06ab30 msgr2=0x7fc81c06af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:48.380 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.389+0000 7fc82439e640 1 -- 172.21.15.193:0/2034761248 shutdown_connections 2024-04-25T08:53:48.380 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.389+0000 7fc82439e640 1 -- 172.21.15.193:0/2034761248 wait complete. 2024-04-25T08:53:48.380 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.390+0000 7fc82439e640 1 Processor -- start 2024-04-25T08:53:48.380 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.390+0000 7fc82439e640 1 -- start start 2024-04-25T08:53:48.383 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.391+0000 7fc82439e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc81c070b80 0x7fc81c1f3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:48.383 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.391+0000 7fc82439e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc81c0aea20 0x7fc81c1f6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:48.383 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.391+0000 7fc82439e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc81c15cab0 0x7fc81c09c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:48.383 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.391+0000 7fc82439e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c1520c0 con 0x7fc81c0aea20 2024-04-25T08:53:48.383 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.391+0000 7fc82439e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c06e170 con 0x7fc81c070b80 2024-04-25T08:53:48.383 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.391+0000 7fc82439e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc81c152240 con 0x7fc81c15cab0 2024-04-25T08:53:48.383 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.391+0000 7fc82213a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc81c070b80 0x7fc81c1f3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:48.384 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.391+0000 7fc82293b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc81c15cab0 0x7fc81c09c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:48.384 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.391+0000 7fc82213a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc81c070b80 0x7fc81c1f3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:55180/0 (socket says 172.21.15.193:55180) 2024-04-25T08:53:48.384 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.391+0000 7fc82293b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc81c15cab0 0x7fc81c09c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.193:57194/0 (socket says 172.21.15.193:57194) 2024-04-25T08:53:48.384 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.391+0000 7fc82213a640 1 -- 172.21.15.193:0/1732017292 learned_addr learned my addr 172.21.15.193:0/1732017292 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-25T08:53:48.384 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.391+0000 7fc821939640 1 --2- 172.21.15.193:0/1732017292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc81c0aea20 0x7fc81c1f6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:48.384 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.391+0000 7fc82213a640 1 -- 172.21.15.193:0/1732017292 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc81c15cab0 msgr2=0x7fc81c09c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:48.384 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.391+0000 7fc82213a640 1 --2- 172.21.15.193:0/1732017292 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc81c15cab0 0x7fc81c09c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.384 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.391+0000 7fc82213a640 1 -- 172.21.15.193:0/1732017292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc81c0aea20 msgr2=0x7fc81c1f6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:48.384 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.391+0000 7fc82213a640 1 --2- 172.21.15.193:0/1732017292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc81c0aea20 0x7fc81c1f6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.384 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.391+0000 7fc82213a640 1 -- 172.21.15.193:0/1732017292 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc81c14f0e0 con 0x7fc81c070b80 2024-04-25T08:53:48.384 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.391+0000 7fc82293b640 1 --2- 172.21.15.193:0/1732017292 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc81c15cab0 0x7fc81c09c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:53:48.384 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.391+0000 7fc821939640 1 --2- 172.21.15.193:0/1732017292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc81c0aea20 0x7fc81c1f6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:48.384 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.391+0000 7fc82213a640 1 --2- 172.21.15.193:0/1732017292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc81c070b80 0x7fc81c1f3750 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc81800c430 tx=0x7fc81800c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:48.385 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.391+0000 7fc812ffd640 1 -- 172.21.15.193:0/1732017292 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8180074e0 con 0x7fc81c070b80 2024-04-25T08:53:48.385 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.391+0000 7fc812ffd640 1 -- 172.21.15.193:0/1732017292 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8180047e0 con 0x7fc81c070b80 2024-04-25T08:53:48.385 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.391+0000 7fc82439e640 1 -- 172.21.15.193:0/1732017292 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc81c14f660 con 0x7fc81c070b80 2024-04-25T08:53:48.385 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.391+0000 7fc812ffd640 1 -- 172.21.15.193:0/1732017292 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc818004bd0 con 0x7fc81c070b80 2024-04-25T08:53:48.385 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.391+0000 7fc82439e640 1 -- 172.21.15.193:0/1732017292 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc81c09d1a0 con 0x7fc81c070b80 2024-04-25T08:53:48.385 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.392+0000 7fc812ffd640 1 -- 172.21.15.193:0/1732017292 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fc818011070 con 0x7fc81c070b80 2024-04-25T08:53:48.385 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.392+0000 7fc812ffd640 1 -- 172.21.15.193:0/1732017292 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fc818013770 con 0x7fc81c070b80 2024-04-25T08:53:48.385 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.392+0000 7fc82439e640 1 -- 172.21.15.193:0/1732017292 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc81c074f60 con 0x7fc81c070b80 2024-04-25T08:53:48.385 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.394+0000 7fc812ffd640 1 -- 172.21.15.193:0/1732017292 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fc8180139c0 con 0x7fc81c070b80 2024-04-25T08:53:48.456 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.466+0000 7fc82439e640 1 -- 172.21.15.193:0/1732017292 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd new", "uuid": "88734834-c723-40a7-914b-1090e7fe47ac", "id": 1} v 0) v1 -- 0x7fc81c06c030 con 0x7fc81c070b80 2024-04-25T08:53:48.461 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.471+0000 7fc812ffd640 1 -- 172.21.15.193:0/1732017292 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "88734834-c723-40a7-914b-1090e7fe47ac", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc818013ba0 con 0x7fc81c070b80 2024-04-25T08:53:48.461 INFO:teuthology.orchestra.run.smithi193.stdout:1 2024-04-25T08:53:48.463 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.472+0000 7fc82439e640 1 -- 172.21.15.193:0/1732017292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc81c070b80 msgr2=0x7fc81c1f3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:48.463 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.472+0000 7fc82439e640 1 --2- 172.21.15.193:0/1732017292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc81c070b80 0x7fc81c1f3750 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc81800c430 tx=0x7fc81800c830 comp rx=0 tx=0).stop 2024-04-25T08:53:48.463 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.472+0000 7fc82439e640 1 -- 172.21.15.193:0/1732017292 shutdown_connections 2024-04-25T08:53:48.463 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.472+0000 7fc82439e640 1 --2- 172.21.15.193:0/1732017292 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc81c15cab0 0x7fc81c09c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.463 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.472+0000 7fc82439e640 1 --2- 172.21.15.193:0/1732017292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc81c0aea20 0x7fc81c1f6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.463 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.472+0000 7fc82439e640 1 --2- 172.21.15.193:0/1732017292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc81c070b80 0x7fc81c1f3750 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.463 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.472+0000 7fc82439e640 1 -- 172.21.15.193:0/1732017292 >> 172.21.15.193:0/1732017292 conn(0x7fc81c06ab30 msgr2=0x7fc81c151920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:48.463 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.472+0000 7fc82439e640 1 -- 172.21.15.193:0/1732017292 shutdown_connections 2024-04-25T08:53:48.463 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.472+0000 7fc82439e640 1 -- 172.21.15.193:0/1732017292 wait complete. 2024-04-25T08:53:48.472 DEBUG:teuthology.orchestra.run.smithi193:> sudo ceph --cluster ceph osd new b938e87d-7038-4a58-acdb-ff9a43a1855b 2 2024-04-25T08:53:48.569 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.579+0000 7f3b685c1640 1 Processor -- start 2024-04-25T08:53:48.570 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.580+0000 7f3b685c1640 1 -- start start 2024-04-25T08:53:48.572 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.581+0000 7f3b685c1640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b6015a750 0x7f3b600cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:48.572 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.581+0000 7f3b685c1640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f3b60071fa0 0x7f3b600d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:48.572 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.581+0000 7f3b685c1640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3b600af850 0x7f3b600d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:48.572 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.581+0000 7f3b685c1640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b600b4350 con 0x7f3b600af850 2024-04-25T08:53:48.572 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.581+0000 7f3b685c1640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b600afda0 con 0x7f3b60071fa0 2024-04-25T08:53:48.573 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.581+0000 7f3b685c1640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b600cf0f0 con 0x7f3b6015a750 2024-04-25T08:53:48.573 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.581+0000 7f3b6635d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b6015a750 0x7f3b600cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:48.573 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.581+0000 7f3b6635d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b6015a750 0x7f3b600cf580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.193:57206/0 (socket says 172.21.15.193:57206) 2024-04-25T08:53:48.573 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.581+0000 7f3b6635d640 1 -- 172.21.15.193:0/143966080 learned_addr learned my addr 172.21.15.193:0/143966080 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-25T08:53:48.573 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.581+0000 7f3b65b5c640 1 --2- 172.21.15.193:0/143966080 >> v2:172.21.15.193:3300/0 conn(0x7f3b60071fa0 0x7f3b600d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:48.573 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.581+0000 7f3b66b5e640 1 --2- 172.21.15.193:0/143966080 >> v2:172.21.15.43:3300/0 conn(0x7f3b600af850 0x7f3b600d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:48.573 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.581+0000 7f3b6635d640 1 -- 172.21.15.193:0/143966080 >> v2:172.21.15.193:3300/0 conn(0x7f3b60071fa0 msgr2=0x7f3b600d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:48.573 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.581+0000 7f3b6635d640 1 --2- 172.21.15.193:0/143966080 >> v2:172.21.15.193:3300/0 conn(0x7f3b60071fa0 0x7f3b600d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.573 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.581+0000 7f3b6635d640 1 -- 172.21.15.193:0/143966080 >> v2:172.21.15.43:3300/0 conn(0x7f3b600af850 msgr2=0x7f3b600d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:48.573 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.581+0000 7f3b6635d640 1 --2- 172.21.15.193:0/143966080 >> v2:172.21.15.43:3300/0 conn(0x7f3b600af850 0x7f3b600d1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.573 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.581+0000 7f3b6635d640 1 -- 172.21.15.193:0/143966080 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b600ce7e0 con 0x7f3b6015a750 2024-04-25T08:53:48.573 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.582+0000 7f3b6635d640 1 --2- 172.21.15.193:0/143966080 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b6015a750 0x7f3b600cf580 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3b600b3410 tx=0x7f3b5c02a1e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48d6038a11adf533 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:48.573 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.582+0000 7f3b6535b640 1 -- 172.21.15.193:0/143966080 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c037070 con 0x7f3b6015a750 2024-04-25T08:53:48.574 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.582+0000 7f3b6535b640 1 -- 172.21.15.193:0/143966080 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b5c032440 con 0x7f3b6015a750 2024-04-25T08:53:48.574 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.582+0000 7f3b6535b640 1 -- 172.21.15.193:0/143966080 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c032810 con 0x7f3b6015a750 2024-04-25T08:53:48.577 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.583+0000 7f3b685c1640 1 -- 172.21.15.193:0/143966080 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b6015a750 msgr2=0x7f3b600cf580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:48.577 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.583+0000 7f3b685c1640 1 --2- 172.21.15.193:0/143966080 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b6015a750 0x7f3b600cf580 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3b600b3410 tx=0x7f3b5c02a1e0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.577 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.583+0000 7f3b685c1640 1 -- 172.21.15.193:0/143966080 shutdown_connections 2024-04-25T08:53:48.577 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.583+0000 7f3b685c1640 1 --2- 172.21.15.193:0/143966080 >> v2:172.21.15.193:3300/0 conn(0x7f3b60071fa0 0x7f3b600d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.577 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.583+0000 7f3b685c1640 1 --2- 172.21.15.193:0/143966080 >> v2:172.21.15.43:3300/0 conn(0x7f3b600af850 0x7f3b600d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.577 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.583+0000 7f3b685c1640 1 --2- 172.21.15.193:0/143966080 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b6015a750 0x7f3b600cf580 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.577 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.583+0000 7f3b685c1640 1 -- 172.21.15.193:0/143966080 >> 172.21.15.193:0/143966080 conn(0x7f3b6006c060 msgr2=0x7f3b6006c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:48.577 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.583+0000 7f3b685c1640 1 -- 172.21.15.193:0/143966080 shutdown_connections 2024-04-25T08:53:48.577 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.583+0000 7f3b685c1640 1 -- 172.21.15.193:0/143966080 wait complete. 2024-04-25T08:53:48.578 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.584+0000 7f3b685c1640 1 Processor -- start 2024-04-25T08:53:48.578 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.584+0000 7f3b685c1640 1 -- start start 2024-04-25T08:53:48.578 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.585+0000 7f3b685c1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b60071fa0 0x7f3b60159080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:48.578 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.585+0000 7f3b685c1640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b600af850 0x7f3b60152280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:48.578 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.585+0000 7f3b685c1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b6015a750 0x7f3b60157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:48.578 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.585+0000 7f3b685c1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b600ced80 con 0x7f3b6015a750 2024-04-25T08:53:48.578 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.585+0000 7f3b685c1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b600ceaa0 con 0x7f3b60071fa0 2024-04-25T08:53:48.578 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.585+0000 7f3b685c1640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6015ab30 con 0x7f3b600af850 2024-04-25T08:53:48.578 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.585+0000 7f3b6635d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b60071fa0 0x7f3b60159080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:48.578 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.585+0000 7f3b65b5c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b600af850 0x7f3b60152280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:48.578 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.585+0000 7f3b65b5c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b600af850 0x7f3b60152280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.193:57220/0 (socket says 172.21.15.193:57220) 2024-04-25T08:53:48.578 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.585+0000 7f3b65b5c640 1 -- 172.21.15.193:0/3714983495 learned_addr learned my addr 172.21.15.193:0/3714983495 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-25T08:53:48.579 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.585+0000 7f3b6635d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b60071fa0 0x7f3b60159080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:55194/0 (socket says 172.21.15.193:55194) 2024-04-25T08:53:48.579 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.585+0000 7f3b66b5e640 1 --2- 172.21.15.193:0/3714983495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b6015a750 0x7f3b60157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:48.579 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.585+0000 7f3b65b5c640 1 -- 172.21.15.193:0/3714983495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b60071fa0 msgr2=0x7f3b60159080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:48.579 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.585+0000 7f3b65b5c640 1 --2- 172.21.15.193:0/3714983495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b60071fa0 0x7f3b60159080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.579 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.585+0000 7f3b65b5c640 1 -- 172.21.15.193:0/3714983495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b6015a750 msgr2=0x7f3b60157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:48.579 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.585+0000 7f3b65b5c640 1 --2- 172.21.15.193:0/3714983495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b6015a750 0x7f3b60157bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.579 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.585+0000 7f3b65b5c640 1 -- 172.21.15.193:0/3714983495 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b601595c0 con 0x7f3b600af850 2024-04-25T08:53:48.579 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.585+0000 7f3b6635d640 1 --2- 172.21.15.193:0/3714983495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b60071fa0 0x7f3b60159080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:53:48.579 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.586+0000 7f3b65b5c640 1 --2- 172.21.15.193:0/3714983495 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b600af850 0x7f3b60152280 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3b50010dc0 tx=0x7f3b5000e730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:48.579 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.586+0000 7f3b56ffd640 1 -- 172.21.15.193:0/3714983495 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5001e070 con 0x7f3b600af850 2024-04-25T08:53:48.579 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.586+0000 7f3b56ffd640 1 -- 172.21.15.193:0/3714983495 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b5000be50 con 0x7f3b600af850 2024-04-25T08:53:48.579 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.586+0000 7f3b56ffd640 1 -- 172.21.15.193:0/3714983495 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b50011a80 con 0x7f3b600af850 2024-04-25T08:53:48.579 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.586+0000 7f3b685c1640 1 -- 172.21.15.193:0/3714983495 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b600a9570 con 0x7f3b600af850 2024-04-25T08:53:48.579 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.586+0000 7f3b685c1640 1 -- 172.21.15.193:0/3714983495 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b601573f0 con 0x7f3b600af850 2024-04-25T08:53:48.579 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.586+0000 7f3b56ffd640 1 -- 172.21.15.193:0/3714983495 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f3b50021070 con 0x7f3b600af850 2024-04-25T08:53:48.580 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.586+0000 7f3b56ffd640 1 -- 172.21.15.193:0/3714983495 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f3b500065d0 con 0x7f3b600af850 2024-04-25T08:53:48.580 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.587+0000 7f3b685c1640 1 -- 172.21.15.193:0/3714983495 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b60074f60 con 0x7f3b600af850 2024-04-25T08:53:48.580 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.589+0000 7f3b56ffd640 1 -- 172.21.15.193:0/3714983495 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f3b50028070 con 0x7f3b600af850 2024-04-25T08:53:48.651 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.661+0000 7f3b685c1640 1 -- 172.21.15.193:0/3714983495 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b938e87d-7038-4a58-acdb-ff9a43a1855b", "id": 2} v 0) v1 -- 0x7f3b600b02f0 con 0x7f3b600af850 2024-04-25T08:53:48.654 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.664+0000 7f3b56ffd640 1 -- 172.21.15.193:0/3714983495 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b938e87d-7038-4a58-acdb-ff9a43a1855b", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3b50011c20 con 0x7f3b600af850 2024-04-25T08:53:48.654 INFO:teuthology.orchestra.run.smithi193.stdout:2 2024-04-25T08:53:48.655 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.665+0000 7f3b685c1640 1 -- 172.21.15.193:0/3714983495 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b600af850 msgr2=0x7f3b60152280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:48.655 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.665+0000 7f3b685c1640 1 --2- 172.21.15.193:0/3714983495 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b600af850 0x7f3b60152280 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3b50010dc0 tx=0x7f3b5000e730 comp rx=0 tx=0).stop 2024-04-25T08:53:48.656 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.665+0000 7f3b685c1640 1 -- 172.21.15.193:0/3714983495 shutdown_connections 2024-04-25T08:53:48.656 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.665+0000 7f3b685c1640 1 --2- 172.21.15.193:0/3714983495 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b6015a750 0x7f3b60157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.656 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.665+0000 7f3b685c1640 1 --2- 172.21.15.193:0/3714983495 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b600af850 0x7f3b60152280 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.656 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.665+0000 7f3b685c1640 1 --2- 172.21.15.193:0/3714983495 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b60071fa0 0x7f3b60159080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.656 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.665+0000 7f3b685c1640 1 -- 172.21.15.193:0/3714983495 >> 172.21.15.193:0/3714983495 conn(0x7f3b6006c060 msgr2=0x7f3b6015eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:48.656 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.665+0000 7f3b685c1640 1 -- 172.21.15.193:0/3714983495 shutdown_connections 2024-04-25T08:53:48.656 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.665+0000 7f3b685c1640 1 -- 172.21.15.193:0/3714983495 wait complete. 2024-04-25T08:53:48.666 DEBUG:teuthology.orchestra.run.smithi193:> sudo ceph --cluster ceph osd new b34857e6-f4b8-4177-82ea-041f4bb1c42c 3 2024-04-25T08:53:48.764 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.774+0000 7f60125fe640 1 Processor -- start 2024-04-25T08:53:48.764 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.774+0000 7f60125fe640 1 -- start start 2024-04-25T08:53:48.766 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.775+0000 7f60125fe640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f600c0cfa00 0x7f600c0720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:48.766 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.775+0000 7f60125fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f600c0ad4b0 0x7f600c15eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:48.766 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.775+0000 7f60125fe640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f600c0af210 con 0x7f600c070b80 2024-04-25T08:53:48.766 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.775+0000 7f60125fe640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f600c0725f0 con 0x7f600c0ad4b0 2024-04-25T08:53:48.766 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.775+0000 7f60125fe640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f600c15f0d0 con 0x7f600c0cfa00 2024-04-25T08:53:48.766 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.775+0000 7f600bfff640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f600c0cfa00 0x7f600c0720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:48.766 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.775+0000 7f600b7fe640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f600c070b80 0x7f600c0acda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.193:40252/0 (socket says 172.21.15.193:40252) 2024-04-25T08:53:48.766 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.775+0000 7f600bfff640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f600c0cfa00 0x7f600c0720b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.193:57234/0 (socket says 172.21.15.193:57234) 2024-04-25T08:53:48.766 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.775+0000 7f600b7fe640 1 -- 172.21.15.193:0/3588169129 learned_addr learned my addr 172.21.15.193:0/3588169129 (peer_addr_for_me v1:172.21.15.193:0/0) 2024-04-25T08:53:48.766 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.775+0000 7f6010b9b640 1 --2- 172.21.15.193:0/3588169129 >> v2:172.21.15.43:3300/0 conn(0x7f600c0ad4b0 0x7f600c15eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:48.766 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.775+0000 7f600bfff640 1 -- 172.21.15.193:0/3588169129 >> v2:172.21.15.43:3300/0 conn(0x7f600c0ad4b0 msgr2=0x7f600c15eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:48.766 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.775+0000 7f600bfff640 1 --2- 172.21.15.193:0/3588169129 >> v2:172.21.15.43:3300/0 conn(0x7f600c0ad4b0 0x7f600c15eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.767 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.775+0000 7f600affd640 1 -- 172.21.15.193:0/3588169129 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1203611692 0 0) 0x7f600c0af210 con 0x7f600c070b80 2024-04-25T08:53:48.767 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.775+0000 7f600bfff640 1 -- 172.21.15.193:0/3588169129 >> v1:172.21.15.193:6789/0 conn(0x7f600c070b80 legacy=0x7f600c0acda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:48.767 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.775+0000 7f600bfff640 1 -- 172.21.15.193:0/3588169129 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f600c071010 con 0x7f600c0cfa00 2024-04-25T08:53:48.767 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.775+0000 7f6010b9b640 1 --2- 172.21.15.193:0/3588169129 >> v2:172.21.15.43:3300/0 conn(0x7f600c0ad4b0 0x7f600c15eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:53:48.767 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.776+0000 7f600bfff640 1 --2- 172.21.15.193:0/3588169129 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f600c0cfa00 0x7f600c0720b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc0097a0 tx=0x7f5ffc02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=399999d740207d2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:48.767 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.776+0000 7f600affd640 1 -- 172.21.15.193:0/3588169129 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc037070 con 0x7f600c0cfa00 2024-04-25T08:53:48.767 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.776+0000 7f600affd640 1 -- 172.21.15.193:0/3588169129 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ffc02ae50 con 0x7f600c0cfa00 2024-04-25T08:53:48.767 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.776+0000 7f600affd640 1 -- 172.21.15.193:0/3588169129 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc032660 con 0x7f600c0cfa00 2024-04-25T08:53:48.770 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.777+0000 7f60125fe640 1 -- 172.21.15.193:0/3588169129 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f600c0cfa00 msgr2=0x7f600c0720b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:48.770 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.777+0000 7f60125fe640 1 --2- 172.21.15.193:0/3588169129 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f600c0cfa00 0x7f600c0720b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc0097a0 tx=0x7f5ffc02a100 comp rx=0 tx=0).stop 2024-04-25T08:53:48.770 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.777+0000 7f60125fe640 1 -- 172.21.15.193:0/3588169129 shutdown_connections 2024-04-25T08:53:48.771 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.777+0000 7f60125fe640 1 --2- 172.21.15.193:0/3588169129 >> v2:172.21.15.43:3300/0 conn(0x7f600c0ad4b0 0x7f600c15eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.771 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.777+0000 7f60125fe640 1 --2- 172.21.15.193:0/3588169129 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f600c0cfa00 0x7f600c0720b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.771 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.777+0000 7f60125fe640 1 -- 172.21.15.193:0/3588169129 >> 172.21.15.193:0/3588169129 conn(0x7f600c06ab30 msgr2=0x7f600c06af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:48.771 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.777+0000 7f60125fe640 1 -- 172.21.15.193:0/3588169129 shutdown_connections 2024-04-25T08:53:48.771 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.777+0000 7f60125fe640 1 -- 172.21.15.193:0/3588169129 wait complete. 2024-04-25T08:53:48.771 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.778+0000 7f60125fe640 1 Processor -- start 2024-04-25T08:53:48.771 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.778+0000 7f60125fe640 1 -- start start 2024-04-25T08:53:48.771 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.779+0000 7f60125fe640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f600c0ad4b0 0x7f600c14c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:48.771 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.779+0000 7f60125fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f600c0cfa00 0x7f600c09cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:48.771 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.779+0000 7f60125fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f600c09d140 0x7f600c14d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:48.772 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.779+0000 7f60125fe640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600c0718b0 con 0x7f600c0cfa00 2024-04-25T08:53:48.772 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.779+0000 7f60125fe640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600c0715d0 con 0x7f600c09d140 2024-04-25T08:53:48.772 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.779+0000 7f60125fe640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f600c071b90 con 0x7f600c0ad4b0 2024-04-25T08:53:48.772 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.779+0000 7f6010b9b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f600c09d140 0x7f600c14d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:48.772 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.779+0000 7f600bfff640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f600c0ad4b0 0x7f600c14c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:48.772 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.779+0000 7f6010b9b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f600c09d140 0x7f600c14d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:55206/0 (socket says 172.21.15.193:55206) 2024-04-25T08:53:48.772 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.779+0000 7f600bfff640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f600c0ad4b0 0x7f600c14c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.193:57248/0 (socket says 172.21.15.193:57248) 2024-04-25T08:53:48.772 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.779+0000 7f6010b9b640 1 -- 172.21.15.193:0/3835181299 learned_addr learned my addr 172.21.15.193:0/3835181299 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-25T08:53:48.772 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.779+0000 7f600b7fe640 1 --2- 172.21.15.193:0/3835181299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f600c0cfa00 0x7f600c09cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:48.772 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.779+0000 7f600bfff640 1 -- 172.21.15.193:0/3835181299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f600c09d140 msgr2=0x7f600c14d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:48.772 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.779+0000 7f600bfff640 1 --2- 172.21.15.193:0/3835181299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f600c09d140 0x7f600c14d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.772 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.779+0000 7f600bfff640 1 -- 172.21.15.193:0/3835181299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f600c0cfa00 msgr2=0x7f600c09cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:48.772 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.779+0000 7f600bfff640 1 --2- 172.21.15.193:0/3835181299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f600c0cfa00 0x7f600c09cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.773 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.779+0000 7f600bfff640 1 -- 172.21.15.193:0/3835181299 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f600c09bb60 con 0x7f600c0ad4b0 2024-04-25T08:53:48.773 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.779+0000 7f6010b9b640 1 --2- 172.21.15.193:0/3835181299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f600c09d140 0x7f600c14d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:53:48.773 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.779+0000 7f600bfff640 1 --2- 172.21.15.193:0/3835181299 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f600c0ad4b0 0x7f600c14c970 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc031000 tx=0x7f5ffc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:48.773 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.779+0000 7f6008ff9640 1 -- 172.21.15.193:0/3835181299 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc042070 con 0x7f600c0ad4b0 2024-04-25T08:53:48.773 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.780+0000 7f6008ff9640 1 -- 172.21.15.193:0/3835181299 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ffc004020 con 0x7f600c0ad4b0 2024-04-25T08:53:48.773 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.780+0000 7f6008ff9640 1 -- 172.21.15.193:0/3835181299 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc037040 con 0x7f600c0ad4b0 2024-04-25T08:53:48.773 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.780+0000 7f60125fe640 1 -- 172.21.15.193:0/3835181299 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f600c09be40 con 0x7f600c0ad4b0 2024-04-25T08:53:48.773 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.780+0000 7f60125fe640 1 -- 172.21.15.193:0/3835181299 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f600c09c400 con 0x7f600c0ad4b0 2024-04-25T08:53:48.773 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.780+0000 7f6008ff9640 1 -- 172.21.15.193:0/3835181299 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f5ffc0039a0 con 0x7f600c0ad4b0 2024-04-25T08:53:48.773 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.780+0000 7f6008ff9640 1 -- 172.21.15.193:0/3835181299 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f5ffc047450 con 0x7f600c0ad4b0 2024-04-25T08:53:48.773 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.781+0000 7f60125fe640 1 -- 172.21.15.193:0/3835181299 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f600c074f60 con 0x7f600c0ad4b0 2024-04-25T08:53:48.774 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.783+0000 7f6008ff9640 1 -- 172.21.15.193:0/3835181299 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f5ffc003e20 con 0x7f600c0ad4b0 2024-04-25T08:53:48.845 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.855+0000 7f60125fe640 1 -- 172.21.15.193:0/3835181299 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b34857e6-f4b8-4177-82ea-041f4bb1c42c", "id": 3} v 0) v1 -- 0x7f600c067180 con 0x7f600c0ad4b0 2024-04-25T08:53:48.848 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.858+0000 7f6008ff9640 1 -- 172.21.15.193:0/3835181299 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b34857e6-f4b8-4177-82ea-041f4bb1c42c", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5ffc004410 con 0x7f600c0ad4b0 2024-04-25T08:53:48.848 INFO:teuthology.orchestra.run.smithi193.stdout:3 2024-04-25T08:53:48.849 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.859+0000 7f60125fe640 1 -- 172.21.15.193:0/3835181299 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f600c0ad4b0 msgr2=0x7f600c14c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:48.850 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.859+0000 7f60125fe640 1 --2- 172.21.15.193:0/3835181299 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f600c0ad4b0 0x7f600c14c970 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc031000 tx=0x7f5ffc0046c0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.850 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.859+0000 7f60125fe640 1 -- 172.21.15.193:0/3835181299 shutdown_connections 2024-04-25T08:53:48.850 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.859+0000 7f60125fe640 1 --2- 172.21.15.193:0/3835181299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f600c09d140 0x7f600c14d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.850 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.859+0000 7f60125fe640 1 --2- 172.21.15.193:0/3835181299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f600c0cfa00 0x7f600c09cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.850 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.859+0000 7f60125fe640 1 --2- 172.21.15.193:0/3835181299 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f600c0ad4b0 0x7f600c14c970 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.850 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.859+0000 7f60125fe640 1 -- 172.21.15.193:0/3835181299 >> 172.21.15.193:0/3835181299 conn(0x7f600c06ab30 msgr2=0x7f600c06d830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:48.850 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.859+0000 7f60125fe640 1 -- 172.21.15.193:0/3835181299 shutdown_connections 2024-04-25T08:53:48.850 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.859+0000 7f60125fe640 1 -- 172.21.15.193:0/3835181299 wait complete. 2024-04-25T08:53:48.860 DEBUG:teuthology.orchestra.run.smithi193:> sudo ceph --cluster ceph osd new cbbc90d8-107f-4d0d-8a3a-38ccca6ff605 4 2024-04-25T08:53:48.913 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-25T08:53:48.923+0000 7f78d0b23040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T08:53:48.918 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-25T08:53:48.928+0000 7facf6922040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T08:53:48.958 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.968+0000 7f39489c8640 1 Processor -- start 2024-04-25T08:53:48.958 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.968+0000 7f39489c8640 1 -- start start 2024-04-25T08:53:48.959 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.969+0000 7f39489c8640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f39441584d0 0x7f3944072300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:48.959 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.969+0000 7f39489c8640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f394406b180 0x7f39440af6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:48.959 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.969+0000 7f39489c8640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f394415e450 con 0x7f394406ada0 2024-04-25T08:53:48.959 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.969+0000 7f39489c8640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39440afe00 con 0x7f394406b180 2024-04-25T08:53:48.959 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.969+0000 7f39489c8640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f394415e710 con 0x7f39441584d0 2024-04-25T08:53:48.959 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.969+0000 7f3941d9b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f39441584d0 0x7f3944072300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:48.960 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.969+0000 7f3942d9d640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f394406b180 0x7f39440af6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:48.960 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.969+0000 7f3941d9b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f39441584d0 0x7f3944072300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.193:57258/0 (socket says 172.21.15.193:57258) 2024-04-25T08:53:48.960 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.969+0000 7f3941d9b640 1 -- 172.21.15.193:0/512885887 learned_addr learned my addr 172.21.15.193:0/512885887 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-25T08:53:48.960 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.969+0000 7f3941d9b640 1 -- 172.21.15.193:0/512885887 >> v2:172.21.15.193:3300/0 conn(0x7f394406b180 msgr2=0x7f39440af6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:48.960 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.969+0000 7f3941d9b640 1 --2- 172.21.15.193:0/512885887 >> v2:172.21.15.193:3300/0 conn(0x7f394406b180 0x7f39440af6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.960 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.969+0000 7f3941d9b640 1 -- 172.21.15.193:0/512885887 >> v1:172.21.15.193:6789/0 conn(0x7f394406ada0 legacy=0x7f3944157dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:48.960 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.969+0000 7f3941d9b640 1 -- 172.21.15.193:0/512885887 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f394415e9f0 con 0x7f39441584d0 2024-04-25T08:53:48.960 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.969+0000 7f394159a640 1 -- 172.21.15.193:0/512885887 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1352233521 0 0) 0x7f394415e450 con 0x7f394406ada0 2024-04-25T08:53:48.960 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.969+0000 7f3942d9d640 1 --2- 172.21.15.193:0/512885887 >> v2:172.21.15.193:3300/0 conn(0x7f394406b180 0x7f39440af6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:53:48.961 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.970+0000 7f3941d9b640 1 --2- 172.21.15.193:0/512885887 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f39441584d0 0x7f3944072300 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3938009870 tx=0x7f393802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7d163494858197d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:48.961 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.970+0000 7f394159a640 1 -- 172.21.15.193:0/512885887 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3938037070 con 0x7f39441584d0 2024-04-25T08:53:48.961 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.970+0000 7f394159a640 1 -- 172.21.15.193:0/512885887 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3938032440 con 0x7f39441584d0 2024-04-25T08:53:48.962 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.970+0000 7f394159a640 1 -- 172.21.15.193:0/512885887 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3938032810 con 0x7f39441584d0 2024-04-25T08:53:48.962 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.971+0000 7f39489c8640 1 -- 172.21.15.193:0/512885887 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f39441584d0 msgr2=0x7f3944072300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:48.962 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.971+0000 7f39489c8640 1 --2- 172.21.15.193:0/512885887 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f39441584d0 0x7f3944072300 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3938009870 tx=0x7f393802a270 comp rx=0 tx=0).stop 2024-04-25T08:53:48.962 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.971+0000 7f39489c8640 1 -- 172.21.15.193:0/512885887 shutdown_connections 2024-04-25T08:53:48.962 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.971+0000 7f39489c8640 1 --2- 172.21.15.193:0/512885887 >> v2:172.21.15.193:3300/0 conn(0x7f394406b180 0x7f39440af6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.962 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.971+0000 7f39489c8640 1 --2- 172.21.15.193:0/512885887 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f39441584d0 0x7f3944072300 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.962 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.971+0000 7f39489c8640 1 -- 172.21.15.193:0/512885887 >> 172.21.15.193:0/512885887 conn(0x7f3944066d10 msgr2=0x7f394406bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:48.962 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.971+0000 7f39489c8640 1 -- 172.21.15.193:0/512885887 shutdown_connections 2024-04-25T08:53:48.962 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.971+0000 7f39489c8640 1 -- 172.21.15.193:0/512885887 wait complete. 2024-04-25T08:53:48.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.972+0000 7f39489c8640 1 Processor -- start 2024-04-25T08:53:48.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.972+0000 7f39489c8640 1 -- start start 2024-04-25T08:53:48.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.973+0000 7f39489c8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f394406ada0 0x7f3944144340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:48.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.973+0000 7f39489c8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f394406b180 0x7f39441588f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:48.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.973+0000 7f39489c8640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f39441584d0 0x7f3944158e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:48.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.973+0000 7f39489c8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39440d0380 con 0x7f394406b180 2024-04-25T08:53:48.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.973+0000 7f39489c8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394415f120 con 0x7f394406ada0 2024-04-25T08:53:48.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.973+0000 7f39489c8640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39440d0500 con 0x7f39441584d0 2024-04-25T08:53:48.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.973+0000 7f3942d9d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f39441584d0 0x7f3944158e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:48.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.973+0000 7f394259c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f394406ada0 0x7f3944144340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:48.965 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.973+0000 7f3942d9d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f39441584d0 0x7f3944158e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.193:57264/0 (socket says 172.21.15.193:57264) 2024-04-25T08:53:48.966 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.973+0000 7f3942d9d640 1 -- 172.21.15.193:0/2400658654 learned_addr learned my addr 172.21.15.193:0/2400658654 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-25T08:53:48.966 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.973+0000 7f3941d9b640 1 --2- 172.21.15.193:0/2400658654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f394406b180 0x7f39441588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:48.966 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.973+0000 7f394259c640 1 -- 172.21.15.193:0/2400658654 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f39441584d0 msgr2=0x7f3944158e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:48.966 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.973+0000 7f394259c640 1 --2- 172.21.15.193:0/2400658654 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f39441584d0 0x7f3944158e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.966 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.973+0000 7f394259c640 1 -- 172.21.15.193:0/2400658654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f394406b180 msgr2=0x7f39441588f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:48.966 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.973+0000 7f394259c640 1 --2- 172.21.15.193:0/2400658654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f394406b180 0x7f39441588f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:48.966 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.973+0000 7f394259c640 1 -- 172.21.15.193:0/2400658654 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3944144880 con 0x7f394406ada0 2024-04-25T08:53:48.966 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.973+0000 7f3942d9d640 1 --2- 172.21.15.193:0/2400658654 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f39441584d0 0x7f3944158e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:53:48.966 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.973+0000 7f394259c640 1 --2- 172.21.15.193:0/2400658654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f394406ada0 0x7f3944144340 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f393400c480 tx=0x7f393400c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:48.966 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.974+0000 7f3932ffd640 1 -- 172.21.15.193:0/2400658654 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3934017070 con 0x7f394406ada0 2024-04-25T08:53:48.966 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.974+0000 7f3932ffd640 1 -- 172.21.15.193:0/2400658654 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3934005480 con 0x7f394406ada0 2024-04-25T08:53:48.966 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.974+0000 7f39489c8640 1 -- 172.21.15.193:0/2400658654 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39440cec30 con 0x7f394406ada0 2024-04-25T08:53:48.966 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.974+0000 7f3932ffd640 1 -- 172.21.15.193:0/2400658654 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3934003830 con 0x7f394406ada0 2024-04-25T08:53:48.966 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.974+0000 7f39489c8640 1 -- 172.21.15.193:0/2400658654 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39440cf1f0 con 0x7f394406ada0 2024-04-25T08:53:48.967 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.974+0000 7f3932ffd640 1 -- 172.21.15.193:0/2400658654 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f393401a070 con 0x7f394406ada0 2024-04-25T08:53:48.967 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.974+0000 7f3932ffd640 1 -- 172.21.15.193:0/2400658654 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f3934012910 con 0x7f394406ada0 2024-04-25T08:53:48.967 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.975+0000 7f39489c8640 1 -- 172.21.15.193:0/2400658654 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3944074f60 con 0x7f394406ada0 2024-04-25T08:53:48.968 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:48.978+0000 7f3932ffd640 1 -- 172.21.15.193:0/2400658654 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f3934021070 con 0x7f394406ada0 2024-04-25T08:53:49.005 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-25T08:53:49.015+0000 7f78d0b23040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T08:53:49.011 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-25T08:53:49.021+0000 7facf6922040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T08:53:49.040 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.051+0000 7f39489c8640 1 -- 172.21.15.193:0/2400658654 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd new", "uuid": "cbbc90d8-107f-4d0d-8a3a-38ccca6ff605", "id": 4} v 0) v1 -- 0x7f3944162720 con 0x7f394406ada0 2024-04-25T08:53:49.044 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.054+0000 7f3932ffd640 1 -- 172.21.15.193:0/2400658654 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cbbc90d8-107f-4d0d-8a3a-38ccca6ff605", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3934012d40 con 0x7f394406ada0 2024-04-25T08:53:49.044 INFO:teuthology.orchestra.run.smithi193.stdout:4 2024-04-25T08:53:49.045 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.055+0000 7f39489c8640 1 -- 172.21.15.193:0/2400658654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f394406ada0 msgr2=0x7f3944144340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:49.045 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.055+0000 7f39489c8640 1 --2- 172.21.15.193:0/2400658654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f394406ada0 0x7f3944144340 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f393400c480 tx=0x7f393400c940 comp rx=0 tx=0).stop 2024-04-25T08:53:49.045 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.055+0000 7f39489c8640 1 -- 172.21.15.193:0/2400658654 shutdown_connections 2024-04-25T08:53:49.045 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.055+0000 7f39489c8640 1 --2- 172.21.15.193:0/2400658654 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f39441584d0 0x7f3944158e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.046 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.055+0000 7f39489c8640 1 --2- 172.21.15.193:0/2400658654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f394406b180 0x7f39441588f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.046 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.055+0000 7f39489c8640 1 --2- 172.21.15.193:0/2400658654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f394406ada0 0x7f3944144340 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.046 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.055+0000 7f39489c8640 1 -- 172.21.15.193:0/2400658654 >> 172.21.15.193:0/2400658654 conn(0x7f3944066d10 msgr2=0x7f39440d0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:49.046 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.055+0000 7f39489c8640 1 -- 172.21.15.193:0/2400658654 shutdown_connections 2024-04-25T08:53:49.046 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.055+0000 7f39489c8640 1 -- 172.21.15.193:0/2400658654 wait complete. 2024-04-25T08:53:49.050 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-25T08:53:49.060+0000 7f78d0b23040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T08:53:49.056 DEBUG:teuthology.orchestra.run.smithi193:> sudo ceph --cluster ceph osd new 69190f87-1f31-4ab9-b6cb-f1904c0ec57a 5 2024-04-25T08:53:49.058 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-25T08:53:49.067+0000 7facf6922040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T08:53:49.155 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.165+0000 7f2c08ce1640 1 Processor -- start 2024-04-25T08:53:49.155 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.165+0000 7f2c08ce1640 1 -- start start 2024-04-25T08:53:49.157 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.166+0000 7f2c08ce1640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c0415e020 0x7f2c04162550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:49.157 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.166+0000 7f2c08ce1640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f2c040ce7e0 0x7f2c040d0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:49.157 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.166+0000 7f2c08ce1640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c040b47c0 con 0x7f2c0415dc40 2024-04-25T08:53:49.158 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.166+0000 7f2c08ce1640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c04071e80 con 0x7f2c040ce7e0 2024-04-25T08:53:49.158 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.166+0000 7f2c08ce1640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c04072160 con 0x7f2c0415e020 2024-04-25T08:53:49.158 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.166+0000 7f2bfbfff640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c0415e020 0x7f2c04162550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:49.158 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.166+0000 7f2c02d9d640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f2c040ce7e0 0x7f2c040d0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:49.158 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.166+0000 7f2bfbfff640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c0415e020 0x7f2c04162550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.193:57274/0 (socket says 172.21.15.193:57274) 2024-04-25T08:53:49.158 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.166+0000 7f2bfbfff640 1 -- 172.21.15.193:0/1745372743 learned_addr learned my addr 172.21.15.193:0/1745372743 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-25T08:53:49.158 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.166+0000 7f2bfbfff640 1 -- 172.21.15.193:0/1745372743 >> v2:172.21.15.193:3300/0 conn(0x7f2c040ce7e0 msgr2=0x7f2c040d0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:49.158 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.166+0000 7f2bfbfff640 1 --2- 172.21.15.193:0/1745372743 >> v2:172.21.15.193:3300/0 conn(0x7f2c040ce7e0 0x7f2c040d0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.158 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.166+0000 7f2bfbfff640 1 -- 172.21.15.193:0/1745372743 >> v1:172.21.15.43:6789/0 conn(0x7f2c0415dc40 legacy=0x7f2c04071670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:49.158 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.166+0000 7f2bfbfff640 1 -- 172.21.15.193:0/1745372743 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c04072440 con 0x7f2c0415e020 2024-04-25T08:53:49.159 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.167+0000 7f2bfbfff640 1 --2- 172.21.15.193:0/1745372743 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c0415e020 0x7f2c04162550 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2bec009870 tx=0x7f2bec02a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab6feee7d59e841b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:49.159 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.167+0000 7f2c01d9b640 1 -- 172.21.15.193:0/1745372743 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bec037070 con 0x7f2c0415e020 2024-04-25T08:53:49.159 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.167+0000 7f2c01d9b640 1 -- 172.21.15.193:0/1745372743 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bec032440 con 0x7f2c0415e020 2024-04-25T08:53:49.159 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.167+0000 7f2c01d9b640 1 -- 172.21.15.193:0/1745372743 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bec032810 con 0x7f2c0415e020 2024-04-25T08:53:49.162 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.167+0000 7f2c08ce1640 1 -- 172.21.15.193:0/1745372743 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c0415e020 msgr2=0x7f2c04162550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:49.163 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.167+0000 7f2c08ce1640 1 --2- 172.21.15.193:0/1745372743 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c0415e020 0x7f2c04162550 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f2bec009870 tx=0x7f2bec02a250 comp rx=0 tx=0).stop 2024-04-25T08:53:49.163 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.168+0000 7f2c08ce1640 1 -- 172.21.15.193:0/1745372743 shutdown_connections 2024-04-25T08:53:49.163 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.168+0000 7f2c08ce1640 1 --2- 172.21.15.193:0/1745372743 >> v2:172.21.15.193:3300/0 conn(0x7f2c040ce7e0 0x7f2c040d0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.163 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.168+0000 7f2c08ce1640 1 --2- 172.21.15.193:0/1745372743 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c0415e020 0x7f2c04162550 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.163 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.168+0000 7f2c08ce1640 1 -- 172.21.15.193:0/1745372743 >> 172.21.15.193:0/1745372743 conn(0x7f2c0406c060 msgr2=0x7f2c0406c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:49.163 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.168+0000 7f2c08ce1640 1 -- 172.21.15.193:0/1745372743 shutdown_connections 2024-04-25T08:53:49.163 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.168+0000 7f2c08ce1640 1 -- 172.21.15.193:0/1745372743 wait complete. 2024-04-25T08:53:49.163 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.169+0000 7f2c08ce1640 1 Processor -- start 2024-04-25T08:53:49.163 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.169+0000 7f2c08ce1640 1 -- start start 2024-04-25T08:53:49.163 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.170+0000 7f2c08ce1640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c040ce7e0 0x7f2c04147bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:49.163 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.170+0000 7f2c08ce1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c0415dc40 0x7f2c04146e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:49.163 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.170+0000 7f2c08ce1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c0415e020 0x7f2c0415c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:49.164 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.170+0000 7f2c08ce1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c040d1030 con 0x7f2c0415dc40 2024-04-25T08:53:49.164 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.170+0000 7f2c08ce1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c040cec90 con 0x7f2c0415e020 2024-04-25T08:53:49.164 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.170+0000 7f2c08ce1640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c040d11b0 con 0x7f2c040ce7e0 2024-04-25T08:53:49.164 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.170+0000 7f2c0259c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c040ce7e0 0x7f2c04147bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:49.164 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.170+0000 7f2c02d9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c0415e020 0x7f2c0415c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:49.164 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.170+0000 7f2c0259c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c040ce7e0 0x7f2c04147bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.193:57280/0 (socket says 172.21.15.193:57280) 2024-04-25T08:53:49.164 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.170+0000 7f2c0259c640 1 -- 172.21.15.193:0/1198570666 learned_addr learned my addr 172.21.15.193:0/1198570666 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-25T08:53:49.164 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.170+0000 7f2c02d9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c0415e020 0x7f2c0415c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:55236/0 (socket says 172.21.15.193:55236) 2024-04-25T08:53:49.164 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.170+0000 7f2bfbfff640 1 --2- 172.21.15.193:0/1198570666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c0415dc40 0x7f2c04146e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:49.164 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.170+0000 7f2c0259c640 1 -- 172.21.15.193:0/1198570666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c0415e020 msgr2=0x7f2c0415c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:49.164 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.170+0000 7f2c0259c640 1 --2- 172.21.15.193:0/1198570666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c0415e020 0x7f2c0415c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.164 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.170+0000 7f2c0259c640 1 -- 172.21.15.193:0/1198570666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c0415dc40 msgr2=0x7f2c04146e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:49.164 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.170+0000 7f2c0259c640 1 --2- 172.21.15.193:0/1198570666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c0415dc40 0x7f2c04146e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.165 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.170+0000 7f2c0259c640 1 -- 172.21.15.193:0/1198570666 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c0406e550 con 0x7f2c040ce7e0 2024-04-25T08:53:49.165 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.170+0000 7f2c02d9d640 1 --2- 172.21.15.193:0/1198570666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c0415e020 0x7f2c0415c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:53:49.165 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.170+0000 7f2c0259c640 1 --2- 172.21.15.193:0/1198570666 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c040ce7e0 0x7f2c04147bc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2bf400c400 tx=0x7f2bf400c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:49.165 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.170+0000 7f2bfaffd640 1 -- 172.21.15.193:0/1198570666 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf40052d0 con 0x7f2c040ce7e0 2024-04-25T08:53:49.165 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.170+0000 7f2bfaffd640 1 -- 172.21.15.193:0/1198570666 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf4005490 con 0x7f2c040ce7e0 2024-04-25T08:53:49.165 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.170+0000 7f2c08ce1640 1 -- 172.21.15.193:0/1198570666 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c0406e830 con 0x7f2c040ce7e0 2024-04-25T08:53:49.165 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.170+0000 7f2bfaffd640 1 -- 172.21.15.193:0/1198570666 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf4007260 con 0x7f2c040ce7e0 2024-04-25T08:53:49.165 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.171+0000 7f2c08ce1640 1 -- 172.21.15.193:0/1198570666 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c0406edf0 con 0x7f2c040ce7e0 2024-04-25T08:53:49.165 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.171+0000 7f2bfaffd640 1 -- 172.21.15.193:0/1198570666 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f2bf4007400 con 0x7f2c040ce7e0 2024-04-25T08:53:49.165 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.171+0000 7f2bfaffd640 1 -- 172.21.15.193:0/1198570666 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f2bf4011590 con 0x7f2c040ce7e0 2024-04-25T08:53:49.165 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.171+0000 7f2c08ce1640 1 -- 172.21.15.193:0/1198570666 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c04074f60 con 0x7f2c040ce7e0 2024-04-25T08:53:49.166 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.173+0000 7f2bfaffd640 1 -- 172.21.15.193:0/1198570666 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f2bf40035f0 con 0x7f2c040ce7e0 2024-04-25T08:53:49.223 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-25T08:53:49.233+0000 7f78d0b23040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T08:53:49.230 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-25T08:53:49.240+0000 7facf6922040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T08:53:49.235 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.245+0000 7f2c08ce1640 1 -- 172.21.15.193:0/1198570666 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "osd new", "uuid": "69190f87-1f31-4ab9-b6cb-f1904c0ec57a", "id": 5} v 0) v1 -- 0x7f2c040ce0a0 con 0x7f2c040ce7e0 2024-04-25T08:53:49.238 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.248+0000 7f2bfaffd640 1 -- 172.21.15.193:0/1198570666 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "69190f87-1f31-4ab9-b6cb-f1904c0ec57a", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2bf4003a60 con 0x7f2c040ce7e0 2024-04-25T08:53:49.238 INFO:teuthology.orchestra.run.smithi193.stdout:5 2024-04-25T08:53:49.239 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.249+0000 7f2c08ce1640 1 -- 172.21.15.193:0/1198570666 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c040ce7e0 msgr2=0x7f2c04147bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:49.240 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.249+0000 7f2c08ce1640 1 --2- 172.21.15.193:0/1198570666 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c040ce7e0 0x7f2c04147bc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2bf400c400 tx=0x7f2bf400c8c0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.240 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.249+0000 7f2c08ce1640 1 -- 172.21.15.193:0/1198570666 shutdown_connections 2024-04-25T08:53:49.240 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.249+0000 7f2c08ce1640 1 --2- 172.21.15.193:0/1198570666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c0415e020 0x7f2c0415c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.240 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.249+0000 7f2c08ce1640 1 --2- 172.21.15.193:0/1198570666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c0415dc40 0x7f2c04146e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.240 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.249+0000 7f2c08ce1640 1 --2- 172.21.15.193:0/1198570666 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c040ce7e0 0x7f2c04147bc0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.240 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.249+0000 7f2c08ce1640 1 -- 172.21.15.193:0/1198570666 >> 172.21.15.193:0/1198570666 conn(0x7f2c0406c060 msgr2=0x7f2c040add30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:49.240 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.249+0000 7f2c08ce1640 1 -- 172.21.15.193:0/1198570666 shutdown_connections 2024-04-25T08:53:49.240 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.249+0000 7f2c08ce1640 1 -- 172.21.15.193:0/1198570666 wait complete. 2024-04-25T08:53:49.250 DEBUG:teuthology.orchestra.run.smithi193:> sudo ceph --cluster ceph osd new 74c61ad4-7316-45da-9aa6-2cdbb3135ae3 6 2024-04-25T08:53:49.273 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-25T08:53:49.283+0000 7f78d0b23040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T08:53:49.288 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-25T08:53:49.299+0000 7facf6922040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T08:53:49.327 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-25T08:53:49.337+0000 7f78d0b23040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T08:53:49.343 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-25T08:53:49.353+0000 7facf6922040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T08:53:49.347 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.357+0000 7fd19e7a9640 1 Processor -- start 2024-04-25T08:53:49.347 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.357+0000 7fd19e7a9640 1 -- start start 2024-04-25T08:53:49.348 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.358+0000 7fd19e7a9640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd19815adf0 0x7fd19815cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:49.348 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.358+0000 7fd19e7a9640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd19815bfc0 con 0x7fd1980b4030 2024-04-25T08:53:49.348 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.358+0000 7fd19e7a9640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd19815b400 con 0x7fd198071460 2024-04-25T08:53:49.349 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.358+0000 7fd19e7a9640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd19815b6e0 con 0x7fd19815adf0 2024-04-25T08:53:49.349 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.358+0000 7fd1977fe640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7fd198071460 0x7fd19815c4c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.193:40268/0 (socket says 172.21.15.193:40268) 2024-04-25T08:53:49.349 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.358+0000 7fd1977fe640 1 -- 172.21.15.193:0/2371064407 learned_addr learned my addr 172.21.15.193:0/2371064407 (peer_addr_for_me v1:172.21.15.193:0/0) 2024-04-25T08:53:49.349 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.358+0000 7fd19cd46640 1 --2- 172.21.15.193:0/2371064407 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd19815adf0 0x7fd19815cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:49.349 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.358+0000 7fd196ffd640 1 -- 172.21.15.193:0/2371064407 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2979357817 0 0) 0x7fd19815b400 con 0x7fd198071460 2024-04-25T08:53:49.349 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.358+0000 7fd196ffd640 1 -- 172.21.15.193:0/2371064407 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd180003520 con 0x7fd198071460 2024-04-25T08:53:49.349 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.358+0000 7fd19cd46640 1 -- 172.21.15.193:0/2371064407 >> v1:172.21.15.193:6789/0 conn(0x7fd198071460 legacy=0x7fd19815c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:49.349 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.358+0000 7fd19cd46640 1 -- 172.21.15.193:0/2371064407 >> v1:172.21.15.43:6789/0 conn(0x7fd1980b4030 legacy=0x7fd1980b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:49.349 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.358+0000 7fd19cd46640 1 -- 172.21.15.193:0/2371064407 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd19815b9c0 con 0x7fd19815adf0 2024-04-25T08:53:49.350 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.359+0000 7fd19cd46640 1 --2- 172.21.15.193:0/2371064407 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd19815adf0 0x7fd19815cbd0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd190009dc0 tx=0x7fd19002a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdf3534a6bf80311 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:49.351 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.359+0000 7fd196ffd640 1 -- 172.21.15.193:0/2371064407 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd190007740 con 0x7fd19815adf0 2024-04-25T08:53:49.351 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.359+0000 7fd196ffd640 1 -- 172.21.15.193:0/2371064407 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1900078e0 con 0x7fd19815adf0 2024-04-25T08:53:49.351 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.359+0000 7fd196ffd640 1 -- 172.21.15.193:0/2371064407 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd190007cb0 con 0x7fd19815adf0 2024-04-25T08:53:49.351 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.360+0000 7fd19e7a9640 1 -- 172.21.15.193:0/2371064407 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd19815adf0 msgr2=0x7fd19815cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:49.351 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.360+0000 7fd19e7a9640 1 --2- 172.21.15.193:0/2371064407 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd19815adf0 0x7fd19815cbd0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd190009dc0 tx=0x7fd19002a6d0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.351 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.360+0000 7fd19e7a9640 1 -- 172.21.15.193:0/2371064407 shutdown_connections 2024-04-25T08:53:49.351 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.360+0000 7fd19e7a9640 1 --2- 172.21.15.193:0/2371064407 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd19815adf0 0x7fd19815cbd0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.351 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.360+0000 7fd19e7a9640 1 -- 172.21.15.193:0/2371064407 >> 172.21.15.193:0/2371064407 conn(0x7fd198066d10 msgr2=0x7fd19806bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:49.351 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.360+0000 7fd19e7a9640 1 -- 172.21.15.193:0/2371064407 shutdown_connections 2024-04-25T08:53:49.351 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.360+0000 7fd19e7a9640 1 -- 172.21.15.193:0/2371064407 wait complete. 2024-04-25T08:53:49.352 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.362+0000 7fd19e7a9640 1 Processor -- start 2024-04-25T08:53:49.352 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.362+0000 7fd19e7a9640 1 -- start start 2024-04-25T08:53:49.355 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.363+0000 7fd19e7a9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd198071460 0x7fd198148240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:49.355 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.363+0000 7fd19e7a9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1980b4030 0x7fd1981552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:49.355 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.363+0000 7fd19e7a9640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd19815adf0 0x7fd198155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:49.355 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.363+0000 7fd19e7a9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1980b0010 con 0x7fd1980b4030 2024-04-25T08:53:49.355 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.363+0000 7fd19e7a9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1980afe90 con 0x7fd198071460 2024-04-25T08:53:49.355 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.363+0000 7fd19e7a9640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1980b4d10 con 0x7fd19815adf0 2024-04-25T08:53:49.356 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.363+0000 7fd197fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd198071460 0x7fd198148240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:49.356 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.363+0000 7fd19cd46640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd19815adf0 0x7fd198155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:49.356 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.363+0000 7fd197fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd198071460 0x7fd198148240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.193:55250/0 (socket says 172.21.15.193:55250) 2024-04-25T08:53:49.356 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.363+0000 7fd197fff640 1 -- 172.21.15.193:0/3618938919 learned_addr learned my addr 172.21.15.193:0/3618938919 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-25T08:53:49.356 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.364+0000 7fd1977fe640 1 --2- 172.21.15.193:0/3618938919 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1980b4030 0x7fd1981552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:49.356 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.364+0000 7fd19cd46640 1 -- 172.21.15.193:0/3618938919 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd198071460 msgr2=0x7fd198148240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:49.356 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.364+0000 7fd19cd46640 1 --2- 172.21.15.193:0/3618938919 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd198071460 0x7fd198148240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.356 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.364+0000 7fd19cd46640 1 -- 172.21.15.193:0/3618938919 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1980b4030 msgr2=0x7fd1981552d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:49.356 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.364+0000 7fd19cd46640 1 --2- 172.21.15.193:0/3618938919 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1980b4030 0x7fd1981552d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.356 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.364+0000 7fd19cd46640 1 -- 172.21.15.193:0/3618938919 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd198156440 con 0x7fd19815adf0 2024-04-25T08:53:49.356 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.364+0000 7fd197fff640 1 --2- 172.21.15.193:0/3618938919 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd198071460 0x7fd198148240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:53:49.356 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.364+0000 7fd19cd46640 1 --2- 172.21.15.193:0/3618938919 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd19815adf0 0x7fd198155810 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd190031000 tx=0x7fd190004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:49.357 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.364+0000 7fd194ff9640 1 -- 172.21.15.193:0/3618938919 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd19003f070 con 0x7fd19815adf0 2024-04-25T08:53:49.357 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.364+0000 7fd194ff9640 1 -- 172.21.15.193:0/3618938919 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd190031bc0 con 0x7fd19815adf0 2024-04-25T08:53:49.357 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.364+0000 7fd19e7a9640 1 -- 172.21.15.193:0/3618938919 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd198156720 con 0x7fd19815adf0 2024-04-25T08:53:49.357 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.364+0000 7fd194ff9640 1 -- 172.21.15.193:0/3618938919 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd190009450 con 0x7fd19815adf0 2024-04-25T08:53:49.357 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.364+0000 7fd19e7a9640 1 -- 172.21.15.193:0/3618938919 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd198156ce0 con 0x7fd19815adf0 2024-04-25T08:53:49.357 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.364+0000 7fd194ff9640 1 -- 172.21.15.193:0/3618938919 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fd190044460 con 0x7fd19815adf0 2024-04-25T08:53:49.357 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.364+0000 7fd194ff9640 1 -- 172.21.15.193:0/3618938919 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fd19003ba00 con 0x7fd19815adf0 2024-04-25T08:53:49.357 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.365+0000 7fd19e7a9640 1 -- 172.21.15.193:0/3618938919 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd198074f60 con 0x7fd19815adf0 2024-04-25T08:53:49.357 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.367+0000 7fd194ff9640 1 -- 172.21.15.193:0/3618938919 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fd190063dd0 con 0x7fd19815adf0 2024-04-25T08:53:49.429 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.439+0000 7fd19e7a9640 1 -- 172.21.15.193:0/3618938919 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "osd new", "uuid": "74c61ad4-7316-45da-9aa6-2cdbb3135ae3", "id": 6} v 0) v1 -- 0x7fd1980b1ea0 con 0x7fd19815adf0 2024-04-25T08:53:49.432 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.443+0000 7fd194ff9640 1 -- 172.21.15.193:0/3618938919 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "74c61ad4-7316-45da-9aa6-2cdbb3135ae3", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd190052070 con 0x7fd19815adf0 2024-04-25T08:53:49.433 INFO:teuthology.orchestra.run.smithi193.stdout:6 2024-04-25T08:53:49.434 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.443+0000 7fd19e7a9640 1 -- 172.21.15.193:0/3618938919 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd19815adf0 msgr2=0x7fd198155810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:49.434 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.443+0000 7fd19e7a9640 1 --2- 172.21.15.193:0/3618938919 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd19815adf0 0x7fd198155810 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd190031000 tx=0x7fd190004a40 comp rx=0 tx=0).stop 2024-04-25T08:53:49.434 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.443+0000 7fd19e7a9640 1 -- 172.21.15.193:0/3618938919 shutdown_connections 2024-04-25T08:53:49.434 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.443+0000 7fd19e7a9640 1 --2- 172.21.15.193:0/3618938919 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd19815adf0 0x7fd198155810 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.435 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.443+0000 7fd19e7a9640 1 --2- 172.21.15.193:0/3618938919 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1980b4030 0x7fd1981552d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.435 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.443+0000 7fd19e7a9640 1 --2- 172.21.15.193:0/3618938919 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd198071460 0x7fd198148240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.435 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.443+0000 7fd19e7a9640 1 -- 172.21.15.193:0/3618938919 >> 172.21.15.193:0/3618938919 conn(0x7fd198066d10 msgr2=0x7fd19806e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:49.435 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.444+0000 7fd19e7a9640 1 -- 172.21.15.193:0/3618938919 shutdown_connections 2024-04-25T08:53:49.435 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.444+0000 7fd19e7a9640 1 -- 172.21.15.193:0/3618938919 wait complete. 2024-04-25T08:53:49.444 DEBUG:teuthology.orchestra.run.smithi193:> sudo ceph --cluster ceph osd new 5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7 7 2024-04-25T08:53:49.445 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-25T08:53:49.455+0000 7f78d0b23040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T08:53:49.463 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-25T08:53:49.473+0000 7facf6922040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T08:53:49.560 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.570+0000 7f61bb9a5640 1 Processor -- start 2024-04-25T08:53:49.560 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.570+0000 7f61bb9a5640 1 -- start start 2024-04-25T08:53:49.563 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.571+0000 7f61bb9a5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f61b40b4030 0x7f61b415adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:49.564 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.571+0000 7f61bb9a5640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f61b4071460 0x7f61b40b4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:49.564 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.571+0000 7f61bb9a5640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61b40b4950 con 0x7f61b406df70 2024-04-25T08:53:49.564 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.571+0000 7f61bb9a5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61b415bfc0 con 0x7f61b40b4030 2024-04-25T08:53:49.564 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.571+0000 7f61bb9a5640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61b415b400 con 0x7f61b4071460 2024-04-25T08:53:49.564 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.571+0000 7f61b8f40640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f61b4071460 0x7f61b40b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:49.565 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.571+0000 7f61b9f42640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f61b406df70 0x7f61b406e400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.193:40280/0 (socket says 172.21.15.193:40280) 2024-04-25T08:53:49.565 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.571+0000 7f61b9f42640 1 -- 172.21.15.193:0/1799149603 learned_addr learned my addr 172.21.15.193:0/1799149603 (peer_addr_for_me v1:172.21.15.193:0/0) 2024-04-25T08:53:49.565 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.571+0000 7f61b9741640 1 --2- 172.21.15.193:0/1799149603 >> v2:172.21.15.43:3300/0 conn(0x7f61b40b4030 0x7f61b415adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:49.565 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.571+0000 7f61b8f40640 1 -- 172.21.15.193:0/1799149603 >> v2:172.21.15.43:3300/0 conn(0x7f61b40b4030 msgr2=0x7f61b415adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:49.565 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.571+0000 7f61b8f40640 1 --2- 172.21.15.193:0/1799149603 >> v2:172.21.15.43:3300/0 conn(0x7f61b40b4030 0x7f61b415adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.566 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.571+0000 7f61b8f40640 1 -- 172.21.15.193:0/1799149603 >> v1:172.21.15.193:6789/0 conn(0x7f61b406df70 legacy=0x7f61b406e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:49.566 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.571+0000 7f61b8f40640 1 -- 172.21.15.193:0/1799149603 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b415b6e0 con 0x7f61b4071460 2024-04-25T08:53:49.566 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.571+0000 7f61abfff640 1 -- 172.21.15.193:0/1799149603 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4212926080 0 0) 0x7f61b40b4950 con 0x7f61b406df70 2024-04-25T08:53:49.566 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.572+0000 7f61b8f40640 1 --2- 172.21.15.193:0/1799149603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61b4071460 0x7f61b40b4410 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f61b406c6a0 tx=0x7f61a402a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17446b2ebca0a3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:49.566 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.572+0000 7f61abfff640 1 -- 172.21.15.193:0/1799149603 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a4037070 con 0x7f61b4071460 2024-04-25T08:53:49.566 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.572+0000 7f61abfff640 1 -- 172.21.15.193:0/1799149603 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a4032440 con 0x7f61b4071460 2024-04-25T08:53:49.566 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.572+0000 7f61abfff640 1 -- 172.21.15.193:0/1799149603 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a4032810 con 0x7f61b4071460 2024-04-25T08:53:49.566 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.572+0000 7f61bb9a5640 1 -- 172.21.15.193:0/1799149603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61b4071460 msgr2=0x7f61b40b4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:49.566 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.572+0000 7f61bb9a5640 1 --2- 172.21.15.193:0/1799149603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61b4071460 0x7f61b40b4410 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f61b406c6a0 tx=0x7f61a402a1a0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.566 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.573+0000 7f61bb9a5640 1 -- 172.21.15.193:0/1799149603 shutdown_connections 2024-04-25T08:53:49.566 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.573+0000 7f61bb9a5640 1 --2- 172.21.15.193:0/1799149603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61b4071460 0x7f61b40b4410 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.566 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.573+0000 7f61bb9a5640 1 --2- 172.21.15.193:0/1799149603 >> v2:172.21.15.43:3300/0 conn(0x7f61b40b4030 0x7f61b415adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.566 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.573+0000 7f61bb9a5640 1 -- 172.21.15.193:0/1799149603 >> 172.21.15.193:0/1799149603 conn(0x7f61b4066d10 msgr2=0x7f61b406bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:49.567 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.573+0000 7f61bb9a5640 1 -- 172.21.15.193:0/1799149603 shutdown_connections 2024-04-25T08:53:49.567 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.573+0000 7f61bb9a5640 1 -- 172.21.15.193:0/1799149603 wait complete. 2024-04-25T08:53:49.567 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.574+0000 7f61bb9a5640 1 Processor -- start 2024-04-25T08:53:49.567 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.574+0000 7f61bb9a5640 1 -- start start 2024-04-25T08:53:49.569 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.575+0000 7f61bb9a5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61b406df70 0x7f61b4149300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:49.569 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.575+0000 7f61bb9a5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f61b4071460 0x7f61b40d0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:49.569 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.575+0000 7f61bb9a5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61b40b4030 0x7f61b41545b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:49.569 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.575+0000 7f61bb9a5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b40afe90 con 0x7f61b406df70 2024-04-25T08:53:49.570 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.575+0000 7f61bb9a5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b415bca0 con 0x7f61b40b4030 2024-04-25T08:53:49.570 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.575+0000 7f61bb9a5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61b40b0010 con 0x7f61b4071460 2024-04-25T08:53:49.570 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.575+0000 7f61b8f40640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f61b4071460 0x7f61b40d0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:49.570 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.575+0000 7f61b9f42640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61b40b4030 0x7f61b41545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:49.570 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.575+0000 7f61b8f40640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f61b4071460 0x7f61b40d0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.193:57302/0 (socket says 172.21.15.193:57302) 2024-04-25T08:53:49.570 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.575+0000 7f61b8f40640 1 -- 172.21.15.193:0/1971925114 learned_addr learned my addr 172.21.15.193:0/1971925114 (peer_addr_for_me v2:172.21.15.193:0/0) 2024-04-25T08:53:49.570 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.575+0000 7f61b9741640 1 --2- 172.21.15.193:0/1971925114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61b406df70 0x7f61b4149300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:49.570 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.575+0000 7f61b8f40640 1 -- 172.21.15.193:0/1971925114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61b40b4030 msgr2=0x7f61b41545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:49.570 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.575+0000 7f61b8f40640 1 --2- 172.21.15.193:0/1971925114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61b40b4030 0x7f61b41545b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.570 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.575+0000 7f61b8f40640 1 -- 172.21.15.193:0/1971925114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61b406df70 msgr2=0x7f61b4149300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:49.570 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.575+0000 7f61b8f40640 1 --2- 172.21.15.193:0/1971925114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61b406df70 0x7f61b4149300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.570 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.575+0000 7f61b8f40640 1 -- 172.21.15.193:0/1971925114 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b40cf370 con 0x7f61b4071460 2024-04-25T08:53:49.570 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.575+0000 7f61b9f42640 1 --2- 172.21.15.193:0/1971925114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61b40b4030 0x7f61b41545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:53:49.570 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.575+0000 7f61b8f40640 1 --2- 172.21.15.193:0/1971925114 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f61b4071460 0x7f61b40d0510 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f61a4035040 tx=0x7f61a40046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:49.571 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.575+0000 7f61a9ffb640 1 -- 172.21.15.193:0/1971925114 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a4037050 con 0x7f61b4071460 2024-04-25T08:53:49.571 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.575+0000 7f61a9ffb640 1 -- 172.21.15.193:0/1971925114 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a4038040 con 0x7f61b4071460 2024-04-25T08:53:49.571 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.575+0000 7f61bb9a5640 1 -- 172.21.15.193:0/1971925114 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61b40cf650 con 0x7f61b4071460 2024-04-25T08:53:49.571 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.575+0000 7f61a9ffb640 1 -- 172.21.15.193:0/1971925114 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a4004450 con 0x7f61b4071460 2024-04-25T08:53:49.571 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.576+0000 7f61bb9a5640 1 -- 172.21.15.193:0/1971925114 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61b40ad980 con 0x7f61b4071460 2024-04-25T08:53:49.571 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.576+0000 7f61a9ffb640 1 -- 172.21.15.193:0/1971925114 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f61a403c6d0 con 0x7f61b4071460 2024-04-25T08:53:49.571 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.576+0000 7f61a9ffb640 1 -- 172.21.15.193:0/1971925114 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f61a403e450 con 0x7f61b4071460 2024-04-25T08:53:49.571 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.576+0000 7f61bb9a5640 1 -- 172.21.15.193:0/1971925114 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61b40d1530 con 0x7f61b4071460 2024-04-25T08:53:49.571 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.578+0000 7f61a9ffb640 1 -- 172.21.15.193:0/1971925114 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f61b40d1530 con 0x7f61b4071460 2024-04-25T08:53:49.640 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.650+0000 7f61bb9a5640 1 -- 172.21.15.193:0/1971925114 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7", "id": 7} v 0) v1 -- 0x7f61b4074f60 con 0x7f61b4071460 2024-04-25T08:53:49.646 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.657+0000 7f61a9ffb640 1 -- 172.21.15.193:0/1971925114 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f61a403cb50 con 0x7f61b4071460 2024-04-25T08:53:49.647 INFO:teuthology.orchestra.run.smithi193.stdout:7 2024-04-25T08:53:49.648 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.658+0000 7f61bb9a5640 1 -- 172.21.15.193:0/1971925114 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f61b4071460 msgr2=0x7f61b40d0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:49.648 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.658+0000 7f61bb9a5640 1 --2- 172.21.15.193:0/1971925114 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f61b4071460 0x7f61b40d0510 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f61a4035040 tx=0x7f61a40046c0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.648 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.658+0000 7f61bb9a5640 1 -- 172.21.15.193:0/1971925114 shutdown_connections 2024-04-25T08:53:49.648 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.658+0000 7f61bb9a5640 1 --2- 172.21.15.193:0/1971925114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f61b40b4030 0x7f61b41545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.648 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.658+0000 7f61bb9a5640 1 --2- 172.21.15.193:0/1971925114 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f61b4071460 0x7f61b40d0510 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.648 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.658+0000 7f61bb9a5640 1 --2- 172.21.15.193:0/1971925114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f61b406df70 0x7f61b4149300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.648 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.658+0000 7f61bb9a5640 1 -- 172.21.15.193:0/1971925114 >> 172.21.15.193:0/1971925114 conn(0x7f61b4066d10 msgr2=0x7f61b41523b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:49.648 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.658+0000 7f61bb9a5640 1 -- 172.21.15.193:0/1971925114 shutdown_connections 2024-04-25T08:53:49.649 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:53:49.658+0000 7f61bb9a5640 1 -- 172.21.15.193:0/1971925114 wait complete. 2024-04-25T08:53:49.658 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T08:53:49.658 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-25T08:53:49.660 INFO:tasks.ceph.osd.0:Started 2024-04-25T08:53:49.661 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T08:53:49.661 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-25T08:53:49.663 INFO:tasks.ceph.osd.1:Started 2024-04-25T08:53:49.663 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T08:53:49.663 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-25T08:53:49.665 INFO:tasks.ceph.osd.2:Started 2024-04-25T08:53:49.666 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T08:53:49.666 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-25T08:53:49.668 INFO:tasks.ceph.osd.3:Started 2024-04-25T08:53:49.669 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-25T08:53:49.669 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-25T08:53:49.671 INFO:tasks.ceph.osd.4:Started 2024-04-25T08:53:49.671 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-25T08:53:49.671 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-25T08:53:49.673 INFO:tasks.ceph.osd.5:Started 2024-04-25T08:53:49.674 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-25T08:53:49.674 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-25T08:53:49.676 INFO:tasks.ceph.osd.6:Started 2024-04-25T08:53:49.676 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-25T08:53:49.676 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-25T08:53:49.678 INFO:tasks.ceph.osd.7:Started 2024-04-25T08:53:49.679 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:53:49.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.820+0000 7f6234e5a640 1 Processor -- start 2024-04-25T08:53:49.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.820+0000 7f6234e5a640 1 -- start start 2024-04-25T08:53:49.935 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-25T08:53:49.883+0000 7facf6922040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T08:53:49.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.821+0000 7f6234e5a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62300d0d90 0x7f62300ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:49.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.821+0000 7f6234e5a640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f62300d2500 0x7f62300d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:49.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.821+0000 7f6234e5a640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62300b6c50 con 0x7f62300d39d0 2024-04-25T08:53:49.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.821+0000 7f6234e5a640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f623015e010 con 0x7f62300d2500 2024-04-25T08:53:49.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.821+0000 7f6234e5a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62300d2020 con 0x7f62300d0d90 2024-04-25T08:53:49.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.822+0000 7f622ed9d640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f62300d2500 0x7f62300d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:49.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.822+0000 7f622dd9b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62300d0d90 0x7f62300ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:49.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.822+0000 7f622e59c640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f62300d39d0 0x7f62300d3db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:52900/0 (socket says 172.21.15.43:52900) 2024-04-25T08:53:49.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.822+0000 7f622e59c640 1 -- 172.21.15.43:0/774679942 learned_addr learned my addr 172.21.15.43:0/774679942 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:53:49.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.822+0000 7f622ed9d640 1 -- 172.21.15.43:0/774679942 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62300d0d90 msgr2=0x7f62300ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:49.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.822+0000 7f622ed9d640 1 --2- 172.21.15.43:0/774679942 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62300d0d90 0x7f62300ce2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.822+0000 7f622ed9d640 1 -- 172.21.15.43:0/774679942 >> v1:172.21.15.193:6789/0 conn(0x7f62300d39d0 legacy=0x7f62300d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:49.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.822+0000 7f622ed9d640 1 -- 172.21.15.43:0/774679942 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62300cec40 con 0x7f62300d2500 2024-04-25T08:53:49.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.823+0000 7f622ed9d640 1 --2- 172.21.15.43:0/774679942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62300d2500 0x7f62300d2990 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6228009d90 tx=0x7f622802a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e88a78e893e9ad5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:49.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.823+0000 7f622d59a640 1 -- 172.21.15.43:0/774679942 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6228031600 con 0x7f62300d2500 2024-04-25T08:53:49.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.823+0000 7f622d59a640 1 -- 172.21.15.43:0/774679942 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62280317a0 con 0x7f62300d2500 2024-04-25T08:53:49.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.823+0000 7f622d59a640 1 -- 172.21.15.43:0/774679942 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6228031b70 con 0x7f62300d2500 2024-04-25T08:53:49.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.823+0000 7f6234e5a640 1 -- 172.21.15.43:0/774679942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62300d2500 msgr2=0x7f62300d2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:49.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.823+0000 7f6234e5a640 1 --2- 172.21.15.43:0/774679942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62300d2500 0x7f62300d2990 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6228009d90 tx=0x7f622802a5d0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.823+0000 7f6234e5a640 1 -- 172.21.15.43:0/774679942 shutdown_connections 2024-04-25T08:53:49.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.823+0000 7f6234e5a640 1 --2- 172.21.15.43:0/774679942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62300d2500 0x7f62300d2990 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.823+0000 7f6234e5a640 1 --2- 172.21.15.43:0/774679942 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62300d0d90 0x7f62300ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.823+0000 7f6234e5a640 1 -- 172.21.15.43:0/774679942 >> 172.21.15.43:0/774679942 conn(0x7f623006b000 msgr2=0x7f623006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:49.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.824+0000 7f6234e5a640 1 -- 172.21.15.43:0/774679942 shutdown_connections 2024-04-25T08:53:49.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.824+0000 7f6234e5a640 1 -- 172.21.15.43:0/774679942 wait complete. 2024-04-25T08:53:49.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.825+0000 7f6234e5a640 1 Processor -- start 2024-04-25T08:53:49.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.825+0000 7f6234e5a640 1 -- start start 2024-04-25T08:53:49.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.826+0000 7f6234e5a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62300d0d90 0x7f6230156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:49.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.826+0000 7f6234e5a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62300d2500 0x7f62300b51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:49.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.826+0000 7f6234e5a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62300d39d0 0x7f62301f71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:49.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.826+0000 7f6234e5a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62300cf200 con 0x7f62300d0d90 2024-04-25T08:53:49.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.826+0000 7f6234e5a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62300b2f00 con 0x7f62300d2500 2024-04-25T08:53:49.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.826+0000 7f6234e5a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62300d2ed0 con 0x7f62300d39d0 2024-04-25T08:53:49.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.826+0000 7f622e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62300d0d90 0x7f6230156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:49.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.826+0000 7f622e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62300d0d90 0x7f6230156b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39436/0 (socket says 172.21.15.43:39436) 2024-04-25T08:53:49.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.826+0000 7f622e59c640 1 -- 172.21.15.43:0/3520386531 learned_addr learned my addr 172.21.15.43:0/3520386531 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:53:49.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.826+0000 7f622dd9b640 1 --2- 172.21.15.43:0/3520386531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62300d2500 0x7f62300b51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:49.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.826+0000 7f622ed9d640 1 --2- 172.21.15.43:0/3520386531 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62300d39d0 0x7f62301f71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:49.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.827+0000 7f622e59c640 1 -- 172.21.15.43:0/3520386531 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62300d39d0 msgr2=0x7f62301f71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:49.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.827+0000 7f622e59c640 1 --2- 172.21.15.43:0/3520386531 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62300d39d0 0x7f62301f71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.827+0000 7f622e59c640 1 -- 172.21.15.43:0/3520386531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62300d2500 msgr2=0x7f62300b51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:49.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.827+0000 7f622e59c640 1 --2- 172.21.15.43:0/3520386531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62300d2500 0x7f62300b51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.827+0000 7f622e59c640 1 -- 172.21.15.43:0/3520386531 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62300b5ad0 con 0x7f62300d0d90 2024-04-25T08:53:49.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.827+0000 7f622e59c640 1 --2- 172.21.15.43:0/3520386531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62300d0d90 0x7f6230156b50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f622000c3f0 tx=0x7f622000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:49.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.827+0000 7f621effd640 1 -- 172.21.15.43:0/3520386531 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6220017070 con 0x7f62300d0d90 2024-04-25T08:53:49.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.827+0000 7f621effd640 1 -- 172.21.15.43:0/3520386531 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6220005ce0 con 0x7f62300d0d90 2024-04-25T08:53:49.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.827+0000 7f6234e5a640 1 -- 172.21.15.43:0/3520386531 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62300b5db0 con 0x7f62300d0d90 2024-04-25T08:53:49.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.827+0000 7f621effd640 1 -- 172.21.15.43:0/3520386531 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62200072f0 con 0x7f62300d0d90 2024-04-25T08:53:49.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.827+0000 7f6234e5a640 1 -- 172.21.15.43:0/3520386531 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62300b5710 con 0x7f62300d0d90 2024-04-25T08:53:49.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.827+0000 7f621effd640 1 -- 172.21.15.43:0/3520386531 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f622001a070 con 0x7f62300d0d90 2024-04-25T08:53:49.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.827+0000 7f621effd640 1 -- 172.21.15.43:0/3520386531 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f6220012ad0 con 0x7f62300d0d90 2024-04-25T08:53:49.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.828+0000 7f6234e5a640 1 -- 172.21.15.43:0/3520386531 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6230074ef0 con 0x7f62300d0d90 2024-04-25T08:53:49.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.830+0000 7f621effd640 1 -- 172.21.15.43:0/3520386531 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f6220021020 con 0x7f62300d0d90 2024-04-25T08:53:49.943 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-25T08:53:49.887+0000 7f78d0b23040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T08:53:49.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.892+0000 7f6234e5a640 1 -- 172.21.15.43:0/3520386531 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62300ce880 con 0x7f62300d0d90 2024-04-25T08:53:49.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.893+0000 7f621effd640 1 -- 172.21.15.43:0/3520386531 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f62300b6c50 con 0x7f62300d0d90 2024-04-25T08:53:49.943 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:53:49.943 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":10,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:53:49.652694+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:53:49.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.894+0000 7f6234e5a640 1 -- 172.21.15.43:0/3520386531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62300d0d90 msgr2=0x7f6230156b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:49.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.894+0000 7f6234e5a640 1 --2- 172.21.15.43:0/3520386531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62300d0d90 0x7f6230156b50 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f622000c3f0 tx=0x7f622000c8b0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.894+0000 7f6234e5a640 1 -- 172.21.15.43:0/3520386531 shutdown_connections 2024-04-25T08:53:49.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.894+0000 7f6234e5a640 1 --2- 172.21.15.43:0/3520386531 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62300d39d0 0x7f62301f71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.894+0000 7f6234e5a640 1 --2- 172.21.15.43:0/3520386531 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62300d2500 0x7f62300b51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.894+0000 7f6234e5a640 1 --2- 172.21.15.43:0/3520386531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62300d0d90 0x7f6230156b50 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:49.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.894+0000 7f6234e5a640 1 -- 172.21.15.43:0/3520386531 >> 172.21.15.43:0/3520386531 conn(0x7f623006b000 msgr2=0x7f623006b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:49.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.894+0000 7f6234e5a640 1 -- 172.21.15.43:0/3520386531 shutdown_connections 2024-04-25T08:53:49.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:53:49.894+0000 7f6234e5a640 1 -- 172.21.15.43:0/3520386531 wait complete. 2024-04-25T08:53:49.946 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-25T08:53:49.946 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-25T08:53:50.129 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-25T08:53:50.139+0000 7facf6922040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T08:53:50.148 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-25T08:53:50.158+0000 7f78d0b23040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T08:53:50.181 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-25T08:53:50.191+0000 7facf6922040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T08:53:50.203 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-25T08:53:50.212+0000 7f78d0b23040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T08:53:50.268 INFO:tasks.ceph.osd.5.smithi193.stderr:2024-04-25T08:53:50.279+0000 7fa5fc030640 -1 Falling back to public interface 2024-04-25T08:53:50.270 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-04-25T08:53:50.280+0000 7ffb98edf640 -1 Falling back to public interface 2024-04-25T08:53:50.282 INFO:tasks.ceph.osd.7.smithi193.stderr:2024-04-25T08:53:50.292+0000 7fcd4700d640 -1 Falling back to public interface 2024-04-25T08:53:50.285 INFO:tasks.ceph.osd.2.smithi043.stderr:2024-04-25T08:53:50.295+0000 7f1658a17640 -1 Falling back to public interface 2024-04-25T08:53:50.293 INFO:tasks.ceph.osd.1.smithi043.stderr:2024-04-25T08:53:50.303+0000 7f859a80f640 -1 Falling back to public interface 2024-04-25T08:53:50.293 INFO:tasks.ceph.osd.0.smithi043.stderr:2024-04-25T08:53:50.303+0000 7fd476c0d640 -1 Falling back to public interface 2024-04-25T08:53:50.299 INFO:tasks.ceph.osd.6.smithi193.stderr:2024-04-25T08:53:50.309+0000 7f4507a82640 -1 Falling back to public interface 2024-04-25T08:53:50.299 INFO:tasks.ceph.osd.4.smithi193.stderr:2024-04-25T08:53:50.309+0000 7ff8acc3f640 -1 Falling back to public interface 2024-04-25T08:53:50.299 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-25T08:53:50.310+0000 7facf6922040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T08:53:50.300 INFO:tasks.ceph.osd.6.smithi193.stderr:2024-04-25T08:53:50.311+0000 7f4506a31640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6820/0: (98) Address already in use 2024-04-25T08:53:50.327 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-25T08:53:50.337+0000 7f78d0b23040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T08:53:50.580 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-25T08:53:50.591+0000 7facf6922040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T08:53:50.621 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-25T08:53:50.631+0000 7f78d0b23040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T08:53:50.628 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-25T08:53:50.639+0000 7facf6922040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T08:53:50.670 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-25T08:53:50.680+0000 7f78d0b23040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T08:53:50.751 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T08:53:50.781 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-25T08:53:50.791+0000 7facf6922040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T08:53:50.836 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-25T08:53:50.846+0000 7f78d0b23040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T08:53:50.844 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.854+0000 7fec91e4a640 1 Processor -- start 2024-04-25T08:53:50.844 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.854+0000 7fec91e4a640 1 -- start start 2024-04-25T08:53:50.845 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.855+0000 7fec91e4a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fec8c158070 0x7fec8c0c6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:50.846 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.855+0000 7fec91e4a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fec8c161790 0x7fec8c06abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:50.846 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.855+0000 7fec91e4a640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec8c161c50 con 0x7fec8c0c7350 2024-04-25T08:53:50.846 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.855+0000 7fec91e4a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec8c15e2a0 con 0x7fec8c161790 2024-04-25T08:53:50.846 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.855+0000 7fec91e4a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec8c161f30 con 0x7fec8c158070 2024-04-25T08:53:50.846 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.855+0000 7fec8bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fec8c161790 0x7fec8c06abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:50.846 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.855+0000 7fec8b7fe640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fec8c158070 0x7fec8c0c6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:50.846 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.855+0000 7fec8affd640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7fec8c0c7350 0x7fec8c071600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:52916/0 (socket says 172.21.15.43:52916) 2024-04-25T08:53:50.846 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.855+0000 7fec8affd640 1 -- 172.21.15.43:0/3728327754 learned_addr learned my addr 172.21.15.43:0/3728327754 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:53:50.846 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.855+0000 7fec8b7fe640 1 -- 172.21.15.43:0/3728327754 >> v2:172.21.15.43:3300/0 conn(0x7fec8c161790 msgr2=0x7fec8c06abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:53:50.846 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.855+0000 7fec8b7fe640 1 --2- 172.21.15.43:0/3728327754 >> v2:172.21.15.43:3300/0 conn(0x7fec8c161790 0x7fec8c06abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.846 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.855+0000 7fec8b7fe640 1 -- 172.21.15.43:0/3728327754 >> v1:172.21.15.193:6789/0 conn(0x7fec8c0c7350 legacy=0x7fec8c071600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:50.846 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.855+0000 7fec8b7fe640 1 -- 172.21.15.43:0/3728327754 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec8c162210 con 0x7fec8c158070 2024-04-25T08:53:50.846 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.855+0000 7fec8a7fc640 1 -- 172.21.15.43:0/3728327754 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2748184808 0 0) 0x7fec8c161c50 con 0x7fec8c0c7350 2024-04-25T08:53:50.848 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.856+0000 7fec8b7fe640 1 --2- 172.21.15.43:0/3728327754 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fec8c158070 0x7fec8c0c6e10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fec8c15e420 tx=0x7fec7c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59150c9e858c234a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:50.848 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.856+0000 7fec8a7fc640 1 -- 172.21.15.43:0/3728327754 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c037070 con 0x7fec8c158070 2024-04-25T08:53:50.848 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.856+0000 7fec8a7fc640 1 -- 172.21.15.43:0/3728327754 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7c032440 con 0x7fec8c158070 2024-04-25T08:53:50.848 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.856+0000 7fec8a7fc640 1 -- 172.21.15.43:0/3728327754 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c032810 con 0x7fec8c158070 2024-04-25T08:53:50.848 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.857+0000 7fec91e4a640 1 -- 172.21.15.43:0/3728327754 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fec8c158070 msgr2=0x7fec8c0c6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:50.848 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.857+0000 7fec91e4a640 1 --2- 172.21.15.43:0/3728327754 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fec8c158070 0x7fec8c0c6e10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fec8c15e420 tx=0x7fec7c02a180 comp rx=0 tx=0).stop 2024-04-25T08:53:50.848 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.857+0000 7fec91e4a640 1 -- 172.21.15.43:0/3728327754 shutdown_connections 2024-04-25T08:53:50.848 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.857+0000 7fec91e4a640 1 --2- 172.21.15.43:0/3728327754 >> v2:172.21.15.43:3300/0 conn(0x7fec8c161790 0x7fec8c06abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.848 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.857+0000 7fec91e4a640 1 --2- 172.21.15.43:0/3728327754 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fec8c158070 0x7fec8c0c6e10 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.848 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.857+0000 7fec91e4a640 1 -- 172.21.15.43:0/3728327754 >> 172.21.15.43:0/3728327754 conn(0x7fec8c0b2330 msgr2=0x7fec8c0b2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:50.848 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.857+0000 7fec91e4a640 1 -- 172.21.15.43:0/3728327754 shutdown_connections 2024-04-25T08:53:50.849 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.857+0000 7fec91e4a640 1 -- 172.21.15.43:0/3728327754 wait complete. 2024-04-25T08:53:50.851 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.858+0000 7fec91e4a640 1 Processor -- start 2024-04-25T08:53:50.852 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.858+0000 7fec91e4a640 1 -- start start 2024-04-25T08:53:50.852 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.859+0000 7fec91e4a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fec8c0c7350 0x7fec8c14c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:50.852 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.859+0000 7fec91e4a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec8c158070 0x7fec8c0cad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:50.852 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.859+0000 7fec91e4a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec8c161790 0x7fec8c0ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:50.852 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.859+0000 7fec91e4a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8c158450 con 0x7fec8c158070 2024-04-25T08:53:50.852 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.859+0000 7fec91e4a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8c071d10 con 0x7fec8c161790 2024-04-25T08:53:50.852 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.859+0000 7fec91e4a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec8c1585d0 con 0x7fec8c0c7350 2024-04-25T08:53:50.852 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.859+0000 7fec8affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec8c158070 0x7fec8c0cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:50.852 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.859+0000 7fec8bfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec8c161790 0x7fec8c0ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:50.852 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.859+0000 7fec8b7fe640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fec8c0c7350 0x7fec8c14c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:50.852 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.859+0000 7fec8bfff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec8c161790 0x7fec8c0ce2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:53954/0 (socket says 172.21.15.43:53954) 2024-04-25T08:53:50.852 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.859+0000 7fec8bfff640 1 -- 172.21.15.43:0/2451184755 learned_addr learned my addr 172.21.15.43:0/2451184755 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:53:50.852 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.860+0000 7fec8b7fe640 1 -- 172.21.15.43:0/2451184755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec8c161790 msgr2=0x7fec8c0ce2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:50.852 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.860+0000 7fec8b7fe640 1 --2- 172.21.15.43:0/2451184755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec8c161790 0x7fec8c0ce2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.853 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.860+0000 7fec8b7fe640 1 -- 172.21.15.43:0/2451184755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec8c158070 msgr2=0x7fec8c0cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:50.853 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.860+0000 7fec8b7fe640 1 --2- 172.21.15.43:0/2451184755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec8c158070 0x7fec8c0cad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.853 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.860+0000 7fec8b7fe640 1 -- 172.21.15.43:0/2451184755 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec8c0c83b0 con 0x7fec8c0c7350 2024-04-25T08:53:50.853 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.860+0000 7fec8bfff640 1 --2- 172.21.15.43:0/2451184755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec8c161790 0x7fec8c0ce2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:53:50.853 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.860+0000 7fec8b7fe640 1 --2- 172.21.15.43:0/2451184755 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fec8c0c7350 0x7fec8c14c2e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fec8c0749f0 tx=0x7fec7c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:50.853 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.860+0000 7fec83fff640 1 -- 172.21.15.43:0/2451184755 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c041020 con 0x7fec8c0c7350 2024-04-25T08:53:50.853 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.860+0000 7fec83fff640 1 -- 172.21.15.43:0/2451184755 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7c030720 con 0x7fec8c0c7350 2024-04-25T08:53:50.853 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.860+0000 7fec91e4a640 1 -- 172.21.15.43:0/2451184755 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec8c0c8690 con 0x7fec8c0c7350 2024-04-25T08:53:50.853 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.860+0000 7fec83fff640 1 -- 172.21.15.43:0/2451184755 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c037040 con 0x7fec8c0c7350 2024-04-25T08:53:50.853 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.860+0000 7fec91e4a640 1 -- 172.21.15.43:0/2451184755 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec8c15f670 con 0x7fec8c0c7350 2024-04-25T08:53:50.853 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.861+0000 7fec83fff640 1 -- 172.21.15.43:0/2451184755 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fec7c044020 con 0x7fec8c0c7350 2024-04-25T08:53:50.853 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.861+0000 7fec83fff640 1 -- 172.21.15.43:0/2451184755 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fec7c03dcc0 con 0x7fec8c0c7350 2024-04-25T08:53:50.853 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.861+0000 7fec91e4a640 1 -- 172.21.15.43:0/2451184755 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec8c0734e0 con 0x7fec8c0c7350 2024-04-25T08:53:50.854 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.864+0000 7fec83fff640 1 -- 172.21.15.43:0/2451184755 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fec7c04b030 con 0x7fec8c0c7350 2024-04-25T08:53:50.916 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.926+0000 7fec91e4a640 1 -- 172.21.15.43:0/2451184755 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec8c0b29a0 con 0x7fec8c0c7350 2024-04-25T08:53:50.916 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.926+0000 7fec83fff640 1 -- 172.21.15.43:0/2451184755 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7fec7c03d420 con 0x7fec8c0c7350 2024-04-25T08:53:50.917 INFO:teuthology.misc.health.smithi043.stdout: 2024-04-25T08:53:50.917 INFO:teuthology.misc.health.smithi043.stdout:{"epoch":10,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:53:49.652694+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:53:50.918 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.927+0000 7fec91e4a640 1 -- 172.21.15.43:0/2451184755 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fec8c0c7350 msgr2=0x7fec8c14c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:50.918 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.927+0000 7fec91e4a640 1 --2- 172.21.15.43:0/2451184755 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fec8c0c7350 0x7fec8c14c2e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fec8c0749f0 tx=0x7fec7c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.918 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.927+0000 7fec91e4a640 1 -- 172.21.15.43:0/2451184755 shutdown_connections 2024-04-25T08:53:50.918 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.927+0000 7fec91e4a640 1 --2- 172.21.15.43:0/2451184755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec8c161790 0x7fec8c0ce2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.918 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.927+0000 7fec91e4a640 1 --2- 172.21.15.43:0/2451184755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec8c158070 0x7fec8c0cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.918 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.927+0000 7fec91e4a640 1 --2- 172.21.15.43:0/2451184755 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fec8c0c7350 0x7fec8c14c2e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:50.918 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.927+0000 7fec91e4a640 1 -- 172.21.15.43:0/2451184755 >> 172.21.15.43:0/2451184755 conn(0x7fec8c0b2330 msgr2=0x7fec8c156520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:50.919 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.928+0000 7fec91e4a640 1 -- 172.21.15.43:0/2451184755 shutdown_connections 2024-04-25T08:53:50.919 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:50.928+0000 7fec91e4a640 1 -- 172.21.15.43:0/2451184755 wait complete. 2024-04-25T08:53:50.919 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-25T08:53:50.928+0000 7facf6922040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T08:53:50.927 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T08:53:50.966 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-25T08:53:50.976+0000 7facf6922040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T08:53:50.982 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-25T08:53:50.992+0000 7f78d0b23040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T08:53:51.034 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-25T08:53:51.044+0000 7f78d0b23040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T08:53:51.034 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-25T08:53:51.044+0000 7facf6922040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T08:53:51.080 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-25T08:53:51.090+0000 7facf6922040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T08:53:51.108 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-25T08:53:51.117+0000 7f78d0b23040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T08:53:51.155 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-25T08:53:51.165+0000 7f78d0b23040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T08:53:51.554 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-25T08:53:51.564+0000 7facf6922040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T08:53:51.654 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-25T08:53:51.664+0000 7f78d0b23040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T08:53:51.671 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-25T08:53:51.681+0000 7facf6922040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T08:53:51.776 INFO:tasks.ceph.mgr.y.smithi043.stderr:2024-04-25T08:53:51.786+0000 7f78d0b23040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T08:53:52.687 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-25T08:53:52.697+0000 7fac97386640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T08:53:52.688 INFO:tasks.ceph.mgr.x.smithi193.stderr:2024-04-25T08:53:52.698+0000 7fac97386640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T08:53:53.724 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-04-25T08:53:53.734+0000 7ffb98edf640 -1 osd.3 0 log_to_monitors true 2024-04-25T08:53:53.729 INFO:tasks.ceph.osd.5.smithi193.stderr:2024-04-25T08:53:53.740+0000 7fa5fc030640 -1 osd.5 0 log_to_monitors true 2024-04-25T08:53:53.764 INFO:tasks.ceph.osd.7.smithi193.stderr:2024-04-25T08:53:53.774+0000 7fcd4700d640 -1 osd.7 0 log_to_monitors true 2024-04-25T08:53:53.782 INFO:tasks.ceph.osd.2.smithi043.stderr:2024-04-25T08:53:53.792+0000 7f1658a17640 -1 osd.2 0 log_to_monitors true 2024-04-25T08:53:53.784 INFO:tasks.ceph.osd.1.smithi043.stderr:2024-04-25T08:53:53.794+0000 7f859a80f640 -1 osd.1 0 log_to_monitors true 2024-04-25T08:53:53.799 INFO:tasks.ceph.osd.0.smithi043.stderr:2024-04-25T08:53:53.809+0000 7fd476c0d640 -1 osd.0 0 log_to_monitors true 2024-04-25T08:53:53.800 INFO:tasks.ceph.osd.4.smithi193.stderr:2024-04-25T08:53:53.811+0000 7ff8acc3f640 -1 osd.4 0 log_to_monitors true 2024-04-25T08:53:53.805 INFO:tasks.ceph.osd.6.smithi193.stderr:2024-04-25T08:53:53.815+0000 7f4507a82640 -1 osd.6 0 log_to_monitors true 2024-04-25T08:53:57.150 INFO:tasks.ceph.osd.0.smithi043.stderr:2024-04-25T08:53:57.160+0000 7fd471dff640 -1 osd.0 0 waiting for initial osdmap 2024-04-25T08:53:57.161 INFO:tasks.ceph.osd.1.smithi043.stderr:2024-04-25T08:53:57.171+0000 7f85959ff640 -1 osd.1 0 waiting for initial osdmap 2024-04-25T08:53:57.162 INFO:tasks.ceph.osd.0.smithi043.stderr:2024-04-25T08:53:57.172+0000 7fd46a431640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:53:57.170 INFO:tasks.ceph.osd.1.smithi043.stderr:2024-04-25T08:53:57.180+0000 7f858e031640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:53:57.176 INFO:tasks.ceph.osd.2.smithi043.stderr:2024-04-25T08:53:57.186+0000 7f1653ae9640 -1 osd.2 0 waiting for initial osdmap 2024-04-25T08:53:57.185 INFO:tasks.ceph.osd.2.smithi043.stderr:2024-04-25T08:53:57.195+0000 7f164c11b640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:53:57.206 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-04-25T08:53:57.216+0000 7ffb94652640 -1 osd.3 0 waiting for initial osdmap 2024-04-25T08:53:57.216 INFO:tasks.ceph.osd.3.smithi043.stderr:2024-04-25T08:53:57.226+0000 7ffb8cc84640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:53:57.673 INFO:tasks.ceph.osd.6.smithi193.stderr:2024-04-25T08:53:57.683+0000 7f45031f5640 -1 osd.6 0 waiting for initial osdmap 2024-04-25T08:53:57.688 INFO:tasks.ceph.osd.6.smithi193.stderr:2024-04-25T08:53:57.698+0000 7f44fb827640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:53:57.712 INFO:tasks.ceph.osd.5.smithi193.stderr:2024-04-25T08:53:57.722+0000 7fa5f858d640 -1 osd.5 0 waiting for initial osdmap 2024-04-25T08:53:57.722 INFO:tasks.ceph.osd.5.smithi193.stderr:2024-04-25T08:53:57.733+0000 7fa5ef387640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:53:57.734 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T08:53:57.740 INFO:tasks.ceph.osd.7.smithi193.stderr:2024-04-25T08:53:57.751+0000 7fcd42101640 -1 osd.7 0 waiting for initial osdmap 2024-04-25T08:53:57.754 INFO:tasks.ceph.osd.7.smithi193.stderr:2024-04-25T08:53:57.765+0000 7fcd3a733640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:53:57.782 INFO:tasks.ceph.osd.4.smithi193.stderr:2024-04-25T08:53:57.792+0000 7ff8a83b2640 -1 osd.4 0 waiting for initial osdmap 2024-04-25T08:53:57.796 INFO:tasks.ceph.osd.4.smithi193.stderr:2024-04-25T08:53:57.806+0000 7ff8a09e4640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T08:53:57.833 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.843+0000 7fa089170640 1 Processor -- start 2024-04-25T08:53:57.833 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.843+0000 7fa089170640 1 -- start start 2024-04-25T08:53:57.836 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.844+0000 7fa089170640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa08415ddd0 0x7fa0840d2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:57.836 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.844+0000 7fa089170640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0840d1a70 con 0x7fa0840b5b40 2024-04-25T08:53:57.836 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.844+0000 7fa089170640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0840d1d50 con 0x7fa0840d33c0 2024-04-25T08:53:57.836 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.844+0000 7fa089170640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0840d3820 con 0x7fa08415ddd0 2024-04-25T08:53:57.836 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.844+0000 7fa08359e640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa0840b5b40 0x7fa0840b6a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35714/0 (socket says 172.21.15.43:35714) 2024-04-25T08:53:57.836 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.844+0000 7fa08359e640 1 -- 172.21.15.43:0/171017649 learned_addr learned my addr 172.21.15.43:0/171017649 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:53:57.836 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.844+0000 7fa082d9d640 1 --2- 172.21.15.43:0/171017649 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa08415ddd0 0x7fa0840d2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:57.837 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.844+0000 7fa08259c640 1 -- 172.21.15.43:0/171017649 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1984019354 0 0) 0x7fa0840d1a70 con 0x7fa0840b5b40 2024-04-25T08:53:57.837 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.844+0000 7fa08259c640 1 -- 172.21.15.43:0/171017649 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa070003520 con 0x7fa0840b5b40 2024-04-25T08:53:57.837 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.844+0000 7fa08259c640 1 -- 172.21.15.43:0/171017649 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3812242693 0 0) 0x7fa0840d1d50 con 0x7fa0840d33c0 2024-04-25T08:53:57.837 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.844+0000 7fa08259c640 1 -- 172.21.15.43:0/171017649 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0840d1a70 con 0x7fa0840d33c0 2024-04-25T08:53:57.837 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.844+0000 7fa082d9d640 1 -- 172.21.15.43:0/171017649 >> v1:172.21.15.193:6789/0 conn(0x7fa0840d33c0 legacy=0x7fa0840b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:57.837 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.844+0000 7fa08259c640 1 -- 172.21.15.43:0/171017649 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3351334581 0 0) 0x7fa070003520 con 0x7fa0840b5b40 2024-04-25T08:53:57.837 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.844+0000 7fa082d9d640 1 -- 172.21.15.43:0/171017649 >> v1:172.21.15.43:6789/0 conn(0x7fa0840b5b40 legacy=0x7fa0840b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:57.837 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.844+0000 7fa082d9d640 1 -- 172.21.15.43:0/171017649 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0840d2030 con 0x7fa08415ddd0 2024-04-25T08:53:57.837 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.845+0000 7fa082d9d640 1 --2- 172.21.15.43:0/171017649 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa08415ddd0 0x7fa0840d2e80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa0740097d0 tx=0x7fa07402a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b4220cf57e960bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:57.837 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.845+0000 7fa08259c640 1 -- 172.21.15.43:0/171017649 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa074037070 con 0x7fa08415ddd0 2024-04-25T08:53:57.837 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.845+0000 7fa08259c640 1 -- 172.21.15.43:0/171017649 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa07402ac30 con 0x7fa08415ddd0 2024-04-25T08:53:57.837 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.845+0000 7fa08259c640 1 -- 172.21.15.43:0/171017649 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07c0019f0 con 0x7fa08415ddd0 2024-04-25T08:53:57.837 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.846+0000 7fa089170640 1 -- 172.21.15.43:0/171017649 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa08415ddd0 msgr2=0x7fa0840d2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:57.837 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.846+0000 7fa089170640 1 --2- 172.21.15.43:0/171017649 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa08415ddd0 0x7fa0840d2e80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa0740097d0 tx=0x7fa07402a1a0 comp rx=0 tx=0).stop 2024-04-25T08:53:57.837 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.846+0000 7fa089170640 1 -- 172.21.15.43:0/171017649 shutdown_connections 2024-04-25T08:53:57.838 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.846+0000 7fa089170640 1 --2- 172.21.15.43:0/171017649 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa08415ddd0 0x7fa0840d2e80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:57.838 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.846+0000 7fa089170640 1 -- 172.21.15.43:0/171017649 >> 172.21.15.43:0/171017649 conn(0x7fa0840d0610 msgr2=0x7fa0840d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:57.838 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.846+0000 7fa089170640 1 -- 172.21.15.43:0/171017649 shutdown_connections 2024-04-25T08:53:57.838 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.846+0000 7fa089170640 1 -- 172.21.15.43:0/171017649 wait complete. 2024-04-25T08:53:57.840 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.847+0000 7fa089170640 1 Processor -- start 2024-04-25T08:53:57.840 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.847+0000 7fa089170640 1 -- start start 2024-04-25T08:53:57.840 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.848+0000 7fa089170640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0840b5b40 0x7fa0840c8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:57.840 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.848+0000 7fa089170640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa0840d33c0 0x7fa0840b89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:57.840 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.848+0000 7fa089170640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa08415ddd0 0x7fa0840ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:57.840 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.848+0000 7fa089170640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0840d25f0 con 0x7fa0840b5b40 2024-04-25T08:53:57.841 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.848+0000 7fa089170640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0840cf070 con 0x7fa08415ddd0 2024-04-25T08:53:57.841 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.848+0000 7fa089170640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0840b7150 con 0x7fa0840d33c0 2024-04-25T08:53:57.841 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.848+0000 7fa082d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0840b5b40 0x7fa0840c8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:57.841 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.848+0000 7fa082d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0840b5b40 0x7fa0840c8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39496/0 (socket says 172.21.15.43:39496) 2024-04-25T08:53:57.841 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.848+0000 7fa082d9d640 1 -- 172.21.15.43:0/4285637539 learned_addr learned my addr 172.21.15.43:0/4285637539 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:53:57.841 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.848+0000 7fa08359e640 1 --2- 172.21.15.43:0/4285637539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa08415ddd0 0x7fa0840ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:57.841 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.848+0000 7fa07bfff640 1 --2- 172.21.15.43:0/4285637539 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa0840d33c0 0x7fa0840b89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:57.841 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.848+0000 7fa082d9d640 1 -- 172.21.15.43:0/4285637539 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa0840d33c0 msgr2=0x7fa0840b89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:57.841 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.848+0000 7fa082d9d640 1 --2- 172.21.15.43:0/4285637539 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa0840d33c0 0x7fa0840b89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:57.841 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.848+0000 7fa082d9d640 1 -- 172.21.15.43:0/4285637539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa08415ddd0 msgr2=0x7fa0840ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:57.841 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.848+0000 7fa082d9d640 1 --2- 172.21.15.43:0/4285637539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa08415ddd0 0x7fa0840ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:57.841 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.848+0000 7fa082d9d640 1 -- 172.21.15.43:0/4285637539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0840c9500 con 0x7fa0840b5b40 2024-04-25T08:53:57.841 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.849+0000 7fa082d9d640 1 --2- 172.21.15.43:0/4285637539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0840b5b40 0x7fa0840c8ef0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa074035040 tx=0x7fa0740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:57.841 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.849+0000 7fa07b7fe640 1 -- 172.21.15.43:0/4285637539 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa074037050 con 0x7fa0840b5b40 2024-04-25T08:53:57.841 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.849+0000 7fa07b7fe640 1 -- 172.21.15.43:0/4285637539 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa074043070 con 0x7fa0840b5b40 2024-04-25T08:53:57.842 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.849+0000 7fa07b7fe640 1 -- 172.21.15.43:0/4285637539 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa074006e10 con 0x7fa0840b5b40 2024-04-25T08:53:57.842 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.849+0000 7fa089170640 1 -- 172.21.15.43:0/4285637539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0840c97e0 con 0x7fa0840b5b40 2024-04-25T08:53:57.842 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.849+0000 7fa089170640 1 -- 172.21.15.43:0/4285637539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0840c9da0 con 0x7fa0840b5b40 2024-04-25T08:53:57.842 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.850+0000 7fa089170640 1 -- 172.21.15.43:0/4285637539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0840734e0 con 0x7fa0840b5b40 2024-04-25T08:53:57.842 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.850+0000 7fa07b7fe640 1 -- 172.21.15.43:0/4285637539 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7fa07403ca30 con 0x7fa0840b5b40 2024-04-25T08:53:57.842 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.851+0000 7fa07b7fe640 1 --2- 172.21.15.43:0/4285637539 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa06406b470 0x7fa06406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:53:57.843 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.851+0000 7fa07b7fe640 1 -- 172.21.15.43:0/4285637539 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7fa0740a9d80 con 0x7fa0840b5b40 2024-04-25T08:53:57.843 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.851+0000 7fa07bfff640 1 --2- 172.21.15.43:0/4285637539 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa06406b470 0x7fa06406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:53:57.843 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.851+0000 7fa07bfff640 1 --2- 172.21.15.43:0/4285637539 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa06406b470 0x7fa06406d930 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa0840c9430 tx=0x7fa06c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:53:57.845 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:57.855+0000 7fa07b7fe640 1 -- 172.21.15.43:0/4285637539 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa074079540 con 0x7fa0840b5b40 2024-04-25T08:53:58.025 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:58.035+0000 7fa089170640 1 -- 172.21.15.43:0/4285637539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa084160f50 con 0x7fa0840b5b40 2024-04-25T08:53:58.026 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:58.036+0000 7fa07b7fe640 1 -- 172.21.15.43:0/4285637539 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7058 (secure 0 0 0) 0x7fa07407d5e0 con 0x7fa0840b5b40 2024-04-25T08:53:58.026 INFO:teuthology.misc.health.smithi043.stdout: 2024-04-25T08:53:58.026 INFO:teuthology.misc.health.smithi043.stdout:{"epoch":12,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:53:55.700869+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:53:58.028 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:58.038+0000 7fa089170640 1 -- 172.21.15.43:0/4285637539 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa06406b470 msgr2=0x7fa06406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:58.028 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:58.038+0000 7fa089170640 1 --2- 172.21.15.43:0/4285637539 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa06406b470 0x7fa06406d930 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa0840c9430 tx=0x7fa06c009040 comp rx=0 tx=0).stop 2024-04-25T08:53:58.028 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:58.038+0000 7fa089170640 1 -- 172.21.15.43:0/4285637539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0840b5b40 msgr2=0x7fa0840c8ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:53:58.028 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:58.038+0000 7fa089170640 1 --2- 172.21.15.43:0/4285637539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0840b5b40 0x7fa0840c8ef0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa074035040 tx=0x7fa0740046c0 comp rx=0 tx=0).stop 2024-04-25T08:53:58.028 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:58.038+0000 7fa089170640 1 -- 172.21.15.43:0/4285637539 shutdown_connections 2024-04-25T08:53:58.028 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:58.038+0000 7fa089170640 1 --2- 172.21.15.43:0/4285637539 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa06406b470 0x7fa06406d930 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:58.028 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:58.038+0000 7fa089170640 1 --2- 172.21.15.43:0/4285637539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa08415ddd0 0x7fa0840ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:58.028 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:58.038+0000 7fa089170640 1 --2- 172.21.15.43:0/4285637539 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa0840d33c0 0x7fa0840b89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:58.028 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:58.038+0000 7fa089170640 1 --2- 172.21.15.43:0/4285637539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa0840b5b40 0x7fa0840c8ef0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:53:58.029 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:58.038+0000 7fa089170640 1 -- 172.21.15.43:0/4285637539 >> 172.21.15.43:0/4285637539 conn(0x7fa0840d0610 msgr2=0x7fa0841601c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:53:58.029 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:58.038+0000 7fa089170640 1 -- 172.21.15.43:0/4285637539 shutdown_connections 2024-04-25T08:53:58.029 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:53:58.038+0000 7fa089170640 1 -- 172.21.15.43:0/4285637539 wait complete. 2024-04-25T08:53:58.037 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T08:54:04.844 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T08:54:04.948 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.958+0000 7f2cedf5c640 1 Processor -- start 2024-04-25T08:54:04.948 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.958+0000 7f2cedf5c640 1 -- start start 2024-04-25T08:54:04.952 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.959+0000 7f2cedf5c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2ce80d0150 0x7f2ce80d05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:04.952 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.959+0000 7f2cedf5c640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f2ce80b67f0 0x7f2ce80b6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:04.952 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.959+0000 7f2cedf5c640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ce80ce650 con 0x7f2ce815dca0 2024-04-25T08:54:04.952 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.959+0000 7f2cedf5c640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ce80d0af0 con 0x7f2ce80b67f0 2024-04-25T08:54:04.952 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.959+0000 7f2cedf5c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce80cf260 con 0x7f2ce80d0150 2024-04-25T08:54:04.952 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.959+0000 7f2ce77fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f2ce815dca0 0x7f2ce80d40b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41852/0 (socket says 172.21.15.43:41852) 2024-04-25T08:54:04.953 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.959+0000 7f2ce77fe640 1 -- 172.21.15.43:0/2106505059 learned_addr learned my addr 172.21.15.43:0/2106505059 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:04.953 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.959+0000 7f2ce7fff640 1 --2- 172.21.15.43:0/2106505059 >> v2:172.21.15.193:3300/0 conn(0x7f2ce80b67f0 0x7f2ce80b6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:04.953 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.959+0000 7f2ce67fc640 1 -- 172.21.15.43:0/2106505059 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3997025862 0 0) 0x7f2ce80ce650 con 0x7f2ce815dca0 2024-04-25T08:54:04.953 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.959+0000 7f2ce6ffd640 1 --2- 172.21.15.43:0/2106505059 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2ce80d0150 0x7f2ce80d05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:04.953 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.959+0000 7f2ce67fc640 1 -- 172.21.15.43:0/2106505059 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cd0003520 con 0x7f2ce815dca0 2024-04-25T08:54:04.953 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.960+0000 7f2ce67fc640 1 -- 172.21.15.43:0/2106505059 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4116639831 0 0) 0x7f2cd0003520 con 0x7f2ce815dca0 2024-04-25T08:54:04.953 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.960+0000 7f2ce67fc640 1 -- 172.21.15.43:0/2106505059 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2ce80d0150 msgr2=0x7f2ce80d05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:04.953 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.960+0000 7f2ce67fc640 1 --2- 172.21.15.43:0/2106505059 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2ce80d0150 0x7f2ce80d05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:04.953 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.960+0000 7f2ce67fc640 1 -- 172.21.15.43:0/2106505059 >> v2:172.21.15.193:3300/0 conn(0x7f2ce80b67f0 msgr2=0x7f2ce80b6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:04.953 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.960+0000 7f2ce67fc640 1 --2- 172.21.15.43:0/2106505059 >> v2:172.21.15.193:3300/0 conn(0x7f2ce80b67f0 0x7f2ce80b6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:04.953 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.960+0000 7f2ce67fc640 1 -- 172.21.15.43:0/2106505059 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce80ce930 con 0x7f2ce815dca0 2024-04-25T08:54:04.953 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.960+0000 7f2ce67fc640 1 -- 172.21.15.43:0/2106505059 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f2cd8001670 con 0x7f2ce815dca0 2024-04-25T08:54:04.953 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.960+0000 7f2ce67fc640 1 -- 172.21.15.43:0/2106505059 >> v1:172.21.15.43:6789/0 conn(0x7f2ce815dca0 legacy=0x7f2ce80d40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:04.954 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.960+0000 7f2ce67fc640 1 --2- 172.21.15.43:0/2106505059 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2cd00039b0 0x7f2cd0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:04.954 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.960+0000 7f2ce67fc640 1 --2- 172.21.15.43:0/2106505059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cd0004800 0x7f2cd0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:04.954 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.960+0000 7f2ce67fc640 1 --2- 172.21.15.43:0/2106505059 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2ce80d0150 0x7f2cd00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:04.954 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.960+0000 7f2ce67fc640 1 -- 172.21.15.43:0/2106505059 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce80cf260 con 0x7f2cd0004800 2024-04-25T08:54:04.954 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.960+0000 7f2ce67fc640 1 -- 172.21.15.43:0/2106505059 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd000b9c0 con 0x7f2cd00039b0 2024-04-25T08:54:04.954 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.960+0000 7f2ce67fc640 1 -- 172.21.15.43:0/2106505059 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cd000bb40 con 0x7f2ce80d0150 2024-04-25T08:54:04.954 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.960+0000 7f2ce6ffd640 1 --2- 172.21.15.43:0/2106505059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cd0004800 0x7f2cd0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:04.954 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.960+0000 7f2ce77fe640 1 --2- 172.21.15.43:0/2106505059 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2cd00039b0 0x7f2cd0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:04.954 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.960+0000 7f2ce7fff640 1 --2- 172.21.15.43:0/2106505059 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2ce80d0150 0x7f2cd00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:04.954 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.960+0000 7f2ce6ffd640 1 -- 172.21.15.43:0/2106505059 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2ce80d0150 msgr2=0x7f2cd00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:04.954 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.960+0000 7f2ce6ffd640 1 --2- 172.21.15.43:0/2106505059 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2ce80d0150 0x7f2cd00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:04.954 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.960+0000 7f2ce6ffd640 1 -- 172.21.15.43:0/2106505059 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2cd00039b0 msgr2=0x7f2cd0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:04.954 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.960+0000 7f2ce6ffd640 1 --2- 172.21.15.43:0/2106505059 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2cd00039b0 0x7f2cd0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:04.954 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.960+0000 7f2ce6ffd640 1 -- 172.21.15.43:0/2106505059 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd0003520 con 0x7f2cd0004800 2024-04-25T08:54:04.955 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.961+0000 7f2ce6ffd640 1 --2- 172.21.15.43:0/2106505059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cd0004800 0x7f2cd0004c80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc009ac0 tx=0x7f2cd0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=596252c7a6c9a819 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:04.955 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.961+0000 7f2ce67fc640 1 -- 172.21.15.43:0/2106505059 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc036070 con 0x7f2cd0004800 2024-04-25T08:54:04.955 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.961+0000 7f2ce67fc640 1 -- 172.21.15.43:0/2106505059 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cdc031400 con 0x7f2cd0004800 2024-04-25T08:54:04.955 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.961+0000 7f2ce67fc640 1 -- 172.21.15.43:0/2106505059 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc009690 con 0x7f2cd0004800 2024-04-25T08:54:04.955 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.962+0000 7f2cedf5c640 1 -- 172.21.15.43:0/2106505059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cd0004800 msgr2=0x7f2cd0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:04.955 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.962+0000 7f2cedf5c640 1 --2- 172.21.15.43:0/2106505059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cd0004800 0x7f2cd0004c80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc009ac0 tx=0x7f2cd0005830 comp rx=0 tx=0).stop 2024-04-25T08:54:04.955 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.962+0000 7f2ce7fff640 1 -- 172.21.15.43:0/2106505059 reap_dead start 2024-04-25T08:54:04.955 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.962+0000 7f2cedf5c640 1 -- 172.21.15.43:0/2106505059 shutdown_connections 2024-04-25T08:54:04.955 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.962+0000 7f2cedf5c640 1 -- 172.21.15.43:0/2106505059 >> 172.21.15.43:0/2106505059 conn(0x7f2ce809ccf0 msgr2=0x7f2ce80d39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:04.955 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.962+0000 7f2cedf5c640 1 -- 172.21.15.43:0/2106505059 shutdown_connections 2024-04-25T08:54:04.955 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.962+0000 7f2cedf5c640 1 -- 172.21.15.43:0/2106505059 wait complete. 2024-04-25T08:54:04.958 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.963+0000 7f2cedf5c640 1 Processor -- start 2024-04-25T08:54:04.958 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.963+0000 7f2cedf5c640 1 -- start start 2024-04-25T08:54:04.958 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.964+0000 7f2cedf5c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cd0004800 0x7f2ce814ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:04.958 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.964+0000 7f2cedf5c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2ce80ca300 0x7f2ce81f3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:04.958 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.964+0000 7f2cedf5c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ce80c8810 0x7f2ce80b84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:04.958 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.964+0000 7f2cedf5c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce815e080 con 0x7f2cd0004800 2024-04-25T08:54:04.958 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.964+0000 7f2cedf5c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce80ceef0 con 0x7f2ce80c8810 2024-04-25T08:54:04.958 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.964+0000 7f2cedf5c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ce81626b0 con 0x7f2ce80ca300 2024-04-25T08:54:04.959 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.964+0000 7f2ce77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cd0004800 0x7f2ce814ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:04.959 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.964+0000 7f2ce77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cd0004800 0x7f2ce814ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41374/0 (socket says 172.21.15.43:41374) 2024-04-25T08:54:04.959 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.964+0000 7f2ce77fe640 1 -- 172.21.15.43:0/3316458880 learned_addr learned my addr 172.21.15.43:0/3316458880 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:04.959 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.964+0000 7f2ce7fff640 1 --2- 172.21.15.43:0/3316458880 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ce80c8810 0x7f2ce80b84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:04.959 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.964+0000 7f2ce6ffd640 1 --2- 172.21.15.43:0/3316458880 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2ce80ca300 0x7f2ce81f3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:04.959 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.964+0000 7f2ce77fe640 1 -- 172.21.15.43:0/3316458880 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2ce80ca300 msgr2=0x7f2ce81f3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:04.959 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.964+0000 7f2ce77fe640 1 --2- 172.21.15.43:0/3316458880 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2ce80ca300 0x7f2ce81f3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:04.959 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.964+0000 7f2ce77fe640 1 -- 172.21.15.43:0/3316458880 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ce80c8810 msgr2=0x7f2ce80b84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:04.959 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.964+0000 7f2ce77fe640 1 --2- 172.21.15.43:0/3316458880 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ce80c8810 0x7f2ce80b84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:04.959 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.964+0000 7f2ce77fe640 1 -- 172.21.15.43:0/3316458880 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ce815f830 con 0x7f2cd0004800 2024-04-25T08:54:04.959 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.964+0000 7f2ce77fe640 1 --2- 172.21.15.43:0/3316458880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cd0004800 0x7f2ce814ae90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8007340 tx=0x7f2cd8007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:04.959 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.965+0000 7f2ccffff640 1 -- 172.21.15.43:0/3316458880 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd800f9c0 con 0x7f2cd0004800 2024-04-25T08:54:04.959 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.965+0000 7f2ccffff640 1 -- 172.21.15.43:0/3316458880 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cd8018070 con 0x7f2cd0004800 2024-04-25T08:54:04.959 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.965+0000 7f2cedf5c640 1 -- 172.21.15.43:0/3316458880 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ce815fb10 con 0x7f2cd0004800 2024-04-25T08:54:04.960 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.965+0000 7f2ccffff640 1 -- 172.21.15.43:0/3316458880 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd800fde0 con 0x7f2cd0004800 2024-04-25T08:54:04.960 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.965+0000 7f2cedf5c640 1 -- 172.21.15.43:0/3316458880 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ce81600d0 con 0x7f2cd0004800 2024-04-25T08:54:04.960 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.966+0000 7f2cedf5c640 1 -- 172.21.15.43:0/3316458880 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ce80734e0 con 0x7f2cd0004800 2024-04-25T08:54:04.960 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.966+0000 7f2ccffff640 1 -- 172.21.15.43:0/3316458880 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2cd801d460 con 0x7f2cd0004800 2024-04-25T08:54:04.960 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.967+0000 7f2ccffff640 1 --2- 172.21.15.43:0/3316458880 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2cd406b4d0 0x7f2cd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:04.960 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.967+0000 7f2ccffff640 1 -- 172.21.15.43:0/3316458880 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2cd808a040 con 0x7f2cd0004800 2024-04-25T08:54:04.960 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.967+0000 7f2ce6ffd640 1 --2- 172.21.15.43:0/3316458880 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2cd406b4d0 0x7f2cd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:04.960 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.967+0000 7f2ce6ffd640 1 --2- 172.21.15.43:0/3316458880 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2cd406b4d0 0x7f2cd406d990 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc033000 tx=0x7f2cdc008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:04.961 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:04.971+0000 7f2ccffff640 1 -- 172.21.15.43:0/3316458880 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2cd8059180 con 0x7f2cd0004800 2024-04-25T08:54:05.142 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:05.152+0000 7f2cedf5c640 1 -- 172.21.15.43:0/3316458880 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ce80d0f50 con 0x7f2cd0004800 2024-04-25T08:54:05.143 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:05.153+0000 7f2ccffff640 1 -- 172.21.15.43:0/3316458880 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12848 (secure 0 0 0) 0x7f2cd8058a80 con 0x7f2cd0004800 2024-04-25T08:54:05.143 INFO:teuthology.misc.health.smithi043.stdout: 2024-04-25T08:54:05.143 INFO:teuthology.misc.health.smithi043.stdout:{"epoch":16,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:54:01.854785+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:54:05.146 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:05.155+0000 7f2cedf5c640 1 -- 172.21.15.43:0/3316458880 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2cd406b4d0 msgr2=0x7f2cd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:05.146 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:05.155+0000 7f2cedf5c640 1 --2- 172.21.15.43:0/3316458880 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2cd406b4d0 0x7f2cd406d990 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc033000 tx=0x7f2cdc008030 comp rx=0 tx=0).stop 2024-04-25T08:54:05.146 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:05.155+0000 7f2cedf5c640 1 -- 172.21.15.43:0/3316458880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cd0004800 msgr2=0x7f2ce814ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:05.146 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:05.155+0000 7f2cedf5c640 1 --2- 172.21.15.43:0/3316458880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cd0004800 0x7f2ce814ae90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8007340 tx=0x7f2cd8007800 comp rx=0 tx=0).stop 2024-04-25T08:54:05.146 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:05.155+0000 7f2cedf5c640 1 -- 172.21.15.43:0/3316458880 shutdown_connections 2024-04-25T08:54:05.146 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:05.155+0000 7f2cedf5c640 1 --2- 172.21.15.43:0/3316458880 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2cd406b4d0 0x7f2cd406d990 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.146 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:05.155+0000 7f2cedf5c640 1 --2- 172.21.15.43:0/3316458880 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ce80c8810 0x7f2ce80b84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.146 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:05.155+0000 7f2cedf5c640 1 --2- 172.21.15.43:0/3316458880 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2ce80ca300 0x7f2ce81f3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.147 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:05.155+0000 7f2cedf5c640 1 --2- 172.21.15.43:0/3316458880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2cd0004800 0x7f2ce814ae90 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.147 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:05.155+0000 7f2cedf5c640 1 -- 172.21.15.43:0/3316458880 >> 172.21.15.43:0/3316458880 conn(0x7f2ce809ccf0 msgr2=0x7f2ce806c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:05.147 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:05.155+0000 7f2cedf5c640 1 -- 172.21.15.43:0/3316458880 shutdown_connections 2024-04-25T08:54:05.147 INFO:teuthology.misc.health.smithi043.stderr:2024-04-25T08:54:05.155+0000 7f2cedf5c640 1 -- 172.21.15.43:0/3316458880 wait complete. 2024-04-25T08:54:05.156 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-25T08:54:05.156 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-25T08:54:05.156 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T08:54:05.156 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T08:54:05.159 INFO:tasks.ceph.mds.a:Started 2024-04-25T08:54:05.159 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T08:54:05.159 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T08:54:05.161 INFO:tasks.ceph.mds.c:Started 2024-04-25T08:54:05.161 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T08:54:05.161 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T08:54:05.164 INFO:tasks.ceph.mds.b:Started 2024-04-25T08:54:05.164 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T08:54:05.164 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T08:54:05.166 INFO:tasks.ceph.mds.d:Started 2024-04-25T08:54:05.166 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-25T08:54:05.167 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:54:05.234 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2024-04-25T08:54:05.234 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-04-25T08:54:05.247 INFO:tasks.ceph.mds.b.smithi193.stdout:starting mds.b at 2024-04-25T08:54:05.249 INFO:tasks.ceph.mds.d.smithi193.stdout:starting mds.d at 2024-04-25T08:54:05.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.282+0000 7f23b344c640 1 Processor -- start 2024-04-25T08:54:05.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.282+0000 7f23b344c640 1 -- start start 2024-04-25T08:54:05.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.283+0000 7f23b344c640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f23ac072ac0 0x7f23ac161f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:05.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.283+0000 7f23b344c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f23ac15e6e0 0x7f23ac15eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:05.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.283+0000 7f23b344c640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23ac0b3170 con 0x7f23ac0b3b90 2024-04-25T08:54:05.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.283+0000 7f23b344c640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23ac0b33e0 con 0x7f23ac072ac0 2024-04-25T08:54:05.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.283+0000 7f23b344c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23ac161790 con 0x7f23ac15e6e0 2024-04-25T08:54:05.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.283+0000 7f23b11e8640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f23ac0b3b90 0x7f23ac0b3f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41878/0 (socket says 172.21.15.43:41878) 2024-04-25T08:54:05.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.283+0000 7f23b11e8640 1 -- 172.21.15.43:0/1330811962 learned_addr learned my addr 172.21.15.43:0/1330811962 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:05.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.283+0000 7f23b19e9640 1 --2- 172.21.15.43:0/1330811962 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f23ac15e6e0 0x7f23ac15eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:05.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.283+0000 7f23a3fff640 1 -- 172.21.15.43:0/1330811962 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1494420192 0 0) 0x7f23ac0b3170 con 0x7f23ac0b3b90 2024-04-25T08:54:05.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.283+0000 7f23a3fff640 1 -- 172.21.15.43:0/1330811962 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2398003520 con 0x7f23ac0b3b90 2024-04-25T08:54:05.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.283+0000 7f23b09e7640 1 --2- 172.21.15.43:0/1330811962 >> v2:172.21.15.193:3300/0 conn(0x7f23ac072ac0 0x7f23ac161f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:05.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.283+0000 7f23a3fff640 1 -- 172.21.15.43:0/1330811962 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3276137219 0 0) 0x7f2398003520 con 0x7f23ac0b3b90 2024-04-25T08:54:05.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.283+0000 7f23a3fff640 1 -- 172.21.15.43:0/1330811962 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f23ac15e6e0 msgr2=0x7f23ac15eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:05.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.283+0000 7f23a3fff640 1 --2- 172.21.15.43:0/1330811962 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f23ac15e6e0 0x7f23ac15eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.283+0000 7f23a3fff640 1 -- 172.21.15.43:0/1330811962 >> v2:172.21.15.193:3300/0 conn(0x7f23ac072ac0 msgr2=0x7f23ac161f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:05.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.283+0000 7f23a3fff640 1 --2- 172.21.15.43:0/1330811962 >> v2:172.21.15.193:3300/0 conn(0x7f23ac072ac0 0x7f23ac161f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.284+0000 7f23a3fff640 1 -- 172.21.15.43:0/1330811962 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23ac161a70 con 0x7f23ac0b3b90 2024-04-25T08:54:05.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.284+0000 7f23a3fff640 1 -- 172.21.15.43:0/1330811962 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f23a8001670 con 0x7f23ac0b3b90 2024-04-25T08:54:05.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.284+0000 7f23a3fff640 1 -- 172.21.15.43:0/1330811962 >> v1:172.21.15.43:6789/0 conn(0x7f23ac0b3b90 legacy=0x7f23ac0b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:05.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.284+0000 7f23a3fff640 1 --2- 172.21.15.43:0/1330811962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23980039b0 0x7f2398003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:05.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.284+0000 7f23a3fff640 1 --2- 172.21.15.43:0/1330811962 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f23ac15e6e0 0x7f2398004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:05.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.284+0000 7f23a3fff640 1 --2- 172.21.15.43:0/1330811962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2398004d40 0x7f23980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:05.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.284+0000 7f23a3fff640 1 -- 172.21.15.43:0/1330811962 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23ac0b33e0 con 0x7f2398004d40 2024-04-25T08:54:05.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.284+0000 7f23a3fff640 1 -- 172.21.15.43:0/1330811962 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23ac161790 con 0x7f23980039b0 2024-04-25T08:54:05.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.284+0000 7f23a3fff640 1 -- 172.21.15.43:0/1330811962 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f239800b9c0 con 0x7f23ac15e6e0 2024-04-25T08:54:05.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.284+0000 7f23b19e9640 1 --2- 172.21.15.43:0/1330811962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2398004d40 0x7f23980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:05.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.284+0000 7f23b11e8640 1 --2- 172.21.15.43:0/1330811962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23980039b0 0x7f2398003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:05.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.284+0000 7f23b09e7640 1 --2- 172.21.15.43:0/1330811962 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f23ac15e6e0 0x7f2398004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:05.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.284+0000 7f23b19e9640 1 -- 172.21.15.43:0/1330811962 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f23ac15e6e0 msgr2=0x7f2398004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:05.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.284+0000 7f23b19e9640 1 --2- 172.21.15.43:0/1330811962 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f23ac15e6e0 0x7f2398004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.284+0000 7f23b19e9640 1 -- 172.21.15.43:0/1330811962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23980039b0 msgr2=0x7f2398003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:05.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.284+0000 7f23b19e9640 1 --2- 172.21.15.43:0/1330811962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23980039b0 0x7f2398003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.284+0000 7f23b19e9640 1 -- 172.21.15.43:0/1330811962 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2398003520 con 0x7f2398004d40 2024-04-25T08:54:05.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.284+0000 7f23b11e8640 1 --2- 172.21.15.43:0/1330811962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23980039b0 0x7f2398003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:54:05.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.285+0000 7f23b19e9640 1 --2- 172.21.15.43:0/1330811962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2398004d40 0x7f23980051c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f23a400e170 tx=0x7f2398005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=721dade82c93ba5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:05.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.285+0000 7f23a3fff640 1 -- 172.21.15.43:0/1330811962 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a4034020 con 0x7f2398004d40 2024-04-25T08:54:05.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.285+0000 7f23a3fff640 1 -- 172.21.15.43:0/1330811962 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a40305f0 con 0x7f2398004d40 2024-04-25T08:54:05.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.285+0000 7f23a3fff640 1 -- 172.21.15.43:0/1330811962 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a4009c10 con 0x7f2398004d40 2024-04-25T08:54:05.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.285+0000 7f23b344c640 1 -- 172.21.15.43:0/1330811962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2398004d40 msgr2=0x7f23980051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:05.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.285+0000 7f23b344c640 1 --2- 172.21.15.43:0/1330811962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2398004d40 0x7f23980051c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f23a400e170 tx=0x7f2398005830 comp rx=0 tx=0).stop 2024-04-25T08:54:05.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.285+0000 7f23b19e9640 1 -- 172.21.15.43:0/1330811962 reap_dead start 2024-04-25T08:54:05.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.285+0000 7f23b344c640 1 -- 172.21.15.43:0/1330811962 shutdown_connections 2024-04-25T08:54:05.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.285+0000 7f23b344c640 1 -- 172.21.15.43:0/1330811962 >> 172.21.15.43:0/1330811962 conn(0x7f23ac06d950 msgr2=0x7f23ac06dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:05.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.286+0000 7f23b344c640 1 -- 172.21.15.43:0/1330811962 shutdown_connections 2024-04-25T08:54:05.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.286+0000 7f23b344c640 1 -- 172.21.15.43:0/1330811962 wait complete. 2024-04-25T08:54:05.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.287+0000 7f23b344c640 1 Processor -- start 2024-04-25T08:54:05.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.287+0000 7f23b344c640 1 -- start start 2024-04-25T08:54:05.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.287+0000 7f23b344c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23ac142830 0x7f23ac143150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:05.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.287+0000 7f23b344c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23ac143690 0x7f23ac15b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:05.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.287+0000 7f23b344c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f23ac142c10 0x7f23ac15bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:05.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.287+0000 7f23b344c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23ac15ecf0 con 0x7f23ac142830 2024-04-25T08:54:05.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.287+0000 7f23b344c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23ac15eb70 con 0x7f23ac143690 2024-04-25T08:54:05.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.287+0000 7f23b344c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23ac15ee70 con 0x7f23ac142c10 2024-04-25T08:54:05.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.288+0000 7f23b11e8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23ac142830 0x7f23ac143150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:05.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.288+0000 7f23b11e8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23ac142830 0x7f23ac143150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41400/0 (socket says 172.21.15.43:41400) 2024-04-25T08:54:05.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.288+0000 7f23b11e8640 1 -- 172.21.15.43:0/1657773840 learned_addr learned my addr 172.21.15.43:0/1657773840 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:05.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.288+0000 7f23b09e7640 1 --2- 172.21.15.43:0/1657773840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23ac143690 0x7f23ac15b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:05.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.288+0000 7f23b19e9640 1 --2- 172.21.15.43:0/1657773840 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f23ac142c10 0x7f23ac15bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:05.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.288+0000 7f23b11e8640 1 -- 172.21.15.43:0/1657773840 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f23ac142c10 msgr2=0x7f23ac15bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:05.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.288+0000 7f23b11e8640 1 --2- 172.21.15.43:0/1657773840 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f23ac142c10 0x7f23ac15bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.288+0000 7f23b11e8640 1 -- 172.21.15.43:0/1657773840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23ac143690 msgr2=0x7f23ac15b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:05.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.288+0000 7f23b11e8640 1 --2- 172.21.15.43:0/1657773840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23ac143690 0x7f23ac15b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.288+0000 7f23b11e8640 1 -- 172.21.15.43:0/1657773840 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23ac159c10 con 0x7f23ac142830 2024-04-25T08:54:05.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.288+0000 7f23b11e8640 1 --2- 172.21.15.43:0/1657773840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23ac142830 0x7f23ac143150 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f23a8007fd0 tx=0x7f23a80076e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:05.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.288+0000 7f23a1ffb640 1 -- 172.21.15.43:0/1657773840 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a801a070 con 0x7f23ac142830 2024-04-25T08:54:05.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.288+0000 7f23a1ffb640 1 -- 172.21.15.43:0/1657773840 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a800f9f0 con 0x7f23ac142830 2024-04-25T08:54:05.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.288+0000 7f23b344c640 1 -- 172.21.15.43:0/1657773840 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23ac159ef0 con 0x7f23ac142830 2024-04-25T08:54:05.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.288+0000 7f23a1ffb640 1 -- 172.21.15.43:0/1657773840 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a800fda0 con 0x7f23ac142830 2024-04-25T08:54:05.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.288+0000 7f23b344c640 1 -- 172.21.15.43:0/1657773840 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23ac15a4b0 con 0x7f23ac142830 2024-04-25T08:54:05.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.289+0000 7f23b344c640 1 -- 172.21.15.43:0/1657773840 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23ac161f90 con 0x7f23ac142830 2024-04-25T08:54:05.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.290+0000 7f23a1ffb640 1 -- 172.21.15.43:0/1657773840 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f23a801d070 con 0x7f23ac142830 2024-04-25T08:54:05.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.290+0000 7f23a1ffb640 1 --2- 172.21.15.43:0/1657773840 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f239006b4d0 0x7f239006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:05.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.291+0000 7f23a1ffb640 1 -- 172.21.15.43:0/1657773840 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f23a8089b40 con 0x7f23ac142830 2024-04-25T08:54:05.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.291+0000 7f23b09e7640 1 --2- 172.21.15.43:0/1657773840 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f239006b4d0 0x7f239006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:05.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.291+0000 7f23b09e7640 1 --2- 172.21.15.43:0/1657773840 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f239006b4d0 0x7f239006d990 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f23ac143a70 tx=0x7f239c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:05.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.295+0000 7f23a1ffb640 1 -- 172.21.15.43:0/1657773840 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f23ac161f90 con 0x7f23ac142830 2024-04-25T08:54:05.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.474+0000 7f23b344c640 1 -- 172.21.15.43:0/1657773840 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23ac074ef0 con 0x7f23ac142830 2024-04-25T08:54:05.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.475+0000 7f23a1ffb640 1 -- 172.21.15.43:0/1657773840 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12848 (secure 0 0 0) 0x7f23a8058630 con 0x7f23ac142830 2024-04-25T08:54:05.465 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:54:05.465 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":16,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:54:01.854785+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:54:05.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.477+0000 7f23b344c640 1 -- 172.21.15.43:0/1657773840 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f239006b4d0 msgr2=0x7f239006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:05.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.477+0000 7f23b344c640 1 --2- 172.21.15.43:0/1657773840 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f239006b4d0 0x7f239006d990 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f23ac143a70 tx=0x7f239c009190 comp rx=0 tx=0).stop 2024-04-25T08:54:05.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.477+0000 7f23b344c640 1 -- 172.21.15.43:0/1657773840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23ac142830 msgr2=0x7f23ac143150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:05.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.477+0000 7f23b344c640 1 --2- 172.21.15.43:0/1657773840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23ac142830 0x7f23ac143150 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f23a8007fd0 tx=0x7f23a80076e0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.477+0000 7f23b344c640 1 -- 172.21.15.43:0/1657773840 shutdown_connections 2024-04-25T08:54:05.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.477+0000 7f23b344c640 1 --2- 172.21.15.43:0/1657773840 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f23ac142c10 0x7f23ac15bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.477+0000 7f23b344c640 1 --2- 172.21.15.43:0/1657773840 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f239006b4d0 0x7f239006d990 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.477+0000 7f23b344c640 1 --2- 172.21.15.43:0/1657773840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23ac143690 0x7f23ac15b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.477+0000 7f23b344c640 1 --2- 172.21.15.43:0/1657773840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23ac142830 0x7f23ac143150 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.477+0000 7f23b344c640 1 -- 172.21.15.43:0/1657773840 >> 172.21.15.43:0/1657773840 conn(0x7f23ac06d950 msgr2=0x7f23ac06be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:05.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.477+0000 7f23b344c640 1 -- 172.21.15.43:0/1657773840 shutdown_connections 2024-04-25T08:54:05.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.477+0000 7f23b344c640 1 -- 172.21.15.43:0/1657773840 wait complete. 2024-04-25T08:54:05.478 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:54:05.478 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:54:05.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.592+0000 7f0d7a1f4640 1 Processor -- start 2024-04-25T08:54:05.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.592+0000 7f0d7a1f4640 1 -- start start 2024-04-25T08:54:05.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.593+0000 7f0d7a1f4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0d7415e090 0x7f0d74161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:05.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.593+0000 7f0d7a1f4640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d74162610 con 0x7f0d740b5c70 2024-04-25T08:54:05.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.593+0000 7f0d7a1f4640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d740d1750 con 0x7f0d74161fe0 2024-04-25T08:54:05.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.593+0000 7f0d7a1f4640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d740d1da0 con 0x7f0d7415e090 2024-04-25T08:54:05.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.593+0000 7f0d737fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0d7415e090 0x7f0d74161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:05.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.593+0000 7f0d73fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f0d740b5c70 0x7f0d740d2230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41886/0 (socket says 172.21.15.43:41886) 2024-04-25T08:54:05.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.593+0000 7f0d73fff640 1 -- 172.21.15.43:0/1946535497 learned_addr learned my addr 172.21.15.43:0/1946535497 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:05.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.593+0000 7f0d737fe640 1 -- 172.21.15.43:0/1946535497 >> v1:172.21.15.193:6789/0 conn(0x7f0d74161fe0 legacy=0x7f0d740b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:05.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.593+0000 7f0d737fe640 1 -- 172.21.15.43:0/1946535497 >> v1:172.21.15.43:6789/0 conn(0x7f0d740b5c70 legacy=0x7f0d740d2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:05.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.594+0000 7f0d737fe640 1 -- 172.21.15.43:0/1946535497 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d740d1a30 con 0x7f0d7415e090 2024-04-25T08:54:05.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.593+0000 7f0d727fc640 1 -- 172.21.15.43:0/1946535497 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4236939621 0 0) 0x7f0d74162610 con 0x7f0d740b5c70 2024-04-25T08:54:05.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.594+0000 7f0d737fe640 1 --2- 172.21.15.43:0/1946535497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d7415e090 0x7f0d74161aa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0d640097a0 tx=0x7f0d6402a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92fbd18ad907157e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:05.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.594+0000 7f0d727fc640 1 -- 172.21.15.43:0/1946535497 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d64037070 con 0x7f0d7415e090 2024-04-25T08:54:05.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.594+0000 7f0d727fc640 1 -- 172.21.15.43:0/1946535497 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d64032640 con 0x7f0d7415e090 2024-04-25T08:54:05.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.594+0000 7f0d727fc640 1 -- 172.21.15.43:0/1946535497 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d64032a10 con 0x7f0d7415e090 2024-04-25T08:54:05.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.595+0000 7f0d7a1f4640 1 -- 172.21.15.43:0/1946535497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d7415e090 msgr2=0x7f0d74161aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:05.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.595+0000 7f0d7a1f4640 1 --2- 172.21.15.43:0/1946535497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d7415e090 0x7f0d74161aa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0d640097a0 tx=0x7f0d6402a2c0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.595+0000 7f0d7a1f4640 1 -- 172.21.15.43:0/1946535497 shutdown_connections 2024-04-25T08:54:05.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.595+0000 7f0d7a1f4640 1 --2- 172.21.15.43:0/1946535497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d7415e090 0x7f0d74161aa0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.595+0000 7f0d7a1f4640 1 -- 172.21.15.43:0/1946535497 >> 172.21.15.43:0/1946535497 conn(0x7f0d740d0560 msgr2=0x7f0d740d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:05.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.595+0000 7f0d7a1f4640 1 -- 172.21.15.43:0/1946535497 shutdown_connections 2024-04-25T08:54:05.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.595+0000 7f0d7a1f4640 1 -- 172.21.15.43:0/1946535497 wait complete. 2024-04-25T08:54:05.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.596+0000 7f0d7a1f4640 1 Processor -- start 2024-04-25T08:54:05.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.596+0000 7f0d7a1f4640 1 -- start start 2024-04-25T08:54:05.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.597+0000 7f0d7a1f4640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0d740b5c70 0x7f0d740b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:05.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.597+0000 7f0d7a1f4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d7415e090 0x7f0d740cde30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:05.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.597+0000 7f0d7a1f4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d74161fe0 0x7f0d740ce370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:05.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.597+0000 7f0d7a1f4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d740d2940 con 0x7f0d7415e090 2024-04-25T08:54:05.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.597+0000 7f0d7a1f4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d740b3280 con 0x7f0d74161fe0 2024-04-25T08:54:05.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.597+0000 7f0d7a1f4640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d740d2ac0 con 0x7f0d740b5c70 2024-04-25T08:54:05.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.597+0000 7f0d72ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d7415e090 0x7f0d740cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:05.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.597+0000 7f0d72ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d7415e090 0x7f0d740cde30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41418/0 (socket says 172.21.15.43:41418) 2024-04-25T08:54:05.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.597+0000 7f0d72ffd640 1 -- 172.21.15.43:0/1709997479 learned_addr learned my addr 172.21.15.43:0/1709997479 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:05.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.597+0000 7f0d737fe640 1 --2- 172.21.15.43:0/1709997479 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0d740b5c70 0x7f0d740b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:05.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.597+0000 7f0d73fff640 1 --2- 172.21.15.43:0/1709997479 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d74161fe0 0x7f0d740ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:05.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.597+0000 7f0d72ffd640 1 -- 172.21.15.43:0/1709997479 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0d740b5c70 msgr2=0x7f0d740b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:05.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.597+0000 7f0d72ffd640 1 --2- 172.21.15.43:0/1709997479 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0d740b5c70 0x7f0d740b1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.597+0000 7f0d72ffd640 1 -- 172.21.15.43:0/1709997479 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d74161fe0 msgr2=0x7f0d740ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:05.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.597+0000 7f0d72ffd640 1 --2- 172.21.15.43:0/1709997479 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d74161fe0 0x7f0d740ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.597+0000 7f0d72ffd640 1 -- 172.21.15.43:0/1709997479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d74160c00 con 0x7f0d7415e090 2024-04-25T08:54:05.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.598+0000 7f0d72ffd640 1 --2- 172.21.15.43:0/1709997479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d7415e090 0x7f0d740cde30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0d6800c400 tx=0x7f0d6800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:05.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.598+0000 7f0d5bfff640 1 -- 172.21.15.43:0/1709997479 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d68017070 con 0x7f0d7415e090 2024-04-25T08:54:05.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.598+0000 7f0d5bfff640 1 -- 172.21.15.43:0/1709997479 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d680052d0 con 0x7f0d7415e090 2024-04-25T08:54:05.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.598+0000 7f0d7a1f4640 1 -- 172.21.15.43:0/1709997479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d74149120 con 0x7f0d7415e090 2024-04-25T08:54:05.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.598+0000 7f0d5bfff640 1 -- 172.21.15.43:0/1709997479 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d68003830 con 0x7f0d7415e090 2024-04-25T08:54:05.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.598+0000 7f0d7a1f4640 1 -- 172.21.15.43:0/1709997479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d741496e0 con 0x7f0d7415e090 2024-04-25T08:54:05.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.599+0000 7f0d7a1f4640 1 -- 172.21.15.43:0/1709997479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d7406bf00 con 0x7f0d7415e090 2024-04-25T08:54:05.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.599+0000 7f0d5bfff640 1 -- 172.21.15.43:0/1709997479 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0d6801a070 con 0x7f0d7415e090 2024-04-25T08:54:05.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.600+0000 7f0d5bfff640 1 --2- 172.21.15.43:0/1709997479 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0d6006b4d0 0x7f0d6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:05.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.600+0000 7f0d5bfff640 1 -- 172.21.15.43:0/1709997479 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0d68089210 con 0x7f0d7415e090 2024-04-25T08:54:05.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.600+0000 7f0d737fe640 1 --2- 172.21.15.43:0/1709997479 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0d6006b4d0 0x7f0d6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:05.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.600+0000 7f0d737fe640 1 --2- 172.21.15.43:0/1709997479 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0d6006b4d0 0x7f0d6006d990 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0d64034000 tx=0x7f0d6402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:05.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.604+0000 7f0d5bfff640 1 -- 172.21.15.43:0/1709997479 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0d7406bf00 con 0x7f0d7415e090 2024-04-25T08:54:05.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.784+0000 7f0d7a1f4640 1 -- 172.21.15.43:0/1709997479 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0d74074f60 con 0x7f0d7415e090 2024-04-25T08:54:05.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.785+0000 7f0d5bfff640 1 -- 172.21.15.43:0/1709997479 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0d680583e0 con 0x7f0d7415e090 2024-04-25T08:54:05.775 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T08:54:05.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.787+0000 7f0d7a1f4640 1 -- 172.21.15.43:0/1709997479 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0d6006b4d0 msgr2=0x7f0d6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:05.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.787+0000 7f0d7a1f4640 1 --2- 172.21.15.43:0/1709997479 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0d6006b4d0 0x7f0d6006d990 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0d64034000 tx=0x7f0d6402e000 comp rx=0 tx=0).stop 2024-04-25T08:54:05.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.787+0000 7f0d7a1f4640 1 -- 172.21.15.43:0/1709997479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d7415e090 msgr2=0x7f0d740cde30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:05.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.787+0000 7f0d7a1f4640 1 --2- 172.21.15.43:0/1709997479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d7415e090 0x7f0d740cde30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0d6800c400 tx=0x7f0d6800c8c0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.787+0000 7f0d7a1f4640 1 -- 172.21.15.43:0/1709997479 shutdown_connections 2024-04-25T08:54:05.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.787+0000 7f0d7a1f4640 1 --2- 172.21.15.43:0/1709997479 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0d6006b4d0 0x7f0d6006d990 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.787+0000 7f0d7a1f4640 1 --2- 172.21.15.43:0/1709997479 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d74161fe0 0x7f0d740ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.787+0000 7f0d7a1f4640 1 --2- 172.21.15.43:0/1709997479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d7415e090 0x7f0d740cde30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.787+0000 7f0d7a1f4640 1 --2- 172.21.15.43:0/1709997479 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0d740b5c70 0x7f0d740b1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.787+0000 7f0d7a1f4640 1 -- 172.21.15.43:0/1709997479 >> 172.21.15.43:0/1709997479 conn(0x7f0d740d0560 msgr2=0x7f0d740cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:05.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.787+0000 7f0d7a1f4640 1 -- 172.21.15.43:0/1709997479 shutdown_connections 2024-04-25T08:54:05.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.787+0000 7f0d7a1f4640 1 -- 172.21.15.43:0/1709997479 wait complete. 2024-04-25T08:54:05.788 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:54:05.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.898+0000 7fb468612640 1 Processor -- start 2024-04-25T08:54:05.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.898+0000 7fb468612640 1 -- start start 2024-04-25T08:54:05.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.899+0000 7fb468612640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb46015df50 0x7fb4601619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:05.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.899+0000 7fb468612640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb460161f30 0x7fb460162380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:05.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.899+0000 7fb468612640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb4600c7cf0 0x7fb4600c8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:05.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.899+0000 7fb468612640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb46015e330 con 0x7fb460161f30 2024-04-25T08:54:05.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.899+0000 7fb468612640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4600d24a0 con 0x7fb46015df50 2024-04-25T08:54:05.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.899+0000 7fb468612640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4600d2740 con 0x7fb4600c7cf0 2024-04-25T08:54:05.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.899+0000 7fb465bad640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb460161f30 0x7fb460162380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:05.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.899+0000 7fb465bad640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb460161f30 0x7fb460162380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41434/0 (socket says 172.21.15.43:41434) 2024-04-25T08:54:05.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.899+0000 7fb465bad640 1 -- 172.21.15.43:0/1944273878 learned_addr learned my addr 172.21.15.43:0/1944273878 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:05.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.899+0000 7fb4663ae640 1 --2- 172.21.15.43:0/1944273878 >> v2:172.21.15.193:3300/0 conn(0x7fb46015df50 0x7fb4601619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:05.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.899+0000 7fb466baf640 1 --2- 172.21.15.43:0/1944273878 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb4600c7cf0 0x7fb4600c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:05.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.899+0000 7fb465bad640 1 -- 172.21.15.43:0/1944273878 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb4600c7cf0 msgr2=0x7fb4600c8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:05.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.899+0000 7fb465bad640 1 --2- 172.21.15.43:0/1944273878 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb4600c7cf0 0x7fb4600c8140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.899+0000 7fb465bad640 1 -- 172.21.15.43:0/1944273878 >> v2:172.21.15.193:3300/0 conn(0x7fb46015df50 msgr2=0x7fb4601619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:05.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.899+0000 7fb465bad640 1 --2- 172.21.15.43:0/1944273878 >> v2:172.21.15.193:3300/0 conn(0x7fb46015df50 0x7fb4601619f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.899+0000 7fb465bad640 1 -- 172.21.15.43:0/1944273878 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4600d2a20 con 0x7fb460161f30 2024-04-25T08:54:05.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.900+0000 7fb465bad640 1 --2- 172.21.15.43:0/1944273878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb460161f30 0x7fb460162380 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb46008c880 tx=0x7fb45402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea5937f3659600b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:05.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.900+0000 7fb4653ac640 1 -- 172.21.15.43:0/1944273878 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb454037070 con 0x7fb460161f30 2024-04-25T08:54:05.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.900+0000 7fb4653ac640 1 -- 172.21.15.43:0/1944273878 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb454032440 con 0x7fb460161f30 2024-04-25T08:54:05.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.900+0000 7fb4653ac640 1 -- 172.21.15.43:0/1944273878 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb454032810 con 0x7fb460161f30 2024-04-25T08:54:05.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.901+0000 7fb468612640 1 -- 172.21.15.43:0/1944273878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb460161f30 msgr2=0x7fb460162380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:05.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.901+0000 7fb468612640 1 --2- 172.21.15.43:0/1944273878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb460161f30 0x7fb460162380 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb46008c880 tx=0x7fb45402a1e0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.901+0000 7fb468612640 1 -- 172.21.15.43:0/1944273878 shutdown_connections 2024-04-25T08:54:05.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.901+0000 7fb468612640 1 --2- 172.21.15.43:0/1944273878 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb4600c7cf0 0x7fb4600c8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.901+0000 7fb468612640 1 --2- 172.21.15.43:0/1944273878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb460161f30 0x7fb460162380 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.901+0000 7fb468612640 1 --2- 172.21.15.43:0/1944273878 >> v2:172.21.15.193:3300/0 conn(0x7fb46015df50 0x7fb4601619f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.901+0000 7fb468612640 1 -- 172.21.15.43:0/1944273878 >> 172.21.15.43:0/1944273878 conn(0x7fb4600d3a30 msgr2=0x7fb4600d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:05.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.901+0000 7fb468612640 1 -- 172.21.15.43:0/1944273878 shutdown_connections 2024-04-25T08:54:05.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.901+0000 7fb468612640 1 -- 172.21.15.43:0/1944273878 wait complete. 2024-04-25T08:54:05.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.902+0000 7fb468612640 1 Processor -- start 2024-04-25T08:54:05.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.902+0000 7fb468612640 1 -- start start 2024-04-25T08:54:05.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.903+0000 7fb468612640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb4600c7cf0 0x7fb46015f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:05.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.903+0000 7fb468612640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb46015df50 0x7fb4601f3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:05.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.903+0000 7fb468612640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb460161f30 0x7fb460156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:05.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.903+0000 7fb468612640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4600d2190 con 0x7fb46015df50 2024-04-25T08:54:05.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.903+0000 7fb468612640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4600c77d0 con 0x7fb460161f30 2024-04-25T08:54:05.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.903+0000 7fb468612640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4600c8680 con 0x7fb4600c7cf0 2024-04-25T08:54:05.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.903+0000 7fb465bad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb46015df50 0x7fb4601f3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:05.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.903+0000 7fb465bad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb46015df50 0x7fb4601f3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41436/0 (socket says 172.21.15.43:41436) 2024-04-25T08:54:05.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.903+0000 7fb465bad640 1 -- 172.21.15.43:0/3243486469 learned_addr learned my addr 172.21.15.43:0/3243486469 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:05.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.903+0000 7fb4663ae640 1 --2- 172.21.15.43:0/3243486469 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb4600c7cf0 0x7fb46015f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:05.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.903+0000 7fb466baf640 1 --2- 172.21.15.43:0/3243486469 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb460161f30 0x7fb460156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:05.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.903+0000 7fb465bad640 1 -- 172.21.15.43:0/3243486469 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb4600c7cf0 msgr2=0x7fb46015f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:05.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.903+0000 7fb465bad640 1 --2- 172.21.15.43:0/3243486469 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb4600c7cf0 0x7fb46015f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.903+0000 7fb465bad640 1 -- 172.21.15.43:0/3243486469 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb460161f30 msgr2=0x7fb460156350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:05.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.903+0000 7fb465bad640 1 --2- 172.21.15.43:0/3243486469 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb460161f30 0x7fb460156350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:05.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.903+0000 7fb465bad640 1 -- 172.21.15.43:0/3243486469 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb46015fb90 con 0x7fb46015df50 2024-04-25T08:54:05.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.903+0000 7fb465bad640 1 --2- 172.21.15.43:0/3243486469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb46015df50 0x7fb4601f3bb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb45402a220 tx=0x7fb4540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:05.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.903+0000 7fb452ffd640 1 -- 172.21.15.43:0/3243486469 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb454041020 con 0x7fb46015df50 2024-04-25T08:54:05.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.903+0000 7fb452ffd640 1 -- 172.21.15.43:0/3243486469 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4540306c0 con 0x7fb46015df50 2024-04-25T08:54:05.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.903+0000 7fb468612640 1 -- 172.21.15.43:0/3243486469 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb46015fe70 con 0x7fb46015df50 2024-04-25T08:54:05.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.903+0000 7fb452ffd640 1 -- 172.21.15.43:0/3243486469 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb454037040 con 0x7fb46015df50 2024-04-25T08:54:05.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.903+0000 7fb468612640 1 -- 172.21.15.43:0/3243486469 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb460159e90 con 0x7fb46015df50 2024-04-25T08:54:05.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.904+0000 7fb468612640 1 -- 172.21.15.43:0/3243486469 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4600680f0 con 0x7fb46015df50 2024-04-25T08:54:05.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.905+0000 7fb452ffd640 1 -- 172.21.15.43:0/3243486469 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb454044020 con 0x7fb46015df50 2024-04-25T08:54:05.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.905+0000 7fb452ffd640 1 --2- 172.21.15.43:0/3243486469 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb44406b4d0 0x7fb44406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:05.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.905+0000 7fb452ffd640 1 -- 172.21.15.43:0/3243486469 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb4540aa650 con 0x7fb46015df50 2024-04-25T08:54:05.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.905+0000 7fb4663ae640 1 --2- 172.21.15.43:0/3243486469 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb44406b4d0 0x7fb44406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:05.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.906+0000 7fb4663ae640 1 --2- 172.21.15.43:0/3243486469 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb44406b4d0 0x7fb44406d990 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb45c006fd0 tx=0x7fb45c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:05.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:05.910+0000 7fb452ffd640 1 -- 172.21.15.43:0/3243486469 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb45404b030 con 0x7fb46015df50 2024-04-25T08:54:06.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:06.134+0000 7fb468612640 1 -- 172.21.15.43:0/3243486469 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb460074ef0 con 0x7fb46015df50 2024-04-25T08:54:06.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:06.135+0000 7fb452ffd640 1 -- 172.21.15.43:0/3243486469 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1324 (secure 0 0 0) 0x7fb4540798b0 con 0x7fb46015df50 2024-04-25T08:54:06.125 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:54:06.125 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3616994003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":3616994003},{"type":"v1","addr":"172.21.15.43:6835","nonce":3616994003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-04-25T08:54:06.125 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 2 2024-04-25T08:54:06.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:06.137+0000 7fb468612640 1 -- 172.21.15.43:0/3243486469 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb44406b4d0 msgr2=0x7fb44406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:06.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:06.137+0000 7fb468612640 1 --2- 172.21.15.43:0/3243486469 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb44406b4d0 0x7fb44406d990 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb45c006fd0 tx=0x7fb45c008040 comp rx=0 tx=0).stop 2024-04-25T08:54:06.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:06.137+0000 7fb468612640 1 -- 172.21.15.43:0/3243486469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb46015df50 msgr2=0x7fb4601f3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:06.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:06.137+0000 7fb468612640 1 --2- 172.21.15.43:0/3243486469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb46015df50 0x7fb4601f3bb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb45402a220 tx=0x7fb4540046c0 comp rx=0 tx=0).stop 2024-04-25T08:54:06.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:06.137+0000 7fb468612640 1 -- 172.21.15.43:0/3243486469 shutdown_connections 2024-04-25T08:54:06.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:06.137+0000 7fb468612640 1 --2- 172.21.15.43:0/3243486469 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb44406b4d0 0x7fb44406d990 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:06.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:06.137+0000 7fb468612640 1 --2- 172.21.15.43:0/3243486469 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb460161f30 0x7fb460156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:06.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:06.137+0000 7fb468612640 1 --2- 172.21.15.43:0/3243486469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb46015df50 0x7fb4601f3bb0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:06.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:06.137+0000 7fb468612640 1 --2- 172.21.15.43:0/3243486469 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb4600c7cf0 0x7fb46015f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:06.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:06.137+0000 7fb468612640 1 -- 172.21.15.43:0/3243486469 >> 172.21.15.43:0/3243486469 conn(0x7fb4600d3a30 msgr2=0x7fb4600c89f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:06.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:06.137+0000 7fb468612640 1 -- 172.21.15.43:0/3243486469 shutdown_connections 2024-04-25T08:54:06.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:06.137+0000 7fb468612640 1 -- 172.21.15.43:0/3243486469 wait complete. 2024-04-25T08:54:08.138 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:54:08.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.261+0000 7f8f13dd5640 1 Processor -- start 2024-04-25T08:54:08.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.261+0000 7f8f13dd5640 1 -- start start 2024-04-25T08:54:08.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.262+0000 7f8f13dd5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f0c15df50 0x7f8f0c1619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:08.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.262+0000 7f8f13dd5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8f0c0cf670 0x7f8f0c162370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:08.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.262+0000 7f8f13dd5640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f0c0d24a0 con 0x7f8f0c161f30 2024-04-25T08:54:08.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.262+0000 7f8f13dd5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f0c0cfa80 con 0x7f8f0c0cf670 2024-04-25T08:54:08.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.262+0000 7f8f13dd5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c15e330 con 0x7f8f0c15df50 2024-04-25T08:54:08.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.262+0000 7f8f12372640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8f0c0cf670 0x7f8f0c162370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:08.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.262+0000 7f8f12372640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8f0c0cf670 0x7f8f0c162370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41450/0 (socket says 172.21.15.43:41450) 2024-04-25T08:54:08.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.262+0000 7f8f12372640 1 -- 172.21.15.43:0/2281761953 learned_addr learned my addr 172.21.15.43:0/2281761953 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:08.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.262+0000 7f8f11b71640 1 --2- 172.21.15.43:0/2281761953 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f0c15df50 0x7f8f0c1619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:08.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.262+0000 7f8f12372640 1 -- 172.21.15.43:0/2281761953 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f0c15df50 msgr2=0x7f8f0c1619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:08.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.262+0000 7f8f12372640 1 --2- 172.21.15.43:0/2281761953 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f0c15df50 0x7f8f0c1619f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.263+0000 7f8f12372640 1 -- 172.21.15.43:0/2281761953 >> v1:172.21.15.193:6789/0 conn(0x7f8f0c161f30 legacy=0x7f8f0c0c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:08.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.263+0000 7f8f12372640 1 -- 172.21.15.43:0/2281761953 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f0c0d2740 con 0x7f8f0c0cf670 2024-04-25T08:54:08.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.263+0000 7f8f11b71640 1 --2- 172.21.15.43:0/2281761953 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f0c15df50 0x7f8f0c1619f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:08.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.263+0000 7f8f12372640 1 --2- 172.21.15.43:0/2281761953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f0c0cf670 0x7f8f0c162370 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8f04009e50 tx=0x7f8f0402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbaa3bf87be8e0bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:08.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.263+0000 7f8f10b6f640 1 -- 172.21.15.43:0/2281761953 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f04036070 con 0x7f8f0c0cf670 2024-04-25T08:54:08.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.263+0000 7f8f10b6f640 1 -- 172.21.15.43:0/2281761953 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f04031600 con 0x7f8f0c0cf670 2024-04-25T08:54:08.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.263+0000 7f8f10b6f640 1 -- 172.21.15.43:0/2281761953 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f040319d0 con 0x7f8f0c0cf670 2024-04-25T08:54:08.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.264+0000 7f8f13dd5640 1 -- 172.21.15.43:0/2281761953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f0c0cf670 msgr2=0x7f8f0c162370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:08.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.264+0000 7f8f13dd5640 1 --2- 172.21.15.43:0/2281761953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f0c0cf670 0x7f8f0c162370 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8f04009e50 tx=0x7f8f0402a730 comp rx=0 tx=0).stop 2024-04-25T08:54:08.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.264+0000 7f8f13dd5640 1 -- 172.21.15.43:0/2281761953 shutdown_connections 2024-04-25T08:54:08.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.264+0000 7f8f13dd5640 1 --2- 172.21.15.43:0/2281761953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f0c0cf670 0x7f8f0c162370 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.264+0000 7f8f13dd5640 1 --2- 172.21.15.43:0/2281761953 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f0c15df50 0x7f8f0c1619f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.264+0000 7f8f13dd5640 1 -- 172.21.15.43:0/2281761953 >> 172.21.15.43:0/2281761953 conn(0x7f8f0c0d3a30 msgr2=0x7f8f0c0d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:08.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.264+0000 7f8f13dd5640 1 -- 172.21.15.43:0/2281761953 shutdown_connections 2024-04-25T08:54:08.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.264+0000 7f8f13dd5640 1 -- 172.21.15.43:0/2281761953 wait complete. 2024-04-25T08:54:08.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.265+0000 7f8f13dd5640 1 Processor -- start 2024-04-25T08:54:08.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.265+0000 7f8f13dd5640 1 -- start start 2024-04-25T08:54:08.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.266+0000 7f8f13dd5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f0c0cf670 0x7f8f0c152020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:08.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.266+0000 7f8f13dd5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f0c15df50 0x7f8f0c0cbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:08.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.266+0000 7f8f13dd5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f0c161f30 0x7f8f0c15cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:08.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.266+0000 7f8f13dd5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c0c77d0 con 0x7f8f0c15df50 2024-04-25T08:54:08.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.266+0000 7f8f13dd5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c0c74f0 con 0x7f8f0c0cf670 2024-04-25T08:54:08.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.266+0000 7f8f13dd5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f0c0d2190 con 0x7f8f0c161f30 2024-04-25T08:54:08.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.266+0000 7f8f11370640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f0c15df50 0x7f8f0c0cbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:08.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.266+0000 7f8f11370640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f0c15df50 0x7f8f0c0cbbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41466/0 (socket says 172.21.15.43:41466) 2024-04-25T08:54:08.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.266+0000 7f8f11370640 1 -- 172.21.15.43:0/863540325 learned_addr learned my addr 172.21.15.43:0/863540325 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:08.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.267+0000 7f8f11b71640 1 --2- 172.21.15.43:0/863540325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f0c0cf670 0x7f8f0c152020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:08.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.267+0000 7f8f12372640 1 --2- 172.21.15.43:0/863540325 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f0c161f30 0x7f8f0c15cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:08.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.267+0000 7f8f11370640 1 -- 172.21.15.43:0/863540325 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f0c161f30 msgr2=0x7f8f0c15cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:08.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.267+0000 7f8f11370640 1 --2- 172.21.15.43:0/863540325 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f0c161f30 0x7f8f0c15cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.267+0000 7f8f11370640 1 -- 172.21.15.43:0/863540325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f0c0cf670 msgr2=0x7f8f0c152020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:08.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.267+0000 7f8f11370640 1 --2- 172.21.15.43:0/863540325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f0c0cf670 0x7f8f0c152020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.267+0000 7f8f11370640 1 -- 172.21.15.43:0/863540325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f0c156580 con 0x7f8f0c15df50 2024-04-25T08:54:08.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.267+0000 7f8f11b71640 1 --2- 172.21.15.43:0/863540325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f0c0cf670 0x7f8f0c152020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:08.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.267+0000 7f8f11370640 1 --2- 172.21.15.43:0/863540325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f0c15df50 0x7f8f0c0cbbe0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8efc00c3c0 tx=0x7f8efc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:08.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.267+0000 7f8f027fc640 1 -- 172.21.15.43:0/863540325 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc0052d0 con 0x7f8f0c15df50 2024-04-25T08:54:08.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.267+0000 7f8f027fc640 1 -- 172.21.15.43:0/863540325 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8efc005490 con 0x7f8f0c15df50 2024-04-25T08:54:08.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.267+0000 7f8f13dd5640 1 -- 172.21.15.43:0/863540325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f0c156860 con 0x7f8f0c15df50 2024-04-25T08:54:08.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.267+0000 7f8f027fc640 1 -- 172.21.15.43:0/863540325 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8efc007260 con 0x7f8f0c15df50 2024-04-25T08:54:08.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.267+0000 7f8f13dd5640 1 -- 172.21.15.43:0/863540325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f0c156e20 con 0x7f8f0c15df50 2024-04-25T08:54:08.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.268+0000 7f8f13dd5640 1 -- 172.21.15.43:0/863540325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f0c0680f0 con 0x7f8f0c15df50 2024-04-25T08:54:08.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.268+0000 7f8f027fc640 1 -- 172.21.15.43:0/863540325 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8efc007400 con 0x7f8f0c15df50 2024-04-25T08:54:08.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.269+0000 7f8f027fc640 1 --2- 172.21.15.43:0/863540325 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8ef006b4d0 0x7f8ef006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:08.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.269+0000 7f8f027fc640 1 -- 172.21.15.43:0/863540325 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8efc0892c0 con 0x7f8f0c15df50 2024-04-25T08:54:08.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.269+0000 7f8f11b71640 1 --2- 172.21.15.43:0/863540325 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8ef006b4d0 0x7f8ef006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:08.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.269+0000 7f8f11b71640 1 --2- 172.21.15.43:0/863540325 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8ef006b4d0 0x7f8ef006d990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8f08006fd0 tx=0x7f8f08008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:08.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.273+0000 7f8f027fc640 1 -- 172.21.15.43:0/863540325 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8efc058520 con 0x7f8f0c15df50 2024-04-25T08:54:08.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.491+0000 7f8f13dd5640 1 -- 172.21.15.43:0/863540325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f0c074ef0 con 0x7f8f0c15df50 2024-04-25T08:54:08.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.491+0000 7f8f027fc640 1 -- 172.21.15.43:0/863540325 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3700 (secure 0 0 0) 0x7f8efc057e20 con 0x7f8f0c15df50 2024-04-25T08:54:08.482 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:54:08.482 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3616994003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":3616994003},{"type":"v1","addr":"172.21.15.43:6835","nonce":3616994003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/4022901908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4022901908},{"type":"v1","addr":"172.21.15.193:6835","nonce":4022901908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2823450590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2823450590},{"type":"v1","addr":"172.21.15.43:6834","nonce":2823450590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4294,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/1197434598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1197434598},{"type":"v1","addr":"172.21.15.193:6837","nonce":1197434598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-04-25T08:54:08.482 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 3 2024-04-25T08:54:08.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.493+0000 7f8f13dd5640 1 -- 172.21.15.43:0/863540325 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8ef006b4d0 msgr2=0x7f8ef006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:08.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.493+0000 7f8f13dd5640 1 --2- 172.21.15.43:0/863540325 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8ef006b4d0 0x7f8ef006d990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8f08006fd0 tx=0x7f8f08008040 comp rx=0 tx=0).stop 2024-04-25T08:54:08.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.493+0000 7f8f13dd5640 1 -- 172.21.15.43:0/863540325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f0c15df50 msgr2=0x7f8f0c0cbbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:08.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.493+0000 7f8f13dd5640 1 --2- 172.21.15.43:0/863540325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f0c15df50 0x7f8f0c0cbbe0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8efc00c3c0 tx=0x7f8efc00c880 comp rx=0 tx=0).stop 2024-04-25T08:54:08.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.493+0000 7f8f13dd5640 1 -- 172.21.15.43:0/863540325 shutdown_connections 2024-04-25T08:54:08.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.493+0000 7f8f13dd5640 1 --2- 172.21.15.43:0/863540325 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f0c161f30 0x7f8f0c15cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.493+0000 7f8f13dd5640 1 --2- 172.21.15.43:0/863540325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f0c15df50 0x7f8f0c0cbbe0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.493+0000 7f8f13dd5640 1 --2- 172.21.15.43:0/863540325 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8ef006b4d0 0x7f8ef006d990 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.493+0000 7f8f13dd5640 1 --2- 172.21.15.43:0/863540325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f0c0cf670 0x7f8f0c152020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.493+0000 7f8f13dd5640 1 -- 172.21.15.43:0/863540325 >> 172.21.15.43:0/863540325 conn(0x7f8f0c0d3a30 msgr2=0x7f8f0c15bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:08.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.493+0000 7f8f13dd5640 1 -- 172.21.15.43:0/863540325 shutdown_connections 2024-04-25T08:54:08.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.493+0000 7f8f13dd5640 1 -- 172.21.15.43:0/863540325 wait complete. 2024-04-25T08:54:08.494 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:54:08.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.606+0000 7f43bddf5640 1 Processor -- start 2024-04-25T08:54:08.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.606+0000 7f43bddf5640 1 -- start start 2024-04-25T08:54:08.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.607+0000 7f43bddf5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f43b815df50 0x7f43b81619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:08.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.607+0000 7f43bddf5640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f43b8161f30 0x7f43b8162380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:08.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.607+0000 7f43bddf5640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43b80d24a0 con 0x7f43b80c7cf0 2024-04-25T08:54:08.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.607+0000 7f43bddf5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43b815e330 con 0x7f43b815df50 2024-04-25T08:54:08.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.607+0000 7f43bddf5640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43b80d2740 con 0x7f43b8161f30 2024-04-25T08:54:08.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.607+0000 7f43b77fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f43b815df50 0x7f43b81619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:08.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.607+0000 7f43b77fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f43b815df50 0x7f43b81619f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41474/0 (socket says 172.21.15.43:41474) 2024-04-25T08:54:08.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.607+0000 7f43b77fe640 1 -- 172.21.15.43:0/4033188094 learned_addr learned my addr 172.21.15.43:0/4033188094 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:08.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.607+0000 7f43b6ffd640 1 --2- 172.21.15.43:0/4033188094 >> v2:172.21.15.193:3300/0 conn(0x7f43b8161f30 0x7f43b8162380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:08.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.607+0000 7f43b77fe640 1 -- 172.21.15.43:0/4033188094 >> v2:172.21.15.193:3300/0 conn(0x7f43b8161f30 msgr2=0x7f43b8162380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:08.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.607+0000 7f43b77fe640 1 --2- 172.21.15.43:0/4033188094 >> v2:172.21.15.193:3300/0 conn(0x7f43b8161f30 0x7f43b8162380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.607+0000 7f43b77fe640 1 -- 172.21.15.43:0/4033188094 >> v1:172.21.15.193:6789/0 conn(0x7f43b80c7cf0 legacy=0x7f43b80c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:08.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.607+0000 7f43b77fe640 1 -- 172.21.15.43:0/4033188094 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b80d2a20 con 0x7f43b815df50 2024-04-25T08:54:08.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.608+0000 7f43b77fe640 1 --2- 172.21.15.43:0/4033188094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43b815df50 0x7f43b81619f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f43b808c880 tx=0x7f43a802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea1a5b35287911e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:08.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.608+0000 7f43b67fc640 1 -- 172.21.15.43:0/4033188094 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a8037070 con 0x7f43b815df50 2024-04-25T08:54:08.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.608+0000 7f43b67fc640 1 -- 172.21.15.43:0/4033188094 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a8032440 con 0x7f43b815df50 2024-04-25T08:54:08.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.608+0000 7f43b67fc640 1 -- 172.21.15.43:0/4033188094 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a8032810 con 0x7f43b815df50 2024-04-25T08:54:08.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.608+0000 7f43bddf5640 1 -- 172.21.15.43:0/4033188094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43b815df50 msgr2=0x7f43b81619f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:08.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.608+0000 7f43bddf5640 1 --2- 172.21.15.43:0/4033188094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43b815df50 0x7f43b81619f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f43b808c880 tx=0x7f43a802a200 comp rx=0 tx=0).stop 2024-04-25T08:54:08.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.609+0000 7f43bddf5640 1 -- 172.21.15.43:0/4033188094 shutdown_connections 2024-04-25T08:54:08.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.609+0000 7f43bddf5640 1 --2- 172.21.15.43:0/4033188094 >> v2:172.21.15.193:3300/0 conn(0x7f43b8161f30 0x7f43b8162380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.609+0000 7f43bddf5640 1 --2- 172.21.15.43:0/4033188094 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43b815df50 0x7f43b81619f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.609+0000 7f43bddf5640 1 -- 172.21.15.43:0/4033188094 >> 172.21.15.43:0/4033188094 conn(0x7f43b80d3a30 msgr2=0x7f43b80d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:08.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.609+0000 7f43bddf5640 1 -- 172.21.15.43:0/4033188094 shutdown_connections 2024-04-25T08:54:08.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.609+0000 7f43bddf5640 1 -- 172.21.15.43:0/4033188094 wait complete. 2024-04-25T08:54:08.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.610+0000 7f43bddf5640 1 Processor -- start 2024-04-25T08:54:08.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.610+0000 7f43bddf5640 1 -- start start 2024-04-25T08:54:08.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.611+0000 7f43bddf5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f43b80c7cf0 0x7f43b809fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:08.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.611+0000 7f43bddf5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43b815df50 0x7f43b815d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.611+0000 7f43bddf5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43b8161f30 0x7f43b81e8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.611+0000 7f43bddf5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b80d2190 con 0x7f43b8161f30 2024-04-25T08:54:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.611+0000 7f43bddf5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b80c77d0 con 0x7f43b815df50 2024-04-25T08:54:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.611+0000 7f43bddf5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43b80cf670 con 0x7f43b80c7cf0 2024-04-25T08:54:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.611+0000 7f43b7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43b8161f30 0x7f43b81e8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.611+0000 7f43b7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43b8161f30 0x7f43b81e8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41480/0 (socket says 172.21.15.43:41480) 2024-04-25T08:54:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.611+0000 7f43b7fff640 1 -- 172.21.15.43:0/3354673318 learned_addr learned my addr 172.21.15.43:0/3354673318 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.611+0000 7f43b77fe640 1 --2- 172.21.15.43:0/3354673318 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f43b80c7cf0 0x7f43b809fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.611+0000 7f43b6ffd640 1 --2- 172.21.15.43:0/3354673318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43b815df50 0x7f43b815d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.611+0000 7f43b7fff640 1 -- 172.21.15.43:0/3354673318 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f43b80c7cf0 msgr2=0x7f43b809fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.611+0000 7f43b7fff640 1 --2- 172.21.15.43:0/3354673318 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f43b80c7cf0 0x7f43b809fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.611+0000 7f43b7fff640 1 -- 172.21.15.43:0/3354673318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43b815df50 msgr2=0x7f43b815d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.611+0000 7f43b7fff640 1 --2- 172.21.15.43:0/3354673318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43b815df50 0x7f43b815d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.611+0000 7f43b7fff640 1 -- 172.21.15.43:0/3354673318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b80a1770 con 0x7f43b8161f30 2024-04-25T08:54:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.611+0000 7f43b6ffd640 1 --2- 172.21.15.43:0/3354673318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43b815df50 0x7f43b815d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:08.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.611+0000 7f43b7fff640 1 --2- 172.21.15.43:0/3354673318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43b8161f30 0x7f43b81e8350 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f43b000c780 tx=0x7f43b000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:08.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.611+0000 7f439ffff640 1 -- 172.21.15.43:0/3354673318 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b0004b60 con 0x7f43b8161f30 2024-04-25T08:54:08.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.611+0000 7f439ffff640 1 -- 172.21.15.43:0/3354673318 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43b0004d00 con 0x7f43b8161f30 2024-04-25T08:54:08.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.611+0000 7f43bddf5640 1 -- 172.21.15.43:0/3354673318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43b80a06d0 con 0x7f43b8161f30 2024-04-25T08:54:08.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.611+0000 7f439ffff640 1 -- 172.21.15.43:0/3354673318 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43b000aa90 con 0x7f43b8161f30 2024-04-25T08:54:08.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.611+0000 7f43bddf5640 1 -- 172.21.15.43:0/3354673318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43b80a0c90 con 0x7f43b8161f30 2024-04-25T08:54:08.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.612+0000 7f43bddf5640 1 -- 172.21.15.43:0/3354673318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43b80680f0 con 0x7f43b8161f30 2024-04-25T08:54:08.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.613+0000 7f439ffff640 1 -- 172.21.15.43:0/3354673318 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f43b000ac30 con 0x7f43b8161f30 2024-04-25T08:54:08.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.613+0000 7f439ffff640 1 --2- 172.21.15.43:0/3354673318 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f43a406b4d0 0x7f43a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:08.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.613+0000 7f439ffff640 1 -- 172.21.15.43:0/3354673318 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f43b008a090 con 0x7f43b8161f30 2024-04-25T08:54:08.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.613+0000 7f43b77fe640 1 --2- 172.21.15.43:0/3354673318 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f43a406b4d0 0x7f43a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:08.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.614+0000 7f43b77fe640 1 --2- 172.21.15.43:0/3354673318 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f43a406b4d0 0x7f43a406d990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f43a8034000 tx=0x7f43a802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:08.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.617+0000 7f439ffff640 1 -- 172.21.15.43:0/3354673318 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f43b0020070 con 0x7f43b8161f30 2024-04-25T08:54:08.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.797+0000 7f43bddf5640 1 -- 172.21.15.43:0/3354673318 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43b8074ef0 con 0x7f43b8161f30 2024-04-25T08:54:08.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.798+0000 7f439ffff640 1 -- 172.21.15.43:0/3354673318 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12848 (secure 0 0 0) 0x7f43b00592f0 con 0x7f43b8161f30 2024-04-25T08:54:08.788 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:54:08.788 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":16,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:54:01.854785+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:54:08.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.800+0000 7f43bddf5640 1 -- 172.21.15.43:0/3354673318 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f43a406b4d0 msgr2=0x7f43a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:08.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.800+0000 7f43bddf5640 1 --2- 172.21.15.43:0/3354673318 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f43a406b4d0 0x7f43a406d990 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f43a8034000 tx=0x7f43a802e000 comp rx=0 tx=0).stop 2024-04-25T08:54:08.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.800+0000 7f43bddf5640 1 -- 172.21.15.43:0/3354673318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43b8161f30 msgr2=0x7f43b81e8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:08.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.800+0000 7f43bddf5640 1 --2- 172.21.15.43:0/3354673318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43b8161f30 0x7f43b81e8350 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f43b000c780 tx=0x7f43b000cc40 comp rx=0 tx=0).stop 2024-04-25T08:54:08.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.800+0000 7f43bddf5640 1 -- 172.21.15.43:0/3354673318 shutdown_connections 2024-04-25T08:54:08.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.800+0000 7f43bddf5640 1 --2- 172.21.15.43:0/3354673318 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43b8161f30 0x7f43b81e8350 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.800+0000 7f43bddf5640 1 --2- 172.21.15.43:0/3354673318 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f43a406b4d0 0x7f43a406d990 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.800+0000 7f43bddf5640 1 --2- 172.21.15.43:0/3354673318 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43b815df50 0x7f43b815d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.800+0000 7f43bddf5640 1 --2- 172.21.15.43:0/3354673318 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f43b80c7cf0 0x7f43b809fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.800+0000 7f43bddf5640 1 -- 172.21.15.43:0/3354673318 >> 172.21.15.43:0/3354673318 conn(0x7f43b80d3a30 msgr2=0x7f43b8153f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:08.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.800+0000 7f43bddf5640 1 -- 172.21.15.43:0/3354673318 shutdown_connections 2024-04-25T08:54:08.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.800+0000 7f43bddf5640 1 -- 172.21.15.43:0/3354673318 wait complete. 2024-04-25T08:54:08.801 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:54:08.802 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:54:08.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.915+0000 7f5025c07640 1 Processor -- start 2024-04-25T08:54:08.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.915+0000 7f5025c07640 1 -- start start 2024-04-25T08:54:08.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.916+0000 7f5025c07640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f502015dd30 0x7f502006c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:08.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.916+0000 7f5025c07640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f502006f700 0x7f50200aeab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:08.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.916+0000 7f5025c07640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f502015e110 con 0x7f502006cb90 2024-04-25T08:54:08.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.916+0000 7f5025c07640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f502006d0a0 con 0x7f502015dd30 2024-04-25T08:54:08.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.916+0000 7f5025c07640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f502006d380 con 0x7f502006f700 2024-04-25T08:54:08.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.916+0000 7f501f7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f502015dd30 0x7f502006c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:08.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.916+0000 7f501effd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f502006cb90 0x7f502006eff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:41900/0 (socket says 172.21.15.43:41900) 2024-04-25T08:54:08.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.916+0000 7f501effd640 1 -- 172.21.15.43:0/3562143229 learned_addr learned my addr 172.21.15.43:0/3562143229 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:08.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.917+0000 7f501ffff640 1 --2- 172.21.15.43:0/3562143229 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f502006f700 0x7f50200aeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:08.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.917+0000 7f501f7fe640 1 -- 172.21.15.43:0/3562143229 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f502006f700 msgr2=0x7f50200aeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:08.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.917+0000 7f501f7fe640 1 --2- 172.21.15.43:0/3562143229 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f502006f700 0x7f50200aeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.917+0000 7f501f7fe640 1 -- 172.21.15.43:0/3562143229 >> v1:172.21.15.43:6789/0 conn(0x7f502006cb90 legacy=0x7f502006eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:08.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.917+0000 7f501f7fe640 1 -- 172.21.15.43:0/3562143229 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f502006d660 con 0x7f502015dd30 2024-04-25T08:54:08.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.917+0000 7f501f7fe640 1 --2- 172.21.15.43:0/3562143229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f502015dd30 0x7f502006c650 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f50100097a0 tx=0x7f501002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=643b1e7c39b1f2e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:08.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.917+0000 7f501e7fc640 1 -- 172.21.15.43:0/3562143229 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5010037070 con 0x7f502015dd30 2024-04-25T08:54:08.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.918+0000 7f501e7fc640 1 -- 172.21.15.43:0/3562143229 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f501002ae50 con 0x7f502015dd30 2024-04-25T08:54:08.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.918+0000 7f501e7fc640 1 -- 172.21.15.43:0/3562143229 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5010032660 con 0x7f502015dd30 2024-04-25T08:54:08.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.918+0000 7f5025c07640 1 -- 172.21.15.43:0/3562143229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f502015dd30 msgr2=0x7f502006c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:08.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.918+0000 7f5025c07640 1 --2- 172.21.15.43:0/3562143229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f502015dd30 0x7f502006c650 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f50100097a0 tx=0x7f501002a100 comp rx=0 tx=0).stop 2024-04-25T08:54:08.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.918+0000 7f5025c07640 1 -- 172.21.15.43:0/3562143229 shutdown_connections 2024-04-25T08:54:08.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.918+0000 7f5025c07640 1 --2- 172.21.15.43:0/3562143229 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f502006f700 0x7f50200aeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.918+0000 7f5025c07640 1 --2- 172.21.15.43:0/3562143229 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f502015dd30 0x7f502006c650 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.918+0000 7f5025c07640 1 -- 172.21.15.43:0/3562143229 >> 172.21.15.43:0/3562143229 conn(0x7f50200b6cf0 msgr2=0x7f50200b7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:08.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.918+0000 7f5025c07640 1 -- 172.21.15.43:0/3562143229 shutdown_connections 2024-04-25T08:54:08.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.918+0000 7f5025c07640 1 -- 172.21.15.43:0/3562143229 wait complete. 2024-04-25T08:54:08.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.919+0000 7f5025c07640 1 Processor -- start 2024-04-25T08:54:08.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.919+0000 7f5025c07640 1 -- start start 2024-04-25T08:54:08.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.920+0000 7f5025c07640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f502006cb90 0x7f502015ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:08.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.920+0000 7f5025c07640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f502006f700 0x7f5020160c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:08.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.920+0000 7f5025c07640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f502015dd30 0x7f5020155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:08.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.920+0000 7f5025c07640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50200b65e0 con 0x7f502015dd30 2024-04-25T08:54:08.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.920+0000 7f5025c07640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50200b6460 con 0x7f502006f700 2024-04-25T08:54:08.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.920+0000 7f5025c07640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50200aeff0 con 0x7f502006cb90 2024-04-25T08:54:08.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.920+0000 7f501ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f502015dd30 0x7f5020155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:08.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.920+0000 7f501ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f502015dd30 0x7f5020155050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41506/0 (socket says 172.21.15.43:41506) 2024-04-25T08:54:08.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.920+0000 7f501ffff640 1 -- 172.21.15.43:0/4121710895 learned_addr learned my addr 172.21.15.43:0/4121710895 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:08.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.920+0000 7f501f7fe640 1 --2- 172.21.15.43:0/4121710895 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f502006cb90 0x7f502015ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:08.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.920+0000 7f501ffff640 1 -- 172.21.15.43:0/4121710895 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f502006cb90 msgr2=0x7f502015ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:08.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.920+0000 7f501effd640 1 --2- 172.21.15.43:0/4121710895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f502006f700 0x7f5020160c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:08.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.920+0000 7f501ffff640 1 --2- 172.21.15.43:0/4121710895 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f502006cb90 0x7f502015ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.920+0000 7f501ffff640 1 -- 172.21.15.43:0/4121710895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f502006f700 msgr2=0x7f5020160c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:08.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.920+0000 7f501ffff640 1 --2- 172.21.15.43:0/4121710895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f502006f700 0x7f5020160c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:08.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.921+0000 7f501ffff640 1 -- 172.21.15.43:0/4121710895 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50201f8270 con 0x7f502015dd30 2024-04-25T08:54:08.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.921+0000 7f501ffff640 1 --2- 172.21.15.43:0/4121710895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f502015dd30 0x7f5020155050 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5018007e30 tx=0x7f501800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:08.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.921+0000 7f5007fff640 1 -- 172.21.15.43:0/4121710895 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5018014620 con 0x7f502015dd30 2024-04-25T08:54:08.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.921+0000 7f5007fff640 1 -- 172.21.15.43:0/4121710895 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5018019070 con 0x7f502015dd30 2024-04-25T08:54:08.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.921+0000 7f5007fff640 1 -- 172.21.15.43:0/4121710895 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5018014a00 con 0x7f502015dd30 2024-04-25T08:54:08.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.921+0000 7f5025c07640 1 -- 172.21.15.43:0/4121710895 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f502015b080 con 0x7f502015dd30 2024-04-25T08:54:08.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.921+0000 7f5025c07640 1 -- 172.21.15.43:0/4121710895 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f502015b640 con 0x7f502015dd30 2024-04-25T08:54:08.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.922+0000 7f5025c07640 1 -- 172.21.15.43:0/4121710895 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5020074f60 con 0x7f502015dd30 2024-04-25T08:54:08.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.922+0000 7f5007fff640 1 -- 172.21.15.43:0/4121710895 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5018014ba0 con 0x7f502015dd30 2024-04-25T08:54:08.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.923+0000 7f5007fff640 1 --2- 172.21.15.43:0/4121710895 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f500c06b4d0 0x7f500c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:08.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.923+0000 7f5007fff640 1 -- 172.21.15.43:0/4121710895 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f501808a710 con 0x7f502015dd30 2024-04-25T08:54:08.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.923+0000 7f501f7fe640 1 --2- 172.21.15.43:0/4121710895 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f500c06b4d0 0x7f500c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:08.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.923+0000 7f501f7fe640 1 --2- 172.21.15.43:0/4121710895 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f500c06b4d0 0x7f500c06d990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5010034000 tx=0x7f501002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:08.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:08.927+0000 7f5007fff640 1 -- 172.21.15.43:0/4121710895 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5018059970 con 0x7f502015dd30 2024-04-25T08:54:09.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.106+0000 7f5025c07640 1 -- 172.21.15.43:0/4121710895 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f502006ebc0 con 0x7f502015dd30 2024-04-25T08:54:09.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.106+0000 7f5007fff640 1 -- 172.21.15.43:0/4121710895 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5018059270 con 0x7f502015dd30 2024-04-25T08:54:09.097 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T08:54:09.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.108+0000 7f5025c07640 1 -- 172.21.15.43:0/4121710895 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f500c06b4d0 msgr2=0x7f500c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:09.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.108+0000 7f5025c07640 1 --2- 172.21.15.43:0/4121710895 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f500c06b4d0 0x7f500c06d990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5010034000 tx=0x7f501002e000 comp rx=0 tx=0).stop 2024-04-25T08:54:09.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.108+0000 7f5025c07640 1 -- 172.21.15.43:0/4121710895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f502015dd30 msgr2=0x7f5020155050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:09.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.108+0000 7f5025c07640 1 --2- 172.21.15.43:0/4121710895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f502015dd30 0x7f5020155050 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5018007e30 tx=0x7f501800e4b0 comp rx=0 tx=0).stop 2024-04-25T08:54:09.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.108+0000 7f5025c07640 1 -- 172.21.15.43:0/4121710895 shutdown_connections 2024-04-25T08:54:09.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.108+0000 7f5025c07640 1 --2- 172.21.15.43:0/4121710895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f502015dd30 0x7f5020155050 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:09.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.108+0000 7f5025c07640 1 --2- 172.21.15.43:0/4121710895 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f500c06b4d0 0x7f500c06d990 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:09.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.108+0000 7f5025c07640 1 --2- 172.21.15.43:0/4121710895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f502006f700 0x7f5020160c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:09.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.108+0000 7f5025c07640 1 --2- 172.21.15.43:0/4121710895 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f502006cb90 0x7f502015ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:09.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.108+0000 7f5025c07640 1 -- 172.21.15.43:0/4121710895 >> 172.21.15.43:0/4121710895 conn(0x7f50200b6cf0 msgr2=0x7f50200b5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:09.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.108+0000 7f5025c07640 1 -- 172.21.15.43:0/4121710895 shutdown_connections 2024-04-25T08:54:09.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.108+0000 7f5025c07640 1 -- 172.21.15.43:0/4121710895 wait complete. 2024-04-25T08:54:09.109 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T08:54:09.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.221+0000 7fd9153c1640 1 Processor -- start 2024-04-25T08:54:09.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.221+0000 7fd9153c1640 1 -- start start 2024-04-25T08:54:09.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.222+0000 7fd9153c1640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd91015dea0 0x7fd9100d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:09.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.222+0000 7fd9153c1640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd9100d1930 0x7fd9100d1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:09.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.222+0000 7fd9153c1640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9100d0d30 con 0x7fd91015e3d0 2024-04-25T08:54:09.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.222+0000 7fd9153c1640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9100d22d0 con 0x7fd91015dea0 2024-04-25T08:54:09.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.222+0000 7fd9153c1640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd91015f140 con 0x7fd9100d1930 2024-04-25T08:54:09.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.222+0000 7fd90effd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd91015dea0 0x7fd9100d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:09.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.222+0000 7fd90effd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd91015dea0 0x7fd9100d07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41516/0 (socket says 172.21.15.43:41516) 2024-04-25T08:54:09.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.222+0000 7fd90effd640 1 -- 172.21.15.43:0/3151506431 learned_addr learned my addr 172.21.15.43:0/3151506431 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:09.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.223+0000 7fd90e7fc640 1 --2- 172.21.15.43:0/3151506431 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd9100d1930 0x7fd9100d1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:09.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.223+0000 7fd90effd640 1 -- 172.21.15.43:0/3151506431 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd9100d1930 msgr2=0x7fd9100d1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:09.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.223+0000 7fd90effd640 1 --2- 172.21.15.43:0/3151506431 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd9100d1930 0x7fd9100d1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:09.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.223+0000 7fd90effd640 1 -- 172.21.15.43:0/3151506431 >> v1:172.21.15.193:6789/0 conn(0x7fd91015e3d0 legacy=0x7fd91015e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:09.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.223+0000 7fd90effd640 1 -- 172.21.15.43:0/3151506431 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9100cfd10 con 0x7fd91015dea0 2024-04-25T08:54:09.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.223+0000 7fd90effd640 1 --2- 172.21.15.43:0/3151506431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd91015dea0 0x7fd9100d07f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd908009870 tx=0x7fd90802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c60200fc43287eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:09.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.223+0000 7fd90dffb640 1 -- 172.21.15.43:0/3151506431 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd908037070 con 0x7fd91015dea0 2024-04-25T08:54:09.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.224+0000 7fd90dffb640 1 -- 172.21.15.43:0/3151506431 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd908032440 con 0x7fd91015dea0 2024-04-25T08:54:09.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.224+0000 7fd90dffb640 1 -- 172.21.15.43:0/3151506431 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd908032810 con 0x7fd91015dea0 2024-04-25T08:54:09.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.224+0000 7fd9153c1640 1 -- 172.21.15.43:0/3151506431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd91015dea0 msgr2=0x7fd9100d07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:09.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.224+0000 7fd9153c1640 1 --2- 172.21.15.43:0/3151506431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd91015dea0 0x7fd9100d07f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd908009870 tx=0x7fd90802a210 comp rx=0 tx=0).stop 2024-04-25T08:54:09.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.224+0000 7fd9153c1640 1 -- 172.21.15.43:0/3151506431 shutdown_connections 2024-04-25T08:54:09.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.224+0000 7fd9153c1640 1 --2- 172.21.15.43:0/3151506431 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd9100d1930 0x7fd9100d1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:09.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.224+0000 7fd9153c1640 1 --2- 172.21.15.43:0/3151506431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd91015dea0 0x7fd9100d07f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:09.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.224+0000 7fd9153c1640 1 -- 172.21.15.43:0/3151506431 >> 172.21.15.43:0/3151506431 conn(0x7fd91009aa70 msgr2=0x7fd91009ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:09.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.224+0000 7fd9153c1640 1 -- 172.21.15.43:0/3151506431 shutdown_connections 2024-04-25T08:54:09.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.224+0000 7fd9153c1640 1 -- 172.21.15.43:0/3151506431 wait complete. 2024-04-25T08:54:09.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.225+0000 7fd9153c1640 1 Processor -- start 2024-04-25T08:54:09.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.226+0000 7fd9153c1640 1 -- start start 2024-04-25T08:54:09.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.226+0000 7fd9153c1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9100d1930 0x7fd910154970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:09.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.226+0000 7fd9153c1640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd91015dea0 0x7fd9100c8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:09.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.226+0000 7fd9153c1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd91015e3d0 0x7fd9100cc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:09.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.226+0000 7fd9153c1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9100cf930 con 0x7fd91015e3d0 2024-04-25T08:54:09.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.226+0000 7fd9153c1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9100d02d0 con 0x7fd9100d1930 2024-04-25T08:54:09.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.226+0000 7fd9153c1640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9100cfab0 con 0x7fd91015dea0 2024-04-25T08:54:09.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.226+0000 7fd90f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd91015e3d0 0x7fd9100cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:09.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.226+0000 7fd90f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd91015e3d0 0x7fd9100cc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41528/0 (socket says 172.21.15.43:41528) 2024-04-25T08:54:09.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.226+0000 7fd90f7fe640 1 -- 172.21.15.43:0/1897979785 learned_addr learned my addr 172.21.15.43:0/1897979785 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:09.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.227+0000 7fd90e7fc640 1 --2- 172.21.15.43:0/1897979785 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd91015dea0 0x7fd9100c8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:09.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.227+0000 7fd90effd640 1 --2- 172.21.15.43:0/1897979785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9100d1930 0x7fd910154970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:09.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.227+0000 7fd90f7fe640 1 -- 172.21.15.43:0/1897979785 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd91015dea0 msgr2=0x7fd9100c8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:09.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.227+0000 7fd90f7fe640 1 --2- 172.21.15.43:0/1897979785 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd91015dea0 0x7fd9100c8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:09.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.227+0000 7fd90f7fe640 1 -- 172.21.15.43:0/1897979785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9100d1930 msgr2=0x7fd910154970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:09.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.227+0000 7fd90f7fe640 1 --2- 172.21.15.43:0/1897979785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9100d1930 0x7fd910154970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:09.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.227+0000 7fd90f7fe640 1 -- 172.21.15.43:0/1897979785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd910154eb0 con 0x7fd91015e3d0 2024-04-25T08:54:09.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.227+0000 7fd90f7fe640 1 --2- 172.21.15.43:0/1897979785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd91015e3d0 0x7fd9100cc270 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd90000b970 tx=0x7fd90000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:09.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.227+0000 7fd9077fe640 1 -- 172.21.15.43:0/1897979785 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd900019070 con 0x7fd91015e3d0 2024-04-25T08:54:09.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.227+0000 7fd9077fe640 1 -- 172.21.15.43:0/1897979785 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90000cb40 con 0x7fd91015e3d0 2024-04-25T08:54:09.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.227+0000 7fd9153c1640 1 -- 172.21.15.43:0/1897979785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd910155150 con 0x7fd91015e3d0 2024-04-25T08:54:09.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.227+0000 7fd9077fe640 1 -- 172.21.15.43:0/1897979785 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd900014440 con 0x7fd91015e3d0 2024-04-25T08:54:09.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.227+0000 7fd9153c1640 1 -- 172.21.15.43:0/1897979785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9100cd5a0 con 0x7fd91015e3d0 2024-04-25T08:54:09.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.228+0000 7fd9153c1640 1 -- 172.21.15.43:0/1897979785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd910074ef0 con 0x7fd91015e3d0 2024-04-25T08:54:09.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.228+0000 7fd9077fe640 1 -- 172.21.15.43:0/1897979785 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd90001c070 con 0x7fd91015e3d0 2024-04-25T08:54:09.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.229+0000 7fd9077fe640 1 --2- 172.21.15.43:0/1897979785 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd8f006b4d0 0x7fd8f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:09.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.229+0000 7fd9077fe640 1 -- 172.21.15.43:0/1897979785 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd900089a90 con 0x7fd91015e3d0 2024-04-25T08:54:09.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.229+0000 7fd90effd640 1 --2- 172.21.15.43:0/1897979785 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd8f006b4d0 0x7fd8f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:09.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.230+0000 7fd90effd640 1 --2- 172.21.15.43:0/1897979785 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd8f006b4d0 0x7fd8f006d990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd908034000 tx=0x7fd90802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:09.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.233+0000 7fd9077fe640 1 -- 172.21.15.43:0/1897979785 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd900058c60 con 0x7fd91015e3d0 2024-04-25T08:54:09.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.414+0000 7fd9153c1640 1 -- 172.21.15.43:0/1897979785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd9100d26e0 con 0x7fd91015e3d0 2024-04-25T08:54:09.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.414+0000 7fd9077fe640 1 -- 172.21.15.43:0/1897979785 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7fd900058560 con 0x7fd91015e3d0 2024-04-25T08:54:09.404 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:54:09.404 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2024-04-25T08:54:09.404 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:54:09.404 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 1, 2024-04-25T08:54:09.405 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": ".mgr" 2024-04-25T08:54:09.405 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:54:09.405 INFO:teuthology.orchestra.run.smithi043.stdout:] 2024-04-25T08:54:09.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.416+0000 7fd9153c1640 1 -- 172.21.15.43:0/1897979785 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd8f006b4d0 msgr2=0x7fd8f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:09.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.416+0000 7fd9153c1640 1 --2- 172.21.15.43:0/1897979785 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd8f006b4d0 0x7fd8f006d990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd908034000 tx=0x7fd90802e000 comp rx=0 tx=0).stop 2024-04-25T08:54:09.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.416+0000 7fd9153c1640 1 -- 172.21.15.43:0/1897979785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd91015e3d0 msgr2=0x7fd9100cc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:09.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.416+0000 7fd9153c1640 1 --2- 172.21.15.43:0/1897979785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd91015e3d0 0x7fd9100cc270 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd90000b970 tx=0x7fd90000be30 comp rx=0 tx=0).stop 2024-04-25T08:54:09.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.416+0000 7fd9153c1640 1 -- 172.21.15.43:0/1897979785 shutdown_connections 2024-04-25T08:54:09.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.416+0000 7fd9153c1640 1 --2- 172.21.15.43:0/1897979785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd91015e3d0 0x7fd9100cc270 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:09.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.416+0000 7fd9153c1640 1 --2- 172.21.15.43:0/1897979785 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd91015dea0 0x7fd9100c8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:09.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.416+0000 7fd9153c1640 1 --2- 172.21.15.43:0/1897979785 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd8f006b4d0 0x7fd8f006d990 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:09.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.416+0000 7fd9153c1640 1 --2- 172.21.15.43:0/1897979785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9100d1930 0x7fd910154970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:09.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.416+0000 7fd9153c1640 1 -- 172.21.15.43:0/1897979785 >> 172.21.15.43:0/1897979785 conn(0x7fd91009aa70 msgr2=0x7fd91006c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:09.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.416+0000 7fd9153c1640 1 -- 172.21.15.43:0/1897979785 shutdown_connections 2024-04-25T08:54:09.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.416+0000 7fd9153c1640 1 -- 172.21.15.43:0/1897979785 wait complete. 2024-04-25T08:54:09.417 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T08:54:09.417 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T08:54:09.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.532+0000 7f4636d3b640 1 Processor -- start 2024-04-25T08:54:09.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.532+0000 7f4636d3b640 1 -- start start 2024-04-25T08:54:09.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.533+0000 7f4636d3b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f46300d03f0 0x7f46300d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:09.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.533+0000 7f4636d3b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f46300b3c40 0x7f46300b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:09.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.533+0000 7f4636d3b640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46300d2370 con 0x7f463015def0 2024-04-25T08:54:09.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.533+0000 7f4636d3b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46300d0d90 con 0x7f46300d03f0 2024-04-25T08:54:09.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.533+0000 7f4636d3b640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46300b37b0 con 0x7f46300b3c40 2024-04-25T08:54:09.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.533+0000 7f462ffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f46300d03f0 0x7f46300d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:09.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.533+0000 7f462ffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f46300d03f0 0x7f46300d0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41544/0 (socket says 172.21.15.43:41544) 2024-04-25T08:54:09.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.533+0000 7f462ffff640 1 -- 172.21.15.43:0/675730651 learned_addr learned my addr 172.21.15.43:0/675730651 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:09.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.533+0000 7f46352d8640 1 --2- 172.21.15.43:0/675730651 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f46300b3c40 0x7f46300b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:09.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.533+0000 7f462ffff640 1 -- 172.21.15.43:0/675730651 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f46300b3c40 msgr2=0x7f46300b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:09.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.533+0000 7f462ffff640 1 --2- 172.21.15.43:0/675730651 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f46300b3c40 0x7f46300b40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:09.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.533+0000 7f462ffff640 1 -- 172.21.15.43:0/675730651 >> v1:172.21.15.193:6789/0 conn(0x7f463015def0 legacy=0x7f46300d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:09.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.533+0000 7f462ffff640 1 -- 172.21.15.43:0/675730651 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46300d2650 con 0x7f46300d03f0 2024-04-25T08:54:09.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.534+0000 7f462ffff640 1 --2- 172.21.15.43:0/675730651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46300d03f0 0x7f46300d0850 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f461c009870 tx=0x7f461c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9991e1e48ac5a29e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:09.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.534+0000 7f462f7fe640 1 -- 172.21.15.43:0/675730651 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461c037070 con 0x7f46300d03f0 2024-04-25T08:54:09.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.534+0000 7f462f7fe640 1 -- 172.21.15.43:0/675730651 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f461c032440 con 0x7f46300d03f0 2024-04-25T08:54:09.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.534+0000 7f462f7fe640 1 -- 172.21.15.43:0/675730651 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461c032810 con 0x7f46300d03f0 2024-04-25T08:54:09.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.534+0000 7f4636d3b640 1 -- 172.21.15.43:0/675730651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46300d03f0 msgr2=0x7f46300d0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:09.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.534+0000 7f4636d3b640 1 --2- 172.21.15.43:0/675730651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46300d03f0 0x7f46300d0850 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f461c009870 tx=0x7f461c02a180 comp rx=0 tx=0).stop 2024-04-25T08:54:09.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.534+0000 7f4636d3b640 1 -- 172.21.15.43:0/675730651 shutdown_connections 2024-04-25T08:54:09.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.534+0000 7f4636d3b640 1 --2- 172.21.15.43:0/675730651 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f46300b3c40 0x7f46300b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:09.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.534+0000 7f4636d3b640 1 --2- 172.21.15.43:0/675730651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46300d03f0 0x7f46300d0850 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:09.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.534+0000 7f4636d3b640 1 -- 172.21.15.43:0/675730651 >> 172.21.15.43:0/675730651 conn(0x7f463009cd60 msgr2=0x7f46300d3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:09.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.535+0000 7f4636d3b640 1 -- 172.21.15.43:0/675730651 shutdown_connections 2024-04-25T08:54:09.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.535+0000 7f4636d3b640 1 -- 172.21.15.43:0/675730651 wait complete. 2024-04-25T08:54:09.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.536+0000 7f4636d3b640 1 Processor -- start 2024-04-25T08:54:09.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.536+0000 7f4636d3b640 1 -- start start 2024-04-25T08:54:09.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.536+0000 7f4636d3b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f46300b3c40 0x7f46301524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:09.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.536+0000 7f4636d3b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f46300d03f0 0x7f46300caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:09.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.536+0000 7f4636d3b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f463015def0 0x7f463015fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:09.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.537+0000 7f4636d3b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46300d2930 con 0x7f463015def0 2024-04-25T08:54:09.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.537+0000 7f4636d3b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46300d4840 con 0x7f46300b3c40 2024-04-25T08:54:09.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.537+0000 7f4636d3b640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46300d1fd0 con 0x7f46300d03f0 2024-04-25T08:54:09.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.537+0000 7f46352d8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f463015def0 0x7f463015fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:09.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.537+0000 7f46352d8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f463015def0 0x7f463015fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41556/0 (socket says 172.21.15.43:41556) 2024-04-25T08:54:09.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.537+0000 7f46352d8640 1 -- 172.21.15.43:0/999582029 learned_addr learned my addr 172.21.15.43:0/999582029 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:09.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.537+0000 7f462ffff640 1 --2- 172.21.15.43:0/999582029 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f46300d03f0 0x7f46300caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:09.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.537+0000 7f4634ad7640 1 --2- 172.21.15.43:0/999582029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f46300b3c40 0x7f46301524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:09.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.537+0000 7f46352d8640 1 -- 172.21.15.43:0/999582029 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f46300d03f0 msgr2=0x7f46300caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:09.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.537+0000 7f46352d8640 1 --2- 172.21.15.43:0/999582029 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f46300d03f0 0x7f46300caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:09.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.537+0000 7f46352d8640 1 -- 172.21.15.43:0/999582029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f46300b3c40 msgr2=0x7f46301524f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:09.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.537+0000 7f46352d8640 1 --2- 172.21.15.43:0/999582029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f46300b3c40 0x7f46301524f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:09.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.537+0000 7f46352d8640 1 -- 172.21.15.43:0/999582029 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f463015f0e0 con 0x7f463015def0 2024-04-25T08:54:09.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.537+0000 7f46352d8640 1 --2- 172.21.15.43:0/999582029 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f463015def0 0x7f463015fe60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4628007e40 tx=0x7f462800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:09.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.537+0000 7f462d7fa640 1 -- 172.21.15.43:0/999582029 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4628014620 con 0x7f463015def0 2024-04-25T08:54:09.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.537+0000 7f462d7fa640 1 -- 172.21.15.43:0/999582029 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4628019070 con 0x7f463015def0 2024-04-25T08:54:09.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.537+0000 7f462d7fa640 1 -- 172.21.15.43:0/999582029 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4628014a20 con 0x7f463015def0 2024-04-25T08:54:09.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.537+0000 7f4636d3b640 1 -- 172.21.15.43:0/999582029 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46300b84a0 con 0x7f463015def0 2024-04-25T08:54:09.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.537+0000 7f4636d3b640 1 -- 172.21.15.43:0/999582029 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f463015f660 con 0x7f463015def0 2024-04-25T08:54:09.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.538+0000 7f4636d3b640 1 -- 172.21.15.43:0/999582029 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4630068a40 con 0x7f463015def0 2024-04-25T08:54:09.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.539+0000 7f462d7fa640 1 -- 172.21.15.43:0/999582029 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4628014bc0 con 0x7f463015def0 2024-04-25T08:54:09.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.539+0000 7f462d7fa640 1 --2- 172.21.15.43:0/999582029 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f461406b4d0 0x7f461406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:09.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.539+0000 7f462d7fa640 1 -- 172.21.15.43:0/999582029 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f462808a890 con 0x7f463015def0 2024-04-25T08:54:09.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.539+0000 7f4634ad7640 1 --2- 172.21.15.43:0/999582029 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f461406b4d0 0x7f461406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:09.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.540+0000 7f4634ad7640 1 --2- 172.21.15.43:0/999582029 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f461406b4d0 0x7f461406d990 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4624006fd0 tx=0x7f4624008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:09.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.543+0000 7f462d7fa640 1 -- 172.21.15.43:0/999582029 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f46280599c0 con 0x7f463015def0 2024-04-25T08:54:09.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:09.726+0000 7f4636d3b640 1 -- 172.21.15.43:0/999582029 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f46300cf340 con 0x7f463015def0 2024-04-25T08:54:10.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.565+0000 7f462d7fa640 1 -- 172.21.15.43:0/999582029 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f46280592c0 con 0x7f463015def0 2024-04-25T08:54:10.555 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' created 2024-04-25T08:54:10.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.569+0000 7f4636d3b640 1 -- 172.21.15.43:0/999582029 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f461406b4d0 msgr2=0x7f461406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:10.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.569+0000 7f4636d3b640 1 --2- 172.21.15.43:0/999582029 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f461406b4d0 0x7f461406d990 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4624006fd0 tx=0x7f4624008040 comp rx=0 tx=0).stop 2024-04-25T08:54:10.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.569+0000 7f4636d3b640 1 -- 172.21.15.43:0/999582029 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f463015def0 msgr2=0x7f463015fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:10.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.569+0000 7f4636d3b640 1 --2- 172.21.15.43:0/999582029 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f463015def0 0x7f463015fe60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4628007e40 tx=0x7f462800e4b0 comp rx=0 tx=0).stop 2024-04-25T08:54:10.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.570+0000 7f4636d3b640 1 -- 172.21.15.43:0/999582029 shutdown_connections 2024-04-25T08:54:10.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.570+0000 7f4636d3b640 1 --2- 172.21.15.43:0/999582029 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f463015def0 0x7f463015fe60 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:10.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.570+0000 7f4636d3b640 1 --2- 172.21.15.43:0/999582029 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f46300d03f0 0x7f46300caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:10.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.570+0000 7f4636d3b640 1 --2- 172.21.15.43:0/999582029 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f461406b4d0 0x7f461406d990 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:10.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.570+0000 7f4636d3b640 1 --2- 172.21.15.43:0/999582029 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f46300b3c40 0x7f46301524f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:10.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.570+0000 7f4636d3b640 1 -- 172.21.15.43:0/999582029 >> 172.21.15.43:0/999582029 conn(0x7f463009cd60 msgr2=0x7f46300d1840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:10.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.572+0000 7f4636d3b640 1 -- 172.21.15.43:0/999582029 shutdown_connections 2024-04-25T08:54:10.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.573+0000 7f4636d3b640 1 -- 172.21.15.43:0/999582029 wait complete. 2024-04-25T08:54:10.578 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T08:54:10.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.701+0000 7f35c2d68640 1 Processor -- start 2024-04-25T08:54:10.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.701+0000 7f35c2d68640 1 -- start start 2024-04-25T08:54:10.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.702+0000 7f35c2d68640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f35bc0d0430 0x7f35bc1621c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:10.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.702+0000 7f35c2d68640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35bc15ded0 0x7f35bc0d2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:10.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.702+0000 7f35c2d68640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35bc162700 con 0x7f35bc0b6d60 2024-04-25T08:54:10.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.702+0000 7f35c2d68640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35bc0d1ed0 con 0x7f35bc0d0430 2024-04-25T08:54:10.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.702+0000 7f35c2d68640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc0ce820 con 0x7f35bc15ded0 2024-04-25T08:54:10.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.702+0000 7f35bbfff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f35bc0b6d60 0x7f35bc0d0810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60766/0 (socket says 172.21.15.43:60766) 2024-04-25T08:54:10.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.702+0000 7f35bbfff640 1 -- 172.21.15.43:0/3179203213 learned_addr learned my addr 172.21.15.43:0/3179203213 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:10.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.703+0000 7f35c1305640 1 --2- 172.21.15.43:0/3179203213 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35bc15ded0 0x7f35bc0d2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:10.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.703+0000 7f35c0b04640 1 --2- 172.21.15.43:0/3179203213 >> v2:172.21.15.193:3300/0 conn(0x7f35bc0d0430 0x7f35bc1621c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:10.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.703+0000 7f35bb7fe640 1 -- 172.21.15.43:0/3179203213 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4036556861 0 0) 0x7f35bc162700 con 0x7f35bc0b6d60 2024-04-25T08:54:10.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.703+0000 7f35bb7fe640 1 -- 172.21.15.43:0/3179203213 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35ac003520 con 0x7f35bc0b6d60 2024-04-25T08:54:10.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.703+0000 7f35bb7fe640 1 -- 172.21.15.43:0/3179203213 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1665327715 0 0) 0x7f35ac003520 con 0x7f35bc0b6d60 2024-04-25T08:54:10.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.703+0000 7f35bb7fe640 1 -- 172.21.15.43:0/3179203213 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35bc15ded0 msgr2=0x7f35bc0d2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:10.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.703+0000 7f35bb7fe640 1 --2- 172.21.15.43:0/3179203213 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35bc15ded0 0x7f35bc0d2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:10.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.703+0000 7f35bb7fe640 1 -- 172.21.15.43:0/3179203213 >> v2:172.21.15.193:3300/0 conn(0x7f35bc0d0430 msgr2=0x7f35bc1621c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:10.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.703+0000 7f35bb7fe640 1 --2- 172.21.15.43:0/3179203213 >> v2:172.21.15.193:3300/0 conn(0x7f35bc0d0430 0x7f35bc1621c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:10.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.703+0000 7f35bb7fe640 1 -- 172.21.15.43:0/3179203213 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35bc0ceb00 con 0x7f35bc0b6d60 2024-04-25T08:54:10.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.703+0000 7f35bb7fe640 1 -- 172.21.15.43:0/3179203213 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f35a8001670 con 0x7f35bc0b6d60 2024-04-25T08:54:10.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.703+0000 7f35bb7fe640 1 -- 172.21.15.43:0/3179203213 >> v1:172.21.15.43:6789/0 conn(0x7f35bc0b6d60 legacy=0x7f35bc0d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:10.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.703+0000 7f35bb7fe640 1 --2- 172.21.15.43:0/3179203213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35ac0039b0 0x7f35ac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:10.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.703+0000 7f35bb7fe640 1 --2- 172.21.15.43:0/3179203213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35ac004800 0x7f35ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:10.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.703+0000 7f35bb7fe640 1 --2- 172.21.15.43:0/3179203213 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35bc15ded0 0x7f35ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:10.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.703+0000 7f35bb7fe640 1 -- 172.21.15.43:0/3179203213 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc0d1ed0 con 0x7f35ac004800 2024-04-25T08:54:10.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.703+0000 7f35bb7fe640 1 -- 172.21.15.43:0/3179203213 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc0ce820 con 0x7f35ac0039b0 2024-04-25T08:54:10.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.703+0000 7f35bb7fe640 1 -- 172.21.15.43:0/3179203213 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac00b9c0 con 0x7f35bc15ded0 2024-04-25T08:54:10.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.703+0000 7f35bbfff640 1 --2- 172.21.15.43:0/3179203213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35ac004800 0x7f35ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:10.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.703+0000 7f35c0b04640 1 --2- 172.21.15.43:0/3179203213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35ac0039b0 0x7f35ac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:10.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.703+0000 7f35c1305640 1 --2- 172.21.15.43:0/3179203213 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35bc15ded0 0x7f35ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:10.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.703+0000 7f35bbfff640 1 -- 172.21.15.43:0/3179203213 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35bc15ded0 msgr2=0x7f35ac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:10.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.703+0000 7f35bbfff640 1 --2- 172.21.15.43:0/3179203213 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35bc15ded0 0x7f35ac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:10.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.703+0000 7f35bbfff640 1 -- 172.21.15.43:0/3179203213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35ac0039b0 msgr2=0x7f35ac003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:10.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.703+0000 7f35bbfff640 1 --2- 172.21.15.43:0/3179203213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35ac0039b0 0x7f35ac003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:10.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.703+0000 7f35bbfff640 1 -- 172.21.15.43:0/3179203213 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35ac003520 con 0x7f35ac004800 2024-04-25T08:54:10.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.704+0000 7f35bbfff640 1 --2- 172.21.15.43:0/3179203213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35ac004800 0x7f35ac004c80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f35a8001050 tx=0x7f35ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f6f781020aa8ec3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:10.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.704+0000 7f35bb7fe640 1 -- 172.21.15.43:0/3179203213 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a8007d70 con 0x7f35ac004800 2024-04-25T08:54:10.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.704+0000 7f35bb7fe640 1 -- 172.21.15.43:0/3179203213 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a8005af0 con 0x7f35ac004800 2024-04-25T08:54:10.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.704+0000 7f35bb7fe640 1 -- 172.21.15.43:0/3179203213 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a8007070 con 0x7f35ac004800 2024-04-25T08:54:10.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.705+0000 7f35c2d68640 1 -- 172.21.15.43:0/3179203213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35ac004800 msgr2=0x7f35ac004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:10.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.705+0000 7f35c2d68640 1 --2- 172.21.15.43:0/3179203213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35ac004800 0x7f35ac004c80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f35a8001050 tx=0x7f35ac005830 comp rx=0 tx=0).stop 2024-04-25T08:54:10.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.705+0000 7f35c1305640 1 -- 172.21.15.43:0/3179203213 reap_dead start 2024-04-25T08:54:10.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.705+0000 7f35c2d68640 1 -- 172.21.15.43:0/3179203213 shutdown_connections 2024-04-25T08:54:10.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.705+0000 7f35c2d68640 1 -- 172.21.15.43:0/3179203213 >> 172.21.15.43:0/3179203213 conn(0x7f35bc06b1e0 msgr2=0x7f35bc06b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:10.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.705+0000 7f35c2d68640 1 -- 172.21.15.43:0/3179203213 shutdown_connections 2024-04-25T08:54:10.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.705+0000 7f35c2d68640 1 -- 172.21.15.43:0/3179203213 wait complete. 2024-04-25T08:54:10.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.706+0000 7f35c2d68640 1 Processor -- start 2024-04-25T08:54:10.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.706+0000 7f35c2d68640 1 -- start start 2024-04-25T08:54:10.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.707+0000 7f35c2d68640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35ac004800 0x7f35bc0cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:10.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.707+0000 7f35c2d68640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35bc150010 0x7f35bc1f7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:10.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.707+0000 7f35c2d68640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35bc0c8230 0x7f35bc0c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:10.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.707+0000 7f35c2d68640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc0ce280 con 0x7f35bc0c8230 2024-04-25T08:54:10.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.707+0000 7f35c2d68640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc0cf0c0 con 0x7f35ac004800 2024-04-25T08:54:10.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.707+0000 7f35c2d68640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc0ce400 con 0x7f35bc150010 2024-04-25T08:54:10.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.707+0000 7f35c1305640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35bc0c8230 0x7f35bc0c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:10.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.707+0000 7f35c1305640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35bc0c8230 0x7f35bc0c86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37152/0 (socket says 172.21.15.43:37152) 2024-04-25T08:54:10.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.707+0000 7f35c1305640 1 -- 172.21.15.43:0/3898829333 learned_addr learned my addr 172.21.15.43:0/3898829333 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:10.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.707+0000 7f35bbfff640 1 --2- 172.21.15.43:0/3898829333 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35bc150010 0x7f35bc1f7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:10.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.707+0000 7f35c0b04640 1 --2- 172.21.15.43:0/3898829333 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35ac004800 0x7f35bc0cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:10.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.707+0000 7f35c1305640 1 -- 172.21.15.43:0/3898829333 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35bc150010 msgr2=0x7f35bc1f7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:10.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.707+0000 7f35c1305640 1 --2- 172.21.15.43:0/3898829333 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35bc150010 0x7f35bc1f7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:10.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.707+0000 7f35c1305640 1 -- 172.21.15.43:0/3898829333 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35ac004800 msgr2=0x7f35bc0cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:10.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.707+0000 7f35c1305640 1 --2- 172.21.15.43:0/3898829333 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35ac004800 0x7f35bc0cc570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:10.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.707+0000 7f35c1305640 1 -- 172.21.15.43:0/3898829333 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35bc150490 con 0x7f35bc0c8230 2024-04-25T08:54:10.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.707+0000 7f35bbfff640 1 --2- 172.21.15.43:0/3898829333 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35bc150010 0x7f35bc1f7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:10.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.708+0000 7f35c1305640 1 --2- 172.21.15.43:0/3898829333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35bc0c8230 0x7f35bc0c86b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f35b4011c80 tx=0x7f35b400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:10.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.708+0000 7f35b97fa640 1 -- 172.21.15.43:0/3898829333 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b4019070 con 0x7f35bc0c8230 2024-04-25T08:54:10.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.708+0000 7f35b97fa640 1 -- 172.21.15.43:0/3898829333 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b4003990 con 0x7f35bc0c8230 2024-04-25T08:54:10.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.708+0000 7f35b97fa640 1 -- 172.21.15.43:0/3898829333 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b4003d80 con 0x7f35bc0c8230 2024-04-25T08:54:10.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.708+0000 7f35c2d68640 1 -- 172.21.15.43:0/3898829333 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35bc1f8200 con 0x7f35bc0c8230 2024-04-25T08:54:10.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.708+0000 7f35c2d68640 1 -- 172.21.15.43:0/3898829333 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35bc1f87c0 con 0x7f35bc0c8230 2024-04-25T08:54:10.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.709+0000 7f35c2d68640 1 -- 172.21.15.43:0/3898829333 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35bc0680f0 con 0x7f35bc0c8230 2024-04-25T08:54:10.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.709+0000 7f35b97fa640 1 -- 172.21.15.43:0/3898829333 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f35b401c070 con 0x7f35bc0c8230 2024-04-25T08:54:10.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.710+0000 7f35b97fa640 1 --2- 172.21.15.43:0/3898829333 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f35a006b4d0 0x7f35a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:10.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.710+0000 7f35b97fa640 1 -- 172.21.15.43:0/3898829333 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f35b408a940 con 0x7f35bc0c8230 2024-04-25T08:54:10.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.710+0000 7f35c0b04640 1 --2- 172.21.15.43:0/3898829333 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f35a006b4d0 0x7f35a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:10.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.711+0000 7f35c0b04640 1 --2- 172.21.15.43:0/3898829333 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f35a006b4d0 0x7f35a006d990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f35b0009980 tx=0x7f35b0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:10.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.715+0000 7f35b97fa640 1 -- 172.21.15.43:0/3898829333 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f35b4010040 con 0x7f35bc0c8230 2024-04-25T08:54:10.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:10.894+0000 7f35c2d68640 1 -- 172.21.15.43:0/3898829333 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f35bc074f60 con 0x7f35bc0c8230 2024-04-25T08:54:11.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.567+0000 7f35b97fa640 1 -- 172.21.15.43:0/3898829333 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) v1 ==== 170+0+0 (secure 0 0 0) 0x7f35b4059990 con 0x7f35bc0c8230 2024-04-25T08:54:11.558 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' created 2024-04-25T08:54:11.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.571+0000 7f35c2d68640 1 -- 172.21.15.43:0/3898829333 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f35a006b4d0 msgr2=0x7f35a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:11.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.571+0000 7f35c2d68640 1 --2- 172.21.15.43:0/3898829333 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f35a006b4d0 0x7f35a006d990 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f35b0009980 tx=0x7f35b0008040 comp rx=0 tx=0).stop 2024-04-25T08:54:11.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.571+0000 7f35c2d68640 1 -- 172.21.15.43:0/3898829333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35bc0c8230 msgr2=0x7f35bc0c86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:11.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.571+0000 7f35c2d68640 1 --2- 172.21.15.43:0/3898829333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35bc0c8230 0x7f35bc0c86b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f35b4011c80 tx=0x7f35b400f580 comp rx=0 tx=0).stop 2024-04-25T08:54:11.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.571+0000 7f35c2d68640 1 -- 172.21.15.43:0/3898829333 shutdown_connections 2024-04-25T08:54:11.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.571+0000 7f35c2d68640 1 --2- 172.21.15.43:0/3898829333 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35bc0c8230 0x7f35bc0c86b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:11.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.571+0000 7f35c2d68640 1 --2- 172.21.15.43:0/3898829333 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35bc150010 0x7f35bc1f7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:11.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.571+0000 7f35c2d68640 1 --2- 172.21.15.43:0/3898829333 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f35a006b4d0 0x7f35a006d990 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f35b0009980 tx=0x7f35b0008040 comp rx=0 tx=0).stop 2024-04-25T08:54:11.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.571+0000 7f35c2d68640 1 --2- 172.21.15.43:0/3898829333 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35ac004800 0x7f35bc0cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:11.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.571+0000 7f35c2d68640 1 -- 172.21.15.43:0/3898829333 >> 172.21.15.43:0/3898829333 conn(0x7f35bc06b1e0 msgr2=0x7f35bc15e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:11.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.572+0000 7f35c2d68640 1 -- 172.21.15.43:0/3898829333 shutdown_connections 2024-04-25T08:54:11.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.572+0000 7f35c2d68640 1 -- 172.21.15.43:0/3898829333 wait complete. 2024-04-25T08:54:11.591 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T08:54:11.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.710+0000 7fa623ee1640 1 Processor -- start 2024-04-25T08:54:11.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.711+0000 7fa623ee1640 1 -- start start 2024-04-25T08:54:11.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.711+0000 7fa623ee1640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa61c0d1a20 0x7fa61c0b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:11.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.711+0000 7fa623ee1640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa61c15bf90 con 0x7fa61c0d1e00 2024-04-25T08:54:11.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.711+0000 7fa623ee1640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa61c0cee70 con 0x7fa61c0d2260 2024-04-25T08:54:11.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.711+0000 7fa623ee1640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c0cf430 con 0x7fa61c0d1a20 2024-04-25T08:54:11.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.712+0000 7fa62147c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa61c0d1e00 0x7fa61c0b5100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60778/0 (socket says 172.21.15.43:60778) 2024-04-25T08:54:11.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.712+0000 7fa62147c640 1 -- 172.21.15.43:0/40501798 learned_addr learned my addr 172.21.15.43:0/40501798 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:11.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.712+0000 7fa621c7d640 1 --2- 172.21.15.43:0/40501798 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa61c0d1a20 0x7fa61c0b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:11.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.712+0000 7fa620c7b640 1 -- 172.21.15.43:0/40501798 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 452007497 0 0) 0x7fa61c15bf90 con 0x7fa61c0d1e00 2024-04-25T08:54:11.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.712+0000 7fa620c7b640 1 -- 172.21.15.43:0/40501798 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa608003520 con 0x7fa61c0d1e00 2024-04-25T08:54:11.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.712+0000 7fa621c7d640 1 -- 172.21.15.43:0/40501798 >> v1:172.21.15.193:6789/0 conn(0x7fa61c0d2260 legacy=0x7fa61c0b5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:11.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.712+0000 7fa620c7b640 1 -- 172.21.15.43:0/40501798 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2815025289 0 0) 0x7fa61c0cee70 con 0x7fa61c0d2260 2024-04-25T08:54:11.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.712+0000 7fa621c7d640 1 -- 172.21.15.43:0/40501798 >> v1:172.21.15.43:6789/0 conn(0x7fa61c0d1e00 legacy=0x7fa61c0b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:11.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.712+0000 7fa621c7d640 1 -- 172.21.15.43:0/40501798 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa61c0cf150 con 0x7fa61c0d1a20 2024-04-25T08:54:11.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.713+0000 7fa621c7d640 1 --2- 172.21.15.43:0/40501798 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa61c0d1a20 0x7fa61c0b2b90 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa6180097d0 tx=0x7fa61802a050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77dfe6ddb9de1961 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:11.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.713+0000 7fa620c7b640 1 -- 172.21.15.43:0/40501798 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa618037070 con 0x7fa61c0d1a20 2024-04-25T08:54:11.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.713+0000 7fa620c7b640 1 -- 172.21.15.43:0/40501798 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa61802aae0 con 0x7fa61c0d1a20 2024-04-25T08:54:11.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.713+0000 7fa620c7b640 1 -- 172.21.15.43:0/40501798 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa60c0015f0 con 0x7fa61c0d1a20 2024-04-25T08:54:11.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.714+0000 7fa623ee1640 1 -- 172.21.15.43:0/40501798 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa61c0d1a20 msgr2=0x7fa61c0b2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:11.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.714+0000 7fa623ee1640 1 --2- 172.21.15.43:0/40501798 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa61c0d1a20 0x7fa61c0b2b90 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa6180097d0 tx=0x7fa61802a050 comp rx=0 tx=0).stop 2024-04-25T08:54:11.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.714+0000 7fa623ee1640 1 -- 172.21.15.43:0/40501798 shutdown_connections 2024-04-25T08:54:11.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.714+0000 7fa623ee1640 1 --2- 172.21.15.43:0/40501798 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa61c0d1a20 0x7fa61c0b2b90 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:11.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.714+0000 7fa623ee1640 1 -- 172.21.15.43:0/40501798 >> 172.21.15.43:0/40501798 conn(0x7fa61c072aa0 msgr2=0x7fa61c09aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:11.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.714+0000 7fa623ee1640 1 -- 172.21.15.43:0/40501798 shutdown_connections 2024-04-25T08:54:11.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.714+0000 7fa623ee1640 1 -- 172.21.15.43:0/40501798 wait complete. 2024-04-25T08:54:11.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.715+0000 7fa623ee1640 1 Processor -- start 2024-04-25T08:54:11.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.715+0000 7fa623ee1640 1 -- start start 2024-04-25T08:54:11.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.716+0000 7fa623ee1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa61c0d1a20 0x7fa61c09f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:11.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.716+0000 7fa623ee1640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa61c0d1e00 0x7fa61c159d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:11.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.716+0000 7fa623ee1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa61c0d2260 0x7fa61c15a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:11.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.716+0000 7fa623ee1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c0cea60 con 0x7fa61c0d2260 2024-04-25T08:54:11.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.716+0000 7fa623ee1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c0b4be0 con 0x7fa61c0d1a20 2024-04-25T08:54:11.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.716+0000 7fa623ee1640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c0cebe0 con 0x7fa61c0d1e00 2024-04-25T08:54:11.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.716+0000 7fa62247e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa61c0d2260 0x7fa61c15a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:11.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.716+0000 7fa62247e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa61c0d2260 0x7fa61c15a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37168/0 (socket says 172.21.15.43:37168) 2024-04-25T08:54:11.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.716+0000 7fa62247e640 1 -- 172.21.15.43:0/3375724890 learned_addr learned my addr 172.21.15.43:0/3375724890 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:11.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.716+0000 7fa62147c640 1 --2- 172.21.15.43:0/3375724890 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa61c0d1e00 0x7fa61c159d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:11.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.716+0000 7fa621c7d640 1 --2- 172.21.15.43:0/3375724890 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa61c0d1a20 0x7fa61c09f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:11.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.716+0000 7fa62247e640 1 -- 172.21.15.43:0/3375724890 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa61c0d1e00 msgr2=0x7fa61c159d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:11.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.716+0000 7fa62247e640 1 --2- 172.21.15.43:0/3375724890 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa61c0d1e00 0x7fa61c159d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:11.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.716+0000 7fa62247e640 1 -- 172.21.15.43:0/3375724890 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa61c0d1a20 msgr2=0x7fa61c09f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:11.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.716+0000 7fa62247e640 1 --2- 172.21.15.43:0/3375724890 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa61c0d1a20 0x7fa61c09f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:11.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.716+0000 7fa62247e640 1 -- 172.21.15.43:0/3375724890 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa61c1f2240 con 0x7fa61c0d2260 2024-04-25T08:54:11.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.717+0000 7fa62247e640 1 --2- 172.21.15.43:0/3375724890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa61c0d2260 0x7fa61c15a290 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa61400c750 tx=0x7fa61400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:11.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.717+0000 7fa6127fc640 1 -- 172.21.15.43:0/3375724890 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa61400ce60 con 0x7fa61c0d2260 2024-04-25T08:54:11.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.717+0000 7fa6127fc640 1 -- 172.21.15.43:0/3375724890 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa614003da0 con 0x7fa61c0d2260 2024-04-25T08:54:11.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.717+0000 7fa623ee1640 1 -- 172.21.15.43:0/3375724890 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa61c1f2800 con 0x7fa61c0d2260 2024-04-25T08:54:11.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.717+0000 7fa6127fc640 1 -- 172.21.15.43:0/3375724890 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa614005290 con 0x7fa61c0d2260 2024-04-25T08:54:11.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.717+0000 7fa623ee1640 1 -- 172.21.15.43:0/3375724890 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa61c1f2dc0 con 0x7fa61c0d2260 2024-04-25T08:54:11.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.718+0000 7fa623ee1640 1 -- 172.21.15.43:0/3375724890 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa61c09f960 con 0x7fa61c0d2260 2024-04-25T08:54:11.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.718+0000 7fa6127fc640 1 -- 172.21.15.43:0/3375724890 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa614005430 con 0x7fa61c0d2260 2024-04-25T08:54:11.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.719+0000 7fa6127fc640 1 --2- 172.21.15.43:0/3375724890 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa60006b500 0x7fa60006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:11.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.719+0000 7fa6127fc640 1 -- 172.21.15.43:0/3375724890 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fa614089fb0 con 0x7fa61c0d2260 2024-04-25T08:54:11.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.719+0000 7fa621c7d640 1 --2- 172.21.15.43:0/3375724890 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa60006b500 0x7fa60006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:11.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.719+0000 7fa621c7d640 1 --2- 172.21.15.43:0/3375724890 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa60006b500 0x7fa60006d9c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa618034000 tx=0x7fa61802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:11.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.723+0000 7fa6127fc640 1 -- 172.21.15.43:0/3375724890 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa614058840 con 0x7fa61c0d2260 2024-04-25T08:54:11.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:11.944+0000 7fa623ee1640 1 -- 172.21.15.43:0/3375724890 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fa61c074f60 con 0x7fa61c0d2260 2024-04-25T08:54:12.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.582+0000 7fa6127fc640 1 -- 172.21.15.43:0/3375724890 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fa61405c8e0 con 0x7fa61c0d2260 2024-04-25T08:54:12.572 INFO:teuthology.orchestra.run.smithi043.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T08:54:12.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.585+0000 7fa623ee1640 1 -- 172.21.15.43:0/3375724890 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa60006b500 msgr2=0x7fa60006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:12.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.585+0000 7fa623ee1640 1 --2- 172.21.15.43:0/3375724890 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa60006b500 0x7fa60006d9c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa618034000 tx=0x7fa61802e000 comp rx=0 tx=0).stop 2024-04-25T08:54:12.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.585+0000 7fa623ee1640 1 -- 172.21.15.43:0/3375724890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa61c0d2260 msgr2=0x7fa61c15a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:12.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.585+0000 7fa623ee1640 1 --2- 172.21.15.43:0/3375724890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa61c0d2260 0x7fa61c15a290 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa61400c750 tx=0x7fa61400cc10 comp rx=0 tx=0).stop 2024-04-25T08:54:12.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.586+0000 7fa623ee1640 1 -- 172.21.15.43:0/3375724890 shutdown_connections 2024-04-25T08:54:12.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.586+0000 7fa623ee1640 1 --2- 172.21.15.43:0/3375724890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa61c0d2260 0x7fa61c15a290 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:12.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.586+0000 7fa623ee1640 1 --2- 172.21.15.43:0/3375724890 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa61c0d1e00 0x7fa61c159d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:12.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.586+0000 7fa623ee1640 1 --2- 172.21.15.43:0/3375724890 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa60006b500 0x7fa60006d9c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:12.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.586+0000 7fa623ee1640 1 --2- 172.21.15.43:0/3375724890 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa61c0d1a20 0x7fa61c09f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:12.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.586+0000 7fa623ee1640 1 -- 172.21.15.43:0/3375724890 >> 172.21.15.43:0/3375724890 conn(0x7fa61c072aa0 msgr2=0x7fa61c06fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:12.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.586+0000 7fa623ee1640 1 -- 172.21.15.43:0/3375724890 shutdown_connections 2024-04-25T08:54:12.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.586+0000 7fa623ee1640 1 -- 172.21.15.43:0/3375724890 wait complete. 2024-04-25T08:54:12.606 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:54:12.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.722+0000 7f03ba06b640 1 Processor -- start 2024-04-25T08:54:12.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.722+0000 7f03ba06b640 1 -- start start 2024-04-25T08:54:12.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.723+0000 7f03ba06b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f03b40cf670 0x7f03b4168d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:12.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.723+0000 7f03ba06b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03b4162810 con 0x7f03b415df50 2024-04-25T08:54:12.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.723+0000 7f03ba06b640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03b40d24a0 con 0x7f03b40c8d00 2024-04-25T08:54:12.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.723+0000 7f03ba06b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03b40cfa80 con 0x7f03b40cf670 2024-04-25T08:54:12.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.723+0000 7f03b3fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f03b40cf670 0x7f03b4168d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:12.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.723+0000 7f03b37fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f03b415df50 0x7f03b41619f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60786/0 (socket says 172.21.15.43:60786) 2024-04-25T08:54:12.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.723+0000 7f03b3fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f03b40cf670 0x7f03b4168d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37174/0 (socket says 172.21.15.43:37174) 2024-04-25T08:54:12.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.723+0000 7f03b37fe640 1 -- 172.21.15.43:0/2777795932 learned_addr learned my addr 172.21.15.43:0/2777795932 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:12.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.723+0000 7f03b3fff640 1 -- 172.21.15.43:0/2777795932 >> v1:172.21.15.193:6789/0 conn(0x7f03b40c8d00 legacy=0x7f03b4162100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:12.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.723+0000 7f03b3fff640 1 -- 172.21.15.43:0/2777795932 >> v1:172.21.15.43:6789/0 conn(0x7f03b415df50 legacy=0x7f03b41619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:12.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.723+0000 7f03b27fc640 1 -- 172.21.15.43:0/2777795932 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2464879705 0 0) 0x7f03b4162810 con 0x7f03b415df50 2024-04-25T08:54:12.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.723+0000 7f03b3fff640 1 -- 172.21.15.43:0/2777795932 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b40d2740 con 0x7f03b40cf670 2024-04-25T08:54:12.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.724+0000 7f03b3fff640 1 --2- 172.21.15.43:0/2777795932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03b40cf670 0x7f03b4168d20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f03ac009d90 tx=0x7f03ac02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36650225694d4f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:12.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.724+0000 7f03b27fc640 1 -- 172.21.15.43:0/2777795932 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac007740 con 0x7f03b40cf670 2024-04-25T08:54:12.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.724+0000 7f03b27fc640 1 -- 172.21.15.43:0/2777795932 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03ac0078e0 con 0x7f03b40cf670 2024-04-25T08:54:12.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.724+0000 7f03b27fc640 1 -- 172.21.15.43:0/2777795932 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac007cb0 con 0x7f03b40cf670 2024-04-25T08:54:12.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.725+0000 7f03ba06b640 1 -- 172.21.15.43:0/2777795932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03b40cf670 msgr2=0x7f03b4168d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:12.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.725+0000 7f03ba06b640 1 --2- 172.21.15.43:0/2777795932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03b40cf670 0x7f03b4168d20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f03ac009d90 tx=0x7f03ac02a6f0 comp rx=0 tx=0).stop 2024-04-25T08:54:12.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.725+0000 7f03ba06b640 1 -- 172.21.15.43:0/2777795932 shutdown_connections 2024-04-25T08:54:12.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.725+0000 7f03ba06b640 1 --2- 172.21.15.43:0/2777795932 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03b40cf670 0x7f03b4168d20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:12.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.725+0000 7f03ba06b640 1 -- 172.21.15.43:0/2777795932 >> 172.21.15.43:0/2777795932 conn(0x7f03b40d3a30 msgr2=0x7f03b40d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:12.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.725+0000 7f03ba06b640 1 -- 172.21.15.43:0/2777795932 shutdown_connections 2024-04-25T08:54:12.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.725+0000 7f03ba06b640 1 -- 172.21.15.43:0/2777795932 wait complete. 2024-04-25T08:54:12.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.726+0000 7f03ba06b640 1 Processor -- start 2024-04-25T08:54:12.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.726+0000 7f03ba06b640 1 -- start start 2024-04-25T08:54:12.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.727+0000 7f03ba06b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03b40c8d00 0x7f03b41599c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:12.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.727+0000 7f03ba06b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03b40cf670 0x7f03b41f7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:12.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.727+0000 7f03ba06b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03b415df50 0x7f03b414fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:12.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.727+0000 7f03ba06b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b40c74f0 con 0x7f03b40cf670 2024-04-25T08:54:12.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.727+0000 7f03ba06b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b415e330 con 0x7f03b40c8d00 2024-04-25T08:54:12.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.727+0000 7f03ba06b640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03b40c77d0 con 0x7f03b415df50 2024-04-25T08:54:12.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.727+0000 7f03b2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03b40cf670 0x7f03b41f7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:12.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.727+0000 7f03b2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03b40cf670 0x7f03b41f7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37180/0 (socket says 172.21.15.43:37180) 2024-04-25T08:54:12.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.727+0000 7f03b2ffd640 1 -- 172.21.15.43:0/217384307 learned_addr learned my addr 172.21.15.43:0/217384307 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:12.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.727+0000 7f03b3fff640 1 --2- 172.21.15.43:0/217384307 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03b415df50 0x7f03b414fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:12.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.727+0000 7f03b37fe640 1 --2- 172.21.15.43:0/217384307 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03b40c8d00 0x7f03b41599c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:12.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.727+0000 7f03b2ffd640 1 -- 172.21.15.43:0/217384307 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03b415df50 msgr2=0x7f03b414fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:12.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.727+0000 7f03b2ffd640 1 --2- 172.21.15.43:0/217384307 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03b415df50 0x7f03b414fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:12.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.727+0000 7f03b2ffd640 1 -- 172.21.15.43:0/217384307 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03b40c8d00 msgr2=0x7f03b41599c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:12.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.727+0000 7f03b2ffd640 1 --2- 172.21.15.43:0/217384307 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03b40c8d00 0x7f03b41599c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:12.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.727+0000 7f03b2ffd640 1 -- 172.21.15.43:0/217384307 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b4159fa0 con 0x7f03b40cf670 2024-04-25T08:54:12.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.727+0000 7f03b2ffd640 1 --2- 172.21.15.43:0/217384307 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03b40cf670 0x7f03b41f7de0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f03a800c450 tx=0x7f03a800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:12.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.728+0000 7f039bfff640 1 -- 172.21.15.43:0/217384307 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a80052d0 con 0x7f03b40cf670 2024-04-25T08:54:12.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.728+0000 7f039bfff640 1 -- 172.21.15.43:0/217384307 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a8005470 con 0x7f03b40cf670 2024-04-25T08:54:12.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.728+0000 7f039bfff640 1 -- 172.21.15.43:0/217384307 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a8012870 con 0x7f03b40cf670 2024-04-25T08:54:12.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.728+0000 7f03ba06b640 1 -- 172.21.15.43:0/217384307 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03b415a280 con 0x7f03b40cf670 2024-04-25T08:54:12.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.728+0000 7f03ba06b640 1 -- 172.21.15.43:0/217384307 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03b41f78c0 con 0x7f03b40cf670 2024-04-25T08:54:12.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.729+0000 7f03ba06b640 1 -- 172.21.15.43:0/217384307 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03b40680f0 con 0x7f03b40cf670 2024-04-25T08:54:12.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.729+0000 7f039bfff640 1 -- 172.21.15.43:0/217384307 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f03a8012a10 con 0x7f03b40cf670 2024-04-25T08:54:12.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.729+0000 7f039bfff640 1 --2- 172.21.15.43:0/217384307 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f03a006b4d0 0x7f03a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:12.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.730+0000 7f039bfff640 1 -- 172.21.15.43:0/217384307 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f03a8089ce0 con 0x7f03b40cf670 2024-04-25T08:54:12.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.730+0000 7f03b37fe640 1 --2- 172.21.15.43:0/217384307 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f03a006b4d0 0x7f03a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:12.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.730+0000 7f03b37fe640 1 --2- 172.21.15.43:0/217384307 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f03a006b4d0 0x7f03a006d990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f03a4000f60 tx=0x7f03a4007e40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:12.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.734+0000 7f039bfff640 1 -- 172.21.15.43:0/217384307 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f03a8058510 con 0x7f03b40cf670 2024-04-25T08:54:12.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.916+0000 7f03ba06b640 1 -- 172.21.15.43:0/217384307 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03b41f73d0 con 0x7f03b40cf670 2024-04-25T08:54:12.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.917+0000 7f039bfff640 1 -- 172.21.15.43:0/217384307 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16582 (secure 0 0 0) 0x7f03a805c5b0 con 0x7f03b40cf670 2024-04-25T08:54:12.908 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:54:12.908 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":19,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:54:11.945660+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:54:09.757978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:54:10.926008+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:54:12.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.919+0000 7f03ba06b640 1 -- 172.21.15.43:0/217384307 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f03a006b4d0 msgr2=0x7f03a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:12.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.919+0000 7f03ba06b640 1 --2- 172.21.15.43:0/217384307 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f03a006b4d0 0x7f03a006d990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f03a4000f60 tx=0x7f03a4007e40 comp rx=0 tx=0).stop 2024-04-25T08:54:12.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.919+0000 7f03ba06b640 1 -- 172.21.15.43:0/217384307 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03b40cf670 msgr2=0x7f03b41f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:12.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.919+0000 7f03ba06b640 1 --2- 172.21.15.43:0/217384307 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03b40cf670 0x7f03b41f7de0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f03a800c450 tx=0x7f03a800c910 comp rx=0 tx=0).stop 2024-04-25T08:54:12.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.919+0000 7f03ba06b640 1 -- 172.21.15.43:0/217384307 shutdown_connections 2024-04-25T08:54:12.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.919+0000 7f03ba06b640 1 --2- 172.21.15.43:0/217384307 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03b415df50 0x7f03b414fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:12.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.919+0000 7f03ba06b640 1 --2- 172.21.15.43:0/217384307 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03b40cf670 0x7f03b41f7de0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:12.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.919+0000 7f03ba06b640 1 --2- 172.21.15.43:0/217384307 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f03a006b4d0 0x7f03a006d990 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:12.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.919+0000 7f03ba06b640 1 --2- 172.21.15.43:0/217384307 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03b40c8d00 0x7f03b41599c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:12.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.919+0000 7f03ba06b640 1 -- 172.21.15.43:0/217384307 >> 172.21.15.43:0/217384307 conn(0x7f03b40d3a30 msgr2=0x7f03b406c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:12.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.919+0000 7f03ba06b640 1 -- 172.21.15.43:0/217384307 shutdown_connections 2024-04-25T08:54:12.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:12.920+0000 7f03ba06b640 1 -- 172.21.15.43:0/217384307 wait complete. 2024-04-25T08:54:12.921 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:54:13.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.031+0000 7f442ce7c640 1 Processor -- start 2024-04-25T08:54:13.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.031+0000 7f442ce7c640 1 -- start start 2024-04-25T08:54:13.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.032+0000 7f442ce7c640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f44280c8d00 0x7f4428162100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:13.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.032+0000 7f442ce7c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f44280ce660 0x7f44280cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:13.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.032+0000 7f442ce7c640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44280d24a0 con 0x7f442815df50 2024-04-25T08:54:13.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.032+0000 7f442ce7c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f442815e330 con 0x7f44280ce660 2024-04-25T08:54:13.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.032+0000 7f442ce7c640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44280d2740 con 0x7f44280c8d00 2024-04-25T08:54:13.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.032+0000 7f4426d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f44280ce660 0x7f44280cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:13.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.032+0000 7f4426d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f44280ce660 0x7f44280cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37184/0 (socket says 172.21.15.43:37184) 2024-04-25T08:54:13.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.032+0000 7f4426d9d640 1 -- 172.21.15.43:0/1362813699 learned_addr learned my addr 172.21.15.43:0/1362813699 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:13.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.032+0000 7f441ffff640 1 --2- 172.21.15.43:0/1362813699 >> v2:172.21.15.193:3300/0 conn(0x7f44280c8d00 0x7f4428162100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:13.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.032+0000 7f4426d9d640 1 -- 172.21.15.43:0/1362813699 >> v2:172.21.15.193:3300/0 conn(0x7f44280c8d00 msgr2=0x7f4428162100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:13.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.032+0000 7f4426d9d640 1 --2- 172.21.15.43:0/1362813699 >> v2:172.21.15.193:3300/0 conn(0x7f44280c8d00 0x7f4428162100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:13.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.032+0000 7f4426d9d640 1 -- 172.21.15.43:0/1362813699 >> v1:172.21.15.193:6789/0 conn(0x7f442815df50 legacy=0x7f44281619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:13.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.032+0000 7f4426d9d640 1 -- 172.21.15.43:0/1362813699 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44280d2a20 con 0x7f44280ce660 2024-04-25T08:54:13.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.033+0000 7f4426d9d640 1 --2- 172.21.15.43:0/1362813699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44280ce660 0x7f44280cea70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f44280caa00 tx=0x7f442002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15134df378787a28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:13.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.033+0000 7f4425d9b640 1 -- 172.21.15.43:0/1362813699 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4420036070 con 0x7f44280ce660 2024-04-25T08:54:13.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.033+0000 7f4425d9b640 1 -- 172.21.15.43:0/1362813699 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4420031600 con 0x7f44280ce660 2024-04-25T08:54:13.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.033+0000 7f4425d9b640 1 -- 172.21.15.43:0/1362813699 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44200319d0 con 0x7f44280ce660 2024-04-25T08:54:13.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.033+0000 7f442ce7c640 1 -- 172.21.15.43:0/1362813699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44280ce660 msgr2=0x7f44280cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:13.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.033+0000 7f442ce7c640 1 --2- 172.21.15.43:0/1362813699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44280ce660 0x7f44280cea70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f44280caa00 tx=0x7f442002a6c0 comp rx=0 tx=0).stop 2024-04-25T08:54:13.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.034+0000 7f442ce7c640 1 -- 172.21.15.43:0/1362813699 shutdown_connections 2024-04-25T08:54:13.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.034+0000 7f442ce7c640 1 --2- 172.21.15.43:0/1362813699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44280ce660 0x7f44280cea70 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:13.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.034+0000 7f442ce7c640 1 --2- 172.21.15.43:0/1362813699 >> v2:172.21.15.193:3300/0 conn(0x7f44280c8d00 0x7f4428162100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:13.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.034+0000 7f442ce7c640 1 -- 172.21.15.43:0/1362813699 >> 172.21.15.43:0/1362813699 conn(0x7f44280d3a30 msgr2=0x7f44280d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:13.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.034+0000 7f442ce7c640 1 -- 172.21.15.43:0/1362813699 shutdown_connections 2024-04-25T08:54:13.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.034+0000 7f442ce7c640 1 -- 172.21.15.43:0/1362813699 wait complete. 2024-04-25T08:54:13.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.035+0000 7f442ce7c640 1 Processor -- start 2024-04-25T08:54:13.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.035+0000 7f442ce7c640 1 -- start start 2024-04-25T08:54:13.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.036+0000 7f442ce7c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44280c8d00 0x7f44280cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:13.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.036+0000 7f442ce7c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44280ce660 0x7f44280ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:13.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.036+0000 7f442ce7c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f442815df50 0x7f4428160ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:13.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.036+0000 7f442ce7c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44280d2190 con 0x7f44280c8d00 2024-04-25T08:54:13.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.036+0000 7f442ce7c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44280c77d0 con 0x7f44280ce660 2024-04-25T08:54:13.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.036+0000 7f442ce7c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4428162640 con 0x7f442815df50 2024-04-25T08:54:13.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.036+0000 7f442659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44280c8d00 0x7f44280cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:13.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.036+0000 7f442659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44280c8d00 0x7f44280cadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37192/0 (socket says 172.21.15.43:37192) 2024-04-25T08:54:13.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.036+0000 7f442659c640 1 -- 172.21.15.43:0/3320494035 learned_addr learned my addr 172.21.15.43:0/3320494035 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:13.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.036+0000 7f441ffff640 1 --2- 172.21.15.43:0/3320494035 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44280ce660 0x7f44280ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:13.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.036+0000 7f4426d9d640 1 --2- 172.21.15.43:0/3320494035 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f442815df50 0x7f4428160ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:13.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.036+0000 7f442659c640 1 -- 172.21.15.43:0/3320494035 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f442815df50 msgr2=0x7f4428160ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:13.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.036+0000 7f442659c640 1 --2- 172.21.15.43:0/3320494035 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f442815df50 0x7f4428160ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:13.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.036+0000 7f442659c640 1 -- 172.21.15.43:0/3320494035 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44280ce660 msgr2=0x7f44280ccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:13.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.036+0000 7f442659c640 1 --2- 172.21.15.43:0/3320494035 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44280ce660 0x7f44280ccba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:13.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.036+0000 7f442659c640 1 -- 172.21.15.43:0/3320494035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4428154a90 con 0x7f44280c8d00 2024-04-25T08:54:13.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.036+0000 7f4426d9d640 1 --2- 172.21.15.43:0/3320494035 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f442815df50 0x7f4428160ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:13.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.036+0000 7f442659c640 1 --2- 172.21.15.43:0/3320494035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44280c8d00 0x7f44280cadc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f441800b3f0 tx=0x7f441800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:13.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.036+0000 7f441effd640 1 -- 172.21.15.43:0/3320494035 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4418015020 con 0x7f44280c8d00 2024-04-25T08:54:13.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.036+0000 7f441effd640 1 -- 172.21.15.43:0/3320494035 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4418011830 con 0x7f44280c8d00 2024-04-25T08:54:13.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.036+0000 7f442ce7c640 1 -- 172.21.15.43:0/3320494035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4428155000 con 0x7f44280c8d00 2024-04-25T08:54:13.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.036+0000 7f441effd640 1 -- 172.21.15.43:0/3320494035 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4418011c20 con 0x7f44280c8d00 2024-04-25T08:54:13.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.036+0000 7f442ce7c640 1 -- 172.21.15.43:0/3320494035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44281555c0 con 0x7f44280c8d00 2024-04-25T08:54:13.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.037+0000 7f442ce7c640 1 -- 172.21.15.43:0/3320494035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44280680f0 con 0x7f44280c8d00 2024-04-25T08:54:13.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.038+0000 7f441effd640 1 -- 172.21.15.43:0/3320494035 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4418018020 con 0x7f44280c8d00 2024-04-25T08:54:13.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.038+0000 7f441effd640 1 --2- 172.21.15.43:0/3320494035 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f440806b500 0x7f440806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:13.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.038+0000 7f441effd640 1 -- 172.21.15.43:0/3320494035 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4418089db0 con 0x7f44280c8d00 2024-04-25T08:54:13.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.038+0000 7f441ffff640 1 --2- 172.21.15.43:0/3320494035 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f440806b500 0x7f440806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:13.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.039+0000 7f441ffff640 1 --2- 172.21.15.43:0/3320494035 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f440806b500 0x7f440806d9c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f44281f7540 tx=0x7f4410008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:13.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.043+0000 7f441effd640 1 -- 172.21.15.43:0/3320494035 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f441801f030 con 0x7f44280c8d00 2024-04-25T08:54:13.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.225+0000 7f442ce7c640 1 -- 172.21.15.43:0/3320494035 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4428068f20 con 0x7f44280c8d00 2024-04-25T08:54:13.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.226+0000 7f441effd640 1 -- 172.21.15.43:0/3320494035 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16582 (secure 0 0 0) 0x7f4428068f20 con 0x7f44280c8d00 2024-04-25T08:54:13.216 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:54:13.217 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":19,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:54:11.945660+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:54:09.757978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:54:10.926008+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:54:13.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.228+0000 7f442ce7c640 1 -- 172.21.15.43:0/3320494035 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f440806b500 msgr2=0x7f440806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:13.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.228+0000 7f442ce7c640 1 --2- 172.21.15.43:0/3320494035 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f440806b500 0x7f440806d9c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f44281f7540 tx=0x7f4410008040 comp rx=0 tx=0).stop 2024-04-25T08:54:13.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.228+0000 7f442ce7c640 1 -- 172.21.15.43:0/3320494035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44280c8d00 msgr2=0x7f44280cadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:13.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.228+0000 7f442ce7c640 1 --2- 172.21.15.43:0/3320494035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44280c8d00 0x7f44280cadc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f441800b3f0 tx=0x7f441800b8b0 comp rx=0 tx=0).stop 2024-04-25T08:54:13.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.228+0000 7f442ce7c640 1 -- 172.21.15.43:0/3320494035 shutdown_connections 2024-04-25T08:54:13.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.228+0000 7f442ce7c640 1 --2- 172.21.15.43:0/3320494035 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f442815df50 0x7f4428160ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:13.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.229+0000 7f442ce7c640 1 --2- 172.21.15.43:0/3320494035 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f440806b500 0x7f440806d9c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:13.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.229+0000 7f442ce7c640 1 --2- 172.21.15.43:0/3320494035 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44280ce660 0x7f44280ccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:13.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.229+0000 7f442ce7c640 1 --2- 172.21.15.43:0/3320494035 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44280c8d00 0x7f44280cadc0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:13.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.229+0000 7f442ce7c640 1 -- 172.21.15.43:0/3320494035 >> 172.21.15.43:0/3320494035 conn(0x7f44280d3a30 msgr2=0x7f44280cf2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:13.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.229+0000 7f442ce7c640 1 -- 172.21.15.43:0/3320494035 shutdown_connections 2024-04-25T08:54:13.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.229+0000 7f442ce7c640 1 -- 172.21.15.43:0/3320494035 wait complete. 2024-04-25T08:54:13.229 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T08:54:13.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.343+0000 7fd84282a640 1 Processor -- start 2024-04-25T08:54:13.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.343+0000 7fd84282a640 1 -- start start 2024-04-25T08:54:13.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.344+0000 7fd84282a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd83c15e090 0x7fd83c161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:13.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.344+0000 7fd84282a640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fd83c0b5c70 0x7fd83c0d2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:13.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.344+0000 7fd84282a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd83c162610 con 0x7fd83c161fe0 2024-04-25T08:54:13.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.344+0000 7fd84282a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd83c0d1da0 con 0x7fd83c15e090 2024-04-25T08:54:13.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.344+0000 7fd84282a640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd83c0d1750 con 0x7fd83c0b5c70 2024-04-25T08:54:13.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.344+0000 7fd83bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd83c15e090 0x7fd83c161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:13.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.344+0000 7fd83bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd83c15e090 0x7fd83c161aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37204/0 (socket says 172.21.15.43:37204) 2024-04-25T08:54:13.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.344+0000 7fd83b7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fd83c161fe0 0x7fd83c0b5560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60798/0 (socket says 172.21.15.43:60798) 2024-04-25T08:54:13.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.344+0000 7fd83bfff640 1 -- 172.21.15.43:0/3865822485 learned_addr learned my addr 172.21.15.43:0/3865822485 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:13.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.345+0000 7fd840dc7640 1 --2- 172.21.15.43:0/3865822485 >> v2:172.21.15.193:3300/0 conn(0x7fd83c0b5c70 0x7fd83c0d2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:13.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.345+0000 7fd83bfff640 1 -- 172.21.15.43:0/3865822485 >> v2:172.21.15.193:3300/0 conn(0x7fd83c0b5c70 msgr2=0x7fd83c0d2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:13.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.345+0000 7fd83bfff640 1 --2- 172.21.15.43:0/3865822485 >> v2:172.21.15.193:3300/0 conn(0x7fd83c0b5c70 0x7fd83c0d2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:13.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.345+0000 7fd83bfff640 1 -- 172.21.15.43:0/3865822485 >> v1:172.21.15.43:6789/0 conn(0x7fd83c161fe0 legacy=0x7fd83c0b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:13.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.345+0000 7fd83bfff640 1 -- 172.21.15.43:0/3865822485 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd83c0d1a30 con 0x7fd83c15e090 2024-04-25T08:54:13.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.345+0000 7fd83affd640 1 -- 172.21.15.43:0/3865822485 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2241960905 0 0) 0x7fd83c162610 con 0x7fd83c161fe0 2024-04-25T08:54:13.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.345+0000 7fd83bfff640 1 --2- 172.21.15.43:0/3865822485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd83c15e090 0x7fd83c161aa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd83c1619e0 tx=0x7fd82c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ffbd0779a8355b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:13.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.345+0000 7fd83affd640 1 -- 172.21.15.43:0/3865822485 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c037070 con 0x7fd83c15e090 2024-04-25T08:54:13.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.345+0000 7fd83affd640 1 -- 172.21.15.43:0/3865822485 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd82c02ae50 con 0x7fd83c15e090 2024-04-25T08:54:13.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.345+0000 7fd83affd640 1 -- 172.21.15.43:0/3865822485 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c032860 con 0x7fd83c15e090 2024-04-25T08:54:13.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.346+0000 7fd84282a640 1 -- 172.21.15.43:0/3865822485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd83c15e090 msgr2=0x7fd83c161aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:13.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.346+0000 7fd84282a640 1 --2- 172.21.15.43:0/3865822485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd83c15e090 0x7fd83c161aa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd83c1619e0 tx=0x7fd82c02a300 comp rx=0 tx=0).stop 2024-04-25T08:54:13.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.346+0000 7fd84282a640 1 -- 172.21.15.43:0/3865822485 shutdown_connections 2024-04-25T08:54:13.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.346+0000 7fd84282a640 1 --2- 172.21.15.43:0/3865822485 >> v2:172.21.15.193:3300/0 conn(0x7fd83c0b5c70 0x7fd83c0d2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:13.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.346+0000 7fd84282a640 1 --2- 172.21.15.43:0/3865822485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd83c15e090 0x7fd83c161aa0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:13.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.346+0000 7fd84282a640 1 -- 172.21.15.43:0/3865822485 >> 172.21.15.43:0/3865822485 conn(0x7fd83c0d0560 msgr2=0x7fd83c0d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:13.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.346+0000 7fd84282a640 1 -- 172.21.15.43:0/3865822485 shutdown_connections 2024-04-25T08:54:13.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.346+0000 7fd84282a640 1 -- 172.21.15.43:0/3865822485 wait complete. 2024-04-25T08:54:13.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.347+0000 7fd84282a640 1 Processor -- start 2024-04-25T08:54:13.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.347+0000 7fd84282a640 1 -- start start 2024-04-25T08:54:13.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.348+0000 7fd84282a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd83c0b5c70 0x7fd83c155b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:13.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.348+0000 7fd84282a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd83c15e090 0x7fd83c141f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:13.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.348+0000 7fd84282a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd83c161fe0 0x7fd83c144210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:13.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.348+0000 7fd84282a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83c0d2770 con 0x7fd83c0b5c70 2024-04-25T08:54:13.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.348+0000 7fd84282a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83c0b3280 con 0x7fd83c15e090 2024-04-25T08:54:13.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.348+0000 7fd84282a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd83c0d28f0 con 0x7fd83c161fe0 2024-04-25T08:54:13.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.348+0000 7fd83bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd83c0b5c70 0x7fd83c155b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:13.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.348+0000 7fd83bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd83c0b5c70 0x7fd83c155b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37212/0 (socket says 172.21.15.43:37212) 2024-04-25T08:54:13.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.348+0000 7fd83bfff640 1 -- 172.21.15.43:0/510121542 learned_addr learned my addr 172.21.15.43:0/510121542 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:13.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.348+0000 7fd83bfff640 1 -- 172.21.15.43:0/510121542 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd83c161fe0 msgr2=0x7fd83c144210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:13.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.348+0000 7fd83bfff640 1 --2- 172.21.15.43:0/510121542 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd83c161fe0 0x7fd83c144210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:13.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.348+0000 7fd83bfff640 1 -- 172.21.15.43:0/510121542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd83c15e090 msgr2=0x7fd83c141f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:13.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.348+0000 7fd83bfff640 1 --2- 172.21.15.43:0/510121542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd83c15e090 0x7fd83c141f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:13.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.348+0000 7fd83bfff640 1 -- 172.21.15.43:0/510121542 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd83c15f5d0 con 0x7fd83c0b5c70 2024-04-25T08:54:13.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.349+0000 7fd83bfff640 1 --2- 172.21.15.43:0/510121542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd83c0b5c70 0x7fd83c155b00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd82c02a340 tx=0x7fd82c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:13.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.349+0000 7fd838ff9640 1 -- 172.21.15.43:0/510121542 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c041070 con 0x7fd83c0b5c70 2024-04-25T08:54:13.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.349+0000 7fd838ff9640 1 -- 172.21.15.43:0/510121542 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd82c03c670 con 0x7fd83c0b5c70 2024-04-25T08:54:13.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.349+0000 7fd84282a640 1 -- 172.21.15.43:0/510121542 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd83c15f8b0 con 0x7fd83c0b5c70 2024-04-25T08:54:13.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.349+0000 7fd838ff9640 1 -- 172.21.15.43:0/510121542 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c037040 con 0x7fd83c0b5c70 2024-04-25T08:54:13.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.349+0000 7fd84282a640 1 -- 172.21.15.43:0/510121542 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd83c1566a0 con 0x7fd83c0b5c70 2024-04-25T08:54:13.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.350+0000 7fd84282a640 1 -- 172.21.15.43:0/510121542 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd83c06bf00 con 0x7fd83c0b5c70 2024-04-25T08:54:13.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.350+0000 7fd838ff9640 1 -- 172.21.15.43:0/510121542 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd82c044070 con 0x7fd83c0b5c70 2024-04-25T08:54:13.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.351+0000 7fd838ff9640 1 --2- 172.21.15.43:0/510121542 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd82806b4d0 0x7fd82806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:13.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.351+0000 7fd838ff9640 1 -- 172.21.15.43:0/510121542 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd82c0aa9a0 con 0x7fd83c0b5c70 2024-04-25T08:54:13.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.351+0000 7fd83b7fe640 1 --2- 172.21.15.43:0/510121542 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd82806b4d0 0x7fd82806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:13.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.351+0000 7fd83b7fe640 1 --2- 172.21.15.43:0/510121542 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd82806b4d0 0x7fd82806d990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd830007fd0 tx=0x7fd830007960 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:13.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.355+0000 7fd838ff9640 1 -- 172.21.15.43:0/510121542 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd83c06bf00 con 0x7fd83c0b5c70 2024-04-25T08:54:13.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.575+0000 7fd84282a640 1 -- 172.21.15.43:0/510121542 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd83c074f60 con 0x7fd83c0b5c70 2024-04-25T08:54:13.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.593+0000 7fd838ff9640 1 -- 172.21.15.43:0/510121542 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd82c0798b0 con 0x7fd83c0b5c70 2024-04-25T08:54:13.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.595+0000 7fd84282a640 1 -- 172.21.15.43:0/510121542 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd82806b4d0 msgr2=0x7fd82806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:13.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.595+0000 7fd84282a640 1 --2- 172.21.15.43:0/510121542 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd82806b4d0 0x7fd82806d990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd830007fd0 tx=0x7fd830007960 comp rx=0 tx=0).stop 2024-04-25T08:54:13.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.595+0000 7fd84282a640 1 -- 172.21.15.43:0/510121542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd83c0b5c70 msgr2=0x7fd83c155b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:13.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.595+0000 7fd84282a640 1 --2- 172.21.15.43:0/510121542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd83c0b5c70 0x7fd83c155b00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd82c02a340 tx=0x7fd82c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:54:13.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.595+0000 7fd84282a640 1 -- 172.21.15.43:0/510121542 shutdown_connections 2024-04-25T08:54:13.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.595+0000 7fd84282a640 1 --2- 172.21.15.43:0/510121542 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd83c161fe0 0x7fd83c144210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:13.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.595+0000 7fd84282a640 1 --2- 172.21.15.43:0/510121542 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd82806b4d0 0x7fd82806d990 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:13.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.595+0000 7fd84282a640 1 --2- 172.21.15.43:0/510121542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd83c15e090 0x7fd83c141f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:13.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.595+0000 7fd84282a640 1 --2- 172.21.15.43:0/510121542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd83c0b5c70 0x7fd83c155b00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:13.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.595+0000 7fd84282a640 1 -- 172.21.15.43:0/510121542 >> 172.21.15.43:0/510121542 conn(0x7fd83c0d0560 msgr2=0x7fd83c0d2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:13.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.595+0000 7fd84282a640 1 -- 172.21.15.43:0/510121542 shutdown_connections 2024-04-25T08:54:13.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.595+0000 7fd84282a640 1 -- 172.21.15.43:0/510121542 wait complete. 2024-04-25T08:54:13.596 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T08:54:13.596 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T08:54:13.675 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:54:13.678 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:54:13.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.710+0000 7f9c17de0640 1 Processor -- start 2024-04-25T08:54:13.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.710+0000 7f9c17de0640 1 -- start start 2024-04-25T08:54:13.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.711+0000 7f9c17de0640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9c10140b80 0x7f9c1015e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:13.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.711+0000 7f9c17de0640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9c100add50 0x7f9c10161d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:13.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.711+0000 7f9c17de0640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c10072620 con 0x7f9c100ad8e0 2024-04-25T08:54:13.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.711+0000 7f9c17de0640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c10140f60 con 0x7f9c10140b80 2024-04-25T08:54:13.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.711+0000 7f9c17de0640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c100ae1e0 con 0x7f9c100add50 2024-04-25T08:54:13.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.711+0000 7f9c15b7c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9c10140b80 0x7f9c1015e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:13.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.711+0000 7f9c1537b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f9c100ad8e0 0x7f9c10161640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60806/0 (socket says 172.21.15.43:60806) 2024-04-25T08:54:13.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.711+0000 7f9c1537b640 1 -- 172.21.15.43:0/3228887585 learned_addr learned my addr 172.21.15.43:0/3228887585 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:13.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.712+0000 7f9c1637d640 1 --2- 172.21.15.43:0/3228887585 >> v2:172.21.15.193:3300/0 conn(0x7f9c100add50 0x7f9c10161d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:13.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.712+0000 7f9c15b7c640 1 -- 172.21.15.43:0/3228887585 >> v2:172.21.15.193:3300/0 conn(0x7f9c100add50 msgr2=0x7f9c10161d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:13.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.712+0000 7f9c15b7c640 1 --2- 172.21.15.43:0/3228887585 >> v2:172.21.15.193:3300/0 conn(0x7f9c100add50 0x7f9c10161d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:13.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.712+0000 7f9c15b7c640 1 -- 172.21.15.43:0/3228887585 >> v1:172.21.15.43:6789/0 conn(0x7f9c100ad8e0 legacy=0x7f9c10161640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:13.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.712+0000 7f9c15b7c640 1 -- 172.21.15.43:0/3228887585 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c10071580 con 0x7f9c10140b80 2024-04-25T08:54:13.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.712+0000 7f9c15b7c640 1 --2- 172.21.15.43:0/3228887585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c10140b80 0x7f9c1015e840 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9c1015e050 tx=0x7f9c0c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44fba97e387bc5c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:13.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.713+0000 7f9c14b7a640 1 -- 172.21.15.43:0/3228887585 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c037070 con 0x7f9c10140b80 2024-04-25T08:54:13.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.713+0000 7f9c14b7a640 1 -- 172.21.15.43:0/3228887585 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c0c032440 con 0x7f9c10140b80 2024-04-25T08:54:13.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.713+0000 7f9c14b7a640 1 -- 172.21.15.43:0/3228887585 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c032810 con 0x7f9c10140b80 2024-04-25T08:54:13.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.713+0000 7f9c17de0640 1 -- 172.21.15.43:0/3228887585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c10140b80 msgr2=0x7f9c1015e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:13.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.713+0000 7f9c17de0640 1 --2- 172.21.15.43:0/3228887585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c10140b80 0x7f9c1015e840 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9c1015e050 tx=0x7f9c0c02a1a0 comp rx=0 tx=0).stop 2024-04-25T08:54:13.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.713+0000 7f9c17de0640 1 -- 172.21.15.43:0/3228887585 shutdown_connections 2024-04-25T08:54:13.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.713+0000 7f9c17de0640 1 --2- 172.21.15.43:0/3228887585 >> v2:172.21.15.193:3300/0 conn(0x7f9c100add50 0x7f9c10161d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:13.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.713+0000 7f9c17de0640 1 --2- 172.21.15.43:0/3228887585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c10140b80 0x7f9c1015e840 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:13.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.713+0000 7f9c17de0640 1 -- 172.21.15.43:0/3228887585 >> 172.21.15.43:0/3228887585 conn(0x7f9c100b1480 msgr2=0x7f9c100b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:13.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.713+0000 7f9c17de0640 1 -- 172.21.15.43:0/3228887585 shutdown_connections 2024-04-25T08:54:13.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.713+0000 7f9c17de0640 1 -- 172.21.15.43:0/3228887585 wait complete. 2024-04-25T08:54:13.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.715+0000 7f9c17de0640 1 Processor -- start 2024-04-25T08:54:13.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.715+0000 7f9c17de0640 1 -- start start 2024-04-25T08:54:13.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.715+0000 7f9c17de0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c100ad8e0 0x7f9c1012f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:13.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.715+0000 7f9c17de0640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9c100add50 0x7f9c10142a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:13.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.715+0000 7f9c17de0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c10140b80 0x7f9c1012c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:13.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.715+0000 7f9c17de0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c10071e20 con 0x7f9c100ad8e0 2024-04-25T08:54:13.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.715+0000 7f9c17de0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c10071b40 con 0x7f9c10140b80 2024-04-25T08:54:13.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.715+0000 7f9c17de0640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c10072100 con 0x7f9c100add50 2024-04-25T08:54:13.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.715+0000 7f9c15b7c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c100ad8e0 0x7f9c1012f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:13.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.715+0000 7f9c15b7c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c100ad8e0 0x7f9c1012f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37220/0 (socket says 172.21.15.43:37220) 2024-04-25T08:54:13.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.715+0000 7f9c15b7c640 1 -- 172.21.15.43:0/3712251413 learned_addr learned my addr 172.21.15.43:0/3712251413 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:13.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.715+0000 7f9c1537b640 1 --2- 172.21.15.43:0/3712251413 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9c100add50 0x7f9c10142a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:13.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.716+0000 7f9c1637d640 1 --2- 172.21.15.43:0/3712251413 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c10140b80 0x7f9c1012c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:13.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.716+0000 7f9c15b7c640 1 -- 172.21.15.43:0/3712251413 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9c100add50 msgr2=0x7f9c10142a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:13.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.716+0000 7f9c15b7c640 1 --2- 172.21.15.43:0/3712251413 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9c100add50 0x7f9c10142a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:13.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.716+0000 7f9c15b7c640 1 -- 172.21.15.43:0/3712251413 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c10140b80 msgr2=0x7f9c1012c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:13.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.716+0000 7f9c15b7c640 1 --2- 172.21.15.43:0/3712251413 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c10140b80 0x7f9c1012c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:13.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.716+0000 7f9c15b7c640 1 -- 172.21.15.43:0/3712251413 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c1013cb80 con 0x7f9c100ad8e0 2024-04-25T08:54:13.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.716+0000 7f9c1637d640 1 --2- 172.21.15.43:0/3712251413 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c10140b80 0x7f9c1012c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:13.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.716+0000 7f9c1537b640 1 --2- 172.21.15.43:0/3712251413 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9c100add50 0x7f9c10142a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:13.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.716+0000 7f9c15b7c640 1 --2- 172.21.15.43:0/3712251413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c100ad8e0 0x7f9c1012f1e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c000c00 tx=0x7f9c0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:13.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.716+0000 7f9c067fc640 1 -- 172.21.15.43:0/3712251413 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c037040 con 0x7f9c100ad8e0 2024-04-25T08:54:13.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.716+0000 7f9c067fc640 1 -- 172.21.15.43:0/3712251413 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c0c042070 con 0x7f9c100ad8e0 2024-04-25T08:54:13.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.716+0000 7f9c17de0640 1 -- 172.21.15.43:0/3712251413 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c1013ce60 con 0x7f9c100ad8e0 2024-04-25T08:54:13.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.716+0000 7f9c067fc640 1 -- 172.21.15.43:0/3712251413 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c030bd0 con 0x7f9c100ad8e0 2024-04-25T08:54:13.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.716+0000 7f9c17de0640 1 -- 172.21.15.43:0/3712251413 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c1013d420 con 0x7f9c100ad8e0 2024-04-25T08:54:13.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.717+0000 7f9c17de0640 1 -- 172.21.15.43:0/3712251413 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c10074f60 con 0x7f9c100ad8e0 2024-04-25T08:54:13.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.718+0000 7f9c067fc640 1 -- 172.21.15.43:0/3712251413 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9c0c047460 con 0x7f9c100ad8e0 2024-04-25T08:54:13.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.718+0000 7f9c067fc640 1 --2- 172.21.15.43:0/3712251413 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9bf406b4d0 0x7f9bf406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:13.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.718+0000 7f9c067fc640 1 -- 172.21.15.43:0/3712251413 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9c0c0aade0 con 0x7f9c100ad8e0 2024-04-25T08:54:13.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.718+0000 7f9c1537b640 1 --2- 172.21.15.43:0/3712251413 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9bf406b4d0 0x7f9bf406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:13.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.719+0000 7f9c1537b640 1 --2- 172.21.15.43:0/3712251413 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9bf406b4d0 0x7f9bf406d990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9c1013c6e0 tx=0x7f9c00009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:13.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.722+0000 7f9c067fc640 1 -- 172.21.15.43:0/3712251413 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9c0c032e20 con 0x7f9c100ad8e0 2024-04-25T08:54:13.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:13.941+0000 7f9c17de0640 1 -- 172.21.15.43:0/3712251413 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9c1006c530 con 0x7f9c100ad8e0 2024-04-25T08:54:14.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.601+0000 7f9c067fc640 1 -- 172.21.15.43:0/3712251413 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9c0c079c60 con 0x7f9c100ad8e0 2024-04-25T08:54:14.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.604+0000 7f9c17de0640 1 -- 172.21.15.43:0/3712251413 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9bf406b4d0 msgr2=0x7f9bf406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:14.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.604+0000 7f9c17de0640 1 --2- 172.21.15.43:0/3712251413 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9bf406b4d0 0x7f9bf406d990 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f9c1013c6e0 tx=0x7f9c00009040 comp rx=0 tx=0).stop 2024-04-25T08:54:14.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.604+0000 7f9c17de0640 1 -- 172.21.15.43:0/3712251413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c100ad8e0 msgr2=0x7f9c1012f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:14.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.604+0000 7f9c17de0640 1 --2- 172.21.15.43:0/3712251413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c100ad8e0 0x7f9c1012f1e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c000c00 tx=0x7f9c0c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:54:14.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.604+0000 7f9c17de0640 1 -- 172.21.15.43:0/3712251413 shutdown_connections 2024-04-25T08:54:14.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.604+0000 7f9c17de0640 1 --2- 172.21.15.43:0/3712251413 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9bf406b4d0 0x7f9bf406d990 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:14.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.604+0000 7f9c17de0640 1 --2- 172.21.15.43:0/3712251413 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c10140b80 0x7f9c1012c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:14.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.604+0000 7f9c17de0640 1 --2- 172.21.15.43:0/3712251413 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9c100add50 0x7f9c10142a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:14.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.604+0000 7f9c17de0640 1 --2- 172.21.15.43:0/3712251413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c100ad8e0 0x7f9c1012f1e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:14.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.604+0000 7f9c17de0640 1 -- 172.21.15.43:0/3712251413 >> 172.21.15.43:0/3712251413 conn(0x7f9c100b1480 msgr2=0x7f9c101615d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:14.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.604+0000 7f9c17de0640 1 -- 172.21.15.43:0/3712251413 shutdown_connections 2024-04-25T08:54:14.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.604+0000 7f9c17de0640 1 -- 172.21.15.43:0/3712251413 wait complete. 2024-04-25T08:54:14.607 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T08:54:14.608 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:54:14.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.724+0000 7f3440d5c640 1 Processor -- start 2024-04-25T08:54:14.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.724+0000 7f3440d5c640 1 -- start start 2024-04-25T08:54:14.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.725+0000 7f3440d5c640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f343c0d35c0 0x7f343c0ce260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:14.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.725+0000 7f3440d5c640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f343c0cec30 con 0x7f343c15dd00 2024-04-25T08:54:14.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.725+0000 7f3440d5c640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f343c15e0e0 con 0x7f343c0ce7a0 2024-04-25T08:54:14.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.725+0000 7f3440d5c640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f343c0d24e0 con 0x7f343c0d35c0 2024-04-25T08:54:14.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.725+0000 7f343a59c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f343c15dd00 0x7f343c0d2eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60808/0 (socket says 172.21.15.43:60808) 2024-04-25T08:54:14.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.725+0000 7f343a59c640 1 -- 172.21.15.43:0/4206288051 learned_addr learned my addr 172.21.15.43:0/4206288051 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:14.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.726+0000 7f3439d9b640 1 --2- 172.21.15.43:0/4206288051 >> v2:172.21.15.193:3300/0 conn(0x7f343c0d35c0 0x7f343c0ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:14.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.726+0000 7f343959a640 1 -- 172.21.15.43:0/4206288051 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3403019121 0 0) 0x7f3434001110 con 0x7f343c15dd00 2024-04-25T08:54:14.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.726+0000 7f343959a640 1 -- 172.21.15.43:0/4206288051 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3424003520 con 0x7f343c15dd00 2024-04-25T08:54:14.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.726+0000 7f3439d9b640 1 -- 172.21.15.43:0/4206288051 >> v1:172.21.15.193:6789/0 conn(0x7f343c0ce7a0 legacy=0x7f343c0b6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:14.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.726+0000 7f343959a640 1 -- 172.21.15.43:0/4206288051 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1306748642 0 0) 0x7f3424003520 con 0x7f343c15dd00 2024-04-25T08:54:14.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.726+0000 7f3439d9b640 1 -- 172.21.15.43:0/4206288051 >> v1:172.21.15.43:6789/0 conn(0x7f343c15dd00 legacy=0x7f343c0d2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:14.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.726+0000 7f3439d9b640 1 -- 172.21.15.43:0/4206288051 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f343c0ceeb0 con 0x7f343c0d35c0 2024-04-25T08:54:14.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.727+0000 7f3439d9b640 1 --2- 172.21.15.43:0/4206288051 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f343c0d35c0 0x7f343c0ce260 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f34300097a0 tx=0x7f343002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=149afe4f44f08845 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:14.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.727+0000 7f343959a640 1 -- 172.21.15.43:0/4206288051 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3430037070 con 0x7f343c0d35c0 2024-04-25T08:54:14.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.727+0000 7f343959a640 1 -- 172.21.15.43:0/4206288051 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f343002ac50 con 0x7f343c0d35c0 2024-04-25T08:54:14.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.727+0000 7f343959a640 1 -- 172.21.15.43:0/4206288051 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3434001840 con 0x7f343c0d35c0 2024-04-25T08:54:14.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.728+0000 7f3440d5c640 1 -- 172.21.15.43:0/4206288051 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f343c0d35c0 msgr2=0x7f343c0ce260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:14.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.728+0000 7f3440d5c640 1 --2- 172.21.15.43:0/4206288051 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f343c0d35c0 0x7f343c0ce260 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f34300097a0 tx=0x7f343002a1c0 comp rx=0 tx=0).stop 2024-04-25T08:54:14.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.728+0000 7f3440d5c640 1 -- 172.21.15.43:0/4206288051 shutdown_connections 2024-04-25T08:54:14.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.728+0000 7f3440d5c640 1 --2- 172.21.15.43:0/4206288051 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f343c0d35c0 0x7f343c0ce260 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:14.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.728+0000 7f3440d5c640 1 -- 172.21.15.43:0/4206288051 >> 172.21.15.43:0/4206288051 conn(0x7f343c06cbd0 msgr2=0x7f343c06cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:14.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.728+0000 7f3440d5c640 1 -- 172.21.15.43:0/4206288051 shutdown_connections 2024-04-25T08:54:14.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.728+0000 7f3440d5c640 1 -- 172.21.15.43:0/4206288051 wait complete. 2024-04-25T08:54:14.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.729+0000 7f3440d5c640 1 Processor -- start 2024-04-25T08:54:14.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.729+0000 7f3440d5c640 1 -- start start 2024-04-25T08:54:14.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.730+0000 7f3440d5c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f343c0ce7a0 0x7f343c0c64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:14.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.730+0000 7f3440d5c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f343c0d35c0 0x7f343c1e6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:14.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.730+0000 7f3440d5c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f343c15dd00 0x7f343c1604b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:14.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.730+0000 7f3440d5c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343c0d2170 con 0x7f343c0ce7a0 2024-04-25T08:54:14.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.730+0000 7f3440d5c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343c0d1e90 con 0x7f343c0d35c0 2024-04-25T08:54:14.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.730+0000 7f3440d5c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f343c0d2970 con 0x7f343c15dd00 2024-04-25T08:54:14.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.730+0000 7f343a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f343c0ce7a0 0x7f343c0c64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:14.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.730+0000 7f343a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f343c0ce7a0 0x7f343c0c64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37236/0 (socket says 172.21.15.43:37236) 2024-04-25T08:54:14.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.730+0000 7f343a59c640 1 -- 172.21.15.43:0/921162376 learned_addr learned my addr 172.21.15.43:0/921162376 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:14.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.730+0000 7f343a59c640 1 -- 172.21.15.43:0/921162376 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f343c15dd00 msgr2=0x7f343c1604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:14.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.730+0000 7f343a59c640 1 --2- 172.21.15.43:0/921162376 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f343c15dd00 0x7f343c1604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:14.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.730+0000 7f343a59c640 1 -- 172.21.15.43:0/921162376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f343c0d35c0 msgr2=0x7f343c1e6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:14.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.730+0000 7f343a59c640 1 --2- 172.21.15.43:0/921162376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f343c0d35c0 0x7f343c1e6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:14.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.730+0000 7f343a59c640 1 -- 172.21.15.43:0/921162376 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f343c14b970 con 0x7f343c0ce7a0 2024-04-25T08:54:14.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.730+0000 7f343a59c640 1 --2- 172.21.15.43:0/921162376 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f343c0ce7a0 0x7f343c0c64e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f343400c7b0 tx=0x7f343400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:14.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.731+0000 7f342affd640 1 -- 172.21.15.43:0/921162376 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3434017070 con 0x7f343c0ce7a0 2024-04-25T08:54:14.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.731+0000 7f342affd640 1 -- 172.21.15.43:0/921162376 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3434005ad0 con 0x7f343c0ce7a0 2024-04-25T08:54:14.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.731+0000 7f342affd640 1 -- 172.21.15.43:0/921162376 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3434007070 con 0x7f343c0ce7a0 2024-04-25T08:54:14.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.731+0000 7f3440d5c640 1 -- 172.21.15.43:0/921162376 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f343c14bc50 con 0x7f343c0ce7a0 2024-04-25T08:54:14.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.731+0000 7f3440d5c640 1 -- 172.21.15.43:0/921162376 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f343c14c210 con 0x7f343c0ce7a0 2024-04-25T08:54:14.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.732+0000 7f3440d5c640 1 -- 172.21.15.43:0/921162376 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f343c074ef0 con 0x7f343c0ce7a0 2024-04-25T08:54:14.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.732+0000 7f342affd640 1 -- 172.21.15.43:0/921162376 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f343401a070 con 0x7f343c0ce7a0 2024-04-25T08:54:14.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.733+0000 7f342affd640 1 --2- 172.21.15.43:0/921162376 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f341c06b4d0 0x7f341c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:14.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.733+0000 7f342affd640 1 -- 172.21.15.43:0/921162376 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f34340897b0 con 0x7f343c0ce7a0 2024-04-25T08:54:14.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.733+0000 7f3439d9b640 1 --2- 172.21.15.43:0/921162376 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f341c06b4d0 0x7f341c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:14.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.733+0000 7f3439d9b640 1 --2- 172.21.15.43:0/921162376 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f341c06b4d0 0x7f341c06d990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3430034000 tx=0x7f343002c000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:14.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.737+0000 7f342affd640 1 -- 172.21.15.43:0/921162376 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f343c0cec30 con 0x7f343c0ce7a0 2024-04-25T08:54:14.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.955+0000 7f3440d5c640 1 -- 172.21.15.43:0/921162376 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f343c0b5310 con 0x7f343c0ce7a0 2024-04-25T08:54:14.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.956+0000 7f342affd640 1 -- 172.21.15.43:0/921162376 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4938 (secure 0 0 0) 0x7f3434058590 con 0x7f343c0ce7a0 2024-04-25T08:54:14.946 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:54:14.946 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3616994003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":3616994003},{"type":"v1","addr":"172.21.15.43:6835","nonce":3616994003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/4022901908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4022901908},{"type":"v1","addr":"172.21.15.193:6835","nonce":4022901908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2823450590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2823450590},{"type":"v1","addr":"172.21.15.43:6834","nonce":2823450590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:54:11.945963+0000","modified":"2024-04-25T08:54:13.943002+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.193:6837/1197434598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1197434598},{"type":"v1","addr":"172.21.15.193:6837","nonce":1197434598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:54:14.946 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 6 2024-04-25T08:54:14.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.958+0000 7f3440d5c640 1 -- 172.21.15.43:0/921162376 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f341c06b4d0 msgr2=0x7f341c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:14.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.958+0000 7f3440d5c640 1 --2- 172.21.15.43:0/921162376 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f341c06b4d0 0x7f341c06d990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3430034000 tx=0x7f343002c000 comp rx=0 tx=0).stop 2024-04-25T08:54:14.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.958+0000 7f3440d5c640 1 -- 172.21.15.43:0/921162376 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f343c0ce7a0 msgr2=0x7f343c0c64e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:14.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.958+0000 7f3440d5c640 1 --2- 172.21.15.43:0/921162376 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f343c0ce7a0 0x7f343c0c64e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f343400c7b0 tx=0x7f343400cc70 comp rx=0 tx=0).stop 2024-04-25T08:54:14.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.958+0000 7f3440d5c640 1 -- 172.21.15.43:0/921162376 shutdown_connections 2024-04-25T08:54:14.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.958+0000 7f3440d5c640 1 --2- 172.21.15.43:0/921162376 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f343c15dd00 0x7f343c1604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:14.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.958+0000 7f3440d5c640 1 --2- 172.21.15.43:0/921162376 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f341c06b4d0 0x7f341c06d990 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:14.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.958+0000 7f3440d5c640 1 --2- 172.21.15.43:0/921162376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f343c0d35c0 0x7f343c1e6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:14.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.958+0000 7f3440d5c640 1 --2- 172.21.15.43:0/921162376 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f343c0ce7a0 0x7f343c0c64e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:14.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.958+0000 7f3440d5c640 1 -- 172.21.15.43:0/921162376 >> 172.21.15.43:0/921162376 conn(0x7f343c06cbd0 msgr2=0x7f343c0b5830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:14.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.958+0000 7f3440d5c640 1 -- 172.21.15.43:0/921162376 shutdown_connections 2024-04-25T08:54:14.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:14.958+0000 7f3440d5c640 1 -- 172.21.15.43:0/921162376 wait complete. 2024-04-25T08:54:14.959 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:54:15.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.069+0000 7f6bc115c640 1 Processor -- start 2024-04-25T08:54:15.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.069+0000 7f6bc115c640 1 -- start start 2024-04-25T08:54:15.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.069+0000 7f6bc115c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bbc0c8d00 0x7f6bbc162100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:15.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.069+0000 7f6bc115c640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bbc0d24a0 con 0x7f6bbc0ce660 2024-04-25T08:54:15.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.069+0000 7f6bc115c640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bbc0d2740 con 0x7f6bbc15df50 2024-04-25T08:54:15.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.069+0000 7f6bc115c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc15e330 con 0x7f6bbc0c8d00 2024-04-25T08:54:15.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.070+0000 7f6bbb59e640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f6bbc0ce660 0x7f6bbc0cea70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60816/0 (socket says 172.21.15.43:60816) 2024-04-25T08:54:15.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.070+0000 7f6bbb59e640 1 -- 172.21.15.43:0/1077882447 learned_addr learned my addr 172.21.15.43:0/1077882447 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:15.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.070+0000 7f6bba59c640 1 --2- 172.21.15.43:0/1077882447 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bbc0c8d00 0x7f6bbc162100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:15.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.070+0000 7f6bb9d9b640 1 -- 172.21.15.43:0/1077882447 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1832261325 0 0) 0x7f6bbc0d24a0 con 0x7f6bbc0ce660 2024-04-25T08:54:15.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.070+0000 7f6bb9d9b640 1 -- 172.21.15.43:0/1077882447 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ba8003520 con 0x7f6bbc0ce660 2024-04-25T08:54:15.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.070+0000 7f6bb9d9b640 1 -- 172.21.15.43:0/1077882447 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1887071179 0 0) 0x7f6bbc0d2740 con 0x7f6bbc15df50 2024-04-25T08:54:15.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.070+0000 7f6bb9d9b640 1 -- 172.21.15.43:0/1077882447 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bbc0d24a0 con 0x7f6bbc15df50 2024-04-25T08:54:15.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.070+0000 7f6bba59c640 1 -- 172.21.15.43:0/1077882447 >> v1:172.21.15.193:6789/0 conn(0x7f6bbc15df50 legacy=0x7f6bbc1619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:15.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.070+0000 7f6bba59c640 1 -- 172.21.15.43:0/1077882447 >> v1:172.21.15.43:6789/0 conn(0x7f6bbc0ce660 legacy=0x7f6bbc0cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:15.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.070+0000 7f6bba59c640 1 -- 172.21.15.43:0/1077882447 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bbc0d2a20 con 0x7f6bbc0c8d00 2024-04-25T08:54:15.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.071+0000 7f6bba59c640 1 --2- 172.21.15.43:0/1077882447 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bbc0c8d00 0x7f6bbc162100 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6ba40097d0 tx=0x7f6ba402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2711fed4809b124d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:15.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.071+0000 7f6bb9d9b640 1 -- 172.21.15.43:0/1077882447 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4037070 con 0x7f6bbc0c8d00 2024-04-25T08:54:15.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.071+0000 7f6bb9d9b640 1 -- 172.21.15.43:0/1077882447 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba4032440 con 0x7f6bbc0c8d00 2024-04-25T08:54:15.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.071+0000 7f6bb9d9b640 1 -- 172.21.15.43:0/1077882447 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba40327f0 con 0x7f6bbc0c8d00 2024-04-25T08:54:15.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.071+0000 7f6bc115c640 1 -- 172.21.15.43:0/1077882447 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bbc0c8d00 msgr2=0x7f6bbc162100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:15.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.071+0000 7f6bc115c640 1 --2- 172.21.15.43:0/1077882447 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bbc0c8d00 0x7f6bbc162100 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6ba40097d0 tx=0x7f6ba402a210 comp rx=0 tx=0).stop 2024-04-25T08:54:15.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.071+0000 7f6bc115c640 1 -- 172.21.15.43:0/1077882447 shutdown_connections 2024-04-25T08:54:15.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.071+0000 7f6bc115c640 1 --2- 172.21.15.43:0/1077882447 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bbc0c8d00 0x7f6bbc162100 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:15.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.071+0000 7f6bc115c640 1 -- 172.21.15.43:0/1077882447 >> 172.21.15.43:0/1077882447 conn(0x7f6bbc0d3a30 msgr2=0x7f6bbc0d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:15.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.071+0000 7f6bc115c640 1 -- 172.21.15.43:0/1077882447 shutdown_connections 2024-04-25T08:54:15.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.071+0000 7f6bc115c640 1 -- 172.21.15.43:0/1077882447 wait complete. 2024-04-25T08:54:15.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.073+0000 7f6bc115c640 1 Processor -- start 2024-04-25T08:54:15.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.073+0000 7f6bc115c640 1 -- start start 2024-04-25T08:54:15.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.074+0000 7f6bc115c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6bbc0c8d00 0x7f6bbc160700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:15.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.074+0000 7f6bc115c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bbc0ce660 0x7f6bbc160c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:15.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.074+0000 7f6bc115c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bbc15df50 0x7f6bbc155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:15.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.074+0000 7f6bc115c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc0d2190 con 0x7f6bbc0c8d00 2024-04-25T08:54:15.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.074+0000 7f6bc115c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc0c77d0 con 0x7f6bbc0ce660 2024-04-25T08:54:15.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.074+0000 7f6bc115c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc162640 con 0x7f6bbc15df50 2024-04-25T08:54:15.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.074+0000 7f6bbad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6bbc0c8d00 0x7f6bbc160700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:15.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.074+0000 7f6bbad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6bbc0c8d00 0x7f6bbc160700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37250/0 (socket says 172.21.15.43:37250) 2024-04-25T08:54:15.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.074+0000 7f6bbad9d640 1 -- 172.21.15.43:0/2405555124 learned_addr learned my addr 172.21.15.43:0/2405555124 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:15.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.074+0000 7f6bbb59e640 1 --2- 172.21.15.43:0/2405555124 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bbc15df50 0x7f6bbc155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:15.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.074+0000 7f6bba59c640 1 --2- 172.21.15.43:0/2405555124 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bbc0ce660 0x7f6bbc160c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:15.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.074+0000 7f6bbad9d640 1 -- 172.21.15.43:0/2405555124 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bbc15df50 msgr2=0x7f6bbc155e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:15.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.074+0000 7f6bbad9d640 1 --2- 172.21.15.43:0/2405555124 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bbc15df50 0x7f6bbc155e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:15.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.074+0000 7f6bbad9d640 1 -- 172.21.15.43:0/2405555124 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bbc0ce660 msgr2=0x7f6bbc160c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:15.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.074+0000 7f6bbad9d640 1 --2- 172.21.15.43:0/2405555124 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bbc0ce660 0x7f6bbc160c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:15.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.074+0000 7f6bbad9d640 1 -- 172.21.15.43:0/2405555124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bbc15f040 con 0x7f6bbc0c8d00 2024-04-25T08:54:15.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.074+0000 7f6bba59c640 1 --2- 172.21.15.43:0/2405555124 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bbc0ce660 0x7f6bbc160c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:15.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.074+0000 7f6bbad9d640 1 --2- 172.21.15.43:0/2405555124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6bbc0c8d00 0x7f6bbc160700 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6bb400c430 tx=0x7f6bb400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:15.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.074+0000 7f6bb37fe640 1 -- 172.21.15.43:0/2405555124 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb4017070 con 0x7f6bbc0c8d00 2024-04-25T08:54:15.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.074+0000 7f6bb37fe640 1 -- 172.21.15.43:0/2405555124 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bb40052f0 con 0x7f6bbc0c8d00 2024-04-25T08:54:15.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.074+0000 7f6bc115c640 1 -- 172.21.15.43:0/2405555124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bbc15f320 con 0x7f6bbc0c8d00 2024-04-25T08:54:15.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.074+0000 7f6bb37fe640 1 -- 172.21.15.43:0/2405555124 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb4003830 con 0x7f6bbc0c8d00 2024-04-25T08:54:15.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.075+0000 7f6bc115c640 1 -- 172.21.15.43:0/2405555124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bbc15f8e0 con 0x7f6bbc0c8d00 2024-04-25T08:54:15.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.076+0000 7f6bc115c640 1 -- 172.21.15.43:0/2405555124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bbc0680f0 con 0x7f6bbc0c8d00 2024-04-25T08:54:15.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.076+0000 7f6bb37fe640 1 -- 172.21.15.43:0/2405555124 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6bb401a070 con 0x7f6bbc0c8d00 2024-04-25T08:54:15.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.076+0000 7f6bb37fe640 1 --2- 172.21.15.43:0/2405555124 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6b9c06b500 0x7f6b9c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:15.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.077+0000 7f6bb37fe640 1 -- 172.21.15.43:0/2405555124 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6bb4089570 con 0x7f6bbc0c8d00 2024-04-25T08:54:15.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.077+0000 7f6bba59c640 1 --2- 172.21.15.43:0/2405555124 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6b9c06b500 0x7f6b9c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:15.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.077+0000 7f6bba59c640 1 --2- 172.21.15.43:0/2405555124 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6b9c06b500 0x7f6b9c06d9c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4034000 tx=0x7f6ba402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:15.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.081+0000 7f6bb37fe640 1 -- 172.21.15.43:0/2405555124 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6bb40583f0 con 0x7f6bbc0c8d00 2024-04-25T08:54:15.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.261+0000 7f6bc115c640 1 -- 172.21.15.43:0/2405555124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6bbc1604e0 con 0x7f6bbc0c8d00 2024-04-25T08:54:15.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.263+0000 7f6bb37fe640 1 -- 172.21.15.43:0/2405555124 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16569 (secure 0 0 0) 0x7f6bb4057cf0 con 0x7f6bbc0c8d00 2024-04-25T08:54:15.253 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:54:15.253 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":20,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:54:13.593188+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:54:09.757978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:54:10.926008+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:54:15.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.265+0000 7f6bc115c640 1 -- 172.21.15.43:0/2405555124 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6b9c06b500 msgr2=0x7f6b9c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:15.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.265+0000 7f6bc115c640 1 --2- 172.21.15.43:0/2405555124 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6b9c06b500 0x7f6b9c06d9c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4034000 tx=0x7f6ba402e000 comp rx=0 tx=0).stop 2024-04-25T08:54:15.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.265+0000 7f6bc115c640 1 -- 172.21.15.43:0/2405555124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6bbc0c8d00 msgr2=0x7f6bbc160700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:15.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.265+0000 7f6bc115c640 1 --2- 172.21.15.43:0/2405555124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6bbc0c8d00 0x7f6bbc160700 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6bb400c430 tx=0x7f6bb400c8f0 comp rx=0 tx=0).stop 2024-04-25T08:54:15.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.265+0000 7f6bc115c640 1 -- 172.21.15.43:0/2405555124 shutdown_connections 2024-04-25T08:54:15.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.265+0000 7f6bc115c640 1 --2- 172.21.15.43:0/2405555124 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bbc15df50 0x7f6bbc155e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:15.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.265+0000 7f6bc115c640 1 --2- 172.21.15.43:0/2405555124 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6b9c06b500 0x7f6b9c06d9c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:15.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.265+0000 7f6bc115c640 1 --2- 172.21.15.43:0/2405555124 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bbc0ce660 0x7f6bbc160c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:15.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.265+0000 7f6bc115c640 1 --2- 172.21.15.43:0/2405555124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6bbc0c8d00 0x7f6bbc160700 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:15.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.265+0000 7f6bc115c640 1 -- 172.21.15.43:0/2405555124 >> 172.21.15.43:0/2405555124 conn(0x7f6bbc0d3a30 msgr2=0x7f6bbc06c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:15.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.265+0000 7f6bc115c640 1 -- 172.21.15.43:0/2405555124 shutdown_connections 2024-04-25T08:54:15.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.265+0000 7f6bc115c640 1 -- 172.21.15.43:0/2405555124 wait complete. 2024-04-25T08:54:15.266 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T08:54:15.266 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:54:15.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.377+0000 7fc518cbd640 1 Processor -- start 2024-04-25T08:54:15.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.377+0000 7fc518cbd640 1 -- start start 2024-04-25T08:54:15.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.377+0000 7fc518cbd640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc514161390 0x7fc5140b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:15.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.377+0000 7fc518cbd640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5140b4c30 con 0x7fc5140d02a0 2024-04-25T08:54:15.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.377+0000 7fc518cbd640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5140cf2c0 con 0x7fc5140cecf0 2024-04-25T08:54:15.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.377+0000 7fc518cbd640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5140d1910 con 0x7fc514161390 2024-04-25T08:54:15.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.378+0000 7fc50bfff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc5140d02a0 0x7fc5140ce5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60828/0 (socket says 172.21.15.43:60828) 2024-04-25T08:54:15.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.378+0000 7fc50bfff640 1 -- 172.21.15.43:0/3995181178 learned_addr learned my addr 172.21.15.43:0/3995181178 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:15.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.378+0000 7fc51259c640 1 --2- 172.21.15.43:0/3995181178 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc514161390 0x7fc5140b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:15.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.378+0000 7fc511d9b640 1 -- 172.21.15.43:0/3995181178 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2287950711 0 0) 0x7fc4fc001110 con 0x7fc5140d02a0 2024-04-25T08:54:15.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.378+0000 7fc511d9b640 1 -- 172.21.15.43:0/3995181178 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc500003520 con 0x7fc5140d02a0 2024-04-25T08:54:15.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.378+0000 7fc511d9b640 1 -- 172.21.15.43:0/3995181178 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1379223573 0 0) 0x7fc5140cf2c0 con 0x7fc5140cecf0 2024-04-25T08:54:15.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.378+0000 7fc511d9b640 1 -- 172.21.15.43:0/3995181178 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4fc001110 con 0x7fc5140cecf0 2024-04-25T08:54:15.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.378+0000 7fc511d9b640 1 -- 172.21.15.43:0/3995181178 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2841001790 0 0) 0x7fc500003520 con 0x7fc5140d02a0 2024-04-25T08:54:15.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.378+0000 7fc51259c640 1 -- 172.21.15.43:0/3995181178 >> v1:172.21.15.193:6789/0 conn(0x7fc5140cecf0 legacy=0x7fc5140d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:15.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.378+0000 7fc51259c640 1 -- 172.21.15.43:0/3995181178 >> v1:172.21.15.43:6789/0 conn(0x7fc5140d02a0 legacy=0x7fc5140ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:15.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.378+0000 7fc51259c640 1 -- 172.21.15.43:0/3995181178 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5140cf5a0 con 0x7fc514161390 2024-04-25T08:54:15.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.379+0000 7fc51259c640 1 --2- 172.21.15.43:0/3995181178 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc514161390 0x7fc5140b6820 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc5040097d0 tx=0x7fc50402a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c382ec9102e9edf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:15.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.379+0000 7fc511d9b640 1 -- 172.21.15.43:0/3995181178 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc504037070 con 0x7fc514161390 2024-04-25T08:54:15.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.379+0000 7fc511d9b640 1 -- 172.21.15.43:0/3995181178 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc50402ac30 con 0x7fc514161390 2024-04-25T08:54:15.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.379+0000 7fc511d9b640 1 -- 172.21.15.43:0/3995181178 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc001840 con 0x7fc514161390 2024-04-25T08:54:15.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.380+0000 7fc518cbd640 1 -- 172.21.15.43:0/3995181178 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc514161390 msgr2=0x7fc5140b6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:15.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.380+0000 7fc518cbd640 1 --2- 172.21.15.43:0/3995181178 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc514161390 0x7fc5140b6820 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc5040097d0 tx=0x7fc50402a1a0 comp rx=0 tx=0).stop 2024-04-25T08:54:15.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.380+0000 7fc518cbd640 1 -- 172.21.15.43:0/3995181178 shutdown_connections 2024-04-25T08:54:15.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.380+0000 7fc518cbd640 1 --2- 172.21.15.43:0/3995181178 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc514161390 0x7fc5140b6820 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:15.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.380+0000 7fc518cbd640 1 -- 172.21.15.43:0/3995181178 >> 172.21.15.43:0/3995181178 conn(0x7fc51409cd20 msgr2=0x7fc5140d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:15.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.380+0000 7fc518cbd640 1 -- 172.21.15.43:0/3995181178 shutdown_connections 2024-04-25T08:54:15.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.380+0000 7fc518cbd640 1 -- 172.21.15.43:0/3995181178 wait complete. 2024-04-25T08:54:15.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.381+0000 7fc518cbd640 1 Processor -- start 2024-04-25T08:54:15.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.381+0000 7fc518cbd640 1 -- start start 2024-04-25T08:54:15.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.382+0000 7fc518cbd640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc5140cecf0 0x7fc5140cca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:15.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.382+0000 7fc518cbd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5140d02a0 0x7fc5140c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:15.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.382+0000 7fc518cbd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc514161390 0x7fc5140b8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:15.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.382+0000 7fc518cbd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc514161770 con 0x7fc5140d02a0 2024-04-25T08:54:15.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.382+0000 7fc518cbd640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5140d15a0 con 0x7fc514161390 2024-04-25T08:54:15.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.382+0000 7fc518cbd640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5140d0720 con 0x7fc5140cecf0 2024-04-25T08:54:15.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.382+0000 7fc50bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5140d02a0 0x7fc5140c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:15.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.382+0000 7fc50bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5140d02a0 0x7fc5140c7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37266/0 (socket says 172.21.15.43:37266) 2024-04-25T08:54:15.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.382+0000 7fc50bfff640 1 -- 172.21.15.43:0/3803084130 learned_addr learned my addr 172.21.15.43:0/3803084130 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:15.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.382+0000 7fc51259c640 1 --2- 172.21.15.43:0/3803084130 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc5140cecf0 0x7fc5140cca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:15.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.382+0000 7fc50bfff640 1 -- 172.21.15.43:0/3803084130 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc5140cecf0 msgr2=0x7fc5140cca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:15.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.382+0000 7fc512d9d640 1 --2- 172.21.15.43:0/3803084130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc514161390 0x7fc5140b8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:15.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.382+0000 7fc50bfff640 1 --2- 172.21.15.43:0/3803084130 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc5140cecf0 0x7fc5140cca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:15.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.382+0000 7fc50bfff640 1 -- 172.21.15.43:0/3803084130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc514161390 msgr2=0x7fc5140b8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:15.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.382+0000 7fc50bfff640 1 --2- 172.21.15.43:0/3803084130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc514161390 0x7fc5140b8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:15.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.382+0000 7fc50bfff640 1 -- 172.21.15.43:0/3803084130 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5140cd320 con 0x7fc5140d02a0 2024-04-25T08:54:15.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.382+0000 7fc51259c640 1 --2- 172.21.15.43:0/3803084130 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc5140cecf0 0x7fc5140cca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:15.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.382+0000 7fc512d9d640 1 --2- 172.21.15.43:0/3803084130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc514161390 0x7fc5140b8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:15.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.382+0000 7fc50bfff640 1 --2- 172.21.15.43:0/3803084130 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5140d02a0 0x7fc5140c7e80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc00cac0 tx=0x7fc4fc00cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:15.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.383+0000 7fc50affd640 1 -- 172.21.15.43:0/3803084130 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc017070 con 0x7fc5140d02a0 2024-04-25T08:54:15.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.383+0000 7fc50affd640 1 -- 172.21.15.43:0/3803084130 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4fc005d00 con 0x7fc5140d02a0 2024-04-25T08:54:15.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.383+0000 7fc518cbd640 1 -- 172.21.15.43:0/3803084130 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5140cd600 con 0x7fc5140d02a0 2024-04-25T08:54:15.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.383+0000 7fc50affd640 1 -- 172.21.15.43:0/3803084130 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4fc0072b0 con 0x7fc5140d02a0 2024-04-25T08:54:15.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.383+0000 7fc518cbd640 1 -- 172.21.15.43:0/3803084130 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5140ccfa0 con 0x7fc5140d02a0 2024-04-25T08:54:15.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.384+0000 7fc518cbd640 1 -- 172.21.15.43:0/3803084130 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5140680f0 con 0x7fc5140d02a0 2024-04-25T08:54:15.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.384+0000 7fc50affd640 1 -- 172.21.15.43:0/3803084130 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc4fc01a070 con 0x7fc5140d02a0 2024-04-25T08:54:15.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.384+0000 7fc50affd640 1 --2- 172.21.15.43:0/3803084130 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc4f406b4d0 0x7fc4f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:15.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.385+0000 7fc50affd640 1 -- 172.21.15.43:0/3803084130 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc4fc0898e0 con 0x7fc5140d02a0 2024-04-25T08:54:15.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.385+0000 7fc51259c640 1 --2- 172.21.15.43:0/3803084130 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc4f406b4d0 0x7fc4f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:15.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.385+0000 7fc51259c640 1 --2- 172.21.15.43:0/3803084130 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc4f406b4d0 0x7fc4f406d990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc504034000 tx=0x7fc50402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:15.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.389+0000 7fc50affd640 1 -- 172.21.15.43:0/3803084130 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc5140b4c30 con 0x7fc5140d02a0 2024-04-25T08:54:15.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.564+0000 7fc518cbd640 1 -- 172.21.15.43:0/3803084130 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc514074ef0 con 0x7fc4f406b4d0 2024-04-25T08:54:15.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.571+0000 7fc50affd640 1 -- 172.21.15.43:0/3803084130 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+237079 (secure 0 0 0) 0x7fc514074ef0 con 0x7fc4f406b4d0 2024-04-25T08:54:15.563 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:54:15.565 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-25T08:54:15.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.573+0000 7fc518cbd640 1 -- 172.21.15.43:0/3803084130 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc4f406b4d0 msgr2=0x7fc4f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:15.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.573+0000 7fc518cbd640 1 --2- 172.21.15.43:0/3803084130 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc4f406b4d0 0x7fc4f406d990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc504034000 tx=0x7fc50402e000 comp rx=0 tx=0).stop 2024-04-25T08:54:15.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.573+0000 7fc518cbd640 1 -- 172.21.15.43:0/3803084130 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5140d02a0 msgr2=0x7fc5140c7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:15.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.573+0000 7fc518cbd640 1 --2- 172.21.15.43:0/3803084130 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5140d02a0 0x7fc5140c7e80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc4fc00cac0 tx=0x7fc4fc00cf80 comp rx=0 tx=0).stop 2024-04-25T08:54:15.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.573+0000 7fc518cbd640 1 -- 172.21.15.43:0/3803084130 shutdown_connections 2024-04-25T08:54:15.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.573+0000 7fc518cbd640 1 --2- 172.21.15.43:0/3803084130 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc4f406b4d0 0x7fc4f406d990 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:15.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.573+0000 7fc518cbd640 1 --2- 172.21.15.43:0/3803084130 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc514161390 0x7fc5140b8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:15.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.573+0000 7fc518cbd640 1 --2- 172.21.15.43:0/3803084130 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5140d02a0 0x7fc5140c7e80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:15.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.573+0000 7fc518cbd640 1 --2- 172.21.15.43:0/3803084130 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc5140cecf0 0x7fc5140cca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:15.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.573+0000 7fc518cbd640 1 -- 172.21.15.43:0/3803084130 >> 172.21.15.43:0/3803084130 conn(0x7fc51409cd20 msgr2=0x7fc5140b6730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:15.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.573+0000 7fc518cbd640 1 -- 172.21.15.43:0/3803084130 shutdown_connections 2024-04-25T08:54:15.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.573+0000 7fc518cbd640 1 -- 172.21.15.43:0/3803084130 wait complete. 2024-04-25T08:54:15.574 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-04-25T08:54:13.695352+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":235188,"kb_used_data":2376,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739532,"statfs":{"total":773094113280,"available":772853280768,"internally_reserved":0,"allocated":2433024,"data_stored":1546872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000712"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:10.561422+0000","last_change":"2024-04-25T08:54:10.561422+0000","last_active":"2024-04-25T08:54:10.561422+0000","last_peered":"2024-04-25T08:54:10.561422+0000","last_clean":"2024-04-25T08:54:10.561422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:10.561422+0000","last_undegraded":"2024-04-25T08:54:10.561422+0000","last_fullsized":"2024-04-25T08:54:10.561422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:10.561422+0000","last_change":"2024-04-25T08:54:10.561422+0000","last_active":"2024-04-25T08:54:10.561422+0000","last_peered":"2024-04-25T08:54:10.561422+0000","last_clean":"2024-04-25T08:54:10.561422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:10.561422+0000","last_undegraded":"2024-04-25T08:54:10.561422+0000","last_fullsized":"2024-04-25T08:54:10.561422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:10.561422+0000","last_change":"2024-04-25T08:54:10.561422+0000","last_active":"2024-04-25T08:54:10.561422+0000","last_peered":"2024-04-25T08:54:10.561422+0000","last_clean":"2024-04-25T08:54:10.561422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:10.561422+0000","last_undegraded":"2024-04-25T08:54:10.561422+0000","last_fullsized":"2024-04-25T08:54:10.561422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:10.561422+0000","last_change":"2024-04-25T08:54:10.561422+0000","last_active":"2024-04-25T08:54:10.561422+0000","last_peered":"2024-04-25T08:54:10.561422+0000","last_clean":"2024-04-25T08:54:10.561422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:10.561422+0000","last_undegraded":"2024-04-25T08:54:10.561422+0000","last_fullsized":"2024-04-25T08:54:10.561422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:10.561422+0000","last_change":"2024-04-25T08:54:10.561422+0000","last_active":"2024-04-25T08:54:10.561422+0000","last_peered":"2024-04-25T08:54:10.561422+0000","last_clean":"2024-04-25T08:54:10.561422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:10.561422+0000","last_undegraded":"2024-04-25T08:54:10.561422+0000","last_fullsized":"2024-04-25T08:54:10.561422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:10.561422+0000","last_change":"2024-04-25T08:54:10.561422+0000","last_active":"2024-04-25T08:54:10.561422+0000","last_peered":"2024-04-25T08:54:10.561422+0000","last_clean":"2024-04-25T08:54:10.561422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:10.561422+0000","last_undegraded":"2024-04-25T08:54:10.561422+0000","last_fullsized":"2024-04-25T08:54:10.561422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:10.561422+0000","last_change":"2024-04-25T08:54:10.561422+0000","last_active":"2024-04-25T08:54:10.561422+0000","last_peered":"2024-04-25T08:54:10.561422+0000","last_clean":"2024-04-25T08:54:10.561422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:10.561422+0000","last_undegraded":"2024-04-25T08:54:10.561422+0000","last_fullsized":"2024-04-25T08:54:10.561422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:10.561422+0000","last_change":"2024-04-25T08:54:10.561422+0000","last_active":"2024-04-25T08:54:10.561422+0000","last_peered":"2024-04-25T08:54:10.561422+0000","last_clean":"2024-04-25T08:54:10.561422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:10.561422+0000","last_undegraded":"2024-04-25T08:54:10.561422+0000","last_fullsized":"2024-04-25T08:54:10.561422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:10.561422+0000","last_change":"2024-04-25T08:54:10.561422+0000","last_active":"2024-04-25T08:54:10.561422+0000","last_peered":"2024-04-25T08:54:10.561422+0000","last_clean":"2024-04-25T08:54:10.561422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:10.561422+0000","last_undegraded":"2024-04-25T08:54:10.561422+0000","last_fullsized":"2024-04-25T08:54:10.561422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:10.561422+0000","last_change":"2024-04-25T08:54:10.561422+0000","last_active":"2024-04-25T08:54:10.561422+0000","last_peered":"2024-04-25T08:54:10.561422+0000","last_clean":"2024-04-25T08:54:10.561422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:10.561422+0000","last_undegraded":"2024-04-25T08:54:10.561422+0000","last_fullsized":"2024-04-25T08:54:10.561422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:10.561422+0000","last_change":"2024-04-25T08:54:10.561422+0000","last_active":"2024-04-25T08:54:10.561422+0000","last_peered":"2024-04-25T08:54:10.561422+0000","last_clean":"2024-04-25T08:54:10.561422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:10.561422+0000","last_undegraded":"2024-04-25T08:54:10.561422+0000","last_fullsized":"2024-04-25T08:54:10.561422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:10.561422+0000","last_change":"2024-04-25T08:54:10.561422+0000","last_active":"2024-04-25T08:54:10.561422+0000","last_peered":"2024-04-25T08:54:10.561422+0000","last_clean":"2024-04-25T08:54:10.561422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:10.561422+0000","last_undegraded":"2024-04-25T08:54:10.561422+0000","last_fullsized":"2024-04-25T08:54:10.561422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:10.561422+0000","last_change":"2024-04-25T08:54:10.561422+0000","last_active":"2024-04-25T08:54:10.561422+0000","last_peered":"2024-04-25T08:54:10.561422+0000","last_clean":"2024-04-25T08:54:10.561422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:10.561422+0000","last_undegraded":"2024-04-25T08:54:10.561422+0000","last_fullsized":"2024-04-25T08:54:10.561422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:10.561422+0000","last_change":"2024-04-25T08:54:10.561422+0000","last_active":"2024-04-25T08:54:10.561422+0000","last_peered":"2024-04-25T08:54:10.561422+0000","last_clean":"2024-04-25T08:54:10.561422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:10.561422+0000","last_undegraded":"2024-04-25T08:54:10.561422+0000","last_fullsized":"2024-04-25T08:54:10.561422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:10.561422+0000","last_change":"2024-04-25T08:54:10.561422+0000","last_active":"2024-04-25T08:54:10.561422+0000","last_peered":"2024-04-25T08:54:10.561422+0000","last_clean":"2024-04-25T08:54:10.561422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:10.561422+0000","last_undegraded":"2024-04-25T08:54:10.561422+0000","last_fullsized":"2024-04-25T08:54:10.561422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:10.561422+0000","last_change":"2024-04-25T08:54:10.561422+0000","last_active":"2024-04-25T08:54:10.561422+0000","last_peered":"2024-04-25T08:54:10.561422+0000","last_clean":"2024-04-25T08:54:10.561422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:10.561422+0000","last_undegraded":"2024-04-25T08:54:10.561422+0000","last_fullsized":"2024-04-25T08:54:10.561422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":162,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-25T08:54:03.116912+0000","last_change":"2024-04-25T08:54:00.818044+0000","last_active":"2024-04-25T08:54:03.116912+0000","last_peered":"2024-04-25T08:54:03.116912+0000","last_clean":"2024-04-25T08:54:03.116912+0000","last_became_active":"2024-04-25T08:54:00.817098+0000","last_became_peered":"2024-04-25T08:54:00.817098+0000","last_unstale":"2024-04-25T08:54:03.116912+0000","last_undegraded":"2024-04-25T08:54:03.116912+0000","last_fullsized":"2024-04-25T08:54:03.116912+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_clean_scrub_stamp":"2024-04-25T08:53:59.739677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:48.129513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:10.561422+0000","last_change":"2024-04-25T08:54:10.561422+0000","last_active":"2024-04-25T08:54:10.561422+0000","last_peered":"2024-04-25T08:54:10.561422+0000","last_clean":"2024-04-25T08:54:10.561422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:10.561422+0000","last_undegraded":"2024-04-25T08:54:10.561422+0000","last_fullsized":"2024-04-25T08:54:10.561422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:10.561422+0000","last_change":"2024-04-25T08:54:10.561422+0000","last_active":"2024-04-25T08:54:10.561422+0000","last_peered":"2024-04-25T08:54:10.561422+0000","last_clean":"2024-04-25T08:54:10.561422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:10.561422+0000","last_undegraded":"2024-04-25T08:54:10.561422+0000","last_fullsized":"2024-04-25T08:54:10.561422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:10.561422+0000","last_change":"2024-04-25T08:54:10.561422+0000","last_active":"2024-04-25T08:54:10.561422+0000","last_peered":"2024-04-25T08:54:10.561422+0000","last_clean":"2024-04-25T08:54:10.561422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:10.561422+0000","last_undegraded":"2024-04-25T08:54:10.561422+0000","last_fullsized":"2024-04-25T08:54:10.561422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:10.561422+0000","last_change":"2024-04-25T08:54:10.561422+0000","last_active":"2024-04-25T08:54:10.561422+0000","last_peered":"2024-04-25T08:54:10.561422+0000","last_clean":"2024-04-25T08:54:10.561422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:10.561422+0000","last_undegraded":"2024-04-25T08:54:10.561422+0000","last_fullsized":"2024-04-25T08:54:10.561422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:10.561422+0000","last_change":"2024-04-25T08:54:10.561422+0000","last_active":"2024-04-25T08:54:10.561422+0000","last_peered":"2024-04-25T08:54:10.561422+0000","last_clean":"2024-04-25T08:54:10.561422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:10.561422+0000","last_undegraded":"2024-04-25T08:54:10.561422+0000","last_fullsized":"2024-04-25T08:54:10.561422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:10.561422+0000","last_change":"2024-04-25T08:54:10.561422+0000","last_active":"2024-04-25T08:54:10.561422+0000","last_peered":"2024-04-25T08:54:10.561422+0000","last_clean":"2024-04-25T08:54:10.561422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:10.561422+0000","last_undegraded":"2024-04-25T08:54:10.561422+0000","last_fullsized":"2024-04-25T08:54:10.561422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:10.561422+0000","last_change":"2024-04-25T08:54:10.561422+0000","last_active":"2024-04-25T08:54:10.561422+0000","last_peered":"2024-04-25T08:54:10.561422+0000","last_clean":"2024-04-25T08:54:10.561422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:10.561422+0000","last_undegraded":"2024-04-25T08:54:10.561422+0000","last_fullsized":"2024-04-25T08:54:10.561422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:10.561422+0000","last_change":"2024-04-25T08:54:10.561422+0000","last_active":"2024-04-25T08:54:10.561422+0000","last_peered":"2024-04-25T08:54:10.561422+0000","last_clean":"2024-04-25T08:54:10.561422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:10.561422+0000","last_undegraded":"2024-04-25T08:54:10.561422+0000","last_fullsized":"2024-04-25T08:54:10.561422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:10.561422+0000","last_change":"2024-04-25T08:54:10.561422+0000","last_active":"2024-04-25T08:54:10.561422+0000","last_peered":"2024-04-25T08:54:10.561422+0000","last_clean":"2024-04-25T08:54:10.561422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:10.561422+0000","last_undegraded":"2024-04-25T08:54:10.561422+0000","last_fullsized":"2024-04-25T08:54:10.561422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:10.561422+0000","last_change":"2024-04-25T08:54:10.561422+0000","last_active":"2024-04-25T08:54:10.561422+0000","last_peered":"2024-04-25T08:54:10.561422+0000","last_clean":"2024-04-25T08:54:10.561422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:10.561422+0000","last_undegraded":"2024-04-25T08:54:10.561422+0000","last_fullsized":"2024-04-25T08:54:10.561422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:10.561422+0000","last_change":"2024-04-25T08:54:10.561422+0000","last_active":"2024-04-25T08:54:10.561422+0000","last_peered":"2024-04-25T08:54:10.561422+0000","last_clean":"2024-04-25T08:54:10.561422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:10.561422+0000","last_undegraded":"2024-04-25T08:54:10.561422+0000","last_fullsized":"2024-04-25T08:54:10.561422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:10.561422+0000","last_change":"2024-04-25T08:54:10.561422+0000","last_active":"2024-04-25T08:54:10.561422+0000","last_peered":"2024-04-25T08:54:10.561422+0000","last_clean":"2024-04-25T08:54:10.561422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:10.561422+0000","last_undegraded":"2024-04-25T08:54:10.561422+0000","last_fullsized":"2024-04-25T08:54:10.561422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:10.561422+0000","last_change":"2024-04-25T08:54:10.561422+0000","last_active":"2024-04-25T08:54:10.561422+0000","last_peered":"2024-04-25T08:54:10.561422+0000","last_clean":"2024-04-25T08:54:10.561422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:10.561422+0000","last_undegraded":"2024-04-25T08:54:10.561422+0000","last_fullsized":"2024-04-25T08:54:10.561422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:10.561422+0000","last_change":"2024-04-25T08:54:10.561422+0000","last_active":"2024-04-25T08:54:10.561422+0000","last_peered":"2024-04-25T08:54:10.561422+0000","last_clean":"2024-04-25T08:54:10.561422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:10.561422+0000","last_undegraded":"2024-04-25T08:54:10.561422+0000","last_fullsized":"2024-04-25T08:54:10.561422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:10.561422+0000","last_change":"2024-04-25T08:54:10.561422+0000","last_active":"2024-04-25T08:54:10.561422+0000","last_peered":"2024-04-25T08:54:10.561422+0000","last_clean":"2024-04-25T08:54:10.561422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:10.561422+0000","last_undegraded":"2024-04-25T08:54:10.561422+0000","last_fullsized":"2024-04-25T08:54:10.561422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:11.566270+0000","last_change":"2024-04-25T08:54:11.566270+0000","last_active":"2024-04-25T08:54:11.566270+0000","last_peered":"2024-04-25T08:54:11.566270+0000","last_clean":"2024-04-25T08:54:11.566270+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:11.566270+0000","last_undegraded":"2024-04-25T08:54:11.566270+0000","last_fullsized":"2024-04-25T08:54:11.566270+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:54:10.561422+0000","last_change":"2024-04-25T08:54:10.561422+0000","last_active":"2024-04-25T08:54:10.561422+0000","last_peered":"2024-04-25T08:54:10.561422+0000","last_clean":"2024-04-25T08:54:10.561422+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:54:10.561422+0000","last_undegraded":"2024-04-25T08:54:10.561422+0000","last_fullsized":"2024-04-25T08:54:10.561422+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27172,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344668,"statfs":{"total":96636764160,"available":96608940032,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31332,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340508,"statfs":{"total":96636764160,"available":96604680192,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31332,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340508,"statfs":{"total":96636764160,"available":96604680192,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31336,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340504,"statfs":{"total":96636764160,"available":96604676096,"internally_reserved":0,"allocated":155648,"data_stored":45767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31912,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339928,"statfs":{"total":96636764160,"available":96604086272,"internally_reserved":0,"allocated":749568,"data_stored":636135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:54:15.581 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:54:15.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.726+0000 7f7882012640 1 Processor -- start 2024-04-25T08:54:15.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.726+0000 7f7882012640 1 -- start start 2024-04-25T08:54:15.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.727+0000 7f7882012640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f787c161390 0x7f787c0b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:15.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.727+0000 7f7882012640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f787c0b4c30 con 0x7f787c0cecf0 2024-04-25T08:54:15.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.727+0000 7f7882012640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f787c0cf2c0 con 0x7f787c0d02a0 2024-04-25T08:54:15.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.727+0000 7f7882012640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f787c0d1910 con 0x7f787c161390 2024-04-25T08:54:15.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.727+0000 7f787bfff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f787c0cecf0 0x7f787c0d1da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60840/0 (socket says 172.21.15.43:60840) 2024-04-25T08:54:15.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.727+0000 7f787bfff640 1 -- 172.21.15.43:0/3276492365 learned_addr learned my addr 172.21.15.43:0/3276492365 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:15.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.727+0000 7f787b7fe640 1 --2- 172.21.15.43:0/3276492365 >> v2:172.21.15.193:3300/0 conn(0x7f787c161390 0x7f787c0b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:15.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.727+0000 7f787a7fc640 1 -- 172.21.15.43:0/3276492365 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3540721498 0 0) 0x7f7874001490 con 0x7f787c0cecf0 2024-04-25T08:54:15.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.727+0000 7f787a7fc640 1 -- 172.21.15.43:0/3276492365 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7864003520 con 0x7f787c0cecf0 2024-04-25T08:54:15.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.727+0000 7f787a7fc640 1 -- 172.21.15.43:0/3276492365 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3109409553 0 0) 0x7f787c0cf2c0 con 0x7f787c0d02a0 2024-04-25T08:54:15.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.727+0000 7f787a7fc640 1 -- 172.21.15.43:0/3276492365 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7874001490 con 0x7f787c0d02a0 2024-04-25T08:54:15.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.727+0000 7f787a7fc640 1 -- 172.21.15.43:0/3276492365 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2860426611 0 0) 0x7f7864003520 con 0x7f787c0cecf0 2024-04-25T08:54:15.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.727+0000 7f787a7fc640 1 -- 172.21.15.43:0/3276492365 >> v2:172.21.15.193:3300/0 conn(0x7f787c161390 msgr2=0x7f787c0b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:15.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.727+0000 7f787a7fc640 1 --2- 172.21.15.43:0/3276492365 >> v2:172.21.15.193:3300/0 conn(0x7f787c161390 0x7f787c0b6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:15.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.727+0000 7f787a7fc640 1 -- 172.21.15.43:0/3276492365 >> v1:172.21.15.193:6789/0 conn(0x7f787c0d02a0 legacy=0x7f787c0ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:15.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.727+0000 7f787a7fc640 1 -- 172.21.15.43:0/3276492365 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f787c0cf5a0 con 0x7f787c0cecf0 2024-04-25T08:54:15.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.727+0000 7f787a7fc640 1 -- 172.21.15.43:0/3276492365 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f7874001bc0 con 0x7f787c0cecf0 2024-04-25T08:54:15.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.727+0000 7f787a7fc640 1 -- 172.21.15.43:0/3276492365 >> v1:172.21.15.43:6789/0 conn(0x7f787c0cecf0 legacy=0x7f787c0d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:15.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.727+0000 7f787a7fc640 1 --2- 172.21.15.43:0/3276492365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7864003b90 0x7f7864003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:15.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.728+0000 7f787a7fc640 1 --2- 172.21.15.43:0/3276492365 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7864004800 0x7f7864004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:15.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.728+0000 7f787a7fc640 1 --2- 172.21.15.43:0/3276492365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78640051c0 0x7f786400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:15.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.728+0000 7f787a7fc640 1 -- 172.21.15.43:0/3276492365 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787c0d1910 con 0x7f78640051c0 2024-04-25T08:54:15.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.728+0000 7f787a7fc640 1 -- 172.21.15.43:0/3276492365 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786400c090 con 0x7f7864003b90 2024-04-25T08:54:15.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.728+0000 7f787a7fc640 1 -- 172.21.15.43:0/3276492365 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f786400c290 con 0x7f7864004800 2024-04-25T08:54:15.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.728+0000 7f787bfff640 1 --2- 172.21.15.43:0/3276492365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78640051c0 0x7f786400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:15.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.728+0000 7f787b7fe640 1 --2- 172.21.15.43:0/3276492365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7864003b90 0x7f7864003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:15.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.728+0000 7f787affd640 1 --2- 172.21.15.43:0/3276492365 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7864004800 0x7f7864004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:15.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.728+0000 7f787bfff640 1 -- 172.21.15.43:0/3276492365 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7864004800 msgr2=0x7f7864004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:15.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.728+0000 7f787bfff640 1 --2- 172.21.15.43:0/3276492365 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7864004800 0x7f7864004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:15.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.728+0000 7f787bfff640 1 -- 172.21.15.43:0/3276492365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7864003b90 msgr2=0x7f7864003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:15.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.728+0000 7f787bfff640 1 --2- 172.21.15.43:0/3276492365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7864003b90 0x7f7864003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:15.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.728+0000 7f787bfff640 1 -- 172.21.15.43:0/3276492365 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7864003520 con 0x7f78640051c0 2024-04-25T08:54:15.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.728+0000 7f787b7fe640 1 --2- 172.21.15.43:0/3276492365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7864003b90 0x7f7864003f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:54:15.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.729+0000 7f787bfff640 1 -- 172.21.15.43:0/3276492365 reap_dead start 2024-04-25T08:54:15.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.729+0000 7f787bfff640 1 --2- 172.21.15.43:0/3276492365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78640051c0 0x7f786400b9c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f787400a900 tx=0x7f7864005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b0e3ee2bc77147 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:15.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.729+0000 7f787a7fc640 1 -- 172.21.15.43:0/3276492365 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c009690 con 0x7f78640051c0 2024-04-25T08:54:15.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.729+0000 7f787a7fc640 1 -- 172.21.15.43:0/3276492365 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f787402f070 con 0x7f78640051c0 2024-04-25T08:54:15.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.729+0000 7f787a7fc640 1 -- 172.21.15.43:0/3276492365 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787400a520 con 0x7f78640051c0 2024-04-25T08:54:15.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.729+0000 7f7882012640 1 -- 172.21.15.43:0/3276492365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78640051c0 msgr2=0x7f786400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:15.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.729+0000 7f7882012640 1 --2- 172.21.15.43:0/3276492365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78640051c0 0x7f786400b9c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f787400a900 tx=0x7f7864005830 comp rx=0 tx=0).stop 2024-04-25T08:54:15.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.729+0000 7f7882012640 1 -- 172.21.15.43:0/3276492365 shutdown_connections 2024-04-25T08:54:15.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.729+0000 7f7882012640 1 --2- 172.21.15.43:0/3276492365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f78640051c0 0x7f786400b9c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:15.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.729+0000 7f7882012640 1 -- 172.21.15.43:0/3276492365 >> 172.21.15.43:0/3276492365 conn(0x7f787c09cd20 msgr2=0x7f787c0d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:15.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.729+0000 7f7882012640 1 -- 172.21.15.43:0/3276492365 shutdown_connections 2024-04-25T08:54:15.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.729+0000 7f7882012640 1 -- 172.21.15.43:0/3276492365 wait complete. 2024-04-25T08:54:15.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.730+0000 7f7882012640 1 Processor -- start 2024-04-25T08:54:15.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.731+0000 7f7882012640 1 -- start start 2024-04-25T08:54:15.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.731+0000 7f7882012640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78640051c0 0x7f787c14cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:15.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.731+0000 7f7882012640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f787c15cc80 0x7f787c15a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:15.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.731+0000 7f7882012640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f787c1f6c90 0x7f787c1f7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:15.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.731+0000 7f7882012640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787c161770 con 0x7f787c15cc80 2024-04-25T08:54:15.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.731+0000 7f7882012640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787c0d15a0 con 0x7f78640051c0 2024-04-25T08:54:15.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.731+0000 7f7882012640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f787c0b6820 con 0x7f787c1f6c90 2024-04-25T08:54:15.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.732+0000 7f787affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f787c15cc80 0x7f787c15a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:15.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.732+0000 7f787affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f787c15cc80 0x7f787c15a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37282/0 (socket says 172.21.15.43:37282) 2024-04-25T08:54:15.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.732+0000 7f787affd640 1 -- 172.21.15.43:0/3466426800 learned_addr learned my addr 172.21.15.43:0/3466426800 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:15.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.732+0000 7f787b7fe640 1 --2- 172.21.15.43:0/3466426800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78640051c0 0x7f787c14cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:15.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.732+0000 7f787bfff640 1 --2- 172.21.15.43:0/3466426800 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f787c1f6c90 0x7f787c1f7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:15.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.732+0000 7f787affd640 1 -- 172.21.15.43:0/3466426800 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f787c1f6c90 msgr2=0x7f787c1f7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:15.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.732+0000 7f787affd640 1 --2- 172.21.15.43:0/3466426800 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f787c1f6c90 0x7f787c1f7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:15.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.732+0000 7f787affd640 1 -- 172.21.15.43:0/3466426800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78640051c0 msgr2=0x7f787c14cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:15.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.732+0000 7f787affd640 1 --2- 172.21.15.43:0/3466426800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78640051c0 0x7f787c14cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:15.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.732+0000 7f787affd640 1 -- 172.21.15.43:0/3466426800 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f787c15d1f0 con 0x7f787c15cc80 2024-04-25T08:54:15.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.732+0000 7f787affd640 1 --2- 172.21.15.43:0/3466426800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f787c15cc80 0x7f787c15a6f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f787000f7a0 tx=0x7f787000fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:15.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.732+0000 7f7863fff640 1 -- 172.21.15.43:0/3466426800 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787001a070 con 0x7f787c15cc80 2024-04-25T08:54:15.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.732+0000 7f7863fff640 1 -- 172.21.15.43:0/3466426800 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7870005320 con 0x7f787c15cc80 2024-04-25T08:54:15.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.732+0000 7f7882012640 1 -- 172.21.15.43:0/3466426800 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f787c15d4d0 con 0x7f787c15cc80 2024-04-25T08:54:15.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.732+0000 7f7863fff640 1 -- 172.21.15.43:0/3466426800 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7870007270 con 0x7f787c15cc80 2024-04-25T08:54:15.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.732+0000 7f7882012640 1 -- 172.21.15.43:0/3466426800 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f787c15ac30 con 0x7f787c15cc80 2024-04-25T08:54:15.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.733+0000 7f7882012640 1 -- 172.21.15.43:0/3466426800 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f787c0680f0 con 0x7f787c15cc80 2024-04-25T08:54:15.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.734+0000 7f7863fff640 1 -- 172.21.15.43:0/3466426800 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f787001d070 con 0x7f787c15cc80 2024-04-25T08:54:15.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.734+0000 7f7863fff640 1 --2- 172.21.15.43:0/3466426800 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f786806b500 0x7f786806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:15.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.734+0000 7f7863fff640 1 -- 172.21.15.43:0/3466426800 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f78700899f0 con 0x7f787c15cc80 2024-04-25T08:54:15.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.734+0000 7f787b7fe640 1 --2- 172.21.15.43:0/3466426800 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f786806b500 0x7f786806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:15.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.735+0000 7f787b7fe640 1 --2- 172.21.15.43:0/3466426800 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f786806b500 0x7f786806d9c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f787c1f6bf0 tx=0x7f786c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:15.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.738+0000 7f7863fff640 1 -- 172.21.15.43:0/3466426800 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7870058870 con 0x7f787c15cc80 2024-04-25T08:54:15.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.914+0000 7f7882012640 1 -- 172.21.15.43:0/3466426800 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f787c074ef0 con 0x7f786806b500 2024-04-25T08:54:15.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.921+0000 7f7863fff640 1 -- 172.21.15.43:0/3466426800 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251903 (secure 0 0 0) 0x7f787c074ef0 con 0x7f786806b500 2024-04-25T08:54:15.914 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:54:15.915 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-25T08:54:15.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.924+0000 7f7882012640 1 -- 172.21.15.43:0/3466426800 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f786806b500 msgr2=0x7f786806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:15.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.924+0000 7f7882012640 1 --2- 172.21.15.43:0/3466426800 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f786806b500 0x7f786806d9c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f787c1f6bf0 tx=0x7f786c008040 comp rx=0 tx=0).stop 2024-04-25T08:54:15.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.924+0000 7f7882012640 1 -- 172.21.15.43:0/3466426800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f787c15cc80 msgr2=0x7f787c15a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:15.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.924+0000 7f7882012640 1 --2- 172.21.15.43:0/3466426800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f787c15cc80 0x7f787c15a6f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f787000f7a0 tx=0x7f787000fc60 comp rx=0 tx=0).stop 2024-04-25T08:54:15.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.924+0000 7f7882012640 1 -- 172.21.15.43:0/3466426800 shutdown_connections 2024-04-25T08:54:15.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.924+0000 7f7882012640 1 --2- 172.21.15.43:0/3466426800 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f787c1f6c90 0x7f787c1f7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:15.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.924+0000 7f7882012640 1 --2- 172.21.15.43:0/3466426800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f787c15cc80 0x7f787c15a6f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:15.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.924+0000 7f7882012640 1 --2- 172.21.15.43:0/3466426800 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f786806b500 0x7f786806d9c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:15.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.924+0000 7f7882012640 1 --2- 172.21.15.43:0/3466426800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f78640051c0 0x7f787c14cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:15.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.924+0000 7f7882012640 1 -- 172.21.15.43:0/3466426800 >> 172.21.15.43:0/3466426800 conn(0x7f787c09cd20 msgr2=0x7f787c0b77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:15.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.924+0000 7f7882012640 1 -- 172.21.15.43:0/3466426800 shutdown_connections 2024-04-25T08:54:15.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:15.924+0000 7f7882012640 1 -- 172.21.15.43:0/3466426800 wait complete. 2024-04-25T08:54:15.925 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2024-04-25T08:54:15.695779+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":100,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235528,"kb_used_data":2728,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739192,"statfs":{"total":773094113280,"available":772852932608,"internally_reserved":0,"allocated":2793472,"data_stored":1702628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.128743"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.616586+0000","last_change":"2024-04-25T08:54:12.612549+0000","last_active":"2024-04-25T08:54:12.616586+0000","last_peered":"2024-04-25T08:54:12.616586+0000","last_clean":"2024-04-25T08:54:12.616586+0000","last_became_active":"2024-04-25T08:54:12.612294+0000","last_became_peered":"2024-04-25T08:54:12.612294+0000","last_unstale":"2024-04-25T08:54:12.616586+0000","last_undegraded":"2024-04-25T08:54:12.616586+0000","last_fullsized":"2024-04-25T08:54:12.616586+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:02.123220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.599873+0000","last_change":"2024-04-25T08:54:12.605023+0000","last_active":"2024-04-25T08:54:13.599873+0000","last_peered":"2024-04-25T08:54:13.599873+0000","last_clean":"2024-04-25T08:54:13.599873+0000","last_became_active":"2024-04-25T08:54:12.604703+0000","last_became_peered":"2024-04-25T08:54:12.604703+0000","last_unstale":"2024-04-25T08:54:13.599873+0000","last_undegraded":"2024-04-25T08:54:13.599873+0000","last_fullsized":"2024-04-25T08:54:13.599873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:57.230096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.614820+0000","last_change":"2024-04-25T08:54:12.611793+0000","last_active":"2024-04-25T08:54:12.614820+0000","last_peered":"2024-04-25T08:54:12.614820+0000","last_clean":"2024-04-25T08:54:12.614820+0000","last_became_active":"2024-04-25T08:54:12.611530+0000","last_became_peered":"2024-04-25T08:54:12.611530+0000","last_unstale":"2024-04-25T08:54:12.614820+0000","last_undegraded":"2024-04-25T08:54:12.614820+0000","last_fullsized":"2024-04-25T08:54:12.614820+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:53.347293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601627+0000","last_change":"2024-04-25T08:54:12.600825+0000","last_active":"2024-04-25T08:54:13.601627+0000","last_peered":"2024-04-25T08:54:13.601627+0000","last_clean":"2024-04-25T08:54:13.601627+0000","last_became_active":"2024-04-25T08:54:12.600531+0000","last_became_peered":"2024-04-25T08:54:12.600531+0000","last_unstale":"2024-04-25T08:54:13.601627+0000","last_undegraded":"2024-04-25T08:54:13.601627+0000","last_fullsized":"2024-04-25T08:54:13.601627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:03.021953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.616590+0000","last_change":"2024-04-25T08:54:12.610934+0000","last_active":"2024-04-25T08:54:12.616590+0000","last_peered":"2024-04-25T08:54:12.616590+0000","last_clean":"2024-04-25T08:54:12.616590+0000","last_became_active":"2024-04-25T08:54:12.610457+0000","last_became_peered":"2024-04-25T08:54:12.610457+0000","last_unstale":"2024-04-25T08:54:12.616590+0000","last_undegraded":"2024-04-25T08:54:12.616590+0000","last_fullsized":"2024-04-25T08:54:12.616590+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:20.570657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601664+0000","last_change":"2024-04-25T08:54:12.612007+0000","last_active":"2024-04-25T08:54:13.601664+0000","last_peered":"2024-04-25T08:54:13.601664+0000","last_clean":"2024-04-25T08:54:13.601664+0000","last_became_active":"2024-04-25T08:54:12.611744+0000","last_became_peered":"2024-04-25T08:54:12.611744+0000","last_unstale":"2024-04-25T08:54:13.601664+0000","last_undegraded":"2024-04-25T08:54:13.601664+0000","last_fullsized":"2024-04-25T08:54:13.601664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:06.222677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.612546+0000","last_change":"2024-04-25T08:54:12.609790+0000","last_active":"2024-04-25T08:54:12.612546+0000","last_peered":"2024-04-25T08:54:12.612546+0000","last_clean":"2024-04-25T08:54:12.612546+0000","last_became_active":"2024-04-25T08:54:12.609504+0000","last_became_peered":"2024-04-25T08:54:12.609504+0000","last_unstale":"2024-04-25T08:54:12.612546+0000","last_undegraded":"2024-04-25T08:54:12.612546+0000","last_fullsized":"2024-04-25T08:54:12.612546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:54.538580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.606699+0000","last_change":"2024-04-25T08:54:12.603756+0000","last_active":"2024-04-25T08:54:12.606699+0000","last_peered":"2024-04-25T08:54:12.606699+0000","last_clean":"2024-04-25T08:54:12.606699+0000","last_became_active":"2024-04-25T08:54:12.602966+0000","last_became_peered":"2024-04-25T08:54:12.602966+0000","last_unstale":"2024-04-25T08:54:12.606699+0000","last_undegraded":"2024-04-25T08:54:12.606699+0000","last_fullsized":"2024-04-25T08:54:12.606699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:46.947007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.619210+0000","last_change":"2024-04-25T08:54:12.617394+0000","last_active":"2024-04-25T08:54:12.619210+0000","last_peered":"2024-04-25T08:54:12.619210+0000","last_clean":"2024-04-25T08:54:12.619210+0000","last_became_active":"2024-04-25T08:54:12.616960+0000","last_became_peered":"2024-04-25T08:54:12.616960+0000","last_unstale":"2024-04-25T08:54:12.619210+0000","last_undegraded":"2024-04-25T08:54:12.619210+0000","last_fullsized":"2024-04-25T08:54:12.619210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:18.500422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.616104+0000","last_change":"2024-04-25T08:54:12.607768+0000","last_active":"2024-04-25T08:54:12.616104+0000","last_peered":"2024-04-25T08:54:12.616104+0000","last_clean":"2024-04-25T08:54:12.616104+0000","last_became_active":"2024-04-25T08:54:12.607503+0000","last_became_peered":"2024-04-25T08:54:12.607503+0000","last_unstale":"2024-04-25T08:54:12.616104+0000","last_undegraded":"2024-04-25T08:54:12.616104+0000","last_fullsized":"2024-04-25T08:54:12.616104+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:39.105098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.615850+0000","last_change":"2024-04-25T08:54:12.607264+0000","last_active":"2024-04-25T08:54:12.615850+0000","last_peered":"2024-04-25T08:54:12.615850+0000","last_clean":"2024-04-25T08:54:12.615850+0000","last_became_active":"2024-04-25T08:54:12.606961+0000","last_became_peered":"2024-04-25T08:54:12.606961+0000","last_unstale":"2024-04-25T08:54:12.615850+0000","last_undegraded":"2024-04-25T08:54:12.615850+0000","last_fullsized":"2024-04-25T08:54:12.615850+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:33.374900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601252+0000","last_change":"2024-04-25T08:54:12.604064+0000","last_active":"2024-04-25T08:54:13.601252+0000","last_peered":"2024-04-25T08:54:13.601252+0000","last_clean":"2024-04-25T08:54:13.601252+0000","last_became_active":"2024-04-25T08:54:12.603807+0000","last_became_peered":"2024-04-25T08:54:12.603807+0000","last_unstale":"2024-04-25T08:54:13.601252+0000","last_undegraded":"2024-04-25T08:54:13.601252+0000","last_fullsized":"2024-04-25T08:54:13.601252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:38.245982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.609320+0000","last_change":"2024-04-25T08:54:12.607207+0000","last_active":"2024-04-25T08:54:12.609320+0000","last_peered":"2024-04-25T08:54:12.609320+0000","last_clean":"2024-04-25T08:54:12.609320+0000","last_became_active":"2024-04-25T08:54:12.606934+0000","last_became_peered":"2024-04-25T08:54:12.606934+0000","last_unstale":"2024-04-25T08:54:12.609320+0000","last_undegraded":"2024-04-25T08:54:12.609320+0000","last_fullsized":"2024-04-25T08:54:12.609320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:33.374900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.602008+0000","last_change":"2024-04-25T08:54:12.596313+0000","last_active":"2024-04-25T08:54:12.602008+0000","last_peered":"2024-04-25T08:54:12.602008+0000","last_clean":"2024-04-25T08:54:12.602008+0000","last_became_active":"2024-04-25T08:54:12.596005+0000","last_became_peered":"2024-04-25T08:54:12.596005+0000","last_unstale":"2024-04-25T08:54:12.602008+0000","last_undegraded":"2024-04-25T08:54:12.602008+0000","last_fullsized":"2024-04-25T08:54:12.602008+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:11.550754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.599534+0000","last_change":"2024-04-25T08:54:12.610230+0000","last_active":"2024-04-25T08:54:13.599534+0000","last_peered":"2024-04-25T08:54:13.599534+0000","last_clean":"2024-04-25T08:54:13.599534+0000","last_became_active":"2024-04-25T08:54:12.609932+0000","last_became_peered":"2024-04-25T08:54:12.609932+0000","last_unstale":"2024-04-25T08:54:13.599534+0000","last_undegraded":"2024-04-25T08:54:13.599534+0000","last_fullsized":"2024-04-25T08:54:13.599534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:54.538580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601103+0000","last_change":"2024-04-25T08:54:12.600584+0000","last_active":"2024-04-25T08:54:13.601103+0000","last_peered":"2024-04-25T08:54:13.601103+0000","last_clean":"2024-04-25T08:54:13.601103+0000","last_became_active":"2024-04-25T08:54:12.598989+0000","last_became_peered":"2024-04-25T08:54:12.598989+0000","last_unstale":"2024-04-25T08:54:13.601103+0000","last_undegraded":"2024-04-25T08:54:13.601103+0000","last_fullsized":"2024-04-25T08:54:13.601103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:45.793417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601517+0000","last_change":"2024-04-25T08:54:12.607275+0000","last_active":"2024-04-25T08:54:13.601517+0000","last_peered":"2024-04-25T08:54:13.601517+0000","last_clean":"2024-04-25T08:54:13.601517+0000","last_became_active":"2024-04-25T08:54:12.606990+0000","last_became_peered":"2024-04-25T08:54:12.606990+0000","last_unstale":"2024-04-25T08:54:13.601517+0000","last_undegraded":"2024-04-25T08:54:13.601517+0000","last_fullsized":"2024-04-25T08:54:13.601517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:18.500422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.608910+0000","last_change":"2024-04-25T08:54:12.607413+0000","last_active":"2024-04-25T08:54:12.608910+0000","last_peered":"2024-04-25T08:54:12.608910+0000","last_clean":"2024-04-25T08:54:12.608910+0000","last_became_active":"2024-04-25T08:54:12.607135+0000","last_became_peered":"2024-04-25T08:54:12.607135+0000","last_unstale":"2024-04-25T08:54:12.608910+0000","last_undegraded":"2024-04-25T08:54:12.608910+0000","last_fullsized":"2024-04-25T08:54:12.608910+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:51.465717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.611452+0000","last_change":"2024-04-25T08:54:12.607734+0000","last_active":"2024-04-25T08:54:12.611452+0000","last_peered":"2024-04-25T08:54:12.611452+0000","last_clean":"2024-04-25T08:54:12.611452+0000","last_became_active":"2024-04-25T08:54:12.607462+0000","last_became_peered":"2024-04-25T08:54:12.607462+0000","last_unstale":"2024-04-25T08:54:12.611452+0000","last_undegraded":"2024-04-25T08:54:12.611452+0000","last_fullsized":"2024-04-25T08:54:12.611452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:14.023628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.616370+0000","last_change":"2024-04-25T08:54:12.609395+0000","last_active":"2024-04-25T08:54:12.616370+0000","last_peered":"2024-04-25T08:54:12.616370+0000","last_clean":"2024-04-25T08:54:12.616370+0000","last_became_active":"2024-04-25T08:54:12.609094+0000","last_became_peered":"2024-04-25T08:54:12.609094+0000","last_unstale":"2024-04-25T08:54:12.616370+0000","last_undegraded":"2024-04-25T08:54:12.616370+0000","last_fullsized":"2024-04-25T08:54:12.616370+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:43.731704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.609911+0000","last_change":"2024-04-25T08:54:12.605364+0000","last_active":"2024-04-25T08:54:12.609911+0000","last_peered":"2024-04-25T08:54:12.609911+0000","last_clean":"2024-04-25T08:54:12.609911+0000","last_became_active":"2024-04-25T08:54:12.605084+0000","last_became_peered":"2024-04-25T08:54:12.605084+0000","last_unstale":"2024-04-25T08:54:12.609911+0000","last_undegraded":"2024-04-25T08:54:12.609911+0000","last_fullsized":"2024-04-25T08:54:12.609911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:33.374900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601962+0000","last_change":"2024-04-25T08:54:12.606453+0000","last_active":"2024-04-25T08:54:13.601962+0000","last_peered":"2024-04-25T08:54:13.601962+0000","last_clean":"2024-04-25T08:54:13.601962+0000","last_became_active":"2024-04-25T08:54:12.606072+0000","last_became_peered":"2024-04-25T08:54:12.606072+0000","last_unstale":"2024-04-25T08:54:13.601962+0000","last_undegraded":"2024-04-25T08:54:13.601962+0000","last_fullsized":"2024-04-25T08:54:13.601962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:36.638705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.616348+0000","last_change":"2024-04-25T08:54:12.615079+0000","last_active":"2024-04-25T08:54:12.616348+0000","last_peered":"2024-04-25T08:54:12.616348+0000","last_clean":"2024-04-25T08:54:12.616348+0000","last_became_active":"2024-04-25T08:54:12.614779+0000","last_became_peered":"2024-04-25T08:54:12.614779+0000","last_unstale":"2024-04-25T08:54:12.616348+0000","last_undegraded":"2024-04-25T08:54:12.616348+0000","last_fullsized":"2024-04-25T08:54:12.616348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:28.227965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601263+0000","last_change":"2024-04-25T08:54:12.605871+0000","last_active":"2024-04-25T08:54:13.601263+0000","last_peered":"2024-04-25T08:54:13.601263+0000","last_clean":"2024-04-25T08:54:13.601263+0000","last_became_active":"2024-04-25T08:54:12.605598+0000","last_became_peered":"2024-04-25T08:54:12.605598+0000","last_unstale":"2024-04-25T08:54:13.601263+0000","last_undegraded":"2024-04-25T08:54:13.601263+0000","last_fullsized":"2024-04-25T08:54:13.601263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:57.230096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.615588+0000","last_change":"2024-04-25T08:54:12.612253+0000","last_active":"2024-04-25T08:54:12.615588+0000","last_peered":"2024-04-25T08:54:12.615588+0000","last_clean":"2024-04-25T08:54:12.615588+0000","last_became_active":"2024-04-25T08:54:12.611923+0000","last_became_peered":"2024-04-25T08:54:12.611923+0000","last_unstale":"2024-04-25T08:54:12.615588+0000","last_undegraded":"2024-04-25T08:54:12.615588+0000","last_fullsized":"2024-04-25T08:54:12.615588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:43.731704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.616126+0000","last_change":"2024-04-25T08:54:12.608764+0000","last_active":"2024-04-25T08:54:12.616126+0000","last_peered":"2024-04-25T08:54:12.616126+0000","last_clean":"2024-04-25T08:54:12.616126+0000","last_became_active":"2024-04-25T08:54:12.608488+0000","last_became_peered":"2024-04-25T08:54:12.608488+0000","last_unstale":"2024-04-25T08:54:12.616126+0000","last_undegraded":"2024-04-25T08:54:12.616126+0000","last_fullsized":"2024-04-25T08:54:12.616126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:53.056423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.599706+0000","last_change":"2024-04-25T08:54:12.602969+0000","last_active":"2024-04-25T08:54:13.599706+0000","last_peered":"2024-04-25T08:54:13.599706+0000","last_clean":"2024-04-25T08:54:13.599706+0000","last_became_active":"2024-04-25T08:54:12.602464+0000","last_became_peered":"2024-04-25T08:54:12.602464+0000","last_unstale":"2024-04-25T08:54:13.599706+0000","last_undegraded":"2024-04-25T08:54:13.599706+0000","last_fullsized":"2024-04-25T08:54:13.599706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:02.123220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.616307+0000","last_change":"2024-04-25T08:54:12.613884+0000","last_active":"2024-04-25T08:54:12.616307+0000","last_peered":"2024-04-25T08:54:12.616307+0000","last_clean":"2024-04-25T08:54:12.616307+0000","last_became_active":"2024-04-25T08:54:12.613610+0000","last_became_peered":"2024-04-25T08:54:12.613610+0000","last_unstale":"2024-04-25T08:54:12.616307+0000","last_undegraded":"2024-04-25T08:54:12.616307+0000","last_fullsized":"2024-04-25T08:54:12.616307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:14.023628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.613597+0000","last_change":"2024-04-25T08:54:12.611194+0000","last_active":"2024-04-25T08:54:12.613597+0000","last_peered":"2024-04-25T08:54:12.613597+0000","last_clean":"2024-04-25T08:54:12.613597+0000","last_became_active":"2024-04-25T08:54:12.610877+0000","last_became_peered":"2024-04-25T08:54:12.610877+0000","last_unstale":"2024-04-25T08:54:12.613597+0000","last_undegraded":"2024-04-25T08:54:12.613597+0000","last_fullsized":"2024-04-25T08:54:12.613597+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:43.731704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.602013+0000","last_change":"2024-04-25T08:54:12.600165+0000","last_active":"2024-04-25T08:54:13.602013+0000","last_peered":"2024-04-25T08:54:13.602013+0000","last_clean":"2024-04-25T08:54:13.602013+0000","last_became_active":"2024-04-25T08:54:12.599911+0000","last_became_peered":"2024-04-25T08:54:12.599911+0000","last_unstale":"2024-04-25T08:54:13.602013+0000","last_undegraded":"2024-04-25T08:54:13.602013+0000","last_fullsized":"2024-04-25T08:54:13.602013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:06.750914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.608815+0000","last_change":"2024-04-25T08:54:12.604655+0000","last_active":"2024-04-25T08:54:12.608815+0000","last_peered":"2024-04-25T08:54:12.608815+0000","last_clean":"2024-04-25T08:54:12.608815+0000","last_became_active":"2024-04-25T08:54:12.604366+0000","last_became_peered":"2024-04-25T08:54:12.604366+0000","last_unstale":"2024-04-25T08:54:12.608815+0000","last_undegraded":"2024-04-25T08:54:12.608815+0000","last_fullsized":"2024-04-25T08:54:12.608815+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:46.947007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.619236+0000","last_change":"2024-04-25T08:54:12.615832+0000","last_active":"2024-04-25T08:54:12.619236+0000","last_peered":"2024-04-25T08:54:12.619236+0000","last_clean":"2024-04-25T08:54:12.619236+0000","last_became_active":"2024-04-25T08:54:12.615442+0000","last_became_peered":"2024-04-25T08:54:12.615442+0000","last_unstale":"2024-04-25T08:54:12.619236+0000","last_undegraded":"2024-04-25T08:54:12.619236+0000","last_fullsized":"2024-04-25T08:54:12.619236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:28.227965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601064+0000","last_change":"2024-04-25T08:54:12.598903+0000","last_active":"2024-04-25T08:54:13.601064+0000","last_peered":"2024-04-25T08:54:13.601064+0000","last_clean":"2024-04-25T08:54:13.601064+0000","last_became_active":"2024-04-25T08:54:12.598005+0000","last_became_peered":"2024-04-25T08:54:12.598005+0000","last_unstale":"2024-04-25T08:54:13.601064+0000","last_undegraded":"2024-04-25T08:54:13.601064+0000","last_fullsized":"2024-04-25T08:54:13.601064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:11.183046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.612320+0000","last_change":"2024-04-25T08:54:11.585710+0000","last_active":"2024-04-25T08:54:12.612320+0000","last_peered":"2024-04-25T08:54:12.612320+0000","last_clean":"2024-04-25T08:54:12.612320+0000","last_became_active":"2024-04-25T08:54:11.585090+0000","last_became_peered":"2024-04-25T08:54:11.585090+0000","last_unstale":"2024-04-25T08:54:12.612320+0000","last_undegraded":"2024-04-25T08:54:12.612320+0000","last_fullsized":"2024-04-25T08:54:12.612320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:10.545906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.603083+0000","last_change":"2024-04-25T08:54:12.600641+0000","last_active":"2024-04-25T08:54:12.603083+0000","last_peered":"2024-04-25T08:54:12.603083+0000","last_clean":"2024-04-25T08:54:12.603083+0000","last_became_active":"2024-04-25T08:54:12.598924+0000","last_became_peered":"2024-04-25T08:54:12.598924+0000","last_unstale":"2024-04-25T08:54:12.603083+0000","last_undegraded":"2024-04-25T08:54:12.603083+0000","last_fullsized":"2024-04-25T08:54:12.603083+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:38.245982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.611849+0000","last_change":"2024-04-25T08:54:11.588038+0000","last_active":"2024-04-25T08:54:12.611849+0000","last_peered":"2024-04-25T08:54:12.611849+0000","last_clean":"2024-04-25T08:54:12.611849+0000","last_became_active":"2024-04-25T08:54:11.587710+0000","last_became_peered":"2024-04-25T08:54:11.587710+0000","last_unstale":"2024-04-25T08:54:12.611849+0000","last_undegraded":"2024-04-25T08:54:12.611849+0000","last_fullsized":"2024-04-25T08:54:12.611849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:37.241134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.592212+0000","last_change":"2024-04-25T08:54:11.590094+0000","last_active":"2024-04-25T08:54:12.592212+0000","last_peered":"2024-04-25T08:54:12.592212+0000","last_clean":"2024-04-25T08:54:12.592212+0000","last_became_active":"2024-04-25T08:54:11.589720+0000","last_became_peered":"2024-04-25T08:54:11.589720+0000","last_unstale":"2024-04-25T08:54:12.592212+0000","last_undegraded":"2024-04-25T08:54:12.592212+0000","last_fullsized":"2024-04-25T08:54:12.592212+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:54.860395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.606681+0000","last_change":"2024-04-25T08:54:12.603523+0000","last_active":"2024-04-25T08:54:12.606681+0000","last_peered":"2024-04-25T08:54:12.606681+0000","last_clean":"2024-04-25T08:54:12.606681+0000","last_became_active":"2024-04-25T08:54:12.602684+0000","last_became_peered":"2024-04-25T08:54:12.602684+0000","last_unstale":"2024-04-25T08:54:12.606681+0000","last_undegraded":"2024-04-25T08:54:12.606681+0000","last_fullsized":"2024-04-25T08:54:12.606681+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:38.245982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601357+0000","last_change":"2024-04-25T08:54:11.578444+0000","last_active":"2024-04-25T08:54:13.601357+0000","last_peered":"2024-04-25T08:54:13.601357+0000","last_clean":"2024-04-25T08:54:13.601357+0000","last_became_active":"2024-04-25T08:54:11.578170+0000","last_became_peered":"2024-04-25T08:54:11.578170+0000","last_unstale":"2024-04-25T08:54:13.601357+0000","last_undegraded":"2024-04-25T08:54:13.601357+0000","last_fullsized":"2024-04-25T08:54:13.601357+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:37.241134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.599533+0000","last_change":"2024-04-25T08:54:12.602572+0000","last_active":"2024-04-25T08:54:13.599533+0000","last_peered":"2024-04-25T08:54:13.599533+0000","last_clean":"2024-04-25T08:54:13.599533+0000","last_became_active":"2024-04-25T08:54:12.601587+0000","last_became_peered":"2024-04-25T08:54:12.601587+0000","last_unstale":"2024-04-25T08:54:13.599533+0000","last_undegraded":"2024-04-25T08:54:13.599533+0000","last_fullsized":"2024-04-25T08:54:13.599533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:56.990780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.602350+0000","last_change":"2024-04-25T08:54:11.592937+0000","last_active":"2024-04-25T08:54:12.602350+0000","last_peered":"2024-04-25T08:54:12.602350+0000","last_clean":"2024-04-25T08:54:12.602350+0000","last_became_active":"2024-04-25T08:54:11.592647+0000","last_became_peered":"2024-04-25T08:54:11.592647+0000","last_unstale":"2024-04-25T08:54:12.602350+0000","last_undegraded":"2024-04-25T08:54:12.602350+0000","last_fullsized":"2024-04-25T08:54:12.602350+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:45.942159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601977+0000","last_change":"2024-04-25T08:54:12.608078+0000","last_active":"2024-04-25T08:54:13.601977+0000","last_peered":"2024-04-25T08:54:13.601977+0000","last_clean":"2024-04-25T08:54:13.601977+0000","last_became_active":"2024-04-25T08:54:12.605851+0000","last_became_peered":"2024-04-25T08:54:12.605851+0000","last_unstale":"2024-04-25T08:54:13.601977+0000","last_undegraded":"2024-04-25T08:54:13.601977+0000","last_fullsized":"2024-04-25T08:54:13.601977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:54.612512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601849+0000","last_change":"2024-04-25T08:54:11.588593+0000","last_active":"2024-04-25T08:54:13.601849+0000","last_peered":"2024-04-25T08:54:13.601849+0000","last_clean":"2024-04-25T08:54:13.601849+0000","last_became_active":"2024-04-25T08:54:11.588087+0000","last_became_peered":"2024-04-25T08:54:11.588087+0000","last_unstale":"2024-04-25T08:54:13.601849+0000","last_undegraded":"2024-04-25T08:54:13.601849+0000","last_fullsized":"2024-04-25T08:54:13.601849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:10.178198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.614127+0000","last_change":"2024-04-25T08:54:12.610492+0000","last_active":"2024-04-25T08:54:12.614127+0000","last_peered":"2024-04-25T08:54:12.614127+0000","last_clean":"2024-04-25T08:54:12.614127+0000","last_became_active":"2024-04-25T08:54:12.610228+0000","last_became_peered":"2024-04-25T08:54:12.610228+0000","last_unstale":"2024-04-25T08:54:12.614127+0000","last_undegraded":"2024-04-25T08:54:12.614127+0000","last_fullsized":"2024-04-25T08:54:12.614127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:53.056423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.599817+0000","last_change":"2024-04-25T08:54:11.585441+0000","last_active":"2024-04-25T08:54:13.599817+0000","last_peered":"2024-04-25T08:54:13.599817+0000","last_clean":"2024-04-25T08:54:13.599817+0000","last_became_active":"2024-04-25T08:54:11.585098+0000","last_became_peered":"2024-04-25T08:54:11.585098+0000","last_unstale":"2024-04-25T08:54:13.599817+0000","last_undegraded":"2024-04-25T08:54:13.599817+0000","last_fullsized":"2024-04-25T08:54:13.599817+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:50.460868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601444+0000","last_change":"2024-04-25T08:54:12.606323+0000","last_active":"2024-04-25T08:54:13.601444+0000","last_peered":"2024-04-25T08:54:13.601444+0000","last_clean":"2024-04-25T08:54:13.601444+0000","last_became_active":"2024-04-25T08:54:12.606063+0000","last_became_peered":"2024-04-25T08:54:12.606063+0000","last_unstale":"2024-04-25T08:54:13.601444+0000","last_undegraded":"2024-04-25T08:54:13.601444+0000","last_fullsized":"2024-04-25T08:54:13.601444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:46.947007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.621900+0000","last_change":"2024-04-25T08:54:11.590170+0000","last_active":"2024-04-25T08:54:12.621900+0000","last_peered":"2024-04-25T08:54:12.621900+0000","last_clean":"2024-04-25T08:54:12.621900+0000","last_became_active":"2024-04-25T08:54:11.589869+0000","last_became_peered":"2024-04-25T08:54:11.589869+0000","last_unstale":"2024-04-25T08:54:12.621900+0000","last_undegraded":"2024-04-25T08:54:12.621900+0000","last_fullsized":"2024-04-25T08:54:12.621900+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:10.545906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.611433+0000","last_change":"2024-04-25T08:54:12.607302+0000","last_active":"2024-04-25T08:54:12.611433+0000","last_peered":"2024-04-25T08:54:12.611433+0000","last_clean":"2024-04-25T08:54:12.611433+0000","last_became_active":"2024-04-25T08:54:12.607042+0000","last_became_peered":"2024-04-25T08:54:12.607042+0000","last_unstale":"2024-04-25T08:54:12.611433+0000","last_undegraded":"2024-04-25T08:54:12.611433+0000","last_fullsized":"2024-04-25T08:54:12.611433+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:53.347293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.616844+0000","last_change":"2024-04-25T08:54:11.586629+0000","last_active":"2024-04-25T08:54:12.616844+0000","last_peered":"2024-04-25T08:54:12.616844+0000","last_clean":"2024-04-25T08:54:12.616844+0000","last_became_active":"2024-04-25T08:54:11.586289+0000","last_became_peered":"2024-04-25T08:54:11.586289+0000","last_unstale":"2024-04-25T08:54:12.616844+0000","last_undegraded":"2024-04-25T08:54:12.616844+0000","last_fullsized":"2024-04-25T08:54:12.616844+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:12.212662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.608489+0000","last_change":"2024-04-25T08:54:12.604702+0000","last_active":"2024-04-25T08:54:12.608489+0000","last_peered":"2024-04-25T08:54:12.608489+0000","last_clean":"2024-04-25T08:54:12.608489+0000","last_became_active":"2024-04-25T08:54:12.604410+0000","last_became_peered":"2024-04-25T08:54:12.604410+0000","last_unstale":"2024-04-25T08:54:12.608489+0000","last_undegraded":"2024-04-25T08:54:12.608489+0000","last_fullsized":"2024-04-25T08:54:12.608489+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:51.465717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.602125+0000","last_change":"2024-04-25T08:54:11.583543+0000","last_active":"2024-04-25T08:54:13.602125+0000","last_peered":"2024-04-25T08:54:13.602125+0000","last_clean":"2024-04-25T08:54:13.602125+0000","last_became_active":"2024-04-25T08:54:11.583254+0000","last_became_peered":"2024-04-25T08:54:11.583254+0000","last_unstale":"2024-04-25T08:54:13.602125+0000","last_undegraded":"2024-04-25T08:54:13.602125+0000","last_fullsized":"2024-04-25T08:54:13.602125+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:45.942159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.612194+0000","last_change":"2024-04-25T08:54:12.609195+0000","last_active":"2024-04-25T08:54:12.612194+0000","last_peered":"2024-04-25T08:54:12.612194+0000","last_clean":"2024-04-25T08:54:12.612194+0000","last_became_active":"2024-04-25T08:54:12.608943+0000","last_became_peered":"2024-04-25T08:54:12.608943+0000","last_unstale":"2024-04-25T08:54:12.612194+0000","last_undegraded":"2024-04-25T08:54:12.612194+0000","last_fullsized":"2024-04-25T08:54:12.612194+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:28.227965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.601096+0000","last_change":"2024-04-25T08:54:11.586926+0000","last_active":"2024-04-25T08:54:12.601096+0000","last_peered":"2024-04-25T08:54:12.601096+0000","last_clean":"2024-04-25T08:54:12.601096+0000","last_became_active":"2024-04-25T08:54:11.586600+0000","last_became_peered":"2024-04-25T08:54:11.586600+0000","last_unstale":"2024-04-25T08:54:12.601096+0000","last_undegraded":"2024-04-25T08:54:12.601096+0000","last_fullsized":"2024-04-25T08:54:12.601096+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:10.178198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.619222+0000","last_change":"2024-04-25T08:54:12.615811+0000","last_active":"2024-04-25T08:54:12.619222+0000","last_peered":"2024-04-25T08:54:12.619222+0000","last_clean":"2024-04-25T08:54:12.619222+0000","last_became_active":"2024-04-25T08:54:12.615492+0000","last_became_peered":"2024-04-25T08:54:12.615492+0000","last_unstale":"2024-04-25T08:54:12.619222+0000","last_undegraded":"2024-04-25T08:54:12.619222+0000","last_fullsized":"2024-04-25T08:54:12.619222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:14.023628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.614336+0000","last_change":"2024-04-25T08:54:11.583593+0000","last_active":"2024-04-25T08:54:12.614336+0000","last_peered":"2024-04-25T08:54:12.614336+0000","last_clean":"2024-04-25T08:54:12.614336+0000","last_became_active":"2024-04-25T08:54:11.582876+0000","last_became_peered":"2024-04-25T08:54:11.582876+0000","last_unstale":"2024-04-25T08:54:12.614336+0000","last_undegraded":"2024-04-25T08:54:12.614336+0000","last_fullsized":"2024-04-25T08:54:12.614336+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:10.545906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.612525+0000","last_change":"2024-04-25T08:54:12.609297+0000","last_active":"2024-04-25T08:54:12.612525+0000","last_peered":"2024-04-25T08:54:12.612525+0000","last_clean":"2024-04-25T08:54:12.612525+0000","last_became_active":"2024-04-25T08:54:12.608948+0000","last_became_peered":"2024-04-25T08:54:12.608948+0000","last_unstale":"2024-04-25T08:54:12.612525+0000","last_undegraded":"2024-04-25T08:54:12.612525+0000","last_fullsized":"2024-04-25T08:54:12.612525+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:05.099361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601349+0000","last_change":"2024-04-25T08:54:11.586306+0000","last_active":"2024-04-25T08:54:13.601349+0000","last_peered":"2024-04-25T08:54:13.601349+0000","last_clean":"2024-04-25T08:54:13.601349+0000","last_became_active":"2024-04-25T08:54:11.585959+0000","last_became_peered":"2024-04-25T08:54:11.585959+0000","last_unstale":"2024-04-25T08:54:13.601349+0000","last_undegraded":"2024-04-25T08:54:13.601349+0000","last_fullsized":"2024-04-25T08:54:13.601349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:40.673650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.615418+0000","last_change":"2024-04-25T08:54:12.609239+0000","last_active":"2024-04-25T08:54:12.615418+0000","last_peered":"2024-04-25T08:54:12.615418+0000","last_clean":"2024-04-25T08:54:12.615418+0000","last_became_active":"2024-04-25T08:54:12.608994+0000","last_became_peered":"2024-04-25T08:54:12.608994+0000","last_unstale":"2024-04-25T08:54:12.615418+0000","last_undegraded":"2024-04-25T08:54:12.615418+0000","last_fullsized":"2024-04-25T08:54:12.615418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:56.990780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.614355+0000","last_change":"2024-04-25T08:54:11.589122+0000","last_active":"2024-04-25T08:54:12.614355+0000","last_peered":"2024-04-25T08:54:12.614355+0000","last_clean":"2024-04-25T08:54:12.614355+0000","last_became_active":"2024-04-25T08:54:11.588772+0000","last_became_peered":"2024-04-25T08:54:11.588772+0000","last_unstale":"2024-04-25T08:54:12.614355+0000","last_undegraded":"2024-04-25T08:54:12.614355+0000","last_fullsized":"2024-04-25T08:54:12.614355+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:37.241134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.616641+0000","last_change":"2024-04-25T08:54:12.613985+0000","last_active":"2024-04-25T08:54:12.616641+0000","last_peered":"2024-04-25T08:54:12.616641+0000","last_clean":"2024-04-25T08:54:12.616641+0000","last_became_active":"2024-04-25T08:54:12.613717+0000","last_became_peered":"2024-04-25T08:54:12.613717+0000","last_unstale":"2024-04-25T08:54:12.616641+0000","last_undegraded":"2024-04-25T08:54:12.616641+0000","last_fullsized":"2024-04-25T08:54:12.616641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:45.793417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.599846+0000","last_change":"2024-04-25T08:54:11.586724+0000","last_active":"2024-04-25T08:54:13.599846+0000","last_peered":"2024-04-25T08:54:13.599846+0000","last_clean":"2024-04-25T08:54:13.599846+0000","last_became_active":"2024-04-25T08:54:11.586219+0000","last_became_peered":"2024-04-25T08:54:11.586219+0000","last_unstale":"2024-04-25T08:54:13.599846+0000","last_undegraded":"2024-04-25T08:54:13.599846+0000","last_fullsized":"2024-04-25T08:54:13.599846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:32.370052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601817+0000","last_change":"2024-04-25T08:54:12.600997+0000","last_active":"2024-04-25T08:54:13.601817+0000","last_peered":"2024-04-25T08:54:13.601817+0000","last_clean":"2024-04-25T08:54:13.601817+0000","last_became_active":"2024-04-25T08:54:12.600704+0000","last_became_peered":"2024-04-25T08:54:12.600704+0000","last_unstale":"2024-04-25T08:54:13.601817+0000","last_undegraded":"2024-04-25T08:54:13.601817+0000","last_fullsized":"2024-04-25T08:54:13.601817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:17.430796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.595415+0000","last_change":"2024-04-25T08:54:11.578803+0000","last_active":"2024-04-25T08:54:12.595415+0000","last_peered":"2024-04-25T08:54:12.595415+0000","last_clean":"2024-04-25T08:54:12.595415+0000","last_became_active":"2024-04-25T08:54:11.578421+0000","last_became_peered":"2024-04-25T08:54:11.578421+0000","last_unstale":"2024-04-25T08:54:12.595415+0000","last_undegraded":"2024-04-25T08:54:12.595415+0000","last_fullsized":"2024-04-25T08:54:12.595415+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:54.860395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.619195+0000","last_change":"2024-04-25T08:54:12.617092+0000","last_active":"2024-04-25T08:54:12.619195+0000","last_peered":"2024-04-25T08:54:12.619195+0000","last_clean":"2024-04-25T08:54:12.619195+0000","last_became_active":"2024-04-25T08:54:12.616799+0000","last_became_peered":"2024-04-25T08:54:12.616799+0000","last_unstale":"2024-04-25T08:54:12.619195+0000","last_undegraded":"2024-04-25T08:54:12.619195+0000","last_fullsized":"2024-04-25T08:54:12.619195+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:57.230096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":168,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.599577+0000","last_change":"2024-04-25T08:54:00.818044+0000","last_active":"2024-04-25T08:54:12.599577+0000","last_peered":"2024-04-25T08:54:12.599577+0000","last_clean":"2024-04-25T08:54:12.599577+0000","last_became_active":"2024-04-25T08:54:00.817098+0000","last_became_peered":"2024-04-25T08:54:00.817098+0000","last_unstale":"2024-04-25T08:54:12.599577+0000","last_undegraded":"2024-04-25T08:54:12.599577+0000","last_fullsized":"2024-04-25T08:54:12.599577+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_clean_scrub_stamp":"2024-04-25T08:53:59.739677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:48.129513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601017+0000","last_change":"2024-04-25T08:54:11.576740+0000","last_active":"2024-04-25T08:54:13.601017+0000","last_peered":"2024-04-25T08:54:13.601017+0000","last_clean":"2024-04-25T08:54:13.601017+0000","last_became_active":"2024-04-25T08:54:11.576444+0000","last_became_peered":"2024-04-25T08:54:11.576444+0000","last_unstale":"2024-04-25T08:54:13.601017+0000","last_undegraded":"2024-04-25T08:54:13.601017+0000","last_fullsized":"2024-04-25T08:54:13.601017+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:10.545906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.608730+0000","last_change":"2024-04-25T08:54:12.606225+0000","last_active":"2024-04-25T08:54:12.608730+0000","last_peered":"2024-04-25T08:54:12.608730+0000","last_clean":"2024-04-25T08:54:12.608730+0000","last_became_active":"2024-04-25T08:54:12.605897+0000","last_became_peered":"2024-04-25T08:54:12.605897+0000","last_unstale":"2024-04-25T08:54:12.608730+0000","last_undegraded":"2024-04-25T08:54:12.608730+0000","last_fullsized":"2024-04-25T08:54:12.608730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:51.465717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.602640+0000","last_change":"2024-04-25T08:54:11.590032+0000","last_active":"2024-04-25T08:54:12.602640+0000","last_peered":"2024-04-25T08:54:12.602640+0000","last_clean":"2024-04-25T08:54:12.602640+0000","last_became_active":"2024-04-25T08:54:11.589701+0000","last_became_peered":"2024-04-25T08:54:11.589701+0000","last_unstale":"2024-04-25T08:54:12.602640+0000","last_undegraded":"2024-04-25T08:54:12.602640+0000","last_fullsized":"2024-04-25T08:54:12.602640+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:52.051575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.607730+0000","last_change":"2024-04-25T08:54:12.603873+0000","last_active":"2024-04-25T08:54:12.607730+0000","last_peered":"2024-04-25T08:54:12.607730+0000","last_clean":"2024-04-25T08:54:12.607730+0000","last_became_active":"2024-04-25T08:54:12.599347+0000","last_became_peered":"2024-04-25T08:54:12.599347+0000","last_unstale":"2024-04-25T08:54:12.607730+0000","last_undegraded":"2024-04-25T08:54:12.607730+0000","last_fullsized":"2024-04-25T08:54:12.607730+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:46.947007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.595592+0000","last_change":"2024-04-25T08:54:11.588495+0000","last_active":"2024-04-25T08:54:12.595592+0000","last_peered":"2024-04-25T08:54:12.595592+0000","last_clean":"2024-04-25T08:54:12.595592+0000","last_became_active":"2024-04-25T08:54:11.588141+0000","last_became_peered":"2024-04-25T08:54:11.588141+0000","last_unstale":"2024-04-25T08:54:12.595592+0000","last_undegraded":"2024-04-25T08:54:12.595592+0000","last_fullsized":"2024-04-25T08:54:12.595592+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:10.178198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.599547+0000","last_change":"2024-04-25T08:54:12.610027+0000","last_active":"2024-04-25T08:54:13.599547+0000","last_peered":"2024-04-25T08:54:13.599547+0000","last_clean":"2024-04-25T08:54:13.599547+0000","last_became_active":"2024-04-25T08:54:12.609091+0000","last_became_peered":"2024-04-25T08:54:12.609091+0000","last_unstale":"2024-04-25T08:54:13.599547+0000","last_undegraded":"2024-04-25T08:54:13.599547+0000","last_fullsized":"2024-04-25T08:54:13.599547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:20.570657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.602128+0000","last_change":"2024-04-25T08:54:11.587597+0000","last_active":"2024-04-25T08:54:13.602128+0000","last_peered":"2024-04-25T08:54:13.602128+0000","last_clean":"2024-04-25T08:54:13.602128+0000","last_became_active":"2024-04-25T08:54:11.587078+0000","last_became_peered":"2024-04-25T08:54:11.587078+0000","last_unstale":"2024-04-25T08:54:13.602128+0000","last_undegraded":"2024-04-25T08:54:13.602128+0000","last_fullsized":"2024-04-25T08:54:13.602128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:50.460868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601656+0000","last_change":"2024-04-25T08:54:12.611656+0000","last_active":"2024-04-25T08:54:13.601656+0000","last_peered":"2024-04-25T08:54:13.601656+0000","last_clean":"2024-04-25T08:54:13.601656+0000","last_became_active":"2024-04-25T08:54:12.611359+0000","last_became_peered":"2024-04-25T08:54:12.611359+0000","last_unstale":"2024-04-25T08:54:13.601656+0000","last_undegraded":"2024-04-25T08:54:13.601656+0000","last_fullsized":"2024-04-25T08:54:13.601656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:33.374900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.599718+0000","last_change":"2024-04-25T08:54:11.589751+0000","last_active":"2024-04-25T08:54:13.599718+0000","last_peered":"2024-04-25T08:54:13.599718+0000","last_clean":"2024-04-25T08:54:13.599718+0000","last_became_active":"2024-04-25T08:54:11.589362+0000","last_became_peered":"2024-04-25T08:54:11.589362+0000","last_unstale":"2024-04-25T08:54:13.599718+0000","last_undegraded":"2024-04-25T08:54:13.599718+0000","last_fullsized":"2024-04-25T08:54:13.599718+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:10.178198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.602180+0000","last_change":"2024-04-25T08:54:12.606441+0000","last_active":"2024-04-25T08:54:13.602180+0000","last_peered":"2024-04-25T08:54:13.602180+0000","last_clean":"2024-04-25T08:54:13.602180+0000","last_became_active":"2024-04-25T08:54:12.605425+0000","last_became_peered":"2024-04-25T08:54:12.605425+0000","last_unstale":"2024-04-25T08:54:13.602180+0000","last_undegraded":"2024-04-25T08:54:13.602180+0000","last_fullsized":"2024-04-25T08:54:13.602180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:31.051483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.595519+0000","last_change":"2024-04-25T08:54:11.591338+0000","last_active":"2024-04-25T08:54:12.595519+0000","last_peered":"2024-04-25T08:54:12.595519+0000","last_clean":"2024-04-25T08:54:12.595519+0000","last_became_active":"2024-04-25T08:54:11.591074+0000","last_became_peered":"2024-04-25T08:54:11.591074+0000","last_unstale":"2024-04-25T08:54:12.595519+0000","last_undegraded":"2024-04-25T08:54:12.595519+0000","last_fullsized":"2024-04-25T08:54:12.595519+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:32.370052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.599726+0000","last_change":"2024-04-25T08:54:12.602183+0000","last_active":"2024-04-25T08:54:13.599726+0000","last_peered":"2024-04-25T08:54:13.599726+0000","last_clean":"2024-04-25T08:54:13.599726+0000","last_became_active":"2024-04-25T08:54:12.601348+0000","last_became_peered":"2024-04-25T08:54:12.601348+0000","last_unstale":"2024-04-25T08:54:13.599726+0000","last_undegraded":"2024-04-25T08:54:13.599726+0000","last_fullsized":"2024-04-25T08:54:13.599726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:45.793417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.599865+0000","last_change":"2024-04-25T08:54:11.592183+0000","last_active":"2024-04-25T08:54:13.599865+0000","last_peered":"2024-04-25T08:54:13.599865+0000","last_clean":"2024-04-25T08:54:13.599865+0000","last_became_active":"2024-04-25T08:54:11.591795+0000","last_became_peered":"2024-04-25T08:54:11.591795+0000","last_unstale":"2024-04-25T08:54:13.599865+0000","last_undegraded":"2024-04-25T08:54:13.599865+0000","last_fullsized":"2024-04-25T08:54:13.599865+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:38.100250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.611511+0000","last_change":"2024-04-25T08:54:12.607792+0000","last_active":"2024-04-25T08:54:12.611511+0000","last_peered":"2024-04-25T08:54:12.611511+0000","last_clean":"2024-04-25T08:54:12.611511+0000","last_became_active":"2024-04-25T08:54:12.607459+0000","last_became_peered":"2024-04-25T08:54:12.607459+0000","last_unstale":"2024-04-25T08:54:12.611511+0000","last_undegraded":"2024-04-25T08:54:12.611511+0000","last_fullsized":"2024-04-25T08:54:12.611511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:39.105098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.615708+0000","last_change":"2024-04-25T08:54:11.591567+0000","last_active":"2024-04-25T08:54:12.615708+0000","last_peered":"2024-04-25T08:54:12.615708+0000","last_clean":"2024-04-25T08:54:12.615708+0000","last_became_active":"2024-04-25T08:54:11.591178+0000","last_became_peered":"2024-04-25T08:54:11.591178+0000","last_unstale":"2024-04-25T08:54:12.615708+0000","last_undegraded":"2024-04-25T08:54:12.615708+0000","last_fullsized":"2024-04-25T08:54:12.615708+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:50.460868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.601765+0000","last_change":"2024-04-25T08:54:12.598577+0000","last_active":"2024-04-25T08:54:12.601765+0000","last_peered":"2024-04-25T08:54:12.601765+0000","last_clean":"2024-04-25T08:54:12.601765+0000","last_became_active":"2024-04-25T08:54:12.597319+0000","last_became_peered":"2024-04-25T08:54:12.597319+0000","last_unstale":"2024-04-25T08:54:12.601765+0000","last_undegraded":"2024-04-25T08:54:12.601765+0000","last_fullsized":"2024-04-25T08:54:12.601765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:11.183046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.596878+0000","last_change":"2024-04-25T08:54:11.591476+0000","last_active":"2024-04-25T08:54:12.596878+0000","last_peered":"2024-04-25T08:54:12.596878+0000","last_clean":"2024-04-25T08:54:12.596878+0000","last_became_active":"2024-04-25T08:54:11.591000+0000","last_became_peered":"2024-04-25T08:54:11.591000+0000","last_unstale":"2024-04-25T08:54:12.596878+0000","last_undegraded":"2024-04-25T08:54:12.596878+0000","last_fullsized":"2024-04-25T08:54:12.596878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:38.100250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601522+0000","last_change":"2024-04-25T08:54:12.602036+0000","last_active":"2024-04-25T08:54:13.601522+0000","last_peered":"2024-04-25T08:54:13.601522+0000","last_clean":"2024-04-25T08:54:13.601522+0000","last_became_active":"2024-04-25T08:54:12.601768+0000","last_became_peered":"2024-04-25T08:54:12.601768+0000","last_unstale":"2024-04-25T08:54:13.601522+0000","last_undegraded":"2024-04-25T08:54:13.601522+0000","last_fullsized":"2024-04-25T08:54:13.601522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:51.465717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.599648+0000","last_change":"2024-04-25T08:54:11.585465+0000","last_active":"2024-04-25T08:54:13.599648+0000","last_peered":"2024-04-25T08:54:13.599648+0000","last_clean":"2024-04-25T08:54:13.599648+0000","last_became_active":"2024-04-25T08:54:11.585184+0000","last_became_peered":"2024-04-25T08:54:11.585184+0000","last_unstale":"2024-04-25T08:54:13.599648+0000","last_undegraded":"2024-04-25T08:54:13.599648+0000","last_fullsized":"2024-04-25T08:54:13.599648+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:37.241134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.606655+0000","last_change":"2024-04-25T08:54:12.603295+0000","last_active":"2024-04-25T08:54:12.606655+0000","last_peered":"2024-04-25T08:54:12.606655+0000","last_clean":"2024-04-25T08:54:12.606655+0000","last_became_active":"2024-04-25T08:54:12.602398+0000","last_became_peered":"2024-04-25T08:54:12.602398+0000","last_unstale":"2024-04-25T08:54:12.606655+0000","last_undegraded":"2024-04-25T08:54:12.606655+0000","last_fullsized":"2024-04-25T08:54:12.606655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:11.183046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.599690+0000","last_change":"2024-04-25T08:54:11.588662+0000","last_active":"2024-04-25T08:54:13.599690+0000","last_peered":"2024-04-25T08:54:13.599690+0000","last_clean":"2024-04-25T08:54:13.599690+0000","last_became_active":"2024-04-25T08:54:11.588350+0000","last_became_peered":"2024-04-25T08:54:11.588350+0000","last_unstale":"2024-04-25T08:54:13.599690+0000","last_undegraded":"2024-04-25T08:54:13.599690+0000","last_fullsized":"2024-04-25T08:54:13.599690+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:45.942159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.616216+0000","last_change":"2024-04-25T08:54:12.612845+0000","last_active":"2024-04-25T08:54:12.616216+0000","last_peered":"2024-04-25T08:54:12.616216+0000","last_clean":"2024-04-25T08:54:12.616216+0000","last_became_active":"2024-04-25T08:54:12.612569+0000","last_became_peered":"2024-04-25T08:54:12.612569+0000","last_unstale":"2024-04-25T08:54:12.616216+0000","last_undegraded":"2024-04-25T08:54:12.616216+0000","last_fullsized":"2024-04-25T08:54:12.616216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:53.347293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.613696+0000","last_change":"2024-04-25T08:54:11.589407+0000","last_active":"2024-04-25T08:54:12.613696+0000","last_peered":"2024-04-25T08:54:12.613696+0000","last_clean":"2024-04-25T08:54:12.613696+0000","last_became_active":"2024-04-25T08:54:11.588226+0000","last_became_peered":"2024-04-25T08:54:11.588226+0000","last_unstale":"2024-04-25T08:54:12.613696+0000","last_undegraded":"2024-04-25T08:54:12.613696+0000","last_fullsized":"2024-04-25T08:54:12.613696+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:42.726856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.616154+0000","last_change":"2024-04-25T08:54:12.609741+0000","last_active":"2024-04-25T08:54:12.616154+0000","last_peered":"2024-04-25T08:54:12.616154+0000","last_clean":"2024-04-25T08:54:12.616154+0000","last_became_active":"2024-04-25T08:54:12.609075+0000","last_became_peered":"2024-04-25T08:54:12.609075+0000","last_unstale":"2024-04-25T08:54:12.616154+0000","last_undegraded":"2024-04-25T08:54:12.616154+0000","last_fullsized":"2024-04-25T08:54:12.616154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:08:53.347293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.602159+0000","last_change":"2024-04-25T08:54:12.603440+0000","last_active":"2024-04-25T08:54:13.602159+0000","last_peered":"2024-04-25T08:54:13.602159+0000","last_clean":"2024-04-25T08:54:13.602159+0000","last_became_active":"2024-04-25T08:54:12.603193+0000","last_became_peered":"2024-04-25T08:54:12.603193+0000","last_unstale":"2024-04-25T08:54:13.602159+0000","last_undegraded":"2024-04-25T08:54:13.602159+0000","last_fullsized":"2024-04-25T08:54:13.602159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:44.251355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.616507+0000","last_change":"2024-04-25T08:54:11.585161+0000","last_active":"2024-04-25T08:54:12.616507+0000","last_peered":"2024-04-25T08:54:12.616507+0000","last_clean":"2024-04-25T08:54:12.616507+0000","last_became_active":"2024-04-25T08:54:11.584842+0000","last_became_peered":"2024-04-25T08:54:11.584842+0000","last_unstale":"2024-04-25T08:54:12.616507+0000","last_undegraded":"2024-04-25T08:54:12.616507+0000","last_fullsized":"2024-04-25T08:54:12.616507+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:45.942159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601990+0000","last_change":"2024-04-25T08:54:12.611429+0000","last_active":"2024-04-25T08:54:13.601990+0000","last_peered":"2024-04-25T08:54:13.601990+0000","last_clean":"2024-04-25T08:54:13.601990+0000","last_became_active":"2024-04-25T08:54:12.611170+0000","last_became_peered":"2024-04-25T08:54:12.611170+0000","last_unstale":"2024-04-25T08:54:13.601990+0000","last_undegraded":"2024-04-25T08:54:13.601990+0000","last_fullsized":"2024-04-25T08:54:13.601990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:21.068533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.616081+0000","last_change":"2024-04-25T08:54:11.589779+0000","last_active":"2024-04-25T08:54:12.616081+0000","last_peered":"2024-04-25T08:54:12.616081+0000","last_clean":"2024-04-25T08:54:12.616081+0000","last_became_active":"2024-04-25T08:54:11.589502+0000","last_became_peered":"2024-04-25T08:54:11.589502+0000","last_unstale":"2024-04-25T08:54:12.616081+0000","last_undegraded":"2024-04-25T08:54:12.616081+0000","last_fullsized":"2024-04-25T08:54:12.616081+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:50.460868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.613568+0000","last_change":"2024-04-25T08:54:12.610745+0000","last_active":"2024-04-25T08:54:12.613568+0000","last_peered":"2024-04-25T08:54:12.613568+0000","last_clean":"2024-04-25T08:54:12.613568+0000","last_became_active":"2024-04-25T08:54:12.610366+0000","last_became_peered":"2024-04-25T08:54:12.610366+0000","last_unstale":"2024-04-25T08:54:12.613568+0000","last_undegraded":"2024-04-25T08:54:12.613568+0000","last_fullsized":"2024-04-25T08:54:12.613568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:39.105098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.614139+0000","last_change":"2024-04-25T08:54:11.586361+0000","last_active":"2024-04-25T08:54:12.614139+0000","last_peered":"2024-04-25T08:54:12.614139+0000","last_clean":"2024-04-25T08:54:12.614139+0000","last_became_active":"2024-04-25T08:54:11.586056+0000","last_became_peered":"2024-04-25T08:54:11.586056+0000","last_unstale":"2024-04-25T08:54:12.614139+0000","last_undegraded":"2024-04-25T08:54:12.614139+0000","last_fullsized":"2024-04-25T08:54:12.614139+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:12.212662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.611464+0000","last_change":"2024-04-25T08:54:12.607913+0000","last_active":"2024-04-25T08:54:12.611464+0000","last_peered":"2024-04-25T08:54:12.611464+0000","last_clean":"2024-04-25T08:54:12.611464+0000","last_became_active":"2024-04-25T08:54:12.607623+0000","last_became_peered":"2024-04-25T08:54:12.607623+0000","last_unstale":"2024-04-25T08:54:12.611464+0000","last_undegraded":"2024-04-25T08:54:12.611464+0000","last_fullsized":"2024-04-25T08:54:12.611464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:53.056423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":19,"state":"active+clean","last_fresh":"2024-04-25T08:54:12.616833+0000","last_change":"2024-04-25T08:54:11.591877+0000","last_active":"2024-04-25T08:54:12.616833+0000","last_peered":"2024-04-25T08:54:12.616833+0000","last_clean":"2024-04-25T08:54:12.616833+0000","last_became_active":"2024-04-25T08:54:11.591562+0000","last_became_peered":"2024-04-25T08:54:11.591562+0000","last_unstale":"2024-04-25T08:54:12.616833+0000","last_undegraded":"2024-04-25T08:54:12.616833+0000","last_fullsized":"2024-04-25T08:54:12.616833+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:54.860395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27792,"kb_used_data":772,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344048,"statfs":{"total":96636764160,"available":96608305152,"internally_reserved":0,"allocated":790528,"data_stored":655584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":66246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":196608,"data_stored":64724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27220,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344620,"statfs":{"total":96636764160,"available":96608890880,"internally_reserved":0,"allocated":204800,"data_stored":64760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31380,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340460,"statfs":{"total":96636764160,"available":96604631040,"internally_reserved":0,"allocated":204800,"data_stored":66246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31368,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340472,"statfs":{"total":96636764160,"available":96604643328,"internally_reserved":0,"allocated":192512,"data_stored":64634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31376,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340464,"statfs":{"total":96636764160,"available":96604635136,"internally_reserved":0,"allocated":200704,"data_stored":64794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31960,"kb_used_data":780,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339880,"statfs":{"total":96636764160,"available":96604037120,"internally_reserved":0,"allocated":798720,"data_stored":655640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:54:15.928 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T08:54:15.929 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T08:54:15.929 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T08:54:15.929 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T08:54:15.929 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-25T08:54:16.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.075+0000 7f480457e640 1 Processor -- start 2024-04-25T08:54:16.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.075+0000 7f480457e640 1 -- start start 2024-04-25T08:54:16.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.076+0000 7f480457e640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f47fc0c8d00 0x7f47fc162100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:16.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.076+0000 7f480457e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f47fc0ce660 0x7f47fc0cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:16.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.076+0000 7f480457e640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47fc0d24a0 con 0x7f47fc15df50 2024-04-25T08:54:16.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.076+0000 7f480457e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47fc15e330 con 0x7f47fc0ce660 2024-04-25T08:54:16.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.076+0000 7f480457e640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47fc0d2740 con 0x7f47fc0c8d00 2024-04-25T08:54:16.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.076+0000 7f4802b1b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f47fc0ce660 0x7f47fc0cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:16.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.076+0000 7f4802b1b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f47fc0ce660 0x7f47fc0cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37284/0 (socket says 172.21.15.43:37284) 2024-04-25T08:54:16.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.076+0000 7f4802b1b640 1 -- 172.21.15.43:0/1958836417 learned_addr learned my addr 172.21.15.43:0/1958836417 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:16.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.077+0000 7f4801b19640 1 --2- 172.21.15.43:0/1958836417 >> v2:172.21.15.193:3300/0 conn(0x7f47fc0c8d00 0x7f47fc162100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:16.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.077+0000 7f4802b1b640 1 -- 172.21.15.43:0/1958836417 >> v2:172.21.15.193:3300/0 conn(0x7f47fc0c8d00 msgr2=0x7f47fc162100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:16.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.077+0000 7f4802b1b640 1 --2- 172.21.15.43:0/1958836417 >> v2:172.21.15.193:3300/0 conn(0x7f47fc0c8d00 0x7f47fc162100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:16.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.077+0000 7f4802b1b640 1 -- 172.21.15.43:0/1958836417 >> v1:172.21.15.193:6789/0 conn(0x7f47fc15df50 legacy=0x7f47fc1619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:16.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.077+0000 7f4802b1b640 1 -- 172.21.15.43:0/1958836417 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47fc0d2a20 con 0x7f47fc0ce660 2024-04-25T08:54:16.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.077+0000 7f4801b19640 1 --2- 172.21.15.43:0/1958836417 >> v2:172.21.15.193:3300/0 conn(0x7f47fc0c8d00 0x7f47fc162100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:16.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.077+0000 7f4802b1b640 1 --2- 172.21.15.43:0/1958836417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47fc0ce660 0x7f47fc0cea70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f47fc0caa00 tx=0x7f47f402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82017e5358154de3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:16.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.077+0000 7f4801318640 1 -- 172.21.15.43:0/1958836417 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f4036070 con 0x7f47fc0ce660 2024-04-25T08:54:16.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.078+0000 7f4801318640 1 -- 172.21.15.43:0/1958836417 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f4031600 con 0x7f47fc0ce660 2024-04-25T08:54:16.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.078+0000 7f4801318640 1 -- 172.21.15.43:0/1958836417 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f40319d0 con 0x7f47fc0ce660 2024-04-25T08:54:16.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.078+0000 7f480457e640 1 -- 172.21.15.43:0/1958836417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47fc0ce660 msgr2=0x7f47fc0cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:16.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.078+0000 7f480457e640 1 --2- 172.21.15.43:0/1958836417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47fc0ce660 0x7f47fc0cea70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f47fc0caa00 tx=0x7f47f402a6c0 comp rx=0 tx=0).stop 2024-04-25T08:54:16.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.078+0000 7f480457e640 1 -- 172.21.15.43:0/1958836417 shutdown_connections 2024-04-25T08:54:16.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.078+0000 7f480457e640 1 --2- 172.21.15.43:0/1958836417 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47fc0ce660 0x7f47fc0cea70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:16.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.078+0000 7f480457e640 1 --2- 172.21.15.43:0/1958836417 >> v2:172.21.15.193:3300/0 conn(0x7f47fc0c8d00 0x7f47fc162100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:16.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.078+0000 7f480457e640 1 -- 172.21.15.43:0/1958836417 >> 172.21.15.43:0/1958836417 conn(0x7f47fc0d3a30 msgr2=0x7f47fc0d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:16.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.078+0000 7f480457e640 1 -- 172.21.15.43:0/1958836417 shutdown_connections 2024-04-25T08:54:16.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.078+0000 7f480457e640 1 -- 172.21.15.43:0/1958836417 wait complete. 2024-04-25T08:54:16.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.079+0000 7f480457e640 1 Processor -- start 2024-04-25T08:54:16.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.079+0000 7f480457e640 1 -- start start 2024-04-25T08:54:16.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.080+0000 7f480457e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47fc0c8d00 0x7f47fc14fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:16.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.080+0000 7f480457e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47fc0ce660 0x7f47fc15f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:16.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.080+0000 7f480457e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f47fc15df50 0x7f47fc14a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:16.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.080+0000 7f480457e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47fc0d2190 con 0x7f47fc0c8d00 2024-04-25T08:54:16.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.080+0000 7f480457e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47fc0c77d0 con 0x7f47fc0ce660 2024-04-25T08:54:16.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.080+0000 7f480457e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47fc162640 con 0x7f47fc15df50 2024-04-25T08:54:16.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.080+0000 7f480231a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47fc0c8d00 0x7f47fc14fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:16.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.080+0000 7f480231a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47fc0c8d00 0x7f47fc14fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37296/0 (socket says 172.21.15.43:37296) 2024-04-25T08:54:16.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.080+0000 7f480231a640 1 -- 172.21.15.43:0/2924673957 learned_addr learned my addr 172.21.15.43:0/2924673957 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:16.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.080+0000 7f4801b19640 1 --2- 172.21.15.43:0/2924673957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47fc0ce660 0x7f47fc15f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:16.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.080+0000 7f4802b1b640 1 --2- 172.21.15.43:0/2924673957 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f47fc15df50 0x7f47fc14a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:16.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.080+0000 7f480231a640 1 -- 172.21.15.43:0/2924673957 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f47fc15df50 msgr2=0x7f47fc14a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:16.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.080+0000 7f480231a640 1 --2- 172.21.15.43:0/2924673957 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f47fc15df50 0x7f47fc14a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:16.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.080+0000 7f480231a640 1 -- 172.21.15.43:0/2924673957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47fc0ce660 msgr2=0x7f47fc15f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:16.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.080+0000 7f480231a640 1 --2- 172.21.15.43:0/2924673957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47fc0ce660 0x7f47fc15f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:16.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.080+0000 7f480231a640 1 -- 172.21.15.43:0/2924673957 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47fc15e4f0 con 0x7f47fc0c8d00 2024-04-25T08:54:16.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.081+0000 7f480231a640 1 --2- 172.21.15.43:0/2924673957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47fc0c8d00 0x7f47fc14fe30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f47f800b3f0 tx=0x7f47f800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:16.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.081+0000 7f47f2ffd640 1 -- 172.21.15.43:0/2924673957 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f8015020 con 0x7f47fc0c8d00 2024-04-25T08:54:16.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.081+0000 7f47f2ffd640 1 -- 172.21.15.43:0/2924673957 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47f8011830 con 0x7f47fc0c8d00 2024-04-25T08:54:16.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.081+0000 7f480457e640 1 -- 172.21.15.43:0/2924673957 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47fc15e7d0 con 0x7f47fc0c8d00 2024-04-25T08:54:16.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.081+0000 7f47f2ffd640 1 -- 172.21.15.43:0/2924673957 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47f8011c20 con 0x7f47fc0c8d00 2024-04-25T08:54:16.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.081+0000 7f480457e640 1 -- 172.21.15.43:0/2924673957 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47fc15ed90 con 0x7f47fc0c8d00 2024-04-25T08:54:16.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.082+0000 7f480457e640 1 -- 172.21.15.43:0/2924673957 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47fc0680f0 con 0x7f47fc0c8d00 2024-04-25T08:54:16.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.082+0000 7f47f2ffd640 1 -- 172.21.15.43:0/2924673957 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f47f8018020 con 0x7f47fc0c8d00 2024-04-25T08:54:16.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.083+0000 7f47f2ffd640 1 --2- 172.21.15.43:0/2924673957 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f47e006b4d0 0x7f47e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:16.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.083+0000 7f47f2ffd640 1 -- 172.21.15.43:0/2924673957 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f47f8089ce0 con 0x7f47fc0c8d00 2024-04-25T08:54:16.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.083+0000 7f4801b19640 1 --2- 172.21.15.43:0/2924673957 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f47e006b4d0 0x7f47e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:16.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.083+0000 7f4801b19640 1 --2- 172.21.15.43:0/2924673957 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f47e006b4d0 0x7f47e006d990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f47fc0c87f0 tx=0x7f47ec008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:16.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.087+0000 7f47f2ffd640 1 -- 172.21.15.43:0/2924673957 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f47f801f030 con 0x7f47fc0c8d00 2024-04-25T08:54:16.248 INFO:tasks.ceph.mds.d.smithi193.stderr:2024-04-25T08:54:16.258+0000 7ff1f5c30640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:54:16.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.315+0000 7f480457e640 1 -- 172.21.15.43:0/2924673957 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f47fc0ca400 con 0x7f47fc0c8d00 2024-04-25T08:54:16.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.318+0000 7f47f2ffd640 1 -- 172.21.15.43:0/2924673957 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171791 (secure 0 0 0) 0x7f47f8058510 con 0x7f47fc0c8d00 2024-04-25T08:54:16.310 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:54:16.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.321+0000 7f480457e640 1 -- 172.21.15.43:0/2924673957 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f47e006b4d0 msgr2=0x7f47e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:16.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.321+0000 7f480457e640 1 --2- 172.21.15.43:0/2924673957 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f47e006b4d0 0x7f47e006d990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f47fc0c87f0 tx=0x7f47ec008040 comp rx=0 tx=0).stop 2024-04-25T08:54:16.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.321+0000 7f480457e640 1 -- 172.21.15.43:0/2924673957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47fc0c8d00 msgr2=0x7f47fc14fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:16.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.321+0000 7f480457e640 1 --2- 172.21.15.43:0/2924673957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47fc0c8d00 0x7f47fc14fe30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f47f800b3f0 tx=0x7f47f800b8b0 comp rx=0 tx=0).stop 2024-04-25T08:54:16.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.321+0000 7f480457e640 1 -- 172.21.15.43:0/2924673957 shutdown_connections 2024-04-25T08:54:16.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.321+0000 7f480457e640 1 --2- 172.21.15.43:0/2924673957 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f47fc15df50 0x7f47fc14a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:16.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.321+0000 7f480457e640 1 --2- 172.21.15.43:0/2924673957 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f47e006b4d0 0x7f47e006d990 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:16.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.321+0000 7f480457e640 1 --2- 172.21.15.43:0/2924673957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47fc0ce660 0x7f47fc15f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:16.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.321+0000 7f480457e640 1 --2- 172.21.15.43:0/2924673957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47fc0c8d00 0x7f47fc14fe30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:16.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.321+0000 7f480457e640 1 -- 172.21.15.43:0/2924673957 >> 172.21.15.43:0/2924673957 conn(0x7f47fc0d3a30 msgr2=0x7f47fc15a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:16.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.321+0000 7f480457e640 1 -- 172.21.15.43:0/2924673957 shutdown_connections 2024-04-25T08:54:16.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.321+0000 7f480457e640 1 -- 172.21.15.43:0/2924673957 wait complete. 2024-04-25T08:54:16.321 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":5,"flags":0,"active_gid":4108,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6832","nonce":1244825175},{"type":"v1","addr":"172.21.15.193:6833","nonce":1244825175}]},"active_addr":"172.21.15.193:6833/1244825175","active_change":"2024-04-25T08:53:51.683999+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4107,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.193:0","nonce":4239924217}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.193:0","nonce":738577069}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.193:0","nonce":3565412970}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.193:0","nonce":1554274444}]}]} 2024-04-25T08:54:16.325 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T08:54:16.325 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T08:54:16.325 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:54:16.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.435+0000 7f97fc8dd640 1 Processor -- start 2024-04-25T08:54:16.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.435+0000 7f97fc8dd640 1 -- start start 2024-04-25T08:54:16.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.436+0000 7f97fc8dd640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f97f80b3ea0 0x7f97f806d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:16.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.436+0000 7f97fc8dd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f97f806f930 0x7f97f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:16.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.436+0000 7f97fc8dd640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f806f4c0 con 0x7f97f806f0e0 2024-04-25T08:54:16.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.436+0000 7f97fc8dd640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97f8072130 con 0x7f97f806f930 2024-04-25T08:54:16.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.436+0000 7f97fc8dd640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97f80b3360 con 0x7f97f80b3ea0 2024-04-25T08:54:16.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.436+0000 7f97f6d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f97f806f930 0x7f97f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:16.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.436+0000 7f97f6d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f97f806f930 0x7f97f806d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37304/0 (socket says 172.21.15.43:37304) 2024-04-25T08:54:16.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.436+0000 7f97f6d9d640 1 -- 172.21.15.43:0/3204500905 learned_addr learned my addr 172.21.15.43:0/3204500905 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:16.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.437+0000 7f97f5d9b640 1 --2- 172.21.15.43:0/3204500905 >> v2:172.21.15.193:3300/0 conn(0x7f97f80b3ea0 0x7f97f806d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:16.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.437+0000 7f97f6d9d640 1 -- 172.21.15.43:0/3204500905 >> v2:172.21.15.193:3300/0 conn(0x7f97f80b3ea0 msgr2=0x7f97f806d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:16.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.437+0000 7f97f6d9d640 1 --2- 172.21.15.43:0/3204500905 >> v2:172.21.15.193:3300/0 conn(0x7f97f80b3ea0 0x7f97f806d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:16.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.437+0000 7f97f6d9d640 1 -- 172.21.15.43:0/3204500905 >> v1:172.21.15.193:6789/0 conn(0x7f97f806f0e0 legacy=0x7f97f80b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:16.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.437+0000 7f97f6d9d640 1 -- 172.21.15.43:0/3204500905 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f8071520 con 0x7f97f806f930 2024-04-25T08:54:16.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.437+0000 7f97f5d9b640 1 --2- 172.21.15.43:0/3204500905 >> v2:172.21.15.193:3300/0 conn(0x7f97f80b3ea0 0x7f97f806d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:16.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.437+0000 7f97f6d9d640 1 --2- 172.21.15.43:0/3204500905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97f806f930 0x7f97f806d990 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f97f80722b0 tx=0x7f97e802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb1e408a91c95b41 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:16.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.438+0000 7f97f559a640 1 -- 172.21.15.43:0/3204500905 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e8036070 con 0x7f97f806f930 2024-04-25T08:54:16.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.438+0000 7f97f559a640 1 -- 172.21.15.43:0/3204500905 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e8031600 con 0x7f97f806f930 2024-04-25T08:54:16.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.438+0000 7f97f559a640 1 -- 172.21.15.43:0/3204500905 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e80319d0 con 0x7f97f806f930 2024-04-25T08:54:16.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.438+0000 7f97fc8dd640 1 -- 172.21.15.43:0/3204500905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97f806f930 msgr2=0x7f97f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:16.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.438+0000 7f97fc8dd640 1 --2- 172.21.15.43:0/3204500905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97f806f930 0x7f97f806d990 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f97f80722b0 tx=0x7f97e802a6a0 comp rx=0 tx=0).stop 2024-04-25T08:54:16.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.438+0000 7f97fc8dd640 1 -- 172.21.15.43:0/3204500905 shutdown_connections 2024-04-25T08:54:16.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.438+0000 7f97fc8dd640 1 --2- 172.21.15.43:0/3204500905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97f806f930 0x7f97f806d990 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:16.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.438+0000 7f97fc8dd640 1 --2- 172.21.15.43:0/3204500905 >> v2:172.21.15.193:3300/0 conn(0x7f97f80b3ea0 0x7f97f806d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:16.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.438+0000 7f97fc8dd640 1 -- 172.21.15.43:0/3204500905 >> 172.21.15.43:0/3204500905 conn(0x7f97f80b2490 msgr2=0x7f97f80b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:16.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.438+0000 7f97fc8dd640 1 -- 172.21.15.43:0/3204500905 shutdown_connections 2024-04-25T08:54:16.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.438+0000 7f97fc8dd640 1 -- 172.21.15.43:0/3204500905 wait complete. 2024-04-25T08:54:16.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.439+0000 7f97fc8dd640 1 Processor -- start 2024-04-25T08:54:16.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.440+0000 7f97fc8dd640 1 -- start start 2024-04-25T08:54:16.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.440+0000 7f97fc8dd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97f806f0e0 0x7f97f8158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:16.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.440+0000 7f97fc8dd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97f806f930 0x7f97f8152b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:16.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.440+0000 7f97fc8dd640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97f80b3ea0 0x7f97f8154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:16.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.440+0000 7f97fc8dd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f8071dc0 con 0x7f97f806f930 2024-04-25T08:54:16.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.440+0000 7f97fc8dd640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f8071ae0 con 0x7f97f806f0e0 2024-04-25T08:54:16.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.440+0000 7f97fc8dd640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f815aee0 con 0x7f97f80b3ea0 2024-04-25T08:54:16.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.440+0000 7f97f5d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97f806f930 0x7f97f8152b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:16.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.440+0000 7f97f5d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97f806f930 0x7f97f8152b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37316/0 (socket says 172.21.15.43:37316) 2024-04-25T08:54:16.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.440+0000 7f97f5d9b640 1 -- 172.21.15.43:0/2616335289 learned_addr learned my addr 172.21.15.43:0/2616335289 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:16.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.441+0000 7f97f659c640 1 --2- 172.21.15.43:0/2616335289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97f806f0e0 0x7f97f8158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:16.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.441+0000 7f97f6d9d640 1 --2- 172.21.15.43:0/2616335289 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97f80b3ea0 0x7f97f8154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:16.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.441+0000 7f97f5d9b640 1 -- 172.21.15.43:0/2616335289 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97f80b3ea0 msgr2=0x7f97f8154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:16.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.441+0000 7f97f5d9b640 1 --2- 172.21.15.43:0/2616335289 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97f80b3ea0 0x7f97f8154060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:16.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.441+0000 7f97f5d9b640 1 -- 172.21.15.43:0/2616335289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97f806f0e0 msgr2=0x7f97f8158120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:16.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.441+0000 7f97f5d9b640 1 --2- 172.21.15.43:0/2616335289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97f806f0e0 0x7f97f8158120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:16.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.441+0000 7f97f5d9b640 1 -- 172.21.15.43:0/2616335289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f81534c0 con 0x7f97f806f930 2024-04-25T08:54:16.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.441+0000 7f97f6d9d640 1 --2- 172.21.15.43:0/2616335289 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97f80b3ea0 0x7f97f8154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:16.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.441+0000 7f97f5d9b640 1 --2- 172.21.15.43:0/2616335289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97f806f930 0x7f97f8152b00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f97ec00d7d0 tx=0x7f97ec00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:16.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.441+0000 7f97e6ffd640 1 -- 172.21.15.43:0/2616335289 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ec004030 con 0x7f97f806f930 2024-04-25T08:54:16.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.441+0000 7f97e6ffd640 1 -- 172.21.15.43:0/2616335289 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97ec0041d0 con 0x7f97f806f930 2024-04-25T08:54:16.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.441+0000 7f97fc8dd640 1 -- 172.21.15.43:0/2616335289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97f8155750 con 0x7f97f806f930 2024-04-25T08:54:16.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.441+0000 7f97e6ffd640 1 -- 172.21.15.43:0/2616335289 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ec00b6c0 con 0x7f97f806f930 2024-04-25T08:54:16.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.441+0000 7f97fc8dd640 1 -- 172.21.15.43:0/2616335289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f81559f0 con 0x7f97f806f930 2024-04-25T08:54:16.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.442+0000 7f97fc8dd640 1 -- 172.21.15.43:0/2616335289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97f80af9a0 con 0x7f97f806f930 2024-04-25T08:54:16.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.442+0000 7f97e6ffd640 1 -- 172.21.15.43:0/2616335289 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f97ec00b860 con 0x7f97f806f930 2024-04-25T08:54:16.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.443+0000 7f97e6ffd640 1 --2- 172.21.15.43:0/2616335289 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f97d806b4d0 0x7f97d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:16.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.443+0000 7f97e6ffd640 1 -- 172.21.15.43:0/2616335289 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f97ec08a450 con 0x7f97f806f930 2024-04-25T08:54:16.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.443+0000 7f97f659c640 1 --2- 172.21.15.43:0/2616335289 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f97d806b4d0 0x7f97d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:16.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.443+0000 7f97f659c640 1 --2- 172.21.15.43:0/2616335289 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f97d806b4d0 0x7f97d806d990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f97f0007fd0 tx=0x7f97f0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:16.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.447+0000 7f97e6ffd640 1 -- 172.21.15.43:0/2616335289 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f97f80af9a0 con 0x7f97f806f930 2024-04-25T08:54:16.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.632+0000 7f97fc8dd640 1 -- 172.21.15.43:0/2616335289 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97f815ab50 con 0x7f97f806f930 2024-04-25T08:54:16.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.634+0000 7f97e6ffd640 1 -- 172.21.15.43:0/2616335289 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16569 (secure 0 0 0) 0x7f97f815ab50 con 0x7f97f806f930 2024-04-25T08:54:16.624 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:54:16.624 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":20,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:54:13.593188+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:54:09.757978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:54:10.926008+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:54:16.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.635+0000 7f97fc8dd640 1 -- 172.21.15.43:0/2616335289 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f97d806b4d0 msgr2=0x7f97d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:16.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.636+0000 7f97fc8dd640 1 --2- 172.21.15.43:0/2616335289 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f97d806b4d0 0x7f97d806d990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f97f0007fd0 tx=0x7f97f0009040 comp rx=0 tx=0).stop 2024-04-25T08:54:16.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.636+0000 7f97fc8dd640 1 -- 172.21.15.43:0/2616335289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97f806f930 msgr2=0x7f97f8152b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:16.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.636+0000 7f97fc8dd640 1 --2- 172.21.15.43:0/2616335289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97f806f930 0x7f97f8152b00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f97ec00d7d0 tx=0x7f97ec00dc90 comp rx=0 tx=0).stop 2024-04-25T08:54:16.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.636+0000 7f97fc8dd640 1 -- 172.21.15.43:0/2616335289 shutdown_connections 2024-04-25T08:54:16.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.636+0000 7f97fc8dd640 1 --2- 172.21.15.43:0/2616335289 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97f80b3ea0 0x7f97f8154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:16.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.636+0000 7f97fc8dd640 1 --2- 172.21.15.43:0/2616335289 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97f806f930 0x7f97f8152b00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:16.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.636+0000 7f97fc8dd640 1 --2- 172.21.15.43:0/2616335289 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f97d806b4d0 0x7f97d806d990 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:16.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.636+0000 7f97fc8dd640 1 --2- 172.21.15.43:0/2616335289 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97f806f0e0 0x7f97f8158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:16.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.636+0000 7f97fc8dd640 1 -- 172.21.15.43:0/2616335289 >> 172.21.15.43:0/2616335289 conn(0x7f97f80b2490 msgr2=0x7f97f8073b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:16.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.636+0000 7f97fc8dd640 1 -- 172.21.15.43:0/2616335289 shutdown_connections 2024-04-25T08:54:16.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.636+0000 7f97fc8dd640 1 -- 172.21.15.43:0/2616335289 wait complete. 2024-04-25T08:54:16.637 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T08:54:16.637 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:54:16.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.747+0000 7f00d243a640 1 Processor -- start 2024-04-25T08:54:16.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.747+0000 7f00d243a640 1 -- start start 2024-04-25T08:54:16.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.748+0000 7f00d243a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f00cc15ddc0 0x7f00cc0b2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:16.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.748+0000 7f00d243a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f00cc161920 0x7f00cc161d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:16.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.748+0000 7f00d243a640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00cc162750 con 0x7f00cc1622c0 2024-04-25T08:54:16.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.748+0000 7f00d243a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00cc0ce4d0 con 0x7f00cc161920 2024-04-25T08:54:16.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.748+0000 7f00d243a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00cc0ce7b0 con 0x7f00cc15ddc0 2024-04-25T08:54:16.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.748+0000 7f00cb7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f00cc161920 0x7f00cc161d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:16.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.748+0000 7f00cb7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f00cc161920 0x7f00cc161d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37320/0 (socket says 172.21.15.43:37320) 2024-04-25T08:54:16.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.748+0000 7f00cb7fe640 1 -- 172.21.15.43:0/4209057555 learned_addr learned my addr 172.21.15.43:0/4209057555 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:16.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.748+0000 7f00cbfff640 1 --2- 172.21.15.43:0/4209057555 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f00cc15ddc0 0x7f00cc0b2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:16.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.748+0000 7f00cb7fe640 1 -- 172.21.15.43:0/4209057555 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f00cc15ddc0 msgr2=0x7f00cc0b2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:16.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.748+0000 7f00cb7fe640 1 --2- 172.21.15.43:0/4209057555 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f00cc15ddc0 0x7f00cc0b2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:16.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.748+0000 7f00cb7fe640 1 -- 172.21.15.43:0/4209057555 >> v1:172.21.15.193:6789/0 conn(0x7f00cc1622c0 legacy=0x7f00cc0d0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:16.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.748+0000 7f00cb7fe640 1 -- 172.21.15.43:0/4209057555 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00cc0ceb40 con 0x7f00cc161920 2024-04-25T08:54:16.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.749+0000 7f00cb7fe640 1 --2- 172.21.15.43:0/4209057555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00cc161920 0x7f00cc161d80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f00bc009870 tx=0x7f00bc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c38e16c03cd34e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:16.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.749+0000 7f00caffd640 1 -- 172.21.15.43:0/4209057555 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc037070 con 0x7f00cc161920 2024-04-25T08:54:16.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.749+0000 7f00caffd640 1 -- 172.21.15.43:0/4209057555 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00bc032440 con 0x7f00cc161920 2024-04-25T08:54:16.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.749+0000 7f00caffd640 1 -- 172.21.15.43:0/4209057555 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc032810 con 0x7f00cc161920 2024-04-25T08:54:16.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.750+0000 7f00d243a640 1 -- 172.21.15.43:0/4209057555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00cc161920 msgr2=0x7f00cc161d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:16.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.750+0000 7f00d243a640 1 --2- 172.21.15.43:0/4209057555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00cc161920 0x7f00cc161d80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f00bc009870 tx=0x7f00bc02a210 comp rx=0 tx=0).stop 2024-04-25T08:54:16.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.750+0000 7f00d243a640 1 -- 172.21.15.43:0/4209057555 shutdown_connections 2024-04-25T08:54:16.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.750+0000 7f00d243a640 1 --2- 172.21.15.43:0/4209057555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00cc161920 0x7f00cc161d80 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:16.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.750+0000 7f00d243a640 1 --2- 172.21.15.43:0/4209057555 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f00cc15ddc0 0x7f00cc0b2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:16.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.750+0000 7f00d243a640 1 -- 172.21.15.43:0/4209057555 >> 172.21.15.43:0/4209057555 conn(0x7f00cc06b000 msgr2=0x7f00cc06b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:16.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.750+0000 7f00d243a640 1 -- 172.21.15.43:0/4209057555 shutdown_connections 2024-04-25T08:54:16.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.750+0000 7f00d243a640 1 -- 172.21.15.43:0/4209057555 wait complete. 2024-04-25T08:54:16.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.751+0000 7f00d243a640 1 Processor -- start 2024-04-25T08:54:16.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.751+0000 7f00d243a640 1 -- start start 2024-04-25T08:54:16.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.752+0000 7f00d243a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00cc15ddc0 0x7f00cc14fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:16.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.752+0000 7f00d243a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f00cc161920 0x7f00cc0cac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:16.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.752+0000 7f00d243a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00cc1622c0 0x7f00cc15f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:16.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.752+0000 7f00d243a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00cc0b8920 con 0x7f00cc1622c0 2024-04-25T08:54:16.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.752+0000 7f00d243a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00cc15e1a0 con 0x7f00cc15ddc0 2024-04-25T08:54:16.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.752+0000 7f00d243a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00cc0b8aa0 con 0x7f00cc161920 2024-04-25T08:54:16.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.752+0000 7f00d09d7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00cc1622c0 0x7f00cc15f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:16.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.752+0000 7f00d09d7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00cc1622c0 0x7f00cc15f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37326/0 (socket says 172.21.15.43:37326) 2024-04-25T08:54:16.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.752+0000 7f00d09d7640 1 -- 172.21.15.43:0/3920725106 learned_addr learned my addr 172.21.15.43:0/3920725106 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:16.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.752+0000 7f00cbfff640 1 --2- 172.21.15.43:0/3920725106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00cc15ddc0 0x7f00cc14fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:16.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.752+0000 7f00cb7fe640 1 --2- 172.21.15.43:0/3920725106 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f00cc161920 0x7f00cc0cac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:16.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.752+0000 7f00d09d7640 1 -- 172.21.15.43:0/3920725106 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f00cc161920 msgr2=0x7f00cc0cac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:16.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.752+0000 7f00d09d7640 1 --2- 172.21.15.43:0/3920725106 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f00cc161920 0x7f00cc0cac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:16.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.752+0000 7f00d09d7640 1 -- 172.21.15.43:0/3920725106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00cc15ddc0 msgr2=0x7f00cc14fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:16.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.752+0000 7f00d09d7640 1 --2- 172.21.15.43:0/3920725106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00cc15ddc0 0x7f00cc14fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:16.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.752+0000 7f00d09d7640 1 -- 172.21.15.43:0/3920725106 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00cc160b40 con 0x7f00cc1622c0 2024-04-25T08:54:16.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.752+0000 7f00cb7fe640 1 --2- 172.21.15.43:0/3920725106 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f00cc161920 0x7f00cc0cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:16.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.753+0000 7f00d09d7640 1 --2- 172.21.15.43:0/3920725106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00cc1622c0 0x7f00cc15f910 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f00c400b970 tx=0x7f00c400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:16.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.753+0000 7f00c8ff9640 1 -- 172.21.15.43:0/3920725106 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c4019070 con 0x7f00cc1622c0 2024-04-25T08:54:16.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.753+0000 7f00c8ff9640 1 -- 172.21.15.43:0/3920725106 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c400cb40 con 0x7f00cc1622c0 2024-04-25T08:54:16.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.753+0000 7f00d243a640 1 -- 172.21.15.43:0/3920725106 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00cc160e20 con 0x7f00cc1622c0 2024-04-25T08:54:16.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.753+0000 7f00c8ff9640 1 -- 172.21.15.43:0/3920725106 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c4014440 con 0x7f00cc1622c0 2024-04-25T08:54:16.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.753+0000 7f00d243a640 1 -- 172.21.15.43:0/3920725106 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00cc1e7890 con 0x7f00cc1622c0 2024-04-25T08:54:16.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.754+0000 7f00d243a640 1 -- 172.21.15.43:0/3920725106 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00cc0680f0 con 0x7f00cc1622c0 2024-04-25T08:54:16.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.754+0000 7f00c8ff9640 1 -- 172.21.15.43:0/3920725106 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f00c401c070 con 0x7f00cc1622c0 2024-04-25T08:54:16.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.755+0000 7f00c8ff9640 1 --2- 172.21.15.43:0/3920725106 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f00ac06b4d0 0x7f00ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:16.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.755+0000 7f00c8ff9640 1 -- 172.21.15.43:0/3920725106 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f00c4089d70 con 0x7f00cc1622c0 2024-04-25T08:54:16.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.755+0000 7f00cbfff640 1 --2- 172.21.15.43:0/3920725106 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f00ac06b4d0 0x7f00ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:16.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.755+0000 7f00cbfff640 1 --2- 172.21.15.43:0/3920725106 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f00ac06b4d0 0x7f00ac06d990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f00b4006fd0 tx=0x7f00b4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:16.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.759+0000 7f00c8ff9640 1 -- 172.21.15.43:0/3920725106 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f00c40585a0 con 0x7f00cc1622c0 2024-04-25T08:54:16.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.941+0000 7f00d243a640 1 -- 172.21.15.43:0/3920725106 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f00cc074ef0 con 0x7f00cc1622c0 2024-04-25T08:54:16.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.942+0000 7f00c8ff9640 1 -- 172.21.15.43:0/3920725106 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16569 (secure 0 0 0) 0x7f00c405c640 con 0x7f00cc1622c0 2024-04-25T08:54:16.932 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:54:16.932 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":20,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:54:13.593188+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:54:09.757978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:54:10.926008+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:54:16.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.943+0000 7f00d243a640 1 -- 172.21.15.43:0/3920725106 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f00ac06b4d0 msgr2=0x7f00ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:16.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.944+0000 7f00d243a640 1 --2- 172.21.15.43:0/3920725106 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f00ac06b4d0 0x7f00ac06d990 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f00b4006fd0 tx=0x7f00b4008040 comp rx=0 tx=0).stop 2024-04-25T08:54:16.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.944+0000 7f00d243a640 1 -- 172.21.15.43:0/3920725106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00cc1622c0 msgr2=0x7f00cc15f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:16.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.944+0000 7f00d243a640 1 --2- 172.21.15.43:0/3920725106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00cc1622c0 0x7f00cc15f910 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f00c400b970 tx=0x7f00c400be30 comp rx=0 tx=0).stop 2024-04-25T08:54:16.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.944+0000 7f00d243a640 1 -- 172.21.15.43:0/3920725106 shutdown_connections 2024-04-25T08:54:16.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.944+0000 7f00d243a640 1 --2- 172.21.15.43:0/3920725106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f00cc1622c0 0x7f00cc15f910 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:16.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.944+0000 7f00d243a640 1 --2- 172.21.15.43:0/3920725106 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f00cc161920 0x7f00cc0cac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:16.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.944+0000 7f00d243a640 1 --2- 172.21.15.43:0/3920725106 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f00ac06b4d0 0x7f00ac06d990 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:16.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.944+0000 7f00d243a640 1 --2- 172.21.15.43:0/3920725106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f00cc15ddc0 0x7f00cc14fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:16.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.944+0000 7f00d243a640 1 -- 172.21.15.43:0/3920725106 >> 172.21.15.43:0/3920725106 conn(0x7f00cc06b000 msgr2=0x7f00cc06b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:16.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.944+0000 7f00d243a640 1 -- 172.21.15.43:0/3920725106 shutdown_connections 2024-04-25T08:54:16.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:16.944+0000 7f00d243a640 1 -- 172.21.15.43:0/3920725106 wait complete. 2024-04-25T08:54:16.945 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T08:54:16.945 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T08:54:16.946 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T08:54:16.947 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T08:54:16.947 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T08:54:16.948 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T08:54:16.948 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T08:54:16.948 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T08:54:17.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.133+0000 7fef7721f640 1 Processor -- start 2024-04-25T08:54:17.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.133+0000 7fef7721f640 1 -- start start 2024-04-25T08:54:17.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.133+0000 7f041c862640 1 Processor -- start 2024-04-25T08:54:17.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.134+0000 7f041c862640 1 -- start start 2024-04-25T08:54:17.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.134+0000 7f2556209640 1 Processor -- start 2024-04-25T08:54:17.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.134+0000 7fc8fe12e640 1 Processor -- start 2024-04-25T08:54:17.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.135+0000 7fef7721f640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fef700ae730 0x7fef700aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.135+0000 7fef7721f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef7015b220 0x7fef700b3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.135+0000 7fef7721f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fef700b3bd0 0x7fef70070d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.135+0000 7fef6ffff640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef7015b220 0x7fef700b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.135+0000 7fef6ffff640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef7015b220 0x7fef700b3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:48834/0 (socket says 172.21.15.43:48834) 2024-04-25T08:54:17.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.135+0000 7fef74fbb640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fef700ae730 0x7fef700aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.135+0000 7f041c862640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0410050dd0 0x7f0410038630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.135+0000 7f041c862640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f041003a5d0 0x7f0410005f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.135+0000 7f041c862640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f041004f540 con 0x7f04100e1090 2024-04-25T08:54:17.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.135+0000 7f041c862640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f041004f820 con 0x7f041003a5d0 2024-04-25T08:54:17.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.135+0000 7f041c862640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f041004fb00 con 0x7f0410050dd0 2024-04-25T08:54:17.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.134+0000 7f04deab1640 1 Processor -- start 2024-04-25T08:54:17.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.134+0000 7f04deab1640 1 -- start start 2024-04-25T08:54:17.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.136+0000 7fef74fbb640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fef700ae730 0x7fef700aeb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:46748/0 (socket says 172.21.15.43:46748) 2024-04-25T08:54:17.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.136+0000 7fef6ffff640 1 -- 172.21.15.43:0/2441240994 learned_addr learned my addr 172.21.15.43:0/2441240994 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:17.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.136+0000 7fef7721f640 1 -- 172.21.15.43:0/2441240994 --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef700b4080 con 0x7fef700b3bd0 2024-04-25T08:54:17.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.136+0000 7f0416ffd640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0410050dd0 0x7f0410038630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.136+0000 7f0416ffd640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0410050dd0 0x7f0410038630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:48832/0 (socket says 172.21.15.43:48832) 2024-04-25T08:54:17.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.136+0000 7f0416ffd640 1 -- 172.21.15.43:0/2475971533 learned_addr learned my addr 172.21.15.43:0/2475971533 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:17.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.136+0000 7fef7721f640 1 -- 172.21.15.43:0/2441240994 --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef700af050 con 0x7fef700ae730 2024-04-25T08:54:17.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.136+0000 7fef7721f640 1 -- 172.21.15.43:0/2441240994 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef700b4310 con 0x7fef7015b220 2024-04-25T08:54:17.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.135+0000 7fc8fe12e640 1 -- start start 2024-04-25T08:54:17.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.134+0000 7f2556209640 1 -- start start 2024-04-25T08:54:17.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.136+0000 7f04deab1640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f04d0049fb0 0x7f04d0051850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.136+0000 7f04deab1640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f04d004a390 0x7f04d00524a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.136+0000 7f04deab1640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04d00545c0 con 0x7f04d0057480 2024-04-25T08:54:17.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.136+0000 7f04deab1640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04d00513c0 con 0x7f04d0049fb0 2024-04-25T08:54:17.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.136+0000 7f04deab1640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04d0007c60 con 0x7f04d004a390 2024-04-25T08:54:17.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.136+0000 7f34d382a640 1 Processor -- start 2024-04-25T08:54:17.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.136+0000 7f04167fc640 1 -- 172.21.15.43:0/2475971533 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 974789355 0 0) 0x7f041004f540 con 0x7f04100e1090 2024-04-25T08:54:17.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.136+0000 7f34d382a640 1 -- start start 2024-04-25T08:54:17.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.136+0000 7f04dd2ae640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f04d0057480 0x7f04d0051d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60850/0 (socket says 172.21.15.43:60850) 2024-04-25T08:54:17.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.136+0000 7f04dd2ae640 1 -- 172.21.15.43:0/167543697 learned_addr learned my addr 172.21.15.43:0/167543697 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:17.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.136+0000 7f04167fc640 1 -- 172.21.15.43:0/2475971533 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0408003520 con 0x7f04100e1090 2024-04-25T08:54:17.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.135+0000 7f2556209640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f254803c0a0 0x7f25480e7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.135+0000 7f2556209640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f2548053cb0 0x7f2548054120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.136+0000 7f2556209640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2548054660 con 0x7f25480e3160 2024-04-25T08:54:17.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.136+0000 7f2556209640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2548007b70 con 0x7f2548053cb0 2024-04-25T08:54:17.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.136+0000 7f2556209640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2548058810 con 0x7f254803c0a0 2024-04-25T08:54:17.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.137+0000 7f2555207640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f25480e3160 0x7f2548058050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:36348/0 (socket says 172.21.15.43:36348) 2024-04-25T08:54:17.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.137+0000 7f2555207640 1 -- 172.21.15.43:0/2387928210 learned_addr learned my addr 172.21.15.43:0/2387928210 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:17.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.137+0000 7f2554a06640 1 --2- 172.21.15.43:0/2387928210 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f254803c0a0 0x7f25480e7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.137+0000 7f2555a08640 1 --2- 172.21.15.43:0/2387928210 >> v2:172.21.15.193:3300/0 conn(0x7f2548053cb0 0x7f2548054120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.136+0000 7f08f19fb640 1 Processor -- start 2024-04-25T08:54:17.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.137+0000 7f08f19fb640 1 -- start start 2024-04-25T08:54:17.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.137+0000 7fc8fe12e640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fc8f81435f0 0x7fc8f8071920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.137+0000 7fc8fe12e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f806ac60 con 0x7fc8f80b3720 2024-04-25T08:54:17.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.137+0000 7fc8fd12c640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fc8f81435f0 0x7fc8f8071920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.137+0000 7fc8fd12c640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fc8f81435f0 0x7fc8f8071920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:46786/0 (socket says 172.21.15.43:46786) 2024-04-25T08:54:17.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.137+0000 7fc8fd12c640 1 -- 172.21.15.43:0/2363522955 learned_addr learned my addr 172.21.15.43:0/2363522955 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:17.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.137+0000 7fc8fe12e640 1 -- 172.21.15.43:0/2363522955 --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8f806af40 con 0x7fc8f80b42c0 2024-04-25T08:54:17.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.137+0000 7fc8fe12e640 1 -- 172.21.15.43:0/2363522955 --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8f806b590 con 0x7fc8f81435f0 2024-04-25T08:54:17.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.136+0000 7fef6ffff640 1 -- 172.21.15.43:0/2441240994 >> v2:172.21.15.193:3300/0 conn(0x7fef700ae730 msgr2=0x7fef700aeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:17.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.136+0000 7fef6ffff640 1 --2- 172.21.15.43:0/2441240994 >> v2:172.21.15.193:3300/0 conn(0x7fef700ae730 0x7fef700aeb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.136+0000 7fef6ffff640 1 -- 172.21.15.43:0/2441240994 >> v2:172.21.15.43:3300/0 conn(0x7fef700b3bd0 msgr2=0x7fef70070d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:17.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.136+0000 7fef6ffff640 1 --2- 172.21.15.43:0/2441240994 >> v2:172.21.15.43:3300/0 conn(0x7fef700b3bd0 0x7fef70070d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.136+0000 7fef6ffff640 1 -- 172.21.15.43:0/2441240994 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef700b45f0 con 0x7fef7015b220 2024-04-25T08:54:17.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.137+0000 7f04167fc640 1 -- 172.21.15.43:0/2475971533 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3756778766 0 0) 0x7f0408003520 con 0x7f04100e1090 2024-04-25T08:54:17.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.137+0000 7f04167fc640 1 -- 172.21.15.43:0/2475971533 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0410050dd0 msgr2=0x7f0410038630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:17.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.137+0000 7f04167fc640 1 --2- 172.21.15.43:0/2475971533 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0410050dd0 0x7f0410038630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.137+0000 7f04167fc640 1 -- 172.21.15.43:0/2475971533 >> v2:172.21.15.193:3300/0 conn(0x7f041003a5d0 msgr2=0x7f0410005f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:17.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.137+0000 7f0417fff640 1 --2- 172.21.15.43:0/2475971533 >> v2:172.21.15.193:3300/0 conn(0x7f041003a5d0 0x7f0410005f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.137+0000 7f04167fc640 1 --2- 172.21.15.43:0/2475971533 >> v2:172.21.15.193:3300/0 conn(0x7f041003a5d0 0x7f0410005f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.136+0000 7f04ddaaf640 1 --2- 172.21.15.43:0/167543697 >> v2:172.21.15.193:3300/0 conn(0x7f04d0049fb0 0x7f04d0051850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.136+0000 7f04de2b0640 1 --2- 172.21.15.43:0/167543697 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f04d004a390 0x7f04d00524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.137+0000 7f04dcaad640 1 -- 172.21.15.43:0/167543697 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1250408867 0 0) 0x7f04d00545c0 con 0x7f04d0057480 2024-04-25T08:54:17.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.137+0000 7f04dcaad640 1 -- 172.21.15.43:0/167543697 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04c4003520 con 0x7f04d0057480 2024-04-25T08:54:17.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.137+0000 7f04167fc640 1 -- 172.21.15.43:0/2475971533 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f041004fde0 con 0x7f04100e1090 2024-04-25T08:54:17.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.138+0000 7fc8fd12c640 1 -- 172.21.15.43:0/2363522955 >> v1:172.21.15.193:6789/0 conn(0x7fc8f80b42c0 legacy=0x7fc8f815a440 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T08:54:17.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.138+0000 7fc8fd12c640 1 -- 172.21.15.43:0/2363522955 >> v1:172.21.15.43:6789/0 conn(0x7fc8f80b3720 legacy=0x7fc8f806ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.138+0000 7f34d382a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f34cc15bb80 0x7f34cc0ce890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.138+0000 7f34d382a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f34cc15f520 0x7f34cc15f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.138+0000 7f34d382a640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f34cc15bf60 0x7f34cc15c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.137+0000 7f04ddaaf640 1 -- 172.21.15.43:0/167543697 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f04d004a390 msgr2=0x7f04d00524a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:17.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.137+0000 7f04ddaaf640 1 --2- 172.21.15.43:0/167543697 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f04d004a390 0x7f04d00524a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.137+0000 7f04ddaaf640 1 -- 172.21.15.43:0/167543697 >> v1:172.21.15.43:6789/0 conn(0x7f04d0057480 legacy=0x7f04d0051d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.137+0000 7f04ddaaf640 1 -- 172.21.15.43:0/167543697 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d0007f40 con 0x7f04d0049fb0 2024-04-25T08:54:17.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.138+0000 7fc8fd12c640 1 -- 172.21.15.43:0/2363522955 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f806b220 con 0x7fc8f81435f0 2024-04-25T08:54:17.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.138+0000 7f34d382a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34cc15fec0 con 0x7f34cc15bb80 2024-04-25T08:54:17.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.138+0000 7f34d382a640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34cc0cd1d0 con 0x7f34cc15bf60 2024-04-25T08:54:17.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.138+0000 7f34d382a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34cc0cc5c0 con 0x7f34cc15f520 2024-04-25T08:54:17.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.138+0000 7f08f19fb640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08ec150af0 0x7f08ec0b4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.138+0000 7f08f19fb640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f08ec0b47b0 0x7f08ec06be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.138+0000 7f08f19fb640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08ec159f40 con 0x7f08ec06c360 2024-04-25T08:54:17.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.138+0000 7f08f19fb640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08ec06c7d0 con 0x7f08ec0b47b0 2024-04-25T08:54:17.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.138+0000 7f08f19fb640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec150ed0 con 0x7f08ec150af0 2024-04-25T08:54:17.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.137+0000 7f2547fff640 1 -- 172.21.15.43:0/2387928210 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3803547603 0 0) 0x7f2548054660 con 0x7f25480e3160 2024-04-25T08:54:17.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.137+0000 7f2547fff640 1 -- 172.21.15.43:0/2387928210 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f253c003520 con 0x7f25480e3160 2024-04-25T08:54:17.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.137+0000 7f2555a08640 1 -- 172.21.15.43:0/2387928210 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f254803c0a0 msgr2=0x7f25480e7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:17.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.137+0000 7f2555a08640 1 --2- 172.21.15.43:0/2387928210 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f254803c0a0 0x7f25480e7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.137+0000 7f2555a08640 1 -- 172.21.15.43:0/2387928210 >> v1:172.21.15.193:6789/0 conn(0x7f25480e3160 legacy=0x7f2548058050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.137+0000 7f2555a08640 1 -- 172.21.15.43:0/2387928210 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2548058af0 con 0x7f2548053cb0 2024-04-25T08:54:17.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.138+0000 7f08ebfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f08ec0b47b0 0x7f08ec06be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.138+0000 7f08f09f9640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08ec150af0 0x7f08ec0b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.137+0000 7fef6ffff640 1 --2- 172.21.15.43:0/2441240994 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef7015b220 0x7fef700b3690 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fef700678b0 tx=0x7fef5c02e4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=109401c387afdfab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.138+0000 7fef6f7fe640 1 -- 172.21.15.43:0/2441240994 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5c03a070 con 0x7fef7015b220 2024-04-25T08:54:17.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.138+0000 7fef6f7fe640 1 -- 172.21.15.43:0/2441240994 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef5c007ce0 con 0x7fef7015b220 2024-04-25T08:54:17.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.138+0000 7fef6f7fe640 1 -- 172.21.15.43:0/2441240994 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5c0092f0 con 0x7fef7015b220 2024-04-25T08:54:17.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.138+0000 7f04167fc640 1 -- 172.21.15.43:0/2475971533 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f041804f770 con 0x7f04100e1090 2024-04-25T08:54:17.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7f0417fff640 1 --2- 172.21.15.43:0/2475971533 >> v2:172.21.15.193:3300/0 conn(0x7f041003a5d0 0x7f0410005f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:17.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.138+0000 7f34d15c6640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f34cc15bb80 0x7f34cc0ce890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7f34d15c6640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f34cc15bb80 0x7f34cc0ce890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37346/0 (socket says 172.21.15.43:37346) 2024-04-25T08:54:17.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7f34d15c6640 1 -- 172.21.15.43:0/3836379625 learned_addr learned my addr 172.21.15.43:0/3836379625 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:17.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7f2555a08640 1 --2- 172.21.15.43:0/2387928210 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2548053cb0 0x7f2548054120 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2550050190 tx=0x7f25500a50a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f748e713c955d792 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.138+0000 7f08f09f9640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08ec150af0 0x7f08ec0b4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:48866/0 (socket says 172.21.15.43:48866) 2024-04-25T08:54:17.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.138+0000 7f08f09f9640 1 -- 172.21.15.43:0/1336371246 learned_addr learned my addr 172.21.15.43:0/1336371246 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:17.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7f04167fc640 1 -- 172.21.15.43:0/2475971533 >> v1:172.21.15.43:6789/0 conn(0x7f04100e1090 legacy=0x7f0410039ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7f34d1dc7640 1 --2- 172.21.15.43:0/3836379625 >> v2:172.21.15.193:3300/0 conn(0x7f34cc15bf60 0x7f34cc15c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7f04ddaaf640 1 --2- 172.21.15.43:0/167543697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04d0049fb0 0x7f04d0051850 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f04d802d370 tx=0x7f04d80a50c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=466753f339d6760 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7fc8fd12c640 1 --2- 172.21.15.43:0/2363522955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8f81435f0 0x7fc8f8071920 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc8f806b710 tx=0x7fc8e802b430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2142b66e40a3ab36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7f04167fc640 1 --2- 172.21.15.43:0/2475971533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04080039b0 0x7f0408003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7f34d0dc5640 1 --2- 172.21.15.43:0/3836379625 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f34cc15f520 0x7f34cc15f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7f04167fc640 1 --2- 172.21.15.43:0/2475971533 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0410050dd0 0x7f0408004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7f04167fc640 1 --2- 172.21.15.43:0/2475971533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0408004d40 0x7f04080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7f04167fc640 1 -- 172.21.15.43:0/2475971533 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041004f820 con 0x7f04080039b0 2024-04-25T08:54:17.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7f04167fc640 1 -- 172.21.15.43:0/2475971533 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041004fb00 con 0x7f0408004d40 2024-04-25T08:54:17.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7f04167fc640 1 -- 172.21.15.43:0/2475971533 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f040800b9f0 con 0x7f0410050dd0 2024-04-25T08:54:17.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7f08f09f9640 1 -- 172.21.15.43:0/1336371246 >> v2:172.21.15.43:3300/0 conn(0x7f08ec0b47b0 msgr2=0x7f08ec06be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:17.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7f08f09f9640 1 --2- 172.21.15.43:0/1336371246 >> v2:172.21.15.43:3300/0 conn(0x7f08ec0b47b0 0x7f08ec06be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7f08f09f9640 1 -- 172.21.15.43:0/1336371246 >> v1:172.21.15.193:6789/0 conn(0x7f08ec06c360 legacy=0x7f08ec15d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:17.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7f2547fff640 1 -- 172.21.15.43:0/2387928210 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25500b2070 con 0x7f2548053cb0 2024-04-25T08:54:17.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7f2547fff640 1 -- 172.21.15.43:0/2387928210 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25500a5ce0 con 0x7f2548053cb0 2024-04-25T08:54:17.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7f08f09f9640 1 -- 172.21.15.43:0/1336371246 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ec15a200 con 0x7f08ec150af0 2024-04-25T08:54:17.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7fc8effff640 1 -- 172.21.15.43:0/2363522955 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e8037070 con 0x7fc8f81435f0 2024-04-25T08:54:17.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7fc8effff640 1 -- 172.21.15.43:0/2363522955 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8e8030710 con 0x7fc8f81435f0 2024-04-25T08:54:17.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7fc8effff640 1 -- 172.21.15.43:0/2363522955 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e8030ae0 con 0x7fc8f81435f0 2024-04-25T08:54:17.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7fef7721f640 1 -- 172.21.15.43:0/2441240994 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef7015b220 msgr2=0x7fef700b3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7fef7721f640 1 --2- 172.21.15.43:0/2441240994 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef7015b220 0x7fef700b3690 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fef700678b0 tx=0x7fef5c02e4e0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7f04177fe640 1 --2- 172.21.15.43:0/2475971533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04080039b0 0x7f0408003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7f04dcaad640 1 -- 172.21.15.43:0/167543697 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d80b2070 con 0x7f04d0049fb0 2024-04-25T08:54:17.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7f04dcaad640 1 -- 172.21.15.43:0/167543697 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04d80a5d00 con 0x7f04d0049fb0 2024-04-25T08:54:17.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7f04dcaad640 1 -- 172.21.15.43:0/167543697 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d80b2070 con 0x7f04d0049fb0 2024-04-25T08:54:17.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7ffb66334640 1 Processor -- start 2024-04-25T08:54:17.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7ffb66334640 1 -- start start 2024-04-25T08:54:17.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7f2547fff640 1 -- 172.21.15.43:0/2387928210 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25500ad640 con 0x7f2548053cb0 2024-04-25T08:54:17.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7f0417fff640 1 --2- 172.21.15.43:0/2475971533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0408004d40 0x7f04080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7fef7721f640 1 -- 172.21.15.43:0/2441240994 shutdown_connections 2024-04-25T08:54:17.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7fef7721f640 1 --2- 172.21.15.43:0/2441240994 >> v2:172.21.15.43:3300/0 conn(0x7fef700b3bd0 0x7fef70070d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7fef7721f640 1 --2- 172.21.15.43:0/2441240994 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef7015b220 0x7fef700b3690 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7fef7721f640 1 --2- 172.21.15.43:0/2441240994 >> v2:172.21.15.193:3300/0 conn(0x7fef700ae730 0x7fef700aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7fef7721f640 1 -- 172.21.15.43:0/2441240994 >> 172.21.15.43:0/2441240994 conn(0x7fef7006ab30 msgr2=0x7fef7006af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:17.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7fef7721f640 1 -- 172.21.15.43:0/2441240994 shutdown_connections 2024-04-25T08:54:17.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.140+0000 7fef7721f640 1 -- 172.21.15.43:0/2441240994 wait complete. 2024-04-25T08:54:17.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.140+0000 7fc8fe12e640 1 -- 172.21.15.43:0/2363522955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8f81435f0 msgr2=0x7fc8f8071920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.140+0000 7f0416ffd640 1 --2- 172.21.15.43:0/2475971533 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0410050dd0 0x7f0408004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.140+0000 7f04177fe640 1 -- 172.21.15.43:0/2475971533 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0410050dd0 msgr2=0x7f0408004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:17.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.140+0000 7f04177fe640 1 --2- 172.21.15.43:0/2475971533 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0410050dd0 0x7f0408004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.140+0000 7f04177fe640 1 -- 172.21.15.43:0/2475971533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0408004d40 msgr2=0x7f04080051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:17.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.140+0000 7f04177fe640 1 --2- 172.21.15.43:0/2475971533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0408004d40 0x7f04080051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.140+0000 7f04177fe640 1 -- 172.21.15.43:0/2475971533 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0408003520 con 0x7f04080039b0 2024-04-25T08:54:17.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7f34d1dc7640 1 -- 172.21.15.43:0/3836379625 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f34cc15f520 msgr2=0x7f34cc15f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:17.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7f34d1dc7640 1 --2- 172.21.15.43:0/3836379625 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f34cc15f520 0x7f34cc15f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7f34d1dc7640 1 -- 172.21.15.43:0/3836379625 >> v2:172.21.15.43:3300/0 conn(0x7f34cc15bb80 msgr2=0x7f34cc0ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:17.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7f34d1dc7640 1 --2- 172.21.15.43:0/3836379625 >> v2:172.21.15.43:3300/0 conn(0x7f34cc15bb80 0x7f34cc0ce890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.139+0000 7f34d1dc7640 1 -- 172.21.15.43:0/3836379625 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34cc0cc8a0 con 0x7f34cc15bf60 2024-04-25T08:54:17.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.140+0000 7fc8fe12e640 1 --2- 172.21.15.43:0/2363522955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8f81435f0 0x7fc8f8071920 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fc8f806b710 tx=0x7fc8e802b430 comp rx=0 tx=0).stop 2024-04-25T08:54:17.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.140+0000 7f34d1dc7640 1 --2- 172.21.15.43:0/3836379625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34cc15bf60 0x7f34cc15c3f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f34cc0c5030 tx=0x7f34bc02a090 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d887d7981ab2a38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.140+0000 7f2556209640 1 -- 172.21.15.43:0/2387928210 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2548053cb0 msgr2=0x7f2548054120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.140+0000 7f2556209640 1 --2- 172.21.15.43:0/2387928210 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2548053cb0 0x7f2548054120 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2550050190 tx=0x7f25500a50a0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.140+0000 7f08f09f9640 1 --2- 172.21.15.43:0/1336371246 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08ec150af0 0x7f08ec0b4270 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f08e400bef0 tx=0x7f08e40093b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80ebf443823575cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.140+0000 7fc8fe12e640 1 -- 172.21.15.43:0/2363522955 shutdown_connections 2024-04-25T08:54:17.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.140+0000 7fc8fe12e640 1 --2- 172.21.15.43:0/2363522955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8f81435f0 0x7fc8f8071920 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.140+0000 7fc8fe12e640 1 -- 172.21.15.43:0/2363522955 >> 172.21.15.43:0/2363522955 conn(0x7fc8f806cb40 msgr2=0x7fc8f806cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:17.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.140+0000 7f2556209640 1 -- 172.21.15.43:0/2387928210 shutdown_connections 2024-04-25T08:54:17.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.140+0000 7f2556209640 1 --2- 172.21.15.43:0/2387928210 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2548053cb0 0x7f2548054120 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.140+0000 7f2556209640 1 --2- 172.21.15.43:0/2387928210 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f254803c0a0 0x7f25480e7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.140+0000 7f2556209640 1 -- 172.21.15.43:0/2387928210 >> 172.21.15.43:0/2387928210 conn(0x7f2548058ee0 msgr2=0x7f25480592c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:17.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.141+0000 7f08eb7fe640 1 -- 172.21.15.43:0/1336371246 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e4036070 con 0x7f08ec150af0 2024-04-25T08:54:17.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.141+0000 7f04deab1640 1 -- 172.21.15.43:0/167543697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04d0049fb0 msgr2=0x7f04d0051850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.141+0000 7ffb66334640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffb600b3d60 0x7ffb60071000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.141+0000 7f04deab1640 1 --2- 172.21.15.43:0/167543697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04d0049fb0 0x7f04d0051850 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f04d802d370 tx=0x7f04d80a50c0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.141+0000 7f08eb7fe640 1 -- 172.21.15.43:0/1336371246 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e4009600 con 0x7f08ec150af0 2024-04-25T08:54:17.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.141+0000 7ffb66334640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ffb600cd3f0 0x7ffb600cd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.141+0000 7f08eb7fe640 1 -- 172.21.15.43:0/1336371246 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e40099f0 con 0x7f08ec150af0 2024-04-25T08:54:17.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.141+0000 7ffb66334640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb600ccc50 con 0x7ffb600b4140 2024-04-25T08:54:17.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.141+0000 7ffb66334640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb60071e80 con 0x7ffb600cd3f0 2024-04-25T08:54:17.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.141+0000 7ffb66334640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb600715e0 con 0x7ffb600b3d60 2024-04-25T08:54:17.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.141+0000 7f34c3fff640 1 -- 172.21.15.43:0/3836379625 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34bc037070 con 0x7f34cc15bf60 2024-04-25T08:54:17.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.141+0000 7fc8fe12e640 1 -- 172.21.15.43:0/2363522955 shutdown_connections 2024-04-25T08:54:17.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.141+0000 7ffb5ffff640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffb600b3d60 0x7ffb60071000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.141+0000 7f34c3fff640 1 -- 172.21.15.43:0/3836379625 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34bc02ade0 con 0x7f34cc15bf60 2024-04-25T08:54:17.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.141+0000 7f34c3fff640 1 -- 172.21.15.43:0/3836379625 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34bc032660 con 0x7f34cc15bf60 2024-04-25T08:54:17.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.141+0000 7fc8fe12e640 1 -- 172.21.15.43:0/2363522955 wait complete. 2024-04-25T08:54:17.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.141+0000 7ffb648d1640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ffb600cd3f0 0x7ffb600cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.141+0000 7ffb5ffff640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffb600b3d60 0x7ffb60071000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:48896/0 (socket says 172.21.15.43:48896) 2024-04-25T08:54:17.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.141+0000 7ffb5ffff640 1 -- 172.21.15.43:0/1594723947 learned_addr learned my addr 172.21.15.43:0/1594723947 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:17.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.141+0000 7f2556209640 1 -- 172.21.15.43:0/2387928210 shutdown_connections 2024-04-25T08:54:17.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.141+0000 7fef7721f640 1 Processor -- start 2024-04-25T08:54:17.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.141+0000 7f04177fe640 1 --2- 172.21.15.43:0/2475971533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04080039b0 0x7f0408003db0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f041806a4e0 tx=0x7f0408005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8da07b380fae5c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.141+0000 7fef7721f640 1 -- start start 2024-04-25T08:54:17.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.142+0000 7f04167fc640 1 -- 172.21.15.43:0/2475971533 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04180b3070 con 0x7f04080039b0 2024-04-25T08:54:17.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.141+0000 7ffb5ffff640 1 -- 172.21.15.43:0/1594723947 >> v2:172.21.15.43:3300/0 conn(0x7ffb600cd3f0 msgr2=0x7ffb600cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:17.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.141+0000 7ffb5ffff640 1 --2- 172.21.15.43:0/1594723947 >> v2:172.21.15.43:3300/0 conn(0x7ffb600cd3f0 0x7ffb600cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.141+0000 7ffb5ffff640 1 -- 172.21.15.43:0/1594723947 >> v1:172.21.15.193:6789/0 conn(0x7ffb600b4140 legacy=0x7ffb6006d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:17.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.141+0000 7ffb5ffff640 1 -- 172.21.15.43:0/1594723947 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb600718c0 con 0x7ffb600b3d60 2024-04-25T08:54:17.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.142+0000 7f04167fc640 1 -- 172.21.15.43:0/2475971533 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04180a6c10 con 0x7f04080039b0 2024-04-25T08:54:17.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.142+0000 7f34d382a640 1 -- 172.21.15.43:0/3836379625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34cc15bf60 msgr2=0x7f34cc15c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.142+0000 7f08f19fb640 1 -- 172.21.15.43:0/1336371246 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08ec150af0 msgr2=0x7f08ec0b4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.142+0000 7f08f19fb640 1 --2- 172.21.15.43:0/1336371246 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08ec150af0 0x7f08ec0b4270 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f08e400bef0 tx=0x7f08e40093b0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.142+0000 7f34d382a640 1 --2- 172.21.15.43:0/3836379625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34cc15bf60 0x7f34cc15c3f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f34cc0c5030 tx=0x7f34bc02a090 comp rx=0 tx=0).stop 2024-04-25T08:54:17.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.142+0000 7f04167fc640 1 -- 172.21.15.43:0/2475971533 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0418069020 con 0x7f04080039b0 2024-04-25T08:54:17.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.142+0000 7f04deab1640 1 -- 172.21.15.43:0/167543697 shutdown_connections 2024-04-25T08:54:17.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.142+0000 7f04deab1640 1 --2- 172.21.15.43:0/167543697 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f04d004a390 0x7f04d00524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.142+0000 7f04deab1640 1 --2- 172.21.15.43:0/167543697 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04d0049fb0 0x7f04d0051850 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.142+0000 7f04deab1640 1 -- 172.21.15.43:0/167543697 >> 172.21.15.43:0/167543697 conn(0x7f04d0020040 msgr2=0x7f04d0056d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:17.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.142+0000 7f34d382a640 1 -- 172.21.15.43:0/3836379625 shutdown_connections 2024-04-25T08:54:17.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.142+0000 7f34d382a640 1 --2- 172.21.15.43:0/3836379625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34cc15bf60 0x7f34cc15c3f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.142+0000 7f34d382a640 1 --2- 172.21.15.43:0/3836379625 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f34cc15f520 0x7f34cc15f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.142+0000 7f34d382a640 1 --2- 172.21.15.43:0/3836379625 >> v2:172.21.15.43:3300/0 conn(0x7f34cc15bb80 0x7f34cc0ce890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.142+0000 7f34d382a640 1 -- 172.21.15.43:0/3836379625 >> 172.21.15.43:0/3836379625 conn(0x7f34cc0b3170 msgr2=0x7f34cc0b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:17.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.141+0000 7f2556209640 1 -- 172.21.15.43:0/2387928210 wait complete. 2024-04-25T08:54:17.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.142+0000 7f08f19fb640 1 -- 172.21.15.43:0/1336371246 shutdown_connections 2024-04-25T08:54:17.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.142+0000 7f08f19fb640 1 --2- 172.21.15.43:0/1336371246 >> v2:172.21.15.43:3300/0 conn(0x7f08ec0b47b0 0x7f08ec06be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.142+0000 7f08f19fb640 1 --2- 172.21.15.43:0/1336371246 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08ec150af0 0x7f08ec0b4270 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.142+0000 7f08f19fb640 1 -- 172.21.15.43:0/1336371246 >> 172.21.15.43:0/1336371246 conn(0x7f08ec06ab30 msgr2=0x7f08ec06af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:17.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.142+0000 7f34d382a640 1 -- 172.21.15.43:0/3836379625 shutdown_connections 2024-04-25T08:54:17.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.143+0000 7ffb5ffff640 1 --2- 172.21.15.43:0/1594723947 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffb600b3d60 0x7ffb60071000 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ffb5800d150 tx=0x7ffb5802c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2afbdb8b4e61c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.143+0000 7f041c862640 1 -- 172.21.15.43:0/2475971533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04080039b0 msgr2=0x7f0408003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.143+0000 7f041c862640 1 --2- 172.21.15.43:0/2475971533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04080039b0 0x7f0408003db0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f041806a4e0 tx=0x7f0408005830 comp rx=0 tx=0).stop 2024-04-25T08:54:17.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.143+0000 7f0417fff640 1 -- 172.21.15.43:0/2475971533 reap_dead start 2024-04-25T08:54:17.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.143+0000 7ffb5effd640 1 -- 172.21.15.43:0/1594723947 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5800b2e0 con 0x7ffb600b3d60 2024-04-25T08:54:17.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.143+0000 7f041c862640 1 -- 172.21.15.43:0/2475971533 shutdown_connections 2024-04-25T08:54:17.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.143+0000 7f041c862640 1 -- 172.21.15.43:0/2475971533 >> 172.21.15.43:0/2475971533 conn(0x7f0410009fb0 msgr2=0x7f041000a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:17.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.143+0000 7ffb5effd640 1 -- 172.21.15.43:0/1594723947 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb58009600 con 0x7ffb600b3d60 2024-04-25T08:54:17.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.143+0000 7ffb5effd640 1 -- 172.21.15.43:0/1594723947 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5800b480 con 0x7ffb600b3d60 2024-04-25T08:54:17.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.142+0000 7f34d382a640 1 -- 172.21.15.43:0/3836379625 wait complete. 2024-04-25T08:54:17.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.143+0000 7f2556209640 1 Processor -- start 2024-04-25T08:54:17.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.143+0000 7f041c862640 1 -- 172.21.15.43:0/2475971533 shutdown_connections 2024-04-25T08:54:17.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.143+0000 7fef7721f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef700ae730 0x7fef702034c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.143+0000 7fef7721f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef700b3bd0 0x7fef700cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.143+0000 7fc8fe12e640 1 Processor -- start 2024-04-25T08:54:17.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.143+0000 7f2556209640 1 -- start start 2024-04-25T08:54:17.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.143+0000 7fef7721f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef7015b220 0x7fef70203c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.143+0000 7f04deab1640 1 -- 172.21.15.43:0/167543697 shutdown_connections 2024-04-25T08:54:17.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.143+0000 7fef7721f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7006c840 con 0x7fef700ae730 2024-04-25T08:54:17.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.144+0000 7f04deab1640 1 -- 172.21.15.43:0/167543697 wait complete. 2024-04-25T08:54:17.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.144+0000 7fef6ffff640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef700b3bd0 0x7fef700cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.144+0000 7fef6ffff640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef700b3bd0 0x7fef700cd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:48898/0 (socket says 172.21.15.43:48898) 2024-04-25T08:54:17.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.144+0000 7fef6ffff640 1 -- 172.21.15.43:0/513698317 learned_addr learned my addr 172.21.15.43:0/513698317 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:17.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.144+0000 7fef757bc640 1 --2- 172.21.15.43:0/513698317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef7015b220 0x7fef70203c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.144+0000 7fef7721f640 1 -- 172.21.15.43:0/513698317 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef7006c560 con 0x7fef7015b220 2024-04-25T08:54:17.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.144+0000 7fef7721f640 1 -- 172.21.15.43:0/513698317 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef70154420 con 0x7fef700b3bd0 2024-04-25T08:54:17.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.143+0000 7f041c862640 1 -- 172.21.15.43:0/2475971533 wait complete. 2024-04-25T08:54:17.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.144+0000 7fc8fe12e640 1 -- start start 2024-04-25T08:54:17.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.144+0000 7ffb66334640 1 -- 172.21.15.43:0/1594723947 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffb600b3d60 msgr2=0x7ffb60071000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.144+0000 7ffb66334640 1 --2- 172.21.15.43:0/1594723947 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffb600b3d60 0x7ffb60071000 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ffb5800d150 tx=0x7ffb5802c8e0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.144+0000 7fef74fbb640 1 --2- 172.21.15.43:0/513698317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef700ae730 0x7fef702034c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.144+0000 7fef74fbb640 1 -- 172.21.15.43:0/513698317 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef700b3bd0 msgr2=0x7fef700cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.144+0000 7f08f19fb640 1 -- 172.21.15.43:0/1336371246 shutdown_connections 2024-04-25T08:54:17.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.144+0000 7f34d382a640 1 Processor -- start 2024-04-25T08:54:17.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.144+0000 7ffb66334640 1 -- 172.21.15.43:0/1594723947 shutdown_connections 2024-04-25T08:54:17.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.144+0000 7fef74fbb640 1 --2- 172.21.15.43:0/513698317 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef700b3bd0 0x7fef700cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.144+0000 7fef74fbb640 1 -- 172.21.15.43:0/513698317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef7015b220 msgr2=0x7fef70203c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.144+0000 7f34d382a640 1 -- start start 2024-04-25T08:54:17.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.144+0000 7fef74fbb640 1 --2- 172.21.15.43:0/513698317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef7015b220 0x7fef70203c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.144+0000 7fef74fbb640 1 -- 172.21.15.43:0/513698317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef700cdfb0 con 0x7fef700ae730 2024-04-25T08:54:17.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.144+0000 7fef757bc640 1 --2- 172.21.15.43:0/513698317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef7015b220 0x7fef70203c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:54:17.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.144+0000 7fef6ffff640 1 --2- 172.21.15.43:0/513698317 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef700b3bd0 0x7fef700cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:54:17.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.144+0000 7ffb66334640 1 --2- 172.21.15.43:0/1594723947 >> v2:172.21.15.43:3300/0 conn(0x7ffb600cd3f0 0x7ffb600cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.144+0000 7ffb66334640 1 --2- 172.21.15.43:0/1594723947 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffb600b3d60 0x7ffb60071000 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.144+0000 7ffb66334640 1 -- 172.21.15.43:0/1594723947 >> 172.21.15.43:0/1594723947 conn(0x7ffb600b1480 msgr2=0x7ffb600b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:17.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.145+0000 7f08f19fb640 1 -- 172.21.15.43:0/1336371246 wait complete. 2024-04-25T08:54:17.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.145+0000 7fef74fbb640 1 --2- 172.21.15.43:0/513698317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef700ae730 0x7fef702034c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fef6400e870 tx=0x7fef6400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.145+0000 7f2556209640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f254803c0a0 0x7f25480e0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.145+0000 7f041c862640 1 Processor -- start 2024-04-25T08:54:17.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.145+0000 7f2556209640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2548053cb0 0x7f25480509c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.145+0000 7f2555207640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f254803c0a0 0x7f25480e0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.145+0000 7fef6d7fa640 1 -- 172.21.15.43:0/513698317 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef64017070 con 0x7fef700ae730 2024-04-25T08:54:17.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.145+0000 7f041c862640 1 -- start start 2024-04-25T08:54:17.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.145+0000 7f2555207640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f254803c0a0 0x7f25480e0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:46824/0 (socket says 172.21.15.43:46824) 2024-04-25T08:54:17.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.145+0000 7fef6d7fa640 1 -- 172.21.15.43:0/513698317 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef6400ee60 con 0x7fef700ae730 2024-04-25T08:54:17.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.145+0000 7fc8fe12e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc8f80b3720 0x7fc8f81597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.145+0000 7f2555207640 1 -- 172.21.15.43:0/491436607 learned_addr learned my addr 172.21.15.43:0/491436607 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:17.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.145+0000 7fc8fe12e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8f80b42c0 0x7fc8f813b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.145+0000 7ffb66334640 1 -- 172.21.15.43:0/1594723947 shutdown_connections 2024-04-25T08:54:17.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.145+0000 7fef7721f640 1 -- 172.21.15.43:0/513698317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef700ce290 con 0x7fef700ae730 2024-04-25T08:54:17.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.145+0000 7fef7721f640 1 -- 172.21.15.43:0/513698317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef700ce850 con 0x7fef700ae730 2024-04-25T08:54:17.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.145+0000 7ffb66334640 1 -- 172.21.15.43:0/1594723947 wait complete. 2024-04-25T08:54:17.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.145+0000 7fc8fe12e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8f81435f0 0x7fc8f8070d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.145+0000 7f2554a06640 1 --2- 172.21.15.43:0/491436607 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2548053cb0 0x7f25480509c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.146+0000 7fef6d7fa640 1 -- 172.21.15.43:0/513698317 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6400cb60 con 0x7fef700ae730 2024-04-25T08:54:17.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.146+0000 7fc8fd12c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc8f80b3720 0x7fc8f81597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.146+0000 7fc8fe12e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f81439d0 con 0x7fc8f81435f0 2024-04-25T08:54:17.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.146+0000 7fc8fe12e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f815ab50 con 0x7fc8f80b42c0 2024-04-25T08:54:17.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.146+0000 7fc8fe12e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f8143b50 con 0x7fc8f80b3720 2024-04-25T08:54:17.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.146+0000 7fc8fd12c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc8f80b3720 0x7fc8f81597a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:48918/0 (socket says 172.21.15.43:48918) 2024-04-25T08:54:17.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.146+0000 7fc8fd12c640 1 -- 172.21.15.43:0/2898367433 learned_addr learned my addr 172.21.15.43:0/2898367433 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:17.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.146+0000 7f04deab1640 1 Processor -- start 2024-04-25T08:54:17.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.146+0000 7f2556209640 1 --2- 172.21.15.43:0/491436607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25480e3160 0x7f25480dd730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.146+0000 7f2556209640 1 -- 172.21.15.43:0/491436607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25480e3540 con 0x7f25480e3160 2024-04-25T08:54:17.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.146+0000 7f2556209640 1 -- 172.21.15.43:0/491436607 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2548006640 con 0x7f254803c0a0 2024-04-25T08:54:17.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.146+0000 7f2556209640 1 -- 172.21.15.43:0/491436607 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2548006b60 con 0x7f2548053cb0 2024-04-25T08:54:17.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.146+0000 7f2555a08640 1 --2- 172.21.15.43:0/491436607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25480e3160 0x7f25480dd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.146+0000 7fc8fc92b640 1 --2- 172.21.15.43:0/2898367433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8f80b42c0 0x7fc8f813b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.146+0000 7fc8fd92d640 1 --2- 172.21.15.43:0/2898367433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8f81435f0 0x7fc8f8070d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.146+0000 7fc8fd12c640 1 -- 172.21.15.43:0/2898367433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8f80b42c0 msgr2=0x7fc8f813b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.146+0000 7fc8fd12c640 1 --2- 172.21.15.43:0/2898367433 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8f80b42c0 0x7fc8f813b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.146+0000 7fc8fd12c640 1 -- 172.21.15.43:0/2898367433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8f81435f0 msgr2=0x7fc8f8070d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.146+0000 7fc8fd12c640 1 --2- 172.21.15.43:0/2898367433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8f81435f0 0x7fc8f8070d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.146+0000 7fc8fd12c640 1 -- 172.21.15.43:0/2898367433 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8f813da40 con 0x7fc8f80b3720 2024-04-25T08:54:17.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.146+0000 7f04deab1640 1 -- start start 2024-04-25T08:54:17.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.146+0000 7f34d382a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f34cc15bb80 0x7f34cc14b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.146+0000 7f34d382a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34cc15bf60 0x7f34cc0b5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.146+0000 7f34d382a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34cc15f520 0x7f34cc1487e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.146+0000 7f34d382a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34cc0cedd0 con 0x7f34cc15bf60 2024-04-25T08:54:17.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.146+0000 7f34d15c6640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f34cc15bb80 0x7f34cc14b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.146+0000 7f34d382a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34cc0cce60 con 0x7f34cc15f520 2024-04-25T08:54:17.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.146+0000 7f34d382a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34cc0cef50 con 0x7f34cc15bb80 2024-04-25T08:54:17.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.146+0000 7f2555207640 1 -- 172.21.15.43:0/491436607 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2548053cb0 msgr2=0x7f25480509c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.146+0000 7f2555207640 1 --2- 172.21.15.43:0/491436607 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2548053cb0 0x7f25480509c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.146+0000 7f2555207640 1 -- 172.21.15.43:0/491436607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25480e3160 msgr2=0x7f25480dd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.146+0000 7f2555207640 1 --2- 172.21.15.43:0/491436607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25480e3160 0x7f25480dd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.147+0000 7f2555207640 1 -- 172.21.15.43:0/491436607 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2548052e00 con 0x7f254803c0a0 2024-04-25T08:54:17.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.147+0000 7f34d1dc7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34cc15f520 0x7f34cc1487e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.147+0000 7f34d15c6640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f34cc15bb80 0x7f34cc14b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:48928/0 (socket says 172.21.15.43:48928) 2024-04-25T08:54:17.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.147+0000 7f34d15c6640 1 -- 172.21.15.43:0/2312740723 learned_addr learned my addr 172.21.15.43:0/2312740723 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:17.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.147+0000 7f34d0dc5640 1 --2- 172.21.15.43:0/2312740723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34cc15bf60 0x7f34cc0b5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.147+0000 7f2555207640 1 --2- 172.21.15.43:0/491436607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f254803c0a0 0x7f25480e0360 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f254000c3c0 tx=0x7f254000c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.147+0000 7f34d15c6640 1 -- 172.21.15.43:0/2312740723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34cc15f520 msgr2=0x7f34cc1487e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.147+0000 7f34d15c6640 1 --2- 172.21.15.43:0/2312740723 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34cc15f520 0x7f34cc1487e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.147+0000 7f34d15c6640 1 -- 172.21.15.43:0/2312740723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34cc15bf60 msgr2=0x7f34cc0b5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.147+0000 7f34d15c6640 1 --2- 172.21.15.43:0/2312740723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34cc15bf60 0x7f34cc0b5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.147+0000 7f34d15c6640 1 -- 172.21.15.43:0/2312740723 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34cc0b4c60 con 0x7f34cc15bb80 2024-04-25T08:54:17.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.147+0000 7f34d0dc5640 1 --2- 172.21.15.43:0/2312740723 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34cc15bf60 0x7f34cc0b5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:54:17.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.148+0000 7f2545ffb640 1 -- 172.21.15.43:0/491436607 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2540017070 con 0x7f254803c0a0 2024-04-25T08:54:17.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.148+0000 7f2545ffb640 1 -- 172.21.15.43:0/491436607 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2540005ce0 con 0x7f254803c0a0 2024-04-25T08:54:17.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.148+0000 7f2545ffb640 1 -- 172.21.15.43:0/491436607 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25400072f0 con 0x7f254803c0a0 2024-04-25T08:54:17.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.148+0000 7ffb66334640 1 Processor -- start 2024-04-25T08:54:17.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.147+0000 7f08f19fb640 1 Processor -- start 2024-04-25T08:54:17.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.147+0000 7f08f19fb640 1 -- start start 2024-04-25T08:54:17.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.148+0000 7ffb66334640 1 -- start start 2024-04-25T08:54:17.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.148+0000 7f34d15c6640 1 --2- 172.21.15.43:0/2312740723 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f34cc15bb80 0x7f34cc14b2c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f34c800d800 tx=0x7f34c800dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.148+0000 7fef7721f640 1 -- 172.21.15.43:0/513698317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fef50001030 con 0x7fef700ae730 2024-04-25T08:54:17.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.147+0000 7fc8fd12c640 1 --2- 172.21.15.43:0/2898367433 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc8f80b3720 0x7fc8f81597a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8000c00 tx=0x7fc8e80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.147+0000 7fc8edffb640 1 -- 172.21.15.43:0/2898367433 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e8037040 con 0x7fc8f80b3720 2024-04-25T08:54:17.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.148+0000 7fc8edffb640 1 -- 172.21.15.43:0/2898367433 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8e803cb30 con 0x7fc8f80b3720 2024-04-25T08:54:17.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.148+0000 7fc8edffb640 1 -- 172.21.15.43:0/2898367433 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e8047480 con 0x7fc8f80b3720 2024-04-25T08:54:17.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.148+0000 7fc8fe12e640 1 -- 172.21.15.43:0/2898367433 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8f813dd20 con 0x7fc8f80b3720 2024-04-25T08:54:17.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.148+0000 7fc8fe12e640 1 -- 172.21.15.43:0/2898367433 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8f813e2e0 con 0x7fc8f80b3720 2024-04-25T08:54:17.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.148+0000 7f041c862640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04100cd050 0x7f041004ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.148+0000 7f34c1ffb640 1 -- 172.21.15.43:0/2312740723 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c8018020 con 0x7f34cc15bb80 2024-04-25T08:54:17.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.147+0000 7f04deab1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04d0049fb0 0x7f04d0034270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.147+0000 7f04deab1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04d004a390 0x7f04d00365a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.147+0000 7f04ddaaf640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04d0049fb0 0x7f04d0034270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.147+0000 7f04deab1640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f04d0057480 0x7f04d00e0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.147+0000 7f04ddaaf640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04d0049fb0 0x7f04d0034270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:46860/0 (socket says 172.21.15.43:46860) 2024-04-25T08:54:17.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.148+0000 7f04ddaaf640 1 -- 172.21.15.43:0/758027632 learned_addr learned my addr 172.21.15.43:0/758027632 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:17.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.148+0000 7f04dd2ae640 1 --2- 172.21.15.43:0/758027632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04d004a390 0x7f04d00365a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.148+0000 7f04deab1640 1 -- 172.21.15.43:0/758027632 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d00529e0 con 0x7f04d004a390 2024-04-25T08:54:17.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.148+0000 7f04deab1640 1 -- 172.21.15.43:0/758027632 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04d0038580 con 0x7f04d0049fb0 2024-04-25T08:54:17.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.148+0000 7f04deab1640 1 -- 172.21.15.43:0/758027632 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04d0052b60 con 0x7f04d0057480 2024-04-25T08:54:17.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.148+0000 7f04dd2ae640 1 -- 172.21.15.43:0/758027632 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f04d0057480 msgr2=0x7f04d00e0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.148+0000 7f04dd2ae640 1 --2- 172.21.15.43:0/758027632 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f04d0057480 0x7f04d00e0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.148+0000 7f04dd2ae640 1 -- 172.21.15.43:0/758027632 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04d0049fb0 msgr2=0x7f04d0034270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.148+0000 7f04dd2ae640 1 --2- 172.21.15.43:0/758027632 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04d0049fb0 0x7f04d0034270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.148+0000 7f04dd2ae640 1 -- 172.21.15.43:0/758027632 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04d00de200 con 0x7f04d004a390 2024-04-25T08:54:17.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.148+0000 7f04dd2ae640 1 --2- 172.21.15.43:0/758027632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04d004a390 0x7f04d00365a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f04d400c390 tx=0x7f04d400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.149+0000 7f08f19fb640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08ec06c360 0x7f08ec1f3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.149+0000 7f08f19fb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08ec0b47b0 0x7f08ec071c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.149+0000 7f2556209640 1 -- 172.21.15.43:0/491436607 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25480ddc70 con 0x7f254803c0a0 2024-04-25T08:54:17.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.149+0000 7f08f19fb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08ec150af0 0x7f08ec13fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.149+0000 7f08f09f9640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08ec06c360 0x7f08ec1f3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.149+0000 7f08f09f9640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08ec06c360 0x7f08ec1f3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:48946/0 (socket says 172.21.15.43:48946) 2024-04-25T08:54:17.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.149+0000 7f08f09f9640 1 -- 172.21.15.43:0/1485790192 learned_addr learned my addr 172.21.15.43:0/1485790192 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:17.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.149+0000 7f08f19fb640 1 -- 172.21.15.43:0/1485790192 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec15a7c0 con 0x7f08ec0b47b0 2024-04-25T08:54:17.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.149+0000 7f08f19fb640 1 -- 172.21.15.43:0/1485790192 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec15ae10 con 0x7f08ec150af0 2024-04-25T08:54:17.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.149+0000 7f08f19fb640 1 -- 172.21.15.43:0/1485790192 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec15aaa0 con 0x7f08ec06c360 2024-04-25T08:54:17.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.149+0000 7f08f11fa640 1 --2- 172.21.15.43:0/1485790192 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08ec150af0 0x7f08ec13fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.149+0000 7f2556209640 1 -- 172.21.15.43:0/491436607 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2548051380 con 0x7f254803c0a0 2024-04-25T08:54:17.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.148+0000 7f34c1ffb640 1 -- 172.21.15.43:0/2312740723 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34c80096a0 con 0x7f34cc15bb80 2024-04-25T08:54:17.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.148+0000 7f041c862640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0410049000 0x7f04100d0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.149+0000 7f08ebfff640 1 --2- 172.21.15.43:0/1485790192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08ec0b47b0 0x7f08ec071c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.150+0000 7ffb66334640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffb600b3d60 0x7ffb601e3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.148+0000 7f041c862640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04100d12c0 0x7f04100d1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.150+0000 7f08f11fa640 1 -- 172.21.15.43:0/1485790192 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08ec06c360 msgr2=0x7f08ec1f3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.150+0000 7f04ce7fc640 1 -- 172.21.15.43:0/758027632 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d40074e0 con 0x7f04d004a390 2024-04-25T08:54:17.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.150+0000 7ffb66334640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb600b4140 0x7ffb601553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.150+0000 7f08f11fa640 1 --2- 172.21.15.43:0/1485790192 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08ec06c360 0x7f08ec1f3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.150+0000 7f08f11fa640 1 -- 172.21.15.43:0/1485790192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08ec0b47b0 msgr2=0x7f08ec071c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.150+0000 7ffb66334640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb600cd3f0 0x7ffb600ce7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.148+0000 7f041c862640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0410038630 con 0x7f04100cd050 2024-04-25T08:54:17.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.148+0000 7f041c862640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04100503a0 con 0x7f04100d12c0 2024-04-25T08:54:17.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.148+0000 7f0416ffd640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0410049000 0x7f04100d0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.149+0000 7f0416ffd640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0410049000 0x7f04100d0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:48944/0 (socket says 172.21.15.43:48944) 2024-04-25T08:54:17.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.149+0000 7f0416ffd640 1 -- 172.21.15.43:0/772494610 learned_addr learned my addr 172.21.15.43:0/772494610 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:17.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.149+0000 7f04177fe640 1 --2- 172.21.15.43:0/772494610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04100cd050 0x7f041004ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.148+0000 7f041c862640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04100387b0 con 0x7f0410049000 2024-04-25T08:54:17.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.150+0000 7f04ce7fc640 1 -- 172.21.15.43:0/758027632 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04d40047e0 con 0x7f04d004a390 2024-04-25T08:54:17.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.148+0000 7f34c1ffb640 1 -- 172.21.15.43:0/2312740723 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c8009a50 con 0x7f34cc15bb80 2024-04-25T08:54:17.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.150+0000 7f04deab1640 1 -- 172.21.15.43:0/758027632 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04d00de4e0 con 0x7f04d004a390 2024-04-25T08:54:17.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.150+0000 7f08f11fa640 1 --2- 172.21.15.43:0/1485790192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08ec0b47b0 0x7f08ec071c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.150+0000 7f08f11fa640 1 -- 172.21.15.43:0/1485790192 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ec0a9fc0 con 0x7f08ec150af0 2024-04-25T08:54:17.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.150+0000 7f08f09f9640 1 --2- 172.21.15.43:0/1485790192 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08ec06c360 0x7f08ec1f3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:54:17.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.150+0000 7f08ebfff640 1 --2- 172.21.15.43:0/1485790192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08ec0b47b0 0x7f08ec071c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:17.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.150+0000 7f08f11fa640 1 --2- 172.21.15.43:0/1485790192 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08ec150af0 0x7f08ec13fa20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00b440 tx=0x7f08dc00b900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.150+0000 7fef6d7fa640 1 -- 172.21.15.43:0/513698317 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fef6401a070 con 0x7fef700ae730 2024-04-25T08:54:17.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.150+0000 7f0416ffd640 1 -- 172.21.15.43:0/772494610 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04100d12c0 msgr2=0x7f04100d1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.150+0000 7f0416ffd640 1 --2- 172.21.15.43:0/772494610 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04100d12c0 0x7f04100d1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.150+0000 7f0416ffd640 1 -- 172.21.15.43:0/772494610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04100cd050 msgr2=0x7f041004ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.150+0000 7f0416ffd640 1 --2- 172.21.15.43:0/772494610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04100cd050 0x7f041004ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.150+0000 7f0416ffd640 1 -- 172.21.15.43:0/772494610 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0410049490 con 0x7f0410049000 2024-04-25T08:54:17.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.150+0000 7f04deab1640 1 -- 172.21.15.43:0/758027632 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04d00deaa0 con 0x7f04d004a390 2024-04-25T08:54:17.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.150+0000 7f04ce7fc640 1 -- 172.21.15.43:0/758027632 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d4004b90 con 0x7f04d004a390 2024-04-25T08:54:17.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.150+0000 7ffb66334640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb600b45a0 con 0x7ffb600cd3f0 2024-04-25T08:54:17.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.150+0000 7ffb66334640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb600cced0 con 0x7ffb600b4140 2024-04-25T08:54:17.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.150+0000 7ffb66334640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb60159ec0 con 0x7ffb600b3d60 2024-04-25T08:54:17.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.150+0000 7ffb5ffff640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffb600b3d60 0x7ffb601e3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.150+0000 7ffb5ffff640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffb600b3d60 0x7ffb601e3930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:48956/0 (socket says 172.21.15.43:48956) 2024-04-25T08:54:17.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.150+0000 7ffb5ffff640 1 -- 172.21.15.43:0/3622186461 learned_addr learned my addr 172.21.15.43:0/3622186461 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:17.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.150+0000 7f34d382a640 1 -- 172.21.15.43:0/2312740723 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34cc0b4f40 con 0x7f34cc15bb80 2024-04-25T08:54:17.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.151+0000 7fef6d7fa640 1 --2- 172.21.15.43:0/513698317 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fef5406b5a0 0x7fef5406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.151+0000 7f34d382a640 1 -- 172.21.15.43:0/2312740723 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34cc0b5500 con 0x7f34cc15bb80 2024-04-25T08:54:17.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.151+0000 7ffb5ffff640 1 -- 172.21.15.43:0/3622186461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb600b4140 msgr2=0x7ffb601553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.151+0000 7ffb5ffff640 1 --2- 172.21.15.43:0/3622186461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffb600b4140 0x7ffb601553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.151+0000 7ffb5ffff640 1 -- 172.21.15.43:0/3622186461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb600cd3f0 msgr2=0x7ffb600ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.151+0000 7ffb5ffff640 1 --2- 172.21.15.43:0/3622186461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb600cd3f0 0x7ffb600ce7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.151+0000 7ffb5ffff640 1 -- 172.21.15.43:0/3622186461 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb60155b10 con 0x7ffb600b3d60 2024-04-25T08:54:17.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.151+0000 7fef6ffff640 1 --2- 172.21.15.43:0/513698317 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fef5406b5a0 0x7fef5406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.151+0000 7fc8edffb640 1 -- 172.21.15.43:0/2898367433 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc8e8047620 con 0x7fc8f80b3720 2024-04-25T08:54:17.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.151+0000 7fc8fe12e640 1 -- 172.21.15.43:0/2898367433 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc8f8074e80 con 0x7fc8f80b3720 2024-04-25T08:54:17.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.151+0000 7f08e97fa640 1 -- 172.21.15.43:0/1485790192 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc015020 con 0x7f08ec150af0 2024-04-25T08:54:17.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.151+0000 7f08f19fb640 1 -- 172.21.15.43:0/1485790192 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08ec0aa2a0 con 0x7f08ec150af0 2024-04-25T08:54:17.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.151+0000 7f2556209640 1 -- 172.21.15.43:0/491436607 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2548005390 con 0x7f254803c0a0 2024-04-25T08:54:17.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.151+0000 7fef6d7fa640 1 -- 172.21.15.43:0/513698317 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fef64089f00 con 0x7fef700ae730 2024-04-25T08:54:17.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.152+0000 7f04deab1640 1 -- 172.21.15.43:0/758027632 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f04c0001030 con 0x7f04d004a390 2024-04-25T08:54:17.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.152+0000 7f08f19fb640 1 -- 172.21.15.43:0/1485790192 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08ec0aa580 con 0x7f08ec150af0 2024-04-25T08:54:17.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.152+0000 7ffb5ffff640 1 --2- 172.21.15.43:0/3622186461 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffb600b3d60 0x7ffb601e3930 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ffb5802ced0 tx=0x7ffb58004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.152+0000 7fef6d7fa640 1 --2- 172.21.15.43:0/513698317 >> [v2:172.21.15.43:6800/1101549916,v1:172.21.15.43:6801/1101549916] conn(0x7fef54073740 0x7fef54075b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.152+0000 7fc8edffb640 1 --2- 172.21.15.43:0/2898367433 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc8dc06b5a0 0x7fc8dc06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.152+0000 7fef6ffff640 1 --2- 172.21.15.43:0/513698317 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fef5406b5a0 0x7fef5406da60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fef7015ca00 tx=0x7fef5c009000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.152+0000 7fef6d7fa640 1 -- 172.21.15.43:0/513698317 --> [v2:172.21.15.43:6800/1101549916,v1:172.21.15.43:6801/1101549916] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fef54076210 con 0x7fef54073740 2024-04-25T08:54:17.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.152+0000 7fc8edffb640 1 -- 172.21.15.43:0/2898367433 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc8e80aedd0 con 0x7fc8f80b3720 2024-04-25T08:54:17.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.152+0000 7fef757bc640 1 --2- 172.21.15.43:0/513698317 >> [v2:172.21.15.43:6800/1101549916,v1:172.21.15.43:6801/1101549916] conn(0x7fef54073740 0x7fef54075b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.152+0000 7f0416ffd640 1 --2- 172.21.15.43:0/772494610 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0410049000 0x7f04100d0d80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f040c010930 tx=0x7f040c010df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.152+0000 7fc8edffb640 1 --2- 172.21.15.43:0/2898367433 >> [v2:172.21.15.43:6808/1175991178,v1:172.21.15.43:6809/1175991178] conn(0x7fc8dc0737c0 0x7fc8dc075be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.152+0000 7fef6d7fa640 1 -- 172.21.15.43:0/513698317 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef640549b0 con 0x7fef700ae730 2024-04-25T08:54:17.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.152+0000 7fc8edffb640 1 -- 172.21.15.43:0/2898367433 --> [v2:172.21.15.43:6808/1175991178,v1:172.21.15.43:6809/1175991178] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc8e803de50 con 0x7fc8dc0737c0 2024-04-25T08:54:17.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.152+0000 7fc8edffb640 1 -- 172.21.15.43:0/2898367433 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8e80b43c0 con 0x7fc8f80b3720 2024-04-25T08:54:17.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.152+0000 7f34d382a640 1 -- 172.21.15.43:0/2312740723 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f34cc074e80 con 0x7f34cc15bb80 2024-04-25T08:54:17.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.152+0000 7ffb5cff9640 1 -- 172.21.15.43:0/3622186461 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb58040030 con 0x7ffb600b3d60 2024-04-25T08:54:17.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.152+0000 7fc8fc92b640 1 --2- 172.21.15.43:0/2898367433 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc8dc06b5a0 0x7fc8dc06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.153+0000 7fc8fd92d640 1 --2- 172.21.15.43:0/2898367433 >> [v2:172.21.15.43:6808/1175991178,v1:172.21.15.43:6809/1175991178] conn(0x7fc8dc0737c0 0x7fc8dc075be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.153+0000 7f08e97fa640 1 -- 172.21.15.43:0/1485790192 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08dc011830 con 0x7f08ec150af0 2024-04-25T08:54:17.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.152+0000 7ffb5cff9640 1 -- 172.21.15.43:0/3622186461 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb5803c640 con 0x7ffb600b3d60 2024-04-25T08:54:17.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.153+0000 7f08e97fa640 1 -- 172.21.15.43:0/1485790192 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc011c20 con 0x7f08ec150af0 2024-04-25T08:54:17.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.152+0000 7ffb5cff9640 1 -- 172.21.15.43:0/3622186461 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5803ca30 con 0x7ffb600b3d60 2024-04-25T08:54:17.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.152+0000 7f0403fff640 1 -- 172.21.15.43:0/772494610 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c00f040 con 0x7f0410049000 2024-04-25T08:54:17.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.152+0000 7f0403fff640 1 -- 172.21.15.43:0/772494610 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f040c00ec20 con 0x7f0410049000 2024-04-25T08:54:17.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.152+0000 7f0403fff640 1 -- 172.21.15.43:0/772494610 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f040c01c400 con 0x7f0410049000 2024-04-25T08:54:17.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.152+0000 7ffb66334640 1 -- 172.21.15.43:0/3622186461 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb60155df0 con 0x7ffb600b3d60 2024-04-25T08:54:17.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.153+0000 7ffb66334640 1 -- 172.21.15.43:0/3622186461 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb601563b0 con 0x7ffb600b3d60 2024-04-25T08:54:17.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.153+0000 7f041c862640 1 -- 172.21.15.43:0/772494610 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0410049770 con 0x7f0410049000 2024-04-25T08:54:17.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.153+0000 7f04ce7fc640 1 -- 172.21.15.43:0/758027632 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f04d4011070 con 0x7f04d004a390 2024-04-25T08:54:17.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.153+0000 7f041c862640 1 -- 172.21.15.43:0/772494610 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0410049d30 con 0x7f0410049000 2024-04-25T08:54:17.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.153+0000 7fc8fd92d640 1 --2- 172.21.15.43:0/2898367433 >> [v2:172.21.15.43:6808/1175991178,v1:172.21.15.43:6809/1175991178] conn(0x7fc8dc0737c0 0x7fc8dc075be0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.154+0000 7f04ce7fc640 1 --2- 172.21.15.43:0/758027632 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f04bc06b5a0 0x7f04bc06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.154+0000 7f04ce7fc640 1 -- 172.21.15.43:0/758027632 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f04d4089d50 con 0x7f04d004a390 2024-04-25T08:54:17.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.154+0000 7f04ce7fc640 1 --2- 172.21.15.43:0/758027632 >> [v2:172.21.15.193:6800/3423021177,v1:172.21.15.193:6801/3423021177] conn(0x7f04bc073820 0x7f04bc075c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.154+0000 7f04ce7fc640 1 -- 172.21.15.43:0/758027632 --> [v2:172.21.15.193:6800/3423021177,v1:172.21.15.193:6801/3423021177] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f04d400ee50 con 0x7f04bc073820 2024-04-25T08:54:17.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.153+0000 7f2545ffb640 1 -- 172.21.15.43:0/491436607 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f254001a070 con 0x7f254803c0a0 2024-04-25T08:54:17.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.154+0000 7f04ce7fc640 1 -- 172.21.15.43:0/758027632 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04d408c3b0 con 0x7f04d004a390 2024-04-25T08:54:17.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.154+0000 7f2545ffb640 1 --2- 172.21.15.43:0/491436607 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f253406b5a0 0x7f253406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.154+0000 7f2545ffb640 1 -- 172.21.15.43:0/491436607 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f25400895e0 con 0x7f254803c0a0 2024-04-25T08:54:17.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.154+0000 7f2545ffb640 1 --2- 172.21.15.43:0/491436607 >> [v2:172.21.15.43:6817/252044399,v1:172.21.15.43:6819/252044399] conn(0x7f2534073820 0x7f2534075c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.154+0000 7ffb66334640 1 -- 172.21.15.43:0/3622186461 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffb40001030 con 0x7ffb600b3d60 2024-04-25T08:54:17.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.154+0000 7f2545ffb640 1 -- 172.21.15.43:0/491436607 --> [v2:172.21.15.43:6817/252044399,v1:172.21.15.43:6819/252044399] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f25340762f0 con 0x7f2534073820 2024-04-25T08:54:17.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.154+0000 7f2545ffb640 1 -- 172.21.15.43:0/491436607 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2540090050 con 0x7f254803c0a0 2024-04-25T08:54:17.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.155+0000 7f04de2b0640 1 --2- 172.21.15.43:0/758027632 >> [v2:172.21.15.193:6800/3423021177,v1:172.21.15.193:6801/3423021177] conn(0x7f04bc073820 0x7f04bc075c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.155+0000 7f04ddaaf640 1 --2- 172.21.15.43:0/758027632 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f04bc06b5a0 0x7f04bc06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.155+0000 7f08d2ffd640 1 -- 172.21.15.43:0/1485790192 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f08ec1f3840 con 0x7f08ec150af0 2024-04-25T08:54:17.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.155+0000 7fc8edffb640 1 -- 172.21.15.43:0/2898367433 <== osd.2 v2:172.21.15.43:6808/1175991178 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc8e803de50 con 0x7fc8dc0737c0 2024-04-25T08:54:17.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.155+0000 7f34c1ffb640 1 -- 172.21.15.43:0/2312740723 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f34c801b020 con 0x7f34cc15bb80 2024-04-25T08:54:17.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.155+0000 7f2554a06640 1 --2- 172.21.15.43:0/491436607 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f253406b5a0 0x7f253406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.155+0000 7f08e97fa640 1 -- 172.21.15.43:0/1485790192 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f08dc018020 con 0x7f08ec150af0 2024-04-25T08:54:17.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.155+0000 7f2555a08640 1 --2- 172.21.15.43:0/491436607 >> [v2:172.21.15.43:6817/252044399,v1:172.21.15.43:6819/252044399] conn(0x7f2534073820 0x7f2534075c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.155+0000 7fc8fc92b640 1 --2- 172.21.15.43:0/2898367433 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc8dc06b5a0 0x7fc8dc06da60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4006fd0 tx=0x7fc8f4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.155+0000 7f2555a08640 1 --2- 172.21.15.43:0/491436607 >> [v2:172.21.15.43:6817/252044399,v1:172.21.15.43:6819/252044399] conn(0x7f2534073820 0x7f2534075c40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.155+0000 7f04ddaaf640 1 --2- 172.21.15.43:0/758027632 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f04bc06b5a0 0x7f04bc06da60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f04d8050470 tx=0x7f04d80a9000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.155+0000 7f04de2b0640 1 --2- 172.21.15.43:0/758027632 >> [v2:172.21.15.193:6800/3423021177,v1:172.21.15.193:6801/3423021177] conn(0x7f04bc073820 0x7f04bc075c40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.155+0000 7ffb5cff9640 1 -- 172.21.15.43:0/3622186461 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffb58065070 con 0x7ffb600b3d60 2024-04-25T08:54:17.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.156+0000 7f34c1ffb640 1 --2- 172.21.15.43:0/2312740723 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f34b006f9b0 0x7f34b0071e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.156+0000 7f2554a06640 1 --2- 172.21.15.43:0/491436607 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f253406b5a0 0x7f253406da60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2548052480 tx=0x7f254c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.156+0000 7f08e97fa640 1 --2- 172.21.15.43:0/1485790192 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f08d806b5a0 0x7f08d806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.156+0000 7f34c1ffb640 1 -- 172.21.15.43:0/2312740723 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f34c808ac20 con 0x7f34cc15bb80 2024-04-25T08:54:17.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.153+0000 7fef757bc640 1 --2- 172.21.15.43:0/513698317 >> [v2:172.21.15.43:6800/1101549916,v1:172.21.15.43:6801/1101549916] conn(0x7fef54073740 0x7fef54075b60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.156+0000 7fef6d7fa640 1 -- 172.21.15.43:0/513698317 <== osd.3 v2:172.21.15.43:6800/1101549916 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fef54076210 con 0x7fef54073740 2024-04-25T08:54:17.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.156+0000 7f0403fff640 1 -- 172.21.15.43:0/772494610 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f040c01c5a0 con 0x7f0410049000 2024-04-25T08:54:17.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.156+0000 7f0401ffb640 1 -- 172.21.15.43:0/772494610 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0410005390 con 0x7f0410049000 2024-04-25T08:54:17.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.156+0000 7f34d0dc5640 1 --2- 172.21.15.43:0/2312740723 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f34b006f9b0 0x7f34b0071e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.156+0000 7ffb5cff9640 1 --2- 172.21.15.43:0/3622186461 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ffb4c06b5a0 0x7ffb4c06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.156+0000 7f34c1ffb640 1 --2- 172.21.15.43:0/2312740723 >> [v2:172.21.15.193:6808/3086001447,v1:172.21.15.193:6809/3086001447] conn(0x7f34b0077bd0 0x7f34b0079ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.156+0000 7f34c1ffb640 1 -- 172.21.15.43:0/2312740723 --> [v2:172.21.15.193:6808/3086001447,v1:172.21.15.193:6809/3086001447] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f34b007a6a0 con 0x7f34b0077bd0 2024-04-25T08:54:17.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.156+0000 7f34c1ffb640 1 -- 172.21.15.43:0/2312740723 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34c808b010 con 0x7f34cc15bb80 2024-04-25T08:54:17.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.156+0000 7f08f09f9640 1 --2- 172.21.15.43:0/1485790192 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f08d806b5a0 0x7f08d806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.156+0000 7ffb5cff9640 1 -- 172.21.15.43:0/3622186461 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffb58030020 con 0x7ffb600b3d60 2024-04-25T08:54:17.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.156+0000 7f2545ffb640 1 -- 172.21.15.43:0/491436607 <== osd.0 v2:172.21.15.43:6817/252044399 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f25340762f0 con 0x7f2534073820 2024-04-25T08:54:17.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.156+0000 7f08e97fa640 1 -- 172.21.15.43:0/1485790192 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f08dc08a050 con 0x7f08ec150af0 2024-04-25T08:54:17.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.156+0000 7ffb5f7fe640 1 --2- 172.21.15.43:0/3622186461 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ffb4c06b5a0 0x7ffb4c06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.156+0000 7ffb5cff9640 1 --2- 172.21.15.43:0/3622186461 >> [v2:172.21.15.193:6821/1737328270,v1:172.21.15.193:6823/1737328270] conn(0x7ffb4c0737c0 0x7ffb4c075be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.156+0000 7f34d1dc7640 1 --2- 172.21.15.43:0/2312740723 >> [v2:172.21.15.193:6808/3086001447,v1:172.21.15.193:6809/3086001447] conn(0x7f34b0077bd0 0x7f34b0079ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.157+0000 7f08e97fa640 1 --2- 172.21.15.43:0/1485790192 >> [v2:172.21.15.43:6816/179790819,v1:172.21.15.43:6818/179790819] conn(0x7f08d80737c0 0x7f08d8075be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.157+0000 7ffb5cff9640 1 -- 172.21.15.43:0/3622186461 --> [v2:172.21.15.193:6821/1737328270,v1:172.21.15.193:6823/1737328270] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffb4c076290 con 0x7ffb4c0737c0 2024-04-25T08:54:17.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.157+0000 7ffb5cff9640 1 -- 172.21.15.43:0/3622186461 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb580abc30 con 0x7ffb600b3d60 2024-04-25T08:54:17.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.157+0000 7f0403fff640 1 --2- 172.21.15.43:0/772494610 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f03fc06b520 0x7f03fc06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.157+0000 7f0403fff640 1 -- 172.21.15.43:0/772494610 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f040c08a740 con 0x7f0410049000 2024-04-25T08:54:17.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.157+0000 7f0403fff640 1 --2- 172.21.15.43:0/772494610 >> [v2:172.21.15.193:6816/77587867,v1:172.21.15.193:6817/77587867] conn(0x7f03fc073740 0x7f03fc075b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.157+0000 7f0403fff640 1 -- 172.21.15.43:0/772494610 --> [v2:172.21.15.193:6816/77587867,v1:172.21.15.193:6817/77587867] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f040c013ea0 con 0x7f03fc073740 2024-04-25T08:54:17.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.157+0000 7f0403fff640 1 -- 172.21.15.43:0/772494610 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f040c091050 con 0x7f0410049000 2024-04-25T08:54:17.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.157+0000 7f04ce7fc640 1 -- 172.21.15.43:0/758027632 <== osd.5 v2:172.21.15.193:6800/3423021177 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f04d400ee50 con 0x7f04bc073820 2024-04-25T08:54:17.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.157+0000 7ffb648d1640 1 --2- 172.21.15.43:0/3622186461 >> [v2:172.21.15.193:6821/1737328270,v1:172.21.15.193:6823/1737328270] conn(0x7ffb4c0737c0 0x7ffb4c075be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.157+0000 7f08e97fa640 1 -- 172.21.15.43:0/1485790192 --> [v2:172.21.15.43:6816/179790819,v1:172.21.15.43:6818/179790819] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f08dc007e50 con 0x7f08d80737c0 2024-04-25T08:54:17.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.157+0000 7f08e97fa640 1 -- 172.21.15.43:0/1485790192 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f08dc091050 con 0x7f08ec150af0 2024-04-25T08:54:17.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.157+0000 7f08f09f9640 1 --2- 172.21.15.43:0/1485790192 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f08d806b5a0 0x7f08d806da60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f08e400bcb0 tx=0x7f08e400b000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.157+0000 7f04177fe640 1 --2- 172.21.15.43:0/772494610 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f03fc06b520 0x7f03fc06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.157+0000 7f08ebfff640 1 --2- 172.21.15.43:0/1485790192 >> [v2:172.21.15.43:6816/179790819,v1:172.21.15.43:6818/179790819] conn(0x7f08d80737c0 0x7f08d8075be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.157+0000 7ffb5f7fe640 1 --2- 172.21.15.43:0/3622186461 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ffb4c06b5a0 0x7ffb4c06da60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ffb60153ae0 tx=0x7ffb500079b0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.157+0000 7f34d0dc5640 1 --2- 172.21.15.43:0/2312740723 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f34b006f9b0 0x7f34b0071e70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f34c4000ba0 tx=0x7f34c400b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.157+0000 7f34d1dc7640 1 --2- 172.21.15.43:0/2312740723 >> [v2:172.21.15.193:6808/3086001447,v1:172.21.15.193:6809/3086001447] conn(0x7f34b0077bd0 0x7f34b0079ff0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.157+0000 7f0417fff640 1 --2- 172.21.15.43:0/772494610 >> [v2:172.21.15.193:6816/77587867,v1:172.21.15.193:6817/77587867] conn(0x7f03fc073740 0x7f03fc075b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.158+0000 7ffb648d1640 1 --2- 172.21.15.43:0/3622186461 >> [v2:172.21.15.193:6821/1737328270,v1:172.21.15.193:6823/1737328270] conn(0x7ffb4c0737c0 0x7ffb4c075be0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.158+0000 7f08ebfff640 1 --2- 172.21.15.43:0/1485790192 >> [v2:172.21.15.43:6816/179790819,v1:172.21.15.43:6818/179790819] conn(0x7f08d80737c0 0x7f08d8075be0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.158+0000 7f0417fff640 1 --2- 172.21.15.43:0/772494610 >> [v2:172.21.15.193:6816/77587867,v1:172.21.15.193:6817/77587867] conn(0x7f03fc073740 0x7f03fc075b60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.158+0000 7f34c1ffb640 1 -- 172.21.15.43:0/2312740723 <== osd.7 v2:172.21.15.193:6808/3086001447 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f34b007a6a0 con 0x7f34b0077bd0 2024-04-25T08:54:17.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.158+0000 7f04177fe640 1 --2- 172.21.15.43:0/772494610 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f03fc06b520 0x7f03fc06d9e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0408004710 tx=0x7f04180aa000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.158+0000 7ffb5cff9640 1 -- 172.21.15.43:0/3622186461 <== osd.6 v2:172.21.15.193:6821/1737328270 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ffb4c076290 con 0x7ffb4c0737c0 2024-04-25T08:54:17.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.159+0000 7f08e97fa640 1 -- 172.21.15.43:0/1485790192 <== osd.1 v2:172.21.15.43:6816/179790819 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f08dc007e50 con 0x7f08d80737c0 2024-04-25T08:54:17.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.159+0000 7f0403fff640 1 -- 172.21.15.43:0/772494610 <== osd.4 v2:172.21.15.193:6816/77587867 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f040c013ea0 con 0x7f03fc073740 2024-04-25T08:54:17.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.188+0000 7fc8fe12e640 1 -- 172.21.15.43:0/2898367433 --> [v2:172.21.15.43:6808/1175991178,v1:172.21.15.43:6809/1175991178] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc8f80b40a0 con 0x7fc8dc0737c0 2024-04-25T08:54:17.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.189+0000 7fef7721f640 1 -- 172.21.15.43:0/513698317 --> [v2:172.21.15.43:6800/1101549916,v1:172.21.15.43:6801/1101549916] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fef50002dd0 con 0x7fef54073740 2024-04-25T08:54:17.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.189+0000 7fc8edffb640 1 -- 172.21.15.43:0/2898367433 <== osd.2 v2:172.21.15.43:6808/1175991178 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc8f80b40a0 con 0x7fc8dc0737c0 2024-04-25T08:54:17.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.189+0000 7fef6d7fa640 1 -- 172.21.15.43:0/513698317 <== osd.3 v2:172.21.15.43:6800/1101549916 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fef50002dd0 con 0x7fef54073740 2024-04-25T08:54:17.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.189+0000 7f2556209640 1 -- 172.21.15.43:0/491436607 --> [v2:172.21.15.43:6817/252044399,v1:172.21.15.43:6819/252044399] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2548038b90 con 0x7f2534073820 2024-04-25T08:54:17.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.189+0000 7fc8e37fe640 1 -- 172.21.15.43:0/2898367433 >> [v2:172.21.15.43:6808/1175991178,v1:172.21.15.43:6809/1175991178] conn(0x7fc8dc0737c0 msgr2=0x7fc8dc075be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.189+0000 7fc8e37fe640 1 --2- 172.21.15.43:0/2898367433 >> [v2:172.21.15.43:6808/1175991178,v1:172.21.15.43:6809/1175991178] conn(0x7fc8dc0737c0 0x7fc8dc075be0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.190+0000 7f2545ffb640 1 -- 172.21.15.43:0/491436607 <== osd.0 v2:172.21.15.43:6817/252044399 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2548038b90 con 0x7f2534073820 2024-04-25T08:54:17.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.190+0000 7fef7721f640 1 -- 172.21.15.43:0/513698317 >> [v2:172.21.15.43:6800/1101549916,v1:172.21.15.43:6801/1101549916] conn(0x7fef54073740 msgr2=0x7fef54075b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.190+0000 7fef7721f640 1 --2- 172.21.15.43:0/513698317 >> [v2:172.21.15.43:6800/1101549916,v1:172.21.15.43:6801/1101549916] conn(0x7fef54073740 0x7fef54075b60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.190+0000 7fc8e37fe640 1 -- 172.21.15.43:0/2898367433 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc8dc06b5a0 msgr2=0x7fc8dc06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.190+0000 7fc8e37fe640 1 --2- 172.21.15.43:0/2898367433 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc8dc06b5a0 0x7fc8dc06da60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc8f4006fd0 tx=0x7fc8f4008040 comp rx=0 tx=0).stop 2024-04-25T08:54:17.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.190+0000 7fc8e37fe640 1 -- 172.21.15.43:0/2898367433 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc8f80b3720 msgr2=0x7fc8f81597a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.190+0000 7fc8e37fe640 1 --2- 172.21.15.43:0/2898367433 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc8f80b3720 0x7fc8f81597a0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8000c00 tx=0x7fc8e80046c0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.190+0000 7fc8fd92d640 1 -- 172.21.15.43:0/2898367433 reap_dead start 2024-04-25T08:54:17.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.190+0000 7fef7721f640 1 -- 172.21.15.43:0/513698317 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fef5406b5a0 msgr2=0x7fef5406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.190+0000 7fef7721f640 1 --2- 172.21.15.43:0/513698317 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fef5406b5a0 0x7fef5406da60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fef7015ca00 tx=0x7fef5c009000 comp rx=0 tx=0).stop 2024-04-25T08:54:17.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.190+0000 7fef7721f640 1 -- 172.21.15.43:0/513698317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef700ae730 msgr2=0x7fef702034c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.190+0000 7fef7721f640 1 --2- 172.21.15.43:0/513698317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef700ae730 0x7fef702034c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fef6400e870 tx=0x7fef6400ed30 comp rx=0 tx=0).stop 2024-04-25T08:54:17.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.190+0000 7fef757bc640 1 -- 172.21.15.43:0/513698317 reap_dead start 2024-04-25T08:54:17.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.190+0000 7f2556209640 1 -- 172.21.15.43:0/491436607 >> [v2:172.21.15.43:6817/252044399,v1:172.21.15.43:6819/252044399] conn(0x7f2534073820 msgr2=0x7f2534075c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.190+0000 7f2556209640 1 --2- 172.21.15.43:0/491436607 >> [v2:172.21.15.43:6817/252044399,v1:172.21.15.43:6819/252044399] conn(0x7f2534073820 0x7f2534075c40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.190+0000 7fef7721f640 1 -- 172.21.15.43:0/513698317 shutdown_connections 2024-04-25T08:54:17.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.190+0000 7f04deab1640 1 -- 172.21.15.43:0/758027632 --> [v2:172.21.15.193:6800/3423021177,v1:172.21.15.193:6801/3423021177] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f04c0002dd0 con 0x7f04bc073820 2024-04-25T08:54:17.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.190+0000 7fef7721f640 1 -- 172.21.15.43:0/513698317 >> 172.21.15.43:0/513698317 conn(0x7fef7006ab30 msgr2=0x7fef70070a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:17.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.191+0000 7fc8e37fe640 1 -- 172.21.15.43:0/2898367433 shutdown_connections 2024-04-25T08:54:17.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.191+0000 7fc8e37fe640 1 -- 172.21.15.43:0/2898367433 >> 172.21.15.43:0/2898367433 conn(0x7fc8f806cb40 msgr2=0x7fc8f80668f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:17.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.191+0000 7fef7721f640 1 -- 172.21.15.43:0/513698317 shutdown_connections 2024-04-25T08:54:17.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.191+0000 7f2556209640 1 -- 172.21.15.43:0/491436607 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f253406b5a0 msgr2=0x7f253406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.191+0000 7f2556209640 1 --2- 172.21.15.43:0/491436607 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f253406b5a0 0x7f253406da60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2548052480 tx=0x7f254c008040 comp rx=0 tx=0).stop 2024-04-25T08:54:17.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.191+0000 7fef7721f640 1 -- 172.21.15.43:0/513698317 wait complete. 2024-04-25T08:54:17.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.191+0000 7f2556209640 1 -- 172.21.15.43:0/491436607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f254803c0a0 msgr2=0x7f25480e0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.191+0000 7f2556209640 1 --2- 172.21.15.43:0/491436607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f254803c0a0 0x7f25480e0360 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f254000c3c0 tx=0x7f254000c880 comp rx=0 tx=0).stop 2024-04-25T08:54:17.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.191+0000 7f2555a08640 1 -- 172.21.15.43:0/491436607 reap_dead start 2024-04-25T08:54:17.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.191+0000 7f2556209640 1 -- 172.21.15.43:0/491436607 shutdown_connections 2024-04-25T08:54:17.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.191+0000 7f2556209640 1 -- 172.21.15.43:0/491436607 >> 172.21.15.43:0/491436607 conn(0x7f2548058ee0 msgr2=0x7f2548006f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:17.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.191+0000 7f04ce7fc640 1 -- 172.21.15.43:0/758027632 <== osd.5 v2:172.21.15.193:6800/3423021177 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f04c0002dd0 con 0x7f04bc073820 2024-04-25T08:54:17.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.191+0000 7fc8e37fe640 1 -- 172.21.15.43:0/2898367433 shutdown_connections 2024-04-25T08:54:17.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.191+0000 7fc8e37fe640 1 -- 172.21.15.43:0/2898367433 wait complete. 2024-04-25T08:54:17.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.191+0000 7f2556209640 1 -- 172.21.15.43:0/491436607 shutdown_connections 2024-04-25T08:54:17.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.191+0000 7f04deab1640 1 -- 172.21.15.43:0/758027632 >> [v2:172.21.15.193:6800/3423021177,v1:172.21.15.193:6801/3423021177] conn(0x7f04bc073820 msgr2=0x7f04bc075c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.191+0000 7f04deab1640 1 --2- 172.21.15.43:0/758027632 >> [v2:172.21.15.193:6800/3423021177,v1:172.21.15.193:6801/3423021177] conn(0x7f04bc073820 0x7f04bc075c40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.192+0000 7f2556209640 1 -- 172.21.15.43:0/491436607 wait complete. 2024-04-25T08:54:17.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.192+0000 7f04deab1640 1 -- 172.21.15.43:0/758027632 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f04bc06b5a0 msgr2=0x7f04bc06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.192+0000 7f04deab1640 1 --2- 172.21.15.43:0/758027632 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f04bc06b5a0 0x7f04bc06da60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f04d8050470 tx=0x7f04d80a9000 comp rx=0 tx=0).stop 2024-04-25T08:54:17.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.192+0000 7f04deab1640 1 -- 172.21.15.43:0/758027632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04d004a390 msgr2=0x7f04d00365a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.192+0000 7f04deab1640 1 --2- 172.21.15.43:0/758027632 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04d004a390 0x7f04d00365a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f04d400c390 tx=0x7f04d400c850 comp rx=0 tx=0).stop 2024-04-25T08:54:17.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.192+0000 7f04de2b0640 1 -- 172.21.15.43:0/758027632 reap_dead start 2024-04-25T08:54:17.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.192+0000 7f04deab1640 1 -- 172.21.15.43:0/758027632 shutdown_connections 2024-04-25T08:54:17.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.192+0000 7f04deab1640 1 -- 172.21.15.43:0/758027632 >> 172.21.15.43:0/758027632 conn(0x7f04d0020040 msgr2=0x7f04d000c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:17.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.192+0000 7ffb66334640 1 -- 172.21.15.43:0/3622186461 --> [v2:172.21.15.193:6821/1737328270,v1:172.21.15.193:6823/1737328270] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ffb40002dd0 con 0x7ffb4c0737c0 2024-04-25T08:54:17.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.192+0000 7f0401ffb640 1 -- 172.21.15.43:0/772494610 --> [v2:172.21.15.193:6816/77587867,v1:172.21.15.193:6817/77587867] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f04100d1c90 con 0x7f03fc073740 2024-04-25T08:54:17.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.192+0000 7f04deab1640 1 -- 172.21.15.43:0/758027632 shutdown_connections 2024-04-25T08:54:17.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.192+0000 7f04deab1640 1 -- 172.21.15.43:0/758027632 wait complete. 2024-04-25T08:54:17.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.192+0000 7f08d2ffd640 1 -- 172.21.15.43:0/1485790192 --> [v2:172.21.15.43:6816/179790819,v1:172.21.15.43:6818/179790819] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f08ec074e80 con 0x7f08d80737c0 2024-04-25T08:54:17.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.193+0000 7f0403fff640 1 -- 172.21.15.43:0/772494610 <== osd.4 v2:172.21.15.193:6816/77587867 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f04100d1c90 con 0x7f03fc073740 2024-04-25T08:54:17.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.193+0000 7f08e97fa640 1 -- 172.21.15.43:0/1485790192 <== osd.1 v2:172.21.15.43:6816/179790819 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f08ec074e80 con 0x7f08d80737c0 2024-04-25T08:54:17.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.193+0000 7ffb5cff9640 1 -- 172.21.15.43:0/3622186461 <== osd.6 v2:172.21.15.193:6821/1737328270 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ffb40002dd0 con 0x7ffb4c0737c0 2024-04-25T08:54:17.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.193+0000 7f34d382a640 1 -- 172.21.15.43:0/2312740723 --> [v2:172.21.15.193:6808/3086001447,v1:172.21.15.193:6809/3086001447] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f34cc072e80 con 0x7f34b0077bd0 2024-04-25T08:54:17.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.193+0000 7f0401ffb640 1 -- 172.21.15.43:0/772494610 >> [v2:172.21.15.193:6816/77587867,v1:172.21.15.193:6817/77587867] conn(0x7f03fc073740 msgr2=0x7f03fc075b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.193+0000 7f0401ffb640 1 --2- 172.21.15.43:0/772494610 >> [v2:172.21.15.193:6816/77587867,v1:172.21.15.193:6817/77587867] conn(0x7f03fc073740 0x7f03fc075b60 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.193+0000 7f34c1ffb640 1 -- 172.21.15.43:0/2312740723 <== osd.7 v2:172.21.15.193:6808/3086001447 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f34cc072e80 con 0x7f34b0077bd0 2024-04-25T08:54:17.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.194+0000 7f0401ffb640 1 -- 172.21.15.43:0/772494610 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f03fc06b520 msgr2=0x7f03fc06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.194+0000 7f0401ffb640 1 --2- 172.21.15.43:0/772494610 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f03fc06b520 0x7f03fc06d9e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0408004710 tx=0x7f04180aa000 comp rx=0 tx=0).stop 2024-04-25T08:54:17.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.194+0000 7f0401ffb640 1 -- 172.21.15.43:0/772494610 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0410049000 msgr2=0x7f04100d0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.194+0000 7f0401ffb640 1 --2- 172.21.15.43:0/772494610 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0410049000 0x7f04100d0d80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f040c010930 tx=0x7f040c010df0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.194+0000 7f08d2ffd640 1 -- 172.21.15.43:0/1485790192 >> [v2:172.21.15.43:6816/179790819,v1:172.21.15.43:6818/179790819] conn(0x7f08d80737c0 msgr2=0x7f08d8075be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.194+0000 7f08d2ffd640 1 --2- 172.21.15.43:0/1485790192 >> [v2:172.21.15.43:6816/179790819,v1:172.21.15.43:6818/179790819] conn(0x7f08d80737c0 0x7f08d8075be0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.194+0000 7f08d2ffd640 1 -- 172.21.15.43:0/1485790192 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f08d806b5a0 msgr2=0x7f08d806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.194+0000 7f08d2ffd640 1 --2- 172.21.15.43:0/1485790192 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f08d806b5a0 0x7f08d806da60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f08e400bcb0 tx=0x7f08e400b000 comp rx=0 tx=0).stop 2024-04-25T08:54:17.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.194+0000 7f08d2ffd640 1 -- 172.21.15.43:0/1485790192 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08ec150af0 msgr2=0x7f08ec13fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.194+0000 7f08d2ffd640 1 --2- 172.21.15.43:0/1485790192 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08ec150af0 0x7f08ec13fa20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f08dc00b440 tx=0x7f08dc00b900 comp rx=0 tx=0).stop 2024-04-25T08:54:17.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.194+0000 7f08f11fa640 1 -- 172.21.15.43:0/1485790192 reap_dead start 2024-04-25T08:54:17.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.194+0000 7f0417fff640 1 -- 172.21.15.43:0/772494610 reap_dead start 2024-04-25T08:54:17.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.194+0000 7f0401ffb640 1 -- 172.21.15.43:0/772494610 shutdown_connections 2024-04-25T08:54:17.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.194+0000 7f0401ffb640 1 -- 172.21.15.43:0/772494610 >> 172.21.15.43:0/772494610 conn(0x7f0410009fb0 msgr2=0x7f04100d5920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:17.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.194+0000 7f0401ffb640 1 -- 172.21.15.43:0/772494610 shutdown_connections 2024-04-25T08:54:17.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.194+0000 7f0401ffb640 1 -- 172.21.15.43:0/772494610 wait complete. 2024-04-25T08:54:17.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.194+0000 7ffb66334640 1 -- 172.21.15.43:0/3622186461 >> [v2:172.21.15.193:6821/1737328270,v1:172.21.15.193:6823/1737328270] conn(0x7ffb4c0737c0 msgr2=0x7ffb4c075be0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.194+0000 7ffb66334640 1 --2- 172.21.15.43:0/3622186461 >> [v2:172.21.15.193:6821/1737328270,v1:172.21.15.193:6823/1737328270] conn(0x7ffb4c0737c0 0x7ffb4c075be0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.194+0000 7f34af7fe640 1 -- 172.21.15.43:0/2312740723 >> [v2:172.21.15.193:6808/3086001447,v1:172.21.15.193:6809/3086001447] conn(0x7f34b0077bd0 msgr2=0x7f34b0079ff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.194+0000 7f34af7fe640 1 --2- 172.21.15.43:0/2312740723 >> [v2:172.21.15.193:6808/3086001447,v1:172.21.15.193:6809/3086001447] conn(0x7f34b0077bd0 0x7f34b0079ff0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.195+0000 7ffb66334640 1 -- 172.21.15.43:0/3622186461 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ffb4c06b5a0 msgr2=0x7ffb4c06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.195+0000 7f08d2ffd640 1 -- 172.21.15.43:0/1485790192 shutdown_connections 2024-04-25T08:54:17.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.195+0000 7f08d2ffd640 1 -- 172.21.15.43:0/1485790192 >> 172.21.15.43:0/1485790192 conn(0x7f08ec06ab30 msgr2=0x7f08ec150070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:17.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.195+0000 7ffb66334640 1 --2- 172.21.15.43:0/3622186461 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ffb4c06b5a0 0x7ffb4c06da60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ffb60153ae0 tx=0x7ffb500079b0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.195+0000 7f34af7fe640 1 -- 172.21.15.43:0/2312740723 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f34b006f9b0 msgr2=0x7f34b0071e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.195+0000 7f34af7fe640 1 --2- 172.21.15.43:0/2312740723 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f34b006f9b0 0x7f34b0071e70 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f34c4000ba0 tx=0x7f34c400b040 comp rx=0 tx=0).stop 2024-04-25T08:54:17.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.195+0000 7ffb66334640 1 -- 172.21.15.43:0/3622186461 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffb600b3d60 msgr2=0x7ffb601e3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.195+0000 7ffb66334640 1 --2- 172.21.15.43:0/3622186461 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffb600b3d60 0x7ffb601e3930 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ffb5802ced0 tx=0x7ffb58004a40 comp rx=0 tx=0).stop 2024-04-25T08:54:17.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.195+0000 7ffb648d1640 1 -- 172.21.15.43:0/3622186461 reap_dead start 2024-04-25T08:54:17.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.195+0000 7f34af7fe640 1 -- 172.21.15.43:0/2312740723 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f34cc15bb80 msgr2=0x7f34cc14b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.195+0000 7f34af7fe640 1 --2- 172.21.15.43:0/2312740723 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f34cc15bb80 0x7f34cc14b2c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f34c800d800 tx=0x7f34c800dcc0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.195+0000 7ffb66334640 1 -- 172.21.15.43:0/3622186461 shutdown_connections 2024-04-25T08:54:17.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.195+0000 7ffb66334640 1 -- 172.21.15.43:0/3622186461 >> 172.21.15.43:0/3622186461 conn(0x7ffb600b1480 msgr2=0x7ffb600aee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:17.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.195+0000 7f34d1dc7640 1 -- 172.21.15.43:0/2312740723 reap_dead start 2024-04-25T08:54:17.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.195+0000 7f34af7fe640 1 -- 172.21.15.43:0/2312740723 shutdown_connections 2024-04-25T08:54:17.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.195+0000 7f34af7fe640 1 -- 172.21.15.43:0/2312740723 >> 172.21.15.43:0/2312740723 conn(0x7f34cc0b3170 msgr2=0x7f34cc0b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:17.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.195+0000 7f08d2ffd640 1 -- 172.21.15.43:0/1485790192 shutdown_connections 2024-04-25T08:54:17.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.195+0000 7f08d2ffd640 1 -- 172.21.15.43:0/1485790192 wait complete. 2024-04-25T08:54:17.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.195+0000 7ffb66334640 1 -- 172.21.15.43:0/3622186461 shutdown_connections 2024-04-25T08:54:17.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.195+0000 7ffb66334640 1 -- 172.21.15.43:0/3622186461 wait complete. 2024-04-25T08:54:17.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.195+0000 7f34af7fe640 1 -- 172.21.15.43:0/2312740723 shutdown_connections 2024-04-25T08:54:17.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.195+0000 7f34af7fe640 1 -- 172.21.15.43:0/2312740723 wait complete. 2024-04-25T08:54:17.251 INFO:teuthology.orchestra.run.smithi043.stdout:55834574854 2024-04-25T08:54:17.252 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T08:54:17.252 INFO:teuthology.orchestra.run.smithi043.stdout:55834574855 2024-04-25T08:54:17.253 INFO:teuthology.orchestra.run.smithi043.stdout:55834574854 2024-04-25T08:54:17.253 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T08:54:17.253 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T08:54:17.254 INFO:teuthology.orchestra.run.smithi043.stdout:55834574856 2024-04-25T08:54:17.255 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T08:54:17.255 INFO:teuthology.orchestra.run.smithi043.stdout:55834574854 2024-04-25T08:54:17.256 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T08:54:17.256 INFO:teuthology.orchestra.run.smithi043.stdout:55834574854 2024-04-25T08:54:17.257 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T08:54:17.257 INFO:teuthology.orchestra.run.smithi043.stdout:55834574853 2024-04-25T08:54:17.258 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T08:54:17.258 INFO:teuthology.orchestra.run.smithi043.stdout:55834574854 2024-04-25T08:54:17.259 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T08:54:17.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.476+0000 7f849450a640 1 Processor -- start 2024-04-25T08:54:17.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.476+0000 7f849450a640 1 -- start start 2024-04-25T08:54:17.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.477+0000 7f7fa0a6b640 1 Processor -- start 2024-04-25T08:54:17.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.477+0000 7f7fa0a6b640 1 -- start start 2024-04-25T08:54:17.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.478+0000 7f2bc359e640 1 Processor -- start 2024-04-25T08:54:17.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.478+0000 7f849450a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f848c0b6130 0x7f848c0d4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.478+0000 7f849450a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f848c0cf310 con 0x7f848c0b5d50 2024-04-25T08:54:17.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.478+0000 7f849450a640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f848c0cf5f0 con 0x7f848c0d20b0 2024-04-25T08:54:17.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.478+0000 7f849450a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f848c0b6940 con 0x7f848c0b6130 2024-04-25T08:54:17.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.478+0000 7f84922a6640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f848c0b5d50 0x7f848c0d19a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60872/0 (socket says 172.21.15.43:60872) 2024-04-25T08:54:17.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.478+0000 7f84922a6640 1 -- 172.21.15.43:0/1714085082 learned_addr learned my addr 172.21.15.43:0/1714085082 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:17.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.478+0000 7f8491aa5640 1 --2- 172.21.15.43:0/1714085082 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f848c0b6130 0x7f848c0d4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.478+0000 7fb93ba1f640 1 Processor -- start 2024-04-25T08:54:17.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.478+0000 7fb93ba1f640 1 -- start start 2024-04-25T08:54:17.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.479+0000 7f84912a4640 1 -- 172.21.15.43:0/1714085082 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2249358760 0 0) 0x7f848c0cf5f0 con 0x7f848c0d20b0 2024-04-25T08:54:17.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.479+0000 7f84912a4640 1 -- 172.21.15.43:0/1714085082 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8478003520 con 0x7f848c0d20b0 2024-04-25T08:54:17.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.479+0000 7f8491aa5640 1 -- 172.21.15.43:0/1714085082 >> v1:172.21.15.193:6789/0 conn(0x7f848c0d20b0 legacy=0x7f848c0b6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.479+0000 7f8491aa5640 1 -- 172.21.15.43:0/1714085082 >> v1:172.21.15.43:6789/0 conn(0x7f848c0b5d50 legacy=0x7f848c0d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.479+0000 7f8491aa5640 1 -- 172.21.15.43:0/1714085082 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f848c0cf8d0 con 0x7f848c0b6130 2024-04-25T08:54:17.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.479+0000 7f7fa0a6b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7f9c15f180 0x7f7f9c0cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.479+0000 7f7f9a59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7f9c15f180 0x7f7f9c0cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.479+0000 7f7fa0a6b640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f7f9c0b23f0 0x7f7f9c0cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.479+0000 7f7fa0a6b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7f9c0b2cc0 0x7f7f9c15f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.479+0000 7f7f9a59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7f9c15f180 0x7f7f9c0cf860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37460/0 (socket says 172.21.15.43:37460) 2024-04-25T08:54:17.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.479+0000 7f7fa0a6b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f9c0cccc0 con 0x7f7f9c15f180 2024-04-25T08:54:17.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.479+0000 7f7f9a59c640 1 -- 172.21.15.43:0/1931528541 learned_addr learned my addr 172.21.15.43:0/1931528541 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:17.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.479+0000 7f7fa0a6b640 1 -- 172.21.15.43:0/1931528541 --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f9c0cd840 con 0x7f7f9c0b23f0 2024-04-25T08:54:17.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.479+0000 7f7fa0a6b640 1 -- 172.21.15.43:0/1931528541 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c0ccf10 con 0x7f7f9c0b2cc0 2024-04-25T08:54:17.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.480+0000 7f7f9a59c640 1 -- 172.21.15.43:0/1931528541 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7f9c0b2cc0 msgr2=0x7f7f9c15f8e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T08:54:17.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.480+0000 7f7f9a59c640 1 --2- 172.21.15.43:0/1931528541 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7f9c0b2cc0 0x7f7f9c15f8e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.480+0000 7f7f9a59c640 1 -- 172.21.15.43:0/1931528541 >> v2:172.21.15.193:3300/0 conn(0x7f7f9c0b23f0 msgr2=0x7f7f9c0cfda0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T08:54:17.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.480+0000 7f7f99d9b640 1 --2- 172.21.15.43:0/1931528541 >> v2:172.21.15.193:3300/0 conn(0x7f7f9c0b23f0 0x7f7f9c0cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.480+0000 7f7f9a59c640 1 --2- 172.21.15.43:0/1931528541 >> v2:172.21.15.193:3300/0 conn(0x7f7f9c0b23f0 0x7f7f9c0cfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.480+0000 7f7f9a59c640 1 -- 172.21.15.43:0/1931528541 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f9c0cd1f0 con 0x7f7f9c15f180 2024-04-25T08:54:17.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.480+0000 7f57a38db640 1 Processor -- start 2024-04-25T08:54:17.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.479+0000 7f2bc359e640 1 -- start start 2024-04-25T08:54:17.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.480+0000 7f20d6458640 1 Processor -- start 2024-04-25T08:54:17.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.480+0000 7f20d6458640 1 -- start start 2024-04-25T08:54:17.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.480+0000 7f2bc359e640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f2bbc00a450 0x7f2bbc055da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.479+0000 7fb93ba1f640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb92c00c090 0x7fb92c038070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.479+0000 7fb93ba1f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb92c053c60 0x7fb92c0defa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.479+0000 7fb93ba1f640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb92c0541d0 con 0x7fb92c0df4e0 2024-04-25T08:54:17.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.479+0000 7fb93ba1f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb92c005f20 con 0x7fb92c053c60 2024-04-25T08:54:17.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.479+0000 7fb93ba1f640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb92c006200 con 0x7fb92c00c090 2024-04-25T08:54:17.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.480+0000 7fb9397bb640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb92c00c090 0x7fb92c038070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.480+0000 7fb939fbc640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7fb92c0df4e0 0x7fb92c04fa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:36374/0 (socket says 172.21.15.43:36374) 2024-04-25T08:54:17.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.480+0000 7fb939fbc640 1 -- 172.21.15.43:0/1685906050 learned_addr learned my addr 172.21.15.43:0/1685906050 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:17.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.480+0000 7fb92bfff640 1 -- 172.21.15.43:0/1685906050 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1982105449 0 0) 0x7fb92c0541d0 con 0x7fb92c0df4e0 2024-04-25T08:54:17.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.480+0000 7fb92bfff640 1 -- 172.21.15.43:0/1685906050 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb920003520 con 0x7fb92c0df4e0 2024-04-25T08:54:17.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.480+0000 7fb938fba640 1 --2- 172.21.15.43:0/1685906050 >> v2:172.21.15.43:3300/0 conn(0x7fb92c053c60 0x7fb92c0defa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.480+0000 7f2bc359e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bbc054d00 con 0x7f2bbc00ab30 2024-04-25T08:54:17.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.480+0000 7f2bc359e640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bbc054fe0 con 0x7f2bbc0e1300 2024-04-25T08:54:17.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.480+0000 7f2bc359e640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bbc055910 con 0x7f2bbc00a450 2024-04-25T08:54:17.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.480+0000 7f2bc1d9b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f2bbc00ab30 0x7f2bbc009d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60882/0 (socket says 172.21.15.43:60882) 2024-04-25T08:54:17.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.481+0000 7f2bc1d9b640 1 -- 172.21.15.43:0/966987933 learned_addr learned my addr 172.21.15.43:0/966987933 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:17.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.481+0000 7fb938fba640 1 -- 172.21.15.43:0/1685906050 >> v2:172.21.15.193:3300/0 conn(0x7fb92c00c090 msgr2=0x7fb92c038070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:17.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.481+0000 7fb938fba640 1 --2- 172.21.15.43:0/1685906050 >> v2:172.21.15.193:3300/0 conn(0x7fb92c00c090 0x7fb92c038070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.481+0000 7fb938fba640 1 -- 172.21.15.43:0/1685906050 >> v1:172.21.15.193:6789/0 conn(0x7fb92c0df4e0 legacy=0x7fb92c04fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.481+0000 7fb938fba640 1 -- 172.21.15.43:0/1685906050 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb92c0064e0 con 0x7fb92c053c60 2024-04-25T08:54:17.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.480+0000 7f57a38db640 1 -- start start 2024-04-25T08:54:17.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.481+0000 7f8491aa5640 1 --2- 172.21.15.43:0/1714085082 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f848c0b6130 0x7f848c0d4550 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f84880097d0 tx=0x7f848802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=674cf89dbf7a283b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.481+0000 7f2bc159a640 1 -- 172.21.15.43:0/966987933 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 120456746 0 0) 0x7f2bbc054d00 con 0x7f2bbc00ab30 2024-04-25T08:54:17.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.481+0000 7f2bc2d9d640 1 --2- 172.21.15.43:0/966987933 >> v2:172.21.15.193:3300/0 conn(0x7f2bbc00a450 0x7f2bbc055da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.481+0000 7f2bc159a640 1 -- 172.21.15.43:0/966987933 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bac003520 con 0x7f2bbc00ab30 2024-04-25T08:54:17.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.481+0000 7f2bc159a640 1 -- 172.21.15.43:0/966987933 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3424383970 0 0) 0x7f2bbc054fe0 con 0x7f2bbc0e1300 2024-04-25T08:54:17.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.481+0000 7f2bc159a640 1 -- 172.21.15.43:0/966987933 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bac0048e0 con 0x7f2bbc0e1300 2024-04-25T08:54:17.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.481+0000 7f20d6458640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20c803e830 0x7f20c8053be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.481+0000 7f20d6458640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f20c8054120 0x7f20c80068c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.481+0000 7f9b44074640 1 Processor -- start 2024-04-25T08:54:17.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.481+0000 7f84912a4640 1 -- 172.21.15.43:0/1714085082 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8488037070 con 0x7f848c0b6130 2024-04-25T08:54:17.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.481+0000 7f84912a4640 1 -- 172.21.15.43:0/1714085082 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8488032440 con 0x7f848c0b6130 2024-04-25T08:54:17.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.481+0000 7f84912a4640 1 -- 172.21.15.43:0/1714085082 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8488032810 con 0x7f848c0b6130 2024-04-25T08:54:17.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.481+0000 7f57a38db640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f5794007170 0x7f5794053850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.481+0000 7f57a38db640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5794051810 con 0x7f57940093e0 2024-04-25T08:54:17.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.482+0000 7f57a38db640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5794050770 con 0x7f57940e77f0 2024-04-25T08:54:17.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.482+0000 7f57a38db640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5794049600 con 0x7f5794007170 2024-04-25T08:54:17.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.482+0000 7f9b44074640 1 -- start start 2024-04-25T08:54:17.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.482+0000 7f20cf7fe640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20c803e830 0x7f20c8053be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.482+0000 7f20d49f5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f20c8054120 0x7f20c80068c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.482+0000 7f57a1e78640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f57940093e0 0x7f5794053d90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60888/0 (socket says 172.21.15.43:60888) 2024-04-25T08:54:17.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.482+0000 7f57a1e78640 1 -- 172.21.15.43:0/2527811454 learned_addr learned my addr 172.21.15.43:0/2527811454 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:17.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.482+0000 7f57a0e76640 1 --2- 172.21.15.43:0/2527811454 >> v2:172.21.15.193:3300/0 conn(0x7f5794007170 0x7f5794053850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.482+0000 7f2bc159a640 1 -- 172.21.15.43:0/966987933 <== mon.1 v1:172.21.15.193:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3262683003 0 0) 0x7f2bac0048e0 con 0x7f2bbc0e1300 2024-04-25T08:54:17.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.482+0000 7f2bc159a640 1 -- 172.21.15.43:0/966987933 >> v2:172.21.15.193:3300/0 conn(0x7f2bbc00a450 msgr2=0x7f2bbc055da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:17.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.482+0000 7f2bc159a640 1 --2- 172.21.15.43:0/966987933 >> v2:172.21.15.193:3300/0 conn(0x7f2bbc00a450 0x7f2bbc055da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.482+0000 7f2bc159a640 1 -- 172.21.15.43:0/966987933 >> v1:172.21.15.43:6789/0 conn(0x7f2bbc00ab30 legacy=0x7f2bbc009d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.482+0000 7f2bc159a640 1 -- 172.21.15.43:0/966987933 --> v1:172.21.15.193:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bbc009b50 con 0x7f2bbc0e1300 2024-04-25T08:54:17.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.482+0000 7f2bc159a640 1 -- 172.21.15.43:0/966987933 <== mon.1 v1:172.21.15.193:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f2bb4001670 con 0x7f2bbc0e1300 2024-04-25T08:54:17.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.482+0000 7f2bc159a640 1 -- 172.21.15.43:0/966987933 >> v1:172.21.15.193:6789/0 conn(0x7f2bbc0e1300 legacy=0x7f2bbc03a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.482+0000 7f2bc159a640 1 --2- 172.21.15.43:0/966987933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bac004d90 0x7f2bac0051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.482+0000 7f2bc159a640 1 --2- 172.21.15.43:0/966987933 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2bac0037f0 0x7f2bac003c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.482+0000 7f2bc159a640 1 --2- 172.21.15.43:0/966987933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2bac0042a0 0x7f2bac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.482+0000 7f2bc159a640 1 -- 172.21.15.43:0/966987933 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc055910 con 0x7f2bac004d90 2024-04-25T08:54:17.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.482+0000 7f2bc159a640 1 -- 172.21.15.43:0/966987933 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bac00c250 con 0x7f2bac0042a0 2024-04-25T08:54:17.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.482+0000 7f2bc159a640 1 -- 172.21.15.43:0/966987933 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bac00c450 con 0x7f2bac0037f0 2024-04-25T08:54:17.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.482+0000 7f20cf7fe640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20c803e830 0x7f20c8053be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:48982/0 (socket says 172.21.15.43:48982) 2024-04-25T08:54:17.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.482+0000 7f20cf7fe640 1 -- 172.21.15.43:0/737178648 learned_addr learned my addr 172.21.15.43:0/737178648 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:17.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.482+0000 7fb938fba640 1 --2- 172.21.15.43:0/1685906050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb92c053c60 0x7fb92c0defa0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb92c00b090 tx=0x7fb92403b1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=245189c8b9141602 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.481+0000 7f1213cc4640 1 Processor -- start 2024-04-25T08:54:17.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.481+0000 7f1213cc4640 1 -- start start 2024-04-25T08:54:17.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.482+0000 7f2bc2d9d640 1 --2- 172.21.15.43:0/966987933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2bac0042a0 0x7f2bac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.482+0000 7f2bc1d9b640 1 --2- 172.21.15.43:0/966987933 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2bac0037f0 0x7f2bac003c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.482+0000 7f5793fff640 1 -- 172.21.15.43:0/2527811454 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 651996444 0 0) 0x7f5794051810 con 0x7f57940093e0 2024-04-25T08:54:17.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.482+0000 7f7f9a59c640 1 --2- 172.21.15.43:0/1931528541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f9c15f180 0x7f7f9c0cf860 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c071a40 tx=0x7f7f8402a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=194be28b24dd3a6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.482+0000 7f5793fff640 1 -- 172.21.15.43:0/2527811454 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5788003520 con 0x7f57940093e0 2024-04-25T08:54:17.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.482+0000 7fb92bfff640 1 -- 172.21.15.43:0/1685906050 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb924048070 con 0x7fb92c053c60 2024-04-25T08:54:17.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.482+0000 7fb92bfff640 1 -- 172.21.15.43:0/1685906050 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb924043440 con 0x7fb92c053c60 2024-04-25T08:54:17.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.482+0000 7fb92bfff640 1 -- 172.21.15.43:0/1685906050 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb924043850 con 0x7fb92c053c60 2024-04-25T08:54:17.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.482+0000 7f2bc259c640 1 --2- 172.21.15.43:0/966987933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bac004d90 0x7f2bac0051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.483+0000 7f2bc1d9b640 1 -- 172.21.15.43:0/966987933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2bac0042a0 msgr2=0x7f2bac00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:17.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.483+0000 7f2bc1d9b640 1 --2- 172.21.15.43:0/966987933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2bac0042a0 0x7f2bac00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.483+0000 7f2bc1d9b640 1 -- 172.21.15.43:0/966987933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bac004d90 msgr2=0x7f2bac0051b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:17.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.483+0000 7f2bc1d9b640 1 --2- 172.21.15.43:0/966987933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bac004d90 0x7f2bac0051b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.483+0000 7f2bc1d9b640 1 -- 172.21.15.43:0/966987933 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bbc009b50 con 0x7f2bac0037f0 2024-04-25T08:54:17.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.483+0000 7f2bc2d9d640 1 -- 172.21.15.43:0/966987933 reap_dead start 2024-04-25T08:54:17.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.482+0000 7f20d49f5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f20c8054120 0x7f20c80068c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37476/0 (socket says 172.21.15.43:37476) 2024-04-25T08:54:17.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.483+0000 7f7f9959a640 1 -- 172.21.15.43:0/1931528541 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f84037070 con 0x7f7f9c15f180 2024-04-25T08:54:17.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.483+0000 7f7f9959a640 1 -- 172.21.15.43:0/1931528541 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f84032440 con 0x7f7f9c15f180 2024-04-25T08:54:17.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.483+0000 7f7f9959a640 1 -- 172.21.15.43:0/1931528541 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f84032810 con 0x7f7f9c15f180 2024-04-25T08:54:17.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.482+0000 7f20d6458640 1 -- 172.21.15.43:0/737178648 --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20c8058560 con 0x7f20c80e30b0 2024-04-25T08:54:17.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.483+0000 7f20d6458640 1 -- 172.21.15.43:0/737178648 --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20c8058840 con 0x7f20c8054120 2024-04-25T08:54:17.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.483+0000 7f1213cc4640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f12040363a0 0x7f1204054370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.483+0000 7f1213cc4640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f120404a250 con 0x7f1204038e10 2024-04-25T08:54:17.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.483+0000 7f1213cc4640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f120404a4c0 con 0x7f1204038a30 2024-04-25T08:54:17.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.483+0000 7f1213cc4640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1204053270 con 0x7f12040363a0 2024-04-25T08:54:17.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.483+0000 7f121125f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f12040363a0 0x7f1204054370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.483+0000 7f121125f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f12040363a0 0x7f1204054370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:48996/0 (socket says 172.21.15.43:48996) 2024-04-25T08:54:17.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.483+0000 7f121125f640 1 -- 172.21.15.43:0/1919225652 learned_addr learned my addr 172.21.15.43:0/1919225652 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:17.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.483+0000 7f20d6458640 1 -- 172.21.15.43:0/737178648 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c8058b20 con 0x7f20c803e830 2024-04-25T08:54:17.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.483+0000 7f849450a640 1 -- 172.21.15.43:0/1714085082 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f848c0b6130 msgr2=0x7f848c0d4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.483+0000 7f9b44074640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b3c0b6300 0x7f9b3c0b53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.483+0000 7f849450a640 1 --2- 172.21.15.43:0/1714085082 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f848c0b6130 0x7f848c0d4550 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f84880097d0 tx=0x7f848802a180 comp rx=0 tx=0).stop 2024-04-25T08:54:17.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.482+0000 7f5793fff640 1 -- 172.21.15.43:0/2527811454 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3524839830 0 0) 0x7f5794050770 con 0x7f57940e77f0 2024-04-25T08:54:17.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.483+0000 7f9b44074640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9b3c072280 0x7f9b3c072710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.483+0000 7fb93ba1f640 1 -- 172.21.15.43:0/1685906050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb92c053c60 msgr2=0x7fb92c0defa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.483+0000 7fb93ba1f640 1 --2- 172.21.15.43:0/1685906050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb92c053c60 0x7fb92c0defa0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb92c00b090 tx=0x7fb92403b1f0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.483+0000 7f849450a640 1 -- 172.21.15.43:0/1714085082 shutdown_connections 2024-04-25T08:54:17.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.482+0000 7f5793fff640 1 -- 172.21.15.43:0/2527811454 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5794051810 con 0x7f57940e77f0 2024-04-25T08:54:17.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.483+0000 7f57a0e76640 1 -- 172.21.15.43:0/2527811454 >> v1:172.21.15.193:6789/0 conn(0x7f57940e77f0 legacy=0x7f5794006a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.483+0000 7f57a0e76640 1 -- 172.21.15.43:0/2527811454 >> v1:172.21.15.43:6789/0 conn(0x7f57940093e0 legacy=0x7f5794053d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.483+0000 7f57a0e76640 1 -- 172.21.15.43:0/2527811454 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5794050a50 con 0x7f5794007170 2024-04-25T08:54:17.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.483+0000 7f121125f640 1 -- 172.21.15.43:0/1919225652 >> v1:172.21.15.193:6789/0 conn(0x7f1204038a30 legacy=0x7f1204007c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:17.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.483+0000 7f121125f640 1 -- 172.21.15.43:0/1919225652 >> v1:172.21.15.43:6789/0 conn(0x7f1204038e10 legacy=0x7f1204008340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:17.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.483+0000 7f121125f640 1 -- 172.21.15.43:0/1919225652 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1204053550 con 0x7f12040363a0 2024-04-25T08:54:17.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.484+0000 7f9b4160f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b3c0b6300 0x7f9b3c0b53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.484+0000 7f9b4160f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b3c0b6300 0x7f9b3c0b53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:49010/0 (socket says 172.21.15.43:49010) 2024-04-25T08:54:17.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.484+0000 7f9b4160f640 1 -- 172.21.15.43:0/3544001237 learned_addr learned my addr 172.21.15.43:0/3544001237 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:17.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.483+0000 7f849450a640 1 --2- 172.21.15.43:0/1714085082 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f848c0b6130 0x7f848c0d4550 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.484+0000 7f9b44074640 1 -- 172.21.15.43:0/3544001237 --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b3c0b5910 con 0x7f9b3c15bb50 2024-04-25T08:54:17.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.484+0000 7f9b44074640 1 -- 172.21.15.43:0/3544001237 --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b3c0b46a0 con 0x7f9b3c072280 2024-04-25T08:54:17.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.484+0000 7f9b44074640 1 -- 172.21.15.43:0/3544001237 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3c0b6970 con 0x7f9b3c0b6300 2024-04-25T08:54:17.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.483+0000 7f849450a640 1 -- 172.21.15.43:0/1714085082 >> 172.21.15.43:0/1714085082 conn(0x7f848c0d3a30 msgr2=0x7f848c0d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:17.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.484+0000 7f2bc1d9b640 1 --2- 172.21.15.43:0/966987933 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2bac0037f0 0x7f2bac003c70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc04a610 tx=0x7f2bac005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ef44ef057053a9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.484+0000 7f849450a640 1 -- 172.21.15.43:0/1714085082 shutdown_connections 2024-04-25T08:54:17.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.484+0000 7f849450a640 1 -- 172.21.15.43:0/1714085082 wait complete. 2024-04-25T08:54:17.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.484+0000 7f7fa0a6b640 1 -- 172.21.15.43:0/1931528541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f9c15f180 msgr2=0x7f7f9c0cf860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.484+0000 7f57a0e76640 1 --2- 172.21.15.43:0/2527811454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5794007170 0x7f5794053850 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f579c02d370 tx=0x7f579c0a5120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f5649d076b8d2a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.484+0000 7f7fa0a6b640 1 --2- 172.21.15.43:0/1931528541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f9c15f180 0x7f7f9c0cf860 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c071a40 tx=0x7f7f8402a160 comp rx=0 tx=0).stop 2024-04-25T08:54:17.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.484+0000 7f5793fff640 1 -- 172.21.15.43:0/2527811454 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c0b2070 con 0x7f5794007170 2024-04-25T08:54:17.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.484+0000 7f5793fff640 1 -- 172.21.15.43:0/2527811454 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f579c0a5ce0 con 0x7f5794007170 2024-04-25T08:54:17.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.484+0000 7f2bc159a640 1 -- 172.21.15.43:0/966987933 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8001550 con 0x7f2bac0037f0 2024-04-25T08:54:17.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.484+0000 7f5793fff640 1 -- 172.21.15.43:0/2527811454 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579c0ad640 con 0x7f5794007170 2024-04-25T08:54:17.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.484+0000 7fb93ba1f640 1 -- 172.21.15.43:0/1685906050 shutdown_connections 2024-04-25T08:54:17.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.484+0000 7f2bc159a640 1 -- 172.21.15.43:0/966987933 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb802e070 con 0x7f2bac0037f0 2024-04-25T08:54:17.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.484+0000 7f2bc159a640 1 -- 172.21.15.43:0/966987933 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8001550 con 0x7f2bac0037f0 2024-04-25T08:54:17.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.484+0000 7f20ceffd640 1 -- 172.21.15.43:0/737178648 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2344354813 0 0) 0x7f20c8058560 con 0x7f20c80e30b0 2024-04-25T08:54:17.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.484+0000 7fb93ba1f640 1 --2- 172.21.15.43:0/1685906050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb92c053c60 0x7fb92c0defa0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.484+0000 7f20ceffd640 1 -- 172.21.15.43:0/737178648 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20c0003520 con 0x7f20c80e30b0 2024-04-25T08:54:17.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.484+0000 7fb93ba1f640 1 --2- 172.21.15.43:0/1685906050 >> v2:172.21.15.193:3300/0 conn(0x7fb92c00c090 0x7fb92c038070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.484+0000 7f121125f640 1 --2- 172.21.15.43:0/1919225652 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f12040363a0 0x7f1204054370 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f120c062780 tx=0x7f120c0a4f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=241f40ccef3208f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.484+0000 7f20cf7fe640 1 -- 172.21.15.43:0/737178648 >> v2:172.21.15.43:3300/0 conn(0x7f20c8054120 msgr2=0x7f20c80068c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:17.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.485+0000 7f20cf7fe640 1 --2- 172.21.15.43:0/737178648 >> v2:172.21.15.43:3300/0 conn(0x7f20c8054120 0x7f20c80068c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.485+0000 7f20cf7fe640 1 -- 172.21.15.43:0/737178648 >> v1:172.21.15.43:6789/0 conn(0x7f20c80e30b0 legacy=0x7f20c8057d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.484+0000 7f9b4160f640 1 -- 172.21.15.43:0/3544001237 >> v2:172.21.15.43:3300/0 conn(0x7f9b3c072280 msgr2=0x7f9b3c072710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:17.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.484+0000 7f9b4160f640 1 --2- 172.21.15.43:0/3544001237 >> v2:172.21.15.43:3300/0 conn(0x7f9b3c072280 0x7f9b3c072710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.484+0000 7f9b4160f640 1 -- 172.21.15.43:0/3544001237 >> v1:172.21.15.43:6789/0 conn(0x7f9b3c15bb50 legacy=0x7f9b3c070520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:17.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.484+0000 7f9b4160f640 1 -- 172.21.15.43:0/3544001237 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b3c0b48f0 con 0x7f9b3c0b6300 2024-04-25T08:54:17.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.485+0000 7f20cf7fe640 1 -- 172.21.15.43:0/737178648 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c80063a0 con 0x7f20c803e830 2024-04-25T08:54:17.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.484+0000 7fb93ba1f640 1 -- 172.21.15.43:0/1685906050 >> 172.21.15.43:0/1685906050 conn(0x7fb92c009fa0 msgr2=0x7fb92c00a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:17.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.484+0000 7fb93ba1f640 1 -- 172.21.15.43:0/1685906050 shutdown_connections 2024-04-25T08:54:17.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.484+0000 7fb93ba1f640 1 -- 172.21.15.43:0/1685906050 wait complete. 2024-04-25T08:54:17.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.485+0000 7f1210a5e640 1 -- 172.21.15.43:0/1919225652 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120c0b2070 con 0x7f12040363a0 2024-04-25T08:54:17.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.485+0000 7f1210a5e640 1 -- 172.21.15.43:0/1919225652 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f120c0a5cd0 con 0x7f12040363a0 2024-04-25T08:54:17.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.485+0000 7f1210a5e640 1 -- 172.21.15.43:0/1919225652 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120c0ad640 con 0x7f12040363a0 2024-04-25T08:54:17.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.485+0000 7f9b4160f640 1 --2- 172.21.15.43:0/3544001237 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b3c0b6300 0x7f9b3c0b53d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9b38009870 tx=0x7f9b3802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7942cfa03dc4b79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.485+0000 7f2bc359e640 1 -- 172.21.15.43:0/966987933 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2bac0037f0 msgr2=0x7f2bac003c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.485+0000 7f2bc359e640 1 --2- 172.21.15.43:0/966987933 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2bac0037f0 0x7f2bac003c70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc04a610 tx=0x7f2bac005830 comp rx=0 tx=0).stop 2024-04-25T08:54:17.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.485+0000 7f7fa0a6b640 1 -- 172.21.15.43:0/1931528541 shutdown_connections 2024-04-25T08:54:17.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.485+0000 7f7fa0a6b640 1 --2- 172.21.15.43:0/1931528541 >> v2:172.21.15.193:3300/0 conn(0x7f7f9c0b23f0 0x7f7f9c0cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.485+0000 7f7fa0a6b640 1 --2- 172.21.15.43:0/1931528541 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7f9c0b2cc0 0x7f7f9c15f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.485+0000 7f7fa0a6b640 1 --2- 172.21.15.43:0/1931528541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f9c15f180 0x7f7f9c0cf860 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.485+0000 7f7fa0a6b640 1 -- 172.21.15.43:0/1931528541 >> 172.21.15.43:0/1931528541 conn(0x7f7f9c0d17f0 msgr2=0x7f7f9c0d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:17.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.486+0000 7f2bc359e640 1 -- 172.21.15.43:0/966987933 shutdown_connections 2024-04-25T08:54:17.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.486+0000 7f2bc359e640 1 --2- 172.21.15.43:0/966987933 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2bac0037f0 0x7f2bac003c70 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.486+0000 7f2bc359e640 1 -- 172.21.15.43:0/966987933 >> 172.21.15.43:0/966987933 conn(0x7f2bbc056cc0 msgr2=0x7f2bbc0570d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:17.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.486+0000 7f9b40e0e640 1 -- 172.21.15.43:0/3544001237 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b38037070 con 0x7f9b3c0b6300 2024-04-25T08:54:17.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.486+0000 7f9b40e0e640 1 -- 172.21.15.43:0/3544001237 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b38032440 con 0x7f9b3c0b6300 2024-04-25T08:54:17.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.486+0000 7f9b40e0e640 1 -- 172.21.15.43:0/3544001237 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b38032810 con 0x7f9b3c0b6300 2024-04-25T08:54:17.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.486+0000 7f20cf7fe640 1 --2- 172.21.15.43:0/737178648 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20c803e830 0x7f20c8053be0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f20bc009870 tx=0x7f20bc02a2a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcab715804923e47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.486+0000 7f849450a640 1 Processor -- start 2024-04-25T08:54:17.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.486+0000 7f849450a640 1 -- start start 2024-04-25T08:54:17.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.486+0000 7f7fa0a6b640 1 -- 172.21.15.43:0/1931528541 shutdown_connections 2024-04-25T08:54:17.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.486+0000 7f57a38db640 1 -- 172.21.15.43:0/2527811454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5794007170 msgr2=0x7f5794053850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.486+0000 7f57a38db640 1 --2- 172.21.15.43:0/2527811454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5794007170 0x7f5794053850 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f579c02d370 tx=0x7f579c0a5120 comp rx=0 tx=0).stop 2024-04-25T08:54:17.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.486+0000 7f1213cc4640 1 -- 172.21.15.43:0/1919225652 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f12040363a0 msgr2=0x7f1204054370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.486+0000 7f1213cc4640 1 --2- 172.21.15.43:0/1919225652 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f12040363a0 0x7f1204054370 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f120c062780 tx=0x7f120c0a4f00 comp rx=0 tx=0).stop 2024-04-25T08:54:17.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.486+0000 7f20ceffd640 1 -- 172.21.15.43:0/737178648 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc037070 con 0x7f20c803e830 2024-04-25T08:54:17.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.486+0000 7f7fa0a6b640 1 -- 172.21.15.43:0/1931528541 wait complete. 2024-04-25T08:54:17.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.486+0000 7f1213cc4640 1 -- 172.21.15.43:0/1919225652 shutdown_connections 2024-04-25T08:54:17.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.486+0000 7f1213cc4640 1 --2- 172.21.15.43:0/1919225652 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f12040363a0 0x7f1204054370 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.486+0000 7f1213cc4640 1 -- 172.21.15.43:0/1919225652 >> 172.21.15.43:0/1919225652 conn(0x7f12040537c0 msgr2=0x7f1204053bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:17.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.486+0000 7f57a38db640 1 -- 172.21.15.43:0/2527811454 shutdown_connections 2024-04-25T08:54:17.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.486+0000 7f57a38db640 1 --2- 172.21.15.43:0/2527811454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5794007170 0x7f5794053850 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.486+0000 7f57a38db640 1 -- 172.21.15.43:0/2527811454 >> 172.21.15.43:0/2527811454 conn(0x7f5794055c80 msgr2=0x7f5794056090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:17.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.486+0000 7fb93ba1f640 1 Processor -- start 2024-04-25T08:54:17.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.486+0000 7fb93ba1f640 1 -- start start 2024-04-25T08:54:17.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.486+0000 7f20ceffd640 1 -- 172.21.15.43:0/737178648 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20bc032440 con 0x7f20c803e830 2024-04-25T08:54:17.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.486+0000 7f20ceffd640 1 -- 172.21.15.43:0/737178648 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc032810 con 0x7f20c803e830 2024-04-25T08:54:17.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.486+0000 7f1213cc4640 1 -- 172.21.15.43:0/1919225652 shutdown_connections 2024-04-25T08:54:17.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.487+0000 7f1213cc4640 1 -- 172.21.15.43:0/1919225652 wait complete. 2024-04-25T08:54:17.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.487+0000 7f9b44074640 1 -- 172.21.15.43:0/3544001237 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b3c0b6300 msgr2=0x7f9b3c0b53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.487+0000 7f9b44074640 1 --2- 172.21.15.43:0/3544001237 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b3c0b6300 0x7f9b3c0b53d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9b38009870 tx=0x7f9b3802a180 comp rx=0 tx=0).stop 2024-04-25T08:54:17.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.487+0000 7fb93ba1f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb92c00c090 0x7fb92c04e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.487+0000 7f9b44074640 1 -- 172.21.15.43:0/3544001237 shutdown_connections 2024-04-25T08:54:17.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.487+0000 7f9b44074640 1 --2- 172.21.15.43:0/3544001237 >> v2:172.21.15.43:3300/0 conn(0x7f9b3c072280 0x7f9b3c072710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.487+0000 7f9b44074640 1 --2- 172.21.15.43:0/3544001237 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b3c0b6300 0x7f9b3c0b53d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.487+0000 7f9b44074640 1 -- 172.21.15.43:0/3544001237 >> 172.21.15.43:0/3544001237 conn(0x7f9b3c0b33a0 msgr2=0x7f9b3c0b37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:17.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.487+0000 7f2bc359e640 1 -- 172.21.15.43:0/966987933 shutdown_connections 2024-04-25T08:54:17.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.487+0000 7f20d6458640 1 -- 172.21.15.43:0/737178648 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20c803e830 msgr2=0x7f20c8053be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.487+0000 7f20d6458640 1 --2- 172.21.15.43:0/737178648 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20c803e830 0x7f20c8053be0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f20bc009870 tx=0x7f20bc02a2a0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.487+0000 7f2bc359e640 1 -- 172.21.15.43:0/966987933 wait complete. 2024-04-25T08:54:17.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.487+0000 7fb93ba1f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb92c053c60 0x7fb92c0e13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.487+0000 7f57a38db640 1 -- 172.21.15.43:0/2527811454 shutdown_connections 2024-04-25T08:54:17.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.487+0000 7fb93ba1f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb92c0df4e0 0x7fb92c04c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.487+0000 7fb93ba1f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92c050310 con 0x7fb92c0df4e0 2024-04-25T08:54:17.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.487+0000 7fb93ba1f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92c050190 con 0x7fb92c053c60 2024-04-25T08:54:17.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.487+0000 7fb93ba1f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb92c033ee0 con 0x7fb92c00c090 2024-04-25T08:54:17.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.487+0000 7fb939fbc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb92c0df4e0 0x7fb92c04c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.487+0000 7fb939fbc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb92c0df4e0 0x7fb92c04c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37516/0 (socket says 172.21.15.43:37516) 2024-04-25T08:54:17.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.487+0000 7fb939fbc640 1 -- 172.21.15.43:0/1266145955 learned_addr learned my addr 172.21.15.43:0/1266145955 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:17.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.487+0000 7fb9397bb640 1 --2- 172.21.15.43:0/1266145955 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb92c00c090 0x7fb92c04e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.487+0000 7fb938fba640 1 --2- 172.21.15.43:0/1266145955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb92c053c60 0x7fb92c0e13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.488+0000 7fb9397bb640 1 -- 172.21.15.43:0/1266145955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb92c053c60 msgr2=0x7fb92c0e13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.488+0000 7fb9397bb640 1 --2- 172.21.15.43:0/1266145955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb92c053c60 0x7fb92c0e13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.488+0000 7fb9397bb640 1 -- 172.21.15.43:0/1266145955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb92c0df4e0 msgr2=0x7fb92c04c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.488+0000 7fb9397bb640 1 --2- 172.21.15.43:0/1266145955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb92c0df4e0 0x7fb92c04c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.488+0000 7fb9397bb640 1 -- 172.21.15.43:0/1266145955 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb92c0e1c00 con 0x7fb92c00c090 2024-04-25T08:54:17.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.488+0000 7fb939fbc640 1 --2- 172.21.15.43:0/1266145955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb92c0df4e0 0x7fb92c04c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:54:17.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.488+0000 7f57a38db640 1 -- 172.21.15.43:0/2527811454 wait complete. 2024-04-25T08:54:17.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.487+0000 7f849450a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f848c0b5d50 0x7f848c0c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.487+0000 7f849450a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f848c0b6130 0x7f848c152ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.487+0000 7f84922a6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f848c0b5d50 0x7f848c0c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.488+0000 7fb9397bb640 1 --2- 172.21.15.43:0/1266145955 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb92c00c090 0x7fb92c04e250 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb93404fcd0 tx=0x7fb934069260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.487+0000 7f84922a6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f848c0b5d50 0x7f848c0c7ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37510/0 (socket says 172.21.15.43:37510) 2024-04-25T08:54:17.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.487+0000 7f84922a6640 1 -- 172.21.15.43:0/2931471298 learned_addr learned my addr 172.21.15.43:0/2931471298 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:17.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.487+0000 7f849450a640 1 --2- 172.21.15.43:0/2931471298 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f848c0d20b0 0x7f848c0ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.487+0000 7f849450a640 1 -- 172.21.15.43:0/2931471298 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848c0d26c0 con 0x7f848c0b5d50 2024-04-25T08:54:17.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.487+0000 7f849450a640 1 -- 172.21.15.43:0/2931471298 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848c0d2540 con 0x7f848c0b6130 2024-04-25T08:54:17.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.487+0000 7f849450a640 1 -- 172.21.15.43:0/2931471298 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f848c0b74e0 con 0x7f848c0d20b0 2024-04-25T08:54:17.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.487+0000 7f8492aa7640 1 --2- 172.21.15.43:0/2931471298 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f848c0d20b0 0x7f848c0ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.488+0000 7f84922a6640 1 -- 172.21.15.43:0/2931471298 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f848c0d20b0 msgr2=0x7f848c0ca1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.488+0000 7f84922a6640 1 --2- 172.21.15.43:0/2931471298 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f848c0d20b0 0x7f848c0ca1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.488+0000 7f84922a6640 1 -- 172.21.15.43:0/2931471298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f848c0b6130 msgr2=0x7f848c152ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.488+0000 7f84922a6640 1 --2- 172.21.15.43:0/2931471298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f848c0b6130 0x7f848c152ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.488+0000 7f84922a6640 1 -- 172.21.15.43:0/2931471298 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f848c1533e0 con 0x7f848c0b5d50 2024-04-25T08:54:17.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.488+0000 7f84922a6640 1 --2- 172.21.15.43:0/2931471298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f848c0b5d50 0x7f848c0c7ee0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f847c00c390 tx=0x7f847c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.488+0000 7f7fa0a6b640 1 Processor -- start 2024-04-25T08:54:17.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.487+0000 7f20d6458640 1 -- 172.21.15.43:0/737178648 shutdown_connections 2024-04-25T08:54:17.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.487+0000 7f20d6458640 1 --2- 172.21.15.43:0/737178648 >> v2:172.21.15.43:3300/0 conn(0x7f20c8054120 0x7f20c80068c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.487+0000 7f20d6458640 1 --2- 172.21.15.43:0/737178648 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20c803e830 0x7f20c8053be0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.487+0000 7f20d6458640 1 -- 172.21.15.43:0/737178648 >> 172.21.15.43:0/737178648 conn(0x7f20c80559d0 msgr2=0x7f20c8055de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:17.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.488+0000 7f20d6458640 1 -- 172.21.15.43:0/737178648 shutdown_connections 2024-04-25T08:54:17.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.488+0000 7f20d6458640 1 -- 172.21.15.43:0/737178648 wait complete. 2024-04-25T08:54:17.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.488+0000 7fb929ffb640 1 -- 172.21.15.43:0/1266145955 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb934068520 con 0x7fb92c00c090 2024-04-25T08:54:17.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.488+0000 7fb93ba1f640 1 -- 172.21.15.43:0/1266145955 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb92c0e1ee0 con 0x7fb92c00c090 2024-04-25T08:54:17.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.488+0000 7f7fa0a6b640 1 -- start start 2024-04-25T08:54:17.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.489+0000 7fb93ba1f640 1 -- 172.21.15.43:0/1266145955 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb92c049570 con 0x7fb92c00c090 2024-04-25T08:54:17.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.489+0000 7fb929ffb640 1 -- 172.21.15.43:0/1266145955 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb93408f070 con 0x7fb92c00c090 2024-04-25T08:54:17.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.489+0000 7fb929ffb640 1 -- 172.21.15.43:0/1266145955 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb934094470 con 0x7fb92c00c090 2024-04-25T08:54:17.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.489+0000 7f9b44074640 1 -- 172.21.15.43:0/3544001237 shutdown_connections 2024-04-25T08:54:17.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.489+0000 7f9b44074640 1 -- 172.21.15.43:0/3544001237 wait complete. 2024-04-25T08:54:17.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.489+0000 7f2bc359e640 1 Processor -- start 2024-04-25T08:54:17.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.489+0000 7f57a38db640 1 Processor -- start 2024-04-25T08:54:17.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.490+0000 7f57a38db640 1 -- start start 2024-04-25T08:54:17.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.488+0000 7f8482ffd640 1 -- 172.21.15.43:0/2931471298 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847c017070 con 0x7f848c0b5d50 2024-04-25T08:54:17.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.488+0000 7f8482ffd640 1 -- 172.21.15.43:0/2931471298 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f847c005ce0 con 0x7f848c0b5d50 2024-04-25T08:54:17.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.489+0000 7f8482ffd640 1 -- 172.21.15.43:0/2931471298 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847c0072d0 con 0x7f848c0b5d50 2024-04-25T08:54:17.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.489+0000 7f849450a640 1 -- 172.21.15.43:0/2931471298 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f848c0ca730 con 0x7f848c0b5d50 2024-04-25T08:54:17.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.490+0000 7f20d6458640 1 Processor -- start 2024-04-25T08:54:17.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.490+0000 7f7fa0a6b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7f9c0b23f0 0x7f7f9c150c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.488+0000 7f1213cc4640 1 Processor -- start 2024-04-25T08:54:17.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.488+0000 7f1213cc4640 1 -- start start 2024-04-25T08:54:17.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.490+0000 7f1213cc4640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f12040363a0 0x7f12040e75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.490+0000 7f1213cc4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1204038a30 0x7f120404f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.490+0000 7f1213cc4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1204038e10 0x7f12040d90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.490+0000 7f1213cc4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1204049ea0 con 0x7f1204038e10 2024-04-25T08:54:17.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.490+0000 7f1213cc4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1204049d20 con 0x7f1204038a30 2024-04-25T08:54:17.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.490+0000 7f1213cc4640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1204052780 con 0x7f12040363a0 2024-04-25T08:54:17.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.490+0000 7f121125f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1204038a30 0x7f120404f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.490+0000 7f7fa0a6b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f9c0b2cc0 0x7f7f9c0b0470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.490+0000 7f20d6458640 1 -- start start 2024-04-25T08:54:17.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.490+0000 7f7f9a59c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7f9c0b23f0 0x7f7f9c150c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.490+0000 7f7fa0a6b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f9c15f180 0x7f7f9c06f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.490+0000 7f2bc359e640 1 -- start start 2024-04-25T08:54:17.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.490+0000 7f7fa0a6b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c15f560 con 0x7f7f9c0b2cc0 2024-04-25T08:54:17.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.490+0000 7f7fa0a6b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c0d23f0 con 0x7f7f9c15f180 2024-04-25T08:54:17.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.490+0000 7f7fa0a6b640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9c1511c0 con 0x7f7f9c0b23f0 2024-04-25T08:54:17.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.490+0000 7f7f99d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f9c0b2cc0 0x7f7f9c0b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.490+0000 7f7f9a59c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7f9c0b23f0 0x7f7f9c150c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:49048/0 (socket says 172.21.15.43:49048) 2024-04-25T08:54:17.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.490+0000 7f7f9a59c640 1 -- 172.21.15.43:0/671274414 learned_addr learned my addr 172.21.15.43:0/671274414 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:17.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.491+0000 7f7f99d9b640 1 -- 172.21.15.43:0/671274414 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7f9c0b23f0 msgr2=0x7f7f9c150c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.491+0000 7f7f99d9b640 1 --2- 172.21.15.43:0/671274414 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7f9c0b23f0 0x7f7f9c150c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.491+0000 7f7f99d9b640 1 -- 172.21.15.43:0/671274414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f9c15f180 msgr2=0x7f7f9c06f590 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T08:54:17.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.491+0000 7f7f99d9b640 1 --2- 172.21.15.43:0/671274414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f9c15f180 0x7f7f9c06f590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.491+0000 7f7f99d9b640 1 -- 172.21.15.43:0/671274414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f9c0c8c80 con 0x7f7f9c0b2cc0 2024-04-25T08:54:17.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.491+0000 7f7f9a59c640 1 --2- 172.21.15.43:0/671274414 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7f9c0b23f0 0x7f7f9c150c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:17.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.491+0000 7fb929ffb640 1 -- 172.21.15.43:0/1266145955 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb934094630 con 0x7fb92c00c090 2024-04-25T08:54:17.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.491+0000 7f7f99d9b640 1 --2- 172.21.15.43:0/671274414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f9c0b2cc0 0x7f7f9c0b0470 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c00c5e0 tx=0x7f7f8c00caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.490+0000 7f1211a60640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f12040363a0 0x7f12040e75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.490+0000 7f121125f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1204038a30 0x7f120404f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:46950/0 (socket says 172.21.15.43:46950) 2024-04-25T08:54:17.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.490+0000 7f121125f640 1 -- 172.21.15.43:0/3362869645 learned_addr learned my addr 172.21.15.43:0/3362869645 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:17.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.491+0000 7f9b44074640 1 Processor -- start 2024-04-25T08:54:17.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.491+0000 7f1212261640 1 --2- 172.21.15.43:0/3362869645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1204038e10 0x7f12040d90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.492+0000 7fb929ffb640 1 --2- 172.21.15.43:0/1266145955 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb91806b5a0 0x7fb91806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.492+0000 7f9b44074640 1 -- start start 2024-04-25T08:54:17.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.491+0000 7f20d6458640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20c803e830 0x7f20c80d6a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.491+0000 7f20d6458640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20c8054120 0x7f20c80dcad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.492+0000 7f20d6458640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20c80e30b0 0x7f20c80cc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.492+0000 7f7f92ffd640 1 -- 172.21.15.43:0/671274414 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c015070 con 0x7f7f9c0b2cc0 2024-04-25T08:54:17.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.492+0000 7f7f92ffd640 1 -- 172.21.15.43:0/671274414 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f8c004220 con 0x7f7f9c0b2cc0 2024-04-25T08:54:17.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.492+0000 7f7f92ffd640 1 -- 172.21.15.43:0/671274414 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8c003610 con 0x7f7f9c0b2cc0 2024-04-25T08:54:17.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.492+0000 7f20d6458640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c8006e00 con 0x7f20c80e30b0 2024-04-25T08:54:17.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.492+0000 7f20d6458640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c80e3490 con 0x7f20c803e830 2024-04-25T08:54:17.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.492+0000 7f20d6458640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c8006f80 con 0x7f20c8054120 2024-04-25T08:54:17.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.492+0000 7f7fa0a6b640 1 -- 172.21.15.43:0/671274414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f9c0c8f60 con 0x7f7f9c0b2cc0 2024-04-25T08:54:17.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.492+0000 7f2bc359e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2bac0037f0 0x7f2bbc04ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.492+0000 7fb929ffb640 1 -- 172.21.15.43:0/1266145955 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb934104a40 con 0x7fb92c00c090 2024-04-25T08:54:17.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.492+0000 7f2bc359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bbc0d31a0 0x7f2bbc0509d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.492+0000 7fb93ba1f640 1 -- 172.21.15.43:0/1266145955 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb914005290 con 0x7fb92c00c090 2024-04-25T08:54:17.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.492+0000 7fb938fba640 1 --2- 172.21.15.43:0/1266145955 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb91806b5a0 0x7fb91806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.492+0000 7f20cffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20c803e830 0x7f20c80d6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.492+0000 7f2bc359e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2bbc050f10 0x7f2bbc0cf090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.492+0000 7f20cf7fe640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20c8054120 0x7f20c80dcad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.492+0000 7f20cf7fe640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20c8054120 0x7f20c80dcad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:49056/0 (socket says 172.21.15.43:49056) 2024-04-25T08:54:17.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.492+0000 7f20cf7fe640 1 -- 172.21.15.43:0/2703390886 learned_addr learned my addr 172.21.15.43:0/2703390886 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:17.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.492+0000 7f57a38db640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5794007170 0x7f579417c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.492+0000 7f2bc259c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2bac0037f0 0x7f2bbc04ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.492+0000 7f2bc259c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2bac0037f0 0x7f2bbc04ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:49060/0 (socket says 172.21.15.43:49060) 2024-04-25T08:54:17.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.492+0000 7f2bc259c640 1 -- 172.21.15.43:0/347814099 learned_addr learned my addr 172.21.15.43:0/347814099 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:17.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.492+0000 7f2bc359e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc054890 con 0x7f2bbc0d31a0 2024-04-25T08:54:17.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.492+0000 7f2bc359e640 1 -- 172.21.15.43:0/347814099 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc0555a0 con 0x7f2bbc050f10 2024-04-25T08:54:17.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.492+0000 7f2bc359e640 1 -- 172.21.15.43:0/347814099 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bbc054a10 con 0x7f2bac0037f0 2024-04-25T08:54:17.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.492+0000 7f57a38db640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f57940093e0 0x7f5794186d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.492+0000 7f1211a60640 1 -- 172.21.15.43:0/3362869645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1204038a30 msgr2=0x7f120404f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.492+0000 7f1211a60640 1 --2- 172.21.15.43:0/3362869645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1204038a30 0x7f120404f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.492+0000 7f1211a60640 1 -- 172.21.15.43:0/3362869645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1204038e10 msgr2=0x7f12040d90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.492+0000 7f1211a60640 1 --2- 172.21.15.43:0/3362869645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1204038e10 0x7f12040d90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.492+0000 7f1211a60640 1 -- 172.21.15.43:0/3362869645 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f120404e2a0 con 0x7f12040363a0 2024-04-25T08:54:17.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.492+0000 7f57a1677640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5794007170 0x7f579417c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.492+0000 7f57a1677640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5794007170 0x7f579417c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37556/0 (socket says 172.21.15.43:37556) 2024-04-25T08:54:17.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.493+0000 7f57a1677640 1 -- 172.21.15.43:0/1844104680 learned_addr learned my addr 172.21.15.43:0/1844104680 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:17.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.493+0000 7f57a38db640 1 --2- 172.21.15.43:0/1844104680 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57940e77f0 0x7f5794185b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.493+0000 7f57a38db640 1 -- 172.21.15.43:0/1844104680 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5794051010 con 0x7f5794007170 2024-04-25T08:54:17.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.493+0000 7f57a38db640 1 -- 172.21.15.43:0/1844104680 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5794051380 con 0x7f57940e77f0 2024-04-25T08:54:17.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.493+0000 7f57a38db640 1 -- 172.21.15.43:0/1844104680 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5794050460 con 0x7f57940093e0 2024-04-25T08:54:17.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.493+0000 7f20cf7fe640 1 -- 172.21.15.43:0/2703390886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20c803e830 msgr2=0x7f20c80d6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.493+0000 7f20cf7fe640 1 --2- 172.21.15.43:0/2703390886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20c803e830 0x7f20c80d6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.493+0000 7f20cf7fe640 1 -- 172.21.15.43:0/2703390886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20c80e30b0 msgr2=0x7f20c80cc240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.493+0000 7f20cf7fe640 1 --2- 172.21.15.43:0/2703390886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20c80e30b0 0x7f20c80cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.493+0000 7f20cf7fe640 1 -- 172.21.15.43:0/2703390886 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c80dba30 con 0x7f20c8054120 2024-04-25T08:54:17.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.493+0000 7f9b44074640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b3c072280 0x7f9b3c157010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.493+0000 7f2bc2d9d640 1 --2- 172.21.15.43:0/347814099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2bbc050f10 0x7f2bbc0cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.493+0000 7f849450a640 1 -- 172.21.15.43:0/2931471298 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f848c0c87a0 con 0x7f848c0b5d50 2024-04-25T08:54:17.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.493+0000 7f57a1e78640 1 --2- 172.21.15.43:0/1844104680 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57940e77f0 0x7f5794185b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.493+0000 7f1211a60640 1 --2- 172.21.15.43:0/3362869645 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f12040363a0 0x7f12040e75a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f11fc00c410 tx=0x7f11fc00c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.493+0000 7f2bc259c640 1 -- 172.21.15.43:0/347814099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2bbc050f10 msgr2=0x7f2bbc0cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.493+0000 7f2bc259c640 1 --2- 172.21.15.43:0/347814099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2bbc050f10 0x7f2bbc0cf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.493+0000 7f2bc259c640 1 -- 172.21.15.43:0/347814099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bbc0d31a0 msgr2=0x7f2bbc0509d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T08:54:17.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.493+0000 7f9b44074640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b3c0b6300 0x7f9b3c0cfe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.493+0000 7f2bc259c640 1 --2- 172.21.15.43:0/347814099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bbc0d31a0 0x7f2bbc0509d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.493+0000 7f2bc259c640 1 -- 172.21.15.43:0/347814099 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bbc0d5d10 con 0x7f2bac0037f0 2024-04-25T08:54:17.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.493+0000 7f2bc2d9d640 1 --2- 172.21.15.43:0/347814099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2bbc050f10 0x7f2bbc0cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:17.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.493+0000 7f9b44074640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b3c15bb50 0x7f9b3c14a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.493+0000 7f2bc259c640 1 --2- 172.21.15.43:0/347814099 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2bac0037f0 0x7f2bbc04ca10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4007fd0 tx=0x7f2bb400ff70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.493+0000 7f57a0e76640 1 --2- 172.21.15.43:0/1844104680 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f57940093e0 0x7f5794186d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.493+0000 7f20cf7fe640 1 --2- 172.21.15.43:0/2703390886 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20c8054120 0x7f20c80dcad0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f20bc000c00 tx=0x7f20bc0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.493+0000 7f57a1677640 1 -- 172.21.15.43:0/1844104680 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f57940093e0 msgr2=0x7f5794186d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.493+0000 7f57a1677640 1 --2- 172.21.15.43:0/1844104680 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f57940093e0 0x7f5794186d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.493+0000 7f57a1677640 1 -- 172.21.15.43:0/1844104680 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57940e77f0 msgr2=0x7f5794185b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.493+0000 7f57a1677640 1 --2- 172.21.15.43:0/1844104680 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57940e77f0 0x7f5794185b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.494+0000 7f57a1677640 1 -- 172.21.15.43:0/1844104680 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5794186b20 con 0x7f5794007170 2024-04-25T08:54:17.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.494+0000 7f57a1e78640 1 --2- 172.21.15.43:0/1844104680 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57940e77f0 0x7f5794185b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:54:17.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.494+0000 7fb938fba640 1 --2- 172.21.15.43:0/1266145955 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb91806b5a0 0x7fb91806da60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb924045000 tx=0x7fb92403f000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.493+0000 7f9b4160f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b3c0b6300 0x7f9b3c0cfe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.493+0000 7f9b4160f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b3c0b6300 0x7f9b3c0cfe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:46990/0 (socket says 172.21.15.43:46990) 2024-04-25T08:54:17.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.494+0000 7f9b44074640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3c0cf670 con 0x7f9b3c072280 2024-04-25T08:54:17.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.494+0000 7f9b44074640 1 -- 172.21.15.43:0/496761864 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3c0b4eb0 con 0x7f9b3c0b6300 2024-04-25T08:54:17.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.494+0000 7f9b4160f640 1 -- 172.21.15.43:0/496761864 learned_addr learned my addr 172.21.15.43:0/496761864 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:17.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.494+0000 7f9b44074640 1 -- 172.21.15.43:0/496761864 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3c0cf7f0 con 0x7f9b3c15bb50 2024-04-25T08:54:17.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.494+0000 7f9b41e10640 1 --2- 172.21.15.43:0/496761864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b3c072280 0x7f9b3c157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.494+0000 7f9b42611640 1 --2- 172.21.15.43:0/496761864 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b3c15bb50 0x7f9b3c14a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.494+0000 7f57a1677640 1 --2- 172.21.15.43:0/1844104680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5794007170 0x7f579417c640 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f578c00c500 tx=0x7f578c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.494+0000 7f8482ffd640 1 -- 172.21.15.43:0/2931471298 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f847c01a070 con 0x7f848c0b5d50 2024-04-25T08:54:17.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.494+0000 7f5791ffb640 1 -- 172.21.15.43:0/1844104680 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578c017070 con 0x7f5794007170 2024-04-25T08:54:17.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.494+0000 7f9b4160f640 1 -- 172.21.15.43:0/496761864 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b3c15bb50 msgr2=0x7f9b3c14a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.494+0000 7f9b4160f640 1 --2- 172.21.15.43:0/496761864 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b3c15bb50 0x7f9b3c14a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.494+0000 7f9b4160f640 1 -- 172.21.15.43:0/496761864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b3c072280 msgr2=0x7f9b3c157010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.494+0000 7f9b4160f640 1 --2- 172.21.15.43:0/496761864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b3c072280 0x7f9b3c157010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.494+0000 7f9b4160f640 1 -- 172.21.15.43:0/496761864 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b3c1576f0 con 0x7f9b3c0b6300 2024-04-25T08:54:17.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.494+0000 7f57a38db640 1 -- 172.21.15.43:0/1844104680 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5794186560 con 0x7f5794007170 2024-04-25T08:54:17.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.494+0000 7f9b4160f640 1 --2- 172.21.15.43:0/496761864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b3c0b6300 0x7f9b3c0cfe80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9b38009870 tx=0x7f9b380046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.494+0000 7f849450a640 1 -- 172.21.15.43:0/2931471298 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f848c0680f0 con 0x7f848c0b5d50 2024-04-25T08:54:17.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.494+0000 7f5791ffb640 1 -- 172.21.15.43:0/1844104680 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578c0035f0 con 0x7f5794007170 2024-04-25T08:54:17.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.494+0000 7f57a38db640 1 -- 172.21.15.43:0/1844104680 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57940e5d90 con 0x7f5794007170 2024-04-25T08:54:17.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.494+0000 7f9b327fc640 1 -- 172.21.15.43:0/496761864 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b38041070 con 0x7f9b3c0b6300 2024-04-25T08:54:17.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.495+0000 7f8482ffd640 1 --2- 172.21.15.43:0/2931471298 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f847006b4d0 0x7f847006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.495+0000 7f9b44074640 1 -- 172.21.15.43:0/496761864 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b3c1579d0 con 0x7f9b3c0b6300 2024-04-25T08:54:17.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.495+0000 7f8482ffd640 1 -- 172.21.15.43:0/2931471298 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f847c08e060 con 0x7f848c0b5d50 2024-04-25T08:54:17.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.495+0000 7f9b44074640 1 -- 172.21.15.43:0/496761864 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b3c06bc40 con 0x7f9b3c0b6300 2024-04-25T08:54:17.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.495+0000 7f8491aa5640 1 --2- 172.21.15.43:0/2931471298 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f847006b4d0 0x7f847006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.495+0000 7f9b327fc640 1 -- 172.21.15.43:0/496761864 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b38030950 con 0x7f9b3c0b6300 2024-04-25T08:54:17.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.495+0000 7f9b327fc640 1 -- 172.21.15.43:0/496761864 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b38037040 con 0x7f9b3c0b6300 2024-04-25T08:54:17.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.495+0000 7f7fa0a6b640 1 -- 172.21.15.43:0/671274414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f9c0c9520 con 0x7f7f9c0b2cc0 2024-04-25T08:54:17.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.495+0000 7f7f92ffd640 1 -- 172.21.15.43:0/671274414 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7f8c0037b0 con 0x7f7f9c0b2cc0 2024-04-25T08:54:17.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.496+0000 7f5791ffb640 1 -- 172.21.15.43:0/1844104680 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f578c0039e0 con 0x7f5794007170 2024-04-25T08:54:17.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.496+0000 7f8491aa5640 1 --2- 172.21.15.43:0/2931471298 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f847006b4d0 0x7f847006d990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8488034000 tx=0x7f848802c000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.496+0000 7f7f92ffd640 1 --2- 172.21.15.43:0/671274414 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7f7c06b520 0x7f7f7c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.496+0000 7f9b44074640 1 -- 172.21.15.43:0/496761864 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b24005290 con 0x7f9b3c0b6300 2024-04-25T08:54:17.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.496+0000 7f57a38db640 1 -- 172.21.15.43:0/1844104680 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5794005810 con 0x7f5794007170 2024-04-25T08:54:17.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.496+0000 7f7f9a59c640 1 --2- 172.21.15.43:0/671274414 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7f7c06b520 0x7f7f7c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.496+0000 7f7fa0a6b640 1 -- 172.21.15.43:0/671274414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f9c067f10 con 0x7f7f9c0b2cc0 2024-04-25T08:54:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.497+0000 7f9b327fc640 1 -- 172.21.15.43:0/496761864 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9b38044070 con 0x7f9b3c0b6300 2024-04-25T08:54:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.497+0000 7f7f9a59c640 1 --2- 172.21.15.43:0/671274414 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7f7c06b520 0x7f7f7c06d9e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7f840090b0 tx=0x7f7f84040040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.497+0000 7f9b327fc640 1 --2- 172.21.15.43:0/496761864 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9b2006b520 0x7f9b2006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.497+0000 7f9b327fc640 1 -- 172.21.15.43:0/496761864 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9b380aae90 con 0x7f9b3c0b6300 2024-04-25T08:54:17.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.497+0000 7f9b41e10640 1 --2- 172.21.15.43:0/496761864 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9b2006b520 0x7f9b2006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.498+0000 7f9b41e10640 1 --2- 172.21.15.43:0/496761864 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9b2006b520 0x7f9b2006d9e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c006fd0 tx=0x7f9b2c009060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.498+0000 7f12027fc640 1 -- 172.21.15.43:0/3362869645 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11fc005230 con 0x7f12040363a0 2024-04-25T08:54:17.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.498+0000 7f2bb2ffd640 1 -- 172.21.15.43:0/347814099 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb40053a0 con 0x7f2bac0037f0 2024-04-25T08:54:17.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.498+0000 7f1213cc4640 1 -- 172.21.15.43:0/3362869645 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f120404e580 con 0x7f12040363a0 2024-04-25T08:54:17.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.498+0000 7f12027fc640 1 -- 172.21.15.43:0/3362869645 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11fc0053d0 con 0x7f12040363a0 2024-04-25T08:54:17.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.499+0000 7f20ccff9640 1 -- 172.21.15.43:0/2703390886 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc041020 con 0x7f20c8054120 2024-04-25T08:54:17.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.499+0000 7f20ccff9640 1 -- 172.21.15.43:0/2703390886 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20bc03d6e0 con 0x7f20c8054120 2024-04-25T08:54:17.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.499+0000 7f20d6458640 1 -- 172.21.15.43:0/2703390886 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20c80dbd10 con 0x7f20c8054120 2024-04-25T08:54:17.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.499+0000 7f2bc359e640 1 -- 172.21.15.43:0/347814099 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bbc0d5ff0 con 0x7f2bac0037f0 2024-04-25T08:54:17.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.499+0000 7f20ccff9640 1 -- 172.21.15.43:0/2703390886 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20bc037040 con 0x7f20c8054120 2024-04-25T08:54:17.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.499+0000 7f20d6458640 1 -- 172.21.15.43:0/2703390886 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20c80dc2d0 con 0x7f20c8054120 2024-04-25T08:54:17.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.499+0000 7f2bc359e640 1 -- 172.21.15.43:0/347814099 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bbc0d65b0 con 0x7f2bac0037f0 2024-04-25T08:54:17.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.499+0000 7fb929ffb640 1 -- 172.21.15.43:0/1266145955 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb9340d3310 con 0x7fb92c00c090 2024-04-25T08:54:17.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.499+0000 7f2bb2ffd640 1 -- 172.21.15.43:0/347814099 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb4018070 con 0x7f2bac0037f0 2024-04-25T08:54:17.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.499+0000 7f12027fc640 1 -- 172.21.15.43:0/3362869645 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11fc005230 con 0x7f12040363a0 2024-04-25T08:54:17.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.499+0000 7f1213cc4640 1 -- 172.21.15.43:0/3362869645 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f120404eb40 con 0x7f12040363a0 2024-04-25T08:54:17.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.500+0000 7f2bb2ffd640 1 -- 172.21.15.43:0/347814099 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb400d8d0 con 0x7f2bac0037f0 2024-04-25T08:54:17.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.500+0000 7f1213cc4640 1 -- 172.21.15.43:0/3362869645 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1204005380 con 0x7f12040363a0 2024-04-25T08:54:17.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.500+0000 7f20d6458640 1 -- 172.21.15.43:0/2703390886 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20c80057d0 con 0x7f20c8054120 2024-04-25T08:54:17.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.500+0000 7f2bc359e640 1 -- 172.21.15.43:0/347814099 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ba8005290 con 0x7f2bac0037f0 2024-04-25T08:54:17.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.501+0000 7f2bb2ffd640 1 -- 172.21.15.43:0/347814099 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2bb400da70 con 0x7f2bac0037f0 2024-04-25T08:54:17.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.501+0000 7f12027fc640 1 -- 172.21.15.43:0/3362869645 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f11fc0045e0 con 0x7f12040363a0 2024-04-25T08:54:17.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.501+0000 7f2bb2ffd640 1 --2- 172.21.15.43:0/347814099 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2ba406b500 0x7f2ba406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.502+0000 7f12027fc640 1 --2- 172.21.15.43:0/3362869645 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f11f006b700 0x7f11f006dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.502+0000 7f20ccff9640 1 -- 172.21.15.43:0/2703390886 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f20bc044020 con 0x7f20c8054120 2024-04-25T08:54:17.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.502+0000 7f2bc1d9b640 1 --2- 172.21.15.43:0/347814099 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2ba406b500 0x7f2ba406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.502+0000 7f12027fc640 1 -- 172.21.15.43:0/3362869645 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f11fc0892d0 con 0x7f12040363a0 2024-04-25T08:54:17.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.502+0000 7f2bb2ffd640 1 -- 172.21.15.43:0/347814099 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2bb408a3b0 con 0x7f2bac0037f0 2024-04-25T08:54:17.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.502+0000 7f121125f640 1 --2- 172.21.15.43:0/3362869645 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f11f006b700 0x7f11f006dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.502+0000 7f20ccff9640 1 --2- 172.21.15.43:0/2703390886 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f20b406b4d0 0x7f20b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.502+0000 7f121125f640 1 --2- 172.21.15.43:0/3362869645 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f11f006b700 0x7f11f006dbc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f120c050190 tx=0x7f120c0a9000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.502+0000 7f20ccff9640 1 -- 172.21.15.43:0/2703390886 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20bc0aaca0 con 0x7f20c8054120 2024-04-25T08:54:17.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.502+0000 7f20cffff640 1 --2- 172.21.15.43:0/2703390886 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f20b406b4d0 0x7f20b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.502+0000 7f2bc1d9b640 1 --2- 172.21.15.43:0/347814099 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2ba406b500 0x7f2ba406d9c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8032000 tx=0x7f2bb8007000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.502+0000 7f8482ffd640 1 -- 172.21.15.43:0/2931471298 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f847c058150 con 0x7f848c0b5d50 2024-04-25T08:54:17.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.503+0000 7f20cffff640 1 --2- 172.21.15.43:0/2703390886 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f20b406b4d0 0x7f20b406d990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f20c4000f60 tx=0x7f20c4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.503+0000 7f7f92ffd640 1 -- 172.21.15.43:0/671274414 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7f8c055c40 con 0x7f7f9c0b2cc0 2024-04-25T08:54:17.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.503+0000 7f9b327fc640 1 -- 172.21.15.43:0/496761864 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9b380798d0 con 0x7f9b3c0b6300 2024-04-25T08:54:17.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.504+0000 7f5791ffb640 1 -- 172.21.15.43:0/1844104680 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f578c01a070 con 0x7f5794007170 2024-04-25T08:54:17.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.504+0000 7f5791ffb640 1 --2- 172.21.15.43:0/1844104680 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f578006b4d0 0x7f578006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:17.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.504+0000 7f57a0e76640 1 --2- 172.21.15.43:0/1844104680 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f578006b4d0 0x7f578006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:17.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.504+0000 7f5791ffb640 1 -- 172.21.15.43:0/1844104680 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f578c056370 con 0x7f5794007170 2024-04-25T08:54:17.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.505+0000 7f57a0e76640 1 --2- 172.21.15.43:0/1844104680 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f578006b4d0 0x7f578006d990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f579400ecf0 tx=0x7f579c0a9000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:17.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.505+0000 7f20ccff9640 1 -- 172.21.15.43:0/2703390886 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f20bc04b030 con 0x7f20c8054120 2024-04-25T08:54:17.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.509+0000 7f2bb2ffd640 1 -- 172.21.15.43:0/347814099 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2bb4059100 con 0x7f2bac0037f0 2024-04-25T08:54:17.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.513+0000 7f7f92ffd640 1 -- 172.21.15.43:0/671274414 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7f9c067f10 con 0x7f7f9c0b2cc0 2024-04-25T08:54:17.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.514+0000 7f12027fc640 1 -- 172.21.15.43:0/3362869645 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f11fc0580c0 con 0x7f12040363a0 2024-04-25T08:54:17.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.521+0000 7f5791ffb640 1 -- 172.21.15.43:0/1844104680 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f578c021020 con 0x7f5794007170 2024-04-25T08:54:17.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.927+0000 7fb93ba1f640 1 -- 172.21.15.43:0/1266145955 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fb914005580 con 0x7fb92c00c090 2024-04-25T08:54:17.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.927+0000 7f849450a640 1 -- 172.21.15.43:0/2931471298 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f848c074ef0 con 0x7f848c0b5d50 2024-04-25T08:54:17.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.927+0000 7f8482ffd640 1 -- 172.21.15.43:0/2931471298 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f847c057a50 con 0x7f848c0b5d50 2024-04-25T08:54:17.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.927+0000 7fb929ffb640 1 -- 172.21.15.43:0/1266145955 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb9340d73b0 con 0x7fb92c00c090 2024-04-25T08:54:17.918 INFO:teuthology.orchestra.run.smithi043.stdout:55834574854 2024-04-25T08:54:17.918 INFO:teuthology.orchestra.run.smithi043.stdout:55834574853 2024-04-25T08:54:17.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.931+0000 7f8480ff9640 1 -- 172.21.15.43:0/2931471298 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f847006b4d0 msgr2=0x7f847006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.931+0000 7f8480ff9640 1 --2- 172.21.15.43:0/2931471298 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f847006b4d0 0x7f847006d990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8488034000 tx=0x7f848802c000 comp rx=0 tx=0).stop 2024-04-25T08:54:17.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.931+0000 7f8480ff9640 1 -- 172.21.15.43:0/2931471298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f848c0b5d50 msgr2=0x7f848c0c7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.931+0000 7f8480ff9640 1 --2- 172.21.15.43:0/2931471298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f848c0b5d50 0x7f848c0c7ee0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f847c00c390 tx=0x7f847c00c850 comp rx=0 tx=0).stop 2024-04-25T08:54:17.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.931+0000 7f8480ff9640 1 -- 172.21.15.43:0/2931471298 shutdown_connections 2024-04-25T08:54:17.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.931+0000 7f8480ff9640 1 --2- 172.21.15.43:0/2931471298 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f848c0d20b0 0x7f848c0ca1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.931+0000 7f8480ff9640 1 --2- 172.21.15.43:0/2931471298 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f847006b4d0 0x7f847006d990 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.931+0000 7f8480ff9640 1 --2- 172.21.15.43:0/2931471298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f848c0b6130 0x7f848c152ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.931+0000 7f8480ff9640 1 --2- 172.21.15.43:0/2931471298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f848c0b5d50 0x7f848c0c7ee0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.931+0000 7f8480ff9640 1 -- 172.21.15.43:0/2931471298 >> 172.21.15.43:0/2931471298 conn(0x7f848c0d3a30 msgr2=0x7f848c0d4020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:17.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.932+0000 7f8480ff9640 1 -- 172.21.15.43:0/2931471298 shutdown_connections 2024-04-25T08:54:17.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.932+0000 7fb91f7fe640 1 -- 172.21.15.43:0/1266145955 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb91806b5a0 msgr2=0x7fb91806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.932+0000 7fb91f7fe640 1 --2- 172.21.15.43:0/1266145955 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb91806b5a0 0x7fb91806da60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb924045000 tx=0x7fb92403f000 comp rx=0 tx=0).stop 2024-04-25T08:54:17.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.932+0000 7f20d6458640 1 -- 172.21.15.43:0/2703390886 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f20c8038a50 con 0x7f20c8054120 2024-04-25T08:54:17.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.932+0000 7fb91f7fe640 1 -- 172.21.15.43:0/1266145955 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb92c00c090 msgr2=0x7fb92c04e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.932+0000 7fb91f7fe640 1 --2- 172.21.15.43:0/1266145955 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb92c00c090 0x7fb92c04e250 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fb93404fcd0 tx=0x7fb934069260 comp rx=0 tx=0).stop 2024-04-25T08:54:17.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.932+0000 7f8480ff9640 1 -- 172.21.15.43:0/2931471298 wait complete. 2024-04-25T08:54:17.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.932+0000 7fb91f7fe640 1 -- 172.21.15.43:0/1266145955 shutdown_connections 2024-04-25T08:54:17.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.932+0000 7fb91f7fe640 1 --2- 172.21.15.43:0/1266145955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb92c0df4e0 0x7fb92c04c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.932+0000 7fb91f7fe640 1 --2- 172.21.15.43:0/1266145955 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb91806b5a0 0x7fb91806da60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.932+0000 7fb91f7fe640 1 --2- 172.21.15.43:0/1266145955 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb92c053c60 0x7fb92c0e13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.932+0000 7fb91f7fe640 1 --2- 172.21.15.43:0/1266145955 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb92c00c090 0x7fb92c04e250 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.932+0000 7fb91f7fe640 1 -- 172.21.15.43:0/1266145955 >> 172.21.15.43:0/1266145955 conn(0x7fb92c009fa0 msgr2=0x7fb92c00a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:17.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.932+0000 7f20ccff9640 1 -- 172.21.15.43:0/2703390886 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f20bc079d20 con 0x7f20c8054120 2024-04-25T08:54:17.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.933+0000 7fb91f7fe640 1 -- 172.21.15.43:0/1266145955 shutdown_connections 2024-04-25T08:54:17.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.933+0000 7fb91f7fe640 1 -- 172.21.15.43:0/1266145955 wait complete. 2024-04-25T08:54:17.926 INFO:teuthology.orchestra.run.smithi043.stdout:55834574852 2024-04-25T08:54:17.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.935+0000 7f9b44074640 1 -- 172.21.15.43:0/496761864 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f9b24005850 con 0x7f9b3c0b6300 2024-04-25T08:54:17.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.935+0000 7f2bc359e640 1 -- 172.21.15.43:0/347814099 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f2ba8005580 con 0x7f2bac0037f0 2024-04-25T08:54:17.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.936+0000 7f20d6458640 1 -- 172.21.15.43:0/2703390886 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f20b406b4d0 msgr2=0x7f20b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.936+0000 7f9b327fc640 1 -- 172.21.15.43:0/496761864 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9b3807d970 con 0x7f9b3c0b6300 2024-04-25T08:54:17.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.936+0000 7f20d6458640 1 --2- 172.21.15.43:0/2703390886 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f20b406b4d0 0x7f20b406d990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f20c4000f60 tx=0x7f20c4009040 comp rx=0 tx=0).stop 2024-04-25T08:54:17.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.936+0000 7f20d6458640 1 -- 172.21.15.43:0/2703390886 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20c8054120 msgr2=0x7f20c80dcad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.936+0000 7f20d6458640 1 --2- 172.21.15.43:0/2703390886 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20c8054120 0x7f20c80dcad0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f20bc000c00 tx=0x7f20bc0046c0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.928 INFO:teuthology.orchestra.run.smithi043.stdout:55834574855 2024-04-25T08:54:17.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.936+0000 7f2bb2ffd640 1 -- 172.21.15.43:0/347814099 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2bb4058a00 con 0x7f2bac0037f0 2024-04-25T08:54:17.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.936+0000 7f20d6458640 1 -- 172.21.15.43:0/2703390886 shutdown_connections 2024-04-25T08:54:17.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.936+0000 7f20d6458640 1 --2- 172.21.15.43:0/2703390886 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20c80e30b0 0x7f20c80cc240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.936+0000 7f20d6458640 1 --2- 172.21.15.43:0/2703390886 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20c8054120 0x7f20c80dcad0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.936+0000 7f20d6458640 1 --2- 172.21.15.43:0/2703390886 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f20b406b4d0 0x7f20b406d990 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.936+0000 7f20d6458640 1 --2- 172.21.15.43:0/2703390886 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20c803e830 0x7f20c80d6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.936+0000 7f20d6458640 1 -- 172.21.15.43:0/2703390886 >> 172.21.15.43:0/2703390886 conn(0x7f20c80559d0 msgr2=0x7f20c8039250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:17.929 INFO:teuthology.orchestra.run.smithi043.stdout:55834574853 2024-04-25T08:54:17.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.938+0000 7f7fa0a6b640 1 -- 172.21.15.43:0/671274414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7f9c160780 con 0x7f7f9c0b2cc0 2024-04-25T08:54:17.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.938+0000 7f1213cc4640 1 -- 172.21.15.43:0/3362869645 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f120400d5e0 con 0x7f12040363a0 2024-04-25T08:54:17.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.938+0000 7f20d6458640 1 -- 172.21.15.43:0/2703390886 shutdown_connections 2024-04-25T08:54:17.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.938+0000 7f20d6458640 1 -- 172.21.15.43:0/2703390886 wait complete. 2024-04-25T08:54:17.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.938+0000 7f12027fc640 1 -- 172.21.15.43:0/3362869645 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f11fc0580c0 con 0x7f12040363a0 2024-04-25T08:54:17.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.939+0000 7f7f92ffd640 1 -- 172.21.15.43:0/671274414 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7f9c160780 con 0x7f7f9c0b2cc0 2024-04-25T08:54:17.931 INFO:teuthology.orchestra.run.smithi043.stdout:55834574853 2024-04-25T08:54:17.931 INFO:teuthology.orchestra.run.smithi043.stdout:55834574853 2024-04-25T08:54:17.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.940+0000 7f2bc359e640 1 -- 172.21.15.43:0/347814099 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2ba406b500 msgr2=0x7f2ba406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.940+0000 7f9b44074640 1 -- 172.21.15.43:0/496761864 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9b2006b520 msgr2=0x7f9b2006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.940+0000 7f2bc359e640 1 --2- 172.21.15.43:0/347814099 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2ba406b500 0x7f2ba406d9c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8032000 tx=0x7f2bb8007000 comp rx=0 tx=0).stop 2024-04-25T08:54:17.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.940+0000 7f2bc359e640 1 -- 172.21.15.43:0/347814099 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2bac0037f0 msgr2=0x7f2bbc04ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.940+0000 7f2bc359e640 1 --2- 172.21.15.43:0/347814099 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2bac0037f0 0x7f2bbc04ca10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4007fd0 tx=0x7f2bb400ff70 comp rx=0 tx=0).stop 2024-04-25T08:54:17.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.940+0000 7f9b44074640 1 --2- 172.21.15.43:0/496761864 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9b2006b520 0x7f9b2006d9e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c006fd0 tx=0x7f9b2c009060 comp rx=0 tx=0).stop 2024-04-25T08:54:17.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.940+0000 7f9b44074640 1 -- 172.21.15.43:0/496761864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b3c0b6300 msgr2=0x7f9b3c0cfe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.940+0000 7f2bc359e640 1 -- 172.21.15.43:0/347814099 shutdown_connections 2024-04-25T08:54:17.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.940+0000 7f2bc359e640 1 --2- 172.21.15.43:0/347814099 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2ba406b500 0x7f2ba406d9c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.940+0000 7f2bc359e640 1 --2- 172.21.15.43:0/347814099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2bbc050f10 0x7f2bbc0cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.940+0000 7f2bc359e640 1 --2- 172.21.15.43:0/347814099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2bbc0d31a0 0x7f2bbc0509d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.940+0000 7f2bc359e640 1 --2- 172.21.15.43:0/347814099 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2bac0037f0 0x7f2bbc04ca10 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.940+0000 7f2bc359e640 1 -- 172.21.15.43:0/347814099 >> 172.21.15.43:0/347814099 conn(0x7f2bbc056cc0 msgr2=0x7f2bbc009d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:17.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.940+0000 7f9b44074640 1 --2- 172.21.15.43:0/496761864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b3c0b6300 0x7f9b3c0cfe80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9b38009870 tx=0x7f9b380046c0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.940+0000 7f2bc359e640 1 -- 172.21.15.43:0/347814099 shutdown_connections 2024-04-25T08:54:17.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.940+0000 7f2bc359e640 1 -- 172.21.15.43:0/347814099 wait complete. 2024-04-25T08:54:17.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.940+0000 7f9b44074640 1 -- 172.21.15.43:0/496761864 shutdown_connections 2024-04-25T08:54:17.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.940+0000 7f9b44074640 1 --2- 172.21.15.43:0/496761864 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b3c15bb50 0x7f9b3c14a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.940+0000 7f9b44074640 1 --2- 172.21.15.43:0/496761864 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9b2006b520 0x7f9b2006d9e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.940+0000 7f9b44074640 1 --2- 172.21.15.43:0/496761864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b3c0b6300 0x7f9b3c0cfe80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.941+0000 7f9b44074640 1 --2- 172.21.15.43:0/496761864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b3c072280 0x7f9b3c157010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.941+0000 7f9b44074640 1 -- 172.21.15.43:0/496761864 >> 172.21.15.43:0/496761864 conn(0x7f9b3c0b33a0 msgr2=0x7f9b3c0cd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:17.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.941+0000 7f9b44074640 1 -- 172.21.15.43:0/496761864 shutdown_connections 2024-04-25T08:54:17.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.941+0000 7f9b44074640 1 -- 172.21.15.43:0/496761864 wait complete. 2024-04-25T08:54:17.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.942+0000 7f7fa0a6b640 1 -- 172.21.15.43:0/671274414 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7f7c06b520 msgr2=0x7f7f7c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.942+0000 7f7fa0a6b640 1 --2- 172.21.15.43:0/671274414 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7f7c06b520 0x7f7f7c06d9e0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7f840090b0 tx=0x7f7f84040040 comp rx=0 tx=0).stop 2024-04-25T08:54:17.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.942+0000 7f1213cc4640 1 -- 172.21.15.43:0/3362869645 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f11f006b700 msgr2=0x7f11f006dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.942+0000 7f1213cc4640 1 --2- 172.21.15.43:0/3362869645 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f11f006b700 0x7f11f006dbc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f120c050190 tx=0x7f120c0a9000 comp rx=0 tx=0).stop 2024-04-25T08:54:17.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.942+0000 7f1213cc4640 1 -- 172.21.15.43:0/3362869645 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f12040363a0 msgr2=0x7f12040e75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.942+0000 7f1213cc4640 1 --2- 172.21.15.43:0/3362869645 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f12040363a0 0x7f12040e75a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f11fc00c410 tx=0x7f11fc00c8d0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.942+0000 7f7fa0a6b640 1 -- 172.21.15.43:0/671274414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f9c0b2cc0 msgr2=0x7f7f9c0b0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.942+0000 7f7fa0a6b640 1 --2- 172.21.15.43:0/671274414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f9c0b2cc0 0x7f7f9c0b0470 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7f8c00c5e0 tx=0x7f7f8c00caa0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.942+0000 7f1213cc4640 1 -- 172.21.15.43:0/3362869645 shutdown_connections 2024-04-25T08:54:17.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.943+0000 7f1213cc4640 1 --2- 172.21.15.43:0/3362869645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1204038e10 0x7f12040d90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.943+0000 7f1213cc4640 1 --2- 172.21.15.43:0/3362869645 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f11f006b700 0x7f11f006dbc0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.943+0000 7f7fa0a6b640 1 -- 172.21.15.43:0/671274414 shutdown_connections 2024-04-25T08:54:17.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.943+0000 7f7fa0a6b640 1 --2- 172.21.15.43:0/671274414 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7f7c06b520 0x7f7f7c06d9e0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.943+0000 7f7fa0a6b640 1 --2- 172.21.15.43:0/671274414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f9c15f180 0x7f7f9c06f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.943+0000 7f7fa0a6b640 1 --2- 172.21.15.43:0/671274414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f9c0b2cc0 0x7f7f9c0b0470 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.943+0000 7f7fa0a6b640 1 --2- 172.21.15.43:0/671274414 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7f9c0b23f0 0x7f7f9c150c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.943+0000 7f7fa0a6b640 1 -- 172.21.15.43:0/671274414 >> 172.21.15.43:0/671274414 conn(0x7f7f9c0d17f0 msgr2=0x7f7f9c0b4510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:17.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.943+0000 7f1213cc4640 1 --2- 172.21.15.43:0/3362869645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1204038a30 0x7f120404f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.943+0000 7f1213cc4640 1 --2- 172.21.15.43:0/3362869645 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f12040363a0 0x7f12040e75a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.943+0000 7f1213cc4640 1 -- 172.21.15.43:0/3362869645 >> 172.21.15.43:0/3362869645 conn(0x7f12040537c0 msgr2=0x7f1204039960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:17.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.943+0000 7f7fa0a6b640 1 -- 172.21.15.43:0/671274414 shutdown_connections 2024-04-25T08:54:17.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.943+0000 7f7fa0a6b640 1 -- 172.21.15.43:0/671274414 wait complete. 2024-04-25T08:54:17.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.943+0000 7f1213cc4640 1 -- 172.21.15.43:0/3362869645 shutdown_connections 2024-04-25T08:54:17.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.943+0000 7f57a38db640 1 -- 172.21.15.43:0/1844104680 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f57940e3310 con 0x7f5794007170 2024-04-25T08:54:17.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.943+0000 7f1213cc4640 1 -- 172.21.15.43:0/3362869645 wait complete. 2024-04-25T08:54:17.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.944+0000 7f5791ffb640 1 -- 172.21.15.43:0/1844104680 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f578c021020 con 0x7f5794007170 2024-04-25T08:54:17.942 INFO:teuthology.orchestra.run.smithi043.stdout:55834574853 2024-04-25T08:54:17.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.948+0000 7f57877fe640 1 -- 172.21.15.43:0/1844104680 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f578006b4d0 msgr2=0x7f578006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.948+0000 7f57877fe640 1 --2- 172.21.15.43:0/1844104680 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f578006b4d0 0x7f578006d990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f579400ecf0 tx=0x7f579c0a9000 comp rx=0 tx=0).stop 2024-04-25T08:54:17.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.949+0000 7f57877fe640 1 -- 172.21.15.43:0/1844104680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5794007170 msgr2=0x7f579417c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:17.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.949+0000 7f57877fe640 1 --2- 172.21.15.43:0/1844104680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5794007170 0x7f579417c640 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f578c00c500 tx=0x7f578c00c9c0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.949+0000 7f57877fe640 1 -- 172.21.15.43:0/1844104680 shutdown_connections 2024-04-25T08:54:17.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.949+0000 7f57877fe640 1 --2- 172.21.15.43:0/1844104680 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f578006b4d0 0x7f578006d990 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.949+0000 7f57877fe640 1 --2- 172.21.15.43:0/1844104680 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57940e77f0 0x7f5794185b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.950+0000 7f57877fe640 1 --2- 172.21.15.43:0/1844104680 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f57940093e0 0x7f5794186d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.950+0000 7f57877fe640 1 --2- 172.21.15.43:0/1844104680 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5794007170 0x7f579417c640 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:17.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.950+0000 7f57877fe640 1 -- 172.21.15.43:0/1844104680 >> 172.21.15.43:0/1844104680 conn(0x7f5794055c80 msgr2=0x7f57940e82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:17.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.950+0000 7f57877fe640 1 -- 172.21.15.43:0/1844104680 shutdown_connections 2024-04-25T08:54:17.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:17.950+0000 7f57877fe640 1 -- 172.21.15.43:0/1844104680 wait complete. 2024-04-25T08:54:17.946 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.0 2024-04-25T08:54:17.948 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.4 2024-04-25T08:54:17.952 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.7 2024-04-25T08:54:17.955 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.1 2024-04-25T08:54:17.956 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.5 2024-04-25T08:54:17.957 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.6 2024-04-25T08:54:17.957 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.3 2024-04-25T08:54:17.958 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.2 2024-04-25T08:54:18.946 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T08:54:18.949 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T08:54:18.953 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T08:54:18.956 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T08:54:18.957 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T08:54:18.957 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T08:54:18.959 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T08:54:18.959 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T08:54:19.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.125+0000 7f8f7cf73640 1 Processor -- start 2024-04-25T08:54:19.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.125+0000 7f8f7cf73640 1 -- start start 2024-04-25T08:54:19.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.126+0000 7f8f7cf73640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f78071c60 0x7f8f7806e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.126+0000 7f8f7cf73640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f8f780d0d10 0x7f8f780cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.126+0000 7f8f7cf73640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f7806ed30 con 0x7f8f780b4f10 2024-04-25T08:54:19.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.126+0000 7f8f7cf73640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f780cf8b0 con 0x7f8f780d0d10 2024-04-25T08:54:19.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.126+0000 7f8f7cf73640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7806f010 con 0x7f8f78071c60 2024-04-25T08:54:19.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.127+0000 7f8f77fff640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f8f780d0d10 0x7f8f780cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.127+0000 7f8f76ffd640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f78071c60 0x7f8f7806e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.127+0000 7f8f77fff640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f8f780d0d10 0x7f8f780cd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:47002/0 (socket says 172.21.15.43:47002) 2024-04-25T08:54:19.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.127+0000 7f8f77fff640 1 -- 172.21.15.43:0/1843023724 learned_addr learned my addr 172.21.15.43:0/1843023724 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:19.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.127+0000 7f8f77fff640 1 -- 172.21.15.43:0/1843023724 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f78071c60 msgr2=0x7f8f7806e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:19.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.127+0000 7f8f77fff640 1 --2- 172.21.15.43:0/1843023724 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f78071c60 0x7f8f7806e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.127+0000 7f8f77fff640 1 -- 172.21.15.43:0/1843023724 >> v1:172.21.15.43:6789/0 conn(0x7f8f780b4f10 legacy=0x7f8f780b52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:19.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.127+0000 7fc45b33c640 1 Processor -- start 2024-04-25T08:54:19.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.127+0000 7fc45b33c640 1 -- start start 2024-04-25T08:54:19.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.128+0000 7fc45b33c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc44c101010 0x7fc44c037c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.128+0000 7fc45b33c640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fc44c054430 0x7fc44c04f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.128+0000 7fc45b33c640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc44c052cf0 con 0x7fc44c033920 2024-04-25T08:54:19.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.128+0000 7fc45b33c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc44c0341c0 con 0x7fc44c101010 2024-04-25T08:54:19.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.128+0000 7fc45b33c640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc44c052f60 con 0x7fc44c054430 2024-04-25T08:54:19.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.129+0000 7fc459b39640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc44c033920 0x7fc44c053d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60926/0 (socket says 172.21.15.43:60926) 2024-04-25T08:54:19.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.129+0000 7fc459b39640 1 -- 172.21.15.43:0/1289812831 learned_addr learned my addr 172.21.15.43:0/1289812831 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:19.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.128+0000 7f8f77fff640 1 -- 172.21.15.43:0/1843023724 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f780d0230 con 0x7f8f780d0d10 2024-04-25T08:54:19.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.129+0000 7f8f77fff640 1 --2- 172.21.15.43:0/1843023724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f780d0d10 0x7f8f780cd010 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8f70009e50 tx=0x7f8f7002a6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4dbc1b04ba347322 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:19.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.129+0000 7fc45ab3b640 1 --2- 172.21.15.43:0/1289812831 >> v2:172.21.15.193:3300/0 conn(0x7fc44c054430 0x7fc44c04f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.129+0000 7fc45a33a640 1 --2- 172.21.15.43:0/1289812831 >> v2:172.21.15.43:3300/0 conn(0x7fc44c101010 0x7fc44c037c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.130+0000 7f8f767fc640 1 -- 172.21.15.43:0/1843023724 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f70036070 con 0x7f8f780d0d10 2024-04-25T08:54:19.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.130+0000 7f8f767fc640 1 -- 172.21.15.43:0/1843023724 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f70031600 con 0x7f8f780d0d10 2024-04-25T08:54:19.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.131+0000 7f8f7cf73640 1 -- 172.21.15.43:0/1843023724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f780d0d10 msgr2=0x7f8f780cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.131+0000 7f8f7cf73640 1 --2- 172.21.15.43:0/1843023724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f780d0d10 0x7f8f780cd010 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8f70009e50 tx=0x7f8f7002a6c0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.130+0000 7fc45a33a640 1 -- 172.21.15.43:0/1289812831 >> v2:172.21.15.193:3300/0 conn(0x7fc44c054430 msgr2=0x7fc44c04f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:19.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.130+0000 7fc45a33a640 1 --2- 172.21.15.43:0/1289812831 >> v2:172.21.15.193:3300/0 conn(0x7fc44c054430 0x7fc44c04f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.130+0000 7fc45a33a640 1 -- 172.21.15.43:0/1289812831 >> v1:172.21.15.43:6789/0 conn(0x7fc44c033920 legacy=0x7fc44c053d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.130+0000 7fc45a33a640 1 -- 172.21.15.43:0/1289812831 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc44c053240 con 0x7fc44c101010 2024-04-25T08:54:19.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.131+0000 7fc45a33a640 1 --2- 172.21.15.43:0/1289812831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc44c101010 0x7fc44c037c40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc44c034450 tx=0x7fc45002c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d174402e08757ea2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:19.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.132+0000 7fc459338640 1 -- 172.21.15.43:0/1289812831 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc450038070 con 0x7fc44c101010 2024-04-25T08:54:19.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.132+0000 7f8f7cf73640 1 -- 172.21.15.43:0/1843023724 shutdown_connections 2024-04-25T08:54:19.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.132+0000 7f8f7cf73640 1 --2- 172.21.15.43:0/1843023724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f780d0d10 0x7f8f780cd010 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.132+0000 7f8f7cf73640 1 --2- 172.21.15.43:0/1843023724 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f78071c60 0x7f8f7806e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.132+0000 7f8f7cf73640 1 -- 172.21.15.43:0/1843023724 >> 172.21.15.43:0/1843023724 conn(0x7f8f7806bc40 msgr2=0x7f8f7806c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:19.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.132+0000 7fc459338640 1 -- 172.21.15.43:0/1289812831 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc450033600 con 0x7fc44c101010 2024-04-25T08:54:19.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.133+0000 7fc45b33c640 1 -- 172.21.15.43:0/1289812831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc44c101010 msgr2=0x7fc44c037c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.133+0000 7fc45b33c640 1 --2- 172.21.15.43:0/1289812831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc44c101010 0x7fc44c037c40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc44c034450 tx=0x7fc45002c6c0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.132+0000 7f8f7cf73640 1 -- 172.21.15.43:0/1843023724 shutdown_connections 2024-04-25T08:54:19.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.132+0000 7f8f7cf73640 1 -- 172.21.15.43:0/1843023724 wait complete. 2024-04-25T08:54:19.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.134+0000 7fc45b33c640 1 -- 172.21.15.43:0/1289812831 shutdown_connections 2024-04-25T08:54:19.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.134+0000 7fc45b33c640 1 --2- 172.21.15.43:0/1289812831 >> v2:172.21.15.193:3300/0 conn(0x7fc44c054430 0x7fc44c04f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.134+0000 7fc45b33c640 1 --2- 172.21.15.43:0/1289812831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc44c101010 0x7fc44c037c40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.134+0000 7fc45b33c640 1 -- 172.21.15.43:0/1289812831 >> 172.21.15.43:0/1289812831 conn(0x7fc44c009fb0 msgr2=0x7fc44c00a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:19.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.134+0000 7f8f7cf73640 1 Processor -- start 2024-04-25T08:54:19.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.134+0000 7f8f7cf73640 1 -- start start 2024-04-25T08:54:19.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.134+0000 7fc45b33c640 1 -- 172.21.15.43:0/1289812831 shutdown_connections 2024-04-25T08:54:19.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.135+0000 7f8f7cf73640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f78071c60 0x7f8f7815bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.135+0000 7f8f7cf73640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f780b4f10 0x7f8f7815fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.136+0000 7f8f76ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f780b4f10 0x7f8f7815fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.136+0000 7f8f76ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f780b4f10 0x7f8f7815fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37594/0 (socket says 172.21.15.43:37594) 2024-04-25T08:54:19.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.136+0000 7f8f777fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f78071c60 0x7f8f7815bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.136+0000 7f8f76ffd640 1 -- 172.21.15.43:0/991765763 learned_addr learned my addr 172.21.15.43:0/991765763 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:19.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.136+0000 7f8f7cf73640 1 --2- 172.21.15.43:0/991765763 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f780d0d10 0x7f8f780cac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.137+0000 7f8f7cf73640 1 -- 172.21.15.43:0/991765763 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7806e970 con 0x7f8f780b4f10 2024-04-25T08:54:19.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.135+0000 7fc45b33c640 1 -- 172.21.15.43:0/1289812831 wait complete. 2024-04-25T08:54:19.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.137+0000 7fc45b33c640 1 Processor -- start 2024-04-25T08:54:19.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.137+0000 7f8f77fff640 1 --2- 172.21.15.43:0/991765763 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f780d0d10 0x7f8f780cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.137+0000 7f8f7cf73640 1 -- 172.21.15.43:0/991765763 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f780d0880 con 0x7f8f78071c60 2024-04-25T08:54:19.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.137+0000 7f8f7cf73640 1 -- 172.21.15.43:0/991765763 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f7806eaf0 con 0x7f8f780d0d10 2024-04-25T08:54:19.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.138+0000 7f8f76ffd640 1 -- 172.21.15.43:0/991765763 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f780d0d10 msgr2=0x7f8f780cac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.138+0000 7f8f76ffd640 1 --2- 172.21.15.43:0/991765763 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f780d0d10 0x7f8f780cac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.138+0000 7f8f76ffd640 1 -- 172.21.15.43:0/991765763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f78071c60 msgr2=0x7f8f7815bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.138+0000 7f8f76ffd640 1 --2- 172.21.15.43:0/991765763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f78071c60 0x7f8f7815bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.138+0000 7f8f76ffd640 1 -- 172.21.15.43:0/991765763 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f780c7170 con 0x7f8f780b4f10 2024-04-25T08:54:19.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.137+0000 7fc45b33c640 1 -- start start 2024-04-25T08:54:19.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.138+0000 7fc45b33c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc44c033920 0x7fc44c1ab960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.138+0000 7fc45b33c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc44c054430 0x7fc44c006f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.138+0000 7fc45b33c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc44c101010 0x7fc44c1a7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.138+0000 7fc45b33c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c054860 con 0x7fc44c033920 2024-04-25T08:54:19.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.138+0000 7fc45b33c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c053800 con 0x7fc44c101010 2024-04-25T08:54:19.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.138+0000 7fc45b33c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc44c038180 con 0x7fc44c054430 2024-04-25T08:54:19.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.138+0000 7fc45a33a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc44c033920 0x7fc44c1ab960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.138+0000 7fc45a33a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc44c033920 0x7fc44c1ab960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37600/0 (socket says 172.21.15.43:37600) 2024-04-25T08:54:19.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.138+0000 7fc45a33a640 1 -- 172.21.15.43:0/1135644383 learned_addr learned my addr 172.21.15.43:0/1135644383 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:19.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.139+0000 7fc459b39640 1 --2- 172.21.15.43:0/1135644383 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc44c054430 0x7fc44c006f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.139+0000 7fc45ab3b640 1 --2- 172.21.15.43:0/1135644383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc44c101010 0x7fc44c1a7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.138+0000 7f8f777fe640 1 --2- 172.21.15.43:0/991765763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f78071c60 0x7f8f7815bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:19.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.139+0000 7f8f77fff640 1 --2- 172.21.15.43:0/991765763 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f780d0d10 0x7f8f780cac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:19.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.139+0000 7fc459b39640 1 -- 172.21.15.43:0/1135644383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc44c101010 msgr2=0x7fc44c1a7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.139+0000 7fc459b39640 1 --2- 172.21.15.43:0/1135644383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc44c101010 0x7fc44c1a7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.139+0000 7fc459b39640 1 -- 172.21.15.43:0/1135644383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc44c033920 msgr2=0x7fc44c1ab960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.139+0000 7fc459b39640 1 --2- 172.21.15.43:0/1135644383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc44c033920 0x7fc44c1ab960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.139+0000 7fc459b39640 1 -- 172.21.15.43:0/1135644383 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc44c04fa50 con 0x7fc44c054430 2024-04-25T08:54:19.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.139+0000 7f8f76ffd640 1 --2- 172.21.15.43:0/991765763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f780b4f10 0x7f8f7815fa20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8f6800eb70 tx=0x7f8f6800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:19.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.140+0000 7fc459b39640 1 --2- 172.21.15.43:0/1135644383 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc44c054430 0x7fc44c006f00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc454069c60 tx=0x7fc45406bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:19.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.140+0000 7fc44affd640 1 -- 172.21.15.43:0/1135644383 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc454068520 con 0x7fc44c054430 2024-04-25T08:54:19.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.140+0000 7f8f5ffff640 1 -- 172.21.15.43:0/991765763 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f68017070 con 0x7f8f780b4f10 2024-04-25T08:54:19.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.140+0000 7f8f5ffff640 1 -- 172.21.15.43:0/991765763 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f6800ed80 con 0x7f8f780b4f10 2024-04-25T08:54:19.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.140+0000 7f8f5ffff640 1 -- 172.21.15.43:0/991765763 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f68013470 con 0x7f8f780b4f10 2024-04-25T08:54:19.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.140+0000 7f8f7cf73640 1 -- 172.21.15.43:0/991765763 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f780c7450 con 0x7f8f780b4f10 2024-04-25T08:54:19.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.140+0000 7f8f7cf73640 1 -- 172.21.15.43:0/991765763 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f780c7a10 con 0x7f8f780b4f10 2024-04-25T08:54:19.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.140+0000 7fc45b33c640 1 -- 172.21.15.43:0/1135644383 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc44c18cec0 con 0x7fc44c054430 2024-04-25T08:54:19.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.140+0000 7fc45b33c640 1 -- 172.21.15.43:0/1135644383 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc44c18d480 con 0x7fc44c054430 2024-04-25T08:54:19.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.142+0000 7fc44affd640 1 -- 172.21.15.43:0/1135644383 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4540686c0 con 0x7fc44c054430 2024-04-25T08:54:19.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.142+0000 7fc44affd640 1 -- 172.21.15.43:0/1135644383 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc454083610 con 0x7fc44c054430 2024-04-25T08:54:19.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.144+0000 7f8f5ffff640 1 -- 172.21.15.43:0/991765763 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8f6801a070 con 0x7f8f780b4f10 2024-04-25T08:54:19.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.144+0000 7f8f5ffff640 1 --2- 172.21.15.43:0/991765763 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8f6406b5d0 0x7f8f6406da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.144+0000 7f8f5ffff640 1 -- 172.21.15.43:0/991765763 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8f6808a180 con 0x7f8f780b4f10 2024-04-25T08:54:19.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.144+0000 7f8f7cf73640 1 -- 172.21.15.43:0/991765763 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f58005290 con 0x7f8f780b4f10 2024-04-25T08:54:19.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.145+0000 7f8f777fe640 1 --2- 172.21.15.43:0/991765763 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8f6406b5d0 0x7f8f6406da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.146+0000 7fc44affd640 1 -- 172.21.15.43:0/1135644383 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc4540837b0 con 0x7fc44c054430 2024-04-25T08:54:19.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.147+0000 7fc44affd640 1 --2- 172.21.15.43:0/1135644383 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc43806b5d0 0x7fc43806da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.148+0000 7fc44affd640 1 -- 172.21.15.43:0/1135644383 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc4540f3d50 con 0x7fc44c054430 2024-04-25T08:54:19.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.148+0000 7f8f777fe640 1 --2- 172.21.15.43:0/991765763 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8f6406b5d0 0x7f8f6406da90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c001710 tx=0x7f8f6c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:19.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.148+0000 7fc45a33a640 1 --2- 172.21.15.43:0/1135644383 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc43806b5d0 0x7fc43806da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.149+0000 7fc45a33a640 1 --2- 172.21.15.43:0/1135644383 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc43806b5d0 0x7fc43806da90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc450009560 tx=0x7fc450008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:19.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.149+0000 7fc45b33c640 1 -- 172.21.15.43:0/1135644383 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc43c005290 con 0x7fc44c054430 2024-04-25T08:54:19.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.153+0000 7f8f5ffff640 1 -- 172.21.15.43:0/991765763 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8f68059070 con 0x7f8f780b4f10 2024-04-25T08:54:19.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.154+0000 7fc44affd640 1 -- 172.21.15.43:0/1135644383 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc4540c25f0 con 0x7fc44c054430 2024-04-25T08:54:19.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.180+0000 7f0891aa7640 1 Processor -- start 2024-04-25T08:54:19.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.180+0000 7f0891aa7640 1 -- start start 2024-04-25T08:54:19.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.181+0000 7f0891aa7640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f088c0cc6e0 0x7f088c0b4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.181+0000 7f0891aa7640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f088c0b4650 0x7f088c06ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.181+0000 7f0891aa7640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f088c0b31b0 con 0x7f088c0cc300 2024-04-25T08:54:19.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.181+0000 7f0891aa7640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f088c0b4ac0 con 0x7f088c0cc6e0 2024-04-25T08:54:19.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.181+0000 7f0891aa7640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f088c0b3490 con 0x7f088c0b4650 2024-04-25T08:54:19.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.182+0000 7f0890aa5640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f088c0cc300 0x7f088c15aed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60928/0 (socket says 172.21.15.43:60928) 2024-04-25T08:54:19.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.182+0000 7f0890aa5640 1 -- 172.21.15.43:0/151852540 learned_addr learned my addr 172.21.15.43:0/151852540 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:19.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.182+0000 7f088bfff640 1 --2- 172.21.15.43:0/151852540 >> v2:172.21.15.43:3300/0 conn(0x7f088c0cc6e0 0x7f088c0b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.182+0000 7f08912a6640 1 --2- 172.21.15.43:0/151852540 >> v2:172.21.15.193:3300/0 conn(0x7f088c0b4650 0x7f088c06ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.182+0000 7f08912a6640 1 -- 172.21.15.43:0/151852540 >> v2:172.21.15.43:3300/0 conn(0x7f088c0cc6e0 msgr2=0x7f088c0b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:19.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.182+0000 7f08912a6640 1 --2- 172.21.15.43:0/151852540 >> v2:172.21.15.43:3300/0 conn(0x7f088c0cc6e0 0x7f088c0b4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.183+0000 7fbc5f593640 1 Processor -- start 2024-04-25T08:54:19.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.182+0000 7f08912a6640 1 -- 172.21.15.43:0/151852540 >> v1:172.21.15.43:6789/0 conn(0x7f088c0cc300 legacy=0x7f088c15aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.182+0000 7f088b7fe640 1 -- 172.21.15.43:0/151852540 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1177779056 0 0) 0x7f088c0b31b0 con 0x7f088c0cc300 2024-04-25T08:54:19.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.183+0000 7f08912a6640 1 -- 172.21.15.43:0/151852540 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f088c0b3770 con 0x7f088c0b4650 2024-04-25T08:54:19.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.184+0000 7f45be8b3640 1 Processor -- start 2024-04-25T08:54:19.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.184+0000 7f2f37ab3640 1 Processor -- start 2024-04-25T08:54:19.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.184+0000 7f2f37ab3640 1 -- start start 2024-04-25T08:54:19.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.185+0000 7f08912a6640 1 --2- 172.21.15.43:0/151852540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f088c0b4650 0x7f088c06ace0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f088c154ac0 tx=0x7f088402a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83d6ed7a17a8660b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:19.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.184+0000 7fbc5f593640 1 -- start start 2024-04-25T08:54:19.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.184+0000 7f45be8b3640 1 -- start start 2024-04-25T08:54:19.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.185+0000 7f088b7fe640 1 -- 172.21.15.43:0/151852540 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0884007740 con 0x7f088c0b4650 2024-04-25T08:54:19.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.185+0000 7f088b7fe640 1 -- 172.21.15.43:0/151852540 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08840078e0 con 0x7f088c0b4650 2024-04-25T08:54:19.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.185+0000 7fbc5f593640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc580b4230 0x7fbc580b4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.185+0000 7f088b7fe640 1 -- 172.21.15.43:0/151852540 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0884007cb0 con 0x7f088c0b4650 2024-04-25T08:54:19.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.185+0000 7f45be8b3640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45b806b620 0x7f45b80ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.185+0000 7f45be8b3640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f45b80b3da0 0x7f45b80aec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.185+0000 7f2f37ab3640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2f300ae730 0x7f2f300aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.185+0000 7f45be8b3640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45b80b4310 con 0x7f45b815b220 2024-04-25T08:54:19.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.185+0000 7f45be8b3640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45b80b45f0 con 0x7f45b80b3da0 2024-04-25T08:54:19.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.186+0000 7f2f37ab3640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f2f300b3da0 0x7f2f3006cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.186+0000 7f2f3584f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2f300ae730 0x7f2f300aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.186+0000 7f2f3584f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2f300ae730 0x7f2f300aeb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37624/0 (socket says 172.21.15.43:37624) 2024-04-25T08:54:19.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.186+0000 7f2f3584f640 1 -- 172.21.15.43:0/382050161 learned_addr learned my addr 172.21.15.43:0/382050161 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:19.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.185+0000 7f45be8b3640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b806c280 con 0x7f45b806b620 2024-04-25T08:54:19.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.186+0000 7f45bce50640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f45b80b3da0 0x7f45b80aec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.186+0000 7f45b77fe640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f45b815b220 0x7f45b80b3690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:36422/0 (socket says 172.21.15.43:36422) 2024-04-25T08:54:19.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.186+0000 7f45b77fe640 1 -- 172.21.15.43:0/2797176891 learned_addr learned my addr 172.21.15.43:0/2797176891 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:19.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.186+0000 7f45b7fff640 1 --2- 172.21.15.43:0/2797176891 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45b806b620 0x7f45b80ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.186+0000 7fbc5e591640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fbc58070250 0x7fbc5806dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60936/0 (socket says 172.21.15.43:60936) 2024-04-25T08:54:19.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.186+0000 7f2f37ab3640 1 -- 172.21.15.43:0/382050161 --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f300af050 con 0x7f2f3015b220 2024-04-25T08:54:19.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.186+0000 7f2f37ab3640 1 -- 172.21.15.43:0/382050161 --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f300b4310 con 0x7f2f300ae730 2024-04-25T08:54:19.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.186+0000 7f2f37ab3640 1 -- 172.21.15.43:0/382050161 --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f300b45f0 con 0x7f2f300b3da0 2024-04-25T08:54:19.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.186+0000 7f2f36050640 1 --2- 172.21.15.43:0/382050161 >> v2:172.21.15.193:3300/0 conn(0x7f2f300b3da0 0x7f2f3006cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.186+0000 7f45b6ffd640 1 -- 172.21.15.43:0/2797176891 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2309753516 0 0) 0x7f45b80b4310 con 0x7f45b815b220 2024-04-25T08:54:19.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.186+0000 7f69eca80640 1 Processor -- start 2024-04-25T08:54:19.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.186+0000 7f2f3484d640 1 -- 172.21.15.43:0/382050161 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2695695668 0 0) 0x7f2f300af050 con 0x7f2f3015b220 2024-04-25T08:54:19.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.186+0000 7f2f3484d640 1 -- 172.21.15.43:0/382050161 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f1c005730 con 0x7f2f3015b220 2024-04-25T08:54:19.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.186+0000 7f2f3584f640 1 -- 172.21.15.43:0/382050161 >> v2:172.21.15.193:3300/0 conn(0x7f2f300b3da0 msgr2=0x7f2f3006cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:19.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.186+0000 7f2f3584f640 1 --2- 172.21.15.43:0/382050161 >> v2:172.21.15.193:3300/0 conn(0x7f2f300b3da0 0x7f2f3006cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.186+0000 7f2f3584f640 1 -- 172.21.15.43:0/382050161 >> v1:172.21.15.193:6789/0 conn(0x7f2f3015b220 legacy=0x7f2f300b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.186+0000 7f2f3584f640 1 -- 172.21.15.43:0/382050161 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f3006c280 con 0x7f2f300ae730 2024-04-25T08:54:19.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.186+0000 7f45b6ffd640 1 -- 172.21.15.43:0/2797176891 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45a0003520 con 0x7f45b815b220 2024-04-25T08:54:19.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.186+0000 7fbc5f593640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc5815e450 con 0x7fbc58070250 2024-04-25T08:54:19.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.186+0000 7fbc5f593640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc5815d690 con 0x7fbc5815a100 2024-04-25T08:54:19.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.186+0000 7fbc5f593640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc580b4bd0 con 0x7fbc580b4230 2024-04-25T08:54:19.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.186+0000 7fbc5e591640 1 -- 172.21.15.43:0/2654855382 learned_addr learned my addr 172.21.15.43:0/2654855382 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:19.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.186+0000 7fbc5dd90640 1 --2- 172.21.15.43:0/2654855382 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc580b4230 0x7fbc580b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.187+0000 7fbc5dd90640 1 -- 172.21.15.43:0/2654855382 >> v1:172.21.15.193:6789/0 conn(0x7fbc5815a100 legacy=0x7fbc5815a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.187+0000 7fbc5dd90640 1 -- 172.21.15.43:0/2654855382 >> v1:172.21.15.43:6789/0 conn(0x7fbc58070250 legacy=0x7fbc5806dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.187+0000 7fbc5dd90640 1 -- 172.21.15.43:0/2654855382 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc5815d970 con 0x7fbc580b4230 2024-04-25T08:54:19.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.186+0000 7f0891aa7640 1 -- 172.21.15.43:0/151852540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f088c0b4650 msgr2=0x7f088c06ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.186+0000 7f0891aa7640 1 --2- 172.21.15.43:0/151852540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f088c0b4650 0x7f088c06ace0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f088c154ac0 tx=0x7f088402a680 comp rx=0 tx=0).stop 2024-04-25T08:54:19.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.187+0000 7f45bce50640 1 -- 172.21.15.43:0/2797176891 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45b806b620 msgr2=0x7f45b80ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:19.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.187+0000 7f45bce50640 1 --2- 172.21.15.43:0/2797176891 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45b806b620 0x7f45b80ae730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.187+0000 7f45bce50640 1 -- 172.21.15.43:0/2797176891 >> v1:172.21.15.193:6789/0 conn(0x7f45b815b220 legacy=0x7f45b80b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.187+0000 7f45bce50640 1 -- 172.21.15.43:0/2797176891 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b806c560 con 0x7f45b80b3da0 2024-04-25T08:54:19.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.187+0000 7f66201b0640 1 Processor -- start 2024-04-25T08:54:19.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.188+0000 7f66201b0640 1 -- start start 2024-04-25T08:54:19.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.188+0000 7f2f3584f640 1 --2- 172.21.15.43:0/382050161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f300ae730 0x7f2f300aeb10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2f300b48d0 tx=0x7f2f2802a8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93872a63cfc5cfc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:19.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.188+0000 7fbc5dd90640 1 --2- 172.21.15.43:0/2654855382 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc580b4230 0x7fbc580b4690 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbc480097d0 tx=0x7fbc4802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff3fd0997034b0d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:19.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.188+0000 7f45bce50640 1 --2- 172.21.15.43:0/2797176891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45b80b3da0 0x7f45b80aec70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f45a80097a0 tx=0x7f45a802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=263ba847945be5af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:19.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.186+0000 7f69eca80640 1 -- start start 2024-04-25T08:54:19.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.188+0000 7f69eca80640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f69e0005e90 0x7f69e00062d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.188+0000 7f69eca80640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69e0053720 con 0x7f69e0006810 2024-04-25T08:54:19.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.188+0000 7f69eca80640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69e0053a00 con 0x7f69e00ead00 2024-04-25T08:54:19.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.188+0000 7f69eca80640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69e0054330 con 0x7f69e0005e90 2024-04-25T08:54:19.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.188+0000 7f45b6ffd640 1 -- 172.21.15.43:0/2797176891 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a8037070 con 0x7f45b80b3da0 2024-04-25T08:54:19.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.188+0000 7fbc5d58f640 1 -- 172.21.15.43:0/2654855382 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc48037070 con 0x7fbc580b4230 2024-04-25T08:54:19.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.188+0000 7f2f3484d640 1 -- 172.21.15.43:0/382050161 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2800c040 con 0x7f2f300ae730 2024-04-25T08:54:19.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.188+0000 7f2f3484d640 1 -- 172.21.15.43:0/382050161 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f28036070 con 0x7f2f300ae730 2024-04-25T08:54:19.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.188+0000 7f2f3484d640 1 -- 172.21.15.43:0/382050161 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f28030a30 con 0x7f2f300ae730 2024-04-25T08:54:19.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.188+0000 7fbc5d58f640 1 -- 172.21.15.43:0/2654855382 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc48032440 con 0x7fbc580b4230 2024-04-25T08:54:19.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.188+0000 7fbc5d58f640 1 -- 172.21.15.43:0/2654855382 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc48032810 con 0x7fbc580b4230 2024-04-25T08:54:19.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.188+0000 7f69e7fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f69e0006810 0x7f69e0006c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:60940/0 (socket says 172.21.15.43:60940) 2024-04-25T08:54:19.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.188+0000 7f69e7fff640 1 -- 172.21.15.43:0/344644895 learned_addr learned my addr 172.21.15.43:0/344644895 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:19.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.188+0000 7f45b6ffd640 1 -- 172.21.15.43:0/2797176891 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45a8032440 con 0x7f45b80b3da0 2024-04-25T08:54:19.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.188+0000 7f0891aa7640 1 -- 172.21.15.43:0/151852540 shutdown_connections 2024-04-25T08:54:19.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.188+0000 7f0891aa7640 1 --2- 172.21.15.43:0/151852540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f088c0b4650 0x7f088c06ace0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.188+0000 7f0891aa7640 1 --2- 172.21.15.43:0/151852540 >> v2:172.21.15.43:3300/0 conn(0x7f088c0cc6e0 0x7f088c0b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.188+0000 7f0891aa7640 1 -- 172.21.15.43:0/151852540 >> 172.21.15.43:0/151852540 conn(0x7f088c0b1fa0 msgr2=0x7f088c0b23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:19.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.188+0000 7f69e67fc640 1 -- 172.21.15.43:0/344644895 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2339926278 0 0) 0x7f69e0053720 con 0x7f69e0006810 2024-04-25T08:54:19.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.189+0000 7f69e67fc640 1 -- 172.21.15.43:0/344644895 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69d8003520 con 0x7f69e0006810 2024-04-25T08:54:19.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.189+0000 7f69e6ffd640 1 --2- 172.21.15.43:0/344644895 >> v2:172.21.15.43:3300/0 conn(0x7f69e0005e90 0x7f69e00062d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.189+0000 7f69e67fc640 1 -- 172.21.15.43:0/344644895 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2301129969 0 0) 0x7f69d8003520 con 0x7f69e0006810 2024-04-25T08:54:19.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.189+0000 7f69e67fc640 1 -- 172.21.15.43:0/344644895 >> v2:172.21.15.43:3300/0 conn(0x7f69e0005e90 msgr2=0x7f69e00062d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:19.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.189+0000 7f69e67fc640 1 --2- 172.21.15.43:0/344644895 >> v2:172.21.15.43:3300/0 conn(0x7f69e0005e90 0x7f69e00062d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.189+0000 7f69e67fc640 1 -- 172.21.15.43:0/344644895 >> v1:172.21.15.193:6789/0 conn(0x7f69e00ead00 legacy=0x7f69e004fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:19.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.189+0000 7f69e67fc640 1 -- 172.21.15.43:0/344644895 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e0053ce0 con 0x7f69e0006810 2024-04-25T08:54:19.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.189+0000 7f69e67fc640 1 -- 172.21.15.43:0/344644895 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f69dc001670 con 0x7f69e0006810 2024-04-25T08:54:19.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.189+0000 7f69e67fc640 1 -- 172.21.15.43:0/344644895 >> v1:172.21.15.43:6789/0 conn(0x7f69e0006810 legacy=0x7f69e0006c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.189+0000 7f69e67fc640 1 --2- 172.21.15.43:0/344644895 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69d80039b0 0x7f69d8003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.189+0000 7f69e67fc640 1 --2- 172.21.15.43:0/344644895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69d8004800 0x7f69d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.189+0000 7f69e67fc640 1 --2- 172.21.15.43:0/344644895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69d80051c0 0x7f69d800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.189+0000 7f69e67fc640 1 -- 172.21.15.43:0/344644895 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e0054330 con 0x7f69d80051c0 2024-04-25T08:54:19.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.189+0000 7f69e67fc640 1 -- 172.21.15.43:0/344644895 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d800c0f0 con 0x7f69d8004800 2024-04-25T08:54:19.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.189+0000 7f69e67fc640 1 -- 172.21.15.43:0/344644895 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69d800c2f0 con 0x7f69d80039b0 2024-04-25T08:54:19.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.189+0000 7f66201b0640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f661004fbd0 0x7f66100e7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.189+0000 7f66201b0640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f661003aa00 0x7f6610006a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.189+0000 7f66201b0640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6610054570 con 0x7f66100e3150 2024-04-25T08:54:19.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.189+0000 7f66201b0640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6610050140 con 0x7f661004fbd0 2024-04-25T08:54:19.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.189+0000 7f66201b0640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66100062d0 con 0x7f661003aa00 2024-04-25T08:54:19.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.189+0000 7f69e6ffd640 1 --2- 172.21.15.43:0/344644895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69d8004800 0x7f69d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.189+0000 7f45be8b3640 1 -- 172.21.15.43:0/2797176891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45b80b3da0 msgr2=0x7f45b80aec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.189+0000 7f45be8b3640 1 --2- 172.21.15.43:0/2797176891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45b80b3da0 0x7f45b80aec70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f45a80097a0 tx=0x7f45a802a230 comp rx=0 tx=0).stop 2024-04-25T08:54:19.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.189+0000 7f2f37ab3640 1 -- 172.21.15.43:0/382050161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f300ae730 msgr2=0x7f2f300aeb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.189+0000 7f2f37ab3640 1 --2- 172.21.15.43:0/382050161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f300ae730 0x7f2f300aeb10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2f300b48d0 tx=0x7f2f2802a8d0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.189+0000 7f2f37ab3640 1 -- 172.21.15.43:0/382050161 shutdown_connections 2024-04-25T08:54:19.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.189+0000 7f2f37ab3640 1 --2- 172.21.15.43:0/382050161 >> v2:172.21.15.193:3300/0 conn(0x7f2f300b3da0 0x7f2f3006cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.189+0000 7f2f37ab3640 1 --2- 172.21.15.43:0/382050161 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f300ae730 0x7f2f300aeb10 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.189+0000 7f2f37ab3640 1 -- 172.21.15.43:0/382050161 >> 172.21.15.43:0/382050161 conn(0x7f2f3006ab30 msgr2=0x7f2f3006af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:19.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.189+0000 7f661e74d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f661003aa00 0x7f6610006a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.189+0000 7f661e74d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f661003aa00 0x7f6610006a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:49150/0 (socket says 172.21.15.43:49150) 2024-04-25T08:54:19.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.189+0000 7fbc5f593640 1 -- 172.21.15.43:0/2654855382 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc580b4230 msgr2=0x7fbc580b4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.189+0000 7f661e74d640 1 -- 172.21.15.43:0/3529057240 learned_addr learned my addr 172.21.15.43:0/3529057240 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:19.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.190+0000 7fbc5f593640 1 --2- 172.21.15.43:0/2654855382 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc580b4230 0x7fbc580b4690 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbc480097d0 tx=0x7fbc4802a180 comp rx=0 tx=0).stop 2024-04-25T08:54:19.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.190+0000 7f45be8b3640 1 -- 172.21.15.43:0/2797176891 shutdown_connections 2024-04-25T08:54:19.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.190+0000 7f45be8b3640 1 --2- 172.21.15.43:0/2797176891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45b80b3da0 0x7f45b80aec70 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.190+0000 7f45be8b3640 1 --2- 172.21.15.43:0/2797176891 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45b806b620 0x7f45b80ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.190+0000 7f45be8b3640 1 -- 172.21.15.43:0/2797176891 >> 172.21.15.43:0/2797176891 conn(0x7f45b806ab30 msgr2=0x7f45b806af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:19.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.190+0000 7fbc5f593640 1 -- 172.21.15.43:0/2654855382 shutdown_connections 2024-04-25T08:54:19.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.190+0000 7fbc5f593640 1 --2- 172.21.15.43:0/2654855382 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc580b4230 0x7fbc580b4690 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.190+0000 7fbc5f593640 1 -- 172.21.15.43:0/2654855382 >> 172.21.15.43:0/2654855382 conn(0x7fbc5806ad10 msgr2=0x7fbc5806b0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:19.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.190+0000 7f2f37ab3640 1 -- 172.21.15.43:0/382050161 shutdown_connections 2024-04-25T08:54:19.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.190+0000 7f2f37ab3640 1 -- 172.21.15.43:0/382050161 wait complete. 2024-04-25T08:54:19.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.190+0000 7f0891aa7640 1 -- 172.21.15.43:0/151852540 shutdown_connections 2024-04-25T08:54:19.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.190+0000 7f69e7fff640 1 --2- 172.21.15.43:0/344644895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69d80051c0 0x7f69d800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.190+0000 7f69e77fe640 1 --2- 172.21.15.43:0/344644895 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69d80039b0 0x7f69d8003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.190+0000 7f661d74b640 1 --2- 172.21.15.43:0/3529057240 >> v2:172.21.15.193:3300/0 conn(0x7f661004fbd0 0x7f66100e7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.190+0000 7f661e74d640 1 -- 172.21.15.43:0/3529057240 >> v2:172.21.15.193:3300/0 conn(0x7f661004fbd0 msgr2=0x7f66100e7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:19.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.190+0000 7f661e74d640 1 --2- 172.21.15.43:0/3529057240 >> v2:172.21.15.193:3300/0 conn(0x7f661004fbd0 0x7f66100e7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.190+0000 7f661e74d640 1 -- 172.21.15.43:0/3529057240 >> v1:172.21.15.193:6789/0 conn(0x7f66100e3150 legacy=0x7f661004f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.190+0000 7f661e74d640 1 -- 172.21.15.43:0/3529057240 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6610054850 con 0x7f661003aa00 2024-04-25T08:54:19.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.190+0000 7f661d74b640 1 --2- 172.21.15.43:0/3529057240 >> v2:172.21.15.193:3300/0 conn(0x7f661004fbd0 0x7f66100e7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:19.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.190+0000 7f45be8b3640 1 -- 172.21.15.43:0/2797176891 shutdown_connections 2024-04-25T08:54:19.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.190+0000 7fbc5f593640 1 -- 172.21.15.43:0/2654855382 shutdown_connections 2024-04-25T08:54:19.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.190+0000 7f0891aa7640 1 -- 172.21.15.43:0/151852540 wait complete. 2024-04-25T08:54:19.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.190+0000 7f45be8b3640 1 -- 172.21.15.43:0/2797176891 wait complete. 2024-04-25T08:54:19.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.191+0000 7f661e74d640 1 --2- 172.21.15.43:0/3529057240 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f661003aa00 0x7f6610006a40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6618062780 tx=0x7f66180a4f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b22639628e6cc1e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:19.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.190+0000 7f69e7fff640 1 -- 172.21.15.43:0/344644895 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69d80039b0 msgr2=0x7f69d8003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:19.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.191+0000 7fbc5f593640 1 -- 172.21.15.43:0/2654855382 wait complete. 2024-04-25T08:54:19.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.190+0000 7f69e7fff640 1 --2- 172.21.15.43:0/344644895 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69d80039b0 0x7f69d8003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.190+0000 7f69e7fff640 1 -- 172.21.15.43:0/344644895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69d8004800 msgr2=0x7f69d8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:19.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.190+0000 7f69e7fff640 1 --2- 172.21.15.43:0/344644895 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69d8004800 0x7f69d8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.190+0000 7f69e7fff640 1 -- 172.21.15.43:0/344644895 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e0053ce0 con 0x7f69d80051c0 2024-04-25T08:54:19.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.191+0000 7f69e77fe640 1 --2- 172.21.15.43:0/344644895 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69d80039b0 0x7f69d8003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:54:19.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.192+0000 7f661cf4a640 1 -- 172.21.15.43:0/3529057240 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66180b2070 con 0x7f661003aa00 2024-04-25T08:54:19.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.192+0000 7f661cf4a640 1 -- 172.21.15.43:0/3529057240 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66180a5d90 con 0x7f661003aa00 2024-04-25T08:54:19.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.192+0000 7f45be8b3640 1 Processor -- start 2024-04-25T08:54:19.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.192+0000 7f0891aa7640 1 Processor -- start 2024-04-25T08:54:19.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.192+0000 7f2f37ab3640 1 Processor -- start 2024-04-25T08:54:19.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.192+0000 7f45be8b3640 1 -- start start 2024-04-25T08:54:19.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.192+0000 7f69e7fff640 1 --2- 172.21.15.43:0/344644895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69d80051c0 0x7f69d800b9c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f69dc007fd0 tx=0x7f69d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a28ecaae0f308b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:19.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.192+0000 7f69e7fff640 1 -- 172.21.15.43:0/344644895 reap_dead start 2024-04-25T08:54:19.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.192+0000 7f69e67fc640 1 -- 172.21.15.43:0/344644895 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69dc036070 con 0x7f69d80051c0 2024-04-25T08:54:19.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.192+0000 7f2f37ab3640 1 -- start start 2024-04-25T08:54:19.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.192+0000 7f69e67fc640 1 -- 172.21.15.43:0/344644895 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69dc005ad0 con 0x7f69d80051c0 2024-04-25T08:54:19.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.193+0000 7f66201b0640 1 -- 172.21.15.43:0/3529057240 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f661003aa00 msgr2=0x7f6610006a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.193+0000 7f66201b0640 1 --2- 172.21.15.43:0/3529057240 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f661003aa00 0x7f6610006a40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6618062780 tx=0x7f66180a4f40 comp rx=0 tx=0).stop 2024-04-25T08:54:19.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.193+0000 7f66201b0640 1 -- 172.21.15.43:0/3529057240 shutdown_connections 2024-04-25T08:54:19.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.193+0000 7f66201b0640 1 --2- 172.21.15.43:0/3529057240 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f661003aa00 0x7f6610006a40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.193+0000 7f66201b0640 1 --2- 172.21.15.43:0/3529057240 >> v2:172.21.15.193:3300/0 conn(0x7f661004fbd0 0x7f66100e7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.193+0000 7f66201b0640 1 -- 172.21.15.43:0/3529057240 >> 172.21.15.43:0/3529057240 conn(0x7f661000a3f0 msgr2=0x7f661000a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:19.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.193+0000 7fbc5f593640 1 Processor -- start 2024-04-25T08:54:19.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.193+0000 7f69e67fc640 1 -- 172.21.15.43:0/344644895 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e804fe10 con 0x7f69d80051c0 2024-04-25T08:54:19.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.193+0000 7fbc5f593640 1 -- start start 2024-04-25T08:54:19.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.192+0000 7f0891aa7640 1 -- start start 2024-04-25T08:54:19.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.194+0000 7f0891aa7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f088c0b4650 0x7f088c146690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.194+0000 7f0891aa7640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f088c0cc300 0x7f088c14f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.194+0000 7f0891aa7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f088c0cc6e0 0x7f088c149140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.194+0000 7f0891aa7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088c072180 con 0x7f088c0b4650 2024-04-25T08:54:19.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.194+0000 7f0891aa7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088c0b2e80 con 0x7f088c0cc6e0 2024-04-25T08:54:19.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.194+0000 7f0891aa7640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f088c072300 con 0x7f088c0cc300 2024-04-25T08:54:19.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.193+0000 7f69eca80640 1 -- 172.21.15.43:0/344644895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69d80051c0 msgr2=0x7f69d800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.193+0000 7f69eca80640 1 --2- 172.21.15.43:0/344644895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69d80051c0 0x7f69d800b9c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f69dc007fd0 tx=0x7f69d8005830 comp rx=0 tx=0).stop 2024-04-25T08:54:19.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.193+0000 7f69eca80640 1 -- 172.21.15.43:0/344644895 shutdown_connections 2024-04-25T08:54:19.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.193+0000 7f69eca80640 1 --2- 172.21.15.43:0/344644895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69d80051c0 0x7f69d800b9c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.193+0000 7f69eca80640 1 -- 172.21.15.43:0/344644895 >> 172.21.15.43:0/344644895 conn(0x7f69e0009fb0 msgr2=0x7f69e000a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:19.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.194+0000 7f69eca80640 1 -- 172.21.15.43:0/344644895 shutdown_connections 2024-04-25T08:54:19.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.194+0000 7f69eca80640 1 -- 172.21.15.43:0/344644895 wait complete. 2024-04-25T08:54:19.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.193+0000 7f45be8b3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45b806b620 0x7f45b815a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.193+0000 7f45be8b3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45b80b3da0 0x7f45b8155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.193+0000 7f45be8b3640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45b815b220 0x7f45b814b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.193+0000 7f45be8b3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b8154510 con 0x7f45b806b620 2024-04-25T08:54:19.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.193+0000 7f45be8b3640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45b8154390 con 0x7f45b80b3da0 2024-04-25T08:54:19.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.193+0000 7f45b7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45b806b620 0x7f45b815a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.193+0000 7f45be8b3640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45b806cd60 con 0x7f45b815b220 2024-04-25T08:54:19.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.193+0000 7f45b7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45b806b620 0x7f45b815a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37654/0 (socket says 172.21.15.43:37654) 2024-04-25T08:54:19.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.194+0000 7f45b7fff640 1 -- 172.21.15.43:0/1345348057 learned_addr learned my addr 172.21.15.43:0/1345348057 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:19.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.194+0000 7f45bce50640 1 --2- 172.21.15.43:0/1345348057 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45b815b220 0x7f45b814b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.194+0000 7f45b77fe640 1 --2- 172.21.15.43:0/1345348057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45b80b3da0 0x7f45b8155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.193+0000 7f66201b0640 1 -- 172.21.15.43:0/3529057240 shutdown_connections 2024-04-25T08:54:19.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.193+0000 7f66201b0640 1 -- 172.21.15.43:0/3529057240 wait complete. 2024-04-25T08:54:19.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.194+0000 7f0890aa5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f088c0b4650 0x7f088c146690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.194+0000 7f45bce50640 1 -- 172.21.15.43:0/1345348057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45b80b3da0 msgr2=0x7f45b8155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.194+0000 7f45bce50640 1 --2- 172.21.15.43:0/1345348057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45b80b3da0 0x7f45b8155ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.194+0000 7f45bce50640 1 -- 172.21.15.43:0/1345348057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45b806b620 msgr2=0x7f45b815a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.194+0000 7f45bce50640 1 --2- 172.21.15.43:0/1345348057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45b806b620 0x7f45b815a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.194+0000 7f45bce50640 1 -- 172.21.15.43:0/1345348057 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b8159580 con 0x7f45b815b220 2024-04-25T08:54:19.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.194+0000 7f45bce50640 1 --2- 172.21.15.43:0/1345348057 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45b815b220 0x7f45b814b6f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f45a8031000 tx=0x7f45a80046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:19.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.194+0000 7f0890aa5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f088c0b4650 0x7f088c146690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37656/0 (socket says 172.21.15.43:37656) 2024-04-25T08:54:19.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.194+0000 7f0890aa5640 1 -- 172.21.15.43:0/1936221786 learned_addr learned my addr 172.21.15.43:0/1936221786 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:19.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.194+0000 7f088bfff640 1 --2- 172.21.15.43:0/1936221786 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f088c0cc300 0x7f088c14f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.194+0000 7f08912a6640 1 --2- 172.21.15.43:0/1936221786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f088c0cc6e0 0x7f088c149140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.195+0000 7fbc5f593640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc58070250 0x7fbc5814c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.194+0000 7f2f37ab3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f300ae730 0x7f2f30150660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.194+0000 7f2f37ab3640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2f300b3da0 0x7f2f30158cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.195+0000 7f45b4ff9640 1 -- 172.21.15.43:0/1345348057 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a8040020 con 0x7f45b815b220 2024-04-25T08:54:19.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.195+0000 7f45b4ff9640 1 -- 172.21.15.43:0/1345348057 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45a8038040 con 0x7f45b815b220 2024-04-25T08:54:19.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.195+0000 7f45b4ff9640 1 -- 172.21.15.43:0/1345348057 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a8037070 con 0x7f45b815b220 2024-04-25T08:54:19.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.195+0000 7f45be8b3640 1 -- 172.21.15.43:0/1345348057 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45b8159860 con 0x7f45b815b220 2024-04-25T08:54:19.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.195+0000 7f45be8b3640 1 -- 172.21.15.43:0/1345348057 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45b8159e20 con 0x7f45b815b220 2024-04-25T08:54:19.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.195+0000 7fbc5f593640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc580b4230 0x7fbc580cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.194+0000 7f2f37ab3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f3015b220 0x7f2f30159210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.194+0000 7f2f37ab3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f30154420 con 0x7f2f300ae730 2024-04-25T08:54:19.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.194+0000 7f2f37ab3640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3006c840 con 0x7f2f3015b220 2024-04-25T08:54:19.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.194+0000 7f2f37ab3640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f301545a0 con 0x7f2f300b3da0 2024-04-25T08:54:19.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.194+0000 7f2f3584f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f300ae730 0x7f2f30150660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.194+0000 7f2f3584f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f300ae730 0x7f2f30150660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37662/0 (socket says 172.21.15.43:37662) 2024-04-25T08:54:19.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.194+0000 7f2f3584f640 1 -- 172.21.15.43:0/422241290 learned_addr learned my addr 172.21.15.43:0/422241290 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:19.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.195+0000 7f66201b0640 1 Processor -- start 2024-04-25T08:54:19.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.195+0000 7fbc5e591640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc58070250 0x7fbc5814c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.195+0000 7fbc5e591640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc58070250 0x7fbc5814c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:49188/0 (socket says 172.21.15.43:49188) 2024-04-25T08:54:19.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.195+0000 7f66201b0640 1 -- start start 2024-04-25T08:54:19.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.195+0000 7fbc5e591640 1 -- 172.21.15.43:0/1688174207 learned_addr learned my addr 172.21.15.43:0/1688174207 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:19.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.195+0000 7fbc5f593640 1 --2- 172.21.15.43:0/1688174207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc5815a100 0x7fbc580cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.195+0000 7fbc5dd90640 1 --2- 172.21.15.43:0/1688174207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc580b4230 0x7fbc580cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.195+0000 7f2f3584f640 1 -- 172.21.15.43:0/422241290 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2f300b3da0 msgr2=0x7f2f30158cd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T08:54:19.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.195+0000 7f2f3584f640 1 --2- 172.21.15.43:0/422241290 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2f300b3da0 0x7f2f30158cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.195+0000 7fbc5f593640 1 -- 172.21.15.43:0/1688174207 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5815df30 con 0x7fbc580b4230 2024-04-25T08:54:19.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.195+0000 7f2f3584f640 1 -- 172.21.15.43:0/422241290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f3015b220 msgr2=0x7f2f30159210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T08:54:19.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.195+0000 7f2f3584f640 1 --2- 172.21.15.43:0/422241290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f3015b220 0x7f2f30159210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.195+0000 7f2f3584f640 1 -- 172.21.15.43:0/422241290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f30155bd0 con 0x7f2f300ae730 2024-04-25T08:54:19.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.195+0000 7fbc5f593640 1 -- 172.21.15.43:0/1688174207 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5815aea0 con 0x7fbc5815a100 2024-04-25T08:54:19.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.196+0000 7fbc5ed92640 1 --2- 172.21.15.43:0/1688174207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc5815a100 0x7fbc580cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.195+0000 7f088bfff640 1 -- 172.21.15.43:0/1936221786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f088c0cc6e0 msgr2=0x7f088c149140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.196+0000 7fbc5f593640 1 -- 172.21.15.43:0/1688174207 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc580719c0 con 0x7fbc58070250 2024-04-25T08:54:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.196+0000 7f2f3584f640 1 --2- 172.21.15.43:0/422241290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f300ae730 0x7f2f30150660 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2f28033040 tx=0x7f2f28004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:19.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.195+0000 7f088bfff640 1 --2- 172.21.15.43:0/1936221786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f088c0cc6e0 0x7f088c149140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.196+0000 7f2f267fc640 1 -- 172.21.15.43:0/422241290 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f28031a10 con 0x7f2f300ae730 2024-04-25T08:54:19.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.196+0000 7f2f267fc640 1 -- 172.21.15.43:0/422241290 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f28031bb0 con 0x7f2f300ae730 2024-04-25T08:54:19.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.196+0000 7f2f267fc640 1 -- 172.21.15.43:0/422241290 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f28044430 con 0x7f2f300ae730 2024-04-25T08:54:19.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.196+0000 7f088bfff640 1 -- 172.21.15.43:0/1936221786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f088c0b4650 msgr2=0x7f088c146690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.196+0000 7f2f37ab3640 1 -- 172.21.15.43:0/422241290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f301458d0 con 0x7f2f300ae730 2024-04-25T08:54:19.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.196+0000 7f2f37ab3640 1 -- 172.21.15.43:0/422241290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f3014b830 con 0x7f2f300ae730 2024-04-25T08:54:19.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.197+0000 7f66201b0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f661003aa00 0x7f66100d9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.197+0000 7fbc5dd90640 1 -- 172.21.15.43:0/1688174207 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc58070250 msgr2=0x7fbc5814c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.197+0000 7fbc5dd90640 1 --2- 172.21.15.43:0/1688174207 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc58070250 0x7fbc5814c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.197+0000 7fbc5dd90640 1 -- 172.21.15.43:0/1688174207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc5815a100 msgr2=0x7fbc580cdd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.197+0000 7fbc5dd90640 1 --2- 172.21.15.43:0/1688174207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc5815a100 0x7fbc580cdd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.197+0000 7fbc5dd90640 1 -- 172.21.15.43:0/1688174207 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc580ccbd0 con 0x7fbc580b4230 2024-04-25T08:54:19.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.197+0000 7fbc5dd90640 1 --2- 172.21.15.43:0/1688174207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc580b4230 0x7fbc580cd7e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbc48031000 tx=0x7fbc480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:19.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.197+0000 7fbc5e591640 1 --2- 172.21.15.43:0/1688174207 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc58070250 0x7fbc5814c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:19.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.197+0000 7fbc5ed92640 1 --2- 172.21.15.43:0/1688174207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc5815a100 0x7fbc580cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:19.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.196+0000 7f088bfff640 1 --2- 172.21.15.43:0/1936221786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f088c0b4650 0x7f088c146690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.196+0000 7f088bfff640 1 -- 172.21.15.43:0/1936221786 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f088c1480a0 con 0x7f088c0cc300 2024-04-25T08:54:19.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.197+0000 7f08912a6640 1 --2- 172.21.15.43:0/1936221786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f088c0cc6e0 0x7f088c149140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:54:19.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.197+0000 7f0890aa5640 1 --2- 172.21.15.43:0/1936221786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f088c0b4650 0x7f088c146690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:54:19.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.197+0000 7f088bfff640 1 --2- 172.21.15.43:0/1936221786 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f088c0cc300 0x7f088c14f3c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f087c00d820 tx=0x7f087c00dce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:19.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.197+0000 7f45b4ff9640 1 -- 172.21.15.43:0/1345348057 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f45a8043020 con 0x7f45b815b220 2024-04-25T08:54:19.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.198+0000 7f45b4ff9640 1 --2- 172.21.15.43:0/1345348057 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f45a406b520 0x7f45a406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.198+0000 7f45b4ff9640 1 -- 172.21.15.43:0/1345348057 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f45a80aa5c0 con 0x7f45b815b220 2024-04-25T08:54:19.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.198+0000 7f66201b0640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f661004fbd0 0x7f66100d96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.198+0000 7f66201b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66100e3150 0x7f66100e5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.198+0000 7f69eca80640 1 Processor -- start 2024-04-25T08:54:19.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.198+0000 7f08897fa640 1 -- 172.21.15.43:0/1936221786 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087c016070 con 0x7f088c0cc300 2024-04-25T08:54:19.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.198+0000 7f66201b0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66100e3530 con 0x7f66100e3150 2024-04-25T08:54:19.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.198+0000 7f66201b0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6610006520 con 0x7f661003aa00 2024-04-25T08:54:19.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.198+0000 7f08897fa640 1 -- 172.21.15.43:0/1936221786 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087c007920 con 0x7f088c0cc300 2024-04-25T08:54:19.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.198+0000 7f08897fa640 1 -- 172.21.15.43:0/1936221786 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087c00aae0 con 0x7f088c0cc300 2024-04-25T08:54:19.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.198+0000 7f2f37ab3640 1 -- 172.21.15.43:0/422241290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f30074ef0 con 0x7f2f300ae730 2024-04-25T08:54:19.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.198+0000 7f45b7fff640 1 --2- 172.21.15.43:0/1345348057 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f45a406b520 0x7f45a406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.199+0000 7f45be8b3640 1 -- 172.21.15.43:0/1345348057 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4598005290 con 0x7f45b815b220 2024-04-25T08:54:19.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.199+0000 7fbc4effd640 1 -- 172.21.15.43:0/1688174207 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc48041030 con 0x7fbc580b4230 2024-04-25T08:54:19.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.199+0000 7fbc4effd640 1 -- 172.21.15.43:0/1688174207 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc48030770 con 0x7fbc580b4230 2024-04-25T08:54:19.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.199+0000 7fbc4effd640 1 -- 172.21.15.43:0/1688174207 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc48037040 con 0x7fbc580b4230 2024-04-25T08:54:19.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.199+0000 7f0891aa7640 1 -- 172.21.15.43:0/1936221786 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f088c148380 con 0x7f088c0cc300 2024-04-25T08:54:19.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.199+0000 7f0891aa7640 1 -- 172.21.15.43:0/1936221786 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f088c148940 con 0x7f088c0cc300 2024-04-25T08:54:19.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.200+0000 7f45b7fff640 1 --2- 172.21.15.43:0/1345348057 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f45a406b520 0x7f45a406d9e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f45b0009a40 tx=0x7f45b000d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:19.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.199+0000 7f69eca80640 1 -- start start 2024-04-25T08:54:19.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.198+0000 7f66201b0640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6610006f80 con 0x7f661004fbd0 2024-04-25T08:54:19.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.198+0000 7f661d74b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f661004fbd0 0x7f66100d96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.198+0000 7f661df4c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f661003aa00 0x7f66100d9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.198+0000 7f661d74b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f661004fbd0 0x7f66100d96f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:49198/0 (socket says 172.21.15.43:49198) 2024-04-25T08:54:19.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.198+0000 7f661df4c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f661003aa00 0x7f66100d9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:47108/0 (socket says 172.21.15.43:47108) 2024-04-25T08:54:19.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.198+0000 7f661d74b640 1 -- 172.21.15.43:0/2880382817 learned_addr learned my addr 172.21.15.43:0/2880382817 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:19.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.199+0000 7f661e74d640 1 --2- 172.21.15.43:0/2880382817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66100e3150 0x7f66100e5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.199+0000 7f661e74d640 1 -- 172.21.15.43:0/2880382817 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f661004fbd0 msgr2=0x7f66100d96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.199+0000 7f661e74d640 1 --2- 172.21.15.43:0/2880382817 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f661004fbd0 0x7f66100d96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.199+0000 7f661e74d640 1 -- 172.21.15.43:0/2880382817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f661003aa00 msgr2=0x7f66100d9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.199+0000 7f661e74d640 1 --2- 172.21.15.43:0/2880382817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f661003aa00 0x7f66100d9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.199+0000 7f661e74d640 1 -- 172.21.15.43:0/2880382817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66100e0e40 con 0x7f66100e3150 2024-04-25T08:54:19.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.200+0000 7f661e74d640 1 --2- 172.21.15.43:0/2880382817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66100e3150 0x7f66100e5e50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6618069c60 tx=0x7f6618069680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:19.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.200+0000 7f69eca80640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69d80051c0 0x7f69e004ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.200+0000 7f69eca80640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69e0036860 0x7f69e00dfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.200+0000 7f69eca80640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69e0036ce0 0x7f69e00e0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.200+0000 7f69e77fe640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69d80051c0 0x7f69e004ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.200+0000 7f69e77fe640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69d80051c0 0x7f69e004ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:49210/0 (socket says 172.21.15.43:49210) 2024-04-25T08:54:19.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.200+0000 7f69e6ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69e0036860 0x7f69e00dfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.200+0000 7f69e6ffd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69e0036860 0x7f69e00dfbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:47112/0 (socket says 172.21.15.43:47112) 2024-04-25T08:54:19.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.200+0000 7f69e7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69e0036ce0 0x7f69e00e0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.200+0000 7f69e7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69e0036ce0 0x7f69e00e0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:37698/0 (socket says 172.21.15.43:37698) 2024-04-25T08:54:19.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.200+0000 7f69eca80640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e0054940 con 0x7f69e0036ce0 2024-04-25T08:54:19.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.200+0000 7f69eca80640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e00547c0 con 0x7f69e0036860 2024-04-25T08:54:19.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.200+0000 7f69eca80640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69e004f550 con 0x7f69d80051c0 2024-04-25T08:54:19.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.200+0000 7f69e77fe640 1 -- 172.21.15.43:0/753495933 learned_addr learned my addr 172.21.15.43:0/753495933 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:19.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.200+0000 7fbc5f593640 1 -- 172.21.15.43:0/1688174207 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc58153d20 con 0x7fbc580b4230 2024-04-25T08:54:19.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.200+0000 7fbc5f593640 1 -- 172.21.15.43:0/1688174207 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc581542e0 con 0x7fbc580b4230 2024-04-25T08:54:19.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.200+0000 7f2f267fc640 1 -- 172.21.15.43:0/422241290 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2f2804d440 con 0x7f2f300ae730 2024-04-25T08:54:19.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.201+0000 7f2f267fc640 1 --2- 172.21.15.43:0/422241290 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2f1406b4d0 0x7f2f1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.201+0000 7f69e77fe640 1 -- 172.21.15.43:0/753495933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69e0036860 msgr2=0x7f69e00dfbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.201+0000 7f69e77fe640 1 --2- 172.21.15.43:0/753495933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69e0036860 0x7f69e00dfbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.201+0000 7f69e77fe640 1 -- 172.21.15.43:0/753495933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69e0036ce0 msgr2=0x7f69e00e0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.201+0000 7f69e77fe640 1 --2- 172.21.15.43:0/753495933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69e0036ce0 0x7f69e00e0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.201+0000 7f69e77fe640 1 -- 172.21.15.43:0/753495933 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e004b380 con 0x7f69d80051c0 2024-04-25T08:54:19.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.201+0000 7f2f267fc640 1 -- 172.21.15.43:0/422241290 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2f280ab7b0 con 0x7f2f300ae730 2024-04-25T08:54:19.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.201+0000 7f0891aa7640 1 -- 172.21.15.43:0/1936221786 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f088c0cce90 con 0x7f088c0cc300 2024-04-25T08:54:19.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.201+0000 7f2f3504e640 1 --2- 172.21.15.43:0/422241290 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2f1406b4d0 0x7f2f1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.201+0000 7f69e77fe640 1 --2- 172.21.15.43:0/753495933 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69d80051c0 0x7f69e004ae40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f69e807aeb0 tx=0x7f69e8069260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:19.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.202+0000 7f2f3504e640 1 --2- 172.21.15.43:0/422241290 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2f1406b4d0 0x7f2f1406d990 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2f3014bf40 tx=0x7f2f2c007840 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:19.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.202+0000 7fbc5f593640 1 -- 172.21.15.43:0/1688174207 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc5806e990 con 0x7fbc580b4230 2024-04-25T08:54:19.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.204+0000 7f45b4ff9640 1 -- 172.21.15.43:0/1345348057 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f45a804a030 con 0x7f45b815b220 2024-04-25T08:54:19.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.204+0000 7f69d3fff640 1 -- 172.21.15.43:0/753495933 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e8098070 con 0x7f69d80051c0 2024-04-25T08:54:19.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.204+0000 7f69d3fff640 1 -- 172.21.15.43:0/753495933 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e8068000 con 0x7f69d80051c0 2024-04-25T08:54:19.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.204+0000 7f69eca80640 1 -- 172.21.15.43:0/753495933 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69e0032e10 con 0x7f69d80051c0 2024-04-25T08:54:19.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.204+0000 7f69eca80640 1 -- 172.21.15.43:0/753495933 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69e00333d0 con 0x7f69d80051c0 2024-04-25T08:54:19.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.205+0000 7f69d3fff640 1 -- 172.21.15.43:0/753495933 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e8068780 con 0x7f69d80051c0 2024-04-25T08:54:19.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.205+0000 7f08897fa640 1 -- 172.21.15.43:0/1936221786 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f087c019070 con 0x7f088c0cc300 2024-04-25T08:54:19.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.206+0000 7f08897fa640 1 --2- 172.21.15.43:0/1936221786 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f087806b5a0 0x7f087806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.206+0000 7f08897fa640 1 -- 172.21.15.43:0/1936221786 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f087c08a420 con 0x7f088c0cc300 2024-04-25T08:54:19.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.206+0000 7f0890aa5640 1 --2- 172.21.15.43:0/1936221786 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f087806b5a0 0x7f087806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.206+0000 7f69eca80640 1 -- 172.21.15.43:0/753495933 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69e0033610 con 0x7f69d80051c0 2024-04-25T08:54:19.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.206+0000 7f660e7fc640 1 -- 172.21.15.43:0/2880382817 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66180b2050 con 0x7f66100e3150 2024-04-25T08:54:19.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.206+0000 7f66201b0640 1 -- 172.21.15.43:0/2880382817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66100e1120 con 0x7f66100e3150 2024-04-25T08:54:19.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.206+0000 7f2f267fc640 1 -- 172.21.15.43:0/422241290 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2f2807a590 con 0x7f2f300ae730 2024-04-25T08:54:19.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.207+0000 7f66201b0640 1 -- 172.21.15.43:0/2880382817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66100e16e0 con 0x7f66100e3150 2024-04-25T08:54:19.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.207+0000 7f0890aa5640 1 --2- 172.21.15.43:0/1936221786 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f087806b5a0 0x7f087806da60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0880007b10 tx=0x7f0880007a60 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:19.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.208+0000 7f66201b0640 1 -- 172.21.15.43:0/2880382817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6610008e10 con 0x7f66100e3150 2024-04-25T08:54:19.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.209+0000 7fbc4effd640 1 -- 172.21.15.43:0/1688174207 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbc48044070 con 0x7fbc580b4230 2024-04-25T08:54:19.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.209+0000 7f660e7fc640 1 -- 172.21.15.43:0/2880382817 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66180b3040 con 0x7f66100e3150 2024-04-25T08:54:19.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.209+0000 7f660e7fc640 1 -- 172.21.15.43:0/2880382817 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66180ad440 con 0x7f66100e3150 2024-04-25T08:54:19.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.209+0000 7f08897fa640 1 -- 172.21.15.43:0/1936221786 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f088c0cce90 con 0x7f088c0cc300 2024-04-25T08:54:19.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.209+0000 7fbc4effd640 1 --2- 172.21.15.43:0/1688174207 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbc3c06b550 0x7fbc3c06da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.209+0000 7fbc4effd640 1 -- 172.21.15.43:0/1688174207 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbc480ab4b0 con 0x7fbc580b4230 2024-04-25T08:54:19.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.210+0000 7fbc5e591640 1 --2- 172.21.15.43:0/1688174207 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbc3c06b550 0x7fbc3c06da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.210+0000 7f660e7fc640 1 -- 172.21.15.43:0/2880382817 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f66180d1020 con 0x7f66100e3150 2024-04-25T08:54:19.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.210+0000 7f69d3fff640 1 -- 172.21.15.43:0/753495933 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f69e809b070 con 0x7f69d80051c0 2024-04-25T08:54:19.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.211+0000 7f660e7fc640 1 --2- 172.21.15.43:0/2880382817 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f65fc06b4d0 0x7f65fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.211+0000 7fbc5e591640 1 --2- 172.21.15.43:0/1688174207 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbc3c06b550 0x7fbc3c06da10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbc54000f60 tx=0x7fbc5400d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:19.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.211+0000 7f69d3fff640 1 --2- 172.21.15.43:0/753495933 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f69cc06b4d0 0x7f69cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.211+0000 7f69d3fff640 1 -- 172.21.15.43:0/753495933 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f69e8104cd0 con 0x7f69d80051c0 2024-04-25T08:54:19.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.211+0000 7f69e6ffd640 1 --2- 172.21.15.43:0/753495933 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f69cc06b4d0 0x7f69cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.212+0000 7f69e6ffd640 1 --2- 172.21.15.43:0/753495933 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f69cc06b4d0 0x7f69cc06d990 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f69e0037420 tx=0x7f69d4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:19.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.214+0000 7f661df4c640 1 --2- 172.21.15.43:0/2880382817 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f65fc06b4d0 0x7f65fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.216+0000 7fbc4effd640 1 -- 172.21.15.43:0/1688174207 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbc5806e990 con 0x7fbc580b4230 2024-04-25T08:54:19.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.217+0000 7f660e7fc640 1 -- 172.21.15.43:0/2880382817 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6618102af0 con 0x7f66100e3150 2024-04-25T08:54:19.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.217+0000 7f69d3fff640 1 -- 172.21.15.43:0/753495933 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f69e0033610 con 0x7f69d80051c0 2024-04-25T08:54:19.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.225+0000 7f660e7fc640 1 -- 172.21.15.43:0/2880382817 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6618104a50 con 0x7f66100e3150 2024-04-25T08:54:19.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.227+0000 7f661df4c640 1 --2- 172.21.15.43:0/2880382817 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f65fc06b4d0 0x7f65fc06d990 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6614000f60 tx=0x7f6614009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:19.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.554+0000 7fc45b33c640 1 -- 172.21.15.43:0/1135644383 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc43c005580 con 0x7fc44c054430 2024-04-25T08:54:19.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.555+0000 7fc44affd640 1 -- 172.21.15.43:0/1135644383 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc4540c6690 con 0x7fc44c054430 2024-04-25T08:54:19.545 INFO:teuthology.orchestra.run.smithi043.stdout:55834574855 2024-04-25T08:54:19.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.558+0000 7fc448ff9640 1 -- 172.21.15.43:0/1135644383 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc43806b5d0 msgr2=0x7fc43806da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.558+0000 7fc448ff9640 1 --2- 172.21.15.43:0/1135644383 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc43806b5d0 0x7fc43806da90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fc450009560 tx=0x7fc450008030 comp rx=0 tx=0).stop 2024-04-25T08:54:19.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.558+0000 7fc448ff9640 1 -- 172.21.15.43:0/1135644383 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc44c054430 msgr2=0x7fc44c006f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.558+0000 7fc448ff9640 1 --2- 172.21.15.43:0/1135644383 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc44c054430 0x7fc44c006f00 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fc454069c60 tx=0x7fc45406bf10 comp rx=0 tx=0).stop 2024-04-25T08:54:19.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.558+0000 7fc448ff9640 1 -- 172.21.15.43:0/1135644383 shutdown_connections 2024-04-25T08:54:19.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.559+0000 7fc448ff9640 1 --2- 172.21.15.43:0/1135644383 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc43806b5d0 0x7fc43806da90 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.559+0000 7fc448ff9640 1 --2- 172.21.15.43:0/1135644383 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc44c101010 0x7fc44c1a7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.559+0000 7fc448ff9640 1 --2- 172.21.15.43:0/1135644383 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc44c054430 0x7fc44c006f00 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.559+0000 7fc448ff9640 1 --2- 172.21.15.43:0/1135644383 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc44c033920 0x7fc44c1ab960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.559+0000 7fc448ff9640 1 -- 172.21.15.43:0/1135644383 >> 172.21.15.43:0/1135644383 conn(0x7fc44c009fb0 msgr2=0x7fc44c00a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:19.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.559+0000 7fc448ff9640 1 -- 172.21.15.43:0/1135644383 shutdown_connections 2024-04-25T08:54:19.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.559+0000 7fc448ff9640 1 -- 172.21.15.43:0/1135644383 wait complete. 2024-04-25T08:54:19.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.569+0000 7f8f7cf73640 1 -- 172.21.15.43:0/991765763 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f8f58005580 con 0x7f8f780b4f10 2024-04-25T08:54:19.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.570+0000 7f8f5ffff640 1 -- 172.21.15.43:0/991765763 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8f68058970 con 0x7f8f780b4f10 2024-04-25T08:54:19.560 INFO:teuthology.orchestra.run.smithi043.stdout:55834574854 2024-04-25T08:54:19.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.573+0000 7f8f5dffb640 1 -- 172.21.15.43:0/991765763 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8f6406b5d0 msgr2=0x7f8f6406da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.573+0000 7f8f5dffb640 1 --2- 172.21.15.43:0/991765763 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8f6406b5d0 0x7f8f6406da90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c001710 tx=0x7f8f6c009040 comp rx=0 tx=0).stop 2024-04-25T08:54:19.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.573+0000 7f8f5dffb640 1 -- 172.21.15.43:0/991765763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f780b4f10 msgr2=0x7f8f7815fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.574+0000 7f8f5dffb640 1 --2- 172.21.15.43:0/991765763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f780b4f10 0x7f8f7815fa20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8f6800eb70 tx=0x7f8f6800c520 comp rx=0 tx=0).stop 2024-04-25T08:54:19.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.574+0000 7f8f5dffb640 1 -- 172.21.15.43:0/991765763 shutdown_connections 2024-04-25T08:54:19.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.574+0000 7f8f5dffb640 1 --2- 172.21.15.43:0/991765763 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f780d0d10 0x7f8f780cac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.574+0000 7f8f5dffb640 1 --2- 172.21.15.43:0/991765763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f780b4f10 0x7f8f7815fa20 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.574+0000 7f8f5dffb640 1 --2- 172.21.15.43:0/991765763 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8f6406b5d0 0x7f8f6406da90 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.574+0000 7f8f5dffb640 1 --2- 172.21.15.43:0/991765763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f78071c60 0x7f8f7815bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.575+0000 7f8f5dffb640 1 -- 172.21.15.43:0/991765763 >> 172.21.15.43:0/991765763 conn(0x7f8f7806bc40 msgr2=0x7f8f780b52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:19.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.575+0000 7f8f5dffb640 1 -- 172.21.15.43:0/991765763 shutdown_connections 2024-04-25T08:54:19.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.575+0000 7f8f5dffb640 1 -- 172.21.15.43:0/991765763 wait complete. 2024-04-25T08:54:19.567 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.0 2024-04-25T08:54:19.567 DEBUG:teuthology.parallel:result is None 2024-04-25T08:54:19.583 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2024-04-25T08:54:19.583 DEBUG:teuthology.parallel:result is None 2024-04-25T08:54:19.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.604+0000 7f69eca80640 1 -- 172.21.15.43:0/753495933 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f69e003a570 con 0x7f69d80051c0 2024-04-25T08:54:19.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.605+0000 7f69d3fff640 1 -- 172.21.15.43:0/753495933 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f69e80d34e0 con 0x7f69d80051c0 2024-04-25T08:54:19.595 INFO:teuthology.orchestra.run.smithi043.stdout:55834574854 2024-04-25T08:54:19.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.606+0000 7f45be8b3640 1 -- 172.21.15.43:0/1345348057 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4598005580 con 0x7f45b815b220 2024-04-25T08:54:19.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.607+0000 7f45b4ff9640 1 -- 172.21.15.43:0/1345348057 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f45a8078dd0 con 0x7f45b815b220 2024-04-25T08:54:19.597 INFO:teuthology.orchestra.run.smithi043.stdout:55834574854 2024-04-25T08:54:19.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.607+0000 7f69eca80640 1 -- 172.21.15.43:0/753495933 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f69cc06b4d0 msgr2=0x7f69cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.607+0000 7f69eca80640 1 --2- 172.21.15.43:0/753495933 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f69cc06b4d0 0x7f69cc06d990 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f69e0037420 tx=0x7f69d4008040 comp rx=0 tx=0).stop 2024-04-25T08:54:19.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.607+0000 7f69eca80640 1 -- 172.21.15.43:0/753495933 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69d80051c0 msgr2=0x7f69e004ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.607+0000 7f69eca80640 1 --2- 172.21.15.43:0/753495933 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69d80051c0 0x7f69e004ae40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f69e807aeb0 tx=0x7f69e8069260 comp rx=0 tx=0).stop 2024-04-25T08:54:19.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.607+0000 7f69eca80640 1 -- 172.21.15.43:0/753495933 shutdown_connections 2024-04-25T08:54:19.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.607+0000 7f69eca80640 1 --2- 172.21.15.43:0/753495933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69e0036ce0 0x7f69e00e0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.607+0000 7f69eca80640 1 --2- 172.21.15.43:0/753495933 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f69cc06b4d0 0x7f69cc06d990 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.607+0000 7f69eca80640 1 --2- 172.21.15.43:0/753495933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69e0036860 0x7f69e00dfbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.607+0000 7f69eca80640 1 --2- 172.21.15.43:0/753495933 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69d80051c0 0x7f69e004ae40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.607+0000 7f69eca80640 1 -- 172.21.15.43:0/753495933 >> 172.21.15.43:0/753495933 conn(0x7f69e0009fb0 msgr2=0x7f69e004dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:19.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.607+0000 7f69eca80640 1 -- 172.21.15.43:0/753495933 shutdown_connections 2024-04-25T08:54:19.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.607+0000 7f69eca80640 1 -- 172.21.15.43:0/753495933 wait complete. 2024-04-25T08:54:19.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.609+0000 7f45be8b3640 1 -- 172.21.15.43:0/1345348057 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f45a406b520 msgr2=0x7f45a406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.609+0000 7f45be8b3640 1 --2- 172.21.15.43:0/1345348057 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f45a406b520 0x7f45a406d9e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f45b0009a40 tx=0x7f45b000d040 comp rx=0 tx=0).stop 2024-04-25T08:54:19.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.609+0000 7f45be8b3640 1 -- 172.21.15.43:0/1345348057 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45b815b220 msgr2=0x7f45b814b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.609+0000 7f45be8b3640 1 --2- 172.21.15.43:0/1345348057 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45b815b220 0x7f45b814b6f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f45a8031000 tx=0x7f45a80046c0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.609+0000 7f45be8b3640 1 -- 172.21.15.43:0/1345348057 shutdown_connections 2024-04-25T08:54:19.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.609+0000 7f45be8b3640 1 --2- 172.21.15.43:0/1345348057 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45b815b220 0x7f45b814b6f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.609+0000 7f45be8b3640 1 --2- 172.21.15.43:0/1345348057 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f45a406b520 0x7f45a406d9e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.609+0000 7f45be8b3640 1 --2- 172.21.15.43:0/1345348057 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45b80b3da0 0x7f45b8155ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.609+0000 7f45be8b3640 1 --2- 172.21.15.43:0/1345348057 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45b806b620 0x7f45b815a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.609+0000 7f45be8b3640 1 -- 172.21.15.43:0/1345348057 >> 172.21.15.43:0/1345348057 conn(0x7f45b806ab30 msgr2=0x7f45b80ceaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:19.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.610+0000 7f45be8b3640 1 -- 172.21.15.43:0/1345348057 shutdown_connections 2024-04-25T08:54:19.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.610+0000 7f45be8b3640 1 -- 172.21.15.43:0/1345348057 wait complete. 2024-04-25T08:54:19.611 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2024-04-25T08:54:19.611 DEBUG:teuthology.parallel:result is None 2024-04-25T08:54:19.614 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.3 2024-04-25T08:54:19.614 DEBUG:teuthology.parallel:result is None 2024-04-25T08:54:19.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.628+0000 7f2f37ab3640 1 -- 172.21.15.43:0/422241290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2f300aecb0 con 0x7f2f300ae730 2024-04-25T08:54:19.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.628+0000 7f2f267fc640 1 -- 172.21.15.43:0/422241290 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2f28079e90 con 0x7f2f300ae730 2024-04-25T08:54:19.618 INFO:teuthology.orchestra.run.smithi043.stdout:55834574854 2024-04-25T08:54:19.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.631+0000 7f2f37ab3640 1 -- 172.21.15.43:0/422241290 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2f1406b4d0 msgr2=0x7f2f1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.631+0000 7f2f37ab3640 1 --2- 172.21.15.43:0/422241290 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2f1406b4d0 0x7f2f1406d990 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2f3014bf40 tx=0x7f2f2c007840 comp rx=0 tx=0).stop 2024-04-25T08:54:19.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.631+0000 7f2f37ab3640 1 -- 172.21.15.43:0/422241290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f300ae730 msgr2=0x7f2f30150660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.631+0000 7f2f37ab3640 1 --2- 172.21.15.43:0/422241290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f300ae730 0x7f2f30150660 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2f28033040 tx=0x7f2f28004a40 comp rx=0 tx=0).stop 2024-04-25T08:54:19.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.631+0000 7f2f37ab3640 1 -- 172.21.15.43:0/422241290 shutdown_connections 2024-04-25T08:54:19.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.631+0000 7f2f37ab3640 1 --2- 172.21.15.43:0/422241290 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2f1406b4d0 0x7f2f1406d990 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.631+0000 7f2f37ab3640 1 --2- 172.21.15.43:0/422241290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f3015b220 0x7f2f30159210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.631+0000 7f2f37ab3640 1 --2- 172.21.15.43:0/422241290 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2f300b3da0 0x7f2f30158cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.631+0000 7f2f37ab3640 1 --2- 172.21.15.43:0/422241290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f300ae730 0x7f2f30150660 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.631+0000 7f2f37ab3640 1 -- 172.21.15.43:0/422241290 >> 172.21.15.43:0/422241290 conn(0x7f2f3006ab30 msgr2=0x7f2f30152b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:19.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.631+0000 7f2f37ab3640 1 -- 172.21.15.43:0/422241290 shutdown_connections 2024-04-25T08:54:19.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.631+0000 7f2f37ab3640 1 -- 172.21.15.43:0/422241290 wait complete. 2024-04-25T08:54:19.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.640+0000 7f0891aa7640 1 -- 172.21.15.43:0/1936221786 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f088c074e80 con 0x7f088c0cc300 2024-04-25T08:54:19.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.640+0000 7f08897fa640 1 -- 172.21.15.43:0/1936221786 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f087c058c30 con 0x7f088c0cc300 2024-04-25T08:54:19.632 INFO:teuthology.orchestra.run.smithi043.stdout:55834574856 2024-04-25T08:54:19.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.640+0000 7fbc5f593640 1 -- 172.21.15.43:0/1688174207 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fbc58074e80 con 0x7fbc580b4230 2024-04-25T08:54:19.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.641+0000 7fbc4effd640 1 -- 172.21.15.43:0/1688174207 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbc4807a3a0 con 0x7fbc580b4230 2024-04-25T08:54:19.633 INFO:teuthology.orchestra.run.smithi043.stdout:55834574853 2024-04-25T08:54:19.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.643+0000 7f0891aa7640 1 -- 172.21.15.43:0/1936221786 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f087806b5a0 msgr2=0x7f087806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.643+0000 7f0891aa7640 1 --2- 172.21.15.43:0/1936221786 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f087806b5a0 0x7f087806da60 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0880007b10 tx=0x7f0880007a60 comp rx=0 tx=0).stop 2024-04-25T08:54:19.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.643+0000 7f0891aa7640 1 -- 172.21.15.43:0/1936221786 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f088c0cc300 msgr2=0x7f088c14f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.643+0000 7f0891aa7640 1 --2- 172.21.15.43:0/1936221786 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f088c0cc300 0x7f088c14f3c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f087c00d820 tx=0x7f087c00dce0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.643+0000 7f0891aa7640 1 -- 172.21.15.43:0/1936221786 shutdown_connections 2024-04-25T08:54:19.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.643+0000 7f0891aa7640 1 --2- 172.21.15.43:0/1936221786 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f087806b5a0 0x7f087806da60 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.643+0000 7f0891aa7640 1 --2- 172.21.15.43:0/1936221786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f088c0cc6e0 0x7f088c149140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.643+0000 7f0891aa7640 1 --2- 172.21.15.43:0/1936221786 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f088c0cc300 0x7f088c14f3c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.643+0000 7f0891aa7640 1 --2- 172.21.15.43:0/1936221786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f088c0b4650 0x7f088c146690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.643+0000 7f0891aa7640 1 -- 172.21.15.43:0/1936221786 >> 172.21.15.43:0/1936221786 conn(0x7f088c0b1fa0 msgr2=0x7f088c0b1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:19.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.643+0000 7f0891aa7640 1 -- 172.21.15.43:0/1936221786 shutdown_connections 2024-04-25T08:54:19.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.643+0000 7f0891aa7640 1 -- 172.21.15.43:0/1936221786 wait complete. 2024-04-25T08:54:19.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.644+0000 7fbc5f593640 1 -- 172.21.15.43:0/1688174207 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbc3c06b550 msgr2=0x7fbc3c06da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.644+0000 7fbc5f593640 1 --2- 172.21.15.43:0/1688174207 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbc3c06b550 0x7fbc3c06da10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbc54000f60 tx=0x7fbc5400d040 comp rx=0 tx=0).stop 2024-04-25T08:54:19.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.644+0000 7fbc5f593640 1 -- 172.21.15.43:0/1688174207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc580b4230 msgr2=0x7fbc580cd7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.644+0000 7fbc5f593640 1 --2- 172.21.15.43:0/1688174207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc580b4230 0x7fbc580cd7e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbc48031000 tx=0x7fbc480046c0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.644+0000 7fbc5f593640 1 -- 172.21.15.43:0/1688174207 shutdown_connections 2024-04-25T08:54:19.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.644+0000 7fbc5f593640 1 --2- 172.21.15.43:0/1688174207 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbc3c06b550 0x7fbc3c06da10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.644+0000 7fbc5f593640 1 --2- 172.21.15.43:0/1688174207 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc5815a100 0x7fbc580cdd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.644+0000 7fbc5f593640 1 --2- 172.21.15.43:0/1688174207 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc580b4230 0x7fbc580cd7e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.644+0000 7fbc5f593640 1 --2- 172.21.15.43:0/1688174207 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc58070250 0x7fbc5814c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.644+0000 7fbc5f593640 1 -- 172.21.15.43:0/1688174207 >> 172.21.15.43:0/1688174207 conn(0x7fbc5806ad10 msgr2=0x7fbc5806dc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:19.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.644+0000 7fbc5f593640 1 -- 172.21.15.43:0/1688174207 shutdown_connections 2024-04-25T08:54:19.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.644+0000 7fbc5f593640 1 -- 172.21.15.43:0/1688174207 wait complete. 2024-04-25T08:54:19.637 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2024-04-25T08:54:19.638 DEBUG:teuthology.parallel:result is None 2024-04-25T08:54:19.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.648+0000 7f66201b0640 1 -- 172.21.15.43:0/2880382817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f66100057d0 con 0x7f66100e3150 2024-04-25T08:54:19.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.648+0000 7f660e7fc640 1 -- 172.21.15.43:0/2880382817 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6618108af0 con 0x7f66100e3150 2024-04-25T08:54:19.639 INFO:teuthology.orchestra.run.smithi043.stdout:55834574854 2024-04-25T08:54:19.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.651+0000 7f66201b0640 1 -- 172.21.15.43:0/2880382817 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f65fc06b4d0 msgr2=0x7f65fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.651+0000 7f66201b0640 1 --2- 172.21.15.43:0/2880382817 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f65fc06b4d0 0x7f65fc06d990 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6614000f60 tx=0x7f6614009040 comp rx=0 tx=0).stop 2024-04-25T08:54:19.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.651+0000 7f66201b0640 1 -- 172.21.15.43:0/2880382817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66100e3150 msgr2=0x7f66100e5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.651+0000 7f66201b0640 1 --2- 172.21.15.43:0/2880382817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66100e3150 0x7f66100e5e50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6618069c60 tx=0x7f6618069680 comp rx=0 tx=0).stop 2024-04-25T08:54:19.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.651+0000 7f66201b0640 1 -- 172.21.15.43:0/2880382817 shutdown_connections 2024-04-25T08:54:19.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.651+0000 7f66201b0640 1 --2- 172.21.15.43:0/2880382817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f66100e3150 0x7f66100e5e50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.651+0000 7f66201b0640 1 --2- 172.21.15.43:0/2880382817 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f661004fbd0 0x7f66100d96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.651+0000 7f66201b0640 1 --2- 172.21.15.43:0/2880382817 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f65fc06b4d0 0x7f65fc06d990 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.651+0000 7f66201b0640 1 --2- 172.21.15.43:0/2880382817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f661003aa00 0x7f66100d9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.651+0000 7f66201b0640 1 -- 172.21.15.43:0/2880382817 >> 172.21.15.43:0/2880382817 conn(0x7f661000a3f0 msgr2=0x7f6610049140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:19.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.651+0000 7f66201b0640 1 -- 172.21.15.43:0/2880382817 shutdown_connections 2024-04-25T08:54:19.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.651+0000 7f66201b0640 1 -- 172.21.15.43:0/2880382817 wait complete. 2024-04-25T08:54:19.646 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.5 2024-04-25T08:54:19.647 DEBUG:teuthology.parallel:result is None 2024-04-25T08:54:19.648 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.7 2024-04-25T08:54:19.648 DEBUG:teuthology.parallel:result is None 2024-04-25T08:54:19.652 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2024-04-25T08:54:19.652 DEBUG:teuthology.parallel:result is None 2024-04-25T08:54:19.652 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T08:54:19.652 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:54:19.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.803+0000 7f1969af7640 1 Processor -- start 2024-04-25T08:54:19.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.803+0000 7f1969af7640 1 -- start start 2024-04-25T08:54:19.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.803+0000 7f1969af7640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f196415dd00 0x7f19640cc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.804+0000 7f1969af7640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f19640c5f70 0x7f19640cda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.804+0000 7f1969af7640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19640c4ed0 con 0x7f19640c6400 2024-04-25T08:54:19.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.804+0000 7f1969af7640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19640c51b0 con 0x7f196415dd00 2024-04-25T08:54:19.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.804+0000 7f1969af7640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19640c5490 con 0x7f19640c5f70 2024-04-25T08:54:19.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.804+0000 7f19637fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f196415dd00 0x7f19640cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.804+0000 7f1963fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f19640c6400 0x7f196415e0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:38178/0 (socket says 172.21.15.43:38178) 2024-04-25T08:54:19.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.804+0000 7f1963fff640 1 -- 172.21.15.43:0/1110208907 learned_addr learned my addr 172.21.15.43:0/1110208907 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:19.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.804+0000 7f19637fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f196415dd00 0x7f19640cc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55886/0 (socket says 172.21.15.43:55886) 2024-04-25T08:54:19.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.804+0000 7f1962ffd640 1 --2- 172.21.15.43:0/1110208907 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f19640c5f70 0x7f19640cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.804+0000 7f19637fe640 1 -- 172.21.15.43:0/1110208907 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f19640c5f70 msgr2=0x7f19640cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:19.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.804+0000 7f19637fe640 1 --2- 172.21.15.43:0/1110208907 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f19640c5f70 0x7f19640cda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.804+0000 7f19637fe640 1 -- 172.21.15.43:0/1110208907 >> v1:172.21.15.43:6789/0 conn(0x7f19640c6400 legacy=0x7f196415e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.804+0000 7f19627fc640 1 -- 172.21.15.43:0/1110208907 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2269310749 0 0) 0x7f19640c4ed0 con 0x7f19640c6400 2024-04-25T08:54:19.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.804+0000 7f19637fe640 1 -- 172.21.15.43:0/1110208907 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19640c5770 con 0x7f196415dd00 2024-04-25T08:54:19.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.804+0000 7f1962ffd640 1 --2- 172.21.15.43:0/1110208907 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f19640c5f70 0x7f19640cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:19.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.805+0000 7f19637fe640 1 --2- 172.21.15.43:0/1110208907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f196415dd00 0x7f19640cc270 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f19540097a0 tx=0x7f195402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cd022c355f47638 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:19.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.805+0000 7f19627fc640 1 -- 172.21.15.43:0/1110208907 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1954037070 con 0x7f196415dd00 2024-04-25T08:54:19.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.805+0000 7f19627fc640 1 -- 172.21.15.43:0/1110208907 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195402ae50 con 0x7f196415dd00 2024-04-25T08:54:19.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.805+0000 7f19627fc640 1 -- 172.21.15.43:0/1110208907 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1954032660 con 0x7f196415dd00 2024-04-25T08:54:19.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.805+0000 7f1969af7640 1 -- 172.21.15.43:0/1110208907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f196415dd00 msgr2=0x7f19640cc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.805+0000 7f1969af7640 1 --2- 172.21.15.43:0/1110208907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f196415dd00 0x7f19640cc270 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f19540097a0 tx=0x7f195402a100 comp rx=0 tx=0).stop 2024-04-25T08:54:19.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.805+0000 7f1969af7640 1 -- 172.21.15.43:0/1110208907 shutdown_connections 2024-04-25T08:54:19.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.805+0000 7f1969af7640 1 --2- 172.21.15.43:0/1110208907 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f19640c5f70 0x7f19640cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.805+0000 7f1969af7640 1 --2- 172.21.15.43:0/1110208907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f196415dd00 0x7f19640cc270 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.805+0000 7f1969af7640 1 -- 172.21.15.43:0/1110208907 >> 172.21.15.43:0/1110208907 conn(0x7f196406ab30 msgr2=0x7f196406af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:19.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.805+0000 7f1969af7640 1 -- 172.21.15.43:0/1110208907 shutdown_connections 2024-04-25T08:54:19.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.805+0000 7f1969af7640 1 -- 172.21.15.43:0/1110208907 wait complete. 2024-04-25T08:54:19.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.807+0000 7f1969af7640 1 Processor -- start 2024-04-25T08:54:19.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.807+0000 7f1969af7640 1 -- start start 2024-04-25T08:54:19.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.807+0000 7f1969af7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19640c5f70 0x7f19641e5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.807+0000 7f1969af7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19640c6400 0x7f196409c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.808+0000 7f1969af7640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f196415dd00 0x7f1964159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.808+0000 7f1969af7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19640ce120 con 0x7f19640c5f70 2024-04-25T08:54:19.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.808+0000 7f1969af7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19640cdfa0 con 0x7f19640c6400 2024-04-25T08:54:19.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.808+0000 7f1969af7640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19640ce2a0 con 0x7f196415dd00 2024-04-25T08:54:19.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.808+0000 7f19637fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19640c5f70 0x7f19641e5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.808+0000 7f19637fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19640c5f70 0x7f19641e5920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55888/0 (socket says 172.21.15.43:55888) 2024-04-25T08:54:19.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.808+0000 7f19637fe640 1 -- 172.21.15.43:0/3362684809 learned_addr learned my addr 172.21.15.43:0/3362684809 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:19.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.808+0000 7f1963fff640 1 --2- 172.21.15.43:0/3362684809 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f196415dd00 0x7f1964159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.808+0000 7f1962ffd640 1 --2- 172.21.15.43:0/3362684809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19640c6400 0x7f196409c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.808+0000 7f19637fe640 1 -- 172.21.15.43:0/3362684809 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f196415dd00 msgr2=0x7f1964159520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.808+0000 7f19637fe640 1 --2- 172.21.15.43:0/3362684809 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f196415dd00 0x7f1964159520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.808+0000 7f19637fe640 1 -- 172.21.15.43:0/3362684809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19640c6400 msgr2=0x7f196409c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.808+0000 7f19637fe640 1 --2- 172.21.15.43:0/3362684809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19640c6400 0x7f196409c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.808+0000 7f19637fe640 1 -- 172.21.15.43:0/3362684809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196409ce40 con 0x7f19640c5f70 2024-04-25T08:54:19.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.808+0000 7f19637fe640 1 --2- 172.21.15.43:0/3362684809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19640c5f70 0x7f19641e5920 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1954035040 tx=0x7f19540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:19.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.808+0000 7f194bfff640 1 -- 172.21.15.43:0/3362684809 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1954041070 con 0x7f19640c5f70 2024-04-25T08:54:19.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.808+0000 7f194bfff640 1 -- 172.21.15.43:0/3362684809 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195403c670 con 0x7f19640c5f70 2024-04-25T08:54:19.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.808+0000 7f194bfff640 1 -- 172.21.15.43:0/3362684809 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1954037040 con 0x7f19640c5f70 2024-04-25T08:54:19.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.808+0000 7f1969af7640 1 -- 172.21.15.43:0/3362684809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f196409cb60 con 0x7f19640c5f70 2024-04-25T08:54:19.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.808+0000 7f1969af7640 1 -- 172.21.15.43:0/3362684809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19641e5120 con 0x7f19640c5f70 2024-04-25T08:54:19.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.809+0000 7f1969af7640 1 -- 172.21.15.43:0/3362684809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1964074ef0 con 0x7f19640c5f70 2024-04-25T08:54:19.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.810+0000 7f194bfff640 1 -- 172.21.15.43:0/3362684809 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f195403a070 con 0x7f19640c5f70 2024-04-25T08:54:19.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.810+0000 7f194bfff640 1 --2- 172.21.15.43:0/3362684809 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f195006b4d0 0x7f195006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:19.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.810+0000 7f194bfff640 1 -- 172.21.15.43:0/3362684809 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f19540aa610 con 0x7f19640c5f70 2024-04-25T08:54:19.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.810+0000 7f1962ffd640 1 --2- 172.21.15.43:0/3362684809 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f195006b4d0 0x7f195006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:19.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.811+0000 7f1962ffd640 1 --2- 172.21.15.43:0/3362684809 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f195006b4d0 0x7f195006d990 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f196409c110 tx=0x7f1958008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:19.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.814+0000 7f194bfff640 1 -- 172.21.15.43:0/3362684809 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f196409ce40 con 0x7f19640c5f70 2024-04-25T08:54:19.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.991+0000 7f1969af7640 1 -- 172.21.15.43:0/3362684809 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f19640cad30 con 0x7f195006b4d0 2024-04-25T08:54:19.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:19.997+0000 7f194bfff640 1 -- 172.21.15.43:0/3362684809 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251836 (secure 0 0 0) 0x7f19640cad30 con 0x7f195006b4d0 2024-04-25T08:54:19.990 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:54:19.991 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-25T08:54:19.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.000+0000 7f1969af7640 1 -- 172.21.15.43:0/3362684809 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f195006b4d0 msgr2=0x7f195006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.000+0000 7f1969af7640 1 --2- 172.21.15.43:0/3362684809 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f195006b4d0 0x7f195006d990 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f196409c110 tx=0x7f1958008040 comp rx=0 tx=0).stop 2024-04-25T08:54:19.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.000+0000 7f1969af7640 1 -- 172.21.15.43:0/3362684809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19640c5f70 msgr2=0x7f19641e5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:19.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.000+0000 7f1969af7640 1 --2- 172.21.15.43:0/3362684809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19640c5f70 0x7f19641e5920 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1954035040 tx=0x7f19540046c0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.000+0000 7f1969af7640 1 -- 172.21.15.43:0/3362684809 shutdown_connections 2024-04-25T08:54:19.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.000+0000 7f1969af7640 1 --2- 172.21.15.43:0/3362684809 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f196415dd00 0x7f1964159520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.000+0000 7f1969af7640 1 --2- 172.21.15.43:0/3362684809 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f195006b4d0 0x7f195006d990 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.000+0000 7f1969af7640 1 --2- 172.21.15.43:0/3362684809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19640c6400 0x7f196409c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.000+0000 7f1969af7640 1 --2- 172.21.15.43:0/3362684809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19640c5f70 0x7f19641e5920 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:19.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.000+0000 7f1969af7640 1 -- 172.21.15.43:0/3362684809 >> 172.21.15.43:0/3362684809 conn(0x7f196406ab30 msgr2=0x7f196415cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:19.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.000+0000 7f1969af7640 1 -- 172.21.15.43:0/3362684809 shutdown_connections 2024-04-25T08:54:19.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.000+0000 7f1969af7640 1 -- 172.21.15.43:0/3362684809 wait complete. 2024-04-25T08:54:20.001 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-25T08:54:19.696847+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235656,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739064,"statfs":{"total":773094113280,"available":772852801536,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001822"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.003073+0000","last_change":"2024-04-25T08:54:12.612549+0000","last_active":"2024-04-25T08:54:14.003073+0000","last_peered":"2024-04-25T08:54:14.003073+0000","last_clean":"2024-04-25T08:54:14.003073+0000","last_became_active":"2024-04-25T08:54:12.612294+0000","last_became_peered":"2024-04-25T08:54:12.612294+0000","last_unstale":"2024-04-25T08:54:14.003073+0000","last_undegraded":"2024-04-25T08:54:14.003073+0000","last_fullsized":"2024-04-25T08:54:14.003073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:08.402155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.599873+0000","last_change":"2024-04-25T08:54:12.605023+0000","last_active":"2024-04-25T08:54:13.599873+0000","last_peered":"2024-04-25T08:54:13.599873+0000","last_clean":"2024-04-25T08:54:13.599873+0000","last_became_active":"2024-04-25T08:54:12.604703+0000","last_became_peered":"2024-04-25T08:54:12.604703+0000","last_unstale":"2024-04-25T08:54:13.599873+0000","last_undegraded":"2024-04-25T08:54:13.599873+0000","last_fullsized":"2024-04-25T08:54:13.599873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:57.230096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002569+0000","last_change":"2024-04-25T08:54:12.611793+0000","last_active":"2024-04-25T08:54:14.002569+0000","last_peered":"2024-04-25T08:54:14.002569+0000","last_clean":"2024-04-25T08:54:14.002569+0000","last_became_active":"2024-04-25T08:54:12.611530+0000","last_became_peered":"2024-04-25T08:54:12.611530+0000","last_unstale":"2024-04-25T08:54:14.002569+0000","last_undegraded":"2024-04-25T08:54:14.002569+0000","last_fullsized":"2024-04-25T08:54:14.002569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:02.123220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601627+0000","last_change":"2024-04-25T08:54:12.600825+0000","last_active":"2024-04-25T08:54:13.601627+0000","last_peered":"2024-04-25T08:54:13.601627+0000","last_clean":"2024-04-25T08:54:13.601627+0000","last_became_active":"2024-04-25T08:54:12.600531+0000","last_became_peered":"2024-04-25T08:54:12.600531+0000","last_unstale":"2024-04-25T08:54:13.601627+0000","last_undegraded":"2024-04-25T08:54:13.601627+0000","last_fullsized":"2024-04-25T08:54:13.601627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:03.021953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002537+0000","last_change":"2024-04-25T08:54:12.610934+0000","last_active":"2024-04-25T08:54:14.002537+0000","last_peered":"2024-04-25T08:54:14.002537+0000","last_clean":"2024-04-25T08:54:14.002537+0000","last_became_active":"2024-04-25T08:54:12.610457+0000","last_became_peered":"2024-04-25T08:54:12.610457+0000","last_unstale":"2024-04-25T08:54:14.002537+0000","last_undegraded":"2024-04-25T08:54:14.002537+0000","last_fullsized":"2024-04-25T08:54:14.002537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:54.612512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601664+0000","last_change":"2024-04-25T08:54:12.612007+0000","last_active":"2024-04-25T08:54:13.601664+0000","last_peered":"2024-04-25T08:54:13.601664+0000","last_clean":"2024-04-25T08:54:13.601664+0000","last_became_active":"2024-04-25T08:54:12.611744+0000","last_became_peered":"2024-04-25T08:54:12.611744+0000","last_unstale":"2024-04-25T08:54:13.601664+0000","last_undegraded":"2024-04-25T08:54:13.601664+0000","last_fullsized":"2024-04-25T08:54:13.601664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:06.222677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002411+0000","last_change":"2024-04-25T08:54:12.609790+0000","last_active":"2024-04-25T08:54:14.002411+0000","last_peered":"2024-04-25T08:54:14.002411+0000","last_clean":"2024-04-25T08:54:14.002411+0000","last_became_active":"2024-04-25T08:54:12.609504+0000","last_became_peered":"2024-04-25T08:54:12.609504+0000","last_unstale":"2024-04-25T08:54:14.002411+0000","last_undegraded":"2024-04-25T08:54:14.002411+0000","last_fullsized":"2024-04-25T08:54:14.002411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:17.430796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.001963+0000","last_change":"2024-04-25T08:54:12.603756+0000","last_active":"2024-04-25T08:54:14.001963+0000","last_peered":"2024-04-25T08:54:14.001963+0000","last_clean":"2024-04-25T08:54:14.001963+0000","last_became_active":"2024-04-25T08:54:12.602966+0000","last_became_peered":"2024-04-25T08:54:12.602966+0000","last_unstale":"2024-04-25T08:54:14.001963+0000","last_undegraded":"2024-04-25T08:54:14.001963+0000","last_fullsized":"2024-04-25T08:54:14.001963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:05.099361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.003077+0000","last_change":"2024-04-25T08:54:12.617394+0000","last_active":"2024-04-25T08:54:14.003077+0000","last_peered":"2024-04-25T08:54:14.003077+0000","last_clean":"2024-04-25T08:54:14.003077+0000","last_became_active":"2024-04-25T08:54:12.616960+0000","last_became_peered":"2024-04-25T08:54:12.616960+0000","last_unstale":"2024-04-25T08:54:14.003077+0000","last_undegraded":"2024-04-25T08:54:14.003077+0000","last_fullsized":"2024-04-25T08:54:14.003077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:31.051483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.003267+0000","last_change":"2024-04-25T08:54:12.607768+0000","last_active":"2024-04-25T08:54:14.003267+0000","last_peered":"2024-04-25T08:54:14.003267+0000","last_clean":"2024-04-25T08:54:14.003267+0000","last_became_active":"2024-04-25T08:54:12.607503+0000","last_became_peered":"2024-04-25T08:54:12.607503+0000","last_unstale":"2024-04-25T08:54:14.003267+0000","last_undegraded":"2024-04-25T08:54:14.003267+0000","last_fullsized":"2024-04-25T08:54:14.003267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:57.230096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002988+0000","last_change":"2024-04-25T08:54:12.607264+0000","last_active":"2024-04-25T08:54:14.002988+0000","last_peered":"2024-04-25T08:54:14.002988+0000","last_clean":"2024-04-25T08:54:14.002988+0000","last_became_active":"2024-04-25T08:54:12.606961+0000","last_became_peered":"2024-04-25T08:54:12.606961+0000","last_unstale":"2024-04-25T08:54:14.002988+0000","last_undegraded":"2024-04-25T08:54:14.002988+0000","last_fullsized":"2024-04-25T08:54:14.002988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:45.793417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601252+0000","last_change":"2024-04-25T08:54:12.604064+0000","last_active":"2024-04-25T08:54:13.601252+0000","last_peered":"2024-04-25T08:54:13.601252+0000","last_clean":"2024-04-25T08:54:13.601252+0000","last_became_active":"2024-04-25T08:54:12.603807+0000","last_became_peered":"2024-04-25T08:54:12.603807+0000","last_unstale":"2024-04-25T08:54:13.601252+0000","last_undegraded":"2024-04-25T08:54:13.601252+0000","last_fullsized":"2024-04-25T08:54:13.601252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:38.245982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.003119+0000","last_change":"2024-04-25T08:54:12.607207+0000","last_active":"2024-04-25T08:54:14.003119+0000","last_peered":"2024-04-25T08:54:14.003119+0000","last_clean":"2024-04-25T08:54:14.003119+0000","last_became_active":"2024-04-25T08:54:12.606934+0000","last_became_peered":"2024-04-25T08:54:12.606934+0000","last_unstale":"2024-04-25T08:54:14.003119+0000","last_undegraded":"2024-04-25T08:54:14.003119+0000","last_fullsized":"2024-04-25T08:54:14.003119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:02.123220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.001959+0000","last_change":"2024-04-25T08:54:12.596313+0000","last_active":"2024-04-25T08:54:14.001959+0000","last_peered":"2024-04-25T08:54:14.001959+0000","last_clean":"2024-04-25T08:54:14.001959+0000","last_became_active":"2024-04-25T08:54:12.596005+0000","last_became_peered":"2024-04-25T08:54:12.596005+0000","last_unstale":"2024-04-25T08:54:14.001959+0000","last_undegraded":"2024-04-25T08:54:14.001959+0000","last_fullsized":"2024-04-25T08:54:14.001959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:28.227965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.599534+0000","last_change":"2024-04-25T08:54:12.610230+0000","last_active":"2024-04-25T08:54:13.599534+0000","last_peered":"2024-04-25T08:54:13.599534+0000","last_clean":"2024-04-25T08:54:13.599534+0000","last_became_active":"2024-04-25T08:54:12.609932+0000","last_became_peered":"2024-04-25T08:54:12.609932+0000","last_unstale":"2024-04-25T08:54:13.599534+0000","last_undegraded":"2024-04-25T08:54:13.599534+0000","last_fullsized":"2024-04-25T08:54:13.599534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:54.538580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601103+0000","last_change":"2024-04-25T08:54:12.600584+0000","last_active":"2024-04-25T08:54:13.601103+0000","last_peered":"2024-04-25T08:54:13.601103+0000","last_clean":"2024-04-25T08:54:13.601103+0000","last_became_active":"2024-04-25T08:54:12.598989+0000","last_became_peered":"2024-04-25T08:54:12.598989+0000","last_unstale":"2024-04-25T08:54:13.601103+0000","last_undegraded":"2024-04-25T08:54:13.601103+0000","last_fullsized":"2024-04-25T08:54:13.601103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:45.793417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601517+0000","last_change":"2024-04-25T08:54:12.607275+0000","last_active":"2024-04-25T08:54:13.601517+0000","last_peered":"2024-04-25T08:54:13.601517+0000","last_clean":"2024-04-25T08:54:13.601517+0000","last_became_active":"2024-04-25T08:54:12.606990+0000","last_became_peered":"2024-04-25T08:54:12.606990+0000","last_unstale":"2024-04-25T08:54:13.601517+0000","last_undegraded":"2024-04-25T08:54:13.601517+0000","last_fullsized":"2024-04-25T08:54:13.601517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:18.500422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002292+0000","last_change":"2024-04-25T08:54:12.607413+0000","last_active":"2024-04-25T08:54:14.002292+0000","last_peered":"2024-04-25T08:54:14.002292+0000","last_clean":"2024-04-25T08:54:14.002292+0000","last_became_active":"2024-04-25T08:54:12.607135+0000","last_became_peered":"2024-04-25T08:54:12.607135+0000","last_unstale":"2024-04-25T08:54:14.002292+0000","last_undegraded":"2024-04-25T08:54:14.002292+0000","last_fullsized":"2024-04-25T08:54:14.002292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:53.056423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002801+0000","last_change":"2024-04-25T08:54:12.607734+0000","last_active":"2024-04-25T08:54:14.002801+0000","last_peered":"2024-04-25T08:54:14.002801+0000","last_clean":"2024-04-25T08:54:14.002801+0000","last_became_active":"2024-04-25T08:54:12.607462+0000","last_became_peered":"2024-04-25T08:54:12.607462+0000","last_unstale":"2024-04-25T08:54:14.002801+0000","last_undegraded":"2024-04-25T08:54:14.002801+0000","last_fullsized":"2024-04-25T08:54:14.002801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:06.750914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002746+0000","last_change":"2024-04-25T08:54:12.609395+0000","last_active":"2024-04-25T08:54:14.002746+0000","last_peered":"2024-04-25T08:54:14.002746+0000","last_clean":"2024-04-25T08:54:14.002746+0000","last_became_active":"2024-04-25T08:54:12.609094+0000","last_became_peered":"2024-04-25T08:54:12.609094+0000","last_unstale":"2024-04-25T08:54:14.002746+0000","last_undegraded":"2024-04-25T08:54:14.002746+0000","last_fullsized":"2024-04-25T08:54:14.002746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:02.123220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002547+0000","last_change":"2024-04-25T08:54:12.605364+0000","last_active":"2024-04-25T08:54:14.002547+0000","last_peered":"2024-04-25T08:54:14.002547+0000","last_clean":"2024-04-25T08:54:14.002547+0000","last_became_active":"2024-04-25T08:54:12.605084+0000","last_became_peered":"2024-04-25T08:54:12.605084+0000","last_unstale":"2024-04-25T08:54:14.002547+0000","last_undegraded":"2024-04-25T08:54:14.002547+0000","last_fullsized":"2024-04-25T08:54:14.002547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:20.570657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601962+0000","last_change":"2024-04-25T08:54:12.606453+0000","last_active":"2024-04-25T08:54:13.601962+0000","last_peered":"2024-04-25T08:54:13.601962+0000","last_clean":"2024-04-25T08:54:13.601962+0000","last_became_active":"2024-04-25T08:54:12.606072+0000","last_became_peered":"2024-04-25T08:54:12.606072+0000","last_unstale":"2024-04-25T08:54:13.601962+0000","last_undegraded":"2024-04-25T08:54:13.601962+0000","last_fullsized":"2024-04-25T08:54:13.601962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:36.638705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002959+0000","last_change":"2024-04-25T08:54:12.615079+0000","last_active":"2024-04-25T08:54:14.002959+0000","last_peered":"2024-04-25T08:54:14.002959+0000","last_clean":"2024-04-25T08:54:14.002959+0000","last_became_active":"2024-04-25T08:54:12.614779+0000","last_became_peered":"2024-04-25T08:54:12.614779+0000","last_unstale":"2024-04-25T08:54:14.002959+0000","last_undegraded":"2024-04-25T08:54:14.002959+0000","last_fullsized":"2024-04-25T08:54:14.002959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:20.570657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601263+0000","last_change":"2024-04-25T08:54:12.605871+0000","last_active":"2024-04-25T08:54:13.601263+0000","last_peered":"2024-04-25T08:54:13.601263+0000","last_clean":"2024-04-25T08:54:13.601263+0000","last_became_active":"2024-04-25T08:54:12.605598+0000","last_became_peered":"2024-04-25T08:54:12.605598+0000","last_unstale":"2024-04-25T08:54:13.601263+0000","last_undegraded":"2024-04-25T08:54:13.601263+0000","last_fullsized":"2024-04-25T08:54:13.601263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:57.230096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.003397+0000","last_change":"2024-04-25T08:54:12.612253+0000","last_active":"2024-04-25T08:54:14.003397+0000","last_peered":"2024-04-25T08:54:14.003397+0000","last_clean":"2024-04-25T08:54:14.003397+0000","last_became_active":"2024-04-25T08:54:12.611923+0000","last_became_peered":"2024-04-25T08:54:12.611923+0000","last_unstale":"2024-04-25T08:54:14.003397+0000","last_undegraded":"2024-04-25T08:54:14.003397+0000","last_fullsized":"2024-04-25T08:54:14.003397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:06.222677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.003254+0000","last_change":"2024-04-25T08:54:12.608764+0000","last_active":"2024-04-25T08:54:14.003254+0000","last_peered":"2024-04-25T08:54:14.003254+0000","last_clean":"2024-04-25T08:54:14.003254+0000","last_became_active":"2024-04-25T08:54:12.608488+0000","last_became_peered":"2024-04-25T08:54:12.608488+0000","last_unstale":"2024-04-25T08:54:14.003254+0000","last_undegraded":"2024-04-25T08:54:14.003254+0000","last_fullsized":"2024-04-25T08:54:14.003254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:18.500422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.599706+0000","last_change":"2024-04-25T08:54:12.602969+0000","last_active":"2024-04-25T08:54:13.599706+0000","last_peered":"2024-04-25T08:54:13.599706+0000","last_clean":"2024-04-25T08:54:13.599706+0000","last_became_active":"2024-04-25T08:54:12.602464+0000","last_became_peered":"2024-04-25T08:54:12.602464+0000","last_unstale":"2024-04-25T08:54:13.599706+0000","last_undegraded":"2024-04-25T08:54:13.599706+0000","last_fullsized":"2024-04-25T08:54:13.599706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:02.123220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.003276+0000","last_change":"2024-04-25T08:54:12.613884+0000","last_active":"2024-04-25T08:54:14.003276+0000","last_peered":"2024-04-25T08:54:14.003276+0000","last_clean":"2024-04-25T08:54:14.003276+0000","last_became_active":"2024-04-25T08:54:12.613610+0000","last_became_peered":"2024-04-25T08:54:12.613610+0000","last_unstale":"2024-04-25T08:54:14.003276+0000","last_undegraded":"2024-04-25T08:54:14.003276+0000","last_fullsized":"2024-04-25T08:54:14.003276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:18.500422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002701+0000","last_change":"2024-04-25T08:54:12.611194+0000","last_active":"2024-04-25T08:54:14.002701+0000","last_peered":"2024-04-25T08:54:14.002701+0000","last_clean":"2024-04-25T08:54:14.002701+0000","last_became_active":"2024-04-25T08:54:12.610877+0000","last_became_peered":"2024-04-25T08:54:12.610877+0000","last_unstale":"2024-04-25T08:54:14.002701+0000","last_undegraded":"2024-04-25T08:54:14.002701+0000","last_fullsized":"2024-04-25T08:54:14.002701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:45.793417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.602013+0000","last_change":"2024-04-25T08:54:12.600165+0000","last_active":"2024-04-25T08:54:13.602013+0000","last_peered":"2024-04-25T08:54:13.602013+0000","last_clean":"2024-04-25T08:54:13.602013+0000","last_became_active":"2024-04-25T08:54:12.599911+0000","last_became_peered":"2024-04-25T08:54:12.599911+0000","last_unstale":"2024-04-25T08:54:13.602013+0000","last_undegraded":"2024-04-25T08:54:13.602013+0000","last_fullsized":"2024-04-25T08:54:13.602013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:06.750914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002238+0000","last_change":"2024-04-25T08:54:12.604655+0000","last_active":"2024-04-25T08:54:14.002238+0000","last_peered":"2024-04-25T08:54:14.002238+0000","last_clean":"2024-04-25T08:54:14.002238+0000","last_became_active":"2024-04-25T08:54:12.604366+0000","last_became_peered":"2024-04-25T08:54:12.604366+0000","last_unstale":"2024-04-25T08:54:14.002238+0000","last_undegraded":"2024-04-25T08:54:14.002238+0000","last_fullsized":"2024-04-25T08:54:14.002238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:39.105098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002363+0000","last_change":"2024-04-25T08:54:12.615832+0000","last_active":"2024-04-25T08:54:14.002363+0000","last_peered":"2024-04-25T08:54:14.002363+0000","last_clean":"2024-04-25T08:54:14.002363+0000","last_became_active":"2024-04-25T08:54:12.615442+0000","last_became_peered":"2024-04-25T08:54:12.615442+0000","last_unstale":"2024-04-25T08:54:14.002363+0000","last_undegraded":"2024-04-25T08:54:14.002363+0000","last_fullsized":"2024-04-25T08:54:14.002363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:54.538580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601064+0000","last_change":"2024-04-25T08:54:12.598903+0000","last_active":"2024-04-25T08:54:13.601064+0000","last_peered":"2024-04-25T08:54:13.601064+0000","last_clean":"2024-04-25T08:54:13.601064+0000","last_became_active":"2024-04-25T08:54:12.598005+0000","last_became_peered":"2024-04-25T08:54:12.598005+0000","last_unstale":"2024-04-25T08:54:13.601064+0000","last_undegraded":"2024-04-25T08:54:13.601064+0000","last_fullsized":"2024-04-25T08:54:13.601064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:11.183046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.003238+0000","last_change":"2024-04-25T08:54:11.585710+0000","last_active":"2024-04-25T08:54:14.003238+0000","last_peered":"2024-04-25T08:54:14.003238+0000","last_clean":"2024-04-25T08:54:14.003238+0000","last_became_active":"2024-04-25T08:54:11.585090+0000","last_became_peered":"2024-04-25T08:54:11.585090+0000","last_unstale":"2024-04-25T08:54:14.003238+0000","last_undegraded":"2024-04-25T08:54:14.003238+0000","last_fullsized":"2024-04-25T08:54:14.003238+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:10.545906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002798+0000","last_change":"2024-04-25T08:54:12.600641+0000","last_active":"2024-04-25T08:54:14.002798+0000","last_peered":"2024-04-25T08:54:14.002798+0000","last_clean":"2024-04-25T08:54:14.002798+0000","last_became_active":"2024-04-25T08:54:12.598924+0000","last_became_peered":"2024-04-25T08:54:12.598924+0000","last_unstale":"2024-04-25T08:54:14.002798+0000","last_undegraded":"2024-04-25T08:54:14.002798+0000","last_fullsized":"2024-04-25T08:54:14.002798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:18.500422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002906+0000","last_change":"2024-04-25T08:54:11.588038+0000","last_active":"2024-04-25T08:54:14.002906+0000","last_peered":"2024-04-25T08:54:14.002906+0000","last_clean":"2024-04-25T08:54:14.002906+0000","last_became_active":"2024-04-25T08:54:11.587710+0000","last_became_peered":"2024-04-25T08:54:11.587710+0000","last_unstale":"2024-04-25T08:54:14.002906+0000","last_undegraded":"2024-04-25T08:54:14.002906+0000","last_fullsized":"2024-04-25T08:54:14.002906+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:37.241134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.003011+0000","last_change":"2024-04-25T08:54:11.590094+0000","last_active":"2024-04-25T08:54:14.003011+0000","last_peered":"2024-04-25T08:54:14.003011+0000","last_clean":"2024-04-25T08:54:14.003011+0000","last_became_active":"2024-04-25T08:54:11.589720+0000","last_became_peered":"2024-04-25T08:54:11.589720+0000","last_unstale":"2024-04-25T08:54:14.003011+0000","last_undegraded":"2024-04-25T08:54:14.003011+0000","last_fullsized":"2024-04-25T08:54:14.003011+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:54.860395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002127+0000","last_change":"2024-04-25T08:54:12.603523+0000","last_active":"2024-04-25T08:54:14.002127+0000","last_peered":"2024-04-25T08:54:14.002127+0000","last_clean":"2024-04-25T08:54:14.002127+0000","last_became_active":"2024-04-25T08:54:12.602684+0000","last_became_peered":"2024-04-25T08:54:12.602684+0000","last_unstale":"2024-04-25T08:54:14.002127+0000","last_undegraded":"2024-04-25T08:54:14.002127+0000","last_fullsized":"2024-04-25T08:54:14.002127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:54.538580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601357+0000","last_change":"2024-04-25T08:54:11.578444+0000","last_active":"2024-04-25T08:54:13.601357+0000","last_peered":"2024-04-25T08:54:13.601357+0000","last_clean":"2024-04-25T08:54:13.601357+0000","last_became_active":"2024-04-25T08:54:11.578170+0000","last_became_peered":"2024-04-25T08:54:11.578170+0000","last_unstale":"2024-04-25T08:54:13.601357+0000","last_undegraded":"2024-04-25T08:54:13.601357+0000","last_fullsized":"2024-04-25T08:54:13.601357+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:37.241134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.599533+0000","last_change":"2024-04-25T08:54:12.602572+0000","last_active":"2024-04-25T08:54:13.599533+0000","last_peered":"2024-04-25T08:54:13.599533+0000","last_clean":"2024-04-25T08:54:13.599533+0000","last_became_active":"2024-04-25T08:54:12.601587+0000","last_became_peered":"2024-04-25T08:54:12.601587+0000","last_unstale":"2024-04-25T08:54:13.599533+0000","last_undegraded":"2024-04-25T08:54:13.599533+0000","last_fullsized":"2024-04-25T08:54:13.599533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:56.990780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.003202+0000","last_change":"2024-04-25T08:54:11.592937+0000","last_active":"2024-04-25T08:54:14.003202+0000","last_peered":"2024-04-25T08:54:14.003202+0000","last_clean":"2024-04-25T08:54:14.003202+0000","last_became_active":"2024-04-25T08:54:11.592647+0000","last_became_peered":"2024-04-25T08:54:11.592647+0000","last_unstale":"2024-04-25T08:54:14.003202+0000","last_undegraded":"2024-04-25T08:54:14.003202+0000","last_fullsized":"2024-04-25T08:54:14.003202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:45.942159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601977+0000","last_change":"2024-04-25T08:54:12.608078+0000","last_active":"2024-04-25T08:54:13.601977+0000","last_peered":"2024-04-25T08:54:13.601977+0000","last_clean":"2024-04-25T08:54:13.601977+0000","last_became_active":"2024-04-25T08:54:12.605851+0000","last_became_peered":"2024-04-25T08:54:12.605851+0000","last_unstale":"2024-04-25T08:54:13.601977+0000","last_undegraded":"2024-04-25T08:54:13.601977+0000","last_fullsized":"2024-04-25T08:54:13.601977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:54.612512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601849+0000","last_change":"2024-04-25T08:54:11.588593+0000","last_active":"2024-04-25T08:54:13.601849+0000","last_peered":"2024-04-25T08:54:13.601849+0000","last_clean":"2024-04-25T08:54:13.601849+0000","last_became_active":"2024-04-25T08:54:11.588087+0000","last_became_peered":"2024-04-25T08:54:11.588087+0000","last_unstale":"2024-04-25T08:54:13.601849+0000","last_undegraded":"2024-04-25T08:54:13.601849+0000","last_fullsized":"2024-04-25T08:54:13.601849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:10.178198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002526+0000","last_change":"2024-04-25T08:54:12.610492+0000","last_active":"2024-04-25T08:54:14.002526+0000","last_peered":"2024-04-25T08:54:14.002526+0000","last_clean":"2024-04-25T08:54:14.002526+0000","last_became_active":"2024-04-25T08:54:12.610228+0000","last_became_peered":"2024-04-25T08:54:12.610228+0000","last_unstale":"2024-04-25T08:54:14.002526+0000","last_undegraded":"2024-04-25T08:54:14.002526+0000","last_fullsized":"2024-04-25T08:54:14.002526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:56.990780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.599817+0000","last_change":"2024-04-25T08:54:11.585441+0000","last_active":"2024-04-25T08:54:13.599817+0000","last_peered":"2024-04-25T08:54:13.599817+0000","last_clean":"2024-04-25T08:54:13.599817+0000","last_became_active":"2024-04-25T08:54:11.585098+0000","last_became_peered":"2024-04-25T08:54:11.585098+0000","last_unstale":"2024-04-25T08:54:13.599817+0000","last_undegraded":"2024-04-25T08:54:13.599817+0000","last_fullsized":"2024-04-25T08:54:13.599817+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:50.460868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601444+0000","last_change":"2024-04-25T08:54:12.606323+0000","last_active":"2024-04-25T08:54:13.601444+0000","last_peered":"2024-04-25T08:54:13.601444+0000","last_clean":"2024-04-25T08:54:13.601444+0000","last_became_active":"2024-04-25T08:54:12.606063+0000","last_became_peered":"2024-04-25T08:54:12.606063+0000","last_unstale":"2024-04-25T08:54:13.601444+0000","last_undegraded":"2024-04-25T08:54:13.601444+0000","last_fullsized":"2024-04-25T08:54:13.601444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:46.947007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002754+0000","last_change":"2024-04-25T08:54:11.590170+0000","last_active":"2024-04-25T08:54:14.002754+0000","last_peered":"2024-04-25T08:54:14.002754+0000","last_clean":"2024-04-25T08:54:14.002754+0000","last_became_active":"2024-04-25T08:54:11.589869+0000","last_became_peered":"2024-04-25T08:54:11.589869+0000","last_unstale":"2024-04-25T08:54:14.002754+0000","last_undegraded":"2024-04-25T08:54:14.002754+0000","last_fullsized":"2024-04-25T08:54:14.002754+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:10.545906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002430+0000","last_change":"2024-04-25T08:54:12.607302+0000","last_active":"2024-04-25T08:54:14.002430+0000","last_peered":"2024-04-25T08:54:14.002430+0000","last_clean":"2024-04-25T08:54:14.002430+0000","last_became_active":"2024-04-25T08:54:12.607042+0000","last_became_peered":"2024-04-25T08:54:12.607042+0000","last_unstale":"2024-04-25T08:54:14.002430+0000","last_undegraded":"2024-04-25T08:54:14.002430+0000","last_fullsized":"2024-04-25T08:54:14.002430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:36.638705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002793+0000","last_change":"2024-04-25T08:54:11.586629+0000","last_active":"2024-04-25T08:54:14.002793+0000","last_peered":"2024-04-25T08:54:14.002793+0000","last_clean":"2024-04-25T08:54:14.002793+0000","last_became_active":"2024-04-25T08:54:11.586289+0000","last_became_peered":"2024-04-25T08:54:11.586289+0000","last_unstale":"2024-04-25T08:54:14.002793+0000","last_undegraded":"2024-04-25T08:54:14.002793+0000","last_fullsized":"2024-04-25T08:54:14.002793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:12.212662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002117+0000","last_change":"2024-04-25T08:54:12.604702+0000","last_active":"2024-04-25T08:54:14.002117+0000","last_peered":"2024-04-25T08:54:14.002117+0000","last_clean":"2024-04-25T08:54:14.002117+0000","last_became_active":"2024-04-25T08:54:12.604410+0000","last_became_peered":"2024-04-25T08:54:12.604410+0000","last_unstale":"2024-04-25T08:54:14.002117+0000","last_undegraded":"2024-04-25T08:54:14.002117+0000","last_fullsized":"2024-04-25T08:54:14.002117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:05.099361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.602125+0000","last_change":"2024-04-25T08:54:11.583543+0000","last_active":"2024-04-25T08:54:13.602125+0000","last_peered":"2024-04-25T08:54:13.602125+0000","last_clean":"2024-04-25T08:54:13.602125+0000","last_became_active":"2024-04-25T08:54:11.583254+0000","last_became_peered":"2024-04-25T08:54:11.583254+0000","last_unstale":"2024-04-25T08:54:13.602125+0000","last_undegraded":"2024-04-25T08:54:13.602125+0000","last_fullsized":"2024-04-25T08:54:13.602125+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:45.942159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002350+0000","last_change":"2024-04-25T08:54:12.609195+0000","last_active":"2024-04-25T08:54:14.002350+0000","last_peered":"2024-04-25T08:54:14.002350+0000","last_clean":"2024-04-25T08:54:14.002350+0000","last_became_active":"2024-04-25T08:54:12.608943+0000","last_became_peered":"2024-04-25T08:54:12.608943+0000","last_unstale":"2024-04-25T08:54:14.002350+0000","last_undegraded":"2024-04-25T08:54:14.002350+0000","last_fullsized":"2024-04-25T08:54:14.002350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:06.222677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002597+0000","last_change":"2024-04-25T08:54:11.586926+0000","last_active":"2024-04-25T08:54:14.002597+0000","last_peered":"2024-04-25T08:54:14.002597+0000","last_clean":"2024-04-25T08:54:14.002597+0000","last_became_active":"2024-04-25T08:54:11.586600+0000","last_became_peered":"2024-04-25T08:54:11.586600+0000","last_unstale":"2024-04-25T08:54:14.002597+0000","last_undegraded":"2024-04-25T08:54:14.002597+0000","last_fullsized":"2024-04-25T08:54:14.002597+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:10.178198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002584+0000","last_change":"2024-04-25T08:54:12.615811+0000","last_active":"2024-04-25T08:54:14.002584+0000","last_peered":"2024-04-25T08:54:14.002584+0000","last_clean":"2024-04-25T08:54:14.002584+0000","last_became_active":"2024-04-25T08:54:12.615492+0000","last_became_peered":"2024-04-25T08:54:12.615492+0000","last_unstale":"2024-04-25T08:54:14.002584+0000","last_undegraded":"2024-04-25T08:54:14.002584+0000","last_fullsized":"2024-04-25T08:54:14.002584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:56.990780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002293+0000","last_change":"2024-04-25T08:54:11.583593+0000","last_active":"2024-04-25T08:54:14.002293+0000","last_peered":"2024-04-25T08:54:14.002293+0000","last_clean":"2024-04-25T08:54:14.002293+0000","last_became_active":"2024-04-25T08:54:11.582876+0000","last_became_peered":"2024-04-25T08:54:11.582876+0000","last_unstale":"2024-04-25T08:54:14.002293+0000","last_undegraded":"2024-04-25T08:54:14.002293+0000","last_fullsized":"2024-04-25T08:54:14.002293+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:10.545906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002256+0000","last_change":"2024-04-25T08:54:12.609297+0000","last_active":"2024-04-25T08:54:14.002256+0000","last_peered":"2024-04-25T08:54:14.002256+0000","last_clean":"2024-04-25T08:54:14.002256+0000","last_became_active":"2024-04-25T08:54:12.608948+0000","last_became_peered":"2024-04-25T08:54:12.608948+0000","last_unstale":"2024-04-25T08:54:14.002256+0000","last_undegraded":"2024-04-25T08:54:14.002256+0000","last_fullsized":"2024-04-25T08:54:14.002256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:03.021953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601349+0000","last_change":"2024-04-25T08:54:11.586306+0000","last_active":"2024-04-25T08:54:13.601349+0000","last_peered":"2024-04-25T08:54:13.601349+0000","last_clean":"2024-04-25T08:54:13.601349+0000","last_became_active":"2024-04-25T08:54:11.585959+0000","last_became_peered":"2024-04-25T08:54:11.585959+0000","last_unstale":"2024-04-25T08:54:13.601349+0000","last_undegraded":"2024-04-25T08:54:13.601349+0000","last_fullsized":"2024-04-25T08:54:13.601349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:40.673650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002907+0000","last_change":"2024-04-25T08:54:12.609239+0000","last_active":"2024-04-25T08:54:14.002907+0000","last_peered":"2024-04-25T08:54:14.002907+0000","last_clean":"2024-04-25T08:54:14.002907+0000","last_became_active":"2024-04-25T08:54:12.608994+0000","last_became_peered":"2024-04-25T08:54:12.608994+0000","last_unstale":"2024-04-25T08:54:14.002907+0000","last_undegraded":"2024-04-25T08:54:14.002907+0000","last_fullsized":"2024-04-25T08:54:14.002907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:44.251355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002393+0000","last_change":"2024-04-25T08:54:11.589122+0000","last_active":"2024-04-25T08:54:14.002393+0000","last_peered":"2024-04-25T08:54:14.002393+0000","last_clean":"2024-04-25T08:54:14.002393+0000","last_became_active":"2024-04-25T08:54:11.588772+0000","last_became_peered":"2024-04-25T08:54:11.588772+0000","last_unstale":"2024-04-25T08:54:14.002393+0000","last_undegraded":"2024-04-25T08:54:14.002393+0000","last_fullsized":"2024-04-25T08:54:14.002393+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:37.241134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.003230+0000","last_change":"2024-04-25T08:54:12.613985+0000","last_active":"2024-04-25T08:54:14.003230+0000","last_peered":"2024-04-25T08:54:14.003230+0000","last_clean":"2024-04-25T08:54:14.003230+0000","last_became_active":"2024-04-25T08:54:12.613717+0000","last_became_peered":"2024-04-25T08:54:12.613717+0000","last_unstale":"2024-04-25T08:54:14.003230+0000","last_undegraded":"2024-04-25T08:54:14.003230+0000","last_fullsized":"2024-04-25T08:54:14.003230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:28.550328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.599846+0000","last_change":"2024-04-25T08:54:11.586724+0000","last_active":"2024-04-25T08:54:13.599846+0000","last_peered":"2024-04-25T08:54:13.599846+0000","last_clean":"2024-04-25T08:54:13.599846+0000","last_became_active":"2024-04-25T08:54:11.586219+0000","last_became_peered":"2024-04-25T08:54:11.586219+0000","last_unstale":"2024-04-25T08:54:13.599846+0000","last_undegraded":"2024-04-25T08:54:13.599846+0000","last_fullsized":"2024-04-25T08:54:13.599846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:32.370052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601817+0000","last_change":"2024-04-25T08:54:12.600997+0000","last_active":"2024-04-25T08:54:13.601817+0000","last_peered":"2024-04-25T08:54:13.601817+0000","last_clean":"2024-04-25T08:54:13.601817+0000","last_became_active":"2024-04-25T08:54:12.600704+0000","last_became_peered":"2024-04-25T08:54:12.600704+0000","last_unstale":"2024-04-25T08:54:13.601817+0000","last_undegraded":"2024-04-25T08:54:13.601817+0000","last_fullsized":"2024-04-25T08:54:13.601817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:17.430796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.001982+0000","last_change":"2024-04-25T08:54:11.578803+0000","last_active":"2024-04-25T08:54:14.001982+0000","last_peered":"2024-04-25T08:54:14.001982+0000","last_clean":"2024-04-25T08:54:14.001982+0000","last_became_active":"2024-04-25T08:54:11.578421+0000","last_became_peered":"2024-04-25T08:54:11.578421+0000","last_unstale":"2024-04-25T08:54:14.001982+0000","last_undegraded":"2024-04-25T08:54:14.001982+0000","last_fullsized":"2024-04-25T08:54:14.001982+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:54.860395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002707+0000","last_change":"2024-04-25T08:54:12.617092+0000","last_active":"2024-04-25T08:54:14.002707+0000","last_peered":"2024-04-25T08:54:14.002707+0000","last_clean":"2024-04-25T08:54:14.002707+0000","last_became_active":"2024-04-25T08:54:12.616799+0000","last_became_peered":"2024-04-25T08:54:12.616799+0000","last_unstale":"2024-04-25T08:54:14.002707+0000","last_undegraded":"2024-04-25T08:54:14.002707+0000","last_fullsized":"2024-04-25T08:54:14.002707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:21.068533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002257+0000","last_change":"2024-04-25T08:54:00.818044+0000","last_active":"2024-04-25T08:54:14.002257+0000","last_peered":"2024-04-25T08:54:14.002257+0000","last_clean":"2024-04-25T08:54:14.002257+0000","last_became_active":"2024-04-25T08:54:00.817098+0000","last_became_peered":"2024-04-25T08:54:00.817098+0000","last_unstale":"2024-04-25T08:54:14.002257+0000","last_undegraded":"2024-04-25T08:54:14.002257+0000","last_fullsized":"2024-04-25T08:54:14.002257+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_clean_scrub_stamp":"2024-04-25T08:53:59.739677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:48.129513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601017+0000","last_change":"2024-04-25T08:54:11.576740+0000","last_active":"2024-04-25T08:54:13.601017+0000","last_peered":"2024-04-25T08:54:13.601017+0000","last_clean":"2024-04-25T08:54:13.601017+0000","last_became_active":"2024-04-25T08:54:11.576444+0000","last_became_peered":"2024-04-25T08:54:11.576444+0000","last_unstale":"2024-04-25T08:54:13.601017+0000","last_undegraded":"2024-04-25T08:54:13.601017+0000","last_fullsized":"2024-04-25T08:54:13.601017+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:10.545906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.003145+0000","last_change":"2024-04-25T08:54:12.606225+0000","last_active":"2024-04-25T08:54:14.003145+0000","last_peered":"2024-04-25T08:54:14.003145+0000","last_clean":"2024-04-25T08:54:14.003145+0000","last_became_active":"2024-04-25T08:54:12.605897+0000","last_became_peered":"2024-04-25T08:54:12.605897+0000","last_unstale":"2024-04-25T08:54:14.003145+0000","last_undegraded":"2024-04-25T08:54:14.003145+0000","last_fullsized":"2024-04-25T08:54:14.003145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:57.230096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002094+0000","last_change":"2024-04-25T08:54:11.590032+0000","last_active":"2024-04-25T08:54:14.002094+0000","last_peered":"2024-04-25T08:54:14.002094+0000","last_clean":"2024-04-25T08:54:14.002094+0000","last_became_active":"2024-04-25T08:54:11.589701+0000","last_became_peered":"2024-04-25T08:54:11.589701+0000","last_unstale":"2024-04-25T08:54:14.002094+0000","last_undegraded":"2024-04-25T08:54:14.002094+0000","last_fullsized":"2024-04-25T08:54:14.002094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:52.051575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.001941+0000","last_change":"2024-04-25T08:54:12.603873+0000","last_active":"2024-04-25T08:54:14.001941+0000","last_peered":"2024-04-25T08:54:14.001941+0000","last_clean":"2024-04-25T08:54:14.001941+0000","last_became_active":"2024-04-25T08:54:12.599347+0000","last_became_peered":"2024-04-25T08:54:12.599347+0000","last_unstale":"2024-04-25T08:54:14.001941+0000","last_undegraded":"2024-04-25T08:54:14.001941+0000","last_fullsized":"2024-04-25T08:54:14.001941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:14.023628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002327+0000","last_change":"2024-04-25T08:54:11.588495+0000","last_active":"2024-04-25T08:54:14.002327+0000","last_peered":"2024-04-25T08:54:14.002327+0000","last_clean":"2024-04-25T08:54:14.002327+0000","last_became_active":"2024-04-25T08:54:11.588141+0000","last_became_peered":"2024-04-25T08:54:11.588141+0000","last_unstale":"2024-04-25T08:54:14.002327+0000","last_undegraded":"2024-04-25T08:54:14.002327+0000","last_fullsized":"2024-04-25T08:54:14.002327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:10.178198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.599547+0000","last_change":"2024-04-25T08:54:12.610027+0000","last_active":"2024-04-25T08:54:13.599547+0000","last_peered":"2024-04-25T08:54:13.599547+0000","last_clean":"2024-04-25T08:54:13.599547+0000","last_became_active":"2024-04-25T08:54:12.609091+0000","last_became_peered":"2024-04-25T08:54:12.609091+0000","last_unstale":"2024-04-25T08:54:13.599547+0000","last_undegraded":"2024-04-25T08:54:13.599547+0000","last_fullsized":"2024-04-25T08:54:13.599547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:20.570657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.602128+0000","last_change":"2024-04-25T08:54:11.587597+0000","last_active":"2024-04-25T08:54:13.602128+0000","last_peered":"2024-04-25T08:54:13.602128+0000","last_clean":"2024-04-25T08:54:13.602128+0000","last_became_active":"2024-04-25T08:54:11.587078+0000","last_became_peered":"2024-04-25T08:54:11.587078+0000","last_unstale":"2024-04-25T08:54:13.602128+0000","last_undegraded":"2024-04-25T08:54:13.602128+0000","last_fullsized":"2024-04-25T08:54:13.602128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:50.460868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601656+0000","last_change":"2024-04-25T08:54:12.611656+0000","last_active":"2024-04-25T08:54:13.601656+0000","last_peered":"2024-04-25T08:54:13.601656+0000","last_clean":"2024-04-25T08:54:13.601656+0000","last_became_active":"2024-04-25T08:54:12.611359+0000","last_became_peered":"2024-04-25T08:54:12.611359+0000","last_unstale":"2024-04-25T08:54:13.601656+0000","last_undegraded":"2024-04-25T08:54:13.601656+0000","last_fullsized":"2024-04-25T08:54:13.601656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:33.374900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.599718+0000","last_change":"2024-04-25T08:54:11.589751+0000","last_active":"2024-04-25T08:54:13.599718+0000","last_peered":"2024-04-25T08:54:13.599718+0000","last_clean":"2024-04-25T08:54:13.599718+0000","last_became_active":"2024-04-25T08:54:11.589362+0000","last_became_peered":"2024-04-25T08:54:11.589362+0000","last_unstale":"2024-04-25T08:54:13.599718+0000","last_undegraded":"2024-04-25T08:54:13.599718+0000","last_fullsized":"2024-04-25T08:54:13.599718+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:10.178198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.602180+0000","last_change":"2024-04-25T08:54:12.606441+0000","last_active":"2024-04-25T08:54:13.602180+0000","last_peered":"2024-04-25T08:54:13.602180+0000","last_clean":"2024-04-25T08:54:13.602180+0000","last_became_active":"2024-04-25T08:54:12.605425+0000","last_became_peered":"2024-04-25T08:54:12.605425+0000","last_unstale":"2024-04-25T08:54:13.602180+0000","last_undegraded":"2024-04-25T08:54:13.602180+0000","last_fullsized":"2024-04-25T08:54:13.602180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:31.051483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002795+0000","last_change":"2024-04-25T08:54:11.591338+0000","last_active":"2024-04-25T08:54:14.002795+0000","last_peered":"2024-04-25T08:54:14.002795+0000","last_clean":"2024-04-25T08:54:14.002795+0000","last_became_active":"2024-04-25T08:54:11.591074+0000","last_became_peered":"2024-04-25T08:54:11.591074+0000","last_unstale":"2024-04-25T08:54:14.002795+0000","last_undegraded":"2024-04-25T08:54:14.002795+0000","last_fullsized":"2024-04-25T08:54:14.002795+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:32.370052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.599726+0000","last_change":"2024-04-25T08:54:12.602183+0000","last_active":"2024-04-25T08:54:13.599726+0000","last_peered":"2024-04-25T08:54:13.599726+0000","last_clean":"2024-04-25T08:54:13.599726+0000","last_became_active":"2024-04-25T08:54:12.601348+0000","last_became_peered":"2024-04-25T08:54:12.601348+0000","last_unstale":"2024-04-25T08:54:13.599726+0000","last_undegraded":"2024-04-25T08:54:13.599726+0000","last_fullsized":"2024-04-25T08:54:13.599726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:45.793417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.714307+0000","last_change":"2024-04-25T08:54:11.592183+0000","last_active":"2024-04-25T08:54:13.714307+0000","last_peered":"2024-04-25T08:54:13.714307+0000","last_clean":"2024-04-25T08:54:13.714307+0000","last_became_active":"2024-04-25T08:54:11.591795+0000","last_became_peered":"2024-04-25T08:54:11.591795+0000","last_unstale":"2024-04-25T08:54:13.714307+0000","last_undegraded":"2024-04-25T08:54:13.714307+0000","last_fullsized":"2024-04-25T08:54:13.714307+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:38.100250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.003444+0000","last_change":"2024-04-25T08:54:12.607792+0000","last_active":"2024-04-25T08:54:14.003444+0000","last_peered":"2024-04-25T08:54:14.003444+0000","last_clean":"2024-04-25T08:54:14.003444+0000","last_became_active":"2024-04-25T08:54:12.607459+0000","last_became_peered":"2024-04-25T08:54:12.607459+0000","last_unstale":"2024-04-25T08:54:14.003444+0000","last_undegraded":"2024-04-25T08:54:14.003444+0000","last_fullsized":"2024-04-25T08:54:14.003444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:17.430796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.003207+0000","last_change":"2024-04-25T08:54:11.591567+0000","last_active":"2024-04-25T08:54:14.003207+0000","last_peered":"2024-04-25T08:54:14.003207+0000","last_clean":"2024-04-25T08:54:14.003207+0000","last_became_active":"2024-04-25T08:54:11.591178+0000","last_became_peered":"2024-04-25T08:54:11.591178+0000","last_unstale":"2024-04-25T08:54:14.003207+0000","last_undegraded":"2024-04-25T08:54:14.003207+0000","last_fullsized":"2024-04-25T08:54:14.003207+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:50.460868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002719+0000","last_change":"2024-04-25T08:54:12.598577+0000","last_active":"2024-04-25T08:54:14.002719+0000","last_peered":"2024-04-25T08:54:14.002719+0000","last_clean":"2024-04-25T08:54:14.002719+0000","last_became_active":"2024-04-25T08:54:12.597319+0000","last_became_peered":"2024-04-25T08:54:12.597319+0000","last_unstale":"2024-04-25T08:54:14.002719+0000","last_undegraded":"2024-04-25T08:54:14.002719+0000","last_fullsized":"2024-04-25T08:54:14.002719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:57.230096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002090+0000","last_change":"2024-04-25T08:54:11.591476+0000","last_active":"2024-04-25T08:54:14.002090+0000","last_peered":"2024-04-25T08:54:14.002090+0000","last_clean":"2024-04-25T08:54:14.002090+0000","last_became_active":"2024-04-25T08:54:11.591000+0000","last_became_peered":"2024-04-25T08:54:11.591000+0000","last_unstale":"2024-04-25T08:54:14.002090+0000","last_undegraded":"2024-04-25T08:54:14.002090+0000","last_fullsized":"2024-04-25T08:54:14.002090+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:38.100250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601522+0000","last_change":"2024-04-25T08:54:12.602036+0000","last_active":"2024-04-25T08:54:13.601522+0000","last_peered":"2024-04-25T08:54:13.601522+0000","last_clean":"2024-04-25T08:54:13.601522+0000","last_became_active":"2024-04-25T08:54:12.601768+0000","last_became_peered":"2024-04-25T08:54:12.601768+0000","last_unstale":"2024-04-25T08:54:13.601522+0000","last_undegraded":"2024-04-25T08:54:13.601522+0000","last_fullsized":"2024-04-25T08:54:13.601522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:51.465717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.599648+0000","last_change":"2024-04-25T08:54:11.585465+0000","last_active":"2024-04-25T08:54:13.599648+0000","last_peered":"2024-04-25T08:54:13.599648+0000","last_clean":"2024-04-25T08:54:13.599648+0000","last_became_active":"2024-04-25T08:54:11.585184+0000","last_became_peered":"2024-04-25T08:54:11.585184+0000","last_unstale":"2024-04-25T08:54:13.599648+0000","last_undegraded":"2024-04-25T08:54:13.599648+0000","last_fullsized":"2024-04-25T08:54:13.599648+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:37.241134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002288+0000","last_change":"2024-04-25T08:54:12.603295+0000","last_active":"2024-04-25T08:54:14.002288+0000","last_peered":"2024-04-25T08:54:14.002288+0000","last_clean":"2024-04-25T08:54:14.002288+0000","last_became_active":"2024-04-25T08:54:12.602398+0000","last_became_peered":"2024-04-25T08:54:12.602398+0000","last_unstale":"2024-04-25T08:54:14.002288+0000","last_undegraded":"2024-04-25T08:54:14.002288+0000","last_fullsized":"2024-04-25T08:54:14.002288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:56.990780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.599690+0000","last_change":"2024-04-25T08:54:11.588662+0000","last_active":"2024-04-25T08:54:13.599690+0000","last_peered":"2024-04-25T08:54:13.599690+0000","last_clean":"2024-04-25T08:54:13.599690+0000","last_became_active":"2024-04-25T08:54:11.588350+0000","last_became_peered":"2024-04-25T08:54:11.588350+0000","last_unstale":"2024-04-25T08:54:13.599690+0000","last_undegraded":"2024-04-25T08:54:13.599690+0000","last_fullsized":"2024-04-25T08:54:13.599690+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:45.942159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.003164+0000","last_change":"2024-04-25T08:54:12.612845+0000","last_active":"2024-04-25T08:54:14.003164+0000","last_peered":"2024-04-25T08:54:14.003164+0000","last_clean":"2024-04-25T08:54:14.003164+0000","last_became_active":"2024-04-25T08:54:12.612569+0000","last_became_peered":"2024-04-25T08:54:12.612569+0000","last_unstale":"2024-04-25T08:54:14.003164+0000","last_undegraded":"2024-04-25T08:54:14.003164+0000","last_fullsized":"2024-04-25T08:54:14.003164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:45.793417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002442+0000","last_change":"2024-04-25T08:54:11.589407+0000","last_active":"2024-04-25T08:54:14.002442+0000","last_peered":"2024-04-25T08:54:14.002442+0000","last_clean":"2024-04-25T08:54:14.002442+0000","last_became_active":"2024-04-25T08:54:11.588226+0000","last_became_peered":"2024-04-25T08:54:11.588226+0000","last_unstale":"2024-04-25T08:54:14.002442+0000","last_undegraded":"2024-04-25T08:54:14.002442+0000","last_fullsized":"2024-04-25T08:54:14.002442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:42.726856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002789+0000","last_change":"2024-04-25T08:54:12.609741+0000","last_active":"2024-04-25T08:54:14.002789+0000","last_peered":"2024-04-25T08:54:14.002789+0000","last_clean":"2024-04-25T08:54:14.002789+0000","last_became_active":"2024-04-25T08:54:12.609075+0000","last_became_peered":"2024-04-25T08:54:12.609075+0000","last_unstale":"2024-04-25T08:54:14.002789+0000","last_undegraded":"2024-04-25T08:54:14.002789+0000","last_fullsized":"2024-04-25T08:54:14.002789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:20.570657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.602159+0000","last_change":"2024-04-25T08:54:12.603440+0000","last_active":"2024-04-25T08:54:13.602159+0000","last_peered":"2024-04-25T08:54:13.602159+0000","last_clean":"2024-04-25T08:54:13.602159+0000","last_became_active":"2024-04-25T08:54:12.603193+0000","last_became_peered":"2024-04-25T08:54:12.603193+0000","last_unstale":"2024-04-25T08:54:13.602159+0000","last_undegraded":"2024-04-25T08:54:13.602159+0000","last_fullsized":"2024-04-25T08:54:13.602159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:44.251355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002297+0000","last_change":"2024-04-25T08:54:11.585161+0000","last_active":"2024-04-25T08:54:14.002297+0000","last_peered":"2024-04-25T08:54:14.002297+0000","last_clean":"2024-04-25T08:54:14.002297+0000","last_became_active":"2024-04-25T08:54:11.584842+0000","last_became_peered":"2024-04-25T08:54:11.584842+0000","last_unstale":"2024-04-25T08:54:14.002297+0000","last_undegraded":"2024-04-25T08:54:14.002297+0000","last_fullsized":"2024-04-25T08:54:14.002297+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:45.942159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601990+0000","last_change":"2024-04-25T08:54:12.611429+0000","last_active":"2024-04-25T08:54:13.601990+0000","last_peered":"2024-04-25T08:54:13.601990+0000","last_clean":"2024-04-25T08:54:13.601990+0000","last_became_active":"2024-04-25T08:54:12.611170+0000","last_became_peered":"2024-04-25T08:54:12.611170+0000","last_unstale":"2024-04-25T08:54:13.601990+0000","last_undegraded":"2024-04-25T08:54:13.601990+0000","last_fullsized":"2024-04-25T08:54:13.601990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:21.068533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002636+0000","last_change":"2024-04-25T08:54:11.589779+0000","last_active":"2024-04-25T08:54:14.002636+0000","last_peered":"2024-04-25T08:54:14.002636+0000","last_clean":"2024-04-25T08:54:14.002636+0000","last_became_active":"2024-04-25T08:54:11.589502+0000","last_became_peered":"2024-04-25T08:54:11.589502+0000","last_unstale":"2024-04-25T08:54:14.002636+0000","last_undegraded":"2024-04-25T08:54:14.002636+0000","last_fullsized":"2024-04-25T08:54:14.002636+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:50.460868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002428+0000","last_change":"2024-04-25T08:54:12.610745+0000","last_active":"2024-04-25T08:54:14.002428+0000","last_peered":"2024-04-25T08:54:14.002428+0000","last_clean":"2024-04-25T08:54:14.002428+0000","last_became_active":"2024-04-25T08:54:12.610366+0000","last_became_peered":"2024-04-25T08:54:12.610366+0000","last_unstale":"2024-04-25T08:54:14.002428+0000","last_undegraded":"2024-04-25T08:54:14.002428+0000","last_fullsized":"2024-04-25T08:54:14.002428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:54.538580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.003385+0000","last_change":"2024-04-25T08:54:11.586361+0000","last_active":"2024-04-25T08:54:14.003385+0000","last_peered":"2024-04-25T08:54:14.003385+0000","last_clean":"2024-04-25T08:54:14.003385+0000","last_became_active":"2024-04-25T08:54:11.586056+0000","last_became_peered":"2024-04-25T08:54:11.586056+0000","last_unstale":"2024-04-25T08:54:14.003385+0000","last_undegraded":"2024-04-25T08:54:14.003385+0000","last_fullsized":"2024-04-25T08:54:14.003385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:12.212662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.003357+0000","last_change":"2024-04-25T08:54:12.607913+0000","last_active":"2024-04-25T08:54:14.003357+0000","last_peered":"2024-04-25T08:54:14.003357+0000","last_clean":"2024-04-25T08:54:14.003357+0000","last_became_active":"2024-04-25T08:54:12.607623+0000","last_became_peered":"2024-04-25T08:54:12.607623+0000","last_unstale":"2024-04-25T08:54:14.003357+0000","last_undegraded":"2024-04-25T08:54:14.003357+0000","last_fullsized":"2024-04-25T08:54:14.003357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:03.021953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002643+0000","last_change":"2024-04-25T08:54:11.591877+0000","last_active":"2024-04-25T08:54:14.002643+0000","last_peered":"2024-04-25T08:54:14.002643+0000","last_clean":"2024-04-25T08:54:14.002643+0000","last_became_active":"2024-04-25T08:54:11.591562+0000","last_became_peered":"2024-04-25T08:54:11.591562+0000","last_unstale":"2024-04-25T08:54:14.002643+0000","last_undegraded":"2024-04-25T08:54:14.002643+0000","last_fullsized":"2024-04-25T08:54:14.002643+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:54.860395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:54:20.005 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:54:20.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.152+0000 7f7a98d90640 1 Processor -- start 2024-04-25T08:54:20.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.152+0000 7f7a98d90640 1 -- start start 2024-04-25T08:54:20.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.153+0000 7f7a98d90640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f7a940cf250 0x7f7a940ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:20.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.153+0000 7f7a98d90640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a940cf6e0 con 0x7f7a9406b240 2024-04-25T08:54:20.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.153+0000 7f7a98d90640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a940cf9c0 con 0x7f7a9415ded0 2024-04-25T08:54:20.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.153+0000 7f7a98d90640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a9415e2b0 con 0x7f7a940cf250 2024-04-25T08:54:20.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.153+0000 7f7a92d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f7a9406b240 0x7f7a940b4db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:38190/0 (socket says 172.21.15.43:38190) 2024-04-25T08:54:20.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.153+0000 7f7a92d9d640 1 -- 172.21.15.43:0/1088045933 learned_addr learned my addr 172.21.15.43:0/1088045933 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:20.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.153+0000 7f7a91d9b640 1 --2- 172.21.15.43:0/1088045933 >> v2:172.21.15.193:3300/0 conn(0x7f7a940cf250 0x7f7a940ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:20.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.153+0000 7f7a9159a640 1 -- 172.21.15.43:0/1088045933 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3111322536 0 0) 0x7f7a940cf6e0 con 0x7f7a9406b240 2024-04-25T08:54:20.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.154+0000 7f7a9159a640 1 -- 172.21.15.43:0/1088045933 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a80003520 con 0x7f7a9406b240 2024-04-25T08:54:20.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.154+0000 7f7a9159a640 1 -- 172.21.15.43:0/1088045933 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1050599153 0 0) 0x7f7a940cf9c0 con 0x7f7a9415ded0 2024-04-25T08:54:20.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.154+0000 7f7a9159a640 1 -- 172.21.15.43:0/1088045933 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a940cf6e0 con 0x7f7a9415ded0 2024-04-25T08:54:20.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.154+0000 7f7a9159a640 1 -- 172.21.15.43:0/1088045933 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2293039846 0 0) 0x7f7a80003520 con 0x7f7a9406b240 2024-04-25T08:54:20.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.154+0000 7f7a9159a640 1 -- 172.21.15.43:0/1088045933 >> v2:172.21.15.193:3300/0 conn(0x7f7a940cf250 msgr2=0x7f7a940ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:20.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.154+0000 7f7a9159a640 1 --2- 172.21.15.43:0/1088045933 >> v2:172.21.15.193:3300/0 conn(0x7f7a940cf250 0x7f7a940ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:20.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.154+0000 7f7a9159a640 1 -- 172.21.15.43:0/1088045933 >> v1:172.21.15.193:6789/0 conn(0x7f7a9415ded0 legacy=0x7f7a9406ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:20.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.154+0000 7f7a9159a640 1 -- 172.21.15.43:0/1088045933 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a940cfca0 con 0x7f7a9406b240 2024-04-25T08:54:20.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.154+0000 7f7a9159a640 1 -- 172.21.15.43:0/1088045933 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f7a840019f0 con 0x7f7a9406b240 2024-04-25T08:54:20.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.154+0000 7f7a9159a640 1 -- 172.21.15.43:0/1088045933 >> v1:172.21.15.43:6789/0 conn(0x7f7a9406b240 legacy=0x7f7a940b4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:20.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.154+0000 7f7a9159a640 1 --2- 172.21.15.43:0/1088045933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a80003b40 0x7f7a80003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:20.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.154+0000 7f7a9159a640 1 --2- 172.21.15.43:0/1088045933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a80004800 0x7f7a80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:20.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.154+0000 7f7a9159a640 1 --2- 172.21.15.43:0/1088045933 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7a800051c0 0x7f7a8000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:20.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.154+0000 7f7a9159a640 1 -- 172.21.15.43:0/1088045933 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8000c010 con 0x7f7a80003b40 2024-04-25T08:54:20.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.154+0000 7f7a9159a640 1 -- 172.21.15.43:0/1088045933 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8000c1c0 con 0x7f7a80004800 2024-04-25T08:54:20.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.154+0000 7f7a9159a640 1 -- 172.21.15.43:0/1088045933 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8000c3c0 con 0x7f7a800051c0 2024-04-25T08:54:20.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.154+0000 7f7a9259c640 1 --2- 172.21.15.43:0/1088045933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a80003b40 0x7f7a80003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:20.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.154+0000 7f7a91d9b640 1 --2- 172.21.15.43:0/1088045933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a80004800 0x7f7a80004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:20.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.154+0000 7f7a92d9d640 1 --2- 172.21.15.43:0/1088045933 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7a800051c0 0x7f7a8000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:20.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.154+0000 7f7a9259c640 1 -- 172.21.15.43:0/1088045933 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7a800051c0 msgr2=0x7f7a8000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:20.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.154+0000 7f7a9259c640 1 --2- 172.21.15.43:0/1088045933 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7a800051c0 0x7f7a8000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:20.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.154+0000 7f7a9259c640 1 -- 172.21.15.43:0/1088045933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a80004800 msgr2=0x7f7a80004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:20.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.154+0000 7f7a9259c640 1 --2- 172.21.15.43:0/1088045933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a80004800 0x7f7a80004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:20.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.154+0000 7f7a92d9d640 1 -- 172.21.15.43:0/1088045933 reap_dead start 2024-04-25T08:54:20.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.154+0000 7f7a9259c640 1 -- 172.21.15.43:0/1088045933 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a80003520 con 0x7f7a80003b40 2024-04-25T08:54:20.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.155+0000 7f7a9259c640 1 --2- 172.21.15.43:0/1088045933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a80003b40 0x7f7a80003f40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c0018d0 tx=0x7f7a80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=732dd24bbddd6c5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:20.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.155+0000 7f7a9159a640 1 -- 172.21.15.43:0/1088045933 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c0014f0 con 0x7f7a80003b40 2024-04-25T08:54:20.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.155+0000 7f7a9159a640 1 -- 172.21.15.43:0/1088045933 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a8c0052d0 con 0x7f7a80003b40 2024-04-25T08:54:20.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.155+0000 7f7a9159a640 1 -- 172.21.15.43:0/1088045933 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c02ac60 con 0x7f7a80003b40 2024-04-25T08:54:20.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.156+0000 7f7a98d90640 1 -- 172.21.15.43:0/1088045933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a80003b40 msgr2=0x7f7a80003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:20.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.156+0000 7f7a98d90640 1 --2- 172.21.15.43:0/1088045933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a80003b40 0x7f7a80003f40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c0018d0 tx=0x7f7a80005830 comp rx=0 tx=0).stop 2024-04-25T08:54:20.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.156+0000 7f7a98d90640 1 -- 172.21.15.43:0/1088045933 shutdown_connections 2024-04-25T08:54:20.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.156+0000 7f7a98d90640 1 --2- 172.21.15.43:0/1088045933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a80003b40 0x7f7a80003f40 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:20.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.156+0000 7f7a98d90640 1 -- 172.21.15.43:0/1088045933 >> 172.21.15.43:0/1088045933 conn(0x7f7a940b2490 msgr2=0x7f7a940b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:20.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.156+0000 7f7a98d90640 1 -- 172.21.15.43:0/1088045933 shutdown_connections 2024-04-25T08:54:20.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.156+0000 7f7a98d90640 1 -- 172.21.15.43:0/1088045933 wait complete. 2024-04-25T08:54:20.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.157+0000 7f7a98d90640 1 Processor -- start 2024-04-25T08:54:20.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.157+0000 7f7a98d90640 1 -- start start 2024-04-25T08:54:20.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.158+0000 7f7a98d90640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a80003b40 0x7f7a94152850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:20.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.158+0000 7f7a98d90640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a9414f1b0 0x7f7a9414ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:20.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.158+0000 7f7a98d90640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7a9415acb0 0x7f7a9415b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:20.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.158+0000 7f7a98d90640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a940cefd0 con 0x7f7a80003b40 2024-04-25T08:54:20.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.158+0000 7f7a98d90640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a940cee50 con 0x7f7a9414f1b0 2024-04-25T08:54:20.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.158+0000 7f7a98d90640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a940ce740 con 0x7f7a9415acb0 2024-04-25T08:54:20.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.158+0000 7f7a9259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a80003b40 0x7f7a94152850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:20.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.158+0000 7f7a9259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a80003b40 0x7f7a94152850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55918/0 (socket says 172.21.15.43:55918) 2024-04-25T08:54:20.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.158+0000 7f7a9259c640 1 -- 172.21.15.43:0/3222609682 learned_addr learned my addr 172.21.15.43:0/3222609682 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:20.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.158+0000 7f7a92d9d640 1 --2- 172.21.15.43:0/3222609682 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7a9415acb0 0x7f7a9415b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:20.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.158+0000 7f7a91d9b640 1 --2- 172.21.15.43:0/3222609682 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a9414f1b0 0x7f7a9414ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:20.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.158+0000 7f7a9259c640 1 -- 172.21.15.43:0/3222609682 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7a9415acb0 msgr2=0x7f7a9415b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:20.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.158+0000 7f7a9259c640 1 --2- 172.21.15.43:0/3222609682 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7a9415acb0 0x7f7a9415b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:20.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.158+0000 7f7a9259c640 1 -- 172.21.15.43:0/3222609682 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a9414f1b0 msgr2=0x7f7a9414ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:20.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.158+0000 7f7a9259c640 1 --2- 172.21.15.43:0/3222609682 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a9414f1b0 0x7f7a9414ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:20.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.158+0000 7f7a9259c640 1 -- 172.21.15.43:0/3222609682 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a94161220 con 0x7f7a80003b40 2024-04-25T08:54:20.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.158+0000 7f7a91d9b640 1 --2- 172.21.15.43:0/3222609682 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a9414f1b0 0x7f7a9414ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:20.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.158+0000 7f7a9259c640 1 --2- 172.21.15.43:0/3222609682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a80003b40 0x7f7a94152850 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c02a6f0 tx=0x7f7a8c0012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:20.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.159+0000 7f7a8affd640 1 -- 172.21.15.43:0/3222609682 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c037040 con 0x7f7a80003b40 2024-04-25T08:54:20.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.159+0000 7f7a8affd640 1 -- 172.21.15.43:0/3222609682 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a8c006e50 con 0x7f7a80003b40 2024-04-25T08:54:20.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.159+0000 7f7a8affd640 1 -- 172.21.15.43:0/3222609682 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c0014f0 con 0x7f7a80003b40 2024-04-25T08:54:20.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.159+0000 7f7a98d90640 1 -- 172.21.15.43:0/3222609682 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a94152e30 con 0x7f7a80003b40 2024-04-25T08:54:20.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.159+0000 7f7a98d90640 1 -- 172.21.15.43:0/3222609682 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a94160a20 con 0x7f7a80003b40 2024-04-25T08:54:20.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.160+0000 7f7a98d90640 1 -- 172.21.15.43:0/3222609682 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a940b3ce0 con 0x7f7a80003b40 2024-04-25T08:54:20.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.160+0000 7f7a8affd640 1 -- 172.21.15.43:0/3222609682 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7a8c043020 con 0x7f7a80003b40 2024-04-25T08:54:20.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.161+0000 7f7a8affd640 1 --2- 172.21.15.43:0/3222609682 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7a7406b4d0 0x7f7a7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:20.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.161+0000 7f7a8affd640 1 -- 172.21.15.43:0/3222609682 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7a8c0a9990 con 0x7f7a80003b40 2024-04-25T08:54:20.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.161+0000 7f7a91d9b640 1 --2- 172.21.15.43:0/3222609682 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7a7406b4d0 0x7f7a7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:20.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.161+0000 7f7a91d9b640 1 --2- 172.21.15.43:0/3222609682 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7a7406b4d0 0x7f7a7406d990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7a94155b20 tx=0x7f7a7c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:20.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.165+0000 7f7a8affd640 1 -- 172.21.15.43:0/3222609682 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7a8c078780 con 0x7f7a80003b40 2024-04-25T08:54:20.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.341+0000 7f7a98d90640 1 -- 172.21.15.43:0/3222609682 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7a94074ef0 con 0x7f7a7406b4d0 2024-04-25T08:54:20.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.348+0000 7f7a8affd640 1 -- 172.21.15.43:0/3222609682 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251836 (secure 0 0 0) 0x7f7a94074ef0 con 0x7f7a7406b4d0 2024-04-25T08:54:20.340 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:54:20.342 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-25T08:54:20.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.350+0000 7f7a98d90640 1 -- 172.21.15.43:0/3222609682 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7a7406b4d0 msgr2=0x7f7a7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:20.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.350+0000 7f7a98d90640 1 --2- 172.21.15.43:0/3222609682 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7a7406b4d0 0x7f7a7406d990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7a94155b20 tx=0x7f7a7c008040 comp rx=0 tx=0).stop 2024-04-25T08:54:20.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.350+0000 7f7a98d90640 1 -- 172.21.15.43:0/3222609682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a80003b40 msgr2=0x7f7a94152850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:20.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.350+0000 7f7a98d90640 1 --2- 172.21.15.43:0/3222609682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a80003b40 0x7f7a94152850 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c02a6f0 tx=0x7f7a8c0012d0 comp rx=0 tx=0).stop 2024-04-25T08:54:20.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.350+0000 7f7a98d90640 1 -- 172.21.15.43:0/3222609682 shutdown_connections 2024-04-25T08:54:20.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.350+0000 7f7a98d90640 1 --2- 172.21.15.43:0/3222609682 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7a9415acb0 0x7f7a9415b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:20.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.350+0000 7f7a98d90640 1 --2- 172.21.15.43:0/3222609682 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7a7406b4d0 0x7f7a7406d990 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:20.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.350+0000 7f7a98d90640 1 --2- 172.21.15.43:0/3222609682 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a9414f1b0 0x7f7a9414ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:20.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.350+0000 7f7a98d90640 1 --2- 172.21.15.43:0/3222609682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a80003b40 0x7f7a94152850 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:20.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.350+0000 7f7a98d90640 1 -- 172.21.15.43:0/3222609682 >> 172.21.15.43:0/3222609682 conn(0x7f7a940b2490 msgr2=0x7f7a940b0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:20.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.350+0000 7f7a98d90640 1 -- 172.21.15.43:0/3222609682 shutdown_connections 2024-04-25T08:54:20.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.350+0000 7f7a98d90640 1 -- 172.21.15.43:0/3222609682 wait complete. 2024-04-25T08:54:20.352 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-25T08:54:19.696847+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235656,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739064,"statfs":{"total":773094113280,"available":772852801536,"internally_reserved":0,"allocated":2891776,"data_stored":1753028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001822"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.003073+0000","last_change":"2024-04-25T08:54:12.612549+0000","last_active":"2024-04-25T08:54:14.003073+0000","last_peered":"2024-04-25T08:54:14.003073+0000","last_clean":"2024-04-25T08:54:14.003073+0000","last_became_active":"2024-04-25T08:54:12.612294+0000","last_became_peered":"2024-04-25T08:54:12.612294+0000","last_unstale":"2024-04-25T08:54:14.003073+0000","last_undegraded":"2024-04-25T08:54:14.003073+0000","last_fullsized":"2024-04-25T08:54:14.003073+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:31:08.402155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.599873+0000","last_change":"2024-04-25T08:54:12.605023+0000","last_active":"2024-04-25T08:54:13.599873+0000","last_peered":"2024-04-25T08:54:13.599873+0000","last_clean":"2024-04-25T08:54:13.599873+0000","last_became_active":"2024-04-25T08:54:12.604703+0000","last_became_peered":"2024-04-25T08:54:12.604703+0000","last_unstale":"2024-04-25T08:54:13.599873+0000","last_undegraded":"2024-04-25T08:54:13.599873+0000","last_fullsized":"2024-04-25T08:54:13.599873+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:57.230096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002569+0000","last_change":"2024-04-25T08:54:12.611793+0000","last_active":"2024-04-25T08:54:14.002569+0000","last_peered":"2024-04-25T08:54:14.002569+0000","last_clean":"2024-04-25T08:54:14.002569+0000","last_became_active":"2024-04-25T08:54:12.611530+0000","last_became_peered":"2024-04-25T08:54:12.611530+0000","last_unstale":"2024-04-25T08:54:14.002569+0000","last_undegraded":"2024-04-25T08:54:14.002569+0000","last_fullsized":"2024-04-25T08:54:14.002569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:02.123220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601627+0000","last_change":"2024-04-25T08:54:12.600825+0000","last_active":"2024-04-25T08:54:13.601627+0000","last_peered":"2024-04-25T08:54:13.601627+0000","last_clean":"2024-04-25T08:54:13.601627+0000","last_became_active":"2024-04-25T08:54:12.600531+0000","last_became_peered":"2024-04-25T08:54:12.600531+0000","last_unstale":"2024-04-25T08:54:13.601627+0000","last_undegraded":"2024-04-25T08:54:13.601627+0000","last_fullsized":"2024-04-25T08:54:13.601627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:03.021953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002537+0000","last_change":"2024-04-25T08:54:12.610934+0000","last_active":"2024-04-25T08:54:14.002537+0000","last_peered":"2024-04-25T08:54:14.002537+0000","last_clean":"2024-04-25T08:54:14.002537+0000","last_became_active":"2024-04-25T08:54:12.610457+0000","last_became_peered":"2024-04-25T08:54:12.610457+0000","last_unstale":"2024-04-25T08:54:14.002537+0000","last_undegraded":"2024-04-25T08:54:14.002537+0000","last_fullsized":"2024-04-25T08:54:14.002537+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:54.612512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601664+0000","last_change":"2024-04-25T08:54:12.612007+0000","last_active":"2024-04-25T08:54:13.601664+0000","last_peered":"2024-04-25T08:54:13.601664+0000","last_clean":"2024-04-25T08:54:13.601664+0000","last_became_active":"2024-04-25T08:54:12.611744+0000","last_became_peered":"2024-04-25T08:54:12.611744+0000","last_unstale":"2024-04-25T08:54:13.601664+0000","last_undegraded":"2024-04-25T08:54:13.601664+0000","last_fullsized":"2024-04-25T08:54:13.601664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:06.222677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002411+0000","last_change":"2024-04-25T08:54:12.609790+0000","last_active":"2024-04-25T08:54:14.002411+0000","last_peered":"2024-04-25T08:54:14.002411+0000","last_clean":"2024-04-25T08:54:14.002411+0000","last_became_active":"2024-04-25T08:54:12.609504+0000","last_became_peered":"2024-04-25T08:54:12.609504+0000","last_unstale":"2024-04-25T08:54:14.002411+0000","last_undegraded":"2024-04-25T08:54:14.002411+0000","last_fullsized":"2024-04-25T08:54:14.002411+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:17.430796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.001963+0000","last_change":"2024-04-25T08:54:12.603756+0000","last_active":"2024-04-25T08:54:14.001963+0000","last_peered":"2024-04-25T08:54:14.001963+0000","last_clean":"2024-04-25T08:54:14.001963+0000","last_became_active":"2024-04-25T08:54:12.602966+0000","last_became_peered":"2024-04-25T08:54:12.602966+0000","last_unstale":"2024-04-25T08:54:14.001963+0000","last_undegraded":"2024-04-25T08:54:14.001963+0000","last_fullsized":"2024-04-25T08:54:14.001963+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:05.099361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.003077+0000","last_change":"2024-04-25T08:54:12.617394+0000","last_active":"2024-04-25T08:54:14.003077+0000","last_peered":"2024-04-25T08:54:14.003077+0000","last_clean":"2024-04-25T08:54:14.003077+0000","last_became_active":"2024-04-25T08:54:12.616960+0000","last_became_peered":"2024-04-25T08:54:12.616960+0000","last_unstale":"2024-04-25T08:54:14.003077+0000","last_undegraded":"2024-04-25T08:54:14.003077+0000","last_fullsized":"2024-04-25T08:54:14.003077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:31.051483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.003267+0000","last_change":"2024-04-25T08:54:12.607768+0000","last_active":"2024-04-25T08:54:14.003267+0000","last_peered":"2024-04-25T08:54:14.003267+0000","last_clean":"2024-04-25T08:54:14.003267+0000","last_became_active":"2024-04-25T08:54:12.607503+0000","last_became_peered":"2024-04-25T08:54:12.607503+0000","last_unstale":"2024-04-25T08:54:14.003267+0000","last_undegraded":"2024-04-25T08:54:14.003267+0000","last_fullsized":"2024-04-25T08:54:14.003267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:57.230096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002988+0000","last_change":"2024-04-25T08:54:12.607264+0000","last_active":"2024-04-25T08:54:14.002988+0000","last_peered":"2024-04-25T08:54:14.002988+0000","last_clean":"2024-04-25T08:54:14.002988+0000","last_became_active":"2024-04-25T08:54:12.606961+0000","last_became_peered":"2024-04-25T08:54:12.606961+0000","last_unstale":"2024-04-25T08:54:14.002988+0000","last_undegraded":"2024-04-25T08:54:14.002988+0000","last_fullsized":"2024-04-25T08:54:14.002988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:45.793417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601252+0000","last_change":"2024-04-25T08:54:12.604064+0000","last_active":"2024-04-25T08:54:13.601252+0000","last_peered":"2024-04-25T08:54:13.601252+0000","last_clean":"2024-04-25T08:54:13.601252+0000","last_became_active":"2024-04-25T08:54:12.603807+0000","last_became_peered":"2024-04-25T08:54:12.603807+0000","last_unstale":"2024-04-25T08:54:13.601252+0000","last_undegraded":"2024-04-25T08:54:13.601252+0000","last_fullsized":"2024-04-25T08:54:13.601252+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:38.245982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.003119+0000","last_change":"2024-04-25T08:54:12.607207+0000","last_active":"2024-04-25T08:54:14.003119+0000","last_peered":"2024-04-25T08:54:14.003119+0000","last_clean":"2024-04-25T08:54:14.003119+0000","last_became_active":"2024-04-25T08:54:12.606934+0000","last_became_peered":"2024-04-25T08:54:12.606934+0000","last_unstale":"2024-04-25T08:54:14.003119+0000","last_undegraded":"2024-04-25T08:54:14.003119+0000","last_fullsized":"2024-04-25T08:54:14.003119+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:02.123220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.001959+0000","last_change":"2024-04-25T08:54:12.596313+0000","last_active":"2024-04-25T08:54:14.001959+0000","last_peered":"2024-04-25T08:54:14.001959+0000","last_clean":"2024-04-25T08:54:14.001959+0000","last_became_active":"2024-04-25T08:54:12.596005+0000","last_became_peered":"2024-04-25T08:54:12.596005+0000","last_unstale":"2024-04-25T08:54:14.001959+0000","last_undegraded":"2024-04-25T08:54:14.001959+0000","last_fullsized":"2024-04-25T08:54:14.001959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:28.227965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.599534+0000","last_change":"2024-04-25T08:54:12.610230+0000","last_active":"2024-04-25T08:54:13.599534+0000","last_peered":"2024-04-25T08:54:13.599534+0000","last_clean":"2024-04-25T08:54:13.599534+0000","last_became_active":"2024-04-25T08:54:12.609932+0000","last_became_peered":"2024-04-25T08:54:12.609932+0000","last_unstale":"2024-04-25T08:54:13.599534+0000","last_undegraded":"2024-04-25T08:54:13.599534+0000","last_fullsized":"2024-04-25T08:54:13.599534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:54.538580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601103+0000","last_change":"2024-04-25T08:54:12.600584+0000","last_active":"2024-04-25T08:54:13.601103+0000","last_peered":"2024-04-25T08:54:13.601103+0000","last_clean":"2024-04-25T08:54:13.601103+0000","last_became_active":"2024-04-25T08:54:12.598989+0000","last_became_peered":"2024-04-25T08:54:12.598989+0000","last_unstale":"2024-04-25T08:54:13.601103+0000","last_undegraded":"2024-04-25T08:54:13.601103+0000","last_fullsized":"2024-04-25T08:54:13.601103+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:45.793417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601517+0000","last_change":"2024-04-25T08:54:12.607275+0000","last_active":"2024-04-25T08:54:13.601517+0000","last_peered":"2024-04-25T08:54:13.601517+0000","last_clean":"2024-04-25T08:54:13.601517+0000","last_became_active":"2024-04-25T08:54:12.606990+0000","last_became_peered":"2024-04-25T08:54:12.606990+0000","last_unstale":"2024-04-25T08:54:13.601517+0000","last_undegraded":"2024-04-25T08:54:13.601517+0000","last_fullsized":"2024-04-25T08:54:13.601517+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:18.500422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002292+0000","last_change":"2024-04-25T08:54:12.607413+0000","last_active":"2024-04-25T08:54:14.002292+0000","last_peered":"2024-04-25T08:54:14.002292+0000","last_clean":"2024-04-25T08:54:14.002292+0000","last_became_active":"2024-04-25T08:54:12.607135+0000","last_became_peered":"2024-04-25T08:54:12.607135+0000","last_unstale":"2024-04-25T08:54:14.002292+0000","last_undegraded":"2024-04-25T08:54:14.002292+0000","last_fullsized":"2024-04-25T08:54:14.002292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:53.056423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002801+0000","last_change":"2024-04-25T08:54:12.607734+0000","last_active":"2024-04-25T08:54:14.002801+0000","last_peered":"2024-04-25T08:54:14.002801+0000","last_clean":"2024-04-25T08:54:14.002801+0000","last_became_active":"2024-04-25T08:54:12.607462+0000","last_became_peered":"2024-04-25T08:54:12.607462+0000","last_unstale":"2024-04-25T08:54:14.002801+0000","last_undegraded":"2024-04-25T08:54:14.002801+0000","last_fullsized":"2024-04-25T08:54:14.002801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:06.750914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002746+0000","last_change":"2024-04-25T08:54:12.609395+0000","last_active":"2024-04-25T08:54:14.002746+0000","last_peered":"2024-04-25T08:54:14.002746+0000","last_clean":"2024-04-25T08:54:14.002746+0000","last_became_active":"2024-04-25T08:54:12.609094+0000","last_became_peered":"2024-04-25T08:54:12.609094+0000","last_unstale":"2024-04-25T08:54:14.002746+0000","last_undegraded":"2024-04-25T08:54:14.002746+0000","last_fullsized":"2024-04-25T08:54:14.002746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:02.123220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002547+0000","last_change":"2024-04-25T08:54:12.605364+0000","last_active":"2024-04-25T08:54:14.002547+0000","last_peered":"2024-04-25T08:54:14.002547+0000","last_clean":"2024-04-25T08:54:14.002547+0000","last_became_active":"2024-04-25T08:54:12.605084+0000","last_became_peered":"2024-04-25T08:54:12.605084+0000","last_unstale":"2024-04-25T08:54:14.002547+0000","last_undegraded":"2024-04-25T08:54:14.002547+0000","last_fullsized":"2024-04-25T08:54:14.002547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:20.570657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601962+0000","last_change":"2024-04-25T08:54:12.606453+0000","last_active":"2024-04-25T08:54:13.601962+0000","last_peered":"2024-04-25T08:54:13.601962+0000","last_clean":"2024-04-25T08:54:13.601962+0000","last_became_active":"2024-04-25T08:54:12.606072+0000","last_became_peered":"2024-04-25T08:54:12.606072+0000","last_unstale":"2024-04-25T08:54:13.601962+0000","last_undegraded":"2024-04-25T08:54:13.601962+0000","last_fullsized":"2024-04-25T08:54:13.601962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:36.638705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002959+0000","last_change":"2024-04-25T08:54:12.615079+0000","last_active":"2024-04-25T08:54:14.002959+0000","last_peered":"2024-04-25T08:54:14.002959+0000","last_clean":"2024-04-25T08:54:14.002959+0000","last_became_active":"2024-04-25T08:54:12.614779+0000","last_became_peered":"2024-04-25T08:54:12.614779+0000","last_unstale":"2024-04-25T08:54:14.002959+0000","last_undegraded":"2024-04-25T08:54:14.002959+0000","last_fullsized":"2024-04-25T08:54:14.002959+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:20.570657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601263+0000","last_change":"2024-04-25T08:54:12.605871+0000","last_active":"2024-04-25T08:54:13.601263+0000","last_peered":"2024-04-25T08:54:13.601263+0000","last_clean":"2024-04-25T08:54:13.601263+0000","last_became_active":"2024-04-25T08:54:12.605598+0000","last_became_peered":"2024-04-25T08:54:12.605598+0000","last_unstale":"2024-04-25T08:54:13.601263+0000","last_undegraded":"2024-04-25T08:54:13.601263+0000","last_fullsized":"2024-04-25T08:54:13.601263+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:57.230096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.003397+0000","last_change":"2024-04-25T08:54:12.612253+0000","last_active":"2024-04-25T08:54:14.003397+0000","last_peered":"2024-04-25T08:54:14.003397+0000","last_clean":"2024-04-25T08:54:14.003397+0000","last_became_active":"2024-04-25T08:54:12.611923+0000","last_became_peered":"2024-04-25T08:54:12.611923+0000","last_unstale":"2024-04-25T08:54:14.003397+0000","last_undegraded":"2024-04-25T08:54:14.003397+0000","last_fullsized":"2024-04-25T08:54:14.003397+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:06.222677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.003254+0000","last_change":"2024-04-25T08:54:12.608764+0000","last_active":"2024-04-25T08:54:14.003254+0000","last_peered":"2024-04-25T08:54:14.003254+0000","last_clean":"2024-04-25T08:54:14.003254+0000","last_became_active":"2024-04-25T08:54:12.608488+0000","last_became_peered":"2024-04-25T08:54:12.608488+0000","last_unstale":"2024-04-25T08:54:14.003254+0000","last_undegraded":"2024-04-25T08:54:14.003254+0000","last_fullsized":"2024-04-25T08:54:14.003254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:18.500422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.599706+0000","last_change":"2024-04-25T08:54:12.602969+0000","last_active":"2024-04-25T08:54:13.599706+0000","last_peered":"2024-04-25T08:54:13.599706+0000","last_clean":"2024-04-25T08:54:13.599706+0000","last_became_active":"2024-04-25T08:54:12.602464+0000","last_became_peered":"2024-04-25T08:54:12.602464+0000","last_unstale":"2024-04-25T08:54:13.599706+0000","last_undegraded":"2024-04-25T08:54:13.599706+0000","last_fullsized":"2024-04-25T08:54:13.599706+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:02.123220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.003276+0000","last_change":"2024-04-25T08:54:12.613884+0000","last_active":"2024-04-25T08:54:14.003276+0000","last_peered":"2024-04-25T08:54:14.003276+0000","last_clean":"2024-04-25T08:54:14.003276+0000","last_became_active":"2024-04-25T08:54:12.613610+0000","last_became_peered":"2024-04-25T08:54:12.613610+0000","last_unstale":"2024-04-25T08:54:14.003276+0000","last_undegraded":"2024-04-25T08:54:14.003276+0000","last_fullsized":"2024-04-25T08:54:14.003276+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:18.500422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002701+0000","last_change":"2024-04-25T08:54:12.611194+0000","last_active":"2024-04-25T08:54:14.002701+0000","last_peered":"2024-04-25T08:54:14.002701+0000","last_clean":"2024-04-25T08:54:14.002701+0000","last_became_active":"2024-04-25T08:54:12.610877+0000","last_became_peered":"2024-04-25T08:54:12.610877+0000","last_unstale":"2024-04-25T08:54:14.002701+0000","last_undegraded":"2024-04-25T08:54:14.002701+0000","last_fullsized":"2024-04-25T08:54:14.002701+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:45.793417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.602013+0000","last_change":"2024-04-25T08:54:12.600165+0000","last_active":"2024-04-25T08:54:13.602013+0000","last_peered":"2024-04-25T08:54:13.602013+0000","last_clean":"2024-04-25T08:54:13.602013+0000","last_became_active":"2024-04-25T08:54:12.599911+0000","last_became_peered":"2024-04-25T08:54:12.599911+0000","last_unstale":"2024-04-25T08:54:13.602013+0000","last_undegraded":"2024-04-25T08:54:13.602013+0000","last_fullsized":"2024-04-25T08:54:13.602013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:06.750914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002238+0000","last_change":"2024-04-25T08:54:12.604655+0000","last_active":"2024-04-25T08:54:14.002238+0000","last_peered":"2024-04-25T08:54:14.002238+0000","last_clean":"2024-04-25T08:54:14.002238+0000","last_became_active":"2024-04-25T08:54:12.604366+0000","last_became_peered":"2024-04-25T08:54:12.604366+0000","last_unstale":"2024-04-25T08:54:14.002238+0000","last_undegraded":"2024-04-25T08:54:14.002238+0000","last_fullsized":"2024-04-25T08:54:14.002238+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:39.105098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002363+0000","last_change":"2024-04-25T08:54:12.615832+0000","last_active":"2024-04-25T08:54:14.002363+0000","last_peered":"2024-04-25T08:54:14.002363+0000","last_clean":"2024-04-25T08:54:14.002363+0000","last_became_active":"2024-04-25T08:54:12.615442+0000","last_became_peered":"2024-04-25T08:54:12.615442+0000","last_unstale":"2024-04-25T08:54:14.002363+0000","last_undegraded":"2024-04-25T08:54:14.002363+0000","last_fullsized":"2024-04-25T08:54:14.002363+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:54.538580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601064+0000","last_change":"2024-04-25T08:54:12.598903+0000","last_active":"2024-04-25T08:54:13.601064+0000","last_peered":"2024-04-25T08:54:13.601064+0000","last_clean":"2024-04-25T08:54:13.601064+0000","last_became_active":"2024-04-25T08:54:12.598005+0000","last_became_peered":"2024-04-25T08:54:12.598005+0000","last_unstale":"2024-04-25T08:54:13.601064+0000","last_undegraded":"2024-04-25T08:54:13.601064+0000","last_fullsized":"2024-04-25T08:54:13.601064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:11.183046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.003238+0000","last_change":"2024-04-25T08:54:11.585710+0000","last_active":"2024-04-25T08:54:14.003238+0000","last_peered":"2024-04-25T08:54:14.003238+0000","last_clean":"2024-04-25T08:54:14.003238+0000","last_became_active":"2024-04-25T08:54:11.585090+0000","last_became_peered":"2024-04-25T08:54:11.585090+0000","last_unstale":"2024-04-25T08:54:14.003238+0000","last_undegraded":"2024-04-25T08:54:14.003238+0000","last_fullsized":"2024-04-25T08:54:14.003238+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:10.545906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002798+0000","last_change":"2024-04-25T08:54:12.600641+0000","last_active":"2024-04-25T08:54:14.002798+0000","last_peered":"2024-04-25T08:54:14.002798+0000","last_clean":"2024-04-25T08:54:14.002798+0000","last_became_active":"2024-04-25T08:54:12.598924+0000","last_became_peered":"2024-04-25T08:54:12.598924+0000","last_unstale":"2024-04-25T08:54:14.002798+0000","last_undegraded":"2024-04-25T08:54:14.002798+0000","last_fullsized":"2024-04-25T08:54:14.002798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:18.500422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002906+0000","last_change":"2024-04-25T08:54:11.588038+0000","last_active":"2024-04-25T08:54:14.002906+0000","last_peered":"2024-04-25T08:54:14.002906+0000","last_clean":"2024-04-25T08:54:14.002906+0000","last_became_active":"2024-04-25T08:54:11.587710+0000","last_became_peered":"2024-04-25T08:54:11.587710+0000","last_unstale":"2024-04-25T08:54:14.002906+0000","last_undegraded":"2024-04-25T08:54:14.002906+0000","last_fullsized":"2024-04-25T08:54:14.002906+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:37.241134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.003011+0000","last_change":"2024-04-25T08:54:11.590094+0000","last_active":"2024-04-25T08:54:14.003011+0000","last_peered":"2024-04-25T08:54:14.003011+0000","last_clean":"2024-04-25T08:54:14.003011+0000","last_became_active":"2024-04-25T08:54:11.589720+0000","last_became_peered":"2024-04-25T08:54:11.589720+0000","last_unstale":"2024-04-25T08:54:14.003011+0000","last_undegraded":"2024-04-25T08:54:14.003011+0000","last_fullsized":"2024-04-25T08:54:14.003011+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:54.860395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002127+0000","last_change":"2024-04-25T08:54:12.603523+0000","last_active":"2024-04-25T08:54:14.002127+0000","last_peered":"2024-04-25T08:54:14.002127+0000","last_clean":"2024-04-25T08:54:14.002127+0000","last_became_active":"2024-04-25T08:54:12.602684+0000","last_became_peered":"2024-04-25T08:54:12.602684+0000","last_unstale":"2024-04-25T08:54:14.002127+0000","last_undegraded":"2024-04-25T08:54:14.002127+0000","last_fullsized":"2024-04-25T08:54:14.002127+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:54.538580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601357+0000","last_change":"2024-04-25T08:54:11.578444+0000","last_active":"2024-04-25T08:54:13.601357+0000","last_peered":"2024-04-25T08:54:13.601357+0000","last_clean":"2024-04-25T08:54:13.601357+0000","last_became_active":"2024-04-25T08:54:11.578170+0000","last_became_peered":"2024-04-25T08:54:11.578170+0000","last_unstale":"2024-04-25T08:54:13.601357+0000","last_undegraded":"2024-04-25T08:54:13.601357+0000","last_fullsized":"2024-04-25T08:54:13.601357+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:37.241134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.599533+0000","last_change":"2024-04-25T08:54:12.602572+0000","last_active":"2024-04-25T08:54:13.599533+0000","last_peered":"2024-04-25T08:54:13.599533+0000","last_clean":"2024-04-25T08:54:13.599533+0000","last_became_active":"2024-04-25T08:54:12.601587+0000","last_became_peered":"2024-04-25T08:54:12.601587+0000","last_unstale":"2024-04-25T08:54:13.599533+0000","last_undegraded":"2024-04-25T08:54:13.599533+0000","last_fullsized":"2024-04-25T08:54:13.599533+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:56.990780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.003202+0000","last_change":"2024-04-25T08:54:11.592937+0000","last_active":"2024-04-25T08:54:14.003202+0000","last_peered":"2024-04-25T08:54:14.003202+0000","last_clean":"2024-04-25T08:54:14.003202+0000","last_became_active":"2024-04-25T08:54:11.592647+0000","last_became_peered":"2024-04-25T08:54:11.592647+0000","last_unstale":"2024-04-25T08:54:14.003202+0000","last_undegraded":"2024-04-25T08:54:14.003202+0000","last_fullsized":"2024-04-25T08:54:14.003202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:45.942159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601977+0000","last_change":"2024-04-25T08:54:12.608078+0000","last_active":"2024-04-25T08:54:13.601977+0000","last_peered":"2024-04-25T08:54:13.601977+0000","last_clean":"2024-04-25T08:54:13.601977+0000","last_became_active":"2024-04-25T08:54:12.605851+0000","last_became_peered":"2024-04-25T08:54:12.605851+0000","last_unstale":"2024-04-25T08:54:13.601977+0000","last_undegraded":"2024-04-25T08:54:13.601977+0000","last_fullsized":"2024-04-25T08:54:13.601977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:54.612512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601849+0000","last_change":"2024-04-25T08:54:11.588593+0000","last_active":"2024-04-25T08:54:13.601849+0000","last_peered":"2024-04-25T08:54:13.601849+0000","last_clean":"2024-04-25T08:54:13.601849+0000","last_became_active":"2024-04-25T08:54:11.588087+0000","last_became_peered":"2024-04-25T08:54:11.588087+0000","last_unstale":"2024-04-25T08:54:13.601849+0000","last_undegraded":"2024-04-25T08:54:13.601849+0000","last_fullsized":"2024-04-25T08:54:13.601849+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:10.178198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002526+0000","last_change":"2024-04-25T08:54:12.610492+0000","last_active":"2024-04-25T08:54:14.002526+0000","last_peered":"2024-04-25T08:54:14.002526+0000","last_clean":"2024-04-25T08:54:14.002526+0000","last_became_active":"2024-04-25T08:54:12.610228+0000","last_became_peered":"2024-04-25T08:54:12.610228+0000","last_unstale":"2024-04-25T08:54:14.002526+0000","last_undegraded":"2024-04-25T08:54:14.002526+0000","last_fullsized":"2024-04-25T08:54:14.002526+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:56.990780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.599817+0000","last_change":"2024-04-25T08:54:11.585441+0000","last_active":"2024-04-25T08:54:13.599817+0000","last_peered":"2024-04-25T08:54:13.599817+0000","last_clean":"2024-04-25T08:54:13.599817+0000","last_became_active":"2024-04-25T08:54:11.585098+0000","last_became_peered":"2024-04-25T08:54:11.585098+0000","last_unstale":"2024-04-25T08:54:13.599817+0000","last_undegraded":"2024-04-25T08:54:13.599817+0000","last_fullsized":"2024-04-25T08:54:13.599817+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:50.460868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601444+0000","last_change":"2024-04-25T08:54:12.606323+0000","last_active":"2024-04-25T08:54:13.601444+0000","last_peered":"2024-04-25T08:54:13.601444+0000","last_clean":"2024-04-25T08:54:13.601444+0000","last_became_active":"2024-04-25T08:54:12.606063+0000","last_became_peered":"2024-04-25T08:54:12.606063+0000","last_unstale":"2024-04-25T08:54:13.601444+0000","last_undegraded":"2024-04-25T08:54:13.601444+0000","last_fullsized":"2024-04-25T08:54:13.601444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:46.947007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002754+0000","last_change":"2024-04-25T08:54:11.590170+0000","last_active":"2024-04-25T08:54:14.002754+0000","last_peered":"2024-04-25T08:54:14.002754+0000","last_clean":"2024-04-25T08:54:14.002754+0000","last_became_active":"2024-04-25T08:54:11.589869+0000","last_became_peered":"2024-04-25T08:54:11.589869+0000","last_unstale":"2024-04-25T08:54:14.002754+0000","last_undegraded":"2024-04-25T08:54:14.002754+0000","last_fullsized":"2024-04-25T08:54:14.002754+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:10.545906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002430+0000","last_change":"2024-04-25T08:54:12.607302+0000","last_active":"2024-04-25T08:54:14.002430+0000","last_peered":"2024-04-25T08:54:14.002430+0000","last_clean":"2024-04-25T08:54:14.002430+0000","last_became_active":"2024-04-25T08:54:12.607042+0000","last_became_peered":"2024-04-25T08:54:12.607042+0000","last_unstale":"2024-04-25T08:54:14.002430+0000","last_undegraded":"2024-04-25T08:54:14.002430+0000","last_fullsized":"2024-04-25T08:54:14.002430+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:36.638705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002793+0000","last_change":"2024-04-25T08:54:11.586629+0000","last_active":"2024-04-25T08:54:14.002793+0000","last_peered":"2024-04-25T08:54:14.002793+0000","last_clean":"2024-04-25T08:54:14.002793+0000","last_became_active":"2024-04-25T08:54:11.586289+0000","last_became_peered":"2024-04-25T08:54:11.586289+0000","last_unstale":"2024-04-25T08:54:14.002793+0000","last_undegraded":"2024-04-25T08:54:14.002793+0000","last_fullsized":"2024-04-25T08:54:14.002793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:12.212662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002117+0000","last_change":"2024-04-25T08:54:12.604702+0000","last_active":"2024-04-25T08:54:14.002117+0000","last_peered":"2024-04-25T08:54:14.002117+0000","last_clean":"2024-04-25T08:54:14.002117+0000","last_became_active":"2024-04-25T08:54:12.604410+0000","last_became_peered":"2024-04-25T08:54:12.604410+0000","last_unstale":"2024-04-25T08:54:14.002117+0000","last_undegraded":"2024-04-25T08:54:14.002117+0000","last_fullsized":"2024-04-25T08:54:14.002117+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:05.099361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.602125+0000","last_change":"2024-04-25T08:54:11.583543+0000","last_active":"2024-04-25T08:54:13.602125+0000","last_peered":"2024-04-25T08:54:13.602125+0000","last_clean":"2024-04-25T08:54:13.602125+0000","last_became_active":"2024-04-25T08:54:11.583254+0000","last_became_peered":"2024-04-25T08:54:11.583254+0000","last_unstale":"2024-04-25T08:54:13.602125+0000","last_undegraded":"2024-04-25T08:54:13.602125+0000","last_fullsized":"2024-04-25T08:54:13.602125+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:45.942159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002350+0000","last_change":"2024-04-25T08:54:12.609195+0000","last_active":"2024-04-25T08:54:14.002350+0000","last_peered":"2024-04-25T08:54:14.002350+0000","last_clean":"2024-04-25T08:54:14.002350+0000","last_became_active":"2024-04-25T08:54:12.608943+0000","last_became_peered":"2024-04-25T08:54:12.608943+0000","last_unstale":"2024-04-25T08:54:14.002350+0000","last_undegraded":"2024-04-25T08:54:14.002350+0000","last_fullsized":"2024-04-25T08:54:14.002350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:06.222677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002597+0000","last_change":"2024-04-25T08:54:11.586926+0000","last_active":"2024-04-25T08:54:14.002597+0000","last_peered":"2024-04-25T08:54:14.002597+0000","last_clean":"2024-04-25T08:54:14.002597+0000","last_became_active":"2024-04-25T08:54:11.586600+0000","last_became_peered":"2024-04-25T08:54:11.586600+0000","last_unstale":"2024-04-25T08:54:14.002597+0000","last_undegraded":"2024-04-25T08:54:14.002597+0000","last_fullsized":"2024-04-25T08:54:14.002597+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:10.178198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002584+0000","last_change":"2024-04-25T08:54:12.615811+0000","last_active":"2024-04-25T08:54:14.002584+0000","last_peered":"2024-04-25T08:54:14.002584+0000","last_clean":"2024-04-25T08:54:14.002584+0000","last_became_active":"2024-04-25T08:54:12.615492+0000","last_became_peered":"2024-04-25T08:54:12.615492+0000","last_unstale":"2024-04-25T08:54:14.002584+0000","last_undegraded":"2024-04-25T08:54:14.002584+0000","last_fullsized":"2024-04-25T08:54:14.002584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:56.990780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002293+0000","last_change":"2024-04-25T08:54:11.583593+0000","last_active":"2024-04-25T08:54:14.002293+0000","last_peered":"2024-04-25T08:54:14.002293+0000","last_clean":"2024-04-25T08:54:14.002293+0000","last_became_active":"2024-04-25T08:54:11.582876+0000","last_became_peered":"2024-04-25T08:54:11.582876+0000","last_unstale":"2024-04-25T08:54:14.002293+0000","last_undegraded":"2024-04-25T08:54:14.002293+0000","last_fullsized":"2024-04-25T08:54:14.002293+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:10.545906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002256+0000","last_change":"2024-04-25T08:54:12.609297+0000","last_active":"2024-04-25T08:54:14.002256+0000","last_peered":"2024-04-25T08:54:14.002256+0000","last_clean":"2024-04-25T08:54:14.002256+0000","last_became_active":"2024-04-25T08:54:12.608948+0000","last_became_peered":"2024-04-25T08:54:12.608948+0000","last_unstale":"2024-04-25T08:54:14.002256+0000","last_undegraded":"2024-04-25T08:54:14.002256+0000","last_fullsized":"2024-04-25T08:54:14.002256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:03.021953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601349+0000","last_change":"2024-04-25T08:54:11.586306+0000","last_active":"2024-04-25T08:54:13.601349+0000","last_peered":"2024-04-25T08:54:13.601349+0000","last_clean":"2024-04-25T08:54:13.601349+0000","last_became_active":"2024-04-25T08:54:11.585959+0000","last_became_peered":"2024-04-25T08:54:11.585959+0000","last_unstale":"2024-04-25T08:54:13.601349+0000","last_undegraded":"2024-04-25T08:54:13.601349+0000","last_fullsized":"2024-04-25T08:54:13.601349+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:40.673650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002907+0000","last_change":"2024-04-25T08:54:12.609239+0000","last_active":"2024-04-25T08:54:14.002907+0000","last_peered":"2024-04-25T08:54:14.002907+0000","last_clean":"2024-04-25T08:54:14.002907+0000","last_became_active":"2024-04-25T08:54:12.608994+0000","last_became_peered":"2024-04-25T08:54:12.608994+0000","last_unstale":"2024-04-25T08:54:14.002907+0000","last_undegraded":"2024-04-25T08:54:14.002907+0000","last_fullsized":"2024-04-25T08:54:14.002907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:44.251355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002393+0000","last_change":"2024-04-25T08:54:11.589122+0000","last_active":"2024-04-25T08:54:14.002393+0000","last_peered":"2024-04-25T08:54:14.002393+0000","last_clean":"2024-04-25T08:54:14.002393+0000","last_became_active":"2024-04-25T08:54:11.588772+0000","last_became_peered":"2024-04-25T08:54:11.588772+0000","last_unstale":"2024-04-25T08:54:14.002393+0000","last_undegraded":"2024-04-25T08:54:14.002393+0000","last_fullsized":"2024-04-25T08:54:14.002393+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:37.241134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.003230+0000","last_change":"2024-04-25T08:54:12.613985+0000","last_active":"2024-04-25T08:54:14.003230+0000","last_peered":"2024-04-25T08:54:14.003230+0000","last_clean":"2024-04-25T08:54:14.003230+0000","last_became_active":"2024-04-25T08:54:12.613717+0000","last_became_peered":"2024-04-25T08:54:12.613717+0000","last_unstale":"2024-04-25T08:54:14.003230+0000","last_undegraded":"2024-04-25T08:54:14.003230+0000","last_fullsized":"2024-04-25T08:54:14.003230+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:28.550328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.599846+0000","last_change":"2024-04-25T08:54:11.586724+0000","last_active":"2024-04-25T08:54:13.599846+0000","last_peered":"2024-04-25T08:54:13.599846+0000","last_clean":"2024-04-25T08:54:13.599846+0000","last_became_active":"2024-04-25T08:54:11.586219+0000","last_became_peered":"2024-04-25T08:54:11.586219+0000","last_unstale":"2024-04-25T08:54:13.599846+0000","last_undegraded":"2024-04-25T08:54:13.599846+0000","last_fullsized":"2024-04-25T08:54:13.599846+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:32.370052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601817+0000","last_change":"2024-04-25T08:54:12.600997+0000","last_active":"2024-04-25T08:54:13.601817+0000","last_peered":"2024-04-25T08:54:13.601817+0000","last_clean":"2024-04-25T08:54:13.601817+0000","last_became_active":"2024-04-25T08:54:12.600704+0000","last_became_peered":"2024-04-25T08:54:12.600704+0000","last_unstale":"2024-04-25T08:54:13.601817+0000","last_undegraded":"2024-04-25T08:54:13.601817+0000","last_fullsized":"2024-04-25T08:54:13.601817+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:17.430796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.001982+0000","last_change":"2024-04-25T08:54:11.578803+0000","last_active":"2024-04-25T08:54:14.001982+0000","last_peered":"2024-04-25T08:54:14.001982+0000","last_clean":"2024-04-25T08:54:14.001982+0000","last_became_active":"2024-04-25T08:54:11.578421+0000","last_became_peered":"2024-04-25T08:54:11.578421+0000","last_unstale":"2024-04-25T08:54:14.001982+0000","last_undegraded":"2024-04-25T08:54:14.001982+0000","last_fullsized":"2024-04-25T08:54:14.001982+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:54.860395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002707+0000","last_change":"2024-04-25T08:54:12.617092+0000","last_active":"2024-04-25T08:54:14.002707+0000","last_peered":"2024-04-25T08:54:14.002707+0000","last_clean":"2024-04-25T08:54:14.002707+0000","last_became_active":"2024-04-25T08:54:12.616799+0000","last_became_peered":"2024-04-25T08:54:12.616799+0000","last_unstale":"2024-04-25T08:54:14.002707+0000","last_undegraded":"2024-04-25T08:54:14.002707+0000","last_fullsized":"2024-04-25T08:54:14.002707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:21.068533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":170,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002257+0000","last_change":"2024-04-25T08:54:00.818044+0000","last_active":"2024-04-25T08:54:14.002257+0000","last_peered":"2024-04-25T08:54:14.002257+0000","last_clean":"2024-04-25T08:54:14.002257+0000","last_became_active":"2024-04-25T08:54:00.817098+0000","last_became_peered":"2024-04-25T08:54:00.817098+0000","last_unstale":"2024-04-25T08:54:14.002257+0000","last_undegraded":"2024-04-25T08:54:14.002257+0000","last_fullsized":"2024-04-25T08:54:14.002257+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_clean_scrub_stamp":"2024-04-25T08:53:59.739677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:48.129513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601017+0000","last_change":"2024-04-25T08:54:11.576740+0000","last_active":"2024-04-25T08:54:13.601017+0000","last_peered":"2024-04-25T08:54:13.601017+0000","last_clean":"2024-04-25T08:54:13.601017+0000","last_became_active":"2024-04-25T08:54:11.576444+0000","last_became_peered":"2024-04-25T08:54:11.576444+0000","last_unstale":"2024-04-25T08:54:13.601017+0000","last_undegraded":"2024-04-25T08:54:13.601017+0000","last_fullsized":"2024-04-25T08:54:13.601017+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:10.545906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.003145+0000","last_change":"2024-04-25T08:54:12.606225+0000","last_active":"2024-04-25T08:54:14.003145+0000","last_peered":"2024-04-25T08:54:14.003145+0000","last_clean":"2024-04-25T08:54:14.003145+0000","last_became_active":"2024-04-25T08:54:12.605897+0000","last_became_peered":"2024-04-25T08:54:12.605897+0000","last_unstale":"2024-04-25T08:54:14.003145+0000","last_undegraded":"2024-04-25T08:54:14.003145+0000","last_fullsized":"2024-04-25T08:54:14.003145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:57.230096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002094+0000","last_change":"2024-04-25T08:54:11.590032+0000","last_active":"2024-04-25T08:54:14.002094+0000","last_peered":"2024-04-25T08:54:14.002094+0000","last_clean":"2024-04-25T08:54:14.002094+0000","last_became_active":"2024-04-25T08:54:11.589701+0000","last_became_peered":"2024-04-25T08:54:11.589701+0000","last_unstale":"2024-04-25T08:54:14.002094+0000","last_undegraded":"2024-04-25T08:54:14.002094+0000","last_fullsized":"2024-04-25T08:54:14.002094+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:24:52.051575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.001941+0000","last_change":"2024-04-25T08:54:12.603873+0000","last_active":"2024-04-25T08:54:14.001941+0000","last_peered":"2024-04-25T08:54:14.001941+0000","last_clean":"2024-04-25T08:54:14.001941+0000","last_became_active":"2024-04-25T08:54:12.599347+0000","last_became_peered":"2024-04-25T08:54:12.599347+0000","last_unstale":"2024-04-25T08:54:14.001941+0000","last_undegraded":"2024-04-25T08:54:14.001941+0000","last_fullsized":"2024-04-25T08:54:14.001941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:14.023628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002327+0000","last_change":"2024-04-25T08:54:11.588495+0000","last_active":"2024-04-25T08:54:14.002327+0000","last_peered":"2024-04-25T08:54:14.002327+0000","last_clean":"2024-04-25T08:54:14.002327+0000","last_became_active":"2024-04-25T08:54:11.588141+0000","last_became_peered":"2024-04-25T08:54:11.588141+0000","last_unstale":"2024-04-25T08:54:14.002327+0000","last_undegraded":"2024-04-25T08:54:14.002327+0000","last_fullsized":"2024-04-25T08:54:14.002327+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:10.178198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.599547+0000","last_change":"2024-04-25T08:54:12.610027+0000","last_active":"2024-04-25T08:54:13.599547+0000","last_peered":"2024-04-25T08:54:13.599547+0000","last_clean":"2024-04-25T08:54:13.599547+0000","last_became_active":"2024-04-25T08:54:12.609091+0000","last_became_peered":"2024-04-25T08:54:12.609091+0000","last_unstale":"2024-04-25T08:54:13.599547+0000","last_undegraded":"2024-04-25T08:54:13.599547+0000","last_fullsized":"2024-04-25T08:54:13.599547+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:20.570657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.602128+0000","last_change":"2024-04-25T08:54:11.587597+0000","last_active":"2024-04-25T08:54:13.602128+0000","last_peered":"2024-04-25T08:54:13.602128+0000","last_clean":"2024-04-25T08:54:13.602128+0000","last_became_active":"2024-04-25T08:54:11.587078+0000","last_became_peered":"2024-04-25T08:54:11.587078+0000","last_unstale":"2024-04-25T08:54:13.602128+0000","last_undegraded":"2024-04-25T08:54:13.602128+0000","last_fullsized":"2024-04-25T08:54:13.602128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:50.460868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601656+0000","last_change":"2024-04-25T08:54:12.611656+0000","last_active":"2024-04-25T08:54:13.601656+0000","last_peered":"2024-04-25T08:54:13.601656+0000","last_clean":"2024-04-25T08:54:13.601656+0000","last_became_active":"2024-04-25T08:54:12.611359+0000","last_became_peered":"2024-04-25T08:54:12.611359+0000","last_unstale":"2024-04-25T08:54:13.601656+0000","last_undegraded":"2024-04-25T08:54:13.601656+0000","last_fullsized":"2024-04-25T08:54:13.601656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:33.374900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.599718+0000","last_change":"2024-04-25T08:54:11.589751+0000","last_active":"2024-04-25T08:54:13.599718+0000","last_peered":"2024-04-25T08:54:13.599718+0000","last_clean":"2024-04-25T08:54:13.599718+0000","last_became_active":"2024-04-25T08:54:11.589362+0000","last_became_peered":"2024-04-25T08:54:11.589362+0000","last_unstale":"2024-04-25T08:54:13.599718+0000","last_undegraded":"2024-04-25T08:54:13.599718+0000","last_fullsized":"2024-04-25T08:54:13.599718+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:10.178198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.602180+0000","last_change":"2024-04-25T08:54:12.606441+0000","last_active":"2024-04-25T08:54:13.602180+0000","last_peered":"2024-04-25T08:54:13.602180+0000","last_clean":"2024-04-25T08:54:13.602180+0000","last_became_active":"2024-04-25T08:54:12.605425+0000","last_became_peered":"2024-04-25T08:54:12.605425+0000","last_unstale":"2024-04-25T08:54:13.602180+0000","last_undegraded":"2024-04-25T08:54:13.602180+0000","last_fullsized":"2024-04-25T08:54:13.602180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:31.051483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002795+0000","last_change":"2024-04-25T08:54:11.591338+0000","last_active":"2024-04-25T08:54:14.002795+0000","last_peered":"2024-04-25T08:54:14.002795+0000","last_clean":"2024-04-25T08:54:14.002795+0000","last_became_active":"2024-04-25T08:54:11.591074+0000","last_became_peered":"2024-04-25T08:54:11.591074+0000","last_unstale":"2024-04-25T08:54:14.002795+0000","last_undegraded":"2024-04-25T08:54:14.002795+0000","last_fullsized":"2024-04-25T08:54:14.002795+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:40:32.370052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.599726+0000","last_change":"2024-04-25T08:54:12.602183+0000","last_active":"2024-04-25T08:54:13.599726+0000","last_peered":"2024-04-25T08:54:13.599726+0000","last_clean":"2024-04-25T08:54:13.599726+0000","last_became_active":"2024-04-25T08:54:12.601348+0000","last_became_peered":"2024-04-25T08:54:12.601348+0000","last_unstale":"2024-04-25T08:54:13.599726+0000","last_undegraded":"2024-04-25T08:54:13.599726+0000","last_fullsized":"2024-04-25T08:54:13.599726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:45.793417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.714307+0000","last_change":"2024-04-25T08:54:11.592183+0000","last_active":"2024-04-25T08:54:13.714307+0000","last_peered":"2024-04-25T08:54:13.714307+0000","last_clean":"2024-04-25T08:54:13.714307+0000","last_became_active":"2024-04-25T08:54:11.591795+0000","last_became_peered":"2024-04-25T08:54:11.591795+0000","last_unstale":"2024-04-25T08:54:13.714307+0000","last_undegraded":"2024-04-25T08:54:13.714307+0000","last_fullsized":"2024-04-25T08:54:13.714307+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:38.100250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.003444+0000","last_change":"2024-04-25T08:54:12.607792+0000","last_active":"2024-04-25T08:54:14.003444+0000","last_peered":"2024-04-25T08:54:14.003444+0000","last_clean":"2024-04-25T08:54:14.003444+0000","last_became_active":"2024-04-25T08:54:12.607459+0000","last_became_peered":"2024-04-25T08:54:12.607459+0000","last_unstale":"2024-04-25T08:54:14.003444+0000","last_undegraded":"2024-04-25T08:54:14.003444+0000","last_fullsized":"2024-04-25T08:54:14.003444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:17.430796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.003207+0000","last_change":"2024-04-25T08:54:11.591567+0000","last_active":"2024-04-25T08:54:14.003207+0000","last_peered":"2024-04-25T08:54:14.003207+0000","last_clean":"2024-04-25T08:54:14.003207+0000","last_became_active":"2024-04-25T08:54:11.591178+0000","last_became_peered":"2024-04-25T08:54:11.591178+0000","last_unstale":"2024-04-25T08:54:14.003207+0000","last_undegraded":"2024-04-25T08:54:14.003207+0000","last_fullsized":"2024-04-25T08:54:14.003207+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:50.460868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002719+0000","last_change":"2024-04-25T08:54:12.598577+0000","last_active":"2024-04-25T08:54:14.002719+0000","last_peered":"2024-04-25T08:54:14.002719+0000","last_clean":"2024-04-25T08:54:14.002719+0000","last_became_active":"2024-04-25T08:54:12.597319+0000","last_became_peered":"2024-04-25T08:54:12.597319+0000","last_unstale":"2024-04-25T08:54:14.002719+0000","last_undegraded":"2024-04-25T08:54:14.002719+0000","last_fullsized":"2024-04-25T08:54:14.002719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:57.230096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002090+0000","last_change":"2024-04-25T08:54:11.591476+0000","last_active":"2024-04-25T08:54:14.002090+0000","last_peered":"2024-04-25T08:54:14.002090+0000","last_clean":"2024-04-25T08:54:14.002090+0000","last_became_active":"2024-04-25T08:54:11.591000+0000","last_became_peered":"2024-04-25T08:54:11.591000+0000","last_unstale":"2024-04-25T08:54:14.002090+0000","last_undegraded":"2024-04-25T08:54:14.002090+0000","last_fullsized":"2024-04-25T08:54:14.002090+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:38.100250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601522+0000","last_change":"2024-04-25T08:54:12.602036+0000","last_active":"2024-04-25T08:54:13.601522+0000","last_peered":"2024-04-25T08:54:13.601522+0000","last_clean":"2024-04-25T08:54:13.601522+0000","last_became_active":"2024-04-25T08:54:12.601768+0000","last_became_peered":"2024-04-25T08:54:12.601768+0000","last_unstale":"2024-04-25T08:54:13.601522+0000","last_undegraded":"2024-04-25T08:54:13.601522+0000","last_fullsized":"2024-04-25T08:54:13.601522+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:51.465717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.599648+0000","last_change":"2024-04-25T08:54:11.585465+0000","last_active":"2024-04-25T08:54:13.599648+0000","last_peered":"2024-04-25T08:54:13.599648+0000","last_clean":"2024-04-25T08:54:13.599648+0000","last_became_active":"2024-04-25T08:54:11.585184+0000","last_became_peered":"2024-04-25T08:54:11.585184+0000","last_unstale":"2024-04-25T08:54:13.599648+0000","last_undegraded":"2024-04-25T08:54:13.599648+0000","last_fullsized":"2024-04-25T08:54:13.599648+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:37.241134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002288+0000","last_change":"2024-04-25T08:54:12.603295+0000","last_active":"2024-04-25T08:54:14.002288+0000","last_peered":"2024-04-25T08:54:14.002288+0000","last_clean":"2024-04-25T08:54:14.002288+0000","last_became_active":"2024-04-25T08:54:12.602398+0000","last_became_peered":"2024-04-25T08:54:12.602398+0000","last_unstale":"2024-04-25T08:54:14.002288+0000","last_undegraded":"2024-04-25T08:54:14.002288+0000","last_fullsized":"2024-04-25T08:54:14.002288+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:56.990780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.599690+0000","last_change":"2024-04-25T08:54:11.588662+0000","last_active":"2024-04-25T08:54:13.599690+0000","last_peered":"2024-04-25T08:54:13.599690+0000","last_clean":"2024-04-25T08:54:13.599690+0000","last_became_active":"2024-04-25T08:54:11.588350+0000","last_became_peered":"2024-04-25T08:54:11.588350+0000","last_unstale":"2024-04-25T08:54:13.599690+0000","last_undegraded":"2024-04-25T08:54:13.599690+0000","last_fullsized":"2024-04-25T08:54:13.599690+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:45.942159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.003164+0000","last_change":"2024-04-25T08:54:12.612845+0000","last_active":"2024-04-25T08:54:14.003164+0000","last_peered":"2024-04-25T08:54:14.003164+0000","last_clean":"2024-04-25T08:54:14.003164+0000","last_became_active":"2024-04-25T08:54:12.612569+0000","last_became_peered":"2024-04-25T08:54:12.612569+0000","last_unstale":"2024-04-25T08:54:14.003164+0000","last_undegraded":"2024-04-25T08:54:14.003164+0000","last_fullsized":"2024-04-25T08:54:14.003164+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:45.793417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002442+0000","last_change":"2024-04-25T08:54:11.589407+0000","last_active":"2024-04-25T08:54:14.002442+0000","last_peered":"2024-04-25T08:54:14.002442+0000","last_clean":"2024-04-25T08:54:14.002442+0000","last_became_active":"2024-04-25T08:54:11.588226+0000","last_became_peered":"2024-04-25T08:54:11.588226+0000","last_unstale":"2024-04-25T08:54:14.002442+0000","last_undegraded":"2024-04-25T08:54:14.002442+0000","last_fullsized":"2024-04-25T08:54:14.002442+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:42.726856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002789+0000","last_change":"2024-04-25T08:54:12.609741+0000","last_active":"2024-04-25T08:54:14.002789+0000","last_peered":"2024-04-25T08:54:14.002789+0000","last_clean":"2024-04-25T08:54:14.002789+0000","last_became_active":"2024-04-25T08:54:12.609075+0000","last_became_peered":"2024-04-25T08:54:12.609075+0000","last_unstale":"2024-04-25T08:54:14.002789+0000","last_undegraded":"2024-04-25T08:54:14.002789+0000","last_fullsized":"2024-04-25T08:54:14.002789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:20.570657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.602159+0000","last_change":"2024-04-25T08:54:12.603440+0000","last_active":"2024-04-25T08:54:13.602159+0000","last_peered":"2024-04-25T08:54:13.602159+0000","last_clean":"2024-04-25T08:54:13.602159+0000","last_became_active":"2024-04-25T08:54:12.603193+0000","last_became_peered":"2024-04-25T08:54:12.603193+0000","last_unstale":"2024-04-25T08:54:13.602159+0000","last_undegraded":"2024-04-25T08:54:13.602159+0000","last_fullsized":"2024-04-25T08:54:13.602159+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:44.251355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002297+0000","last_change":"2024-04-25T08:54:11.585161+0000","last_active":"2024-04-25T08:54:14.002297+0000","last_peered":"2024-04-25T08:54:14.002297+0000","last_clean":"2024-04-25T08:54:14.002297+0000","last_became_active":"2024-04-25T08:54:11.584842+0000","last_became_peered":"2024-04-25T08:54:11.584842+0000","last_unstale":"2024-04-25T08:54:14.002297+0000","last_undegraded":"2024-04-25T08:54:14.002297+0000","last_fullsized":"2024-04-25T08:54:14.002297+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:45.942159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:13.601990+0000","last_change":"2024-04-25T08:54:12.611429+0000","last_active":"2024-04-25T08:54:13.601990+0000","last_peered":"2024-04-25T08:54:13.601990+0000","last_clean":"2024-04-25T08:54:13.601990+0000","last_became_active":"2024-04-25T08:54:12.611170+0000","last_became_peered":"2024-04-25T08:54:12.611170+0000","last_unstale":"2024-04-25T08:54:13.601990+0000","last_undegraded":"2024-04-25T08:54:13.601990+0000","last_fullsized":"2024-04-25T08:54:13.601990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:21.068533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002636+0000","last_change":"2024-04-25T08:54:11.589779+0000","last_active":"2024-04-25T08:54:14.002636+0000","last_peered":"2024-04-25T08:54:14.002636+0000","last_clean":"2024-04-25T08:54:14.002636+0000","last_became_active":"2024-04-25T08:54:11.589502+0000","last_became_peered":"2024-04-25T08:54:11.589502+0000","last_unstale":"2024-04-25T08:54:14.002636+0000","last_undegraded":"2024-04-25T08:54:14.002636+0000","last_fullsized":"2024-04-25T08:54:14.002636+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:50.460868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002428+0000","last_change":"2024-04-25T08:54:12.610745+0000","last_active":"2024-04-25T08:54:14.002428+0000","last_peered":"2024-04-25T08:54:14.002428+0000","last_clean":"2024-04-25T08:54:14.002428+0000","last_became_active":"2024-04-25T08:54:12.610366+0000","last_became_peered":"2024-04-25T08:54:12.610366+0000","last_unstale":"2024-04-25T08:54:14.002428+0000","last_undegraded":"2024-04-25T08:54:14.002428+0000","last_fullsized":"2024-04-25T08:54:14.002428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:54.538580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.003385+0000","last_change":"2024-04-25T08:54:11.586361+0000","last_active":"2024-04-25T08:54:14.003385+0000","last_peered":"2024-04-25T08:54:14.003385+0000","last_clean":"2024-04-25T08:54:14.003385+0000","last_became_active":"2024-04-25T08:54:11.586056+0000","last_became_peered":"2024-04-25T08:54:11.586056+0000","last_unstale":"2024-04-25T08:54:14.003385+0000","last_undegraded":"2024-04-25T08:54:14.003385+0000","last_fullsized":"2024-04-25T08:54:14.003385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:51:12.212662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.003357+0000","last_change":"2024-04-25T08:54:12.607913+0000","last_active":"2024-04-25T08:54:14.003357+0000","last_peered":"2024-04-25T08:54:14.003357+0000","last_clean":"2024-04-25T08:54:14.003357+0000","last_became_active":"2024-04-25T08:54:12.607623+0000","last_became_peered":"2024-04-25T08:54:12.607623+0000","last_unstale":"2024-04-25T08:54:14.003357+0000","last_undegraded":"2024-04-25T08:54:14.003357+0000","last_fullsized":"2024-04-25T08:54:14.003357+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:11.566270+0000","last_clean_scrub_stamp":"2024-04-25T08:54:11.566270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:03.021953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-04-25T08:54:14.002643+0000","last_change":"2024-04-25T08:54:11.591877+0000","last_active":"2024-04-25T08:54:14.002643+0000","last_peered":"2024-04-25T08:54:14.002643+0000","last_clean":"2024-04-25T08:54:14.002643+0000","last_became_active":"2024-04-25T08:54:11.591562+0000","last_became_peered":"2024-04-25T08:54:11.591562+0000","last_unstale":"2024-04-25T08:54:14.002643+0000","last_undegraded":"2024-04-25T08:54:14.002643+0000","last_fullsized":"2024-04-25T08:54:14.002643+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:54:10.561422+0000","last_clean_scrub_stamp":"2024-04-25T08:54:10.561422+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:30:54.860395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31396,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340444,"statfs":{"total":96636764160,"available":96604614656,"internally_reserved":0,"allocated":217088,"data_stored":72546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31384,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340456,"statfs":{"total":96636764160,"available":96604626944,"internally_reserved":0,"allocated":204800,"data_stored":70934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":212992,"data_stored":71094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31976,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339864,"statfs":{"total":96636764160,"available":96604020736,"internally_reserved":0,"allocated":811008,"data_stored":661940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:54:20.357 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T08:54:20.358 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T08:54:20.358 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T08:54:20.358 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T08:54:20.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.503+0000 7fd822d77640 1 Processor -- start 2024-04-25T08:54:20.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.503+0000 7fd822d77640 1 -- start start 2024-04-25T08:54:20.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.503+0000 7fd822d77640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd81c15df50 0x7fd81c1619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:20.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.503+0000 7fd822d77640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd81c161f30 0x7fd81c162370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:20.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.503+0000 7fd822d77640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fd81c0c7cf0 0x7fd81c0c8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:20.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.503+0000 7fd822d77640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd81c15e330 con 0x7fd81c161f30 2024-04-25T08:54:20.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.503+0000 7fd822d77640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd81c0d24a0 con 0x7fd81c0c7cf0 2024-04-25T08:54:20.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.503+0000 7fd822d77640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c0d2740 con 0x7fd81c15df50 2024-04-25T08:54:20.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.504+0000 7fd81bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd81c161f30 0x7fd81c162370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:20.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.504+0000 7fd81bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd81c161f30 0x7fd81c162370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55934/0 (socket says 172.21.15.43:55934) 2024-04-25T08:54:20.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.504+0000 7fd81bfff640 1 -- 172.21.15.43:0/3532725974 learned_addr learned my addr 172.21.15.43:0/3532725974 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:20.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.504+0000 7fd820b13640 1 --2- 172.21.15.43:0/3532725974 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd81c15df50 0x7fd81c1619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:20.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.504+0000 7fd821314640 1 --2- 172.21.15.43:0/3532725974 >> v2:172.21.15.193:3300/0 conn(0x7fd81c0c7cf0 0x7fd81c0c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:20.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.504+0000 7fd81bfff640 1 -- 172.21.15.43:0/3532725974 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd81c15df50 msgr2=0x7fd81c1619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:20.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.504+0000 7fd81bfff640 1 --2- 172.21.15.43:0/3532725974 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd81c15df50 0x7fd81c1619f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:20.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.504+0000 7fd81bfff640 1 -- 172.21.15.43:0/3532725974 >> v2:172.21.15.193:3300/0 conn(0x7fd81c0c7cf0 msgr2=0x7fd81c0c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:20.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.504+0000 7fd81bfff640 1 --2- 172.21.15.43:0/3532725974 >> v2:172.21.15.193:3300/0 conn(0x7fd81c0c7cf0 0x7fd81c0c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:20.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.504+0000 7fd81bfff640 1 -- 172.21.15.43:0/3532725974 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd81c0d2a20 con 0x7fd81c161f30 2024-04-25T08:54:20.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.505+0000 7fd81bfff640 1 --2- 172.21.15.43:0/3532725974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd81c161f30 0x7fd81c162370 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd81c073000 tx=0x7fd80802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=808d949d9bfb5a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:20.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.505+0000 7fd81b7fe640 1 -- 172.21.15.43:0/3532725974 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd808037070 con 0x7fd81c161f30 2024-04-25T08:54:20.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.505+0000 7fd81b7fe640 1 -- 172.21.15.43:0/3532725974 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd808032440 con 0x7fd81c161f30 2024-04-25T08:54:20.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.505+0000 7fd81b7fe640 1 -- 172.21.15.43:0/3532725974 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd808032810 con 0x7fd81c161f30 2024-04-25T08:54:20.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.505+0000 7fd822d77640 1 -- 172.21.15.43:0/3532725974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd81c161f30 msgr2=0x7fd81c162370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:20.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.505+0000 7fd822d77640 1 --2- 172.21.15.43:0/3532725974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd81c161f30 0x7fd81c162370 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd81c073000 tx=0x7fd80802a1e0 comp rx=0 tx=0).stop 2024-04-25T08:54:20.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.505+0000 7fd822d77640 1 -- 172.21.15.43:0/3532725974 shutdown_connections 2024-04-25T08:54:20.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.505+0000 7fd822d77640 1 --2- 172.21.15.43:0/3532725974 >> v2:172.21.15.193:3300/0 conn(0x7fd81c0c7cf0 0x7fd81c0c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:20.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.505+0000 7fd822d77640 1 --2- 172.21.15.43:0/3532725974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd81c161f30 0x7fd81c162370 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:20.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.505+0000 7fd822d77640 1 --2- 172.21.15.43:0/3532725974 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd81c15df50 0x7fd81c1619f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:20.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.505+0000 7fd822d77640 1 -- 172.21.15.43:0/3532725974 >> 172.21.15.43:0/3532725974 conn(0x7fd81c0d3a30 msgr2=0x7fd81c0d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:20.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.505+0000 7fd822d77640 1 -- 172.21.15.43:0/3532725974 shutdown_connections 2024-04-25T08:54:20.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.505+0000 7fd822d77640 1 -- 172.21.15.43:0/3532725974 wait complete. 2024-04-25T08:54:20.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.507+0000 7fd822d77640 1 Processor -- start 2024-04-25T08:54:20.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.507+0000 7fd822d77640 1 -- start start 2024-04-25T08:54:20.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.507+0000 7fd822d77640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd81c0c7cf0 0x7fd81c154fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:20.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.507+0000 7fd822d77640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd81c15df50 0x7fd81c0ce620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:20.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.507+0000 7fd822d77640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd81c161f30 0x7fd81c0ceb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:20.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.507+0000 7fd822d77640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c0d2190 con 0x7fd81c161f30 2024-04-25T08:54:20.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.507+0000 7fd822d77640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c0c77d0 con 0x7fd81c15df50 2024-04-25T08:54:20.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.507+0000 7fd822d77640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c0c8640 con 0x7fd81c0c7cf0 2024-04-25T08:54:20.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.508+0000 7fd821314640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd81c161f30 0x7fd81c0ceb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:20.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.508+0000 7fd821314640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd81c161f30 0x7fd81c0ceb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55942/0 (socket says 172.21.15.43:55942) 2024-04-25T08:54:20.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.508+0000 7fd821314640 1 -- 172.21.15.43:0/1693529844 learned_addr learned my addr 172.21.15.43:0/1693529844 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:20.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.508+0000 7fd820b13640 1 --2- 172.21.15.43:0/1693529844 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd81c0c7cf0 0x7fd81c154fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:20.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.508+0000 7fd81bfff640 1 --2- 172.21.15.43:0/1693529844 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd81c15df50 0x7fd81c0ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:20.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.508+0000 7fd821314640 1 -- 172.21.15.43:0/1693529844 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd81c0c7cf0 msgr2=0x7fd81c154fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:20.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.508+0000 7fd821314640 1 --2- 172.21.15.43:0/1693529844 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd81c0c7cf0 0x7fd81c154fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:20.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.508+0000 7fd821314640 1 -- 172.21.15.43:0/1693529844 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd81c15df50 msgr2=0x7fd81c0ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:20.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.508+0000 7fd821314640 1 --2- 172.21.15.43:0/1693529844 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd81c15df50 0x7fd81c0ce620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:20.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.508+0000 7fd821314640 1 -- 172.21.15.43:0/1693529844 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd81c155520 con 0x7fd81c161f30 2024-04-25T08:54:20.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.508+0000 7fd821314640 1 --2- 172.21.15.43:0/1693529844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd81c161f30 0x7fd81c0ceb60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd81400eed0 tx=0x7fd81400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:20.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.508+0000 7fd8197fa640 1 -- 172.21.15.43:0/1693529844 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd814018020 con 0x7fd81c161f30 2024-04-25T08:54:20.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.508+0000 7fd8197fa640 1 -- 172.21.15.43:0/1693529844 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd814014400 con 0x7fd81c161f30 2024-04-25T08:54:20.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.508+0000 7fd822d77640 1 -- 172.21.15.43:0/1693529844 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd81c15f6e0 con 0x7fd81c161f30 2024-04-25T08:54:20.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.508+0000 7fd8197fa640 1 -- 172.21.15.43:0/1693529844 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8140147f0 con 0x7fd81c161f30 2024-04-25T08:54:20.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.508+0000 7fd822d77640 1 -- 172.21.15.43:0/1693529844 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd81c15fca0 con 0x7fd81c161f30 2024-04-25T08:54:20.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.509+0000 7fd822d77640 1 -- 172.21.15.43:0/1693529844 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd81c0680f0 con 0x7fd81c161f30 2024-04-25T08:54:20.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.510+0000 7fd8197fa640 1 -- 172.21.15.43:0/1693529844 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd814005d10 con 0x7fd81c161f30 2024-04-25T08:54:20.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.510+0000 7fd8197fa640 1 --2- 172.21.15.43:0/1693529844 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd80006b4d0 0x7fd80006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:20.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.510+0000 7fd8197fa640 1 -- 172.21.15.43:0/1693529844 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd81408aa00 con 0x7fd81c161f30 2024-04-25T08:54:20.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.510+0000 7fd820b13640 1 --2- 172.21.15.43:0/1693529844 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd80006b4d0 0x7fd80006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:20.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.511+0000 7fd820b13640 1 --2- 172.21.15.43:0/1693529844 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd80006b4d0 0x7fd80006d990 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd81c156c10 tx=0x7fd810009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:20.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.514+0000 7fd8197fa640 1 -- 172.21.15.43:0/1693529844 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd814022030 con 0x7fd81c161f30 2024-04-25T08:54:20.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.735+0000 7fd822d77640 1 -- 172.21.15.43:0/1693529844 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd81c0ca400 con 0x7fd81c161f30 2024-04-25T08:54:20.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.735+0000 7fd8197fa640 1 -- 172.21.15.43:0/1693529844 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fd81c155520 con 0x7fd81c161f30 2024-04-25T08:54:20.726 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:54:20.726 INFO:teuthology.orchestra.run.smithi043.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T08:54:20.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.737+0000 7fd822d77640 1 -- 172.21.15.43:0/1693529844 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd80006b4d0 msgr2=0x7fd80006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:20.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.737+0000 7fd822d77640 1 --2- 172.21.15.43:0/1693529844 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd80006b4d0 0x7fd80006d990 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd81c156c10 tx=0x7fd810009190 comp rx=0 tx=0).stop 2024-04-25T08:54:20.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.737+0000 7fd822d77640 1 -- 172.21.15.43:0/1693529844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd81c161f30 msgr2=0x7fd81c0ceb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:20.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.737+0000 7fd822d77640 1 --2- 172.21.15.43:0/1693529844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd81c161f30 0x7fd81c0ceb60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd81400eed0 tx=0x7fd81400b480 comp rx=0 tx=0).stop 2024-04-25T08:54:20.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.737+0000 7fd822d77640 1 -- 172.21.15.43:0/1693529844 shutdown_connections 2024-04-25T08:54:20.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.737+0000 7fd822d77640 1 --2- 172.21.15.43:0/1693529844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd81c161f30 0x7fd81c0ceb60 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:20.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.737+0000 7fd822d77640 1 --2- 172.21.15.43:0/1693529844 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd80006b4d0 0x7fd80006d990 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:20.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.738+0000 7fd822d77640 1 --2- 172.21.15.43:0/1693529844 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd81c15df50 0x7fd81c0ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:20.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.738+0000 7fd822d77640 1 --2- 172.21.15.43:0/1693529844 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd81c0c7cf0 0x7fd81c154fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:20.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.738+0000 7fd822d77640 1 -- 172.21.15.43:0/1693529844 >> 172.21.15.43:0/1693529844 conn(0x7fd81c0d3a30 msgr2=0x7fd81c073c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:20.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.738+0000 7fd822d77640 1 -- 172.21.15.43:0/1693529844 shutdown_connections 2024-04-25T08:54:20.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.738+0000 7fd822d77640 1 -- 172.21.15.43:0/1693529844 wait complete. 2024-04-25T08:54:20.738 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T08:54:20.739 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:54:20.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.849+0000 7f0fdd05d640 1 Processor -- start 2024-04-25T08:54:20.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.849+0000 7f0fdd05d640 1 -- start start 2024-04-25T08:54:20.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.850+0000 7f0fdd05d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0fd80b2920 0x7f0fd806c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:20.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.850+0000 7f0fdd05d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fd806caa0 con 0x7f0fd81614c0 2024-04-25T08:54:20.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.850+0000 7f0fdd05d640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fd80cc8c0 con 0x7f0fd80ce3b0 2024-04-25T08:54:20.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.850+0000 7f0fdd05d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fd80ccf10 con 0x7f0fd80b2920 2024-04-25T08:54:20.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.850+0000 7f0fd759e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0fd80b2920 0x7f0fd806c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:20.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.850+0000 7f0fd6d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f0fd81614c0 0x7f0fd806bc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:38192/0 (socket says 172.21.15.43:38192) 2024-04-25T08:54:20.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.850+0000 7f0fd6d9d640 1 -- 172.21.15.43:0/3804587018 learned_addr learned my addr 172.21.15.43:0/3804587018 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:20.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.850+0000 7f0fd759e640 1 -- 172.21.15.43:0/3804587018 >> v1:172.21.15.193:6789/0 conn(0x7f0fd80ce3b0 legacy=0x7f0fd80cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:20.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.851+0000 7f0fd759e640 1 -- 172.21.15.43:0/3804587018 >> v1:172.21.15.43:6789/0 conn(0x7f0fd81614c0 legacy=0x7f0fd806bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:20.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.851+0000 7f0fd759e640 1 -- 172.21.15.43:0/3804587018 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd80ccba0 con 0x7f0fd80b2920 2024-04-25T08:54:20.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.851+0000 7f0fd5d9b640 1 -- 172.21.15.43:0/3804587018 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2883436807 0 0) 0x7f0fd806caa0 con 0x7f0fd81614c0 2024-04-25T08:54:20.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.851+0000 7f0fd759e640 1 --2- 172.21.15.43:0/3804587018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0fd80b2920 0x7f0fd806c350 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8009d90 tx=0x7f0fc802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9543e370e304f651 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:20.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.851+0000 7f0fd5d9b640 1 -- 172.21.15.43:0/3804587018 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc8031600 con 0x7f0fd80b2920 2024-04-25T08:54:20.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.851+0000 7f0fd5d9b640 1 -- 172.21.15.43:0/3804587018 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc80317a0 con 0x7f0fd80b2920 2024-04-25T08:54:20.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.851+0000 7f0fd5d9b640 1 -- 172.21.15.43:0/3804587018 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc8031b70 con 0x7f0fd80b2920 2024-04-25T08:54:20.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.852+0000 7f0fdd05d640 1 -- 172.21.15.43:0/3804587018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0fd80b2920 msgr2=0x7f0fd806c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:20.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.852+0000 7f0fdd05d640 1 --2- 172.21.15.43:0/3804587018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0fd80b2920 0x7f0fd806c350 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0fc8009d90 tx=0x7f0fc802a610 comp rx=0 tx=0).stop 2024-04-25T08:54:20.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.852+0000 7f0fdd05d640 1 -- 172.21.15.43:0/3804587018 shutdown_connections 2024-04-25T08:54:20.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.852+0000 7f0fdd05d640 1 --2- 172.21.15.43:0/3804587018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0fd80b2920 0x7f0fd806c350 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:20.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.852+0000 7f0fdd05d640 1 -- 172.21.15.43:0/3804587018 >> 172.21.15.43:0/3804587018 conn(0x7f0fd80b32c0 msgr2=0x7f0fd80b36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:20.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.852+0000 7f0fdd05d640 1 -- 172.21.15.43:0/3804587018 shutdown_connections 2024-04-25T08:54:20.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.852+0000 7f0fdd05d640 1 -- 172.21.15.43:0/3804587018 wait complete. 2024-04-25T08:54:20.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.853+0000 7f0fdd05d640 1 Processor -- start 2024-04-25T08:54:20.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.853+0000 7f0fdd05d640 1 -- start start 2024-04-25T08:54:20.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.854+0000 7f0fdd05d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0fd80b2920 0x7f0fd80c6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:20.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.854+0000 7f0fdd05d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fd80ce3b0 0x7f0fd80c6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:20.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.854+0000 7f0fdd05d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0fd81614c0 0x7f0fd8150630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:20.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.854+0000 7f0fdd05d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd80d0290 con 0x7f0fd81614c0 2024-04-25T08:54:20.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.854+0000 7f0fdd05d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd80d0110 con 0x7f0fd80ce3b0 2024-04-25T08:54:20.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.854+0000 7f0fdd05d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd80d0410 con 0x7f0fd80b2920 2024-04-25T08:54:20.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.854+0000 7f0fd759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0fd81614c0 0x7f0fd8150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:20.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.854+0000 7f0fd759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0fd81614c0 0x7f0fd8150630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55966/0 (socket says 172.21.15.43:55966) 2024-04-25T08:54:20.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.854+0000 7f0fd759e640 1 -- 172.21.15.43:0/1047256518 learned_addr learned my addr 172.21.15.43:0/1047256518 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:20.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.854+0000 7f0fd659c640 1 --2- 172.21.15.43:0/1047256518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fd80ce3b0 0x7f0fd80c6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:20.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.855+0000 7f0fd6d9d640 1 --2- 172.21.15.43:0/1047256518 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0fd80b2920 0x7f0fd80c6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:20.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.855+0000 7f0fd759e640 1 -- 172.21.15.43:0/1047256518 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0fd80b2920 msgr2=0x7f0fd80c6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:20.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.855+0000 7f0fd759e640 1 --2- 172.21.15.43:0/1047256518 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0fd80b2920 0x7f0fd80c6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:20.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.855+0000 7f0fd759e640 1 -- 172.21.15.43:0/1047256518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fd80ce3b0 msgr2=0x7f0fd80c6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:20.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.855+0000 7f0fd759e640 1 --2- 172.21.15.43:0/1047256518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fd80ce3b0 0x7f0fd80c6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:20.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.855+0000 7f0fd759e640 1 -- 172.21.15.43:0/1047256518 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd8154920 con 0x7f0fd81614c0 2024-04-25T08:54:20.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.855+0000 7f0fd759e640 1 --2- 172.21.15.43:0/1047256518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0fd81614c0 0x7f0fd8150630 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0fc802ac00 tx=0x7f0fc8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:20.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.855+0000 7f0fcf7fe640 1 -- 172.21.15.43:0/1047256518 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc8040070 con 0x7f0fd81614c0 2024-04-25T08:54:20.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.855+0000 7f0fcf7fe640 1 -- 172.21.15.43:0/1047256518 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fc8030b60 con 0x7f0fd81614c0 2024-04-25T08:54:20.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.855+0000 7f0fdd05d640 1 -- 172.21.15.43:0/1047256518 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fd80c6b90 con 0x7f0fd81614c0 2024-04-25T08:54:20.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.855+0000 7f0fcf7fe640 1 -- 172.21.15.43:0/1047256518 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fc8045460 con 0x7f0fd81614c0 2024-04-25T08:54:20.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.855+0000 7f0fdd05d640 1 -- 172.21.15.43:0/1047256518 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fd814e760 con 0x7f0fd81614c0 2024-04-25T08:54:20.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.856+0000 7f0fdd05d640 1 -- 172.21.15.43:0/1047256518 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fd8074ef0 con 0x7f0fd81614c0 2024-04-25T08:54:20.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.857+0000 7f0fcf7fe640 1 -- 172.21.15.43:0/1047256518 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0fc8039040 con 0x7f0fd81614c0 2024-04-25T08:54:20.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.857+0000 7f0fcf7fe640 1 --2- 172.21.15.43:0/1047256518 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0fb806b4d0 0x7f0fb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:20.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.857+0000 7f0fcf7fe640 1 -- 172.21.15.43:0/1047256518 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0fc80aba50 con 0x7f0fd81614c0 2024-04-25T08:54:20.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.857+0000 7f0fd6d9d640 1 --2- 172.21.15.43:0/1047256518 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0fb806b4d0 0x7f0fb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:20.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.858+0000 7f0fd6d9d640 1 --2- 172.21.15.43:0/1047256518 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0fb806b4d0 0x7f0fb806d990 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0000f60 tx=0x7f0fd0007c00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:20.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:20.861+0000 7f0fcf7fe640 1 -- 172.21.15.43:0/1047256518 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0fc807a990 con 0x7f0fd81614c0 2024-04-25T08:54:21.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.041+0000 7f0fdd05d640 1 -- 172.21.15.43:0/1047256518 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0fd80c70b0 con 0x7f0fd81614c0 2024-04-25T08:54:21.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.042+0000 7f0fcf7fe640 1 -- 172.21.15.43:0/1047256518 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16569 (secure 0 0 0) 0x7f0fc807a290 con 0x7f0fd81614c0 2024-04-25T08:54:21.032 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:54:21.033 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":20,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:54:13.593188+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:54:09.757978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:54:10.926008+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:54:21.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.044+0000 7f0fdd05d640 1 -- 172.21.15.43:0/1047256518 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0fb806b4d0 msgr2=0x7f0fb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:21.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.044+0000 7f0fdd05d640 1 --2- 172.21.15.43:0/1047256518 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0fb806b4d0 0x7f0fb806d990 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0000f60 tx=0x7f0fd0007c00 comp rx=0 tx=0).stop 2024-04-25T08:54:21.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.044+0000 7f0fdd05d640 1 -- 172.21.15.43:0/1047256518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0fd81614c0 msgr2=0x7f0fd8150630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:21.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.044+0000 7f0fdd05d640 1 --2- 172.21.15.43:0/1047256518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0fd81614c0 0x7f0fd8150630 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0fc802ac00 tx=0x7f0fc8004a40 comp rx=0 tx=0).stop 2024-04-25T08:54:21.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.044+0000 7f0fdd05d640 1 -- 172.21.15.43:0/1047256518 shutdown_connections 2024-04-25T08:54:21.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.044+0000 7f0fdd05d640 1 --2- 172.21.15.43:0/1047256518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0fd81614c0 0x7f0fd8150630 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:21.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.044+0000 7f0fdd05d640 1 --2- 172.21.15.43:0/1047256518 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0fb806b4d0 0x7f0fb806d990 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:21.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.044+0000 7f0fdd05d640 1 --2- 172.21.15.43:0/1047256518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fd80ce3b0 0x7f0fd80c6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:21.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.044+0000 7f0fdd05d640 1 --2- 172.21.15.43:0/1047256518 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0fd80b2920 0x7f0fd80c6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:21.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.044+0000 7f0fdd05d640 1 -- 172.21.15.43:0/1047256518 >> 172.21.15.43:0/1047256518 conn(0x7f0fd80b32c0 msgr2=0x7f0fd806e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:21.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.044+0000 7f0fdd05d640 1 -- 172.21.15.43:0/1047256518 shutdown_connections 2024-04-25T08:54:21.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.044+0000 7f0fdd05d640 1 -- 172.21.15.43:0/1047256518 wait complete. 2024-04-25T08:54:21.046 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:54:09.757978+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:54:10.926008+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:54:21.046 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:54:21.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.159+0000 7f32b7789640 1 Processor -- start 2024-04-25T08:54:21.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.160+0000 7f32b7789640 1 -- start start 2024-04-25T08:54:21.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.160+0000 7f32b7789640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f32b015e080 0x7f32b006be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:21.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.160+0000 7f32b7789640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f32b006c650 0x7f32b006ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:21.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.160+0000 7f32b7789640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32b00d16e0 con 0x7f32b006ec90 2024-04-25T08:54:21.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.160+0000 7f32b7789640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32b006cfd0 con 0x7f32b015e080 2024-04-25T08:54:21.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.160+0000 7f32b7789640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b00d0640 con 0x7f32b006c650 2024-04-25T08:54:21.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.161+0000 7f32b4d24640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f32b006c650 0x7f32b006ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:21.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.161+0000 7f32b5525640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f32b015e080 0x7f32b006be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:21.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.161+0000 7f32b4d24640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f32b006c650 0x7f32b006ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:42756/0 (socket says 172.21.15.43:42756) 2024-04-25T08:54:21.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.161+0000 7f32b4d24640 1 -- 172.21.15.43:0/1062333902 learned_addr learned my addr 172.21.15.43:0/1062333902 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:21.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.161+0000 7f32b4d24640 1 -- 172.21.15.43:0/1062333902 >> v2:172.21.15.193:3300/0 conn(0x7f32b015e080 msgr2=0x7f32b006be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:21.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.161+0000 7f32b4d24640 1 --2- 172.21.15.43:0/1062333902 >> v2:172.21.15.193:3300/0 conn(0x7f32b015e080 0x7f32b006be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:21.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.161+0000 7f32b4d24640 1 -- 172.21.15.43:0/1062333902 >> v1:172.21.15.193:6789/0 conn(0x7f32b006ec90 legacy=0x7f32b006f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:21.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.161+0000 7f32b4d24640 1 -- 172.21.15.43:0/1062333902 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b00d0920 con 0x7f32b006c650 2024-04-25T08:54:21.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.161+0000 7f32b5525640 1 --2- 172.21.15.43:0/1062333902 >> v2:172.21.15.193:3300/0 conn(0x7f32b015e080 0x7f32b006be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:54:21.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.162+0000 7f32b4d24640 1 --2- 172.21.15.43:0/1062333902 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f32b006c650 0x7f32b006ca90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f32a4009870 tx=0x7f32a402a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=182d37f520b527eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:21.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.162+0000 7f32affff640 1 -- 172.21.15.43:0/1062333902 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a4037070 con 0x7f32b006c650 2024-04-25T08:54:21.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.162+0000 7f32affff640 1 -- 172.21.15.43:0/1062333902 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a402ae50 con 0x7f32b006c650 2024-04-25T08:54:21.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.162+0000 7f32affff640 1 -- 172.21.15.43:0/1062333902 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a4032860 con 0x7f32b006c650 2024-04-25T08:54:21.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.162+0000 7f32b7789640 1 -- 172.21.15.43:0/1062333902 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f32b006c650 msgr2=0x7f32b006ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:21.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.162+0000 7f32b7789640 1 --2- 172.21.15.43:0/1062333902 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f32b006c650 0x7f32b006ca90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f32a4009870 tx=0x7f32a402a300 comp rx=0 tx=0).stop 2024-04-25T08:54:21.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.162+0000 7f32b7789640 1 -- 172.21.15.43:0/1062333902 shutdown_connections 2024-04-25T08:54:21.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.162+0000 7f32b7789640 1 --2- 172.21.15.43:0/1062333902 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f32b006c650 0x7f32b006ca90 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:21.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.162+0000 7f32b7789640 1 --2- 172.21.15.43:0/1062333902 >> v2:172.21.15.193:3300/0 conn(0x7f32b015e080 0x7f32b006be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:21.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.163+0000 7f32b7789640 1 -- 172.21.15.43:0/1062333902 >> 172.21.15.43:0/1062333902 conn(0x7f32b00b34d0 msgr2=0x7f32b00b38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:21.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.163+0000 7f32b7789640 1 -- 172.21.15.43:0/1062333902 shutdown_connections 2024-04-25T08:54:21.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.163+0000 7f32b7789640 1 -- 172.21.15.43:0/1062333902 wait complete. 2024-04-25T08:54:21.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.164+0000 7f32b7789640 1 Processor -- start 2024-04-25T08:54:21.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.164+0000 7f32b7789640 1 -- start start 2024-04-25T08:54:21.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.164+0000 7f32b7789640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32b006c650 0x7f32b014cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:21.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.164+0000 7f32b7789640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32b006ec90 0x7f32b00b4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:21.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.164+0000 7f32b7789640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f32b015e080 0x7f32b01f7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:21.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.164+0000 7f32b7789640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b00d11c0 con 0x7f32b006ec90 2024-04-25T08:54:21.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.164+0000 7f32b7789640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b00d0ee0 con 0x7f32b006c650 2024-04-25T08:54:21.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.164+0000 7f32b7789640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b00af7c0 con 0x7f32b015e080 2024-04-25T08:54:21.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.165+0000 7f32b4d24640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32b006ec90 0x7f32b00b4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:21.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.165+0000 7f32b4d24640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32b006ec90 0x7f32b00b4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55968/0 (socket says 172.21.15.43:55968) 2024-04-25T08:54:21.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.165+0000 7f32b4d24640 1 -- 172.21.15.43:0/3840143607 learned_addr learned my addr 172.21.15.43:0/3840143607 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:21.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.165+0000 7f32b5525640 1 --2- 172.21.15.43:0/3840143607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32b006c650 0x7f32b014cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:21.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.165+0000 7f32b5d26640 1 --2- 172.21.15.43:0/3840143607 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f32b015e080 0x7f32b01f7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:21.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.165+0000 7f32b4d24640 1 -- 172.21.15.43:0/3840143607 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f32b015e080 msgr2=0x7f32b01f7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:21.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.165+0000 7f32b4d24640 1 --2- 172.21.15.43:0/3840143607 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f32b015e080 0x7f32b01f7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:21.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.165+0000 7f32b4d24640 1 -- 172.21.15.43:0/3840143607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32b006c650 msgr2=0x7f32b014cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:21.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.165+0000 7f32b4d24640 1 --2- 172.21.15.43:0/3840143607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32b006c650 0x7f32b014cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:21.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.165+0000 7f32b4d24640 1 -- 172.21.15.43:0/3840143607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b009d450 con 0x7f32b006ec90 2024-04-25T08:54:21.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.165+0000 7f32b4d24640 1 --2- 172.21.15.43:0/3840143607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32b006ec90 0x7f32b00b4a80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f32a4009870 tx=0x7f32a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:21.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.165+0000 7f32adffb640 1 -- 172.21.15.43:0/3840143607 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a4042070 con 0x7f32b006ec90 2024-04-25T08:54:21.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.165+0000 7f32adffb640 1 -- 172.21.15.43:0/3840143607 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a403e660 con 0x7f32b006ec90 2024-04-25T08:54:21.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.165+0000 7f32b7789640 1 -- 172.21.15.43:0/3840143607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32b014d490 con 0x7f32b006ec90 2024-04-25T08:54:21.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.165+0000 7f32adffb640 1 -- 172.21.15.43:0/3840143607 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a4037040 con 0x7f32b006ec90 2024-04-25T08:54:21.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.165+0000 7f32b7789640 1 -- 172.21.15.43:0/3840143607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b00b5540 con 0x7f32b006ec90 2024-04-25T08:54:21.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.167+0000 7f32b7789640 1 -- 172.21.15.43:0/3840143607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32b0074f60 con 0x7f32b006ec90 2024-04-25T08:54:21.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.167+0000 7f32adffb640 1 -- 172.21.15.43:0/3840143607 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f32a4045070 con 0x7f32b006ec90 2024-04-25T08:54:21.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.167+0000 7f32adffb640 1 --2- 172.21.15.43:0/3840143607 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f329c06b4d0 0x7f329c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:21.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.167+0000 7f32adffb640 1 -- 172.21.15.43:0/3840143607 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f32a40aa8a0 con 0x7f32b006ec90 2024-04-25T08:54:21.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.168+0000 7f32b5525640 1 --2- 172.21.15.43:0/3840143607 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f329c06b4d0 0x7f329c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:21.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.168+0000 7f32b5525640 1 --2- 172.21.15.43:0/3840143607 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f329c06b4d0 0x7f329c06d990 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f32a8009bd0 tx=0x7f32a800b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:21.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.172+0000 7f32adffb640 1 -- 172.21.15.43:0/3840143607 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f32b009d450 con 0x7f32b006ec90 2024-04-25T08:54:21.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.350+0000 7f32b7789640 1 -- 172.21.15.43:0/3840143607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f32b006c430 con 0x7f32b006ec90 2024-04-25T08:54:21.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.350+0000 7f32adffb640 1 -- 172.21.15.43:0/3840143607 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f32a4079160 con 0x7f32b006ec90 2024-04-25T08:54:21.340 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T08:54:21.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.352+0000 7f32b7789640 1 -- 172.21.15.43:0/3840143607 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f329c06b4d0 msgr2=0x7f329c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:21.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.352+0000 7f32b7789640 1 --2- 172.21.15.43:0/3840143607 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f329c06b4d0 0x7f329c06d990 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f32a8009bd0 tx=0x7f32a800b040 comp rx=0 tx=0).stop 2024-04-25T08:54:21.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.352+0000 7f32b7789640 1 -- 172.21.15.43:0/3840143607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32b006ec90 msgr2=0x7f32b00b4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:21.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.352+0000 7f32b7789640 1 --2- 172.21.15.43:0/3840143607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32b006ec90 0x7f32b00b4a80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f32a4009870 tx=0x7f32a40046c0 comp rx=0 tx=0).stop 2024-04-25T08:54:21.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.352+0000 7f32b7789640 1 -- 172.21.15.43:0/3840143607 shutdown_connections 2024-04-25T08:54:21.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.352+0000 7f32b7789640 1 --2- 172.21.15.43:0/3840143607 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f32b015e080 0x7f32b01f7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:21.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.352+0000 7f32b7789640 1 --2- 172.21.15.43:0/3840143607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32b006ec90 0x7f32b00b4a80 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:21.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.352+0000 7f32b7789640 1 --2- 172.21.15.43:0/3840143607 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f329c06b4d0 0x7f329c06d990 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:21.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.352+0000 7f32b7789640 1 --2- 172.21.15.43:0/3840143607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32b006c650 0x7f32b014cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:21.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.352+0000 7f32b7789640 1 -- 172.21.15.43:0/3840143607 >> 172.21.15.43:0/3840143607 conn(0x7f32b00b34d0 msgr2=0x7f32b00b04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:21.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.352+0000 7f32b7789640 1 -- 172.21.15.43:0/3840143607 shutdown_connections 2024-04-25T08:54:21.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.352+0000 7f32b7789640 1 -- 172.21.15.43:0/3840143607 wait complete. 2024-04-25T08:54:21.353 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:54:21.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.466+0000 7f82ef9f5640 1 Processor -- start 2024-04-25T08:54:21.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.467+0000 7f82ef9f5640 1 -- start start 2024-04-25T08:54:21.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.467+0000 7f82ef9f5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f82e80b5560 0x7f82e80b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:21.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.467+0000 7f82ef9f5640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f82e80d2230 0x7f82e80d26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:21.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.467+0000 7f82ef9f5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82e8162610 con 0x7f82e815e090 2024-04-25T08:54:21.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.467+0000 7f82ef9f5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82e80d1da0 con 0x7f82e80b5560 2024-04-25T08:54:21.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.467+0000 7f82ef9f5640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82e80d1750 con 0x7f82e80d2230 2024-04-25T08:54:21.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.467+0000 7f82ecf90640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f82e80b5560 0x7f82e80b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:21.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.467+0000 7f82ed791640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f82e815e090 0x7f82e8161aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:38196/0 (socket says 172.21.15.43:38196) 2024-04-25T08:54:21.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.467+0000 7f82ecf90640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f82e80b5560 0x7f82e80b59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55982/0 (socket says 172.21.15.43:55982) 2024-04-25T08:54:21.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.467+0000 7f82ed791640 1 -- 172.21.15.43:0/3676508884 learned_addr learned my addr 172.21.15.43:0/3676508884 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:21.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.468+0000 7f82edf92640 1 --2- 172.21.15.43:0/3676508884 >> v2:172.21.15.193:3300/0 conn(0x7f82e80d2230 0x7f82e80d26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:21.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.468+0000 7f82ecf90640 1 -- 172.21.15.43:0/3676508884 >> v2:172.21.15.193:3300/0 conn(0x7f82e80d2230 msgr2=0x7f82e80d26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:21.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.468+0000 7f82ecf90640 1 --2- 172.21.15.43:0/3676508884 >> v2:172.21.15.193:3300/0 conn(0x7f82e80d2230 0x7f82e80d26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:21.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.468+0000 7f82ecf90640 1 -- 172.21.15.43:0/3676508884 >> v1:172.21.15.43:6789/0 conn(0x7f82e815e090 legacy=0x7f82e8161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:21.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.468+0000 7f82ecf90640 1 -- 172.21.15.43:0/3676508884 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e80d1a30 con 0x7f82e80b5560 2024-04-25T08:54:21.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.468+0000 7f82e7fff640 1 -- 172.21.15.43:0/3676508884 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4266232864 0 0) 0x7f82e8162610 con 0x7f82e815e090 2024-04-25T08:54:21.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.468+0000 7f82ecf90640 1 --2- 172.21.15.43:0/3676508884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82e80b5560 0x7f82e80b59e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f82e80b24d0 tx=0x7f82dc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eeb75d04d5b1752 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:21.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.469+0000 7f82e7fff640 1 -- 172.21.15.43:0/3676508884 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82dc037070 con 0x7f82e80b5560 2024-04-25T08:54:21.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.469+0000 7f82e7fff640 1 -- 172.21.15.43:0/3676508884 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82dc032440 con 0x7f82e80b5560 2024-04-25T08:54:21.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.469+0000 7f82e7fff640 1 -- 172.21.15.43:0/3676508884 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82dc032810 con 0x7f82e80b5560 2024-04-25T08:54:21.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.469+0000 7f82ef9f5640 1 -- 172.21.15.43:0/3676508884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82e80b5560 msgr2=0x7f82e80b59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:21.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.469+0000 7f82ef9f5640 1 --2- 172.21.15.43:0/3676508884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82e80b5560 0x7f82e80b59e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f82e80b24d0 tx=0x7f82dc02a230 comp rx=0 tx=0).stop 2024-04-25T08:54:21.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.469+0000 7f82ef9f5640 1 -- 172.21.15.43:0/3676508884 shutdown_connections 2024-04-25T08:54:21.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.469+0000 7f82ef9f5640 1 --2- 172.21.15.43:0/3676508884 >> v2:172.21.15.193:3300/0 conn(0x7f82e80d2230 0x7f82e80d26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:21.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.469+0000 7f82ef9f5640 1 --2- 172.21.15.43:0/3676508884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82e80b5560 0x7f82e80b59e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:21.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.469+0000 7f82ef9f5640 1 -- 172.21.15.43:0/3676508884 >> 172.21.15.43:0/3676508884 conn(0x7f82e80d0560 msgr2=0x7f82e80d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:21.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.469+0000 7f82ef9f5640 1 -- 172.21.15.43:0/3676508884 shutdown_connections 2024-04-25T08:54:21.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.469+0000 7f82ef9f5640 1 -- 172.21.15.43:0/3676508884 wait complete. 2024-04-25T08:54:21.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.470+0000 7f82ef9f5640 1 Processor -- start 2024-04-25T08:54:21.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.470+0000 7f82ef9f5640 1 -- start start 2024-04-25T08:54:21.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.471+0000 7f82ef9f5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82e80b5560 0x7f82e81f4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:21.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.471+0000 7f82ef9f5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82e80d2230 0x7f82e80c9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:21.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.471+0000 7f82ef9f5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82e815e090 0x7f82e81f1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:21.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.471+0000 7f82ef9f5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e81621b0 con 0x7f82e815e090 2024-04-25T08:54:21.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.471+0000 7f82ef9f5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e80b3280 con 0x7f82e80b5560 2024-04-25T08:54:21.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.471+0000 7f82ef9f5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82e8162330 con 0x7f82e80d2230 2024-04-25T08:54:21.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.471+0000 7f82edf92640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82e815e090 0x7f82e81f1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:21.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.471+0000 7f82edf92640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82e815e090 0x7f82e81f1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55998/0 (socket says 172.21.15.43:55998) 2024-04-25T08:54:21.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.471+0000 7f82edf92640 1 -- 172.21.15.43:0/2870136378 learned_addr learned my addr 172.21.15.43:0/2870136378 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:21.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.471+0000 7f82ecf90640 1 --2- 172.21.15.43:0/2870136378 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82e80d2230 0x7f82e80c9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:21.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.471+0000 7f82ed791640 1 --2- 172.21.15.43:0/2870136378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82e80b5560 0x7f82e81f4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:21.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.472+0000 7f82edf92640 1 -- 172.21.15.43:0/2870136378 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82e80d2230 msgr2=0x7f82e80c9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:21.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.472+0000 7f82edf92640 1 --2- 172.21.15.43:0/2870136378 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82e80d2230 0x7f82e80c9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:21.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.472+0000 7f82edf92640 1 -- 172.21.15.43:0/2870136378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82e80b5560 msgr2=0x7f82e81f4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:21.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.472+0000 7f82edf92640 1 --2- 172.21.15.43:0/2870136378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82e80b5560 0x7f82e81f4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:21.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.472+0000 7f82edf92640 1 -- 172.21.15.43:0/2870136378 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e80c9fe0 con 0x7f82e815e090 2024-04-25T08:54:21.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.472+0000 7f82ecf90640 1 --2- 172.21.15.43:0/2870136378 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82e80d2230 0x7f82e80c9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:21.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.472+0000 7f82edf92640 1 --2- 172.21.15.43:0/2870136378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82e815e090 0x7f82e81f1810 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f82d8007e40 tx=0x7f82d800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:21.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.472+0000 7f82e5ffb640 1 -- 172.21.15.43:0/2870136378 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d8014620 con 0x7f82e815e090 2024-04-25T08:54:21.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.472+0000 7f82e5ffb640 1 -- 172.21.15.43:0/2870136378 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82d8019070 con 0x7f82e815e090 2024-04-25T08:54:21.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.472+0000 7f82ef9f5640 1 -- 172.21.15.43:0/2870136378 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82e81f5400 con 0x7f82e815e090 2024-04-25T08:54:21.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.472+0000 7f82e5ffb640 1 -- 172.21.15.43:0/2870136378 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d8014a20 con 0x7f82e815e090 2024-04-25T08:54:21.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.472+0000 7f82ef9f5640 1 -- 172.21.15.43:0/2870136378 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82e81f59c0 con 0x7f82e815e090 2024-04-25T08:54:21.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.473+0000 7f82ef9f5640 1 -- 172.21.15.43:0/2870136378 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82e806bf00 con 0x7f82e815e090 2024-04-25T08:54:21.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.473+0000 7f82e5ffb640 1 -- 172.21.15.43:0/2870136378 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f82d8014bc0 con 0x7f82e815e090 2024-04-25T08:54:21.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.474+0000 7f82e5ffb640 1 --2- 172.21.15.43:0/2870136378 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f82d406b4d0 0x7f82d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:21.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.474+0000 7f82e5ffb640 1 -- 172.21.15.43:0/2870136378 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82d808aab0 con 0x7f82e815e090 2024-04-25T08:54:21.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.474+0000 7f82ed791640 1 --2- 172.21.15.43:0/2870136378 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f82d406b4d0 0x7f82d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:21.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.474+0000 7f82ed791640 1 --2- 172.21.15.43:0/2870136378 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f82d406b4d0 0x7f82d406d990 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f82e0000f60 tx=0x7f82e0007c00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:21.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.478+0000 7f82e5ffb640 1 -- 172.21.15.43:0/2870136378 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f82e806bf00 con 0x7f82e815e090 2024-04-25T08:54:21.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.659+0000 7f82ef9f5640 1 -- 172.21.15.43:0/2870136378 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f82e8074f60 con 0x7f82e815e090 2024-04-25T08:54:21.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.659+0000 7f82e5ffb640 1 -- 172.21.15.43:0/2870136378 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f82d80599c0 con 0x7f82e815e090 2024-04-25T08:54:21.649 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T08:54:21.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.661+0000 7f82ef9f5640 1 -- 172.21.15.43:0/2870136378 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f82d406b4d0 msgr2=0x7f82d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:21.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.661+0000 7f82ef9f5640 1 --2- 172.21.15.43:0/2870136378 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f82d406b4d0 0x7f82d406d990 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f82e0000f60 tx=0x7f82e0007c00 comp rx=0 tx=0).stop 2024-04-25T08:54:21.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.661+0000 7f82ef9f5640 1 -- 172.21.15.43:0/2870136378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82e815e090 msgr2=0x7f82e81f1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:21.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.661+0000 7f82ef9f5640 1 --2- 172.21.15.43:0/2870136378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82e815e090 0x7f82e81f1810 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f82d8007e40 tx=0x7f82d800e4b0 comp rx=0 tx=0).stop 2024-04-25T08:54:21.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.661+0000 7f82ef9f5640 1 -- 172.21.15.43:0/2870136378 shutdown_connections 2024-04-25T08:54:21.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.661+0000 7f82ef9f5640 1 --2- 172.21.15.43:0/2870136378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82e815e090 0x7f82e81f1810 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:21.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.661+0000 7f82ef9f5640 1 --2- 172.21.15.43:0/2870136378 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82e80d2230 0x7f82e80c9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:21.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.661+0000 7f82ef9f5640 1 --2- 172.21.15.43:0/2870136378 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f82d406b4d0 0x7f82d406d990 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:21.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.661+0000 7f82ef9f5640 1 --2- 172.21.15.43:0/2870136378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82e80b5560 0x7f82e81f4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:21.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.661+0000 7f82ef9f5640 1 -- 172.21.15.43:0/2870136378 >> 172.21.15.43:0/2870136378 conn(0x7f82e80d0560 msgr2=0x7f82e80cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:21.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.661+0000 7f82ef9f5640 1 -- 172.21.15.43:0/2870136378 shutdown_connections 2024-04-25T08:54:21.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.661+0000 7f82ef9f5640 1 -- 172.21.15.43:0/2870136378 wait complete. 2024-04-25T08:54:21.662 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:54:21.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.775+0000 7fb9b7607640 1 Processor -- start 2024-04-25T08:54:21.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.775+0000 7fb9b7607640 1 -- start start 2024-04-25T08:54:21.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.776+0000 7fb9b7607640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb9b00b6110 0x7fb9b00d1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:21.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.776+0000 7fb9b7607640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb9b00d2340 0x7fb9b01656d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:21.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.776+0000 7fb9b7607640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9b00b32a0 con 0x7fb9b00d1a20 2024-04-25T08:54:21.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.776+0000 7fb9b7607640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9b00cf430 con 0x7fb9b00b6110 2024-04-25T08:54:21.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.776+0000 7fb9b7607640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9b015bf90 con 0x7fb9b00d2340 2024-04-25T08:54:21.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.776+0000 7fb9b4ba2640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb9b00b6110 0x7fb9b00d1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:21.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.776+0000 7fb9b4ba2640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb9b00b6110 0x7fb9b00d1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56006/0 (socket says 172.21.15.43:56006) 2024-04-25T08:54:21.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.776+0000 7fb9b4ba2640 1 -- 172.21.15.43:0/4212682580 learned_addr learned my addr 172.21.15.43:0/4212682580 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:21.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.776+0000 7fb9b5ba4640 1 --2- 172.21.15.43:0/4212682580 >> v2:172.21.15.193:3300/0 conn(0x7fb9b00d2340 0x7fb9b01656d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:21.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.776+0000 7fb9b4ba2640 1 -- 172.21.15.43:0/4212682580 >> v2:172.21.15.193:3300/0 conn(0x7fb9b00d2340 msgr2=0x7fb9b01656d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:21.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.776+0000 7fb9b4ba2640 1 --2- 172.21.15.43:0/4212682580 >> v2:172.21.15.193:3300/0 conn(0x7fb9b00d2340 0x7fb9b01656d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:21.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.776+0000 7fb9b4ba2640 1 -- 172.21.15.43:0/4212682580 >> v1:172.21.15.193:6789/0 conn(0x7fb9b00d1a20 legacy=0x7fb9b00b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:21.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.776+0000 7fb9b4ba2640 1 -- 172.21.15.43:0/4212682580 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b00cee70 con 0x7fb9b00b6110 2024-04-25T08:54:21.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.777+0000 7fb9b4ba2640 1 --2- 172.21.15.43:0/4212682580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b00b6110 0x7fb9b00d1e00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb9b00b65a0 tx=0x7fb9a402a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1edb21128d44a9cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:21.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.777+0000 7fb9affff640 1 -- 172.21.15.43:0/4212682580 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4037070 con 0x7fb9b00b6110 2024-04-25T08:54:21.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.777+0000 7fb9affff640 1 -- 172.21.15.43:0/4212682580 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9a4032440 con 0x7fb9b00b6110 2024-04-25T08:54:21.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.777+0000 7fb9affff640 1 -- 172.21.15.43:0/4212682580 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a4032810 con 0x7fb9b00b6110 2024-04-25T08:54:21.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.778+0000 7fb9b7607640 1 -- 172.21.15.43:0/4212682580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b00b6110 msgr2=0x7fb9b00d1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:21.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.778+0000 7fb9b7607640 1 --2- 172.21.15.43:0/4212682580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b00b6110 0x7fb9b00d1e00 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb9b00b65a0 tx=0x7fb9a402a150 comp rx=0 tx=0).stop 2024-04-25T08:54:21.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.778+0000 7fb9b7607640 1 -- 172.21.15.43:0/4212682580 shutdown_connections 2024-04-25T08:54:21.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.778+0000 7fb9b7607640 1 --2- 172.21.15.43:0/4212682580 >> v2:172.21.15.193:3300/0 conn(0x7fb9b00d2340 0x7fb9b01656d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:21.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.778+0000 7fb9b7607640 1 --2- 172.21.15.43:0/4212682580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b00b6110 0x7fb9b00d1e00 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:21.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.778+0000 7fb9b7607640 1 -- 172.21.15.43:0/4212682580 >> 172.21.15.43:0/4212682580 conn(0x7fb9b0072aa0 msgr2=0x7fb9b009aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:21.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.778+0000 7fb9b7607640 1 -- 172.21.15.43:0/4212682580 shutdown_connections 2024-04-25T08:54:21.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.778+0000 7fb9b7607640 1 -- 172.21.15.43:0/4212682580 wait complete. 2024-04-25T08:54:21.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.779+0000 7fb9b7607640 1 Processor -- start 2024-04-25T08:54:21.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.779+0000 7fb9b7607640 1 -- start start 2024-04-25T08:54:21.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.780+0000 7fb9b7607640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9b00b6110 0x7fb9b0143350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:21.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.780+0000 7fb9b7607640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9b00d1a20 0x7fb9b0156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:21.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.780+0000 7fb9b7607640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b00d2340 0x7fb9b0156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:21.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.780+0000 7fb9b7607640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b00b4be0 con 0x7fb9b00d2340 2024-04-25T08:54:21.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.780+0000 7fb9b7607640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b00b4900 con 0x7fb9b00d1a20 2024-04-25T08:54:21.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.780+0000 7fb9b7607640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b00cea60 con 0x7fb9b00b6110 2024-04-25T08:54:21.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.780+0000 7fb9b5ba4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b00d2340 0x7fb9b0156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:21.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.780+0000 7fb9b5ba4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b00d2340 0x7fb9b0156b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56022/0 (socket says 172.21.15.43:56022) 2024-04-25T08:54:21.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.780+0000 7fb9b5ba4640 1 -- 172.21.15.43:0/3934671102 learned_addr learned my addr 172.21.15.43:0/3934671102 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:21.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.780+0000 7fb9b53a3640 1 --2- 172.21.15.43:0/3934671102 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9b00b6110 0x7fb9b0143350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:21.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.780+0000 7fb9b4ba2640 1 --2- 172.21.15.43:0/3934671102 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9b00d1a20 0x7fb9b0156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:21.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.780+0000 7fb9b5ba4640 1 -- 172.21.15.43:0/3934671102 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9b00b6110 msgr2=0x7fb9b0143350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:21.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.780+0000 7fb9b5ba4640 1 --2- 172.21.15.43:0/3934671102 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9b00b6110 0x7fb9b0143350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:21.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.780+0000 7fb9b5ba4640 1 -- 172.21.15.43:0/3934671102 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9b00d1a20 msgr2=0x7fb9b0156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:21.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.780+0000 7fb9b5ba4640 1 --2- 172.21.15.43:0/3934671102 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9b00d1a20 0x7fb9b0156650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:21.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.780+0000 7fb9b5ba4640 1 -- 172.21.15.43:0/3934671102 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b01584b0 con 0x7fb9b00d2340 2024-04-25T08:54:21.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.780+0000 7fb9b5ba4640 1 --2- 172.21.15.43:0/3934671102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b00d2340 0x7fb9b0156b90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0007d80 tx=0x7fb9a000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:21.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.781+0000 7fb9adffb640 1 -- 172.21.15.43:0/3934671102 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a0014420 con 0x7fb9b00d2340 2024-04-25T08:54:21.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.781+0000 7fb9adffb640 1 -- 172.21.15.43:0/3934671102 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9a0019070 con 0x7fb9b00d2340 2024-04-25T08:54:21.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.781+0000 7fb9b7607640 1 -- 172.21.15.43:0/3934671102 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9b0157410 con 0x7fb9b00d2340 2024-04-25T08:54:21.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.781+0000 7fb9adffb640 1 -- 172.21.15.43:0/3934671102 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9a0014800 con 0x7fb9b00d2340 2024-04-25T08:54:21.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.781+0000 7fb9b7607640 1 -- 172.21.15.43:0/3934671102 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9b01579d0 con 0x7fb9b00d2340 2024-04-25T08:54:21.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.782+0000 7fb9b7607640 1 -- 172.21.15.43:0/3934671102 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9b0074f60 con 0x7fb9b00d2340 2024-04-25T08:54:21.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.782+0000 7fb9adffb640 1 -- 172.21.15.43:0/3934671102 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb9a00149a0 con 0x7fb9b00d2340 2024-04-25T08:54:21.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.782+0000 7fb9adffb640 1 --2- 172.21.15.43:0/3934671102 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb99c06b4d0 0x7fb99c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:21.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.783+0000 7fb9adffb640 1 -- 172.21.15.43:0/3934671102 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb9a008a8e0 con 0x7fb9b00d2340 2024-04-25T08:54:21.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.783+0000 7fb9b53a3640 1 --2- 172.21.15.43:0/3934671102 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb99c06b4d0 0x7fb99c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:21.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.783+0000 7fb9b53a3640 1 --2- 172.21.15.43:0/3934671102 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb99c06b4d0 0x7fb99c06d990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8000f60 tx=0x7fb9a8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:21.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.787+0000 7fb9adffb640 1 -- 172.21.15.43:0/3934671102 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb9a0059110 con 0x7fb9b00d2340 2024-04-25T08:54:21.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.966+0000 7fb9b7607640 1 -- 172.21.15.43:0/3934671102 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb9b0152a20 con 0x7fb9b00d2340 2024-04-25T08:54:21.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.967+0000 7fb9adffb640 1 -- 172.21.15.43:0/3934671102 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb9a005d1b0 con 0x7fb9b00d2340 2024-04-25T08:54:21.957 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T08:54:21.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.968+0000 7fb9b7607640 1 -- 172.21.15.43:0/3934671102 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb99c06b4d0 msgr2=0x7fb99c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:21.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.968+0000 7fb9b7607640 1 --2- 172.21.15.43:0/3934671102 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb99c06b4d0 0x7fb99c06d990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb9a8000f60 tx=0x7fb9a8009040 comp rx=0 tx=0).stop 2024-04-25T08:54:21.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.968+0000 7fb9b7607640 1 -- 172.21.15.43:0/3934671102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b00d2340 msgr2=0x7fb9b0156b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:21.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.968+0000 7fb9b7607640 1 --2- 172.21.15.43:0/3934671102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b00d2340 0x7fb9b0156b90 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb9a0007d80 tx=0x7fb9a000e4b0 comp rx=0 tx=0).stop 2024-04-25T08:54:21.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.969+0000 7fb9b7607640 1 -- 172.21.15.43:0/3934671102 shutdown_connections 2024-04-25T08:54:21.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.969+0000 7fb9b7607640 1 --2- 172.21.15.43:0/3934671102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9b00d2340 0x7fb9b0156b90 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:21.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.969+0000 7fb9b7607640 1 --2- 172.21.15.43:0/3934671102 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb99c06b4d0 0x7fb99c06d990 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:21.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.969+0000 7fb9b7607640 1 --2- 172.21.15.43:0/3934671102 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9b00d1a20 0x7fb9b0156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:21.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.969+0000 7fb9b7607640 1 --2- 172.21.15.43:0/3934671102 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9b00b6110 0x7fb9b0143350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:21.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.969+0000 7fb9b7607640 1 -- 172.21.15.43:0/3934671102 >> 172.21.15.43:0/3934671102 conn(0x7fb9b0072aa0 msgr2=0x7fb9b006eef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:21.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.969+0000 7fb9b7607640 1 -- 172.21.15.43:0/3934671102 shutdown_connections 2024-04-25T08:54:21.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:21.969+0000 7fb9b7607640 1 -- 172.21.15.43:0/3934671102 wait complete. 2024-04-25T08:54:21.969 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:54:22.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.080+0000 7ff4bd528640 1 Processor -- start 2024-04-25T08:54:22.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.080+0000 7ff4bd528640 1 -- start start 2024-04-25T08:54:22.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.080+0000 7ff4bd528640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff4b815df50 0x7ff4b81619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:22.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.081+0000 7ff4bd528640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4b80cf670 0x7ff4b8162380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:22.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.081+0000 7ff4bd528640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b80d24a0 con 0x7ff4b8161f30 2024-04-25T08:54:22.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.081+0000 7ff4bd528640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b815e330 con 0x7ff4b815df50 2024-04-25T08:54:22.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.081+0000 7ff4bd528640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b80d2740 con 0x7ff4b80cf670 2024-04-25T08:54:22.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.081+0000 7ff4b77fe640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4b80cf670 0x7ff4b8162380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:22.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.081+0000 7ff4b77fe640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4b80cf670 0x7ff4b8162380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:42790/0 (socket says 172.21.15.43:42790) 2024-04-25T08:54:22.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.081+0000 7ff4affff640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7ff4b8161f30 0x7ff4b80c7cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:58400/0 (socket says 172.21.15.43:58400) 2024-04-25T08:54:22.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.081+0000 7ff4b77fe640 1 -- 172.21.15.43:0/22631742 learned_addr learned my addr 172.21.15.43:0/22631742 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:22.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.081+0000 7ff4b6ffd640 1 --2- 172.21.15.43:0/22631742 >> v2:172.21.15.193:3300/0 conn(0x7ff4b815df50 0x7ff4b81619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:22.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.081+0000 7ff4b67fc640 1 -- 172.21.15.43:0/22631742 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 615012945 0 0) 0x7ff4b80d24a0 con 0x7ff4b8161f30 2024-04-25T08:54:22.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.081+0000 7ff4b77fe640 1 -- 172.21.15.43:0/22631742 >> v2:172.21.15.193:3300/0 conn(0x7ff4b815df50 msgr2=0x7ff4b81619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:22.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.081+0000 7ff4b77fe640 1 --2- 172.21.15.43:0/22631742 >> v2:172.21.15.193:3300/0 conn(0x7ff4b815df50 0x7ff4b81619f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:22.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.081+0000 7ff4b77fe640 1 -- 172.21.15.43:0/22631742 >> v1:172.21.15.193:6789/0 conn(0x7ff4b8161f30 legacy=0x7ff4b80c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:22.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.081+0000 7ff4b77fe640 1 -- 172.21.15.43:0/22631742 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b80d2a20 con 0x7ff4b80cf670 2024-04-25T08:54:22.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.082+0000 7ff4b77fe640 1 --2- 172.21.15.43:0/22631742 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4b80cf670 0x7ff4b8162380 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0009e50 tx=0x7ff4b002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80968fc1b717e1e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:22.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.082+0000 7ff4b67fc640 1 -- 172.21.15.43:0/22631742 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b0036070 con 0x7ff4b80cf670 2024-04-25T08:54:22.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.082+0000 7ff4b67fc640 1 -- 172.21.15.43:0/22631742 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4b0031600 con 0x7ff4b80cf670 2024-04-25T08:54:22.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.082+0000 7ff4b67fc640 1 -- 172.21.15.43:0/22631742 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b00319d0 con 0x7ff4b80cf670 2024-04-25T08:54:22.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.083+0000 7ff4bd528640 1 -- 172.21.15.43:0/22631742 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4b80cf670 msgr2=0x7ff4b8162380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:22.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.083+0000 7ff4bd528640 1 --2- 172.21.15.43:0/22631742 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4b80cf670 0x7ff4b8162380 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff4b0009e50 tx=0x7ff4b002a710 comp rx=0 tx=0).stop 2024-04-25T08:54:22.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.083+0000 7ff4bd528640 1 -- 172.21.15.43:0/22631742 shutdown_connections 2024-04-25T08:54:22.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.083+0000 7ff4bd528640 1 --2- 172.21.15.43:0/22631742 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4b80cf670 0x7ff4b8162380 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:22.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.083+0000 7ff4bd528640 1 --2- 172.21.15.43:0/22631742 >> v2:172.21.15.193:3300/0 conn(0x7ff4b815df50 0x7ff4b81619f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:22.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.083+0000 7ff4bd528640 1 -- 172.21.15.43:0/22631742 >> 172.21.15.43:0/22631742 conn(0x7ff4b80d3a30 msgr2=0x7ff4b80d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:22.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.083+0000 7ff4bd528640 1 -- 172.21.15.43:0/22631742 shutdown_connections 2024-04-25T08:54:22.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.083+0000 7ff4bd528640 1 -- 172.21.15.43:0/22631742 wait complete. 2024-04-25T08:54:22.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.084+0000 7ff4bd528640 1 Processor -- start 2024-04-25T08:54:22.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.084+0000 7ff4bd528640 1 -- start start 2024-04-25T08:54:22.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.085+0000 7ff4bd528640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4b80cf670 0x7ff4b8160520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:22.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.085+0000 7ff4bd528640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4b815df50 0x7ff4b81f7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:22.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.085+0000 7ff4bd528640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4b8161f30 0x7ff4b80caa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:22.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.085+0000 7ff4bd528640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b80d2190 con 0x7ff4b815df50 2024-04-25T08:54:22.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.085+0000 7ff4bd528640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b80c77d0 con 0x7ff4b8161f30 2024-04-25T08:54:22.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.085+0000 7ff4bd528640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b80c8400 con 0x7ff4b80cf670 2024-04-25T08:54:22.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.085+0000 7ff4affff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4b815df50 0x7ff4b81f7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:22.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.085+0000 7ff4affff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4b815df50 0x7ff4b81f7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56028/0 (socket says 172.21.15.43:56028) 2024-04-25T08:54:22.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.085+0000 7ff4affff640 1 -- 172.21.15.43:0/2507333685 learned_addr learned my addr 172.21.15.43:0/2507333685 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:22.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.085+0000 7ff4b6ffd640 1 --2- 172.21.15.43:0/2507333685 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4b80cf670 0x7ff4b8160520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:22.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.085+0000 7ff4b77fe640 1 --2- 172.21.15.43:0/2507333685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4b8161f30 0x7ff4b80caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:22.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.085+0000 7ff4b6ffd640 1 -- 172.21.15.43:0/2507333685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4b8161f30 msgr2=0x7ff4b80caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:22.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.085+0000 7ff4b6ffd640 1 --2- 172.21.15.43:0/2507333685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4b8161f30 0x7ff4b80caa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:22.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.085+0000 7ff4b6ffd640 1 -- 172.21.15.43:0/2507333685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4b815df50 msgr2=0x7ff4b81f7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:22.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.085+0000 7ff4b6ffd640 1 --2- 172.21.15.43:0/2507333685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4b815df50 0x7ff4b81f7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:22.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.085+0000 7ff4b6ffd640 1 -- 172.21.15.43:0/2507333685 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b81f4570 con 0x7ff4b80cf670 2024-04-25T08:54:22.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.085+0000 7ff4affff640 1 --2- 172.21.15.43:0/2507333685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4b815df50 0x7ff4b81f7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:54:22.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.085+0000 7ff4b6ffd640 1 --2- 172.21.15.43:0/2507333685 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4b80cf670 0x7ff4b8160520 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff4a800e850 tx=0x7ff4a800ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:22.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.086+0000 7ff4af7fe640 1 -- 172.21.15.43:0/2507333685 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a8017070 con 0x7ff4b80cf670 2024-04-25T08:54:22.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.086+0000 7ff4af7fe640 1 -- 172.21.15.43:0/2507333685 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a800c940 con 0x7ff4b80cf670 2024-04-25T08:54:22.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.086+0000 7ff4af7fe640 1 -- 172.21.15.43:0/2507333685 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a800cd10 con 0x7ff4b80cf670 2024-04-25T08:54:22.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.086+0000 7ff4bd528640 1 -- 172.21.15.43:0/2507333685 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b81f4850 con 0x7ff4b80cf670 2024-04-25T08:54:22.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.086+0000 7ff4bd528640 1 -- 172.21.15.43:0/2507333685 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b81f75e0 con 0x7ff4b80cf670 2024-04-25T08:54:22.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.087+0000 7ff4bd528640 1 -- 172.21.15.43:0/2507333685 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b80680f0 con 0x7ff4b80cf670 2024-04-25T08:54:22.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.087+0000 7ff4af7fe640 1 -- 172.21.15.43:0/2507333685 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff4a801a070 con 0x7ff4b80cf670 2024-04-25T08:54:22.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.087+0000 7ff4af7fe640 1 --2- 172.21.15.43:0/2507333685 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff49806b500 0x7ff49806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:22.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.088+0000 7ff4af7fe640 1 -- 172.21.15.43:0/2507333685 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff4a808a530 con 0x7ff4b80cf670 2024-04-25T08:54:22.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.088+0000 7ff4affff640 1 --2- 172.21.15.43:0/2507333685 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff49806b500 0x7ff49806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:22.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.088+0000 7ff4affff640 1 --2- 172.21.15.43:0/2507333685 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff49806b500 0x7ff49806d9c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0007c40 tx=0x7ff4a0007b90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:22.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.092+0000 7ff4af7fe640 1 -- 172.21.15.43:0/2507333685 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff4a8058c40 con 0x7ff4b80cf670 2024-04-25T08:54:22.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.315+0000 7ff4bd528640 1 -- 172.21.15.43:0/2507333685 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff4b8074ef0 con 0x7ff4b80cf670 2024-04-25T08:54:22.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.316+0000 7ff4af7fe640 1 -- 172.21.15.43:0/2507333685 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4938 (secure 0 0 0) 0x7ff4a8088e20 con 0x7ff4b80cf670 2024-04-25T08:54:22.306 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:54:22.306 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3616994003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":3616994003},{"type":"v1","addr":"172.21.15.43:6835","nonce":3616994003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/4022901908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4022901908},{"type":"v1","addr":"172.21.15.193:6835","nonce":4022901908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2823450590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2823450590},{"type":"v1","addr":"172.21.15.43:6834","nonce":2823450590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:54:11.945963+0000","modified":"2024-04-25T08:54:13.943002+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.193:6837/1197434598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1197434598},{"type":"v1","addr":"172.21.15.193:6837","nonce":1197434598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:54:22.306 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 6 2024-04-25T08:54:22.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.318+0000 7ff4bd528640 1 -- 172.21.15.43:0/2507333685 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff49806b500 msgr2=0x7ff49806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:22.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.318+0000 7ff4bd528640 1 --2- 172.21.15.43:0/2507333685 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff49806b500 0x7ff49806d9c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0007c40 tx=0x7ff4a0007b90 comp rx=0 tx=0).stop 2024-04-25T08:54:22.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.318+0000 7ff4bd528640 1 -- 172.21.15.43:0/2507333685 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4b80cf670 msgr2=0x7ff4b8160520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:22.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.318+0000 7ff4bd528640 1 --2- 172.21.15.43:0/2507333685 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4b80cf670 0x7ff4b8160520 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff4a800e850 tx=0x7ff4a800ed10 comp rx=0 tx=0).stop 2024-04-25T08:54:22.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.318+0000 7ff4bd528640 1 -- 172.21.15.43:0/2507333685 shutdown_connections 2024-04-25T08:54:22.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.318+0000 7ff4bd528640 1 --2- 172.21.15.43:0/2507333685 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff49806b500 0x7ff49806d9c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:22.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.318+0000 7ff4bd528640 1 --2- 172.21.15.43:0/2507333685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4b8161f30 0x7ff4b80caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:22.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.318+0000 7ff4bd528640 1 --2- 172.21.15.43:0/2507333685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4b815df50 0x7ff4b81f7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:22.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.318+0000 7ff4bd528640 1 --2- 172.21.15.43:0/2507333685 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4b80cf670 0x7ff4b8160520 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:22.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.318+0000 7ff4bd528640 1 -- 172.21.15.43:0/2507333685 >> 172.21.15.43:0/2507333685 conn(0x7ff4b80d3a30 msgr2=0x7ff4b8154080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:22.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.318+0000 7ff4bd528640 1 -- 172.21.15.43:0/2507333685 shutdown_connections 2024-04-25T08:54:22.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.318+0000 7ff4bd528640 1 -- 172.21.15.43:0/2507333685 wait complete. 2024-04-25T08:54:22.318 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:54:22.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.429+0000 7f8fa9509640 1 Processor -- start 2024-04-25T08:54:22.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.429+0000 7f8fa9509640 1 -- start start 2024-04-25T08:54:22.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.430+0000 7f8fa9509640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8fa415df50 0x7f8fa41619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:22.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.430+0000 7f8fa9509640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f8fa40cf670 0x7f8fa4162380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:22.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.430+0000 7f8fa9509640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fa40d24a0 con 0x7f8fa4161f30 2024-04-25T08:54:22.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.430+0000 7f8fa9509640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fa40cfa80 con 0x7f8fa415df50 2024-04-25T08:54:22.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.430+0000 7f8fa9509640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fa415e330 con 0x7f8fa40cf670 2024-04-25T08:54:22.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.430+0000 7f8fa2ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8fa415df50 0x7f8fa41619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:22.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.430+0000 7f8fa2ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8fa415df50 0x7f8fa41619f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56044/0 (socket says 172.21.15.43:56044) 2024-04-25T08:54:22.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.430+0000 7f8fa2ffd640 1 -- 172.21.15.43:0/2495946871 learned_addr learned my addr 172.21.15.43:0/2495946871 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:22.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.430+0000 7f8fa37fe640 1 --2- 172.21.15.43:0/2495946871 >> v2:172.21.15.193:3300/0 conn(0x7f8fa40cf670 0x7f8fa4162380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:22.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.430+0000 7f8fa2ffd640 1 -- 172.21.15.43:0/2495946871 >> v2:172.21.15.193:3300/0 conn(0x7f8fa40cf670 msgr2=0x7f8fa4162380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:22.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.430+0000 7f8fa2ffd640 1 --2- 172.21.15.43:0/2495946871 >> v2:172.21.15.193:3300/0 conn(0x7f8fa40cf670 0x7f8fa4162380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:22.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.430+0000 7f8fa2ffd640 1 -- 172.21.15.43:0/2495946871 >> v1:172.21.15.193:6789/0 conn(0x7f8fa4161f30 legacy=0x7f8fa40c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:22.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.430+0000 7f8fa2ffd640 1 -- 172.21.15.43:0/2495946871 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa40d2740 con 0x7f8fa415df50 2024-04-25T08:54:22.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.431+0000 7f8fa2ffd640 1 --2- 172.21.15.43:0/2495946871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fa415df50 0x7f8fa41619f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8fa408c880 tx=0x7f8f9402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80b43ef170539fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:22.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.431+0000 7f8fa27fc640 1 -- 172.21.15.43:0/2495946871 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f94037070 con 0x7f8fa415df50 2024-04-25T08:54:22.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.431+0000 7f8fa27fc640 1 -- 172.21.15.43:0/2495946871 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f94032440 con 0x7f8fa415df50 2024-04-25T08:54:22.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.431+0000 7f8fa27fc640 1 -- 172.21.15.43:0/2495946871 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f94032810 con 0x7f8fa415df50 2024-04-25T08:54:22.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.432+0000 7f8fa9509640 1 -- 172.21.15.43:0/2495946871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fa415df50 msgr2=0x7f8fa41619f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:22.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.432+0000 7f8fa9509640 1 --2- 172.21.15.43:0/2495946871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fa415df50 0x7f8fa41619f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8fa408c880 tx=0x7f8f9402a200 comp rx=0 tx=0).stop 2024-04-25T08:54:22.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.432+0000 7f8fa9509640 1 -- 172.21.15.43:0/2495946871 shutdown_connections 2024-04-25T08:54:22.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.432+0000 7f8fa9509640 1 --2- 172.21.15.43:0/2495946871 >> v2:172.21.15.193:3300/0 conn(0x7f8fa40cf670 0x7f8fa4162380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:22.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.432+0000 7f8fa9509640 1 --2- 172.21.15.43:0/2495946871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fa415df50 0x7f8fa41619f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:22.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.432+0000 7f8fa9509640 1 -- 172.21.15.43:0/2495946871 >> 172.21.15.43:0/2495946871 conn(0x7f8fa40d3a30 msgr2=0x7f8fa40d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:22.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.432+0000 7f8fa9509640 1 -- 172.21.15.43:0/2495946871 shutdown_connections 2024-04-25T08:54:22.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.432+0000 7f8fa9509640 1 -- 172.21.15.43:0/2495946871 wait complete. 2024-04-25T08:54:22.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.433+0000 7f8fa9509640 1 Processor -- start 2024-04-25T08:54:22.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.433+0000 7f8fa9509640 1 -- start start 2024-04-25T08:54:22.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.434+0000 7f8fa9509640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8fa40cf670 0x7f8fa4159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:22.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.434+0000 7f8fa9509640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fa415df50 0x7f8fa41f7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:22.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.434+0000 7f8fa9509640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fa4161f30 0x7f8fa41f7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:22.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.434+0000 7f8fa9509640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa40c77d0 con 0x7f8fa4161f30 2024-04-25T08:54:22.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.434+0000 7f8fa9509640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa40c74f0 con 0x7f8fa415df50 2024-04-25T08:54:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.434+0000 7f8fa9509640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa40d2190 con 0x7f8fa40cf670 2024-04-25T08:54:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.434+0000 7f8fa37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fa4161f30 0x7f8fa41f7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.434+0000 7f8fa37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fa4161f30 0x7f8fa41f7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56052/0 (socket says 172.21.15.43:56052) 2024-04-25T08:54:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.434+0000 7f8fa37fe640 1 -- 172.21.15.43:0/809223527 learned_addr learned my addr 172.21.15.43:0/809223527 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.434+0000 7f8fa2ffd640 1 --2- 172.21.15.43:0/809223527 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8fa40cf670 0x7f8fa4159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.434+0000 7f8f9bfff640 1 --2- 172.21.15.43:0/809223527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fa415df50 0x7f8fa41f7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.434+0000 7f8fa37fe640 1 -- 172.21.15.43:0/809223527 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8fa40cf670 msgr2=0x7f8fa4159b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.434+0000 7f8fa37fe640 1 --2- 172.21.15.43:0/809223527 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8fa40cf670 0x7f8fa4159b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.434+0000 7f8fa37fe640 1 -- 172.21.15.43:0/809223527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fa415df50 msgr2=0x7f8fa41f7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.434+0000 7f8fa37fe640 1 --2- 172.21.15.43:0/809223527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fa415df50 0x7f8fa41f7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.434+0000 7f8fa37fe640 1 -- 172.21.15.43:0/809223527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa409f5e0 con 0x7f8fa4161f30 2024-04-25T08:54:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.434+0000 7f8f9bfff640 1 --2- 172.21.15.43:0/809223527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fa415df50 0x7f8fa41f7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.434+0000 7f8fa37fe640 1 --2- 172.21.15.43:0/809223527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fa4161f30 0x7f8fa41f7b50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c007e40 tx=0x7f8f9c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:22.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.434+0000 7f8f9b7fe640 1 -- 172.21.15.43:0/809223527 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c014620 con 0x7f8fa4161f30 2024-04-25T08:54:22.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.434+0000 7f8f9b7fe640 1 -- 172.21.15.43:0/809223527 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f9c019070 con 0x7f8fa4161f30 2024-04-25T08:54:22.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.434+0000 7f8fa9509640 1 -- 172.21.15.43:0/809223527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fa415c900 con 0x7f8fa4161f30 2024-04-25T08:54:22.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.434+0000 7f8f9b7fe640 1 -- 172.21.15.43:0/809223527 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9c014a20 con 0x7f8fa4161f30 2024-04-25T08:54:22.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.434+0000 7f8fa9509640 1 -- 172.21.15.43:0/809223527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fa409f850 con 0x7f8fa4161f30 2024-04-25T08:54:22.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.435+0000 7f8fa9509640 1 -- 172.21.15.43:0/809223527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fa40680f0 con 0x7f8fa4161f30 2024-04-25T08:54:22.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.436+0000 7f8f9b7fe640 1 -- 172.21.15.43:0/809223527 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8f9c014bc0 con 0x7f8fa4161f30 2024-04-25T08:54:22.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.436+0000 7f8f9b7fe640 1 --2- 172.21.15.43:0/809223527 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8f8406b4d0 0x7f8f8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:22.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.436+0000 7f8f9b7fe640 1 -- 172.21.15.43:0/809223527 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8f9c08aab0 con 0x7f8fa4161f30 2024-04-25T08:54:22.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.436+0000 7f8fa2ffd640 1 --2- 172.21.15.43:0/809223527 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8f8406b4d0 0x7f8f8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:22.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.437+0000 7f8fa2ffd640 1 --2- 172.21.15.43:0/809223527 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8f8406b4d0 0x7f8f8406d990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8f94034000 tx=0x7f8f9402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:22.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.441+0000 7f8f9b7fe640 1 -- 172.21.15.43:0/809223527 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8f9c0599c0 con 0x7f8fa4161f30 2024-04-25T08:54:22.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.621+0000 7f8fa9509640 1 -- 172.21.15.43:0/809223527 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8fa4074ef0 con 0x7f8fa4161f30 2024-04-25T08:54:22.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.623+0000 7f8f9b7fe640 1 -- 172.21.15.43:0/809223527 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16569 (secure 0 0 0) 0x7f8f9c0592c0 con 0x7f8fa4161f30 2024-04-25T08:54:22.613 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:54:22.613 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":20,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:54:13.593188+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:54:09.757978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:54:10.926008+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:54:22.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.625+0000 7f8fa9509640 1 -- 172.21.15.43:0/809223527 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8f8406b4d0 msgr2=0x7f8f8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:22.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.625+0000 7f8fa9509640 1 --2- 172.21.15.43:0/809223527 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8f8406b4d0 0x7f8f8406d990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8f94034000 tx=0x7f8f9402e000 comp rx=0 tx=0).stop 2024-04-25T08:54:22.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.625+0000 7f8fa9509640 1 -- 172.21.15.43:0/809223527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fa4161f30 msgr2=0x7f8fa41f7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:22.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.625+0000 7f8fa9509640 1 --2- 172.21.15.43:0/809223527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fa4161f30 0x7f8fa41f7b50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8f9c007e40 tx=0x7f8f9c00e4b0 comp rx=0 tx=0).stop 2024-04-25T08:54:22.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.625+0000 7f8fa9509640 1 -- 172.21.15.43:0/809223527 shutdown_connections 2024-04-25T08:54:22.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.625+0000 7f8fa9509640 1 --2- 172.21.15.43:0/809223527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fa4161f30 0x7f8fa41f7b50 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:22.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.625+0000 7f8fa9509640 1 --2- 172.21.15.43:0/809223527 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8f8406b4d0 0x7f8f8406d990 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:22.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.625+0000 7f8fa9509640 1 --2- 172.21.15.43:0/809223527 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fa415df50 0x7f8fa41f7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:22.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.625+0000 7f8fa9509640 1 --2- 172.21.15.43:0/809223527 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8fa40cf670 0x7f8fa4159b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:22.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.625+0000 7f8fa9509640 1 -- 172.21.15.43:0/809223527 >> 172.21.15.43:0/809223527 conn(0x7f8fa40d3a30 msgr2=0x7f8fa40cd390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:22.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.625+0000 7f8fa9509640 1 -- 172.21.15.43:0/809223527 shutdown_connections 2024-04-25T08:54:22.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.625+0000 7f8fa9509640 1 -- 172.21.15.43:0/809223527 wait complete. 2024-04-25T08:54:22.626 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:54:09.757978+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:54:10.926008+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:54:22.627 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:54:22.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.740+0000 7fa7e53c8640 1 Processor -- start 2024-04-25T08:54:22.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.740+0000 7fa7e53c8640 1 -- start start 2024-04-25T08:54:22.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.741+0000 7fa7e53c8640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7e00d19f0 0x7fa7e00d1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:22.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.741+0000 7fa7e53c8640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7e00cf030 con 0x7fa7e00d2370 2024-04-25T08:54:22.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.741+0000 7fa7e53c8640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7e00cf310 con 0x7fa7e0161310 2024-04-25T08:54:22.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.741+0000 7fa7e53c8640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e00d4870 con 0x7fa7e00d19f0 2024-04-25T08:54:22.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.741+0000 7fa7df7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa7e00d2370 0x7fa7e00c7860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:38208/0 (socket says 172.21.15.43:38208) 2024-04-25T08:54:22.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.741+0000 7fa7df7fe640 1 -- 172.21.15.43:0/2550993901 learned_addr learned my addr 172.21.15.43:0/2550993901 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:22.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.741+0000 7fa7de7fc640 1 --2- 172.21.15.43:0/2550993901 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7e00d19f0 0x7fa7e00d1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:22.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.741+0000 7fa7ddffb640 1 -- 172.21.15.43:0/2550993901 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2876246173 0 0) 0x7fa7e00cf030 con 0x7fa7e00d2370 2024-04-25T08:54:22.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.741+0000 7fa7ddffb640 1 -- 172.21.15.43:0/2550993901 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7c8003520 con 0x7fa7e00d2370 2024-04-25T08:54:22.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.741+0000 7fa7ddffb640 1 -- 172.21.15.43:0/2550993901 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3287212828 0 0) 0x7fa7e00cf310 con 0x7fa7e0161310 2024-04-25T08:54:22.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.742+0000 7fa7ddffb640 1 -- 172.21.15.43:0/2550993901 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7e00cf030 con 0x7fa7e0161310 2024-04-25T08:54:22.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.742+0000 7fa7ddffb640 1 -- 172.21.15.43:0/2550993901 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1775526124 0 0) 0x7fa7c8003520 con 0x7fa7e00d2370 2024-04-25T08:54:22.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.742+0000 7fa7de7fc640 1 -- 172.21.15.43:0/2550993901 >> v1:172.21.15.193:6789/0 conn(0x7fa7e0161310 legacy=0x7fa7e00d4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:22.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.742+0000 7fa7de7fc640 1 -- 172.21.15.43:0/2550993901 >> v1:172.21.15.43:6789/0 conn(0x7fa7e00d2370 legacy=0x7fa7e00c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:22.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.742+0000 7fa7de7fc640 1 -- 172.21.15.43:0/2550993901 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e00cf5f0 con 0x7fa7e00d19f0 2024-04-25T08:54:22.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.742+0000 7fa7de7fc640 1 --2- 172.21.15.43:0/2550993901 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7e00d19f0 0x7fa7e00d1e30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa7d40097d0 tx=0x7fa7d402a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46ccfe8f5d86118e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:22.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.742+0000 7fa7ddffb640 1 -- 172.21.15.43:0/2550993901 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d4037070 con 0x7fa7e00d19f0 2024-04-25T08:54:22.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.743+0000 7fa7ddffb640 1 -- 172.21.15.43:0/2550993901 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d402ac30 con 0x7fa7e00d19f0 2024-04-25T08:54:22.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.743+0000 7fa7ddffb640 1 -- 172.21.15.43:0/2550993901 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d00019f0 con 0x7fa7e00d19f0 2024-04-25T08:54:22.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.743+0000 7fa7e53c8640 1 -- 172.21.15.43:0/2550993901 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7e00d19f0 msgr2=0x7fa7e00d1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:22.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.743+0000 7fa7e53c8640 1 --2- 172.21.15.43:0/2550993901 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7e00d19f0 0x7fa7e00d1e30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa7d40097d0 tx=0x7fa7d402a1a0 comp rx=0 tx=0).stop 2024-04-25T08:54:22.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.743+0000 7fa7e53c8640 1 -- 172.21.15.43:0/2550993901 shutdown_connections 2024-04-25T08:54:22.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.743+0000 7fa7e53c8640 1 --2- 172.21.15.43:0/2550993901 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7e00d19f0 0x7fa7e00d1e30 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:22.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.743+0000 7fa7e53c8640 1 -- 172.21.15.43:0/2550993901 >> 172.21.15.43:0/2550993901 conn(0x7fa7e009cd20 msgr2=0x7fa7e00d3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:22.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.743+0000 7fa7e53c8640 1 -- 172.21.15.43:0/2550993901 shutdown_connections 2024-04-25T08:54:22.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.743+0000 7fa7e53c8640 1 -- 172.21.15.43:0/2550993901 wait complete. 2024-04-25T08:54:22.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.744+0000 7fa7e53c8640 1 Processor -- start 2024-04-25T08:54:22.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.744+0000 7fa7e53c8640 1 -- start start 2024-04-25T08:54:22.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.745+0000 7fa7e53c8640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7e00d19f0 0x7fa7e015a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:22.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.745+0000 7fa7e53c8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e00d2370 0x7fa7e00cf830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:22.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.745+0000 7fa7e53c8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7e0161310 0x7fa7e00cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:22.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.745+0000 7fa7e53c8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e00c7f70 con 0x7fa7e00d2370 2024-04-25T08:54:22.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.745+0000 7fa7e53c8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e00d2800 con 0x7fa7e0161310 2024-04-25T08:54:22.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.745+0000 7fa7e53c8640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e00c80f0 con 0x7fa7e00d19f0 2024-04-25T08:54:22.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.745+0000 7fa7de7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e00d2370 0x7fa7e00cf830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:22.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.745+0000 7fa7de7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e00d2370 0x7fa7e00cf830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56058/0 (socket says 172.21.15.43:56058) 2024-04-25T08:54:22.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.745+0000 7fa7de7fc640 1 -- 172.21.15.43:0/367243656 learned_addr learned my addr 172.21.15.43:0/367243656 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:22.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.746+0000 7fa7deffd640 1 --2- 172.21.15.43:0/367243656 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7e00d19f0 0x7fa7e015a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:22.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.746+0000 7fa7de7fc640 1 -- 172.21.15.43:0/367243656 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7e00d19f0 msgr2=0x7fa7e015a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:22.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.746+0000 7fa7de7fc640 1 --2- 172.21.15.43:0/367243656 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7e00d19f0 0x7fa7e015a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:22.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.746+0000 7fa7de7fc640 1 -- 172.21.15.43:0/367243656 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7e0161310 msgr2=0x7fa7e00cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:22.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.746+0000 7fa7de7fc640 1 --2- 172.21.15.43:0/367243656 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7e0161310 0x7fa7e00cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:22.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.746+0000 7fa7de7fc640 1 -- 172.21.15.43:0/367243656 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e015a7e0 con 0x7fa7e00d2370 2024-04-25T08:54:22.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.746+0000 7fa7de7fc640 1 --2- 172.21.15.43:0/367243656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e00d2370 0x7fa7e00cf830 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa7d40097d0 tx=0x7fa7d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:22.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.746+0000 7fa7c77fe640 1 -- 172.21.15.43:0/367243656 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d4041030 con 0x7fa7e00d2370 2024-04-25T08:54:22.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.746+0000 7fa7c77fe640 1 -- 172.21.15.43:0/367243656 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d403d420 con 0x7fa7e00d2370 2024-04-25T08:54:22.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.746+0000 7fa7c77fe640 1 -- 172.21.15.43:0/367243656 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d4037040 con 0x7fa7e00d2370 2024-04-25T08:54:22.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.746+0000 7fa7e53c8640 1 -- 172.21.15.43:0/367243656 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7e01fb3b0 con 0x7fa7e00d2370 2024-04-25T08:54:22.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.746+0000 7fa7e53c8640 1 -- 172.21.15.43:0/367243656 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e01fb970 con 0x7fa7e00d2370 2024-04-25T08:54:22.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.747+0000 7fa7e53c8640 1 -- 172.21.15.43:0/367243656 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7e0074f60 con 0x7fa7e00d2370 2024-04-25T08:54:22.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.747+0000 7fa7c77fe640 1 -- 172.21.15.43:0/367243656 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa7d4044070 con 0x7fa7e00d2370 2024-04-25T08:54:22.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.748+0000 7fa7c77fe640 1 --2- 172.21.15.43:0/367243656 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa7cc06b4d0 0x7fa7cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:22.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.748+0000 7fa7c77fe640 1 -- 172.21.15.43:0/367243656 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa7d40aa6c0 con 0x7fa7e00d2370 2024-04-25T08:54:22.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.748+0000 7fa7deffd640 1 --2- 172.21.15.43:0/367243656 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa7cc06b4d0 0x7fa7cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:22.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.749+0000 7fa7deffd640 1 --2- 172.21.15.43:0/367243656 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa7cc06b4d0 0x7fa7cc06d990 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa7d80010e0 tx=0x7fa7d8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:22.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.752+0000 7fa7c77fe640 1 -- 172.21.15.43:0/367243656 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa7d4079540 con 0x7fa7e00d2370 2024-04-25T08:54:22.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.931+0000 7fa7e53c8640 1 -- 172.21.15.43:0/367243656 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa7e006bbf0 con 0x7fa7e00d2370 2024-04-25T08:54:22.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.931+0000 7fa7c77fe640 1 -- 172.21.15.43:0/367243656 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa7d4078e40 con 0x7fa7e00d2370 2024-04-25T08:54:22.922 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T08:54:22.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.933+0000 7fa7e53c8640 1 -- 172.21.15.43:0/367243656 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa7cc06b4d0 msgr2=0x7fa7cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:22.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.933+0000 7fa7e53c8640 1 --2- 172.21.15.43:0/367243656 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa7cc06b4d0 0x7fa7cc06d990 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa7d80010e0 tx=0x7fa7d8009040 comp rx=0 tx=0).stop 2024-04-25T08:54:22.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.933+0000 7fa7e53c8640 1 -- 172.21.15.43:0/367243656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e00d2370 msgr2=0x7fa7e00cf830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:22.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.933+0000 7fa7e53c8640 1 --2- 172.21.15.43:0/367243656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e00d2370 0x7fa7e00cf830 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa7d40097d0 tx=0x7fa7d40046c0 comp rx=0 tx=0).stop 2024-04-25T08:54:22.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.934+0000 7fa7e53c8640 1 -- 172.21.15.43:0/367243656 shutdown_connections 2024-04-25T08:54:22.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.934+0000 7fa7e53c8640 1 --2- 172.21.15.43:0/367243656 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa7cc06b4d0 0x7fa7cc06d990 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:22.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.934+0000 7fa7e53c8640 1 --2- 172.21.15.43:0/367243656 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7e0161310 0x7fa7e00cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:22.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.934+0000 7fa7e53c8640 1 --2- 172.21.15.43:0/367243656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e00d2370 0x7fa7e00cf830 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:22.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.934+0000 7fa7e53c8640 1 --2- 172.21.15.43:0/367243656 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7e00d19f0 0x7fa7e015a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:22.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.934+0000 7fa7e53c8640 1 -- 172.21.15.43:0/367243656 >> 172.21.15.43:0/367243656 conn(0x7fa7e009cd20 msgr2=0x7fa7e006b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:22.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.934+0000 7fa7e53c8640 1 -- 172.21.15.43:0/367243656 shutdown_connections 2024-04-25T08:54:22.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:22.934+0000 7fa7e53c8640 1 -- 172.21.15.43:0/367243656 wait complete. 2024-04-25T08:54:22.935 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:54:23.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.048+0000 7fc2e45e0640 1 Processor -- start 2024-04-25T08:54:23.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.048+0000 7fc2e45e0640 1 -- start start 2024-04-25T08:54:23.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.049+0000 7fc2e45e0640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fc2dc0d19f0 0x7fc2dc0d1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:23.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.049+0000 7fc2e45e0640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc2dc0d2370 0x7fc2dc0c7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:23.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.049+0000 7fc2e45e0640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2dc0cf030 con 0x7fc2dc161310 2024-04-25T08:54:23.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.049+0000 7fc2e45e0640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2dc0d4870 con 0x7fc2dc0d2370 2024-04-25T08:54:23.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.049+0000 7fc2e45e0640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2dc0cf310 con 0x7fc2dc0d19f0 2024-04-25T08:54:23.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.049+0000 7fc2e2b7d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc2dc0d2370 0x7fc2dc0c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:23.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.049+0000 7fc2e2b7d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc2dc0d2370 0x7fc2dc0c7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56074/0 (socket says 172.21.15.43:56074) 2024-04-25T08:54:23.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.049+0000 7fc2e2b7d640 1 -- 172.21.15.43:0/1797065565 learned_addr learned my addr 172.21.15.43:0/1797065565 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:23.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.049+0000 7fc2e1b7b640 1 --2- 172.21.15.43:0/1797065565 >> v2:172.21.15.193:3300/0 conn(0x7fc2dc0d19f0 0x7fc2dc0d1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:23.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.050+0000 7fc2e2b7d640 1 -- 172.21.15.43:0/1797065565 >> v2:172.21.15.193:3300/0 conn(0x7fc2dc0d19f0 msgr2=0x7fc2dc0d1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:23.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.050+0000 7fc2e2b7d640 1 --2- 172.21.15.43:0/1797065565 >> v2:172.21.15.193:3300/0 conn(0x7fc2dc0d19f0 0x7fc2dc0d1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:23.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.050+0000 7fc2e2b7d640 1 -- 172.21.15.43:0/1797065565 >> v1:172.21.15.193:6789/0 conn(0x7fc2dc161310 legacy=0x7fc2dc0d4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:23.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.050+0000 7fc2e2b7d640 1 -- 172.21.15.43:0/1797065565 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2dc0cf5f0 con 0x7fc2dc0d2370 2024-04-25T08:54:23.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.050+0000 7fc2e2b7d640 1 --2- 172.21.15.43:0/1797065565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2dc0d2370 0x7fc2dc0c7860 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc08d2c0 tx=0x7fc2d402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb4e3d28aded7cc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:23.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.050+0000 7fc2e137a640 1 -- 172.21.15.43:0/1797065565 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d4037070 con 0x7fc2dc0d2370 2024-04-25T08:54:23.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.050+0000 7fc2e137a640 1 -- 172.21.15.43:0/1797065565 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d4032440 con 0x7fc2dc0d2370 2024-04-25T08:54:23.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.050+0000 7fc2e137a640 1 -- 172.21.15.43:0/1797065565 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d4032810 con 0x7fc2dc0d2370 2024-04-25T08:54:23.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.051+0000 7fc2e45e0640 1 -- 172.21.15.43:0/1797065565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2dc0d2370 msgr2=0x7fc2dc0c7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:23.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.051+0000 7fc2e45e0640 1 --2- 172.21.15.43:0/1797065565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2dc0d2370 0x7fc2dc0c7860 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc08d2c0 tx=0x7fc2d402a230 comp rx=0 tx=0).stop 2024-04-25T08:54:23.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.051+0000 7fc2e45e0640 1 -- 172.21.15.43:0/1797065565 shutdown_connections 2024-04-25T08:54:23.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.051+0000 7fc2e45e0640 1 --2- 172.21.15.43:0/1797065565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2dc0d2370 0x7fc2dc0c7860 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:23.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.051+0000 7fc2e45e0640 1 --2- 172.21.15.43:0/1797065565 >> v2:172.21.15.193:3300/0 conn(0x7fc2dc0d19f0 0x7fc2dc0d1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:23.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.051+0000 7fc2e45e0640 1 -- 172.21.15.43:0/1797065565 >> 172.21.15.43:0/1797065565 conn(0x7fc2dc09cd20 msgr2=0x7fc2dc0d3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:23.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.051+0000 7fc2e45e0640 1 -- 172.21.15.43:0/1797065565 shutdown_connections 2024-04-25T08:54:23.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.051+0000 7fc2e45e0640 1 -- 172.21.15.43:0/1797065565 wait complete. 2024-04-25T08:54:23.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.052+0000 7fc2e45e0640 1 Processor -- start 2024-04-25T08:54:23.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.052+0000 7fc2e45e0640 1 -- start start 2024-04-25T08:54:23.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.053+0000 7fc2e45e0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2dc0d19f0 0x7fc2dc0ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:23.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.053+0000 7fc2e45e0640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc2dc0d2370 0x7fc2dc0b6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:23.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.053+0000 7fc2e45e0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc2dc161310 0x7fc2dc0b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:23.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.053+0000 7fc2e45e0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc0c7da0 con 0x7fc2dc0d19f0 2024-04-25T08:54:23.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.053+0000 7fc2e45e0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc0d27e0 con 0x7fc2dc161310 2024-04-25T08:54:23.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.053+0000 7fc2e45e0640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2dc0c7f20 con 0x7fc2dc0d2370 2024-04-25T08:54:23.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.053+0000 7fc2e237c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2dc0d19f0 0x7fc2dc0ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:23.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.053+0000 7fc2e237c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2dc0d19f0 0x7fc2dc0ccd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56080/0 (socket says 172.21.15.43:56080) 2024-04-25T08:54:23.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.053+0000 7fc2e237c640 1 -- 172.21.15.43:0/2105840639 learned_addr learned my addr 172.21.15.43:0/2105840639 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:23.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.053+0000 7fc2e2b7d640 1 --2- 172.21.15.43:0/2105840639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc2dc161310 0x7fc2dc0b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:23.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.053+0000 7fc2e1b7b640 1 --2- 172.21.15.43:0/2105840639 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc2dc0d2370 0x7fc2dc0b6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:23.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.053+0000 7fc2e237c640 1 -- 172.21.15.43:0/2105840639 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc2dc0d2370 msgr2=0x7fc2dc0b6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:23.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.053+0000 7fc2e237c640 1 --2- 172.21.15.43:0/2105840639 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc2dc0d2370 0x7fc2dc0b6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:23.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.053+0000 7fc2e237c640 1 -- 172.21.15.43:0/2105840639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc2dc161310 msgr2=0x7fc2dc0b39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:23.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.053+0000 7fc2e237c640 1 --2- 172.21.15.43:0/2105840639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc2dc161310 0x7fc2dc0b39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:23.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.053+0000 7fc2e237c640 1 -- 172.21.15.43:0/2105840639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2dc0b2940 con 0x7fc2dc0d19f0 2024-04-25T08:54:23.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.054+0000 7fc2e237c640 1 --2- 172.21.15.43:0/2105840639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2dc0d19f0 0x7fc2dc0ccd40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc00ba50 tx=0x7fc2cc00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:23.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.054+0000 7fc2daffd640 1 -- 172.21.15.43:0/2105840639 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc019070 con 0x7fc2dc0d19f0 2024-04-25T08:54:23.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.054+0000 7fc2daffd640 1 -- 172.21.15.43:0/2105840639 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2cc00cc10 con 0x7fc2dc0d19f0 2024-04-25T08:54:23.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.054+0000 7fc2e45e0640 1 -- 172.21.15.43:0/2105840639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2dc0b2c20 con 0x7fc2dc0d19f0 2024-04-25T08:54:23.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.054+0000 7fc2daffd640 1 -- 172.21.15.43:0/2105840639 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc014440 con 0x7fc2dc0d19f0 2024-04-25T08:54:23.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.054+0000 7fc2e45e0640 1 -- 172.21.15.43:0/2105840639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2dc0b31e0 con 0x7fc2dc0d19f0 2024-04-25T08:54:23.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.055+0000 7fc2e45e0640 1 -- 172.21.15.43:0/2105840639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2dc074f60 con 0x7fc2dc0d19f0 2024-04-25T08:54:23.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.055+0000 7fc2daffd640 1 -- 172.21.15.43:0/2105840639 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc2cc01c070 con 0x7fc2dc0d19f0 2024-04-25T08:54:23.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.056+0000 7fc2daffd640 1 --2- 172.21.15.43:0/2105840639 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc2c806b4d0 0x7fc2c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:23.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.056+0000 7fc2daffd640 1 -- 172.21.15.43:0/2105840639 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc2cc08a090 con 0x7fc2dc0d19f0 2024-04-25T08:54:23.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.056+0000 7fc2e1b7b640 1 --2- 172.21.15.43:0/2105840639 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc2c806b4d0 0x7fc2c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:23.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.057+0000 7fc2e1b7b640 1 --2- 172.21.15.43:0/2105840639 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc2c806b4d0 0x7fc2c806d990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0006fd0 tx=0x7fc2d0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:23.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.060+0000 7fc2daffd640 1 -- 172.21.15.43:0/2105840639 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc2cc058e70 con 0x7fc2dc0d19f0 2024-04-25T08:54:23.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.239+0000 7fc2e45e0640 1 -- 172.21.15.43:0/2105840639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc2dc15d7b0 con 0x7fc2dc0d19f0 2024-04-25T08:54:23.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.240+0000 7fc2daffd640 1 -- 172.21.15.43:0/2105840639 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc2cc058770 con 0x7fc2dc0d19f0 2024-04-25T08:54:23.230 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T08:54:23.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.242+0000 7fc2e45e0640 1 -- 172.21.15.43:0/2105840639 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc2c806b4d0 msgr2=0x7fc2c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:23.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.242+0000 7fc2e45e0640 1 --2- 172.21.15.43:0/2105840639 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc2c806b4d0 0x7fc2c806d990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0006fd0 tx=0x7fc2d0008040 comp rx=0 tx=0).stop 2024-04-25T08:54:23.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.242+0000 7fc2e45e0640 1 -- 172.21.15.43:0/2105840639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2dc0d19f0 msgr2=0x7fc2dc0ccd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:23.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.242+0000 7fc2e45e0640 1 --2- 172.21.15.43:0/2105840639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2dc0d19f0 0x7fc2dc0ccd40 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc2cc00ba50 tx=0x7fc2cc00bf10 comp rx=0 tx=0).stop 2024-04-25T08:54:23.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.242+0000 7fc2e45e0640 1 -- 172.21.15.43:0/2105840639 shutdown_connections 2024-04-25T08:54:23.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.242+0000 7fc2e45e0640 1 --2- 172.21.15.43:0/2105840639 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc2c806b4d0 0x7fc2c806d990 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:23.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.242+0000 7fc2e45e0640 1 --2- 172.21.15.43:0/2105840639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc2dc161310 0x7fc2dc0b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:23.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.242+0000 7fc2e45e0640 1 --2- 172.21.15.43:0/2105840639 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc2dc0d2370 0x7fc2dc0b6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:23.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.242+0000 7fc2e45e0640 1 --2- 172.21.15.43:0/2105840639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc2dc0d19f0 0x7fc2dc0ccd40 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:23.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.242+0000 7fc2e45e0640 1 -- 172.21.15.43:0/2105840639 >> 172.21.15.43:0/2105840639 conn(0x7fc2dc09cd20 msgr2=0x7fc2dc0d3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:23.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.242+0000 7fc2e45e0640 1 -- 172.21.15.43:0/2105840639 shutdown_connections 2024-04-25T08:54:23.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.242+0000 7fc2e45e0640 1 -- 172.21.15.43:0/2105840639 wait complete. 2024-04-25T08:54:23.243 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:54:23.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.356+0000 7ffbdb0ea640 1 Processor -- start 2024-04-25T08:54:23.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.356+0000 7ffbdb0ea640 1 -- start start 2024-04-25T08:54:23.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.357+0000 7ffbdb0ea640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ffbd4161310 0x7ffbd40d4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:23.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.357+0000 7ffbdb0ea640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ffbd40d0120 0x7ffbd40d0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:23.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.357+0000 7ffbdb0ea640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbd40d45c0 con 0x7ffbd40d0aa0 2024-04-25T08:54:23.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.357+0000 7ffbdb0ea640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbd40d4870 con 0x7ffbd40d0120 2024-04-25T08:54:23.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.357+0000 7ffbdb0ea640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbd40cf030 con 0x7ffbd4161310 2024-04-25T08:54:23.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.357+0000 7ffbd3fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ffbd40d0120 0x7ffbd40d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:23.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.357+0000 7ffbd3fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ffbd40d0120 0x7ffbd40d0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56084/0 (socket says 172.21.15.43:56084) 2024-04-25T08:54:23.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.357+0000 7ffbd3fff640 1 -- 172.21.15.43:0/3278959871 learned_addr learned my addr 172.21.15.43:0/3278959871 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:23.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.357+0000 7ffbd8e86640 1 --2- 172.21.15.43:0/3278959871 >> v2:172.21.15.193:3300/0 conn(0x7ffbd4161310 0x7ffbd40d4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:23.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.357+0000 7ffbd3fff640 1 -- 172.21.15.43:0/3278959871 >> v2:172.21.15.193:3300/0 conn(0x7ffbd4161310 msgr2=0x7ffbd40d4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:23.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.357+0000 7ffbd3fff640 1 --2- 172.21.15.43:0/3278959871 >> v2:172.21.15.193:3300/0 conn(0x7ffbd4161310 0x7ffbd40d4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:23.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.357+0000 7ffbd3fff640 1 -- 172.21.15.43:0/3278959871 >> v1:172.21.15.193:6789/0 conn(0x7ffbd40d0aa0 legacy=0x7ffbd40d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:23.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.357+0000 7ffbd3fff640 1 -- 172.21.15.43:0/3278959871 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd40cf310 con 0x7ffbd40d0120 2024-04-25T08:54:23.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.357+0000 7ffbd8e86640 1 --2- 172.21.15.43:0/3278959871 >> v2:172.21.15.193:3300/0 conn(0x7ffbd4161310 0x7ffbd40d4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:23.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.358+0000 7ffbd3fff640 1 --2- 172.21.15.43:0/3278959871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbd40d0120 0x7ffbd40d0560 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ffbd408d2f0 tx=0x7ffbc002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=524d4eb559a0fbfc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:23.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.358+0000 7ffbd37fe640 1 -- 172.21.15.43:0/3278959871 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc0037070 con 0x7ffbd40d0120 2024-04-25T08:54:23.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.358+0000 7ffbd37fe640 1 -- 172.21.15.43:0/3278959871 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc0032440 con 0x7ffbd40d0120 2024-04-25T08:54:23.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.358+0000 7ffbd37fe640 1 -- 172.21.15.43:0/3278959871 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc0032810 con 0x7ffbd40d0120 2024-04-25T08:54:23.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.358+0000 7ffbdb0ea640 1 -- 172.21.15.43:0/3278959871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbd40d0120 msgr2=0x7ffbd40d0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:23.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.358+0000 7ffbdb0ea640 1 --2- 172.21.15.43:0/3278959871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbd40d0120 0x7ffbd40d0560 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ffbd408d2f0 tx=0x7ffbc002a230 comp rx=0 tx=0).stop 2024-04-25T08:54:23.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.359+0000 7ffbdb0ea640 1 -- 172.21.15.43:0/3278959871 shutdown_connections 2024-04-25T08:54:23.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.359+0000 7ffbdb0ea640 1 --2- 172.21.15.43:0/3278959871 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbd40d0120 0x7ffbd40d0560 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:23.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.359+0000 7ffbdb0ea640 1 --2- 172.21.15.43:0/3278959871 >> v2:172.21.15.193:3300/0 conn(0x7ffbd4161310 0x7ffbd40d4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:23.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.359+0000 7ffbdb0ea640 1 -- 172.21.15.43:0/3278959871 >> 172.21.15.43:0/3278959871 conn(0x7ffbd409cd20 msgr2=0x7ffbd40d3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:23.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.359+0000 7ffbdb0ea640 1 -- 172.21.15.43:0/3278959871 shutdown_connections 2024-04-25T08:54:23.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.359+0000 7ffbdb0ea640 1 -- 172.21.15.43:0/3278959871 wait complete. 2024-04-25T08:54:23.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.360+0000 7ffbdb0ea640 1 Processor -- start 2024-04-25T08:54:23.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.360+0000 7ffbdb0ea640 1 -- start start 2024-04-25T08:54:23.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.361+0000 7ffbdb0ea640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbd40d0120 0x7ffbd415cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:23.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.361+0000 7ffbdb0ea640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffbd40d0aa0 0x7ffbd4146e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:23.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.361+0000 7ffbdb0ea640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffbd4161310 0x7ffbd40b8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:23.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.361+0000 7ffbdb0ea640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd41616f0 con 0x7ffbd40d0120 2024-04-25T08:54:23.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.361+0000 7ffbdb0ea640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd40d0ef0 con 0x7ffbd40d0aa0 2024-04-25T08:54:23.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.361+0000 7ffbdb0ea640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd40c8870 con 0x7ffbd4161310 2024-04-25T08:54:23.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.361+0000 7ffbd8e86640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbd40d0120 0x7ffbd415cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:23.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.361+0000 7ffbd8e86640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbd40d0120 0x7ffbd415cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56100/0 (socket says 172.21.15.43:56100) 2024-04-25T08:54:23.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.361+0000 7ffbd8e86640 1 -- 172.21.15.43:0/602001835 learned_addr learned my addr 172.21.15.43:0/602001835 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:23.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.361+0000 7ffbd3fff640 1 --2- 172.21.15.43:0/602001835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffbd40d0aa0 0x7ffbd4146e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:23.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.361+0000 7ffbd9687640 1 --2- 172.21.15.43:0/602001835 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffbd4161310 0x7ffbd40b8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:23.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.361+0000 7ffbd8e86640 1 -- 172.21.15.43:0/602001835 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffbd4161310 msgr2=0x7ffbd40b8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:23.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.361+0000 7ffbd8e86640 1 --2- 172.21.15.43:0/602001835 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffbd4161310 0x7ffbd40b8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:23.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.361+0000 7ffbd8e86640 1 -- 172.21.15.43:0/602001835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffbd40d0aa0 msgr2=0x7ffbd4146e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:23.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.361+0000 7ffbd8e86640 1 --2- 172.21.15.43:0/602001835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffbd40d0aa0 0x7ffbd4146e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:23.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.361+0000 7ffbd8e86640 1 -- 172.21.15.43:0/602001835 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd4157d20 con 0x7ffbd40d0120 2024-04-25T08:54:23.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.361+0000 7ffbd9687640 1 --2- 172.21.15.43:0/602001835 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffbd4161310 0x7ffbd40b8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:23.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.361+0000 7ffbd8e86640 1 --2- 172.21.15.43:0/602001835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbd40d0120 0x7ffbd415cfc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ffbc800c820 tx=0x7ffbc800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:23.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.361+0000 7ffbd17fa640 1 -- 172.21.15.43:0/602001835 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8015070 con 0x7ffbd40d0120 2024-04-25T08:54:23.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.361+0000 7ffbd17fa640 1 -- 172.21.15.43:0/602001835 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc8007a30 con 0x7ffbd40d0120 2024-04-25T08:54:23.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.361+0000 7ffbd17fa640 1 -- 172.21.15.43:0/602001835 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8007de0 con 0x7ffbd40d0120 2024-04-25T08:54:23.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.361+0000 7ffbdb0ea640 1 -- 172.21.15.43:0/602001835 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbd4158000 con 0x7ffbd40d0120 2024-04-25T08:54:23.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.361+0000 7ffbdb0ea640 1 -- 172.21.15.43:0/602001835 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbd415c470 con 0x7ffbd40d0120 2024-04-25T08:54:23.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.362+0000 7ffbdb0ea640 1 -- 172.21.15.43:0/602001835 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbd408dab0 con 0x7ffbd40d0120 2024-04-25T08:54:23.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.363+0000 7ffbd17fa640 1 -- 172.21.15.43:0/602001835 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffbc8018070 con 0x7ffbd40d0120 2024-04-25T08:54:23.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.363+0000 7ffbd17fa640 1 --2- 172.21.15.43:0/602001835 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ffbb806b4d0 0x7ffbb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:23.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.363+0000 7ffbd17fa640 1 -- 172.21.15.43:0/602001835 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffbc808a080 con 0x7ffbd40d0120 2024-04-25T08:54:23.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.364+0000 7ffbd3fff640 1 --2- 172.21.15.43:0/602001835 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ffbb806b4d0 0x7ffbb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:23.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.364+0000 7ffbd3fff640 1 --2- 172.21.15.43:0/602001835 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ffbb806b4d0 0x7ffbb806d990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0034000 tx=0x7ffbc002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:23.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.368+0000 7ffbd17fa640 1 -- 172.21.15.43:0/602001835 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffbd408dab0 con 0x7ffbd40d0120 2024-04-25T08:54:23.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.552+0000 7ffbdb0ea640 1 -- 172.21.15.43:0/602001835 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ffbd4074f60 con 0x7ffbd40d0120 2024-04-25T08:54:23.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.553+0000 7ffbd17fa640 1 -- 172.21.15.43:0/602001835 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7ffbc80588b0 con 0x7ffbd40d0120 2024-04-25T08:54:23.543 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T08:54:23.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.554+0000 7ffbdb0ea640 1 -- 172.21.15.43:0/602001835 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ffbb806b4d0 msgr2=0x7ffbb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:23.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.554+0000 7ffbdb0ea640 1 --2- 172.21.15.43:0/602001835 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ffbb806b4d0 0x7ffbb806d990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0034000 tx=0x7ffbc002e000 comp rx=0 tx=0).stop 2024-04-25T08:54:23.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.554+0000 7ffbdb0ea640 1 -- 172.21.15.43:0/602001835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbd40d0120 msgr2=0x7ffbd415cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:23.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.554+0000 7ffbdb0ea640 1 --2- 172.21.15.43:0/602001835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbd40d0120 0x7ffbd415cfc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ffbc800c820 tx=0x7ffbc800cce0 comp rx=0 tx=0).stop 2024-04-25T08:54:23.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.554+0000 7ffbdb0ea640 1 -- 172.21.15.43:0/602001835 shutdown_connections 2024-04-25T08:54:23.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.554+0000 7ffbdb0ea640 1 --2- 172.21.15.43:0/602001835 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffbd4161310 0x7ffbd40b8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:23.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.554+0000 7ffbdb0ea640 1 --2- 172.21.15.43:0/602001835 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ffbb806b4d0 0x7ffbb806d990 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:23.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.554+0000 7ffbdb0ea640 1 --2- 172.21.15.43:0/602001835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffbd40d0aa0 0x7ffbd4146e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:23.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.554+0000 7ffbdb0ea640 1 --2- 172.21.15.43:0/602001835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbd40d0120 0x7ffbd415cfc0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:23.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.554+0000 7ffbdb0ea640 1 -- 172.21.15.43:0/602001835 >> 172.21.15.43:0/602001835 conn(0x7ffbd409cd20 msgr2=0x7ffbd40d1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:23.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.554+0000 7ffbdb0ea640 1 -- 172.21.15.43:0/602001835 shutdown_connections 2024-04-25T08:54:23.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.555+0000 7ffbdb0ea640 1 -- 172.21.15.43:0/602001835 wait complete. 2024-04-25T08:54:23.555 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:54:23.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.666+0000 7fa6a5cb3640 1 Processor -- start 2024-04-25T08:54:23.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.666+0000 7fa6a5cb3640 1 -- start start 2024-04-25T08:54:23.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.667+0000 7fa6a5cb3640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fa6a006b540 0x7fa6a00b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:23.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.667+0000 7fa6a5cb3640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6a00b5060 con 0x7fa6a015de60 2024-04-25T08:54:23.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.667+0000 7fa6a5cb3640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6a00b5340 con 0x7fa6a0071ca0 2024-04-25T08:54:23.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.667+0000 7fa6a5cb3640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6a00b4e10 con 0x7fa6a006b540 2024-04-25T08:54:23.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.667+0000 7fa69f7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa6a015de60 0x7fa6a0071590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:38222/0 (socket says 172.21.15.43:38222) 2024-04-25T08:54:23.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.667+0000 7fa69f7fe640 1 -- 172.21.15.43:0/4279434595 learned_addr learned my addr 172.21.15.43:0/4279434595 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:23.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.668+0000 7fa69ffff640 1 --2- 172.21.15.43:0/4279434595 >> v2:172.21.15.193:3300/0 conn(0x7fa6a006b540 0x7fa6a00b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:23.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.668+0000 7fa69e7fc640 1 -- 172.21.15.43:0/4279434595 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 663794594 0 0) 0x7fa6a00b5060 con 0x7fa6a015de60 2024-04-25T08:54:23.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.668+0000 7fa69e7fc640 1 -- 172.21.15.43:0/4279434595 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa688003520 con 0x7fa6a015de60 2024-04-25T08:54:23.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.668+0000 7fa69e7fc640 1 -- 172.21.15.43:0/4279434595 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 575128114 0 0) 0x7fa688003520 con 0x7fa6a015de60 2024-04-25T08:54:23.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.668+0000 7fa69e7fc640 1 -- 172.21.15.43:0/4279434595 >> v2:172.21.15.193:3300/0 conn(0x7fa6a006b540 msgr2=0x7fa6a00b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:23.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.668+0000 7fa69e7fc640 1 --2- 172.21.15.43:0/4279434595 >> v2:172.21.15.193:3300/0 conn(0x7fa6a006b540 0x7fa6a00b48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:23.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.668+0000 7fa69e7fc640 1 -- 172.21.15.43:0/4279434595 >> v1:172.21.15.193:6789/0 conn(0x7fa6a0071ca0 legacy=0x7fa6a00cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:23.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.668+0000 7fa69e7fc640 1 -- 172.21.15.43:0/4279434595 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a00b5620 con 0x7fa6a015de60 2024-04-25T08:54:23.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.668+0000 7fa69e7fc640 1 -- 172.21.15.43:0/4279434595 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7fa690001670 con 0x7fa6a015de60 2024-04-25T08:54:23.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.668+0000 7fa69e7fc640 1 -- 172.21.15.43:0/4279434595 >> v1:172.21.15.43:6789/0 conn(0x7fa6a015de60 legacy=0x7fa6a0071590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:23.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.668+0000 7fa69e7fc640 1 --2- 172.21.15.43:0/4279434595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6880039b0 0x7fa688003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:23.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.668+0000 7fa69e7fc640 1 --2- 172.21.15.43:0/4279434595 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa688004800 0x7fa688004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:23.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.668+0000 7fa69e7fc640 1 --2- 172.21.15.43:0/4279434595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6880051c0 0x7fa68800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:23.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.668+0000 7fa69e7fc640 1 -- 172.21.15.43:0/4279434595 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a00b4e10 con 0x7fa6880039b0 2024-04-25T08:54:23.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.668+0000 7fa69e7fc640 1 -- 172.21.15.43:0/4279434595 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa68800c060 con 0x7fa6880051c0 2024-04-25T08:54:23.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.668+0000 7fa69e7fc640 1 -- 172.21.15.43:0/4279434595 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa68800c260 con 0x7fa688004800 2024-04-25T08:54:23.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.668+0000 7fa69f7fe640 1 --2- 172.21.15.43:0/4279434595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6880039b0 0x7fa688003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:23.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.668+0000 7fa69ffff640 1 --2- 172.21.15.43:0/4279434595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6880051c0 0x7fa68800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:23.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.668+0000 7fa69effd640 1 --2- 172.21.15.43:0/4279434595 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa688004800 0x7fa688004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:23.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.668+0000 7fa69f7fe640 1 -- 172.21.15.43:0/4279434595 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa688004800 msgr2=0x7fa688004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:23.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.668+0000 7fa69f7fe640 1 --2- 172.21.15.43:0/4279434595 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa688004800 0x7fa688004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:23.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.668+0000 7fa69f7fe640 1 -- 172.21.15.43:0/4279434595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6880051c0 msgr2=0x7fa68800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:23.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.668+0000 7fa69f7fe640 1 --2- 172.21.15.43:0/4279434595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6880051c0 0x7fa68800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:23.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.668+0000 7fa69f7fe640 1 -- 172.21.15.43:0/4279434595 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa688003520 con 0x7fa6880039b0 2024-04-25T08:54:23.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.668+0000 7fa69ffff640 1 --2- 172.21.15.43:0/4279434595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6880051c0 0x7fa68800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:23.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.668+0000 7fa69ffff640 1 -- 172.21.15.43:0/4279434595 reap_dead start 2024-04-25T08:54:23.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.669+0000 7fa69f7fe640 1 --2- 172.21.15.43:0/4279434595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6880039b0 0x7fa688003dd0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa69000c240 tx=0x7fa688005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a96baaeb11b9ba4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:23.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.669+0000 7fa69e7fc640 1 -- 172.21.15.43:0/4279434595 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa690007cf0 con 0x7fa6880039b0 2024-04-25T08:54:23.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.669+0000 7fa69e7fc640 1 -- 172.21.15.43:0/4279434595 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa690005af0 con 0x7fa6880039b0 2024-04-25T08:54:23.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.669+0000 7fa69e7fc640 1 -- 172.21.15.43:0/4279434595 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa690007070 con 0x7fa6880039b0 2024-04-25T08:54:23.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.670+0000 7fa6a5cb3640 1 -- 172.21.15.43:0/4279434595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6880039b0 msgr2=0x7fa688003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:23.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.670+0000 7fa6a5cb3640 1 --2- 172.21.15.43:0/4279434595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6880039b0 0x7fa688003dd0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa69000c240 tx=0x7fa688005830 comp rx=0 tx=0).stop 2024-04-25T08:54:23.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.670+0000 7fa6a5cb3640 1 -- 172.21.15.43:0/4279434595 shutdown_connections 2024-04-25T08:54:23.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.670+0000 7fa6a5cb3640 1 --2- 172.21.15.43:0/4279434595 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6880039b0 0x7fa688003dd0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:23.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.670+0000 7fa6a5cb3640 1 -- 172.21.15.43:0/4279434595 >> 172.21.15.43:0/4279434595 conn(0x7fa6a00b2490 msgr2=0x7fa6a00b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:23.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.670+0000 7fa6a5cb3640 1 -- 172.21.15.43:0/4279434595 shutdown_connections 2024-04-25T08:54:23.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.670+0000 7fa6a5cb3640 1 -- 172.21.15.43:0/4279434595 wait complete. 2024-04-25T08:54:23.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.671+0000 7fa6a5cb3640 1 Processor -- start 2024-04-25T08:54:23.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.671+0000 7fa6a5cb3640 1 -- start start 2024-04-25T08:54:23.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.672+0000 7fa6a5cb3640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa6880039b0 0x7fa6a0160c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:23.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.672+0000 7fa6a5cb3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6a015cba0 0x7fa6a00aed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:23.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.672+0000 7fa6a5cb3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6a00af2b0 0x7fa6a015ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:23.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.672+0000 7fa6a5cb3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a015e240 con 0x7fa6a015cba0 2024-04-25T08:54:23.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.672+0000 7fa6a5cb3640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a00ceae0 con 0x7fa6a00af2b0 2024-04-25T08:54:23.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.672+0000 7fa6a5cb3640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6a006b950 con 0x7fa6880039b0 2024-04-25T08:54:23.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.672+0000 7fa69effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6a015cba0 0x7fa6a00aed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:23.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.672+0000 7fa69effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6a015cba0 0x7fa6a00aed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56114/0 (socket says 172.21.15.43:56114) 2024-04-25T08:54:23.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.672+0000 7fa69effd640 1 -- 172.21.15.43:0/769805982 learned_addr learned my addr 172.21.15.43:0/769805982 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:23.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.672+0000 7fa69f7fe640 1 --2- 172.21.15.43:0/769805982 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa6880039b0 0x7fa6a0160c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:23.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.672+0000 7fa69ffff640 1 --2- 172.21.15.43:0/769805982 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6a00af2b0 0x7fa6a015ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:23.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.672+0000 7fa69effd640 1 -- 172.21.15.43:0/769805982 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa6880039b0 msgr2=0x7fa6a0160c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:23.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.672+0000 7fa69effd640 1 --2- 172.21.15.43:0/769805982 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa6880039b0 0x7fa6a0160c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:23.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.672+0000 7fa69effd640 1 -- 172.21.15.43:0/769805982 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6a00af2b0 msgr2=0x7fa6a015ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:23.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.672+0000 7fa69effd640 1 --2- 172.21.15.43:0/769805982 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6a00af2b0 0x7fa6a015ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:23.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.672+0000 7fa69effd640 1 -- 172.21.15.43:0/769805982 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6a01f7950 con 0x7fa6a015cba0 2024-04-25T08:54:23.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.672+0000 7fa69ffff640 1 --2- 172.21.15.43:0/769805982 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6a00af2b0 0x7fa6a015ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:23.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.672+0000 7fa69effd640 1 --2- 172.21.15.43:0/769805982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6a015cba0 0x7fa6a00aed70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa69400e500 tx=0x7fa69400e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:23.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.673+0000 7fa687fff640 1 -- 172.21.15.43:0/769805982 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6940053a0 con 0x7fa6a015cba0 2024-04-25T08:54:23.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.673+0000 7fa687fff640 1 -- 172.21.15.43:0/769805982 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa694018070 con 0x7fa6a015cba0 2024-04-25T08:54:23.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.673+0000 7fa6a5cb3640 1 -- 172.21.15.43:0/769805982 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6a01f7c30 con 0x7fa6a015cba0 2024-04-25T08:54:23.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.673+0000 7fa687fff640 1 -- 172.21.15.43:0/769805982 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa694003810 con 0x7fa6a015cba0 2024-04-25T08:54:23.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.673+0000 7fa6a5cb3640 1 -- 172.21.15.43:0/769805982 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6a01f81f0 con 0x7fa6a015cba0 2024-04-25T08:54:23.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.673+0000 7fa6a5cb3640 1 -- 172.21.15.43:0/769805982 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6a0067c10 con 0x7fa6a015cba0 2024-04-25T08:54:23.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.674+0000 7fa687fff640 1 -- 172.21.15.43:0/769805982 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa6940039b0 con 0x7fa6a015cba0 2024-04-25T08:54:23.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.674+0000 7fa687fff640 1 --2- 172.21.15.43:0/769805982 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa68c06b4d0 0x7fa68c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:23.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.674+0000 7fa687fff640 1 -- 172.21.15.43:0/769805982 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa694089ee0 con 0x7fa6a015cba0 2024-04-25T08:54:23.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.675+0000 7fa69f7fe640 1 --2- 172.21.15.43:0/769805982 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa68c06b4d0 0x7fa68c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:23.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.675+0000 7fa69f7fe640 1 --2- 172.21.15.43:0/769805982 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa68c06b4d0 0x7fa68c06d990 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa690032000 tx=0x7fa690009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:23.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.678+0000 7fa687fff640 1 -- 172.21.15.43:0/769805982 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa6a0067c10 con 0x7fa6a015cba0 2024-04-25T08:54:23.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.894+0000 7fa6a5cb3640 1 -- 172.21.15.43:0/769805982 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa6a0074ef0 con 0x7fa6a015cba0 2024-04-25T08:54:23.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.894+0000 7fa687fff640 1 -- 172.21.15.43:0/769805982 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4938 (secure 0 0 0) 0x7fa694058d70 con 0x7fa6a015cba0 2024-04-25T08:54:23.884 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:54:23.884 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3616994003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":3616994003},{"type":"v1","addr":"172.21.15.43:6835","nonce":3616994003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/4022901908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4022901908},{"type":"v1","addr":"172.21.15.193:6835","nonce":4022901908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2823450590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2823450590},{"type":"v1","addr":"172.21.15.43:6834","nonce":2823450590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:54:11.945963+0000","modified":"2024-04-25T08:54:13.943002+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.193:6837/1197434598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1197434598},{"type":"v1","addr":"172.21.15.193:6837","nonce":1197434598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:54:23.885 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 6 2024-04-25T08:54:23.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.896+0000 7fa6a5cb3640 1 -- 172.21.15.43:0/769805982 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa68c06b4d0 msgr2=0x7fa68c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:23.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.896+0000 7fa6a5cb3640 1 --2- 172.21.15.43:0/769805982 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa68c06b4d0 0x7fa68c06d990 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa690032000 tx=0x7fa690009030 comp rx=0 tx=0).stop 2024-04-25T08:54:23.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.896+0000 7fa6a5cb3640 1 -- 172.21.15.43:0/769805982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6a015cba0 msgr2=0x7fa6a00aed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:23.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.896+0000 7fa6a5cb3640 1 --2- 172.21.15.43:0/769805982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6a015cba0 0x7fa6a00aed70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa69400e500 tx=0x7fa69400e9c0 comp rx=0 tx=0).stop 2024-04-25T08:54:23.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.896+0000 7fa6a5cb3640 1 -- 172.21.15.43:0/769805982 shutdown_connections 2024-04-25T08:54:23.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.896+0000 7fa6a5cb3640 1 --2- 172.21.15.43:0/769805982 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa68c06b4d0 0x7fa68c06d990 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:23.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.896+0000 7fa6a5cb3640 1 --2- 172.21.15.43:0/769805982 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa6a00af2b0 0x7fa6a015ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:23.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.896+0000 7fa6a5cb3640 1 --2- 172.21.15.43:0/769805982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa6a015cba0 0x7fa6a00aed70 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:23.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.896+0000 7fa6a5cb3640 1 --2- 172.21.15.43:0/769805982 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa6880039b0 0x7fa6a0160c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:23.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.896+0000 7fa6a5cb3640 1 -- 172.21.15.43:0/769805982 >> 172.21.15.43:0/769805982 conn(0x7fa6a00b2490 msgr2=0x7fa6a006d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:23.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.896+0000 7fa6a5cb3640 1 -- 172.21.15.43:0/769805982 shutdown_connections 2024-04-25T08:54:23.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:23.896+0000 7fa6a5cb3640 1 -- 172.21.15.43:0/769805982 wait complete. 2024-04-25T08:54:23.897 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:54:23.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.007+0000 7f88731b7640 1 Processor -- start 2024-04-25T08:54:23.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.007+0000 7f88731b7640 1 -- start start 2024-04-25T08:54:23.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.008+0000 7f88731b7640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f886c0d24b0 0x7f886c0cffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:23.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.008+0000 7f88731b7640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f886c0d41b0 con 0x7f886c1631d0 2024-04-25T08:54:23.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.008+0000 7f88731b7640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f886c0d19d0 con 0x7f886c0ce090 2024-04-25T08:54:23.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.008+0000 7f88731b7640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f886c0d1cb0 con 0x7f886c0d24b0 2024-04-25T08:54:23.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.008+0000 7f8871754640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f886c0d24b0 0x7f886c0cffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:23.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.008+0000 7f886bfff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f886c1631d0 0x7f886c0ce470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:38228/0 (socket says 172.21.15.43:38228) 2024-04-25T08:54:23.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.008+0000 7f8871754640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f886c0d24b0 0x7f886c0cffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56124/0 (socket says 172.21.15.43:56124) 2024-04-25T08:54:23.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.008+0000 7f886bfff640 1 -- 172.21.15.43:0/3147001046 learned_addr learned my addr 172.21.15.43:0/3147001046 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:23.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.009+0000 7f8871754640 1 -- 172.21.15.43:0/3147001046 >> v1:172.21.15.193:6789/0 conn(0x7f886c0ce090 legacy=0x7f886c0d39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:23.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.009+0000 7f8871754640 1 -- 172.21.15.43:0/3147001046 >> v1:172.21.15.43:6789/0 conn(0x7f886c1631d0 legacy=0x7f886c0ce470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:23.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.009+0000 7f8871754640 1 -- 172.21.15.43:0/3147001046 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f886c0d1f90 con 0x7f886c0d24b0 2024-04-25T08:54:24.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.009+0000 7f8871754640 1 --2- 172.21.15.43:0/3147001046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f886c0d24b0 0x7f886c0cffa0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f885c0097a0 tx=0x7f885c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99269ac0865ba5c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:24.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.009+0000 7f886b7fe640 1 -- 172.21.15.43:0/3147001046 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885c037070 con 0x7f886c0d24b0 2024-04-25T08:54:24.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.009+0000 7f886b7fe640 1 -- 172.21.15.43:0/3147001046 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f885c032440 con 0x7f886c0d24b0 2024-04-25T08:54:24.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.010+0000 7f886b7fe640 1 -- 172.21.15.43:0/3147001046 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885c032810 con 0x7f886c0d24b0 2024-04-25T08:54:24.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.010+0000 7f88731b7640 1 -- 172.21.15.43:0/3147001046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f886c0d24b0 msgr2=0x7f886c0cffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:24.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.010+0000 7f88731b7640 1 --2- 172.21.15.43:0/3147001046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f886c0d24b0 0x7f886c0cffa0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f885c0097a0 tx=0x7f885c02a1e0 comp rx=0 tx=0).stop 2024-04-25T08:54:24.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.010+0000 7f88731b7640 1 -- 172.21.15.43:0/3147001046 shutdown_connections 2024-04-25T08:54:24.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.010+0000 7f88731b7640 1 --2- 172.21.15.43:0/3147001046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f886c0d24b0 0x7f886c0cffa0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:24.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.010+0000 7f88731b7640 1 -- 172.21.15.43:0/3147001046 >> 172.21.15.43:0/3147001046 conn(0x7f886c06b000 msgr2=0x7f886c06b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:24.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.010+0000 7f88731b7640 1 -- 172.21.15.43:0/3147001046 shutdown_connections 2024-04-25T08:54:24.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.010+0000 7f88731b7640 1 -- 172.21.15.43:0/3147001046 wait complete. 2024-04-25T08:54:24.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.011+0000 7f88731b7640 1 Processor -- start 2024-04-25T08:54:24.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.011+0000 7f88731b7640 1 -- start start 2024-04-25T08:54:24.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.012+0000 7f88731b7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f886c0ce090 0x7f886c1ebc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:24.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.012+0000 7f88731b7640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f886c0d24b0 0x7f886c14ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:24.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.012+0000 7f88731b7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f886c1631d0 0x7f886c0b84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:24.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.012+0000 7f88731b7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886c0d04e0 con 0x7f886c0ce090 2024-04-25T08:54:24.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.012+0000 7f88731b7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886c1623e0 con 0x7f886c1631d0 2024-04-25T08:54:24.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.012+0000 7f88731b7640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f886c0d0660 con 0x7f886c0d24b0 2024-04-25T08:54:24.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.012+0000 7f8870f53640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f886c0ce090 0x7f886c1ebc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:24.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.012+0000 7f8870f53640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f886c0ce090 0x7f886c1ebc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56128/0 (socket says 172.21.15.43:56128) 2024-04-25T08:54:24.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.012+0000 7f8870f53640 1 -- 172.21.15.43:0/3891938652 learned_addr learned my addr 172.21.15.43:0/3891938652 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:24.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.013+0000 7f886bfff640 1 --2- 172.21.15.43:0/3891938652 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f886c0d24b0 0x7f886c14ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:24.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.013+0000 7f8871754640 1 --2- 172.21.15.43:0/3891938652 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f886c1631d0 0x7f886c0b84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:24.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.013+0000 7f8870f53640 1 -- 172.21.15.43:0/3891938652 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f886c0d24b0 msgr2=0x7f886c14ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:24.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.013+0000 7f8870f53640 1 --2- 172.21.15.43:0/3891938652 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f886c0d24b0 0x7f886c14ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:24.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.013+0000 7f8870f53640 1 -- 172.21.15.43:0/3891938652 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f886c1631d0 msgr2=0x7f886c0b84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:24.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.013+0000 7f8870f53640 1 --2- 172.21.15.43:0/3891938652 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f886c1631d0 0x7f886c0b84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:24.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.013+0000 7f8870f53640 1 -- 172.21.15.43:0/3891938652 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f886c15e010 con 0x7f886c0ce090 2024-04-25T08:54:24.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.013+0000 7f8871754640 1 --2- 172.21.15.43:0/3891938652 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f886c1631d0 0x7f886c0b84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:24.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.013+0000 7f8870f53640 1 --2- 172.21.15.43:0/3891938652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f886c0ce090 0x7f886c1ebc00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f886400c7c0 tx=0x7f886400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:24.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.013+0000 7f88697fa640 1 -- 172.21.15.43:0/3891938652 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8864016020 con 0x7f886c0ce090 2024-04-25T08:54:24.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.013+0000 7f88697fa640 1 -- 172.21.15.43:0/3891938652 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886400aa50 con 0x7f886c0ce090 2024-04-25T08:54:24.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.013+0000 7f88697fa640 1 -- 172.21.15.43:0/3891938652 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8864012420 con 0x7f886c0ce090 2024-04-25T08:54:24.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.013+0000 7f88731b7640 1 -- 172.21.15.43:0/3891938652 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f886c1662a0 con 0x7f886c0ce090 2024-04-25T08:54:24.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.013+0000 7f88731b7640 1 -- 172.21.15.43:0/3891938652 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f886c166860 con 0x7f886c0ce090 2024-04-25T08:54:24.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.014+0000 7f88731b7640 1 -- 172.21.15.43:0/3891938652 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f886c074ef0 con 0x7f886c0ce090 2024-04-25T08:54:24.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.015+0000 7f88697fa640 1 -- 172.21.15.43:0/3891938652 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8864019020 con 0x7f886c0ce090 2024-04-25T08:54:24.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.015+0000 7f88697fa640 1 --2- 172.21.15.43:0/3891938652 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f885806b4d0 0x7f885806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:24.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.015+0000 7f88697fa640 1 -- 172.21.15.43:0/3891938652 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8864089ed0 con 0x7f886c0ce090 2024-04-25T08:54:24.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.015+0000 7f886bfff640 1 --2- 172.21.15.43:0/3891938652 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f885806b4d0 0x7f885806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:24.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.016+0000 7f886bfff640 1 --2- 172.21.15.43:0/3891938652 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f885806b4d0 0x7f885806d990 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f886c06b830 tx=0x7f8860009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:24.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.019+0000 7f88697fa640 1 -- 172.21.15.43:0/3891938652 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8864020070 con 0x7f886c0ce090 2024-04-25T08:54:24.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.203+0000 7f88731b7640 1 -- 172.21.15.43:0/3891938652 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f886c0d1610 con 0x7f886c0ce090 2024-04-25T08:54:24.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.204+0000 7f88697fa640 1 -- 172.21.15.43:0/3891938652 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16569 (secure 0 0 0) 0x7f8864058de0 con 0x7f886c0ce090 2024-04-25T08:54:24.194 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:54:24.194 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":20,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:54:13.593188+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:54:09.757978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:54:10.926008+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:54:24.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.206+0000 7f88731b7640 1 -- 172.21.15.43:0/3891938652 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f885806b4d0 msgr2=0x7f885806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:24.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.206+0000 7f88731b7640 1 --2- 172.21.15.43:0/3891938652 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f885806b4d0 0x7f885806d990 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f886c06b830 tx=0x7f8860009040 comp rx=0 tx=0).stop 2024-04-25T08:54:24.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.206+0000 7f88731b7640 1 -- 172.21.15.43:0/3891938652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f886c0ce090 msgr2=0x7f886c1ebc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:24.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.206+0000 7f88731b7640 1 --2- 172.21.15.43:0/3891938652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f886c0ce090 0x7f886c1ebc00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f886400c7c0 tx=0x7f886400cc80 comp rx=0 tx=0).stop 2024-04-25T08:54:24.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.206+0000 7f88731b7640 1 -- 172.21.15.43:0/3891938652 shutdown_connections 2024-04-25T08:54:24.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.206+0000 7f88731b7640 1 --2- 172.21.15.43:0/3891938652 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f885806b4d0 0x7f885806d990 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:24.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.206+0000 7f88731b7640 1 --2- 172.21.15.43:0/3891938652 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f886c1631d0 0x7f886c0b84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:24.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.206+0000 7f88731b7640 1 --2- 172.21.15.43:0/3891938652 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f886c0d24b0 0x7f886c14ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:24.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.206+0000 7f88731b7640 1 --2- 172.21.15.43:0/3891938652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f886c0ce090 0x7f886c1ebc00 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:24.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.206+0000 7f88731b7640 1 -- 172.21.15.43:0/3891938652 >> 172.21.15.43:0/3891938652 conn(0x7f886c06b000 msgr2=0x7f886c0ce470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:24.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.206+0000 7f88731b7640 1 -- 172.21.15.43:0/3891938652 shutdown_connections 2024-04-25T08:54:24.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.206+0000 7f88731b7640 1 -- 172.21.15.43:0/3891938652 wait complete. 2024-04-25T08:54:24.207 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:54:24.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.318+0000 7f56f05c8640 1 Processor -- start 2024-04-25T08:54:24.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.318+0000 7f56f05c8640 1 -- start start 2024-04-25T08:54:24.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.319+0000 7f56f05c8640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f56e815df70 0x7f56e80cc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:24.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.319+0000 7f56f05c8640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56e80b0870 0x7f56e80ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:24.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.319+0000 7f56f05c8640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56e80cbee0 con 0x7f56e806c510 2024-04-25T08:54:24.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.319+0000 7f56f05c8640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56e80d0400 con 0x7f56e815df70 2024-04-25T08:54:24.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.319+0000 7f56f05c8640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e80d0650 con 0x7f56e80b0870 2024-04-25T08:54:24.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.319+0000 7f56ee364640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f56e806c510 0x7f56e806c8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:38230/0 (socket says 172.21.15.43:38230) 2024-04-25T08:54:24.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.319+0000 7f56ee364640 1 -- 172.21.15.43:0/3489205935 learned_addr learned my addr 172.21.15.43:0/3489205935 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:24.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.319+0000 7f56eeb65640 1 --2- 172.21.15.43:0/3489205935 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56e80b0870 0x7f56e80ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:24.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.319+0000 7f56edb63640 1 --2- 172.21.15.43:0/3489205935 >> v2:172.21.15.193:3300/0 conn(0x7f56e815df70 0x7f56e80cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:24.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.319+0000 7f56ed362640 1 -- 172.21.15.43:0/3489205935 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1365182615 0 0) 0x7f56e80cbee0 con 0x7f56e806c510 2024-04-25T08:54:24.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.319+0000 7f56ed362640 1 -- 172.21.15.43:0/3489205935 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56d4003520 con 0x7f56e806c510 2024-04-25T08:54:24.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.319+0000 7f56ed362640 1 -- 172.21.15.43:0/3489205935 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3703332450 0 0) 0x7f56d4003520 con 0x7f56e806c510 2024-04-25T08:54:24.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.319+0000 7f56ed362640 1 -- 172.21.15.43:0/3489205935 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56e80b0870 msgr2=0x7f56e80ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:24.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.319+0000 7f56ed362640 1 --2- 172.21.15.43:0/3489205935 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56e80b0870 0x7f56e80ccbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:24.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.319+0000 7f56ed362640 1 -- 172.21.15.43:0/3489205935 >> v2:172.21.15.193:3300/0 conn(0x7f56e815df70 msgr2=0x7f56e80cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:24.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.319+0000 7f56ed362640 1 --2- 172.21.15.43:0/3489205935 >> v2:172.21.15.193:3300/0 conn(0x7f56e815df70 0x7f56e80cc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:24.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.319+0000 7f56ed362640 1 -- 172.21.15.43:0/3489205935 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e80d0930 con 0x7f56e806c510 2024-04-25T08:54:24.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.319+0000 7f56ed362640 1 -- 172.21.15.43:0/3489205935 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f56e4001670 con 0x7f56e806c510 2024-04-25T08:54:24.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.319+0000 7f56eeb65640 1 --2- 172.21.15.43:0/3489205935 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56e80b0870 0x7f56e80ccbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:54:24.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.319+0000 7f56ed362640 1 -- 172.21.15.43:0/3489205935 >> v1:172.21.15.43:6789/0 conn(0x7f56e806c510 legacy=0x7f56e806c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:24.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.319+0000 7f56ed362640 1 --2- 172.21.15.43:0/3489205935 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56e80b0870 0x7f56d40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:24.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.319+0000 7f56ed362640 1 --2- 172.21.15.43:0/3489205935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56d4003ef0 0x7f56d4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:24.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.319+0000 7f56ed362640 1 --2- 172.21.15.43:0/3489205935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56d4004d40 0x7f56d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:24.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.319+0000 7f56ed362640 1 -- 172.21.15.43:0/3489205935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e80d0400 con 0x7f56d4003ef0 2024-04-25T08:54:24.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.319+0000 7f56ed362640 1 -- 172.21.15.43:0/3489205935 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e80d0650 con 0x7f56d4004d40 2024-04-25T08:54:24.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.319+0000 7f56ed362640 1 -- 172.21.15.43:0/3489205935 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56d400b9c0 con 0x7f56e80b0870 2024-04-25T08:54:24.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.319+0000 7f56edb63640 1 --2- 172.21.15.43:0/3489205935 >> v2:172.21.15.193:3300/0 conn(0x7f56e815df70 0x7f56e80cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:54:24.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.320+0000 7f56ee364640 1 --2- 172.21.15.43:0/3489205935 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56e80b0870 0x7f56d40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:24.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.320+0000 7f56edb63640 1 --2- 172.21.15.43:0/3489205935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56d4003ef0 0x7f56d4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:24.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.320+0000 7f56eeb65640 1 --2- 172.21.15.43:0/3489205935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56d4004d40 0x7f56d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:24.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.320+0000 7f56edb63640 1 -- 172.21.15.43:0/3489205935 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56e80b0870 msgr2=0x7f56d40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:24.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.320+0000 7f56edb63640 1 --2- 172.21.15.43:0/3489205935 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56e80b0870 0x7f56d40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:24.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.320+0000 7f56edb63640 1 -- 172.21.15.43:0/3489205935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56d4004d40 msgr2=0x7f56d40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:24.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.320+0000 7f56edb63640 1 --2- 172.21.15.43:0/3489205935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56d4004d40 0x7f56d40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:24.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.320+0000 7f56edb63640 1 -- 172.21.15.43:0/3489205935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d4003520 con 0x7f56d4003ef0 2024-04-25T08:54:24.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.320+0000 7f56ee364640 1 --2- 172.21.15.43:0/3489205935 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56e80b0870 0x7f56d40039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:54:24.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.321+0000 7f56edb63640 1 --2- 172.21.15.43:0/3489205935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56d4003ef0 0x7f56d4004800 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f56dc009c40 tx=0x7f56d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73a6239a72401908 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:24.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.321+0000 7f56ed362640 1 -- 172.21.15.43:0/3489205935 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56dc009840 con 0x7f56d4003ef0 2024-04-25T08:54:24.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.321+0000 7f56ed362640 1 -- 172.21.15.43:0/3489205935 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56dc004020 con 0x7f56d4003ef0 2024-04-25T08:54:24.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.321+0000 7f56ed362640 1 -- 172.21.15.43:0/3489205935 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56dc0043d0 con 0x7f56d4003ef0 2024-04-25T08:54:24.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.321+0000 7f56f05c8640 1 -- 172.21.15.43:0/3489205935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56d4003ef0 msgr2=0x7f56d4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:24.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.321+0000 7f56f05c8640 1 --2- 172.21.15.43:0/3489205935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56d4003ef0 0x7f56d4004800 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f56dc009c40 tx=0x7f56d4005830 comp rx=0 tx=0).stop 2024-04-25T08:54:24.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.321+0000 7f56eeb65640 1 -- 172.21.15.43:0/3489205935 reap_dead start 2024-04-25T08:54:24.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.321+0000 7f56f05c8640 1 -- 172.21.15.43:0/3489205935 shutdown_connections 2024-04-25T08:54:24.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.321+0000 7f56f05c8640 1 -- 172.21.15.43:0/3489205935 >> 172.21.15.43:0/3489205935 conn(0x7f56e80d17f0 msgr2=0x7f56e80d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:24.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.321+0000 7f56f05c8640 1 -- 172.21.15.43:0/3489205935 shutdown_connections 2024-04-25T08:54:24.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.321+0000 7f56f05c8640 1 -- 172.21.15.43:0/3489205935 wait complete. 2024-04-25T08:54:24.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.322+0000 7f56f05c8640 1 Processor -- start 2024-04-25T08:54:24.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.323+0000 7f56f05c8640 1 -- start start 2024-04-25T08:54:24.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.323+0000 7f56f05c8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56d4003ef0 0x7f56e80c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:24.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.323+0000 7f56f05c8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56e81f6f00 0x7f56e80cac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:24.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.323+0000 7f56f05c8640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56e81f7380 0x7f56e80cb160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:24.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.323+0000 7f56f05c8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e806bd10 con 0x7f56e81f6f00 2024-04-25T08:54:24.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.323+0000 7f56f05c8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e80d0c10 con 0x7f56d4003ef0 2024-04-25T08:54:24.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.323+0000 7f56f05c8640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e806be90 con 0x7f56e81f7380 2024-04-25T08:54:24.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.323+0000 7f56edb63640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56e81f6f00 0x7f56e80cac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:24.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.323+0000 7f56edb63640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56e81f6f00 0x7f56e80cac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56146/0 (socket says 172.21.15.43:56146) 2024-04-25T08:54:24.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.323+0000 7f56edb63640 1 -- 172.21.15.43:0/304991334 learned_addr learned my addr 172.21.15.43:0/304991334 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:24.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.324+0000 7f56eeb65640 1 --2- 172.21.15.43:0/304991334 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56e81f7380 0x7f56e80cb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:24.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.324+0000 7f56ee364640 1 --2- 172.21.15.43:0/304991334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56d4003ef0 0x7f56e80c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:24.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.324+0000 7f56edb63640 1 -- 172.21.15.43:0/304991334 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56e81f7380 msgr2=0x7f56e80cb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:24.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.324+0000 7f56edb63640 1 --2- 172.21.15.43:0/304991334 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56e81f7380 0x7f56e80cb160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:24.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.324+0000 7f56edb63640 1 -- 172.21.15.43:0/304991334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56d4003ef0 msgr2=0x7f56e80c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:24.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.324+0000 7f56edb63640 1 --2- 172.21.15.43:0/304991334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56d4003ef0 0x7f56e80c98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:24.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.324+0000 7f56edb63640 1 -- 172.21.15.43:0/304991334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e80c9e10 con 0x7f56e81f6f00 2024-04-25T08:54:24.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.324+0000 7f56eeb65640 1 --2- 172.21.15.43:0/304991334 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56e81f7380 0x7f56e80cb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:24.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.324+0000 7f56edb63640 1 --2- 172.21.15.43:0/304991334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56e81f6f00 0x7f56e80cac20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f56dc0099b0 tx=0x7f56dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:24.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.324+0000 7f56daffd640 1 -- 172.21.15.43:0/304991334 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56dc040070 con 0x7f56e81f6f00 2024-04-25T08:54:24.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.324+0000 7f56daffd640 1 -- 172.21.15.43:0/304991334 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56dc003820 con 0x7f56e81f6f00 2024-04-25T08:54:24.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.324+0000 7f56daffd640 1 -- 172.21.15.43:0/304991334 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56dc003c10 con 0x7f56e81f6f00 2024-04-25T08:54:24.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.324+0000 7f56f05c8640 1 -- 172.21.15.43:0/304991334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56e80c8b10 con 0x7f56e81f6f00 2024-04-25T08:54:24.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.324+0000 7f56f05c8640 1 -- 172.21.15.43:0/304991334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56e80c90d0 con 0x7f56e81f6f00 2024-04-25T08:54:24.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.325+0000 7f56f05c8640 1 -- 172.21.15.43:0/304991334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56e8074ef0 con 0x7f56e81f6f00 2024-04-25T08:54:24.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.326+0000 7f56daffd640 1 -- 172.21.15.43:0/304991334 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f56dc043070 con 0x7f56e81f6f00 2024-04-25T08:54:24.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.326+0000 7f56daffd640 1 --2- 172.21.15.43:0/304991334 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f56cc06b4d0 0x7f56cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:24.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.326+0000 7f56daffd640 1 -- 172.21.15.43:0/304991334 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f56dc0aaad0 con 0x7f56e81f6f00 2024-04-25T08:54:24.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.326+0000 7f56ee364640 1 --2- 172.21.15.43:0/304991334 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f56cc06b4d0 0x7f56cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:24.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.327+0000 7f56ee364640 1 --2- 172.21.15.43:0/304991334 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f56cc06b4d0 0x7f56cc06d990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f56e4001030 tx=0x7f56e4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:24.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.330+0000 7f56daffd640 1 -- 172.21.15.43:0/304991334 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f56dc079950 con 0x7f56e81f6f00 2024-04-25T08:54:24.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.546+0000 7f56f05c8640 1 -- 172.21.15.43:0/304991334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f56e806d3e0 con 0x7f56e81f6f00 2024-04-25T08:54:24.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.547+0000 7f56daffd640 1 -- 172.21.15.43:0/304991334 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4938 (secure 0 0 0) 0x7f56dc079250 con 0x7f56e81f6f00 2024-04-25T08:54:24.537 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:54:24.537 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3616994003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":3616994003},{"type":"v1","addr":"172.21.15.43:6835","nonce":3616994003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/4022901908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4022901908},{"type":"v1","addr":"172.21.15.193:6835","nonce":4022901908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2823450590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2823450590},{"type":"v1","addr":"172.21.15.43:6834","nonce":2823450590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:54:11.945963+0000","modified":"2024-04-25T08:54:13.943002+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.193:6837/1197434598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1197434598},{"type":"v1","addr":"172.21.15.193:6837","nonce":1197434598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:54:24.537 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 6 2024-04-25T08:54:24.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.549+0000 7f56f05c8640 1 -- 172.21.15.43:0/304991334 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f56cc06b4d0 msgr2=0x7f56cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:24.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.549+0000 7f56f05c8640 1 --2- 172.21.15.43:0/304991334 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f56cc06b4d0 0x7f56cc06d990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f56e4001030 tx=0x7f56e4009040 comp rx=0 tx=0).stop 2024-04-25T08:54:24.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.549+0000 7f56f05c8640 1 -- 172.21.15.43:0/304991334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56e81f6f00 msgr2=0x7f56e80cac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:24.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.549+0000 7f56f05c8640 1 --2- 172.21.15.43:0/304991334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56e81f6f00 0x7f56e80cac20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f56dc0099b0 tx=0x7f56dc004730 comp rx=0 tx=0).stop 2024-04-25T08:54:24.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.549+0000 7f56f05c8640 1 -- 172.21.15.43:0/304991334 shutdown_connections 2024-04-25T08:54:24.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.549+0000 7f56f05c8640 1 --2- 172.21.15.43:0/304991334 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56e81f7380 0x7f56e80cb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:24.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.549+0000 7f56f05c8640 1 --2- 172.21.15.43:0/304991334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56e81f6f00 0x7f56e80cac20 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:24.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.549+0000 7f56f05c8640 1 --2- 172.21.15.43:0/304991334 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f56cc06b4d0 0x7f56cc06d990 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:24.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.549+0000 7f56f05c8640 1 --2- 172.21.15.43:0/304991334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56d4003ef0 0x7f56e80c98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:24.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.549+0000 7f56f05c8640 1 -- 172.21.15.43:0/304991334 >> 172.21.15.43:0/304991334 conn(0x7f56e80d17f0 msgr2=0x7f56e80ccf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:24.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.549+0000 7f56f05c8640 1 -- 172.21.15.43:0/304991334 shutdown_connections 2024-04-25T08:54:24.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:24.549+0000 7f56f05c8640 1 -- 172.21.15.43:0/304991334 wait complete. 2024-04-25T08:54:24.952 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:54:11.945963+0000', 'modified': '2024-04-25T08:54:13.943002+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4294}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4294': {'gid': 4294, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.193:6837/1197434598', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6836', 'nonce': 1197434598}, {'type': 'v1', 'addr': '172.21.15.193:6837', 'nonce': 1197434598}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:54:24.953 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:54:24.953 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T08:54:25.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.070+0000 7eff43e15640 1 Processor -- start 2024-04-25T08:54:25.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.070+0000 7eff43e15640 1 -- start start 2024-04-25T08:54:25.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.071+0000 7eff43e15640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff3c15df50 0x7eff3c1619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:25.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.071+0000 7eff43e15640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7eff3c161f30 0x7eff3c162370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:25.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.071+0000 7eff43e15640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7eff3c0c7cf0 0x7eff3c0c8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:25.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.071+0000 7eff43e15640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff3c15e330 con 0x7eff3c0c7cf0 2024-04-25T08:54:25.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.071+0000 7eff43e15640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff3c0d24a0 con 0x7eff3c161f30 2024-04-25T08:54:25.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.071+0000 7eff43e15640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c0d2740 con 0x7eff3c15df50 2024-04-25T08:54:25.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.071+0000 7eff423b2640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7eff3c0c7cf0 0x7eff3c0c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:25.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.071+0000 7eff423b2640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7eff3c0c7cf0 0x7eff3c0c8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56150/0 (socket says 172.21.15.43:56150) 2024-04-25T08:54:25.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.071+0000 7eff423b2640 1 -- 172.21.15.43:0/817760699 learned_addr learned my addr 172.21.15.43:0/817760699 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:25.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.071+0000 7eff41bb1640 1 --2- 172.21.15.43:0/817760699 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff3c15df50 0x7eff3c1619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:25.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.071+0000 7eff413b0640 1 --2- 172.21.15.43:0/817760699 >> v2:172.21.15.193:3300/0 conn(0x7eff3c161f30 0x7eff3c162370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:25.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.071+0000 7eff423b2640 1 -- 172.21.15.43:0/817760699 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff3c15df50 msgr2=0x7eff3c1619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:25.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.071+0000 7eff423b2640 1 --2- 172.21.15.43:0/817760699 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff3c15df50 0x7eff3c1619f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.071+0000 7eff423b2640 1 -- 172.21.15.43:0/817760699 >> v2:172.21.15.193:3300/0 conn(0x7eff3c161f30 msgr2=0x7eff3c162370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:25.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.071+0000 7eff423b2640 1 --2- 172.21.15.43:0/817760699 >> v2:172.21.15.193:3300/0 conn(0x7eff3c161f30 0x7eff3c162370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.071+0000 7eff423b2640 1 -- 172.21.15.43:0/817760699 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff3c0d2a20 con 0x7eff3c0c7cf0 2024-04-25T08:54:25.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.071+0000 7eff413b0640 1 --2- 172.21.15.43:0/817760699 >> v2:172.21.15.193:3300/0 conn(0x7eff3c161f30 0x7eff3c162370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:25.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.072+0000 7eff423b2640 1 --2- 172.21.15.43:0/817760699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff3c0c7cf0 0x7eff3c0c8100 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7eff3c08c950 tx=0x7eff3402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d67c1c6f829351d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:25.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.072+0000 7eff40baf640 1 -- 172.21.15.43:0/817760699 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff34036070 con 0x7eff3c0c7cf0 2024-04-25T08:54:25.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.072+0000 7eff40baf640 1 -- 172.21.15.43:0/817760699 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff34031600 con 0x7eff3c0c7cf0 2024-04-25T08:54:25.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.072+0000 7eff40baf640 1 -- 172.21.15.43:0/817760699 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff340319d0 con 0x7eff3c0c7cf0 2024-04-25T08:54:25.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.073+0000 7eff43e15640 1 -- 172.21.15.43:0/817760699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff3c0c7cf0 msgr2=0x7eff3c0c8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:25.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.073+0000 7eff43e15640 1 --2- 172.21.15.43:0/817760699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff3c0c7cf0 0x7eff3c0c8100 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7eff3c08c950 tx=0x7eff3402a6a0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.073+0000 7eff43e15640 1 -- 172.21.15.43:0/817760699 shutdown_connections 2024-04-25T08:54:25.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.073+0000 7eff43e15640 1 --2- 172.21.15.43:0/817760699 >> v2:172.21.15.193:3300/0 conn(0x7eff3c161f30 0x7eff3c162370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.073+0000 7eff43e15640 1 --2- 172.21.15.43:0/817760699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff3c0c7cf0 0x7eff3c0c8100 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.073+0000 7eff43e15640 1 --2- 172.21.15.43:0/817760699 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff3c15df50 0x7eff3c1619f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.073+0000 7eff43e15640 1 -- 172.21.15.43:0/817760699 >> 172.21.15.43:0/817760699 conn(0x7eff3c0d3a30 msgr2=0x7eff3c0d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:25.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.073+0000 7eff43e15640 1 -- 172.21.15.43:0/817760699 shutdown_connections 2024-04-25T08:54:25.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.073+0000 7eff43e15640 1 -- 172.21.15.43:0/817760699 wait complete. 2024-04-25T08:54:25.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.074+0000 7eff43e15640 1 Processor -- start 2024-04-25T08:54:25.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.074+0000 7eff43e15640 1 -- start start 2024-04-25T08:54:25.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.075+0000 7eff43e15640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff3c0c7cf0 0x7eff3c15f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:25.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.075+0000 7eff43e15640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff3c15df50 0x7eff3c15cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:25.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.075+0000 7eff43e15640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff3c161f30 0x7eff3c1e8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:25.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.075+0000 7eff43e15640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c0d2190 con 0x7eff3c15df50 2024-04-25T08:54:25.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.075+0000 7eff43e15640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c0c77d0 con 0x7eff3c0c7cf0 2024-04-25T08:54:25.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.075+0000 7eff43e15640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c0c8640 con 0x7eff3c161f30 2024-04-25T08:54:25.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.075+0000 7eff413b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff3c15df50 0x7eff3c15cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:25.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.075+0000 7eff413b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff3c15df50 0x7eff3c15cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56154/0 (socket says 172.21.15.43:56154) 2024-04-25T08:54:25.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.075+0000 7eff413b0640 1 -- 172.21.15.43:0/4107489091 learned_addr learned my addr 172.21.15.43:0/4107489091 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:25.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.075+0000 7eff423b2640 1 --2- 172.21.15.43:0/4107489091 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff3c161f30 0x7eff3c1e8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:25.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.075+0000 7eff41bb1640 1 --2- 172.21.15.43:0/4107489091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff3c0c7cf0 0x7eff3c15f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:25.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.075+0000 7eff413b0640 1 -- 172.21.15.43:0/4107489091 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff3c161f30 msgr2=0x7eff3c1e8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:25.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.075+0000 7eff413b0640 1 --2- 172.21.15.43:0/4107489091 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff3c161f30 0x7eff3c1e8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.075+0000 7eff413b0640 1 -- 172.21.15.43:0/4107489091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff3c0c7cf0 msgr2=0x7eff3c15f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:25.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.075+0000 7eff413b0640 1 --2- 172.21.15.43:0/4107489091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff3c0c7cf0 0x7eff3c15f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.075+0000 7eff413b0640 1 -- 172.21.15.43:0/4107489091 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff3c15a330 con 0x7eff3c15df50 2024-04-25T08:54:25.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.075+0000 7eff423b2640 1 --2- 172.21.15.43:0/4107489091 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff3c161f30 0x7eff3c1e8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:25.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.075+0000 7eff413b0640 1 --2- 172.21.15.43:0/4107489091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff3c15df50 0x7eff3c15cb90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7eff2c00c820 tx=0x7eff2c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:25.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.075+0000 7eff327fc640 1 -- 172.21.15.43:0/4107489091 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2c015070 con 0x7eff3c15df50 2024-04-25T08:54:25.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.076+0000 7eff327fc640 1 -- 172.21.15.43:0/4107489091 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff2c007a30 con 0x7eff3c15df50 2024-04-25T08:54:25.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.076+0000 7eff43e15640 1 -- 172.21.15.43:0/4107489091 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff3c15a610 con 0x7eff3c15df50 2024-04-25T08:54:25.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.076+0000 7eff327fc640 1 -- 172.21.15.43:0/4107489091 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2c007de0 con 0x7eff3c15df50 2024-04-25T08:54:25.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.076+0000 7eff43e15640 1 -- 172.21.15.43:0/4107489091 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff3c15abd0 con 0x7eff3c15df50 2024-04-25T08:54:25.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.077+0000 7eff327fc640 1 -- 172.21.15.43:0/4107489091 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7eff2c018070 con 0x7eff3c15df50 2024-04-25T08:54:25.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.078+0000 7eff327fc640 1 --2- 172.21.15.43:0/4107489091 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7eff2006b4d0 0x7eff2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:25.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.078+0000 7eff327fc640 1 -- 172.21.15.43:0/4107489091 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7eff2c08a0a0 con 0x7eff3c15df50 2024-04-25T08:54:25.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.078+0000 7eff41bb1640 1 --2- 172.21.15.43:0/4107489091 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7eff2006b4d0 0x7eff2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:25.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.078+0000 7eff41bb1640 1 --2- 172.21.15.43:0/4107489091 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7eff2006b4d0 0x7eff2006d990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7eff38006fd0 tx=0x7eff38008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:25.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.079+0000 7eff46bfa740 1 Processor -- start 2024-04-25T08:54:25.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.079+0000 7eff46bfa740 1 -- start start 2024-04-25T08:54:25.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.080+0000 7eff46bfa740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55888685f230 0x55888685f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:25.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.080+0000 7eff46bfa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55888687cad0 0x55888687d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:25.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.080+0000 7eff46bfa740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558886834f10 0x55888687b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:25.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.080+0000 7eff46bfa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x558886797a10 con 0x55888687cad0 2024-04-25T08:54:25.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.080+0000 7eff46bfa740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x558886740b30 con 0x55888685f230 2024-04-25T08:54:25.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.080+0000 7eff46bfa740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x558886793c50 con 0x558886834f10 2024-04-25T08:54:25.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.080+0000 7eff413b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55888687cad0 0x55888687d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:25.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.080+0000 7eff413b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55888687cad0 0x55888687d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56162/0 (socket says 172.21.15.43:56162) 2024-04-25T08:54:25.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.080+0000 7eff413b0640 1 -- 172.21.15.43:0/788763302 learned_addr learned my addr 172.21.15.43:0/788763302 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:25.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.080+0000 7eff423b2640 1 --2- 172.21.15.43:0/788763302 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558886834f10 0x55888687b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:25.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.080+0000 7eff41bb1640 1 --2- 172.21.15.43:0/788763302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55888685f230 0x55888685f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:25.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.080+0000 7eff413b0640 1 -- 172.21.15.43:0/788763302 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558886834f10 msgr2=0x55888687b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:25.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.080+0000 7eff413b0640 1 --2- 172.21.15.43:0/788763302 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558886834f10 0x55888687b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.080+0000 7eff413b0640 1 -- 172.21.15.43:0/788763302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55888685f230 msgr2=0x55888685f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:25.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.080+0000 7eff413b0640 1 --2- 172.21.15.43:0/788763302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55888685f230 0x55888685f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.080+0000 7eff413b0640 1 -- 172.21.15.43:0/788763302 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558886755e10 con 0x55888687cad0 2024-04-25T08:54:25.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.080+0000 7eff41bb1640 1 --2- 172.21.15.43:0/788763302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55888685f230 0x55888685f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:25.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.080+0000 7eff413b0640 1 --2- 172.21.15.43:0/788763302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55888687cad0 0x55888687d020 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7eff2c088680 tx=0x7eff2c0590e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cea76a25e7b98573 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:25.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.080+0000 7eff1f7fe640 1 -- 172.21.15.43:0/788763302 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2c01f070 con 0x55888687cad0 2024-04-25T08:54:25.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.080+0000 7eff1f7fe640 1 -- 172.21.15.43:0/788763302 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff2c00ce10 con 0x55888687cad0 2024-04-25T08:54:25.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.080+0000 7eff1f7fe640 1 -- 172.21.15.43:0/788763302 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2c054f20 con 0x55888687cad0 2024-04-25T08:54:25.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.081+0000 7eff46bfa740 1 -- 172.21.15.43:0/788763302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55888687cad0 msgr2=0x55888687d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:25.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.081+0000 7eff46bfa740 1 --2- 172.21.15.43:0/788763302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55888687cad0 0x55888687d020 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7eff2c088680 tx=0x7eff2c0590e0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.081+0000 7eff46bfa740 1 -- 172.21.15.43:0/788763302 shutdown_connections 2024-04-25T08:54:25.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.081+0000 7eff46bfa740 1 --2- 172.21.15.43:0/788763302 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558886834f10 0x55888687b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.081+0000 7eff46bfa740 1 --2- 172.21.15.43:0/788763302 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55888687cad0 0x55888687d020 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.081+0000 7eff46bfa740 1 --2- 172.21.15.43:0/788763302 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55888685f230 0x55888685f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.081+0000 7eff46bfa740 1 -- 172.21.15.43:0/788763302 >> 172.21.15.43:0/788763302 conn(0x558886831840 msgr2=0x55888687a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:25.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.081+0000 7eff46bfa740 1 -- 172.21.15.43:0/788763302 shutdown_connections 2024-04-25T08:54:25.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.081+0000 7eff46bfa740 1 -- 172.21.15.43:0/788763302 wait complete. 2024-04-25T08:54:25.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.081+0000 7eff46bfa740 1 -- start start 2024-04-25T08:54:25.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.082+0000 7eff46bfa740 1 Processor -- start 2024-04-25T08:54:25.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.082+0000 7eff46bfa740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55888685f230 0x5588868785e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:25.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.082+0000 7eff46bfa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55888687cad0 0x55888688b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:25.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.082+0000 7eff46bfa740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558886860f40 0x55888687d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:25.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.082+0000 7eff46bfa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x558886835cb0 con 0x55888687cad0 2024-04-25T08:54:25.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.082+0000 7eff46bfa740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588867540c0 con 0x55888685f230 2024-04-25T08:54:25.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.082+0000 7eff46bfa740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55888682f770 con 0x558886860f40 2024-04-25T08:54:25.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.082+0000 7eff46bfa740 10 client.0 _ll_register_callbacks cb 0x558886747ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:54:25.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.083+0000 7eff413b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55888687cad0 0x55888688b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:25.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.083+0000 7eff413b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55888687cad0 0x55888688b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56168/0 (socket says 172.21.15.43:56168) 2024-04-25T08:54:25.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.083+0000 7eff413b0640 1 -- 172.21.15.43:0/3027299897 learned_addr learned my addr 172.21.15.43:0/3027299897 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:25.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.083+0000 7eff423b2640 1 --2- 172.21.15.43:0/3027299897 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558886860f40 0x55888687d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:25.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.083+0000 7eff41bb1640 1 --2- 172.21.15.43:0/3027299897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55888685f230 0x5588868785e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:25.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.083+0000 7eff413b0640 1 -- 172.21.15.43:0/3027299897 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558886860f40 msgr2=0x55888687d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:25.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.083+0000 7eff413b0640 1 --2- 172.21.15.43:0/3027299897 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558886860f40 0x55888687d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.083+0000 7eff413b0640 1 -- 172.21.15.43:0/3027299897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55888685f230 msgr2=0x5588868785e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:25.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.083+0000 7eff413b0640 1 --2- 172.21.15.43:0/3027299897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55888685f230 0x5588868785e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.083+0000 7eff413b0640 1 -- 172.21.15.43:0/3027299897 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5588864ac2d0 con 0x55888687cad0 2024-04-25T08:54:25.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.083+0000 7eff41bb1640 1 --2- 172.21.15.43:0/3027299897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55888685f230 0x5588868785e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:25.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.083+0000 7eff413b0640 1 --2- 172.21.15.43:0/3027299897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55888687cad0 0x55888688b190 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7eff2c00c7f0 tx=0x7eff2c01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21753fe1ab95814d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:25.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.083+0000 7eff1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:54:25.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.083+0000 7eff1cff9640 1 -- 172.21.15.43:0/3027299897 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2c077300 con 0x55888687cad0 2024-04-25T08:54:25.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.083+0000 7eff1cff9640 1 -- 172.21.15.43:0/3027299897 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7eff2c0701c0 con 0x55888687cad0 2024-04-25T08:54:25.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.083+0000 7eff46bfa740 1 -- 172.21.15.43:0/3027299897 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff2c00ce10 con 0x55888687cad0 2024-04-25T08:54:25.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.083+0000 7eff1cff9640 1 -- 172.21.15.43:0/3027299897 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff2c0774a0 con 0x55888687cad0 2024-04-25T08:54:25.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.083+0000 7eff46bfa740 10 client.4635 fetch_fsmap learned FSMap version 6 2024-04-25T08:54:25.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.083+0000 7eff46bfa740 1 -- 172.21.15.43:0/3027299897 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x5588865766b0 con 0x55888687cad0 2024-04-25T08:54:25.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.083+0000 7eff1cff9640 1 -- 172.21.15.43:0/3027299897 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff2c00ce10 con 0x55888687cad0 2024-04-25T08:54:25.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.084+0000 7eff1cff9640 1 -- 172.21.15.43:0/3027299897 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 6) v1 ==== 3294+0+0 (secure 0 0 0) 0x7eff2c059be0 con 0x55888687cad0 2024-04-25T08:54:25.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.084+0000 7eff46bfa740 10 client.4635 fetch_fsmap finished waiting for FSMap version 6 2024-04-25T08:54:25.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.084+0000 7eff46bfa740 10 client.4635 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T08:54:25.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.084+0000 7eff46bfa740 20 client.4635 populate_metadata read hostname 'smithi043' 2024-04-25T08:54:25.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.084+0000 7eff46bfa740 1 --2- 172.21.15.43:0/3027299897 >> [v2:172.21.15.193:6836/1197434598,v1:172.21.15.193:6837/1197434598] conn(0x55888688dd00 0x55888688e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:25.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.084+0000 7eff46bfa740 4 client.4635 mds_command: new command op to 4294 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:54:25.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.084+0000 7eff46bfa740 1 -- 172.21.15.43:0/3027299897 --> [v2:172.21.15.193:6836/1197434598,v1:172.21.15.193:6837/1197434598] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5588867f6920 con 0x55888688dd00 2024-04-25T08:54:25.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.084+0000 7eff41bb1640 1 --2- 172.21.15.43:0/3027299897 >> [v2:172.21.15.193:6836/1197434598,v1:172.21.15.193:6837/1197434598] conn(0x55888688dd00 0x55888688e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:25.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.084+0000 7eff41bb1640 1 --2- 172.21.15.43:0/3027299897 >> [v2:172.21.15.193:6836/1197434598,v1:172.21.15.193:6837/1197434598] conn(0x55888688dd00 0x55888688e0e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4974056c1a512729 server_cookie=1e1e314d27f54973 in_seq=0 out_seq=0 2024-04-25T08:54:25.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.084+0000 7eff1cff9640 10 client.4635 ms_handle_connect on v2:172.21.15.193:6836/1197434598 2024-04-25T08:54:25.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.085+0000 7eff1cff9640 1 -- 172.21.15.43:0/3027299897 <== mds.0 v2:172.21.15.193:6836/1197434598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5588867f6920 con 0x55888688dd00 2024-04-25T08:54:25.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.085+0000 7eff1cff9640 10 client.4635 handle_command_reply: tid=0 2024-04-25T08:54:25.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.085+0000 7eff46bfa740 1 client.4635 shutdown 2024-04-25T08:54:25.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.085+0000 7eff46bfa740 1 -- 172.21.15.43:0/3027299897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55888687cad0 msgr2=0x55888688b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:25.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.085+0000 7eff46bfa740 1 --2- 172.21.15.43:0/3027299897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55888687cad0 0x55888688b190 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7eff2c00c7f0 tx=0x7eff2c01d000 comp rx=0 tx=0).stop 2024-04-25T08:54:25.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.085+0000 7eff46bfa740 1 -- 172.21.15.43:0/3027299897 shutdown_connections 2024-04-25T08:54:25.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.085+0000 7eff46bfa740 1 --2- 172.21.15.43:0/3027299897 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558886860f40 0x55888687d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.085+0000 7eff46bfa740 1 --2- 172.21.15.43:0/3027299897 >> [v2:172.21.15.193:6836/1197434598,v1:172.21.15.193:6837/1197434598] conn(0x55888688dd00 0x55888688e0e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.085+0000 7eff46bfa740 1 --2- 172.21.15.43:0/3027299897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55888687cad0 0x55888688b190 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.085+0000 7eff46bfa740 1 --2- 172.21.15.43:0/3027299897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55888685f230 0x5588868785e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.085+0000 7eff1cff9640 0 client.4635 ms_handle_reset on v2:172.21.15.193:6836/1197434598 2024-04-25T08:54:25.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.085+0000 7eff46bfa740 1 -- 172.21.15.43:0/3027299897 >> 172.21.15.43:0/3027299897 conn(0x558886834f10 msgr2=0x55888682eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:25.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.085+0000 7eff46bfa740 1 -- 172.21.15.43:0/3027299897 shutdown_connections 2024-04-25T08:54:25.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.085+0000 7eff46bfa740 1 -- 172.21.15.43:0/3027299897 wait complete. 2024-04-25T08:54:25.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.085+0000 7eff46bfa740 20 client.4635 trim_cache size 0 max 16384 2024-04-25T08:54:25.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.101+0000 7eff46bfa740 1 Processor -- start 2024-04-25T08:54:25.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.101+0000 7eff46bfa740 1 -- start start 2024-04-25T08:54:25.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.102+0000 7eff46bfa740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55888685f230 0x55888687cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:25.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.102+0000 7eff46bfa740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558886860f40 0x55888687d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:25.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.102+0000 7eff46bfa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55888687cad0 0x558886871cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:25.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.102+0000 7eff46bfa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c1ff110 con 0x55888687cad0 2024-04-25T08:54:25.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.102+0000 7eff46bfa740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c1fe570 con 0x55888685f230 2024-04-25T08:54:25.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.102+0000 7eff46bfa740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff3c1feb40 con 0x558886860f40 2024-04-25T08:54:25.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.102+0000 7eff423b2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55888687cad0 0x558886871cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:25.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.102+0000 7eff423b2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55888687cad0 0x558886871cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56172/0 (socket says 172.21.15.43:56172) 2024-04-25T08:54:25.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.102+0000 7eff423b2640 1 -- 172.21.15.43:0/1068636578 learned_addr learned my addr 172.21.15.43:0/1068636578 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:25.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.102+0000 7eff413b0640 1 --2- 172.21.15.43:0/1068636578 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558886860f40 0x55888687d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:25.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.102+0000 7eff41bb1640 1 --2- 172.21.15.43:0/1068636578 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55888685f230 0x55888687cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:25.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.102+0000 7eff423b2640 1 -- 172.21.15.43:0/1068636578 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558886860f40 msgr2=0x55888687d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:25.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.102+0000 7eff423b2640 1 --2- 172.21.15.43:0/1068636578 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558886860f40 0x55888687d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.102+0000 7eff423b2640 1 -- 172.21.15.43:0/1068636578 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55888685f230 msgr2=0x55888687cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:25.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.102+0000 7eff423b2640 1 --2- 172.21.15.43:0/1068636578 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55888685f230 0x55888687cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.102+0000 7eff423b2640 1 -- 172.21.15.43:0/1068636578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588865766b0 con 0x55888687cad0 2024-04-25T08:54:25.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.102+0000 7eff413b0640 1 --2- 172.21.15.43:0/1068636578 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558886860f40 0x55888687d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:25.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.103+0000 7eff423b2640 1 --2- 172.21.15.43:0/1068636578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55888687cad0 0x558886871cb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7eff340314b0 tx=0x7eff3402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48b2d0c58486bac2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:25.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.103+0000 7eff1effd640 1 -- 172.21.15.43:0/1068636578 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff34036040 con 0x55888687cad0 2024-04-25T08:54:25.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.103+0000 7eff1effd640 1 -- 172.21.15.43:0/1068636578 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff34042070 con 0x55888687cad0 2024-04-25T08:54:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.103+0000 7eff1effd640 1 -- 172.21.15.43:0/1068636578 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff34031b30 con 0x55888687cad0 2024-04-25T08:54:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.103+0000 7eff46bfa740 1 -- 172.21.15.43:0/1068636578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55888687cad0 msgr2=0x558886871cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.103+0000 7eff46bfa740 1 --2- 172.21.15.43:0/1068636578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55888687cad0 0x558886871cb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7eff340314b0 tx=0x7eff3402e000 comp rx=0 tx=0).stop 2024-04-25T08:54:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.103+0000 7eff46bfa740 1 -- 172.21.15.43:0/1068636578 shutdown_connections 2024-04-25T08:54:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.103+0000 7eff46bfa740 1 --2- 172.21.15.43:0/1068636578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55888687cad0 0x558886871cb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.103+0000 7eff46bfa740 1 --2- 172.21.15.43:0/1068636578 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558886860f40 0x55888687d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.103+0000 7eff46bfa740 1 --2- 172.21.15.43:0/1068636578 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55888685f230 0x55888687cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.103+0000 7eff46bfa740 1 -- 172.21.15.43:0/1068636578 >> 172.21.15.43:0/1068636578 conn(0x558886834f10 msgr2=0x55888682eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.103+0000 7eff46bfa740 1 -- 172.21.15.43:0/1068636578 shutdown_connections 2024-04-25T08:54:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.103+0000 7eff46bfa740 1 -- 172.21.15.43:0/1068636578 wait complete. 2024-04-25T08:54:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.104+0000 7eff46bfa740 1 -- start start 2024-04-25T08:54:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.104+0000 7eff46bfa740 1 Processor -- start 2024-04-25T08:54:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.105+0000 7eff46bfa740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55888685f230 0x55888689d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.105+0000 7eff46bfa740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558886860f40 0x558886886d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:25.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.105+0000 7eff46bfa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55888687cad0 0x5588868721d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:25.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.105+0000 7eff46bfa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x558886835cb0 con 0x55888687cad0 2024-04-25T08:54:25.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.105+0000 7eff46bfa740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588867540c0 con 0x558886860f40 2024-04-25T08:54:25.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.105+0000 7eff46bfa740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x558886871090 con 0x55888685f230 2024-04-25T08:54:25.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.105+0000 7eff46bfa740 10 client.0 _ll_register_callbacks cb 0x55888682fc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:54:25.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.105+0000 7eff423b2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55888687cad0 0x5588868721d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:25.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.105+0000 7eff423b2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55888687cad0 0x5588868721d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56182/0 (socket says 172.21.15.43:56182) 2024-04-25T08:54:25.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.105+0000 7eff423b2640 1 -- 172.21.15.43:0/61503638 learned_addr learned my addr 172.21.15.43:0/61503638 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:25.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.105+0000 7eff41bb1640 1 --2- 172.21.15.43:0/61503638 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55888685f230 0x55888689d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:25.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.105+0000 7eff413b0640 1 --2- 172.21.15.43:0/61503638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558886860f40 0x558886886d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:25.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.105+0000 7eff423b2640 1 -- 172.21.15.43:0/61503638 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55888685f230 msgr2=0x55888689d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:25.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.105+0000 7eff423b2640 1 --2- 172.21.15.43:0/61503638 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55888685f230 0x55888689d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.105+0000 7eff423b2640 1 -- 172.21.15.43:0/61503638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558886860f40 msgr2=0x558886886d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:25.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.105+0000 7eff423b2640 1 --2- 172.21.15.43:0/61503638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558886860f40 0x558886886d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.105+0000 7eff423b2640 1 -- 172.21.15.43:0/61503638 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55888685ee70 con 0x55888687cad0 2024-04-25T08:54:25.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.105+0000 7eff413b0640 1 --2- 172.21.15.43:0/61503638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558886860f40 0x558886886d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:25.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.105+0000 7eff423b2640 1 --2- 172.21.15.43:0/61503638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55888687cad0 0x5588868721d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7eff340093e0 tx=0x7eff34004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2391e34cef7c8165 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:25.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.105+0000 7eff1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:54:25.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.105+0000 7eff1cff9640 1 -- 172.21.15.43:0/61503638 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff3404e070 con 0x55888687cad0 2024-04-25T08:54:25.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.105+0000 7eff1cff9640 1 -- 172.21.15.43:0/61503638 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7eff340318d0 con 0x55888687cad0 2024-04-25T08:54:25.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.105+0000 7eff46bfa740 1 -- 172.21.15.43:0/61503638 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7eff34042070 con 0x55888687cad0 2024-04-25T08:54:25.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.105+0000 7eff1cff9640 1 -- 172.21.15.43:0/61503638 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff34036050 con 0x55888687cad0 2024-04-25T08:54:25.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.105+0000 7eff46bfa740 10 client.4641 fetch_fsmap learned FSMap version 6 2024-04-25T08:54:25.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.106+0000 7eff46bfa740 1 -- 172.21.15.43:0/61503638 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x558886894bb0 con 0x55888687cad0 2024-04-25T08:54:25.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.106+0000 7eff1cff9640 1 -- 172.21.15.43:0/61503638 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff34042070 con 0x55888687cad0 2024-04-25T08:54:25.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.106+0000 7eff1cff9640 1 -- 172.21.15.43:0/61503638 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 6) v1 ==== 3294+0+0 (secure 0 0 0) 0x7eff34008040 con 0x55888687cad0 2024-04-25T08:54:25.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.106+0000 7eff46bfa740 10 client.4641 fetch_fsmap finished waiting for FSMap version 6 2024-04-25T08:54:25.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.106+0000 7eff46bfa740 10 client.4641 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T08:54:25.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.106+0000 7eff46bfa740 20 client.4641 populate_metadata read hostname 'smithi043' 2024-04-25T08:54:25.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.106+0000 7eff46bfa740 1 --2- 172.21.15.43:0/61503638 >> [v2:172.21.15.193:6836/1197434598,v1:172.21.15.193:6837/1197434598] conn(0x55888687d0a0 0x55888687d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:25.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.106+0000 7eff46bfa740 4 client.4641 mds_command: new command op to 4294 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:54:25.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.106+0000 7eff46bfa740 1 -- 172.21.15.43:0/61503638 --> [v2:172.21.15.193:6836/1197434598,v1:172.21.15.193:6837/1197434598] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5588867a4f50 con 0x55888687d0a0 2024-04-25T08:54:25.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.106+0000 7eff41bb1640 1 --2- 172.21.15.43:0/61503638 >> [v2:172.21.15.193:6836/1197434598,v1:172.21.15.193:6837/1197434598] conn(0x55888687d0a0 0x55888687d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:25.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.106+0000 7eff41bb1640 1 --2- 172.21.15.43:0/61503638 >> [v2:172.21.15.193:6836/1197434598,v1:172.21.15.193:6837/1197434598] conn(0x55888687d0a0 0x55888687d480 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9020ed5c82fd519f server_cookie=263c0fedc983cdab in_seq=0 out_seq=0 2024-04-25T08:54:25.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.106+0000 7eff1cff9640 10 client.4641 ms_handle_connect on v2:172.21.15.193:6836/1197434598 2024-04-25T08:54:25.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.107+0000 7eff1cff9640 1 -- 172.21.15.43:0/61503638 <== mds.0 v2:172.21.15.193:6836/1197434598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x5588867a4f50 con 0x55888687d0a0 2024-04-25T08:54:25.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.107+0000 7eff1cff9640 10 client.4641 handle_command_reply: tid=0 2024-04-25T08:54:25.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.107+0000 7eff46bfa740 1 client.4641 shutdown 2024-04-25T08:54:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.107+0000 7eff46bfa740 1 -- 172.21.15.43:0/61503638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55888687cad0 msgr2=0x5588868721d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.107+0000 7eff46bfa740 1 --2- 172.21.15.43:0/61503638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55888687cad0 0x5588868721d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7eff340093e0 tx=0x7eff34004ab0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.107+0000 7eff46bfa740 1 -- 172.21.15.43:0/61503638 shutdown_connections 2024-04-25T08:54:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.107+0000 7eff46bfa740 1 --2- 172.21.15.43:0/61503638 >> [v2:172.21.15.193:6836/1197434598,v1:172.21.15.193:6837/1197434598] conn(0x55888687d0a0 0x55888687d480 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.107+0000 7eff46bfa740 1 --2- 172.21.15.43:0/61503638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55888687cad0 0x5588868721d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.107+0000 7eff46bfa740 1 --2- 172.21.15.43:0/61503638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558886860f40 0x558886886d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.107+0000 7eff1cff9640 0 client.4641 ms_handle_reset on v2:172.21.15.193:6836/1197434598 2024-04-25T08:54:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.107+0000 7eff46bfa740 1 --2- 172.21.15.43:0/61503638 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55888685f230 0x55888689d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.107+0000 7eff46bfa740 1 -- 172.21.15.43:0/61503638 >> 172.21.15.43:0/61503638 conn(0x558886834f10 msgr2=0x55888688b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.107+0000 7eff46bfa740 1 -- 172.21.15.43:0/61503638 shutdown_connections 2024-04-25T08:54:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.107+0000 7eff46bfa740 1 -- 172.21.15.43:0/61503638 wait complete. 2024-04-25T08:54:25.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.107+0000 7eff46bfa740 20 client.4641 trim_cache size 0 max 16384 2024-04-25T08:54:25.105 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T08:54:25.105 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "6c4bd50e-aa00-4a7f-8d09-352344ef5a4b", 2024-04-25T08:54:25.105 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-04-25T08:54:25.105 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 4294, 2024-04-25T08:54:25.105 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-04-25T08:54:25.105 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-04-25T08:54:25.105 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-04-25T08:54:25.105 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 12.521912368000001, 2024-04-25T08:54:25.105 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 6, 2024-04-25T08:54:25.105 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 20, 2024-04-25T08:54:25.105 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 20, 2024-04-25T08:54:25.105 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 19.845652683000001 2024-04-25T08:54:25.106 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T08:54:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.107+0000 7eff43e15640 1 -- 172.21.15.43:0/4107489091 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7eff2006b4d0 msgr2=0x7eff2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.107+0000 7eff43e15640 1 --2- 172.21.15.43:0/4107489091 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7eff2006b4d0 0x7eff2006d990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7eff38006fd0 tx=0x7eff38008040 comp rx=0 tx=0).stop 2024-04-25T08:54:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.107+0000 7eff43e15640 1 -- 172.21.15.43:0/4107489091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff3c15df50 msgr2=0x7eff3c15cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.107+0000 7eff43e15640 1 --2- 172.21.15.43:0/4107489091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff3c15df50 0x7eff3c15cb90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7eff2c00c820 tx=0x7eff2c00cce0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.107+0000 7eff43e15640 1 -- 172.21.15.43:0/4107489091 shutdown_connections 2024-04-25T08:54:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.107+0000 7eff43e15640 1 --2- 172.21.15.43:0/4107489091 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff3c161f30 0x7eff3c1e8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.107+0000 7eff43e15640 1 --2- 172.21.15.43:0/4107489091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff3c15df50 0x7eff3c15cb90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.107+0000 7eff43e15640 1 --2- 172.21.15.43:0/4107489091 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7eff2006b4d0 0x7eff2006d990 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.107+0000 7eff43e15640 1 --2- 172.21.15.43:0/4107489091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff3c0c7cf0 0x7eff3c15f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.107+0000 7eff43e15640 1 -- 172.21.15.43:0/4107489091 >> 172.21.15.43:0/4107489091 conn(0x7eff3c0d3a30 msgr2=0x7eff3c0ce660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.107+0000 7eff43e15640 1 -- 172.21.15.43:0/4107489091 shutdown_connections 2024-04-25T08:54:25.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.107+0000 7eff43e15640 1 -- 172.21.15.43:0/4107489091 wait complete. 2024-04-25T08:54:25.109 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-25T08:54:25.134 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-25T08:54:25.135 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-25T08:54:25.135 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-25T08:54:25.135 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T08:54:25.135 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:54:25.135 INFO:tasks.ceph_fuse:client.2 config is {} 2024-04-25T08:54:25.135 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T08:54:25.135 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T08:54:25.135 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-25T08:54:25.135 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T08:54:25.136 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T08:54:25.136 INFO:tasks.ceph_fuse:client.3 config is {} 2024-04-25T08:54:25.136 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T08:54:25.136 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T08:54:25.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:25.136 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-04-25T08:54:25.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:25.152 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link delete ceph-brx 2024-04-25T08:54:25.218 INFO:teuthology.orchestra.run.smithi043.stderr:Cannot find device "ceph-brx" 2024-04-25T08:54:25.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:54:25.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:25.220 DEBUG:teuthology.orchestra.run.smithi193:> ip netns list 2024-04-25T08:54:25.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:25.239 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip link delete ceph-brx 2024-04-25T08:54:25.308 INFO:teuthology.orchestra.run.smithi193.stderr:Cannot find device "ceph-brx" 2024-04-25T08:54:25.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:54:25.310 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-25T08:54:25.311 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:54:25.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.422+0000 7f50dde07640 1 Processor -- start 2024-04-25T08:54:25.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.422+0000 7f50dde07640 1 -- start start 2024-04-25T08:54:25.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.422+0000 7f50dde07640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f50d8161390 0x7f50d80b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:25.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.422+0000 7f50dde07640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f50d80d02a0 0x7f50d80d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:25.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.422+0000 7f50dde07640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50d80b4c30 con 0x7f50d80ce5e0 2024-04-25T08:54:25.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.422+0000 7f50dde07640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50d80d1910 con 0x7f50d8161390 2024-04-25T08:54:25.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.422+0000 7f50dde07640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50d80cf2c0 con 0x7f50d80d02a0 2024-04-25T08:54:25.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.423+0000 7f50d7fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f50d80ce5e0 0x7f50d80cea70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:38238/0 (socket says 172.21.15.43:38238) 2024-04-25T08:54:25.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.423+0000 7f50d7fff640 1 -- 172.21.15.43:0/3552212823 learned_addr learned my addr 172.21.15.43:0/3552212823 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:25.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.423+0000 7f50d6ffd640 1 --2- 172.21.15.43:0/3552212823 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f50d80d02a0 0x7f50d80d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:25.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.423+0000 7f50d77fe640 1 --2- 172.21.15.43:0/3552212823 >> v2:172.21.15.193:3300/0 conn(0x7f50d8161390 0x7f50d80b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:25.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.423+0000 7f50d67fc640 1 -- 172.21.15.43:0/3552212823 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 385565059 0 0) 0x7f50d0001490 con 0x7f50d80ce5e0 2024-04-25T08:54:25.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.423+0000 7f50d67fc640 1 -- 172.21.15.43:0/3552212823 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50c0003520 con 0x7f50d80ce5e0 2024-04-25T08:54:25.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.423+0000 7f50d77fe640 1 -- 172.21.15.43:0/3552212823 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f50d80d02a0 msgr2=0x7f50d80d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:25.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.423+0000 7f50d77fe640 1 --2- 172.21.15.43:0/3552212823 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f50d80d02a0 0x7f50d80d0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.423+0000 7f50d77fe640 1 -- 172.21.15.43:0/3552212823 >> v1:172.21.15.43:6789/0 conn(0x7f50d80ce5e0 legacy=0x7f50d80cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:25.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.423+0000 7f50d67fc640 1 -- 172.21.15.43:0/3552212823 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3624670317 0 0) 0x7f50c0003520 con 0x7f50d80ce5e0 2024-04-25T08:54:25.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.423+0000 7f50d77fe640 1 -- 172.21.15.43:0/3552212823 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d80cf5a0 con 0x7f50d8161390 2024-04-25T08:54:25.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.423+0000 7f50d6ffd640 1 --2- 172.21.15.43:0/3552212823 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f50d80d02a0 0x7f50d80d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:54:25.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.424+0000 7f50d77fe640 1 --2- 172.21.15.43:0/3552212823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50d8161390 0x7f50d80b6820 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f50c80097a0 tx=0x7f50c802a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9271a8407e307dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:25.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.424+0000 7f50d67fc640 1 -- 172.21.15.43:0/3552212823 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c8037070 con 0x7f50d8161390 2024-04-25T08:54:25.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.424+0000 7f50d67fc640 1 -- 172.21.15.43:0/3552212823 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50c802aba0 con 0x7f50d8161390 2024-04-25T08:54:25.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.424+0000 7f50d67fc640 1 -- 172.21.15.43:0/3552212823 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d0001bc0 con 0x7f50d8161390 2024-04-25T08:54:25.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.425+0000 7f50dde07640 1 -- 172.21.15.43:0/3552212823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50d8161390 msgr2=0x7f50d80b6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:25.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.425+0000 7f50dde07640 1 --2- 172.21.15.43:0/3552212823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50d8161390 0x7f50d80b6820 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f50c80097a0 tx=0x7f50c802a110 comp rx=0 tx=0).stop 2024-04-25T08:54:25.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.425+0000 7f50dde07640 1 -- 172.21.15.43:0/3552212823 shutdown_connections 2024-04-25T08:54:25.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.425+0000 7f50dde07640 1 --2- 172.21.15.43:0/3552212823 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f50d80d02a0 0x7f50d80d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.425+0000 7f50dde07640 1 --2- 172.21.15.43:0/3552212823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50d8161390 0x7f50d80b6820 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.425+0000 7f50dde07640 1 -- 172.21.15.43:0/3552212823 >> 172.21.15.43:0/3552212823 conn(0x7f50d809cd20 msgr2=0x7f50d80d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:25.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.425+0000 7f50dde07640 1 -- 172.21.15.43:0/3552212823 shutdown_connections 2024-04-25T08:54:25.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.425+0000 7f50dde07640 1 -- 172.21.15.43:0/3552212823 wait complete. 2024-04-25T08:54:25.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.426+0000 7f50dde07640 1 Processor -- start 2024-04-25T08:54:25.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.426+0000 7f50dde07640 1 -- start start 2024-04-25T08:54:25.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.427+0000 7f50dde07640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f50d80ce5e0 0x7f50d80cd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:25.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.427+0000 7f50dde07640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50d80d02a0 0x7f50d80cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:25.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.427+0000 7f50dde07640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50d8161390 0x7f50d81f6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:25.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.427+0000 7f50dde07640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d8161770 con 0x7f50d80d02a0 2024-04-25T08:54:25.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.427+0000 7f50dde07640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50d80d15a0 con 0x7f50d8161390 2024-04-25T08:54:25.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.427+0000 7f50dde07640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50d80d1da0 con 0x7f50d80ce5e0 2024-04-25T08:54:25.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.427+0000 7f50d6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50d80d02a0 0x7f50d80cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:25.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.427+0000 7f50d6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50d80d02a0 0x7f50d80cd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56196/0 (socket says 172.21.15.43:56196) 2024-04-25T08:54:25.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.427+0000 7f50d6ffd640 1 -- 172.21.15.43:0/1156545328 learned_addr learned my addr 172.21.15.43:0/1156545328 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:25.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.427+0000 7f50d77fe640 1 --2- 172.21.15.43:0/1156545328 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f50d80ce5e0 0x7f50d80cd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:25.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.427+0000 7f50d7fff640 1 --2- 172.21.15.43:0/1156545328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50d8161390 0x7f50d81f6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:25.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.427+0000 7f50d6ffd640 1 -- 172.21.15.43:0/1156545328 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f50d80ce5e0 msgr2=0x7f50d80cd0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:25.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.427+0000 7f50d6ffd640 1 --2- 172.21.15.43:0/1156545328 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f50d80ce5e0 0x7f50d80cd0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.427+0000 7f50d6ffd640 1 -- 172.21.15.43:0/1156545328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50d8161390 msgr2=0x7f50d81f6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:25.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.427+0000 7f50d6ffd640 1 --2- 172.21.15.43:0/1156545328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50d8161390 0x7f50d81f6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.427+0000 7f50d6ffd640 1 -- 172.21.15.43:0/1156545328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d81f3d50 con 0x7f50d80d02a0 2024-04-25T08:54:25.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.427+0000 7f50d77fe640 1 --2- 172.21.15.43:0/1156545328 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f50d80ce5e0 0x7f50d80cd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:25.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.427+0000 7f50d7fff640 1 --2- 172.21.15.43:0/1156545328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50d8161390 0x7f50d81f6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:25.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.427+0000 7f50d6ffd640 1 --2- 172.21.15.43:0/1156545328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50d80d02a0 0x7f50d80cd5f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f50cc00ebe0 tx=0x7f50cc00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:25.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.428+0000 7f50bffff640 1 -- 172.21.15.43:0/1156545328 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50cc017070 con 0x7f50d80d02a0 2024-04-25T08:54:25.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.428+0000 7f50bffff640 1 -- 172.21.15.43:0/1156545328 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50cc00cc30 con 0x7f50d80d02a0 2024-04-25T08:54:25.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.428+0000 7f50dde07640 1 -- 172.21.15.43:0/1156545328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50d81f4030 con 0x7f50d80d02a0 2024-04-25T08:54:25.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.428+0000 7f50bffff640 1 -- 172.21.15.43:0/1156545328 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50cc013470 con 0x7f50d80d02a0 2024-04-25T08:54:25.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.428+0000 7f50dde07640 1 -- 172.21.15.43:0/1156545328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50d80d47c0 con 0x7f50d80d02a0 2024-04-25T08:54:25.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.429+0000 7f50dde07640 1 -- 172.21.15.43:0/1156545328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50d80680f0 con 0x7f50d80d02a0 2024-04-25T08:54:25.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.429+0000 7f50bffff640 1 -- 172.21.15.43:0/1156545328 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f50cc01a070 con 0x7f50d80d02a0 2024-04-25T08:54:25.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.430+0000 7f50bffff640 1 --2- 172.21.15.43:0/1156545328 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f50c406b4d0 0x7f50c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:25.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.430+0000 7f50bffff640 1 -- 172.21.15.43:0/1156545328 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f50cc08a500 con 0x7f50d80d02a0 2024-04-25T08:54:25.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.430+0000 7f50d77fe640 1 --2- 172.21.15.43:0/1156545328 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f50c406b4d0 0x7f50c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:25.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.430+0000 7f50d77fe640 1 --2- 172.21.15.43:0/1156545328 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f50c406b4d0 0x7f50c406d990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f50c8034000 tx=0x7f50c802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:25.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.434+0000 7f50bffff640 1 -- 172.21.15.43:0/1156545328 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f50cc059380 con 0x7f50d80d02a0 2024-04-25T08:54:25.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.651+0000 7f50dde07640 1 -- 172.21.15.43:0/1156545328 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f50d8074ef0 con 0x7f50d80d02a0 2024-04-25T08:54:25.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.652+0000 7f50bffff640 1 -- 172.21.15.43:0/1156545328 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f50cc058c80 con 0x7f50d80d02a0 2024-04-25T08:54:25.642 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:54:25.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.653+0000 7f50dde07640 1 -- 172.21.15.43:0/1156545328 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f50c406b4d0 msgr2=0x7f50c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:25.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.653+0000 7f50dde07640 1 --2- 172.21.15.43:0/1156545328 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f50c406b4d0 0x7f50c406d990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f50c8034000 tx=0x7f50c802e000 comp rx=0 tx=0).stop 2024-04-25T08:54:25.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.653+0000 7f50dde07640 1 -- 172.21.15.43:0/1156545328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50d80d02a0 msgr2=0x7f50d80cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:25.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.653+0000 7f50dde07640 1 --2- 172.21.15.43:0/1156545328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50d80d02a0 0x7f50d80cd5f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f50cc00ebe0 tx=0x7f50cc00c520 comp rx=0 tx=0).stop 2024-04-25T08:54:25.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.654+0000 7f50dde07640 1 -- 172.21.15.43:0/1156545328 shutdown_connections 2024-04-25T08:54:25.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.654+0000 7f50dde07640 1 --2- 172.21.15.43:0/1156545328 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f50c406b4d0 0x7f50c406d990 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.654+0000 7f50dde07640 1 --2- 172.21.15.43:0/1156545328 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50d8161390 0x7f50d81f6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.654+0000 7f50dde07640 1 --2- 172.21.15.43:0/1156545328 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f50d80d02a0 0x7f50d80cd5f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.654+0000 7f50dde07640 1 --2- 172.21.15.43:0/1156545328 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f50d80ce5e0 0x7f50d80cd0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:25.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.654+0000 7f50dde07640 1 -- 172.21.15.43:0/1156545328 >> 172.21.15.43:0/1156545328 conn(0x7f50d809cd20 msgr2=0x7f50d8148940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:25.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.654+0000 7f50dde07640 1 -- 172.21.15.43:0/1156545328 shutdown_connections 2024-04-25T08:54:25.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:25.654+0000 7f50dde07640 1 -- 172.21.15.43:0/1156545328 wait complete. 2024-04-25T08:54:25.654 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:54:25.655 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:54:25.655 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-04-25T08:54:25.655 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:54:25.655 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:54:25.655 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:54:25.655 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:54:25.655 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:54:25.655 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:54:25.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:25.655 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-04-25T08:54:25.670 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:54:25.671 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:54:25.671 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:54:25.671 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:25.671 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-04-25T08:54:25.671 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:25.671 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:54:25.671 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-04-25T08:54:25.671 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-04-25T08:54:25.671 INFO:teuthology.orchestra.run.smithi043.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:54:25.671 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:54:25.671 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-04-25T08:54:25.671 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T08:54:25.671 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 42752sec preferred_lft 42752sec 2024-04-25T08:54:25.672 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link noprefixroute 2024-04-25T08:54:25.672 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:25.672 INFO:teuthology.orchestra.run.smithi043.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:54:25.672 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-04-25T08:54:25.672 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-04-25T08:54:25.672 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:54:25.672 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:54:25.672 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-04-25T08:54:25.672 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T08:54:25.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:25.673 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2024-04-25T08:54:25.673 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T08:54:25.673 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link add name ceph-brx type bridge 2024-04-25T08:54:25.673 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip addr flush dev ceph-brx 2024-04-25T08:54:25.673 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set ceph-brx up 2024-04-25T08:54:25.673 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T08:54:25.673 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T08:54:25.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:25.825 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:54:25.849 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-04-25T08:54:25.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:25.852 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-04-25T08:54:25.918 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-04-25T08:54:25.918 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:54:25.918 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T08:54:25.918 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T08:54:25.919 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:54:25.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:25.919 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2024-04-25T08:54:25.919 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T08:54:25.919 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:54:25.919 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:54:25.919 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:54:25.919 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T08:54:26.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:26.151 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-04-25T08:54:26.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:26.167 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list-id 2024-04-25T08:54:26.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:26.222 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2024-04-25T08:54:26.222 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T08:54:26.222 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:54:26.222 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T08:54:26.222 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T08:54:26.298 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T08:54:26.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:26.298 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2024-04-25T08:54:26.298 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T08:54:26.298 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T08:54:26.299 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T08:54:26.299 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T08:54:26.299 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T08:54:26.299 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T08:54:26.299 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T08:54:26.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:26.419 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2024-04-25T08:54:26.419 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T08:54:26.419 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-04-25T08:54:26.419 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T08:54:26.420 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T08:54:26.456 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T08:54:26.456 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:54:26.456 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:54:26.511 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:54:26.511 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:54:26.512 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:54:26.567 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-04-25T08:54:26.632 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:54:26.688 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-04-25T08:54:26.688 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-04-25T08:54:26.688 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-04-25T08:54:26.688 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-04-25T08:54:26.688 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-04-25T08:54:26.688 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-04-25T08:54:26.688 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-04-25T08:54:26.688 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-04-25T08:54:26.688 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-04-25T08:54:26.688 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-04-25T08:54:26.688 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-04-25T08:54:26.688 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/selinux 2024-04-25T08:54:26.689 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:54:26.689 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-04-25T08:54:26.689 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-04-25T08:54:26.689 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-04-25T08:54:26.689 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-04-25T08:54:26.689 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:54:26.689 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-04-25T08:54:26.689 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-04-25T08:54:26.689 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:54:26.689 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:54:26.689 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:54:26.689 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:54:26.689 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-04-25T08:54:26.689 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-04-25T08:54:26.689 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:54:26.690 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:54:26.690 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:54:26.690 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:54:26.690 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-04-25T08:54:26.690 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:54:26.690 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:54:26.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:26.690 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-04-25T08:54:26.743 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T08:54:26.743 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T08:54:26.785 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-04-25T08:54:26.810 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:54:26.865 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-04-25T08:54:26.865 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-04-25T08:54:26.865 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-04-25T08:54:26.866 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-04-25T08:54:26.866 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-04-25T08:54:26.866 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-04-25T08:54:26.866 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-04-25T08:54:26.866 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-04-25T08:54:26.866 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-04-25T08:54:26.866 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-04-25T08:54:26.866 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-04-25T08:54:26.866 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/selinux 2024-04-25T08:54:26.866 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:54:26.866 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-04-25T08:54:26.866 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-04-25T08:54:26.866 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-04-25T08:54:26.866 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-04-25T08:54:26.867 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:54:26.867 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-04-25T08:54:26.867 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-04-25T08:54:26.867 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:54:26.867 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:54:26.867 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:54:26.867 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:54:26.867 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-04-25T08:54:26.867 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-04-25T08:54:26.867 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:54:26.867 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:54:26.867 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:54:26.867 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:54:26.867 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-04-25T08:54:26.868 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:54:26.868 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:54:26.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:26.868 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-04-25T08:54:26.898 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:2024-04-25T08:54:26.908+0000 7f2cc760c480 -1 init, newargv = 0x557ae3e29730 newargc=15 2024-04-25T08:54:26.899 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[43192]: starting ceph client 2024-04-25T08:54:26.903 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[43192]: starting fuse 2024-04-25T08:54:26.914 INFO:teuthology.orchestra.run.smithi043.stdout:43 2024-04-25T08:54:26.914 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-25T08:54:26.914 DEBUG:teuthology.orchestra.run.smithi043:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:54:26.914 DEBUG:teuthology.orchestra.run.smithi043:> import glob 2024-04-25T08:54:26.915 DEBUG:teuthology.orchestra.run.smithi043:> import re 2024-04-25T08:54:26.915 DEBUG:teuthology.orchestra.run.smithi043:> import os 2024-04-25T08:54:26.915 DEBUG:teuthology.orchestra.run.smithi043:> import subprocess 2024-04-25T08:54:26.915 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T08:54:26.915 DEBUG:teuthology.orchestra.run.smithi043:> def _find_admin_socket(client_name): 2024-04-25T08:54:26.915 DEBUG:teuthology.orchestra.run.smithi043:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T08:54:26.915 DEBUG:teuthology.orchestra.run.smithi043:> files = glob.glob(asok_path) 2024-04-25T08:54:26.915 DEBUG:teuthology.orchestra.run.smithi043:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T08:54:26.915 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T08:54:26.915 DEBUG:teuthology.orchestra.run.smithi043:> # Given a non-glob path, it better be there 2024-04-25T08:54:26.915 DEBUG:teuthology.orchestra.run.smithi043:> if "*" not in asok_path: 2024-04-25T08:54:26.915 DEBUG:teuthology.orchestra.run.smithi043:> assert(len(files) == 1) 2024-04-25T08:54:26.915 DEBUG:teuthology.orchestra.run.smithi043:> return files[0] 2024-04-25T08:54:26.915 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T08:54:26.915 DEBUG:teuthology.orchestra.run.smithi043:> for f in files: 2024-04-25T08:54:26.915 DEBUG:teuthology.orchestra.run.smithi043:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:54:26.916 DEBUG:teuthology.orchestra.run.smithi043:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:54:26.916 DEBUG:teuthology.orchestra.run.smithi043:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:54:26.916 DEBUG:teuthology.orchestra.run.smithi043:> contents = proc_f.read() 2024-04-25T08:54:26.916 DEBUG:teuthology.orchestra.run.smithi043:> if mountpoint in contents: 2024-04-25T08:54:26.916 DEBUG:teuthology.orchestra.run.smithi043:> return f 2024-04-25T08:54:26.916 DEBUG:teuthology.orchestra.run.smithi043:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:54:26.916 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T08:54:26.916 DEBUG:teuthology.orchestra.run.smithi043:> print(_find_admin_socket("client.0")) 2024-04-25T08:54:26.916 DEBUG:teuthology.orchestra.run.smithi043:> ' 2024-04-25T08:54:27.034 INFO:teuthology.orchestra.run.smithi043.stdout:/var/run/ceph/ceph-client.0.43192.asok 2024-04-25T08:54:27.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:54:27.045 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.43192.asok 2024-04-25T08:54:27.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:27.046 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.43192.asok status 2024-04-25T08:54:27.182 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T08:54:27.183 INFO:teuthology.orchestra.run.smithi043.stdout: "metadata": { 2024-04-25T08:54:27.183 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:54:27.183 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:54:27.183 INFO:teuthology.orchestra.run.smithi043.stdout: "entity_id": "0", 2024-04-25T08:54:27.183 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": "smithi043", 2024-04-25T08:54:27.183 INFO:teuthology.orchestra.run.smithi043.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T08:54:27.183 INFO:teuthology.orchestra.run.smithi043.stdout: "pid": "43192", 2024-04-25T08:54:27.183 INFO:teuthology.orchestra.run.smithi043.stdout: "root": "/" 2024-04-25T08:54:27.183 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:54:27.183 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_count": 0, 2024-04-25T08:54:27.183 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_pinned_count": 0, 2024-04-25T08:54:27.183 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 4653, 2024-04-25T08:54:27.183 INFO:teuthology.orchestra.run.smithi043.stdout: "inst": { 2024-04-25T08:54:27.184 INFO:teuthology.orchestra.run.smithi043.stdout: "name": { 2024-04-25T08:54:27.184 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "client", 2024-04-25T08:54:27.184 INFO:teuthology.orchestra.run.smithi043.stdout: "num": 4653 2024-04-25T08:54:27.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:54:27.184 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-04-25T08:54:27.184 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-04-25T08:54:27.184 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:54:27.184 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 180376349 2024-04-25T08:54:27.184 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:54:27.184 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:54:27.184 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-04-25T08:54:27.184 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-04-25T08:54:27.184 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:54:27.184 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 180376349 2024-04-25T08:54:27.185 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:54:27.185 INFO:teuthology.orchestra.run.smithi043.stdout: "inst_str": "client.4653 192.168.0.1:0/180376349", 2024-04-25T08:54:27.185 INFO:teuthology.orchestra.run.smithi043.stdout: "addr_str": "192.168.0.1:0/180376349", 2024-04-25T08:54:27.185 INFO:teuthology.orchestra.run.smithi043.stdout: "inode_count": 1, 2024-04-25T08:54:27.185 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_epoch": 6, 2024-04-25T08:54:27.185 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch": 20, 2024-04-25T08:54:27.185 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:54:27.185 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklisted": false, 2024-04-25T08:54:27.185 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs" 2024-04-25T08:54:27.185 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T08:54:27.191 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:54:27.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.301+0000 7f0953d0a640 1 Processor -- start 2024-04-25T08:54:27.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.301+0000 7f0953d0a640 1 -- start start 2024-04-25T08:54:27.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.302+0000 7f0953d0a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f094c161390 0x7f094c0b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:27.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.302+0000 7f0953d0a640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f094c0d02a0 0x7f094c0d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:27.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.302+0000 7f0953d0a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f094c0ce5e0 0x7f094c0cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:27.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.302+0000 7f0953d0a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f094c0d1910 con 0x7f094c0ce5e0 2024-04-25T08:54:27.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.302+0000 7f0953d0a640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f094c0b4c30 con 0x7f094c0d02a0 2024-04-25T08:54:27.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.302+0000 7f0953d0a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f094c0cf2c0 con 0x7f094c161390 2024-04-25T08:54:27.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.302+0000 7f09522a7640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f094c0ce5e0 0x7f094c0cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:27.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.302+0000 7f09522a7640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f094c0ce5e0 0x7f094c0cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56202/0 (socket says 172.21.15.43:56202) 2024-04-25T08:54:27.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.302+0000 7f09522a7640 1 -- 172.21.15.43:0/3994305981 learned_addr learned my addr 172.21.15.43:0/3994305981 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:27.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.302+0000 7f09512a5640 1 --2- 172.21.15.43:0/3994305981 >> v2:172.21.15.193:3300/0 conn(0x7f094c0d02a0 0x7f094c0d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:27.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.302+0000 7f0951aa6640 1 --2- 172.21.15.43:0/3994305981 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f094c161390 0x7f094c0b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:27.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.302+0000 7f09522a7640 1 -- 172.21.15.43:0/3994305981 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f094c161390 msgr2=0x7f094c0b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:27.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.302+0000 7f09522a7640 1 --2- 172.21.15.43:0/3994305981 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f094c161390 0x7f094c0b6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.302+0000 7f09522a7640 1 -- 172.21.15.43:0/3994305981 >> v2:172.21.15.193:3300/0 conn(0x7f094c0d02a0 msgr2=0x7f094c0d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:27.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.302+0000 7f09522a7640 1 --2- 172.21.15.43:0/3994305981 >> v2:172.21.15.193:3300/0 conn(0x7f094c0d02a0 0x7f094c0d0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.302+0000 7f09522a7640 1 -- 172.21.15.43:0/3994305981 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f094c0cf5a0 con 0x7f094c0ce5e0 2024-04-25T08:54:27.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.302+0000 7f0951aa6640 1 --2- 172.21.15.43:0/3994305981 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f094c161390 0x7f094c0b6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:27.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.303+0000 7f09522a7640 1 --2- 172.21.15.43:0/3994305981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f094c0ce5e0 0x7f094c0cea70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f094c06d8a0 tx=0x7f093c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c62da4b2a612c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:27.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.303+0000 7f0950aa4640 1 -- 172.21.15.43:0/3994305981 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093c036070 con 0x7f094c0ce5e0 2024-04-25T08:54:27.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.303+0000 7f0950aa4640 1 -- 172.21.15.43:0/3994305981 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f093c031600 con 0x7f094c0ce5e0 2024-04-25T08:54:27.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.303+0000 7f0950aa4640 1 -- 172.21.15.43:0/3994305981 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093c0319d0 con 0x7f094c0ce5e0 2024-04-25T08:54:27.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.304+0000 7f0953d0a640 1 -- 172.21.15.43:0/3994305981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f094c0ce5e0 msgr2=0x7f094c0cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:27.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.304+0000 7f0953d0a640 1 --2- 172.21.15.43:0/3994305981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f094c0ce5e0 0x7f094c0cea70 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f094c06d8a0 tx=0x7f093c02a6a0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.304+0000 7f0953d0a640 1 -- 172.21.15.43:0/3994305981 shutdown_connections 2024-04-25T08:54:27.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.304+0000 7f0953d0a640 1 --2- 172.21.15.43:0/3994305981 >> v2:172.21.15.193:3300/0 conn(0x7f094c0d02a0 0x7f094c0d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.304+0000 7f0953d0a640 1 --2- 172.21.15.43:0/3994305981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f094c0ce5e0 0x7f094c0cea70 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.304+0000 7f0953d0a640 1 --2- 172.21.15.43:0/3994305981 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f094c161390 0x7f094c0b6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.304+0000 7f0953d0a640 1 -- 172.21.15.43:0/3994305981 >> 172.21.15.43:0/3994305981 conn(0x7f094c09cd20 msgr2=0x7f094c0d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:27.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.304+0000 7f0953d0a640 1 -- 172.21.15.43:0/3994305981 shutdown_connections 2024-04-25T08:54:27.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.304+0000 7f0953d0a640 1 -- 172.21.15.43:0/3994305981 wait complete. 2024-04-25T08:54:27.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.305+0000 7f0953d0a640 1 Processor -- start 2024-04-25T08:54:27.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.305+0000 7f0953d0a640 1 -- start start 2024-04-25T08:54:27.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.306+0000 7f0953d0a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f094c0ce5e0 0x7f094c14d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:27.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.306+0000 7f0953d0a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f094c0d02a0 0x7f094c0cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:27.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.306+0000 7f0953d0a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f094c161390 0x7f094c0b0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:27.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.306+0000 7f0953d0a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f094c161770 con 0x7f094c161390 2024-04-25T08:54:27.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.306+0000 7f0953d0a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f094c0d15a0 con 0x7f094c0d02a0 2024-04-25T08:54:27.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.306+0000 7f0953d0a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f094c0cefb0 con 0x7f094c0ce5e0 2024-04-25T08:54:27.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.306+0000 7f09522a7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f094c161390 0x7f094c0b0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:27.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.306+0000 7f09522a7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f094c161390 0x7f094c0b0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56210/0 (socket says 172.21.15.43:56210) 2024-04-25T08:54:27.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.306+0000 7f09522a7640 1 -- 172.21.15.43:0/826724162 learned_addr learned my addr 172.21.15.43:0/826724162 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:27.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.306+0000 7f0951aa6640 1 --2- 172.21.15.43:0/826724162 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f094c0ce5e0 0x7f094c14d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:27.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.306+0000 7f09512a5640 1 --2- 172.21.15.43:0/826724162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f094c0d02a0 0x7f094c0cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:27.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.306+0000 7f09522a7640 1 -- 172.21.15.43:0/826724162 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f094c0ce5e0 msgr2=0x7f094c14d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:27.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.306+0000 7f09522a7640 1 --2- 172.21.15.43:0/826724162 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f094c0ce5e0 0x7f094c14d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.306+0000 7f09522a7640 1 -- 172.21.15.43:0/826724162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f094c0d02a0 msgr2=0x7f094c0cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:27.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.306+0000 7f09522a7640 1 --2- 172.21.15.43:0/826724162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f094c0d02a0 0x7f094c0cbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.306+0000 7f09522a7640 1 -- 172.21.15.43:0/826724162 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f094c0cc460 con 0x7f094c161390 2024-04-25T08:54:27.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.306+0000 7f09522a7640 1 --2- 172.21.15.43:0/826724162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f094c161390 0x7f094c0b0680 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f093c034040 tx=0x7f093c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:27.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.307+0000 7f094a7fc640 1 -- 172.21.15.43:0/826724162 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093c041020 con 0x7f094c161390 2024-04-25T08:54:27.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.307+0000 7f094a7fc640 1 -- 172.21.15.43:0/826724162 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f093c0077a0 con 0x7f094c161390 2024-04-25T08:54:27.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.307+0000 7f0953d0a640 1 -- 172.21.15.43:0/826724162 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f094c0cc740 con 0x7f094c161390 2024-04-25T08:54:27.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.307+0000 7f094a7fc640 1 -- 172.21.15.43:0/826724162 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093c036040 con 0x7f094c161390 2024-04-25T08:54:27.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.307+0000 7f0953d0a640 1 -- 172.21.15.43:0/826724162 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f094c0ccd00 con 0x7f094c161390 2024-04-25T08:54:27.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.308+0000 7f0953d0a640 1 -- 172.21.15.43:0/826724162 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f094c0680f0 con 0x7f094c161390 2024-04-25T08:54:27.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.308+0000 7f094a7fc640 1 -- 172.21.15.43:0/826724162 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f093c044020 con 0x7f094c161390 2024-04-25T08:54:27.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.309+0000 7f094a7fc640 1 --2- 172.21.15.43:0/826724162 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f093806b4d0 0x7f093806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:27.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.309+0000 7f094a7fc640 1 -- 172.21.15.43:0/826724162 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f093c0aaea0 con 0x7f094c161390 2024-04-25T08:54:27.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.309+0000 7f0951aa6640 1 --2- 172.21.15.43:0/826724162 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f093806b4d0 0x7f093806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:27.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.309+0000 7f0951aa6640 1 --2- 172.21.15.43:0/826724162 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f093806b4d0 0x7f093806d990 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f094400a990 tx=0x7f0944008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:27.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.313+0000 7f094a7fc640 1 -- 172.21.15.43:0/826724162 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f093c04b030 con 0x7f094c161390 2024-04-25T08:54:27.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.530+0000 7f0953d0a640 1 -- 172.21.15.43:0/826724162 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f094c074ef0 con 0x7f094c161390 2024-04-25T08:54:27.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.531+0000 7f094a7fc640 1 -- 172.21.15.43:0/826724162 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f093c079db0 con 0x7f094c161390 2024-04-25T08:54:27.521 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:54:27.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.532+0000 7f0953d0a640 1 -- 172.21.15.43:0/826724162 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f093806b4d0 msgr2=0x7f093806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:27.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.532+0000 7f0953d0a640 1 --2- 172.21.15.43:0/826724162 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f093806b4d0 0x7f093806d990 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f094400a990 tx=0x7f0944008040 comp rx=0 tx=0).stop 2024-04-25T08:54:27.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.532+0000 7f0953d0a640 1 -- 172.21.15.43:0/826724162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f094c161390 msgr2=0x7f094c0b0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:27.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.532+0000 7f0953d0a640 1 --2- 172.21.15.43:0/826724162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f094c161390 0x7f094c0b0680 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f093c034040 tx=0x7f093c004a40 comp rx=0 tx=0).stop 2024-04-25T08:54:27.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.532+0000 7f0953d0a640 1 -- 172.21.15.43:0/826724162 shutdown_connections 2024-04-25T08:54:27.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.532+0000 7f0953d0a640 1 --2- 172.21.15.43:0/826724162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f094c161390 0x7f094c0b0680 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.532+0000 7f0953d0a640 1 --2- 172.21.15.43:0/826724162 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f093806b4d0 0x7f093806d990 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.532+0000 7f0953d0a640 1 --2- 172.21.15.43:0/826724162 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f094c0d02a0 0x7f094c0cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.532+0000 7f0953d0a640 1 --2- 172.21.15.43:0/826724162 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f094c0ce5e0 0x7f094c14d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:27.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.532+0000 7f0953d0a640 1 -- 172.21.15.43:0/826724162 >> 172.21.15.43:0/826724162 conn(0x7f094c09cd20 msgr2=0x7f094c0b65c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:27.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.532+0000 7f0953d0a640 1 -- 172.21.15.43:0/826724162 shutdown_connections 2024-04-25T08:54:27.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:27.533+0000 7f0953d0a640 1 -- 172.21.15.43:0/826724162 wait complete. 2024-04-25T08:54:27.533 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:54:27.533 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:54:27.533 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-04-25T08:54:27.533 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-25T08:54:27.533 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T08:54:27.533 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:54:27.533 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:54:27.533 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:54:27.533 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-25T08:54:27.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:27.534 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-04-25T08:54:27.549 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:54:27.549 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:54:27.549 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:54:27.549 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:27.550 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-04-25T08:54:27.550 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:27.550 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:54:27.550 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-04-25T08:54:27.550 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-04-25T08:54:27.550 INFO:teuthology.orchestra.run.smithi043.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:54:27.550 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:54:27.550 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-04-25T08:54:27.550 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T08:54:27.550 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 42750sec preferred_lft 42750sec 2024-04-25T08:54:27.550 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link noprefixroute 2024-04-25T08:54:27.550 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:27.550 INFO:teuthology.orchestra.run.smithi043.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:54:27.551 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-04-25T08:54:27.551 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-04-25T08:54:27.551 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:54:27.551 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:54:27.551 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-04-25T08:54:27.551 INFO:teuthology.orchestra.run.smithi043.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:54:27.551 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 16:9a:63:37:46:24 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:54:27.551 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:54:27.551 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:27.551 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ccff:4eff:fe9d:c8bf/64 scope link 2024-04-25T08:54:27.551 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:27.551 INFO:teuthology.orchestra.run.smithi043.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:54:27.551 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 16:9a:63:37:46:24 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:54:27.551 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::149a:63ff:fe37:4624/64 scope link tentative 2024-04-25T08:54:27.552 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:27.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:27.552 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:54:27.613 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-04-25T08:54:27.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:27.615 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-04-25T08:54:27.671 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-04-25T08:54:27.671 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:54:27.671 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T08:54:27.671 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T08:54:27.671 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:54:27.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:27.672 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2024-04-25T08:54:27.672 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T08:54:27.672 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:54:27.672 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:54:27.672 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:54:27.672 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T08:54:27.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:27.759 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-04-25T08:54:27.774 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:54:27.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:27.774 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list-id 2024-04-25T08:54:27.829 INFO:teuthology.orchestra.run.smithi043.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-25T08:54:27.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:27.830 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2024-04-25T08:54:27.830 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T08:54:27.830 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:54:27.830 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-25T08:54:27.830 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T08:54:27.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:27.907 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T08:54:27.971 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:54:27.971 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:54:27.971 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:54:27.971 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:27.971 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-04-25T08:54:27.971 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:27.971 INFO:teuthology.orchestra.run.smithi043.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:54:27.971 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 4e:69:49:fe:4d:73 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T08:54:27.971 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T08:54:27.972 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:27.972 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::4c69:49ff:fefe:4d73/64 scope link 2024-04-25T08:54:27.972 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:27.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:27.973 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T08:54:28.036 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:54:28.036 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:54:28.036 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:54:28.036 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:28.037 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-04-25T08:54:28.037 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:28.037 INFO:teuthology.orchestra.run.smithi043.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:54:28.037 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 4e:69:49:fe:4d:73 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T08:54:28.037 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T08:54:28.037 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:28.037 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::4c69:49ff:fefe:4d73/64 scope link 2024-04-25T08:54:28.037 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:28.039 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-25T08:54:28.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:28.040 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2024-04-25T08:54:28.040 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T08:54:28.040 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-25T08:54:28.040 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T08:54:28.040 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-25T08:54:28.040 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-25T08:54:28.040 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-25T08:54:28.040 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T08:54:28.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:28.149 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2024-04-25T08:54:28.149 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T08:54:28.149 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.1 up 2024-04-25T08:54:28.149 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T08:54:28.149 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T08:54:28.185 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-04-25T08:54:28.185 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:54:28.185 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-25T08:54:28.241 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-25T08:54:28.241 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:54:28.241 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-25T08:54:28.297 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-04-25T08:54:28.362 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:54:28.418 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-04-25T08:54:28.418 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-04-25T08:54:28.418 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-04-25T08:54:28.418 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-04-25T08:54:28.418 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-04-25T08:54:28.419 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-04-25T08:54:28.419 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-04-25T08:54:28.419 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-04-25T08:54:28.419 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-04-25T08:54:28.419 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-04-25T08:54:28.419 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-04-25T08:54:28.419 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/selinux 2024-04-25T08:54:28.419 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:54:28.419 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-04-25T08:54:28.419 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-04-25T08:54:28.419 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-04-25T08:54:28.419 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-04-25T08:54:28.419 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:54:28.420 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-04-25T08:54:28.420 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-04-25T08:54:28.420 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:54:28.420 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:54:28.420 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:54:28.420 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:54:28.420 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-04-25T08:54:28.420 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-04-25T08:54:28.420 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:54:28.420 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:54:28.420 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:54:28.420 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:54:28.420 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-04-25T08:54:28.420 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:54:28.421 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:54:28.421 INFO:teuthology.orchestra.run.smithi043.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T08:54:28.421 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:54:28.421 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:54:28.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:28.421 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-04-25T08:54:28.474 INFO:teuthology.orchestra.run.smithi043.stdout:43 2024-04-25T08:54:28.474 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2024-04-25T08:54:28.474 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-04-25T08:54:28.516 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-04-25T08:54:28.540 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:54:28.597 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-04-25T08:54:28.598 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-04-25T08:54:28.598 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-04-25T08:54:28.598 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-04-25T08:54:28.598 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-04-25T08:54:28.598 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-04-25T08:54:28.598 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-04-25T08:54:28.598 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-04-25T08:54:28.598 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-04-25T08:54:28.598 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-04-25T08:54:28.598 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-04-25T08:54:28.598 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/selinux 2024-04-25T08:54:28.598 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:54:28.599 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-04-25T08:54:28.599 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-04-25T08:54:28.599 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-04-25T08:54:28.599 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-04-25T08:54:28.599 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:54:28.599 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-04-25T08:54:28.599 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-04-25T08:54:28.599 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:54:28.599 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:54:28.599 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:54:28.599 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:54:28.599 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-04-25T08:54:28.599 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-04-25T08:54:28.599 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:54:28.600 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:54:28.600 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:54:28.600 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:54:28.600 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-04-25T08:54:28.600 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:54:28.600 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:54:28.600 INFO:teuthology.orchestra.run.smithi043.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-25T08:54:28.600 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:54:28.600 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:54:28.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:28.601 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-04-25T08:54:28.602 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi043.stderr:2024-04-25T08:54:28.609+0000 7ff8b2e21480 -1 init, newargv = 0x55fb0ef76910 newargc=15 2024-04-25T08:54:28.602 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi043.stderr:ceph-fuse[43668]: starting ceph client 2024-04-25T08:54:28.604 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi043.stderr:ceph-fuse[43668]: starting fuse 2024-04-25T08:54:28.618 INFO:teuthology.orchestra.run.smithi043.stdout:43 2024-04-25T08:54:28.618 INFO:teuthology.orchestra.run.smithi043.stdout:44 2024-04-25T08:54:28.618 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2024-04-25T08:54:28.619 DEBUG:teuthology.orchestra.run.smithi043:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:54:28.619 DEBUG:teuthology.orchestra.run.smithi043:> import glob 2024-04-25T08:54:28.619 DEBUG:teuthology.orchestra.run.smithi043:> import re 2024-04-25T08:54:28.619 DEBUG:teuthology.orchestra.run.smithi043:> import os 2024-04-25T08:54:28.619 DEBUG:teuthology.orchestra.run.smithi043:> import subprocess 2024-04-25T08:54:28.619 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T08:54:28.619 DEBUG:teuthology.orchestra.run.smithi043:> def _find_admin_socket(client_name): 2024-04-25T08:54:28.619 DEBUG:teuthology.orchestra.run.smithi043:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-04-25T08:54:28.619 DEBUG:teuthology.orchestra.run.smithi043:> files = glob.glob(asok_path) 2024-04-25T08:54:28.619 DEBUG:teuthology.orchestra.run.smithi043:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-04-25T08:54:28.619 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T08:54:28.619 DEBUG:teuthology.orchestra.run.smithi043:> # Given a non-glob path, it better be there 2024-04-25T08:54:28.619 DEBUG:teuthology.orchestra.run.smithi043:> if "*" not in asok_path: 2024-04-25T08:54:28.620 DEBUG:teuthology.orchestra.run.smithi043:> assert(len(files) == 1) 2024-04-25T08:54:28.620 DEBUG:teuthology.orchestra.run.smithi043:> return files[0] 2024-04-25T08:54:28.620 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T08:54:28.620 DEBUG:teuthology.orchestra.run.smithi043:> for f in files: 2024-04-25T08:54:28.620 DEBUG:teuthology.orchestra.run.smithi043:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:54:28.620 DEBUG:teuthology.orchestra.run.smithi043:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:54:28.620 DEBUG:teuthology.orchestra.run.smithi043:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:54:28.620 DEBUG:teuthology.orchestra.run.smithi043:> contents = proc_f.read() 2024-04-25T08:54:28.620 DEBUG:teuthology.orchestra.run.smithi043:> if mountpoint in contents: 2024-04-25T08:54:28.620 DEBUG:teuthology.orchestra.run.smithi043:> return f 2024-04-25T08:54:28.620 DEBUG:teuthology.orchestra.run.smithi043:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:54:28.620 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T08:54:28.620 DEBUG:teuthology.orchestra.run.smithi043:> print(_find_admin_socket("client.2")) 2024-04-25T08:54:28.620 DEBUG:teuthology.orchestra.run.smithi043:> ' 2024-04-25T08:54:28.737 INFO:teuthology.orchestra.run.smithi043.stdout:/var/run/ceph/ceph-client.2.43668.asok 2024-04-25T08:54:28.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:54:28.748 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.43668.asok 2024-04-25T08:54:28.748 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:28.748 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.43668.asok status 2024-04-25T08:54:28.884 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T08:54:28.884 INFO:teuthology.orchestra.run.smithi043.stdout: "metadata": { 2024-04-25T08:54:28.884 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:54:28.884 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:54:28.884 INFO:teuthology.orchestra.run.smithi043.stdout: "entity_id": "2", 2024-04-25T08:54:28.884 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": "smithi043", 2024-04-25T08:54:28.884 INFO:teuthology.orchestra.run.smithi043.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-04-25T08:54:28.884 INFO:teuthology.orchestra.run.smithi043.stdout: "pid": "43668", 2024-04-25T08:54:28.884 INFO:teuthology.orchestra.run.smithi043.stdout: "root": "/" 2024-04-25T08:54:28.884 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:54:28.885 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_count": 0, 2024-04-25T08:54:28.885 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_pinned_count": 0, 2024-04-25T08:54:28.885 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 4665, 2024-04-25T08:54:28.885 INFO:teuthology.orchestra.run.smithi043.stdout: "inst": { 2024-04-25T08:54:28.885 INFO:teuthology.orchestra.run.smithi043.stdout: "name": { 2024-04-25T08:54:28.885 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "client", 2024-04-25T08:54:28.885 INFO:teuthology.orchestra.run.smithi043.stdout: "num": 4665 2024-04-25T08:54:28.885 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:54:28.885 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-04-25T08:54:28.885 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-04-25T08:54:28.885 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.2:0", 2024-04-25T08:54:28.886 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1902357962 2024-04-25T08:54:28.886 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:54:28.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:54:28.886 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-04-25T08:54:28.886 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-04-25T08:54:28.886 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.2:0", 2024-04-25T08:54:28.886 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1902357962 2024-04-25T08:54:28.886 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:54:28.886 INFO:teuthology.orchestra.run.smithi043.stdout: "inst_str": "client.4665 192.168.0.2:0/1902357962", 2024-04-25T08:54:28.886 INFO:teuthology.orchestra.run.smithi043.stdout: "addr_str": "192.168.0.2:0/1902357962", 2024-04-25T08:54:28.886 INFO:teuthology.orchestra.run.smithi043.stdout: "inode_count": 1, 2024-04-25T08:54:28.886 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_epoch": 6, 2024-04-25T08:54:28.886 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch": 20, 2024-04-25T08:54:28.886 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:54:28.887 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklisted": false, 2024-04-25T08:54:28.887 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs" 2024-04-25T08:54:28.887 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T08:54:28.892 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:54:28.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.002+0000 7f3fe7549640 1 Processor -- start 2024-04-25T08:54:28.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.002+0000 7f3fe7549640 1 -- start start 2024-04-25T08:54:28.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.003+0000 7f3fe7549640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f3fe006c510 0x7f3fe006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:28.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.003+0000 7f3fe7549640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fe00cbee0 con 0x7f3fe00cd680 2024-04-25T08:54:28.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.003+0000 7f3fe7549640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fe00cee90 con 0x7f3fe015df70 2024-04-25T08:54:28.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.003+0000 7f3fe7549640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fe00d0400 con 0x7f3fe006c510 2024-04-25T08:54:28.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.003+0000 7f3fe5ae6640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f3fe00cd680 0x7f3fe00ce780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:38248/0 (socket says 172.21.15.43:38248) 2024-04-25T08:54:28.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.003+0000 7f3fe5ae6640 1 -- 172.21.15.43:0/996577504 learned_addr learned my addr 172.21.15.43:0/996577504 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:28.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.004+0000 7f3fe52e5640 1 --2- 172.21.15.43:0/996577504 >> v2:172.21.15.193:3300/0 conn(0x7f3fe006c510 0x7f3fe006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:28.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.004+0000 7f3fdffff640 1 -- 172.21.15.43:0/996577504 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2496678102 0 0) 0x7f3fe00cbee0 con 0x7f3fe00cd680 2024-04-25T08:54:28.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.004+0000 7f3fdffff640 1 -- 172.21.15.43:0/996577504 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fc8003520 con 0x7f3fe00cd680 2024-04-25T08:54:28.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.004+0000 7f3fdffff640 1 -- 172.21.15.43:0/996577504 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3116092065 0 0) 0x7f3fc8003520 con 0x7f3fe00cd680 2024-04-25T08:54:28.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.004+0000 7f3fdffff640 1 -- 172.21.15.43:0/996577504 >> v2:172.21.15.193:3300/0 conn(0x7f3fe006c510 msgr2=0x7f3fe006c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:28.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.004+0000 7f3fdffff640 1 --2- 172.21.15.43:0/996577504 >> v2:172.21.15.193:3300/0 conn(0x7f3fe006c510 0x7f3fe006c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:28.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.004+0000 7f3fdffff640 1 -- 172.21.15.43:0/996577504 >> v1:172.21.15.193:6789/0 conn(0x7f3fe015df70 legacy=0x7f3fe00ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:28.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.004+0000 7f3fdffff640 1 -- 172.21.15.43:0/996577504 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe006ce30 con 0x7f3fe00cd680 2024-04-25T08:54:28.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.004+0000 7f3fdffff640 1 -- 172.21.15.43:0/996577504 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f3fd00019f0 con 0x7f3fe00cd680 2024-04-25T08:54:28.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.004+0000 7f3fdffff640 1 -- 172.21.15.43:0/996577504 >> v1:172.21.15.43:6789/0 conn(0x7f3fe00cd680 legacy=0x7f3fe00ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:28.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.004+0000 7f3fdffff640 1 --2- 172.21.15.43:0/996577504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fc80039b0 0x7f3fc8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:28.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.004+0000 7f3fdffff640 1 --2- 172.21.15.43:0/996577504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3fc8004800 0x7f3fc8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:28.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.004+0000 7f3fdffff640 1 --2- 172.21.15.43:0/996577504 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3fc80051c0 0x7f3fc800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:28.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.004+0000 7f3fdffff640 1 -- 172.21.15.43:0/996577504 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe00d0400 con 0x7f3fc80039b0 2024-04-25T08:54:28.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.004+0000 7f3fdffff640 1 -- 172.21.15.43:0/996577504 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc800c060 con 0x7f3fc8004800 2024-04-25T08:54:28.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.004+0000 7f3fdffff640 1 -- 172.21.15.43:0/996577504 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc800c260 con 0x7f3fc80051c0 2024-04-25T08:54:28.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.004+0000 7f3fe52e5640 1 --2- 172.21.15.43:0/996577504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fc80039b0 0x7f3fc8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:28.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.004+0000 7f3fe4ae4640 1 --2- 172.21.15.43:0/996577504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3fc8004800 0x7f3fc8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:28.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.004+0000 7f3fe5ae6640 1 --2- 172.21.15.43:0/996577504 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3fc80051c0 0x7f3fc800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:28.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.004+0000 7f3fe52e5640 1 -- 172.21.15.43:0/996577504 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3fc80051c0 msgr2=0x7f3fc800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:28.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.004+0000 7f3fe52e5640 1 --2- 172.21.15.43:0/996577504 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3fc80051c0 0x7f3fc800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:28.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.004+0000 7f3fe52e5640 1 -- 172.21.15.43:0/996577504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3fc8004800 msgr2=0x7f3fc8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:28.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.004+0000 7f3fe52e5640 1 --2- 172.21.15.43:0/996577504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3fc8004800 0x7f3fc8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:28.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.004+0000 7f3fe52e5640 1 -- 172.21.15.43:0/996577504 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc8003520 con 0x7f3fc80039b0 2024-04-25T08:54:28.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.004+0000 7f3fe5ae6640 1 -- 172.21.15.43:0/996577504 reap_dead start 2024-04-25T08:54:28.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.004+0000 7f3fe4ae4640 1 --2- 172.21.15.43:0/996577504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3fc8004800 0x7f3fc8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:28.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.005+0000 7f3fe52e5640 1 --2- 172.21.15.43:0/996577504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fc80039b0 0x7f3fc8003dd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8009ac0 tx=0x7f3fc8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e024a8880cda075d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:28.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.005+0000 7f3fdffff640 1 -- 172.21.15.43:0/996577504 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8036070 con 0x7f3fc80039b0 2024-04-25T08:54:28.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.005+0000 7f3fdffff640 1 -- 172.21.15.43:0/996577504 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd8031400 con 0x7f3fc80039b0 2024-04-25T08:54:28.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.005+0000 7f3fdffff640 1 -- 172.21.15.43:0/996577504 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8009690 con 0x7f3fc80039b0 2024-04-25T08:54:28.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.006+0000 7f3fe7549640 1 -- 172.21.15.43:0/996577504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fc80039b0 msgr2=0x7f3fc8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:28.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.006+0000 7f3fe7549640 1 --2- 172.21.15.43:0/996577504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fc80039b0 0x7f3fc8003dd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8009ac0 tx=0x7f3fc8005830 comp rx=0 tx=0).stop 2024-04-25T08:54:28.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.006+0000 7f3fe7549640 1 -- 172.21.15.43:0/996577504 shutdown_connections 2024-04-25T08:54:28.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.006+0000 7f3fe7549640 1 --2- 172.21.15.43:0/996577504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fc80039b0 0x7f3fc8003dd0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:28.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.006+0000 7f3fe7549640 1 -- 172.21.15.43:0/996577504 >> 172.21.15.43:0/996577504 conn(0x7f3fe00d17f0 msgr2=0x7f3fe00d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:28.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.006+0000 7f3fe7549640 1 -- 172.21.15.43:0/996577504 shutdown_connections 2024-04-25T08:54:28.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.006+0000 7f3fe7549640 1 -- 172.21.15.43:0/996577504 wait complete. 2024-04-25T08:54:29.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.007+0000 7f3fe7549640 1 Processor -- start 2024-04-25T08:54:29.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.007+0000 7f3fe7549640 1 -- start start 2024-04-25T08:54:29.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.008+0000 7f3fe7549640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3fc80039b0 0x7f3fe009b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:29.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.008+0000 7f3fe7549640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fe01f6ff0 0x7f3fe00b3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:29.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.008+0000 7f3fe7549640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3fe00caa90 0x7f3fe00caf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:29.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.008+0000 7f3fe7549640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe00cc150 con 0x7f3fe01f6ff0 2024-04-25T08:54:29.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.008+0000 7f3fe7549640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe00d0930 con 0x7f3fe00caa90 2024-04-25T08:54:29.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.008+0000 7f3fe7549640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe00d0c10 con 0x7f3fc80039b0 2024-04-25T08:54:29.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.008+0000 7f3fe4ae4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fe01f6ff0 0x7f3fe00b3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:29.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.008+0000 7f3fe4ae4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fe01f6ff0 0x7f3fe00b3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:56216/0 (socket says 172.21.15.43:56216) 2024-04-25T08:54:29.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.008+0000 7f3fe4ae4640 1 -- 172.21.15.43:0/1054481840 learned_addr learned my addr 172.21.15.43:0/1054481840 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:29.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.008+0000 7f3fe52e5640 1 --2- 172.21.15.43:0/1054481840 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3fc80039b0 0x7f3fe009b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:29.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.008+0000 7f3fe5ae6640 1 --2- 172.21.15.43:0/1054481840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3fe00caa90 0x7f3fe00caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:29.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.008+0000 7f3fe4ae4640 1 -- 172.21.15.43:0/1054481840 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3fc80039b0 msgr2=0x7f3fe009b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:29.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.008+0000 7f3fe4ae4640 1 --2- 172.21.15.43:0/1054481840 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3fc80039b0 0x7f3fe009b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:29.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.008+0000 7f3fe4ae4640 1 -- 172.21.15.43:0/1054481840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3fe00caa90 msgr2=0x7f3fe00caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:29.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.008+0000 7f3fe4ae4640 1 --2- 172.21.15.43:0/1054481840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3fe00caa90 0x7f3fe00caf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:29.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.008+0000 7f3fe4ae4640 1 -- 172.21.15.43:0/1054481840 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe01f7760 con 0x7f3fe01f6ff0 2024-04-25T08:54:29.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.008+0000 7f3fe5ae6640 1 --2- 172.21.15.43:0/1054481840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3fe00caa90 0x7f3fe00caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:29.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.009+0000 7f3fe4ae4640 1 --2- 172.21.15.43:0/1054481840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fe01f6ff0 0x7f3fe00b3ae0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3fd400e480 tx=0x7f3fd400e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:29.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.009+0000 7f3fddffb640 1 -- 172.21.15.43:0/1054481840 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd40052e0 con 0x7f3fe01f6ff0 2024-04-25T08:54:29.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.009+0000 7f3fddffb640 1 -- 172.21.15.43:0/1054481840 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd4018070 con 0x7f3fe01f6ff0 2024-04-25T08:54:29.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.009+0000 7f3fe7549640 1 -- 172.21.15.43:0/1054481840 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fe01f7a40 con 0x7f3fe01f6ff0 2024-04-25T08:54:29.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.009+0000 7f3fddffb640 1 -- 172.21.15.43:0/1054481840 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4003830 con 0x7f3fe01f6ff0 2024-04-25T08:54:29.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.009+0000 7f3fe7549640 1 -- 172.21.15.43:0/1054481840 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe01f7470 con 0x7f3fe01f6ff0 2024-04-25T08:54:29.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.010+0000 7f3fe7549640 1 -- 172.21.15.43:0/1054481840 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fe00cb450 con 0x7f3fe01f6ff0 2024-04-25T08:54:29.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.010+0000 7f3fddffb640 1 -- 172.21.15.43:0/1054481840 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3fd40039d0 con 0x7f3fe01f6ff0 2024-04-25T08:54:29.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.011+0000 7f3fddffb640 1 --2- 172.21.15.43:0/1054481840 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3fcc06b500 0x7f3fcc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:29.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.011+0000 7f3fddffb640 1 -- 172.21.15.43:0/1054481840 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3fd4089d90 con 0x7f3fe01f6ff0 2024-04-25T08:54:29.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.011+0000 7f3fe52e5640 1 --2- 172.21.15.43:0/1054481840 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3fcc06b500 0x7f3fcc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:29.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.011+0000 7f3fe52e5640 1 --2- 172.21.15.43:0/1054481840 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3fcc06b500 0x7f3fcc06d9c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8033000 tx=0x7f3fd8008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:29.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.015+0000 7f3fddffb640 1 -- 172.21.15.43:0/1054481840 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3fd4058c00 con 0x7f3fe01f6ff0 2024-04-25T08:54:29.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.235+0000 7f3fe7549640 1 -- 172.21.15.43:0/1054481840 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3fe0074ef0 con 0x7f3fe01f6ff0 2024-04-25T08:54:29.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.235+0000 7f3fddffb640 1 -- 172.21.15.43:0/1054481840 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3fd4058500 con 0x7f3fe01f6ff0 2024-04-25T08:54:29.225 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:54:29.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.237+0000 7f3fe7549640 1 -- 172.21.15.43:0/1054481840 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3fcc06b500 msgr2=0x7f3fcc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:29.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.237+0000 7f3fe7549640 1 --2- 172.21.15.43:0/1054481840 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3fcc06b500 0x7f3fcc06d9c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8033000 tx=0x7f3fd8008030 comp rx=0 tx=0).stop 2024-04-25T08:54:29.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.237+0000 7f3fe7549640 1 -- 172.21.15.43:0/1054481840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fe01f6ff0 msgr2=0x7f3fe00b3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:29.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.237+0000 7f3fe7549640 1 --2- 172.21.15.43:0/1054481840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fe01f6ff0 0x7f3fe00b3ae0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3fd400e480 tx=0x7f3fd400e940 comp rx=0 tx=0).stop 2024-04-25T08:54:29.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.237+0000 7f3fe7549640 1 -- 172.21.15.43:0/1054481840 shutdown_connections 2024-04-25T08:54:29.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.237+0000 7f3fe7549640 1 --2- 172.21.15.43:0/1054481840 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3fcc06b500 0x7f3fcc06d9c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:29.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.237+0000 7f3fe7549640 1 --2- 172.21.15.43:0/1054481840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3fe00caa90 0x7f3fe00caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:29.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.237+0000 7f3fe7549640 1 --2- 172.21.15.43:0/1054481840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3fe01f6ff0 0x7f3fe00b3ae0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:29.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.237+0000 7f3fe7549640 1 --2- 172.21.15.43:0/1054481840 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3fc80039b0 0x7f3fe009b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:29.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.237+0000 7f3fe7549640 1 -- 172.21.15.43:0/1054481840 >> 172.21.15.43:0/1054481840 conn(0x7f3fe00d17f0 msgr2=0x7f3fe00ae5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:29.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.237+0000 7f3fe7549640 1 -- 172.21.15.43:0/1054481840 shutdown_connections 2024-04-25T08:54:29.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:29.237+0000 7f3fe7549640 1 -- 172.21.15.43:0/1054481840 wait complete. 2024-04-25T08:54:29.238 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:54:29.238 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:54:29.238 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi193.front.sepia.ceph.com 2024-04-25T08:54:29.238 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T08:54:29.238 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T08:54:29.238 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:54:29.238 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:54:29.238 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:54:29.238 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T08:54:29.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:29.239 DEBUG:teuthology.orchestra.run.smithi193:> ip addr 2024-04-25T08:54:29.257 INFO:teuthology.orchestra.run.smithi193.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:54:29.257 INFO:teuthology.orchestra.run.smithi193.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:54:29.258 INFO:teuthology.orchestra.run.smithi193.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:54:29.258 INFO:teuthology.orchestra.run.smithi193.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:29.258 INFO:teuthology.orchestra.run.smithi193.stdout: inet6 ::1/128 scope host 2024-04-25T08:54:29.258 INFO:teuthology.orchestra.run.smithi193.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:29.258 INFO:teuthology.orchestra.run.smithi193.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:54:29.258 INFO:teuthology.orchestra.run.smithi193.stdout: link/ether 0c:c4:7a:d9:38:b6 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:54:29.258 INFO:teuthology.orchestra.run.smithi193.stdout: altname enp7s0f0 2024-04-25T08:54:29.258 INFO:teuthology.orchestra.run.smithi193.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:54:29.258 INFO:teuthology.orchestra.run.smithi193.stdout: link/ether 0c:c4:7a:d9:38:b7 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:54:29.258 INFO:teuthology.orchestra.run.smithi193.stdout: altname enp7s0f1 2024-04-25T08:54:29.258 INFO:teuthology.orchestra.run.smithi193.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:54:29.258 INFO:teuthology.orchestra.run.smithi193.stdout: link/ether 0c:c4:7a:88:72:52 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:54:29.258 INFO:teuthology.orchestra.run.smithi193.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:54:29.259 INFO:teuthology.orchestra.run.smithi193.stdout: link/ether 0c:c4:7a:88:72:53 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:54:29.259 INFO:teuthology.orchestra.run.smithi193.stdout: inet 172.21.15.193/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T08:54:29.259 INFO:teuthology.orchestra.run.smithi193.stdout: valid_lft 42757sec preferred_lft 42757sec 2024-04-25T08:54:29.259 INFO:teuthology.orchestra.run.smithi193.stdout: inet6 fe80::3d1:9f1f:4d9e:4aa9/64 scope link noprefixroute 2024-04-25T08:54:29.259 INFO:teuthology.orchestra.run.smithi193.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:29.259 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T08:54:29.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:29.259 DEBUG:teuthology.orchestra.run.smithi193:> (cd / && exec bash -c ' 2024-04-25T08:54:29.259 DEBUG:teuthology.orchestra.run.smithi193:> set -e 2024-04-25T08:54:29.259 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip link add name ceph-brx type bridge 2024-04-25T08:54:29.260 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip addr flush dev ceph-brx 2024-04-25T08:54:29.260 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip link set ceph-brx up 2024-04-25T08:54:29.260 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T08:54:29.260 DEBUG:teuthology.orchestra.run.smithi193:> ') 2024-04-25T08:54:29.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:29.406 DEBUG:teuthology.orchestra.run.smithi193:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:54:29.430 INFO:teuthology.orchestra.run.smithi193.stdout:1 2024-04-25T08:54:29.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:29.432 DEBUG:teuthology.orchestra.run.smithi193:> route 2024-04-25T08:54:29.498 INFO:teuthology.orchestra.run.smithi193.stdout:Kernel IP routing table 2024-04-25T08:54:29.498 INFO:teuthology.orchestra.run.smithi193.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:54:29.498 INFO:teuthology.orchestra.run.smithi193.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T08:54:29.498 INFO:teuthology.orchestra.run.smithi193.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T08:54:29.498 INFO:teuthology.orchestra.run.smithi193.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:54:29.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:29.499 DEBUG:teuthology.orchestra.run.smithi193:> (cd / && exec bash -c ' 2024-04-25T08:54:29.499 DEBUG:teuthology.orchestra.run.smithi193:> set -e 2024-04-25T08:54:29.499 DEBUG:teuthology.orchestra.run.smithi193:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T08:54:29.499 DEBUG:teuthology.orchestra.run.smithi193:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T08:54:29.499 DEBUG:teuthology.orchestra.run.smithi193:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T08:54:29.499 DEBUG:teuthology.orchestra.run.smithi193:> ') 2024-04-25T08:54:29.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:29.705 DEBUG:teuthology.orchestra.run.smithi193:> ip netns list 2024-04-25T08:54:29.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:29.720 DEBUG:teuthology.orchestra.run.smithi193:> ip netns list-id 2024-04-25T08:54:29.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:29.776 DEBUG:teuthology.orchestra.run.smithi193:> (cd / && exec bash -c ' 2024-04-25T08:54:29.776 DEBUG:teuthology.orchestra.run.smithi193:> set -e 2024-04-25T08:54:29.776 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:54:29.776 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-25T08:54:29.776 DEBUG:teuthology.orchestra.run.smithi193:> ') 2024-04-25T08:54:29.851 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-25T08:54:29.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:29.852 DEBUG:teuthology.orchestra.run.smithi193:> (cd / && exec bash -c ' 2024-04-25T08:54:29.852 DEBUG:teuthology.orchestra.run.smithi193:> set -e 2024-04-25T08:54:29.852 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-25T08:54:29.852 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T08:54:29.852 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-25T08:54:29.852 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-25T08:54:29.852 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-25T08:54:29.852 DEBUG:teuthology.orchestra.run.smithi193:> ') 2024-04-25T08:54:29.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:29.974 DEBUG:teuthology.orchestra.run.smithi193:> (cd / && exec bash -c ' 2024-04-25T08:54:29.974 DEBUG:teuthology.orchestra.run.smithi193:> set -e 2024-04-25T08:54:29.974 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip link set brx.0 up 2024-04-25T08:54:29.974 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T08:54:29.974 DEBUG:teuthology.orchestra.run.smithi193:> ') 2024-04-25T08:54:30.009 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-25T08:54:30.009 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:54:30.009 DEBUG:teuthology.orchestra.run.smithi193:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T08:54:30.064 INFO:teuthology.orchestra.run.smithi193.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T08:54:30.064 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:54:30.064 DEBUG:teuthology.orchestra.run.smithi193:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T08:54:30.119 DEBUG:teuthology.orchestra.run.smithi193:> sudo modprobe fuse 2024-04-25T08:54:30.183 DEBUG:teuthology.orchestra.run.smithi193:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:54:30.239 INFO:teuthology.orchestra.run.smithi193.stdout:/proc 2024-04-25T08:54:30.239 INFO:teuthology.orchestra.run.smithi193.stdout:/sys 2024-04-25T08:54:30.239 INFO:teuthology.orchestra.run.smithi193.stdout:/dev 2024-04-25T08:54:30.239 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/kernel/security 2024-04-25T08:54:30.239 INFO:teuthology.orchestra.run.smithi193.stdout:/dev/shm 2024-04-25T08:54:30.240 INFO:teuthology.orchestra.run.smithi193.stdout:/dev/pts 2024-04-25T08:54:30.240 INFO:teuthology.orchestra.run.smithi193.stdout:/run 2024-04-25T08:54:30.240 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/cgroup 2024-04-25T08:54:30.240 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/pstore 2024-04-25T08:54:30.240 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/bpf 2024-04-25T08:54:30.240 INFO:teuthology.orchestra.run.smithi193.stdout:/ 2024-04-25T08:54:30.240 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/selinux 2024-04-25T08:54:30.240 INFO:teuthology.orchestra.run.smithi193.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:54:30.240 INFO:teuthology.orchestra.run.smithi193.stdout:/dev/hugepages 2024-04-25T08:54:30.240 INFO:teuthology.orchestra.run.smithi193.stdout:/dev/mqueue 2024-04-25T08:54:30.240 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/kernel/debug 2024-04-25T08:54:30.240 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/kernel/tracing 2024-04-25T08:54:30.240 INFO:teuthology.orchestra.run.smithi193.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:54:30.240 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/kernel/config 2024-04-25T08:54:30.240 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/fuse/connections 2024-04-25T08:54:30.241 INFO:teuthology.orchestra.run.smithi193.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:54:30.241 INFO:teuthology.orchestra.run.smithi193.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:54:30.241 INFO:teuthology.orchestra.run.smithi193.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:54:30.241 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:54:30.241 INFO:teuthology.orchestra.run.smithi193.stdout:/run/user/1000 2024-04-25T08:54:30.241 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/ceph 2024-04-25T08:54:30.241 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T08:54:30.241 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T08:54:30.241 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T08:54:30.241 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T08:54:30.241 INFO:teuthology.orchestra.run.smithi193.stdout:/run/netns 2024-04-25T08:54:30.241 INFO:teuthology.orchestra.run.smithi193.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:54:30.241 INFO:teuthology.orchestra.run.smithi193.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:54:30.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:30.242 DEBUG:teuthology.orchestra.run.smithi193:> ls /sys/fs/fuse/connections 2024-04-25T08:54:30.295 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T08:54:30.295 DEBUG:teuthology.orchestra.run.smithi193:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-25T08:54:30.337 DEBUG:teuthology.orchestra.run.smithi193:> sudo modprobe fuse 2024-04-25T08:54:30.360 DEBUG:teuthology.orchestra.run.smithi193:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:54:30.417 INFO:teuthology.orchestra.run.smithi193.stdout:/proc 2024-04-25T08:54:30.417 INFO:teuthology.orchestra.run.smithi193.stdout:/sys 2024-04-25T08:54:30.417 INFO:teuthology.orchestra.run.smithi193.stdout:/dev 2024-04-25T08:54:30.417 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/kernel/security 2024-04-25T08:54:30.417 INFO:teuthology.orchestra.run.smithi193.stdout:/dev/shm 2024-04-25T08:54:30.418 INFO:teuthology.orchestra.run.smithi193.stdout:/dev/pts 2024-04-25T08:54:30.418 INFO:teuthology.orchestra.run.smithi193.stdout:/run 2024-04-25T08:54:30.418 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/cgroup 2024-04-25T08:54:30.418 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/pstore 2024-04-25T08:54:30.418 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/bpf 2024-04-25T08:54:30.418 INFO:teuthology.orchestra.run.smithi193.stdout:/ 2024-04-25T08:54:30.418 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/selinux 2024-04-25T08:54:30.418 INFO:teuthology.orchestra.run.smithi193.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:54:30.418 INFO:teuthology.orchestra.run.smithi193.stdout:/dev/hugepages 2024-04-25T08:54:30.418 INFO:teuthology.orchestra.run.smithi193.stdout:/dev/mqueue 2024-04-25T08:54:30.418 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/kernel/debug 2024-04-25T08:54:30.418 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/kernel/tracing 2024-04-25T08:54:30.418 INFO:teuthology.orchestra.run.smithi193.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:54:30.419 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/kernel/config 2024-04-25T08:54:30.419 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/fuse/connections 2024-04-25T08:54:30.419 INFO:teuthology.orchestra.run.smithi193.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:54:30.419 INFO:teuthology.orchestra.run.smithi193.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:54:30.419 INFO:teuthology.orchestra.run.smithi193.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:54:30.419 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:54:30.419 INFO:teuthology.orchestra.run.smithi193.stdout:/run/user/1000 2024-04-25T08:54:30.419 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/ceph 2024-04-25T08:54:30.419 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T08:54:30.419 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T08:54:30.419 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T08:54:30.419 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T08:54:30.419 INFO:teuthology.orchestra.run.smithi193.stdout:/run/netns 2024-04-25T08:54:30.419 INFO:teuthology.orchestra.run.smithi193.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:54:30.420 INFO:teuthology.orchestra.run.smithi193.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:54:30.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:30.420 DEBUG:teuthology.orchestra.run.smithi193:> ls /sys/fs/fuse/connections 2024-04-25T08:54:30.425 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi193.stderr:2024-04-25T08:54:30.435+0000 7f8bbce0b480 -1 init, newargv = 0x55b289a07750 newargc=15 2024-04-25T08:54:30.426 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi193.stderr:ceph-fuse[40994]: starting ceph client 2024-04-25T08:54:30.429 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi193.stderr:ceph-fuse[40994]: starting fuse 2024-04-25T08:54:30.440 INFO:teuthology.orchestra.run.smithi193.stdout:43 2024-04-25T08:54:30.440 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-25T08:54:30.440 DEBUG:teuthology.orchestra.run.smithi193:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:54:30.440 DEBUG:teuthology.orchestra.run.smithi193:> import glob 2024-04-25T08:54:30.441 DEBUG:teuthology.orchestra.run.smithi193:> import re 2024-04-25T08:54:30.441 DEBUG:teuthology.orchestra.run.smithi193:> import os 2024-04-25T08:54:30.441 DEBUG:teuthology.orchestra.run.smithi193:> import subprocess 2024-04-25T08:54:30.441 DEBUG:teuthology.orchestra.run.smithi193:> 2024-04-25T08:54:30.441 DEBUG:teuthology.orchestra.run.smithi193:> def _find_admin_socket(client_name): 2024-04-25T08:54:30.441 DEBUG:teuthology.orchestra.run.smithi193:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-25T08:54:30.441 DEBUG:teuthology.orchestra.run.smithi193:> files = glob.glob(asok_path) 2024-04-25T08:54:30.441 DEBUG:teuthology.orchestra.run.smithi193:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-25T08:54:30.441 DEBUG:teuthology.orchestra.run.smithi193:> 2024-04-25T08:54:30.441 DEBUG:teuthology.orchestra.run.smithi193:> # Given a non-glob path, it better be there 2024-04-25T08:54:30.441 DEBUG:teuthology.orchestra.run.smithi193:> if "*" not in asok_path: 2024-04-25T08:54:30.441 DEBUG:teuthology.orchestra.run.smithi193:> assert(len(files) == 1) 2024-04-25T08:54:30.441 DEBUG:teuthology.orchestra.run.smithi193:> return files[0] 2024-04-25T08:54:30.441 DEBUG:teuthology.orchestra.run.smithi193:> 2024-04-25T08:54:30.442 DEBUG:teuthology.orchestra.run.smithi193:> for f in files: 2024-04-25T08:54:30.442 DEBUG:teuthology.orchestra.run.smithi193:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:54:30.442 DEBUG:teuthology.orchestra.run.smithi193:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:54:30.442 DEBUG:teuthology.orchestra.run.smithi193:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:54:30.442 DEBUG:teuthology.orchestra.run.smithi193:> contents = proc_f.read() 2024-04-25T08:54:30.442 DEBUG:teuthology.orchestra.run.smithi193:> if mountpoint in contents: 2024-04-25T08:54:30.442 DEBUG:teuthology.orchestra.run.smithi193:> return f 2024-04-25T08:54:30.442 DEBUG:teuthology.orchestra.run.smithi193:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:54:30.442 DEBUG:teuthology.orchestra.run.smithi193:> 2024-04-25T08:54:30.442 DEBUG:teuthology.orchestra.run.smithi193:> print(_find_admin_socket("client.1")) 2024-04-25T08:54:30.442 DEBUG:teuthology.orchestra.run.smithi193:> ' 2024-04-25T08:54:30.557 INFO:teuthology.orchestra.run.smithi193.stdout:/var/run/ceph/ceph-client.1.40994.asok 2024-04-25T08:54:30.560 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:54:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:54:30.568 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.40994.asok 2024-04-25T08:54:30.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:30.568 DEBUG:teuthology.orchestra.run.smithi193:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.40994.asok status 2024-04-25T08:54:30.703 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-04-25T08:54:30.703 INFO:teuthology.orchestra.run.smithi193.stdout: "metadata": { 2024-04-25T08:54:30.703 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:54:30.703 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:54:30.703 INFO:teuthology.orchestra.run.smithi193.stdout: "entity_id": "1", 2024-04-25T08:54:30.703 INFO:teuthology.orchestra.run.smithi193.stdout: "hostname": "smithi193", 2024-04-25T08:54:30.703 INFO:teuthology.orchestra.run.smithi193.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-25T08:54:30.703 INFO:teuthology.orchestra.run.smithi193.stdout: "pid": "40994", 2024-04-25T08:54:30.704 INFO:teuthology.orchestra.run.smithi193.stdout: "root": "/" 2024-04-25T08:54:30.704 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T08:54:30.704 INFO:teuthology.orchestra.run.smithi193.stdout: "dentry_count": 0, 2024-04-25T08:54:30.704 INFO:teuthology.orchestra.run.smithi193.stdout: "dentry_pinned_count": 0, 2024-04-25T08:54:30.704 INFO:teuthology.orchestra.run.smithi193.stdout: "id": 4526, 2024-04-25T08:54:30.704 INFO:teuthology.orchestra.run.smithi193.stdout: "inst": { 2024-04-25T08:54:30.704 INFO:teuthology.orchestra.run.smithi193.stdout: "name": { 2024-04-25T08:54:30.704 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "client", 2024-04-25T08:54:30.704 INFO:teuthology.orchestra.run.smithi193.stdout: "num": 4526 2024-04-25T08:54:30.704 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T08:54:30.704 INFO:teuthology.orchestra.run.smithi193.stdout: "addr": { 2024-04-25T08:54:30.704 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "v1", 2024-04-25T08:54:30.704 INFO:teuthology.orchestra.run.smithi193.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:54:30.704 INFO:teuthology.orchestra.run.smithi193.stdout: "nonce": 812122462 2024-04-25T08:54:30.704 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-04-25T08:54:30.704 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T08:54:30.705 INFO:teuthology.orchestra.run.smithi193.stdout: "addr": { 2024-04-25T08:54:30.705 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "v1", 2024-04-25T08:54:30.705 INFO:teuthology.orchestra.run.smithi193.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:54:30.705 INFO:teuthology.orchestra.run.smithi193.stdout: "nonce": 812122462 2024-04-25T08:54:30.705 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T08:54:30.705 INFO:teuthology.orchestra.run.smithi193.stdout: "inst_str": "client.4526 192.168.0.1:0/812122462", 2024-04-25T08:54:30.705 INFO:teuthology.orchestra.run.smithi193.stdout: "addr_str": "192.168.0.1:0/812122462", 2024-04-25T08:54:30.705 INFO:teuthology.orchestra.run.smithi193.stdout: "inode_count": 1, 2024-04-25T08:54:30.705 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_epoch": 6, 2024-04-25T08:54:30.705 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_epoch": 20, 2024-04-25T08:54:30.705 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:54:30.705 INFO:teuthology.orchestra.run.smithi193.stdout: "blocklisted": false, 2024-04-25T08:54:30.705 INFO:teuthology.orchestra.run.smithi193.stdout: "fs_name": "cephfs" 2024-04-25T08:54:30.705 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-04-25T08:54:30.710 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:54:30.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.832+0000 7ff9b793b640 1 Processor -- start 2024-04-25T08:54:30.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.833+0000 7ff9b793b640 1 -- start start 2024-04-25T08:54:30.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.834+0000 7ff9b793b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff9b00cbfc0 0x7ff9b006dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:30.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.834+0000 7ff9b793b640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff9b00ccbb0 0x7ff9b00cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:30.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.834+0000 7ff9b793b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff9b015f5e0 0x7ff9b015fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:30.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.834+0000 7ff9b793b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9b00cd550 con 0x7ff9b00cbfc0 2024-04-25T08:54:30.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.834+0000 7ff9b793b640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9b00b2780 con 0x7ff9b00ccbb0 2024-04-25T08:54:30.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.834+0000 7ff9b793b640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b00b2a60 con 0x7ff9b015f5e0 2024-04-25T08:54:30.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.834+0000 7ff9b56d7640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff9b00cbfc0 0x7ff9b006dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:30.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.834+0000 7ff9b56d7640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff9b00cbfc0 0x7ff9b006dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39954/0 (socket says 172.21.15.43:39954) 2024-04-25T08:54:30.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.834+0000 7ff9b56d7640 1 -- 172.21.15.43:0/3316621971 learned_addr learned my addr 172.21.15.43:0/3316621971 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:30.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.834+0000 7ff9b5ed8640 1 --2- 172.21.15.43:0/3316621971 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff9b015f5e0 0x7ff9b015fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:30.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.834+0000 7ff9b4ed6640 1 --2- 172.21.15.43:0/3316621971 >> v2:172.21.15.193:3300/0 conn(0x7ff9b00ccbb0 0x7ff9b00cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:30.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.834+0000 7ff9b56d7640 1 -- 172.21.15.43:0/3316621971 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff9b015f5e0 msgr2=0x7ff9b015fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:30.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.834+0000 7ff9b56d7640 1 --2- 172.21.15.43:0/3316621971 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff9b015f5e0 0x7ff9b015fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:30.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.834+0000 7ff9b56d7640 1 -- 172.21.15.43:0/3316621971 >> v2:172.21.15.193:3300/0 conn(0x7ff9b00ccbb0 msgr2=0x7ff9b00cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:30.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.834+0000 7ff9b56d7640 1 --2- 172.21.15.43:0/3316621971 >> v2:172.21.15.193:3300/0 conn(0x7ff9b00ccbb0 0x7ff9b00cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:30.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.834+0000 7ff9b56d7640 1 -- 172.21.15.43:0/3316621971 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b00b2d40 con 0x7ff9b00cbfc0 2024-04-25T08:54:30.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.835+0000 7ff9b56d7640 1 --2- 172.21.15.43:0/3316621971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9b00cbfc0 0x7ff9b006dc60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff9b00747f0 tx=0x7ff9ac02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f63c3dd20136591 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:30.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.835+0000 7ff9a7fff640 1 -- 172.21.15.43:0/3316621971 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac037070 con 0x7ff9b00cbfc0 2024-04-25T08:54:30.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.835+0000 7ff9a7fff640 1 -- 172.21.15.43:0/3316621971 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9ac032440 con 0x7ff9b00cbfc0 2024-04-25T08:54:30.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.835+0000 7ff9a7fff640 1 -- 172.21.15.43:0/3316621971 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac032810 con 0x7ff9b00cbfc0 2024-04-25T08:54:30.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.836+0000 7ff9b793b640 1 -- 172.21.15.43:0/3316621971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9b00cbfc0 msgr2=0x7ff9b006dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:30.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.836+0000 7ff9b793b640 1 --2- 172.21.15.43:0/3316621971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9b00cbfc0 0x7ff9b006dc60 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ff9b00747f0 tx=0x7ff9ac02a1e0 comp rx=0 tx=0).stop 2024-04-25T08:54:30.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.836+0000 7ff9b793b640 1 -- 172.21.15.43:0/3316621971 shutdown_connections 2024-04-25T08:54:30.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.836+0000 7ff9b793b640 1 --2- 172.21.15.43:0/3316621971 >> v2:172.21.15.193:3300/0 conn(0x7ff9b00ccbb0 0x7ff9b00cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:30.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.836+0000 7ff9b793b640 1 --2- 172.21.15.43:0/3316621971 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff9b015f5e0 0x7ff9b015fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:30.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.836+0000 7ff9b793b640 1 --2- 172.21.15.43:0/3316621971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9b00cbfc0 0x7ff9b006dc60 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:30.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.836+0000 7ff9b793b640 1 -- 172.21.15.43:0/3316621971 >> 172.21.15.43:0/3316621971 conn(0x7ff9b00b1480 msgr2=0x7ff9b00b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:30.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.836+0000 7ff9b793b640 1 -- 172.21.15.43:0/3316621971 shutdown_connections 2024-04-25T08:54:30.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.836+0000 7ff9b793b640 1 -- 172.21.15.43:0/3316621971 wait complete. 2024-04-25T08:54:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.837+0000 7ff9b793b640 1 Processor -- start 2024-04-25T08:54:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.837+0000 7ff9b793b640 1 -- start start 2024-04-25T08:54:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.838+0000 7ff9b793b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9b00cbfc0 0x7ff9b01410b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.838+0000 7ff9b793b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9b00ccbb0 0x7ff9b00c46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.838+0000 7ff9b793b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff9b015f5e0 0x7ff9b01f17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.838+0000 7ff9b793b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b00b35e0 con 0x7ff9b00cbfc0 2024-04-25T08:54:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.838+0000 7ff9b793b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b00b3300 con 0x7ff9b00ccbb0 2024-04-25T08:54:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.838+0000 7ff9b793b640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b006e1a0 con 0x7ff9b015f5e0 2024-04-25T08:54:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.838+0000 7ff9b56d7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9b00cbfc0 0x7ff9b01410b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.838+0000 7ff9b56d7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9b00cbfc0 0x7ff9b01410b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39960/0 (socket says 172.21.15.43:39960) 2024-04-25T08:54:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.838+0000 7ff9b56d7640 1 -- 172.21.15.43:0/3666493093 learned_addr learned my addr 172.21.15.43:0/3666493093 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.838+0000 7ff9b5ed8640 1 --2- 172.21.15.43:0/3666493093 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff9b015f5e0 0x7ff9b01f17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.838+0000 7ff9b4ed6640 1 --2- 172.21.15.43:0/3666493093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9b00ccbb0 0x7ff9b00c46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.838+0000 7ff9b56d7640 1 -- 172.21.15.43:0/3666493093 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff9b015f5e0 msgr2=0x7ff9b01f17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.838+0000 7ff9b56d7640 1 --2- 172.21.15.43:0/3666493093 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff9b015f5e0 0x7ff9b01f17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.838+0000 7ff9b56d7640 1 -- 172.21.15.43:0/3666493093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9b00ccbb0 msgr2=0x7ff9b00c46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.838+0000 7ff9b56d7640 1 --2- 172.21.15.43:0/3666493093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9b00ccbb0 0x7ff9b00c46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.838+0000 7ff9b56d7640 1 -- 172.21.15.43:0/3666493093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b01415f0 con 0x7ff9b00cbfc0 2024-04-25T08:54:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.838+0000 7ff9b4ed6640 1 --2- 172.21.15.43:0/3666493093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9b00ccbb0 0x7ff9b00c46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.839+0000 7ff9b56d7640 1 --2- 172.21.15.43:0/3666493093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9b00cbfc0 0x7ff9b01410b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac000c00 tx=0x7ff9ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.839+0000 7ff9a5ffb640 1 -- 172.21.15.43:0/3666493093 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac041020 con 0x7ff9b00cbfc0 2024-04-25T08:54:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.839+0000 7ff9a5ffb640 1 -- 172.21.15.43:0/3666493093 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9ac03d620 con 0x7ff9b00cbfc0 2024-04-25T08:54:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.839+0000 7ff9a5ffb640 1 -- 172.21.15.43:0/3666493093 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac037040 con 0x7ff9b00cbfc0 2024-04-25T08:54:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.839+0000 7ff9b793b640 1 -- 172.21.15.43:0/3666493093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9b00a9f80 con 0x7ff9b00cbfc0 2024-04-25T08:54:30.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.839+0000 7ff9b793b640 1 -- 172.21.15.43:0/3666493093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b00aa540 con 0x7ff9b00cbfc0 2024-04-25T08:54:30.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.840+0000 7ff9b793b640 1 -- 172.21.15.43:0/3666493093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9b0074ef0 con 0x7ff9b00cbfc0 2024-04-25T08:54:30.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.841+0000 7ff9a5ffb640 1 -- 172.21.15.43:0/3666493093 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff9ac044020 con 0x7ff9b00cbfc0 2024-04-25T08:54:30.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.841+0000 7ff9a5ffb640 1 --2- 172.21.15.43:0/3666493093 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff99406b4d0 0x7ff99406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:30.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.841+0000 7ff9a5ffb640 1 -- 172.21.15.43:0/3666493093 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff9ac0aa7c0 con 0x7ff9b00cbfc0 2024-04-25T08:54:30.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.841+0000 7ff9b4ed6640 1 --2- 172.21.15.43:0/3666493093 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff99406b4d0 0x7ff99406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:30.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.842+0000 7ff9b4ed6640 1 --2- 172.21.15.43:0/3666493093 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff99406b4d0 0x7ff99406d990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff9b00ae030 tx=0x7ff9a0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:30.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:30.846+0000 7ff9a5ffb640 1 -- 172.21.15.43:0/3666493093 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff9ac04b030 con 0x7ff9b00cbfc0 2024-04-25T08:54:31.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:31.063+0000 7ff9b793b640 1 -- 172.21.15.43:0/3666493093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff9b0144750 con 0x7ff9b00cbfc0 2024-04-25T08:54:31.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:31.063+0000 7ff9a5ffb640 1 -- 172.21.15.43:0/3666493093 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff9ac079640 con 0x7ff9b00cbfc0 2024-04-25T08:54:31.053 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:54:31.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:31.065+0000 7ff9b793b640 1 -- 172.21.15.43:0/3666493093 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff99406b4d0 msgr2=0x7ff99406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:31.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:31.065+0000 7ff9b793b640 1 --2- 172.21.15.43:0/3666493093 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff99406b4d0 0x7ff99406d990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff9b00ae030 tx=0x7ff9a0008040 comp rx=0 tx=0).stop 2024-04-25T08:54:31.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:31.065+0000 7ff9b793b640 1 -- 172.21.15.43:0/3666493093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9b00cbfc0 msgr2=0x7ff9b01410b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:31.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:31.065+0000 7ff9b793b640 1 --2- 172.21.15.43:0/3666493093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9b00cbfc0 0x7ff9b01410b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac000c00 tx=0x7ff9ac0046c0 comp rx=0 tx=0).stop 2024-04-25T08:54:31.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:31.065+0000 7ff9b793b640 1 -- 172.21.15.43:0/3666493093 shutdown_connections 2024-04-25T08:54:31.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:31.065+0000 7ff9b793b640 1 --2- 172.21.15.43:0/3666493093 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff9b015f5e0 0x7ff9b01f17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:31.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:31.065+0000 7ff9b793b640 1 --2- 172.21.15.43:0/3666493093 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff99406b4d0 0x7ff99406d990 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:31.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:31.065+0000 7ff9b793b640 1 --2- 172.21.15.43:0/3666493093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9b00ccbb0 0x7ff9b00c46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:31.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:31.065+0000 7ff9b793b640 1 --2- 172.21.15.43:0/3666493093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9b00cbfc0 0x7ff9b01410b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:31.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:31.065+0000 7ff9b793b640 1 -- 172.21.15.43:0/3666493093 >> 172.21.15.43:0/3666493093 conn(0x7ff9b00b1480 msgr2=0x7ff9b00b0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:31.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:31.065+0000 7ff9b793b640 1 -- 172.21.15.43:0/3666493093 shutdown_connections 2024-04-25T08:54:31.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:31.065+0000 7ff9b793b640 1 -- 172.21.15.43:0/3666493093 wait complete. 2024-04-25T08:54:31.066 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:54:31.066 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:54:31.066 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi193.front.sepia.ceph.com 2024-04-25T08:54:31.066 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-25T08:54:31.066 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T08:54:31.066 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:54:31.066 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:54:31.066 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:54:31.066 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-25T08:54:31.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:31.067 DEBUG:teuthology.orchestra.run.smithi193:> ip addr 2024-04-25T08:54:31.082 INFO:teuthology.orchestra.run.smithi193.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:54:31.082 INFO:teuthology.orchestra.run.smithi193.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:54:31.082 INFO:teuthology.orchestra.run.smithi193.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:54:31.082 INFO:teuthology.orchestra.run.smithi193.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:31.082 INFO:teuthology.orchestra.run.smithi193.stdout: inet6 ::1/128 scope host 2024-04-25T08:54:31.083 INFO:teuthology.orchestra.run.smithi193.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:31.083 INFO:teuthology.orchestra.run.smithi193.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:54:31.083 INFO:teuthology.orchestra.run.smithi193.stdout: link/ether 0c:c4:7a:d9:38:b6 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:54:31.083 INFO:teuthology.orchestra.run.smithi193.stdout: altname enp7s0f0 2024-04-25T08:54:31.083 INFO:teuthology.orchestra.run.smithi193.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:54:31.083 INFO:teuthology.orchestra.run.smithi193.stdout: link/ether 0c:c4:7a:d9:38:b7 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:54:31.083 INFO:teuthology.orchestra.run.smithi193.stdout: altname enp7s0f1 2024-04-25T08:54:31.083 INFO:teuthology.orchestra.run.smithi193.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:54:31.083 INFO:teuthology.orchestra.run.smithi193.stdout: link/ether 0c:c4:7a:88:72:52 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:54:31.083 INFO:teuthology.orchestra.run.smithi193.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:54:31.083 INFO:teuthology.orchestra.run.smithi193.stdout: link/ether 0c:c4:7a:88:72:53 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:54:31.083 INFO:teuthology.orchestra.run.smithi193.stdout: inet 172.21.15.193/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-25T08:54:31.083 INFO:teuthology.orchestra.run.smithi193.stdout: valid_lft 42755sec preferred_lft 42755sec 2024-04-25T08:54:31.084 INFO:teuthology.orchestra.run.smithi193.stdout: inet6 fe80::3d1:9f1f:4d9e:4aa9/64 scope link noprefixroute 2024-04-25T08:54:31.084 INFO:teuthology.orchestra.run.smithi193.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:31.084 INFO:teuthology.orchestra.run.smithi193.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:54:31.084 INFO:teuthology.orchestra.run.smithi193.stdout: link/ether 62:ec:9d:ae:91:6a brd ff:ff:ff:ff:ff:ff 2024-04-25T08:54:31.084 INFO:teuthology.orchestra.run.smithi193.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:54:31.084 INFO:teuthology.orchestra.run.smithi193.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:31.084 INFO:teuthology.orchestra.run.smithi193.stdout: inet6 fe80::740f:c5ff:fe21:56db/64 scope link 2024-04-25T08:54:31.084 INFO:teuthology.orchestra.run.smithi193.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:31.084 INFO:teuthology.orchestra.run.smithi193.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:54:31.084 INFO:teuthology.orchestra.run.smithi193.stdout: link/ether 62:ec:9d:ae:91:6a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:54:31.084 INFO:teuthology.orchestra.run.smithi193.stdout: inet6 fe80::60ec:9dff:feae:916a/64 scope link tentative 2024-04-25T08:54:31.084 INFO:teuthology.orchestra.run.smithi193.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:31.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:31.085 DEBUG:teuthology.orchestra.run.smithi193:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:54:31.145 INFO:teuthology.orchestra.run.smithi193.stdout:1 2024-04-25T08:54:31.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:31.147 DEBUG:teuthology.orchestra.run.smithi193:> route 2024-04-25T08:54:31.202 INFO:teuthology.orchestra.run.smithi193.stdout:Kernel IP routing table 2024-04-25T08:54:31.202 INFO:teuthology.orchestra.run.smithi193.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:54:31.202 INFO:teuthology.orchestra.run.smithi193.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T08:54:31.202 INFO:teuthology.orchestra.run.smithi193.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T08:54:31.202 INFO:teuthology.orchestra.run.smithi193.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:54:31.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:31.203 DEBUG:teuthology.orchestra.run.smithi193:> (cd / && exec bash -c ' 2024-04-25T08:54:31.203 DEBUG:teuthology.orchestra.run.smithi193:> set -e 2024-04-25T08:54:31.203 DEBUG:teuthology.orchestra.run.smithi193:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T08:54:31.203 DEBUG:teuthology.orchestra.run.smithi193:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T08:54:31.203 DEBUG:teuthology.orchestra.run.smithi193:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T08:54:31.203 DEBUG:teuthology.orchestra.run.smithi193:> ') 2024-04-25T08:54:31.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:31.286 DEBUG:teuthology.orchestra.run.smithi193:> ip netns list 2024-04-25T08:54:31.341 INFO:teuthology.orchestra.run.smithi193.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T08:54:31.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:31.342 DEBUG:teuthology.orchestra.run.smithi193:> ip netns list-id 2024-04-25T08:54:31.396 INFO:teuthology.orchestra.run.smithi193.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-25T08:54:31.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:31.396 DEBUG:teuthology.orchestra.run.smithi193:> (cd / && exec bash -c ' 2024-04-25T08:54:31.396 DEBUG:teuthology.orchestra.run.smithi193:> set -e 2024-04-25T08:54:31.397 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T08:54:31.397 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-25T08:54:31.397 DEBUG:teuthology.orchestra.run.smithi193:> ') 2024-04-25T08:54:31.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:31.476 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T08:54:31.540 INFO:teuthology.orchestra.run.smithi193.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:54:31.541 INFO:teuthology.orchestra.run.smithi193.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:54:31.541 INFO:teuthology.orchestra.run.smithi193.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:54:31.541 INFO:teuthology.orchestra.run.smithi193.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:31.541 INFO:teuthology.orchestra.run.smithi193.stdout: inet6 ::1/128 scope host 2024-04-25T08:54:31.541 INFO:teuthology.orchestra.run.smithi193.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:31.541 INFO:teuthology.orchestra.run.smithi193.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:54:31.541 INFO:teuthology.orchestra.run.smithi193.stdout: link/ether 1a:17:75:7d:dd:65 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T08:54:31.541 INFO:teuthology.orchestra.run.smithi193.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T08:54:31.541 INFO:teuthology.orchestra.run.smithi193.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:31.541 INFO:teuthology.orchestra.run.smithi193.stdout: inet6 fe80::1817:75ff:fe7d:dd65/64 scope link tentative 2024-04-25T08:54:31.541 INFO:teuthology.orchestra.run.smithi193.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:31.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:31.542 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T08:54:31.607 INFO:teuthology.orchestra.run.smithi193.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:54:31.607 INFO:teuthology.orchestra.run.smithi193.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:54:31.607 INFO:teuthology.orchestra.run.smithi193.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:54:31.607 INFO:teuthology.orchestra.run.smithi193.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:31.607 INFO:teuthology.orchestra.run.smithi193.stdout: inet6 ::1/128 scope host 2024-04-25T08:54:31.607 INFO:teuthology.orchestra.run.smithi193.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:31.607 INFO:teuthology.orchestra.run.smithi193.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:54:31.607 INFO:teuthology.orchestra.run.smithi193.stdout: link/ether 1a:17:75:7d:dd:65 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T08:54:31.607 INFO:teuthology.orchestra.run.smithi193.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T08:54:31.607 INFO:teuthology.orchestra.run.smithi193.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:31.607 INFO:teuthology.orchestra.run.smithi193.stdout: inet6 fe80::1817:75ff:fe7d:dd65/64 scope link 2024-04-25T08:54:31.608 INFO:teuthology.orchestra.run.smithi193.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:54:31.609 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-25T08:54:31.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:31.609 DEBUG:teuthology.orchestra.run.smithi193:> (cd / && exec bash -c ' 2024-04-25T08:54:31.609 DEBUG:teuthology.orchestra.run.smithi193:> set -e 2024-04-25T08:54:31.609 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-25T08:54:31.609 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T08:54:31.609 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-25T08:54:31.609 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-25T08:54:31.609 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-25T08:54:31.609 DEBUG:teuthology.orchestra.run.smithi193:> ') 2024-04-25T08:54:31.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:31.719 DEBUG:teuthology.orchestra.run.smithi193:> (cd / && exec bash -c ' 2024-04-25T08:54:31.719 DEBUG:teuthology.orchestra.run.smithi193:> set -e 2024-04-25T08:54:31.719 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip link set brx.1 up 2024-04-25T08:54:31.719 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T08:54:31.719 DEBUG:teuthology.orchestra.run.smithi193:> ') 2024-04-25T08:54:31.755 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-04-25T08:54:31.755 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:54:31.755 DEBUG:teuthology.orchestra.run.smithi193:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-25T08:54:31.811 INFO:teuthology.orchestra.run.smithi193.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-25T08:54:31.811 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:54:31.811 DEBUG:teuthology.orchestra.run.smithi193:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-25T08:54:31.865 DEBUG:teuthology.orchestra.run.smithi193:> sudo modprobe fuse 2024-04-25T08:54:31.930 DEBUG:teuthology.orchestra.run.smithi193:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:54:31.986 INFO:teuthology.orchestra.run.smithi193.stdout:/proc 2024-04-25T08:54:31.986 INFO:teuthology.orchestra.run.smithi193.stdout:/sys 2024-04-25T08:54:31.987 INFO:teuthology.orchestra.run.smithi193.stdout:/dev 2024-04-25T08:54:31.987 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/kernel/security 2024-04-25T08:54:31.987 INFO:teuthology.orchestra.run.smithi193.stdout:/dev/shm 2024-04-25T08:54:31.987 INFO:teuthology.orchestra.run.smithi193.stdout:/dev/pts 2024-04-25T08:54:31.987 INFO:teuthology.orchestra.run.smithi193.stdout:/run 2024-04-25T08:54:31.987 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/cgroup 2024-04-25T08:54:31.987 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/pstore 2024-04-25T08:54:31.987 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/bpf 2024-04-25T08:54:31.987 INFO:teuthology.orchestra.run.smithi193.stdout:/ 2024-04-25T08:54:31.987 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/selinux 2024-04-25T08:54:31.987 INFO:teuthology.orchestra.run.smithi193.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:54:31.987 INFO:teuthology.orchestra.run.smithi193.stdout:/dev/hugepages 2024-04-25T08:54:31.987 INFO:teuthology.orchestra.run.smithi193.stdout:/dev/mqueue 2024-04-25T08:54:31.987 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/kernel/debug 2024-04-25T08:54:31.988 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/kernel/tracing 2024-04-25T08:54:31.988 INFO:teuthology.orchestra.run.smithi193.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:54:31.988 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/kernel/config 2024-04-25T08:54:31.988 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/fuse/connections 2024-04-25T08:54:31.988 INFO:teuthology.orchestra.run.smithi193.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:54:31.988 INFO:teuthology.orchestra.run.smithi193.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:54:31.988 INFO:teuthology.orchestra.run.smithi193.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:54:31.988 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:54:31.988 INFO:teuthology.orchestra.run.smithi193.stdout:/run/user/1000 2024-04-25T08:54:31.988 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/ceph 2024-04-25T08:54:31.988 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T08:54:31.988 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T08:54:31.988 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T08:54:31.988 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T08:54:31.989 INFO:teuthology.orchestra.run.smithi193.stdout:/run/netns 2024-04-25T08:54:31.989 INFO:teuthology.orchestra.run.smithi193.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:54:31.989 INFO:teuthology.orchestra.run.smithi193.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:54:31.990 INFO:teuthology.orchestra.run.smithi193.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T08:54:31.990 INFO:teuthology.orchestra.run.smithi193.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T08:54:31.990 INFO:teuthology.orchestra.run.smithi193.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T08:54:31.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:31.991 DEBUG:teuthology.orchestra.run.smithi193:> ls /sys/fs/fuse/connections 2024-04-25T08:54:32.041 INFO:teuthology.orchestra.run.smithi193.stdout:43 2024-04-25T08:54:32.041 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [43] 2024-04-25T08:54:32.041 DEBUG:teuthology.orchestra.run.smithi193:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-04-25T08:54:32.084 DEBUG:teuthology.orchestra.run.smithi193:> sudo modprobe fuse 2024-04-25T08:54:32.107 DEBUG:teuthology.orchestra.run.smithi193:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:54:32.164 INFO:teuthology.orchestra.run.smithi193.stdout:/proc 2024-04-25T08:54:32.164 INFO:teuthology.orchestra.run.smithi193.stdout:/sys 2024-04-25T08:54:32.165 INFO:teuthology.orchestra.run.smithi193.stdout:/dev 2024-04-25T08:54:32.165 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/kernel/security 2024-04-25T08:54:32.165 INFO:teuthology.orchestra.run.smithi193.stdout:/dev/shm 2024-04-25T08:54:32.165 INFO:teuthology.orchestra.run.smithi193.stdout:/dev/pts 2024-04-25T08:54:32.165 INFO:teuthology.orchestra.run.smithi193.stdout:/run 2024-04-25T08:54:32.165 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/cgroup 2024-04-25T08:54:32.165 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/pstore 2024-04-25T08:54:32.165 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/bpf 2024-04-25T08:54:32.165 INFO:teuthology.orchestra.run.smithi193.stdout:/ 2024-04-25T08:54:32.165 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/selinux 2024-04-25T08:54:32.165 INFO:teuthology.orchestra.run.smithi193.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:54:32.165 INFO:teuthology.orchestra.run.smithi193.stdout:/dev/hugepages 2024-04-25T08:54:32.165 INFO:teuthology.orchestra.run.smithi193.stdout:/dev/mqueue 2024-04-25T08:54:32.165 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/kernel/debug 2024-04-25T08:54:32.166 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/kernel/tracing 2024-04-25T08:54:32.166 INFO:teuthology.orchestra.run.smithi193.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:54:32.166 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/kernel/config 2024-04-25T08:54:32.166 INFO:teuthology.orchestra.run.smithi193.stdout:/sys/fs/fuse/connections 2024-04-25T08:54:32.166 INFO:teuthology.orchestra.run.smithi193.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:54:32.166 INFO:teuthology.orchestra.run.smithi193.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:54:32.166 INFO:teuthology.orchestra.run.smithi193.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:54:32.166 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:54:32.166 INFO:teuthology.orchestra.run.smithi193.stdout:/run/user/1000 2024-04-25T08:54:32.166 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/ceph 2024-04-25T08:54:32.166 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/ceph/osd/ceph-4 2024-04-25T08:54:32.166 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/ceph/osd/ceph-5 2024-04-25T08:54:32.166 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/ceph/osd/ceph-6 2024-04-25T08:54:32.166 INFO:teuthology.orchestra.run.smithi193.stdout:/var/lib/ceph/osd/ceph-7 2024-04-25T08:54:32.167 INFO:teuthology.orchestra.run.smithi193.stdout:/run/netns 2024-04-25T08:54:32.167 INFO:teuthology.orchestra.run.smithi193.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:54:32.167 INFO:teuthology.orchestra.run.smithi193.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T08:54:32.167 INFO:teuthology.orchestra.run.smithi193.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-25T08:54:32.167 INFO:teuthology.orchestra.run.smithi193.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T08:54:32.167 INFO:teuthology.orchestra.run.smithi193.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T08:54:32.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:32.167 DEBUG:teuthology.orchestra.run.smithi193:> ls /sys/fs/fuse/connections 2024-04-25T08:54:32.168 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi193.stderr:2024-04-25T08:54:32.174+0000 7f705b58f480 -1 init, newargv = 0x55707dcb89e0 newargc=15 2024-04-25T08:54:32.168 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi193.stderr:ceph-fuse[41456]: starting ceph client 2024-04-25T08:54:32.168 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi193.stderr:ceph-fuse[41456]: starting fuse 2024-04-25T08:54:32.183 INFO:teuthology.orchestra.run.smithi193.stdout:43 2024-04-25T08:54:32.183 INFO:teuthology.orchestra.run.smithi193.stdout:44 2024-04-25T08:54:32.183 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43, 44] 2024-04-25T08:54:32.184 DEBUG:teuthology.orchestra.run.smithi193:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:54:32.184 DEBUG:teuthology.orchestra.run.smithi193:> import glob 2024-04-25T08:54:32.184 DEBUG:teuthology.orchestra.run.smithi193:> import re 2024-04-25T08:54:32.184 DEBUG:teuthology.orchestra.run.smithi193:> import os 2024-04-25T08:54:32.184 DEBUG:teuthology.orchestra.run.smithi193:> import subprocess 2024-04-25T08:54:32.184 DEBUG:teuthology.orchestra.run.smithi193:> 2024-04-25T08:54:32.184 DEBUG:teuthology.orchestra.run.smithi193:> def _find_admin_socket(client_name): 2024-04-25T08:54:32.184 DEBUG:teuthology.orchestra.run.smithi193:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-04-25T08:54:32.184 DEBUG:teuthology.orchestra.run.smithi193:> files = glob.glob(asok_path) 2024-04-25T08:54:32.185 DEBUG:teuthology.orchestra.run.smithi193:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-04-25T08:54:32.185 DEBUG:teuthology.orchestra.run.smithi193:> 2024-04-25T08:54:32.185 DEBUG:teuthology.orchestra.run.smithi193:> # Given a non-glob path, it better be there 2024-04-25T08:54:32.185 DEBUG:teuthology.orchestra.run.smithi193:> if "*" not in asok_path: 2024-04-25T08:54:32.185 DEBUG:teuthology.orchestra.run.smithi193:> assert(len(files) == 1) 2024-04-25T08:54:32.185 DEBUG:teuthology.orchestra.run.smithi193:> return files[0] 2024-04-25T08:54:32.185 DEBUG:teuthology.orchestra.run.smithi193:> 2024-04-25T08:54:32.185 DEBUG:teuthology.orchestra.run.smithi193:> for f in files: 2024-04-25T08:54:32.185 DEBUG:teuthology.orchestra.run.smithi193:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:54:32.185 DEBUG:teuthology.orchestra.run.smithi193:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:54:32.185 DEBUG:teuthology.orchestra.run.smithi193:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:54:32.185 DEBUG:teuthology.orchestra.run.smithi193:> contents = proc_f.read() 2024-04-25T08:54:32.185 DEBUG:teuthology.orchestra.run.smithi193:> if mountpoint in contents: 2024-04-25T08:54:32.185 DEBUG:teuthology.orchestra.run.smithi193:> return f 2024-04-25T08:54:32.185 DEBUG:teuthology.orchestra.run.smithi193:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:54:32.186 DEBUG:teuthology.orchestra.run.smithi193:> 2024-04-25T08:54:32.186 DEBUG:teuthology.orchestra.run.smithi193:> print(_find_admin_socket("client.3")) 2024-04-25T08:54:32.186 DEBUG:teuthology.orchestra.run.smithi193:> ' 2024-04-25T08:54:32.299 INFO:teuthology.orchestra.run.smithi193.stdout:/var/run/ceph/ceph-client.3.41456.asok 2024-04-25T08:54:32.303 INFO:teuthology.orchestra.run.smithi193.stderr:2024-04-25T08:54:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:54:32.310 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.41456.asok 2024-04-25T08:54:32.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:32.310 DEBUG:teuthology.orchestra.run.smithi193:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.41456.asok status 2024-04-25T08:54:32.444 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-04-25T08:54:32.444 INFO:teuthology.orchestra.run.smithi193.stdout: "metadata": { 2024-04-25T08:54:32.444 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:54:32.444 INFO:teuthology.orchestra.run.smithi193.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:54:32.444 INFO:teuthology.orchestra.run.smithi193.stdout: "entity_id": "3", 2024-04-25T08:54:32.445 INFO:teuthology.orchestra.run.smithi193.stdout: "hostname": "smithi193", 2024-04-25T08:54:32.445 INFO:teuthology.orchestra.run.smithi193.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-04-25T08:54:32.445 INFO:teuthology.orchestra.run.smithi193.stdout: "pid": "41456", 2024-04-25T08:54:32.445 INFO:teuthology.orchestra.run.smithi193.stdout: "root": "/" 2024-04-25T08:54:32.445 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T08:54:32.445 INFO:teuthology.orchestra.run.smithi193.stdout: "dentry_count": 0, 2024-04-25T08:54:32.445 INFO:teuthology.orchestra.run.smithi193.stdout: "dentry_pinned_count": 0, 2024-04-25T08:54:32.445 INFO:teuthology.orchestra.run.smithi193.stdout: "id": 4627, 2024-04-25T08:54:32.445 INFO:teuthology.orchestra.run.smithi193.stdout: "inst": { 2024-04-25T08:54:32.445 INFO:teuthology.orchestra.run.smithi193.stdout: "name": { 2024-04-25T08:54:32.445 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "client", 2024-04-25T08:54:32.445 INFO:teuthology.orchestra.run.smithi193.stdout: "num": 4627 2024-04-25T08:54:32.445 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T08:54:32.445 INFO:teuthology.orchestra.run.smithi193.stdout: "addr": { 2024-04-25T08:54:32.446 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "v1", 2024-04-25T08:54:32.446 INFO:teuthology.orchestra.run.smithi193.stdout: "addr": "192.168.0.2:0", 2024-04-25T08:54:32.446 INFO:teuthology.orchestra.run.smithi193.stdout: "nonce": 1877569740 2024-04-25T08:54:32.446 INFO:teuthology.orchestra.run.smithi193.stdout: } 2024-04-25T08:54:32.446 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T08:54:32.446 INFO:teuthology.orchestra.run.smithi193.stdout: "addr": { 2024-04-25T08:54:32.446 INFO:teuthology.orchestra.run.smithi193.stdout: "type": "v1", 2024-04-25T08:54:32.446 INFO:teuthology.orchestra.run.smithi193.stdout: "addr": "192.168.0.2:0", 2024-04-25T08:54:32.446 INFO:teuthology.orchestra.run.smithi193.stdout: "nonce": 1877569740 2024-04-25T08:54:32.446 INFO:teuthology.orchestra.run.smithi193.stdout: }, 2024-04-25T08:54:32.446 INFO:teuthology.orchestra.run.smithi193.stdout: "inst_str": "client.4627 192.168.0.2:0/1877569740", 2024-04-25T08:54:32.446 INFO:teuthology.orchestra.run.smithi193.stdout: "addr_str": "192.168.0.2:0/1877569740", 2024-04-25T08:54:32.446 INFO:teuthology.orchestra.run.smithi193.stdout: "inode_count": 1, 2024-04-25T08:54:32.447 INFO:teuthology.orchestra.run.smithi193.stdout: "mds_epoch": 6, 2024-04-25T08:54:32.447 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_epoch": 20, 2024-04-25T08:54:32.447 INFO:teuthology.orchestra.run.smithi193.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:54:32.447 INFO:teuthology.orchestra.run.smithi193.stdout: "blocklisted": false, 2024-04-25T08:54:32.447 INFO:teuthology.orchestra.run.smithi193.stdout: "fs_name": "cephfs" 2024-04-25T08:54:32.447 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-04-25T08:54:32.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:32.453 DEBUG:teuthology.orchestra.run.smithi043:> stat --file-system '--printf=%T 2024-04-25T08:54:32.453 DEBUG:teuthology.orchestra.run.smithi043:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T08:54:32.470 INFO:teuthology.orchestra.run.smithi043.stdout:fuseblk 2024-04-25T08:54:32.470 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T08:54:32.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:32.470 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:54:32.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:32.539 DEBUG:teuthology.orchestra.run.smithi043:> stat --file-system '--printf=%T 2024-04-25T08:54:32.540 DEBUG:teuthology.orchestra.run.smithi043:> ' -- /home/ubuntu/cephtest/mnt.2 2024-04-25T08:54:32.596 INFO:teuthology.orchestra.run.smithi043.stdout:fuseblk 2024-04-25T08:54:32.596 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-04-25T08:54:32.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:32.597 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-25T08:54:32.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:32.665 DEBUG:teuthology.orchestra.run.smithi193:> stat --file-system '--printf=%T 2024-04-25T08:54:32.665 DEBUG:teuthology.orchestra.run.smithi193:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-25T08:54:32.680 INFO:teuthology.orchestra.run.smithi193.stdout:fuseblk 2024-04-25T08:54:32.680 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-25T08:54:32.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:32.681 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T08:54:32.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:32.745 DEBUG:teuthology.orchestra.run.smithi193:> stat --file-system '--printf=%T 2024-04-25T08:54:32.745 DEBUG:teuthology.orchestra.run.smithi193:> ' -- /home/ubuntu/cephtest/mnt.3 2024-04-25T08:54:32.801 INFO:teuthology.orchestra.run.smithi193.stdout:fuseblk 2024-04-25T08:54:32.801 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-04-25T08:54:32.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:32.801 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-25T08:54:32.867 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-25T08:54:32.884 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:54:32.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:32.998+0000 7fe844570640 1 Processor -- start 2024-04-25T08:54:32.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:32.998+0000 7fe844570640 1 -- start start 2024-04-25T08:54:32.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:32.998+0000 7fe844570640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe83c15dcf0 0x7fe83c06e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:32.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:32.999+0000 7fe844570640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fe83c06ebf0 0x7fe83c06f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:32.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:32.999+0000 7fe844570640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe83c0d0d80 con 0x7fe83c1616e0 2024-04-25T08:54:32.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:32.999+0000 7fe844570640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe83c06f590 con 0x7fe83c06ebf0 2024-04-25T08:54:32.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:32.999+0000 7fe844570640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c06c050 con 0x7fe83c15dcf0 2024-04-25T08:54:32.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:32.999+0000 7fe842b0d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe83c1616e0 0x7fe83c161b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:55640/0 (socket says 172.21.15.43:55640) 2024-04-25T08:54:32.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:32.999+0000 7fe842b0d640 1 -- 172.21.15.43:0/1293118028 learned_addr learned my addr 172.21.15.43:0/1293118028 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:32.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:32.999+0000 7fe84130a640 1 -- 172.21.15.43:0/1293118028 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3027246944 0 0) 0x7fe83c0d0d80 con 0x7fe83c1616e0 2024-04-25T08:54:32.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:32.999+0000 7fe841b0b640 1 --2- 172.21.15.43:0/1293118028 >> v2:172.21.15.193:3300/0 conn(0x7fe83c06ebf0 0x7fe83c06f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:32.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:32.999+0000 7fe84130a640 1 -- 172.21.15.43:0/1293118028 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe828003520 con 0x7fe83c1616e0 2024-04-25T08:54:32.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:32.999+0000 7fe84230c640 1 --2- 172.21.15.43:0/1293118028 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe83c15dcf0 0x7fe83c06e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:32.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:32.999+0000 7fe84130a640 1 -- 172.21.15.43:0/1293118028 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1353941511 0 0) 0x7fe828003520 con 0x7fe83c1616e0 2024-04-25T08:54:32.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:32.999+0000 7fe84130a640 1 -- 172.21.15.43:0/1293118028 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe83c15dcf0 msgr2=0x7fe83c06e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:32.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:32.999+0000 7fe84130a640 1 --2- 172.21.15.43:0/1293118028 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe83c15dcf0 0x7fe83c06e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:32.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:32.999+0000 7fe84130a640 1 -- 172.21.15.43:0/1293118028 >> v2:172.21.15.193:3300/0 conn(0x7fe83c06ebf0 msgr2=0x7fe83c06f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:32.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:32.999+0000 7fe84130a640 1 --2- 172.21.15.43:0/1293118028 >> v2:172.21.15.193:3300/0 conn(0x7fe83c06ebf0 0x7fe83c06f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:32.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:32.999+0000 7fe84130a640 1 -- 172.21.15.43:0/1293118028 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe83c0d04e0 con 0x7fe83c1616e0 2024-04-25T08:54:32.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:32.999+0000 7fe84130a640 1 -- 172.21.15.43:0/1293118028 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7fe8340019f0 con 0x7fe83c1616e0 2024-04-25T08:54:32.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:32.999+0000 7fe84130a640 1 -- 172.21.15.43:0/1293118028 >> v1:172.21.15.43:6789/0 conn(0x7fe83c1616e0 legacy=0x7fe83c161b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:32.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:32.999+0000 7fe84130a640 1 --2- 172.21.15.43:0/1293118028 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe83c15dcf0 0x7fe8280039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:32.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:32.999+0000 7fe84130a640 1 --2- 172.21.15.43:0/1293118028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe828003f30 0x7fe828004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:32.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:32.999+0000 7fe84130a640 1 --2- 172.21.15.43:0/1293118028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe828004d40 0x7fe8280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:32.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:32.999+0000 7fe84130a640 1 -- 172.21.15.43:0/1293118028 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c06c050 con 0x7fe828003f30 2024-04-25T08:54:32.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:32.999+0000 7fe84130a640 1 -- 172.21.15.43:0/1293118028 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe82800b9c0 con 0x7fe828004d40 2024-04-25T08:54:32.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:32.999+0000 7fe84130a640 1 -- 172.21.15.43:0/1293118028 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe82800bb90 con 0x7fe83c15dcf0 2024-04-25T08:54:32.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.000+0000 7fe84230c640 1 --2- 172.21.15.43:0/1293118028 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe83c15dcf0 0x7fe8280039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:32.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.000+0000 7fe841b0b640 1 --2- 172.21.15.43:0/1293118028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe828003f30 0x7fe828004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:32.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.000+0000 7fe842b0d640 1 --2- 172.21.15.43:0/1293118028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe828004d40 0x7fe8280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:32.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.000+0000 7fe84230c640 1 -- 172.21.15.43:0/1293118028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe828004d40 msgr2=0x7fe8280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:32.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.000+0000 7fe84230c640 1 --2- 172.21.15.43:0/1293118028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe828004d40 0x7fe8280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:32.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.000+0000 7fe84230c640 1 -- 172.21.15.43:0/1293118028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe828003f30 msgr2=0x7fe828004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:32.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.000+0000 7fe84230c640 1 --2- 172.21.15.43:0/1293118028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe828003f30 0x7fe828004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:32.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.000+0000 7fe84230c640 1 -- 172.21.15.43:0/1293118028 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe828003520 con 0x7fe83c15dcf0 2024-04-25T08:54:32.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.000+0000 7fe841b0b640 1 --2- 172.21.15.43:0/1293118028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe828003f30 0x7fe828004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:54:32.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.000+0000 7fe84230c640 1 --2- 172.21.15.43:0/1293118028 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe83c15dcf0 0x7fe8280039b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe838009af0 tx=0x7fe828005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2cf34c557e96842 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:32.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.001+0000 7fe84130a640 1 -- 172.21.15.43:0/1293118028 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe838036070 con 0x7fe83c15dcf0 2024-04-25T08:54:32.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.001+0000 7fe84130a640 1 -- 172.21.15.43:0/1293118028 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe838031400 con 0x7fe83c15dcf0 2024-04-25T08:54:32.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.001+0000 7fe84130a640 1 -- 172.21.15.43:0/1293118028 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe838009690 con 0x7fe83c15dcf0 2024-04-25T08:54:32.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.001+0000 7fe844570640 1 -- 172.21.15.43:0/1293118028 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe83c15dcf0 msgr2=0x7fe8280039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:32.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.001+0000 7fe844570640 1 --2- 172.21.15.43:0/1293118028 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe83c15dcf0 0x7fe8280039b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe838009af0 tx=0x7fe828005830 comp rx=0 tx=0).stop 2024-04-25T08:54:32.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.001+0000 7fe842b0d640 1 -- 172.21.15.43:0/1293118028 reap_dead start 2024-04-25T08:54:32.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.001+0000 7fe844570640 1 -- 172.21.15.43:0/1293118028 shutdown_connections 2024-04-25T08:54:32.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.001+0000 7fe844570640 1 -- 172.21.15.43:0/1293118028 >> 172.21.15.43:0/1293118028 conn(0x7fe83c0d1700 msgr2=0x7fe83c0d1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:32.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.001+0000 7fe844570640 1 -- 172.21.15.43:0/1293118028 shutdown_connections 2024-04-25T08:54:32.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.001+0000 7fe844570640 1 -- 172.21.15.43:0/1293118028 wait complete. 2024-04-25T08:54:32.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.002+0000 7fe844570640 1 Processor -- start 2024-04-25T08:54:32.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.003+0000 7fe844570640 1 -- start start 2024-04-25T08:54:32.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.003+0000 7fe844570640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe83c15dcf0 0x7fe83c0c5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:32.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.003+0000 7fe844570640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe83c0c72c0 0x7fe83c160a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:32.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.003+0000 7fe844570640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe83c156bf0 0x7fe83c15b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:32.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.003+0000 7fe844570640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c06bce0 con 0x7fe83c156bf0 2024-04-25T08:54:32.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.003+0000 7fe844570640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c0d0aa0 con 0x7fe83c0c72c0 2024-04-25T08:54:32.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.003+0000 7fe844570640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c0b0d50 con 0x7fe83c15dcf0 2024-04-25T08:54:32.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.003+0000 7fe842b0d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe83c156bf0 0x7fe83c15b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:32.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.003+0000 7fe842b0d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe83c156bf0 0x7fe83c15b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39976/0 (socket says 172.21.15.43:39976) 2024-04-25T08:54:32.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.003+0000 7fe842b0d640 1 -- 172.21.15.43:0/4018988717 learned_addr learned my addr 172.21.15.43:0/4018988717 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:32.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.003+0000 7fe841b0b640 1 --2- 172.21.15.43:0/4018988717 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe83c0c72c0 0x7fe83c160a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:32.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.003+0000 7fe84230c640 1 --2- 172.21.15.43:0/4018988717 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe83c15dcf0 0x7fe83c0c5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:32.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.004+0000 7fe842b0d640 1 -- 172.21.15.43:0/4018988717 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe83c15dcf0 msgr2=0x7fe83c0c5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:32.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.004+0000 7fe842b0d640 1 --2- 172.21.15.43:0/4018988717 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe83c15dcf0 0x7fe83c0c5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:32.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.004+0000 7fe842b0d640 1 -- 172.21.15.43:0/4018988717 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe83c0c72c0 msgr2=0x7fe83c160a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:32.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.004+0000 7fe842b0d640 1 --2- 172.21.15.43:0/4018988717 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe83c0c72c0 0x7fe83c160a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:32.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.004+0000 7fe842b0d640 1 -- 172.21.15.43:0/4018988717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe83c0c6220 con 0x7fe83c156bf0 2024-04-25T08:54:32.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.004+0000 7fe842b0d640 1 --2- 172.21.15.43:0/4018988717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe83c156bf0 0x7fe83c15b080 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe83c161ac0 tx=0x7fe834007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:33.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.004+0000 7fe832ffd640 1 -- 172.21.15.43:0/4018988717 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe834017020 con 0x7fe83c156bf0 2024-04-25T08:54:33.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.004+0000 7fe832ffd640 1 -- 172.21.15.43:0/4018988717 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe83400ac20 con 0x7fe83c156bf0 2024-04-25T08:54:33.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.004+0000 7fe844570640 1 -- 172.21.15.43:0/4018988717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe83c0c6500 con 0x7fe83c156bf0 2024-04-25T08:54:33.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.004+0000 7fe832ffd640 1 -- 172.21.15.43:0/4018988717 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82c009690 con 0x7fe83c156bf0 2024-04-25T08:54:33.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.004+0000 7fe844570640 1 -- 172.21.15.43:0/4018988717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe83c0c6ac0 con 0x7fe83c156bf0 2024-04-25T08:54:33.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.005+0000 7fe844570640 1 -- 172.21.15.43:0/4018988717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe83c06fbb0 con 0x7fe83c156bf0 2024-04-25T08:54:33.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.005+0000 7fe832ffd640 1 -- 172.21.15.43:0/4018988717 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe83401a020 con 0x7fe83c156bf0 2024-04-25T08:54:33.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.006+0000 7fe832ffd640 1 --2- 172.21.15.43:0/4018988717 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe82006b4d0 0x7fe82006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:33.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.006+0000 7fe832ffd640 1 -- 172.21.15.43:0/4018988717 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe83408a670 con 0x7fe83c156bf0 2024-04-25T08:54:33.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.006+0000 7fe84230c640 1 --2- 172.21.15.43:0/4018988717 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe82006b4d0 0x7fe82006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:33.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.007+0000 7fe84230c640 1 --2- 172.21.15.43:0/4018988717 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe82006b4d0 0x7fe82006d990 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe838033000 tx=0x7fe838008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:33.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.010+0000 7fe832ffd640 1 -- 172.21.15.43:0/4018988717 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe83c06fbb0 con 0x7fe83c156bf0 2024-04-25T08:54:33.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.191+0000 7fe844570640 1 -- 172.21.15.43:0/4018988717 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe83c074ef0 con 0x7fe83c156bf0 2024-04-25T08:54:33.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.192+0000 7fe832ffd640 1 -- 172.21.15.43:0/4018988717 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16569 (secure 0 0 0) 0x7fe834021020 con 0x7fe83c156bf0 2024-04-25T08:54:33.182 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:54:33.182 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":20,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:54:13.593188+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:54:09.757978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:54:10.926008+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:54:33.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.194+0000 7fe844570640 1 -- 172.21.15.43:0/4018988717 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe82006b4d0 msgr2=0x7fe82006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:33.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.194+0000 7fe844570640 1 --2- 172.21.15.43:0/4018988717 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe82006b4d0 0x7fe82006d990 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe838033000 tx=0x7fe838008030 comp rx=0 tx=0).stop 2024-04-25T08:54:33.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.194+0000 7fe844570640 1 -- 172.21.15.43:0/4018988717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe83c156bf0 msgr2=0x7fe83c15b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:33.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.194+0000 7fe844570640 1 --2- 172.21.15.43:0/4018988717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe83c156bf0 0x7fe83c15b080 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe83c161ac0 tx=0x7fe834007bd0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.194+0000 7fe844570640 1 -- 172.21.15.43:0/4018988717 shutdown_connections 2024-04-25T08:54:33.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.194+0000 7fe844570640 1 --2- 172.21.15.43:0/4018988717 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe83c156bf0 0x7fe83c15b080 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.194+0000 7fe844570640 1 --2- 172.21.15.43:0/4018988717 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe82006b4d0 0x7fe82006d990 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.194+0000 7fe844570640 1 --2- 172.21.15.43:0/4018988717 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe83c0c72c0 0x7fe83c160a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.194+0000 7fe844570640 1 --2- 172.21.15.43:0/4018988717 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe83c15dcf0 0x7fe83c0c5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.194+0000 7fe844570640 1 -- 172.21.15.43:0/4018988717 >> 172.21.15.43:0/4018988717 conn(0x7fe83c0d1700 msgr2=0x7fe83c0b27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:33.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.194+0000 7fe844570640 1 -- 172.21.15.43:0/4018988717 shutdown_connections 2024-04-25T08:54:33.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.194+0000 7fe844570640 1 -- 172.21.15.43:0/4018988717 wait complete. 2024-04-25T08:54:33.195 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:54:09.757978+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:54:10.926008+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:54:33.195 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:54:33.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.309+0000 7fe6ba7e0640 1 Processor -- start 2024-04-25T08:54:33.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.309+0000 7fe6ba7e0640 1 -- start start 2024-04-25T08:54:33.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.310+0000 7fe6ba7e0640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe6b415e090 0x7fe6b4161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:33.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.310+0000 7fe6ba7e0640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe6b40b5c70 0x7fe6b40d2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:33.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.310+0000 7fe6ba7e0640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6b4162610 con 0x7fe6b4161fe0 2024-04-25T08:54:33.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.310+0000 7fe6ba7e0640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6b40d1da0 con 0x7fe6b415e090 2024-04-25T08:54:33.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.310+0000 7fe6ba7e0640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b40d1750 con 0x7fe6b40b5c70 2024-04-25T08:54:33.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.310+0000 7fe6b3fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe6b415e090 0x7fe6b4161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:33.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.310+0000 7fe6b37fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe6b4161fe0 0x7fe6b40b5560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:55644/0 (socket says 172.21.15.43:55644) 2024-04-25T08:54:33.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.310+0000 7fe6b3fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe6b415e090 0x7fe6b4161aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39986/0 (socket says 172.21.15.43:39986) 2024-04-25T08:54:33.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.310+0000 7fe6b37fe640 1 -- 172.21.15.43:0/3451378084 learned_addr learned my addr 172.21.15.43:0/3451378084 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:33.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.310+0000 7fe6b8d7d640 1 --2- 172.21.15.43:0/3451378084 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe6b40b5c70 0x7fe6b40d2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:33.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.310+0000 7fe6b3fff640 1 -- 172.21.15.43:0/3451378084 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe6b40b5c70 msgr2=0x7fe6b40d2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:33.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.310+0000 7fe6b3fff640 1 --2- 172.21.15.43:0/3451378084 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe6b40b5c70 0x7fe6b40d2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.310+0000 7fe6b3fff640 1 -- 172.21.15.43:0/3451378084 >> v1:172.21.15.43:6789/0 conn(0x7fe6b4161fe0 legacy=0x7fe6b40b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:33.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.310+0000 7fe6b3fff640 1 -- 172.21.15.43:0/3451378084 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b40d1a30 con 0x7fe6b415e090 2024-04-25T08:54:33.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.310+0000 7fe6b8d7d640 1 --2- 172.21.15.43:0/3451378084 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe6b40b5c70 0x7fe6b40d2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:33.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.310+0000 7fe6b2ffd640 1 -- 172.21.15.43:0/3451378084 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 123550950 0 0) 0x7fe6b4162610 con 0x7fe6b4161fe0 2024-04-25T08:54:33.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.311+0000 7fe6b3fff640 1 --2- 172.21.15.43:0/3451378084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6b415e090 0x7fe6b4161aa0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe69c0097a0 tx=0x7fe69c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9403d768c05b2e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:33.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.311+0000 7fe6b2ffd640 1 -- 172.21.15.43:0/3451378084 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c037070 con 0x7fe6b415e090 2024-04-25T08:54:33.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.311+0000 7fe6b2ffd640 1 -- 172.21.15.43:0/3451378084 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe69c032440 con 0x7fe6b415e090 2024-04-25T08:54:33.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.311+0000 7fe6b2ffd640 1 -- 172.21.15.43:0/3451378084 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c032810 con 0x7fe6b415e090 2024-04-25T08:54:33.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.312+0000 7fe6ba7e0640 1 -- 172.21.15.43:0/3451378084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6b415e090 msgr2=0x7fe6b4161aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:33.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.312+0000 7fe6ba7e0640 1 --2- 172.21.15.43:0/3451378084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6b415e090 0x7fe6b4161aa0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe69c0097a0 tx=0x7fe69c02a210 comp rx=0 tx=0).stop 2024-04-25T08:54:33.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.312+0000 7fe6ba7e0640 1 -- 172.21.15.43:0/3451378084 shutdown_connections 2024-04-25T08:54:33.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.312+0000 7fe6ba7e0640 1 --2- 172.21.15.43:0/3451378084 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe6b40b5c70 0x7fe6b40d2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.312+0000 7fe6ba7e0640 1 --2- 172.21.15.43:0/3451378084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6b415e090 0x7fe6b4161aa0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.312+0000 7fe6ba7e0640 1 -- 172.21.15.43:0/3451378084 >> 172.21.15.43:0/3451378084 conn(0x7fe6b40d0560 msgr2=0x7fe6b40d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:33.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.312+0000 7fe6ba7e0640 1 -- 172.21.15.43:0/3451378084 shutdown_connections 2024-04-25T08:54:33.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.312+0000 7fe6ba7e0640 1 -- 172.21.15.43:0/3451378084 wait complete. 2024-04-25T08:54:33.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.313+0000 7fe6ba7e0640 1 Processor -- start 2024-04-25T08:54:33.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.313+0000 7fe6ba7e0640 1 -- start start 2024-04-25T08:54:33.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.314+0000 7fe6ba7e0640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe6b40b5c70 0x7fe6b40cae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:33.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.314+0000 7fe6ba7e0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6b415e090 0x7fe6b415fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:33.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.314+0000 7fe6ba7e0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6b4161fe0 0x7fe6b4160410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:33.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.314+0000 7fe6ba7e0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b40d2770 con 0x7fe6b415e090 2024-04-25T08:54:33.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.314+0000 7fe6ba7e0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b40b3280 con 0x7fe6b4161fe0 2024-04-25T08:54:33.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.314+0000 7fe6ba7e0640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b40d28f0 con 0x7fe6b40b5c70 2024-04-25T08:54:33.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.314+0000 7fe6b37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6b415e090 0x7fe6b415fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:33.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.314+0000 7fe6b37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6b415e090 0x7fe6b415fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39998/0 (socket says 172.21.15.43:39998) 2024-04-25T08:54:33.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.314+0000 7fe6b37fe640 1 -- 172.21.15.43:0/466317725 learned_addr learned my addr 172.21.15.43:0/466317725 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:33.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.314+0000 7fe6b3fff640 1 --2- 172.21.15.43:0/466317725 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe6b40b5c70 0x7fe6b40cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:33.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.314+0000 7fe6b8d7d640 1 --2- 172.21.15.43:0/466317725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6b4161fe0 0x7fe6b4160410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:33.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.314+0000 7fe6b37fe640 1 -- 172.21.15.43:0/466317725 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe6b40b5c70 msgr2=0x7fe6b40cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:33.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.314+0000 7fe6b37fe640 1 --2- 172.21.15.43:0/466317725 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe6b40b5c70 0x7fe6b40cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.314+0000 7fe6b37fe640 1 -- 172.21.15.43:0/466317725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6b4161fe0 msgr2=0x7fe6b4160410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:33.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.314+0000 7fe6b37fe640 1 --2- 172.21.15.43:0/466317725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6b4161fe0 0x7fe6b4160410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.314+0000 7fe6b37fe640 1 -- 172.21.15.43:0/466317725 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b40ccb30 con 0x7fe6b415e090 2024-04-25T08:54:33.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.314+0000 7fe6b37fe640 1 --2- 172.21.15.43:0/466317725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6b415e090 0x7fe6b415fed0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe6a400c430 tx=0x7fe6a400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:33.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.314+0000 7fe6b0ff9640 1 -- 172.21.15.43:0/466317725 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a40074e0 con 0x7fe6b415e090 2024-04-25T08:54:33.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.314+0000 7fe6b0ff9640 1 -- 172.21.15.43:0/466317725 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a40047e0 con 0x7fe6b415e090 2024-04-25T08:54:33.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.315+0000 7fe6ba7e0640 1 -- 172.21.15.43:0/466317725 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6b4160b90 con 0x7fe6b415e090 2024-04-25T08:54:33.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.315+0000 7fe6b0ff9640 1 -- 172.21.15.43:0/466317725 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4004bd0 con 0x7fe6b415e090 2024-04-25T08:54:33.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.315+0000 7fe6ba7e0640 1 -- 172.21.15.43:0/466317725 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6b40cd0b0 con 0x7fe6b415e090 2024-04-25T08:54:33.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.315+0000 7fe6ba7e0640 1 -- 172.21.15.43:0/466317725 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6b406bf00 con 0x7fe6b415e090 2024-04-25T08:54:33.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.316+0000 7fe6b0ff9640 1 -- 172.21.15.43:0/466317725 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe6a4011070 con 0x7fe6b415e090 2024-04-25T08:54:33.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.316+0000 7fe6b0ff9640 1 --2- 172.21.15.43:0/466317725 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe69406b4d0 0x7fe69406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:33.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.316+0000 7fe6b0ff9640 1 -- 172.21.15.43:0/466317725 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe6a4089d30 con 0x7fe6b415e090 2024-04-25T08:54:33.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.317+0000 7fe6b3fff640 1 --2- 172.21.15.43:0/466317725 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe69406b4d0 0x7fe69406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:33.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.317+0000 7fe6b3fff640 1 --2- 172.21.15.43:0/466317725 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe69406b4d0 0x7fe69406d990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe69c034000 tx=0x7fe69c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:33.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.321+0000 7fe6b0ff9640 1 -- 172.21.15.43:0/466317725 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe6b406bf00 con 0x7fe6b415e090 2024-04-25T08:54:33.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.498+0000 7fe6ba7e0640 1 -- 172.21.15.43:0/466317725 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe6b4074f60 con 0x7fe6b415e090 2024-04-25T08:54:33.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.499+0000 7fe6b0ff9640 1 -- 172.21.15.43:0/466317725 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe6a4058560 con 0x7fe6b415e090 2024-04-25T08:54:33.489 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T08:54:33.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.501+0000 7fe6ba7e0640 1 -- 172.21.15.43:0/466317725 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe69406b4d0 msgr2=0x7fe69406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:33.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.501+0000 7fe6ba7e0640 1 --2- 172.21.15.43:0/466317725 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe69406b4d0 0x7fe69406d990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe69c034000 tx=0x7fe69c02e000 comp rx=0 tx=0).stop 2024-04-25T08:54:33.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.501+0000 7fe6ba7e0640 1 -- 172.21.15.43:0/466317725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6b415e090 msgr2=0x7fe6b415fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:33.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.501+0000 7fe6ba7e0640 1 --2- 172.21.15.43:0/466317725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6b415e090 0x7fe6b415fed0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe6a400c430 tx=0x7fe6a400c8f0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.501+0000 7fe6ba7e0640 1 -- 172.21.15.43:0/466317725 shutdown_connections 2024-04-25T08:54:33.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.501+0000 7fe6ba7e0640 1 --2- 172.21.15.43:0/466317725 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe69406b4d0 0x7fe69406d990 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.501+0000 7fe6ba7e0640 1 --2- 172.21.15.43:0/466317725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6b4161fe0 0x7fe6b4160410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.501+0000 7fe6ba7e0640 1 --2- 172.21.15.43:0/466317725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6b415e090 0x7fe6b415fed0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.501+0000 7fe6ba7e0640 1 --2- 172.21.15.43:0/466317725 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe6b40b5c70 0x7fe6b40cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.501+0000 7fe6ba7e0640 1 -- 172.21.15.43:0/466317725 >> 172.21.15.43:0/466317725 conn(0x7fe6b40d0560 msgr2=0x7fe6b40c6db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:33.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.501+0000 7fe6ba7e0640 1 -- 172.21.15.43:0/466317725 shutdown_connections 2024-04-25T08:54:33.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.501+0000 7fe6ba7e0640 1 -- 172.21.15.43:0/466317725 wait complete. 2024-04-25T08:54:33.502 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:54:33.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.616+0000 7f997be09640 1 Processor -- start 2024-04-25T08:54:33.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.617+0000 7f997be09640 1 -- start start 2024-04-25T08:54:33.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.617+0000 7f997be09640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f99740b59e0 0x7f99740d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:33.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.617+0000 7f997be09640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99740b5e70 con 0x7f99740b5560 2024-04-25T08:54:33.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.618+0000 7f997be09640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9974162610 con 0x7f997415e090 2024-04-25T08:54:33.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.618+0000 7f997be09640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99740d1da0 con 0x7f99740b59e0 2024-04-25T08:54:33.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.618+0000 7f99793a4640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f99740b5560 0x7f99740d2230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:55652/0 (socket says 172.21.15.43:55652) 2024-04-25T08:54:33.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.618+0000 7f99793a4640 1 -- 172.21.15.43:0/3661694465 learned_addr learned my addr 172.21.15.43:0/3661694465 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:33.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.618+0000 7f997a3a6640 1 --2- 172.21.15.43:0/3661694465 >> v2:172.21.15.43:3300/0 conn(0x7f99740b59e0 0x7f99740d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:33.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.618+0000 7f997a3a6640 1 -- 172.21.15.43:0/3661694465 >> v1:172.21.15.193:6789/0 conn(0x7f997415e090 legacy=0x7f9974161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:33.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.618+0000 7f9978ba3640 1 -- 172.21.15.43:0/3661694465 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3463707340 0 0) 0x7f9974162610 con 0x7f997415e090 2024-04-25T08:54:33.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.618+0000 7f997a3a6640 1 -- 172.21.15.43:0/3661694465 >> v1:172.21.15.43:6789/0 conn(0x7f99740b5560 legacy=0x7f99740d2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:33.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.618+0000 7f997a3a6640 1 -- 172.21.15.43:0/3661694465 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99740d1750 con 0x7f99740b59e0 2024-04-25T08:54:33.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.619+0000 7f997a3a6640 1 --2- 172.21.15.43:0/3661694465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99740b59e0 0x7f99740d2940 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9970009d90 tx=0x7f997002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7d89ef69df9743c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:33.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.619+0000 7f9978ba3640 1 -- 172.21.15.43:0/3661694465 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9970007740 con 0x7f99740b59e0 2024-04-25T08:54:33.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.619+0000 7f9978ba3640 1 -- 172.21.15.43:0/3661694465 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99700078e0 con 0x7f99740b59e0 2024-04-25T08:54:33.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.619+0000 7f9978ba3640 1 -- 172.21.15.43:0/3661694465 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9970007cb0 con 0x7f99740b59e0 2024-04-25T08:54:33.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.619+0000 7f997be09640 1 -- 172.21.15.43:0/3661694465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99740b59e0 msgr2=0x7f99740d2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:33.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.619+0000 7f997be09640 1 --2- 172.21.15.43:0/3661694465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99740b59e0 0x7f99740d2940 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9970009d90 tx=0x7f997002a680 comp rx=0 tx=0).stop 2024-04-25T08:54:33.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.619+0000 7f997be09640 1 -- 172.21.15.43:0/3661694465 shutdown_connections 2024-04-25T08:54:33.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.619+0000 7f997be09640 1 --2- 172.21.15.43:0/3661694465 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99740b59e0 0x7f99740d2940 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.619+0000 7f997be09640 1 -- 172.21.15.43:0/3661694465 >> 172.21.15.43:0/3661694465 conn(0x7f99740d0560 msgr2=0x7f99740d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:33.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.620+0000 7f997be09640 1 -- 172.21.15.43:0/3661694465 shutdown_connections 2024-04-25T08:54:33.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.620+0000 7f997be09640 1 -- 172.21.15.43:0/3661694465 wait complete. 2024-04-25T08:54:33.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.621+0000 7f997be09640 1 Processor -- start 2024-04-25T08:54:33.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.621+0000 7f997be09640 1 -- start start 2024-04-25T08:54:33.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.621+0000 7f997be09640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99740b5560 0x7f9974160fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:33.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.621+0000 7f997be09640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f99740b59e0 0x7f99740cac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:33.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.621+0000 7f997be09640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f997415e090 0x7f997414ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:33.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.621+0000 7f997be09640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99740b3280 con 0x7f99740b5560 2024-04-25T08:54:33.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.621+0000 7f997be09640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99741628f0 con 0x7f997415e090 2024-04-25T08:54:33.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.621+0000 7f997be09640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99741621b0 con 0x7f99740b59e0 2024-04-25T08:54:33.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.622+0000 7f9979ba5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99740b5560 0x7f9974160fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:33.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.622+0000 7f9979ba5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99740b5560 0x7f9974160fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40010/0 (socket says 172.21.15.43:40010) 2024-04-25T08:54:33.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.622+0000 7f9979ba5640 1 -- 172.21.15.43:0/1014606334 learned_addr learned my addr 172.21.15.43:0/1014606334 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:33.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.622+0000 7f997a3a6640 1 --2- 172.21.15.43:0/1014606334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f997415e090 0x7f997414ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:33.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.622+0000 7f99793a4640 1 --2- 172.21.15.43:0/1014606334 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f99740b59e0 0x7f99740cac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:33.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.622+0000 7f9979ba5640 1 -- 172.21.15.43:0/1014606334 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f99740b59e0 msgr2=0x7f99740cac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:33.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.622+0000 7f9979ba5640 1 --2- 172.21.15.43:0/1014606334 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f99740b59e0 0x7f99740cac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.622+0000 7f9979ba5640 1 -- 172.21.15.43:0/1014606334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f997415e090 msgr2=0x7f997414ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:33.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.622+0000 7f9979ba5640 1 --2- 172.21.15.43:0/1014606334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f997415e090 0x7f997414ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.622+0000 7f9979ba5640 1 -- 172.21.15.43:0/1014606334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f997415f100 con 0x7f99740b5560 2024-04-25T08:54:33.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.622+0000 7f9979ba5640 1 --2- 172.21.15.43:0/1014606334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99740b5560 0x7f9974160fc0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f996c00c3f0 tx=0x7f996c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:33.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.622+0000 7f996a7fc640 1 -- 172.21.15.43:0/1014606334 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c017070 con 0x7f99740b5560 2024-04-25T08:54:33.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.622+0000 7f996a7fc640 1 -- 172.21.15.43:0/1014606334 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996c0035f0 con 0x7f99740b5560 2024-04-25T08:54:33.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.622+0000 7f997be09640 1 -- 172.21.15.43:0/1014606334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f997415f3c0 con 0x7f99740b5560 2024-04-25T08:54:33.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.622+0000 7f996a7fc640 1 -- 172.21.15.43:0/1014606334 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c0039e0 con 0x7f99740b5560 2024-04-25T08:54:33.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.622+0000 7f997be09640 1 -- 172.21.15.43:0/1014606334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f997415f980 con 0x7f99740b5560 2024-04-25T08:54:33.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.623+0000 7f997be09640 1 -- 172.21.15.43:0/1014606334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f997406bf00 con 0x7f99740b5560 2024-04-25T08:54:33.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.624+0000 7f996a7fc640 1 -- 172.21.15.43:0/1014606334 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f996c01a070 con 0x7f99740b5560 2024-04-25T08:54:33.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.624+0000 7f996a7fc640 1 --2- 172.21.15.43:0/1014606334 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f995806b4d0 0x7f995806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:33.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.624+0000 7f996a7fc640 1 -- 172.21.15.43:0/1014606334 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f996c0895b0 con 0x7f99740b5560 2024-04-25T08:54:33.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.625+0000 7f99793a4640 1 --2- 172.21.15.43:0/1014606334 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f995806b4d0 0x7f995806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:33.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.625+0000 7f99793a4640 1 --2- 172.21.15.43:0/1014606334 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f995806b4d0 0x7f995806d990 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9974158cf0 tx=0x7f9964000bc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:33.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.629+0000 7f996a7fc640 1 -- 172.21.15.43:0/1014606334 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f997406bf00 con 0x7f99740b5560 2024-04-25T08:54:33.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.808+0000 7f997be09640 1 -- 172.21.15.43:0/1014606334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9974074f60 con 0x7f99740b5560 2024-04-25T08:54:33.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.809+0000 7f996a7fc640 1 -- 172.21.15.43:0/1014606334 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f996c057d50 con 0x7f99740b5560 2024-04-25T08:54:33.799 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T08:54:33.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.811+0000 7f997be09640 1 -- 172.21.15.43:0/1014606334 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f995806b4d0 msgr2=0x7f995806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:33.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.811+0000 7f997be09640 1 --2- 172.21.15.43:0/1014606334 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f995806b4d0 0x7f995806d990 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9974158cf0 tx=0x7f9964000bc0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.811+0000 7f997be09640 1 -- 172.21.15.43:0/1014606334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99740b5560 msgr2=0x7f9974160fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:33.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.811+0000 7f997be09640 1 --2- 172.21.15.43:0/1014606334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99740b5560 0x7f9974160fc0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f996c00c3f0 tx=0x7f996c00c8b0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.811+0000 7f997be09640 1 -- 172.21.15.43:0/1014606334 shutdown_connections 2024-04-25T08:54:33.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.811+0000 7f997be09640 1 --2- 172.21.15.43:0/1014606334 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f995806b4d0 0x7f995806d990 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.811+0000 7f997be09640 1 --2- 172.21.15.43:0/1014606334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f997415e090 0x7f997414ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.811+0000 7f997be09640 1 --2- 172.21.15.43:0/1014606334 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f99740b59e0 0x7f99740cac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.811+0000 7f997be09640 1 --2- 172.21.15.43:0/1014606334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99740b5560 0x7f9974160fc0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.811+0000 7f997be09640 1 -- 172.21.15.43:0/1014606334 >> 172.21.15.43:0/1014606334 conn(0x7f99740d0560 msgr2=0x7f99740c69c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:33.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.811+0000 7f997be09640 1 -- 172.21.15.43:0/1014606334 shutdown_connections 2024-04-25T08:54:33.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.811+0000 7f997be09640 1 -- 172.21.15.43:0/1014606334 wait complete. 2024-04-25T08:54:33.811 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:54:33.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.926+0000 7f8949088640 1 Processor -- start 2024-04-25T08:54:33.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.926+0000 7f8949088640 1 -- start start 2024-04-25T08:54:33.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.927+0000 7f8949088640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f894415e090 0x7f8944161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:33.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.927+0000 7f8949088640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f89440b5c70 0x7f89440d2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:33.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.927+0000 7f8949088640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8944162610 con 0x7f8944161fe0 2024-04-25T08:54:33.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.927+0000 7f8949088640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89440d1da0 con 0x7f894415e090 2024-04-25T08:54:33.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.927+0000 7f8949088640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89440d1750 con 0x7f89440b5c70 2024-04-25T08:54:33.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.927+0000 7f8942d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f894415e090 0x7f8944161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:33.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.927+0000 7f893bfff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8944161fe0 0x7f89440b5560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:55658/0 (socket says 172.21.15.43:55658) 2024-04-25T08:54:33.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.927+0000 7f893bfff640 1 -- 172.21.15.43:0/3662735312 learned_addr learned my addr 172.21.15.43:0/3662735312 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:33.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.927+0000 7f894359e640 1 --2- 172.21.15.43:0/3662735312 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f89440b5c70 0x7f89440d2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:33.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.927+0000 7f8942d9d640 1 -- 172.21.15.43:0/3662735312 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f89440b5c70 msgr2=0x7f89440d2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:33.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.927+0000 7f8942d9d640 1 --2- 172.21.15.43:0/3662735312 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f89440b5c70 0x7f89440d2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.927+0000 7f8942d9d640 1 -- 172.21.15.43:0/3662735312 >> v1:172.21.15.43:6789/0 conn(0x7f8944161fe0 legacy=0x7f89440b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:33.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.927+0000 7f894259c640 1 -- 172.21.15.43:0/3662735312 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3146515314 0 0) 0x7f8944162610 con 0x7f8944161fe0 2024-04-25T08:54:33.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.927+0000 7f8942d9d640 1 -- 172.21.15.43:0/3662735312 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89440d1a30 con 0x7f894415e090 2024-04-25T08:54:33.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.928+0000 7f8942d9d640 1 --2- 172.21.15.43:0/3662735312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f894415e090 0x7f8944161aa0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f89340097a0 tx=0x7f893402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191ecf5306ffc269 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:33.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.928+0000 7f894259c640 1 -- 172.21.15.43:0/3662735312 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8934037070 con 0x7f894415e090 2024-04-25T08:54:33.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.928+0000 7f894259c640 1 -- 172.21.15.43:0/3662735312 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8934032440 con 0x7f894415e090 2024-04-25T08:54:33.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.928+0000 7f894259c640 1 -- 172.21.15.43:0/3662735312 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8934032810 con 0x7f894415e090 2024-04-25T08:54:33.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.928+0000 7f8949088640 1 -- 172.21.15.43:0/3662735312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f894415e090 msgr2=0x7f8944161aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:33.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.928+0000 7f8949088640 1 --2- 172.21.15.43:0/3662735312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f894415e090 0x7f8944161aa0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f89340097a0 tx=0x7f893402a210 comp rx=0 tx=0).stop 2024-04-25T08:54:33.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.929+0000 7f8949088640 1 -- 172.21.15.43:0/3662735312 shutdown_connections 2024-04-25T08:54:33.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.929+0000 7f8949088640 1 --2- 172.21.15.43:0/3662735312 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f89440b5c70 0x7f89440d2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.929+0000 7f8949088640 1 --2- 172.21.15.43:0/3662735312 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f894415e090 0x7f8944161aa0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.929+0000 7f8949088640 1 -- 172.21.15.43:0/3662735312 >> 172.21.15.43:0/3662735312 conn(0x7f89440d0560 msgr2=0x7f89440d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:33.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.929+0000 7f8949088640 1 -- 172.21.15.43:0/3662735312 shutdown_connections 2024-04-25T08:54:33.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.929+0000 7f8949088640 1 -- 172.21.15.43:0/3662735312 wait complete. 2024-04-25T08:54:33.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.930+0000 7f8949088640 1 Processor -- start 2024-04-25T08:54:33.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.930+0000 7f8949088640 1 -- start start 2024-04-25T08:54:33.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.931+0000 7f8949088640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89440b5c70 0x7f8944142cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:33.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.931+0000 7f8949088640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f894415e090 0x7f89441f2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:33.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.931+0000 7f8949088640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8944161fe0 0x7f8944157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:33.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.931+0000 7f8949088640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89440d2770 con 0x7f89440b5c70 2024-04-25T08:54:33.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.931+0000 7f8949088640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89440b3280 con 0x7f894415e090 2024-04-25T08:54:33.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.931+0000 7f8949088640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89440d28f0 con 0x7f8944161fe0 2024-04-25T08:54:33.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.931+0000 7f8942d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89440b5c70 0x7f8944142cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:33.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.931+0000 7f8942d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89440b5c70 0x7f8944142cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40036/0 (socket says 172.21.15.43:40036) 2024-04-25T08:54:33.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.931+0000 7f8942d9d640 1 -- 172.21.15.43:0/59431426 learned_addr learned my addr 172.21.15.43:0/59431426 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:33.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.931+0000 7f893bfff640 1 --2- 172.21.15.43:0/59431426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f894415e090 0x7f89441f2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:33.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.931+0000 7f894359e640 1 --2- 172.21.15.43:0/59431426 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8944161fe0 0x7f8944157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:33.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.931+0000 7f8942d9d640 1 -- 172.21.15.43:0/59431426 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8944161fe0 msgr2=0x7f8944157580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:33.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.931+0000 7f8942d9d640 1 --2- 172.21.15.43:0/59431426 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8944161fe0 0x7f8944157580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.931+0000 7f8942d9d640 1 -- 172.21.15.43:0/59431426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f894415e090 msgr2=0x7f89441f2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:33.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.931+0000 7f8942d9d640 1 --2- 172.21.15.43:0/59431426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f894415e090 0x7f89441f2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:33.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.931+0000 7f8942d9d640 1 -- 172.21.15.43:0/59431426 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8944157ac0 con 0x7f89440b5c70 2024-04-25T08:54:33.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.931+0000 7f8942d9d640 1 --2- 172.21.15.43:0/59431426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89440b5c70 0x7f8944142cb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f89340097a0 tx=0x7f89340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:33.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.932+0000 7f893b7fe640 1 -- 172.21.15.43:0/59431426 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8934037050 con 0x7f89440b5c70 2024-04-25T08:54:33.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.932+0000 7f893b7fe640 1 -- 172.21.15.43:0/59431426 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8934038040 con 0x7f89440b5c70 2024-04-25T08:54:33.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.932+0000 7f8949088640 1 -- 172.21.15.43:0/59431426 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8944160df0 con 0x7f89440b5c70 2024-04-25T08:54:33.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.932+0000 7f893b7fe640 1 -- 172.21.15.43:0/59431426 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8934004470 con 0x7f89440b5c70 2024-04-25T08:54:33.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.932+0000 7f8949088640 1 -- 172.21.15.43:0/59431426 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8944159220 con 0x7f89440b5c70 2024-04-25T08:54:33.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.933+0000 7f8949088640 1 -- 172.21.15.43:0/59431426 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f894406bf00 con 0x7f89440b5c70 2024-04-25T08:54:33.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.933+0000 7f893b7fe640 1 -- 172.21.15.43:0/59431426 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8934030900 con 0x7f89440b5c70 2024-04-25T08:54:33.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.934+0000 7f893b7fe640 1 --2- 172.21.15.43:0/59431426 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f892406b4d0 0x7f892406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:33.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.934+0000 7f893b7fe640 1 -- 172.21.15.43:0/59431426 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f89340aad20 con 0x7f89440b5c70 2024-04-25T08:54:33.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.934+0000 7f893bfff640 1 --2- 172.21.15.43:0/59431426 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f892406b4d0 0x7f892406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:33.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.934+0000 7f893bfff640 1 --2- 172.21.15.43:0/59431426 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f892406b4d0 0x7f892406d990 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8944148d00 tx=0x7f892c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:33.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:33.938+0000 7f893b7fe640 1 -- 172.21.15.43:0/59431426 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f894406bf00 con 0x7f89440b5c70 2024-04-25T08:54:34.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.121+0000 7f8949088640 1 -- 172.21.15.43:0/59431426 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8944074f60 con 0x7f89440b5c70 2024-04-25T08:54:34.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.121+0000 7f893b7fe640 1 -- 172.21.15.43:0/59431426 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8944157ac0 con 0x7f89440b5c70 2024-04-25T08:54:34.112 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T08:54:34.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.123+0000 7f8949088640 1 -- 172.21.15.43:0/59431426 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f892406b4d0 msgr2=0x7f892406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:34.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.123+0000 7f8949088640 1 --2- 172.21.15.43:0/59431426 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f892406b4d0 0x7f892406d990 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8944148d00 tx=0x7f892c009040 comp rx=0 tx=0).stop 2024-04-25T08:54:34.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.123+0000 7f8949088640 1 -- 172.21.15.43:0/59431426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89440b5c70 msgr2=0x7f8944142cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:34.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.123+0000 7f8949088640 1 --2- 172.21.15.43:0/59431426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89440b5c70 0x7f8944142cb0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f89340097a0 tx=0x7f89340046c0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.123+0000 7f8949088640 1 -- 172.21.15.43:0/59431426 shutdown_connections 2024-04-25T08:54:34.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.123+0000 7f8949088640 1 --2- 172.21.15.43:0/59431426 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8944161fe0 0x7f8944157580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.123+0000 7f8949088640 1 --2- 172.21.15.43:0/59431426 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f892406b4d0 0x7f892406d990 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.123+0000 7f8949088640 1 --2- 172.21.15.43:0/59431426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f894415e090 0x7f89441f2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.123+0000 7f8949088640 1 --2- 172.21.15.43:0/59431426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89440b5c70 0x7f8944142cb0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.123+0000 7f8949088640 1 -- 172.21.15.43:0/59431426 >> 172.21.15.43:0/59431426 conn(0x7f89440d0560 msgr2=0x7f89440c6e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:34.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.123+0000 7f8949088640 1 -- 172.21.15.43:0/59431426 shutdown_connections 2024-04-25T08:54:34.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.123+0000 7f8949088640 1 -- 172.21.15.43:0/59431426 wait complete. 2024-04-25T08:54:34.125 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:54:34.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.236+0000 7fefba0e4640 1 Processor -- start 2024-04-25T08:54:34.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.236+0000 7fefba0e4640 1 -- start start 2024-04-25T08:54:34.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.236+0000 7fefba0e4640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fefb415dce0 0x7fefb415ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:34.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.236+0000 7fefba0e4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fefb40ce550 0x7fefb40d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:34.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.236+0000 7fefba0e4640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefb40cee80 con 0x7fefb40ce990 2024-04-25T08:54:34.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.236+0000 7fefba0e4640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefb40d2920 con 0x7fefb40ce550 2024-04-25T08:54:34.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.236+0000 7fefba0e4640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb40cf160 con 0x7fefb415dce0 2024-04-25T08:54:34.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.237+0000 7fefb2ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fefb40ce550 0x7fefb40d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:34.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.237+0000 7fefb2ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fefb40ce550 0x7fefb40d03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40040/0 (socket says 172.21.15.43:40040) 2024-04-25T08:54:34.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.237+0000 7fefb2ffd640 1 -- 172.21.15.43:0/4069686433 learned_addr learned my addr 172.21.15.43:0/4069686433 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:34.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.237+0000 7fefb37fe640 1 --2- 172.21.15.43:0/4069686433 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fefb415dce0 0x7fefb415ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:34.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.237+0000 7fefb2ffd640 1 -- 172.21.15.43:0/4069686433 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fefb415dce0 msgr2=0x7fefb415ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:34.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.237+0000 7fefb2ffd640 1 --2- 172.21.15.43:0/4069686433 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fefb415dce0 0x7fefb415ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.237+0000 7fefb2ffd640 1 -- 172.21.15.43:0/4069686433 >> v1:172.21.15.193:6789/0 conn(0x7fefb40ce990 legacy=0x7fefb4160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:34.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.237+0000 7fefb2ffd640 1 -- 172.21.15.43:0/4069686433 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb40cf440 con 0x7fefb40ce550 2024-04-25T08:54:34.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.238+0000 7fefb2ffd640 1 --2- 172.21.15.43:0/4069686433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefb40ce550 0x7fefb40d03e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fefa4009870 tx=0x7fefa402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e38c00b46adc67a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:34.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.238+0000 7fefb27fc640 1 -- 172.21.15.43:0/4069686433 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa4037070 con 0x7fefb40ce550 2024-04-25T08:54:34.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.238+0000 7fefb27fc640 1 -- 172.21.15.43:0/4069686433 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa4032440 con 0x7fefb40ce550 2024-04-25T08:54:34.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.238+0000 7fefb27fc640 1 -- 172.21.15.43:0/4069686433 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa4032810 con 0x7fefb40ce550 2024-04-25T08:54:34.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.238+0000 7fefba0e4640 1 -- 172.21.15.43:0/4069686433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefb40ce550 msgr2=0x7fefb40d03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:34.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.238+0000 7fefba0e4640 1 --2- 172.21.15.43:0/4069686433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefb40ce550 0x7fefb40d03e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fefa4009870 tx=0x7fefa402a1c0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.238+0000 7fefba0e4640 1 -- 172.21.15.43:0/4069686433 shutdown_connections 2024-04-25T08:54:34.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.238+0000 7fefba0e4640 1 --2- 172.21.15.43:0/4069686433 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefb40ce550 0x7fefb40d03e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.238+0000 7fefba0e4640 1 --2- 172.21.15.43:0/4069686433 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fefb415dce0 0x7fefb415ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.238+0000 7fefba0e4640 1 -- 172.21.15.43:0/4069686433 >> 172.21.15.43:0/4069686433 conn(0x7fefb409cd00 msgr2=0x7fefb40d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:34.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.238+0000 7fefba0e4640 1 -- 172.21.15.43:0/4069686433 shutdown_connections 2024-04-25T08:54:34.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.239+0000 7fefba0e4640 1 -- 172.21.15.43:0/4069686433 wait complete. 2024-04-25T08:54:34.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.240+0000 7fefba0e4640 1 Processor -- start 2024-04-25T08:54:34.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.240+0000 7fefba0e4640 1 -- start start 2024-04-25T08:54:34.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.240+0000 7fefba0e4640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fefb40ce550 0x7fefb4149750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:34.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.241+0000 7fefba0e4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefb40ce990 0x7fefb414bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:34.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.241+0000 7fefba0e4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefb415dce0 0x7fefb414c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:34.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.241+0000 7fefba0e4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb415e0c0 con 0x7fefb415dce0 2024-04-25T08:54:34.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.241+0000 7fefba0e4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb415f6b0 con 0x7fefb40ce990 2024-04-25T08:54:34.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.241+0000 7fefba0e4640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb40d0920 con 0x7fefb40ce550 2024-04-25T08:54:34.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.241+0000 7fefb3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefb415dce0 0x7fefb414c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:34.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.241+0000 7fefb3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefb415dce0 0x7fefb414c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40042/0 (socket says 172.21.15.43:40042) 2024-04-25T08:54:34.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.241+0000 7fefb3fff640 1 -- 172.21.15.43:0/2995477540 learned_addr learned my addr 172.21.15.43:0/2995477540 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:34.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.241+0000 7fefb2ffd640 1 --2- 172.21.15.43:0/2995477540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefb40ce990 0x7fefb414bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:34.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.241+0000 7fefb37fe640 1 --2- 172.21.15.43:0/2995477540 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fefb40ce550 0x7fefb4149750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:34.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.241+0000 7fefb3fff640 1 -- 172.21.15.43:0/2995477540 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fefb40ce550 msgr2=0x7fefb4149750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:34.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.241+0000 7fefb3fff640 1 --2- 172.21.15.43:0/2995477540 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fefb40ce550 0x7fefb4149750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.241+0000 7fefb3fff640 1 -- 172.21.15.43:0/2995477540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefb40ce990 msgr2=0x7fefb414bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:34.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.241+0000 7fefb3fff640 1 --2- 172.21.15.43:0/2995477540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefb40ce990 0x7fefb414bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.241+0000 7fefb3fff640 1 -- 172.21.15.43:0/2995477540 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb41f3fb0 con 0x7fefb415dce0 2024-04-25T08:54:34.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.241+0000 7fefb3fff640 1 --2- 172.21.15.43:0/2995477540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefb415dce0 0x7fefb414c3e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fefac00c7a0 tx=0x7fefac00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:34.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.241+0000 7fef9bfff640 1 -- 172.21.15.43:0/2995477540 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefac003b70 con 0x7fefb415dce0 2024-04-25T08:54:34.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.241+0000 7fef9bfff640 1 -- 172.21.15.43:0/2995477540 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefac003d30 con 0x7fefb415dce0 2024-04-25T08:54:34.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.241+0000 7fef9bfff640 1 -- 172.21.15.43:0/2995477540 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefac00aa90 con 0x7fefb415dce0 2024-04-25T08:54:34.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.241+0000 7fefba0e4640 1 -- 172.21.15.43:0/2995477540 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefb414a060 con 0x7fefb415dce0 2024-04-25T08:54:34.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.241+0000 7fefba0e4640 1 -- 172.21.15.43:0/2995477540 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefb414a620 con 0x7fefb415dce0 2024-04-25T08:54:34.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.242+0000 7fefba0e4640 1 -- 172.21.15.43:0/2995477540 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefb40680f0 con 0x7fefb415dce0 2024-04-25T08:54:34.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.243+0000 7fef9bfff640 1 -- 172.21.15.43:0/2995477540 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fefac00ac30 con 0x7fefb415dce0 2024-04-25T08:54:34.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.243+0000 7fef9bfff640 1 --2- 172.21.15.43:0/2995477540 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fefa006b4d0 0x7fefa006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:34.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.243+0000 7fef9bfff640 1 -- 172.21.15.43:0/2995477540 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fefac08a010 con 0x7fefb415dce0 2024-04-25T08:54:34.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.243+0000 7fefb37fe640 1 --2- 172.21.15.43:0/2995477540 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fefa006b4d0 0x7fefa006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:34.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.244+0000 7fefb37fe640 1 --2- 172.21.15.43:0/2995477540 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fefa006b4d0 0x7fefa006d990 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fefa8009aa0 tx=0x7fefa8009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:34.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.247+0000 7fef9bfff640 1 -- 172.21.15.43:0/2995477540 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fefac058830 con 0x7fefb415dce0 2024-04-25T08:54:34.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.430+0000 7fefba0e4640 1 -- 172.21.15.43:0/2995477540 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fefb4074ef0 con 0x7fefb415dce0 2024-04-25T08:54:34.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.432+0000 7fef9bfff640 1 -- 172.21.15.43:0/2995477540 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16569 (secure 0 0 0) 0x7fefac05c8d0 con 0x7fefb415dce0 2024-04-25T08:54:34.422 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:54:34.422 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":20,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:54:13.593188+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:54:09.757978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:54:10.926008+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:54:34.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.434+0000 7fefba0e4640 1 -- 172.21.15.43:0/2995477540 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fefa006b4d0 msgr2=0x7fefa006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:34.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.434+0000 7fefba0e4640 1 --2- 172.21.15.43:0/2995477540 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fefa006b4d0 0x7fefa006d990 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fefa8009aa0 tx=0x7fefa8009190 comp rx=0 tx=0).stop 2024-04-25T08:54:34.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.434+0000 7fefba0e4640 1 -- 172.21.15.43:0/2995477540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefb415dce0 msgr2=0x7fefb414c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:34.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.434+0000 7fefba0e4640 1 --2- 172.21.15.43:0/2995477540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefb415dce0 0x7fefb414c3e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fefac00c7a0 tx=0x7fefac00cc60 comp rx=0 tx=0).stop 2024-04-25T08:54:34.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.434+0000 7fefba0e4640 1 -- 172.21.15.43:0/2995477540 shutdown_connections 2024-04-25T08:54:34.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.434+0000 7fefba0e4640 1 --2- 172.21.15.43:0/2995477540 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefb415dce0 0x7fefb414c3e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.434+0000 7fefba0e4640 1 --2- 172.21.15.43:0/2995477540 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fefa006b4d0 0x7fefa006d990 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.434+0000 7fefba0e4640 1 --2- 172.21.15.43:0/2995477540 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefb40ce990 0x7fefb414bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.434+0000 7fefba0e4640 1 --2- 172.21.15.43:0/2995477540 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fefb40ce550 0x7fefb4149750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.434+0000 7fefba0e4640 1 -- 172.21.15.43:0/2995477540 >> 172.21.15.43:0/2995477540 conn(0x7fefb409cd00 msgr2=0x7fefb406c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:34.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.434+0000 7fefba0e4640 1 -- 172.21.15.43:0/2995477540 shutdown_connections 2024-04-25T08:54:34.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.434+0000 7fefba0e4640 1 -- 172.21.15.43:0/2995477540 wait complete. 2024-04-25T08:54:34.435 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:54:09.757978+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:54:10.926008+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:54:34.435 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:54:34.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.549+0000 7fb1085e7640 1 Processor -- start 2024-04-25T08:54:34.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.549+0000 7fb1085e7640 1 -- start start 2024-04-25T08:54:34.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.550+0000 7fb1085e7640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb10006ed70 0x7fb10006f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:34.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.550+0000 7fb1085e7640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb10015bf00 con 0x7fb1000cbda0 2024-04-25T08:54:34.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.550+0000 7fb1085e7640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb10006f690 con 0x7fb1000d1090 2024-04-25T08:54:34.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.550+0000 7fb1085e7640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1000cfdb0 con 0x7fb10006ed70 2024-04-25T08:54:34.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.550+0000 7fb105b82640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb1000cbda0 0x7fb1000d0980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:55670/0 (socket says 172.21.15.43:55670) 2024-04-25T08:54:34.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.550+0000 7fb105b82640 1 -- 172.21.15.43:0/1785400846 learned_addr learned my addr 172.21.15.43:0/1785400846 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:34.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.550+0000 7fb106383640 1 --2- 172.21.15.43:0/1785400846 >> v2:172.21.15.193:3300/0 conn(0x7fb10006ed70 0x7fb10006f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:34.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.550+0000 7fb105381640 1 -- 172.21.15.43:0/1785400846 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 111261036 0 0) 0x7fb10015bf00 con 0x7fb1000cbda0 2024-04-25T08:54:34.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.550+0000 7fb105381640 1 -- 172.21.15.43:0/1785400846 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0e8003520 con 0x7fb1000cbda0 2024-04-25T08:54:34.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.551+0000 7fb106383640 1 -- 172.21.15.43:0/1785400846 >> v1:172.21.15.193:6789/0 conn(0x7fb1000d1090 legacy=0x7fb1000c77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:34.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.551+0000 7fb106383640 1 -- 172.21.15.43:0/1785400846 >> v1:172.21.15.43:6789/0 conn(0x7fb1000cbda0 legacy=0x7fb1000d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:34.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.551+0000 7fb106383640 1 -- 172.21.15.43:0/1785400846 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1000c6740 con 0x7fb10006ed70 2024-04-25T08:54:34.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.551+0000 7fb106383640 1 --2- 172.21.15.43:0/1785400846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb10006ed70 0x7fb10006f150 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0009d90 tx=0x7fb0f002a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1462ff948d68bef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:34.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.551+0000 7fb105381640 1 -- 172.21.15.43:0/1785400846 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f0007740 con 0x7fb10006ed70 2024-04-25T08:54:34.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.552+0000 7fb105381640 1 -- 172.21.15.43:0/1785400846 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0f00078e0 con 0x7fb10006ed70 2024-04-25T08:54:34.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.552+0000 7fb105381640 1 -- 172.21.15.43:0/1785400846 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f0007cb0 con 0x7fb10006ed70 2024-04-25T08:54:34.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.552+0000 7fb1085e7640 1 -- 172.21.15.43:0/1785400846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb10006ed70 msgr2=0x7fb10006f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:34.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.552+0000 7fb1085e7640 1 --2- 172.21.15.43:0/1785400846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb10006ed70 0x7fb10006f150 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0009d90 tx=0x7fb0f002a6f0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.552+0000 7fb1085e7640 1 -- 172.21.15.43:0/1785400846 shutdown_connections 2024-04-25T08:54:34.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.552+0000 7fb1085e7640 1 --2- 172.21.15.43:0/1785400846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb10006ed70 0x7fb10006f150 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.552+0000 7fb1085e7640 1 -- 172.21.15.43:0/1785400846 >> 172.21.15.43:0/1785400846 conn(0x7fb1000d17e0 msgr2=0x7fb1000d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:34.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.552+0000 7fb1085e7640 1 -- 172.21.15.43:0/1785400846 shutdown_connections 2024-04-25T08:54:34.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.552+0000 7fb1085e7640 1 -- 172.21.15.43:0/1785400846 wait complete. 2024-04-25T08:54:34.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.553+0000 7fb1085e7640 1 Processor -- start 2024-04-25T08:54:34.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.553+0000 7fb1085e7640 1 -- start start 2024-04-25T08:54:34.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.554+0000 7fb1085e7640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb10006ed70 0x7fb10009b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:34.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.554+0000 7fb1085e7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1000cbda0 0x7fb1001f4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:34.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.554+0000 7fb1085e7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1000d1090 0x7fb1001f1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:34.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.554+0000 7fb1085e7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1000c6fe0 con 0x7fb1000cbda0 2024-04-25T08:54:34.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.554+0000 7fb1085e7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1000c6d00 con 0x7fb1000d1090 2024-04-25T08:54:34.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.554+0000 7fb1085e7640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1000c72c0 con 0x7fb10006ed70 2024-04-25T08:54:34.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.554+0000 7fb105b82640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1000cbda0 0x7fb1001f4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:34.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.554+0000 7fb105b82640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1000cbda0 0x7fb1001f4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40046/0 (socket says 172.21.15.43:40046) 2024-04-25T08:54:34.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.554+0000 7fb105b82640 1 -- 172.21.15.43:0/381084630 learned_addr learned my addr 172.21.15.43:0/381084630 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:34.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.554+0000 7fb106383640 1 --2- 172.21.15.43:0/381084630 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb10006ed70 0x7fb10009b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:34.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.554+0000 7fb106b84640 1 --2- 172.21.15.43:0/381084630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1000d1090 0x7fb1001f1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:34.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.554+0000 7fb105b82640 1 -- 172.21.15.43:0/381084630 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb10006ed70 msgr2=0x7fb10009b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:34.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.554+0000 7fb105b82640 1 --2- 172.21.15.43:0/381084630 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb10006ed70 0x7fb10009b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.554+0000 7fb105b82640 1 -- 172.21.15.43:0/381084630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1000d1090 msgr2=0x7fb1001f1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:34.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.554+0000 7fb105b82640 1 --2- 172.21.15.43:0/381084630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1000d1090 0x7fb1001f1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.554+0000 7fb105b82640 1 -- 172.21.15.43:0/381084630 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1000c9230 con 0x7fb1000cbda0 2024-04-25T08:54:34.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.555+0000 7fb106b84640 1 --2- 172.21.15.43:0/381084630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1000d1090 0x7fb1001f1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:34.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.555+0000 7fb105b82640 1 --2- 172.21.15.43:0/381084630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1000cbda0 0x7fb1001f4fe0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb0f400c440 tx=0x7fb0f400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:34.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.555+0000 7fb0feffd640 1 -- 172.21.15.43:0/381084630 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f4017070 con 0x7fb1000cbda0 2024-04-25T08:54:34.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.555+0000 7fb0feffd640 1 -- 172.21.15.43:0/381084630 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0f4005ce0 con 0x7fb1000cbda0 2024-04-25T08:54:34.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.555+0000 7fb1085e7640 1 -- 172.21.15.43:0/381084630 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1000c9510 con 0x7fb1000cbda0 2024-04-25T08:54:34.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.555+0000 7fb0feffd640 1 -- 172.21.15.43:0/381084630 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f40072b0 con 0x7fb1000cbda0 2024-04-25T08:54:34.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.555+0000 7fb1085e7640 1 -- 172.21.15.43:0/381084630 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1000c9db0 con 0x7fb1000cbda0 2024-04-25T08:54:34.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.556+0000 7fb1085e7640 1 -- 172.21.15.43:0/381084630 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1000c9900 con 0x7fb1000cbda0 2024-04-25T08:54:34.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.556+0000 7fb0feffd640 1 -- 172.21.15.43:0/381084630 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb0f401a070 con 0x7fb1000cbda0 2024-04-25T08:54:34.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.557+0000 7fb0feffd640 1 --2- 172.21.15.43:0/381084630 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb0ec06b4d0 0x7fb0ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:34.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.557+0000 7fb0feffd640 1 -- 172.21.15.43:0/381084630 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb0f40894a0 con 0x7fb1000cbda0 2024-04-25T08:54:34.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.557+0000 7fb106383640 1 --2- 172.21.15.43:0/381084630 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb0ec06b4d0 0x7fb0ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:34.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.557+0000 7fb106383640 1 --2- 172.21.15.43:0/381084630 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb0ec06b4d0 0x7fb0ec06d990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0034000 tx=0x7fb0f002f000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:34.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.561+0000 7fb0feffd640 1 -- 172.21.15.43:0/381084630 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb0f4057d90 con 0x7fb1000cbda0 2024-04-25T08:54:34.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.740+0000 7fb1085e7640 1 -- 172.21.15.43:0/381084630 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb1000cd9c0 con 0x7fb1000cbda0 2024-04-25T08:54:34.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.740+0000 7fb0feffd640 1 -- 172.21.15.43:0/381084630 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb1000cd9c0 con 0x7fb1000cbda0 2024-04-25T08:54:34.730 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T08:54:34.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.742+0000 7fb1085e7640 1 -- 172.21.15.43:0/381084630 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb0ec06b4d0 msgr2=0x7fb0ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:34.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.742+0000 7fb1085e7640 1 --2- 172.21.15.43:0/381084630 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb0ec06b4d0 0x7fb0ec06d990 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0034000 tx=0x7fb0f002f000 comp rx=0 tx=0).stop 2024-04-25T08:54:34.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.742+0000 7fb1085e7640 1 -- 172.21.15.43:0/381084630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1000cbda0 msgr2=0x7fb1001f4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:34.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.742+0000 7fb1085e7640 1 --2- 172.21.15.43:0/381084630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1000cbda0 0x7fb1001f4fe0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb0f400c440 tx=0x7fb0f400c900 comp rx=0 tx=0).stop 2024-04-25T08:54:34.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.742+0000 7fb1085e7640 1 -- 172.21.15.43:0/381084630 shutdown_connections 2024-04-25T08:54:34.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.742+0000 7fb1085e7640 1 --2- 172.21.15.43:0/381084630 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb0ec06b4d0 0x7fb0ec06d990 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.742+0000 7fb1085e7640 1 --2- 172.21.15.43:0/381084630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb1000d1090 0x7fb1001f1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.742+0000 7fb1085e7640 1 --2- 172.21.15.43:0/381084630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1000cbda0 0x7fb1001f4fe0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.742+0000 7fb1085e7640 1 --2- 172.21.15.43:0/381084630 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb10006ed70 0x7fb10009b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.742+0000 7fb1085e7640 1 -- 172.21.15.43:0/381084630 >> 172.21.15.43:0/381084630 conn(0x7fb1000d17e0 msgr2=0x7fb1000c76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:34.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.742+0000 7fb1085e7640 1 -- 172.21.15.43:0/381084630 shutdown_connections 2024-04-25T08:54:34.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.742+0000 7fb1085e7640 1 -- 172.21.15.43:0/381084630 wait complete. 2024-04-25T08:54:34.743 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:54:34.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.857+0000 7fa201438640 1 Processor -- start 2024-04-25T08:54:34.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.857+0000 7fa201438640 1 -- start start 2024-04-25T08:54:34.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.858+0000 7fa201438640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa1fc15e090 0x7fa1fc161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:34.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.858+0000 7fa201438640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa1fc161fe0 0x7fa1fc0b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:34.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.858+0000 7fa201438640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1fc162610 con 0x7fa1fc0b5aa0 2024-04-25T08:54:34.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.858+0000 7fa201438640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1fc0b5f00 con 0x7fa1fc161fe0 2024-04-25T08:54:34.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.858+0000 7fa201438640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc0d1da0 con 0x7fa1fc15e090 2024-04-25T08:54:34.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.858+0000 7fa1fa7fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa1fc161fe0 0x7fa1fc0b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:34.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.858+0000 7fa1fb7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa1fc0b5aa0 0x7fa1fc0ce560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:55680/0 (socket says 172.21.15.43:55680) 2024-04-25T08:54:34.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.858+0000 7fa1fb7fe640 1 -- 172.21.15.43:0/2462957517 learned_addr learned my addr 172.21.15.43:0/2462957517 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:34.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.858+0000 7fa1fa7fc640 1 -- 172.21.15.43:0/2462957517 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa1fc15e090 msgr2=0x7fa1fc161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:34.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.858+0000 7fa1faffd640 1 --2- 172.21.15.43:0/2462957517 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa1fc15e090 0x7fa1fc161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:34.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.858+0000 7fa1fa7fc640 1 --2- 172.21.15.43:0/2462957517 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa1fc15e090 0x7fa1fc161aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.858+0000 7fa1fa7fc640 1 -- 172.21.15.43:0/2462957517 >> v1:172.21.15.43:6789/0 conn(0x7fa1fc0b5aa0 legacy=0x7fa1fc0ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:34.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.858+0000 7fa1fa7fc640 1 -- 172.21.15.43:0/2462957517 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1fc0d1750 con 0x7fa1fc161fe0 2024-04-25T08:54:34.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.858+0000 7fa1f9ffb640 1 -- 172.21.15.43:0/2462957517 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3197687314 0 0) 0x7fa1fc162610 con 0x7fa1fc0b5aa0 2024-04-25T08:54:34.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.859+0000 7fa1fa7fc640 1 --2- 172.21.15.43:0/2462957517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1fc161fe0 0x7fa1fc0b5560 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00097a0 tx=0x7fa1f002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f42f6d295a95971b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:34.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.859+0000 7fa1f9ffb640 1 -- 172.21.15.43:0/2462957517 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0037070 con 0x7fa1fc161fe0 2024-04-25T08:54:34.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.859+0000 7fa1f9ffb640 1 -- 172.21.15.43:0/2462957517 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f0032440 con 0x7fa1fc161fe0 2024-04-25T08:54:34.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.859+0000 7fa1f9ffb640 1 -- 172.21.15.43:0/2462957517 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0032810 con 0x7fa1fc161fe0 2024-04-25T08:54:34.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.860+0000 7fa201438640 1 -- 172.21.15.43:0/2462957517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1fc161fe0 msgr2=0x7fa1fc0b5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:34.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.860+0000 7fa201438640 1 --2- 172.21.15.43:0/2462957517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1fc161fe0 0x7fa1fc0b5560 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00097a0 tx=0x7fa1f002a210 comp rx=0 tx=0).stop 2024-04-25T08:54:34.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.860+0000 7fa201438640 1 -- 172.21.15.43:0/2462957517 shutdown_connections 2024-04-25T08:54:34.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.860+0000 7fa201438640 1 --2- 172.21.15.43:0/2462957517 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1fc161fe0 0x7fa1fc0b5560 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.860+0000 7fa201438640 1 --2- 172.21.15.43:0/2462957517 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa1fc15e090 0x7fa1fc161aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.860+0000 7fa201438640 1 -- 172.21.15.43:0/2462957517 >> 172.21.15.43:0/2462957517 conn(0x7fa1fc0d0560 msgr2=0x7fa1fc0d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:34.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.860+0000 7fa201438640 1 -- 172.21.15.43:0/2462957517 shutdown_connections 2024-04-25T08:54:34.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.860+0000 7fa201438640 1 -- 172.21.15.43:0/2462957517 wait complete. 2024-04-25T08:54:34.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.861+0000 7fa201438640 1 Processor -- start 2024-04-25T08:54:34.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.861+0000 7fa201438640 1 -- start start 2024-04-25T08:54:34.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.862+0000 7fa201438640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1fc0b5aa0 0x7fa1fc15bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:34.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.862+0000 7fa201438640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1fc15e090 0x7fa1fc143d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:34.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.862+0000 7fa201438640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa1fc161fe0 0x7fa1fc144290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:34.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.862+0000 7fa201438640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc0b3280 con 0x7fa1fc15e090 2024-04-25T08:54:34.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.862+0000 7fa201438640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc1628f0 con 0x7fa1fc0b5aa0 2024-04-25T08:54:34.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.862+0000 7fa201438640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1fc0d3240 con 0x7fa1fc161fe0 2024-04-25T08:54:34.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.862+0000 7fa1fa7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1fc15e090 0x7fa1fc143d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:34.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.862+0000 7fa1fa7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1fc15e090 0x7fa1fc143d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40062/0 (socket says 172.21.15.43:40062) 2024-04-25T08:54:34.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.862+0000 7fa1fa7fc640 1 -- 172.21.15.43:0/1399798767 learned_addr learned my addr 172.21.15.43:0/1399798767 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:34.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.862+0000 7fa1fa7fc640 1 -- 172.21.15.43:0/1399798767 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa1fc161fe0 msgr2=0x7fa1fc144290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:34.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.862+0000 7fa1fa7fc640 1 --2- 172.21.15.43:0/1399798767 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa1fc161fe0 0x7fa1fc144290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.862+0000 7fa1fa7fc640 1 -- 172.21.15.43:0/1399798767 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1fc0b5aa0 msgr2=0x7fa1fc15bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:34.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.862+0000 7fa1fa7fc640 1 --2- 172.21.15.43:0/1399798767 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1fc0b5aa0 0x7fa1fc15bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:34.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.862+0000 7fa1fa7fc640 1 -- 172.21.15.43:0/1399798767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1fc15c180 con 0x7fa1fc15e090 2024-04-25T08:54:34.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.862+0000 7fa1fa7fc640 1 --2- 172.21.15.43:0/1399798767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1fc15e090 0x7fa1fc143d50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00098c0 tx=0x7fa1f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:34.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.863+0000 7fa1e37fe640 1 -- 172.21.15.43:0/1399798767 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0037040 con 0x7fa1fc15e090 2024-04-25T08:54:34.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.863+0000 7fa1e37fe640 1 -- 172.21.15.43:0/1399798767 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f0030770 con 0x7fa1fc15e090 2024-04-25T08:54:34.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.863+0000 7fa201438640 1 -- 172.21.15.43:0/1399798767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1fc15aba0 con 0x7fa1fc15e090 2024-04-25T08:54:34.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.863+0000 7fa1e37fe640 1 -- 172.21.15.43:0/1399798767 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0030b40 con 0x7fa1fc15e090 2024-04-25T08:54:34.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.863+0000 7fa201438640 1 -- 172.21.15.43:0/1399798767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1fc15b160 con 0x7fa1fc15e090 2024-04-25T08:54:34.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.864+0000 7fa201438640 1 -- 172.21.15.43:0/1399798767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1fc06bf00 con 0x7fa1fc15e090 2024-04-25T08:54:34.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.864+0000 7fa1e37fe640 1 -- 172.21.15.43:0/1399798767 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa1f0044020 con 0x7fa1fc15e090 2024-04-25T08:54:34.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.865+0000 7fa1e37fe640 1 --2- 172.21.15.43:0/1399798767 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa1e806b4d0 0x7fa1e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:34.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.865+0000 7fa1e37fe640 1 -- 172.21.15.43:0/1399798767 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1f00aa7f0 con 0x7fa1fc15e090 2024-04-25T08:54:34.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.865+0000 7fa1faffd640 1 --2- 172.21.15.43:0/1399798767 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa1e806b4d0 0x7fa1e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:34.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.865+0000 7fa1faffd640 1 --2- 172.21.15.43:0/1399798767 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa1e806b4d0 0x7fa1e806d990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4006fd0 tx=0x7fa1f4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:34.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:34.869+0000 7fa1e37fe640 1 -- 172.21.15.43:0/1399798767 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa1fc06bf00 con 0x7fa1fc15e090 2024-04-25T08:54:35.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.047+0000 7fa201438640 1 -- 172.21.15.43:0/1399798767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa1fc074f60 con 0x7fa1fc15e090 2024-04-25T08:54:35.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.048+0000 7fa1e37fe640 1 -- 172.21.15.43:0/1399798767 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa1f0032e20 con 0x7fa1fc15e090 2024-04-25T08:54:35.038 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T08:54:35.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.049+0000 7fa201438640 1 -- 172.21.15.43:0/1399798767 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa1e806b4d0 msgr2=0x7fa1e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:35.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.049+0000 7fa201438640 1 --2- 172.21.15.43:0/1399798767 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa1e806b4d0 0x7fa1e806d990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4006fd0 tx=0x7fa1f4008040 comp rx=0 tx=0).stop 2024-04-25T08:54:35.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.049+0000 7fa201438640 1 -- 172.21.15.43:0/1399798767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1fc15e090 msgr2=0x7fa1fc143d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:35.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.049+0000 7fa201438640 1 --2- 172.21.15.43:0/1399798767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1fc15e090 0x7fa1fc143d50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa1f00098c0 tx=0x7fa1f00046c0 comp rx=0 tx=0).stop 2024-04-25T08:54:35.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.049+0000 7fa201438640 1 -- 172.21.15.43:0/1399798767 shutdown_connections 2024-04-25T08:54:35.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.049+0000 7fa201438640 1 --2- 172.21.15.43:0/1399798767 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa1fc161fe0 0x7fa1fc144290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:35.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.049+0000 7fa201438640 1 --2- 172.21.15.43:0/1399798767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1fc15e090 0x7fa1fc143d50 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:35.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.049+0000 7fa201438640 1 --2- 172.21.15.43:0/1399798767 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa1e806b4d0 0x7fa1e806d990 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:35.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.049+0000 7fa201438640 1 --2- 172.21.15.43:0/1399798767 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1fc0b5aa0 0x7fa1fc15bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:35.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.049+0000 7fa201438640 1 -- 172.21.15.43:0/1399798767 >> 172.21.15.43:0/1399798767 conn(0x7fa1fc0d0560 msgr2=0x7fa1fc160480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:35.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.050+0000 7fa201438640 1 -- 172.21.15.43:0/1399798767 shutdown_connections 2024-04-25T08:54:35.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.050+0000 7fa201438640 1 -- 172.21.15.43:0/1399798767 wait complete. 2024-04-25T08:54:35.051 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:54:35.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.164+0000 7f189f4f7640 1 Processor -- start 2024-04-25T08:54:35.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.165+0000 7f189f4f7640 1 -- start start 2024-04-25T08:54:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.165+0000 7f189f4f7640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f189815dd80 0x7f18980cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.165+0000 7f189f4f7640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18980b2fe0 con 0x7f18980b1cd0 2024-04-25T08:54:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.165+0000 7f189f4f7640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18980ced70 con 0x7f18980d17b0 2024-04-25T08:54:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.165+0000 7f189f4f7640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18980cfc60 con 0x7f189815dd80 2024-04-25T08:54:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.165+0000 7f189da94640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f18980b1cd0 0x7f189815e870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:55694/0 (socket says 172.21.15.43:55694) 2024-04-25T08:54:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.165+0000 7f189da94640 1 -- 172.21.15.43:0/954587073 learned_addr learned my addr 172.21.15.43:0/954587073 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.166+0000 7f189d293640 1 --2- 172.21.15.43:0/954587073 >> v2:172.21.15.193:3300/0 conn(0x7f189815dd80 0x7f18980cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.166+0000 7f1897fff640 1 -- 172.21.15.43:0/954587073 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2947159025 0 0) 0x7f18980b2fe0 con 0x7f18980b1cd0 2024-04-25T08:54:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.166+0000 7f1897fff640 1 -- 172.21.15.43:0/954587073 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1880003520 con 0x7f18980b1cd0 2024-04-25T08:54:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.166+0000 7f1897fff640 1 -- 172.21.15.43:0/954587073 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4209258832 0 0) 0x7f1880003520 con 0x7f18980b1cd0 2024-04-25T08:54:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.166+0000 7f1897fff640 1 -- 172.21.15.43:0/954587073 >> v2:172.21.15.193:3300/0 conn(0x7f189815dd80 msgr2=0x7f18980cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.166+0000 7f1897fff640 1 --2- 172.21.15.43:0/954587073 >> v2:172.21.15.193:3300/0 conn(0x7f189815dd80 0x7f18980cf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:35.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.166+0000 7f1897fff640 1 -- 172.21.15.43:0/954587073 >> v1:172.21.15.193:6789/0 conn(0x7f18980d17b0 legacy=0x7f189815e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:35.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.166+0000 7f1897fff640 1 -- 172.21.15.43:0/954587073 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18980cf050 con 0x7f18980b1cd0 2024-04-25T08:54:35.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.166+0000 7f1897fff640 1 -- 172.21.15.43:0/954587073 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f18880019f0 con 0x7f18980b1cd0 2024-04-25T08:54:35.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.166+0000 7f1897fff640 1 -- 172.21.15.43:0/954587073 >> v1:172.21.15.43:6789/0 conn(0x7f18980b1cd0 legacy=0x7f189815e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:35.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.166+0000 7f1897fff640 1 --2- 172.21.15.43:0/954587073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18800039b0 0x7f1880003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:35.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.166+0000 7f1897fff640 1 --2- 172.21.15.43:0/954587073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1880004800 0x7f1880004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:35.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.166+0000 7f1897fff640 1 --2- 172.21.15.43:0/954587073 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f18800051c0 0x7f188000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:35.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.166+0000 7f1897fff640 1 -- 172.21.15.43:0/954587073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18980cfc60 con 0x7f18800039b0 2024-04-25T08:54:35.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.166+0000 7f1897fff640 1 -- 172.21.15.43:0/954587073 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188000c160 con 0x7f1880004800 2024-04-25T08:54:35.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.166+0000 7f1897fff640 1 -- 172.21.15.43:0/954587073 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f188000c360 con 0x7f18800051c0 2024-04-25T08:54:35.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.166+0000 7f189d293640 1 --2- 172.21.15.43:0/954587073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18800039b0 0x7f1880003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:35.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.166+0000 7f189ca92640 1 --2- 172.21.15.43:0/954587073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1880004800 0x7f1880004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:35.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.166+0000 7f189da94640 1 --2- 172.21.15.43:0/954587073 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f18800051c0 0x7f188000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:35.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.166+0000 7f189d293640 1 -- 172.21.15.43:0/954587073 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f18800051c0 msgr2=0x7f188000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:35.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.166+0000 7f189d293640 1 --2- 172.21.15.43:0/954587073 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f18800051c0 0x7f188000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:35.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.166+0000 7f189d293640 1 -- 172.21.15.43:0/954587073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1880004800 msgr2=0x7f1880004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:35.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.166+0000 7f189d293640 1 --2- 172.21.15.43:0/954587073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1880004800 0x7f1880004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:35.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.167+0000 7f189da94640 1 -- 172.21.15.43:0/954587073 reap_dead start 2024-04-25T08:54:35.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.167+0000 7f189d293640 1 -- 172.21.15.43:0/954587073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1880003520 con 0x7f18800039b0 2024-04-25T08:54:35.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.167+0000 7f189ca92640 1 --2- 172.21.15.43:0/954587073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1880004800 0x7f1880004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:35.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.167+0000 7f189d293640 1 --2- 172.21.15.43:0/954587073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18800039b0 0x7f1880003dd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1890009ac0 tx=0x7f1880005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ff14059efac737f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:35.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.167+0000 7f1897fff640 1 -- 172.21.15.43:0/954587073 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1890036070 con 0x7f18800039b0 2024-04-25T08:54:35.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.167+0000 7f1897fff640 1 -- 172.21.15.43:0/954587073 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1890031400 con 0x7f18800039b0 2024-04-25T08:54:35.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.167+0000 7f1897fff640 1 -- 172.21.15.43:0/954587073 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1890009690 con 0x7f18800039b0 2024-04-25T08:54:35.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.168+0000 7f189f4f7640 1 -- 172.21.15.43:0/954587073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18800039b0 msgr2=0x7f1880003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:35.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.168+0000 7f189f4f7640 1 --2- 172.21.15.43:0/954587073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18800039b0 0x7f1880003dd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1890009ac0 tx=0x7f1880005830 comp rx=0 tx=0).stop 2024-04-25T08:54:35.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.168+0000 7f189f4f7640 1 -- 172.21.15.43:0/954587073 shutdown_connections 2024-04-25T08:54:35.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.168+0000 7f189f4f7640 1 --2- 172.21.15.43:0/954587073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18800039b0 0x7f1880003dd0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:35.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.168+0000 7f189f4f7640 1 -- 172.21.15.43:0/954587073 >> 172.21.15.43:0/954587073 conn(0x7f1898072aa0 msgr2=0x7f189809aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:35.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.168+0000 7f189f4f7640 1 -- 172.21.15.43:0/954587073 shutdown_connections 2024-04-25T08:54:35.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.168+0000 7f189f4f7640 1 -- 172.21.15.43:0/954587073 wait complete. 2024-04-25T08:54:35.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.169+0000 7f189f4f7640 1 Processor -- start 2024-04-25T08:54:35.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.169+0000 7f189f4f7640 1 -- start start 2024-04-25T08:54:35.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.170+0000 7f189f4f7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18800039b0 0x7f1898143e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:35.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.170+0000 7f189f4f7640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f18980ae7b0 0x7f1898146ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:35.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.170+0000 7f189f4f7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18980afdf0 0x7f18980cab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:35.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.170+0000 7f189f4f7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18980b2c40 con 0x7f18800039b0 2024-04-25T08:54:35.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.170+0000 7f189f4f7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18980b32c0 con 0x7f18980afdf0 2024-04-25T08:54:35.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.170+0000 7f189f4f7640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18980b2dc0 con 0x7f18980ae7b0 2024-04-25T08:54:35.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.170+0000 7f189d293640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18800039b0 0x7f1898143e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:35.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.170+0000 7f189d293640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18800039b0 0x7f1898143e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40092/0 (socket says 172.21.15.43:40092) 2024-04-25T08:54:35.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.170+0000 7f189d293640 1 -- 172.21.15.43:0/695240897 learned_addr learned my addr 172.21.15.43:0/695240897 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:35.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.170+0000 7f189da94640 1 --2- 172.21.15.43:0/695240897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18980afdf0 0x7f18980cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:35.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.170+0000 7f189ca92640 1 --2- 172.21.15.43:0/695240897 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f18980ae7b0 0x7f1898146ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:35.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.170+0000 7f189d293640 1 -- 172.21.15.43:0/695240897 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f18980ae7b0 msgr2=0x7f1898146ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:35.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.170+0000 7f189d293640 1 --2- 172.21.15.43:0/695240897 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f18980ae7b0 0x7f1898146ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:35.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.170+0000 7f189d293640 1 -- 172.21.15.43:0/695240897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18980afdf0 msgr2=0x7f18980cab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:35.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.170+0000 7f189d293640 1 --2- 172.21.15.43:0/695240897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18980afdf0 0x7f18980cab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:35.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.170+0000 7f189d293640 1 -- 172.21.15.43:0/695240897 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18980aed50 con 0x7f18800039b0 2024-04-25T08:54:35.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.170+0000 7f189ca92640 1 --2- 172.21.15.43:0/695240897 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f18980ae7b0 0x7f1898146ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:35.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.171+0000 7f189d293640 1 --2- 172.21.15.43:0/695240897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18800039b0 0x7f1898143e10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1890009be0 tx=0x7f1890004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:35.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.171+0000 7f1895ffb640 1 -- 172.21.15.43:0/695240897 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189003f020 con 0x7f18800039b0 2024-04-25T08:54:35.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.171+0000 7f1895ffb640 1 -- 172.21.15.43:0/695240897 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1890037040 con 0x7f18800039b0 2024-04-25T08:54:35.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.171+0000 7f189f4f7640 1 -- 172.21.15.43:0/695240897 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18980af030 con 0x7f18800039b0 2024-04-25T08:54:35.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.171+0000 7f1895ffb640 1 -- 172.21.15.43:0/695240897 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1890036070 con 0x7f18800039b0 2024-04-25T08:54:35.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.171+0000 7f189f4f7640 1 -- 172.21.15.43:0/695240897 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18980af5f0 con 0x7f18800039b0 2024-04-25T08:54:35.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.172+0000 7f189f4f7640 1 -- 172.21.15.43:0/695240897 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1898074f60 con 0x7f18800039b0 2024-04-25T08:54:35.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.172+0000 7f1895ffb640 1 -- 172.21.15.43:0/695240897 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1890042020 con 0x7f18800039b0 2024-04-25T08:54:35.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.173+0000 7f1895ffb640 1 --2- 172.21.15.43:0/695240897 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f188406b4d0 0x7f188406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:35.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.173+0000 7f1895ffb640 1 -- 172.21.15.43:0/695240897 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f18900aa180 con 0x7f18800039b0 2024-04-25T08:54:35.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.173+0000 7f189ca92640 1 --2- 172.21.15.43:0/695240897 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f188406b4d0 0x7f188406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:35.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.174+0000 7f189ca92640 1 --2- 172.21.15.43:0/695240897 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f188406b4d0 0x7f188406d990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f188c000c00 tx=0x7f188c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:35.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.177+0000 7f1895ffb640 1 -- 172.21.15.43:0/695240897 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1890049030 con 0x7f18800039b0 2024-04-25T08:54:35.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.356+0000 7f189f4f7640 1 -- 172.21.15.43:0/695240897 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f18980b2630 con 0x7f18800039b0 2024-04-25T08:54:35.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.356+0000 7f1895ffb640 1 -- 172.21.15.43:0/695240897 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1890079090 con 0x7f18800039b0 2024-04-25T08:54:35.347 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T08:54:35.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.358+0000 7f189f4f7640 1 -- 172.21.15.43:0/695240897 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f188406b4d0 msgr2=0x7f188406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:35.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.358+0000 7f189f4f7640 1 --2- 172.21.15.43:0/695240897 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f188406b4d0 0x7f188406d990 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f188c000c00 tx=0x7f188c009040 comp rx=0 tx=0).stop 2024-04-25T08:54:35.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.358+0000 7f189f4f7640 1 -- 172.21.15.43:0/695240897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18800039b0 msgr2=0x7f1898143e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:35.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.358+0000 7f189f4f7640 1 --2- 172.21.15.43:0/695240897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18800039b0 0x7f1898143e10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1890009be0 tx=0x7f1890004730 comp rx=0 tx=0).stop 2024-04-25T08:54:35.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.358+0000 7f189f4f7640 1 -- 172.21.15.43:0/695240897 shutdown_connections 2024-04-25T08:54:35.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.358+0000 7f189f4f7640 1 --2- 172.21.15.43:0/695240897 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f188406b4d0 0x7f188406d990 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:35.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.358+0000 7f189f4f7640 1 --2- 172.21.15.43:0/695240897 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18980afdf0 0x7f18980cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:35.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.358+0000 7f189f4f7640 1 --2- 172.21.15.43:0/695240897 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f18980ae7b0 0x7f1898146ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:35.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.358+0000 7f189f4f7640 1 --2- 172.21.15.43:0/695240897 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18800039b0 0x7f1898143e10 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:35.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.358+0000 7f189f4f7640 1 -- 172.21.15.43:0/695240897 >> 172.21.15.43:0/695240897 conn(0x7f1898072aa0 msgr2=0x7f189806ebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:35.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.358+0000 7f189f4f7640 1 -- 172.21.15.43:0/695240897 shutdown_connections 2024-04-25T08:54:35.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.358+0000 7f189f4f7640 1 -- 172.21.15.43:0/695240897 wait complete. 2024-04-25T08:54:35.359 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:54:35.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.469+0000 7feffb5d5640 1 Processor -- start 2024-04-25T08:54:35.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.469+0000 7feffb5d5640 1 -- start start 2024-04-25T08:54:35.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.470+0000 7feffb5d5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feff40d39d0 0x7feff40d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:35.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.470+0000 7feffb5d5640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7feff40d2500 0x7feff40d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:35.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.470+0000 7feffb5d5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feff40b6c50 con 0x7feff40ce2a0 2024-04-25T08:54:35.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.470+0000 7feffb5d5640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feff415e010 con 0x7feff40d2500 2024-04-25T08:54:35.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.470+0000 7feffb5d5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff40d2020 con 0x7feff40d39d0 2024-04-25T08:54:35.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.470+0000 7feff8b70640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7feff40ce2a0 0x7feff40d3db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:55696/0 (socket says 172.21.15.43:55696) 2024-04-25T08:54:35.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.470+0000 7feff8b70640 1 -- 172.21.15.43:0/4061051074 learned_addr learned my addr 172.21.15.43:0/4061051074 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:35.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.471+0000 7feff9b72640 1 --2- 172.21.15.43:0/4061051074 >> v2:172.21.15.193:3300/0 conn(0x7feff40d2500 0x7feff40d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:35.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.471+0000 7feff3fff640 1 -- 172.21.15.43:0/4061051074 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3789357730 0 0) 0x7fefe8001110 con 0x7feff40ce2a0 2024-04-25T08:54:35.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.471+0000 7feff9371640 1 --2- 172.21.15.43:0/4061051074 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feff40d39d0 0x7feff40d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:35.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.471+0000 7feff3fff640 1 -- 172.21.15.43:0/4061051074 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fefdc003520 con 0x7feff40ce2a0 2024-04-25T08:54:35.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.471+0000 7feff3fff640 1 -- 172.21.15.43:0/4061051074 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 401202121 0 0) 0x7fefdc003520 con 0x7feff40ce2a0 2024-04-25T08:54:35.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.471+0000 7feff3fff640 1 -- 172.21.15.43:0/4061051074 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feff40d39d0 msgr2=0x7feff40d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:35.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.471+0000 7feff3fff640 1 --2- 172.21.15.43:0/4061051074 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feff40d39d0 0x7feff40d0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:35.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.471+0000 7feff3fff640 1 -- 172.21.15.43:0/4061051074 >> v2:172.21.15.193:3300/0 conn(0x7feff40d2500 msgr2=0x7feff40d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:35.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.471+0000 7feff3fff640 1 --2- 172.21.15.43:0/4061051074 >> v2:172.21.15.193:3300/0 conn(0x7feff40d2500 0x7feff40d2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:35.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.471+0000 7feff3fff640 1 -- 172.21.15.43:0/4061051074 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff40cec40 con 0x7feff40ce2a0 2024-04-25T08:54:35.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.471+0000 7feff3fff640 1 -- 172.21.15.43:0/4061051074 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7fefe8001840 con 0x7feff40ce2a0 2024-04-25T08:54:35.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.471+0000 7feff3fff640 1 -- 172.21.15.43:0/4061051074 >> v1:172.21.15.43:6789/0 conn(0x7feff40ce2a0 legacy=0x7feff40d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:35.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.471+0000 7feff3fff640 1 --2- 172.21.15.43:0/4061051074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefdc0039b0 0x7fefdc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:35.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.471+0000 7feff3fff640 1 --2- 172.21.15.43:0/4061051074 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefdc004800 0x7fefdc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:35.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.471+0000 7feff3fff640 1 --2- 172.21.15.43:0/4061051074 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feff40d39d0 0x7fefdc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:35.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.471+0000 7feff3fff640 1 -- 172.21.15.43:0/4061051074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff415e010 con 0x7fefdc0039b0 2024-04-25T08:54:35.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.471+0000 7feff3fff640 1 -- 172.21.15.43:0/4061051074 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff40d2020 con 0x7fefdc004800 2024-04-25T08:54:35.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.471+0000 7feff3fff640 1 -- 172.21.15.43:0/4061051074 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefdc00b9c0 con 0x7feff40d39d0 2024-04-25T08:54:35.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.471+0000 7feff9371640 1 --2- 172.21.15.43:0/4061051074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefdc0039b0 0x7fefdc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:35.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.471+0000 7feff8b70640 1 --2- 172.21.15.43:0/4061051074 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefdc004800 0x7fefdc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:35.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.471+0000 7feff9b72640 1 --2- 172.21.15.43:0/4061051074 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feff40d39d0 0x7fefdc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:35.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.471+0000 7feff9371640 1 -- 172.21.15.43:0/4061051074 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feff40d39d0 msgr2=0x7fefdc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:35.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.471+0000 7feff9371640 1 --2- 172.21.15.43:0/4061051074 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feff40d39d0 0x7fefdc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:35.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.471+0000 7feff9371640 1 -- 172.21.15.43:0/4061051074 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefdc004800 msgr2=0x7fefdc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:35.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.471+0000 7feff9371640 1 --2- 172.21.15.43:0/4061051074 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fefdc004800 0x7fefdc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:35.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.471+0000 7feff9371640 1 -- 172.21.15.43:0/4061051074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefdc003520 con 0x7fefdc0039b0 2024-04-25T08:54:35.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.472+0000 7feff9371640 1 --2- 172.21.15.43:0/4061051074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefdc0039b0 0x7fefdc003dd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fefec009af0 tx=0x7fefdc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d3c8a3b2e3e3068 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:35.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.472+0000 7feff3fff640 1 -- 172.21.15.43:0/4061051074 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec036070 con 0x7fefdc0039b0 2024-04-25T08:54:35.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.472+0000 7feff3fff640 1 -- 172.21.15.43:0/4061051074 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefec031400 con 0x7fefdc0039b0 2024-04-25T08:54:35.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.472+0000 7feff3fff640 1 -- 172.21.15.43:0/4061051074 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefec009690 con 0x7fefdc0039b0 2024-04-25T08:54:35.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.473+0000 7feffb5d5640 1 -- 172.21.15.43:0/4061051074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefdc0039b0 msgr2=0x7fefdc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:35.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.473+0000 7feffb5d5640 1 --2- 172.21.15.43:0/4061051074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fefdc0039b0 0x7fefdc003dd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fefec009af0 tx=0x7fefdc005830 comp rx=0 tx=0).stop 2024-04-25T08:54:35.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.473+0000 7feff9b72640 1 -- 172.21.15.43:0/4061051074 reap_dead start 2024-04-25T08:54:35.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.473+0000 7feffb5d5640 1 -- 172.21.15.43:0/4061051074 shutdown_connections 2024-04-25T08:54:35.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.473+0000 7feffb5d5640 1 -- 172.21.15.43:0/4061051074 >> 172.21.15.43:0/4061051074 conn(0x7feff406b000 msgr2=0x7feff406b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:35.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.473+0000 7feffb5d5640 1 -- 172.21.15.43:0/4061051074 shutdown_connections 2024-04-25T08:54:35.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.473+0000 7feffb5d5640 1 -- 172.21.15.43:0/4061051074 wait complete. 2024-04-25T08:54:35.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.474+0000 7feffb5d5640 1 Processor -- start 2024-04-25T08:54:35.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.474+0000 7feffb5d5640 1 -- start start 2024-04-25T08:54:35.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.475+0000 7feffb5d5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fefdc0039b0 0x7feff4159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:35.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.475+0000 7feffb5d5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feff40b54d0 0x7feff4155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:35.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.475+0000 7feffb5d5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feff4156970 0x7feff4156df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:35.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.475+0000 7feffb5d5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff40cf200 con 0x7feff40b54d0 2024-04-25T08:54:35.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.475+0000 7feffb5d5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff40b2f00 con 0x7feff4156970 2024-04-25T08:54:35.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.475+0000 7feffb5d5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff40d0d90 con 0x7fefdc0039b0 2024-04-25T08:54:35.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.475+0000 7feff8b70640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feff40b54d0 0x7feff4155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:35.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.475+0000 7feff8b70640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feff40b54d0 0x7feff4155d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40110/0 (socket says 172.21.15.43:40110) 2024-04-25T08:54:35.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.475+0000 7feff8b70640 1 -- 172.21.15.43:0/1903048250 learned_addr learned my addr 172.21.15.43:0/1903048250 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:35.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.475+0000 7feff8b70640 1 -- 172.21.15.43:0/1903048250 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fefdc0039b0 msgr2=0x7feff4159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:35.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.475+0000 7feff8b70640 1 --2- 172.21.15.43:0/1903048250 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fefdc0039b0 0x7feff4159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:35.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.475+0000 7feff8b70640 1 -- 172.21.15.43:0/1903048250 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feff4156970 msgr2=0x7feff4156df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:35.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.475+0000 7feff8b70640 1 --2- 172.21.15.43:0/1903048250 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feff4156970 0x7feff4156df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:35.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.475+0000 7feff8b70640 1 -- 172.21.15.43:0/1903048250 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff415a4d0 con 0x7feff40b54d0 2024-04-25T08:54:35.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.475+0000 7feff8b70640 1 --2- 172.21.15.43:0/1903048250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feff40b54d0 0x7feff4155d40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fefe800e780 tx=0x7fefe800ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:35.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.476+0000 7feff1ffb640 1 -- 172.21.15.43:0/1903048250 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe800c640 con 0x7feff40b54d0 2024-04-25T08:54:35.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.476+0000 7feff1ffb640 1 -- 172.21.15.43:0/1903048250 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefe8018070 con 0x7feff40b54d0 2024-04-25T08:54:35.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.476+0000 7feffb5d5640 1 -- 172.21.15.43:0/1903048250 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feff415be90 con 0x7feff40b54d0 2024-04-25T08:54:35.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.476+0000 7feff1ffb640 1 -- 172.21.15.43:0/1903048250 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefe800ca40 con 0x7feff40b54d0 2024-04-25T08:54:35.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.476+0000 7feffb5d5640 1 -- 172.21.15.43:0/1903048250 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feff415c450 con 0x7feff40b54d0 2024-04-25T08:54:35.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.477+0000 7feffb5d5640 1 -- 172.21.15.43:0/1903048250 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feff415c650 con 0x7feff40b54d0 2024-04-25T08:54:35.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.477+0000 7feff1ffb640 1 -- 172.21.15.43:0/1903048250 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fefe800cbe0 con 0x7feff40b54d0 2024-04-25T08:54:35.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.477+0000 7feff1ffb640 1 --2- 172.21.15.43:0/1903048250 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fefe006b4d0 0x7fefe006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:35.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.478+0000 7feff1ffb640 1 -- 172.21.15.43:0/1903048250 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fefe808a000 con 0x7feff40b54d0 2024-04-25T08:54:35.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.478+0000 7feff9371640 1 --2- 172.21.15.43:0/1903048250 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fefe006b4d0 0x7fefe006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:35.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.478+0000 7feff9371640 1 --2- 172.21.15.43:0/1903048250 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fefe006b4d0 0x7fefe006d990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fefec033000 tx=0x7fefec02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:35.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.482+0000 7feff1ffb640 1 -- 172.21.15.43:0/1903048250 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feff415c650 con 0x7feff40b54d0 2024-04-25T08:54:35.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.663+0000 7feffb5d5640 1 -- 172.21.15.43:0/1903048250 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feff40ce880 con 0x7feff40b54d0 2024-04-25T08:54:35.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.664+0000 7feff1ffb640 1 -- 172.21.15.43:0/1903048250 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16569 (secure 0 0 0) 0x7feff40b6c50 con 0x7feff40b54d0 2024-04-25T08:54:35.654 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:54:35.654 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":20,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:54:13.593188+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:54:09.757978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:54:10.926008+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:54:35.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.666+0000 7feffb5d5640 1 -- 172.21.15.43:0/1903048250 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fefe006b4d0 msgr2=0x7fefe006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:35.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.666+0000 7feffb5d5640 1 --2- 172.21.15.43:0/1903048250 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fefe006b4d0 0x7fefe006d990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fefec033000 tx=0x7fefec02e000 comp rx=0 tx=0).stop 2024-04-25T08:54:35.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.666+0000 7feffb5d5640 1 -- 172.21.15.43:0/1903048250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feff40b54d0 msgr2=0x7feff4155d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:35.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.666+0000 7feffb5d5640 1 --2- 172.21.15.43:0/1903048250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feff40b54d0 0x7feff4155d40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fefe800e780 tx=0x7fefe800ec40 comp rx=0 tx=0).stop 2024-04-25T08:54:35.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.666+0000 7feffb5d5640 1 -- 172.21.15.43:0/1903048250 shutdown_connections 2024-04-25T08:54:35.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.666+0000 7feffb5d5640 1 --2- 172.21.15.43:0/1903048250 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fefe006b4d0 0x7fefe006d990 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:35.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.666+0000 7feffb5d5640 1 --2- 172.21.15.43:0/1903048250 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feff4156970 0x7feff4156df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:35.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.666+0000 7feffb5d5640 1 --2- 172.21.15.43:0/1903048250 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feff40b54d0 0x7feff4155d40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:35.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.666+0000 7feffb5d5640 1 --2- 172.21.15.43:0/1903048250 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fefdc0039b0 0x7feff4159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:35.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.666+0000 7feffb5d5640 1 -- 172.21.15.43:0/1903048250 >> 172.21.15.43:0/1903048250 conn(0x7feff406b000 msgr2=0x7feff40c93d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:35.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.666+0000 7feffb5d5640 1 -- 172.21.15.43:0/1903048250 shutdown_connections 2024-04-25T08:54:35.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.666+0000 7feffb5d5640 1 -- 172.21.15.43:0/1903048250 wait complete. 2024-04-25T08:54:35.667 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:54:09.757978+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:54:10.926008+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:54:35.667 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:54:35.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.781+0000 7fcc0b41b640 1 Processor -- start 2024-04-25T08:54:35.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.781+0000 7fcc0b41b640 1 -- start start 2024-04-25T08:54:35.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.782+0000 7fcc0b41b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc040c8870 0x7fcc040d2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:35.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.782+0000 7fcc0b41b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc040cf030 con 0x7fcc040d19f0 2024-04-25T08:54:35.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.782+0000 7fcc0b41b640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc040cf310 con 0x7fcc04161310 2024-04-25T08:54:35.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.782+0000 7fcc0b41b640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc040d4870 con 0x7fcc040c8870 2024-04-25T08:54:35.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.782+0000 7fcc089b6640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fcc040d19f0 0x7fcc040d1e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:55698/0 (socket says 172.21.15.43:55698) 2024-04-25T08:54:35.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.782+0000 7fcc089b6640 1 -- 172.21.15.43:0/1900877921 learned_addr learned my addr 172.21.15.43:0/1900877921 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:35.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.782+0000 7fcc03fff640 1 -- 172.21.15.43:0/1900877921 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2405497167 0 0) 0x7fcc040cf030 con 0x7fcc040d19f0 2024-04-25T08:54:35.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.782+0000 7fcc03fff640 1 -- 172.21.15.43:0/1900877921 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbec003520 con 0x7fcc040d19f0 2024-04-25T08:54:35.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.782+0000 7fcc099b8640 1 --2- 172.21.15.43:0/1900877921 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc040c8870 0x7fcc040d2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:35.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.783+0000 7fcc03fff640 1 -- 172.21.15.43:0/1900877921 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3951724098 0 0) 0x7fcbec003520 con 0x7fcc040d19f0 2024-04-25T08:54:35.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.783+0000 7fcc03fff640 1 -- 172.21.15.43:0/1900877921 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc040c8870 msgr2=0x7fcc040d2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:35.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.783+0000 7fcc03fff640 1 --2- 172.21.15.43:0/1900877921 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc040c8870 0x7fcc040d2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:35.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.783+0000 7fcc03fff640 1 -- 172.21.15.43:0/1900877921 >> v1:172.21.15.193:6789/0 conn(0x7fcc04161310 legacy=0x7fcc040d4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:35.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.783+0000 7fcc03fff640 1 -- 172.21.15.43:0/1900877921 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc040cf5f0 con 0x7fcc040d19f0 2024-04-25T08:54:35.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.783+0000 7fcc03fff640 1 -- 172.21.15.43:0/1900877921 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7fcbf8001670 con 0x7fcc040d19f0 2024-04-25T08:54:35.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.783+0000 7fcc03fff640 1 -- 172.21.15.43:0/1900877921 >> v1:172.21.15.43:6789/0 conn(0x7fcc040d19f0 legacy=0x7fcc040d1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:35.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.783+0000 7fcc03fff640 1 --2- 172.21.15.43:0/1900877921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbec0039e0 0x7fcbec003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:35.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.783+0000 7fcc03fff640 1 --2- 172.21.15.43:0/1900877921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbec004800 0x7fcbec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:35.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.783+0000 7fcc03fff640 1 --2- 172.21.15.43:0/1900877921 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc040c8870 0x7fcbec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:35.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.783+0000 7fcc03fff640 1 -- 172.21.15.43:0/1900877921 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc040d4870 con 0x7fcbec004800 2024-04-25T08:54:35.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.783+0000 7fcc03fff640 1 -- 172.21.15.43:0/1900877921 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbec00ba70 con 0x7fcbec0039e0 2024-04-25T08:54:35.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.783+0000 7fcc03fff640 1 -- 172.21.15.43:0/1900877921 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbec00bc70 con 0x7fcc040c8870 2024-04-25T08:54:35.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.783+0000 7fcc089b6640 1 --2- 172.21.15.43:0/1900877921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbec004800 0x7fcbec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:35.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.783+0000 7fcc091b7640 1 --2- 172.21.15.43:0/1900877921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbec0039e0 0x7fcbec003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:35.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.783+0000 7fcc099b8640 1 --2- 172.21.15.43:0/1900877921 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc040c8870 0x7fcbec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:35.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.783+0000 7fcc089b6640 1 -- 172.21.15.43:0/1900877921 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc040c8870 msgr2=0x7fcbec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:35.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.783+0000 7fcc089b6640 1 --2- 172.21.15.43:0/1900877921 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc040c8870 0x7fcbec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:35.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.783+0000 7fcc089b6640 1 -- 172.21.15.43:0/1900877921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbec0039e0 msgr2=0x7fcbec003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:35.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.783+0000 7fcc089b6640 1 --2- 172.21.15.43:0/1900877921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbec0039e0 0x7fcbec003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:35.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.783+0000 7fcc089b6640 1 -- 172.21.15.43:0/1900877921 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbec003520 con 0x7fcbec004800 2024-04-25T08:54:35.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.784+0000 7fcc089b6640 1 --2- 172.21.15.43:0/1900877921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbec004800 0x7fcbec004c80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8001050 tx=0x7fcbec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=460e087c6e6777f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:35.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.784+0000 7fcc03fff640 1 -- 172.21.15.43:0/1900877921 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8007d30 con 0x7fcbec004800 2024-04-25T08:54:35.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.784+0000 7fcc03fff640 1 -- 172.21.15.43:0/1900877921 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbf8005af0 con 0x7fcbec004800 2024-04-25T08:54:35.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.784+0000 7fcc03fff640 1 -- 172.21.15.43:0/1900877921 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8007070 con 0x7fcbec004800 2024-04-25T08:54:35.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.785+0000 7fcc0b41b640 1 -- 172.21.15.43:0/1900877921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbec004800 msgr2=0x7fcbec004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:35.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.785+0000 7fcc0b41b640 1 --2- 172.21.15.43:0/1900877921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbec004800 0x7fcbec004c80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8001050 tx=0x7fcbec005830 comp rx=0 tx=0).stop 2024-04-25T08:54:35.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.785+0000 7fcc099b8640 1 -- 172.21.15.43:0/1900877921 reap_dead start 2024-04-25T08:54:35.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.785+0000 7fcc0b41b640 1 -- 172.21.15.43:0/1900877921 shutdown_connections 2024-04-25T08:54:35.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.785+0000 7fcc0b41b640 1 -- 172.21.15.43:0/1900877921 >> 172.21.15.43:0/1900877921 conn(0x7fcc0409cd20 msgr2=0x7fcc040d3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:35.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.785+0000 7fcc0b41b640 1 -- 172.21.15.43:0/1900877921 shutdown_connections 2024-04-25T08:54:35.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.785+0000 7fcc0b41b640 1 -- 172.21.15.43:0/1900877921 wait complete. 2024-04-25T08:54:35.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.786+0000 7fcc0b41b640 1 Processor -- start 2024-04-25T08:54:35.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.786+0000 7fcc0b41b640 1 -- start start 2024-04-25T08:54:35.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.787+0000 7fcc0b41b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbec004800 0x7fcc041f39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:35.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.787+0000 7fcc0b41b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc04156dd0 0x7fcc04157230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:35.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.787+0000 7fcc0b41b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc040cbed0 0x7fcc040cc350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:35.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.787+0000 7fcc0b41b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc040d26c0 con 0x7fcc040cbed0 2024-04-25T08:54:35.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.787+0000 7fcc0b41b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc040d2540 con 0x7fcbec004800 2024-04-25T08:54:35.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.787+0000 7fcc0b41b640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc040d2840 con 0x7fcc04156dd0 2024-04-25T08:54:35.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.787+0000 7fcc099b8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc040cbed0 0x7fcc040cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:35.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.787+0000 7fcc099b8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc040cbed0 0x7fcc040cc350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40126/0 (socket says 172.21.15.43:40126) 2024-04-25T08:54:35.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.787+0000 7fcc099b8640 1 -- 172.21.15.43:0/2518146439 learned_addr learned my addr 172.21.15.43:0/2518146439 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:35.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.787+0000 7fcc091b7640 1 --2- 172.21.15.43:0/2518146439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbec004800 0x7fcc041f39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:35.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.787+0000 7fcc089b6640 1 --2- 172.21.15.43:0/2518146439 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc04156dd0 0x7fcc04157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:35.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.787+0000 7fcc099b8640 1 -- 172.21.15.43:0/2518146439 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc04156dd0 msgr2=0x7fcc04157230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:35.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.787+0000 7fcc099b8640 1 --2- 172.21.15.43:0/2518146439 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc04156dd0 0x7fcc04157230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:35.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.787+0000 7fcc099b8640 1 -- 172.21.15.43:0/2518146439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbec004800 msgr2=0x7fcc041f39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:35.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.787+0000 7fcc099b8640 1 --2- 172.21.15.43:0/2518146439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbec004800 0x7fcc041f39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:35.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.787+0000 7fcc099b8640 1 -- 172.21.15.43:0/2518146439 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc040ccca0 con 0x7fcc040cbed0 2024-04-25T08:54:35.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.787+0000 7fcc099b8640 1 --2- 172.21.15.43:0/2518146439 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc040cbed0 0x7fcc040cc350 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcc0408dee0 tx=0x7fcbf400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:35.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.787+0000 7fcc01ffb640 1 -- 172.21.15.43:0/2518146439 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf4019070 con 0x7fcc040cbed0 2024-04-25T08:54:35.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.787+0000 7fcc01ffb640 1 -- 172.21.15.43:0/2518146439 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbf4003990 con 0x7fcc040cbed0 2024-04-25T08:54:35.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.787+0000 7fcc0b41b640 1 -- 172.21.15.43:0/2518146439 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc040ccf80 con 0x7fcc040cbed0 2024-04-25T08:54:35.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.787+0000 7fcc01ffb640 1 -- 172.21.15.43:0/2518146439 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf4003d80 con 0x7fcc040cbed0 2024-04-25T08:54:35.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.788+0000 7fcc0b41b640 1 -- 172.21.15.43:0/2518146439 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc040cd540 con 0x7fcc040cbed0 2024-04-25T08:54:35.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.789+0000 7fcc0b41b640 1 -- 172.21.15.43:0/2518146439 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc04074f60 con 0x7fcc040cbed0 2024-04-25T08:54:35.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.789+0000 7fcc01ffb640 1 -- 172.21.15.43:0/2518146439 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcbf401c070 con 0x7fcc040cbed0 2024-04-25T08:54:35.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.789+0000 7fcc01ffb640 1 --2- 172.21.15.43:0/2518146439 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcbf006b4d0 0x7fcbf006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:35.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.789+0000 7fcc01ffb640 1 -- 172.21.15.43:0/2518146439 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcbf408ac80 con 0x7fcc040cbed0 2024-04-25T08:54:35.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.790+0000 7fcc091b7640 1 --2- 172.21.15.43:0/2518146439 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcbf006b4d0 0x7fcbf006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:35.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.790+0000 7fcc091b7640 1 --2- 172.21.15.43:0/2518146439 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcbf006b4d0 0x7fcbf006d990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc005590 tx=0x7fcbfc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:35.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.794+0000 7fcc01ffb640 1 -- 172.21.15.43:0/2518146439 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcbf4010040 con 0x7fcc040cbed0 2024-04-25T08:54:35.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.976+0000 7fcc0b41b640 1 -- 172.21.15.43:0/2518146439 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcc040b4db0 con 0x7fcc040cbed0 2024-04-25T08:54:35.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.976+0000 7fcc01ffb640 1 -- 172.21.15.43:0/2518146439 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcbf4059af0 con 0x7fcc040cbed0 2024-04-25T08:54:35.966 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T08:54:35.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.978+0000 7fcc0b41b640 1 -- 172.21.15.43:0/2518146439 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcbf006b4d0 msgr2=0x7fcbf006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:35.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.978+0000 7fcc0b41b640 1 --2- 172.21.15.43:0/2518146439 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcbf006b4d0 0x7fcbf006d990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc005590 tx=0x7fcbfc009040 comp rx=0 tx=0).stop 2024-04-25T08:54:35.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.978+0000 7fcc0b41b640 1 -- 172.21.15.43:0/2518146439 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc040cbed0 msgr2=0x7fcc040cc350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:35.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.978+0000 7fcc0b41b640 1 --2- 172.21.15.43:0/2518146439 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc040cbed0 0x7fcc040cc350 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fcc0408dee0 tx=0x7fcbf400f580 comp rx=0 tx=0).stop 2024-04-25T08:54:35.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.978+0000 7fcc0b41b640 1 -- 172.21.15.43:0/2518146439 shutdown_connections 2024-04-25T08:54:35.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.978+0000 7fcc0b41b640 1 --2- 172.21.15.43:0/2518146439 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc040cbed0 0x7fcc040cc350 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:35.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.978+0000 7fcc0b41b640 1 --2- 172.21.15.43:0/2518146439 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc04156dd0 0x7fcc04157230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:35.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.978+0000 7fcc0b41b640 1 --2- 172.21.15.43:0/2518146439 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcbf006b4d0 0x7fcbf006d990 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:35.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.978+0000 7fcc0b41b640 1 --2- 172.21.15.43:0/2518146439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbec004800 0x7fcc041f39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:35.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.978+0000 7fcc0b41b640 1 -- 172.21.15.43:0/2518146439 >> 172.21.15.43:0/2518146439 conn(0x7fcc0409cd20 msgr2=0x7fcc0406b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:35.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.978+0000 7fcc0b41b640 1 -- 172.21.15.43:0/2518146439 shutdown_connections 2024-04-25T08:54:35.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:35.978+0000 7fcc0b41b640 1 -- 172.21.15.43:0/2518146439 wait complete. 2024-04-25T08:54:35.979 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:54:36.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.093+0000 7ff5698e1640 1 Processor -- start 2024-04-25T08:54:36.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.093+0000 7ff5698e1640 1 -- start start 2024-04-25T08:54:36.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.094+0000 7ff5698e1640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff5640b5560 0x7ff5640b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:36.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.094+0000 7ff5698e1640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff564162610 con 0x7ff5640d2230 2024-04-25T08:54:36.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.094+0000 7ff5698e1640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5640d1750 con 0x7ff56415e090 2024-04-25T08:54:36.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.094+0000 7ff5698e1640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5640d1da0 con 0x7ff5640b5560 2024-04-25T08:54:36.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.094+0000 7ff5637fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff5640d2230 0x7ff5640d2690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:55710/0 (socket says 172.21.15.43:55710) 2024-04-25T08:54:36.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.094+0000 7ff5637fe640 1 -- 172.21.15.43:0/542967796 learned_addr learned my addr 172.21.15.43:0/542967796 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:36.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.094+0000 7ff5627fc640 1 --2- 172.21.15.43:0/542967796 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff5640b5560 0x7ff5640b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:36.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.094+0000 7ff561ffb640 1 -- 172.21.15.43:0/542967796 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1996671594 0 0) 0x7ff564162610 con 0x7ff5640d2230 2024-04-25T08:54:36.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.094+0000 7ff561ffb640 1 -- 172.21.15.43:0/542967796 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff550003520 con 0x7ff5640d2230 2024-04-25T08:54:36.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.094+0000 7ff561ffb640 1 -- 172.21.15.43:0/542967796 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2672117156 0 0) 0x7ff5640d1750 con 0x7ff56415e090 2024-04-25T08:54:36.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.094+0000 7ff561ffb640 1 -- 172.21.15.43:0/542967796 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff564162610 con 0x7ff56415e090 2024-04-25T08:54:36.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.094+0000 7ff561ffb640 1 -- 172.21.15.43:0/542967796 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2967389761 0 0) 0x7ff550003520 con 0x7ff5640d2230 2024-04-25T08:54:36.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.094+0000 7ff5627fc640 1 -- 172.21.15.43:0/542967796 >> v1:172.21.15.193:6789/0 conn(0x7ff56415e090 legacy=0x7ff564161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:36.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.094+0000 7ff5627fc640 1 -- 172.21.15.43:0/542967796 >> v1:172.21.15.43:6789/0 conn(0x7ff5640d2230 legacy=0x7ff5640d2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:36.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.094+0000 7ff5627fc640 1 -- 172.21.15.43:0/542967796 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5640d1a30 con 0x7ff5640b5560 2024-04-25T08:54:36.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.095+0000 7ff5627fc640 1 --2- 172.21.15.43:0/542967796 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff5640b5560 0x7ff5640b59e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff5640d1f20 tx=0x7ff54c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d781bfedb0ceac6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:36.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.095+0000 7ff561ffb640 1 -- 172.21.15.43:0/542967796 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54c037070 con 0x7ff5640b5560 2024-04-25T08:54:36.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.095+0000 7ff561ffb640 1 -- 172.21.15.43:0/542967796 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff54c02aca0 con 0x7ff5640b5560 2024-04-25T08:54:36.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.095+0000 7ff561ffb640 1 -- 172.21.15.43:0/542967796 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55c001670 con 0x7ff5640b5560 2024-04-25T08:54:36.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.096+0000 7ff5698e1640 1 -- 172.21.15.43:0/542967796 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff5640b5560 msgr2=0x7ff5640b59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:36.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.096+0000 7ff5698e1640 1 --2- 172.21.15.43:0/542967796 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff5640b5560 0x7ff5640b59e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff5640d1f20 tx=0x7ff54c02a210 comp rx=0 tx=0).stop 2024-04-25T08:54:36.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.096+0000 7ff5698e1640 1 -- 172.21.15.43:0/542967796 shutdown_connections 2024-04-25T08:54:36.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.096+0000 7ff5698e1640 1 --2- 172.21.15.43:0/542967796 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff5640b5560 0x7ff5640b59e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:36.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.096+0000 7ff5698e1640 1 -- 172.21.15.43:0/542967796 >> 172.21.15.43:0/542967796 conn(0x7ff5640d0560 msgr2=0x7ff5640d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:36.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.096+0000 7ff5698e1640 1 -- 172.21.15.43:0/542967796 shutdown_connections 2024-04-25T08:54:36.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.096+0000 7ff5698e1640 1 -- 172.21.15.43:0/542967796 wait complete. 2024-04-25T08:54:36.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.097+0000 7ff5698e1640 1 Processor -- start 2024-04-25T08:54:36.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.097+0000 7ff5698e1640 1 -- start start 2024-04-25T08:54:36.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.098+0000 7ff5698e1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5640b5560 0x7ff5640ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:36.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.098+0000 7ff5698e1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5640d2230 0x7ff56414eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:36.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.098+0000 7ff5698e1640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff56415e090 0x7ff5640abc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:36.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.098+0000 7ff5698e1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5641621b0 con 0x7ff5640d2230 2024-04-25T08:54:36.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.098+0000 7ff5698e1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5640b3280 con 0x7ff5640b5560 2024-04-25T08:54:36.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.098+0000 7ff5698e1640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff564162330 con 0x7ff56415e090 2024-04-25T08:54:36.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.098+0000 7ff5627fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5640d2230 0x7ff56414eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:36.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.098+0000 7ff5627fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5640d2230 0x7ff56414eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40132/0 (socket says 172.21.15.43:40132) 2024-04-25T08:54:36.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.098+0000 7ff5627fc640 1 -- 172.21.15.43:0/3516846299 learned_addr learned my addr 172.21.15.43:0/3516846299 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:36.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.098+0000 7ff5637fe640 1 --2- 172.21.15.43:0/3516846299 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff56415e090 0x7ff5640abc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:36.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.098+0000 7ff5627fc640 1 -- 172.21.15.43:0/3516846299 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff56415e090 msgr2=0x7ff5640abc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:36.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.098+0000 7ff562ffd640 1 --2- 172.21.15.43:0/3516846299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5640b5560 0x7ff5640ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:36.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.099+0000 7ff5627fc640 1 --2- 172.21.15.43:0/3516846299 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff56415e090 0x7ff5640abc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:36.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.099+0000 7ff5627fc640 1 -- 172.21.15.43:0/3516846299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5640b5560 msgr2=0x7ff5640ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:36.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.099+0000 7ff5627fc640 1 --2- 172.21.15.43:0/3516846299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5640b5560 0x7ff5640ceea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:36.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.099+0000 7ff5627fc640 1 -- 172.21.15.43:0/3516846299 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5641f3e00 con 0x7ff5640d2230 2024-04-25T08:54:36.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.099+0000 7ff5637fe640 1 --2- 172.21.15.43:0/3516846299 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff56415e090 0x7ff5640abc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:36.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.099+0000 7ff562ffd640 1 --2- 172.21.15.43:0/3516846299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5640b5560 0x7ff5640ceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:36.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.099+0000 7ff5627fc640 1 --2- 172.21.15.43:0/3516846299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5640d2230 0x7ff56414eea0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff54c035040 tx=0x7ff54c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:36.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.099+0000 7ff55bfff640 1 -- 172.21.15.43:0/3516846299 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54c037050 con 0x7ff5640d2230 2024-04-25T08:54:36.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.099+0000 7ff55bfff640 1 -- 172.21.15.43:0/3516846299 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff54c043070 con 0x7ff5640d2230 2024-04-25T08:54:36.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.099+0000 7ff5698e1640 1 -- 172.21.15.43:0/3516846299 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5641f40e0 con 0x7ff5640d2230 2024-04-25T08:54:36.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.099+0000 7ff55bfff640 1 -- 172.21.15.43:0/3516846299 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54c006e10 con 0x7ff5640d2230 2024-04-25T08:54:36.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.099+0000 7ff5698e1640 1 -- 172.21.15.43:0/3516846299 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5641f46a0 con 0x7ff5640d2230 2024-04-25T08:54:36.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.100+0000 7ff5698e1640 1 -- 172.21.15.43:0/3516846299 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff56406bf00 con 0x7ff5640d2230 2024-04-25T08:54:36.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.100+0000 7ff55bfff640 1 -- 172.21.15.43:0/3516846299 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff54c03ca30 con 0x7ff5640d2230 2024-04-25T08:54:36.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.101+0000 7ff55bfff640 1 --2- 172.21.15.43:0/3516846299 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff54406b500 0x7ff54406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:36.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.101+0000 7ff55bfff640 1 -- 172.21.15.43:0/3516846299 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff54c0aad50 con 0x7ff5640d2230 2024-04-25T08:54:36.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.101+0000 7ff562ffd640 1 --2- 172.21.15.43:0/3516846299 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff54406b500 0x7ff54406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:36.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.102+0000 7ff562ffd640 1 --2- 172.21.15.43:0/3516846299 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff54406b500 0x7ff54406d9c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff554007fd0 tx=0x7ff554009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:36.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.105+0000 7ff55bfff640 1 -- 172.21.15.43:0/3516846299 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff56406bf00 con 0x7ff5640d2230 2024-04-25T08:54:36.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.287+0000 7ff5698e1640 1 -- 172.21.15.43:0/3516846299 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff564074f60 con 0x7ff5640d2230 2024-04-25T08:54:36.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.288+0000 7ff55bfff640 1 -- 172.21.15.43:0/3516846299 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff54c0ae030 con 0x7ff5640d2230 2024-04-25T08:54:36.278 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T08:54:36.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.290+0000 7ff5698e1640 1 -- 172.21.15.43:0/3516846299 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff54406b500 msgr2=0x7ff54406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:36.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.290+0000 7ff5698e1640 1 --2- 172.21.15.43:0/3516846299 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff54406b500 0x7ff54406d9c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff554007fd0 tx=0x7ff554009040 comp rx=0 tx=0).stop 2024-04-25T08:54:36.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.290+0000 7ff5698e1640 1 -- 172.21.15.43:0/3516846299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5640d2230 msgr2=0x7ff56414eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:36.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.290+0000 7ff5698e1640 1 --2- 172.21.15.43:0/3516846299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5640d2230 0x7ff56414eea0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff54c035040 tx=0x7ff54c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:54:36.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.290+0000 7ff5698e1640 1 -- 172.21.15.43:0/3516846299 shutdown_connections 2024-04-25T08:54:36.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.290+0000 7ff5698e1640 1 --2- 172.21.15.43:0/3516846299 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff56415e090 0x7ff5640abc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:36.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.290+0000 7ff5698e1640 1 --2- 172.21.15.43:0/3516846299 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5640d2230 0x7ff56414eea0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:36.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.290+0000 7ff5698e1640 1 --2- 172.21.15.43:0/3516846299 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff54406b500 0x7ff54406d9c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:36.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.290+0000 7ff5698e1640 1 --2- 172.21.15.43:0/3516846299 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5640b5560 0x7ff5640ceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:36.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.290+0000 7ff5698e1640 1 -- 172.21.15.43:0/3516846299 >> 172.21.15.43:0/3516846299 conn(0x7ff5640d0560 msgr2=0x7ff5640b59c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:36.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.290+0000 7ff5698e1640 1 -- 172.21.15.43:0/3516846299 shutdown_connections 2024-04-25T08:54:36.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.290+0000 7ff5698e1640 1 -- 172.21.15.43:0/3516846299 wait complete. 2024-04-25T08:54:36.291 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:54:36.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.405+0000 7f584ead8640 1 Processor -- start 2024-04-25T08:54:36.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.405+0000 7f584ead8640 1 -- start start 2024-04-25T08:54:36.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.406+0000 7f584ead8640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f58480b40a0 0x7f58480ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:36.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.406+0000 7f584ead8640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58480af0a0 con 0x7f58480aec70 2024-04-25T08:54:36.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.406+0000 7f584ead8640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f584815b110 con 0x7f584815a960 2024-04-25T08:54:36.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.406+0000 7f584ead8640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f584815b3f0 con 0x7f58480b40a0 2024-04-25T08:54:36.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.406+0000 7f584d075640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f58480aec70 0x7f5848070330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:55720/0 (socket says 172.21.15.43:55720) 2024-04-25T08:54:36.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.406+0000 7f584d075640 1 -- 172.21.15.43:0/3020368408 learned_addr learned my addr 172.21.15.43:0/3020368408 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:36.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.406+0000 7f5847fff640 1 --2- 172.21.15.43:0/3020368408 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f58480b40a0 0x7f58480ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:36.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.406+0000 7f58477fe640 1 -- 172.21.15.43:0/3020368408 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2171335682 0 0) 0x7f58480af0a0 con 0x7f58480aec70 2024-04-25T08:54:36.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.406+0000 7f58477fe640 1 -- 172.21.15.43:0/3020368408 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5838003520 con 0x7f58480aec70 2024-04-25T08:54:36.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.406+0000 7f58477fe640 1 -- 172.21.15.43:0/3020368408 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1285799511 0 0) 0x7f584815b110 con 0x7f584815a960 2024-04-25T08:54:36.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.406+0000 7f58477fe640 1 -- 172.21.15.43:0/3020368408 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58480af0a0 con 0x7f584815a960 2024-04-25T08:54:36.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.406+0000 7f5847fff640 1 -- 172.21.15.43:0/3020368408 >> v1:172.21.15.193:6789/0 conn(0x7f584815a960 legacy=0x7f58480b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:36.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.406+0000 7f5847fff640 1 -- 172.21.15.43:0/3020368408 >> v1:172.21.15.43:6789/0 conn(0x7f58480aec70 legacy=0x7f5848070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:36.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.406+0000 7f5847fff640 1 -- 172.21.15.43:0/3020368408 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f584815b6d0 con 0x7f58480b40a0 2024-04-25T08:54:36.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.407+0000 7f5847fff640 1 --2- 172.21.15.43:0/3020368408 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f58480b40a0 0x7f58480ae730 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f58340097d0 tx=0x7f583402a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea9eac660419ca48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:36.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.407+0000 7f58477fe640 1 -- 172.21.15.43:0/3020368408 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5834037070 con 0x7f58480b40a0 2024-04-25T08:54:36.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.407+0000 7f58477fe640 1 -- 172.21.15.43:0/3020368408 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5834032440 con 0x7f58480b40a0 2024-04-25T08:54:36.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.407+0000 7f58477fe640 1 -- 172.21.15.43:0/3020368408 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58340327f0 con 0x7f58480b40a0 2024-04-25T08:54:36.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.408+0000 7f584ead8640 1 -- 172.21.15.43:0/3020368408 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f58480b40a0 msgr2=0x7f58480ae730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:36.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.408+0000 7f584ead8640 1 --2- 172.21.15.43:0/3020368408 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f58480b40a0 0x7f58480ae730 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f58340097d0 tx=0x7f583402a210 comp rx=0 tx=0).stop 2024-04-25T08:54:36.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.408+0000 7f584ead8640 1 -- 172.21.15.43:0/3020368408 shutdown_connections 2024-04-25T08:54:36.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.408+0000 7f584ead8640 1 --2- 172.21.15.43:0/3020368408 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f58480b40a0 0x7f58480ae730 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:36.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.408+0000 7f584ead8640 1 -- 172.21.15.43:0/3020368408 >> 172.21.15.43:0/3020368408 conn(0x7f584806ab30 msgr2=0x7f584806af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:36.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.408+0000 7f584ead8640 1 -- 172.21.15.43:0/3020368408 shutdown_connections 2024-04-25T08:54:36.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.408+0000 7f584ead8640 1 -- 172.21.15.43:0/3020368408 wait complete. 2024-04-25T08:54:36.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.409+0000 7f584ead8640 1 Processor -- start 2024-04-25T08:54:36.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.409+0000 7f584ead8640 1 -- start start 2024-04-25T08:54:36.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.410+0000 7f584ead8640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f58480aec70 0x7f584815dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:36.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.410+0000 7f584ead8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58480b40a0 0x7f584814b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:36.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.410+0000 7f584ead8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f584815a960 0x7f584815c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:36.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.410+0000 7f584ead8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58480cd940 con 0x7f58480b40a0 2024-04-25T08:54:36.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.410+0000 7f584ead8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58480cc440 con 0x7f584815a960 2024-04-25T08:54:36.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.410+0000 7f584ead8640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f584815ad40 con 0x7f58480aec70 2024-04-25T08:54:36.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.410+0000 7f5847fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58480b40a0 0x7f584814b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:36.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.410+0000 7f5847fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58480b40a0 0x7f584814b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40148/0 (socket says 172.21.15.43:40148) 2024-04-25T08:54:36.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.410+0000 7f5847fff640 1 -- 172.21.15.43:0/3212775260 learned_addr learned my addr 172.21.15.43:0/3212775260 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:36.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.410+0000 7f584d075640 1 --2- 172.21.15.43:0/3212775260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f584815a960 0x7f584815c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:36.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.410+0000 7f5847fff640 1 -- 172.21.15.43:0/3212775260 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f58480aec70 msgr2=0x7f584815dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:36.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.410+0000 7f5847fff640 1 --2- 172.21.15.43:0/3212775260 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f58480aec70 0x7f584815dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:36.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.410+0000 7f5847fff640 1 -- 172.21.15.43:0/3212775260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f584815a960 msgr2=0x7f584815c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:36.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.410+0000 7f5847fff640 1 --2- 172.21.15.43:0/3212775260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f584815a960 0x7f584815c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:36.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.410+0000 7f5847fff640 1 -- 172.21.15.43:0/3212775260 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f584815cc40 con 0x7f58480b40a0 2024-04-25T08:54:36.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.411+0000 7f5847fff640 1 --2- 172.21.15.43:0/3212775260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58480b40a0 0x7f584814b170 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f583402a250 tx=0x7f58340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:36.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.411+0000 7f58457fa640 1 -- 172.21.15.43:0/3212775260 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5834041020 con 0x7f58480b40a0 2024-04-25T08:54:36.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.411+0000 7f58457fa640 1 -- 172.21.15.43:0/3212775260 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f583403d6e0 con 0x7f58480b40a0 2024-04-25T08:54:36.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.411+0000 7f58457fa640 1 -- 172.21.15.43:0/3212775260 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5834037040 con 0x7f58480b40a0 2024-04-25T08:54:36.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.411+0000 7f584ead8640 1 -- 172.21.15.43:0/3212775260 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f584815cf20 con 0x7f58480b40a0 2024-04-25T08:54:36.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.411+0000 7f584ead8640 1 -- 172.21.15.43:0/3212775260 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f584815d4e0 con 0x7f58480b40a0 2024-04-25T08:54:36.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.412+0000 7f584ead8640 1 -- 172.21.15.43:0/3212775260 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5848074f60 con 0x7f58480b40a0 2024-04-25T08:54:36.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.413+0000 7f58457fa640 1 -- 172.21.15.43:0/3212775260 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5834044020 con 0x7f58480b40a0 2024-04-25T08:54:36.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.413+0000 7f58457fa640 1 --2- 172.21.15.43:0/3212775260 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f582c06b4d0 0x7f582c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:36.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.413+0000 7f58457fa640 1 -- 172.21.15.43:0/3212775260 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f58340aa970 con 0x7f58480b40a0 2024-04-25T08:54:36.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.413+0000 7f584c874640 1 --2- 172.21.15.43:0/3212775260 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f582c06b4d0 0x7f582c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:36.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.414+0000 7f584c874640 1 --2- 172.21.15.43:0/3212775260 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f582c06b4d0 0x7f582c06d990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f583c0011e0 tx=0x7f583c0078f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:36.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.417+0000 7f58457fa640 1 -- 172.21.15.43:0/3212775260 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f583404b030 con 0x7f58480b40a0 2024-04-25T08:54:36.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.596+0000 7f584ead8640 1 -- 172.21.15.43:0/3212775260 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f58480cbab0 con 0x7f58480b40a0 2024-04-25T08:54:36.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.597+0000 7f58457fa640 1 -- 172.21.15.43:0/3212775260 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5834079800 con 0x7f58480b40a0 2024-04-25T08:54:36.588 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T08:54:36.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.599+0000 7f584ead8640 1 -- 172.21.15.43:0/3212775260 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f582c06b4d0 msgr2=0x7f582c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:36.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.599+0000 7f584ead8640 1 --2- 172.21.15.43:0/3212775260 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f582c06b4d0 0x7f582c06d990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f583c0011e0 tx=0x7f583c0078f0 comp rx=0 tx=0).stop 2024-04-25T08:54:36.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.599+0000 7f584ead8640 1 -- 172.21.15.43:0/3212775260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58480b40a0 msgr2=0x7f584814b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:36.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.599+0000 7f584ead8640 1 --2- 172.21.15.43:0/3212775260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58480b40a0 0x7f584814b170 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f583402a250 tx=0x7f58340046c0 comp rx=0 tx=0).stop 2024-04-25T08:54:36.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.599+0000 7f584ead8640 1 -- 172.21.15.43:0/3212775260 shutdown_connections 2024-04-25T08:54:36.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.599+0000 7f584ead8640 1 --2- 172.21.15.43:0/3212775260 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f582c06b4d0 0x7f582c06d990 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:36.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.599+0000 7f584ead8640 1 --2- 172.21.15.43:0/3212775260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f584815a960 0x7f584815c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:36.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.599+0000 7f584ead8640 1 --2- 172.21.15.43:0/3212775260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58480b40a0 0x7f584814b170 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:36.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.599+0000 7f584ead8640 1 --2- 172.21.15.43:0/3212775260 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f58480aec70 0x7f584815dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:36.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.599+0000 7f584ead8640 1 -- 172.21.15.43:0/3212775260 >> 172.21.15.43:0/3212775260 conn(0x7f584806ab30 msgr2=0x7f58480b0d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:36.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.599+0000 7f584ead8640 1 -- 172.21.15.43:0/3212775260 shutdown_connections 2024-04-25T08:54:36.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.599+0000 7f584ead8640 1 -- 172.21.15.43:0/3212775260 wait complete. 2024-04-25T08:54:36.601 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:54:36.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.712+0000 7f77aa8b3640 1 Processor -- start 2024-04-25T08:54:36.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.712+0000 7f77aa8b3640 1 -- start start 2024-04-25T08:54:36.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.713+0000 7f77aa8b3640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f77a40cd010 0x7f77a40b4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:36.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.713+0000 7f77aa8b3640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77a40b2780 con 0x7f77a40ccbb0 2024-04-25T08:54:36.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.713+0000 7f77aa8b3640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77a40b2a60 con 0x7f77a40cbfc0 2024-04-25T08:54:36.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.713+0000 7f77aa8b3640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77a40b2d40 con 0x7f77a40cd010 2024-04-25T08:54:36.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.713+0000 7f77a37fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f77a40ccbb0 0x7f77a40b3b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:55734/0 (socket says 172.21.15.43:55734) 2024-04-25T08:54:36.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.713+0000 7f77a37fe640 1 -- 172.21.15.43:0/2445193304 learned_addr learned my addr 172.21.15.43:0/2445193304 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:36.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.713+0000 7f77a8e50640 1 --2- 172.21.15.43:0/2445193304 >> v2:172.21.15.193:3300/0 conn(0x7f77a40cd010 0x7f77a40b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:36.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.713+0000 7f77a2ffd640 1 -- 172.21.15.43:0/2445193304 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 456754280 0 0) 0x7f77a40b2780 con 0x7f77a40ccbb0 2024-04-25T08:54:36.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.713+0000 7f77a2ffd640 1 -- 172.21.15.43:0/2445193304 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f778c003520 con 0x7f77a40ccbb0 2024-04-25T08:54:36.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.713+0000 7f77a2ffd640 1 -- 172.21.15.43:0/2445193304 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4149650226 0 0) 0x7f778c003520 con 0x7f77a40ccbb0 2024-04-25T08:54:36.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.713+0000 7f77a2ffd640 1 -- 172.21.15.43:0/2445193304 >> v2:172.21.15.193:3300/0 conn(0x7f77a40cd010 msgr2=0x7f77a40b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:36.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.713+0000 7f77a2ffd640 1 --2- 172.21.15.43:0/2445193304 >> v2:172.21.15.193:3300/0 conn(0x7f77a40cd010 0x7f77a40b4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:36.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.713+0000 7f77a2ffd640 1 -- 172.21.15.43:0/2445193304 >> v1:172.21.15.193:6789/0 conn(0x7f77a40cbfc0 legacy=0x7f77a406dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:36.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.713+0000 7f77a2ffd640 1 -- 172.21.15.43:0/2445193304 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a40b3020 con 0x7f77a40ccbb0 2024-04-25T08:54:36.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.714+0000 7f77a2ffd640 1 -- 172.21.15.43:0/2445193304 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f7794001670 con 0x7f77a40ccbb0 2024-04-25T08:54:36.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.714+0000 7f77a2ffd640 1 -- 172.21.15.43:0/2445193304 >> v1:172.21.15.43:6789/0 conn(0x7f77a40ccbb0 legacy=0x7f77a40b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:36.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.714+0000 7f77a2ffd640 1 --2- 172.21.15.43:0/2445193304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f778c0039b0 0x7f778c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:36.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.714+0000 7f77a2ffd640 1 --2- 172.21.15.43:0/2445193304 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f778c004800 0x7f778c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:36.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.714+0000 7f77a2ffd640 1 --2- 172.21.15.43:0/2445193304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f778c0051c0 0x7f778c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:36.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.714+0000 7f77a2ffd640 1 -- 172.21.15.43:0/2445193304 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a40b2d40 con 0x7f778c0051c0 2024-04-25T08:54:36.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.714+0000 7f77a2ffd640 1 -- 172.21.15.43:0/2445193304 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f778c00c060 con 0x7f778c0039b0 2024-04-25T08:54:36.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.714+0000 7f77a2ffd640 1 -- 172.21.15.43:0/2445193304 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f778c00c260 con 0x7f778c004800 2024-04-25T08:54:36.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.714+0000 7f77a8e50640 1 --2- 172.21.15.43:0/2445193304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f778c0051c0 0x7f778c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:36.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.714+0000 7f77a3fff640 1 --2- 172.21.15.43:0/2445193304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f778c0039b0 0x7f778c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:36.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.714+0000 7f77a37fe640 1 --2- 172.21.15.43:0/2445193304 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f778c004800 0x7f778c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:36.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.714+0000 7f77a8e50640 1 -- 172.21.15.43:0/2445193304 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f778c004800 msgr2=0x7f778c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:36.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.714+0000 7f77a8e50640 1 --2- 172.21.15.43:0/2445193304 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f778c004800 0x7f778c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:36.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.714+0000 7f77a8e50640 1 -- 172.21.15.43:0/2445193304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f778c0039b0 msgr2=0x7f778c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:36.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.714+0000 7f77a8e50640 1 --2- 172.21.15.43:0/2445193304 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f778c0039b0 0x7f778c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:36.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.714+0000 7f77a8e50640 1 -- 172.21.15.43:0/2445193304 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f778c003520 con 0x7f778c0051c0 2024-04-25T08:54:36.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.715+0000 7f77a8e50640 1 -- 172.21.15.43:0/2445193304 reap_dead start 2024-04-25T08:54:36.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.715+0000 7f77a8e50640 1 --2- 172.21.15.43:0/2445193304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f778c0051c0 0x7f778c00b9c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f779c009fc0 tx=0x7f778c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ade72d439116325 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:36.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.715+0000 7f77a2ffd640 1 -- 172.21.15.43:0/2445193304 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779c035020 con 0x7f778c0051c0 2024-04-25T08:54:36.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.715+0000 7f77a2ffd640 1 -- 172.21.15.43:0/2445193304 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f779c003b90 con 0x7f778c0051c0 2024-04-25T08:54:36.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.715+0000 7f77a2ffd640 1 -- 172.21.15.43:0/2445193304 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779c009c10 con 0x7f778c0051c0 2024-04-25T08:54:36.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.715+0000 7f77aa8b3640 1 -- 172.21.15.43:0/2445193304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f778c0051c0 msgr2=0x7f778c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:36.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.715+0000 7f77aa8b3640 1 --2- 172.21.15.43:0/2445193304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f778c0051c0 0x7f778c00b9c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f779c009fc0 tx=0x7f778c005830 comp rx=0 tx=0).stop 2024-04-25T08:54:36.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.716+0000 7f77aa8b3640 1 -- 172.21.15.43:0/2445193304 shutdown_connections 2024-04-25T08:54:36.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.716+0000 7f77aa8b3640 1 --2- 172.21.15.43:0/2445193304 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f778c0051c0 0x7f778c00b9c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:36.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.716+0000 7f77aa8b3640 1 -- 172.21.15.43:0/2445193304 >> 172.21.15.43:0/2445193304 conn(0x7f77a40b1480 msgr2=0x7f77a40b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:36.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.716+0000 7f77aa8b3640 1 -- 172.21.15.43:0/2445193304 shutdown_connections 2024-04-25T08:54:36.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.716+0000 7f77aa8b3640 1 -- 172.21.15.43:0/2445193304 wait complete. 2024-04-25T08:54:36.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.717+0000 7f77aa8b3640 1 Processor -- start 2024-04-25T08:54:36.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.717+0000 7f77aa8b3640 1 -- start start 2024-04-25T08:54:36.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.718+0000 7f77aa8b3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f778c0051c0 0x7f77a4151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:36.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.718+0000 7f77aa8b3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77a413e790 0x7f77a4140770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:36.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.718+0000 7f77aa8b3640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77a40a94d0 0x7f77a40a9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:36.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.718+0000 7f77aa8b3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a40cd480 con 0x7f77a413e790 2024-04-25T08:54:36.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.718+0000 7f77aa8b3640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a40b35e0 con 0x7f778c0051c0 2024-04-25T08:54:36.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.718+0000 7f77aa8b3640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a40cc3a0 con 0x7f77a40a94d0 2024-04-25T08:54:36.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.718+0000 7f77a37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77a413e790 0x7f77a4140770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:36.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.718+0000 7f77a37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77a413e790 0x7f77a4140770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40160/0 (socket says 172.21.15.43:40160) 2024-04-25T08:54:36.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.718+0000 7f77a37fe640 1 -- 172.21.15.43:0/495626262 learned_addr learned my addr 172.21.15.43:0/495626262 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:36.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.718+0000 7f77a3fff640 1 --2- 172.21.15.43:0/495626262 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f778c0051c0 0x7f77a4151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:36.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.718+0000 7f77a8e50640 1 --2- 172.21.15.43:0/495626262 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77a40a94d0 0x7f77a40a9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:36.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.718+0000 7f77a37fe640 1 -- 172.21.15.43:0/495626262 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77a40a94d0 msgr2=0x7f77a40a9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:36.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.718+0000 7f77a37fe640 1 --2- 172.21.15.43:0/495626262 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77a40a94d0 0x7f77a40a9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:36.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.718+0000 7f77a37fe640 1 -- 172.21.15.43:0/495626262 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f778c0051c0 msgr2=0x7f77a4151d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:36.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.718+0000 7f77a37fe640 1 --2- 172.21.15.43:0/495626262 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f778c0051c0 0x7f77a4151d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:36.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.718+0000 7f77a37fe640 1 -- 172.21.15.43:0/495626262 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a41410e0 con 0x7f77a413e790 2024-04-25T08:54:36.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.718+0000 7f77a8e50640 1 --2- 172.21.15.43:0/495626262 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77a40a94d0 0x7f77a40a9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:36.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.718+0000 7f77a37fe640 1 --2- 172.21.15.43:0/495626262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77a413e790 0x7f77a4140770 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7794004ef0 tx=0x7f7794007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:36.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.718+0000 7f77a0ff9640 1 -- 172.21.15.43:0/495626262 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779400f750 con 0x7f77a413e790 2024-04-25T08:54:36.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.718+0000 7f77a0ff9640 1 -- 172.21.15.43:0/495626262 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7794018070 con 0x7f77a413e790 2024-04-25T08:54:36.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.718+0000 7f77a0ff9640 1 -- 172.21.15.43:0/495626262 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f779400fb10 con 0x7f77a413e790 2024-04-25T08:54:36.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.718+0000 7f77aa8b3640 1 -- 172.21.15.43:0/495626262 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77a41413c0 con 0x7f77a413e790 2024-04-25T08:54:36.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.719+0000 7f77aa8b3640 1 -- 172.21.15.43:0/495626262 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77a4141980 con 0x7f77a413e790 2024-04-25T08:54:36.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.720+0000 7f77aa8b3640 1 -- 172.21.15.43:0/495626262 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77a4074ef0 con 0x7f77a413e790 2024-04-25T08:54:36.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.720+0000 7f77a0ff9640 1 -- 172.21.15.43:0/495626262 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f779400fcb0 con 0x7f77a413e790 2024-04-25T08:54:36.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.720+0000 7f77a0ff9640 1 --2- 172.21.15.43:0/495626262 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f779006b500 0x7f779006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:36.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.721+0000 7f77a0ff9640 1 -- 172.21.15.43:0/495626262 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f779408a100 con 0x7f77a413e790 2024-04-25T08:54:36.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.721+0000 7f77a3fff640 1 --2- 172.21.15.43:0/495626262 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f779006b500 0x7f779006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:36.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.721+0000 7f77a3fff640 1 --2- 172.21.15.43:0/495626262 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f779006b500 0x7f779006d9c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f77a413ef80 tx=0x7f7798009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:36.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.725+0000 7f77a0ff9640 1 -- 172.21.15.43:0/495626262 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7794058f80 con 0x7f77a413e790 2024-04-25T08:54:36.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.905+0000 7f77aa8b3640 1 -- 172.21.15.43:0/495626262 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f77a4144ac0 con 0x7f77a413e790 2024-04-25T08:54:36.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.906+0000 7f77a0ff9640 1 -- 172.21.15.43:0/495626262 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16569 (secure 0 0 0) 0x7f7794058880 con 0x7f77a413e790 2024-04-25T08:54:36.897 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:54:36.897 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":20,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:54:13.593188+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:54:09.757978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:54:10.926008+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:54:36.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.908+0000 7f77aa8b3640 1 -- 172.21.15.43:0/495626262 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f779006b500 msgr2=0x7f779006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:36.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.908+0000 7f77aa8b3640 1 --2- 172.21.15.43:0/495626262 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f779006b500 0x7f779006d9c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f77a413ef80 tx=0x7f7798009040 comp rx=0 tx=0).stop 2024-04-25T08:54:36.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.908+0000 7f77aa8b3640 1 -- 172.21.15.43:0/495626262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77a413e790 msgr2=0x7f77a4140770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:36.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.908+0000 7f77aa8b3640 1 --2- 172.21.15.43:0/495626262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77a413e790 0x7f77a4140770 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7794004ef0 tx=0x7f7794007430 comp rx=0 tx=0).stop 2024-04-25T08:54:36.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.908+0000 7f77aa8b3640 1 -- 172.21.15.43:0/495626262 shutdown_connections 2024-04-25T08:54:36.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.908+0000 7f77aa8b3640 1 --2- 172.21.15.43:0/495626262 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77a40a94d0 0x7f77a40a9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:36.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.908+0000 7f77aa8b3640 1 --2- 172.21.15.43:0/495626262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77a413e790 0x7f77a4140770 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:36.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.908+0000 7f77aa8b3640 1 --2- 172.21.15.43:0/495626262 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f779006b500 0x7f779006d9c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:36.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.908+0000 7f77aa8b3640 1 --2- 172.21.15.43:0/495626262 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f778c0051c0 0x7f77a4151d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:36.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.908+0000 7f77aa8b3640 1 -- 172.21.15.43:0/495626262 >> 172.21.15.43:0/495626262 conn(0x7f77a40b1480 msgr2=0x7f77a406d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:36.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.908+0000 7f77aa8b3640 1 -- 172.21.15.43:0/495626262 shutdown_connections 2024-04-25T08:54:36.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:36.908+0000 7f77aa8b3640 1 -- 172.21.15.43:0/495626262 wait complete. 2024-04-25T08:54:36.909 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:54:09.757978+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:54:10.926008+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:54:36.910 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:54:37.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.023+0000 7f053b3b0640 1 Processor -- start 2024-04-25T08:54:37.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.023+0000 7f053b3b0640 1 -- start start 2024-04-25T08:54:37.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.024+0000 7f053b3b0640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f053415e090 0x7f0534161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:37.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.024+0000 7f053b3b0640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0534161fe0 0x7f05340b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:37.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.024+0000 7f053b3b0640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0534162610 con 0x7f05340b5aa0 2024-04-25T08:54:37.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.024+0000 7f053b3b0640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05340d1da0 con 0x7f053415e090 2024-04-25T08:54:37.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.024+0000 7f053b3b0640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05340d1750 con 0x7f0534161fe0 2024-04-25T08:54:37.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.024+0000 7f053914c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f053415e090 0x7f0534161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:37.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.024+0000 7f053914c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f053415e090 0x7f0534161aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40164/0 (socket says 172.21.15.43:40164) 2024-04-25T08:54:37.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.024+0000 7f053994d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f05340b5aa0 0x7f05340ce560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:55742/0 (socket says 172.21.15.43:55742) 2024-04-25T08:54:37.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.024+0000 7f053914c640 1 -- 172.21.15.43:0/3819281397 learned_addr learned my addr 172.21.15.43:0/3819281397 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:37.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.024+0000 7f053894b640 1 --2- 172.21.15.43:0/3819281397 >> v2:172.21.15.193:3300/0 conn(0x7f0534161fe0 0x7f05340b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:37.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.024+0000 7f053914c640 1 -- 172.21.15.43:0/3819281397 >> v2:172.21.15.193:3300/0 conn(0x7f0534161fe0 msgr2=0x7f05340b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:37.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.024+0000 7f053914c640 1 --2- 172.21.15.43:0/3819281397 >> v2:172.21.15.193:3300/0 conn(0x7f0534161fe0 0x7f05340b5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.024+0000 7f053914c640 1 -- 172.21.15.43:0/3819281397 >> v1:172.21.15.43:6789/0 conn(0x7f05340b5aa0 legacy=0x7f05340ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:37.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.024+0000 7f053914c640 1 -- 172.21.15.43:0/3819281397 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05340d1a30 con 0x7f053415e090 2024-04-25T08:54:37.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.024+0000 7f0533fff640 1 -- 172.21.15.43:0/3819281397 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 758548923 0 0) 0x7f0534162610 con 0x7f05340b5aa0 2024-04-25T08:54:37.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.024+0000 7f053894b640 1 --2- 172.21.15.43:0/3819281397 >> v2:172.21.15.193:3300/0 conn(0x7f0534161fe0 0x7f05340b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:37.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.025+0000 7f053914c640 1 --2- 172.21.15.43:0/3819281397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f053415e090 0x7f0534161aa0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f05340cfe10 tx=0x7f052c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d042057dfbfb709 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:37.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.025+0000 7f0533fff640 1 -- 172.21.15.43:0/3819281397 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c037070 con 0x7f053415e090 2024-04-25T08:54:37.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.025+0000 7f0533fff640 1 -- 172.21.15.43:0/3819281397 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f052c02ae50 con 0x7f053415e090 2024-04-25T08:54:37.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.025+0000 7f0533fff640 1 -- 172.21.15.43:0/3819281397 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c032860 con 0x7f053415e090 2024-04-25T08:54:37.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.026+0000 7f053b3b0640 1 -- 172.21.15.43:0/3819281397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f053415e090 msgr2=0x7f0534161aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:37.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.026+0000 7f053b3b0640 1 --2- 172.21.15.43:0/3819281397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f053415e090 0x7f0534161aa0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f05340cfe10 tx=0x7f052c02a300 comp rx=0 tx=0).stop 2024-04-25T08:54:37.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.026+0000 7f053b3b0640 1 -- 172.21.15.43:0/3819281397 shutdown_connections 2024-04-25T08:54:37.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.026+0000 7f053b3b0640 1 --2- 172.21.15.43:0/3819281397 >> v2:172.21.15.193:3300/0 conn(0x7f0534161fe0 0x7f05340b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.026+0000 7f053b3b0640 1 --2- 172.21.15.43:0/3819281397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f053415e090 0x7f0534161aa0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.026+0000 7f053b3b0640 1 -- 172.21.15.43:0/3819281397 >> 172.21.15.43:0/3819281397 conn(0x7f05340d0560 msgr2=0x7f05340d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:37.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.026+0000 7f053b3b0640 1 -- 172.21.15.43:0/3819281397 shutdown_connections 2024-04-25T08:54:37.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.026+0000 7f053b3b0640 1 -- 172.21.15.43:0/3819281397 wait complete. 2024-04-25T08:54:37.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.027+0000 7f053b3b0640 1 Processor -- start 2024-04-25T08:54:37.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.027+0000 7f053b3b0640 1 -- start start 2024-04-25T08:54:37.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.028+0000 7f053b3b0640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f05340b5aa0 0x7f05340b70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:37.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.028+0000 7f053b3b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f053415e090 0x7f0534154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:37.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.028+0000 7f053b3b0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0534161fe0 0x7f0534155130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:37.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.028+0000 7f053b3b0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05340d3240 con 0x7f053415e090 2024-04-25T08:54:37.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.028+0000 7f053b3b0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05340b3280 con 0x7f0534161fe0 2024-04-25T08:54:37.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.028+0000 7f053b3b0640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05340d33c0 con 0x7f05340b5aa0 2024-04-25T08:54:37.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.028+0000 7f053894b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f053415e090 0x7f0534154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:37.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.028+0000 7f053894b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f053415e090 0x7f0534154bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40166/0 (socket says 172.21.15.43:40166) 2024-04-25T08:54:37.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.028+0000 7f053894b640 1 -- 172.21.15.43:0/524940257 learned_addr learned my addr 172.21.15.43:0/524940257 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:37.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.028+0000 7f053994d640 1 --2- 172.21.15.43:0/524940257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0534161fe0 0x7f0534155130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:37.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.028+0000 7f053914c640 1 --2- 172.21.15.43:0/524940257 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f05340b5aa0 0x7f05340b70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:37.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.028+0000 7f053894b640 1 -- 172.21.15.43:0/524940257 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f05340b5aa0 msgr2=0x7f05340b70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:37.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.028+0000 7f053894b640 1 --2- 172.21.15.43:0/524940257 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f05340b5aa0 0x7f05340b70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.028+0000 7f053894b640 1 -- 172.21.15.43:0/524940257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0534161fe0 msgr2=0x7f0534155130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:37.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.028+0000 7f053894b640 1 --2- 172.21.15.43:0/524940257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0534161fe0 0x7f0534155130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.028+0000 7f053894b640 1 -- 172.21.15.43:0/524940257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05341f4330 con 0x7f053415e090 2024-04-25T08:54:37.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.029+0000 7f053894b640 1 --2- 172.21.15.43:0/524940257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f053415e090 0x7f0534154bf0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f052800d800 tx=0x7f052800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:37.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.029+0000 7f0531ffb640 1 -- 172.21.15.43:0/524940257 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0528004030 con 0x7f053415e090 2024-04-25T08:54:37.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.029+0000 7f0531ffb640 1 -- 172.21.15.43:0/524940257 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05280041d0 con 0x7f053415e090 2024-04-25T08:54:37.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.029+0000 7f053b3b0640 1 -- 172.21.15.43:0/524940257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05341f6af0 con 0x7f053415e090 2024-04-25T08:54:37.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.029+0000 7f0531ffb640 1 -- 172.21.15.43:0/524940257 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05280096a0 con 0x7f053415e090 2024-04-25T08:54:37.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.029+0000 7f053b3b0640 1 -- 172.21.15.43:0/524940257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05341f70b0 con 0x7f053415e090 2024-04-25T08:54:37.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.030+0000 7f053b3b0640 1 -- 172.21.15.43:0/524940257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f053406bf00 con 0x7f053415e090 2024-04-25T08:54:37.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.030+0000 7f0531ffb640 1 -- 172.21.15.43:0/524940257 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0528009840 con 0x7f053415e090 2024-04-25T08:54:37.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.031+0000 7f0531ffb640 1 --2- 172.21.15.43:0/524940257 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f052006b4d0 0x7f052006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:37.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.031+0000 7f0531ffb640 1 -- 172.21.15.43:0/524940257 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f052808a500 con 0x7f053415e090 2024-04-25T08:54:37.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.031+0000 7f053914c640 1 --2- 172.21.15.43:0/524940257 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f052006b4d0 0x7f052006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:37.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.032+0000 7f053914c640 1 --2- 172.21.15.43:0/524940257 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f052006b4d0 0x7f052006d990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f052c034000 tx=0x7f052c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:37.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.035+0000 7f0531ffb640 1 -- 172.21.15.43:0/524940257 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f053406bf00 con 0x7f053415e090 2024-04-25T08:54:37.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.214+0000 7f053b3b0640 1 -- 172.21.15.43:0/524940257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0534074f60 con 0x7f053415e090 2024-04-25T08:54:37.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.214+0000 7f0531ffb640 1 -- 172.21.15.43:0/524940257 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0528059380 con 0x7f053415e090 2024-04-25T08:54:37.204 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T08:54:37.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.216+0000 7f053b3b0640 1 -- 172.21.15.43:0/524940257 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f052006b4d0 msgr2=0x7f052006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:37.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.216+0000 7f053b3b0640 1 --2- 172.21.15.43:0/524940257 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f052006b4d0 0x7f052006d990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f052c034000 tx=0x7f052c02e000 comp rx=0 tx=0).stop 2024-04-25T08:54:37.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.216+0000 7f053b3b0640 1 -- 172.21.15.43:0/524940257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f053415e090 msgr2=0x7f0534154bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:37.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.216+0000 7f053b3b0640 1 --2- 172.21.15.43:0/524940257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f053415e090 0x7f0534154bf0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f052800d800 tx=0x7f052800dcc0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.216+0000 7f053b3b0640 1 -- 172.21.15.43:0/524940257 shutdown_connections 2024-04-25T08:54:37.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.216+0000 7f053b3b0640 1 --2- 172.21.15.43:0/524940257 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f052006b4d0 0x7f052006d990 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.216+0000 7f053b3b0640 1 --2- 172.21.15.43:0/524940257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0534161fe0 0x7f0534155130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.216+0000 7f053b3b0640 1 --2- 172.21.15.43:0/524940257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f053415e090 0x7f0534154bf0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.216+0000 7f053b3b0640 1 --2- 172.21.15.43:0/524940257 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f05340b5aa0 0x7f05340b70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.216+0000 7f053b3b0640 1 -- 172.21.15.43:0/524940257 >> 172.21.15.43:0/524940257 conn(0x7f05340d0560 msgr2=0x7f05340cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:37.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.216+0000 7f053b3b0640 1 -- 172.21.15.43:0/524940257 shutdown_connections 2024-04-25T08:54:37.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.216+0000 7f053b3b0640 1 -- 172.21.15.43:0/524940257 wait complete. 2024-04-25T08:54:37.217 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:54:37.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.330+0000 7f8fa77f4640 1 Processor -- start 2024-04-25T08:54:37.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.330+0000 7f8fa77f4640 1 -- start start 2024-04-25T08:54:37.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.331+0000 7f8fa77f4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8fa00b59e0 0x7f8fa00d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:37.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.331+0000 7f8fa77f4640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fa00b5e70 con 0x7f8fa015e090 2024-04-25T08:54:37.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.331+0000 7f8fa77f4640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fa0162610 con 0x7f8fa00b5560 2024-04-25T08:54:37.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.331+0000 7f8fa77f4640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fa00d1da0 con 0x7f8fa00b59e0 2024-04-25T08:54:37.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.331+0000 7f8fa5590640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8fa015e090 0x7f8fa0161aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:55744/0 (socket says 172.21.15.43:55744) 2024-04-25T08:54:37.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.331+0000 7f8fa5590640 1 -- 172.21.15.43:0/657146075 learned_addr learned my addr 172.21.15.43:0/657146075 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:37.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.331+0000 7f8fa5d91640 1 --2- 172.21.15.43:0/657146075 >> v2:172.21.15.43:3300/0 conn(0x7f8fa00b59e0 0x7f8fa00d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:37.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.332+0000 7f8f9ffff640 1 -- 172.21.15.43:0/657146075 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4285842891 0 0) 0x7f8fa00b5e70 con 0x7f8fa015e090 2024-04-25T08:54:37.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.332+0000 7f8f9ffff640 1 -- 172.21.15.43:0/657146075 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f88003520 con 0x7f8fa015e090 2024-04-25T08:54:37.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.332+0000 7f8fa5d91640 1 -- 172.21.15.43:0/657146075 >> v1:172.21.15.193:6789/0 conn(0x7f8fa00b5560 legacy=0x7f8fa00d2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:37.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.332+0000 7f8fa5d91640 1 -- 172.21.15.43:0/657146075 >> v1:172.21.15.43:6789/0 conn(0x7f8fa015e090 legacy=0x7f8fa0161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:37.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.332+0000 7f8fa5d91640 1 -- 172.21.15.43:0/657146075 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa00d1750 con 0x7f8fa00b59e0 2024-04-25T08:54:37.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.332+0000 7f8fa5d91640 1 --2- 172.21.15.43:0/657146075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fa00b59e0 0x7f8fa00d2940 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8f90009d90 tx=0x7f8f9002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a3f4d479ad9a265 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:37.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.332+0000 7f8f9ffff640 1 -- 172.21.15.43:0/657146075 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f90007740 con 0x7f8fa00b59e0 2024-04-25T08:54:37.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.333+0000 7f8f9ffff640 1 -- 172.21.15.43:0/657146075 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f900078e0 con 0x7f8fa00b59e0 2024-04-25T08:54:37.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.333+0000 7f8f9ffff640 1 -- 172.21.15.43:0/657146075 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f90007cb0 con 0x7f8fa00b59e0 2024-04-25T08:54:37.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.333+0000 7f8fa77f4640 1 -- 172.21.15.43:0/657146075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fa00b59e0 msgr2=0x7f8fa00d2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:37.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.333+0000 7f8fa77f4640 1 --2- 172.21.15.43:0/657146075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fa00b59e0 0x7f8fa00d2940 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8f90009d90 tx=0x7f8f9002a730 comp rx=0 tx=0).stop 2024-04-25T08:54:37.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.333+0000 7f8fa77f4640 1 -- 172.21.15.43:0/657146075 shutdown_connections 2024-04-25T08:54:37.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.333+0000 7f8fa77f4640 1 --2- 172.21.15.43:0/657146075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fa00b59e0 0x7f8fa00d2940 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.333+0000 7f8fa77f4640 1 -- 172.21.15.43:0/657146075 >> 172.21.15.43:0/657146075 conn(0x7f8fa00d0560 msgr2=0x7f8fa00d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:37.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.333+0000 7f8fa77f4640 1 -- 172.21.15.43:0/657146075 shutdown_connections 2024-04-25T08:54:37.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.333+0000 7f8fa77f4640 1 -- 172.21.15.43:0/657146075 wait complete. 2024-04-25T08:54:37.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.334+0000 7f8fa77f4640 1 Processor -- start 2024-04-25T08:54:37.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.334+0000 7f8fa77f4640 1 -- start start 2024-04-25T08:54:37.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.335+0000 7f8fa77f4640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8fa00b5560 0x7f8fa015f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:37.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.335+0000 7f8fa77f4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fa00b59e0 0x7f8fa00c6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:37.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.335+0000 7f8fa77f4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fa015e090 0x7f8fa00c6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:37.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.335+0000 7f8fa77f4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa00b3280 con 0x7f8fa00b59e0 2024-04-25T08:54:37.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.335+0000 7f8fa77f4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa01628f0 con 0x7f8fa015e090 2024-04-25T08:54:37.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.335+0000 7f8fa77f4640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa01621b0 con 0x7f8fa00b5560 2024-04-25T08:54:37.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.335+0000 7f8fa4d8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fa00b59e0 0x7f8fa00c6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:37.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.335+0000 7f8fa4d8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fa00b59e0 0x7f8fa00c6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40184/0 (socket says 172.21.15.43:40184) 2024-04-25T08:54:37.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.335+0000 7f8fa4d8f640 1 -- 172.21.15.43:0/2520121991 learned_addr learned my addr 172.21.15.43:0/2520121991 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:37.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.335+0000 7f8fa5d91640 1 --2- 172.21.15.43:0/2520121991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fa015e090 0x7f8fa00c6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:37.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.335+0000 7f8fa5590640 1 --2- 172.21.15.43:0/2520121991 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8fa00b5560 0x7f8fa015f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:37.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.335+0000 7f8fa4d8f640 1 -- 172.21.15.43:0/2520121991 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8fa00b5560 msgr2=0x7f8fa015f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:37.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.335+0000 7f8fa4d8f640 1 --2- 172.21.15.43:0/2520121991 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8fa00b5560 0x7f8fa015f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.335+0000 7f8fa4d8f640 1 -- 172.21.15.43:0/2520121991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fa015e090 msgr2=0x7f8fa00c6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:37.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.335+0000 7f8fa4d8f640 1 --2- 172.21.15.43:0/2520121991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fa015e090 0x7f8fa00c6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.335+0000 7f8fa4d8f640 1 -- 172.21.15.43:0/2520121991 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa015e610 con 0x7f8fa00b59e0 2024-04-25T08:54:37.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.335+0000 7f8fa5d91640 1 --2- 172.21.15.43:0/2520121991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fa015e090 0x7f8fa00c6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:37.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.336+0000 7f8fa4d8f640 1 --2- 172.21.15.43:0/2520121991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fa00b59e0 0x7f8fa00c6970 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8f9400c430 tx=0x7f8f9400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:37.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.336+0000 7f8f9dffb640 1 -- 172.21.15.43:0/2520121991 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f94011790 con 0x7f8fa00b59e0 2024-04-25T08:54:37.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.336+0000 7f8f9dffb640 1 -- 172.21.15.43:0/2520121991 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f94017070 con 0x7f8fa00b59e0 2024-04-25T08:54:37.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.336+0000 7f8f9dffb640 1 -- 172.21.15.43:0/2520121991 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f94011b70 con 0x7f8fa00b59e0 2024-04-25T08:54:37.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.336+0000 7f8fa77f4640 1 -- 172.21.15.43:0/2520121991 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fa015e8f0 con 0x7f8fa00b59e0 2024-04-25T08:54:37.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.336+0000 7f8fa77f4640 1 -- 172.21.15.43:0/2520121991 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fa015eeb0 con 0x7f8fa00b59e0 2024-04-25T08:54:37.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.337+0000 7f8fa77f4640 1 -- 172.21.15.43:0/2520121991 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fa006bf00 con 0x7f8fa00b59e0 2024-04-25T08:54:37.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.337+0000 7f8f9dffb640 1 -- 172.21.15.43:0/2520121991 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8f94011d10 con 0x7f8fa00b59e0 2024-04-25T08:54:37.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.338+0000 7f8f9dffb640 1 --2- 172.21.15.43:0/2520121991 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8f8c06b4d0 0x7f8f8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:37.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.338+0000 7f8f9dffb640 1 -- 172.21.15.43:0/2520121991 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8f9408a090 con 0x7f8fa00b59e0 2024-04-25T08:54:37.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.338+0000 7f8fa5590640 1 --2- 172.21.15.43:0/2520121991 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8f8c06b4d0 0x7f8f8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:37.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.339+0000 7f8fa5590640 1 --2- 172.21.15.43:0/2520121991 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8f8c06b4d0 0x7f8f8c06d990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8f98000f60 tx=0x7f8f98009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:37.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.342+0000 7f8f9dffb640 1 -- 172.21.15.43:0/2520121991 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8fa006bf00 con 0x7f8fa00b59e0 2024-04-25T08:54:37.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.521+0000 7f8fa77f4640 1 -- 172.21.15.43:0/2520121991 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8fa0074f60 con 0x7f8fa00b59e0 2024-04-25T08:54:37.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.521+0000 7f8f9dffb640 1 -- 172.21.15.43:0/2520121991 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8f94058f10 con 0x7f8fa00b59e0 2024-04-25T08:54:37.511 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T08:54:37.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.523+0000 7f8fa77f4640 1 -- 172.21.15.43:0/2520121991 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8f8c06b4d0 msgr2=0x7f8f8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:37.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.523+0000 7f8fa77f4640 1 --2- 172.21.15.43:0/2520121991 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8f8c06b4d0 0x7f8f8c06d990 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8f98000f60 tx=0x7f8f98009040 comp rx=0 tx=0).stop 2024-04-25T08:54:37.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.523+0000 7f8fa77f4640 1 -- 172.21.15.43:0/2520121991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fa00b59e0 msgr2=0x7f8fa00c6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:37.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.523+0000 7f8fa77f4640 1 --2- 172.21.15.43:0/2520121991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fa00b59e0 0x7f8fa00c6970 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8f9400c430 tx=0x7f8f9400c8f0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.523+0000 7f8fa77f4640 1 -- 172.21.15.43:0/2520121991 shutdown_connections 2024-04-25T08:54:37.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.523+0000 7f8fa77f4640 1 --2- 172.21.15.43:0/2520121991 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8f8c06b4d0 0x7f8f8c06d990 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.523+0000 7f8fa77f4640 1 --2- 172.21.15.43:0/2520121991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8fa015e090 0x7f8fa00c6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.523+0000 7f8fa77f4640 1 --2- 172.21.15.43:0/2520121991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8fa00b59e0 0x7f8fa00c6970 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.523+0000 7f8fa77f4640 1 --2- 172.21.15.43:0/2520121991 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8fa00b5560 0x7f8fa015f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.523+0000 7f8fa77f4640 1 -- 172.21.15.43:0/2520121991 >> 172.21.15.43:0/2520121991 conn(0x7f8fa00d0560 msgr2=0x7f8fa00d2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:37.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.523+0000 7f8fa77f4640 1 -- 172.21.15.43:0/2520121991 shutdown_connections 2024-04-25T08:54:37.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.523+0000 7f8fa77f4640 1 -- 172.21.15.43:0/2520121991 wait complete. 2024-04-25T08:54:37.524 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:54:37.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.638+0000 7f39a29f4640 1 Processor -- start 2024-04-25T08:54:37.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.638+0000 7f39a29f4640 1 -- start start 2024-04-25T08:54:37.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.639+0000 7f39a29f4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f399c15e090 0x7f399c161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:37.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.639+0000 7f39a29f4640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f399c161fe0 0x7f399c0b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:37.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.639+0000 7f39a29f4640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f399c0b5aa0 0x7f399c0ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:37.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.639+0000 7f39a29f4640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f399c0d1da0 con 0x7f399c15e090 2024-04-25T08:54:37.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.639+0000 7f39a29f4640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f399c162610 con 0x7f399c161fe0 2024-04-25T08:54:37.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.639+0000 7f39a29f4640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f399c0d1750 con 0x7f399c0b5aa0 2024-04-25T08:54:37.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.639+0000 7f399bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f399c15e090 0x7f399c161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:37.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.639+0000 7f399bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f399c15e090 0x7f399c161aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40200/0 (socket says 172.21.15.43:40200) 2024-04-25T08:54:37.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.639+0000 7f399bfff640 1 -- 172.21.15.43:0/634725809 learned_addr learned my addr 172.21.15.43:0/634725809 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:37.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.639+0000 7f399b7fe640 1 --2- 172.21.15.43:0/634725809 >> v2:172.21.15.193:3300/0 conn(0x7f399c161fe0 0x7f399c0b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:37.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.639+0000 7f39a0f91640 1 --2- 172.21.15.43:0/634725809 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f399c0b5aa0 0x7f399c0ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:37.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.640+0000 7f399bfff640 1 -- 172.21.15.43:0/634725809 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f399c0b5aa0 msgr2=0x7f399c0ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:37.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.640+0000 7f399bfff640 1 --2- 172.21.15.43:0/634725809 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f399c0b5aa0 0x7f399c0ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.640+0000 7f399bfff640 1 -- 172.21.15.43:0/634725809 >> v2:172.21.15.193:3300/0 conn(0x7f399c161fe0 msgr2=0x7f399c0b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:37.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.640+0000 7f399bfff640 1 --2- 172.21.15.43:0/634725809 >> v2:172.21.15.193:3300/0 conn(0x7f399c161fe0 0x7f399c0b5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.640+0000 7f399bfff640 1 -- 172.21.15.43:0/634725809 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f399c0d1a30 con 0x7f399c15e090 2024-04-25T08:54:37.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.640+0000 7f39a0f91640 1 --2- 172.21.15.43:0/634725809 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f399c0b5aa0 0x7f399c0ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:37.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.640+0000 7f399bfff640 1 --2- 172.21.15.43:0/634725809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f399c15e090 0x7f399c161aa0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f399c0cfe10 tx=0x7f398402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2556ff3a4fc11e5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:37.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.640+0000 7f399affd640 1 -- 172.21.15.43:0/634725809 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3984037070 con 0x7f399c15e090 2024-04-25T08:54:37.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.640+0000 7f399affd640 1 -- 172.21.15.43:0/634725809 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3984032440 con 0x7f399c15e090 2024-04-25T08:54:37.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.640+0000 7f399affd640 1 -- 172.21.15.43:0/634725809 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3984032810 con 0x7f399c15e090 2024-04-25T08:54:37.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.641+0000 7f39a29f4640 1 -- 172.21.15.43:0/634725809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f399c15e090 msgr2=0x7f399c161aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:37.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.641+0000 7f39a29f4640 1 --2- 172.21.15.43:0/634725809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f399c15e090 0x7f399c161aa0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f399c0cfe10 tx=0x7f398402a270 comp rx=0 tx=0).stop 2024-04-25T08:54:37.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.641+0000 7f39a29f4640 1 -- 172.21.15.43:0/634725809 shutdown_connections 2024-04-25T08:54:37.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.641+0000 7f39a29f4640 1 --2- 172.21.15.43:0/634725809 >> v2:172.21.15.193:3300/0 conn(0x7f399c161fe0 0x7f399c0b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.641+0000 7f39a29f4640 1 --2- 172.21.15.43:0/634725809 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f399c0b5aa0 0x7f399c0ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.641+0000 7f39a29f4640 1 --2- 172.21.15.43:0/634725809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f399c15e090 0x7f399c161aa0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.641+0000 7f39a29f4640 1 -- 172.21.15.43:0/634725809 >> 172.21.15.43:0/634725809 conn(0x7f399c0d0560 msgr2=0x7f399c0d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:37.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.641+0000 7f39a29f4640 1 -- 172.21.15.43:0/634725809 shutdown_connections 2024-04-25T08:54:37.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.641+0000 7f39a29f4640 1 -- 172.21.15.43:0/634725809 wait complete. 2024-04-25T08:54:37.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.642+0000 7f39a29f4640 1 Processor -- start 2024-04-25T08:54:37.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.642+0000 7f39a29f4640 1 -- start start 2024-04-25T08:54:37.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.643+0000 7f39a29f4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f399c0b5aa0 0x7f399c155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:37.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.643+0000 7f39a29f4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f399c15e090 0x7f399c0cba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:37.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.643+0000 7f39a29f4640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f399c161fe0 0x7f399c1f39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:37.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.643+0000 7f39a29f4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399c0ceaa0 con 0x7f399c15e090 2024-04-25T08:54:37.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.643+0000 7f39a29f4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399c0b3280 con 0x7f399c0b5aa0 2024-04-25T08:54:37.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.643+0000 7f39a29f4640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f399c0cec20 con 0x7f399c161fe0 2024-04-25T08:54:37.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.643+0000 7f399b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f399c15e090 0x7f399c0cba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:37.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.643+0000 7f399b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f399c15e090 0x7f399c0cba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40216/0 (socket says 172.21.15.43:40216) 2024-04-25T08:54:37.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.643+0000 7f399b7fe640 1 -- 172.21.15.43:0/1844462765 learned_addr learned my addr 172.21.15.43:0/1844462765 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:37.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.644+0000 7f39a0f91640 1 --2- 172.21.15.43:0/1844462765 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f399c161fe0 0x7f399c1f39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:37.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.644+0000 7f399bfff640 1 --2- 172.21.15.43:0/1844462765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f399c0b5aa0 0x7f399c155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:37.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.644+0000 7f399b7fe640 1 -- 172.21.15.43:0/1844462765 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f399c161fe0 msgr2=0x7f399c1f39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:37.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.644+0000 7f399b7fe640 1 --2- 172.21.15.43:0/1844462765 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f399c161fe0 0x7f399c1f39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.644+0000 7f399b7fe640 1 -- 172.21.15.43:0/1844462765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f399c0b5aa0 msgr2=0x7f399c155870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:37.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.644+0000 7f399b7fe640 1 --2- 172.21.15.43:0/1844462765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f399c0b5aa0 0x7f399c155870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.644+0000 7f399b7fe640 1 -- 172.21.15.43:0/1844462765 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f399c15f560 con 0x7f399c15e090 2024-04-25T08:54:37.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.644+0000 7f39a0f91640 1 --2- 172.21.15.43:0/1844462765 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f399c161fe0 0x7f399c1f39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:37.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.644+0000 7f399b7fe640 1 --2- 172.21.15.43:0/1844462765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f399c15e090 0x7f399c0cba90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f398c00e8a0 tx=0x7f398c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:37.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.644+0000 7f3998ff9640 1 -- 172.21.15.43:0/1844462765 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c017070 con 0x7f399c15e090 2024-04-25T08:54:37.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.644+0000 7f3998ff9640 1 -- 172.21.15.43:0/1844462765 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f398c00c940 con 0x7f399c15e090 2024-04-25T08:54:37.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.644+0000 7f39a29f4640 1 -- 172.21.15.43:0/1844462765 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f399c15f840 con 0x7f399c15e090 2024-04-25T08:54:37.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.644+0000 7f3998ff9640 1 -- 172.21.15.43:0/1844462765 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398c00cd30 con 0x7f399c15e090 2024-04-25T08:54:37.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.644+0000 7f39a29f4640 1 -- 172.21.15.43:0/1844462765 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f399c15fe00 con 0x7f399c15e090 2024-04-25T08:54:37.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.645+0000 7f39a29f4640 1 -- 172.21.15.43:0/1844462765 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f399c06bf00 con 0x7f399c15e090 2024-04-25T08:54:37.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.645+0000 7f3998ff9640 1 -- 172.21.15.43:0/1844462765 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f398c01a070 con 0x7f399c15e090 2024-04-25T08:54:37.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.646+0000 7f3998ff9640 1 --2- 172.21.15.43:0/1844462765 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f397c06b4d0 0x7f397c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:37.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.646+0000 7f3998ff9640 1 -- 172.21.15.43:0/1844462765 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f398c08a430 con 0x7f399c15e090 2024-04-25T08:54:37.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.646+0000 7f399bfff640 1 --2- 172.21.15.43:0/1844462765 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f397c06b4d0 0x7f397c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:37.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.647+0000 7f399bfff640 1 --2- 172.21.15.43:0/1844462765 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f397c06b4d0 0x7f397c06d990 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3984034000 tx=0x7f398402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:37.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.650+0000 7f3998ff9640 1 -- 172.21.15.43:0/1844462765 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f399c06bf00 con 0x7f399c15e090 2024-04-25T08:54:37.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.832+0000 7f39a29f4640 1 -- 172.21.15.43:0/1844462765 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f399c074f60 con 0x7f399c15e090 2024-04-25T08:54:37.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.832+0000 7f3998ff9640 1 -- 172.21.15.43:0/1844462765 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f398c0592b0 con 0x7f399c15e090 2024-04-25T08:54:37.822 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T08:54:37.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.834+0000 7f39a29f4640 1 -- 172.21.15.43:0/1844462765 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f397c06b4d0 msgr2=0x7f397c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:37.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.834+0000 7f39a29f4640 1 --2- 172.21.15.43:0/1844462765 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f397c06b4d0 0x7f397c06d990 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f3984034000 tx=0x7f398402e000 comp rx=0 tx=0).stop 2024-04-25T08:54:37.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.834+0000 7f39a29f4640 1 -- 172.21.15.43:0/1844462765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f399c15e090 msgr2=0x7f399c0cba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:37.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.834+0000 7f39a29f4640 1 --2- 172.21.15.43:0/1844462765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f399c15e090 0x7f399c0cba90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f398c00e8a0 tx=0x7f398c00ed60 comp rx=0 tx=0).stop 2024-04-25T08:54:37.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.834+0000 7f39a29f4640 1 -- 172.21.15.43:0/1844462765 shutdown_connections 2024-04-25T08:54:37.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.834+0000 7f39a29f4640 1 --2- 172.21.15.43:0/1844462765 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f399c161fe0 0x7f399c1f39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.834+0000 7f39a29f4640 1 --2- 172.21.15.43:0/1844462765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f399c15e090 0x7f399c0cba90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.834+0000 7f39a29f4640 1 --2- 172.21.15.43:0/1844462765 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f397c06b4d0 0x7f397c06d990 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.834+0000 7f39a29f4640 1 --2- 172.21.15.43:0/1844462765 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f399c0b5aa0 0x7f399c155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.834+0000 7f39a29f4640 1 -- 172.21.15.43:0/1844462765 >> 172.21.15.43:0/1844462765 conn(0x7f399c0d0560 msgr2=0x7f399c0b7390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:37.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.834+0000 7f39a29f4640 1 -- 172.21.15.43:0/1844462765 shutdown_connections 2024-04-25T08:54:37.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.834+0000 7f39a29f4640 1 -- 172.21.15.43:0/1844462765 wait complete. 2024-04-25T08:54:37.845 INFO:tasks.cephfs_test_runner:Starting test: test_flag_scrub_mdsdir (tasks.cephfs.test_scrub_checks.TestScrubChecks) test flag scrub_mdsdir 2024-04-25T08:54:37.845 INFO:tasks.cephfs_test_runner:test_flag_scrub_mdsdir (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T08:54:37.845 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:54:37.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.956+0000 7f734d6b6640 1 Processor -- start 2024-04-25T08:54:37.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.956+0000 7f734d6b6640 1 -- start start 2024-04-25T08:54:37.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.957+0000 7f734d6b6640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f73480c8d00 0x7f7348162100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:37.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.957+0000 7f734d6b6640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f73480ce660 0x7f73480cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:37.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.957+0000 7f734d6b6640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73480d24a0 con 0x7f734815df50 2024-04-25T08:54:37.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.957+0000 7f734d6b6640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f734815e330 con 0x7f73480ce660 2024-04-25T08:54:37.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.957+0000 7f734d6b6640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73480d2740 con 0x7f73480c8d00 2024-04-25T08:54:37.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.957+0000 7f73477fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f73480ce660 0x7f73480cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:37.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.957+0000 7f73477fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f73480ce660 0x7f73480cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40230/0 (socket says 172.21.15.43:40230) 2024-04-25T08:54:37.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.957+0000 7f73477fe640 1 -- 172.21.15.43:0/3008211669 learned_addr learned my addr 172.21.15.43:0/3008211669 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:37.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.957+0000 7f733e7fc640 1 --2- 172.21.15.43:0/3008211669 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f73480c8d00 0x7f7348162100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:37.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.957+0000 7f73477fe640 1 -- 172.21.15.43:0/3008211669 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f73480c8d00 msgr2=0x7f7348162100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:37.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.957+0000 7f73477fe640 1 --2- 172.21.15.43:0/3008211669 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f73480c8d00 0x7f7348162100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.957+0000 7f73477fe640 1 -- 172.21.15.43:0/3008211669 >> v1:172.21.15.193:6789/0 conn(0x7f734815df50 legacy=0x7f73481619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:37.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.957+0000 7f73477fe640 1 -- 172.21.15.43:0/3008211669 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73480d2a20 con 0x7f73480ce660 2024-04-25T08:54:37.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.958+0000 7f73477fe640 1 --2- 172.21.15.43:0/3008211669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73480ce660 0x7f73480cea70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7340009e50 tx=0x7f734002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22c75c0d11bbe9bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:37.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.958+0000 7f73467fc640 1 -- 172.21.15.43:0/3008211669 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7340036070 con 0x7f73480ce660 2024-04-25T08:54:37.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.958+0000 7f73467fc640 1 -- 172.21.15.43:0/3008211669 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7340031600 con 0x7f73480ce660 2024-04-25T08:54:37.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.958+0000 7f73467fc640 1 -- 172.21.15.43:0/3008211669 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73400319d0 con 0x7f73480ce660 2024-04-25T08:54:37.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.959+0000 7f734d6b6640 1 -- 172.21.15.43:0/3008211669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73480ce660 msgr2=0x7f73480cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:37.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.959+0000 7f734d6b6640 1 --2- 172.21.15.43:0/3008211669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73480ce660 0x7f73480cea70 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7340009e50 tx=0x7f734002a710 comp rx=0 tx=0).stop 2024-04-25T08:54:37.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.959+0000 7f734d6b6640 1 -- 172.21.15.43:0/3008211669 shutdown_connections 2024-04-25T08:54:37.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.959+0000 7f734d6b6640 1 --2- 172.21.15.43:0/3008211669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73480ce660 0x7f73480cea70 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.959+0000 7f734d6b6640 1 --2- 172.21.15.43:0/3008211669 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f73480c8d00 0x7f7348162100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.959+0000 7f734d6b6640 1 -- 172.21.15.43:0/3008211669 >> 172.21.15.43:0/3008211669 conn(0x7f73480d3a30 msgr2=0x7f73480d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:37.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.959+0000 7f734d6b6640 1 -- 172.21.15.43:0/3008211669 shutdown_connections 2024-04-25T08:54:37.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.959+0000 7f734d6b6640 1 -- 172.21.15.43:0/3008211669 wait complete. 2024-04-25T08:54:37.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.960+0000 7f734d6b6640 1 Processor -- start 2024-04-25T08:54:37.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.960+0000 7f734d6b6640 1 -- start start 2024-04-25T08:54:37.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.961+0000 7f734d6b6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73480c8d00 0x7f734814c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:37.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.961+0000 7f734d6b6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73480ce660 0x7f7348157080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:37.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.961+0000 7f734d6b6640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f734815df50 0x7f7348154a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:37.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.961+0000 7f734d6b6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73480d2190 con 0x7f73480ce660 2024-04-25T08:54:37.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.961+0000 7f734d6b6640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73480c77d0 con 0x7f73480c8d00 2024-04-25T08:54:37.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.961+0000 7f734d6b6640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7348162640 con 0x7f734815df50 2024-04-25T08:54:37.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.961+0000 7f733e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73480ce660 0x7f7348157080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:37.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.961+0000 7f733e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73480ce660 0x7f7348157080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40240/0 (socket says 172.21.15.43:40240) 2024-04-25T08:54:37.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.961+0000 7f733e7fc640 1 -- 172.21.15.43:0/2927411840 learned_addr learned my addr 172.21.15.43:0/2927411840 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:37.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.961+0000 7f73477fe640 1 --2- 172.21.15.43:0/2927411840 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f734815df50 0x7f7348154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:37.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.961+0000 7f7346ffd640 1 --2- 172.21.15.43:0/2927411840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73480c8d00 0x7f734814c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:37.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.961+0000 7f733e7fc640 1 -- 172.21.15.43:0/2927411840 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f734815df50 msgr2=0x7f7348154a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:37.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.961+0000 7f733e7fc640 1 --2- 172.21.15.43:0/2927411840 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f734815df50 0x7f7348154a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.961+0000 7f733e7fc640 1 -- 172.21.15.43:0/2927411840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73480c8d00 msgr2=0x7f734814c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:37.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.961+0000 7f733e7fc640 1 --2- 172.21.15.43:0/2927411840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73480c8d00 0x7f734814c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:37.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.961+0000 7f733e7fc640 1 -- 172.21.15.43:0/2927411840 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73480cab10 con 0x7f73480ce660 2024-04-25T08:54:37.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.961+0000 7f733e7fc640 1 --2- 172.21.15.43:0/2927411840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73480ce660 0x7f7348157080 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f733000d830 tx=0x7f733000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:37.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.961+0000 7f733ffff640 1 -- 172.21.15.43:0/2927411840 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733000b6c0 con 0x7f73480ce660 2024-04-25T08:54:37.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.961+0000 7f733ffff640 1 -- 172.21.15.43:0/2927411840 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7330019070 con 0x7f73480ce660 2024-04-25T08:54:37.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.961+0000 7f734d6b6640 1 -- 172.21.15.43:0/2927411840 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73480cadf0 con 0x7f73480ce660 2024-04-25T08:54:37.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.961+0000 7f733ffff640 1 -- 172.21.15.43:0/2927411840 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733000ba80 con 0x7f73480ce660 2024-04-25T08:54:37.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.962+0000 7f734d6b6640 1 -- 172.21.15.43:0/2927411840 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73480cb3b0 con 0x7f73480ce660 2024-04-25T08:54:37.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.963+0000 7f734d6b6640 1 -- 172.21.15.43:0/2927411840 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73480680f0 con 0x7f73480ce660 2024-04-25T08:54:37.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.963+0000 7f733ffff640 1 -- 172.21.15.43:0/2927411840 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f733001e460 con 0x7f73480ce660 2024-04-25T08:54:37.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.963+0000 7f733ffff640 1 --2- 172.21.15.43:0/2927411840 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f732806b4d0 0x7f732806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:37.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.964+0000 7f733ffff640 1 -- 172.21.15.43:0/2927411840 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f733008a870 con 0x7f73480ce660 2024-04-25T08:54:37.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.964+0000 7f7346ffd640 1 --2- 172.21.15.43:0/2927411840 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f732806b4d0 0x7f732806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:37.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.964+0000 7f7346ffd640 1 --2- 172.21.15.43:0/2927411840 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f732806b4d0 0x7f732806d990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7338001500 tx=0x7f7338009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:37.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:37.968+0000 7f733ffff640 1 -- 172.21.15.43:0/2927411840 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7330059780 con 0x7f73480ce660 2024-04-25T08:54:38.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.148+0000 7f734d6b6640 1 -- 172.21.15.43:0/2927411840 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f73480ca400 con 0x7f73480ce660 2024-04-25T08:54:38.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.149+0000 7f733ffff640 1 -- 172.21.15.43:0/2927411840 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16569 (secure 0 0 0) 0x7f7330059080 con 0x7f73480ce660 2024-04-25T08:54:38.139 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:54:38.140 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":20,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:54:13.593188+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:54:09.757978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:54:10.926008+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:54:38.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.151+0000 7f734d6b6640 1 -- 172.21.15.43:0/2927411840 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f732806b4d0 msgr2=0x7f732806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:38.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.151+0000 7f734d6b6640 1 --2- 172.21.15.43:0/2927411840 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f732806b4d0 0x7f732806d990 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7338001500 tx=0x7f7338009040 comp rx=0 tx=0).stop 2024-04-25T08:54:38.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.151+0000 7f734d6b6640 1 -- 172.21.15.43:0/2927411840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73480ce660 msgr2=0x7f7348157080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:38.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.151+0000 7f734d6b6640 1 --2- 172.21.15.43:0/2927411840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73480ce660 0x7f7348157080 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f733000d830 tx=0x7f733000dcf0 comp rx=0 tx=0).stop 2024-04-25T08:54:38.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.151+0000 7f734d6b6640 1 -- 172.21.15.43:0/2927411840 shutdown_connections 2024-04-25T08:54:38.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.151+0000 7f734d6b6640 1 --2- 172.21.15.43:0/2927411840 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f734815df50 0x7f7348154a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:38.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.151+0000 7f734d6b6640 1 --2- 172.21.15.43:0/2927411840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73480ce660 0x7f7348157080 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:38.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.151+0000 7f734d6b6640 1 --2- 172.21.15.43:0/2927411840 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f732806b4d0 0x7f732806d990 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:38.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.151+0000 7f734d6b6640 1 --2- 172.21.15.43:0/2927411840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73480c8d00 0x7f734814c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:38.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.151+0000 7f734d6b6640 1 -- 172.21.15.43:0/2927411840 >> 172.21.15.43:0/2927411840 conn(0x7f73480d3a30 msgr2=0x7f7348161710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:38.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.151+0000 7f734d6b6640 1 -- 172.21.15.43:0/2927411840 shutdown_connections 2024-04-25T08:54:38.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.151+0000 7f734d6b6640 1 -- 172.21.15.43:0/2927411840 wait complete. 2024-04-25T08:54:38.152 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:54:09.757978+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:54:10.926008+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:54:38.153 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:54:38.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.266+0000 7f0070e38640 1 Processor -- start 2024-04-25T08:54:38.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.266+0000 7f0070e38640 1 -- start start 2024-04-25T08:54:38.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.267+0000 7f0070e38640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f006c15e020 0x7f006c162550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:38.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.267+0000 7f0070e38640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f006c0b47c0 con 0x7f006c15dc40 2024-04-25T08:54:38.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.267+0000 7f0070e38640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f006c071e80 con 0x7f006c0ce7e0 2024-04-25T08:54:38.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.267+0000 7f0070e38640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f006c072160 con 0x7f006c15e020 2024-04-25T08:54:38.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.267+0000 7f006a59c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f006c15dc40 0x7f006c071670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:55758/0 (socket says 172.21.15.43:55758) 2024-04-25T08:54:38.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.267+0000 7f006a59c640 1 -- 172.21.15.43:0/426729833 learned_addr learned my addr 172.21.15.43:0/426729833 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:38.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.268+0000 7f0069d9b640 1 --2- 172.21.15.43:0/426729833 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f006c15e020 0x7f006c162550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:38.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.268+0000 7f006959a640 1 -- 172.21.15.43:0/426729833 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3372638011 0 0) 0x7f006c0b47c0 con 0x7f006c15dc40 2024-04-25T08:54:38.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.268+0000 7f006959a640 1 -- 172.21.15.43:0/426729833 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0058003520 con 0x7f006c15dc40 2024-04-25T08:54:38.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.268+0000 7f006959a640 1 -- 172.21.15.43:0/426729833 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2466571764 0 0) 0x7f0058003520 con 0x7f006c15dc40 2024-04-25T08:54:38.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.268+0000 7f006959a640 1 -- 172.21.15.43:0/426729833 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f006c15e020 msgr2=0x7f006c162550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:38.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.268+0000 7f006959a640 1 --2- 172.21.15.43:0/426729833 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f006c15e020 0x7f006c162550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:38.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.268+0000 7f006959a640 1 -- 172.21.15.43:0/426729833 >> v1:172.21.15.193:6789/0 conn(0x7f006c0ce7e0 legacy=0x7f006c0d0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:38.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.268+0000 7f006959a640 1 -- 172.21.15.43:0/426729833 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f006c072440 con 0x7f006c15dc40 2024-04-25T08:54:38.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.268+0000 7f006959a640 1 -- 172.21.15.43:0/426729833 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f0064001670 con 0x7f006c15dc40 2024-04-25T08:54:38.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.268+0000 7f006959a640 1 -- 172.21.15.43:0/426729833 >> v1:172.21.15.43:6789/0 conn(0x7f006c15dc40 legacy=0x7f006c071670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:38.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.268+0000 7f006959a640 1 --2- 172.21.15.43:0/426729833 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f006c15e020 0x7f00580039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:38.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.268+0000 7f006959a640 1 --2- 172.21.15.43:0/426729833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0058003f20 0x7f0058004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:38.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.268+0000 7f006959a640 1 --2- 172.21.15.43:0/426729833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0058004d40 0x7f00580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:38.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.268+0000 7f006959a640 1 -- 172.21.15.43:0/426729833 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006c072160 con 0x7f0058003f20 2024-04-25T08:54:38.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.268+0000 7f006959a640 1 -- 172.21.15.43:0/426729833 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005800ba20 con 0x7f0058004d40 2024-04-25T08:54:38.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.268+0000 7f006959a640 1 -- 172.21.15.43:0/426729833 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f005800bc20 con 0x7f006c15e020 2024-04-25T08:54:38.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.268+0000 7f0069d9b640 1 --2- 172.21.15.43:0/426729833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0058003f20 0x7f0058004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:38.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.268+0000 7f006a59c640 1 --2- 172.21.15.43:0/426729833 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f006c15e020 0x7f00580039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:38.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.268+0000 7f006ad9d640 1 --2- 172.21.15.43:0/426729833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0058004d40 0x7f00580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:38.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.268+0000 7f0069d9b640 1 -- 172.21.15.43:0/426729833 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f006c15e020 msgr2=0x7f00580039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:38.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.268+0000 7f0069d9b640 1 --2- 172.21.15.43:0/426729833 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f006c15e020 0x7f00580039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:38.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.268+0000 7f0069d9b640 1 -- 172.21.15.43:0/426729833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0058004d40 msgr2=0x7f00580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:38.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.268+0000 7f0069d9b640 1 --2- 172.21.15.43:0/426729833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0058004d40 0x7f00580051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:38.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.268+0000 7f0069d9b640 1 -- 172.21.15.43:0/426729833 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0058003520 con 0x7f0058003f20 2024-04-25T08:54:38.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.269+0000 7f0069d9b640 1 --2- 172.21.15.43:0/426729833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0058003f20 0x7f0058004800 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0054009a90 tx=0x7f0058005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c57d1ed71cf64a36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:38.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.269+0000 7f006959a640 1 -- 172.21.15.43:0/426729833 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0054036070 con 0x7f0058003f20 2024-04-25T08:54:38.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.269+0000 7f006959a640 1 -- 172.21.15.43:0/426729833 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0054031400 con 0x7f0058003f20 2024-04-25T08:54:38.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.269+0000 7f006959a640 1 -- 172.21.15.43:0/426729833 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0054009690 con 0x7f0058003f20 2024-04-25T08:54:38.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.270+0000 7f0070e38640 1 -- 172.21.15.43:0/426729833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0058003f20 msgr2=0x7f0058004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:38.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.270+0000 7f0070e38640 1 --2- 172.21.15.43:0/426729833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0058003f20 0x7f0058004800 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0054009a90 tx=0x7f0058005830 comp rx=0 tx=0).stop 2024-04-25T08:54:38.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.270+0000 7f006ad9d640 1 -- 172.21.15.43:0/426729833 reap_dead start 2024-04-25T08:54:38.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.270+0000 7f0070e38640 1 -- 172.21.15.43:0/426729833 shutdown_connections 2024-04-25T08:54:38.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.270+0000 7f0070e38640 1 -- 172.21.15.43:0/426729833 >> 172.21.15.43:0/426729833 conn(0x7f006c06c060 msgr2=0x7f006c06c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:38.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.270+0000 7f0070e38640 1 -- 172.21.15.43:0/426729833 shutdown_connections 2024-04-25T08:54:38.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.270+0000 7f0070e38640 1 -- 172.21.15.43:0/426729833 wait complete. 2024-04-25T08:54:38.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.271+0000 7f0070e38640 1 Processor -- start 2024-04-25T08:54:38.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.271+0000 7f0070e38640 1 -- start start 2024-04-25T08:54:38.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.272+0000 7f0070e38640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0058003f20 0x7f006c06ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:38.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.272+0000 7f0070e38640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f006c151830 0x7f006c06ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:38.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.272+0000 7f0070e38640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f006c151c50 0x7f006c06b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:38.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.272+0000 7f0070e38640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006c0d1200 con 0x7f0058003f20 2024-04-25T08:54:38.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.272+0000 7f0070e38640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006c0cec90 con 0x7f006c151c50 2024-04-25T08:54:38.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.272+0000 7f0070e38640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f006c0d1380 con 0x7f006c151830 2024-04-25T08:54:38.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.272+0000 7f006a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0058003f20 0x7f006c06ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:38.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.272+0000 7f006a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0058003f20 0x7f006c06ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40272/0 (socket says 172.21.15.43:40272) 2024-04-25T08:54:38.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.272+0000 7f006a59c640 1 -- 172.21.15.43:0/951307520 learned_addr learned my addr 172.21.15.43:0/951307520 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:38.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.272+0000 7f006a59c640 1 -- 172.21.15.43:0/951307520 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f006c151830 msgr2=0x7f006c06ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:38.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.272+0000 7f006ad9d640 1 --2- 172.21.15.43:0/951307520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f006c151c50 0x7f006c06b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:38.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.272+0000 7f0069d9b640 1 --2- 172.21.15.43:0/951307520 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f006c151830 0x7f006c06ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:38.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.272+0000 7f006a59c640 1 --2- 172.21.15.43:0/951307520 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f006c151830 0x7f006c06ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:38.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.272+0000 7f006a59c640 1 -- 172.21.15.43:0/951307520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f006c151c50 msgr2=0x7f006c06b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:38.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.272+0000 7f006a59c640 1 --2- 172.21.15.43:0/951307520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f006c151c50 0x7f006c06b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:38.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.272+0000 7f006a59c640 1 -- 172.21.15.43:0/951307520 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f006c06ef60 con 0x7f0058003f20 2024-04-25T08:54:38.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.273+0000 7f006ad9d640 1 --2- 172.21.15.43:0/951307520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f006c151c50 0x7f006c06b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:38.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.273+0000 7f0069d9b640 1 --2- 172.21.15.43:0/951307520 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f006c151830 0x7f006c06ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:38.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.273+0000 7f006a59c640 1 --2- 172.21.15.43:0/951307520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0058003f20 0x7f006c06ea20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0064007370 tx=0x7f0064007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:38.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.273+0000 7f0062ffd640 1 -- 172.21.15.43:0/951307520 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006401a070 con 0x7f0058003f20 2024-04-25T08:54:38.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.273+0000 7f0062ffd640 1 -- 172.21.15.43:0/951307520 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006400f9c0 con 0x7f0058003f20 2024-04-25T08:54:38.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.273+0000 7f0070e38640 1 -- 172.21.15.43:0/951307520 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f006c157d50 con 0x7f0058003f20 2024-04-25T08:54:38.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.273+0000 7f0062ffd640 1 -- 172.21.15.43:0/951307520 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006400fd90 con 0x7f0058003f20 2024-04-25T08:54:38.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.273+0000 7f0070e38640 1 -- 172.21.15.43:0/951307520 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f006c06e220 con 0x7f0058003f20 2024-04-25T08:54:38.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.274+0000 7f0070e38640 1 -- 172.21.15.43:0/951307520 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f006c074f60 con 0x7f0058003f20 2024-04-25T08:54:38.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.274+0000 7f0062ffd640 1 -- 172.21.15.43:0/951307520 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f006401d070 con 0x7f0058003f20 2024-04-25T08:54:38.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.275+0000 7f0062ffd640 1 --2- 172.21.15.43:0/951307520 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f004c06b500 0x7f004c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:38.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.275+0000 7f0062ffd640 1 -- 172.21.15.43:0/951307520 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0064089b40 con 0x7f0058003f20 2024-04-25T08:54:38.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.275+0000 7f0069d9b640 1 --2- 172.21.15.43:0/951307520 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f004c06b500 0x7f004c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:38.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.275+0000 7f0069d9b640 1 --2- 172.21.15.43:0/951307520 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f004c06b500 0x7f004c06d9c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0054033000 tx=0x7f0054008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:38.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.279+0000 7f0062ffd640 1 -- 172.21.15.43:0/951307520 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0064058a50 con 0x7f0058003f20 2024-04-25T08:54:38.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.458+0000 7f0070e38640 1 -- 172.21.15.43:0/951307520 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f006c0d0d70 con 0x7f0058003f20 2024-04-25T08:54:38.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.458+0000 7f0062ffd640 1 -- 172.21.15.43:0/951307520 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0064058350 con 0x7f0058003f20 2024-04-25T08:54:38.448 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T08:54:38.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.460+0000 7f0070e38640 1 -- 172.21.15.43:0/951307520 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f004c06b500 msgr2=0x7f004c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:38.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.460+0000 7f0070e38640 1 --2- 172.21.15.43:0/951307520 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f004c06b500 0x7f004c06d9c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0054033000 tx=0x7f0054008030 comp rx=0 tx=0).stop 2024-04-25T08:54:38.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.460+0000 7f0070e38640 1 -- 172.21.15.43:0/951307520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0058003f20 msgr2=0x7f006c06ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:38.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.460+0000 7f0070e38640 1 --2- 172.21.15.43:0/951307520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0058003f20 0x7f006c06ea20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0064007370 tx=0x7f0064007830 comp rx=0 tx=0).stop 2024-04-25T08:54:38.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.460+0000 7f0070e38640 1 -- 172.21.15.43:0/951307520 shutdown_connections 2024-04-25T08:54:38.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.460+0000 7f0070e38640 1 --2- 172.21.15.43:0/951307520 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f004c06b500 0x7f004c06d9c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:38.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.460+0000 7f0070e38640 1 --2- 172.21.15.43:0/951307520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f006c151c50 0x7f006c06b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:38.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.460+0000 7f0070e38640 1 --2- 172.21.15.43:0/951307520 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f006c151830 0x7f006c06ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:38.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.460+0000 7f0070e38640 1 --2- 172.21.15.43:0/951307520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0058003f20 0x7f006c06ea20 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:38.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.460+0000 7f0070e38640 1 -- 172.21.15.43:0/951307520 >> 172.21.15.43:0/951307520 conn(0x7f006c06c060 msgr2=0x7f006c0715f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:38.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.460+0000 7f0070e38640 1 -- 172.21.15.43:0/951307520 shutdown_connections 2024-04-25T08:54:38.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.460+0000 7f0070e38640 1 -- 172.21.15.43:0/951307520 wait complete. 2024-04-25T08:54:38.461 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:54:38.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.575+0000 7f1f68534640 1 Processor -- start 2024-04-25T08:54:38.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.575+0000 7f1f68534640 1 -- start start 2024-04-25T08:54:38.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.576+0000 7f1f68534640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1f6015e090 0x7f1f60161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:38.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.576+0000 7f1f68534640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f1f60161fe0 0x7f1f600b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:38.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.576+0000 7f1f68534640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f60162610 con 0x7f1f600b5aa0 2024-04-25T08:54:38.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.576+0000 7f1f68534640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f600d1da0 con 0x7f1f6015e090 2024-04-25T08:54:38.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.576+0000 7f1f68534640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f600d1750 con 0x7f1f60161fe0 2024-04-25T08:54:38.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.576+0000 7f1f662d0640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1f6015e090 0x7f1f60161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:38.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.576+0000 7f1f662d0640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1f6015e090 0x7f1f60161aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40284/0 (socket says 172.21.15.43:40284) 2024-04-25T08:54:38.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.576+0000 7f1f662d0640 1 -- 172.21.15.43:0/3860388616 learned_addr learned my addr 172.21.15.43:0/3860388616 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:38.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.576+0000 7f1f65acf640 1 --2- 172.21.15.43:0/3860388616 >> v2:172.21.15.193:3300/0 conn(0x7f1f60161fe0 0x7f1f600b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:38.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.576+0000 7f1f662d0640 1 -- 172.21.15.43:0/3860388616 >> v2:172.21.15.193:3300/0 conn(0x7f1f60161fe0 msgr2=0x7f1f600b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:38.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.576+0000 7f1f662d0640 1 --2- 172.21.15.43:0/3860388616 >> v2:172.21.15.193:3300/0 conn(0x7f1f60161fe0 0x7f1f600b5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:38.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.576+0000 7f1f662d0640 1 -- 172.21.15.43:0/3860388616 >> v1:172.21.15.193:6789/0 conn(0x7f1f600b5aa0 legacy=0x7f1f600ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:38.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.576+0000 7f1f662d0640 1 -- 172.21.15.43:0/3860388616 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f600d1a30 con 0x7f1f6015e090 2024-04-25T08:54:38.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.577+0000 7f1f662d0640 1 --2- 172.21.15.43:0/3860388616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f6015e090 0x7f1f60161aa0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1f600cfe10 tx=0x7f1f5c02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a8bf343c9f287ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:38.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.577+0000 7f1f652ce640 1 -- 172.21.15.43:0/3860388616 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c037070 con 0x7f1f6015e090 2024-04-25T08:54:38.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.577+0000 7f1f652ce640 1 -- 172.21.15.43:0/3860388616 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5c02ae50 con 0x7f1f6015e090 2024-04-25T08:54:38.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.577+0000 7f1f652ce640 1 -- 172.21.15.43:0/3860388616 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c032860 con 0x7f1f6015e090 2024-04-25T08:54:38.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.578+0000 7f1f68534640 1 -- 172.21.15.43:0/3860388616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f6015e090 msgr2=0x7f1f60161aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:38.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.578+0000 7f1f68534640 1 --2- 172.21.15.43:0/3860388616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f6015e090 0x7f1f60161aa0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f1f600cfe10 tx=0x7f1f5c02a300 comp rx=0 tx=0).stop 2024-04-25T08:54:38.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.578+0000 7f1f68534640 1 -- 172.21.15.43:0/3860388616 shutdown_connections 2024-04-25T08:54:38.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.578+0000 7f1f68534640 1 --2- 172.21.15.43:0/3860388616 >> v2:172.21.15.193:3300/0 conn(0x7f1f60161fe0 0x7f1f600b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:38.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.578+0000 7f1f68534640 1 --2- 172.21.15.43:0/3860388616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f6015e090 0x7f1f60161aa0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:38.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.578+0000 7f1f68534640 1 -- 172.21.15.43:0/3860388616 >> 172.21.15.43:0/3860388616 conn(0x7f1f600d0560 msgr2=0x7f1f600d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:38.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.578+0000 7f1f68534640 1 -- 172.21.15.43:0/3860388616 shutdown_connections 2024-04-25T08:54:38.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.578+0000 7f1f68534640 1 -- 172.21.15.43:0/3860388616 wait complete. 2024-04-25T08:54:38.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.579+0000 7f1f68534640 1 Processor -- start 2024-04-25T08:54:38.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.579+0000 7f1f68534640 1 -- start start 2024-04-25T08:54:38.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.580+0000 7f1f68534640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f600b5aa0 0x7f1f601586a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:38.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.580+0000 7f1f68534640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1f6015e090 0x7f1f6015ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:38.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.580+0000 7f1f68534640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f60161fe0 0x7f1f60160a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:38.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.580+0000 7f1f68534640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f600d3240 con 0x7f1f600b5aa0 2024-04-25T08:54:38.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.580+0000 7f1f68534640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f600b3280 con 0x7f1f60161fe0 2024-04-25T08:54:38.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.580+0000 7f1f68534640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f600d33c0 con 0x7f1f6015e090 2024-04-25T08:54:38.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.580+0000 7f1f662d0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f600b5aa0 0x7f1f601586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:38.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.580+0000 7f1f662d0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f600b5aa0 0x7f1f601586a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40300/0 (socket says 172.21.15.43:40300) 2024-04-25T08:54:38.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.580+0000 7f1f662d0640 1 -- 172.21.15.43:0/2079402226 learned_addr learned my addr 172.21.15.43:0/2079402226 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:38.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.580+0000 7f1f662d0640 1 -- 172.21.15.43:0/2079402226 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1f6015e090 msgr2=0x7f1f6015ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:38.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.580+0000 7f1f662d0640 1 --2- 172.21.15.43:0/2079402226 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1f6015e090 0x7f1f6015ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:38.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.580+0000 7f1f662d0640 1 -- 172.21.15.43:0/2079402226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f60161fe0 msgr2=0x7f1f60160a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:38.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.580+0000 7f1f662d0640 1 --2- 172.21.15.43:0/2079402226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f60161fe0 0x7f1f60160a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:38.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.580+0000 7f1f662d0640 1 -- 172.21.15.43:0/2079402226 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f600b35a0 con 0x7f1f600b5aa0 2024-04-25T08:54:38.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.580+0000 7f1f662d0640 1 --2- 172.21.15.43:0/2079402226 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f600b5aa0 0x7f1f601586a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c000c00 tx=0x7f1f5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:38.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.580+0000 7f1f56ffd640 1 -- 172.21.15.43:0/2079402226 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c042070 con 0x7f1f600b5aa0 2024-04-25T08:54:38.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.580+0000 7f1f56ffd640 1 -- 172.21.15.43:0/2079402226 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5c03e670 con 0x7f1f600b5aa0 2024-04-25T08:54:38.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.581+0000 7f1f56ffd640 1 -- 172.21.15.43:0/2079402226 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c037040 con 0x7f1f600b5aa0 2024-04-25T08:54:38.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.581+0000 7f1f68534640 1 -- 172.21.15.43:0/2079402226 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f6015f900 con 0x7f1f600b5aa0 2024-04-25T08:54:38.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.581+0000 7f1f68534640 1 -- 172.21.15.43:0/2079402226 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f600b3840 con 0x7f1f600b5aa0 2024-04-25T08:54:38.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.581+0000 7f1f68534640 1 -- 172.21.15.43:0/2079402226 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f6006bf00 con 0x7f1f600b5aa0 2024-04-25T08:54:38.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.582+0000 7f1f56ffd640 1 -- 172.21.15.43:0/2079402226 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1f5c045070 con 0x7f1f600b5aa0 2024-04-25T08:54:38.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.582+0000 7f1f56ffd640 1 --2- 172.21.15.43:0/2079402226 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1f4406b4d0 0x7f1f4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:38.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.582+0000 7f1f56ffd640 1 -- 172.21.15.43:0/2079402226 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1f5c0aac00 con 0x7f1f600b5aa0 2024-04-25T08:54:38.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.583+0000 7f1f65acf640 1 --2- 172.21.15.43:0/2079402226 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1f4406b4d0 0x7f1f4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:38.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.583+0000 7f1f65acf640 1 --2- 172.21.15.43:0/2079402226 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1f4406b4d0 0x7f1f4406d990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1f50006fd0 tx=0x7f1f50008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:38.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.587+0000 7f1f56ffd640 1 -- 172.21.15.43:0/2079402226 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1f6006bf00 con 0x7f1f600b5aa0 2024-04-25T08:54:38.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.767+0000 7f1f68534640 1 -- 172.21.15.43:0/2079402226 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1f60074f60 con 0x7f1f600b5aa0 2024-04-25T08:54:38.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.767+0000 7f1f56ffd640 1 -- 172.21.15.43:0/2079402226 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1f5c079b90 con 0x7f1f600b5aa0 2024-04-25T08:54:38.757 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T08:54:38.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.769+0000 7f1f68534640 1 -- 172.21.15.43:0/2079402226 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1f4406b4d0 msgr2=0x7f1f4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:38.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.769+0000 7f1f68534640 1 --2- 172.21.15.43:0/2079402226 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1f4406b4d0 0x7f1f4406d990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1f50006fd0 tx=0x7f1f50008040 comp rx=0 tx=0).stop 2024-04-25T08:54:38.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.769+0000 7f1f68534640 1 -- 172.21.15.43:0/2079402226 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f600b5aa0 msgr2=0x7f1f601586a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:38.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.769+0000 7f1f68534640 1 --2- 172.21.15.43:0/2079402226 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f600b5aa0 0x7f1f601586a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c000c00 tx=0x7f1f5c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:54:38.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.769+0000 7f1f68534640 1 -- 172.21.15.43:0/2079402226 shutdown_connections 2024-04-25T08:54:38.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.769+0000 7f1f68534640 1 --2- 172.21.15.43:0/2079402226 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1f4406b4d0 0x7f1f4406d990 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:38.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.769+0000 7f1f68534640 1 --2- 172.21.15.43:0/2079402226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f60161fe0 0x7f1f60160a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:38.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.769+0000 7f1f68534640 1 --2- 172.21.15.43:0/2079402226 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1f6015e090 0x7f1f6015ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:38.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.769+0000 7f1f68534640 1 --2- 172.21.15.43:0/2079402226 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f600b5aa0 0x7f1f601586a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:38.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.769+0000 7f1f68534640 1 -- 172.21.15.43:0/2079402226 >> 172.21.15.43:0/2079402226 conn(0x7f1f600d0560 msgr2=0x7f1f600b7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:38.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.769+0000 7f1f68534640 1 -- 172.21.15.43:0/2079402226 shutdown_connections 2024-04-25T08:54:38.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.769+0000 7f1f68534640 1 -- 172.21.15.43:0/2079402226 wait complete. 2024-04-25T08:54:38.770 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:54:38.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.884+0000 7fea566e1640 1 Processor -- start 2024-04-25T08:54:38.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.884+0000 7fea566e1640 1 -- start start 2024-04-25T08:54:38.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.885+0000 7fea566e1640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fea5015dd00 0x7fea500d25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:38.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.885+0000 7fea566e1640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fea500cf0a0 0x7fea500d02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:38.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.885+0000 7fea566e1640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea5015e0e0 con 0x7fea500cf4e0 2024-04-25T08:54:38.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.885+0000 7fea566e1640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea500d15c0 con 0x7fea5015dd00 2024-04-25T08:54:38.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.885+0000 7fea566e1640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea500d1810 con 0x7fea500cf0a0 2024-04-25T08:54:38.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.885+0000 7fea4ffff640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fea5015dd00 0x7fea500d25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:38.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.885+0000 7fea54c7e640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7fea500cf4e0 0x7fea5006b040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:48644/0 (socket says 172.21.15.43:48644) 2024-04-25T08:54:38.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.885+0000 7fea54c7e640 1 -- 172.21.15.43:0/447229873 learned_addr learned my addr 172.21.15.43:0/447229873 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:38.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.885+0000 7fea4f7fe640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fea500cf0a0 0x7fea500d02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:38.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.886+0000 7fea4ffff640 1 -- 172.21.15.43:0/447229873 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fea500cf0a0 msgr2=0x7fea500d02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:38.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.886+0000 7fea4ffff640 1 --2- 172.21.15.43:0/447229873 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fea500cf0a0 0x7fea500d02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:38.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.886+0000 7fea4ffff640 1 -- 172.21.15.43:0/447229873 >> v1:172.21.15.193:6789/0 conn(0x7fea500cf4e0 legacy=0x7fea5006b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:38.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.886+0000 7fea4ffff640 1 -- 172.21.15.43:0/447229873 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea500d1af0 con 0x7fea5015dd00 2024-04-25T08:54:38.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.886+0000 7fea4effd640 1 -- 172.21.15.43:0/447229873 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2816415239 0 0) 0x7fea5015e0e0 con 0x7fea500cf4e0 2024-04-25T08:54:38.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.887+0000 7fea4ffff640 1 --2- 172.21.15.43:0/447229873 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea5015dd00 0x7fea500d25d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fea380097a0 tx=0x7fea3802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91ede2eb69e47bce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:38.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.887+0000 7fea4effd640 1 -- 172.21.15.43:0/447229873 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea38037070 con 0x7fea5015dd00 2024-04-25T08:54:38.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.887+0000 7fea4effd640 1 -- 172.21.15.43:0/447229873 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3802ae50 con 0x7fea5015dd00 2024-04-25T08:54:38.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.887+0000 7fea4effd640 1 -- 172.21.15.43:0/447229873 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea38032660 con 0x7fea5015dd00 2024-04-25T08:54:38.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.887+0000 7fea566e1640 1 -- 172.21.15.43:0/447229873 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea5015dd00 msgr2=0x7fea500d25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:38.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.887+0000 7fea566e1640 1 --2- 172.21.15.43:0/447229873 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea5015dd00 0x7fea500d25d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fea380097a0 tx=0x7fea3802a100 comp rx=0 tx=0).stop 2024-04-25T08:54:38.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.887+0000 7fea566e1640 1 -- 172.21.15.43:0/447229873 shutdown_connections 2024-04-25T08:54:38.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.887+0000 7fea566e1640 1 --2- 172.21.15.43:0/447229873 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fea500cf0a0 0x7fea500d02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:38.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.887+0000 7fea566e1640 1 --2- 172.21.15.43:0/447229873 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea5015dd00 0x7fea500d25d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:38.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.887+0000 7fea566e1640 1 -- 172.21.15.43:0/447229873 >> 172.21.15.43:0/447229873 conn(0x7fea5009cd20 msgr2=0x7fea500d39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:38.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.887+0000 7fea566e1640 1 -- 172.21.15.43:0/447229873 shutdown_connections 2024-04-25T08:54:38.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.887+0000 7fea566e1640 1 -- 172.21.15.43:0/447229873 wait complete. 2024-04-25T08:54:38.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.889+0000 7fea566e1640 1 Processor -- start 2024-04-25T08:54:38.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.889+0000 7fea566e1640 1 -- start start 2024-04-25T08:54:38.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.889+0000 7fea566e1640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fea500cf0a0 0x7fea5014e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:38.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.889+0000 7fea566e1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea500cf4e0 0x7fea500b43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:38.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.889+0000 7fea566e1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea5015dd00 0x7fea500b48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:38.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.889+0000 7fea566e1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea500d42c0 con 0x7fea500cf4e0 2024-04-25T08:54:38.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.889+0000 7fea566e1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea500d20b0 con 0x7fea5015dd00 2024-04-25T08:54:38.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.889+0000 7fea566e1640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea500d4440 con 0x7fea500cf0a0 2024-04-25T08:54:38.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.890+0000 7fea4f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea500cf4e0 0x7fea500b43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:38.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.890+0000 7fea4f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea500cf4e0 0x7fea500b43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40316/0 (socket says 172.21.15.43:40316) 2024-04-25T08:54:38.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.890+0000 7fea4f7fe640 1 -- 172.21.15.43:0/1567237580 learned_addr learned my addr 172.21.15.43:0/1567237580 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:38.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.890+0000 7fea4ffff640 1 --2- 172.21.15.43:0/1567237580 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fea500cf0a0 0x7fea5014e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:38.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.890+0000 7fea54c7e640 1 --2- 172.21.15.43:0/1567237580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea5015dd00 0x7fea500b48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:38.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.890+0000 7fea4f7fe640 1 -- 172.21.15.43:0/1567237580 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fea500cf0a0 msgr2=0x7fea5014e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:38.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.890+0000 7fea4f7fe640 1 --2- 172.21.15.43:0/1567237580 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fea500cf0a0 0x7fea5014e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:38.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.890+0000 7fea4f7fe640 1 -- 172.21.15.43:0/1567237580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea5015dd00 msgr2=0x7fea500b48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:38.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.890+0000 7fea4f7fe640 1 --2- 172.21.15.43:0/1567237580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea5015dd00 0x7fea500b48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:38.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.890+0000 7fea4f7fe640 1 -- 172.21.15.43:0/1567237580 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea500cba20 con 0x7fea500cf4e0 2024-04-25T08:54:38.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.890+0000 7fea4f7fe640 1 --2- 172.21.15.43:0/1567237580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea500cf4e0 0x7fea500b43a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fea4000e9e0 tx=0x7fea4000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:38.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.890+0000 7fea4cff9640 1 -- 172.21.15.43:0/1567237580 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea40017070 con 0x7fea500cf4e0 2024-04-25T08:54:38.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.890+0000 7fea4cff9640 1 -- 172.21.15.43:0/1567237580 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea4000c9d0 con 0x7fea500cf4e0 2024-04-25T08:54:38.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.890+0000 7fea566e1640 1 -- 172.21.15.43:0/1567237580 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea500cbd00 con 0x7fea500cf4e0 2024-04-25T08:54:38.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.890+0000 7fea4cff9640 1 -- 172.21.15.43:0/1567237580 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea4000cdc0 con 0x7fea500cf4e0 2024-04-25T08:54:38.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.890+0000 7fea566e1640 1 -- 172.21.15.43:0/1567237580 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea500cc2c0 con 0x7fea500cf4e0 2024-04-25T08:54:38.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.891+0000 7fea566e1640 1 -- 172.21.15.43:0/1567237580 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea50074f60 con 0x7fea500cf4e0 2024-04-25T08:54:38.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.892+0000 7fea4cff9640 1 -- 172.21.15.43:0/1567237580 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fea4001a070 con 0x7fea500cf4e0 2024-04-25T08:54:38.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.892+0000 7fea4cff9640 1 --2- 172.21.15.43:0/1567237580 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fea3006b4d0 0x7fea3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:38.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.892+0000 7fea4cff9640 1 -- 172.21.15.43:0/1567237580 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fea4008a5b0 con 0x7fea500cf4e0 2024-04-25T08:54:38.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.893+0000 7fea4ffff640 1 --2- 172.21.15.43:0/1567237580 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fea3006b4d0 0x7fea3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:38.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.893+0000 7fea4ffff640 1 --2- 172.21.15.43:0/1567237580 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fea3006b4d0 0x7fea3006d990 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fea38034000 tx=0x7fea3802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:38.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:38.896+0000 7fea4cff9640 1 -- 172.21.15.43:0/1567237580 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fea40058d50 con 0x7fea500cf4e0 2024-04-25T08:54:39.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.076+0000 7fea566e1640 1 -- 172.21.15.43:0/1567237580 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fea500685d0 con 0x7fea500cf4e0 2024-04-25T08:54:39.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.076+0000 7fea4cff9640 1 -- 172.21.15.43:0/1567237580 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fea4005cdf0 con 0x7fea500cf4e0 2024-04-25T08:54:39.066 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T08:54:39.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.078+0000 7fea566e1640 1 -- 172.21.15.43:0/1567237580 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fea3006b4d0 msgr2=0x7fea3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:39.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.078+0000 7fea566e1640 1 --2- 172.21.15.43:0/1567237580 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fea3006b4d0 0x7fea3006d990 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fea38034000 tx=0x7fea3802e000 comp rx=0 tx=0).stop 2024-04-25T08:54:39.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.078+0000 7fea566e1640 1 -- 172.21.15.43:0/1567237580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea500cf4e0 msgr2=0x7fea500b43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:39.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.078+0000 7fea566e1640 1 --2- 172.21.15.43:0/1567237580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea500cf4e0 0x7fea500b43a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fea4000e9e0 tx=0x7fea4000eea0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.078+0000 7fea566e1640 1 -- 172.21.15.43:0/1567237580 shutdown_connections 2024-04-25T08:54:39.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.078+0000 7fea566e1640 1 --2- 172.21.15.43:0/1567237580 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fea3006b4d0 0x7fea3006d990 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.078+0000 7fea566e1640 1 --2- 172.21.15.43:0/1567237580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fea5015dd00 0x7fea500b48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.078+0000 7fea566e1640 1 --2- 172.21.15.43:0/1567237580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fea500cf4e0 0x7fea500b43a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.078+0000 7fea566e1640 1 --2- 172.21.15.43:0/1567237580 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fea500cf0a0 0x7fea5014e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.078+0000 7fea566e1640 1 -- 172.21.15.43:0/1567237580 >> 172.21.15.43:0/1567237580 conn(0x7fea5009cd20 msgr2=0x7fea500b2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:39.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.078+0000 7fea566e1640 1 -- 172.21.15.43:0/1567237580 shutdown_connections 2024-04-25T08:54:39.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.078+0000 7fea566e1640 1 -- 172.21.15.43:0/1567237580 wait complete. 2024-04-25T08:54:39.079 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir' 2024-04-25T08:54:39.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.190+0000 7f815d0dd640 1 Processor -- start 2024-04-25T08:54:39.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.190+0000 7f815d0dd640 1 -- start start 2024-04-25T08:54:39.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.191+0000 7f815d0dd640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f81580d0d90 0x7f81580ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:39.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.191+0000 7f815d0dd640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f81580d2500 0x7f81580d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:39.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.191+0000 7f815d0dd640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81580b6c50 con 0x7f81580d39d0 2024-04-25T08:54:39.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.191+0000 7f815d0dd640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f815815e010 con 0x7f81580d2500 2024-04-25T08:54:39.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.191+0000 7f815d0dd640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81580d2020 con 0x7f81580d0d90 2024-04-25T08:54:39.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.191+0000 7f8156d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f81580d39d0 0x7f81580d3db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:55768/0 (socket says 172.21.15.43:55768) 2024-04-25T08:54:39.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.191+0000 7f8156d9d640 1 -- 172.21.15.43:0/3198581530 learned_addr learned my addr 172.21.15.43:0/3198581530 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:39.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.191+0000 7f815759e640 1 --2- 172.21.15.43:0/3198581530 >> v2:172.21.15.193:3300/0 conn(0x7f81580d2500 0x7f81580d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:39.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.191+0000 7f815659c640 1 --2- 172.21.15.43:0/3198581530 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f81580d0d90 0x7f81580ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:39.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.192+0000 7f8155d9b640 1 -- 172.21.15.43:0/3198581530 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2679365685 0 0) 0x7f8150001110 con 0x7f81580d39d0 2024-04-25T08:54:39.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.192+0000 7f8155d9b640 1 -- 172.21.15.43:0/3198581530 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8144003520 con 0x7f81580d39d0 2024-04-25T08:54:39.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.192+0000 7f8155d9b640 1 -- 172.21.15.43:0/3198581530 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 390132912 0 0) 0x7f8144003520 con 0x7f81580d39d0 2024-04-25T08:54:39.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.192+0000 7f815759e640 1 -- 172.21.15.43:0/3198581530 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f81580d0d90 msgr2=0x7f81580ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:39.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.192+0000 7f815759e640 1 --2- 172.21.15.43:0/3198581530 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f81580d0d90 0x7f81580ce2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.192+0000 7f815759e640 1 -- 172.21.15.43:0/3198581530 >> v1:172.21.15.43:6789/0 conn(0x7f81580d39d0 legacy=0x7f81580d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:39.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.192+0000 7f815759e640 1 -- 172.21.15.43:0/3198581530 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81580cec40 con 0x7f81580d2500 2024-04-25T08:54:39.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.193+0000 7f815759e640 1 --2- 172.21.15.43:0/3198581530 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81580d2500 0x7f81580d2990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8148009d90 tx=0x7f814802a560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf330d25847f625d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:39.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.193+0000 7f8155d9b640 1 -- 172.21.15.43:0/3198581530 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8150001840 con 0x7f81580d2500 2024-04-25T08:54:39.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.193+0000 7f8155d9b640 1 -- 172.21.15.43:0/3198581530 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8148007540 con 0x7f81580d2500 2024-04-25T08:54:39.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.193+0000 7f8155d9b640 1 -- 172.21.15.43:0/3198581530 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8148007910 con 0x7f81580d2500 2024-04-25T08:54:39.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.193+0000 7f815d0dd640 1 -- 172.21.15.43:0/3198581530 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81580d2500 msgr2=0x7f81580d2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:39.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.193+0000 7f815d0dd640 1 --2- 172.21.15.43:0/3198581530 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81580d2500 0x7f81580d2990 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8148009d90 tx=0x7f814802a560 comp rx=0 tx=0).stop 2024-04-25T08:54:39.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.193+0000 7f815d0dd640 1 -- 172.21.15.43:0/3198581530 shutdown_connections 2024-04-25T08:54:39.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.193+0000 7f815d0dd640 1 --2- 172.21.15.43:0/3198581530 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81580d2500 0x7f81580d2990 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.193+0000 7f815d0dd640 1 --2- 172.21.15.43:0/3198581530 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f81580d0d90 0x7f81580ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.193+0000 7f815d0dd640 1 -- 172.21.15.43:0/3198581530 >> 172.21.15.43:0/3198581530 conn(0x7f815806b000 msgr2=0x7f815806b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:39.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.193+0000 7f815d0dd640 1 -- 172.21.15.43:0/3198581530 shutdown_connections 2024-04-25T08:54:39.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.193+0000 7f815d0dd640 1 -- 172.21.15.43:0/3198581530 wait complete. 2024-04-25T08:54:39.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.195+0000 7f815d0dd640 1 Processor -- start 2024-04-25T08:54:39.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.195+0000 7f815d0dd640 1 -- start start 2024-04-25T08:54:39.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.196+0000 7f815d0dd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81580d0d90 0x7f81580cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:39.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.196+0000 7f815d0dd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81580d2500 0x7f81581605f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:39.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.196+0000 7f815d0dd640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f81580d39d0 0x7f81580cc390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:39.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.196+0000 7f815d0dd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81580cf200 con 0x7f81580d0d90 2024-04-25T08:54:39.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.196+0000 7f815d0dd640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81580b2f00 con 0x7f81580d2500 2024-04-25T08:54:39.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.196+0000 7f815d0dd640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81580d2ed0 con 0x7f81580d39d0 2024-04-25T08:54:39.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.196+0000 7f8156d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81580d0d90 0x7f81580cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:39.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.196+0000 7f8156d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81580d0d90 0x7f81580cab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40330/0 (socket says 172.21.15.43:40330) 2024-04-25T08:54:39.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.196+0000 7f8156d9d640 1 -- 172.21.15.43:0/1106490643 learned_addr learned my addr 172.21.15.43:0/1106490643 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:39.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.196+0000 7f815659c640 1 --2- 172.21.15.43:0/1106490643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81580d2500 0x7f81581605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:39.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.196+0000 7f815759e640 1 --2- 172.21.15.43:0/1106490643 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f81580d39d0 0x7f81580cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:39.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.196+0000 7f8156d9d640 1 -- 172.21.15.43:0/1106490643 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f81580d39d0 msgr2=0x7f81580cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:39.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.196+0000 7f8156d9d640 1 --2- 172.21.15.43:0/1106490643 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f81580d39d0 0x7f81580cc390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.196+0000 7f8156d9d640 1 -- 172.21.15.43:0/1106490643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81580d2500 msgr2=0x7f81581605f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:39.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.196+0000 7f8156d9d640 1 --2- 172.21.15.43:0/1106490643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81580d2500 0x7f81581605f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.196+0000 7f8156d9d640 1 -- 172.21.15.43:0/1106490643 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81580cb050 con 0x7f81580d0d90 2024-04-25T08:54:39.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.196+0000 7f815659c640 1 --2- 172.21.15.43:0/1106490643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81580d2500 0x7f81581605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:39.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.196+0000 7f8156d9d640 1 --2- 172.21.15.43:0/1106490643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81580d0d90 0x7f81580cab10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f815000c730 tx=0x7f815000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:39.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.196+0000 7f814f7fe640 1 -- 172.21.15.43:0/1106490643 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8150017070 con 0x7f81580d0d90 2024-04-25T08:54:39.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.196+0000 7f814f7fe640 1 -- 172.21.15.43:0/1106490643 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8150005af0 con 0x7f81580d0d90 2024-04-25T08:54:39.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.196+0000 7f814f7fe640 1 -- 172.21.15.43:0/1106490643 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8150007070 con 0x7f81580d0d90 2024-04-25T08:54:39.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.196+0000 7f815d0dd640 1 -- 172.21.15.43:0/1106490643 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81580cb2f0 con 0x7f81580d0d90 2024-04-25T08:54:39.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.196+0000 7f815d0dd640 1 -- 172.21.15.43:0/1106490643 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81580cb8b0 con 0x7f81580d0d90 2024-04-25T08:54:39.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.197+0000 7f815d0dd640 1 -- 172.21.15.43:0/1106490643 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8158074ef0 con 0x7f81580d0d90 2024-04-25T08:54:39.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.198+0000 7f814f7fe640 1 -- 172.21.15.43:0/1106490643 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f815001a070 con 0x7f81580d0d90 2024-04-25T08:54:39.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.198+0000 7f814f7fe640 1 --2- 172.21.15.43:0/1106490643 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f813806b500 0x7f813806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:39.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.198+0000 7f814f7fe640 1 -- 172.21.15.43:0/1106490643 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8150089610 con 0x7f81580d0d90 2024-04-25T08:54:39.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.199+0000 7f815659c640 1 --2- 172.21.15.43:0/1106490643 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f813806b500 0x7f813806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:39.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.199+0000 7f815659c640 1 --2- 172.21.15.43:0/1106490643 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f813806b500 0x7f813806d9c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f81581f7770 tx=0x7f8140008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:39.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.202+0000 7f814f7fe640 1 -- 172.21.15.43:0/1106490643 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f81580b6c50 con 0x7f81580d0d90 2024-04-25T08:54:39.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.452+0000 7f815d0dd640 1 -- 172.21.15.43:0/1106490643 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]} v 0) v1 -- 0x7f81580ce880 con 0x7f81580d0d90 2024-04-25T08:54:39.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.809+0000 7f814f7fe640 1 -- 172.21.15.43:0/1106490643 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]}]=0 v52) v1 ==== 151+0+0 (secure 0 0 0) 0x7f8150058490 con 0x7f81580d0d90 2024-04-25T08:54:39.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.812+0000 7f815d0dd640 1 -- 172.21.15.43:0/1106490643 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f813806b500 msgr2=0x7f813806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:39.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.812+0000 7f815d0dd640 1 --2- 172.21.15.43:0/1106490643 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f813806b500 0x7f813806d9c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f81581f7770 tx=0x7f8140008040 comp rx=0 tx=0).stop 2024-04-25T08:54:39.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.812+0000 7f815d0dd640 1 -- 172.21.15.43:0/1106490643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81580d0d90 msgr2=0x7f81580cab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:39.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.812+0000 7f815d0dd640 1 --2- 172.21.15.43:0/1106490643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81580d0d90 0x7f81580cab10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f815000c730 tx=0x7f815000cbf0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.812+0000 7f815d0dd640 1 -- 172.21.15.43:0/1106490643 shutdown_connections 2024-04-25T08:54:39.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.812+0000 7f815d0dd640 1 --2- 172.21.15.43:0/1106490643 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f81580d39d0 0x7f81580cc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.812+0000 7f815d0dd640 1 --2- 172.21.15.43:0/1106490643 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f813806b500 0x7f813806d9c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.812+0000 7f815d0dd640 1 --2- 172.21.15.43:0/1106490643 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f81580d2500 0x7f81581605f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.812+0000 7f815d0dd640 1 --2- 172.21.15.43:0/1106490643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f81580d0d90 0x7f81580cab10 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.812+0000 7f815d0dd640 1 -- 172.21.15.43:0/1106490643 >> 172.21.15.43:0/1106490643 conn(0x7f815806b000 msgr2=0x7f81580c93d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:39.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.812+0000 7f815d0dd640 1 -- 172.21.15.43:0/1106490643 shutdown_connections 2024-04-25T08:54:39.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.812+0000 7f815d0dd640 1 -- 172.21.15.43:0/1106490643 wait complete. 2024-04-25T08:54:39.813 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T08:54:39.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.928+0000 7fb890d63640 1 Processor -- start 2024-04-25T08:54:39.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.928+0000 7fb890d63640 1 -- start start 2024-04-25T08:54:39.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.929+0000 7fb890d63640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb88c15dd00 0x7fb88c0d25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:39.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.929+0000 7fb890d63640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb88c15e0e0 con 0x7fb88c0d09d0 2024-04-25T08:54:39.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.929+0000 7fb890d63640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb88c0d1810 con 0x7fb88c0cf0a0 2024-04-25T08:54:39.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.929+0000 7fb890d63640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb88c0d15c0 con 0x7fb88c15dd00 2024-04-25T08:54:39.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.929+0000 7fb88ad9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb88c0d09d0 0x7fb88c06b040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39632/0 (socket says 172.21.15.43:39632) 2024-04-25T08:54:39.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.929+0000 7fb88ad9d640 1 -- 172.21.15.43:0/1587123444 learned_addr learned my addr 172.21.15.43:0/1587123444 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:39.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.929+0000 7fb88a59c640 1 --2- 172.21.15.43:0/1587123444 >> v2:172.21.15.193:3300/0 conn(0x7fb88c15dd00 0x7fb88c0d25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:39.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.929+0000 7fb88959a640 1 -- 172.21.15.43:0/1587123444 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2392148387 0 0) 0x7fb88c15e0e0 con 0x7fb88c0d09d0 2024-04-25T08:54:39.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.930+0000 7fb88959a640 1 -- 172.21.15.43:0/1587123444 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb874003520 con 0x7fb88c0d09d0 2024-04-25T08:54:39.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.930+0000 7fb88959a640 1 -- 172.21.15.43:0/1587123444 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2383366755 0 0) 0x7fb88c0d1810 con 0x7fb88c0cf0a0 2024-04-25T08:54:39.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.930+0000 7fb88959a640 1 -- 172.21.15.43:0/1587123444 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb88c15e0e0 con 0x7fb88c0cf0a0 2024-04-25T08:54:39.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.930+0000 7fb88959a640 1 -- 172.21.15.43:0/1587123444 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2687827627 0 0) 0x7fb874003520 con 0x7fb88c0d09d0 2024-04-25T08:54:39.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.930+0000 7fb88a59c640 1 -- 172.21.15.43:0/1587123444 >> v1:172.21.15.193:6789/0 conn(0x7fb88c0cf0a0 legacy=0x7fb88c0d02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:39.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.930+0000 7fb88a59c640 1 -- 172.21.15.43:0/1587123444 >> v1:172.21.15.43:6789/0 conn(0x7fb88c0d09d0 legacy=0x7fb88c06b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:39.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.930+0000 7fb88a59c640 1 -- 172.21.15.43:0/1587123444 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb88c0d1af0 con 0x7fb88c15dd00 2024-04-25T08:54:39.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.931+0000 7fb88a59c640 1 --2- 172.21.15.43:0/1587123444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb88c15dd00 0x7fb88c0d25d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb88c0d1740 tx=0x7fb87c02a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2487c6b80a2b3d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:39.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.931+0000 7fb88959a640 1 -- 172.21.15.43:0/1587123444 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87c037070 con 0x7fb88c15dd00 2024-04-25T08:54:39.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.931+0000 7fb88959a640 1 -- 172.21.15.43:0/1587123444 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb87c02ac30 con 0x7fb88c15dd00 2024-04-25T08:54:39.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.931+0000 7fb88959a640 1 -- 172.21.15.43:0/1587123444 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8840019f0 con 0x7fb88c15dd00 2024-04-25T08:54:39.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.931+0000 7fb890d63640 1 -- 172.21.15.43:0/1587123444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb88c15dd00 msgr2=0x7fb88c0d25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:39.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.931+0000 7fb890d63640 1 --2- 172.21.15.43:0/1587123444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb88c15dd00 0x7fb88c0d25d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb88c0d1740 tx=0x7fb87c02a1a0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.931+0000 7fb890d63640 1 -- 172.21.15.43:0/1587123444 shutdown_connections 2024-04-25T08:54:39.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.931+0000 7fb890d63640 1 --2- 172.21.15.43:0/1587123444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb88c15dd00 0x7fb88c0d25d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.931+0000 7fb890d63640 1 -- 172.21.15.43:0/1587123444 >> 172.21.15.43:0/1587123444 conn(0x7fb88c09cd20 msgr2=0x7fb88c0d39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:39.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.931+0000 7fb890d63640 1 -- 172.21.15.43:0/1587123444 shutdown_connections 2024-04-25T08:54:39.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.931+0000 7fb890d63640 1 -- 172.21.15.43:0/1587123444 wait complete. 2024-04-25T08:54:39.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.933+0000 7fb890d63640 1 Processor -- start 2024-04-25T08:54:39.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.933+0000 7fb890d63640 1 -- start start 2024-04-25T08:54:39.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.933+0000 7fb890d63640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb88c0cf0a0 0x7fb88c0cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:39.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.933+0000 7fb890d63640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb88c0d09d0 0x7fb88c1602e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:39.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.933+0000 7fb890d63640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb88c15dd00 0x7fb88c0b1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:39.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.933+0000 7fb890d63640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c0d42c0 con 0x7fb88c15dd00 2024-04-25T08:54:39.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.933+0000 7fb890d63640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c0d20b0 con 0x7fb88c0d09d0 2024-04-25T08:54:39.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.933+0000 7fb890d63640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb88c0d4440 con 0x7fb88c0cf0a0 2024-04-25T08:54:39.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.934+0000 7fb88ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb88c15dd00 0x7fb88c0b1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:39.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.934+0000 7fb88ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb88c15dd00 0x7fb88c0b1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38096/0 (socket says 172.21.15.43:38096) 2024-04-25T08:54:39.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.934+0000 7fb88ad9d640 1 -- 172.21.15.43:0/3674038630 learned_addr learned my addr 172.21.15.43:0/3674038630 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:39.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.934+0000 7fb88a59c640 1 --2- 172.21.15.43:0/3674038630 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb88c0cf0a0 0x7fb88c0cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:39.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.934+0000 7fb889d9b640 1 --2- 172.21.15.43:0/3674038630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb88c0d09d0 0x7fb88c1602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:39.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.934+0000 7fb88ad9d640 1 -- 172.21.15.43:0/3674038630 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb88c0cf0a0 msgr2=0x7fb88c0cb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:39.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.934+0000 7fb88ad9d640 1 --2- 172.21.15.43:0/3674038630 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb88c0cf0a0 0x7fb88c0cb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.934+0000 7fb88ad9d640 1 -- 172.21.15.43:0/3674038630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb88c0d09d0 msgr2=0x7fb88c1602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:39.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.934+0000 7fb88ad9d640 1 --2- 172.21.15.43:0/3674038630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb88c0d09d0 0x7fb88c1602e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:39.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.934+0000 7fb88ad9d640 1 -- 172.21.15.43:0/3674038630 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb88c0cbb60 con 0x7fb88c15dd00 2024-04-25T08:54:39.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.934+0000 7fb88ad9d640 1 --2- 172.21.15.43:0/3674038630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb88c15dd00 0x7fb88c0b1610 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb884005c00 tx=0x7fb88400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:39.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.934+0000 7fb87affd640 1 -- 172.21.15.43:0/3674038630 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb884017070 con 0x7fb88c15dd00 2024-04-25T08:54:39.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.934+0000 7fb87affd640 1 -- 172.21.15.43:0/3674038630 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb884012400 con 0x7fb88c15dd00 2024-04-25T08:54:39.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.934+0000 7fb890d63640 1 -- 172.21.15.43:0/3674038630 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb88c0cbe40 con 0x7fb88c15dd00 2024-04-25T08:54:39.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.934+0000 7fb87affd640 1 -- 172.21.15.43:0/3674038630 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8840127f0 con 0x7fb88c15dd00 2024-04-25T08:54:39.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.934+0000 7fb890d63640 1 -- 172.21.15.43:0/3674038630 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb88c0cc400 con 0x7fb88c15dd00 2024-04-25T08:54:39.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.936+0000 7fb890d63640 1 -- 172.21.15.43:0/3674038630 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb88c074f60 con 0x7fb88c15dd00 2024-04-25T08:54:39.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.936+0000 7fb87affd640 1 -- 172.21.15.43:0/3674038630 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb88401a070 con 0x7fb88c15dd00 2024-04-25T08:54:39.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.936+0000 7fb87affd640 1 --2- 172.21.15.43:0/3674038630 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb86c06b4d0 0x7fb86c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:39.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.936+0000 7fb87affd640 1 -- 172.21.15.43:0/3674038630 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb88408a0f0 con 0x7fb88c15dd00 2024-04-25T08:54:39.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.937+0000 7fb88a59c640 1 --2- 172.21.15.43:0/3674038630 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb86c06b4d0 0x7fb86c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:39.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.937+0000 7fb88a59c640 1 --2- 172.21.15.43:0/3674038630 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb86c06b4d0 0x7fb86c06d990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb87c034000 tx=0x7fb87c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:39.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:39.941+0000 7fb87affd640 1 -- 172.21.15.43:0/3674038630 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb884058920 con 0x7fb88c15dd00 2024-04-25T08:54:40.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.119+0000 7fb890d63640 1 -- 172.21.15.43:0/3674038630 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb88c161300 con 0x7fb88c15dd00 2024-04-25T08:54:40.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.122+0000 7fb87affd640 1 -- 172.21.15.43:0/3674038630 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb88405c9c0 con 0x7fb88c15dd00 2024-04-25T08:54:40.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.124+0000 7fb890d63640 1 -- 172.21.15.43:0/3674038630 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb86c06b4d0 msgr2=0x7fb86c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:40.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.124+0000 7fb890d63640 1 --2- 172.21.15.43:0/3674038630 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb86c06b4d0 0x7fb86c06d990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb87c034000 tx=0x7fb87c02e000 comp rx=0 tx=0).stop 2024-04-25T08:54:40.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.124+0000 7fb890d63640 1 -- 172.21.15.43:0/3674038630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb88c15dd00 msgr2=0x7fb88c0b1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:40.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.124+0000 7fb890d63640 1 --2- 172.21.15.43:0/3674038630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb88c15dd00 0x7fb88c0b1610 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb884005c00 tx=0x7fb88400a430 comp rx=0 tx=0).stop 2024-04-25T08:54:40.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.124+0000 7fb890d63640 1 -- 172.21.15.43:0/3674038630 shutdown_connections 2024-04-25T08:54:40.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.124+0000 7fb890d63640 1 --2- 172.21.15.43:0/3674038630 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb88c15dd00 0x7fb88c0b1610 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:40.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.124+0000 7fb890d63640 1 --2- 172.21.15.43:0/3674038630 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb86c06b4d0 0x7fb86c06d990 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:40.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.124+0000 7fb890d63640 1 --2- 172.21.15.43:0/3674038630 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb88c0d09d0 0x7fb88c1602e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:40.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.124+0000 7fb890d63640 1 --2- 172.21.15.43:0/3674038630 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb88c0cf0a0 0x7fb88c0cb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:40.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.124+0000 7fb890d63640 1 -- 172.21.15.43:0/3674038630 >> 172.21.15.43:0/3674038630 conn(0x7fb88c09cd20 msgr2=0x7fb88c0b3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:40.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.124+0000 7fb890d63640 1 -- 172.21.15.43:0/3674038630 shutdown_connections 2024-04-25T08:54:40.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.124+0000 7fb890d63640 1 -- 172.21.15.43:0/3674038630 wait complete. 2024-04-25T08:54:40.125 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:54:40.125 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:54:40.176 DEBUG:teuthology.orchestra.run.smithi193:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:54:40.217 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:54:40.217 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:54:40.217 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:54:40.234 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:54:40.234 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:54:40.289 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:54:40.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.440+0000 7f8034d29640 1 Processor -- start 2024-04-25T08:54:40.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.440+0000 7f8034d29640 1 -- start start 2024-04-25T08:54:40.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.441+0000 7f8034d29640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f803015a9d0 0x7f803006d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:40.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.441+0000 7f8034d29640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8030072160 0x7f80300cd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:40.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.441+0000 7f8034d29640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80300704f0 con 0x7f803006d7d0 2024-04-25T08:54:40.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.441+0000 7f8034d29640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f803006dc30 con 0x7f803015a9d0 2024-04-25T08:54:40.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.441+0000 7f8034d29640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80300707d0 con 0x7f8030072160 2024-04-25T08:54:40.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.441+0000 7f802e59c640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f803015a9d0 0x7f803006d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:40.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.441+0000 7f8025d9b640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f803006d7d0 0x7f8030071a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:38672/0 (socket says 172.21.15.43:38672) 2024-04-25T08:54:40.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.441+0000 7f8025d9b640 1 -- 172.21.15.43:0/2992355027 learned_addr learned my addr 172.21.15.43:0/2992355027 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:40.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.441+0000 7f802ed9d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8030072160 0x7f80300cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:40.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.441+0000 7f802ed9d640 1 -- 172.21.15.43:0/2992355027 >> v2:172.21.15.193:3300/0 conn(0x7f803015a9d0 msgr2=0x7f803006d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:40.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.441+0000 7f802ed9d640 1 --2- 172.21.15.43:0/2992355027 >> v2:172.21.15.193:3300/0 conn(0x7f803015a9d0 0x7f803006d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:40.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.441+0000 7f802ed9d640 1 -- 172.21.15.43:0/2992355027 >> v1:172.21.15.193:6789/0 conn(0x7f803006d7d0 legacy=0x7f8030071a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:40.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.441+0000 7f802ed9d640 1 -- 172.21.15.43:0/2992355027 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8030070ab0 con 0x7f8030072160 2024-04-25T08:54:40.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.441+0000 7f802e59c640 1 --2- 172.21.15.43:0/2992355027 >> v2:172.21.15.193:3300/0 conn(0x7f803015a9d0 0x7f803006d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:54:40.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.442+0000 7f802ed9d640 1 --2- 172.21.15.43:0/2992355027 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8030072160 0x7f80300cd940 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8028009d90 tx=0x7f802802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a37a21b1408e050a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:40.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.442+0000 7f802dd9b640 1 -- 172.21.15.43:0/2992355027 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8028007740 con 0x7f8030072160 2024-04-25T08:54:40.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.442+0000 7f802dd9b640 1 -- 172.21.15.43:0/2992355027 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80280078e0 con 0x7f8030072160 2024-04-25T08:54:40.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.442+0000 7f802dd9b640 1 -- 172.21.15.43:0/2992355027 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8028007cb0 con 0x7f8030072160 2024-04-25T08:54:40.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.443+0000 7f8034d29640 1 -- 172.21.15.43:0/2992355027 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8030072160 msgr2=0x7f80300cd940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:40.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.443+0000 7f8034d29640 1 --2- 172.21.15.43:0/2992355027 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8030072160 0x7f80300cd940 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f8028009d90 tx=0x7f802802a660 comp rx=0 tx=0).stop 2024-04-25T08:54:40.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.443+0000 7f8034d29640 1 -- 172.21.15.43:0/2992355027 shutdown_connections 2024-04-25T08:54:40.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.443+0000 7f8034d29640 1 --2- 172.21.15.43:0/2992355027 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8030072160 0x7f80300cd940 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:40.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.443+0000 7f8034d29640 1 --2- 172.21.15.43:0/2992355027 >> v2:172.21.15.193:3300/0 conn(0x7f803015a9d0 0x7f803006d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:40.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.443+0000 7f8034d29640 1 -- 172.21.15.43:0/2992355027 >> 172.21.15.43:0/2992355027 conn(0x7f80300b1480 msgr2=0x7f80300b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:40.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.443+0000 7f8034d29640 1 -- 172.21.15.43:0/2992355027 shutdown_connections 2024-04-25T08:54:40.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.443+0000 7f8034d29640 1 -- 172.21.15.43:0/2992355027 wait complete. 2024-04-25T08:54:40.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.444+0000 7f8034d29640 1 Processor -- start 2024-04-25T08:54:40.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.444+0000 7f8034d29640 1 -- start start 2024-04-25T08:54:40.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.445+0000 7f8034d29640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f803006d7d0 0x7f8030140ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:40.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.445+0000 7f8034d29640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8030072160 0x7f803015ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:40.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.445+0000 7f8034d29640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f803015d340 0x7f80301f4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:40.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.445+0000 7f8034d29640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80300cde80 con 0x7f803006d7d0 2024-04-25T08:54:40.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.445+0000 7f8034d29640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80300712b0 con 0x7f8030072160 2024-04-25T08:54:40.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.445+0000 7f8034d29640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80300ce000 con 0x7f803015d340 2024-04-25T08:54:40.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.445+0000 7f802e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f803006d7d0 0x7f8030140ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:40.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.445+0000 7f802e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f803006d7d0 0x7f8030140ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38098/0 (socket says 172.21.15.43:38098) 2024-04-25T08:54:40.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.445+0000 7f802e59c640 1 -- 172.21.15.43:0/4266199201 learned_addr learned my addr 172.21.15.43:0/4266199201 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:40.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.446+0000 7f802ed9d640 1 --2- 172.21.15.43:0/4266199201 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f803015d340 0x7f80301f4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:40.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.446+0000 7f8025d9b640 1 --2- 172.21.15.43:0/4266199201 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8030072160 0x7f803015ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:40.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.446+0000 7f802e59c640 1 -- 172.21.15.43:0/4266199201 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f803015d340 msgr2=0x7f80301f4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:40.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.446+0000 7f802e59c640 1 --2- 172.21.15.43:0/4266199201 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f803015d340 0x7f80301f4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:40.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.446+0000 7f802e59c640 1 -- 172.21.15.43:0/4266199201 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8030072160 msgr2=0x7f803015ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:40.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.446+0000 7f802e59c640 1 --2- 172.21.15.43:0/4266199201 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8030072160 0x7f803015ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:40.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.446+0000 7f802e59c640 1 -- 172.21.15.43:0/4266199201 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80301f5c20 con 0x7f803006d7d0 2024-04-25T08:54:40.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.446+0000 7f8025d9b640 1 --2- 172.21.15.43:0/4266199201 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8030072160 0x7f803015ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:40.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.446+0000 7f802e59c640 1 --2- 172.21.15.43:0/4266199201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f803006d7d0 0x7f8030140ea0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f802000e960 tx=0x7f802000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:40.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.446+0000 7f80277fe640 1 -- 172.21.15.43:0/4266199201 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8020017070 con 0x7f803006d7d0 2024-04-25T08:54:40.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.446+0000 7f80277fe640 1 -- 172.21.15.43:0/4266199201 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f802000ca00 con 0x7f803006d7d0 2024-04-25T08:54:40.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.446+0000 7f80277fe640 1 -- 172.21.15.43:0/4266199201 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802000cdd0 con 0x7f803006d7d0 2024-04-25T08:54:40.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.446+0000 7f8034d29640 1 -- 172.21.15.43:0/4266199201 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80301f5940 con 0x7f803006d7d0 2024-04-25T08:54:40.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.446+0000 7f8034d29640 1 -- 172.21.15.43:0/4266199201 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80301e3370 con 0x7f803006d7d0 2024-04-25T08:54:40.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.447+0000 7f8034d29640 1 -- 172.21.15.43:0/4266199201 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8030074ef0 con 0x7f803006d7d0 2024-04-25T08:54:40.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.447+0000 7f80277fe640 1 -- 172.21.15.43:0/4266199201 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f802001a070 con 0x7f803006d7d0 2024-04-25T08:54:40.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.448+0000 7f80277fe640 1 --2- 172.21.15.43:0/4266199201 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f801006b4d0 0x7f801006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:40.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.448+0000 7f80277fe640 1 -- 172.21.15.43:0/4266199201 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f802008a0d0 con 0x7f803006d7d0 2024-04-25T08:54:40.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.448+0000 7f8025d9b640 1 --2- 172.21.15.43:0/4266199201 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f801006b4d0 0x7f801006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:40.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.449+0000 7f8025d9b640 1 --2- 172.21.15.43:0/4266199201 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f801006b4d0 0x7f801006d990 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8030155170 tx=0x7f8018009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:40.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.452+0000 7f80277fe640 1 -- 172.21.15.43:0/4266199201 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8020058f50 con 0x7f803006d7d0 2024-04-25T08:54:40.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.631+0000 7f8034d29640 1 -- 172.21.15.43:0/4266199201 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f80300cf290 con 0x7f803006d7d0 2024-04-25T08:54:40.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.632+0000 7f80277fe640 1 -- 172.21.15.43:0/4266199201 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16569 (secure 0 0 0) 0x7f8020058850 con 0x7f803006d7d0 2024-04-25T08:54:40.623 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:54:40.623 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":20,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:54:13.593188+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:54:09.757978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:54:10.926008+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:54:40.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.635+0000 7f8034d29640 1 -- 172.21.15.43:0/4266199201 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f801006b4d0 msgr2=0x7f801006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:40.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.635+0000 7f8034d29640 1 --2- 172.21.15.43:0/4266199201 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f801006b4d0 0x7f801006d990 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8030155170 tx=0x7f8018009040 comp rx=0 tx=0).stop 2024-04-25T08:54:40.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.635+0000 7f8034d29640 1 -- 172.21.15.43:0/4266199201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f803006d7d0 msgr2=0x7f8030140ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:40.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.635+0000 7f8034d29640 1 --2- 172.21.15.43:0/4266199201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f803006d7d0 0x7f8030140ea0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f802000e960 tx=0x7f802000ee20 comp rx=0 tx=0).stop 2024-04-25T08:54:40.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.635+0000 7f8034d29640 1 -- 172.21.15.43:0/4266199201 shutdown_connections 2024-04-25T08:54:40.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.635+0000 7f8034d29640 1 --2- 172.21.15.43:0/4266199201 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f803015d340 0x7f80301f4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:40.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.635+0000 7f8034d29640 1 --2- 172.21.15.43:0/4266199201 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f801006b4d0 0x7f801006d990 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:40.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.635+0000 7f8034d29640 1 --2- 172.21.15.43:0/4266199201 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8030072160 0x7f803015ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:40.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.635+0000 7f8034d29640 1 --2- 172.21.15.43:0/4266199201 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f803006d7d0 0x7f8030140ea0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:40.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.635+0000 7f8034d29640 1 -- 172.21.15.43:0/4266199201 >> 172.21.15.43:0/4266199201 conn(0x7f80300b1480 msgr2=0x7f80300b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:40.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.635+0000 7f8034d29640 1 -- 172.21.15.43:0/4266199201 shutdown_connections 2024-04-25T08:54:40.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.635+0000 7f8034d29640 1 -- 172.21.15.43:0/4266199201 wait complete. 2024-04-25T08:54:40.636 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:54:09.757978+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:54:10.926008+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:54:40.636 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:54:40.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.749+0000 7fe8b5b22640 1 Processor -- start 2024-04-25T08:54:40.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.749+0000 7fe8b5b22640 1 -- start start 2024-04-25T08:54:40.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.750+0000 7fe8b5b22640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fe8b015dd80 0x7fe8b00cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:40.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.750+0000 7fe8b5b22640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8b00b2fe0 con 0x7fe8b00d17b0 2024-04-25T08:54:40.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.750+0000 7fe8b5b22640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8b00ced70 con 0x7fe8b00b1cd0 2024-04-25T08:54:40.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.750+0000 7fe8b5b22640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8b00cfc60 con 0x7fe8b015dd80 2024-04-25T08:54:40.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.750+0000 7fe8aeffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe8b00d17b0 0x7fe8b015e160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39634/0 (socket says 172.21.15.43:39634) 2024-04-25T08:54:40.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.750+0000 7fe8aeffd640 1 -- 172.21.15.43:0/3041871595 learned_addr learned my addr 172.21.15.43:0/3041871595 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:40.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.751+0000 7fe8af7fe640 1 --2- 172.21.15.43:0/3041871595 >> v2:172.21.15.193:3300/0 conn(0x7fe8b015dd80 0x7fe8b00cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:40.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.751+0000 7fe8ae7fc640 1 -- 172.21.15.43:0/3041871595 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1624742582 0 0) 0x7fe8b00b2fe0 con 0x7fe8b00d17b0 2024-04-25T08:54:40.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.751+0000 7fe8ae7fc640 1 -- 172.21.15.43:0/3041871595 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe898003520 con 0x7fe8b00d17b0 2024-04-25T08:54:40.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.751+0000 7fe8ae7fc640 1 -- 172.21.15.43:0/3041871595 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1202403326 0 0) 0x7fe898003520 con 0x7fe8b00d17b0 2024-04-25T08:54:40.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.751+0000 7fe8af7fe640 1 -- 172.21.15.43:0/3041871595 >> v1:172.21.15.193:6789/0 conn(0x7fe8b00b1cd0 legacy=0x7fe8b015e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:40.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.751+0000 7fe8af7fe640 1 -- 172.21.15.43:0/3041871595 >> v1:172.21.15.43:6789/0 conn(0x7fe8b00d17b0 legacy=0x7fe8b015e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:40.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.751+0000 7fe8af7fe640 1 -- 172.21.15.43:0/3041871595 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b00cf050 con 0x7fe8b015dd80 2024-04-25T08:54:40.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.752+0000 7fe8af7fe640 1 --2- 172.21.15.43:0/3041871595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8b015dd80 0x7fe8b00cf720 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe8a00097a0 tx=0x7fe8a002a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e4e2eb68482ae0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:40.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.752+0000 7fe8ae7fc640 1 -- 172.21.15.43:0/3041871595 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0037070 con 0x7fe8b015dd80 2024-04-25T08:54:40.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.752+0000 7fe8ae7fc640 1 -- 172.21.15.43:0/3041871595 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a002aad0 con 0x7fe8b015dd80 2024-04-25T08:54:40.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.752+0000 7fe8ae7fc640 1 -- 172.21.15.43:0/3041871595 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a4001670 con 0x7fe8b015dd80 2024-04-25T08:54:40.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.752+0000 7fe8b5b22640 1 -- 172.21.15.43:0/3041871595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8b015dd80 msgr2=0x7fe8b00cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:40.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.752+0000 7fe8b5b22640 1 --2- 172.21.15.43:0/3041871595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8b015dd80 0x7fe8b00cf720 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe8a00097a0 tx=0x7fe8a002a040 comp rx=0 tx=0).stop 2024-04-25T08:54:40.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.752+0000 7fe8b5b22640 1 -- 172.21.15.43:0/3041871595 shutdown_connections 2024-04-25T08:54:40.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.752+0000 7fe8b5b22640 1 --2- 172.21.15.43:0/3041871595 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8b015dd80 0x7fe8b00cf720 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:40.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.752+0000 7fe8b5b22640 1 -- 172.21.15.43:0/3041871595 >> 172.21.15.43:0/3041871595 conn(0x7fe8b0072aa0 msgr2=0x7fe8b009aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:40.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.753+0000 7fe8b5b22640 1 -- 172.21.15.43:0/3041871595 shutdown_connections 2024-04-25T08:54:40.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.753+0000 7fe8b5b22640 1 -- 172.21.15.43:0/3041871595 wait complete. 2024-04-25T08:54:40.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.754+0000 7fe8b5b22640 1 Processor -- start 2024-04-25T08:54:40.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.754+0000 7fe8b5b22640 1 -- start start 2024-04-25T08:54:40.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.755+0000 7fe8b5b22640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe8b00b1cd0 0x7fe8b0154cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:40.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.755+0000 7fe8b5b22640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8b00d17b0 0x7fe8b00b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:40.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.755+0000 7fe8b5b22640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8b015dd80 0x7fe8b01606d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:40.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.755+0000 7fe8b5b22640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b00b2c40 con 0x7fe8b015dd80 2024-04-25T08:54:40.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.755+0000 7fe8b5b22640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b00b32c0 con 0x7fe8b00d17b0 2024-04-25T08:54:40.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.755+0000 7fe8b5b22640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8b00b2dc0 con 0x7fe8b00b1cd0 2024-04-25T08:54:40.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.755+0000 7fe8affff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8b015dd80 0x7fe8b01606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:40.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.755+0000 7fe8affff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8b015dd80 0x7fe8b01606d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38100/0 (socket says 172.21.15.43:38100) 2024-04-25T08:54:40.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.755+0000 7fe8affff640 1 -- 172.21.15.43:0/2275992086 learned_addr learned my addr 172.21.15.43:0/2275992086 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:40.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.755+0000 7fe8affff640 1 -- 172.21.15.43:0/2275992086 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe8b00b1cd0 msgr2=0x7fe8b0154cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:40.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.755+0000 7fe8affff640 1 --2- 172.21.15.43:0/2275992086 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe8b00b1cd0 0x7fe8b0154cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:40.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.755+0000 7fe8affff640 1 -- 172.21.15.43:0/2275992086 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8b00d17b0 msgr2=0x7fe8b00b44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:40.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.755+0000 7fe8affff640 1 --2- 172.21.15.43:0/2275992086 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8b00d17b0 0x7fe8b00b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:40.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.755+0000 7fe8affff640 1 -- 172.21.15.43:0/2275992086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b01f3210 con 0x7fe8b015dd80 2024-04-25T08:54:40.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.755+0000 7fe8affff640 1 --2- 172.21.15.43:0/2275992086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8b015dd80 0x7fe8b01606d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe8a800cb20 tx=0x7fe8a8002090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:40.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.755+0000 7fe897fff640 1 -- 172.21.15.43:0/2275992086 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8018070 con 0x7fe8b015dd80 2024-04-25T08:54:40.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.755+0000 7fe897fff640 1 -- 172.21.15.43:0/2275992086 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a8003da0 con 0x7fe8b015dd80 2024-04-25T08:54:40.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.755+0000 7fe8b5b22640 1 -- 172.21.15.43:0/2275992086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8b0155230 con 0x7fe8b015dd80 2024-04-25T08:54:40.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.755+0000 7fe897fff640 1 -- 172.21.15.43:0/2275992086 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a8006d70 con 0x7fe8b015dd80 2024-04-25T08:54:40.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.756+0000 7fe8b5b22640 1 -- 172.21.15.43:0/2275992086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8b00b5480 con 0x7fe8b015dd80 2024-04-25T08:54:40.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.756+0000 7fe8b5b22640 1 -- 172.21.15.43:0/2275992086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8b0074f60 con 0x7fe8b015dd80 2024-04-25T08:54:40.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.757+0000 7fe897fff640 1 -- 172.21.15.43:0/2275992086 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe8a801b070 con 0x7fe8b015dd80 2024-04-25T08:54:40.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.757+0000 7fe897fff640 1 --2- 172.21.15.43:0/2275992086 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe89c06b4d0 0x7fe89c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:40.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.757+0000 7fe897fff640 1 -- 172.21.15.43:0/2275992086 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe8a8089880 con 0x7fe8b015dd80 2024-04-25T08:54:40.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.758+0000 7fe8af7fe640 1 --2- 172.21.15.43:0/2275992086 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe89c06b4d0 0x7fe89c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:40.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.758+0000 7fe8af7fe640 1 --2- 172.21.15.43:0/2275992086 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe89c06b4d0 0x7fe89c06d990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0034000 tx=0x7fe8a002c000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:40.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.762+0000 7fe897fff640 1 -- 172.21.15.43:0/2275992086 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe8a8058700 con 0x7fe8b015dd80 2024-04-25T08:54:40.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.941+0000 7fe8b5b22640 1 -- 172.21.15.43:0/2275992086 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe8b00743d0 con 0x7fe8b015dd80 2024-04-25T08:54:40.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.941+0000 7fe897fff640 1 -- 172.21.15.43:0/2275992086 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe8a8058000 con 0x7fe8b015dd80 2024-04-25T08:54:40.932 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T08:54:40.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.944+0000 7fe8b5b22640 1 -- 172.21.15.43:0/2275992086 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe89c06b4d0 msgr2=0x7fe89c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:40.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.944+0000 7fe8b5b22640 1 --2- 172.21.15.43:0/2275992086 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe89c06b4d0 0x7fe89c06d990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0034000 tx=0x7fe8a002c000 comp rx=0 tx=0).stop 2024-04-25T08:54:40.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.944+0000 7fe8b5b22640 1 -- 172.21.15.43:0/2275992086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8b015dd80 msgr2=0x7fe8b01606d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:40.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.944+0000 7fe8b5b22640 1 --2- 172.21.15.43:0/2275992086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8b015dd80 0x7fe8b01606d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fe8a800cb20 tx=0x7fe8a8002090 comp rx=0 tx=0).stop 2024-04-25T08:54:40.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.944+0000 7fe8b5b22640 1 -- 172.21.15.43:0/2275992086 shutdown_connections 2024-04-25T08:54:40.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.944+0000 7fe8b5b22640 1 --2- 172.21.15.43:0/2275992086 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8b015dd80 0x7fe8b01606d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:40.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.944+0000 7fe8b5b22640 1 --2- 172.21.15.43:0/2275992086 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe89c06b4d0 0x7fe89c06d990 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:40.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.944+0000 7fe8b5b22640 1 --2- 172.21.15.43:0/2275992086 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8b00d17b0 0x7fe8b00b44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:40.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.944+0000 7fe8b5b22640 1 --2- 172.21.15.43:0/2275992086 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe8b00b1cd0 0x7fe8b0154cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:40.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.944+0000 7fe8b5b22640 1 -- 172.21.15.43:0/2275992086 >> 172.21.15.43:0/2275992086 conn(0x7fe8b0072aa0 msgr2=0x7fe8b00d1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:40.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.944+0000 7fe8b5b22640 1 -- 172.21.15.43:0/2275992086 shutdown_connections 2024-04-25T08:54:40.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:40.944+0000 7fe8b5b22640 1 -- 172.21.15.43:0/2275992086 wait complete. 2024-04-25T08:54:40.945 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:54:41.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.058+0000 7f0a20415640 1 Processor -- start 2024-04-25T08:54:41.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.058+0000 7f0a20415640 1 -- start start 2024-04-25T08:54:41.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.059+0000 7f0a20415640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0a180d1a20 0x7f0a180b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:41.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.059+0000 7f0a20415640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0a180d1e00 0x7f0a180d2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:41.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.059+0000 7f0a20415640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a1815bf90 con 0x7f0a180b5100 2024-04-25T08:54:41.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.059+0000 7f0a20415640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a180cf430 con 0x7f0a180d1a20 2024-04-25T08:54:41.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.059+0000 7f0a20415640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a180cee70 con 0x7f0a180d1e00 2024-04-25T08:54:41.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.059+0000 7f0a1e1b1640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0a180d1a20 0x7f0a180b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:41.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.059+0000 7f0a1e1b1640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0a180d1a20 0x7f0a180b2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38116/0 (socket says 172.21.15.43:38116) 2024-04-25T08:54:41.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.059+0000 7f0a1e1b1640 1 -- 172.21.15.43:0/2952313345 learned_addr learned my addr 172.21.15.43:0/2952313345 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:41.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.060+0000 7f0a1e1b1640 1 -- 172.21.15.43:0/2952313345 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0a180d1e00 msgr2=0x7f0a180d2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:41.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.060+0000 7f0a1d9b0640 1 --2- 172.21.15.43:0/2952313345 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0a180d1e00 0x7f0a180d2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:41.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.060+0000 7f0a1e1b1640 1 --2- 172.21.15.43:0/2952313345 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0a180d1e00 0x7f0a180d2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.060+0000 7f0a1e1b1640 1 -- 172.21.15.43:0/2952313345 >> v1:172.21.15.193:6789/0 conn(0x7f0a180b5100 legacy=0x7f0a180b55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:41.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.060+0000 7f0a1e1b1640 1 -- 172.21.15.43:0/2952313345 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a180cf150 con 0x7f0a180d1a20 2024-04-25T08:54:41.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.060+0000 7f0a1d9b0640 1 --2- 172.21.15.43:0/2952313345 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0a180d1e00 0x7f0a180d2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:41.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.060+0000 7f0a1e1b1640 1 --2- 172.21.15.43:0/2952313345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a180d1a20 0x7f0a180b2b90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0a10009870 tx=0x7f0a1002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c44fd4f4190fc463 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:41.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.061+0000 7f0a1d1af640 1 -- 172.21.15.43:0/2952313345 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a10037070 con 0x7f0a180d1a20 2024-04-25T08:54:41.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.061+0000 7f0a1d1af640 1 -- 172.21.15.43:0/2952313345 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a1002ae10 con 0x7f0a180d1a20 2024-04-25T08:54:41.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.061+0000 7f0a1d1af640 1 -- 172.21.15.43:0/2952313345 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a10032660 con 0x7f0a180d1a20 2024-04-25T08:54:41.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.061+0000 7f0a20415640 1 -- 172.21.15.43:0/2952313345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a180d1a20 msgr2=0x7f0a180b2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:41.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.061+0000 7f0a20415640 1 --2- 172.21.15.43:0/2952313345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a180d1a20 0x7f0a180b2b90 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0a10009870 tx=0x7f0a1002a100 comp rx=0 tx=0).stop 2024-04-25T08:54:41.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.061+0000 7f0a20415640 1 -- 172.21.15.43:0/2952313345 shutdown_connections 2024-04-25T08:54:41.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.061+0000 7f0a20415640 1 --2- 172.21.15.43:0/2952313345 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0a180d1e00 0x7f0a180d2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.061+0000 7f0a20415640 1 --2- 172.21.15.43:0/2952313345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a180d1a20 0x7f0a180b2b90 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.061+0000 7f0a20415640 1 -- 172.21.15.43:0/2952313345 >> 172.21.15.43:0/2952313345 conn(0x7f0a18072aa0 msgr2=0x7f0a1809aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:41.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.061+0000 7f0a20415640 1 -- 172.21.15.43:0/2952313345 shutdown_connections 2024-04-25T08:54:41.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.061+0000 7f0a20415640 1 -- 172.21.15.43:0/2952313345 wait complete. 2024-04-25T08:54:41.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.062+0000 7f0a20415640 1 Processor -- start 2024-04-25T08:54:41.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.062+0000 7f0a20415640 1 -- start start 2024-04-25T08:54:41.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.063+0000 7f0a20415640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a180b5100 0x7f0a180cab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:41.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.063+0000 7f0a20415640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0a180d1a20 0x7f0a1814d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:41.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.063+0000 7f0a20415640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a180d1e00 0x7f0a1806d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:41.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.063+0000 7f0a20415640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a180cea60 con 0x7f0a180b5100 2024-04-25T08:54:41.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.063+0000 7f0a20415640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a180b4be0 con 0x7f0a180d1e00 2024-04-25T08:54:41.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.063+0000 7f0a20415640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a180cebe0 con 0x7f0a180d1a20 2024-04-25T08:54:41.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.063+0000 7f0a1e1b1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a180b5100 0x7f0a180cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:41.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.063+0000 7f0a1e1b1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a180b5100 0x7f0a180cab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38122/0 (socket says 172.21.15.43:38122) 2024-04-25T08:54:41.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.063+0000 7f0a1e1b1640 1 -- 172.21.15.43:0/3863799606 learned_addr learned my addr 172.21.15.43:0/3863799606 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:41.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.064+0000 7f0a1e1b1640 1 -- 172.21.15.43:0/3863799606 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0a180d1a20 msgr2=0x7f0a1814d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:41.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.064+0000 7f0a1e1b1640 1 --2- 172.21.15.43:0/3863799606 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0a180d1a20 0x7f0a1814d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.064+0000 7f0a1e1b1640 1 -- 172.21.15.43:0/3863799606 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a180d1e00 msgr2=0x7f0a1806d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:41.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.064+0000 7f0a1e1b1640 1 --2- 172.21.15.43:0/3863799606 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a180d1e00 0x7f0a1806d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.064+0000 7f0a1e1b1640 1 -- 172.21.15.43:0/3863799606 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a181f3d70 con 0x7f0a180b5100 2024-04-25T08:54:41.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.064+0000 7f0a1e1b1640 1 --2- 172.21.15.43:0/3863799606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a180b5100 0x7f0a180cab50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0a10009870 tx=0x7f0a100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:41.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.064+0000 7f0a16ffd640 1 -- 172.21.15.43:0/3863799606 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a10041070 con 0x7f0a180b5100 2024-04-25T08:54:41.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.064+0000 7f0a16ffd640 1 -- 172.21.15.43:0/3863799606 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a1003c680 con 0x7f0a180b5100 2024-04-25T08:54:41.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.064+0000 7f0a20415640 1 -- 172.21.15.43:0/3863799606 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a181f4050 con 0x7f0a180b5100 2024-04-25T08:54:41.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.064+0000 7f0a16ffd640 1 -- 172.21.15.43:0/3863799606 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a10037040 con 0x7f0a180b5100 2024-04-25T08:54:41.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.064+0000 7f0a20415640 1 -- 172.21.15.43:0/3863799606 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a181f4610 con 0x7f0a180b5100 2024-04-25T08:54:41.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.065+0000 7f0a20415640 1 -- 172.21.15.43:0/3863799606 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a181f4850 con 0x7f0a180b5100 2024-04-25T08:54:41.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.065+0000 7f0a16ffd640 1 -- 172.21.15.43:0/3863799606 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0a10044070 con 0x7f0a180b5100 2024-04-25T08:54:41.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.066+0000 7f0a16ffd640 1 --2- 172.21.15.43:0/3863799606 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0a0406b4d0 0x7f0a0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:41.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.066+0000 7f0a16ffd640 1 -- 172.21.15.43:0/3863799606 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0a100aaa00 con 0x7f0a180b5100 2024-04-25T08:54:41.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.066+0000 7f0a1d9b0640 1 --2- 172.21.15.43:0/3863799606 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0a0406b4d0 0x7f0a0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:41.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.067+0000 7f0a1d9b0640 1 --2- 172.21.15.43:0/3863799606 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0a0406b4d0 0x7f0a0406d990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c006fd0 tx=0x7f0a0c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:41.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.070+0000 7f0a16ffd640 1 -- 172.21.15.43:0/3863799606 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0a181f4850 con 0x7f0a180b5100 2024-04-25T08:54:41.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.252+0000 7f0a20415640 1 -- 172.21.15.43:0/3863799606 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0a180cdbe0 con 0x7f0a180b5100 2024-04-25T08:54:41.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.253+0000 7f0a16ffd640 1 -- 172.21.15.43:0/3863799606 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0a10079880 con 0x7f0a180b5100 2024-04-25T08:54:41.243 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T08:54:41.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.255+0000 7f0a20415640 1 -- 172.21.15.43:0/3863799606 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0a0406b4d0 msgr2=0x7f0a0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:41.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.255+0000 7f0a20415640 1 --2- 172.21.15.43:0/3863799606 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0a0406b4d0 0x7f0a0406d990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c006fd0 tx=0x7f0a0c008040 comp rx=0 tx=0).stop 2024-04-25T08:54:41.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.255+0000 7f0a20415640 1 -- 172.21.15.43:0/3863799606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a180b5100 msgr2=0x7f0a180cab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:41.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.255+0000 7f0a20415640 1 --2- 172.21.15.43:0/3863799606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a180b5100 0x7f0a180cab50 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f0a10009870 tx=0x7f0a100046c0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.255+0000 7f0a20415640 1 -- 172.21.15.43:0/3863799606 shutdown_connections 2024-04-25T08:54:41.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.255+0000 7f0a20415640 1 --2- 172.21.15.43:0/3863799606 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0a0406b4d0 0x7f0a0406d990 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.255+0000 7f0a20415640 1 --2- 172.21.15.43:0/3863799606 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a180d1e00 0x7f0a1806d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.255+0000 7f0a20415640 1 --2- 172.21.15.43:0/3863799606 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0a180d1a20 0x7f0a1814d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.255+0000 7f0a20415640 1 --2- 172.21.15.43:0/3863799606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a180b5100 0x7f0a180cab50 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.255+0000 7f0a20415640 1 -- 172.21.15.43:0/3863799606 >> 172.21.15.43:0/3863799606 conn(0x7f0a18072aa0 msgr2=0x7f0a1806fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:41.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.255+0000 7f0a20415640 1 -- 172.21.15.43:0/3863799606 shutdown_connections 2024-04-25T08:54:41.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.255+0000 7f0a20415640 1 -- 172.21.15.43:0/3863799606 wait complete. 2024-04-25T08:54:41.256 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:54:41.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.369+0000 7fba1e476640 1 Processor -- start 2024-04-25T08:54:41.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.369+0000 7fba1e476640 1 -- start start 2024-04-25T08:54:41.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.370+0000 7fba1e476640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fba1815e090 0x7fba18161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:41.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.370+0000 7fba1e476640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fba180b5c70 0x7fba180d2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:41.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.370+0000 7fba1e476640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba18162610 con 0x7fba18161fe0 2024-04-25T08:54:41.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.370+0000 7fba1e476640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba180d1da0 con 0x7fba180b5c70 2024-04-25T08:54:41.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.370+0000 7fba1e476640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba180d1750 con 0x7fba1815e090 2024-04-25T08:54:41.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.370+0000 7fba1ca13640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fba180b5c70 0x7fba180d2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:41.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.370+0000 7fba1ca13640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fba180b5c70 0x7fba180d2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38138/0 (socket says 172.21.15.43:38138) 2024-04-25T08:54:41.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.370+0000 7fba177fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fba18161fe0 0x7fba180b5560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39648/0 (socket says 172.21.15.43:39648) 2024-04-25T08:54:41.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.370+0000 7fba1ca13640 1 -- 172.21.15.43:0/3328895747 learned_addr learned my addr 172.21.15.43:0/3328895747 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:41.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.370+0000 7fba17fff640 1 --2- 172.21.15.43:0/3328895747 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fba1815e090 0x7fba18161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:41.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.371+0000 7fba1ca13640 1 -- 172.21.15.43:0/3328895747 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fba1815e090 msgr2=0x7fba18161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:41.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.371+0000 7fba1ca13640 1 --2- 172.21.15.43:0/3328895747 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fba1815e090 0x7fba18161aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.371+0000 7fba1ca13640 1 -- 172.21.15.43:0/3328895747 >> v1:172.21.15.43:6789/0 conn(0x7fba18161fe0 legacy=0x7fba180b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:41.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.371+0000 7fba16ffd640 1 -- 172.21.15.43:0/3328895747 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3879556581 0 0) 0x7fba18162610 con 0x7fba18161fe0 2024-04-25T08:54:41.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.371+0000 7fba1ca13640 1 -- 172.21.15.43:0/3328895747 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba180d1a30 con 0x7fba180b5c70 2024-04-25T08:54:41.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.371+0000 7fba17fff640 1 --2- 172.21.15.43:0/3328895747 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fba1815e090 0x7fba18161aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:41.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.371+0000 7fba1ca13640 1 --2- 172.21.15.43:0/3328895747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba180b5c70 0x7fba180d2230 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fba10009e50 tx=0x7fba1002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0d0391b296d1bf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:41.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.372+0000 7fba16ffd640 1 -- 172.21.15.43:0/3328895747 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba10036070 con 0x7fba180b5c70 2024-04-25T08:54:41.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.372+0000 7fba16ffd640 1 -- 172.21.15.43:0/3328895747 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba10031600 con 0x7fba180b5c70 2024-04-25T08:54:41.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.372+0000 7fba16ffd640 1 -- 172.21.15.43:0/3328895747 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba100319d0 con 0x7fba180b5c70 2024-04-25T08:54:41.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.372+0000 7fba1e476640 1 -- 172.21.15.43:0/3328895747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba180b5c70 msgr2=0x7fba180d2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:41.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.372+0000 7fba1e476640 1 --2- 172.21.15.43:0/3328895747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba180b5c70 0x7fba180d2230 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fba10009e50 tx=0x7fba1002a7a0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.372+0000 7fba1e476640 1 -- 172.21.15.43:0/3328895747 shutdown_connections 2024-04-25T08:54:41.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.372+0000 7fba1e476640 1 --2- 172.21.15.43:0/3328895747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba180b5c70 0x7fba180d2230 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.372+0000 7fba1e476640 1 --2- 172.21.15.43:0/3328895747 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fba1815e090 0x7fba18161aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.372+0000 7fba1e476640 1 -- 172.21.15.43:0/3328895747 >> 172.21.15.43:0/3328895747 conn(0x7fba180d0560 msgr2=0x7fba180d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:41.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.372+0000 7fba1e476640 1 -- 172.21.15.43:0/3328895747 shutdown_connections 2024-04-25T08:54:41.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.372+0000 7fba1e476640 1 -- 172.21.15.43:0/3328895747 wait complete. 2024-04-25T08:54:41.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.373+0000 7fba1e476640 1 Processor -- start 2024-04-25T08:54:41.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.374+0000 7fba1e476640 1 -- start start 2024-04-25T08:54:41.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.374+0000 7fba1e476640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fba180b5c70 0x7fba18160e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:41.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.374+0000 7fba1e476640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba1815e090 0x7fba180cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:41.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.374+0000 7fba1e476640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba18161fe0 0x7fba180b7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:41.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.374+0000 7fba1e476640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba180d2770 con 0x7fba1815e090 2024-04-25T08:54:41.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.374+0000 7fba1e476640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba180b3280 con 0x7fba18161fe0 2024-04-25T08:54:41.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.374+0000 7fba1e476640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba180d28f0 con 0x7fba180b5c70 2024-04-25T08:54:41.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.374+0000 7fba177fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba1815e090 0x7fba180cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:41.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.374+0000 7fba177fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba1815e090 0x7fba180cd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38142/0 (socket says 172.21.15.43:38142) 2024-04-25T08:54:41.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.374+0000 7fba177fe640 1 -- 172.21.15.43:0/4052594359 learned_addr learned my addr 172.21.15.43:0/4052594359 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:41.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.375+0000 7fba17fff640 1 --2- 172.21.15.43:0/4052594359 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fba180b5c70 0x7fba18160e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:41.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.375+0000 7fba1ca13640 1 --2- 172.21.15.43:0/4052594359 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba18161fe0 0x7fba180b7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:41.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.375+0000 7fba177fe640 1 -- 172.21.15.43:0/4052594359 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fba180b5c70 msgr2=0x7fba18160e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:41.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.375+0000 7fba177fe640 1 --2- 172.21.15.43:0/4052594359 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fba180b5c70 0x7fba18160e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.375+0000 7fba177fe640 1 -- 172.21.15.43:0/4052594359 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba18161fe0 msgr2=0x7fba180b7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:41.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.375+0000 7fba177fe640 1 --2- 172.21.15.43:0/4052594359 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba18161fe0 0x7fba180b7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.375+0000 7fba177fe640 1 -- 172.21.15.43:0/4052594359 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba181f3e10 con 0x7fba1815e090 2024-04-25T08:54:41.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.375+0000 7fba1ca13640 1 --2- 172.21.15.43:0/4052594359 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba18161fe0 0x7fba180b7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:41.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.375+0000 7fba177fe640 1 --2- 172.21.15.43:0/4052594359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba1815e090 0x7fba180cd9a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fba0000c3f0 tx=0x7fba0000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:41.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.375+0000 7fba14ff9640 1 -- 172.21.15.43:0/4052594359 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba000074e0 con 0x7fba1815e090 2024-04-25T08:54:41.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.375+0000 7fba14ff9640 1 -- 172.21.15.43:0/4052594359 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba000047e0 con 0x7fba1815e090 2024-04-25T08:54:41.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.375+0000 7fba1e476640 1 -- 172.21.15.43:0/4052594359 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba181f40f0 con 0x7fba1815e090 2024-04-25T08:54:41.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.375+0000 7fba14ff9640 1 -- 172.21.15.43:0/4052594359 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba00004c10 con 0x7fba1815e090 2024-04-25T08:54:41.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.375+0000 7fba1e476640 1 -- 172.21.15.43:0/4052594359 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba181f46b0 con 0x7fba1815e090 2024-04-25T08:54:41.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.376+0000 7fba1e476640 1 -- 172.21.15.43:0/4052594359 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba1806bf00 con 0x7fba1815e090 2024-04-25T08:54:41.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.376+0000 7fba14ff9640 1 -- 172.21.15.43:0/4052594359 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fba00011070 con 0x7fba1815e090 2024-04-25T08:54:41.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.377+0000 7fba14ff9640 1 --2- 172.21.15.43:0/4052594359 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb9f806b500 0x7fb9f806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:41.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.377+0000 7fba14ff9640 1 -- 172.21.15.43:0/4052594359 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fba00089cd0 con 0x7fba1815e090 2024-04-25T08:54:41.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.377+0000 7fba17fff640 1 --2- 172.21.15.43:0/4052594359 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb9f806b500 0x7fb9f806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:41.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.378+0000 7fba17fff640 1 --2- 172.21.15.43:0/4052594359 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb9f806b500 0x7fb9f806d9c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fba180cee10 tx=0x7fba08008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:41.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.381+0000 7fba14ff9640 1 -- 172.21.15.43:0/4052594359 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fba1806bf00 con 0x7fba1815e090 2024-04-25T08:54:41.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.560+0000 7fba1e476640 1 -- 172.21.15.43:0/4052594359 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fba18074f60 con 0x7fba1815e090 2024-04-25T08:54:41.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.560+0000 7fba14ff9640 1 -- 172.21.15.43:0/4052594359 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fba00058b50 con 0x7fba1815e090 2024-04-25T08:54:41.551 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T08:54:41.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.562+0000 7fba1e476640 1 -- 172.21.15.43:0/4052594359 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb9f806b500 msgr2=0x7fb9f806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:41.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.562+0000 7fba1e476640 1 --2- 172.21.15.43:0/4052594359 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb9f806b500 0x7fb9f806d9c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fba180cee10 tx=0x7fba08008040 comp rx=0 tx=0).stop 2024-04-25T08:54:41.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.562+0000 7fba1e476640 1 -- 172.21.15.43:0/4052594359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba1815e090 msgr2=0x7fba180cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:41.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.562+0000 7fba1e476640 1 --2- 172.21.15.43:0/4052594359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba1815e090 0x7fba180cd9a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fba0000c3f0 tx=0x7fba0000c8b0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.562+0000 7fba1e476640 1 -- 172.21.15.43:0/4052594359 shutdown_connections 2024-04-25T08:54:41.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.562+0000 7fba1e476640 1 --2- 172.21.15.43:0/4052594359 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb9f806b500 0x7fb9f806d9c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.562+0000 7fba1e476640 1 --2- 172.21.15.43:0/4052594359 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fba18161fe0 0x7fba180b7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.562+0000 7fba1e476640 1 --2- 172.21.15.43:0/4052594359 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fba1815e090 0x7fba180cd9a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.562+0000 7fba1e476640 1 --2- 172.21.15.43:0/4052594359 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fba180b5c70 0x7fba18160e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.562+0000 7fba1e476640 1 -- 172.21.15.43:0/4052594359 >> 172.21.15.43:0/4052594359 conn(0x7fba180d0560 msgr2=0x7fba180c9620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:41.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.562+0000 7fba1e476640 1 -- 172.21.15.43:0/4052594359 shutdown_connections 2024-04-25T08:54:41.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.562+0000 7fba1e476640 1 -- 172.21.15.43:0/4052594359 wait complete. 2024-04-25T08:54:41.563 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:54:41.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.675+0000 7f0c26bb7640 1 Processor -- start 2024-04-25T08:54:41.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.675+0000 7f0c26bb7640 1 -- start start 2024-04-25T08:54:41.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.676+0000 7f0c26bb7640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0c2015df50 0x7f0c201619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:41.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.676+0000 7f0c26bb7640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0c20161f30 0x7f0c201623a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:41.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.676+0000 7f0c26bb7640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0c200c7cf0 0x7f0c200c8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:41.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.676+0000 7f0c26bb7640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c2015e330 con 0x7f0c2015df50 2024-04-25T08:54:41.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.676+0000 7f0c26bb7640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c200d24a0 con 0x7f0c200c7cf0 2024-04-25T08:54:41.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.676+0000 7f0c26bb7640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c200d2740 con 0x7f0c20161f30 2024-04-25T08:54:41.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.676+0000 7f0c24953640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0c2015df50 0x7f0c201619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:41.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.676+0000 7f0c24953640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0c2015df50 0x7f0c201619f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38148/0 (socket says 172.21.15.43:38148) 2024-04-25T08:54:41.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.676+0000 7f0c24953640 1 -- 172.21.15.43:0/2084974473 learned_addr learned my addr 172.21.15.43:0/2084974473 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:41.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.676+0000 7f0c17fff640 1 --2- 172.21.15.43:0/2084974473 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0c20161f30 0x7f0c201623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:41.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.676+0000 7f0c25154640 1 --2- 172.21.15.43:0/2084974473 >> v2:172.21.15.193:3300/0 conn(0x7f0c200c7cf0 0x7f0c200c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:41.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.676+0000 7f0c24953640 1 -- 172.21.15.43:0/2084974473 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0c20161f30 msgr2=0x7f0c201623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:41.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.676+0000 7f0c24953640 1 --2- 172.21.15.43:0/2084974473 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0c20161f30 0x7f0c201623a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.676+0000 7f0c24953640 1 -- 172.21.15.43:0/2084974473 >> v2:172.21.15.193:3300/0 conn(0x7f0c200c7cf0 msgr2=0x7f0c200c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:41.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.676+0000 7f0c24953640 1 --2- 172.21.15.43:0/2084974473 >> v2:172.21.15.193:3300/0 conn(0x7f0c200c7cf0 0x7f0c200c8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.676+0000 7f0c24953640 1 -- 172.21.15.43:0/2084974473 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c200d2a20 con 0x7f0c2015df50 2024-04-25T08:54:41.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.677+0000 7f0c24953640 1 --2- 172.21.15.43:0/2084974473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c2015df50 0x7f0c201619f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0c200cdb50 tx=0x7f0c1c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f8d51f04fc91938 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:41.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.677+0000 7f0c177fe640 1 -- 172.21.15.43:0/2084974473 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c037070 con 0x7f0c2015df50 2024-04-25T08:54:41.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.677+0000 7f0c177fe640 1 -- 172.21.15.43:0/2084974473 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1c032440 con 0x7f0c2015df50 2024-04-25T08:54:41.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.677+0000 7f0c177fe640 1 -- 172.21.15.43:0/2084974473 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c032810 con 0x7f0c2015df50 2024-04-25T08:54:41.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.678+0000 7f0c26bb7640 1 -- 172.21.15.43:0/2084974473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c2015df50 msgr2=0x7f0c201619f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:41.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.678+0000 7f0c26bb7640 1 --2- 172.21.15.43:0/2084974473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c2015df50 0x7f0c201619f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f0c200cdb50 tx=0x7f0c1c02a1e0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.678+0000 7f0c26bb7640 1 -- 172.21.15.43:0/2084974473 shutdown_connections 2024-04-25T08:54:41.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.678+0000 7f0c26bb7640 1 --2- 172.21.15.43:0/2084974473 >> v2:172.21.15.193:3300/0 conn(0x7f0c200c7cf0 0x7f0c200c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.678+0000 7f0c26bb7640 1 --2- 172.21.15.43:0/2084974473 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0c20161f30 0x7f0c201623a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.678+0000 7f0c26bb7640 1 --2- 172.21.15.43:0/2084974473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c2015df50 0x7f0c201619f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.678+0000 7f0c26bb7640 1 -- 172.21.15.43:0/2084974473 >> 172.21.15.43:0/2084974473 conn(0x7f0c200d3a30 msgr2=0x7f0c200d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:41.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.678+0000 7f0c26bb7640 1 -- 172.21.15.43:0/2084974473 shutdown_connections 2024-04-25T08:54:41.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.678+0000 7f0c26bb7640 1 -- 172.21.15.43:0/2084974473 wait complete. 2024-04-25T08:54:41.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.679+0000 7f0c26bb7640 1 Processor -- start 2024-04-25T08:54:41.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.679+0000 7f0c26bb7640 1 -- start start 2024-04-25T08:54:41.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.680+0000 7f0c26bb7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c200c7cf0 0x7f0c20148e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:41.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.680+0000 7f0c26bb7640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0c2015df50 0x7f0c2009fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:41.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.680+0000 7f0c26bb7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c20161f30 0x7f0c2009fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:41.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.680+0000 7f0c26bb7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c200d2190 con 0x7f0c200c7cf0 2024-04-25T08:54:41.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.680+0000 7f0c26bb7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c200c77d0 con 0x7f0c20161f30 2024-04-25T08:54:41.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.680+0000 7f0c26bb7640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c200c8640 con 0x7f0c2015df50 2024-04-25T08:54:41.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.680+0000 7f0c24953640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c200c7cf0 0x7f0c20148e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:41.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.680+0000 7f0c24953640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c200c7cf0 0x7f0c20148e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38162/0 (socket says 172.21.15.43:38162) 2024-04-25T08:54:41.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.680+0000 7f0c24953640 1 -- 172.21.15.43:0/2422202780 learned_addr learned my addr 172.21.15.43:0/2422202780 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:41.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.680+0000 7f0c25154640 1 --2- 172.21.15.43:0/2422202780 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c20161f30 0x7f0c2009fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:41.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.680+0000 7f0c17fff640 1 --2- 172.21.15.43:0/2422202780 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0c2015df50 0x7f0c2009fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:41.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.680+0000 7f0c24953640 1 -- 172.21.15.43:0/2422202780 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0c2015df50 msgr2=0x7f0c2009fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:41.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.680+0000 7f0c24953640 1 --2- 172.21.15.43:0/2422202780 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0c2015df50 0x7f0c2009fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.680+0000 7f0c24953640 1 -- 172.21.15.43:0/2422202780 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c20161f30 msgr2=0x7f0c2009fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:41.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.680+0000 7f0c24953640 1 --2- 172.21.15.43:0/2422202780 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c20161f30 0x7f0c2009fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.680+0000 7f0c24953640 1 -- 172.21.15.43:0/2422202780 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c2015f500 con 0x7f0c200c7cf0 2024-04-25T08:54:41.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.680+0000 7f0c25154640 1 --2- 172.21.15.43:0/2422202780 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c20161f30 0x7f0c2009fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:41.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.680+0000 7f0c17fff640 1 --2- 172.21.15.43:0/2422202780 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0c2015df50 0x7f0c2009fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:41.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.680+0000 7f0c24953640 1 --2- 172.21.15.43:0/2422202780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c200c7cf0 0x7f0c20148e90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c02a220 tx=0x7f0c1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:41.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.680+0000 7f0c157fa640 1 -- 172.21.15.43:0/2422202780 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c037040 con 0x7f0c200c7cf0 2024-04-25T08:54:41.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.680+0000 7f0c157fa640 1 -- 172.21.15.43:0/2422202780 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1c042070 con 0x7f0c200c7cf0 2024-04-25T08:54:41.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.680+0000 7f0c26bb7640 1 -- 172.21.15.43:0/2422202780 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c2015f7a0 con 0x7f0c200c7cf0 2024-04-25T08:54:41.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.680+0000 7f0c157fa640 1 -- 172.21.15.43:0/2422202780 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c030bb0 con 0x7f0c200c7cf0 2024-04-25T08:54:41.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.681+0000 7f0c26bb7640 1 -- 172.21.15.43:0/2422202780 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c2015fd60 con 0x7f0c200c7cf0 2024-04-25T08:54:41.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.681+0000 7f0c26bb7640 1 -- 172.21.15.43:0/2422202780 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c200680f0 con 0x7f0c200c7cf0 2024-04-25T08:54:41.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.682+0000 7f0c157fa640 1 -- 172.21.15.43:0/2422202780 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0c1c047460 con 0x7f0c200c7cf0 2024-04-25T08:54:41.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.682+0000 7f0c157fa640 1 --2- 172.21.15.43:0/2422202780 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0c0c06b4d0 0x7f0c0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:41.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.682+0000 7f0c157fa640 1 -- 172.21.15.43:0/2422202780 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0c1c0aaef0 con 0x7f0c200c7cf0 2024-04-25T08:54:41.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.683+0000 7f0c17fff640 1 --2- 172.21.15.43:0/2422202780 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0c0c06b4d0 0x7f0c0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:41.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.683+0000 7f0c17fff640 1 --2- 172.21.15.43:0/2422202780 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0c0c06b4d0 0x7f0c0c06d990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0c20154220 tx=0x7f0c10008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:41.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.687+0000 7f0c157fa640 1 -- 172.21.15.43:0/2422202780 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0c1c045360 con 0x7f0c200c7cf0 2024-04-25T08:54:41.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.883+0000 7f0c26bb7640 1 -- 172.21.15.43:0/2422202780 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0c20073270 con 0x7f0c200c7cf0 2024-04-25T08:54:41.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.883+0000 7f0c157fa640 1 -- 172.21.15.43:0/2422202780 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0c20073270 con 0x7f0c200c7cf0 2024-04-25T08:54:41.873 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-04-25T08:54:41.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.885+0000 7f0c26bb7640 1 -- 172.21.15.43:0/2422202780 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0c0c06b4d0 msgr2=0x7f0c0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:41.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.885+0000 7f0c26bb7640 1 --2- 172.21.15.43:0/2422202780 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0c0c06b4d0 0x7f0c0c06d990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0c20154220 tx=0x7f0c10008040 comp rx=0 tx=0).stop 2024-04-25T08:54:41.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.885+0000 7f0c26bb7640 1 -- 172.21.15.43:0/2422202780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c200c7cf0 msgr2=0x7f0c20148e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:41.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.885+0000 7f0c26bb7640 1 --2- 172.21.15.43:0/2422202780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c200c7cf0 0x7f0c20148e90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c02a220 tx=0x7f0c1c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.885+0000 7f0c26bb7640 1 -- 172.21.15.43:0/2422202780 shutdown_connections 2024-04-25T08:54:41.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.885+0000 7f0c26bb7640 1 --2- 172.21.15.43:0/2422202780 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0c0c06b4d0 0x7f0c0c06d990 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.885+0000 7f0c26bb7640 1 --2- 172.21.15.43:0/2422202780 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c20161f30 0x7f0c2009fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.885+0000 7f0c26bb7640 1 --2- 172.21.15.43:0/2422202780 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0c2015df50 0x7f0c2009fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.885+0000 7f0c26bb7640 1 --2- 172.21.15.43:0/2422202780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c200c7cf0 0x7f0c20148e90 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:41.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.886+0000 7f0c26bb7640 1 -- 172.21.15.43:0/2422202780 >> 172.21.15.43:0/2422202780 conn(0x7f0c200d3a30 msgr2=0x7f0c200cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:41.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.886+0000 7f0c26bb7640 1 -- 172.21.15.43:0/2422202780 shutdown_connections 2024-04-25T08:54:41.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:41.886+0000 7f0c26bb7640 1 -- 172.21.15.43:0/2422202780 wait complete. 2024-04-25T08:54:41.886 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:54:41.886 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:54:41.902 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:54:42.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.053+0000 7f153084a640 1 Processor -- start 2024-04-25T08:54:42.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.053+0000 7f153084a640 1 -- start start 2024-04-25T08:54:42.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.054+0000 7f153084a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f152c15dce0 0x7f152c15ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:42.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.054+0000 7f153084a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f152c0d0af0 0x7f152c160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:42.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.054+0000 7f153084a640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f152c0cee80 con 0x7f152c0ce550 2024-04-25T08:54:42.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.054+0000 7f153084a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f152c0d2920 con 0x7f152c15dce0 2024-04-25T08:54:42.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.054+0000 7f153084a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f152c0cf160 con 0x7f152c0d0af0 2024-04-25T08:54:42.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.054+0000 7f152a59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f152c15dce0 0x7f152c15ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:42.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.054+0000 7f152a59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f152c15dce0 0x7f152c15ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38176/0 (socket says 172.21.15.43:38176) 2024-04-25T08:54:42.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.054+0000 7f152a59c640 1 -- 172.21.15.43:0/3558743007 learned_addr learned my addr 172.21.15.43:0/3558743007 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:42.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.054+0000 7f152a59c640 1 -- 172.21.15.43:0/3558743007 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f152c0d0af0 msgr2=0x7f152c160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:42.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.054+0000 7f152a59c640 1 --2- 172.21.15.43:0/3558743007 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f152c0d0af0 0x7f152c160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.054+0000 7f152a59c640 1 -- 172.21.15.43:0/3558743007 >> v1:172.21.15.193:6789/0 conn(0x7f152c0ce550 legacy=0x7f152c0d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:42.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.054+0000 7f152a59c640 1 -- 172.21.15.43:0/3558743007 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f152c0cf440 con 0x7f152c15dce0 2024-04-25T08:54:42.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.055+0000 7f152a59c640 1 --2- 172.21.15.43:0/3558743007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f152c15dce0 0x7f152c15ef50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f152c09d3c0 tx=0x7f151c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de80f52c4ebf543 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:42.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.055+0000 7f1529d9b640 1 -- 172.21.15.43:0/3558743007 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151c037070 con 0x7f152c15dce0 2024-04-25T08:54:42.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.055+0000 7f1529d9b640 1 -- 172.21.15.43:0/3558743007 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f151c02ae50 con 0x7f152c15dce0 2024-04-25T08:54:42.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.055+0000 7f1529d9b640 1 -- 172.21.15.43:0/3558743007 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151c032660 con 0x7f152c15dce0 2024-04-25T08:54:42.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.055+0000 7f153084a640 1 -- 172.21.15.43:0/3558743007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f152c15dce0 msgr2=0x7f152c15ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:42.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.055+0000 7f153084a640 1 --2- 172.21.15.43:0/3558743007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f152c15dce0 0x7f152c15ef50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f152c09d3c0 tx=0x7f151c02a140 comp rx=0 tx=0).stop 2024-04-25T08:54:42.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.055+0000 7f153084a640 1 -- 172.21.15.43:0/3558743007 shutdown_connections 2024-04-25T08:54:42.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.055+0000 7f153084a640 1 --2- 172.21.15.43:0/3558743007 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f152c0d0af0 0x7f152c160980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.055+0000 7f153084a640 1 --2- 172.21.15.43:0/3558743007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f152c15dce0 0x7f152c15ef50 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.056+0000 7f153084a640 1 -- 172.21.15.43:0/3558743007 >> 172.21.15.43:0/3558743007 conn(0x7f152c09cd00 msgr2=0x7f152c0d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:42.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.056+0000 7f153084a640 1 -- 172.21.15.43:0/3558743007 shutdown_connections 2024-04-25T08:54:42.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.056+0000 7f153084a640 1 -- 172.21.15.43:0/3558743007 wait complete. 2024-04-25T08:54:42.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.057+0000 7f153084a640 1 Processor -- start 2024-04-25T08:54:42.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.057+0000 7f153084a640 1 -- start start 2024-04-25T08:54:42.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.058+0000 7f153084a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f152c0ce550 0x7f152c1f4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:42.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.058+0000 7f153084a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f152c0d0af0 0x7f152c15c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:42.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.058+0000 7f153084a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f152c15dce0 0x7f152c0a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:42.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.058+0000 7f153084a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152c15e0c0 con 0x7f152c0ce550 2024-04-25T08:54:42.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.058+0000 7f153084a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f152c15f6b0 con 0x7f152c15dce0 2024-04-25T08:54:42.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.058+0000 7f153084a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f152c0ce9c0 con 0x7f152c0d0af0 2024-04-25T08:54:42.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.058+0000 7f152a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f152c0ce550 0x7f152c1f4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:42.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.058+0000 7f152a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f152c0ce550 0x7f152c1f4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38190/0 (socket says 172.21.15.43:38190) 2024-04-25T08:54:42.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.058+0000 7f152a59c640 1 -- 172.21.15.43:0/695508180 learned_addr learned my addr 172.21.15.43:0/695508180 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:42.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.058+0000 7f152ad9d640 1 --2- 172.21.15.43:0/695508180 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f152c15dce0 0x7f152c0a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:42.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.058+0000 7f1521d9b640 1 --2- 172.21.15.43:0/695508180 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f152c0d0af0 0x7f152c15c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:42.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.058+0000 7f152a59c640 1 -- 172.21.15.43:0/695508180 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f152c0d0af0 msgr2=0x7f152c15c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:42.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.058+0000 7f152a59c640 1 --2- 172.21.15.43:0/695508180 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f152c0d0af0 0x7f152c15c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.058+0000 7f152a59c640 1 -- 172.21.15.43:0/695508180 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f152c15dce0 msgr2=0x7f152c0a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:42.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.058+0000 7f152a59c640 1 --2- 172.21.15.43:0/695508180 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f152c15dce0 0x7f152c0a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.058+0000 7f152a59c640 1 -- 172.21.15.43:0/695508180 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f152c0cbc90 con 0x7f152c0ce550 2024-04-25T08:54:42.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.058+0000 7f152ad9d640 1 --2- 172.21.15.43:0/695508180 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f152c15dce0 0x7f152c0a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:42.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.059+0000 7f152a59c640 1 --2- 172.21.15.43:0/695508180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f152c0ce550 0x7f152c1f4370 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f151c02a180 tx=0x7f151c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:42.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.059+0000 7f15237fe640 1 -- 172.21.15.43:0/695508180 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151c042070 con 0x7f152c0ce550 2024-04-25T08:54:42.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.059+0000 7f15237fe640 1 -- 172.21.15.43:0/695508180 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f151c03e670 con 0x7f152c0ce550 2024-04-25T08:54:42.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.059+0000 7f15237fe640 1 -- 172.21.15.43:0/695508180 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151c037040 con 0x7f152c0ce550 2024-04-25T08:54:42.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.059+0000 7f153084a640 1 -- 172.21.15.43:0/695508180 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f152c0cbf30 con 0x7f152c0ce550 2024-04-25T08:54:42.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.059+0000 7f153084a640 1 -- 172.21.15.43:0/695508180 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f152c0cc4f0 con 0x7f152c0ce550 2024-04-25T08:54:42.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.060+0000 7f153084a640 1 -- 172.21.15.43:0/695508180 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f152c0680f0 con 0x7f152c0ce550 2024-04-25T08:54:42.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.060+0000 7f15237fe640 1 -- 172.21.15.43:0/695508180 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f151c045070 con 0x7f152c0ce550 2024-04-25T08:54:42.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.061+0000 7f15237fe640 1 --2- 172.21.15.43:0/695508180 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f151006b4d0 0x7f151006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:42.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.061+0000 7f15237fe640 1 -- 172.21.15.43:0/695508180 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f151c0aa9e0 con 0x7f152c0ce550 2024-04-25T08:54:42.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.061+0000 7f1521d9b640 1 --2- 172.21.15.43:0/695508180 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f151006b4d0 0x7f151006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:42.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.062+0000 7f1521d9b640 1 --2- 172.21.15.43:0/695508180 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f151006b4d0 0x7f151006d990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f152c06c6d0 tx=0x7f1514008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:42.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.065+0000 7f15237fe640 1 -- 172.21.15.43:0/695508180 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f151c079980 con 0x7f152c0ce550 2024-04-25T08:54:42.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.246+0000 7f153084a640 1 -- 172.21.15.43:0/695508180 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f152c074ef0 con 0x7f152c0ce550 2024-04-25T08:54:42.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.248+0000 7f15237fe640 1 -- 172.21.15.43:0/695508180 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16569 (secure 0 0 0) 0x7f151c079280 con 0x7f152c0ce550 2024-04-25T08:54:42.238 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:54:42.238 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":20,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:54:13.593188+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:54:09.757978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:54:10.926008+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:54:42.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.250+0000 7f153084a640 1 -- 172.21.15.43:0/695508180 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f151006b4d0 msgr2=0x7f151006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:42.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.250+0000 7f153084a640 1 --2- 172.21.15.43:0/695508180 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f151006b4d0 0x7f151006d990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f152c06c6d0 tx=0x7f1514008040 comp rx=0 tx=0).stop 2024-04-25T08:54:42.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.250+0000 7f153084a640 1 -- 172.21.15.43:0/695508180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f152c0ce550 msgr2=0x7f152c1f4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:42.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.250+0000 7f153084a640 1 --2- 172.21.15.43:0/695508180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f152c0ce550 0x7f152c1f4370 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f151c02a180 tx=0x7f151c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.250+0000 7f153084a640 1 -- 172.21.15.43:0/695508180 shutdown_connections 2024-04-25T08:54:42.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.250+0000 7f153084a640 1 --2- 172.21.15.43:0/695508180 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f151006b4d0 0x7f151006d990 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.250+0000 7f153084a640 1 --2- 172.21.15.43:0/695508180 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f152c15dce0 0x7f152c0a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.250+0000 7f153084a640 1 --2- 172.21.15.43:0/695508180 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f152c0d0af0 0x7f152c15c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.250+0000 7f153084a640 1 --2- 172.21.15.43:0/695508180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f152c0ce550 0x7f152c1f4370 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.250+0000 7f153084a640 1 -- 172.21.15.43:0/695508180 >> 172.21.15.43:0/695508180 conn(0x7f152c09cd00 msgr2=0x7f152c0c8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:42.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.250+0000 7f153084a640 1 -- 172.21.15.43:0/695508180 shutdown_connections 2024-04-25T08:54:42.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.250+0000 7f153084a640 1 -- 172.21.15.43:0/695508180 wait complete. 2024-04-25T08:54:42.251 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:54:09.757978+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:54:10.926008+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:54:42.251 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:54:42.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.367+0000 7fd7ba241640 1 Processor -- start 2024-04-25T08:54:42.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.367+0000 7fd7ba241640 1 -- start start 2024-04-25T08:54:42.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.368+0000 7fd7ba241640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fd7b415b9d0 0x7fd7b415bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:42.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.368+0000 7fd7ba241640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd7b406e600 0x7fd7b40b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:42.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.368+0000 7fd7ba241640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd7b40b06c0 0x7fd7b40b0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:42.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.368+0000 7fd7ba241640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7b40b1070 con 0x7fd7b40b06c0 2024-04-25T08:54:42.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.368+0000 7fd7ba241640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7b415c2f0 con 0x7fd7b415b9d0 2024-04-25T08:54:42.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.368+0000 7fd7ba241640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b415c570 con 0x7fd7b406e600 2024-04-25T08:54:42.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.368+0000 7fd7b3fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd7b40b06c0 0x7fd7b40b0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:42.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.368+0000 7fd7b3fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd7b40b06c0 0x7fd7b40b0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38198/0 (socket says 172.21.15.43:38198) 2024-04-25T08:54:42.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.368+0000 7fd7b3fff640 1 -- 172.21.15.43:0/3878266120 learned_addr learned my addr 172.21.15.43:0/3878266120 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:42.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.368+0000 7fd7b37fe640 1 --2- 172.21.15.43:0/3878266120 >> v2:172.21.15.193:3300/0 conn(0x7fd7b415b9d0 0x7fd7b415bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:42.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.368+0000 7fd7b2ffd640 1 --2- 172.21.15.43:0/3878266120 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd7b406e600 0x7fd7b40b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:42.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.368+0000 7fd7b3fff640 1 -- 172.21.15.43:0/3878266120 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd7b406e600 msgr2=0x7fd7b40b49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:42.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.368+0000 7fd7b3fff640 1 --2- 172.21.15.43:0/3878266120 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd7b406e600 0x7fd7b40b49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.368+0000 7fd7b3fff640 1 -- 172.21.15.43:0/3878266120 >> v2:172.21.15.193:3300/0 conn(0x7fd7b415b9d0 msgr2=0x7fd7b415bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:42.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.368+0000 7fd7b3fff640 1 --2- 172.21.15.43:0/3878266120 >> v2:172.21.15.193:3300/0 conn(0x7fd7b415b9d0 0x7fd7b415bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.368+0000 7fd7b3fff640 1 -- 172.21.15.43:0/3878266120 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b415c850 con 0x7fd7b40b06c0 2024-04-25T08:54:42.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.368+0000 7fd7b37fe640 1 --2- 172.21.15.43:0/3878266120 >> v2:172.21.15.193:3300/0 conn(0x7fd7b415b9d0 0x7fd7b415bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:42.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.369+0000 7fd7b3fff640 1 --2- 172.21.15.43:0/3878266120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7b40b06c0 0x7fd7b40b0b30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4070850 tx=0x7fd7ac02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eba7b9e8f2adc9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:42.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.370+0000 7fd7b27fc640 1 -- 172.21.15.43:0/3878266120 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac036070 con 0x7fd7b40b06c0 2024-04-25T08:54:42.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.370+0000 7fd7b27fc640 1 -- 172.21.15.43:0/3878266120 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7ac031600 con 0x7fd7b40b06c0 2024-04-25T08:54:42.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.370+0000 7fd7b27fc640 1 -- 172.21.15.43:0/3878266120 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac0319d0 con 0x7fd7b40b06c0 2024-04-25T08:54:42.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.370+0000 7fd7ba241640 1 -- 172.21.15.43:0/3878266120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7b40b06c0 msgr2=0x7fd7b40b0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:42.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.370+0000 7fd7ba241640 1 --2- 172.21.15.43:0/3878266120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7b40b06c0 0x7fd7b40b0b30 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4070850 tx=0x7fd7ac02a610 comp rx=0 tx=0).stop 2024-04-25T08:54:42.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.370+0000 7fd7ba241640 1 -- 172.21.15.43:0/3878266120 shutdown_connections 2024-04-25T08:54:42.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.370+0000 7fd7ba241640 1 --2- 172.21.15.43:0/3878266120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7b40b06c0 0x7fd7b40b0b30 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.370+0000 7fd7ba241640 1 --2- 172.21.15.43:0/3878266120 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd7b406e600 0x7fd7b40b49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.370+0000 7fd7ba241640 1 --2- 172.21.15.43:0/3878266120 >> v2:172.21.15.193:3300/0 conn(0x7fd7b415b9d0 0x7fd7b415bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.370+0000 7fd7ba241640 1 -- 172.21.15.43:0/3878266120 >> 172.21.15.43:0/3878266120 conn(0x7fd7b40b6cf0 msgr2=0x7fd7b40b7100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:42.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.370+0000 7fd7ba241640 1 -- 172.21.15.43:0/3878266120 shutdown_connections 2024-04-25T08:54:42.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.370+0000 7fd7ba241640 1 -- 172.21.15.43:0/3878266120 wait complete. 2024-04-25T08:54:42.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.372+0000 7fd7ba241640 1 Processor -- start 2024-04-25T08:54:42.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.372+0000 7fd7ba241640 1 -- start start 2024-04-25T08:54:42.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.373+0000 7fd7ba241640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd7b406e600 0x7fd7b40cb760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:42.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.373+0000 7fd7ba241640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7b40b06c0 0x7fd7b415fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:42.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.373+0000 7fd7ba241640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd7b415b9d0 0x7fd7b406cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:42.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.373+0000 7fd7ba241640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b40aeee0 con 0x7fd7b40b06c0 2024-04-25T08:54:42.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.373+0000 7fd7ba241640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b415ce10 con 0x7fd7b415b9d0 2024-04-25T08:54:42.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.373+0000 7fd7ba241640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b40af060 con 0x7fd7b406e600 2024-04-25T08:54:42.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.373+0000 7fd7b2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7b40b06c0 0x7fd7b415fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:42.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.373+0000 7fd7b2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7b40b06c0 0x7fd7b415fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38202/0 (socket says 172.21.15.43:38202) 2024-04-25T08:54:42.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.373+0000 7fd7b2ffd640 1 -- 172.21.15.43:0/1225232655 learned_addr learned my addr 172.21.15.43:0/1225232655 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:42.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.373+0000 7fd7b3fff640 1 --2- 172.21.15.43:0/1225232655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd7b415b9d0 0x7fd7b406cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:42.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.373+0000 7fd7b37fe640 1 --2- 172.21.15.43:0/1225232655 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd7b406e600 0x7fd7b40cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:42.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.373+0000 7fd7b2ffd640 1 -- 172.21.15.43:0/1225232655 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd7b406e600 msgr2=0x7fd7b40cb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:42.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.373+0000 7fd7b2ffd640 1 --2- 172.21.15.43:0/1225232655 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd7b406e600 0x7fd7b40cb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.373+0000 7fd7b2ffd640 1 -- 172.21.15.43:0/1225232655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd7b415b9d0 msgr2=0x7fd7b406cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:42.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.373+0000 7fd7b2ffd640 1 --2- 172.21.15.43:0/1225232655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd7b415b9d0 0x7fd7b406cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.373+0000 7fd7b2ffd640 1 -- 172.21.15.43:0/1225232655 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b406dc20 con 0x7fd7b40b06c0 2024-04-25T08:54:42.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.373+0000 7fd7b2ffd640 1 --2- 172.21.15.43:0/1225232655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7b40b06c0 0x7fd7b415fd30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd7a800d830 tx=0x7fd7a800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:42.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.373+0000 7fd79bfff640 1 -- 172.21.15.43:0/1225232655 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8018030 con 0x7fd7b40b06c0 2024-04-25T08:54:42.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.373+0000 7fd79bfff640 1 -- 172.21.15.43:0/1225232655 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a800be10 con 0x7fd7b40b06c0 2024-04-25T08:54:42.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.373+0000 7fd7ba241640 1 -- 172.21.15.43:0/1225232655 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7b40cbca0 con 0x7fd7b40b06c0 2024-04-25T08:54:42.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.374+0000 7fd79bfff640 1 -- 172.21.15.43:0/1225232655 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a80098b0 con 0x7fd7b40b06c0 2024-04-25T08:54:42.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.374+0000 7fd7ba241640 1 -- 172.21.15.43:0/1225232655 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7b40caf60 con 0x7fd7b40b06c0 2024-04-25T08:54:42.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.375+0000 7fd7ba241640 1 -- 172.21.15.43:0/1225232655 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7b4074f60 con 0x7fd7b40b06c0 2024-04-25T08:54:42.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.375+0000 7fd79bfff640 1 -- 172.21.15.43:0/1225232655 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd7a801b070 con 0x7fd7b40b06c0 2024-04-25T08:54:42.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.376+0000 7fd79bfff640 1 --2- 172.21.15.43:0/1225232655 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd7a006b4d0 0x7fd7a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:42.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.376+0000 7fd79bfff640 1 -- 172.21.15.43:0/1225232655 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd7a808a420 con 0x7fd7b40b06c0 2024-04-25T08:54:42.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.376+0000 7fd7b37fe640 1 --2- 172.21.15.43:0/1225232655 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd7a006b4d0 0x7fd7a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:42.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.376+0000 7fd7b37fe640 1 --2- 172.21.15.43:0/1225232655 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd7a006b4d0 0x7fd7a006d990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4006fd0 tx=0x7fd7a4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:42.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.380+0000 7fd79bfff640 1 -- 172.21.15.43:0/1225232655 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd7a8059290 con 0x7fd7b40b06c0 2024-04-25T08:54:42.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.560+0000 7fd7ba241640 1 -- 172.21.15.43:0/1225232655 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd7b40676e0 con 0x7fd7b40b06c0 2024-04-25T08:54:42.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.560+0000 7fd79bfff640 1 -- 172.21.15.43:0/1225232655 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd7a8058b90 con 0x7fd7b40b06c0 2024-04-25T08:54:42.550 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T08:54:42.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.562+0000 7fd7ba241640 1 -- 172.21.15.43:0/1225232655 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd7a006b4d0 msgr2=0x7fd7a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:42.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.562+0000 7fd7ba241640 1 --2- 172.21.15.43:0/1225232655 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd7a006b4d0 0x7fd7a006d990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd7a4006fd0 tx=0x7fd7a4008040 comp rx=0 tx=0).stop 2024-04-25T08:54:42.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.562+0000 7fd7ba241640 1 -- 172.21.15.43:0/1225232655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7b40b06c0 msgr2=0x7fd7b415fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:42.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.562+0000 7fd7ba241640 1 --2- 172.21.15.43:0/1225232655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7b40b06c0 0x7fd7b415fd30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd7a800d830 tx=0x7fd7a800dcf0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.562+0000 7fd7ba241640 1 -- 172.21.15.43:0/1225232655 shutdown_connections 2024-04-25T08:54:42.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.562+0000 7fd7ba241640 1 --2- 172.21.15.43:0/1225232655 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd7a006b4d0 0x7fd7a006d990 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.562+0000 7fd7ba241640 1 --2- 172.21.15.43:0/1225232655 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd7b415b9d0 0x7fd7b406cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.562+0000 7fd7ba241640 1 --2- 172.21.15.43:0/1225232655 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd7b40b06c0 0x7fd7b415fd30 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.562+0000 7fd7ba241640 1 --2- 172.21.15.43:0/1225232655 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd7b406e600 0x7fd7b40cb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.562+0000 7fd7ba241640 1 -- 172.21.15.43:0/1225232655 >> 172.21.15.43:0/1225232655 conn(0x7fd7b40b6cf0 msgr2=0x7fd7b4159260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:42.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.562+0000 7fd7ba241640 1 -- 172.21.15.43:0/1225232655 shutdown_connections 2024-04-25T08:54:42.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.562+0000 7fd7ba241640 1 -- 172.21.15.43:0/1225232655 wait complete. 2024-04-25T08:54:42.565 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:54:42.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.680+0000 7f90f8f89640 1 Processor -- start 2024-04-25T08:54:42.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.680+0000 7f90f8f89640 1 -- start start 2024-04-25T08:54:42.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.680+0000 7f90f8f89640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f90f40d1a20 0x7f90f40b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:42.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.680+0000 7f90f8f89640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90f415bf90 con 0x7f90f40d2260 2024-04-25T08:54:42.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.680+0000 7f90f8f89640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90f40cee70 con 0x7f90f40d1e00 2024-04-25T08:54:42.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.680+0000 7f90f8f89640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90f40cf430 con 0x7f90f40d1a20 2024-04-25T08:54:42.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.681+0000 7f90f2d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f90f40d2260 0x7f90f40b5810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39650/0 (socket says 172.21.15.43:39650) 2024-04-25T08:54:42.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.681+0000 7f90f2d9d640 1 -- 172.21.15.43:0/2008648485 learned_addr learned my addr 172.21.15.43:0/2008648485 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:42.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.681+0000 7f90f259c640 1 --2- 172.21.15.43:0/2008648485 >> v2:172.21.15.193:3300/0 conn(0x7f90f40d1a20 0x7f90f40b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:42.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.681+0000 7f90f1d9b640 1 -- 172.21.15.43:0/2008648485 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 305758974 0 0) 0x7f90f415bf90 con 0x7f90f40d2260 2024-04-25T08:54:42.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.681+0000 7f90f1d9b640 1 -- 172.21.15.43:0/2008648485 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90e0003520 con 0x7f90f40d2260 2024-04-25T08:54:42.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.681+0000 7f90f1d9b640 1 -- 172.21.15.43:0/2008648485 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3963413371 0 0) 0x7f90e0003520 con 0x7f90f40d2260 2024-04-25T08:54:42.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.681+0000 7f90f1d9b640 1 -- 172.21.15.43:0/2008648485 >> v2:172.21.15.193:3300/0 conn(0x7f90f40d1a20 msgr2=0x7f90f40b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:42.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.681+0000 7f90f1d9b640 1 --2- 172.21.15.43:0/2008648485 >> v2:172.21.15.193:3300/0 conn(0x7f90f40d1a20 0x7f90f40b2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.681+0000 7f90f1d9b640 1 -- 172.21.15.43:0/2008648485 >> v1:172.21.15.193:6789/0 conn(0x7f90f40d1e00 legacy=0x7f90f40b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:42.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.681+0000 7f90f1d9b640 1 -- 172.21.15.43:0/2008648485 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f40cf150 con 0x7f90f40d2260 2024-04-25T08:54:42.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.681+0000 7f90f1d9b640 1 -- 172.21.15.43:0/2008648485 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f90ec0019f0 con 0x7f90f40d2260 2024-04-25T08:54:42.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.681+0000 7f90f1d9b640 1 -- 172.21.15.43:0/2008648485 >> v1:172.21.15.43:6789/0 conn(0x7f90f40d2260 legacy=0x7f90f40b5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:42.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.681+0000 7f90f1d9b640 1 --2- 172.21.15.43:0/2008648485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90e00039b0 0x7f90e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:42.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.681+0000 7f90f1d9b640 1 --2- 172.21.15.43:0/2008648485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90e0004800 0x7f90e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:42.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.681+0000 7f90f1d9b640 1 --2- 172.21.15.43:0/2008648485 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f90e00052e0 0x7f90e000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:42.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.681+0000 7f90f1d9b640 1 -- 172.21.15.43:0/2008648485 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f40cf430 con 0x7f90e00039b0 2024-04-25T08:54:42.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.681+0000 7f90f1d9b640 1 -- 172.21.15.43:0/2008648485 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90e000c1b0 con 0x7f90e0004800 2024-04-25T08:54:42.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.681+0000 7f90f1d9b640 1 -- 172.21.15.43:0/2008648485 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90e000c3b0 con 0x7f90e00052e0 2024-04-25T08:54:42.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.681+0000 7f90f259c640 1 --2- 172.21.15.43:0/2008648485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90e00039b0 0x7f90e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:42.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.681+0000 7f90f2d9d640 1 --2- 172.21.15.43:0/2008648485 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f90e00052e0 0x7f90e000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:42.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.681+0000 7f90ebfff640 1 --2- 172.21.15.43:0/2008648485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90e0004800 0x7f90e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:42.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.682+0000 7f90f259c640 1 -- 172.21.15.43:0/2008648485 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f90e00052e0 msgr2=0x7f90e000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:42.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.682+0000 7f90f259c640 1 --2- 172.21.15.43:0/2008648485 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f90e00052e0 0x7f90e000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.682+0000 7f90f259c640 1 -- 172.21.15.43:0/2008648485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90e0004800 msgr2=0x7f90e0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:42.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.682+0000 7f90f259c640 1 --2- 172.21.15.43:0/2008648485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90e0004800 0x7f90e0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.682+0000 7f90f259c640 1 -- 172.21.15.43:0/2008648485 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90e0003520 con 0x7f90e00039b0 2024-04-25T08:54:42.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.682+0000 7f90f2d9d640 1 -- 172.21.15.43:0/2008648485 reap_dead start 2024-04-25T08:54:42.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.682+0000 7f90ebfff640 1 --2- 172.21.15.43:0/2008648485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90e0004800 0x7f90e0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:42.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.682+0000 7f90f259c640 1 --2- 172.21.15.43:0/2008648485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90e00039b0 0x7f90e0003dd0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f90e4009a90 tx=0x7f90e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38fc7c9e2cc1df12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:42.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.682+0000 7f90f1d9b640 1 -- 172.21.15.43:0/2008648485 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e4036070 con 0x7f90e00039b0 2024-04-25T08:54:42.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.682+0000 7f90f1d9b640 1 -- 172.21.15.43:0/2008648485 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e4031400 con 0x7f90e00039b0 2024-04-25T08:54:42.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.682+0000 7f90f1d9b640 1 -- 172.21.15.43:0/2008648485 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e4009690 con 0x7f90e00039b0 2024-04-25T08:54:42.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.683+0000 7f90f8f89640 1 -- 172.21.15.43:0/2008648485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90e00039b0 msgr2=0x7f90e0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:42.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.683+0000 7f90f8f89640 1 --2- 172.21.15.43:0/2008648485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90e00039b0 0x7f90e0003dd0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f90e4009a90 tx=0x7f90e0005830 comp rx=0 tx=0).stop 2024-04-25T08:54:42.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.683+0000 7f90f8f89640 1 -- 172.21.15.43:0/2008648485 shutdown_connections 2024-04-25T08:54:42.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.683+0000 7f90f8f89640 1 --2- 172.21.15.43:0/2008648485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90e00039b0 0x7f90e0003dd0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.683+0000 7f90f8f89640 1 -- 172.21.15.43:0/2008648485 >> 172.21.15.43:0/2008648485 conn(0x7f90f4072aa0 msgr2=0x7f90f409aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:42.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.683+0000 7f90f8f89640 1 -- 172.21.15.43:0/2008648485 shutdown_connections 2024-04-25T08:54:42.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.683+0000 7f90f8f89640 1 -- 172.21.15.43:0/2008648485 wait complete. 2024-04-25T08:54:42.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.685+0000 7f90f8f89640 1 Processor -- start 2024-04-25T08:54:42.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.685+0000 7f90f8f89640 1 -- start start 2024-04-25T08:54:42.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.685+0000 7f90f8f89640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90e00039b0 0x7f90f4151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:42.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.685+0000 7f90f8f89640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90f41432c0 0x7f90f4152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:42.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.685+0000 7f90f8f89640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f90f4159840 0x7f90f4159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:42.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.685+0000 7f90f8f89640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f40cea60 con 0x7f90e00039b0 2024-04-25T08:54:42.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.685+0000 7f90f8f89640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f40b4be0 con 0x7f90f41432c0 2024-04-25T08:54:42.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.685+0000 7f90f8f89640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90f40cebe0 con 0x7f90f4159840 2024-04-25T08:54:42.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.686+0000 7f90f259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90e00039b0 0x7f90f4151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:42.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.686+0000 7f90f259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90e00039b0 0x7f90f4151c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38218/0 (socket says 172.21.15.43:38218) 2024-04-25T08:54:42.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.686+0000 7f90f259c640 1 -- 172.21.15.43:0/228741064 learned_addr learned my addr 172.21.15.43:0/228741064 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:42.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.686+0000 7f90f2d9d640 1 --2- 172.21.15.43:0/228741064 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f90f4159840 0x7f90f4159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:42.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.686+0000 7f90ebfff640 1 --2- 172.21.15.43:0/228741064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90f41432c0 0x7f90f4152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:42.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.686+0000 7f90f259c640 1 -- 172.21.15.43:0/228741064 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f90f4159840 msgr2=0x7f90f4159cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:42.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.686+0000 7f90f259c640 1 --2- 172.21.15.43:0/228741064 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f90f4159840 0x7f90f4159cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.686+0000 7f90f259c640 1 -- 172.21.15.43:0/228741064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90f41432c0 msgr2=0x7f90f4152620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:42.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.686+0000 7f90f259c640 1 --2- 172.21.15.43:0/228741064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90f41432c0 0x7f90f4152620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.686+0000 7f90f259c640 1 -- 172.21.15.43:0/228741064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f4143740 con 0x7f90e00039b0 2024-04-25T08:54:42.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.686+0000 7f90f259c640 1 --2- 172.21.15.43:0/228741064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90e00039b0 0x7f90f4151c90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f90e4034040 tx=0x7f90e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:42.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.686+0000 7f90eaffd640 1 -- 172.21.15.43:0/228741064 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e403f020 con 0x7f90e00039b0 2024-04-25T08:54:42.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.686+0000 7f90eaffd640 1 -- 172.21.15.43:0/228741064 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e4037040 con 0x7f90e00039b0 2024-04-25T08:54:42.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.686+0000 7f90eaffd640 1 -- 172.21.15.43:0/228741064 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e4036070 con 0x7f90e00039b0 2024-04-25T08:54:42.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.686+0000 7f90f8f89640 1 -- 172.21.15.43:0/228741064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90f41522c0 con 0x7f90e00039b0 2024-04-25T08:54:42.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.686+0000 7f90f8f89640 1 -- 172.21.15.43:0/228741064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90f4158d60 con 0x7f90e00039b0 2024-04-25T08:54:42.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.688+0000 7f90f8f89640 1 -- 172.21.15.43:0/228741064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90f4074f60 con 0x7f90e00039b0 2024-04-25T08:54:42.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.688+0000 7f90eaffd640 1 -- 172.21.15.43:0/228741064 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f90e4042020 con 0x7f90e00039b0 2024-04-25T08:54:42.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.688+0000 7f90eaffd640 1 --2- 172.21.15.43:0/228741064 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f90d406b4d0 0x7f90d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:42.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.688+0000 7f90eaffd640 1 -- 172.21.15.43:0/228741064 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f90e40aa110 con 0x7f90e00039b0 2024-04-25T08:54:42.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.689+0000 7f90ebfff640 1 --2- 172.21.15.43:0/228741064 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f90d406b4d0 0x7f90d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:42.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.689+0000 7f90ebfff640 1 --2- 172.21.15.43:0/228741064 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f90d406b4d0 0x7f90d406d990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f90dc000c00 tx=0x7f90dc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:42.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.693+0000 7f90eaffd640 1 -- 172.21.15.43:0/228741064 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f90e4049030 con 0x7f90e00039b0 2024-04-25T08:54:42.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.876+0000 7f90f8f89640 1 -- 172.21.15.43:0/228741064 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f90f40b5510 con 0x7f90e00039b0 2024-04-25T08:54:42.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.876+0000 7f90eaffd640 1 -- 172.21.15.43:0/228741064 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f90e4078a00 con 0x7f90e00039b0 2024-04-25T08:54:42.866 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T08:54:42.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.878+0000 7f90f8f89640 1 -- 172.21.15.43:0/228741064 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f90d406b4d0 msgr2=0x7f90d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:42.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.878+0000 7f90f8f89640 1 --2- 172.21.15.43:0/228741064 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f90d406b4d0 0x7f90d406d990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f90dc000c00 tx=0x7f90dc009040 comp rx=0 tx=0).stop 2024-04-25T08:54:42.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.879+0000 7f90f8f89640 1 -- 172.21.15.43:0/228741064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90e00039b0 msgr2=0x7f90f4151c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:42.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.879+0000 7f90f8f89640 1 --2- 172.21.15.43:0/228741064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90e00039b0 0x7f90f4151c90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f90e4034040 tx=0x7f90e4004730 comp rx=0 tx=0).stop 2024-04-25T08:54:42.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.879+0000 7f90f8f89640 1 -- 172.21.15.43:0/228741064 shutdown_connections 2024-04-25T08:54:42.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.879+0000 7f90f8f89640 1 --2- 172.21.15.43:0/228741064 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f90f4159840 0x7f90f4159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.879+0000 7f90f8f89640 1 --2- 172.21.15.43:0/228741064 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f90d406b4d0 0x7f90d406d990 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.879+0000 7f90f8f89640 1 --2- 172.21.15.43:0/228741064 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f90f41432c0 0x7f90f4152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.879+0000 7f90f8f89640 1 --2- 172.21.15.43:0/228741064 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90e00039b0 0x7f90f4151c90 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.879+0000 7f90f8f89640 1 -- 172.21.15.43:0/228741064 >> 172.21.15.43:0/228741064 conn(0x7f90f4072aa0 msgr2=0x7f90f406f560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:42.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.879+0000 7f90f8f89640 1 -- 172.21.15.43:0/228741064 shutdown_connections 2024-04-25T08:54:42.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.879+0000 7f90f8f89640 1 -- 172.21.15.43:0/228741064 wait complete. 2024-04-25T08:54:42.879 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:54:42.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.993+0000 7f7427ce2640 1 Processor -- start 2024-04-25T08:54:42.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.993+0000 7f7427ce2640 1 -- start start 2024-04-25T08:54:42.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.994+0000 7f7427ce2640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f742006ab30 0x7f742006af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:42.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.994+0000 7f7427ce2640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f74200b42a0 0x7f74200b4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:42.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.994+0000 7f7427ce2640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f742015e160 con 0x7f7420071440 2024-04-25T08:54:42.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.994+0000 7f7427ce2640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f742006b450 con 0x7f742006ab30 2024-04-25T08:54:42.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.994+0000 7f7427ce2640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f742015e400 con 0x7f74200b42a0 2024-04-25T08:54:42.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.994+0000 7f7425a7e640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f742006ab30 0x7f742006af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:42.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.995+0000 7f742527d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f74200b42a0 0x7f74200b4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:42.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.995+0000 7f742627f640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f7420071440 0x7f74200718d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:38730/0 (socket says 172.21.15.43:38730) 2024-04-25T08:54:42.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.995+0000 7f742627f640 1 -- 172.21.15.43:0/1767609380 learned_addr learned my addr 172.21.15.43:0/1767609380 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:42.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.995+0000 7f7424a7c640 1 -- 172.21.15.43:0/1767609380 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3856625447 0 0) 0x7f742015e160 con 0x7f7420071440 2024-04-25T08:54:42.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.995+0000 7f7424a7c640 1 -- 172.21.15.43:0/1767609380 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7408003520 con 0x7f7420071440 2024-04-25T08:54:42.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.995+0000 7f7425a7e640 1 -- 172.21.15.43:0/1767609380 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f74200b42a0 msgr2=0x7f74200b4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:42.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.995+0000 7f7425a7e640 1 --2- 172.21.15.43:0/1767609380 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f74200b42a0 0x7f74200b4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.995+0000 7f7425a7e640 1 -- 172.21.15.43:0/1767609380 >> v1:172.21.15.193:6789/0 conn(0x7f7420071440 legacy=0x7f74200718d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:42.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.995+0000 7f7425a7e640 1 -- 172.21.15.43:0/1767609380 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f742015e6e0 con 0x7f742006ab30 2024-04-25T08:54:42.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.995+0000 7f742527d640 1 --2- 172.21.15.43:0/1767609380 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f74200b42a0 0x7f74200b4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:54:42.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.996+0000 7f7425a7e640 1 --2- 172.21.15.43:0/1767609380 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f742006ab30 0x7f742006af10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f74180097a0 tx=0x7f741802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=561b31d14e5bf9fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:42.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.996+0000 7f7424a7c640 1 -- 172.21.15.43:0/1767609380 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7418037070 con 0x7f742006ab30 2024-04-25T08:54:42.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.996+0000 7f7424a7c640 1 -- 172.21.15.43:0/1767609380 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7418032440 con 0x7f742006ab30 2024-04-25T08:54:42.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.996+0000 7f7424a7c640 1 -- 172.21.15.43:0/1767609380 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74180327f0 con 0x7f742006ab30 2024-04-25T08:54:42.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.996+0000 7f7427ce2640 1 -- 172.21.15.43:0/1767609380 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f742006ab30 msgr2=0x7f742006af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:42.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.996+0000 7f7427ce2640 1 --2- 172.21.15.43:0/1767609380 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f742006ab30 0x7f742006af10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f74180097a0 tx=0x7f741802a180 comp rx=0 tx=0).stop 2024-04-25T08:54:42.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.996+0000 7f7427ce2640 1 -- 172.21.15.43:0/1767609380 shutdown_connections 2024-04-25T08:54:42.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.996+0000 7f7427ce2640 1 --2- 172.21.15.43:0/1767609380 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f74200b42a0 0x7f74200b4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.996+0000 7f7427ce2640 1 --2- 172.21.15.43:0/1767609380 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f742006ab30 0x7f742006af10 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.996+0000 7f7427ce2640 1 -- 172.21.15.43:0/1767609380 >> 172.21.15.43:0/1767609380 conn(0x7f74200b2590 msgr2=0x7f74200b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:42.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.996+0000 7f7427ce2640 1 -- 172.21.15.43:0/1767609380 shutdown_connections 2024-04-25T08:54:42.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.997+0000 7f7427ce2640 1 -- 172.21.15.43:0/1767609380 wait complete. 2024-04-25T08:54:42.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.998+0000 7f7427ce2640 1 Processor -- start 2024-04-25T08:54:42.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.998+0000 7f7427ce2640 1 -- start start 2024-04-25T08:54:42.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.998+0000 7f7427ce2640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f742006ab30 0x7f74201e56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:42.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.998+0000 7f7427ce2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7420071440 0x7f74200b1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:42.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.998+0000 7f7427ce2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74200b42a0 0x7f74200b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:42.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.998+0000 7f7427ce2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742015b950 con 0x7f74200b42a0 2024-04-25T08:54:42.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.998+0000 7f7427ce2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742015eca0 con 0x7f7420071440 2024-04-25T08:54:42.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.998+0000 7f7427ce2640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f742015bad0 con 0x7f742006ab30 2024-04-25T08:54:42.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.998+0000 7f742627f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74200b42a0 0x7f74200b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:42.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.999+0000 7f7425a7e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f742006ab30 0x7f74201e56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:42.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.999+0000 7f742527d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7420071440 0x7f74200b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:42.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.999+0000 7f742627f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74200b42a0 0x7f74200b15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38222/0 (socket says 172.21.15.43:38222) 2024-04-25T08:54:42.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.999+0000 7f742627f640 1 -- 172.21.15.43:0/1146347615 learned_addr learned my addr 172.21.15.43:0/1146347615 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:42.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.999+0000 7f742627f640 1 -- 172.21.15.43:0/1146347615 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f742006ab30 msgr2=0x7f74201e56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:42.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.999+0000 7f742627f640 1 --2- 172.21.15.43:0/1146347615 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f742006ab30 0x7f74201e56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.999+0000 7f742627f640 1 -- 172.21.15.43:0/1146347615 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7420071440 msgr2=0x7f74200b1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:42.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.999+0000 7f742627f640 1 --2- 172.21.15.43:0/1146347615 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7420071440 0x7f74200b1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:42.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.999+0000 7f742627f640 1 -- 172.21.15.43:0/1146347615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74201e5bf0 con 0x7f74200b42a0 2024-04-25T08:54:42.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.999+0000 7f742627f640 1 --2- 172.21.15.43:0/1146347615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74200b42a0 0x7f74200b15d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f741000cb40 tx=0x7f741000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:42.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.999+0000 7f741e7fc640 1 -- 172.21.15.43:0/1146347615 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7410017070 con 0x7f74200b42a0 2024-04-25T08:54:42.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.999+0000 7f741e7fc640 1 -- 172.21.15.43:0/1146347615 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f741000aca0 con 0x7f74200b42a0 2024-04-25T08:54:42.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.999+0000 7f7427ce2640 1 -- 172.21.15.43:0/1146347615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74201e3fe0 con 0x7f74200b42a0 2024-04-25T08:54:42.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.999+0000 7f741e7fc640 1 -- 172.21.15.43:0/1146347615 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7410012620 con 0x7f74200b42a0 2024-04-25T08:54:42.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:42.999+0000 7f7427ce2640 1 -- 172.21.15.43:0/1146347615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74201e45a0 con 0x7f74200b42a0 2024-04-25T08:54:42.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.000+0000 7f7427ce2640 1 -- 172.21.15.43:0/1146347615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7420074f60 con 0x7f74200b42a0 2024-04-25T08:54:42.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.001+0000 7f741e7fc640 1 -- 172.21.15.43:0/1146347615 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f741001a070 con 0x7f74200b42a0 2024-04-25T08:54:42.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.001+0000 7f741e7fc640 1 --2- 172.21.15.43:0/1146347615 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f740c06b4d0 0x7f740c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:42.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.001+0000 7f741e7fc640 1 -- 172.21.15.43:0/1146347615 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f741008a040 con 0x7f74200b42a0 2024-04-25T08:54:42.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.002+0000 7f7425a7e640 1 --2- 172.21.15.43:0/1146347615 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f740c06b4d0 0x7f740c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:42.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.002+0000 7f7425a7e640 1 --2- 172.21.15.43:0/1146347615 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f740c06b4d0 0x7f740c06d990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7418034000 tx=0x7f741802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:42.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.006+0000 7f741e7fc640 1 -- 172.21.15.43:0/1146347615 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f74100587e0 con 0x7f74200b42a0 2024-04-25T08:54:43.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.184+0000 7f7427ce2640 1 -- 172.21.15.43:0/1146347615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f74200b2e40 con 0x7f74200b42a0 2024-04-25T08:54:43.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.184+0000 7f741e7fc640 1 -- 172.21.15.43:0/1146347615 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f741005c880 con 0x7f74200b42a0 2024-04-25T08:54:43.174 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T08:54:43.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.186+0000 7f7427ce2640 1 -- 172.21.15.43:0/1146347615 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f740c06b4d0 msgr2=0x7f740c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:43.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.186+0000 7f7427ce2640 1 --2- 172.21.15.43:0/1146347615 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f740c06b4d0 0x7f740c06d990 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7418034000 tx=0x7f741802e000 comp rx=0 tx=0).stop 2024-04-25T08:54:43.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.186+0000 7f7427ce2640 1 -- 172.21.15.43:0/1146347615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74200b42a0 msgr2=0x7f74200b15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:43.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.186+0000 7f7427ce2640 1 --2- 172.21.15.43:0/1146347615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74200b42a0 0x7f74200b15d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f741000cb40 tx=0x7f741000a060 comp rx=0 tx=0).stop 2024-04-25T08:54:43.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.186+0000 7f7427ce2640 1 -- 172.21.15.43:0/1146347615 shutdown_connections 2024-04-25T08:54:43.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.186+0000 7f7427ce2640 1 --2- 172.21.15.43:0/1146347615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74200b42a0 0x7f74200b15d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:43.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.186+0000 7f7427ce2640 1 --2- 172.21.15.43:0/1146347615 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f740c06b4d0 0x7f740c06d990 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:43.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.186+0000 7f7427ce2640 1 --2- 172.21.15.43:0/1146347615 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7420071440 0x7f74200b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:43.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.186+0000 7f7427ce2640 1 --2- 172.21.15.43:0/1146347615 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f742006ab30 0x7f74201e56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:43.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.186+0000 7f7427ce2640 1 -- 172.21.15.43:0/1146347615 >> 172.21.15.43:0/1146347615 conn(0x7f74200b2590 msgr2=0x7f74200aeae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:43.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.186+0000 7f7427ce2640 1 -- 172.21.15.43:0/1146347615 shutdown_connections 2024-04-25T08:54:43.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.186+0000 7f7427ce2640 1 -- 172.21.15.43:0/1146347615 wait complete. 2024-04-25T08:54:43.187 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:54:43.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.298+0000 7f305a2b3640 1 Processor -- start 2024-04-25T08:54:43.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.298+0000 7f305a2b3640 1 -- start start 2024-04-25T08:54:43.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.299+0000 7f305a2b3640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f305415df50 0x7f30541619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:43.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.299+0000 7f305a2b3640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3054161f30 0x7f3054162380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:43.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.299+0000 7f305a2b3640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30540d24a0 con 0x7f30540c7cf0 2024-04-25T08:54:43.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.299+0000 7f305a2b3640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f305415e330 con 0x7f3054161f30 2024-04-25T08:54:43.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.299+0000 7f305a2b3640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30540d2740 con 0x7f305415df50 2024-04-25T08:54:43.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.299+0000 7f30537fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3054161f30 0x7f3054162380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:43.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.299+0000 7f30537fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3054161f30 0x7f3054162380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38230/0 (socket says 172.21.15.43:38230) 2024-04-25T08:54:43.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.299+0000 7f30537fe640 1 -- 172.21.15.43:0/3750337219 learned_addr learned my addr 172.21.15.43:0/3750337219 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:43.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.300+0000 7f3053fff640 1 --2- 172.21.15.43:0/3750337219 >> v2:172.21.15.193:3300/0 conn(0x7f305415df50 0x7f30541619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:43.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.300+0000 7f30537fe640 1 -- 172.21.15.43:0/3750337219 >> v2:172.21.15.193:3300/0 conn(0x7f305415df50 msgr2=0x7f30541619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:43.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.300+0000 7f30537fe640 1 --2- 172.21.15.43:0/3750337219 >> v2:172.21.15.193:3300/0 conn(0x7f305415df50 0x7f30541619f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:43.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.300+0000 7f30537fe640 1 -- 172.21.15.43:0/3750337219 >> v1:172.21.15.193:6789/0 conn(0x7f30540c7cf0 legacy=0x7f30540c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:43.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.300+0000 7f30537fe640 1 -- 172.21.15.43:0/3750337219 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30540d2a20 con 0x7f3054161f30 2024-04-25T08:54:43.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.300+0000 7f30537fe640 1 --2- 172.21.15.43:0/3750337219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3054161f30 0x7f3054162380 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f305408c880 tx=0x7f304802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce55d2488f6d4698 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:43.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.301+0000 7f3052ffd640 1 -- 172.21.15.43:0/3750337219 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3048037070 con 0x7f3054161f30 2024-04-25T08:54:43.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.301+0000 7f3052ffd640 1 -- 172.21.15.43:0/3750337219 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3048032440 con 0x7f3054161f30 2024-04-25T08:54:43.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.301+0000 7f3052ffd640 1 -- 172.21.15.43:0/3750337219 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3048032810 con 0x7f3054161f30 2024-04-25T08:54:43.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.301+0000 7f305a2b3640 1 -- 172.21.15.43:0/3750337219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3054161f30 msgr2=0x7f3054162380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:43.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.301+0000 7f305a2b3640 1 --2- 172.21.15.43:0/3750337219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3054161f30 0x7f3054162380 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f305408c880 tx=0x7f304802a270 comp rx=0 tx=0).stop 2024-04-25T08:54:43.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.301+0000 7f305a2b3640 1 -- 172.21.15.43:0/3750337219 shutdown_connections 2024-04-25T08:54:43.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.301+0000 7f305a2b3640 1 --2- 172.21.15.43:0/3750337219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3054161f30 0x7f3054162380 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:43.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.301+0000 7f305a2b3640 1 --2- 172.21.15.43:0/3750337219 >> v2:172.21.15.193:3300/0 conn(0x7f305415df50 0x7f30541619f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:43.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.301+0000 7f305a2b3640 1 -- 172.21.15.43:0/3750337219 >> 172.21.15.43:0/3750337219 conn(0x7f30540d3a30 msgr2=0x7f30540d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:43.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.301+0000 7f305a2b3640 1 -- 172.21.15.43:0/3750337219 shutdown_connections 2024-04-25T08:54:43.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.301+0000 7f305a2b3640 1 -- 172.21.15.43:0/3750337219 wait complete. 2024-04-25T08:54:43.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.302+0000 7f305a2b3640 1 Processor -- start 2024-04-25T08:54:43.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.303+0000 7f305a2b3640 1 -- start start 2024-04-25T08:54:43.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.303+0000 7f305a2b3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30540c7cf0 0x7f3054151f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:43.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.303+0000 7f305a2b3640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f305415df50 0x7f30541f42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:43.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.303+0000 7f305a2b3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3054161f30 0x7f30541558b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:43.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.303+0000 7f305a2b3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30540d2190 con 0x7f30540c7cf0 2024-04-25T08:54:43.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.303+0000 7f305a2b3640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30540c77d0 con 0x7f3054161f30 2024-04-25T08:54:43.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.303+0000 7f305a2b3640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30540cf670 con 0x7f305415df50 2024-04-25T08:54:43.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.303+0000 7f3053fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30540c7cf0 0x7f3054151f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:43.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.303+0000 7f3053fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30540c7cf0 0x7f3054151f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38236/0 (socket says 172.21.15.43:38236) 2024-04-25T08:54:43.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.303+0000 7f3053fff640 1 -- 172.21.15.43:0/3083150703 learned_addr learned my addr 172.21.15.43:0/3083150703 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:43.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.304+0000 7f3053fff640 1 -- 172.21.15.43:0/3083150703 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f305415df50 msgr2=0x7f30541f42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:43.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.304+0000 7f3053fff640 1 --2- 172.21.15.43:0/3083150703 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f305415df50 0x7f30541f42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:43.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.304+0000 7f3053fff640 1 -- 172.21.15.43:0/3083150703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3054161f30 msgr2=0x7f30541558b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:43.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.304+0000 7f3058850640 1 --2- 172.21.15.43:0/3083150703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3054161f30 0x7f30541558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:43.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.304+0000 7f3053fff640 1 --2- 172.21.15.43:0/3083150703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3054161f30 0x7f30541558b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:43.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.304+0000 7f3053fff640 1 -- 172.21.15.43:0/3083150703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30541f47f0 con 0x7f30540c7cf0 2024-04-25T08:54:43.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.304+0000 7f3058850640 1 --2- 172.21.15.43:0/3083150703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3054161f30 0x7f30541558b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:43.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.304+0000 7f3053fff640 1 --2- 172.21.15.43:0/3083150703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30540c7cf0 0x7f3054151f40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f304400d800 tx=0x7f304400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:43.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.304+0000 7f3050ff9640 1 -- 172.21.15.43:0/3083150703 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304400b6c0 con 0x7f30540c7cf0 2024-04-25T08:54:43.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.304+0000 7f3050ff9640 1 -- 172.21.15.43:0/3083150703 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3044019070 con 0x7f30540c7cf0 2024-04-25T08:54:43.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.304+0000 7f305a2b3640 1 -- 172.21.15.43:0/3083150703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3054156190 con 0x7f30540c7cf0 2024-04-25T08:54:43.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.304+0000 7f3050ff9640 1 -- 172.21.15.43:0/3083150703 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304400ba80 con 0x7f30540c7cf0 2024-04-25T08:54:43.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.304+0000 7f305a2b3640 1 -- 172.21.15.43:0/3083150703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30540cc470 con 0x7f30540c7cf0 2024-04-25T08:54:43.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.305+0000 7f305a2b3640 1 -- 172.21.15.43:0/3083150703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30540680f0 con 0x7f30540c7cf0 2024-04-25T08:54:43.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.306+0000 7f3050ff9640 1 -- 172.21.15.43:0/3083150703 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f304400bc20 con 0x7f30540c7cf0 2024-04-25T08:54:43.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.306+0000 7f3050ff9640 1 --2- 172.21.15.43:0/3083150703 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f304006b4d0 0x7f304006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:43.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.306+0000 7f3050ff9640 1 -- 172.21.15.43:0/3083150703 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f304408a670 con 0x7f30540c7cf0 2024-04-25T08:54:43.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.306+0000 7f30537fe640 1 --2- 172.21.15.43:0/3083150703 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f304006b4d0 0x7f304006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:43.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.307+0000 7f30537fe640 1 --2- 172.21.15.43:0/3083150703 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f304006b4d0 0x7f304006d990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3048034000 tx=0x7f304802c000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:43.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.310+0000 7f3050ff9640 1 -- 172.21.15.43:0/3083150703 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f30541f47f0 con 0x7f30540c7cf0 2024-04-25T08:54:43.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.501+0000 7f305a2b3640 1 -- 172.21.15.43:0/3083150703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3054074ef0 con 0x7f30540c7cf0 2024-04-25T08:54:43.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.502+0000 7f3050ff9640 1 -- 172.21.15.43:0/3083150703 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3044059580 con 0x7f30540c7cf0 2024-04-25T08:54:43.492 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-04-25T08:54:43.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.503+0000 7f305a2b3640 1 -- 172.21.15.43:0/3083150703 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f304006b4d0 msgr2=0x7f304006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:43.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.503+0000 7f305a2b3640 1 --2- 172.21.15.43:0/3083150703 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f304006b4d0 0x7f304006d990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3048034000 tx=0x7f304802c000 comp rx=0 tx=0).stop 2024-04-25T08:54:43.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.503+0000 7f305a2b3640 1 -- 172.21.15.43:0/3083150703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30540c7cf0 msgr2=0x7f3054151f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:43.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.503+0000 7f305a2b3640 1 --2- 172.21.15.43:0/3083150703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30540c7cf0 0x7f3054151f40 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f304400d800 tx=0x7f304400dcc0 comp rx=0 tx=0).stop 2024-04-25T08:54:43.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.504+0000 7f305a2b3640 1 -- 172.21.15.43:0/3083150703 shutdown_connections 2024-04-25T08:54:43.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.504+0000 7f305a2b3640 1 --2- 172.21.15.43:0/3083150703 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f304006b4d0 0x7f304006d990 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:43.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.504+0000 7f305a2b3640 1 --2- 172.21.15.43:0/3083150703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3054161f30 0x7f30541558b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:43.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.504+0000 7f305a2b3640 1 --2- 172.21.15.43:0/3083150703 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f305415df50 0x7f30541f42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:43.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.504+0000 7f305a2b3640 1 --2- 172.21.15.43:0/3083150703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30540c7cf0 0x7f3054151f40 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:43.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.504+0000 7f305a2b3640 1 -- 172.21.15.43:0/3083150703 >> 172.21.15.43:0/3083150703 conn(0x7f30540d3a30 msgr2=0x7f3054153e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:43.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.504+0000 7f305a2b3640 1 -- 172.21.15.43:0/3083150703 shutdown_connections 2024-04-25T08:54:43.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.504+0000 7f305a2b3640 1 -- 172.21.15.43:0/3083150703 wait complete. 2024-04-25T08:54:43.505 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi043.front.sepia.ceph.com... 2024-04-25T08:54:43.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:43.505 DEBUG:teuthology.orchestra.run.smithi043:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T08:54:43.529 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:54:43.529 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-04-25T08:54:43.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:43.530 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:54:43.597 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T08:54:43.597 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:54:43.618 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T08:54:43.618 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:54:43.677 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:54:43.769 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:54:43.772 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:54:43.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.788+0000 7fda7bb89640 1 Processor -- start 2024-04-25T08:54:43.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.789+0000 7fda7bb89640 1 -- start start 2024-04-25T08:54:43.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.789+0000 7fda7bb89640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fda7415a9d0 0x7fda7406d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:43.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.789+0000 7fda7bb89640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda740704f0 con 0x7fda7406d7d0 2024-04-25T08:54:43.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.789+0000 7fda7bb89640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda740707d0 con 0x7fda74072160 2024-04-25T08:54:43.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.789+0000 7fda7bb89640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda7406dc30 con 0x7fda7415a9d0 2024-04-25T08:54:43.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.790+0000 7fda79124640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fda7406d7d0 0x7fda74071a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39654/0 (socket says 172.21.15.43:39654) 2024-04-25T08:54:43.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.790+0000 7fda79124640 1 -- 172.21.15.43:0/1643672353 learned_addr learned my addr 172.21.15.43:0/1643672353 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:43.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.790+0000 7fda79925640 1 --2- 172.21.15.43:0/1643672353 >> v2:172.21.15.193:3300/0 conn(0x7fda7415a9d0 0x7fda7406d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:43.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.790+0000 7fda78923640 1 -- 172.21.15.43:0/1643672353 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 996721552 0 0) 0x7fda740704f0 con 0x7fda7406d7d0 2024-04-25T08:54:43.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.790+0000 7fda78923640 1 -- 172.21.15.43:0/1643672353 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda60005730 con 0x7fda7406d7d0 2024-04-25T08:54:43.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.790+0000 7fda78923640 1 -- 172.21.15.43:0/1643672353 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1649769917 0 0) 0x7fda740707d0 con 0x7fda74072160 2024-04-25T08:54:43.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.790+0000 7fda78923640 1 -- 172.21.15.43:0/1643672353 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda740704f0 con 0x7fda74072160 2024-04-25T08:54:43.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.790+0000 7fda79925640 1 -- 172.21.15.43:0/1643672353 >> v1:172.21.15.193:6789/0 conn(0x7fda74072160 legacy=0x7fda740cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:43.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.790+0000 7fda79925640 1 -- 172.21.15.43:0/1643672353 >> v1:172.21.15.43:6789/0 conn(0x7fda7406d7d0 legacy=0x7fda74071a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:43.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.790+0000 7fda79925640 1 -- 172.21.15.43:0/1643672353 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda74070ab0 con 0x7fda7415a9d0 2024-04-25T08:54:43.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.791+0000 7fda79925640 1 --2- 172.21.15.43:0/1643672353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda7415a9d0 0x7fda7406d290 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fda700097a0 tx=0x7fda7002a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65b5f7d98471af3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:43.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.791+0000 7fda78923640 1 -- 172.21.15.43:0/1643672353 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda70037070 con 0x7fda7415a9d0 2024-04-25T08:54:43.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.791+0000 7fda78923640 1 -- 172.21.15.43:0/1643672353 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda70032440 con 0x7fda7415a9d0 2024-04-25T08:54:43.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.791+0000 7fda78923640 1 -- 172.21.15.43:0/1643672353 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda70032810 con 0x7fda7415a9d0 2024-04-25T08:54:43.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.792+0000 7fda7bb89640 1 -- 172.21.15.43:0/1643672353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda7415a9d0 msgr2=0x7fda7406d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:43.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.792+0000 7fda7bb89640 1 --2- 172.21.15.43:0/1643672353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda7415a9d0 0x7fda7406d290 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fda700097a0 tx=0x7fda7002a230 comp rx=0 tx=0).stop 2024-04-25T08:54:43.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.792+0000 7fda7bb89640 1 -- 172.21.15.43:0/1643672353 shutdown_connections 2024-04-25T08:54:43.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.792+0000 7fda7bb89640 1 --2- 172.21.15.43:0/1643672353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda7415a9d0 0x7fda7406d290 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:43.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.792+0000 7fda7bb89640 1 -- 172.21.15.43:0/1643672353 >> 172.21.15.43:0/1643672353 conn(0x7fda740b1480 msgr2=0x7fda740b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:43.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.792+0000 7fda7bb89640 1 -- 172.21.15.43:0/1643672353 shutdown_connections 2024-04-25T08:54:43.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.792+0000 7fda7bb89640 1 -- 172.21.15.43:0/1643672353 wait complete. 2024-04-25T08:54:43.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.793+0000 7fda7bb89640 1 Processor -- start 2024-04-25T08:54:43.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.793+0000 7fda7bb89640 1 -- start start 2024-04-25T08:54:43.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.794+0000 7fda7bb89640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda7406d7d0 0x7fda7413bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:43.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.794+0000 7fda7bb89640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda74072160 0x7fda741567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:43.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.794+0000 7fda7bb89640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fda74156d10 0x7fda7413e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:43.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.794+0000 7fda7bb89640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda740ce050 con 0x7fda7406d7d0 2024-04-25T08:54:43.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.794+0000 7fda7bb89640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda740712b0 con 0x7fda74072160 2024-04-25T08:54:43.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.794+0000 7fda7bb89640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda740ce1d0 con 0x7fda74156d10 2024-04-25T08:54:43.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.794+0000 7fda79925640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda7406d7d0 0x7fda7413bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:43.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.794+0000 7fda79925640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda7406d7d0 0x7fda7413bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38242/0 (socket says 172.21.15.43:38242) 2024-04-25T08:54:43.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.794+0000 7fda79925640 1 -- 172.21.15.43:0/3350020478 learned_addr learned my addr 172.21.15.43:0/3350020478 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:43.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.794+0000 7fda79124640 1 --2- 172.21.15.43:0/3350020478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda74072160 0x7fda741567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:43.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.794+0000 7fda7a126640 1 --2- 172.21.15.43:0/3350020478 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fda74156d10 0x7fda7413e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:43.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.794+0000 7fda79925640 1 -- 172.21.15.43:0/3350020478 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fda74156d10 msgr2=0x7fda7413e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:43.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.794+0000 7fda79925640 1 --2- 172.21.15.43:0/3350020478 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fda74156d10 0x7fda7413e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:43.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.794+0000 7fda79925640 1 -- 172.21.15.43:0/3350020478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda74072160 msgr2=0x7fda741567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:43.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.794+0000 7fda79925640 1 --2- 172.21.15.43:0/3350020478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda74072160 0x7fda741567d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:43.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.794+0000 7fda79925640 1 -- 172.21.15.43:0/3350020478 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda741e3120 con 0x7fda7406d7d0 2024-04-25T08:54:43.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.794+0000 7fda79925640 1 --2- 172.21.15.43:0/3350020478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda7406d7d0 0x7fda7413bca0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fda70035040 tx=0x7fda700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:43.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.795+0000 7fda6a7fc640 1 -- 172.21.15.43:0/3350020478 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda70041030 con 0x7fda7406d7d0 2024-04-25T08:54:43.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.795+0000 7fda6a7fc640 1 -- 172.21.15.43:0/3350020478 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda7003d4e0 con 0x7fda7406d7d0 2024-04-25T08:54:43.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.795+0000 7fda7bb89640 1 -- 172.21.15.43:0/3350020478 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda741e3400 con 0x7fda7406d7d0 2024-04-25T08:54:43.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.795+0000 7fda6a7fc640 1 -- 172.21.15.43:0/3350020478 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda70037040 con 0x7fda7406d7d0 2024-04-25T08:54:43.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.795+0000 7fda7bb89640 1 -- 172.21.15.43:0/3350020478 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda741e39c0 con 0x7fda7406d7d0 2024-04-25T08:54:43.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.796+0000 7fda7bb89640 1 -- 172.21.15.43:0/3350020478 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda74157120 con 0x7fda7406d7d0 2024-04-25T08:54:43.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.796+0000 7fda6a7fc640 1 -- 172.21.15.43:0/3350020478 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fda70044070 con 0x7fda7406d7d0 2024-04-25T08:54:43.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.797+0000 7fda6a7fc640 1 --2- 172.21.15.43:0/3350020478 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fda5806b500 0x7fda5806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:43.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.797+0000 7fda6a7fc640 1 -- 172.21.15.43:0/3350020478 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda700aa740 con 0x7fda7406d7d0 2024-04-25T08:54:43.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.797+0000 7fda79124640 1 --2- 172.21.15.43:0/3350020478 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fda5806b500 0x7fda5806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:43.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.797+0000 7fda79124640 1 --2- 172.21.15.43:0/3350020478 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fda5806b500 0x7fda5806d9c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fda74154fc0 tx=0x7fda64007d70 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:43.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.801+0000 7fda6a7fc640 1 -- 172.21.15.43:0/3350020478 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fda74157120 con 0x7fda7406d7d0 2024-04-25T08:54:43.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.983+0000 7fda7bb89640 1 -- 172.21.15.43:0/3350020478 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fda74074ef0 con 0x7fda7406d7d0 2024-04-25T08:54:43.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.984+0000 7fda6a7fc640 1 -- 172.21.15.43:0/3350020478 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16569 (secure 0 0 0) 0x7fda70079000 con 0x7fda7406d7d0 2024-04-25T08:54:43.974 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:54:43.974 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":20,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:54:13.593188+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:54:09.757978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:54:10.926008+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:54:43.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.986+0000 7fda7bb89640 1 -- 172.21.15.43:0/3350020478 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fda5806b500 msgr2=0x7fda5806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:43.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.986+0000 7fda7bb89640 1 --2- 172.21.15.43:0/3350020478 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fda5806b500 0x7fda5806d9c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fda74154fc0 tx=0x7fda64007d70 comp rx=0 tx=0).stop 2024-04-25T08:54:43.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.986+0000 7fda7bb89640 1 -- 172.21.15.43:0/3350020478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda7406d7d0 msgr2=0x7fda7413bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:43.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.986+0000 7fda7bb89640 1 --2- 172.21.15.43:0/3350020478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda7406d7d0 0x7fda7413bca0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fda70035040 tx=0x7fda700046c0 comp rx=0 tx=0).stop 2024-04-25T08:54:43.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.986+0000 7fda7bb89640 1 -- 172.21.15.43:0/3350020478 shutdown_connections 2024-04-25T08:54:43.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.986+0000 7fda7bb89640 1 --2- 172.21.15.43:0/3350020478 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fda74156d10 0x7fda7413e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:43.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.986+0000 7fda7bb89640 1 --2- 172.21.15.43:0/3350020478 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fda5806b500 0x7fda5806d9c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:43.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.986+0000 7fda7bb89640 1 --2- 172.21.15.43:0/3350020478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fda74072160 0x7fda741567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:43.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.986+0000 7fda7bb89640 1 --2- 172.21.15.43:0/3350020478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fda7406d7d0 0x7fda7413bca0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:43.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.986+0000 7fda7bb89640 1 -- 172.21.15.43:0/3350020478 >> 172.21.15.43:0/3350020478 conn(0x7fda740b1480 msgr2=0x7fda740ccbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:43.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.986+0000 7fda7bb89640 1 -- 172.21.15.43:0/3350020478 shutdown_connections 2024-04-25T08:54:43.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:43.986+0000 7fda7bb89640 1 -- 172.21.15.43:0/3350020478 wait complete. 2024-04-25T08:54:43.987 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:54:09.757978+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:54:10.926008+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:54:43.988 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:54:44.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.101+0000 7fe2fc46b640 1 Processor -- start 2024-04-25T08:54:44.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.101+0000 7fe2fc46b640 1 -- start start 2024-04-25T08:54:44.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.102+0000 7fe2fc46b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2f415e090 0x7fe2f4161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:44.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.102+0000 7fe2fc46b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2f4162610 con 0x7fe2f4161fe0 2024-04-25T08:54:44.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.102+0000 7fe2fc46b640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2f40d1750 con 0x7fe2f40b5c70 2024-04-25T08:54:44.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.102+0000 7fe2fc46b640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f40d1da0 con 0x7fe2f415e090 2024-04-25T08:54:44.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.103+0000 7fe2f9a06640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe2f4161fe0 0x7fe2f40b5560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39656/0 (socket says 172.21.15.43:39656) 2024-04-25T08:54:44.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.103+0000 7fe2f9a06640 1 -- 172.21.15.43:0/639282918 learned_addr learned my addr 172.21.15.43:0/639282918 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:44.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.103+0000 7fe2fa207640 1 --2- 172.21.15.43:0/639282918 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2f415e090 0x7fe2f4161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:44.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.103+0000 7fe2f9205640 1 -- 172.21.15.43:0/639282918 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1709662949 0 0) 0x7fe2f4162610 con 0x7fe2f4161fe0 2024-04-25T08:54:44.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.103+0000 7fe2f9205640 1 -- 172.21.15.43:0/639282918 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2e0003520 con 0x7fe2f4161fe0 2024-04-25T08:54:44.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.103+0000 7fe2f9205640 1 -- 172.21.15.43:0/639282918 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1684035736 0 0) 0x7fe2e0003520 con 0x7fe2f4161fe0 2024-04-25T08:54:44.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.103+0000 7fe2f9205640 1 -- 172.21.15.43:0/639282918 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2f415e090 msgr2=0x7fe2f4161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:44.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.103+0000 7fe2f9205640 1 --2- 172.21.15.43:0/639282918 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2f415e090 0x7fe2f4161aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:44.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.103+0000 7fe2f9205640 1 -- 172.21.15.43:0/639282918 >> v1:172.21.15.193:6789/0 conn(0x7fe2f40b5c70 legacy=0x7fe2f40d2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:44.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.103+0000 7fe2f9205640 1 -- 172.21.15.43:0/639282918 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f40d1a30 con 0x7fe2f4161fe0 2024-04-25T08:54:44.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.103+0000 7fe2f9205640 1 -- 172.21.15.43:0/639282918 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7fe2e4001670 con 0x7fe2f4161fe0 2024-04-25T08:54:44.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.103+0000 7fe2fa207640 1 --2- 172.21.15.43:0/639282918 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2f415e090 0x7fe2f4161aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:54:44.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.103+0000 7fe2f9205640 1 -- 172.21.15.43:0/639282918 >> v1:172.21.15.43:6789/0 conn(0x7fe2f4161fe0 legacy=0x7fe2f40b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:44.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.103+0000 7fe2f9205640 1 --2- 172.21.15.43:0/639282918 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2f415e090 0x7fe2e00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:44.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.103+0000 7fe2f9205640 1 --2- 172.21.15.43:0/639282918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2e0003f20 0x7fe2e0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:44.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.103+0000 7fe2f9205640 1 --2- 172.21.15.43:0/639282918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2e0004d40 0x7fe2e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:44.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.103+0000 7fe2f9205640 1 -- 172.21.15.43:0/639282918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f40d1da0 con 0x7fe2e0004d40 2024-04-25T08:54:44.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.103+0000 7fe2f9205640 1 -- 172.21.15.43:0/639282918 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e000ba20 con 0x7fe2e0003f20 2024-04-25T08:54:44.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.103+0000 7fe2f9205640 1 -- 172.21.15.43:0/639282918 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e000bc20 con 0x7fe2f415e090 2024-04-25T08:54:44.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.103+0000 7fe2faa08640 1 --2- 172.21.15.43:0/639282918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2e0004d40 0x7fe2e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:44.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.103+0000 7fe2fa207640 1 --2- 172.21.15.43:0/639282918 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2f415e090 0x7fe2e00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:44.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.103+0000 7fe2f9a06640 1 --2- 172.21.15.43:0/639282918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2e0003f20 0x7fe2e0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:44.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.104+0000 7fe2faa08640 1 -- 172.21.15.43:0/639282918 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2f415e090 msgr2=0x7fe2e00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:44.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.104+0000 7fe2faa08640 1 --2- 172.21.15.43:0/639282918 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2f415e090 0x7fe2e00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:44.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.104+0000 7fe2faa08640 1 -- 172.21.15.43:0/639282918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2e0003f20 msgr2=0x7fe2e0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:44.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.104+0000 7fe2faa08640 1 --2- 172.21.15.43:0/639282918 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2e0003f20 0x7fe2e0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:44.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.104+0000 7fe2faa08640 1 -- 172.21.15.43:0/639282918 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e0003520 con 0x7fe2e0004d40 2024-04-25T08:54:44.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.104+0000 7fe2fa207640 1 --2- 172.21.15.43:0/639282918 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2f415e090 0x7fe2e00039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:54:44.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.104+0000 7fe2faa08640 1 --2- 172.21.15.43:0/639282918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2e0004d40 0x7fe2e00051c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec001f60 tx=0x7fe2e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a8a931ad5571cbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:44.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.105+0000 7fe2f9205640 1 -- 172.21.15.43:0/639282918 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec001b30 con 0x7fe2e0004d40 2024-04-25T08:54:44.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.105+0000 7fe2f9205640 1 -- 172.21.15.43:0/639282918 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2ec031620 con 0x7fe2e0004d40 2024-04-25T08:54:44.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.105+0000 7fe2f9205640 1 -- 172.21.15.43:0/639282918 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec031a10 con 0x7fe2e0004d40 2024-04-25T08:54:44.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.105+0000 7fe2fc46b640 1 -- 172.21.15.43:0/639282918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2e0004d40 msgr2=0x7fe2e00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:44.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.105+0000 7fe2fc46b640 1 --2- 172.21.15.43:0/639282918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2e0004d40 0x7fe2e00051c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec001f60 tx=0x7fe2e0005830 comp rx=0 tx=0).stop 2024-04-25T08:54:44.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.105+0000 7fe2faa08640 1 -- 172.21.15.43:0/639282918 reap_dead start 2024-04-25T08:54:44.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.105+0000 7fe2fc46b640 1 -- 172.21.15.43:0/639282918 shutdown_connections 2024-04-25T08:54:44.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.105+0000 7fe2fc46b640 1 -- 172.21.15.43:0/639282918 >> 172.21.15.43:0/639282918 conn(0x7fe2f40d0560 msgr2=0x7fe2f40d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:44.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.105+0000 7fe2fc46b640 1 -- 172.21.15.43:0/639282918 shutdown_connections 2024-04-25T08:54:44.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.105+0000 7fe2fc46b640 1 -- 172.21.15.43:0/639282918 wait complete. 2024-04-25T08:54:44.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.106+0000 7fe2fc46b640 1 Processor -- start 2024-04-25T08:54:44.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.107+0000 7fe2fc46b640 1 -- start start 2024-04-25T08:54:44.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.107+0000 7fe2fc46b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2e0004d40 0x7fe2f4160f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:44.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.107+0000 7fe2fc46b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2f415f0b0 0x7fe2f40c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:44.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.107+0000 7fe2fc46b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2f40c9150 0x7fe2f41f3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:44.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.107+0000 7fe2fc46b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f4161aa0 con 0x7fe2e0004d40 2024-04-25T08:54:44.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.107+0000 7fe2fc46b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f40b3280 con 0x7fe2f40c9150 2024-04-25T08:54:44.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.107+0000 7fe2fc46b640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2f4161c20 con 0x7fe2f415f0b0 2024-04-25T08:54:44.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.107+0000 7fe2fa207640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2e0004d40 0x7fe2f4160f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:44.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.107+0000 7fe2fa207640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2e0004d40 0x7fe2f4160f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38252/0 (socket says 172.21.15.43:38252) 2024-04-25T08:54:44.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.107+0000 7fe2fa207640 1 -- 172.21.15.43:0/4117826663 learned_addr learned my addr 172.21.15.43:0/4117826663 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:44.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.108+0000 7fe2faa08640 1 --2- 172.21.15.43:0/4117826663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2f40c9150 0x7fe2f41f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:44.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.108+0000 7fe2f9a06640 1 --2- 172.21.15.43:0/4117826663 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2f415f0b0 0x7fe2f40c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:44.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.108+0000 7fe2fa207640 1 -- 172.21.15.43:0/4117826663 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2f415f0b0 msgr2=0x7fe2f40c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:44.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.108+0000 7fe2fa207640 1 --2- 172.21.15.43:0/4117826663 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2f415f0b0 0x7fe2f40c8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:44.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.108+0000 7fe2fa207640 1 -- 172.21.15.43:0/4117826663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2f40c9150 msgr2=0x7fe2f41f3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:44.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.108+0000 7fe2fa207640 1 --2- 172.21.15.43:0/4117826663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2f40c9150 0x7fe2f41f3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:44.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.108+0000 7fe2fa207640 1 -- 172.21.15.43:0/4117826663 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f415e5f0 con 0x7fe2e0004d40 2024-04-25T08:54:44.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.108+0000 7fe2f9a06640 1 --2- 172.21.15.43:0/4117826663 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2f415f0b0 0x7fe2f40c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:44.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.108+0000 7fe2faa08640 1 --2- 172.21.15.43:0/4117826663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2f40c9150 0x7fe2f41f3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:44.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.108+0000 7fe2fa207640 1 --2- 172.21.15.43:0/4117826663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2e0004d40 0x7fe2f4160f00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0011ca0 tx=0x7fe2f000f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:44.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.108+0000 7fe2eaffd640 1 -- 172.21.15.43:0/4117826663 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f000fcc0 con 0x7fe2e0004d40 2024-04-25T08:54:44.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.108+0000 7fe2eaffd640 1 -- 172.21.15.43:0/4117826663 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f000fe60 con 0x7fe2e0004d40 2024-04-25T08:54:44.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.108+0000 7fe2eaffd640 1 -- 172.21.15.43:0/4117826663 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f0014640 con 0x7fe2e0004d40 2024-04-25T08:54:44.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.108+0000 7fe2fc46b640 1 -- 172.21.15.43:0/4117826663 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2f41f4500 con 0x7fe2e0004d40 2024-04-25T08:54:44.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.108+0000 7fe2fc46b640 1 -- 172.21.15.43:0/4117826663 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2f415f530 con 0x7fe2e0004d40 2024-04-25T08:54:44.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.109+0000 7fe2fc46b640 1 -- 172.21.15.43:0/4117826663 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2f406bf00 con 0x7fe2e0004d40 2024-04-25T08:54:44.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.109+0000 7fe2eaffd640 1 -- 172.21.15.43:0/4117826663 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe2f0012040 con 0x7fe2e0004d40 2024-04-25T08:54:44.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.110+0000 7fe2eaffd640 1 --2- 172.21.15.43:0/4117826663 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe2d806b500 0x7fe2d806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:44.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.110+0000 7fe2eaffd640 1 -- 172.21.15.43:0/4117826663 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2f008aa30 con 0x7fe2e0004d40 2024-04-25T08:54:44.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.110+0000 7fe2f9a06640 1 --2- 172.21.15.43:0/4117826663 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe2d806b500 0x7fe2d806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:44.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.111+0000 7fe2f9a06640 1 --2- 172.21.15.43:0/4117826663 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe2d806b500 0x7fe2d806d9c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4001050 tx=0x7fe2e4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:44.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.114+0000 7fe2eaffd640 1 -- 172.21.15.43:0/4117826663 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe2f406bf00 con 0x7fe2e0004d40 2024-04-25T08:54:44.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.294+0000 7fe2fc46b640 1 -- 172.21.15.43:0/4117826663 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe2f4074f60 con 0x7fe2e0004d40 2024-04-25T08:54:44.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.294+0000 7fe2eaffd640 1 -- 172.21.15.43:0/4117826663 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe2f0019030 con 0x7fe2e0004d40 2024-04-25T08:54:44.284 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T08:54:44.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.297+0000 7fe2fc46b640 1 -- 172.21.15.43:0/4117826663 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe2d806b500 msgr2=0x7fe2d806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:44.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.297+0000 7fe2fc46b640 1 --2- 172.21.15.43:0/4117826663 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe2d806b500 0x7fe2d806d9c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe2e4001050 tx=0x7fe2e4009040 comp rx=0 tx=0).stop 2024-04-25T08:54:44.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.297+0000 7fe2fc46b640 1 -- 172.21.15.43:0/4117826663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2e0004d40 msgr2=0x7fe2f4160f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:44.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.297+0000 7fe2fc46b640 1 --2- 172.21.15.43:0/4117826663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2e0004d40 0x7fe2f4160f00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe2f0011ca0 tx=0x7fe2f000f610 comp rx=0 tx=0).stop 2024-04-25T08:54:44.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.297+0000 7fe2fc46b640 1 -- 172.21.15.43:0/4117826663 shutdown_connections 2024-04-25T08:54:44.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.297+0000 7fe2fc46b640 1 --2- 172.21.15.43:0/4117826663 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe2d806b500 0x7fe2d806d9c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:44.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.297+0000 7fe2fc46b640 1 --2- 172.21.15.43:0/4117826663 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2f40c9150 0x7fe2f41f3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:44.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.297+0000 7fe2fc46b640 1 --2- 172.21.15.43:0/4117826663 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2f415f0b0 0x7fe2f40c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:44.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.297+0000 7fe2fc46b640 1 --2- 172.21.15.43:0/4117826663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2e0004d40 0x7fe2f4160f00 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:44.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.297+0000 7fe2fc46b640 1 -- 172.21.15.43:0/4117826663 >> 172.21.15.43:0/4117826663 conn(0x7fe2f40d0560 msgr2=0x7fe2f40b7430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:44.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.297+0000 7fe2fc46b640 1 -- 172.21.15.43:0/4117826663 shutdown_connections 2024-04-25T08:54:44.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.297+0000 7fe2fc46b640 1 -- 172.21.15.43:0/4117826663 wait complete. 2024-04-25T08:54:44.298 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:54:44.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.411+0000 7ff714ff5640 1 Processor -- start 2024-04-25T08:54:44.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.411+0000 7ff714ff5640 1 -- start start 2024-04-25T08:54:44.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.412+0000 7ff714ff5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff71015e090 0x7ff710161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:44.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.412+0000 7ff714ff5640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff710161fe0 0x7ff7100b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:44.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.412+0000 7ff714ff5640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff710162610 con 0x7ff7100b5aa0 2024-04-25T08:54:44.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.412+0000 7ff714ff5640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7100b5f00 con 0x7ff710161fe0 2024-04-25T08:54:44.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.412+0000 7ff714ff5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7100d1da0 con 0x7ff71015e090 2024-04-25T08:54:44.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.413+0000 7ff70e59c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff71015e090 0x7ff710161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:44.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.413+0000 7ff70dd9b640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff710161fe0 0x7ff7100b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:44.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.413+0000 7ff70ed9d640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7ff7100b5aa0 0x7ff7100ce560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:38760/0 (socket says 172.21.15.43:38760) 2024-04-25T08:54:44.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.413+0000 7ff70dd9b640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff710161fe0 0x7ff7100b5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:52798/0 (socket says 172.21.15.43:52798) 2024-04-25T08:54:44.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.413+0000 7ff70e59c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff71015e090 0x7ff710161aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:33508/0 (socket says 172.21.15.43:33508) 2024-04-25T08:54:44.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.413+0000 7ff70ed9d640 1 -- 172.21.15.43:0/309147055 learned_addr learned my addr 172.21.15.43:0/309147055 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:44.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.413+0000 7ff70e59c640 1 -- 172.21.15.43:0/309147055 >> v2:172.21.15.193:3300/0 conn(0x7ff710161fe0 msgr2=0x7ff7100b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:44.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.413+0000 7ff70e59c640 1 --2- 172.21.15.43:0/309147055 >> v2:172.21.15.193:3300/0 conn(0x7ff710161fe0 0x7ff7100b5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:44.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.413+0000 7ff70e59c640 1 -- 172.21.15.43:0/309147055 >> v1:172.21.15.193:6789/0 conn(0x7ff7100b5aa0 legacy=0x7ff7100ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:44.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.413+0000 7ff70e59c640 1 -- 172.21.15.43:0/309147055 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7100d1750 con 0x7ff71015e090 2024-04-25T08:54:44.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.414+0000 7ff70e59c640 1 --2- 172.21.15.43:0/309147055 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff71015e090 0x7ff710161aa0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff7080097a0 tx=0x7ff70802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2f1bf094ea68e86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:44.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.414+0000 7ff70d59a640 1 -- 172.21.15.43:0/309147055 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff708037070 con 0x7ff71015e090 2024-04-25T08:54:44.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.414+0000 7ff70d59a640 1 -- 172.21.15.43:0/309147055 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff708032440 con 0x7ff71015e090 2024-04-25T08:54:44.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.414+0000 7ff70d59a640 1 -- 172.21.15.43:0/309147055 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff708032810 con 0x7ff71015e090 2024-04-25T08:54:44.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.415+0000 7ff714ff5640 1 -- 172.21.15.43:0/309147055 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff71015e090 msgr2=0x7ff710161aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:44.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.415+0000 7ff714ff5640 1 --2- 172.21.15.43:0/309147055 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff71015e090 0x7ff710161aa0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff7080097a0 tx=0x7ff70802a210 comp rx=0 tx=0).stop 2024-04-25T08:54:44.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.415+0000 7ff714ff5640 1 -- 172.21.15.43:0/309147055 shutdown_connections 2024-04-25T08:54:44.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.415+0000 7ff714ff5640 1 --2- 172.21.15.43:0/309147055 >> v2:172.21.15.193:3300/0 conn(0x7ff710161fe0 0x7ff7100b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:44.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.415+0000 7ff714ff5640 1 --2- 172.21.15.43:0/309147055 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff71015e090 0x7ff710161aa0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:44.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.415+0000 7ff714ff5640 1 -- 172.21.15.43:0/309147055 >> 172.21.15.43:0/309147055 conn(0x7ff7100d0560 msgr2=0x7ff7100d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:44.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.415+0000 7ff714ff5640 1 -- 172.21.15.43:0/309147055 shutdown_connections 2024-04-25T08:54:44.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.415+0000 7ff714ff5640 1 -- 172.21.15.43:0/309147055 wait complete. 2024-04-25T08:54:44.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.416+0000 7ff714ff5640 1 Processor -- start 2024-04-25T08:54:44.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.416+0000 7ff714ff5640 1 -- start start 2024-04-25T08:54:44.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.417+0000 7ff714ff5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7100b5aa0 0x7ff710156470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:44.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.417+0000 7ff714ff5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff71015e090 0x7ff710160990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:44.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.417+0000 7ff714ff5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff710161fe0 0x7ff710160ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:44.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.417+0000 7ff714ff5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7100b3280 con 0x7ff71015e090 2024-04-25T08:54:44.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.417+0000 7ff714ff5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7101628f0 con 0x7ff710161fe0 2024-04-25T08:54:44.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.417+0000 7ff714ff5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7100d3240 con 0x7ff7100b5aa0 2024-04-25T08:54:44.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.417+0000 7ff70dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff71015e090 0x7ff710160990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:44.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.417+0000 7ff70dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff71015e090 0x7ff710160990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38260/0 (socket says 172.21.15.43:38260) 2024-04-25T08:54:44.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.417+0000 7ff70dd9b640 1 -- 172.21.15.43:0/1213948294 learned_addr learned my addr 172.21.15.43:0/1213948294 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:44.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.417+0000 7ff70e59c640 1 --2- 172.21.15.43:0/1213948294 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7100b5aa0 0x7ff710156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:44.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.417+0000 7ff70ed9d640 1 --2- 172.21.15.43:0/1213948294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff710161fe0 0x7ff710160ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:44.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.417+0000 7ff70dd9b640 1 -- 172.21.15.43:0/1213948294 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7100b5aa0 msgr2=0x7ff710156470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:44.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.417+0000 7ff70dd9b640 1 --2- 172.21.15.43:0/1213948294 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7100b5aa0 0x7ff710156470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:44.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.417+0000 7ff70dd9b640 1 -- 172.21.15.43:0/1213948294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff710161fe0 msgr2=0x7ff710160ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:44.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.417+0000 7ff70dd9b640 1 --2- 172.21.15.43:0/1213948294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff710161fe0 0x7ff710160ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:44.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.417+0000 7ff70dd9b640 1 -- 172.21.15.43:0/1213948294 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7101589f0 con 0x7ff71015e090 2024-04-25T08:54:44.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.418+0000 7ff70dd9b640 1 --2- 172.21.15.43:0/1213948294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff71015e090 0x7ff710160990 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff6f800e8a0 tx=0x7ff6f800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:44.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.418+0000 7ff706ffd640 1 -- 172.21.15.43:0/1213948294 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f8017070 con 0x7ff71015e090 2024-04-25T08:54:44.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.418+0000 7ff706ffd640 1 -- 172.21.15.43:0/1213948294 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f800c940 con 0x7ff71015e090 2024-04-25T08:54:44.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.418+0000 7ff706ffd640 1 -- 172.21.15.43:0/1213948294 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f800cd30 con 0x7ff71015e090 2024-04-25T08:54:44.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.418+0000 7ff714ff5640 1 -- 172.21.15.43:0/1213948294 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff710158cd0 con 0x7ff71015e090 2024-04-25T08:54:44.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.418+0000 7ff714ff5640 1 -- 172.21.15.43:0/1213948294 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff710159290 con 0x7ff71015e090 2024-04-25T08:54:44.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.419+0000 7ff714ff5640 1 -- 172.21.15.43:0/1213948294 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff71006bf00 con 0x7ff71015e090 2024-04-25T08:54:44.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.419+0000 7ff706ffd640 1 -- 172.21.15.43:0/1213948294 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff6f801a070 con 0x7ff71015e090 2024-04-25T08:54:44.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.420+0000 7ff706ffd640 1 --2- 172.21.15.43:0/1213948294 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff6f006b4d0 0x7ff6f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:44.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.420+0000 7ff706ffd640 1 -- 172.21.15.43:0/1213948294 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff6f808a430 con 0x7ff71015e090 2024-04-25T08:54:44.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.420+0000 7ff70e59c640 1 --2- 172.21.15.43:0/1213948294 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff6f006b4d0 0x7ff6f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:44.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.420+0000 7ff70e59c640 1 --2- 172.21.15.43:0/1213948294 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff6f006b4d0 0x7ff6f006d990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff708034000 tx=0x7ff70802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:44.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.424+0000 7ff706ffd640 1 -- 172.21.15.43:0/1213948294 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff71006bf00 con 0x7ff71015e090 2024-04-25T08:54:44.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.606+0000 7ff714ff5640 1 -- 172.21.15.43:0/1213948294 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff710074f60 con 0x7ff71015e090 2024-04-25T08:54:44.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.606+0000 7ff706ffd640 1 -- 172.21.15.43:0/1213948294 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff6f80592b0 con 0x7ff71015e090 2024-04-25T08:54:44.596 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T08:54:44.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.608+0000 7ff714ff5640 1 -- 172.21.15.43:0/1213948294 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff6f006b4d0 msgr2=0x7ff6f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:44.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.608+0000 7ff714ff5640 1 --2- 172.21.15.43:0/1213948294 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff6f006b4d0 0x7ff6f006d990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff708034000 tx=0x7ff70802e000 comp rx=0 tx=0).stop 2024-04-25T08:54:44.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.608+0000 7ff714ff5640 1 -- 172.21.15.43:0/1213948294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff71015e090 msgr2=0x7ff710160990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:44.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.608+0000 7ff714ff5640 1 --2- 172.21.15.43:0/1213948294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff71015e090 0x7ff710160990 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff6f800e8a0 tx=0x7ff6f800ed60 comp rx=0 tx=0).stop 2024-04-25T08:54:44.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.608+0000 7ff714ff5640 1 -- 172.21.15.43:0/1213948294 shutdown_connections 2024-04-25T08:54:44.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.608+0000 7ff714ff5640 1 --2- 172.21.15.43:0/1213948294 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff6f006b4d0 0x7ff6f006d990 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:44.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.608+0000 7ff714ff5640 1 --2- 172.21.15.43:0/1213948294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff710161fe0 0x7ff710160ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:44.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.608+0000 7ff714ff5640 1 --2- 172.21.15.43:0/1213948294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff71015e090 0x7ff710160990 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:44.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.608+0000 7ff714ff5640 1 --2- 172.21.15.43:0/1213948294 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7100b5aa0 0x7ff710156470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:44.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.608+0000 7ff714ff5640 1 -- 172.21.15.43:0/1213948294 >> 172.21.15.43:0/1213948294 conn(0x7ff7100d0560 msgr2=0x7ff7100b7c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:44.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.608+0000 7ff714ff5640 1 -- 172.21.15.43:0/1213948294 shutdown_connections 2024-04-25T08:54:44.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.608+0000 7ff714ff5640 1 -- 172.21.15.43:0/1213948294 wait complete. 2024-04-25T08:54:44.609 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:54:44.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.723+0000 7ff391438640 1 Processor -- start 2024-04-25T08:54:44.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.724+0000 7ff391438640 1 -- start start 2024-04-25T08:54:44.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.724+0000 7ff391438640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff38c15a960 0x7ff38c0b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:44.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.724+0000 7ff391438640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff38c15b110 con 0x7ff38c0b3ed0 2024-04-25T08:54:44.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.724+0000 7ff391438640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff38c15b3f0 con 0x7ff38c0aee40 2024-04-25T08:54:44.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.724+0000 7ff391438640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c15b6d0 con 0x7ff38c15a960 2024-04-25T08:54:44.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.724+0000 7ff38a7fc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff38c0b3ed0 0x7ff38c0ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39666/0 (socket says 172.21.15.43:39666) 2024-04-25T08:54:44.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.725+0000 7ff38a7fc640 1 -- 172.21.15.43:0/3055236798 learned_addr learned my addr 172.21.15.43:0/3055236798 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:44.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.725+0000 7ff38affd640 1 --2- 172.21.15.43:0/3055236798 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff38c15a960 0x7ff38c0b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:44.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.725+0000 7ff389ffb640 1 -- 172.21.15.43:0/3055236798 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2879804537 0 0) 0x7ff38c15b110 con 0x7ff38c0b3ed0 2024-04-25T08:54:44.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.725+0000 7ff389ffb640 1 -- 172.21.15.43:0/3055236798 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff374003520 con 0x7ff38c0b3ed0 2024-04-25T08:54:44.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.725+0000 7ff389ffb640 1 -- 172.21.15.43:0/3055236798 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 342151046 0 0) 0x7ff38c15b3f0 con 0x7ff38c0aee40 2024-04-25T08:54:44.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.725+0000 7ff389ffb640 1 -- 172.21.15.43:0/3055236798 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff38c15b110 con 0x7ff38c0aee40 2024-04-25T08:54:44.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.725+0000 7ff38affd640 1 -- 172.21.15.43:0/3055236798 >> v1:172.21.15.193:6789/0 conn(0x7ff38c0aee40 legacy=0x7ff38c070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:44.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.725+0000 7ff38affd640 1 -- 172.21.15.43:0/3055236798 >> v1:172.21.15.43:6789/0 conn(0x7ff38c0b3ed0 legacy=0x7ff38c0ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:44.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.725+0000 7ff38affd640 1 -- 172.21.15.43:0/3055236798 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff38c15b9b0 con 0x7ff38c15a960 2024-04-25T08:54:44.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.726+0000 7ff38affd640 1 --2- 172.21.15.43:0/3055236798 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff38c15a960 0x7ff38c0b3990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff37c0097d0 tx=0x7ff37c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba22005ab405776c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:44.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.726+0000 7ff389ffb640 1 -- 172.21.15.43:0/3055236798 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37c037070 con 0x7ff38c15a960 2024-04-25T08:54:44.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.726+0000 7ff389ffb640 1 -- 172.21.15.43:0/3055236798 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff37c032440 con 0x7ff38c15a960 2024-04-25T08:54:44.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.726+0000 7ff389ffb640 1 -- 172.21.15.43:0/3055236798 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37c032810 con 0x7ff38c15a960 2024-04-25T08:54:44.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.727+0000 7ff391438640 1 -- 172.21.15.43:0/3055236798 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff38c15a960 msgr2=0x7ff38c0b3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:44.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.727+0000 7ff391438640 1 --2- 172.21.15.43:0/3055236798 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff38c15a960 0x7ff38c0b3990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff37c0097d0 tx=0x7ff37c02a210 comp rx=0 tx=0).stop 2024-04-25T08:54:44.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.727+0000 7ff391438640 1 -- 172.21.15.43:0/3055236798 shutdown_connections 2024-04-25T08:54:44.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.727+0000 7ff391438640 1 --2- 172.21.15.43:0/3055236798 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff38c15a960 0x7ff38c0b3990 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:44.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.727+0000 7ff391438640 1 -- 172.21.15.43:0/3055236798 >> 172.21.15.43:0/3055236798 conn(0x7ff38c06ab30 msgr2=0x7ff38c06af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:44.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.727+0000 7ff391438640 1 -- 172.21.15.43:0/3055236798 shutdown_connections 2024-04-25T08:54:44.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.727+0000 7ff391438640 1 -- 172.21.15.43:0/3055236798 wait complete. 2024-04-25T08:54:44.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.728+0000 7ff391438640 1 Processor -- start 2024-04-25T08:54:44.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.728+0000 7ff391438640 1 -- start start 2024-04-25T08:54:44.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.729+0000 7ff391438640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff38c0aee40 0x7ff38c159650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:44.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.729+0000 7ff391438640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff38c0b3ed0 0x7ff38c1f5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:44.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.729+0000 7ff391438640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff38c15a960 0x7ff38c1f5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:44.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.729+0000 7ff391438640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c15ad40 con 0x7ff38c0aee40 2024-04-25T08:54:44.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.729+0000 7ff391438640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c0cd940 con 0x7ff38c0b3ed0 2024-04-25T08:54:44.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.729+0000 7ff391438640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c15aec0 con 0x7ff38c15a960 2024-04-25T08:54:44.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.729+0000 7ff38affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff38c0aee40 0x7ff38c159650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:44.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.729+0000 7ff38affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff38c0aee40 0x7ff38c159650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38262/0 (socket says 172.21.15.43:38262) 2024-04-25T08:54:44.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.729+0000 7ff38affd640 1 -- 172.21.15.43:0/674005518 learned_addr learned my addr 172.21.15.43:0/674005518 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:44.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.729+0000 7ff38b7fe640 1 --2- 172.21.15.43:0/674005518 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff38c15a960 0x7ff38c1f5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:44.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.729+0000 7ff38a7fc640 1 --2- 172.21.15.43:0/674005518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff38c0b3ed0 0x7ff38c1f5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:44.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.729+0000 7ff38affd640 1 -- 172.21.15.43:0/674005518 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff38c15a960 msgr2=0x7ff38c1f5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:44.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.729+0000 7ff38affd640 1 --2- 172.21.15.43:0/674005518 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff38c15a960 0x7ff38c1f5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:44.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.729+0000 7ff38affd640 1 -- 172.21.15.43:0/674005518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff38c0b3ed0 msgr2=0x7ff38c1f5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:44.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.729+0000 7ff38affd640 1 --2- 172.21.15.43:0/674005518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff38c0b3ed0 0x7ff38c1f5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:44.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.729+0000 7ff38affd640 1 -- 172.21.15.43:0/674005518 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff38c14d920 con 0x7ff38c0aee40 2024-04-25T08:54:44.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.729+0000 7ff38a7fc640 1 --2- 172.21.15.43:0/674005518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff38c0b3ed0 0x7ff38c1f5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:44.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.729+0000 7ff38affd640 1 --2- 172.21.15.43:0/674005518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff38c0aee40 0x7ff38c159650 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff37c000c00 tx=0x7ff37c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:44.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.730+0000 7ff3737fe640 1 -- 172.21.15.43:0/674005518 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37c037040 con 0x7ff38c0aee40 2024-04-25T08:54:44.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.730+0000 7ff3737fe640 1 -- 172.21.15.43:0/674005518 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff37c042070 con 0x7ff38c0aee40 2024-04-25T08:54:44.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.730+0000 7ff391438640 1 -- 172.21.15.43:0/674005518 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff38c1f5c10 con 0x7ff38c0aee40 2024-04-25T08:54:44.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.730+0000 7ff3737fe640 1 -- 172.21.15.43:0/674005518 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37c03b870 con 0x7ff38c0aee40 2024-04-25T08:54:44.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.730+0000 7ff391438640 1 -- 172.21.15.43:0/674005518 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff38c0ab3e0 con 0x7ff38c0aee40 2024-04-25T08:54:44.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.731+0000 7ff391438640 1 -- 172.21.15.43:0/674005518 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff38c0cd240 con 0x7ff38c0aee40 2024-04-25T08:54:44.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.731+0000 7ff3737fe640 1 -- 172.21.15.43:0/674005518 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff37c03ba10 con 0x7ff38c0aee40 2024-04-25T08:54:44.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.732+0000 7ff3737fe640 1 --2- 172.21.15.43:0/674005518 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff37806b4d0 0x7ff37806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:44.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.732+0000 7ff3737fe640 1 -- 172.21.15.43:0/674005518 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff37c0aa8f0 con 0x7ff38c0aee40 2024-04-25T08:54:44.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.732+0000 7ff38a7fc640 1 --2- 172.21.15.43:0/674005518 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff37806b4d0 0x7ff37806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:44.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.733+0000 7ff38a7fc640 1 --2- 172.21.15.43:0/674005518 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff37806b4d0 0x7ff37806d990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff38c0cb730 tx=0x7ff380009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:44.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.736+0000 7ff3737fe640 1 -- 172.21.15.43:0/674005518 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff38c0cd240 con 0x7ff38c0aee40 2024-04-25T08:54:44.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.916+0000 7ff391438640 1 -- 172.21.15.43:0/674005518 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff38c0cc220 con 0x7ff38c0aee40 2024-04-25T08:54:44.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.916+0000 7ff3737fe640 1 -- 172.21.15.43:0/674005518 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff38c14d920 con 0x7ff38c0aee40 2024-04-25T08:54:44.906 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T08:54:44.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.918+0000 7ff391438640 1 -- 172.21.15.43:0/674005518 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff37806b4d0 msgr2=0x7ff37806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:44.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.918+0000 7ff391438640 1 --2- 172.21.15.43:0/674005518 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff37806b4d0 0x7ff37806d990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff38c0cb730 tx=0x7ff380009040 comp rx=0 tx=0).stop 2024-04-25T08:54:44.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.918+0000 7ff391438640 1 -- 172.21.15.43:0/674005518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff38c0aee40 msgr2=0x7ff38c159650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:44.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.918+0000 7ff391438640 1 --2- 172.21.15.43:0/674005518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff38c0aee40 0x7ff38c159650 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff37c000c00 tx=0x7ff37c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:54:44.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.918+0000 7ff391438640 1 -- 172.21.15.43:0/674005518 shutdown_connections 2024-04-25T08:54:44.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.918+0000 7ff391438640 1 --2- 172.21.15.43:0/674005518 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff38c15a960 0x7ff38c1f5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:44.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.918+0000 7ff391438640 1 --2- 172.21.15.43:0/674005518 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff37806b4d0 0x7ff37806d990 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:44.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.918+0000 7ff391438640 1 --2- 172.21.15.43:0/674005518 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff38c0b3ed0 0x7ff38c1f5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:44.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.918+0000 7ff391438640 1 --2- 172.21.15.43:0/674005518 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff38c0aee40 0x7ff38c159650 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:44.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.918+0000 7ff391438640 1 -- 172.21.15.43:0/674005518 >> 172.21.15.43:0/674005518 conn(0x7ff38c06ab30 msgr2=0x7ff38c15c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:44.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.918+0000 7ff391438640 1 -- 172.21.15.43:0/674005518 shutdown_connections 2024-04-25T08:54:44.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:44.918+0000 7ff391438640 1 -- 172.21.15.43:0/674005518 wait complete. 2024-04-25T08:54:44.919 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:54:45.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.031+0000 7f6706df4640 1 Processor -- start 2024-04-25T08:54:45.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.031+0000 7f6706df4640 1 -- start start 2024-04-25T08:54:45.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.031+0000 7f6706df4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f67000cbfc0 0x7f670006dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:45.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.031+0000 7f6706df4640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f67000ccbb0 0x7f67000cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:45.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.031+0000 7f6706df4640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67000b2780 con 0x7f670015f5e0 2024-04-25T08:54:45.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.031+0000 7f6706df4640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67000cd550 con 0x7f67000cbfc0 2024-04-25T08:54:45.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.031+0000 7f6706df4640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67000b2a60 con 0x7f67000ccbb0 2024-04-25T08:54:45.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.032+0000 7f6704b90640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f67000cbfc0 0x7f670006dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:45.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.032+0000 7f6704b90640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f67000cbfc0 0x7f670006dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38272/0 (socket says 172.21.15.43:38272) 2024-04-25T08:54:45.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.032+0000 7f6704b90640 1 -- 172.21.15.43:0/807773695 learned_addr learned my addr 172.21.15.43:0/807773695 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:45.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.032+0000 7f6705391640 1 --1- 172.21.15.43:0/807773695 >> v1:172.21.15.43:6789/0 conn(0x7f670015f5e0 0x7f670015fa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39672/0 (socket says 172.21.15.43:39672) 2024-04-25T08:54:45.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.032+0000 7f66fffff640 1 --2- 172.21.15.43:0/807773695 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f67000ccbb0 0x7f67000cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:45.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.032+0000 7f6704b90640 1 -- 172.21.15.43:0/807773695 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f67000ccbb0 msgr2=0x7f67000cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:45.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.032+0000 7f6704b90640 1 --2- 172.21.15.43:0/807773695 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f67000ccbb0 0x7f67000cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.032+0000 7f6704b90640 1 -- 172.21.15.43:0/807773695 >> v1:172.21.15.43:6789/0 conn(0x7f670015f5e0 legacy=0x7f670015fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:45.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.032+0000 7f66ff7fe640 1 -- 172.21.15.43:0/807773695 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 345740873 0 0) 0x7f67000b2780 con 0x7f670015f5e0 2024-04-25T08:54:45.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.032+0000 7f6704b90640 1 -- 172.21.15.43:0/807773695 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67000b2d40 con 0x7f67000cbfc0 2024-04-25T08:54:45.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.032+0000 7f66fffff640 1 --2- 172.21.15.43:0/807773695 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f67000ccbb0 0x7f67000cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:45.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.033+0000 7f6704b90640 1 --2- 172.21.15.43:0/807773695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67000cbfc0 0x7f670006dc60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f66f8009e50 tx=0x7f66f802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ef03fd2621b8bbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:45.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.033+0000 7f66ff7fe640 1 -- 172.21.15.43:0/807773695 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f8036070 con 0x7f67000cbfc0 2024-04-25T08:54:45.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.033+0000 7f66ff7fe640 1 -- 172.21.15.43:0/807773695 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f8031600 con 0x7f67000cbfc0 2024-04-25T08:54:45.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.033+0000 7f66ff7fe640 1 -- 172.21.15.43:0/807773695 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f80319d0 con 0x7f67000cbfc0 2024-04-25T08:54:45.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.033+0000 7f6706df4640 1 -- 172.21.15.43:0/807773695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67000cbfc0 msgr2=0x7f670006dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:45.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.033+0000 7f6706df4640 1 --2- 172.21.15.43:0/807773695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67000cbfc0 0x7f670006dc60 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f66f8009e50 tx=0x7f66f802a710 comp rx=0 tx=0).stop 2024-04-25T08:54:45.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.033+0000 7f6706df4640 1 -- 172.21.15.43:0/807773695 shutdown_connections 2024-04-25T08:54:45.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.033+0000 7f6706df4640 1 --2- 172.21.15.43:0/807773695 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f67000ccbb0 0x7f67000cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.033+0000 7f6706df4640 1 --2- 172.21.15.43:0/807773695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67000cbfc0 0x7f670006dc60 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.033+0000 7f6706df4640 1 -- 172.21.15.43:0/807773695 >> 172.21.15.43:0/807773695 conn(0x7f67000b1480 msgr2=0x7f67000b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:45.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.033+0000 7f6706df4640 1 -- 172.21.15.43:0/807773695 shutdown_connections 2024-04-25T08:54:45.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.033+0000 7f6706df4640 1 -- 172.21.15.43:0/807773695 wait complete. 2024-04-25T08:54:45.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.035+0000 7f6706df4640 1 Processor -- start 2024-04-25T08:54:45.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.035+0000 7f6706df4640 1 -- start start 2024-04-25T08:54:45.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.035+0000 7f6706df4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f67000cbfc0 0x7f67000aa520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:45.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.036+0000 7f6706df4640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f67000ccbb0 0x7f67000cea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:45.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.036+0000 7f6706df4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f670015f5e0 0x7f67000cefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:45.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.036+0000 7f6706df4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67000b35e0 con 0x7f670015f5e0 2024-04-25T08:54:45.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.036+0000 7f6706df4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67000b3300 con 0x7f67000cbfc0 2024-04-25T08:54:45.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.036+0000 7f6706df4640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f670006e1a0 con 0x7f67000ccbb0 2024-04-25T08:54:45.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.036+0000 7f6705391640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f670015f5e0 0x7f67000cefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:45.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.036+0000 7f6705391640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f670015f5e0 0x7f67000cefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38288/0 (socket says 172.21.15.43:38288) 2024-04-25T08:54:45.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.036+0000 7f6705391640 1 -- 172.21.15.43:0/310236305 learned_addr learned my addr 172.21.15.43:0/310236305 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:45.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.036+0000 7f66fffff640 1 --2- 172.21.15.43:0/310236305 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f67000ccbb0 0x7f67000cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:45.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.036+0000 7f6704b90640 1 --2- 172.21.15.43:0/310236305 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f67000cbfc0 0x7f67000aa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:45.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.036+0000 7f6705391640 1 -- 172.21.15.43:0/310236305 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f67000ccbb0 msgr2=0x7f67000cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:45.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.036+0000 7f6705391640 1 --2- 172.21.15.43:0/310236305 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f67000ccbb0 0x7f67000cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.036+0000 7f6705391640 1 -- 172.21.15.43:0/310236305 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f67000cbfc0 msgr2=0x7f67000aa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:45.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.036+0000 7f6705391640 1 --2- 172.21.15.43:0/310236305 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f67000cbfc0 0x7f67000aa520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.036+0000 7f6705391640 1 -- 172.21.15.43:0/310236305 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67000aad70 con 0x7f670015f5e0 2024-04-25T08:54:45.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.036+0000 7f6704b90640 1 --2- 172.21.15.43:0/310236305 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f67000cbfc0 0x7f67000aa520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:45.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.036+0000 7f6705391640 1 --2- 172.21.15.43:0/310236305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f670015f5e0 0x7f67000cefd0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f66f000c3c0 tx=0x7f66f000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:45.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.036+0000 7f66fd7fa640 1 -- 172.21.15.43:0/310236305 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f00074e0 con 0x7f670015f5e0 2024-04-25T08:54:45.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.036+0000 7f66fd7fa640 1 -- 172.21.15.43:0/310236305 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f00047e0 con 0x7f670015f5e0 2024-04-25T08:54:45.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.036+0000 7f6706df4640 1 -- 172.21.15.43:0/310236305 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67000ab050 con 0x7f670015f5e0 2024-04-25T08:54:45.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.036+0000 7f66fd7fa640 1 -- 172.21.15.43:0/310236305 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f0004c50 con 0x7f670015f5e0 2024-04-25T08:54:45.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.036+0000 7f6706df4640 1 -- 172.21.15.43:0/310236305 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67000aaa60 con 0x7f670015f5e0 2024-04-25T08:54:45.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.037+0000 7f6706df4640 1 -- 172.21.15.43:0/310236305 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6700074ef0 con 0x7f670015f5e0 2024-04-25T08:54:45.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.038+0000 7f66fd7fa640 1 -- 172.21.15.43:0/310236305 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f66f0011070 con 0x7f670015f5e0 2024-04-25T08:54:45.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.038+0000 7f66fd7fa640 1 --2- 172.21.15.43:0/310236305 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f66ec06b4d0 0x7f66ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:45.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.038+0000 7f66fd7fa640 1 -- 172.21.15.43:0/310236305 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f66f0089c80 con 0x7f670015f5e0 2024-04-25T08:54:45.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.038+0000 7f6704b90640 1 --2- 172.21.15.43:0/310236305 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f66ec06b4d0 0x7f66ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:45.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.039+0000 7f6704b90640 1 --2- 172.21.15.43:0/310236305 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f66ec06b4d0 0x7f66ec06d990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f66f8033000 tx=0x7f66f8009000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:45.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.042+0000 7f66fd7fa640 1 -- 172.21.15.43:0/310236305 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f66f0058b90 con 0x7f670015f5e0 2024-04-25T08:54:45.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.233+0000 7f6706df4640 1 -- 172.21.15.43:0/310236305 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f67000b0610 con 0x7f670015f5e0 2024-04-25T08:54:45.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.234+0000 7f66fd7fa640 1 -- 172.21.15.43:0/310236305 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f66f0058490 con 0x7f670015f5e0 2024-04-25T08:54:45.224 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-04-25T08:54:45.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.236+0000 7f6706df4640 1 -- 172.21.15.43:0/310236305 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f66ec06b4d0 msgr2=0x7f66ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:45.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.236+0000 7f6706df4640 1 --2- 172.21.15.43:0/310236305 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f66ec06b4d0 0x7f66ec06d990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f66f8033000 tx=0x7f66f8009000 comp rx=0 tx=0).stop 2024-04-25T08:54:45.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.236+0000 7f6706df4640 1 -- 172.21.15.43:0/310236305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f670015f5e0 msgr2=0x7f67000cefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:45.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.236+0000 7f6706df4640 1 --2- 172.21.15.43:0/310236305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f670015f5e0 0x7f67000cefd0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f66f000c3c0 tx=0x7f66f000c880 comp rx=0 tx=0).stop 2024-04-25T08:54:45.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.236+0000 7f6706df4640 1 -- 172.21.15.43:0/310236305 shutdown_connections 2024-04-25T08:54:45.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.236+0000 7f6706df4640 1 --2- 172.21.15.43:0/310236305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f670015f5e0 0x7f67000cefd0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.236+0000 7f6706df4640 1 --2- 172.21.15.43:0/310236305 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f67000ccbb0 0x7f67000cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.236+0000 7f6706df4640 1 --2- 172.21.15.43:0/310236305 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f66ec06b4d0 0x7f66ec06d990 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.236+0000 7f6706df4640 1 --2- 172.21.15.43:0/310236305 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f67000cbfc0 0x7f67000aa520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.236+0000 7f6706df4640 1 -- 172.21.15.43:0/310236305 >> 172.21.15.43:0/310236305 conn(0x7f67000b1480 msgr2=0x7f670015b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:45.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.236+0000 7f6706df4640 1 -- 172.21.15.43:0/310236305 shutdown_connections 2024-04-25T08:54:45.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.236+0000 7f6706df4640 1 -- 172.21.15.43:0/310236305 wait complete. 2024-04-25T08:54:45.237 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T08:54:45.237 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:54:45.257 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:54:45.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.368+0000 7f2f83f6a640 1 Processor -- start 2024-04-25T08:54:45.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.368+0000 7f2f83f6a640 1 -- start start 2024-04-25T08:54:45.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.369+0000 7f2f83f6a640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f2f7c15def0 0x7f2f7c06be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:45.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.369+0000 7f2f83f6a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2f7c162560 0x7f2f7c0b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:45.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.369+0000 7f2f83f6a640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f7c15e2d0 con 0x7f2f7c1619f0 2024-04-25T08:54:45.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.369+0000 7f2f83f6a640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f7c0d0e90 con 0x7f2f7c15def0 2024-04-25T08:54:45.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.369+0000 7f2f83f6a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7c06c390 con 0x7f2f7c162560 2024-04-25T08:54:45.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.369+0000 7f2f82507640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2f7c162560 0x7f2f7c0b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:45.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.369+0000 7f2f82507640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2f7c162560 0x7f2f7c0b20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:33578/0 (socket says 172.21.15.43:33578) 2024-04-25T08:54:45.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.369+0000 7f2f81505640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f2f7c1619f0 0x7f2f7c161e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:38778/0 (socket says 172.21.15.43:38778) 2024-04-25T08:54:45.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.369+0000 7f2f82507640 1 -- 172.21.15.43:0/2318565505 learned_addr learned my addr 172.21.15.43:0/2318565505 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:45.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.369+0000 7f2f81d06640 1 --2- 172.21.15.43:0/2318565505 >> v2:172.21.15.193:3300/0 conn(0x7f2f7c15def0 0x7f2f7c06be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:45.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.370+0000 7f2f80d04640 1 -- 172.21.15.43:0/2318565505 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 935592166 0 0) 0x7f2f7c15e2d0 con 0x7f2f7c1619f0 2024-04-25T08:54:45.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.370+0000 7f2f82507640 1 -- 172.21.15.43:0/2318565505 >> v2:172.21.15.193:3300/0 conn(0x7f2f7c15def0 msgr2=0x7f2f7c06be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:45.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.370+0000 7f2f82507640 1 --2- 172.21.15.43:0/2318565505 >> v2:172.21.15.193:3300/0 conn(0x7f2f7c15def0 0x7f2f7c06be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.370+0000 7f2f82507640 1 -- 172.21.15.43:0/2318565505 >> v1:172.21.15.193:6789/0 conn(0x7f2f7c1619f0 legacy=0x7f2f7c161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:45.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.370+0000 7f2f82507640 1 -- 172.21.15.43:0/2318565505 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f7c0b1020 con 0x7f2f7c162560 2024-04-25T08:54:45.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.371+0000 7f2f82507640 1 --2- 172.21.15.43:0/2318565505 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2f7c162560 0x7f2f7c0b20c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2f74009e50 tx=0x7f2f7402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42ac9f2d9b5b96a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:45.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.371+0000 7f2f80d04640 1 -- 172.21.15.43:0/2318565505 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f74036070 con 0x7f2f7c162560 2024-04-25T08:54:45.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.371+0000 7f2f80d04640 1 -- 172.21.15.43:0/2318565505 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f74031600 con 0x7f2f7c162560 2024-04-25T08:54:45.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.371+0000 7f2f80d04640 1 -- 172.21.15.43:0/2318565505 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f740319d0 con 0x7f2f7c162560 2024-04-25T08:54:45.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.371+0000 7f2f83f6a640 1 -- 172.21.15.43:0/2318565505 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2f7c162560 msgr2=0x7f2f7c0b20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:45.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.371+0000 7f2f83f6a640 1 --2- 172.21.15.43:0/2318565505 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2f7c162560 0x7f2f7c0b20c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2f74009e50 tx=0x7f2f7402a710 comp rx=0 tx=0).stop 2024-04-25T08:54:45.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.371+0000 7f2f83f6a640 1 -- 172.21.15.43:0/2318565505 shutdown_connections 2024-04-25T08:54:45.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.371+0000 7f2f83f6a640 1 --2- 172.21.15.43:0/2318565505 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2f7c162560 0x7f2f7c0b20c0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.371+0000 7f2f83f6a640 1 --2- 172.21.15.43:0/2318565505 >> v2:172.21.15.193:3300/0 conn(0x7f2f7c15def0 0x7f2f7c06be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.371+0000 7f2f83f6a640 1 -- 172.21.15.43:0/2318565505 >> 172.21.15.43:0/2318565505 conn(0x7f2f7c0b32e0 msgr2=0x7f2f7c0b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:45.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.371+0000 7f2f83f6a640 1 -- 172.21.15.43:0/2318565505 shutdown_connections 2024-04-25T08:54:45.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.371+0000 7f2f83f6a640 1 -- 172.21.15.43:0/2318565505 wait complete. 2024-04-25T08:54:45.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.373+0000 7f2f83f6a640 1 Processor -- start 2024-04-25T08:54:45.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.373+0000 7f2f83f6a640 1 -- start start 2024-04-25T08:54:45.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.373+0000 7f2f83f6a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2f7c15def0 0x7f2f7c149780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:45.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.373+0000 7f2f83f6a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f7c1619f0 0x7f2f7c149cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:45.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.373+0000 7f2f83f6a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f7c162560 0x7f2f7c1f6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:45.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.373+0000 7f2f83f6a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7c0b18c0 con 0x7f2f7c1619f0 2024-04-25T08:54:45.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.373+0000 7f2f83f6a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7c0b15e0 con 0x7f2f7c162560 2024-04-25T08:54:45.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.373+0000 7f2f83f6a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7c0b1ba0 con 0x7f2f7c15def0 2024-04-25T08:54:45.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.374+0000 7f2f81505640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f7c1619f0 0x7f2f7c149cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:45.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.374+0000 7f2f81505640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f7c1619f0 0x7f2f7c149cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38304/0 (socket says 172.21.15.43:38304) 2024-04-25T08:54:45.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.374+0000 7f2f81505640 1 -- 172.21.15.43:0/1985572575 learned_addr learned my addr 172.21.15.43:0/1985572575 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:45.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.374+0000 7f2f81d06640 1 --2- 172.21.15.43:0/1985572575 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2f7c15def0 0x7f2f7c149780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:45.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.374+0000 7f2f82507640 1 --2- 172.21.15.43:0/1985572575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f7c162560 0x7f2f7c1f6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:45.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.374+0000 7f2f81505640 1 -- 172.21.15.43:0/1985572575 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2f7c15def0 msgr2=0x7f2f7c149780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:45.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.374+0000 7f2f81505640 1 --2- 172.21.15.43:0/1985572575 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2f7c15def0 0x7f2f7c149780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.374+0000 7f2f81505640 1 -- 172.21.15.43:0/1985572575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f7c162560 msgr2=0x7f2f7c1f6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:45.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.374+0000 7f2f81505640 1 --2- 172.21.15.43:0/1985572575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f7c162560 0x7f2f7c1f6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.374+0000 7f2f81505640 1 -- 172.21.15.43:0/1985572575 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f7c1612d0 con 0x7f2f7c1619f0 2024-04-25T08:54:45.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.374+0000 7f2f81505640 1 --2- 172.21.15.43:0/1985572575 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f7c1619f0 0x7f2f7c149cc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c00c460 tx=0x7f2f6c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:45.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.374+0000 7f2f727fc640 1 -- 172.21.15.43:0/1985572575 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c018070 con 0x7f2f7c1619f0 2024-04-25T08:54:45.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.374+0000 7f2f727fc640 1 -- 172.21.15.43:0/1985572575 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f6c0074e0 con 0x7f2f7c1619f0 2024-04-25T08:54:45.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.374+0000 7f2f83f6a640 1 -- 172.21.15.43:0/1985572575 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f7c0c6cf0 con 0x7f2f7c1619f0 2024-04-25T08:54:45.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.374+0000 7f2f727fc640 1 -- 172.21.15.43:0/1985572575 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c004a50 con 0x7f2f7c1619f0 2024-04-25T08:54:45.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.374+0000 7f2f83f6a640 1 -- 172.21.15.43:0/1985572575 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f7c0c7590 con 0x7f2f7c1619f0 2024-04-25T08:54:45.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.375+0000 7f2f83f6a640 1 -- 172.21.15.43:0/1985572575 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f7c06fc20 con 0x7f2f7c1619f0 2024-04-25T08:54:45.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.376+0000 7f2f727fc640 1 -- 172.21.15.43:0/1985572575 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2f6c004bf0 con 0x7f2f7c1619f0 2024-04-25T08:54:45.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.376+0000 7f2f727fc640 1 --2- 172.21.15.43:0/1985572575 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2f6006b4d0 0x7f2f6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:45.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.376+0000 7f2f727fc640 1 -- 172.21.15.43:0/1985572575 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2f6c089480 con 0x7f2f7c1619f0 2024-04-25T08:54:45.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.376+0000 7f2f81d06640 1 --2- 172.21.15.43:0/1985572575 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2f6006b4d0 0x7f2f6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:45.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.377+0000 7f2f81d06640 1 --2- 172.21.15.43:0/1985572575 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2f6006b4d0 0x7f2f6006d990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2f78009aa0 tx=0x7f2f78009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:45.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.380+0000 7f2f727fc640 1 -- 172.21.15.43:0/1985572575 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2f7c06fc20 con 0x7f2f7c1619f0 2024-04-25T08:54:45.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.562+0000 7f2f83f6a640 1 -- 172.21.15.43:0/1985572575 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f7c074ef0 con 0x7f2f7c1619f0 2024-04-25T08:54:45.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.563+0000 7f2f727fc640 1 -- 172.21.15.43:0/1985572575 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16569 (secure 0 0 0) 0x7f2f6c057cf0 con 0x7f2f7c1619f0 2024-04-25T08:54:45.554 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:54:45.554 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":20,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:54:13.593188+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:54:09.757978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:54:10.926008+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:54:45.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.566+0000 7f2f83f6a640 1 -- 172.21.15.43:0/1985572575 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2f6006b4d0 msgr2=0x7f2f6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:45.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.566+0000 7f2f83f6a640 1 --2- 172.21.15.43:0/1985572575 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2f6006b4d0 0x7f2f6006d990 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2f78009aa0 tx=0x7f2f78009190 comp rx=0 tx=0).stop 2024-04-25T08:54:45.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.566+0000 7f2f83f6a640 1 -- 172.21.15.43:0/1985572575 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f7c1619f0 msgr2=0x7f2f7c149cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:45.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.566+0000 7f2f83f6a640 1 --2- 172.21.15.43:0/1985572575 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f7c1619f0 0x7f2f7c149cc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c00c460 tx=0x7f2f6c00c920 comp rx=0 tx=0).stop 2024-04-25T08:54:45.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.566+0000 7f2f83f6a640 1 -- 172.21.15.43:0/1985572575 shutdown_connections 2024-04-25T08:54:45.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.566+0000 7f2f83f6a640 1 --2- 172.21.15.43:0/1985572575 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2f6006b4d0 0x7f2f6006d990 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.566+0000 7f2f83f6a640 1 --2- 172.21.15.43:0/1985572575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2f7c162560 0x7f2f7c1f6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.566+0000 7f2f83f6a640 1 --2- 172.21.15.43:0/1985572575 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2f7c1619f0 0x7f2f7c149cc0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.566+0000 7f2f83f6a640 1 --2- 172.21.15.43:0/1985572575 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2f7c15def0 0x7f2f7c149780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.566+0000 7f2f83f6a640 1 -- 172.21.15.43:0/1985572575 >> 172.21.15.43:0/1985572575 conn(0x7f2f7c0b32e0 msgr2=0x7f2f7c161e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:45.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.566+0000 7f2f83f6a640 1 -- 172.21.15.43:0/1985572575 shutdown_connections 2024-04-25T08:54:45.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.566+0000 7f2f83f6a640 1 -- 172.21.15.43:0/1985572575 wait complete. 2024-04-25T08:54:45.567 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:54:09.757978+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:54:10.926008+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:54:45.567 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:54:45.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.680+0000 7f41d479e640 1 Processor -- start 2024-04-25T08:54:45.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.680+0000 7f41d479e640 1 -- start start 2024-04-25T08:54:45.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.681+0000 7f41d479e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f41cc15dd60 0x7f41cc0d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:45.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.681+0000 7f41d479e640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f41cc0d1fa0 0x7f41cc0d0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:45.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.681+0000 7f41d479e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f41cc0d08d0 0x7f41cc0ce5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:45.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.681+0000 7f41d479e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41cc0d0d60 con 0x7f41cc15dd60 2024-04-25T08:54:45.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.681+0000 7f41d479e640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41cc15e140 con 0x7f41cc0d1fa0 2024-04-25T08:54:45.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.681+0000 7f41d479e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41cc0d27d0 con 0x7f41cc0d08d0 2024-04-25T08:54:45.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.681+0000 7f41d253a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f41cc15dd60 0x7f41cc0d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:45.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.682+0000 7f41d253a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f41cc15dd60 0x7f41cc0d1000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38308/0 (socket says 172.21.15.43:38308) 2024-04-25T08:54:45.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.682+0000 7f41d253a640 1 -- 172.21.15.43:0/3905090818 learned_addr learned my addr 172.21.15.43:0/3905090818 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:45.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.682+0000 7f41d2d3b640 1 --2- 172.21.15.43:0/3905090818 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f41cc0d08d0 0x7f41cc0ce5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:45.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.682+0000 7f41d1d39640 1 --2- 172.21.15.43:0/3905090818 >> v2:172.21.15.193:3300/0 conn(0x7f41cc0d1fa0 0x7f41cc0d0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:45.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.682+0000 7f41d253a640 1 -- 172.21.15.43:0/3905090818 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f41cc0d08d0 msgr2=0x7f41cc0ce5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:45.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.682+0000 7f41d253a640 1 --2- 172.21.15.43:0/3905090818 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f41cc0d08d0 0x7f41cc0ce5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.682+0000 7f41d253a640 1 -- 172.21.15.43:0/3905090818 >> v2:172.21.15.193:3300/0 conn(0x7f41cc0d1fa0 msgr2=0x7f41cc0d0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:45.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.682+0000 7f41d253a640 1 --2- 172.21.15.43:0/3905090818 >> v2:172.21.15.193:3300/0 conn(0x7f41cc0d1fa0 0x7f41cc0d0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.682+0000 7f41d253a640 1 -- 172.21.15.43:0/3905090818 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41cc06b1e0 con 0x7f41cc15dd60 2024-04-25T08:54:45.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.682+0000 7f41d1d39640 1 --2- 172.21.15.43:0/3905090818 >> v2:172.21.15.193:3300/0 conn(0x7f41cc0d1fa0 0x7f41cc0d0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:45.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.683+0000 7f41d253a640 1 --2- 172.21.15.43:0/3905090818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41cc15dd60 0x7f41cc0d1000 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f41cc0d1540 tx=0x7f41c802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37ee7a13b3684274 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:45.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.683+0000 7f41d1538640 1 -- 172.21.15.43:0/3905090818 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c8037070 con 0x7f41cc15dd60 2024-04-25T08:54:45.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.683+0000 7f41d1538640 1 -- 172.21.15.43:0/3905090818 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41c8032440 con 0x7f41cc15dd60 2024-04-25T08:54:45.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.683+0000 7f41d1538640 1 -- 172.21.15.43:0/3905090818 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c8032810 con 0x7f41cc15dd60 2024-04-25T08:54:45.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.683+0000 7f41d479e640 1 -- 172.21.15.43:0/3905090818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41cc15dd60 msgr2=0x7f41cc0d1000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:45.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.683+0000 7f41d479e640 1 --2- 172.21.15.43:0/3905090818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41cc15dd60 0x7f41cc0d1000 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f41cc0d1540 tx=0x7f41c802a1e0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.683+0000 7f41d479e640 1 -- 172.21.15.43:0/3905090818 shutdown_connections 2024-04-25T08:54:45.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.683+0000 7f41d479e640 1 --2- 172.21.15.43:0/3905090818 >> v2:172.21.15.193:3300/0 conn(0x7f41cc0d1fa0 0x7f41cc0d0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.683+0000 7f41d479e640 1 --2- 172.21.15.43:0/3905090818 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f41cc0d08d0 0x7f41cc0ce5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.683+0000 7f41d479e640 1 --2- 172.21.15.43:0/3905090818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41cc15dd60 0x7f41cc0d1000 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.683+0000 7f41d479e640 1 -- 172.21.15.43:0/3905090818 >> 172.21.15.43:0/3905090818 conn(0x7f41cc0d39c0 msgr2=0x7f41cc0d3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:45.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.683+0000 7f41d479e640 1 -- 172.21.15.43:0/3905090818 shutdown_connections 2024-04-25T08:54:45.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.683+0000 7f41d479e640 1 -- 172.21.15.43:0/3905090818 wait complete. 2024-04-25T08:54:45.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.684+0000 7f41d479e640 1 Processor -- start 2024-04-25T08:54:45.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.685+0000 7f41d479e640 1 -- start start 2024-04-25T08:54:45.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.685+0000 7f41d479e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41cc0d08d0 0x7f41cc0b8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:45.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.685+0000 7f41d479e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f41cc0d1fa0 0x7f41cc0b91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:45.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.685+0000 7f41d479e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41cc15dd60 0x7f41cc15faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:45.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.685+0000 7f41d479e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41cc0d2430 con 0x7f41cc0d08d0 2024-04-25T08:54:45.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.685+0000 7f41d479e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41cc06b7a0 con 0x7f41cc15dd60 2024-04-25T08:54:45.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.685+0000 7f41d479e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41cc0d25b0 con 0x7f41cc0d1fa0 2024-04-25T08:54:45.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.685+0000 7f41d253a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41cc0d08d0 0x7f41cc0b8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:45.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.685+0000 7f41d253a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41cc0d08d0 0x7f41cc0b8c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38322/0 (socket says 172.21.15.43:38322) 2024-04-25T08:54:45.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.685+0000 7f41d253a640 1 -- 172.21.15.43:0/1007040764 learned_addr learned my addr 172.21.15.43:0/1007040764 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:45.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.686+0000 7f41d2d3b640 1 --2- 172.21.15.43:0/1007040764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41cc15dd60 0x7f41cc15faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:45.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.686+0000 7f41d1d39640 1 --2- 172.21.15.43:0/1007040764 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f41cc0d1fa0 0x7f41cc0b91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:45.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.686+0000 7f41d253a640 1 -- 172.21.15.43:0/1007040764 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f41cc0d1fa0 msgr2=0x7f41cc0b91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:45.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.686+0000 7f41d253a640 1 --2- 172.21.15.43:0/1007040764 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f41cc0d1fa0 0x7f41cc0b91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.686+0000 7f41d253a640 1 -- 172.21.15.43:0/1007040764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41cc15dd60 msgr2=0x7f41cc15faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:45.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.686+0000 7f41d253a640 1 --2- 172.21.15.43:0/1007040764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41cc15dd60 0x7f41cc15faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.686+0000 7f41d253a640 1 -- 172.21.15.43:0/1007040764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41cc1534b0 con 0x7f41cc0d08d0 2024-04-25T08:54:45.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.686+0000 7f41d2d3b640 1 --2- 172.21.15.43:0/1007040764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41cc15dd60 0x7f41cc15faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:45.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.686+0000 7f41d1d39640 1 --2- 172.21.15.43:0/1007040764 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f41cc0d1fa0 0x7f41cc0b91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:45.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.686+0000 7f41d253a640 1 --2- 172.21.15.43:0/1007040764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41cc0d08d0 0x7f41cc0b8c80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f41c802a220 tx=0x7f41c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:45.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.686+0000 7f41c2ffd640 1 -- 172.21.15.43:0/1007040764 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c8041020 con 0x7f41cc0d08d0 2024-04-25T08:54:45.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.686+0000 7f41c2ffd640 1 -- 172.21.15.43:0/1007040764 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41c80306c0 con 0x7f41cc0d08d0 2024-04-25T08:54:45.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.686+0000 7f41d479e640 1 -- 172.21.15.43:0/1007040764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41cc160b00 con 0x7f41cc0d08d0 2024-04-25T08:54:45.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.686+0000 7f41c2ffd640 1 -- 172.21.15.43:0/1007040764 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c8037040 con 0x7f41cc0d08d0 2024-04-25T08:54:45.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.686+0000 7f41d479e640 1 -- 172.21.15.43:0/1007040764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41cc1610c0 con 0x7f41cc0d08d0 2024-04-25T08:54:45.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.687+0000 7f41d479e640 1 -- 172.21.15.43:0/1007040764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41cc074f60 con 0x7f41cc0d08d0 2024-04-25T08:54:45.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.688+0000 7f41c2ffd640 1 -- 172.21.15.43:0/1007040764 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f41c8044020 con 0x7f41cc0d08d0 2024-04-25T08:54:45.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.688+0000 7f41c2ffd640 1 --2- 172.21.15.43:0/1007040764 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f41b006b4d0 0x7f41b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:45.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.688+0000 7f41c2ffd640 1 -- 172.21.15.43:0/1007040764 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f41c80aaa10 con 0x7f41cc0d08d0 2024-04-25T08:54:45.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.688+0000 7f41d1d39640 1 --2- 172.21.15.43:0/1007040764 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f41b006b4d0 0x7f41b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:45.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.689+0000 7f41d1d39640 1 --2- 172.21.15.43:0/1007040764 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f41b006b4d0 0x7f41b006d990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f41bc006fd0 tx=0x7f41bc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:45.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.692+0000 7f41c2ffd640 1 -- 172.21.15.43:0/1007040764 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f41c804b030 con 0x7f41cc0d08d0 2024-04-25T08:54:45.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.877+0000 7f41d479e640 1 -- 172.21.15.43:0/1007040764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f41cc0cf1b0 con 0x7f41cc0d08d0 2024-04-25T08:54:45.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.877+0000 7f41c2ffd640 1 -- 172.21.15.43:0/1007040764 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f41cc1534b0 con 0x7f41cc0d08d0 2024-04-25T08:54:45.867 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T08:54:45.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.879+0000 7f41d479e640 1 -- 172.21.15.43:0/1007040764 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f41b006b4d0 msgr2=0x7f41b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:45.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.879+0000 7f41d479e640 1 --2- 172.21.15.43:0/1007040764 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f41b006b4d0 0x7f41b006d990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f41bc006fd0 tx=0x7f41bc008040 comp rx=0 tx=0).stop 2024-04-25T08:54:45.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.879+0000 7f41d479e640 1 -- 172.21.15.43:0/1007040764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41cc0d08d0 msgr2=0x7f41cc0b8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:45.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.879+0000 7f41d479e640 1 --2- 172.21.15.43:0/1007040764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41cc0d08d0 0x7f41cc0b8c80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f41c802a220 tx=0x7f41c80046c0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.879+0000 7f41d479e640 1 -- 172.21.15.43:0/1007040764 shutdown_connections 2024-04-25T08:54:45.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.879+0000 7f41d479e640 1 --2- 172.21.15.43:0/1007040764 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f41b006b4d0 0x7f41b006d990 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.879+0000 7f41d479e640 1 --2- 172.21.15.43:0/1007040764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41cc15dd60 0x7f41cc15faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.879+0000 7f41d479e640 1 --2- 172.21.15.43:0/1007040764 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f41cc0d1fa0 0x7f41cc0b91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.879+0000 7f41d479e640 1 --2- 172.21.15.43:0/1007040764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41cc0d08d0 0x7f41cc0b8c80 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.879+0000 7f41d479e640 1 -- 172.21.15.43:0/1007040764 >> 172.21.15.43:0/1007040764 conn(0x7f41cc0d39c0 msgr2=0x7f41cc0cf990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:45.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.879+0000 7f41d479e640 1 -- 172.21.15.43:0/1007040764 shutdown_connections 2024-04-25T08:54:45.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.880+0000 7f41d479e640 1 -- 172.21.15.43:0/1007040764 wait complete. 2024-04-25T08:54:45.880 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:54:45.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.995+0000 7f2004b10640 1 Processor -- start 2024-04-25T08:54:45.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.995+0000 7f2004b10640 1 -- start start 2024-04-25T08:54:45.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.995+0000 7f2004b10640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f200015ddd0 0x7f20000b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:45.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.995+0000 7f2004b10640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f200006d4f0 0x7f200006f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:45.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.995+0000 7f2004b10640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f200006d920 con 0x7f200006ef90 2024-04-25T08:54:45.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.995+0000 7f2004b10640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20000d2620 con 0x7f200015ddd0 2024-04-25T08:54:45.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.995+0000 7f2004b10640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20000b0b80 con 0x7f200006d4f0 2024-04-25T08:54:45.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.996+0000 7f1ff7fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f200006ef90 0x7f20000cc460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39676/0 (socket says 172.21.15.43:39676) 2024-04-25T08:54:45.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.996+0000 7f1ff7fff640 1 -- 172.21.15.43:0/1578118474 learned_addr learned my addr 172.21.15.43:0/1578118474 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:45.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.996+0000 7f1ffed9d640 1 --2- 172.21.15.43:0/1578118474 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f200006d4f0 0x7f200006f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:45.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.996+0000 7f1ffe59c640 1 --2- 172.21.15.43:0/1578118474 >> v2:172.21.15.193:3300/0 conn(0x7f200015ddd0 0x7f20000b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:45.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.996+0000 7f1ffdd9b640 1 -- 172.21.15.43:0/1578118474 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2265613440 0 0) 0x7f200006d920 con 0x7f200006ef90 2024-04-25T08:54:45.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.996+0000 7f1ffdd9b640 1 -- 172.21.15.43:0/1578118474 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fec003520 con 0x7f200006ef90 2024-04-25T08:54:45.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.996+0000 7f1ffdd9b640 1 -- 172.21.15.43:0/1578118474 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2285524562 0 0) 0x7f1fec003520 con 0x7f200006ef90 2024-04-25T08:54:45.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.996+0000 7f1ffdd9b640 1 -- 172.21.15.43:0/1578118474 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f200006d4f0 msgr2=0x7f200006f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:45.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.996+0000 7f1ffdd9b640 1 --2- 172.21.15.43:0/1578118474 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f200006d4f0 0x7f200006f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.996+0000 7f1ffdd9b640 1 -- 172.21.15.43:0/1578118474 >> v2:172.21.15.193:3300/0 conn(0x7f200015ddd0 msgr2=0x7f20000b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:45.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.996+0000 7f1ffdd9b640 1 --2- 172.21.15.43:0/1578118474 >> v2:172.21.15.193:3300/0 conn(0x7f200015ddd0 0x7f20000b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.996+0000 7f1ffdd9b640 1 -- 172.21.15.43:0/1578118474 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20000b0e60 con 0x7f200006ef90 2024-04-25T08:54:45.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.996+0000 7f1ffdd9b640 1 -- 172.21.15.43:0/1578118474 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f1fe8001670 con 0x7f200006ef90 2024-04-25T08:54:45.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.996+0000 7f1ffdd9b640 1 -- 172.21.15.43:0/1578118474 >> v1:172.21.15.43:6789/0 conn(0x7f200006ef90 legacy=0x7f20000cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:45.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.996+0000 7f1ffdd9b640 1 --2- 172.21.15.43:0/1578118474 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f200006d4f0 0x7f1fec0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:45.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.996+0000 7f1ffdd9b640 1 --2- 172.21.15.43:0/1578118474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fec003f30 0x7f1fec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:45.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.996+0000 7f1ffdd9b640 1 --2- 172.21.15.43:0/1578118474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1fec004d40 0x7f1fec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:45.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.996+0000 7f1ffdd9b640 1 -- 172.21.15.43:0/1578118474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20000d2620 con 0x7f1fec003f30 2024-04-25T08:54:45.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.996+0000 7f1ffdd9b640 1 -- 172.21.15.43:0/1578118474 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20000b0b80 con 0x7f1fec004d40 2024-04-25T08:54:45.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.996+0000 7f1ffdd9b640 1 -- 172.21.15.43:0/1578118474 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fec00b9c0 con 0x7f200006d4f0 2024-04-25T08:54:45.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.996+0000 7f1ff7fff640 1 --2- 172.21.15.43:0/1578118474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fec003f30 0x7f1fec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:45.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.996+0000 7f1ffe59c640 1 --2- 172.21.15.43:0/1578118474 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f200006d4f0 0x7f1fec0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:45.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.996+0000 7f1ffed9d640 1 --2- 172.21.15.43:0/1578118474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1fec004d40 0x7f1fec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:45.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.997+0000 7f1ff7fff640 1 -- 172.21.15.43:0/1578118474 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f200006d4f0 msgr2=0x7f1fec0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:45.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.997+0000 7f1ff7fff640 1 --2- 172.21.15.43:0/1578118474 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f200006d4f0 0x7f1fec0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.997+0000 7f1ff7fff640 1 -- 172.21.15.43:0/1578118474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1fec004d40 msgr2=0x7f1fec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:45.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.997+0000 7f1ff7fff640 1 --2- 172.21.15.43:0/1578118474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1fec004d40 0x7f1fec0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.997+0000 7f1ff7fff640 1 -- 172.21.15.43:0/1578118474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fec003520 con 0x7f1fec003f30 2024-04-25T08:54:45.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.997+0000 7f1ff7fff640 1 --2- 172.21.15.43:0/1578118474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fec003f30 0x7f1fec004800 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1fe800c210 tx=0x7f1fec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2fc5e22e122ea7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:45.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.997+0000 7f1ffdd9b640 1 -- 172.21.15.43:0/1578118474 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe8007d50 con 0x7f1fec003f30 2024-04-25T08:54:45.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.997+0000 7f1ffdd9b640 1 -- 172.21.15.43:0/1578118474 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fe802e070 con 0x7f1fec003f30 2024-04-25T08:54:45.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.997+0000 7f1ffdd9b640 1 -- 172.21.15.43:0/1578118474 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fe8005d70 con 0x7f1fec003f30 2024-04-25T08:54:45.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.998+0000 7f2004b10640 1 -- 172.21.15.43:0/1578118474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fec003f30 msgr2=0x7f1fec004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:45.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.998+0000 7f2004b10640 1 --2- 172.21.15.43:0/1578118474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fec003f30 0x7f1fec004800 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1fe800c210 tx=0x7f1fec005830 comp rx=0 tx=0).stop 2024-04-25T08:54:45.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.998+0000 7f1ffed9d640 1 -- 172.21.15.43:0/1578118474 reap_dead start 2024-04-25T08:54:45.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.998+0000 7f2004b10640 1 -- 172.21.15.43:0/1578118474 shutdown_connections 2024-04-25T08:54:45.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.998+0000 7f2004b10640 1 -- 172.21.15.43:0/1578118474 >> 172.21.15.43:0/1578118474 conn(0x7f20000d17e0 msgr2=0x7f20000d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:45.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.998+0000 7f2004b10640 1 -- 172.21.15.43:0/1578118474 shutdown_connections 2024-04-25T08:54:45.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.998+0000 7f2004b10640 1 -- 172.21.15.43:0/1578118474 wait complete. 2024-04-25T08:54:45.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:45.999+0000 7f2004b10640 1 Processor -- start 2024-04-25T08:54:45.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.000+0000 7f2004b10640 1 -- start start 2024-04-25T08:54:45.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.000+0000 7f2004b10640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1fec003f30 0x7f20000b4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:45.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.000+0000 7f2004b10640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20000b20d0 0x7f200015a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:45.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.000+0000 7f2004b10640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20000b6900 0x7f20000b6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:45.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.000+0000 7f2004b10640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200006f5a0 con 0x7f20000b6900 2024-04-25T08:54:45.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.000+0000 7f2004b10640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200006f420 con 0x7f1fec003f30 2024-04-25T08:54:45.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.000+0000 7f2004b10640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f200006f720 con 0x7f20000b20d0 2024-04-25T08:54:45.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.000+0000 7f1ffed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20000b6900 0x7f20000b6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:45.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.000+0000 7f1ffed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20000b6900 0x7f20000b6d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38330/0 (socket says 172.21.15.43:38330) 2024-04-25T08:54:45.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.000+0000 7f1ffed9d640 1 -- 172.21.15.43:0/972060504 learned_addr learned my addr 172.21.15.43:0/972060504 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:45.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.001+0000 7f1ff7fff640 1 --2- 172.21.15.43:0/972060504 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20000b20d0 0x7f200015a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:45.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.001+0000 7f1ffe59c640 1 --2- 172.21.15.43:0/972060504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1fec003f30 0x7f20000b4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:45.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.001+0000 7f1ffed9d640 1 -- 172.21.15.43:0/972060504 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20000b20d0 msgr2=0x7f200015a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:45.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.001+0000 7f1ffed9d640 1 --2- 172.21.15.43:0/972060504 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20000b20d0 0x7f200015a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.001+0000 7f1ffed9d640 1 -- 172.21.15.43:0/972060504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1fec003f30 msgr2=0x7f20000b4fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:45.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.001+0000 7f1ffed9d640 1 --2- 172.21.15.43:0/972060504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1fec003f30 0x7f20000b4fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:45.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.001+0000 7f1ffed9d640 1 -- 172.21.15.43:0/972060504 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20000b2920 con 0x7f20000b6900 2024-04-25T08:54:45.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.001+0000 7f1ffe59c640 1 --2- 172.21.15.43:0/972060504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1fec003f30 0x7f20000b4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:45.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.001+0000 7f1ffed9d640 1 --2- 172.21.15.43:0/972060504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20000b6900 0x7f20000b6d80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2000161710 tx=0x7f1ff800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:45.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.001+0000 7f1ff6ffd640 1 -- 172.21.15.43:0/972060504 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff800fde0 con 0x7f20000b6900 2024-04-25T08:54:45.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.001+0000 7f1ff6ffd640 1 -- 172.21.15.43:0/972060504 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff8015430 con 0x7f20000b6900 2024-04-25T08:54:45.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.001+0000 7f2004b10640 1 -- 172.21.15.43:0/972060504 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20000b2c00 con 0x7f20000b6900 2024-04-25T08:54:45.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.001+0000 7f1ff6ffd640 1 -- 172.21.15.43:0/972060504 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff8015820 con 0x7f20000b6900 2024-04-25T08:54:45.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.001+0000 7f2004b10640 1 -- 172.21.15.43:0/972060504 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20000b31c0 con 0x7f20000b6900 2024-04-25T08:54:45.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.002+0000 7f2004b10640 1 -- 172.21.15.43:0/972060504 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2000074f60 con 0x7f20000b6900 2024-04-25T08:54:45.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.003+0000 7f1ff6ffd640 1 -- 172.21.15.43:0/972060504 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1ff80159c0 con 0x7f20000b6900 2024-04-25T08:54:45.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.003+0000 7f1ff6ffd640 1 --2- 172.21.15.43:0/972060504 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1fe006b4d0 0x7f1fe006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:45.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.003+0000 7f1ff6ffd640 1 -- 172.21.15.43:0/972060504 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ff808af60 con 0x7f20000b6900 2024-04-25T08:54:45.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.003+0000 7f1ffe59c640 1 --2- 172.21.15.43:0/972060504 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1fe006b4d0 0x7f1fe006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:45.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.004+0000 7f1ffe59c640 1 --2- 172.21.15.43:0/972060504 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1fe006b4d0 0x7f1fe006d990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f20000b3d50 tx=0x7f1ff0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:45.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.007+0000 7f1ff6ffd640 1 -- 172.21.15.43:0/972060504 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1ff8019030 con 0x7f20000b6900 2024-04-25T08:54:46.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.189+0000 7f2004b10640 1 -- 172.21.15.43:0/972060504 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f200015d4d0 con 0x7f20000b6900 2024-04-25T08:54:46.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.189+0000 7f1ff6ffd640 1 -- 172.21.15.43:0/972060504 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1ff8010040 con 0x7f20000b6900 2024-04-25T08:54:46.179 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T08:54:46.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.191+0000 7f2004b10640 1 -- 172.21.15.43:0/972060504 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1fe006b4d0 msgr2=0x7f1fe006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:46.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.191+0000 7f2004b10640 1 --2- 172.21.15.43:0/972060504 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1fe006b4d0 0x7f1fe006d990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f20000b3d50 tx=0x7f1ff0008040 comp rx=0 tx=0).stop 2024-04-25T08:54:46.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.191+0000 7f2004b10640 1 -- 172.21.15.43:0/972060504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20000b6900 msgr2=0x7f20000b6d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:46.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.191+0000 7f2004b10640 1 --2- 172.21.15.43:0/972060504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20000b6900 0x7f20000b6d80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2000161710 tx=0x7f1ff800f580 comp rx=0 tx=0).stop 2024-04-25T08:54:46.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.191+0000 7f2004b10640 1 -- 172.21.15.43:0/972060504 shutdown_connections 2024-04-25T08:54:46.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.191+0000 7f2004b10640 1 --2- 172.21.15.43:0/972060504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20000b6900 0x7f20000b6d80 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:46.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.191+0000 7f2004b10640 1 --2- 172.21.15.43:0/972060504 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20000b20d0 0x7f200015a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:46.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.191+0000 7f2004b10640 1 --2- 172.21.15.43:0/972060504 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1fe006b4d0 0x7f1fe006d990 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:46.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.191+0000 7f2004b10640 1 --2- 172.21.15.43:0/972060504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1fec003f30 0x7f20000b4fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:46.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.191+0000 7f2004b10640 1 -- 172.21.15.43:0/972060504 >> 172.21.15.43:0/972060504 conn(0x7f20000d17e0 msgr2=0x7f20000ae710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:46.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.191+0000 7f2004b10640 1 -- 172.21.15.43:0/972060504 shutdown_connections 2024-04-25T08:54:46.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.191+0000 7f2004b10640 1 -- 172.21.15.43:0/972060504 wait complete. 2024-04-25T08:54:46.192 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:54:46.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.306+0000 7fb9d19c8640 1 Processor -- start 2024-04-25T08:54:46.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.306+0000 7fb9d19c8640 1 -- start start 2024-04-25T08:54:46.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.307+0000 7fb9d19c8640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9cc15e020 0x7fb9cc162550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:46.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.307+0000 7fb9d19c8640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb9cc0ce7e0 0x7fb9cc0d0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:46.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.307+0000 7fb9d19c8640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9cc0b47c0 con 0x7fb9cc15dc40 2024-04-25T08:54:46.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.307+0000 7fb9d19c8640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9cc071e80 con 0x7fb9cc0ce7e0 2024-04-25T08:54:46.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.307+0000 7fb9d19c8640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc072160 con 0x7fb9cc15e020 2024-04-25T08:54:46.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.307+0000 7fb9caffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb9cc15dc40 0x7fb9cc071670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39692/0 (socket says 172.21.15.43:39692) 2024-04-25T08:54:46.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.307+0000 7fb9caffd640 1 -- 172.21.15.43:0/3211589516 learned_addr learned my addr 172.21.15.43:0/3211589516 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:46.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.307+0000 7fb9ca7fc640 1 --2- 172.21.15.43:0/3211589516 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9cc15e020 0x7fb9cc162550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:46.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.308+0000 7fb9cb7fe640 1 --2- 172.21.15.43:0/3211589516 >> v2:172.21.15.193:3300/0 conn(0x7fb9cc0ce7e0 0x7fb9cc0d0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:46.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.308+0000 7fb9c9ffb640 1 -- 172.21.15.43:0/3211589516 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1715998346 0 0) 0x7fb9cc0b47c0 con 0x7fb9cc15dc40 2024-04-25T08:54:46.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.308+0000 7fb9c9ffb640 1 -- 172.21.15.43:0/3211589516 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9b4003520 con 0x7fb9cc15dc40 2024-04-25T08:54:46.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.308+0000 7fb9c9ffb640 1 -- 172.21.15.43:0/3211589516 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 739912039 0 0) 0x7fb9b4003520 con 0x7fb9cc15dc40 2024-04-25T08:54:46.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.308+0000 7fb9ca7fc640 1 -- 172.21.15.43:0/3211589516 >> v2:172.21.15.193:3300/0 conn(0x7fb9cc0ce7e0 msgr2=0x7fb9cc0d0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:46.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.308+0000 7fb9ca7fc640 1 --2- 172.21.15.43:0/3211589516 >> v2:172.21.15.193:3300/0 conn(0x7fb9cc0ce7e0 0x7fb9cc0d0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:46.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.308+0000 7fb9ca7fc640 1 -- 172.21.15.43:0/3211589516 >> v1:172.21.15.43:6789/0 conn(0x7fb9cc15dc40 legacy=0x7fb9cc071670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:46.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.308+0000 7fb9ca7fc640 1 -- 172.21.15.43:0/3211589516 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9cc072440 con 0x7fb9cc15e020 2024-04-25T08:54:46.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.309+0000 7fb9ca7fc640 1 --2- 172.21.15.43:0/3211589516 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9cc15e020 0x7fb9cc162550 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb9c00097a0 tx=0x7fb9c002a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6799c702a34b80bc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:46.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.309+0000 7fb9c9ffb640 1 -- 172.21.15.43:0/3211589516 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0037070 con 0x7fb9cc15e020 2024-04-25T08:54:46.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.309+0000 7fb9c9ffb640 1 -- 172.21.15.43:0/3211589516 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c002aba0 con 0x7fb9cc15e020 2024-04-25T08:54:46.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.309+0000 7fb9c9ffb640 1 -- 172.21.15.43:0/3211589516 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc0019f0 con 0x7fb9cc15e020 2024-04-25T08:54:46.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.309+0000 7fb9d19c8640 1 -- 172.21.15.43:0/3211589516 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9cc15e020 msgr2=0x7fb9cc162550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:46.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.309+0000 7fb9d19c8640 1 --2- 172.21.15.43:0/3211589516 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9cc15e020 0x7fb9cc162550 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb9c00097a0 tx=0x7fb9c002a110 comp rx=0 tx=0).stop 2024-04-25T08:54:46.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.309+0000 7fb9d19c8640 1 -- 172.21.15.43:0/3211589516 shutdown_connections 2024-04-25T08:54:46.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.309+0000 7fb9d19c8640 1 --2- 172.21.15.43:0/3211589516 >> v2:172.21.15.193:3300/0 conn(0x7fb9cc0ce7e0 0x7fb9cc0d0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:46.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.309+0000 7fb9d19c8640 1 --2- 172.21.15.43:0/3211589516 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9cc15e020 0x7fb9cc162550 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:46.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.309+0000 7fb9d19c8640 1 -- 172.21.15.43:0/3211589516 >> 172.21.15.43:0/3211589516 conn(0x7fb9cc06c060 msgr2=0x7fb9cc06c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:46.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.309+0000 7fb9d19c8640 1 -- 172.21.15.43:0/3211589516 shutdown_connections 2024-04-25T08:54:46.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.310+0000 7fb9d19c8640 1 -- 172.21.15.43:0/3211589516 wait complete. 2024-04-25T08:54:46.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.311+0000 7fb9d19c8640 1 Processor -- start 2024-04-25T08:54:46.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.311+0000 7fb9d19c8640 1 -- start start 2024-04-25T08:54:46.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.311+0000 7fb9d19c8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9cc0ce7e0 0x7fb9cc1557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:46.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.311+0000 7fb9d19c8640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9cc15dc40 0x7fb9cc159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:46.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.312+0000 7fb9d19c8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9cc15e020 0x7fb9cc15f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:46.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.312+0000 7fb9d19c8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc0d1030 con 0x7fb9cc15e020 2024-04-25T08:54:46.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.312+0000 7fb9d19c8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc0cec90 con 0x7fb9cc0ce7e0 2024-04-25T08:54:46.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.312+0000 7fb9d19c8640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc0d11b0 con 0x7fb9cc15dc40 2024-04-25T08:54:46.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.312+0000 7fb9cb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9cc15e020 0x7fb9cc15f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:46.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.312+0000 7fb9cb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9cc15e020 0x7fb9cc15f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38332/0 (socket says 172.21.15.43:38332) 2024-04-25T08:54:46.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.312+0000 7fb9cb7fe640 1 -- 172.21.15.43:0/991576713 learned_addr learned my addr 172.21.15.43:0/991576713 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:46.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.312+0000 7fb9ca7fc640 1 --2- 172.21.15.43:0/991576713 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9cc15dc40 0x7fb9cc159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:46.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.312+0000 7fb9caffd640 1 --2- 172.21.15.43:0/991576713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9cc0ce7e0 0x7fb9cc1557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:46.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.312+0000 7fb9cb7fe640 1 -- 172.21.15.43:0/991576713 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9cc15dc40 msgr2=0x7fb9cc159390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:46.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.312+0000 7fb9cb7fe640 1 --2- 172.21.15.43:0/991576713 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9cc15dc40 0x7fb9cc159390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:46.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.312+0000 7fb9cb7fe640 1 -- 172.21.15.43:0/991576713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9cc0ce7e0 msgr2=0x7fb9cc1557e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:46.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.312+0000 7fb9cb7fe640 1 --2- 172.21.15.43:0/991576713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9cc0ce7e0 0x7fb9cc1557e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:46.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.312+0000 7fb9cb7fe640 1 -- 172.21.15.43:0/991576713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9cc158220 con 0x7fb9cc15e020 2024-04-25T08:54:46.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.312+0000 7fb9cb7fe640 1 --2- 172.21.15.43:0/991576713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9cc15e020 0x7fb9cc15f820 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb9c400e960 tx=0x7fb9c400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:46.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.312+0000 7fb9b3fff640 1 -- 172.21.15.43:0/991576713 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c4017070 con 0x7fb9cc15e020 2024-04-25T08:54:46.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.312+0000 7fb9b3fff640 1 -- 172.21.15.43:0/991576713 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c400c980 con 0x7fb9cc15e020 2024-04-25T08:54:46.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.312+0000 7fb9d19c8640 1 -- 172.21.15.43:0/991576713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9cc158500 con 0x7fb9cc15e020 2024-04-25T08:54:46.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.312+0000 7fb9b3fff640 1 -- 172.21.15.43:0/991576713 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c400cd70 con 0x7fb9cc15e020 2024-04-25T08:54:46.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.312+0000 7fb9d19c8640 1 -- 172.21.15.43:0/991576713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9cc158ac0 con 0x7fb9cc15e020 2024-04-25T08:54:46.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.313+0000 7fb9d19c8640 1 -- 172.21.15.43:0/991576713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9cc074f60 con 0x7fb9cc15e020 2024-04-25T08:54:46.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.314+0000 7fb9b3fff640 1 -- 172.21.15.43:0/991576713 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb9c401a070 con 0x7fb9cc15e020 2024-04-25T08:54:46.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.314+0000 7fb9b3fff640 1 --2- 172.21.15.43:0/991576713 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb9b806b4d0 0x7fb9b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:46.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.314+0000 7fb9b3fff640 1 -- 172.21.15.43:0/991576713 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb9c408a520 con 0x7fb9cc15e020 2024-04-25T08:54:46.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.315+0000 7fb9caffd640 1 --2- 172.21.15.43:0/991576713 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb9b806b4d0 0x7fb9b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:46.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.315+0000 7fb9caffd640 1 --2- 172.21.15.43:0/991576713 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb9b806b4d0 0x7fb9b806d990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc005c00 tx=0x7fb9bc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:46.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.318+0000 7fb9b3fff640 1 -- 172.21.15.43:0/991576713 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb9c4058d50 con 0x7fb9cc15e020 2024-04-25T08:54:46.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.499+0000 7fb9d19c8640 1 -- 172.21.15.43:0/991576713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb9cc0ce0a0 con 0x7fb9cc15e020 2024-04-25T08:54:46.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.500+0000 7fb9b3fff640 1 -- 172.21.15.43:0/991576713 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb9c405cdf0 con 0x7fb9cc15e020 2024-04-25T08:54:46.490 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T08:54:46.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.502+0000 7fb9d19c8640 1 -- 172.21.15.43:0/991576713 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb9b806b4d0 msgr2=0x7fb9b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:46.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.502+0000 7fb9d19c8640 1 --2- 172.21.15.43:0/991576713 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb9b806b4d0 0x7fb9b806d990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc005c00 tx=0x7fb9bc009040 comp rx=0 tx=0).stop 2024-04-25T08:54:46.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.502+0000 7fb9d19c8640 1 -- 172.21.15.43:0/991576713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9cc15e020 msgr2=0x7fb9cc15f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:46.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.502+0000 7fb9d19c8640 1 --2- 172.21.15.43:0/991576713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9cc15e020 0x7fb9cc15f820 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fb9c400e960 tx=0x7fb9c400ee20 comp rx=0 tx=0).stop 2024-04-25T08:54:46.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.502+0000 7fb9d19c8640 1 -- 172.21.15.43:0/991576713 shutdown_connections 2024-04-25T08:54:46.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.502+0000 7fb9d19c8640 1 --2- 172.21.15.43:0/991576713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9cc15e020 0x7fb9cc15f820 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:46.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.502+0000 7fb9d19c8640 1 --2- 172.21.15.43:0/991576713 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9cc15dc40 0x7fb9cc159390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:46.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.502+0000 7fb9d19c8640 1 --2- 172.21.15.43:0/991576713 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb9b806b4d0 0x7fb9b806d990 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:46.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.502+0000 7fb9d19c8640 1 --2- 172.21.15.43:0/991576713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9cc0ce7e0 0x7fb9cc1557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:46.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.502+0000 7fb9d19c8640 1 -- 172.21.15.43:0/991576713 >> 172.21.15.43:0/991576713 conn(0x7fb9cc06c060 msgr2=0x7fb9cc0b2360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:46.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.502+0000 7fb9d19c8640 1 -- 172.21.15.43:0/991576713 shutdown_connections 2024-04-25T08:54:46.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.502+0000 7fb9d19c8640 1 -- 172.21.15.43:0/991576713 wait complete. 2024-04-25T08:54:46.503 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:54:46.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.615+0000 7f3e2ad0a640 1 Processor -- start 2024-04-25T08:54:46.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.615+0000 7f3e2ad0a640 1 -- start start 2024-04-25T08:54:46.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.616+0000 7f3e2ad0a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3e24161390 0x7f3e240b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:46.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.616+0000 7f3e2ad0a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3e240d02a0 0x7f3e240d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:46.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.616+0000 7f3e2ad0a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e240b4c30 con 0x7f3e240ce5e0 2024-04-25T08:54:46.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.616+0000 7f3e2ad0a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e240d1910 con 0x7f3e24161390 2024-04-25T08:54:46.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.616+0000 7f3e2ad0a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e240cf2c0 con 0x7f3e240d02a0 2024-04-25T08:54:46.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.616+0000 7f3e28aa6640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3e24161390 0x7f3e240b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:46.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.616+0000 7f3e292a7640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f3e240ce5e0 0x7f3e240cea70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39708/0 (socket says 172.21.15.43:39708) 2024-04-25T08:54:46.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.616+0000 7f3e292a7640 1 -- 172.21.15.43:0/2891877969 learned_addr learned my addr 172.21.15.43:0/2891877969 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:46.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.616+0000 7f3e28aa6640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3e24161390 0x7f3e240b6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38340/0 (socket says 172.21.15.43:38340) 2024-04-25T08:54:46.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.616+0000 7f3e23fff640 1 --2- 172.21.15.43:0/2891877969 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3e240d02a0 0x7f3e240d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:46.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.616+0000 7f3e28aa6640 1 -- 172.21.15.43:0/2891877969 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3e240d02a0 msgr2=0x7f3e240d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:46.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.616+0000 7f3e28aa6640 1 --2- 172.21.15.43:0/2891877969 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3e240d02a0 0x7f3e240d0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:46.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.616+0000 7f3e28aa6640 1 -- 172.21.15.43:0/2891877969 >> v1:172.21.15.43:6789/0 conn(0x7f3e240ce5e0 legacy=0x7f3e240cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:46.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.616+0000 7f3e28aa6640 1 -- 172.21.15.43:0/2891877969 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e240cf5a0 con 0x7f3e24161390 2024-04-25T08:54:46.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.616+0000 7f3e237fe640 1 -- 172.21.15.43:0/2891877969 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 35755234 0 0) 0x7f3e1c001490 con 0x7f3e240ce5e0 2024-04-25T08:54:46.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.616+0000 7f3e23fff640 1 --2- 172.21.15.43:0/2891877969 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3e240d02a0 0x7f3e240d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:46.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.617+0000 7f3e28aa6640 1 --2- 172.21.15.43:0/2891877969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e24161390 0x7f3e240b6820 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3e180097a0 tx=0x7f3e1802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b8f3bcda303cc99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:46.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.617+0000 7f3e237fe640 1 -- 172.21.15.43:0/2891877969 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e18037070 con 0x7f3e24161390 2024-04-25T08:54:46.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.617+0000 7f3e237fe640 1 -- 172.21.15.43:0/2891877969 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e18032440 con 0x7f3e24161390 2024-04-25T08:54:46.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.617+0000 7f3e237fe640 1 -- 172.21.15.43:0/2891877969 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e18032810 con 0x7f3e24161390 2024-04-25T08:54:46.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.618+0000 7f3e2ad0a640 1 -- 172.21.15.43:0/2891877969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e24161390 msgr2=0x7f3e240b6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:46.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.618+0000 7f3e2ad0a640 1 --2- 172.21.15.43:0/2891877969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e24161390 0x7f3e240b6820 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3e180097a0 tx=0x7f3e1802a180 comp rx=0 tx=0).stop 2024-04-25T08:54:46.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.618+0000 7f3e2ad0a640 1 -- 172.21.15.43:0/2891877969 shutdown_connections 2024-04-25T08:54:46.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.618+0000 7f3e2ad0a640 1 --2- 172.21.15.43:0/2891877969 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3e240d02a0 0x7f3e240d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:46.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.618+0000 7f3e2ad0a640 1 --2- 172.21.15.43:0/2891877969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e24161390 0x7f3e240b6820 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:46.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.618+0000 7f3e2ad0a640 1 -- 172.21.15.43:0/2891877969 >> 172.21.15.43:0/2891877969 conn(0x7f3e2409cd20 msgr2=0x7f3e240d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:46.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.618+0000 7f3e2ad0a640 1 -- 172.21.15.43:0/2891877969 shutdown_connections 2024-04-25T08:54:46.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.618+0000 7f3e2ad0a640 1 -- 172.21.15.43:0/2891877969 wait complete. 2024-04-25T08:54:46.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.619+0000 7f3e2ad0a640 1 Processor -- start 2024-04-25T08:54:46.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.619+0000 7f3e2ad0a640 1 -- start start 2024-04-25T08:54:46.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.620+0000 7f3e2ad0a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e240ce5e0 0x7f3e2415d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:46.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.620+0000 7f3e2ad0a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3e240d02a0 0x7f3e240cbb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:46.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.620+0000 7f3e2ad0a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e24161390 0x7f3e241f36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:46.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.620+0000 7f3e2ad0a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e24161770 con 0x7f3e24161390 2024-04-25T08:54:46.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.620+0000 7f3e2ad0a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e240d15a0 con 0x7f3e240ce5e0 2024-04-25T08:54:46.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.620+0000 7f3e2ad0a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e240d1da0 con 0x7f3e240d02a0 2024-04-25T08:54:46.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.620+0000 7f3e292a7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e24161390 0x7f3e241f36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:46.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.620+0000 7f3e292a7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e24161390 0x7f3e241f36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38356/0 (socket says 172.21.15.43:38356) 2024-04-25T08:54:46.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.620+0000 7f3e292a7640 1 -- 172.21.15.43:0/4254876741 learned_addr learned my addr 172.21.15.43:0/4254876741 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:46.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.620+0000 7f3e28aa6640 1 --2- 172.21.15.43:0/4254876741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e240ce5e0 0x7f3e2415d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:46.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.620+0000 7f3e23fff640 1 --2- 172.21.15.43:0/4254876741 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3e240d02a0 0x7f3e240cbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:46.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.620+0000 7f3e292a7640 1 -- 172.21.15.43:0/4254876741 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3e240d02a0 msgr2=0x7f3e240cbb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:46.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.620+0000 7f3e292a7640 1 --2- 172.21.15.43:0/4254876741 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3e240d02a0 0x7f3e240cbb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:46.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.620+0000 7f3e292a7640 1 -- 172.21.15.43:0/4254876741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e240ce5e0 msgr2=0x7f3e2415d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:46.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.620+0000 7f3e292a7640 1 --2- 172.21.15.43:0/4254876741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e240ce5e0 0x7f3e2415d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:46.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.620+0000 7f3e292a7640 1 -- 172.21.15.43:0/4254876741 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e240cc0d0 con 0x7f3e24161390 2024-04-25T08:54:46.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.620+0000 7f3e292a7640 1 --2- 172.21.15.43:0/4254876741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e24161390 0x7f3e241f36d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c00c820 tx=0x7f3e1c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:46.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.621+0000 7f3e217fa640 1 -- 172.21.15.43:0/4254876741 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c018070 con 0x7f3e24161390 2024-04-25T08:54:46.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.621+0000 7f3e217fa640 1 -- 172.21.15.43:0/4254876741 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e1c00a5e0 con 0x7f3e24161390 2024-04-25T08:54:46.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.621+0000 7f3e217fa640 1 -- 172.21.15.43:0/4254876741 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1c00a9d0 con 0x7f3e24161390 2024-04-25T08:54:46.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.621+0000 7f3e2ad0a640 1 -- 172.21.15.43:0/4254876741 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e240cc3b0 con 0x7f3e24161390 2024-04-25T08:54:46.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.621+0000 7f3e2ad0a640 1 -- 172.21.15.43:0/4254876741 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e240ccbf0 con 0x7f3e24161390 2024-04-25T08:54:46.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.622+0000 7f3e2ad0a640 1 -- 172.21.15.43:0/4254876741 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e240680f0 con 0x7f3e24161390 2024-04-25T08:54:46.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.622+0000 7f3e217fa640 1 -- 172.21.15.43:0/4254876741 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3e1c01b070 con 0x7f3e24161390 2024-04-25T08:54:46.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.622+0000 7f3e217fa640 1 --2- 172.21.15.43:0/4254876741 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3e0806b4d0 0x7f3e0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:46.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.623+0000 7f3e217fa640 1 -- 172.21.15.43:0/4254876741 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e1c089830 con 0x7f3e24161390 2024-04-25T08:54:46.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.623+0000 7f3e28aa6640 1 --2- 172.21.15.43:0/4254876741 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3e0806b4d0 0x7f3e0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:46.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.623+0000 7f3e28aa6640 1 --2- 172.21.15.43:0/4254876741 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3e0806b4d0 0x7f3e0806d990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3e18034000 tx=0x7f3e1802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:46.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.627+0000 7f3e217fa640 1 -- 172.21.15.43:0/4254876741 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3e240cc3b0 con 0x7f3e24161390 2024-04-25T08:54:46.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.823+0000 7f3e2ad0a640 1 -- 172.21.15.43:0/4254876741 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3e24074ef0 con 0x7f3e24161390 2024-04-25T08:54:46.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.823+0000 7f3e217fa640 1 -- 172.21.15.43:0/4254876741 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3e240cc0d0 con 0x7f3e24161390 2024-04-25T08:54:46.813 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-04-25T08:54:46.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.825+0000 7f3e2ad0a640 1 -- 172.21.15.43:0/4254876741 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3e0806b4d0 msgr2=0x7f3e0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:46.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.825+0000 7f3e2ad0a640 1 --2- 172.21.15.43:0/4254876741 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3e0806b4d0 0x7f3e0806d990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3e18034000 tx=0x7f3e1802e000 comp rx=0 tx=0).stop 2024-04-25T08:54:46.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.825+0000 7f3e2ad0a640 1 -- 172.21.15.43:0/4254876741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e24161390 msgr2=0x7f3e241f36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:46.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.825+0000 7f3e2ad0a640 1 --2- 172.21.15.43:0/4254876741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e24161390 0x7f3e241f36d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c00c820 tx=0x7f3e1c00cce0 comp rx=0 tx=0).stop 2024-04-25T08:54:46.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.825+0000 7f3e2ad0a640 1 -- 172.21.15.43:0/4254876741 shutdown_connections 2024-04-25T08:54:46.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.825+0000 7f3e2ad0a640 1 --2- 172.21.15.43:0/4254876741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3e24161390 0x7f3e241f36d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:46.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.825+0000 7f3e2ad0a640 1 --2- 172.21.15.43:0/4254876741 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3e240d02a0 0x7f3e240cbb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:46.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.825+0000 7f3e2ad0a640 1 --2- 172.21.15.43:0/4254876741 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3e0806b4d0 0x7f3e0806d990 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:46.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.825+0000 7f3e2ad0a640 1 --2- 172.21.15.43:0/4254876741 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3e240ce5e0 0x7f3e2415d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:46.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.825+0000 7f3e2ad0a640 1 -- 172.21.15.43:0/4254876741 >> 172.21.15.43:0/4254876741 conn(0x7f3e2409cd20 msgr2=0x7f3e24148960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:46.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.825+0000 7f3e2ad0a640 1 -- 172.21.15.43:0/4254876741 shutdown_connections 2024-04-25T08:54:46.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:46.825+0000 7f3e2ad0a640 1 -- 172.21.15.43:0/4254876741 wait complete. 2024-04-25T08:54:46.826 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi193.front.sepia.ceph.com... 2024-04-25T08:54:46.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:46.827 DEBUG:teuthology.orchestra.run.smithi193:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-25T08:54:46.856 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:54:46.856 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi193.front.sepia.ceph.com 2024-04-25T08:54:46.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:46.856 DEBUG:teuthology.orchestra.run.smithi193:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T08:54:46.926 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:54:46.927 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:54:46.942 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:54:46.943 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:54:46.998 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:54:47.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.149+0000 7fcc712cb640 1 Processor -- start 2024-04-25T08:54:47.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.150+0000 7fcc712cb640 1 -- start start 2024-04-25T08:54:47.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.150+0000 7fcc712cb640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fcc6c15dcf0 0x7fcc6c06e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:47.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.150+0000 7fcc712cb640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc6c162500 con 0x7fcc6c06ebf0 2024-04-25T08:54:47.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.150+0000 7fcc712cb640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc6c0d0d80 con 0x7fcc6c06d4f0 2024-04-25T08:54:47.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.150+0000 7fcc712cb640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc6c06c050 con 0x7fcc6c15dcf0 2024-04-25T08:54:47.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.151+0000 7fcc6a7fc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fcc6c06ebf0 0x7fcc6c1616e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39720/0 (socket says 172.21.15.43:39720) 2024-04-25T08:54:47.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.151+0000 7fcc6a7fc640 1 -- 172.21.15.43:0/1998778370 learned_addr learned my addr 172.21.15.43:0/1998778370 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:47.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.151+0000 7fcc6affd640 1 --2- 172.21.15.43:0/1998778370 >> v2:172.21.15.193:3300/0 conn(0x7fcc6c15dcf0 0x7fcc6c06e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:47.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.151+0000 7fcc69ffb640 1 -- 172.21.15.43:0/1998778370 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2480735156 0 0) 0x7fcc6c162500 con 0x7fcc6c06ebf0 2024-04-25T08:54:47.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.151+0000 7fcc69ffb640 1 -- 172.21.15.43:0/1998778370 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc54003520 con 0x7fcc6c06ebf0 2024-04-25T08:54:47.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.151+0000 7fcc69ffb640 1 -- 172.21.15.43:0/1998778370 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3482704448 0 0) 0x7fcc54003520 con 0x7fcc6c06ebf0 2024-04-25T08:54:47.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.151+0000 7fcc69ffb640 1 -- 172.21.15.43:0/1998778370 >> v2:172.21.15.193:3300/0 conn(0x7fcc6c15dcf0 msgr2=0x7fcc6c06e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:47.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.151+0000 7fcc69ffb640 1 --2- 172.21.15.43:0/1998778370 >> v2:172.21.15.193:3300/0 conn(0x7fcc6c15dcf0 0x7fcc6c06e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.151+0000 7fcc69ffb640 1 -- 172.21.15.43:0/1998778370 >> v1:172.21.15.193:6789/0 conn(0x7fcc6c06d4f0 legacy=0x7fcc6c161df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:47.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.151+0000 7fcc69ffb640 1 -- 172.21.15.43:0/1998778370 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc6c0d04e0 con 0x7fcc6c06ebf0 2024-04-25T08:54:47.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.151+0000 7fcc69ffb640 1 -- 172.21.15.43:0/1998778370 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7fcc60001670 con 0x7fcc6c06ebf0 2024-04-25T08:54:47.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.151+0000 7fcc69ffb640 1 -- 172.21.15.43:0/1998778370 >> v1:172.21.15.43:6789/0 conn(0x7fcc6c06ebf0 legacy=0x7fcc6c1616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:47.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.151+0000 7fcc69ffb640 1 --2- 172.21.15.43:0/1998778370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc540039b0 0x7fcc54003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:47.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.151+0000 7fcc69ffb640 1 --2- 172.21.15.43:0/1998778370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc54004800 0x7fcc54004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:47.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.151+0000 7fcc69ffb640 1 --2- 172.21.15.43:0/1998778370 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc540051c0 0x7fcc5400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:47.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.151+0000 7fcc69ffb640 1 -- 172.21.15.43:0/1998778370 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6c06c050 con 0x7fcc540039b0 2024-04-25T08:54:47.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.151+0000 7fcc69ffb640 1 -- 172.21.15.43:0/1998778370 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5400c0c0 con 0x7fcc54004800 2024-04-25T08:54:47.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.151+0000 7fcc69ffb640 1 -- 172.21.15.43:0/1998778370 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5400c2c0 con 0x7fcc540051c0 2024-04-25T08:54:47.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.151+0000 7fcc6affd640 1 --2- 172.21.15.43:0/1998778370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc540039b0 0x7fcc54003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:47.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.151+0000 7fcc6a7fc640 1 --2- 172.21.15.43:0/1998778370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc54004800 0x7fcc54004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:47.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.151+0000 7fcc6b7fe640 1 --2- 172.21.15.43:0/1998778370 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc540051c0 0x7fcc5400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:47.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.152+0000 7fcc6affd640 1 -- 172.21.15.43:0/1998778370 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc540051c0 msgr2=0x7fcc5400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:47.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.152+0000 7fcc6affd640 1 --2- 172.21.15.43:0/1998778370 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc540051c0 0x7fcc5400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.152+0000 7fcc6affd640 1 -- 172.21.15.43:0/1998778370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc54004800 msgr2=0x7fcc54004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:47.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.152+0000 7fcc6affd640 1 --2- 172.21.15.43:0/1998778370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc54004800 0x7fcc54004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.152+0000 7fcc6affd640 1 -- 172.21.15.43:0/1998778370 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc54003520 con 0x7fcc540039b0 2024-04-25T08:54:47.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.152+0000 7fcc6b7fe640 1 --2- 172.21.15.43:0/1998778370 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc540051c0 0x7fcc5400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:47.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.152+0000 7fcc6a7fc640 1 --2- 172.21.15.43:0/1998778370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc54004800 0x7fcc54004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:47.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.152+0000 7fcc6b7fe640 1 -- 172.21.15.43:0/1998778370 reap_dead start 2024-04-25T08:54:47.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.152+0000 7fcc6affd640 1 --2- 172.21.15.43:0/1998778370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc540039b0 0x7fcc54003d90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c009ac0 tx=0x7fcc54005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e531104e5ba33db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:47.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.152+0000 7fcc69ffb640 1 -- 172.21.15.43:0/1998778370 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c036070 con 0x7fcc540039b0 2024-04-25T08:54:47.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.152+0000 7fcc69ffb640 1 -- 172.21.15.43:0/1998778370 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc5c031400 con 0x7fcc540039b0 2024-04-25T08:54:47.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.152+0000 7fcc69ffb640 1 -- 172.21.15.43:0/1998778370 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c009690 con 0x7fcc540039b0 2024-04-25T08:54:47.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.153+0000 7fcc712cb640 1 -- 172.21.15.43:0/1998778370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc540039b0 msgr2=0x7fcc54003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:47.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.153+0000 7fcc712cb640 1 --2- 172.21.15.43:0/1998778370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc540039b0 0x7fcc54003d90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c009ac0 tx=0x7fcc54005830 comp rx=0 tx=0).stop 2024-04-25T08:54:47.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.153+0000 7fcc712cb640 1 -- 172.21.15.43:0/1998778370 shutdown_connections 2024-04-25T08:54:47.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.153+0000 7fcc712cb640 1 --2- 172.21.15.43:0/1998778370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc540039b0 0x7fcc54003d90 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.153+0000 7fcc712cb640 1 -- 172.21.15.43:0/1998778370 >> 172.21.15.43:0/1998778370 conn(0x7fcc6c0d1700 msgr2=0x7fcc6c0d1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:47.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.153+0000 7fcc712cb640 1 -- 172.21.15.43:0/1998778370 shutdown_connections 2024-04-25T08:54:47.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.153+0000 7fcc712cb640 1 -- 172.21.15.43:0/1998778370 wait complete. 2024-04-25T08:54:47.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.154+0000 7fcc712cb640 1 Processor -- start 2024-04-25T08:54:47.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.154+0000 7fcc712cb640 1 -- start start 2024-04-25T08:54:47.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.155+0000 7fcc712cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc540039b0 0x7fcc6c1e8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:47.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.155+0000 7fcc712cb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc6c1f3bb0 0x7fcc6c0afed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:47.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.155+0000 7fcc712cb640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc6c15a9e0 0x7fcc6c153790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:47.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.155+0000 7fcc712cb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6c06bce0 con 0x7fcc540039b0 2024-04-25T08:54:47.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.155+0000 7fcc712cb640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6c0d0aa0 con 0x7fcc6c1f3bb0 2024-04-25T08:54:47.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.155+0000 7fcc712cb640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6c0b0d50 con 0x7fcc6c15a9e0 2024-04-25T08:54:47.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.155+0000 7fcc6affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc540039b0 0x7fcc6c1e8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:47.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.155+0000 7fcc6affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc540039b0 0x7fcc6c1e8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38366/0 (socket says 172.21.15.43:38366) 2024-04-25T08:54:47.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.155+0000 7fcc6affd640 1 -- 172.21.15.43:0/1608685787 learned_addr learned my addr 172.21.15.43:0/1608685787 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:47.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.155+0000 7fcc6a7fc640 1 --2- 172.21.15.43:0/1608685787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc6c1f3bb0 0x7fcc6c0afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:47.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.155+0000 7fcc6b7fe640 1 --2- 172.21.15.43:0/1608685787 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc6c15a9e0 0x7fcc6c153790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:47.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.156+0000 7fcc6affd640 1 -- 172.21.15.43:0/1608685787 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc6c15a9e0 msgr2=0x7fcc6c153790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:47.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.156+0000 7fcc6affd640 1 --2- 172.21.15.43:0/1608685787 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc6c15a9e0 0x7fcc6c153790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.156+0000 7fcc6affd640 1 -- 172.21.15.43:0/1608685787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc6c1f3bb0 msgr2=0x7fcc6c0afed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:47.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.156+0000 7fcc6affd640 1 --2- 172.21.15.43:0/1608685787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc6c1f3bb0 0x7fcc6c0afed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.156+0000 7fcc6affd640 1 -- 172.21.15.43:0/1608685787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc6c1f4040 con 0x7fcc540039b0 2024-04-25T08:54:47.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.156+0000 7fcc6b7fe640 1 --2- 172.21.15.43:0/1608685787 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc6c15a9e0 0x7fcc6c153790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:47.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.156+0000 7fcc6affd640 1 --2- 172.21.15.43:0/1608685787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc540039b0 0x7fcc6c1e8d90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c034040 tx=0x7fcc5c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:47.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.156+0000 7fcc537fe640 1 -- 172.21.15.43:0/1608685787 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c03f020 con 0x7fcc540039b0 2024-04-25T08:54:47.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.156+0000 7fcc537fe640 1 -- 172.21.15.43:0/1608685787 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc5c037040 con 0x7fcc540039b0 2024-04-25T08:54:47.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.156+0000 7fcc537fe640 1 -- 172.21.15.43:0/1608685787 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c036070 con 0x7fcc540039b0 2024-04-25T08:54:47.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.156+0000 7fcc712cb640 1 -- 172.21.15.43:0/1608685787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc6c1f4320 con 0x7fcc540039b0 2024-04-25T08:54:47.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.156+0000 7fcc712cb640 1 -- 172.21.15.43:0/1608685787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc6c1f48e0 con 0x7fcc540039b0 2024-04-25T08:54:47.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.157+0000 7fcc712cb640 1 -- 172.21.15.43:0/1608685787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc6c074ef0 con 0x7fcc540039b0 2024-04-25T08:54:47.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.157+0000 7fcc537fe640 1 -- 172.21.15.43:0/1608685787 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcc5c042020 con 0x7fcc540039b0 2024-04-25T08:54:47.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.158+0000 7fcc537fe640 1 --2- 172.21.15.43:0/1608685787 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcc5806b500 0x7fcc5806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:47.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.158+0000 7fcc537fe640 1 -- 172.21.15.43:0/1608685787 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcc5c0aa1f0 con 0x7fcc540039b0 2024-04-25T08:54:47.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.158+0000 7fcc6a7fc640 1 --2- 172.21.15.43:0/1608685787 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcc5806b500 0x7fcc5806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:47.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.159+0000 7fcc6a7fc640 1 --2- 172.21.15.43:0/1608685787 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcc5806b500 0x7fcc5806d9c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcc60000ff0 tx=0x7fcc60009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:47.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.162+0000 7fcc537fe640 1 -- 172.21.15.43:0/1608685787 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcc5c049030 con 0x7fcc540039b0 2024-04-25T08:54:47.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.342+0000 7fcc712cb640 1 -- 172.21.15.43:0/1608685787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc6c0d0d80 con 0x7fcc540039b0 2024-04-25T08:54:47.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.343+0000 7fcc537fe640 1 -- 172.21.15.43:0/1608685787 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16569 (secure 0 0 0) 0x7fcc5c079130 con 0x7fcc540039b0 2024-04-25T08:54:47.334 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:54:47.334 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":20,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:54:13.593188+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:54:09.757978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:54:10.926008+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:54:47.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.346+0000 7fcc712cb640 1 -- 172.21.15.43:0/1608685787 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcc5806b500 msgr2=0x7fcc5806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:47.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.346+0000 7fcc712cb640 1 --2- 172.21.15.43:0/1608685787 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcc5806b500 0x7fcc5806d9c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcc60000ff0 tx=0x7fcc60009040 comp rx=0 tx=0).stop 2024-04-25T08:54:47.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.346+0000 7fcc712cb640 1 -- 172.21.15.43:0/1608685787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc540039b0 msgr2=0x7fcc6c1e8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:47.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.346+0000 7fcc712cb640 1 --2- 172.21.15.43:0/1608685787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc540039b0 0x7fcc6c1e8d90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c034040 tx=0x7fcc5c004730 comp rx=0 tx=0).stop 2024-04-25T08:54:47.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.346+0000 7fcc712cb640 1 -- 172.21.15.43:0/1608685787 shutdown_connections 2024-04-25T08:54:47.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.346+0000 7fcc712cb640 1 --2- 172.21.15.43:0/1608685787 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc6c15a9e0 0x7fcc6c153790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.346+0000 7fcc712cb640 1 --2- 172.21.15.43:0/1608685787 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcc5806b500 0x7fcc5806d9c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.346+0000 7fcc712cb640 1 --2- 172.21.15.43:0/1608685787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc6c1f3bb0 0x7fcc6c0afed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.346+0000 7fcc712cb640 1 --2- 172.21.15.43:0/1608685787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc540039b0 0x7fcc6c1e8d90 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.346+0000 7fcc712cb640 1 -- 172.21.15.43:0/1608685787 >> 172.21.15.43:0/1608685787 conn(0x7fcc6c0d1700 msgr2=0x7fcc6c0b26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:47.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.346+0000 7fcc712cb640 1 -- 172.21.15.43:0/1608685787 shutdown_connections 2024-04-25T08:54:47.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.346+0000 7fcc712cb640 1 -- 172.21.15.43:0/1608685787 wait complete. 2024-04-25T08:54:47.349 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:54:09.757978+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:54:10.926008+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:54:47.349 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:54:47.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.463+0000 7fa5dbf9e640 1 Processor -- start 2024-04-25T08:54:47.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.463+0000 7fa5dbf9e640 1 -- start start 2024-04-25T08:54:47.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.464+0000 7fa5dbf9e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5d40d1fa0 0x7fa5d40cece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:47.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.464+0000 7fa5dbf9e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa5d40cf220 0x7fa5d406b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:47.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.464+0000 7fa5dbf9e640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5d40d47f0 con 0x7fa5d415dc40 2024-04-25T08:54:47.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.464+0000 7fa5dbf9e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5d40d13c0 con 0x7fa5d40cf220 2024-04-25T08:54:47.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.464+0000 7fa5dbf9e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d40d24b0 con 0x7fa5d40d1fa0 2024-04-25T08:54:47.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.464+0000 7fa5da53b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa5d40cf220 0x7fa5d406b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:47.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.464+0000 7fa5da53b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa5d40cf220 0x7fa5d406b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38368/0 (socket says 172.21.15.43:38368) 2024-04-25T08:54:47.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.464+0000 7fa5da53b640 1 -- 172.21.15.43:0/283808913 learned_addr learned my addr 172.21.15.43:0/283808913 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:47.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.464+0000 7fa5d9539640 1 --2- 172.21.15.43:0/283808913 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5d40d1fa0 0x7fa5d40cece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:47.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.464+0000 7fa5da53b640 1 -- 172.21.15.43:0/283808913 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5d40d1fa0 msgr2=0x7fa5d40cece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:47.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.464+0000 7fa5da53b640 1 --2- 172.21.15.43:0/283808913 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5d40d1fa0 0x7fa5d40cece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.464+0000 7fa5da53b640 1 -- 172.21.15.43:0/283808913 >> v1:172.21.15.193:6789/0 conn(0x7fa5d415dc40 legacy=0x7fa5d40ce5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:47.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.464+0000 7fa5da53b640 1 -- 172.21.15.43:0/283808913 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d40d2750 con 0x7fa5d40cf220 2024-04-25T08:54:47.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.464+0000 7fa5d9539640 1 --2- 172.21.15.43:0/283808913 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5d40d1fa0 0x7fa5d40cece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:47.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.465+0000 7fa5da53b640 1 --2- 172.21.15.43:0/283808913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5d40cf220 0x7fa5d406b970 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc009e50 tx=0x7fa5cc02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c80026dce3f1b086 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:47.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.465+0000 7fa5d8d38640 1 -- 172.21.15.43:0/283808913 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc036070 con 0x7fa5d40cf220 2024-04-25T08:54:47.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.465+0000 7fa5d8d38640 1 -- 172.21.15.43:0/283808913 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5cc031600 con 0x7fa5d40cf220 2024-04-25T08:54:47.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.465+0000 7fa5d8d38640 1 -- 172.21.15.43:0/283808913 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc0319d0 con 0x7fa5d40cf220 2024-04-25T08:54:47.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.465+0000 7fa5dbf9e640 1 -- 172.21.15.43:0/283808913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5d40cf220 msgr2=0x7fa5d406b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:47.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.466+0000 7fa5dbf9e640 1 --2- 172.21.15.43:0/283808913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5d40cf220 0x7fa5d406b970 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc009e50 tx=0x7fa5cc02a680 comp rx=0 tx=0).stop 2024-04-25T08:54:47.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.466+0000 7fa5dbf9e640 1 -- 172.21.15.43:0/283808913 shutdown_connections 2024-04-25T08:54:47.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.466+0000 7fa5dbf9e640 1 --2- 172.21.15.43:0/283808913 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5d40cf220 0x7fa5d406b970 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.466+0000 7fa5dbf9e640 1 --2- 172.21.15.43:0/283808913 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5d40d1fa0 0x7fa5d40cece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.466+0000 7fa5dbf9e640 1 -- 172.21.15.43:0/283808913 >> 172.21.15.43:0/283808913 conn(0x7fa5d40d39c0 msgr2=0x7fa5d40d3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:47.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.466+0000 7fa5dbf9e640 1 -- 172.21.15.43:0/283808913 shutdown_connections 2024-04-25T08:54:47.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.466+0000 7fa5dbf9e640 1 -- 172.21.15.43:0/283808913 wait complete. 2024-04-25T08:54:47.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.467+0000 7fa5dbf9e640 1 Processor -- start 2024-04-25T08:54:47.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.467+0000 7fa5dbf9e640 1 -- start start 2024-04-25T08:54:47.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.468+0000 7fa5dbf9e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5d40cf220 0x7fa5d40ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:47.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.468+0000 7fa5dbf9e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5d40d1fa0 0x7fa5d40caa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:47.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.468+0000 7fa5dbf9e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5d415dc40 0x7fa5d40b8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:47.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.468+0000 7fa5dbf9e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d40d2a30 con 0x7fa5d415dc40 2024-04-25T08:54:47.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.468+0000 7fa5dbf9e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d406b450 con 0x7fa5d40cf220 2024-04-25T08:54:47.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.468+0000 7fa5dbf9e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d415e020 con 0x7fa5d40d1fa0 2024-04-25T08:54:47.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.468+0000 7fa5da53b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5d415dc40 0x7fa5d40b8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:47.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.468+0000 7fa5da53b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5d415dc40 0x7fa5d40b8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38372/0 (socket says 172.21.15.43:38372) 2024-04-25T08:54:47.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.468+0000 7fa5da53b640 1 -- 172.21.15.43:0/157237370 learned_addr learned my addr 172.21.15.43:0/157237370 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:47.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.468+0000 7fa5d9539640 1 --2- 172.21.15.43:0/157237370 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5d40d1fa0 0x7fa5d40caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:47.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.468+0000 7fa5d9d3a640 1 --2- 172.21.15.43:0/157237370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5d40cf220 0x7fa5d40ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:47.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.468+0000 7fa5da53b640 1 -- 172.21.15.43:0/157237370 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5d40d1fa0 msgr2=0x7fa5d40caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:47.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.468+0000 7fa5da53b640 1 --2- 172.21.15.43:0/157237370 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5d40d1fa0 0x7fa5d40caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.468+0000 7fa5da53b640 1 -- 172.21.15.43:0/157237370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5d40cf220 msgr2=0x7fa5d40ccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:47.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.468+0000 7fa5da53b640 1 --2- 172.21.15.43:0/157237370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5d40cf220 0x7fa5d40ccd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.468+0000 7fa5da53b640 1 -- 172.21.15.43:0/157237370 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d40cb4e0 con 0x7fa5d415dc40 2024-04-25T08:54:47.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.468+0000 7fa5d9539640 1 --2- 172.21.15.43:0/157237370 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5d40d1fa0 0x7fa5d40caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:47.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.468+0000 7fa5da53b640 1 --2- 172.21.15.43:0/157237370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5d415dc40 0x7fa5d40b8c50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc034040 tx=0x7fa5cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:47.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.468+0000 7fa5ca7fc640 1 -- 172.21.15.43:0/157237370 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc03f020 con 0x7fa5d415dc40 2024-04-25T08:54:47.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.468+0000 7fa5ca7fc640 1 -- 172.21.15.43:0/157237370 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5cc037040 con 0x7fa5d415dc40 2024-04-25T08:54:47.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.468+0000 7fa5dbf9e640 1 -- 172.21.15.43:0/157237370 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5d40cb7c0 con 0x7fa5d415dc40 2024-04-25T08:54:47.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.468+0000 7fa5ca7fc640 1 -- 172.21.15.43:0/157237370 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc036070 con 0x7fa5d415dc40 2024-04-25T08:54:47.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.468+0000 7fa5dbf9e640 1 -- 172.21.15.43:0/157237370 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5d40cbd80 con 0x7fa5d415dc40 2024-04-25T08:54:47.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.469+0000 7fa5dbf9e640 1 -- 172.21.15.43:0/157237370 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5d406ac30 con 0x7fa5d415dc40 2024-04-25T08:54:47.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.470+0000 7fa5ca7fc640 1 -- 172.21.15.43:0/157237370 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa5cc042020 con 0x7fa5d415dc40 2024-04-25T08:54:47.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.470+0000 7fa5ca7fc640 1 --2- 172.21.15.43:0/157237370 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa5b806b4d0 0x7fa5b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:47.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.470+0000 7fa5ca7fc640 1 -- 172.21.15.43:0/157237370 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa5cc0aa6d0 con 0x7fa5d415dc40 2024-04-25T08:54:47.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.470+0000 7fa5d9d3a640 1 --2- 172.21.15.43:0/157237370 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa5b806b4d0 0x7fa5b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:47.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.471+0000 7fa5d9d3a640 1 --2- 172.21.15.43:0/157237370 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa5b806b4d0 0x7fa5b806d990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0001500 tx=0x7fa5d0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:47.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.475+0000 7fa5ca7fc640 1 -- 172.21.15.43:0/157237370 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa5d406ac30 con 0x7fa5d415dc40 2024-04-25T08:54:47.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.654+0000 7fa5dbf9e640 1 -- 172.21.15.43:0/157237370 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa5d406ce40 con 0x7fa5d415dc40 2024-04-25T08:54:47.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.655+0000 7fa5ca7fc640 1 -- 172.21.15.43:0/157237370 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa5cc049030 con 0x7fa5d415dc40 2024-04-25T08:54:47.645 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T08:54:47.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.657+0000 7fa5dbf9e640 1 -- 172.21.15.43:0/157237370 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa5b806b4d0 msgr2=0x7fa5b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:47.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.657+0000 7fa5dbf9e640 1 --2- 172.21.15.43:0/157237370 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa5b806b4d0 0x7fa5b806d990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa5d0001500 tx=0x7fa5d0009040 comp rx=0 tx=0).stop 2024-04-25T08:54:47.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.657+0000 7fa5dbf9e640 1 -- 172.21.15.43:0/157237370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5d415dc40 msgr2=0x7fa5d40b8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:47.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.657+0000 7fa5dbf9e640 1 --2- 172.21.15.43:0/157237370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5d415dc40 0x7fa5d40b8c50 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc034040 tx=0x7fa5cc004a40 comp rx=0 tx=0).stop 2024-04-25T08:54:47.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.657+0000 7fa5dbf9e640 1 -- 172.21.15.43:0/157237370 shutdown_connections 2024-04-25T08:54:47.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.657+0000 7fa5dbf9e640 1 --2- 172.21.15.43:0/157237370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5d415dc40 0x7fa5d40b8c50 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.657+0000 7fa5dbf9e640 1 --2- 172.21.15.43:0/157237370 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5d40d1fa0 0x7fa5d40caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.657+0000 7fa5dbf9e640 1 --2- 172.21.15.43:0/157237370 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa5b806b4d0 0x7fa5b806d990 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.657+0000 7fa5dbf9e640 1 --2- 172.21.15.43:0/157237370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5d40cf220 0x7fa5d40ccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.657+0000 7fa5dbf9e640 1 -- 172.21.15.43:0/157237370 >> 172.21.15.43:0/157237370 conn(0x7fa5d40d39c0 msgr2=0x7fa5d406b870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:47.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.657+0000 7fa5dbf9e640 1 -- 172.21.15.43:0/157237370 shutdown_connections 2024-04-25T08:54:47.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.657+0000 7fa5dbf9e640 1 -- 172.21.15.43:0/157237370 wait complete. 2024-04-25T08:54:47.658 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:54:47.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.772+0000 7f9716038640 1 Processor -- start 2024-04-25T08:54:47.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.772+0000 7f9716038640 1 -- start start 2024-04-25T08:54:47.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.773+0000 7f9716038640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f97100aee80 0x7f971006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:47.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.773+0000 7f9716038640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f971015c280 con 0x7f97100cd2f0 2024-04-25T08:54:47.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.773+0000 7f9716038640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f971015c560 con 0x7f971006d600 2024-04-25T08:54:47.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.773+0000 7f9716038640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97100af260 con 0x7f97100aee80 2024-04-25T08:54:47.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.773+0000 7f970ffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f97100cd2f0 0x7f971015d590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39722/0 (socket says 172.21.15.43:39722) 2024-04-25T08:54:47.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.773+0000 7f970ffff640 1 -- 172.21.15.43:0/2136846803 learned_addr learned my addr 172.21.15.43:0/2136846803 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:47.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.774+0000 7f970f7fe640 1 --2- 172.21.15.43:0/2136846803 >> v2:172.21.15.193:3300/0 conn(0x7f97100aee80 0x7f971006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:47.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.774+0000 7f970e7fc640 1 -- 172.21.15.43:0/2136846803 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1261331786 0 0) 0x7f971015c280 con 0x7f97100cd2f0 2024-04-25T08:54:47.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.774+0000 7f970e7fc640 1 -- 172.21.15.43:0/2136846803 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96f8003520 con 0x7f97100cd2f0 2024-04-25T08:54:47.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.774+0000 7f970e7fc640 1 -- 172.21.15.43:0/2136846803 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 858795193 0 0) 0x7f96f8003520 con 0x7f97100cd2f0 2024-04-25T08:54:47.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.774+0000 7f970e7fc640 1 -- 172.21.15.43:0/2136846803 >> v2:172.21.15.193:3300/0 conn(0x7f97100aee80 msgr2=0x7f971006d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:47.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.774+0000 7f970e7fc640 1 --2- 172.21.15.43:0/2136846803 >> v2:172.21.15.193:3300/0 conn(0x7f97100aee80 0x7f971006d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.774+0000 7f970e7fc640 1 -- 172.21.15.43:0/2136846803 >> v1:172.21.15.193:6789/0 conn(0x7f971006d600 legacy=0x7f97100ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:47.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.774+0000 7f970e7fc640 1 -- 172.21.15.43:0/2136846803 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f971015c840 con 0x7f97100cd2f0 2024-04-25T08:54:47.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.774+0000 7f970e7fc640 1 -- 172.21.15.43:0/2136846803 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f97080019f0 con 0x7f97100cd2f0 2024-04-25T08:54:47.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.774+0000 7f970e7fc640 1 -- 172.21.15.43:0/2136846803 >> v1:172.21.15.43:6789/0 conn(0x7f97100cd2f0 legacy=0x7f971015d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:47.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.774+0000 7f970e7fc640 1 --2- 172.21.15.43:0/2136846803 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96f8003a00 0x7f96f8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:47.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.774+0000 7f970e7fc640 1 --2- 172.21.15.43:0/2136846803 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f96f8004800 0x7f96f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:47.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.774+0000 7f970e7fc640 1 --2- 172.21.15.43:0/2136846803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96f80051c0 0x7f96f800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:47.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.774+0000 7f970e7fc640 1 -- 172.21.15.43:0/2136846803 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f800c090 con 0x7f96f80051c0 2024-04-25T08:54:47.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.774+0000 7f970e7fc640 1 -- 172.21.15.43:0/2136846803 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f800c240 con 0x7f96f8003a00 2024-04-25T08:54:47.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.774+0000 7f970e7fc640 1 -- 172.21.15.43:0/2136846803 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f800c440 con 0x7f96f8004800 2024-04-25T08:54:47.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.774+0000 7f970ffff640 1 --2- 172.21.15.43:0/2136846803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96f80051c0 0x7f96f800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:47.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.774+0000 7f970f7fe640 1 --2- 172.21.15.43:0/2136846803 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96f8003a00 0x7f96f8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:47.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.774+0000 7f970effd640 1 --2- 172.21.15.43:0/2136846803 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f96f8004800 0x7f96f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:47.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.774+0000 7f970ffff640 1 -- 172.21.15.43:0/2136846803 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f96f8004800 msgr2=0x7f96f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:47.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.774+0000 7f970ffff640 1 --2- 172.21.15.43:0/2136846803 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f96f8004800 0x7f96f8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.774+0000 7f970ffff640 1 -- 172.21.15.43:0/2136846803 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96f8003a00 msgr2=0x7f96f8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:47.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.774+0000 7f970ffff640 1 --2- 172.21.15.43:0/2136846803 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96f8003a00 0x7f96f8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.775+0000 7f970ffff640 1 -- 172.21.15.43:0/2136846803 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f8003520 con 0x7f96f80051c0 2024-04-25T08:54:47.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.775+0000 7f970effd640 1 --2- 172.21.15.43:0/2136846803 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f96f8004800 0x7f96f8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:47.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.775+0000 7f970ffff640 1 -- 172.21.15.43:0/2136846803 reap_dead start 2024-04-25T08:54:47.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.775+0000 7f970ffff640 1 --2- 172.21.15.43:0/2136846803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96f80051c0 0x7f96f800b9c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9708001d80 tx=0x7f96f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a4347aea521b297 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:47.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.775+0000 7f970e7fc640 1 -- 172.21.15.43:0/2136846803 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970800a390 con 0x7f96f80051c0 2024-04-25T08:54:47.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.775+0000 7f970e7fc640 1 -- 172.21.15.43:0/2136846803 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f970803b650 con 0x7f96f80051c0 2024-04-25T08:54:47.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.775+0000 7f970e7fc640 1 -- 172.21.15.43:0/2136846803 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970803ba40 con 0x7f96f80051c0 2024-04-25T08:54:47.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.776+0000 7f9716038640 1 -- 172.21.15.43:0/2136846803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96f80051c0 msgr2=0x7f96f800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:47.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.776+0000 7f9716038640 1 --2- 172.21.15.43:0/2136846803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96f80051c0 0x7f96f800b9c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9708001d80 tx=0x7f96f8005830 comp rx=0 tx=0).stop 2024-04-25T08:54:47.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.776+0000 7f9716038640 1 -- 172.21.15.43:0/2136846803 shutdown_connections 2024-04-25T08:54:47.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.776+0000 7f9716038640 1 --2- 172.21.15.43:0/2136846803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96f80051c0 0x7f96f800b9c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.776+0000 7f9716038640 1 -- 172.21.15.43:0/2136846803 >> 172.21.15.43:0/2136846803 conn(0x7f971006ab30 msgr2=0x7f971006af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:47.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.776+0000 7f9716038640 1 -- 172.21.15.43:0/2136846803 shutdown_connections 2024-04-25T08:54:47.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.776+0000 7f9716038640 1 -- 172.21.15.43:0/2136846803 wait complete. 2024-04-25T08:54:47.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.777+0000 7f9716038640 1 Processor -- start 2024-04-25T08:54:47.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.777+0000 7f9716038640 1 -- start start 2024-04-25T08:54:47.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.778+0000 7f9716038640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96f80051c0 0x7f971014e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:47.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.778+0000 7f9716038640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97101517b0 0x7f9710151c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:47.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.778+0000 7f9716038640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f971015e500 0x7f971015e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:47.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.778+0000 7f9716038640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971006d240 con 0x7f97101517b0 2024-04-25T08:54:47.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.778+0000 7f9716038640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971006d0c0 con 0x7f96f80051c0 2024-04-25T08:54:47.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.778+0000 7f9716038640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f971006d3c0 con 0x7f971015e500 2024-04-25T08:54:47.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.778+0000 7f970effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97101517b0 0x7f9710151c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:47.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.778+0000 7f970effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97101517b0 0x7f9710151c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38394/0 (socket says 172.21.15.43:38394) 2024-04-25T08:54:47.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.778+0000 7f970effd640 1 -- 172.21.15.43:0/3025877656 learned_addr learned my addr 172.21.15.43:0/3025877656 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:47.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.778+0000 7f970ffff640 1 --2- 172.21.15.43:0/3025877656 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f971015e500 0x7f971015e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:47.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.778+0000 7f970f7fe640 1 --2- 172.21.15.43:0/3025877656 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96f80051c0 0x7f971014e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:47.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.778+0000 7f970effd640 1 -- 172.21.15.43:0/3025877656 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f971015e500 msgr2=0x7f971015e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:47.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.778+0000 7f970effd640 1 --2- 172.21.15.43:0/3025877656 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f971015e500 0x7f971015e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.778+0000 7f970effd640 1 -- 172.21.15.43:0/3025877656 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96f80051c0 msgr2=0x7f971014e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:47.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.778+0000 7f970effd640 1 --2- 172.21.15.43:0/3025877656 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96f80051c0 0x7f971014e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.778+0000 7f970effd640 1 -- 172.21.15.43:0/3025877656 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9710152600 con 0x7f97101517b0 2024-04-25T08:54:47.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.778+0000 7f970f7fe640 1 --2- 172.21.15.43:0/3025877656 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96f80051c0 0x7f971014e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:47.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.779+0000 7f970effd640 1 --2- 172.21.15.43:0/3025877656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97101517b0 0x7f9710151c30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f970400e460 tx=0x7f970400e920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:47.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.779+0000 7f96f7fff640 1 -- 172.21.15.43:0/3025877656 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97040052e0 con 0x7f97101517b0 2024-04-25T08:54:47.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.779+0000 7f96f7fff640 1 -- 172.21.15.43:0/3025877656 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9704018070 con 0x7f97101517b0 2024-04-25T08:54:47.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.779+0000 7f9716038640 1 -- 172.21.15.43:0/3025877656 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f971014f410 con 0x7f97101517b0 2024-04-25T08:54:47.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.779+0000 7f96f7fff640 1 -- 172.21.15.43:0/3025877656 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9704003830 con 0x7f97101517b0 2024-04-25T08:54:47.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.779+0000 7f9716038640 1 -- 172.21.15.43:0/3025877656 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97100b2710 con 0x7f97101517b0 2024-04-25T08:54:47.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.780+0000 7f9716038640 1 -- 172.21.15.43:0/3025877656 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9710074f60 con 0x7f97101517b0 2024-04-25T08:54:47.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.780+0000 7f96f7fff640 1 -- 172.21.15.43:0/3025877656 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f97040039d0 con 0x7f97101517b0 2024-04-25T08:54:47.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.781+0000 7f96f7fff640 1 --2- 172.21.15.43:0/3025877656 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f96fc06b500 0x7f96fc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:47.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.781+0000 7f96f7fff640 1 -- 172.21.15.43:0/3025877656 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9704089e40 con 0x7f97101517b0 2024-04-25T08:54:47.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.781+0000 7f970f7fe640 1 --2- 172.21.15.43:0/3025877656 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f96fc06b500 0x7f96fc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:47.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.781+0000 7f970f7fe640 1 --2- 172.21.15.43:0/3025877656 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f96fc06b500 0x7f96fc06d9c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f97000093d0 tx=0x7f9700008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:47.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.785+0000 7f96f7fff640 1 -- 172.21.15.43:0/3025877656 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9704058c20 con 0x7f97101517b0 2024-04-25T08:54:47.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.966+0000 7f9716038640 1 -- 172.21.15.43:0/3025877656 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f97100725f0 con 0x7f97101517b0 2024-04-25T08:54:47.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.966+0000 7f96f7fff640 1 -- 172.21.15.43:0/3025877656 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9704058520 con 0x7f97101517b0 2024-04-25T08:54:47.956 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T08:54:47.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.968+0000 7f9716038640 1 -- 172.21.15.43:0/3025877656 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f96fc06b500 msgr2=0x7f96fc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:47.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.968+0000 7f9716038640 1 --2- 172.21.15.43:0/3025877656 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f96fc06b500 0x7f96fc06d9c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f97000093d0 tx=0x7f9700008040 comp rx=0 tx=0).stop 2024-04-25T08:54:47.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.968+0000 7f9716038640 1 -- 172.21.15.43:0/3025877656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97101517b0 msgr2=0x7f9710151c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:47.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.968+0000 7f9716038640 1 --2- 172.21.15.43:0/3025877656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97101517b0 0x7f9710151c30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f970400e460 tx=0x7f970400e920 comp rx=0 tx=0).stop 2024-04-25T08:54:47.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.968+0000 7f9716038640 1 -- 172.21.15.43:0/3025877656 shutdown_connections 2024-04-25T08:54:47.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.968+0000 7f9716038640 1 --2- 172.21.15.43:0/3025877656 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f971015e500 0x7f971015e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.968+0000 7f9716038640 1 --2- 172.21.15.43:0/3025877656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97101517b0 0x7f9710151c30 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.968+0000 7f9716038640 1 --2- 172.21.15.43:0/3025877656 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f96fc06b500 0x7f96fc06d9c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.968+0000 7f9716038640 1 --2- 172.21.15.43:0/3025877656 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96f80051c0 0x7f971014e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:47.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.968+0000 7f9716038640 1 -- 172.21.15.43:0/3025877656 >> 172.21.15.43:0/3025877656 conn(0x7f971006ab30 msgr2=0x7f971015a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:47.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.968+0000 7f9716038640 1 -- 172.21.15.43:0/3025877656 shutdown_connections 2024-04-25T08:54:47.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:47.968+0000 7f9716038640 1 -- 172.21.15.43:0/3025877656 wait complete. 2024-04-25T08:54:47.969 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:54:48.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.083+0000 7f7936207640 1 Processor -- start 2024-04-25T08:54:48.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.083+0000 7f7936207640 1 -- start start 2024-04-25T08:54:48.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.084+0000 7f7936207640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79300aee80 0x7f793006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:48.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.084+0000 7f7936207640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f793006d600 0x7f79300ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:48.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.084+0000 7f7936207640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f793015c280 con 0x7f79300cd120 2024-04-25T08:54:48.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.084+0000 7f7936207640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79300af260 con 0x7f793006d600 2024-04-25T08:54:48.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.084+0000 7f7936207640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f793006da60 con 0x7f79300aee80 2024-04-25T08:54:48.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.084+0000 7f792ffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f79300cd120 0x7f793015f6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39728/0 (socket says 172.21.15.43:39728) 2024-04-25T08:54:48.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.084+0000 7f792ffff640 1 -- 172.21.15.43:0/1576893554 learned_addr learned my addr 172.21.15.43:0/1576893554 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:48.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.084+0000 7f792e7fc640 1 -- 172.21.15.43:0/1576893554 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1978593115 0 0) 0x7f793015c280 con 0x7f79300cd120 2024-04-25T08:54:48.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.084+0000 7f792f7fe640 1 --2- 172.21.15.43:0/1576893554 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79300aee80 0x7f793006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:48.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.084+0000 7f792effd640 1 --2- 172.21.15.43:0/1576893554 >> v2:172.21.15.193:3300/0 conn(0x7f793006d600 0x7f79300ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:48.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.084+0000 7f792e7fc640 1 -- 172.21.15.43:0/1576893554 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f791c003520 con 0x7f79300cd120 2024-04-25T08:54:48.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.084+0000 7f792e7fc640 1 -- 172.21.15.43:0/1576893554 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 954096256 0 0) 0x7f791c003520 con 0x7f79300cd120 2024-04-25T08:54:48.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.084+0000 7f792e7fc640 1 -- 172.21.15.43:0/1576893554 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79300aee80 msgr2=0x7f793006d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:48.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.084+0000 7f792e7fc640 1 --2- 172.21.15.43:0/1576893554 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79300aee80 0x7f793006d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.084+0000 7f792e7fc640 1 -- 172.21.15.43:0/1576893554 >> v2:172.21.15.193:3300/0 conn(0x7f793006d600 msgr2=0x7f79300ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:48.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.084+0000 7f792e7fc640 1 --2- 172.21.15.43:0/1576893554 >> v2:172.21.15.193:3300/0 conn(0x7f793006d600 0x7f79300ccbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.084+0000 7f792e7fc640 1 -- 172.21.15.43:0/1576893554 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f793015c560 con 0x7f79300cd120 2024-04-25T08:54:48.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.084+0000 7f792e7fc640 1 -- 172.21.15.43:0/1576893554 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f79280019f0 con 0x7f79300cd120 2024-04-25T08:54:48.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.084+0000 7f792e7fc640 1 -- 172.21.15.43:0/1576893554 >> v1:172.21.15.43:6789/0 conn(0x7f79300cd120 legacy=0x7f793015f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.085+0000 7f792e7fc640 1 --2- 172.21.15.43:0/1576893554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f791c0039b0 0x7f791c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.085+0000 7f792e7fc640 1 --2- 172.21.15.43:0/1576893554 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f791c004800 0x7f791c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.085+0000 7f792e7fc640 1 --2- 172.21.15.43:0/1576893554 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79300aee80 0x7f791c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.085+0000 7f792e7fc640 1 -- 172.21.15.43:0/1576893554 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791c00b9c0 con 0x7f791c0039b0 2024-04-25T08:54:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.085+0000 7f792e7fc640 1 -- 172.21.15.43:0/1576893554 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791c00bb40 con 0x7f791c004800 2024-04-25T08:54:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.085+0000 7f792e7fc640 1 -- 172.21.15.43:0/1576893554 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f791c00bcc0 con 0x7f79300aee80 2024-04-25T08:54:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.085+0000 7f792f7fe640 1 --2- 172.21.15.43:0/1576893554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f791c0039b0 0x7f791c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.085+0000 7f792effd640 1 --2- 172.21.15.43:0/1576893554 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f791c004800 0x7f791c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.085+0000 7f792ffff640 1 --2- 172.21.15.43:0/1576893554 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79300aee80 0x7f791c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.085+0000 7f792f7fe640 1 -- 172.21.15.43:0/1576893554 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79300aee80 msgr2=0x7f791c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.085+0000 7f792f7fe640 1 --2- 172.21.15.43:0/1576893554 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79300aee80 0x7f791c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.085+0000 7f792f7fe640 1 -- 172.21.15.43:0/1576893554 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f791c004800 msgr2=0x7f791c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.085+0000 7f792f7fe640 1 --2- 172.21.15.43:0/1576893554 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f791c004800 0x7f791c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.085+0000 7f792f7fe640 1 -- 172.21.15.43:0/1576893554 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f791c003520 con 0x7f791c0039b0 2024-04-25T08:54:48.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.085+0000 7f792ffff640 1 --2- 172.21.15.43:0/1576893554 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79300aee80 0x7f791c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:48.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.086+0000 7f792f7fe640 1 --2- 172.21.15.43:0/1576893554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f791c0039b0 0x7f791c003dd0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7920009ac0 tx=0x7f791c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e19ab58de1a39cf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:48.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.086+0000 7f792e7fc640 1 -- 172.21.15.43:0/1576893554 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7920036070 con 0x7f791c0039b0 2024-04-25T08:54:48.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.086+0000 7f792e7fc640 1 -- 172.21.15.43:0/1576893554 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7920031400 con 0x7f791c0039b0 2024-04-25T08:54:48.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.086+0000 7f792e7fc640 1 -- 172.21.15.43:0/1576893554 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7920009690 con 0x7f791c0039b0 2024-04-25T08:54:48.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.086+0000 7f7936207640 1 -- 172.21.15.43:0/1576893554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f791c0039b0 msgr2=0x7f791c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:48.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.086+0000 7f7936207640 1 --2- 172.21.15.43:0/1576893554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f791c0039b0 0x7f791c003dd0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7920009ac0 tx=0x7f791c005830 comp rx=0 tx=0).stop 2024-04-25T08:54:48.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.086+0000 7f792ffff640 1 -- 172.21.15.43:0/1576893554 reap_dead start 2024-04-25T08:54:48.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.086+0000 7f7936207640 1 -- 172.21.15.43:0/1576893554 shutdown_connections 2024-04-25T08:54:48.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.086+0000 7f7936207640 1 -- 172.21.15.43:0/1576893554 >> 172.21.15.43:0/1576893554 conn(0x7f793006ab30 msgr2=0x7f793006af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:48.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.086+0000 7f7936207640 1 -- 172.21.15.43:0/1576893554 shutdown_connections 2024-04-25T08:54:48.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.086+0000 7f7936207640 1 -- 172.21.15.43:0/1576893554 wait complete. 2024-04-25T08:54:48.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.088+0000 7f7936207640 1 Processor -- start 2024-04-25T08:54:48.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.088+0000 7f7936207640 1 -- start start 2024-04-25T08:54:48.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.088+0000 7f7936207640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f791c0039b0 0x7f79301425e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:48.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.088+0000 7f7936207640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79301f51d0 0x7f7930158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:48.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.088+0000 7f7936207640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79301f5650 0x7f7930158790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:48.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.088+0000 7f7936207640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f793015cb20 con 0x7f79301f5650 2024-04-25T08:54:48.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.088+0000 7f7936207640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79300cd5b0 con 0x7f791c0039b0 2024-04-25T08:54:48.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.088+0000 7f7936207640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f793015e5a0 con 0x7f79301f51d0 2024-04-25T08:54:48.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.089+0000 7f792ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79301f5650 0x7f7930158790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:48.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.089+0000 7f792ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79301f5650 0x7f7930158790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38406/0 (socket says 172.21.15.43:38406) 2024-04-25T08:54:48.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.089+0000 7f792ffff640 1 -- 172.21.15.43:0/3014657078 learned_addr learned my addr 172.21.15.43:0/3014657078 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:48.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.089+0000 7f792effd640 1 --2- 172.21.15.43:0/3014657078 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79301f51d0 0x7f7930158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:48.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.089+0000 7f792f7fe640 1 --2- 172.21.15.43:0/3014657078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f791c0039b0 0x7f79301425e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:48.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.089+0000 7f792ffff640 1 -- 172.21.15.43:0/3014657078 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79301f51d0 msgr2=0x7f7930158250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:48.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.089+0000 7f792ffff640 1 --2- 172.21.15.43:0/3014657078 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79301f51d0 0x7f7930158250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.089+0000 7f792ffff640 1 -- 172.21.15.43:0/3014657078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f791c0039b0 msgr2=0x7f79301425e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:48.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.089+0000 7f792ffff640 1 --2- 172.21.15.43:0/3014657078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f791c0039b0 0x7f79301425e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.089+0000 7f792ffff640 1 -- 172.21.15.43:0/3014657078 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79301f26f0 con 0x7f79301f5650 2024-04-25T08:54:48.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.089+0000 7f792effd640 1 --2- 172.21.15.43:0/3014657078 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79301f51d0 0x7f7930158250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:48.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.089+0000 7f792f7fe640 1 --2- 172.21.15.43:0/3014657078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f791c0039b0 0x7f79301425e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:48.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.089+0000 7f792ffff640 1 --2- 172.21.15.43:0/3014657078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79301f5650 0x7f7930158790 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7928001350 tx=0x7f792800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:48.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.089+0000 7f7927fff640 1 -- 172.21.15.43:0/3014657078 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f792800eb90 con 0x7f79301f5650 2024-04-25T08:54:48.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.089+0000 7f7927fff640 1 -- 172.21.15.43:0/3014657078 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7928018070 con 0x7f79301f5650 2024-04-25T08:54:48.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.089+0000 7f7936207640 1 -- 172.21.15.43:0/3014657078 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7930153580 con 0x7f79301f5650 2024-04-25T08:54:48.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.089+0000 7f7927fff640 1 -- 172.21.15.43:0/3014657078 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7928013420 con 0x7f79301f5650 2024-04-25T08:54:48.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.089+0000 7f7936207640 1 -- 172.21.15.43:0/3014657078 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79301f18f0 con 0x7f79301f5650 2024-04-25T08:54:48.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.090+0000 7f7936207640 1 -- 172.21.15.43:0/3014657078 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7930074f60 con 0x7f79301f5650 2024-04-25T08:54:48.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.091+0000 7f7927fff640 1 -- 172.21.15.43:0/3014657078 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f79280135c0 con 0x7f79301f5650 2024-04-25T08:54:48.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.091+0000 7f7927fff640 1 --2- 172.21.15.43:0/3014657078 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f791006b4d0 0x7f791006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:48.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.091+0000 7f7927fff640 1 -- 172.21.15.43:0/3014657078 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f792808a520 con 0x7f79301f5650 2024-04-25T08:54:48.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.092+0000 7f792f7fe640 1 --2- 172.21.15.43:0/3014657078 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f791006b4d0 0x7f791006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:48.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.092+0000 7f792f7fe640 1 --2- 172.21.15.43:0/3014657078 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f791006b4d0 0x7f791006d990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7920033000 tx=0x7f7920008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:48.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.096+0000 7f7927fff640 1 -- 172.21.15.43:0/3014657078 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f79280593a0 con 0x7f79301f5650 2024-04-25T08:54:48.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.274+0000 7f7936207640 1 -- 172.21.15.43:0/3014657078 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f79300725f0 con 0x7f79301f5650 2024-04-25T08:54:48.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.274+0000 7f7927fff640 1 -- 172.21.15.43:0/3014657078 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7928058ca0 con 0x7f79301f5650 2024-04-25T08:54:48.264 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T08:54:48.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.276+0000 7f7936207640 1 -- 172.21.15.43:0/3014657078 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f791006b4d0 msgr2=0x7f791006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:48.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.276+0000 7f7936207640 1 --2- 172.21.15.43:0/3014657078 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f791006b4d0 0x7f791006d990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7920033000 tx=0x7f7920008030 comp rx=0 tx=0).stop 2024-04-25T08:54:48.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.276+0000 7f7936207640 1 -- 172.21.15.43:0/3014657078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79301f5650 msgr2=0x7f7930158790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:48.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.276+0000 7f7936207640 1 --2- 172.21.15.43:0/3014657078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79301f5650 0x7f7930158790 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7928001350 tx=0x7f792800e530 comp rx=0 tx=0).stop 2024-04-25T08:54:48.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.276+0000 7f7936207640 1 -- 172.21.15.43:0/3014657078 shutdown_connections 2024-04-25T08:54:48.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.276+0000 7f7936207640 1 --2- 172.21.15.43:0/3014657078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79301f5650 0x7f7930158790 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.276+0000 7f7936207640 1 --2- 172.21.15.43:0/3014657078 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79301f51d0 0x7f7930158250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.276+0000 7f7936207640 1 --2- 172.21.15.43:0/3014657078 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f791006b4d0 0x7f791006d990 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.276+0000 7f7936207640 1 --2- 172.21.15.43:0/3014657078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f791c0039b0 0x7f79301425e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.276+0000 7f7936207640 1 -- 172.21.15.43:0/3014657078 >> 172.21.15.43:0/3014657078 conn(0x7f793006ab30 msgr2=0x7f79300ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:48.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.276+0000 7f7936207640 1 -- 172.21.15.43:0/3014657078 shutdown_connections 2024-04-25T08:54:48.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.276+0000 7f7936207640 1 -- 172.21.15.43:0/3014657078 wait complete. 2024-04-25T08:54:48.277 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:54:48.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.389+0000 7f85a6d89640 1 Processor -- start 2024-04-25T08:54:48.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.389+0000 7f85a6d89640 1 -- start start 2024-04-25T08:54:48.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.390+0000 7f85a6d89640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f85a0161390 0x7f85a00b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:48.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.390+0000 7f85a6d89640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f85a00d02a0 0x7f85a00d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:48.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.390+0000 7f85a6d89640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f85a00ce5e0 0x7f85a00cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:48.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.390+0000 7f85a6d89640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85a00d1910 con 0x7f85a00ce5e0 2024-04-25T08:54:48.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.390+0000 7f85a6d89640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85a00b4c30 con 0x7f85a0161390 2024-04-25T08:54:48.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.390+0000 7f85a6d89640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85a00cf2c0 con 0x7f85a00d02a0 2024-04-25T08:54:48.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.390+0000 7f85a5326640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f85a00ce5e0 0x7f85a00cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:48.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.390+0000 7f85a5326640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f85a00ce5e0 0x7f85a00cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38408/0 (socket says 172.21.15.43:38408) 2024-04-25T08:54:48.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.390+0000 7f85a5326640 1 -- 172.21.15.43:0/3495434377 learned_addr learned my addr 172.21.15.43:0/3495434377 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:48.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.390+0000 7f8597fff640 1 --2- 172.21.15.43:0/3495434377 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f85a00d02a0 0x7f85a00d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:48.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.390+0000 7f85a4b25640 1 --2- 172.21.15.43:0/3495434377 >> v2:172.21.15.193:3300/0 conn(0x7f85a0161390 0x7f85a00b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:48.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.390+0000 7f85a5326640 1 -- 172.21.15.43:0/3495434377 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f85a00d02a0 msgr2=0x7f85a00d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:48.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.390+0000 7f85a5326640 1 --2- 172.21.15.43:0/3495434377 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f85a00d02a0 0x7f85a00d0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.390+0000 7f85a5326640 1 -- 172.21.15.43:0/3495434377 >> v2:172.21.15.193:3300/0 conn(0x7f85a0161390 msgr2=0x7f85a00b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:48.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.390+0000 7f85a5326640 1 --2- 172.21.15.43:0/3495434377 >> v2:172.21.15.193:3300/0 conn(0x7f85a0161390 0x7f85a00b6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.390+0000 7f85a5326640 1 -- 172.21.15.43:0/3495434377 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a00cf5a0 con 0x7f85a00ce5e0 2024-04-25T08:54:48.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.390+0000 7f85a4b25640 1 --2- 172.21.15.43:0/3495434377 >> v2:172.21.15.193:3300/0 conn(0x7f85a0161390 0x7f85a00b6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:48.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.391+0000 7f85a5326640 1 --2- 172.21.15.43:0/3495434377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a00ce5e0 0x7f85a00cea70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f85a00c74d0 tx=0x7f859802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d91229920a89295a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:48.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.391+0000 7f85977fe640 1 -- 172.21.15.43:0/3495434377 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8598036070 con 0x7f85a00ce5e0 2024-04-25T08:54:48.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.391+0000 7f85977fe640 1 -- 172.21.15.43:0/3495434377 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8598031600 con 0x7f85a00ce5e0 2024-04-25T08:54:48.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.391+0000 7f85977fe640 1 -- 172.21.15.43:0/3495434377 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85980319d0 con 0x7f85a00ce5e0 2024-04-25T08:54:48.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.392+0000 7f85a6d89640 1 -- 172.21.15.43:0/3495434377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a00ce5e0 msgr2=0x7f85a00cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:48.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.392+0000 7f85a6d89640 1 --2- 172.21.15.43:0/3495434377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a00ce5e0 0x7f85a00cea70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f85a00c74d0 tx=0x7f859802a6a0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.392+0000 7f85a6d89640 1 -- 172.21.15.43:0/3495434377 shutdown_connections 2024-04-25T08:54:48.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.392+0000 7f85a6d89640 1 --2- 172.21.15.43:0/3495434377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a00ce5e0 0x7f85a00cea70 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.392+0000 7f85a6d89640 1 --2- 172.21.15.43:0/3495434377 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f85a00d02a0 0x7f85a00d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.392+0000 7f85a6d89640 1 --2- 172.21.15.43:0/3495434377 >> v2:172.21.15.193:3300/0 conn(0x7f85a0161390 0x7f85a00b6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.392+0000 7f85a6d89640 1 -- 172.21.15.43:0/3495434377 >> 172.21.15.43:0/3495434377 conn(0x7f85a009cd20 msgr2=0x7f85a00d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:48.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.392+0000 7f85a6d89640 1 -- 172.21.15.43:0/3495434377 shutdown_connections 2024-04-25T08:54:48.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.392+0000 7f85a6d89640 1 -- 172.21.15.43:0/3495434377 wait complete. 2024-04-25T08:54:48.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.393+0000 7f85a6d89640 1 Processor -- start 2024-04-25T08:54:48.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.393+0000 7f85a6d89640 1 -- start start 2024-04-25T08:54:48.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.394+0000 7f85a6d89640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85a00ce5e0 0x7f85a00c7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:48.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.394+0000 7f85a6d89640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a00d02a0 0x7f85a01f7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:48.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.394+0000 7f85a6d89640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f85a0161390 0x7f85a00cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:48.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.394+0000 7f85a6d89640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a0161770 con 0x7f85a00d02a0 2024-04-25T08:54:48.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.394+0000 7f85a6d89640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a00d15a0 con 0x7f85a00ce5e0 2024-04-25T08:54:48.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.394+0000 7f85a6d89640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85a00cefb0 con 0x7f85a0161390 2024-04-25T08:54:48.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.394+0000 7f8597fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a00d02a0 0x7f85a01f7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:48.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.394+0000 7f8597fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a00d02a0 0x7f85a01f7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38418/0 (socket says 172.21.15.43:38418) 2024-04-25T08:54:48.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.394+0000 7f8597fff640 1 -- 172.21.15.43:0/190325076 learned_addr learned my addr 172.21.15.43:0/190325076 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:48.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.394+0000 7f85a5326640 1 --2- 172.21.15.43:0/190325076 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f85a0161390 0x7f85a00cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:48.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.394+0000 7f85a4b25640 1 --2- 172.21.15.43:0/190325076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85a00ce5e0 0x7f85a00c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:48.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.394+0000 7f8597fff640 1 -- 172.21.15.43:0/190325076 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f85a0161390 msgr2=0x7f85a00cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:48.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.394+0000 7f8597fff640 1 --2- 172.21.15.43:0/190325076 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f85a0161390 0x7f85a00cc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.394+0000 7f8597fff640 1 -- 172.21.15.43:0/190325076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85a00ce5e0 msgr2=0x7f85a00c7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:48.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.394+0000 7f8597fff640 1 --2- 172.21.15.43:0/190325076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85a00ce5e0 0x7f85a00c7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.394+0000 7f8597fff640 1 -- 172.21.15.43:0/190325076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a00c7780 con 0x7f85a00d02a0 2024-04-25T08:54:48.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.394+0000 7f85a5326640 1 --2- 172.21.15.43:0/190325076 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f85a0161390 0x7f85a00cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:48.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.394+0000 7f85a4b25640 1 --2- 172.21.15.43:0/190325076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85a00ce5e0 0x7f85a00c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:48.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.395+0000 7f8597fff640 1 --2- 172.21.15.43:0/190325076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a00d02a0 0x7f85a01f7b00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f859000d830 tx=0x7f859000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:48.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.395+0000 7f85957fa640 1 -- 172.21.15.43:0/190325076 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8590018020 con 0x7f85a00d02a0 2024-04-25T08:54:48.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.395+0000 7f85957fa640 1 -- 172.21.15.43:0/190325076 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85900096a0 con 0x7f85a00d02a0 2024-04-25T08:54:48.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.395+0000 7f85957fa640 1 -- 172.21.15.43:0/190325076 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8590009a70 con 0x7f85a00d02a0 2024-04-25T08:54:48.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.395+0000 7f85a6d89640 1 -- 172.21.15.43:0/190325076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85a00c7a60 con 0x7f85a00d02a0 2024-04-25T08:54:48.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.395+0000 7f85a6d89640 1 -- 172.21.15.43:0/190325076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85a00c8020 con 0x7f85a00d02a0 2024-04-25T08:54:48.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.396+0000 7f85a6d89640 1 -- 172.21.15.43:0/190325076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85a00680f0 con 0x7f85a00d02a0 2024-04-25T08:54:48.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.396+0000 7f85957fa640 1 -- 172.21.15.43:0/190325076 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f859001b020 con 0x7f85a00d02a0 2024-04-25T08:54:48.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.397+0000 7f85957fa640 1 --2- 172.21.15.43:0/190325076 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f858c06b4d0 0x7f858c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:48.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.397+0000 7f85957fa640 1 -- 172.21.15.43:0/190325076 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f859008a6d0 con 0x7f85a00d02a0 2024-04-25T08:54:48.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.397+0000 7f85a4b25640 1 --2- 172.21.15.43:0/190325076 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f858c06b4d0 0x7f858c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:48.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.397+0000 7f85a4b25640 1 --2- 172.21.15.43:0/190325076 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f858c06b4d0 0x7f858c06d990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f859c006fd0 tx=0x7f859c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:48.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.401+0000 7f85957fa640 1 -- 172.21.15.43:0/190325076 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8590022030 con 0x7f85a00d02a0 2024-04-25T08:54:48.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.591+0000 7f85a6d89640 1 -- 172.21.15.43:0/190325076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f85a0068e90 con 0x7f85a00d02a0 2024-04-25T08:54:48.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.591+0000 7f85957fa640 1 -- 172.21.15.43:0/190325076 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f85a0068e90 con 0x7f85a00d02a0 2024-04-25T08:54:48.581 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-04-25T08:54:48.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.593+0000 7f85a6d89640 1 -- 172.21.15.43:0/190325076 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f858c06b4d0 msgr2=0x7f858c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:48.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.593+0000 7f85a6d89640 1 --2- 172.21.15.43:0/190325076 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f858c06b4d0 0x7f858c06d990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f859c006fd0 tx=0x7f859c008040 comp rx=0 tx=0).stop 2024-04-25T08:54:48.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.593+0000 7f85a6d89640 1 -- 172.21.15.43:0/190325076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a00d02a0 msgr2=0x7f85a01f7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:48.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.593+0000 7f85a6d89640 1 --2- 172.21.15.43:0/190325076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a00d02a0 0x7f85a01f7b00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f859000d830 tx=0x7f859000dcf0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.593+0000 7f85a6d89640 1 -- 172.21.15.43:0/190325076 shutdown_connections 2024-04-25T08:54:48.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.593+0000 7f85a6d89640 1 --2- 172.21.15.43:0/190325076 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f85a0161390 0x7f85a00cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.593+0000 7f85a6d89640 1 --2- 172.21.15.43:0/190325076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85a00d02a0 0x7f85a01f7b00 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.593+0000 7f85a6d89640 1 --2- 172.21.15.43:0/190325076 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f858c06b4d0 0x7f858c06d990 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.593+0000 7f85a6d89640 1 --2- 172.21.15.43:0/190325076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85a00ce5e0 0x7f85a00c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.593+0000 7f85a6d89640 1 -- 172.21.15.43:0/190325076 >> 172.21.15.43:0/190325076 conn(0x7f85a009cd20 msgr2=0x7f85a01606a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:48.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.593+0000 7f85a6d89640 1 -- 172.21.15.43:0/190325076 shutdown_connections 2024-04-25T08:54:48.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.593+0000 7f85a6d89640 1 -- 172.21.15.43:0/190325076 wait complete. 2024-04-25T08:54:48.594 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:54:48.595 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:54:48.610 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:54:48.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.762+0000 7fb204fd5640 1 Processor -- start 2024-04-25T08:54:48.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.762+0000 7fb204fd5640 1 -- start start 2024-04-25T08:54:48.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.763+0000 7fb204fd5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb20015dcf0 0x7fb20006e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:48.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.763+0000 7fb204fd5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb20006ebf0 0x7fb20006f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:48.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.763+0000 7fb204fd5640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb2001616e0 0x7fb200161b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:48.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.763+0000 7fb204fd5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb20006f590 con 0x7fb20015dcf0 2024-04-25T08:54:48.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.763+0000 7fb204fd5640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb20006c050 con 0x7fb2001616e0 2024-04-25T08:54:48.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.763+0000 7fb204fd5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2000d0d80 con 0x7fb20006ebf0 2024-04-25T08:54:48.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.763+0000 7fb1fe59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb20015dcf0 0x7fb20006e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:48.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.763+0000 7fb1fe59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb20015dcf0 0x7fb20006e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38424/0 (socket says 172.21.15.43:38424) 2024-04-25T08:54:48.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.763+0000 7fb1fe59c640 1 -- 172.21.15.43:0/1895889908 learned_addr learned my addr 172.21.15.43:0/1895889908 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:48.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.763+0000 7fb1fed9d640 1 --2- 172.21.15.43:0/1895889908 >> v2:172.21.15.193:3300/0 conn(0x7fb2001616e0 0x7fb200161b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:48.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.763+0000 7fb1fdd9b640 1 --2- 172.21.15.43:0/1895889908 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb20006ebf0 0x7fb20006f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:48.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.763+0000 7fb1fe59c640 1 -- 172.21.15.43:0/1895889908 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb20006ebf0 msgr2=0x7fb20006f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:48.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.763+0000 7fb1fe59c640 1 --2- 172.21.15.43:0/1895889908 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb20006ebf0 0x7fb20006f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.763+0000 7fb1fe59c640 1 -- 172.21.15.43:0/1895889908 >> v2:172.21.15.193:3300/0 conn(0x7fb2001616e0 msgr2=0x7fb200161b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:48.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.763+0000 7fb1fe59c640 1 --2- 172.21.15.43:0/1895889908 >> v2:172.21.15.193:3300/0 conn(0x7fb2001616e0 0x7fb200161b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.763+0000 7fb1fe59c640 1 -- 172.21.15.43:0/1895889908 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2000d04e0 con 0x7fb20015dcf0 2024-04-25T08:54:48.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.763+0000 7fb1fed9d640 1 --2- 172.21.15.43:0/1895889908 >> v2:172.21.15.193:3300/0 conn(0x7fb2001616e0 0x7fb200161b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:48.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.763+0000 7fb1fdd9b640 1 --2- 172.21.15.43:0/1895889908 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb20006ebf0 0x7fb20006f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:48.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.764+0000 7fb1fe59c640 1 --2- 172.21.15.43:0/1895889908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb20015dcf0 0x7fb20006e530 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb2000b4230 tx=0x7fb1f002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c90c378a5df74c40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:48.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.764+0000 7fb1fd59a640 1 -- 172.21.15.43:0/1895889908 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f0037070 con 0x7fb20015dcf0 2024-04-25T08:54:48.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.764+0000 7fb1fd59a640 1 -- 172.21.15.43:0/1895889908 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f0032440 con 0x7fb20015dcf0 2024-04-25T08:54:48.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.764+0000 7fb1fd59a640 1 -- 172.21.15.43:0/1895889908 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f0032810 con 0x7fb20015dcf0 2024-04-25T08:54:48.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.764+0000 7fb204fd5640 1 -- 172.21.15.43:0/1895889908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb20015dcf0 msgr2=0x7fb20006e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:48.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.764+0000 7fb204fd5640 1 --2- 172.21.15.43:0/1895889908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb20015dcf0 0x7fb20006e530 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb2000b4230 tx=0x7fb1f002a140 comp rx=0 tx=0).stop 2024-04-25T08:54:48.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.765+0000 7fb204fd5640 1 -- 172.21.15.43:0/1895889908 shutdown_connections 2024-04-25T08:54:48.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.765+0000 7fb204fd5640 1 --2- 172.21.15.43:0/1895889908 >> v2:172.21.15.193:3300/0 conn(0x7fb2001616e0 0x7fb200161b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.765+0000 7fb204fd5640 1 --2- 172.21.15.43:0/1895889908 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb20006ebf0 0x7fb20006f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.765+0000 7fb204fd5640 1 --2- 172.21.15.43:0/1895889908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb20015dcf0 0x7fb20006e530 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.765+0000 7fb204fd5640 1 -- 172.21.15.43:0/1895889908 >> 172.21.15.43:0/1895889908 conn(0x7fb2000d1700 msgr2=0x7fb2000d1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:48.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.765+0000 7fb204fd5640 1 -- 172.21.15.43:0/1895889908 shutdown_connections 2024-04-25T08:54:48.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.765+0000 7fb204fd5640 1 -- 172.21.15.43:0/1895889908 wait complete. 2024-04-25T08:54:48.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.766+0000 7fb204fd5640 1 Processor -- start 2024-04-25T08:54:48.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.766+0000 7fb204fd5640 1 -- start start 2024-04-25T08:54:48.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.767+0000 7fb204fd5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb20006ebf0 0x7fb2001f4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:48.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.767+0000 7fb204fd5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb20015dcf0 0x7fb2001f0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:48.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.767+0000 7fb204fd5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2001616e0 0x7fb2000c54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:48.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.767+0000 7fb204fd5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20006bce0 con 0x7fb20006ebf0 2024-04-25T08:54:48.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.767+0000 7fb204fd5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2000d0aa0 con 0x7fb2001616e0 2024-04-25T08:54:48.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.767+0000 7fb204fd5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2000b0d50 con 0x7fb20015dcf0 2024-04-25T08:54:48.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.767+0000 7fb1fe59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb20006ebf0 0x7fb2001f4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:48.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.767+0000 7fb1fe59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb20006ebf0 0x7fb2001f4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38440/0 (socket says 172.21.15.43:38440) 2024-04-25T08:54:48.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.767+0000 7fb1fe59c640 1 -- 172.21.15.43:0/434172658 learned_addr learned my addr 172.21.15.43:0/434172658 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:48.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.767+0000 7fb1fdd9b640 1 --2- 172.21.15.43:0/434172658 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb20015dcf0 0x7fb2001f0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:48.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.767+0000 7fb1fed9d640 1 --2- 172.21.15.43:0/434172658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2001616e0 0x7fb2000c54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:48.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.767+0000 7fb1fe59c640 1 -- 172.21.15.43:0/434172658 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb20015dcf0 msgr2=0x7fb2001f0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:48.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.767+0000 7fb1fe59c640 1 --2- 172.21.15.43:0/434172658 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb20015dcf0 0x7fb2001f0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.767+0000 7fb1fe59c640 1 -- 172.21.15.43:0/434172658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2001616e0 msgr2=0x7fb2000c54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:48.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.767+0000 7fb1fe59c640 1 --2- 172.21.15.43:0/434172658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2001616e0 0x7fb2000c54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.767+0000 7fb1fe59c640 1 -- 172.21.15.43:0/434172658 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2000b5130 con 0x7fb20006ebf0 2024-04-25T08:54:48.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.767+0000 7fb1fed9d640 1 --2- 172.21.15.43:0/434172658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2001616e0 0x7fb2000c54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:48.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.767+0000 7fb1fe59c640 1 --2- 172.21.15.43:0/434172658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb20006ebf0 0x7fb2001f4c70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb1f002a180 tx=0x7fb1f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:48.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.767+0000 7fb1eeffd640 1 -- 172.21.15.43:0/434172658 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f0037040 con 0x7fb20006ebf0 2024-04-25T08:54:48.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.767+0000 7fb1eeffd640 1 -- 172.21.15.43:0/434172658 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f0042070 con 0x7fb20006ebf0 2024-04-25T08:54:48.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.767+0000 7fb1eeffd640 1 -- 172.21.15.43:0/434172658 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f0030bb0 con 0x7fb20006ebf0 2024-04-25T08:54:48.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.767+0000 7fb204fd5640 1 -- 172.21.15.43:0/434172658 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2000b5410 con 0x7fb20006ebf0 2024-04-25T08:54:48.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.768+0000 7fb204fd5640 1 -- 172.21.15.43:0/434172658 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2000b59d0 con 0x7fb20006ebf0 2024-04-25T08:54:48.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.769+0000 7fb204fd5640 1 -- 172.21.15.43:0/434172658 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb200074ef0 con 0x7fb20006ebf0 2024-04-25T08:54:48.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.769+0000 7fb1eeffd640 1 -- 172.21.15.43:0/434172658 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb1f0047460 con 0x7fb20006ebf0 2024-04-25T08:54:48.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.769+0000 7fb1eeffd640 1 --2- 172.21.15.43:0/434172658 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb1e006b500 0x7fb1e006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:48.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.770+0000 7fb1eeffd640 1 -- 172.21.15.43:0/434172658 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb1f00aad70 con 0x7fb20006ebf0 2024-04-25T08:54:48.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.770+0000 7fb1fdd9b640 1 --2- 172.21.15.43:0/434172658 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb1e006b500 0x7fb1e006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:48.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.770+0000 7fb1fdd9b640 1 --2- 172.21.15.43:0/434172658 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb1e006b500 0x7fb1e006d9c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb2000b7170 tx=0x7fb1f4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:48.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.774+0000 7fb1eeffd640 1 -- 172.21.15.43:0/434172658 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb1f0045360 con 0x7fb20006ebf0 2024-04-25T08:54:48.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.959+0000 7fb204fd5640 1 -- 172.21.15.43:0/434172658 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2000b7680 con 0x7fb20006ebf0 2024-04-25T08:54:48.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.960+0000 7fb1eeffd640 1 -- 172.21.15.43:0/434172658 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16569 (secure 0 0 0) 0x7fb1f0079c80 con 0x7fb20006ebf0 2024-04-25T08:54:48.951 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:54:48.951 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":20,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:54:13.593188+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:54:09.757978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:54:10.926008+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:54:48.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.962+0000 7fb204fd5640 1 -- 172.21.15.43:0/434172658 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb1e006b500 msgr2=0x7fb1e006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:48.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.962+0000 7fb204fd5640 1 --2- 172.21.15.43:0/434172658 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb1e006b500 0x7fb1e006d9c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb2000b7170 tx=0x7fb1f4008040 comp rx=0 tx=0).stop 2024-04-25T08:54:48.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.962+0000 7fb204fd5640 1 -- 172.21.15.43:0/434172658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb20006ebf0 msgr2=0x7fb2001f4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:48.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.962+0000 7fb204fd5640 1 --2- 172.21.15.43:0/434172658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb20006ebf0 0x7fb2001f4c70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb1f002a180 tx=0x7fb1f00046c0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.962+0000 7fb204fd5640 1 -- 172.21.15.43:0/434172658 shutdown_connections 2024-04-25T08:54:48.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.962+0000 7fb204fd5640 1 --2- 172.21.15.43:0/434172658 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb1e006b500 0x7fb1e006d9c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.962+0000 7fb204fd5640 1 --2- 172.21.15.43:0/434172658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2001616e0 0x7fb2000c54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.962+0000 7fb204fd5640 1 --2- 172.21.15.43:0/434172658 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb20015dcf0 0x7fb2001f0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.962+0000 7fb204fd5640 1 --2- 172.21.15.43:0/434172658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb20006ebf0 0x7fb2001f4c70 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:48.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.962+0000 7fb204fd5640 1 -- 172.21.15.43:0/434172658 >> 172.21.15.43:0/434172658 conn(0x7fb2000d1700 msgr2=0x7fb2000aece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:48.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.962+0000 7fb204fd5640 1 -- 172.21.15.43:0/434172658 shutdown_connections 2024-04-25T08:54:48.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:48.962+0000 7fb204fd5640 1 -- 172.21.15.43:0/434172658 wait complete. 2024-04-25T08:54:48.963 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:54:09.757978+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:54:10.926008+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:54:48.964 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:54:49.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.078+0000 7f3becd1d640 1 Processor -- start 2024-04-25T08:54:49.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.078+0000 7f3becd1d640 1 -- start start 2024-04-25T08:54:49.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.079+0000 7f3becd1d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3be815dd80 0x7f3be80cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:49.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.079+0000 7f3becd1d640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f3be80b1cd0 0x7f3be80d1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:49.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.079+0000 7f3becd1d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3be80b2fe0 con 0x7f3be80d17b0 2024-04-25T08:54:49.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.079+0000 7f3becd1d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3be80cfc60 con 0x7f3be815dd80 2024-04-25T08:54:49.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.079+0000 7f3becd1d640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3be80b32c0 con 0x7f3be80b1cd0 2024-04-25T08:54:49.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.079+0000 7f3be659c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3be815dd80 0x7f3be80cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:49.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.079+0000 7f3be5d9b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f3be80d17b0 0x7f3be815e160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39730/0 (socket says 172.21.15.43:39730) 2024-04-25T08:54:49.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.079+0000 7f3be659c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3be815dd80 0x7f3be80cf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38450/0 (socket says 172.21.15.43:38450) 2024-04-25T08:54:49.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.079+0000 7f3be5d9b640 1 -- 172.21.15.43:0/2549910006 learned_addr learned my addr 172.21.15.43:0/2549910006 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:49.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.080+0000 7f3be6d9d640 1 --2- 172.21.15.43:0/2549910006 >> v2:172.21.15.193:3300/0 conn(0x7f3be80b1cd0 0x7f3be80d1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:49.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.080+0000 7f3be659c640 1 -- 172.21.15.43:0/2549910006 >> v2:172.21.15.193:3300/0 conn(0x7f3be80b1cd0 msgr2=0x7f3be80d1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:49.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.080+0000 7f3be659c640 1 --2- 172.21.15.43:0/2549910006 >> v2:172.21.15.193:3300/0 conn(0x7f3be80b1cd0 0x7f3be80d1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:49.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.080+0000 7f3be659c640 1 -- 172.21.15.43:0/2549910006 >> v1:172.21.15.43:6789/0 conn(0x7f3be80d17b0 legacy=0x7f3be815e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:49.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.080+0000 7f3be659c640 1 -- 172.21.15.43:0/2549910006 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be80ced70 con 0x7f3be815dd80 2024-04-25T08:54:49.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.080+0000 7f3be659c640 1 --2- 172.21.15.43:0/2549910006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3be815dd80 0x7f3be80cf720 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3be806e510 tx=0x7f3be002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a939f96f96cf23 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:49.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.081+0000 7f3be559a640 1 -- 172.21.15.43:0/2549910006 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be0037070 con 0x7f3be815dd80 2024-04-25T08:54:49.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.081+0000 7f3be559a640 1 -- 172.21.15.43:0/2549910006 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be0032440 con 0x7f3be815dd80 2024-04-25T08:54:49.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.081+0000 7f3be559a640 1 -- 172.21.15.43:0/2549910006 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be002ae60 con 0x7f3be815dd80 2024-04-25T08:54:49.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.081+0000 7f3becd1d640 1 -- 172.21.15.43:0/2549910006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3be815dd80 msgr2=0x7f3be80cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:49.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.081+0000 7f3becd1d640 1 --2- 172.21.15.43:0/2549910006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3be815dd80 0x7f3be80cf720 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3be806e510 tx=0x7f3be002a110 comp rx=0 tx=0).stop 2024-04-25T08:54:49.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.081+0000 7f3becd1d640 1 -- 172.21.15.43:0/2549910006 shutdown_connections 2024-04-25T08:54:49.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.081+0000 7f3becd1d640 1 --2- 172.21.15.43:0/2549910006 >> v2:172.21.15.193:3300/0 conn(0x7f3be80b1cd0 0x7f3be80d1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:49.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.081+0000 7f3becd1d640 1 --2- 172.21.15.43:0/2549910006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3be815dd80 0x7f3be80cf720 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:49.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.081+0000 7f3becd1d640 1 -- 172.21.15.43:0/2549910006 >> 172.21.15.43:0/2549910006 conn(0x7f3be8072aa0 msgr2=0x7f3be809aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:49.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.081+0000 7f3becd1d640 1 -- 172.21.15.43:0/2549910006 shutdown_connections 2024-04-25T08:54:49.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.081+0000 7f3becd1d640 1 -- 172.21.15.43:0/2549910006 wait complete. 2024-04-25T08:54:49.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.083+0000 7f3becd1d640 1 Processor -- start 2024-04-25T08:54:49.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.083+0000 7f3becd1d640 1 -- start start 2024-04-25T08:54:49.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.083+0000 7f3becd1d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3be80b1cd0 0x7f3be8160870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:49.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.083+0000 7f3becd1d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3be80d17b0 0x7f3be81532c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:49.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.083+0000 7f3becd1d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3be815dd80 0x7f3be806edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:49.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.083+0000 7f3becd1d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be80b2c40 con 0x7f3be80d17b0 2024-04-25T08:54:49.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.083+0000 7f3becd1d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be80cf330 con 0x7f3be80b1cd0 2024-04-25T08:54:49.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.083+0000 7f3becd1d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3be80b2dc0 con 0x7f3be815dd80 2024-04-25T08:54:49.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.083+0000 7f3be5d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3be80d17b0 0x7f3be81532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:49.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.084+0000 7f3be5d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3be80d17b0 0x7f3be81532c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38462/0 (socket says 172.21.15.43:38462) 2024-04-25T08:54:49.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.084+0000 7f3be5d9b640 1 -- 172.21.15.43:0/2974593734 learned_addr learned my addr 172.21.15.43:0/2974593734 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:49.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.084+0000 7f3be659c640 1 --2- 172.21.15.43:0/2974593734 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3be80b1cd0 0x7f3be8160870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:49.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.084+0000 7f3be6d9d640 1 --2- 172.21.15.43:0/2974593734 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3be815dd80 0x7f3be806edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:49.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.084+0000 7f3be5d9b640 1 -- 172.21.15.43:0/2974593734 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3be815dd80 msgr2=0x7f3be806edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:49.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.084+0000 7f3be5d9b640 1 --2- 172.21.15.43:0/2974593734 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3be815dd80 0x7f3be806edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:49.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.084+0000 7f3be5d9b640 1 -- 172.21.15.43:0/2974593734 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3be80b1cd0 msgr2=0x7f3be8160870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:49.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.084+0000 7f3be5d9b640 1 --2- 172.21.15.43:0/2974593734 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3be80b1cd0 0x7f3be8160870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:49.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.084+0000 7f3be5d9b640 1 -- 172.21.15.43:0/2974593734 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be81611a0 con 0x7f3be80d17b0 2024-04-25T08:54:49.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.084+0000 7f3be5d9b640 1 --2- 172.21.15.43:0/2974593734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3be80d17b0 0x7f3be81532c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3bd000c4a0 tx=0x7f3bd000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:49.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.084+0000 7f3bdeffd640 1 -- 172.21.15.43:0/2974593734 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0017070 con 0x7f3be80d17b0 2024-04-25T08:54:49.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.084+0000 7f3bdeffd640 1 -- 172.21.15.43:0/2974593734 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bd0005ce0 con 0x7f3be80d17b0 2024-04-25T08:54:49.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.084+0000 7f3becd1d640 1 -- 172.21.15.43:0/2974593734 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3be80cae60 con 0x7f3be80d17b0 2024-04-25T08:54:49.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.084+0000 7f3bdeffd640 1 -- 172.21.15.43:0/2974593734 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd0007270 con 0x7f3be80d17b0 2024-04-25T08:54:49.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.084+0000 7f3becd1d640 1 -- 172.21.15.43:0/2974593734 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3be80cb420 con 0x7f3be80d17b0 2024-04-25T08:54:49.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.085+0000 7f3becd1d640 1 -- 172.21.15.43:0/2974593734 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3be8074f60 con 0x7f3be80d17b0 2024-04-25T08:54:49.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.086+0000 7f3bdeffd640 1 -- 172.21.15.43:0/2974593734 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3bd001a070 con 0x7f3be80d17b0 2024-04-25T08:54:49.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.086+0000 7f3bdeffd640 1 --2- 172.21.15.43:0/2974593734 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3bc806b4d0 0x7f3bc806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:49.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.086+0000 7f3bdeffd640 1 -- 172.21.15.43:0/2974593734 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3bd0089400 con 0x7f3be80d17b0 2024-04-25T08:54:49.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.087+0000 7f3be659c640 1 --2- 172.21.15.43:0/2974593734 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3bc806b4d0 0x7f3bc806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:49.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.087+0000 7f3be659c640 1 --2- 172.21.15.43:0/2974593734 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3bc806b4d0 0x7f3bc806d990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3be0034000 tx=0x7f3be002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:49.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.091+0000 7f3bdeffd640 1 -- 172.21.15.43:0/2974593734 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3bd0057cc0 con 0x7f3be80d17b0 2024-04-25T08:54:49.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.270+0000 7f3becd1d640 1 -- 172.21.15.43:0/2974593734 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3be80b1790 con 0x7f3be80d17b0 2024-04-25T08:54:49.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.270+0000 7f3bdeffd640 1 -- 172.21.15.43:0/2974593734 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3bd005bd60 con 0x7f3be80d17b0 2024-04-25T08:54:49.260 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T08:54:49.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.272+0000 7f3becd1d640 1 -- 172.21.15.43:0/2974593734 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3bc806b4d0 msgr2=0x7f3bc806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:49.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.272+0000 7f3becd1d640 1 --2- 172.21.15.43:0/2974593734 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3bc806b4d0 0x7f3bc806d990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3be0034000 tx=0x7f3be002e000 comp rx=0 tx=0).stop 2024-04-25T08:54:49.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.272+0000 7f3becd1d640 1 -- 172.21.15.43:0/2974593734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3be80d17b0 msgr2=0x7f3be81532c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:49.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.272+0000 7f3becd1d640 1 --2- 172.21.15.43:0/2974593734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3be80d17b0 0x7f3be81532c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f3bd000c4a0 tx=0x7f3bd000c960 comp rx=0 tx=0).stop 2024-04-25T08:54:49.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.272+0000 7f3becd1d640 1 -- 172.21.15.43:0/2974593734 shutdown_connections 2024-04-25T08:54:49.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.272+0000 7f3becd1d640 1 --2- 172.21.15.43:0/2974593734 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3be815dd80 0x7f3be806edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:49.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.272+0000 7f3becd1d640 1 --2- 172.21.15.43:0/2974593734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3be80d17b0 0x7f3be81532c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:49.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.272+0000 7f3becd1d640 1 --2- 172.21.15.43:0/2974593734 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3bc806b4d0 0x7f3bc806d990 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:49.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.272+0000 7f3becd1d640 1 --2- 172.21.15.43:0/2974593734 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3be80b1cd0 0x7f3be8160870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:49.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.272+0000 7f3becd1d640 1 -- 172.21.15.43:0/2974593734 >> 172.21.15.43:0/2974593734 conn(0x7f3be8072aa0 msgr2=0x7f3be80b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:49.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.272+0000 7f3becd1d640 1 -- 172.21.15.43:0/2974593734 shutdown_connections 2024-04-25T08:54:49.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.272+0000 7f3becd1d640 1 -- 172.21.15.43:0/2974593734 wait complete. 2024-04-25T08:54:49.273 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:54:49.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.386+0000 7f460901f640 1 Processor -- start 2024-04-25T08:54:49.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.386+0000 7f460901f640 1 -- start start 2024-04-25T08:54:49.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.387+0000 7f460901f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f460415ddd0 0x7f46040b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:49.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.387+0000 7f460901f640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f460406ef90 0x7f460406f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:49.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.387+0000 7f460901f640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46040d2620 con 0x7f46040cc460 2024-04-25T08:54:49.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.387+0000 7f460901f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46040b0b80 con 0x7f460415ddd0 2024-04-25T08:54:49.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.387+0000 7f460901f640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46040b0e60 con 0x7f460406ef90 2024-04-25T08:54:49.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.387+0000 7f4602d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f460415ddd0 0x7f46040b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:49.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.387+0000 7f4602d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f460415ddd0 0x7f46040b3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38472/0 (socket says 172.21.15.43:38472) 2024-04-25T08:54:49.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.388+0000 7f4602d9d640 1 -- 172.21.15.43:0/3828553800 learned_addr learned my addr 172.21.15.43:0/3828553800 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:49.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.388+0000 7f460259c640 1 --2- 172.21.15.43:0/3828553800 >> v2:172.21.15.193:3300/0 conn(0x7f460406ef90 0x7f460406f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:49.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.388+0000 7f4602d9d640 1 -- 172.21.15.43:0/3828553800 >> v2:172.21.15.193:3300/0 conn(0x7f460406ef90 msgr2=0x7f460406f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:49.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.388+0000 7f4602d9d640 1 --2- 172.21.15.43:0/3828553800 >> v2:172.21.15.193:3300/0 conn(0x7f460406ef90 0x7f460406f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:49.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.388+0000 7f4602d9d640 1 -- 172.21.15.43:0/3828553800 >> v1:172.21.15.193:6789/0 conn(0x7f46040cc460 legacy=0x7f46040cc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:49.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.388+0000 7f4602d9d640 1 -- 172.21.15.43:0/3828553800 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46040cbf40 con 0x7f460415ddd0 2024-04-25T08:54:49.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.389+0000 7f4602d9d640 1 --2- 172.21.15.43:0/3828553800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f460415ddd0 0x7f46040b3dd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f46040cdd70 tx=0x7f45f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4206000d27c3f514 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:49.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.389+0000 7f4601d9b640 1 -- 172.21.15.43:0/3828553800 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f4037070 con 0x7f460415ddd0 2024-04-25T08:54:49.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.389+0000 7f4601d9b640 1 -- 172.21.15.43:0/3828553800 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f4032440 con 0x7f460415ddd0 2024-04-25T08:54:49.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.389+0000 7f4601d9b640 1 -- 172.21.15.43:0/3828553800 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f4032810 con 0x7f460415ddd0 2024-04-25T08:54:49.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.389+0000 7f460901f640 1 -- 172.21.15.43:0/3828553800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f460415ddd0 msgr2=0x7f46040b3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:49.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.389+0000 7f460901f640 1 --2- 172.21.15.43:0/3828553800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f460415ddd0 0x7f46040b3dd0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f46040cdd70 tx=0x7f45f402a270 comp rx=0 tx=0).stop 2024-04-25T08:54:49.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.389+0000 7f460901f640 1 -- 172.21.15.43:0/3828553800 shutdown_connections 2024-04-25T08:54:49.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.389+0000 7f460901f640 1 --2- 172.21.15.43:0/3828553800 >> v2:172.21.15.193:3300/0 conn(0x7f460406ef90 0x7f460406f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:49.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.389+0000 7f460901f640 1 --2- 172.21.15.43:0/3828553800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f460415ddd0 0x7f46040b3dd0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:49.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.389+0000 7f460901f640 1 -- 172.21.15.43:0/3828553800 >> 172.21.15.43:0/3828553800 conn(0x7f46040d17e0 msgr2=0x7f46040d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:49.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.389+0000 7f460901f640 1 -- 172.21.15.43:0/3828553800 shutdown_connections 2024-04-25T08:54:49.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.389+0000 7f460901f640 1 -- 172.21.15.43:0/3828553800 wait complete. 2024-04-25T08:54:49.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.391+0000 7f460901f640 1 Processor -- start 2024-04-25T08:54:49.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.391+0000 7f460901f640 1 -- start start 2024-04-25T08:54:49.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.391+0000 7f460901f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f460406ef90 0x7f46040b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:49.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.391+0000 7f460901f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46040cc460 0x7f46040cb160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:49.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.391+0000 7f460901f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f460415ddd0 0x7f4604151850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:49.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.391+0000 7f460901f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460406d880 con 0x7f46040cc460 2024-04-25T08:54:49.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.391+0000 7f460901f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460406d700 con 0x7f460415ddd0 2024-04-25T08:54:49.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.392+0000 7f460901f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46040ccf80 con 0x7f460406ef90 2024-04-25T08:54:49.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.392+0000 7f460259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46040cc460 0x7f46040cb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:49.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.392+0000 7f460259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46040cc460 0x7f46040cb160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38486/0 (socket says 172.21.15.43:38486) 2024-04-25T08:54:49.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.392+0000 7f460259c640 1 -- 172.21.15.43:0/2745257766 learned_addr learned my addr 172.21.15.43:0/2745257766 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:49.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.392+0000 7f4602d9d640 1 --2- 172.21.15.43:0/2745257766 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f460406ef90 0x7f46040b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:49.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.392+0000 7f460359e640 1 --2- 172.21.15.43:0/2745257766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f460415ddd0 0x7f4604151850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:49.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.392+0000 7f460259c640 1 -- 172.21.15.43:0/2745257766 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f460406ef90 msgr2=0x7f46040b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:49.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.392+0000 7f460259c640 1 --2- 172.21.15.43:0/2745257766 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f460406ef90 0x7f46040b69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:49.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.392+0000 7f460259c640 1 -- 172.21.15.43:0/2745257766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f460415ddd0 msgr2=0x7f4604151850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:49.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.392+0000 7f460259c640 1 --2- 172.21.15.43:0/2745257766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f460415ddd0 0x7f4604151850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:49.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.392+0000 7f460259c640 1 -- 172.21.15.43:0/2745257766 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4604151d90 con 0x7f46040cc460 2024-04-25T08:54:49.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.392+0000 7f460359e640 1 --2- 172.21.15.43:0/2745257766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f460415ddd0 0x7f4604151850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:49.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.392+0000 7f4602d9d640 1 --2- 172.21.15.43:0/2745257766 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f460406ef90 0x7f46040b69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:49.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.392+0000 7f460259c640 1 --2- 172.21.15.43:0/2745257766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46040cc460 0x7f46040cb160 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f45f800e9b0 tx=0x7f45f800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:49.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.392+0000 7f45eb7fe640 1 -- 172.21.15.43:0/2745257766 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f8017070 con 0x7f46040cc460 2024-04-25T08:54:49.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.392+0000 7f45eb7fe640 1 -- 172.21.15.43:0/2745257766 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f800c9a0 con 0x7f46040cc460 2024-04-25T08:54:49.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.392+0000 7f45eb7fe640 1 -- 172.21.15.43:0/2745257766 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f800cd90 con 0x7f46040cc460 2024-04-25T08:54:49.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.392+0000 7f460901f640 1 -- 172.21.15.43:0/2745257766 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46040b73c0 con 0x7f46040cc460 2024-04-25T08:54:49.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.392+0000 7f460901f640 1 -- 172.21.15.43:0/2745257766 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46040c88c0 con 0x7f46040cc460 2024-04-25T08:54:49.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.393+0000 7f460901f640 1 -- 172.21.15.43:0/2745257766 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4604074f60 con 0x7f46040cc460 2024-04-25T08:54:49.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.394+0000 7f45eb7fe640 1 -- 172.21.15.43:0/2745257766 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f45f801a070 con 0x7f46040cc460 2024-04-25T08:54:49.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.394+0000 7f45eb7fe640 1 --2- 172.21.15.43:0/2745257766 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f45f006b4d0 0x7f45f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:49.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.394+0000 7f45eb7fe640 1 -- 172.21.15.43:0/2745257766 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f45f808a1f0 con 0x7f46040cc460 2024-04-25T08:54:49.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.394+0000 7f4602d9d640 1 --2- 172.21.15.43:0/2745257766 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f45f006b4d0 0x7f45f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:49.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.395+0000 7f4602d9d640 1 --2- 172.21.15.43:0/2745257766 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f45f006b4d0 0x7f45f006d990 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f45f4034000 tx=0x7f45f402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:49.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.399+0000 7f45eb7fe640 1 -- 172.21.15.43:0/2745257766 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4604151d90 con 0x7f46040cc460 2024-04-25T08:54:49.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.576+0000 7f460901f640 1 -- 172.21.15.43:0/2745257766 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f46040cb850 con 0x7f46040cc460 2024-04-25T08:54:49.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.576+0000 7f45eb7fe640 1 -- 172.21.15.43:0/2745257766 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f45f8059070 con 0x7f46040cc460 2024-04-25T08:54:49.566 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T08:54:49.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.578+0000 7f460901f640 1 -- 172.21.15.43:0/2745257766 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f45f006b4d0 msgr2=0x7f45f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:49.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.578+0000 7f460901f640 1 --2- 172.21.15.43:0/2745257766 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f45f006b4d0 0x7f45f006d990 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f45f4034000 tx=0x7f45f402e000 comp rx=0 tx=0).stop 2024-04-25T08:54:49.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.578+0000 7f460901f640 1 -- 172.21.15.43:0/2745257766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46040cc460 msgr2=0x7f46040cb160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:49.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.578+0000 7f460901f640 1 --2- 172.21.15.43:0/2745257766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46040cc460 0x7f46040cb160 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f45f800e9b0 tx=0x7f45f800ee70 comp rx=0 tx=0).stop 2024-04-25T08:54:49.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.578+0000 7f460901f640 1 -- 172.21.15.43:0/2745257766 shutdown_connections 2024-04-25T08:54:49.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.578+0000 7f460901f640 1 --2- 172.21.15.43:0/2745257766 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f45f006b4d0 0x7f45f006d990 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:49.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.578+0000 7f460901f640 1 --2- 172.21.15.43:0/2745257766 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f460415ddd0 0x7f4604151850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:49.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.578+0000 7f460901f640 1 --2- 172.21.15.43:0/2745257766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46040cc460 0x7f46040cb160 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:49.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.578+0000 7f460901f640 1 --2- 172.21.15.43:0/2745257766 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f460406ef90 0x7f46040b69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:49.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.578+0000 7f460901f640 1 -- 172.21.15.43:0/2745257766 >> 172.21.15.43:0/2745257766 conn(0x7f46040d17e0 msgr2=0x7f46040c7040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:49.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.579+0000 7f460901f640 1 -- 172.21.15.43:0/2745257766 shutdown_connections 2024-04-25T08:54:49.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.579+0000 7f460901f640 1 -- 172.21.15.43:0/2745257766 wait complete. 2024-04-25T08:54:49.580 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:54:49.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.694+0000 7fd652a74640 1 Processor -- start 2024-04-25T08:54:49.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.694+0000 7fd652a74640 1 -- start start 2024-04-25T08:54:49.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.695+0000 7fd652a74640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd64c0b5560 0x7fd64c0b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:49.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.695+0000 7fd652a74640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd64c162610 con 0x7fd64c0d2230 2024-04-25T08:54:49.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.695+0000 7fd652a74640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd64c0d1750 con 0x7fd64c15e090 2024-04-25T08:54:49.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.695+0000 7fd652a74640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd64c0d1da0 con 0x7fd64c0b5560 2024-04-25T08:54:49.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.695+0000 7fd64bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd64c0b5560 0x7fd64c0b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:49.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.695+0000 7fd64bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd64c0b5560 0x7fd64c0b59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49514/0 (socket says 172.21.15.43:49514) 2024-04-25T08:54:49.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.695+0000 7fd651011640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fd64c0d2230 0x7fd64c0d26c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48324/0 (socket says 172.21.15.43:48324) 2024-04-25T08:54:49.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.695+0000 7fd64bfff640 1 -- 172.21.15.43:0/1384177661 learned_addr learned my addr 172.21.15.43:0/1384177661 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:49.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.695+0000 7fd64bfff640 1 -- 172.21.15.43:0/1384177661 >> v1:172.21.15.193:6789/0 conn(0x7fd64c15e090 legacy=0x7fd64c161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:49.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.695+0000 7fd64bfff640 1 -- 172.21.15.43:0/1384177661 >> v1:172.21.15.43:6789/0 conn(0x7fd64c0d2230 legacy=0x7fd64c0d26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:49.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.695+0000 7fd64bfff640 1 -- 172.21.15.43:0/1384177661 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd64c0d1a30 con 0x7fd64c0b5560 2024-04-25T08:54:49.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.695+0000 7fd64b7fe640 1 -- 172.21.15.43:0/1384177661 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3935461537 0 0) 0x7fd64c162610 con 0x7fd64c0d2230 2024-04-25T08:54:49.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.696+0000 7fd64bfff640 1 --2- 172.21.15.43:0/1384177661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd64c0b5560 0x7fd64c0b59e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd638009870 tx=0x7fd63802a3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57c2784361c2aea1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:49.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.696+0000 7fd64b7fe640 1 -- 172.21.15.43:0/1384177661 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd638037070 con 0x7fd64c0b5560 2024-04-25T08:54:49.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.696+0000 7fd64b7fe640 1 -- 172.21.15.43:0/1384177661 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd638032640 con 0x7fd64c0b5560 2024-04-25T08:54:49.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.696+0000 7fd64b7fe640 1 -- 172.21.15.43:0/1384177661 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd638032a10 con 0x7fd64c0b5560 2024-04-25T08:54:49.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.697+0000 7fd652a74640 1 -- 172.21.15.43:0/1384177661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd64c0b5560 msgr2=0x7fd64c0b59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:49.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.697+0000 7fd652a74640 1 --2- 172.21.15.43:0/1384177661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd64c0b5560 0x7fd64c0b59e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fd638009870 tx=0x7fd63802a3c0 comp rx=0 tx=0).stop 2024-04-25T08:54:49.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.697+0000 7fd652a74640 1 -- 172.21.15.43:0/1384177661 shutdown_connections 2024-04-25T08:54:49.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.697+0000 7fd652a74640 1 --2- 172.21.15.43:0/1384177661 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd64c0b5560 0x7fd64c0b59e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:49.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.697+0000 7fd652a74640 1 -- 172.21.15.43:0/1384177661 >> 172.21.15.43:0/1384177661 conn(0x7fd64c0d0560 msgr2=0x7fd64c0d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:49.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.697+0000 7fd652a74640 1 -- 172.21.15.43:0/1384177661 shutdown_connections 2024-04-25T08:54:49.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.697+0000 7fd652a74640 1 -- 172.21.15.43:0/1384177661 wait complete. 2024-04-25T08:54:49.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.698+0000 7fd652a74640 1 Processor -- start 2024-04-25T08:54:49.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.698+0000 7fd652a74640 1 -- start start 2024-04-25T08:54:49.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.699+0000 7fd652a74640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd64c0b5560 0x7fd64c0c67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:49.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.699+0000 7fd652a74640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd64c0d2230 0x7fd64c158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:49.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.699+0000 7fd652a74640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd64c15e090 0x7fd64c1606d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:49.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.699+0000 7fd652a74640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c1621b0 con 0x7fd64c0b5560 2024-04-25T08:54:49.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.699+0000 7fd652a74640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c0b3280 con 0x7fd64c15e090 2024-04-25T08:54:49.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.699+0000 7fd652a74640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c162330 con 0x7fd64c0d2230 2024-04-25T08:54:49.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.699+0000 7fd650810640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd64c0b5560 0x7fd64c0c67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:49.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.699+0000 7fd650810640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd64c0b5560 0x7fd64c0c67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49516/0 (socket says 172.21.15.43:49516) 2024-04-25T08:54:49.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.699+0000 7fd650810640 1 -- 172.21.15.43:0/1015917847 learned_addr learned my addr 172.21.15.43:0/1015917847 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:49.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.699+0000 7fd64bfff640 1 --2- 172.21.15.43:0/1015917847 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd64c0d2230 0x7fd64c158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:49.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.699+0000 7fd651011640 1 --2- 172.21.15.43:0/1015917847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd64c15e090 0x7fd64c1606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:49.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.699+0000 7fd650810640 1 -- 172.21.15.43:0/1015917847 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd64c0d2230 msgr2=0x7fd64c158830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:49.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.699+0000 7fd650810640 1 --2- 172.21.15.43:0/1015917847 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd64c0d2230 0x7fd64c158830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:49.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.699+0000 7fd650810640 1 -- 172.21.15.43:0/1015917847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd64c15e090 msgr2=0x7fd64c1606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:49.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.699+0000 7fd650810640 1 --2- 172.21.15.43:0/1015917847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd64c15e090 0x7fd64c1606d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:49.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.699+0000 7fd650810640 1 -- 172.21.15.43:0/1015917847 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd64c15f440 con 0x7fd64c0b5560 2024-04-25T08:54:49.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.699+0000 7fd650810640 1 --2- 172.21.15.43:0/1015917847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd64c0b5560 0x7fd64c0c67f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd64000c400 tx=0x7fd64000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:49.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.700+0000 7fd6497fa640 1 -- 172.21.15.43:0/1015917847 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd640017070 con 0x7fd64c0b5560 2024-04-25T08:54:49.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.700+0000 7fd6497fa640 1 -- 172.21.15.43:0/1015917847 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6400052d0 con 0x7fd64c0b5560 2024-04-25T08:54:49.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.700+0000 7fd6497fa640 1 -- 172.21.15.43:0/1015917847 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd640003830 con 0x7fd64c0b5560 2024-04-25T08:54:49.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.700+0000 7fd652a74640 1 -- 172.21.15.43:0/1015917847 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd64c15f720 con 0x7fd64c0b5560 2024-04-25T08:54:49.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.700+0000 7fd652a74640 1 -- 172.21.15.43:0/1015917847 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd64c0c7de0 con 0x7fd64c0b5560 2024-04-25T08:54:49.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.701+0000 7fd652a74640 1 -- 172.21.15.43:0/1015917847 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd64c06bf00 con 0x7fd64c0b5560 2024-04-25T08:54:49.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.701+0000 7fd6497fa640 1 -- 172.21.15.43:0/1015917847 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd64001a070 con 0x7fd64c0b5560 2024-04-25T08:54:49.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.701+0000 7fd6497fa640 1 --2- 172.21.15.43:0/1015917847 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd63006b4d0 0x7fd63006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:49.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.702+0000 7fd6497fa640 1 -- 172.21.15.43:0/1015917847 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd6400894d0 con 0x7fd64c0b5560 2024-04-25T08:54:49.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.702+0000 7fd64bfff640 1 --2- 172.21.15.43:0/1015917847 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd63006b4d0 0x7fd63006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:49.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.702+0000 7fd64bfff640 1 --2- 172.21.15.43:0/1015917847 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd63006b4d0 0x7fd63006d990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd638034000 tx=0x7fd63802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:49.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.706+0000 7fd6497fa640 1 -- 172.21.15.43:0/1015917847 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd64c06bf00 con 0x7fd64c0b5560 2024-04-25T08:54:49.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.886+0000 7fd652a74640 1 -- 172.21.15.43:0/1015917847 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd64c0c7910 con 0x7fd64c0b5560 2024-04-25T08:54:49.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.886+0000 7fd6497fa640 1 -- 172.21.15.43:0/1015917847 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd640087e20 con 0x7fd64c0b5560 2024-04-25T08:54:49.877 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T08:54:49.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.888+0000 7fd652a74640 1 -- 172.21.15.43:0/1015917847 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd63006b4d0 msgr2=0x7fd63006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:49.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.888+0000 7fd652a74640 1 --2- 172.21.15.43:0/1015917847 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd63006b4d0 0x7fd63006d990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd638034000 tx=0x7fd63802e000 comp rx=0 tx=0).stop 2024-04-25T08:54:49.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.888+0000 7fd652a74640 1 -- 172.21.15.43:0/1015917847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd64c0b5560 msgr2=0x7fd64c0c67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:49.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.888+0000 7fd652a74640 1 --2- 172.21.15.43:0/1015917847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd64c0b5560 0x7fd64c0c67f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd64000c400 tx=0x7fd64000c8c0 comp rx=0 tx=0).stop 2024-04-25T08:54:49.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.888+0000 7fd652a74640 1 -- 172.21.15.43:0/1015917847 shutdown_connections 2024-04-25T08:54:49.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.888+0000 7fd652a74640 1 --2- 172.21.15.43:0/1015917847 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd63006b4d0 0x7fd63006d990 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:49.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.888+0000 7fd652a74640 1 --2- 172.21.15.43:0/1015917847 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd64c15e090 0x7fd64c1606d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:49.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.888+0000 7fd652a74640 1 --2- 172.21.15.43:0/1015917847 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd64c0d2230 0x7fd64c158830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:49.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.888+0000 7fd652a74640 1 --2- 172.21.15.43:0/1015917847 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd64c0b5560 0x7fd64c0c67f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:49.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.888+0000 7fd652a74640 1 -- 172.21.15.43:0/1015917847 >> 172.21.15.43:0/1015917847 conn(0x7fd64c0d0560 msgr2=0x7fd64c0cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:49.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.888+0000 7fd652a74640 1 -- 172.21.15.43:0/1015917847 shutdown_connections 2024-04-25T08:54:49.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:49.888+0000 7fd652a74640 1 -- 172.21.15.43:0/1015917847 wait complete. 2024-04-25T08:54:49.890 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:54:49.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.002+0000 7fc39905d640 1 Processor -- start 2024-04-25T08:54:49.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.002+0000 7fc39905d640 1 -- start start 2024-04-25T08:54:49.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.002+0000 7fc39905d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc39415dce0 0x7fc39415ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:49.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.002+0000 7fc39905d640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fc3940ce550 0x7fc3940d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:49.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.002+0000 7fc39905d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc3940ce990 0x7fc3940d0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:49.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.002+0000 7fc39905d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3940cee80 con 0x7fc3940ce990 2024-04-25T08:54:49.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.002+0000 7fc39905d640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3940cf160 con 0x7fc3940ce550 2024-04-25T08:54:49.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.002+0000 7fc39905d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3940cf440 con 0x7fc39415dce0 2024-04-25T08:54:49.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.003+0000 7fc39359e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc3940ce990 0x7fc3940d0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:49.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.003+0000 7fc39359e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc3940ce990 0x7fc3940d0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49530/0 (socket says 172.21.15.43:49530) 2024-04-25T08:54:49.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.003+0000 7fc39359e640 1 -- 172.21.15.43:0/1536846795 learned_addr learned my addr 172.21.15.43:0/1536846795 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:49.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.003+0000 7fc39259c640 1 --2- 172.21.15.43:0/1536846795 >> v2:172.21.15.193:3300/0 conn(0x7fc3940ce550 0x7fc3940d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:49.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.003+0000 7fc392d9d640 1 --2- 172.21.15.43:0/1536846795 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc39415dce0 0x7fc39415ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:49.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.003+0000 7fc39359e640 1 -- 172.21.15.43:0/1536846795 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc39415dce0 msgr2=0x7fc39415ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:49.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.003+0000 7fc39359e640 1 --2- 172.21.15.43:0/1536846795 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc39415dce0 0x7fc39415ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:49.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.003+0000 7fc39359e640 1 -- 172.21.15.43:0/1536846795 >> v2:172.21.15.193:3300/0 conn(0x7fc3940ce550 msgr2=0x7fc3940d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:49.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.003+0000 7fc39359e640 1 --2- 172.21.15.43:0/1536846795 >> v2:172.21.15.193:3300/0 conn(0x7fc3940ce550 0x7fc3940d03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:49.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.003+0000 7fc39359e640 1 -- 172.21.15.43:0/1536846795 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3940cf720 con 0x7fc3940ce990 2024-04-25T08:54:49.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.004+0000 7fc39359e640 1 --2- 172.21.15.43:0/1536846795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3940ce990 0x7fc3940d0920 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc39409d430 tx=0x7fc38c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=440cc01931fd9ea0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:49.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.004+0000 7fc391d9b640 1 -- 172.21.15.43:0/1536846795 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38c037070 con 0x7fc3940ce990 2024-04-25T08:54:49.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.004+0000 7fc391d9b640 1 -- 172.21.15.43:0/1536846795 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc38c032440 con 0x7fc3940ce990 2024-04-25T08:54:49.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.004+0000 7fc391d9b640 1 -- 172.21.15.43:0/1536846795 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38c032810 con 0x7fc3940ce990 2024-04-25T08:54:49.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.004+0000 7fc39905d640 1 -- 172.21.15.43:0/1536846795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3940ce990 msgr2=0x7fc3940d0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:49.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.004+0000 7fc39905d640 1 --2- 172.21.15.43:0/1536846795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3940ce990 0x7fc3940d0920 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc39409d430 tx=0x7fc38c02a140 comp rx=0 tx=0).stop 2024-04-25T08:54:49.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.004+0000 7fc39905d640 1 -- 172.21.15.43:0/1536846795 shutdown_connections 2024-04-25T08:54:49.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.004+0000 7fc39905d640 1 --2- 172.21.15.43:0/1536846795 >> v2:172.21.15.193:3300/0 conn(0x7fc3940ce550 0x7fc3940d03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:49.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.004+0000 7fc39905d640 1 --2- 172.21.15.43:0/1536846795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3940ce990 0x7fc3940d0920 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:49.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.004+0000 7fc39905d640 1 --2- 172.21.15.43:0/1536846795 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc39415dce0 0x7fc39415ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:49.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.004+0000 7fc39905d640 1 -- 172.21.15.43:0/1536846795 >> 172.21.15.43:0/1536846795 conn(0x7fc39409cd00 msgr2=0x7fc3940d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:49.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.004+0000 7fc39905d640 1 -- 172.21.15.43:0/1536846795 shutdown_connections 2024-04-25T08:54:49.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.004+0000 7fc39905d640 1 -- 172.21.15.43:0/1536846795 wait complete. 2024-04-25T08:54:49.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.006+0000 7fc39905d640 1 Processor -- start 2024-04-25T08:54:49.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.006+0000 7fc39905d640 1 -- start start 2024-04-25T08:54:49.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.006+0000 7fc39905d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc3940ce550 0x7fc39415ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:49.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.007+0000 7fc39905d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3940ce990 0x7fc3941f3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:49.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.007+0000 7fc39905d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc39415dce0 0x7fc3940d1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:49.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.007+0000 7fc39905d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc394160980 con 0x7fc3940ce990 2024-04-25T08:54:49.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.007+0000 7fc39905d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39415e0c0 con 0x7fc39415dce0 2024-04-25T08:54:49.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.007+0000 7fc39905d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc394160b00 con 0x7fc3940ce550 2024-04-25T08:54:49.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.007+0000 7fc39259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3940ce990 0x7fc3941f3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:49.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.007+0000 7fc39259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3940ce990 0x7fc3941f3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49542/0 (socket says 172.21.15.43:49542) 2024-04-25T08:54:49.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.007+0000 7fc39259c640 1 -- 172.21.15.43:0/2553719994 learned_addr learned my addr 172.21.15.43:0/2553719994 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:49.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.007+0000 7fc392d9d640 1 --2- 172.21.15.43:0/2553719994 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc3940ce550 0x7fc39415ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:49.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.007+0000 7fc39359e640 1 --2- 172.21.15.43:0/2553719994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc39415dce0 0x7fc3940d1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:49.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.007+0000 7fc39259c640 1 -- 172.21.15.43:0/2553719994 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc3940ce550 msgr2=0x7fc39415ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:49.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.007+0000 7fc39259c640 1 --2- 172.21.15.43:0/2553719994 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc3940ce550 0x7fc39415ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:49.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.007+0000 7fc39259c640 1 -- 172.21.15.43:0/2553719994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc39415dce0 msgr2=0x7fc3940d1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:49.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.007+0000 7fc39259c640 1 --2- 172.21.15.43:0/2553719994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc39415dce0 0x7fc3940d1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:50.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.007+0000 7fc39259c640 1 -- 172.21.15.43:0/2553719994 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3941f4700 con 0x7fc3940ce990 2024-04-25T08:54:50.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.007+0000 7fc39359e640 1 --2- 172.21.15.43:0/2553719994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc39415dce0 0x7fc3940d1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:50.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.007+0000 7fc39259c640 1 --2- 172.21.15.43:0/2553719994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3940ce990 0x7fc3941f3d90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc37c00d800 tx=0x7fc37c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:50.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.007+0000 7fc38b7fe640 1 -- 172.21.15.43:0/2553719994 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37c00b6c0 con 0x7fc3940ce990 2024-04-25T08:54:50.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.007+0000 7fc38b7fe640 1 -- 172.21.15.43:0/2553719994 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc37c019070 con 0x7fc3940ce990 2024-04-25T08:54:50.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.007+0000 7fc39905d640 1 -- 172.21.15.43:0/2553719994 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3941f49e0 con 0x7fc3940ce990 2024-04-25T08:54:50.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.007+0000 7fc38b7fe640 1 -- 172.21.15.43:0/2553719994 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc37c00ba80 con 0x7fc3940ce990 2024-04-25T08:54:50.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.007+0000 7fc39905d640 1 -- 172.21.15.43:0/2553719994 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3940a0120 con 0x7fc3940ce990 2024-04-25T08:54:50.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.008+0000 7fc39905d640 1 -- 172.21.15.43:0/2553719994 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3940680f0 con 0x7fc3940ce990 2024-04-25T08:54:50.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.009+0000 7fc38b7fe640 1 -- 172.21.15.43:0/2553719994 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc37c00bc20 con 0x7fc3940ce990 2024-04-25T08:54:50.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.009+0000 7fc38b7fe640 1 --2- 172.21.15.43:0/2553719994 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc37406b4d0 0x7fc37406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:50.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.009+0000 7fc38b7fe640 1 -- 172.21.15.43:0/2553719994 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc37c08a6b0 con 0x7fc3940ce990 2024-04-25T08:54:50.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.009+0000 7fc392d9d640 1 --2- 172.21.15.43:0/2553719994 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc37406b4d0 0x7fc37406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:50.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.010+0000 7fc392d9d640 1 --2- 172.21.15.43:0/2553719994 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc37406b4d0 0x7fc37406d990 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc3940d0a00 tx=0x7fc384003990 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:50.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.013+0000 7fc38b7fe640 1 -- 172.21.15.43:0/2553719994 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc37c059530 con 0x7fc3940ce990 2024-04-25T08:54:50.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.205+0000 7fc39905d640 1 -- 172.21.15.43:0/2553719994 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc394074ef0 con 0x7fc3940ce990 2024-04-25T08:54:50.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.205+0000 7fc38b7fe640 1 -- 172.21.15.43:0/2553719994 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc37c058e30 con 0x7fc3940ce990 2024-04-25T08:54:50.195 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-04-25T08:54:50.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.207+0000 7fc39905d640 1 -- 172.21.15.43:0/2553719994 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc37406b4d0 msgr2=0x7fc37406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:50.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.207+0000 7fc39905d640 1 --2- 172.21.15.43:0/2553719994 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc37406b4d0 0x7fc37406d990 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc3940d0a00 tx=0x7fc384003990 comp rx=0 tx=0).stop 2024-04-25T08:54:50.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.207+0000 7fc39905d640 1 -- 172.21.15.43:0/2553719994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3940ce990 msgr2=0x7fc3941f3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:50.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.207+0000 7fc39905d640 1 --2- 172.21.15.43:0/2553719994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3940ce990 0x7fc3941f3d90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc37c00d800 tx=0x7fc37c00dcc0 comp rx=0 tx=0).stop 2024-04-25T08:54:50.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.207+0000 7fc39905d640 1 -- 172.21.15.43:0/2553719994 shutdown_connections 2024-04-25T08:54:50.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.207+0000 7fc39905d640 1 --2- 172.21.15.43:0/2553719994 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc37406b4d0 0x7fc37406d990 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:50.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.207+0000 7fc39905d640 1 --2- 172.21.15.43:0/2553719994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc39415dce0 0x7fc3940d1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:50.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.207+0000 7fc39905d640 1 --2- 172.21.15.43:0/2553719994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3940ce990 0x7fc3941f3d90 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:50.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.207+0000 7fc39905d640 1 --2- 172.21.15.43:0/2553719994 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc3940ce550 0x7fc39415ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:50.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.207+0000 7fc39905d640 1 -- 172.21.15.43:0/2553719994 >> 172.21.15.43:0/2553719994 conn(0x7fc39409cd00 msgr2=0x7fc39406b6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:50.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.207+0000 7fc39905d640 1 -- 172.21.15.43:0/2553719994 shutdown_connections 2024-04-25T08:54:50.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.207+0000 7fc39905d640 1 -- 172.21.15.43:0/2553719994 wait complete. 2024-04-25T08:54:50.208 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi043.front.sepia.ceph.com... 2024-04-25T08:54:50.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:50.208 DEBUG:teuthology.orchestra.run.smithi043:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-04-25T08:54:50.233 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:54:50.233 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-04-25T08:54:50.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:50.233 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T08:54:50.289 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T08:54:50.289 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:54:50.308 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T08:54:50.308 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:54:50.367 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:54:50.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.478+0000 7f1292322640 1 Processor -- start 2024-04-25T08:54:50.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.478+0000 7f1292322640 1 -- start start 2024-04-25T08:54:50.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.479+0000 7f1292322640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f128c06d9a0 0x7f128c071a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:50.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.479+0000 7f1292322640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f128c071f90 0x7f128c0cd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:50.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.479+0000 7f1292322640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f128c0723e0 con 0x7f128c15a9d0 2024-04-25T08:54:50.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.479+0000 7f1292322640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f128c0704f0 con 0x7f128c071f90 2024-04-25T08:54:50.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.479+0000 7f1292322640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f128c0707d0 con 0x7f128c06d9a0 2024-04-25T08:54:50.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.479+0000 7f12908bf640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f128c071f90 0x7f128c0cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:50.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.479+0000 7f128bfff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f128c15a9d0 0x7f128c06d290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48338/0 (socket says 172.21.15.43:48338) 2024-04-25T08:54:50.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.479+0000 7f128bfff640 1 -- 172.21.15.43:0/1977423824 learned_addr learned my addr 172.21.15.43:0/1977423824 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:50.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.480+0000 7f128b7fe640 1 --2- 172.21.15.43:0/1977423824 >> v2:172.21.15.193:3300/0 conn(0x7f128c06d9a0 0x7f128c071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:50.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.480+0000 7f12908bf640 1 -- 172.21.15.43:0/1977423824 >> v2:172.21.15.193:3300/0 conn(0x7f128c06d9a0 msgr2=0x7f128c071a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:50.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.480+0000 7f12908bf640 1 --2- 172.21.15.43:0/1977423824 >> v2:172.21.15.193:3300/0 conn(0x7f128c06d9a0 0x7f128c071a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:50.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.480+0000 7f12908bf640 1 -- 172.21.15.43:0/1977423824 >> v1:172.21.15.43:6789/0 conn(0x7f128c15a9d0 legacy=0x7f128c06d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:50.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.480+0000 7f12908bf640 1 -- 172.21.15.43:0/1977423824 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f128c070ab0 con 0x7f128c071f90 2024-04-25T08:54:50.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.480+0000 7f128affd640 1 -- 172.21.15.43:0/1977423824 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2540963916 0 0) 0x7f128c0723e0 con 0x7f128c15a9d0 2024-04-25T08:54:50.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.480+0000 7f128b7fe640 1 --2- 172.21.15.43:0/1977423824 >> v2:172.21.15.193:3300/0 conn(0x7f128c06d9a0 0x7f128c071a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:50.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.480+0000 7f12908bf640 1 --2- 172.21.15.43:0/1977423824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f128c071f90 0x7f128c0cd940 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f128c06baf0 tx=0x7f128402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=878c3f82f30cbb73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:50.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.480+0000 7f128affd640 1 -- 172.21.15.43:0/1977423824 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1284007740 con 0x7f128c071f90 2024-04-25T08:54:50.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.480+0000 7f128affd640 1 -- 172.21.15.43:0/1977423824 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12840078e0 con 0x7f128c071f90 2024-04-25T08:54:50.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.481+0000 7f128affd640 1 -- 172.21.15.43:0/1977423824 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1284007cb0 con 0x7f128c071f90 2024-04-25T08:54:50.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.481+0000 7f1292322640 1 -- 172.21.15.43:0/1977423824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f128c071f90 msgr2=0x7f128c0cd940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:50.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.481+0000 7f1292322640 1 --2- 172.21.15.43:0/1977423824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f128c071f90 0x7f128c0cd940 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f128c06baf0 tx=0x7f128402a680 comp rx=0 tx=0).stop 2024-04-25T08:54:50.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.481+0000 7f1292322640 1 -- 172.21.15.43:0/1977423824 shutdown_connections 2024-04-25T08:54:50.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.481+0000 7f1292322640 1 --2- 172.21.15.43:0/1977423824 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f128c071f90 0x7f128c0cd940 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:50.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.481+0000 7f1292322640 1 --2- 172.21.15.43:0/1977423824 >> v2:172.21.15.193:3300/0 conn(0x7f128c06d9a0 0x7f128c071a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:50.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.481+0000 7f1292322640 1 -- 172.21.15.43:0/1977423824 >> 172.21.15.43:0/1977423824 conn(0x7f128c0b1480 msgr2=0x7f128c0b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:50.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.481+0000 7f1292322640 1 -- 172.21.15.43:0/1977423824 shutdown_connections 2024-04-25T08:54:50.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.481+0000 7f1292322640 1 -- 172.21.15.43:0/1977423824 wait complete. 2024-04-25T08:54:50.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.482+0000 7f1292322640 1 Processor -- start 2024-04-25T08:54:50.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.482+0000 7f1292322640 1 -- start start 2024-04-25T08:54:50.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.483+0000 7f1292322640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f128c06d9a0 0x7f128c144ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:50.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.483+0000 7f1292322640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f128c071f90 0x7f128c1e5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:50.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.483+0000 7f1292322640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f128c1e63b0 0x7f128c1e3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:50.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.483+0000 7f1292322640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128c0cde80 con 0x7f128c071f90 2024-04-25T08:54:50.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.483+0000 7f1292322640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128c0712b0 con 0x7f128c06d9a0 2024-04-25T08:54:50.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.483+0000 7f1292322640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f128c0ce000 con 0x7f128c1e63b0 2024-04-25T08:54:50.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.483+0000 7f128b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f128c071f90 0x7f128c1e5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:50.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.483+0000 7f128b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f128c071f90 0x7f128c1e5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49564/0 (socket says 172.21.15.43:49564) 2024-04-25T08:54:50.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.483+0000 7f128b7fe640 1 -- 172.21.15.43:0/201007598 learned_addr learned my addr 172.21.15.43:0/201007598 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:50.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.483+0000 7f12908bf640 1 --2- 172.21.15.43:0/201007598 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f128c1e63b0 0x7f128c1e3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:50.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.483+0000 7f128bfff640 1 --2- 172.21.15.43:0/201007598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f128c06d9a0 0x7f128c144ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:50.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.484+0000 7f128b7fe640 1 -- 172.21.15.43:0/201007598 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f128c1e63b0 msgr2=0x7f128c1e3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:50.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.484+0000 7f128b7fe640 1 --2- 172.21.15.43:0/201007598 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f128c1e63b0 0x7f128c1e3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:50.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.484+0000 7f128b7fe640 1 -- 172.21.15.43:0/201007598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f128c06d9a0 msgr2=0x7f128c144ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:50.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.484+0000 7f128b7fe640 1 --2- 172.21.15.43:0/201007598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f128c06d9a0 0x7f128c144ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:50.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.484+0000 7f128b7fe640 1 -- 172.21.15.43:0/201007598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f128c145530 con 0x7f128c071f90 2024-04-25T08:54:50.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.484+0000 7f12908bf640 1 --2- 172.21.15.43:0/201007598 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f128c1e63b0 0x7f128c1e3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:50.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.484+0000 7f128b7fe640 1 --2- 172.21.15.43:0/201007598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f128c071f90 0x7f128c1e5e70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f128000c7f0 tx=0x7f128000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:50.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.484+0000 7f1288ff9640 1 -- 172.21.15.43:0/201007598 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1280015070 con 0x7f128c071f90 2024-04-25T08:54:50.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.484+0000 7f1288ff9640 1 -- 172.21.15.43:0/201007598 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1280007a30 con 0x7f128c071f90 2024-04-25T08:54:50.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.484+0000 7f1292322640 1 -- 172.21.15.43:0/201007598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f128c1e46f0 con 0x7f128c071f90 2024-04-25T08:54:50.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.484+0000 7f1288ff9640 1 -- 172.21.15.43:0/201007598 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1280007e00 con 0x7f128c071f90 2024-04-25T08:54:50.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.484+0000 7f1292322640 1 -- 172.21.15.43:0/201007598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f128c1e4cb0 con 0x7f128c071f90 2024-04-25T08:54:50.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.485+0000 7f1292322640 1 -- 172.21.15.43:0/201007598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f128c06dfd0 con 0x7f128c071f90 2024-04-25T08:54:50.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.485+0000 7f1288ff9640 1 -- 172.21.15.43:0/201007598 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1280018070 con 0x7f128c071f90 2024-04-25T08:54:50.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.486+0000 7f1288ff9640 1 --2- 172.21.15.43:0/201007598 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f127806b4d0 0x7f127806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:50.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.486+0000 7f1288ff9640 1 -- 172.21.15.43:0/201007598 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f128008a170 con 0x7f128c071f90 2024-04-25T08:54:50.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.486+0000 7f128bfff640 1 --2- 172.21.15.43:0/201007598 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f127806b4d0 0x7f127806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:50.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.487+0000 7f128bfff640 1 --2- 172.21.15.43:0/201007598 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f127806b4d0 0x7f127806d990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f127c000f60 tx=0x7f127c007c00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:50.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.490+0000 7f1288ff9640 1 -- 172.21.15.43:0/201007598 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f128c06dfd0 con 0x7f128c071f90 2024-04-25T08:54:50.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.672+0000 7f1292322640 1 -- 172.21.15.43:0/201007598 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f128c074ef0 con 0x7f128c071f90 2024-04-25T08:54:50.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.673+0000 7f1288ff9640 1 -- 172.21.15.43:0/201007598 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16569 (secure 0 0 0) 0x7f1280058ff0 con 0x7f128c071f90 2024-04-25T08:54:50.663 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:54:50.663 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":20,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:54:13.593188+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:54:09.757978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:54:10.926008+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:54:50.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.675+0000 7f1292322640 1 -- 172.21.15.43:0/201007598 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f127806b4d0 msgr2=0x7f127806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:50.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.675+0000 7f1292322640 1 --2- 172.21.15.43:0/201007598 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f127806b4d0 0x7f127806d990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f127c000f60 tx=0x7f127c007c00 comp rx=0 tx=0).stop 2024-04-25T08:54:50.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.675+0000 7f1292322640 1 -- 172.21.15.43:0/201007598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f128c071f90 msgr2=0x7f128c1e5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:50.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.675+0000 7f1292322640 1 --2- 172.21.15.43:0/201007598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f128c071f90 0x7f128c1e5e70 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f128000c7f0 tx=0x7f128000cbf0 comp rx=0 tx=0).stop 2024-04-25T08:54:50.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.675+0000 7f1292322640 1 -- 172.21.15.43:0/201007598 shutdown_connections 2024-04-25T08:54:50.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.675+0000 7f1292322640 1 --2- 172.21.15.43:0/201007598 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f128c1e63b0 0x7f128c1e3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:50.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.675+0000 7f1292322640 1 --2- 172.21.15.43:0/201007598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f128c071f90 0x7f128c1e5e70 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:50.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.675+0000 7f1292322640 1 --2- 172.21.15.43:0/201007598 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f127806b4d0 0x7f127806d990 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:50.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.675+0000 7f1292322640 1 --2- 172.21.15.43:0/201007598 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f128c06d9a0 0x7f128c144ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:50.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.675+0000 7f1292322640 1 -- 172.21.15.43:0/201007598 >> 172.21.15.43:0/201007598 conn(0x7f128c0b1480 msgr2=0x7f128c15d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:50.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.675+0000 7f1292322640 1 -- 172.21.15.43:0/201007598 shutdown_connections 2024-04-25T08:54:50.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.675+0000 7f1292322640 1 -- 172.21.15.43:0/201007598 wait complete. 2024-04-25T08:54:50.676 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:54:09.757978+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:54:10.926008+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:54:50.676 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:54:50.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.790+0000 7f582fc09640 1 Processor -- start 2024-04-25T08:54:50.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.791+0000 7f582fc09640 1 -- start start 2024-04-25T08:54:50.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.791+0000 7f582fc09640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f58280b5560 0x7f58280b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:50.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.791+0000 7f582fc09640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5828162610 con 0x7f58280d2230 2024-04-25T08:54:50.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.791+0000 7f582fc09640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58280d1750 con 0x7f582815e090 2024-04-25T08:54:50.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.791+0000 7f582fc09640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58280d1da0 con 0x7f58280b5560 2024-04-25T08:54:50.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.791+0000 7f582d1a4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f58280b5560 0x7f58280b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:50.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.792+0000 7f582e1a6640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f58280d2230 0x7f58280d26c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48352/0 (socket says 172.21.15.43:48352) 2024-04-25T08:54:50.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.792+0000 7f582e1a6640 1 -- 172.21.15.43:0/3278024617 learned_addr learned my addr 172.21.15.43:0/3278024617 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:50.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.792+0000 7f582d1a4640 1 -- 172.21.15.43:0/3278024617 >> v1:172.21.15.193:6789/0 conn(0x7f582815e090 legacy=0x7f5828161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:50.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.792+0000 7f582d1a4640 1 -- 172.21.15.43:0/3278024617 >> v1:172.21.15.43:6789/0 conn(0x7f58280d2230 legacy=0x7f58280d26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:50.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.792+0000 7f582c9a3640 1 -- 172.21.15.43:0/3278024617 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 30399082 0 0) 0x7f5828162610 con 0x7f58280d2230 2024-04-25T08:54:50.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.792+0000 7f582d1a4640 1 -- 172.21.15.43:0/3278024617 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58280d1a30 con 0x7f58280b5560 2024-04-25T08:54:50.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.793+0000 7f582d1a4640 1 --2- 172.21.15.43:0/3278024617 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58280b5560 0x7f58280b59e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f58180097a0 tx=0x7f581802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f84954044b34679e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:50.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.793+0000 7f582c9a3640 1 -- 172.21.15.43:0/3278024617 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5818037070 con 0x7f58280b5560 2024-04-25T08:54:50.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.793+0000 7f582c9a3640 1 -- 172.21.15.43:0/3278024617 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5818032440 con 0x7f58280b5560 2024-04-25T08:54:50.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.793+0000 7f582c9a3640 1 -- 172.21.15.43:0/3278024617 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5818032810 con 0x7f58280b5560 2024-04-25T08:54:50.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.793+0000 7f582fc09640 1 -- 172.21.15.43:0/3278024617 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58280b5560 msgr2=0x7f58280b59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:50.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.793+0000 7f582fc09640 1 --2- 172.21.15.43:0/3278024617 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58280b5560 0x7f58280b59e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f58180097a0 tx=0x7f581802a270 comp rx=0 tx=0).stop 2024-04-25T08:54:50.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.793+0000 7f582fc09640 1 -- 172.21.15.43:0/3278024617 shutdown_connections 2024-04-25T08:54:50.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.793+0000 7f582fc09640 1 --2- 172.21.15.43:0/3278024617 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58280b5560 0x7f58280b59e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:50.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.793+0000 7f582fc09640 1 -- 172.21.15.43:0/3278024617 >> 172.21.15.43:0/3278024617 conn(0x7f58280d0560 msgr2=0x7f58280d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:50.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.793+0000 7f582fc09640 1 -- 172.21.15.43:0/3278024617 shutdown_connections 2024-04-25T08:54:50.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.793+0000 7f582fc09640 1 -- 172.21.15.43:0/3278024617 wait complete. 2024-04-25T08:54:50.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.794+0000 7f582fc09640 1 Processor -- start 2024-04-25T08:54:50.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.795+0000 7f582fc09640 1 -- start start 2024-04-25T08:54:50.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.795+0000 7f582fc09640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58280b5560 0x7f582815e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:50.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.795+0000 7f582fc09640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f58280d2230 0x7f5828160d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:50.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.795+0000 7f582fc09640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f582815e090 0x7f582814be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:50.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.795+0000 7f582fc09640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58281621b0 con 0x7f582815e090 2024-04-25T08:54:50.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.795+0000 7f582fc09640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58280b3280 con 0x7f58280b5560 2024-04-25T08:54:50.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.795+0000 7f582fc09640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5828162330 con 0x7f58280d2230 2024-04-25T08:54:50.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.795+0000 7f582e1a6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f582815e090 0x7f582814be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:50.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.795+0000 7f582e1a6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f582815e090 0x7f582814be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49572/0 (socket says 172.21.15.43:49572) 2024-04-25T08:54:50.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.795+0000 7f582e1a6640 1 -- 172.21.15.43:0/3965742636 learned_addr learned my addr 172.21.15.43:0/3965742636 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:50.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.796+0000 7f582e1a6640 1 -- 172.21.15.43:0/3965742636 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f58280d2230 msgr2=0x7f5828160d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:50.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.796+0000 7f582e1a6640 1 --2- 172.21.15.43:0/3965742636 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f58280d2230 0x7f5828160d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:50.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.796+0000 7f582e1a6640 1 -- 172.21.15.43:0/3965742636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58280b5560 msgr2=0x7f582815e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:50.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.796+0000 7f582e1a6640 1 --2- 172.21.15.43:0/3965742636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58280b5560 0x7f582815e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:50.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.796+0000 7f582e1a6640 1 -- 172.21.15.43:0/3965742636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58281612c0 con 0x7f582815e090 2024-04-25T08:54:50.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.796+0000 7f582e1a6640 1 --2- 172.21.15.43:0/3965742636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f582815e090 0x7f582814be70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f582000c7a0 tx=0x7f582000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:50.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.796+0000 7f581e7fc640 1 -- 172.21.15.43:0/3965742636 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5820018070 con 0x7f582815e090 2024-04-25T08:54:50.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.796+0000 7f581e7fc640 1 -- 172.21.15.43:0/3965742636 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f582000a640 con 0x7f582815e090 2024-04-25T08:54:50.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.796+0000 7f582fc09640 1 -- 172.21.15.43:0/3965742636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58281f3e50 con 0x7f582815e090 2024-04-25T08:54:50.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.796+0000 7f581e7fc640 1 -- 172.21.15.43:0/3965742636 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582000aa70 con 0x7f582815e090 2024-04-25T08:54:50.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.796+0000 7f582fc09640 1 -- 172.21.15.43:0/3965742636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58281e7d40 con 0x7f582815e090 2024-04-25T08:54:50.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.797+0000 7f582fc09640 1 -- 172.21.15.43:0/3965742636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f582806bf00 con 0x7f582815e090 2024-04-25T08:54:50.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.797+0000 7f581e7fc640 1 -- 172.21.15.43:0/3965742636 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f582001b070 con 0x7f582815e090 2024-04-25T08:54:50.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.798+0000 7f581e7fc640 1 --2- 172.21.15.43:0/3965742636 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f580c06b4d0 0x7f580c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:50.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.798+0000 7f581e7fc640 1 -- 172.21.15.43:0/3965742636 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5820089820 con 0x7f582815e090 2024-04-25T08:54:50.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.798+0000 7f582d9a5640 1 --2- 172.21.15.43:0/3965742636 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f580c06b4d0 0x7f580c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:50.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.799+0000 7f582d9a5640 1 --2- 172.21.15.43:0/3965742636 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f580c06b4d0 0x7f580c06d990 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5824006fd0 tx=0x7f5824006bc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:50.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.802+0000 7f581e7fc640 1 -- 172.21.15.43:0/3965742636 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f582806bf00 con 0x7f582815e090 2024-04-25T08:54:50.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.981+0000 7f582fc09640 1 -- 172.21.15.43:0/3965742636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5828074f60 con 0x7f582815e090 2024-04-25T08:54:50.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.981+0000 7f581e7fc640 1 -- 172.21.15.43:0/3965742636 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f58200586a0 con 0x7f582815e090 2024-04-25T08:54:50.971 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T08:54:50.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.983+0000 7f582fc09640 1 -- 172.21.15.43:0/3965742636 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f580c06b4d0 msgr2=0x7f580c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:50.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.983+0000 7f582fc09640 1 --2- 172.21.15.43:0/3965742636 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f580c06b4d0 0x7f580c06d990 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5824006fd0 tx=0x7f5824006bc0 comp rx=0 tx=0).stop 2024-04-25T08:54:50.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.983+0000 7f582fc09640 1 -- 172.21.15.43:0/3965742636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f582815e090 msgr2=0x7f582814be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:50.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.983+0000 7f582fc09640 1 --2- 172.21.15.43:0/3965742636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f582815e090 0x7f582814be70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f582000c7a0 tx=0x7f582000cc60 comp rx=0 tx=0).stop 2024-04-25T08:54:50.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.983+0000 7f582fc09640 1 -- 172.21.15.43:0/3965742636 shutdown_connections 2024-04-25T08:54:50.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.983+0000 7f582fc09640 1 --2- 172.21.15.43:0/3965742636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f582815e090 0x7f582814be70 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:50.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.983+0000 7f582fc09640 1 --2- 172.21.15.43:0/3965742636 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f58280d2230 0x7f5828160d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:50.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.983+0000 7f582fc09640 1 --2- 172.21.15.43:0/3965742636 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f580c06b4d0 0x7f580c06d990 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:50.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.983+0000 7f582fc09640 1 --2- 172.21.15.43:0/3965742636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58280b5560 0x7f582815e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:50.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.983+0000 7f582fc09640 1 -- 172.21.15.43:0/3965742636 >> 172.21.15.43:0/3965742636 conn(0x7f58280d0560 msgr2=0x7f58280b7c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:50.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.983+0000 7f582fc09640 1 -- 172.21.15.43:0/3965742636 shutdown_connections 2024-04-25T08:54:50.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:50.983+0000 7f582fc09640 1 -- 172.21.15.43:0/3965742636 wait complete. 2024-04-25T08:54:50.984 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:54:51.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.098+0000 7fc49ff09640 1 Processor -- start 2024-04-25T08:54:51.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.098+0000 7fc49ff09640 1 -- start start 2024-04-25T08:54:51.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.099+0000 7fc49ff09640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc4980d04a0 0x7fc49815ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:51.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.099+0000 7fc49ff09640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fc4980ce8c0 0x7fc4980d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:51.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.099+0000 7fc49ff09640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4980cf110 con 0x7fc49815dec0 2024-04-25T08:54:51.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.099+0000 7fc49ff09640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4980cf3f0 con 0x7fc4980d04a0 2024-04-25T08:54:51.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.099+0000 7fc49ff09640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4980cf6d0 con 0x7fc4980ce8c0 2024-04-25T08:54:51.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.099+0000 7fc49dca5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc4980d04a0 0x7fc49815ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:51.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.099+0000 7fc49dca5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc4980d04a0 0x7fc49815ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49574/0 (socket says 172.21.15.43:49574) 2024-04-25T08:54:51.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.099+0000 7fc49dca5640 1 -- 172.21.15.43:0/1185363284 learned_addr learned my addr 172.21.15.43:0/1185363284 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:51.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.099+0000 7fc49dca5640 1 -- 172.21.15.43:0/1185363284 >> v2:172.21.15.193:3300/0 conn(0x7fc4980ce8c0 msgr2=0x7fc4980d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:51.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.099+0000 7fc49dca5640 1 --2- 172.21.15.43:0/1185363284 >> v2:172.21.15.193:3300/0 conn(0x7fc4980ce8c0 0x7fc4980d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:51.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.099+0000 7fc49dca5640 1 -- 172.21.15.43:0/1185363284 >> v1:172.21.15.193:6789/0 conn(0x7fc49815dec0 legacy=0x7fc4980d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:51.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.099+0000 7fc49dca5640 1 -- 172.21.15.43:0/1185363284 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4980cf9b0 con 0x7fc4980d04a0 2024-04-25T08:54:51.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.100+0000 7fc49dca5640 1 --2- 172.21.15.43:0/1185363284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4980d04a0 0x7fc49815ed90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc49808e260 tx=0x7fc49402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b5a9629c33b943 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:51.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.100+0000 7fc49cca3640 1 -- 172.21.15.43:0/1185363284 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc494037070 con 0x7fc4980d04a0 2024-04-25T08:54:51.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.100+0000 7fc49cca3640 1 -- 172.21.15.43:0/1185363284 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc494032440 con 0x7fc4980d04a0 2024-04-25T08:54:51.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.100+0000 7fc49cca3640 1 -- 172.21.15.43:0/1185363284 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc494032810 con 0x7fc4980d04a0 2024-04-25T08:54:51.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.100+0000 7fc49ff09640 1 -- 172.21.15.43:0/1185363284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4980d04a0 msgr2=0x7fc49815ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:51.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.100+0000 7fc49ff09640 1 --2- 172.21.15.43:0/1185363284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4980d04a0 0x7fc49815ed90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc49808e260 tx=0x7fc49402a230 comp rx=0 tx=0).stop 2024-04-25T08:54:51.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.100+0000 7fc49ff09640 1 -- 172.21.15.43:0/1185363284 shutdown_connections 2024-04-25T08:54:51.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.100+0000 7fc49ff09640 1 --2- 172.21.15.43:0/1185363284 >> v2:172.21.15.193:3300/0 conn(0x7fc4980ce8c0 0x7fc4980d17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:51.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.100+0000 7fc49ff09640 1 --2- 172.21.15.43:0/1185363284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4980d04a0 0x7fc49815ed90 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:51.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.100+0000 7fc49ff09640 1 -- 172.21.15.43:0/1185363284 >> 172.21.15.43:0/1185363284 conn(0x7fc49809cd40 msgr2=0x7fc4980d3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:51.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.101+0000 7fc49ff09640 1 -- 172.21.15.43:0/1185363284 shutdown_connections 2024-04-25T08:54:51.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.101+0000 7fc49ff09640 1 -- 172.21.15.43:0/1185363284 wait complete. 2024-04-25T08:54:51.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.102+0000 7fc49ff09640 1 Processor -- start 2024-04-25T08:54:51.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.102+0000 7fc49ff09640 1 -- start start 2024-04-25T08:54:51.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.102+0000 7fc49ff09640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4980ce8c0 0x7fc49815b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:51.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.102+0000 7fc49ff09640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc4980d04a0 0x7fc4980b6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:51.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.102+0000 7fc49ff09640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc49815dec0 0x7fc4980b6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:51.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.102+0000 7fc49ff09640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4980cee50 con 0x7fc49815dec0 2024-04-25T08:54:51.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.102+0000 7fc49ff09640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4980cecd0 con 0x7fc4980ce8c0 2024-04-25T08:54:51.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.102+0000 7fc49ff09640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4980d1ce0 con 0x7fc4980d04a0 2024-04-25T08:54:51.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.103+0000 7fc49e4a6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc49815dec0 0x7fc4980b6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:51.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.103+0000 7fc49e4a6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc49815dec0 0x7fc4980b6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49588/0 (socket says 172.21.15.43:49588) 2024-04-25T08:54:51.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.103+0000 7fc49e4a6640 1 -- 172.21.15.43:0/1318065400 learned_addr learned my addr 172.21.15.43:0/1318065400 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:51.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.103+0000 7fc49dca5640 1 --2- 172.21.15.43:0/1318065400 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4980ce8c0 0x7fc49815b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:51.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.103+0000 7fc49d4a4640 1 --2- 172.21.15.43:0/1318065400 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc4980d04a0 0x7fc4980b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:51.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.103+0000 7fc49e4a6640 1 -- 172.21.15.43:0/1318065400 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc4980d04a0 msgr2=0x7fc4980b6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:51.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.103+0000 7fc49e4a6640 1 --2- 172.21.15.43:0/1318065400 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc4980d04a0 0x7fc4980b6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:51.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.103+0000 7fc49e4a6640 1 -- 172.21.15.43:0/1318065400 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4980ce8c0 msgr2=0x7fc49815b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:51.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.103+0000 7fc49e4a6640 1 --2- 172.21.15.43:0/1318065400 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4980ce8c0 0x7fc49815b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:51.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.103+0000 7fc49e4a6640 1 -- 172.21.15.43:0/1318065400 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc49815b620 con 0x7fc49815dec0 2024-04-25T08:54:51.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.103+0000 7fc49d4a4640 1 --2- 172.21.15.43:0/1318065400 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc4980d04a0 0x7fc4980b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:51.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.103+0000 7fc49e4a6640 1 --2- 172.21.15.43:0/1318065400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc49815dec0 0x7fc4980b6bd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc49000b970 tx=0x7fc49000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:51.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.103+0000 7fc48e7fc640 1 -- 172.21.15.43:0/1318065400 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc490019070 con 0x7fc49815dec0 2024-04-25T08:54:51.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.103+0000 7fc48e7fc640 1 -- 172.21.15.43:0/1318065400 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc49000cb60 con 0x7fc49815dec0 2024-04-25T08:54:51.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.103+0000 7fc49ff09640 1 -- 172.21.15.43:0/1318065400 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4980b72a0 con 0x7fc49815dec0 2024-04-25T08:54:51.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.103+0000 7fc48e7fc640 1 -- 172.21.15.43:0/1318065400 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc490014440 con 0x7fc49815dec0 2024-04-25T08:54:51.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.103+0000 7fc49ff09640 1 -- 172.21.15.43:0/1318065400 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4980b7860 con 0x7fc49815dec0 2024-04-25T08:54:51.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.104+0000 7fc49ff09640 1 -- 172.21.15.43:0/1318065400 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc498074f60 con 0x7fc49815dec0 2024-04-25T08:54:51.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.105+0000 7fc48e7fc640 1 -- 172.21.15.43:0/1318065400 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc49001c070 con 0x7fc49815dec0 2024-04-25T08:54:51.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.105+0000 7fc48e7fc640 1 --2- 172.21.15.43:0/1318065400 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc47c06b4d0 0x7fc47c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:51.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.105+0000 7fc48e7fc640 1 -- 172.21.15.43:0/1318065400 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc490089e90 con 0x7fc49815dec0 2024-04-25T08:54:51.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.105+0000 7fc49dca5640 1 --2- 172.21.15.43:0/1318065400 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc47c06b4d0 0x7fc47c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:51.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.106+0000 7fc49dca5640 1 --2- 172.21.15.43:0/1318065400 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc47c06b4d0 0x7fc47c06d990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc494034000 tx=0x7fc49402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:51.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.109+0000 7fc48e7fc640 1 -- 172.21.15.43:0/1318065400 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc49815b620 con 0x7fc49815dec0 2024-04-25T08:54:51.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.287+0000 7fc49ff09640 1 -- 172.21.15.43:0/1318065400 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc49808daf0 con 0x7fc49815dec0 2024-04-25T08:54:51.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.287+0000 7fc48e7fc640 1 -- 172.21.15.43:0/1318065400 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc4900586c0 con 0x7fc49815dec0 2024-04-25T08:54:51.277 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T08:54:51.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.289+0000 7fc49ff09640 1 -- 172.21.15.43:0/1318065400 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc47c06b4d0 msgr2=0x7fc47c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:51.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.289+0000 7fc49ff09640 1 --2- 172.21.15.43:0/1318065400 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc47c06b4d0 0x7fc47c06d990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc494034000 tx=0x7fc49402e000 comp rx=0 tx=0).stop 2024-04-25T08:54:51.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.289+0000 7fc49ff09640 1 -- 172.21.15.43:0/1318065400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc49815dec0 msgr2=0x7fc4980b6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:51.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.289+0000 7fc49ff09640 1 --2- 172.21.15.43:0/1318065400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc49815dec0 0x7fc4980b6bd0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc49000b970 tx=0x7fc49000be30 comp rx=0 tx=0).stop 2024-04-25T08:54:51.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.290+0000 7fc49ff09640 1 -- 172.21.15.43:0/1318065400 shutdown_connections 2024-04-25T08:54:51.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.290+0000 7fc49ff09640 1 --2- 172.21.15.43:0/1318065400 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc49815dec0 0x7fc4980b6bd0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:51.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.290+0000 7fc49ff09640 1 --2- 172.21.15.43:0/1318065400 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc4980d04a0 0x7fc4980b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:51.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.290+0000 7fc49ff09640 1 --2- 172.21.15.43:0/1318065400 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc47c06b4d0 0x7fc47c06d990 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:51.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.290+0000 7fc49ff09640 1 --2- 172.21.15.43:0/1318065400 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4980ce8c0 0x7fc49815b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:51.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.290+0000 7fc49ff09640 1 -- 172.21.15.43:0/1318065400 >> 172.21.15.43:0/1318065400 conn(0x7fc49809cd40 msgr2=0x7fc49815ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:51.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.290+0000 7fc49ff09640 1 -- 172.21.15.43:0/1318065400 shutdown_connections 2024-04-25T08:54:51.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.290+0000 7fc49ff09640 1 -- 172.21.15.43:0/1318065400 wait complete. 2024-04-25T08:54:51.290 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:54:51.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.404+0000 7f4344809640 1 Processor -- start 2024-04-25T08:54:51.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.404+0000 7f4344809640 1 -- start start 2024-04-25T08:54:51.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.405+0000 7f4344809640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f43400d04a0 0x7f434015ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:51.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.405+0000 7f4344809640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f434015dec0 0x7f43400d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:51.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.405+0000 7f4344809640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43400d0dc0 con 0x7f43400ce8c0 2024-04-25T08:54:51.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.405+0000 7f4344809640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43400cf110 con 0x7f43400d04a0 2024-04-25T08:54:51.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.405+0000 7f4344809640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43400cf3f0 con 0x7f434015dec0 2024-04-25T08:54:51.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.405+0000 7f433ed9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f43400ce8c0 0x7f43401617a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48360/0 (socket says 172.21.15.43:48360) 2024-04-25T08:54:51.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.405+0000 7f433ed9d640 1 -- 172.21.15.43:0/956907721 learned_addr learned my addr 172.21.15.43:0/956907721 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:51.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.405+0000 7f4337fff640 1 --2- 172.21.15.43:0/956907721 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f434015dec0 0x7f43400d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:51.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.405+0000 7f433e59c640 1 --2- 172.21.15.43:0/956907721 >> v2:172.21.15.193:3300/0 conn(0x7f43400d04a0 0x7f434015ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:51.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.405+0000 7f433dd9b640 1 -- 172.21.15.43:0/956907721 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1017268839 0 0) 0x7f43400d0dc0 con 0x7f43400ce8c0 2024-04-25T08:54:51.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.405+0000 7f433dd9b640 1 -- 172.21.15.43:0/956907721 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f432c003520 con 0x7f43400ce8c0 2024-04-25T08:54:51.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.405+0000 7f4337fff640 1 -- 172.21.15.43:0/956907721 >> v2:172.21.15.193:3300/0 conn(0x7f43400d04a0 msgr2=0x7f434015ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:51.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.405+0000 7f4337fff640 1 --2- 172.21.15.43:0/956907721 >> v2:172.21.15.193:3300/0 conn(0x7f43400d04a0 0x7f434015ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:51.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.405+0000 7f4337fff640 1 -- 172.21.15.43:0/956907721 >> v1:172.21.15.43:6789/0 conn(0x7f43400ce8c0 legacy=0x7f43401617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:51.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.405+0000 7f433dd9b640 1 -- 172.21.15.43:0/956907721 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3997541459 0 0) 0x7f432c003520 con 0x7f43400ce8c0 2024-04-25T08:54:51.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.405+0000 7f4337fff640 1 -- 172.21.15.43:0/956907721 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43400cf6d0 con 0x7f434015dec0 2024-04-25T08:54:51.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.406+0000 7f4337fff640 1 --2- 172.21.15.43:0/956907721 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f434015dec0 0x7f43400d0880 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f43280097a0 tx=0x7f432802a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2504ef4e39e2bfd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:51.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.406+0000 7f433dd9b640 1 -- 172.21.15.43:0/956907721 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4328037070 con 0x7f434015dec0 2024-04-25T08:54:51.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.406+0000 7f433dd9b640 1 -- 172.21.15.43:0/956907721 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f432802aba0 con 0x7f434015dec0 2024-04-25T08:54:51.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.406+0000 7f433dd9b640 1 -- 172.21.15.43:0/956907721 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43380019f0 con 0x7f434015dec0 2024-04-25T08:54:51.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.407+0000 7f4344809640 1 -- 172.21.15.43:0/956907721 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f434015dec0 msgr2=0x7f43400d0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:51.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.407+0000 7f4344809640 1 --2- 172.21.15.43:0/956907721 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f434015dec0 0x7f43400d0880 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f43280097a0 tx=0x7f432802a110 comp rx=0 tx=0).stop 2024-04-25T08:54:51.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.407+0000 7f4344809640 1 -- 172.21.15.43:0/956907721 shutdown_connections 2024-04-25T08:54:51.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.407+0000 7f4344809640 1 --2- 172.21.15.43:0/956907721 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f434015dec0 0x7f43400d0880 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:51.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.407+0000 7f4344809640 1 --2- 172.21.15.43:0/956907721 >> v2:172.21.15.193:3300/0 conn(0x7f43400d04a0 0x7f434015ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:51.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.407+0000 7f4344809640 1 -- 172.21.15.43:0/956907721 >> 172.21.15.43:0/956907721 conn(0x7f434009cd40 msgr2=0x7f43400d3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:51.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.407+0000 7f4344809640 1 -- 172.21.15.43:0/956907721 shutdown_connections 2024-04-25T08:54:51.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.407+0000 7f4344809640 1 -- 172.21.15.43:0/956907721 wait complete. 2024-04-25T08:54:51.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.408+0000 7f4344809640 1 Processor -- start 2024-04-25T08:54:51.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.408+0000 7f4344809640 1 -- start start 2024-04-25T08:54:51.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.409+0000 7f4344809640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43400ce8c0 0x7f43400b6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:51.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.409+0000 7f4344809640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43400d04a0 0x7f43401f7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:51.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.409+0000 7f4344809640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f434015dec0 0x7f434006b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:51.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.409+0000 7f4344809640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43400cecf0 con 0x7f43400d04a0 2024-04-25T08:54:51.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.409+0000 7f4344809640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43400cfc90 con 0x7f43400ce8c0 2024-04-25T08:54:51.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.409+0000 7f4344809640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43400cee70 con 0x7f434015dec0 2024-04-25T08:54:51.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.409+0000 7f4337fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43400d04a0 0x7f43401f7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:51.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.409+0000 7f4337fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43400d04a0 0x7f43401f7e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49602/0 (socket says 172.21.15.43:49602) 2024-04-25T08:54:51.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.409+0000 7f4337fff640 1 -- 172.21.15.43:0/810123347 learned_addr learned my addr 172.21.15.43:0/810123347 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:51.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.409+0000 7f4337fff640 1 -- 172.21.15.43:0/810123347 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f434015dec0 msgr2=0x7f434006b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:51.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.409+0000 7f4337fff640 1 --2- 172.21.15.43:0/810123347 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f434015dec0 0x7f434006b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:51.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.409+0000 7f4337fff640 1 -- 172.21.15.43:0/810123347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43400ce8c0 msgr2=0x7f43400b6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:51.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.409+0000 7f4337fff640 1 --2- 172.21.15.43:0/810123347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43400ce8c0 0x7f43400b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:51.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.409+0000 7f4337fff640 1 -- 172.21.15.43:0/810123347 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43400b4d40 con 0x7f43400d04a0 2024-04-25T08:54:51.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.410+0000 7f4337fff640 1 --2- 172.21.15.43:0/810123347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43400d04a0 0x7f43401f7e60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4328035040 tx=0x7f43280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:51.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.410+0000 7f4336ffd640 1 -- 172.21.15.43:0/810123347 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4328041020 con 0x7f43400d04a0 2024-04-25T08:54:51.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.410+0000 7f4336ffd640 1 -- 172.21.15.43:0/810123347 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f432803d620 con 0x7f43400d04a0 2024-04-25T08:54:51.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.410+0000 7f4344809640 1 -- 172.21.15.43:0/810123347 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f434015b530 con 0x7f43400d04a0 2024-04-25T08:54:51.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.410+0000 7f4336ffd640 1 -- 172.21.15.43:0/810123347 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4328037040 con 0x7f43400d04a0 2024-04-25T08:54:51.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.410+0000 7f4344809640 1 -- 172.21.15.43:0/810123347 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f434015bb80 con 0x7f43400d04a0 2024-04-25T08:54:51.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.411+0000 7f4344809640 1 -- 172.21.15.43:0/810123347 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4340074f60 con 0x7f43400d04a0 2024-04-25T08:54:51.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.411+0000 7f4336ffd640 1 -- 172.21.15.43:0/810123347 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4328044020 con 0x7f43400d04a0 2024-04-25T08:54:51.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.412+0000 7f4336ffd640 1 --2- 172.21.15.43:0/810123347 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f432006b4d0 0x7f432006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:51.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.412+0000 7f4336ffd640 1 -- 172.21.15.43:0/810123347 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f43280aa7e0 con 0x7f43400d04a0 2024-04-25T08:54:51.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.412+0000 7f433e59c640 1 --2- 172.21.15.43:0/810123347 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f432006b4d0 0x7f432006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:51.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.413+0000 7f433e59c640 1 --2- 172.21.15.43:0/810123347 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f432006b4d0 0x7f432006d990 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4330006a90 tx=0x7f4330006a20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:51.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.416+0000 7f4336ffd640 1 -- 172.21.15.43:0/810123347 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f432804b030 con 0x7f43400d04a0 2024-04-25T08:54:51.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.598+0000 7f4344809640 1 -- 172.21.15.43:0/810123347 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f43400b61e0 con 0x7f43400d04a0 2024-04-25T08:54:51.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.598+0000 7f4336ffd640 1 -- 172.21.15.43:0/810123347 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f43280795e0 con 0x7f43400d04a0 2024-04-25T08:54:51.588 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T08:54:51.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.600+0000 7f4344809640 1 -- 172.21.15.43:0/810123347 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f432006b4d0 msgr2=0x7f432006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:51.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.600+0000 7f4344809640 1 --2- 172.21.15.43:0/810123347 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f432006b4d0 0x7f432006d990 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4330006a90 tx=0x7f4330006a20 comp rx=0 tx=0).stop 2024-04-25T08:54:51.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.600+0000 7f4344809640 1 -- 172.21.15.43:0/810123347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43400d04a0 msgr2=0x7f43401f7e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:51.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.600+0000 7f4344809640 1 --2- 172.21.15.43:0/810123347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43400d04a0 0x7f43401f7e60 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f4328035040 tx=0x7f43280046c0 comp rx=0 tx=0).stop 2024-04-25T08:54:51.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.600+0000 7f4344809640 1 -- 172.21.15.43:0/810123347 shutdown_connections 2024-04-25T08:54:51.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.600+0000 7f4344809640 1 --2- 172.21.15.43:0/810123347 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f434015dec0 0x7f434006b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:51.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.600+0000 7f4344809640 1 --2- 172.21.15.43:0/810123347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43400d04a0 0x7f43401f7e60 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:51.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.600+0000 7f4344809640 1 --2- 172.21.15.43:0/810123347 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f432006b4d0 0x7f432006d990 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:51.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.600+0000 7f4344809640 1 --2- 172.21.15.43:0/810123347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43400ce8c0 0x7f43400b6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:51.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.600+0000 7f4344809640 1 -- 172.21.15.43:0/810123347 >> 172.21.15.43:0/810123347 conn(0x7f434009cd40 msgr2=0x7f4340160ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:51.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.600+0000 7f4344809640 1 -- 172.21.15.43:0/810123347 shutdown_connections 2024-04-25T08:54:51.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.600+0000 7f4344809640 1 -- 172.21.15.43:0/810123347 wait complete. 2024-04-25T08:54:51.601 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:54:51.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.714+0000 7f0c1445a640 1 Processor -- start 2024-04-25T08:54:51.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.714+0000 7f0c1445a640 1 -- start start 2024-04-25T08:54:51.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.715+0000 7f0c1445a640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0c0c15df50 0x7f0c0c1619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:51.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.715+0000 7f0c1445a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0c0c161f30 0x7f0c0c162380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:51.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.715+0000 7f0c1445a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c0c0d24a0 con 0x7f0c0c0c7cf0 2024-04-25T08:54:51.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.715+0000 7f0c1445a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c0c15e330 con 0x7f0c0c161f30 2024-04-25T08:54:51.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.715+0000 7f0c1445a640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c0c0d2740 con 0x7f0c0c15df50 2024-04-25T08:54:51.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.715+0000 7f0c119f5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0c0c161f30 0x7f0c0c162380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:51.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.715+0000 7f0c129f7640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f0c0c0c7cf0 0x7f0c0c0c8100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48374/0 (socket says 172.21.15.43:48374) 2024-04-25T08:54:51.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.715+0000 7f0c129f7640 1 -- 172.21.15.43:0/998359498 learned_addr learned my addr 172.21.15.43:0/998359498 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:51.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.716+0000 7f0c121f6640 1 --2- 172.21.15.43:0/998359498 >> v2:172.21.15.193:3300/0 conn(0x7f0c0c15df50 0x7f0c0c1619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:51.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.716+0000 7f0c119f5640 1 -- 172.21.15.43:0/998359498 >> v2:172.21.15.193:3300/0 conn(0x7f0c0c15df50 msgr2=0x7f0c0c1619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:51.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.716+0000 7f0c119f5640 1 --2- 172.21.15.43:0/998359498 >> v2:172.21.15.193:3300/0 conn(0x7f0c0c15df50 0x7f0c0c1619f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:51.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.716+0000 7f0c119f5640 1 -- 172.21.15.43:0/998359498 >> v1:172.21.15.43:6789/0 conn(0x7f0c0c0c7cf0 legacy=0x7f0c0c0c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:51.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.716+0000 7f0c111f4640 1 -- 172.21.15.43:0/998359498 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1371314103 0 0) 0x7f0c0c0d24a0 con 0x7f0c0c0c7cf0 2024-04-25T08:54:51.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.716+0000 7f0c119f5640 1 -- 172.21.15.43:0/998359498 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c0c0d2a20 con 0x7f0c0c161f30 2024-04-25T08:54:51.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.717+0000 7f0c119f5640 1 --2- 172.21.15.43:0/998359498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c0c161f30 0x7f0c0c162380 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c08c880 tx=0x7f0c0002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c938a16324f7ae82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:51.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.717+0000 7f0c111f4640 1 -- 172.21.15.43:0/998359498 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c00037070 con 0x7f0c0c161f30 2024-04-25T08:54:51.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.717+0000 7f0c111f4640 1 -- 172.21.15.43:0/998359498 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c00032440 con 0x7f0c0c161f30 2024-04-25T08:54:51.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.717+0000 7f0c111f4640 1 -- 172.21.15.43:0/998359498 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c00032810 con 0x7f0c0c161f30 2024-04-25T08:54:51.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.717+0000 7f0c1445a640 1 -- 172.21.15.43:0/998359498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c0c161f30 msgr2=0x7f0c0c162380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:51.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.717+0000 7f0c1445a640 1 --2- 172.21.15.43:0/998359498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c0c161f30 0x7f0c0c162380 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c08c880 tx=0x7f0c0002a1a0 comp rx=0 tx=0).stop 2024-04-25T08:54:51.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.717+0000 7f0c1445a640 1 -- 172.21.15.43:0/998359498 shutdown_connections 2024-04-25T08:54:51.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.717+0000 7f0c1445a640 1 --2- 172.21.15.43:0/998359498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c0c161f30 0x7f0c0c162380 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:51.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.717+0000 7f0c1445a640 1 --2- 172.21.15.43:0/998359498 >> v2:172.21.15.193:3300/0 conn(0x7f0c0c15df50 0x7f0c0c1619f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:51.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.717+0000 7f0c1445a640 1 -- 172.21.15.43:0/998359498 >> 172.21.15.43:0/998359498 conn(0x7f0c0c0d3a30 msgr2=0x7f0c0c0d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:51.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.717+0000 7f0c1445a640 1 -- 172.21.15.43:0/998359498 shutdown_connections 2024-04-25T08:54:51.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.717+0000 7f0c1445a640 1 -- 172.21.15.43:0/998359498 wait complete. 2024-04-25T08:54:51.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.718+0000 7f0c1445a640 1 Processor -- start 2024-04-25T08:54:51.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.719+0000 7f0c1445a640 1 -- start start 2024-04-25T08:54:51.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.719+0000 7f0c1445a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0c0c0c7cf0 0x7f0c0c161720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:51.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.719+0000 7f0c1445a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c0c15df50 0x7f0c0c159010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:51.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.719+0000 7f0c1445a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c0c161f30 0x7f0c0c159550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:51.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.719+0000 7f0c1445a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c0d2190 con 0x7f0c0c161f30 2024-04-25T08:54:51.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.719+0000 7f0c1445a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c0c77d0 con 0x7f0c0c15df50 2024-04-25T08:54:51.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.719+0000 7f0c1445a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c0c0cf670 con 0x7f0c0c0c7cf0 2024-04-25T08:54:51.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.719+0000 7f0c129f7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c0c161f30 0x7f0c0c159550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:51.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.719+0000 7f0c129f7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c0c161f30 0x7f0c0c159550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49626/0 (socket says 172.21.15.43:49626) 2024-04-25T08:54:51.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.719+0000 7f0c129f7640 1 -- 172.21.15.43:0/756587982 learned_addr learned my addr 172.21.15.43:0/756587982 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:51.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.720+0000 7f0c119f5640 1 --2- 172.21.15.43:0/756587982 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c0c15df50 0x7f0c0c159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:51.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.720+0000 7f0c121f6640 1 --2- 172.21.15.43:0/756587982 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0c0c0c7cf0 0x7f0c0c161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:51.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.720+0000 7f0c129f7640 1 -- 172.21.15.43:0/756587982 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0c0c0c7cf0 msgr2=0x7f0c0c161720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:51.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.720+0000 7f0c129f7640 1 --2- 172.21.15.43:0/756587982 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0c0c0c7cf0 0x7f0c0c161720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:51.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.720+0000 7f0c129f7640 1 -- 172.21.15.43:0/756587982 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c0c15df50 msgr2=0x7f0c0c159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:51.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.720+0000 7f0c129f7640 1 --2- 172.21.15.43:0/756587982 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c0c15df50 0x7f0c0c159010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:51.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.720+0000 7f0c129f7640 1 -- 172.21.15.43:0/756587982 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c0c0cabf0 con 0x7f0c0c161f30 2024-04-25T08:54:51.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.720+0000 7f0c121f6640 1 --2- 172.21.15.43:0/756587982 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0c0c0c7cf0 0x7f0c0c161720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:51.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.720+0000 7f0c119f5640 1 --2- 172.21.15.43:0/756587982 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c0c15df50 0x7f0c0c159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:51.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.720+0000 7f0c129f7640 1 --2- 172.21.15.43:0/756587982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c0c161f30 0x7f0c0c159550 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc00caf0 tx=0x7f0bfc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:51.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.720+0000 7f0c0affd640 1 -- 172.21.15.43:0/756587982 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc00aca0 con 0x7f0c0c161f30 2024-04-25T08:54:51.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.720+0000 7f0c0affd640 1 -- 172.21.15.43:0/756587982 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bfc00ae40 con 0x7f0c0c161f30 2024-04-25T08:54:51.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.720+0000 7f0c0affd640 1 -- 172.21.15.43:0/756587982 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc012620 con 0x7f0c0c161f30 2024-04-25T08:54:51.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.720+0000 7f0c1445a640 1 -- 172.21.15.43:0/756587982 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c0c0caed0 con 0x7f0c0c161f30 2024-04-25T08:54:51.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.720+0000 7f0c1445a640 1 -- 172.21.15.43:0/756587982 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c0c0cb490 con 0x7f0c0c161f30 2024-04-25T08:54:51.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.721+0000 7f0c1445a640 1 -- 172.21.15.43:0/756587982 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c0c0680f0 con 0x7f0c0c161f30 2024-04-25T08:54:51.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.722+0000 7f0c0affd640 1 -- 172.21.15.43:0/756587982 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0bfc012880 con 0x7f0c0c161f30 2024-04-25T08:54:51.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.722+0000 7f0c0affd640 1 --2- 172.21.15.43:0/756587982 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0bf806b4d0 0x7f0bf806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:51.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.722+0000 7f0c0affd640 1 -- 172.21.15.43:0/756587982 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0bfc08a810 con 0x7f0c0c161f30 2024-04-25T08:54:51.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.722+0000 7f0c121f6640 1 --2- 172.21.15.43:0/756587982 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0bf806b4d0 0x7f0bf806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:51.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.723+0000 7f0c121f6640 1 --2- 172.21.15.43:0/756587982 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0bf806b4d0 0x7f0bf806d990 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c15e980 tx=0x7f0c04008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:51.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.726+0000 7f0c0affd640 1 -- 172.21.15.43:0/756587982 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0bfc059720 con 0x7f0c0c161f30 2024-04-25T08:54:51.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.923+0000 7f0c1445a640 1 -- 172.21.15.43:0/756587982 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0c0c0ca400 con 0x7f0c0c161f30 2024-04-25T08:54:51.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.923+0000 7f0c0affd640 1 -- 172.21.15.43:0/756587982 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0bfc059020 con 0x7f0c0c161f30 2024-04-25T08:54:51.913 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-04-25T08:54:51.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.925+0000 7f0c1445a640 1 -- 172.21.15.43:0/756587982 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0bf806b4d0 msgr2=0x7f0bf806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:51.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.925+0000 7f0c1445a640 1 --2- 172.21.15.43:0/756587982 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0bf806b4d0 0x7f0bf806d990 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c15e980 tx=0x7f0c04008040 comp rx=0 tx=0).stop 2024-04-25T08:54:51.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.925+0000 7f0c1445a640 1 -- 172.21.15.43:0/756587982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c0c161f30 msgr2=0x7f0c0c159550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:51.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.925+0000 7f0c1445a640 1 --2- 172.21.15.43:0/756587982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c0c161f30 0x7f0c0c159550 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc00caf0 tx=0x7f0bfc00a060 comp rx=0 tx=0).stop 2024-04-25T08:54:51.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.925+0000 7f0c1445a640 1 -- 172.21.15.43:0/756587982 shutdown_connections 2024-04-25T08:54:51.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.925+0000 7f0c1445a640 1 --2- 172.21.15.43:0/756587982 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0c0c161f30 0x7f0c0c159550 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:51.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.925+0000 7f0c1445a640 1 --2- 172.21.15.43:0/756587982 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0bf806b4d0 0x7f0bf806d990 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:51.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.925+0000 7f0c1445a640 1 --2- 172.21.15.43:0/756587982 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0c0c15df50 0x7f0c0c159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:51.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.925+0000 7f0c1445a640 1 --2- 172.21.15.43:0/756587982 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0c0c0c7cf0 0x7f0c0c161720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:51.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.925+0000 7f0c1445a640 1 -- 172.21.15.43:0/756587982 >> 172.21.15.43:0/756587982 conn(0x7f0c0c0d3a30 msgr2=0x7f0c0c0c9430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:51.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.925+0000 7f0c1445a640 1 -- 172.21.15.43:0/756587982 shutdown_connections 2024-04-25T08:54:51.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:51.925+0000 7f0c1445a640 1 -- 172.21.15.43:0/756587982 wait complete. 2024-04-25T08:54:51.926 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T08:54:51.926 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:54:51.945 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:54:52.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.056+0000 7fc3157e2640 1 Processor -- start 2024-04-25T08:54:52.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.056+0000 7fc3157e2640 1 -- start start 2024-04-25T08:54:52.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.056+0000 7fc3157e2640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc3100d21f0 0x7fc3100cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.056+0000 7fc3157e2640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc31006e530 0x7fc31006e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.057+0000 7fc3157e2640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3100cef00 con 0x7fc31015f4e0 2024-04-25T08:54:52.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.057+0000 7fc3157e2640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3100d26a0 con 0x7fc3100d21f0 2024-04-25T08:54:52.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.057+0000 7fc3157e2640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3100cc5b0 con 0x7fc31006e530 2024-04-25T08:54:52.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.057+0000 7fc30e7fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc3100d21f0 0x7fc3100cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.057+0000 7fc30effd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc31015f4e0 0x7fc31015f8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48380/0 (socket says 172.21.15.43:48380) 2024-04-25T08:54:52.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.057+0000 7fc30effd640 1 -- 172.21.15.43:0/4145649132 learned_addr learned my addr 172.21.15.43:0/4145649132 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:52.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.057+0000 7fc30e7fc640 1 -- 172.21.15.43:0/4145649132 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc31006e530 msgr2=0x7fc31006e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:52.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.057+0000 7fc30e7fc640 1 --2- 172.21.15.43:0/4145649132 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc31006e530 0x7fc31006e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.057+0000 7fc30e7fc640 1 -- 172.21.15.43:0/4145649132 >> v1:172.21.15.43:6789/0 conn(0x7fc31015f4e0 legacy=0x7fc31015f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:52.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.057+0000 7fc30e7fc640 1 -- 172.21.15.43:0/4145649132 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3100cc890 con 0x7fc3100d21f0 2024-04-25T08:54:52.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.058+0000 7fc30e7fc640 1 --2- 172.21.15.43:0/4145649132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3100d21f0 0x7fc3100cd090 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc3040097a0 tx=0x7fc30402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97467f354d7b8cae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:52.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.058+0000 7fc30dffb640 1 -- 172.21.15.43:0/4145649132 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc304037070 con 0x7fc3100d21f0 2024-04-25T08:54:52.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.058+0000 7fc30dffb640 1 -- 172.21.15.43:0/4145649132 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc304032440 con 0x7fc3100d21f0 2024-04-25T08:54:52.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.058+0000 7fc30dffb640 1 -- 172.21.15.43:0/4145649132 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc304032810 con 0x7fc3100d21f0 2024-04-25T08:54:52.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.058+0000 7fc3157e2640 1 -- 172.21.15.43:0/4145649132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3100d21f0 msgr2=0x7fc3100cd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:52.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.058+0000 7fc3157e2640 1 --2- 172.21.15.43:0/4145649132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3100d21f0 0x7fc3100cd090 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc3040097a0 tx=0x7fc30402a180 comp rx=0 tx=0).stop 2024-04-25T08:54:52.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.058+0000 7fc3157e2640 1 -- 172.21.15.43:0/4145649132 shutdown_connections 2024-04-25T08:54:52.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.058+0000 7fc3157e2640 1 --2- 172.21.15.43:0/4145649132 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc31006e530 0x7fc31006e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.058+0000 7fc3157e2640 1 --2- 172.21.15.43:0/4145649132 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3100d21f0 0x7fc3100cd090 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.058+0000 7fc3157e2640 1 -- 172.21.15.43:0/4145649132 >> 172.21.15.43:0/4145649132 conn(0x7fc3100d1700 msgr2=0x7fc3100d1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:52.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.058+0000 7fc3157e2640 1 -- 172.21.15.43:0/4145649132 shutdown_connections 2024-04-25T08:54:52.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.058+0000 7fc3157e2640 1 -- 172.21.15.43:0/4145649132 wait complete. 2024-04-25T08:54:52.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.060+0000 7fc3157e2640 1 Processor -- start 2024-04-25T08:54:52.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.060+0000 7fc3157e2640 1 -- start start 2024-04-25T08:54:52.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.060+0000 7fc3157e2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc31006e530 0x7fc3100b68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.060+0000 7fc3157e2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3100d21f0 0x7fc3101f5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.061+0000 7fc3157e2640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc31015f4e0 0x7fc3101f5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.061+0000 7fc3157e2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3100cd5d0 con 0x7fc31006e530 2024-04-25T08:54:52.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.061+0000 7fc3157e2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3100cc2a0 con 0x7fc3100d21f0 2024-04-25T08:54:52.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.061+0000 7fc3157e2640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc31006ef00 con 0x7fc31015f4e0 2024-04-25T08:54:52.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.061+0000 7fc30effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc31006e530 0x7fc3100b68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.061+0000 7fc30effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc31006e530 0x7fc3100b68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49650/0 (socket says 172.21.15.43:49650) 2024-04-25T08:54:52.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.061+0000 7fc30effd640 1 -- 172.21.15.43:0/3920454461 learned_addr learned my addr 172.21.15.43:0/3920454461 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:52.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.061+0000 7fc30f7fe640 1 --2- 172.21.15.43:0/3920454461 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc31015f4e0 0x7fc3101f5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.061+0000 7fc30e7fc640 1 --2- 172.21.15.43:0/3920454461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3100d21f0 0x7fc3101f5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.061+0000 7fc30effd640 1 -- 172.21.15.43:0/3920454461 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc31015f4e0 msgr2=0x7fc3101f5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:52.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.061+0000 7fc30effd640 1 --2- 172.21.15.43:0/3920454461 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc31015f4e0 0x7fc3101f5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.061+0000 7fc30effd640 1 -- 172.21.15.43:0/3920454461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3100d21f0 msgr2=0x7fc3101f5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:52.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.061+0000 7fc30effd640 1 --2- 172.21.15.43:0/3920454461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3100d21f0 0x7fc3101f5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.061+0000 7fc30effd640 1 -- 172.21.15.43:0/3920454461 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3100b21b0 con 0x7fc31006e530 2024-04-25T08:54:52.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.061+0000 7fc30effd640 1 --2- 172.21.15.43:0/3920454461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc31006e530 0x7fc3100b68e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc30000c3c0 tx=0x7fc30000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:52.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.061+0000 7fc2f77fe640 1 -- 172.21.15.43:0/3920454461 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc300017070 con 0x7fc31006e530 2024-04-25T08:54:52.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.061+0000 7fc2f77fe640 1 -- 172.21.15.43:0/3920454461 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc300005ce0 con 0x7fc31006e530 2024-04-25T08:54:52.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.061+0000 7fc3157e2640 1 -- 172.21.15.43:0/3920454461 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3100b2490 con 0x7fc31006e530 2024-04-25T08:54:52.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.061+0000 7fc2f77fe640 1 -- 172.21.15.43:0/3920454461 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3000072f0 con 0x7fc31006e530 2024-04-25T08:54:52.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.061+0000 7fc3157e2640 1 -- 172.21.15.43:0/3920454461 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3100b2a50 con 0x7fc31006e530 2024-04-25T08:54:52.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.062+0000 7fc3157e2640 1 -- 172.21.15.43:0/3920454461 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc310074ef0 con 0x7fc31006e530 2024-04-25T08:54:52.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.063+0000 7fc2f77fe640 1 -- 172.21.15.43:0/3920454461 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc30001a070 con 0x7fc31006e530 2024-04-25T08:54:52.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.063+0000 7fc2f77fe640 1 --2- 172.21.15.43:0/3920454461 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc2fc06b4d0 0x7fc2fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.063+0000 7fc2f77fe640 1 -- 172.21.15.43:0/3920454461 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fc300089680 con 0x7fc31006e530 2024-04-25T08:54:52.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.064+0000 7fc30e7fc640 1 --2- 172.21.15.43:0/3920454461 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc2fc06b4d0 0x7fc2fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.064+0000 7fc30e7fc640 1 --2- 172.21.15.43:0/3920454461 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc2fc06b4d0 0x7fc2fc06d990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc304034000 tx=0x7fc30402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:52.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.067+0000 7fc2f77fe640 1 -- 172.21.15.43:0/3920454461 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc300058460 con 0x7fc31006e530 2024-04-25T08:54:52.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.248+0000 7fc3157e2640 1 -- 172.21.15.43:0/3920454461 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3100cdef0 con 0x7fc31006e530 2024-04-25T08:54:52.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.249+0000 7fc2f77fe640 1 -- 172.21.15.43:0/3920454461 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16809 (secure 0 0 0) 0x7fc300057d60 con 0x7fc31006e530 2024-04-25T08:54:52.239 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:54:52.240 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":21,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:54:51.843324+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:54:09.757978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:54:10.926008+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.18","mappings":[{"from":3,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2d","mappings":[{"from":7,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:54:52.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.251+0000 7fc3157e2640 1 -- 172.21.15.43:0/3920454461 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc2fc06b4d0 msgr2=0x7fc2fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:52.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.251+0000 7fc3157e2640 1 --2- 172.21.15.43:0/3920454461 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc2fc06b4d0 0x7fc2fc06d990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc304034000 tx=0x7fc30402e000 comp rx=0 tx=0).stop 2024-04-25T08:54:52.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.251+0000 7fc3157e2640 1 -- 172.21.15.43:0/3920454461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc31006e530 msgr2=0x7fc3100b68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:52.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.251+0000 7fc3157e2640 1 --2- 172.21.15.43:0/3920454461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc31006e530 0x7fc3100b68e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc30000c3c0 tx=0x7fc30000c880 comp rx=0 tx=0).stop 2024-04-25T08:54:52.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.251+0000 7fc3157e2640 1 -- 172.21.15.43:0/3920454461 shutdown_connections 2024-04-25T08:54:52.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.251+0000 7fc3157e2640 1 --2- 172.21.15.43:0/3920454461 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc31015f4e0 0x7fc3101f5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.251+0000 7fc3157e2640 1 --2- 172.21.15.43:0/3920454461 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc2fc06b4d0 0x7fc2fc06d990 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.251+0000 7fc3157e2640 1 --2- 172.21.15.43:0/3920454461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3100d21f0 0x7fc3101f5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.251+0000 7fc3157e2640 1 --2- 172.21.15.43:0/3920454461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc31006e530 0x7fc3100b68e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.251+0000 7fc3157e2640 1 -- 172.21.15.43:0/3920454461 >> 172.21.15.43:0/3920454461 conn(0x7fc3100d1700 msgr2=0x7fc310158b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:52.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.251+0000 7fc3157e2640 1 -- 172.21.15.43:0/3920454461 shutdown_connections 2024-04-25T08:54:52.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.251+0000 7fc3157e2640 1 -- 172.21.15.43:0/3920454461 wait complete. 2024-04-25T08:54:52.252 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:54:09.757978+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:54:10.926008+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:54:52.253 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:54:52.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.366+0000 7fca9ca2a640 1 Processor -- start 2024-04-25T08:54:52.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.366+0000 7fca9ca2a640 1 -- start start 2024-04-25T08:54:52.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.367+0000 7fca9ca2a640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fca9806f6a0 0x7fca980b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.367+0000 7fca9ca2a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fca980cd470 0x7fca9806c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.367+0000 7fca9ca2a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca980d2620 con 0x7fca9815ddd0 2024-04-25T08:54:52.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.367+0000 7fca9ca2a640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca980b0b80 con 0x7fca9806f6a0 2024-04-25T08:54:52.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.367+0000 7fca9ca2a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca980b0e60 con 0x7fca980cd470 2024-04-25T08:54:52.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.367+0000 7fca9659c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fca9815ddd0 0x7fca9806ef90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48396/0 (socket says 172.21.15.43:48396) 2024-04-25T08:54:52.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.367+0000 7fca9659c640 1 -- 172.21.15.43:0/3499389854 learned_addr learned my addr 172.21.15.43:0/3499389854 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:52.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.367+0000 7fca96d9d640 1 --2- 172.21.15.43:0/3499389854 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fca980cd470 0x7fca9806c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.367+0000 7fca95d9b640 1 --2- 172.21.15.43:0/3499389854 >> v2:172.21.15.193:3300/0 conn(0x7fca9806f6a0 0x7fca980b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.367+0000 7fca9559a640 1 -- 172.21.15.43:0/3499389854 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3946390575 0 0) 0x7fca980d2620 con 0x7fca9815ddd0 2024-04-25T08:54:52.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.368+0000 7fca9559a640 1 -- 172.21.15.43:0/3499389854 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca84003520 con 0x7fca9815ddd0 2024-04-25T08:54:52.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.368+0000 7fca9559a640 1 -- 172.21.15.43:0/3499389854 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 164183830 0 0) 0x7fca84003520 con 0x7fca9815ddd0 2024-04-25T08:54:52.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.368+0000 7fca96d9d640 1 -- 172.21.15.43:0/3499389854 >> v2:172.21.15.193:3300/0 conn(0x7fca9806f6a0 msgr2=0x7fca980b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:52.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.368+0000 7fca96d9d640 1 --2- 172.21.15.43:0/3499389854 >> v2:172.21.15.193:3300/0 conn(0x7fca9806f6a0 0x7fca980b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.368+0000 7fca96d9d640 1 -- 172.21.15.43:0/3499389854 >> v1:172.21.15.43:6789/0 conn(0x7fca9815ddd0 legacy=0x7fca9806ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:52.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.368+0000 7fca96d9d640 1 -- 172.21.15.43:0/3499389854 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca980cbf40 con 0x7fca980cd470 2024-04-25T08:54:52.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.368+0000 7fca95d9b640 1 --2- 172.21.15.43:0/3499389854 >> v2:172.21.15.193:3300/0 conn(0x7fca9806f6a0 0x7fca980b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:54:52.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.369+0000 7fca96d9d640 1 --2- 172.21.15.43:0/3499389854 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fca980cd470 0x7fca9806c4e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fca900097a0 tx=0x7fca9002a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc0874ac2e28cd6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:52.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.369+0000 7fca9559a640 1 -- 172.21.15.43:0/3499389854 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca90037070 con 0x7fca980cd470 2024-04-25T08:54:52.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.369+0000 7fca9559a640 1 -- 172.21.15.43:0/3499389854 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca9002aba0 con 0x7fca980cd470 2024-04-25T08:54:52.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.369+0000 7fca9559a640 1 -- 172.21.15.43:0/3499389854 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca880019f0 con 0x7fca980cd470 2024-04-25T08:54:52.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.369+0000 7fca9ca2a640 1 -- 172.21.15.43:0/3499389854 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fca980cd470 msgr2=0x7fca9806c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:52.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.369+0000 7fca9ca2a640 1 --2- 172.21.15.43:0/3499389854 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fca980cd470 0x7fca9806c4e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fca900097a0 tx=0x7fca9002a110 comp rx=0 tx=0).stop 2024-04-25T08:54:52.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.369+0000 7fca9ca2a640 1 -- 172.21.15.43:0/3499389854 shutdown_connections 2024-04-25T08:54:52.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.369+0000 7fca9ca2a640 1 --2- 172.21.15.43:0/3499389854 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fca980cd470 0x7fca9806c4e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.369+0000 7fca9ca2a640 1 --2- 172.21.15.43:0/3499389854 >> v2:172.21.15.193:3300/0 conn(0x7fca9806f6a0 0x7fca980b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.369+0000 7fca9ca2a640 1 -- 172.21.15.43:0/3499389854 >> 172.21.15.43:0/3499389854 conn(0x7fca980d17e0 msgr2=0x7fca980d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:52.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.369+0000 7fca9ca2a640 1 -- 172.21.15.43:0/3499389854 shutdown_connections 2024-04-25T08:54:52.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.369+0000 7fca9ca2a640 1 -- 172.21.15.43:0/3499389854 wait complete. 2024-04-25T08:54:52.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.371+0000 7fca9ca2a640 1 Processor -- start 2024-04-25T08:54:52.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.371+0000 7fca9ca2a640 1 -- start start 2024-04-25T08:54:52.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.371+0000 7fca9ca2a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca9806f6a0 0x7fca981e9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.371+0000 7fca9ca2a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca980cd470 0x7fca980c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.371+0000 7fca9ca2a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fca9815ddd0 0x7fca980c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.371+0000 7fca9ca2a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9806cdb0 con 0x7fca9806f6a0 2024-04-25T08:54:52.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.371+0000 7fca9ca2a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca9806cc30 con 0x7fca980cd470 2024-04-25T08:54:52.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.371+0000 7fca9ca2a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca9806cf30 con 0x7fca9815ddd0 2024-04-25T08:54:52.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.372+0000 7fca9659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca9806f6a0 0x7fca981e9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.372+0000 7fca9659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca9806f6a0 0x7fca981e9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49662/0 (socket says 172.21.15.43:49662) 2024-04-25T08:54:52.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.372+0000 7fca9659c640 1 -- 172.21.15.43:0/1682812120 learned_addr learned my addr 172.21.15.43:0/1682812120 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:52.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.372+0000 7fca96d9d640 1 --2- 172.21.15.43:0/1682812120 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fca9815ddd0 0x7fca980c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.372+0000 7fca95d9b640 1 --2- 172.21.15.43:0/1682812120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca980cd470 0x7fca980c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.372+0000 7fca9659c640 1 -- 172.21.15.43:0/1682812120 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fca9815ddd0 msgr2=0x7fca980c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:52.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.372+0000 7fca9659c640 1 --2- 172.21.15.43:0/1682812120 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fca9815ddd0 0x7fca980c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.372+0000 7fca9659c640 1 -- 172.21.15.43:0/1682812120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca980cd470 msgr2=0x7fca980c7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:52.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.372+0000 7fca9659c640 1 --2- 172.21.15.43:0/1682812120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca980cd470 0x7fca980c7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.372+0000 7fca9659c640 1 -- 172.21.15.43:0/1682812120 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca98153470 con 0x7fca9806f6a0 2024-04-25T08:54:52.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.372+0000 7fca95d9b640 1 --2- 172.21.15.43:0/1682812120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca980cd470 0x7fca980c7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:52.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.372+0000 7fca9659c640 1 --2- 172.21.15.43:0/1682812120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca9806f6a0 0x7fca981e9830 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fca8800cc40 tx=0x7fca8800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:52.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.372+0000 7fca8effd640 1 -- 172.21.15.43:0/1682812120 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca88017070 con 0x7fca9806f6a0 2024-04-25T08:54:52.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.372+0000 7fca8effd640 1 -- 172.21.15.43:0/1682812120 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca8800ce50 con 0x7fca9806f6a0 2024-04-25T08:54:52.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.372+0000 7fca9ca2a640 1 -- 172.21.15.43:0/1682812120 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca980d0890 con 0x7fca9806f6a0 2024-04-25T08:54:52.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.372+0000 7fca8effd640 1 -- 172.21.15.43:0/1682812120 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca88012600 con 0x7fca9806f6a0 2024-04-25T08:54:52.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.372+0000 7fca9ca2a640 1 -- 172.21.15.43:0/1682812120 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca980d0e10 con 0x7fca9806f6a0 2024-04-25T08:54:52.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.373+0000 7fca9ca2a640 1 -- 172.21.15.43:0/1682812120 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca98073130 con 0x7fca9806f6a0 2024-04-25T08:54:52.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.374+0000 7fca8effd640 1 -- 172.21.15.43:0/1682812120 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fca8801a070 con 0x7fca9806f6a0 2024-04-25T08:54:52.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.374+0000 7fca8effd640 1 --2- 172.21.15.43:0/1682812120 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fca7806b4d0 0x7fca7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.374+0000 7fca8effd640 1 -- 172.21.15.43:0/1682812120 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fca88089ff0 con 0x7fca9806f6a0 2024-04-25T08:54:52.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.375+0000 7fca95d9b640 1 --2- 172.21.15.43:0/1682812120 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fca7806b4d0 0x7fca7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.375+0000 7fca95d9b640 1 --2- 172.21.15.43:0/1682812120 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fca7806b4d0 0x7fca7806d990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fca980c76d0 tx=0x7fca80008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:52.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.379+0000 7fca8effd640 1 -- 172.21.15.43:0/1682812120 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fca98073130 con 0x7fca9806f6a0 2024-04-25T08:54:52.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.559+0000 7fca9ca2a640 1 -- 172.21.15.43:0/1682812120 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fca98074f60 con 0x7fca9806f6a0 2024-04-25T08:54:52.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.560+0000 7fca8effd640 1 -- 172.21.15.43:0/1682812120 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fca88058820 con 0x7fca9806f6a0 2024-04-25T08:54:52.550 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T08:54:52.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.562+0000 7fca9ca2a640 1 -- 172.21.15.43:0/1682812120 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fca7806b4d0 msgr2=0x7fca7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:52.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.562+0000 7fca9ca2a640 1 --2- 172.21.15.43:0/1682812120 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fca7806b4d0 0x7fca7806d990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fca980c76d0 tx=0x7fca80008040 comp rx=0 tx=0).stop 2024-04-25T08:54:52.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.562+0000 7fca9ca2a640 1 -- 172.21.15.43:0/1682812120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca9806f6a0 msgr2=0x7fca981e9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:52.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.562+0000 7fca9ca2a640 1 --2- 172.21.15.43:0/1682812120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca9806f6a0 0x7fca981e9830 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fca8800cc40 tx=0x7fca8800a430 comp rx=0 tx=0).stop 2024-04-25T08:54:52.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.562+0000 7fca9ca2a640 1 -- 172.21.15.43:0/1682812120 shutdown_connections 2024-04-25T08:54:52.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.562+0000 7fca9ca2a640 1 --2- 172.21.15.43:0/1682812120 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fca9815ddd0 0x7fca980c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.562+0000 7fca9ca2a640 1 --2- 172.21.15.43:0/1682812120 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fca7806b4d0 0x7fca7806d990 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.562+0000 7fca9ca2a640 1 --2- 172.21.15.43:0/1682812120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca980cd470 0x7fca980c7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.562+0000 7fca9ca2a640 1 --2- 172.21.15.43:0/1682812120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca9806f6a0 0x7fca981e9830 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.562+0000 7fca9ca2a640 1 -- 172.21.15.43:0/1682812120 >> 172.21.15.43:0/1682812120 conn(0x7fca980d17e0 msgr2=0x7fca9815cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:52.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.562+0000 7fca9ca2a640 1 -- 172.21.15.43:0/1682812120 shutdown_connections 2024-04-25T08:54:52.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.562+0000 7fca9ca2a640 1 -- 172.21.15.43:0/1682812120 wait complete. 2024-04-25T08:54:52.563 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:54:52.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.678+0000 7fb4136ac640 1 Processor -- start 2024-04-25T08:54:52.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.678+0000 7fb4136ac640 1 -- start start 2024-04-25T08:54:52.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.679+0000 7fb4136ac640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb40c15a960 0x7fb40c0b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.679+0000 7fb4136ac640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb40c0b3ed0 0x7fb40c0ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.679+0000 7fb4136ac640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb40c0aec70 0x7fb40c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.679+0000 7fb4136ac640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb40c0af0c0 con 0x7fb40c15a960 2024-04-25T08:54:52.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.679+0000 7fb4136ac640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb40c15b110 con 0x7fb40c0aec70 2024-04-25T08:54:52.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.679+0000 7fb4136ac640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb40c15b3f0 con 0x7fb40c0b3ed0 2024-04-25T08:54:52.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.679+0000 7fb411448640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb40c15a960 0x7fb40c0b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.679+0000 7fb411448640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb40c15a960 0x7fb40c0b3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49666/0 (socket says 172.21.15.43:49666) 2024-04-25T08:54:52.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.679+0000 7fb411448640 1 -- 172.21.15.43:0/1538456731 learned_addr learned my addr 172.21.15.43:0/1538456731 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:52.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.679+0000 7fb410c47640 1 --2- 172.21.15.43:0/1538456731 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb40c0b3ed0 0x7fb40c0ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.679+0000 7fb411c49640 1 --2- 172.21.15.43:0/1538456731 >> v2:172.21.15.193:3300/0 conn(0x7fb40c0aec70 0x7fb40c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.679+0000 7fb411448640 1 -- 172.21.15.43:0/1538456731 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb40c0b3ed0 msgr2=0x7fb40c0ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:52.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.679+0000 7fb411448640 1 --2- 172.21.15.43:0/1538456731 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb40c0b3ed0 0x7fb40c0ae730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.679+0000 7fb411448640 1 -- 172.21.15.43:0/1538456731 >> v2:172.21.15.193:3300/0 conn(0x7fb40c0aec70 msgr2=0x7fb40c06d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:52.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.679+0000 7fb411448640 1 --2- 172.21.15.43:0/1538456731 >> v2:172.21.15.193:3300/0 conn(0x7fb40c0aec70 0x7fb40c06d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.679+0000 7fb411448640 1 -- 172.21.15.43:0/1538456731 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb40c15b6d0 con 0x7fb40c15a960 2024-04-25T08:54:52.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.679+0000 7fb410c47640 1 --2- 172.21.15.43:0/1538456731 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb40c0b3ed0 0x7fb40c0ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:52.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.679+0000 7fb411c49640 1 --2- 172.21.15.43:0/1538456731 >> v2:172.21.15.193:3300/0 conn(0x7fb40c0aec70 0x7fb40c06d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:52.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.680+0000 7fb411448640 1 --2- 172.21.15.43:0/1538456731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb40c15a960 0x7fb40c0b3990 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb40c0b2810 tx=0x7fb40802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61c6c5c27aa95e43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:52.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.680+0000 7fb403fff640 1 -- 172.21.15.43:0/1538456731 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb408037070 con 0x7fb40c15a960 2024-04-25T08:54:52.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.680+0000 7fb403fff640 1 -- 172.21.15.43:0/1538456731 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb408032440 con 0x7fb40c15a960 2024-04-25T08:54:52.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.680+0000 7fb403fff640 1 -- 172.21.15.43:0/1538456731 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb408032810 con 0x7fb40c15a960 2024-04-25T08:54:52.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.681+0000 7fb4136ac640 1 -- 172.21.15.43:0/1538456731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb40c15a960 msgr2=0x7fb40c0b3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:52.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.681+0000 7fb4136ac640 1 --2- 172.21.15.43:0/1538456731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb40c15a960 0x7fb40c0b3990 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb40c0b2810 tx=0x7fb40802a1e0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.681+0000 7fb4136ac640 1 -- 172.21.15.43:0/1538456731 shutdown_connections 2024-04-25T08:54:52.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.681+0000 7fb4136ac640 1 --2- 172.21.15.43:0/1538456731 >> v2:172.21.15.193:3300/0 conn(0x7fb40c0aec70 0x7fb40c06d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.681+0000 7fb4136ac640 1 --2- 172.21.15.43:0/1538456731 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb40c0b3ed0 0x7fb40c0ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.681+0000 7fb4136ac640 1 --2- 172.21.15.43:0/1538456731 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb40c15a960 0x7fb40c0b3990 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.681+0000 7fb4136ac640 1 -- 172.21.15.43:0/1538456731 >> 172.21.15.43:0/1538456731 conn(0x7fb40c06ab30 msgr2=0x7fb40c06af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:52.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.681+0000 7fb4136ac640 1 -- 172.21.15.43:0/1538456731 shutdown_connections 2024-04-25T08:54:52.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.681+0000 7fb4136ac640 1 -- 172.21.15.43:0/1538456731 wait complete. 2024-04-25T08:54:52.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.682+0000 7fb4136ac640 1 Processor -- start 2024-04-25T08:54:52.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.682+0000 7fb4136ac640 1 -- start start 2024-04-25T08:54:52.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.683+0000 7fb4136ac640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb40c0aec70 0x7fb40c0a9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.683+0000 7fb4136ac640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb40c0b3ed0 0x7fb40c145f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.683+0000 7fb4136ac640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb40c15a960 0x7fb40c1f4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.683+0000 7fb4136ac640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40c0cd940 con 0x7fb40c0aec70 2024-04-25T08:54:52.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.683+0000 7fb4136ac640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb40c0cc440 con 0x7fb40c0b3ed0 2024-04-25T08:54:52.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.683+0000 7fb4136ac640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb40c15ad40 con 0x7fb40c15a960 2024-04-25T08:54:52.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.683+0000 7fb411448640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb40c0aec70 0x7fb40c0a9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.683+0000 7fb411448640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb40c0aec70 0x7fb40c0a9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49676/0 (socket says 172.21.15.43:49676) 2024-04-25T08:54:52.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.683+0000 7fb411448640 1 -- 172.21.15.43:0/2908378654 learned_addr learned my addr 172.21.15.43:0/2908378654 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:52.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.683+0000 7fb411c49640 1 --2- 172.21.15.43:0/2908378654 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb40c15a960 0x7fb40c1f4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.683+0000 7fb410c47640 1 --2- 172.21.15.43:0/2908378654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb40c0b3ed0 0x7fb40c145f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.683+0000 7fb411448640 1 -- 172.21.15.43:0/2908378654 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb40c15a960 msgr2=0x7fb40c1f4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:52.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.683+0000 7fb411448640 1 --2- 172.21.15.43:0/2908378654 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb40c15a960 0x7fb40c1f4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.683+0000 7fb411448640 1 -- 172.21.15.43:0/2908378654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb40c0b3ed0 msgr2=0x7fb40c145f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:52.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.683+0000 7fb411448640 1 --2- 172.21.15.43:0/2908378654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb40c0b3ed0 0x7fb40c145f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.683+0000 7fb411448640 1 -- 172.21.15.43:0/2908378654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb40c156770 con 0x7fb40c0aec70 2024-04-25T08:54:52.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.683+0000 7fb411448640 1 --2- 172.21.15.43:0/2908378654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb40c0aec70 0x7fb40c0a9ee0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb40802a220 tx=0x7fb4080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:52.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.683+0000 7fb401ffb640 1 -- 172.21.15.43:0/2908378654 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb408037040 con 0x7fb40c0aec70 2024-04-25T08:54:52.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.683+0000 7fb401ffb640 1 -- 172.21.15.43:0/2908378654 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb408042070 con 0x7fb40c0aec70 2024-04-25T08:54:52.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.684+0000 7fb4136ac640 1 -- 172.21.15.43:0/2908378654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb40c1e3a80 con 0x7fb40c0aec70 2024-04-25T08:54:52.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.684+0000 7fb401ffb640 1 -- 172.21.15.43:0/2908378654 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb408030bb0 con 0x7fb40c0aec70 2024-04-25T08:54:52.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.684+0000 7fb4136ac640 1 -- 172.21.15.43:0/2908378654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb40c1e3d40 con 0x7fb40c0aec70 2024-04-25T08:54:52.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.685+0000 7fb4136ac640 1 -- 172.21.15.43:0/2908378654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb40c074f60 con 0x7fb40c0aec70 2024-04-25T08:54:52.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.685+0000 7fb401ffb640 1 -- 172.21.15.43:0/2908378654 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb408047460 con 0x7fb40c0aec70 2024-04-25T08:54:52.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.685+0000 7fb401ffb640 1 --2- 172.21.15.43:0/2908378654 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb3f006b4d0 0x7fb3f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.686+0000 7fb401ffb640 1 -- 172.21.15.43:0/2908378654 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fb4080aae90 con 0x7fb40c0aec70 2024-04-25T08:54:52.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.686+0000 7fb410c47640 1 --2- 172.21.15.43:0/2908378654 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb3f006b4d0 0x7fb3f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.686+0000 7fb410c47640 1 --2- 172.21.15.43:0/2908378654 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb3f006b4d0 0x7fb3f006d990 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb40c0ab8d0 tx=0x7fb3fc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:52.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.690+0000 7fb401ffb640 1 -- 172.21.15.43:0/2908378654 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb408045360 con 0x7fb40c0aec70 2024-04-25T08:54:52.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.871+0000 7fb4136ac640 1 -- 172.21.15.43:0/2908378654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb40c15c3c0 con 0x7fb40c0aec70 2024-04-25T08:54:52.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.872+0000 7fb401ffb640 1 -- 172.21.15.43:0/2908378654 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb408079d10 con 0x7fb40c0aec70 2024-04-25T08:54:52.862 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T08:54:52.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.874+0000 7fb4136ac640 1 -- 172.21.15.43:0/2908378654 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb3f006b4d0 msgr2=0x7fb3f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:52.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.874+0000 7fb4136ac640 1 --2- 172.21.15.43:0/2908378654 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb3f006b4d0 0x7fb3f006d990 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb40c0ab8d0 tx=0x7fb3fc008040 comp rx=0 tx=0).stop 2024-04-25T08:54:52.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.874+0000 7fb4136ac640 1 -- 172.21.15.43:0/2908378654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb40c0aec70 msgr2=0x7fb40c0a9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:52.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.874+0000 7fb4136ac640 1 --2- 172.21.15.43:0/2908378654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb40c0aec70 0x7fb40c0a9ee0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fb40802a220 tx=0x7fb4080046c0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.874+0000 7fb4136ac640 1 -- 172.21.15.43:0/2908378654 shutdown_connections 2024-04-25T08:54:52.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.874+0000 7fb4136ac640 1 --2- 172.21.15.43:0/2908378654 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb40c15a960 0x7fb40c1f4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.874+0000 7fb4136ac640 1 --2- 172.21.15.43:0/2908378654 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb3f006b4d0 0x7fb3f006d990 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.874+0000 7fb4136ac640 1 --2- 172.21.15.43:0/2908378654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb40c0b3ed0 0x7fb40c145f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.874+0000 7fb4136ac640 1 --2- 172.21.15.43:0/2908378654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb40c0aec70 0x7fb40c0a9ee0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.874+0000 7fb4136ac640 1 -- 172.21.15.43:0/2908378654 >> 172.21.15.43:0/2908378654 conn(0x7fb40c06ab30 msgr2=0x7fb40c1599a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:52.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.874+0000 7fb4136ac640 1 -- 172.21.15.43:0/2908378654 shutdown_connections 2024-04-25T08:54:52.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.874+0000 7fb4136ac640 1 -- 172.21.15.43:0/2908378654 wait complete. 2024-04-25T08:54:52.875 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:54:52.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.990+0000 7f9f95bc8640 1 Processor -- start 2024-04-25T08:54:52.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.991+0000 7f9f95bc8640 1 -- start start 2024-04-25T08:54:52.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.991+0000 7f9f95bc8640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9f9015e090 0x7f9f90161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.991+0000 7f9f95bc8640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9f90161fe0 0x7f9f900b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.991+0000 7f9f95bc8640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f90162610 con 0x7f9f900b5aa0 2024-04-25T08:54:52.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.991+0000 7f9f95bc8640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f900b5f00 con 0x7f9f9015e090 2024-04-25T08:54:52.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.991+0000 7f9f95bc8640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f900d1da0 con 0x7f9f90161fe0 2024-04-25T08:54:52.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.992+0000 7f9f8effd640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9f90161fe0 0x7f9f900b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.992+0000 7f9f8f7fe640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9f9015e090 0x7f9f90161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.992+0000 7f9f8effd640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9f90161fe0 0x7f9f900b5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:57816/0 (socket says 172.21.15.43:57816) 2024-04-25T08:54:52.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.992+0000 7f9f8effd640 1 -- 172.21.15.43:0/2325878267 learned_addr learned my addr 172.21.15.43:0/2325878267 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:52.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.992+0000 7f9f8effd640 1 -- 172.21.15.43:0/2325878267 >> v2:172.21.15.193:3300/0 conn(0x7f9f9015e090 msgr2=0x7f9f90161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:52.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.992+0000 7f9f8effd640 1 --2- 172.21.15.43:0/2325878267 >> v2:172.21.15.193:3300/0 conn(0x7f9f9015e090 0x7f9f90161aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.992+0000 7f9f8effd640 1 -- 172.21.15.43:0/2325878267 >> v1:172.21.15.193:6789/0 conn(0x7f9f900b5aa0 legacy=0x7f9f900ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:52.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.992+0000 7f9f8effd640 1 -- 172.21.15.43:0/2325878267 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f900d1750 con 0x7f9f90161fe0 2024-04-25T08:54:52.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.993+0000 7f9f8effd640 1 --2- 172.21.15.43:0/2325878267 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9f90161fe0 0x7f9f900b5560 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9f84009870 tx=0x7f9f8402a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c31c948c0649ed18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:52.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.993+0000 7f9f8e7fc640 1 -- 172.21.15.43:0/2325878267 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f84037070 con 0x7f9f90161fe0 2024-04-25T08:54:52.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.993+0000 7f9f8e7fc640 1 -- 172.21.15.43:0/2325878267 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f8402ae50 con 0x7f9f90161fe0 2024-04-25T08:54:52.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.993+0000 7f9f8e7fc640 1 -- 172.21.15.43:0/2325878267 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f84032860 con 0x7f9f90161fe0 2024-04-25T08:54:52.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.994+0000 7f9f95bc8640 1 -- 172.21.15.43:0/2325878267 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9f90161fe0 msgr2=0x7f9f900b5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:52.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.994+0000 7f9f95bc8640 1 --2- 172.21.15.43:0/2325878267 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9f90161fe0 0x7f9f900b5560 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f9f84009870 tx=0x7f9f8402a300 comp rx=0 tx=0).stop 2024-04-25T08:54:52.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.994+0000 7f9f95bc8640 1 -- 172.21.15.43:0/2325878267 shutdown_connections 2024-04-25T08:54:52.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.994+0000 7f9f95bc8640 1 --2- 172.21.15.43:0/2325878267 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9f90161fe0 0x7f9f900b5560 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.994+0000 7f9f95bc8640 1 --2- 172.21.15.43:0/2325878267 >> v2:172.21.15.193:3300/0 conn(0x7f9f9015e090 0x7f9f90161aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.994+0000 7f9f95bc8640 1 -- 172.21.15.43:0/2325878267 >> 172.21.15.43:0/2325878267 conn(0x7f9f900d0560 msgr2=0x7f9f900d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:52.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.994+0000 7f9f95bc8640 1 -- 172.21.15.43:0/2325878267 shutdown_connections 2024-04-25T08:54:52.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.994+0000 7f9f95bc8640 1 -- 172.21.15.43:0/2325878267 wait complete. 2024-04-25T08:54:52.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.995+0000 7f9f95bc8640 1 Processor -- start 2024-04-25T08:54:52.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.995+0000 7f9f95bc8640 1 -- start start 2024-04-25T08:54:52.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.996+0000 7f9f95bc8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f900b5aa0 0x7f9f901f3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.996+0000 7f9f95bc8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f9015e090 0x7f9f9015e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.996+0000 7f9f95bc8640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9f90161fe0 0x7f9f9015f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.996+0000 7f9f95bc8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f900b3280 con 0x7f9f9015e090 2024-04-25T08:54:52.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.996+0000 7f9f95bc8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f901628f0 con 0x7f9f900b5aa0 2024-04-25T08:54:52.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.996+0000 7f9f95bc8640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f900d3240 con 0x7f9f90161fe0 2024-04-25T08:54:52.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.996+0000 7f9f8effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f9015e090 0x7f9f9015e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.996+0000 7f9f8effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f9015e090 0x7f9f9015e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49682/0 (socket says 172.21.15.43:49682) 2024-04-25T08:54:52.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.996+0000 7f9f8effd640 1 -- 172.21.15.43:0/2219903809 learned_addr learned my addr 172.21.15.43:0/2219903809 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:52.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.996+0000 7f9f8ffff640 1 --2- 172.21.15.43:0/2219903809 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9f90161fe0 0x7f9f9015f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.996+0000 7f9f8f7fe640 1 --2- 172.21.15.43:0/2219903809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f900b5aa0 0x7f9f901f3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.996+0000 7f9f8effd640 1 -- 172.21.15.43:0/2219903809 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9f90161fe0 msgr2=0x7f9f9015f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:52.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.996+0000 7f9f8effd640 1 --2- 172.21.15.43:0/2219903809 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9f90161fe0 0x7f9f9015f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.996+0000 7f9f8effd640 1 -- 172.21.15.43:0/2219903809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f900b5aa0 msgr2=0x7f9f901f3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:52.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.996+0000 7f9f8effd640 1 --2- 172.21.15.43:0/2219903809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f900b5aa0 0x7f9f901f3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:52.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.996+0000 7f9f8effd640 1 -- 172.21.15.43:0/2219903809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f9015eb00 con 0x7f9f9015e090 2024-04-25T08:54:52.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.996+0000 7f9f8effd640 1 --2- 172.21.15.43:0/2219903809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f9015e090 0x7f9f9015e5c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9f84000c00 tx=0x7f9f840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:52.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.996+0000 7f9f77fff640 1 -- 172.21.15.43:0/2219903809 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f84041070 con 0x7f9f9015e090 2024-04-25T08:54:52.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.996+0000 7f9f77fff640 1 -- 172.21.15.43:0/2219903809 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f8403c660 con 0x7f9f9015e090 2024-04-25T08:54:52.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.996+0000 7f9f95bc8640 1 -- 172.21.15.43:0/2219903809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f901f3fc0 con 0x7f9f9015e090 2024-04-25T08:54:52.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.997+0000 7f9f77fff640 1 -- 172.21.15.43:0/2219903809 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f84037040 con 0x7f9f9015e090 2024-04-25T08:54:52.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.997+0000 7f9f95bc8640 1 -- 172.21.15.43:0/2219903809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f901f4580 con 0x7f9f9015e090 2024-04-25T08:54:52.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.997+0000 7f9f95bc8640 1 -- 172.21.15.43:0/2219903809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f9006bf00 con 0x7f9f9015e090 2024-04-25T08:54:52.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.998+0000 7f9f77fff640 1 -- 172.21.15.43:0/2219903809 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9f84044070 con 0x7f9f9015e090 2024-04-25T08:54:52.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.998+0000 7f9f77fff640 1 --2- 172.21.15.43:0/2219903809 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9f7c06b4d0 0x7f9f7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:52.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.998+0000 7f9f77fff640 1 -- 172.21.15.43:0/2219903809 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f9f840aa930 con 0x7f9f9015e090 2024-04-25T08:54:52.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.999+0000 7f9f8f7fe640 1 --2- 172.21.15.43:0/2219903809 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9f7c06b4d0 0x7f9f7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:52.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:52.999+0000 7f9f8f7fe640 1 --2- 172.21.15.43:0/2219903809 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9f7c06b4d0 0x7f9f7c06d990 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9f901f8040 tx=0x7f9f80009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:52.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.003+0000 7f9f77fff640 1 -- 172.21.15.43:0/2219903809 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9f9006bf00 con 0x7f9f9015e090 2024-04-25T08:54:53.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.181+0000 7f9f95bc8640 1 -- 172.21.15.43:0/2219903809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9f90074f60 con 0x7f9f9015e090 2024-04-25T08:54:53.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.182+0000 7f9f77fff640 1 -- 172.21.15.43:0/2219903809 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9f8407f070 con 0x7f9f9015e090 2024-04-25T08:54:53.172 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T08:54:53.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.184+0000 7f9f95bc8640 1 -- 172.21.15.43:0/2219903809 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9f7c06b4d0 msgr2=0x7f9f7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:53.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.184+0000 7f9f95bc8640 1 --2- 172.21.15.43:0/2219903809 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9f7c06b4d0 0x7f9f7c06d990 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9f901f8040 tx=0x7f9f80009190 comp rx=0 tx=0).stop 2024-04-25T08:54:53.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.184+0000 7f9f95bc8640 1 -- 172.21.15.43:0/2219903809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f9015e090 msgr2=0x7f9f9015e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:53.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.184+0000 7f9f95bc8640 1 --2- 172.21.15.43:0/2219903809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f9015e090 0x7f9f9015e5c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9f84000c00 tx=0x7f9f840046c0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.184+0000 7f9f95bc8640 1 -- 172.21.15.43:0/2219903809 shutdown_connections 2024-04-25T08:54:53.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.184+0000 7f9f95bc8640 1 --2- 172.21.15.43:0/2219903809 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9f90161fe0 0x7f9f9015f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.184+0000 7f9f95bc8640 1 --2- 172.21.15.43:0/2219903809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f9015e090 0x7f9f9015e5c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.184+0000 7f9f95bc8640 1 --2- 172.21.15.43:0/2219903809 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9f7c06b4d0 0x7f9f7c06d990 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.184+0000 7f9f95bc8640 1 --2- 172.21.15.43:0/2219903809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f900b5aa0 0x7f9f901f3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.184+0000 7f9f95bc8640 1 -- 172.21.15.43:0/2219903809 >> 172.21.15.43:0/2219903809 conn(0x7f9f900d0560 msgr2=0x7f9f90159b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:53.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.184+0000 7f9f95bc8640 1 -- 172.21.15.43:0/2219903809 shutdown_connections 2024-04-25T08:54:53.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.184+0000 7f9f95bc8640 1 -- 172.21.15.43:0/2219903809 wait complete. 2024-04-25T08:54:53.184 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:54:53.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.296+0000 7f15be65a640 1 Processor -- start 2024-04-25T08:54:53.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.296+0000 7f15be65a640 1 -- start start 2024-04-25T08:54:53.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.297+0000 7f15be65a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f15b815dce0 0x7f15b815ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.297+0000 7f15be65a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f15b80ce550 0x7f15b80d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.297+0000 7f15be65a640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15b80cee80 con 0x7f15b80ce990 2024-04-25T08:54:53.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.297+0000 7f15be65a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15b80d2920 con 0x7f15b80ce550 2024-04-25T08:54:53.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.297+0000 7f15be65a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15b80cf160 con 0x7f15b815dce0 2024-04-25T08:54:53.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.297+0000 7f15b77fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f15b80ce550 0x7f15b80d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:53.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.297+0000 7f15b77fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f15b80ce550 0x7f15b80d03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49696/0 (socket says 172.21.15.43:49696) 2024-04-25T08:54:53.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.297+0000 7f15b77fe640 1 -- 172.21.15.43:0/1912681274 learned_addr learned my addr 172.21.15.43:0/1912681274 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:53.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.297+0000 7f15b7fff640 1 --2- 172.21.15.43:0/1912681274 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f15b815dce0 0x7f15b815ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:53.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.298+0000 7f15b77fe640 1 -- 172.21.15.43:0/1912681274 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f15b815dce0 msgr2=0x7f15b815ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:53.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.298+0000 7f15b77fe640 1 --2- 172.21.15.43:0/1912681274 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f15b815dce0 0x7f15b815ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.298+0000 7f15b77fe640 1 -- 172.21.15.43:0/1912681274 >> v1:172.21.15.193:6789/0 conn(0x7f15b80ce990 legacy=0x7f15b8160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:53.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.298+0000 7f15b77fe640 1 -- 172.21.15.43:0/1912681274 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b80cf440 con 0x7f15b80ce550 2024-04-25T08:54:53.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.298+0000 7f15b77fe640 1 --2- 172.21.15.43:0/1912681274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15b80ce550 0x7f15b80d03e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f15a8009870 tx=0x7f15a802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ed65a423a279703 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:53.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.298+0000 7f15b6ffd640 1 -- 172.21.15.43:0/1912681274 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a8037070 con 0x7f15b80ce550 2024-04-25T08:54:53.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.298+0000 7f15b6ffd640 1 -- 172.21.15.43:0/1912681274 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a8032440 con 0x7f15b80ce550 2024-04-25T08:54:53.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.298+0000 7f15b6ffd640 1 -- 172.21.15.43:0/1912681274 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a8032810 con 0x7f15b80ce550 2024-04-25T08:54:53.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.299+0000 7f15be65a640 1 -- 172.21.15.43:0/1912681274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15b80ce550 msgr2=0x7f15b80d03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:53.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.299+0000 7f15be65a640 1 --2- 172.21.15.43:0/1912681274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15b80ce550 0x7f15b80d03e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f15a8009870 tx=0x7f15a802a1c0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.299+0000 7f15be65a640 1 -- 172.21.15.43:0/1912681274 shutdown_connections 2024-04-25T08:54:53.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.299+0000 7f15be65a640 1 --2- 172.21.15.43:0/1912681274 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15b80ce550 0x7f15b80d03e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.299+0000 7f15be65a640 1 --2- 172.21.15.43:0/1912681274 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f15b815dce0 0x7f15b815ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.299+0000 7f15be65a640 1 -- 172.21.15.43:0/1912681274 >> 172.21.15.43:0/1912681274 conn(0x7f15b809cd00 msgr2=0x7f15b80d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:53.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.299+0000 7f15be65a640 1 -- 172.21.15.43:0/1912681274 shutdown_connections 2024-04-25T08:54:53.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.299+0000 7f15be65a640 1 -- 172.21.15.43:0/1912681274 wait complete. 2024-04-25T08:54:53.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.300+0000 7f15be65a640 1 Processor -- start 2024-04-25T08:54:53.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.300+0000 7f15be65a640 1 -- start start 2024-04-25T08:54:53.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.301+0000 7f15be65a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15b80ce550 0x7f15b814a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.301+0000 7f15be65a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15b80ce990 0x7f15b81f3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.301+0000 7f15be65a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f15b815dce0 0x7f15b8159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.301+0000 7f15be65a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b815e0c0 con 0x7f15b80ce990 2024-04-25T08:54:53.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.301+0000 7f15be65a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b815f6b0 con 0x7f15b80ce550 2024-04-25T08:54:53.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.301+0000 7f15be65a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15b80d0920 con 0x7f15b815dce0 2024-04-25T08:54:53.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.301+0000 7f15b77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15b80ce990 0x7f15b81f3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:53.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.301+0000 7f15b77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15b80ce990 0x7f15b81f3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49712/0 (socket says 172.21.15.43:49712) 2024-04-25T08:54:53.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.301+0000 7f15b77fe640 1 -- 172.21.15.43:0/2164533325 learned_addr learned my addr 172.21.15.43:0/2164533325 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:53.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.302+0000 7f15b7fff640 1 --2- 172.21.15.43:0/2164533325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15b80ce550 0x7f15b814a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:53.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.302+0000 7f15bcbf7640 1 --2- 172.21.15.43:0/2164533325 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f15b815dce0 0x7f15b8159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:53.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.302+0000 7f15b77fe640 1 -- 172.21.15.43:0/2164533325 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f15b815dce0 msgr2=0x7f15b8159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:53.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.302+0000 7f15b77fe640 1 --2- 172.21.15.43:0/2164533325 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f15b815dce0 0x7f15b8159fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.302+0000 7f15b77fe640 1 -- 172.21.15.43:0/2164533325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15b80ce550 msgr2=0x7f15b814a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:53.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.302+0000 7f15b77fe640 1 --2- 172.21.15.43:0/2164533325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15b80ce550 0x7f15b814a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.302+0000 7f15b77fe640 1 -- 172.21.15.43:0/2164533325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b814a670 con 0x7f15b80ce990 2024-04-25T08:54:53.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.302+0000 7f15b7fff640 1 --2- 172.21.15.43:0/2164533325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15b80ce550 0x7f15b814a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:53.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.302+0000 7f15b77fe640 1 --2- 172.21.15.43:0/2164533325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15b80ce990 0x7f15b81f3630 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f15a8000c00 tx=0x7f15a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:53.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.302+0000 7f15b4ff9640 1 -- 172.21.15.43:0/2164533325 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a8037040 con 0x7f15b80ce990 2024-04-25T08:54:53.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.302+0000 7f15b4ff9640 1 -- 172.21.15.43:0/2164533325 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a8042070 con 0x7f15b80ce990 2024-04-25T08:54:53.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.302+0000 7f15be65a640 1 -- 172.21.15.43:0/2164533325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15b80cbc20 con 0x7f15b80ce990 2024-04-25T08:54:53.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.302+0000 7f15b4ff9640 1 -- 172.21.15.43:0/2164533325 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a8030880 con 0x7f15b80ce990 2024-04-25T08:54:53.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.302+0000 7f15be65a640 1 -- 172.21.15.43:0/2164533325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15b80cc1e0 con 0x7f15b80ce990 2024-04-25T08:54:53.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.303+0000 7f15be65a640 1 -- 172.21.15.43:0/2164533325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15b80680f0 con 0x7f15b80ce990 2024-04-25T08:54:53.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.304+0000 7f15b4ff9640 1 -- 172.21.15.43:0/2164533325 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f15a802ad10 con 0x7f15b80ce990 2024-04-25T08:54:53.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.304+0000 7f15b4ff9640 1 --2- 172.21.15.43:0/2164533325 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f15a406b4d0 0x7f15a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.304+0000 7f15b4ff9640 1 -- 172.21.15.43:0/2164533325 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f15a80aa720 con 0x7f15b80ce990 2024-04-25T08:54:53.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.304+0000 7f15b7fff640 1 --2- 172.21.15.43:0/2164533325 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f15a406b4d0 0x7f15a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:53.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.305+0000 7f15b7fff640 1 --2- 172.21.15.43:0/2164533325 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f15a406b4d0 0x7f15a406d990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f15b80ce060 tx=0x7f15ac008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:53.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.308+0000 7f15b4ff9640 1 -- 172.21.15.43:0/2164533325 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f15a80795a0 con 0x7f15b80ce990 2024-04-25T08:54:53.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.502+0000 7f15be65a640 1 -- 172.21.15.43:0/2164533325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f15b8074ef0 con 0x7f15b80ce990 2024-04-25T08:54:53.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.502+0000 7f15b4ff9640 1 -- 172.21.15.43:0/2164533325 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f15a8078ea0 con 0x7f15b80ce990 2024-04-25T08:54:53.492 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-04-25T08:54:53.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.504+0000 7f15be65a640 1 -- 172.21.15.43:0/2164533325 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f15a406b4d0 msgr2=0x7f15a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:53.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.504+0000 7f15be65a640 1 --2- 172.21.15.43:0/2164533325 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f15a406b4d0 0x7f15a406d990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f15b80ce060 tx=0x7f15ac008040 comp rx=0 tx=0).stop 2024-04-25T08:54:53.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.504+0000 7f15be65a640 1 -- 172.21.15.43:0/2164533325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15b80ce990 msgr2=0x7f15b81f3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:53.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.504+0000 7f15be65a640 1 --2- 172.21.15.43:0/2164533325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15b80ce990 0x7f15b81f3630 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f15a8000c00 tx=0x7f15a80046c0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.504+0000 7f15be65a640 1 -- 172.21.15.43:0/2164533325 shutdown_connections 2024-04-25T08:54:53.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.504+0000 7f15be65a640 1 --2- 172.21.15.43:0/2164533325 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f15b815dce0 0x7f15b8159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.504+0000 7f15be65a640 1 --2- 172.21.15.43:0/2164533325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15b80ce990 0x7f15b81f3630 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.504+0000 7f15be65a640 1 --2- 172.21.15.43:0/2164533325 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f15a406b4d0 0x7f15a406d990 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.504+0000 7f15be65a640 1 --2- 172.21.15.43:0/2164533325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15b80ce550 0x7f15b814a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.504+0000 7f15be65a640 1 -- 172.21.15.43:0/2164533325 >> 172.21.15.43:0/2164533325 conn(0x7f15b809cd00 msgr2=0x7f15b80cf960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:53.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.505+0000 7f15be65a640 1 -- 172.21.15.43:0/2164533325 shutdown_connections 2024-04-25T08:54:53.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.505+0000 7f15be65a640 1 -- 172.21.15.43:0/2164533325 wait complete. 2024-04-25T08:54:53.505 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi193.front.sepia.ceph.com... 2024-04-25T08:54:53.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:53.506 DEBUG:teuthology.orchestra.run.smithi193:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-04-25T08:54:53.535 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:54:53.535 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi193.front.sepia.ceph.com 2024-04-25T08:54:53.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:54:53.535 DEBUG:teuthology.orchestra.run.smithi193:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T08:54:53.594 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T08:54:53.594 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:54:53.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.705+0000 7f82a9d6a640 1 Processor -- start 2024-04-25T08:54:53.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.705+0000 7f82a9d6a640 1 -- start start 2024-04-25T08:54:53.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.706+0000 7f82a9d6a640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f82a406ab30 0x7f82a406afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.706+0000 7f82a9d6a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82a40ce6e0 con 0x7f82a415bcb0 2024-04-25T08:54:53.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.706+0000 7f82a9d6a640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82a40ce9c0 con 0x7f82a4071850 2024-04-25T08:54:53.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.706+0000 7f82a9d6a640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82a4160620 con 0x7f82a406ab30 2024-04-25T08:54:53.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.706+0000 7f82a37fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f82a415bcb0 0x7f82a406e220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48404/0 (socket says 172.21.15.43:48404) 2024-04-25T08:54:53.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.706+0000 7f82a37fe640 1 -- 172.21.15.43:0/1867231320 learned_addr learned my addr 172.21.15.43:0/1867231320 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:53.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.707+0000 7f82a2ffd640 1 --2- 172.21.15.43:0/1867231320 >> v2:172.21.15.193:3300/0 conn(0x7f82a406ab30 0x7f82a406afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:53.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.707+0000 7f82a27fc640 1 -- 172.21.15.43:0/1867231320 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2279253977 0 0) 0x7f82a40ce6e0 con 0x7f82a415bcb0 2024-04-25T08:54:53.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.707+0000 7f82a27fc640 1 -- 172.21.15.43:0/1867231320 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f828c003520 con 0x7f82a415bcb0 2024-04-25T08:54:53.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.707+0000 7f82a27fc640 1 -- 172.21.15.43:0/1867231320 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2154231338 0 0) 0x7f828c003520 con 0x7f82a415bcb0 2024-04-25T08:54:53.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.707+0000 7f82a27fc640 1 -- 172.21.15.43:0/1867231320 >> v2:172.21.15.193:3300/0 conn(0x7f82a406ab30 msgr2=0x7f82a406afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:53.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.707+0000 7f82a27fc640 1 --2- 172.21.15.43:0/1867231320 >> v2:172.21.15.193:3300/0 conn(0x7f82a406ab30 0x7f82a406afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.707+0000 7f82a27fc640 1 -- 172.21.15.43:0/1867231320 >> v1:172.21.15.193:6789/0 conn(0x7f82a4071850 legacy=0x7f82a40cf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:53.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.707+0000 7f82a27fc640 1 -- 172.21.15.43:0/1867231320 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a40ceca0 con 0x7f82a415bcb0 2024-04-25T08:54:53.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.707+0000 7f82a27fc640 1 -- 172.21.15.43:0/1867231320 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f8298001670 con 0x7f82a415bcb0 2024-04-25T08:54:53.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.707+0000 7f82a27fc640 1 -- 172.21.15.43:0/1867231320 >> v1:172.21.15.43:6789/0 conn(0x7f82a415bcb0 legacy=0x7f82a406e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:53.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.707+0000 7f82a27fc640 1 --2- 172.21.15.43:0/1867231320 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f828c0039b0 0x7f828c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.707+0000 7f82a27fc640 1 --2- 172.21.15.43:0/1867231320 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f828c004800 0x7f828c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.707+0000 7f82a27fc640 1 --2- 172.21.15.43:0/1867231320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f828c0051c0 0x7f828c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.707+0000 7f82a27fc640 1 -- 172.21.15.43:0/1867231320 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828c00c030 con 0x7f828c0051c0 2024-04-25T08:54:53.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.707+0000 7f82a27fc640 1 -- 172.21.15.43:0/1867231320 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828c00c1e0 con 0x7f828c0039b0 2024-04-25T08:54:53.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.707+0000 7f82a27fc640 1 -- 172.21.15.43:0/1867231320 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f828c00c3e0 con 0x7f828c004800 2024-04-25T08:54:53.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.707+0000 7f82a3fff640 1 --2- 172.21.15.43:0/1867231320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f828c0051c0 0x7f828c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:53.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.707+0000 7f82a37fe640 1 --2- 172.21.15.43:0/1867231320 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f828c0039b0 0x7f828c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:53.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.707+0000 7f82a2ffd640 1 --2- 172.21.15.43:0/1867231320 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f828c004800 0x7f828c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:53.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.708+0000 7f82a3fff640 1 -- 172.21.15.43:0/1867231320 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f828c004800 msgr2=0x7f828c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:53.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.708+0000 7f82a3fff640 1 --2- 172.21.15.43:0/1867231320 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f828c004800 0x7f828c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.708+0000 7f82a3fff640 1 -- 172.21.15.43:0/1867231320 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f828c0039b0 msgr2=0x7f828c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:53.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.708+0000 7f82a3fff640 1 --2- 172.21.15.43:0/1867231320 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f828c0039b0 0x7f828c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.708+0000 7f82a3fff640 1 -- 172.21.15.43:0/1867231320 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f828c003520 con 0x7f828c0051c0 2024-04-25T08:54:53.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.708+0000 7f82a3fff640 1 -- 172.21.15.43:0/1867231320 reap_dead start 2024-04-25T08:54:53.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.708+0000 7f82a3fff640 1 --2- 172.21.15.43:0/1867231320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f828c0051c0 0x7f828c00b9c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f829c001e50 tx=0x7f828c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=551661610c4e26ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:53.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.708+0000 7f82a27fc640 1 -- 172.21.15.43:0/1867231320 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829c001a70 con 0x7f828c0051c0 2024-04-25T08:54:53.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.708+0000 7f82a27fc640 1 -- 172.21.15.43:0/1867231320 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f829c003970 con 0x7f828c0051c0 2024-04-25T08:54:53.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.708+0000 7f82a27fc640 1 -- 172.21.15.43:0/1867231320 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829c003d60 con 0x7f828c0051c0 2024-04-25T08:54:53.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.709+0000 7f82a9d6a640 1 -- 172.21.15.43:0/1867231320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f828c0051c0 msgr2=0x7f828c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:53.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.709+0000 7f82a9d6a640 1 --2- 172.21.15.43:0/1867231320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f828c0051c0 0x7f828c00b9c0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f829c001e50 tx=0x7f828c005830 comp rx=0 tx=0).stop 2024-04-25T08:54:53.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.709+0000 7f82a9d6a640 1 -- 172.21.15.43:0/1867231320 shutdown_connections 2024-04-25T08:54:53.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.709+0000 7f82a9d6a640 1 --2- 172.21.15.43:0/1867231320 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f828c0051c0 0x7f828c00b9c0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.709+0000 7f82a9d6a640 1 -- 172.21.15.43:0/1867231320 >> 172.21.15.43:0/1867231320 conn(0x7f82a40b2490 msgr2=0x7f82a40b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:53.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.709+0000 7f82a9d6a640 1 -- 172.21.15.43:0/1867231320 shutdown_connections 2024-04-25T08:54:53.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.709+0000 7f82a9d6a640 1 -- 172.21.15.43:0/1867231320 wait complete. 2024-04-25T08:54:53.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.710+0000 7f82a9d6a640 1 Processor -- start 2024-04-25T08:54:53.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.710+0000 7f82a9d6a640 1 -- start start 2024-04-25T08:54:53.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.711+0000 7f82a9d6a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f828c0051c0 0x7f82a414d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.711+0000 7f82a9d6a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82a414ecf0 0x7f82a414f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.711+0000 7f82a9d6a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82a4150450 0x7f82a41508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.711+0000 7f82a9d6a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a4071d00 con 0x7f828c0051c0 2024-04-25T08:54:53.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.711+0000 7f82a9d6a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a40cf010 con 0x7f82a4150450 2024-04-25T08:54:53.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.711+0000 7f82a9d6a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82a4071e80 con 0x7f82a414ecf0 2024-04-25T08:54:53.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.711+0000 7f82a37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f828c0051c0 0x7f82a414d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:53.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.711+0000 7f82a37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f828c0051c0 0x7f82a414d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49742/0 (socket says 172.21.15.43:49742) 2024-04-25T08:54:53.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.711+0000 7f82a37fe640 1 -- 172.21.15.43:0/1824499209 learned_addr learned my addr 172.21.15.43:0/1824499209 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:53.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.711+0000 7f82a37fe640 1 -- 172.21.15.43:0/1824499209 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82a414ecf0 msgr2=0x7f82a414f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:53.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.711+0000 7f82a37fe640 1 --2- 172.21.15.43:0/1824499209 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82a414ecf0 0x7f82a414f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.711+0000 7f82a37fe640 1 -- 172.21.15.43:0/1824499209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82a4150450 msgr2=0x7f82a41508d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:53.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.711+0000 7f82a37fe640 1 --2- 172.21.15.43:0/1824499209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82a4150450 0x7f82a41508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.711+0000 7f82a37fe640 1 -- 172.21.15.43:0/1824499209 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a4150e10 con 0x7f828c0051c0 2024-04-25T08:54:53.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.712+0000 7f82a37fe640 1 --2- 172.21.15.43:0/1824499209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f828c0051c0 0x7f82a414d860 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f82980072c0 tx=0x7f8298007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:53.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.712+0000 7f828bfff640 1 -- 172.21.15.43:0/1824499209 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829800f8e0 con 0x7f828c0051c0 2024-04-25T08:54:53.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.712+0000 7f828bfff640 1 -- 172.21.15.43:0/1824499209 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8298018070 con 0x7f828c0051c0 2024-04-25T08:54:53.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.712+0000 7f82a9d6a640 1 -- 172.21.15.43:0/1824499209 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82a40ad8e0 con 0x7f828c0051c0 2024-04-25T08:54:53.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.712+0000 7f828bfff640 1 -- 172.21.15.43:0/1824499209 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829800fce0 con 0x7f828c0051c0 2024-04-25T08:54:53.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.712+0000 7f82a9d6a640 1 -- 172.21.15.43:0/1824499209 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82a414fa70 con 0x7f828c0051c0 2024-04-25T08:54:53.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.713+0000 7f82a9d6a640 1 -- 172.21.15.43:0/1824499209 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82a4074ef0 con 0x7f828c0051c0 2024-04-25T08:54:53.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.713+0000 7f828bfff640 1 -- 172.21.15.43:0/1824499209 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f829801d460 con 0x7f828c0051c0 2024-04-25T08:54:53.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.714+0000 7f828bfff640 1 --2- 172.21.15.43:0/1824499209 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f829006b500 0x7f829006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:53.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.714+0000 7f828bfff640 1 -- 172.21.15.43:0/1824499209 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f829808a280 con 0x7f828c0051c0 2024-04-25T08:54:53.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.714+0000 7f82a2ffd640 1 --2- 172.21.15.43:0/1824499209 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f829006b500 0x7f829006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:53.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.714+0000 7f82a2ffd640 1 --2- 172.21.15.43:0/1824499209 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f829006b500 0x7f829006d9c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f82a40b3c00 tx=0x7f8294009320 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:53.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.718+0000 7f828bfff640 1 -- 172.21.15.43:0/1824499209 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f82a4150e10 con 0x7f828c0051c0 2024-04-25T08:54:53.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.936+0000 7f82a9d6a640 1 -- 172.21.15.43:0/1824499209 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82a40adab0 con 0x7f828c0051c0 2024-04-25T08:54:53.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.937+0000 7f828bfff640 1 -- 172.21.15.43:0/1824499209 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4938 (secure 0 0 0) 0x7f8298059100 con 0x7f828c0051c0 2024-04-25T08:54:53.927 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:54:53.927 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3616994003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":3616994003},{"type":"v1","addr":"172.21.15.43:6835","nonce":3616994003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/4022901908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4022901908},{"type":"v1","addr":"172.21.15.193:6835","nonce":4022901908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2823450590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2823450590},{"type":"v1","addr":"172.21.15.43:6834","nonce":2823450590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:54:11.945963+0000","modified":"2024-04-25T08:54:13.943002+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.193:6837/1197434598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1197434598},{"type":"v1","addr":"172.21.15.193:6837","nonce":1197434598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:54:53.927 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 6 2024-04-25T08:54:53.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.939+0000 7f82a9d6a640 1 -- 172.21.15.43:0/1824499209 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f829006b500 msgr2=0x7f829006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:53.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.939+0000 7f82a9d6a640 1 --2- 172.21.15.43:0/1824499209 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f829006b500 0x7f829006d9c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f82a40b3c00 tx=0x7f8294009320 comp rx=0 tx=0).stop 2024-04-25T08:54:53.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.939+0000 7f82a9d6a640 1 -- 172.21.15.43:0/1824499209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f828c0051c0 msgr2=0x7f82a414d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:53.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.939+0000 7f82a9d6a640 1 --2- 172.21.15.43:0/1824499209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f828c0051c0 0x7f82a414d860 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f82980072c0 tx=0x7f8298007780 comp rx=0 tx=0).stop 2024-04-25T08:54:53.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.939+0000 7f82a9d6a640 1 -- 172.21.15.43:0/1824499209 shutdown_connections 2024-04-25T08:54:53.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.939+0000 7f82a9d6a640 1 --2- 172.21.15.43:0/1824499209 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f829006b500 0x7f829006d9c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.939+0000 7f82a9d6a640 1 --2- 172.21.15.43:0/1824499209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82a4150450 0x7f82a41508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.939+0000 7f82a9d6a640 1 --2- 172.21.15.43:0/1824499209 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82a414ecf0 0x7f82a414f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.939+0000 7f82a9d6a640 1 --2- 172.21.15.43:0/1824499209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f828c0051c0 0x7f82a414d860 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:53.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.939+0000 7f82a9d6a640 1 -- 172.21.15.43:0/1824499209 >> 172.21.15.43:0/1824499209 conn(0x7f82a40b2490 msgr2=0x7f82a40cf4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:53.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.939+0000 7f82a9d6a640 1 -- 172.21.15.43:0/1824499209 shutdown_connections 2024-04-25T08:54:53.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:53.939+0000 7f82a9d6a640 1 -- 172.21.15.43:0/1824499209 wait complete. 2024-04-25T08:54:53.940 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:54:54.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.051+0000 7f2c52c74640 1 Processor -- start 2024-04-25T08:54:54.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.051+0000 7f2c52c74640 1 -- start start 2024-04-25T08:54:54.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.052+0000 7f2c52c74640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f2c4c15dce0 0x7f2c4c15ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:54.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.052+0000 7f2c52c74640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c4c0d2920 con 0x7f2c4c0ce550 2024-04-25T08:54:54.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.052+0000 7f2c52c74640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c4c0cee80 con 0x7f2c4c0d0af0 2024-04-25T08:54:54.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.052+0000 7f2c52c74640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c4c0cf160 con 0x7f2c4c15dce0 2024-04-25T08:54:54.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.052+0000 7f2c4bfff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f2c4c0ce550 0x7f2c4c0d03e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48414/0 (socket says 172.21.15.43:48414) 2024-04-25T08:54:54.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.052+0000 7f2c4bfff640 1 -- 172.21.15.43:0/1981353074 learned_addr learned my addr 172.21.15.43:0/1981353074 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:54.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.053+0000 7f2c50a10640 1 --2- 172.21.15.43:0/1981353074 >> v2:172.21.15.193:3300/0 conn(0x7f2c4c15dce0 0x7f2c4c15ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:54.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.053+0000 7f2c4b7fe640 1 -- 172.21.15.43:0/1981353074 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1415651586 0 0) 0x7f2c4c0d2920 con 0x7f2c4c0ce550 2024-04-25T08:54:54.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.053+0000 7f2c4b7fe640 1 -- 172.21.15.43:0/1981353074 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c38003520 con 0x7f2c4c0ce550 2024-04-25T08:54:54.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.053+0000 7f2c4b7fe640 1 -- 172.21.15.43:0/1981353074 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1361406621 0 0) 0x7f2c38003520 con 0x7f2c4c0ce550 2024-04-25T08:54:54.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.053+0000 7f2c4b7fe640 1 -- 172.21.15.43:0/1981353074 >> v2:172.21.15.193:3300/0 conn(0x7f2c4c15dce0 msgr2=0x7f2c4c15ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:54.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.053+0000 7f2c4b7fe640 1 --2- 172.21.15.43:0/1981353074 >> v2:172.21.15.193:3300/0 conn(0x7f2c4c15dce0 0x7f2c4c15ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:54.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.053+0000 7f2c4b7fe640 1 -- 172.21.15.43:0/1981353074 >> v1:172.21.15.193:6789/0 conn(0x7f2c4c0d0af0 legacy=0x7f2c4c160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:54.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.053+0000 7f2c4b7fe640 1 -- 172.21.15.43:0/1981353074 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c4c0cf440 con 0x7f2c4c0ce550 2024-04-25T08:54:54.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.053+0000 7f2c4b7fe640 1 -- 172.21.15.43:0/1981353074 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f2c3c001690 con 0x7f2c4c0ce550 2024-04-25T08:54:54.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.053+0000 7f2c4b7fe640 1 -- 172.21.15.43:0/1981353074 >> v1:172.21.15.43:6789/0 conn(0x7f2c4c0ce550 legacy=0x7f2c4c0d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:54.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.053+0000 7f2c4b7fe640 1 --2- 172.21.15.43:0/1981353074 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c380039b0 0x7f2c38003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:54.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.053+0000 7f2c4b7fe640 1 --2- 172.21.15.43:0/1981353074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c380043b0 0x7f2c38004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:54.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.053+0000 7f2c4b7fe640 1 --2- 172.21.15.43:0/1981353074 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c38005120 0x7f2c3800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:54.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.053+0000 7f2c4b7fe640 1 -- 172.21.15.43:0/1981353074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c0cf160 con 0x7f2c380043b0 2024-04-25T08:54:54.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.053+0000 7f2c4b7fe640 1 -- 172.21.15.43:0/1981353074 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3800c060 con 0x7f2c380039b0 2024-04-25T08:54:54.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.053+0000 7f2c4b7fe640 1 -- 172.21.15.43:0/1981353074 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3800c260 con 0x7f2c38005120 2024-04-25T08:54:54.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.053+0000 7f2c4bfff640 1 --2- 172.21.15.43:0/1981353074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c380043b0 0x7f2c38004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:54.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.053+0000 7f2c50a10640 1 --2- 172.21.15.43:0/1981353074 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c380039b0 0x7f2c38003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:54.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.053+0000 7f2c51211640 1 --2- 172.21.15.43:0/1981353074 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c38005120 0x7f2c3800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:54.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.053+0000 7f2c4bfff640 1 -- 172.21.15.43:0/1981353074 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c38005120 msgr2=0x7f2c3800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:54.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.053+0000 7f2c4bfff640 1 --2- 172.21.15.43:0/1981353074 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c38005120 0x7f2c3800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:54.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.053+0000 7f2c4bfff640 1 -- 172.21.15.43:0/1981353074 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c380039b0 msgr2=0x7f2c38003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:54.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.053+0000 7f2c4bfff640 1 --2- 172.21.15.43:0/1981353074 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c380039b0 0x7f2c38003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:54.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.053+0000 7f2c4bfff640 1 -- 172.21.15.43:0/1981353074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c38003520 con 0x7f2c380043b0 2024-04-25T08:54:54.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.053+0000 7f2c51211640 1 --2- 172.21.15.43:0/1981353074 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c38005120 0x7f2c3800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:54.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.053+0000 7f2c51211640 1 -- 172.21.15.43:0/1981353074 reap_dead start 2024-04-25T08:54:54.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.054+0000 7f2c4bfff640 1 --2- 172.21.15.43:0/1981353074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c380043b0 0x7f2c38004880 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c00c210 tx=0x7f2c38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37eb2e09828ae7e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:54.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.054+0000 7f2c4b7fe640 1 -- 172.21.15.43:0/1981353074 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c007d30 con 0x7f2c380043b0 2024-04-25T08:54:54.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.054+0000 7f2c4b7fe640 1 -- 172.21.15.43:0/1981353074 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c3c005ad0 con 0x7f2c380043b0 2024-04-25T08:54:54.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.054+0000 7f2c4b7fe640 1 -- 172.21.15.43:0/1981353074 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c007070 con 0x7f2c380043b0 2024-04-25T08:54:54.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.055+0000 7f2c52c74640 1 -- 172.21.15.43:0/1981353074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c380043b0 msgr2=0x7f2c38004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:54.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.055+0000 7f2c52c74640 1 --2- 172.21.15.43:0/1981353074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c380043b0 0x7f2c38004880 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c00c210 tx=0x7f2c38005830 comp rx=0 tx=0).stop 2024-04-25T08:54:54.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.055+0000 7f2c52c74640 1 -- 172.21.15.43:0/1981353074 shutdown_connections 2024-04-25T08:54:54.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.055+0000 7f2c52c74640 1 --2- 172.21.15.43:0/1981353074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c380043b0 0x7f2c38004880 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:54.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.055+0000 7f2c52c74640 1 -- 172.21.15.43:0/1981353074 >> 172.21.15.43:0/1981353074 conn(0x7f2c4c09cd00 msgr2=0x7f2c4c0d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:54.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.055+0000 7f2c52c74640 1 -- 172.21.15.43:0/1981353074 shutdown_connections 2024-04-25T08:54:54.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.055+0000 7f2c52c74640 1 -- 172.21.15.43:0/1981353074 wait complete. 2024-04-25T08:54:54.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.056+0000 7f2c52c74640 1 Processor -- start 2024-04-25T08:54:54.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.056+0000 7f2c52c74640 1 -- start start 2024-04-25T08:54:54.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.057+0000 7f2c52c74640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c380043b0 0x7f2c4c154a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:54.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.057+0000 7f2c52c74640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c4c15abd0 0x7f2c4c15b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:54.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.057+0000 7f2c52c74640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c4c159190 0x7f2c4c159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:54.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.057+0000 7f2c52c74640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c15e0c0 con 0x7f2c4c15abd0 2024-04-25T08:54:54.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.057+0000 7f2c52c74640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c15f6b0 con 0x7f2c4c159190 2024-04-25T08:54:54.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.057+0000 7f2c52c74640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c0ce9c0 con 0x7f2c380043b0 2024-04-25T08:54:54.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.057+0000 7f2c4bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c4c15abd0 0x7f2c4c15b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:54.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.057+0000 7f2c4bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c4c15abd0 0x7f2c4c15b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49754/0 (socket says 172.21.15.43:49754) 2024-04-25T08:54:54.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.057+0000 7f2c4bfff640 1 -- 172.21.15.43:0/3425394371 learned_addr learned my addr 172.21.15.43:0/3425394371 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:54.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.057+0000 7f2c50a10640 1 --2- 172.21.15.43:0/3425394371 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c380043b0 0x7f2c4c154a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:54.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.057+0000 7f2c51211640 1 --2- 172.21.15.43:0/3425394371 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c4c159190 0x7f2c4c159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:54.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.057+0000 7f2c4bfff640 1 -- 172.21.15.43:0/3425394371 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c380043b0 msgr2=0x7f2c4c154a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:54.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.057+0000 7f2c4bfff640 1 --2- 172.21.15.43:0/3425394371 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c380043b0 0x7f2c4c154a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:54.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.057+0000 7f2c4bfff640 1 -- 172.21.15.43:0/3425394371 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c4c159190 msgr2=0x7f2c4c159610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:54.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.057+0000 7f2c4bfff640 1 --2- 172.21.15.43:0/3425394371 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c4c159190 0x7f2c4c159610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:54.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.057+0000 7f2c4bfff640 1 -- 172.21.15.43:0/3425394371 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c4c15a610 con 0x7f2c4c15abd0 2024-04-25T08:54:54.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.057+0000 7f2c4bfff640 1 --2- 172.21.15.43:0/3425394371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c4c15abd0 0x7f2c4c15b050 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c00c210 tx=0x7f2c4c0cf050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:54.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.058+0000 7f2c497fa640 1 -- 172.21.15.43:0/3425394371 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c037040 con 0x7f2c4c15abd0 2024-04-25T08:54:54.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.058+0000 7f2c497fa640 1 -- 172.21.15.43:0/3425394371 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c3c041070 con 0x7f2c4c15abd0 2024-04-25T08:54:54.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.058+0000 7f2c52c74640 1 -- 172.21.15.43:0/3425394371 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c4c159d70 con 0x7f2c4c15abd0 2024-04-25T08:54:54.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.058+0000 7f2c497fa640 1 -- 172.21.15.43:0/3425394371 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c004a20 con 0x7f2c4c15abd0 2024-04-25T08:54:54.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.058+0000 7f2c52c74640 1 -- 172.21.15.43:0/3425394371 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c4c15a330 con 0x7f2c4c15abd0 2024-04-25T08:54:54.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.059+0000 7f2c52c74640 1 -- 172.21.15.43:0/3425394371 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c4c0680f0 con 0x7f2c4c15abd0 2024-04-25T08:54:54.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.059+0000 7f2c497fa640 1 -- 172.21.15.43:0/3425394371 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2c3c004bc0 con 0x7f2c4c15abd0 2024-04-25T08:54:54.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.060+0000 7f2c497fa640 1 --2- 172.21.15.43:0/3425394371 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2c2c06b500 0x7f2c2c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:54.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.060+0000 7f2c497fa640 1 -- 172.21.15.43:0/3425394371 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f2c3c0a9a60 con 0x7f2c4c15abd0 2024-04-25T08:54:54.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.060+0000 7f2c50a10640 1 --2- 172.21.15.43:0/3425394371 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2c2c06b500 0x7f2c2c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:54.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.060+0000 7f2c50a10640 1 --2- 172.21.15.43:0/3425394371 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2c2c06b500 0x7f2c2c06d9c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2c44014510 tx=0x7f2c4400b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:54.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.064+0000 7f2c497fa640 1 -- 172.21.15.43:0/3425394371 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2c3c078200 con 0x7f2c4c15abd0 2024-04-25T08:54:54.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.246+0000 7f2c52c74640 1 -- 172.21.15.43:0/3425394371 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2c4c074ef0 con 0x7f2c4c15abd0 2024-04-25T08:54:54.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.247+0000 7f2c497fa640 1 -- 172.21.15.43:0/3425394371 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16809 (secure 0 0 0) 0x7f2c3c07c2a0 con 0x7f2c4c15abd0 2024-04-25T08:54:54.237 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:54:54.237 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":22,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:54:52.847242+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:54:09.757978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:54:10.926008+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.18","mappings":[{"from":3,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2d","mappings":[{"from":7,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:54:54.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.249+0000 7f2c52c74640 1 -- 172.21.15.43:0/3425394371 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2c2c06b500 msgr2=0x7f2c2c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:54.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.249+0000 7f2c52c74640 1 --2- 172.21.15.43:0/3425394371 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2c2c06b500 0x7f2c2c06d9c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2c44014510 tx=0x7f2c4400b040 comp rx=0 tx=0).stop 2024-04-25T08:54:54.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.249+0000 7f2c52c74640 1 -- 172.21.15.43:0/3425394371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c4c15abd0 msgr2=0x7f2c4c15b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:54.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.249+0000 7f2c52c74640 1 --2- 172.21.15.43:0/3425394371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c4c15abd0 0x7f2c4c15b050 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2c3c00c210 tx=0x7f2c4c0cf050 comp rx=0 tx=0).stop 2024-04-25T08:54:54.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.249+0000 7f2c52c74640 1 -- 172.21.15.43:0/3425394371 shutdown_connections 2024-04-25T08:54:54.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.249+0000 7f2c52c74640 1 --2- 172.21.15.43:0/3425394371 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2c2c06b500 0x7f2c2c06d9c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:54.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.249+0000 7f2c52c74640 1 --2- 172.21.15.43:0/3425394371 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c4c159190 0x7f2c4c159610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:54.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.249+0000 7f2c52c74640 1 --2- 172.21.15.43:0/3425394371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c4c15abd0 0x7f2c4c15b050 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:54.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.249+0000 7f2c52c74640 1 --2- 172.21.15.43:0/3425394371 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c380043b0 0x7f2c4c154a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:54.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.249+0000 7f2c52c74640 1 -- 172.21.15.43:0/3425394371 >> 172.21.15.43:0/3425394371 conn(0x7f2c4c09cd00 msgr2=0x7f2c4c06c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:54.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.249+0000 7f2c52c74640 1 -- 172.21.15.43:0/3425394371 shutdown_connections 2024-04-25T08:54:54.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.249+0000 7f2c52c74640 1 -- 172.21.15.43:0/3425394371 wait complete. 2024-04-25T08:54:54.250 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:54:09.757978+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:54:10.926008+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:54:54.250 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:54:54.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.364+0000 7ff74896a640 1 Processor -- start 2024-04-25T08:54:54.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.364+0000 7ff74896a640 1 -- start start 2024-04-25T08:54:54.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.365+0000 7ff74896a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff74415a1d0 0x7ff74406d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:54.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.365+0000 7ff74896a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff744071ab0 0x7ff744070a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:54.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.365+0000 7ff74896a640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7440b3160 con 0x7ff744071650 2024-04-25T08:54:54.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.365+0000 7ff74896a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7440b3ff0 con 0x7ff74415a1d0 2024-04-25T08:54:54.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.365+0000 7ff74896a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff74415a5b0 con 0x7ff744071ab0 2024-04-25T08:54:54.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.365+0000 7ff74259c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff74415a1d0 0x7ff74406d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:54.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.365+0000 7ff74259c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff74415a1d0 0x7ff74406d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49764/0 (socket says 172.21.15.43:49764) 2024-04-25T08:54:54.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.365+0000 7ff74259c640 1 -- 172.21.15.43:0/1923960600 learned_addr learned my addr 172.21.15.43:0/1923960600 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:54.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.365+0000 7ff742d9d640 1 --2- 172.21.15.43:0/1923960600 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff744071ab0 0x7ff744070a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:54.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.365+0000 7ff74259c640 1 -- 172.21.15.43:0/1923960600 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff744071ab0 msgr2=0x7ff744070a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:54.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.365+0000 7ff74259c640 1 --2- 172.21.15.43:0/1923960600 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff744071ab0 0x7ff744070a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:54.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.365+0000 7ff74259c640 1 -- 172.21.15.43:0/1923960600 >> v1:172.21.15.193:6789/0 conn(0x7ff744071650 legacy=0x7ff744070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:54.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.365+0000 7ff74259c640 1 -- 172.21.15.43:0/1923960600 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7440b33e0 con 0x7ff74415a1d0 2024-04-25T08:54:54.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.365+0000 7ff742d9d640 1 --2- 172.21.15.43:0/1923960600 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff744071ab0 0x7ff744070a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:54.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.366+0000 7ff74259c640 1 --2- 172.21.15.43:0/1923960600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff74415a1d0 0x7ff74406d950 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff734009870 tx=0x7ff73402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8421295529f4e08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:54.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.366+0000 7ff741d9b640 1 -- 172.21.15.43:0/1923960600 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff734037070 con 0x7ff74415a1d0 2024-04-25T08:54:54.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.366+0000 7ff741d9b640 1 -- 172.21.15.43:0/1923960600 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff734032440 con 0x7ff74415a1d0 2024-04-25T08:54:54.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.366+0000 7ff741d9b640 1 -- 172.21.15.43:0/1923960600 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff734032810 con 0x7ff74415a1d0 2024-04-25T08:54:54.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.367+0000 7ff74896a640 1 -- 172.21.15.43:0/1923960600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff74415a1d0 msgr2=0x7ff74406d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:54.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.367+0000 7ff74896a640 1 --2- 172.21.15.43:0/1923960600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff74415a1d0 0x7ff74406d950 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ff734009870 tx=0x7ff73402a210 comp rx=0 tx=0).stop 2024-04-25T08:54:54.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.367+0000 7ff74896a640 1 -- 172.21.15.43:0/1923960600 shutdown_connections 2024-04-25T08:54:54.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.367+0000 7ff74896a640 1 --2- 172.21.15.43:0/1923960600 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff744071ab0 0x7ff744070a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:54.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.367+0000 7ff74896a640 1 --2- 172.21.15.43:0/1923960600 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff74415a1d0 0x7ff74406d950 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:54.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.367+0000 7ff74896a640 1 -- 172.21.15.43:0/1923960600 >> 172.21.15.43:0/1923960600 conn(0x7ff74406ab30 msgr2=0x7ff74406af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:54.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.367+0000 7ff74896a640 1 -- 172.21.15.43:0/1923960600 shutdown_connections 2024-04-25T08:54:54.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.367+0000 7ff74896a640 1 -- 172.21.15.43:0/1923960600 wait complete. 2024-04-25T08:54:54.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.368+0000 7ff74896a640 1 Processor -- start 2024-04-25T08:54:54.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.368+0000 7ff74896a640 1 -- start start 2024-04-25T08:54:54.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.369+0000 7ff74896a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff744071650 0x7ff7441592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:54.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.369+0000 7ff74896a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff744071ab0 0x7ff744148d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:54.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.369+0000 7ff74896a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff74415a1d0 0x7ff7441492b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:54.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.369+0000 7ff74896a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7440b3c80 con 0x7ff74415a1d0 2024-04-25T08:54:54.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.369+0000 7ff74896a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7440b39a0 con 0x7ff744071650 2024-04-25T08:54:54.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.369+0000 7ff74896a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff744071f40 con 0x7ff744071ab0 2024-04-25T08:54:54.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.369+0000 7ff742d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff74415a1d0 0x7ff7441492b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:54.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.369+0000 7ff742d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff74415a1d0 0x7ff7441492b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49776/0 (socket says 172.21.15.43:49776) 2024-04-25T08:54:54.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.369+0000 7ff742d9d640 1 -- 172.21.15.43:0/1192403432 learned_addr learned my addr 172.21.15.43:0/1192403432 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:54.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.369+0000 7ff74259c640 1 --2- 172.21.15.43:0/1192403432 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff744071650 0x7ff7441592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:54.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.369+0000 7ff73bfff640 1 --2- 172.21.15.43:0/1192403432 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff744071ab0 0x7ff744148d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:54.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.369+0000 7ff742d9d640 1 -- 172.21.15.43:0/1192403432 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff744071ab0 msgr2=0x7ff744148d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:54.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.369+0000 7ff742d9d640 1 --2- 172.21.15.43:0/1192403432 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff744071ab0 0x7ff744148d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:54.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.369+0000 7ff742d9d640 1 -- 172.21.15.43:0/1192403432 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff744071650 msgr2=0x7ff7441592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:54.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.369+0000 7ff742d9d640 1 --2- 172.21.15.43:0/1192403432 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff744071650 0x7ff7441592d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:54.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.369+0000 7ff742d9d640 1 -- 172.21.15.43:0/1192403432 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff74414fcd0 con 0x7ff74415a1d0 2024-04-25T08:54:54.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.369+0000 7ff742d9d640 1 --2- 172.21.15.43:0/1192403432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff74415a1d0 0x7ff7441492b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff73c00cef0 tx=0x7ff73c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:54.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.369+0000 7ff73affd640 1 -- 172.21.15.43:0/1192403432 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c01a070 con 0x7ff74415a1d0 2024-04-25T08:54:54.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.370+0000 7ff73affd640 1 -- 172.21.15.43:0/1192403432 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff73c00d860 con 0x7ff74415a1d0 2024-04-25T08:54:54.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.370+0000 7ff74896a640 1 -- 172.21.15.43:0/1192403432 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7441f78e0 con 0x7ff74415a1d0 2024-04-25T08:54:54.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.370+0000 7ff73affd640 1 -- 172.21.15.43:0/1192403432 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c00dc50 con 0x7ff74415a1d0 2024-04-25T08:54:54.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.370+0000 7ff74896a640 1 -- 172.21.15.43:0/1192403432 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff74414ae20 con 0x7ff74415a1d0 2024-04-25T08:54:54.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.371+0000 7ff74896a640 1 -- 172.21.15.43:0/1192403432 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff744074f60 con 0x7ff74415a1d0 2024-04-25T08:54:54.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.371+0000 7ff73affd640 1 -- 172.21.15.43:0/1192403432 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff73c01d070 con 0x7ff74415a1d0 2024-04-25T08:54:54.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.371+0000 7ff73affd640 1 --2- 172.21.15.43:0/1192403432 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff72406b500 0x7ff72406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:54.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.372+0000 7ff73affd640 1 -- 172.21.15.43:0/1192403432 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7ff73c08aaa0 con 0x7ff74415a1d0 2024-04-25T08:54:54.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.372+0000 7ff74259c640 1 --2- 172.21.15.43:0/1192403432 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff72406b500 0x7ff72406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:54.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.372+0000 7ff74259c640 1 --2- 172.21.15.43:0/1192403432 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff72406b500 0x7ff72406d9c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff734034000 tx=0x7ff73402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:54.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.376+0000 7ff73affd640 1 -- 172.21.15.43:0/1192403432 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff73c059300 con 0x7ff74415a1d0 2024-04-25T08:54:54.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.554+0000 7ff74896a640 1 -- 172.21.15.43:0/1192403432 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff744150740 con 0x7ff74415a1d0 2024-04-25T08:54:54.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.555+0000 7ff73affd640 1 -- 172.21.15.43:0/1192403432 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff73c05d3a0 con 0x7ff74415a1d0 2024-04-25T08:54:54.545 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T08:54:54.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.557+0000 7ff74896a640 1 -- 172.21.15.43:0/1192403432 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff72406b500 msgr2=0x7ff72406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:54.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.557+0000 7ff74896a640 1 --2- 172.21.15.43:0/1192403432 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff72406b500 0x7ff72406d9c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff734034000 tx=0x7ff73402e000 comp rx=0 tx=0).stop 2024-04-25T08:54:54.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.557+0000 7ff74896a640 1 -- 172.21.15.43:0/1192403432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff74415a1d0 msgr2=0x7ff7441492b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:54.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.557+0000 7ff74896a640 1 --2- 172.21.15.43:0/1192403432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff74415a1d0 0x7ff7441492b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7ff73c00cef0 tx=0x7ff73c00a710 comp rx=0 tx=0).stop 2024-04-25T08:54:54.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.557+0000 7ff74896a640 1 -- 172.21.15.43:0/1192403432 shutdown_connections 2024-04-25T08:54:54.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.557+0000 7ff74896a640 1 --2- 172.21.15.43:0/1192403432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff74415a1d0 0x7ff7441492b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:54.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.557+0000 7ff74896a640 1 --2- 172.21.15.43:0/1192403432 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff744071ab0 0x7ff744148d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:54.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.557+0000 7ff74896a640 1 --2- 172.21.15.43:0/1192403432 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff72406b500 0x7ff72406d9c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:54.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.557+0000 7ff74896a640 1 --2- 172.21.15.43:0/1192403432 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff744071650 0x7ff7441592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:54.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.557+0000 7ff74896a640 1 -- 172.21.15.43:0/1192403432 >> 172.21.15.43:0/1192403432 conn(0x7ff74406ab30 msgr2=0x7ff744150f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:54.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.557+0000 7ff74896a640 1 -- 172.21.15.43:0/1192403432 shutdown_connections 2024-04-25T08:54:54.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.557+0000 7ff74896a640 1 -- 172.21.15.43:0/1192403432 wait complete. 2024-04-25T08:54:54.558 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:54:54.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.672+0000 7f110c15c640 1 Processor -- start 2024-04-25T08:54:54.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.672+0000 7f110c15c640 1 -- start start 2024-04-25T08:54:54.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.673+0000 7f110c15c640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f11040c8870 0x7f11040d2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:54.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.673+0000 7f110c15c640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11040cf030 con 0x7f11040d19f0 2024-04-25T08:54:54.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.673+0000 7f110c15c640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11040cf310 con 0x7f1104161310 2024-04-25T08:54:54.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.673+0000 7f110c15c640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11040d4870 con 0x7f11040c8870 2024-04-25T08:54:54.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.673+0000 7f11096f7640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f11040d19f0 0x7f11040d1e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48422/0 (socket says 172.21.15.43:48422) 2024-04-25T08:54:54.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.673+0000 7f11096f7640 1 -- 172.21.15.43:0/794170641 learned_addr learned my addr 172.21.15.43:0/794170641 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:54.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.673+0000 7f110a6f9640 1 --2- 172.21.15.43:0/794170641 >> v2:172.21.15.193:3300/0 conn(0x7f11040c8870 0x7f11040d2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:54.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.673+0000 7f1108ef6640 1 -- 172.21.15.43:0/794170641 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2787235965 0 0) 0x7f11040cf030 con 0x7f11040d19f0 2024-04-25T08:54:54.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.673+0000 7f1108ef6640 1 -- 172.21.15.43:0/794170641 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10f0003520 con 0x7f11040d19f0 2024-04-25T08:54:54.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.673+0000 7f1108ef6640 1 -- 172.21.15.43:0/794170641 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1781303884 0 0) 0x7f10f0003520 con 0x7f11040d19f0 2024-04-25T08:54:54.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.673+0000 7f1108ef6640 1 -- 172.21.15.43:0/794170641 >> v2:172.21.15.193:3300/0 conn(0x7f11040c8870 msgr2=0x7f11040d2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:54.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.673+0000 7f1108ef6640 1 --2- 172.21.15.43:0/794170641 >> v2:172.21.15.193:3300/0 conn(0x7f11040c8870 0x7f11040d2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:54.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.673+0000 7f1108ef6640 1 -- 172.21.15.43:0/794170641 >> v1:172.21.15.193:6789/0 conn(0x7f1104161310 legacy=0x7f11040d4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:54.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.673+0000 7f1108ef6640 1 -- 172.21.15.43:0/794170641 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11040cf5f0 con 0x7f11040d19f0 2024-04-25T08:54:54.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.673+0000 7f1108ef6640 1 -- 172.21.15.43:0/794170641 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f10f4001670 con 0x7f11040d19f0 2024-04-25T08:54:54.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.673+0000 7f1108ef6640 1 -- 172.21.15.43:0/794170641 >> v1:172.21.15.43:6789/0 conn(0x7f11040d19f0 legacy=0x7f11040d1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:54.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.673+0000 7f1108ef6640 1 --2- 172.21.15.43:0/794170641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10f00039b0 0x7f10f0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:54.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.673+0000 7f1108ef6640 1 --2- 172.21.15.43:0/794170641 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f10f0004800 0x7f10f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:54.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.673+0000 7f1108ef6640 1 --2- 172.21.15.43:0/794170641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10f00051c0 0x7f10f000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:54.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.673+0000 7f1108ef6640 1 -- 172.21.15.43:0/794170641 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11040d4870 con 0x7f10f00039b0 2024-04-25T08:54:54.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.673+0000 7f1108ef6640 1 -- 172.21.15.43:0/794170641 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f000c060 con 0x7f10f00051c0 2024-04-25T08:54:54.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.673+0000 7f1108ef6640 1 -- 172.21.15.43:0/794170641 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f000c260 con 0x7f10f0004800 2024-04-25T08:54:54.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.674+0000 7f1109ef8640 1 --2- 172.21.15.43:0/794170641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10f00039b0 0x7f10f0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:54.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.674+0000 7f11096f7640 1 --2- 172.21.15.43:0/794170641 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f10f0004800 0x7f10f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:54.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.674+0000 7f110a6f9640 1 --2- 172.21.15.43:0/794170641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10f00051c0 0x7f10f000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:54.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.674+0000 7f1109ef8640 1 -- 172.21.15.43:0/794170641 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f10f0004800 msgr2=0x7f10f0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:54.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.674+0000 7f1109ef8640 1 --2- 172.21.15.43:0/794170641 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f10f0004800 0x7f10f0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:54.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.674+0000 7f1109ef8640 1 -- 172.21.15.43:0/794170641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10f00051c0 msgr2=0x7f10f000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:54.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.674+0000 7f1109ef8640 1 --2- 172.21.15.43:0/794170641 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10f00051c0 0x7f10f000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:54.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.674+0000 7f1109ef8640 1 -- 172.21.15.43:0/794170641 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f0003520 con 0x7f10f00039b0 2024-04-25T08:54:54.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.674+0000 7f110a6f9640 1 -- 172.21.15.43:0/794170641 reap_dead start 2024-04-25T08:54:54.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.675+0000 7f1109ef8640 1 --2- 172.21.15.43:0/794170641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10f00039b0 0x7f10f0003dd0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1100001750 tx=0x7f10f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28d1d0402a379fab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:54.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.675+0000 7f1108ef6640 1 -- 172.21.15.43:0/794170641 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1100001370 con 0x7f10f00039b0 2024-04-25T08:54:54.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.675+0000 7f1108ef6640 1 -- 172.21.15.43:0/794170641 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11000052d0 con 0x7f10f00039b0 2024-04-25T08:54:54.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.675+0000 7f1108ef6640 1 -- 172.21.15.43:0/794170641 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110002abc0 con 0x7f10f00039b0 2024-04-25T08:54:54.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.675+0000 7f110c15c640 1 -- 172.21.15.43:0/794170641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10f00039b0 msgr2=0x7f10f0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:54.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.675+0000 7f110c15c640 1 --2- 172.21.15.43:0/794170641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10f00039b0 0x7f10f0003dd0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1100001750 tx=0x7f10f0005830 comp rx=0 tx=0).stop 2024-04-25T08:54:54.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.675+0000 7f110c15c640 1 -- 172.21.15.43:0/794170641 shutdown_connections 2024-04-25T08:54:54.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.675+0000 7f110c15c640 1 --2- 172.21.15.43:0/794170641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10f00039b0 0x7f10f0003dd0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:54.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.675+0000 7f110c15c640 1 -- 172.21.15.43:0/794170641 >> 172.21.15.43:0/794170641 conn(0x7f110409cd20 msgr2=0x7f11040d3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:54.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.675+0000 7f110c15c640 1 -- 172.21.15.43:0/794170641 shutdown_connections 2024-04-25T08:54:54.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.675+0000 7f110c15c640 1 -- 172.21.15.43:0/794170641 wait complete. 2024-04-25T08:54:54.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.676+0000 7f110c15c640 1 Processor -- start 2024-04-25T08:54:54.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.677+0000 7f110c15c640 1 -- start start 2024-04-25T08:54:54.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.677+0000 7f110c15c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10f00039b0 0x7f11040b8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:54.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.677+0000 7f110c15c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f11041592b0 0x7f1104159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:54.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.677+0000 7f110c15c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11041f4a40 0x7f11041f4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:54.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.677+0000 7f110c15c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11040d4200 con 0x7f10f00039b0 2024-04-25T08:54:54.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.677+0000 7f110c15c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11040d4080 con 0x7f11041f4a40 2024-04-25T08:54:54.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.677+0000 7f110c15c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11040d4380 con 0x7f11041592b0 2024-04-25T08:54:54.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.677+0000 7f1109ef8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10f00039b0 0x7f11040b8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:54.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.677+0000 7f1109ef8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10f00039b0 0x7f11040b8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49788/0 (socket says 172.21.15.43:49788) 2024-04-25T08:54:54.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.677+0000 7f1109ef8640 1 -- 172.21.15.43:0/4253478534 learned_addr learned my addr 172.21.15.43:0/4253478534 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:54.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.678+0000 7f11096f7640 1 --2- 172.21.15.43:0/4253478534 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f11041592b0 0x7f1104159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:54.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.678+0000 7f110a6f9640 1 --2- 172.21.15.43:0/4253478534 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11041f4a40 0x7f11041f4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:54.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.678+0000 7f1109ef8640 1 -- 172.21.15.43:0/4253478534 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f11041592b0 msgr2=0x7f1104159730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:54.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.678+0000 7f1109ef8640 1 --2- 172.21.15.43:0/4253478534 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f11041592b0 0x7f1104159730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:54.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.678+0000 7f1109ef8640 1 -- 172.21.15.43:0/4253478534 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11041f4a40 msgr2=0x7f11041f4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:54.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.678+0000 7f1109ef8640 1 --2- 172.21.15.43:0/4253478534 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11041f4a40 0x7f11041f4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:54.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.678+0000 7f1109ef8640 1 -- 172.21.15.43:0/4253478534 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11040b8f80 con 0x7f10f00039b0 2024-04-25T08:54:54.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.678+0000 7f1109ef8640 1 --2- 172.21.15.43:0/4253478534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10f00039b0 0x7f11040b8a40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1100000c00 tx=0x7f1100001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:54.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.678+0000 7f10fa7fc640 1 -- 172.21.15.43:0/4253478534 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110003f030 con 0x7f10f00039b0 2024-04-25T08:54:54.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.678+0000 7f10fa7fc640 1 -- 172.21.15.43:0/4253478534 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1100005480 con 0x7f10f00039b0 2024-04-25T08:54:54.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.678+0000 7f110c15c640 1 -- 172.21.15.43:0/4253478534 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11040b79a0 con 0x7f10f00039b0 2024-04-25T08:54:54.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.678+0000 7f10fa7fc640 1 -- 172.21.15.43:0/4253478534 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1100037040 con 0x7f10f00039b0 2024-04-25T08:54:54.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.678+0000 7f110c15c640 1 -- 172.21.15.43:0/4253478534 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11040b7f60 con 0x7f10f00039b0 2024-04-25T08:54:54.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.679+0000 7f110c15c640 1 -- 172.21.15.43:0/4253478534 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1104074f60 con 0x7f10f00039b0 2024-04-25T08:54:54.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.679+0000 7f10fa7fc640 1 -- 172.21.15.43:0/4253478534 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1100042070 con 0x7f10f00039b0 2024-04-25T08:54:54.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.680+0000 7f10fa7fc640 1 --2- 172.21.15.43:0/4253478534 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f10e806b500 0x7f10e806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:54.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.680+0000 7f10fa7fc640 1 -- 172.21.15.43:0/4253478534 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f11000a9650 con 0x7f10f00039b0 2024-04-25T08:54:54.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.680+0000 7f11096f7640 1 --2- 172.21.15.43:0/4253478534 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f10e806b500 0x7f10e806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:54.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.681+0000 7f11096f7640 1 --2- 172.21.15.43:0/4253478534 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f10e806b500 0x7f10e806d9c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f11040b9320 tx=0x7f10f4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:54.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.684+0000 7f10fa7fc640 1 -- 172.21.15.43:0/4253478534 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f110004a020 con 0x7f10f00039b0 2024-04-25T08:54:54.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.862+0000 7f110c15c640 1 -- 172.21.15.43:0/4253478534 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f11040c7fb0 con 0x7f10f00039b0 2024-04-25T08:54:54.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.863+0000 7f10fa7fc640 1 -- 172.21.15.43:0/4253478534 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f11000783b0 con 0x7f10f00039b0 2024-04-25T08:54:54.853 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T08:54:54.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.865+0000 7f110c15c640 1 -- 172.21.15.43:0/4253478534 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f10e806b500 msgr2=0x7f10e806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:54.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.865+0000 7f110c15c640 1 --2- 172.21.15.43:0/4253478534 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f10e806b500 0x7f10e806d9c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f11040b9320 tx=0x7f10f4009040 comp rx=0 tx=0).stop 2024-04-25T08:54:54.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.865+0000 7f110c15c640 1 -- 172.21.15.43:0/4253478534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10f00039b0 msgr2=0x7f11040b8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:54.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.865+0000 7f110c15c640 1 --2- 172.21.15.43:0/4253478534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10f00039b0 0x7f11040b8a40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f1100000c00 tx=0x7f1100001120 comp rx=0 tx=0).stop 2024-04-25T08:54:54.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.865+0000 7f110c15c640 1 -- 172.21.15.43:0/4253478534 shutdown_connections 2024-04-25T08:54:54.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.865+0000 7f110c15c640 1 --2- 172.21.15.43:0/4253478534 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f10e806b500 0x7f10e806d9c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:54.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.865+0000 7f110c15c640 1 --2- 172.21.15.43:0/4253478534 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11041f4a40 0x7f11041f4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:54.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.865+0000 7f110c15c640 1 --2- 172.21.15.43:0/4253478534 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f11041592b0 0x7f1104159730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:54.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.865+0000 7f110c15c640 1 --2- 172.21.15.43:0/4253478534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10f00039b0 0x7f11040b8a40 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:54.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.865+0000 7f110c15c640 1 -- 172.21.15.43:0/4253478534 >> 172.21.15.43:0/4253478534 conn(0x7f110409cd20 msgr2=0x7f11040d0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:54.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.865+0000 7f110c15c640 1 -- 172.21.15.43:0/4253478534 shutdown_connections 2024-04-25T08:54:54.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.865+0000 7f110c15c640 1 -- 172.21.15.43:0/4253478534 wait complete. 2024-04-25T08:54:54.866 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:54:54.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.980+0000 7f84d353b640 1 Processor -- start 2024-04-25T08:54:54.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.980+0000 7f84d353b640 1 -- start start 2024-04-25T08:54:54.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.981+0000 7f84d353b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f84cc0d1fa0 0x7f84cc0cece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:54.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.981+0000 7f84d353b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f84cc0cf220 0x7f84cc06b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:54.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.981+0000 7f84d353b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84cc0d47f0 con 0x7f84cc15dc40 2024-04-25T08:54:54.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.981+0000 7f84d353b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84cc0d13c0 con 0x7f84cc0cf220 2024-04-25T08:54:54.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.981+0000 7f84d353b640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84cc0d24b0 con 0x7f84cc0d1fa0 2024-04-25T08:54:54.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.981+0000 7f84d12d7640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f84cc15dc40 0x7f84cc0ce5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48430/0 (socket says 172.21.15.43:48430) 2024-04-25T08:54:54.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.981+0000 7f84d12d7640 1 -- 172.21.15.43:0/4177733591 learned_addr learned my addr 172.21.15.43:0/4177733591 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:54.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.981+0000 7f84d1ad8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f84cc0cf220 0x7f84cc06b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:54.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.981+0000 7f84d0ad6640 1 --2- 172.21.15.43:0/4177733591 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f84cc0d1fa0 0x7f84cc0cece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:54.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.981+0000 7f84d1ad8640 1 -- 172.21.15.43:0/4177733591 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f84cc0d1fa0 msgr2=0x7f84cc0cece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:54.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.981+0000 7f84d1ad8640 1 --2- 172.21.15.43:0/4177733591 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f84cc0d1fa0 0x7f84cc0cece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:54.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.981+0000 7f84d1ad8640 1 -- 172.21.15.43:0/4177733591 >> v1:172.21.15.43:6789/0 conn(0x7f84cc15dc40 legacy=0x7f84cc0ce5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:54.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.981+0000 7f84d1ad8640 1 -- 172.21.15.43:0/4177733591 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84cc0d2750 con 0x7f84cc0cf220 2024-04-25T08:54:54.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.981+0000 7f84c3fff640 1 -- 172.21.15.43:0/4177733591 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3965240039 0 0) 0x7f84cc0d47f0 con 0x7f84cc15dc40 2024-04-25T08:54:54.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.982+0000 7f84d1ad8640 1 --2- 172.21.15.43:0/4177733591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84cc0cf220 0x7f84cc06b970 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f84c4009d90 tx=0x7f84c402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb881c641c1ed45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:54.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.982+0000 7f84c3fff640 1 -- 172.21.15.43:0/4177733591 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c4031600 con 0x7f84cc0cf220 2024-04-25T08:54:54.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.982+0000 7f84c3fff640 1 -- 172.21.15.43:0/4177733591 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84c40317a0 con 0x7f84cc0cf220 2024-04-25T08:54:54.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.982+0000 7f84c3fff640 1 -- 172.21.15.43:0/4177733591 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c4031b70 con 0x7f84cc0cf220 2024-04-25T08:54:54.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.982+0000 7f84d353b640 1 -- 172.21.15.43:0/4177733591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84cc0cf220 msgr2=0x7f84cc06b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:54.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.983+0000 7f84d353b640 1 --2- 172.21.15.43:0/4177733591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84cc0cf220 0x7f84cc06b970 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f84c4009d90 tx=0x7f84c402a5d0 comp rx=0 tx=0).stop 2024-04-25T08:54:54.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.983+0000 7f84d353b640 1 -- 172.21.15.43:0/4177733591 shutdown_connections 2024-04-25T08:54:54.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.983+0000 7f84d353b640 1 --2- 172.21.15.43:0/4177733591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84cc0cf220 0x7f84cc06b970 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:54.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.983+0000 7f84d353b640 1 --2- 172.21.15.43:0/4177733591 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f84cc0d1fa0 0x7f84cc0cece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:54.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.983+0000 7f84d353b640 1 -- 172.21.15.43:0/4177733591 >> 172.21.15.43:0/4177733591 conn(0x7f84cc0d39c0 msgr2=0x7f84cc0d3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:54.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.983+0000 7f84d353b640 1 -- 172.21.15.43:0/4177733591 shutdown_connections 2024-04-25T08:54:54.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.983+0000 7f84d353b640 1 -- 172.21.15.43:0/4177733591 wait complete. 2024-04-25T08:54:54.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.984+0000 7f84d353b640 1 Processor -- start 2024-04-25T08:54:54.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.984+0000 7f84d353b640 1 -- start start 2024-04-25T08:54:54.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.985+0000 7f84d353b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84cc0cf220 0x7f84cc0b6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:54.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.985+0000 7f84d353b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f84cc0d1fa0 0x7f84cc14ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:54.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.985+0000 7f84d353b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84cc15dc40 0x7f84cc0cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:54.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.985+0000 7f84d353b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84cc0d2a30 con 0x7f84cc15dc40 2024-04-25T08:54:54.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.985+0000 7f84d353b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84cc06b450 con 0x7f84cc0cf220 2024-04-25T08:54:54.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.985+0000 7f84d353b640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84cc15e020 con 0x7f84cc0d1fa0 2024-04-25T08:54:54.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.985+0000 7f84d1ad8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84cc15dc40 0x7f84cc0cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:54.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.985+0000 7f84d1ad8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84cc15dc40 0x7f84cc0cd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49804/0 (socket says 172.21.15.43:49804) 2024-04-25T08:54:54.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.985+0000 7f84d1ad8640 1 -- 172.21.15.43:0/107689602 learned_addr learned my addr 172.21.15.43:0/107689602 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:54.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.985+0000 7f84d0ad6640 1 --2- 172.21.15.43:0/107689602 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f84cc0d1fa0 0x7f84cc14ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:54.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.985+0000 7f84d12d7640 1 --2- 172.21.15.43:0/107689602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84cc0cf220 0x7f84cc0b6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:54.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.985+0000 7f84d1ad8640 1 -- 172.21.15.43:0/107689602 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f84cc0d1fa0 msgr2=0x7f84cc14ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:54.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.985+0000 7f84d1ad8640 1 --2- 172.21.15.43:0/107689602 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f84cc0d1fa0 0x7f84cc14ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:54.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.985+0000 7f84d1ad8640 1 -- 172.21.15.43:0/107689602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84cc0cf220 msgr2=0x7f84cc0b6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:54.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.985+0000 7f84d1ad8640 1 --2- 172.21.15.43:0/107689602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84cc0cf220 0x7f84cc0b6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:54.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.985+0000 7f84d1ad8640 1 -- 172.21.15.43:0/107689602 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84cc15f370 con 0x7f84cc15dc40 2024-04-25T08:54:54.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.985+0000 7f84d0ad6640 1 --2- 172.21.15.43:0/107689602 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f84cc0d1fa0 0x7f84cc14ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:54.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.985+0000 7f84d1ad8640 1 --2- 172.21.15.43:0/107689602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84cc15dc40 0x7f84cc0cd3b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f84c402abc0 tx=0x7f84c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:54.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.985+0000 7f84c1ffb640 1 -- 172.21.15.43:0/107689602 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c403f070 con 0x7f84cc15dc40 2024-04-25T08:54:54.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.985+0000 7f84c1ffb640 1 -- 172.21.15.43:0/107689602 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84c4030b60 con 0x7f84cc15dc40 2024-04-25T08:54:54.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.985+0000 7f84d353b640 1 -- 172.21.15.43:0/107689602 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84cc15f650 con 0x7f84cc15dc40 2024-04-25T08:54:54.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.985+0000 7f84c1ffb640 1 -- 172.21.15.43:0/107689602 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c4044460 con 0x7f84cc15dc40 2024-04-25T08:54:54.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.986+0000 7f84d353b640 1 -- 172.21.15.43:0/107689602 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84cc15fc10 con 0x7f84cc15dc40 2024-04-25T08:54:54.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.987+0000 7f84d353b640 1 -- 172.21.15.43:0/107689602 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84cc074f60 con 0x7f84cc15dc40 2024-04-25T08:54:54.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.987+0000 7f84c1ffb640 1 -- 172.21.15.43:0/107689602 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f84c403b410 con 0x7f84cc15dc40 2024-04-25T08:54:54.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.988+0000 7f84c1ffb640 1 --2- 172.21.15.43:0/107689602 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f84b006b4d0 0x7f84b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:54.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.988+0000 7f84c1ffb640 1 -- 172.21.15.43:0/107689602 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f84c40ab990 con 0x7f84cc15dc40 2024-04-25T08:54:54.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.988+0000 7f84d12d7640 1 --2- 172.21.15.43:0/107689602 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f84b006b4d0 0x7f84b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:54.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.988+0000 7f84d12d7640 1 --2- 172.21.15.43:0/107689602 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f84b006b4d0 0x7f84b006d990 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f84c8000f60 tx=0x7f84c8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:54.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:54.992+0000 7f84c1ffb640 1 -- 172.21.15.43:0/107689602 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f84c407a840 con 0x7f84cc15dc40 2024-04-25T08:54:55.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.170+0000 7f84d353b640 1 -- 172.21.15.43:0/107689602 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f84cc0cdd80 con 0x7f84cc15dc40 2024-04-25T08:54:55.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.170+0000 7f84c1ffb640 1 -- 172.21.15.43:0/107689602 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f84c4080070 con 0x7f84cc15dc40 2024-04-25T08:54:55.161 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T08:54:55.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.172+0000 7f84d353b640 1 -- 172.21.15.43:0/107689602 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f84b006b4d0 msgr2=0x7f84b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:55.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.172+0000 7f84d353b640 1 --2- 172.21.15.43:0/107689602 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f84b006b4d0 0x7f84b006d990 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f84c8000f60 tx=0x7f84c8009040 comp rx=0 tx=0).stop 2024-04-25T08:54:55.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.173+0000 7f84d353b640 1 -- 172.21.15.43:0/107689602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84cc15dc40 msgr2=0x7f84cc0cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:55.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.173+0000 7f84d353b640 1 --2- 172.21.15.43:0/107689602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84cc15dc40 0x7f84cc0cd3b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f84c402abc0 tx=0x7f84c4004a40 comp rx=0 tx=0).stop 2024-04-25T08:54:55.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.173+0000 7f84d353b640 1 -- 172.21.15.43:0/107689602 shutdown_connections 2024-04-25T08:54:55.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.173+0000 7f84d353b640 1 --2- 172.21.15.43:0/107689602 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f84cc15dc40 0x7f84cc0cd3b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:55.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.173+0000 7f84d353b640 1 --2- 172.21.15.43:0/107689602 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f84cc0d1fa0 0x7f84cc14ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:55.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.173+0000 7f84d353b640 1 --2- 172.21.15.43:0/107689602 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f84b006b4d0 0x7f84b006d990 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:55.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.173+0000 7f84d353b640 1 --2- 172.21.15.43:0/107689602 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f84cc0cf220 0x7f84cc0b6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:55.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.173+0000 7f84d353b640 1 -- 172.21.15.43:0/107689602 >> 172.21.15.43:0/107689602 conn(0x7f84cc0d39c0 msgr2=0x7f84cc0d3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:55.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.173+0000 7f84d353b640 1 -- 172.21.15.43:0/107689602 shutdown_connections 2024-04-25T08:54:55.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.173+0000 7f84d353b640 1 -- 172.21.15.43:0/107689602 wait complete. 2024-04-25T08:54:55.173 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:54:55.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.284+0000 7ff558d53640 1 Processor -- start 2024-04-25T08:54:55.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.284+0000 7ff558d53640 1 -- start start 2024-04-25T08:54:55.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.285+0000 7ff558d53640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff55415dd00 0x7ff5540d2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:55.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.285+0000 7ff558d53640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff5540ce970 0x7ff5540b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:55.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.285+0000 7ff558d53640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff55415e0e0 con 0x7ff5540d33f0 2024-04-25T08:54:55.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.285+0000 7ff558d53640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5540d3850 con 0x7ff5540ce970 2024-04-25T08:54:55.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.285+0000 7ff558d53640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5540d24e0 con 0x7ff55415dd00 2024-04-25T08:54:55.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.285+0000 7ff55259c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff55415dd00 0x7ff5540d2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:55.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.285+0000 7ff552d9d640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff5540ce970 0x7ff5540b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:55.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.285+0000 7ff55259c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff55415dd00 0x7ff5540d2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:57962/0 (socket says 172.21.15.43:57962) 2024-04-25T08:54:55.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.285+0000 7ff552d9d640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff5540ce970 0x7ff5540b6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:46492/0 (socket says 172.21.15.43:46492) 2024-04-25T08:54:55.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.285+0000 7ff551d9b640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7ff5540d33f0 0x7ff5540ce260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:39460/0 (socket says 172.21.15.43:39460) 2024-04-25T08:54:55.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.285+0000 7ff55259c640 1 -- 172.21.15.43:0/2161530619 learned_addr learned my addr 172.21.15.43:0/2161530619 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:55.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.285+0000 7ff552d9d640 1 -- 172.21.15.43:0/2161530619 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff55415dd00 msgr2=0x7ff5540d2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:55.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.285+0000 7ff552d9d640 1 --2- 172.21.15.43:0/2161530619 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff55415dd00 0x7ff5540d2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:55.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.285+0000 7ff552d9d640 1 -- 172.21.15.43:0/2161530619 >> v1:172.21.15.193:6789/0 conn(0x7ff5540d33f0 legacy=0x7ff5540ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:55.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.285+0000 7ff552d9d640 1 -- 172.21.15.43:0/2161530619 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5540ceeb0 con 0x7ff5540ce970 2024-04-25T08:54:55.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.285+0000 7ff55259c640 1 --2- 172.21.15.43:0/2161530619 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff55415dd00 0x7ff5540d2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:54:55.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.285+0000 7ff55159a640 1 -- 172.21.15.43:0/2161530619 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 444775617 0 0) 0x7ff55415e0e0 con 0x7ff5540d33f0 2024-04-25T08:54:55.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.286+0000 7ff552d9d640 1 --2- 172.21.15.43:0/2161530619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5540ce970 0x7ff5540b6a00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff5540d2660 tx=0x7ff54c02a640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa0fd8f2449b4795 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:55.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.286+0000 7ff55159a640 1 -- 172.21.15.43:0/2161530619 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54c02ae60 con 0x7ff5540ce970 2024-04-25T08:54:55.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.286+0000 7ff55159a640 1 -- 172.21.15.43:0/2161530619 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff54c007740 con 0x7ff5540ce970 2024-04-25T08:54:55.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.286+0000 7ff55159a640 1 -- 172.21.15.43:0/2161530619 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54c007b10 con 0x7ff5540ce970 2024-04-25T08:54:55.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.287+0000 7ff558d53640 1 -- 172.21.15.43:0/2161530619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5540ce970 msgr2=0x7ff5540b6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:55.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.287+0000 7ff558d53640 1 --2- 172.21.15.43:0/2161530619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5540ce970 0x7ff5540b6a00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff5540d2660 tx=0x7ff54c02a640 comp rx=0 tx=0).stop 2024-04-25T08:54:55.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.287+0000 7ff558d53640 1 -- 172.21.15.43:0/2161530619 shutdown_connections 2024-04-25T08:54:55.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.287+0000 7ff558d53640 1 --2- 172.21.15.43:0/2161530619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5540ce970 0x7ff5540b6a00 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:55.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.287+0000 7ff558d53640 1 --2- 172.21.15.43:0/2161530619 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff55415dd00 0x7ff5540d2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:55.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.287+0000 7ff558d53640 1 -- 172.21.15.43:0/2161530619 >> 172.21.15.43:0/2161530619 conn(0x7ff55406cbd0 msgr2=0x7ff55406cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:55.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.287+0000 7ff558d53640 1 -- 172.21.15.43:0/2161530619 shutdown_connections 2024-04-25T08:54:55.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.287+0000 7ff558d53640 1 -- 172.21.15.43:0/2161530619 wait complete. 2024-04-25T08:54:55.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.288+0000 7ff558d53640 1 Processor -- start 2024-04-25T08:54:55.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.288+0000 7ff558d53640 1 -- start start 2024-04-25T08:54:55.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.289+0000 7ff558d53640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5540ce970 0x7ff5540afe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:55.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.289+0000 7ff558d53640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff5540d33f0 0x7ff55414f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:55.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.289+0000 7ff558d53640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff55415dd00 0x7ff5540abc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:55.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.289+0000 7ff558d53640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5540d2170 con 0x7ff5540ce970 2024-04-25T08:54:55.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.289+0000 7ff558d53640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5540d1e90 con 0x7ff55415dd00 2024-04-25T08:54:55.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.289+0000 7ff558d53640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5540d2970 con 0x7ff5540d33f0 2024-04-25T08:54:55.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.289+0000 7ff55259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5540ce970 0x7ff5540afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:55.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.289+0000 7ff55259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5540ce970 0x7ff5540afe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49818/0 (socket says 172.21.15.43:49818) 2024-04-25T08:54:55.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.289+0000 7ff55259c640 1 -- 172.21.15.43:0/3252766337 learned_addr learned my addr 172.21.15.43:0/3252766337 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:55.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.289+0000 7ff551d9b640 1 --2- 172.21.15.43:0/3252766337 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff5540d33f0 0x7ff55414f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:55.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.289+0000 7ff552d9d640 1 --2- 172.21.15.43:0/3252766337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff55415dd00 0x7ff5540abc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:55.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.290+0000 7ff55259c640 1 -- 172.21.15.43:0/3252766337 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff5540d33f0 msgr2=0x7ff55414f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:55.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.290+0000 7ff55259c640 1 --2- 172.21.15.43:0/3252766337 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff5540d33f0 0x7ff55414f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:55.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.290+0000 7ff55259c640 1 -- 172.21.15.43:0/3252766337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff55415dd00 msgr2=0x7ff5540abc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:55.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.290+0000 7ff55259c640 1 --2- 172.21.15.43:0/3252766337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff55415dd00 0x7ff5540abc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:55.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.290+0000 7ff55259c640 1 -- 172.21.15.43:0/3252766337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5541501b0 con 0x7ff5540ce970 2024-04-25T08:54:55.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.290+0000 7ff55259c640 1 --2- 172.21.15.43:0/3252766337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5540ce970 0x7ff5540afe10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff54400e9d0 tx=0x7ff54400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:55.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.290+0000 7ff542ffd640 1 -- 172.21.15.43:0/3252766337 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff544017070 con 0x7ff5540ce970 2024-04-25T08:54:55.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.290+0000 7ff542ffd640 1 -- 172.21.15.43:0/3252766337 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff54400ca60 con 0x7ff5540ce970 2024-04-25T08:54:55.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.290+0000 7ff542ffd640 1 -- 172.21.15.43:0/3252766337 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54400ce10 con 0x7ff5540ce970 2024-04-25T08:54:55.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.290+0000 7ff558d53640 1 -- 172.21.15.43:0/3252766337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff55414fbf0 con 0x7ff5540ce970 2024-04-25T08:54:55.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.290+0000 7ff558d53640 1 -- 172.21.15.43:0/3252766337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5540ab450 con 0x7ff5540ce970 2024-04-25T08:54:55.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.291+0000 7ff558d53640 1 -- 172.21.15.43:0/3252766337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff554074ef0 con 0x7ff5540ce970 2024-04-25T08:54:55.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.292+0000 7ff542ffd640 1 -- 172.21.15.43:0/3252766337 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff54401a070 con 0x7ff5540ce970 2024-04-25T08:54:55.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.292+0000 7ff542ffd640 1 --2- 172.21.15.43:0/3252766337 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff53406b500 0x7ff53406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:55.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.292+0000 7ff542ffd640 1 -- 172.21.15.43:0/3252766337 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7ff54408a1f0 con 0x7ff5540ce970 2024-04-25T08:54:55.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.292+0000 7ff551d9b640 1 --2- 172.21.15.43:0/3252766337 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff53406b500 0x7ff53406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:55.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.293+0000 7ff551d9b640 1 --2- 172.21.15.43:0/3252766337 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff53406b500 0x7ff53406d9c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff55415c280 tx=0x7ff548007b50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:55.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.297+0000 7ff542ffd640 1 -- 172.21.15.43:0/3252766337 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff544058fe0 con 0x7ff5540ce970 2024-04-25T08:54:55.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.513+0000 7ff558d53640 1 -- 172.21.15.43:0/3252766337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff5540b25f0 con 0x7ff5540ce970 2024-04-25T08:54:55.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.514+0000 7ff542ffd640 1 -- 172.21.15.43:0/3252766337 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4938 (secure 0 0 0) 0x7ff5440588e0 con 0x7ff5540ce970 2024-04-25T08:54:55.505 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:54:55.505 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3616994003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":3616994003},{"type":"v1","addr":"172.21.15.43:6835","nonce":3616994003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/4022901908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4022901908},{"type":"v1","addr":"172.21.15.193:6835","nonce":4022901908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2823450590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2823450590},{"type":"v1","addr":"172.21.15.43:6834","nonce":2823450590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:54:11.945963+0000","modified":"2024-04-25T08:54:13.943002+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.193:6837/1197434598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1197434598},{"type":"v1","addr":"172.21.15.193:6837","nonce":1197434598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:54:55.505 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 6 2024-04-25T08:54:55.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.516+0000 7ff558d53640 1 -- 172.21.15.43:0/3252766337 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff53406b500 msgr2=0x7ff53406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:55.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.516+0000 7ff558d53640 1 --2- 172.21.15.43:0/3252766337 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff53406b500 0x7ff53406d9c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff55415c280 tx=0x7ff548007b50 comp rx=0 tx=0).stop 2024-04-25T08:54:55.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.516+0000 7ff558d53640 1 -- 172.21.15.43:0/3252766337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5540ce970 msgr2=0x7ff5540afe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:55.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.516+0000 7ff558d53640 1 --2- 172.21.15.43:0/3252766337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5540ce970 0x7ff5540afe10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff54400e9d0 tx=0x7ff54400ee90 comp rx=0 tx=0).stop 2024-04-25T08:54:55.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.516+0000 7ff558d53640 1 -- 172.21.15.43:0/3252766337 shutdown_connections 2024-04-25T08:54:55.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.516+0000 7ff558d53640 1 --2- 172.21.15.43:0/3252766337 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff53406b500 0x7ff53406d9c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:55.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.516+0000 7ff558d53640 1 --2- 172.21.15.43:0/3252766337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff55415dd00 0x7ff5540abc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:55.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.516+0000 7ff558d53640 1 --2- 172.21.15.43:0/3252766337 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff5540d33f0 0x7ff55414f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:55.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.516+0000 7ff558d53640 1 --2- 172.21.15.43:0/3252766337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5540ce970 0x7ff5540afe10 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:55.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.516+0000 7ff558d53640 1 -- 172.21.15.43:0/3252766337 >> 172.21.15.43:0/3252766337 conn(0x7ff55406cbd0 msgr2=0x7ff5540b5070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:55.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.516+0000 7ff558d53640 1 -- 172.21.15.43:0/3252766337 shutdown_connections 2024-04-25T08:54:55.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.516+0000 7ff558d53640 1 -- 172.21.15.43:0/3252766337 wait complete. 2024-04-25T08:54:55.517 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:54:55.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.628+0000 7f46a5fd5640 1 Processor -- start 2024-04-25T08:54:55.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.628+0000 7f46a5fd5640 1 -- start start 2024-04-25T08:54:55.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.629+0000 7f46a5fd5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f46a00d0af0 0x7f46a0160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:55.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.629+0000 7f46a5fd5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46a00d2920 con 0x7f46a015dce0 2024-04-25T08:54:55.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.629+0000 7f46a5fd5640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46a00cee80 con 0x7f46a015ef50 2024-04-25T08:54:55.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.629+0000 7f46a5fd5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46a00cec60 con 0x7f46a00d0af0 2024-04-25T08:54:55.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.629+0000 7f469ffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f46a00d0af0 0x7f46a0160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:55.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.629+0000 7f469f7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f46a015dce0 0x7f46a00ce550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48446/0 (socket says 172.21.15.43:48446) 2024-04-25T08:54:55.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.629+0000 7f469f7fe640 1 -- 172.21.15.43:0/2032513698 learned_addr learned my addr 172.21.15.43:0/2032513698 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:55.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.629+0000 7f469ffff640 1 -- 172.21.15.43:0/2032513698 >> v1:172.21.15.193:6789/0 conn(0x7f46a015ef50 legacy=0x7f46a00d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:55.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.629+0000 7f469ffff640 1 -- 172.21.15.43:0/2032513698 >> v1:172.21.15.43:6789/0 conn(0x7f46a015dce0 legacy=0x7f46a00ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:55.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.629+0000 7f469ffff640 1 -- 172.21.15.43:0/2032513698 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a00cf160 con 0x7f46a00d0af0 2024-04-25T08:54:55.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.630+0000 7f469ffff640 1 --2- 172.21.15.43:0/2032513698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46a00d0af0 0x7f46a0160980 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4698009d90 tx=0x7f469802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=689901d6882cff0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:55.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.630+0000 7f469e7fc640 1 -- 172.21.15.43:0/2032513698 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4698007740 con 0x7f46a00d0af0 2024-04-25T08:54:55.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.630+0000 7f469e7fc640 1 -- 172.21.15.43:0/2032513698 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46980078e0 con 0x7f46a00d0af0 2024-04-25T08:54:55.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.630+0000 7f469e7fc640 1 -- 172.21.15.43:0/2032513698 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4698007cb0 con 0x7f46a00d0af0 2024-04-25T08:54:55.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.631+0000 7f46a5fd5640 1 -- 172.21.15.43:0/2032513698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46a00d0af0 msgr2=0x7f46a0160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:55.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.631+0000 7f46a5fd5640 1 --2- 172.21.15.43:0/2032513698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46a00d0af0 0x7f46a0160980 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f4698009d90 tx=0x7f469802a660 comp rx=0 tx=0).stop 2024-04-25T08:54:55.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.631+0000 7f46a5fd5640 1 -- 172.21.15.43:0/2032513698 shutdown_connections 2024-04-25T08:54:55.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.631+0000 7f46a5fd5640 1 --2- 172.21.15.43:0/2032513698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46a00d0af0 0x7f46a0160980 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:55.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.631+0000 7f46a5fd5640 1 -- 172.21.15.43:0/2032513698 >> 172.21.15.43:0/2032513698 conn(0x7f46a009cd00 msgr2=0x7f46a00d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:55.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.631+0000 7f46a5fd5640 1 -- 172.21.15.43:0/2032513698 shutdown_connections 2024-04-25T08:54:55.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.631+0000 7f46a5fd5640 1 -- 172.21.15.43:0/2032513698 wait complete. 2024-04-25T08:54:55.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.632+0000 7f46a5fd5640 1 Processor -- start 2024-04-25T08:54:55.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.632+0000 7f46a5fd5640 1 -- start start 2024-04-25T08:54:55.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.633+0000 7f46a5fd5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f46a00d0af0 0x7f46a01f4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:55.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.633+0000 7f46a5fd5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f46a015dce0 0x7f46a00ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:55.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.633+0000 7f46a5fd5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46a015ef50 0x7f46a00c7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:55.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.633+0000 7f46a5fd5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a015f6b0 con 0x7f46a015ef50 2024-04-25T08:54:55.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.633+0000 7f46a5fd5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a00cf720 con 0x7f46a00d0af0 2024-04-25T08:54:55.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.633+0000 7f46a5fd5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46a015e0c0 con 0x7f46a015dce0 2024-04-25T08:54:55.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.633+0000 7f469ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46a015ef50 0x7f46a00c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:55.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.633+0000 7f469ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46a015ef50 0x7f46a00c7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49838/0 (socket says 172.21.15.43:49838) 2024-04-25T08:54:55.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.633+0000 7f469ffff640 1 -- 172.21.15.43:0/759932631 learned_addr learned my addr 172.21.15.43:0/759932631 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:55.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.633+0000 7f469f7fe640 1 --2- 172.21.15.43:0/759932631 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f46a00d0af0 0x7f46a01f4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:55.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.633+0000 7f469effd640 1 --2- 172.21.15.43:0/759932631 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f46a015dce0 0x7f46a00ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:55.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.633+0000 7f469ffff640 1 -- 172.21.15.43:0/759932631 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f46a015dce0 msgr2=0x7f46a00ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:55.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.633+0000 7f469ffff640 1 --2- 172.21.15.43:0/759932631 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f46a015dce0 0x7f46a00ccd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:55.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.633+0000 7f469ffff640 1 -- 172.21.15.43:0/759932631 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f46a00d0af0 msgr2=0x7f46a01f4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:55.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.633+0000 7f469ffff640 1 --2- 172.21.15.43:0/759932631 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f46a00d0af0 0x7f46a01f4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:55.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.633+0000 7f469ffff640 1 -- 172.21.15.43:0/759932631 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a00cd270 con 0x7f46a015ef50 2024-04-25T08:54:55.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.633+0000 7f469f7fe640 1 --2- 172.21.15.43:0/759932631 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f46a00d0af0 0x7f46a01f4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:55.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.633+0000 7f469effd640 1 --2- 172.21.15.43:0/759932631 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f46a015dce0 0x7f46a00ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:55.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.634+0000 7f469ffff640 1 --2- 172.21.15.43:0/759932631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46a015ef50 0x7f46a00c7520 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4698009eb0 tx=0x7f4698004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:55.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.634+0000 7f4687fff640 1 -- 172.21.15.43:0/759932631 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4698040070 con 0x7f46a015ef50 2024-04-25T08:54:55.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.634+0000 7f4687fff640 1 -- 172.21.15.43:0/759932631 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4698031c90 con 0x7f46a015ef50 2024-04-25T08:54:55.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.634+0000 7f46a5fd5640 1 -- 172.21.15.43:0/759932631 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46a00cd530 con 0x7f46a015ef50 2024-04-25T08:54:55.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.634+0000 7f4687fff640 1 -- 172.21.15.43:0/759932631 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4698045680 con 0x7f46a015ef50 2024-04-25T08:54:55.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.634+0000 7f46a5fd5640 1 -- 172.21.15.43:0/759932631 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46a00cdaf0 con 0x7f46a015ef50 2024-04-25T08:54:55.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.635+0000 7f46a5fd5640 1 -- 172.21.15.43:0/759932631 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46a00680f0 con 0x7f46a015ef50 2024-04-25T08:54:55.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.635+0000 7f4687fff640 1 -- 172.21.15.43:0/759932631 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4698039040 con 0x7f46a015ef50 2024-04-25T08:54:55.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.636+0000 7f4687fff640 1 --2- 172.21.15.43:0/759932631 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f468c06b4d0 0x7f468c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:55.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.636+0000 7f4687fff640 1 -- 172.21.15.43:0/759932631 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f46980abce0 con 0x7f46a015ef50 2024-04-25T08:54:55.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.636+0000 7f469f7fe640 1 --2- 172.21.15.43:0/759932631 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f468c06b4d0 0x7f468c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:55.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.637+0000 7f469f7fe640 1 --2- 172.21.15.43:0/759932631 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f468c06b4d0 0x7f468c06d990 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4690000f60 tx=0x7f4690009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:55.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.640+0000 7f4687fff640 1 -- 172.21.15.43:0/759932631 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f469807ab10 con 0x7f46a015ef50 2024-04-25T08:54:55.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.819+0000 7f46a5fd5640 1 -- 172.21.15.43:0/759932631 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46a009d1a0 con 0x7f46a015ef50 2024-04-25T08:54:55.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.820+0000 7f4687fff640 1 -- 172.21.15.43:0/759932631 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16809 (secure 0 0 0) 0x7f469807a410 con 0x7f46a015ef50 2024-04-25T08:54:55.810 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:54:55.811 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":22,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:54:52.847242+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:54:09.757978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:54:10.926008+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.18","mappings":[{"from":3,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2d","mappings":[{"from":7,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:54:55.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.822+0000 7f46a5fd5640 1 -- 172.21.15.43:0/759932631 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f468c06b4d0 msgr2=0x7f468c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:55.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.822+0000 7f46a5fd5640 1 --2- 172.21.15.43:0/759932631 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f468c06b4d0 0x7f468c06d990 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4690000f60 tx=0x7f4690009040 comp rx=0 tx=0).stop 2024-04-25T08:54:55.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.822+0000 7f46a5fd5640 1 -- 172.21.15.43:0/759932631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46a015ef50 msgr2=0x7f46a00c7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:55.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.822+0000 7f46a5fd5640 1 --2- 172.21.15.43:0/759932631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46a015ef50 0x7f46a00c7520 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4698009eb0 tx=0x7f4698004a40 comp rx=0 tx=0).stop 2024-04-25T08:54:55.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.822+0000 7f46a5fd5640 1 -- 172.21.15.43:0/759932631 shutdown_connections 2024-04-25T08:54:55.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.822+0000 7f46a5fd5640 1 --2- 172.21.15.43:0/759932631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46a015ef50 0x7f46a00c7520 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:55.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.822+0000 7f46a5fd5640 1 --2- 172.21.15.43:0/759932631 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f46a015dce0 0x7f46a00ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:55.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.822+0000 7f46a5fd5640 1 --2- 172.21.15.43:0/759932631 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f468c06b4d0 0x7f468c06d990 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:55.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.822+0000 7f46a5fd5640 1 --2- 172.21.15.43:0/759932631 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f46a00d0af0 0x7f46a01f4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:55.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.822+0000 7f46a5fd5640 1 -- 172.21.15.43:0/759932631 >> 172.21.15.43:0/759932631 conn(0x7f46a009cd00 msgr2=0x7f46a00ce510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:55.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.823+0000 7f46a5fd5640 1 -- 172.21.15.43:0/759932631 shutdown_connections 2024-04-25T08:54:55.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.823+0000 7f46a5fd5640 1 -- 172.21.15.43:0/759932631 wait complete. 2024-04-25T08:54:55.824 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T08:54:55.824 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:54:55.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.935+0000 7feb3df5c640 1 Processor -- start 2024-04-25T08:54:55.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.935+0000 7feb3df5c640 1 -- start start 2024-04-25T08:54:55.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.936+0000 7feb3df5c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7feb380715a0 0x7feb38071a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:55.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.936+0000 7feb3df5c640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7feb380b3100 0x7feb380b3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:55.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.936+0000 7feb3df5c640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb3806dd30 con 0x7feb3815a250 2024-04-25T08:54:55.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.936+0000 7feb3df5c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb3806e010 con 0x7feb380715a0 2024-04-25T08:54:55.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.936+0000 7feb3df5c640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb380b2900 con 0x7feb380b3100 2024-04-25T08:54:55.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.936+0000 7feb36ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7feb380715a0 0x7feb38071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:55.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.936+0000 7feb377fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7feb3815a250 0x7feb38070d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48460/0 (socket says 172.21.15.43:48460) 2024-04-25T08:54:55.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.936+0000 7feb377fe640 1 -- 172.21.15.43:0/3316972991 learned_addr learned my addr 172.21.15.43:0/3316972991 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:55.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.936+0000 7feb37fff640 1 --2- 172.21.15.43:0/3316972991 >> v2:172.21.15.193:3300/0 conn(0x7feb380b3100 0x7feb380b3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:55.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.937+0000 7feb36ffd640 1 -- 172.21.15.43:0/3316972991 >> v2:172.21.15.193:3300/0 conn(0x7feb380b3100 msgr2=0x7feb380b3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:55.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.937+0000 7feb36ffd640 1 --2- 172.21.15.43:0/3316972991 >> v2:172.21.15.193:3300/0 conn(0x7feb380b3100 0x7feb380b3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:55.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.937+0000 7feb36ffd640 1 -- 172.21.15.43:0/3316972991 >> v1:172.21.15.43:6789/0 conn(0x7feb3815a250 legacy=0x7feb38070d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:55.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.937+0000 7feb36ffd640 1 -- 172.21.15.43:0/3316972991 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb380b2be0 con 0x7feb380715a0 2024-04-25T08:54:55.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.937+0000 7feb36ffd640 1 --2- 172.21.15.43:0/3316972991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb380715a0 0x7feb38071a30 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7feb380b21e0 tx=0x7feb2c02b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=301513e0af47aa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:55.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.938+0000 7feb367fc640 1 -- 172.21.15.43:0/3316972991 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2c038070 con 0x7feb380715a0 2024-04-25T08:54:55.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.938+0000 7feb367fc640 1 -- 172.21.15.43:0/3316972991 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb2c033640 con 0x7feb380715a0 2024-04-25T08:54:55.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.938+0000 7feb367fc640 1 -- 172.21.15.43:0/3316972991 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2c00b040 con 0x7feb380715a0 2024-04-25T08:54:55.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.938+0000 7feb3df5c640 1 -- 172.21.15.43:0/3316972991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb380715a0 msgr2=0x7feb38071a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:55.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.938+0000 7feb3df5c640 1 --2- 172.21.15.43:0/3316972991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb380715a0 0x7feb38071a30 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7feb380b21e0 tx=0x7feb2c02b460 comp rx=0 tx=0).stop 2024-04-25T08:54:55.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.938+0000 7feb3df5c640 1 -- 172.21.15.43:0/3316972991 shutdown_connections 2024-04-25T08:54:55.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.938+0000 7feb3df5c640 1 --2- 172.21.15.43:0/3316972991 >> v2:172.21.15.193:3300/0 conn(0x7feb380b3100 0x7feb380b3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:55.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.938+0000 7feb3df5c640 1 --2- 172.21.15.43:0/3316972991 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb380715a0 0x7feb38071a30 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:55.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.938+0000 7feb3df5c640 1 -- 172.21.15.43:0/3316972991 >> 172.21.15.43:0/3316972991 conn(0x7feb380b1480 msgr2=0x7feb380b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:55.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.938+0000 7feb3df5c640 1 -- 172.21.15.43:0/3316972991 shutdown_connections 2024-04-25T08:54:55.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.938+0000 7feb3df5c640 1 -- 172.21.15.43:0/3316972991 wait complete. 2024-04-25T08:54:55.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.939+0000 7feb3df5c640 1 Processor -- start 2024-04-25T08:54:55.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.939+0000 7feb3df5c640 1 -- start start 2024-04-25T08:54:55.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.940+0000 7feb3df5c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feb380715a0 0x7feb381e3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:55.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.940+0000 7feb3df5c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb380b3100 0x7feb381ef260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:55.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.940+0000 7feb3df5c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb3815a250 0x7feb381ef7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:55.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.940+0000 7feb3df5c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3806d410 con 0x7feb3815a250 2024-04-25T08:54:55.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.940+0000 7feb3df5c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3806d290 con 0x7feb380b3100 2024-04-25T08:54:55.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.940+0000 7feb3df5c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb3806d590 con 0x7feb380715a0 2024-04-25T08:54:55.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.940+0000 7feb37fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb3815a250 0x7feb381ef7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:55.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.940+0000 7feb37fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb3815a250 0x7feb381ef7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49858/0 (socket says 172.21.15.43:49858) 2024-04-25T08:54:55.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.940+0000 7feb37fff640 1 -- 172.21.15.43:0/665638505 learned_addr learned my addr 172.21.15.43:0/665638505 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:55.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.941+0000 7feb37fff640 1 -- 172.21.15.43:0/665638505 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feb380715a0 msgr2=0x7feb381e3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:55.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.941+0000 7feb37fff640 1 --2- 172.21.15.43:0/665638505 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feb380715a0 0x7feb381e3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:55.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.941+0000 7feb37fff640 1 -- 172.21.15.43:0/665638505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb380b3100 msgr2=0x7feb381ef260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:55.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.941+0000 7feb37fff640 1 --2- 172.21.15.43:0/665638505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb380b3100 0x7feb381ef260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:55.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.941+0000 7feb37fff640 1 -- 172.21.15.43:0/665638505 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb380aa7d0 con 0x7feb3815a250 2024-04-25T08:54:55.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.941+0000 7feb37fff640 1 --2- 172.21.15.43:0/665638505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb3815a250 0x7feb381ef7a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7feb3000efc0 tx=0x7feb3000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:55.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.941+0000 7feb1ffff640 1 -- 172.21.15.43:0/665638505 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb30018020 con 0x7feb3815a250 2024-04-25T08:54:55.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.941+0000 7feb1ffff640 1 -- 172.21.15.43:0/665638505 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb30009b10 con 0x7feb3815a250 2024-04-25T08:54:55.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.941+0000 7feb3df5c640 1 -- 172.21.15.43:0/665638505 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb380aaab0 con 0x7feb3815a250 2024-04-25T08:54:55.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.941+0000 7feb1ffff640 1 -- 172.21.15.43:0/665638505 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb300146c0 con 0x7feb3815a250 2024-04-25T08:54:55.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.941+0000 7feb3df5c640 1 -- 172.21.15.43:0/665638505 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb380ab070 con 0x7feb3815a250 2024-04-25T08:54:55.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.942+0000 7feb3df5c640 1 -- 172.21.15.43:0/665638505 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb38074ef0 con 0x7feb3815a250 2024-04-25T08:54:55.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.943+0000 7feb1ffff640 1 -- 172.21.15.43:0/665638505 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feb30005d10 con 0x7feb3815a250 2024-04-25T08:54:55.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.943+0000 7feb1ffff640 1 --2- 172.21.15.43:0/665638505 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7feb2406b4d0 0x7feb2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:55.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.943+0000 7feb1ffff640 1 -- 172.21.15.43:0/665638505 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7feb3008ac90 con 0x7feb3815a250 2024-04-25T08:54:55.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.943+0000 7feb377fe640 1 --2- 172.21.15.43:0/665638505 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7feb2406b4d0 0x7feb2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:55.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.944+0000 7feb377fe640 1 --2- 172.21.15.43:0/665638505 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7feb2406b4d0 0x7feb2406d990 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7feb28007fd0 tx=0x7feb280079a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:55.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:55.947+0000 7feb1ffff640 1 -- 172.21.15.43:0/665638505 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feb30022070 con 0x7feb3815a250 2024-04-25T08:54:56.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.166+0000 7feb3df5c640 1 -- 172.21.15.43:0/665638505 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb380ae8e0 con 0x7feb3815a250 2024-04-25T08:54:56.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.166+0000 7feb1ffff640 1 -- 172.21.15.43:0/665638505 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4938 (secure 0 0 0) 0x7feb30010040 con 0x7feb3815a250 2024-04-25T08:54:56.156 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:54:56.156 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3616994003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":3616994003},{"type":"v1","addr":"172.21.15.43:6835","nonce":3616994003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/4022901908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4022901908},{"type":"v1","addr":"172.21.15.193:6835","nonce":4022901908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2823450590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2823450590},{"type":"v1","addr":"172.21.15.43:6834","nonce":2823450590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:54:11.945963+0000","modified":"2024-04-25T08:54:13.943002+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.193:6837/1197434598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1197434598},{"type":"v1","addr":"172.21.15.193:6837","nonce":1197434598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:54:56.157 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 6 2024-04-25T08:54:56.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.168+0000 7feb3df5c640 1 -- 172.21.15.43:0/665638505 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7feb2406b4d0 msgr2=0x7feb2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:56.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.168+0000 7feb3df5c640 1 --2- 172.21.15.43:0/665638505 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7feb2406b4d0 0x7feb2406d990 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7feb28007fd0 tx=0x7feb280079a0 comp rx=0 tx=0).stop 2024-04-25T08:54:56.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.168+0000 7feb3df5c640 1 -- 172.21.15.43:0/665638505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb3815a250 msgr2=0x7feb381ef7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:56.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.168+0000 7feb3df5c640 1 --2- 172.21.15.43:0/665638505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb3815a250 0x7feb381ef7a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7feb3000efc0 tx=0x7feb3000c490 comp rx=0 tx=0).stop 2024-04-25T08:54:56.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.168+0000 7feb3df5c640 1 -- 172.21.15.43:0/665638505 shutdown_connections 2024-04-25T08:54:56.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.168+0000 7feb3df5c640 1 --2- 172.21.15.43:0/665638505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb3815a250 0x7feb381ef7a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:56.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.168+0000 7feb3df5c640 1 --2- 172.21.15.43:0/665638505 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7feb2406b4d0 0x7feb2406d990 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:56.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.168+0000 7feb3df5c640 1 --2- 172.21.15.43:0/665638505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb380b3100 0x7feb381ef260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:56.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.168+0000 7feb3df5c640 1 --2- 172.21.15.43:0/665638505 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feb380715a0 0x7feb381e3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:56.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.168+0000 7feb3df5c640 1 -- 172.21.15.43:0/665638505 >> 172.21.15.43:0/665638505 conn(0x7feb380b1480 msgr2=0x7feb380cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:56.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.168+0000 7feb3df5c640 1 -- 172.21.15.43:0/665638505 shutdown_connections 2024-04-25T08:54:56.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.168+0000 7feb3df5c640 1 -- 172.21.15.43:0/665638505 wait complete. 2024-04-25T08:54:56.169 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:54:56.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.280+0000 7fb352a2c640 1 Processor -- start 2024-04-25T08:54:56.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.280+0000 7fb352a2c640 1 -- start start 2024-04-25T08:54:56.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.281+0000 7fb352a2c640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb34c0cbfc0 0x7fb34c06dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:56.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.281+0000 7fb352a2c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb34c0ccbb0 0x7fb34c0cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:56.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.281+0000 7fb352a2c640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb34c0b2780 con 0x7fb34c15f5e0 2024-04-25T08:54:56.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.281+0000 7fb352a2c640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb34c0cd550 con 0x7fb34c0cbfc0 2024-04-25T08:54:56.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.281+0000 7fb352a2c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb34c0b2a60 con 0x7fb34c0ccbb0 2024-04-25T08:54:56.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.281+0000 7fb350fc9640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb34c15f5e0 0x7fb34c15fa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48476/0 (socket says 172.21.15.43:48476) 2024-04-25T08:54:56.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.281+0000 7fb350fc9640 1 -- 172.21.15.43:0/3266373075 learned_addr learned my addr 172.21.15.43:0/3266373075 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:56.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.281+0000 7fb34b7fe640 1 --2- 172.21.15.43:0/3266373075 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb34c0ccbb0 0x7fb34c0cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:56.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.281+0000 7fb34bfff640 1 --2- 172.21.15.43:0/3266373075 >> v2:172.21.15.193:3300/0 conn(0x7fb34c0cbfc0 0x7fb34c06dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:56.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.281+0000 7fb34affd640 1 -- 172.21.15.43:0/3266373075 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3397694690 0 0) 0x7fb34c0b2780 con 0x7fb34c15f5e0 2024-04-25T08:54:56.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.281+0000 7fb34affd640 1 -- 172.21.15.43:0/3266373075 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb334003520 con 0x7fb34c15f5e0 2024-04-25T08:54:56.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.282+0000 7fb34b7fe640 1 -- 172.21.15.43:0/3266373075 >> v2:172.21.15.193:3300/0 conn(0x7fb34c0cbfc0 msgr2=0x7fb34c06dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:56.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.282+0000 7fb34b7fe640 1 --2- 172.21.15.43:0/3266373075 >> v2:172.21.15.193:3300/0 conn(0x7fb34c0cbfc0 0x7fb34c06dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:56.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.282+0000 7fb34affd640 1 -- 172.21.15.43:0/3266373075 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2502431041 0 0) 0x7fb334003520 con 0x7fb34c15f5e0 2024-04-25T08:54:56.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.282+0000 7fb34b7fe640 1 -- 172.21.15.43:0/3266373075 >> v1:172.21.15.43:6789/0 conn(0x7fb34c15f5e0 legacy=0x7fb34c15fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:56.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.282+0000 7fb34b7fe640 1 -- 172.21.15.43:0/3266373075 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb34c0b2d40 con 0x7fb34c0ccbb0 2024-04-25T08:54:56.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.282+0000 7fb34bfff640 1 --2- 172.21.15.43:0/3266373075 >> v2:172.21.15.193:3300/0 conn(0x7fb34c0cbfc0 0x7fb34c06dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:54:56.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.282+0000 7fb34b7fe640 1 --2- 172.21.15.43:0/3266373075 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb34c0ccbb0 0x7fb34c0cd010 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb33c0097a0 tx=0x7fb33c02a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf7518bfc3863914 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:56.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.283+0000 7fb34affd640 1 -- 172.21.15.43:0/3266373075 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c037070 con 0x7fb34c0ccbb0 2024-04-25T08:54:56.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.283+0000 7fb34affd640 1 -- 172.21.15.43:0/3266373075 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb33c02aba0 con 0x7fb34c0ccbb0 2024-04-25T08:54:56.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.283+0000 7fb34affd640 1 -- 172.21.15.43:0/3266373075 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3440019f0 con 0x7fb34c0ccbb0 2024-04-25T08:54:56.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.283+0000 7fb352a2c640 1 -- 172.21.15.43:0/3266373075 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb34c0ccbb0 msgr2=0x7fb34c0cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:56.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.283+0000 7fb352a2c640 1 --2- 172.21.15.43:0/3266373075 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb34c0ccbb0 0x7fb34c0cd010 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb33c0097a0 tx=0x7fb33c02a110 comp rx=0 tx=0).stop 2024-04-25T08:54:56.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.283+0000 7fb352a2c640 1 -- 172.21.15.43:0/3266373075 shutdown_connections 2024-04-25T08:54:56.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.283+0000 7fb352a2c640 1 --2- 172.21.15.43:0/3266373075 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb34c0ccbb0 0x7fb34c0cd010 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:56.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.283+0000 7fb352a2c640 1 --2- 172.21.15.43:0/3266373075 >> v2:172.21.15.193:3300/0 conn(0x7fb34c0cbfc0 0x7fb34c06dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:56.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.283+0000 7fb352a2c640 1 -- 172.21.15.43:0/3266373075 >> 172.21.15.43:0/3266373075 conn(0x7fb34c0b1480 msgr2=0x7fb34c0b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:56.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.283+0000 7fb352a2c640 1 -- 172.21.15.43:0/3266373075 shutdown_connections 2024-04-25T08:54:56.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.283+0000 7fb352a2c640 1 -- 172.21.15.43:0/3266373075 wait complete. 2024-04-25T08:54:56.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.284+0000 7fb352a2c640 1 Processor -- start 2024-04-25T08:54:56.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.284+0000 7fb352a2c640 1 -- start start 2024-04-25T08:54:56.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.285+0000 7fb352a2c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb34c0cbfc0 0x7fb34c1f2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:56.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.285+0000 7fb352a2c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb34c0ccbb0 0x7fb34c140270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:56.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.285+0000 7fb352a2c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb34c15f5e0 0x7fb34c1e5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:56.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.285+0000 7fb352a2c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34c0b35e0 con 0x7fb34c0cbfc0 2024-04-25T08:54:56.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.285+0000 7fb352a2c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34c0b3300 con 0x7fb34c0ccbb0 2024-04-25T08:54:56.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.285+0000 7fb352a2c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb34c06e1a0 con 0x7fb34c15f5e0 2024-04-25T08:54:56.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.285+0000 7fb34bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb34c0cbfc0 0x7fb34c1f2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:56.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.285+0000 7fb34bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb34c0cbfc0 0x7fb34c1f2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49860/0 (socket says 172.21.15.43:49860) 2024-04-25T08:54:56.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.285+0000 7fb34bfff640 1 -- 172.21.15.43:0/3028975437 learned_addr learned my addr 172.21.15.43:0/3028975437 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:56.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.286+0000 7fb350fc9640 1 --2- 172.21.15.43:0/3028975437 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb34c15f5e0 0x7fb34c1e5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:56.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.286+0000 7fb34b7fe640 1 --2- 172.21.15.43:0/3028975437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb34c0ccbb0 0x7fb34c140270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:56.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.286+0000 7fb34bfff640 1 -- 172.21.15.43:0/3028975437 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb34c15f5e0 msgr2=0x7fb34c1e5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:56.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.286+0000 7fb34bfff640 1 --2- 172.21.15.43:0/3028975437 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb34c15f5e0 0x7fb34c1e5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:56.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.286+0000 7fb34bfff640 1 -- 172.21.15.43:0/3028975437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb34c0ccbb0 msgr2=0x7fb34c140270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:56.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.286+0000 7fb34bfff640 1 --2- 172.21.15.43:0/3028975437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb34c0ccbb0 0x7fb34c140270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:56.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.286+0000 7fb34bfff640 1 -- 172.21.15.43:0/3028975437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb34c1e4bb0 con 0x7fb34c0cbfc0 2024-04-25T08:54:56.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.286+0000 7fb350fc9640 1 --2- 172.21.15.43:0/3028975437 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb34c15f5e0 0x7fb34c1e5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:56.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.286+0000 7fb34b7fe640 1 --2- 172.21.15.43:0/3028975437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb34c0ccbb0 0x7fb34c140270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:54:56.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.286+0000 7fb34bfff640 1 --2- 172.21.15.43:0/3028975437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb34c0cbfc0 0x7fb34c1f2740 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb34000ebe0 tx=0x7fb34000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:56.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.286+0000 7fb348ff9640 1 -- 172.21.15.43:0/3028975437 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb340017070 con 0x7fb34c0cbfc0 2024-04-25T08:54:56.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.286+0000 7fb352a2c640 1 -- 172.21.15.43:0/3028975437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb34c1e4e90 con 0x7fb34c0cbfc0 2024-04-25T08:54:56.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.286+0000 7fb348ff9640 1 -- 172.21.15.43:0/3028975437 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb34000cc30 con 0x7fb34c0cbfc0 2024-04-25T08:54:56.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.286+0000 7fb348ff9640 1 -- 172.21.15.43:0/3028975437 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb340013470 con 0x7fb34c0cbfc0 2024-04-25T08:54:56.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.286+0000 7fb352a2c640 1 -- 172.21.15.43:0/3028975437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb34c1e5450 con 0x7fb34c0cbfc0 2024-04-25T08:54:56.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.287+0000 7fb352a2c640 1 -- 172.21.15.43:0/3028975437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb34c074ef0 con 0x7fb34c0cbfc0 2024-04-25T08:54:56.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.288+0000 7fb348ff9640 1 -- 172.21.15.43:0/3028975437 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb34001a070 con 0x7fb34c0cbfc0 2024-04-25T08:54:56.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.288+0000 7fb348ff9640 1 --2- 172.21.15.43:0/3028975437 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb33806b4d0 0x7fb33806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:56.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.288+0000 7fb348ff9640 1 -- 172.21.15.43:0/3028975437 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fb34008a5b0 con 0x7fb34c0cbfc0 2024-04-25T08:54:56.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.288+0000 7fb34b7fe640 1 --2- 172.21.15.43:0/3028975437 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb33806b4d0 0x7fb33806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:56.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.289+0000 7fb34b7fe640 1 --2- 172.21.15.43:0/3028975437 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb33806b4d0 0x7fb33806d990 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb33c034000 tx=0x7fb33c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:56.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.292+0000 7fb348ff9640 1 -- 172.21.15.43:0/3028975437 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb3400593b0 con 0x7fb34c0cbfc0 2024-04-25T08:54:56.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.510+0000 7fb352a2c640 1 -- 172.21.15.43:0/3028975437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb34c0af0d0 con 0x7fb34c0cbfc0 2024-04-25T08:54:56.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.510+0000 7fb348ff9640 1 -- 172.21.15.43:0/3028975437 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4938 (secure 0 0 0) 0x7fb340058cb0 con 0x7fb34c0cbfc0 2024-04-25T08:54:56.500 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:54:56.500 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3616994003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":3616994003},{"type":"v1","addr":"172.21.15.43:6835","nonce":3616994003}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4253,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/4022901908","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":4022901908},{"type":"v1","addr":"172.21.15.193:6835","nonce":4022901908}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2823450590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2823450590},{"type":"v1","addr":"172.21.15.43:6834","nonce":2823450590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:54:11.945963+0000","modified":"2024-04-25T08:54:13.943002+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4294},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4294":{"gid":4294,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.193:6837/1197434598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":1197434598},{"type":"v1","addr":"172.21.15.193:6837","nonce":1197434598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T08:54:56.501 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 6 2024-04-25T08:54:56.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.512+0000 7fb352a2c640 1 -- 172.21.15.43:0/3028975437 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb33806b4d0 msgr2=0x7fb33806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:56.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.512+0000 7fb352a2c640 1 --2- 172.21.15.43:0/3028975437 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb33806b4d0 0x7fb33806d990 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb33c034000 tx=0x7fb33c02e000 comp rx=0 tx=0).stop 2024-04-25T08:54:56.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.512+0000 7fb352a2c640 1 -- 172.21.15.43:0/3028975437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb34c0cbfc0 msgr2=0x7fb34c1f2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:56.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.512+0000 7fb352a2c640 1 --2- 172.21.15.43:0/3028975437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb34c0cbfc0 0x7fb34c1f2740 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fb34000ebe0 tx=0x7fb34000c520 comp rx=0 tx=0).stop 2024-04-25T08:54:56.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.512+0000 7fb352a2c640 1 -- 172.21.15.43:0/3028975437 shutdown_connections 2024-04-25T08:54:56.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.512+0000 7fb352a2c640 1 --2- 172.21.15.43:0/3028975437 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb34c15f5e0 0x7fb34c1e5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:56.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.512+0000 7fb352a2c640 1 --2- 172.21.15.43:0/3028975437 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb33806b4d0 0x7fb33806d990 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:56.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.512+0000 7fb352a2c640 1 --2- 172.21.15.43:0/3028975437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb34c0ccbb0 0x7fb34c140270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:56.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.512+0000 7fb352a2c640 1 --2- 172.21.15.43:0/3028975437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb34c0cbfc0 0x7fb34c1f2740 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:56.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.512+0000 7fb352a2c640 1 -- 172.21.15.43:0/3028975437 >> 172.21.15.43:0/3028975437 conn(0x7fb34c0b1480 msgr2=0x7fb34c06d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:56.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.512+0000 7fb352a2c640 1 -- 172.21.15.43:0/3028975437 shutdown_connections 2024-04-25T08:54:56.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.512+0000 7fb352a2c640 1 -- 172.21.15.43:0/3028975437 wait complete. 2024-04-25T08:54:56.513 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:54:56.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.624+0000 7f6da381f640 1 Processor -- start 2024-04-25T08:54:56.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.624+0000 7f6da381f640 1 -- start start 2024-04-25T08:54:56.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.625+0000 7f6da381f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6d9c15df50 0x7f6d9c1619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:56.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.625+0000 7f6da381f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d9c0d24a0 con 0x7f6d9c0cf670 2024-04-25T08:54:56.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.625+0000 7f6da381f640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d9c0d2740 con 0x7f6d9c161f30 2024-04-25T08:54:56.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.625+0000 7f6da381f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d9c0cfa80 con 0x7f6d9c15df50 2024-04-25T08:54:56.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.625+0000 7f6da15bb640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6d9c15df50 0x7f6d9c1619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:56.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.625+0000 7f6da15bb640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6d9c15df50 0x7f6d9c1619f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49862/0 (socket says 172.21.15.43:49862) 2024-04-25T08:54:56.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.625+0000 7f6da1dbc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f6d9c0cf670 0x7f6d9c0c8400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48492/0 (socket says 172.21.15.43:48492) 2024-04-25T08:54:56.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.625+0000 7f6da15bb640 1 -- 172.21.15.43:0/3648413252 learned_addr learned my addr 172.21.15.43:0/3648413252 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:56.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.625+0000 7f6da15bb640 1 -- 172.21.15.43:0/3648413252 >> v1:172.21.15.193:6789/0 conn(0x7f6d9c161f30 legacy=0x7f6d9c0c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:56.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.625+0000 7f6da15bb640 1 -- 172.21.15.43:0/3648413252 >> v1:172.21.15.43:6789/0 conn(0x7f6d9c0cf670 legacy=0x7f6d9c0c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:56.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.625+0000 7f6da15bb640 1 -- 172.21.15.43:0/3648413252 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d9c0d2a20 con 0x7f6d9c15df50 2024-04-25T08:54:56.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.625+0000 7f6d93fff640 1 -- 172.21.15.43:0/3648413252 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1628915155 0 0) 0x7f6d9c0d24a0 con 0x7f6d9c0cf670 2024-04-25T08:54:56.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.626+0000 7f6da15bb640 1 --2- 172.21.15.43:0/3648413252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d9c15df50 0x7f6d9c1619f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6d98009870 tx=0x7f6d9802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f0780becf0e5c49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:56.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.626+0000 7f6d93fff640 1 -- 172.21.15.43:0/3648413252 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d98037070 con 0x7f6d9c15df50 2024-04-25T08:54:56.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.626+0000 7f6d93fff640 1 -- 172.21.15.43:0/3648413252 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d98032440 con 0x7f6d9c15df50 2024-04-25T08:54:56.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.626+0000 7f6d93fff640 1 -- 172.21.15.43:0/3648413252 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d98032810 con 0x7f6d9c15df50 2024-04-25T08:54:56.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.627+0000 7f6da381f640 1 -- 172.21.15.43:0/3648413252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d9c15df50 msgr2=0x7f6d9c1619f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:56.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.627+0000 7f6da381f640 1 --2- 172.21.15.43:0/3648413252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d9c15df50 0x7f6d9c1619f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6d98009870 tx=0x7f6d9802a2a0 comp rx=0 tx=0).stop 2024-04-25T08:54:56.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.627+0000 7f6da381f640 1 -- 172.21.15.43:0/3648413252 shutdown_connections 2024-04-25T08:54:56.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.627+0000 7f6da381f640 1 --2- 172.21.15.43:0/3648413252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d9c15df50 0x7f6d9c1619f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:56.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.627+0000 7f6da381f640 1 -- 172.21.15.43:0/3648413252 >> 172.21.15.43:0/3648413252 conn(0x7f6d9c0d3a30 msgr2=0x7f6d9c0d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:56.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.627+0000 7f6da381f640 1 -- 172.21.15.43:0/3648413252 shutdown_connections 2024-04-25T08:54:56.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.627+0000 7f6da381f640 1 -- 172.21.15.43:0/3648413252 wait complete. 2024-04-25T08:54:56.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.628+0000 7f6da381f640 1 Processor -- start 2024-04-25T08:54:56.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.628+0000 7f6da381f640 1 -- start start 2024-04-25T08:54:56.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.629+0000 7f6da381f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d9c0cf670 0x7f6d9c1608f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:56.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.629+0000 7f6da381f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6d9c15df50 0x7f6d9c09d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:56.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.629+0000 7f6da381f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d9c161f30 0x7f6d9c1f3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:56.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.629+0000 7f6da381f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9c0c77d0 con 0x7f6d9c161f30 2024-04-25T08:54:56.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.629+0000 7f6da381f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9c15e330 con 0x7f6d9c0cf670 2024-04-25T08:54:56.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.629+0000 7f6da381f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d9c0d2190 con 0x7f6d9c15df50 2024-04-25T08:54:56.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.629+0000 7f6da1dbc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d9c161f30 0x7f6d9c1f3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:56.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.629+0000 7f6da1dbc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d9c161f30 0x7f6d9c1f3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49870/0 (socket says 172.21.15.43:49870) 2024-04-25T08:54:56.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.629+0000 7f6da1dbc640 1 -- 172.21.15.43:0/3245434030 learned_addr learned my addr 172.21.15.43:0/3245434030 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:56.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.629+0000 7f6da15bb640 1 --2- 172.21.15.43:0/3245434030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d9c0cf670 0x7f6d9c1608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:56.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.629+0000 7f6da0dba640 1 --2- 172.21.15.43:0/3245434030 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6d9c15df50 0x7f6d9c09d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:56.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.629+0000 7f6da1dbc640 1 -- 172.21.15.43:0/3245434030 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6d9c15df50 msgr2=0x7f6d9c09d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:56.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.629+0000 7f6da1dbc640 1 --2- 172.21.15.43:0/3245434030 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6d9c15df50 0x7f6d9c09d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:56.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.629+0000 7f6da1dbc640 1 -- 172.21.15.43:0/3245434030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d9c0cf670 msgr2=0x7f6d9c1608f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:56.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.629+0000 7f6da1dbc640 1 --2- 172.21.15.43:0/3245434030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d9c0cf670 0x7f6d9c1608f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:56.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.629+0000 7f6da1dbc640 1 -- 172.21.15.43:0/3245434030 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d9c09e230 con 0x7f6d9c161f30 2024-04-25T08:54:56.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.629+0000 7f6da1dbc640 1 --2- 172.21.15.43:0/3245434030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d9c161f30 0x7f6d9c1f3380 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6d9400c760 tx=0x7f6d9400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:56.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.629+0000 7f6d91ffb640 1 -- 172.21.15.43:0/3245434030 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9400a640 con 0x7f6d9c161f30 2024-04-25T08:54:56.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.629+0000 7f6d91ffb640 1 -- 172.21.15.43:0/3245434030 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d9400a800 con 0x7f6d9c161f30 2024-04-25T08:54:56.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.629+0000 7f6d91ffb640 1 -- 172.21.15.43:0/3245434030 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9400abf0 con 0x7f6d9c161f30 2024-04-25T08:54:56.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.629+0000 7f6da381f640 1 -- 172.21.15.43:0/3245434030 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d9c09e510 con 0x7f6d9c161f30 2024-04-25T08:54:56.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.630+0000 7f6da381f640 1 -- 172.21.15.43:0/3245434030 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d9c149c20 con 0x7f6d9c161f30 2024-04-25T08:54:56.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.631+0000 7f6da381f640 1 -- 172.21.15.43:0/3245434030 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d9c0680f0 con 0x7f6d9c161f30 2024-04-25T08:54:56.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.631+0000 7f6d91ffb640 1 -- 172.21.15.43:0/3245434030 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6d94007070 con 0x7f6d9c161f30 2024-04-25T08:54:56.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.631+0000 7f6d91ffb640 1 --2- 172.21.15.43:0/3245434030 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6d8006b4d0 0x7f6d8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:56.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.631+0000 7f6d91ffb640 1 -- 172.21.15.43:0/3245434030 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f6d9408a0d0 con 0x7f6d9c161f30 2024-04-25T08:54:56.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.632+0000 7f6da15bb640 1 --2- 172.21.15.43:0/3245434030 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6d8006b4d0 0x7f6d8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:56.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.632+0000 7f6da15bb640 1 --2- 172.21.15.43:0/3245434030 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6d8006b4d0 0x7f6d8006d990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6d98034000 tx=0x7f6d9802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:56.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.636+0000 7f6d91ffb640 1 -- 172.21.15.43:0/3245434030 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6d94058870 con 0x7f6d9c161f30 2024-04-25T08:54:56.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.815+0000 7f6da381f640 1 -- 172.21.15.43:0/3245434030 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6d9c074ef0 con 0x7f6d9c161f30 2024-04-25T08:54:56.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.816+0000 7f6d91ffb640 1 -- 172.21.15.43:0/3245434030 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16809 (secure 0 0 0) 0x7f6d9405c910 con 0x7f6d9c161f30 2024-04-25T08:54:56.807 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:54:56.808 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":22,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:54:52.847242+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:54:09.757978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T08:54:10.926008+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":5,"to":2}]},{"pgid":"3.18","mappings":[{"from":3,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2d","mappings":[{"from":7,"to":1}]},{"pgid":"3.35","mappings":[{"from":5,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:54:56.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.818+0000 7f6da381f640 1 -- 172.21.15.43:0/3245434030 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6d8006b4d0 msgr2=0x7f6d8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:56.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.818+0000 7f6da381f640 1 --2- 172.21.15.43:0/3245434030 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6d8006b4d0 0x7f6d8006d990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6d98034000 tx=0x7f6d9802e000 comp rx=0 tx=0).stop 2024-04-25T08:54:56.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.818+0000 7f6da381f640 1 -- 172.21.15.43:0/3245434030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d9c161f30 msgr2=0x7f6d9c1f3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:56.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.818+0000 7f6da381f640 1 --2- 172.21.15.43:0/3245434030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d9c161f30 0x7f6d9c1f3380 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6d9400c760 tx=0x7f6d9400cc20 comp rx=0 tx=0).stop 2024-04-25T08:54:56.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.819+0000 7f6da381f640 1 -- 172.21.15.43:0/3245434030 shutdown_connections 2024-04-25T08:54:56.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.819+0000 7f6da381f640 1 --2- 172.21.15.43:0/3245434030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d9c161f30 0x7f6d9c1f3380 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:56.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.819+0000 7f6da381f640 1 --2- 172.21.15.43:0/3245434030 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6d9c15df50 0x7f6d9c09d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:56.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.819+0000 7f6da381f640 1 --2- 172.21.15.43:0/3245434030 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6d8006b4d0 0x7f6d8006d990 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:56.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.819+0000 7f6da381f640 1 --2- 172.21.15.43:0/3245434030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d9c0cf670 0x7f6d9c1608f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:56.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.819+0000 7f6da381f640 1 -- 172.21.15.43:0/3245434030 >> 172.21.15.43:0/3245434030 conn(0x7f6d9c0d3a30 msgr2=0x7f6d9c154080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:56.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.819+0000 7f6da381f640 1 -- 172.21.15.43:0/3245434030 shutdown_connections 2024-04-25T08:54:56.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.819+0000 7f6da381f640 1 -- 172.21.15.43:0/3245434030 wait complete. 2024-04-25T08:54:56.820 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:54:56.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.934+0000 7f0155189640 1 Processor -- start 2024-04-25T08:54:56.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.934+0000 7f0155189640 1 -- start start 2024-04-25T08:54:56.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.934+0000 7f0155189640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f015015df50 0x7f01501619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:56.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.934+0000 7f0155189640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01500cf670 0x7f0150162380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:56.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.934+0000 7f0155189640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01500d24a0 con 0x7f0150161f30 2024-04-25T08:54:56.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.934+0000 7f0155189640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f015015e330 con 0x7f015015df50 2024-04-25T08:54:56.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.934+0000 7f0155189640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01500d2740 con 0x7f01500cf670 2024-04-25T08:54:56.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.935+0000 7f014ed9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f015015df50 0x7f01501619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:56.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.935+0000 7f014ed9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f015015df50 0x7f01501619f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49886/0 (socket says 172.21.15.43:49886) 2024-04-25T08:54:56.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.935+0000 7f014ed9d640 1 -- 172.21.15.43:0/3108623354 learned_addr learned my addr 172.21.15.43:0/3108623354 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:56.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.935+0000 7f014f59e640 1 --2- 172.21.15.43:0/3108623354 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01500cf670 0x7f0150162380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:56.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.935+0000 7f014ed9d640 1 -- 172.21.15.43:0/3108623354 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01500cf670 msgr2=0x7f0150162380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:56.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.935+0000 7f014ed9d640 1 --2- 172.21.15.43:0/3108623354 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01500cf670 0x7f0150162380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:56.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.935+0000 7f014ed9d640 1 -- 172.21.15.43:0/3108623354 >> v1:172.21.15.193:6789/0 conn(0x7f0150161f30 legacy=0x7f01500c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:56.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.935+0000 7f014ed9d640 1 -- 172.21.15.43:0/3108623354 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01500d2a20 con 0x7f015015df50 2024-04-25T08:54:56.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.936+0000 7f014ed9d640 1 --2- 172.21.15.43:0/3108623354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f015015df50 0x7f01501619f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0148009870 tx=0x7f014802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1923b0bf87c3714b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:56.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.936+0000 7f014dd9b640 1 -- 172.21.15.43:0/3108623354 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0148037070 con 0x7f015015df50 2024-04-25T08:54:56.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.936+0000 7f014dd9b640 1 -- 172.21.15.43:0/3108623354 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0148032440 con 0x7f015015df50 2024-04-25T08:54:56.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.936+0000 7f014dd9b640 1 -- 172.21.15.43:0/3108623354 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0148032810 con 0x7f015015df50 2024-04-25T08:54:56.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.936+0000 7f0155189640 1 -- 172.21.15.43:0/3108623354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f015015df50 msgr2=0x7f01501619f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:56.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.936+0000 7f0155189640 1 --2- 172.21.15.43:0/3108623354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f015015df50 0x7f01501619f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0148009870 tx=0x7f014802a270 comp rx=0 tx=0).stop 2024-04-25T08:54:56.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.936+0000 7f0155189640 1 -- 172.21.15.43:0/3108623354 shutdown_connections 2024-04-25T08:54:56.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.936+0000 7f0155189640 1 --2- 172.21.15.43:0/3108623354 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01500cf670 0x7f0150162380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:56.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.936+0000 7f0155189640 1 --2- 172.21.15.43:0/3108623354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f015015df50 0x7f01501619f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:56.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.936+0000 7f0155189640 1 -- 172.21.15.43:0/3108623354 >> 172.21.15.43:0/3108623354 conn(0x7f01500d3a30 msgr2=0x7f01500d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:56.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.936+0000 7f0155189640 1 -- 172.21.15.43:0/3108623354 shutdown_connections 2024-04-25T08:54:56.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.936+0000 7f0155189640 1 -- 172.21.15.43:0/3108623354 wait complete. 2024-04-25T08:54:56.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.938+0000 7f0155189640 1 Processor -- start 2024-04-25T08:54:56.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.938+0000 7f0155189640 1 -- start start 2024-04-25T08:54:56.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.938+0000 7f0155189640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01500cf670 0x7f01500a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:56.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.938+0000 7f0155189640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f015015df50 0x7f015006c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:56.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.938+0000 7f0155189640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0150161f30 0x7f015009e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:56.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.939+0000 7f0155189640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01500d2190 con 0x7f01500cf670 2024-04-25T08:54:56.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.939+0000 7f0155189640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01500c77d0 con 0x7f015015df50 2024-04-25T08:54:56.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.939+0000 7f0155189640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01500c8400 con 0x7f0150161f30 2024-04-25T08:54:56.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.939+0000 7f014ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01500cf670 0x7f01500a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:56.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.939+0000 7f014ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01500cf670 0x7f01500a1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49894/0 (socket says 172.21.15.43:49894) 2024-04-25T08:54:56.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.939+0000 7f014ed9d640 1 -- 172.21.15.43:0/3635555771 learned_addr learned my addr 172.21.15.43:0/3635555771 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:56.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.939+0000 7f014f59e640 1 --2- 172.21.15.43:0/3635555771 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0150161f30 0x7f015009e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:56.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.939+0000 7f014ed9d640 1 -- 172.21.15.43:0/3635555771 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0150161f30 msgr2=0x7f015009e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:56.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.939+0000 7f014ed9d640 1 --2- 172.21.15.43:0/3635555771 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0150161f30 0x7f015009e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:56.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.939+0000 7f014e59c640 1 --2- 172.21.15.43:0/3635555771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f015015df50 0x7f015006c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:56.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.939+0000 7f014ed9d640 1 -- 172.21.15.43:0/3635555771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f015015df50 msgr2=0x7f015006c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:56.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.939+0000 7f014ed9d640 1 --2- 172.21.15.43:0/3635555771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f015015df50 0x7f015006c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:56.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.939+0000 7f014ed9d640 1 -- 172.21.15.43:0/3635555771 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01501f7860 con 0x7f01500cf670 2024-04-25T08:54:56.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.939+0000 7f014ed9d640 1 --2- 172.21.15.43:0/3635555771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01500cf670 0x7f01500a1130 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0148000c00 tx=0x7f01480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:56.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.939+0000 7f01477fe640 1 -- 172.21.15.43:0/3635555771 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0148037040 con 0x7f01500cf670 2024-04-25T08:54:56.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.939+0000 7f01477fe640 1 -- 172.21.15.43:0/3635555771 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0148042070 con 0x7f01500cf670 2024-04-25T08:54:56.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.939+0000 7f0155189640 1 -- 172.21.15.43:0/3635555771 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f015009dce0 con 0x7f01500cf670 2024-04-25T08:54:56.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.939+0000 7f01477fe640 1 -- 172.21.15.43:0/3635555771 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0148030880 con 0x7f01500cf670 2024-04-25T08:54:56.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.939+0000 7f0155189640 1 -- 172.21.15.43:0/3635555771 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f015009e580 con 0x7f01500cf670 2024-04-25T08:54:56.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.940+0000 7f0155189640 1 -- 172.21.15.43:0/3635555771 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f015009e0d0 con 0x7f01500cf670 2024-04-25T08:54:56.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.941+0000 7f01477fe640 1 -- 172.21.15.43:0/3635555771 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0148030a20 con 0x7f01500cf670 2024-04-25T08:54:56.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.941+0000 7f01477fe640 1 --2- 172.21.15.43:0/3635555771 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f013006b4d0 0x7f013006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:56.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.941+0000 7f01477fe640 1 -- 172.21.15.43:0/3635555771 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f01480aaa40 con 0x7f01500cf670 2024-04-25T08:54:56.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.942+0000 7f014e59c640 1 --2- 172.21.15.43:0/3635555771 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f013006b4d0 0x7f013006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:56.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.942+0000 7f014e59c640 1 --2- 172.21.15.43:0/3635555771 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f013006b4d0 0x7f013006d990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f015006ba70 tx=0x7f0138009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:56.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:56.945+0000 7f01477fe640 1 -- 172.21.15.43:0/3635555771 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f01480798c0 con 0x7f01500cf670 2024-04-25T08:54:57.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.164+0000 7f0155189640 1 -- 172.21.15.43:0/3635555771 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f01500680f0 con 0x7f01500cf670 2024-04-25T08:54:57.154 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-04-25T08:54:57.164+0000 7fe4fb198640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:54:57.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.166+0000 7f01477fe640 1 -- 172.21.15.43:0/3635555771 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v7) v1 ==== 161+0+0 (secure 0 0 0) 0x7f01480791c0 con 0x7f01500cf670 2024-04-25T08:54:57.156 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:54:57.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.168+0000 7f0155189640 1 -- 172.21.15.43:0/3635555771 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f013006b4d0 msgr2=0x7f013006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:57.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.168+0000 7f0155189640 1 --2- 172.21.15.43:0/3635555771 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f013006b4d0 0x7f013006d990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f015006ba70 tx=0x7f0138009040 comp rx=0 tx=0).stop 2024-04-25T08:54:57.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.168+0000 7f0155189640 1 -- 172.21.15.43:0/3635555771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01500cf670 msgr2=0x7f01500a1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:57.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.168+0000 7f0155189640 1 --2- 172.21.15.43:0/3635555771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01500cf670 0x7f01500a1130 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0148000c00 tx=0x7f01480046c0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.168+0000 7f0155189640 1 -- 172.21.15.43:0/3635555771 shutdown_connections 2024-04-25T08:54:57.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.169+0000 7f0155189640 1 --2- 172.21.15.43:0/3635555771 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0150161f30 0x7f015009e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.169+0000 7f0155189640 1 --2- 172.21.15.43:0/3635555771 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f013006b4d0 0x7f013006d990 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.169+0000 7f0155189640 1 --2- 172.21.15.43:0/3635555771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f015015df50 0x7f015006c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.169+0000 7f0155189640 1 --2- 172.21.15.43:0/3635555771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01500cf670 0x7f01500a1130 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.169+0000 7f0155189640 1 -- 172.21.15.43:0/3635555771 >> 172.21.15.43:0/3635555771 conn(0x7f01500d3a30 msgr2=0x7f0150152f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:57.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.169+0000 7f0155189640 1 -- 172.21.15.43:0/3635555771 shutdown_connections 2024-04-25T08:54:57.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.169+0000 7f0155189640 1 -- 172.21.15.43:0/3635555771 wait complete. 2024-04-25T08:54:57.161 INFO:tasks.ceph.mds.d.smithi193.stderr: -2908> 2024-04-25T08:54:16.258+0000 7ff1f5c30640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:54:57.170 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T08:54:57.191 INFO:tasks.ceph.mds.d.smithi193.stdout:starting mds.d at 2024-04-25T08:54:57.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.283+0000 7f971a274640 1 Processor -- start 2024-04-25T08:54:57.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.283+0000 7f971a274640 1 -- start start 2024-04-25T08:54:57.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.284+0000 7f971a274640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97140b6110 0x7f97140d1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.284+0000 7f971a274640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f97140d2340 0x7f97141656d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.284+0000 7f971a274640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97140b32a0 con 0x7f97140d1a20 2024-04-25T08:54:57.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.284+0000 7f971a274640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97140cf430 con 0x7f97140d2340 2024-04-25T08:54:57.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.284+0000 7f971a274640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f971415bf90 con 0x7f97140b6110 2024-04-25T08:54:57.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.284+0000 7f9713fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f97140d1a20 0x7f97140b2b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48500/0 (socket says 172.21.15.43:48500) 2024-04-25T08:54:57.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.284+0000 7f9713fff640 1 -- 172.21.15.43:0/4263996121 learned_addr learned my addr 172.21.15.43:0/4263996121 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:57.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.284+0000 7f9712ffd640 1 -- 172.21.15.43:0/4263996121 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3341033236 0 0) 0x7f97140b32a0 con 0x7f97140d1a20 2024-04-25T08:54:57.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.284+0000 7f9718811640 1 --2- 172.21.15.43:0/4263996121 >> v2:172.21.15.193:3300/0 conn(0x7f97140d2340 0x7f97141656d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.284+0000 7f9712ffd640 1 -- 172.21.15.43:0/4263996121 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9700003520 con 0x7f97140d1a20 2024-04-25T08:54:57.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.284+0000 7f97137fe640 1 --2- 172.21.15.43:0/4263996121 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97140b6110 0x7f97140d1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.285+0000 7f9712ffd640 1 -- 172.21.15.43:0/4263996121 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2651848446 0 0) 0x7f9700003520 con 0x7f97140d1a20 2024-04-25T08:54:57.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.285+0000 7f9712ffd640 1 -- 172.21.15.43:0/4263996121 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97140b6110 msgr2=0x7f97140d1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:57.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.285+0000 7f9712ffd640 1 --2- 172.21.15.43:0/4263996121 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97140b6110 0x7f97140d1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.285+0000 7f9712ffd640 1 -- 172.21.15.43:0/4263996121 >> v2:172.21.15.193:3300/0 conn(0x7f97140d2340 msgr2=0x7f97141656d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:57.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.285+0000 7f9712ffd640 1 --2- 172.21.15.43:0/4263996121 >> v2:172.21.15.193:3300/0 conn(0x7f97140d2340 0x7f97141656d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.285+0000 7f9712ffd640 1 -- 172.21.15.43:0/4263996121 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97140cee70 con 0x7f97140d1a20 2024-04-25T08:54:57.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.285+0000 7f9712ffd640 1 -- 172.21.15.43:0/4263996121 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f96fc001690 con 0x7f97140d1a20 2024-04-25T08:54:57.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.285+0000 7f9712ffd640 1 -- 172.21.15.43:0/4263996121 >> v1:172.21.15.43:6789/0 conn(0x7f97140d1a20 legacy=0x7f97140b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:57.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.285+0000 7f9712ffd640 1 --2- 172.21.15.43:0/4263996121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97000039b0 0x7f9700003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.285+0000 7f9712ffd640 1 --2- 172.21.15.43:0/4263996121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9700004800 0x7f9700004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.285+0000 7f9712ffd640 1 --2- 172.21.15.43:0/4263996121 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97140b6110 0x7f97000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.285+0000 7f9712ffd640 1 -- 172.21.15.43:0/4263996121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97140cf430 con 0x7f9700004800 2024-04-25T08:54:57.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.285+0000 7f9712ffd640 1 -- 172.21.15.43:0/4263996121 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971415bf90 con 0x7f97000039b0 2024-04-25T08:54:57.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.285+0000 7f9712ffd640 1 -- 172.21.15.43:0/4263996121 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f970000ba40 con 0x7f97140b6110 2024-04-25T08:54:57.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.285+0000 7f97137fe640 1 --2- 172.21.15.43:0/4263996121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9700004800 0x7f9700004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.285+0000 7f9713fff640 1 --2- 172.21.15.43:0/4263996121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97000039b0 0x7f9700003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.285+0000 7f9718811640 1 --2- 172.21.15.43:0/4263996121 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97140b6110 0x7f97000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.285+0000 7f97137fe640 1 -- 172.21.15.43:0/4263996121 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97140b6110 msgr2=0x7f97000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:57.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.285+0000 7f97137fe640 1 --2- 172.21.15.43:0/4263996121 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97140b6110 0x7f97000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.285+0000 7f97137fe640 1 -- 172.21.15.43:0/4263996121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97000039b0 msgr2=0x7f9700003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:57.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.285+0000 7f97137fe640 1 --2- 172.21.15.43:0/4263996121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97000039b0 0x7f9700003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.285+0000 7f97137fe640 1 -- 172.21.15.43:0/4263996121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9700003520 con 0x7f9700004800 2024-04-25T08:54:57.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.286+0000 7f97137fe640 1 --2- 172.21.15.43:0/4263996121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9700004800 0x7f9700004c80 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9704009ac0 tx=0x7f9700005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e079577c7adb763b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:57.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.286+0000 7f9712ffd640 1 -- 172.21.15.43:0/4263996121 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9704036070 con 0x7f9700004800 2024-04-25T08:54:57.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.286+0000 7f9712ffd640 1 -- 172.21.15.43:0/4263996121 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9704031400 con 0x7f9700004800 2024-04-25T08:54:57.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.286+0000 7f9712ffd640 1 -- 172.21.15.43:0/4263996121 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9704009690 con 0x7f9700004800 2024-04-25T08:54:57.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.286+0000 7f971a274640 1 -- 172.21.15.43:0/4263996121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9700004800 msgr2=0x7f9700004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:57.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.286+0000 7f971a274640 1 --2- 172.21.15.43:0/4263996121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9700004800 0x7f9700004c80 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9704009ac0 tx=0x7f9700005830 comp rx=0 tx=0).stop 2024-04-25T08:54:57.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.286+0000 7f9718811640 1 -- 172.21.15.43:0/4263996121 reap_dead start 2024-04-25T08:54:57.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.286+0000 7f971a274640 1 -- 172.21.15.43:0/4263996121 shutdown_connections 2024-04-25T08:54:57.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.286+0000 7f971a274640 1 -- 172.21.15.43:0/4263996121 >> 172.21.15.43:0/4263996121 conn(0x7f9714072aa0 msgr2=0x7f971409aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:57.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.287+0000 7f971a274640 1 -- 172.21.15.43:0/4263996121 shutdown_connections 2024-04-25T08:54:57.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.287+0000 7f971a274640 1 -- 172.21.15.43:0/4263996121 wait complete. 2024-04-25T08:54:57.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.288+0000 7f971a274640 1 Processor -- start 2024-04-25T08:54:57.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.288+0000 7f971a274640 1 -- start start 2024-04-25T08:54:57.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.289+0000 7f971a274640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9700004800 0x7f9714140940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.289+0000 7f971a274640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f971415afb0 0x7f971415b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.289+0000 7f971a274640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97141f49a0 0x7f97141f4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.289+0000 7f971a274640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97140b4be0 con 0x7f9700004800 2024-04-25T08:54:57.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.289+0000 7f971a274640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97140b4900 con 0x7f971415afb0 2024-04-25T08:54:57.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.289+0000 7f971a274640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97140cea60 con 0x7f97141f49a0 2024-04-25T08:54:57.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.289+0000 7f9713fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9700004800 0x7f9714140940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.289+0000 7f9713fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9700004800 0x7f9714140940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49906/0 (socket says 172.21.15.43:49906) 2024-04-25T08:54:57.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.289+0000 7f9713fff640 1 -- 172.21.15.43:0/3344361406 learned_addr learned my addr 172.21.15.43:0/3344361406 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:57.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.289+0000 7f97137fe640 1 --2- 172.21.15.43:0/3344361406 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f971415afb0 0x7f971415b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.289+0000 7f9718811640 1 --2- 172.21.15.43:0/3344361406 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97141f49a0 0x7f97141f4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.289+0000 7f9713fff640 1 -- 172.21.15.43:0/3344361406 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97141f49a0 msgr2=0x7f97141f4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:57.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.289+0000 7f9713fff640 1 --2- 172.21.15.43:0/3344361406 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97141f49a0 0x7f97141f4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.289+0000 7f9713fff640 1 -- 172.21.15.43:0/3344361406 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f971415afb0 msgr2=0x7f971415b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:57.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.289+0000 7f9713fff640 1 --2- 172.21.15.43:0/3344361406 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f971415afb0 0x7f971415b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:57.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.289+0000 7f9713fff640 1 -- 172.21.15.43:0/3344361406 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9714140e80 con 0x7f9700004800 2024-04-25T08:54:57.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.289+0000 7f97137fe640 1 --2- 172.21.15.43:0/3344361406 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f971415afb0 0x7f971415b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:57.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.289+0000 7f9718811640 1 --2- 172.21.15.43:0/3344361406 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97141f49a0 0x7f97141f4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:57.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.289+0000 7f9713fff640 1 --2- 172.21.15.43:0/3344361406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9700004800 0x7f9714140940 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f96fc007310 tx=0x7f96fc0077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:57.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.289+0000 7f9710ff9640 1 -- 172.21.15.43:0/3344361406 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96fc00f960 con 0x7f9700004800 2024-04-25T08:54:57.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.289+0000 7f9710ff9640 1 -- 172.21.15.43:0/3344361406 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96fc018070 con 0x7f9700004800 2024-04-25T08:54:57.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.289+0000 7f971a274640 1 -- 172.21.15.43:0/3344361406 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97141f5760 con 0x7f9700004800 2024-04-25T08:54:57.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.289+0000 7f9710ff9640 1 -- 172.21.15.43:0/3344361406 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96fc00fd60 con 0x7f9700004800 2024-04-25T08:54:57.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.289+0000 7f971a274640 1 -- 172.21.15.43:0/3344361406 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9714143560 con 0x7f9700004800 2024-04-25T08:54:57.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.290+0000 7f971a274640 1 -- 172.21.15.43:0/3344361406 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f971415b950 con 0x7f9700004800 2024-04-25T08:54:57.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.291+0000 7f9710ff9640 1 -- 172.21.15.43:0/3344361406 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f96fc01d460 con 0x7f9700004800 2024-04-25T08:54:57.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.291+0000 7f9710ff9640 1 --2- 172.21.15.43:0/3344361406 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f96f406b4d0 0x7f96f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:57.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.291+0000 7f9710ff9640 1 -- 172.21.15.43:0/3344361406 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f96fc08a260 con 0x7f9700004800 2024-04-25T08:54:57.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.292+0000 7f97137fe640 1 --2- 172.21.15.43:0/3344361406 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f96f406b4d0 0x7f96f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:57.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.292+0000 7f97137fe640 1 --2- 172.21.15.43:0/3344361406 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f96f406b4d0 0x7f96f406d990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9704033000 tx=0x7f9704008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:57.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.295+0000 7f9710ff9640 1 -- 172.21.15.43:0/3344361406 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f96fc058ff0 con 0x7f9700004800 2024-04-25T08:54:57.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:57.515+0000 7f971a274640 1 -- 172.21.15.43:0/3344361406 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f9714074f60 con 0x7f9700004800 2024-04-25T08:54:58.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.171+0000 7f9710ff9640 1 -- 172.21.15.43:0/3344361406 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v8) v1 ==== 104+0+0 (secure 0 0 0) 0x7f96fc0588f0 con 0x7f9700004800 2024-04-25T08:54:58.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.174+0000 7f971a274640 1 -- 172.21.15.43:0/3344361406 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f96f406b4d0 msgr2=0x7f96f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:58.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.174+0000 7f971a274640 1 --2- 172.21.15.43:0/3344361406 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f96f406b4d0 0x7f96f406d990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9704033000 tx=0x7f9704008030 comp rx=0 tx=0).stop 2024-04-25T08:54:58.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.174+0000 7f971a274640 1 -- 172.21.15.43:0/3344361406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9700004800 msgr2=0x7f9714140940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:58.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.174+0000 7f971a274640 1 --2- 172.21.15.43:0/3344361406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9700004800 0x7f9714140940 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f96fc007310 tx=0x7f96fc0077d0 comp rx=0 tx=0).stop 2024-04-25T08:54:58.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.174+0000 7f971a274640 1 -- 172.21.15.43:0/3344361406 shutdown_connections 2024-04-25T08:54:58.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.174+0000 7f971a274640 1 --2- 172.21.15.43:0/3344361406 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97141f49a0 0x7f97141f4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:58.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.174+0000 7f971a274640 1 --2- 172.21.15.43:0/3344361406 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f96f406b4d0 0x7f96f406d990 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:58.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.174+0000 7f971a274640 1 --2- 172.21.15.43:0/3344361406 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f971415afb0 0x7f971415b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:58.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.174+0000 7f971a274640 1 --2- 172.21.15.43:0/3344361406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9700004800 0x7f9714140940 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:58.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.174+0000 7f971a274640 1 -- 172.21.15.43:0/3344361406 >> 172.21.15.43:0/3344361406 conn(0x7f9714072aa0 msgr2=0x7f971415cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:58.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.174+0000 7f971a274640 1 -- 172.21.15.43:0/3344361406 shutdown_connections 2024-04-25T08:54:58.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.174+0000 7f971a274640 1 -- 172.21.15.43:0/3344361406 wait complete. 2024-04-25T08:54:58.179 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T08:54:58.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.301+0000 7f76dce5d640 1 Processor -- start 2024-04-25T08:54:58.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.301+0000 7f76dce5d640 1 -- start start 2024-04-25T08:54:58.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.302+0000 7f76dce5d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f76d815ba10 0x7f76d806df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:58.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.302+0000 7f76dce5d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f76d806e460 0x7f76d80aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:58.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.302+0000 7f76dce5d640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f76d80af0a0 0x7f76d80acda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:58.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.302+0000 7f76dce5d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76d80af510 con 0x7f76d806e460 2024-04-25T08:54:58.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.302+0000 7f76dce5d640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76d815bdf0 con 0x7f76d80af0a0 2024-04-25T08:54:58.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.302+0000 7f76dce5d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76d806e9e0 con 0x7f76d815ba10 2024-04-25T08:54:58.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.302+0000 7f76cdd9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f76d806e460 0x7f76d80aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:58.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.302+0000 7f76cdd9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f76d806e460 0x7f76d80aeb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49914/0 (socket says 172.21.15.43:49914) 2024-04-25T08:54:58.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.302+0000 7f76cdd9b640 1 -- 172.21.15.43:0/3380145734 learned_addr learned my addr 172.21.15.43:0/3380145734 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:58.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.302+0000 7f76d6d9d640 1 --2- 172.21.15.43:0/3380145734 >> v2:172.21.15.193:3300/0 conn(0x7f76d80af0a0 0x7f76d80acda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:58.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.302+0000 7f76d659c640 1 --2- 172.21.15.43:0/3380145734 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f76d815ba10 0x7f76d806df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:58.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.302+0000 7f76cdd9b640 1 -- 172.21.15.43:0/3380145734 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f76d815ba10 msgr2=0x7f76d806df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:58.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.302+0000 7f76cdd9b640 1 --2- 172.21.15.43:0/3380145734 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f76d815ba10 0x7f76d806df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:58.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.302+0000 7f76cdd9b640 1 -- 172.21.15.43:0/3380145734 >> v2:172.21.15.193:3300/0 conn(0x7f76d80af0a0 msgr2=0x7f76d80acda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:58.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.302+0000 7f76cdd9b640 1 --2- 172.21.15.43:0/3380145734 >> v2:172.21.15.193:3300/0 conn(0x7f76d80af0a0 0x7f76d80acda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:58.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.302+0000 7f76cdd9b640 1 -- 172.21.15.43:0/3380145734 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d806ecc0 con 0x7f76d806e460 2024-04-25T08:54:58.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.303+0000 7f76cdd9b640 1 --2- 172.21.15.43:0/3380145734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76d806e460 0x7f76d80aeb60 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f76d8070a90 tx=0x7f76c002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea8fbd0b5b4d4a90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:58.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.303+0000 7f76d5d9b640 1 -- 172.21.15.43:0/3380145734 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c0037070 con 0x7f76d806e460 2024-04-25T08:54:58.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.303+0000 7f76d5d9b640 1 -- 172.21.15.43:0/3380145734 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76c0032440 con 0x7f76d806e460 2024-04-25T08:54:58.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.303+0000 7f76d5d9b640 1 -- 172.21.15.43:0/3380145734 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c0032810 con 0x7f76d806e460 2024-04-25T08:54:58.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.304+0000 7f76dce5d640 1 -- 172.21.15.43:0/3380145734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76d806e460 msgr2=0x7f76d80aeb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:58.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.304+0000 7f76dce5d640 1 --2- 172.21.15.43:0/3380145734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76d806e460 0x7f76d80aeb60 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f76d8070a90 tx=0x7f76c002a1e0 comp rx=0 tx=0).stop 2024-04-25T08:54:58.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.304+0000 7f76dce5d640 1 -- 172.21.15.43:0/3380145734 shutdown_connections 2024-04-25T08:54:58.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.304+0000 7f76dce5d640 1 --2- 172.21.15.43:0/3380145734 >> v2:172.21.15.193:3300/0 conn(0x7f76d80af0a0 0x7f76d80acda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:58.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.304+0000 7f76dce5d640 1 --2- 172.21.15.43:0/3380145734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76d806e460 0x7f76d80aeb60 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:58.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.304+0000 7f76dce5d640 1 --2- 172.21.15.43:0/3380145734 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f76d815ba10 0x7f76d806df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:58.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.304+0000 7f76dce5d640 1 -- 172.21.15.43:0/3380145734 >> 172.21.15.43:0/3380145734 conn(0x7f76d806ab30 msgr2=0x7f76d806af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:58.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.304+0000 7f76dce5d640 1 -- 172.21.15.43:0/3380145734 shutdown_connections 2024-04-25T08:54:58.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.304+0000 7f76dce5d640 1 -- 172.21.15.43:0/3380145734 wait complete. 2024-04-25T08:54:58.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.305+0000 7f76dce5d640 1 Processor -- start 2024-04-25T08:54:58.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.305+0000 7f76dce5d640 1 -- start start 2024-04-25T08:54:58.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.306+0000 7f76dce5d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f76d806e460 0x7f76d80c9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:58.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.306+0000 7f76dce5d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f76d80af0a0 0x7f76d814b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:58.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.306+0000 7f76dce5d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76d815ba10 0x7f76d815d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:58.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.306+0000 7f76dce5d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d815c120 con 0x7f76d815ba10 2024-04-25T08:54:58.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.306+0000 7f76dce5d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d806f280 con 0x7f76d80af0a0 2024-04-25T08:54:58.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.306+0000 7f76dce5d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76d815c2a0 con 0x7f76d806e460 2024-04-25T08:54:58.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.306+0000 7f76d6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76d815ba10 0x7f76d815d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:58.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.306+0000 7f76d6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76d815ba10 0x7f76d815d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49922/0 (socket says 172.21.15.43:49922) 2024-04-25T08:54:58.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.306+0000 7f76d6d9d640 1 -- 172.21.15.43:0/2396591387 learned_addr learned my addr 172.21.15.43:0/2396591387 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:58.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.306+0000 7f76d659c640 1 --2- 172.21.15.43:0/2396591387 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f76d806e460 0x7f76d80c9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:58.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.306+0000 7f76cdd9b640 1 --2- 172.21.15.43:0/2396591387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f76d80af0a0 0x7f76d814b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:58.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.306+0000 7f76d6d9d640 1 -- 172.21.15.43:0/2396591387 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f76d806e460 msgr2=0x7f76d80c9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:58.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.306+0000 7f76d6d9d640 1 --2- 172.21.15.43:0/2396591387 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f76d806e460 0x7f76d80c9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:58.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.306+0000 7f76d6d9d640 1 -- 172.21.15.43:0/2396591387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f76d80af0a0 msgr2=0x7f76d814b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:58.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.306+0000 7f76d6d9d640 1 --2- 172.21.15.43:0/2396591387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f76d80af0a0 0x7f76d814b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:58.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.306+0000 7f76d6d9d640 1 -- 172.21.15.43:0/2396591387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d8072960 con 0x7f76d815ba10 2024-04-25T08:54:58.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.306+0000 7f76d659c640 1 --2- 172.21.15.43:0/2396591387 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f76d806e460 0x7f76d80c9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:58.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.307+0000 7f76d6d9d640 1 --2- 172.21.15.43:0/2396591387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76d815ba10 0x7f76d815d560 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f76d0007e60 tx=0x7f76d000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:58.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.307+0000 7f76cf7fe640 1 -- 172.21.15.43:0/2396591387 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d0014620 con 0x7f76d815ba10 2024-04-25T08:54:58.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.307+0000 7f76cf7fe640 1 -- 172.21.15.43:0/2396591387 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76d0019070 con 0x7f76d815ba10 2024-04-25T08:54:58.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.307+0000 7f76dce5d640 1 -- 172.21.15.43:0/2396591387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76d80c9350 con 0x7f76d815ba10 2024-04-25T08:54:58.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.307+0000 7f76cf7fe640 1 -- 172.21.15.43:0/2396591387 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76d0014a20 con 0x7f76d815ba10 2024-04-25T08:54:58.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.307+0000 7f76dce5d640 1 -- 172.21.15.43:0/2396591387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76d814bd90 con 0x7f76d815ba10 2024-04-25T08:54:58.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.308+0000 7f76dce5d640 1 -- 172.21.15.43:0/2396591387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76d80c9140 con 0x7f76d815ba10 2024-04-25T08:54:58.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.308+0000 7f76cf7fe640 1 -- 172.21.15.43:0/2396591387 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f76d0014bc0 con 0x7f76d815ba10 2024-04-25T08:54:58.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.309+0000 7f76cf7fe640 1 --2- 172.21.15.43:0/2396591387 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f76b806b500 0x7f76b806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:58.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.309+0000 7f76cf7fe640 1 -- 172.21.15.43:0/2396591387 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f76d008ab10 con 0x7f76d815ba10 2024-04-25T08:54:58.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.309+0000 7f76d659c640 1 --2- 172.21.15.43:0/2396591387 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f76b806b500 0x7f76b806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:58.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.309+0000 7f76d659c640 1 --2- 172.21.15.43:0/2396591387 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f76b806b500 0x7f76b806d9c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f76c8006fd0 tx=0x7f76c8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:58.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.313+0000 7f76cf7fe640 1 -- 172.21.15.43:0/2396591387 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f76d0059800 con 0x7f76d815ba10 2024-04-25T08:54:58.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:58.493+0000 7f76dce5d640 1 -- 172.21.15.43:0/2396591387 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f76d8068a40 con 0x7f76d815ba10 2024-04-25T08:54:59.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.173+0000 7f76cf7fe640 1 -- 172.21.15.43:0/2396591387 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) v1 ==== 181+0+0 (secure 0 0 0) 0x7f76d0059100 con 0x7f76d815ba10 2024-04-25T08:54:59.163 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' removed 2024-04-25T08:54:59.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.177+0000 7f76dce5d640 1 -- 172.21.15.43:0/2396591387 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f76b806b500 msgr2=0x7f76b806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:59.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.177+0000 7f76dce5d640 1 --2- 172.21.15.43:0/2396591387 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f76b806b500 0x7f76b806d9c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f76c8006fd0 tx=0x7f76c8008040 comp rx=0 tx=0).stop 2024-04-25T08:54:59.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.177+0000 7f76dce5d640 1 -- 172.21.15.43:0/2396591387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76d815ba10 msgr2=0x7f76d815d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:59.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.177+0000 7f76dce5d640 1 --2- 172.21.15.43:0/2396591387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76d815ba10 0x7f76d815d560 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f76d0007e60 tx=0x7f76d000e4b0 comp rx=0 tx=0).stop 2024-04-25T08:54:59.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.177+0000 7f76dce5d640 1 -- 172.21.15.43:0/2396591387 shutdown_connections 2024-04-25T08:54:59.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.177+0000 7f76dce5d640 1 --2- 172.21.15.43:0/2396591387 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f76d815ba10 0x7f76d815d560 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:59.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.177+0000 7f76dce5d640 1 --2- 172.21.15.43:0/2396591387 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f76b806b500 0x7f76b806d9c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:59.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.177+0000 7f76dce5d640 1 --2- 172.21.15.43:0/2396591387 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f76d80af0a0 0x7f76d814b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:59.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.177+0000 7f76dce5d640 1 --2- 172.21.15.43:0/2396591387 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f76d806e460 0x7f76d80c9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:59.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.177+0000 7f76dce5d640 1 -- 172.21.15.43:0/2396591387 >> 172.21.15.43:0/2396591387 conn(0x7f76d806ab30 msgr2=0x7f76d80b0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:59.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.177+0000 7f76dce5d640 1 -- 172.21.15.43:0/2396591387 shutdown_connections 2024-04-25T08:54:59.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.177+0000 7f76dce5d640 1 -- 172.21.15.43:0/2396591387 wait complete. 2024-04-25T08:54:59.182 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T08:54:59.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.305+0000 7fa5dc74f640 1 Processor -- start 2024-04-25T08:54:59.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.305+0000 7fa5dc74f640 1 -- start start 2024-04-25T08:54:59.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.306+0000 7fa5dc74f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5d415bb40 0x7fa5d40ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:59.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.306+0000 7fa5dc74f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa5d40cca30 0x7fa5d4070b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:59.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.306+0000 7fa5dc74f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5d4160510 con 0x7fa5d40aec70 2024-04-25T08:54:59.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.306+0000 7fa5dc74f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5d40af0d0 con 0x7fa5d40cca30 2024-04-25T08:54:59.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.306+0000 7fa5dc74f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d415f750 con 0x7fa5d415bb40 2024-04-25T08:54:59.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.306+0000 7fa5dacec640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa5d40cca30 0x7fa5d4070b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:59.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.306+0000 7fa5d9cea640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa5d40aec70 0x7fa5d40cc320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48504/0 (socket says 172.21.15.43:48504) 2024-04-25T08:54:59.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.306+0000 7fa5dacec640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa5d40cca30 0x7fa5d4070b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49928/0 (socket says 172.21.15.43:49928) 2024-04-25T08:54:59.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.306+0000 7fa5d9cea640 1 -- 172.21.15.43:0/3071210172 learned_addr learned my addr 172.21.15.43:0/3071210172 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:54:59.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.306+0000 7fa5da4eb640 1 --2- 172.21.15.43:0/3071210172 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5d415bb40 0x7fa5d40ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:59.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.306+0000 7fa5dacec640 1 -- 172.21.15.43:0/3071210172 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5d415bb40 msgr2=0x7fa5d40ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:54:59.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.306+0000 7fa5dacec640 1 --2- 172.21.15.43:0/3071210172 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5d415bb40 0x7fa5d40ae730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:59.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.306+0000 7fa5dacec640 1 -- 172.21.15.43:0/3071210172 >> v1:172.21.15.43:6789/0 conn(0x7fa5d40aec70 legacy=0x7fa5d40cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:59.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.306+0000 7fa5dacec640 1 -- 172.21.15.43:0/3071210172 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d415fa30 con 0x7fa5d40cca30 2024-04-25T08:54:59.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.306+0000 7fa5d94e9640 1 -- 172.21.15.43:0/3071210172 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 336015490 0 0) 0x7fa5d4160510 con 0x7fa5d40aec70 2024-04-25T08:54:59.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.307+0000 7fa5dacec640 1 --2- 172.21.15.43:0/3071210172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5d40cca30 0x7fa5d4070b80 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc009d90 tx=0x7fa5cc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9693b9bb1ec0c70d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:59.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.307+0000 7fa5d94e9640 1 -- 172.21.15.43:0/3071210172 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc007740 con 0x7fa5d40cca30 2024-04-25T08:54:59.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.307+0000 7fa5d94e9640 1 -- 172.21.15.43:0/3071210172 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5cc0078e0 con 0x7fa5d40cca30 2024-04-25T08:54:59.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.307+0000 7fa5d94e9640 1 -- 172.21.15.43:0/3071210172 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc007cb0 con 0x7fa5d40cca30 2024-04-25T08:54:59.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.308+0000 7fa5dc74f640 1 -- 172.21.15.43:0/3071210172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5d40cca30 msgr2=0x7fa5d4070b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:59.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.308+0000 7fa5dc74f640 1 --2- 172.21.15.43:0/3071210172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5d40cca30 0x7fa5d4070b80 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc009d90 tx=0x7fa5cc02a660 comp rx=0 tx=0).stop 2024-04-25T08:54:59.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.308+0000 7fa5dc74f640 1 -- 172.21.15.43:0/3071210172 shutdown_connections 2024-04-25T08:54:59.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.308+0000 7fa5dc74f640 1 --2- 172.21.15.43:0/3071210172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5d40cca30 0x7fa5d4070b80 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:59.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.308+0000 7fa5dc74f640 1 --2- 172.21.15.43:0/3071210172 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5d415bb40 0x7fa5d40ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:59.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.308+0000 7fa5dc74f640 1 -- 172.21.15.43:0/3071210172 >> 172.21.15.43:0/3071210172 conn(0x7fa5d406ab30 msgr2=0x7fa5d406af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:54:59.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.308+0000 7fa5dc74f640 1 -- 172.21.15.43:0/3071210172 shutdown_connections 2024-04-25T08:54:59.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.308+0000 7fa5dc74f640 1 -- 172.21.15.43:0/3071210172 wait complete. 2024-04-25T08:54:59.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.309+0000 7fa5dc74f640 1 Processor -- start 2024-04-25T08:54:59.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.309+0000 7fa5dc74f640 1 -- start start 2024-04-25T08:54:59.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.310+0000 7fa5dc74f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5d40aec70 0x7fa5d4155000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:59.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.310+0000 7fa5dc74f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5d40cca30 0x7fa5d41e7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:59.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.310+0000 7fa5dc74f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5d415bb40 0x7fa5d41e7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:59.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.310+0000 7fa5dc74f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d40710c0 con 0x7fa5d415bb40 2024-04-25T08:54:59.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.310+0000 7fa5dc74f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d415fff0 con 0x7fa5d40cca30 2024-04-25T08:54:59.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.310+0000 7fa5dc74f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d4071240 con 0x7fa5d40aec70 2024-04-25T08:54:59.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.310+0000 7fa5dacec640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5d415bb40 0x7fa5d41e7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:59.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.310+0000 7fa5dacec640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5d415bb40 0x7fa5d41e7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49930/0 (socket says 172.21.15.43:49930) 2024-04-25T08:54:59.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.310+0000 7fa5dacec640 1 -- 172.21.15.43:0/3783171901 learned_addr learned my addr 172.21.15.43:0/3783171901 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:54:59.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.310+0000 7fa5da4eb640 1 --2- 172.21.15.43:0/3783171901 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5d40aec70 0x7fa5d4155000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:59.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.310+0000 7fa5d9cea640 1 --2- 172.21.15.43:0/3783171901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5d40cca30 0x7fa5d41e7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:59.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.310+0000 7fa5dacec640 1 -- 172.21.15.43:0/3783171901 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5d40aec70 msgr2=0x7fa5d4155000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:59.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.310+0000 7fa5dacec640 1 --2- 172.21.15.43:0/3783171901 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5d40aec70 0x7fa5d4155000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:59.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.310+0000 7fa5dacec640 1 -- 172.21.15.43:0/3783171901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5d40cca30 msgr2=0x7fa5d41e7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:54:59.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.310+0000 7fa5dacec640 1 --2- 172.21.15.43:0/3783171901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5d40cca30 0x7fa5d41e7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:54:59.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.310+0000 7fa5dacec640 1 -- 172.21.15.43:0/3783171901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d4155540 con 0x7fa5d415bb40 2024-04-25T08:54:59.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.310+0000 7fa5d9cea640 1 --2- 172.21.15.43:0/3783171901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5d40cca30 0x7fa5d41e7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:54:59.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.311+0000 7fa5dacec640 1 --2- 172.21.15.43:0/3783171901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5d415bb40 0x7fa5d41e7b40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc009d90 tx=0x7fa5cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:59.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.311+0000 7fa5caffd640 1 -- 172.21.15.43:0/3783171901 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc040070 con 0x7fa5d415bb40 2024-04-25T08:54:59.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.311+0000 7fa5caffd640 1 -- 172.21.15.43:0/3783171901 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5cc031c40 con 0x7fa5d415bb40 2024-04-25T08:54:59.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.311+0000 7fa5caffd640 1 -- 172.21.15.43:0/3783171901 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc009450 con 0x7fa5d415bb40 2024-04-25T08:54:59.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.311+0000 7fa5dc74f640 1 -- 172.21.15.43:0/3783171901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5d41f5f60 con 0x7fa5d415bb40 2024-04-25T08:54:59.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.311+0000 7fa5dc74f640 1 -- 172.21.15.43:0/3783171901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5d415bfc0 con 0x7fa5d415bb40 2024-04-25T08:54:59.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.312+0000 7fa5dc74f640 1 -- 172.21.15.43:0/3783171901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5d4070ee0 con 0x7fa5d415bb40 2024-04-25T08:54:59.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.312+0000 7fa5caffd640 1 -- 172.21.15.43:0/3783171901 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa5cc039040 con 0x7fa5d415bb40 2024-04-25T08:54:59.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.313+0000 7fa5caffd640 1 --2- 172.21.15.43:0/3783171901 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa5b806b4d0 0x7fa5b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:54:59.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.313+0000 7fa5caffd640 1 -- 172.21.15.43:0/3783171901 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5502+0+0 (secure 0 0 0) 0x7fa5cc0aafe0 con 0x7fa5d415bb40 2024-04-25T08:54:59.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.313+0000 7fa5da4eb640 1 --2- 172.21.15.43:0/3783171901 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa5b806b4d0 0x7fa5b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:54:59.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.313+0000 7fa5da4eb640 1 --2- 172.21.15.43:0/3783171901 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa5b806b4d0 0x7fa5b806d990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa5d00079a0 tx=0x7fa5d0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:54:59.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.317+0000 7fa5caffd640 1 -- 172.21.15.43:0/3783171901 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa5d4070ee0 con 0x7fa5d415bb40 2024-04-25T08:54:59.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:54:59.503+0000 7fa5dc74f640 1 -- 172.21.15.43:0/3783171901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa5d4068a40 con 0x7fa5d415bb40 2024-04-25T08:55:00.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.178+0000 7fa5caffd640 1 -- 172.21.15.43:0/3783171901 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa5cc0798d0 con 0x7fa5d415bb40 2024-04-25T08:55:00.168 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' removed 2024-04-25T08:55:00.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.181+0000 7fa5dc74f640 1 -- 172.21.15.43:0/3783171901 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa5b806b4d0 msgr2=0x7fa5b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:00.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.181+0000 7fa5dc74f640 1 --2- 172.21.15.43:0/3783171901 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa5b806b4d0 0x7fa5b806d990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa5d00079a0 tx=0x7fa5d0008040 comp rx=0 tx=0).stop 2024-04-25T08:55:00.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.181+0000 7fa5dc74f640 1 -- 172.21.15.43:0/3783171901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5d415bb40 msgr2=0x7fa5d41e7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:00.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.181+0000 7fa5dc74f640 1 --2- 172.21.15.43:0/3783171901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5d415bb40 0x7fa5d41e7b40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc009d90 tx=0x7fa5cc004a40 comp rx=0 tx=0).stop 2024-04-25T08:55:00.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.181+0000 7fa5dc74f640 1 -- 172.21.15.43:0/3783171901 shutdown_connections 2024-04-25T08:55:00.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.181+0000 7fa5dc74f640 1 --2- 172.21.15.43:0/3783171901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5d415bb40 0x7fa5d41e7b40 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:00.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.181+0000 7fa5dc74f640 1 --2- 172.21.15.43:0/3783171901 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa5b806b4d0 0x7fa5b806d990 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:00.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.181+0000 7fa5dc74f640 1 --2- 172.21.15.43:0/3783171901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5d40cca30 0x7fa5d41e7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:00.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.181+0000 7fa5dc74f640 1 --2- 172.21.15.43:0/3783171901 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5d40aec70 0x7fa5d4155000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:00.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.182+0000 7fa5dc74f640 1 -- 172.21.15.43:0/3783171901 >> 172.21.15.43:0/3783171901 conn(0x7fa5d406ab30 msgr2=0x7fa5d4156660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:00.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.182+0000 7fa5dc74f640 1 -- 172.21.15.43:0/3783171901 shutdown_connections 2024-04-25T08:55:00.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.182+0000 7fa5dc74f640 1 -- 172.21.15.43:0/3783171901 wait complete. 2024-04-25T08:55:00.188 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T08:55:00.188 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T08:55:00.188 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T08:55:00.188 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:55:00.189 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T08:55:00.189 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T08:55:00.189 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T08:55:00.189 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:55:00.189 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T08:55:00.189 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T08:55:00.190 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T08:55:00.190 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:55:00.190 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T08:55:00.190 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T08:55:00.190 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T08:55:00.190 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:55:00.200 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T08:55:00.200 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T08:55:00.201 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T08:55:00.201 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T08:55:00.202 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T08:55:00.202 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T08:55:00.203 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T08:55:00.203 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T08:55:00.243 INFO:tasks.ceph.mds.a:Started 2024-04-25T08:55:00.243 DEBUG:teuthology.parallel:result is None 2024-04-25T08:55:00.244 INFO:tasks.ceph.mds.c:Started 2024-04-25T08:55:00.245 DEBUG:teuthology.parallel:result is None 2024-04-25T08:55:00.245 INFO:tasks.ceph.mds.b:Started 2024-04-25T08:55:00.246 DEBUG:teuthology.parallel:result is None 2024-04-25T08:55:00.246 INFO:tasks.ceph.mds.d:Started 2024-04-25T08:55:00.246 DEBUG:teuthology.parallel:result is None 2024-04-25T08:55:00.247 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T08:55:00.323 INFO:tasks.ceph.mds.d.smithi193.stdout:starting mds.d at 2024-04-25T08:55:00.323 INFO:tasks.ceph.mds.b.smithi193.stdout:starting mds.b at 2024-04-25T08:55:00.324 INFO:tasks.ceph.mds.d.smithi193.stderr:2024-04-25T08:55:00.334+0000 7f924e5fe640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-04-25T08:55:00.324 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-04-25T08:55:00.324 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2024-04-25T08:55:00.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.364+0000 7f8f89b5c640 1 Processor -- start 2024-04-25T08:55:00.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.364+0000 7f8f89b5c640 1 -- start start 2024-04-25T08:55:00.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.365+0000 7f8f89b5c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8f8415df50 0x7f8f841619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:00.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.365+0000 7f8f89b5c640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f8f840cf670 0x7f8f84162380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:00.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.365+0000 7f8f89b5c640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f840d24a0 con 0x7f8f84161f30 2024-04-25T08:55:00.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.365+0000 7f8f89b5c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f840cfa80 con 0x7f8f8415df50 2024-04-25T08:55:00.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.365+0000 7f8f89b5c640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f8415e330 con 0x7f8f840cf670 2024-04-25T08:55:00.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.365+0000 7f8f837fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8f8415df50 0x7f8f841619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:00.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.365+0000 7f8f837fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8f8415df50 0x7f8f841619f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41876/0 (socket says 172.21.15.43:41876) 2024-04-25T08:55:00.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.365+0000 7f8f837fe640 1 -- 172.21.15.43:0/605699615 learned_addr learned my addr 172.21.15.43:0/605699615 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:00.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.365+0000 7f8f83fff640 1 --2- 172.21.15.43:0/605699615 >> v2:172.21.15.193:3300/0 conn(0x7f8f840cf670 0x7f8f84162380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:00.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.366+0000 7f8f837fe640 1 -- 172.21.15.43:0/605699615 >> v2:172.21.15.193:3300/0 conn(0x7f8f840cf670 msgr2=0x7f8f84162380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:00.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.366+0000 7f8f837fe640 1 --2- 172.21.15.43:0/605699615 >> v2:172.21.15.193:3300/0 conn(0x7f8f840cf670 0x7f8f84162380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:00.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.366+0000 7f8f837fe640 1 -- 172.21.15.43:0/605699615 >> v1:172.21.15.193:6789/0 conn(0x7f8f84161f30 legacy=0x7f8f840c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.366+0000 7f8f837fe640 1 -- 172.21.15.43:0/605699615 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f840d2740 con 0x7f8f8415df50 2024-04-25T08:55:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.366+0000 7f8f837fe640 1 --2- 172.21.15.43:0/605699615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f8415df50 0x7f8f841619f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8f840caa00 tx=0x7f8f7402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3174a048fa9afd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.367+0000 7f8f827fc640 1 -- 172.21.15.43:0/605699615 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f74037070 con 0x7f8f8415df50 2024-04-25T08:55:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.367+0000 7f8f827fc640 1 -- 172.21.15.43:0/605699615 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f74032440 con 0x7f8f8415df50 2024-04-25T08:55:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.367+0000 7f8f827fc640 1 -- 172.21.15.43:0/605699615 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f74032810 con 0x7f8f8415df50 2024-04-25T08:55:00.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.367+0000 7f8f89b5c640 1 -- 172.21.15.43:0/605699615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f8415df50 msgr2=0x7f8f841619f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:00.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.367+0000 7f8f89b5c640 1 --2- 172.21.15.43:0/605699615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f8415df50 0x7f8f841619f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8f840caa00 tx=0x7f8f7402a270 comp rx=0 tx=0).stop 2024-04-25T08:55:00.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.367+0000 7f8f89b5c640 1 -- 172.21.15.43:0/605699615 shutdown_connections 2024-04-25T08:55:00.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.367+0000 7f8f89b5c640 1 --2- 172.21.15.43:0/605699615 >> v2:172.21.15.193:3300/0 conn(0x7f8f840cf670 0x7f8f84162380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:00.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.367+0000 7f8f89b5c640 1 --2- 172.21.15.43:0/605699615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f8415df50 0x7f8f841619f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:00.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.367+0000 7f8f89b5c640 1 -- 172.21.15.43:0/605699615 >> 172.21.15.43:0/605699615 conn(0x7f8f840d3a30 msgr2=0x7f8f840d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:00.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.367+0000 7f8f89b5c640 1 -- 172.21.15.43:0/605699615 shutdown_connections 2024-04-25T08:55:00.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.367+0000 7f8f89b5c640 1 -- 172.21.15.43:0/605699615 wait complete. 2024-04-25T08:55:00.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.369+0000 7f8f89b5c640 1 Processor -- start 2024-04-25T08:55:00.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.369+0000 7f8f89b5c640 1 -- start start 2024-04-25T08:55:00.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.370+0000 7f8f89b5c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f840cf670 0x7f8f8414ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:00.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.370+0000 7f8f89b5c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f8415df50 0x7f8f8415ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:00.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.370+0000 7f8f89b5c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f84161f30 0x7f8f840cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:00.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.370+0000 7f8f89b5c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f840c77d0 con 0x7f8f84161f30 2024-04-25T08:55:00.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.370+0000 7f8f89b5c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f840c74f0 con 0x7f8f8415df50 2024-04-25T08:55:00.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.370+0000 7f8f89b5c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f840d2190 con 0x7f8f840cf670 2024-04-25T08:55:00.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.370+0000 7f8f83fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f84161f30 0x7f8f840cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:00.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.370+0000 7f8f83fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f84161f30 0x7f8f840cbb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41888/0 (socket says 172.21.15.43:41888) 2024-04-25T08:55:00.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.370+0000 7f8f83fff640 1 -- 172.21.15.43:0/552960128 learned_addr learned my addr 172.21.15.43:0/552960128 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:00.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.370+0000 7f8f837fe640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f840cf670 0x7f8f8414ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:00.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.370+0000 7f8f82ffd640 1 --2- 172.21.15.43:0/552960128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f8415df50 0x7f8f8415ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:00.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.370+0000 7f8f83fff640 1 -- 172.21.15.43:0/552960128 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f840cf670 msgr2=0x7f8f8414ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:00.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.370+0000 7f8f83fff640 1 --2- 172.21.15.43:0/552960128 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f840cf670 0x7f8f8414ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:00.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.370+0000 7f8f83fff640 1 -- 172.21.15.43:0/552960128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f8415df50 msgr2=0x7f8f8415ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:00.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.370+0000 7f8f83fff640 1 --2- 172.21.15.43:0/552960128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f8415df50 0x7f8f8415ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:00.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.370+0000 7f8f83fff640 1 -- 172.21.15.43:0/552960128 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f8415aad0 con 0x7f8f84161f30 2024-04-25T08:55:00.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.370+0000 7f8f83fff640 1 --2- 172.21.15.43:0/552960128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f84161f30 0x7f8f840cbb40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c00efc0 tx=0x7f8f7c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:00.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.371+0000 7f8f6bfff640 1 -- 172.21.15.43:0/552960128 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c018020 con 0x7f8f84161f30 2024-04-25T08:55:00.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.371+0000 7f8f6bfff640 1 -- 172.21.15.43:0/552960128 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f7c014480 con 0x7f8f84161f30 2024-04-25T08:55:00.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.371+0000 7f8f6bfff640 1 -- 172.21.15.43:0/552960128 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c014870 con 0x7f8f84161f30 2024-04-25T08:55:00.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.371+0000 7f8f89b5c640 1 -- 172.21.15.43:0/552960128 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f8415e990 con 0x7f8f84161f30 2024-04-25T08:55:00.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.371+0000 7f8f89b5c640 1 -- 172.21.15.43:0/552960128 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f8415a7f0 con 0x7f8f84161f30 2024-04-25T08:55:00.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.372+0000 7f8f89b5c640 1 -- 172.21.15.43:0/552960128 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f840680f0 con 0x7f8f84161f30 2024-04-25T08:55:00.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.372+0000 7f8f6bfff640 1 -- 172.21.15.43:0/552960128 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8f7c005d10 con 0x7f8f84161f30 2024-04-25T08:55:00.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.373+0000 7f8f6bfff640 1 --2- 172.21.15.43:0/552960128 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8f7006b4d0 0x7f8f7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:00.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.373+0000 7f8f6bfff640 1 -- 172.21.15.43:0/552960128 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8f7c08abc0 con 0x7f8f84161f30 2024-04-25T08:55:00.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.373+0000 7f8f837fe640 1 --2- 172.21.15.43:0/552960128 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8f7006b4d0 0x7f8f7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:00.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.374+0000 7f8f837fe640 1 --2- 172.21.15.43:0/552960128 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8f7006b4d0 0x7f8f7006d990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8f74034000 tx=0x7f8f7402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:00.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.377+0000 7f8f6bfff640 1 -- 172.21.15.43:0/552960128 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8f7c022030 con 0x7f8f84161f30 2024-04-25T08:55:00.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:00.570+0000 7f8f89b5c640 1 -- 172.21.15.43:0/552960128 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f8f84074ef0 con 0x7f8f84161f30 2024-04-25T08:55:01.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.189+0000 7f8f6bfff640 1 -- 172.21.15.43:0/552960128 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v27) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8f7c010040 con 0x7f8f84161f30 2024-04-25T08:55:01.179 INFO:teuthology.orchestra.run.smithi043.stderr: removed all blocklist entries 2024-04-25T08:55:01.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.192+0000 7f8f89b5c640 1 -- 172.21.15.43:0/552960128 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8f7006b4d0 msgr2=0x7f8f7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:01.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.192+0000 7f8f89b5c640 1 --2- 172.21.15.43:0/552960128 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8f7006b4d0 0x7f8f7006d990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8f74034000 tx=0x7f8f7402e000 comp rx=0 tx=0).stop 2024-04-25T08:55:01.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.192+0000 7f8f89b5c640 1 -- 172.21.15.43:0/552960128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f84161f30 msgr2=0x7f8f840cbb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:01.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.192+0000 7f8f89b5c640 1 --2- 172.21.15.43:0/552960128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f84161f30 0x7f8f840cbb40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c00efc0 tx=0x7f8f7c00c490 comp rx=0 tx=0).stop 2024-04-25T08:55:01.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.192+0000 7f8f89b5c640 1 -- 172.21.15.43:0/552960128 shutdown_connections 2024-04-25T08:55:01.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.192+0000 7f8f89b5c640 1 --2- 172.21.15.43:0/552960128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f84161f30 0x7f8f840cbb40 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:01.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.192+0000 7f8f89b5c640 1 --2- 172.21.15.43:0/552960128 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8f7006b4d0 0x7f8f7006d990 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:01.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.192+0000 7f8f89b5c640 1 --2- 172.21.15.43:0/552960128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f8415df50 0x7f8f8415ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:01.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.192+0000 7f8f89b5c640 1 --2- 172.21.15.43:0/552960128 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f840cf670 0x7f8f8414ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:01.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.192+0000 7f8f89b5c640 1 -- 172.21.15.43:0/552960128 >> 172.21.15.43:0/552960128 conn(0x7f8f840d3a30 msgr2=0x7f8f84153e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:01.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.192+0000 7f8f89b5c640 1 -- 172.21.15.43:0/552960128 shutdown_connections 2024-04-25T08:55:01.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.192+0000 7f8f89b5c640 1 -- 172.21.15.43:0/552960128 wait complete. 2024-04-25T08:55:01.196 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T08:55:01.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.313+0000 7f7529293640 1 Processor -- start 2024-04-25T08:55:01.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.313+0000 7f7529293640 1 -- start start 2024-04-25T08:55:01.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.314+0000 7f7529293640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7524161390 0x7f75240b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:01.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.314+0000 7f7529293640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f75240d02a0 0x7f75240d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:01.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.314+0000 7f7529293640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75240b4c30 con 0x7f75240ce5e0 2024-04-25T08:55:01.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.314+0000 7f7529293640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75240d1910 con 0x7f7524161390 2024-04-25T08:55:01.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.314+0000 7f7529293640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75240cf2c0 con 0x7f75240d02a0 2024-04-25T08:55:01.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.314+0000 7f7522ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7524161390 0x7f75240b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:01.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.314+0000 7f7522ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7524161390 0x7f75240b6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41904/0 (socket says 172.21.15.43:41904) 2024-04-25T08:55:01.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.314+0000 7f7522ffd640 1 -- 172.21.15.43:0/2378301682 learned_addr learned my addr 172.21.15.43:0/2378301682 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:01.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.314+0000 7f75237fe640 1 --1- 172.21.15.43:0/2378301682 >> v1:172.21.15.43:6789/0 conn(0x7f75240ce5e0 0x7f75240cea70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:55440/0 (socket says 172.21.15.43:55440) 2024-04-25T08:55:01.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.314+0000 7f7522ffd640 1 -- 172.21.15.43:0/2378301682 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f75240d02a0 msgr2=0x7f75240d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:01.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.314+0000 7f7522ffd640 1 --2- 172.21.15.43:0/2378301682 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f75240d02a0 0x7f75240d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:01.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.314+0000 7f7522ffd640 1 -- 172.21.15.43:0/2378301682 >> v1:172.21.15.43:6789/0 conn(0x7f75240ce5e0 legacy=0x7f75240cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:01.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.314+0000 7f7522ffd640 1 -- 172.21.15.43:0/2378301682 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75240cf5a0 con 0x7f7524161390 2024-04-25T08:55:01.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.314+0000 7f7521ffb640 1 -- 172.21.15.43:0/2378301682 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2964900834 0 0) 0x7f751c001490 con 0x7f75240ce5e0 2024-04-25T08:55:01.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.315+0000 7f7522ffd640 1 --2- 172.21.15.43:0/2378301682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7524161390 0x7f75240b6820 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7514009870 tx=0x7f751402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8082c2160f1fabb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:01.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.315+0000 7f7521ffb640 1 -- 172.21.15.43:0/2378301682 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7514037070 con 0x7f7524161390 2024-04-25T08:55:01.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.315+0000 7f7521ffb640 1 -- 172.21.15.43:0/2378301682 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7514032440 con 0x7f7524161390 2024-04-25T08:55:01.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.315+0000 7f7521ffb640 1 -- 172.21.15.43:0/2378301682 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7514032810 con 0x7f7524161390 2024-04-25T08:55:01.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.316+0000 7f7529293640 1 -- 172.21.15.43:0/2378301682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7524161390 msgr2=0x7f75240b6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:01.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.316+0000 7f7529293640 1 --2- 172.21.15.43:0/2378301682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7524161390 0x7f75240b6820 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7514009870 tx=0x7f751402a270 comp rx=0 tx=0).stop 2024-04-25T08:55:01.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.316+0000 7f7529293640 1 -- 172.21.15.43:0/2378301682 shutdown_connections 2024-04-25T08:55:01.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.316+0000 7f7529293640 1 --2- 172.21.15.43:0/2378301682 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f75240d02a0 0x7f75240d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:01.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.316+0000 7f7529293640 1 --2- 172.21.15.43:0/2378301682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7524161390 0x7f75240b6820 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:01.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.316+0000 7f7529293640 1 -- 172.21.15.43:0/2378301682 >> 172.21.15.43:0/2378301682 conn(0x7f752409cd20 msgr2=0x7f75240d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:01.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.316+0000 7f7529293640 1 -- 172.21.15.43:0/2378301682 shutdown_connections 2024-04-25T08:55:01.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.316+0000 7f7529293640 1 -- 172.21.15.43:0/2378301682 wait complete. 2024-04-25T08:55:01.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.317+0000 7f7529293640 1 Processor -- start 2024-04-25T08:55:01.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.317+0000 7f7529293640 1 -- start start 2024-04-25T08:55:01.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.318+0000 7f7529293640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75240ce5e0 0x7f75240cd780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:01.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.318+0000 7f7529293640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f75240d02a0 0x7f75240c5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:01.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.318+0000 7f7529293640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7524161390 0x7f75240c5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:01.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.318+0000 7f7529293640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7524161770 con 0x7f7524161390 2024-04-25T08:55:01.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.318+0000 7f7529293640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75240d15a0 con 0x7f75240ce5e0 2024-04-25T08:55:01.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.318+0000 7f7529293640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75240d1da0 con 0x7f75240d02a0 2024-04-25T08:55:01.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.318+0000 7f75237fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7524161390 0x7f75240c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:01.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.318+0000 7f75237fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7524161390 0x7f75240c5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41908/0 (socket says 172.21.15.43:41908) 2024-04-25T08:55:01.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.318+0000 7f75237fe640 1 -- 172.21.15.43:0/795684845 learned_addr learned my addr 172.21.15.43:0/795684845 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:01.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.318+0000 7f75227fc640 1 --2- 172.21.15.43:0/795684845 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f75240d02a0 0x7f75240c5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:01.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.318+0000 7f7522ffd640 1 --2- 172.21.15.43:0/795684845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75240ce5e0 0x7f75240cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:01.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.318+0000 7f75237fe640 1 -- 172.21.15.43:0/795684845 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f75240d02a0 msgr2=0x7f75240c5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:01.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.318+0000 7f75237fe640 1 --2- 172.21.15.43:0/795684845 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f75240d02a0 0x7f75240c5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:01.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.318+0000 7f75237fe640 1 -- 172.21.15.43:0/795684845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75240ce5e0 msgr2=0x7f75240cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:01.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.318+0000 7f75237fe640 1 --2- 172.21.15.43:0/795684845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75240ce5e0 0x7f75240cd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:01.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.318+0000 7f75237fe640 1 -- 172.21.15.43:0/795684845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75240cb8b0 con 0x7f7524161390 2024-04-25T08:55:01.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.319+0000 7f75237fe640 1 --2- 172.21.15.43:0/795684845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7524161390 0x7f75240c5540 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f751c00c7f0 tx=0x7f751c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:01.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.319+0000 7f750b7fe640 1 -- 172.21.15.43:0/795684845 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751c018070 con 0x7f7524161390 2024-04-25T08:55:01.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.319+0000 7f750b7fe640 1 -- 172.21.15.43:0/795684845 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f751c00a5e0 con 0x7f7524161390 2024-04-25T08:55:01.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.319+0000 7f750b7fe640 1 -- 172.21.15.43:0/795684845 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f751c00aa10 con 0x7f7524161390 2024-04-25T08:55:01.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.319+0000 7f7529293640 1 -- 172.21.15.43:0/795684845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75241f4db0 con 0x7f7524161390 2024-04-25T08:55:01.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.319+0000 7f7529293640 1 -- 172.21.15.43:0/795684845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75240cbb30 con 0x7f7524161390 2024-04-25T08:55:01.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.320+0000 7f7529293640 1 -- 172.21.15.43:0/795684845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75240680f0 con 0x7f7524161390 2024-04-25T08:55:01.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.320+0000 7f750b7fe640 1 -- 172.21.15.43:0/795684845 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f751c00abb0 con 0x7f7524161390 2024-04-25T08:55:01.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.321+0000 7f750b7fe640 1 --2- 172.21.15.43:0/795684845 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f751006b4d0 0x7f751006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:01.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.321+0000 7f750b7fe640 1 -- 172.21.15.43:0/795684845 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f751c0894f0 con 0x7f7524161390 2024-04-25T08:55:01.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.321+0000 7f7522ffd640 1 --2- 172.21.15.43:0/795684845 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f751006b4d0 0x7f751006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:01.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.322+0000 7f7522ffd640 1 --2- 172.21.15.43:0/795684845 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f751006b4d0 0x7f751006d990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7514034000 tx=0x7f751402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:01.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.325+0000 7f750b7fe640 1 -- 172.21.15.43:0/795684845 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f75240cb8b0 con 0x7f7524161390 2024-04-25T08:55:01.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.574+0000 7f7529293640 1 -- 172.21.15.43:0/795684845 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f75240cbd30 con 0x7f7524161390 2024-04-25T08:55:01.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.574+0000 7f750b7fe640 1 -- 172.21.15.43:0/795684845 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f75240cbd30 con 0x7f7524161390 2024-04-25T08:55:01.565 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:55:01.565 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T08:55:01.565 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_dump": [ 2024-04-25T08:55:01.565 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:55:01.565 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.a", 2024-04-25T08:55:01.565 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpmIjzEBRAAs2L10XrFO3mThjZevna5wA==", 2024-04-25T08:55:01.565 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:55:01.565 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T08:55:01.565 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T08:55:01.565 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T08:55:01.565 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:55:01.565 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:55:01.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:55:01.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:55:01.566 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.b", 2024-04-25T08:55:01.566 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm51cmERAAij4vy8xyihh8u/4dkmcllw==", 2024-04-25T08:55:01.566 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:55:01.566 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T08:55:01.566 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T08:55:01.566 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T08:55:01.566 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:55:01.566 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:55:01.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:55:01.566 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:55:01.566 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.c", 2024-04-25T08:55:01.566 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm1OOtDBAAlUYbCh1mBEEka9CWRnUz9A==", 2024-04-25T08:55:01.566 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:55:01.567 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T08:55:01.567 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T08:55:01.567 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T08:55:01.567 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:55:01.567 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:55:01.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:55:01.567 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:55:01.567 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.d", 2024-04-25T08:55:01.567 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpmpEvsFxAAXX1h37UWdFkyJ3Iw3xd8kw==", 2024-04-25T08:55:01.567 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:55:01.567 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T08:55:01.567 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T08:55:01.567 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T08:55:01.567 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:55:01.567 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:55:01.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:55:01.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:55:01.568 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.0", 2024-04-25T08:55:01.568 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD9GSpmi7qkORAAwSNgwzcVMitrPPf3nDyqvQ==", 2024-04-25T08:55:01.568 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:55:01.568 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T08:55:01.568 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T08:55:01.568 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:55:01.568 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:55:01.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:55:01.568 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:55:01.568 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.1", 2024-04-25T08:55:01.568 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAAGipmJDT3NxAAcWBCd9nt7Au+Xftjqcuc5w==", 2024-04-25T08:55:01.568 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:55:01.568 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T08:55:01.569 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T08:55:01.569 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:55:01.569 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:55:01.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:55:01.569 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:55:01.569 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.2", 2024-04-25T08:55:01.569 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQADGipm68kzNhAA0ZOnlr4edWDjx6RfoEHYIw==", 2024-04-25T08:55:01.569 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:55:01.569 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T08:55:01.569 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T08:55:01.569 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:55:01.570 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:55:01.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:55:01.570 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:55:01.570 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.3", 2024-04-25T08:55:01.570 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAGGipmT1AeNBAAoNPiEtkznSxxhVJ1EmZ1ZA==", 2024-04-25T08:55:01.570 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:55:01.570 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T08:55:01.570 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T08:55:01.570 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:55:01.571 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:55:01.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:55:01.571 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:55:01.571 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.4", 2024-04-25T08:55:01.571 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAMGipmBjHHNxAAH/SCzKHus8fKxxI3UN4wWQ==", 2024-04-25T08:55:01.571 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:55:01.571 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T08:55:01.571 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T08:55:01.571 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:55:01.571 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:55:01.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:55:01.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:55:01.572 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.5", 2024-04-25T08:55:01.572 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPGipm3FXKNRAAL97YYWONg4QHr4ajC5D7fA==", 2024-04-25T08:55:01.572 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:55:01.572 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T08:55:01.572 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T08:55:01.572 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:55:01.572 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:55:01.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:55:01.572 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:55:01.572 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.6", 2024-04-25T08:55:01.573 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQASGipmCwKaMxAAcM0p9Q2sbNQviGkwSLg8AA==", 2024-04-25T08:55:01.573 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:55:01.573 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T08:55:01.573 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T08:55:01.573 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:55:01.573 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:55:01.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:55:01.573 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:55:01.573 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.7", 2024-04-25T08:55:01.573 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAVGipmP7A2MRAAxnFJFosMcbZ6VDbgH8r6wA==", 2024-04-25T08:55:01.573 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:55:01.574 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T08:55:01.574 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T08:55:01.574 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:55:01.574 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:55:01.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:55:01.574 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:55:01.574 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.0", 2024-04-25T08:55:01.574 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpmVOjWGxAAFNmxeGi0h2D//pa6Ne3eQw==", 2024-04-25T08:55:01.574 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:55:01.574 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T08:55:01.574 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-04-25T08:55:01.574 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-04-25T08:55:01.574 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-04-25T08:55:01.575 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:55:01.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:55:01.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:55:01.575 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.1", 2024-04-25T08:55:01.575 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm24laJBAA9ijbDlFDTmF/ZZ4IhLfBVQ==", 2024-04-25T08:55:01.575 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:55:01.575 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T08:55:01.575 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-04-25T08:55:01.575 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-04-25T08:55:01.575 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-04-25T08:55:01.575 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:55:01.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:55:01.575 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:55:01.575 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.2", 2024-04-25T08:55:01.576 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm8b5UIRAACcOwDL630oJrXfp7s+fNTA==", 2024-04-25T08:55:01.576 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:55:01.576 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T08:55:01.576 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-04-25T08:55:01.576 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-04-25T08:55:01.576 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-04-25T08:55:01.576 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:55:01.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:55:01.576 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:55:01.576 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.3", 2024-04-25T08:55:01.576 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpmLsbBKRAAIKWtRdpuyCaW1LnKMWTMiw==", 2024-04-25T08:55:01.576 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:55:01.576 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T08:55:01.576 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-04-25T08:55:01.576 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-04-25T08:55:01.576 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-04-25T08:55:01.577 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:55:01.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:55:01.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:55:01.577 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.admin", 2024-04-25T08:55:01.577 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD7GSpmE9C8JRAAS263n6a4L2DXzMyS9o42NQ==", 2024-04-25T08:55:01.577 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:55:01.577 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-04-25T08:55:01.577 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T08:55:01.577 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T08:55:01.577 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:55:01.577 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:55:01.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:55:01.577 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:55:01.577 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.bootstrap-mds", 2024-04-25T08:55:01.577 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAbGipmDVInIhAAgvnM1bNCJoQAgTojRSwbIA==", 2024-04-25T08:55:01.578 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:55:01.578 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile bootstrap-mds" 2024-04-25T08:55:01.578 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:55:01.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:55:01.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:55:01.578 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.bootstrap-mgr", 2024-04-25T08:55:01.578 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAbGipmTFsnIhAAtkKI471sWJeSidsk3hv61w==", 2024-04-25T08:55:01.578 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:55:01.578 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-25T08:55:01.578 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:55:01.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:55:01.578 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:55:01.578 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.bootstrap-osd", 2024-04-25T08:55:01.579 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAbGipmq2InIhAAQh2VMOB9JPyL6q9PZO6HEQ==", 2024-04-25T08:55:01.579 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:55:01.579 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile bootstrap-osd" 2024-04-25T08:55:01.579 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:55:01.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:55:01.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:55:01.579 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.bootstrap-rbd", 2024-04-25T08:55:01.579 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAbGipm/mknIhAA3oYlei/iGcIoJWP9nq2H1g==", 2024-04-25T08:55:01.579 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:55:01.579 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-25T08:55:01.579 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:55:01.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:55:01.579 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:55:01.579 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-25T08:55:01.579 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAbGipmh3InIhAAdcB5Zkwo1LqtiP3REM38NQ==", 2024-04-25T08:55:01.580 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:55:01.580 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-25T08:55:01.580 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:55:01.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:55:01.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:55:01.580 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.bootstrap-rgw", 2024-04-25T08:55:01.580 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAbGipmLHonIhAA4i6pOKvg/aRi2QXGRWiZvw==", 2024-04-25T08:55:01.580 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:55:01.580 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-25T08:55:01.580 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:55:01.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:55:01.580 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:55:01.580 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.x", 2024-04-25T08:55:01.580 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm/uGhAhAAr0FaJ6SwZ8Su1MEqyZwNRA==", 2024-04-25T08:55:01.581 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:55:01.581 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-04-25T08:55:01.581 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2024-04-25T08:55:01.581 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:55:01.581 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:55:01.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:55:01.581 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:55:01.581 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.y", 2024-04-25T08:55:01.581 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD7GSpmPwwpOhAALeiQ/LKM2zgjE7yu16RQOg==", 2024-04-25T08:55:01.581 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:55:01.581 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-04-25T08:55:01.581 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2024-04-25T08:55:01.581 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:55:01.581 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:55:01.581 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:55:01.582 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-04-25T08:55:01.582 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T08:55:01.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.576+0000 7f7529293640 1 -- 172.21.15.43:0/795684845 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f751006b4d0 msgr2=0x7f751006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:01.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.576+0000 7f7529293640 1 --2- 172.21.15.43:0/795684845 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f751006b4d0 0x7f751006d990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7514034000 tx=0x7f751402e000 comp rx=0 tx=0).stop 2024-04-25T08:55:01.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.576+0000 7f7529293640 1 -- 172.21.15.43:0/795684845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7524161390 msgr2=0x7f75240c5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:01.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.576+0000 7f7529293640 1 --2- 172.21.15.43:0/795684845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7524161390 0x7f75240c5540 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f751c00c7f0 tx=0x7f751c00ccb0 comp rx=0 tx=0).stop 2024-04-25T08:55:01.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.576+0000 7f7529293640 1 -- 172.21.15.43:0/795684845 shutdown_connections 2024-04-25T08:55:01.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.576+0000 7f7529293640 1 --2- 172.21.15.43:0/795684845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7524161390 0x7f75240c5540 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:01.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.576+0000 7f7529293640 1 --2- 172.21.15.43:0/795684845 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f75240d02a0 0x7f75240c5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:01.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.576+0000 7f7529293640 1 --2- 172.21.15.43:0/795684845 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f751006b4d0 0x7f751006d990 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:01.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.576+0000 7f7529293640 1 --2- 172.21.15.43:0/795684845 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f75240ce5e0 0x7f75240cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:01.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.576+0000 7f7529293640 1 -- 172.21.15.43:0/795684845 >> 172.21.15.43:0/795684845 conn(0x7f752409cd20 msgr2=0x7f75240b77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:01.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.577+0000 7f7529293640 1 -- 172.21.15.43:0/795684845 shutdown_connections 2024-04-25T08:55:01.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.577+0000 7f7529293640 1 -- 172.21.15.43:0/795684845 wait complete. 2024-04-25T08:55:01.584 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-25T08:55:01.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.696+0000 7ff3f0f9d640 1 Processor -- start 2024-04-25T08:55:01.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.696+0000 7ff3f0f9d640 1 -- start start 2024-04-25T08:55:01.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.697+0000 7ff3f0f9d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff3ec0cf250 0x7ff3ec0ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:01.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.697+0000 7ff3f0f9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff3ec06b240 0x7ff3ec0ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:01.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.697+0000 7ff3f0f9d640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3ec0cf6e0 con 0x7ff3ec15ded0 2024-04-25T08:55:01.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.697+0000 7ff3f0f9d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3ec15e2b0 con 0x7ff3ec06b240 2024-04-25T08:55:01.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.697+0000 7ff3f0f9d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec0cf9c0 con 0x7ff3ec0cf250 2024-04-25T08:55:01.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.697+0000 7ff3ead9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff3ec06b240 0x7ff3ec0ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:01.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.697+0000 7ff3ead9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff3ec06b240 0x7ff3ec0ad8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41920/0 (socket says 172.21.15.43:41920) 2024-04-25T08:55:01.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.697+0000 7ff3ead9d640 1 -- 172.21.15.43:0/234136811 learned_addr learned my addr 172.21.15.43:0/234136811 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:01.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.698+0000 7ff3e9d9b640 1 --2- 172.21.15.43:0/234136811 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff3ec0cf250 0x7ff3ec0ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:01.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.698+0000 7ff3ead9d640 1 -- 172.21.15.43:0/234136811 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff3ec0cf250 msgr2=0x7ff3ec0ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:01.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.698+0000 7ff3ead9d640 1 --2- 172.21.15.43:0/234136811 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff3ec0cf250 0x7ff3ec0ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:01.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.698+0000 7ff3ead9d640 1 -- 172.21.15.43:0/234136811 >> v1:172.21.15.193:6789/0 conn(0x7ff3ec15ded0 legacy=0x7ff3ec06ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:01.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.698+0000 7ff3ead9d640 1 -- 172.21.15.43:0/234136811 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ec0cfca0 con 0x7ff3ec06b240 2024-04-25T08:55:01.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.698+0000 7ff3ead9d640 1 --2- 172.21.15.43:0/234136811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3ec06b240 0x7ff3ec0ad8e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc009e50 tx=0x7ff3dc02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2561c12dd453ebc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:01.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.698+0000 7ff3e959a640 1 -- 172.21.15.43:0/234136811 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc036070 con 0x7ff3ec06b240 2024-04-25T08:55:01.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.699+0000 7ff3e959a640 1 -- 172.21.15.43:0/234136811 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3dc031600 con 0x7ff3ec06b240 2024-04-25T08:55:01.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.699+0000 7ff3e959a640 1 -- 172.21.15.43:0/234136811 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc0319d0 con 0x7ff3ec06b240 2024-04-25T08:55:01.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.699+0000 7ff3f0f9d640 1 -- 172.21.15.43:0/234136811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3ec06b240 msgr2=0x7ff3ec0ad8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:01.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.699+0000 7ff3f0f9d640 1 --2- 172.21.15.43:0/234136811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3ec06b240 0x7ff3ec0ad8e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc009e50 tx=0x7ff3dc02a710 comp rx=0 tx=0).stop 2024-04-25T08:55:01.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.699+0000 7ff3f0f9d640 1 -- 172.21.15.43:0/234136811 shutdown_connections 2024-04-25T08:55:01.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.699+0000 7ff3f0f9d640 1 --2- 172.21.15.43:0/234136811 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3ec06b240 0x7ff3ec0ad8e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:01.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.699+0000 7ff3f0f9d640 1 --2- 172.21.15.43:0/234136811 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff3ec0cf250 0x7ff3ec0ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:01.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.699+0000 7ff3f0f9d640 1 -- 172.21.15.43:0/234136811 >> 172.21.15.43:0/234136811 conn(0x7ff3ec0b2490 msgr2=0x7ff3ec0b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:01.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.699+0000 7ff3f0f9d640 1 -- 172.21.15.43:0/234136811 shutdown_connections 2024-04-25T08:55:01.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.699+0000 7ff3f0f9d640 1 -- 172.21.15.43:0/234136811 wait complete. 2024-04-25T08:55:01.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.700+0000 7ff3f0f9d640 1 Processor -- start 2024-04-25T08:55:01.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.701+0000 7ff3f0f9d640 1 -- start start 2024-04-25T08:55:01.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.701+0000 7ff3f0f9d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3ec06b240 0x7ff3ec15e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:01.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.701+0000 7ff3f0f9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3ec15ded0 0x7ff3ec1f4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:01.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.701+0000 7ff3f0f9d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff3ec1605e0 0x7ff3ec15b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:01.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.701+0000 7ff3f0f9d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec0cefd0 con 0x7ff3ec15ded0 2024-04-25T08:55:01.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.701+0000 7ff3f0f9d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec0cee50 con 0x7ff3ec06b240 2024-04-25T08:55:01.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.701+0000 7ff3f0f9d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec0b4db0 con 0x7ff3ec1605e0 2024-04-25T08:55:01.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.701+0000 7ff3e9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3ec15ded0 0x7ff3ec1f4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:01.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.701+0000 7ff3e9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3ec15ded0 0x7ff3ec1f4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41936/0 (socket says 172.21.15.43:41936) 2024-04-25T08:55:01.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.701+0000 7ff3e9d9b640 1 -- 172.21.15.43:0/4642769 learned_addr learned my addr 172.21.15.43:0/4642769 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:01.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.702+0000 7ff3e9d9b640 1 -- 172.21.15.43:0/4642769 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff3ec1605e0 msgr2=0x7ff3ec15b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:01.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.702+0000 7ff3e9d9b640 1 --2- 172.21.15.43:0/4642769 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff3ec1605e0 0x7ff3ec15b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:01.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.702+0000 7ff3e9d9b640 1 -- 172.21.15.43:0/4642769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3ec06b240 msgr2=0x7ff3ec15e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:01.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.702+0000 7ff3ea59c640 1 --2- 172.21.15.43:0/4642769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3ec06b240 0x7ff3ec15e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:01.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.702+0000 7ff3e9d9b640 1 --2- 172.21.15.43:0/4642769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3ec06b240 0x7ff3ec15e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:01.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.702+0000 7ff3e9d9b640 1 -- 172.21.15.43:0/4642769 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ec160dd0 con 0x7ff3ec15ded0 2024-04-25T08:55:01.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.702+0000 7ff3ea59c640 1 --2- 172.21.15.43:0/4642769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3ec06b240 0x7ff3ec15e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:01.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.702+0000 7ff3e9d9b640 1 --2- 172.21.15.43:0/4642769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3ec15ded0 0x7ff3ec1f4480 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff3d400d800 tx=0x7ff3d400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:01.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.702+0000 7ff3e2ffd640 1 -- 172.21.15.43:0/4642769 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d400b6c0 con 0x7ff3ec15ded0 2024-04-25T08:55:01.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.702+0000 7ff3e2ffd640 1 -- 172.21.15.43:0/4642769 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d4019070 con 0x7ff3ec15ded0 2024-04-25T08:55:01.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.702+0000 7ff3f0f9d640 1 -- 172.21.15.43:0/4642769 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3ec1610b0 con 0x7ff3ec15ded0 2024-04-25T08:55:01.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.702+0000 7ff3e2ffd640 1 -- 172.21.15.43:0/4642769 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d400bac0 con 0x7ff3ec15ded0 2024-04-25T08:55:01.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.702+0000 7ff3f0f9d640 1 -- 172.21.15.43:0/4642769 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3ec160a90 con 0x7ff3ec15ded0 2024-04-25T08:55:01.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.703+0000 7ff3f0f9d640 1 -- 172.21.15.43:0/4642769 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3ec074ef0 con 0x7ff3ec15ded0 2024-04-25T08:55:01.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.703+0000 7ff3e2ffd640 1 -- 172.21.15.43:0/4642769 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff3d400bc60 con 0x7ff3ec15ded0 2024-04-25T08:55:01.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.704+0000 7ff3e2ffd640 1 --2- 172.21.15.43:0/4642769 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff3cc06b500 0x7ff3cc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:01.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.704+0000 7ff3e2ffd640 1 -- 172.21.15.43:0/4642769 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff3d408a2c0 con 0x7ff3ec15ded0 2024-04-25T08:55:01.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.704+0000 7ff3ea59c640 1 --2- 172.21.15.43:0/4642769 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff3cc06b500 0x7ff3cc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:01.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.705+0000 7ff3ea59c640 1 --2- 172.21.15.43:0/4642769 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff3cc06b500 0x7ff3cc06d9c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff3e40015e0 tx=0x7ff3e4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:01.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.708+0000 7ff3e2ffd640 1 -- 172.21.15.43:0/4642769 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff3d4059520 con 0x7ff3ec15ded0 2024-04-25T08:55:01.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.963+0000 7ff3f0f9d640 1 -- 172.21.15.43:0/4642769 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7ff3ec15d6d0 con 0x7ff3ec15ded0 2024-04-25T08:55:01.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.965+0000 7ff3e2ffd640 1 -- 172.21.15.43:0/4642769 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7ff3d4058e20 con 0x7ff3ec15ded0 2024-04-25T08:55:01.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.967+0000 7ff3f0f9d640 1 -- 172.21.15.43:0/4642769 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff3cc06b500 msgr2=0x7ff3cc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:01.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.967+0000 7ff3f0f9d640 1 --2- 172.21.15.43:0/4642769 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff3cc06b500 0x7ff3cc06d9c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff3e40015e0 tx=0x7ff3e4009040 comp rx=0 tx=0).stop 2024-04-25T08:55:01.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.967+0000 7ff3f0f9d640 1 -- 172.21.15.43:0/4642769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3ec15ded0 msgr2=0x7ff3ec1f4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:01.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.967+0000 7ff3f0f9d640 1 --2- 172.21.15.43:0/4642769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3ec15ded0 0x7ff3ec1f4480 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff3d400d800 tx=0x7ff3d400dcc0 comp rx=0 tx=0).stop 2024-04-25T08:55:01.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.967+0000 7ff3f0f9d640 1 -- 172.21.15.43:0/4642769 shutdown_connections 2024-04-25T08:55:01.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.967+0000 7ff3f0f9d640 1 --2- 172.21.15.43:0/4642769 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff3ec1605e0 0x7ff3ec15b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:01.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.967+0000 7ff3f0f9d640 1 --2- 172.21.15.43:0/4642769 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3ec15ded0 0x7ff3ec1f4480 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:01.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.967+0000 7ff3f0f9d640 1 --2- 172.21.15.43:0/4642769 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff3cc06b500 0x7ff3cc06d9c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:01.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.967+0000 7ff3f0f9d640 1 --2- 172.21.15.43:0/4642769 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff3ec06b240 0x7ff3ec15e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:01.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.967+0000 7ff3f0f9d640 1 -- 172.21.15.43:0/4642769 >> 172.21.15.43:0/4642769 conn(0x7ff3ec0b2490 msgr2=0x7ff3ec15beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:01.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.967+0000 7ff3f0f9d640 1 -- 172.21.15.43:0/4642769 shutdown_connections 2024-04-25T08:55:01.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:01.967+0000 7ff3f0f9d640 1 -- 172.21.15.43:0/4642769 wait complete. 2024-04-25T08:55:01.968 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-25T08:55:02.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.079+0000 7f075b9e7640 1 Processor -- start 2024-04-25T08:55:02.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.080+0000 7f075b9e7640 1 -- start start 2024-04-25T08:55:02.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.080+0000 7f075b9e7640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f07540d0d90 0x7f07540ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:02.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.080+0000 7f075b9e7640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f07540d2500 0x7f07540d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:02.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.080+0000 7f075b9e7640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07540b6c50 con 0x7f07540d39d0 2024-04-25T08:55:02.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.080+0000 7f075b9e7640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f075415e010 con 0x7f07540d0d90 2024-04-25T08:55:02.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.080+0000 7f075b9e7640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07540d2020 con 0x7f07540d2500 2024-04-25T08:55:02.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.081+0000 7f0758f82640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f07540d0d90 0x7f07540ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:02.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.081+0000 7f0758f82640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f07540d0d90 0x7f07540ce2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41940/0 (socket says 172.21.15.43:41940) 2024-04-25T08:55:02.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.081+0000 7f0758f82640 1 -- 172.21.15.43:0/2442776439 learned_addr learned my addr 172.21.15.43:0/2442776439 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:02.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.081+0000 7f0759f84640 1 --2- 172.21.15.43:0/2442776439 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f07540d2500 0x7f07540d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:02.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.081+0000 7f0758f82640 1 -- 172.21.15.43:0/2442776439 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f07540d2500 msgr2=0x7f07540d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:02.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.081+0000 7f0758f82640 1 --2- 172.21.15.43:0/2442776439 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f07540d2500 0x7f07540d2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:02.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.081+0000 7f0758f82640 1 -- 172.21.15.43:0/2442776439 >> v1:172.21.15.193:6789/0 conn(0x7f07540d39d0 legacy=0x7f07540d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:02.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.081+0000 7f0758f82640 1 -- 172.21.15.43:0/2442776439 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07540cec40 con 0x7f07540d0d90 2024-04-25T08:55:02.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.082+0000 7f0758f82640 1 --2- 172.21.15.43:0/2442776439 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07540d0d90 0x7f07540ce2a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0744009870 tx=0x7f074402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c0a197b85f757ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:02.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.082+0000 7f074bfff640 1 -- 172.21.15.43:0/2442776439 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0744037070 con 0x7f07540d0d90 2024-04-25T08:55:02.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.082+0000 7f074bfff640 1 -- 172.21.15.43:0/2442776439 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0744032440 con 0x7f07540d0d90 2024-04-25T08:55:02.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.082+0000 7f074bfff640 1 -- 172.21.15.43:0/2442776439 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0744032810 con 0x7f07540d0d90 2024-04-25T08:55:02.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.082+0000 7f075b9e7640 1 -- 172.21.15.43:0/2442776439 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07540d0d90 msgr2=0x7f07540ce2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:02.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.082+0000 7f075b9e7640 1 --2- 172.21.15.43:0/2442776439 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07540d0d90 0x7f07540ce2a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0744009870 tx=0x7f074402a1c0 comp rx=0 tx=0).stop 2024-04-25T08:55:02.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.082+0000 7f075b9e7640 1 -- 172.21.15.43:0/2442776439 shutdown_connections 2024-04-25T08:55:02.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.082+0000 7f075b9e7640 1 --2- 172.21.15.43:0/2442776439 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f07540d2500 0x7f07540d2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:02.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.082+0000 7f075b9e7640 1 --2- 172.21.15.43:0/2442776439 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07540d0d90 0x7f07540ce2a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:02.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.082+0000 7f075b9e7640 1 -- 172.21.15.43:0/2442776439 >> 172.21.15.43:0/2442776439 conn(0x7f075406b000 msgr2=0x7f075406b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:02.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.082+0000 7f075b9e7640 1 -- 172.21.15.43:0/2442776439 shutdown_connections 2024-04-25T08:55:02.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.082+0000 7f075b9e7640 1 -- 172.21.15.43:0/2442776439 wait complete. 2024-04-25T08:55:02.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.084+0000 7f075b9e7640 1 Processor -- start 2024-04-25T08:55:02.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.084+0000 7f075b9e7640 1 -- start start 2024-04-25T08:55:02.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.084+0000 7f075b9e7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07540d0d90 0x7f07540c9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:02.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.084+0000 7f075b9e7640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f07540d2500 0x7f075414ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:02.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.084+0000 7f075b9e7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07540d39d0 0x7f07540c7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:02.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.084+0000 7f075b9e7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07540cf200 con 0x7f07540d39d0 2024-04-25T08:55:02.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.084+0000 7f075b9e7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07540b2f00 con 0x7f07540d0d90 2024-04-25T08:55:02.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.084+0000 7f075b9e7640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07540d2ed0 con 0x7f07540d2500 2024-04-25T08:55:02.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.084+0000 7f0759f84640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07540d39d0 0x7f07540c7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:02.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.085+0000 7f0759f84640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07540d39d0 0x7f07540c7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41946/0 (socket says 172.21.15.43:41946) 2024-04-25T08:55:02.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.085+0000 7f0759f84640 1 -- 172.21.15.43:0/1519127787 learned_addr learned my addr 172.21.15.43:0/1519127787 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:02.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.085+0000 7f0758f82640 1 --2- 172.21.15.43:0/1519127787 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f07540d2500 0x7f075414ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:02.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.085+0000 7f0759783640 1 --2- 172.21.15.43:0/1519127787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07540d0d90 0x7f07540c9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:02.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.085+0000 7f0759f84640 1 -- 172.21.15.43:0/1519127787 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f07540d2500 msgr2=0x7f075414ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:02.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.085+0000 7f0759f84640 1 --2- 172.21.15.43:0/1519127787 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f07540d2500 0x7f075414ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:02.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.085+0000 7f0759f84640 1 -- 172.21.15.43:0/1519127787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07540d0d90 msgr2=0x7f07540c9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:02.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.085+0000 7f0759f84640 1 --2- 172.21.15.43:0/1519127787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07540d0d90 0x7f07540c9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:02.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.085+0000 7f0759f84640 1 -- 172.21.15.43:0/1519127787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07540b3120 con 0x7f07540d39d0 2024-04-25T08:55:02.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.085+0000 7f0759f84640 1 --2- 172.21.15.43:0/1519127787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07540d39d0 0x7f07540c7480 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f074c007e40 tx=0x7f074c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:02.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.085+0000 7f0749ffb640 1 -- 172.21.15.43:0/1519127787 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074c014620 con 0x7f07540d39d0 2024-04-25T08:55:02.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.085+0000 7f0749ffb640 1 -- 172.21.15.43:0/1519127787 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f074c019070 con 0x7f07540d39d0 2024-04-25T08:55:02.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.085+0000 7f075b9e7640 1 -- 172.21.15.43:0/1519127787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07540b3400 con 0x7f07540d39d0 2024-04-25T08:55:02.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.085+0000 7f0749ffb640 1 -- 172.21.15.43:0/1519127787 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f074c014a20 con 0x7f07540d39d0 2024-04-25T08:55:02.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.085+0000 7f075b9e7640 1 -- 172.21.15.43:0/1519127787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07541504e0 con 0x7f07540d39d0 2024-04-25T08:55:02.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.086+0000 7f075b9e7640 1 -- 172.21.15.43:0/1519127787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0754074ef0 con 0x7f07540d39d0 2024-04-25T08:55:02.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.087+0000 7f0749ffb640 1 -- 172.21.15.43:0/1519127787 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f074c014bc0 con 0x7f07540d39d0 2024-04-25T08:55:02.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.087+0000 7f0749ffb640 1 --2- 172.21.15.43:0/1519127787 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f073806b4d0 0x7f073806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:02.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.087+0000 7f0749ffb640 1 -- 172.21.15.43:0/1519127787 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f074c08a6e0 con 0x7f07540d39d0 2024-04-25T08:55:02.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.087+0000 7f0759783640 1 --2- 172.21.15.43:0/1519127787 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f073806b4d0 0x7f073806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:02.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.088+0000 7f0759783640 1 --2- 172.21.15.43:0/1519127787 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f073806b4d0 0x7f073806d990 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0750007fd0 tx=0x7f0750009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:02.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.091+0000 7f0749ffb640 1 -- 172.21.15.43:0/1519127787 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f074c0598b0 con 0x7f07540d39d0 2024-04-25T08:55:02.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.342+0000 7f075b9e7640 1 -- 172.21.15.43:0/1519127787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f07540b5e90 con 0x7f07540d39d0 2024-04-25T08:55:02.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.344+0000 7f0749ffb640 1 -- 172.21.15.43:0/1519127787 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f074c0591b0 con 0x7f07540d39d0 2024-04-25T08:55:02.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.346+0000 7f075b9e7640 1 -- 172.21.15.43:0/1519127787 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f073806b4d0 msgr2=0x7f073806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:02.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.346+0000 7f075b9e7640 1 --2- 172.21.15.43:0/1519127787 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f073806b4d0 0x7f073806d990 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0750007fd0 tx=0x7f0750009040 comp rx=0 tx=0).stop 2024-04-25T08:55:02.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.346+0000 7f075b9e7640 1 -- 172.21.15.43:0/1519127787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07540d39d0 msgr2=0x7f07540c7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:02.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.346+0000 7f075b9e7640 1 --2- 172.21.15.43:0/1519127787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07540d39d0 0x7f07540c7480 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f074c007e40 tx=0x7f074c00e4b0 comp rx=0 tx=0).stop 2024-04-25T08:55:02.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.346+0000 7f075b9e7640 1 -- 172.21.15.43:0/1519127787 shutdown_connections 2024-04-25T08:55:02.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.346+0000 7f075b9e7640 1 --2- 172.21.15.43:0/1519127787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f07540d39d0 0x7f07540c7480 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:02.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.346+0000 7f075b9e7640 1 --2- 172.21.15.43:0/1519127787 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f07540d2500 0x7f075414ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:02.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.346+0000 7f075b9e7640 1 --2- 172.21.15.43:0/1519127787 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f073806b4d0 0x7f073806d990 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:02.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.346+0000 7f075b9e7640 1 --2- 172.21.15.43:0/1519127787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f07540d0d90 0x7f07540c9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:02.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.346+0000 7f075b9e7640 1 -- 172.21.15.43:0/1519127787 >> 172.21.15.43:0/1519127787 conn(0x7f075406b000 msgr2=0x7f075415e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:02.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.346+0000 7f075b9e7640 1 -- 172.21.15.43:0/1519127787 shutdown_connections 2024-04-25T08:55:02.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.346+0000 7f075b9e7640 1 -- 172.21.15.43:0/1519127787 wait complete. 2024-04-25T08:55:02.347 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-25T08:55:02.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.458+0000 7f99469b7640 1 Processor -- start 2024-04-25T08:55:02.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.459+0000 7f99469b7640 1 -- start start 2024-04-25T08:55:02.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.459+0000 7f99469b7640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f99400cff70 0x7f99400cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:02.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.459+0000 7f99469b7640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f99400b23f0 0x7f99400ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:02.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.459+0000 7f99469b7640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99400ccf10 con 0x7f994015f180 2024-04-25T08:55:02.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.459+0000 7f99469b7640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99400cccc0 con 0x7f99400b23f0 2024-04-25T08:55:02.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.459+0000 7f99469b7640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99400cd840 con 0x7f99400cff70 2024-04-25T08:55:02.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.459+0000 7f993ffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f994015f180 0x7f99400cf860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:55456/0 (socket says 172.21.15.43:55456) 2024-04-25T08:55:02.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.459+0000 7f993ffff640 1 -- 172.21.15.43:0/1982207112 learned_addr learned my addr 172.21.15.43:0/1982207112 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:55:02.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.459+0000 7f9944f54640 1 --2- 172.21.15.43:0/1982207112 >> v2:172.21.15.43:3300/0 conn(0x7f99400b23f0 0x7f99400ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:02.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.460+0000 7f993f7fe640 1 --2- 172.21.15.43:0/1982207112 >> v2:172.21.15.193:3300/0 conn(0x7f99400cff70 0x7f99400cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:02.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.460+0000 7f9944f54640 1 -- 172.21.15.43:0/1982207112 >> v2:172.21.15.193:3300/0 conn(0x7f99400cff70 msgr2=0x7f99400cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:02.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.460+0000 7f9944f54640 1 --2- 172.21.15.43:0/1982207112 >> v2:172.21.15.193:3300/0 conn(0x7f99400cff70 0x7f99400cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:02.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.460+0000 7f9944f54640 1 -- 172.21.15.43:0/1982207112 >> v1:172.21.15.43:6789/0 conn(0x7f994015f180 legacy=0x7f99400cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:02.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.460+0000 7f993effd640 1 -- 172.21.15.43:0/1982207112 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 598299608 0 0) 0x7f99400ccf10 con 0x7f994015f180 2024-04-25T08:55:02.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.460+0000 7f9944f54640 1 -- 172.21.15.43:0/1982207112 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99400cd1f0 con 0x7f99400b23f0 2024-04-25T08:55:02.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.460+0000 7f993f7fe640 1 --2- 172.21.15.43:0/1982207112 >> v2:172.21.15.193:3300/0 conn(0x7f99400cff70 0x7f99400cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:02.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.460+0000 7f9944f54640 1 --2- 172.21.15.43:0/1982207112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99400b23f0 0x7f99400ce210 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9940071a40 tx=0x7f993802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92f4baa9b645556d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:02.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.461+0000 7f993effd640 1 -- 172.21.15.43:0/1982207112 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9938007740 con 0x7f99400b23f0 2024-04-25T08:55:02.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.461+0000 7f993effd640 1 -- 172.21.15.43:0/1982207112 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99380078e0 con 0x7f99400b23f0 2024-04-25T08:55:02.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.461+0000 7f993effd640 1 -- 172.21.15.43:0/1982207112 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9938007cb0 con 0x7f99400b23f0 2024-04-25T08:55:02.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.461+0000 7f99469b7640 1 -- 172.21.15.43:0/1982207112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99400b23f0 msgr2=0x7f99400ce210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:02.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.461+0000 7f99469b7640 1 --2- 172.21.15.43:0/1982207112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99400b23f0 0x7f99400ce210 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9940071a40 tx=0x7f993802a5f0 comp rx=0 tx=0).stop 2024-04-25T08:55:02.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.461+0000 7f99469b7640 1 -- 172.21.15.43:0/1982207112 shutdown_connections 2024-04-25T08:55:02.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.461+0000 7f99469b7640 1 --2- 172.21.15.43:0/1982207112 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99400b23f0 0x7f99400ce210 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:02.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.461+0000 7f99469b7640 1 --2- 172.21.15.43:0/1982207112 >> v2:172.21.15.193:3300/0 conn(0x7f99400cff70 0x7f99400cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:02.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.461+0000 7f99469b7640 1 -- 172.21.15.43:0/1982207112 >> 172.21.15.43:0/1982207112 conn(0x7f99400d17f0 msgr2=0x7f99400d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:02.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.461+0000 7f99469b7640 1 -- 172.21.15.43:0/1982207112 shutdown_connections 2024-04-25T08:55:02.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.461+0000 7f99469b7640 1 -- 172.21.15.43:0/1982207112 wait complete. 2024-04-25T08:55:02.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.462+0000 7f99469b7640 1 Processor -- start 2024-04-25T08:55:02.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.463+0000 7f99469b7640 1 -- start start 2024-04-25T08:55:02.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.463+0000 7f99469b7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99400b23f0 0x7f99400cad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:02.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.463+0000 7f99469b7640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f99400cff70 0x7f99401f1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:02.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.463+0000 7f99469b7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f994015f180 0x7f994014f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:02.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.463+0000 7f99469b7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994015f560 con 0x7f99400b23f0 2024-04-25T08:55:02.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.463+0000 7f99469b7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99400d23f0 con 0x7f994015f180 2024-04-25T08:55:02.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.463+0000 7f99469b7640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f994015cd80 con 0x7f99400cff70 2024-04-25T08:55:02.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.463+0000 7f993ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99400b23f0 0x7f99400cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:02.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.463+0000 7f993ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99400b23f0 0x7f99400cad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41962/0 (socket says 172.21.15.43:41962) 2024-04-25T08:55:02.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.463+0000 7f993ffff640 1 -- 172.21.15.43:0/2546843639 learned_addr learned my addr 172.21.15.43:0/2546843639 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:02.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.464+0000 7f993f7fe640 1 --2- 172.21.15.43:0/2546843639 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f99400cff70 0x7f99401f1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:02.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.464+0000 7f9944f54640 1 --2- 172.21.15.43:0/2546843639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f994015f180 0x7f994014f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:02.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.464+0000 7f993ffff640 1 -- 172.21.15.43:0/2546843639 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f99400cff70 msgr2=0x7f99401f1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:02.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.464+0000 7f993ffff640 1 --2- 172.21.15.43:0/2546843639 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f99400cff70 0x7f99401f1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:02.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.464+0000 7f993ffff640 1 -- 172.21.15.43:0/2546843639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f994015f180 msgr2=0x7f994014f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:02.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.464+0000 7f993ffff640 1 --2- 172.21.15.43:0/2546843639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f994015f180 0x7f994014f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:02.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.464+0000 7f993ffff640 1 -- 172.21.15.43:0/2546843639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f994015c170 con 0x7f99400b23f0 2024-04-25T08:55:02.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.464+0000 7f993ffff640 1 --2- 172.21.15.43:0/2546843639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99400b23f0 0x7f99400cad00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f993400c430 tx=0x7f993400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:02.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.464+0000 7f993cff9640 1 -- 172.21.15.43:0/2546843639 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99340074e0 con 0x7f99400b23f0 2024-04-25T08:55:02.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.464+0000 7f993cff9640 1 -- 172.21.15.43:0/2546843639 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99340047e0 con 0x7f99400b23f0 2024-04-25T08:55:02.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.464+0000 7f99469b7640 1 -- 172.21.15.43:0/2546843639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f994015c4b0 con 0x7f99400b23f0 2024-04-25T08:55:02.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.464+0000 7f993cff9640 1 -- 172.21.15.43:0/2546843639 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9934004bd0 con 0x7f99400b23f0 2024-04-25T08:55:02.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.464+0000 7f99469b7640 1 -- 172.21.15.43:0/2546843639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99400cb240 con 0x7f99400b23f0 2024-04-25T08:55:02.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.465+0000 7f99469b7640 1 -- 172.21.15.43:0/2546843639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99400b2cc0 con 0x7f99400b23f0 2024-04-25T08:55:02.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.466+0000 7f993cff9640 1 -- 172.21.15.43:0/2546843639 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9934011070 con 0x7f99400b23f0 2024-04-25T08:55:02.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.466+0000 7f993cff9640 1 --2- 172.21.15.43:0/2546843639 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f992c06b4d0 0x7f992c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:02.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.466+0000 7f993cff9640 1 -- 172.21.15.43:0/2546843639 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9934089950 con 0x7f99400b23f0 2024-04-25T08:55:02.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.466+0000 7f993f7fe640 1 --2- 172.21.15.43:0/2546843639 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f992c06b4d0 0x7f992c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:02.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.467+0000 7f993f7fe640 1 --2- 172.21.15.43:0/2546843639 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f992c06b4d0 0x7f992c06d990 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f994015a470 tx=0x7f9930008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:02.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.471+0000 7f993cff9640 1 -- 172.21.15.43:0/2546843639 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f99400b2cc0 con 0x7f99400b23f0 2024-04-25T08:55:02.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.729+0000 7f99469b7640 1 -- 172.21.15.43:0/2546843639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f9940074ef0 con 0x7f99400b23f0 2024-04-25T08:55:02.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.731+0000 7f993cff9640 1 -- 172.21.15.43:0/2546843639 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f994015c4b0 con 0x7f99400b23f0 2024-04-25T08:55:02.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.734+0000 7f99469b7640 1 -- 172.21.15.43:0/2546843639 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f992c06b4d0 msgr2=0x7f992c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:02.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.734+0000 7f99469b7640 1 --2- 172.21.15.43:0/2546843639 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f992c06b4d0 0x7f992c06d990 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f994015a470 tx=0x7f9930008040 comp rx=0 tx=0).stop 2024-04-25T08:55:02.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.734+0000 7f99469b7640 1 -- 172.21.15.43:0/2546843639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99400b23f0 msgr2=0x7f99400cad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:02.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.734+0000 7f99469b7640 1 --2- 172.21.15.43:0/2546843639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99400b23f0 0x7f99400cad00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f993400c430 tx=0x7f993400c8f0 comp rx=0 tx=0).stop 2024-04-25T08:55:02.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.734+0000 7f99469b7640 1 -- 172.21.15.43:0/2546843639 shutdown_connections 2024-04-25T08:55:02.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.734+0000 7f99469b7640 1 --2- 172.21.15.43:0/2546843639 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f992c06b4d0 0x7f992c06d990 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:02.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.734+0000 7f99469b7640 1 --2- 172.21.15.43:0/2546843639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f994015f180 0x7f994014f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:02.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.734+0000 7f99469b7640 1 --2- 172.21.15.43:0/2546843639 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f99400cff70 0x7f99401f1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:02.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.734+0000 7f99469b7640 1 --2- 172.21.15.43:0/2546843639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99400b23f0 0x7f99400cad00 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:02.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.734+0000 7f99469b7640 1 -- 172.21.15.43:0/2546843639 >> 172.21.15.43:0/2546843639 conn(0x7f99400d17f0 msgr2=0x7f99400cf3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:02.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.734+0000 7f99469b7640 1 -- 172.21.15.43:0/2546843639 shutdown_connections 2024-04-25T08:55:02.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.734+0000 7f99469b7640 1 -- 172.21.15.43:0/2546843639 wait complete. 2024-04-25T08:55:02.734 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-25T08:55:02.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.846+0000 7f1343dc1640 1 Processor -- start 2024-04-25T08:55:02.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.846+0000 7f1343dc1640 1 -- start start 2024-04-25T08:55:02.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.847+0000 7f1343dc1640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f133c15a9d0 0x7f133c06d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:02.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.847+0000 7f1343dc1640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f133c06d7d0 0x7f133c071a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:02.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.847+0000 7f1343dc1640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f133c071f90 0x7f133c0b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:02.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.847+0000 7f1343dc1640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f133c06dc30 con 0x7f133c06d7d0 2024-04-25T08:55:02.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.847+0000 7f1343dc1640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f133c072420 con 0x7f133c071f90 2024-04-25T08:55:02.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.847+0000 7f1343dc1640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f133c0704f0 con 0x7f133c15a9d0 2024-04-25T08:55:02.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.847+0000 7f134135c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f133c06d7d0 0x7f133c071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:02.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.847+0000 7f134135c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f133c06d7d0 0x7f133c071a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41968/0 (socket says 172.21.15.43:41968) 2024-04-25T08:55:02.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.847+0000 7f134135c640 1 -- 172.21.15.43:0/1678328085 learned_addr learned my addr 172.21.15.43:0/1678328085 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:02.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.847+0000 7f134235e640 1 --2- 172.21.15.43:0/1678328085 >> v2:172.21.15.193:3300/0 conn(0x7f133c071f90 0x7f133c0b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:02.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.847+0000 7f1341b5d640 1 --2- 172.21.15.43:0/1678328085 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f133c15a9d0 0x7f133c06d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:02.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.848+0000 7f134135c640 1 -- 172.21.15.43:0/1678328085 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f133c15a9d0 msgr2=0x7f133c06d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:02.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.848+0000 7f134135c640 1 --2- 172.21.15.43:0/1678328085 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f133c15a9d0 0x7f133c06d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:02.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.848+0000 7f134135c640 1 -- 172.21.15.43:0/1678328085 >> v2:172.21.15.193:3300/0 conn(0x7f133c071f90 msgr2=0x7f133c0b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:02.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.848+0000 7f134135c640 1 --2- 172.21.15.43:0/1678328085 >> v2:172.21.15.193:3300/0 conn(0x7f133c071f90 0x7f133c0b3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:02.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.848+0000 7f134135c640 1 -- 172.21.15.43:0/1678328085 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f133c0707d0 con 0x7f133c06d7d0 2024-04-25T08:55:02.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.848+0000 7f134135c640 1 --2- 172.21.15.43:0/1678328085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f133c06d7d0 0x7f133c071a50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f133c0702e0 tx=0x7f133002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa9fb4fb2bcad66a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:02.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.848+0000 7f1340b5b640 1 -- 172.21.15.43:0/1678328085 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1330037070 con 0x7f133c06d7d0 2024-04-25T08:55:02.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.848+0000 7f1340b5b640 1 -- 172.21.15.43:0/1678328085 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1330032440 con 0x7f133c06d7d0 2024-04-25T08:55:02.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.848+0000 7f1340b5b640 1 -- 172.21.15.43:0/1678328085 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1330032810 con 0x7f133c06d7d0 2024-04-25T08:55:02.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.849+0000 7f1343dc1640 1 -- 172.21.15.43:0/1678328085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f133c06d7d0 msgr2=0x7f133c071a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:02.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.849+0000 7f1343dc1640 1 --2- 172.21.15.43:0/1678328085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f133c06d7d0 0x7f133c071a50 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f133c0702e0 tx=0x7f133002a1e0 comp rx=0 tx=0).stop 2024-04-25T08:55:02.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.849+0000 7f1343dc1640 1 -- 172.21.15.43:0/1678328085 shutdown_connections 2024-04-25T08:55:02.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.849+0000 7f1343dc1640 1 --2- 172.21.15.43:0/1678328085 >> v2:172.21.15.193:3300/0 conn(0x7f133c071f90 0x7f133c0b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:02.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.849+0000 7f1343dc1640 1 --2- 172.21.15.43:0/1678328085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f133c06d7d0 0x7f133c071a50 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:02.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.849+0000 7f1343dc1640 1 --2- 172.21.15.43:0/1678328085 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f133c15a9d0 0x7f133c06d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:02.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.849+0000 7f1343dc1640 1 -- 172.21.15.43:0/1678328085 >> 172.21.15.43:0/1678328085 conn(0x7f133c0b1480 msgr2=0x7f133c0b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:02.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.849+0000 7f1343dc1640 1 -- 172.21.15.43:0/1678328085 shutdown_connections 2024-04-25T08:55:02.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.849+0000 7f1343dc1640 1 -- 172.21.15.43:0/1678328085 wait complete. 2024-04-25T08:55:02.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.850+0000 7f1343dc1640 1 Processor -- start 2024-04-25T08:55:02.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.850+0000 7f1343dc1640 1 -- start start 2024-04-25T08:55:02.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.851+0000 7f1343dc1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f133c06d7d0 0x7f133c0af430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:02.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.851+0000 7f1343dc1640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f133c071f90 0x7f133c158c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:02.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.851+0000 7f1343dc1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f133c14ff40 0x7f133c154790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:02.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.851+0000 7f1343dc1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133c0712b0 con 0x7f133c14ff40 2024-04-25T08:55:02.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.851+0000 7f1343dc1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133c070d90 con 0x7f133c06d7d0 2024-04-25T08:55:02.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.851+0000 7f1343dc1640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f133c0b3ed0 con 0x7f133c071f90 2024-04-25T08:55:02.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.851+0000 7f134235e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f133c14ff40 0x7f133c154790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:02.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.851+0000 7f134235e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f133c14ff40 0x7f133c154790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41978/0 (socket says 172.21.15.43:41978) 2024-04-25T08:55:02.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.851+0000 7f134235e640 1 -- 172.21.15.43:0/2370262689 learned_addr learned my addr 172.21.15.43:0/2370262689 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:02.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.851+0000 7f1341b5d640 1 --2- 172.21.15.43:0/2370262689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f133c06d7d0 0x7f133c0af430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:02.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.851+0000 7f134135c640 1 --2- 172.21.15.43:0/2370262689 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f133c071f90 0x7f133c158c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:02.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.851+0000 7f134235e640 1 -- 172.21.15.43:0/2370262689 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f133c071f90 msgr2=0x7f133c158c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:02.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.851+0000 7f134235e640 1 --2- 172.21.15.43:0/2370262689 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f133c071f90 0x7f133c158c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:02.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.851+0000 7f134235e640 1 -- 172.21.15.43:0/2370262689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f133c06d7d0 msgr2=0x7f133c0af430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:02.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.851+0000 7f134235e640 1 --2- 172.21.15.43:0/2370262689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f133c06d7d0 0x7f133c0af430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:02.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.851+0000 7f134235e640 1 -- 172.21.15.43:0/2370262689 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f133c0af970 con 0x7f133c14ff40 2024-04-25T08:55:02.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.852+0000 7f1341b5d640 1 --2- 172.21.15.43:0/2370262689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f133c06d7d0 0x7f133c0af430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:55:02.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.852+0000 7f134235e640 1 --2- 172.21.15.43:0/2370262689 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f133c14ff40 0x7f133c154790 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f132c00d580 tx=0x7f132c00da40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:02.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.852+0000 7f133a7fc640 1 -- 172.21.15.43:0/2370262689 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132c0125f0 con 0x7f133c14ff40 2024-04-25T08:55:02.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.852+0000 7f133a7fc640 1 -- 172.21.15.43:0/2370262689 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f132c019070 con 0x7f133c14ff40 2024-04-25T08:55:02.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.852+0000 7f1343dc1640 1 -- 172.21.15.43:0/2370262689 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f133c159150 con 0x7f133c14ff40 2024-04-25T08:55:02.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.852+0000 7f133a7fc640 1 -- 172.21.15.43:0/2370262689 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132c0129f0 con 0x7f133c14ff40 2024-04-25T08:55:02.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.852+0000 7f1343dc1640 1 -- 172.21.15.43:0/2370262689 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f133c0ae670 con 0x7f133c14ff40 2024-04-25T08:55:02.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.853+0000 7f1343dc1640 1 -- 172.21.15.43:0/2370262689 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f133c074ef0 con 0x7f133c14ff40 2024-04-25T08:55:02.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.853+0000 7f133a7fc640 1 -- 172.21.15.43:0/2370262689 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f132c012b90 con 0x7f133c14ff40 2024-04-25T08:55:02.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.854+0000 7f133a7fc640 1 --2- 172.21.15.43:0/2370262689 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f132806b4d0 0x7f132806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:02.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.854+0000 7f133a7fc640 1 -- 172.21.15.43:0/2370262689 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f132c08a1f0 con 0x7f133c14ff40 2024-04-25T08:55:02.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.854+0000 7f1341b5d640 1 --2- 172.21.15.43:0/2370262689 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f132806b4d0 0x7f132806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:02.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.855+0000 7f1341b5d640 1 --2- 172.21.15.43:0/2370262689 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f132806b4d0 0x7f132806d990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1334009a00 tx=0x7f1334003990 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:02.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:02.858+0000 7f133a7fc640 1 -- 172.21.15.43:0/2370262689 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f132c0593d0 con 0x7f133c14ff40 2024-04-25T08:55:03.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.119+0000 7f1343dc1640 1 -- 172.21.15.43:0/2370262689 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f133c159380 con 0x7f133c14ff40 2024-04-25T08:55:03.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.121+0000 7f133a7fc640 1 -- 172.21.15.43:0/2370262689 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f132c058cd0 con 0x7f133c14ff40 2024-04-25T08:55:03.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.123+0000 7f1343dc1640 1 -- 172.21.15.43:0/2370262689 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f132806b4d0 msgr2=0x7f132806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:03.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.123+0000 7f1343dc1640 1 --2- 172.21.15.43:0/2370262689 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f132806b4d0 0x7f132806d990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1334009a00 tx=0x7f1334003990 comp rx=0 tx=0).stop 2024-04-25T08:55:03.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.123+0000 7f1343dc1640 1 -- 172.21.15.43:0/2370262689 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f133c14ff40 msgr2=0x7f133c154790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:03.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.123+0000 7f1343dc1640 1 --2- 172.21.15.43:0/2370262689 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f133c14ff40 0x7f133c154790 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f132c00d580 tx=0x7f132c00da40 comp rx=0 tx=0).stop 2024-04-25T08:55:03.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.123+0000 7f1343dc1640 1 -- 172.21.15.43:0/2370262689 shutdown_connections 2024-04-25T08:55:03.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.123+0000 7f1343dc1640 1 --2- 172.21.15.43:0/2370262689 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f133c14ff40 0x7f133c154790 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:03.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.123+0000 7f1343dc1640 1 --2- 172.21.15.43:0/2370262689 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f133c071f90 0x7f133c158c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:03.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.123+0000 7f1343dc1640 1 --2- 172.21.15.43:0/2370262689 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f132806b4d0 0x7f132806d990 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:03.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.123+0000 7f1343dc1640 1 --2- 172.21.15.43:0/2370262689 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f133c06d7d0 0x7f133c0af430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:03.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.123+0000 7f1343dc1640 1 -- 172.21.15.43:0/2370262689 >> 172.21.15.43:0/2370262689 conn(0x7f133c0b1480 msgr2=0x7f133c06d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:03.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.123+0000 7f1343dc1640 1 -- 172.21.15.43:0/2370262689 shutdown_connections 2024-04-25T08:55:03.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.123+0000 7f1343dc1640 1 -- 172.21.15.43:0/2370262689 wait complete. 2024-04-25T08:55:03.124 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-25T08:55:03.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.236+0000 7f2a9719e640 1 Processor -- start 2024-04-25T08:55:03.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.236+0000 7f2a9719e640 1 -- start start 2024-04-25T08:55:03.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.237+0000 7f2a9719e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a900d0d90 0x7f2a900ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:03.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.237+0000 7f2a9719e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2a900d2500 0x7f2a900d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:03.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.237+0000 7f2a9719e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a900b6c50 con 0x7f2a900d39d0 2024-04-25T08:55:03.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.237+0000 7f2a9719e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a9015e010 con 0x7f2a900d2500 2024-04-25T08:55:03.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.237+0000 7f2a9719e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a900d2020 con 0x7f2a900d0d90 2024-04-25T08:55:03.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.237+0000 7f2a9573b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2a900d2500 0x7f2a900d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:03.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.237+0000 7f2a94f3a640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f2a900d39d0 0x7f2a900d3db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:55464/0 (socket says 172.21.15.43:55464) 2024-04-25T08:55:03.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.237+0000 7f2a94f3a640 1 -- 172.21.15.43:0/3056270789 learned_addr learned my addr 172.21.15.43:0/3056270789 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:55:03.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.238+0000 7f2a8ffff640 1 --2- 172.21.15.43:0/3056270789 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a900d0d90 0x7f2a900ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:03.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.238+0000 7f2a8f7fe640 1 -- 172.21.15.43:0/3056270789 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 569099878 0 0) 0x7f2a88001490 con 0x7f2a900d39d0 2024-04-25T08:55:03.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.238+0000 7f2a8f7fe640 1 -- 172.21.15.43:0/3056270789 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a7c003520 con 0x7f2a900d39d0 2024-04-25T08:55:03.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.238+0000 7f2a9573b640 1 -- 172.21.15.43:0/3056270789 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a900d0d90 msgr2=0x7f2a900ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:03.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.238+0000 7f2a9573b640 1 --2- 172.21.15.43:0/3056270789 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a900d0d90 0x7f2a900ce2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:03.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.238+0000 7f2a9573b640 1 -- 172.21.15.43:0/3056270789 >> v1:172.21.15.43:6789/0 conn(0x7f2a900d39d0 legacy=0x7f2a900d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:03.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.238+0000 7f2a9573b640 1 -- 172.21.15.43:0/3056270789 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a900cec40 con 0x7f2a900d2500 2024-04-25T08:55:03.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.238+0000 7f2a9573b640 1 --2- 172.21.15.43:0/3056270789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a900d2500 0x7f2a900d2990 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2a800097a0 tx=0x7f2a8002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be85d6550f6085d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:03.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.238+0000 7f2a8f7fe640 1 -- 172.21.15.43:0/3056270789 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a80037070 con 0x7f2a900d2500 2024-04-25T08:55:03.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.239+0000 7f2a8f7fe640 1 -- 172.21.15.43:0/3056270789 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a8002ae50 con 0x7f2a900d2500 2024-04-25T08:55:03.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.239+0000 7f2a8f7fe640 1 -- 172.21.15.43:0/3056270789 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a80032660 con 0x7f2a900d2500 2024-04-25T08:55:03.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.239+0000 7f2a9719e640 1 -- 172.21.15.43:0/3056270789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a900d2500 msgr2=0x7f2a900d2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:03.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.239+0000 7f2a9719e640 1 --2- 172.21.15.43:0/3056270789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a900d2500 0x7f2a900d2990 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2a800097a0 tx=0x7f2a8002a100 comp rx=0 tx=0).stop 2024-04-25T08:55:03.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.239+0000 7f2a9719e640 1 -- 172.21.15.43:0/3056270789 shutdown_connections 2024-04-25T08:55:03.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.239+0000 7f2a9719e640 1 --2- 172.21.15.43:0/3056270789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a900d2500 0x7f2a900d2990 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:03.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.239+0000 7f2a9719e640 1 --2- 172.21.15.43:0/3056270789 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a900d0d90 0x7f2a900ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:03.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.239+0000 7f2a9719e640 1 -- 172.21.15.43:0/3056270789 >> 172.21.15.43:0/3056270789 conn(0x7f2a9006b000 msgr2=0x7f2a9006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:03.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.239+0000 7f2a9719e640 1 -- 172.21.15.43:0/3056270789 shutdown_connections 2024-04-25T08:55:03.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.239+0000 7f2a9719e640 1 -- 172.21.15.43:0/3056270789 wait complete. 2024-04-25T08:55:03.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.241+0000 7f2a9719e640 1 Processor -- start 2024-04-25T08:55:03.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.241+0000 7f2a9719e640 1 -- start start 2024-04-25T08:55:03.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.242+0000 7f2a9719e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a900d0d90 0x7f2a900cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:03.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.242+0000 7f2a9719e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a900d2500 0x7f2a90149e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:03.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.242+0000 7f2a9719e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a900d39d0 0x7f2a9014c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:03.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.242+0000 7f2a9719e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a900cf200 con 0x7f2a900d39d0 2024-04-25T08:55:03.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.242+0000 7f2a9719e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a900b2f00 con 0x7f2a900d0d90 2024-04-25T08:55:03.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.242+0000 7f2a9719e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a900d2ed0 con 0x7f2a900d2500 2024-04-25T08:55:03.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.242+0000 7f2a9573b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a900d39d0 0x7f2a9014c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:03.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.242+0000 7f2a9573b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a900d39d0 0x7f2a9014c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42004/0 (socket says 172.21.15.43:42004) 2024-04-25T08:55:03.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.242+0000 7f2a9573b640 1 -- 172.21.15.43:0/4238072452 learned_addr learned my addr 172.21.15.43:0/4238072452 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:03.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.242+0000 7f2a8ffff640 1 --2- 172.21.15.43:0/4238072452 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a900d2500 0x7f2a90149e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:03.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.242+0000 7f2a94f3a640 1 --2- 172.21.15.43:0/4238072452 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a900d0d90 0x7f2a900cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:03.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.242+0000 7f2a9573b640 1 -- 172.21.15.43:0/4238072452 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a900d2500 msgr2=0x7f2a90149e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:03.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.242+0000 7f2a9573b640 1 --2- 172.21.15.43:0/4238072452 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a900d2500 0x7f2a90149e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:03.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.242+0000 7f2a9573b640 1 -- 172.21.15.43:0/4238072452 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a900d0d90 msgr2=0x7f2a900cab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:03.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.242+0000 7f2a9573b640 1 --2- 172.21.15.43:0/4238072452 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a900d0d90 0x7f2a900cab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:03.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.242+0000 7f2a9573b640 1 -- 172.21.15.43:0/4238072452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a9014ce00 con 0x7f2a900d39d0 2024-04-25T08:55:03.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.242+0000 7f2a8ffff640 1 --2- 172.21.15.43:0/4238072452 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a900d2500 0x7f2a90149e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:55:03.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.242+0000 7f2a94f3a640 1 --2- 172.21.15.43:0/4238072452 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a900d0d90 0x7f2a900cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:55:03.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.242+0000 7f2a9573b640 1 --2- 172.21.15.43:0/4238072452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a900d39d0 0x7f2a9014c8c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2a80035040 tx=0x7f2a800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:03.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.242+0000 7f2a8d7fa640 1 -- 172.21.15.43:0/4238072452 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a80041070 con 0x7f2a900d39d0 2024-04-25T08:55:03.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.242+0000 7f2a8d7fa640 1 -- 172.21.15.43:0/4238072452 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a8003c460 con 0x7f2a900d39d0 2024-04-25T08:55:03.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.242+0000 7f2a8d7fa640 1 -- 172.21.15.43:0/4238072452 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a80037040 con 0x7f2a900d39d0 2024-04-25T08:55:03.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.242+0000 7f2a9719e640 1 -- 172.21.15.43:0/4238072452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a900cc880 con 0x7f2a900d39d0 2024-04-25T08:55:03.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.242+0000 7f2a9719e640 1 -- 172.21.15.43:0/4238072452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a900cce40 con 0x7f2a900d39d0 2024-04-25T08:55:03.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.243+0000 7f2a9719e640 1 -- 172.21.15.43:0/4238072452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a90074ef0 con 0x7f2a900d39d0 2024-04-25T08:55:03.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.244+0000 7f2a8d7fa640 1 -- 172.21.15.43:0/4238072452 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2a80044070 con 0x7f2a900d39d0 2024-04-25T08:55:03.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.244+0000 7f2a8d7fa640 1 --2- 172.21.15.43:0/4238072452 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2a7006b4d0 0x7f2a7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:03.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.244+0000 7f2a8d7fa640 1 -- 172.21.15.43:0/4238072452 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2a800aa420 con 0x7f2a900d39d0 2024-04-25T08:55:03.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.245+0000 7f2a94f3a640 1 --2- 172.21.15.43:0/4238072452 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2a7006b4d0 0x7f2a7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:03.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.245+0000 7f2a94f3a640 1 --2- 172.21.15.43:0/4238072452 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2a7006b4d0 0x7f2a7006d990 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2a88005c00 tx=0x7f2a88009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:03.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.248+0000 7f2a8d7fa640 1 -- 172.21.15.43:0/4238072452 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2a80079680 con 0x7f2a900d39d0 2024-04-25T08:55:03.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.495+0000 7f2a9719e640 1 -- 172.21.15.43:0/4238072452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f2a900ce880 con 0x7f2a900d39d0 2024-04-25T08:55:03.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.497+0000 7f2a8d7fa640 1 -- 172.21.15.43:0/4238072452 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2a80078f80 con 0x7f2a900d39d0 2024-04-25T08:55:03.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.499+0000 7f2a9719e640 1 -- 172.21.15.43:0/4238072452 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2a7006b4d0 msgr2=0x7f2a7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:03.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.499+0000 7f2a9719e640 1 --2- 172.21.15.43:0/4238072452 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2a7006b4d0 0x7f2a7006d990 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f2a88005c00 tx=0x7f2a88009040 comp rx=0 tx=0).stop 2024-04-25T08:55:03.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.499+0000 7f2a9719e640 1 -- 172.21.15.43:0/4238072452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a900d39d0 msgr2=0x7f2a9014c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:03.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.499+0000 7f2a9719e640 1 --2- 172.21.15.43:0/4238072452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a900d39d0 0x7f2a9014c8c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2a80035040 tx=0x7f2a800046c0 comp rx=0 tx=0).stop 2024-04-25T08:55:03.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.499+0000 7f2a9719e640 1 -- 172.21.15.43:0/4238072452 shutdown_connections 2024-04-25T08:55:03.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.499+0000 7f2a9719e640 1 --2- 172.21.15.43:0/4238072452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a900d39d0 0x7f2a9014c8c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:03.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.499+0000 7f2a9719e640 1 --2- 172.21.15.43:0/4238072452 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a900d2500 0x7f2a90149e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:03.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.499+0000 7f2a9719e640 1 --2- 172.21.15.43:0/4238072452 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2a7006b4d0 0x7f2a7006d990 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:03.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.499+0000 7f2a9719e640 1 --2- 172.21.15.43:0/4238072452 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a900d0d90 0x7f2a900cab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:03.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.499+0000 7f2a9719e640 1 -- 172.21.15.43:0/4238072452 >> 172.21.15.43:0/4238072452 conn(0x7f2a9006b000 msgr2=0x7f2a9015fc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:03.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.500+0000 7f2a9719e640 1 -- 172.21.15.43:0/4238072452 shutdown_connections 2024-04-25T08:55:03.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.500+0000 7f2a9719e640 1 -- 172.21.15.43:0/4238072452 wait complete. 2024-04-25T08:55:03.500 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-25T08:55:03.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.612+0000 7f985d153640 1 Processor -- start 2024-04-25T08:55:03.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.612+0000 7f985d153640 1 -- start start 2024-04-25T08:55:03.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.613+0000 7f985d153640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f985815dce0 0x7f985815ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:03.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.613+0000 7f985d153640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f98580d0af0 0x7f9858160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:03.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.613+0000 7f985d153640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98580cee80 con 0x7f98580ce550 2024-04-25T08:55:03.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.613+0000 7f985d153640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98580d2920 con 0x7f98580d0af0 2024-04-25T08:55:03.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.613+0000 7f985d153640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98580cf160 con 0x7f985815dce0 2024-04-25T08:55:03.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.613+0000 7f985759e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f98580d0af0 0x7f9858160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:03.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.613+0000 7f985759e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f98580d0af0 0x7f9858160980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42018/0 (socket says 172.21.15.43:42018) 2024-04-25T08:55:03.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.613+0000 7f985759e640 1 -- 172.21.15.43:0/2327819163 learned_addr learned my addr 172.21.15.43:0/2327819163 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:03.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.613+0000 7f985659c640 1 --1- 172.21.15.43:0/2327819163 >> v1:172.21.15.43:6789/0 conn(0x7f98580ce550 0x7f98580d03e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:55474/0 (socket says 172.21.15.43:55474) 2024-04-25T08:55:03.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.614+0000 7f9856d9d640 1 --2- 172.21.15.43:0/2327819163 >> v2:172.21.15.193:3300/0 conn(0x7f985815dce0 0x7f985815ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:03.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.614+0000 7f985759e640 1 -- 172.21.15.43:0/2327819163 >> v2:172.21.15.193:3300/0 conn(0x7f985815dce0 msgr2=0x7f985815ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:03.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.614+0000 7f985759e640 1 --2- 172.21.15.43:0/2327819163 >> v2:172.21.15.193:3300/0 conn(0x7f985815dce0 0x7f985815ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:03.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.614+0000 7f985759e640 1 -- 172.21.15.43:0/2327819163 >> v1:172.21.15.43:6789/0 conn(0x7f98580ce550 legacy=0x7f98580d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:03.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.614+0000 7f985759e640 1 -- 172.21.15.43:0/2327819163 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98580cf440 con 0x7f98580d0af0 2024-04-25T08:55:03.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.614+0000 7f985759e640 1 --2- 172.21.15.43:0/2327819163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98580d0af0 0x7f9858160980 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f985809d3c0 tx=0x7f984802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce20d8ac648da1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:03.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.615+0000 7f9855d9b640 1 -- 172.21.15.43:0/2327819163 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9848036070 con 0x7f98580d0af0 2024-04-25T08:55:03.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.615+0000 7f9855d9b640 1 -- 172.21.15.43:0/2327819163 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9848031600 con 0x7f98580d0af0 2024-04-25T08:55:03.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.615+0000 7f9855d9b640 1 -- 172.21.15.43:0/2327819163 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98480319d0 con 0x7f98580d0af0 2024-04-25T08:55:03.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.615+0000 7f985d153640 1 -- 172.21.15.43:0/2327819163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98580d0af0 msgr2=0x7f9858160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:03.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.615+0000 7f985d153640 1 --2- 172.21.15.43:0/2327819163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98580d0af0 0x7f9858160980 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f985809d3c0 tx=0x7f984802a6a0 comp rx=0 tx=0).stop 2024-04-25T08:55:03.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.615+0000 7f985d153640 1 -- 172.21.15.43:0/2327819163 shutdown_connections 2024-04-25T08:55:03.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.615+0000 7f985d153640 1 --2- 172.21.15.43:0/2327819163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98580d0af0 0x7f9858160980 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:03.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.615+0000 7f985d153640 1 --2- 172.21.15.43:0/2327819163 >> v2:172.21.15.193:3300/0 conn(0x7f985815dce0 0x7f985815ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:03.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.615+0000 7f985d153640 1 -- 172.21.15.43:0/2327819163 >> 172.21.15.43:0/2327819163 conn(0x7f985809cd00 msgr2=0x7f98580d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:03.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.615+0000 7f985d153640 1 -- 172.21.15.43:0/2327819163 shutdown_connections 2024-04-25T08:55:03.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.615+0000 7f985d153640 1 -- 172.21.15.43:0/2327819163 wait complete. 2024-04-25T08:55:03.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.617+0000 7f985d153640 1 Processor -- start 2024-04-25T08:55:03.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.617+0000 7f985d153640 1 -- start start 2024-04-25T08:55:03.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.617+0000 7f985d153640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98580ce550 0x7f98581f3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:03.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.617+0000 7f985d153640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f98580d0af0 0x7f98580b8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:03.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.617+0000 7f985d153640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f985815dce0 0x7f98580b8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:03.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.617+0000 7f985d153640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985815e0c0 con 0x7f985815dce0 2024-04-25T08:55:03.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.617+0000 7f985d153640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985815f6b0 con 0x7f98580ce550 2024-04-25T08:55:03.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.617+0000 7f985d153640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98580ce9c0 con 0x7f98580d0af0 2024-04-25T08:55:03.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.617+0000 7f985759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f985815dce0 0x7f98580b8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:03.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.617+0000 7f985759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f985815dce0 0x7f98580b8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42022/0 (socket says 172.21.15.43:42022) 2024-04-25T08:55:03.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.617+0000 7f985759e640 1 -- 172.21.15.43:0/446411864 learned_addr learned my addr 172.21.15.43:0/446411864 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:03.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.618+0000 7f9856d9d640 1 --2- 172.21.15.43:0/446411864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98580ce550 0x7f98581f3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:03.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.618+0000 7f985659c640 1 --2- 172.21.15.43:0/446411864 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f98580d0af0 0x7f98580b8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:03.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.618+0000 7f985759e640 1 -- 172.21.15.43:0/446411864 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f98580d0af0 msgr2=0x7f98580b8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:03.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.618+0000 7f985759e640 1 --2- 172.21.15.43:0/446411864 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f98580d0af0 0x7f98580b8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:03.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.618+0000 7f985759e640 1 -- 172.21.15.43:0/446411864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98580ce550 msgr2=0x7f98581f3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:03.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.618+0000 7f985759e640 1 --2- 172.21.15.43:0/446411864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98580ce550 0x7f98581f3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:03.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.618+0000 7f985759e640 1 -- 172.21.15.43:0/446411864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f985815a950 con 0x7f985815dce0 2024-04-25T08:55:03.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.618+0000 7f985759e640 1 --2- 172.21.15.43:0/446411864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f985815dce0 0x7f98580b8ab0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9848009f40 tx=0x7f9848004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:03.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.618+0000 7f984f7fe640 1 -- 172.21.15.43:0/446411864 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9848036050 con 0x7f985815dce0 2024-04-25T08:55:03.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.618+0000 7f984f7fe640 1 -- 172.21.15.43:0/446411864 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9848042070 con 0x7f985815dce0 2024-04-25T08:55:03.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.618+0000 7f985d153640 1 -- 172.21.15.43:0/446411864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98580cc920 con 0x7f985815dce0 2024-04-25T08:55:03.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.618+0000 7f984f7fe640 1 -- 172.21.15.43:0/446411864 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9848009510 con 0x7f985815dce0 2024-04-25T08:55:03.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.618+0000 7f985d153640 1 -- 172.21.15.43:0/446411864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f985815aeb0 con 0x7f985815dce0 2024-04-25T08:55:03.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.619+0000 7f985d153640 1 -- 172.21.15.43:0/446411864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98580680f0 con 0x7f985815dce0 2024-04-25T08:55:03.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.620+0000 7f984f7fe640 1 -- 172.21.15.43:0/446411864 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9848007bd0 con 0x7f985815dce0 2024-04-25T08:55:03.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.620+0000 7f984f7fe640 1 --2- 172.21.15.43:0/446411864 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f983806b4d0 0x7f983806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:03.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.620+0000 7f984f7fe640 1 -- 172.21.15.43:0/446411864 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f98480aae00 con 0x7f985815dce0 2024-04-25T08:55:03.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.620+0000 7f9856d9d640 1 --2- 172.21.15.43:0/446411864 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f983806b4d0 0x7f983806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:03.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.621+0000 7f9856d9d640 1 --2- 172.21.15.43:0/446411864 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f983806b4d0 0x7f983806d990 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9850006fd0 tx=0x7f9850008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:03.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.624+0000 7f984f7fe640 1 -- 172.21.15.43:0/446411864 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f985815a950 con 0x7f985815dce0 2024-04-25T08:55:03.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.876+0000 7f985d153640 1 -- 172.21.15.43:0/446411864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f985809d1a0 con 0x7f985815dce0 2024-04-25T08:55:03.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.878+0000 7f984f7fe640 1 -- 172.21.15.43:0/446411864 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9848079fd0 con 0x7f985815dce0 2024-04-25T08:55:03.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.881+0000 7f985d153640 1 -- 172.21.15.43:0/446411864 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f983806b4d0 msgr2=0x7f983806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:03.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.881+0000 7f985d153640 1 --2- 172.21.15.43:0/446411864 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f983806b4d0 0x7f983806d990 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9850006fd0 tx=0x7f9850008040 comp rx=0 tx=0).stop 2024-04-25T08:55:03.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.881+0000 7f985d153640 1 -- 172.21.15.43:0/446411864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f985815dce0 msgr2=0x7f98580b8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:03.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.881+0000 7f985d153640 1 --2- 172.21.15.43:0/446411864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f985815dce0 0x7f98580b8ab0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9848009f40 tx=0x7f9848004a40 comp rx=0 tx=0).stop 2024-04-25T08:55:03.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.881+0000 7f985d153640 1 -- 172.21.15.43:0/446411864 shutdown_connections 2024-04-25T08:55:03.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.881+0000 7f985d153640 1 --2- 172.21.15.43:0/446411864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f985815dce0 0x7f98580b8ab0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:03.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.881+0000 7f985d153640 1 --2- 172.21.15.43:0/446411864 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f98580d0af0 0x7f98580b8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:03.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.881+0000 7f985d153640 1 --2- 172.21.15.43:0/446411864 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f983806b4d0 0x7f983806d990 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:03.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.881+0000 7f985d153640 1 --2- 172.21.15.43:0/446411864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98580ce550 0x7f98581f3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:03.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.881+0000 7f985d153640 1 -- 172.21.15.43:0/446411864 >> 172.21.15.43:0/446411864 conn(0x7f985809cd00 msgr2=0x7f98580c9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:03.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.881+0000 7f985d153640 1 -- 172.21.15.43:0/446411864 shutdown_connections 2024-04-25T08:55:03.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.881+0000 7f985d153640 1 -- 172.21.15.43:0/446411864 wait complete. 2024-04-25T08:55:03.882 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:55:03.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.993+0000 7f143bc4d640 1 Processor -- start 2024-04-25T08:55:03.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.993+0000 7f143bc4d640 1 -- start start 2024-04-25T08:55:03.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.994+0000 7f143bc4d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f14340ccbb0 0x7f14340cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:03.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.994+0000 7f143bc4d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14340b2780 con 0x7f14340cbfc0 2024-04-25T08:55:03.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.994+0000 7f143bc4d640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14340b2a60 con 0x7f14340b3b00 2024-04-25T08:55:03.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.994+0000 7f143bc4d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14340cd550 con 0x7f14340ccbb0 2024-04-25T08:55:03.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.994+0000 7f14399e9640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f14340cbfc0 0x7f143406dc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:55484/0 (socket says 172.21.15.43:55484) 2024-04-25T08:55:03.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.994+0000 7f14399e9640 1 -- 172.21.15.43:0/2947476492 learned_addr learned my addr 172.21.15.43:0/2947476492 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:55:03.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.994+0000 7f14391e8640 1 --2- 172.21.15.43:0/2947476492 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f14340ccbb0 0x7f14340cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:03.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.994+0000 7f14389e7640 1 -- 172.21.15.43:0/2947476492 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 560448529 0 0) 0x7f14340b2780 con 0x7f14340cbfc0 2024-04-25T08:55:03.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.994+0000 7f14389e7640 1 -- 172.21.15.43:0/2947476492 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1420003520 con 0x7f14340cbfc0 2024-04-25T08:55:03.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.994+0000 7f14389e7640 1 -- 172.21.15.43:0/2947476492 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3870443269 0 0) 0x7f14340b2a60 con 0x7f14340b3b00 2024-04-25T08:55:03.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.994+0000 7f14389e7640 1 -- 172.21.15.43:0/2947476492 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14340b2780 con 0x7f14340b3b00 2024-04-25T08:55:03.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.994+0000 7f14389e7640 1 -- 172.21.15.43:0/2947476492 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 234147687 0 0) 0x7f1420003520 con 0x7f14340cbfc0 2024-04-25T08:55:03.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.994+0000 7f14391e8640 1 -- 172.21.15.43:0/2947476492 >> v1:172.21.15.193:6789/0 conn(0x7f14340b3b00 legacy=0x7f14340b3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:03.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.994+0000 7f14391e8640 1 -- 172.21.15.43:0/2947476492 >> v1:172.21.15.43:6789/0 conn(0x7f14340cbfc0 legacy=0x7f143406dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:03.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.994+0000 7f14391e8640 1 -- 172.21.15.43:0/2947476492 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14340b2d40 con 0x7f14340ccbb0 2024-04-25T08:55:03.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.995+0000 7f14391e8640 1 --2- 172.21.15.43:0/2947476492 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f14340ccbb0 0x7f14340cd010 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f14240097d0 tx=0x7f142402a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2148a2cbca85ffd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:03.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.995+0000 7f14389e7640 1 -- 172.21.15.43:0/2947476492 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1424037070 con 0x7f14340ccbb0 2024-04-25T08:55:03.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.996+0000 7f14389e7640 1 -- 172.21.15.43:0/2947476492 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f142402ac30 con 0x7f14340ccbb0 2024-04-25T08:55:03.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.996+0000 7f14389e7640 1 -- 172.21.15.43:0/2947476492 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1430001670 con 0x7f14340ccbb0 2024-04-25T08:55:03.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.996+0000 7f143bc4d640 1 -- 172.21.15.43:0/2947476492 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f14340ccbb0 msgr2=0x7f14340cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:03.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.996+0000 7f143bc4d640 1 --2- 172.21.15.43:0/2947476492 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f14340ccbb0 0x7f14340cd010 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f14240097d0 tx=0x7f142402a1a0 comp rx=0 tx=0).stop 2024-04-25T08:55:03.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.996+0000 7f143bc4d640 1 -- 172.21.15.43:0/2947476492 shutdown_connections 2024-04-25T08:55:03.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.996+0000 7f143bc4d640 1 --2- 172.21.15.43:0/2947476492 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f14340ccbb0 0x7f14340cd010 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:03.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.996+0000 7f143bc4d640 1 -- 172.21.15.43:0/2947476492 >> 172.21.15.43:0/2947476492 conn(0x7f14340b1480 msgr2=0x7f14340b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:03.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.996+0000 7f143bc4d640 1 -- 172.21.15.43:0/2947476492 shutdown_connections 2024-04-25T08:55:03.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.996+0000 7f143bc4d640 1 -- 172.21.15.43:0/2947476492 wait complete. 2024-04-25T08:55:03.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.997+0000 7f143bc4d640 1 Processor -- start 2024-04-25T08:55:03.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.997+0000 7f143bc4d640 1 -- start start 2024-04-25T08:55:03.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.998+0000 7f143bc4d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14340b3b00 0x7f143413d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:03.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.998+0000 7f143bc4d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14340cbfc0 0x7f1434071f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:03.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.998+0000 7f143bc4d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f14340ccbb0 0x7f14341404f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:03.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.998+0000 7f143bc4d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14340b35e0 con 0x7f14340b3b00 2024-04-25T08:55:03.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.998+0000 7f143bc4d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14340b3300 con 0x7f14340cbfc0 2024-04-25T08:55:03.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.998+0000 7f143bc4d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14340cc3a0 con 0x7f14340ccbb0 2024-04-25T08:55:03.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.998+0000 7f14399e9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14340b3b00 0x7f143413d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:03.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.998+0000 7f14399e9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14340b3b00 0x7f143413d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42038/0 (socket says 172.21.15.43:42038) 2024-04-25T08:55:03.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.998+0000 7f14399e9640 1 -- 172.21.15.43:0/237745821 learned_addr learned my addr 172.21.15.43:0/237745821 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:03.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.998+0000 7f14391e8640 1 --2- 172.21.15.43:0/237745821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14340cbfc0 0x7f1434071f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:03.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.998+0000 7f143a1ea640 1 --2- 172.21.15.43:0/237745821 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f14340ccbb0 0x7f14341404f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:03.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.999+0000 7f14399e9640 1 -- 172.21.15.43:0/237745821 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f14340ccbb0 msgr2=0x7f14341404f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:03.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.999+0000 7f14399e9640 1 --2- 172.21.15.43:0/237745821 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f14340ccbb0 0x7f14341404f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:03.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.999+0000 7f14399e9640 1 -- 172.21.15.43:0/237745821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14340cbfc0 msgr2=0x7f1434071f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:03.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.999+0000 7f14399e9640 1 --2- 172.21.15.43:0/237745821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14340cbfc0 0x7f1434071f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:03.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.999+0000 7f14399e9640 1 -- 172.21.15.43:0/237745821 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1434070c40 con 0x7f14340b3b00 2024-04-25T08:55:03.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.999+0000 7f143a1ea640 1 --2- 172.21.15.43:0/237745821 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f14340ccbb0 0x7f14341404f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:55:03.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.999+0000 7f14391e8640 1 --2- 172.21.15.43:0/237745821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14340cbfc0 0x7f1434071f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:55:03.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.999+0000 7f14399e9640 1 --2- 172.21.15.43:0/237745821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14340b3b00 0x7f143413d300 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f143000c5d0 tx=0x7f143000ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:03.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.999+0000 7f142a7fc640 1 -- 172.21.15.43:0/237745821 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1430017070 con 0x7f14340b3b00 2024-04-25T08:55:03.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.999+0000 7f142a7fc640 1 -- 172.21.15.43:0/237745821 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1430005ce0 con 0x7f14340b3b00 2024-04-25T08:55:03.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.999+0000 7f143bc4d640 1 -- 172.21.15.43:0/237745821 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1434070860 con 0x7f14340b3b00 2024-04-25T08:55:03.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.999+0000 7f142a7fc640 1 -- 172.21.15.43:0/237745821 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14300072b0 con 0x7f14340b3b00 2024-04-25T08:55:03.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:03.999+0000 7f143bc4d640 1 -- 172.21.15.43:0/237745821 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14340711c0 con 0x7f14340b3b00 2024-04-25T08:55:03.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.000+0000 7f143bc4d640 1 -- 172.21.15.43:0/237745821 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1434074ef0 con 0x7f14340b3b00 2024-04-25T08:55:03.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.001+0000 7f142a7fc640 1 -- 172.21.15.43:0/237745821 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f143001a070 con 0x7f14340b3b00 2024-04-25T08:55:03.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.001+0000 7f142a7fc640 1 --2- 172.21.15.43:0/237745821 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f141806b4d0 0x7f141806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:03.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.001+0000 7f142a7fc640 1 -- 172.21.15.43:0/237745821 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1430089490 con 0x7f14340b3b00 2024-04-25T08:55:03.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.001+0000 7f14391e8640 1 --2- 172.21.15.43:0/237745821 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f141806b4d0 0x7f141806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:03.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.002+0000 7f14391e8640 1 --2- 172.21.15.43:0/237745821 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f141806b4d0 0x7f141806d990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1424034000 tx=0x7f142402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:03.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.005+0000 7f142a7fc640 1 -- 172.21.15.43:0/237745821 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f14300585e0 con 0x7f14340b3b00 2024-04-25T08:55:04.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.189+0000 7f143bc4d640 1 -- 172.21.15.43:0/237745821 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14340af0d0 con 0x7f14340b3b00 2024-04-25T08:55:04.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.190+0000 7f142a7fc640 1 -- 172.21.15.43:0/237745821 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) v1 ==== 74+0+12855 (secure 0 0 0) 0x7f1430057ee0 con 0x7f14340b3b00 2024-04-25T08:55:04.180 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:55:04.180 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":29,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:55:01.197206+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:55:04.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.193+0000 7f143bc4d640 1 -- 172.21.15.43:0/237745821 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f141806b4d0 msgr2=0x7f141806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:04.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.193+0000 7f143bc4d640 1 --2- 172.21.15.43:0/237745821 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f141806b4d0 0x7f141806d990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1424034000 tx=0x7f142402e000 comp rx=0 tx=0).stop 2024-04-25T08:55:04.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.193+0000 7f143bc4d640 1 -- 172.21.15.43:0/237745821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14340b3b00 msgr2=0x7f143413d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:04.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.193+0000 7f143bc4d640 1 --2- 172.21.15.43:0/237745821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14340b3b00 0x7f143413d300 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f143000c5d0 tx=0x7f143000ca90 comp rx=0 tx=0).stop 2024-04-25T08:55:04.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.193+0000 7f143bc4d640 1 -- 172.21.15.43:0/237745821 shutdown_connections 2024-04-25T08:55:04.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.193+0000 7f143bc4d640 1 --2- 172.21.15.43:0/237745821 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f14340ccbb0 0x7f14341404f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:04.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.193+0000 7f143bc4d640 1 --2- 172.21.15.43:0/237745821 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f141806b4d0 0x7f141806d990 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:04.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.193+0000 7f143bc4d640 1 --2- 172.21.15.43:0/237745821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f14340cbfc0 0x7f1434071f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:04.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.193+0000 7f143bc4d640 1 --2- 172.21.15.43:0/237745821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14340b3b00 0x7f143413d300 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:04.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.193+0000 7f143bc4d640 1 -- 172.21.15.43:0/237745821 >> 172.21.15.43:0/237745821 conn(0x7f14340b1480 msgr2=0x7f143406d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:04.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.193+0000 7f143bc4d640 1 -- 172.21.15.43:0/237745821 shutdown_connections 2024-04-25T08:55:04.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.193+0000 7f143bc4d640 1 -- 172.21.15.43:0/237745821 wait complete. 2024-04-25T08:55:04.194 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:55:04.194 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:55:04.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.308+0000 7f3ad3b4c640 1 Processor -- start 2024-04-25T08:55:04.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.308+0000 7f3ad3b4c640 1 -- start start 2024-04-25T08:55:04.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.309+0000 7f3ad3b4c640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f3acc0cbda0 0x7f3acc0d0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:04.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.309+0000 7f3ad3b4c640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3acc15bf00 con 0x7f3acc06ed70 2024-04-25T08:55:04.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.309+0000 7f3ad3b4c640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3acc0d1350 con 0x7f3acc0d0ec0 2024-04-25T08:55:04.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.309+0000 7f3ad3b4c640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3acc0cfdb0 con 0x7f3acc0cbda0 2024-04-25T08:55:04.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.309+0000 7f3ad18e8640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f3acc06ed70 0x7f3acc06f150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:55498/0 (socket says 172.21.15.43:55498) 2024-04-25T08:55:04.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.309+0000 7f3ad18e8640 1 -- 172.21.15.43:0/2265303960 learned_addr learned my addr 172.21.15.43:0/2265303960 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:55:04.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.309+0000 7f3ad10e7640 1 --2- 172.21.15.43:0/2265303960 >> v2:172.21.15.193:3300/0 conn(0x7f3acc0cbda0 0x7f3acc0d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:04.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.310+0000 7f3ad08e6640 1 -- 172.21.15.43:0/2265303960 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1635080655 0 0) 0x7f3acc15bf00 con 0x7f3acc06ed70 2024-04-25T08:55:04.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.310+0000 7f3ad08e6640 1 -- 172.21.15.43:0/2265303960 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ab8003520 con 0x7f3acc06ed70 2024-04-25T08:55:04.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.310+0000 7f3ad08e6640 1 -- 172.21.15.43:0/2265303960 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2088213414 0 0) 0x7f3acc0d1350 con 0x7f3acc0d0ec0 2024-04-25T08:55:04.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.310+0000 7f3ad08e6640 1 -- 172.21.15.43:0/2265303960 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3acc15bf00 con 0x7f3acc0d0ec0 2024-04-25T08:55:04.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.310+0000 7f3ad08e6640 1 -- 172.21.15.43:0/2265303960 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 367406429 0 0) 0x7f3ab8003520 con 0x7f3acc06ed70 2024-04-25T08:55:04.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.310+0000 7f3ad08e6640 1 -- 172.21.15.43:0/2265303960 >> v2:172.21.15.193:3300/0 conn(0x7f3acc0cbda0 msgr2=0x7f3acc0d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:04.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.310+0000 7f3ad08e6640 1 --2- 172.21.15.43:0/2265303960 >> v2:172.21.15.193:3300/0 conn(0x7f3acc0cbda0 0x7f3acc0d0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:04.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.310+0000 7f3ad08e6640 1 -- 172.21.15.43:0/2265303960 >> v1:172.21.15.193:6789/0 conn(0x7f3acc0d0ec0 legacy=0x7f3acc0c77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:04.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.310+0000 7f3ad08e6640 1 -- 172.21.15.43:0/2265303960 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3acc0c6740 con 0x7f3acc06ed70 2024-04-25T08:55:04.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.310+0000 7f3ad08e6640 1 -- 172.21.15.43:0/2265303960 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f3ac8001670 con 0x7f3acc06ed70 2024-04-25T08:55:04.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.310+0000 7f3ad08e6640 1 -- 172.21.15.43:0/2265303960 >> v1:172.21.15.43:6789/0 conn(0x7f3acc06ed70 legacy=0x7f3acc06f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:04.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.310+0000 7f3ad08e6640 1 --2- 172.21.15.43:0/2265303960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ab8003b40 0x7f3ab8003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:04.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.310+0000 7f3ad08e6640 1 --2- 172.21.15.43:0/2265303960 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ab8004800 0x7f3ab8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:04.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.310+0000 7f3ad08e6640 1 --2- 172.21.15.43:0/2265303960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ab80051c0 0x7f3ab800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:04.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.310+0000 7f3ad08e6640 1 -- 172.21.15.43:0/2265303960 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3acc0cfdb0 con 0x7f3ab80051c0 2024-04-25T08:55:04.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.310+0000 7f3ad08e6640 1 -- 172.21.15.43:0/2265303960 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab800c040 con 0x7f3ab8003b40 2024-04-25T08:55:04.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.310+0000 7f3ad08e6640 1 -- 172.21.15.43:0/2265303960 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab800c240 con 0x7f3ab8004800 2024-04-25T08:55:04.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.310+0000 7f3ad20e9640 1 --2- 172.21.15.43:0/2265303960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ab80051c0 0x7f3ab800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:04.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.310+0000 7f3ad18e8640 1 --2- 172.21.15.43:0/2265303960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ab8003b40 0x7f3ab8003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:04.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.310+0000 7f3ad10e7640 1 --2- 172.21.15.43:0/2265303960 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ab8004800 0x7f3ab8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:04.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.310+0000 7f3ad20e9640 1 -- 172.21.15.43:0/2265303960 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ab8004800 msgr2=0x7f3ab8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:04.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.310+0000 7f3ad20e9640 1 --2- 172.21.15.43:0/2265303960 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ab8004800 0x7f3ab8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:04.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.310+0000 7f3ad20e9640 1 -- 172.21.15.43:0/2265303960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ab8003b40 msgr2=0x7f3ab8003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:04.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.310+0000 7f3ad20e9640 1 --2- 172.21.15.43:0/2265303960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ab8003b40 0x7f3ab8003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:04.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.310+0000 7f3ad20e9640 1 -- 172.21.15.43:0/2265303960 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab8003520 con 0x7f3ab80051c0 2024-04-25T08:55:04.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.310+0000 7f3ad10e7640 1 --2- 172.21.15.43:0/2265303960 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ab8004800 0x7f3ab8004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:55:04.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.311+0000 7f3ad20e9640 1 -- 172.21.15.43:0/2265303960 reap_dead start 2024-04-25T08:55:04.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.311+0000 7f3ad20e9640 1 --2- 172.21.15.43:0/2265303960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ab80051c0 0x7f3ab800b9c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3abc0018d0 tx=0x7f3ab8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ffe1c2847de4645 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:04.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.311+0000 7f3ad08e6640 1 -- 172.21.15.43:0/2265303960 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc0014f0 con 0x7f3ab80051c0 2024-04-25T08:55:04.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.311+0000 7f3ad08e6640 1 -- 172.21.15.43:0/2265303960 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3abc0052d0 con 0x7f3ab80051c0 2024-04-25T08:55:04.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.311+0000 7f3ad08e6640 1 -- 172.21.15.43:0/2265303960 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc02ac80 con 0x7f3ab80051c0 2024-04-25T08:55:04.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.312+0000 7f3ad3b4c640 1 -- 172.21.15.43:0/2265303960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ab80051c0 msgr2=0x7f3ab800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:04.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.312+0000 7f3ad3b4c640 1 --2- 172.21.15.43:0/2265303960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ab80051c0 0x7f3ab800b9c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3abc0018d0 tx=0x7f3ab8005830 comp rx=0 tx=0).stop 2024-04-25T08:55:04.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.312+0000 7f3ad3b4c640 1 -- 172.21.15.43:0/2265303960 shutdown_connections 2024-04-25T08:55:04.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.312+0000 7f3ad3b4c640 1 --2- 172.21.15.43:0/2265303960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ab80051c0 0x7f3ab800b9c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:04.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.312+0000 7f3ad3b4c640 1 -- 172.21.15.43:0/2265303960 >> 172.21.15.43:0/2265303960 conn(0x7f3acc0d17e0 msgr2=0x7f3acc0d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:04.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.312+0000 7f3ad3b4c640 1 -- 172.21.15.43:0/2265303960 shutdown_connections 2024-04-25T08:55:04.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.312+0000 7f3ad3b4c640 1 -- 172.21.15.43:0/2265303960 wait complete. 2024-04-25T08:55:04.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.313+0000 7f3ad3b4c640 1 Processor -- start 2024-04-25T08:55:04.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.313+0000 7f3ad3b4c640 1 -- start start 2024-04-25T08:55:04.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.314+0000 7f3ad3b4c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ab80051c0 0x7f3acc152700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:04.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.314+0000 7f3ad3b4c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3acc154f90 0x7f3acc15d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:04.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.314+0000 7f3ad3b4c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3acc1502f0 0x7f3acc15d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:04.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.314+0000 7f3ad3b4c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3acc0c6fe0 con 0x7f3acc1502f0 2024-04-25T08:55:04.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.314+0000 7f3ad3b4c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3acc0c6d00 con 0x7f3ab80051c0 2024-04-25T08:55:04.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.314+0000 7f3ad3b4c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3acc0c72c0 con 0x7f3acc154f90 2024-04-25T08:55:04.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.314+0000 7f3ad20e9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3acc1502f0 0x7f3acc15d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:04.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.314+0000 7f3ad20e9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3acc1502f0 0x7f3acc15d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42064/0 (socket says 172.21.15.43:42064) 2024-04-25T08:55:04.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.314+0000 7f3ad20e9640 1 -- 172.21.15.43:0/2644421740 learned_addr learned my addr 172.21.15.43:0/2644421740 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:04.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.314+0000 7f3ad18e8640 1 --2- 172.21.15.43:0/2644421740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ab80051c0 0x7f3acc152700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:04.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.314+0000 7f3ad10e7640 1 --2- 172.21.15.43:0/2644421740 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3acc154f90 0x7f3acc15d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:04.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.314+0000 7f3ad20e9640 1 -- 172.21.15.43:0/2644421740 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3acc154f90 msgr2=0x7f3acc15d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:04.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.314+0000 7f3ad20e9640 1 --2- 172.21.15.43:0/2644421740 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3acc154f90 0x7f3acc15d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:04.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.314+0000 7f3ad20e9640 1 -- 172.21.15.43:0/2644421740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ab80051c0 msgr2=0x7f3acc152700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:04.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.314+0000 7f3ad20e9640 1 --2- 172.21.15.43:0/2644421740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ab80051c0 0x7f3acc152700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:04.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.314+0000 7f3ad20e9640 1 -- 172.21.15.43:0/2644421740 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3acc150b60 con 0x7f3acc1502f0 2024-04-25T08:55:04.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.314+0000 7f3ad20e9640 1 --2- 172.21.15.43:0/2644421740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3acc1502f0 0x7f3acc15d980 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3abc0019c0 tx=0x7f3abc0012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:04.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.315+0000 7f3ac27fc640 1 -- 172.21.15.43:0/2644421740 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc0014f0 con 0x7f3acc1502f0 2024-04-25T08:55:04.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.315+0000 7f3ac27fc640 1 -- 172.21.15.43:0/2644421740 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3abc005480 con 0x7f3acc1502f0 2024-04-25T08:55:04.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.315+0000 7f3ad3b4c640 1 -- 172.21.15.43:0/2644421740 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3acc152e00 con 0x7f3acc1502f0 2024-04-25T08:55:04.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.315+0000 7f3ac27fc640 1 -- 172.21.15.43:0/2644421740 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3abc004d40 con 0x7f3acc1502f0 2024-04-25T08:55:04.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.315+0000 7f3ad3b4c640 1 -- 172.21.15.43:0/2644421740 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3acc158890 con 0x7f3acc1502f0 2024-04-25T08:55:04.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.316+0000 7f3ad3b4c640 1 -- 172.21.15.43:0/2644421740 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3acc074f60 con 0x7f3acc1502f0 2024-04-25T08:55:04.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.316+0000 7f3ac27fc640 1 -- 172.21.15.43:0/2644421740 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3abc0067d0 con 0x7f3acc1502f0 2024-04-25T08:55:04.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.317+0000 7f3ac27fc640 1 --2- 172.21.15.43:0/2644421740 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3ab006b500 0x7f3ab006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:04.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.317+0000 7f3ac27fc640 1 -- 172.21.15.43:0/2644421740 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3abc0a8ee0 con 0x7f3acc1502f0 2024-04-25T08:55:04.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.317+0000 7f3ad18e8640 1 --2- 172.21.15.43:0/2644421740 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3ab006b500 0x7f3ab006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:04.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.318+0000 7f3ad18e8640 1 --2- 172.21.15.43:0/2644421740 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3ab006b500 0x7f3ab006d9c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8001eb0 tx=0x7f3ac8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:04.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.321+0000 7f3ac27fc640 1 -- 172.21.15.43:0/2644421740 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3abc078020 con 0x7f3acc1502f0 2024-04-25T08:55:04.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.499+0000 7f3ad3b4c640 1 -- 172.21.15.43:0/2644421740 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3acc15b050 con 0x7f3acc1502f0 2024-04-25T08:55:04.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.500+0000 7f3ac27fc640 1 -- 172.21.15.43:0/2644421740 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3abc077920 con 0x7f3acc1502f0 2024-04-25T08:55:04.490 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T08:55:04.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.502+0000 7f3ad3b4c640 1 -- 172.21.15.43:0/2644421740 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3ab006b500 msgr2=0x7f3ab006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:04.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.502+0000 7f3ad3b4c640 1 --2- 172.21.15.43:0/2644421740 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3ab006b500 0x7f3ab006d9c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3ac8001eb0 tx=0x7f3ac8009040 comp rx=0 tx=0).stop 2024-04-25T08:55:04.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.502+0000 7f3ad3b4c640 1 -- 172.21.15.43:0/2644421740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3acc1502f0 msgr2=0x7f3acc15d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:04.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.502+0000 7f3ad3b4c640 1 --2- 172.21.15.43:0/2644421740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3acc1502f0 0x7f3acc15d980 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3abc0019c0 tx=0x7f3abc0012d0 comp rx=0 tx=0).stop 2024-04-25T08:55:04.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.502+0000 7f3ad3b4c640 1 -- 172.21.15.43:0/2644421740 shutdown_connections 2024-04-25T08:55:04.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.502+0000 7f3ad3b4c640 1 --2- 172.21.15.43:0/2644421740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3acc1502f0 0x7f3acc15d980 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:04.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.502+0000 7f3ad3b4c640 1 --2- 172.21.15.43:0/2644421740 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3acc154f90 0x7f3acc15d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:04.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.502+0000 7f3ad3b4c640 1 --2- 172.21.15.43:0/2644421740 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3ab006b500 0x7f3ab006d9c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:04.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.502+0000 7f3ad3b4c640 1 --2- 172.21.15.43:0/2644421740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ab80051c0 0x7f3acc152700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:04.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.502+0000 7f3ad3b4c640 1 -- 172.21.15.43:0/2644421740 >> 172.21.15.43:0/2644421740 conn(0x7f3acc0d17e0 msgr2=0x7f3acc0aecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:04.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.503+0000 7f3ad3b4c640 1 -- 172.21.15.43:0/2644421740 shutdown_connections 2024-04-25T08:55:04.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.503+0000 7f3ad3b4c640 1 -- 172.21.15.43:0/2644421740 wait complete. 2024-04-25T08:55:04.503 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T08:55:04.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.615+0000 7f4401f7e640 1 Processor -- start 2024-04-25T08:55:04.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.615+0000 7f4401f7e640 1 -- start start 2024-04-25T08:55:04.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.616+0000 7f4401f7e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f43fc0b2ea0 0x7f43fc162030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:04.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.616+0000 7f4401f7e640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f43fc0b7910 0x7f43fc0b7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:04.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.616+0000 7f4401f7e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43fc0b3300 con 0x7f43fc15ddc0 2024-04-25T08:55:04.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.616+0000 7f4401f7e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43fc0ce4d0 con 0x7f43fc0b2ea0 2024-04-25T08:55:04.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.616+0000 7f4401f7e640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43fc0ce7b0 con 0x7f43fc0b7910 2024-04-25T08:55:04.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.616+0000 7f43faffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f43fc0b2ea0 0x7f43fc162030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:04.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.616+0000 7f43fb7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f43fc15ddc0 0x7f43fc161920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:55506/0 (socket says 172.21.15.43:55506) 2024-04-25T08:55:04.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.616+0000 7f43faffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f43fc0b2ea0 0x7f43fc162030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42066/0 (socket says 172.21.15.43:42066) 2024-04-25T08:55:04.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.616+0000 7f43fb7fe640 1 -- 172.21.15.43:0/2466317993 learned_addr learned my addr 172.21.15.43:0/2466317993 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:55:04.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.616+0000 7f43fbfff640 1 --2- 172.21.15.43:0/2466317993 >> v2:172.21.15.193:3300/0 conn(0x7f43fc0b7910 0x7f43fc0b7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:04.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.616+0000 7f43faffd640 1 -- 172.21.15.43:0/2466317993 >> v2:172.21.15.193:3300/0 conn(0x7f43fc0b7910 msgr2=0x7f43fc0b7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:04.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.616+0000 7f43faffd640 1 --2- 172.21.15.43:0/2466317993 >> v2:172.21.15.193:3300/0 conn(0x7f43fc0b7910 0x7f43fc0b7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:04.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.616+0000 7f43faffd640 1 -- 172.21.15.43:0/2466317993 >> v1:172.21.15.43:6789/0 conn(0x7f43fc15ddc0 legacy=0x7f43fc161920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:04.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.616+0000 7f43faffd640 1 -- 172.21.15.43:0/2466317993 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43fc0ceb40 con 0x7f43fc0b2ea0 2024-04-25T08:55:04.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.616+0000 7f43fa7fc640 1 -- 172.21.15.43:0/2466317993 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 693612834 0 0) 0x7f43fc0b3300 con 0x7f43fc15ddc0 2024-04-25T08:55:04.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.616+0000 7f43faffd640 1 --2- 172.21.15.43:0/2466317993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43fc0b2ea0 0x7f43fc162030 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f43fc0caa90 tx=0x7f43ec02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=102b85b8f202a4b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:04.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.616+0000 7f43fa7fc640 1 -- 172.21.15.43:0/2466317993 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ec037070 con 0x7f43fc0b2ea0 2024-04-25T08:55:04.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.616+0000 7f43fa7fc640 1 -- 172.21.15.43:0/2466317993 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43ec032440 con 0x7f43fc0b2ea0 2024-04-25T08:55:04.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.616+0000 7f43fa7fc640 1 -- 172.21.15.43:0/2466317993 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ec032810 con 0x7f43fc0b2ea0 2024-04-25T08:55:04.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.616+0000 7f4401f7e640 1 -- 172.21.15.43:0/2466317993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43fc0b2ea0 msgr2=0x7f43fc162030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:04.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.616+0000 7f4401f7e640 1 --2- 172.21.15.43:0/2466317993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43fc0b2ea0 0x7f43fc162030 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f43fc0caa90 tx=0x7f43ec02a1a0 comp rx=0 tx=0).stop 2024-04-25T08:55:04.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.617+0000 7f4401f7e640 1 -- 172.21.15.43:0/2466317993 shutdown_connections 2024-04-25T08:55:04.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.617+0000 7f4401f7e640 1 --2- 172.21.15.43:0/2466317993 >> v2:172.21.15.193:3300/0 conn(0x7f43fc0b7910 0x7f43fc0b7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:04.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.617+0000 7f4401f7e640 1 --2- 172.21.15.43:0/2466317993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43fc0b2ea0 0x7f43fc162030 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:04.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.617+0000 7f4401f7e640 1 -- 172.21.15.43:0/2466317993 >> 172.21.15.43:0/2466317993 conn(0x7f43fc06b000 msgr2=0x7f43fc06b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:04.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.617+0000 7f4401f7e640 1 -- 172.21.15.43:0/2466317993 shutdown_connections 2024-04-25T08:55:04.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.617+0000 7f4401f7e640 1 -- 172.21.15.43:0/2466317993 wait complete. 2024-04-25T08:55:04.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.618+0000 7f4401f7e640 1 Processor -- start 2024-04-25T08:55:04.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.618+0000 7f4401f7e640 1 -- start start 2024-04-25T08:55:04.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.619+0000 7f4401f7e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43fc0b2ea0 0x7f43fc158ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:04.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.619+0000 7f4401f7e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43fc0b7910 0x7f43fc0cc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:04.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.619+0000 7f4401f7e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f43fc15ddc0 0x7f43fc15bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:04.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.619+0000 7f4401f7e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43fc162570 con 0x7f43fc0b7910 2024-04-25T08:55:04.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.619+0000 7f4401f7e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43fc15e1a0 con 0x7f43fc0b2ea0 2024-04-25T08:55:04.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.619+0000 7f4401f7e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43fc1626f0 con 0x7f43fc15ddc0 2024-04-25T08:55:04.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.619+0000 7f43faffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43fc0b7910 0x7f43fc0cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:04.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.619+0000 7f43faffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43fc0b7910 0x7f43fc0cc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42072/0 (socket says 172.21.15.43:42072) 2024-04-25T08:55:04.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.619+0000 7f43faffd640 1 -- 172.21.15.43:0/1436470800 learned_addr learned my addr 172.21.15.43:0/1436470800 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:04.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.619+0000 7f43fbfff640 1 --2- 172.21.15.43:0/1436470800 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f43fc15ddc0 0x7f43fc15bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:04.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.619+0000 7f43fb7fe640 1 --2- 172.21.15.43:0/1436470800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43fc0b2ea0 0x7f43fc158ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:04.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.619+0000 7f43faffd640 1 -- 172.21.15.43:0/1436470800 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f43fc15ddc0 msgr2=0x7f43fc15bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:04.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.619+0000 7f43faffd640 1 --2- 172.21.15.43:0/1436470800 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f43fc15ddc0 0x7f43fc15bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:04.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.619+0000 7f43faffd640 1 -- 172.21.15.43:0/1436470800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43fc0b2ea0 msgr2=0x7f43fc158ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:04.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.619+0000 7f43faffd640 1 --2- 172.21.15.43:0/1436470800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43fc0b2ea0 0x7f43fc158ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:04.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.619+0000 7f43faffd640 1 -- 172.21.15.43:0/1436470800 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43fc0cceb0 con 0x7f43fc0b7910 2024-04-25T08:55:04.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.619+0000 7f43fbfff640 1 --2- 172.21.15.43:0/1436470800 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f43fc15ddc0 0x7f43fc15bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:55:04.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.619+0000 7f43fb7fe640 1 --2- 172.21.15.43:0/1436470800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43fc0b2ea0 0x7f43fc158ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:55:04.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.619+0000 7f43faffd640 1 --2- 172.21.15.43:0/1436470800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43fc0b7910 0x7f43fc0cc410 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f43ec000c00 tx=0x7f43ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:04.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.619+0000 7f43e3fff640 1 -- 172.21.15.43:0/1436470800 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ec037040 con 0x7f43fc0b7910 2024-04-25T08:55:04.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.619+0000 7f43e3fff640 1 -- 172.21.15.43:0/1436470800 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43ec042070 con 0x7f43fc0b7910 2024-04-25T08:55:04.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.619+0000 7f4401f7e640 1 -- 172.21.15.43:0/1436470800 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43fc15e450 con 0x7f43fc0b7910 2024-04-25T08:55:04.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.619+0000 7f43e3fff640 1 -- 172.21.15.43:0/1436470800 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ec030bd0 con 0x7f43fc0b7910 2024-04-25T08:55:04.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.620+0000 7f4401f7e640 1 -- 172.21.15.43:0/1436470800 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43fc0b5f20 con 0x7f43fc0b7910 2024-04-25T08:55:04.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.621+0000 7f4401f7e640 1 -- 172.21.15.43:0/1436470800 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43fc0680f0 con 0x7f43fc0b7910 2024-04-25T08:55:04.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.621+0000 7f43e3fff640 1 -- 172.21.15.43:0/1436470800 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f43ec047460 con 0x7f43fc0b7910 2024-04-25T08:55:04.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.621+0000 7f43e3fff640 1 --2- 172.21.15.43:0/1436470800 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f43e806b4d0 0x7f43e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:04.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.622+0000 7f43e3fff640 1 -- 172.21.15.43:0/1436470800 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f43ec0aaad0 con 0x7f43fc0b7910 2024-04-25T08:55:04.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.622+0000 7f43fb7fe640 1 --2- 172.21.15.43:0/1436470800 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f43e806b4d0 0x7f43e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:04.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.622+0000 7f43fb7fe640 1 --2- 172.21.15.43:0/1436470800 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f43e806b4d0 0x7f43e806d990 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f43f0000f60 tx=0x7f43f0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:04.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.626+0000 7f43e3fff640 1 -- 172.21.15.43:0/1436470800 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f43ec032e20 con 0x7f43fc0b7910 2024-04-25T08:55:04.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.807+0000 7f4401f7e640 1 -- 172.21.15.43:0/1436470800 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f43fc074ef0 con 0x7f43fc0b7910 2024-04-25T08:55:04.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.807+0000 7f43e3fff640 1 -- 172.21.15.43:0/1436470800 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v29) v1 ==== 84+0+65 (secure 0 0 0) 0x7f43ec079c90 con 0x7f43fc0b7910 2024-04-25T08:55:04.797 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:55:04.798 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2024-04-25T08:55:04.798 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:55:04.798 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 1, 2024-04-25T08:55:04.798 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": ".mgr" 2024-04-25T08:55:04.798 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:55:04.798 INFO:teuthology.orchestra.run.smithi043.stdout:] 2024-04-25T08:55:04.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.809+0000 7f4401f7e640 1 -- 172.21.15.43:0/1436470800 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f43e806b4d0 msgr2=0x7f43e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:04.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.809+0000 7f4401f7e640 1 --2- 172.21.15.43:0/1436470800 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f43e806b4d0 0x7f43e806d990 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f43f0000f60 tx=0x7f43f0009040 comp rx=0 tx=0).stop 2024-04-25T08:55:04.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.809+0000 7f4401f7e640 1 -- 172.21.15.43:0/1436470800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43fc0b7910 msgr2=0x7f43fc0cc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:04.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.809+0000 7f4401f7e640 1 --2- 172.21.15.43:0/1436470800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43fc0b7910 0x7f43fc0cc410 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f43ec000c00 tx=0x7f43ec0046c0 comp rx=0 tx=0).stop 2024-04-25T08:55:04.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.809+0000 7f4401f7e640 1 -- 172.21.15.43:0/1436470800 shutdown_connections 2024-04-25T08:55:04.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.809+0000 7f4401f7e640 1 --2- 172.21.15.43:0/1436470800 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f43fc15ddc0 0x7f43fc15bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:04.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.809+0000 7f4401f7e640 1 --2- 172.21.15.43:0/1436470800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f43fc0b7910 0x7f43fc0cc410 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:04.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.809+0000 7f4401f7e640 1 --2- 172.21.15.43:0/1436470800 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f43e806b4d0 0x7f43e806d990 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:04.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.809+0000 7f4401f7e640 1 --2- 172.21.15.43:0/1436470800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f43fc0b2ea0 0x7f43fc158ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:04.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.809+0000 7f4401f7e640 1 -- 172.21.15.43:0/1436470800 >> 172.21.15.43:0/1436470800 conn(0x7f43fc06b000 msgr2=0x7f43fc0b85c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:04.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.809+0000 7f4401f7e640 1 -- 172.21.15.43:0/1436470800 shutdown_connections 2024-04-25T08:55:04.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.809+0000 7f4401f7e640 1 -- 172.21.15.43:0/1436470800 wait complete. 2024-04-25T08:55:04.810 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T08:55:04.810 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T08:55:04.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.924+0000 7f04bd41f640 1 Processor -- start 2024-04-25T08:55:04.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.924+0000 7f04bd41f640 1 -- start start 2024-04-25T08:55:04.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.925+0000 7f04bd41f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f04b815def0 0x7f04b80d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:04.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.925+0000 7f04bd41f640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f04b80d4670 0x7f04b80d03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:04.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.925+0000 7f04bd41f640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04b80d2370 con 0x7f04b80d0930 2024-04-25T08:55:04.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.925+0000 7f04bd41f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04b80b37b0 con 0x7f04b815def0 2024-04-25T08:55:04.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.925+0000 7f04bd41f640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04b80d2650 con 0x7f04b80d4670 2024-04-25T08:55:04.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.925+0000 7f04b6ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f04b815def0 0x7f04b80d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:04.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.925+0000 7f04b6ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f04b815def0 0x7f04b80d4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42082/0 (socket says 172.21.15.43:42082) 2024-04-25T08:55:04.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.925+0000 7f04b6ffd640 1 -- 172.21.15.43:0/3650474358 learned_addr learned my addr 172.21.15.43:0/3650474358 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:04.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.926+0000 7f04b67fc640 1 --2- 172.21.15.43:0/3650474358 >> v2:172.21.15.193:3300/0 conn(0x7f04b80d4670 0x7f04b80d03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:04.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.926+0000 7f04b6ffd640 1 -- 172.21.15.43:0/3650474358 >> v2:172.21.15.193:3300/0 conn(0x7f04b80d4670 msgr2=0x7f04b80d03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:04.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.926+0000 7f04b6ffd640 1 --2- 172.21.15.43:0/3650474358 >> v2:172.21.15.193:3300/0 conn(0x7f04b80d4670 0x7f04b80d03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:04.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.926+0000 7f04b6ffd640 1 -- 172.21.15.43:0/3650474358 >> v1:172.21.15.193:6789/0 conn(0x7f04b80d0930 legacy=0x7f04b80ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:04.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.926+0000 7f04b6ffd640 1 -- 172.21.15.43:0/3650474358 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b80b3440 con 0x7f04b815def0 2024-04-25T08:55:04.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.926+0000 7f04b6ffd640 1 --2- 172.21.15.43:0/3650474358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04b815def0 0x7f04b80d4130 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f04b80cfbe0 tx=0x7f04b002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4524a15d3b2a8967 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:04.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.927+0000 7f04b5ffb640 1 -- 172.21.15.43:0/3650474358 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b0037070 con 0x7f04b815def0 2024-04-25T08:55:04.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.927+0000 7f04b5ffb640 1 -- 172.21.15.43:0/3650474358 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b0032440 con 0x7f04b815def0 2024-04-25T08:55:04.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.927+0000 7f04b5ffb640 1 -- 172.21.15.43:0/3650474358 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b0032810 con 0x7f04b815def0 2024-04-25T08:55:04.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.927+0000 7f04bd41f640 1 -- 172.21.15.43:0/3650474358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04b815def0 msgr2=0x7f04b80d4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:04.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.927+0000 7f04bd41f640 1 --2- 172.21.15.43:0/3650474358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04b815def0 0x7f04b80d4130 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f04b80cfbe0 tx=0x7f04b002a1e0 comp rx=0 tx=0).stop 2024-04-25T08:55:04.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.927+0000 7f04bd41f640 1 -- 172.21.15.43:0/3650474358 shutdown_connections 2024-04-25T08:55:04.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.927+0000 7f04bd41f640 1 --2- 172.21.15.43:0/3650474358 >> v2:172.21.15.193:3300/0 conn(0x7f04b80d4670 0x7f04b80d03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:04.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.927+0000 7f04bd41f640 1 --2- 172.21.15.43:0/3650474358 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04b815def0 0x7f04b80d4130 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:04.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.927+0000 7f04bd41f640 1 -- 172.21.15.43:0/3650474358 >> 172.21.15.43:0/3650474358 conn(0x7f04b809cd60 msgr2=0x7f04b80d3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:04.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.927+0000 7f04bd41f640 1 -- 172.21.15.43:0/3650474358 shutdown_connections 2024-04-25T08:55:04.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.927+0000 7f04bd41f640 1 -- 172.21.15.43:0/3650474358 wait complete. 2024-04-25T08:55:04.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.929+0000 7f04bd41f640 1 Processor -- start 2024-04-25T08:55:04.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.929+0000 7f04bd41f640 1 -- start start 2024-04-25T08:55:04.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.929+0000 7f04bd41f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04b80d0930 0x7f04b8160850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:04.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.929+0000 7f04bd41f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04b80d4670 0x7f04b8160d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:04.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.929+0000 7f04bd41f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f04b815def0 0x7f04b815e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:04.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.929+0000 7f04bd41f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b80d2150 con 0x7f04b80d4670 2024-04-25T08:55:04.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.929+0000 7f04bd41f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04b80d1fd0 con 0x7f04b80d0930 2024-04-25T08:55:04.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.929+0000 7f04bd41f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04b80b4c50 con 0x7f04b815def0 2024-04-25T08:55:04.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.929+0000 7f04b67fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04b80d4670 0x7f04b8160d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:04.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.929+0000 7f04b67fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04b80d4670 0x7f04b8160d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42096/0 (socket says 172.21.15.43:42096) 2024-04-25T08:55:04.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.929+0000 7f04b67fc640 1 -- 172.21.15.43:0/2425880271 learned_addr learned my addr 172.21.15.43:0/2425880271 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:04.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.930+0000 7f04b6ffd640 1 --2- 172.21.15.43:0/2425880271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04b80d0930 0x7f04b8160850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:04.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.930+0000 7f04b77fe640 1 --2- 172.21.15.43:0/2425880271 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f04b815def0 0x7f04b815e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:04.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.930+0000 7f04b67fc640 1 -- 172.21.15.43:0/2425880271 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f04b815def0 msgr2=0x7f04b815e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:04.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.930+0000 7f04b67fc640 1 --2- 172.21.15.43:0/2425880271 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f04b815def0 0x7f04b815e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:04.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.930+0000 7f04b67fc640 1 -- 172.21.15.43:0/2425880271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04b80d0930 msgr2=0x7f04b8160850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:04.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.930+0000 7f04b67fc640 1 --2- 172.21.15.43:0/2425880271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04b80d0930 0x7f04b8160850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:04.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.930+0000 7f04b67fc640 1 -- 172.21.15.43:0/2425880271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b815ef90 con 0x7f04b80d4670 2024-04-25T08:55:04.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.930+0000 7f04b6ffd640 1 --2- 172.21.15.43:0/2425880271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04b80d0930 0x7f04b8160850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:55:04.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.930+0000 7f04b67fc640 1 --2- 172.21.15.43:0/2425880271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04b80d4670 0x7f04b8160d90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f04a000d800 tx=0x7f04a000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:04.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.930+0000 7f04af7fe640 1 -- 172.21.15.43:0/2425880271 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a000b660 con 0x7f04b80d4670 2024-04-25T08:55:04.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.930+0000 7f04af7fe640 1 -- 172.21.15.43:0/2425880271 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04a0019070 con 0x7f04b80d4670 2024-04-25T08:55:04.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.930+0000 7f04bd41f640 1 -- 172.21.15.43:0/2425880271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04b815f270 con 0x7f04b80d4670 2024-04-25T08:55:04.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.930+0000 7f04af7fe640 1 -- 172.21.15.43:0/2425880271 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04a000ba20 con 0x7f04b80d4670 2024-04-25T08:55:04.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.930+0000 7f04bd41f640 1 -- 172.21.15.43:0/2425880271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04b815f830 con 0x7f04b80d4670 2024-04-25T08:55:04.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.931+0000 7f04bd41f640 1 -- 172.21.15.43:0/2425880271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04b8068a40 con 0x7f04b80d4670 2024-04-25T08:55:04.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.932+0000 7f04af7fe640 1 -- 172.21.15.43:0/2425880271 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f04a000bbc0 con 0x7f04b80d4670 2024-04-25T08:55:04.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.932+0000 7f04af7fe640 1 --2- 172.21.15.43:0/2425880271 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f049806b4d0 0x7f049806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:04.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.932+0000 7f04af7fe640 1 -- 172.21.15.43:0/2425880271 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f04a008a2f0 con 0x7f04b80d4670 2024-04-25T08:55:04.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.932+0000 7f04b6ffd640 1 --2- 172.21.15.43:0/2425880271 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f049806b4d0 0x7f049806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:04.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.933+0000 7f04b6ffd640 1 --2- 172.21.15.43:0/2425880271 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f049806b4d0 0x7f049806d990 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f04b0034000 tx=0x7f04b002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:04.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:04.936+0000 7f04af7fe640 1 -- 172.21.15.43:0/2425880271 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f04a00594c0 con 0x7f04b80d4670 2024-04-25T08:55:05.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.119+0000 7f04bd41f640 1 -- 172.21.15.43:0/2425880271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f04b80cff10 con 0x7f04b80d4670 2024-04-25T08:55:05.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.501+0000 7f04af7fe640 1 -- 172.21.15.43:0/2425880271 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v30) v1 ==== 138+0+0 (secure 0 0 0) 0x7f04a0058dc0 con 0x7f04b80d4670 2024-04-25T08:55:05.490 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' created 2024-04-25T08:55:05.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.503+0000 7f04bd41f640 1 -- 172.21.15.43:0/2425880271 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f049806b4d0 msgr2=0x7f049806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:05.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.503+0000 7f04bd41f640 1 --2- 172.21.15.43:0/2425880271 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f049806b4d0 0x7f049806d990 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f04b0034000 tx=0x7f04b002e000 comp rx=0 tx=0).stop 2024-04-25T08:55:05.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.503+0000 7f04bd41f640 1 -- 172.21.15.43:0/2425880271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04b80d4670 msgr2=0x7f04b8160d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:05.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.503+0000 7f04bd41f640 1 --2- 172.21.15.43:0/2425880271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04b80d4670 0x7f04b8160d90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f04a000d800 tx=0x7f04a000dcc0 comp rx=0 tx=0).stop 2024-04-25T08:55:05.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.503+0000 7f04bd41f640 1 -- 172.21.15.43:0/2425880271 shutdown_connections 2024-04-25T08:55:05.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.503+0000 7f04bd41f640 1 --2- 172.21.15.43:0/2425880271 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f04b815def0 0x7f04b815e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:05.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.503+0000 7f04bd41f640 1 --2- 172.21.15.43:0/2425880271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04b80d4670 0x7f04b8160d90 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:05.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.503+0000 7f04bd41f640 1 --2- 172.21.15.43:0/2425880271 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f049806b4d0 0x7f049806d990 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:05.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.503+0000 7f04bd41f640 1 --2- 172.21.15.43:0/2425880271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04b80d0930 0x7f04b8160850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:05.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.503+0000 7f04bd41f640 1 -- 172.21.15.43:0/2425880271 >> 172.21.15.43:0/2425880271 conn(0x7f04b809cd60 msgr2=0x7f04b80b4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:05.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.503+0000 7f04bd41f640 1 -- 172.21.15.43:0/2425880271 shutdown_connections 2024-04-25T08:55:05.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.503+0000 7f04bd41f640 1 -- 172.21.15.43:0/2425880271 wait complete. 2024-04-25T08:55:05.508 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T08:55:05.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.628+0000 7ff51e5e2640 1 Processor -- start 2024-04-25T08:55:05.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.628+0000 7ff51e5e2640 1 -- start start 2024-04-25T08:55:05.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.629+0000 7ff51e5e2640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff5180d0190 0x7ff5180d0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:05.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.629+0000 7ff51e5e2640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff5180d0ab0 0x7ff51806b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:05.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.629+0000 7ff51e5e2640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5180d2310 con 0x7ff51806bd80 2024-04-25T08:55:05.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.629+0000 7ff51e5e2640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff51806b3b0 con 0x7ff5180d0190 2024-04-25T08:55:05.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.629+0000 7ff51e5e2640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5180d25f0 con 0x7ff5180d0ab0 2024-04-25T08:55:05.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.629+0000 7ff517fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff5180d0190 0x7ff5180d0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:05.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.629+0000 7ff517fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff5180d0190 0x7ff5180d0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42100/0 (socket says 172.21.15.43:42100) 2024-04-25T08:55:05.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.629+0000 7ff517fff640 1 -- 172.21.15.43:0/2154693122 learned_addr learned my addr 172.21.15.43:0/2154693122 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:05.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.629+0000 7ff5177fe640 1 --2- 172.21.15.43:0/2154693122 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff5180d0ab0 0x7ff51806b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:05.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.629+0000 7ff517fff640 1 -- 172.21.15.43:0/2154693122 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff5180d0ab0 msgr2=0x7ff51806b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:05.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.629+0000 7ff517fff640 1 --2- 172.21.15.43:0/2154693122 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff5180d0ab0 0x7ff51806b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:05.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.629+0000 7ff517fff640 1 -- 172.21.15.43:0/2154693122 >> v1:172.21.15.193:6789/0 conn(0x7ff51806bd80 legacy=0x7ff51806c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:05.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.629+0000 7ff517fff640 1 -- 172.21.15.43:0/2154693122 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5180d28d0 con 0x7ff5180d0190 2024-04-25T08:55:05.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.629+0000 7ff5177fe640 1 --2- 172.21.15.43:0/2154693122 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff5180d0ab0 0x7ff51806b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:05.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.630+0000 7ff517fff640 1 --2- 172.21.15.43:0/2154693122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5180d0190 0x7ff5180d0570 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff50c009870 tx=0x7ff50c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f67bf514ae4b5fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:05.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.630+0000 7ff516ffd640 1 -- 172.21.15.43:0/2154693122 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50c037070 con 0x7ff5180d0190 2024-04-25T08:55:05.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.630+0000 7ff516ffd640 1 -- 172.21.15.43:0/2154693122 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff50c032440 con 0x7ff5180d0190 2024-04-25T08:55:05.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.630+0000 7ff516ffd640 1 -- 172.21.15.43:0/2154693122 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50c032810 con 0x7ff5180d0190 2024-04-25T08:55:05.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.631+0000 7ff51e5e2640 1 -- 172.21.15.43:0/2154693122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5180d0190 msgr2=0x7ff5180d0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:05.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.631+0000 7ff51e5e2640 1 --2- 172.21.15.43:0/2154693122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5180d0190 0x7ff5180d0570 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff50c009870 tx=0x7ff50c02a180 comp rx=0 tx=0).stop 2024-04-25T08:55:05.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.631+0000 7ff51e5e2640 1 -- 172.21.15.43:0/2154693122 shutdown_connections 2024-04-25T08:55:05.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.631+0000 7ff51e5e2640 1 --2- 172.21.15.43:0/2154693122 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff5180d0ab0 0x7ff51806b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:05.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.631+0000 7ff51e5e2640 1 --2- 172.21.15.43:0/2154693122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff5180d0190 0x7ff5180d0570 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:05.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.631+0000 7ff51e5e2640 1 -- 172.21.15.43:0/2154693122 >> 172.21.15.43:0/2154693122 conn(0x7ff5180d39e0 msgr2=0x7ff5180d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:05.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.631+0000 7ff51e5e2640 1 -- 172.21.15.43:0/2154693122 shutdown_connections 2024-04-25T08:55:05.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.631+0000 7ff51e5e2640 1 -- 172.21.15.43:0/2154693122 wait complete. 2024-04-25T08:55:05.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.632+0000 7ff51e5e2640 1 Processor -- start 2024-04-25T08:55:05.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.632+0000 7ff51e5e2640 1 -- start start 2024-04-25T08:55:05.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.634+0000 7ff51e5e2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff51806bd80 0x7ff5181f33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:05.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.634+0000 7ff51e5e2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5180d0190 0x7ff5180b7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:05.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.634+0000 7ff51e5e2640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff5180d0ab0 0x7ff51814f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:05.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.634+0000 7ff51e5e2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5180d1f90 con 0x7ff51806bd80 2024-04-25T08:55:05.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.634+0000 7ff51e5e2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5180d0f30 con 0x7ff5180d0190 2024-04-25T08:55:05.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.634+0000 7ff51e5e2640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff51815df20 con 0x7ff5180d0ab0 2024-04-25T08:55:05.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.634+0000 7ff517fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff51806bd80 0x7ff5181f33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:05.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.634+0000 7ff517fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff51806bd80 0x7ff5181f33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42116/0 (socket says 172.21.15.43:42116) 2024-04-25T08:55:05.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.634+0000 7ff517fff640 1 -- 172.21.15.43:0/791505560 learned_addr learned my addr 172.21.15.43:0/791505560 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:05.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.634+0000 7ff51cb7f640 1 --2- 172.21.15.43:0/791505560 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff5180d0ab0 0x7ff51814f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:05.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.634+0000 7ff5177fe640 1 --2- 172.21.15.43:0/791505560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5180d0190 0x7ff5180b7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:05.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.634+0000 7ff517fff640 1 -- 172.21.15.43:0/791505560 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff5180d0ab0 msgr2=0x7ff51814f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:05.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.634+0000 7ff517fff640 1 --2- 172.21.15.43:0/791505560 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff5180d0ab0 0x7ff51814f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:05.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.634+0000 7ff517fff640 1 -- 172.21.15.43:0/791505560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5180d0190 msgr2=0x7ff5180b7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:05.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.634+0000 7ff517fff640 1 --2- 172.21.15.43:0/791505560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5180d0190 0x7ff5180b7c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:05.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.634+0000 7ff517fff640 1 -- 172.21.15.43:0/791505560 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff51814bad0 con 0x7ff51806bd80 2024-04-25T08:55:05.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.634+0000 7ff5177fe640 1 --2- 172.21.15.43:0/791505560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5180d0190 0x7ff5180b7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:05.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.634+0000 7ff517fff640 1 --2- 172.21.15.43:0/791505560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff51806bd80 0x7ff5181f33e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff50c000c00 tx=0x7ff50c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:05.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.634+0000 7ff514ff9640 1 -- 172.21.15.43:0/791505560 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50c037040 con 0x7ff51806bd80 2024-04-25T08:55:05.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.634+0000 7ff514ff9640 1 -- 172.21.15.43:0/791505560 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff50c042070 con 0x7ff51806bd80 2024-04-25T08:55:05.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.634+0000 7ff51e5e2640 1 -- 172.21.15.43:0/791505560 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff51814bdb0 con 0x7ff51806bd80 2024-04-25T08:55:05.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.634+0000 7ff514ff9640 1 -- 172.21.15.43:0/791505560 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff50c030bb0 con 0x7ff51806bd80 2024-04-25T08:55:05.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.634+0000 7ff51e5e2640 1 -- 172.21.15.43:0/791505560 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff51814c370 con 0x7ff51806bd80 2024-04-25T08:55:05.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.635+0000 7ff51e5e2640 1 -- 172.21.15.43:0/791505560 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff518074f60 con 0x7ff51806bd80 2024-04-25T08:55:05.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.635+0000 7ff514ff9640 1 -- 172.21.15.43:0/791505560 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff50c047460 con 0x7ff51806bd80 2024-04-25T08:55:05.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.636+0000 7ff514ff9640 1 --2- 172.21.15.43:0/791505560 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff50406b4d0 0x7ff50406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:05.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.636+0000 7ff514ff9640 1 -- 172.21.15.43:0/791505560 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5227+0+0 (secure 0 0 0) 0x7ff50c0aacb0 con 0x7ff51806bd80 2024-04-25T08:55:05.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.636+0000 7ff5177fe640 1 --2- 172.21.15.43:0/791505560 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff50406b4d0 0x7ff50406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:05.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.636+0000 7ff5177fe640 1 --2- 172.21.15.43:0/791505560 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff50406b4d0 0x7ff50406d990 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff508006fd0 tx=0x7ff508008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:05.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.640+0000 7ff514ff9640 1 -- 172.21.15.43:0/791505560 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff50c045360 con 0x7ff51806bd80 2024-04-25T08:55:05.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:05.822+0000 7ff51e5e2640 1 -- 172.21.15.43:0/791505560 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff5180d3e60 con 0x7ff51806bd80 2024-04-25T08:55:06.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.506+0000 7ff514ff9640 1 -- 172.21.15.43:0/791505560 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v31) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff50c079d20 con 0x7ff51806bd80 2024-04-25T08:55:06.496 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' created 2024-04-25T08:55:06.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.510+0000 7ff51e5e2640 1 -- 172.21.15.43:0/791505560 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff50406b4d0 msgr2=0x7ff50406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:06.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.510+0000 7ff51e5e2640 1 --2- 172.21.15.43:0/791505560 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff50406b4d0 0x7ff50406d990 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff508006fd0 tx=0x7ff508008040 comp rx=0 tx=0).stop 2024-04-25T08:55:06.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.510+0000 7ff51e5e2640 1 -- 172.21.15.43:0/791505560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff51806bd80 msgr2=0x7ff5181f33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:06.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.510+0000 7ff51e5e2640 1 --2- 172.21.15.43:0/791505560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff51806bd80 0x7ff5181f33e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff50c000c00 tx=0x7ff50c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:55:06.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.510+0000 7ff51e5e2640 1 -- 172.21.15.43:0/791505560 shutdown_connections 2024-04-25T08:55:06.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.510+0000 7ff51e5e2640 1 --2- 172.21.15.43:0/791505560 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff5180d0ab0 0x7ff51814f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:06.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.510+0000 7ff51e5e2640 1 --2- 172.21.15.43:0/791505560 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff50406b4d0 0x7ff50406d990 secure :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff508006fd0 tx=0x7ff508008040 comp rx=0 tx=0).stop 2024-04-25T08:55:06.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.510+0000 7ff51e5e2640 1 --2- 172.21.15.43:0/791505560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5180d0190 0x7ff5180b7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:06.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.510+0000 7ff51e5e2640 1 --2- 172.21.15.43:0/791505560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff51806bd80 0x7ff5181f33e0 secure :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff50c000c00 tx=0x7ff50c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:55:06.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.510+0000 7ff51e5e2640 1 -- 172.21.15.43:0/791505560 >> 172.21.15.43:0/791505560 conn(0x7ff5180d39e0 msgr2=0x7ff5180b4490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:06.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.512+0000 7ff51e5e2640 1 -- 172.21.15.43:0/791505560 shutdown_connections 2024-04-25T08:55:06.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.517+0000 7ff51e5e2640 1 -- 172.21.15.43:0/791505560 wait complete. 2024-04-25T08:55:06.530 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T08:55:06.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.649+0000 7f301129e640 1 Processor -- start 2024-04-25T08:55:06.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.649+0000 7f301129e640 1 -- start start 2024-04-25T08:55:06.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.650+0000 7f301129e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f300c15e090 0x7f300c161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:06.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.650+0000 7f301129e640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f300c161fe0 0x7f300c0b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:06.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.650+0000 7f301129e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f300c162610 con 0x7f300c0b5aa0 2024-04-25T08:55:06.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.650+0000 7f301129e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f300c0d1da0 con 0x7f300c15e090 2024-04-25T08:55:06.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.650+0000 7f301129e640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f300c0d1750 con 0x7f300c161fe0 2024-04-25T08:55:06.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.650+0000 7f300affd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f300c15e090 0x7f300c161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:06.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.650+0000 7f300affd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f300c15e090 0x7f300c161aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42120/0 (socket says 172.21.15.43:42120) 2024-04-25T08:55:06.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.650+0000 7f300affd640 1 -- 172.21.15.43:0/2436517432 learned_addr learned my addr 172.21.15.43:0/2436517432 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:06.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.650+0000 7f300b7fe640 1 --1- 172.21.15.43:0/2436517432 >> v1:172.21.15.43:6789/0 conn(0x7f300c0b5aa0 0x7f300c0ce560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:55520/0 (socket says 172.21.15.43:55520) 2024-04-25T08:55:06.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.650+0000 7f3003fff640 1 --2- 172.21.15.43:0/2436517432 >> v2:172.21.15.193:3300/0 conn(0x7f300c161fe0 0x7f300c0b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:06.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.651+0000 7f300affd640 1 -- 172.21.15.43:0/2436517432 >> v2:172.21.15.193:3300/0 conn(0x7f300c161fe0 msgr2=0x7f300c0b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:06.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.651+0000 7f300affd640 1 --2- 172.21.15.43:0/2436517432 >> v2:172.21.15.193:3300/0 conn(0x7f300c161fe0 0x7f300c0b5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:06.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.651+0000 7f300affd640 1 -- 172.21.15.43:0/2436517432 >> v1:172.21.15.43:6789/0 conn(0x7f300c0b5aa0 legacy=0x7f300c0ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:06.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.651+0000 7f300a7fc640 1 -- 172.21.15.43:0/2436517432 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1883668246 0 0) 0x7f300c162610 con 0x7f300c0b5aa0 2024-04-25T08:55:06.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.651+0000 7f300affd640 1 -- 172.21.15.43:0/2436517432 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f300c0d1a30 con 0x7f300c15e090 2024-04-25T08:55:06.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.651+0000 7f300affd640 1 --2- 172.21.15.43:0/2436517432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f300c15e090 0x7f300c161aa0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f300c0cfe10 tx=0x7f2ffc02a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b0ed7cb6ab8e753 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:06.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.651+0000 7f300a7fc640 1 -- 172.21.15.43:0/2436517432 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc037070 con 0x7f300c15e090 2024-04-25T08:55:06.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.651+0000 7f300a7fc640 1 -- 172.21.15.43:0/2436517432 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ffc02ae50 con 0x7f300c15e090 2024-04-25T08:55:06.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.651+0000 7f300a7fc640 1 -- 172.21.15.43:0/2436517432 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc032860 con 0x7f300c15e090 2024-04-25T08:55:06.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.652+0000 7f301129e640 1 -- 172.21.15.43:0/2436517432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f300c15e090 msgr2=0x7f300c161aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:06.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.652+0000 7f301129e640 1 --2- 172.21.15.43:0/2436517432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f300c15e090 0x7f300c161aa0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f300c0cfe10 tx=0x7f2ffc02a300 comp rx=0 tx=0).stop 2024-04-25T08:55:06.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.652+0000 7f301129e640 1 -- 172.21.15.43:0/2436517432 shutdown_connections 2024-04-25T08:55:06.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.652+0000 7f301129e640 1 --2- 172.21.15.43:0/2436517432 >> v2:172.21.15.193:3300/0 conn(0x7f300c161fe0 0x7f300c0b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:06.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.652+0000 7f301129e640 1 --2- 172.21.15.43:0/2436517432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f300c15e090 0x7f300c161aa0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:06.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.652+0000 7f301129e640 1 -- 172.21.15.43:0/2436517432 >> 172.21.15.43:0/2436517432 conn(0x7f300c0d0560 msgr2=0x7f300c0d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:06.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.652+0000 7f301129e640 1 -- 172.21.15.43:0/2436517432 shutdown_connections 2024-04-25T08:55:06.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.652+0000 7f301129e640 1 -- 172.21.15.43:0/2436517432 wait complete. 2024-04-25T08:55:06.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.653+0000 7f301129e640 1 Processor -- start 2024-04-25T08:55:06.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.653+0000 7f301129e640 1 -- start start 2024-04-25T08:55:06.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.654+0000 7f301129e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f300c0b5aa0 0x7f300c1484a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:06.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.654+0000 7f301129e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f300c15e090 0x7f300c14ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:06.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.654+0000 7f301129e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f300c161fe0 0x7f300c14b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:06.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.654+0000 7f301129e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300c0d3240 con 0x7f300c0b5aa0 2024-04-25T08:55:06.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.654+0000 7f301129e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300c0b3280 con 0x7f300c161fe0 2024-04-25T08:55:06.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.654+0000 7f301129e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f300c0d33c0 con 0x7f300c15e090 2024-04-25T08:55:06.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.654+0000 7f300affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f300c0b5aa0 0x7f300c1484a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:06.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.654+0000 7f300affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f300c0b5aa0 0x7f300c1484a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42128/0 (socket says 172.21.15.43:42128) 2024-04-25T08:55:06.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.654+0000 7f300affd640 1 -- 172.21.15.43:0/3057720829 learned_addr learned my addr 172.21.15.43:0/3057720829 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:06.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.654+0000 7f300b7fe640 1 --2- 172.21.15.43:0/3057720829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f300c161fe0 0x7f300c14b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:06.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.654+0000 7f3003fff640 1 --2- 172.21.15.43:0/3057720829 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f300c15e090 0x7f300c14ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:06.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.654+0000 7f300affd640 1 -- 172.21.15.43:0/3057720829 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f300c15e090 msgr2=0x7f300c14ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:06.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.654+0000 7f300affd640 1 --2- 172.21.15.43:0/3057720829 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f300c15e090 0x7f300c14ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:06.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.654+0000 7f300affd640 1 -- 172.21.15.43:0/3057720829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f300c161fe0 msgr2=0x7f300c14b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:06.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.654+0000 7f300affd640 1 --2- 172.21.15.43:0/3057720829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f300c161fe0 0x7f300c14b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:06.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.654+0000 7f300affd640 1 -- 172.21.15.43:0/3057720829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f300c149ae0 con 0x7f300c0b5aa0 2024-04-25T08:55:06.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.655+0000 7f300affd640 1 --2- 172.21.15.43:0/3057720829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f300c0b5aa0 0x7f300c1484a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc000c00 tx=0x7f2ffc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:06.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.655+0000 7f30037fe640 1 -- 172.21.15.43:0/3057720829 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc042070 con 0x7f300c0b5aa0 2024-04-25T08:55:06.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.655+0000 7f30037fe640 1 -- 172.21.15.43:0/3057720829 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ffc03e670 con 0x7f300c0b5aa0 2024-04-25T08:55:06.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.655+0000 7f301129e640 1 -- 172.21.15.43:0/3057720829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f300c149dc0 con 0x7f300c0b5aa0 2024-04-25T08:55:06.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.655+0000 7f30037fe640 1 -- 172.21.15.43:0/3057720829 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc037040 con 0x7f300c0b5aa0 2024-04-25T08:55:06.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.655+0000 7f301129e640 1 -- 172.21.15.43:0/3057720829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f300c14a380 con 0x7f300c0b5aa0 2024-04-25T08:55:06.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.656+0000 7f301129e640 1 -- 172.21.15.43:0/3057720829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f300c06bf00 con 0x7f300c0b5aa0 2024-04-25T08:55:06.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.656+0000 7f30037fe640 1 -- 172.21.15.43:0/3057720829 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2ffc045070 con 0x7f300c0b5aa0 2024-04-25T08:55:06.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.657+0000 7f30037fe640 1 --2- 172.21.15.43:0/3057720829 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2fec06b500 0x7f2fec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:06.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.657+0000 7f30037fe640 1 -- 172.21.15.43:0/3057720829 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f2ffc0aa9c0 con 0x7f300c0b5aa0 2024-04-25T08:55:06.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.657+0000 7f3003fff640 1 --2- 172.21.15.43:0/3057720829 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2fec06b500 0x7f2fec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:06.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.658+0000 7f3003fff640 1 --2- 172.21.15.43:0/3057720829 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2fec06b500 0x7f2fec06d9c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4006fd0 tx=0x7f2ff4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:06.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.661+0000 7f30037fe640 1 -- 172.21.15.43:0/3057720829 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f300c06bf00 con 0x7f300c0b5aa0 2024-04-25T08:55:06.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:06.884+0000 7f301129e640 1 -- 172.21.15.43:0/3057720829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f300c074f60 con 0x7f300c0b5aa0 2024-04-25T08:55:07.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.508+0000 7f30037fe640 1 -- 172.21.15.43:0/3057720829 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v11) v1 ==== 172+0+0 (secure 0 0 0) 0x7f2ffc0ace20 con 0x7f300c0b5aa0 2024-04-25T08:55:07.497 INFO:teuthology.orchestra.run.smithi043.stderr:new fs with metadata pool 4 and data pool 5 2024-04-25T08:55:07.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.513+0000 7f301129e640 1 -- 172.21.15.43:0/3057720829 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2fec06b500 msgr2=0x7f2fec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:07.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.513+0000 7f301129e640 1 --2- 172.21.15.43:0/3057720829 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2fec06b500 0x7f2fec06d9c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2ff4006fd0 tx=0x7f2ff4008040 comp rx=0 tx=0).stop 2024-04-25T08:55:07.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.513+0000 7f301129e640 1 -- 172.21.15.43:0/3057720829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f300c0b5aa0 msgr2=0x7f300c1484a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:07.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.513+0000 7f301129e640 1 --2- 172.21.15.43:0/3057720829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f300c0b5aa0 0x7f300c1484a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc000c00 tx=0x7f2ffc0046c0 comp rx=0 tx=0).stop 2024-04-25T08:55:07.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.513+0000 7f301129e640 1 -- 172.21.15.43:0/3057720829 shutdown_connections 2024-04-25T08:55:07.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.513+0000 7f301129e640 1 --2- 172.21.15.43:0/3057720829 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2fec06b500 0x7f2fec06d9c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:07.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.513+0000 7f301129e640 1 --2- 172.21.15.43:0/3057720829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f300c161fe0 0x7f300c14b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:07.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.513+0000 7f301129e640 1 --2- 172.21.15.43:0/3057720829 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f300c15e090 0x7f300c14ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:07.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.513+0000 7f301129e640 1 --2- 172.21.15.43:0/3057720829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f300c0b5aa0 0x7f300c1484a0 secure :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc000c00 tx=0x7f2ffc0046c0 comp rx=0 tx=0).stop 2024-04-25T08:55:07.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.513+0000 7f301129e640 1 -- 172.21.15.43:0/3057720829 >> 172.21.15.43:0/3057720829 conn(0x7f300c0d0560 msgr2=0x7f300c0b70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:07.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.515+0000 7f301129e640 1 -- 172.21.15.43:0/3057720829 shutdown_connections 2024-04-25T08:55:07.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.516+0000 7f301129e640 1 -- 172.21.15.43:0/3057720829 wait complete. 2024-04-25T08:55:07.534 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:55:07.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.647+0000 7f0dc2ecb640 1 Processor -- start 2024-04-25T08:55:07.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.648+0000 7f0dc2ecb640 1 -- start start 2024-04-25T08:55:07.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.648+0000 7f0dc2ecb640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0dbc15df50 0x7f0dbc1619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:07.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.648+0000 7f0dc2ecb640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dbc0d24a0 con 0x7f0dbc161f30 2024-04-25T08:55:07.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.648+0000 7f0dc2ecb640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dbc0d2740 con 0x7f0dbc0cf670 2024-04-25T08:55:07.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.648+0000 7f0dc2ecb640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc0cfa80 con 0x7f0dbc15df50 2024-04-25T08:55:07.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.648+0000 7f0dbbfff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f0dbc161f30 0x7f0dbc0c7cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:55522/0 (socket says 172.21.15.43:55522) 2024-04-25T08:55:07.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.648+0000 7f0dbbfff640 1 -- 172.21.15.43:0/750087159 learned_addr learned my addr 172.21.15.43:0/750087159 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:55:07.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.649+0000 7f0dbb7fe640 1 -- 172.21.15.43:0/750087159 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4253031074 0 0) 0x7f0dbc0d24a0 con 0x7f0dbc161f30 2024-04-25T08:55:07.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.649+0000 7f0dbb7fe640 1 -- 172.21.15.43:0/750087159 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0da4003520 con 0x7f0dbc161f30 2024-04-25T08:55:07.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.649+0000 7f0dc0c67640 1 --2- 172.21.15.43:0/750087159 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0dbc15df50 0x7f0dbc1619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:07.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.649+0000 7f0dbb7fe640 1 -- 172.21.15.43:0/750087159 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3153220976 0 0) 0x7f0da4003520 con 0x7f0dbc161f30 2024-04-25T08:55:07.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.649+0000 7f0dbb7fe640 1 -- 172.21.15.43:0/750087159 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0dbc15df50 msgr2=0x7f0dbc1619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:07.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.649+0000 7f0dbb7fe640 1 --2- 172.21.15.43:0/750087159 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0dbc15df50 0x7f0dbc1619f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:07.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.649+0000 7f0dbb7fe640 1 -- 172.21.15.43:0/750087159 >> v1:172.21.15.193:6789/0 conn(0x7f0dbc0cf670 legacy=0x7f0dbc0c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:07.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.649+0000 7f0dbb7fe640 1 -- 172.21.15.43:0/750087159 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dbc0d2a20 con 0x7f0dbc161f30 2024-04-25T08:55:07.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.649+0000 7f0dbb7fe640 1 -- 172.21.15.43:0/750087159 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f0db0001670 con 0x7f0dbc161f30 2024-04-25T08:55:07.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.649+0000 7f0dbb7fe640 1 -- 172.21.15.43:0/750087159 >> v1:172.21.15.43:6789/0 conn(0x7f0dbc161f30 legacy=0x7f0dbc0c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:07.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.649+0000 7f0dbb7fe640 1 --2- 172.21.15.43:0/750087159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0da40039e0 0x7f0da4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:07.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.649+0000 7f0dbb7fe640 1 --2- 172.21.15.43:0/750087159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0da4004800 0x7f0da4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:07.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.649+0000 7f0dbb7fe640 1 --2- 172.21.15.43:0/750087159 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0dbc15df50 0x7f0da40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:07.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.649+0000 7f0dbb7fe640 1 -- 172.21.15.43:0/750087159 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc0cfa80 con 0x7f0da4004800 2024-04-25T08:55:07.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.649+0000 7f0dbb7fe640 1 -- 172.21.15.43:0/750087159 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da400ba70 con 0x7f0da40039e0 2024-04-25T08:55:07.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.649+0000 7f0dbb7fe640 1 -- 172.21.15.43:0/750087159 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0da400bc70 con 0x7f0dbc15df50 2024-04-25T08:55:07.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.649+0000 7f0dbbfff640 1 --2- 172.21.15.43:0/750087159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0da4004800 0x7f0da4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:07.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.649+0000 7f0dc0c67640 1 --2- 172.21.15.43:0/750087159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0da40039e0 0x7f0da4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:07.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.649+0000 7f0dc1468640 1 --2- 172.21.15.43:0/750087159 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0dbc15df50 0x7f0da40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:07.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.650+0000 7f0dbbfff640 1 -- 172.21.15.43:0/750087159 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0dbc15df50 msgr2=0x7f0da40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:07.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.650+0000 7f0dbbfff640 1 --2- 172.21.15.43:0/750087159 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0dbc15df50 0x7f0da40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:07.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.650+0000 7f0dbbfff640 1 -- 172.21.15.43:0/750087159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0da40039e0 msgr2=0x7f0da4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:07.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.650+0000 7f0dbbfff640 1 --2- 172.21.15.43:0/750087159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0da40039e0 0x7f0da4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:07.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.650+0000 7f0dbbfff640 1 -- 172.21.15.43:0/750087159 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da4003520 con 0x7f0da4004800 2024-04-25T08:55:07.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.650+0000 7f0dc1468640 1 --2- 172.21.15.43:0/750087159 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0dbc15df50 0x7f0da40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:55:07.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.650+0000 7f0dbbfff640 1 --2- 172.21.15.43:0/750087159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0da4004800 0x7f0da4004c80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0db0001eb0 tx=0x7f0da4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=821f08887e58a1f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:07.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.650+0000 7f0dbb7fe640 1 -- 172.21.15.43:0/750087159 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db0007d30 con 0x7f0da4004800 2024-04-25T08:55:07.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.650+0000 7f0dbb7fe640 1 -- 172.21.15.43:0/750087159 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0db0005af0 con 0x7f0da4004800 2024-04-25T08:55:07.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.650+0000 7f0dbb7fe640 1 -- 172.21.15.43:0/750087159 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0db0007070 con 0x7f0da4004800 2024-04-25T08:55:07.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.651+0000 7f0dc2ecb640 1 -- 172.21.15.43:0/750087159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0da4004800 msgr2=0x7f0da4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:07.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.651+0000 7f0dc2ecb640 1 --2- 172.21.15.43:0/750087159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0da4004800 0x7f0da4004c80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f0db0001eb0 tx=0x7f0da4005830 comp rx=0 tx=0).stop 2024-04-25T08:55:07.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.651+0000 7f0dc1468640 1 -- 172.21.15.43:0/750087159 reap_dead start 2024-04-25T08:55:07.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.651+0000 7f0dc2ecb640 1 -- 172.21.15.43:0/750087159 shutdown_connections 2024-04-25T08:55:07.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.651+0000 7f0dc2ecb640 1 -- 172.21.15.43:0/750087159 >> 172.21.15.43:0/750087159 conn(0x7f0dbc0d3a30 msgr2=0x7f0dbc0d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:07.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.651+0000 7f0dc2ecb640 1 -- 172.21.15.43:0/750087159 shutdown_connections 2024-04-25T08:55:07.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.651+0000 7f0dc2ecb640 1 -- 172.21.15.43:0/750087159 wait complete. 2024-04-25T08:55:07.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.652+0000 7f0dc2ecb640 1 Processor -- start 2024-04-25T08:55:07.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.652+0000 7f0dc2ecb640 1 -- start start 2024-04-25T08:55:07.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.653+0000 7f0dc2ecb640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0da4004800 0x7f0dbc0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:07.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.653+0000 7f0dc2ecb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dbc0cbac0 0x7f0dbc15a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:07.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.653+0000 7f0dc2ecb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dbc15a6b0 0x7f0dbc141c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:07.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.653+0000 7f0dc2ecb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc0c77d0 con 0x7f0dbc15a6b0 2024-04-25T08:55:07.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.653+0000 7f0dc2ecb640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc15e330 con 0x7f0dbc0cbac0 2024-04-25T08:55:07.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.653+0000 7f0dc2ecb640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc0d2190 con 0x7f0da4004800 2024-04-25T08:55:07.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.653+0000 7f0dc1468640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dbc15a6b0 0x7f0dbc141c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:07.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.653+0000 7f0dc1468640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dbc15a6b0 0x7f0dbc141c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42148/0 (socket says 172.21.15.43:42148) 2024-04-25T08:55:07.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.653+0000 7f0dc1468640 1 -- 172.21.15.43:0/1920921756 learned_addr learned my addr 172.21.15.43:0/1920921756 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:07.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.653+0000 7f0dbbfff640 1 --2- 172.21.15.43:0/1920921756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dbc0cbac0 0x7f0dbc15a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:07.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.653+0000 7f0dc0c67640 1 --2- 172.21.15.43:0/1920921756 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0da4004800 0x7f0dbc0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:07.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.653+0000 7f0dc1468640 1 -- 172.21.15.43:0/1920921756 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0da4004800 msgr2=0x7f0dbc0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:07.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.653+0000 7f0dc1468640 1 --2- 172.21.15.43:0/1920921756 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0da4004800 0x7f0dbc0ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:07.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.654+0000 7f0dc1468640 1 -- 172.21.15.43:0/1920921756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dbc0cbac0 msgr2=0x7f0dbc15a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:07.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.654+0000 7f0dc1468640 1 --2- 172.21.15.43:0/1920921756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dbc0cbac0 0x7f0dbc15a170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:07.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.654+0000 7f0dc1468640 1 -- 172.21.15.43:0/1920921756 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dbc0cbf50 con 0x7f0dbc15a6b0 2024-04-25T08:55:07.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.654+0000 7f0dc0c67640 1 --2- 172.21.15.43:0/1920921756 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0da4004800 0x7f0dbc0ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:07.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.654+0000 7f0dc1468640 1 --2- 172.21.15.43:0/1920921756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dbc15a6b0 0x7f0dbc141c60 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc0d39e0 tx=0x7f0dac00e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:07.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.654+0000 7f0db97fa640 1 -- 172.21.15.43:0/1920921756 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dac019070 con 0x7f0dbc15a6b0 2024-04-25T08:55:07.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.654+0000 7f0db97fa640 1 -- 172.21.15.43:0/1920921756 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dac0052e0 con 0x7f0dbc15a6b0 2024-04-25T08:55:07.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.654+0000 7f0dc2ecb640 1 -- 172.21.15.43:0/1920921756 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dbc0cc230 con 0x7f0dbc15a6b0 2024-04-25T08:55:07.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.654+0000 7f0db97fa640 1 -- 172.21.15.43:0/1920921756 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dac003830 con 0x7f0dbc15a6b0 2024-04-25T08:55:07.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.654+0000 7f0dc2ecb640 1 -- 172.21.15.43:0/1920921756 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dbc0cc7f0 con 0x7f0dbc15a6b0 2024-04-25T08:55:07.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.655+0000 7f0dc2ecb640 1 -- 172.21.15.43:0/1920921756 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dbc0680f0 con 0x7f0dbc15a6b0 2024-04-25T08:55:07.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.655+0000 7f0db97fa640 1 -- 172.21.15.43:0/1920921756 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0dac01c070 con 0x7f0dbc15a6b0 2024-04-25T08:55:07.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.656+0000 7f0db97fa640 1 --2- 172.21.15.43:0/1920921756 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0da806b4d0 0x7f0da806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:07.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.656+0000 7f0db97fa640 1 -- 172.21.15.43:0/1920921756 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0dac089590 con 0x7f0dbc15a6b0 2024-04-25T08:55:07.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.656+0000 7f0dc0c67640 1 --2- 172.21.15.43:0/1920921756 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0da806b4d0 0x7f0da806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:07.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.657+0000 7f0dc0c67640 1 --2- 172.21.15.43:0/1920921756 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0da806b4d0 0x7f0da806d990 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0db4007440 tx=0x7f0db400b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:07.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.660+0000 7f0db97fa640 1 -- 172.21.15.43:0/1920921756 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0dac0584a0 con 0x7f0dbc15a6b0 2024-04-25T08:55:07.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.841+0000 7f0dc2ecb640 1 -- 172.21.15.43:0/1920921756 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0dbc074ef0 con 0x7f0dbc15a6b0 2024-04-25T08:55:07.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.842+0000 7f0db97fa640 1 -- 172.21.15.43:0/1920921756 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16646 (secure 0 0 0) 0x7f0dac057da0 con 0x7f0dbc15a6b0 2024-04-25T08:55:07.831 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:55:07.831 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":32,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:55:06.884709+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:55:05.150275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T08:55:05.853905+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:55:07.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.844+0000 7f0dc2ecb640 1 -- 172.21.15.43:0/1920921756 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0da806b4d0 msgr2=0x7f0da806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:07.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.844+0000 7f0dc2ecb640 1 --2- 172.21.15.43:0/1920921756 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0da806b4d0 0x7f0da806d990 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0db4007440 tx=0x7f0db400b040 comp rx=0 tx=0).stop 2024-04-25T08:55:07.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.844+0000 7f0dc2ecb640 1 -- 172.21.15.43:0/1920921756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dbc15a6b0 msgr2=0x7f0dbc141c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:07.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.844+0000 7f0dc2ecb640 1 --2- 172.21.15.43:0/1920921756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dbc15a6b0 0x7f0dbc141c60 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc0d39e0 tx=0x7f0dac00e820 comp rx=0 tx=0).stop 2024-04-25T08:55:07.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.844+0000 7f0dc2ecb640 1 -- 172.21.15.43:0/1920921756 shutdown_connections 2024-04-25T08:55:07.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.844+0000 7f0dc2ecb640 1 --2- 172.21.15.43:0/1920921756 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0dbc15a6b0 0x7f0dbc141c60 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:07.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.844+0000 7f0dc2ecb640 1 --2- 172.21.15.43:0/1920921756 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0da806b4d0 0x7f0da806d990 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:07.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.844+0000 7f0dc2ecb640 1 --2- 172.21.15.43:0/1920921756 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0dbc0cbac0 0x7f0dbc15a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:07.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.844+0000 7f0dc2ecb640 1 --2- 172.21.15.43:0/1920921756 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0da4004800 0x7f0dbc0ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:07.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.844+0000 7f0dc2ecb640 1 -- 172.21.15.43:0/1920921756 >> 172.21.15.43:0/1920921756 conn(0x7f0dbc0d3a30 msgr2=0x7f0dbc0d0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:07.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.844+0000 7f0dc2ecb640 1 -- 172.21.15.43:0/1920921756 shutdown_connections 2024-04-25T08:55:07.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.844+0000 7f0dc2ecb640 1 -- 172.21.15.43:0/1920921756 wait complete. 2024-04-25T08:55:07.844 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:55:07.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.956+0000 7fa710342640 1 Processor -- start 2024-04-25T08:55:07.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.956+0000 7fa710342640 1 -- start start 2024-04-25T08:55:07.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.957+0000 7fa710342640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fa70815f180 0x7fa7080cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:07.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.957+0000 7fa710342640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7080b23f0 0x7fa7080cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:07.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.957+0000 7fa710342640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7080ccf10 con 0x7fa7080b2cc0 2024-04-25T08:55:07.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.957+0000 7fa710342640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7080cccc0 con 0x7fa70815f180 2024-04-25T08:55:07.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.957+0000 7fa710342640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7080cd840 con 0x7fa7080b23f0 2024-04-25T08:55:07.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.958+0000 7fa70e0de640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fa70815f180 0x7fa7080cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:07.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.958+0000 7fa70d8dd640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7080b23f0 0x7fa7080cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:07.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.958+0000 7fa70e0de640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fa70815f180 0x7fa7080cf860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:54050/0 (socket says 172.21.15.43:54050) 2024-04-25T08:55:07.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.958+0000 7fa70e0de640 1 -- 172.21.15.43:0/819143568 learned_addr learned my addr 172.21.15.43:0/819143568 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:07.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.958+0000 7fa70e0de640 1 -- 172.21.15.43:0/819143568 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7080b23f0 msgr2=0x7fa7080cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:07.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.958+0000 7fa70e0de640 1 --2- 172.21.15.43:0/819143568 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7080b23f0 0x7fa7080cfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:07.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.958+0000 7fa70e0de640 1 -- 172.21.15.43:0/819143568 >> v1:172.21.15.193:6789/0 conn(0x7fa7080b2cc0 legacy=0x7fa70815f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:07.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.958+0000 7fa70e0de640 1 -- 172.21.15.43:0/819143568 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7080cd1f0 con 0x7fa70815f180 2024-04-25T08:55:07.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.959+0000 7fa70e0de640 1 --2- 172.21.15.43:0/819143568 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa70815f180 0x7fa7080cf860 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa700009870 tx=0x7fa70002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4379419a8da21d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:07.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.959+0000 7fa70d0dc640 1 -- 172.21.15.43:0/819143568 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa700037070 con 0x7fa70815f180 2024-04-25T08:55:07.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.959+0000 7fa70d0dc640 1 -- 172.21.15.43:0/819143568 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa700032440 con 0x7fa70815f180 2024-04-25T08:55:07.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.959+0000 7fa70d0dc640 1 -- 172.21.15.43:0/819143568 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa700032810 con 0x7fa70815f180 2024-04-25T08:55:07.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.960+0000 7fa710342640 1 -- 172.21.15.43:0/819143568 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa70815f180 msgr2=0x7fa7080cf860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:07.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.960+0000 7fa710342640 1 --2- 172.21.15.43:0/819143568 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa70815f180 0x7fa7080cf860 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa700009870 tx=0x7fa70002a1c0 comp rx=0 tx=0).stop 2024-04-25T08:55:07.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.960+0000 7fa710342640 1 -- 172.21.15.43:0/819143568 shutdown_connections 2024-04-25T08:55:07.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.960+0000 7fa710342640 1 --2- 172.21.15.43:0/819143568 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7080b23f0 0x7fa7080cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:07.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.960+0000 7fa710342640 1 --2- 172.21.15.43:0/819143568 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa70815f180 0x7fa7080cf860 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:07.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.960+0000 7fa710342640 1 -- 172.21.15.43:0/819143568 >> 172.21.15.43:0/819143568 conn(0x7fa7080d17f0 msgr2=0x7fa7080d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:07.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.960+0000 7fa710342640 1 -- 172.21.15.43:0/819143568 shutdown_connections 2024-04-25T08:55:07.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.960+0000 7fa710342640 1 -- 172.21.15.43:0/819143568 wait complete. 2024-04-25T08:55:07.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.961+0000 7fa710342640 1 Processor -- start 2024-04-25T08:55:07.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.961+0000 7fa710342640 1 -- start start 2024-04-25T08:55:07.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.961+0000 7fa710342640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7080b23f0 0x7fa7080af160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:07.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.961+0000 7fa710342640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7080b2cc0 0x7fa708156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:07.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.962+0000 7fa710342640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa70815f180 0x7fa7080c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:07.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.962+0000 7fa710342640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa70815f560 con 0x7fa7080b23f0 2024-04-25T08:55:07.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.962+0000 7fa710342640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7080cece0 con 0x7fa70815f180 2024-04-25T08:55:07.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.962+0000 7fa710342640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7080ca450 con 0x7fa7080b2cc0 2024-04-25T08:55:07.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.962+0000 7fa70e0de640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7080b23f0 0x7fa7080af160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:07.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.962+0000 7fa70e0de640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7080b23f0 0x7fa7080af160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42152/0 (socket says 172.21.15.43:42152) 2024-04-25T08:55:07.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.962+0000 7fa70e0de640 1 -- 172.21.15.43:0/239269337 learned_addr learned my addr 172.21.15.43:0/239269337 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:07.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.962+0000 7fa70e8df640 1 --2- 172.21.15.43:0/239269337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa70815f180 0x7fa7080c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:07.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.962+0000 7fa70d8dd640 1 --2- 172.21.15.43:0/239269337 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7080b2cc0 0x7fa708156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:07.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.962+0000 7fa70e0de640 1 -- 172.21.15.43:0/239269337 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7080b2cc0 msgr2=0x7fa708156de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:07.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.962+0000 7fa70e0de640 1 --2- 172.21.15.43:0/239269337 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7080b2cc0 0x7fa708156de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:07.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.962+0000 7fa70e0de640 1 -- 172.21.15.43:0/239269337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa70815f180 msgr2=0x7fa7080c9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:07.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.962+0000 7fa70e0de640 1 --2- 172.21.15.43:0/239269337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa70815f180 0x7fa7080c9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:07.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.962+0000 7fa70e0de640 1 -- 172.21.15.43:0/239269337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7080af740 con 0x7fa7080b23f0 2024-04-25T08:55:07.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.962+0000 7fa70e8df640 1 --2- 172.21.15.43:0/239269337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa70815f180 0x7fa7080c9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:55:07.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.962+0000 7fa70e0de640 1 --2- 172.21.15.43:0/239269337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7080b23f0 0x7fa7080af160 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa70002a200 tx=0x7fa7000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:07.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.962+0000 7fa706ffd640 1 -- 172.21.15.43:0/239269337 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa700037040 con 0x7fa7080b23f0 2024-04-25T08:55:07.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.962+0000 7fa706ffd640 1 -- 172.21.15.43:0/239269337 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa700042070 con 0x7fa7080b23f0 2024-04-25T08:55:07.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.962+0000 7fa706ffd640 1 -- 172.21.15.43:0/239269337 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa700030860 con 0x7fa7080b23f0 2024-04-25T08:55:07.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.962+0000 7fa710342640 1 -- 172.21.15.43:0/239269337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7080afa20 con 0x7fa7080b23f0 2024-04-25T08:55:07.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.963+0000 7fa710342640 1 -- 172.21.15.43:0/239269337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7080cabd0 con 0x7fa7080b23f0 2024-04-25T08:55:07.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.963+0000 7fa710342640 1 -- 172.21.15.43:0/239269337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa708074ef0 con 0x7fa7080b23f0 2024-04-25T08:55:07.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.964+0000 7fa706ffd640 1 -- 172.21.15.43:0/239269337 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa70002ad10 con 0x7fa7080b23f0 2024-04-25T08:55:07.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.964+0000 7fa706ffd640 1 --2- 172.21.15.43:0/239269337 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa6f406b500 0x7fa6f406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:07.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.964+0000 7fa706ffd640 1 -- 172.21.15.43:0/239269337 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa7000aa850 con 0x7fa7080b23f0 2024-04-25T08:55:07.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.964+0000 7fa70d8dd640 1 --2- 172.21.15.43:0/239269337 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa6f406b500 0x7fa6f406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:07.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.965+0000 7fa70d8dd640 1 --2- 172.21.15.43:0/239269337 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa6f406b500 0x7fa6f406d9c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa7080d0b10 tx=0x7fa6fc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:07.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:07.969+0000 7fa706ffd640 1 -- 172.21.15.43:0/239269337 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa700079640 con 0x7fa7080b23f0 2024-04-25T08:55:08.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.152+0000 7fa710342640 1 -- 172.21.15.43:0/239269337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa7080cf050 con 0x7fa7080b23f0 2024-04-25T08:55:08.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.153+0000 7fa706ffd640 1 -- 172.21.15.43:0/239269337 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+16646 (secure 0 0 0) 0x7fa700078f40 con 0x7fa7080b23f0 2024-04-25T08:55:08.142 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:55:08.142 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":32,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:55:06.884709+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:55:05.150275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T08:55:05.853905+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:55:08.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.155+0000 7fa710342640 1 -- 172.21.15.43:0/239269337 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa6f406b500 msgr2=0x7fa6f406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:08.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.155+0000 7fa710342640 1 --2- 172.21.15.43:0/239269337 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa6f406b500 0x7fa6f406d9c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa7080d0b10 tx=0x7fa6fc008040 comp rx=0 tx=0).stop 2024-04-25T08:55:08.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.155+0000 7fa710342640 1 -- 172.21.15.43:0/239269337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7080b23f0 msgr2=0x7fa7080af160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:08.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.155+0000 7fa710342640 1 --2- 172.21.15.43:0/239269337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7080b23f0 0x7fa7080af160 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa70002a200 tx=0x7fa7000046c0 comp rx=0 tx=0).stop 2024-04-25T08:55:08.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.155+0000 7fa710342640 1 -- 172.21.15.43:0/239269337 shutdown_connections 2024-04-25T08:55:08.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.155+0000 7fa710342640 1 --2- 172.21.15.43:0/239269337 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa6f406b500 0x7fa6f406d9c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:08.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.155+0000 7fa710342640 1 --2- 172.21.15.43:0/239269337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa70815f180 0x7fa7080c9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:08.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.155+0000 7fa710342640 1 --2- 172.21.15.43:0/239269337 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7080b2cc0 0x7fa708156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:08.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.155+0000 7fa710342640 1 --2- 172.21.15.43:0/239269337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7080b23f0 0x7fa7080af160 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:08.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.155+0000 7fa710342640 1 -- 172.21.15.43:0/239269337 >> 172.21.15.43:0/239269337 conn(0x7fa7080d17f0 msgr2=0x7fa7080b4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:08.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.155+0000 7fa710342640 1 -- 172.21.15.43:0/239269337 shutdown_connections 2024-04-25T08:55:08.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.155+0000 7fa710342640 1 -- 172.21.15.43:0/239269337 wait complete. 2024-04-25T08:55:08.155 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T08:55:08.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.269+0000 7f77349d5640 1 Processor -- start 2024-04-25T08:55:08.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.269+0000 7f77349d5640 1 -- start start 2024-04-25T08:55:08.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.270+0000 7f77349d5640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f77300d04a0 0x7f773015ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:08.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.270+0000 7f77349d5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f773015dec0 0x7f77300d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:08.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.270+0000 7f77349d5640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77300d0dc0 con 0x7f77300ce8c0 2024-04-25T08:55:08.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.270+0000 7f77349d5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77300cf110 con 0x7f773015dec0 2024-04-25T08:55:08.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.270+0000 7f77349d5640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77300cf3f0 con 0x7f77300d04a0 2024-04-25T08:55:08.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.270+0000 7f772dd9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f773015dec0 0x7f77300d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:08.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.270+0000 7f772dd9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f773015dec0 0x7f77300d0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42154/0 (socket says 172.21.15.43:42154) 2024-04-25T08:55:08.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.270+0000 7f772dd9b640 1 -- 172.21.15.43:0/1886890104 learned_addr learned my addr 172.21.15.43:0/1886890104 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:08.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.270+0000 7f772dd9b640 1 -- 172.21.15.43:0/1886890104 >> v2:172.21.15.193:3300/0 conn(0x7f77300d04a0 msgr2=0x7f773015ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:08.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.270+0000 7f772dd9b640 1 --2- 172.21.15.43:0/1886890104 >> v2:172.21.15.193:3300/0 conn(0x7f77300d04a0 0x7f773015ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:08.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.270+0000 7f772dd9b640 1 -- 172.21.15.43:0/1886890104 >> v1:172.21.15.193:6789/0 conn(0x7f77300ce8c0 legacy=0x7f77301617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:08.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.271+0000 7f772dd9b640 1 -- 172.21.15.43:0/1886890104 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77300cf6d0 con 0x7f773015dec0 2024-04-25T08:55:08.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.271+0000 7f772dd9b640 1 --2- 172.21.15.43:0/1886890104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f773015dec0 0x7f77300d0880 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f77300b6690 tx=0x7f772402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3325f00332c9a1a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:08.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.271+0000 7f772d59a640 1 -- 172.21.15.43:0/1886890104 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7724037070 con 0x7f773015dec0 2024-04-25T08:55:08.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.271+0000 7f772d59a640 1 -- 172.21.15.43:0/1886890104 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7724032440 con 0x7f773015dec0 2024-04-25T08:55:08.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.271+0000 7f772d59a640 1 -- 172.21.15.43:0/1886890104 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7724032810 con 0x7f773015dec0 2024-04-25T08:55:08.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.272+0000 7f77349d5640 1 -- 172.21.15.43:0/1886890104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f773015dec0 msgr2=0x7f77300d0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:08.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.272+0000 7f77349d5640 1 --2- 172.21.15.43:0/1886890104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f773015dec0 0x7f77300d0880 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f77300b6690 tx=0x7f772402a230 comp rx=0 tx=0).stop 2024-04-25T08:55:08.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.272+0000 7f77349d5640 1 -- 172.21.15.43:0/1886890104 shutdown_connections 2024-04-25T08:55:08.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.272+0000 7f77349d5640 1 --2- 172.21.15.43:0/1886890104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f773015dec0 0x7f77300d0880 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:08.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.272+0000 7f77349d5640 1 --2- 172.21.15.43:0/1886890104 >> v2:172.21.15.193:3300/0 conn(0x7f77300d04a0 0x7f773015ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:08.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.272+0000 7f77349d5640 1 -- 172.21.15.43:0/1886890104 >> 172.21.15.43:0/1886890104 conn(0x7f773009cd40 msgr2=0x7f77300d3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:08.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.272+0000 7f77349d5640 1 -- 172.21.15.43:0/1886890104 shutdown_connections 2024-04-25T08:55:08.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.272+0000 7f77349d5640 1 -- 172.21.15.43:0/1886890104 wait complete. 2024-04-25T08:55:08.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.273+0000 7f77349d5640 1 Processor -- start 2024-04-25T08:55:08.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.273+0000 7f77349d5640 1 -- start start 2024-04-25T08:55:08.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.274+0000 7f77349d5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77300ce8c0 0x7f77301597c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:08.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.274+0000 7f77349d5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77300d04a0 0x7f7730154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:08.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.274+0000 7f77349d5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f773015dec0 0x7f77300b5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:08.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.274+0000 7f77349d5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77300ced10 con 0x7f773015dec0 2024-04-25T08:55:08.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.274+0000 7f77349d5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77300cfc90 con 0x7f77300d04a0 2024-04-25T08:55:08.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.274+0000 7f77349d5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77300cee90 con 0x7f77300ce8c0 2024-04-25T08:55:08.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.274+0000 7f772ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f773015dec0 0x7f77300b5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:08.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.274+0000 7f772ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f773015dec0 0x7f77300b5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42160/0 (socket says 172.21.15.43:42160) 2024-04-25T08:55:08.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.274+0000 7f772ed9d640 1 -- 172.21.15.43:0/1434263992 learned_addr learned my addr 172.21.15.43:0/1434263992 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:08.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.274+0000 7f772dd9b640 1 --2- 172.21.15.43:0/1434263992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77300d04a0 0x7f7730154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:08.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.275+0000 7f772ed9d640 1 -- 172.21.15.43:0/1434263992 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77300ce8c0 msgr2=0x7f77301597c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:08.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.275+0000 7f772e59c640 1 --2- 172.21.15.43:0/1434263992 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77300ce8c0 0x7f77301597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:08.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.275+0000 7f772ed9d640 1 --2- 172.21.15.43:0/1434263992 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77300ce8c0 0x7f77301597c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:08.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.275+0000 7f772ed9d640 1 -- 172.21.15.43:0/1434263992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77300d04a0 msgr2=0x7f7730154e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:08.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.275+0000 7f772ed9d640 1 --2- 172.21.15.43:0/1434263992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77300d04a0 0x7f7730154e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:08.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.275+0000 7f772ed9d640 1 -- 172.21.15.43:0/1434263992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77300b8910 con 0x7f773015dec0 2024-04-25T08:55:08.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.275+0000 7f772dd9b640 1 --2- 172.21.15.43:0/1434263992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77300d04a0 0x7f7730154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:08.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.275+0000 7f772e59c640 1 --2- 172.21.15.43:0/1434263992 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77300ce8c0 0x7f77301597c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:08.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.275+0000 7f772ed9d640 1 --2- 172.21.15.43:0/1434263992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f773015dec0 0x7f77300b5250 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f772800c390 tx=0x7f772800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:08.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.275+0000 7f771effd640 1 -- 172.21.15.43:0/1434263992 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7728017070 con 0x7f773015dec0 2024-04-25T08:55:08.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.275+0000 7f771effd640 1 -- 172.21.15.43:0/1434263992 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77280035f0 con 0x7f773015dec0 2024-04-25T08:55:08.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.275+0000 7f77349d5640 1 -- 172.21.15.43:0/1434263992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77300b30b0 con 0x7f773015dec0 2024-04-25T08:55:08.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.275+0000 7f771effd640 1 -- 172.21.15.43:0/1434263992 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77280039e0 con 0x7f773015dec0 2024-04-25T08:55:08.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.275+0000 7f77349d5640 1 -- 172.21.15.43:0/1434263992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77300b8bf0 con 0x7f773015dec0 2024-04-25T08:55:08.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.276+0000 7f77349d5640 1 -- 172.21.15.43:0/1434263992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7730074f60 con 0x7f773015dec0 2024-04-25T08:55:08.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.277+0000 7f771effd640 1 -- 172.21.15.43:0/1434263992 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f772801a070 con 0x7f773015dec0 2024-04-25T08:55:08.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.277+0000 7f771effd640 1 --2- 172.21.15.43:0/1434263992 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f771006b4d0 0x7f771006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:08.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.277+0000 7f771effd640 1 -- 172.21.15.43:0/1434263992 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7728089600 con 0x7f773015dec0 2024-04-25T08:55:08.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.277+0000 7f772e59c640 1 --2- 172.21.15.43:0/1434263992 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f771006b4d0 0x7f771006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:08.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.278+0000 7f772e59c640 1 --2- 172.21.15.43:0/1434263992 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f771006b4d0 0x7f771006d990 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7720007eb0 tx=0x7f77200073c0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:08.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.281+0000 7f771effd640 1 -- 172.21.15.43:0/1434263992 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7728058470 con 0x7f773015dec0 2024-04-25T08:55:08.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.497+0000 7f77349d5640 1 -- 172.21.15.43:0/1434263992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f77300d09c0 con 0x7f773015dec0 2024-04-25T08:55:08.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.528+0000 7f771effd640 1 -- 172.21.15.43:0/1434263992 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v12) v1 ==== 118+0+0 (secure 0 0 0) 0x7f7728057d70 con 0x7f773015dec0 2024-04-25T08:55:08.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.530+0000 7f77349d5640 1 -- 172.21.15.43:0/1434263992 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f771006b4d0 msgr2=0x7f771006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:08.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.530+0000 7f77349d5640 1 --2- 172.21.15.43:0/1434263992 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f771006b4d0 0x7f771006d990 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7720007eb0 tx=0x7f77200073c0 comp rx=0 tx=0).stop 2024-04-25T08:55:08.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.530+0000 7f77349d5640 1 -- 172.21.15.43:0/1434263992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f773015dec0 msgr2=0x7f77300b5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:08.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.530+0000 7f77349d5640 1 --2- 172.21.15.43:0/1434263992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f773015dec0 0x7f77300b5250 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f772800c390 tx=0x7f772800c850 comp rx=0 tx=0).stop 2024-04-25T08:55:08.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.530+0000 7f77349d5640 1 -- 172.21.15.43:0/1434263992 shutdown_connections 2024-04-25T08:55:08.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.530+0000 7f77349d5640 1 --2- 172.21.15.43:0/1434263992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f773015dec0 0x7f77300b5250 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:08.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.530+0000 7f77349d5640 1 --2- 172.21.15.43:0/1434263992 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f771006b4d0 0x7f771006d990 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:08.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.530+0000 7f77349d5640 1 --2- 172.21.15.43:0/1434263992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77300d04a0 0x7f7730154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:08.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.530+0000 7f77349d5640 1 --2- 172.21.15.43:0/1434263992 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77300ce8c0 0x7f77301597c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:08.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.530+0000 7f77349d5640 1 -- 172.21.15.43:0/1434263992 >> 172.21.15.43:0/1434263992 conn(0x7f773009cd40 msgr2=0x7f7730160ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:08.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.530+0000 7f77349d5640 1 -- 172.21.15.43:0/1434263992 shutdown_connections 2024-04-25T08:55:08.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.530+0000 7f77349d5640 1 -- 172.21.15.43:0/1434263992 wait complete. 2024-04-25T08:55:08.531 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T08:55:08.531 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T08:55:08.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.645+0000 7fab74f48640 1 Processor -- start 2024-04-25T08:55:08.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.645+0000 7fab74f48640 1 -- start start 2024-04-25T08:55:08.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.646+0000 7fab74f48640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fab7015dec0 0x7fab7015ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:08.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.646+0000 7fab74f48640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fab700ce8c0 0x7fab700d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:08.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.646+0000 7fab74f48640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab700cf110 con 0x7fab700d04a0 2024-04-25T08:55:08.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.646+0000 7fab74f48640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab700cf3f0 con 0x7fab700ce8c0 2024-04-25T08:55:08.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.646+0000 7fab74f48640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab700cf6d0 con 0x7fab7015dec0 2024-04-25T08:55:08.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.646+0000 7fab6ed9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fab700ce8c0 0x7fab700d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:08.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.646+0000 7fab6ed9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fab700ce8c0 0x7fab700d17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42162/0 (socket says 172.21.15.43:42162) 2024-04-25T08:55:08.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.646+0000 7fab6e59c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fab700d04a0 0x7fab700d0880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:55536/0 (socket says 172.21.15.43:55536) 2024-04-25T08:55:08.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.646+0000 7fab6ed9d640 1 -- 172.21.15.43:0/3868459659 learned_addr learned my addr 172.21.15.43:0/3868459659 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:08.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.646+0000 7fab6dd9b640 1 --2- 172.21.15.43:0/3868459659 >> v2:172.21.15.193:3300/0 conn(0x7fab7015dec0 0x7fab7015ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:08.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.646+0000 7fab6ed9d640 1 -- 172.21.15.43:0/3868459659 >> v2:172.21.15.193:3300/0 conn(0x7fab7015dec0 msgr2=0x7fab7015ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:08.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.646+0000 7fab6ed9d640 1 --2- 172.21.15.43:0/3868459659 >> v2:172.21.15.193:3300/0 conn(0x7fab7015dec0 0x7fab7015ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:08.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.646+0000 7fab6ed9d640 1 -- 172.21.15.43:0/3868459659 >> v1:172.21.15.43:6789/0 conn(0x7fab700d04a0 legacy=0x7fab700d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:08.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.646+0000 7fab6ed9d640 1 -- 172.21.15.43:0/3868459659 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab700cf9b0 con 0x7fab700ce8c0 2024-04-25T08:55:08.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.646+0000 7fab6d59a640 1 -- 172.21.15.43:0/3868459659 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2111401393 0 0) 0x7fab700cf110 con 0x7fab700d04a0 2024-04-25T08:55:08.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.647+0000 7fab6ed9d640 1 --2- 172.21.15.43:0/3868459659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab700ce8c0 0x7fab700d17a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fab700d0070 tx=0x7fab6802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff0e3d7f03999b78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:08.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.647+0000 7fab6d59a640 1 -- 172.21.15.43:0/3868459659 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab68036070 con 0x7fab700ce8c0 2024-04-25T08:55:08.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.647+0000 7fab6d59a640 1 -- 172.21.15.43:0/3868459659 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab68031600 con 0x7fab700ce8c0 2024-04-25T08:55:08.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.647+0000 7fab6d59a640 1 -- 172.21.15.43:0/3868459659 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab680319d0 con 0x7fab700ce8c0 2024-04-25T08:55:08.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.647+0000 7fab74f48640 1 -- 172.21.15.43:0/3868459659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab700ce8c0 msgr2=0x7fab700d17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:08.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.647+0000 7fab74f48640 1 --2- 172.21.15.43:0/3868459659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab700ce8c0 0x7fab700d17a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fab700d0070 tx=0x7fab6802a730 comp rx=0 tx=0).stop 2024-04-25T08:55:08.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.648+0000 7fab74f48640 1 -- 172.21.15.43:0/3868459659 shutdown_connections 2024-04-25T08:55:08.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.648+0000 7fab74f48640 1 --2- 172.21.15.43:0/3868459659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab700ce8c0 0x7fab700d17a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:08.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.648+0000 7fab74f48640 1 --2- 172.21.15.43:0/3868459659 >> v2:172.21.15.193:3300/0 conn(0x7fab7015dec0 0x7fab7015ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:08.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.648+0000 7fab74f48640 1 -- 172.21.15.43:0/3868459659 >> 172.21.15.43:0/3868459659 conn(0x7fab7009cd40 msgr2=0x7fab700d3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:08.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.648+0000 7fab74f48640 1 -- 172.21.15.43:0/3868459659 shutdown_connections 2024-04-25T08:55:08.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.648+0000 7fab74f48640 1 -- 172.21.15.43:0/3868459659 wait complete. 2024-04-25T08:55:08.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.649+0000 7fab74f48640 1 Processor -- start 2024-04-25T08:55:08.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.649+0000 7fab74f48640 1 -- start start 2024-04-25T08:55:08.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.650+0000 7fab74f48640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fab700ce8c0 0x7fab700b5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:08.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.650+0000 7fab74f48640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab700d04a0 0x7fab70150fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:08.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.650+0000 7fab74f48640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab7015dec0 0x7fab700b8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:08.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.650+0000 7fab74f48640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab700cee70 con 0x7fab7015dec0 2024-04-25T08:55:08.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.650+0000 7fab74f48640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab700cecf0 con 0x7fab700d04a0 2024-04-25T08:55:08.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.650+0000 7fab74f48640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab700d1ce0 con 0x7fab700ce8c0 2024-04-25T08:55:08.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.650+0000 7fab6ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab7015dec0 0x7fab700b8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:08.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.650+0000 7fab6ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab7015dec0 0x7fab700b8120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42176/0 (socket says 172.21.15.43:42176) 2024-04-25T08:55:08.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.650+0000 7fab6ed9d640 1 -- 172.21.15.43:0/3657386997 learned_addr learned my addr 172.21.15.43:0/3657386997 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:08.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.650+0000 7fab6dd9b640 1 --2- 172.21.15.43:0/3657386997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab700d04a0 0x7fab70150fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:08.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.650+0000 7fab6e59c640 1 --2- 172.21.15.43:0/3657386997 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fab700ce8c0 0x7fab700b5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:08.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.650+0000 7fab6ed9d640 1 -- 172.21.15.43:0/3657386997 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fab700ce8c0 msgr2=0x7fab700b5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:08.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.650+0000 7fab6ed9d640 1 --2- 172.21.15.43:0/3657386997 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fab700ce8c0 0x7fab700b5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:08.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.650+0000 7fab6ed9d640 1 -- 172.21.15.43:0/3657386997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab700d04a0 msgr2=0x7fab70150fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:08.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.650+0000 7fab6ed9d640 1 --2- 172.21.15.43:0/3657386997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab700d04a0 0x7fab70150fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:08.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.650+0000 7fab6ed9d640 1 -- 172.21.15.43:0/3657386997 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab700b8ef0 con 0x7fab7015dec0 2024-04-25T08:55:08.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.650+0000 7fab6dd9b640 1 --2- 172.21.15.43:0/3657386997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab700d04a0 0x7fab70150fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:55:08.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.650+0000 7fab6ed9d640 1 --2- 172.21.15.43:0/3657386997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab7015dec0 0x7fab700b8120 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fab68009f40 tx=0x7fab68004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:08.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.650+0000 7fab5effd640 1 -- 172.21.15.43:0/3657386997 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab68036050 con 0x7fab7015dec0 2024-04-25T08:55:08.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.650+0000 7fab5effd640 1 -- 172.21.15.43:0/3657386997 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab68042070 con 0x7fab7015dec0 2024-04-25T08:55:08.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.650+0000 7fab74f48640 1 -- 172.21.15.43:0/3657386997 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab700b6c80 con 0x7fab7015dec0 2024-04-25T08:55:08.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.651+0000 7fab5effd640 1 -- 172.21.15.43:0/3657386997 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab68009510 con 0x7fab7015dec0 2024-04-25T08:55:08.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.651+0000 7fab74f48640 1 -- 172.21.15.43:0/3657386997 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab700b9470 con 0x7fab7015dec0 2024-04-25T08:55:08.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.652+0000 7fab74f48640 1 -- 172.21.15.43:0/3657386997 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab70074f60 con 0x7fab7015dec0 2024-04-25T08:55:08.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.652+0000 7fab5effd640 1 -- 172.21.15.43:0/3657386997 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fab68007bf0 con 0x7fab7015dec0 2024-04-25T08:55:08.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.652+0000 7fab5effd640 1 --2- 172.21.15.43:0/3657386997 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fab5006b4d0 0x7fab5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:08.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.652+0000 7fab5effd640 1 -- 172.21.15.43:0/3657386997 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fab680aaf20 con 0x7fab7015dec0 2024-04-25T08:55:08.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.653+0000 7fab6e59c640 1 --2- 172.21.15.43:0/3657386997 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fab5006b4d0 0x7fab5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:08.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.653+0000 7fab6e59c640 1 --2- 172.21.15.43:0/3657386997 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fab5006b4d0 0x7fab5006d990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fab60000f60 tx=0x7fab60007b50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:08.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.657+0000 7fab5effd640 1 -- 172.21.15.43:0/3657386997 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fab68079e30 con 0x7fab7015dec0 2024-04-25T08:55:08.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:08.874+0000 7fab74f48640 1 -- 172.21.15.43:0/3657386997 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fab7008daf0 con 0x7fab7015dec0 2024-04-25T08:55:09.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.533+0000 7fab5effd640 1 -- 172.21.15.43:0/3657386997 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v13) v1 ==== 122+0+0 (secure 0 0 0) 0x7fab68079730 con 0x7fab7015dec0 2024-04-25T08:55:09.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.536+0000 7fab74f48640 1 -- 172.21.15.43:0/3657386997 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fab5006b4d0 msgr2=0x7fab5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:09.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.536+0000 7fab74f48640 1 --2- 172.21.15.43:0/3657386997 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fab5006b4d0 0x7fab5006d990 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fab60000f60 tx=0x7fab60007b50 comp rx=0 tx=0).stop 2024-04-25T08:55:09.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.536+0000 7fab74f48640 1 -- 172.21.15.43:0/3657386997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab7015dec0 msgr2=0x7fab700b8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:09.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.536+0000 7fab74f48640 1 --2- 172.21.15.43:0/3657386997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab7015dec0 0x7fab700b8120 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fab68009f40 tx=0x7fab68004a40 comp rx=0 tx=0).stop 2024-04-25T08:55:09.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.536+0000 7fab74f48640 1 -- 172.21.15.43:0/3657386997 shutdown_connections 2024-04-25T08:55:09.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.536+0000 7fab74f48640 1 --2- 172.21.15.43:0/3657386997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fab7015dec0 0x7fab700b8120 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:09.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.536+0000 7fab74f48640 1 --2- 172.21.15.43:0/3657386997 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fab5006b4d0 0x7fab5006d990 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:09.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.536+0000 7fab74f48640 1 --2- 172.21.15.43:0/3657386997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fab700d04a0 0x7fab70150fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:09.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.536+0000 7fab74f48640 1 --2- 172.21.15.43:0/3657386997 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fab700ce8c0 0x7fab700b5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:09.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.536+0000 7fab74f48640 1 -- 172.21.15.43:0/3657386997 >> 172.21.15.43:0/3657386997 conn(0x7fab7009cd40 msgr2=0x7fab700b0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:09.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.536+0000 7fab74f48640 1 -- 172.21.15.43:0/3657386997 shutdown_connections 2024-04-25T08:55:09.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.536+0000 7fab74f48640 1 -- 172.21.15.43:0/3657386997 wait complete. 2024-04-25T08:55:09.538 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T08:55:09.539 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:55:09.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.655+0000 7f8aa51d5640 1 Processor -- start 2024-04-25T08:55:09.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.655+0000 7f8aa51d5640 1 -- start start 2024-04-25T08:55:09.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.656+0000 7f8aa51d5640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f8aa00cbfc0 0x7f8aa006dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:09.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.656+0000 7f8aa51d5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8aa00ccbb0 0x7f8aa00cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:09.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.656+0000 7f8aa51d5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8aa015f5e0 0x7f8aa015fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:09.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.656+0000 7f8aa51d5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aa00cd550 con 0x7f8aa00ccbb0 2024-04-25T08:55:09.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.656+0000 7f8aa51d5640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aa00b2780 con 0x7f8aa00cbfc0 2024-04-25T08:55:09.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.656+0000 7f8aa51d5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa00b2a60 con 0x7f8aa015f5e0 2024-04-25T08:55:09.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.656+0000 7f8a97fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8aa00ccbb0 0x7f8aa00cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:09.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.656+0000 7f8a97fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8aa00ccbb0 0x7f8aa00cd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38220/0 (socket says 172.21.15.43:38220) 2024-04-25T08:55:09.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.656+0000 7f8a97fff640 1 -- 172.21.15.43:0/3433541047 learned_addr learned my addr 172.21.15.43:0/3433541047 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:09.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.656+0000 7f8a97fff640 1 -- 172.21.15.43:0/3433541047 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8aa015f5e0 msgr2=0x7f8aa015fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:09.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.656+0000 7f8a9f59e640 1 --2- 172.21.15.43:0/3433541047 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8aa015f5e0 0x7f8aa015fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:09.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.656+0000 7f8a97fff640 1 --2- 172.21.15.43:0/3433541047 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8aa015f5e0 0x7f8aa015fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:09.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.656+0000 7f8a9ed9d640 1 --2- 172.21.15.43:0/3433541047 >> v2:172.21.15.193:3300/0 conn(0x7f8aa00cbfc0 0x7f8aa006dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:09.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.656+0000 7f8a97fff640 1 -- 172.21.15.43:0/3433541047 >> v2:172.21.15.193:3300/0 conn(0x7f8aa00cbfc0 msgr2=0x7f8aa006dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:09.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.656+0000 7f8a97fff640 1 --2- 172.21.15.43:0/3433541047 >> v2:172.21.15.193:3300/0 conn(0x7f8aa00cbfc0 0x7f8aa006dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:09.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.656+0000 7f8a97fff640 1 -- 172.21.15.43:0/3433541047 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa00b2d40 con 0x7f8aa00ccbb0 2024-04-25T08:55:09.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.657+0000 7f8a97fff640 1 --2- 172.21.15.43:0/3433541047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8aa00ccbb0 0x7f8aa00cd010 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8aa00cf630 tx=0x7f8a8802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=705f9e7fe9c10c83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:09.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.657+0000 7f8a9e59c640 1 -- 172.21.15.43:0/3433541047 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a88037070 con 0x7f8aa00ccbb0 2024-04-25T08:55:09.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.657+0000 7f8a9e59c640 1 -- 172.21.15.43:0/3433541047 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a88032440 con 0x7f8aa00ccbb0 2024-04-25T08:55:09.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.657+0000 7f8a9e59c640 1 -- 172.21.15.43:0/3433541047 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a88032810 con 0x7f8aa00ccbb0 2024-04-25T08:55:09.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.658+0000 7f8aa51d5640 1 -- 172.21.15.43:0/3433541047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8aa00ccbb0 msgr2=0x7f8aa00cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:09.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.658+0000 7f8aa51d5640 1 --2- 172.21.15.43:0/3433541047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8aa00ccbb0 0x7f8aa00cd010 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8aa00cf630 tx=0x7f8a8802a1e0 comp rx=0 tx=0).stop 2024-04-25T08:55:09.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.658+0000 7f8aa51d5640 1 -- 172.21.15.43:0/3433541047 shutdown_connections 2024-04-25T08:55:09.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.658+0000 7f8aa51d5640 1 --2- 172.21.15.43:0/3433541047 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8aa015f5e0 0x7f8aa015fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:09.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.658+0000 7f8aa51d5640 1 --2- 172.21.15.43:0/3433541047 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8aa00ccbb0 0x7f8aa00cd010 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:09.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.658+0000 7f8aa51d5640 1 --2- 172.21.15.43:0/3433541047 >> v2:172.21.15.193:3300/0 conn(0x7f8aa00cbfc0 0x7f8aa006dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:09.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.658+0000 7f8aa51d5640 1 -- 172.21.15.43:0/3433541047 >> 172.21.15.43:0/3433541047 conn(0x7f8aa00b1480 msgr2=0x7f8aa00b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:09.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.658+0000 7f8aa51d5640 1 -- 172.21.15.43:0/3433541047 shutdown_connections 2024-04-25T08:55:09.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.658+0000 7f8aa51d5640 1 -- 172.21.15.43:0/3433541047 wait complete. 2024-04-25T08:55:09.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.659+0000 7f8aa51d5640 1 Processor -- start 2024-04-25T08:55:09.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.659+0000 7f8aa51d5640 1 -- start start 2024-04-25T08:55:09.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.660+0000 7f8aa51d5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8aa00cbfc0 0x7f8aa01f45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:09.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.660+0000 7f8aa51d5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8aa00ccbb0 0x7f8aa0070350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:09.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.660+0000 7f8aa51d5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8aa015f5e0 0x7f8aa00c6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:09.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.660+0000 7f8aa51d5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa00b35e0 con 0x7f8aa00ccbb0 2024-04-25T08:55:09.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.660+0000 7f8aa51d5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa00b3300 con 0x7f8aa015f5e0 2024-04-25T08:55:09.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.660+0000 7f8aa51d5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa006e1a0 con 0x7f8aa00cbfc0 2024-04-25T08:55:09.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.660+0000 7f8a97fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8aa00ccbb0 0x7f8aa0070350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:09.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.660+0000 7f8a97fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8aa00ccbb0 0x7f8aa0070350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38224/0 (socket says 172.21.15.43:38224) 2024-04-25T08:55:09.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.660+0000 7f8a97fff640 1 -- 172.21.15.43:0/2491013420 learned_addr learned my addr 172.21.15.43:0/2491013420 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:09.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.660+0000 7f8a9ed9d640 1 --2- 172.21.15.43:0/2491013420 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8aa00cbfc0 0x7f8aa01f45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:09.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.660+0000 7f8a9f59e640 1 --2- 172.21.15.43:0/2491013420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8aa015f5e0 0x7f8aa00c6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:09.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.660+0000 7f8a97fff640 1 -- 172.21.15.43:0/2491013420 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8aa00cbfc0 msgr2=0x7f8aa01f45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:09.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.660+0000 7f8a97fff640 1 --2- 172.21.15.43:0/2491013420 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8aa00cbfc0 0x7f8aa01f45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:09.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.660+0000 7f8a97fff640 1 -- 172.21.15.43:0/2491013420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8aa015f5e0 msgr2=0x7f8aa00c6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:09.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.660+0000 7f8a97fff640 1 --2- 172.21.15.43:0/2491013420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8aa015f5e0 0x7f8aa00c6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:09.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.660+0000 7f8a97fff640 1 -- 172.21.15.43:0/2491013420 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa0070b80 con 0x7f8aa00ccbb0 2024-04-25T08:55:09.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.660+0000 7f8a9ed9d640 1 --2- 172.21.15.43:0/2491013420 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8aa00cbfc0 0x7f8aa01f45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:09.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.660+0000 7f8a9f59e640 1 --2- 172.21.15.43:0/2491013420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8aa015f5e0 0x7f8aa00c6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:09.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.661+0000 7f8a97fff640 1 --2- 172.21.15.43:0/2491013420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8aa00ccbb0 0x7f8aa0070350 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8a8802a220 tx=0x7f8a880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:09.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.661+0000 7f8a977fe640 1 -- 172.21.15.43:0/2491013420 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a88037040 con 0x7f8aa00ccbb0 2024-04-25T08:55:09.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.661+0000 7f8a977fe640 1 -- 172.21.15.43:0/2491013420 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a88042070 con 0x7f8aa00ccbb0 2024-04-25T08:55:09.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.661+0000 7f8aa51d5640 1 -- 172.21.15.43:0/2491013420 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aa0070e60 con 0x7f8aa00ccbb0 2024-04-25T08:55:09.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.661+0000 7f8a977fe640 1 -- 172.21.15.43:0/2491013420 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a88030bb0 con 0x7f8aa00ccbb0 2024-04-25T08:55:09.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.661+0000 7f8aa51d5640 1 -- 172.21.15.43:0/2491013420 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aa01f4d40 con 0x7f8aa00ccbb0 2024-04-25T08:55:09.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.662+0000 7f8aa51d5640 1 -- 172.21.15.43:0/2491013420 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aa0074ef0 con 0x7f8aa00ccbb0 2024-04-25T08:55:09.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.662+0000 7f8a977fe640 1 -- 172.21.15.43:0/2491013420 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8a88047460 con 0x7f8aa00ccbb0 2024-04-25T08:55:09.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.663+0000 7f8a977fe640 1 --2- 172.21.15.43:0/2491013420 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8a8006b4d0 0x7f8a8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:09.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.663+0000 7f8a977fe640 1 -- 172.21.15.43:0/2491013420 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a880aae40 con 0x7f8aa00ccbb0 2024-04-25T08:55:09.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.663+0000 7f8a9ed9d640 1 --2- 172.21.15.43:0/2491013420 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8a8006b4d0 0x7f8a8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:09.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.664+0000 7f8a9ed9d640 1 --2- 172.21.15.43:0/2491013420 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8a8006b4d0 0x7f8a8006d990 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8a9000b800 tx=0x7f8a90008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:09.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.667+0000 7f8a977fe640 1 -- 172.21.15.43:0/2491013420 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8a88045360 con 0x7f8aa00ccbb0 2024-04-25T08:55:09.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.884+0000 7f8aa51d5640 1 -- 172.21.15.43:0/2491013420 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8aa0072340 con 0x7f8aa00ccbb0 2024-04-25T08:55:09.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.884+0000 7f8a977fe640 1 -- 172.21.15.43:0/2491013420 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4934 (secure 0 0 0) 0x7f8a88079cc0 con 0x7f8aa00ccbb0 2024-04-25T08:55:09.875 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:55:09.875 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4984,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/715858990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":715858990},{"type":"v1","addr":"172.21.15.193:6836","nonce":715858990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4987,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2342354979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2342354979},{"type":"v1","addr":"172.21.15.193:6837","nonce":2342354979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":5205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/491504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":491504707},{"type":"v1","addr":"172.21.15.43:6834","nonce":491504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:55:06.885018+0000","modified":"2024-04-25T08:55:08.875946+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5208},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5208":{"gid":5208,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/242762542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":242762542},{"type":"v1","addr":"172.21.15.43:6835","nonce":242762542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T08:55:09.875 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 13 2024-04-25T08:55:09.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.886+0000 7f8aa51d5640 1 -- 172.21.15.43:0/2491013420 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8a8006b4d0 msgr2=0x7f8a8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:09.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.886+0000 7f8aa51d5640 1 --2- 172.21.15.43:0/2491013420 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8a8006b4d0 0x7f8a8006d990 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8a9000b800 tx=0x7f8a90008040 comp rx=0 tx=0).stop 2024-04-25T08:55:09.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.886+0000 7f8aa51d5640 1 -- 172.21.15.43:0/2491013420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8aa00ccbb0 msgr2=0x7f8aa0070350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:09.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.886+0000 7f8aa51d5640 1 --2- 172.21.15.43:0/2491013420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8aa00ccbb0 0x7f8aa0070350 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8a8802a220 tx=0x7f8a880046c0 comp rx=0 tx=0).stop 2024-04-25T08:55:09.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.886+0000 7f8aa51d5640 1 -- 172.21.15.43:0/2491013420 shutdown_connections 2024-04-25T08:55:09.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.886+0000 7f8aa51d5640 1 --2- 172.21.15.43:0/2491013420 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8a8006b4d0 0x7f8a8006d990 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:09.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.886+0000 7f8aa51d5640 1 --2- 172.21.15.43:0/2491013420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8aa015f5e0 0x7f8aa00c6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:09.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.886+0000 7f8aa51d5640 1 --2- 172.21.15.43:0/2491013420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8aa00ccbb0 0x7f8aa0070350 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:09.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.886+0000 7f8aa51d5640 1 --2- 172.21.15.43:0/2491013420 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8aa00cbfc0 0x7f8aa01f45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:09.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.886+0000 7f8aa51d5640 1 -- 172.21.15.43:0/2491013420 >> 172.21.15.43:0/2491013420 conn(0x7f8aa00b1480 msgr2=0x7f8aa01444d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:09.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.886+0000 7f8aa51d5640 1 -- 172.21.15.43:0/2491013420 shutdown_connections 2024-04-25T08:55:09.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.886+0000 7f8aa51d5640 1 -- 172.21.15.43:0/2491013420 wait complete. 2024-04-25T08:55:09.887 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:55:09.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.998+0000 7fb38ad2b640 1 Processor -- start 2024-04-25T08:55:09.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.998+0000 7fb38ad2b640 1 -- start start 2024-04-25T08:55:09.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.999+0000 7fb38ad2b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb3840ce2a0 0x7fb3840d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:09.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.999+0000 7fb38ad2b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3840b6c50 con 0x7fb3840d39d0 2024-04-25T08:55:09.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.999+0000 7fb38ad2b640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb38415e010 con 0x7fb3840d0d90 2024-04-25T08:55:09.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.999+0000 7fb38ad2b640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3840d2020 con 0x7fb3840ce2a0 2024-04-25T08:55:09.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.999+0000 7fb388ac7640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb3840d39d0 0x7fb3840d3db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32964/0 (socket says 172.21.15.43:32964) 2024-04-25T08:55:09.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:09.999+0000 7fb388ac7640 1 -- 172.21.15.43:0/1737988245 learned_addr learned my addr 172.21.15.43:0/1737988245 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:55:09.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.000+0000 7fb3892c8640 1 --2- 172.21.15.43:0/1737988245 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb3840ce2a0 0x7fb3840d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:09.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.000+0000 7fb3837fe640 1 -- 172.21.15.43:0/1737988245 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 808671091 0 0) 0x7fb37c001490 con 0x7fb3840d39d0 2024-04-25T08:55:09.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.000+0000 7fb3837fe640 1 -- 172.21.15.43:0/1737988245 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb370003520 con 0x7fb3840d39d0 2024-04-25T08:55:09.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.000+0000 7fb3837fe640 1 -- 172.21.15.43:0/1737988245 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3287971939 0 0) 0x7fb38415e010 con 0x7fb3840d0d90 2024-04-25T08:55:09.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.000+0000 7fb3837fe640 1 -- 172.21.15.43:0/1737988245 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb37c001490 con 0x7fb3840d0d90 2024-04-25T08:55:09.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.000+0000 7fb3837fe640 1 -- 172.21.15.43:0/1737988245 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1905347720 0 0) 0x7fb370003520 con 0x7fb3840d39d0 2024-04-25T08:55:09.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.000+0000 7fb3892c8640 1 -- 172.21.15.43:0/1737988245 >> v1:172.21.15.193:6789/0 conn(0x7fb3840d0d90 legacy=0x7fb3840d2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:09.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.000+0000 7fb3892c8640 1 -- 172.21.15.43:0/1737988245 >> v1:172.21.15.43:6789/0 conn(0x7fb3840d39d0 legacy=0x7fb3840d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:09.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.000+0000 7fb3892c8640 1 -- 172.21.15.43:0/1737988245 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3840cec40 con 0x7fb3840ce2a0 2024-04-25T08:55:09.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.001+0000 7fb3892c8640 1 --2- 172.21.15.43:0/1737988245 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb3840ce2a0 0x7fb3840d2c10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb3740097d0 tx=0x7fb37402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6748e715a9547ee7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:09.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.001+0000 7fb3837fe640 1 -- 172.21.15.43:0/1737988245 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb374037070 con 0x7fb3840ce2a0 2024-04-25T08:55:09.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.001+0000 7fb3837fe640 1 -- 172.21.15.43:0/1737988245 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb37402ab90 con 0x7fb3840ce2a0 2024-04-25T08:55:09.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.001+0000 7fb3837fe640 1 -- 172.21.15.43:0/1737988245 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c001bc0 con 0x7fb3840ce2a0 2024-04-25T08:55:09.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.001+0000 7fb38ad2b640 1 -- 172.21.15.43:0/1737988245 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb3840ce2a0 msgr2=0x7fb3840d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:09.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.001+0000 7fb38ad2b640 1 --2- 172.21.15.43:0/1737988245 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb3840ce2a0 0x7fb3840d2c10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb3740097d0 tx=0x7fb37402a100 comp rx=0 tx=0).stop 2024-04-25T08:55:09.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.002+0000 7fb38ad2b640 1 -- 172.21.15.43:0/1737988245 shutdown_connections 2024-04-25T08:55:09.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.002+0000 7fb38ad2b640 1 --2- 172.21.15.43:0/1737988245 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb3840ce2a0 0x7fb3840d2c10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:09.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.002+0000 7fb38ad2b640 1 -- 172.21.15.43:0/1737988245 >> 172.21.15.43:0/1737988245 conn(0x7fb38406b000 msgr2=0x7fb38406b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:09.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.002+0000 7fb38ad2b640 1 -- 172.21.15.43:0/1737988245 shutdown_connections 2024-04-25T08:55:09.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.002+0000 7fb38ad2b640 1 -- 172.21.15.43:0/1737988245 wait complete. 2024-04-25T08:55:09.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.003+0000 7fb38ad2b640 1 Processor -- start 2024-04-25T08:55:09.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.003+0000 7fb38ad2b640 1 -- start start 2024-04-25T08:55:09.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.004+0000 7fb38ad2b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3840ce2a0 0x7fb38415c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:09.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.004+0000 7fb38ad2b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3840d0d90 0x7fb3840c9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:09.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.004+0000 7fb38ad2b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb3840d39d0 0x7fb3841e9120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:09.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.004+0000 7fb38ad2b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3840cf200 con 0x7fb3840ce2a0 2024-04-25T08:55:09.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.004+0000 7fb38ad2b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3840b2f00 con 0x7fb3840d0d90 2024-04-25T08:55:09.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.004+0000 7fb38ad2b640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3840d3150 con 0x7fb3840d39d0 2024-04-25T08:55:09.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.004+0000 7fb388ac7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3840ce2a0 0x7fb38415c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:09.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.004+0000 7fb388ac7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3840ce2a0 0x7fb38415c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38234/0 (socket says 172.21.15.43:38234) 2024-04-25T08:55:09.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.004+0000 7fb388ac7640 1 -- 172.21.15.43:0/645905996 learned_addr learned my addr 172.21.15.43:0/645905996 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:09.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.004+0000 7fb383fff640 1 --2- 172.21.15.43:0/645905996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3840d0d90 0x7fb3840c9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:09.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.004+0000 7fb3892c8640 1 --2- 172.21.15.43:0/645905996 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb3840d39d0 0x7fb3841e9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:09.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.004+0000 7fb388ac7640 1 -- 172.21.15.43:0/645905996 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb3840d39d0 msgr2=0x7fb3841e9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:09.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.004+0000 7fb388ac7640 1 --2- 172.21.15.43:0/645905996 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb3840d39d0 0x7fb3841e9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:09.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.004+0000 7fb388ac7640 1 -- 172.21.15.43:0/645905996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3840d0d90 msgr2=0x7fb3840c9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:09.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.004+0000 7fb388ac7640 1 --2- 172.21.15.43:0/645905996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3840d0d90 0x7fb3840c9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:09.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.004+0000 7fb388ac7640 1 -- 172.21.15.43:0/645905996 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3841e7710 con 0x7fb3840ce2a0 2024-04-25T08:55:09.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.004+0000 7fb388ac7640 1 --2- 172.21.15.43:0/645905996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3840ce2a0 0x7fb38415c720 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb37c001b80 tx=0x7fb37c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:09.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.004+0000 7fb3817fa640 1 -- 172.21.15.43:0/645905996 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c017070 con 0x7fb3840ce2a0 2024-04-25T08:55:09.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.004+0000 7fb3817fa640 1 -- 172.21.15.43:0/645905996 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb37c012400 con 0x7fb3840ce2a0 2024-04-25T08:55:09.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.004+0000 7fb38ad2b640 1 -- 172.21.15.43:0/645905996 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3841e79f0 con 0x7fb3840ce2a0 2024-04-25T08:55:09.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.004+0000 7fb3817fa640 1 -- 172.21.15.43:0/645905996 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c012830 con 0x7fb3840ce2a0 2024-04-25T08:55:09.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.004+0000 7fb38ad2b640 1 -- 172.21.15.43:0/645905996 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3841e7fb0 con 0x7fb3840ce2a0 2024-04-25T08:55:09.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.005+0000 7fb38ad2b640 1 -- 172.21.15.43:0/645905996 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb384074ef0 con 0x7fb3840ce2a0 2024-04-25T08:55:09.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.006+0000 7fb3817fa640 1 -- 172.21.15.43:0/645905996 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb37c01a070 con 0x7fb3840ce2a0 2024-04-25T08:55:09.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.006+0000 7fb3817fa640 1 --2- 172.21.15.43:0/645905996 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb36406b4d0 0x7fb36406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:09.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.006+0000 7fb3817fa640 1 -- 172.21.15.43:0/645905996 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb37c08a340 con 0x7fb3840ce2a0 2024-04-25T08:55:09.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.006+0000 7fb383fff640 1 --2- 172.21.15.43:0/645905996 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb36406b4d0 0x7fb36406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:09.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.007+0000 7fb383fff640 1 --2- 172.21.15.43:0/645905996 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb36406b4d0 0x7fb36406d990 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb3840c9800 tx=0x7fb36c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:10.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.010+0000 7fb3817fa640 1 -- 172.21.15.43:0/645905996 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb3840b6c50 con 0x7fb3840ce2a0 2024-04-25T08:55:10.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.190+0000 7fb38ad2b640 1 -- 172.21.15.43:0/645905996 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3841e81f0 con 0x7fb3840ce2a0 2024-04-25T08:55:10.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.191+0000 7fb3817fa640 1 -- 172.21.15.43:0/645905996 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16633 (secure 0 0 0) 0x7fb37c0591c0 con 0x7fb3840ce2a0 2024-04-25T08:55:10.181 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:55:10.181 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":33,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:55:08.508624+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:55:05.150275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T08:55:05.853905+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:55:10.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.193+0000 7fb38ad2b640 1 -- 172.21.15.43:0/645905996 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb36406b4d0 msgr2=0x7fb36406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:10.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.193+0000 7fb38ad2b640 1 --2- 172.21.15.43:0/645905996 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb36406b4d0 0x7fb36406d990 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb3840c9800 tx=0x7fb36c009040 comp rx=0 tx=0).stop 2024-04-25T08:55:10.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.193+0000 7fb38ad2b640 1 -- 172.21.15.43:0/645905996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3840ce2a0 msgr2=0x7fb38415c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:10.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.193+0000 7fb38ad2b640 1 --2- 172.21.15.43:0/645905996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3840ce2a0 0x7fb38415c720 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb37c001b80 tx=0x7fb37c00a430 comp rx=0 tx=0).stop 2024-04-25T08:55:10.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.193+0000 7fb38ad2b640 1 -- 172.21.15.43:0/645905996 shutdown_connections 2024-04-25T08:55:10.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.193+0000 7fb38ad2b640 1 --2- 172.21.15.43:0/645905996 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb3840d39d0 0x7fb3841e9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:10.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.193+0000 7fb38ad2b640 1 --2- 172.21.15.43:0/645905996 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb36406b4d0 0x7fb36406d990 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:10.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.193+0000 7fb38ad2b640 1 --2- 172.21.15.43:0/645905996 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb3840d0d90 0x7fb3840c9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:10.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.193+0000 7fb38ad2b640 1 --2- 172.21.15.43:0/645905996 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3840ce2a0 0x7fb38415c720 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:10.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.193+0000 7fb38ad2b640 1 -- 172.21.15.43:0/645905996 >> 172.21.15.43:0/645905996 conn(0x7fb38406b000 msgr2=0x7fb3840d2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:10.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.193+0000 7fb38ad2b640 1 -- 172.21.15.43:0/645905996 shutdown_connections 2024-04-25T08:55:10.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.193+0000 7fb38ad2b640 1 -- 172.21.15.43:0/645905996 wait complete. 2024-04-25T08:55:10.194 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T08:55:10.194 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:55:10.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.305+0000 7f5a168dd640 1 Processor -- start 2024-04-25T08:55:10.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.305+0000 7f5a168dd640 1 -- start start 2024-04-25T08:55:10.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.306+0000 7f5a168dd640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5a1015def0 0x7f5a1006be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:10.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.306+0000 7f5a168dd640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f5a10162560 0x7f5a100b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:10.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.306+0000 7f5a168dd640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a1015e2d0 con 0x7f5a101619f0 2024-04-25T08:55:10.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.306+0000 7f5a168dd640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a100d0e90 con 0x7f5a10162560 2024-04-25T08:55:10.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.306+0000 7f5a168dd640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1006c390 con 0x7f5a1015def0 2024-04-25T08:55:10.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.306+0000 7f5a0f7fe640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f5a101619f0 0x7f5a10161e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:47304/0 (socket says 172.21.15.43:47304) 2024-04-25T08:55:10.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.306+0000 7f5a0f7fe640 1 -- 172.21.15.43:0/1302623223 learned_addr learned my addr 172.21.15.43:0/1302623223 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:55:10.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.306+0000 7f5a0ffff640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5a1015def0 0x7f5a1006be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:10.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.306+0000 7f5a14e7a640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f5a10162560 0x7f5a100b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:10.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.306+0000 7f5a14e7a640 1 -- 172.21.15.43:0/1302623223 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5a1015def0 msgr2=0x7f5a1006be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:10.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.306+0000 7f5a14e7a640 1 --2- 172.21.15.43:0/1302623223 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5a1015def0 0x7f5a1006be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:10.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.306+0000 7f5a14e7a640 1 -- 172.21.15.43:0/1302623223 >> v1:172.21.15.193:6789/0 conn(0x7f5a101619f0 legacy=0x7f5a10161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:10.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.306+0000 7f5a0effd640 1 -- 172.21.15.43:0/1302623223 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2744017031 0 0) 0x7f5a1015e2d0 con 0x7f5a101619f0 2024-04-25T08:55:10.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.307+0000 7f5a14e7a640 1 -- 172.21.15.43:0/1302623223 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a100b1020 con 0x7f5a10162560 2024-04-25T08:55:10.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.307+0000 7f5a14e7a640 1 --2- 172.21.15.43:0/1302623223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a10162560 0x7f5a100b20c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5a08009dc0 tx=0x7f5a0802a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ab0b49d8382213d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:10.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.307+0000 7f5a0effd640 1 -- 172.21.15.43:0/1302623223 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0802ae60 con 0x7f5a10162560 2024-04-25T08:55:10.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.307+0000 7f5a0effd640 1 -- 172.21.15.43:0/1302623223 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a08007740 con 0x7f5a10162560 2024-04-25T08:55:10.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.307+0000 7f5a0effd640 1 -- 172.21.15.43:0/1302623223 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a08007b10 con 0x7f5a10162560 2024-04-25T08:55:10.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.308+0000 7f5a168dd640 1 -- 172.21.15.43:0/1302623223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a10162560 msgr2=0x7f5a100b20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:10.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.308+0000 7f5a168dd640 1 --2- 172.21.15.43:0/1302623223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a10162560 0x7f5a100b20c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5a08009dc0 tx=0x7f5a0802a680 comp rx=0 tx=0).stop 2024-04-25T08:55:10.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.308+0000 7f5a168dd640 1 -- 172.21.15.43:0/1302623223 shutdown_connections 2024-04-25T08:55:10.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.308+0000 7f5a168dd640 1 --2- 172.21.15.43:0/1302623223 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a10162560 0x7f5a100b20c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:10.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.308+0000 7f5a168dd640 1 --2- 172.21.15.43:0/1302623223 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5a1015def0 0x7f5a1006be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:10.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.308+0000 7f5a168dd640 1 -- 172.21.15.43:0/1302623223 >> 172.21.15.43:0/1302623223 conn(0x7f5a100b32e0 msgr2=0x7f5a100b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:10.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.308+0000 7f5a168dd640 1 -- 172.21.15.43:0/1302623223 shutdown_connections 2024-04-25T08:55:10.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.308+0000 7f5a168dd640 1 -- 172.21.15.43:0/1302623223 wait complete. 2024-04-25T08:55:10.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.309+0000 7f5a168dd640 1 Processor -- start 2024-04-25T08:55:10.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.309+0000 7f5a168dd640 1 -- start start 2024-04-25T08:55:10.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.310+0000 7f5a168dd640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5a1015def0 0x7f5a1006f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:10.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.310+0000 7f5a168dd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a101619f0 0x7f5a1015f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:10.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.310+0000 7f5a168dd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a10162560 0x7f5a101f3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:10.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.310+0000 7f5a168dd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a100b18c0 con 0x7f5a101619f0 2024-04-25T08:55:10.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.310+0000 7f5a168dd640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a100b15e0 con 0x7f5a10162560 2024-04-25T08:55:10.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.310+0000 7f5a168dd640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a100b1ba0 con 0x7f5a1015def0 2024-04-25T08:55:10.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.310+0000 7f5a0f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a101619f0 0x7f5a1015f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:10.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.310+0000 7f5a0f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a101619f0 0x7f5a1015f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38248/0 (socket says 172.21.15.43:38248) 2024-04-25T08:55:10.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.310+0000 7f5a0f7fe640 1 -- 172.21.15.43:0/4112726446 learned_addr learned my addr 172.21.15.43:0/4112726446 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:10.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.310+0000 7f5a14e7a640 1 --2- 172.21.15.43:0/4112726446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a10162560 0x7f5a101f3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:10.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.310+0000 7f5a0ffff640 1 --2- 172.21.15.43:0/4112726446 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5a1015def0 0x7f5a1006f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:10.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.311+0000 7f5a0f7fe640 1 -- 172.21.15.43:0/4112726446 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5a1015def0 msgr2=0x7f5a1006f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:10.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.311+0000 7f5a0f7fe640 1 --2- 172.21.15.43:0/4112726446 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5a1015def0 0x7f5a1006f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:10.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.311+0000 7f5a0f7fe640 1 -- 172.21.15.43:0/4112726446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a10162560 msgr2=0x7f5a101f3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:10.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.311+0000 7f5a0f7fe640 1 --2- 172.21.15.43:0/4112726446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a10162560 0x7f5a101f3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:10.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.311+0000 7f5a0f7fe640 1 -- 172.21.15.43:0/4112726446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a100af2d0 con 0x7f5a101619f0 2024-04-25T08:55:10.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.311+0000 7f5a0f7fe640 1 --2- 172.21.15.43:0/4112726446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a101619f0 0x7f5a1015f190 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5a0000c460 tx=0x7f5a0000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:10.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.311+0000 7f5a0cff9640 1 -- 172.21.15.43:0/4112726446 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a00018070 con 0x7f5a101619f0 2024-04-25T08:55:10.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.311+0000 7f5a0cff9640 1 -- 172.21.15.43:0/4112726446 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a000074e0 con 0x7f5a101619f0 2024-04-25T08:55:10.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.311+0000 7f5a168dd640 1 -- 172.21.15.43:0/4112726446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a100af570 con 0x7f5a101619f0 2024-04-25T08:55:10.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.311+0000 7f5a0cff9640 1 -- 172.21.15.43:0/4112726446 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a00004a50 con 0x7f5a101619f0 2024-04-25T08:55:10.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.311+0000 7f5a168dd640 1 -- 172.21.15.43:0/4112726446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a100b6050 con 0x7f5a101619f0 2024-04-25T08:55:10.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.312+0000 7f5a168dd640 1 -- 172.21.15.43:0/4112726446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a10074ef0 con 0x7f5a101619f0 2024-04-25T08:55:10.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.312+0000 7f5a0cff9640 1 -- 172.21.15.43:0/4112726446 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5a00004bf0 con 0x7f5a101619f0 2024-04-25T08:55:10.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.313+0000 7f5a0cff9640 1 --2- 172.21.15.43:0/4112726446 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f59f006b4d0 0x7f59f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:10.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.313+0000 7f5a0cff9640 1 -- 172.21.15.43:0/4112726446 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a000894f0 con 0x7f5a101619f0 2024-04-25T08:55:10.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.313+0000 7f5a0ffff640 1 --2- 172.21.15.43:0/4112726446 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f59f006b4d0 0x7f59f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:10.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.314+0000 7f5a0ffff640 1 --2- 172.21.15.43:0/4112726446 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f59f006b4d0 0x7f59f006d990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f59f8009aa0 tx=0x7f59f8009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:10.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.317+0000 7f5a0cff9640 1 -- 172.21.15.43:0/4112726446 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5a00057d60 con 0x7f5a101619f0 2024-04-25T08:55:10.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.492+0000 7f5a168dd640 1 -- 172.21.15.43:0/4112726446 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5a1015b230 con 0x7f59f006b4d0 2024-04-25T08:55:10.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.499+0000 7f5a0cff9640 1 -- 172.21.15.43:0/4112726446 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251907 (secure 0 0 0) 0x7f5a1015b230 con 0x7f59f006b4d0 2024-04-25T08:55:10.492 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:55:10.493 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-25T08:55:10.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.502+0000 7f5a168dd640 1 -- 172.21.15.43:0/4112726446 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f59f006b4d0 msgr2=0x7f59f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:10.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.502+0000 7f5a168dd640 1 --2- 172.21.15.43:0/4112726446 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f59f006b4d0 0x7f59f006d990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f59f8009aa0 tx=0x7f59f8009190 comp rx=0 tx=0).stop 2024-04-25T08:55:10.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.502+0000 7f5a168dd640 1 -- 172.21.15.43:0/4112726446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a101619f0 msgr2=0x7f5a1015f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:10.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.502+0000 7f5a168dd640 1 --2- 172.21.15.43:0/4112726446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a101619f0 0x7f5a1015f190 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5a0000c460 tx=0x7f5a0000c920 comp rx=0 tx=0).stop 2024-04-25T08:55:10.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.502+0000 7f5a168dd640 1 -- 172.21.15.43:0/4112726446 shutdown_connections 2024-04-25T08:55:10.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.502+0000 7f5a168dd640 1 --2- 172.21.15.43:0/4112726446 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f59f006b4d0 0x7f59f006d990 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:10.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.502+0000 7f5a168dd640 1 --2- 172.21.15.43:0/4112726446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5a10162560 0x7f5a101f3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:10.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.502+0000 7f5a168dd640 1 --2- 172.21.15.43:0/4112726446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5a101619f0 0x7f5a1015f190 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:10.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.502+0000 7f5a168dd640 1 --2- 172.21.15.43:0/4112726446 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5a1015def0 0x7f5a1006f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:10.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.502+0000 7f5a168dd640 1 -- 172.21.15.43:0/4112726446 >> 172.21.15.43:0/4112726446 conn(0x7f5a100b32e0 msgr2=0x7f5a10161e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:10.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.502+0000 7f5a168dd640 1 -- 172.21.15.43:0/4112726446 shutdown_connections 2024-04-25T08:55:10.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.502+0000 7f5a168dd640 1 -- 172.21.15.43:0/4112726446 wait complete. 2024-04-25T08:55:10.503 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":62,"stamp":"2024-04-25T08:55:09.707776+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":157,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236888,"kb_used_data":4052,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737832,"statfs":{"total":773094113280,"available":772851539968,"internally_reserved":0,"allocated":4149248,"data_stored":2315680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.204898"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.544412+0000","last_change":"2024-04-25T08:55:07.536694+0000","last_active":"2024-04-25T08:55:07.544412+0000","last_peered":"2024-04-25T08:55:07.544412+0000","last_clean":"2024-04-25T08:55:07.544412+0000","last_became_active":"2024-04-25T08:55:07.536387+0000","last_became_peered":"2024-04-25T08:55:07.536387+0000","last_unstale":"2024-04-25T08:55:07.544412+0000","last_undegraded":"2024-04-25T08:55:07.544412+0000","last_fullsized":"2024-04-25T08:55:07.544412+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:36.981521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.540328+0000","last_change":"2024-04-25T08:55:07.530129+0000","last_active":"2024-04-25T08:55:07.540328+0000","last_peered":"2024-04-25T08:55:07.540328+0000","last_clean":"2024-04-25T08:55:07.540328+0000","last_became_active":"2024-04-25T08:55:07.529658+0000","last_became_peered":"2024-04-25T08:55:07.529658+0000","last_unstale":"2024-04-25T08:55:07.540328+0000","last_undegraded":"2024-04-25T08:55:07.540328+0000","last_fullsized":"2024-04-25T08:55:07.540328+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:56.437421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.515504+0000","last_change":"2024-04-25T08:55:07.533341+0000","last_active":"2024-04-25T08:55:08.515504+0000","last_peered":"2024-04-25T08:55:08.515504+0000","last_clean":"2024-04-25T08:55:08.515504+0000","last_became_active":"2024-04-25T08:55:07.532998+0000","last_became_peered":"2024-04-25T08:55:07.532998+0000","last_unstale":"2024-04-25T08:55:08.515504+0000","last_undegraded":"2024-04-25T08:55:08.515504+0000","last_fullsized":"2024-04-25T08:55:08.515504+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:18.826664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.536370+0000","last_change":"2024-04-25T08:55:07.533504+0000","last_active":"2024-04-25T08:55:07.536370+0000","last_peered":"2024-04-25T08:55:07.536370+0000","last_clean":"2024-04-25T08:55:07.536370+0000","last_became_active":"2024-04-25T08:55:07.533240+0000","last_became_peered":"2024-04-25T08:55:07.533240+0000","last_unstale":"2024-04-25T08:55:07.536370+0000","last_undegraded":"2024-04-25T08:55:07.536370+0000","last_fullsized":"2024-04-25T08:55:07.536370+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:42.117269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.527256+0000","last_change":"2024-04-25T08:55:07.526227+0000","last_active":"2024-04-25T08:55:07.527256+0000","last_peered":"2024-04-25T08:55:07.527256+0000","last_clean":"2024-04-25T08:55:07.527256+0000","last_became_active":"2024-04-25T08:55:07.525948+0000","last_became_peered":"2024-04-25T08:55:07.525948+0000","last_unstale":"2024-04-25T08:55:07.527256+0000","last_undegraded":"2024-04-25T08:55:07.527256+0000","last_fullsized":"2024-04-25T08:55:07.527256+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:28.891499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.534490+0000","last_change":"2024-04-25T08:55:07.525333+0000","last_active":"2024-04-25T08:55:07.534490+0000","last_peered":"2024-04-25T08:55:07.534490+0000","last_clean":"2024-04-25T08:55:07.534490+0000","last_became_active":"2024-04-25T08:55:07.525073+0000","last_became_peered":"2024-04-25T08:55:07.525073+0000","last_unstale":"2024-04-25T08:55:07.534490+0000","last_undegraded":"2024-04-25T08:55:07.534490+0000","last_fullsized":"2024-04-25T08:55:07.534490+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:26.245579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.530912+0000","last_change":"2024-04-25T08:55:07.527951+0000","last_active":"2024-04-25T08:55:07.530912+0000","last_peered":"2024-04-25T08:55:07.530912+0000","last_clean":"2024-04-25T08:55:07.530912+0000","last_became_active":"2024-04-25T08:55:07.527682+0000","last_became_peered":"2024-04-25T08:55:07.527682+0000","last_unstale":"2024-04-25T08:55:07.530912+0000","last_undegraded":"2024-04-25T08:55:07.530912+0000","last_fullsized":"2024-04-25T08:55:07.530912+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:22.632076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.530108+0000","last_change":"2024-04-25T08:55:07.524922+0000","last_active":"2024-04-25T08:55:07.530108+0000","last_peered":"2024-04-25T08:55:07.530108+0000","last_clean":"2024-04-25T08:55:07.530108+0000","last_became_active":"2024-04-25T08:55:07.524332+0000","last_became_peered":"2024-04-25T08:55:07.524332+0000","last_unstale":"2024-04-25T08:55:07.530108+0000","last_undegraded":"2024-04-25T08:55:07.530108+0000","last_fullsized":"2024-04-25T08:55:07.530108+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:14:13.129793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.516679+0000","last_change":"2024-04-25T08:55:07.528710+0000","last_active":"2024-04-25T08:55:08.516679+0000","last_peered":"2024-04-25T08:55:08.516679+0000","last_clean":"2024-04-25T08:55:08.516679+0000","last_became_active":"2024-04-25T08:55:07.526398+0000","last_became_peered":"2024-04-25T08:55:07.526398+0000","last_unstale":"2024-04-25T08:55:08.516679+0000","last_undegraded":"2024-04-25T08:55:08.516679+0000","last_fullsized":"2024-04-25T08:55:08.516679+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:25.454711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.542514+0000","last_change":"2024-04-25T08:55:07.541632+0000","last_active":"2024-04-25T08:55:07.542514+0000","last_peered":"2024-04-25T08:55:07.542514+0000","last_clean":"2024-04-25T08:55:07.542514+0000","last_became_active":"2024-04-25T08:55:07.541117+0000","last_became_peered":"2024-04-25T08:55:07.541117+0000","last_unstale":"2024-04-25T08:55:07.542514+0000","last_undegraded":"2024-04-25T08:55:07.542514+0000","last_fullsized":"2024-04-25T08:55:07.542514+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:09.909329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.536869+0000","last_change":"2024-04-25T08:55:07.533404+0000","last_active":"2024-04-25T08:55:07.536869+0000","last_peered":"2024-04-25T08:55:07.536869+0000","last_clean":"2024-04-25T08:55:07.536869+0000","last_became_active":"2024-04-25T08:55:07.533015+0000","last_became_peered":"2024-04-25T08:55:07.533015+0000","last_unstale":"2024-04-25T08:55:07.536869+0000","last_undegraded":"2024-04-25T08:55:07.536869+0000","last_fullsized":"2024-04-25T08:55:07.536869+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:01.586986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.515864+0000","last_change":"2024-04-25T08:55:07.538554+0000","last_active":"2024-04-25T08:55:08.515864+0000","last_peered":"2024-04-25T08:55:08.515864+0000","last_clean":"2024-04-25T08:55:08.515864+0000","last_became_active":"2024-04-25T08:55:07.538295+0000","last_became_peered":"2024-04-25T08:55:07.538295+0000","last_unstale":"2024-04-25T08:55:08.515864+0000","last_undegraded":"2024-04-25T08:55:08.515864+0000","last_fullsized":"2024-04-25T08:55:08.515864+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:47.319053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.526373+0000","last_change":"2024-04-25T08:55:07.521745+0000","last_active":"2024-04-25T08:55:07.526373+0000","last_peered":"2024-04-25T08:55:07.526373+0000","last_clean":"2024-04-25T08:55:07.526373+0000","last_became_active":"2024-04-25T08:55:07.521460+0000","last_became_peered":"2024-04-25T08:55:07.521460+0000","last_unstale":"2024-04-25T08:55:07.526373+0000","last_undegraded":"2024-04-25T08:55:07.526373+0000","last_fullsized":"2024-04-25T08:55:07.526373+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:42.854344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.530942+0000","last_change":"2024-04-25T08:55:07.519469+0000","last_active":"2024-04-25T08:55:07.530942+0000","last_peered":"2024-04-25T08:55:07.530942+0000","last_clean":"2024-04-25T08:55:07.530942+0000","last_became_active":"2024-04-25T08:55:07.519169+0000","last_became_peered":"2024-04-25T08:55:07.519169+0000","last_unstale":"2024-04-25T08:55:07.530942+0000","last_undegraded":"2024-04-25T08:55:07.530942+0000","last_fullsized":"2024-04-25T08:55:07.530942+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:29.002070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.515778+0000","last_change":"2024-04-25T08:55:07.535390+0000","last_active":"2024-04-25T08:55:08.515778+0000","last_peered":"2024-04-25T08:55:08.515778+0000","last_clean":"2024-04-25T08:55:08.515778+0000","last_became_active":"2024-04-25T08:55:07.535119+0000","last_became_peered":"2024-04-25T08:55:07.535119+0000","last_unstale":"2024-04-25T08:55:08.515778+0000","last_undegraded":"2024-04-25T08:55:08.515778+0000","last_fullsized":"2024-04-25T08:55:08.515778+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:39.948659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.536169+0000","last_change":"2024-04-25T08:55:07.529965+0000","last_active":"2024-04-25T08:55:07.536169+0000","last_peered":"2024-04-25T08:55:07.536169+0000","last_clean":"2024-04-25T08:55:07.536169+0000","last_became_active":"2024-04-25T08:55:07.529695+0000","last_became_peered":"2024-04-25T08:55:07.529695+0000","last_unstale":"2024-04-25T08:55:07.536169+0000","last_undegraded":"2024-04-25T08:55:07.536169+0000","last_fullsized":"2024-04-25T08:55:07.536169+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:42.117269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.537095+0000","last_change":"2024-04-25T08:55:07.531585+0000","last_active":"2024-04-25T08:55:07.537095+0000","last_peered":"2024-04-25T08:55:07.537095+0000","last_clean":"2024-04-25T08:55:07.537095+0000","last_became_active":"2024-04-25T08:55:07.531331+0000","last_became_peered":"2024-04-25T08:55:07.531331+0000","last_unstale":"2024-04-25T08:55:07.537095+0000","last_undegraded":"2024-04-25T08:55:07.537095+0000","last_fullsized":"2024-04-25T08:55:07.537095+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:26.245579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.540348+0000","last_change":"2024-04-25T08:55:07.530786+0000","last_active":"2024-04-25T08:55:07.540348+0000","last_peered":"2024-04-25T08:55:07.540348+0000","last_clean":"2024-04-25T08:55:07.540348+0000","last_became_active":"2024-04-25T08:55:07.530441+0000","last_became_peered":"2024-04-25T08:55:07.530441+0000","last_unstale":"2024-04-25T08:55:07.540348+0000","last_undegraded":"2024-04-25T08:55:07.540348+0000","last_fullsized":"2024-04-25T08:55:07.540348+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:15.381709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.538324+0000","last_change":"2024-04-25T08:55:07.528902+0000","last_active":"2024-04-25T08:55:07.538324+0000","last_peered":"2024-04-25T08:55:07.538324+0000","last_clean":"2024-04-25T08:55:07.538324+0000","last_became_active":"2024-04-25T08:55:07.528582+0000","last_became_peered":"2024-04-25T08:55:07.528582+0000","last_unstale":"2024-04-25T08:55:07.538324+0000","last_undegraded":"2024-04-25T08:55:07.538324+0000","last_fullsized":"2024-04-25T08:55:07.538324+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:01.586986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.535350+0000","last_change":"2024-04-25T08:55:07.531670+0000","last_active":"2024-04-25T08:55:07.535350+0000","last_peered":"2024-04-25T08:55:07.535350+0000","last_clean":"2024-04-25T08:55:07.535350+0000","last_became_active":"2024-04-25T08:55:07.531371+0000","last_became_peered":"2024-04-25T08:55:07.531371+0000","last_unstale":"2024-04-25T08:55:07.535350+0000","last_undegraded":"2024-04-25T08:55:07.535350+0000","last_fullsized":"2024-04-25T08:55:07.535350+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:10.672523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.516226+0000","last_change":"2024-04-25T08:55:07.532759+0000","last_active":"2024-04-25T08:55:08.516226+0000","last_peered":"2024-04-25T08:55:08.516226+0000","last_clean":"2024-04-25T08:55:08.516226+0000","last_became_active":"2024-04-25T08:55:07.531953+0000","last_became_peered":"2024-04-25T08:55:07.531953+0000","last_unstale":"2024-04-25T08:55:08.516226+0000","last_undegraded":"2024-04-25T08:55:08.516226+0000","last_fullsized":"2024-04-25T08:55:08.516226+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:09.261521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.516399+0000","last_change":"2024-04-25T08:55:07.519517+0000","last_active":"2024-04-25T08:55:08.516399+0000","last_peered":"2024-04-25T08:55:08.516399+0000","last_clean":"2024-04-25T08:55:08.516399+0000","last_became_active":"2024-04-25T08:55:07.519246+0000","last_became_peered":"2024-04-25T08:55:07.519246+0000","last_unstale":"2024-04-25T08:55:08.516399+0000","last_undegraded":"2024-04-25T08:55:08.516399+0000","last_fullsized":"2024-04-25T08:55:08.516399+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:29.002070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.511605+0000","last_change":"2024-04-25T08:55:07.532420+0000","last_active":"2024-04-25T08:55:08.511605+0000","last_peered":"2024-04-25T08:55:08.511605+0000","last_clean":"2024-04-25T08:55:08.511605+0000","last_became_active":"2024-04-25T08:55:07.532161+0000","last_became_peered":"2024-04-25T08:55:07.532161+0000","last_unstale":"2024-04-25T08:55:08.511605+0000","last_undegraded":"2024-04-25T08:55:08.511605+0000","last_fullsized":"2024-04-25T08:55:08.511605+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:09.261521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.515822+0000","last_change":"2024-04-25T08:55:07.532589+0000","last_active":"2024-04-25T08:55:08.515822+0000","last_peered":"2024-04-25T08:55:08.515822+0000","last_clean":"2024-04-25T08:55:08.515822+0000","last_became_active":"2024-04-25T08:55:07.532321+0000","last_became_peered":"2024-04-25T08:55:07.532321+0000","last_unstale":"2024-04-25T08:55:08.515822+0000","last_undegraded":"2024-04-25T08:55:08.515822+0000","last_fullsized":"2024-04-25T08:55:08.515822+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:15.381709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.531128+0000","last_change":"2024-04-25T08:55:07.530094+0000","last_active":"2024-04-25T08:55:07.531128+0000","last_peered":"2024-04-25T08:55:07.531128+0000","last_clean":"2024-04-25T08:55:07.531128+0000","last_became_active":"2024-04-25T08:55:07.529829+0000","last_became_peered":"2024-04-25T08:55:07.529829+0000","last_unstale":"2024-04-25T08:55:07.531128+0000","last_undegraded":"2024-04-25T08:55:07.531128+0000","last_fullsized":"2024-04-25T08:55:07.531128+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:32.166165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.540306+0000","last_change":"2024-04-25T08:55:07.529894+0000","last_active":"2024-04-25T08:55:07.540306+0000","last_peered":"2024-04-25T08:55:07.540306+0000","last_clean":"2024-04-25T08:55:07.540306+0000","last_became_active":"2024-04-25T08:55:07.529301+0000","last_became_peered":"2024-04-25T08:55:07.529301+0000","last_unstale":"2024-04-25T08:55:07.540306+0000","last_undegraded":"2024-04-25T08:55:07.540306+0000","last_fullsized":"2024-04-25T08:55:07.540306+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:09.909329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.540650+0000","last_change":"2024-04-25T08:55:07.538292+0000","last_active":"2024-04-25T08:55:07.540650+0000","last_peered":"2024-04-25T08:55:07.540650+0000","last_clean":"2024-04-25T08:55:07.540650+0000","last_became_active":"2024-04-25T08:55:07.537992+0000","last_became_peered":"2024-04-25T08:55:07.537992+0000","last_unstale":"2024-04-25T08:55:07.540650+0000","last_undegraded":"2024-04-25T08:55:07.540650+0000","last_fullsized":"2024-04-25T08:55:07.540650+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:26.245579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.516022+0000","last_change":"2024-04-25T08:55:07.530437+0000","last_active":"2024-04-25T08:55:08.516022+0000","last_peered":"2024-04-25T08:55:08.516022+0000","last_clean":"2024-04-25T08:55:08.516022+0000","last_became_active":"2024-04-25T08:55:07.530143+0000","last_became_peered":"2024-04-25T08:55:07.530143+0000","last_unstale":"2024-04-25T08:55:08.516022+0000","last_undegraded":"2024-04-25T08:55:08.516022+0000","last_fullsized":"2024-04-25T08:55:08.516022+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:20.700269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.516555+0000","last_change":"2024-04-25T08:55:07.520199+0000","last_active":"2024-04-25T08:55:08.516555+0000","last_peered":"2024-04-25T08:55:08.516555+0000","last_clean":"2024-04-25T08:55:08.516555+0000","last_became_active":"2024-04-25T08:55:07.519924+0000","last_became_peered":"2024-04-25T08:55:07.519924+0000","last_unstale":"2024-04-25T08:55:08.516555+0000","last_undegraded":"2024-04-25T08:55:08.516555+0000","last_fullsized":"2024-04-25T08:55:08.516555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:26.245579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.530743+0000","last_change":"2024-04-25T08:55:07.529557+0000","last_active":"2024-04-25T08:55:07.530743+0000","last_peered":"2024-04-25T08:55:07.530743+0000","last_clean":"2024-04-25T08:55:07.530743+0000","last_became_active":"2024-04-25T08:55:07.529259+0000","last_became_peered":"2024-04-25T08:55:07.529259+0000","last_unstale":"2024-04-25T08:55:07.530743+0000","last_undegraded":"2024-04-25T08:55:07.530743+0000","last_fullsized":"2024-04-25T08:55:07.530743+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:09.261521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.541637+0000","last_change":"2024-04-25T08:55:07.532819+0000","last_active":"2024-04-25T08:55:07.541637+0000","last_peered":"2024-04-25T08:55:07.541637+0000","last_clean":"2024-04-25T08:55:07.541637+0000","last_became_active":"2024-04-25T08:55:07.532527+0000","last_became_peered":"2024-04-25T08:55:07.532527+0000","last_unstale":"2024-04-25T08:55:07.541637+0000","last_undegraded":"2024-04-25T08:55:07.541637+0000","last_fullsized":"2024-04-25T08:55:07.541637+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:53.530214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.530128+0000","last_change":"2024-04-25T08:55:07.525052+0000","last_active":"2024-04-25T08:55:07.530128+0000","last_peered":"2024-04-25T08:55:07.530128+0000","last_clean":"2024-04-25T08:55:07.530128+0000","last_became_active":"2024-04-25T08:55:07.524758+0000","last_became_peered":"2024-04-25T08:55:07.524758+0000","last_unstale":"2024-04-25T08:55:07.530128+0000","last_undegraded":"2024-04-25T08:55:07.530128+0000","last_fullsized":"2024-04-25T08:55:07.530128+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:36.981521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.511749+0000","last_change":"2024-04-25T08:55:07.533290+0000","last_active":"2024-04-25T08:55:08.511749+0000","last_peered":"2024-04-25T08:55:08.511749+0000","last_clean":"2024-04-25T08:55:08.511749+0000","last_became_active":"2024-04-25T08:55:07.532818+0000","last_became_peered":"2024-04-25T08:55:07.532818+0000","last_unstale":"2024-04-25T08:55:08.511749+0000","last_undegraded":"2024-04-25T08:55:08.511749+0000","last_fullsized":"2024-04-25T08:55:08.511749+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:42.117269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.540469+0000","last_change":"2024-04-25T08:55:06.532332+0000","last_active":"2024-04-25T08:55:07.540469+0000","last_peered":"2024-04-25T08:55:07.540469+0000","last_clean":"2024-04-25T08:55:07.540469+0000","last_became_active":"2024-04-25T08:55:06.532043+0000","last_became_peered":"2024-04-25T08:55:06.532043+0000","last_unstale":"2024-04-25T08:55:07.540469+0000","last_undegraded":"2024-04-25T08:55:07.540469+0000","last_fullsized":"2024-04-25T08:55:07.540469+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:57.871761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.535951+0000","last_change":"2024-04-25T08:55:07.530580+0000","last_active":"2024-04-25T08:55:07.535951+0000","last_peered":"2024-04-25T08:55:07.535951+0000","last_clean":"2024-04-25T08:55:07.535951+0000","last_became_active":"2024-04-25T08:55:07.530275+0000","last_became_peered":"2024-04-25T08:55:07.530275+0000","last_unstale":"2024-04-25T08:55:07.535951+0000","last_undegraded":"2024-04-25T08:55:07.535951+0000","last_fullsized":"2024-04-25T08:55:07.535951+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:07.265287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.511762+0000","last_change":"2024-04-25T08:55:06.526275+0000","last_active":"2024-04-25T08:55:08.511762+0000","last_peered":"2024-04-25T08:55:08.511762+0000","last_clean":"2024-04-25T08:55:08.511762+0000","last_became_active":"2024-04-25T08:55:06.525943+0000","last_became_peered":"2024-04-25T08:55:06.525943+0000","last_unstale":"2024-04-25T08:55:08.511762+0000","last_undegraded":"2024-04-25T08:55:08.511762+0000","last_fullsized":"2024-04-25T08:55:08.511762+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:19.451377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.516609+0000","last_change":"2024-04-25T08:55:07.527049+0000","last_active":"2024-04-25T08:55:08.516609+0000","last_peered":"2024-04-25T08:55:08.516609+0000","last_clean":"2024-04-25T08:55:08.516609+0000","last_became_active":"2024-04-25T08:55:07.526777+0000","last_became_peered":"2024-04-25T08:55:07.526777+0000","last_unstale":"2024-04-25T08:55:08.516609+0000","last_undegraded":"2024-04-25T08:55:08.516609+0000","last_fullsized":"2024-04-25T08:55:08.516609+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:01.586986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.542825+0000","last_change":"2024-04-25T08:55:06.529754+0000","last_active":"2024-04-25T08:55:07.542825+0000","last_peered":"2024-04-25T08:55:07.542825+0000","last_clean":"2024-04-25T08:55:07.542825+0000","last_became_active":"2024-04-25T08:55:06.529439+0000","last_became_peered":"2024-04-25T08:55:06.529439+0000","last_unstale":"2024-04-25T08:55:07.542825+0000","last_undegraded":"2024-04-25T08:55:07.542825+0000","last_fullsized":"2024-04-25T08:55:07.542825+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:31.164120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.540673+0000","last_change":"2024-04-25T08:55:07.538476+0000","last_active":"2024-04-25T08:55:07.540673+0000","last_peered":"2024-04-25T08:55:07.540673+0000","last_clean":"2024-04-25T08:55:07.540673+0000","last_became_active":"2024-04-25T08:55:07.537354+0000","last_became_peered":"2024-04-25T08:55:07.537354+0000","last_unstale":"2024-04-25T08:55:07.540673+0000","last_undegraded":"2024-04-25T08:55:07.540673+0000","last_fullsized":"2024-04-25T08:55:07.540673+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:01.586986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.532620+0000","last_change":"2024-04-25T08:55:06.522349+0000","last_active":"2024-04-25T08:55:07.532620+0000","last_peered":"2024-04-25T08:55:07.532620+0000","last_clean":"2024-04-25T08:55:07.532620+0000","last_became_active":"2024-04-25T08:55:06.522066+0000","last_became_peered":"2024-04-25T08:55:06.522066+0000","last_unstale":"2024-04-25T08:55:07.532620+0000","last_undegraded":"2024-04-25T08:55:07.532620+0000","last_fullsized":"2024-04-25T08:55:07.532620+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:00.350645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.536945+0000","last_change":"2024-04-25T08:55:07.534022+0000","last_active":"2024-04-25T08:55:07.536945+0000","last_peered":"2024-04-25T08:55:07.536945+0000","last_clean":"2024-04-25T08:55:07.536945+0000","last_became_active":"2024-04-25T08:55:07.533736+0000","last_became_peered":"2024-04-25T08:55:07.533736+0000","last_unstale":"2024-04-25T08:55:07.536945+0000","last_undegraded":"2024-04-25T08:55:07.536945+0000","last_fullsized":"2024-04-25T08:55:07.536945+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:25.454711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.516361+0000","last_change":"2024-04-25T08:55:06.526224+0000","last_active":"2024-04-25T08:55:08.516361+0000","last_peered":"2024-04-25T08:55:08.516361+0000","last_clean":"2024-04-25T08:55:08.516361+0000","last_became_active":"2024-04-25T08:55:06.525952+0000","last_became_peered":"2024-04-25T08:55:06.525952+0000","last_unstale":"2024-04-25T08:55:08.516361+0000","last_undegraded":"2024-04-25T08:55:08.516361+0000","last_fullsized":"2024-04-25T08:55:08.516361+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:12.207955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.515656+0000","last_change":"2024-04-25T08:55:07.531473+0000","last_active":"2024-04-25T08:55:08.515656+0000","last_peered":"2024-04-25T08:55:08.515656+0000","last_clean":"2024-04-25T08:55:08.515656+0000","last_became_active":"2024-04-25T08:55:07.531204+0000","last_became_peered":"2024-04-25T08:55:07.531204+0000","last_unstale":"2024-04-25T08:55:08.515656+0000","last_undegraded":"2024-04-25T08:55:08.515656+0000","last_fullsized":"2024-04-25T08:55:08.515656+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:22.996931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.516506+0000","last_change":"2024-04-25T08:55:06.526543+0000","last_active":"2024-04-25T08:55:08.516506+0000","last_peered":"2024-04-25T08:55:08.516506+0000","last_clean":"2024-04-25T08:55:08.516506+0000","last_became_active":"2024-04-25T08:55:06.526141+0000","last_became_peered":"2024-04-25T08:55:06.526141+0000","last_unstale":"2024-04-25T08:55:08.516506+0000","last_undegraded":"2024-04-25T08:55:08.516506+0000","last_fullsized":"2024-04-25T08:55:08.516506+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:43.138025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.516258+0000","last_change":"2024-04-25T08:55:07.532267+0000","last_active":"2024-04-25T08:55:08.516258+0000","last_peered":"2024-04-25T08:55:08.516258+0000","last_clean":"2024-04-25T08:55:08.516258+0000","last_became_active":"2024-04-25T08:55:07.531661+0000","last_became_peered":"2024-04-25T08:55:07.531661+0000","last_unstale":"2024-04-25T08:55:08.516258+0000","last_undegraded":"2024-04-25T08:55:08.516258+0000","last_fullsized":"2024-04-25T08:55:08.516258+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:42.117269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.511488+0000","last_change":"2024-04-25T08:55:06.524478+0000","last_active":"2024-04-25T08:55:07.511488+0000","last_peered":"2024-04-25T08:55:07.511488+0000","last_clean":"2024-04-25T08:55:07.511488+0000","last_became_active":"2024-04-25T08:55:06.524189+0000","last_became_peered":"2024-04-25T08:55:06.524189+0000","last_unstale":"2024-04-25T08:55:07.511488+0000","last_undegraded":"2024-04-25T08:55:07.511488+0000","last_fullsized":"2024-04-25T08:55:07.511488+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:14:12.127747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.540440+0000","last_change":"2024-04-25T08:55:07.539319+0000","last_active":"2024-04-25T08:55:07.540440+0000","last_peered":"2024-04-25T08:55:07.540440+0000","last_clean":"2024-04-25T08:55:07.540440+0000","last_became_active":"2024-04-25T08:55:07.538641+0000","last_became_peered":"2024-04-25T08:55:07.538641+0000","last_unstale":"2024-04-25T08:55:07.540440+0000","last_undegraded":"2024-04-25T08:55:07.540440+0000","last_fullsized":"2024-04-25T08:55:07.540440+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:25.454711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.516287+0000","last_change":"2024-04-25T08:55:06.525229+0000","last_active":"2024-04-25T08:55:08.516287+0000","last_peered":"2024-04-25T08:55:08.516287+0000","last_clean":"2024-04-25T08:55:08.516287+0000","last_became_active":"2024-04-25T08:55:06.524943+0000","last_became_peered":"2024-04-25T08:55:06.524943+0000","last_unstale":"2024-04-25T08:55:08.516287+0000","last_undegraded":"2024-04-25T08:55:08.516287+0000","last_fullsized":"2024-04-25T08:55:08.516287+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:09.670477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.515541+0000","last_change":"2024-04-25T08:55:07.538304+0000","last_active":"2024-04-25T08:55:08.515541+0000","last_peered":"2024-04-25T08:55:08.515541+0000","last_clean":"2024-04-25T08:55:08.515541+0000","last_became_active":"2024-04-25T08:55:07.538057+0000","last_became_peered":"2024-04-25T08:55:07.538057+0000","last_unstale":"2024-04-25T08:55:08.515541+0000","last_undegraded":"2024-04-25T08:55:08.515541+0000","last_fullsized":"2024-04-25T08:55:08.515541+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:53.019544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.513080+0000","last_change":"2024-04-25T08:55:06.529815+0000","last_active":"2024-04-25T08:55:07.513080+0000","last_peered":"2024-04-25T08:55:07.513080+0000","last_clean":"2024-04-25T08:55:07.513080+0000","last_became_active":"2024-04-25T08:55:06.529540+0000","last_became_peered":"2024-04-25T08:55:06.529540+0000","last_unstale":"2024-04-25T08:55:07.513080+0000","last_undegraded":"2024-04-25T08:55:07.513080+0000","last_fullsized":"2024-04-25T08:55:07.513080+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:08.259475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.539547+0000","last_change":"2024-04-25T08:55:07.531978+0000","last_active":"2024-04-25T08:55:07.539547+0000","last_peered":"2024-04-25T08:55:07.539547+0000","last_clean":"2024-04-25T08:55:07.539547+0000","last_became_active":"2024-04-25T08:55:07.530117+0000","last_became_peered":"2024-04-25T08:55:07.530117+0000","last_unstale":"2024-04-25T08:55:07.539547+0000","last_undegraded":"2024-04-25T08:55:07.539547+0000","last_fullsized":"2024-04-25T08:55:07.539547+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:18.826664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.511854+0000","last_change":"2024-04-25T08:55:06.525065+0000","last_active":"2024-04-25T08:55:08.511854+0000","last_peered":"2024-04-25T08:55:08.511854+0000","last_clean":"2024-04-25T08:55:08.511854+0000","last_became_active":"2024-04-25T08:55:06.523538+0000","last_became_peered":"2024-04-25T08:55:06.523538+0000","last_unstale":"2024-04-25T08:55:08.511854+0000","last_undegraded":"2024-04-25T08:55:08.511854+0000","last_fullsized":"2024-04-25T08:55:08.511854+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:57.871761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.541822+0000","last_change":"2024-04-25T08:55:07.529022+0000","last_active":"2024-04-25T08:55:07.541822+0000","last_peered":"2024-04-25T08:55:07.541822+0000","last_clean":"2024-04-25T08:55:07.541822+0000","last_became_active":"2024-04-25T08:55:07.528742+0000","last_became_peered":"2024-04-25T08:55:07.528742+0000","last_unstale":"2024-04-25T08:55:07.541822+0000","last_undegraded":"2024-04-25T08:55:07.541822+0000","last_fullsized":"2024-04-25T08:55:07.541822+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:25.454711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.515606+0000","last_change":"2024-04-25T08:55:06.523452+0000","last_active":"2024-04-25T08:55:08.515606+0000","last_peered":"2024-04-25T08:55:08.515606+0000","last_clean":"2024-04-25T08:55:08.515606+0000","last_became_active":"2024-04-25T08:55:06.523154+0000","last_became_peered":"2024-04-25T08:55:06.523154+0000","last_unstale":"2024-04-25T08:55:08.515606+0000","last_undegraded":"2024-04-25T08:55:08.515606+0000","last_fullsized":"2024-04-25T08:55:08.515606+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:35.574390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.515527+0000","last_change":"2024-04-25T08:55:07.536086+0000","last_active":"2024-04-25T08:55:08.515527+0000","last_peered":"2024-04-25T08:55:08.515527+0000","last_clean":"2024-04-25T08:55:08.515527+0000","last_became_active":"2024-04-25T08:55:07.535072+0000","last_became_peered":"2024-04-25T08:55:07.535072+0000","last_unstale":"2024-04-25T08:55:08.515527+0000","last_undegraded":"2024-04-25T08:55:08.515527+0000","last_fullsized":"2024-04-25T08:55:08.515527+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:19.649060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.546095+0000","last_change":"2024-04-25T08:55:06.523087+0000","last_active":"2024-04-25T08:55:07.546095+0000","last_peered":"2024-04-25T08:55:07.546095+0000","last_clean":"2024-04-25T08:55:07.546095+0000","last_became_active":"2024-04-25T08:55:06.522699+0000","last_became_peered":"2024-04-25T08:55:06.522699+0000","last_unstale":"2024-04-25T08:55:07.546095+0000","last_undegraded":"2024-04-25T08:55:07.546095+0000","last_fullsized":"2024-04-25T08:55:07.546095+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:41.115223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.537822+0000","last_change":"2024-04-25T08:55:07.536920+0000","last_active":"2024-04-25T08:55:07.537822+0000","last_peered":"2024-04-25T08:55:07.537822+0000","last_clean":"2024-04-25T08:55:07.537822+0000","last_became_active":"2024-04-25T08:55:07.536620+0000","last_became_peered":"2024-04-25T08:55:07.536620+0000","last_unstale":"2024-04-25T08:55:07.537822+0000","last_undegraded":"2024-04-25T08:55:07.537822+0000","last_fullsized":"2024-04-25T08:55:07.537822+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:29.002070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.516501+0000","last_change":"2024-04-25T08:55:06.524172+0000","last_active":"2024-04-25T08:55:08.516501+0000","last_peered":"2024-04-25T08:55:08.516501+0000","last_clean":"2024-04-25T08:55:08.516501+0000","last_became_active":"2024-04-25T08:55:06.523899+0000","last_became_peered":"2024-04-25T08:55:06.523899+0000","last_unstale":"2024-04-25T08:55:08.516501+0000","last_undegraded":"2024-04-25T08:55:08.516501+0000","last_fullsized":"2024-04-25T08:55:08.516501+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:45.224210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.541250+0000","last_change":"2024-04-25T08:55:07.539175+0000","last_active":"2024-04-25T08:55:07.541250+0000","last_peered":"2024-04-25T08:55:07.541250+0000","last_clean":"2024-04-25T08:55:07.541250+0000","last_became_active":"2024-04-25T08:55:07.538897+0000","last_became_peered":"2024-04-25T08:55:07.538897+0000","last_unstale":"2024-04-25T08:55:07.541250+0000","last_undegraded":"2024-04-25T08:55:07.541250+0000","last_fullsized":"2024-04-25T08:55:07.541250+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:19.649060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.520259+0000","last_change":"2024-04-25T08:55:06.525766+0000","last_active":"2024-04-25T08:55:07.520259+0000","last_peered":"2024-04-25T08:55:07.520259+0000","last_clean":"2024-04-25T08:55:07.520259+0000","last_became_active":"2024-04-25T08:55:06.525489+0000","last_became_peered":"2024-04-25T08:55:06.525489+0000","last_unstale":"2024-04-25T08:55:07.520259+0000","last_undegraded":"2024-04-25T08:55:07.520259+0000","last_fullsized":"2024-04-25T08:55:07.520259+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:52.528168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.530147+0000","last_change":"2024-04-25T08:55:07.526998+0000","last_active":"2024-04-25T08:55:07.530147+0000","last_peered":"2024-04-25T08:55:07.530147+0000","last_clean":"2024-04-25T08:55:07.530147+0000","last_became_active":"2024-04-25T08:55:07.526703+0000","last_became_peered":"2024-04-25T08:55:07.526703+0000","last_unstale":"2024-04-25T08:55:07.530147+0000","last_undegraded":"2024-04-25T08:55:07.530147+0000","last_fullsized":"2024-04-25T08:55:07.530147+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:36.576436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.511708+0000","last_change":"2024-04-25T08:55:06.529432+0000","last_active":"2024-04-25T08:55:07.511708+0000","last_peered":"2024-04-25T08:55:07.511708+0000","last_clean":"2024-04-25T08:55:07.511708+0000","last_became_active":"2024-04-25T08:55:06.529118+0000","last_became_peered":"2024-04-25T08:55:06.529118+0000","last_unstale":"2024-04-25T08:55:07.511708+0000","last_undegraded":"2024-04-25T08:55:07.511708+0000","last_fullsized":"2024-04-25T08:55:07.511708+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:35.979475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.541695+0000","last_change":"2024-04-25T08:55:07.533478+0000","last_active":"2024-04-25T08:55:07.541695+0000","last_peered":"2024-04-25T08:55:07.541695+0000","last_clean":"2024-04-25T08:55:07.541695+0000","last_became_active":"2024-04-25T08:55:07.533151+0000","last_became_peered":"2024-04-25T08:55:07.533151+0000","last_unstale":"2024-04-25T08:55:07.541695+0000","last_undegraded":"2024-04-25T08:55:07.541695+0000","last_fullsized":"2024-04-25T08:55:07.541695+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:14:13.129793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.542584+0000","last_change":"2024-04-25T08:55:06.522276+0000","last_active":"2024-04-25T08:55:07.542584+0000","last_peered":"2024-04-25T08:55:07.542584+0000","last_clean":"2024-04-25T08:55:07.542584+0000","last_became_active":"2024-04-25T08:55:06.521990+0000","last_became_peered":"2024-04-25T08:55:06.521990+0000","last_unstale":"2024-04-25T08:55:07.542584+0000","last_undegraded":"2024-04-25T08:55:07.542584+0000","last_fullsized":"2024-04-25T08:55:07.542584+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:35.574390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.515512+0000","last_change":"2024-04-25T08:55:07.534236+0000","last_active":"2024-04-25T08:55:08.515512+0000","last_peered":"2024-04-25T08:55:08.515512+0000","last_clean":"2024-04-25T08:55:08.515512+0000","last_became_active":"2024-04-25T08:55:07.532222+0000","last_became_peered":"2024-04-25T08:55:07.532222+0000","last_unstale":"2024-04-25T08:55:08.515512+0000","last_undegraded":"2024-04-25T08:55:08.515512+0000","last_fullsized":"2024-04-25T08:55:08.515512+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:22.632076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.543003+0000","last_change":"2024-04-25T08:55:06.523259+0000","last_active":"2024-04-25T08:55:07.543003+0000","last_peered":"2024-04-25T08:55:07.543003+0000","last_clean":"2024-04-25T08:55:07.543003+0000","last_became_active":"2024-04-25T08:55:06.522945+0000","last_became_peered":"2024-04-25T08:55:06.522945+0000","last_unstale":"2024-04-25T08:55:07.543003+0000","last_undegraded":"2024-04-25T08:55:07.543003+0000","last_fullsized":"2024-04-25T08:55:07.543003+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:14:12.127747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.516287+0000","last_change":"2024-04-25T08:55:07.536941+0000","last_active":"2024-04-25T08:55:08.516287+0000","last_peered":"2024-04-25T08:55:08.516287+0000","last_clean":"2024-04-25T08:55:08.516287+0000","last_became_active":"2024-04-25T08:55:07.536550+0000","last_became_peered":"2024-04-25T08:55:07.536550+0000","last_unstale":"2024-04-25T08:55:08.516287+0000","last_undegraded":"2024-04-25T08:55:08.516287+0000","last_fullsized":"2024-04-25T08:55:08.516287+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:27.043053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.515625+0000","last_change":"2024-04-25T08:55:06.523436+0000","last_active":"2024-04-25T08:55:08.515625+0000","last_peered":"2024-04-25T08:55:08.515625+0000","last_clean":"2024-04-25T08:55:08.515625+0000","last_became_active":"2024-04-25T08:55:06.523079+0000","last_became_peered":"2024-04-25T08:55:06.523079+0000","last_unstale":"2024-04-25T08:55:08.515625+0000","last_undegraded":"2024-04-25T08:55:08.515625+0000","last_fullsized":"2024-04-25T08:55:08.515625+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:25.799809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.536346+0000","last_change":"2024-04-25T08:55:07.532736+0000","last_active":"2024-04-25T08:55:07.536346+0000","last_peered":"2024-04-25T08:55:07.536346+0000","last_clean":"2024-04-25T08:55:07.536346+0000","last_became_active":"2024-04-25T08:55:07.532438+0000","last_became_peered":"2024-04-25T08:55:07.532438+0000","last_unstale":"2024-04-25T08:55:07.536346+0000","last_undegraded":"2024-04-25T08:55:07.536346+0000","last_fullsized":"2024-04-25T08:55:07.536346+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:09.261521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.516130+0000","last_change":"2024-04-25T08:55:06.524726+0000","last_active":"2024-04-25T08:55:08.516130+0000","last_peered":"2024-04-25T08:55:08.516130+0000","last_clean":"2024-04-25T08:55:08.516130+0000","last_became_active":"2024-04-25T08:55:06.524417+0000","last_became_peered":"2024-04-25T08:55:06.524417+0000","last_unstale":"2024-04-25T08:55:08.516130+0000","last_undegraded":"2024-04-25T08:55:08.516130+0000","last_fullsized":"2024-04-25T08:55:08.516130+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:23.342472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.531059+0000","last_change":"2024-04-25T08:55:07.527592+0000","last_active":"2024-04-25T08:55:07.531059+0000","last_peered":"2024-04-25T08:55:07.531059+0000","last_clean":"2024-04-25T08:55:07.531059+0000","last_became_active":"2024-04-25T08:55:07.527318+0000","last_became_peered":"2024-04-25T08:55:07.527318+0000","last_unstale":"2024-04-25T08:55:07.531059+0000","last_undegraded":"2024-04-25T08:55:07.531059+0000","last_fullsized":"2024-04-25T08:55:07.531059+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:24.344517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.516528+0000","last_change":"2024-04-25T08:55:06.524005+0000","last_active":"2024-04-25T08:55:08.516528+0000","last_peered":"2024-04-25T08:55:08.516528+0000","last_clean":"2024-04-25T08:55:08.516528+0000","last_became_active":"2024-04-25T08:55:06.522976+0000","last_became_peered":"2024-04-25T08:55:06.522976+0000","last_unstale":"2024-04-25T08:55:08.516528+0000","last_undegraded":"2024-04-25T08:55:08.516528+0000","last_fullsized":"2024-04-25T08:55:08.516528+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:08.772119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.527238+0000","last_change":"2024-04-25T08:55:07.524575+0000","last_active":"2024-04-25T08:55:07.527238+0000","last_peered":"2024-04-25T08:55:07.527238+0000","last_clean":"2024-04-25T08:55:07.527238+0000","last_became_active":"2024-04-25T08:55:07.524283+0000","last_became_peered":"2024-04-25T08:55:07.524283+0000","last_unstale":"2024-04-25T08:55:07.527238+0000","last_undegraded":"2024-04-25T08:55:07.527238+0000","last_fullsized":"2024-04-25T08:55:07.527238+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:11.974656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.511899+0000","last_change":"2024-04-25T08:55:06.522913+0000","last_active":"2024-04-25T08:55:08.511899+0000","last_peered":"2024-04-25T08:55:08.511899+0000","last_clean":"2024-04-25T08:55:08.511899+0000","last_became_active":"2024-04-25T08:55:06.520103+0000","last_became_peered":"2024-04-25T08:55:06.520103+0000","last_unstale":"2024-04-25T08:55:08.511899+0000","last_undegraded":"2024-04-25T08:55:08.511899+0000","last_fullsized":"2024-04-25T08:55:08.511899+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:10.907040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.535417+0000","last_change":"2024-04-25T08:55:07.531740+0000","last_active":"2024-04-25T08:55:07.535417+0000","last_peered":"2024-04-25T08:55:07.535417+0000","last_clean":"2024-04-25T08:55:07.535417+0000","last_became_active":"2024-04-25T08:55:07.531475+0000","last_became_peered":"2024-04-25T08:55:07.531475+0000","last_unstale":"2024-04-25T08:55:07.535417+0000","last_undegraded":"2024-04-25T08:55:07.535417+0000","last_fullsized":"2024-04-25T08:55:07.535417+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:26.801855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.522734+0000","last_change":"2024-04-25T08:55:06.528210+0000","last_active":"2024-04-25T08:55:07.522734+0000","last_peered":"2024-04-25T08:55:07.522734+0000","last_clean":"2024-04-25T08:55:07.522734+0000","last_became_active":"2024-04-25T08:55:06.527957+0000","last_became_peered":"2024-04-25T08:55:06.527957+0000","last_unstale":"2024-04-25T08:55:07.522734+0000","last_undegraded":"2024-04-25T08:55:07.522734+0000","last_fullsized":"2024-04-25T08:55:07.522734+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:19.451377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.531069+0000","last_change":"2024-04-25T08:55:07.527730+0000","last_active":"2024-04-25T08:55:07.531069+0000","last_peered":"2024-04-25T08:55:07.531069+0000","last_clean":"2024-04-25T08:55:07.531069+0000","last_became_active":"2024-04-25T08:55:07.527457+0000","last_became_peered":"2024-04-25T08:55:07.527457+0000","last_unstale":"2024-04-25T08:55:07.531069+0000","last_undegraded":"2024-04-25T08:55:07.531069+0000","last_fullsized":"2024-04-25T08:55:07.531069+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:44.140070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.515646+0000","last_change":"2024-04-25T08:55:06.527054+0000","last_active":"2024-04-25T08:55:08.515646+0000","last_peered":"2024-04-25T08:55:08.515646+0000","last_clean":"2024-04-25T08:55:08.515646+0000","last_became_active":"2024-04-25T08:55:06.526762+0000","last_became_peered":"2024-04-25T08:55:06.526762+0000","last_unstale":"2024-04-25T08:55:08.515646+0000","last_undegraded":"2024-04-25T08:55:08.515646+0000","last_fullsized":"2024-04-25T08:55:08.515646+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:59.799720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.536974+0000","last_change":"2024-04-25T08:55:07.530782+0000","last_active":"2024-04-25T08:55:07.536974+0000","last_peered":"2024-04-25T08:55:07.536974+0000","last_clean":"2024-04-25T08:55:07.536974+0000","last_became_active":"2024-04-25T08:55:07.530527+0000","last_became_peered":"2024-04-25T08:55:07.530527+0000","last_unstale":"2024-04-25T08:55:07.536974+0000","last_undegraded":"2024-04-25T08:55:07.536974+0000","last_fullsized":"2024-04-25T08:55:07.536974+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:29.002070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.515024+0000","last_change":"2024-04-25T08:55:06.525945+0000","last_active":"2024-04-25T08:55:08.515024+0000","last_peered":"2024-04-25T08:55:08.515024+0000","last_clean":"2024-04-25T08:55:08.515024+0000","last_became_active":"2024-04-25T08:55:06.525643+0000","last_became_peered":"2024-04-25T08:55:06.525643+0000","last_unstale":"2024-04-25T08:55:08.515024+0000","last_undegraded":"2024-04-25T08:55:08.515024+0000","last_fullsized":"2024-04-25T08:55:08.515024+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:35.979475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.535702+0000","last_change":"2024-04-25T08:55:07.533596+0000","last_active":"2024-04-25T08:55:07.535702+0000","last_peered":"2024-04-25T08:55:07.535702+0000","last_clean":"2024-04-25T08:55:07.535702+0000","last_became_active":"2024-04-25T08:55:07.533325+0000","last_became_peered":"2024-04-25T08:55:07.533325+0000","last_unstale":"2024-04-25T08:55:07.535702+0000","last_undegraded":"2024-04-25T08:55:07.535702+0000","last_fullsized":"2024-04-25T08:55:07.535702+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:00.801766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.539070+0000","last_change":"2024-04-25T08:55:06.522733+0000","last_active":"2024-04-25T08:55:07.539070+0000","last_peered":"2024-04-25T08:55:07.539070+0000","last_clean":"2024-04-25T08:55:07.539070+0000","last_became_active":"2024-04-25T08:55:06.522285+0000","last_became_peered":"2024-04-25T08:55:06.522285+0000","last_unstale":"2024-04-25T08:55:07.539070+0000","last_undegraded":"2024-04-25T08:55:07.539070+0000","last_fullsized":"2024-04-25T08:55:07.539070+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:35.574390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.516272+0000","last_change":"2024-04-25T08:55:07.534209+0000","last_active":"2024-04-25T08:55:08.516272+0000","last_peered":"2024-04-25T08:55:08.516272+0000","last_clean":"2024-04-25T08:55:08.516272+0000","last_became_active":"2024-04-25T08:55:07.533152+0000","last_became_peered":"2024-04-25T08:55:07.533152+0000","last_unstale":"2024-04-25T08:55:08.516272+0000","last_undegraded":"2024-04-25T08:55:08.516272+0000","last_fullsized":"2024-04-25T08:55:08.516272+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:43.220568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.511874+0000","last_change":"2024-04-25T08:55:06.519043+0000","last_active":"2024-04-25T08:55:08.511874+0000","last_peered":"2024-04-25T08:55:08.511874+0000","last_clean":"2024-04-25T08:55:08.511874+0000","last_became_active":"2024-04-25T08:55:06.518727+0000","last_became_peered":"2024-04-25T08:55:06.518727+0000","last_unstale":"2024-04-25T08:55:08.511874+0000","last_undegraded":"2024-04-25T08:55:08.511874+0000","last_fullsized":"2024-04-25T08:55:08.511874+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:06.263241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.511827+0000","last_change":"2024-04-25T08:55:07.530829+0000","last_active":"2024-04-25T08:55:08.511827+0000","last_peered":"2024-04-25T08:55:08.511827+0000","last_clean":"2024-04-25T08:55:08.511827+0000","last_became_active":"2024-04-25T08:55:07.530565+0000","last_became_peered":"2024-04-25T08:55:07.530565+0000","last_unstale":"2024-04-25T08:55:08.511827+0000","last_undegraded":"2024-04-25T08:55:08.511827+0000","last_fullsized":"2024-04-25T08:55:08.511827+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:26.245579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.520886+0000","last_change":"2024-04-25T08:55:06.524167+0000","last_active":"2024-04-25T08:55:07.520886+0000","last_peered":"2024-04-25T08:55:07.520886+0000","last_clean":"2024-04-25T08:55:07.520886+0000","last_became_active":"2024-04-25T08:55:06.523879+0000","last_became_peered":"2024-04-25T08:55:06.523879+0000","last_unstale":"2024-04-25T08:55:07.520886+0000","last_undegraded":"2024-04-25T08:55:07.520886+0000","last_fullsized":"2024-04-25T08:55:07.520886+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:21.630030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.536182+0000","last_change":"2024-04-25T08:55:07.530975+0000","last_active":"2024-04-25T08:55:07.536182+0000","last_peered":"2024-04-25T08:55:07.536182+0000","last_clean":"2024-04-25T08:55:07.536182+0000","last_became_active":"2024-04-25T08:55:07.530233+0000","last_became_peered":"2024-04-25T08:55:07.530233+0000","last_unstale":"2024-04-25T08:55:07.536182+0000","last_undegraded":"2024-04-25T08:55:07.536182+0000","last_fullsized":"2024-04-25T08:55:07.536182+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:11.909086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.511994+0000","last_change":"2024-04-25T08:55:06.524320+0000","last_active":"2024-04-25T08:55:08.511994+0000","last_peered":"2024-04-25T08:55:08.511994+0000","last_clean":"2024-04-25T08:55:08.511994+0000","last_became_active":"2024-04-25T08:55:06.523284+0000","last_became_peered":"2024-04-25T08:55:06.523284+0000","last_unstale":"2024-04-25T08:55:08.511994+0000","last_undegraded":"2024-04-25T08:55:08.511994+0000","last_fullsized":"2024-04-25T08:55:08.511994+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:52.528168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.511814+0000","last_change":"2024-04-25T08:55:07.533703+0000","last_active":"2024-04-25T08:55:08.511814+0000","last_peered":"2024-04-25T08:55:08.511814+0000","last_clean":"2024-04-25T08:55:08.511814+0000","last_became_active":"2024-04-25T08:55:07.533445+0000","last_became_peered":"2024-04-25T08:55:07.533445+0000","last_unstale":"2024-04-25T08:55:08.511814+0000","last_undegraded":"2024-04-25T08:55:08.511814+0000","last_fullsized":"2024-04-25T08:55:08.511814+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:29.002070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.520411+0000","last_change":"2024-04-25T08:55:06.521544+0000","last_active":"2024-04-25T08:55:07.520411+0000","last_peered":"2024-04-25T08:55:07.520411+0000","last_clean":"2024-04-25T08:55:07.520411+0000","last_became_active":"2024-04-25T08:55:06.521266+0000","last_became_peered":"2024-04-25T08:55:06.521266+0000","last_unstale":"2024-04-25T08:55:07.520411+0000","last_undegraded":"2024-04-25T08:55:07.520411+0000","last_fullsized":"2024-04-25T08:55:07.520411+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:35.979475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.541167+0000","last_change":"2024-04-25T08:55:07.530484+0000","last_active":"2024-04-25T08:55:07.541167+0000","last_peered":"2024-04-25T08:55:07.541167+0000","last_clean":"2024-04-25T08:55:07.541167+0000","last_became_active":"2024-04-25T08:55:07.530195+0000","last_became_peered":"2024-04-25T08:55:07.530195+0000","last_unstale":"2024-04-25T08:55:07.541167+0000","last_undegraded":"2024-04-25T08:55:07.541167+0000","last_fullsized":"2024-04-25T08:55:07.541167+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:01.806799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.515754+0000","last_change":"2024-04-25T08:55:06.523594+0000","last_active":"2024-04-25T08:55:08.515754+0000","last_peered":"2024-04-25T08:55:08.515754+0000","last_clean":"2024-04-25T08:55:08.515754+0000","last_became_active":"2024-04-25T08:55:06.523317+0000","last_became_peered":"2024-04-25T08:55:06.523317+0000","last_unstale":"2024-04-25T08:55:08.515754+0000","last_undegraded":"2024-04-25T08:55:08.515754+0000","last_fullsized":"2024-04-25T08:55:08.515754+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:03.203548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.515547+0000","last_change":"2024-04-25T08:55:07.523690+0000","last_active":"2024-04-25T08:55:08.515547+0000","last_peered":"2024-04-25T08:55:08.515547+0000","last_clean":"2024-04-25T08:55:08.515547+0000","last_became_active":"2024-04-25T08:55:07.523370+0000","last_became_peered":"2024-04-25T08:55:07.523370+0000","last_unstale":"2024-04-25T08:55:08.515547+0000","last_undegraded":"2024-04-25T08:55:08.515547+0000","last_fullsized":"2024-04-25T08:55:08.515547+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:32.166165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.512062+0000","last_change":"2024-04-25T08:55:06.527555+0000","last_active":"2024-04-25T08:55:07.512062+0000","last_peered":"2024-04-25T08:55:07.512062+0000","last_clean":"2024-04-25T08:55:07.512062+0000","last_became_active":"2024-04-25T08:55:06.527229+0000","last_became_peered":"2024-04-25T08:55:06.527229+0000","last_unstale":"2024-04-25T08:55:07.512062+0000","last_undegraded":"2024-04-25T08:55:07.512062+0000","last_fullsized":"2024-04-25T08:55:07.512062+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:21.630030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.530589+0000","last_change":"2024-04-25T08:55:07.529229+0000","last_active":"2024-04-25T08:55:07.530589+0000","last_peered":"2024-04-25T08:55:07.530589+0000","last_clean":"2024-04-25T08:55:07.530589+0000","last_became_active":"2024-04-25T08:55:07.528886+0000","last_became_peered":"2024-04-25T08:55:07.528886+0000","last_unstale":"2024-04-25T08:55:07.530589+0000","last_undegraded":"2024-04-25T08:55:07.530589+0000","last_fullsized":"2024-04-25T08:55:07.530589+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:32.166165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.521546+0000","last_change":"2024-04-25T08:55:06.527498+0000","last_active":"2024-04-25T08:55:07.521546+0000","last_peered":"2024-04-25T08:55:07.521546+0000","last_clean":"2024-04-25T08:55:07.521546+0000","last_became_active":"2024-04-25T08:55:06.527168+0000","last_became_peered":"2024-04-25T08:55:06.527168+0000","last_unstale":"2024-04-25T08:55:07.521546+0000","last_undegraded":"2024-04-25T08:55:07.521546+0000","last_fullsized":"2024-04-25T08:55:07.521546+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:59.329089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":192,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.518362+0000","last_change":"2024-04-25T08:54:00.818044+0000","last_active":"2024-04-25T08:55:07.518362+0000","last_peered":"2024-04-25T08:55:07.518362+0000","last_clean":"2024-04-25T08:55:07.518362+0000","last_became_active":"2024-04-25T08:54:00.817098+0000","last_became_peered":"2024-04-25T08:54:00.817098+0000","last_unstale":"2024-04-25T08:55:07.518362+0000","last_undegraded":"2024-04-25T08:55:07.518362+0000","last_fullsized":"2024-04-25T08:55:07.518362+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_clean_scrub_stamp":"2024-04-25T08:53:59.739677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:48.129513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":958464,"data_stored":730936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":141094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":141150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":140582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31552,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":376832,"data_stored":142112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31540,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340300,"statfs":{"total":96636764160,"available":96604467200,"internally_reserved":0,"allocated":364544,"data_stored":140548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31552,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":376832,"data_stored":142112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":966656,"data_stored":737146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:55:10.508 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:55:10.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.655+0000 7fef15570640 1 Processor -- start 2024-04-25T08:55:10.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.655+0000 7fef15570640 1 -- start start 2024-04-25T08:55:10.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.656+0000 7fef15570640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fef1015f180 0x7fef100cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:10.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.656+0000 7fef15570640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef100b23f0 0x7fef100cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:10.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.656+0000 7fef15570640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef100ccf10 con 0x7fef100b2cc0 2024-04-25T08:55:10.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.656+0000 7fef15570640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef100cccc0 con 0x7fef1015f180 2024-04-25T08:55:10.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.656+0000 7fef15570640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef100cd840 con 0x7fef100b23f0 2024-04-25T08:55:10.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.656+0000 7fef0effd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fef1015f180 0x7fef100cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:10.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.656+0000 7fef0effd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fef1015f180 0x7fef100cf860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38254/0 (socket says 172.21.15.43:38254) 2024-04-25T08:55:10.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.656+0000 7fef0effd640 1 -- 172.21.15.43:0/3702680610 learned_addr learned my addr 172.21.15.43:0/3702680610 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:10.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.657+0000 7fef0e7fc640 1 --2- 172.21.15.43:0/3702680610 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef100b23f0 0x7fef100cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:10.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.657+0000 7fef0effd640 1 -- 172.21.15.43:0/3702680610 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef100b23f0 msgr2=0x7fef100cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:10.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.657+0000 7fef0effd640 1 --2- 172.21.15.43:0/3702680610 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef100b23f0 0x7fef100cfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:10.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.657+0000 7fef0effd640 1 -- 172.21.15.43:0/3702680610 >> v1:172.21.15.193:6789/0 conn(0x7fef100b2cc0 legacy=0x7fef1015f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:10.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.657+0000 7fef0effd640 1 -- 172.21.15.43:0/3702680610 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef100cd1f0 con 0x7fef1015f180 2024-04-25T08:55:10.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.657+0000 7fef0e7fc640 1 --2- 172.21.15.43:0/3702680610 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef100b23f0 0x7fef100cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:10.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.657+0000 7fef0effd640 1 --2- 172.21.15.43:0/3702680610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef1015f180 0x7fef100cf860 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fef00009e50 tx=0x7fef0002a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4148058e39aa8e9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:10.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.658+0000 7fef0dffb640 1 -- 172.21.15.43:0/3702680610 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef00036070 con 0x7fef1015f180 2024-04-25T08:55:10.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.658+0000 7fef0dffb640 1 -- 172.21.15.43:0/3702680610 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef00031600 con 0x7fef1015f180 2024-04-25T08:55:10.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.658+0000 7fef0dffb640 1 -- 172.21.15.43:0/3702680610 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef000319d0 con 0x7fef1015f180 2024-04-25T08:55:10.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.658+0000 7fef15570640 1 -- 172.21.15.43:0/3702680610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef1015f180 msgr2=0x7fef100cf860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:10.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.658+0000 7fef15570640 1 --2- 172.21.15.43:0/3702680610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef1015f180 0x7fef100cf860 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fef00009e50 tx=0x7fef0002a640 comp rx=0 tx=0).stop 2024-04-25T08:55:10.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.658+0000 7fef15570640 1 -- 172.21.15.43:0/3702680610 shutdown_connections 2024-04-25T08:55:10.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.658+0000 7fef15570640 1 --2- 172.21.15.43:0/3702680610 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef100b23f0 0x7fef100cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:10.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.658+0000 7fef15570640 1 --2- 172.21.15.43:0/3702680610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef1015f180 0x7fef100cf860 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:10.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.658+0000 7fef15570640 1 -- 172.21.15.43:0/3702680610 >> 172.21.15.43:0/3702680610 conn(0x7fef100d17f0 msgr2=0x7fef100d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:10.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.658+0000 7fef15570640 1 -- 172.21.15.43:0/3702680610 shutdown_connections 2024-04-25T08:55:10.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.658+0000 7fef15570640 1 -- 172.21.15.43:0/3702680610 wait complete. 2024-04-25T08:55:10.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.660+0000 7fef15570640 1 Processor -- start 2024-04-25T08:55:10.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.660+0000 7fef15570640 1 -- start start 2024-04-25T08:55:10.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.660+0000 7fef15570640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef100b23f0 0x7fef1015c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:10.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.660+0000 7fef15570640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef100b2cc0 0x7fef100c73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:10.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.660+0000 7fef15570640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef1015f180 0x7fef1006d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:10.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.660+0000 7fef15570640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1015f560 con 0x7fef100b23f0 2024-04-25T08:55:10.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.660+0000 7fef15570640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef100cece0 con 0x7fef1015f180 2024-04-25T08:55:10.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.660+0000 7fef15570640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef101f4090 con 0x7fef100b2cc0 2024-04-25T08:55:10.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.660+0000 7fef0effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef100b23f0 0x7fef1015c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:10.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.660+0000 7fef0effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef100b23f0 0x7fef1015c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38268/0 (socket says 172.21.15.43:38268) 2024-04-25T08:55:10.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.660+0000 7fef0effd640 1 -- 172.21.15.43:0/80608389 learned_addr learned my addr 172.21.15.43:0/80608389 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:10.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.661+0000 7fef0e7fc640 1 --2- 172.21.15.43:0/80608389 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef100b2cc0 0x7fef100c73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:10.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.661+0000 7fef0f7fe640 1 --2- 172.21.15.43:0/80608389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef1015f180 0x7fef1006d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:10.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.661+0000 7fef0effd640 1 -- 172.21.15.43:0/80608389 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef100b2cc0 msgr2=0x7fef100c73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:10.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.661+0000 7fef0effd640 1 --2- 172.21.15.43:0/80608389 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef100b2cc0 0x7fef100c73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:10.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.661+0000 7fef0effd640 1 -- 172.21.15.43:0/80608389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef1015f180 msgr2=0x7fef1006d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:10.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.661+0000 7fef0effd640 1 --2- 172.21.15.43:0/80608389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef1015f180 0x7fef1006d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:10.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.661+0000 7fef0effd640 1 -- 172.21.15.43:0/80608389 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef101f4370 con 0x7fef100b23f0 2024-04-25T08:55:10.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.661+0000 7fef0effd640 1 --2- 172.21.15.43:0/80608389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef100b23f0 0x7fef1015c940 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fef00009e50 tx=0x7fef00004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:10.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.661+0000 7fef077fe640 1 -- 172.21.15.43:0/80608389 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef00036050 con 0x7fef100b23f0 2024-04-25T08:55:10.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.661+0000 7fef077fe640 1 -- 172.21.15.43:0/80608389 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef00042070 con 0x7fef100b23f0 2024-04-25T08:55:10.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.661+0000 7fef15570640 1 -- 172.21.15.43:0/80608389 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef101f4650 con 0x7fef100b23f0 2024-04-25T08:55:10.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.661+0000 7fef077fe640 1 -- 172.21.15.43:0/80608389 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef00009510 con 0x7fef100b23f0 2024-04-25T08:55:10.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.661+0000 7fef15570640 1 -- 172.21.15.43:0/80608389 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef100c7930 con 0x7fef100b23f0 2024-04-25T08:55:10.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.662+0000 7fef15570640 1 -- 172.21.15.43:0/80608389 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef10074ef0 con 0x7fef100b23f0 2024-04-25T08:55:10.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.663+0000 7fef077fe640 1 -- 172.21.15.43:0/80608389 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fef00007c50 con 0x7fef100b23f0 2024-04-25T08:55:10.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.663+0000 7fef077fe640 1 --2- 172.21.15.43:0/80608389 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7feef006b4d0 0x7feef006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:10.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.663+0000 7fef077fe640 1 -- 172.21.15.43:0/80608389 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fef000ab010 con 0x7fef100b23f0 2024-04-25T08:55:10.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.663+0000 7fef0e7fc640 1 --2- 172.21.15.43:0/80608389 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7feef006b4d0 0x7feef006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:10.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.664+0000 7fef0e7fc640 1 --2- 172.21.15.43:0/80608389 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7feef006b4d0 0x7feef006d990 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fef1015a9c0 tx=0x7feef8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:10.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.667+0000 7fef077fe640 1 -- 172.21.15.43:0/80608389 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fef0002ae20 con 0x7fef100b23f0 2024-04-25T08:55:10.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.843+0000 7fef15570640 1 -- 172.21.15.43:0/80608389 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fef1006bcd0 con 0x7feef006b4d0 2024-04-25T08:55:10.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.850+0000 7fef077fe640 1 -- 172.21.15.43:0/80608389 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251907 (secure 0 0 0) 0x7feef8004030 con 0x7feef006b4d0 2024-04-25T08:55:10.842 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:55:10.843 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-25T08:55:10.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.853+0000 7fef15570640 1 -- 172.21.15.43:0/80608389 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7feef006b4d0 msgr2=0x7feef006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:10.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.853+0000 7fef15570640 1 --2- 172.21.15.43:0/80608389 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7feef006b4d0 0x7feef006d990 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fef1015a9c0 tx=0x7feef8008040 comp rx=0 tx=0).stop 2024-04-25T08:55:10.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.853+0000 7fef15570640 1 -- 172.21.15.43:0/80608389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef100b23f0 msgr2=0x7fef1015c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:10.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.853+0000 7fef15570640 1 --2- 172.21.15.43:0/80608389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef100b23f0 0x7fef1015c940 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fef00009e50 tx=0x7fef00004a40 comp rx=0 tx=0).stop 2024-04-25T08:55:10.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.853+0000 7fef15570640 1 -- 172.21.15.43:0/80608389 shutdown_connections 2024-04-25T08:55:10.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.853+0000 7fef15570640 1 --2- 172.21.15.43:0/80608389 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7feef006b4d0 0x7feef006d990 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:10.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.853+0000 7fef15570640 1 --2- 172.21.15.43:0/80608389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef1015f180 0x7fef1006d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:10.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.853+0000 7fef15570640 1 --2- 172.21.15.43:0/80608389 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef100b2cc0 0x7fef100c73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:10.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.853+0000 7fef15570640 1 --2- 172.21.15.43:0/80608389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef100b23f0 0x7fef1015c940 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:10.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.853+0000 7fef15570640 1 -- 172.21.15.43:0/80608389 >> 172.21.15.43:0/80608389 conn(0x7fef100d17f0 msgr2=0x7fef100b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:10.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.853+0000 7fef15570640 1 -- 172.21.15.43:0/80608389 shutdown_connections 2024-04-25T08:55:10.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:10.853+0000 7fef15570640 1 -- 172.21.15.43:0/80608389 wait complete. 2024-04-25T08:55:10.854 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":62,"stamp":"2024-04-25T08:55:09.707776+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":157,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":236888,"kb_used_data":4052,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737832,"statfs":{"total":773094113280,"available":772851539968,"internally_reserved":0,"allocated":4149248,"data_stored":2315680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.204898"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.544412+0000","last_change":"2024-04-25T08:55:07.536694+0000","last_active":"2024-04-25T08:55:07.544412+0000","last_peered":"2024-04-25T08:55:07.544412+0000","last_clean":"2024-04-25T08:55:07.544412+0000","last_became_active":"2024-04-25T08:55:07.536387+0000","last_became_peered":"2024-04-25T08:55:07.536387+0000","last_unstale":"2024-04-25T08:55:07.544412+0000","last_undegraded":"2024-04-25T08:55:07.544412+0000","last_fullsized":"2024-04-25T08:55:07.544412+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:36.981521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.540328+0000","last_change":"2024-04-25T08:55:07.530129+0000","last_active":"2024-04-25T08:55:07.540328+0000","last_peered":"2024-04-25T08:55:07.540328+0000","last_clean":"2024-04-25T08:55:07.540328+0000","last_became_active":"2024-04-25T08:55:07.529658+0000","last_became_peered":"2024-04-25T08:55:07.529658+0000","last_unstale":"2024-04-25T08:55:07.540328+0000","last_undegraded":"2024-04-25T08:55:07.540328+0000","last_fullsized":"2024-04-25T08:55:07.540328+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:56.437421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.515504+0000","last_change":"2024-04-25T08:55:07.533341+0000","last_active":"2024-04-25T08:55:08.515504+0000","last_peered":"2024-04-25T08:55:08.515504+0000","last_clean":"2024-04-25T08:55:08.515504+0000","last_became_active":"2024-04-25T08:55:07.532998+0000","last_became_peered":"2024-04-25T08:55:07.532998+0000","last_unstale":"2024-04-25T08:55:08.515504+0000","last_undegraded":"2024-04-25T08:55:08.515504+0000","last_fullsized":"2024-04-25T08:55:08.515504+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:18.826664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.536370+0000","last_change":"2024-04-25T08:55:07.533504+0000","last_active":"2024-04-25T08:55:07.536370+0000","last_peered":"2024-04-25T08:55:07.536370+0000","last_clean":"2024-04-25T08:55:07.536370+0000","last_became_active":"2024-04-25T08:55:07.533240+0000","last_became_peered":"2024-04-25T08:55:07.533240+0000","last_unstale":"2024-04-25T08:55:07.536370+0000","last_undegraded":"2024-04-25T08:55:07.536370+0000","last_fullsized":"2024-04-25T08:55:07.536370+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:42.117269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.527256+0000","last_change":"2024-04-25T08:55:07.526227+0000","last_active":"2024-04-25T08:55:07.527256+0000","last_peered":"2024-04-25T08:55:07.527256+0000","last_clean":"2024-04-25T08:55:07.527256+0000","last_became_active":"2024-04-25T08:55:07.525948+0000","last_became_peered":"2024-04-25T08:55:07.525948+0000","last_unstale":"2024-04-25T08:55:07.527256+0000","last_undegraded":"2024-04-25T08:55:07.527256+0000","last_fullsized":"2024-04-25T08:55:07.527256+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:28.891499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.534490+0000","last_change":"2024-04-25T08:55:07.525333+0000","last_active":"2024-04-25T08:55:07.534490+0000","last_peered":"2024-04-25T08:55:07.534490+0000","last_clean":"2024-04-25T08:55:07.534490+0000","last_became_active":"2024-04-25T08:55:07.525073+0000","last_became_peered":"2024-04-25T08:55:07.525073+0000","last_unstale":"2024-04-25T08:55:07.534490+0000","last_undegraded":"2024-04-25T08:55:07.534490+0000","last_fullsized":"2024-04-25T08:55:07.534490+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:26.245579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.530912+0000","last_change":"2024-04-25T08:55:07.527951+0000","last_active":"2024-04-25T08:55:07.530912+0000","last_peered":"2024-04-25T08:55:07.530912+0000","last_clean":"2024-04-25T08:55:07.530912+0000","last_became_active":"2024-04-25T08:55:07.527682+0000","last_became_peered":"2024-04-25T08:55:07.527682+0000","last_unstale":"2024-04-25T08:55:07.530912+0000","last_undegraded":"2024-04-25T08:55:07.530912+0000","last_fullsized":"2024-04-25T08:55:07.530912+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:22.632076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.530108+0000","last_change":"2024-04-25T08:55:07.524922+0000","last_active":"2024-04-25T08:55:07.530108+0000","last_peered":"2024-04-25T08:55:07.530108+0000","last_clean":"2024-04-25T08:55:07.530108+0000","last_became_active":"2024-04-25T08:55:07.524332+0000","last_became_peered":"2024-04-25T08:55:07.524332+0000","last_unstale":"2024-04-25T08:55:07.530108+0000","last_undegraded":"2024-04-25T08:55:07.530108+0000","last_fullsized":"2024-04-25T08:55:07.530108+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:14:13.129793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.516679+0000","last_change":"2024-04-25T08:55:07.528710+0000","last_active":"2024-04-25T08:55:08.516679+0000","last_peered":"2024-04-25T08:55:08.516679+0000","last_clean":"2024-04-25T08:55:08.516679+0000","last_became_active":"2024-04-25T08:55:07.526398+0000","last_became_peered":"2024-04-25T08:55:07.526398+0000","last_unstale":"2024-04-25T08:55:08.516679+0000","last_undegraded":"2024-04-25T08:55:08.516679+0000","last_fullsized":"2024-04-25T08:55:08.516679+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:25.454711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.542514+0000","last_change":"2024-04-25T08:55:07.541632+0000","last_active":"2024-04-25T08:55:07.542514+0000","last_peered":"2024-04-25T08:55:07.542514+0000","last_clean":"2024-04-25T08:55:07.542514+0000","last_became_active":"2024-04-25T08:55:07.541117+0000","last_became_peered":"2024-04-25T08:55:07.541117+0000","last_unstale":"2024-04-25T08:55:07.542514+0000","last_undegraded":"2024-04-25T08:55:07.542514+0000","last_fullsized":"2024-04-25T08:55:07.542514+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:09.909329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.536869+0000","last_change":"2024-04-25T08:55:07.533404+0000","last_active":"2024-04-25T08:55:07.536869+0000","last_peered":"2024-04-25T08:55:07.536869+0000","last_clean":"2024-04-25T08:55:07.536869+0000","last_became_active":"2024-04-25T08:55:07.533015+0000","last_became_peered":"2024-04-25T08:55:07.533015+0000","last_unstale":"2024-04-25T08:55:07.536869+0000","last_undegraded":"2024-04-25T08:55:07.536869+0000","last_fullsized":"2024-04-25T08:55:07.536869+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:01.586986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.515864+0000","last_change":"2024-04-25T08:55:07.538554+0000","last_active":"2024-04-25T08:55:08.515864+0000","last_peered":"2024-04-25T08:55:08.515864+0000","last_clean":"2024-04-25T08:55:08.515864+0000","last_became_active":"2024-04-25T08:55:07.538295+0000","last_became_peered":"2024-04-25T08:55:07.538295+0000","last_unstale":"2024-04-25T08:55:08.515864+0000","last_undegraded":"2024-04-25T08:55:08.515864+0000","last_fullsized":"2024-04-25T08:55:08.515864+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:47.319053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.526373+0000","last_change":"2024-04-25T08:55:07.521745+0000","last_active":"2024-04-25T08:55:07.526373+0000","last_peered":"2024-04-25T08:55:07.526373+0000","last_clean":"2024-04-25T08:55:07.526373+0000","last_became_active":"2024-04-25T08:55:07.521460+0000","last_became_peered":"2024-04-25T08:55:07.521460+0000","last_unstale":"2024-04-25T08:55:07.526373+0000","last_undegraded":"2024-04-25T08:55:07.526373+0000","last_fullsized":"2024-04-25T08:55:07.526373+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:42.854344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.530942+0000","last_change":"2024-04-25T08:55:07.519469+0000","last_active":"2024-04-25T08:55:07.530942+0000","last_peered":"2024-04-25T08:55:07.530942+0000","last_clean":"2024-04-25T08:55:07.530942+0000","last_became_active":"2024-04-25T08:55:07.519169+0000","last_became_peered":"2024-04-25T08:55:07.519169+0000","last_unstale":"2024-04-25T08:55:07.530942+0000","last_undegraded":"2024-04-25T08:55:07.530942+0000","last_fullsized":"2024-04-25T08:55:07.530942+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:29.002070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.515778+0000","last_change":"2024-04-25T08:55:07.535390+0000","last_active":"2024-04-25T08:55:08.515778+0000","last_peered":"2024-04-25T08:55:08.515778+0000","last_clean":"2024-04-25T08:55:08.515778+0000","last_became_active":"2024-04-25T08:55:07.535119+0000","last_became_peered":"2024-04-25T08:55:07.535119+0000","last_unstale":"2024-04-25T08:55:08.515778+0000","last_undegraded":"2024-04-25T08:55:08.515778+0000","last_fullsized":"2024-04-25T08:55:08.515778+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:39.948659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.536169+0000","last_change":"2024-04-25T08:55:07.529965+0000","last_active":"2024-04-25T08:55:07.536169+0000","last_peered":"2024-04-25T08:55:07.536169+0000","last_clean":"2024-04-25T08:55:07.536169+0000","last_became_active":"2024-04-25T08:55:07.529695+0000","last_became_peered":"2024-04-25T08:55:07.529695+0000","last_unstale":"2024-04-25T08:55:07.536169+0000","last_undegraded":"2024-04-25T08:55:07.536169+0000","last_fullsized":"2024-04-25T08:55:07.536169+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:42.117269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.537095+0000","last_change":"2024-04-25T08:55:07.531585+0000","last_active":"2024-04-25T08:55:07.537095+0000","last_peered":"2024-04-25T08:55:07.537095+0000","last_clean":"2024-04-25T08:55:07.537095+0000","last_became_active":"2024-04-25T08:55:07.531331+0000","last_became_peered":"2024-04-25T08:55:07.531331+0000","last_unstale":"2024-04-25T08:55:07.537095+0000","last_undegraded":"2024-04-25T08:55:07.537095+0000","last_fullsized":"2024-04-25T08:55:07.537095+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:26.245579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.540348+0000","last_change":"2024-04-25T08:55:07.530786+0000","last_active":"2024-04-25T08:55:07.540348+0000","last_peered":"2024-04-25T08:55:07.540348+0000","last_clean":"2024-04-25T08:55:07.540348+0000","last_became_active":"2024-04-25T08:55:07.530441+0000","last_became_peered":"2024-04-25T08:55:07.530441+0000","last_unstale":"2024-04-25T08:55:07.540348+0000","last_undegraded":"2024-04-25T08:55:07.540348+0000","last_fullsized":"2024-04-25T08:55:07.540348+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:15.381709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.538324+0000","last_change":"2024-04-25T08:55:07.528902+0000","last_active":"2024-04-25T08:55:07.538324+0000","last_peered":"2024-04-25T08:55:07.538324+0000","last_clean":"2024-04-25T08:55:07.538324+0000","last_became_active":"2024-04-25T08:55:07.528582+0000","last_became_peered":"2024-04-25T08:55:07.528582+0000","last_unstale":"2024-04-25T08:55:07.538324+0000","last_undegraded":"2024-04-25T08:55:07.538324+0000","last_fullsized":"2024-04-25T08:55:07.538324+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:01.586986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.535350+0000","last_change":"2024-04-25T08:55:07.531670+0000","last_active":"2024-04-25T08:55:07.535350+0000","last_peered":"2024-04-25T08:55:07.535350+0000","last_clean":"2024-04-25T08:55:07.535350+0000","last_became_active":"2024-04-25T08:55:07.531371+0000","last_became_peered":"2024-04-25T08:55:07.531371+0000","last_unstale":"2024-04-25T08:55:07.535350+0000","last_undegraded":"2024-04-25T08:55:07.535350+0000","last_fullsized":"2024-04-25T08:55:07.535350+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:10.672523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.516226+0000","last_change":"2024-04-25T08:55:07.532759+0000","last_active":"2024-04-25T08:55:08.516226+0000","last_peered":"2024-04-25T08:55:08.516226+0000","last_clean":"2024-04-25T08:55:08.516226+0000","last_became_active":"2024-04-25T08:55:07.531953+0000","last_became_peered":"2024-04-25T08:55:07.531953+0000","last_unstale":"2024-04-25T08:55:08.516226+0000","last_undegraded":"2024-04-25T08:55:08.516226+0000","last_fullsized":"2024-04-25T08:55:08.516226+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:09.261521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.516399+0000","last_change":"2024-04-25T08:55:07.519517+0000","last_active":"2024-04-25T08:55:08.516399+0000","last_peered":"2024-04-25T08:55:08.516399+0000","last_clean":"2024-04-25T08:55:08.516399+0000","last_became_active":"2024-04-25T08:55:07.519246+0000","last_became_peered":"2024-04-25T08:55:07.519246+0000","last_unstale":"2024-04-25T08:55:08.516399+0000","last_undegraded":"2024-04-25T08:55:08.516399+0000","last_fullsized":"2024-04-25T08:55:08.516399+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:29.002070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.511605+0000","last_change":"2024-04-25T08:55:07.532420+0000","last_active":"2024-04-25T08:55:08.511605+0000","last_peered":"2024-04-25T08:55:08.511605+0000","last_clean":"2024-04-25T08:55:08.511605+0000","last_became_active":"2024-04-25T08:55:07.532161+0000","last_became_peered":"2024-04-25T08:55:07.532161+0000","last_unstale":"2024-04-25T08:55:08.511605+0000","last_undegraded":"2024-04-25T08:55:08.511605+0000","last_fullsized":"2024-04-25T08:55:08.511605+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:09.261521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.515822+0000","last_change":"2024-04-25T08:55:07.532589+0000","last_active":"2024-04-25T08:55:08.515822+0000","last_peered":"2024-04-25T08:55:08.515822+0000","last_clean":"2024-04-25T08:55:08.515822+0000","last_became_active":"2024-04-25T08:55:07.532321+0000","last_became_peered":"2024-04-25T08:55:07.532321+0000","last_unstale":"2024-04-25T08:55:08.515822+0000","last_undegraded":"2024-04-25T08:55:08.515822+0000","last_fullsized":"2024-04-25T08:55:08.515822+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:15.381709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.531128+0000","last_change":"2024-04-25T08:55:07.530094+0000","last_active":"2024-04-25T08:55:07.531128+0000","last_peered":"2024-04-25T08:55:07.531128+0000","last_clean":"2024-04-25T08:55:07.531128+0000","last_became_active":"2024-04-25T08:55:07.529829+0000","last_became_peered":"2024-04-25T08:55:07.529829+0000","last_unstale":"2024-04-25T08:55:07.531128+0000","last_undegraded":"2024-04-25T08:55:07.531128+0000","last_fullsized":"2024-04-25T08:55:07.531128+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:32.166165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.540306+0000","last_change":"2024-04-25T08:55:07.529894+0000","last_active":"2024-04-25T08:55:07.540306+0000","last_peered":"2024-04-25T08:55:07.540306+0000","last_clean":"2024-04-25T08:55:07.540306+0000","last_became_active":"2024-04-25T08:55:07.529301+0000","last_became_peered":"2024-04-25T08:55:07.529301+0000","last_unstale":"2024-04-25T08:55:07.540306+0000","last_undegraded":"2024-04-25T08:55:07.540306+0000","last_fullsized":"2024-04-25T08:55:07.540306+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:09.909329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.540650+0000","last_change":"2024-04-25T08:55:07.538292+0000","last_active":"2024-04-25T08:55:07.540650+0000","last_peered":"2024-04-25T08:55:07.540650+0000","last_clean":"2024-04-25T08:55:07.540650+0000","last_became_active":"2024-04-25T08:55:07.537992+0000","last_became_peered":"2024-04-25T08:55:07.537992+0000","last_unstale":"2024-04-25T08:55:07.540650+0000","last_undegraded":"2024-04-25T08:55:07.540650+0000","last_fullsized":"2024-04-25T08:55:07.540650+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:26.245579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.516022+0000","last_change":"2024-04-25T08:55:07.530437+0000","last_active":"2024-04-25T08:55:08.516022+0000","last_peered":"2024-04-25T08:55:08.516022+0000","last_clean":"2024-04-25T08:55:08.516022+0000","last_became_active":"2024-04-25T08:55:07.530143+0000","last_became_peered":"2024-04-25T08:55:07.530143+0000","last_unstale":"2024-04-25T08:55:08.516022+0000","last_undegraded":"2024-04-25T08:55:08.516022+0000","last_fullsized":"2024-04-25T08:55:08.516022+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:20.700269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.516555+0000","last_change":"2024-04-25T08:55:07.520199+0000","last_active":"2024-04-25T08:55:08.516555+0000","last_peered":"2024-04-25T08:55:08.516555+0000","last_clean":"2024-04-25T08:55:08.516555+0000","last_became_active":"2024-04-25T08:55:07.519924+0000","last_became_peered":"2024-04-25T08:55:07.519924+0000","last_unstale":"2024-04-25T08:55:08.516555+0000","last_undegraded":"2024-04-25T08:55:08.516555+0000","last_fullsized":"2024-04-25T08:55:08.516555+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:26.245579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.530743+0000","last_change":"2024-04-25T08:55:07.529557+0000","last_active":"2024-04-25T08:55:07.530743+0000","last_peered":"2024-04-25T08:55:07.530743+0000","last_clean":"2024-04-25T08:55:07.530743+0000","last_became_active":"2024-04-25T08:55:07.529259+0000","last_became_peered":"2024-04-25T08:55:07.529259+0000","last_unstale":"2024-04-25T08:55:07.530743+0000","last_undegraded":"2024-04-25T08:55:07.530743+0000","last_fullsized":"2024-04-25T08:55:07.530743+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:09.261521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.541637+0000","last_change":"2024-04-25T08:55:07.532819+0000","last_active":"2024-04-25T08:55:07.541637+0000","last_peered":"2024-04-25T08:55:07.541637+0000","last_clean":"2024-04-25T08:55:07.541637+0000","last_became_active":"2024-04-25T08:55:07.532527+0000","last_became_peered":"2024-04-25T08:55:07.532527+0000","last_unstale":"2024-04-25T08:55:07.541637+0000","last_undegraded":"2024-04-25T08:55:07.541637+0000","last_fullsized":"2024-04-25T08:55:07.541637+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:53.530214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.530128+0000","last_change":"2024-04-25T08:55:07.525052+0000","last_active":"2024-04-25T08:55:07.530128+0000","last_peered":"2024-04-25T08:55:07.530128+0000","last_clean":"2024-04-25T08:55:07.530128+0000","last_became_active":"2024-04-25T08:55:07.524758+0000","last_became_peered":"2024-04-25T08:55:07.524758+0000","last_unstale":"2024-04-25T08:55:07.530128+0000","last_undegraded":"2024-04-25T08:55:07.530128+0000","last_fullsized":"2024-04-25T08:55:07.530128+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:36.981521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.511749+0000","last_change":"2024-04-25T08:55:07.533290+0000","last_active":"2024-04-25T08:55:08.511749+0000","last_peered":"2024-04-25T08:55:08.511749+0000","last_clean":"2024-04-25T08:55:08.511749+0000","last_became_active":"2024-04-25T08:55:07.532818+0000","last_became_peered":"2024-04-25T08:55:07.532818+0000","last_unstale":"2024-04-25T08:55:08.511749+0000","last_undegraded":"2024-04-25T08:55:08.511749+0000","last_fullsized":"2024-04-25T08:55:08.511749+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:42.117269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"32'4","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.540469+0000","last_change":"2024-04-25T08:55:06.532332+0000","last_active":"2024-04-25T08:55:07.540469+0000","last_peered":"2024-04-25T08:55:07.540469+0000","last_clean":"2024-04-25T08:55:07.540469+0000","last_became_active":"2024-04-25T08:55:06.532043+0000","last_became_peered":"2024-04-25T08:55:06.532043+0000","last_unstale":"2024-04-25T08:55:07.540469+0000","last_undegraded":"2024-04-25T08:55:07.540469+0000","last_fullsized":"2024-04-25T08:55:07.540469+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:57.871761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.535951+0000","last_change":"2024-04-25T08:55:07.530580+0000","last_active":"2024-04-25T08:55:07.535951+0000","last_peered":"2024-04-25T08:55:07.535951+0000","last_clean":"2024-04-25T08:55:07.535951+0000","last_became_active":"2024-04-25T08:55:07.530275+0000","last_became_peered":"2024-04-25T08:55:07.530275+0000","last_unstale":"2024-04-25T08:55:07.535951+0000","last_undegraded":"2024-04-25T08:55:07.535951+0000","last_fullsized":"2024-04-25T08:55:07.535951+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:07.265287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.511762+0000","last_change":"2024-04-25T08:55:06.526275+0000","last_active":"2024-04-25T08:55:08.511762+0000","last_peered":"2024-04-25T08:55:08.511762+0000","last_clean":"2024-04-25T08:55:08.511762+0000","last_became_active":"2024-04-25T08:55:06.525943+0000","last_became_peered":"2024-04-25T08:55:06.525943+0000","last_unstale":"2024-04-25T08:55:08.511762+0000","last_undegraded":"2024-04-25T08:55:08.511762+0000","last_fullsized":"2024-04-25T08:55:08.511762+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:19.451377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.516609+0000","last_change":"2024-04-25T08:55:07.527049+0000","last_active":"2024-04-25T08:55:08.516609+0000","last_peered":"2024-04-25T08:55:08.516609+0000","last_clean":"2024-04-25T08:55:08.516609+0000","last_became_active":"2024-04-25T08:55:07.526777+0000","last_became_peered":"2024-04-25T08:55:07.526777+0000","last_unstale":"2024-04-25T08:55:08.516609+0000","last_undegraded":"2024-04-25T08:55:08.516609+0000","last_fullsized":"2024-04-25T08:55:08.516609+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:01.586986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.542825+0000","last_change":"2024-04-25T08:55:06.529754+0000","last_active":"2024-04-25T08:55:07.542825+0000","last_peered":"2024-04-25T08:55:07.542825+0000","last_clean":"2024-04-25T08:55:07.542825+0000","last_became_active":"2024-04-25T08:55:06.529439+0000","last_became_peered":"2024-04-25T08:55:06.529439+0000","last_unstale":"2024-04-25T08:55:07.542825+0000","last_undegraded":"2024-04-25T08:55:07.542825+0000","last_fullsized":"2024-04-25T08:55:07.542825+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:31.164120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.540673+0000","last_change":"2024-04-25T08:55:07.538476+0000","last_active":"2024-04-25T08:55:07.540673+0000","last_peered":"2024-04-25T08:55:07.540673+0000","last_clean":"2024-04-25T08:55:07.540673+0000","last_became_active":"2024-04-25T08:55:07.537354+0000","last_became_peered":"2024-04-25T08:55:07.537354+0000","last_unstale":"2024-04-25T08:55:07.540673+0000","last_undegraded":"2024-04-25T08:55:07.540673+0000","last_fullsized":"2024-04-25T08:55:07.540673+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:01.586986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.532620+0000","last_change":"2024-04-25T08:55:06.522349+0000","last_active":"2024-04-25T08:55:07.532620+0000","last_peered":"2024-04-25T08:55:07.532620+0000","last_clean":"2024-04-25T08:55:07.532620+0000","last_became_active":"2024-04-25T08:55:06.522066+0000","last_became_peered":"2024-04-25T08:55:06.522066+0000","last_unstale":"2024-04-25T08:55:07.532620+0000","last_undegraded":"2024-04-25T08:55:07.532620+0000","last_fullsized":"2024-04-25T08:55:07.532620+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:00.350645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.536945+0000","last_change":"2024-04-25T08:55:07.534022+0000","last_active":"2024-04-25T08:55:07.536945+0000","last_peered":"2024-04-25T08:55:07.536945+0000","last_clean":"2024-04-25T08:55:07.536945+0000","last_became_active":"2024-04-25T08:55:07.533736+0000","last_became_peered":"2024-04-25T08:55:07.533736+0000","last_unstale":"2024-04-25T08:55:07.536945+0000","last_undegraded":"2024-04-25T08:55:07.536945+0000","last_fullsized":"2024-04-25T08:55:07.536945+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:25.454711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.516361+0000","last_change":"2024-04-25T08:55:06.526224+0000","last_active":"2024-04-25T08:55:08.516361+0000","last_peered":"2024-04-25T08:55:08.516361+0000","last_clean":"2024-04-25T08:55:08.516361+0000","last_became_active":"2024-04-25T08:55:06.525952+0000","last_became_peered":"2024-04-25T08:55:06.525952+0000","last_unstale":"2024-04-25T08:55:08.516361+0000","last_undegraded":"2024-04-25T08:55:08.516361+0000","last_fullsized":"2024-04-25T08:55:08.516361+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:12.207955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.515656+0000","last_change":"2024-04-25T08:55:07.531473+0000","last_active":"2024-04-25T08:55:08.515656+0000","last_peered":"2024-04-25T08:55:08.515656+0000","last_clean":"2024-04-25T08:55:08.515656+0000","last_became_active":"2024-04-25T08:55:07.531204+0000","last_became_peered":"2024-04-25T08:55:07.531204+0000","last_unstale":"2024-04-25T08:55:08.515656+0000","last_undegraded":"2024-04-25T08:55:08.515656+0000","last_fullsized":"2024-04-25T08:55:08.515656+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:22.996931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.516506+0000","last_change":"2024-04-25T08:55:06.526543+0000","last_active":"2024-04-25T08:55:08.516506+0000","last_peered":"2024-04-25T08:55:08.516506+0000","last_clean":"2024-04-25T08:55:08.516506+0000","last_became_active":"2024-04-25T08:55:06.526141+0000","last_became_peered":"2024-04-25T08:55:06.526141+0000","last_unstale":"2024-04-25T08:55:08.516506+0000","last_undegraded":"2024-04-25T08:55:08.516506+0000","last_fullsized":"2024-04-25T08:55:08.516506+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:43.138025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.516258+0000","last_change":"2024-04-25T08:55:07.532267+0000","last_active":"2024-04-25T08:55:08.516258+0000","last_peered":"2024-04-25T08:55:08.516258+0000","last_clean":"2024-04-25T08:55:08.516258+0000","last_became_active":"2024-04-25T08:55:07.531661+0000","last_became_peered":"2024-04-25T08:55:07.531661+0000","last_unstale":"2024-04-25T08:55:08.516258+0000","last_undegraded":"2024-04-25T08:55:08.516258+0000","last_fullsized":"2024-04-25T08:55:08.516258+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:42.117269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.511488+0000","last_change":"2024-04-25T08:55:06.524478+0000","last_active":"2024-04-25T08:55:07.511488+0000","last_peered":"2024-04-25T08:55:07.511488+0000","last_clean":"2024-04-25T08:55:07.511488+0000","last_became_active":"2024-04-25T08:55:06.524189+0000","last_became_peered":"2024-04-25T08:55:06.524189+0000","last_unstale":"2024-04-25T08:55:07.511488+0000","last_undegraded":"2024-04-25T08:55:07.511488+0000","last_fullsized":"2024-04-25T08:55:07.511488+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:14:12.127747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.540440+0000","last_change":"2024-04-25T08:55:07.539319+0000","last_active":"2024-04-25T08:55:07.540440+0000","last_peered":"2024-04-25T08:55:07.540440+0000","last_clean":"2024-04-25T08:55:07.540440+0000","last_became_active":"2024-04-25T08:55:07.538641+0000","last_became_peered":"2024-04-25T08:55:07.538641+0000","last_unstale":"2024-04-25T08:55:07.540440+0000","last_undegraded":"2024-04-25T08:55:07.540440+0000","last_fullsized":"2024-04-25T08:55:07.540440+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:25.454711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.516287+0000","last_change":"2024-04-25T08:55:06.525229+0000","last_active":"2024-04-25T08:55:08.516287+0000","last_peered":"2024-04-25T08:55:08.516287+0000","last_clean":"2024-04-25T08:55:08.516287+0000","last_became_active":"2024-04-25T08:55:06.524943+0000","last_became_peered":"2024-04-25T08:55:06.524943+0000","last_unstale":"2024-04-25T08:55:08.516287+0000","last_undegraded":"2024-04-25T08:55:08.516287+0000","last_fullsized":"2024-04-25T08:55:08.516287+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:09.670477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.515541+0000","last_change":"2024-04-25T08:55:07.538304+0000","last_active":"2024-04-25T08:55:08.515541+0000","last_peered":"2024-04-25T08:55:08.515541+0000","last_clean":"2024-04-25T08:55:08.515541+0000","last_became_active":"2024-04-25T08:55:07.538057+0000","last_became_peered":"2024-04-25T08:55:07.538057+0000","last_unstale":"2024-04-25T08:55:08.515541+0000","last_undegraded":"2024-04-25T08:55:08.515541+0000","last_fullsized":"2024-04-25T08:55:08.515541+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:53.019544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.513080+0000","last_change":"2024-04-25T08:55:06.529815+0000","last_active":"2024-04-25T08:55:07.513080+0000","last_peered":"2024-04-25T08:55:07.513080+0000","last_clean":"2024-04-25T08:55:07.513080+0000","last_became_active":"2024-04-25T08:55:06.529540+0000","last_became_peered":"2024-04-25T08:55:06.529540+0000","last_unstale":"2024-04-25T08:55:07.513080+0000","last_undegraded":"2024-04-25T08:55:07.513080+0000","last_fullsized":"2024-04-25T08:55:07.513080+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:08.259475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.539547+0000","last_change":"2024-04-25T08:55:07.531978+0000","last_active":"2024-04-25T08:55:07.539547+0000","last_peered":"2024-04-25T08:55:07.539547+0000","last_clean":"2024-04-25T08:55:07.539547+0000","last_became_active":"2024-04-25T08:55:07.530117+0000","last_became_peered":"2024-04-25T08:55:07.530117+0000","last_unstale":"2024-04-25T08:55:07.539547+0000","last_undegraded":"2024-04-25T08:55:07.539547+0000","last_fullsized":"2024-04-25T08:55:07.539547+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:18.826664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.511854+0000","last_change":"2024-04-25T08:55:06.525065+0000","last_active":"2024-04-25T08:55:08.511854+0000","last_peered":"2024-04-25T08:55:08.511854+0000","last_clean":"2024-04-25T08:55:08.511854+0000","last_became_active":"2024-04-25T08:55:06.523538+0000","last_became_peered":"2024-04-25T08:55:06.523538+0000","last_unstale":"2024-04-25T08:55:08.511854+0000","last_undegraded":"2024-04-25T08:55:08.511854+0000","last_fullsized":"2024-04-25T08:55:08.511854+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:57.871761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.541822+0000","last_change":"2024-04-25T08:55:07.529022+0000","last_active":"2024-04-25T08:55:07.541822+0000","last_peered":"2024-04-25T08:55:07.541822+0000","last_clean":"2024-04-25T08:55:07.541822+0000","last_became_active":"2024-04-25T08:55:07.528742+0000","last_became_peered":"2024-04-25T08:55:07.528742+0000","last_unstale":"2024-04-25T08:55:07.541822+0000","last_undegraded":"2024-04-25T08:55:07.541822+0000","last_fullsized":"2024-04-25T08:55:07.541822+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:25.454711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"32'3","reported_seq":17,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.515606+0000","last_change":"2024-04-25T08:55:06.523452+0000","last_active":"2024-04-25T08:55:08.515606+0000","last_peered":"2024-04-25T08:55:08.515606+0000","last_clean":"2024-04-25T08:55:08.515606+0000","last_became_active":"2024-04-25T08:55:06.523154+0000","last_became_peered":"2024-04-25T08:55:06.523154+0000","last_unstale":"2024-04-25T08:55:08.515606+0000","last_undegraded":"2024-04-25T08:55:08.515606+0000","last_fullsized":"2024-04-25T08:55:08.515606+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:35.574390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.515527+0000","last_change":"2024-04-25T08:55:07.536086+0000","last_active":"2024-04-25T08:55:08.515527+0000","last_peered":"2024-04-25T08:55:08.515527+0000","last_clean":"2024-04-25T08:55:08.515527+0000","last_became_active":"2024-04-25T08:55:07.535072+0000","last_became_peered":"2024-04-25T08:55:07.535072+0000","last_unstale":"2024-04-25T08:55:08.515527+0000","last_undegraded":"2024-04-25T08:55:08.515527+0000","last_fullsized":"2024-04-25T08:55:08.515527+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:19.649060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"32'6","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.546095+0000","last_change":"2024-04-25T08:55:06.523087+0000","last_active":"2024-04-25T08:55:07.546095+0000","last_peered":"2024-04-25T08:55:07.546095+0000","last_clean":"2024-04-25T08:55:07.546095+0000","last_became_active":"2024-04-25T08:55:06.522699+0000","last_became_peered":"2024-04-25T08:55:06.522699+0000","last_unstale":"2024-04-25T08:55:07.546095+0000","last_undegraded":"2024-04-25T08:55:07.546095+0000","last_fullsized":"2024-04-25T08:55:07.546095+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:07:41.115223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.537822+0000","last_change":"2024-04-25T08:55:07.536920+0000","last_active":"2024-04-25T08:55:07.537822+0000","last_peered":"2024-04-25T08:55:07.537822+0000","last_clean":"2024-04-25T08:55:07.537822+0000","last_became_active":"2024-04-25T08:55:07.536620+0000","last_became_peered":"2024-04-25T08:55:07.536620+0000","last_unstale":"2024-04-25T08:55:07.537822+0000","last_undegraded":"2024-04-25T08:55:07.537822+0000","last_fullsized":"2024-04-25T08:55:07.537822+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:29.002070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.516501+0000","last_change":"2024-04-25T08:55:06.524172+0000","last_active":"2024-04-25T08:55:08.516501+0000","last_peered":"2024-04-25T08:55:08.516501+0000","last_clean":"2024-04-25T08:55:08.516501+0000","last_became_active":"2024-04-25T08:55:06.523899+0000","last_became_peered":"2024-04-25T08:55:06.523899+0000","last_unstale":"2024-04-25T08:55:08.516501+0000","last_undegraded":"2024-04-25T08:55:08.516501+0000","last_fullsized":"2024-04-25T08:55:08.516501+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:45.224210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.541250+0000","last_change":"2024-04-25T08:55:07.539175+0000","last_active":"2024-04-25T08:55:07.541250+0000","last_peered":"2024-04-25T08:55:07.541250+0000","last_clean":"2024-04-25T08:55:07.541250+0000","last_became_active":"2024-04-25T08:55:07.538897+0000","last_became_peered":"2024-04-25T08:55:07.538897+0000","last_unstale":"2024-04-25T08:55:07.541250+0000","last_undegraded":"2024-04-25T08:55:07.541250+0000","last_fullsized":"2024-04-25T08:55:07.541250+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:19.649060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.520259+0000","last_change":"2024-04-25T08:55:06.525766+0000","last_active":"2024-04-25T08:55:07.520259+0000","last_peered":"2024-04-25T08:55:07.520259+0000","last_clean":"2024-04-25T08:55:07.520259+0000","last_became_active":"2024-04-25T08:55:06.525489+0000","last_became_peered":"2024-04-25T08:55:06.525489+0000","last_unstale":"2024-04-25T08:55:07.520259+0000","last_undegraded":"2024-04-25T08:55:07.520259+0000","last_fullsized":"2024-04-25T08:55:07.520259+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:52.528168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.530147+0000","last_change":"2024-04-25T08:55:07.526998+0000","last_active":"2024-04-25T08:55:07.530147+0000","last_peered":"2024-04-25T08:55:07.530147+0000","last_clean":"2024-04-25T08:55:07.530147+0000","last_became_active":"2024-04-25T08:55:07.526703+0000","last_became_peered":"2024-04-25T08:55:07.526703+0000","last_unstale":"2024-04-25T08:55:07.530147+0000","last_undegraded":"2024-04-25T08:55:07.530147+0000","last_fullsized":"2024-04-25T08:55:07.530147+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:36.576436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.511708+0000","last_change":"2024-04-25T08:55:06.529432+0000","last_active":"2024-04-25T08:55:07.511708+0000","last_peered":"2024-04-25T08:55:07.511708+0000","last_clean":"2024-04-25T08:55:07.511708+0000","last_became_active":"2024-04-25T08:55:06.529118+0000","last_became_peered":"2024-04-25T08:55:06.529118+0000","last_unstale":"2024-04-25T08:55:07.511708+0000","last_undegraded":"2024-04-25T08:55:07.511708+0000","last_fullsized":"2024-04-25T08:55:07.511708+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:35.979475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.541695+0000","last_change":"2024-04-25T08:55:07.533478+0000","last_active":"2024-04-25T08:55:07.541695+0000","last_peered":"2024-04-25T08:55:07.541695+0000","last_clean":"2024-04-25T08:55:07.541695+0000","last_became_active":"2024-04-25T08:55:07.533151+0000","last_became_peered":"2024-04-25T08:55:07.533151+0000","last_unstale":"2024-04-25T08:55:07.541695+0000","last_undegraded":"2024-04-25T08:55:07.541695+0000","last_fullsized":"2024-04-25T08:55:07.541695+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:14:13.129793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"32'8","reported_seq":20,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.542584+0000","last_change":"2024-04-25T08:55:06.522276+0000","last_active":"2024-04-25T08:55:07.542584+0000","last_peered":"2024-04-25T08:55:07.542584+0000","last_clean":"2024-04-25T08:55:07.542584+0000","last_became_active":"2024-04-25T08:55:06.521990+0000","last_became_peered":"2024-04-25T08:55:06.521990+0000","last_unstale":"2024-04-25T08:55:07.542584+0000","last_undegraded":"2024-04-25T08:55:07.542584+0000","last_fullsized":"2024-04-25T08:55:07.542584+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:35.574390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.515512+0000","last_change":"2024-04-25T08:55:07.534236+0000","last_active":"2024-04-25T08:55:08.515512+0000","last_peered":"2024-04-25T08:55:08.515512+0000","last_clean":"2024-04-25T08:55:08.515512+0000","last_became_active":"2024-04-25T08:55:07.532222+0000","last_became_peered":"2024-04-25T08:55:07.532222+0000","last_unstale":"2024-04-25T08:55:08.515512+0000","last_undegraded":"2024-04-25T08:55:08.515512+0000","last_fullsized":"2024-04-25T08:55:08.515512+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:22.632076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"32'1","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.543003+0000","last_change":"2024-04-25T08:55:06.523259+0000","last_active":"2024-04-25T08:55:07.543003+0000","last_peered":"2024-04-25T08:55:07.543003+0000","last_clean":"2024-04-25T08:55:07.543003+0000","last_became_active":"2024-04-25T08:55:06.522945+0000","last_became_peered":"2024-04-25T08:55:06.522945+0000","last_unstale":"2024-04-25T08:55:07.543003+0000","last_undegraded":"2024-04-25T08:55:07.543003+0000","last_fullsized":"2024-04-25T08:55:07.543003+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:14:12.127747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.516287+0000","last_change":"2024-04-25T08:55:07.536941+0000","last_active":"2024-04-25T08:55:08.516287+0000","last_peered":"2024-04-25T08:55:08.516287+0000","last_clean":"2024-04-25T08:55:08.516287+0000","last_became_active":"2024-04-25T08:55:07.536550+0000","last_became_peered":"2024-04-25T08:55:07.536550+0000","last_unstale":"2024-04-25T08:55:08.516287+0000","last_undegraded":"2024-04-25T08:55:08.516287+0000","last_fullsized":"2024-04-25T08:55:08.516287+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:27.043053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.515625+0000","last_change":"2024-04-25T08:55:06.523436+0000","last_active":"2024-04-25T08:55:08.515625+0000","last_peered":"2024-04-25T08:55:08.515625+0000","last_clean":"2024-04-25T08:55:08.515625+0000","last_became_active":"2024-04-25T08:55:06.523079+0000","last_became_peered":"2024-04-25T08:55:06.523079+0000","last_unstale":"2024-04-25T08:55:08.515625+0000","last_undegraded":"2024-04-25T08:55:08.515625+0000","last_fullsized":"2024-04-25T08:55:08.515625+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:25.799809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.536346+0000","last_change":"2024-04-25T08:55:07.532736+0000","last_active":"2024-04-25T08:55:07.536346+0000","last_peered":"2024-04-25T08:55:07.536346+0000","last_clean":"2024-04-25T08:55:07.536346+0000","last_became_active":"2024-04-25T08:55:07.532438+0000","last_became_peered":"2024-04-25T08:55:07.532438+0000","last_unstale":"2024-04-25T08:55:07.536346+0000","last_undegraded":"2024-04-25T08:55:07.536346+0000","last_fullsized":"2024-04-25T08:55:07.536346+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:09.261521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"32'1","reported_seq":15,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.516130+0000","last_change":"2024-04-25T08:55:06.524726+0000","last_active":"2024-04-25T08:55:08.516130+0000","last_peered":"2024-04-25T08:55:08.516130+0000","last_clean":"2024-04-25T08:55:08.516130+0000","last_became_active":"2024-04-25T08:55:06.524417+0000","last_became_peered":"2024-04-25T08:55:06.524417+0000","last_unstale":"2024-04-25T08:55:08.516130+0000","last_undegraded":"2024-04-25T08:55:08.516130+0000","last_fullsized":"2024-04-25T08:55:08.516130+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:23.342472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.531059+0000","last_change":"2024-04-25T08:55:07.527592+0000","last_active":"2024-04-25T08:55:07.531059+0000","last_peered":"2024-04-25T08:55:07.531059+0000","last_clean":"2024-04-25T08:55:07.531059+0000","last_became_active":"2024-04-25T08:55:07.527318+0000","last_became_peered":"2024-04-25T08:55:07.527318+0000","last_unstale":"2024-04-25T08:55:07.531059+0000","last_undegraded":"2024-04-25T08:55:07.531059+0000","last_fullsized":"2024-04-25T08:55:07.531059+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:26:24.344517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.516528+0000","last_change":"2024-04-25T08:55:06.524005+0000","last_active":"2024-04-25T08:55:08.516528+0000","last_peered":"2024-04-25T08:55:08.516528+0000","last_clean":"2024-04-25T08:55:08.516528+0000","last_became_active":"2024-04-25T08:55:06.522976+0000","last_became_peered":"2024-04-25T08:55:06.522976+0000","last_unstale":"2024-04-25T08:55:08.516528+0000","last_undegraded":"2024-04-25T08:55:08.516528+0000","last_fullsized":"2024-04-25T08:55:08.516528+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:31:08.772119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.527238+0000","last_change":"2024-04-25T08:55:07.524575+0000","last_active":"2024-04-25T08:55:07.527238+0000","last_peered":"2024-04-25T08:55:07.527238+0000","last_clean":"2024-04-25T08:55:07.527238+0000","last_became_active":"2024-04-25T08:55:07.524283+0000","last_became_peered":"2024-04-25T08:55:07.524283+0000","last_unstale":"2024-04-25T08:55:07.527238+0000","last_undegraded":"2024-04-25T08:55:07.527238+0000","last_fullsized":"2024-04-25T08:55:07.527238+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:11.974656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"32'2","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.511899+0000","last_change":"2024-04-25T08:55:06.522913+0000","last_active":"2024-04-25T08:55:08.511899+0000","last_peered":"2024-04-25T08:55:08.511899+0000","last_clean":"2024-04-25T08:55:08.511899+0000","last_became_active":"2024-04-25T08:55:06.520103+0000","last_became_peered":"2024-04-25T08:55:06.520103+0000","last_unstale":"2024-04-25T08:55:08.511899+0000","last_undegraded":"2024-04-25T08:55:08.511899+0000","last_fullsized":"2024-04-25T08:55:08.511899+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:10.907040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.535417+0000","last_change":"2024-04-25T08:55:07.531740+0000","last_active":"2024-04-25T08:55:07.535417+0000","last_peered":"2024-04-25T08:55:07.535417+0000","last_clean":"2024-04-25T08:55:07.535417+0000","last_became_active":"2024-04-25T08:55:07.531475+0000","last_became_peered":"2024-04-25T08:55:07.531475+0000","last_unstale":"2024-04-25T08:55:07.535417+0000","last_undegraded":"2024-04-25T08:55:07.535417+0000","last_fullsized":"2024-04-25T08:55:07.535417+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:26.801855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.522734+0000","last_change":"2024-04-25T08:55:06.528210+0000","last_active":"2024-04-25T08:55:07.522734+0000","last_peered":"2024-04-25T08:55:07.522734+0000","last_clean":"2024-04-25T08:55:07.522734+0000","last_became_active":"2024-04-25T08:55:06.527957+0000","last_became_peered":"2024-04-25T08:55:06.527957+0000","last_unstale":"2024-04-25T08:55:07.522734+0000","last_undegraded":"2024-04-25T08:55:07.522734+0000","last_fullsized":"2024-04-25T08:55:07.522734+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:19.451377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.531069+0000","last_change":"2024-04-25T08:55:07.527730+0000","last_active":"2024-04-25T08:55:07.531069+0000","last_peered":"2024-04-25T08:55:07.531069+0000","last_clean":"2024-04-25T08:55:07.531069+0000","last_became_active":"2024-04-25T08:55:07.527457+0000","last_became_peered":"2024-04-25T08:55:07.527457+0000","last_unstale":"2024-04-25T08:55:07.531069+0000","last_undegraded":"2024-04-25T08:55:07.531069+0000","last_fullsized":"2024-04-25T08:55:07.531069+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:40:44.140070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.515646+0000","last_change":"2024-04-25T08:55:06.527054+0000","last_active":"2024-04-25T08:55:08.515646+0000","last_peered":"2024-04-25T08:55:08.515646+0000","last_clean":"2024-04-25T08:55:08.515646+0000","last_became_active":"2024-04-25T08:55:06.526762+0000","last_became_peered":"2024-04-25T08:55:06.526762+0000","last_unstale":"2024-04-25T08:55:08.515646+0000","last_undegraded":"2024-04-25T08:55:08.515646+0000","last_fullsized":"2024-04-25T08:55:08.515646+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:59.799720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.536974+0000","last_change":"2024-04-25T08:55:07.530782+0000","last_active":"2024-04-25T08:55:07.536974+0000","last_peered":"2024-04-25T08:55:07.536974+0000","last_clean":"2024-04-25T08:55:07.536974+0000","last_became_active":"2024-04-25T08:55:07.530527+0000","last_became_peered":"2024-04-25T08:55:07.530527+0000","last_unstale":"2024-04-25T08:55:07.536974+0000","last_undegraded":"2024-04-25T08:55:07.536974+0000","last_fullsized":"2024-04-25T08:55:07.536974+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:29.002070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.515024+0000","last_change":"2024-04-25T08:55:06.525945+0000","last_active":"2024-04-25T08:55:08.515024+0000","last_peered":"2024-04-25T08:55:08.515024+0000","last_clean":"2024-04-25T08:55:08.515024+0000","last_became_active":"2024-04-25T08:55:06.525643+0000","last_became_peered":"2024-04-25T08:55:06.525643+0000","last_unstale":"2024-04-25T08:55:08.515024+0000","last_undegraded":"2024-04-25T08:55:08.515024+0000","last_fullsized":"2024-04-25T08:55:08.515024+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:35.979475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.535702+0000","last_change":"2024-04-25T08:55:07.533596+0000","last_active":"2024-04-25T08:55:07.535702+0000","last_peered":"2024-04-25T08:55:07.535702+0000","last_clean":"2024-04-25T08:55:07.535702+0000","last_became_active":"2024-04-25T08:55:07.533325+0000","last_became_peered":"2024-04-25T08:55:07.533325+0000","last_unstale":"2024-04-25T08:55:07.535702+0000","last_undegraded":"2024-04-25T08:55:07.535702+0000","last_fullsized":"2024-04-25T08:55:07.535702+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:00.801766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"32'2","reported_seq":14,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.539070+0000","last_change":"2024-04-25T08:55:06.522733+0000","last_active":"2024-04-25T08:55:07.539070+0000","last_peered":"2024-04-25T08:55:07.539070+0000","last_clean":"2024-04-25T08:55:07.539070+0000","last_became_active":"2024-04-25T08:55:06.522285+0000","last_became_peered":"2024-04-25T08:55:06.522285+0000","last_unstale":"2024-04-25T08:55:07.539070+0000","last_undegraded":"2024-04-25T08:55:07.539070+0000","last_fullsized":"2024-04-25T08:55:07.539070+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:45:35.574390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.516272+0000","last_change":"2024-04-25T08:55:07.534209+0000","last_active":"2024-04-25T08:55:08.516272+0000","last_peered":"2024-04-25T08:55:08.516272+0000","last_clean":"2024-04-25T08:55:08.516272+0000","last_became_active":"2024-04-25T08:55:07.533152+0000","last_became_peered":"2024-04-25T08:55:07.533152+0000","last_unstale":"2024-04-25T08:55:08.516272+0000","last_undegraded":"2024-04-25T08:55:08.516272+0000","last_fullsized":"2024-04-25T08:55:08.516272+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:43.220568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.511874+0000","last_change":"2024-04-25T08:55:06.519043+0000","last_active":"2024-04-25T08:55:08.511874+0000","last_peered":"2024-04-25T08:55:08.511874+0000","last_clean":"2024-04-25T08:55:08.511874+0000","last_became_active":"2024-04-25T08:55:06.518727+0000","last_became_peered":"2024-04-25T08:55:06.518727+0000","last_unstale":"2024-04-25T08:55:08.511874+0000","last_undegraded":"2024-04-25T08:55:08.511874+0000","last_fullsized":"2024-04-25T08:55:08.511874+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:06.263241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.511827+0000","last_change":"2024-04-25T08:55:07.530829+0000","last_active":"2024-04-25T08:55:08.511827+0000","last_peered":"2024-04-25T08:55:08.511827+0000","last_clean":"2024-04-25T08:55:08.511827+0000","last_became_active":"2024-04-25T08:55:07.530565+0000","last_became_peered":"2024-04-25T08:55:07.530565+0000","last_unstale":"2024-04-25T08:55:08.511827+0000","last_undegraded":"2024-04-25T08:55:08.511827+0000","last_fullsized":"2024-04-25T08:55:08.511827+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:26.245579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.520886+0000","last_change":"2024-04-25T08:55:06.524167+0000","last_active":"2024-04-25T08:55:07.520886+0000","last_peered":"2024-04-25T08:55:07.520886+0000","last_clean":"2024-04-25T08:55:07.520886+0000","last_became_active":"2024-04-25T08:55:06.523879+0000","last_became_peered":"2024-04-25T08:55:06.523879+0000","last_unstale":"2024-04-25T08:55:07.520886+0000","last_undegraded":"2024-04-25T08:55:07.520886+0000","last_fullsized":"2024-04-25T08:55:07.520886+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:21.630030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.536182+0000","last_change":"2024-04-25T08:55:07.530975+0000","last_active":"2024-04-25T08:55:07.536182+0000","last_peered":"2024-04-25T08:55:07.536182+0000","last_clean":"2024-04-25T08:55:07.536182+0000","last_became_active":"2024-04-25T08:55:07.530233+0000","last_became_peered":"2024-04-25T08:55:07.530233+0000","last_unstale":"2024-04-25T08:55:07.536182+0000","last_undegraded":"2024-04-25T08:55:07.536182+0000","last_fullsized":"2024-04-25T08:55:07.536182+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:17:11.909086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.511994+0000","last_change":"2024-04-25T08:55:06.524320+0000","last_active":"2024-04-25T08:55:08.511994+0000","last_peered":"2024-04-25T08:55:08.511994+0000","last_clean":"2024-04-25T08:55:08.511994+0000","last_became_active":"2024-04-25T08:55:06.523284+0000","last_became_peered":"2024-04-25T08:55:06.523284+0000","last_unstale":"2024-04-25T08:55:08.511994+0000","last_undegraded":"2024-04-25T08:55:08.511994+0000","last_fullsized":"2024-04-25T08:55:08.511994+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:43:52.528168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.511814+0000","last_change":"2024-04-25T08:55:07.533703+0000","last_active":"2024-04-25T08:55:08.511814+0000","last_peered":"2024-04-25T08:55:08.511814+0000","last_clean":"2024-04-25T08:55:08.511814+0000","last_became_active":"2024-04-25T08:55:07.533445+0000","last_became_peered":"2024-04-25T08:55:07.533445+0000","last_unstale":"2024-04-25T08:55:08.511814+0000","last_undegraded":"2024-04-25T08:55:08.511814+0000","last_fullsized":"2024-04-25T08:55:08.511814+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:29.002070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.520411+0000","last_change":"2024-04-25T08:55:06.521544+0000","last_active":"2024-04-25T08:55:07.520411+0000","last_peered":"2024-04-25T08:55:07.520411+0000","last_clean":"2024-04-25T08:55:07.520411+0000","last_became_active":"2024-04-25T08:55:06.521266+0000","last_became_peered":"2024-04-25T08:55:06.521266+0000","last_unstale":"2024-04-25T08:55:07.520411+0000","last_undegraded":"2024-04-25T08:55:07.520411+0000","last_fullsized":"2024-04-25T08:55:07.520411+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:35.979475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.541167+0000","last_change":"2024-04-25T08:55:07.530484+0000","last_active":"2024-04-25T08:55:07.541167+0000","last_peered":"2024-04-25T08:55:07.541167+0000","last_clean":"2024-04-25T08:55:07.541167+0000","last_became_active":"2024-04-25T08:55:07.530195+0000","last_became_peered":"2024-04-25T08:55:07.530195+0000","last_unstale":"2024-04-25T08:55:07.541167+0000","last_undegraded":"2024-04-25T08:55:07.541167+0000","last_fullsized":"2024-04-25T08:55:07.541167+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:01.806799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.515754+0000","last_change":"2024-04-25T08:55:06.523594+0000","last_active":"2024-04-25T08:55:08.515754+0000","last_peered":"2024-04-25T08:55:08.515754+0000","last_clean":"2024-04-25T08:55:08.515754+0000","last_became_active":"2024-04-25T08:55:06.523317+0000","last_became_peered":"2024-04-25T08:55:06.523317+0000","last_unstale":"2024-04-25T08:55:08.515754+0000","last_undegraded":"2024-04-25T08:55:08.515754+0000","last_fullsized":"2024-04-25T08:55:08.515754+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:03.203548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-04-25T08:55:08.515547+0000","last_change":"2024-04-25T08:55:07.523690+0000","last_active":"2024-04-25T08:55:08.515547+0000","last_peered":"2024-04-25T08:55:08.515547+0000","last_clean":"2024-04-25T08:55:08.515547+0000","last_became_active":"2024-04-25T08:55:07.523370+0000","last_became_peered":"2024-04-25T08:55:07.523370+0000","last_unstale":"2024-04-25T08:55:08.515547+0000","last_undegraded":"2024-04-25T08:55:08.515547+0000","last_fullsized":"2024-04-25T08:55:08.515547+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:32.166165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.512062+0000","last_change":"2024-04-25T08:55:06.527555+0000","last_active":"2024-04-25T08:55:07.512062+0000","last_peered":"2024-04-25T08:55:07.512062+0000","last_clean":"2024-04-25T08:55:07.512062+0000","last_became_active":"2024-04-25T08:55:06.527229+0000","last_became_peered":"2024-04-25T08:55:06.527229+0000","last_unstale":"2024-04-25T08:55:07.512062+0000","last_undegraded":"2024-04-25T08:55:07.512062+0000","last_fullsized":"2024-04-25T08:55:07.512062+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:21.630030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.530589+0000","last_change":"2024-04-25T08:55:07.529229+0000","last_active":"2024-04-25T08:55:07.530589+0000","last_peered":"2024-04-25T08:55:07.530589+0000","last_clean":"2024-04-25T08:55:07.530589+0000","last_became_active":"2024-04-25T08:55:07.528886+0000","last_became_peered":"2024-04-25T08:55:07.528886+0000","last_unstale":"2024-04-25T08:55:07.530589+0000","last_undegraded":"2024-04-25T08:55:07.530589+0000","last_fullsized":"2024-04-25T08:55:07.530589+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:06.502256+0000","last_clean_scrub_stamp":"2024-04-25T08:55:06.502256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:32.166165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.521546+0000","last_change":"2024-04-25T08:55:06.527498+0000","last_active":"2024-04-25T08:55:07.521546+0000","last_peered":"2024-04-25T08:55:07.521546+0000","last_clean":"2024-04-25T08:55:07.521546+0000","last_became_active":"2024-04-25T08:55:06.527168+0000","last_became_peered":"2024-04-25T08:55:06.527168+0000","last_unstale":"2024-04-25T08:55:07.521546+0000","last_undegraded":"2024-04-25T08:55:07.521546+0000","last_fullsized":"2024-04-25T08:55:07.521546+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:55:05.500210+0000","last_clean_scrub_stamp":"2024-04-25T08:55:05.500210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:59.329089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":192,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-25T08:55:07.518362+0000","last_change":"2024-04-25T08:54:00.818044+0000","last_active":"2024-04-25T08:55:07.518362+0000","last_peered":"2024-04-25T08:55:07.518362+0000","last_clean":"2024-04-25T08:55:07.518362+0000","last_became_active":"2024-04-25T08:54:00.817098+0000","last_became_peered":"2024-04-25T08:54:00.817098+0000","last_unstale":"2024-04-25T08:55:07.518362+0000","last_undegraded":"2024-04-25T08:55:07.518362+0000","last_fullsized":"2024-04-25T08:55:07.518362+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_clean_scrub_stamp":"2024-04-25T08:53:59.739677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:48.129513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27960,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343880,"statfs":{"total":96636764160,"available":96608133120,"internally_reserved":0,"allocated":958464,"data_stored":730936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":141094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":141150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":368640,"data_stored":140582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31552,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":376832,"data_stored":142112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31540,"kb_used_data":356,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340300,"statfs":{"total":96636764160,"available":96604467200,"internally_reserved":0,"allocated":364544,"data_stored":140548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31552,"kb_used_data":368,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":376832,"data_stored":142112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32132,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339708,"statfs":{"total":96636764160,"available":96603860992,"internally_reserved":0,"allocated":966656,"data_stored":737146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:55:10.858 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T08:55:10.859 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T08:55:11.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.012+0000 7f16ada38640 1 Processor -- start 2024-04-25T08:55:11.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.012+0000 7f16ada38640 1 -- start start 2024-04-25T08:55:11.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.012+0000 7f16ada38640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f16a815a6b0 0x7f16a80cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:11.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.012+0000 7f16ada38640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f16a806dc00 0x7f16a80b35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:11.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.012+0000 7f16ada38640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16a815e3c0 con 0x7f16a80ae9e0 2024-04-25T08:55:11.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.012+0000 7f16ada38640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16a815e680 con 0x7f16a806dc00 2024-04-25T08:55:11.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.012+0000 7f16ada38640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a815e960 con 0x7f16a815a6b0 2024-04-25T08:55:11.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.013+0000 7f16a77fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f16a806dc00 0x7f16a80b35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:11.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.013+0000 7f169ffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f16a80ae9e0 0x7f16a806d4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32978/0 (socket says 172.21.15.43:32978) 2024-04-25T08:55:11.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.013+0000 7f169ffff640 1 -- 172.21.15.43:0/4192604196 learned_addr learned my addr 172.21.15.43:0/4192604196 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:55:11.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.013+0000 7f16a6ffd640 1 --2- 172.21.15.43:0/4192604196 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f16a815a6b0 0x7f16a80cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:11.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.013+0000 7f16a77fe640 1 -- 172.21.15.43:0/4192604196 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f16a815a6b0 msgr2=0x7f16a80cc6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:11.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.013+0000 7f16a77fe640 1 --2- 172.21.15.43:0/4192604196 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f16a815a6b0 0x7f16a80cc6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:11.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.013+0000 7f16a77fe640 1 -- 172.21.15.43:0/4192604196 >> v1:172.21.15.43:6789/0 conn(0x7f16a80ae9e0 legacy=0x7f16a806d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:11.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.013+0000 7f16a77fe640 1 -- 172.21.15.43:0/4192604196 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a815ec40 con 0x7f16a806dc00 2024-04-25T08:55:11.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.013+0000 7f16a67fc640 1 -- 172.21.15.43:0/4192604196 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 231200344 0 0) 0x7f16a815e3c0 con 0x7f16a80ae9e0 2024-04-25T08:55:11.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.013+0000 7f16a6ffd640 1 --2- 172.21.15.43:0/4192604196 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f16a815a6b0 0x7f16a80cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:55:11.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.014+0000 7f16a77fe640 1 --2- 172.21.15.43:0/4192604196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16a806dc00 0x7f16a80b35c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f16a0009d90 tx=0x7f16a002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97cde4f0bdade7ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:11.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.014+0000 7f16a67fc640 1 -- 172.21.15.43:0/4192604196 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a0007740 con 0x7f16a806dc00 2024-04-25T08:55:11.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.014+0000 7f16a67fc640 1 -- 172.21.15.43:0/4192604196 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16a00078e0 con 0x7f16a806dc00 2024-04-25T08:55:11.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.014+0000 7f16a67fc640 1 -- 172.21.15.43:0/4192604196 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a0007cb0 con 0x7f16a806dc00 2024-04-25T08:55:11.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.014+0000 7f16ada38640 1 -- 172.21.15.43:0/4192604196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16a806dc00 msgr2=0x7f16a80b35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:11.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.014+0000 7f16ada38640 1 --2- 172.21.15.43:0/4192604196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16a806dc00 0x7f16a80b35c0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f16a0009d90 tx=0x7f16a002a660 comp rx=0 tx=0).stop 2024-04-25T08:55:11.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.014+0000 7f16ada38640 1 -- 172.21.15.43:0/4192604196 shutdown_connections 2024-04-25T08:55:11.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.014+0000 7f16ada38640 1 --2- 172.21.15.43:0/4192604196 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16a806dc00 0x7f16a80b35c0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:11.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.014+0000 7f16ada38640 1 --2- 172.21.15.43:0/4192604196 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f16a815a6b0 0x7f16a80cc6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:11.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.014+0000 7f16ada38640 1 -- 172.21.15.43:0/4192604196 >> 172.21.15.43:0/4192604196 conn(0x7f16a806ab30 msgr2=0x7f16a806af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:11.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.014+0000 7f16ada38640 1 -- 172.21.15.43:0/4192604196 shutdown_connections 2024-04-25T08:55:11.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.015+0000 7f16ada38640 1 -- 172.21.15.43:0/4192604196 wait complete. 2024-04-25T08:55:11.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.016+0000 7f16ada38640 1 Processor -- start 2024-04-25T08:55:11.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.016+0000 7f16ada38640 1 -- start start 2024-04-25T08:55:11.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.016+0000 7f16ada38640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16a806dc00 0x7f16a80adb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:11.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.016+0000 7f16ada38640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f16a80ae9e0 0x7f16a8167050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:11.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.016+0000 7f16ada38640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16a815a6b0 0x7f16a81525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:11.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.016+0000 7f16ada38640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a815aa90 con 0x7f16a806dc00 2024-04-25T08:55:11.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.016+0000 7f16ada38640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a815f200 con 0x7f16a815a6b0 2024-04-25T08:55:11.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.016+0000 7f16ada38640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a815ac10 con 0x7f16a80ae9e0 2024-04-25T08:55:11.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.016+0000 7f16a6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16a806dc00 0x7f16a80adb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:11.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.016+0000 7f16a6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16a806dc00 0x7f16a80adb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38298/0 (socket says 172.21.15.43:38298) 2024-04-25T08:55:11.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.016+0000 7f16a6ffd640 1 -- 172.21.15.43:0/713838502 learned_addr learned my addr 172.21.15.43:0/713838502 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:11.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.017+0000 7f169ffff640 1 --2- 172.21.15.43:0/713838502 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f16a80ae9e0 0x7f16a8167050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:11.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.017+0000 7f16a77fe640 1 --2- 172.21.15.43:0/713838502 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16a815a6b0 0x7f16a81525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:11.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.017+0000 7f16a6ffd640 1 -- 172.21.15.43:0/713838502 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f16a80ae9e0 msgr2=0x7f16a8167050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:11.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.017+0000 7f16a6ffd640 1 --2- 172.21.15.43:0/713838502 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f16a80ae9e0 0x7f16a8167050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:11.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.017+0000 7f16a6ffd640 1 -- 172.21.15.43:0/713838502 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16a815a6b0 msgr2=0x7f16a81525e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:11.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.017+0000 7f16a6ffd640 1 --2- 172.21.15.43:0/713838502 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16a815a6b0 0x7f16a81525e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:11.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.017+0000 7f16a6ffd640 1 -- 172.21.15.43:0/713838502 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a80cd520 con 0x7f16a806dc00 2024-04-25T08:55:11.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.017+0000 7f16a77fe640 1 --2- 172.21.15.43:0/713838502 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16a815a6b0 0x7f16a81525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:55:11.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.017+0000 7f16a6ffd640 1 --2- 172.21.15.43:0/713838502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16a806dc00 0x7f16a80adb50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f169800e760 tx=0x7f169800ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:11.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.017+0000 7f16a4ff9640 1 -- 172.21.15.43:0/713838502 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169800f040 con 0x7f16a806dc00 2024-04-25T08:55:11.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.017+0000 7f16a4ff9640 1 -- 172.21.15.43:0/713838502 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1698004230 con 0x7f16a806dc00 2024-04-25T08:55:11.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.017+0000 7f16a4ff9640 1 -- 172.21.15.43:0/713838502 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169800ee70 con 0x7f16a806dc00 2024-04-25T08:55:11.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.017+0000 7f16ada38640 1 -- 172.21.15.43:0/713838502 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16a8165be0 con 0x7f16a806dc00 2024-04-25T08:55:11.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.017+0000 7f16ada38640 1 -- 172.21.15.43:0/713838502 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16a8166480 con 0x7f16a806dc00 2024-04-25T08:55:11.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.018+0000 7f16ada38640 1 -- 172.21.15.43:0/713838502 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16a8165fd0 con 0x7f16a806dc00 2024-04-25T08:55:11.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.019+0000 7f16a4ff9640 1 -- 172.21.15.43:0/713838502 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f169800c6d0 con 0x7f16a806dc00 2024-04-25T08:55:11.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.019+0000 7f16a4ff9640 1 --2- 172.21.15.43:0/713838502 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f168806b4d0 0x7f168806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:11.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.019+0000 7f16a4ff9640 1 -- 172.21.15.43:0/713838502 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1698089e30 con 0x7f16a806dc00 2024-04-25T08:55:11.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.020+0000 7f169ffff640 1 --2- 172.21.15.43:0/713838502 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f168806b4d0 0x7f168806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:11.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.020+0000 7f169ffff640 1 --2- 172.21.15.43:0/713838502 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f168806b4d0 0x7f168806d990 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f16a815c110 tx=0x7f1690007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:11.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.023+0000 7f16a4ff9640 1 -- 172.21.15.43:0/713838502 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1698017020 con 0x7f16a806dc00 2024-04-25T08:55:11.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.272+0000 7f16ada38640 1 -- 172.21.15.43:0/713838502 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f16a8071e20 con 0x7f16a806dc00 2024-04-25T08:55:11.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.274+0000 7f16a4ff9640 1 -- 172.21.15.43:0/713838502 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+147 (secure 0 0 0) 0x7f16a8071e20 con 0x7f16a806dc00 2024-04-25T08:55:11.264 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2024-04-25T08:55:11.264 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQD8GSpmVOjWGxAAFNmxeGi0h2D//pa6Ne3eQw== 2024-04-25T08:55:11.264 INFO:teuthology.orchestra.run.smithi043.stdout: caps mds = "allow" 2024-04-25T08:55:11.265 INFO:teuthology.orchestra.run.smithi043.stdout: caps mon = "allow r" 2024-04-25T08:55:11.265 INFO:teuthology.orchestra.run.smithi043.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T08:55:11.265 INFO:teuthology.orchestra.run.smithi043.stderr:updated caps for client.0 2024-04-25T08:55:11.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.276+0000 7f16ada38640 1 -- 172.21.15.43:0/713838502 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f168806b4d0 msgr2=0x7f168806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:11.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.276+0000 7f16ada38640 1 --2- 172.21.15.43:0/713838502 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f168806b4d0 0x7f168806d990 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f16a815c110 tx=0x7f1690007de0 comp rx=0 tx=0).stop 2024-04-25T08:55:11.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.276+0000 7f16ada38640 1 -- 172.21.15.43:0/713838502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16a806dc00 msgr2=0x7f16a80adb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:11.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.276+0000 7f16ada38640 1 --2- 172.21.15.43:0/713838502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16a806dc00 0x7f16a80adb50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f169800e760 tx=0x7f169800ec20 comp rx=0 tx=0).stop 2024-04-25T08:55:11.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.276+0000 7f16ada38640 1 -- 172.21.15.43:0/713838502 shutdown_connections 2024-04-25T08:55:11.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.276+0000 7f16ada38640 1 --2- 172.21.15.43:0/713838502 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f168806b4d0 0x7f168806d990 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:11.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.276+0000 7f16ada38640 1 --2- 172.21.15.43:0/713838502 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16a815a6b0 0x7f16a81525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:11.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.276+0000 7f16ada38640 1 --2- 172.21.15.43:0/713838502 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f16a80ae9e0 0x7f16a8167050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:11.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.276+0000 7f16ada38640 1 --2- 172.21.15.43:0/713838502 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f16a806dc00 0x7f16a80adb50 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:11.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.276+0000 7f16ada38640 1 -- 172.21.15.43:0/713838502 >> 172.21.15.43:0/713838502 conn(0x7f16a806ab30 msgr2=0x7f16a80b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:11.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.276+0000 7f16ada38640 1 -- 172.21.15.43:0/713838502 shutdown_connections 2024-04-25T08:55:11.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.276+0000 7f16ada38640 1 -- 172.21.15.43:0/713838502 wait complete. 2024-04-25T08:55:11.277 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:55:11.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.388+0000 7f4db923a640 1 Processor -- start 2024-04-25T08:55:11.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.388+0000 7f4db923a640 1 -- start start 2024-04-25T08:55:11.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.389+0000 7f4db923a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4db4161390 0x7f4db40b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:11.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.389+0000 7f4db923a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4db40b4c30 con 0x7f4db40cecf0 2024-04-25T08:55:11.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.389+0000 7f4db923a640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4db40cf2c0 con 0x7f4db40d02a0 2024-04-25T08:55:11.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.389+0000 7f4db923a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4db40d1910 con 0x7f4db4161390 2024-04-25T08:55:11.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.389+0000 7f4db359e640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4db40cecf0 0x7f4db40d1da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32980/0 (socket says 172.21.15.43:32980) 2024-04-25T08:55:11.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.389+0000 7f4db359e640 1 -- 172.21.15.43:0/3029940477 learned_addr learned my addr 172.21.15.43:0/3029940477 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:55:11.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.389+0000 7f4db2d9d640 1 --2- 172.21.15.43:0/3029940477 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4db4161390 0x7f4db40b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:11.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.389+0000 7f4db259c640 1 -- 172.21.15.43:0/3029940477 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3951377004 0 0) 0x7f4dac001490 con 0x7f4db40cecf0 2024-04-25T08:55:11.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.389+0000 7f4db259c640 1 -- 172.21.15.43:0/3029940477 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4da0003520 con 0x7f4db40cecf0 2024-04-25T08:55:11.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.389+0000 7f4db259c640 1 -- 172.21.15.43:0/3029940477 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 876246217 0 0) 0x7f4db40cf2c0 con 0x7f4db40d02a0 2024-04-25T08:55:11.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.389+0000 7f4db259c640 1 -- 172.21.15.43:0/3029940477 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4dac001490 con 0x7f4db40d02a0 2024-04-25T08:55:11.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.389+0000 7f4db259c640 1 -- 172.21.15.43:0/3029940477 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1783562216 0 0) 0x7f4da0003520 con 0x7f4db40cecf0 2024-04-25T08:55:11.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.389+0000 7f4db259c640 1 -- 172.21.15.43:0/3029940477 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4db4161390 msgr2=0x7f4db40b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:11.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.390+0000 7f4db259c640 1 --2- 172.21.15.43:0/3029940477 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4db4161390 0x7f4db40b6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:11.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.390+0000 7f4db259c640 1 -- 172.21.15.43:0/3029940477 >> v1:172.21.15.193:6789/0 conn(0x7f4db40d02a0 legacy=0x7f4db40ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:11.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.390+0000 7f4db259c640 1 -- 172.21.15.43:0/3029940477 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db40cf5a0 con 0x7f4db40cecf0 2024-04-25T08:55:11.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.390+0000 7f4db259c640 1 -- 172.21.15.43:0/3029940477 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f4dac001bc0 con 0x7f4db40cecf0 2024-04-25T08:55:11.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.390+0000 7f4db2d9d640 1 --2- 172.21.15.43:0/3029940477 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4db4161390 0x7f4db40b6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:55:11.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.390+0000 7f4db259c640 1 -- 172.21.15.43:0/3029940477 >> v1:172.21.15.43:6789/0 conn(0x7f4db40cecf0 legacy=0x7f4db40d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:11.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.390+0000 7f4db259c640 1 --2- 172.21.15.43:0/3029940477 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4db4161390 0x7f4da0003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:11.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.390+0000 7f4db259c640 1 --2- 172.21.15.43:0/3029940477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4da0004100 0x7f4da0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:11.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.390+0000 7f4db259c640 1 --2- 172.21.15.43:0/3029940477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4da0004d40 0x7f4da00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:11.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.390+0000 7f4db259c640 1 -- 172.21.15.43:0/3029940477 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db40d1910 con 0x7f4da0004d40 2024-04-25T08:55:11.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.390+0000 7f4db259c640 1 -- 172.21.15.43:0/3029940477 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da000bad0 con 0x7f4da0004100 2024-04-25T08:55:11.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.390+0000 7f4db259c640 1 -- 172.21.15.43:0/3029940477 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4da000bcd0 con 0x7f4db4161390 2024-04-25T08:55:11.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.390+0000 7f4db359e640 1 --2- 172.21.15.43:0/3029940477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4da0004d40 0x7f4da00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:11.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.390+0000 7f4db2d9d640 1 --2- 172.21.15.43:0/3029940477 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4db4161390 0x7f4da0003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:11.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.390+0000 7f4dabfff640 1 --2- 172.21.15.43:0/3029940477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4da0004100 0x7f4da0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:11.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.390+0000 7f4db359e640 1 -- 172.21.15.43:0/3029940477 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4db4161390 msgr2=0x7f4da0003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:11.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.390+0000 7f4db359e640 1 --2- 172.21.15.43:0/3029940477 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4db4161390 0x7f4da0003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:11.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.390+0000 7f4db359e640 1 -- 172.21.15.43:0/3029940477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4da0004100 msgr2=0x7f4da0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:11.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.390+0000 7f4db359e640 1 --2- 172.21.15.43:0/3029940477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4da0004100 0x7f4da0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:11.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.390+0000 7f4db359e640 1 -- 172.21.15.43:0/3029940477 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da0003520 con 0x7f4da0004d40 2024-04-25T08:55:11.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.391+0000 7f4db359e640 1 --2- 172.21.15.43:0/3029940477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4da0004d40 0x7f4da00051c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4dac00a8d0 tx=0x7f4da0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f96d3f4b04dc2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:11.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.391+0000 7f4db259c640 1 -- 172.21.15.43:0/3029940477 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dac00a520 con 0x7f4da0004d40 2024-04-25T08:55:11.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.391+0000 7f4db259c640 1 -- 172.21.15.43:0/3029940477 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dac02f070 con 0x7f4da0004d40 2024-04-25T08:55:11.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.391+0000 7f4db259c640 1 -- 172.21.15.43:0/3029940477 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dac02bd50 con 0x7f4da0004d40 2024-04-25T08:55:11.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.392+0000 7f4db923a640 1 -- 172.21.15.43:0/3029940477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4da0004d40 msgr2=0x7f4da00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:11.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.392+0000 7f4db923a640 1 --2- 172.21.15.43:0/3029940477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4da0004d40 0x7f4da00051c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f4dac00a8d0 tx=0x7f4da0005830 comp rx=0 tx=0).stop 2024-04-25T08:55:11.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.392+0000 7f4db359e640 1 -- 172.21.15.43:0/3029940477 reap_dead start 2024-04-25T08:55:11.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.392+0000 7f4db923a640 1 -- 172.21.15.43:0/3029940477 shutdown_connections 2024-04-25T08:55:11.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.392+0000 7f4db923a640 1 -- 172.21.15.43:0/3029940477 >> 172.21.15.43:0/3029940477 conn(0x7f4db409cd20 msgr2=0x7f4db40d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:11.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.392+0000 7f4db923a640 1 -- 172.21.15.43:0/3029940477 shutdown_connections 2024-04-25T08:55:11.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.392+0000 7f4db923a640 1 -- 172.21.15.43:0/3029940477 wait complete. 2024-04-25T08:55:11.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.393+0000 7f4db923a640 1 Processor -- start 2024-04-25T08:55:11.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.393+0000 7f4db923a640 1 -- start start 2024-04-25T08:55:11.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.394+0000 7f4db923a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4da0004d40 0x7f4db415a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:11.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.394+0000 7f4db923a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4db415d480 0x7f4db415c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:11.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.394+0000 7f4db923a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4db415c6d0 0x7f4db415cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:11.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.394+0000 7f4db923a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db4161770 con 0x7f4da0004d40 2024-04-25T08:55:11.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.394+0000 7f4db923a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db40d15a0 con 0x7f4db415c6d0 2024-04-25T08:55:11.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.394+0000 7f4db923a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4db40b6820 con 0x7f4db415d480 2024-04-25T08:55:11.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.394+0000 7f4db2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4da0004d40 0x7f4db415a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:11.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.394+0000 7f4db2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4da0004d40 0x7f4db415a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38312/0 (socket says 172.21.15.43:38312) 2024-04-25T08:55:11.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.394+0000 7f4db2d9d640 1 -- 172.21.15.43:0/1668343872 learned_addr learned my addr 172.21.15.43:0/1668343872 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:11.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.394+0000 7f4dabfff640 1 --2- 172.21.15.43:0/1668343872 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4db415d480 0x7f4db415c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:11.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.394+0000 7f4db359e640 1 --2- 172.21.15.43:0/1668343872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4db415c6d0 0x7f4db415cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:11.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.394+0000 7f4db2d9d640 1 -- 172.21.15.43:0/1668343872 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4db415d480 msgr2=0x7f4db415c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:11.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.394+0000 7f4db2d9d640 1 --2- 172.21.15.43:0/1668343872 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4db415d480 0x7f4db415c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:11.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.394+0000 7f4db2d9d640 1 -- 172.21.15.43:0/1668343872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4db415c6d0 msgr2=0x7f4db415cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:11.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.394+0000 7f4db2d9d640 1 --2- 172.21.15.43:0/1668343872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4db415c6d0 0x7f4db415cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:11.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.394+0000 7f4db2d9d640 1 -- 172.21.15.43:0/1668343872 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db40cc6d0 con 0x7f4da0004d40 2024-04-25T08:55:11.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.394+0000 7f4db2d9d640 1 --2- 172.21.15.43:0/1668343872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4da0004d40 0x7f4db415a4b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4da4011a00 tx=0x7f4da4011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:11.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.395+0000 7f4dab7fe640 1 -- 172.21.15.43:0/1668343872 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da400faf0 con 0x7f4da0004d40 2024-04-25T08:55:11.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.395+0000 7f4dab7fe640 1 -- 172.21.15.43:0/1668343872 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4da400fc90 con 0x7f4da0004d40 2024-04-25T08:55:11.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.395+0000 7f4dab7fe640 1 -- 172.21.15.43:0/1668343872 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da4014440 con 0x7f4da0004d40 2024-04-25T08:55:11.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.395+0000 7f4db923a640 1 -- 172.21.15.43:0/1668343872 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4db40cb670 con 0x7f4da0004d40 2024-04-25T08:55:11.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.395+0000 7f4db923a640 1 -- 172.21.15.43:0/1668343872 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4db40cbbf0 con 0x7f4da0004d40 2024-04-25T08:55:11.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.396+0000 7f4db923a640 1 -- 172.21.15.43:0/1668343872 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4db40680f0 con 0x7f4da0004d40 2024-04-25T08:55:11.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.396+0000 7f4dab7fe640 1 -- 172.21.15.43:0/1668343872 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4da4012040 con 0x7f4da0004d40 2024-04-25T08:55:11.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.397+0000 7f4dab7fe640 1 --2- 172.21.15.43:0/1668343872 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4d9406b500 0x7f4d9406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:11.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.397+0000 7f4dab7fe640 1 -- 172.21.15.43:0/1668343872 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4da408a820 con 0x7f4da0004d40 2024-04-25T08:55:11.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.397+0000 7f4dabfff640 1 --2- 172.21.15.43:0/1668343872 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4d9406b500 0x7f4d9406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:11.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.397+0000 7f4dabfff640 1 --2- 172.21.15.43:0/1668343872 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4d9406b500 0x7f4d9406d9c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4db40ca1d0 tx=0x7f4d9c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:11.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.401+0000 7f4dab7fe640 1 -- 172.21.15.43:0/1668343872 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4da4019030 con 0x7f4da0004d40 2024-04-25T08:55:11.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.619+0000 7f4db923a640 1 -- 172.21.15.43:0/1668343872 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4db4074ef0 con 0x7f4da0004d40 2024-04-25T08:55:11.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.619+0000 7f4dab7fe640 1 -- 172.21.15.43:0/1668343872 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+4934 (secure 0 0 0) 0x7f4da4059730 con 0x7f4da0004d40 2024-04-25T08:55:11.609 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:55:11.609 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":13,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4984,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/715858990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":715858990},{"type":"v1","addr":"172.21.15.193:6836","nonce":715858990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4987,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2342354979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2342354979},{"type":"v1","addr":"172.21.15.193:6837","nonce":2342354979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":5205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/491504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":491504707},{"type":"v1","addr":"172.21.15.43:6834","nonce":491504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":13,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:55:06.885018+0000","modified":"2024-04-25T08:55:08.875946+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5208},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5208":{"gid":5208,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/242762542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":242762542},{"type":"v1","addr":"172.21.15.43:6835","nonce":242762542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T08:55:11.610 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 13 2024-04-25T08:55:11.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.621+0000 7f4db923a640 1 -- 172.21.15.43:0/1668343872 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4d9406b500 msgr2=0x7f4d9406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:11.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.621+0000 7f4db923a640 1 --2- 172.21.15.43:0/1668343872 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4d9406b500 0x7f4d9406d9c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4db40ca1d0 tx=0x7f4d9c009040 comp rx=0 tx=0).stop 2024-04-25T08:55:11.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.621+0000 7f4db923a640 1 -- 172.21.15.43:0/1668343872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4da0004d40 msgr2=0x7f4db415a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:11.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.621+0000 7f4db923a640 1 --2- 172.21.15.43:0/1668343872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4da0004d40 0x7f4db415a4b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f4da4011a00 tx=0x7f4da4011ec0 comp rx=0 tx=0).stop 2024-04-25T08:55:11.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.621+0000 7f4db923a640 1 -- 172.21.15.43:0/1668343872 shutdown_connections 2024-04-25T08:55:11.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.621+0000 7f4db923a640 1 --2- 172.21.15.43:0/1668343872 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4d9406b500 0x7f4d9406d9c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:11.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.621+0000 7f4db923a640 1 --2- 172.21.15.43:0/1668343872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4db415c6d0 0x7f4db415cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:11.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.621+0000 7f4db923a640 1 --2- 172.21.15.43:0/1668343872 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4db415d480 0x7f4db415c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:11.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.621+0000 7f4db923a640 1 --2- 172.21.15.43:0/1668343872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4da0004d40 0x7f4db415a4b0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:11.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.621+0000 7f4db923a640 1 -- 172.21.15.43:0/1668343872 >> 172.21.15.43:0/1668343872 conn(0x7f4db409cd20 msgr2=0x7f4db40ce5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:11.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.621+0000 7f4db923a640 1 -- 172.21.15.43:0/1668343872 shutdown_connections 2024-04-25T08:55:11.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:11.621+0000 7f4db923a640 1 -- 172.21.15.43:0/1668343872 wait complete. 2024-04-25T08:55:12.025 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 13, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:55:06.885018+0000', 'modified': '2024-04-25T08:55:08.875946+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5208}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5208': {'gid': 5208, 'name': 'c', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.43:6835/242762542', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6833', 'nonce': 242762542}, {'type': 'v1', 'addr': '172.21.15.43:6835', 'nonce': 242762542}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:55:12.025 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:55:12.025 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T08:55:12.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.142+0000 7ff7f9161640 1 Processor -- start 2024-04-25T08:55:12.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.142+0000 7ff7f9161640 1 -- start start 2024-04-25T08:55:12.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.143+0000 7ff7f9161640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7f406d9a0 0x7ff7f4071a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:12.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.143+0000 7ff7f9161640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7f4072420 con 0x7ff7f415a9d0 2024-04-25T08:55:12.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.143+0000 7ff7f9161640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7f40704f0 con 0x7ff7f4071f90 2024-04-25T08:55:12.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.143+0000 7ff7f9161640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f40707d0 con 0x7ff7f406d9a0 2024-04-25T08:55:12.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.143+0000 7ff7f2d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff7f415a9d0 0x7ff7f406d290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32990/0 (socket says 172.21.15.43:32990) 2024-04-25T08:55:12.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.143+0000 7ff7f2d9d640 1 -- 172.21.15.43:0/2978273532 learned_addr learned my addr 172.21.15.43:0/2978273532 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:55:12.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.143+0000 7ff7f259c640 1 -- 172.21.15.43:0/2978273532 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1030221539 0 0) 0x7ff7e4001110 con 0x7ff7f415a9d0 2024-04-25T08:55:12.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.143+0000 7ff7f259c640 1 -- 172.21.15.43:0/2978273532 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7e0003520 con 0x7ff7f415a9d0 2024-04-25T08:55:12.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.143+0000 7ff7ebfff640 1 --2- 172.21.15.43:0/2978273532 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7f406d9a0 0x7ff7f4071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:12.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.143+0000 7ff7f259c640 1 -- 172.21.15.43:0/2978273532 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2326217400 0 0) 0x7ff7e0003520 con 0x7ff7f415a9d0 2024-04-25T08:55:12.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.143+0000 7ff7f259c640 1 -- 172.21.15.43:0/2978273532 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7f406d9a0 msgr2=0x7ff7f4071a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:12.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.143+0000 7ff7f259c640 1 --2- 172.21.15.43:0/2978273532 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7f406d9a0 0x7ff7f4071a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:12.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.143+0000 7ff7f259c640 1 -- 172.21.15.43:0/2978273532 >> v1:172.21.15.193:6789/0 conn(0x7ff7f4071f90 legacy=0x7ff7f40cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:12.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.144+0000 7ff7f259c640 1 -- 172.21.15.43:0/2978273532 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f4070ab0 con 0x7ff7f415a9d0 2024-04-25T08:55:12.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.144+0000 7ff7f259c640 1 -- 172.21.15.43:0/2978273532 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7ff7e4001840 con 0x7ff7f415a9d0 2024-04-25T08:55:12.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.144+0000 7ff7f259c640 1 -- 172.21.15.43:0/2978273532 >> v1:172.21.15.43:6789/0 conn(0x7ff7f415a9d0 legacy=0x7ff7f406d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:12.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.144+0000 7ff7f259c640 1 --2- 172.21.15.43:0/2978273532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7e00039e0 0x7ff7e0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:12.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.144+0000 7ff7f259c640 1 --2- 172.21.15.43:0/2978273532 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7f406d9a0 0x7ff7e0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:12.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.144+0000 7ff7f259c640 1 --2- 172.21.15.43:0/2978273532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7e0004d40 0x7ff7e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:12.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.144+0000 7ff7f2d9d640 1 --2- 172.21.15.43:0/2978273532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7e00039e0 0x7ff7e0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:12.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.144+0000 7ff7f259c640 1 -- 172.21.15.43:0/2978273532 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f40707d0 con 0x7ff7e00039e0 2024-04-25T08:55:12.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.144+0000 7ff7f259c640 1 -- 172.21.15.43:0/2978273532 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e000ba70 con 0x7ff7e0004d40 2024-04-25T08:55:12.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.144+0000 7ff7f259c640 1 -- 172.21.15.43:0/2978273532 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e000bc70 con 0x7ff7f406d9a0 2024-04-25T08:55:12.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.144+0000 7ff7ebfff640 1 --2- 172.21.15.43:0/2978273532 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7f406d9a0 0x7ff7e0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:12.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.144+0000 7ff7f359e640 1 --2- 172.21.15.43:0/2978273532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7e0004d40 0x7ff7e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:12.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.144+0000 7ff7f2d9d640 1 -- 172.21.15.43:0/2978273532 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7f406d9a0 msgr2=0x7ff7e0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:12.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.144+0000 7ff7f2d9d640 1 --2- 172.21.15.43:0/2978273532 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7f406d9a0 0x7ff7e0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:12.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.144+0000 7ff7f2d9d640 1 -- 172.21.15.43:0/2978273532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7e0004d40 msgr2=0x7ff7e00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:12.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.144+0000 7ff7f2d9d640 1 --2- 172.21.15.43:0/2978273532 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7e0004d40 0x7ff7e00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:12.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.144+0000 7ff7f2d9d640 1 -- 172.21.15.43:0/2978273532 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e0003520 con 0x7ff7e00039e0 2024-04-25T08:55:12.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.145+0000 7ff7f2d9d640 1 --2- 172.21.15.43:0/2978273532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7e00039e0 0x7ff7e0003e00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff7e400c490 tx=0x7ff7e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89753776b749182 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:12.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.145+0000 7ff7f259c640 1 -- 172.21.15.43:0/2978273532 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e400c0e0 con 0x7ff7e00039e0 2024-04-25T08:55:12.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.145+0000 7ff7f259c640 1 -- 172.21.15.43:0/2978273532 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e4034070 con 0x7ff7e00039e0 2024-04-25T08:55:12.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.145+0000 7ff7f259c640 1 -- 172.21.15.43:0/2978273532 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4005d30 con 0x7ff7e00039e0 2024-04-25T08:55:12.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.145+0000 7ff7f9161640 1 -- 172.21.15.43:0/2978273532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7e00039e0 msgr2=0x7ff7e0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:12.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.145+0000 7ff7f9161640 1 --2- 172.21.15.43:0/2978273532 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7e00039e0 0x7ff7e0003e00 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7ff7e400c490 tx=0x7ff7e0005830 comp rx=0 tx=0).stop 2024-04-25T08:55:12.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.145+0000 7ff7f359e640 1 -- 172.21.15.43:0/2978273532 reap_dead start 2024-04-25T08:55:12.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.145+0000 7ff7f9161640 1 -- 172.21.15.43:0/2978273532 shutdown_connections 2024-04-25T08:55:12.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.145+0000 7ff7f9161640 1 -- 172.21.15.43:0/2978273532 >> 172.21.15.43:0/2978273532 conn(0x7ff7f40b1480 msgr2=0x7ff7f40b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:12.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.145+0000 7ff7f9161640 1 -- 172.21.15.43:0/2978273532 shutdown_connections 2024-04-25T08:55:12.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.146+0000 7ff7f9161640 1 -- 172.21.15.43:0/2978273532 wait complete. 2024-04-25T08:55:12.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.147+0000 7ff7f9161640 1 Processor -- start 2024-04-25T08:55:12.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.147+0000 7ff7f9161640 1 -- start start 2024-04-25T08:55:12.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.148+0000 7ff7f9161640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7e00039e0 0x7ff7f413fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:12.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.148+0000 7ff7f9161640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7f4150a90 0x7ff7f4153270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:12.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.148+0000 7ff7f9161640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7f41537b0 0x7ff7f41426e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:12.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.148+0000 7ff7f9161640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f4071a50 con 0x7ff7f41537b0 2024-04-25T08:55:12.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.148+0000 7ff7f9161640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f40712b0 con 0x7ff7e00039e0 2024-04-25T08:55:12.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.148+0000 7ff7f9161640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f4071bd0 con 0x7ff7f4150a90 2024-04-25T08:55:12.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.148+0000 7ff7f359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7f41537b0 0x7ff7f41426e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:12.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.148+0000 7ff7f359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7f41537b0 0x7ff7f41426e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38338/0 (socket says 172.21.15.43:38338) 2024-04-25T08:55:12.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.148+0000 7ff7f359e640 1 -- 172.21.15.43:0/854096711 learned_addr learned my addr 172.21.15.43:0/854096711 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:12.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.148+0000 7ff7ebfff640 1 --2- 172.21.15.43:0/854096711 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7f4150a90 0x7ff7f4153270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:12.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.148+0000 7ff7f2d9d640 1 --2- 172.21.15.43:0/854096711 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7e00039e0 0x7ff7f413fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:12.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.148+0000 7ff7f359e640 1 -- 172.21.15.43:0/854096711 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7f4150a90 msgr2=0x7ff7f4153270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:12.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.148+0000 7ff7f359e640 1 --2- 172.21.15.43:0/854096711 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7f4150a90 0x7ff7f4153270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:12.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.148+0000 7ff7f359e640 1 -- 172.21.15.43:0/854096711 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7e00039e0 msgr2=0x7ff7f413fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:12.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.148+0000 7ff7f359e640 1 --2- 172.21.15.43:0/854096711 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7e00039e0 0x7ff7f413fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:12.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.148+0000 7ff7f359e640 1 -- 172.21.15.43:0/854096711 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f4143520 con 0x7ff7f41537b0 2024-04-25T08:55:12.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.148+0000 7ff7ebfff640 1 --2- 172.21.15.43:0/854096711 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7f4150a90 0x7ff7f4153270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:55:12.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.148+0000 7ff7f359e640 1 --2- 172.21.15.43:0/854096711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7f41537b0 0x7ff7f41426e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4072e00 tx=0x7ff7ec00fc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:12.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.148+0000 7ff7eb7fe640 1 -- 172.21.15.43:0/854096711 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec01a070 con 0x7ff7f41537b0 2024-04-25T08:55:12.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.148+0000 7ff7eb7fe640 1 -- 172.21.15.43:0/854096711 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7ec00fe60 con 0x7ff7f41537b0 2024-04-25T08:55:12.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.148+0000 7ff7eb7fe640 1 -- 172.21.15.43:0/854096711 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec003970 con 0x7ff7f41537b0 2024-04-25T08:55:12.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.148+0000 7ff7f9161640 1 -- 172.21.15.43:0/854096711 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7f4140650 con 0x7ff7f41537b0 2024-04-25T08:55:12.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.148+0000 7ff7f9161640 1 -- 172.21.15.43:0/854096711 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7f4140c10 con 0x7ff7f41537b0 2024-04-25T08:55:12.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.150+0000 7ff7eb7fe640 1 -- 172.21.15.43:0/854096711 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff7ec01d070 con 0x7ff7f41537b0 2024-04-25T08:55:12.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.150+0000 7ff7eb7fe640 1 --2- 172.21.15.43:0/854096711 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff7d406b4d0 0x7ff7d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:12.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.150+0000 7ff7eb7fe640 1 -- 172.21.15.43:0/854096711 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff7ec089710 con 0x7ff7f41537b0 2024-04-25T08:55:12.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.151+0000 7ff7f2d9d640 1 --2- 172.21.15.43:0/854096711 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff7d406b4d0 0x7ff7d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:12.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.151+0000 7ff7f2d9d640 1 --2- 172.21.15.43:0/854096711 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff7d406b4d0 0x7ff7d406d990 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff7e00057e0 tx=0x7ff7e402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:12.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.151+0000 7ff7fbe5e740 1 Processor -- start 2024-04-25T08:55:12.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.151+0000 7ff7fbe5e740 1 -- start start 2024-04-25T08:55:12.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.152+0000 7ff7fbe5e740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x557fcd682230 0x557fcd682610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:12.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.152+0000 7ff7fbe5e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fcd69fad0 0x557fcd6a0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:12.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.152+0000 7ff7fbe5e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557fcd657f10 0x557fcd69e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:12.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.152+0000 7ff7fbe5e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fcd5baa10 con 0x557fcd69fad0 2024-04-25T08:55:12.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.152+0000 7ff7fbe5e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fcd563b30 con 0x557fcd657f10 2024-04-25T08:55:12.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.152+0000 7ff7fbe5e740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x557fcd5b6c50 con 0x557fcd682230 2024-04-25T08:55:12.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.152+0000 7ff7ebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fcd69fad0 0x557fcd6a0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:12.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.152+0000 7ff7ebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fcd69fad0 0x557fcd6a0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38342/0 (socket says 172.21.15.43:38342) 2024-04-25T08:55:12.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.152+0000 7ff7ebfff640 1 -- 172.21.15.43:0/2147766584 learned_addr learned my addr 172.21.15.43:0/2147766584 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:12.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.152+0000 7ff7f2d9d640 1 --2- 172.21.15.43:0/2147766584 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x557fcd682230 0x557fcd682610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:12.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.152+0000 7ff7f359e640 1 --2- 172.21.15.43:0/2147766584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557fcd657f10 0x557fcd69e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:12.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.153+0000 7ff7ebfff640 1 -- 172.21.15.43:0/2147766584 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x557fcd682230 msgr2=0x557fcd682610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:12.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.153+0000 7ff7ebfff640 1 --2- 172.21.15.43:0/2147766584 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x557fcd682230 0x557fcd682610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:12.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.153+0000 7ff7ebfff640 1 -- 172.21.15.43:0/2147766584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557fcd657f10 msgr2=0x557fcd69e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:12.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.153+0000 7ff7ebfff640 1 --2- 172.21.15.43:0/2147766584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557fcd657f10 0x557fcd69e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:12.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.153+0000 7ff7ebfff640 1 -- 172.21.15.43:0/2147766584 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fcd578e10 con 0x557fcd69fad0 2024-04-25T08:55:12.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.153+0000 7ff7f359e640 1 --2- 172.21.15.43:0/2147766584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557fcd657f10 0x557fcd69e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:55:12.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.153+0000 7ff7ebfff640 1 --2- 172.21.15.43:0/2147766584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fcd69fad0 0x557fcd6a0020 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc009b90 tx=0x7ff7dc009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dcf73a0c6253cf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:12.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.153+0000 7ff7e8ff9640 1 -- 172.21.15.43:0/2147766584 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc01f480 con 0x557fcd69fad0 2024-04-25T08:55:12.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.153+0000 7ff7e8ff9640 1 -- 172.21.15.43:0/2147766584 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7dc01f620 con 0x557fcd69fad0 2024-04-25T08:55:12.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.153+0000 7ff7e8ff9640 1 -- 172.21.15.43:0/2147766584 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc01f9d0 con 0x557fcd69fad0 2024-04-25T08:55:12.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.153+0000 7ff7fbe5e740 1 -- 172.21.15.43:0/2147766584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fcd69fad0 msgr2=0x557fcd6a0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:12.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.154+0000 7ff7fbe5e740 1 --2- 172.21.15.43:0/2147766584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fcd69fad0 0x557fcd6a0020 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc009b90 tx=0x7ff7dc009190 comp rx=0 tx=0).stop 2024-04-25T08:55:12.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.154+0000 7ff7fbe5e740 1 -- 172.21.15.43:0/2147766584 shutdown_connections 2024-04-25T08:55:12.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.154+0000 7ff7fbe5e740 1 --2- 172.21.15.43:0/2147766584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557fcd657f10 0x557fcd69e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:12.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.154+0000 7ff7fbe5e740 1 --2- 172.21.15.43:0/2147766584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fcd69fad0 0x557fcd6a0020 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:12.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.154+0000 7ff7fbe5e740 1 --2- 172.21.15.43:0/2147766584 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x557fcd682230 0x557fcd682610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:12.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.154+0000 7ff7fbe5e740 1 -- 172.21.15.43:0/2147766584 >> 172.21.15.43:0/2147766584 conn(0x557fcd654840 msgr2=0x557fcd69d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:12.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.154+0000 7ff7fbe5e740 1 -- 172.21.15.43:0/2147766584 shutdown_connections 2024-04-25T08:55:12.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.154+0000 7ff7fbe5e740 1 -- 172.21.15.43:0/2147766584 wait complete. 2024-04-25T08:55:12.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.154+0000 7ff7fbe5e740 1 -- start start 2024-04-25T08:55:12.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.154+0000 7ff7fbe5e740 1 Processor -- start 2024-04-25T08:55:12.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.155+0000 7ff7fbe5e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fcd682230 0x557fcd685ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:12.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.155+0000 7ff7fbe5e740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x557fcd69fad0 0x557fcd682f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:12.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.155+0000 7ff7fbe5e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557fcd6834c0 0x557fcd67b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:12.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.155+0000 7ff7fbe5e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fcd658cb0 con 0x557fcd682230 2024-04-25T08:55:12.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.155+0000 7ff7fbe5e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fcd5770c0 con 0x557fcd6834c0 2024-04-25T08:55:12.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.155+0000 7ff7fbe5e740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x557fcd652770 con 0x557fcd69fad0 2024-04-25T08:55:12.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.155+0000 7ff7fbe5e740 10 client.0 _ll_register_callbacks cb 0x557fcd56aec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:55:12.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.155+0000 7ff7f2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fcd682230 0x557fcd685ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:12.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.155+0000 7ff7f2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fcd682230 0x557fcd685ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38352/0 (socket says 172.21.15.43:38352) 2024-04-25T08:55:12.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.155+0000 7ff7f2d9d640 1 -- 172.21.15.43:0/936796645 learned_addr learned my addr 172.21.15.43:0/936796645 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:12.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.155+0000 7ff7ebfff640 1 --2- 172.21.15.43:0/936796645 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x557fcd69fad0 0x557fcd682f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:12.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.155+0000 7ff7f359e640 1 --2- 172.21.15.43:0/936796645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557fcd6834c0 0x557fcd67b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:12.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.155+0000 7ff7f2d9d640 1 -- 172.21.15.43:0/936796645 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x557fcd69fad0 msgr2=0x557fcd682f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:12.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.155+0000 7ff7f2d9d640 1 --2- 172.21.15.43:0/936796645 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x557fcd69fad0 0x557fcd682f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:12.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.155+0000 7ff7f2d9d640 1 -- 172.21.15.43:0/936796645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557fcd6834c0 msgr2=0x557fcd67b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:12.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.155+0000 7ff7f2d9d640 1 --2- 172.21.15.43:0/936796645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557fcd6834c0 0x557fcd67b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:12.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.155+0000 7ff7f2d9d640 1 -- 172.21.15.43:0/936796645 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557fcd2cf2d0 con 0x557fcd682230 2024-04-25T08:55:12.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.155+0000 7ff7f359e640 1 --2- 172.21.15.43:0/936796645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557fcd6834c0 0x557fcd67b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:55:12.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.156+0000 7ff7f2d9d640 1 --2- 172.21.15.43:0/936796645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fcd682230 0x557fcd685ab0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff7e400c580 tx=0x7ff7e4007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58abdeee94c3ddfe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:12.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.156+0000 7ff7d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:55:12.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.156+0000 7ff7d1ffb640 1 -- 172.21.15.43:0/936796645 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4056060 con 0x557fcd682230 2024-04-25T08:55:12.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.156+0000 7ff7d1ffb640 1 -- 172.21.15.43:0/936796645 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff7e4007070 con 0x557fcd682230 2024-04-25T08:55:12.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.156+0000 7ff7fbe5e740 1 -- 172.21.15.43:0/936796645 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff7dc01f620 con 0x557fcd682230 2024-04-25T08:55:12.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.156+0000 7ff7d1ffb640 1 -- 172.21.15.43:0/936796645 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7e405c050 con 0x557fcd682230 2024-04-25T08:55:12.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.156+0000 7ff7fbe5e740 10 client.5382 fetch_fsmap learned FSMap version 13 2024-04-25T08:55:12.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.156+0000 7ff7fbe5e740 1 -- 172.21.15.43:0/936796645 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x557fcd3996b0 con 0x557fcd682230 2024-04-25T08:55:12.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.156+0000 7ff7d1ffb640 1 -- 172.21.15.43:0/936796645 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7dc01f620 con 0x557fcd682230 2024-04-25T08:55:12.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.156+0000 7ff7d1ffb640 1 -- 172.21.15.43:0/936796645 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 13) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff7e405f020 con 0x557fcd682230 2024-04-25T08:55:12.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.156+0000 7ff7fbe5e740 10 client.5382 fetch_fsmap finished waiting for FSMap version 13 2024-04-25T08:55:12.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.156+0000 7ff7fbe5e740 10 client.5382 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:55:12.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.156+0000 7ff7fbe5e740 20 client.5382 populate_metadata read hostname 'smithi043' 2024-04-25T08:55:12.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.156+0000 7ff7fbe5e740 1 --2- 172.21.15.43:0/936796645 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x557fcd6b1280 0x557fcd694960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:12.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.156+0000 7ff7fbe5e740 4 client.5382 mds_command: new command op to 5208 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:55:12.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.156+0000 7ff7fbe5e740 1 -- 172.21.15.43:0/936796645 --> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557fcd619920 con 0x557fcd6b1280 2024-04-25T08:55:12.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.156+0000 7ff7ebfff640 1 --2- 172.21.15.43:0/936796645 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x557fcd6b1280 0x557fcd694960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:12.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.157+0000 7ff7d1ffb640 10 client.5382 ms_handle_connect on v2:172.21.15.43:6833/242762542 2024-04-25T08:55:12.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.157+0000 7ff7ebfff640 1 --2- 172.21.15.43:0/936796645 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x557fcd6b1280 0x557fcd694960 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48f5424317cec1cd server_cookie=335d4180117a26fc in_seq=0 out_seq=0 2024-04-25T08:55:12.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.157+0000 7ff7d1ffb640 1 -- 172.21.15.43:0/936796645 <== mds.0 v2:172.21.15.43:6833/242762542 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557fcd619920 con 0x557fcd6b1280 2024-04-25T08:55:12.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.157+0000 7ff7d1ffb640 10 client.5382 handle_command_reply: tid=0 2024-04-25T08:55:12.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.157+0000 7ff7fbe5e740 1 client.5382 shutdown 2024-04-25T08:55:12.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.157+0000 7ff7fbe5e740 1 -- 172.21.15.43:0/936796645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fcd682230 msgr2=0x557fcd685ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:12.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.157+0000 7ff7fbe5e740 1 --2- 172.21.15.43:0/936796645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fcd682230 0x557fcd685ab0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7ff7e400c580 tx=0x7ff7e4007000 comp rx=0 tx=0).stop 2024-04-25T08:55:12.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.158+0000 7ff7fbe5e740 1 -- 172.21.15.43:0/936796645 shutdown_connections 2024-04-25T08:55:12.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.158+0000 7ff7fbe5e740 1 --2- 172.21.15.43:0/936796645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557fcd6834c0 0x557fcd67b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:12.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.158+0000 7ff7fbe5e740 1 --2- 172.21.15.43:0/936796645 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x557fcd69fad0 0x557fcd682f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:12.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.158+0000 7ff7fbe5e740 1 --2- 172.21.15.43:0/936796645 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x557fcd6b1280 0x557fcd694960 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:12.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.158+0000 7ff7fbe5e740 1 --2- 172.21.15.43:0/936796645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fcd682230 0x557fcd685ab0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:12.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.158+0000 7ff7d1ffb640 0 client.5382 ms_handle_reset on v2:172.21.15.43:6833/242762542 2024-04-25T08:55:12.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.158+0000 7ff7fbe5e740 1 -- 172.21.15.43:0/936796645 >> 172.21.15.43:0/936796645 conn(0x557fcd657f10 msgr2=0x557fcd6a0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:12.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.158+0000 7ff7fbe5e740 1 -- 172.21.15.43:0/936796645 shutdown_connections 2024-04-25T08:55:12.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.158+0000 7ff7fbe5e740 1 -- 172.21.15.43:0/936796645 wait complete. 2024-04-25T08:55:12.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.158+0000 7ff7fbe5e740 20 client.5382 trim_cache size 0 max 16384 2024-04-25T08:55:12.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.173+0000 7ff7fbe5e740 1 Processor -- start 2024-04-25T08:55:12.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.173+0000 7ff7fbe5e740 1 -- start start 2024-04-25T08:55:12.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.174+0000 7ff7fbe5e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557fcd682230 0x557fcd697280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:12.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.174+0000 7ff7fbe5e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fcd6834c0 0x557fcd694c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:12.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.174+0000 7ff7fbe5e740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x557fcd69fad0 0x557fcd6584e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:12.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.174+0000 7ff7fbe5e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f4157ee0 con 0x557fcd6834c0 2024-04-25T08:55:12.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.174+0000 7ff7fbe5e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f41530a0 con 0x557fcd682230 2024-04-25T08:55:12.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.174+0000 7ff7fbe5e740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f4142410 con 0x557fcd69fad0 2024-04-25T08:55:12.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.174+0000 7ff7ebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fcd6834c0 0x557fcd694c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:12.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.174+0000 7ff7ebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fcd6834c0 0x557fcd694c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38364/0 (socket says 172.21.15.43:38364) 2024-04-25T08:55:12.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.174+0000 7ff7ebfff640 1 -- 172.21.15.43:0/4291584056 learned_addr learned my addr 172.21.15.43:0/4291584056 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:12.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.175+0000 7ff7ebfff640 1 -- 172.21.15.43:0/4291584056 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x557fcd69fad0 msgr2=0x557fcd6584e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:12.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.175+0000 7ff7ebfff640 1 --2- 172.21.15.43:0/4291584056 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x557fcd69fad0 0x557fcd6584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:12.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.175+0000 7ff7ebfff640 1 -- 172.21.15.43:0/4291584056 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557fcd682230 msgr2=0x557fcd697280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:12.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.175+0000 7ff7f2d9d640 1 --2- 172.21.15.43:0/4291584056 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557fcd682230 0x557fcd697280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:12.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.175+0000 7ff7ebfff640 1 --2- 172.21.15.43:0/4291584056 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557fcd682230 0x557fcd697280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:12.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.175+0000 7ff7ebfff640 1 -- 172.21.15.43:0/4291584056 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557fcd69f8c0 con 0x557fcd6834c0 2024-04-25T08:55:12.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.175+0000 7ff7ebfff640 1 --2- 172.21.15.43:0/4291584056 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fcd6834c0 0x557fcd694c20 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc009b60 tx=0x7ff7dc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ec82762520c669 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:12.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.175+0000 7ff7d3fff640 1 -- 172.21.15.43:0/4291584056 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc004230 con 0x557fcd6834c0 2024-04-25T08:55:12.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.175+0000 7ff7d3fff640 1 -- 172.21.15.43:0/4291584056 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7dc030070 con 0x557fcd6834c0 2024-04-25T08:55:12.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.175+0000 7ff7d3fff640 1 -- 172.21.15.43:0/4291584056 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7dc02b400 con 0x557fcd6834c0 2024-04-25T08:55:12.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.175+0000 7ff7fbe5e740 1 -- 172.21.15.43:0/4291584056 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fcd6834c0 msgr2=0x557fcd694c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:12.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.175+0000 7ff7fbe5e740 1 --2- 172.21.15.43:0/4291584056 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fcd6834c0 0x557fcd694c20 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7ff7dc009b60 tx=0x7ff7dc019000 comp rx=0 tx=0).stop 2024-04-25T08:55:12.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.175+0000 7ff7fbe5e740 1 -- 172.21.15.43:0/4291584056 shutdown_connections 2024-04-25T08:55:12.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.175+0000 7ff7fbe5e740 1 --2- 172.21.15.43:0/4291584056 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x557fcd69fad0 0x557fcd6584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:12.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.175+0000 7ff7fbe5e740 1 --2- 172.21.15.43:0/4291584056 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fcd6834c0 0x557fcd694c20 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:12.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.175+0000 7ff7fbe5e740 1 --2- 172.21.15.43:0/4291584056 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557fcd682230 0x557fcd697280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:12.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.175+0000 7ff7fbe5e740 1 -- 172.21.15.43:0/4291584056 >> 172.21.15.43:0/4291584056 conn(0x557fcd657f10 msgr2=0x557fcd681290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:12.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.176+0000 7ff7fbe5e740 1 -- 172.21.15.43:0/4291584056 shutdown_connections 2024-04-25T08:55:12.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.176+0000 7ff7fbe5e740 1 -- 172.21.15.43:0/4291584056 wait complete. 2024-04-25T08:55:12.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.176+0000 7ff7fbe5e740 1 -- start start 2024-04-25T08:55:12.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.176+0000 7ff7fbe5e740 1 Processor -- start 2024-04-25T08:55:12.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.177+0000 7ff7fbe5e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557fcd682230 0x557fcd6fe350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:12.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.177+0000 7ff7fbe5e740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x557fcd6834c0 0x557fcd696a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:12.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.177+0000 7ff7fbe5e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fcd69fad0 0x557fcd696f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:12.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.177+0000 7ff7fbe5e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fcd6b1100 con 0x557fcd69fad0 2024-04-25T08:55:12.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.177+0000 7ff7fbe5e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x557fcd684810 con 0x557fcd682230 2024-04-25T08:55:12.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.177+0000 7ff7fbe5e740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x557fcd6b1700 con 0x557fcd6834c0 2024-04-25T08:55:12.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.177+0000 7ff7fbe5e740 10 client.0 _ll_register_callbacks cb 0x557fcd652c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:55:12.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.177+0000 7ff7f359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fcd69fad0 0x557fcd696f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:12.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.177+0000 7ff7ebfff640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x557fcd6834c0 0x557fcd696a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:12.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.177+0000 7ff7f359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fcd69fad0 0x557fcd696f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38374/0 (socket says 172.21.15.43:38374) 2024-04-25T08:55:12.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.177+0000 7ff7f359e640 1 -- 172.21.15.43:0/1668149240 learned_addr learned my addr 172.21.15.43:0/1668149240 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:12.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.177+0000 7ff7f2d9d640 1 --2- 172.21.15.43:0/1668149240 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557fcd682230 0x557fcd6fe350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:12.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.177+0000 7ff7f359e640 1 -- 172.21.15.43:0/1668149240 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x557fcd6834c0 msgr2=0x557fcd696a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:12.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.177+0000 7ff7f359e640 1 --2- 172.21.15.43:0/1668149240 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x557fcd6834c0 0x557fcd696a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:12.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.177+0000 7ff7f359e640 1 -- 172.21.15.43:0/1668149240 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557fcd682230 msgr2=0x557fcd6fe350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:12.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.177+0000 7ff7f359e640 1 --2- 172.21.15.43:0/1668149240 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557fcd682230 0x557fcd6fe350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:12.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.177+0000 7ff7f359e640 1 -- 172.21.15.43:0/1668149240 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557fcd6a0c70 con 0x557fcd69fad0 2024-04-25T08:55:12.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.177+0000 7ff7f359e640 1 --2- 172.21.15.43:0/1668149240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fcd69fad0 0x557fcd696f40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec057e20 tx=0x7ff7ec0586d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d67ae4fce2d1c01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:12.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.177+0000 7ff7d1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:55:12.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.178+0000 7ff7d1ffb640 1 -- 172.21.15.43:0/1668149240 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec06ce60 con 0x557fcd69fad0 2024-04-25T08:55:12.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.178+0000 7ff7d1ffb640 1 -- 172.21.15.43:0/1668149240 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff7ec060110 con 0x557fcd69fad0 2024-04-25T08:55:12.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.178+0000 7ff7fbe5e740 1 -- 172.21.15.43:0/1668149240 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff7dc030070 con 0x557fcd69fad0 2024-04-25T08:55:12.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.178+0000 7ff7d1ffb640 1 -- 172.21.15.43:0/1668149240 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=13) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7ec060310 con 0x557fcd69fad0 2024-04-25T08:55:12.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.178+0000 7ff7fbe5e740 10 client.5388 fetch_fsmap learned FSMap version 13 2024-04-25T08:55:12.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.178+0000 7ff7fbe5e740 1 -- 172.21.15.43:0/1668149240 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=13,monmap=2+}) v3 -- 0x557fcd6a42a0 con 0x557fcd69fad0 2024-04-25T08:55:12.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.178+0000 7ff7d1ffb640 1 -- 172.21.15.43:0/1668149240 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7dc030070 con 0x557fcd69fad0 2024-04-25T08:55:12.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.178+0000 7ff7d1ffb640 1 -- 172.21.15.43:0/1668149240 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 13) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff7ec0572e0 con 0x557fcd69fad0 2024-04-25T08:55:12.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.178+0000 7ff7fbe5e740 10 client.5388 fetch_fsmap finished waiting for FSMap version 13 2024-04-25T08:55:12.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.178+0000 7ff7fbe5e740 10 client.5388 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:55:12.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.178+0000 7ff7fbe5e740 20 client.5388 populate_metadata read hostname 'smithi043' 2024-04-25T08:55:12.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.178+0000 7ff7fbe5e740 1 --2- 172.21.15.43:0/1668149240 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x557fcd708470 0x557fcd655de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:12.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.178+0000 7ff7fbe5e740 4 client.5388 mds_command: new command op to 5208 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:55:12.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.178+0000 7ff7fbe5e740 1 -- 172.21.15.43:0/1668149240 --> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557fcd5c7f50 con 0x557fcd708470 2024-04-25T08:55:12.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.178+0000 7ff7f2d9d640 1 --2- 172.21.15.43:0/1668149240 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x557fcd708470 0x557fcd655de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:12.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.178+0000 7ff7d1ffb640 10 client.5388 ms_handle_connect on v2:172.21.15.43:6833/242762542 2024-04-25T08:55:12.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.178+0000 7ff7f2d9d640 1 --2- 172.21.15.43:0/1668149240 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x557fcd708470 0x557fcd655de0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd2c5140dc668f06 server_cookie=4c15d827abee068f in_seq=0 out_seq=0 2024-04-25T08:55:12.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.179+0000 7ff7d1ffb640 1 -- 172.21.15.43:0/1668149240 <== mds.0 v2:172.21.15.43:6833/242762542 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x557fcd5c7f50 con 0x557fcd708470 2024-04-25T08:55:12.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.179+0000 7ff7d1ffb640 10 client.5388 handle_command_reply: tid=0 2024-04-25T08:55:12.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.179+0000 7ff7fbe5e740 1 client.5388 shutdown 2024-04-25T08:55:12.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.179+0000 7ff7fbe5e740 1 -- 172.21.15.43:0/1668149240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fcd69fad0 msgr2=0x557fcd696f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:12.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.179+0000 7ff7fbe5e740 1 --2- 172.21.15.43:0/1668149240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fcd69fad0 0x557fcd696f40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec057e20 tx=0x7ff7ec0586d0 comp rx=0 tx=0).stop 2024-04-25T08:55:12.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.179+0000 7ff7fbe5e740 1 -- 172.21.15.43:0/1668149240 shutdown_connections 2024-04-25T08:55:12.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.179+0000 7ff7fbe5e740 1 --2- 172.21.15.43:0/1668149240 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x557fcd708470 0x557fcd655de0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:12.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.179+0000 7ff7fbe5e740 1 --2- 172.21.15.43:0/1668149240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x557fcd69fad0 0x557fcd696f40 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:12.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.179+0000 7ff7fbe5e740 1 --2- 172.21.15.43:0/1668149240 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x557fcd6834c0 0x557fcd696a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:12.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.179+0000 7ff7d1ffb640 0 client.5388 ms_handle_reset on v2:172.21.15.43:6833/242762542 2024-04-25T08:55:12.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.179+0000 7ff7fbe5e740 1 --2- 172.21.15.43:0/1668149240 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x557fcd682230 0x557fcd6fe350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:12.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.179+0000 7ff7fbe5e740 1 -- 172.21.15.43:0/1668149240 >> 172.21.15.43:0/1668149240 conn(0x557fcd657f10 msgr2=0x557fcd6a39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:12.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.179+0000 7ff7fbe5e740 1 -- 172.21.15.43:0/1668149240 shutdown_connections 2024-04-25T08:55:12.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.179+0000 7ff7fbe5e740 1 -- 172.21.15.43:0/1668149240 wait complete. 2024-04-25T08:55:12.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.179+0000 7ff7fbe5e740 20 client.5388 trim_cache size 0 max 16384 2024-04-25T08:55:12.175 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T08:55:12.175 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "6c4bd50e-aa00-4a7f-8d09-352344ef5a4b", 2024-04-25T08:55:12.175 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-04-25T08:55:12.175 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 5208, 2024-04-25T08:55:12.175 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-04-25T08:55:12.175 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-04-25T08:55:12.175 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-04-25T08:55:12.175 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 4.6708952659999996, 2024-04-25T08:55:12.175 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 13, 2024-04-25T08:55:12.175 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 33, 2024-04-25T08:55:12.175 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 33, 2024-04-25T08:55:12.175 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 11.843293284 2024-04-25T08:55:12.175 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T08:55:12.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.179+0000 7ff7f9161640 1 -- 172.21.15.43:0/854096711 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff7d406b4d0 msgr2=0x7ff7d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:12.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.179+0000 7ff7f9161640 1 --2- 172.21.15.43:0/854096711 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff7d406b4d0 0x7ff7d406d990 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff7e00057e0 tx=0x7ff7e402e000 comp rx=0 tx=0).stop 2024-04-25T08:55:12.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.179+0000 7ff7f9161640 1 -- 172.21.15.43:0/854096711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7f41537b0 msgr2=0x7ff7f41426e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:12.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.179+0000 7ff7f9161640 1 --2- 172.21.15.43:0/854096711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7f41537b0 0x7ff7f41426e0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7ff7f4072e00 tx=0x7ff7ec00fc70 comp rx=0 tx=0).stop 2024-04-25T08:55:12.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.179+0000 7ff7f9161640 1 -- 172.21.15.43:0/854096711 shutdown_connections 2024-04-25T08:55:12.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.179+0000 7ff7f9161640 1 --2- 172.21.15.43:0/854096711 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7f41537b0 0x7ff7f41426e0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:12.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.179+0000 7ff7f9161640 1 --2- 172.21.15.43:0/854096711 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7f4150a90 0x7ff7f4153270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:12.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.179+0000 7ff7f9161640 1 --2- 172.21.15.43:0/854096711 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff7d406b4d0 0x7ff7d406d990 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:12.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.179+0000 7ff7f9161640 1 --2- 172.21.15.43:0/854096711 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7e00039e0 0x7ff7f413fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:12.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.180+0000 7ff7f9161640 1 -- 172.21.15.43:0/854096711 >> 172.21.15.43:0/854096711 conn(0x7ff7f40b1480 msgr2=0x7ff7f40cd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:12.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.180+0000 7ff7f9161640 1 -- 172.21.15.43:0/854096711 shutdown_connections 2024-04-25T08:55:12.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.180+0000 7ff7f9161640 1 -- 172.21.15.43:0/854096711 wait complete. 2024-04-25T08:55:12.180 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:55:12.183 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-04-25T08:55:12.193+0000 7f6a2285a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:55:12.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.291+0000 7f9d32390640 1 Processor -- start 2024-04-25T08:55:12.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.292+0000 7f9d32390640 1 -- start start 2024-04-25T08:55:12.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.292+0000 7f9d32390640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d2c0cd010 0x7f9d2c0b4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:12.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.292+0000 7f9d32390640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d2c0b2780 con 0x7f9d2c0ccbb0 2024-04-25T08:55:12.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.292+0000 7f9d32390640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d2c0b2a60 con 0x7f9d2c0cbfc0 2024-04-25T08:55:12.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.292+0000 7f9d32390640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c0b2d40 con 0x7f9d2c0cd010 2024-04-25T08:55:12.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.293+0000 7f9d2b7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f9d2c0ccbb0 0x7f9d2c0b3b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:33002/0 (socket says 172.21.15.43:33002) 2024-04-25T08:55:12.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.293+0000 7f9d2b7fe640 1 -- 172.21.15.43:0/1260571102 learned_addr learned my addr 172.21.15.43:0/1260571102 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:55:12.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.293+0000 7f9d3092d640 1 --2- 172.21.15.43:0/1260571102 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d2c0cd010 0x7f9d2c0b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:12.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.293+0000 7f9d2affd640 1 -- 172.21.15.43:0/1260571102 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1163253870 0 0) 0x7f9d2c0b2780 con 0x7f9d2c0ccbb0 2024-04-25T08:55:12.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.293+0000 7f9d2affd640 1 -- 172.21.15.43:0/1260571102 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d14003520 con 0x7f9d2c0ccbb0 2024-04-25T08:55:12.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.293+0000 7f9d2affd640 1 -- 172.21.15.43:0/1260571102 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 577942670 0 0) 0x7f9d14003520 con 0x7f9d2c0ccbb0 2024-04-25T08:55:12.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.293+0000 7f9d2affd640 1 -- 172.21.15.43:0/1260571102 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d2c0cd010 msgr2=0x7f9d2c0b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:12.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.293+0000 7f9d2affd640 1 --2- 172.21.15.43:0/1260571102 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d2c0cd010 0x7f9d2c0b4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:12.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.293+0000 7f9d2affd640 1 -- 172.21.15.43:0/1260571102 >> v1:172.21.15.193:6789/0 conn(0x7f9d2c0cbfc0 legacy=0x7f9d2c06dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:12.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.293+0000 7f9d2affd640 1 -- 172.21.15.43:0/1260571102 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d2c0b3020 con 0x7f9d2c0ccbb0 2024-04-25T08:55:12.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.293+0000 7f9d2affd640 1 -- 172.21.15.43:0/1260571102 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f9d20001670 con 0x7f9d2c0ccbb0 2024-04-25T08:55:12.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.293+0000 7f9d2affd640 1 -- 172.21.15.43:0/1260571102 >> v1:172.21.15.43:6789/0 conn(0x7f9d2c0ccbb0 legacy=0x7f9d2c0b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:12.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.293+0000 7f9d2affd640 1 --2- 172.21.15.43:0/1260571102 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d140039e0 0x7f9d14003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:12.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.293+0000 7f9d2affd640 1 --2- 172.21.15.43:0/1260571102 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d2c0cd010 0x7f9d14004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:12.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.293+0000 7f9d2affd640 1 --2- 172.21.15.43:0/1260571102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d14004d40 0x7f9d140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:12.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.293+0000 7f9d2affd640 1 -- 172.21.15.43:0/1260571102 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c0b2d40 con 0x7f9d14004d40 2024-04-25T08:55:12.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.293+0000 7f9d2affd640 1 -- 172.21.15.43:0/1260571102 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1400b9c0 con 0x7f9d140039e0 2024-04-25T08:55:12.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.293+0000 7f9d2affd640 1 -- 172.21.15.43:0/1260571102 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d1400bbc0 con 0x7f9d2c0cd010 2024-04-25T08:55:12.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.293+0000 7f9d3092d640 1 --2- 172.21.15.43:0/1260571102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d14004d40 0x7f9d140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:12.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.293+0000 7f9d2bfff640 1 --2- 172.21.15.43:0/1260571102 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d140039e0 0x7f9d14003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:12.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.293+0000 7f9d2b7fe640 1 --2- 172.21.15.43:0/1260571102 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d2c0cd010 0x7f9d14004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:12.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.294+0000 7f9d3092d640 1 -- 172.21.15.43:0/1260571102 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d2c0cd010 msgr2=0x7f9d14004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:12.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.294+0000 7f9d3092d640 1 --2- 172.21.15.43:0/1260571102 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d2c0cd010 0x7f9d14004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:12.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.294+0000 7f9d3092d640 1 -- 172.21.15.43:0/1260571102 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d140039e0 msgr2=0x7f9d14003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:12.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.294+0000 7f9d3092d640 1 --2- 172.21.15.43:0/1260571102 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d140039e0 0x7f9d14003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:12.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.294+0000 7f9d3092d640 1 -- 172.21.15.43:0/1260571102 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d14003520 con 0x7f9d14004d40 2024-04-25T08:55:12.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.294+0000 7f9d2b7fe640 1 --2- 172.21.15.43:0/1260571102 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d2c0cd010 0x7f9d14004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:55:12.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.294+0000 7f9d3092d640 1 --2- 172.21.15.43:0/1260571102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d14004d40 0x7f9d140051c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9d24009fc0 tx=0x7f9d14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fdb65679a510166 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:12.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.295+0000 7f9d2affd640 1 -- 172.21.15.43:0/1260571102 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d24009c10 con 0x7f9d14004d40 2024-04-25T08:55:12.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.295+0000 7f9d2affd640 1 -- 172.21.15.43:0/1260571102 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d24036070 con 0x7f9d14004d40 2024-04-25T08:55:12.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.295+0000 7f9d2affd640 1 -- 172.21.15.43:0/1260571102 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d24031880 con 0x7f9d14004d40 2024-04-25T08:55:12.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.295+0000 7f9d32390640 1 -- 172.21.15.43:0/1260571102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d14004d40 msgr2=0x7f9d140051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:12.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.295+0000 7f9d32390640 1 --2- 172.21.15.43:0/1260571102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d14004d40 0x7f9d140051c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9d24009fc0 tx=0x7f9d14005830 comp rx=0 tx=0).stop 2024-04-25T08:55:12.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.295+0000 7f9d3092d640 1 -- 172.21.15.43:0/1260571102 reap_dead start 2024-04-25T08:55:12.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.295+0000 7f9d32390640 1 -- 172.21.15.43:0/1260571102 shutdown_connections 2024-04-25T08:55:12.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.295+0000 7f9d32390640 1 -- 172.21.15.43:0/1260571102 >> 172.21.15.43:0/1260571102 conn(0x7f9d2c0b1480 msgr2=0x7f9d2c0b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:12.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.295+0000 7f9d32390640 1 -- 172.21.15.43:0/1260571102 shutdown_connections 2024-04-25T08:55:12.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.295+0000 7f9d32390640 1 -- 172.21.15.43:0/1260571102 wait complete. 2024-04-25T08:55:12.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.296+0000 7f9d32390640 1 Processor -- start 2024-04-25T08:55:12.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.297+0000 7f9d32390640 1 -- start start 2024-04-25T08:55:12.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.297+0000 7f9d32390640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d14004d40 0x7f9d2c0ae450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:12.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.297+0000 7f9d32390640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d2c0ae990 0x7f9d2c1f3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:12.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.297+0000 7f9d32390640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d2c1f4080 0x7f9d2c1f4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:12.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.297+0000 7f9d32390640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c0cd4a0 con 0x7f9d2c1f4080 2024-04-25T08:55:12.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.297+0000 7f9d32390640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c0b35e0 con 0x7f9d14004d40 2024-04-25T08:55:12.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.297+0000 7f9d32390640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c0cc3a0 con 0x7f9d2c0ae990 2024-04-25T08:55:12.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.297+0000 7f9d3092d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d2c1f4080 0x7f9d2c1f4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:12.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.297+0000 7f9d3092d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d2c1f4080 0x7f9d2c1f4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38382/0 (socket says 172.21.15.43:38382) 2024-04-25T08:55:12.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.297+0000 7f9d3092d640 1 -- 172.21.15.43:0/210418145 learned_addr learned my addr 172.21.15.43:0/210418145 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:12.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.298+0000 7f9d3092d640 1 -- 172.21.15.43:0/210418145 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d2c0ae990 msgr2=0x7f9d2c1f3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:12.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.298+0000 7f9d3092d640 1 --2- 172.21.15.43:0/210418145 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d2c0ae990 0x7f9d2c1f3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:12.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.298+0000 7f9d3092d640 1 -- 172.21.15.43:0/210418145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d14004d40 msgr2=0x7f9d2c0ae450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:12.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.298+0000 7f9d3092d640 1 --2- 172.21.15.43:0/210418145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d14004d40 0x7f9d2c0ae450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:12.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.298+0000 7f9d3092d640 1 -- 172.21.15.43:0/210418145 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d2c143bb0 con 0x7f9d2c1f4080 2024-04-25T08:55:12.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.298+0000 7f9d3092d640 1 --2- 172.21.15.43:0/210418145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d2c1f4080 0x7f9d2c1f4500 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9d2402aee0 tx=0x7f9d24009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:12.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.298+0000 7f9d28ff9640 1 -- 172.21.15.43:0/210418145 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d24031d30 con 0x7f9d2c1f4080 2024-04-25T08:55:12.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.298+0000 7f9d28ff9640 1 -- 172.21.15.43:0/210418145 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d24003990 con 0x7f9d2c1f4080 2024-04-25T08:55:12.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.298+0000 7f9d32390640 1 -- 172.21.15.43:0/210418145 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d2c142870 con 0x7f9d2c1f4080 2024-04-25T08:55:12.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.298+0000 7f9d28ff9640 1 -- 172.21.15.43:0/210418145 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d24003d80 con 0x7f9d2c1f4080 2024-04-25T08:55:12.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.298+0000 7f9d32390640 1 -- 172.21.15.43:0/210418145 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d2c142df0 con 0x7f9d2c1f4080 2024-04-25T08:55:12.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.299+0000 7f9d32390640 1 -- 172.21.15.43:0/210418145 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d2c074ef0 con 0x7f9d2c1f4080 2024-04-25T08:55:12.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.300+0000 7f9d28ff9640 1 -- 172.21.15.43:0/210418145 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9d24005d10 con 0x7f9d2c1f4080 2024-04-25T08:55:12.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.300+0000 7f9d28ff9640 1 --2- 172.21.15.43:0/210418145 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9d1806b4d0 0x7f9d1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:12.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.300+0000 7f9d28ff9640 1 -- 172.21.15.43:0/210418145 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9d240ab460 con 0x7f9d2c1f4080 2024-04-25T08:55:12.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.300+0000 7f9d2bfff640 1 --2- 172.21.15.43:0/210418145 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9d1806b4d0 0x7f9d1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:12.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.301+0000 7f9d2bfff640 1 --2- 172.21.15.43:0/210418145 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9d1806b4d0 0x7f9d1806d990 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c001900 tx=0x7f9d1c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:12.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.304+0000 7f9d28ff9640 1 -- 172.21.15.43:0/210418145 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9d2404a060 con 0x7f9d2c1f4080 2024-04-25T08:55:12.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.522+0000 7f9d32390640 1 -- 172.21.15.43:0/210418145 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f9d2c140630 con 0x7f9d2c1f4080 2024-04-25T08:55:12.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.522+0000 7f9d28ff9640 1 -- 172.21.15.43:0/210418145 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v13) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9d2407a370 con 0x7f9d2c1f4080 2024-04-25T08:55:12.512 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:55:12.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.524+0000 7f9d32390640 1 -- 172.21.15.43:0/210418145 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9d1806b4d0 msgr2=0x7f9d1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:12.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.524+0000 7f9d32390640 1 --2- 172.21.15.43:0/210418145 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9d1806b4d0 0x7f9d1806d990 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c001900 tx=0x7f9d1c009040 comp rx=0 tx=0).stop 2024-04-25T08:55:12.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.524+0000 7f9d32390640 1 -- 172.21.15.43:0/210418145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d2c1f4080 msgr2=0x7f9d2c1f4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:12.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.524+0000 7f9d32390640 1 --2- 172.21.15.43:0/210418145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d2c1f4080 0x7f9d2c1f4500 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9d2402aee0 tx=0x7f9d24009750 comp rx=0 tx=0).stop 2024-04-25T08:55:12.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.524+0000 7f9d32390640 1 -- 172.21.15.43:0/210418145 shutdown_connections 2024-04-25T08:55:12.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.524+0000 7f9d32390640 1 --2- 172.21.15.43:0/210418145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d2c1f4080 0x7f9d2c1f4500 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:12.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.524+0000 7f9d32390640 1 --2- 172.21.15.43:0/210418145 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d2c0ae990 0x7f9d2c1f3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:12.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.524+0000 7f9d32390640 1 --2- 172.21.15.43:0/210418145 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9d1806b4d0 0x7f9d1806d990 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:12.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.524+0000 7f9d32390640 1 --2- 172.21.15.43:0/210418145 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d14004d40 0x7f9d2c0ae450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:12.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.524+0000 7f9d32390640 1 -- 172.21.15.43:0/210418145 >> 172.21.15.43:0/210418145 conn(0x7f9d2c0b1480 msgr2=0x7f9d2c0cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:12.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.524+0000 7f9d32390640 1 -- 172.21.15.43:0/210418145 shutdown_connections 2024-04-25T08:55:12.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:12.524+0000 7f9d32390640 1 -- 172.21.15.43:0/210418145 wait complete. 2024-04-25T08:55:12.524 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:55:12.524 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:55:12.525 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-04-25T08:55:12.525 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:55:12.525 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:55:12.525 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:55:12.525 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:55:12.525 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:55:12.525 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:55:12.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:55:12.525 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-04-25T08:55:12.541 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:55:12.541 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:55:12.541 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:55:12.541 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:55:12.542 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-04-25T08:55:12.542 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:55:12.542 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:55:12.542 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-04-25T08:55:12.542 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-04-25T08:55:12.542 INFO:teuthology.orchestra.run.smithi043.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:55:12.542 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:55:12.542 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-04-25T08:55:12.542 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T08:55:12.542 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 42705sec preferred_lft 42705sec 2024-04-25T08:55:12.542 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link noprefixroute 2024-04-25T08:55:12.542 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:55:12.543 INFO:teuthology.orchestra.run.smithi043.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:55:12.543 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-04-25T08:55:12.543 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-04-25T08:55:12.543 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:55:12.543 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:55:12.543 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-04-25T08:55:12.543 INFO:teuthology.orchestra.run.smithi043.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:55:12.543 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 12:2c:7a:72:36:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:55:12.543 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:55:12.543 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:55:12.543 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ccff:4eff:fe9d:c8bf/64 scope link 2024-04-25T08:55:12.543 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:55:12.543 INFO:teuthology.orchestra.run.smithi043.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:55:12.543 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 16:9a:63:37:46:24 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:55:12.543 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::149a:63ff:fe37:4624/64 scope link 2024-04-25T08:55:12.544 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:55:12.544 INFO:teuthology.orchestra.run.smithi043.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:55:12.544 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 12:2c:7a:72:36:14 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:55:12.544 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::102c:7aff:fe72:3614/64 scope link 2024-04-25T08:55:12.544 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:55:12.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:55:12.544 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:55:12.605 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-04-25T08:55:12.606 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:55:12.606 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-04-25T08:55:12.663 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-04-25T08:55:12.664 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:55:12.664 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T08:55:12.664 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T08:55:12.664 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:55:12.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:55:12.664 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2024-04-25T08:55:12.664 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T08:55:12.664 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:55:12.664 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:55:12.665 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:55:12.665 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T08:55:12.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:55:12.751 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-04-25T08:55:12.765 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:55:12.766 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:55:12.766 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:55:12.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:55:12.766 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-04-25T08:55:12.830 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T08:55:12.830 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:55:12.830 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:55:12.886 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:55:12.886 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:55:12.886 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:55:12.941 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-04-25T08:55:13.006 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:55:13.063 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-04-25T08:55:13.063 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-04-25T08:55:13.063 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-04-25T08:55:13.063 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-04-25T08:55:13.063 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-04-25T08:55:13.063 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-04-25T08:55:13.063 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-04-25T08:55:13.063 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-04-25T08:55:13.063 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-04-25T08:55:13.063 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-04-25T08:55:13.063 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-04-25T08:55:13.063 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/selinux 2024-04-25T08:55:13.064 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:55:13.064 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-04-25T08:55:13.064 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-04-25T08:55:13.064 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-04-25T08:55:13.064 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-04-25T08:55:13.064 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:55:13.064 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-04-25T08:55:13.064 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-04-25T08:55:13.064 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:55:13.064 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:55:13.064 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:55:13.064 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:55:13.065 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-04-25T08:55:13.065 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-04-25T08:55:13.065 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:55:13.065 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:55:13.065 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:55:13.065 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:55:13.065 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-04-25T08:55:13.065 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:55:13.065 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:55:13.065 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:55:13.065 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:55:13.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:55:13.066 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-04-25T08:55:13.117 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T08:55:13.117 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T08:55:13.160 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-04-25T08:55:13.184 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:55:13.239 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-04-25T08:55:13.239 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-04-25T08:55:13.239 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-04-25T08:55:13.240 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-04-25T08:55:13.240 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-04-25T08:55:13.240 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-04-25T08:55:13.240 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-04-25T08:55:13.240 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-04-25T08:55:13.240 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-04-25T08:55:13.240 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-04-25T08:55:13.240 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-04-25T08:55:13.240 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/selinux 2024-04-25T08:55:13.240 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:55:13.240 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-04-25T08:55:13.240 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-04-25T08:55:13.241 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-04-25T08:55:13.241 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-04-25T08:55:13.241 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:55:13.241 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-04-25T08:55:13.241 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-04-25T08:55:13.241 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:55:13.241 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:55:13.241 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:55:13.241 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:55:13.241 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-04-25T08:55:13.241 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-04-25T08:55:13.241 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:55:13.241 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:55:13.241 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:55:13.241 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:55:13.242 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-04-25T08:55:13.242 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:55:13.242 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:55:13.242 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:55:13.242 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:55:13.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:55:13.242 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-04-25T08:55:13.243 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:2024-04-25T08:55:13.251+0000 7f2322d5c480 -1 init, newargv = 0x55c4c26f73d0 newargc=15 2024-04-25T08:55:13.243 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[48779]: starting ceph client 2024-04-25T08:55:13.245 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[48779]: starting fuse 2024-04-25T08:55:13.258 INFO:teuthology.orchestra.run.smithi043.stdout:43 2024-04-25T08:55:13.258 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-25T08:55:13.258 DEBUG:teuthology.orchestra.run.smithi043:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:55:13.258 DEBUG:teuthology.orchestra.run.smithi043:> import glob 2024-04-25T08:55:13.258 DEBUG:teuthology.orchestra.run.smithi043:> import re 2024-04-25T08:55:13.258 DEBUG:teuthology.orchestra.run.smithi043:> import os 2024-04-25T08:55:13.258 DEBUG:teuthology.orchestra.run.smithi043:> import subprocess 2024-04-25T08:55:13.258 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T08:55:13.258 DEBUG:teuthology.orchestra.run.smithi043:> def _find_admin_socket(client_name): 2024-04-25T08:55:13.259 DEBUG:teuthology.orchestra.run.smithi043:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T08:55:13.259 DEBUG:teuthology.orchestra.run.smithi043:> files = glob.glob(asok_path) 2024-04-25T08:55:13.259 DEBUG:teuthology.orchestra.run.smithi043:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T08:55:13.259 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T08:55:13.259 DEBUG:teuthology.orchestra.run.smithi043:> # Given a non-glob path, it better be there 2024-04-25T08:55:13.259 DEBUG:teuthology.orchestra.run.smithi043:> if "*" not in asok_path: 2024-04-25T08:55:13.259 DEBUG:teuthology.orchestra.run.smithi043:> assert(len(files) == 1) 2024-04-25T08:55:13.259 DEBUG:teuthology.orchestra.run.smithi043:> return files[0] 2024-04-25T08:55:13.259 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T08:55:13.259 DEBUG:teuthology.orchestra.run.smithi043:> for f in files: 2024-04-25T08:55:13.259 DEBUG:teuthology.orchestra.run.smithi043:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:55:13.259 DEBUG:teuthology.orchestra.run.smithi043:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:55:13.259 DEBUG:teuthology.orchestra.run.smithi043:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:55:13.259 DEBUG:teuthology.orchestra.run.smithi043:> contents = proc_f.read() 2024-04-25T08:55:13.259 DEBUG:teuthology.orchestra.run.smithi043:> if mountpoint in contents: 2024-04-25T08:55:13.260 DEBUG:teuthology.orchestra.run.smithi043:> return f 2024-04-25T08:55:13.260 DEBUG:teuthology.orchestra.run.smithi043:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:55:13.260 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T08:55:13.260 DEBUG:teuthology.orchestra.run.smithi043:> print(_find_admin_socket("client.0")) 2024-04-25T08:55:13.260 DEBUG:teuthology.orchestra.run.smithi043:> ' 2024-04-25T08:55:13.377 INFO:teuthology.orchestra.run.smithi043.stdout:/var/run/ceph/ceph-client.0.48779.asok 2024-04-25T08:55:13.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:55:13.387 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.48779.asok 2024-04-25T08:55:13.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:55:13.387 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.48779.asok status 2024-04-25T08:55:13.524 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T08:55:13.524 INFO:teuthology.orchestra.run.smithi043.stdout: "metadata": { 2024-04-25T08:55:13.524 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:55:13.525 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:55:13.525 INFO:teuthology.orchestra.run.smithi043.stdout: "entity_id": "0", 2024-04-25T08:55:13.525 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": "smithi043", 2024-04-25T08:55:13.525 INFO:teuthology.orchestra.run.smithi043.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T08:55:13.525 INFO:teuthology.orchestra.run.smithi043.stdout: "pid": "48779", 2024-04-25T08:55:13.525 INFO:teuthology.orchestra.run.smithi043.stdout: "root": "/" 2024-04-25T08:55:13.525 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:55:13.525 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_count": 0, 2024-04-25T08:55:13.525 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_pinned_count": 0, 2024-04-25T08:55:13.525 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 5397, 2024-04-25T08:55:13.525 INFO:teuthology.orchestra.run.smithi043.stdout: "inst": { 2024-04-25T08:55:13.525 INFO:teuthology.orchestra.run.smithi043.stdout: "name": { 2024-04-25T08:55:13.525 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "client", 2024-04-25T08:55:13.526 INFO:teuthology.orchestra.run.smithi043.stdout: "num": 5397 2024-04-25T08:55:13.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:55:13.526 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-04-25T08:55:13.526 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-04-25T08:55:13.526 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:55:13.526 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 315578127 2024-04-25T08:55:13.526 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:55:13.526 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:55:13.526 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-04-25T08:55:13.526 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-04-25T08:55:13.526 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:55:13.526 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 315578127 2024-04-25T08:55:13.527 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:55:13.527 INFO:teuthology.orchestra.run.smithi043.stdout: "inst_str": "client.5397 192.168.0.1:0/315578127", 2024-04-25T08:55:13.527 INFO:teuthology.orchestra.run.smithi043.stdout: "addr_str": "192.168.0.1:0/315578127", 2024-04-25T08:55:13.527 INFO:teuthology.orchestra.run.smithi043.stdout: "inode_count": 1, 2024-04-25T08:55:13.527 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_epoch": 13, 2024-04-25T08:55:13.527 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch": 33, 2024-04-25T08:55:13.527 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:55:13.527 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklisted": false, 2024-04-25T08:55:13.527 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs" 2024-04-25T08:55:13.527 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T08:55:13.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:55:13.533 DEBUG:teuthology.orchestra.run.smithi043:> stat --file-system '--printf=%T 2024-04-25T08:55:13.533 DEBUG:teuthology.orchestra.run.smithi043:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T08:55:13.548 INFO:teuthology.orchestra.run.smithi043.stdout:fuseblk 2024-04-25T08:55:13.549 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T08:55:13.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:55:13.549 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:55:13.614 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2024-04-25T08:55:13.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.769+0000 7fd5b90cd640 1 Processor -- start 2024-04-25T08:55:13.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.769+0000 7fd5b90cd640 1 -- start start 2024-04-25T08:55:13.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.770+0000 7fd5b90cd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd5b40b16d0 0x7fd5b4168ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:13.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.770+0000 7fd5b90cd640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5b415e260 con 0x7fd5b415de80 2024-04-25T08:55:13.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.770+0000 7fd5b90cd640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5b40b52a0 con 0x7fd5b40d3760 2024-04-25T08:55:13.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.770+0000 7fd5b90cd640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5b4161970 con 0x7fd5b40b16d0 2024-04-25T08:55:13.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.770+0000 7fd5b2d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fd5b415de80 0x7fd5b40b4b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:33004/0 (socket says 172.21.15.43:33004) 2024-04-25T08:55:13.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.770+0000 7fd5b2d9d640 1 -- 172.21.15.43:0/2804907503 learned_addr learned my addr 172.21.15.43:0/2804907503 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:55:13.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.770+0000 7fd5b359e640 1 --2- 172.21.15.43:0/2804907503 >> v2:172.21.15.43:3300/0 conn(0x7fd5b40b16d0 0x7fd5b4168ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:13.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.770+0000 7fd5b359e640 1 -- 172.21.15.43:0/2804907503 >> v1:172.21.15.193:6789/0 conn(0x7fd5b40d3760 legacy=0x7fd5b40d3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:55:13.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.770+0000 7fd5b359e640 1 -- 172.21.15.43:0/2804907503 >> v1:172.21.15.43:6789/0 conn(0x7fd5b415de80 legacy=0x7fd5b40b4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:13.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.770+0000 7fd5b359e640 1 -- 172.21.15.43:0/2804907503 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b4161c50 con 0x7fd5b40b16d0 2024-04-25T08:55:13.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.770+0000 7fd5b259c640 1 -- 172.21.15.43:0/2804907503 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3519154814 0 0) 0x7fd5b415e260 con 0x7fd5b415de80 2024-04-25T08:55:13.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.771+0000 7fd5b359e640 1 --2- 172.21.15.43:0/2804907503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5b40b16d0 0x7fd5b4168ca0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac009dc0 tx=0x7fd5ac02a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f897818b6319a5f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:13.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.771+0000 7fd5b259c640 1 -- 172.21.15.43:0/2804907503 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac007740 con 0x7fd5b40b16d0 2024-04-25T08:55:13.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.771+0000 7fd5b259c640 1 -- 172.21.15.43:0/2804907503 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5ac0078e0 con 0x7fd5b40b16d0 2024-04-25T08:55:13.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.771+0000 7fd5b259c640 1 -- 172.21.15.43:0/2804907503 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac007cb0 con 0x7fd5b40b16d0 2024-04-25T08:55:13.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.771+0000 7fd5b90cd640 1 -- 172.21.15.43:0/2804907503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5b40b16d0 msgr2=0x7fd5b4168ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:13.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.771+0000 7fd5b90cd640 1 --2- 172.21.15.43:0/2804907503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5b40b16d0 0x7fd5b4168ca0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac009dc0 tx=0x7fd5ac02a640 comp rx=0 tx=0).stop 2024-04-25T08:55:13.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.771+0000 7fd5b90cd640 1 -- 172.21.15.43:0/2804907503 shutdown_connections 2024-04-25T08:55:13.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.772+0000 7fd5b90cd640 1 --2- 172.21.15.43:0/2804907503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5b40b16d0 0x7fd5b4168ca0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:13.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.772+0000 7fd5b90cd640 1 -- 172.21.15.43:0/2804907503 >> 172.21.15.43:0/2804907503 conn(0x7fd5b408d460 msgr2=0x7fd5b409cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:13.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.772+0000 7fd5b90cd640 1 -- 172.21.15.43:0/2804907503 shutdown_connections 2024-04-25T08:55:13.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.772+0000 7fd5b90cd640 1 -- 172.21.15.43:0/2804907503 wait complete. 2024-04-25T08:55:13.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.773+0000 7fd5b90cd640 1 Processor -- start 2024-04-25T08:55:13.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.773+0000 7fd5b90cd640 1 -- start start 2024-04-25T08:55:13.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.774+0000 7fd5b90cd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5b40b16d0 0x7fd5b415fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:13.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.774+0000 7fd5b90cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5b40d3760 0x7fd5b4153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:13.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.774+0000 7fd5b90cd640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd5b415de80 0x7fd5b40cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:13.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.774+0000 7fd5b90cd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b41624f0 con 0x7fd5b40d3760 2024-04-25T08:55:13.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.774+0000 7fd5b90cd640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b4162210 con 0x7fd5b40b16d0 2024-04-25T08:55:13.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.774+0000 7fd5b90cd640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b40b3010 con 0x7fd5b415de80 2024-04-25T08:55:13.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.774+0000 7fd5abfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5b40d3760 0x7fd5b4153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:13.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.774+0000 7fd5abfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5b40d3760 0x7fd5b4153fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38396/0 (socket says 172.21.15.43:38396) 2024-04-25T08:55:13.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.774+0000 7fd5abfff640 1 -- 172.21.15.43:0/2086654118 learned_addr learned my addr 172.21.15.43:0/2086654118 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:55:13.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.774+0000 7fd5b359e640 1 --2- 172.21.15.43:0/2086654118 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd5b415de80 0x7fd5b40cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:13.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.774+0000 7fd5b2d9d640 1 --2- 172.21.15.43:0/2086654118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5b40b16d0 0x7fd5b415fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:13.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.774+0000 7fd5abfff640 1 -- 172.21.15.43:0/2086654118 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd5b415de80 msgr2=0x7fd5b40cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:13.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.774+0000 7fd5abfff640 1 --2- 172.21.15.43:0/2086654118 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd5b415de80 0x7fd5b40cc9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:13.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.774+0000 7fd5abfff640 1 -- 172.21.15.43:0/2086654118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5b40b16d0 msgr2=0x7fd5b415fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:13.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.774+0000 7fd5abfff640 1 --2- 172.21.15.43:0/2086654118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5b40b16d0 0x7fd5b415fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:13.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.774+0000 7fd5abfff640 1 -- 172.21.15.43:0/2086654118 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b41544e0 con 0x7fd5b40d3760 2024-04-25T08:55:13.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.774+0000 7fd5b2d9d640 1 --2- 172.21.15.43:0/2086654118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5b40b16d0 0x7fd5b415fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:55:13.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.774+0000 7fd5abfff640 1 --2- 172.21.15.43:0/2086654118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5b40d3760 0x7fd5b4153fa0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd59c00c430 tx=0x7fd59c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:13.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.774+0000 7fd5ab7fe640 1 -- 172.21.15.43:0/2086654118 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59c011790 con 0x7fd5b40d3760 2024-04-25T08:55:13.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.774+0000 7fd5ab7fe640 1 -- 172.21.15.43:0/2086654118 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd59c017070 con 0x7fd5b40d3760 2024-04-25T08:55:13.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.774+0000 7fd5b90cd640 1 -- 172.21.15.43:0/2086654118 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5b40cd280 con 0x7fd5b40d3760 2024-04-25T08:55:13.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.774+0000 7fd5ab7fe640 1 -- 172.21.15.43:0/2086654118 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59c011b70 con 0x7fd5b40d3760 2024-04-25T08:55:13.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.775+0000 7fd5b90cd640 1 -- 172.21.15.43:0/2086654118 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5b40cd840 con 0x7fd5b40d3760 2024-04-25T08:55:13.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.776+0000 7fd5b90cd640 1 -- 172.21.15.43:0/2086654118 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5b40cda80 con 0x7fd5b40d3760 2024-04-25T08:55:13.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.776+0000 7fd5ab7fe640 1 -- 172.21.15.43:0/2086654118 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd59c011d10 con 0x7fd5b40d3760 2024-04-25T08:55:13.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.776+0000 7fd5ab7fe640 1 --2- 172.21.15.43:0/2086654118 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd59406b500 0x7fd59406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:55:13.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.777+0000 7fd5ab7fe640 1 -- 172.21.15.43:0/2086654118 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd59c089f70 con 0x7fd5b40d3760 2024-04-25T08:55:13.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.777+0000 7fd5b2d9d640 1 --2- 172.21.15.43:0/2086654118 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd59406b500 0x7fd59406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:55:13.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.777+0000 7fd5b2d9d640 1 --2- 172.21.15.43:0/2086654118 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd59406b500 0x7fd59406d9c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4000f60 tx=0x7fd5a4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:55:13.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:13.781+0000 7fd5ab7fe640 1 -- 172.21.15.43:0/2086654118 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd5b40cda80 con 0x7fd5b40d3760 2024-04-25T08:55:13.836 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:55:13.839 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:55:13.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:14.000+0000 7fd5b90cd640 1 -- 172.21.15.43:0/2086654118 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd5b4068a40 con 0x7fd5b40d3760 2024-04-25T08:55:14.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:14.549+0000 7fd5ab7fe640 1 -- 172.21.15.43:0/2086654118 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v14) v1 ==== 167+0+0 (secure 0 0 0) 0x7fd5b41544e0 con 0x7fd5b40d3760 2024-04-25T08:55:14.539 INFO:teuthology.orchestra.run.smithi043.stderr:enabled new snapshots 2024-04-25T08:55:14.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:14.553+0000 7fd5b90cd640 1 -- 172.21.15.43:0/2086654118 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd59406b500 msgr2=0x7fd59406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:14.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:14.553+0000 7fd5b90cd640 1 --2- 172.21.15.43:0/2086654118 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd59406b500 0x7fd59406d9c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4000f60 tx=0x7fd5a4009040 comp rx=0 tx=0).stop 2024-04-25T08:55:14.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:14.553+0000 7fd5b90cd640 1 -- 172.21.15.43:0/2086654118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5b40d3760 msgr2=0x7fd5b4153fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:55:14.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:14.553+0000 7fd5b90cd640 1 --2- 172.21.15.43:0/2086654118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5b40d3760 0x7fd5b4153fa0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd59c00c430 tx=0x7fd59c00c8f0 comp rx=0 tx=0).stop 2024-04-25T08:55:14.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:14.553+0000 7fd5b90cd640 1 -- 172.21.15.43:0/2086654118 shutdown_connections 2024-04-25T08:55:14.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:14.553+0000 7fd5b90cd640 1 --2- 172.21.15.43:0/2086654118 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd5b415de80 0x7fd5b40cc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:14.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:14.553+0000 7fd5b90cd640 1 --2- 172.21.15.43:0/2086654118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd5b40d3760 0x7fd5b4153fa0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:14.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:14.553+0000 7fd5b90cd640 1 --2- 172.21.15.43:0/2086654118 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd59406b500 0x7fd59406d9c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:14.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:14.553+0000 7fd5b90cd640 1 --2- 172.21.15.43:0/2086654118 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd5b40b16d0 0x7fd5b415fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:55:14.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:14.553+0000 7fd5b90cd640 1 -- 172.21.15.43:0/2086654118 >> 172.21.15.43:0/2086654118 conn(0x7fd5b408d460 msgr2=0x7fd5b40b5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:55:14.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:14.553+0000 7fd5b90cd640 1 -- 172.21.15.43:0/2086654118 shutdown_connections 2024-04-25T08:55:14.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:14.553+0000 7fd5b90cd640 1 -- 172.21.15.43:0/2086654118 wait complete. 2024-04-25T08:55:14.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:55:14.556 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir stray_eval_dir) 2024-04-25T08:55:14.579 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:55:14.579 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:55:14.637 DEBUG:teuthology.orchestra.run.smithi043:> stdin-killer -- python3 -c ' 2024-04-25T08:55:14.637 DEBUG:teuthology.orchestra.run.smithi043:> import os 2024-04-25T08:55:14.638 DEBUG:teuthology.orchestra.run.smithi043:> import uuid 2024-04-25T08:55:14.638 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T08:55:14.638 DEBUG:teuthology.orchestra.run.smithi043:> n = 2000 2024-04-25T08:55:14.638 DEBUG:teuthology.orchestra.run.smithi043:> create_hard_links = False 2024-04-25T08:55:14.638 DEBUG:teuthology.orchestra.run.smithi043:> if 3 > 0: 2024-04-25T08:55:14.638 DEBUG:teuthology.orchestra.run.smithi043:> create_hard_links = True 2024-04-25T08:55:14.638 DEBUG:teuthology.orchestra.run.smithi043:> path = "/home/ubuntu/cephtest/mnt.0/stray_eval_dir/file" 2024-04-25T08:55:14.638 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T08:55:14.638 DEBUG:teuthology.orchestra.run.smithi043:> dpath = os.path.dirname(path) 2024-04-25T08:55:14.638 DEBUG:teuthology.orchestra.run.smithi043:> fnameprefix = os.path.basename(path) 2024-04-25T08:55:14.638 DEBUG:teuthology.orchestra.run.smithi043:> os.makedirs(dpath, exist_ok=True) 2024-04-25T08:55:14.638 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T08:55:14.638 DEBUG:teuthology.orchestra.run.smithi043:> try: 2024-04-25T08:55:14.638 DEBUG:teuthology.orchestra.run.smithi043:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-25T08:55:14.638 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T08:55:14.639 DEBUG:teuthology.orchestra.run.smithi043:> for i in range(n): 2024-04-25T08:55:14.639 DEBUG:teuthology.orchestra.run.smithi043:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-25T08:55:14.639 DEBUG:teuthology.orchestra.run.smithi043:> with open(fpath, '"'"'w'"'"') as f: 2024-04-25T08:55:14.639 DEBUG:teuthology.orchestra.run.smithi043:> f.write(f"{i}") 2024-04-25T08:55:14.639 DEBUG:teuthology.orchestra.run.smithi043:> if False: 2024-04-25T08:55:14.639 DEBUG:teuthology.orchestra.run.smithi043:> f.flush() 2024-04-25T08:55:14.639 DEBUG:teuthology.orchestra.run.smithi043:> os.fsync(f.fileno()) 2024-04-25T08:55:14.639 DEBUG:teuthology.orchestra.run.smithi043:> if False: 2024-04-25T08:55:14.639 DEBUG:teuthology.orchestra.run.smithi043:> os.unlink(fpath) 2024-04-25T08:55:14.639 DEBUG:teuthology.orchestra.run.smithi043:> if False: 2024-04-25T08:55:14.639 DEBUG:teuthology.orchestra.run.smithi043:> os.fsync(dirfd) 2024-04-25T08:55:14.639 DEBUG:teuthology.orchestra.run.smithi043:> if create_hard_links: 2024-04-25T08:55:14.639 DEBUG:teuthology.orchestra.run.smithi043:> for j in range(3): 2024-04-25T08:55:14.639 DEBUG:teuthology.orchestra.run.smithi043:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-25T08:55:14.639 DEBUG:teuthology.orchestra.run.smithi043:> if False: 2024-04-25T08:55:14.639 DEBUG:teuthology.orchestra.run.smithi043:> os.fsync(dirfd) 2024-04-25T08:55:14.640 DEBUG:teuthology.orchestra.run.smithi043:> finally: 2024-04-25T08:55:14.640 DEBUG:teuthology.orchestra.run.smithi043:> os.close(dirfd) 2024-04-25T08:55:14.640 DEBUG:teuthology.orchestra.run.smithi043:> ' 2024-04-25T08:55:43.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:55:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:55:43.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:55:43.175 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T08:55:43.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:55:43.268 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec find /home/ubuntu/cephtest/mnt.0/stray_eval_dir/ -type f -delete) 2024-04-25T08:55:43.984 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:55:43.986 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:56:13.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:56:13.294 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T08:56:13.337 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:56:13.337 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 perf dump 2024-04-25T08:56:13.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.490+0000 7f0929888640 1 Processor -- start 2024-04-25T08:56:13.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.490+0000 7f0929888640 1 -- start start 2024-04-25T08:56:13.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.491+0000 7f0929888640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f09240d1ec0 0x7f09240cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.491+0000 7f0929888640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f09240b0cc0 0x7f09240b1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.491+0000 7f0929888640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f092415f170 con 0x7f092415dd80 2024-04-25T08:56:13.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.491+0000 7f0929888640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09240cfc60 con 0x7f09240b0cc0 2024-04-25T08:56:13.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.491+0000 7f0929888640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09240b32c0 con 0x7f09240d1ec0 2024-04-25T08:56:13.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.491+0000 7f09237fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f09240b0cc0 0x7f09240b1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.491+0000 7f0922ffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f092415dd80 0x7f09240d17b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:46216/0 (socket says 172.21.15.43:46216) 2024-04-25T08:56:13.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.491+0000 7f0922ffd640 1 -- 172.21.15.43:0/3982118374 learned_addr learned my addr 172.21.15.43:0/3982118374 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:56:13.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.492+0000 7f09237fe640 1 -- 172.21.15.43:0/3982118374 >> v2:172.21.15.193:3300/0 conn(0x7f09240d1ec0 msgr2=0x7f09240cf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:13.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.492+0000 7f09237fe640 1 --2- 172.21.15.43:0/3982118374 >> v2:172.21.15.193:3300/0 conn(0x7f09240d1ec0 0x7f09240cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.492+0000 7f09237fe640 1 -- 172.21.15.43:0/3982118374 >> v1:172.21.15.43:6789/0 conn(0x7f092415dd80 legacy=0x7f09240d17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:13.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.492+0000 7f09237fe640 1 -- 172.21.15.43:0/3982118374 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09240b2fe0 con 0x7f09240b0cc0 2024-04-25T08:56:13.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.492+0000 7f0921ffb640 1 -- 172.21.15.43:0/3982118374 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 497433095 0 0) 0x7f092415f170 con 0x7f092415dd80 2024-04-25T08:56:13.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.492+0000 7f09237fe640 1 --2- 172.21.15.43:0/3982118374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09240b0cc0 0x7f09240b1130 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0924069280 tx=0x7f091402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=813c83155293f32d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:13.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.493+0000 7f0921ffb640 1 -- 172.21.15.43:0/3982118374 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0914007740 con 0x7f09240b0cc0 2024-04-25T08:56:13.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.493+0000 7f0921ffb640 1 -- 172.21.15.43:0/3982118374 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09140078e0 con 0x7f09240b0cc0 2024-04-25T08:56:13.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.493+0000 7f0921ffb640 1 -- 172.21.15.43:0/3982118374 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0914007cb0 con 0x7f09240b0cc0 2024-04-25T08:56:13.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.493+0000 7f0929888640 1 -- 172.21.15.43:0/3982118374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09240b0cc0 msgr2=0x7f09240b1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:13.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.493+0000 7f0929888640 1 --2- 172.21.15.43:0/3982118374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09240b0cc0 0x7f09240b1130 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0924069280 tx=0x7f091402a5f0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.493+0000 7f0929888640 1 -- 172.21.15.43:0/3982118374 shutdown_connections 2024-04-25T08:56:13.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.493+0000 7f0929888640 1 --2- 172.21.15.43:0/3982118374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09240b0cc0 0x7f09240b1130 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.493+0000 7f0929888640 1 --2- 172.21.15.43:0/3982118374 >> v2:172.21.15.193:3300/0 conn(0x7f09240d1ec0 0x7f09240cf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.493+0000 7f0929888640 1 -- 172.21.15.43:0/3982118374 >> 172.21.15.43:0/3982118374 conn(0x7f0924072aa0 msgr2=0x7f092409aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:13.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.493+0000 7f0929888640 1 -- 172.21.15.43:0/3982118374 shutdown_connections 2024-04-25T08:56:13.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.493+0000 7f0929888640 1 -- 172.21.15.43:0/3982118374 wait complete. 2024-04-25T08:56:13.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.494+0000 7f0929888640 1 Processor -- start 2024-04-25T08:56:13.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.495+0000 7f0929888640 1 -- start start 2024-04-25T08:56:13.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.495+0000 7f0929888640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09240b0cc0 0x7f092415b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.495+0000 7f0929888640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f09240d1ec0 0x7f0924155470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.495+0000 7f0929888640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f092415dd80 0x7f09241f3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.495+0000 7f0929888640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09240cf330 con 0x7f092415dd80 2024-04-25T08:56:13.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.495+0000 7f0929888640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09240cf050 con 0x7f09240b0cc0 2024-04-25T08:56:13.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.495+0000 7f0929888640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09240b2c40 con 0x7f09240d1ec0 2024-04-25T08:56:13.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.495+0000 7f09237fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f092415dd80 0x7f09241f3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.496+0000 7f09237fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f092415dd80 0x7f09241f3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39764/0 (socket says 172.21.15.43:39764) 2024-04-25T08:56:13.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.496+0000 7f09237fe640 1 -- 172.21.15.43:0/3732790147 learned_addr learned my addr 172.21.15.43:0/3732790147 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:13.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.496+0000 7f0922ffd640 1 --2- 172.21.15.43:0/3732790147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09240b0cc0 0x7f092415b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.496+0000 7f09227fc640 1 --2- 172.21.15.43:0/3732790147 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f09240d1ec0 0x7f0924155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.496+0000 7f09237fe640 1 -- 172.21.15.43:0/3732790147 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f09240d1ec0 msgr2=0x7f0924155470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:13.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.496+0000 7f09237fe640 1 --2- 172.21.15.43:0/3732790147 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f09240d1ec0 0x7f0924155470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.496+0000 7f09237fe640 1 -- 172.21.15.43:0/3732790147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09240b0cc0 msgr2=0x7f092415b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:13.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.496+0000 7f09237fe640 1 --2- 172.21.15.43:0/3732790147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09240b0cc0 0x7f092415b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.496+0000 7f09237fe640 1 -- 172.21.15.43:0/3732790147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092415fd70 con 0x7f092415dd80 2024-04-25T08:56:13.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.496+0000 7f0922ffd640 1 --2- 172.21.15.43:0/3732790147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09240b0cc0 0x7f092415b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:56:13.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.496+0000 7f09237fe640 1 --2- 172.21.15.43:0/3732790147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f092415dd80 0x7f09241f3530 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f091402abe0 tx=0x7f0914004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:13.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.496+0000 7f091bfff640 1 -- 172.21.15.43:0/3732790147 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0914040070 con 0x7f092415dd80 2024-04-25T08:56:13.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.496+0000 7f091bfff640 1 -- 172.21.15.43:0/3732790147 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0914031c90 con 0x7f092415dd80 2024-04-25T08:56:13.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.496+0000 7f0929888640 1 -- 172.21.15.43:0/3732790147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0924160050 con 0x7f092415dd80 2024-04-25T08:56:13.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.496+0000 7f091bfff640 1 -- 172.21.15.43:0/3732790147 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0914045680 con 0x7f092415dd80 2024-04-25T08:56:13.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.496+0000 7f0929888640 1 -- 172.21.15.43:0/3732790147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0924160610 con 0x7f092415dd80 2024-04-25T08:56:13.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.498+0000 7f091bfff640 1 -- 172.21.15.43:0/3732790147 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0914039040 con 0x7f092415dd80 2024-04-25T08:56:13.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.498+0000 7f091bfff640 1 --2- 172.21.15.43:0/3732790147 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f090406b4d0 0x7f090406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.498+0000 7f091bfff640 1 -- 172.21.15.43:0/3732790147 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f09140abbd0 con 0x7f092415dd80 2024-04-25T08:56:13.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.498+0000 7f0922ffd640 1 --2- 172.21.15.43:0/3732790147 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f090406b4d0 0x7f090406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.499+0000 7f0922ffd640 1 --2- 172.21.15.43:0/3732790147 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f090406b4d0 0x7f090406d990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f091c000f60 tx=0x7f091c007ca0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:13.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.499+0000 7f092c587740 1 Processor -- start 2024-04-25T08:56:13.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.499+0000 7f092c587740 1 -- start start 2024-04-25T08:56:13.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.500+0000 7f092c587740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556a9001aca0 0x556a9001ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.500+0000 7f092c587740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a90057370 0x556a9001b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.500+0000 7f092c587740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a90045760 0x556a90045be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.500+0000 7f092c587740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a8ff7db90 con 0x556a90045760 2024-04-25T08:56:13.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.500+0000 7f092c587740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a8ff26e40 con 0x556a90057370 2024-04-25T08:56:13.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.500+0000 7f092c587740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x556a8ff79c30 con 0x556a9001aca0 2024-04-25T08:56:13.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.500+0000 7f09237fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a90045760 0x556a90045be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.500+0000 7f09237fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a90045760 0x556a90045be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39770/0 (socket says 172.21.15.43:39770) 2024-04-25T08:56:13.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.500+0000 7f09237fe640 1 -- 172.21.15.43:0/3681060242 learned_addr learned my addr 172.21.15.43:0/3681060242 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:13.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.500+0000 7f09227fc640 1 --2- 172.21.15.43:0/3681060242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a90057370 0x556a9001b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.500+0000 7f0922ffd640 1 --2- 172.21.15.43:0/3681060242 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556a9001aca0 0x556a9001ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.500+0000 7f09237fe640 1 -- 172.21.15.43:0/3681060242 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556a9001aca0 msgr2=0x556a9001ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:13.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.500+0000 7f09237fe640 1 --2- 172.21.15.43:0/3681060242 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556a9001aca0 0x556a9001ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.500+0000 7f09237fe640 1 -- 172.21.15.43:0/3681060242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a90057370 msgr2=0x556a9001b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:13.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.500+0000 7f09237fe640 1 --2- 172.21.15.43:0/3681060242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a90057370 0x556a9001b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.500+0000 7f09237fe640 1 -- 172.21.15.43:0/3681060242 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a8fd5c7d0 con 0x556a90045760 2024-04-25T08:56:13.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.501+0000 7f09237fe640 1 --2- 172.21.15.43:0/3681060242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a90045760 0x556a90045be0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f09140aa0d0 tx=0x7f091407a3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f37f900aad59f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:13.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.501+0000 7f09197fa640 1 -- 172.21.15.43:0/3681060242 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0914093020 con 0x556a90045760 2024-04-25T08:56:13.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.501+0000 7f09197fa640 1 -- 172.21.15.43:0/3681060242 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f091407a4a0 con 0x556a90045760 2024-04-25T08:56:13.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.501+0000 7f09197fa640 1 -- 172.21.15.43:0/3681060242 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09140a7020 con 0x556a90045760 2024-04-25T08:56:13.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.501+0000 7f092c587740 1 -- 172.21.15.43:0/3681060242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a90045760 msgr2=0x556a90045be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:13.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.501+0000 7f092c587740 1 --2- 172.21.15.43:0/3681060242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a90045760 0x556a90045be0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f09140aa0d0 tx=0x7f091407a3a0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.501+0000 7f092c587740 1 -- 172.21.15.43:0/3681060242 shutdown_connections 2024-04-25T08:56:13.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.501+0000 7f092c587740 1 --2- 172.21.15.43:0/3681060242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a90045760 0x556a90045be0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.501+0000 7f092c587740 1 --2- 172.21.15.43:0/3681060242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a90057370 0x556a9001b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.501+0000 7f092c587740 1 --2- 172.21.15.43:0/3681060242 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556a9001aca0 0x556a9001ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.501+0000 7f092c587740 1 -- 172.21.15.43:0/3681060242 >> 172.21.15.43:0/3681060242 conn(0x556a8ffe2870 msgr2=0x556a90060760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:13.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.502+0000 7f092c587740 1 -- 172.21.15.43:0/3681060242 shutdown_connections 2024-04-25T08:56:13.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.502+0000 7f092c587740 1 -- 172.21.15.43:0/3681060242 wait complete. 2024-04-25T08:56:13.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.502+0000 7f092c587740 1 -- start start 2024-04-25T08:56:13.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.502+0000 7f092c587740 1 Processor -- start 2024-04-25T08:56:13.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.503+0000 7f092c587740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556a9001aca0 0x556a90033790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.503+0000 7f092c587740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a90045760 0x556a90032fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.503+0000 7f092c587740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a90057370 0x556a90048c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.503+0000 7f092c587740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a8ff7db90 con 0x556a90057370 2024-04-25T08:56:13.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.503+0000 7f092c587740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a8ff3a3e0 con 0x556a90045760 2024-04-25T08:56:13.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.503+0000 7f092c587740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x556a90033510 con 0x556a9001aca0 2024-04-25T08:56:13.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.503+0000 7f092c587740 10 client.0 _ll_register_callbacks cb 0x556a8ff39db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:56:13.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.503+0000 7f09237fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a90057370 0x556a90048c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.503+0000 7f09237fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a90057370 0x556a90048c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39776/0 (socket says 172.21.15.43:39776) 2024-04-25T08:56:13.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.503+0000 7f09237fe640 1 -- 172.21.15.43:0/281093608 learned_addr learned my addr 172.21.15.43:0/281093608 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:13.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.503+0000 7f0922ffd640 1 --2- 172.21.15.43:0/281093608 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556a9001aca0 0x556a90033790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.503+0000 7f09227fc640 1 --2- 172.21.15.43:0/281093608 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a90045760 0x556a90032fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.503+0000 7f09237fe640 1 -- 172.21.15.43:0/281093608 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556a9001aca0 msgr2=0x556a90033790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:13.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.503+0000 7f09237fe640 1 --2- 172.21.15.43:0/281093608 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556a9001aca0 0x556a90033790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.503+0000 7f09237fe640 1 -- 172.21.15.43:0/281093608 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a90045760 msgr2=0x556a90032fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:13.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.503+0000 7f09237fe640 1 --2- 172.21.15.43:0/281093608 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a90045760 0x556a90032fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.503+0000 7f09237fe640 1 -- 172.21.15.43:0/281093608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a8fc520f0 con 0x556a90057370 2024-04-25T08:56:13.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.504+0000 7f09237fe640 1 --2- 172.21.15.43:0/281093608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a90057370 0x556a90048c30 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f091407a1a0 tx=0x7f0914007fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d5c34a84b9b2aa1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:13.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.504+0000 7f09027fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:56:13.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.504+0000 7f09027fc640 1 -- 172.21.15.43:0/281093608 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09140a1060 con 0x556a90057370 2024-04-25T08:56:13.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.504+0000 7f09027fc640 1 -- 172.21.15.43:0/281093608 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f091407e170 con 0x556a90057370 2024-04-25T08:56:13.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.504+0000 7f092c587740 1 -- 172.21.15.43:0/281093608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f091407a4a0 con 0x556a90057370 2024-04-25T08:56:13.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.504+0000 7f09027fc640 1 -- 172.21.15.43:0/281093608 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0914093040 con 0x556a90057370 2024-04-25T08:56:13.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.504+0000 7f092c587740 10 client.5421 fetch_fsmap learned FSMap version 14 2024-04-25T08:56:13.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.504+0000 7f092c587740 1 -- 172.21.15.43:0/281093608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x556a8fdbcf60 con 0x556a90057370 2024-04-25T08:56:13.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.504+0000 7f09027fc640 1 -- 172.21.15.43:0/281093608 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f091407a4a0 con 0x556a90057370 2024-04-25T08:56:13.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.504+0000 7f09027fc640 1 -- 172.21.15.43:0/281093608 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 14) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f091407a700 con 0x556a90057370 2024-04-25T08:56:13.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.504+0000 7f092c587740 10 client.5421 fetch_fsmap finished waiting for FSMap version 14 2024-04-25T08:56:13.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.504+0000 7f092c587740 10 client.5421 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T08:56:13.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.504+0000 7f092c587740 20 client.5421 populate_metadata read hostname 'smithi043' 2024-04-25T08:56:13.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.504+0000 7f092c587740 1 --2- 172.21.15.43:0/281093608 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x556a90047290 0x556a9005bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.504+0000 7f092c587740 4 client.5421 mds_command: new command op to 5208 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:56:13.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.504+0000 7f092c587740 1 -- 172.21.15.43:0/281093608 --> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556a8ff8b0d0 con 0x556a90047290 2024-04-25T08:56:13.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.504+0000 7f0922ffd640 1 --2- 172.21.15.43:0/281093608 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x556a90047290 0x556a9005bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.505+0000 7f0922ffd640 1 --2- 172.21.15.43:0/281093608 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x556a90047290 0x556a9005bca0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2832bf9396b487ab server_cookie=5a95e891e55953b8 in_seq=0 out_seq=0 2024-04-25T08:56:13.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.505+0000 7f09027fc640 10 client.5421 ms_handle_connect on v2:172.21.15.43:6833/242762542 2024-04-25T08:56:13.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.505+0000 7f09027fc640 1 -- 172.21.15.43:0/281093608 <== mds.0 v2:172.21.15.43:6833/242762542 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556a8ff8b0d0 con 0x556a90047290 2024-04-25T08:56:13.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.505+0000 7f09027fc640 10 client.5421 handle_command_reply: tid=0 2024-04-25T08:56:13.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.505+0000 7f092c587740 1 client.5421 shutdown 2024-04-25T08:56:13.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.505+0000 7f092c587740 1 -- 172.21.15.43:0/281093608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a90057370 msgr2=0x556a90048c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:13.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.505+0000 7f092c587740 1 --2- 172.21.15.43:0/281093608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a90057370 0x556a90048c30 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f091407a1a0 tx=0x7f0914007fc0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.505+0000 7f092c587740 1 -- 172.21.15.43:0/281093608 shutdown_connections 2024-04-25T08:56:13.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.505+0000 7f092c587740 1 --2- 172.21.15.43:0/281093608 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x556a90047290 0x556a9005bca0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.505+0000 7f092c587740 1 --2- 172.21.15.43:0/281093608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a90057370 0x556a90048c30 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.505+0000 7f092c587740 1 --2- 172.21.15.43:0/281093608 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a90045760 0x556a90032fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.505+0000 7f09027fc640 0 client.5421 ms_handle_reset on v2:172.21.15.43:6833/242762542 2024-04-25T08:56:13.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.505+0000 7f092c587740 1 --2- 172.21.15.43:0/281093608 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556a9001aca0 0x556a90033790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.505+0000 7f092c587740 1 -- 172.21.15.43:0/281093608 >> 172.21.15.43:0/281093608 conn(0x556a8ffe2870 msgr2=0x556a90063c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:13.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.505+0000 7f092c587740 1 -- 172.21.15.43:0/281093608 shutdown_connections 2024-04-25T08:56:13.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.505+0000 7f092c587740 1 -- 172.21.15.43:0/281093608 wait complete. 2024-04-25T08:56:13.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.506+0000 7f092c587740 20 client.5421 trim_cache size 0 max 16384 2024-04-25T08:56:13.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.520+0000 7f092c587740 1 Processor -- start 2024-04-25T08:56:13.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.520+0000 7f092c587740 1 -- start start 2024-04-25T08:56:13.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.521+0000 7f092c587740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556a9001aca0 0x556a90048c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.521+0000 7f092c587740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a90045760 0x556a9007a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.521+0000 7f092c587740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a90057370 0x556a90059760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.521+0000 7f092c587740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09241ffc60 con 0x556a90045760 2024-04-25T08:56:13.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.521+0000 7f092c587740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09241fee50 con 0x556a90057370 2024-04-25T08:56:13.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.521+0000 7f092c587740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09240b5060 con 0x556a9001aca0 2024-04-25T08:56:13.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.521+0000 7f09227fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a90045760 0x556a9007a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.521+0000 7f09227fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a90045760 0x556a9007a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39792/0 (socket says 172.21.15.43:39792) 2024-04-25T08:56:13.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.521+0000 7f09227fc640 1 -- 172.21.15.43:0/1420195063 learned_addr learned my addr 172.21.15.43:0/1420195063 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:13.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.521+0000 7f0922ffd640 1 --2- 172.21.15.43:0/1420195063 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556a9001aca0 0x556a90048c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.521+0000 7f09237fe640 1 --2- 172.21.15.43:0/1420195063 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a90057370 0x556a90059760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.521+0000 7f09227fc640 1 -- 172.21.15.43:0/1420195063 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556a9001aca0 msgr2=0x556a90048c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:13.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.521+0000 7f09227fc640 1 --2- 172.21.15.43:0/1420195063 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556a9001aca0 0x556a90048c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.521+0000 7f09227fc640 1 -- 172.21.15.43:0/1420195063 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a90057370 msgr2=0x556a90059760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:13.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.521+0000 7f09227fc640 1 --2- 172.21.15.43:0/1420195063 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a90057370 0x556a90059760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.521+0000 7f09227fc640 1 -- 172.21.15.43:0/1420195063 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556a8fc2b1b0 con 0x556a90045760 2024-04-25T08:56:13.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.521+0000 7f0922ffd640 1 --2- 172.21.15.43:0/1420195063 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556a9001aca0 0x556a90048c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:13.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.521+0000 7f09237fe640 1 --2- 172.21.15.43:0/1420195063 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a90057370 0x556a90059760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:13.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.522+0000 7f09227fc640 1 --2- 172.21.15.43:0/1420195063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a90045760 0x556a9007a550 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f090c00b8a0 tx=0x7f090c00bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef9ae5bc8659a5e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:13.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.522+0000 7f0918ff9640 1 -- 172.21.15.43:0/1420195063 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090c013ab0 con 0x556a90045760 2024-04-25T08:56:13.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.522+0000 7f0918ff9640 1 -- 172.21.15.43:0/1420195063 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f090c013c70 con 0x556a90045760 2024-04-25T08:56:13.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.522+0000 7f0918ff9640 1 -- 172.21.15.43:0/1420195063 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090c015420 con 0x556a90045760 2024-04-25T08:56:13.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.522+0000 7f092c587740 1 -- 172.21.15.43:0/1420195063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a90045760 msgr2=0x556a9007a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:13.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.522+0000 7f092c587740 1 --2- 172.21.15.43:0/1420195063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a90045760 0x556a9007a550 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f090c00b8a0 tx=0x7f090c00bd60 comp rx=0 tx=0).stop 2024-04-25T08:56:13.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.522+0000 7f092c587740 1 -- 172.21.15.43:0/1420195063 shutdown_connections 2024-04-25T08:56:13.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.522+0000 7f092c587740 1 --2- 172.21.15.43:0/1420195063 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a90057370 0x556a90059760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.522+0000 7f092c587740 1 --2- 172.21.15.43:0/1420195063 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a90045760 0x556a9007a550 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.522+0000 7f092c587740 1 --2- 172.21.15.43:0/1420195063 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556a9001aca0 0x556a90048c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.522+0000 7f092c587740 1 -- 172.21.15.43:0/1420195063 >> 172.21.15.43:0/1420195063 conn(0x556a8ffe2870 msgr2=0x556a90018a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:13.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.522+0000 7f092c587740 1 -- 172.21.15.43:0/1420195063 shutdown_connections 2024-04-25T08:56:13.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.522+0000 7f092c587740 1 -- 172.21.15.43:0/1420195063 wait complete. 2024-04-25T08:56:13.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.523+0000 7f092c587740 1 -- start start 2024-04-25T08:56:13.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.523+0000 7f092c587740 1 Processor -- start 2024-04-25T08:56:13.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.524+0000 7f092c587740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a9001aca0 0x556a900c9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.524+0000 7f092c587740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a90045760 0x556a90040ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.524+0000 7f092c587740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556a90057370 0x556a900442d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.524+0000 7f092c587740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09240b5660 con 0x556a90045760 2024-04-25T08:56:13.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.524+0000 7f092c587740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x556a9001c060 con 0x556a9001aca0 2024-04-25T08:56:13.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.524+0000 7f092c587740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x556a90044810 con 0x556a90057370 2024-04-25T08:56:13.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.524+0000 7f092c587740 10 client.0 _ll_register_callbacks cb 0x7f090c00b180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:56:13.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.524+0000 7f09227fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a90045760 0x556a90040ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.524+0000 7f09227fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a90045760 0x556a90040ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39798/0 (socket says 172.21.15.43:39798) 2024-04-25T08:56:13.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.524+0000 7f09227fc640 1 -- 172.21.15.43:0/1340628046 learned_addr learned my addr 172.21.15.43:0/1340628046 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:13.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.524+0000 7f09237fe640 1 --2- 172.21.15.43:0/1340628046 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556a90057370 0x556a900442d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.524+0000 7f0922ffd640 1 --2- 172.21.15.43:0/1340628046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a9001aca0 0x556a900c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.524+0000 7f09227fc640 1 -- 172.21.15.43:0/1340628046 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556a90057370 msgr2=0x556a900442d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:13.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.524+0000 7f09227fc640 1 --2- 172.21.15.43:0/1340628046 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556a90057370 0x556a900442d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.524+0000 7f09227fc640 1 -- 172.21.15.43:0/1340628046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a9001aca0 msgr2=0x556a900c9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:13.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.524+0000 7f09227fc640 1 --2- 172.21.15.43:0/1340628046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a9001aca0 0x556a900c9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.524+0000 7f09227fc640 1 -- 172.21.15.43:0/1340628046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556a90048230 con 0x556a90045760 2024-04-25T08:56:13.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.524+0000 7f09227fc640 1 --2- 172.21.15.43:0/1340628046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a90045760 0x556a90040ea0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f092415c780 tx=0x7f090c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f698a88b69b2ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:13.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.524+0000 7f09027fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:56:13.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.525+0000 7f09027fc640 1 -- 172.21.15.43:0/1340628046 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090c013ab0 con 0x556a90045760 2024-04-25T08:56:13.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.525+0000 7f09027fc640 1 -- 172.21.15.43:0/1340628046 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f090c015bd0 con 0x556a90045760 2024-04-25T08:56:13.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.525+0000 7f092c587740 1 -- 172.21.15.43:0/1340628046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f090c013c70 con 0x556a90045760 2024-04-25T08:56:13.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.525+0000 7f09027fc640 1 -- 172.21.15.43:0/1340628046 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7f090c032050 con 0x556a90045760 2024-04-25T08:56:13.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.525+0000 7f092c587740 10 client.5427 fetch_fsmap learned FSMap version 14 2024-04-25T08:56:13.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.525+0000 7f092c587740 1 -- 172.21.15.43:0/1340628046 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x556a900cab00 con 0x556a90045760 2024-04-25T08:56:13.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.525+0000 7f09027fc640 1 -- 172.21.15.43:0/1340628046 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f090c013c70 con 0x556a90045760 2024-04-25T08:56:13.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.525+0000 7f09027fc640 1 -- 172.21.15.43:0/1340628046 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 14) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f090c035020 con 0x556a90045760 2024-04-25T08:56:13.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.525+0000 7f092c587740 10 client.5427 fetch_fsmap finished waiting for FSMap version 14 2024-04-25T08:56:13.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.525+0000 7f092c587740 10 client.5427 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T08:56:13.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.525+0000 7f092c587740 20 client.5427 populate_metadata read hostname 'smithi043' 2024-04-25T08:56:13.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.525+0000 7f092c587740 1 --2- 172.21.15.43:0/1340628046 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x556a90059940 0x556a90059d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.525+0000 7f092c587740 4 client.5427 mds_command: new command op to 5208 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T08:56:13.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.525+0000 7f092c587740 1 -- 172.21.15.43:0/1340628046 --> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x556a8ff85420 con 0x556a90059940 2024-04-25T08:56:13.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.525+0000 7f0922ffd640 1 --2- 172.21.15.43:0/1340628046 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x556a90059940 0x556a90059d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.526+0000 7f0922ffd640 1 --2- 172.21.15.43:0/1340628046 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x556a90059940 0x556a90059d50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a31261fd8b77c03d server_cookie=2df8b9f7110fd8da in_seq=0 out_seq=0 2024-04-25T08:56:13.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.526+0000 7f09027fc640 10 client.5427 ms_handle_connect on v2:172.21.15.43:6833/242762542 2024-04-25T08:56:13.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.526+0000 7f09027fc640 1 -- 172.21.15.43:0/1340628046 <== mds.0 v2:172.21.15.43:6833/242762542 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17926 (crc 0 0 0) 0x556a8ff85420 con 0x556a90059940 2024-04-25T08:56:13.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.526+0000 7f09027fc640 10 client.5427 handle_command_reply: tid=0 2024-04-25T08:56:13.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.526+0000 7f092c587740 1 client.5427 shutdown 2024-04-25T08:56:13.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.526+0000 7f092c587740 1 -- 172.21.15.43:0/1340628046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a90045760 msgr2=0x556a90040ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:13.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.526+0000 7f092c587740 1 --2- 172.21.15.43:0/1340628046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a90045760 0x556a90040ea0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f092415c780 tx=0x7f090c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.526+0000 7f092c587740 1 -- 172.21.15.43:0/1340628046 shutdown_connections 2024-04-25T08:56:13.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.526+0000 7f092c587740 1 --2- 172.21.15.43:0/1340628046 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556a90057370 0x556a900442d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.526+0000 7f092c587740 1 --2- 172.21.15.43:0/1340628046 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x556a90059940 0x556a90059d50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.526+0000 7f092c587740 1 --2- 172.21.15.43:0/1340628046 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556a90045760 0x556a90040ea0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.526+0000 7f092c587740 1 --2- 172.21.15.43:0/1340628046 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556a9001aca0 0x556a900c9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.526+0000 7f09027fc640 0 client.5427 ms_handle_reset on v2:172.21.15.43:6833/242762542 2024-04-25T08:56:13.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.526+0000 7f092c587740 1 -- 172.21.15.43:0/1340628046 >> 172.21.15.43:0/1340628046 conn(0x556a8ffe2870 msgr2=0x556a90032de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:13.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.526+0000 7f092c587740 1 -- 172.21.15.43:0/1340628046 shutdown_connections 2024-04-25T08:56:13.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.526+0000 7f092c587740 1 -- 172.21.15.43:0/1340628046 wait complete. 2024-04-25T08:56:13.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.526+0000 7f092c587740 20 client.5427 trim_cache size 0 max 16384 2024-04-25T08:56:13.545 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T08:56:13.545 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T08:56:13.545 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 3669, 2024-04-25T08:56:13.546 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 3670, 2024-04-25T08:56:13.546 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 908980, 2024-04-25T08:56:13.546 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 34722122, 2024-04-25T08:56:13.546 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 9, 2024-04-25T08:56:13.546 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 3, 2024-04-25T08:56:13.546 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.327797657, 2024-04-25T08:56:13.546 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.150899697, 2024-04-25T08:56:13.546 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.062974753, 2024-04-25T08:56:13.546 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.090926397, 2024-04-25T08:56:13.546 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T08:56:13.546 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 3668, 2024-04-25T08:56:13.547 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.339801284, 2024-04-25T08:56:13.547 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000092639 2024-04-25T08:56:13.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.547 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-25T08:56:13.547 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 2, 2024-04-25T08:56:13.547 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000209, 2024-04-25T08:56:13.547 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000104 2024-04-25T08:56:13.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.547 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T08:56:13.547 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T08:56:13.547 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.547 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T08:56:13.547 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 33167, 2024-04-25T08:56:13.547 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 59110, 2024-04-25T08:56:13.548 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 11657348, 2024-04-25T08:56:13.548 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 41262818, 2024-04-25T08:56:13.548 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 8, 2024-04-25T08:56:13.548 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 4, 2024-04-25T08:56:13.548 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 2.900632877, 2024-04-25T08:56:13.548 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 1.451499481, 2024-04-25T08:56:13.548 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 1.726647807, 2024-04-25T08:56:13.548 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.030656646, 2024-04-25T08:56:13.548 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T08:56:13.548 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 59109, 2024-04-25T08:56:13.548 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 1.033918070, 2024-04-25T08:56:13.548 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000017491 2024-04-25T08:56:13.548 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.549 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-25T08:56:13.549 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 88540, 2024-04-25T08:56:13.549 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.013826062, 2024-04-25T08:56:13.549 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000156 2024-04-25T08:56:13.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.549 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T08:56:13.549 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 110304 2024-04-25T08:56:13.549 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.549 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T08:56:13.549 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 3840, 2024-04-25T08:56:13.549 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 3833, 2024-04-25T08:56:13.549 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 1210127, 2024-04-25T08:56:13.550 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 34856570, 2024-04-25T08:56:13.550 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 5, 2024-04-25T08:56:13.550 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 4, 2024-04-25T08:56:13.550 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.303329219, 2024-04-25T08:56:13.550 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.118262476, 2024-04-25T08:56:13.550 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.075591907, 2024-04-25T08:56:13.550 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.094261256, 2024-04-25T08:56:13.550 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T08:56:13.550 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 3833, 2024-04-25T08:56:13.550 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.368267789, 2024-04-25T08:56:13.550 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000096078 2024-04-25T08:56:13.550 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.551 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-25T08:56:13.551 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:13.551 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:13.551 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:13.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.551 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 117632, 2024-04-25T08:56:13.551 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 20160 2024-04-25T08:56:13.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.551 INFO:teuthology.orchestra.run.smithi043.stdout: "cct": { 2024-04-25T08:56:13.551 INFO:teuthology.orchestra.run.smithi043.stdout: "total_workers": 1, 2024-04-25T08:56:13.551 INFO:teuthology.orchestra.run.smithi043.stdout: "unhealthy_workers": 0 2024-04-25T08:56:13.551 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.552 INFO:teuthology.orchestra.run.smithi043.stdout: "finisher-MDSRank": { 2024-04-25T08:56:13.552 INFO:teuthology.orchestra.run.smithi043.stdout: "queue_len": 0, 2024-04-25T08:56:13.552 INFO:teuthology.orchestra.run.smithi043.stdout: "complete_latency": { 2024-04-25T08:56:13.552 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 9360, 2024-04-25T08:56:13.552 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 12.517619952, 2024-04-25T08:56:13.552 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.001337352 2024-04-25T08:56:13.552 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:56:13.552 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.552 INFO:teuthology.orchestra.run.smithi043.stdout: "finisher-PurgeQueue": { 2024-04-25T08:56:13.552 INFO:teuthology.orchestra.run.smithi043.stdout: "queue_len": 0, 2024-04-25T08:56:13.553 INFO:teuthology.orchestra.run.smithi043.stdout: "complete_latency": { 2024-04-25T08:56:13.553 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 1635, 2024-04-25T08:56:13.553 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.432804591, 2024-04-25T08:56:13.553 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000264712 2024-04-25T08:56:13.553 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:56:13.553 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.553 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": { 2024-04-25T08:56:13.553 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 28251, 2024-04-25T08:56:13.553 INFO:teuthology.orchestra.run.smithi043.stdout: "reply": 28251, 2024-04-25T08:56:13.553 INFO:teuthology.orchestra.run.smithi043.stdout: "reply_latency": { 2024-04-25T08:56:13.553 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 28251, 2024-04-25T08:56:13.553 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 243.042693723, 2024-04-25T08:56:13.554 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.008602976 2024-04-25T08:56:13.554 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.554 INFO:teuthology.orchestra.run.smithi043.stdout: "slow_reply": 0, 2024-04-25T08:56:13.554 INFO:teuthology.orchestra.run.smithi043.stdout: "forward": 0, 2024-04-25T08:56:13.554 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_fetch_complete": 0, 2024-04-25T08:56:13.554 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_fetch_keys": 0, 2024-04-25T08:56:13.554 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_commit": 12, 2024-04-25T08:56:13.554 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_split": 0, 2024-04-25T08:56:13.554 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_merge": 0, 2024-04-25T08:56:13.554 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes": 19818, 2024-04-25T08:56:13.554 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_top": 0, 2024-04-25T08:56:13.554 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_bottom": 19818, 2024-04-25T08:56:13.555 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_pin_tail": 0, 2024-04-25T08:56:13.555 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_pinned": 19818, 2024-04-25T08:56:13.555 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_expired": 0, 2024-04-25T08:56:13.555 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_with_caps": 521, 2024-04-25T08:56:13.555 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 521, 2024-04-25T08:56:13.555 INFO:teuthology.orchestra.run.smithi043.stdout: "subtrees": 2, 2024-04-25T08:56:13.555 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse": 46869, 2024-04-25T08:56:13.555 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_hit": 46381, 2024-04-25T08:56:13.555 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_forward": 0, 2024-04-25T08:56:13.555 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_discover": 0, 2024-04-25T08:56:13.555 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_dir_fetch": 0, 2024-04-25T08:56:13.555 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_remote_ino": 0, 2024-04-25T08:56:13.556 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_lock": 168, 2024-04-25T08:56:13.556 INFO:teuthology.orchestra.run.smithi043.stdout: "load_cent": 0, 2024-04-25T08:56:13.556 INFO:teuthology.orchestra.run.smithi043.stdout: "q": 0, 2024-04-25T08:56:13.556 INFO:teuthology.orchestra.run.smithi043.stdout: "exported": 0, 2024-04-25T08:56:13.556 INFO:teuthology.orchestra.run.smithi043.stdout: "exported_inodes": 0, 2024-04-25T08:56:13.556 INFO:teuthology.orchestra.run.smithi043.stdout: "imported": 0, 2024-04-25T08:56:13.556 INFO:teuthology.orchestra.run.smithi043.stdout: "imported_inodes": 0, 2024-04-25T08:56:13.556 INFO:teuthology.orchestra.run.smithi043.stdout: "openino_dir_fetch": 0, 2024-04-25T08:56:13.556 INFO:teuthology.orchestra.run.smithi043.stdout: "openino_backtrace_fetch": 0, 2024-04-25T08:56:13.556 INFO:teuthology.orchestra.run.smithi043.stdout: "openino_peer_discover": 0, 2024-04-25T08:56:13.556 INFO:teuthology.orchestra.run.smithi043.stdout: "root_rfiles": 0, 2024-04-25T08:56:13.556 INFO:teuthology.orchestra.run.smithi043.stdout: "root_rbytes": 0, 2024-04-25T08:56:13.557 INFO:teuthology.orchestra.run.smithi043.stdout: "root_rsnaps": 0, 2024-04-25T08:56:13.557 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T08:56:13.557 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_set_tag": 0, 2024-04-25T08:56:13.557 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T08:56:13.557 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_inotable_repaired": 0, 2024-04-25T08:56:13.557 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_dir_inodes": 0, 2024-04-25T08:56:13.557 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T08:56:13.557 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T08:56:13.557 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_file_inodes": 0, 2024-04-25T08:56:13.557 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_inode_file_caps": 0, 2024-04-25T08:56:13.557 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_revoke": 6190, 2024-04-25T08:56:13.558 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_grant": 6006, 2024-04-25T08:56:13.558 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T08:56:13.558 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T08:56:13.558 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T08:56:13.558 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_caps": 6190, 2024-04-25T08:56:13.558 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_caps_dirty": 2000, 2024-04-25T08:56:13.558 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_cap_release": 25, 2024-04-25T08:56:13.558 INFO:teuthology.orchestra.run.smithi043.stdout: "process_request_cap_release": 17846 2024-04-25T08:56:13.558 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.558 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_cache": { 2024-04-25T08:56:13.558 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_update": 0, 2024-04-25T08:56:13.558 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_update_receipt": 0, 2024-04-25T08:56:13.559 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_try_discover": 0, 2024-04-25T08:56:13.559 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_send_discover": 0, 2024-04-25T08:56:13.559 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_handle_discover": 0, 2024-04-25T08:56:13.559 INFO:teuthology.orchestra.run.smithi043.stdout: "num_strays": 2000, 2024-04-25T08:56:13.559 INFO:teuthology.orchestra.run.smithi043.stdout: "num_strays_delayed": 249, 2024-04-25T08:56:13.559 INFO:teuthology.orchestra.run.smithi043.stdout: "num_strays_enqueuing": 0, 2024-04-25T08:56:13.559 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_created": 4088, 2024-04-25T08:56:13.559 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_enqueued": 1481, 2024-04-25T08:56:13.559 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_reintegrated": 2196, 2024-04-25T08:56:13.559 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_migrated": 0, 2024-04-25T08:56:13.559 INFO:teuthology.orchestra.run.smithi043.stdout: "num_recovering_processing": 0, 2024-04-25T08:56:13.559 INFO:teuthology.orchestra.run.smithi043.stdout: "num_recovering_enqueued": 0, 2024-04-25T08:56:13.560 INFO:teuthology.orchestra.run.smithi043.stdout: "num_recovering_prioritized": 0, 2024-04-25T08:56:13.560 INFO:teuthology.orchestra.run.smithi043.stdout: "recovery_started": 0, 2024-04-25T08:56:13.560 INFO:teuthology.orchestra.run.smithi043.stdout: "recovery_completed": 0, 2024-04-25T08:56:13.560 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T08:56:13.560 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_exportdir": 0, 2024-04-25T08:56:13.560 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_flush": 0, 2024-04-25T08:56:13.560 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_fragmentdir": 0, 2024-04-25T08:56:13.560 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_fragstats": 0, 2024-04-25T08:56:13.560 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_inodestats": 0 2024-04-25T08:56:13.560 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.560 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_log": { 2024-04-25T08:56:13.560 INFO:teuthology.orchestra.run.smithi043.stdout: "evlrg": 0, 2024-04-25T08:56:13.561 INFO:teuthology.orchestra.run.smithi043.stdout: "evadd": 21633, 2024-04-25T08:56:13.561 INFO:teuthology.orchestra.run.smithi043.stdout: "evex": 0, 2024-04-25T08:56:13.561 INFO:teuthology.orchestra.run.smithi043.stdout: "evtrm": 0, 2024-04-25T08:56:13.561 INFO:teuthology.orchestra.run.smithi043.stdout: "ev": 21633, 2024-04-25T08:56:13.561 INFO:teuthology.orchestra.run.smithi043.stdout: "evexg": 0, 2024-04-25T08:56:13.561 INFO:teuthology.orchestra.run.smithi043.stdout: "evexd": 0, 2024-04-25T08:56:13.561 INFO:teuthology.orchestra.run.smithi043.stdout: "segadd": 28, 2024-04-25T08:56:13.561 INFO:teuthology.orchestra.run.smithi043.stdout: "segex": 0, 2024-04-25T08:56:13.561 INFO:teuthology.orchestra.run.smithi043.stdout: "segtrm": 0, 2024-04-25T08:56:13.561 INFO:teuthology.orchestra.run.smithi043.stdout: "seg": 28, 2024-04-25T08:56:13.561 INFO:teuthology.orchestra.run.smithi043.stdout: "segmjr": 3, 2024-04-25T08:56:13.561 INFO:teuthology.orchestra.run.smithi043.stdout: "segexg": 0, 2024-04-25T08:56:13.561 INFO:teuthology.orchestra.run.smithi043.stdout: "segexd": 0, 2024-04-25T08:56:13.562 INFO:teuthology.orchestra.run.smithi043.stdout: "expos": 4194304, 2024-04-25T08:56:13.562 INFO:teuthology.orchestra.run.smithi043.stdout: "wrpos": 80869790, 2024-04-25T08:56:13.562 INFO:teuthology.orchestra.run.smithi043.stdout: "rdpos": 0, 2024-04-25T08:56:13.562 INFO:teuthology.orchestra.run.smithi043.stdout: "jlat": { 2024-04-25T08:56:13.562 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 6494, 2024-04-25T08:56:13.562 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 72.929510444, 2024-04-25T08:56:13.562 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.011230291 2024-04-25T08:56:13.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.562 INFO:teuthology.orchestra.run.smithi043.stdout: "replayed": 0 2024-04-25T08:56:13.562 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.562 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_mem": { 2024-04-25T08:56:13.563 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 2014, 2024-04-25T08:56:13.563 INFO:teuthology.orchestra.run.smithi043.stdout: "ino+": 2014, 2024-04-25T08:56:13.564 INFO:teuthology.orchestra.run.smithi043.stdout: "ino-": 0, 2024-04-25T08:56:13.564 INFO:teuthology.orchestra.run.smithi043.stdout: "dir": 13, 2024-04-25T08:56:13.564 INFO:teuthology.orchestra.run.smithi043.stdout: "dir+": 13, 2024-04-25T08:56:13.564 INFO:teuthology.orchestra.run.smithi043.stdout: "dir-": 0, 2024-04-25T08:56:13.564 INFO:teuthology.orchestra.run.smithi043.stdout: "dn": 19818, 2024-04-25T08:56:13.564 INFO:teuthology.orchestra.run.smithi043.stdout: "dn+": 19818, 2024-04-25T08:56:13.564 INFO:teuthology.orchestra.run.smithi043.stdout: "dn-": 0, 2024-04-25T08:56:13.564 INFO:teuthology.orchestra.run.smithi043.stdout: "cap": 274, 2024-04-25T08:56:13.564 INFO:teuthology.orchestra.run.smithi043.stdout: "cap+": 2002, 2024-04-25T08:56:13.564 INFO:teuthology.orchestra.run.smithi043.stdout: "cap-": 1728, 2024-04-25T08:56:13.564 INFO:teuthology.orchestra.run.smithi043.stdout: "rss": 106376, 2024-04-25T08:56:13.564 INFO:teuthology.orchestra.run.smithi043.stdout: "heap": 215396 2024-04-25T08:56:13.564 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.565 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_server": { 2024-04-25T08:56:13.565 INFO:teuthology.orchestra.run.smithi043.stdout: "dispatch_client_request": 37664, 2024-04-25T08:56:13.565 INFO:teuthology.orchestra.run.smithi043.stdout: "dispatch_server_request": 0, 2024-04-25T08:56:13.565 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_request": 28251, 2024-04-25T08:56:13.565 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_session": 5, 2024-04-25T08:56:13.565 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_peer_request": 0, 2024-04-25T08:56:13.565 INFO:teuthology.orchestra.run.smithi043.stdout: "req_create_latency": { 2024-04-25T08:56:13.565 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 2000, 2024-04-25T08:56:13.565 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 1.369929300, 2024-04-25T08:56:13.565 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000684964 2024-04-25T08:56:13.565 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.565 INFO:teuthology.orchestra.run.smithi043.stdout: "req_getattr_latency": { 2024-04-25T08:56:13.566 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 1999, 2024-04-25T08:56:13.566 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.608105825, 2024-04-25T08:56:13.566 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000304205 2024-04-25T08:56:13.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.566 INFO:teuthology.orchestra.run.smithi043.stdout: "req_getfilelock_latency": { 2024-04-25T08:56:13.566 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:13.566 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:13.566 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:13.566 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.566 INFO:teuthology.orchestra.run.smithi043.stdout: "req_link_latency": { 2024-04-25T08:56:13.566 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 6000, 2024-04-25T08:56:13.567 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 9.535551052, 2024-04-25T08:56:13.567 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.001589258 2024-04-25T08:56:13.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.567 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookup_latency": { 2024-04-25T08:56:13.567 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 8043, 2024-04-25T08:56:13.567 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 7.183828452, 2024-04-25T08:56:13.567 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000893177 2024-04-25T08:56:13.567 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.567 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookuphash_latency": { 2024-04-25T08:56:13.567 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:13.567 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:13.567 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:13.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.568 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupino_latency": { 2024-04-25T08:56:13.568 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:13.568 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:13.568 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:13.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.568 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupname_latency": { 2024-04-25T08:56:13.568 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:13.568 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:13.568 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:13.568 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.569 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupparent_latency": { 2024-04-25T08:56:13.569 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:13.569 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:13.569 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:13.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.569 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupsnap_latency": { 2024-04-25T08:56:13.569 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 3, 2024-04-25T08:56:13.569 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.006938573, 2024-04-25T08:56:13.569 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.002312857 2024-04-25T08:56:13.569 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.569 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lssnap_latency": { 2024-04-25T08:56:13.569 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:13.570 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:13.570 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:13.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.570 INFO:teuthology.orchestra.run.smithi043.stdout: "req_mkdir_latency": { 2024-04-25T08:56:13.570 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 1, 2024-04-25T08:56:13.570 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.002562965, 2024-04-25T08:56:13.570 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.002562965 2024-04-25T08:56:13.570 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.570 INFO:teuthology.orchestra.run.smithi043.stdout: "req_mknod_latency": { 2024-04-25T08:56:13.570 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:13.571 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:13.571 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:13.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.571 INFO:teuthology.orchestra.run.smithi043.stdout: "req_mksnap_latency": { 2024-04-25T08:56:13.571 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 1, 2024-04-25T08:56:13.571 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.007145409, 2024-04-25T08:56:13.571 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.007145409 2024-04-25T08:56:13.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.571 INFO:teuthology.orchestra.run.smithi043.stdout: "req_open_latency": { 2024-04-25T08:56:13.571 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:13.571 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:13.571 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:13.571 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.572 INFO:teuthology.orchestra.run.smithi043.stdout: "req_readdir_latency": { 2024-04-25T08:56:13.572 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 6, 2024-04-25T08:56:13.572 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.093349188, 2024-04-25T08:56:13.572 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.015558198 2024-04-25T08:56:13.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.572 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rename_latency": { 2024-04-25T08:56:13.572 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 2196, 2024-04-25T08:56:13.572 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 208.693899538, 2024-04-25T08:56:13.572 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.095033651 2024-04-25T08:56:13.572 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.572 INFO:teuthology.orchestra.run.smithi043.stdout: "req_renamesnap_latency": { 2024-04-25T08:56:13.572 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:13.573 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:13.573 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:13.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.573 INFO:teuthology.orchestra.run.smithi043.stdout: "req_snapdiff_latency": { 2024-04-25T08:56:13.573 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:13.573 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:13.573 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:13.573 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.573 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rmdir_latency": { 2024-04-25T08:56:13.573 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:13.573 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:13.573 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:13.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.574 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rmsnap_latency": { 2024-04-25T08:56:13.574 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 1, 2024-04-25T08:56:13.574 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.004814886, 2024-04-25T08:56:13.574 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.004814886 2024-04-25T08:56:13.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.574 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rmxattr_latency": { 2024-04-25T08:56:13.574 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:13.574 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:13.574 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:13.574 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.574 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setattr_latency": { 2024-04-25T08:56:13.575 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 1, 2024-04-25T08:56:13.575 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000225467, 2024-04-25T08:56:13.575 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000225467 2024-04-25T08:56:13.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.575 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setdirlayout_latency": { 2024-04-25T08:56:13.575 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:13.575 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:13.575 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:13.575 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.575 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setfilelock_latency": { 2024-04-25T08:56:13.575 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:13.575 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:13.575 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:13.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.576 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setlayout_latency": { 2024-04-25T08:56:13.576 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:13.576 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:13.576 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:13.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.576 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setxattr_latency": { 2024-04-25T08:56:13.576 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:13.576 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:13.576 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:13.576 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.576 INFO:teuthology.orchestra.run.smithi043.stdout: "req_symlink_latency": { 2024-04-25T08:56:13.576 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:13.577 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:13.577 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:13.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.577 INFO:teuthology.orchestra.run.smithi043.stdout: "req_unlink_latency": { 2024-04-25T08:56:13.577 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 8000, 2024-04-25T08:56:13.577 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 15.536343068, 2024-04-25T08:56:13.577 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.001942042 2024-04-25T08:56:13.577 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.577 INFO:teuthology.orchestra.run.smithi043.stdout: "cap_revoke_eviction": 0, 2024-04-25T08:56:13.577 INFO:teuthology.orchestra.run.smithi043.stdout: "cap_acquisition_throttle": 0, 2024-04-25T08:56:13.577 INFO:teuthology.orchestra.run.smithi043.stdout: "req_getvxattr_latency": { 2024-04-25T08:56:13.577 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:13.577 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:13.578 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:13.578 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:56:13.578 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.578 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_sessions": { 2024-04-25T08:56:13.578 INFO:teuthology.orchestra.run.smithi043.stdout: "session_count": 1, 2024-04-25T08:56:13.578 INFO:teuthology.orchestra.run.smithi043.stdout: "session_add": 1, 2024-04-25T08:56:13.578 INFO:teuthology.orchestra.run.smithi043.stdout: "session_remove": 0, 2024-04-25T08:56:13.578 INFO:teuthology.orchestra.run.smithi043.stdout: "sessions_open": 1, 2024-04-25T08:56:13.578 INFO:teuthology.orchestra.run.smithi043.stdout: "sessions_stale": 0, 2024-04-25T08:56:13.578 INFO:teuthology.orchestra.run.smithi043.stdout: "total_load": 19520, 2024-04-25T08:56:13.578 INFO:teuthology.orchestra.run.smithi043.stdout: "average_load": 19520, 2024-04-25T08:56:13.578 INFO:teuthology.orchestra.run.smithi043.stdout: "avg_session_uptime": 57, 2024-04-25T08:56:13.579 INFO:teuthology.orchestra.run.smithi043.stdout: "mdthresh_evicted": 0 2024-04-25T08:56:13.579 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.579 INFO:teuthology.orchestra.run.smithi043.stdout: "mempool": { 2024-04-25T08:56:13.579 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_bytes": 0, 2024-04-25T08:56:13.579 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_items": 0, 2024-04-25T08:56:13.579 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T08:56:13.579 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_items": 0, 2024-04-25T08:56:13.579 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T08:56:13.579 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_items": 0, 2024-04-25T08:56:13.579 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T08:56:13.579 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T08:56:13.579 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T08:56:13.579 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T08:56:13.579 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T08:56:13.579 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_items": 0, 2024-04-25T08:56:13.580 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T08:56:13.580 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T08:56:13.580 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_bytes": 0, 2024-04-25T08:56:13.580 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_items": 0, 2024-04-25T08:56:13.580 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_bytes": 0, 2024-04-25T08:56:13.580 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_items": 0, 2024-04-25T08:56:13.580 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T08:56:13.580 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T08:56:13.580 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T08:56:13.580 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T08:56:13.580 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T08:56:13.580 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_items": 0, 2024-04-25T08:56:13.581 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_bytes": 0, 2024-04-25T08:56:13.581 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_items": 0, 2024-04-25T08:56:13.581 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T08:56:13.581 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T08:56:13.581 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_bytes": 0, 2024-04-25T08:56:13.581 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_items": 0, 2024-04-25T08:56:13.581 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_bytes": 0, 2024-04-25T08:56:13.581 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_items": 0, 2024-04-25T08:56:13.581 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T08:56:13.581 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_items": 0, 2024-04-25T08:56:13.581 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T08:56:13.581 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_items": 0, 2024-04-25T08:56:13.582 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_bytes": 16621972, 2024-04-25T08:56:13.582 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_items": 4190, 2024-04-25T08:56:13.582 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_bytes": 0, 2024-04-25T08:56:13.582 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_items": 0, 2024-04-25T08:56:13.582 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_bytes": 0, 2024-04-25T08:56:13.582 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_items": 0, 2024-04-25T08:56:13.582 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_bytes": 0, 2024-04-25T08:56:13.582 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_items": 0, 2024-04-25T08:56:13.582 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_bytes": 0, 2024-04-25T08:56:13.582 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_items": 0, 2024-04-25T08:56:13.582 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_bytes": 5312, 2024-04-25T08:56:13.582 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_items": 75, 2024-04-25T08:56:13.582 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T08:56:13.583 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_items": 0, 2024-04-25T08:56:13.583 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_bytes": 0, 2024-04-25T08:56:13.583 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_items": 0, 2024-04-25T08:56:13.583 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_bytes": 20459656, 2024-04-25T08:56:13.583 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_items": 800709, 2024-04-25T08:56:13.583 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_bytes": 0, 2024-04-25T08:56:13.583 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_items": 0, 2024-04-25T08:56:13.583 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_bytes": 0, 2024-04-25T08:56:13.583 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_items": 0 2024-04-25T08:56:13.583 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.583 INFO:teuthology.orchestra.run.smithi043.stdout: "objecter": { 2024-04-25T08:56:13.584 INFO:teuthology.orchestra.run.smithi043.stdout: "op_active": 0, 2024-04-25T08:56:13.584 INFO:teuthology.orchestra.run.smithi043.stdout: "op_laggy": 0, 2024-04-25T08:56:13.584 INFO:teuthology.orchestra.run.smithi043.stdout: "op_send": 8214, 2024-04-25T08:56:13.584 INFO:teuthology.orchestra.run.smithi043.stdout: "op_send_bytes": 76853661, 2024-04-25T08:56:13.584 INFO:teuthology.orchestra.run.smithi043.stdout: "op_resend": 37, 2024-04-25T08:56:13.584 INFO:teuthology.orchestra.run.smithi043.stdout: "op_reply": 8214, 2024-04-25T08:56:13.584 INFO:teuthology.orchestra.run.smithi043.stdout: "op_latency": { 2024-04-25T08:56:13.584 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 8214, 2024-04-25T08:56:13.584 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 22.710729216, 2024-04-25T08:56:13.584 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.002764880 2024-04-25T08:56:13.584 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.584 INFO:teuthology.orchestra.run.smithi043.stdout: "op_inflight": 0, 2024-04-25T08:56:13.585 INFO:teuthology.orchestra.run.smithi043.stdout: "oplen_avg": { 2024-04-25T08:56:13.585 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 8214, 2024-04-25T08:56:13.585 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 8270 2024-04-25T08:56:13.585 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.585 INFO:teuthology.orchestra.run.smithi043.stdout: "op": 8214, 2024-04-25T08:56:13.585 INFO:teuthology.orchestra.run.smithi043.stdout: "op_r": 55, 2024-04-25T08:56:13.585 INFO:teuthology.orchestra.run.smithi043.stdout: "op_w": 8159, 2024-04-25T08:56:13.585 INFO:teuthology.orchestra.run.smithi043.stdout: "op_rmw": 0, 2024-04-25T08:56:13.585 INFO:teuthology.orchestra.run.smithi043.stdout: "op_pg": 0, 2024-04-25T08:56:13.585 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_stat": 0, 2024-04-25T08:56:13.585 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_create": 11, 2024-04-25T08:56:13.585 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_read": 55, 2024-04-25T08:56:13.585 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_write": 6549, 2024-04-25T08:56:13.586 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_writefull": 36, 2024-04-25T08:56:13.586 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_writesame": 0, 2024-04-25T08:56:13.586 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_append": 0, 2024-04-25T08:56:13.586 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_zero": 1481, 2024-04-25T08:56:13.586 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_truncate": 0, 2024-04-25T08:56:13.586 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_delete": 30, 2024-04-25T08:56:13.586 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_mapext": 0, 2024-04-25T08:56:13.586 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_sparse_read": 0, 2024-04-25T08:56:13.586 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_clonerange": 0, 2024-04-25T08:56:13.586 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_getxattr": 0, 2024-04-25T08:56:13.586 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_setxattr": 22, 2024-04-25T08:56:13.587 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_cmpxattr": 0, 2024-04-25T08:56:13.587 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_rmxattr": 0, 2024-04-25T08:56:13.587 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_resetxattrs": 0, 2024-04-25T08:56:13.587 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_call": 0, 2024-04-25T08:56:13.587 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_watch": 0, 2024-04-25T08:56:13.587 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_notify": 0, 2024-04-25T08:56:13.587 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T08:56:13.587 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_pgls": 0, 2024-04-25T08:56:13.587 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_pgls_filter": 0, 2024-04-25T08:56:13.587 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_other": 0, 2024-04-25T08:56:13.587 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_active": 0, 2024-04-25T08:56:13.587 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_send": 0, 2024-04-25T08:56:13.587 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_resend": 0, 2024-04-25T08:56:13.588 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_ping": 0, 2024-04-25T08:56:13.588 INFO:teuthology.orchestra.run.smithi043.stdout: "poolop_active": 0, 2024-04-25T08:56:13.588 INFO:teuthology.orchestra.run.smithi043.stdout: "poolop_send": 0, 2024-04-25T08:56:13.588 INFO:teuthology.orchestra.run.smithi043.stdout: "poolop_resend": 0, 2024-04-25T08:56:13.588 INFO:teuthology.orchestra.run.smithi043.stdout: "poolstat_active": 0, 2024-04-25T08:56:13.588 INFO:teuthology.orchestra.run.smithi043.stdout: "poolstat_send": 0, 2024-04-25T08:56:13.588 INFO:teuthology.orchestra.run.smithi043.stdout: "poolstat_resend": 0, 2024-04-25T08:56:13.588 INFO:teuthology.orchestra.run.smithi043.stdout: "statfs_active": 0, 2024-04-25T08:56:13.588 INFO:teuthology.orchestra.run.smithi043.stdout: "statfs_send": 0, 2024-04-25T08:56:13.588 INFO:teuthology.orchestra.run.smithi043.stdout: "statfs_resend": 0, 2024-04-25T08:56:13.588 INFO:teuthology.orchestra.run.smithi043.stdout: "command_active": 0, 2024-04-25T08:56:13.588 INFO:teuthology.orchestra.run.smithi043.stdout: "command_send": 0, 2024-04-25T08:56:13.588 INFO:teuthology.orchestra.run.smithi043.stdout: "command_resend": 0, 2024-04-25T08:56:13.589 INFO:teuthology.orchestra.run.smithi043.stdout: "map_epoch": 33, 2024-04-25T08:56:13.589 INFO:teuthology.orchestra.run.smithi043.stdout: "map_full": 0, 2024-04-25T08:56:13.589 INFO:teuthology.orchestra.run.smithi043.stdout: "map_inc": 1, 2024-04-25T08:56:13.589 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_sessions": 8, 2024-04-25T08:56:13.589 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_session_open": 8, 2024-04-25T08:56:13.589 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_session_close": 0, 2024-04-25T08:56:13.589 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_laggy": 0, 2024-04-25T08:56:13.589 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_wr": 73, 2024-04-25T08:56:13.589 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_rd": 0, 2024-04-25T08:56:13.589 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_del": 13 2024-04-25T08:56:13.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.589 INFO:teuthology.orchestra.run.smithi043.stdout: "oft": { 2024-04-25T08:56:13.589 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_objs": 1, 2024-04-25T08:56:13.590 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_kv_pairs": 0, 2024-04-25T08:56:13.590 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_updates": 0, 2024-04-25T08:56:13.590 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_removes": 0 2024-04-25T08:56:13.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.590 INFO:teuthology.orchestra.run.smithi043.stdout: "osd-slow-ops": { 2024-04-25T08:56:13.590 INFO:teuthology.orchestra.run.smithi043.stdout: "slow_ops_count": 0 2024-04-25T08:56:13.590 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.590 INFO:teuthology.orchestra.run.smithi043.stdout: "purge_queue": { 2024-04-25T08:56:13.590 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing_ops": 0, 2024-04-25T08:56:13.590 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T08:56:13.590 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing": 0, 2024-04-25T08:56:13.591 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing_high_water": 32, 2024-04-25T08:56:13.591 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executed_ops": 1481, 2024-04-25T08:56:13.591 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executed": 1481, 2024-04-25T08:56:13.591 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_item_in_journal": 0 2024-04-25T08:56:13.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.591 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T08:56:13.591 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-25T08:56:13.591 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 104857600, 2024-04-25T08:56:13.591 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-25T08:56:13.591 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 40672, 2024-04-25T08:56:13.591 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 10111020, 2024-04-25T08:56:13.591 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-25T08:56:13.592 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 40672, 2024-04-25T08:56:13.592 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-04-25T08:56:13.592 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-04-25T08:56:13.592 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 40672, 2024-04-25T08:56:13.592 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 10111020, 2024-04-25T08:56:13.592 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-25T08:56:13.592 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:13.592 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:13.592 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:13.592 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:56:13.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.592 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-objecter_bytes": { 2024-04-25T08:56:13.592 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-25T08:56:13.593 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 104857600, 2024-04-25T08:56:13.593 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-25T08:56:13.593 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 0, 2024-04-25T08:56:13.593 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 0, 2024-04-25T08:56:13.593 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-25T08:56:13.593 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 0, 2024-04-25T08:56:13.593 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 8214, 2024-04-25T08:56:13.593 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 77015090, 2024-04-25T08:56:13.593 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 8214, 2024-04-25T08:56:13.593 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 77015090, 2024-04-25T08:56:13.593 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-25T08:56:13.593 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:13.594 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:13.594 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:13.594 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:56:13.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.594 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-objecter_ops": { 2024-04-25T08:56:13.594 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-25T08:56:13.594 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 1024, 2024-04-25T08:56:13.594 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-25T08:56:13.594 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 0, 2024-04-25T08:56:13.594 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 0, 2024-04-25T08:56:13.594 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-25T08:56:13.594 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 0, 2024-04-25T08:56:13.594 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 8214, 2024-04-25T08:56:13.595 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 8214, 2024-04-25T08:56:13.595 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 8214, 2024-04-25T08:56:13.595 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 8214, 2024-04-25T08:56:13.595 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-25T08:56:13.595 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:13.595 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:13.595 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:13.595 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:56:13.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.595 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-write_buf_throttle": { 2024-04-25T08:56:13.595 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-25T08:56:13.595 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 3758096384, 2024-04-25T08:56:13.595 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-25T08:56:13.595 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 1481, 2024-04-25T08:56:13.596 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 161429, 2024-04-25T08:56:13.596 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-25T08:56:13.596 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 1481, 2024-04-25T08:56:13.596 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-04-25T08:56:13.596 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-04-25T08:56:13.596 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 55, 2024-04-25T08:56:13.596 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 161429, 2024-04-25T08:56:13.596 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-25T08:56:13.596 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:13.596 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:13.596 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:13.596 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:56:13.596 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:13.597 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-write_buf_throttle-0x55d8ed2fa6e0": { 2024-04-25T08:56:13.597 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 84, 2024-04-25T08:56:13.597 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 3758096384, 2024-04-25T08:56:13.597 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-25T08:56:13.597 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 21633, 2024-04-25T08:56:13.597 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 76675486, 2024-04-25T08:56:13.597 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-25T08:56:13.597 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 21633, 2024-04-25T08:56:13.597 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-04-25T08:56:13.597 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-04-25T08:56:13.597 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 6494, 2024-04-25T08:56:13.597 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 76675402, 2024-04-25T08:56:13.598 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-25T08:56:13.598 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:13.598 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:13.598 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:13.598 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:56:13.598 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:56:13.598 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T08:56:13.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.527+0000 7f0929888640 1 -- 172.21.15.43:0/3732790147 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f090406b4d0 msgr2=0x7f090406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:13.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.527+0000 7f0929888640 1 --2- 172.21.15.43:0/3732790147 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f090406b4d0 0x7f090406d990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f091c000f60 tx=0x7f091c007ca0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.527+0000 7f0929888640 1 -- 172.21.15.43:0/3732790147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f092415dd80 msgr2=0x7f09241f3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:13.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.527+0000 7f0929888640 1 --2- 172.21.15.43:0/3732790147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f092415dd80 0x7f09241f3530 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f091402abe0 tx=0x7f0914004a40 comp rx=0 tx=0).stop 2024-04-25T08:56:13.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.527+0000 7f0929888640 1 -- 172.21.15.43:0/3732790147 shutdown_connections 2024-04-25T08:56:13.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.527+0000 7f0929888640 1 --2- 172.21.15.43:0/3732790147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f092415dd80 0x7f09241f3530 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.527+0000 7f0929888640 1 --2- 172.21.15.43:0/3732790147 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f09240d1ec0 0x7f0924155470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.527+0000 7f0929888640 1 --2- 172.21.15.43:0/3732790147 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f090406b4d0 0x7f090406d990 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.527+0000 7f0929888640 1 --2- 172.21.15.43:0/3732790147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09240b0cc0 0x7f092415b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.527+0000 7f0929888640 1 -- 172.21.15.43:0/3732790147 >> 172.21.15.43:0/3732790147 conn(0x7f0924072aa0 msgr2=0x7f092406c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:13.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.527+0000 7f0929888640 1 -- 172.21.15.43:0/3732790147 shutdown_connections 2024-04-25T08:56:13.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.527+0000 7f0929888640 1 -- 172.21.15.43:0/3732790147 wait complete. 2024-04-25T08:56:13.601 INFO:tasks.cephfs.test_scrub_checks:num of strays: 2000 2024-04-25T08:56:13.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:56:13.601 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub start / recursive,scrub_mdsdir 2024-04-25T08:56:13.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.735+0000 7fc9599b0640 1 Processor -- start 2024-04-25T08:56:13.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.735+0000 7fc9599b0640 1 -- start start 2024-04-25T08:56:13.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.736+0000 7fc9599b0640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc9540d3e70 0x7fc95415e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.736+0000 7fc9599b0640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9540b37a0 con 0x7fc95415b6c0 2024-04-25T08:56:13.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.736+0000 7fc9599b0640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc95415eab0 con 0x7fc95415baa0 2024-04-25T08:56:13.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.736+0000 7fc9599b0640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95415fce0 con 0x7fc9540d3e70 2024-04-25T08:56:13.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.736+0000 7fc952ffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc95415b6c0 0x7fc95415f5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:46220/0 (socket says 172.21.15.43:46220) 2024-04-25T08:56:13.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.736+0000 7fc952ffd640 1 -- 172.21.15.43:0/951248771 learned_addr learned my addr 172.21.15.43:0/951248771 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:56:13.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.736+0000 7fc951ffb640 1 -- 172.21.15.43:0/951248771 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1249268438 0 0) 0x7fc9540b37a0 con 0x7fc95415b6c0 2024-04-25T08:56:13.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.736+0000 7fc951ffb640 1 -- 172.21.15.43:0/951248771 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc93c003520 con 0x7fc95415b6c0 2024-04-25T08:56:13.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.736+0000 7fc9537fe640 1 --2- 172.21.15.43:0/951248771 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc9540d3e70 0x7fc95415e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.736+0000 7fc951ffb640 1 -- 172.21.15.43:0/951248771 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 978776592 0 0) 0x7fc93c003520 con 0x7fc95415b6c0 2024-04-25T08:56:13.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.736+0000 7fc951ffb640 1 -- 172.21.15.43:0/951248771 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc9540d3e70 msgr2=0x7fc95415e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:13.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.736+0000 7fc951ffb640 1 --2- 172.21.15.43:0/951248771 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc9540d3e70 0x7fc95415e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.736+0000 7fc951ffb640 1 -- 172.21.15.43:0/951248771 >> v1:172.21.15.193:6789/0 conn(0x7fc95415baa0 legacy=0x7fc9540d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:13.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.736+0000 7fc951ffb640 1 -- 172.21.15.43:0/951248771 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc95415ed90 con 0x7fc95415b6c0 2024-04-25T08:56:13.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.736+0000 7fc951ffb640 1 -- 172.21.15.43:0/951248771 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7fc94c001690 con 0x7fc95415b6c0 2024-04-25T08:56:13.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.737+0000 7fc951ffb640 1 -- 172.21.15.43:0/951248771 >> v1:172.21.15.43:6789/0 conn(0x7fc95415b6c0 legacy=0x7fc95415f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:13.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.737+0000 7fc951ffb640 1 --2- 172.21.15.43:0/951248771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc93c0039e0 0x7fc93c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.737+0000 7fc951ffb640 1 --2- 172.21.15.43:0/951248771 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc9540d3e70 0x7fc93c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.737+0000 7fc951ffb640 1 --2- 172.21.15.43:0/951248771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc93c004d40 0x7fc93c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.737+0000 7fc951ffb640 1 -- 172.21.15.43:0/951248771 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95415fce0 con 0x7fc93c0039e0 2024-04-25T08:56:13.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.737+0000 7fc951ffb640 1 -- 172.21.15.43:0/951248771 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93c00ba70 con 0x7fc93c004d40 2024-04-25T08:56:13.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.737+0000 7fc951ffb640 1 -- 172.21.15.43:0/951248771 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc93c00bc70 con 0x7fc9540d3e70 2024-04-25T08:56:13.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.737+0000 7fc952ffd640 1 --2- 172.21.15.43:0/951248771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc93c0039e0 0x7fc93c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.737+0000 7fc9537fe640 1 --2- 172.21.15.43:0/951248771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc93c004d40 0x7fc93c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.737+0000 7fc9527fc640 1 --2- 172.21.15.43:0/951248771 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc9540d3e70 0x7fc93c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.737+0000 7fc952ffd640 1 -- 172.21.15.43:0/951248771 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc9540d3e70 msgr2=0x7fc93c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:13.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.737+0000 7fc952ffd640 1 --2- 172.21.15.43:0/951248771 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc9540d3e70 0x7fc93c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.737+0000 7fc952ffd640 1 -- 172.21.15.43:0/951248771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc93c004d40 msgr2=0x7fc93c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:13.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.737+0000 7fc952ffd640 1 --2- 172.21.15.43:0/951248771 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc93c004d40 0x7fc93c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.737+0000 7fc952ffd640 1 -- 172.21.15.43:0/951248771 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc93c003520 con 0x7fc93c0039e0 2024-04-25T08:56:13.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.738+0000 7fc952ffd640 1 --2- 172.21.15.43:0/951248771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc93c0039e0 0x7fc93c003e00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc9540b3970 tx=0x7fc93c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c1ce6ec4ac422e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:13.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.738+0000 7fc951ffb640 1 -- 172.21.15.43:0/951248771 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c007d30 con 0x7fc93c0039e0 2024-04-25T08:56:13.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.738+0000 7fc951ffb640 1 -- 172.21.15.43:0/951248771 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc94c005ad0 con 0x7fc93c0039e0 2024-04-25T08:56:13.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.738+0000 7fc951ffb640 1 -- 172.21.15.43:0/951248771 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c007070 con 0x7fc93c0039e0 2024-04-25T08:56:13.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.738+0000 7fc9599b0640 1 -- 172.21.15.43:0/951248771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc93c0039e0 msgr2=0x7fc93c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:13.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.738+0000 7fc9599b0640 1 --2- 172.21.15.43:0/951248771 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc93c0039e0 0x7fc93c003e00 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc9540b3970 tx=0x7fc93c005830 comp rx=0 tx=0).stop 2024-04-25T08:56:13.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.738+0000 7fc9537fe640 1 -- 172.21.15.43:0/951248771 reap_dead start 2024-04-25T08:56:13.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.738+0000 7fc9599b0640 1 -- 172.21.15.43:0/951248771 shutdown_connections 2024-04-25T08:56:13.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.738+0000 7fc9599b0640 1 -- 172.21.15.43:0/951248771 >> 172.21.15.43:0/951248771 conn(0x7fc95408d460 msgr2=0x7fc95409cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:13.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.738+0000 7fc9599b0640 1 -- 172.21.15.43:0/951248771 shutdown_connections 2024-04-25T08:56:13.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.739+0000 7fc9599b0640 1 -- 172.21.15.43:0/951248771 wait complete. 2024-04-25T08:56:13.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.740+0000 7fc9599b0640 1 Processor -- start 2024-04-25T08:56:13.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.740+0000 7fc9599b0640 1 -- start start 2024-04-25T08:56:13.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.741+0000 7fc9599b0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc93c0039e0 0x7fc9540cd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.741+0000 7fc9599b0640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc9540c75c0 0x7fc9540cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.741+0000 7fc9599b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9540c7a40 0x7fc9540d0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.741+0000 7fc9599b0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95415e410 con 0x7fc9540c7a40 2024-04-25T08:56:13.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.741+0000 7fc9599b0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95415f350 con 0x7fc93c0039e0 2024-04-25T08:56:13.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.741+0000 7fc9599b0640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95415e590 con 0x7fc9540c75c0 2024-04-25T08:56:13.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.741+0000 7fc9537fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9540c7a40 0x7fc9540d0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.741+0000 7fc9537fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9540c7a40 0x7fc9540d0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39808/0 (socket says 172.21.15.43:39808) 2024-04-25T08:56:13.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.741+0000 7fc9537fe640 1 -- 172.21.15.43:0/2740651259 learned_addr learned my addr 172.21.15.43:0/2740651259 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:13.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.741+0000 7fc9527fc640 1 --2- 172.21.15.43:0/2740651259 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc9540c75c0 0x7fc9540cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.741+0000 7fc952ffd640 1 --2- 172.21.15.43:0/2740651259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc93c0039e0 0x7fc9540cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.741+0000 7fc9537fe640 1 -- 172.21.15.43:0/2740651259 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc9540c75c0 msgr2=0x7fc9540cfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:13.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.741+0000 7fc9537fe640 1 --2- 172.21.15.43:0/2740651259 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc9540c75c0 0x7fc9540cfac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.741+0000 7fc9537fe640 1 -- 172.21.15.43:0/2740651259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc93c0039e0 msgr2=0x7fc9540cd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:13.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.741+0000 7fc9537fe640 1 --2- 172.21.15.43:0/2740651259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc93c0039e0 0x7fc9540cd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.741+0000 7fc9537fe640 1 -- 172.21.15.43:0/2740651259 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9540b6730 con 0x7fc9540c7a40 2024-04-25T08:56:13.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.741+0000 7fc952ffd640 1 --2- 172.21.15.43:0/2740651259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc93c0039e0 0x7fc9540cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:56:13.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.741+0000 7fc9537fe640 1 --2- 172.21.15.43:0/2740651259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9540c7a40 0x7fc9540d0000 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc944011bf0 tx=0x7fc94400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:13.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.741+0000 7fc93bfff640 1 -- 172.21.15.43:0/2740651259 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94400fe10 con 0x7fc9540c7a40 2024-04-25T08:56:13.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.741+0000 7fc93bfff640 1 -- 172.21.15.43:0/2740651259 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc944015430 con 0x7fc9540c7a40 2024-04-25T08:56:13.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.741+0000 7fc9599b0640 1 -- 172.21.15.43:0/2740651259 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9540c7ff0 con 0x7fc9540c7a40 2024-04-25T08:56:13.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.741+0000 7fc93bfff640 1 -- 172.21.15.43:0/2740651259 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc944015820 con 0x7fc9540c7a40 2024-04-25T08:56:13.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.741+0000 7fc9599b0640 1 -- 172.21.15.43:0/2740651259 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9540cd880 con 0x7fc9540c7a40 2024-04-25T08:56:13.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.743+0000 7fc93bfff640 1 -- 172.21.15.43:0/2740651259 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc9440159c0 con 0x7fc9540c7a40 2024-04-25T08:56:13.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.743+0000 7fc93bfff640 1 --2- 172.21.15.43:0/2740651259 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc94006b4d0 0x7fc94006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.743+0000 7fc93bfff640 1 -- 172.21.15.43:0/2740651259 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc94408aee0 con 0x7fc9540c7a40 2024-04-25T08:56:13.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.743+0000 7fc952ffd640 1 --2- 172.21.15.43:0/2740651259 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc94006b4d0 0x7fc94006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.744+0000 7fc952ffd640 1 --2- 172.21.15.43:0/2740651259 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc94006b4d0 0x7fc94006d990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc94c032000 tx=0x7fc94c02d000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:13.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.744+0000 7fc95c794740 1 Processor -- start 2024-04-25T08:56:13.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.744+0000 7fc95c794740 1 -- start start 2024-04-25T08:56:13.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.745+0000 7fc95c794740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560571dfe710 0x560571e00f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.745+0000 7fc95c794740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x560571dfeaf0 0x560571e01460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.745+0000 7fc95c794740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560571dd1370 0x560571dd4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.745+0000 7fc95c794740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x560571d39bc0 con 0x560571dd1370 2024-04-25T08:56:13.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.745+0000 7fc95c794740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x560571ce2e70 con 0x560571dfe710 2024-04-25T08:56:13.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.745+0000 7fc95c794740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x560571d35c60 con 0x560571dfeaf0 2024-04-25T08:56:13.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.745+0000 7fc9537fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560571dd1370 0x560571dd4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.745+0000 7fc9537fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560571dd1370 0x560571dd4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39818/0 (socket says 172.21.15.43:39818) 2024-04-25T08:56:13.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.745+0000 7fc9537fe640 1 -- 172.21.15.43:0/1034141608 learned_addr learned my addr 172.21.15.43:0/1034141608 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:13.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.745+0000 7fc9527fc640 1 --2- 172.21.15.43:0/1034141608 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x560571dfeaf0 0x560571e01460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.745+0000 7fc952ffd640 1 --2- 172.21.15.43:0/1034141608 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560571dfe710 0x560571e00f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.746+0000 7fc9537fe640 1 -- 172.21.15.43:0/1034141608 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x560571dfeaf0 msgr2=0x560571e01460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:13.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.746+0000 7fc9537fe640 1 --2- 172.21.15.43:0/1034141608 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x560571dfeaf0 0x560571e01460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.746+0000 7fc9537fe640 1 -- 172.21.15.43:0/1034141608 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560571dfe710 msgr2=0x560571e00f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:13.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.746+0000 7fc9537fe640 1 --2- 172.21.15.43:0/1034141608 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560571dfe710 0x560571e00f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.746+0000 7fc9537fe640 1 -- 172.21.15.43:0/1034141608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560571d03ad0 con 0x560571dd1370 2024-04-25T08:56:13.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.746+0000 7fc9527fc640 1 --2- 172.21.15.43:0/1034141608 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x560571dfeaf0 0x560571e01460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:56:13.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.746+0000 7fc952ffd640 1 --2- 172.21.15.43:0/1034141608 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560571dfe710 0x560571e00f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:56:13.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.746+0000 7fc9537fe640 1 --2- 172.21.15.43:0/1034141608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560571dd1370 0x560571dd4b80 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc944055ab0 tx=0x7fc944015db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7af8ce32dda91fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:13.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.746+0000 7fc9397fa640 1 -- 172.21.15.43:0/1034141608 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc944061d60 con 0x560571dd1370 2024-04-25T08:56:13.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.746+0000 7fc9397fa640 1 -- 172.21.15.43:0/1034141608 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc944057950 con 0x560571dd1370 2024-04-25T08:56:13.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.746+0000 7fc9397fa640 1 -- 172.21.15.43:0/1034141608 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc944055cc0 con 0x560571dd1370 2024-04-25T08:56:13.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.746+0000 7fc95c794740 1 -- 172.21.15.43:0/1034141608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560571dd1370 msgr2=0x560571dd4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:13.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.746+0000 7fc95c794740 1 --2- 172.21.15.43:0/1034141608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560571dd1370 0x560571dd4b80 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fc944055ab0 tx=0x7fc944015db0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.747+0000 7fc95c794740 1 -- 172.21.15.43:0/1034141608 shutdown_connections 2024-04-25T08:56:13.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.747+0000 7fc95c794740 1 --2- 172.21.15.43:0/1034141608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560571dd1370 0x560571dd4b80 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.747+0000 7fc95c794740 1 --2- 172.21.15.43:0/1034141608 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x560571dfeaf0 0x560571e01460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.747+0000 7fc95c794740 1 --2- 172.21.15.43:0/1034141608 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560571dfe710 0x560571e00f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.747+0000 7fc95c794740 1 -- 172.21.15.43:0/1034141608 >> 172.21.15.43:0/1034141608 conn(0x560571dd35e0 msgr2=0x560571e1bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:13.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.747+0000 7fc95c794740 1 -- 172.21.15.43:0/1034141608 shutdown_connections 2024-04-25T08:56:13.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.747+0000 7fc95c794740 1 -- 172.21.15.43:0/1034141608 wait complete. 2024-04-25T08:56:13.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.747+0000 7fc95c794740 1 -- start start 2024-04-25T08:56:13.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.747+0000 7fc95c794740 1 Processor -- start 2024-04-25T08:56:13.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.748+0000 7fc95c794740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x560571dfe710 0x560571df4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.748+0000 7fc95c794740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560571dfeaf0 0x560571e173f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.748+0000 7fc95c794740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560571e1d050 0x560571e17930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.748+0000 7fc95c794740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x560571dd6790 con 0x560571e1d050 2024-04-25T08:56:13.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.748+0000 7fc95c794740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x560571cd84e0 con 0x560571dfeaf0 2024-04-25T08:56:13.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.748+0000 7fc95c794740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x560571dd6a70 con 0x560571dfe710 2024-04-25T08:56:13.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.748+0000 7fc95c794740 10 client.0 _ll_register_callbacks cb 0x560571cf5de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:56:13.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.748+0000 7fc9537fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560571e1d050 0x560571e17930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.748+0000 7fc9537fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560571e1d050 0x560571e17930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39822/0 (socket says 172.21.15.43:39822) 2024-04-25T08:56:13.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.748+0000 7fc9537fe640 1 -- 172.21.15.43:0/804227941 learned_addr learned my addr 172.21.15.43:0/804227941 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:13.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.748+0000 7fc952ffd640 1 --2- 172.21.15.43:0/804227941 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x560571dfe710 0x560571df4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.748+0000 7fc9527fc640 1 --2- 172.21.15.43:0/804227941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560571dfeaf0 0x560571e173f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.749+0000 7fc9537fe640 1 -- 172.21.15.43:0/804227941 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x560571dfe710 msgr2=0x560571df4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:13.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.749+0000 7fc9537fe640 1 --2- 172.21.15.43:0/804227941 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x560571dfe710 0x560571df4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.749+0000 7fc9537fe640 1 -- 172.21.15.43:0/804227941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560571dfeaf0 msgr2=0x560571e173f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:13.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.749+0000 7fc9537fe640 1 --2- 172.21.15.43:0/804227941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560571dfeaf0 0x560571e173f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.749+0000 7fc9537fe640 1 -- 172.21.15.43:0/804227941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560571b1a940 con 0x560571e1d050 2024-04-25T08:56:13.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.749+0000 7fc9537fe640 1 --2- 172.21.15.43:0/804227941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560571e1d050 0x560571e17930 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc944059590 tx=0x7fc94400ffb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9d106528cabcd52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:13.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.749+0000 7fc9327fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:56:13.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.749+0000 7fc9327fc640 1 -- 172.21.15.43:0/804227941 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc944058aa0 con 0x560571e1d050 2024-04-25T08:56:13.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.749+0000 7fc9327fc640 1 -- 172.21.15.43:0/804227941 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc944058c40 con 0x560571e1d050 2024-04-25T08:56:13.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.749+0000 7fc95c794740 1 -- 172.21.15.43:0/804227941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc944057950 con 0x560571e1d050 2024-04-25T08:56:13.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.749+0000 7fc9327fc640 1 -- 172.21.15.43:0/804227941 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9440780a0 con 0x560571e1d050 2024-04-25T08:56:13.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.749+0000 7fc95c794740 10 client.5439 fetch_fsmap learned FSMap version 14 2024-04-25T08:56:13.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.749+0000 7fc95c794740 1 -- 172.21.15.43:0/804227941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x560571b7b0d0 con 0x560571e1d050 2024-04-25T08:56:13.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.749+0000 7fc9327fc640 1 -- 172.21.15.43:0/804227941 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc944057950 con 0x560571e1d050 2024-04-25T08:56:13.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.749+0000 7fc9327fc640 1 -- 172.21.15.43:0/804227941 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 14) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc944025030 con 0x560571e1d050 2024-04-25T08:56:13.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.749+0000 7fc95c794740 10 client.5439 fetch_fsmap finished waiting for FSMap version 14 2024-04-25T08:56:13.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.749+0000 7fc95c794740 10 client.5439 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T08:56:13.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.749+0000 7fc95c794740 20 client.5439 populate_metadata read hostname 'smithi043' 2024-04-25T08:56:13.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.749+0000 7fc95c794740 1 --2- 172.21.15.43:0/804227941 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x560571df5700 0x560571e1f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.749+0000 7fc95c794740 4 client.5439 mds_command: new command op to 5208 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:56:13.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.749+0000 7fc95c794740 1 -- 172.21.15.43:0/804227941 --> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560571d47100 con 0x560571df5700 2024-04-25T08:56:13.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.749+0000 7fc952ffd640 1 --2- 172.21.15.43:0/804227941 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x560571df5700 0x560571e1f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.750+0000 7fc9327fc640 10 client.5439 ms_handle_connect on v2:172.21.15.43:6833/242762542 2024-04-25T08:56:13.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.750+0000 7fc952ffd640 1 --2- 172.21.15.43:0/804227941 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x560571df5700 0x560571e1f860 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2aaac9c4ba9e8f5d server_cookie=a7ccc65b4ab4a09d in_seq=0 out_seq=0 2024-04-25T08:56:13.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.750+0000 7fc9327fc640 1 -- 172.21.15.43:0/804227941 <== mds.0 v2:172.21.15.43:6833/242762542 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560571d47100 con 0x560571df5700 2024-04-25T08:56:13.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.750+0000 7fc9327fc640 10 client.5439 handle_command_reply: tid=0 2024-04-25T08:56:13.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.750+0000 7fc95c794740 1 client.5439 shutdown 2024-04-25T08:56:13.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.750+0000 7fc95c794740 1 -- 172.21.15.43:0/804227941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560571e1d050 msgr2=0x560571e17930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:13.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.750+0000 7fc95c794740 1 --2- 172.21.15.43:0/804227941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560571e1d050 0x560571e17930 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc944059590 tx=0x7fc94400ffb0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.750+0000 7fc95c794740 1 -- 172.21.15.43:0/804227941 shutdown_connections 2024-04-25T08:56:13.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.750+0000 7fc95c794740 1 --2- 172.21.15.43:0/804227941 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x560571df5700 0x560571e1f860 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.750+0000 7fc95c794740 1 --2- 172.21.15.43:0/804227941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560571e1d050 0x560571e17930 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.750+0000 7fc95c794740 1 --2- 172.21.15.43:0/804227941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560571dfeaf0 0x560571e173f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.750+0000 7fc9327fc640 0 client.5439 ms_handle_reset on v2:172.21.15.43:6833/242762542 2024-04-25T08:56:13.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.750+0000 7fc95c794740 1 --2- 172.21.15.43:0/804227941 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x560571dfe710 0x560571df4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.750+0000 7fc95c794740 1 -- 172.21.15.43:0/804227941 >> 172.21.15.43:0/804227941 conn(0x560571dd1370 msgr2=0x560571dfcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:13.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.751+0000 7fc95c794740 1 -- 172.21.15.43:0/804227941 shutdown_connections 2024-04-25T08:56:13.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.751+0000 7fc95c794740 1 -- 172.21.15.43:0/804227941 wait complete. 2024-04-25T08:56:13.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.751+0000 7fc95c794740 20 client.5439 trim_cache size 0 max 16384 2024-04-25T08:56:13.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.766+0000 7fc95c794740 1 Processor -- start 2024-04-25T08:56:13.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.766+0000 7fc95c794740 1 -- start start 2024-04-25T08:56:13.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.767+0000 7fc95c794740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560571dfe710 0x560571dd1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.767+0000 7fc95c794740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x560571dfeaf0 0x560571df6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.767+0000 7fc95c794740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560571e1d050 0x560571df69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.767+0000 7fc95c794740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9542006e0 con 0x560571dfe710 2024-04-25T08:56:13.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.767+0000 7fc95c794740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9541ff9b0 con 0x560571e1d050 2024-04-25T08:56:13.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.767+0000 7fc95c794740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9540b7ac0 con 0x560571dfeaf0 2024-04-25T08:56:13.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.767+0000 7fc952ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560571dfe710 0x560571dd1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.767+0000 7fc952ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560571dfe710 0x560571dd1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39824/0 (socket says 172.21.15.43:39824) 2024-04-25T08:56:13.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.767+0000 7fc952ffd640 1 -- 172.21.15.43:0/1550371218 learned_addr learned my addr 172.21.15.43:0/1550371218 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:13.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.767+0000 7fc9537fe640 1 --2- 172.21.15.43:0/1550371218 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560571e1d050 0x560571df69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.767+0000 7fc9527fc640 1 --2- 172.21.15.43:0/1550371218 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x560571dfeaf0 0x560571df6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.767+0000 7fc952ffd640 1 -- 172.21.15.43:0/1550371218 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x560571dfeaf0 msgr2=0x560571df6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:13.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.767+0000 7fc952ffd640 1 --2- 172.21.15.43:0/1550371218 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x560571dfeaf0 0x560571df6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.767+0000 7fc952ffd640 1 -- 172.21.15.43:0/1550371218 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560571e1d050 msgr2=0x560571df69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:13.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.767+0000 7fc952ffd640 1 --2- 172.21.15.43:0/1550371218 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560571e1d050 0x560571df69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.767+0000 7fc952ffd640 1 -- 172.21.15.43:0/1550371218 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560571a0e230 con 0x560571dfe710 2024-04-25T08:56:13.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.767+0000 7fc952ffd640 1 --2- 172.21.15.43:0/1550371218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560571dfe710 0x560571dd1d40 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc94c00c300 tx=0x7fc94c009030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e338765712e4755b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:13.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.767+0000 7fc938ff9640 1 -- 172.21.15.43:0/1550371218 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c041040 con 0x560571dfe710 2024-04-25T08:56:13.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.767+0000 7fc938ff9640 1 -- 172.21.15.43:0/1550371218 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc94c038040 con 0x560571dfe710 2024-04-25T08:56:13.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.767+0000 7fc938ff9640 1 -- 172.21.15.43:0/1550371218 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c007540 con 0x560571dfe710 2024-04-25T08:56:13.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.768+0000 7fc95c794740 1 -- 172.21.15.43:0/1550371218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560571dfe710 msgr2=0x560571dd1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:13.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.768+0000 7fc95c794740 1 --2- 172.21.15.43:0/1550371218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560571dfe710 0x560571dd1d40 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc94c00c300 tx=0x7fc94c009030 comp rx=0 tx=0).stop 2024-04-25T08:56:13.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.768+0000 7fc95c794740 1 -- 172.21.15.43:0/1550371218 shutdown_connections 2024-04-25T08:56:13.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.768+0000 7fc95c794740 1 --2- 172.21.15.43:0/1550371218 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560571e1d050 0x560571df69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.768+0000 7fc95c794740 1 --2- 172.21.15.43:0/1550371218 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x560571dfeaf0 0x560571df6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.768+0000 7fc95c794740 1 --2- 172.21.15.43:0/1550371218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560571dfe710 0x560571dd1d40 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.768+0000 7fc95c794740 1 -- 172.21.15.43:0/1550371218 >> 172.21.15.43:0/1550371218 conn(0x560571dd1370 msgr2=0x560571e3dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:13.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.768+0000 7fc95c794740 1 -- 172.21.15.43:0/1550371218 shutdown_connections 2024-04-25T08:56:13.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.768+0000 7fc95c794740 1 -- 172.21.15.43:0/1550371218 wait complete. 2024-04-25T08:56:13.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.769+0000 7fc95c794740 1 -- start start 2024-04-25T08:56:13.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.769+0000 7fc95c794740 1 Processor -- start 2024-04-25T08:56:13.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.769+0000 7fc95c794740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560571dfe710 0x560571df36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.769+0000 7fc95c794740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x560571dfeaf0 0x560571e22b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.769+0000 7fc95c794740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560571e1d050 0x560571e23080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.769+0000 7fc95c794740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x560571dcbf00 con 0x560571e1d050 2024-04-25T08:56:13.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.769+0000 7fc95c794740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9540b80c0 con 0x560571dfe710 2024-04-25T08:56:13.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.769+0000 7fc95c794740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x560571e368d0 con 0x560571dfeaf0 2024-04-25T08:56:13.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.769+0000 7fc95c794740 10 client.0 _ll_register_callbacks cb 0x560571dd8780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:56:13.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.769+0000 7fc9537fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560571e1d050 0x560571e23080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.770+0000 7fc9537fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560571e1d050 0x560571e23080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39826/0 (socket says 172.21.15.43:39826) 2024-04-25T08:56:13.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.770+0000 7fc9537fe640 1 -- 172.21.15.43:0/1687259450 learned_addr learned my addr 172.21.15.43:0/1687259450 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:13.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.770+0000 7fc952ffd640 1 --2- 172.21.15.43:0/1687259450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560571dfe710 0x560571df36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.770+0000 7fc9527fc640 1 --2- 172.21.15.43:0/1687259450 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x560571dfeaf0 0x560571e22b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.770+0000 7fc9537fe640 1 -- 172.21.15.43:0/1687259450 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x560571dfeaf0 msgr2=0x560571e22b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:13.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.770+0000 7fc9537fe640 1 --2- 172.21.15.43:0/1687259450 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x560571dfeaf0 0x560571e22b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.770+0000 7fc9537fe640 1 -- 172.21.15.43:0/1687259450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560571dfe710 msgr2=0x560571df36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:13.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.770+0000 7fc9537fe640 1 --2- 172.21.15.43:0/1687259450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560571dfe710 0x560571df36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.770+0000 7fc9537fe640 1 -- 172.21.15.43:0/1687259450 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560571dd6520 con 0x560571e1d050 2024-04-25T08:56:13.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.770+0000 7fc9537fe640 1 --2- 172.21.15.43:0/1687259450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560571e1d050 0x560571e23080 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc944061f20 tx=0x7fc9440559a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ae43809aea5be8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:13.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.770+0000 7fc9327fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:56:13.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.770+0000 7fc9327fc640 1 -- 172.21.15.43:0/1687259450 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc944078240 con 0x560571e1d050 2024-04-25T08:56:13.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.770+0000 7fc9327fc640 1 -- 172.21.15.43:0/1687259450 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc944058aa0 con 0x560571e1d050 2024-04-25T08:56:13.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.770+0000 7fc95c794740 1 -- 172.21.15.43:0/1687259450 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc94c038040 con 0x560571e1d050 2024-04-25T08:56:13.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.770+0000 7fc9327fc640 1 -- 172.21.15.43:0/1687259450 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc944058ca0 con 0x560571e1d050 2024-04-25T08:56:13.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.770+0000 7fc95c794740 10 client.5445 fetch_fsmap learned FSMap version 14 2024-04-25T08:56:13.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.770+0000 7fc95c794740 1 -- 172.21.15.43:0/1687259450 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x560571dd5da0 con 0x560571e1d050 2024-04-25T08:56:13.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.770+0000 7fc9327fc640 1 -- 172.21.15.43:0/1687259450 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc94c038040 con 0x560571e1d050 2024-04-25T08:56:13.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.770+0000 7fc9327fc640 1 -- 172.21.15.43:0/1687259450 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 14) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc944025040 con 0x560571e1d050 2024-04-25T08:56:13.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.770+0000 7fc95c794740 10 client.5445 fetch_fsmap finished waiting for FSMap version 14 2024-04-25T08:56:13.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.770+0000 7fc95c794740 10 client.5445 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T08:56:13.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.771+0000 7fc95c794740 20 client.5445 populate_metadata read hostname 'smithi043' 2024-04-25T08:56:13.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.771+0000 7fc95c794740 1 --2- 172.21.15.43:0/1687259450 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x560571e866c0 0x560571e86ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.771+0000 7fc95c794740 4 client.5445 mds_command: new command op to 5208 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "scrub_mdsdir"]}] 2024-04-25T08:56:13.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.771+0000 7fc95c794740 1 -- 172.21.15.43:0/1687259450 --> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "scrub_mdsdir"]}) v1 -- 0x560571d41450 con 0x560571e866c0 2024-04-25T08:56:13.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.771+0000 7fc952ffd640 1 --2- 172.21.15.43:0/1687259450 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x560571e866c0 0x560571e86ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.771+0000 7fc9327fc640 10 client.5445 ms_handle_connect on v2:172.21.15.43:6833/242762542 2024-04-25T08:56:13.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.771+0000 7fc952ffd640 1 --2- 172.21.15.43:0/1687259450 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x560571e866c0 0x560571e86ad0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eed2c225e2495818 server_cookie=cc8ff4b04bd62d31 in_seq=0 out_seq=0 2024-04-25T08:56:13.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.777+0000 7fc9327fc640 1 -- 172.21.15.43:0/1687259450 <== mds.0 v2:172.21.15.43:6833/242762542 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560571d41450 con 0x560571e866c0 2024-04-25T08:56:13.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.777+0000 7fc9327fc640 10 client.5445 handle_command_reply: tid=0 2024-04-25T08:56:13.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.777+0000 7fc95c794740 1 client.5445 shutdown 2024-04-25T08:56:13.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.777+0000 7fc95c794740 1 -- 172.21.15.43:0/1687259450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560571e1d050 msgr2=0x560571e23080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:13.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.777+0000 7fc95c794740 1 --2- 172.21.15.43:0/1687259450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560571e1d050 0x560571e23080 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fc944061f20 tx=0x7fc9440559a0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.778+0000 7fc95c794740 1 -- 172.21.15.43:0/1687259450 shutdown_connections 2024-04-25T08:56:13.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.778+0000 7fc95c794740 1 --2- 172.21.15.43:0/1687259450 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x560571e866c0 0x560571e86ad0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.778+0000 7fc95c794740 1 --2- 172.21.15.43:0/1687259450 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560571e1d050 0x560571e23080 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.778+0000 7fc95c794740 1 --2- 172.21.15.43:0/1687259450 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x560571dfeaf0 0x560571e22b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.778+0000 7fc95c794740 1 --2- 172.21.15.43:0/1687259450 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560571dfe710 0x560571df36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.778+0000 7fc95c794740 1 -- 172.21.15.43:0/1687259450 >> 172.21.15.43:0/1687259450 conn(0x560571dd1370 msgr2=0x560571e00f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:13.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.778+0000 7fc9327fc640 0 client.5445 ms_handle_reset on v2:172.21.15.43:6833/242762542 2024-04-25T08:56:13.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.778+0000 7fc95c794740 1 -- 172.21.15.43:0/1687259450 shutdown_connections 2024-04-25T08:56:13.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.778+0000 7fc95c794740 1 -- 172.21.15.43:0/1687259450 wait complete. 2024-04-25T08:56:13.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.778+0000 7fc95c794740 20 client.5445 trim_cache size 0 max 16384 2024-04-25T08:56:13.770 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T08:56:13.770 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-25T08:56:13.770 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "4df37e95-019e-47b0-8cef-9fc7f27ab48e", 2024-04-25T08:56:13.771 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-25T08:56:13.771 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T08:56:13.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.778+0000 7fc9599b0640 1 -- 172.21.15.43:0/2740651259 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc94006b4d0 msgr2=0x7fc94006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:13.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.778+0000 7fc9599b0640 1 --2- 172.21.15.43:0/2740651259 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc94006b4d0 0x7fc94006d990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc94c032000 tx=0x7fc94c02d000 comp rx=0 tx=0).stop 2024-04-25T08:56:13.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.778+0000 7fc9599b0640 1 -- 172.21.15.43:0/2740651259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9540c7a40 msgr2=0x7fc9540d0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:13.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.778+0000 7fc9599b0640 1 --2- 172.21.15.43:0/2740651259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9540c7a40 0x7fc9540d0000 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc944011bf0 tx=0x7fc94400f580 comp rx=0 tx=0).stop 2024-04-25T08:56:13.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.778+0000 7fc9599b0640 1 -- 172.21.15.43:0/2740651259 shutdown_connections 2024-04-25T08:56:13.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.778+0000 7fc9599b0640 1 --2- 172.21.15.43:0/2740651259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9540c7a40 0x7fc9540d0000 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.778+0000 7fc9599b0640 1 --2- 172.21.15.43:0/2740651259 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc9540c75c0 0x7fc9540cfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.778+0000 7fc9599b0640 1 --2- 172.21.15.43:0/2740651259 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc94006b4d0 0x7fc94006d990 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.778+0000 7fc9599b0640 1 --2- 172.21.15.43:0/2740651259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc93c0039e0 0x7fc9540cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.778+0000 7fc9599b0640 1 -- 172.21.15.43:0/2740651259 >> 172.21.15.43:0/2740651259 conn(0x7fc95408d460 msgr2=0x7fc95415ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:13.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.779+0000 7fc9599b0640 1 -- 172.21.15.43:0/2740651259 shutdown_connections 2024-04-25T08:56:13.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.779+0000 7fc9599b0640 1 -- 172.21.15.43:0/2740651259 wait complete. 2024-04-25T08:56:13.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:56:13.780 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-04-25T08:56:13.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.921+0000 7fcb96f28640 1 Processor -- start 2024-04-25T08:56:13.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.921+0000 7fcb96f28640 1 -- start start 2024-04-25T08:56:13.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.922+0000 7fcb96f28640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fcb900d1a20 0x7fcb900b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.922+0000 7fcb96f28640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcb900d1e00 0x7fcb900d2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.922+0000 7fcb96f28640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb9015bf90 con 0x7fcb900b5100 2024-04-25T08:56:13.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.922+0000 7fcb96f28640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb900cf430 con 0x7fcb900d1e00 2024-04-25T08:56:13.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.922+0000 7fcb96f28640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb900cee70 con 0x7fcb900d1a20 2024-04-25T08:56:13.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.922+0000 7fcb954c5640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7fcb900b5100 0x7fcb900b5570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:48166/0 (socket says 172.21.15.43:48166) 2024-04-25T08:56:13.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.922+0000 7fcb954c5640 1 -- 172.21.15.43:0/1466543610 learned_addr learned my addr 172.21.15.43:0/1466543610 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:56:13.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.922+0000 7fcb94cc4640 1 --2- 172.21.15.43:0/1466543610 >> v2:172.21.15.193:3300/0 conn(0x7fcb900d1a20 0x7fcb900b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.922+0000 7fcb8ffff640 1 --2- 172.21.15.43:0/1466543610 >> v2:172.21.15.43:3300/0 conn(0x7fcb900d1e00 0x7fcb900d2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.923+0000 7fcb8f7fe640 1 -- 172.21.15.43:0/1466543610 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2354691855 0 0) 0x7fcb9015bf90 con 0x7fcb900b5100 2024-04-25T08:56:13.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.923+0000 7fcb8f7fe640 1 -- 172.21.15.43:0/1466543610 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb80003520 con 0x7fcb900b5100 2024-04-25T08:56:13.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.923+0000 7fcb94cc4640 1 -- 172.21.15.43:0/1466543610 >> v2:172.21.15.43:3300/0 conn(0x7fcb900d1e00 msgr2=0x7fcb900d2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:13.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.923+0000 7fcb94cc4640 1 --2- 172.21.15.43:0/1466543610 >> v2:172.21.15.43:3300/0 conn(0x7fcb900d1e00 0x7fcb900d2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.923+0000 7fcb94cc4640 1 -- 172.21.15.43:0/1466543610 >> v1:172.21.15.193:6789/0 conn(0x7fcb900b5100 legacy=0x7fcb900b5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:13.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.923+0000 7fcb94cc4640 1 -- 172.21.15.43:0/1466543610 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb900cf150 con 0x7fcb900d1a20 2024-04-25T08:56:13.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.924+0000 7fcb94cc4640 1 --2- 172.21.15.43:0/1466543610 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb900d1a20 0x7fcb900b2b90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcb90074860 tx=0x7fcb8402a080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f30f7a1dadfa307b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:13.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.924+0000 7fcb8f7fe640 1 -- 172.21.15.43:0/1466543610 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb84037070 con 0x7fcb900d1a20 2024-04-25T08:56:13.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.924+0000 7fcb8f7fe640 1 -- 172.21.15.43:0/1466543610 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb8402add0 con 0x7fcb900d1a20 2024-04-25T08:56:13.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.924+0000 7fcb8f7fe640 1 -- 172.21.15.43:0/1466543610 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb84032660 con 0x7fcb900d1a20 2024-04-25T08:56:13.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.925+0000 7fcb96f28640 1 -- 172.21.15.43:0/1466543610 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb900d1a20 msgr2=0x7fcb900b2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:13.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.925+0000 7fcb96f28640 1 --2- 172.21.15.43:0/1466543610 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb900d1a20 0x7fcb900b2b90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcb90074860 tx=0x7fcb8402a080 comp rx=0 tx=0).stop 2024-04-25T08:56:13.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.925+0000 7fcb96f28640 1 -- 172.21.15.43:0/1466543610 shutdown_connections 2024-04-25T08:56:13.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.925+0000 7fcb96f28640 1 --2- 172.21.15.43:0/1466543610 >> v2:172.21.15.43:3300/0 conn(0x7fcb900d1e00 0x7fcb900d2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.925+0000 7fcb96f28640 1 --2- 172.21.15.43:0/1466543610 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb900d1a20 0x7fcb900b2b90 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.925+0000 7fcb96f28640 1 -- 172.21.15.43:0/1466543610 >> 172.21.15.43:0/1466543610 conn(0x7fcb90072aa0 msgr2=0x7fcb9009aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:13.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.925+0000 7fcb96f28640 1 -- 172.21.15.43:0/1466543610 shutdown_connections 2024-04-25T08:56:13.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.925+0000 7fcb96f28640 1 -- 172.21.15.43:0/1466543610 wait complete. 2024-04-25T08:56:13.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.927+0000 7fcb96f28640 1 Processor -- start 2024-04-25T08:56:13.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.927+0000 7fcb96f28640 1 -- start start 2024-04-25T08:56:13.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.928+0000 7fcb96f28640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb900b5100 0x7fcb901522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.928+0000 7fcb96f28640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcb900d1a20 0x7fcb9009d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.928+0000 7fcb96f28640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb900d1e00 0x7fcb900cc140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.928+0000 7fcb96f28640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb900cea60 con 0x7fcb900b5100 2024-04-25T08:56:13.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.928+0000 7fcb96f28640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb900b4be0 con 0x7fcb900d1e00 2024-04-25T08:56:13.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.928+0000 7fcb96f28640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb900cebe0 con 0x7fcb900d1a20 2024-04-25T08:56:13.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.929+0000 7fcb8ffff640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcb900d1a20 0x7fcb9009d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.929+0000 7fcb8ffff640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcb900d1a20 0x7fcb9009d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:56296/0 (socket says 172.21.15.43:56296) 2024-04-25T08:56:13.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.929+0000 7fcb8ffff640 1 -- 172.21.15.43:0/2072202987 learned_addr learned my addr 172.21.15.43:0/2072202987 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:13.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.929+0000 7fcb954c5640 1 --2- 172.21.15.43:0/2072202987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb900d1e00 0x7fcb900cc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.929+0000 7fcb94cc4640 1 --2- 172.21.15.43:0/2072202987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb900b5100 0x7fcb901522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.929+0000 7fcb954c5640 1 -- 172.21.15.43:0/2072202987 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcb900d1a20 msgr2=0x7fcb9009d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:13.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.929+0000 7fcb954c5640 1 --2- 172.21.15.43:0/2072202987 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcb900d1a20 0x7fcb9009d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.929+0000 7fcb954c5640 1 -- 172.21.15.43:0/2072202987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb900b5100 msgr2=0x7fcb901522d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:13.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.929+0000 7fcb954c5640 1 --2- 172.21.15.43:0/2072202987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb900b5100 0x7fcb901522d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.929+0000 7fcb954c5640 1 -- 172.21.15.43:0/2072202987 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb900b0fb0 con 0x7fcb900d1e00 2024-04-25T08:56:13.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.929+0000 7fcb8ffff640 1 --2- 172.21.15.43:0/2072202987 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcb900d1a20 0x7fcb9009d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:56:13.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.929+0000 7fcb94cc4640 1 --2- 172.21.15.43:0/2072202987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb900b5100 0x7fcb901522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:56:13.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.929+0000 7fcb954c5640 1 --2- 172.21.15.43:0/2072202987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb900d1e00 0x7fcb900cc140 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcb8800cab0 tx=0x7fcb8800cf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:13.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.930+0000 7fcb8d7fa640 1 -- 172.21.15.43:0/2072202987 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb88017070 con 0x7fcb900d1e00 2024-04-25T08:56:13.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.930+0000 7fcb8d7fa640 1 -- 172.21.15.43:0/2072202987 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb8800abd0 con 0x7fcb900d1e00 2024-04-25T08:56:13.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.930+0000 7fcb8d7fa640 1 -- 172.21.15.43:0/2072202987 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb88012400 con 0x7fcb900d1e00 2024-04-25T08:56:13.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.930+0000 7fcb96f28640 1 -- 172.21.15.43:0/2072202987 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb900b1290 con 0x7fcb900d1e00 2024-04-25T08:56:13.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.930+0000 7fcb96f28640 1 -- 172.21.15.43:0/2072202987 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb900b1850 con 0x7fcb900d1e00 2024-04-25T08:56:13.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.933+0000 7fcb8d7fa640 1 -- 172.21.15.43:0/2072202987 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcb8801a070 con 0x7fcb900d1e00 2024-04-25T08:56:13.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.934+0000 7fcb8d7fa640 1 --2- 172.21.15.43:0/2072202987 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcb7406b4d0 0x7fcb7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.934+0000 7fcb99c1f740 1 Processor -- start 2024-04-25T08:56:13.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.934+0000 7fcb94cc4640 1 --2- 172.21.15.43:0/2072202987 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcb7406b4d0 0x7fcb7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.934+0000 7fcb8d7fa640 1 -- 172.21.15.43:0/2072202987 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcb88089e20 con 0x7fcb900d1e00 2024-04-25T08:56:13.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.934+0000 7fcb99c1f740 1 -- start start 2024-04-25T08:56:13.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.935+0000 7fcb94cc4640 1 --2- 172.21.15.43:0/2072202987 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcb7406b4d0 0x7fcb7406d990 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcb84034000 tx=0x7fcb8402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:13.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.935+0000 7fcb99c1f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f3ce2fb10 0x564f3ce32320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.935+0000 7fcb99c1f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f3ce2fef0 0x564f3ce32860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.935+0000 7fcb99c1f740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564f3ce02770 0x564f3ce05f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.935+0000 7fcb99c1f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f3cd6bda0 con 0x564f3ce2fef0 2024-04-25T08:56:13.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.935+0000 7fcb99c1f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f3cd14b80 con 0x564f3ce2fb10 2024-04-25T08:56:13.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.935+0000 7fcb99c1f740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x564f3cd67e40 con 0x564f3ce02770 2024-04-25T08:56:13.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.935+0000 7fcb8ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f3ce2fef0 0x564f3ce32860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.935+0000 7fcb94cc4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f3ce2fb10 0x564f3ce32320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.935+0000 7fcb8ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f3ce2fef0 0x564f3ce32860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39852/0 (socket says 172.21.15.43:39852) 2024-04-25T08:56:13.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.935+0000 7fcb8ffff640 1 -- 172.21.15.43:0/297227282 learned_addr learned my addr 172.21.15.43:0/297227282 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:13.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.935+0000 7fcb94cc4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f3ce2fb10 0x564f3ce32320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:55848/0 (socket says 172.21.15.43:55848) 2024-04-25T08:56:13.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.935+0000 7fcb954c5640 1 --2- 172.21.15.43:0/297227282 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564f3ce02770 0x564f3ce05f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.936+0000 7fcb8ffff640 1 -- 172.21.15.43:0/297227282 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564f3ce02770 msgr2=0x564f3ce05f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:13.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.936+0000 7fcb8ffff640 1 --2- 172.21.15.43:0/297227282 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564f3ce02770 0x564f3ce05f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.936+0000 7fcb8ffff640 1 -- 172.21.15.43:0/297227282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f3ce2fb10 msgr2=0x564f3ce32320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:13.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.936+0000 7fcb8ffff640 1 --2- 172.21.15.43:0/297227282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f3ce2fb10 0x564f3ce32320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.936+0000 7fcb8ffff640 1 -- 172.21.15.43:0/297227282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f3cd1ef10 con 0x564f3ce2fef0 2024-04-25T08:56:13.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.936+0000 7fcb8ffff640 1 --2- 172.21.15.43:0/297227282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f3ce2fef0 0x564f3ce32860 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c009fd0 tx=0x7fcb7c0117d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a3552e9dcd8122d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:13.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.936+0000 7fcb7a7fc640 1 -- 172.21.15.43:0/297227282 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c014650 con 0x564f3ce2fef0 2024-04-25T08:56:13.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.936+0000 7fcb7a7fc640 1 -- 172.21.15.43:0/297227282 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb7c0147f0 con 0x564f3ce2fef0 2024-04-25T08:56:13.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.936+0000 7fcb7a7fc640 1 -- 172.21.15.43:0/297227282 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c014ba0 con 0x564f3ce2fef0 2024-04-25T08:56:13.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.937+0000 7fcb99c1f740 1 -- 172.21.15.43:0/297227282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f3ce2fef0 msgr2=0x564f3ce32860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:13.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.937+0000 7fcb99c1f740 1 --2- 172.21.15.43:0/297227282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f3ce2fef0 0x564f3ce32860 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c009fd0 tx=0x7fcb7c0117d0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.937+0000 7fcb99c1f740 1 -- 172.21.15.43:0/297227282 shutdown_connections 2024-04-25T08:56:13.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.937+0000 7fcb99c1f740 1 --2- 172.21.15.43:0/297227282 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564f3ce02770 0x564f3ce05f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.937+0000 7fcb99c1f740 1 --2- 172.21.15.43:0/297227282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f3ce2fef0 0x564f3ce32860 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.937+0000 7fcb99c1f740 1 --2- 172.21.15.43:0/297227282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f3ce2fb10 0x564f3ce32320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.937+0000 7fcb99c1f740 1 -- 172.21.15.43:0/297227282 >> 172.21.15.43:0/297227282 conn(0x564f3ce049e0 msgr2=0x564f3ce4d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:13.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.937+0000 7fcb99c1f740 1 -- 172.21.15.43:0/297227282 shutdown_connections 2024-04-25T08:56:13.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.937+0000 7fcb99c1f740 1 -- 172.21.15.43:0/297227282 wait complete. 2024-04-25T08:56:13.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.938+0000 7fcb99c1f740 1 -- start start 2024-04-25T08:56:13.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.938+0000 7fcb99c1f740 1 Processor -- start 2024-04-25T08:56:13.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.939+0000 7fcb99c1f740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564f3ce2fb10 0x564f3ce28450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.939+0000 7fcb99c1f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f3ce2fef0 0x564f3ce4f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.940+0000 7fcb99c1f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f3ce256a0 0x564f3ce265b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.940+0000 7fcb99c1f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f3ce07b90 con 0x564f3ce2fef0 2024-04-25T08:56:13.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.940+0000 7fcb99c1f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f3cd0a4b0 con 0x564f3ce256a0 2024-04-25T08:56:13.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.940+0000 7fcb99c1f740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x564f3ce07e70 con 0x564f3ce2fb10 2024-04-25T08:56:13.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.940+0000 7fcb94cc4640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564f3ce2fb10 0x564f3ce28450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.940+0000 7fcb94cc4640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564f3ce2fb10 0x564f3ce28450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:56322/0 (socket says 172.21.15.43:56322) 2024-04-25T08:56:13.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.940+0000 7fcb94cc4640 1 -- 172.21.15.43:0/3440890492 learned_addr learned my addr 172.21.15.43:0/3440890492 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:13.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.940+0000 7fcb8ffff640 1 --2- 172.21.15.43:0/3440890492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f3ce2fef0 0x564f3ce4f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.940+0000 7fcb99c1f740 10 client.0 _ll_register_callbacks cb 0x564f3cd4a180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:56:13.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.940+0000 7fcb954c5640 1 --2- 172.21.15.43:0/3440890492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f3ce256a0 0x564f3ce265b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.940+0000 7fcb94cc4640 1 -- 172.21.15.43:0/3440890492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f3ce256a0 msgr2=0x564f3ce265b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:13.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.940+0000 7fcb94cc4640 1 --2- 172.21.15.43:0/3440890492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f3ce256a0 0x564f3ce265b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.940+0000 7fcb94cc4640 1 -- 172.21.15.43:0/3440890492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f3ce2fef0 msgr2=0x564f3ce4f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:13.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.940+0000 7fcb94cc4640 1 --2- 172.21.15.43:0/3440890492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f3ce2fef0 0x564f3ce4f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.940+0000 7fcb94cc4640 1 -- 172.21.15.43:0/3440890492 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f3cb4a7d0 con 0x564f3ce2fb10 2024-04-25T08:56:13.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.940+0000 7fcb8ffff640 1 --2- 172.21.15.43:0/3440890492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f3ce2fef0 0x564f3ce4f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:56:13.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.941+0000 7fcb94cc4640 1 --2- 172.21.15.43:0/3440890492 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564f3ce2fb10 0x564f3ce28450 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcb84000c00 tx=0x7fcb84004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec50f648bd720efe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:13.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.941+0000 7fcb6f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.193:3301/0 2024-04-25T08:56:13.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.941+0000 7fcb6f7fe640 1 -- 172.21.15.43:0/3440890492 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8404e020 con 0x564f3ce2fb10 2024-04-25T08:56:13.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.941+0000 7fcb6f7fe640 1 -- 172.21.15.43:0/3440890492 <== mon.2 v2:172.21.15.193:3301/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcb8404a660 con 0x564f3ce2fb10 2024-04-25T08:56:13.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.941+0000 7fcb99c1f740 1 -- 172.21.15.43:0/3440890492 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb7c0147f0 con 0x564f3ce2fb10 2024-04-25T08:56:13.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.941+0000 7fcb6f7fe640 1 -- 172.21.15.43:0/3440890492 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb84037040 con 0x564f3ce2fb10 2024-04-25T08:56:13.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.941+0000 7fcb99c1f740 10 client.4955 fetch_fsmap learned FSMap version 14 2024-04-25T08:56:13.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.941+0000 7fcb99c1f740 1 -- 172.21.15.43:0/3440890492 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x564f3ca803f0 con 0x564f3ce2fb10 2024-04-25T08:56:13.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.941+0000 7fcb6f7fe640 1 -- 172.21.15.43:0/3440890492 <== mon.2 v2:172.21.15.193:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb8404a860 con 0x564f3ce2fb10 2024-04-25T08:56:13.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.941+0000 7fcb6f7fe640 1 -- 172.21.15.43:0/3440890492 <== mon.2 v2:172.21.15.193:3301/0 5 ==== fsmap(e 14) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcb8404aa10 con 0x564f3ce2fb10 2024-04-25T08:56:13.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.941+0000 7fcb99c1f740 10 client.4955 fetch_fsmap finished waiting for FSMap version 14 2024-04-25T08:56:13.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.941+0000 7fcb99c1f740 10 client.4955 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T08:56:13.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.941+0000 7fcb99c1f740 20 client.4955 populate_metadata read hostname 'smithi043' 2024-04-25T08:56:13.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.941+0000 7fcb99c1f740 1 --2- 172.21.15.43:0/3440890492 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x564f3ce50930 0x564f3ce50d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.942+0000 7fcb99c1f740 4 client.4955 mds_command: new command op to 5208 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:56:13.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.942+0000 7fcb99c1f740 1 -- 172.21.15.43:0/3440890492 --> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcb7c0147f0 con 0x564f3ce50930 2024-04-25T08:56:13.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.942+0000 7fcb8ffff640 1 --2- 172.21.15.43:0/3440890492 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x564f3ce50930 0x564f3ce50d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.942+0000 7fcb6f7fe640 10 client.4955 ms_handle_connect on v2:172.21.15.43:6833/242762542 2024-04-25T08:56:13.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.942+0000 7fcb8ffff640 1 --2- 172.21.15.43:0/3440890492 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x564f3ce50930 0x564f3ce50d10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6561bc4fd19ecdb7 server_cookie=c6648afbcc323b30 in_seq=0 out_seq=0 2024-04-25T08:56:13.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.943+0000 7fcb6f7fe640 1 -- 172.21.15.43:0/3440890492 <== mds.0 v2:172.21.15.43:6833/242762542 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7fcb7c0147f0 con 0x564f3ce50930 2024-04-25T08:56:13.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.943+0000 7fcb6f7fe640 10 client.4955 handle_command_reply: tid=0 2024-04-25T08:56:13.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.943+0000 7fcb99c1f740 1 client.4955 shutdown 2024-04-25T08:56:13.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.943+0000 7fcb99c1f740 1 -- 172.21.15.43:0/3440890492 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564f3ce2fb10 msgr2=0x564f3ce28450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:13.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.943+0000 7fcb99c1f740 1 --2- 172.21.15.43:0/3440890492 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564f3ce2fb10 0x564f3ce28450 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fcb84000c00 tx=0x7fcb84004730 comp rx=0 tx=0).stop 2024-04-25T08:56:13.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.943+0000 7fcb99c1f740 1 -- 172.21.15.43:0/3440890492 shutdown_connections 2024-04-25T08:56:13.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.943+0000 7fcb99c1f740 1 --2- 172.21.15.43:0/3440890492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f3ce256a0 0x564f3ce265b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.943+0000 7fcb99c1f740 1 --2- 172.21.15.43:0/3440890492 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x564f3ce50930 0x564f3ce50d10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.943+0000 7fcb6f7fe640 0 client.4955 ms_handle_reset on v2:172.21.15.43:6833/242762542 2024-04-25T08:56:13.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.943+0000 7fcb99c1f740 1 --2- 172.21.15.43:0/3440890492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f3ce2fef0 0x564f3ce4f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.943+0000 7fcb99c1f740 1 --2- 172.21.15.43:0/3440890492 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564f3ce2fb10 0x564f3ce28450 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.943+0000 7fcb99c1f740 1 -- 172.21.15.43:0/3440890492 >> 172.21.15.43:0/3440890492 conn(0x564f3ce02770 msgr2=0x564f3ce07330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:13.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.943+0000 7fcb99c1f740 1 -- 172.21.15.43:0/3440890492 shutdown_connections 2024-04-25T08:56:13.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.944+0000 7fcb99c1f740 1 -- 172.21.15.43:0/3440890492 wait complete. 2024-04-25T08:56:13.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.944+0000 7fcb99c1f740 20 client.4955 trim_cache size 0 max 16384 2024-04-25T08:56:13.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.972+0000 7fcb99c1f740 1 Processor -- start 2024-04-25T08:56:13.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.972+0000 7fcb99c1f740 1 -- start start 2024-04-25T08:56:13.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.973+0000 7fcb99c1f740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564f3ce256a0 0x564f3ce2f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.973+0000 7fcb99c1f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f3ce2fb10 0x564f3ce01600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.973+0000 7fcb99c1f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f3ce2fef0 0x564f3ce01b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.973+0000 7fcb99c1f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb901ffc90 con 0x564f3ce2fef0 2024-04-25T08:56:13.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.973+0000 7fcb99c1f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9009bc70 con 0x564f3ce2fb10 2024-04-25T08:56:13.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.973+0000 7fcb99c1f740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb900b6400 con 0x564f3ce256a0 2024-04-25T08:56:13.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.973+0000 7fcb8ffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f3ce2fb10 0x564f3ce01600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.973+0000 7fcb94cc4640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564f3ce256a0 0x564f3ce2f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.973+0000 7fcb8ffff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f3ce2fb10 0x564f3ce01600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:55874/0 (socket says 172.21.15.43:55874) 2024-04-25T08:56:13.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.973+0000 7fcb8ffff640 1 -- 172.21.15.43:0/3770554474 learned_addr learned my addr 172.21.15.43:0/3770554474 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:13.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.973+0000 7fcb954c5640 1 --2- 172.21.15.43:0/3770554474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f3ce2fef0 0x564f3ce01b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.974+0000 7fcb8ffff640 1 -- 172.21.15.43:0/3770554474 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564f3ce256a0 msgr2=0x564f3ce2f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:13.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.974+0000 7fcb8ffff640 1 --2- 172.21.15.43:0/3770554474 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564f3ce256a0 0x564f3ce2f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.974+0000 7fcb8ffff640 1 -- 172.21.15.43:0/3770554474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f3ce2fef0 msgr2=0x564f3ce01b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:13.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.974+0000 7fcb8ffff640 1 --2- 172.21.15.43:0/3770554474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f3ce2fef0 0x564f3ce01b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.974+0000 7fcb8ffff640 1 -- 172.21.15.43:0/3770554474 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564f3ca400f0 con 0x564f3ce2fb10 2024-04-25T08:56:13.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.974+0000 7fcb954c5640 1 --2- 172.21.15.43:0/3770554474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f3ce2fef0 0x564f3ce01b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:56:13.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.974+0000 7fcb8ffff640 1 --2- 172.21.15.43:0/3770554474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f3ce2fb10 0x564f3ce01600 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c019000 tx=0x7fcb7c016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6d7c31aa337ad7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:13.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.974+0000 7fcb79ffb640 1 -- 172.21.15.43:0/3770554474 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c02b230 con 0x564f3ce2fb10 2024-04-25T08:56:13.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.974+0000 7fcb79ffb640 1 -- 172.21.15.43:0/3770554474 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb7c02b3d0 con 0x564f3ce2fb10 2024-04-25T08:56:13.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.974+0000 7fcb79ffb640 1 -- 172.21.15.43:0/3770554474 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7c02b780 con 0x564f3ce2fb10 2024-04-25T08:56:13.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.975+0000 7fcb99c1f740 1 -- 172.21.15.43:0/3770554474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f3ce2fb10 msgr2=0x564f3ce01600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:13.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.975+0000 7fcb99c1f740 1 --2- 172.21.15.43:0/3770554474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f3ce2fb10 0x564f3ce01600 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c019000 tx=0x7fcb7c016000 comp rx=0 tx=0).stop 2024-04-25T08:56:13.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.975+0000 7fcb99c1f740 1 -- 172.21.15.43:0/3770554474 shutdown_connections 2024-04-25T08:56:13.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.975+0000 7fcb99c1f740 1 --2- 172.21.15.43:0/3770554474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f3ce2fef0 0x564f3ce01b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.975+0000 7fcb99c1f740 1 --2- 172.21.15.43:0/3770554474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f3ce2fb10 0x564f3ce01600 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.975+0000 7fcb99c1f740 1 --2- 172.21.15.43:0/3770554474 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564f3ce256a0 0x564f3ce2f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.975+0000 7fcb99c1f740 1 -- 172.21.15.43:0/3770554474 >> 172.21.15.43:0/3770554474 conn(0x564f3ce02770 msgr2=0x564f3ce02c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:13.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.976+0000 7fcb99c1f740 1 -- 172.21.15.43:0/3770554474 shutdown_connections 2024-04-25T08:56:13.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.976+0000 7fcb99c1f740 1 -- 172.21.15.43:0/3770554474 wait complete. 2024-04-25T08:56:13.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.976+0000 7fcb99c1f740 1 -- start start 2024-04-25T08:56:13.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.977+0000 7fcb99c1f740 1 Processor -- start 2024-04-25T08:56:13.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.978+0000 7fcb99c1f740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564f3ce256a0 0x564f3cdfdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.978+0000 7fcb99c1f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f3ce2fb10 0x564f3ce01600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.978+0000 7fcb99c1f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f3ce2fef0 0x564f3ce01b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.978+0000 7fcb99c1f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x564f3ce475f0 con 0x564f3ce2fb10 2024-04-25T08:56:13.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.978+0000 7fcb99c1f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb900b6a00 con 0x564f3ce2fef0 2024-04-25T08:56:13.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.978+0000 7fcb99c1f740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x564f3cd2d4b0 con 0x564f3ce256a0 2024-04-25T08:56:13.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.978+0000 7fcb99c1f740 10 client.0 _ll_register_callbacks cb 0x7fcb7c009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:56:13.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.978+0000 7fcb94cc4640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564f3ce256a0 0x564f3cdfdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.978+0000 7fcb94cc4640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564f3ce256a0 0x564f3cdfdc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:56330/0 (socket says 172.21.15.43:56330) 2024-04-25T08:56:13.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.978+0000 7fcb94cc4640 1 -- 172.21.15.43:0/4013723163 learned_addr learned my addr 172.21.15.43:0/4013723163 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:13.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.978+0000 7fcb8ffff640 1 --2- 172.21.15.43:0/4013723163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f3ce2fb10 0x564f3ce01600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.978+0000 7fcb954c5640 1 --2- 172.21.15.43:0/4013723163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f3ce2fef0 0x564f3ce01b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.979+0000 7fcb94cc4640 1 -- 172.21.15.43:0/4013723163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f3ce2fef0 msgr2=0x564f3ce01b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:13.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.979+0000 7fcb94cc4640 1 --2- 172.21.15.43:0/4013723163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f3ce2fef0 0x564f3ce01b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.979+0000 7fcb94cc4640 1 -- 172.21.15.43:0/4013723163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f3ce2fb10 msgr2=0x564f3ce01600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:13.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.979+0000 7fcb94cc4640 1 --2- 172.21.15.43:0/4013723163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f3ce2fb10 0x564f3ce01600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.979+0000 7fcb94cc4640 1 -- 172.21.15.43:0/4013723163 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564f3ce32a70 con 0x564f3ce256a0 2024-04-25T08:56:13.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.979+0000 7fcb8ffff640 1 --2- 172.21.15.43:0/4013723163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f3ce2fb10 0x564f3ce01600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:56:13.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.979+0000 7fcb6f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.193:3301/0 2024-04-25T08:56:13.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.979+0000 7fcb94cc4640 1 --2- 172.21.15.43:0/4013723163 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564f3ce256a0 0x564f3cdfdc80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcb840096a0 tx=0x7fcb84009190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=410d621e2714ffff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:13.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.979+0000 7fcb6f7fe640 1 -- 172.21.15.43:0/4013723163 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb84066030 con 0x564f3ce256a0 2024-04-25T08:56:13.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.979+0000 7fcb6f7fe640 1 -- 172.21.15.43:0/4013723163 <== mon.2 v2:172.21.15.193:3301/0 2 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcb84062d60 con 0x564f3ce256a0 2024-04-25T08:56:13.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.979+0000 7fcb99c1f740 1 -- 172.21.15.43:0/4013723163 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb7c02b3d0 con 0x564f3ce256a0 2024-04-25T08:56:13.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.980+0000 7fcb6f7fe640 1 -- 172.21.15.43:0/4013723163 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb8406f050 con 0x564f3ce256a0 2024-04-25T08:56:13.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.980+0000 7fcb99c1f740 10 client.4961 fetch_fsmap learned FSMap version 14 2024-04-25T08:56:13.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.980+0000 7fcb99c1f740 1 -- 172.21.15.43:0/4013723163 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x564f3ce4d6f0 con 0x564f3ce256a0 2024-04-25T08:56:13.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.980+0000 7fcb6f7fe640 1 -- 172.21.15.43:0/4013723163 <== mon.2 v2:172.21.15.193:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb8403c440 con 0x564f3ce256a0 2024-04-25T08:56:13.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.980+0000 7fcb6f7fe640 1 -- 172.21.15.43:0/4013723163 <== mon.2 v2:172.21.15.193:3301/0 5 ==== fsmap(e 14) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcb8403c5f0 con 0x564f3ce256a0 2024-04-25T08:56:13.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.980+0000 7fcb99c1f740 10 client.4961 fetch_fsmap finished waiting for FSMap version 14 2024-04-25T08:56:13.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.980+0000 7fcb99c1f740 10 client.4961 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T08:56:13.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.980+0000 7fcb99c1f740 20 client.4961 populate_metadata read hostname 'smithi043' 2024-04-25T08:56:13.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.980+0000 7fcb99c1f740 1 --2- 172.21.15.43:0/4013723163 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x564f3ce50930 0x564f3ce241c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:13.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.980+0000 7fcb99c1f740 4 client.4961 mds_command: new command op to 5208 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:56:13.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.980+0000 7fcb99c1f740 1 -- 172.21.15.43:0/4013723163 --> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x7fcb7c02b3d0 con 0x564f3ce50930 2024-04-25T08:56:13.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.980+0000 7fcb8ffff640 1 --2- 172.21.15.43:0/4013723163 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x564f3ce50930 0x564f3ce241c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:13.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.981+0000 7fcb6f7fe640 10 client.4961 ms_handle_connect on v2:172.21.15.43:6833/242762542 2024-04-25T08:56:13.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.981+0000 7fcb8ffff640 1 --2- 172.21.15.43:0/4013723163 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x564f3ce50930 0x564f3ce241c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21f78acafd94a5d2 server_cookie=9be1c14f152ffaec in_seq=0 out_seq=0 2024-04-25T08:56:13.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.981+0000 7fcb6f7fe640 1 -- 172.21.15.43:0/4013723163 <== mds.0 v2:172.21.15.43:6833/242762542 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+274 (crc 0 0 0) 0x7fcb7c02b3d0 con 0x564f3ce50930 2024-04-25T08:56:13.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.981+0000 7fcb6f7fe640 10 client.4961 handle_command_reply: tid=0 2024-04-25T08:56:13.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.981+0000 7fcb99c1f740 1 client.4961 shutdown 2024-04-25T08:56:13.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.982+0000 7fcb99c1f740 1 -- 172.21.15.43:0/4013723163 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564f3ce256a0 msgr2=0x564f3cdfdc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:13.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.982+0000 7fcb99c1f740 1 --2- 172.21.15.43:0/4013723163 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564f3ce256a0 0x564f3cdfdc80 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fcb840096a0 tx=0x7fcb84009190 comp rx=0 tx=0).stop 2024-04-25T08:56:13.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.982+0000 7fcb99c1f740 1 -- 172.21.15.43:0/4013723163 shutdown_connections 2024-04-25T08:56:13.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.982+0000 7fcb99c1f740 1 --2- 172.21.15.43:0/4013723163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564f3ce2fef0 0x564f3ce01b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.982+0000 7fcb99c1f740 1 --2- 172.21.15.43:0/4013723163 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x564f3ce50930 0x564f3ce241c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.982+0000 7fcb99c1f740 1 --2- 172.21.15.43:0/4013723163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564f3ce2fb10 0x564f3ce01600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.982+0000 7fcb99c1f740 1 --2- 172.21.15.43:0/4013723163 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564f3ce256a0 0x564f3cdfdc80 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.982+0000 7fcb99c1f740 1 -- 172.21.15.43:0/4013723163 >> 172.21.15.43:0/4013723163 conn(0x564f3ce02770 msgr2=0x564f3ce27170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:13.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.982+0000 7fcb6f7fe640 0 client.4961 ms_handle_reset on v2:172.21.15.43:6833/242762542 2024-04-25T08:56:13.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.982+0000 7fcb99c1f740 1 -- 172.21.15.43:0/4013723163 shutdown_connections 2024-04-25T08:56:13.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.982+0000 7fcb99c1f740 1 -- 172.21.15.43:0/4013723163 wait complete. 2024-04-25T08:56:13.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.982+0000 7fcb99c1f740 20 client.4961 trim_cache size 0 max 16384 2024-04-25T08:56:13.977 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T08:56:13.977 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "scrub active (603 inodes in the stack)", 2024-04-25T08:56:13.977 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-25T08:56:13.977 INFO:teuthology.orchestra.run.smithi043.stdout: "4df37e95-019e-47b0-8cef-9fc7f27ab48e": { 2024-04-25T08:56:13.977 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-25T08:56:13.977 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "4df37e95-019e-47b0-8cef-9fc7f27ab48e", 2024-04-25T08:56:13.977 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive,scrub_mdsdir" 2024-04-25T08:56:13.978 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:56:13.978 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:56:13.978 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T08:56:13.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.983+0000 7fcb96f28640 1 -- 172.21.15.43:0/2072202987 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcb7406b4d0 msgr2=0x7fcb7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:13.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.983+0000 7fcb96f28640 1 --2- 172.21.15.43:0/2072202987 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcb7406b4d0 0x7fcb7406d990 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcb84034000 tx=0x7fcb8402e000 comp rx=0 tx=0).stop 2024-04-25T08:56:13.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.983+0000 7fcb96f28640 1 -- 172.21.15.43:0/2072202987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb900d1e00 msgr2=0x7fcb900cc140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:13.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.983+0000 7fcb96f28640 1 --2- 172.21.15.43:0/2072202987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb900d1e00 0x7fcb900cc140 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcb8800cab0 tx=0x7fcb8800cf70 comp rx=0 tx=0).stop 2024-04-25T08:56:13.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.983+0000 7fcb96f28640 1 -- 172.21.15.43:0/2072202987 shutdown_connections 2024-04-25T08:56:13.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.983+0000 7fcb96f28640 1 --2- 172.21.15.43:0/2072202987 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcb7406b4d0 0x7fcb7406d990 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.983+0000 7fcb96f28640 1 --2- 172.21.15.43:0/2072202987 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb900d1e00 0x7fcb900cc140 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.983+0000 7fcb96f28640 1 --2- 172.21.15.43:0/2072202987 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcb900d1a20 0x7fcb9009d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.983+0000 7fcb96f28640 1 --2- 172.21.15.43:0/2072202987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb900b5100 0x7fcb901522d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:13.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.983+0000 7fcb96f28640 1 -- 172.21.15.43:0/2072202987 >> 172.21.15.43:0/2072202987 conn(0x7fcb90072aa0 msgr2=0x7fcb900cd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:13.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.983+0000 7fcb96f28640 1 -- 172.21.15.43:0/2072202987 shutdown_connections 2024-04-25T08:56:13.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:13.983+0000 7fcb96f28640 1 -- 172.21.15.43:0/2072202987 wait complete. 2024-04-25T08:56:13.988 INFO:tasks.cephfs.filesystem:scrub status for tag:4df37e95-019e-47b0-8cef-9fc7f27ab48e - {'path': '/', 'tag': '4df37e95-019e-47b0-8cef-9fc7f27ab48e', 'options': 'recursive,scrub_mdsdir'} 2024-04-25T08:56:14.053 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:56:14.056 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:56:43.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:56:43.989 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-04-25T08:56:44.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.116+0000 7ff4fe815640 1 Processor -- start 2024-04-25T08:56:44.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.116+0000 7ff4fe815640 1 -- start start 2024-04-25T08:56:44.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.116+0000 7ff4fe815640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff4f815e090 0x7ff4f8161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.116+0000 7ff4fe815640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4f8162610 con 0x7ff4f80b5c70 2024-04-25T08:56:44.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.116+0000 7ff4fe815640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4f80d1750 con 0x7ff4f8161fe0 2024-04-25T08:56:44.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.116+0000 7ff4fe815640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4f80d1da0 con 0x7ff4f815e090 2024-04-25T08:56:44.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.117+0000 7ff4fcdb2640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff4f80b5c70 0x7ff4f80d2230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39466/0 (socket says 172.21.15.43:39466) 2024-04-25T08:56:44.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.117+0000 7ff4fcdb2640 1 -- 172.21.15.43:0/461268613 learned_addr learned my addr 172.21.15.43:0/461268613 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:56:44.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.117+0000 7ff4f7fff640 1 --2- 172.21.15.43:0/461268613 >> v2:172.21.15.193:3300/0 conn(0x7ff4f815e090 0x7ff4f8161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.117+0000 7ff4f6ffd640 1 -- 172.21.15.43:0/461268613 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3636797442 0 0) 0x7ff4f8162610 con 0x7ff4f80b5c70 2024-04-25T08:56:44.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.117+0000 7ff4f6ffd640 1 -- 172.21.15.43:0/461268613 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4e0003520 con 0x7ff4f80b5c70 2024-04-25T08:56:44.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.117+0000 7ff4f6ffd640 1 -- 172.21.15.43:0/461268613 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1905451494 0 0) 0x7ff4f80d1750 con 0x7ff4f8161fe0 2024-04-25T08:56:44.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.117+0000 7ff4f6ffd640 1 -- 172.21.15.43:0/461268613 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4f8162610 con 0x7ff4f8161fe0 2024-04-25T08:56:44.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.117+0000 7ff4f7fff640 1 -- 172.21.15.43:0/461268613 >> v1:172.21.15.193:6789/0 conn(0x7ff4f8161fe0 legacy=0x7ff4f80b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.117+0000 7ff4f7fff640 1 -- 172.21.15.43:0/461268613 >> v1:172.21.15.43:6789/0 conn(0x7ff4f80b5c70 legacy=0x7ff4f80d2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.117+0000 7ff4f6ffd640 1 -- 172.21.15.43:0/461268613 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1299946322 0 0) 0x7ff4e0003520 con 0x7ff4f80b5c70 2024-04-25T08:56:44.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.117+0000 7ff4f7fff640 1 -- 172.21.15.43:0/461268613 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f80d1a30 con 0x7ff4f815e090 2024-04-25T08:56:44.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.118+0000 7ff4f7fff640 1 --2- 172.21.15.43:0/461268613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4f815e090 0x7ff4f8161aa0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff4e80097a0 tx=0x7ff4e802a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=234372ecb54ee555 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:44.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.118+0000 7ff4f6ffd640 1 -- 172.21.15.43:0/461268613 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8037070 con 0x7ff4f815e090 2024-04-25T08:56:44.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.118+0000 7ff4f6ffd640 1 -- 172.21.15.43:0/461268613 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e802ace0 con 0x7ff4f815e090 2024-04-25T08:56:44.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.118+0000 7ff4f6ffd640 1 -- 172.21.15.43:0/461268613 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f00019f0 con 0x7ff4f815e090 2024-04-25T08:56:44.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.119+0000 7ff4fe815640 1 -- 172.21.15.43:0/461268613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4f815e090 msgr2=0x7ff4f8161aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.119+0000 7ff4fe815640 1 --2- 172.21.15.43:0/461268613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4f815e090 0x7ff4f8161aa0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff4e80097a0 tx=0x7ff4e802a250 comp rx=0 tx=0).stop 2024-04-25T08:56:44.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.119+0000 7ff4fe815640 1 -- 172.21.15.43:0/461268613 shutdown_connections 2024-04-25T08:56:44.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.119+0000 7ff4fe815640 1 --2- 172.21.15.43:0/461268613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4f815e090 0x7ff4f8161aa0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.119+0000 7ff4fe815640 1 -- 172.21.15.43:0/461268613 >> 172.21.15.43:0/461268613 conn(0x7ff4f80d0560 msgr2=0x7ff4f80d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:44.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.119+0000 7ff4fe815640 1 -- 172.21.15.43:0/461268613 shutdown_connections 2024-04-25T08:56:44.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.119+0000 7ff4fe815640 1 -- 172.21.15.43:0/461268613 wait complete. 2024-04-25T08:56:44.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.120+0000 7ff4fe815640 1 Processor -- start 2024-04-25T08:56:44.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.120+0000 7ff4fe815640 1 -- start start 2024-04-25T08:56:44.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.121+0000 7ff4fe815640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4f80b5c70 0x7ff4f80ca200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.121+0000 7ff4fe815640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4f815e090 0x7ff4f80cd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.121+0000 7ff4fe815640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4f8161fe0 0x7ff4f80b3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.121+0000 7ff4fe815640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f80d2940 con 0x7ff4f8161fe0 2024-04-25T08:56:44.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.121+0000 7ff4fe815640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f80b3280 con 0x7ff4f80b5c70 2024-04-25T08:56:44.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.121+0000 7ff4fe815640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f80d2ac0 con 0x7ff4f815e090 2024-04-25T08:56:44.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.121+0000 7ff4fcdb2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4f8161fe0 0x7ff4f80b3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.121+0000 7ff4fcdb2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4f8161fe0 0x7ff4f80b3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59032/0 (socket says 172.21.15.43:59032) 2024-04-25T08:56:44.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.121+0000 7ff4fcdb2640 1 -- 172.21.15.43:0/1768084942 learned_addr learned my addr 172.21.15.43:0/1768084942 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:44.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.122+0000 7ff4f7fff640 1 --2- 172.21.15.43:0/1768084942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4f80b5c70 0x7ff4f80ca200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.122+0000 7ff4f77fe640 1 --2- 172.21.15.43:0/1768084942 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4f815e090 0x7ff4f80cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.122+0000 7ff4fcdb2640 1 -- 172.21.15.43:0/1768084942 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4f815e090 msgr2=0x7ff4f80cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.122+0000 7ff4fcdb2640 1 --2- 172.21.15.43:0/1768084942 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4f815e090 0x7ff4f80cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.122+0000 7ff4fcdb2640 1 -- 172.21.15.43:0/1768084942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4f80b5c70 msgr2=0x7ff4f80ca200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.122+0000 7ff4fcdb2640 1 --2- 172.21.15.43:0/1768084942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4f80b5c70 0x7ff4f80ca200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.122+0000 7ff4fcdb2640 1 -- 172.21.15.43:0/1768084942 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f8157ab0 con 0x7ff4f8161fe0 2024-04-25T08:56:44.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.122+0000 7ff4fcdb2640 1 --2- 172.21.15.43:0/1768084942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4f8161fe0 0x7ff4f80b3eb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0001350 tx=0x7ff4f000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:44.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.122+0000 7ff4f4ff9640 1 -- 172.21.15.43:0/1768084942 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0017070 con 0x7ff4f8161fe0 2024-04-25T08:56:44.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.122+0000 7ff4f4ff9640 1 -- 172.21.15.43:0/1768084942 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f0012400 con 0x7ff4f8161fe0 2024-04-25T08:56:44.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.122+0000 7ff4fe815640 1 -- 172.21.15.43:0/1768084942 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4f8158880 con 0x7ff4f8161fe0 2024-04-25T08:56:44.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.122+0000 7ff4f4ff9640 1 -- 172.21.15.43:0/1768084942 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0012830 con 0x7ff4f8161fe0 2024-04-25T08:56:44.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.122+0000 7ff4fe815640 1 -- 172.21.15.43:0/1768084942 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4f80ca740 con 0x7ff4f8161fe0 2024-04-25T08:56:44.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.124+0000 7ff4f4ff9640 1 -- 172.21.15.43:0/1768084942 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff4f001a070 con 0x7ff4f8161fe0 2024-04-25T08:56:44.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.124+0000 7ff4f4ff9640 1 --2- 172.21.15.43:0/1768084942 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff4e406b4d0 0x7ff4e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.124+0000 7ff4f4ff9640 1 -- 172.21.15.43:0/1768084942 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7ff4f008a080 con 0x7ff4f8161fe0 2024-04-25T08:56:44.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.124+0000 7ff4f7fff640 1 --2- 172.21.15.43:0/1768084942 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff4e406b4d0 0x7ff4e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.125+0000 7ff4f7fff640 1 --2- 172.21.15.43:0/1768084942 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff4e406b4d0 0x7ff4e406d990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8034000 tx=0x7ff4e802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:44.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.125+0000 7ff5015f9740 1 Processor -- start 2024-04-25T08:56:44.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.125+0000 7ff5015f9740 1 -- start start 2024-04-25T08:56:44.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.126+0000 7ff5015f9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b2eee2b10 0x558b2eee5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.126+0000 7ff5015f9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b2eee2ef0 0x558b2eee5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.126+0000 7ff5015f9740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558b2eeb5770 0x558b2eeb8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.126+0000 7ff5015f9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b2ee1eda0 con 0x558b2eee2b10 2024-04-25T08:56:44.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.126+0000 7ff5015f9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b2edc7b80 con 0x558b2eee2ef0 2024-04-25T08:56:44.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.126+0000 7ff5015f9740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x558b2ee1ae40 con 0x558b2eeb5770 2024-04-25T08:56:44.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.126+0000 7ff4f7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b2eee2b10 0x558b2eee5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.126+0000 7ff4f7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b2eee2b10 0x558b2eee5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59048/0 (socket says 172.21.15.43:59048) 2024-04-25T08:56:44.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.126+0000 7ff4f7fff640 1 -- 172.21.15.43:0/2820435673 learned_addr learned my addr 172.21.15.43:0/2820435673 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:44.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.126+0000 7ff4fcdb2640 1 --2- 172.21.15.43:0/2820435673 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558b2eeb5770 0x558b2eeb8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.126+0000 7ff4f77fe640 1 --2- 172.21.15.43:0/2820435673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b2eee2ef0 0x558b2eee5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.126+0000 7ff4f7fff640 1 -- 172.21.15.43:0/2820435673 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558b2eeb5770 msgr2=0x558b2eeb8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:44.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.126+0000 7ff4f7fff640 1 --2- 172.21.15.43:0/2820435673 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558b2eeb5770 0x558b2eeb8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.126+0000 7ff4f7fff640 1 -- 172.21.15.43:0/2820435673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b2eee2ef0 msgr2=0x558b2eee5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:44.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.126+0000 7ff4f7fff640 1 --2- 172.21.15.43:0/2820435673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b2eee2ef0 0x558b2eee5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.126+0000 7ff4f7fff640 1 -- 172.21.15.43:0/2820435673 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b2edd1f10 con 0x558b2eee2b10 2024-04-25T08:56:44.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.127+0000 7ff4f7fff640 1 --2- 172.21.15.43:0/2820435673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b2eee2b10 0x558b2eee5320 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8006f80 tx=0x7ff4e802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15257ef0cccc5c02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:44.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.127+0000 7ff4ddffb640 1 -- 172.21.15.43:0/2820435673 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e8037040 con 0x558b2eee2b10 2024-04-25T08:56:44.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.127+0000 7ff4ddffb640 1 -- 172.21.15.43:0/2820435673 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e803c780 con 0x558b2eee2b10 2024-04-25T08:56:44.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.127+0000 7ff4ddffb640 1 -- 172.21.15.43:0/2820435673 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e803cb30 con 0x558b2eee2b10 2024-04-25T08:56:44.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.127+0000 7ff5015f9740 1 -- 172.21.15.43:0/2820435673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b2eee2b10 msgr2=0x558b2eee5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.127+0000 7ff5015f9740 1 --2- 172.21.15.43:0/2820435673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b2eee2b10 0x558b2eee5320 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8006f80 tx=0x7ff4e802c000 comp rx=0 tx=0).stop 2024-04-25T08:56:44.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.127+0000 7ff5015f9740 1 -- 172.21.15.43:0/2820435673 shutdown_connections 2024-04-25T08:56:44.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.127+0000 7ff5015f9740 1 --2- 172.21.15.43:0/2820435673 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558b2eeb5770 0x558b2eeb8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.127+0000 7ff5015f9740 1 --2- 172.21.15.43:0/2820435673 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b2eee2ef0 0x558b2eee5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.127+0000 7ff5015f9740 1 --2- 172.21.15.43:0/2820435673 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b2eee2b10 0x558b2eee5320 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.127+0000 7ff5015f9740 1 -- 172.21.15.43:0/2820435673 >> 172.21.15.43:0/2820435673 conn(0x558b2eeb79e0 msgr2=0x558b2ef00320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:44.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.127+0000 7ff5015f9740 1 -- 172.21.15.43:0/2820435673 shutdown_connections 2024-04-25T08:56:44.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.127+0000 7ff5015f9740 1 -- 172.21.15.43:0/2820435673 wait complete. 2024-04-25T08:56:44.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.128+0000 7ff5015f9740 1 -- start start 2024-04-25T08:56:44.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.128+0000 7ff5015f9740 1 Processor -- start 2024-04-25T08:56:44.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.129+0000 7ff5015f9740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558b2eee2b10 0x558b2eed2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.129+0000 7ff5015f9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b2eee2ef0 0x558b2eefc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.129+0000 7ff5015f9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b2eefccd0 0x558b2eee8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.129+0000 7ff5015f9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b2eebab90 con 0x558b2eee2ef0 2024-04-25T08:56:44.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.129+0000 7ff5015f9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b2edbd4b0 con 0x558b2eefccd0 2024-04-25T08:56:44.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.129+0000 7ff5015f9740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x558b2eebae70 con 0x558b2eee2b10 2024-04-25T08:56:44.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.129+0000 7ff5015f9740 10 client.0 _ll_register_callbacks cb 0x558b2edfd180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:56:44.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.129+0000 7ff4f77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b2eee2ef0 0x558b2eefc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.129+0000 7ff4f77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b2eee2ef0 0x558b2eefc790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59062/0 (socket says 172.21.15.43:59062) 2024-04-25T08:56:44.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.129+0000 7ff4f77fe640 1 -- 172.21.15.43:0/296982232 learned_addr learned my addr 172.21.15.43:0/296982232 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:44.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.129+0000 7ff4f7fff640 1 --2- 172.21.15.43:0/296982232 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558b2eee2b10 0x558b2eed2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.129+0000 7ff4fcdb2640 1 --2- 172.21.15.43:0/296982232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b2eefccd0 0x558b2eee8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.129+0000 7ff4f77fe640 1 -- 172.21.15.43:0/296982232 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558b2eee2b10 msgr2=0x558b2eed2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:44.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.129+0000 7ff4f77fe640 1 --2- 172.21.15.43:0/296982232 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558b2eee2b10 0x558b2eed2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.129+0000 7ff4f77fe640 1 -- 172.21.15.43:0/296982232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b2eefccd0 msgr2=0x558b2eee8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:44.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.130+0000 7ff4f77fe640 1 --2- 172.21.15.43:0/296982232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b2eefccd0 0x558b2eee8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.130+0000 7ff4f77fe640 1 -- 172.21.15.43:0/296982232 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b2ebfd7d0 con 0x558b2eee2ef0 2024-04-25T08:56:44.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.130+0000 7ff4f77fe640 1 --2- 172.21.15.43:0/296982232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b2eee2ef0 0x558b2eefc790 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff4f81f6e90 tx=0x7ff4ec012750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a96424db35d81e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:44.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.130+0000 7ff4d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:56:44.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.130+0000 7ff4d6ffd640 1 -- 172.21.15.43:0/296982232 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec005320 con 0x558b2eee2ef0 2024-04-25T08:56:44.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.130+0000 7ff4d6ffd640 1 -- 172.21.15.43:0/296982232 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7ff4ec007030 con 0x558b2eee2ef0 2024-04-25T08:56:44.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.130+0000 7ff5015f9740 1 -- 172.21.15.43:0/296982232 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff4e803c780 con 0x558b2eee2ef0 2024-04-25T08:56:44.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.130+0000 7ff4d6ffd640 1 -- 172.21.15.43:0/296982232 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4ec007230 con 0x558b2eee2ef0 2024-04-25T08:56:44.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.130+0000 7ff5015f9740 10 client.5475 fetch_fsmap learned FSMap version 14 2024-04-25T08:56:44.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.130+0000 7ff5015f9740 1 -- 172.21.15.43:0/296982232 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x558b2eb333f0 con 0x558b2eee2ef0 2024-04-25T08:56:44.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.130+0000 7ff4d6ffd640 1 -- 172.21.15.43:0/296982232 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e803c780 con 0x558b2eee2ef0 2024-04-25T08:56:44.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.130+0000 7ff4d6ffd640 1 -- 172.21.15.43:0/296982232 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 14) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff4ec0073d0 con 0x558b2eee2ef0 2024-04-25T08:56:44.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.130+0000 7ff5015f9740 10 client.5475 fetch_fsmap finished waiting for FSMap version 14 2024-04-25T08:56:44.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.130+0000 7ff5015f9740 10 client.5475 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T08:56:44.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.130+0000 7ff5015f9740 20 client.5475 populate_metadata read hostname 'smithi043' 2024-04-25T08:56:44.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.130+0000 7ff5015f9740 1 --2- 172.21.15.43:0/296982232 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x558b2ef036c0 0x558b2ef03aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.130+0000 7ff5015f9740 4 client.5475 mds_command: new command op to 5208 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:56:44.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.130+0000 7ff5015f9740 1 -- 172.21.15.43:0/296982232 --> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558b2ee2c2e0 con 0x558b2ef036c0 2024-04-25T08:56:44.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.131+0000 7ff4f7fff640 1 --2- 172.21.15.43:0/296982232 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x558b2ef036c0 0x558b2ef03aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.131+0000 7ff4d6ffd640 10 client.5475 ms_handle_connect on v2:172.21.15.43:6833/242762542 2024-04-25T08:56:44.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.131+0000 7ff4f7fff640 1 --2- 172.21.15.43:0/296982232 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x558b2ef036c0 0x558b2ef03aa0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1baf2956a2c9d633 server_cookie=185ea39b79143819 in_seq=0 out_seq=0 2024-04-25T08:56:44.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.131+0000 7ff4d6ffd640 1 -- 172.21.15.43:0/296982232 <== mds.0 v2:172.21.15.43:6833/242762542 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558b2ee2c2e0 con 0x558b2ef036c0 2024-04-25T08:56:44.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.131+0000 7ff4d6ffd640 10 client.5475 handle_command_reply: tid=0 2024-04-25T08:56:44.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.131+0000 7ff5015f9740 1 client.5475 shutdown 2024-04-25T08:56:44.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.132+0000 7ff5015f9740 1 -- 172.21.15.43:0/296982232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b2eee2ef0 msgr2=0x558b2eefc790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.132+0000 7ff5015f9740 1 --2- 172.21.15.43:0/296982232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b2eee2ef0 0x558b2eefc790 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff4f81f6e90 tx=0x7ff4ec012750 comp rx=0 tx=0).stop 2024-04-25T08:56:44.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.132+0000 7ff5015f9740 1 -- 172.21.15.43:0/296982232 shutdown_connections 2024-04-25T08:56:44.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.132+0000 7ff5015f9740 1 --2- 172.21.15.43:0/296982232 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b2eefccd0 0x558b2eee8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.132+0000 7ff5015f9740 1 --2- 172.21.15.43:0/296982232 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x558b2ef036c0 0x558b2ef03aa0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.132+0000 7ff5015f9740 1 --2- 172.21.15.43:0/296982232 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b2eee2ef0 0x558b2eefc790 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.132+0000 7ff5015f9740 1 --2- 172.21.15.43:0/296982232 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558b2eee2b10 0x558b2eed2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.132+0000 7ff4d6ffd640 0 client.5475 ms_handle_reset on v2:172.21.15.43:6833/242762542 2024-04-25T08:56:44.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.132+0000 7ff5015f9740 1 -- 172.21.15.43:0/296982232 >> 172.21.15.43:0/296982232 conn(0x558b2eeb5770 msgr2=0x558b2eeb4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:44.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.132+0000 7ff5015f9740 1 -- 172.21.15.43:0/296982232 shutdown_connections 2024-04-25T08:56:44.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.132+0000 7ff5015f9740 1 -- 172.21.15.43:0/296982232 wait complete. 2024-04-25T08:56:44.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.132+0000 7ff5015f9740 20 client.5475 trim_cache size 0 max 16384 2024-04-25T08:56:44.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.147+0000 7ff5015f9740 1 Processor -- start 2024-04-25T08:56:44.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.147+0000 7ff5015f9740 1 -- start start 2024-04-25T08:56:44.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.148+0000 7ff5015f9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b2eee2b10 0x558b2eebc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.148+0000 7ff5015f9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b2eee2ef0 0x558b2eefc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.148+0000 7ff5015f9740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558b2eefccd0 0x558b2eeb4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.148+0000 7ff5015f9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f81ffc90 con 0x558b2eee2ef0 2024-04-25T08:56:44.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.148+0000 7ff5015f9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f81f7b80 con 0x558b2eee2b10 2024-04-25T08:56:44.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.148+0000 7ff5015f9740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f81f6f80 con 0x558b2eefccd0 2024-04-25T08:56:44.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.148+0000 7ff4f77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b2eee2ef0 0x558b2eefc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.148+0000 7ff4f77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b2eee2ef0 0x558b2eefc220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59074/0 (socket says 172.21.15.43:59074) 2024-04-25T08:56:44.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.148+0000 7ff4f77fe640 1 -- 172.21.15.43:0/3960815828 learned_addr learned my addr 172.21.15.43:0/3960815828 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:44.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.148+0000 7ff4f77fe640 1 -- 172.21.15.43:0/3960815828 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558b2eefccd0 msgr2=0x558b2eeb4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:44.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.148+0000 7ff4f77fe640 1 --2- 172.21.15.43:0/3960815828 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558b2eefccd0 0x558b2eeb4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.148+0000 7ff4f77fe640 1 -- 172.21.15.43:0/3960815828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b2eee2b10 msgr2=0x558b2eebc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:44.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.148+0000 7ff4f77fe640 1 --2- 172.21.15.43:0/3960815828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b2eee2b10 0x558b2eebc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.148+0000 7ff4f77fe640 1 -- 172.21.15.43:0/3960815828 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b2eaf30f0 con 0x558b2eee2ef0 2024-04-25T08:56:44.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.149+0000 7ff4f77fe640 1 --2- 172.21.15.43:0/3960815828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b2eee2ef0 0x558b2eefc220 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec005620 tx=0x7ff4ec00c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58c350ef9b16edf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:44.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.149+0000 7ff4dd7fa640 1 -- 172.21.15.43:0/3960815828 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec007db0 con 0x558b2eee2ef0 2024-04-25T08:56:44.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.149+0000 7ff4dd7fa640 1 -- 172.21.15.43:0/3960815828 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4ec0073d0 con 0x558b2eee2ef0 2024-04-25T08:56:44.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.149+0000 7ff4dd7fa640 1 -- 172.21.15.43:0/3960815828 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec006c40 con 0x558b2eee2ef0 2024-04-25T08:56:44.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.149+0000 7ff5015f9740 1 -- 172.21.15.43:0/3960815828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b2eee2ef0 msgr2=0x558b2eefc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.149+0000 7ff5015f9740 1 --2- 172.21.15.43:0/3960815828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b2eee2ef0 0x558b2eefc220 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec005620 tx=0x7ff4ec00c000 comp rx=0 tx=0).stop 2024-04-25T08:56:44.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.149+0000 7ff5015f9740 1 -- 172.21.15.43:0/3960815828 shutdown_connections 2024-04-25T08:56:44.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.149+0000 7ff5015f9740 1 --2- 172.21.15.43:0/3960815828 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558b2eefccd0 0x558b2eeb4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.149+0000 7ff5015f9740 1 --2- 172.21.15.43:0/3960815828 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b2eee2ef0 0x558b2eefc220 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.149+0000 7ff5015f9740 1 --2- 172.21.15.43:0/3960815828 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b2eee2b10 0x558b2eebc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.149+0000 7ff5015f9740 1 -- 172.21.15.43:0/3960815828 >> 172.21.15.43:0/3960815828 conn(0x558b2eeb5770 msgr2=0x558b2eebb7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:44.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.149+0000 7ff5015f9740 1 -- 172.21.15.43:0/3960815828 shutdown_connections 2024-04-25T08:56:44.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.149+0000 7ff5015f9740 1 -- 172.21.15.43:0/3960815828 wait complete. 2024-04-25T08:56:44.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.150+0000 7ff5015f9740 1 -- start start 2024-04-25T08:56:44.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.150+0000 7ff5015f9740 1 Processor -- start 2024-04-25T08:56:44.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.151+0000 7ff5015f9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b2eee2b10 0x558b2ef60ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.151+0000 7ff5015f9740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558b2eee2ef0 0x558b2ef6a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.151+0000 7ff5015f9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b2eefccd0 0x558b2ef008a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.151+0000 7ff5015f9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f81f7580 con 0x558b2eefccd0 2024-04-25T08:56:44.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.151+0000 7ff5015f9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b2ef6acb0 con 0x558b2eee2b10 2024-04-25T08:56:44.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.151+0000 7ff5015f9740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x558b2eef66a0 con 0x558b2eee2ef0 2024-04-25T08:56:44.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.151+0000 7ff5015f9740 10 client.0 _ll_register_callbacks cb 0x7ff4e00036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:56:44.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.151+0000 7ff4fcdb2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b2eefccd0 0x558b2ef008a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.151+0000 7ff4fcdb2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b2eefccd0 0x558b2ef008a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59080/0 (socket says 172.21.15.43:59080) 2024-04-25T08:56:44.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.151+0000 7ff4fcdb2640 1 -- 172.21.15.43:0/724765816 learned_addr learned my addr 172.21.15.43:0/724765816 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:44.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.151+0000 7ff4f7fff640 1 --2- 172.21.15.43:0/724765816 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b2eee2b10 0x558b2ef60ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.151+0000 7ff4f77fe640 1 --2- 172.21.15.43:0/724765816 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558b2eee2ef0 0x558b2ef6a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.151+0000 7ff4fcdb2640 1 -- 172.21.15.43:0/724765816 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558b2eee2ef0 msgr2=0x558b2ef6a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:44.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.151+0000 7ff4fcdb2640 1 --2- 172.21.15.43:0/724765816 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558b2eee2ef0 0x558b2ef6a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.151+0000 7ff4fcdb2640 1 -- 172.21.15.43:0/724765816 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b2eee2b10 msgr2=0x558b2ef60ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:44.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.151+0000 7ff4fcdb2640 1 --2- 172.21.15.43:0/724765816 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b2eee2b10 0x558b2ef60ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.151+0000 7ff4fcdb2640 1 -- 172.21.15.43:0/724765816 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b2ef193d0 con 0x558b2eefccd0 2024-04-25T08:56:44.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.151+0000 7ff4fcdb2640 1 --2- 172.21.15.43:0/724765816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b2eefccd0 0x558b2ef008a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0005c00 tx=0x7ff4f005ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e65004d1870c14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:44.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.151+0000 7ff4d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:56:44.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.152+0000 7ff4d6ffd640 1 -- 172.21.15.43:0/724765816 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0056c60 con 0x558b2eefccd0 2024-04-25T08:56:44.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.152+0000 7ff4d6ffd640 1 -- 172.21.15.43:0/724765816 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7ff4f00701c0 con 0x558b2eefccd0 2024-04-25T08:56:44.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.152+0000 7ff5015f9740 1 -- 172.21.15.43:0/724765816 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff4ec0073d0 con 0x558b2eefccd0 2024-04-25T08:56:44.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.152+0000 7ff4d6ffd640 1 -- 172.21.15.43:0/724765816 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4f0004b60 con 0x558b2eefccd0 2024-04-25T08:56:44.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.152+0000 7ff5015f9740 10 client.5481 fetch_fsmap learned FSMap version 14 2024-04-25T08:56:44.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.152+0000 7ff5015f9740 1 -- 172.21.15.43:0/724765816 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x558b2eefa290 con 0x558b2eefccd0 2024-04-25T08:56:44.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.152+0000 7ff4d6ffd640 1 -- 172.21.15.43:0/724765816 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4ec0073d0 con 0x558b2eefccd0 2024-04-25T08:56:44.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.152+0000 7ff4d6ffd640 1 -- 172.21.15.43:0/724765816 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 14) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff4f0057cf0 con 0x558b2eefccd0 2024-04-25T08:56:44.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.152+0000 7ff5015f9740 10 client.5481 fetch_fsmap finished waiting for FSMap version 14 2024-04-25T08:56:44.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.152+0000 7ff5015f9740 10 client.5481 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T08:56:44.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.152+0000 7ff5015f9740 20 client.5481 populate_metadata read hostname 'smithi043' 2024-04-25T08:56:44.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.152+0000 7ff5015f9740 1 --2- 172.21.15.43:0/724765816 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x558b2ef00de0 0x558b2eee7480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.152+0000 7ff5015f9740 4 client.5481 mds_command: new command op to 5208 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:56:44.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.152+0000 7ff5015f9740 1 -- 172.21.15.43:0/724765816 --> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558b2ee26630 con 0x558b2ef00de0 2024-04-25T08:56:44.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.152+0000 7ff4f7fff640 1 --2- 172.21.15.43:0/724765816 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x558b2ef00de0 0x558b2eee7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.152+0000 7ff4d6ffd640 10 client.5481 ms_handle_connect on v2:172.21.15.43:6833/242762542 2024-04-25T08:56:44.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.153+0000 7ff4f7fff640 1 --2- 172.21.15.43:0/724765816 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x558b2ef00de0 0x558b2eee7480 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19b86dfbcf981ae1 server_cookie=c51372216b093b2d in_seq=0 out_seq=0 2024-04-25T08:56:44.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.153+0000 7ff4d6ffd640 1 -- 172.21.15.43:0/724765816 <== mds.0 v2:172.21.15.43:6833/242762542 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558b2ee26630 con 0x558b2ef00de0 2024-04-25T08:56:44.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.153+0000 7ff4d6ffd640 10 client.5481 handle_command_reply: tid=0 2024-04-25T08:56:44.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.153+0000 7ff5015f9740 1 client.5481 shutdown 2024-04-25T08:56:44.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.153+0000 7ff5015f9740 1 -- 172.21.15.43:0/724765816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b2eefccd0 msgr2=0x558b2ef008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.153+0000 7ff5015f9740 1 --2- 172.21.15.43:0/724765816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b2eefccd0 0x558b2ef008a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0005c00 tx=0x7ff4f005ca00 comp rx=0 tx=0).stop 2024-04-25T08:56:44.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.153+0000 7ff5015f9740 1 -- 172.21.15.43:0/724765816 shutdown_connections 2024-04-25T08:56:44.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.153+0000 7ff5015f9740 1 --2- 172.21.15.43:0/724765816 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x558b2ef00de0 0x558b2eee7480 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.153+0000 7ff5015f9740 1 --2- 172.21.15.43:0/724765816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b2eefccd0 0x558b2ef008a0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.153+0000 7ff5015f9740 1 --2- 172.21.15.43:0/724765816 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558b2eee2ef0 0x558b2ef6a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.153+0000 7ff5015f9740 1 --2- 172.21.15.43:0/724765816 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b2eee2b10 0x558b2ef60ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.153+0000 7ff4d6ffd640 0 client.5481 ms_handle_reset on v2:172.21.15.43:6833/242762542 2024-04-25T08:56:44.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.153+0000 7ff5015f9740 1 -- 172.21.15.43:0/724765816 >> 172.21.15.43:0/724765816 conn(0x558b2eeb5770 msgr2=0x558b2eefc220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:44.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.153+0000 7ff5015f9740 1 -- 172.21.15.43:0/724765816 shutdown_connections 2024-04-25T08:56:44.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.153+0000 7ff5015f9740 1 -- 172.21.15.43:0/724765816 wait complete. 2024-04-25T08:56:44.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.153+0000 7ff5015f9740 20 client.5481 trim_cache size 0 max 16384 2024-04-25T08:56:44.149 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T08:56:44.149 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-25T08:56:44.149 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-25T08:56:44.149 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T08:56:44.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.154+0000 7ff4fe815640 1 -- 172.21.15.43:0/1768084942 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff4e406b4d0 msgr2=0x7ff4e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.154+0000 7ff4fe815640 1 --2- 172.21.15.43:0/1768084942 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff4e406b4d0 0x7ff4e406d990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8034000 tx=0x7ff4e802e000 comp rx=0 tx=0).stop 2024-04-25T08:56:44.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.154+0000 7ff4fe815640 1 -- 172.21.15.43:0/1768084942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4f8161fe0 msgr2=0x7ff4f80b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.154+0000 7ff4fe815640 1 --2- 172.21.15.43:0/1768084942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4f8161fe0 0x7ff4f80b3eb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0001350 tx=0x7ff4f000cf90 comp rx=0 tx=0).stop 2024-04-25T08:56:44.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.154+0000 7ff4fe815640 1 -- 172.21.15.43:0/1768084942 shutdown_connections 2024-04-25T08:56:44.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.154+0000 7ff4fe815640 1 --2- 172.21.15.43:0/1768084942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff4f8161fe0 0x7ff4f80b3eb0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.154+0000 7ff4fe815640 1 --2- 172.21.15.43:0/1768084942 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff4f815e090 0x7ff4f80cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.154+0000 7ff4fe815640 1 --2- 172.21.15.43:0/1768084942 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff4e406b4d0 0x7ff4e406d990 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.154+0000 7ff4fe815640 1 --2- 172.21.15.43:0/1768084942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff4f80b5c70 0x7ff4f80ca200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.154+0000 7ff4fe815640 1 -- 172.21.15.43:0/1768084942 >> 172.21.15.43:0/1768084942 conn(0x7ff4f80d0560 msgr2=0x7ff4f80b3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:44.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.154+0000 7ff4fe815640 1 -- 172.21.15.43:0/1768084942 shutdown_connections 2024-04-25T08:56:44.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.154+0000 7ff4fe815640 1 -- 172.21.15.43:0/1768084942 wait complete. 2024-04-25T08:56:44.154 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T08:56:44.154 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:56:44.155 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 perf dump 2024-04-25T08:56:44.200 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:56:44.203 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:56:44.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.268+0000 7fde60812640 1 Processor -- start 2024-04-25T08:56:44.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.268+0000 7fde60812640 1 -- start start 2024-04-25T08:56:44.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.268+0000 7fde60812640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fde5c15e090 0x7fde5c161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.269+0000 7fde60812640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde5c162610 con 0x7fde5c161fe0 2024-04-25T08:56:44.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.269+0000 7fde60812640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde5c0d1750 con 0x7fde5c0b5c70 2024-04-25T08:56:44.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.269+0000 7fde60812640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde5c0d1da0 con 0x7fde5c15e090 2024-04-25T08:56:44.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.269+0000 7fde5a59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fde5c15e090 0x7fde5c161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.269+0000 7fde5a59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fde5c15e090 0x7fde5c161aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59084/0 (socket says 172.21.15.43:59084) 2024-04-25T08:56:44.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.269+0000 7fde59d9b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fde5c161fe0 0x7fde5c0b5560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39470/0 (socket says 172.21.15.43:39470) 2024-04-25T08:56:44.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.269+0000 7fde5a59c640 1 -- 172.21.15.43:0/2434246252 learned_addr learned my addr 172.21.15.43:0/2434246252 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:44.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.269+0000 7fde5a59c640 1 -- 172.21.15.43:0/2434246252 >> v1:172.21.15.193:6789/0 conn(0x7fde5c0b5c70 legacy=0x7fde5c0d2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.269+0000 7fde5a59c640 1 -- 172.21.15.43:0/2434246252 >> v1:172.21.15.43:6789/0 conn(0x7fde5c161fe0 legacy=0x7fde5c0b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.269+0000 7fde5959a640 1 -- 172.21.15.43:0/2434246252 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3298721415 0 0) 0x7fde5c162610 con 0x7fde5c161fe0 2024-04-25T08:56:44.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.269+0000 7fde5a59c640 1 -- 172.21.15.43:0/2434246252 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde5c0d1a30 con 0x7fde5c15e090 2024-04-25T08:56:44.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.270+0000 7fde5a59c640 1 --2- 172.21.15.43:0/2434246252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde5c15e090 0x7fde5c161aa0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fde4c009e50 tx=0x7fde4c02a840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74fa8ffb881d116f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:44.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.270+0000 7fde5959a640 1 -- 172.21.15.43:0/2434246252 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4c037070 con 0x7fde5c15e090 2024-04-25T08:56:44.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.270+0000 7fde5959a640 1 -- 172.21.15.43:0/2434246252 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde4c032630 con 0x7fde5c15e090 2024-04-25T08:56:44.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.270+0000 7fde5959a640 1 -- 172.21.15.43:0/2434246252 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4c02e050 con 0x7fde5c15e090 2024-04-25T08:56:44.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.270+0000 7fde60812640 1 -- 172.21.15.43:0/2434246252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde5c15e090 msgr2=0x7fde5c161aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.270+0000 7fde60812640 1 --2- 172.21.15.43:0/2434246252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde5c15e090 0x7fde5c161aa0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fde4c009e50 tx=0x7fde4c02a840 comp rx=0 tx=0).stop 2024-04-25T08:56:44.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.271+0000 7fde60812640 1 -- 172.21.15.43:0/2434246252 shutdown_connections 2024-04-25T08:56:44.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.271+0000 7fde60812640 1 --2- 172.21.15.43:0/2434246252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde5c15e090 0x7fde5c161aa0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.271+0000 7fde60812640 1 -- 172.21.15.43:0/2434246252 >> 172.21.15.43:0/2434246252 conn(0x7fde5c0d0560 msgr2=0x7fde5c0d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:44.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.271+0000 7fde60812640 1 -- 172.21.15.43:0/2434246252 shutdown_connections 2024-04-25T08:56:44.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.271+0000 7fde60812640 1 -- 172.21.15.43:0/2434246252 wait complete. 2024-04-25T08:56:44.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.272+0000 7fde60812640 1 Processor -- start 2024-04-25T08:56:44.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.272+0000 7fde60812640 1 -- start start 2024-04-25T08:56:44.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.273+0000 7fde60812640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fde5c0b5c70 0x7fde5c15b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.273+0000 7fde60812640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde5c15e090 0x7fde5c0cec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.273+0000 7fde60812640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde5c161fe0 0x7fde5c0b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.273+0000 7fde60812640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c0d2940 con 0x7fde5c161fe0 2024-04-25T08:56:44.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.273+0000 7fde60812640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c0b3280 con 0x7fde5c15e090 2024-04-25T08:56:44.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.273+0000 7fde60812640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c0d2ac0 con 0x7fde5c0b5c70 2024-04-25T08:56:44.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.273+0000 7fde5ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde5c161fe0 0x7fde5c0b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.273+0000 7fde5ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde5c161fe0 0x7fde5c0b3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59092/0 (socket says 172.21.15.43:59092) 2024-04-25T08:56:44.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.273+0000 7fde5ad9d640 1 -- 172.21.15.43:0/548198445 learned_addr learned my addr 172.21.15.43:0/548198445 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:44.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.273+0000 7fde59d9b640 1 --2- 172.21.15.43:0/548198445 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde5c15e090 0x7fde5c0cec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.273+0000 7fde5a59c640 1 --2- 172.21.15.43:0/548198445 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fde5c0b5c70 0x7fde5c15b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.273+0000 7fde5ad9d640 1 -- 172.21.15.43:0/548198445 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fde5c0b5c70 msgr2=0x7fde5c15b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.273+0000 7fde5ad9d640 1 --2- 172.21.15.43:0/548198445 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fde5c0b5c70 0x7fde5c15b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.273+0000 7fde5ad9d640 1 -- 172.21.15.43:0/548198445 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde5c15e090 msgr2=0x7fde5c0cec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.273+0000 7fde5ad9d640 1 --2- 172.21.15.43:0/548198445 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde5c15e090 0x7fde5c0cec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.273+0000 7fde5ad9d640 1 -- 172.21.15.43:0/548198445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde5c0aa4f0 con 0x7fde5c161fe0 2024-04-25T08:56:44.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.273+0000 7fde59d9b640 1 --2- 172.21.15.43:0/548198445 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde5c15e090 0x7fde5c0cec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:44.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.273+0000 7fde5ad9d640 1 --2- 172.21.15.43:0/548198445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde5c161fe0 0x7fde5c0b3f60 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fde54001eb0 tx=0x7fde5400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:44.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.273+0000 7fde52ffd640 1 -- 172.21.15.43:0/548198445 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde54017070 con 0x7fde5c161fe0 2024-04-25T08:56:44.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.273+0000 7fde52ffd640 1 -- 172.21.15.43:0/548198445 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde540052d0 con 0x7fde5c161fe0 2024-04-25T08:56:44.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.273+0000 7fde52ffd640 1 -- 172.21.15.43:0/548198445 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde54003810 con 0x7fde5c161fe0 2024-04-25T08:56:44.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.273+0000 7fde60812640 1 -- 172.21.15.43:0/548198445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde5c0aa7d0 con 0x7fde5c161fe0 2024-04-25T08:56:44.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.274+0000 7fde60812640 1 -- 172.21.15.43:0/548198445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde5c0aad90 con 0x7fde5c161fe0 2024-04-25T08:56:44.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.275+0000 7fde52ffd640 1 -- 172.21.15.43:0/548198445 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fde5401a070 con 0x7fde5c161fe0 2024-04-25T08:56:44.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.275+0000 7fde52ffd640 1 --2- 172.21.15.43:0/548198445 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fde3c06b4d0 0x7fde3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.276+0000 7fde52ffd640 1 -- 172.21.15.43:0/548198445 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fde54089530 con 0x7fde5c161fe0 2024-04-25T08:56:44.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.276+0000 7fde5a59c640 1 --2- 172.21.15.43:0/548198445 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fde3c06b4d0 0x7fde3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.276+0000 7fde5a59c640 1 --2- 172.21.15.43:0/548198445 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fde3c06b4d0 0x7fde3c06d990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fde4c034000 tx=0x7fde4c009000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:44.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.277+0000 7fde635ef740 1 Processor -- start 2024-04-25T08:56:44.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.277+0000 7fde635ef740 1 -- start start 2024-04-25T08:56:44.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.278+0000 7fde635ef740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590df011ca0 0x5590df013e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.278+0000 7fde635ef740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5590df04e370 0x5590df012080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.278+0000 7fde635ef740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590df03c760 0x5590df03cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.278+0000 7fde635ef740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590def74b90 con 0x5590df03c760 2024-04-25T08:56:44.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.278+0000 7fde635ef740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590def1de40 con 0x5590df011ca0 2024-04-25T08:56:44.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.278+0000 7fde635ef740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x5590def70c30 con 0x5590df04e370 2024-04-25T08:56:44.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.278+0000 7fde5ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590df03c760 0x5590df03cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.278+0000 7fde5ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590df03c760 0x5590df03cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59108/0 (socket says 172.21.15.43:59108) 2024-04-25T08:56:44.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.278+0000 7fde5ad9d640 1 -- 172.21.15.43:0/2063517873 learned_addr learned my addr 172.21.15.43:0/2063517873 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:44.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.278+0000 7fde59d9b640 1 --2- 172.21.15.43:0/2063517873 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5590df04e370 0x5590df012080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.278+0000 7fde5a59c640 1 --2- 172.21.15.43:0/2063517873 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590df011ca0 0x5590df013e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.278+0000 7fde5ad9d640 1 -- 172.21.15.43:0/2063517873 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5590df04e370 msgr2=0x5590df012080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:44.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.278+0000 7fde5ad9d640 1 --2- 172.21.15.43:0/2063517873 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5590df04e370 0x5590df012080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.278+0000 7fde5ad9d640 1 -- 172.21.15.43:0/2063517873 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590df011ca0 msgr2=0x5590df013e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:44.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.278+0000 7fde5ad9d640 1 --2- 172.21.15.43:0/2063517873 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590df011ca0 0x5590df013e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.278+0000 7fde5ad9d640 1 -- 172.21.15.43:0/2063517873 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590ded537d0 con 0x5590df03c760 2024-04-25T08:56:44.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.278+0000 7fde5ad9d640 1 --2- 172.21.15.43:0/2063517873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590df03c760 0x5590df03cbe0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fde54087a70 tx=0x7fde540577b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edebb1c4621cb67c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:44.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.278+0000 7fde3bfff640 1 -- 172.21.15.43:0/2063517873 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde54087e60 con 0x5590df03c760 2024-04-25T08:56:44.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.278+0000 7fde3bfff640 1 -- 172.21.15.43:0/2063517873 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde5405e0c0 con 0x5590df03c760 2024-04-25T08:56:44.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.278+0000 7fde3bfff640 1 -- 172.21.15.43:0/2063517873 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde54054390 con 0x5590df03c760 2024-04-25T08:56:44.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.279+0000 7fde635ef740 1 -- 172.21.15.43:0/2063517873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590df03c760 msgr2=0x5590df03cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.279+0000 7fde635ef740 1 --2- 172.21.15.43:0/2063517873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590df03c760 0x5590df03cbe0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fde54087a70 tx=0x7fde540577b0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.279+0000 7fde635ef740 1 -- 172.21.15.43:0/2063517873 shutdown_connections 2024-04-25T08:56:44.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.279+0000 7fde635ef740 1 --2- 172.21.15.43:0/2063517873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590df03c760 0x5590df03cbe0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.279+0000 7fde635ef740 1 --2- 172.21.15.43:0/2063517873 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5590df04e370 0x5590df012080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.279+0000 7fde635ef740 1 --2- 172.21.15.43:0/2063517873 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590df011ca0 0x5590df013e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.279+0000 7fde635ef740 1 -- 172.21.15.43:0/2063517873 >> 172.21.15.43:0/2063517873 conn(0x5590defd9870 msgr2=0x5590df057760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:44.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.279+0000 7fde635ef740 1 -- 172.21.15.43:0/2063517873 shutdown_connections 2024-04-25T08:56:44.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.279+0000 7fde635ef740 1 -- 172.21.15.43:0/2063517873 wait complete. 2024-04-25T08:56:44.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.280+0000 7fde635ef740 1 -- start start 2024-04-25T08:56:44.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.280+0000 7fde635ef740 1 Processor -- start 2024-04-25T08:56:44.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.280+0000 7fde635ef740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5590df011ca0 0x5590df023090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.281+0000 7fde635ef740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590df03c760 0x5590df033310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.281+0000 7fde635ef740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590df04e370 0x5590df023830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.281+0000 7fde635ef740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590def313e0 con 0x5590df04e370 2024-04-25T08:56:44.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.281+0000 7fde635ef740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590df03a2b0 con 0x5590df03c760 2024-04-25T08:56:44.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.281+0000 7fde635ef740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x5590df0242b0 con 0x5590df011ca0 2024-04-25T08:56:44.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.281+0000 7fde635ef740 10 client.0 _ll_register_callbacks cb 0x5590def30db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:56:44.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.281+0000 7fde5ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590df04e370 0x5590df023830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.281+0000 7fde5ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590df04e370 0x5590df023830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59116/0 (socket says 172.21.15.43:59116) 2024-04-25T08:56:44.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.281+0000 7fde5ad9d640 1 -- 172.21.15.43:0/4282377473 learned_addr learned my addr 172.21.15.43:0/4282377473 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:44.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.281+0000 7fde59d9b640 1 --2- 172.21.15.43:0/4282377473 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590df03c760 0x5590df033310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.281+0000 7fde5a59c640 1 --2- 172.21.15.43:0/4282377473 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5590df011ca0 0x5590df023090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.281+0000 7fde5ad9d640 1 -- 172.21.15.43:0/4282377473 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5590df011ca0 msgr2=0x5590df023090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:44.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.281+0000 7fde5ad9d640 1 --2- 172.21.15.43:0/4282377473 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5590df011ca0 0x5590df023090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.281+0000 7fde5ad9d640 1 -- 172.21.15.43:0/4282377473 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590df03c760 msgr2=0x5590df033310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:44.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.281+0000 7fde5ad9d640 1 --2- 172.21.15.43:0/4282377473 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590df03c760 0x5590df033310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.281+0000 7fde5ad9d640 1 -- 172.21.15.43:0/4282377473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590dec490f0 con 0x5590df04e370 2024-04-25T08:56:44.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.281+0000 7fde5ad9d640 1 --2- 172.21.15.43:0/4282377473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590df04e370 0x5590df023830 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fde54057cb0 tx=0x7fde54000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28759f066260f82e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:44.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.281+0000 7fde397fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:56:44.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.281+0000 7fde397fa640 1 -- 172.21.15.43:0/4282377473 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde54087e60 con 0x5590df04e370 2024-04-25T08:56:44.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.281+0000 7fde397fa640 1 -- 172.21.15.43:0/4282377473 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fde540890e0 con 0x5590df04e370 2024-04-25T08:56:44.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.281+0000 7fde635ef740 1 -- 172.21.15.43:0/4282377473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fde5405e0c0 con 0x5590df04e370 2024-04-25T08:56:44.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.281+0000 7fde397fa640 1 -- 172.21.15.43:0/4282377473 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde54053fc0 con 0x5590df04e370 2024-04-25T08:56:44.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.281+0000 7fde635ef740 10 client.5496 fetch_fsmap learned FSMap version 14 2024-04-25T08:56:44.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.281+0000 7fde635ef740 1 -- 172.21.15.43:0/4282377473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x5590dedb3f60 con 0x5590df04e370 2024-04-25T08:56:44.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.282+0000 7fde397fa640 1 -- 172.21.15.43:0/4282377473 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde5405e0c0 con 0x5590df04e370 2024-04-25T08:56:44.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.282+0000 7fde397fa640 1 -- 172.21.15.43:0/4282377473 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 14) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fde5406bce0 con 0x5590df04e370 2024-04-25T08:56:44.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.282+0000 7fde635ef740 10 client.5496 fetch_fsmap finished waiting for FSMap version 14 2024-04-25T08:56:44.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.282+0000 7fde635ef740 10 client.5496 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T08:56:44.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.282+0000 7fde635ef740 20 client.5496 populate_metadata read hostname 'smithi043' 2024-04-25T08:56:44.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.282+0000 7fde635ef740 1 --2- 172.21.15.43:0/4282377473 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x5590df036db0 0x5590df03e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.282+0000 7fde635ef740 4 client.5496 mds_command: new command op to 5208 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:56:44.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.282+0000 7fde635ef740 1 -- 172.21.15.43:0/4282377473 --> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5590def820d0 con 0x5590df036db0 2024-04-25T08:56:44.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.282+0000 7fde5a59c640 1 --2- 172.21.15.43:0/4282377473 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x5590df036db0 0x5590df03e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.282+0000 7fde397fa640 10 client.5496 ms_handle_connect on v2:172.21.15.43:6833/242762542 2024-04-25T08:56:44.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.282+0000 7fde5a59c640 1 --2- 172.21.15.43:0/4282377473 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x5590df036db0 0x5590df03e040 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3efdd9fe1bef44b server_cookie=4c40abbea2957561 in_seq=0 out_seq=0 2024-04-25T08:56:44.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.283+0000 7fde397fa640 1 -- 172.21.15.43:0/4282377473 <== mds.0 v2:172.21.15.43:6833/242762542 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5590def820d0 con 0x5590df036db0 2024-04-25T08:56:44.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.283+0000 7fde397fa640 10 client.5496 handle_command_reply: tid=0 2024-04-25T08:56:44.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.283+0000 7fde635ef740 1 client.5496 shutdown 2024-04-25T08:56:44.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.283+0000 7fde635ef740 1 -- 172.21.15.43:0/4282377473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590df04e370 msgr2=0x5590df023830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.283+0000 7fde635ef740 1 --2- 172.21.15.43:0/4282377473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590df04e370 0x5590df023830 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fde54057cb0 tx=0x7fde54000ed0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.283+0000 7fde635ef740 1 -- 172.21.15.43:0/4282377473 shutdown_connections 2024-04-25T08:56:44.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.283+0000 7fde635ef740 1 --2- 172.21.15.43:0/4282377473 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x5590df036db0 0x5590df03e040 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.283+0000 7fde635ef740 1 --2- 172.21.15.43:0/4282377473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590df04e370 0x5590df023830 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.283+0000 7fde635ef740 1 --2- 172.21.15.43:0/4282377473 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590df03c760 0x5590df033310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.283+0000 7fde397fa640 0 client.5496 ms_handle_reset on v2:172.21.15.43:6833/242762542 2024-04-25T08:56:44.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.283+0000 7fde635ef740 1 --2- 172.21.15.43:0/4282377473 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5590df011ca0 0x5590df023090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.283+0000 7fde635ef740 1 -- 172.21.15.43:0/4282377473 >> 172.21.15.43:0/4282377473 conn(0x5590defd9870 msgr2=0x5590df05ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:44.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.283+0000 7fde635ef740 1 -- 172.21.15.43:0/4282377473 shutdown_connections 2024-04-25T08:56:44.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.283+0000 7fde635ef740 1 -- 172.21.15.43:0/4282377473 wait complete. 2024-04-25T08:56:44.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.283+0000 7fde635ef740 20 client.5496 trim_cache size 0 max 16384 2024-04-25T08:56:44.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.297+0000 7fde635ef740 1 Processor -- start 2024-04-25T08:56:44.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.297+0000 7fde635ef740 1 -- start start 2024-04-25T08:56:44.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.298+0000 7fde635ef740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590df011ca0 0x5590df07a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.298+0000 7fde635ef740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590df03c760 0x5590df013d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.298+0000 7fde635ef740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5590df04e370 0x5590df0324f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.298+0000 7fde635ef740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c1ffd60 con 0x5590df03c760 2024-04-25T08:56:44.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.298+0000 7fde635ef740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c1f4860 con 0x5590df011ca0 2024-04-25T08:56:44.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.298+0000 7fde635ef740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c1f3c60 con 0x5590df04e370 2024-04-25T08:56:44.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.298+0000 7fde59d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590df03c760 0x5590df013d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.298+0000 7fde59d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590df03c760 0x5590df013d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59128/0 (socket says 172.21.15.43:59128) 2024-04-25T08:56:44.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.298+0000 7fde59d9b640 1 -- 172.21.15.43:0/1960303339 learned_addr learned my addr 172.21.15.43:0/1960303339 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:44.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.298+0000 7fde5ad9d640 1 --2- 172.21.15.43:0/1960303339 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5590df04e370 0x5590df0324f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.298+0000 7fde5a59c640 1 --2- 172.21.15.43:0/1960303339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590df011ca0 0x5590df07a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.298+0000 7fde59d9b640 1 -- 172.21.15.43:0/1960303339 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5590df04e370 msgr2=0x5590df0324f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:44.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.298+0000 7fde59d9b640 1 --2- 172.21.15.43:0/1960303339 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5590df04e370 0x5590df0324f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.298+0000 7fde59d9b640 1 -- 172.21.15.43:0/1960303339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590df011ca0 msgr2=0x5590df07a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:44.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.298+0000 7fde59d9b640 1 --2- 172.21.15.43:0/1960303339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590df011ca0 0x5590df07a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.298+0000 7fde59d9b640 1 -- 172.21.15.43:0/1960303339 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5590dec221b0 con 0x5590df03c760 2024-04-25T08:56:44.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.299+0000 7fde59d9b640 1 --2- 172.21.15.43:0/1960303339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590df03c760 0x5590df013d50 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fde440077d0 tx=0x7fde44005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76820356c94c3df4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:44.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.299+0000 7fde3b7fe640 1 -- 172.21.15.43:0/1960303339 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde44019070 con 0x5590df03c760 2024-04-25T08:56:44.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.299+0000 7fde3b7fe640 1 -- 172.21.15.43:0/1960303339 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde44012ab0 con 0x5590df03c760 2024-04-25T08:56:44.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.299+0000 7fde3b7fe640 1 -- 172.21.15.43:0/1960303339 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde44012e60 con 0x5590df03c760 2024-04-25T08:56:44.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.299+0000 7fde635ef740 1 -- 172.21.15.43:0/1960303339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590df03c760 msgr2=0x5590df013d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.299+0000 7fde635ef740 1 --2- 172.21.15.43:0/1960303339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590df03c760 0x5590df013d50 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fde440077d0 tx=0x7fde44005c80 comp rx=0 tx=0).stop 2024-04-25T08:56:44.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.299+0000 7fde635ef740 1 -- 172.21.15.43:0/1960303339 shutdown_connections 2024-04-25T08:56:44.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.299+0000 7fde635ef740 1 --2- 172.21.15.43:0/1960303339 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5590df04e370 0x5590df0324f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.299+0000 7fde635ef740 1 --2- 172.21.15.43:0/1960303339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590df03c760 0x5590df013d50 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.299+0000 7fde635ef740 1 --2- 172.21.15.43:0/1960303339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590df011ca0 0x5590df07a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.299+0000 7fde635ef740 1 -- 172.21.15.43:0/1960303339 >> 172.21.15.43:0/1960303339 conn(0x5590defd9870 msgr2=0x5590df05b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:44.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.299+0000 7fde635ef740 1 -- 172.21.15.43:0/1960303339 shutdown_connections 2024-04-25T08:56:44.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.299+0000 7fde635ef740 1 -- 172.21.15.43:0/1960303339 wait complete. 2024-04-25T08:56:44.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.300+0000 7fde635ef740 1 -- start start 2024-04-25T08:56:44.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.300+0000 7fde635ef740 1 Processor -- start 2024-04-25T08:56:44.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.301+0000 7fde635ef740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590df011ca0 0x5590df0385a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.301+0000 7fde635ef740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5590df03c760 0x5590df030e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.301+0000 7fde635ef740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590df04e370 0x5590df033930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.301+0000 7fde635ef740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x5590df06a4c0 con 0x5590df011ca0 2024-04-25T08:56:44.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.301+0000 7fde635ef740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde5c1f4260 con 0x5590df04e370 2024-04-25T08:56:44.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.301+0000 7fde635ef740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x5590df033040 con 0x5590df03c760 2024-04-25T08:56:44.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.301+0000 7fde635ef740 10 client.0 _ll_register_callbacks cb 0x7fde54058420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:56:44.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.301+0000 7fde5a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590df011ca0 0x5590df0385a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.301+0000 7fde5a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590df011ca0 0x5590df0385a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59136/0 (socket says 172.21.15.43:59136) 2024-04-25T08:56:44.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.301+0000 7fde5a59c640 1 -- 172.21.15.43:0/3325252653 learned_addr learned my addr 172.21.15.43:0/3325252653 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:44.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.301+0000 7fde59d9b640 1 --2- 172.21.15.43:0/3325252653 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5590df03c760 0x5590df030e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.301+0000 7fde5ad9d640 1 --2- 172.21.15.43:0/3325252653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590df04e370 0x5590df033930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.301+0000 7fde5a59c640 1 -- 172.21.15.43:0/3325252653 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5590df03c760 msgr2=0x5590df030e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:44.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.301+0000 7fde5a59c640 1 --2- 172.21.15.43:0/3325252653 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5590df03c760 0x5590df030e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.301+0000 7fde5a59c640 1 -- 172.21.15.43:0/3325252653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590df04e370 msgr2=0x5590df033930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:44.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.301+0000 7fde5a59c640 1 --2- 172.21.15.43:0/3325252653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590df04e370 0x5590df033930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.301+0000 7fde5a59c640 1 -- 172.21.15.43:0/3325252653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5590df0c2b20 con 0x5590df011ca0 2024-04-25T08:56:44.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.301+0000 7fde5a59c640 1 --2- 172.21.15.43:0/3325252653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590df011ca0 0x5590df0385a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fde4c009e50 tx=0x7fde4c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88bc136fc4520487 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:44.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.301+0000 7fde397fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:56:44.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.302+0000 7fde397fa640 1 -- 172.21.15.43:0/3325252653 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4c037040 con 0x5590df011ca0 2024-04-25T08:56:44.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.302+0000 7fde397fa640 1 -- 172.21.15.43:0/3325252653 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fde4c04d440 con 0x5590df011ca0 2024-04-25T08:56:44.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.302+0000 7fde635ef740 1 -- 172.21.15.43:0/3325252653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fde44012ab0 con 0x5590df011ca0 2024-04-25T08:56:44.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.302+0000 7fde397fa640 1 -- 172.21.15.43:0/3325252653 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7fde4c05a050 con 0x5590df011ca0 2024-04-25T08:56:44.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.302+0000 7fde635ef740 10 client.5502 fetch_fsmap learned FSMap version 14 2024-04-25T08:56:44.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.302+0000 7fde635ef740 1 -- 172.21.15.43:0/3325252653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x5590df0345c0 con 0x5590df011ca0 2024-04-25T08:56:44.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.302+0000 7fde397fa640 1 -- 172.21.15.43:0/3325252653 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde44012ab0 con 0x5590df011ca0 2024-04-25T08:56:44.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.302+0000 7fde397fa640 1 -- 172.21.15.43:0/3325252653 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 14) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fde4c05d020 con 0x5590df011ca0 2024-04-25T08:56:44.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.302+0000 7fde635ef740 10 client.5502 fetch_fsmap finished waiting for FSMap version 14 2024-04-25T08:56:44.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.302+0000 7fde635ef740 10 client.5502 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.c 2024-04-25T08:56:44.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.302+0000 7fde635ef740 20 client.5502 populate_metadata read hostname 'smithi043' 2024-04-25T08:56:44.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.302+0000 7fde635ef740 1 --2- 172.21.15.43:0/3325252653 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x5590df069900 0x5590df0c5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.302+0000 7fde635ef740 4 client.5502 mds_command: new command op to 5208 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T08:56:44.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.302+0000 7fde635ef740 1 -- 172.21.15.43:0/3325252653 --> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x5590def7c420 con 0x5590df069900 2024-04-25T08:56:44.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.302+0000 7fde59d9b640 1 --2- 172.21.15.43:0/3325252653 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x5590df069900 0x5590df0c5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.302+0000 7fde397fa640 10 client.5502 ms_handle_connect on v2:172.21.15.43:6833/242762542 2024-04-25T08:56:44.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.302+0000 7fde59d9b640 1 --2- 172.21.15.43:0/3325252653 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x5590df069900 0x5590df0c5520 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a06067c5b11de4 server_cookie=46705868d9b8568e in_seq=0 out_seq=0 2024-04-25T08:56:44.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.303+0000 7fde397fa640 1 -- 172.21.15.43:0/3325252653 <== mds.0 v2:172.21.15.43:6833/242762542 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17949 (crc 0 0 0) 0x5590def7c420 con 0x5590df069900 2024-04-25T08:56:44.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.303+0000 7fde397fa640 10 client.5502 handle_command_reply: tid=0 2024-04-25T08:56:44.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.303+0000 7fde635ef740 1 client.5502 shutdown 2024-04-25T08:56:44.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.303+0000 7fde635ef740 1 -- 172.21.15.43:0/3325252653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590df011ca0 msgr2=0x5590df0385a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.303+0000 7fde635ef740 1 --2- 172.21.15.43:0/3325252653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590df011ca0 0x5590df0385a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fde4c009e50 tx=0x7fde4c004ab0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.303+0000 7fde635ef740 1 -- 172.21.15.43:0/3325252653 shutdown_connections 2024-04-25T08:56:44.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.303+0000 7fde635ef740 1 --2- 172.21.15.43:0/3325252653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5590df04e370 0x5590df033930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.303+0000 7fde635ef740 1 --2- 172.21.15.43:0/3325252653 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5590df03c760 0x5590df030e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.303+0000 7fde635ef740 1 --2- 172.21.15.43:0/3325252653 >> [v2:172.21.15.43:6833/242762542,v1:172.21.15.43:6835/242762542] conn(0x5590df069900 0x5590df0c5520 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.303+0000 7fde635ef740 1 --2- 172.21.15.43:0/3325252653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5590df011ca0 0x5590df0385a0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.303+0000 7fde397fa640 0 client.5502 ms_handle_reset on v2:172.21.15.43:6833/242762542 2024-04-25T08:56:44.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.303+0000 7fde635ef740 1 -- 172.21.15.43:0/3325252653 >> 172.21.15.43:0/3325252653 conn(0x5590defd9870 msgr2=0x5590df013ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:44.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.303+0000 7fde635ef740 1 -- 172.21.15.43:0/3325252653 shutdown_connections 2024-04-25T08:56:44.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.303+0000 7fde635ef740 1 -- 172.21.15.43:0/3325252653 wait complete. 2024-04-25T08:56:44.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.303+0000 7fde635ef740 20 client.5502 trim_cache size 0 max 16384 2024-04-25T08:56:44.300 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T08:56:44.300 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T08:56:44.300 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 5096, 2024-04-25T08:56:44.301 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 5097, 2024-04-25T08:56:44.301 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 1271672, 2024-04-25T08:56:44.301 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 35284707, 2024-04-25T08:56:44.301 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 17, 2024-04-25T08:56:44.301 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 6, 2024-04-25T08:56:44.301 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.427469721, 2024-04-25T08:56:44.301 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.189485487, 2024-04-25T08:56:44.301 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.086453088, 2024-04-25T08:56:44.301 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.122294573, 2024-04-25T08:56:44.301 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T08:56:44.301 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 5095, 2024-04-25T08:56:44.301 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.462797538, 2024-04-25T08:56:44.301 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000090833 2024-04-25T08:56:44.301 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.301 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-25T08:56:44.302 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 10, 2024-04-25T08:56:44.302 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000001073, 2024-04-25T08:56:44.302 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000107 2024-04-25T08:56:44.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.302 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T08:56:44.302 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T08:56:44.302 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.302 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T08:56:44.302 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 34162, 2024-04-25T08:56:44.302 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 60080, 2024-04-25T08:56:44.302 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 11921206, 2024-04-25T08:56:44.302 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 41590096, 2024-04-25T08:56:44.302 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 8, 2024-04-25T08:56:44.303 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 4, 2024-04-25T08:56:44.303 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 2.973798073, 2024-04-25T08:56:44.303 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 1.484310794, 2024-04-25T08:56:44.303 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 1.741889878, 2024-04-25T08:56:44.303 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.051691999, 2024-04-25T08:56:44.303 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T08:56:44.303 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 60079, 2024-04-25T08:56:44.303 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 1.107849405, 2024-04-25T08:56:44.303 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000018439 2024-04-25T08:56:44.303 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.304 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-25T08:56:44.304 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 88575, 2024-04-25T08:56:44.304 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.013831713, 2024-04-25T08:56:44.304 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000156 2024-04-25T08:56:44.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.304 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T08:56:44.304 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 121056 2024-04-25T08:56:44.304 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.304 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T08:56:44.304 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 5265, 2024-04-25T08:56:44.304 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 5258, 2024-04-25T08:56:44.304 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 1775165, 2024-04-25T08:56:44.304 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 36731786, 2024-04-25T08:56:44.304 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 5, 2024-04-25T08:56:44.304 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 4, 2024-04-25T08:56:44.305 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.386283433, 2024-04-25T08:56:44.305 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.146214953, 2024-04-25T08:56:44.305 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.094652200, 2024-04-25T08:56:44.305 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.123961232, 2024-04-25T08:56:44.305 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T08:56:44.305 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 5258, 2024-04-25T08:56:44.305 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.481678912, 2024-04-25T08:56:44.305 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000091608 2024-04-25T08:56:44.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.305 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-25T08:56:44.305 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:44.305 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:44.305 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:44.305 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.305 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 121776, 2024-04-25T08:56:44.305 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 27008 2024-04-25T08:56:44.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.306 INFO:teuthology.orchestra.run.smithi043.stdout: "cct": { 2024-04-25T08:56:44.306 INFO:teuthology.orchestra.run.smithi043.stdout: "total_workers": 1, 2024-04-25T08:56:44.306 INFO:teuthology.orchestra.run.smithi043.stdout: "unhealthy_workers": 0 2024-04-25T08:56:44.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.306 INFO:teuthology.orchestra.run.smithi043.stdout: "finisher-MDSRank": { 2024-04-25T08:56:44.306 INFO:teuthology.orchestra.run.smithi043.stdout: "queue_len": 0, 2024-04-25T08:56:44.306 INFO:teuthology.orchestra.run.smithi043.stdout: "complete_latency": { 2024-04-25T08:56:44.306 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 10506, 2024-04-25T08:56:44.306 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 12.866656598, 2024-04-25T08:56:44.306 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.001224696 2024-04-25T08:56:44.306 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:56:44.306 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.307 INFO:teuthology.orchestra.run.smithi043.stdout: "finisher-PurgeQueue": { 2024-04-25T08:56:44.307 INFO:teuthology.orchestra.run.smithi043.stdout: "queue_len": 0, 2024-04-25T08:56:44.307 INFO:teuthology.orchestra.run.smithi043.stdout: "complete_latency": { 2024-04-25T08:56:44.307 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 3636, 2024-04-25T08:56:44.307 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.648177216, 2024-04-25T08:56:44.307 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000178266 2024-04-25T08:56:44.307 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:56:44.307 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.307 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": { 2024-04-25T08:56:44.307 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 28251, 2024-04-25T08:56:44.307 INFO:teuthology.orchestra.run.smithi043.stdout: "reply": 28251, 2024-04-25T08:56:44.307 INFO:teuthology.orchestra.run.smithi043.stdout: "reply_latency": { 2024-04-25T08:56:44.307 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 28251, 2024-04-25T08:56:44.307 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 243.042693723, 2024-04-25T08:56:44.308 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.008602976 2024-04-25T08:56:44.308 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.308 INFO:teuthology.orchestra.run.smithi043.stdout: "slow_reply": 0, 2024-04-25T08:56:44.308 INFO:teuthology.orchestra.run.smithi043.stdout: "forward": 0, 2024-04-25T08:56:44.308 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_fetch_complete": 0, 2024-04-25T08:56:44.308 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_fetch_keys": 0, 2024-04-25T08:56:44.308 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_commit": 12, 2024-04-25T08:56:44.308 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_split": 0, 2024-04-25T08:56:44.308 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_merge": 0, 2024-04-25T08:56:44.308 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes": 17818, 2024-04-25T08:56:44.308 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_top": 0, 2024-04-25T08:56:44.308 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_bottom": 17818, 2024-04-25T08:56:44.308 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_pin_tail": 0, 2024-04-25T08:56:44.308 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_pinned": 17818, 2024-04-25T08:56:44.308 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_expired": 0, 2024-04-25T08:56:44.309 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_with_caps": 2, 2024-04-25T08:56:44.309 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 2, 2024-04-25T08:56:44.309 INFO:teuthology.orchestra.run.smithi043.stdout: "subtrees": 2, 2024-04-25T08:56:44.309 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse": 46870, 2024-04-25T08:56:44.309 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_hit": 46382, 2024-04-25T08:56:44.309 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_forward": 0, 2024-04-25T08:56:44.309 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_discover": 0, 2024-04-25T08:56:44.309 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_dir_fetch": 0, 2024-04-25T08:56:44.309 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_remote_ino": 0, 2024-04-25T08:56:44.309 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_lock": 168, 2024-04-25T08:56:44.309 INFO:teuthology.orchestra.run.smithi043.stdout: "load_cent": 0, 2024-04-25T08:56:44.309 INFO:teuthology.orchestra.run.smithi043.stdout: "q": 0, 2024-04-25T08:56:44.309 INFO:teuthology.orchestra.run.smithi043.stdout: "exported": 0, 2024-04-25T08:56:44.309 INFO:teuthology.orchestra.run.smithi043.stdout: "exported_inodes": 0, 2024-04-25T08:56:44.309 INFO:teuthology.orchestra.run.smithi043.stdout: "imported": 0, 2024-04-25T08:56:44.309 INFO:teuthology.orchestra.run.smithi043.stdout: "imported_inodes": 0, 2024-04-25T08:56:44.310 INFO:teuthology.orchestra.run.smithi043.stdout: "openino_dir_fetch": 0, 2024-04-25T08:56:44.310 INFO:teuthology.orchestra.run.smithi043.stdout: "openino_backtrace_fetch": 1766, 2024-04-25T08:56:44.310 INFO:teuthology.orchestra.run.smithi043.stdout: "openino_peer_discover": 0, 2024-04-25T08:56:44.310 INFO:teuthology.orchestra.run.smithi043.stdout: "root_rfiles": 0, 2024-04-25T08:56:44.310 INFO:teuthology.orchestra.run.smithi043.stdout: "root_rbytes": 0, 2024-04-25T08:56:44.310 INFO:teuthology.orchestra.run.smithi043.stdout: "root_rsnaps": 0, 2024-04-25T08:56:44.310 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_backtrace_fetch": 1766, 2024-04-25T08:56:44.310 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_set_tag": 0, 2024-04-25T08:56:44.310 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T08:56:44.310 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_inotable_repaired": 0, 2024-04-25T08:56:44.310 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_dir_inodes": 13, 2024-04-25T08:56:44.310 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_dir_base_inodes": 2, 2024-04-25T08:56:44.310 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_dirfrag_rstats": 11, 2024-04-25T08:56:44.311 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_file_inodes": 1753, 2024-04-25T08:56:44.311 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_inode_file_caps": 0, 2024-04-25T08:56:44.311 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_revoke": 6190, 2024-04-25T08:56:44.311 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_grant": 6006, 2024-04-25T08:56:44.311 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T08:56:44.311 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T08:56:44.311 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T08:56:44.311 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_caps": 6190, 2024-04-25T08:56:44.311 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_caps_dirty": 2000, 2024-04-25T08:56:44.311 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_cap_release": 26, 2024-04-25T08:56:44.311 INFO:teuthology.orchestra.run.smithi043.stdout: "process_request_cap_release": 17846 2024-04-25T08:56:44.311 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.311 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_cache": { 2024-04-25T08:56:44.312 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_update": 0, 2024-04-25T08:56:44.312 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_update_receipt": 0, 2024-04-25T08:56:44.312 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_try_discover": 0, 2024-04-25T08:56:44.312 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_send_discover": 0, 2024-04-25T08:56:44.312 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_handle_discover": 0, 2024-04-25T08:56:44.312 INFO:teuthology.orchestra.run.smithi043.stdout: "num_strays": 0, 2024-04-25T08:56:44.312 INFO:teuthology.orchestra.run.smithi043.stdout: "num_strays_delayed": 0, 2024-04-25T08:56:44.312 INFO:teuthology.orchestra.run.smithi043.stdout: "num_strays_enqueuing": 0, 2024-04-25T08:56:44.312 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_created": 4088, 2024-04-25T08:56:44.312 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_enqueued": 3481, 2024-04-25T08:56:44.312 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_reintegrated": 2196, 2024-04-25T08:56:44.312 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_migrated": 0, 2024-04-25T08:56:44.312 INFO:teuthology.orchestra.run.smithi043.stdout: "num_recovering_processing": 0, 2024-04-25T08:56:44.312 INFO:teuthology.orchestra.run.smithi043.stdout: "num_recovering_enqueued": 0, 2024-04-25T08:56:44.312 INFO:teuthology.orchestra.run.smithi043.stdout: "num_recovering_prioritized": 0, 2024-04-25T08:56:44.313 INFO:teuthology.orchestra.run.smithi043.stdout: "recovery_started": 0, 2024-04-25T08:56:44.313 INFO:teuthology.orchestra.run.smithi043.stdout: "recovery_completed": 0, 2024-04-25T08:56:44.313 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_enqueue_scrub": 1, 2024-04-25T08:56:44.313 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_exportdir": 0, 2024-04-25T08:56:44.313 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_flush": 0, 2024-04-25T08:56:44.313 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_fragmentdir": 0, 2024-04-25T08:56:44.313 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_fragstats": 0, 2024-04-25T08:56:44.313 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_inodestats": 0 2024-04-25T08:56:44.313 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.313 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_log": { 2024-04-25T08:56:44.313 INFO:teuthology.orchestra.run.smithi043.stdout: "evlrg": 0, 2024-04-25T08:56:44.313 INFO:teuthology.orchestra.run.smithi043.stdout: "evadd": 23640, 2024-04-25T08:56:44.313 INFO:teuthology.orchestra.run.smithi043.stdout: "evex": 0, 2024-04-25T08:56:44.313 INFO:teuthology.orchestra.run.smithi043.stdout: "evtrm": 0, 2024-04-25T08:56:44.313 INFO:teuthology.orchestra.run.smithi043.stdout: "ev": 23640, 2024-04-25T08:56:44.314 INFO:teuthology.orchestra.run.smithi043.stdout: "evexg": 0, 2024-04-25T08:56:44.314 INFO:teuthology.orchestra.run.smithi043.stdout: "evexd": 0, 2024-04-25T08:56:44.314 INFO:teuthology.orchestra.run.smithi043.stdout: "segadd": 30, 2024-04-25T08:56:44.314 INFO:teuthology.orchestra.run.smithi043.stdout: "segex": 0, 2024-04-25T08:56:44.314 INFO:teuthology.orchestra.run.smithi043.stdout: "segtrm": 0, 2024-04-25T08:56:44.314 INFO:teuthology.orchestra.run.smithi043.stdout: "seg": 30, 2024-04-25T08:56:44.314 INFO:teuthology.orchestra.run.smithi043.stdout: "segmjr": 3, 2024-04-25T08:56:44.314 INFO:teuthology.orchestra.run.smithi043.stdout: "segexg": 0, 2024-04-25T08:56:44.314 INFO:teuthology.orchestra.run.smithi043.stdout: "segexd": 0, 2024-04-25T08:56:44.314 INFO:teuthology.orchestra.run.smithi043.stdout: "expos": 4194304, 2024-04-25T08:56:44.314 INFO:teuthology.orchestra.run.smithi043.stdout: "wrpos": 82072346, 2024-04-25T08:56:44.314 INFO:teuthology.orchestra.run.smithi043.stdout: "rdpos": 0, 2024-04-25T08:56:44.314 INFO:teuthology.orchestra.run.smithi043.stdout: "jlat": { 2024-04-25T08:56:44.314 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 6498, 2024-04-25T08:56:44.314 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 72.957287392, 2024-04-25T08:56:44.315 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.011227652 2024-04-25T08:56:44.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.315 INFO:teuthology.orchestra.run.smithi043.stdout: "replayed": 0 2024-04-25T08:56:44.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.315 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_mem": { 2024-04-25T08:56:44.315 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 14, 2024-04-25T08:56:44.315 INFO:teuthology.orchestra.run.smithi043.stdout: "ino+": 2016, 2024-04-25T08:56:44.315 INFO:teuthology.orchestra.run.smithi043.stdout: "ino-": 2002, 2024-04-25T08:56:44.315 INFO:teuthology.orchestra.run.smithi043.stdout: "dir": 13, 2024-04-25T08:56:44.315 INFO:teuthology.orchestra.run.smithi043.stdout: "dir+": 13, 2024-04-25T08:56:44.315 INFO:teuthology.orchestra.run.smithi043.stdout: "dir-": 0, 2024-04-25T08:56:44.315 INFO:teuthology.orchestra.run.smithi043.stdout: "dn": 17818, 2024-04-25T08:56:44.315 INFO:teuthology.orchestra.run.smithi043.stdout: "dn+": 19818, 2024-04-25T08:56:44.316 INFO:teuthology.orchestra.run.smithi043.stdout: "dn-": 2000, 2024-04-25T08:56:44.316 INFO:teuthology.orchestra.run.smithi043.stdout: "cap": 2, 2024-04-25T08:56:44.316 INFO:teuthology.orchestra.run.smithi043.stdout: "cap+": 2002, 2024-04-25T08:56:44.316 INFO:teuthology.orchestra.run.smithi043.stdout: "cap-": 2000, 2024-04-25T08:56:44.316 INFO:teuthology.orchestra.run.smithi043.stdout: "rss": 100648, 2024-04-25T08:56:44.316 INFO:teuthology.orchestra.run.smithi043.stdout: "heap": 215396 2024-04-25T08:56:44.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.316 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_server": { 2024-04-25T08:56:44.316 INFO:teuthology.orchestra.run.smithi043.stdout: "dispatch_client_request": 37664, 2024-04-25T08:56:44.316 INFO:teuthology.orchestra.run.smithi043.stdout: "dispatch_server_request": 0, 2024-04-25T08:56:44.316 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_request": 28251, 2024-04-25T08:56:44.316 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_session": 7, 2024-04-25T08:56:44.316 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_peer_request": 0, 2024-04-25T08:56:44.316 INFO:teuthology.orchestra.run.smithi043.stdout: "req_create_latency": { 2024-04-25T08:56:44.316 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 2000, 2024-04-25T08:56:44.316 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 1.369929300, 2024-04-25T08:56:44.317 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000684964 2024-04-25T08:56:44.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.317 INFO:teuthology.orchestra.run.smithi043.stdout: "req_getattr_latency": { 2024-04-25T08:56:44.317 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 1999, 2024-04-25T08:56:44.317 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.608105825, 2024-04-25T08:56:44.317 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000304205 2024-04-25T08:56:44.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.317 INFO:teuthology.orchestra.run.smithi043.stdout: "req_getfilelock_latency": { 2024-04-25T08:56:44.317 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:44.317 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:44.317 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:44.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.317 INFO:teuthology.orchestra.run.smithi043.stdout: "req_link_latency": { 2024-04-25T08:56:44.317 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 6000, 2024-04-25T08:56:44.317 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 9.535551052, 2024-04-25T08:56:44.317 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.001589258 2024-04-25T08:56:44.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.318 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookup_latency": { 2024-04-25T08:56:44.318 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 8043, 2024-04-25T08:56:44.318 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 7.183828452, 2024-04-25T08:56:44.318 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000893177 2024-04-25T08:56:44.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.318 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookuphash_latency": { 2024-04-25T08:56:44.318 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:44.318 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:44.318 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:44.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.318 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupino_latency": { 2024-04-25T08:56:44.318 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:44.318 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:44.319 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:44.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.319 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupname_latency": { 2024-04-25T08:56:44.319 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:44.319 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:44.319 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:44.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.319 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupparent_latency": { 2024-04-25T08:56:44.319 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:44.319 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:44.319 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:44.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.319 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupsnap_latency": { 2024-04-25T08:56:44.319 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 3, 2024-04-25T08:56:44.319 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.006938573, 2024-04-25T08:56:44.320 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.002312857 2024-04-25T08:56:44.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.320 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lssnap_latency": { 2024-04-25T08:56:44.320 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:44.320 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:44.320 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:44.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.320 INFO:teuthology.orchestra.run.smithi043.stdout: "req_mkdir_latency": { 2024-04-25T08:56:44.320 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 1, 2024-04-25T08:56:44.320 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.002562965, 2024-04-25T08:56:44.320 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.002562965 2024-04-25T08:56:44.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.320 INFO:teuthology.orchestra.run.smithi043.stdout: "req_mknod_latency": { 2024-04-25T08:56:44.321 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:44.321 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:44.321 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:44.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.321 INFO:teuthology.orchestra.run.smithi043.stdout: "req_mksnap_latency": { 2024-04-25T08:56:44.321 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 1, 2024-04-25T08:56:44.321 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.007145409, 2024-04-25T08:56:44.321 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.007145409 2024-04-25T08:56:44.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.321 INFO:teuthology.orchestra.run.smithi043.stdout: "req_open_latency": { 2024-04-25T08:56:44.321 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:44.321 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:44.321 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:44.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.322 INFO:teuthology.orchestra.run.smithi043.stdout: "req_readdir_latency": { 2024-04-25T08:56:44.322 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 6, 2024-04-25T08:56:44.322 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.093349188, 2024-04-25T08:56:44.322 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.015558198 2024-04-25T08:56:44.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.322 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rename_latency": { 2024-04-25T08:56:44.322 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 2196, 2024-04-25T08:56:44.322 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 208.693899538, 2024-04-25T08:56:44.322 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.095033651 2024-04-25T08:56:44.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.322 INFO:teuthology.orchestra.run.smithi043.stdout: "req_renamesnap_latency": { 2024-04-25T08:56:44.322 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:44.322 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:44.322 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:44.322 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.323 INFO:teuthology.orchestra.run.smithi043.stdout: "req_snapdiff_latency": { 2024-04-25T08:56:44.323 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:44.323 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:44.323 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:44.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.323 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rmdir_latency": { 2024-04-25T08:56:44.323 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:44.323 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:44.323 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:44.323 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.324 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rmsnap_latency": { 2024-04-25T08:56:44.324 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 1, 2024-04-25T08:56:44.324 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.004814886, 2024-04-25T08:56:44.324 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.004814886 2024-04-25T08:56:44.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.324 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rmxattr_latency": { 2024-04-25T08:56:44.324 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:44.324 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:44.324 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:44.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.324 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setattr_latency": { 2024-04-25T08:56:44.324 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 1, 2024-04-25T08:56:44.324 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000225467, 2024-04-25T08:56:44.324 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000225467 2024-04-25T08:56:44.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.325 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setdirlayout_latency": { 2024-04-25T08:56:44.325 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:44.325 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:44.325 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:44.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.325 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setfilelock_latency": { 2024-04-25T08:56:44.325 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:44.325 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:44.325 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:44.325 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.325 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setlayout_latency": { 2024-04-25T08:56:44.325 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:44.325 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:44.326 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:44.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.326 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setxattr_latency": { 2024-04-25T08:56:44.326 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:44.326 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:44.326 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:44.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.326 INFO:teuthology.orchestra.run.smithi043.stdout: "req_symlink_latency": { 2024-04-25T08:56:44.326 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:44.326 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:44.326 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:44.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.326 INFO:teuthology.orchestra.run.smithi043.stdout: "req_unlink_latency": { 2024-04-25T08:56:44.326 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 8000, 2024-04-25T08:56:44.327 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 15.536343068, 2024-04-25T08:56:44.327 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.001942042 2024-04-25T08:56:44.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.327 INFO:teuthology.orchestra.run.smithi043.stdout: "cap_revoke_eviction": 0, 2024-04-25T08:56:44.327 INFO:teuthology.orchestra.run.smithi043.stdout: "cap_acquisition_throttle": 0, 2024-04-25T08:56:44.327 INFO:teuthology.orchestra.run.smithi043.stdout: "req_getvxattr_latency": { 2024-04-25T08:56:44.327 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:44.327 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:44.327 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:44.327 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:56:44.327 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.327 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_sessions": { 2024-04-25T08:56:44.327 INFO:teuthology.orchestra.run.smithi043.stdout: "session_count": 1, 2024-04-25T08:56:44.327 INFO:teuthology.orchestra.run.smithi043.stdout: "session_add": 1, 2024-04-25T08:56:44.328 INFO:teuthology.orchestra.run.smithi043.stdout: "session_remove": 0, 2024-04-25T08:56:44.328 INFO:teuthology.orchestra.run.smithi043.stdout: "sessions_open": 1, 2024-04-25T08:56:44.328 INFO:teuthology.orchestra.run.smithi043.stdout: "sessions_stale": 0, 2024-04-25T08:56:44.328 INFO:teuthology.orchestra.run.smithi043.stdout: "total_load": 19520, 2024-04-25T08:56:44.328 INFO:teuthology.orchestra.run.smithi043.stdout: "average_load": 19520, 2024-04-25T08:56:44.328 INFO:teuthology.orchestra.run.smithi043.stdout: "avg_session_uptime": 87, 2024-04-25T08:56:44.328 INFO:teuthology.orchestra.run.smithi043.stdout: "mdthresh_evicted": 0 2024-04-25T08:56:44.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.328 INFO:teuthology.orchestra.run.smithi043.stdout: "mempool": { 2024-04-25T08:56:44.328 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_bytes": 0, 2024-04-25T08:56:44.328 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_items": 0, 2024-04-25T08:56:44.328 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T08:56:44.328 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_items": 0, 2024-04-25T08:56:44.328 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T08:56:44.328 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_items": 0, 2024-04-25T08:56:44.329 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T08:56:44.329 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T08:56:44.329 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T08:56:44.329 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T08:56:44.329 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T08:56:44.329 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_items": 0, 2024-04-25T08:56:44.329 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T08:56:44.329 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T08:56:44.329 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_bytes": 0, 2024-04-25T08:56:44.329 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_items": 0, 2024-04-25T08:56:44.329 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_bytes": 0, 2024-04-25T08:56:44.329 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_items": 0, 2024-04-25T08:56:44.329 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T08:56:44.329 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T08:56:44.329 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T08:56:44.330 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T08:56:44.330 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T08:56:44.330 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_items": 0, 2024-04-25T08:56:44.330 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_bytes": 0, 2024-04-25T08:56:44.330 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_items": 0, 2024-04-25T08:56:44.330 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T08:56:44.330 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T08:56:44.330 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_bytes": 0, 2024-04-25T08:56:44.330 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_items": 0, 2024-04-25T08:56:44.330 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_bytes": 0, 2024-04-25T08:56:44.330 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_items": 0, 2024-04-25T08:56:44.330 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T08:56:44.330 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_items": 0, 2024-04-25T08:56:44.330 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T08:56:44.331 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_items": 0, 2024-04-25T08:56:44.331 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_bytes": 251336, 2024-04-25T08:56:44.331 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_items": 86, 2024-04-25T08:56:44.331 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_bytes": 0, 2024-04-25T08:56:44.331 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_items": 0, 2024-04-25T08:56:44.331 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_bytes": 0, 2024-04-25T08:56:44.331 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_items": 0, 2024-04-25T08:56:44.331 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_bytes": 0, 2024-04-25T08:56:44.331 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_items": 0, 2024-04-25T08:56:44.331 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_bytes": 0, 2024-04-25T08:56:44.331 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_items": 0, 2024-04-25T08:56:44.331 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_bytes": 5416, 2024-04-25T08:56:44.331 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_items": 77, 2024-04-25T08:56:44.331 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T08:56:44.331 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_items": 0, 2024-04-25T08:56:44.331 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_bytes": 0, 2024-04-25T08:56:44.332 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_items": 0, 2024-04-25T08:56:44.332 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_bytes": 13417716, 2024-04-25T08:56:44.332 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_items": 660757, 2024-04-25T08:56:44.332 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_bytes": 0, 2024-04-25T08:56:44.332 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_items": 0, 2024-04-25T08:56:44.332 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_bytes": 0, 2024-04-25T08:56:44.332 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_items": 0 2024-04-25T08:56:44.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.332 INFO:teuthology.orchestra.run.smithi043.stdout: "objecter": { 2024-04-25T08:56:44.332 INFO:teuthology.orchestra.run.smithi043.stdout: "op_active": 0, 2024-04-25T08:56:44.332 INFO:teuthology.orchestra.run.smithi043.stdout: "op_laggy": 0, 2024-04-25T08:56:44.332 INFO:teuthology.orchestra.run.smithi043.stdout: "op_send": 12006, 2024-04-25T08:56:44.332 INFO:teuthology.orchestra.run.smithi043.stdout: "op_send_bytes": 78269435, 2024-04-25T08:56:44.332 INFO:teuthology.orchestra.run.smithi043.stdout: "op_resend": 37, 2024-04-25T08:56:44.332 INFO:teuthology.orchestra.run.smithi043.stdout: "op_reply": 12006, 2024-04-25T08:56:44.332 INFO:teuthology.orchestra.run.smithi043.stdout: "op_latency": { 2024-04-25T08:56:44.333 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 12006, 2024-04-25T08:56:44.333 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 32.495612579, 2024-04-25T08:56:44.333 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.002706614 2024-04-25T08:56:44.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.333 INFO:teuthology.orchestra.run.smithi043.stdout: "op_inflight": 0, 2024-04-25T08:56:44.333 INFO:teuthology.orchestra.run.smithi043.stdout: "oplen_avg": { 2024-04-25T08:56:44.333 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 12006, 2024-04-25T08:56:44.333 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 12065 2024-04-25T08:56:44.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.333 INFO:teuthology.orchestra.run.smithi043.stdout: "op": 12006, 2024-04-25T08:56:44.333 INFO:teuthology.orchestra.run.smithi043.stdout: "op_r": 1831, 2024-04-25T08:56:44.333 INFO:teuthology.orchestra.run.smithi043.stdout: "op_w": 10175, 2024-04-25T08:56:44.333 INFO:teuthology.orchestra.run.smithi043.stdout: "op_rmw": 0, 2024-04-25T08:56:44.333 INFO:teuthology.orchestra.run.smithi043.stdout: "op_pg": 0, 2024-04-25T08:56:44.333 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_stat": 0, 2024-04-25T08:56:44.333 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_create": 11, 2024-04-25T08:56:44.334 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_read": 63, 2024-04-25T08:56:44.334 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_write": 6559, 2024-04-25T08:56:44.334 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_writefull": 39, 2024-04-25T08:56:44.334 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_writesame": 0, 2024-04-25T08:56:44.334 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_append": 0, 2024-04-25T08:56:44.334 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_zero": 1481, 2024-04-25T08:56:44.334 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_truncate": 0, 2024-04-25T08:56:44.334 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_delete": 2030, 2024-04-25T08:56:44.334 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_mapext": 0, 2024-04-25T08:56:44.334 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_sparse_read": 0, 2024-04-25T08:56:44.334 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_clonerange": 0, 2024-04-25T08:56:44.334 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_getxattr": 1768, 2024-04-25T08:56:44.334 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_setxattr": 22, 2024-04-25T08:56:44.334 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_cmpxattr": 0, 2024-04-25T08:56:44.334 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_rmxattr": 0, 2024-04-25T08:56:44.335 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_resetxattrs": 0, 2024-04-25T08:56:44.335 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_call": 0, 2024-04-25T08:56:44.335 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_watch": 0, 2024-04-25T08:56:44.335 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_notify": 0, 2024-04-25T08:56:44.335 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T08:56:44.335 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_pgls": 0, 2024-04-25T08:56:44.335 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_pgls_filter": 0, 2024-04-25T08:56:44.335 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_other": 0, 2024-04-25T08:56:44.335 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_active": 0, 2024-04-25T08:56:44.335 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_send": 0, 2024-04-25T08:56:44.335 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_resend": 0, 2024-04-25T08:56:44.335 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_ping": 0, 2024-04-25T08:56:44.335 INFO:teuthology.orchestra.run.smithi043.stdout: "poolop_active": 0, 2024-04-25T08:56:44.335 INFO:teuthology.orchestra.run.smithi043.stdout: "poolop_send": 0, 2024-04-25T08:56:44.335 INFO:teuthology.orchestra.run.smithi043.stdout: "poolop_resend": 0, 2024-04-25T08:56:44.335 INFO:teuthology.orchestra.run.smithi043.stdout: "poolstat_active": 0, 2024-04-25T08:56:44.335 INFO:teuthology.orchestra.run.smithi043.stdout: "poolstat_send": 0, 2024-04-25T08:56:44.336 INFO:teuthology.orchestra.run.smithi043.stdout: "poolstat_resend": 0, 2024-04-25T08:56:44.336 INFO:teuthology.orchestra.run.smithi043.stdout: "statfs_active": 0, 2024-04-25T08:56:44.336 INFO:teuthology.orchestra.run.smithi043.stdout: "statfs_send": 0, 2024-04-25T08:56:44.336 INFO:teuthology.orchestra.run.smithi043.stdout: "statfs_resend": 0, 2024-04-25T08:56:44.336 INFO:teuthology.orchestra.run.smithi043.stdout: "command_active": 0, 2024-04-25T08:56:44.336 INFO:teuthology.orchestra.run.smithi043.stdout: "command_send": 0, 2024-04-25T08:56:44.336 INFO:teuthology.orchestra.run.smithi043.stdout: "command_resend": 0, 2024-04-25T08:56:44.336 INFO:teuthology.orchestra.run.smithi043.stdout: "map_epoch": 35, 2024-04-25T08:56:44.336 INFO:teuthology.orchestra.run.smithi043.stdout: "map_full": 0, 2024-04-25T08:56:44.336 INFO:teuthology.orchestra.run.smithi043.stdout: "map_inc": 3, 2024-04-25T08:56:44.336 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_sessions": 8, 2024-04-25T08:56:44.336 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_session_open": 8, 2024-04-25T08:56:44.336 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_session_close": 0, 2024-04-25T08:56:44.336 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_laggy": 0, 2024-04-25T08:56:44.336 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_wr": 77, 2024-04-25T08:56:44.336 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_rd": 0, 2024-04-25T08:56:44.337 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_del": 15 2024-04-25T08:56:44.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.337 INFO:teuthology.orchestra.run.smithi043.stdout: "oft": { 2024-04-25T08:56:44.337 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_objs": 1, 2024-04-25T08:56:44.337 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_kv_pairs": 0, 2024-04-25T08:56:44.337 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_updates": 0, 2024-04-25T08:56:44.337 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_removes": 0 2024-04-25T08:56:44.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.337 INFO:teuthology.orchestra.run.smithi043.stdout: "osd-slow-ops": { 2024-04-25T08:56:44.337 INFO:teuthology.orchestra.run.smithi043.stdout: "slow_ops_count": 0 2024-04-25T08:56:44.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.337 INFO:teuthology.orchestra.run.smithi043.stdout: "purge_queue": { 2024-04-25T08:56:44.337 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing_ops": 0, 2024-04-25T08:56:44.337 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T08:56:44.337 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing": 0, 2024-04-25T08:56:44.338 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing_high_water": 32, 2024-04-25T08:56:44.338 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executed_ops": 3481, 2024-04-25T08:56:44.338 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executed": 3481, 2024-04-25T08:56:44.338 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_item_in_journal": 0 2024-04-25T08:56:44.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.338 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T08:56:44.338 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-25T08:56:44.338 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 104857600, 2024-04-25T08:56:44.338 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-25T08:56:44.338 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 44519, 2024-04-25T08:56:44.338 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 10955705, 2024-04-25T08:56:44.338 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-25T08:56:44.338 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 44519, 2024-04-25T08:56:44.338 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-04-25T08:56:44.339 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-04-25T08:56:44.339 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 44519, 2024-04-25T08:56:44.339 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 10955705, 2024-04-25T08:56:44.339 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-25T08:56:44.339 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:44.339 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:44.339 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:44.339 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:56:44.339 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.339 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-objecter_bytes": { 2024-04-25T08:56:44.339 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-25T08:56:44.339 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 104857600, 2024-04-25T08:56:44.339 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-25T08:56:44.339 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 0, 2024-04-25T08:56:44.340 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 0, 2024-04-25T08:56:44.340 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-25T08:56:44.340 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 0, 2024-04-25T08:56:44.340 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 12006, 2024-04-25T08:56:44.340 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 78632864, 2024-04-25T08:56:44.340 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 12006, 2024-04-25T08:56:44.340 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 78632864, 2024-04-25T08:56:44.340 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-25T08:56:44.340 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:44.340 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:44.341 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:44.341 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:56:44.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.341 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-objecter_ops": { 2024-04-25T08:56:44.341 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-25T08:56:44.341 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 1024, 2024-04-25T08:56:44.341 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-25T08:56:44.341 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 0, 2024-04-25T08:56:44.341 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 0, 2024-04-25T08:56:44.341 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-25T08:56:44.341 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 0, 2024-04-25T08:56:44.341 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 12006, 2024-04-25T08:56:44.341 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 12006, 2024-04-25T08:56:44.342 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 12006, 2024-04-25T08:56:44.342 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 12006, 2024-04-25T08:56:44.342 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-25T08:56:44.342 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:44.342 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:44.342 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:44.342 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:56:44.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.342 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-write_buf_throttle": { 2024-04-25T08:56:44.342 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-25T08:56:44.342 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 3758096384, 2024-04-25T08:56:44.342 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-25T08:56:44.343 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 3481, 2024-04-25T08:56:44.343 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 363429, 2024-04-25T08:56:44.343 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-25T08:56:44.343 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 3481, 2024-04-25T08:56:44.343 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-04-25T08:56:44.343 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-04-25T08:56:44.343 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 61, 2024-04-25T08:56:44.343 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 363429, 2024-04-25T08:56:44.343 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-25T08:56:44.343 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:44.343 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:44.343 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:44.343 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:56:44.343 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:44.343 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-write_buf_throttle-0x55d8ed2fa6e0": { 2024-04-25T08:56:44.343 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-25T08:56:44.344 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 3758096384, 2024-04-25T08:56:44.344 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-25T08:56:44.344 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 23640, 2024-04-25T08:56:44.344 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 77878042, 2024-04-25T08:56:44.344 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-25T08:56:44.344 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 23640, 2024-04-25T08:56:44.344 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-04-25T08:56:44.344 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-04-25T08:56:44.344 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 6498, 2024-04-25T08:56:44.344 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 77878042, 2024-04-25T08:56:44.344 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-25T08:56:44.344 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:56:44.344 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:56:44.344 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:56:44.345 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:56:44.345 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:56:44.345 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T08:56:44.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.304+0000 7fde60812640 1 -- 172.21.15.43:0/548198445 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fde3c06b4d0 msgr2=0x7fde3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.304+0000 7fde60812640 1 --2- 172.21.15.43:0/548198445 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fde3c06b4d0 0x7fde3c06d990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fde4c034000 tx=0x7fde4c009000 comp rx=0 tx=0).stop 2024-04-25T08:56:44.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.304+0000 7fde60812640 1 -- 172.21.15.43:0/548198445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde5c161fe0 msgr2=0x7fde5c0b3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.304+0000 7fde60812640 1 --2- 172.21.15.43:0/548198445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde5c161fe0 0x7fde5c0b3f60 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fde54001eb0 tx=0x7fde5400c890 comp rx=0 tx=0).stop 2024-04-25T08:56:44.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.304+0000 7fde60812640 1 -- 172.21.15.43:0/548198445 shutdown_connections 2024-04-25T08:56:44.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.304+0000 7fde60812640 1 --2- 172.21.15.43:0/548198445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde5c161fe0 0x7fde5c0b3f60 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.304+0000 7fde60812640 1 --2- 172.21.15.43:0/548198445 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fde3c06b4d0 0x7fde3c06d990 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.304+0000 7fde60812640 1 --2- 172.21.15.43:0/548198445 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde5c15e090 0x7fde5c0cec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.304+0000 7fde60812640 1 --2- 172.21.15.43:0/548198445 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fde5c0b5c70 0x7fde5c15b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.304+0000 7fde60812640 1 -- 172.21.15.43:0/548198445 >> 172.21.15.43:0/548198445 conn(0x7fde5c0d0560 msgr2=0x7fde5c160580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:44.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.304+0000 7fde60812640 1 -- 172.21.15.43:0/548198445 shutdown_connections 2024-04-25T08:56:44.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.304+0000 7fde60812640 1 -- 172.21.15.43:0/548198445 wait complete. 2024-04-25T08:56:44.348 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:56:44.348 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:56:44.428 DEBUG:teuthology.orchestra.run.smithi193:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:56:44.476 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:56:44.476 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:56:44.476 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:56:44.492 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:56:44.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.642+0000 7fef7c89e640 1 Processor -- start 2024-04-25T08:56:44.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.642+0000 7fef7c89e640 1 -- start start 2024-04-25T08:56:44.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.643+0000 7fef7c89e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef7806c510 0x7fef7806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.643+0000 7fef7c89e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fef7815df70 0x7fef780ce070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.643+0000 7fef7c89e640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fef780b0870 0x7fef780ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.643+0000 7fef7c89e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef780d0400 con 0x7fef7815df70 2024-04-25T08:56:44.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.643+0000 7fef7c89e640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef780cbee0 con 0x7fef780b0870 2024-04-25T08:56:44.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.643+0000 7fef7c89e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef7806ce30 con 0x7fef7806c510 2024-04-25T08:56:44.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.643+0000 7fef75d9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fef7815df70 0x7fef780ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.643+0000 7fef75d9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fef7815df70 0x7fef780ce070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59148/0 (socket says 172.21.15.43:59148) 2024-04-25T08:56:44.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.643+0000 7fef75d9b640 1 -- 172.21.15.43:0/3153714377 learned_addr learned my addr 172.21.15.43:0/3153714377 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:44.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.643+0000 7fef7659c640 1 --2- 172.21.15.43:0/3153714377 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef7806c510 0x7fef7806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.643+0000 7fef76d9d640 1 --2- 172.21.15.43:0/3153714377 >> v2:172.21.15.193:3300/0 conn(0x7fef780b0870 0x7fef780ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.643+0000 7fef75d9b640 1 -- 172.21.15.43:0/3153714377 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef7806c510 msgr2=0x7fef7806c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:44.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.643+0000 7fef75d9b640 1 --2- 172.21.15.43:0/3153714377 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef7806c510 0x7fef7806c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.643+0000 7fef75d9b640 1 -- 172.21.15.43:0/3153714377 >> v2:172.21.15.193:3300/0 conn(0x7fef780b0870 msgr2=0x7fef780ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:44.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.643+0000 7fef75d9b640 1 --2- 172.21.15.43:0/3153714377 >> v2:172.21.15.193:3300/0 conn(0x7fef780b0870 0x7fef780ce5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.643+0000 7fef75d9b640 1 -- 172.21.15.43:0/3153714377 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef780d0650 con 0x7fef7815df70 2024-04-25T08:56:44.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.644+0000 7fef75d9b640 1 --2- 172.21.15.43:0/3153714377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef7815df70 0x7fef780ce070 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fef78072750 tx=0x7fef6c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=317d01ea461e4359 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:44.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.644+0000 7fef7559a640 1 -- 172.21.15.43:0/3153714377 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c037070 con 0x7fef7815df70 2024-04-25T08:56:44.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.644+0000 7fef7559a640 1 -- 172.21.15.43:0/3153714377 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef6c032440 con 0x7fef7815df70 2024-04-25T08:56:44.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.644+0000 7fef7559a640 1 -- 172.21.15.43:0/3153714377 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c032810 con 0x7fef7815df70 2024-04-25T08:56:44.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.645+0000 7fef7c89e640 1 -- 172.21.15.43:0/3153714377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef7815df70 msgr2=0x7fef780ce070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.645+0000 7fef7c89e640 1 --2- 172.21.15.43:0/3153714377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef7815df70 0x7fef780ce070 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fef78072750 tx=0x7fef6c02a1e0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.645+0000 7fef7c89e640 1 -- 172.21.15.43:0/3153714377 shutdown_connections 2024-04-25T08:56:44.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.645+0000 7fef7c89e640 1 --2- 172.21.15.43:0/3153714377 >> v2:172.21.15.193:3300/0 conn(0x7fef780b0870 0x7fef780ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.645+0000 7fef7c89e640 1 --2- 172.21.15.43:0/3153714377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef7815df70 0x7fef780ce070 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.645+0000 7fef7c89e640 1 --2- 172.21.15.43:0/3153714377 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef7806c510 0x7fef7806c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.645+0000 7fef7c89e640 1 -- 172.21.15.43:0/3153714377 >> 172.21.15.43:0/3153714377 conn(0x7fef780d17f0 msgr2=0x7fef780d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:44.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.645+0000 7fef7c89e640 1 -- 172.21.15.43:0/3153714377 shutdown_connections 2024-04-25T08:56:44.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.645+0000 7fef7c89e640 1 -- 172.21.15.43:0/3153714377 wait complete. 2024-04-25T08:56:44.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.646+0000 7fef7c89e640 1 Processor -- start 2024-04-25T08:56:44.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.646+0000 7fef7c89e640 1 -- start start 2024-04-25T08:56:44.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.647+0000 7fef7c89e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef7806c510 0x7fef7814be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.647+0000 7fef7c89e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef780b0870 0x7fef780c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.647+0000 7fef7c89e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef7815df70 0x7fef780ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.647+0000 7fef7c89e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef780d0c10 con 0x7fef780b0870 2024-04-25T08:56:44.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.647+0000 7fef7c89e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef780cc150 con 0x7fef7815df70 2024-04-25T08:56:44.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.647+0000 7fef7c89e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef7806bd10 con 0x7fef7806c510 2024-04-25T08:56:44.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.647+0000 7fef75d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef780b0870 0x7fef780c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.647+0000 7fef75d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef780b0870 0x7fef780c71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59162/0 (socket says 172.21.15.43:59162) 2024-04-25T08:56:44.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.647+0000 7fef75d9b640 1 -- 172.21.15.43:0/1310209104 learned_addr learned my addr 172.21.15.43:0/1310209104 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:44.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.647+0000 7fef7659c640 1 --2- 172.21.15.43:0/1310209104 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef7806c510 0x7fef7814be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.647+0000 7fef76d9d640 1 --2- 172.21.15.43:0/1310209104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef7815df70 0x7fef780ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.647+0000 7fef75d9b640 1 -- 172.21.15.43:0/1310209104 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef7806c510 msgr2=0x7fef7814be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.647+0000 7fef75d9b640 1 --2- 172.21.15.43:0/1310209104 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef7806c510 0x7fef7814be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.647+0000 7fef75d9b640 1 -- 172.21.15.43:0/1310209104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef7815df70 msgr2=0x7fef780ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.647+0000 7fef75d9b640 1 --2- 172.21.15.43:0/1310209104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef7815df70 0x7fef780ccc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.647+0000 7fef75d9b640 1 -- 172.21.15.43:0/1310209104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef7814c390 con 0x7fef780b0870 2024-04-25T08:56:44.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.647+0000 7fef76d9d640 1 --2- 172.21.15.43:0/1310209104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef7815df70 0x7fef780ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:56:44.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.648+0000 7fef75d9b640 1 --2- 172.21.15.43:0/1310209104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef780b0870 0x7fef780c71e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fef6c02a220 tx=0x7fef6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:44.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.648+0000 7fef66ffd640 1 -- 172.21.15.43:0/1310209104 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c041020 con 0x7fef780b0870 2024-04-25T08:56:44.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.648+0000 7fef66ffd640 1 -- 172.21.15.43:0/1310209104 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef6c0306c0 con 0x7fef780b0870 2024-04-25T08:56:44.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.648+0000 7fef7c89e640 1 -- 172.21.15.43:0/1310209104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef780c7a30 con 0x7fef780b0870 2024-04-25T08:56:44.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.648+0000 7fef66ffd640 1 -- 172.21.15.43:0/1310209104 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6c037040 con 0x7fef780b0870 2024-04-25T08:56:44.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.648+0000 7fef7c89e640 1 -- 172.21.15.43:0/1310209104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef780c7ff0 con 0x7fef780b0870 2024-04-25T08:56:44.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.649+0000 7fef7c89e640 1 -- 172.21.15.43:0/1310209104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef78074ef0 con 0x7fef780b0870 2024-04-25T08:56:44.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.649+0000 7fef66ffd640 1 -- 172.21.15.43:0/1310209104 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fef6c044020 con 0x7fef780b0870 2024-04-25T08:56:44.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.650+0000 7fef66ffd640 1 --2- 172.21.15.43:0/1310209104 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fef5806b4d0 0x7fef5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.650+0000 7fef66ffd640 1 -- 172.21.15.43:0/1310209104 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fef6c0aa9d0 con 0x7fef780b0870 2024-04-25T08:56:44.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.650+0000 7fef7659c640 1 --2- 172.21.15.43:0/1310209104 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fef5806b4d0 0x7fef5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.651+0000 7fef7659c640 1 --2- 172.21.15.43:0/1310209104 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fef5806b4d0 0x7fef5806d990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fef68003aa0 tx=0x7fef68009830 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:44.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.654+0000 7fef66ffd640 1 -- 172.21.15.43:0/1310209104 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fef6c04b030 con 0x7fef780b0870 2024-04-25T08:56:44.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.834+0000 7fef7c89e640 1 -- 172.21.15.43:0/1310209104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef780af430 con 0x7fef780b0870 2024-04-25T08:56:44.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.835+0000 7fef66ffd640 1 -- 172.21.15.43:0/1310209104 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16698 (secure 0 0 0) 0x7fef6c079830 con 0x7fef780b0870 2024-04-25T08:56:44.825 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:56:44.825 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":35,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:56:22.378131+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:55:05.150275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T08:55:05.853905+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":34,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:56:44.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.837+0000 7fef7c89e640 1 -- 172.21.15.43:0/1310209104 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fef5806b4d0 msgr2=0x7fef5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.837+0000 7fef7c89e640 1 --2- 172.21.15.43:0/1310209104 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fef5806b4d0 0x7fef5806d990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fef68003aa0 tx=0x7fef68009830 comp rx=0 tx=0).stop 2024-04-25T08:56:44.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.837+0000 7fef7c89e640 1 -- 172.21.15.43:0/1310209104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef780b0870 msgr2=0x7fef780c71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.837+0000 7fef7c89e640 1 --2- 172.21.15.43:0/1310209104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef780b0870 0x7fef780c71e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fef6c02a220 tx=0x7fef6c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.837+0000 7fef7c89e640 1 -- 172.21.15.43:0/1310209104 shutdown_connections 2024-04-25T08:56:44.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.837+0000 7fef7c89e640 1 --2- 172.21.15.43:0/1310209104 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fef5806b4d0 0x7fef5806d990 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.837+0000 7fef7c89e640 1 --2- 172.21.15.43:0/1310209104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef7815df70 0x7fef780ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.837+0000 7fef7c89e640 1 --2- 172.21.15.43:0/1310209104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef780b0870 0x7fef780c71e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.837+0000 7fef7c89e640 1 --2- 172.21.15.43:0/1310209104 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef7806c510 0x7fef7814be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.837+0000 7fef7c89e640 1 -- 172.21.15.43:0/1310209104 >> 172.21.15.43:0/1310209104 conn(0x7fef780d17f0 msgr2=0x7fef7806d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:44.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.837+0000 7fef7c89e640 1 -- 172.21.15.43:0/1310209104 shutdown_connections 2024-04-25T08:56:44.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.837+0000 7fef7c89e640 1 -- 172.21.15.43:0/1310209104 wait complete. 2024-04-25T08:56:44.837 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:55:05.150275+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:55:05.853905+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 34, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:56:44.838 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:56:44.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.952+0000 7f7060a07640 1 Processor -- start 2024-04-25T08:56:44.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.952+0000 7f7060a07640 1 -- start start 2024-04-25T08:56:44.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.953+0000 7f7060a07640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f705c15e090 0x7f705c161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.953+0000 7f7060a07640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f705c0b5c70 0x7f705c0d2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.953+0000 7f7060a07640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f705c162610 con 0x7f705c161fe0 2024-04-25T08:56:44.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.953+0000 7f7060a07640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f705c0d1da0 con 0x7f705c15e090 2024-04-25T08:56:44.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.953+0000 7f7060a07640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f705c0d1750 con 0x7f705c0b5c70 2024-04-25T08:56:44.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.954+0000 7f705a59c640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f705c15e090 0x7f705c161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.954+0000 7f705ad9d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f705c0b5c70 0x7f705c0d2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.954+0000 7f7059d9b640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f705c161fe0 0x7f705c0b5560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:52314/0 (socket says 172.21.15.43:52314) 2024-04-25T08:56:44.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.954+0000 7f7059d9b640 1 -- 172.21.15.43:0/1066139684 learned_addr learned my addr 172.21.15.43:0/1066139684 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:56:44.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.954+0000 7f705ad9d640 1 -- 172.21.15.43:0/1066139684 >> v2:172.21.15.193:3300/0 conn(0x7f705c15e090 msgr2=0x7f705c161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:44.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.954+0000 7f705ad9d640 1 --2- 172.21.15.43:0/1066139684 >> v2:172.21.15.193:3300/0 conn(0x7f705c15e090 0x7f705c161aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.954+0000 7f705959a640 1 -- 172.21.15.43:0/1066139684 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1227695228 0 0) 0x7f705c162610 con 0x7f705c161fe0 2024-04-25T08:56:44.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.954+0000 7f705ad9d640 1 -- 172.21.15.43:0/1066139684 >> v1:172.21.15.193:6789/0 conn(0x7f705c161fe0 legacy=0x7f705c0b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.954+0000 7f705ad9d640 1 -- 172.21.15.43:0/1066139684 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f705c0d1a30 con 0x7f705c0b5c70 2024-04-25T08:56:44.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.955+0000 7f705ad9d640 1 --2- 172.21.15.43:0/1066139684 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f705c0b5c70 0x7f705c0d2230 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f704c009d90 tx=0x7f704c02a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f83e0f4e83c968f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:44.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.955+0000 7f705959a640 1 -- 172.21.15.43:0/1066139684 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704c007740 con 0x7f705c0b5c70 2024-04-25T08:56:44.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.955+0000 7f705959a640 1 -- 172.21.15.43:0/1066139684 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f704c0078e0 con 0x7f705c0b5c70 2024-04-25T08:56:44.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.955+0000 7f705959a640 1 -- 172.21.15.43:0/1066139684 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704c007cb0 con 0x7f705c0b5c70 2024-04-25T08:56:44.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.955+0000 7f7060a07640 1 -- 172.21.15.43:0/1066139684 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f705c0b5c70 msgr2=0x7f705c0d2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.955+0000 7f7060a07640 1 --2- 172.21.15.43:0/1066139684 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f705c0b5c70 0x7f705c0d2230 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f704c009d90 tx=0x7f704c02a6f0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.956+0000 7f7060a07640 1 -- 172.21.15.43:0/1066139684 shutdown_connections 2024-04-25T08:56:44.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.956+0000 7f7060a07640 1 --2- 172.21.15.43:0/1066139684 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f705c0b5c70 0x7f705c0d2230 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.956+0000 7f7060a07640 1 --2- 172.21.15.43:0/1066139684 >> v2:172.21.15.193:3300/0 conn(0x7f705c15e090 0x7f705c161aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.956+0000 7f7060a07640 1 -- 172.21.15.43:0/1066139684 >> 172.21.15.43:0/1066139684 conn(0x7f705c0d0560 msgr2=0x7f705c0d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:44.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.956+0000 7f7060a07640 1 -- 172.21.15.43:0/1066139684 shutdown_connections 2024-04-25T08:56:44.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.956+0000 7f7060a07640 1 -- 172.21.15.43:0/1066139684 wait complete. 2024-04-25T08:56:44.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.957+0000 7f7060a07640 1 Processor -- start 2024-04-25T08:56:44.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.957+0000 7f7060a07640 1 -- start start 2024-04-25T08:56:44.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.958+0000 7f7060a07640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f705c0b5c70 0x7f705c157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.958+0000 7f7060a07640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f705c15e090 0x7f705c160e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.958+0000 7f7060a07640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f705c161fe0 0x7f705c161340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.958+0000 7f7060a07640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705c0d2770 con 0x7f705c0b5c70 2024-04-25T08:56:44.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.958+0000 7f7060a07640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f705c0b3280 con 0x7f705c15e090 2024-04-25T08:56:44.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.958+0000 7f7060a07640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f705c0d28f0 con 0x7f705c161fe0 2024-04-25T08:56:44.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.958+0000 7f705a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f705c0b5c70 0x7f705c157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.958+0000 7f705a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f705c0b5c70 0x7f705c157330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59168/0 (socket says 172.21.15.43:59168) 2024-04-25T08:56:44.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.958+0000 7f705a59c640 1 -- 172.21.15.43:0/1311815570 learned_addr learned my addr 172.21.15.43:0/1311815570 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:44.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.958+0000 7f705ad9d640 1 --2- 172.21.15.43:0/1311815570 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f705c161fe0 0x7f705c161340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.958+0000 7f7059d9b640 1 --2- 172.21.15.43:0/1311815570 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f705c15e090 0x7f705c160e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.958+0000 7f705a59c640 1 -- 172.21.15.43:0/1311815570 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f705c161fe0 msgr2=0x7f705c161340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.958+0000 7f705a59c640 1 --2- 172.21.15.43:0/1311815570 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f705c161fe0 0x7f705c161340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.958+0000 7f705a59c640 1 -- 172.21.15.43:0/1311815570 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f705c15e090 msgr2=0x7f705c160e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:44.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.958+0000 7f705a59c640 1 --2- 172.21.15.43:0/1311815570 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f705c15e090 0x7f705c160e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:44.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.958+0000 7f705a59c640 1 -- 172.21.15.43:0/1311815570 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f705c157c80 con 0x7f705c0b5c70 2024-04-25T08:56:44.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.958+0000 7f705a59c640 1 --2- 172.21.15.43:0/1311815570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f705c0b5c70 0x7f705c157330 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f705400e9b0 tx=0x7f705400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:44.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.958+0000 7f7052ffd640 1 -- 172.21.15.43:0/1311815570 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7054017070 con 0x7f705c0b5c70 2024-04-25T08:56:44.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.958+0000 7f7052ffd640 1 -- 172.21.15.43:0/1311815570 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705400c9a0 con 0x7f705c0b5c70 2024-04-25T08:56:44.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.958+0000 7f7060a07640 1 -- 172.21.15.43:0/1311815570 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f705c157f60 con 0x7f705c0b5c70 2024-04-25T08:56:44.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.958+0000 7f7052ffd640 1 -- 172.21.15.43:0/1311815570 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705400cd90 con 0x7f705c0b5c70 2024-04-25T08:56:44.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.959+0000 7f7060a07640 1 -- 172.21.15.43:0/1311815570 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f705c0cc440 con 0x7f705c0b5c70 2024-04-25T08:56:44.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.959+0000 7f7060a07640 1 -- 172.21.15.43:0/1311815570 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f705c06bf00 con 0x7f705c0b5c70 2024-04-25T08:56:44.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.960+0000 7f7052ffd640 1 -- 172.21.15.43:0/1311815570 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f705401a070 con 0x7f705c0b5c70 2024-04-25T08:56:44.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.960+0000 7f7052ffd640 1 --2- 172.21.15.43:0/1311815570 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f703c06b4d0 0x7f703c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:44.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.960+0000 7f7052ffd640 1 -- 172.21.15.43:0/1311815570 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f705408a600 con 0x7f705c0b5c70 2024-04-25T08:56:44.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.961+0000 7f7059d9b640 1 --2- 172.21.15.43:0/1311815570 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f703c06b4d0 0x7f703c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:44.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.961+0000 7f7059d9b640 1 --2- 172.21.15.43:0/1311815570 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f703c06b4d0 0x7f703c06d990 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f705c161aa0 tx=0x7f7044007c00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:44.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:44.964+0000 7f7052ffd640 1 -- 172.21.15.43:0/1311815570 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f705c06bf00 con 0x7f705c0b5c70 2024-04-25T08:56:45.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.143+0000 7f7060a07640 1 -- 172.21.15.43:0/1311815570 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f705c074f60 con 0x7f705c0b5c70 2024-04-25T08:56:45.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.144+0000 7f7052ffd640 1 -- 172.21.15.43:0/1311815570 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7054059460 con 0x7f705c0b5c70 2024-04-25T08:56:45.133 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T08:56:45.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.145+0000 7f7060a07640 1 -- 172.21.15.43:0/1311815570 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f703c06b4d0 msgr2=0x7f703c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:45.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.145+0000 7f7060a07640 1 --2- 172.21.15.43:0/1311815570 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f703c06b4d0 0x7f703c06d990 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f705c161aa0 tx=0x7f7044007c00 comp rx=0 tx=0).stop 2024-04-25T08:56:45.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.146+0000 7f7060a07640 1 -- 172.21.15.43:0/1311815570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f705c0b5c70 msgr2=0x7f705c157330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:45.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.146+0000 7f7060a07640 1 --2- 172.21.15.43:0/1311815570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f705c0b5c70 0x7f705c157330 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f705400e9b0 tx=0x7f705400ee70 comp rx=0 tx=0).stop 2024-04-25T08:56:45.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.146+0000 7f7060a07640 1 -- 172.21.15.43:0/1311815570 shutdown_connections 2024-04-25T08:56:45.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.146+0000 7f7060a07640 1 --2- 172.21.15.43:0/1311815570 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f705c161fe0 0x7f705c161340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:45.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.146+0000 7f7060a07640 1 --2- 172.21.15.43:0/1311815570 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f703c06b4d0 0x7f703c06d990 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:45.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.146+0000 7f7060a07640 1 --2- 172.21.15.43:0/1311815570 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f705c15e090 0x7f705c160e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:45.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.146+0000 7f7060a07640 1 --2- 172.21.15.43:0/1311815570 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f705c0b5c70 0x7f705c157330 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:45.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.146+0000 7f7060a07640 1 -- 172.21.15.43:0/1311815570 >> 172.21.15.43:0/1311815570 conn(0x7f705c0d0560 msgr2=0x7f705c0b50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:45.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.146+0000 7f7060a07640 1 -- 172.21.15.43:0/1311815570 shutdown_connections 2024-04-25T08:56:45.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.146+0000 7f7060a07640 1 -- 172.21.15.43:0/1311815570 wait complete. 2024-04-25T08:56:45.146 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:56:45.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.260+0000 7ff7bb01f640 1 Processor -- start 2024-04-25T08:56:45.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.260+0000 7ff7bb01f640 1 -- start start 2024-04-25T08:56:45.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.261+0000 7ff7bb01f640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff7b40d04a0 0x7ff7b415ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:45.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.261+0000 7ff7bb01f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7b40ce8c0 0x7ff7b40d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:45.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.261+0000 7ff7bb01f640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7b40cf110 con 0x7ff7b415dec0 2024-04-25T08:56:45.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.261+0000 7ff7bb01f640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7b40cf3f0 con 0x7ff7b40d04a0 2024-04-25T08:56:45.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.261+0000 7ff7bb01f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b40cf6d0 con 0x7ff7b40ce8c0 2024-04-25T08:56:45.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.261+0000 7ff7b95bc640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7b40ce8c0 0x7ff7b40d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:45.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.261+0000 7ff7b8dbb640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff7b40d04a0 0x7ff7b415ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:45.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.261+0000 7ff7b95bc640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7b40ce8c0 0x7ff7b40d17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:60000/0 (socket says 172.21.15.43:60000) 2024-04-25T08:56:45.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.261+0000 7ff7b95bc640 1 -- 172.21.15.43:0/748474264 learned_addr learned my addr 172.21.15.43:0/748474264 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:45.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.261+0000 7ff7b8dbb640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff7b40d04a0 0x7ff7b415ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:34868/0 (socket says 172.21.15.43:34868) 2024-04-25T08:56:45.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.262+0000 7ff7b95bc640 1 -- 172.21.15.43:0/748474264 >> v2:172.21.15.193:3300/0 conn(0x7ff7b40d04a0 msgr2=0x7ff7b415ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:45.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.262+0000 7ff7b95bc640 1 --2- 172.21.15.43:0/748474264 >> v2:172.21.15.193:3300/0 conn(0x7ff7b40d04a0 0x7ff7b415ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:45.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.262+0000 7ff7b95bc640 1 -- 172.21.15.43:0/748474264 >> v1:172.21.15.193:6789/0 conn(0x7ff7b415dec0 legacy=0x7ff7b40d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:45.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.262+0000 7ff7b95bc640 1 -- 172.21.15.43:0/748474264 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b40cf9b0 con 0x7ff7b40ce8c0 2024-04-25T08:56:45.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.262+0000 7ff7b8dbb640 1 --2- 172.21.15.43:0/748474264 >> v2:172.21.15.193:3300/0 conn(0x7ff7b40d04a0 0x7ff7b415ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:56:45.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.263+0000 7ff7b95bc640 1 --2- 172.21.15.43:0/748474264 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7b40ce8c0 0x7ff7b40d17a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac009e50 tx=0x7ff7ac02a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=597c16205d326948 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:45.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.263+0000 7ff7b37fe640 1 -- 172.21.15.43:0/748474264 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac036070 con 0x7ff7b40ce8c0 2024-04-25T08:56:45.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.263+0000 7ff7b37fe640 1 -- 172.21.15.43:0/748474264 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7ac031600 con 0x7ff7b40ce8c0 2024-04-25T08:56:45.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.263+0000 7ff7b37fe640 1 -- 172.21.15.43:0/748474264 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac0319d0 con 0x7ff7b40ce8c0 2024-04-25T08:56:45.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.263+0000 7ff7bb01f640 1 -- 172.21.15.43:0/748474264 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7b40ce8c0 msgr2=0x7ff7b40d17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:45.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.263+0000 7ff7bb01f640 1 --2- 172.21.15.43:0/748474264 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7b40ce8c0 0x7ff7b40d17a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac009e50 tx=0x7ff7ac02a710 comp rx=0 tx=0).stop 2024-04-25T08:56:45.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.263+0000 7ff7bb01f640 1 -- 172.21.15.43:0/748474264 shutdown_connections 2024-04-25T08:56:45.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.263+0000 7ff7bb01f640 1 --2- 172.21.15.43:0/748474264 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7b40ce8c0 0x7ff7b40d17a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:45.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.263+0000 7ff7bb01f640 1 --2- 172.21.15.43:0/748474264 >> v2:172.21.15.193:3300/0 conn(0x7ff7b40d04a0 0x7ff7b415ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:45.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.263+0000 7ff7bb01f640 1 -- 172.21.15.43:0/748474264 >> 172.21.15.43:0/748474264 conn(0x7ff7b409cd40 msgr2=0x7ff7b40d3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:45.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.263+0000 7ff7bb01f640 1 -- 172.21.15.43:0/748474264 shutdown_connections 2024-04-25T08:56:45.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.263+0000 7ff7bb01f640 1 -- 172.21.15.43:0/748474264 wait complete. 2024-04-25T08:56:45.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.265+0000 7ff7bb01f640 1 Processor -- start 2024-04-25T08:56:45.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.265+0000 7ff7bb01f640 1 -- start start 2024-04-25T08:56:45.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.265+0000 7ff7bb01f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7b40ce8c0 0x7ff7b40b70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:45.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.266+0000 7ff7bb01f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7b40d04a0 0x7ff7b41556a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:45.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.266+0000 7ff7bb01f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7b415dec0 0x7ff7b40cc070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:45.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.266+0000 7ff7bb01f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b40cee50 con 0x7ff7b40ce8c0 2024-04-25T08:56:45.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.266+0000 7ff7bb01f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b40cecd0 con 0x7ff7b415dec0 2024-04-25T08:56:45.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.266+0000 7ff7bb01f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b40d1ce0 con 0x7ff7b40d04a0 2024-04-25T08:56:45.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.266+0000 7ff7b8dbb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7b40ce8c0 0x7ff7b40b70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:45.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.266+0000 7ff7b8dbb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7b40ce8c0 0x7ff7b40b70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59174/0 (socket says 172.21.15.43:59174) 2024-04-25T08:56:45.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.266+0000 7ff7b8dbb640 1 -- 172.21.15.43:0/2384740832 learned_addr learned my addr 172.21.15.43:0/2384740832 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:45.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.266+0000 7ff7b3fff640 1 --2- 172.21.15.43:0/2384740832 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7b40d04a0 0x7ff7b41556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:45.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.266+0000 7ff7b95bc640 1 --2- 172.21.15.43:0/2384740832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7b415dec0 0x7ff7b40cc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:45.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.266+0000 7ff7b8dbb640 1 -- 172.21.15.43:0/2384740832 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7b40d04a0 msgr2=0x7ff7b41556a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:45.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.266+0000 7ff7b8dbb640 1 --2- 172.21.15.43:0/2384740832 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7b40d04a0 0x7ff7b41556a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:45.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.266+0000 7ff7b8dbb640 1 -- 172.21.15.43:0/2384740832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7b415dec0 msgr2=0x7ff7b40cc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:45.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.266+0000 7ff7b8dbb640 1 --2- 172.21.15.43:0/2384740832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7b415dec0 0x7ff7b40cc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:45.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.266+0000 7ff7b8dbb640 1 -- 172.21.15.43:0/2384740832 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b40b5e70 con 0x7ff7b40ce8c0 2024-04-25T08:56:45.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.266+0000 7ff7b95bc640 1 --2- 172.21.15.43:0/2384740832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7b415dec0 0x7ff7b40cc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:56:45.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.266+0000 7ff7b3fff640 1 --2- 172.21.15.43:0/2384740832 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7b40d04a0 0x7ff7b41556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:56:45.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.266+0000 7ff7b8dbb640 1 --2- 172.21.15.43:0/2384740832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7b40ce8c0 0x7ff7b40b70a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff7a800ebe0 tx=0x7ff7a800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:45.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.266+0000 7ff7b17fa640 1 -- 172.21.15.43:0/2384740832 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a8017070 con 0x7ff7b40ce8c0 2024-04-25T08:56:45.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.266+0000 7ff7b17fa640 1 -- 172.21.15.43:0/2384740832 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7a800cc30 con 0x7ff7b40ce8c0 2024-04-25T08:56:45.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.266+0000 7ff7b17fa640 1 -- 172.21.15.43:0/2384740832 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7a8013470 con 0x7ff7b40ce8c0 2024-04-25T08:56:45.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.266+0000 7ff7bb01f640 1 -- 172.21.15.43:0/2384740832 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7b40b75e0 con 0x7ff7b40ce8c0 2024-04-25T08:56:45.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.266+0000 7ff7bb01f640 1 -- 172.21.15.43:0/2384740832 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7b40b3d50 con 0x7ff7b40ce8c0 2024-04-25T08:56:45.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.267+0000 7ff7bb01f640 1 -- 172.21.15.43:0/2384740832 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7b4074f60 con 0x7ff7b40ce8c0 2024-04-25T08:56:45.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.268+0000 7ff7b17fa640 1 -- 172.21.15.43:0/2384740832 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff7a801a070 con 0x7ff7b40ce8c0 2024-04-25T08:56:45.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.268+0000 7ff7b17fa640 1 --2- 172.21.15.43:0/2384740832 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff79806b4d0 0x7ff79806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:45.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.268+0000 7ff7b17fa640 1 -- 172.21.15.43:0/2384740832 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7ff7a808a230 con 0x7ff7b40ce8c0 2024-04-25T08:56:45.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.268+0000 7ff7b3fff640 1 --2- 172.21.15.43:0/2384740832 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff79806b4d0 0x7ff79806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:45.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.269+0000 7ff7b3fff640 1 --2- 172.21.15.43:0/2384740832 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff79806b4d0 0x7ff79806d990 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff7b40cd860 tx=0x7ff7a0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:45.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.272+0000 7ff7b17fa640 1 -- 172.21.15.43:0/2384740832 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff7a8059010 con 0x7ff7b40ce8c0 2024-04-25T08:56:45.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.450+0000 7ff7bb01f640 1 -- 172.21.15.43:0/2384740832 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff7b40b9730 con 0x7ff7b40ce8c0 2024-04-25T08:56:45.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.450+0000 7ff7b17fa640 1 -- 172.21.15.43:0/2384740832 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff7a8058910 con 0x7ff7b40ce8c0 2024-04-25T08:56:45.440 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T08:56:45.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.452+0000 7ff7bb01f640 1 -- 172.21.15.43:0/2384740832 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff79806b4d0 msgr2=0x7ff79806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:45.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.452+0000 7ff7bb01f640 1 --2- 172.21.15.43:0/2384740832 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff79806b4d0 0x7ff79806d990 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff7b40cd860 tx=0x7ff7a0009040 comp rx=0 tx=0).stop 2024-04-25T08:56:45.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.452+0000 7ff7bb01f640 1 -- 172.21.15.43:0/2384740832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7b40ce8c0 msgr2=0x7ff7b40b70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:45.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.452+0000 7ff7bb01f640 1 --2- 172.21.15.43:0/2384740832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7b40ce8c0 0x7ff7b40b70a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff7a800ebe0 tx=0x7ff7a800c520 comp rx=0 tx=0).stop 2024-04-25T08:56:45.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.453+0000 7ff7bb01f640 1 -- 172.21.15.43:0/2384740832 shutdown_connections 2024-04-25T08:56:45.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.453+0000 7ff7bb01f640 1 --2- 172.21.15.43:0/2384740832 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff79806b4d0 0x7ff79806d990 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:45.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.453+0000 7ff7bb01f640 1 --2- 172.21.15.43:0/2384740832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7b415dec0 0x7ff7b40cc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:45.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.453+0000 7ff7bb01f640 1 --2- 172.21.15.43:0/2384740832 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7b40d04a0 0x7ff7b41556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:45.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.453+0000 7ff7bb01f640 1 --2- 172.21.15.43:0/2384740832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7b40ce8c0 0x7ff7b40b70a0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:45.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.453+0000 7ff7bb01f640 1 -- 172.21.15.43:0/2384740832 >> 172.21.15.43:0/2384740832 conn(0x7ff7b409cd40 msgr2=0x7ff7b4160f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:45.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.453+0000 7ff7bb01f640 1 -- 172.21.15.43:0/2384740832 shutdown_connections 2024-04-25T08:56:45.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.453+0000 7ff7bb01f640 1 -- 172.21.15.43:0/2384740832 wait complete. 2024-04-25T08:56:45.453 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:56:45.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.567+0000 7fdc668ac640 1 Processor -- start 2024-04-25T08:56:45.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.567+0000 7fdc668ac640 1 -- start start 2024-04-25T08:56:45.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.568+0000 7fdc668ac640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdc60162550 0x7fdc600d0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:45.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.568+0000 7fdc668ac640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc600b47c0 con 0x7fdc6015dc40 2024-04-25T08:56:45.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.568+0000 7fdc668ac640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc60071e80 con 0x7fdc6015e020 2024-04-25T08:56:45.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.568+0000 7fdc668ac640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc60072160 con 0x7fdc60162550 2024-04-25T08:56:45.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.568+0000 7fdc5ffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fdc6015dc40 0x7fdc60071670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39472/0 (socket says 172.21.15.43:39472) 2024-04-25T08:56:45.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.568+0000 7fdc5ffff640 1 -- 172.21.15.43:0/1571087534 learned_addr learned my addr 172.21.15.43:0/1571087534 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:56:45.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.568+0000 7fdc64e49640 1 --2- 172.21.15.43:0/1571087534 >> v2:172.21.15.43:3300/0 conn(0x7fdc60162550 0x7fdc600d0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:45.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.569+0000 7fdc64e49640 1 -- 172.21.15.43:0/1571087534 >> v1:172.21.15.193:6789/0 conn(0x7fdc6015e020 legacy=0x7fdc600ce7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:45.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.569+0000 7fdc64e49640 1 -- 172.21.15.43:0/1571087534 >> v1:172.21.15.43:6789/0 conn(0x7fdc6015dc40 legacy=0x7fdc60071670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:45.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.569+0000 7fdc64e49640 1 -- 172.21.15.43:0/1571087534 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc60072440 con 0x7fdc60162550 2024-04-25T08:56:45.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.569+0000 7fdc64e49640 1 --2- 172.21.15.43:0/1571087534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc60162550 0x7fdc600d0af0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fdc58009d90 tx=0x7fdc5802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33ac6d8f2be890a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:45.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.569+0000 7fdc5effd640 1 -- 172.21.15.43:0/1571087534 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc58007740 con 0x7fdc60162550 2024-04-25T08:56:45.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.569+0000 7fdc5effd640 1 -- 172.21.15.43:0/1571087534 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc580078e0 con 0x7fdc60162550 2024-04-25T08:56:45.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.569+0000 7fdc5effd640 1 -- 172.21.15.43:0/1571087534 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc58007cb0 con 0x7fdc60162550 2024-04-25T08:56:45.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.570+0000 7fdc668ac640 1 -- 172.21.15.43:0/1571087534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc60162550 msgr2=0x7fdc600d0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:45.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.570+0000 7fdc668ac640 1 --2- 172.21.15.43:0/1571087534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc60162550 0x7fdc600d0af0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fdc58009d90 tx=0x7fdc5802a6a0 comp rx=0 tx=0).stop 2024-04-25T08:56:45.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.570+0000 7fdc668ac640 1 -- 172.21.15.43:0/1571087534 shutdown_connections 2024-04-25T08:56:45.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.570+0000 7fdc668ac640 1 --2- 172.21.15.43:0/1571087534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc60162550 0x7fdc600d0af0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:45.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.570+0000 7fdc668ac640 1 -- 172.21.15.43:0/1571087534 >> 172.21.15.43:0/1571087534 conn(0x7fdc6006c060 msgr2=0x7fdc6006c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:45.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.570+0000 7fdc668ac640 1 -- 172.21.15.43:0/1571087534 shutdown_connections 2024-04-25T08:56:45.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.570+0000 7fdc668ac640 1 -- 172.21.15.43:0/1571087534 wait complete. 2024-04-25T08:56:45.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.571+0000 7fdc668ac640 1 Processor -- start 2024-04-25T08:56:45.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.571+0000 7fdc668ac640 1 -- start start 2024-04-25T08:56:45.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.572+0000 7fdc668ac640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc6015dc40 0x7fdc6015e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:45.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.572+0000 7fdc668ac640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdc6015e020 0x7fdc60146230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:45.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.572+0000 7fdc668ac640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc60162550 0x7fdc601f76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:45.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.572+0000 7fdc668ac640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc600d11b0 con 0x7fdc6015dc40 2024-04-25T08:56:45.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.572+0000 7fdc668ac640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc600d1030 con 0x7fdc60162550 2024-04-25T08:56:45.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.572+0000 7fdc668ac640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc600d1330 con 0x7fdc6015e020 2024-04-25T08:56:45.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.572+0000 7fdc5ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc6015dc40 0x7fdc6015e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:45.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.572+0000 7fdc5ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc6015dc40 0x7fdc6015e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59198/0 (socket says 172.21.15.43:59198) 2024-04-25T08:56:45.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.572+0000 7fdc5ffff640 1 -- 172.21.15.43:0/3486602419 learned_addr learned my addr 172.21.15.43:0/3486602419 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:45.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.572+0000 7fdc64e49640 1 --2- 172.21.15.43:0/3486602419 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc60162550 0x7fdc601f76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:45.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.573+0000 7fdc5ffff640 1 -- 172.21.15.43:0/3486602419 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdc6015e020 msgr2=0x7fdc60146230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:45.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.573+0000 7fdc5ffff640 1 --2- 172.21.15.43:0/3486602419 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdc6015e020 0x7fdc60146230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:45.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.573+0000 7fdc5ffff640 1 -- 172.21.15.43:0/3486602419 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc60162550 msgr2=0x7fdc601f76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:45.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.573+0000 7fdc5ffff640 1 --2- 172.21.15.43:0/3486602419 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc60162550 0x7fdc601f76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:45.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.573+0000 7fdc5ffff640 1 -- 172.21.15.43:0/3486602419 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc6015b360 con 0x7fdc6015dc40 2024-04-25T08:56:45.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.573+0000 7fdc64e49640 1 --2- 172.21.15.43:0/3486602419 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc60162550 0x7fdc601f76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:45.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.573+0000 7fdc5ffff640 1 --2- 172.21.15.43:0/3486602419 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc6015dc40 0x7fdc6015e4b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fdc4800c3c0 tx=0x7fdc4800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:45.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.573+0000 7fdc5cff9640 1 -- 172.21.15.43:0/3486602419 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc48017070 con 0x7fdc6015dc40 2024-04-25T08:56:45.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.573+0000 7fdc5cff9640 1 -- 172.21.15.43:0/3486602419 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc48005ce0 con 0x7fdc6015dc40 2024-04-25T08:56:45.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.573+0000 7fdc668ac640 1 -- 172.21.15.43:0/3486602419 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc6015eb70 con 0x7fdc6015dc40 2024-04-25T08:56:45.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.573+0000 7fdc5cff9640 1 -- 172.21.15.43:0/3486602419 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc480072f0 con 0x7fdc6015dc40 2024-04-25T08:56:45.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.573+0000 7fdc668ac640 1 -- 172.21.15.43:0/3486602419 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc6015f130 con 0x7fdc6015dc40 2024-04-25T08:56:45.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.574+0000 7fdc668ac640 1 -- 172.21.15.43:0/3486602419 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc600b3b00 con 0x7fdc6015dc40 2024-04-25T08:56:45.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.574+0000 7fdc5cff9640 1 -- 172.21.15.43:0/3486602419 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdc4801a070 con 0x7fdc6015dc40 2024-04-25T08:56:45.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.575+0000 7fdc5cff9640 1 --2- 172.21.15.43:0/3486602419 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdc4006b500 0x7fdc4006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:45.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.575+0000 7fdc5cff9640 1 -- 172.21.15.43:0/3486602419 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fdc480894b0 con 0x7fdc6015dc40 2024-04-25T08:56:45.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.575+0000 7fdc5f7fe640 1 --2- 172.21.15.43:0/3486602419 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdc4006b500 0x7fdc4006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:45.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.575+0000 7fdc5f7fe640 1 --2- 172.21.15.43:0/3486602419 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdc4006b500 0x7fdc4006d9c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdc50006fd0 tx=0x7fdc50006bc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:45.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.579+0000 7fdc5cff9640 1 -- 172.21.15.43:0/3486602419 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdc600b3b00 con 0x7fdc6015dc40 2024-04-25T08:56:45.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.759+0000 7fdc668ac640 1 -- 172.21.15.43:0/3486602419 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdc60074f60 con 0x7fdc6015dc40 2024-04-25T08:56:45.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.759+0000 7fdc5cff9640 1 -- 172.21.15.43:0/3486602419 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdc480583a0 con 0x7fdc6015dc40 2024-04-25T08:56:45.749 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T08:56:45.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.761+0000 7fdc668ac640 1 -- 172.21.15.43:0/3486602419 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdc4006b500 msgr2=0x7fdc4006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:45.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.761+0000 7fdc668ac640 1 --2- 172.21.15.43:0/3486602419 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdc4006b500 0x7fdc4006d9c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdc50006fd0 tx=0x7fdc50006bc0 comp rx=0 tx=0).stop 2024-04-25T08:56:45.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.761+0000 7fdc668ac640 1 -- 172.21.15.43:0/3486602419 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc6015dc40 msgr2=0x7fdc6015e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:45.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.761+0000 7fdc668ac640 1 --2- 172.21.15.43:0/3486602419 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc6015dc40 0x7fdc6015e4b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fdc4800c3c0 tx=0x7fdc4800c880 comp rx=0 tx=0).stop 2024-04-25T08:56:45.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.761+0000 7fdc668ac640 1 -- 172.21.15.43:0/3486602419 shutdown_connections 2024-04-25T08:56:45.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.761+0000 7fdc668ac640 1 --2- 172.21.15.43:0/3486602419 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdc4006b500 0x7fdc4006d9c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:45.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.761+0000 7fdc668ac640 1 --2- 172.21.15.43:0/3486602419 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc60162550 0x7fdc601f76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:45.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.761+0000 7fdc668ac640 1 --2- 172.21.15.43:0/3486602419 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdc6015e020 0x7fdc60146230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:45.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.761+0000 7fdc668ac640 1 --2- 172.21.15.43:0/3486602419 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc6015dc40 0x7fdc6015e4b0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:45.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.761+0000 7fdc668ac640 1 -- 172.21.15.43:0/3486602419 >> 172.21.15.43:0/3486602419 conn(0x7fdc6006c060 msgr2=0x7fdc6006c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:45.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.761+0000 7fdc668ac640 1 -- 172.21.15.43:0/3486602419 shutdown_connections 2024-04-25T08:56:45.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.761+0000 7fdc668ac640 1 -- 172.21.15.43:0/3486602419 wait complete. 2024-04-25T08:56:45.762 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:56:45.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.874+0000 7f950a7e7640 1 Processor -- start 2024-04-25T08:56:45.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.874+0000 7f950a7e7640 1 -- start start 2024-04-25T08:56:45.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.875+0000 7f950a7e7640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f950406b240 0x7f95040ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:45.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.875+0000 7f950a7e7640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95040cf6e0 con 0x7f95040cf250 2024-04-25T08:56:45.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.875+0000 7f950a7e7640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95040cf9c0 con 0x7f950415ded0 2024-04-25T08:56:45.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.875+0000 7f950a7e7640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f950415e2b0 con 0x7f950406b240 2024-04-25T08:56:45.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.875+0000 7f9508d84640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f950406b240 0x7f95040ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:45.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.875+0000 7f95037fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f95040cf250 0x7f95040b4db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39488/0 (socket says 172.21.15.43:39488) 2024-04-25T08:56:45.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.875+0000 7f95037fe640 1 -- 172.21.15.43:0/145763073 learned_addr learned my addr 172.21.15.43:0/145763073 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:56:45.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.875+0000 7f9508d84640 1 -- 172.21.15.43:0/145763073 >> v1:172.21.15.193:6789/0 conn(0x7f950415ded0 legacy=0x7f950406ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:45.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.875+0000 7f9508d84640 1 -- 172.21.15.43:0/145763073 >> v1:172.21.15.43:6789/0 conn(0x7f95040cf250 legacy=0x7f95040b4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:45.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.875+0000 7f9502ffd640 1 -- 172.21.15.43:0/145763073 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1262021572 0 0) 0x7f95040cf6e0 con 0x7f95040cf250 2024-04-25T08:56:45.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.875+0000 7f9508d84640 1 -- 172.21.15.43:0/145763073 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95040cfca0 con 0x7f950406b240 2024-04-25T08:56:45.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.876+0000 7f9508d84640 1 --2- 172.21.15.43:0/145763073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f950406b240 0x7f95040ce740 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f94fc009d90 tx=0x7f94fc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3acc7995a18045d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:45.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.876+0000 7f9502ffd640 1 -- 172.21.15.43:0/145763073 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94fc007740 con 0x7f950406b240 2024-04-25T08:56:45.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.876+0000 7f9502ffd640 1 -- 172.21.15.43:0/145763073 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94fc0078e0 con 0x7f950406b240 2024-04-25T08:56:45.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.876+0000 7f9502ffd640 1 -- 172.21.15.43:0/145763073 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94fc007cb0 con 0x7f950406b240 2024-04-25T08:56:45.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.877+0000 7f950a7e7640 1 -- 172.21.15.43:0/145763073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f950406b240 msgr2=0x7f95040ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:45.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.877+0000 7f950a7e7640 1 --2- 172.21.15.43:0/145763073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f950406b240 0x7f95040ce740 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f94fc009d90 tx=0x7f94fc02a6a0 comp rx=0 tx=0).stop 2024-04-25T08:56:45.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.877+0000 7f950a7e7640 1 -- 172.21.15.43:0/145763073 shutdown_connections 2024-04-25T08:56:45.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.877+0000 7f950a7e7640 1 --2- 172.21.15.43:0/145763073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f950406b240 0x7f95040ce740 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:45.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.877+0000 7f950a7e7640 1 -- 172.21.15.43:0/145763073 >> 172.21.15.43:0/145763073 conn(0x7f95040b2490 msgr2=0x7f95040b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:45.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.877+0000 7f950a7e7640 1 -- 172.21.15.43:0/145763073 shutdown_connections 2024-04-25T08:56:45.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.877+0000 7f950a7e7640 1 -- 172.21.15.43:0/145763073 wait complete. 2024-04-25T08:56:45.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.878+0000 7f950a7e7640 1 Processor -- start 2024-04-25T08:56:45.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.878+0000 7f950a7e7640 1 -- start start 2024-04-25T08:56:45.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.879+0000 7f950a7e7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f950406b240 0x7f9504160da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:45.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.879+0000 7f950a7e7640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f950415ded0 0x7f950415af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:45.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.879+0000 7f950a7e7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9504143fe0 0x7f9504144450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:45.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.879+0000 7f950a7e7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95040cefd0 con 0x7f9504143fe0 2024-04-25T08:56:45.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.879+0000 7f950a7e7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95040cee50 con 0x7f950406b240 2024-04-25T08:56:45.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.879+0000 7f950a7e7640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95040ad8e0 con 0x7f950415ded0 2024-04-25T08:56:45.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.879+0000 7f9508d84640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9504143fe0 0x7f9504144450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:45.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.879+0000 7f9508d84640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9504143fe0 0x7f9504144450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59216/0 (socket says 172.21.15.43:59216) 2024-04-25T08:56:45.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.879+0000 7f9508d84640 1 -- 172.21.15.43:0/1635838061 learned_addr learned my addr 172.21.15.43:0/1635838061 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:45.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.879+0000 7f95037fe640 1 --2- 172.21.15.43:0/1635838061 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f950415ded0 0x7f950415af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:45.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.879+0000 7f9503fff640 1 --2- 172.21.15.43:0/1635838061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f950406b240 0x7f9504160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:45.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.879+0000 7f9508d84640 1 -- 172.21.15.43:0/1635838061 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f950415ded0 msgr2=0x7f950415af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:45.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.879+0000 7f9508d84640 1 --2- 172.21.15.43:0/1635838061 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f950415ded0 0x7f950415af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:45.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.879+0000 7f9508d84640 1 -- 172.21.15.43:0/1635838061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f950406b240 msgr2=0x7f9504160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:45.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.879+0000 7f9508d84640 1 --2- 172.21.15.43:0/1635838061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f950406b240 0x7f9504160da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:45.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.879+0000 7f9508d84640 1 -- 172.21.15.43:0/1635838061 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f950415a500 con 0x7f9504143fe0 2024-04-25T08:56:45.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.879+0000 7f9508d84640 1 --2- 172.21.15.43:0/1635838061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9504143fe0 0x7f9504144450 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f94fc009d90 tx=0x7f94fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:45.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.879+0000 7f9500ff9640 1 -- 172.21.15.43:0/1635838061 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94fc040070 con 0x7f9504143fe0 2024-04-25T08:56:45.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.879+0000 7f9500ff9640 1 -- 172.21.15.43:0/1635838061 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94fc031c90 con 0x7f9504143fe0 2024-04-25T08:56:45.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.879+0000 7f950a7e7640 1 -- 172.21.15.43:0/1635838061 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95041533b0 con 0x7f9504143fe0 2024-04-25T08:56:45.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.879+0000 7f9500ff9640 1 -- 172.21.15.43:0/1635838061 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94fc03c640 con 0x7f9504143fe0 2024-04-25T08:56:45.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.880+0000 7f950a7e7640 1 -- 172.21.15.43:0/1635838061 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f950415a7a0 con 0x7f9504143fe0 2024-04-25T08:56:45.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.881+0000 7f950a7e7640 1 -- 172.21.15.43:0/1635838061 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f950406b800 con 0x7f9504143fe0 2024-04-25T08:56:45.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.881+0000 7f9500ff9640 1 -- 172.21.15.43:0/1635838061 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f94fc039040 con 0x7f9504143fe0 2024-04-25T08:56:45.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.881+0000 7f9500ff9640 1 --2- 172.21.15.43:0/1635838061 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f94e406b4d0 0x7f94e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:45.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.882+0000 7f9500ff9640 1 -- 172.21.15.43:0/1635838061 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f94fc0ab7c0 con 0x7f9504143fe0 2024-04-25T08:56:45.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.882+0000 7f9503fff640 1 --2- 172.21.15.43:0/1635838061 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f94e406b4d0 0x7f94e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:45.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.882+0000 7f9503fff640 1 --2- 172.21.15.43:0/1635838061 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f94e406b4d0 0x7f94e406d990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f94ec006fd0 tx=0x7f94ec008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:45.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:45.886+0000 7f9500ff9640 1 -- 172.21.15.43:0/1635838061 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f950406b800 con 0x7f9504143fe0 2024-04-25T08:56:46.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.080+0000 7f950a7e7640 1 -- 172.21.15.43:0/1635838061 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f95040b2870 con 0x7f9504143fe0 2024-04-25T08:56:46.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.080+0000 7f9500ff9640 1 -- 172.21.15.43:0/1635838061 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v35) v1 ==== 80+0+0 (secure 0 0 0) 0x7f95040b2870 con 0x7f9504143fe0 2024-04-25T08:56:46.070 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-04-25T08:56:46.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.083+0000 7f950a7e7640 1 -- 172.21.15.43:0/1635838061 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f94e406b4d0 msgr2=0x7f94e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:46.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.083+0000 7f950a7e7640 1 --2- 172.21.15.43:0/1635838061 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f94e406b4d0 0x7f94e406d990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f94ec006fd0 tx=0x7f94ec008040 comp rx=0 tx=0).stop 2024-04-25T08:56:46.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.083+0000 7f950a7e7640 1 -- 172.21.15.43:0/1635838061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9504143fe0 msgr2=0x7f9504144450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:46.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.083+0000 7f950a7e7640 1 --2- 172.21.15.43:0/1635838061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9504143fe0 0x7f9504144450 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f94fc009d90 tx=0x7f94fc004a40 comp rx=0 tx=0).stop 2024-04-25T08:56:46.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.083+0000 7f950a7e7640 1 -- 172.21.15.43:0/1635838061 shutdown_connections 2024-04-25T08:56:46.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.083+0000 7f950a7e7640 1 --2- 172.21.15.43:0/1635838061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9504143fe0 0x7f9504144450 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:46.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.083+0000 7f950a7e7640 1 --2- 172.21.15.43:0/1635838061 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f950415ded0 0x7f950415af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:46.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.083+0000 7f950a7e7640 1 --2- 172.21.15.43:0/1635838061 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f94e406b4d0 0x7f94e406d990 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:46.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.083+0000 7f950a7e7640 1 --2- 172.21.15.43:0/1635838061 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f950406b240 0x7f9504160da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:46.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.083+0000 7f950a7e7640 1 -- 172.21.15.43:0/1635838061 >> 172.21.15.43:0/1635838061 conn(0x7f95040b2490 msgr2=0x7f95040ae8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:46.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.083+0000 7f950a7e7640 1 -- 172.21.15.43:0/1635838061 shutdown_connections 2024-04-25T08:56:46.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.083+0000 7f950a7e7640 1 -- 172.21.15.43:0/1635838061 wait complete. 2024-04-25T08:56:46.083 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi043.front.sepia.ceph.com... 2024-04-25T08:56:46.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:56:46.083 DEBUG:teuthology.orchestra.run.smithi043:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T08:56:46.106 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[48779]: fuse finished with error 0 and tester_r 0 2024-04-25T08:56:46.110 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-04-25T08:56:46.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:56:46.110 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:56:46.280 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T08:56:46.280 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:56:46.299 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19508 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:56:46.299 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi193.front.sepia.ceph.com 2024-04-25T08:56:46.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:56:46.299 DEBUG:teuthology.orchestra.run.smithi193:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T08:56:46.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:56:46.357 INFO:teuthology.orchestra.run.smithi193.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T08:56:46.357 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:56:46.357 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:56:46.374 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2374 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:56:46.375 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-04-25T08:56:46.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:56:46.375 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T08:56:46.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:56:46.430 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T08:56:46.430 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T08:56:46.431 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:56:46.449 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19508 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:56:46.449 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi193.front.sepia.ceph.com 2024-04-25T08:56:46.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:56:46.450 DEBUG:teuthology.orchestra.run.smithi193:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T08:56:46.505 INFO:teuthology.orchestra.run.smithi193.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T08:56:46.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:56:46.506 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:56:46.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.618+0000 7f6c2d65a640 1 Processor -- start 2024-04-25T08:56:46.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.618+0000 7f6c2d65a640 1 -- start start 2024-04-25T08:56:46.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.618+0000 7f6c2d65a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c2815ded0 0x7f6c280ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:46.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.618+0000 7f6c2d65a640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6c2806b240 0x7f6c280ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:46.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.618+0000 7f6c2d65a640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c280cf6e0 con 0x7f6c280cf250 2024-04-25T08:56:46.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.618+0000 7f6c2d65a640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c2815e2b0 con 0x7f6c2806b240 2024-04-25T08:56:46.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.618+0000 7f6c2d65a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c280cf9c0 con 0x7f6c2815ded0 2024-04-25T08:56:46.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.619+0000 7f6c26ffd640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c2815ded0 0x7f6c280ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:46.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.619+0000 7f6c277fe640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6c2806b240 0x7f6c280ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:46.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.619+0000 7f6c267fc640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f6c280cf250 0x7f6c2806ab30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:52334/0 (socket says 172.21.15.43:52334) 2024-04-25T08:56:46.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.619+0000 7f6c267fc640 1 -- 172.21.15.43:0/835176095 learned_addr learned my addr 172.21.15.43:0/835176095 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:56:46.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.619+0000 7f6c26ffd640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c2815ded0 0x7f6c280ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:60030/0 (socket says 172.21.15.43:60030) 2024-04-25T08:56:46.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.619+0000 7f6c25ffb640 1 -- 172.21.15.43:0/835176095 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3650487574 0 0) 0x7f6c280cf6e0 con 0x7f6c280cf250 2024-04-25T08:56:46.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.619+0000 7f6c25ffb640 1 -- 172.21.15.43:0/835176095 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c10003520 con 0x7f6c280cf250 2024-04-25T08:56:46.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.619+0000 7f6c277fe640 1 -- 172.21.15.43:0/835176095 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c2815ded0 msgr2=0x7f6c280ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:46.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.619+0000 7f6c277fe640 1 --2- 172.21.15.43:0/835176095 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c2815ded0 0x7f6c280ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:46.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.619+0000 7f6c277fe640 1 -- 172.21.15.43:0/835176095 >> v1:172.21.15.193:6789/0 conn(0x7f6c280cf250 legacy=0x7f6c2806ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:46.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.619+0000 7f6c277fe640 1 -- 172.21.15.43:0/835176095 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c280cfca0 con 0x7f6c2806b240 2024-04-25T08:56:46.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.619+0000 7f6c26ffd640 1 --2- 172.21.15.43:0/835176095 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c2815ded0 0x7f6c280ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:56:46.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.620+0000 7f6c277fe640 1 --2- 172.21.15.43:0/835176095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c2806b240 0x7f6c280ad8e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6c20009d90 tx=0x7f6c2002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22fe79c2d5de60ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:46.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.620+0000 7f6c25ffb640 1 -- 172.21.15.43:0/835176095 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c20007740 con 0x7f6c2806b240 2024-04-25T08:56:46.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.620+0000 7f6c25ffb640 1 -- 172.21.15.43:0/835176095 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c200078e0 con 0x7f6c2806b240 2024-04-25T08:56:46.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.620+0000 7f6c25ffb640 1 -- 172.21.15.43:0/835176095 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c20007cb0 con 0x7f6c2806b240 2024-04-25T08:56:46.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.621+0000 7f6c2d65a640 1 -- 172.21.15.43:0/835176095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c2806b240 msgr2=0x7f6c280ad8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:46.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.621+0000 7f6c2d65a640 1 --2- 172.21.15.43:0/835176095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c2806b240 0x7f6c280ad8e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6c20009d90 tx=0x7f6c2002a660 comp rx=0 tx=0).stop 2024-04-25T08:56:46.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.621+0000 7f6c2d65a640 1 -- 172.21.15.43:0/835176095 shutdown_connections 2024-04-25T08:56:46.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.621+0000 7f6c2d65a640 1 --2- 172.21.15.43:0/835176095 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c2806b240 0x7f6c280ad8e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:46.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.621+0000 7f6c2d65a640 1 --2- 172.21.15.43:0/835176095 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c2815ded0 0x7f6c280ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:46.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.621+0000 7f6c2d65a640 1 -- 172.21.15.43:0/835176095 >> 172.21.15.43:0/835176095 conn(0x7f6c280b2490 msgr2=0x7f6c280b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:46.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.621+0000 7f6c2d65a640 1 -- 172.21.15.43:0/835176095 shutdown_connections 2024-04-25T08:56:46.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.621+0000 7f6c2d65a640 1 -- 172.21.15.43:0/835176095 wait complete. 2024-04-25T08:56:46.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.622+0000 7f6c2d65a640 1 Processor -- start 2024-04-25T08:56:46.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.622+0000 7f6c2d65a640 1 -- start start 2024-04-25T08:56:46.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.623+0000 7f6c2d65a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c2806b240 0x7f6c280aeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:46.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.623+0000 7f6c2d65a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c2815ded0 0x7f6c28160890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:46.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.623+0000 7f6c2d65a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c28160dd0 0x7f6c281f7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:46.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.623+0000 7f6c2d65a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c280cefd0 con 0x7f6c28160dd0 2024-04-25T08:56:46.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.623+0000 7f6c2d65a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c280cee50 con 0x7f6c2815ded0 2024-04-25T08:56:46.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.623+0000 7f6c2d65a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c280b4db0 con 0x7f6c2806b240 2024-04-25T08:56:46.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.623+0000 7f6c277fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c28160dd0 0x7f6c281f7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:46.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.623+0000 7f6c277fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c28160dd0 0x7f6c281f7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59218/0 (socket says 172.21.15.43:59218) 2024-04-25T08:56:46.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.623+0000 7f6c277fe640 1 -- 172.21.15.43:0/1709421596 learned_addr learned my addr 172.21.15.43:0/1709421596 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:46.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.623+0000 7f6c267fc640 1 --2- 172.21.15.43:0/1709421596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c2815ded0 0x7f6c28160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:46.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.623+0000 7f6c26ffd640 1 --2- 172.21.15.43:0/1709421596 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c2806b240 0x7f6c280aeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:46.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.623+0000 7f6c277fe640 1 -- 172.21.15.43:0/1709421596 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c2806b240 msgr2=0x7f6c280aeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:46.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.623+0000 7f6c277fe640 1 --2- 172.21.15.43:0/1709421596 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c2806b240 0x7f6c280aeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:46.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.623+0000 7f6c277fe640 1 -- 172.21.15.43:0/1709421596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c2815ded0 msgr2=0x7f6c28160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:46.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.623+0000 7f6c277fe640 1 --2- 172.21.15.43:0/1709421596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c2815ded0 0x7f6c28160890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:46.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.623+0000 7f6c277fe640 1 -- 172.21.15.43:0/1709421596 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c28161260 con 0x7f6c28160dd0 2024-04-25T08:56:46.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.623+0000 7f6c277fe640 1 --2- 172.21.15.43:0/1709421596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c28160dd0 0x7f6c281f7310 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6c2002ac50 tx=0x7f6c20004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:46.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.624+0000 7f6c0f7fe640 1 -- 172.21.15.43:0/1709421596 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c20040070 con 0x7f6c28160dd0 2024-04-25T08:56:46.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.624+0000 7f6c0f7fe640 1 -- 172.21.15.43:0/1709421596 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c20031bc0 con 0x7f6c28160dd0 2024-04-25T08:56:46.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.624+0000 7f6c2d65a640 1 -- 172.21.15.43:0/1709421596 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c281f3ae0 con 0x7f6c28160dd0 2024-04-25T08:56:46.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.624+0000 7f6c0f7fe640 1 -- 172.21.15.43:0/1709421596 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c20009450 con 0x7f6c28160dd0 2024-04-25T08:56:46.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.624+0000 7f6c2d65a640 1 -- 172.21.15.43:0/1709421596 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c281f40a0 con 0x7f6c28160dd0 2024-04-25T08:56:46.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.625+0000 7f6c2d65a640 1 -- 172.21.15.43:0/1709421596 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c28074ef0 con 0x7f6c28160dd0 2024-04-25T08:56:46.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.625+0000 7f6c0f7fe640 1 -- 172.21.15.43:0/1709421596 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6c20039040 con 0x7f6c28160dd0 2024-04-25T08:56:46.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.625+0000 7f6c0f7fe640 1 --2- 172.21.15.43:0/1709421596 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6c1406b4d0 0x7f6c1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:46.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.626+0000 7f6c0f7fe640 1 -- 172.21.15.43:0/1709421596 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f6c200aba20 con 0x7f6c28160dd0 2024-04-25T08:56:46.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.626+0000 7f6c26ffd640 1 --2- 172.21.15.43:0/1709421596 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6c1406b4d0 0x7f6c1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:46.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.626+0000 7f6c26ffd640 1 --2- 172.21.15.43:0/1709421596 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6c1406b4d0 0x7f6c1406d990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6c18009a30 tx=0x7f6c18008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:46.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.630+0000 7f6c0f7fe640 1 -- 172.21.15.43:0/1709421596 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6c28161260 con 0x7f6c28160dd0 2024-04-25T08:56:46.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.846+0000 7f6c2d65a640 1 -- 172.21.15.43:0/1709421596 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c280b31e0 con 0x7f6c28160dd0 2024-04-25T08:56:46.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.846+0000 7f6c0f7fe640 1 -- 172.21.15.43:0/1709421596 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4934 (secure 0 0 0) 0x7f6c2007a9a0 con 0x7f6c28160dd0 2024-04-25T08:56:46.836 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:56:46.836 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4984,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/715858990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":715858990},{"type":"v1","addr":"172.21.15.193:6836","nonce":715858990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4987,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2342354979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2342354979},{"type":"v1","addr":"172.21.15.193:6837","nonce":2342354979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":5205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/491504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":491504707},{"type":"v1","addr":"172.21.15.43:6834","nonce":491504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T08:55:06.885018+0000","modified":"2024-04-25T08:55:14.001880+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5208},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5208":{"gid":5208,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/242762542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":242762542},{"type":"v1","addr":"172.21.15.43:6835","nonce":242762542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T08:56:46.836 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 14 2024-04-25T08:56:46.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.848+0000 7f6c2d65a640 1 -- 172.21.15.43:0/1709421596 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6c1406b4d0 msgr2=0x7f6c1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:46.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.848+0000 7f6c2d65a640 1 --2- 172.21.15.43:0/1709421596 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6c1406b4d0 0x7f6c1406d990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6c18009a30 tx=0x7f6c18008040 comp rx=0 tx=0).stop 2024-04-25T08:56:46.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.848+0000 7f6c2d65a640 1 -- 172.21.15.43:0/1709421596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c28160dd0 msgr2=0x7f6c281f7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:46.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.848+0000 7f6c2d65a640 1 --2- 172.21.15.43:0/1709421596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c28160dd0 0x7f6c281f7310 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f6c2002ac50 tx=0x7f6c20004a40 comp rx=0 tx=0).stop 2024-04-25T08:56:46.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.849+0000 7f6c2d65a640 1 -- 172.21.15.43:0/1709421596 shutdown_connections 2024-04-25T08:56:46.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.849+0000 7f6c2d65a640 1 --2- 172.21.15.43:0/1709421596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6c28160dd0 0x7f6c281f7310 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:46.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.849+0000 7f6c2d65a640 1 --2- 172.21.15.43:0/1709421596 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6c1406b4d0 0x7f6c1406d990 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:46.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.849+0000 7f6c2d65a640 1 --2- 172.21.15.43:0/1709421596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6c2815ded0 0x7f6c28160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:46.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.849+0000 7f6c2d65a640 1 --2- 172.21.15.43:0/1709421596 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6c2806b240 0x7f6c280aeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:46.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.849+0000 7f6c2d65a640 1 -- 172.21.15.43:0/1709421596 >> 172.21.15.43:0/1709421596 conn(0x7f6c280b2490 msgr2=0x7f6c280b3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:46.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.849+0000 7f6c2d65a640 1 -- 172.21.15.43:0/1709421596 shutdown_connections 2024-04-25T08:56:46.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.849+0000 7f6c2d65a640 1 -- 172.21.15.43:0/1709421596 wait complete. 2024-04-25T08:56:46.850 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:56:46.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.961+0000 7f541e4b6640 1 Processor -- start 2024-04-25T08:56:46.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.961+0000 7f541e4b6640 1 -- start start 2024-04-25T08:56:46.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.962+0000 7f541e4b6640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f541815a9d0 0x7f541806d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:46.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.962+0000 7f541e4b6640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f541806d7d0 0x7f5418071a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:46.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.962+0000 7f541e4b6640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f5418071f90 0x7f54180b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:46.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.962+0000 7f541e4b6640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f541806dc30 con 0x7f541806d7d0 2024-04-25T08:56:46.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.962+0000 7f541e4b6640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5418072420 con 0x7f5418071f90 2024-04-25T08:56:46.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.962+0000 7f541e4b6640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54180704f0 con 0x7f541815a9d0 2024-04-25T08:56:46.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.962+0000 7f54177fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f541806d7d0 0x7f5418071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:46.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.962+0000 7f54177fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f541806d7d0 0x7f5418071a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59222/0 (socket says 172.21.15.43:59222) 2024-04-25T08:56:46.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.962+0000 7f54177fe640 1 -- 172.21.15.43:0/3957753061 learned_addr learned my addr 172.21.15.43:0/3957753061 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:46.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.962+0000 7f5417fff640 1 --2- 172.21.15.43:0/3957753061 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f541815a9d0 0x7f541806d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:46.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.962+0000 7f541ca53640 1 --2- 172.21.15.43:0/3957753061 >> v2:172.21.15.193:3300/0 conn(0x7f5418071f90 0x7f54180b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:46.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.962+0000 7f54177fe640 1 -- 172.21.15.43:0/3957753061 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f541815a9d0 msgr2=0x7f541806d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:46.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.962+0000 7f54177fe640 1 --2- 172.21.15.43:0/3957753061 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f541815a9d0 0x7f541806d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:46.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.962+0000 7f54177fe640 1 -- 172.21.15.43:0/3957753061 >> v2:172.21.15.193:3300/0 conn(0x7f5418071f90 msgr2=0x7f54180b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:46.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.962+0000 7f54177fe640 1 --2- 172.21.15.43:0/3957753061 >> v2:172.21.15.193:3300/0 conn(0x7f5418071f90 0x7f54180b3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:46.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.962+0000 7f54177fe640 1 -- 172.21.15.43:0/3957753061 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54180707d0 con 0x7f541806d7d0 2024-04-25T08:56:46.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.963+0000 7f54177fe640 1 --2- 172.21.15.43:0/3957753061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f541806d7d0 0x7f5418071a50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f54180702e0 tx=0x7f540c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=959a59d6ba996226 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:46.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.963+0000 7f5416ffd640 1 -- 172.21.15.43:0/3957753061 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540c037070 con 0x7f541806d7d0 2024-04-25T08:56:46.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.963+0000 7f5416ffd640 1 -- 172.21.15.43:0/3957753061 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f540c032440 con 0x7f541806d7d0 2024-04-25T08:56:46.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.963+0000 7f5416ffd640 1 -- 172.21.15.43:0/3957753061 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540c032810 con 0x7f541806d7d0 2024-04-25T08:56:46.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.964+0000 7f541e4b6640 1 -- 172.21.15.43:0/3957753061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f541806d7d0 msgr2=0x7f5418071a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:46.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.964+0000 7f541e4b6640 1 --2- 172.21.15.43:0/3957753061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f541806d7d0 0x7f5418071a50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f54180702e0 tx=0x7f540c02a1e0 comp rx=0 tx=0).stop 2024-04-25T08:56:46.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.964+0000 7f541e4b6640 1 -- 172.21.15.43:0/3957753061 shutdown_connections 2024-04-25T08:56:46.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.964+0000 7f541e4b6640 1 --2- 172.21.15.43:0/3957753061 >> v2:172.21.15.193:3300/0 conn(0x7f5418071f90 0x7f54180b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:46.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.964+0000 7f541e4b6640 1 --2- 172.21.15.43:0/3957753061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f541806d7d0 0x7f5418071a50 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:46.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.964+0000 7f541e4b6640 1 --2- 172.21.15.43:0/3957753061 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f541815a9d0 0x7f541806d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:46.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.964+0000 7f541e4b6640 1 -- 172.21.15.43:0/3957753061 >> 172.21.15.43:0/3957753061 conn(0x7f54180b1480 msgr2=0x7f54180b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:46.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.964+0000 7f541e4b6640 1 -- 172.21.15.43:0/3957753061 shutdown_connections 2024-04-25T08:56:46.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.964+0000 7f541e4b6640 1 -- 172.21.15.43:0/3957753061 wait complete. 2024-04-25T08:56:46.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.965+0000 7f541e4b6640 1 Processor -- start 2024-04-25T08:56:46.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.965+0000 7f541e4b6640 1 -- start start 2024-04-25T08:56:46.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.966+0000 7f541e4b6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f541806d7d0 0x7f5418150380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:46.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.966+0000 7f541e4b6640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5418071f90 0x7f5418144ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:46.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.966+0000 7f541e4b6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5418150900 0x7f5418152180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:46.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.966+0000 7f541e4b6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54180712b0 con 0x7f5418150900 2024-04-25T08:56:46.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.966+0000 7f541e4b6640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5418070d90 con 0x7f541806d7d0 2024-04-25T08:56:46.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.966+0000 7f541e4b6640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54180b3ed0 con 0x7f5418071f90 2024-04-25T08:56:46.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.966+0000 7f541ca53640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5418150900 0x7f5418152180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:46.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.966+0000 7f541ca53640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5418150900 0x7f5418152180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59230/0 (socket says 172.21.15.43:59230) 2024-04-25T08:56:46.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.966+0000 7f541ca53640 1 -- 172.21.15.43:0/3775319478 learned_addr learned my addr 172.21.15.43:0/3775319478 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:46.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.966+0000 7f5417fff640 1 --2- 172.21.15.43:0/3775319478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f541806d7d0 0x7f5418150380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:46.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.966+0000 7f54177fe640 1 --2- 172.21.15.43:0/3775319478 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5418071f90 0x7f5418144ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:46.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.966+0000 7f541ca53640 1 -- 172.21.15.43:0/3775319478 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5418071f90 msgr2=0x7f5418144ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:46.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.966+0000 7f541ca53640 1 --2- 172.21.15.43:0/3775319478 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5418071f90 0x7f5418144ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:46.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.966+0000 7f541ca53640 1 -- 172.21.15.43:0/3775319478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f541806d7d0 msgr2=0x7f5418150380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:46.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.966+0000 7f541ca53640 1 --2- 172.21.15.43:0/3775319478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f541806d7d0 0x7f5418150380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:46.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.966+0000 7f541ca53640 1 -- 172.21.15.43:0/3775319478 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54181510e0 con 0x7f5418150900 2024-04-25T08:56:46.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.966+0000 7f5417fff640 1 --2- 172.21.15.43:0/3775319478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f541806d7d0 0x7f5418150380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:46.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.966+0000 7f54177fe640 1 --2- 172.21.15.43:0/3775319478 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5418071f90 0x7f5418144ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:46.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.966+0000 7f541ca53640 1 --2- 172.21.15.43:0/3775319478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5418150900 0x7f5418152180 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5410009d20 tx=0x7f541000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:46.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.967+0000 7f5414ff9640 1 -- 172.21.15.43:0/3775319478 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5410018020 con 0x7f5418150900 2024-04-25T08:56:46.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.967+0000 7f5414ff9640 1 -- 172.21.15.43:0/3775319478 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5410009b10 con 0x7f5418150900 2024-04-25T08:56:46.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.967+0000 7f541e4b6640 1 -- 172.21.15.43:0/3775319478 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54181513c0 con 0x7f5418150900 2024-04-25T08:56:46.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.967+0000 7f5414ff9640 1 -- 172.21.15.43:0/3775319478 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54100146c0 con 0x7f5418150900 2024-04-25T08:56:46.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.967+0000 7f541e4b6640 1 -- 172.21.15.43:0/3775319478 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5418151980 con 0x7f5418150900 2024-04-25T08:56:46.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.968+0000 7f541e4b6640 1 -- 172.21.15.43:0/3775319478 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5418074ef0 con 0x7f5418150900 2024-04-25T08:56:46.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.968+0000 7f5414ff9640 1 -- 172.21.15.43:0/3775319478 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5410005d10 con 0x7f5418150900 2024-04-25T08:56:46.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.969+0000 7f5414ff9640 1 --2- 172.21.15.43:0/3775319478 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f540406b500 0x7f540406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:46.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.969+0000 7f5414ff9640 1 -- 172.21.15.43:0/3775319478 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f541008b0e0 con 0x7f5418150900 2024-04-25T08:56:46.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.969+0000 7f5417fff640 1 --2- 172.21.15.43:0/3775319478 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f540406b500 0x7f540406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:46.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.969+0000 7f5417fff640 1 --2- 172.21.15.43:0/3775319478 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f540406b500 0x7f540406d9c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f54080099a0 tx=0x7f5408008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:46.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:46.973+0000 7f5414ff9640 1 -- 172.21.15.43:0/3775319478 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5410022030 con 0x7f5418150900 2024-04-25T08:56:47.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.154+0000 7f541e4b6640 1 -- 172.21.15.43:0/3775319478 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f541815d600 con 0x7f5418150900 2024-04-25T08:56:47.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.155+0000 7f5414ff9640 1 -- 172.21.15.43:0/3775319478 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16698 (secure 0 0 0) 0x7f5410010040 con 0x7f5418150900 2024-04-25T08:56:47.145 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:56:47.145 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":35,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:56:22.378131+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:55:05.150275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T08:55:05.853905+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":34,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:56:47.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.157+0000 7f541e4b6640 1 -- 172.21.15.43:0/3775319478 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f540406b500 msgr2=0x7f540406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:47.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.157+0000 7f541e4b6640 1 --2- 172.21.15.43:0/3775319478 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f540406b500 0x7f540406d9c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f54080099a0 tx=0x7f5408008040 comp rx=0 tx=0).stop 2024-04-25T08:56:47.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.157+0000 7f541e4b6640 1 -- 172.21.15.43:0/3775319478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5418150900 msgr2=0x7f5418152180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:47.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.157+0000 7f541e4b6640 1 --2- 172.21.15.43:0/3775319478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5418150900 0x7f5418152180 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f5410009d20 tx=0x7f541000c490 comp rx=0 tx=0).stop 2024-04-25T08:56:47.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.157+0000 7f541e4b6640 1 -- 172.21.15.43:0/3775319478 shutdown_connections 2024-04-25T08:56:47.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.157+0000 7f541e4b6640 1 --2- 172.21.15.43:0/3775319478 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5418150900 0x7f5418152180 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:47.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.157+0000 7f541e4b6640 1 --2- 172.21.15.43:0/3775319478 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5418071f90 0x7f5418144ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:47.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.157+0000 7f541e4b6640 1 --2- 172.21.15.43:0/3775319478 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f540406b500 0x7f540406d9c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:47.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.157+0000 7f541e4b6640 1 --2- 172.21.15.43:0/3775319478 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f541806d7d0 0x7f5418150380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:47.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.157+0000 7f541e4b6640 1 -- 172.21.15.43:0/3775319478 >> 172.21.15.43:0/3775319478 conn(0x7f54180b1480 msgr2=0x7f54180cba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:47.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.157+0000 7f541e4b6640 1 -- 172.21.15.43:0/3775319478 shutdown_connections 2024-04-25T08:56:47.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.157+0000 7f541e4b6640 1 -- 172.21.15.43:0/3775319478 wait complete. 2024-04-25T08:56:47.158 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:55:05.150275+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '32', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:55:05.853905+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 34, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:56:47.158 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:56:47.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.272+0000 7ff52c970640 1 Processor -- start 2024-04-25T08:56:47.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.272+0000 7ff52c970640 1 -- start start 2024-04-25T08:56:47.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.273+0000 7ff52c970640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff52815dc40 0x7ff528162550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:47.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.273+0000 7ff52c970640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff52815e020 0x7ff528071670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:47.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.273+0000 7ff52c970640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5280b47c0 con 0x7ff5280ce7e0 2024-04-25T08:56:47.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.273+0000 7ff52c970640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff528071bb0 con 0x7ff52815e020 2024-04-25T08:56:47.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.273+0000 7ff52c970640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff528071e80 con 0x7ff52815dc40 2024-04-25T08:56:47.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.273+0000 7ff526d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff5280ce7e0 0x7ff5280d0af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39496/0 (socket says 172.21.15.43:39496) 2024-04-25T08:56:47.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.273+0000 7ff526d9d640 1 -- 172.21.15.43:0/3870347142 learned_addr learned my addr 172.21.15.43:0/3870347142 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:56:47.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.274+0000 7ff525d9b640 1 --2- 172.21.15.43:0/3870347142 >> v2:172.21.15.193:3300/0 conn(0x7ff52815e020 0x7ff528071670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:47.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.274+0000 7ff52659c640 1 --2- 172.21.15.43:0/3870347142 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff52815dc40 0x7ff528162550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:47.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.274+0000 7ff52559a640 1 -- 172.21.15.43:0/3870347142 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1677645528 0 0) 0x7ff5280b47c0 con 0x7ff5280ce7e0 2024-04-25T08:56:47.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.274+0000 7ff52559a640 1 -- 172.21.15.43:0/3870347142 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff514003520 con 0x7ff5280ce7e0 2024-04-25T08:56:47.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.274+0000 7ff52559a640 1 -- 172.21.15.43:0/3870347142 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 619266720 0 0) 0x7ff514003520 con 0x7ff5280ce7e0 2024-04-25T08:56:47.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.274+0000 7ff52559a640 1 -- 172.21.15.43:0/3870347142 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff52815dc40 msgr2=0x7ff528162550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:47.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.274+0000 7ff52559a640 1 --2- 172.21.15.43:0/3870347142 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff52815dc40 0x7ff528162550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:47.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.274+0000 7ff52559a640 1 -- 172.21.15.43:0/3870347142 >> v2:172.21.15.193:3300/0 conn(0x7ff52815e020 msgr2=0x7ff528071670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:47.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.274+0000 7ff52559a640 1 --2- 172.21.15.43:0/3870347142 >> v2:172.21.15.193:3300/0 conn(0x7ff52815e020 0x7ff528071670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:47.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.274+0000 7ff52559a640 1 -- 172.21.15.43:0/3870347142 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff528072160 con 0x7ff5280ce7e0 2024-04-25T08:56:47.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.274+0000 7ff52559a640 1 -- 172.21.15.43:0/3870347142 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7ff5180019f0 con 0x7ff5280ce7e0 2024-04-25T08:56:47.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.274+0000 7ff52559a640 1 -- 172.21.15.43:0/3870347142 >> v1:172.21.15.43:6789/0 conn(0x7ff5280ce7e0 legacy=0x7ff5280d0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:47.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.274+0000 7ff52559a640 1 --2- 172.21.15.43:0/3870347142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5140039b0 0x7ff514003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:47.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.274+0000 7ff52559a640 1 --2- 172.21.15.43:0/3870347142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff514004800 0x7ff514004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:47.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.274+0000 7ff52559a640 1 --2- 172.21.15.43:0/3870347142 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff5140051c0 0x7ff51400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:47.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.274+0000 7ff52559a640 1 -- 172.21.15.43:0/3870347142 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff528071bb0 con 0x7ff514004800 2024-04-25T08:56:47.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.274+0000 7ff52559a640 1 -- 172.21.15.43:0/3870347142 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff528071e80 con 0x7ff5140039b0 2024-04-25T08:56:47.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.274+0000 7ff52559a640 1 -- 172.21.15.43:0/3870347142 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff51400bf00 con 0x7ff5140051c0 2024-04-25T08:56:47.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.274+0000 7ff52659c640 1 --2- 172.21.15.43:0/3870347142 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff52815dc40 0x7ff528162550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:56:47.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.274+0000 7ff525d9b640 1 --2- 172.21.15.43:0/3870347142 >> v2:172.21.15.193:3300/0 conn(0x7ff52815e020 0x7ff528071670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:56:47.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.274+0000 7ff525d9b640 1 --2- 172.21.15.43:0/3870347142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff514004800 0x7ff514004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:47.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.274+0000 7ff526d9d640 1 --2- 172.21.15.43:0/3870347142 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff5140051c0 0x7ff51400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:47.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.274+0000 7ff52659c640 1 --2- 172.21.15.43:0/3870347142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5140039b0 0x7ff514003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:47.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.274+0000 7ff525d9b640 1 -- 172.21.15.43:0/3870347142 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff5140051c0 msgr2=0x7ff51400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:47.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.275+0000 7ff525d9b640 1 --2- 172.21.15.43:0/3870347142 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff5140051c0 0x7ff51400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:47.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.275+0000 7ff525d9b640 1 -- 172.21.15.43:0/3870347142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5140039b0 msgr2=0x7ff514003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:47.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.275+0000 7ff525d9b640 1 --2- 172.21.15.43:0/3870347142 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5140039b0 0x7ff514003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:47.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.275+0000 7ff525d9b640 1 -- 172.21.15.43:0/3870347142 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff514003520 con 0x7ff514004800 2024-04-25T08:56:47.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.275+0000 7ff525d9b640 1 --2- 172.21.15.43:0/3870347142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff514004800 0x7ff514004c80 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff510009c90 tx=0x7ff514005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2c39f17a6f8be08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:47.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.275+0000 7ff52559a640 1 -- 172.21.15.43:0/3870347142 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff510036070 con 0x7ff514004800 2024-04-25T08:56:47.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.275+0000 7ff52559a640 1 -- 172.21.15.43:0/3870347142 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff510031400 con 0x7ff514004800 2024-04-25T08:56:47.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.275+0000 7ff52559a640 1 -- 172.21.15.43:0/3870347142 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff510009860 con 0x7ff514004800 2024-04-25T08:56:47.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.276+0000 7ff52c970640 1 -- 172.21.15.43:0/3870347142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff514004800 msgr2=0x7ff514004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:47.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.276+0000 7ff52c970640 1 --2- 172.21.15.43:0/3870347142 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff514004800 0x7ff514004c80 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff510009c90 tx=0x7ff514005830 comp rx=0 tx=0).stop 2024-04-25T08:56:47.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.276+0000 7ff526d9d640 1 -- 172.21.15.43:0/3870347142 reap_dead start 2024-04-25T08:56:47.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.276+0000 7ff52c970640 1 -- 172.21.15.43:0/3870347142 shutdown_connections 2024-04-25T08:56:47.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.276+0000 7ff52c970640 1 -- 172.21.15.43:0/3870347142 >> 172.21.15.43:0/3870347142 conn(0x7ff52806c060 msgr2=0x7ff52806c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:47.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.276+0000 7ff52c970640 1 -- 172.21.15.43:0/3870347142 shutdown_connections 2024-04-25T08:56:47.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.276+0000 7ff52c970640 1 -- 172.21.15.43:0/3870347142 wait complete. 2024-04-25T08:56:47.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.277+0000 7ff52c970640 1 Processor -- start 2024-04-25T08:56:47.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.277+0000 7ff52c970640 1 -- start start 2024-04-25T08:56:47.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.278+0000 7ff52c970640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff514004800 0x7ff528160cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:47.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.278+0000 7ff52c970640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff52815ee20 0x7ff5281f38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:47.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.278+0000 7ff52c970640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5281f3e10 0x7ff52806e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:47.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.278+0000 7ff52c970640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5280cec90 con 0x7ff52815ee20 2024-04-25T08:56:47.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.278+0000 7ff52c970640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5280b4a60 con 0x7ff5281f3e10 2024-04-25T08:56:47.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.278+0000 7ff52c970640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5280d1200 con 0x7ff514004800 2024-04-25T08:56:47.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.278+0000 7ff525d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff52815ee20 0x7ff5281f38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:47.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.278+0000 7ff525d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff52815ee20 0x7ff5281f38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59238/0 (socket says 172.21.15.43:59238) 2024-04-25T08:56:47.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.278+0000 7ff525d9b640 1 -- 172.21.15.43:0/2513140321 learned_addr learned my addr 172.21.15.43:0/2513140321 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:47.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.278+0000 7ff52659c640 1 --2- 172.21.15.43:0/2513140321 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff514004800 0x7ff528160cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:47.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.278+0000 7ff526d9d640 1 --2- 172.21.15.43:0/2513140321 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5281f3e10 0x7ff52806e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:47.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.278+0000 7ff525d9b640 1 -- 172.21.15.43:0/2513140321 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff514004800 msgr2=0x7ff528160cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:47.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.278+0000 7ff525d9b640 1 --2- 172.21.15.43:0/2513140321 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff514004800 0x7ff528160cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:47.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.278+0000 7ff525d9b640 1 -- 172.21.15.43:0/2513140321 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5281f3e10 msgr2=0x7ff52806e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:47.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.278+0000 7ff525d9b640 1 --2- 172.21.15.43:0/2513140321 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5281f3e10 0x7ff52806e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:47.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.278+0000 7ff525d9b640 1 -- 172.21.15.43:0/2513140321 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5281f4370 con 0x7ff52815ee20 2024-04-25T08:56:47.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.278+0000 7ff52659c640 1 --2- 172.21.15.43:0/2513140321 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff514004800 0x7ff528160cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:47.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.279+0000 7ff526d9d640 1 --2- 172.21.15.43:0/2513140321 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5281f3e10 0x7ff52806e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:47.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.279+0000 7ff525d9b640 1 --2- 172.21.15.43:0/2513140321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff52815ee20 0x7ff5281f38d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff510034040 tx=0x7ff510004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:47.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.279+0000 7ff51effd640 1 -- 172.21.15.43:0/2513140321 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff510040020 con 0x7ff52815ee20 2024-04-25T08:56:47.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.279+0000 7ff51effd640 1 -- 172.21.15.43:0/2513140321 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff51003b6a0 con 0x7ff52815ee20 2024-04-25T08:56:47.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.279+0000 7ff52c970640 1 -- 172.21.15.43:0/2513140321 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5281f4650 con 0x7ff52815ee20 2024-04-25T08:56:47.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.279+0000 7ff51effd640 1 -- 172.21.15.43:0/2513140321 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff510036040 con 0x7ff52815ee20 2024-04-25T08:56:47.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.279+0000 7ff52c970640 1 -- 172.21.15.43:0/2513140321 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff52815e900 con 0x7ff52815ee20 2024-04-25T08:56:47.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.280+0000 7ff52c970640 1 -- 172.21.15.43:0/2513140321 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff528074f60 con 0x7ff52815ee20 2024-04-25T08:56:47.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.280+0000 7ff51effd640 1 -- 172.21.15.43:0/2513140321 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff510043020 con 0x7ff52815ee20 2024-04-25T08:56:47.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.281+0000 7ff51effd640 1 --2- 172.21.15.43:0/2513140321 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff50806b500 0x7ff50806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:47.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.281+0000 7ff51effd640 1 -- 172.21.15.43:0/2513140321 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7ff5100aa950 con 0x7ff52815ee20 2024-04-25T08:56:47.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.281+0000 7ff52659c640 1 --2- 172.21.15.43:0/2513140321 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff50806b500 0x7ff50806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:47.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.282+0000 7ff52659c640 1 --2- 172.21.15.43:0/2513140321 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff50806b500 0x7ff50806d9c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff52806bbf0 tx=0x7ff520008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:47.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.285+0000 7ff51effd640 1 -- 172.21.15.43:0/2513140321 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff51004a030 con 0x7ff52815ee20 2024-04-25T08:56:47.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.463+0000 7ff52c970640 1 -- 172.21.15.43:0/2513140321 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff5280ce0a0 con 0x7ff52815ee20 2024-04-25T08:56:47.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.463+0000 7ff51effd640 1 -- 172.21.15.43:0/2513140321 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff510079720 con 0x7ff52815ee20 2024-04-25T08:56:47.453 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T08:56:47.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.465+0000 7ff52c970640 1 -- 172.21.15.43:0/2513140321 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff50806b500 msgr2=0x7ff50806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:47.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.465+0000 7ff52c970640 1 --2- 172.21.15.43:0/2513140321 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff50806b500 0x7ff50806d9c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff52806bbf0 tx=0x7ff520008040 comp rx=0 tx=0).stop 2024-04-25T08:56:47.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.465+0000 7ff52c970640 1 -- 172.21.15.43:0/2513140321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff52815ee20 msgr2=0x7ff5281f38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:47.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.465+0000 7ff52c970640 1 --2- 172.21.15.43:0/2513140321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff52815ee20 0x7ff5281f38d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff510034040 tx=0x7ff510004730 comp rx=0 tx=0).stop 2024-04-25T08:56:47.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.465+0000 7ff52c970640 1 -- 172.21.15.43:0/2513140321 shutdown_connections 2024-04-25T08:56:47.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.465+0000 7ff52c970640 1 --2- 172.21.15.43:0/2513140321 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff50806b500 0x7ff50806d9c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:47.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.465+0000 7ff52c970640 1 --2- 172.21.15.43:0/2513140321 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff5281f3e10 0x7ff52806e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:47.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.465+0000 7ff52c970640 1 --2- 172.21.15.43:0/2513140321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff52815ee20 0x7ff5281f38d0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:47.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.465+0000 7ff52c970640 1 --2- 172.21.15.43:0/2513140321 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff514004800 0x7ff528160cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:47.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.465+0000 7ff52c970640 1 -- 172.21.15.43:0/2513140321 >> 172.21.15.43:0/2513140321 conn(0x7ff52806c060 msgr2=0x7ff5280ad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:47.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.465+0000 7ff52c970640 1 -- 172.21.15.43:0/2513140321 shutdown_connections 2024-04-25T08:56:47.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.465+0000 7ff52c970640 1 -- 172.21.15.43:0/2513140321 wait complete. 2024-04-25T08:56:47.466 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:56:47.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.580+0000 7f9fb5d16640 1 Processor -- start 2024-04-25T08:56:47.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.580+0000 7f9fb5d16640 1 -- start start 2024-04-25T08:56:47.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.581+0000 7f9fb5d16640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9fb00b6110 0x7f9fb00d1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:47.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.581+0000 7f9fb5d16640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9fb00d2340 0x7f9fb01656d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.581+0000 7f9fb5d16640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fb00b32a0 con 0x7f9fb00d1a20 2024-04-25T08:56:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.581+0000 7f9fb5d16640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fb00cf430 con 0x7f9fb00d2340 2024-04-25T08:56:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.581+0000 7f9fb5d16640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb015bf90 con 0x7f9fb00b6110 2024-04-25T08:56:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.581+0000 7f9faf7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f9fb00d1a20 0x7f9fb00b2b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39498/0 (socket says 172.21.15.43:39498) 2024-04-25T08:56:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.581+0000 7f9faf7fe640 1 -- 172.21.15.43:0/2023328225 learned_addr learned my addr 172.21.15.43:0/2023328225 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:56:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.582+0000 7f9fae7fc640 1 -- 172.21.15.43:0/2023328225 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2584193117 0 0) 0x7f9fb00b32a0 con 0x7f9fb00d1a20 2024-04-25T08:56:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.582+0000 7f9fae7fc640 1 -- 172.21.15.43:0/2023328225 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f98003520 con 0x7f9fb00d1a20 2024-04-25T08:56:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.582+0000 7f9faeffd640 1 --2- 172.21.15.43:0/2023328225 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9fb00b6110 0x7f9fb00d1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.582+0000 7f9faffff640 1 --2- 172.21.15.43:0/2023328225 >> v2:172.21.15.193:3300/0 conn(0x7f9fb00d2340 0x7f9fb01656d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.582+0000 7f9fae7fc640 1 -- 172.21.15.43:0/2023328225 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 958154559 0 0) 0x7f9f98003520 con 0x7f9fb00d1a20 2024-04-25T08:56:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.582+0000 7f9fae7fc640 1 -- 172.21.15.43:0/2023328225 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9fb00b6110 msgr2=0x7f9fb00d1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.582+0000 7f9fae7fc640 1 --2- 172.21.15.43:0/2023328225 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9fb00b6110 0x7f9fb00d1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.582+0000 7f9fae7fc640 1 -- 172.21.15.43:0/2023328225 >> v2:172.21.15.193:3300/0 conn(0x7f9fb00d2340 msgr2=0x7f9fb01656d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:47.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.582+0000 7f9fae7fc640 1 --2- 172.21.15.43:0/2023328225 >> v2:172.21.15.193:3300/0 conn(0x7f9fb00d2340 0x7f9fb01656d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:47.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.582+0000 7f9fae7fc640 1 -- 172.21.15.43:0/2023328225 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb00cee70 con 0x7f9fb00d1a20 2024-04-25T08:56:47.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.582+0000 7f9fae7fc640 1 -- 172.21.15.43:0/2023328225 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f9fa0001690 con 0x7f9fb00d1a20 2024-04-25T08:56:47.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.582+0000 7f9fae7fc640 1 -- 172.21.15.43:0/2023328225 >> v1:172.21.15.43:6789/0 conn(0x7f9fb00d1a20 legacy=0x7f9fb00b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:47.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.582+0000 7f9fae7fc640 1 --2- 172.21.15.43:0/2023328225 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9fb00b6110 0x7f9f980039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:47.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.582+0000 7f9fae7fc640 1 --2- 172.21.15.43:0/2023328225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f98003f30 0x7f9f98004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:47.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.582+0000 7f9fae7fc640 1 --2- 172.21.15.43:0/2023328225 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f98004d40 0x7f9f980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:47.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.582+0000 7f9fae7fc640 1 -- 172.21.15.43:0/2023328225 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb00cf430 con 0x7f9f98003f30 2024-04-25T08:56:47.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.582+0000 7f9fae7fc640 1 -- 172.21.15.43:0/2023328225 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb015bf90 con 0x7f9f98004d40 2024-04-25T08:56:47.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.582+0000 7f9fae7fc640 1 -- 172.21.15.43:0/2023328225 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9800baa0 con 0x7f9fb00b6110 2024-04-25T08:56:47.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.582+0000 7f9faeffd640 1 --2- 172.21.15.43:0/2023328225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f98003f30 0x7f9f98004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:47.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.582+0000 7f9faf7fe640 1 --2- 172.21.15.43:0/2023328225 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9fb00b6110 0x7f9f980039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:47.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.582+0000 7f9faffff640 1 --2- 172.21.15.43:0/2023328225 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f98004d40 0x7f9f980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:47.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.582+0000 7f9faeffd640 1 -- 172.21.15.43:0/2023328225 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9fb00b6110 msgr2=0x7f9f980039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:47.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.582+0000 7f9faeffd640 1 --2- 172.21.15.43:0/2023328225 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9fb00b6110 0x7f9f980039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:47.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.582+0000 7f9faeffd640 1 -- 172.21.15.43:0/2023328225 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f98004d40 msgr2=0x7f9f980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:47.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.582+0000 7f9faeffd640 1 --2- 172.21.15.43:0/2023328225 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9f98004d40 0x7f9f980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:47.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.582+0000 7f9faeffd640 1 -- 172.21.15.43:0/2023328225 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f98003520 con 0x7f9f98003f30 2024-04-25T08:56:47.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.583+0000 7f9faeffd640 1 --2- 172.21.15.43:0/2023328225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f98003f30 0x7f9f98004800 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4009ac0 tx=0x7f9f98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1791034604ca1f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:47.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.583+0000 7f9fae7fc640 1 -- 172.21.15.43:0/2023328225 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa4036070 con 0x7f9f98003f30 2024-04-25T08:56:47.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.583+0000 7f9fae7fc640 1 -- 172.21.15.43:0/2023328225 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa4031400 con 0x7f9f98003f30 2024-04-25T08:56:47.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.583+0000 7f9fae7fc640 1 -- 172.21.15.43:0/2023328225 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa4009690 con 0x7f9f98003f30 2024-04-25T08:56:47.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.584+0000 7f9fb5d16640 1 -- 172.21.15.43:0/2023328225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f98003f30 msgr2=0x7f9f98004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:47.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.584+0000 7f9fb5d16640 1 --2- 172.21.15.43:0/2023328225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9f98003f30 0x7f9f98004800 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4009ac0 tx=0x7f9f98005830 comp rx=0 tx=0).stop 2024-04-25T08:56:47.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.584+0000 7f9faffff640 1 -- 172.21.15.43:0/2023328225 reap_dead start 2024-04-25T08:56:47.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.584+0000 7f9fb5d16640 1 -- 172.21.15.43:0/2023328225 shutdown_connections 2024-04-25T08:56:47.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.584+0000 7f9fb5d16640 1 -- 172.21.15.43:0/2023328225 >> 172.21.15.43:0/2023328225 conn(0x7f9fb0072aa0 msgr2=0x7f9fb009aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:47.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.584+0000 7f9fb5d16640 1 -- 172.21.15.43:0/2023328225 shutdown_connections 2024-04-25T08:56:47.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.584+0000 7f9fb5d16640 1 -- 172.21.15.43:0/2023328225 wait complete. 2024-04-25T08:56:47.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.585+0000 7f9fb5d16640 1 Processor -- start 2024-04-25T08:56:47.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.585+0000 7f9fb5d16640 1 -- start start 2024-04-25T08:56:47.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.586+0000 7f9fb5d16640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9fb01503e0 0x7f9fb01f5db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:47.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.586+0000 7f9fb5d16640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9fb009c2d0 0x7f9fb014cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:47.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.586+0000 7f9fb5d16640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fb01f2aa0 0x7f9fb014d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:47.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.586+0000 7f9fb5d16640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb00b4be0 con 0x7f9fb01f2aa0 2024-04-25T08:56:47.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.586+0000 7f9fb5d16640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb00b4900 con 0x7f9fb009c2d0 2024-04-25T08:56:47.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.586+0000 7f9fb5d16640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb00cea60 con 0x7f9fb01503e0 2024-04-25T08:56:47.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.586+0000 7f9faffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fb01f2aa0 0x7f9fb014d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:47.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.586+0000 7f9faffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fb01f2aa0 0x7f9fb014d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59254/0 (socket says 172.21.15.43:59254) 2024-04-25T08:56:47.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.586+0000 7f9faffff640 1 -- 172.21.15.43:0/3868426509 learned_addr learned my addr 172.21.15.43:0/3868426509 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:47.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.586+0000 7f9faf7fe640 1 --2- 172.21.15.43:0/3868426509 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9fb01503e0 0x7f9fb01f5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:47.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.586+0000 7f9faeffd640 1 --2- 172.21.15.43:0/3868426509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9fb009c2d0 0x7f9fb014cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:47.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.586+0000 7f9faffff640 1 -- 172.21.15.43:0/3868426509 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9fb01503e0 msgr2=0x7f9fb01f5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:47.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.586+0000 7f9faffff640 1 --2- 172.21.15.43:0/3868426509 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9fb01503e0 0x7f9fb01f5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:47.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.586+0000 7f9faffff640 1 -- 172.21.15.43:0/3868426509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9fb009c2d0 msgr2=0x7f9fb014cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:47.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.586+0000 7f9faffff640 1 --2- 172.21.15.43:0/3868426509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9fb009c2d0 0x7f9fb014cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:47.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.586+0000 7f9faffff640 1 -- 172.21.15.43:0/3868426509 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb009ca70 con 0x7f9fb01f2aa0 2024-04-25T08:56:47.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.587+0000 7f9faffff640 1 --2- 172.21.15.43:0/3868426509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fb01f2aa0 0x7f9fb014d170 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9fb00d0830 tx=0x7f9fa80115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:47.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.587+0000 7f9f97fff640 1 -- 172.21.15.43:0/3868426509 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8011e10 con 0x7f9fb01f2aa0 2024-04-25T08:56:47.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.587+0000 7f9f97fff640 1 -- 172.21.15.43:0/3868426509 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa801b070 con 0x7f9fb01f2aa0 2024-04-25T08:56:47.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.587+0000 7f9f97fff640 1 -- 172.21.15.43:0/3868426509 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8003990 con 0x7f9fb01f2aa0 2024-04-25T08:56:47.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.587+0000 7f9fb5d16640 1 -- 172.21.15.43:0/3868426509 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fb01f62f0 con 0x7f9fb01f2aa0 2024-04-25T08:56:47.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.587+0000 7f9fb5d16640 1 -- 172.21.15.43:0/3868426509 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fb01f1fc0 con 0x7f9fb01f2aa0 2024-04-25T08:56:47.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.588+0000 7f9fb5d16640 1 -- 172.21.15.43:0/3868426509 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fb0074f60 con 0x7f9fb01f2aa0 2024-04-25T08:56:47.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.588+0000 7f9f97fff640 1 -- 172.21.15.43:0/3868426509 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9fa8005d10 con 0x7f9fb01f2aa0 2024-04-25T08:56:47.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.589+0000 7f9f97fff640 1 --2- 172.21.15.43:0/3868426509 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9f9c06b4d0 0x7f9f9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:47.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.589+0000 7f9f97fff640 1 -- 172.21.15.43:0/3868426509 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f9fa808a830 con 0x7f9fb01f2aa0 2024-04-25T08:56:47.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.589+0000 7f9faf7fe640 1 --2- 172.21.15.43:0/3868426509 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9f9c06b4d0 0x7f9f9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:47.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.589+0000 7f9faf7fe640 1 --2- 172.21.15.43:0/3868426509 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9f9c06b4d0 0x7f9f9c06d990 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0001120 tx=0x7f9fa0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:47.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.593+0000 7f9f97fff640 1 -- 172.21.15.43:0/3868426509 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9fa8025020 con 0x7f9fb01f2aa0 2024-04-25T08:56:47.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.772+0000 7f9fb5d16640 1 -- 172.21.15.43:0/3868426509 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9fb009c0b0 con 0x7f9fb01f2aa0 2024-04-25T08:56:47.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.772+0000 7f9f97fff640 1 -- 172.21.15.43:0/3868426509 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v35) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9fb01f62f0 con 0x7f9fb01f2aa0 2024-04-25T08:56:47.762 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T08:56:47.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.774+0000 7f9fb5d16640 1 -- 172.21.15.43:0/3868426509 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9f9c06b4d0 msgr2=0x7f9f9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:47.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.774+0000 7f9fb5d16640 1 --2- 172.21.15.43:0/3868426509 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9f9c06b4d0 0x7f9f9c06d990 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0001120 tx=0x7f9fa0009040 comp rx=0 tx=0).stop 2024-04-25T08:56:47.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.774+0000 7f9fb5d16640 1 -- 172.21.15.43:0/3868426509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fb01f2aa0 msgr2=0x7f9fb014d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:47.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.774+0000 7f9fb5d16640 1 --2- 172.21.15.43:0/3868426509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fb01f2aa0 0x7f9fb014d170 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9fb00d0830 tx=0x7f9fa80115a0 comp rx=0 tx=0).stop 2024-04-25T08:56:47.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.774+0000 7f9fb5d16640 1 -- 172.21.15.43:0/3868426509 shutdown_connections 2024-04-25T08:56:47.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.774+0000 7f9fb5d16640 1 --2- 172.21.15.43:0/3868426509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fb01f2aa0 0x7f9fb014d170 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:47.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.774+0000 7f9fb5d16640 1 --2- 172.21.15.43:0/3868426509 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9f9c06b4d0 0x7f9f9c06d990 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:47.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.774+0000 7f9fb5d16640 1 --2- 172.21.15.43:0/3868426509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9fb009c2d0 0x7f9fb014cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:47.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.774+0000 7f9fb5d16640 1 --2- 172.21.15.43:0/3868426509 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9fb01503e0 0x7f9fb01f5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:47.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.774+0000 7f9fb5d16640 1 -- 172.21.15.43:0/3868426509 >> 172.21.15.43:0/3868426509 conn(0x7f9fb0072aa0 msgr2=0x7f9fb006e600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:47.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.774+0000 7f9fb5d16640 1 -- 172.21.15.43:0/3868426509 shutdown_connections 2024-04-25T08:56:47.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.774+0000 7f9fb5d16640 1 -- 172.21.15.43:0/3868426509 wait complete. 2024-04-25T08:56:47.775 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:56:47.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.890+0000 7fa4dd4fc640 1 Processor -- start 2024-04-25T08:56:47.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.890+0000 7fa4dd4fc640 1 -- start start 2024-04-25T08:56:47.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.890+0000 7fa4dd4fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa4d80cda20 0x7fa4d81621d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:47.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.890+0000 7fa4dd4fc640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4d815bdd0 con 0x7fa4d815b9f0 2024-04-25T08:56:47.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.890+0000 7fa4dd4fc640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4d815ec80 con 0x7fa4d8162710 2024-04-25T08:56:47.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.890+0000 7fa4dd4fc640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4d815ef60 con 0x7fa4d80cda20 2024-04-25T08:56:47.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.891+0000 7fa4d67fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa4d80cda20 0x7fa4d81621d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:47.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.891+0000 7fa4d6ffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa4d815b9f0 0x7fa4d815e360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39512/0 (socket says 172.21.15.43:39512) 2024-04-25T08:56:47.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.891+0000 7fa4d6ffd640 1 -- 172.21.15.43:0/3761374437 learned_addr learned my addr 172.21.15.43:0/3761374437 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:56:47.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.891+0000 7fa4d67fc640 1 -- 172.21.15.43:0/3761374437 >> v1:172.21.15.193:6789/0 conn(0x7fa4d8162710 legacy=0x7fa4d80cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:47.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.891+0000 7fa4d67fc640 1 -- 172.21.15.43:0/3761374437 >> v1:172.21.15.43:6789/0 conn(0x7fa4d815b9f0 legacy=0x7fa4d815e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:47.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.891+0000 7fa4d5ffb640 1 -- 172.21.15.43:0/3761374437 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 245528860 0 0) 0x7fa4d815bdd0 con 0x7fa4d815b9f0 2024-04-25T08:56:47.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.891+0000 7fa4d67fc640 1 -- 172.21.15.43:0/3761374437 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d815f240 con 0x7fa4d80cda20 2024-04-25T08:56:47.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.892+0000 7fa4d67fc640 1 --2- 172.21.15.43:0/3761374437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4d80cda20 0x7fa4d81621d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc0097a0 tx=0x7fa4cc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e76a14402f8cbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:47.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.892+0000 7fa4d5ffb640 1 -- 172.21.15.43:0/3761374437 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc037070 con 0x7fa4d80cda20 2024-04-25T08:56:47.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.892+0000 7fa4d5ffb640 1 -- 172.21.15.43:0/3761374437 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4cc02ae50 con 0x7fa4d80cda20 2024-04-25T08:56:47.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.892+0000 7fa4d5ffb640 1 -- 172.21.15.43:0/3761374437 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc032660 con 0x7fa4d80cda20 2024-04-25T08:56:47.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.892+0000 7fa4dd4fc640 1 -- 172.21.15.43:0/3761374437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4d80cda20 msgr2=0x7fa4d81621d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:47.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.892+0000 7fa4dd4fc640 1 --2- 172.21.15.43:0/3761374437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4d80cda20 0x7fa4d81621d0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc0097a0 tx=0x7fa4cc02a140 comp rx=0 tx=0).stop 2024-04-25T08:56:47.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.892+0000 7fa4dd4fc640 1 -- 172.21.15.43:0/3761374437 shutdown_connections 2024-04-25T08:56:47.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.892+0000 7fa4dd4fc640 1 --2- 172.21.15.43:0/3761374437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4d80cda20 0x7fa4d81621d0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:47.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.892+0000 7fa4dd4fc640 1 -- 172.21.15.43:0/3761374437 >> 172.21.15.43:0/3761374437 conn(0x7fa4d806ab30 msgr2=0x7fa4d806af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:47.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.892+0000 7fa4dd4fc640 1 -- 172.21.15.43:0/3761374437 shutdown_connections 2024-04-25T08:56:47.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.892+0000 7fa4dd4fc640 1 -- 172.21.15.43:0/3761374437 wait complete. 2024-04-25T08:56:47.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.894+0000 7fa4dd4fc640 1 Processor -- start 2024-04-25T08:56:47.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.894+0000 7fa4dd4fc640 1 -- start start 2024-04-25T08:56:47.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.894+0000 7fa4dd4fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4d80cda20 0x7fa4d80b2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:47.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.894+0000 7fa4dd4fc640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa4d815b9f0 0x7fa4d815c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:47.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.894+0000 7fa4dd4fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4d8162710 0x7fa4d815ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:47.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.894+0000 7fa4dd4fc640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d80d0310 con 0x7fa4d8162710 2024-04-25T08:56:47.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.894+0000 7fa4dd4fc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d80d0190 con 0x7fa4d80cda20 2024-04-25T08:56:47.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.894+0000 7fa4dd4fc640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d806d450 con 0x7fa4d815b9f0 2024-04-25T08:56:47.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.894+0000 7fa4d77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4d8162710 0x7fa4d815ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:47.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.895+0000 7fa4d77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4d8162710 0x7fa4d815ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59280/0 (socket says 172.21.15.43:59280) 2024-04-25T08:56:47.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.895+0000 7fa4d77fe640 1 -- 172.21.15.43:0/3982156389 learned_addr learned my addr 172.21.15.43:0/3982156389 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:47.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.895+0000 7fa4d6ffd640 1 --2- 172.21.15.43:0/3982156389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4d80cda20 0x7fa4d80b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:47.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.895+0000 7fa4d67fc640 1 --2- 172.21.15.43:0/3982156389 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa4d815b9f0 0x7fa4d815c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:47.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.895+0000 7fa4d77fe640 1 -- 172.21.15.43:0/3982156389 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa4d815b9f0 msgr2=0x7fa4d815c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:47.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.895+0000 7fa4d77fe640 1 --2- 172.21.15.43:0/3982156389 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa4d815b9f0 0x7fa4d815c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:47.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.895+0000 7fa4d77fe640 1 -- 172.21.15.43:0/3982156389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4d80cda20 msgr2=0x7fa4d80b2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:47.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.895+0000 7fa4d77fe640 1 --2- 172.21.15.43:0/3982156389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4d80cda20 0x7fa4d80b2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:47.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.895+0000 7fa4d77fe640 1 -- 172.21.15.43:0/3982156389 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d81f7010 con 0x7fa4d8162710 2024-04-25T08:56:47.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.895+0000 7fa4d77fe640 1 --2- 172.21.15.43:0/3982156389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4d8162710 0x7fa4d815ce60 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa4d000b9a0 tx=0x7fa4d000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:47.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.895+0000 7fa4bf7fe640 1 -- 172.21.15.43:0/3982156389 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d0019070 con 0x7fa4d8162710 2024-04-25T08:56:47.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.895+0000 7fa4bf7fe640 1 -- 172.21.15.43:0/3982156389 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d000cba0 con 0x7fa4d8162710 2024-04-25T08:56:47.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.895+0000 7fa4dd4fc640 1 -- 172.21.15.43:0/3982156389 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4d81f72b0 con 0x7fa4d8162710 2024-04-25T08:56:47.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.895+0000 7fa4bf7fe640 1 -- 172.21.15.43:0/3982156389 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d0014440 con 0x7fa4d8162710 2024-04-25T08:56:47.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.895+0000 7fa4dd4fc640 1 -- 172.21.15.43:0/3982156389 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4d81f7870 con 0x7fa4d8162710 2024-04-25T08:56:47.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.896+0000 7fa4dd4fc640 1 -- 172.21.15.43:0/3982156389 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4d80add70 con 0x7fa4d8162710 2024-04-25T08:56:47.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.897+0000 7fa4bf7fe640 1 -- 172.21.15.43:0/3982156389 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa4d001c070 con 0x7fa4d8162710 2024-04-25T08:56:47.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.897+0000 7fa4bf7fe640 1 --2- 172.21.15.43:0/3982156389 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa4c406b4d0 0x7fa4c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:47.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.897+0000 7fa4bf7fe640 1 -- 172.21.15.43:0/3982156389 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fa4d0089d20 con 0x7fa4d8162710 2024-04-25T08:56:47.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.897+0000 7fa4d6ffd640 1 --2- 172.21.15.43:0/3982156389 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa4c406b4d0 0x7fa4c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:47.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.898+0000 7fa4d6ffd640 1 --2- 172.21.15.43:0/3982156389 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa4c406b4d0 0x7fa4c406d990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8000f60 tx=0x7fa4c8007d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:47.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:47.901+0000 7fa4bf7fe640 1 -- 172.21.15.43:0/3982156389 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa4d00585c0 con 0x7fa4d8162710 2024-04-25T08:56:48.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.081+0000 7fa4dd4fc640 1 -- 172.21.15.43:0/3982156389 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa4d806c790 con 0x7fa4d8162710 2024-04-25T08:56:48.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.082+0000 7fa4bf7fe640 1 -- 172.21.15.43:0/3982156389 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v35) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa4d005c660 con 0x7fa4d8162710 2024-04-25T08:56:48.072 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T08:56:48.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.084+0000 7fa4dd4fc640 1 -- 172.21.15.43:0/3982156389 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa4c406b4d0 msgr2=0x7fa4c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.084+0000 7fa4dd4fc640 1 --2- 172.21.15.43:0/3982156389 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa4c406b4d0 0x7fa4c406d990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8000f60 tx=0x7fa4c8007d50 comp rx=0 tx=0).stop 2024-04-25T08:56:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.084+0000 7fa4dd4fc640 1 -- 172.21.15.43:0/3982156389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4d8162710 msgr2=0x7fa4d815ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.084+0000 7fa4dd4fc640 1 --2- 172.21.15.43:0/3982156389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4d8162710 0x7fa4d815ce60 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa4d000b9a0 tx=0x7fa4d000be60 comp rx=0 tx=0).stop 2024-04-25T08:56:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.084+0000 7fa4dd4fc640 1 -- 172.21.15.43:0/3982156389 shutdown_connections 2024-04-25T08:56:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.084+0000 7fa4dd4fc640 1 --2- 172.21.15.43:0/3982156389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4d8162710 0x7fa4d815ce60 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.084+0000 7fa4dd4fc640 1 --2- 172.21.15.43:0/3982156389 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa4d815b9f0 0x7fa4d815c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.084+0000 7fa4dd4fc640 1 --2- 172.21.15.43:0/3982156389 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa4c406b4d0 0x7fa4c406d990 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.084+0000 7fa4dd4fc640 1 --2- 172.21.15.43:0/3982156389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4d80cda20 0x7fa4d80b2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.084+0000 7fa4dd4fc640 1 -- 172.21.15.43:0/3982156389 >> 172.21.15.43:0/3982156389 conn(0x7fa4d806ab30 msgr2=0x7fa4d806af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.084+0000 7fa4dd4fc640 1 -- 172.21.15.43:0/3982156389 shutdown_connections 2024-04-25T08:56:48.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.084+0000 7fa4dd4fc640 1 -- 172.21.15.43:0/3982156389 wait complete. 2024-04-25T08:56:48.084 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:56:48.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.195+0000 7f556858b640 1 Processor -- start 2024-04-25T08:56:48.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.195+0000 7f556858b640 1 -- start start 2024-04-25T08:56:48.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.196+0000 7f556858b640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f55600d09b0 0x7f55600b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:48.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.196+0000 7f556858b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55600b4c30 con 0x7f55600ce5e0 2024-04-25T08:56:48.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.196+0000 7f556858b640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55600cf2c0 con 0x7f5560161390 2024-04-25T08:56:48.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.196+0000 7f556858b640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55600d1910 con 0x7f55600d09b0 2024-04-25T08:56:48.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.197+0000 7f5566b28640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f55600ce5e0 0x7f55600cea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39522/0 (socket says 172.21.15.43:39522) 2024-04-25T08:56:48.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.197+0000 7f5566b28640 1 -- 172.21.15.43:0/1171915509 learned_addr learned my addr 172.21.15.43:0/1171915509 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:56:48.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.197+0000 7f5565b26640 1 --2- 172.21.15.43:0/1171915509 >> v2:172.21.15.193:3300/0 conn(0x7f55600d09b0 0x7f55600b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:48.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.197+0000 7f5565325640 1 -- 172.21.15.43:0/1171915509 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1650719048 0 0) 0x7f5558001490 con 0x7f55600ce5e0 2024-04-25T08:56:48.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.197+0000 7f5565325640 1 -- 172.21.15.43:0/1171915509 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f554c003520 con 0x7f55600ce5e0 2024-04-25T08:56:48.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.197+0000 7f5565325640 1 -- 172.21.15.43:0/1171915509 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3424545501 0 0) 0x7f55600cf2c0 con 0x7f5560161390 2024-04-25T08:56:48.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.197+0000 7f5565325640 1 -- 172.21.15.43:0/1171915509 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5558001490 con 0x7f5560161390 2024-04-25T08:56:48.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.197+0000 7f5565325640 1 -- 172.21.15.43:0/1171915509 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2758151990 0 0) 0x7f554c003520 con 0x7f55600ce5e0 2024-04-25T08:56:48.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.197+0000 7f5565325640 1 -- 172.21.15.43:0/1171915509 >> v2:172.21.15.193:3300/0 conn(0x7f55600d09b0 msgr2=0x7f55600b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:48.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.197+0000 7f5565325640 1 --2- 172.21.15.43:0/1171915509 >> v2:172.21.15.193:3300/0 conn(0x7f55600d09b0 0x7f55600b6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.197+0000 7f5565325640 1 -- 172.21.15.43:0/1171915509 >> v1:172.21.15.193:6789/0 conn(0x7f5560161390 legacy=0x7f55600d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:48.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.197+0000 7f5565325640 1 -- 172.21.15.43:0/1171915509 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55600cf5a0 con 0x7f55600ce5e0 2024-04-25T08:56:48.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.197+0000 7f5565325640 1 -- 172.21.15.43:0/1171915509 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f5558001bc0 con 0x7f55600ce5e0 2024-04-25T08:56:48.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.197+0000 7f5565b26640 1 --2- 172.21.15.43:0/1171915509 >> v2:172.21.15.193:3300/0 conn(0x7f55600d09b0 0x7f55600b6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:56:48.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.197+0000 7f5565325640 1 -- 172.21.15.43:0/1171915509 >> v1:172.21.15.43:6789/0 conn(0x7f55600ce5e0 legacy=0x7f55600cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:48.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.197+0000 7f5565325640 1 --2- 172.21.15.43:0/1171915509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f554c003b40 0x7f554c003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:48.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.197+0000 7f5565325640 1 --2- 172.21.15.43:0/1171915509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f554c004800 0x7f554c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:48.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.197+0000 7f5565325640 1 --2- 172.21.15.43:0/1171915509 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f554c0051c0 0x7f554c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:48.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.197+0000 7f5565325640 1 -- 172.21.15.43:0/1171915509 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55600d1910 con 0x7f554c004800 2024-04-25T08:56:48.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.197+0000 7f5565325640 1 -- 172.21.15.43:0/1171915509 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554c00c040 con 0x7f554c003b40 2024-04-25T08:56:48.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.197+0000 7f5565325640 1 -- 172.21.15.43:0/1171915509 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f554c00c240 con 0x7f554c0051c0 2024-04-25T08:56:48.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.197+0000 7f5565b26640 1 --2- 172.21.15.43:0/1171915509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f554c004800 0x7f554c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:48.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.197+0000 7f5566327640 1 --2- 172.21.15.43:0/1171915509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f554c003b40 0x7f554c003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:48.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.198+0000 7f5566b28640 1 --2- 172.21.15.43:0/1171915509 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f554c0051c0 0x7f554c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:48.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.198+0000 7f5565b26640 1 -- 172.21.15.43:0/1171915509 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f554c0051c0 msgr2=0x7f554c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:48.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.198+0000 7f5565b26640 1 --2- 172.21.15.43:0/1171915509 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f554c0051c0 0x7f554c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.198+0000 7f5565b26640 1 -- 172.21.15.43:0/1171915509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f554c003b40 msgr2=0x7f554c003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:48.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.198+0000 7f5565b26640 1 --2- 172.21.15.43:0/1171915509 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f554c003b40 0x7f554c003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.198+0000 7f5565b26640 1 -- 172.21.15.43:0/1171915509 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f554c003520 con 0x7f554c004800 2024-04-25T08:56:48.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.198+0000 7f5566b28640 1 -- 172.21.15.43:0/1171915509 reap_dead start 2024-04-25T08:56:48.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.198+0000 7f5565b26640 1 --2- 172.21.15.43:0/1171915509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f554c004800 0x7f554c004c80 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5550009c90 tx=0x7f554c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a3c400aca3e8b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:48.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.198+0000 7f5565325640 1 -- 172.21.15.43:0/1171915509 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5550036070 con 0x7f554c004800 2024-04-25T08:56:48.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.199+0000 7f5565325640 1 -- 172.21.15.43:0/1171915509 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5550031400 con 0x7f554c004800 2024-04-25T08:56:48.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.199+0000 7f5565325640 1 -- 172.21.15.43:0/1171915509 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5550009860 con 0x7f554c004800 2024-04-25T08:56:48.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.199+0000 7f556858b640 1 -- 172.21.15.43:0/1171915509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f554c004800 msgr2=0x7f554c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:48.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.199+0000 7f556858b640 1 --2- 172.21.15.43:0/1171915509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f554c004800 0x7f554c004c80 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f5550009c90 tx=0x7f554c005830 comp rx=0 tx=0).stop 2024-04-25T08:56:48.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.199+0000 7f556858b640 1 -- 172.21.15.43:0/1171915509 shutdown_connections 2024-04-25T08:56:48.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.199+0000 7f556858b640 1 --2- 172.21.15.43:0/1171915509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f554c004800 0x7f554c004c80 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.199+0000 7f556858b640 1 -- 172.21.15.43:0/1171915509 >> 172.21.15.43:0/1171915509 conn(0x7f556009cd20 msgr2=0x7f55600d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:48.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.199+0000 7f556858b640 1 -- 172.21.15.43:0/1171915509 shutdown_connections 2024-04-25T08:56:48.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.199+0000 7f556858b640 1 -- 172.21.15.43:0/1171915509 wait complete. 2024-04-25T08:56:48.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.200+0000 7f556858b640 1 Processor -- start 2024-04-25T08:56:48.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.200+0000 7f556858b640 1 -- start start 2024-04-25T08:56:48.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.201+0000 7f556858b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f554c004800 0x7f55600cc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:48.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.201+0000 7f556858b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f55600cafc0 0x7f55601f7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:48.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.201+0000 7f556858b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5560155f70 0x7f55600b2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:48.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.201+0000 7f556858b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5560161770 con 0x7f5560155f70 2024-04-25T08:56:48.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.201+0000 7f556858b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55600d15a0 con 0x7f554c004800 2024-04-25T08:56:48.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.201+0000 7f556858b640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55600b6820 con 0x7f55600cafc0 2024-04-25T08:56:48.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.201+0000 7f5566b28640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5560155f70 0x7f55600b2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:48.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.201+0000 7f5566b28640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5560155f70 0x7f55600b2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59298/0 (socket says 172.21.15.43:59298) 2024-04-25T08:56:48.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.201+0000 7f5566b28640 1 -- 172.21.15.43:0/3069229863 learned_addr learned my addr 172.21.15.43:0/3069229863 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:48.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.201+0000 7f5565b26640 1 --2- 172.21.15.43:0/3069229863 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f55600cafc0 0x7f55601f7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:48.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.201+0000 7f5566b28640 1 -- 172.21.15.43:0/3069229863 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f55600cafc0 msgr2=0x7f55601f7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:48.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.201+0000 7f5566327640 1 --2- 172.21.15.43:0/3069229863 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f554c004800 0x7f55600cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:48.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.202+0000 7f5566b28640 1 --2- 172.21.15.43:0/3069229863 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f55600cafc0 0x7f55601f7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.202+0000 7f5566b28640 1 -- 172.21.15.43:0/3069229863 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f554c004800 msgr2=0x7f55600cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:48.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.202+0000 7f5566b28640 1 --2- 172.21.15.43:0/3069229863 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f554c004800 0x7f55600cc620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.202+0000 7f5566b28640 1 -- 172.21.15.43:0/3069229863 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55601f81a0 con 0x7f5560155f70 2024-04-25T08:56:48.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.202+0000 7f5566b28640 1 --2- 172.21.15.43:0/3069229863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5560155f70 0x7f55600b2de0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f55600cfb60 tx=0x7f555800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:48.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.202+0000 7f5556ffd640 1 -- 172.21.15.43:0/3069229863 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f555800ee20 con 0x7f5560155f70 2024-04-25T08:56:48.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.202+0000 7f5556ffd640 1 -- 172.21.15.43:0/3069229863 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5558018070 con 0x7f5560155f70 2024-04-25T08:56:48.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.202+0000 7f556858b640 1 -- 172.21.15.43:0/3069229863 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55600cb580 con 0x7f5560155f70 2024-04-25T08:56:48.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.202+0000 7f5556ffd640 1 -- 172.21.15.43:0/3069229863 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5558013640 con 0x7f5560155f70 2024-04-25T08:56:48.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.202+0000 7f556858b640 1 -- 172.21.15.43:0/3069229863 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55600cbb40 con 0x7f5560155f70 2024-04-25T08:56:48.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.203+0000 7f556858b640 1 -- 172.21.15.43:0/3069229863 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55600680f0 con 0x7f5560155f70 2024-04-25T08:56:48.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.203+0000 7f5556ffd640 1 -- 172.21.15.43:0/3069229863 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f55580137e0 con 0x7f5560155f70 2024-04-25T08:56:48.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.204+0000 7f5556ffd640 1 --2- 172.21.15.43:0/3069229863 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f554406b500 0x7f554406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:48.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.204+0000 7f5556ffd640 1 -- 172.21.15.43:0/3069229863 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f555808a650 con 0x7f5560155f70 2024-04-25T08:56:48.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.204+0000 7f5566327640 1 --2- 172.21.15.43:0/3069229863 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f554406b500 0x7f554406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:48.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.205+0000 7f5566327640 1 --2- 172.21.15.43:0/3069229863 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f554406b500 0x7f554406d9c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f555c005570 tx=0x7f555c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:48.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.208+0000 7f5556ffd640 1 -- 172.21.15.43:0/3069229863 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f55601f81a0 con 0x7f5560155f70 2024-04-25T08:56:48.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.426+0000 7f556858b640 1 -- 172.21.15.43:0/3069229863 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f55600d1050 con 0x7f5560155f70 2024-04-25T08:56:48.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.426+0000 7f5556ffd640 1 -- 172.21.15.43:0/3069229863 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4934 (secure 0 0 0) 0x7f555805de20 con 0x7f5560155f70 2024-04-25T08:56:48.416 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:56:48.416 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4984,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/715858990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":715858990},{"type":"v1","addr":"172.21.15.193:6836","nonce":715858990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4987,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2342354979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2342354979},{"type":"v1","addr":"172.21.15.193:6837","nonce":2342354979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":5205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/491504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":491504707},{"type":"v1","addr":"172.21.15.43:6834","nonce":491504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T08:55:06.885018+0000","modified":"2024-04-25T08:55:14.001880+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5208},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5208":{"gid":5208,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/242762542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":242762542},{"type":"v1","addr":"172.21.15.43:6835","nonce":242762542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T08:56:48.417 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 14 2024-04-25T08:56:48.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.429+0000 7f556858b640 1 -- 172.21.15.43:0/3069229863 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f554406b500 msgr2=0x7f554406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:48.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.429+0000 7f556858b640 1 --2- 172.21.15.43:0/3069229863 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f554406b500 0x7f554406d9c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f555c005570 tx=0x7f555c009040 comp rx=0 tx=0).stop 2024-04-25T08:56:48.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.429+0000 7f556858b640 1 -- 172.21.15.43:0/3069229863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5560155f70 msgr2=0x7f55600b2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:48.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.429+0000 7f556858b640 1 --2- 172.21.15.43:0/3069229863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5560155f70 0x7f55600b2de0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f55600cfb60 tx=0x7f555800e530 comp rx=0 tx=0).stop 2024-04-25T08:56:48.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.429+0000 7f556858b640 1 -- 172.21.15.43:0/3069229863 shutdown_connections 2024-04-25T08:56:48.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.429+0000 7f556858b640 1 --2- 172.21.15.43:0/3069229863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5560155f70 0x7f55600b2de0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.429+0000 7f556858b640 1 --2- 172.21.15.43:0/3069229863 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f55600cafc0 0x7f55601f7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.429+0000 7f556858b640 1 --2- 172.21.15.43:0/3069229863 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f554406b500 0x7f554406d9c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.429+0000 7f556858b640 1 --2- 172.21.15.43:0/3069229863 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f554c004800 0x7f55600cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.429+0000 7f556858b640 1 -- 172.21.15.43:0/3069229863 >> 172.21.15.43:0/3069229863 conn(0x7f556009cd20 msgr2=0x7f55600b76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:48.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.429+0000 7f556858b640 1 -- 172.21.15.43:0/3069229863 shutdown_connections 2024-04-25T08:56:48.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.429+0000 7f556858b640 1 -- 172.21.15.43:0/3069229863 wait complete. 2024-04-25T08:56:48.430 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:56:48.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.541+0000 7f2ffd238640 1 Processor -- start 2024-04-25T08:56:48.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.541+0000 7f2ffd238640 1 -- start start 2024-04-25T08:56:48.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.542+0000 7f2ffd238640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2ff80d09b0 0x7f2ff80b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:48.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.542+0000 7f2ffd238640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2ff80ce5e0 0x7f2ff80cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:48.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.542+0000 7f2ffd238640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ff80b4c30 con 0x7f2ff8161390 2024-04-25T08:56:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.542+0000 7f2ffd238640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ff80d1910 con 0x7f2ff80d09b0 2024-04-25T08:56:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.542+0000 7f2ffd238640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff80cf2c0 con 0x7f2ff80ce5e0 2024-04-25T08:56:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.542+0000 7f2feffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2ff80d09b0 0x7f2ff80b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.542+0000 7f2feffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2ff80d09b0 0x7f2ff80b6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59310/0 (socket says 172.21.15.43:59310) 2024-04-25T08:56:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.542+0000 7f2feffff640 1 -- 172.21.15.43:0/3896634965 learned_addr learned my addr 172.21.15.43:0/3896634965 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.542+0000 7f2ff759e640 1 --2- 172.21.15.43:0/3896634965 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2ff80ce5e0 0x7f2ff80cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.542+0000 7f2feffff640 1 -- 172.21.15.43:0/3896634965 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2ff80ce5e0 msgr2=0x7f2ff80cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.542+0000 7f2feffff640 1 --2- 172.21.15.43:0/3896634965 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2ff80ce5e0 0x7f2ff80cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.542+0000 7f2feffff640 1 -- 172.21.15.43:0/3896634965 >> v1:172.21.15.193:6789/0 conn(0x7f2ff8161390 legacy=0x7f2ff80d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.542+0000 7f2feffff640 1 -- 172.21.15.43:0/3896634965 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff80cf5a0 con 0x7f2ff80d09b0 2024-04-25T08:56:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.543+0000 7f2feffff640 1 --2- 172.21.15.43:0/3896634965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ff80d09b0 0x7f2ff80b6820 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0009870 tx=0x7f2fe002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a17f164eb7324ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.543+0000 7f2ff659c640 1 -- 172.21.15.43:0/3896634965 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0037070 con 0x7f2ff80d09b0 2024-04-25T08:56:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.543+0000 7f2ff659c640 1 -- 172.21.15.43:0/3896634965 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe0032440 con 0x7f2ff80d09b0 2024-04-25T08:56:48.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.543+0000 7f2ff659c640 1 -- 172.21.15.43:0/3896634965 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0032810 con 0x7f2ff80d09b0 2024-04-25T08:56:48.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.543+0000 7f2ffd238640 1 -- 172.21.15.43:0/3896634965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ff80d09b0 msgr2=0x7f2ff80b6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:48.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.543+0000 7f2ffd238640 1 --2- 172.21.15.43:0/3896634965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ff80d09b0 0x7f2ff80b6820 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0009870 tx=0x7f2fe002a210 comp rx=0 tx=0).stop 2024-04-25T08:56:48.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.543+0000 7f2ffd238640 1 -- 172.21.15.43:0/3896634965 shutdown_connections 2024-04-25T08:56:48.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.543+0000 7f2ffd238640 1 --2- 172.21.15.43:0/3896634965 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2ff80ce5e0 0x7f2ff80cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.543+0000 7f2ffd238640 1 --2- 172.21.15.43:0/3896634965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ff80d09b0 0x7f2ff80b6820 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.544+0000 7f2ffd238640 1 -- 172.21.15.43:0/3896634965 >> 172.21.15.43:0/3896634965 conn(0x7f2ff809cd20 msgr2=0x7f2ff80d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:48.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.544+0000 7f2ffd238640 1 -- 172.21.15.43:0/3896634965 shutdown_connections 2024-04-25T08:56:48.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.544+0000 7f2ffd238640 1 -- 172.21.15.43:0/3896634965 wait complete. 2024-04-25T08:56:48.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.545+0000 7f2ffd238640 1 Processor -- start 2024-04-25T08:56:48.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.545+0000 7f2ffd238640 1 -- start start 2024-04-25T08:56:48.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.546+0000 7f2ffd238640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ff80ce5e0 0x7f2ff815cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:48.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.546+0000 7f2ffd238640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ff80d09b0 0x7f2ff806c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:48.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.546+0000 7f2ffd238640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2ff8161390 0x7f2ff806c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:48.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.546+0000 7f2ffd238640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff8161770 con 0x7f2ff80ce5e0 2024-04-25T08:56:48.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.546+0000 7f2ffd238640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff80d15a0 con 0x7f2ff80d09b0 2024-04-25T08:56:48.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.546+0000 7f2ffd238640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff80cefb0 con 0x7f2ff8161390 2024-04-25T08:56:48.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.546+0000 7f2ff6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ff80ce5e0 0x7f2ff815cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:48.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.546+0000 7f2ff6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ff80ce5e0 0x7f2ff815cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59324/0 (socket says 172.21.15.43:59324) 2024-04-25T08:56:48.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.546+0000 7f2ff6d9d640 1 -- 172.21.15.43:0/3253880423 learned_addr learned my addr 172.21.15.43:0/3253880423 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:48.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.546+0000 7f2feffff640 1 --2- 172.21.15.43:0/3253880423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ff80d09b0 0x7f2ff806c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:48.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.546+0000 7f2ff759e640 1 --2- 172.21.15.43:0/3253880423 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2ff8161390 0x7f2ff806c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:48.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.546+0000 7f2ff6d9d640 1 -- 172.21.15.43:0/3253880423 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2ff8161390 msgr2=0x7f2ff806c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:48.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.546+0000 7f2ff6d9d640 1 --2- 172.21.15.43:0/3253880423 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2ff8161390 0x7f2ff806c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.546+0000 7f2ff6d9d640 1 -- 172.21.15.43:0/3253880423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ff80d09b0 msgr2=0x7f2ff806c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:48.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.546+0000 7f2ff6d9d640 1 --2- 172.21.15.43:0/3253880423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ff80d09b0 0x7f2ff806c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.546+0000 7f2ff6d9d640 1 -- 172.21.15.43:0/3253880423 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff81f3a90 con 0x7f2ff80ce5e0 2024-04-25T08:56:48.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.546+0000 7f2ff6d9d640 1 --2- 172.21.15.43:0/3253880423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ff80ce5e0 0x7f2ff815cd30 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800b3f0 tx=0x7f2fe800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:48.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.546+0000 7f2fef7fe640 1 -- 172.21.15.43:0/3253880423 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8015020 con 0x7f2ff80ce5e0 2024-04-25T08:56:48.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.546+0000 7f2fef7fe640 1 -- 172.21.15.43:0/3253880423 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe8011830 con 0x7f2ff80ce5e0 2024-04-25T08:56:48.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.546+0000 7f2ffd238640 1 -- 172.21.15.43:0/3253880423 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ff815d730 con 0x7f2ff80ce5e0 2024-04-25T08:56:48.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.546+0000 7f2fef7fe640 1 -- 172.21.15.43:0/3253880423 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8011c20 con 0x7f2ff80ce5e0 2024-04-25T08:56:48.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.546+0000 7f2ffd238640 1 -- 172.21.15.43:0/3253880423 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ff815ab40 con 0x7f2ff80ce5e0 2024-04-25T08:56:48.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.547+0000 7f2ffd238640 1 -- 172.21.15.43:0/3253880423 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ff80680f0 con 0x7f2ff80ce5e0 2024-04-25T08:56:48.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.548+0000 7f2fef7fe640 1 -- 172.21.15.43:0/3253880423 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2fe8018020 con 0x7f2ff80ce5e0 2024-04-25T08:56:48.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.548+0000 7f2fef7fe640 1 --2- 172.21.15.43:0/3253880423 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2fd806b500 0x7f2fd806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:48.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.549+0000 7f2fef7fe640 1 -- 172.21.15.43:0/3253880423 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f2fe8089fe0 con 0x7f2ff80ce5e0 2024-04-25T08:56:48.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.549+0000 7f2feffff640 1 --2- 172.21.15.43:0/3253880423 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2fd806b500 0x7f2fd806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:48.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.549+0000 7f2feffff640 1 --2- 172.21.15.43:0/3253880423 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2fd806b500 0x7f2fd806d9c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0034000 tx=0x7f2fe002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:48.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.553+0000 7f2fef7fe640 1 -- 172.21.15.43:0/3253880423 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2fe801f020 con 0x7f2ff80ce5e0 2024-04-25T08:56:48.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.735+0000 7f2ffd238640 1 -- 172.21.15.43:0/3253880423 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ff80c7080 con 0x7f2ff80ce5e0 2024-04-25T08:56:48.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.736+0000 7f2fef7fe640 1 -- 172.21.15.43:0/3253880423 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16698 (secure 0 0 0) 0x7f2fe80587f0 con 0x7f2ff80ce5e0 2024-04-25T08:56:48.726 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:56:48.726 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":35,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:56:22.378131+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:55:05.150275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T08:55:05.853905+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":34,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:56:48.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.738+0000 7f2ffd238640 1 -- 172.21.15.43:0/3253880423 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2fd806b500 msgr2=0x7f2fd806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:48.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.738+0000 7f2ffd238640 1 --2- 172.21.15.43:0/3253880423 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2fd806b500 0x7f2fd806d9c0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0034000 tx=0x7f2fe002e000 comp rx=0 tx=0).stop 2024-04-25T08:56:48.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.738+0000 7f2ffd238640 1 -- 172.21.15.43:0/3253880423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ff80ce5e0 msgr2=0x7f2ff815cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:48.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.738+0000 7f2ffd238640 1 --2- 172.21.15.43:0/3253880423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ff80ce5e0 0x7f2ff815cd30 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800b3f0 tx=0x7f2fe800b8b0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.738+0000 7f2ffd238640 1 -- 172.21.15.43:0/3253880423 shutdown_connections 2024-04-25T08:56:48.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.738+0000 7f2ffd238640 1 --2- 172.21.15.43:0/3253880423 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2ff8161390 0x7f2ff806c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.738+0000 7f2ffd238640 1 --2- 172.21.15.43:0/3253880423 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2fd806b500 0x7f2fd806d9c0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.738+0000 7f2ffd238640 1 --2- 172.21.15.43:0/3253880423 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2ff80d09b0 0x7f2ff806c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.738+0000 7f2ffd238640 1 --2- 172.21.15.43:0/3253880423 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2ff80ce5e0 0x7f2ff815cd30 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.738+0000 7f2ffd238640 1 -- 172.21.15.43:0/3253880423 >> 172.21.15.43:0/3253880423 conn(0x7f2ff809cd20 msgr2=0x7f2ff80d38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:48.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.738+0000 7f2ffd238640 1 -- 172.21.15.43:0/3253880423 shutdown_connections 2024-04-25T08:56:48.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.738+0000 7f2ffd238640 1 -- 172.21.15.43:0/3253880423 wait complete. 2024-04-25T08:56:48.739 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T08:56:48.739 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:56:48.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.850+0000 7fdbd4a9e640 1 Processor -- start 2024-04-25T08:56:48.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.850+0000 7fdbd4a9e640 1 -- start start 2024-04-25T08:56:48.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.851+0000 7fdbd4a9e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdbd015df50 0x7fdbd01619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:48.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.851+0000 7fdbd4a9e640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fdbd0161f30 0x7fdbd0162380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:48.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.851+0000 7fdbd4a9e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdbd00c7cf0 0x7fdbd00c8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:48.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.851+0000 7fdbd4a9e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbd015e330 con 0x7fdbd015df50 2024-04-25T08:56:48.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.851+0000 7fdbd4a9e640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbd00d24a0 con 0x7fdbd0161f30 2024-04-25T08:56:48.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.851+0000 7fdbd4a9e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd00d2740 con 0x7fdbd00c7cf0 2024-04-25T08:56:48.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.852+0000 7fdbce59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdbd015df50 0x7fdbd01619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:48.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.852+0000 7fdbce59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdbd015df50 0x7fdbd01619f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59338/0 (socket says 172.21.15.43:59338) 2024-04-25T08:56:48.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.852+0000 7fdbce59c640 1 -- 172.21.15.43:0/3375951151 learned_addr learned my addr 172.21.15.43:0/3375951151 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:48.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.852+0000 7fdbced9d640 1 --2- 172.21.15.43:0/3375951151 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdbd00c7cf0 0x7fdbd00c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:48.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.852+0000 7fdbcdd9b640 1 --2- 172.21.15.43:0/3375951151 >> v2:172.21.15.193:3300/0 conn(0x7fdbd0161f30 0x7fdbd0162380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:48.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.852+0000 7fdbce59c640 1 -- 172.21.15.43:0/3375951151 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdbd00c7cf0 msgr2=0x7fdbd00c8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:48.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.852+0000 7fdbce59c640 1 --2- 172.21.15.43:0/3375951151 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdbd00c7cf0 0x7fdbd00c8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.852+0000 7fdbce59c640 1 -- 172.21.15.43:0/3375951151 >> v2:172.21.15.193:3300/0 conn(0x7fdbd0161f30 msgr2=0x7fdbd0162380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:48.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.852+0000 7fdbce59c640 1 --2- 172.21.15.43:0/3375951151 >> v2:172.21.15.193:3300/0 conn(0x7fdbd0161f30 0x7fdbd0162380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.852+0000 7fdbce59c640 1 -- 172.21.15.43:0/3375951151 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd00d2a20 con 0x7fdbd015df50 2024-04-25T08:56:48.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.853+0000 7fdbce59c640 1 --2- 172.21.15.43:0/3375951151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbd015df50 0x7fdbd01619f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fdbd00d45e0 tx=0x7fdbc002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b6d90c1bebfde56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:48.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.853+0000 7fdbcd59a640 1 -- 172.21.15.43:0/3375951151 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc0036070 con 0x7fdbd015df50 2024-04-25T08:56:48.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.853+0000 7fdbcd59a640 1 -- 172.21.15.43:0/3375951151 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbc0031600 con 0x7fdbd015df50 2024-04-25T08:56:48.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.853+0000 7fdbcd59a640 1 -- 172.21.15.43:0/3375951151 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbc00319d0 con 0x7fdbd015df50 2024-04-25T08:56:48.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.853+0000 7fdbd4a9e640 1 -- 172.21.15.43:0/3375951151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbd015df50 msgr2=0x7fdbd01619f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:48.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.853+0000 7fdbd4a9e640 1 --2- 172.21.15.43:0/3375951151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbd015df50 0x7fdbd01619f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fdbd00d45e0 tx=0x7fdbc002a6a0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.853+0000 7fdbd4a9e640 1 -- 172.21.15.43:0/3375951151 shutdown_connections 2024-04-25T08:56:48.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.853+0000 7fdbd4a9e640 1 --2- 172.21.15.43:0/3375951151 >> v2:172.21.15.193:3300/0 conn(0x7fdbd0161f30 0x7fdbd0162380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.853+0000 7fdbd4a9e640 1 --2- 172.21.15.43:0/3375951151 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdbd00c7cf0 0x7fdbd00c8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.853+0000 7fdbd4a9e640 1 --2- 172.21.15.43:0/3375951151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbd015df50 0x7fdbd01619f0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.853+0000 7fdbd4a9e640 1 -- 172.21.15.43:0/3375951151 >> 172.21.15.43:0/3375951151 conn(0x7fdbd00d3a30 msgr2=0x7fdbd00d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:48.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.853+0000 7fdbd4a9e640 1 -- 172.21.15.43:0/3375951151 shutdown_connections 2024-04-25T08:56:48.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.853+0000 7fdbd4a9e640 1 -- 172.21.15.43:0/3375951151 wait complete. 2024-04-25T08:56:48.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.855+0000 7fdbd4a9e640 1 Processor -- start 2024-04-25T08:56:48.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.855+0000 7fdbd4a9e640 1 -- start start 2024-04-25T08:56:48.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.855+0000 7fdbd4a9e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdbd00c7cf0 0x7fdbd015c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:48.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.855+0000 7fdbd4a9e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbd015df50 0x7fdbd009fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:48.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.855+0000 7fdbd4a9e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdbd0161f30 0x7fdbd00a0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:48.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.855+0000 7fdbd4a9e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd00d2190 con 0x7fdbd015df50 2024-04-25T08:56:48.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.855+0000 7fdbd4a9e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd00c77d0 con 0x7fdbd00c7cf0 2024-04-25T08:56:48.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.855+0000 7fdbd4a9e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbd00c8680 con 0x7fdbd0161f30 2024-04-25T08:56:48.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.856+0000 7fdbcdd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbd015df50 0x7fdbd009fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:48.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.856+0000 7fdbcdd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbd015df50 0x7fdbd009fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59344/0 (socket says 172.21.15.43:59344) 2024-04-25T08:56:48.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.856+0000 7fdbcdd9b640 1 -- 172.21.15.43:0/343811101 learned_addr learned my addr 172.21.15.43:0/343811101 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:48.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.856+0000 7fdbce59c640 1 --2- 172.21.15.43:0/343811101 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdbd00c7cf0 0x7fdbd015c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:48.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.856+0000 7fdbced9d640 1 --2- 172.21.15.43:0/343811101 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdbd0161f30 0x7fdbd00a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:48.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.856+0000 7fdbcdd9b640 1 -- 172.21.15.43:0/343811101 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdbd0161f30 msgr2=0x7fdbd00a0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:48.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.856+0000 7fdbcdd9b640 1 --2- 172.21.15.43:0/343811101 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdbd0161f30 0x7fdbd00a0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.856+0000 7fdbcdd9b640 1 -- 172.21.15.43:0/343811101 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdbd00c7cf0 msgr2=0x7fdbd015c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:48.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.856+0000 7fdbcdd9b640 1 --2- 172.21.15.43:0/343811101 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdbd00c7cf0 0x7fdbd015c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:48.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.856+0000 7fdbcdd9b640 1 -- 172.21.15.43:0/343811101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd009d770 con 0x7fdbd015df50 2024-04-25T08:56:48.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.856+0000 7fdbcdd9b640 1 --2- 172.21.15.43:0/343811101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbd015df50 0x7fdbd009fc40 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fdbb800c7f0 tx=0x7fdbb800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:48.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.856+0000 7fdbc6ffd640 1 -- 172.21.15.43:0/343811101 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8015070 con 0x7fdbd015df50 2024-04-25T08:56:48.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.856+0000 7fdbc6ffd640 1 -- 172.21.15.43:0/343811101 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb8004220 con 0x7fdbd015df50 2024-04-25T08:56:48.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.856+0000 7fdbd4a9e640 1 -- 172.21.15.43:0/343811101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbd009da50 con 0x7fdbd015df50 2024-04-25T08:56:48.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.856+0000 7fdbc6ffd640 1 -- 172.21.15.43:0/343811101 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8007620 con 0x7fdbd015df50 2024-04-25T08:56:48.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.856+0000 7fdbd4a9e640 1 -- 172.21.15.43:0/343811101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd009e010 con 0x7fdbd015df50 2024-04-25T08:56:48.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.857+0000 7fdbd4a9e640 1 -- 172.21.15.43:0/343811101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbd00680f0 con 0x7fdbd015df50 2024-04-25T08:56:48.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.858+0000 7fdbc6ffd640 1 -- 172.21.15.43:0/343811101 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdbb80079d0 con 0x7fdbd015df50 2024-04-25T08:56:48.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.858+0000 7fdbc6ffd640 1 --2- 172.21.15.43:0/343811101 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdbb006b4d0 0x7fdbb006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:48.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.858+0000 7fdbc6ffd640 1 -- 172.21.15.43:0/343811101 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fdbb808a4a0 con 0x7fdbd015df50 2024-04-25T08:56:48.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.858+0000 7fdbce59c640 1 --2- 172.21.15.43:0/343811101 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdbb006b4d0 0x7fdbb006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:48.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.859+0000 7fdbce59c640 1 --2- 172.21.15.43:0/343811101 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdbb006b4d0 0x7fdbb006d990 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0033000 tx=0x7fdbc0009000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:48.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:48.862+0000 7fdbc6ffd640 1 -- 172.21.15.43:0/343811101 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdbb8058d40 con 0x7fdbd015df50 2024-04-25T08:56:49.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.078+0000 7fdbd4a9e640 1 -- 172.21.15.43:0/343811101 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdbd0074ef0 con 0x7fdbd015df50 2024-04-25T08:56:49.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.079+0000 7fdbc6ffd640 1 -- 172.21.15.43:0/343811101 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4934 (secure 0 0 0) 0x7fdbb805cde0 con 0x7fdbd015df50 2024-04-25T08:56:49.069 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:56:49.069 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4984,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/715858990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":715858990},{"type":"v1","addr":"172.21.15.193:6836","nonce":715858990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4987,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2342354979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2342354979},{"type":"v1","addr":"172.21.15.193:6837","nonce":2342354979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":5205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/491504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":491504707},{"type":"v1","addr":"172.21.15.43:6834","nonce":491504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T08:55:06.885018+0000","modified":"2024-04-25T08:55:14.001880+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5208},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5208":{"gid":5208,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/242762542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":242762542},{"type":"v1","addr":"172.21.15.43:6835","nonce":242762542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T08:56:49.069 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 14 2024-04-25T08:56:49.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.081+0000 7fdbd4a9e640 1 -- 172.21.15.43:0/343811101 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdbb006b4d0 msgr2=0x7fdbb006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:49.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.081+0000 7fdbd4a9e640 1 --2- 172.21.15.43:0/343811101 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdbb006b4d0 0x7fdbb006d990 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0033000 tx=0x7fdbc0009000 comp rx=0 tx=0).stop 2024-04-25T08:56:49.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.081+0000 7fdbd4a9e640 1 -- 172.21.15.43:0/343811101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbd015df50 msgr2=0x7fdbd009fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:49.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.081+0000 7fdbd4a9e640 1 --2- 172.21.15.43:0/343811101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbd015df50 0x7fdbd009fc40 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fdbb800c7f0 tx=0x7fdbb800ccb0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.081+0000 7fdbd4a9e640 1 -- 172.21.15.43:0/343811101 shutdown_connections 2024-04-25T08:56:49.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.081+0000 7fdbd4a9e640 1 --2- 172.21.15.43:0/343811101 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdbd0161f30 0x7fdbd00a0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.081+0000 7fdbd4a9e640 1 --2- 172.21.15.43:0/343811101 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdbd015df50 0x7fdbd009fc40 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.081+0000 7fdbd4a9e640 1 --2- 172.21.15.43:0/343811101 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdbb006b4d0 0x7fdbb006d990 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.081+0000 7fdbd4a9e640 1 --2- 172.21.15.43:0/343811101 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdbd00c7cf0 0x7fdbd015c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.081+0000 7fdbd4a9e640 1 -- 172.21.15.43:0/343811101 >> 172.21.15.43:0/343811101 conn(0x7fdbd00d3a30 msgr2=0x7fdbd015cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:49.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.081+0000 7fdbd4a9e640 1 -- 172.21.15.43:0/343811101 shutdown_connections 2024-04-25T08:56:49.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.081+0000 7fdbd4a9e640 1 -- 172.21.15.43:0/343811101 wait complete. 2024-04-25T08:56:49.084 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:56:49.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.196+0000 7f6edcfe0640 1 Processor -- start 2024-04-25T08:56:49.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.196+0000 7f6edcfe0640 1 -- start start 2024-04-25T08:56:49.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.197+0000 7f6edcfe0640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6ed815df50 0x7f6ed81619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:49.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.197+0000 7f6edcfe0640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ed80d24a0 con 0x7f6ed8161f30 2024-04-25T08:56:49.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.197+0000 7f6edcfe0640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ed80d2740 con 0x7f6ed80cf670 2024-04-25T08:56:49.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.197+0000 7f6edcfe0640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ed80cfa80 con 0x7f6ed815df50 2024-04-25T08:56:49.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.197+0000 7f6ed5d9b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f6ed8161f30 0x7f6ed80c7cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39524/0 (socket says 172.21.15.43:39524) 2024-04-25T08:56:49.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.197+0000 7f6ed5d9b640 1 -- 172.21.15.43:0/1900564920 learned_addr learned my addr 172.21.15.43:0/1900564920 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:56:49.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.198+0000 7f6ed559a640 1 -- 172.21.15.43:0/1900564920 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3593053644 0 0) 0x7f6ed80d24a0 con 0x7f6ed8161f30 2024-04-25T08:56:49.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.198+0000 7f6ed659c640 1 --2- 172.21.15.43:0/1900564920 >> v2:172.21.15.193:3300/0 conn(0x7f6ed815df50 0x7f6ed81619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:49.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.198+0000 7f6ed559a640 1 -- 172.21.15.43:0/1900564920 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ec4003520 con 0x7f6ed8161f30 2024-04-25T08:56:49.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.198+0000 7f6ed559a640 1 -- 172.21.15.43:0/1900564920 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1359146732 0 0) 0x7f6ec4003520 con 0x7f6ed8161f30 2024-04-25T08:56:49.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.198+0000 7f6ed559a640 1 -- 172.21.15.43:0/1900564920 >> v2:172.21.15.193:3300/0 conn(0x7f6ed815df50 msgr2=0x7f6ed81619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:49.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.198+0000 7f6ed559a640 1 --2- 172.21.15.43:0/1900564920 >> v2:172.21.15.193:3300/0 conn(0x7f6ed815df50 0x7f6ed81619f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.198+0000 7f6ed559a640 1 -- 172.21.15.43:0/1900564920 >> v1:172.21.15.193:6789/0 conn(0x7f6ed80cf670 legacy=0x7f6ed80c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:49.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.198+0000 7f6ed559a640 1 -- 172.21.15.43:0/1900564920 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed80d2a20 con 0x7f6ed8161f30 2024-04-25T08:56:49.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.198+0000 7f6ed559a640 1 -- 172.21.15.43:0/1900564920 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f6ec0001690 con 0x7f6ed8161f30 2024-04-25T08:56:49.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.198+0000 7f6ed559a640 1 -- 172.21.15.43:0/1900564920 >> v1:172.21.15.43:6789/0 conn(0x7f6ed8161f30 legacy=0x7f6ed80c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:49.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.198+0000 7f6ed559a640 1 --2- 172.21.15.43:0/1900564920 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ec40039b0 0x7f6ec4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:49.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.198+0000 7f6ed559a640 1 --2- 172.21.15.43:0/1900564920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ec4004800 0x7f6ec4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:49.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.198+0000 7f6ed559a640 1 --2- 172.21.15.43:0/1900564920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ec40051c0 0x7f6ec400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:49.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.198+0000 7f6ed559a640 1 -- 172.21.15.43:0/1900564920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed80cfa80 con 0x7f6ec40051c0 2024-04-25T08:56:49.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.198+0000 7f6ed559a640 1 -- 172.21.15.43:0/1900564920 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec400c060 con 0x7f6ec4004800 2024-04-25T08:56:49.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.198+0000 7f6ed559a640 1 -- 172.21.15.43:0/1900564920 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec400c260 con 0x7f6ec40039b0 2024-04-25T08:56:49.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.198+0000 7f6ed6d9d640 1 --2- 172.21.15.43:0/1900564920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ec40051c0 0x7f6ec400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:49.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.198+0000 7f6ed659c640 1 --2- 172.21.15.43:0/1900564920 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ec40039b0 0x7f6ec4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:49.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.198+0000 7f6ed5d9b640 1 --2- 172.21.15.43:0/1900564920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ec4004800 0x7f6ec4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:49.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.198+0000 7f6ed6d9d640 1 -- 172.21.15.43:0/1900564920 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ec40039b0 msgr2=0x7f6ec4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:49.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.198+0000 7f6ed6d9d640 1 --2- 172.21.15.43:0/1900564920 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ec40039b0 0x7f6ec4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.198+0000 7f6ed6d9d640 1 -- 172.21.15.43:0/1900564920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ec4004800 msgr2=0x7f6ec4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:49.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.198+0000 7f6ed6d9d640 1 --2- 172.21.15.43:0/1900564920 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ec4004800 0x7f6ec4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.198+0000 7f6ed6d9d640 1 -- 172.21.15.43:0/1900564920 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec4003520 con 0x7f6ec40051c0 2024-04-25T08:56:49.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.199+0000 7f6ed6d9d640 1 -- 172.21.15.43:0/1900564920 reap_dead start 2024-04-25T08:56:49.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.199+0000 7f6ed6d9d640 1 --2- 172.21.15.43:0/1900564920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ec40051c0 0x7f6ec400b9c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8001e50 tx=0x7f6ec4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b3ae4e9222b1c96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:49.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.199+0000 7f6ed559a640 1 -- 172.21.15.43:0/1900564920 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8001a70 con 0x7f6ec40051c0 2024-04-25T08:56:49.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.199+0000 7f6ed559a640 1 -- 172.21.15.43:0/1900564920 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ec8003970 con 0x7f6ec40051c0 2024-04-25T08:56:49.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.199+0000 7f6ed559a640 1 -- 172.21.15.43:0/1900564920 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8003d60 con 0x7f6ec40051c0 2024-04-25T08:56:49.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.200+0000 7f6edcfe0640 1 -- 172.21.15.43:0/1900564920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ec40051c0 msgr2=0x7f6ec400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:49.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.200+0000 7f6edcfe0640 1 --2- 172.21.15.43:0/1900564920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ec40051c0 0x7f6ec400b9c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6ec8001e50 tx=0x7f6ec4005830 comp rx=0 tx=0).stop 2024-04-25T08:56:49.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.200+0000 7f6edcfe0640 1 -- 172.21.15.43:0/1900564920 shutdown_connections 2024-04-25T08:56:49.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.200+0000 7f6edcfe0640 1 --2- 172.21.15.43:0/1900564920 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ec40051c0 0x7f6ec400b9c0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.200+0000 7f6edcfe0640 1 -- 172.21.15.43:0/1900564920 >> 172.21.15.43:0/1900564920 conn(0x7f6ed80d3a30 msgr2=0x7f6ed80d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:49.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.200+0000 7f6edcfe0640 1 -- 172.21.15.43:0/1900564920 shutdown_connections 2024-04-25T08:56:49.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.200+0000 7f6edcfe0640 1 -- 172.21.15.43:0/1900564920 wait complete. 2024-04-25T08:56:49.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.201+0000 7f6edcfe0640 1 Processor -- start 2024-04-25T08:56:49.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.201+0000 7f6edcfe0640 1 -- start start 2024-04-25T08:56:49.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.202+0000 7f6edcfe0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ec40051c0 0x7f6ed815ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:49.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.202+0000 7f6edcfe0640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ed8147590 0x7f6ed8157910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:49.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.202+0000 7f6edcfe0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ed8147a10 0x7f6ed8157e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:49.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.202+0000 7f6edcfe0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed80c77d0 con 0x7f6ec40051c0 2024-04-25T08:56:49.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.202+0000 7f6edcfe0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed815e330 con 0x7f6ed8147a10 2024-04-25T08:56:49.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.202+0000 7f6edcfe0640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed80d2190 con 0x7f6ed8147590 2024-04-25T08:56:49.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.202+0000 7f6ed659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ec40051c0 0x7f6ed815ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:49.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.202+0000 7f6ed659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ec40051c0 0x7f6ed815ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59364/0 (socket says 172.21.15.43:59364) 2024-04-25T08:56:49.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.202+0000 7f6ed659c640 1 -- 172.21.15.43:0/944764601 learned_addr learned my addr 172.21.15.43:0/944764601 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:49.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.202+0000 7f6ed659c640 1 -- 172.21.15.43:0/944764601 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ed8147590 msgr2=0x7f6ed8157910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:49.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.202+0000 7f6ed659c640 1 --2- 172.21.15.43:0/944764601 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ed8147590 0x7f6ed8157910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.202+0000 7f6ed659c640 1 -- 172.21.15.43:0/944764601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ed8147a10 msgr2=0x7f6ed8157e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:49.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.202+0000 7f6ed659c640 1 --2- 172.21.15.43:0/944764601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ed8147a10 0x7f6ed8157e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.202+0000 7f6ed659c640 1 -- 172.21.15.43:0/944764601 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed81f4420 con 0x7f6ec40051c0 2024-04-25T08:56:49.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.203+0000 7f6ed659c640 1 --2- 172.21.15.43:0/944764601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ec40051c0 0x7f6ed815ac10 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6ed0011910 tx=0x7f6ed0011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:49.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.203+0000 7f6eceffd640 1 -- 172.21.15.43:0/944764601 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed000faf0 con 0x7f6ec40051c0 2024-04-25T08:56:49.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.203+0000 7f6eceffd640 1 -- 172.21.15.43:0/944764601 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed000fc90 con 0x7f6ec40051c0 2024-04-25T08:56:49.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.203+0000 7f6edcfe0640 1 -- 172.21.15.43:0/944764601 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ed81f4700 con 0x7f6ec40051c0 2024-04-25T08:56:49.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.203+0000 7f6eceffd640 1 -- 172.21.15.43:0/944764601 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed0014440 con 0x7f6ec40051c0 2024-04-25T08:56:49.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.203+0000 7f6edcfe0640 1 -- 172.21.15.43:0/944764601 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ed81f4cc0 con 0x7f6ec40051c0 2024-04-25T08:56:49.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.204+0000 7f6edcfe0640 1 -- 172.21.15.43:0/944764601 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ed80680f0 con 0x7f6ec40051c0 2024-04-25T08:56:49.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.204+0000 7f6eceffd640 1 -- 172.21.15.43:0/944764601 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6ed0012040 con 0x7f6ec40051c0 2024-04-25T08:56:49.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.205+0000 7f6eceffd640 1 --2- 172.21.15.43:0/944764601 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6eb806b4d0 0x7f6eb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:49.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.205+0000 7f6eceffd640 1 -- 172.21.15.43:0/944764601 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f6ed008a870 con 0x7f6ec40051c0 2024-04-25T08:56:49.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.205+0000 7f6ed5d9b640 1 --2- 172.21.15.43:0/944764601 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6eb806b4d0 0x7f6eb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:49.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.206+0000 7f6ed5d9b640 1 --2- 172.21.15.43:0/944764601 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6eb806b4d0 0x7f6eb806d990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0000f60 tx=0x7f6ec0000ed0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:49.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.209+0000 7f6eceffd640 1 -- 172.21.15.43:0/944764601 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6ed0019030 con 0x7f6ec40051c0 2024-04-25T08:56:49.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.426+0000 7f6edcfe0640 1 -- 172.21.15.43:0/944764601 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ed81f4f00 con 0x7f6ec40051c0 2024-04-25T08:56:49.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.426+0000 7f6eceffd640 1 -- 172.21.15.43:0/944764601 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4934 (secure 0 0 0) 0x7f6ed00596d0 con 0x7f6ec40051c0 2024-04-25T08:56:49.416 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:56:49.417 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4984,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/715858990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":715858990},{"type":"v1","addr":"172.21.15.193:6836","nonce":715858990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4987,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2342354979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2342354979},{"type":"v1","addr":"172.21.15.193:6837","nonce":2342354979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":5205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/491504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":491504707},{"type":"v1","addr":"172.21.15.43:6834","nonce":491504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T08:55:06.885018+0000","modified":"2024-04-25T08:55:14.001880+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5208},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5208":{"gid":5208,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/242762542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":242762542},{"type":"v1","addr":"172.21.15.43:6835","nonce":242762542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T08:56:49.417 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 14 2024-04-25T08:56:49.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.428+0000 7f6edcfe0640 1 -- 172.21.15.43:0/944764601 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6eb806b4d0 msgr2=0x7f6eb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:49.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.428+0000 7f6edcfe0640 1 --2- 172.21.15.43:0/944764601 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6eb806b4d0 0x7f6eb806d990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f6ec0000f60 tx=0x7f6ec0000ed0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.428+0000 7f6edcfe0640 1 -- 172.21.15.43:0/944764601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ec40051c0 msgr2=0x7f6ed815ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:49.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.428+0000 7f6edcfe0640 1 --2- 172.21.15.43:0/944764601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ec40051c0 0x7f6ed815ac10 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6ed0011910 tx=0x7f6ed0011dd0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.428+0000 7f6edcfe0640 1 -- 172.21.15.43:0/944764601 shutdown_connections 2024-04-25T08:56:49.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.428+0000 7f6edcfe0640 1 --2- 172.21.15.43:0/944764601 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6eb806b4d0 0x7f6eb806d990 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.428+0000 7f6edcfe0640 1 --2- 172.21.15.43:0/944764601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ed8147a10 0x7f6ed8157e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.428+0000 7f6edcfe0640 1 --2- 172.21.15.43:0/944764601 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ed8147590 0x7f6ed8157910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.428+0000 7f6edcfe0640 1 --2- 172.21.15.43:0/944764601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ec40051c0 0x7f6ed815ac10 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.428+0000 7f6edcfe0640 1 -- 172.21.15.43:0/944764601 >> 172.21.15.43:0/944764601 conn(0x7f6ed80d3a30 msgr2=0x7f6ed80d1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:49.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.429+0000 7f6edcfe0640 1 -- 172.21.15.43:0/944764601 shutdown_connections 2024-04-25T08:56:49.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.429+0000 7f6edcfe0640 1 -- 172.21.15.43:0/944764601 wait complete. 2024-04-25T08:56:49.429 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:56:49.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.540+0000 7fec67bf9640 1 Processor -- start 2024-04-25T08:56:49.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.541+0000 7fec67bf9640 1 -- start start 2024-04-25T08:56:49.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.541+0000 7fec67bf9640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fec600d09b0 0x7fec600b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:49.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.541+0000 7fec67bf9640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec600b4c30 con 0x7fec60161390 2024-04-25T08:56:49.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.541+0000 7fec67bf9640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec600cf2c0 con 0x7fec600ce5e0 2024-04-25T08:56:49.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.541+0000 7fec67bf9640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec600d1910 con 0x7fec600d09b0 2024-04-25T08:56:49.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.542+0000 7fec65194640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fec600d09b0 0x7fec600b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:49.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.542+0000 7fec65194640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fec600d09b0 0x7fec600b6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59374/0 (socket says 172.21.15.43:59374) 2024-04-25T08:56:49.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.542+0000 7fec65194640 1 -- 172.21.15.43:0/419728298 learned_addr learned my addr 172.21.15.43:0/419728298 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:49.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.542+0000 7fec65995640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fec60161390 0x7fec600d02a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39534/0 (socket says 172.21.15.43:39534) 2024-04-25T08:56:49.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.542+0000 7fec65194640 1 -- 172.21.15.43:0/419728298 >> v1:172.21.15.193:6789/0 conn(0x7fec600ce5e0 legacy=0x7fec600cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:49.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.542+0000 7fec65194640 1 -- 172.21.15.43:0/419728298 >> v1:172.21.15.43:6789/0 conn(0x7fec60161390 legacy=0x7fec600d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:49.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.542+0000 7fec65194640 1 -- 172.21.15.43:0/419728298 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec600cf5a0 con 0x7fec600d09b0 2024-04-25T08:56:49.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.542+0000 7fec64993640 1 -- 172.21.15.43:0/419728298 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3230764625 0 0) 0x7fec5c001110 con 0x7fec60161390 2024-04-25T08:56:49.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.543+0000 7fec65194640 1 --2- 172.21.15.43:0/419728298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec600d09b0 0x7fec600b6820 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fec50009870 tx=0x7fec5002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f89c617763182431 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:49.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.543+0000 7fec64993640 1 -- 172.21.15.43:0/419728298 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec50037070 con 0x7fec600d09b0 2024-04-25T08:56:49.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.543+0000 7fec64993640 1 -- 172.21.15.43:0/419728298 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec50032440 con 0x7fec600d09b0 2024-04-25T08:56:49.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.543+0000 7fec64993640 1 -- 172.21.15.43:0/419728298 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec50032810 con 0x7fec600d09b0 2024-04-25T08:56:49.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.543+0000 7fec67bf9640 1 -- 172.21.15.43:0/419728298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec600d09b0 msgr2=0x7fec600b6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:49.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.543+0000 7fec67bf9640 1 --2- 172.21.15.43:0/419728298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec600d09b0 0x7fec600b6820 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fec50009870 tx=0x7fec5002a2a0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.543+0000 7fec67bf9640 1 -- 172.21.15.43:0/419728298 shutdown_connections 2024-04-25T08:56:49.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.543+0000 7fec67bf9640 1 --2- 172.21.15.43:0/419728298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec600d09b0 0x7fec600b6820 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.543+0000 7fec67bf9640 1 -- 172.21.15.43:0/419728298 >> 172.21.15.43:0/419728298 conn(0x7fec6009cd20 msgr2=0x7fec600d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:49.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.543+0000 7fec67bf9640 1 -- 172.21.15.43:0/419728298 shutdown_connections 2024-04-25T08:56:49.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.543+0000 7fec67bf9640 1 -- 172.21.15.43:0/419728298 wait complete. 2024-04-25T08:56:49.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.545+0000 7fec67bf9640 1 Processor -- start 2024-04-25T08:56:49.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.545+0000 7fec67bf9640 1 -- start start 2024-04-25T08:56:49.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.545+0000 7fec67bf9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec600ce5e0 0x7fec600c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:49.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.545+0000 7fec67bf9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec600d09b0 0x7fec600cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:49.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.545+0000 7fec67bf9640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fec60161390 0x7fec6014be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:49.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.545+0000 7fec67bf9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec60161770 con 0x7fec600d09b0 2024-04-25T08:56:49.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.545+0000 7fec67bf9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec600d15a0 con 0x7fec600ce5e0 2024-04-25T08:56:49.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.545+0000 7fec67bf9640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec600d1da0 con 0x7fec60161390 2024-04-25T08:56:49.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.545+0000 7fec65194640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec600d09b0 0x7fec600cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:49.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.546+0000 7fec65194640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec600d09b0 0x7fec600cb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59386/0 (socket says 172.21.15.43:59386) 2024-04-25T08:56:49.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.546+0000 7fec65194640 1 -- 172.21.15.43:0/2767268269 learned_addr learned my addr 172.21.15.43:0/2767268269 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:49.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.546+0000 7fec65995640 1 --2- 172.21.15.43:0/2767268269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec600ce5e0 0x7fec600c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:49.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.546+0000 7fec66196640 1 --2- 172.21.15.43:0/2767268269 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fec60161390 0x7fec6014be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:49.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.546+0000 7fec65194640 1 -- 172.21.15.43:0/2767268269 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fec60161390 msgr2=0x7fec6014be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:49.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.546+0000 7fec65194640 1 --2- 172.21.15.43:0/2767268269 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fec60161390 0x7fec6014be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.546+0000 7fec65194640 1 -- 172.21.15.43:0/2767268269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec600ce5e0 msgr2=0x7fec600c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:49.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.546+0000 7fec65194640 1 --2- 172.21.15.43:0/2767268269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec600ce5e0 0x7fec600c81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.546+0000 7fec65194640 1 -- 172.21.15.43:0/2767268269 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec600cc7a0 con 0x7fec600d09b0 2024-04-25T08:56:49.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.546+0000 7fec65194640 1 --2- 172.21.15.43:0/2767268269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec600d09b0 0x7fec600cb4b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fec50009870 tx=0x7fec500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:49.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.546+0000 7fec567fc640 1 -- 172.21.15.43:0/2767268269 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec50037040 con 0x7fec600d09b0 2024-04-25T08:56:49.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.546+0000 7fec567fc640 1 -- 172.21.15.43:0/2767268269 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec50004250 con 0x7fec600d09b0 2024-04-25T08:56:49.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.546+0000 7fec67bf9640 1 -- 172.21.15.43:0/2767268269 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec600cbc20 con 0x7fec600d09b0 2024-04-25T08:56:49.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.546+0000 7fec567fc640 1 -- 172.21.15.43:0/2767268269 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5003c670 con 0x7fec600d09b0 2024-04-25T08:56:49.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.546+0000 7fec67bf9640 1 -- 172.21.15.43:0/2767268269 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec600cc1e0 con 0x7fec600d09b0 2024-04-25T08:56:49.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.547+0000 7fec67bf9640 1 -- 172.21.15.43:0/2767268269 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec600680f0 con 0x7fec600d09b0 2024-04-25T08:56:49.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.548+0000 7fec567fc640 1 -- 172.21.15.43:0/2767268269 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fec5003a070 con 0x7fec600d09b0 2024-04-25T08:56:49.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.548+0000 7fec567fc640 1 --2- 172.21.15.43:0/2767268269 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fec4406b500 0x7fec4406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:49.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.548+0000 7fec567fc640 1 -- 172.21.15.43:0/2767268269 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fec500aaa00 con 0x7fec600d09b0 2024-04-25T08:56:49.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.548+0000 7fec65995640 1 --2- 172.21.15.43:0/2767268269 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fec4406b500 0x7fec4406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:49.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.549+0000 7fec65995640 1 --2- 172.21.15.43:0/2767268269 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fec4406b500 0x7fec4406d9c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fec5c007fd0 tx=0x7fec5c007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:49.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.552+0000 7fec567fc640 1 -- 172.21.15.43:0/2767268269 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fec50041020 con 0x7fec600d09b0 2024-04-25T08:56:49.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.734+0000 7fec67bf9640 1 -- 172.21.15.43:0/2767268269 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec60074ef0 con 0x7fec600d09b0 2024-04-25T08:56:49.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.735+0000 7fec567fc640 1 -- 172.21.15.43:0/2767268269 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16698 (secure 0 0 0) 0x7fec500792a0 con 0x7fec600d09b0 2024-04-25T08:56:49.725 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:56:49.725 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":35,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:56:22.378131+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:55:05.150275+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-04-25T08:55:05.853905+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":34,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":5,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:56:49.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.737+0000 7fec67bf9640 1 -- 172.21.15.43:0/2767268269 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fec4406b500 msgr2=0x7fec4406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:49.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.737+0000 7fec67bf9640 1 --2- 172.21.15.43:0/2767268269 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fec4406b500 0x7fec4406d9c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fec5c007fd0 tx=0x7fec5c007de0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.737+0000 7fec67bf9640 1 -- 172.21.15.43:0/2767268269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec600d09b0 msgr2=0x7fec600cb4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:49.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.737+0000 7fec67bf9640 1 --2- 172.21.15.43:0/2767268269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec600d09b0 0x7fec600cb4b0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fec50009870 tx=0x7fec500046c0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.737+0000 7fec67bf9640 1 -- 172.21.15.43:0/2767268269 shutdown_connections 2024-04-25T08:56:49.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.737+0000 7fec67bf9640 1 --2- 172.21.15.43:0/2767268269 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fec60161390 0x7fec6014be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.737+0000 7fec67bf9640 1 --2- 172.21.15.43:0/2767268269 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec600d09b0 0x7fec600cb4b0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.737+0000 7fec67bf9640 1 --2- 172.21.15.43:0/2767268269 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fec4406b500 0x7fec4406d9c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.737+0000 7fec67bf9640 1 --2- 172.21.15.43:0/2767268269 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec600ce5e0 0x7fec600c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.737+0000 7fec67bf9640 1 -- 172.21.15.43:0/2767268269 >> 172.21.15.43:0/2767268269 conn(0x7fec6009cd20 msgr2=0x7fec600b7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:49.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.737+0000 7fec67bf9640 1 -- 172.21.15.43:0/2767268269 shutdown_connections 2024-04-25T08:56:49.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.737+0000 7fec67bf9640 1 -- 172.21.15.43:0/2767268269 wait complete. 2024-04-25T08:56:49.738 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:56:49.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.851+0000 7f5b4deac640 1 Processor -- start 2024-04-25T08:56:49.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.852+0000 7f5b4deac640 1 -- start start 2024-04-25T08:56:49.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.852+0000 7f5b4deac640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f5b480cff70 0x7f5b480cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:49.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.852+0000 7f5b4deac640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5b480b23f0 0x7f5b480ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:49.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.852+0000 7f5b4deac640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b480ccf10 con 0x7f5b4815f180 2024-04-25T08:56:49.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.852+0000 7f5b4deac640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b480cccc0 con 0x7f5b480b23f0 2024-04-25T08:56:49.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.852+0000 7f5b4deac640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b480cd840 con 0x7f5b480cff70 2024-04-25T08:56:49.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.852+0000 7f5b47fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5b480b23f0 0x7f5b480ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:49.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.853+0000 7f5b47fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5b480b23f0 0x7f5b480ce210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57646/0 (socket says 172.21.15.43:57646) 2024-04-25T08:56:49.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.853+0000 7f5b47fff640 1 -- 172.21.15.43:0/476967640 learned_addr learned my addr 172.21.15.43:0/476967640 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:49.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.853+0000 7f5b46ffd640 1 --2- 172.21.15.43:0/476967640 >> v2:172.21.15.193:3300/0 conn(0x7f5b480cff70 0x7f5b480cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:49.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.853+0000 7f5b47fff640 1 -- 172.21.15.43:0/476967640 >> v2:172.21.15.193:3300/0 conn(0x7f5b480cff70 msgr2=0x7f5b480cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:49.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.853+0000 7f5b47fff640 1 --2- 172.21.15.43:0/476967640 >> v2:172.21.15.193:3300/0 conn(0x7f5b480cff70 0x7f5b480cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.853+0000 7f5b47fff640 1 -- 172.21.15.43:0/476967640 >> v1:172.21.15.193:6789/0 conn(0x7f5b4815f180 legacy=0x7f5b480cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:49.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.853+0000 7f5b47fff640 1 -- 172.21.15.43:0/476967640 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b480cd1f0 con 0x7f5b480b23f0 2024-04-25T08:56:49.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.853+0000 7f5b46ffd640 1 --2- 172.21.15.43:0/476967640 >> v2:172.21.15.193:3300/0 conn(0x7f5b480cff70 0x7f5b480cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:56:49.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.854+0000 7f5b47fff640 1 --2- 172.21.15.43:0/476967640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b480b23f0 0x7f5b480ce210 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5b48071a40 tx=0x7f5b4002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ebbcd8afc79f98b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:49.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.854+0000 7f5b467fc640 1 -- 172.21.15.43:0/476967640 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b40036070 con 0x7f5b480b23f0 2024-04-25T08:56:49.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.854+0000 7f5b467fc640 1 -- 172.21.15.43:0/476967640 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b40031600 con 0x7f5b480b23f0 2024-04-25T08:56:49.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.854+0000 7f5b467fc640 1 -- 172.21.15.43:0/476967640 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b400319d0 con 0x7f5b480b23f0 2024-04-25T08:56:49.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.854+0000 7f5b4deac640 1 -- 172.21.15.43:0/476967640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b480b23f0 msgr2=0x7f5b480ce210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:49.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.854+0000 7f5b4deac640 1 --2- 172.21.15.43:0/476967640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b480b23f0 0x7f5b480ce210 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5b48071a40 tx=0x7f5b4002a6a0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.854+0000 7f5b4deac640 1 -- 172.21.15.43:0/476967640 shutdown_connections 2024-04-25T08:56:49.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.854+0000 7f5b4deac640 1 --2- 172.21.15.43:0/476967640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b480b23f0 0x7f5b480ce210 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.854+0000 7f5b4deac640 1 --2- 172.21.15.43:0/476967640 >> v2:172.21.15.193:3300/0 conn(0x7f5b480cff70 0x7f5b480cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.854+0000 7f5b4deac640 1 -- 172.21.15.43:0/476967640 >> 172.21.15.43:0/476967640 conn(0x7f5b480d17f0 msgr2=0x7f5b480d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:49.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.854+0000 7f5b4deac640 1 -- 172.21.15.43:0/476967640 shutdown_connections 2024-04-25T08:56:49.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.854+0000 7f5b4deac640 1 -- 172.21.15.43:0/476967640 wait complete. 2024-04-25T08:56:49.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.855+0000 7f5b4deac640 1 Processor -- start 2024-04-25T08:56:49.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.856+0000 7f5b4deac640 1 -- start start 2024-04-25T08:56:49.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.856+0000 7f5b4deac640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5b480b23f0 0x7f5b480c9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:49.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.856+0000 7f5b4deac640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b480cff70 0x7f5b481e6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:49.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.856+0000 7f5b4deac640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b4815f180 0x7f5b480cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:49.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.856+0000 7f5b4deac640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4815f560 con 0x7f5b4815f180 2024-04-25T08:56:49.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.856+0000 7f5b4deac640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b480d23f0 con 0x7f5b480cff70 2024-04-25T08:56:49.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.856+0000 7f5b4deac640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b480af540 con 0x7f5b480b23f0 2024-04-25T08:56:49.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.856+0000 7f5b47fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b4815f180 0x7f5b480cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:49.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.856+0000 7f5b47fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b4815f180 0x7f5b480cadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57656/0 (socket says 172.21.15.43:57656) 2024-04-25T08:56:49.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.856+0000 7f5b47fff640 1 -- 172.21.15.43:0/1022063565 learned_addr learned my addr 172.21.15.43:0/1022063565 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:49.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.857+0000 7f5b46ffd640 1 --2- 172.21.15.43:0/1022063565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b480cff70 0x7f5b481e6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:49.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.857+0000 7f5b477fe640 1 --2- 172.21.15.43:0/1022063565 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5b480b23f0 0x7f5b480c9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:49.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.857+0000 7f5b47fff640 1 -- 172.21.15.43:0/1022063565 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5b480b23f0 msgr2=0x7f5b480c9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:49.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.857+0000 7f5b47fff640 1 --2- 172.21.15.43:0/1022063565 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5b480b23f0 0x7f5b480c9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.857+0000 7f5b47fff640 1 -- 172.21.15.43:0/1022063565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b480cff70 msgr2=0x7f5b481e6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:49.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.857+0000 7f5b47fff640 1 --2- 172.21.15.43:0/1022063565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b480cff70 0x7f5b481e6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:49.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.857+0000 7f5b47fff640 1 -- 172.21.15.43:0/1022063565 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b481e73a0 con 0x7f5b4815f180 2024-04-25T08:56:49.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.857+0000 7f5b477fe640 1 --2- 172.21.15.43:0/1022063565 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5b480b23f0 0x7f5b480c9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:49.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.857+0000 7f5b47fff640 1 --2- 172.21.15.43:0/1022063565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b4815f180 0x7f5b480cadb0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f5b40009f40 tx=0x7f5b40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:49.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.857+0000 7f5b2ffff640 1 -- 172.21.15.43:0/1022063565 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b40036050 con 0x7f5b4815f180 2024-04-25T08:56:49.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.857+0000 7f5b2ffff640 1 -- 172.21.15.43:0/1022063565 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b40042070 con 0x7f5b4815f180 2024-04-25T08:56:49.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.857+0000 7f5b4deac640 1 -- 172.21.15.43:0/1022063565 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b481e6860 con 0x7f5b4815f180 2024-04-25T08:56:49.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.857+0000 7f5b2ffff640 1 -- 172.21.15.43:0/1022063565 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b40009510 con 0x7f5b4815f180 2024-04-25T08:56:49.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.857+0000 7f5b4deac640 1 -- 172.21.15.43:0/1022063565 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b481e6de0 con 0x7f5b4815f180 2024-04-25T08:56:49.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.858+0000 7f5b4deac640 1 -- 172.21.15.43:0/1022063565 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b48160710 con 0x7f5b4815f180 2024-04-25T08:56:49.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.859+0000 7f5b2ffff640 1 -- 172.21.15.43:0/1022063565 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5b40007bd0 con 0x7f5b4815f180 2024-04-25T08:56:49.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.859+0000 7f5b2ffff640 1 --2- 172.21.15.43:0/1022063565 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5b3406b4d0 0x7f5b3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:49.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.859+0000 7f5b2ffff640 1 -- 172.21.15.43:0/1022063565 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f5b400ab170 con 0x7f5b4815f180 2024-04-25T08:56:49.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.859+0000 7f5b477fe640 1 --2- 172.21.15.43:0/1022063565 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5b3406b4d0 0x7f5b3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:49.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.860+0000 7f5b477fe640 1 --2- 172.21.15.43:0/1022063565 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5b3406b4d0 0x7f5b3406d990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5b480ce250 tx=0x7f5b38009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:49.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:49.863+0000 7f5b2ffff640 1 -- 172.21.15.43:0/1022063565 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5b48160710 con 0x7f5b4815f180 2024-04-25T08:56:50.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.081+0000 7f5b4deac640 1 -- 172.21.15.43:0/1022063565 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5b48074ef0 con 0x7f5b4815f180 2024-04-25T08:56:50.071 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-04-25T08:56:50.081+0000 7fe4fb198640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:56:50.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.082+0000 7f5b2ffff640 1 -- 172.21.15.43:0/1022063565 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v15) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5b40079fd0 con 0x7f5b4815f180 2024-04-25T08:56:50.072 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:56:50.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.085+0000 7f5b4deac640 1 -- 172.21.15.43:0/1022063565 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5b3406b4d0 msgr2=0x7f5b3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:50.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.085+0000 7f5b4deac640 1 --2- 172.21.15.43:0/1022063565 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5b3406b4d0 0x7f5b3406d990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f5b480ce250 tx=0x7f5b38009040 comp rx=0 tx=0).stop 2024-04-25T08:56:50.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.085+0000 7f5b4deac640 1 -- 172.21.15.43:0/1022063565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b4815f180 msgr2=0x7f5b480cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:50.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.085+0000 7f5b4deac640 1 --2- 172.21.15.43:0/1022063565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b4815f180 0x7f5b480cadb0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f5b40009f40 tx=0x7f5b40004a40 comp rx=0 tx=0).stop 2024-04-25T08:56:50.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.085+0000 7f5b4deac640 1 -- 172.21.15.43:0/1022063565 shutdown_connections 2024-04-25T08:56:50.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.085+0000 7f5b4deac640 1 --2- 172.21.15.43:0/1022063565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b4815f180 0x7f5b480cadb0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:50.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.085+0000 7f5b4deac640 1 --2- 172.21.15.43:0/1022063565 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5b3406b4d0 0x7f5b3406d990 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:50.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.085+0000 7f5b4deac640 1 --2- 172.21.15.43:0/1022063565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b480cff70 0x7f5b481e6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:50.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.085+0000 7f5b4deac640 1 --2- 172.21.15.43:0/1022063565 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5b480b23f0 0x7f5b480c9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:50.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.085+0000 7f5b4deac640 1 -- 172.21.15.43:0/1022063565 >> 172.21.15.43:0/1022063565 conn(0x7f5b480d17f0 msgr2=0x7f5b4815f870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:50.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.085+0000 7f5b4deac640 1 -- 172.21.15.43:0/1022063565 shutdown_connections 2024-04-25T08:56:50.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.086+0000 7f5b4deac640 1 -- 172.21.15.43:0/1022063565 wait complete. 2024-04-25T08:56:50.087 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T08:56:50.116 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-04-25T08:56:50.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.203+0000 7f377b934640 1 Processor -- start 2024-04-25T08:56:50.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.203+0000 7f377b934640 1 -- start start 2024-04-25T08:56:50.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.204+0000 7f377b934640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f377406ab30 0x7f377406af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:50.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.204+0000 7f377b934640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f37740ade30 0x7f37740ae2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:50.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.204+0000 7f377b934640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37740b4e60 con 0x7f37740b42a0 2024-04-25T08:56:50.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.204+0000 7f377b934640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f377415e160 con 0x7f377406ab30 2024-04-25T08:56:50.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.204+0000 7f377b934640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f377406b450 con 0x7f37740ade30 2024-04-25T08:56:50.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.204+0000 7f37796d0640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f377406ab30 0x7f377406af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:50.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.204+0000 7f3778ecf640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f37740b42a0 0x7f37740b4750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49718/0 (socket says 172.21.15.43:49718) 2024-04-25T08:56:50.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.204+0000 7f3778ecf640 1 -- 172.21.15.43:0/341405643 learned_addr learned my addr 172.21.15.43:0/341405643 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:56:50.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.204+0000 7f3779ed1640 1 --2- 172.21.15.43:0/341405643 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f37740ade30 0x7f37740ae2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:50.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.204+0000 7f37796d0640 1 -- 172.21.15.43:0/341405643 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f37740ade30 msgr2=0x7f37740ae2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:50.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.204+0000 7f37796d0640 1 --2- 172.21.15.43:0/341405643 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f37740ade30 0x7f37740ae2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:50.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.204+0000 7f37796d0640 1 -- 172.21.15.43:0/341405643 >> v1:172.21.15.43:6789/0 conn(0x7f37740b42a0 legacy=0x7f37740b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:50.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.204+0000 7f37796d0640 1 -- 172.21.15.43:0/341405643 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f377415e400 con 0x7f377406ab30 2024-04-25T08:56:50.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.204+0000 7f3773fff640 1 -- 172.21.15.43:0/341405643 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1977159185 0 0) 0x7f37740b4e60 con 0x7f37740b42a0 2024-04-25T08:56:50.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.204+0000 7f3779ed1640 1 --2- 172.21.15.43:0/341405643 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f37740ade30 0x7f37740ae2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:50.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.205+0000 7f37796d0640 1 --2- 172.21.15.43:0/341405643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f377406ab30 0x7f377406af10 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f376c009d90 tx=0x7f376c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9160c56f0b033fc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:50.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.205+0000 7f3773fff640 1 -- 172.21.15.43:0/341405643 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376c007740 con 0x7f377406ab30 2024-04-25T08:56:50.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.205+0000 7f3773fff640 1 -- 172.21.15.43:0/341405643 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f376c0078e0 con 0x7f377406ab30 2024-04-25T08:56:50.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.205+0000 7f3773fff640 1 -- 172.21.15.43:0/341405643 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376c007cb0 con 0x7f377406ab30 2024-04-25T08:56:50.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.206+0000 7f377b934640 1 -- 172.21.15.43:0/341405643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f377406ab30 msgr2=0x7f377406af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:50.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.206+0000 7f377b934640 1 --2- 172.21.15.43:0/341405643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f377406ab30 0x7f377406af10 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f376c009d90 tx=0x7f376c02a660 comp rx=0 tx=0).stop 2024-04-25T08:56:50.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.206+0000 7f377b934640 1 -- 172.21.15.43:0/341405643 shutdown_connections 2024-04-25T08:56:50.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.206+0000 7f377b934640 1 --2- 172.21.15.43:0/341405643 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f37740ade30 0x7f37740ae2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:50.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.206+0000 7f377b934640 1 --2- 172.21.15.43:0/341405643 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f377406ab30 0x7f377406af10 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:50.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.206+0000 7f377b934640 1 -- 172.21.15.43:0/341405643 >> 172.21.15.43:0/341405643 conn(0x7f37740b2590 msgr2=0x7f37740b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:50.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.206+0000 7f377b934640 1 -- 172.21.15.43:0/341405643 shutdown_connections 2024-04-25T08:56:50.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.206+0000 7f377b934640 1 -- 172.21.15.43:0/341405643 wait complete. 2024-04-25T08:56:50.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.207+0000 7f377b934640 1 Processor -- start 2024-04-25T08:56:50.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.207+0000 7f377b934640 1 -- start start 2024-04-25T08:56:50.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.208+0000 7f377b934640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f377406ab30 0x7f3774152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:50.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.208+0000 7f377b934640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37740ade30 0x7f3774154500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:50.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.208+0000 7f377b934640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f37740b42a0 0x7f3774155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:50.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.208+0000 7f377b934640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377415eca0 con 0x7f377406ab30 2024-04-25T08:56:50.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.208+0000 7f377b934640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377415e9c0 con 0x7f37740ade30 2024-04-25T08:56:50.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.208+0000 7f377b934640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f377415b950 con 0x7f37740b42a0 2024-04-25T08:56:50.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.208+0000 7f37796d0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f377406ab30 0x7f3774152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:50.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.208+0000 7f37796d0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f377406ab30 0x7f3774152220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57688/0 (socket says 172.21.15.43:57688) 2024-04-25T08:56:50.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.208+0000 7f37796d0640 1 -- 172.21.15.43:0/2791176809 learned_addr learned my addr 172.21.15.43:0/2791176809 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:50.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.208+0000 7f3778ecf640 1 --2- 172.21.15.43:0/2791176809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37740ade30 0x7f3774154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:50.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.208+0000 7f3779ed1640 1 --2- 172.21.15.43:0/2791176809 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f37740b42a0 0x7f3774155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:50.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.208+0000 7f37796d0640 1 -- 172.21.15.43:0/2791176809 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f37740b42a0 msgr2=0x7f3774155970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:50.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.208+0000 7f37796d0640 1 --2- 172.21.15.43:0/2791176809 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f37740b42a0 0x7f3774155970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:50.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.208+0000 7f37796d0640 1 -- 172.21.15.43:0/2791176809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37740ade30 msgr2=0x7f3774154500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:50.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.208+0000 7f37796d0640 1 --2- 172.21.15.43:0/2791176809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37740ade30 0x7f3774154500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:50.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.208+0000 7f37796d0640 1 -- 172.21.15.43:0/2791176809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37741561a0 con 0x7f377406ab30 2024-04-25T08:56:50.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.209+0000 7f37796d0640 1 --2- 172.21.15.43:0/2791176809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f377406ab30 0x7f3774152220 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f376c009eb0 tx=0x7f376c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:50.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.209+0000 7f3771ffb640 1 -- 172.21.15.43:0/2791176809 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376c03f070 con 0x7f377406ab30 2024-04-25T08:56:50.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.209+0000 7f3771ffb640 1 -- 172.21.15.43:0/2791176809 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f376c031c40 con 0x7f377406ab30 2024-04-25T08:56:50.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.209+0000 7f377b934640 1 -- 172.21.15.43:0/2791176809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3774151740 con 0x7f377406ab30 2024-04-25T08:56:50.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.209+0000 7f3771ffb640 1 -- 172.21.15.43:0/2791176809 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376c043410 con 0x7f377406ab30 2024-04-25T08:56:50.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.209+0000 7f377b934640 1 -- 172.21.15.43:0/2791176809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3774151d00 con 0x7f377406ab30 2024-04-25T08:56:50.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.210+0000 7f377b934640 1 -- 172.21.15.43:0/2791176809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3774074f60 con 0x7f377406ab30 2024-04-25T08:56:50.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.210+0000 7f3771ffb640 1 -- 172.21.15.43:0/2791176809 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f376c0435b0 con 0x7f377406ab30 2024-04-25T08:56:50.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.211+0000 7f3771ffb640 1 --2- 172.21.15.43:0/2791176809 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f376006b4d0 0x7f376006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:50.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.211+0000 7f3771ffb640 1 -- 172.21.15.43:0/2791176809 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f376c0ab620 con 0x7f377406ab30 2024-04-25T08:56:50.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.211+0000 7f3778ecf640 1 --2- 172.21.15.43:0/2791176809 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f376006b4d0 0x7f376006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:50.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.211+0000 7f3778ecf640 1 --2- 172.21.15.43:0/2791176809 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f376006b4d0 0x7f376006d990 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3774152f10 tx=0x7f3768007d50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:50.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.215+0000 7f3771ffb640 1 -- 172.21.15.43:0/2791176809 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f376c079e80 con 0x7f377406ab30 2024-04-25T08:56:50.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:50.432+0000 7f377b934640 1 -- 172.21.15.43:0/2791176809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f377406ea40 con 0x7f377406ab30 2024-04-25T08:56:51.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.084+0000 7f3771ffb640 1 -- 172.21.15.43:0/2791176809 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v16) v1 ==== 104+0+0 (secure 0 0 0) 0x7f376c07df20 con 0x7f377406ab30 2024-04-25T08:56:51.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.087+0000 7f377b934640 1 -- 172.21.15.43:0/2791176809 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f376006b4d0 msgr2=0x7f376006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:51.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.087+0000 7f377b934640 1 --2- 172.21.15.43:0/2791176809 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f376006b4d0 0x7f376006d990 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3774152f10 tx=0x7f3768007d50 comp rx=0 tx=0).stop 2024-04-25T08:56:51.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.087+0000 7f377b934640 1 -- 172.21.15.43:0/2791176809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f377406ab30 msgr2=0x7f3774152220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:51.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.087+0000 7f377b934640 1 --2- 172.21.15.43:0/2791176809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f377406ab30 0x7f3774152220 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f376c009eb0 tx=0x7f376c004a40 comp rx=0 tx=0).stop 2024-04-25T08:56:51.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.087+0000 7f377b934640 1 -- 172.21.15.43:0/2791176809 shutdown_connections 2024-04-25T08:56:51.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.087+0000 7f377b934640 1 --2- 172.21.15.43:0/2791176809 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f37740b42a0 0x7f3774155970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:51.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.087+0000 7f377b934640 1 --2- 172.21.15.43:0/2791176809 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f376006b4d0 0x7f376006d990 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:51.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.087+0000 7f377b934640 1 --2- 172.21.15.43:0/2791176809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37740ade30 0x7f3774154500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:51.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.087+0000 7f377b934640 1 --2- 172.21.15.43:0/2791176809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f377406ab30 0x7f3774152220 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:51.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.087+0000 7f377b934640 1 -- 172.21.15.43:0/2791176809 >> 172.21.15.43:0/2791176809 conn(0x7f37740b2590 msgr2=0x7f377415bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:51.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.087+0000 7f377b934640 1 -- 172.21.15.43:0/2791176809 shutdown_connections 2024-04-25T08:56:51.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.087+0000 7f377b934640 1 -- 172.21.15.43:0/2791176809 wait complete. 2024-04-25T08:56:51.091 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T08:56:51.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.255+0000 7f72d152b640 1 Processor -- start 2024-04-25T08:56:51.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.255+0000 7f72d152b640 1 -- start start 2024-04-25T08:56:51.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.256+0000 7f72d152b640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f72cc0d3710 0x7f72cc0d3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:51.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.256+0000 7f72d152b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f72cc0d2010 0x7f72cc0d24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:51.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.256+0000 7f72d152b640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72cc0b3070 con 0x7f72cc15df10 2024-04-25T08:56:51.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.256+0000 7f72d152b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72cc0b3f20 con 0x7f72cc0d2010 2024-04-25T08:56:51.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.256+0000 7f72d152b640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72cc0b4ac0 con 0x7f72cc0d3710 2024-04-25T08:56:51.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.256+0000 7f72cb7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f72cc0d2010 0x7f72cc0d24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:51.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.256+0000 7f72cb7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f72cc0d2010 0x7f72cc0d24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57694/0 (socket says 172.21.15.43:57694) 2024-04-25T08:56:51.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.256+0000 7f72cb7fe640 1 -- 172.21.15.43:0/4190190469 learned_addr learned my addr 172.21.15.43:0/4190190469 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:51.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.256+0000 7f72caffd640 1 --2- 172.21.15.43:0/4190190469 >> v2:172.21.15.193:3300/0 conn(0x7f72cc0d3710 0x7f72cc0d3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:51.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.256+0000 7f72cb7fe640 1 -- 172.21.15.43:0/4190190469 >> v2:172.21.15.193:3300/0 conn(0x7f72cc0d3710 msgr2=0x7f72cc0d3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:51.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.256+0000 7f72cb7fe640 1 --2- 172.21.15.43:0/4190190469 >> v2:172.21.15.193:3300/0 conn(0x7f72cc0d3710 0x7f72cc0d3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:51.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.256+0000 7f72cb7fe640 1 -- 172.21.15.43:0/4190190469 >> v1:172.21.15.193:6789/0 conn(0x7f72cc15df10 legacy=0x7f72cc0b43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:51.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.256+0000 7f72cb7fe640 1 -- 172.21.15.43:0/4190190469 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72cc0b3310 con 0x7f72cc0d2010 2024-04-25T08:56:51.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.256+0000 7f72caffd640 1 --2- 172.21.15.43:0/4190190469 >> v2:172.21.15.193:3300/0 conn(0x7f72cc0d3710 0x7f72cc0d3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:51.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.257+0000 7f72cb7fe640 1 --2- 172.21.15.43:0/4190190469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72cc0d2010 0x7f72cc0d24a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f72cc068670 tx=0x7f72c402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a81443c8f4a74e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:51.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.257+0000 7f72ca7fc640 1 -- 172.21.15.43:0/4190190469 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c4036070 con 0x7f72cc0d2010 2024-04-25T08:56:51.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.257+0000 7f72ca7fc640 1 -- 172.21.15.43:0/4190190469 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c4031600 con 0x7f72cc0d2010 2024-04-25T08:56:51.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.257+0000 7f72ca7fc640 1 -- 172.21.15.43:0/4190190469 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c40319d0 con 0x7f72cc0d2010 2024-04-25T08:56:51.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.258+0000 7f72d152b640 1 -- 172.21.15.43:0/4190190469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72cc0d2010 msgr2=0x7f72cc0d24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:51.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.258+0000 7f72d152b640 1 --2- 172.21.15.43:0/4190190469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72cc0d2010 0x7f72cc0d24a0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f72cc068670 tx=0x7f72c402a660 comp rx=0 tx=0).stop 2024-04-25T08:56:51.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.258+0000 7f72d152b640 1 -- 172.21.15.43:0/4190190469 shutdown_connections 2024-04-25T08:56:51.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.258+0000 7f72d152b640 1 --2- 172.21.15.43:0/4190190469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72cc0d2010 0x7f72cc0d24a0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:51.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.258+0000 7f72d152b640 1 --2- 172.21.15.43:0/4190190469 >> v2:172.21.15.193:3300/0 conn(0x7f72cc0d3710 0x7f72cc0d3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:51.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.258+0000 7f72d152b640 1 -- 172.21.15.43:0/4190190469 >> 172.21.15.43:0/4190190469 conn(0x7f72cc09cd50 msgr2=0x7f72cc08cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:51.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.258+0000 7f72d152b640 1 -- 172.21.15.43:0/4190190469 shutdown_connections 2024-04-25T08:56:51.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.258+0000 7f72d152b640 1 -- 172.21.15.43:0/4190190469 wait complete. 2024-04-25T08:56:51.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.259+0000 7f72d152b640 1 Processor -- start 2024-04-25T08:56:51.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.259+0000 7f72d152b640 1 -- start start 2024-04-25T08:56:51.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.260+0000 7f72d152b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f72cc0d2010 0x7f72cc152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:51.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.260+0000 7f72d152b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72cc0d3710 0x7f72cc0b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:51.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.260+0000 7f72d152b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72cc15df10 0x7f72cc0b2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:51.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.260+0000 7f72d152b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc0b3bb0 con 0x7f72cc15df10 2024-04-25T08:56:51.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.260+0000 7f72d152b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc0b38d0 con 0x7f72cc0d3710 2024-04-25T08:56:51.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.260+0000 7f72d152b640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc0d29e0 con 0x7f72cc0d2010 2024-04-25T08:56:51.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.260+0000 7f72cb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72cc15df10 0x7f72cc0b2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:51.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.260+0000 7f72cb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72cc15df10 0x7f72cc0b2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57702/0 (socket says 172.21.15.43:57702) 2024-04-25T08:56:51.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.260+0000 7f72cb7fe640 1 -- 172.21.15.43:0/295909928 learned_addr learned my addr 172.21.15.43:0/295909928 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:51.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.260+0000 7f72caffd640 1 --2- 172.21.15.43:0/295909928 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f72cc0d2010 0x7f72cc152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:51.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.260+0000 7f72c3fff640 1 --2- 172.21.15.43:0/295909928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72cc0d3710 0x7f72cc0b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:51.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.260+0000 7f72cb7fe640 1 -- 172.21.15.43:0/295909928 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f72cc0d2010 msgr2=0x7f72cc152aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:51.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.260+0000 7f72cb7fe640 1 --2- 172.21.15.43:0/295909928 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f72cc0d2010 0x7f72cc152aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:51.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.260+0000 7f72cb7fe640 1 -- 172.21.15.43:0/295909928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72cc0d3710 msgr2=0x7f72cc0b1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:51.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.260+0000 7f72cb7fe640 1 --2- 172.21.15.43:0/295909928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72cc0d3710 0x7f72cc0b1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:51.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.260+0000 7f72cb7fe640 1 -- 172.21.15.43:0/295909928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72cc1f4600 con 0x7f72cc15df10 2024-04-25T08:56:51.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.261+0000 7f72cb7fe640 1 --2- 172.21.15.43:0/295909928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72cc15df10 0x7f72cc0b2150 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f72c402ac50 tx=0x7f72c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:51.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.261+0000 7f72c37fe640 1 -- 172.21.15.43:0/295909928 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c403f020 con 0x7f72cc15df10 2024-04-25T08:56:51.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.261+0000 7f72c37fe640 1 -- 172.21.15.43:0/295909928 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c4037040 con 0x7f72cc15df10 2024-04-25T08:56:51.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.261+0000 7f72d152b640 1 -- 172.21.15.43:0/295909928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72cc1f48e0 con 0x7f72cc15df10 2024-04-25T08:56:51.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.261+0000 7f72c37fe640 1 -- 172.21.15.43:0/295909928 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c4036070 con 0x7f72cc15df10 2024-04-25T08:56:51.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.261+0000 7f72d152b640 1 -- 172.21.15.43:0/295909928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72cc1f5020 con 0x7f72cc15df10 2024-04-25T08:56:51.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.262+0000 7f72d152b640 1 -- 172.21.15.43:0/295909928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72cc068a40 con 0x7f72cc15df10 2024-04-25T08:56:51.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.262+0000 7f72c37fe640 1 -- 172.21.15.43:0/295909928 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f72c4042020 con 0x7f72cc15df10 2024-04-25T08:56:51.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.263+0000 7f72c37fe640 1 --2- 172.21.15.43:0/295909928 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f72ac06b4d0 0x7f72ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:51.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.263+0000 7f72c37fe640 1 -- 172.21.15.43:0/295909928 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f72c40aa620 con 0x7f72cc15df10 2024-04-25T08:56:51.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.263+0000 7f72caffd640 1 --2- 172.21.15.43:0/295909928 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f72ac06b4d0 0x7f72ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:51.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.264+0000 7f72caffd640 1 --2- 172.21.15.43:0/295909928 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f72ac06b4d0 0x7f72ac06d990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f72bc00ab60 tx=0x7f72bc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:51.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.267+0000 7f72c37fe640 1 -- 172.21.15.43:0/295909928 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f72c4049030 con 0x7f72cc15df10 2024-04-25T08:56:51.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:51.451+0000 7f72d152b640 1 -- 172.21.15.43:0/295909928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f72cc0c7260 con 0x7f72cc15df10 2024-04-25T08:56:52.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.437+0000 7f72c37fe640 1 -- 172.21.15.43:0/295909928 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v37) v1 ==== 181+0+0 (secure 0 0 0) 0x7f72cc0b3070 con 0x7f72cc15df10 2024-04-25T08:56:52.427 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' removed 2024-04-25T08:56:52.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.441+0000 7f72d152b640 1 -- 172.21.15.43:0/295909928 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f72ac06b4d0 msgr2=0x7f72ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:52.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.441+0000 7f72d152b640 1 --2- 172.21.15.43:0/295909928 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f72ac06b4d0 0x7f72ac06d990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f72bc00ab60 tx=0x7f72bc008040 comp rx=0 tx=0).stop 2024-04-25T08:56:52.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.441+0000 7f72d152b640 1 -- 172.21.15.43:0/295909928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72cc15df10 msgr2=0x7f72cc0b2150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:52.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.441+0000 7f72d152b640 1 --2- 172.21.15.43:0/295909928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72cc15df10 0x7f72cc0b2150 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f72c402ac50 tx=0x7f72c4004a40 comp rx=0 tx=0).stop 2024-04-25T08:56:52.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.441+0000 7f72d152b640 1 -- 172.21.15.43:0/295909928 shutdown_connections 2024-04-25T08:56:52.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.441+0000 7f72d152b640 1 --2- 172.21.15.43:0/295909928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f72cc15df10 0x7f72cc0b2150 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:52.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.441+0000 7f72d152b640 1 --2- 172.21.15.43:0/295909928 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f72ac06b4d0 0x7f72ac06d990 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:52.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.441+0000 7f72d152b640 1 --2- 172.21.15.43:0/295909928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f72cc0d3710 0x7f72cc0b1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:52.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.441+0000 7f72d152b640 1 --2- 172.21.15.43:0/295909928 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f72cc0d2010 0x7f72cc152aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:52.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.441+0000 7f72d152b640 1 -- 172.21.15.43:0/295909928 >> 172.21.15.43:0/295909928 conn(0x7f72cc09cd50 msgr2=0x7f72cc0d1610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:52.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.441+0000 7f72d152b640 1 -- 172.21.15.43:0/295909928 shutdown_connections 2024-04-25T08:56:52.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.441+0000 7f72d152b640 1 -- 172.21.15.43:0/295909928 wait complete. 2024-04-25T08:56:52.445 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T08:56:52.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.575+0000 7fe053929640 1 Processor -- start 2024-04-25T08:56:52.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.575+0000 7fe053929640 1 -- start start 2024-04-25T08:56:52.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.576+0000 7fe053929640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fe04c15b6c0 0x7fe04c15baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:52.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.576+0000 7fe053929640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe04c0b4ea0 0x7fe04c0d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:52.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.576+0000 7fe053929640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe04c0b37a0 con 0x7fe04c15e410 2024-04-25T08:56:52.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.576+0000 7fe053929640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe04c15fce0 con 0x7fe04c0b4ea0 2024-04-25T08:56:52.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.576+0000 7fe053929640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe04c15eab0 con 0x7fe04c15b6c0 2024-04-25T08:56:52.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.576+0000 7fe051ec6640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe04c0b4ea0 0x7fe04c0d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:52.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.576+0000 7fe050ec4640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe04c15e410 0x7fe04c15f5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49730/0 (socket says 172.21.15.43:49730) 2024-04-25T08:56:52.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.576+0000 7fe050ec4640 1 -- 172.21.15.43:0/1346996812 learned_addr learned my addr 172.21.15.43:0/1346996812 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:56:52.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.576+0000 7fe051ec6640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe04c0b4ea0 0x7fe04c0d3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57716/0 (socket says 172.21.15.43:57716) 2024-04-25T08:56:52.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.576+0000 7fe0516c5640 1 --2- 172.21.15.43:0/1346996812 >> v2:172.21.15.193:3300/0 conn(0x7fe04c15b6c0 0x7fe04c15baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:52.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.576+0000 7fe051ec6640 1 -- 172.21.15.43:0/1346996812 >> v2:172.21.15.193:3300/0 conn(0x7fe04c15b6c0 msgr2=0x7fe04c15baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:52.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.576+0000 7fe051ec6640 1 --2- 172.21.15.43:0/1346996812 >> v2:172.21.15.193:3300/0 conn(0x7fe04c15b6c0 0x7fe04c15baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:52.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.576+0000 7fe051ec6640 1 -- 172.21.15.43:0/1346996812 >> v1:172.21.15.43:6789/0 conn(0x7fe04c15e410 legacy=0x7fe04c15f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:52.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.576+0000 7fe051ec6640 1 -- 172.21.15.43:0/1346996812 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe04c15ed90 con 0x7fe04c0b4ea0 2024-04-25T08:56:52.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.576+0000 7fe043fff640 1 -- 172.21.15.43:0/1346996812 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 899776212 0 0) 0x7fe04c0b37a0 con 0x7fe04c15e410 2024-04-25T08:56:52.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.577+0000 7fe051ec6640 1 --2- 172.21.15.43:0/1346996812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe04c0b4ea0 0x7fe04c0d3760 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe04c08dfc0 tx=0x7fe04402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5d7c17f58a9e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:52.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.577+0000 7fe043fff640 1 -- 172.21.15.43:0/1346996812 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe044007740 con 0x7fe04c0b4ea0 2024-04-25T08:56:52.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.577+0000 7fe043fff640 1 -- 172.21.15.43:0/1346996812 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0440078e0 con 0x7fe04c0b4ea0 2024-04-25T08:56:52.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.577+0000 7fe043fff640 1 -- 172.21.15.43:0/1346996812 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe044007cb0 con 0x7fe04c0b4ea0 2024-04-25T08:56:52.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.578+0000 7fe053929640 1 -- 172.21.15.43:0/1346996812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe04c0b4ea0 msgr2=0x7fe04c0d3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:52.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.578+0000 7fe053929640 1 --2- 172.21.15.43:0/1346996812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe04c0b4ea0 0x7fe04c0d3760 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe04c08dfc0 tx=0x7fe04402a680 comp rx=0 tx=0).stop 2024-04-25T08:56:52.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.578+0000 7fe053929640 1 -- 172.21.15.43:0/1346996812 shutdown_connections 2024-04-25T08:56:52.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.578+0000 7fe053929640 1 --2- 172.21.15.43:0/1346996812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe04c0b4ea0 0x7fe04c0d3760 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:52.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.578+0000 7fe053929640 1 --2- 172.21.15.43:0/1346996812 >> v2:172.21.15.193:3300/0 conn(0x7fe04c15b6c0 0x7fe04c15baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:52.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.578+0000 7fe053929640 1 -- 172.21.15.43:0/1346996812 >> 172.21.15.43:0/1346996812 conn(0x7fe04c08d460 msgr2=0x7fe04c09cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:52.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.578+0000 7fe053929640 1 -- 172.21.15.43:0/1346996812 shutdown_connections 2024-04-25T08:56:52.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.578+0000 7fe053929640 1 -- 172.21.15.43:0/1346996812 wait complete. 2024-04-25T08:56:52.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.579+0000 7fe053929640 1 Processor -- start 2024-04-25T08:56:52.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.579+0000 7fe053929640 1 -- start start 2024-04-25T08:56:52.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.580+0000 7fe053929640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe04c0b4ea0 0x7fe04c159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:52.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.580+0000 7fe053929640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe04c15b6c0 0x7fe04c0b7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:52.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.580+0000 7fe053929640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe04c15e410 0x7fe04c0b8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:52.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.580+0000 7fe053929640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04c0b52f0 con 0x7fe04c15b6c0 2024-04-25T08:56:52.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.580+0000 7fe053929640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04c15f350 con 0x7fe04c0b4ea0 2024-04-25T08:56:52.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.580+0000 7fe053929640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe04c0d3ca0 con 0x7fe04c15e410 2024-04-25T08:56:52.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.580+0000 7fe050ec4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe04c15b6c0 0x7fe04c0b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:52.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.580+0000 7fe050ec4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe04c15b6c0 0x7fe04c0b7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57722/0 (socket says 172.21.15.43:57722) 2024-04-25T08:56:52.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.580+0000 7fe050ec4640 1 -- 172.21.15.43:0/3325309901 learned_addr learned my addr 172.21.15.43:0/3325309901 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:52.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.580+0000 7fe0516c5640 1 --2- 172.21.15.43:0/3325309901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe04c0b4ea0 0x7fe04c159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:52.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.580+0000 7fe051ec6640 1 --2- 172.21.15.43:0/3325309901 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe04c15e410 0x7fe04c0b8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:52.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.580+0000 7fe050ec4640 1 -- 172.21.15.43:0/3325309901 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe04c15e410 msgr2=0x7fe04c0b8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:52.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.580+0000 7fe050ec4640 1 --2- 172.21.15.43:0/3325309901 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe04c15e410 0x7fe04c0b8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:52.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.580+0000 7fe050ec4640 1 -- 172.21.15.43:0/3325309901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe04c0b4ea0 msgr2=0x7fe04c159400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:52.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.580+0000 7fe050ec4640 1 --2- 172.21.15.43:0/3325309901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe04c0b4ea0 0x7fe04c159400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:52.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.580+0000 7fe050ec4640 1 -- 172.21.15.43:0/3325309901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe04c0b7b60 con 0x7fe04c15b6c0 2024-04-25T08:56:52.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.580+0000 7fe050ec4640 1 --2- 172.21.15.43:0/3325309901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe04c15b6c0 0x7fe04c0b7180 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe03c00c460 tx=0x7fe03c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:52.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.580+0000 7fe041ffb640 1 -- 172.21.15.43:0/3325309901 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03c018070 con 0x7fe04c15b6c0 2024-04-25T08:56:52.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.580+0000 7fe041ffb640 1 -- 172.21.15.43:0/3325309901 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe03c007500 con 0x7fe04c15b6c0 2024-04-25T08:56:52.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.580+0000 7fe053929640 1 -- 172.21.15.43:0/3325309901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe04c0b7e40 con 0x7fe04c15b6c0 2024-04-25T08:56:52.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.580+0000 7fe041ffb640 1 -- 172.21.15.43:0/3325309901 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03c004a20 con 0x7fe04c15b6c0 2024-04-25T08:56:52.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.581+0000 7fe053929640 1 -- 172.21.15.43:0/3325309901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe04c0b8400 con 0x7fe04c15b6c0 2024-04-25T08:56:52.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.582+0000 7fe053929640 1 -- 172.21.15.43:0/3325309901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe04c068a40 con 0x7fe04c15b6c0 2024-04-25T08:56:52.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.582+0000 7fe041ffb640 1 -- 172.21.15.43:0/3325309901 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe03c01b070 con 0x7fe04c15b6c0 2024-04-25T08:56:52.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.583+0000 7fe041ffb640 1 --2- 172.21.15.43:0/3325309901 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe03006b4d0 0x7fe03006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:52.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.583+0000 7fe041ffb640 1 -- 172.21.15.43:0/3325309901 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5357+0+0 (secure 0 0 0) 0x7fe03c0892c0 con 0x7fe04c15b6c0 2024-04-25T08:56:52.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.583+0000 7fe0516c5640 1 --2- 172.21.15.43:0/3325309901 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe03006b4d0 0x7fe03006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:52.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.583+0000 7fe0516c5640 1 --2- 172.21.15.43:0/3325309901 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe03006b4d0 0x7fe03006d990 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe048006fd0 tx=0x7fe048008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:52.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.587+0000 7fe041ffb640 1 -- 172.21.15.43:0/3325309901 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe03c057c50 con 0x7fe04c15b6c0 2024-04-25T08:56:52.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:52.768+0000 7fe053929640 1 -- 172.21.15.43:0/3325309901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe04c0b8800 con 0x7fe04c15b6c0 2024-04-25T08:56:53.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.438+0000 7fe041ffb640 1 -- 172.21.15.43:0/3325309901 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v38) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe03c05bcf0 con 0x7fe04c15b6c0 2024-04-25T08:56:53.428 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' removed 2024-04-25T08:56:53.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.443+0000 7fe053929640 1 -- 172.21.15.43:0/3325309901 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe03006b4d0 msgr2=0x7fe03006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:53.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.443+0000 7fe053929640 1 --2- 172.21.15.43:0/3325309901 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe03006b4d0 0x7fe03006d990 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe048006fd0 tx=0x7fe048008040 comp rx=0 tx=0).stop 2024-04-25T08:56:53.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.443+0000 7fe053929640 1 -- 172.21.15.43:0/3325309901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe04c15b6c0 msgr2=0x7fe04c0b7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:53.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.443+0000 7fe053929640 1 --2- 172.21.15.43:0/3325309901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe04c15b6c0 0x7fe04c0b7180 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe03c00c460 tx=0x7fe03c00c920 comp rx=0 tx=0).stop 2024-04-25T08:56:53.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.443+0000 7fe053929640 1 -- 172.21.15.43:0/3325309901 shutdown_connections 2024-04-25T08:56:53.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.443+0000 7fe053929640 1 --2- 172.21.15.43:0/3325309901 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe04c15e410 0x7fe04c0b8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:53.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.443+0000 7fe053929640 1 --2- 172.21.15.43:0/3325309901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe04c15b6c0 0x7fe04c0b7180 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:53.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.443+0000 7fe053929640 1 --2- 172.21.15.43:0/3325309901 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe03006b4d0 0x7fe03006d990 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:53.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.443+0000 7fe053929640 1 --2- 172.21.15.43:0/3325309901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe04c0b4ea0 0x7fe04c159400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:53.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.443+0000 7fe053929640 1 -- 172.21.15.43:0/3325309901 >> 172.21.15.43:0/3325309901 conn(0x7fe04c08d460 msgr2=0x7fe04c06bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:53.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.444+0000 7fe053929640 1 -- 172.21.15.43:0/3325309901 shutdown_connections 2024-04-25T08:56:53.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.444+0000 7fe053929640 1 -- 172.21.15.43:0/3325309901 wait complete. 2024-04-25T08:56:53.451 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T08:56:53.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.570+0000 7fccb1edb640 1 Processor -- start 2024-04-25T08:56:53.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.570+0000 7fccb1edb640 1 -- start start 2024-04-25T08:56:53.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.571+0000 7fccb1edb640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fccac0b5560 0x7fccac0b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:53.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.571+0000 7fccb1edb640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fccac0d2230 0x7fccac0d26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:53.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.571+0000 7fccb1edb640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccac162610 con 0x7fccac15e090 2024-04-25T08:56:53.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.571+0000 7fccb1edb640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccac0d1da0 con 0x7fccac0d2230 2024-04-25T08:56:53.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.571+0000 7fccb1edb640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccac0d1750 con 0x7fccac0b5560 2024-04-25T08:56:53.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.571+0000 7fccab7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fccac15e090 0x7fccac161aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49742/0 (socket says 172.21.15.43:49742) 2024-04-25T08:56:53.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.571+0000 7fccab7fe640 1 -- 172.21.15.43:0/1463252212 learned_addr learned my addr 172.21.15.43:0/1463252212 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:56:53.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.571+0000 7fccabfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fccac0d2230 0x7fccac0d26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:53.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.572+0000 7fccaaffd640 1 --2- 172.21.15.43:0/1463252212 >> v2:172.21.15.193:3300/0 conn(0x7fccac0b5560 0x7fccac0b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:53.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.572+0000 7fccabfff640 1 -- 172.21.15.43:0/1463252212 >> v2:172.21.15.193:3300/0 conn(0x7fccac0b5560 msgr2=0x7fccac0b59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:53.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.572+0000 7fccabfff640 1 --2- 172.21.15.43:0/1463252212 >> v2:172.21.15.193:3300/0 conn(0x7fccac0b5560 0x7fccac0b59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:53.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.572+0000 7fccabfff640 1 -- 172.21.15.43:0/1463252212 >> v1:172.21.15.43:6789/0 conn(0x7fccac15e090 legacy=0x7fccac161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:53.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.572+0000 7fccabfff640 1 -- 172.21.15.43:0/1463252212 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccac0d1a30 con 0x7fccac0d2230 2024-04-25T08:56:53.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.572+0000 7fccaa7fc640 1 -- 172.21.15.43:0/1463252212 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3495138946 0 0) 0x7fccac162610 con 0x7fccac15e090 2024-04-25T08:56:53.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.572+0000 7fccabfff640 1 --2- 172.21.15.43:0/1463252212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccac0d2230 0x7fccac0d26c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fccac0cfe10 tx=0x7fcca402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27075ff60e3d4a71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:53.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.573+0000 7fccaa7fc640 1 -- 172.21.15.43:0/1463252212 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca4007740 con 0x7fccac0d2230 2024-04-25T08:56:53.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.573+0000 7fccaa7fc640 1 -- 172.21.15.43:0/1463252212 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcca40078e0 con 0x7fccac0d2230 2024-04-25T08:56:53.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.573+0000 7fccaa7fc640 1 -- 172.21.15.43:0/1463252212 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca4007cb0 con 0x7fccac0d2230 2024-04-25T08:56:53.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.573+0000 7fccb1edb640 1 -- 172.21.15.43:0/1463252212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccac0d2230 msgr2=0x7fccac0d26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:53.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.573+0000 7fccb1edb640 1 --2- 172.21.15.43:0/1463252212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccac0d2230 0x7fccac0d26c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fccac0cfe10 tx=0x7fcca402a710 comp rx=0 tx=0).stop 2024-04-25T08:56:53.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.573+0000 7fccb1edb640 1 -- 172.21.15.43:0/1463252212 shutdown_connections 2024-04-25T08:56:53.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.573+0000 7fccb1edb640 1 --2- 172.21.15.43:0/1463252212 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccac0d2230 0x7fccac0d26c0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:53.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.573+0000 7fccb1edb640 1 --2- 172.21.15.43:0/1463252212 >> v2:172.21.15.193:3300/0 conn(0x7fccac0b5560 0x7fccac0b59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:53.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.573+0000 7fccb1edb640 1 -- 172.21.15.43:0/1463252212 >> 172.21.15.43:0/1463252212 conn(0x7fccac0d0560 msgr2=0x7fccac0d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:53.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.573+0000 7fccb1edb640 1 -- 172.21.15.43:0/1463252212 shutdown_connections 2024-04-25T08:56:53.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.573+0000 7fccb1edb640 1 -- 172.21.15.43:0/1463252212 wait complete. 2024-04-25T08:56:53.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.574+0000 7fccb1edb640 1 Processor -- start 2024-04-25T08:56:53.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.575+0000 7fccb1edb640 1 -- start start 2024-04-25T08:56:53.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.575+0000 7fccb1edb640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fccac0b5560 0x7fccac1524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:53.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.575+0000 7fccb1edb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fccac0d2230 0x7fccac14eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:53.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.575+0000 7fccb1edb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccac15e090 0x7fccac160010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:53.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.575+0000 7fccb1edb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccac1621b0 con 0x7fccac15e090 2024-04-25T08:56:53.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.575+0000 7fccb1edb640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccac0b3280 con 0x7fccac0d2230 2024-04-25T08:56:53.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.575+0000 7fccb1edb640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccac162330 con 0x7fccac0b5560 2024-04-25T08:56:53.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.575+0000 7fccabfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccac15e090 0x7fccac160010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:53.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.575+0000 7fccabfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccac15e090 0x7fccac160010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57742/0 (socket says 172.21.15.43:57742) 2024-04-25T08:56:53.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.575+0000 7fccabfff640 1 -- 172.21.15.43:0/3842270273 learned_addr learned my addr 172.21.15.43:0/3842270273 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:53.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.576+0000 7fccaaffd640 1 --2- 172.21.15.43:0/3842270273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fccac0d2230 0x7fccac14eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:53.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.576+0000 7fccab7fe640 1 --2- 172.21.15.43:0/3842270273 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fccac0b5560 0x7fccac1524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:53.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.576+0000 7fccabfff640 1 -- 172.21.15.43:0/3842270273 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fccac0b5560 msgr2=0x7fccac1524c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:53.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.576+0000 7fccabfff640 1 --2- 172.21.15.43:0/3842270273 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fccac0b5560 0x7fccac1524c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:53.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.576+0000 7fccabfff640 1 -- 172.21.15.43:0/3842270273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fccac0d2230 msgr2=0x7fccac14eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:53.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.576+0000 7fccabfff640 1 --2- 172.21.15.43:0/3842270273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fccac0d2230 0x7fccac14eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:53.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.576+0000 7fccabfff640 1 -- 172.21.15.43:0/3842270273 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccac14fcf0 con 0x7fccac15e090 2024-04-25T08:56:53.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.576+0000 7fccabfff640 1 --2- 172.21.15.43:0/3842270273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccac15e090 0x7fccac160010 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fcca4009e50 tx=0x7fcca4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:53.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.576+0000 7fcc93fff640 1 -- 172.21.15.43:0/3842270273 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca4040070 con 0x7fccac15e090 2024-04-25T08:56:53.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.576+0000 7fcc93fff640 1 -- 172.21.15.43:0/3842270273 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcca4031bf0 con 0x7fccac15e090 2024-04-25T08:56:53.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.576+0000 7fccb1edb640 1 -- 172.21.15.43:0/3842270273 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccac14ffd0 con 0x7fccac15e090 2024-04-25T08:56:53.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.576+0000 7fcc93fff640 1 -- 172.21.15.43:0/3842270273 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca4045460 con 0x7fccac15e090 2024-04-25T08:56:53.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.576+0000 7fccb1edb640 1 -- 172.21.15.43:0/3842270273 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccac150590 con 0x7fccac15e090 2024-04-25T08:56:53.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.577+0000 7fccb1edb640 1 -- 172.21.15.43:0/3842270273 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccac06bf00 con 0x7fccac15e090 2024-04-25T08:56:53.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.578+0000 7fcc93fff640 1 -- 172.21.15.43:0/3842270273 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcca4039040 con 0x7fccac15e090 2024-04-25T08:56:53.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.578+0000 7fcc93fff640 1 --2- 172.21.15.43:0/3842270273 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcc9806b500 0x7fcc9806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:53.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.578+0000 7fcc93fff640 1 -- 172.21.15.43:0/3842270273 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fcca40ab7d0 con 0x7fccac15e090 2024-04-25T08:56:53.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.578+0000 7fccab7fe640 1 --2- 172.21.15.43:0/3842270273 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcc9806b500 0x7fcc9806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:53.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.579+0000 7fccab7fe640 1 --2- 172.21.15.43:0/3842270273 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcc9806b500 0x7fcc9806d9c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c000f60 tx=0x7fcc9c007c00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:53.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.582+0000 7fcc93fff640 1 -- 172.21.15.43:0/3842270273 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fccac06bf00 con 0x7fccac15e090 2024-04-25T08:56:53.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.762+0000 7fccb1edb640 1 -- 172.21.15.43:0/3842270273 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fccac074f60 con 0x7fccac15e090 2024-04-25T08:56:53.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.765+0000 7fcc93fff640 1 -- 172.21.15.43:0/3842270273 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcca407a950 con 0x7fccac15e090 2024-04-25T08:56:53.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.767+0000 7fccb1edb640 1 -- 172.21.15.43:0/3842270273 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcc9806b500 msgr2=0x7fcc9806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:53.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.767+0000 7fccb1edb640 1 --2- 172.21.15.43:0/3842270273 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcc9806b500 0x7fcc9806d9c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c000f60 tx=0x7fcc9c007c00 comp rx=0 tx=0).stop 2024-04-25T08:56:53.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.767+0000 7fccb1edb640 1 -- 172.21.15.43:0/3842270273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccac15e090 msgr2=0x7fccac160010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:53.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.767+0000 7fccb1edb640 1 --2- 172.21.15.43:0/3842270273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccac15e090 0x7fccac160010 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fcca4009e50 tx=0x7fcca4004a40 comp rx=0 tx=0).stop 2024-04-25T08:56:53.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.767+0000 7fccb1edb640 1 -- 172.21.15.43:0/3842270273 shutdown_connections 2024-04-25T08:56:53.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.767+0000 7fccb1edb640 1 --2- 172.21.15.43:0/3842270273 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fccac15e090 0x7fccac160010 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:53.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.767+0000 7fccb1edb640 1 --2- 172.21.15.43:0/3842270273 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcc9806b500 0x7fcc9806d9c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:53.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.767+0000 7fccb1edb640 1 --2- 172.21.15.43:0/3842270273 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fccac0d2230 0x7fccac14eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:53.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.767+0000 7fccb1edb640 1 --2- 172.21.15.43:0/3842270273 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fccac0b5560 0x7fccac1524c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:53.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.767+0000 7fccb1edb640 1 -- 172.21.15.43:0/3842270273 >> 172.21.15.43:0/3842270273 conn(0x7fccac0d0560 msgr2=0x7fccac0b7d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:53.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.767+0000 7fccb1edb640 1 -- 172.21.15.43:0/3842270273 shutdown_connections 2024-04-25T08:56:53.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.767+0000 7fccb1edb640 1 -- 172.21.15.43:0/3842270273 wait complete. 2024-04-25T08:56:53.768 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir' 2024-04-25T08:56:53.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.879+0000 7f4ad74bc640 1 Processor -- start 2024-04-25T08:56:53.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.880+0000 7f4ad74bc640 1 -- start start 2024-04-25T08:56:53.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.880+0000 7f4ad74bc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4ad00d0d90 0x7f4ad00ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:53.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.880+0000 7f4ad74bc640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f4ad00d2500 0x7f4ad00d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:53.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.880+0000 7f4ad74bc640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ad00b6c50 con 0x7f4ad00d39d0 2024-04-25T08:56:53.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.880+0000 7f4ad74bc640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ad015e010 con 0x7f4ad00d0d90 2024-04-25T08:56:53.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.880+0000 7f4ad74bc640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ad00d2020 con 0x7f4ad00d2500 2024-04-25T08:56:53.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.881+0000 7f4ad4a57640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4ad00d0d90 0x7f4ad00ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:53.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.881+0000 7f4ad4a57640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4ad00d0d90 0x7f4ad00ce2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57746/0 (socket says 172.21.15.43:57746) 2024-04-25T08:56:53.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.881+0000 7f4ad5258640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4ad00d39d0 0x7f4ad00d3db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49750/0 (socket says 172.21.15.43:49750) 2024-04-25T08:56:53.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.881+0000 7f4ad4a57640 1 -- 172.21.15.43:0/2938080118 learned_addr learned my addr 172.21.15.43:0/2938080118 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:53.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.881+0000 7f4ad4a57640 1 -- 172.21.15.43:0/2938080118 >> v2:172.21.15.193:3300/0 conn(0x7f4ad00d2500 msgr2=0x7f4ad00d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:53.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.881+0000 7f4ad4a57640 1 --2- 172.21.15.43:0/2938080118 >> v2:172.21.15.193:3300/0 conn(0x7f4ad00d2500 0x7f4ad00d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:53.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.881+0000 7f4ad4a57640 1 -- 172.21.15.43:0/2938080118 >> v1:172.21.15.43:6789/0 conn(0x7f4ad00d39d0 legacy=0x7f4ad00d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:53.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.881+0000 7f4ad4a57640 1 -- 172.21.15.43:0/2938080118 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad00cec40 con 0x7f4ad00d0d90 2024-04-25T08:56:53.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.882+0000 7f4ad4a57640 1 --2- 172.21.15.43:0/2938080118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ad00d0d90 0x7f4ad00ce2a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4ad00ce7e0 tx=0x7f4ac402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa8cf2bc8d02b51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:53.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.882+0000 7f4acffff640 1 -- 172.21.15.43:0/2938080118 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac4037070 con 0x7f4ad00d0d90 2024-04-25T08:56:53.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.882+0000 7f4acffff640 1 -- 172.21.15.43:0/2938080118 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac4032440 con 0x7f4ad00d0d90 2024-04-25T08:56:53.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.882+0000 7f4acffff640 1 -- 172.21.15.43:0/2938080118 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac4032810 con 0x7f4ad00d0d90 2024-04-25T08:56:53.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.882+0000 7f4ad74bc640 1 -- 172.21.15.43:0/2938080118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ad00d0d90 msgr2=0x7f4ad00ce2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:53.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.882+0000 7f4ad74bc640 1 --2- 172.21.15.43:0/2938080118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ad00d0d90 0x7f4ad00ce2a0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4ad00ce7e0 tx=0x7f4ac402a1e0 comp rx=0 tx=0).stop 2024-04-25T08:56:53.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.882+0000 7f4ad74bc640 1 -- 172.21.15.43:0/2938080118 shutdown_connections 2024-04-25T08:56:53.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.882+0000 7f4ad74bc640 1 --2- 172.21.15.43:0/2938080118 >> v2:172.21.15.193:3300/0 conn(0x7f4ad00d2500 0x7f4ad00d2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:53.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.882+0000 7f4ad74bc640 1 --2- 172.21.15.43:0/2938080118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ad00d0d90 0x7f4ad00ce2a0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:53.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.882+0000 7f4ad74bc640 1 -- 172.21.15.43:0/2938080118 >> 172.21.15.43:0/2938080118 conn(0x7f4ad006b000 msgr2=0x7f4ad006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:53.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.882+0000 7f4ad74bc640 1 -- 172.21.15.43:0/2938080118 shutdown_connections 2024-04-25T08:56:53.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.882+0000 7f4ad74bc640 1 -- 172.21.15.43:0/2938080118 wait complete. 2024-04-25T08:56:53.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.884+0000 7f4ad74bc640 1 Processor -- start 2024-04-25T08:56:53.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.884+0000 7f4ad74bc640 1 -- start start 2024-04-25T08:56:53.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.884+0000 7f4ad74bc640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4ad00d0d90 0x7f4ad0153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:53.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.884+0000 7f4ad74bc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ad00d2500 0x7f4ad00c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:53.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.884+0000 7f4ad74bc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ad00d39d0 0x7f4ad00b4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:53.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.884+0000 7f4ad74bc640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad00cf200 con 0x7f4ad00d39d0 2024-04-25T08:56:53.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.884+0000 7f4ad74bc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad00b2f00 con 0x7f4ad00d2500 2024-04-25T08:56:53.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.884+0000 7f4ad74bc640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ad00d2ed0 con 0x7f4ad00d0d90 2024-04-25T08:56:53.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.884+0000 7f4ad5a59640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ad00d39d0 0x7f4ad00b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:53.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.884+0000 7f4ad5a59640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ad00d39d0 0x7f4ad00b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57754/0 (socket says 172.21.15.43:57754) 2024-04-25T08:56:53.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.884+0000 7f4ad5a59640 1 -- 172.21.15.43:0/3522212533 learned_addr learned my addr 172.21.15.43:0/3522212533 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:53.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.885+0000 7f4ad4a57640 1 --2- 172.21.15.43:0/3522212533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ad00d2500 0x7f4ad00c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:53.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.885+0000 7f4ad5258640 1 --2- 172.21.15.43:0/3522212533 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4ad00d0d90 0x7f4ad0153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:53.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.885+0000 7f4ad5a59640 1 -- 172.21.15.43:0/3522212533 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4ad00d0d90 msgr2=0x7f4ad0153db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:53.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.885+0000 7f4ad5a59640 1 --2- 172.21.15.43:0/3522212533 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4ad00d0d90 0x7f4ad0153db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:53.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.885+0000 7f4ad5a59640 1 -- 172.21.15.43:0/3522212533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ad00d2500 msgr2=0x7f4ad00c7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:53.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.885+0000 7f4ad5a59640 1 --2- 172.21.15.43:0/3522212533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ad00d2500 0x7f4ad00c7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:53.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.885+0000 7f4ad5a59640 1 -- 172.21.15.43:0/3522212533 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ad01f5390 con 0x7f4ad00d39d0 2024-04-25T08:56:53.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.885+0000 7f4ad4a57640 1 --2- 172.21.15.43:0/3522212533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ad00d2500 0x7f4ad00c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:53.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.885+0000 7f4ad5258640 1 --2- 172.21.15.43:0/3522212533 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4ad00d0d90 0x7f4ad0153db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:53.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.885+0000 7f4ad5a59640 1 --2- 172.21.15.43:0/3522212533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ad00d39d0 0x7f4ad00b4670 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4ac000b970 tx=0x7f4ac000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:53.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.885+0000 7f4acdffb640 1 -- 172.21.15.43:0/3522212533 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0019070 con 0x7f4ad00d39d0 2024-04-25T08:56:53.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.885+0000 7f4acdffb640 1 -- 172.21.15.43:0/3522212533 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ac000cb60 con 0x7f4ad00d39d0 2024-04-25T08:56:53.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.885+0000 7f4ad74bc640 1 -- 172.21.15.43:0/3522212533 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ad01f5670 con 0x7f4ad00d39d0 2024-04-25T08:56:53.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.885+0000 7f4acdffb640 1 -- 172.21.15.43:0/3522212533 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0014440 con 0x7f4ad00d39d0 2024-04-25T08:56:53.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.885+0000 7f4ad74bc640 1 -- 172.21.15.43:0/3522212533 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ad01f5c30 con 0x7f4ad00d39d0 2024-04-25T08:56:53.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.886+0000 7f4ad74bc640 1 -- 172.21.15.43:0/3522212533 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ad0074ef0 con 0x7f4ad00d39d0 2024-04-25T08:56:53.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.887+0000 7f4acdffb640 1 -- 172.21.15.43:0/3522212533 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4ac001c070 con 0x7f4ad00d39d0 2024-04-25T08:56:53.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.887+0000 7f4acdffb640 1 --2- 172.21.15.43:0/3522212533 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4abc06b4d0 0x7f4abc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:53.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.887+0000 7f4acdffb640 1 -- 172.21.15.43:0/3522212533 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4ac0089c60 con 0x7f4ad00d39d0 2024-04-25T08:56:53.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.887+0000 7f4ad5258640 1 --2- 172.21.15.43:0/3522212533 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4abc06b4d0 0x7f4abc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:53.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.888+0000 7f4ad5258640 1 --2- 172.21.15.43:0/3522212533 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4abc06b4d0 0x7f4abc06d990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8000f60 tx=0x7f4ac8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:53.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:53.891+0000 7f4acdffb640 1 -- 172.21.15.43:0/3522212533 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4ac0058dc0 con 0x7f4ad00d39d0 2024-04-25T08:56:54.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.147+0000 7f4ad74bc640 1 -- 172.21.15.43:0/3522212533 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]} v 0) v1 -- 0x7f4ad00ce880 con 0x7f4ad00d39d0 2024-04-25T08:56:54.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.449+0000 7f4acdffb640 1 -- 172.21.15.43:0/3522212533 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]}]=0 v141) v1 ==== 148+0+0 (secure 0 0 0) 0x7f4ac00586c0 con 0x7f4ad00d39d0 2024-04-25T08:56:54.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.452+0000 7f4ad74bc640 1 -- 172.21.15.43:0/3522212533 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4abc06b4d0 msgr2=0x7f4abc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:54.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.452+0000 7f4ad74bc640 1 --2- 172.21.15.43:0/3522212533 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4abc06b4d0 0x7f4abc06d990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f4ac8000f60 tx=0x7f4ac8009040 comp rx=0 tx=0).stop 2024-04-25T08:56:54.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.452+0000 7f4ad74bc640 1 -- 172.21.15.43:0/3522212533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ad00d39d0 msgr2=0x7f4ad00b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:54.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.452+0000 7f4ad74bc640 1 --2- 172.21.15.43:0/3522212533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ad00d39d0 0x7f4ad00b4670 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f4ac000b970 tx=0x7f4ac000be30 comp rx=0 tx=0).stop 2024-04-25T08:56:54.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.452+0000 7f4ad74bc640 1 -- 172.21.15.43:0/3522212533 shutdown_connections 2024-04-25T08:56:54.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.452+0000 7f4ad74bc640 1 --2- 172.21.15.43:0/3522212533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ad00d39d0 0x7f4ad00b4670 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:54.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.452+0000 7f4ad74bc640 1 --2- 172.21.15.43:0/3522212533 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4abc06b4d0 0x7f4abc06d990 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:54.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.452+0000 7f4ad74bc640 1 --2- 172.21.15.43:0/3522212533 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ad00d2500 0x7f4ad00c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:54.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.452+0000 7f4ad74bc640 1 --2- 172.21.15.43:0/3522212533 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4ad00d0d90 0x7f4ad0153db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:54.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.452+0000 7f4ad74bc640 1 -- 172.21.15.43:0/3522212533 >> 172.21.15.43:0/3522212533 conn(0x7f4ad006b000 msgr2=0x7f4ad015e500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:54.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.452+0000 7f4ad74bc640 1 -- 172.21.15.43:0/3522212533 shutdown_connections 2024-04-25T08:56:54.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.452+0000 7f4ad74bc640 1 -- 172.21.15.43:0/3522212533 wait complete. 2024-04-25T08:56:54.453 INFO:tasks.cephfs_test_runner:test flag scrub_mdsdir ... ok 2024-04-25T08:56:54.453 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_checks (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T08:56:54.454 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:56:54.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.565+0000 7f4a5cc10640 1 Processor -- start 2024-04-25T08:56:54.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.565+0000 7f4a5cc10640 1 -- start start 2024-04-25T08:56:54.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.566+0000 7f4a5cc10640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4a580d0af0 0x7f4a58160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:54.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.566+0000 7f4a5cc10640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a580d2920 con 0x7f4a5815dce0 2024-04-25T08:56:54.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.566+0000 7f4a5cc10640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a580cee80 con 0x7f4a5815ef50 2024-04-25T08:56:54.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.566+0000 7f4a5cc10640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a580cec60 con 0x7f4a580d0af0 2024-04-25T08:56:54.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.566+0000 7f4a56d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4a580d0af0 0x7f4a58160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:54.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.566+0000 7f4a56d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4a580d0af0 0x7f4a58160980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57756/0 (socket says 172.21.15.43:57756) 2024-04-25T08:56:54.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.566+0000 7f4a5659c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4a5815dce0 0x7f4a580ce550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49760/0 (socket says 172.21.15.43:49760) 2024-04-25T08:56:54.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.566+0000 7f4a56d9d640 1 -- 172.21.15.43:0/2583300894 learned_addr learned my addr 172.21.15.43:0/2583300894 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:54.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.566+0000 7f4a56d9d640 1 -- 172.21.15.43:0/2583300894 >> v1:172.21.15.193:6789/0 conn(0x7f4a5815ef50 legacy=0x7f4a580d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:54.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.566+0000 7f4a56d9d640 1 -- 172.21.15.43:0/2583300894 >> v1:172.21.15.43:6789/0 conn(0x7f4a5815dce0 legacy=0x7f4a580ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:54.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.566+0000 7f4a5559a640 1 -- 172.21.15.43:0/2583300894 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3966983570 0 0) 0x7f4a580d2920 con 0x7f4a5815dce0 2024-04-25T08:56:54.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.566+0000 7f4a56d9d640 1 -- 172.21.15.43:0/2583300894 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a580cf160 con 0x7f4a580d0af0 2024-04-25T08:56:54.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.567+0000 7f4a56d9d640 1 --2- 172.21.15.43:0/2583300894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a580d0af0 0x7f4a58160980 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4a48009e50 tx=0x7f4a4802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6560c462d278dc34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:54.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.567+0000 7f4a5559a640 1 -- 172.21.15.43:0/2583300894 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a48036070 con 0x7f4a580d0af0 2024-04-25T08:56:54.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.567+0000 7f4a5559a640 1 -- 172.21.15.43:0/2583300894 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a48031600 con 0x7f4a580d0af0 2024-04-25T08:56:54.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.567+0000 7f4a5559a640 1 -- 172.21.15.43:0/2583300894 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a480319d0 con 0x7f4a580d0af0 2024-04-25T08:56:54.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.568+0000 7f4a5cc10640 1 -- 172.21.15.43:0/2583300894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a580d0af0 msgr2=0x7f4a58160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:54.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.568+0000 7f4a5cc10640 1 --2- 172.21.15.43:0/2583300894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a580d0af0 0x7f4a58160980 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4a48009e50 tx=0x7f4a4802a730 comp rx=0 tx=0).stop 2024-04-25T08:56:54.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.568+0000 7f4a5cc10640 1 -- 172.21.15.43:0/2583300894 shutdown_connections 2024-04-25T08:56:54.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.568+0000 7f4a5cc10640 1 --2- 172.21.15.43:0/2583300894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a580d0af0 0x7f4a58160980 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:54.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.568+0000 7f4a5cc10640 1 -- 172.21.15.43:0/2583300894 >> 172.21.15.43:0/2583300894 conn(0x7f4a5809cd00 msgr2=0x7f4a580d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:54.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.568+0000 7f4a5cc10640 1 -- 172.21.15.43:0/2583300894 shutdown_connections 2024-04-25T08:56:54.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.568+0000 7f4a5cc10640 1 -- 172.21.15.43:0/2583300894 wait complete. 2024-04-25T08:56:54.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.569+0000 7f4a5cc10640 1 Processor -- start 2024-04-25T08:56:54.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.569+0000 7f4a5cc10640 1 -- start start 2024-04-25T08:56:54.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.570+0000 7f4a5cc10640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a580d0af0 0x7f4a58158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:54.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.570+0000 7f4a5cc10640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a5815dce0 0x7f4a581554d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:54.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.570+0000 7f4a5cc10640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4a5815ef50 0x7f4a580b6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:54.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.570+0000 7f4a5cc10640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5815f6b0 con 0x7f4a580d0af0 2024-04-25T08:56:54.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.570+0000 7f4a5cc10640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a580cf720 con 0x7f4a5815dce0 2024-04-25T08:56:54.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.570+0000 7f4a5cc10640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5815e0c0 con 0x7f4a5815ef50 2024-04-25T08:56:54.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.570+0000 7f4a5659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a580d0af0 0x7f4a58158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:54.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.570+0000 7f4a5659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a580d0af0 0x7f4a58158390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57770/0 (socket says 172.21.15.43:57770) 2024-04-25T08:56:54.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.570+0000 7f4a5659c640 1 -- 172.21.15.43:0/1678331825 learned_addr learned my addr 172.21.15.43:0/1678331825 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:54.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.570+0000 7f4a5659c640 1 -- 172.21.15.43:0/1678331825 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4a5815ef50 msgr2=0x7f4a580b6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:54.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.570+0000 7f4a5659c640 1 --2- 172.21.15.43:0/1678331825 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4a5815ef50 0x7f4a580b6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:54.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.570+0000 7f4a5659c640 1 -- 172.21.15.43:0/1678331825 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a5815dce0 msgr2=0x7f4a581554d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:54.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.570+0000 7f4a5659c640 1 --2- 172.21.15.43:0/1678331825 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a5815dce0 0x7f4a581554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:54.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.570+0000 7f4a5659c640 1 -- 172.21.15.43:0/1678331825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a580b76e0 con 0x7f4a580d0af0 2024-04-25T08:56:54.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.571+0000 7f4a5659c640 1 --2- 172.21.15.43:0/1678331825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a580d0af0 0x7f4a58158390 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4a5000c490 tx=0x7f4a5000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:54.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.571+0000 7f4a4effd640 1 -- 172.21.15.43:0/1678331825 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a500074e0 con 0x7f4a580d0af0 2024-04-25T08:56:54.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.571+0000 7f4a4effd640 1 -- 172.21.15.43:0/1678331825 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a500047e0 con 0x7f4a580d0af0 2024-04-25T08:56:54.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.571+0000 7f4a5cc10640 1 -- 172.21.15.43:0/1678331825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a580b79c0 con 0x7f4a580d0af0 2024-04-25T08:56:54.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.571+0000 7f4a4effd640 1 -- 172.21.15.43:0/1678331825 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a50004bd0 con 0x7f4a580d0af0 2024-04-25T08:56:54.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.571+0000 7f4a5cc10640 1 -- 172.21.15.43:0/1678331825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a580b7f80 con 0x7f4a580d0af0 2024-04-25T08:56:54.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.572+0000 7f4a5cc10640 1 -- 172.21.15.43:0/1678331825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a580680f0 con 0x7f4a580d0af0 2024-04-25T08:56:54.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.572+0000 7f4a4effd640 1 -- 172.21.15.43:0/1678331825 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4a5001a020 con 0x7f4a580d0af0 2024-04-25T08:56:54.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.573+0000 7f4a4effd640 1 --2- 172.21.15.43:0/1678331825 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4a3806b500 0x7f4a3806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:54.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.573+0000 7f4a4effd640 1 -- 172.21.15.43:0/1678331825 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4a500896f0 con 0x7f4a580d0af0 2024-04-25T08:56:54.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.573+0000 7f4a55d9b640 1 --2- 172.21.15.43:0/1678331825 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4a3806b500 0x7f4a3806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:54.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.573+0000 7f4a55d9b640 1 --2- 172.21.15.43:0/1678331825 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4a3806b500 0x7f4a3806d9c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4a40000f60 tx=0x7f4a400079b0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:54.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.577+0000 7f4a4effd640 1 -- 172.21.15.43:0/1678331825 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4a50058880 con 0x7f4a580d0af0 2024-04-25T08:56:54.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.758+0000 7f4a5cc10640 1 -- 172.21.15.43:0/1678331825 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4a58074ef0 con 0x7f4a580d0af0 2024-04-25T08:56:54.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.758+0000 7f4a4effd640 1 -- 172.21.15.43:0/1678331825 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+12982 (secure 0 0 0) 0x7f4a50058180 con 0x7f4a580d0af0 2024-04-25T08:56:54.748 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:56:54.748 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":38,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:56:53.438365+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6835/242762542":"2024-04-26T08:56:50.082445+0000","172.21.15.43:6833/242762542":"2024-04-26T08:56:50.082445+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:56:54.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.760+0000 7f4a5cc10640 1 -- 172.21.15.43:0/1678331825 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4a3806b500 msgr2=0x7f4a3806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:54.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.760+0000 7f4a5cc10640 1 --2- 172.21.15.43:0/1678331825 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4a3806b500 0x7f4a3806d9c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4a40000f60 tx=0x7f4a400079b0 comp rx=0 tx=0).stop 2024-04-25T08:56:54.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.760+0000 7f4a5cc10640 1 -- 172.21.15.43:0/1678331825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a580d0af0 msgr2=0x7f4a58158390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:54.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.760+0000 7f4a5cc10640 1 --2- 172.21.15.43:0/1678331825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a580d0af0 0x7f4a58158390 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4a5000c490 tx=0x7f4a5000c950 comp rx=0 tx=0).stop 2024-04-25T08:56:54.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.760+0000 7f4a5cc10640 1 -- 172.21.15.43:0/1678331825 shutdown_connections 2024-04-25T08:56:54.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.760+0000 7f4a5cc10640 1 --2- 172.21.15.43:0/1678331825 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4a5815ef50 0x7f4a580b6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:54.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.760+0000 7f4a5cc10640 1 --2- 172.21.15.43:0/1678331825 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4a3806b500 0x7f4a3806d9c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:54.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.760+0000 7f4a5cc10640 1 --2- 172.21.15.43:0/1678331825 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a5815dce0 0x7f4a581554d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:54.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.760+0000 7f4a5cc10640 1 --2- 172.21.15.43:0/1678331825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a580d0af0 0x7f4a58158390 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:54.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.760+0000 7f4a5cc10640 1 -- 172.21.15.43:0/1678331825 >> 172.21.15.43:0/1678331825 conn(0x7f4a5809cd00 msgr2=0x7f4a580c9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:54.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.761+0000 7f4a5cc10640 1 -- 172.21.15.43:0/1678331825 shutdown_connections 2024-04-25T08:56:54.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.761+0000 7f4a5cc10640 1 -- 172.21.15.43:0/1678331825 wait complete. 2024-04-25T08:56:54.761 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:56:54.762 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:56:54.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.877+0000 7f37d097e640 1 Processor -- start 2024-04-25T08:56:54.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.877+0000 7f37d097e640 1 -- start start 2024-04-25T08:56:54.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.877+0000 7f37d097e640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f37cc15dd60 0x7f37cc0d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:54.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.877+0000 7f37d097e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f37cc0d0aa0 0x7f37cc06bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:54.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.877+0000 7f37d097e640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37cc15e140 con 0x7f37cc0d1fa0 2024-04-25T08:56:54.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.877+0000 7f37d097e640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37cc0d27d0 con 0x7f37cc15dd60 2024-04-25T08:56:54.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.877+0000 7f37d097e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37cc06b1e0 con 0x7f37cc0d0aa0 2024-04-25T08:56:54.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.878+0000 7f37ca59c640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f37cc15dd60 0x7f37cc0d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:54.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.878+0000 7f37cad9d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f37cc0d0aa0 0x7f37cc06bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:54.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.878+0000 7f37c3fff640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f37cc0d1fa0 0x7f37cc0d0390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:54620/0 (socket says 172.21.15.43:54620) 2024-04-25T08:56:54.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.878+0000 7f37c3fff640 1 -- 172.21.15.43:0/1246939421 learned_addr learned my addr 172.21.15.43:0/1246939421 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:56:54.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.878+0000 7f37c9d9b640 1 -- 172.21.15.43:0/1246939421 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 922840540 0 0) 0x7f37cc15e140 con 0x7f37cc0d1fa0 2024-04-25T08:56:54.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.878+0000 7f37c9d9b640 1 -- 172.21.15.43:0/1246939421 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37b8003520 con 0x7f37cc0d1fa0 2024-04-25T08:56:54.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.878+0000 7f37cad9d640 1 -- 172.21.15.43:0/1246939421 >> v2:172.21.15.193:3300/0 conn(0x7f37cc15dd60 msgr2=0x7f37cc0d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:54.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.878+0000 7f37cad9d640 1 --2- 172.21.15.43:0/1246939421 >> v2:172.21.15.193:3300/0 conn(0x7f37cc15dd60 0x7f37cc0d1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:54.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.878+0000 7f37cad9d640 1 -- 172.21.15.43:0/1246939421 >> v1:172.21.15.193:6789/0 conn(0x7f37cc0d1fa0 legacy=0x7f37cc0d0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:54.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.878+0000 7f37cad9d640 1 -- 172.21.15.43:0/1246939421 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37cc06b4c0 con 0x7f37cc0d0aa0 2024-04-25T08:56:54.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.878+0000 7f37ca59c640 1 --2- 172.21.15.43:0/1246939421 >> v2:172.21.15.193:3300/0 conn(0x7f37cc15dd60 0x7f37cc0d1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:56:54.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.879+0000 7f37cad9d640 1 --2- 172.21.15.43:0/1246939421 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f37cc0d0aa0 0x7f37cc06bcc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f37c4009d90 tx=0x7f37c402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=330d7e2ea978fc9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:54.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.879+0000 7f37c9d9b640 1 -- 172.21.15.43:0/1246939421 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c4007740 con 0x7f37cc0d0aa0 2024-04-25T08:56:54.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.879+0000 7f37c9d9b640 1 -- 172.21.15.43:0/1246939421 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37c40078e0 con 0x7f37cc0d0aa0 2024-04-25T08:56:54.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.879+0000 7f37c9d9b640 1 -- 172.21.15.43:0/1246939421 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c4007c90 con 0x7f37cc0d0aa0 2024-04-25T08:56:54.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.880+0000 7f37d097e640 1 -- 172.21.15.43:0/1246939421 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f37cc0d0aa0 msgr2=0x7f37cc06bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:54.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.880+0000 7f37d097e640 1 --2- 172.21.15.43:0/1246939421 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f37cc0d0aa0 0x7f37cc06bcc0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f37c4009d90 tx=0x7f37c402a660 comp rx=0 tx=0).stop 2024-04-25T08:56:54.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.880+0000 7f37d097e640 1 -- 172.21.15.43:0/1246939421 shutdown_connections 2024-04-25T08:56:54.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.880+0000 7f37d097e640 1 --2- 172.21.15.43:0/1246939421 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f37cc0d0aa0 0x7f37cc06bcc0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:54.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.880+0000 7f37d097e640 1 --2- 172.21.15.43:0/1246939421 >> v2:172.21.15.193:3300/0 conn(0x7f37cc15dd60 0x7f37cc0d1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:54.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.880+0000 7f37d097e640 1 -- 172.21.15.43:0/1246939421 >> 172.21.15.43:0/1246939421 conn(0x7f37cc0d39c0 msgr2=0x7f37cc0d3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:54.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.880+0000 7f37d097e640 1 -- 172.21.15.43:0/1246939421 shutdown_connections 2024-04-25T08:56:54.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.880+0000 7f37d097e640 1 -- 172.21.15.43:0/1246939421 wait complete. 2024-04-25T08:56:54.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.881+0000 7f37d097e640 1 Processor -- start 2024-04-25T08:56:54.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.881+0000 7f37d097e640 1 -- start start 2024-04-25T08:56:54.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.882+0000 7f37d097e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37cc0d0aa0 0x7f37cc0b6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:54.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.882+0000 7f37d097e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f37cc0d1fa0 0x7f37cc1f6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:54.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.882+0000 7f37d097e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37cc15dd60 0x7f37cc1f6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:54.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.882+0000 7f37d097e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37cc0d25b0 con 0x7f37cc0d0aa0 2024-04-25T08:56:54.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.882+0000 7f37d097e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37cc0d2430 con 0x7f37cc15dd60 2024-04-25T08:56:54.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.882+0000 7f37d097e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37cc06c200 con 0x7f37cc0d1fa0 2024-04-25T08:56:54.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.882+0000 7f37ca59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37cc0d0aa0 0x7f37cc0b6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:54.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.882+0000 7f37ca59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37cc0d0aa0 0x7f37cc0b6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57782/0 (socket says 172.21.15.43:57782) 2024-04-25T08:56:54.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.882+0000 7f37ca59c640 1 -- 172.21.15.43:0/3020092072 learned_addr learned my addr 172.21.15.43:0/3020092072 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:54.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.882+0000 7f37c3fff640 1 --2- 172.21.15.43:0/3020092072 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f37cc0d1fa0 0x7f37cc1f6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:54.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.882+0000 7f37cad9d640 1 --2- 172.21.15.43:0/3020092072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37cc15dd60 0x7f37cc1f6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:54.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.882+0000 7f37ca59c640 1 -- 172.21.15.43:0/3020092072 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f37cc0d1fa0 msgr2=0x7f37cc1f6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:54.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.882+0000 7f37ca59c640 1 --2- 172.21.15.43:0/3020092072 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f37cc0d1fa0 0x7f37cc1f6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:54.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.882+0000 7f37ca59c640 1 -- 172.21.15.43:0/3020092072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37cc15dd60 msgr2=0x7f37cc1f6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:54.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.882+0000 7f37ca59c640 1 --2- 172.21.15.43:0/3020092072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37cc15dd60 0x7f37cc1f6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:54.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.882+0000 7f37ca59c640 1 -- 172.21.15.43:0/3020092072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37cc0b9060 con 0x7f37cc0d0aa0 2024-04-25T08:56:54.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.883+0000 7f37ca59c640 1 --2- 172.21.15.43:0/3020092072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37cc0d0aa0 0x7f37cc0b6810 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f37bc00e960 tx=0x7f37bc00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:54.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.883+0000 7f37c2ffd640 1 -- 172.21.15.43:0/3020092072 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc017070 con 0x7f37cc0d0aa0 2024-04-25T08:56:54.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.883+0000 7f37c2ffd640 1 -- 172.21.15.43:0/3020092072 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37bc00ca00 con 0x7f37cc0d0aa0 2024-04-25T08:56:54.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.883+0000 7f37d097e640 1 -- 172.21.15.43:0/3020092072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37cc0b9340 con 0x7f37cc0d0aa0 2024-04-25T08:56:54.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.883+0000 7f37c2ffd640 1 -- 172.21.15.43:0/3020092072 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc00cdd0 con 0x7f37cc0d0aa0 2024-04-25T08:56:54.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.883+0000 7f37d097e640 1 -- 172.21.15.43:0/3020092072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37cc0b43b0 con 0x7f37cc0d0aa0 2024-04-25T08:56:54.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.884+0000 7f37d097e640 1 -- 172.21.15.43:0/3020092072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37cc074f60 con 0x7f37cc0d0aa0 2024-04-25T08:56:54.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.884+0000 7f37c2ffd640 1 -- 172.21.15.43:0/3020092072 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f37bc01a070 con 0x7f37cc0d0aa0 2024-04-25T08:56:54.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.885+0000 7f37c2ffd640 1 --2- 172.21.15.43:0/3020092072 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f37ac06b4d0 0x7f37ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:54.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.885+0000 7f37c2ffd640 1 -- 172.21.15.43:0/3020092072 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f37bc08a160 con 0x7f37cc0d0aa0 2024-04-25T08:56:54.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.885+0000 7f37c3fff640 1 --2- 172.21.15.43:0/3020092072 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f37ac06b4d0 0x7f37ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:54.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.885+0000 7f37c3fff640 1 --2- 172.21.15.43:0/3020092072 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f37ac06b4d0 0x7f37ac06d990 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f37cc161430 tx=0x7f37b4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:54.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:54.889+0000 7f37c2ffd640 1 -- 172.21.15.43:0/3020092072 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f37bc058bf0 con 0x7f37cc0d0aa0 2024-04-25T08:56:55.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.078+0000 7f37d097e640 1 -- 172.21.15.43:0/3020092072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f37cc0b15c0 con 0x7f37cc0d0aa0 2024-04-25T08:56:55.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.078+0000 7f37c2ffd640 1 -- 172.21.15.43:0/3020092072 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) v1 ==== 93+0+10 (secure 0 0 0) 0x7f37bc05cc90 con 0x7f37cc0d0aa0 2024-04-25T08:56:55.068 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T08:56:55.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.080+0000 7f37d097e640 1 -- 172.21.15.43:0/3020092072 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f37ac06b4d0 msgr2=0x7f37ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:55.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.080+0000 7f37d097e640 1 --2- 172.21.15.43:0/3020092072 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f37ac06b4d0 0x7f37ac06d990 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f37cc161430 tx=0x7f37b4009040 comp rx=0 tx=0).stop 2024-04-25T08:56:55.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.080+0000 7f37d097e640 1 -- 172.21.15.43:0/3020092072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37cc0d0aa0 msgr2=0x7f37cc0b6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:55.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.080+0000 7f37d097e640 1 --2- 172.21.15.43:0/3020092072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37cc0d0aa0 0x7f37cc0b6810 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f37bc00e960 tx=0x7f37bc00ee20 comp rx=0 tx=0).stop 2024-04-25T08:56:55.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.080+0000 7f37d097e640 1 -- 172.21.15.43:0/3020092072 shutdown_connections 2024-04-25T08:56:55.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.080+0000 7f37d097e640 1 --2- 172.21.15.43:0/3020092072 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f37ac06b4d0 0x7f37ac06d990 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:55.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.080+0000 7f37d097e640 1 --2- 172.21.15.43:0/3020092072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37cc15dd60 0x7f37cc1f6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:55.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.080+0000 7f37d097e640 1 --2- 172.21.15.43:0/3020092072 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f37cc0d1fa0 0x7f37cc1f6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:55.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.080+0000 7f37d097e640 1 --2- 172.21.15.43:0/3020092072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37cc0d0aa0 0x7f37cc0b6810 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:55.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.080+0000 7f37d097e640 1 -- 172.21.15.43:0/3020092072 >> 172.21.15.43:0/3020092072 conn(0x7f37cc0d39c0 msgr2=0x7f37cc0c7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:55.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.080+0000 7f37d097e640 1 -- 172.21.15.43:0/3020092072 shutdown_connections 2024-04-25T08:56:55.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.080+0000 7f37d097e640 1 -- 172.21.15.43:0/3020092072 wait complete. 2024-04-25T08:56:55.081 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks' 2024-04-25T08:56:55.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.192+0000 7f9229913640 1 Processor -- start 2024-04-25T08:56:55.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.192+0000 7f9229913640 1 -- start start 2024-04-25T08:56:55.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.193+0000 7f9229913640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f922406c510 0x7f922406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:55.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.193+0000 7f9229913640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f922415df70 0x7f92240ce070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:55.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.193+0000 7f9229913640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f92240b0870 0x7f92240ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:55.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.193+0000 7f9229913640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92240d0400 con 0x7f92240b0870 2024-04-25T08:56:55.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.193+0000 7f9229913640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92240cbee0 con 0x7f922415df70 2024-04-25T08:56:55.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.193+0000 7f9229913640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f922406ce30 con 0x7f922406c510 2024-04-25T08:56:55.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.193+0000 7f92237fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f92240b0870 0x7f92240ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:55.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.193+0000 7f92237fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f92240b0870 0x7f92240ce5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57798/0 (socket says 172.21.15.43:57798) 2024-04-25T08:56:55.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.193+0000 7f92237fe640 1 -- 172.21.15.43:0/3680787141 learned_addr learned my addr 172.21.15.43:0/3680787141 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:55.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.194+0000 7f92237fe640 1 -- 172.21.15.43:0/3680787141 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f922406c510 msgr2=0x7f922406c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:55.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.194+0000 7f92237fe640 1 --2- 172.21.15.43:0/3680787141 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f922406c510 0x7f922406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:55.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.194+0000 7f92237fe640 1 -- 172.21.15.43:0/3680787141 >> v2:172.21.15.193:3300/0 conn(0x7f922415df70 msgr2=0x7f92240ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:55.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.194+0000 7f92227fc640 1 --2- 172.21.15.43:0/3680787141 >> v2:172.21.15.193:3300/0 conn(0x7f922415df70 0x7f92240ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:55.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.194+0000 7f92237fe640 1 --2- 172.21.15.43:0/3680787141 >> v2:172.21.15.193:3300/0 conn(0x7f922415df70 0x7f92240ce070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:55.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.194+0000 7f92237fe640 1 -- 172.21.15.43:0/3680787141 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92240d0650 con 0x7f92240b0870 2024-04-25T08:56:55.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.194+0000 7f92237fe640 1 --2- 172.21.15.43:0/3680787141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92240b0870 0x7f92240ce5b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f92240b4360 tx=0x7f921402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33fd27a11275902a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:55.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.195+0000 7f9221ffb640 1 -- 172.21.15.43:0/3680787141 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9214037070 con 0x7f92240b0870 2024-04-25T08:56:55.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.195+0000 7f9221ffb640 1 -- 172.21.15.43:0/3680787141 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9214032440 con 0x7f92240b0870 2024-04-25T08:56:55.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.195+0000 7f9221ffb640 1 -- 172.21.15.43:0/3680787141 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9214032810 con 0x7f92240b0870 2024-04-25T08:56:55.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.195+0000 7f9229913640 1 -- 172.21.15.43:0/3680787141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92240b0870 msgr2=0x7f92240ce5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:55.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.195+0000 7f9229913640 1 --2- 172.21.15.43:0/3680787141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92240b0870 0x7f92240ce5b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f92240b4360 tx=0x7f921402a1e0 comp rx=0 tx=0).stop 2024-04-25T08:56:55.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.195+0000 7f9229913640 1 -- 172.21.15.43:0/3680787141 shutdown_connections 2024-04-25T08:56:55.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.195+0000 7f9229913640 1 --2- 172.21.15.43:0/3680787141 >> v2:172.21.15.193:3300/0 conn(0x7f922415df70 0x7f92240ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:55.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.195+0000 7f9229913640 1 --2- 172.21.15.43:0/3680787141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92240b0870 0x7f92240ce5b0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:55.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.195+0000 7f9229913640 1 --2- 172.21.15.43:0/3680787141 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f922406c510 0x7f922406c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:55.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.195+0000 7f9229913640 1 -- 172.21.15.43:0/3680787141 >> 172.21.15.43:0/3680787141 conn(0x7f92240d17f0 msgr2=0x7f92240d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:55.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.195+0000 7f9229913640 1 -- 172.21.15.43:0/3680787141 shutdown_connections 2024-04-25T08:56:55.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.195+0000 7f9229913640 1 -- 172.21.15.43:0/3680787141 wait complete. 2024-04-25T08:56:55.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.196+0000 7f9229913640 1 Processor -- start 2024-04-25T08:56:55.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.197+0000 7f9229913640 1 -- start start 2024-04-25T08:56:55.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.197+0000 7f9229913640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f922406c510 0x7f92240b6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:55.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.197+0000 7f9229913640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f92240b0870 0x7f92241f2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:55.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.197+0000 7f9229913640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f922415df70 0x7f92240c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:55.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.197+0000 7f9229913640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92240d0c10 con 0x7f922406c510 2024-04-25T08:56:55.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.197+0000 7f9229913640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92240cc150 con 0x7f922415df70 2024-04-25T08:56:55.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.197+0000 7f9229913640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f922406bd10 con 0x7f92240b0870 2024-04-25T08:56:55.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.197+0000 7f9222ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f922406c510 0x7f92240b6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:55.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.197+0000 7f9222ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f922406c510 0x7f92240b6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57808/0 (socket says 172.21.15.43:57808) 2024-04-25T08:56:55.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.197+0000 7f9222ffd640 1 -- 172.21.15.43:0/633468285 learned_addr learned my addr 172.21.15.43:0/633468285 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:55.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.198+0000 7f92237fe640 1 --2- 172.21.15.43:0/633468285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f922415df70 0x7f92240c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:55.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.198+0000 7f92227fc640 1 --2- 172.21.15.43:0/633468285 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f92240b0870 0x7f92241f2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:55.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.198+0000 7f9222ffd640 1 -- 172.21.15.43:0/633468285 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f92240b0870 msgr2=0x7f92241f2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:55.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.198+0000 7f9222ffd640 1 --2- 172.21.15.43:0/633468285 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f92240b0870 0x7f92241f2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:55.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.198+0000 7f9222ffd640 1 -- 172.21.15.43:0/633468285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f922415df70 msgr2=0x7f92240c6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:55.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.198+0000 7f9222ffd640 1 --2- 172.21.15.43:0/633468285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f922415df70 0x7f92240c6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:55.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.198+0000 7f9222ffd640 1 -- 172.21.15.43:0/633468285 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92241f3210 con 0x7f922406c510 2024-04-25T08:56:55.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.198+0000 7f9222ffd640 1 --2- 172.21.15.43:0/633468285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f922406c510 0x7f92240b6ed0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f921c00ba50 tx=0x7f921c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:55.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.198+0000 7f920bfff640 1 -- 172.21.15.43:0/633468285 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921c019070 con 0x7f922406c510 2024-04-25T08:56:55.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.198+0000 7f920bfff640 1 -- 172.21.15.43:0/633468285 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f921c00cc00 con 0x7f922406c510 2024-04-25T08:56:55.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.198+0000 7f9229913640 1 -- 172.21.15.43:0/633468285 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92241f6eb0 con 0x7f922406c510 2024-04-25T08:56:55.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.198+0000 7f920bfff640 1 -- 172.21.15.43:0/633468285 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f921c014440 con 0x7f922406c510 2024-04-25T08:56:55.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.198+0000 7f9229913640 1 -- 172.21.15.43:0/633468285 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92241f7470 con 0x7f922406c510 2024-04-25T08:56:55.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.199+0000 7f9229913640 1 -- 172.21.15.43:0/633468285 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9224074ef0 con 0x7f922406c510 2024-04-25T08:56:55.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.200+0000 7f920bfff640 1 -- 172.21.15.43:0/633468285 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f921c01c070 con 0x7f922406c510 2024-04-25T08:56:55.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.200+0000 7f920bfff640 1 --2- 172.21.15.43:0/633468285 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f921006b4d0 0x7f921006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:55.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.200+0000 7f920bfff640 1 -- 172.21.15.43:0/633468285 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f921c089c30 con 0x7f922406c510 2024-04-25T08:56:55.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.200+0000 7f92227fc640 1 --2- 172.21.15.43:0/633468285 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f921006b4d0 0x7f921006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:55.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.201+0000 7f92227fc640 1 --2- 172.21.15.43:0/633468285 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f921006b4d0 0x7f921006d990 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f92240b3d70 tx=0x7f9218008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:55.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.204+0000 7f920bfff640 1 -- 172.21.15.43:0/633468285 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f921c058760 con 0x7f922406c510 2024-04-25T08:56:55.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:55.453+0000 7f9229913640 1 -- 172.21.15.43:0/633468285 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]} v 0) v1 -- 0x7f92240743d0 con 0x7f922406c510 2024-04-25T08:56:56.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.455+0000 7f920bfff640 1 -- 172.21.15.43:0/633468285 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]}]=0 v143) v1 ==== 146+0+0 (secure 0 0 0) 0x7f921c05c800 con 0x7f922406c510 2024-04-25T08:56:56.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.458+0000 7f9229913640 1 -- 172.21.15.43:0/633468285 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f921006b4d0 msgr2=0x7f921006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:56.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.458+0000 7f9229913640 1 --2- 172.21.15.43:0/633468285 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f921006b4d0 0x7f921006d990 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f92240b3d70 tx=0x7f9218008040 comp rx=0 tx=0).stop 2024-04-25T08:56:56.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.458+0000 7f9229913640 1 -- 172.21.15.43:0/633468285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f922406c510 msgr2=0x7f92240b6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:56.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.458+0000 7f9229913640 1 --2- 172.21.15.43:0/633468285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f922406c510 0x7f92240b6ed0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f921c00ba50 tx=0x7f921c00bf10 comp rx=0 tx=0).stop 2024-04-25T08:56:56.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.458+0000 7f9229913640 1 -- 172.21.15.43:0/633468285 shutdown_connections 2024-04-25T08:56:56.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.458+0000 7f9229913640 1 --2- 172.21.15.43:0/633468285 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f921006b4d0 0x7f921006d990 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:56.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.458+0000 7f9229913640 1 --2- 172.21.15.43:0/633468285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f922415df70 0x7f92240c6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:56.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.458+0000 7f9229913640 1 --2- 172.21.15.43:0/633468285 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f92240b0870 0x7f92241f2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:56.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.458+0000 7f9229913640 1 --2- 172.21.15.43:0/633468285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f922406c510 0x7f92240b6ed0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:56.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.458+0000 7f9229913640 1 -- 172.21.15.43:0/633468285 >> 172.21.15.43:0/633468285 conn(0x7f92240d17f0 msgr2=0x7f922406d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:56.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.458+0000 7f9229913640 1 -- 172.21.15.43:0/633468285 shutdown_connections 2024-04-25T08:56:56.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.458+0000 7f9229913640 1 -- 172.21.15.43:0/633468285 wait complete. 2024-04-25T08:56:56.462 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T08:56:56.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.582+0000 7fa156529640 1 Processor -- start 2024-04-25T08:56:56.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.582+0000 7fa156529640 1 -- start start 2024-04-25T08:56:56.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.583+0000 7fa156529640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa15015a960 0x7fa1500b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:56.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.583+0000 7fa156529640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa1500b3ed0 0x7fa1500ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:56.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.583+0000 7fa156529640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1500af0a0 con 0x7fa1500aec70 2024-04-25T08:56:56.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.583+0000 7fa156529640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa150071340 con 0x7fa1500b3ed0 2024-04-25T08:56:56.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.583+0000 7fa156529640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa15015b110 con 0x7fa15015a960 2024-04-25T08:56:56.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.583+0000 7fa14f7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa1500b3ed0 0x7fa1500ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:56.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.583+0000 7fa14f7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa1500b3ed0 0x7fa1500ae730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57820/0 (socket says 172.21.15.43:57820) 2024-04-25T08:56:56.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.583+0000 7fa14f7fe640 1 -- 172.21.15.43:0/139639675 learned_addr learned my addr 172.21.15.43:0/139639675 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:56.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.583+0000 7fa14ffff640 1 --2- 172.21.15.43:0/139639675 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa15015a960 0x7fa1500b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:56.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.583+0000 7fa14f7fe640 1 -- 172.21.15.43:0/139639675 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa15015a960 msgr2=0x7fa1500b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:56.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.583+0000 7fa14f7fe640 1 --2- 172.21.15.43:0/139639675 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa15015a960 0x7fa1500b3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:56.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.583+0000 7fa14f7fe640 1 -- 172.21.15.43:0/139639675 >> v1:172.21.15.193:6789/0 conn(0x7fa1500aec70 legacy=0x7fa15006d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:56.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.584+0000 7fa14f7fe640 1 -- 172.21.15.43:0/139639675 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa15015b3f0 con 0x7fa1500b3ed0 2024-04-25T08:56:56.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.584+0000 7fa14f7fe640 1 --2- 172.21.15.43:0/139639675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1500b3ed0 0x7fa1500ae730 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fa14000bb10 tx=0x7fa14002c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee53f4ccda8d70dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:56.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.584+0000 7fa14effd640 1 -- 172.21.15.43:0/139639675 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa140036020 con 0x7fa1500b3ed0 2024-04-25T08:56:56.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.584+0000 7fa14effd640 1 -- 172.21.15.43:0/139639675 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa140032820 con 0x7fa1500b3ed0 2024-04-25T08:56:56.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.584+0000 7fa14effd640 1 -- 172.21.15.43:0/139639675 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa140032bf0 con 0x7fa1500b3ed0 2024-04-25T08:56:56.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.585+0000 7fa156529640 1 -- 172.21.15.43:0/139639675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1500b3ed0 msgr2=0x7fa1500ae730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:56.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.585+0000 7fa156529640 1 --2- 172.21.15.43:0/139639675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1500b3ed0 0x7fa1500ae730 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fa14000bb10 tx=0x7fa14002c920 comp rx=0 tx=0).stop 2024-04-25T08:56:56.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.585+0000 7fa156529640 1 -- 172.21.15.43:0/139639675 shutdown_connections 2024-04-25T08:56:56.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.585+0000 7fa156529640 1 --2- 172.21.15.43:0/139639675 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1500b3ed0 0x7fa1500ae730 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:56.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.585+0000 7fa156529640 1 --2- 172.21.15.43:0/139639675 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa15015a960 0x7fa1500b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:56.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.585+0000 7fa156529640 1 -- 172.21.15.43:0/139639675 >> 172.21.15.43:0/139639675 conn(0x7fa15006ab30 msgr2=0x7fa15006af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:56.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.585+0000 7fa156529640 1 -- 172.21.15.43:0/139639675 shutdown_connections 2024-04-25T08:56:56.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.585+0000 7fa156529640 1 -- 172.21.15.43:0/139639675 wait complete. 2024-04-25T08:56:56.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.586+0000 7fa156529640 1 Processor -- start 2024-04-25T08:56:56.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.586+0000 7fa156529640 1 -- start start 2024-04-25T08:56:56.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.587+0000 7fa156529640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa1500aec70 0x7fa150147030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:56.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.587+0000 7fa156529640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1500b3ed0 0x7fa1500aabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:56.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.587+0000 7fa156529640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa15015a960 0x7fa1500ab100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:56.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.587+0000 7fa156529640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1500cc440 con 0x7fa1500b3ed0 2024-04-25T08:56:56.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.587+0000 7fa156529640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15015b9b0 con 0x7fa15015a960 2024-04-25T08:56:56.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.587+0000 7fa156529640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1500cd940 con 0x7fa1500aec70 2024-04-25T08:56:56.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.587+0000 7fa14f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1500b3ed0 0x7fa1500aabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:56.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.587+0000 7fa14f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1500b3ed0 0x7fa1500aabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57834/0 (socket says 172.21.15.43:57834) 2024-04-25T08:56:56.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.587+0000 7fa14f7fe640 1 -- 172.21.15.43:0/2235767173 learned_addr learned my addr 172.21.15.43:0/2235767173 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:56.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.587+0000 7fa14ffff640 1 --2- 172.21.15.43:0/2235767173 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa1500aec70 0x7fa150147030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:56.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.587+0000 7fa154ac6640 1 --2- 172.21.15.43:0/2235767173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa15015a960 0x7fa1500ab100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:56.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.588+0000 7fa14f7fe640 1 -- 172.21.15.43:0/2235767173 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa1500aec70 msgr2=0x7fa150147030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:56.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.588+0000 7fa14f7fe640 1 --2- 172.21.15.43:0/2235767173 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa1500aec70 0x7fa150147030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:56.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.588+0000 7fa14f7fe640 1 -- 172.21.15.43:0/2235767173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa15015a960 msgr2=0x7fa1500ab100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:56.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.588+0000 7fa14f7fe640 1 --2- 172.21.15.43:0/2235767173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa15015a960 0x7fa1500ab100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:56.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.588+0000 7fa14f7fe640 1 -- 172.21.15.43:0/2235767173 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa150147570 con 0x7fa1500b3ed0 2024-04-25T08:56:56.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.588+0000 7fa154ac6640 1 --2- 172.21.15.43:0/2235767173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa15015a960 0x7fa1500ab100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:56:56.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.588+0000 7fa14f7fe640 1 --2- 172.21.15.43:0/2235767173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1500b3ed0 0x7fa1500aabc0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa14002c960 tx=0x7fa1400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:56.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.588+0000 7fa14cff9640 1 -- 172.21.15.43:0/2235767173 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa140036040 con 0x7fa1500b3ed0 2024-04-25T08:56:56.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.588+0000 7fa14cff9640 1 -- 172.21.15.43:0/2235767173 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa14003cd70 con 0x7fa1500b3ed0 2024-04-25T08:56:56.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.588+0000 7fa14cff9640 1 -- 172.21.15.43:0/2235767173 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa140046630 con 0x7fa1500b3ed0 2024-04-25T08:56:56.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.588+0000 7fa156529640 1 -- 172.21.15.43:0/2235767173 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1500a94d0 con 0x7fa1500b3ed0 2024-04-25T08:56:56.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.588+0000 7fa156529640 1 -- 172.21.15.43:0/2235767173 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1500a9a90 con 0x7fa1500b3ed0 2024-04-25T08:56:56.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.589+0000 7fa156529640 1 -- 172.21.15.43:0/2235767173 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa150074f60 con 0x7fa1500b3ed0 2024-04-25T08:56:56.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.589+0000 7fa14cff9640 1 -- 172.21.15.43:0/2235767173 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa14004d060 con 0x7fa1500b3ed0 2024-04-25T08:56:56.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.590+0000 7fa14cff9640 1 --2- 172.21.15.43:0/2235767173 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa13c06b4d0 0x7fa13c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:56.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.590+0000 7fa14cff9640 1 -- 172.21.15.43:0/2235767173 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa1400acff0 con 0x7fa1500b3ed0 2024-04-25T08:56:56.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.590+0000 7fa14ffff640 1 --2- 172.21.15.43:0/2235767173 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa13c06b4d0 0x7fa13c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:56.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.591+0000 7fa14ffff640 1 --2- 172.21.15.43:0/2235767173 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa13c06b4d0 0x7fa13c06d990 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa1440079a0 tx=0x7fa144008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:56.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.594+0000 7fa14cff9640 1 -- 172.21.15.43:0/2235767173 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa150147570 con 0x7fa1500b3ed0 2024-04-25T08:56:56.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.772+0000 7fa156529640 1 -- 172.21.15.43:0/2235767173 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa1500cf3f0 con 0x7fa1500b3ed0 2024-04-25T08:56:56.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.776+0000 7fa14cff9640 1 -- 172.21.15.43:0/2235767173 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa140041070 con 0x7fa1500b3ed0 2024-04-25T08:56:56.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.778+0000 7fa156529640 1 -- 172.21.15.43:0/2235767173 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa13c06b4d0 msgr2=0x7fa13c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:56.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.778+0000 7fa156529640 1 --2- 172.21.15.43:0/2235767173 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa13c06b4d0 0x7fa13c06d990 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa1440079a0 tx=0x7fa144008040 comp rx=0 tx=0).stop 2024-04-25T08:56:56.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.778+0000 7fa156529640 1 -- 172.21.15.43:0/2235767173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1500b3ed0 msgr2=0x7fa1500aabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:56.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.778+0000 7fa156529640 1 --2- 172.21.15.43:0/2235767173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1500b3ed0 0x7fa1500aabc0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa14002c960 tx=0x7fa1400046c0 comp rx=0 tx=0).stop 2024-04-25T08:56:56.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.778+0000 7fa156529640 1 -- 172.21.15.43:0/2235767173 shutdown_connections 2024-04-25T08:56:56.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.778+0000 7fa156529640 1 --2- 172.21.15.43:0/2235767173 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa13c06b4d0 0x7fa13c06d990 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:56.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.778+0000 7fa156529640 1 --2- 172.21.15.43:0/2235767173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa15015a960 0x7fa1500ab100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:56.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.778+0000 7fa156529640 1 --2- 172.21.15.43:0/2235767173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1500b3ed0 0x7fa1500aabc0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:56.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.778+0000 7fa156529640 1 --2- 172.21.15.43:0/2235767173 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa1500aec70 0x7fa150147030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:56.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.778+0000 7fa156529640 1 -- 172.21.15.43:0/2235767173 >> 172.21.15.43:0/2235767173 conn(0x7fa15006ab30 msgr2=0x7fa1500b2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:56.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.778+0000 7fa156529640 1 -- 172.21.15.43:0/2235767173 shutdown_connections 2024-04-25T08:56:56.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:56.778+0000 7fa156529640 1 -- 172.21.15.43:0/2235767173 wait complete. 2024-04-25T08:56:56.778 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:56:56.779 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:56:56.818 DEBUG:teuthology.orchestra.run.smithi193:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:56:56.862 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:56:56.862 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:56:56.862 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:56:56.878 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2374 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:56:56.878 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T08:56:56.878 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:56:56.920 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19508 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:56:56.920 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:56:56.920 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:56:56.936 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2374 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:56:56.936 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T08:56:56.936 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:56:56.979 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19508 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:56:56.979 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T08:56:56.979 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:56:57.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.090+0000 7f90487b6640 1 Processor -- start 2024-04-25T08:56:57.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.090+0000 7f90487b6640 1 -- start start 2024-04-25T08:56:57.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.091+0000 7f90487b6640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9040072160 0x7f9040165370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:57.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.091+0000 7f90487b6640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90400704f0 con 0x7f904015a9d0 2024-04-25T08:56:57.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.091+0000 7f90487b6640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90400707d0 con 0x7f904006d9a0 2024-04-25T08:56:57.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.091+0000 7f90487b6640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9040070ab0 con 0x7f9040072160 2024-04-25T08:56:57.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.091+0000 7f9046d53640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9040072160 0x7f9040165370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:57.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.092+0000 7f9046552640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f904015a9d0 0x7f904006d290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49762/0 (socket says 172.21.15.43:49762) 2024-04-25T08:56:57.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.092+0000 7f9046552640 1 -- 172.21.15.43:0/782564034 learned_addr learned my addr 172.21.15.43:0/782564034 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:56:57.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.092+0000 7f9046d53640 1 -- 172.21.15.43:0/782564034 >> v1:172.21.15.193:6789/0 conn(0x7f904006d9a0 legacy=0x7f9040071a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:57.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.092+0000 7f9046d53640 1 -- 172.21.15.43:0/782564034 >> v1:172.21.15.43:6789/0 conn(0x7f904015a9d0 legacy=0x7f904006d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:57.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.092+0000 7f9046d53640 1 -- 172.21.15.43:0/782564034 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9040070d90 con 0x7f9040072160 2024-04-25T08:56:57.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.092+0000 7f9046d53640 1 --2- 172.21.15.43:0/782564034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9040072160 0x7f9040165370 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f9038009d90 tx=0x7f903802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5859270428bdf606 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:57.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.093+0000 7f9045550640 1 -- 172.21.15.43:0/782564034 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9038007740 con 0x7f9040072160 2024-04-25T08:56:57.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.093+0000 7f9045550640 1 -- 172.21.15.43:0/782564034 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90380078e0 con 0x7f9040072160 2024-04-25T08:56:57.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.093+0000 7f9045550640 1 -- 172.21.15.43:0/782564034 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9038007cb0 con 0x7f9040072160 2024-04-25T08:56:57.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.093+0000 7f90487b6640 1 -- 172.21.15.43:0/782564034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9040072160 msgr2=0x7f9040165370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:57.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.093+0000 7f90487b6640 1 --2- 172.21.15.43:0/782564034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9040072160 0x7f9040165370 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f9038009d90 tx=0x7f903802a6a0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.093+0000 7f90487b6640 1 -- 172.21.15.43:0/782564034 shutdown_connections 2024-04-25T08:56:57.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.093+0000 7f90487b6640 1 --2- 172.21.15.43:0/782564034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9040072160 0x7f9040165370 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.093+0000 7f90487b6640 1 -- 172.21.15.43:0/782564034 >> 172.21.15.43:0/782564034 conn(0x7f90400b1480 msgr2=0x7f90400b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:57.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.093+0000 7f90487b6640 1 -- 172.21.15.43:0/782564034 shutdown_connections 2024-04-25T08:56:57.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.093+0000 7f90487b6640 1 -- 172.21.15.43:0/782564034 wait complete. 2024-04-25T08:56:57.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.094+0000 7f90487b6640 1 Processor -- start 2024-04-25T08:56:57.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.095+0000 7f90487b6640 1 -- start start 2024-04-25T08:56:57.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.095+0000 7f90487b6640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f904006d9a0 0x7f904015d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:57.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.095+0000 7f90487b6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9040072160 0x7f904013d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:57.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.095+0000 7f90487b6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f904014d2d0 0x7f904015b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:57.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.095+0000 7f90487b6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90401658b0 con 0x7f904014d2d0 2024-04-25T08:56:57.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.095+0000 7f90487b6640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90400ce950 con 0x7f9040072160 2024-04-25T08:56:57.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.095+0000 7f90487b6640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9040165a30 con 0x7f904006d9a0 2024-04-25T08:56:57.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.095+0000 7f9046d53640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f904014d2d0 0x7f904015b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:57.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.095+0000 7f9046d53640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f904014d2d0 0x7f904015b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57838/0 (socket says 172.21.15.43:57838) 2024-04-25T08:56:57.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.095+0000 7f9046d53640 1 -- 172.21.15.43:0/2728997160 learned_addr learned my addr 172.21.15.43:0/2728997160 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:57.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.096+0000 7f9045d51640 1 --2- 172.21.15.43:0/2728997160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9040072160 0x7f904013d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:57.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.096+0000 7f9046552640 1 --2- 172.21.15.43:0/2728997160 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f904006d9a0 0x7f904015d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:57.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.096+0000 7f9046d53640 1 -- 172.21.15.43:0/2728997160 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f904006d9a0 msgr2=0x7f904015d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:57.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.096+0000 7f9046d53640 1 --2- 172.21.15.43:0/2728997160 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f904006d9a0 0x7f904015d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.096+0000 7f9046d53640 1 -- 172.21.15.43:0/2728997160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9040072160 msgr2=0x7f904013d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:57.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.096+0000 7f9046d53640 1 --2- 172.21.15.43:0/2728997160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9040072160 0x7f904013d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.096+0000 7f9046d53640 1 -- 172.21.15.43:0/2728997160 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9040153b40 con 0x7f904014d2d0 2024-04-25T08:56:57.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.096+0000 7f9045d51640 1 --2- 172.21.15.43:0/2728997160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9040072160 0x7f904013d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:56:57.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.096+0000 7f9046d53640 1 --2- 172.21.15.43:0/2728997160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f904014d2d0 0x7f904015b890 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f9038009eb0 tx=0x7f9038004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:57.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.096+0000 7f9036ffd640 1 -- 172.21.15.43:0/2728997160 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f903803f070 con 0x7f904014d2d0 2024-04-25T08:56:57.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.096+0000 7f9036ffd640 1 -- 172.21.15.43:0/2728997160 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9038031bc0 con 0x7f904014d2d0 2024-04-25T08:56:57.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.096+0000 7f9036ffd640 1 -- 172.21.15.43:0/2728997160 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9038043410 con 0x7f904014d2d0 2024-04-25T08:56:57.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.096+0000 7f90487b6640 1 -- 172.21.15.43:0/2728997160 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f904014d840 con 0x7f904014d2d0 2024-04-25T08:56:57.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.096+0000 7f90487b6640 1 -- 172.21.15.43:0/2728997160 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f904014de00 con 0x7f904014d2d0 2024-04-25T08:56:57.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.097+0000 7f90487b6640 1 -- 172.21.15.43:0/2728997160 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9040074ef0 con 0x7f904014d2d0 2024-04-25T08:56:57.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.098+0000 7f9036ffd640 1 -- 172.21.15.43:0/2728997160 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f903803b410 con 0x7f904014d2d0 2024-04-25T08:56:57.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.098+0000 7f9036ffd640 1 --2- 172.21.15.43:0/2728997160 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f902406b4d0 0x7f902406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:57.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.098+0000 7f9036ffd640 1 -- 172.21.15.43:0/2728997160 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f90380ab3b0 con 0x7f904014d2d0 2024-04-25T08:56:57.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.098+0000 7f9046552640 1 --2- 172.21.15.43:0/2728997160 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f902406b4d0 0x7f902406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:57.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.099+0000 7f9046552640 1 --2- 172.21.15.43:0/2728997160 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f902406b4d0 0x7f902406d990 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f903c000f60 tx=0x7f903c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:57.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.102+0000 7f9036ffd640 1 -- 172.21.15.43:0/2728997160 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f903807a680 con 0x7f904014d2d0 2024-04-25T08:56:57.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.322+0000 7f90487b6640 1 -- 172.21.15.43:0/2728997160 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9040153d10 con 0x7f904014d2d0 2024-04-25T08:56:57.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.322+0000 7f9036ffd640 1 -- 172.21.15.43:0/2728997160 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+3698 (secure 0 0 0) 0x7f9038079f80 con 0x7f904014d2d0 2024-04-25T08:56:57.312 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:56:57.312 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":16,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4984,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/715858990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":715858990},{"type":"v1","addr":"172.21.15.193:6836","nonce":715858990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4987,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2342354979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2342354979},{"type":"v1","addr":"172.21.15.193:6837","nonce":2342354979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":5205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/491504707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":491504707},{"type":"v1","addr":"172.21.15.43:6834","nonce":491504707}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5607,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2735005981","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2735005981},{"type":"v1","addr":"172.21.15.43:6835","nonce":2735005981}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":16}],"filesystems":[]} 2024-04-25T08:56:57.312 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 16 2024-04-25T08:56:57.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.324+0000 7f90487b6640 1 -- 172.21.15.43:0/2728997160 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f902406b4d0 msgr2=0x7f902406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:57.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.324+0000 7f90487b6640 1 --2- 172.21.15.43:0/2728997160 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f902406b4d0 0x7f902406d990 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f903c000f60 tx=0x7f903c009040 comp rx=0 tx=0).stop 2024-04-25T08:56:57.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.324+0000 7f90487b6640 1 -- 172.21.15.43:0/2728997160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f904014d2d0 msgr2=0x7f904015b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:57.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.324+0000 7f90487b6640 1 --2- 172.21.15.43:0/2728997160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f904014d2d0 0x7f904015b890 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f9038009eb0 tx=0x7f9038004a40 comp rx=0 tx=0).stop 2024-04-25T08:56:57.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.324+0000 7f90487b6640 1 -- 172.21.15.43:0/2728997160 shutdown_connections 2024-04-25T08:56:57.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.324+0000 7f90487b6640 1 --2- 172.21.15.43:0/2728997160 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f904014d2d0 0x7f904015b890 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.324+0000 7f90487b6640 1 --2- 172.21.15.43:0/2728997160 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f902406b4d0 0x7f902406d990 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.324+0000 7f90487b6640 1 --2- 172.21.15.43:0/2728997160 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9040072160 0x7f904013d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.324+0000 7f90487b6640 1 --2- 172.21.15.43:0/2728997160 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f904006d9a0 0x7f904015d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.324+0000 7f90487b6640 1 -- 172.21.15.43:0/2728997160 >> 172.21.15.43:0/2728997160 conn(0x7f90400b1480 msgr2=0x7f9040157e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:57.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.324+0000 7f90487b6640 1 -- 172.21.15.43:0/2728997160 shutdown_connections 2024-04-25T08:56:57.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.324+0000 7f90487b6640 1 -- 172.21.15.43:0/2728997160 wait complete. 2024-04-25T08:56:57.325 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T08:56:57.325 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T08:56:57.326 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T08:56:57.326 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:56:57.326 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T08:56:57.326 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T08:56:57.326 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T08:56:57.326 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:56:57.326 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T08:56:57.327 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T08:56:57.327 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T08:56:57.327 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:56:57.327 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T08:56:57.327 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T08:56:57.327 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T08:56:57.327 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:56:57.335 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T08:56:57.335 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T08:56:57.336 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T08:56:57.336 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T08:56:57.338 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T08:56:57.338 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T08:56:57.339 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T08:56:57.339 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T08:56:57.377 INFO:tasks.ceph.mds.a:Started 2024-04-25T08:56:57.378 DEBUG:teuthology.parallel:result is None 2024-04-25T08:56:57.378 INFO:tasks.ceph.mds.c:Started 2024-04-25T08:56:57.378 DEBUG:teuthology.parallel:result is None 2024-04-25T08:56:57.380 INFO:tasks.ceph.mds.b:Started 2024-04-25T08:56:57.381 DEBUG:teuthology.parallel:result is None 2024-04-25T08:56:57.381 INFO:tasks.ceph.mds.d:Started 2024-04-25T08:56:57.381 DEBUG:teuthology.parallel:result is None 2024-04-25T08:56:57.381 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T08:56:57.453 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-04-25T08:56:57.454 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2024-04-25T08:56:57.460 INFO:tasks.ceph.mds.b.smithi193.stdout:starting mds.b at 2024-04-25T08:56:57.460 INFO:tasks.ceph.mds.d.smithi193.stdout:starting mds.d at 2024-04-25T08:56:57.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.498+0000 7f3014f3b640 1 Processor -- start 2024-04-25T08:56:57.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.499+0000 7f3014f3b640 1 -- start start 2024-04-25T08:56:57.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.499+0000 7f3014f3b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f30100d39d0 0x7f30100d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:57.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.499+0000 7f3014f3b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f30100ce2a0 0x7f30100d3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:57.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.499+0000 7f3014f3b640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f3010161710 0x7f3010161ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:57.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.499+0000 7f3014f3b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30100b6c50 con 0x7f30100ce2a0 2024-04-25T08:56:57.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.499+0000 7f3014f3b640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f301015e010 con 0x7f3010161710 2024-04-25T08:56:57.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.499+0000 7f3014f3b640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30100d2020 con 0x7f30100d39d0 2024-04-25T08:56:57.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.500+0000 7f300dd9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f30100ce2a0 0x7f30100d3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:57.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.500+0000 7f300dd9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f30100ce2a0 0x7f30100d3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57866/0 (socket says 172.21.15.43:57866) 2024-04-25T08:56:57.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.500+0000 7f300dd9b640 1 -- 172.21.15.43:0/1353822608 learned_addr learned my addr 172.21.15.43:0/1353822608 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:57.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.500+0000 7f300e59c640 1 --2- 172.21.15.43:0/1353822608 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f30100d39d0 0x7f30100d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:57.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.500+0000 7f300ed9d640 1 --2- 172.21.15.43:0/1353822608 >> v2:172.21.15.193:3300/0 conn(0x7f3010161710 0x7f3010161ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:57.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.500+0000 7f300dd9b640 1 -- 172.21.15.43:0/1353822608 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f30100d39d0 msgr2=0x7f30100d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:57.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.500+0000 7f300dd9b640 1 --2- 172.21.15.43:0/1353822608 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f30100d39d0 0x7f30100d0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.500+0000 7f300dd9b640 1 -- 172.21.15.43:0/1353822608 >> v2:172.21.15.193:3300/0 conn(0x7f3010161710 msgr2=0x7f3010161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:57.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.500+0000 7f300dd9b640 1 --2- 172.21.15.43:0/1353822608 >> v2:172.21.15.193:3300/0 conn(0x7f3010161710 0x7f3010161ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.500+0000 7f300dd9b640 1 -- 172.21.15.43:0/1353822608 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30100d42f0 con 0x7f30100ce2a0 2024-04-25T08:56:57.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.501+0000 7f300dd9b640 1 --2- 172.21.15.43:0/1353822608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30100ce2a0 0x7f30100d3db0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f301006ccf0 tx=0x7f300002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=262138d505b016fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:57.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.501+0000 7f300d59a640 1 -- 172.21.15.43:0/1353822608 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3000037070 con 0x7f30100ce2a0 2024-04-25T08:56:57.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.501+0000 7f300d59a640 1 -- 172.21.15.43:0/1353822608 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3000032440 con 0x7f30100ce2a0 2024-04-25T08:56:57.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.501+0000 7f300d59a640 1 -- 172.21.15.43:0/1353822608 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3000032810 con 0x7f30100ce2a0 2024-04-25T08:56:57.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.501+0000 7f3014f3b640 1 -- 172.21.15.43:0/1353822608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30100ce2a0 msgr2=0x7f30100d3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:57.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.501+0000 7f3014f3b640 1 --2- 172.21.15.43:0/1353822608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30100ce2a0 0x7f30100d3db0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f301006ccf0 tx=0x7f300002a140 comp rx=0 tx=0).stop 2024-04-25T08:56:57.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.502+0000 7f3014f3b640 1 -- 172.21.15.43:0/1353822608 shutdown_connections 2024-04-25T08:56:57.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.502+0000 7f3014f3b640 1 --2- 172.21.15.43:0/1353822608 >> v2:172.21.15.193:3300/0 conn(0x7f3010161710 0x7f3010161ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.502+0000 7f3014f3b640 1 --2- 172.21.15.43:0/1353822608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30100ce2a0 0x7f30100d3db0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.502+0000 7f3014f3b640 1 --2- 172.21.15.43:0/1353822608 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f30100d39d0 0x7f30100d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.502+0000 7f3014f3b640 1 -- 172.21.15.43:0/1353822608 >> 172.21.15.43:0/1353822608 conn(0x7f301006b000 msgr2=0x7f301006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:57.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.502+0000 7f3014f3b640 1 -- 172.21.15.43:0/1353822608 shutdown_connections 2024-04-25T08:56:57.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.502+0000 7f3014f3b640 1 -- 172.21.15.43:0/1353822608 wait complete. 2024-04-25T08:56:57.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.503+0000 7f3014f3b640 1 Processor -- start 2024-04-25T08:56:57.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.503+0000 7f3014f3b640 1 -- start start 2024-04-25T08:56:57.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.504+0000 7f3014f3b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f30100ce2a0 0x7f30100b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:57.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.504+0000 7f3014f3b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30100d39d0 0x7f3010159500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:57.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.504+0000 7f3014f3b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3010161710 0x7f3010156970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:57.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.504+0000 7f3014f3b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30100b2f00 con 0x7f30100d39d0 2024-04-25T08:56:57.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.504+0000 7f3014f3b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30100cef20 con 0x7f3010161710 2024-04-25T08:56:57.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.504+0000 7f3014f3b640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30100cf200 con 0x7f30100ce2a0 2024-04-25T08:56:57.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.504+0000 7f300dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30100d39d0 0x7f3010159500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:57.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.504+0000 7f300dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30100d39d0 0x7f3010159500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57876/0 (socket says 172.21.15.43:57876) 2024-04-25T08:56:57.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.504+0000 7f300dd9b640 1 -- 172.21.15.43:0/3438339565 learned_addr learned my addr 172.21.15.43:0/3438339565 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:57.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.504+0000 7f300e59c640 1 --2- 172.21.15.43:0/3438339565 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f30100ce2a0 0x7f30100b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:57.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.504+0000 7f300ed9d640 1 --2- 172.21.15.43:0/3438339565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3010161710 0x7f3010156970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:57.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.504+0000 7f300dd9b640 1 -- 172.21.15.43:0/3438339565 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f30100ce2a0 msgr2=0x7f30100b54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:57.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.504+0000 7f300dd9b640 1 --2- 172.21.15.43:0/3438339565 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f30100ce2a0 0x7f30100b54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.504+0000 7f300dd9b640 1 -- 172.21.15.43:0/3438339565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3010161710 msgr2=0x7f3010156970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:57.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.504+0000 7f300dd9b640 1 --2- 172.21.15.43:0/3438339565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3010161710 0x7f3010156970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:57.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.504+0000 7f300dd9b640 1 -- 172.21.15.43:0/3438339565 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30100b4160 con 0x7f30100d39d0 2024-04-25T08:56:57.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.504+0000 7f300dd9b640 1 --2- 172.21.15.43:0/3438339565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30100d39d0 0x7f3010159500 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f3000000c00 tx=0x7f30000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:57.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.504+0000 7f2ffeffd640 1 -- 172.21.15.43:0/3438339565 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3000041070 con 0x7f30100d39d0 2024-04-25T08:56:57.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.504+0000 7f2ffeffd640 1 -- 172.21.15.43:0/3438339565 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3000032e50 con 0x7f30100d39d0 2024-04-25T08:56:57.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.504+0000 7f3014f3b640 1 -- 172.21.15.43:0/3438339565 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30100b4440 con 0x7f30100d39d0 2024-04-25T08:56:57.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.505+0000 7f2ffeffd640 1 -- 172.21.15.43:0/3438339565 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3000037040 con 0x7f30100d39d0 2024-04-25T08:56:57.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.505+0000 7f3014f3b640 1 -- 172.21.15.43:0/3438339565 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30100b4a00 con 0x7f30100d39d0 2024-04-25T08:56:57.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.506+0000 7f3014f3b640 1 -- 172.21.15.43:0/3438339565 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3010074ef0 con 0x7f30100d39d0 2024-04-25T08:56:57.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.506+0000 7f2ffeffd640 1 -- 172.21.15.43:0/3438339565 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3000044070 con 0x7f30100d39d0 2024-04-25T08:56:57.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.506+0000 7f2ffeffd640 1 --2- 172.21.15.43:0/3438339565 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2ff006b500 0x7f2ff006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:57.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.506+0000 7f2ffeffd640 1 -- 172.21.15.43:0/3438339565 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f30000aab30 con 0x7f30100d39d0 2024-04-25T08:56:57.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.506+0000 7f300e59c640 1 --2- 172.21.15.43:0/3438339565 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2ff006b500 0x7f2ff006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:57.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.507+0000 7f300e59c640 1 --2- 172.21.15.43:0/3438339565 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2ff006b500 0x7f2ff006d9c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3004009990 tx=0x7f3004008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:57.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.511+0000 7f2ffeffd640 1 -- 172.21.15.43:0/3438339565 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3000079dd0 con 0x7f30100d39d0 2024-04-25T08:56:57.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:57.707+0000 7f3014f3b640 1 -- 172.21.15.43:0/3438339565 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f30100ce880 con 0x7f30100d39d0 2024-04-25T08:56:58.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.466+0000 7f2ffeffd640 1 -- 172.21.15.43:0/3438339565 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v40) v1 ==== 97+0+0 (secure 0 0 0) 0x7f30000796d0 con 0x7f30100d39d0 2024-04-25T08:56:58.456 INFO:teuthology.orchestra.run.smithi043.stderr: removed all blocklist entries 2024-04-25T08:56:58.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.470+0000 7f3014f3b640 1 -- 172.21.15.43:0/3438339565 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2ff006b500 msgr2=0x7f2ff006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:58.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.470+0000 7f3014f3b640 1 --2- 172.21.15.43:0/3438339565 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2ff006b500 0x7f2ff006d9c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3004009990 tx=0x7f3004008040 comp rx=0 tx=0).stop 2024-04-25T08:56:58.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.470+0000 7f3014f3b640 1 -- 172.21.15.43:0/3438339565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30100d39d0 msgr2=0x7f3010159500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:58.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.470+0000 7f3014f3b640 1 --2- 172.21.15.43:0/3438339565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30100d39d0 0x7f3010159500 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f3000000c00 tx=0x7f30000046c0 comp rx=0 tx=0).stop 2024-04-25T08:56:58.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.470+0000 7f3014f3b640 1 -- 172.21.15.43:0/3438339565 shutdown_connections 2024-04-25T08:56:58.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.470+0000 7f3014f3b640 1 --2- 172.21.15.43:0/3438339565 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2ff006b500 0x7f2ff006d9c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:58.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.470+0000 7f3014f3b640 1 --2- 172.21.15.43:0/3438339565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3010161710 0x7f3010156970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:58.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.470+0000 7f3014f3b640 1 --2- 172.21.15.43:0/3438339565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30100d39d0 0x7f3010159500 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:58.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.470+0000 7f3014f3b640 1 --2- 172.21.15.43:0/3438339565 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f30100ce2a0 0x7f30100b54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:58.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.470+0000 7f3014f3b640 1 -- 172.21.15.43:0/3438339565 >> 172.21.15.43:0/3438339565 conn(0x7f301006b000 msgr2=0x7f301015f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:58.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.470+0000 7f3014f3b640 1 -- 172.21.15.43:0/3438339565 shutdown_connections 2024-04-25T08:56:58.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.470+0000 7f3014f3b640 1 -- 172.21.15.43:0/3438339565 wait complete. 2024-04-25T08:56:58.475 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T08:56:58.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.594+0000 7fac2c893640 1 Processor -- start 2024-04-25T08:56:58.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.594+0000 7fac2c893640 1 -- start start 2024-04-25T08:56:58.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.595+0000 7fac2c893640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fac280cc270 0x7fac280cda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:58.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.595+0000 7fac2c893640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac280ce3d0 con 0x7fac280cdfa0 2024-04-25T08:56:58.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.595+0000 7fac2c893640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac280cc700 con 0x7fac2815dd00 2024-04-25T08:56:58.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.595+0000 7fac2c893640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac280c4ed0 con 0x7fac280cc270 2024-04-25T08:56:58.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.595+0000 7fac26d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fac280cdfa0 0x7fac28161760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49788/0 (socket says 172.21.15.43:49788) 2024-04-25T08:56:58.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.595+0000 7fac26d9d640 1 -- 172.21.15.43:0/2016779972 learned_addr learned my addr 172.21.15.43:0/2016779972 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:56:58.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.595+0000 7fac1ffff640 1 --2- 172.21.15.43:0/2016779972 >> v2:172.21.15.193:3300/0 conn(0x7fac280cc270 0x7fac280cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:58.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.595+0000 7fac25d9b640 1 -- 172.21.15.43:0/2016779972 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1506918118 0 0) 0x7fac280ce3d0 con 0x7fac280cdfa0 2024-04-25T08:56:58.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.595+0000 7fac25d9b640 1 -- 172.21.15.43:0/2016779972 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac14003520 con 0x7fac280cdfa0 2024-04-25T08:56:58.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.596+0000 7fac25d9b640 1 -- 172.21.15.43:0/2016779972 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4052040653 0 0) 0x7fac14003520 con 0x7fac280cdfa0 2024-04-25T08:56:58.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.596+0000 7fac1ffff640 1 -- 172.21.15.43:0/2016779972 >> v1:172.21.15.193:6789/0 conn(0x7fac2815dd00 legacy=0x7fac280c5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:58.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.596+0000 7fac1ffff640 1 -- 172.21.15.43:0/2016779972 >> v1:172.21.15.43:6789/0 conn(0x7fac280cdfa0 legacy=0x7fac28161760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:58.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.596+0000 7fac1ffff640 1 -- 172.21.15.43:0/2016779972 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac280c51b0 con 0x7fac280cc270 2024-04-25T08:56:58.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.596+0000 7fac1ffff640 1 --2- 172.21.15.43:0/2016779972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fac280cc270 0x7fac280cda60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fac100097a0 tx=0x7fac1002a0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10f406a6c5f1b451 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:58.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.597+0000 7fac25d9b640 1 -- 172.21.15.43:0/2016779972 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac10037070 con 0x7fac280cc270 2024-04-25T08:56:58.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.597+0000 7fac25d9b640 1 -- 172.21.15.43:0/2016779972 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac1002ab60 con 0x7fac280cc270 2024-04-25T08:56:58.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.597+0000 7fac25d9b640 1 -- 172.21.15.43:0/2016779972 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac200019f0 con 0x7fac280cc270 2024-04-25T08:56:58.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.597+0000 7fac2c893640 1 -- 172.21.15.43:0/2016779972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fac280cc270 msgr2=0x7fac280cda60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:58.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.597+0000 7fac2c893640 1 --2- 172.21.15.43:0/2016779972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fac280cc270 0x7fac280cda60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fac100097a0 tx=0x7fac1002a0d0 comp rx=0 tx=0).stop 2024-04-25T08:56:58.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.597+0000 7fac2c893640 1 -- 172.21.15.43:0/2016779972 shutdown_connections 2024-04-25T08:56:58.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.597+0000 7fac2c893640 1 --2- 172.21.15.43:0/2016779972 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fac280cc270 0x7fac280cda60 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:58.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.597+0000 7fac2c893640 1 -- 172.21.15.43:0/2016779972 >> 172.21.15.43:0/2016779972 conn(0x7fac2806ab30 msgr2=0x7fac2806af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:58.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.597+0000 7fac2c893640 1 -- 172.21.15.43:0/2016779972 shutdown_connections 2024-04-25T08:56:58.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.597+0000 7fac2c893640 1 -- 172.21.15.43:0/2016779972 wait complete. 2024-04-25T08:56:58.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.599+0000 7fac2c893640 1 Processor -- start 2024-04-25T08:56:58.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.599+0000 7fac2c893640 1 -- start start 2024-04-25T08:56:58.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.599+0000 7fac2c893640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fac280cc270 0x7fac2806e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:58.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.599+0000 7fac2c893640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fac280cdfa0 0x7fac281543c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:58.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.599+0000 7fac2c893640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac2815dd00 0x7fac2806cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:58.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.599+0000 7fac2c893640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac280c5a50 con 0x7fac2815dd00 2024-04-25T08:56:58.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.599+0000 7fac2c893640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac280c5770 con 0x7fac280cdfa0 2024-04-25T08:56:58.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.599+0000 7fac2c893640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac280c6680 con 0x7fac280cc270 2024-04-25T08:56:58.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.599+0000 7fac26d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac2815dd00 0x7fac2806cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:58.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.599+0000 7fac26d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac2815dd00 0x7fac2806cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57884/0 (socket says 172.21.15.43:57884) 2024-04-25T08:56:58.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.599+0000 7fac26d9d640 1 -- 172.21.15.43:0/3192869695 learned_addr learned my addr 172.21.15.43:0/3192869695 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:58.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.600+0000 7fac1ffff640 1 --2- 172.21.15.43:0/3192869695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fac280cdfa0 0x7fac281543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:58.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.600+0000 7fac2659c640 1 --2- 172.21.15.43:0/3192869695 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fac280cc270 0x7fac2806e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:58.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.600+0000 7fac26d9d640 1 -- 172.21.15.43:0/3192869695 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fac280cc270 msgr2=0x7fac2806e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:58.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.600+0000 7fac26d9d640 1 --2- 172.21.15.43:0/3192869695 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fac280cc270 0x7fac2806e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:58.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.600+0000 7fac26d9d640 1 -- 172.21.15.43:0/3192869695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fac280cdfa0 msgr2=0x7fac281543c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:58.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.600+0000 7fac26d9d640 1 --2- 172.21.15.43:0/3192869695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fac280cdfa0 0x7fac281543c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:58.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.600+0000 7fac26d9d640 1 -- 172.21.15.43:0/3192869695 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac2815b2b0 con 0x7fac2815dd00 2024-04-25T08:56:58.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.600+0000 7fac26d9d640 1 --2- 172.21.15.43:0/3192869695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac2815dd00 0x7fac2806cb20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fac20001560 tx=0x7fac2000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:58.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.600+0000 7fac1effd640 1 -- 172.21.15.43:0/3192869695 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac20018070 con 0x7fac2815dd00 2024-04-25T08:56:58.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.600+0000 7fac1effd640 1 -- 172.21.15.43:0/3192869695 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac2000a6a0 con 0x7fac2815dd00 2024-04-25T08:56:58.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.600+0000 7fac2c893640 1 -- 172.21.15.43:0/3192869695 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac2815b590 con 0x7fac2815dd00 2024-04-25T08:56:58.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.600+0000 7fac1effd640 1 -- 172.21.15.43:0/3192869695 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2000aa90 con 0x7fac2815dd00 2024-04-25T08:56:58.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.600+0000 7fac2c893640 1 -- 172.21.15.43:0/3192869695 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac2815bb50 con 0x7fac2815dd00 2024-04-25T08:56:58.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.601+0000 7fac2c893640 1 -- 172.21.15.43:0/3192869695 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac28074ef0 con 0x7fac2815dd00 2024-04-25T08:56:58.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.602+0000 7fac1effd640 1 -- 172.21.15.43:0/3192869695 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fac2000ac30 con 0x7fac2815dd00 2024-04-25T08:56:58.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.602+0000 7fac1effd640 1 --2- 172.21.15.43:0/3192869695 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fac0806b4d0 0x7fac0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:58.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.602+0000 7fac1effd640 1 -- 172.21.15.43:0/3192869695 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fac20089610 con 0x7fac2815dd00 2024-04-25T08:56:58.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.603+0000 7fac2659c640 1 --2- 172.21.15.43:0/3192869695 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fac0806b4d0 0x7fac0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:58.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.603+0000 7fac2659c640 1 --2- 172.21.15.43:0/3192869695 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fac0806b4d0 0x7fac0806d990 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fac18000f60 tx=0x7fac18009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:58.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.606+0000 7fac1effd640 1 -- 172.21.15.43:0/3192869695 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fac200587f0 con 0x7fac2815dd00 2024-04-25T08:56:58.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.858+0000 7fac2c893640 1 -- 172.21.15.43:0/3192869695 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fac280669c0 con 0x7fac2815dd00 2024-04-25T08:56:58.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.858+0000 7fac1effd640 1 -- 172.21.15.43:0/3192869695 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fac200580f0 con 0x7fac2815dd00 2024-04-25T08:56:58.848 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:56:58.848 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T08:56:58.848 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_dump": [ 2024-04-25T08:56:58.848 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:56:58.848 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.a", 2024-04-25T08:56:58.848 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpmIjzEBRAAs2L10XrFO3mThjZevna5wA==", 2024-04-25T08:56:58.848 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:56:58.848 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T08:56:58.848 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T08:56:58.848 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T08:56:58.848 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:56:58.848 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:56:58.849 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:58.849 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:56:58.849 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.b", 2024-04-25T08:56:58.849 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm51cmERAAij4vy8xyihh8u/4dkmcllw==", 2024-04-25T08:56:58.849 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:56:58.849 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T08:56:58.849 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T08:56:58.849 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T08:56:58.849 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:56:58.849 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:56:58.849 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:58.849 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:56:58.849 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.c", 2024-04-25T08:56:58.849 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm1OOtDBAAlUYbCh1mBEEka9CWRnUz9A==", 2024-04-25T08:56:58.850 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:56:58.850 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T08:56:58.850 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T08:56:58.850 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T08:56:58.850 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:56:58.850 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:56:58.850 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:58.850 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:56:58.850 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.d", 2024-04-25T08:56:58.850 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpmpEvsFxAAXX1h37UWdFkyJ3Iw3xd8kw==", 2024-04-25T08:56:58.850 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:56:58.850 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T08:56:58.850 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T08:56:58.850 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T08:56:58.850 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:56:58.851 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:56:58.851 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:58.851 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:56:58.851 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.0", 2024-04-25T08:56:58.851 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD9GSpmi7qkORAAwSNgwzcVMitrPPf3nDyqvQ==", 2024-04-25T08:56:58.851 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:56:58.851 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T08:56:58.851 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T08:56:58.851 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:56:58.851 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:56:58.851 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:58.851 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:56:58.851 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.1", 2024-04-25T08:56:58.851 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAAGipmJDT3NxAAcWBCd9nt7Au+Xftjqcuc5w==", 2024-04-25T08:56:58.851 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:56:58.852 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T08:56:58.852 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T08:56:58.852 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:56:58.852 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:56:58.852 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:58.852 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:56:58.852 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.2", 2024-04-25T08:56:58.852 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQADGipm68kzNhAA0ZOnlr4edWDjx6RfoEHYIw==", 2024-04-25T08:56:58.852 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:56:58.852 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T08:56:58.852 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T08:56:58.852 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:56:58.852 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:56:58.852 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:58.852 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:56:58.852 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.3", 2024-04-25T08:56:58.853 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAGGipmT1AeNBAAoNPiEtkznSxxhVJ1EmZ1ZA==", 2024-04-25T08:56:58.853 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:56:58.853 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T08:56:58.853 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T08:56:58.853 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:56:58.853 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:56:58.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:58.853 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:56:58.853 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.4", 2024-04-25T08:56:58.853 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAMGipmBjHHNxAAH/SCzKHus8fKxxI3UN4wWQ==", 2024-04-25T08:56:58.853 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:56:58.853 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T08:56:58.853 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T08:56:58.853 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:56:58.853 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:56:58.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:58.854 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:56:58.854 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.5", 2024-04-25T08:56:58.854 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPGipm3FXKNRAAL97YYWONg4QHr4ajC5D7fA==", 2024-04-25T08:56:58.854 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:56:58.854 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T08:56:58.854 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T08:56:58.854 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:56:58.854 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:56:58.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:58.854 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:56:58.854 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.6", 2024-04-25T08:56:58.854 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQASGipmCwKaMxAAcM0p9Q2sbNQviGkwSLg8AA==", 2024-04-25T08:56:58.854 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:56:58.855 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T08:56:58.855 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T08:56:58.855 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:56:58.855 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:56:58.855 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:58.855 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:56:58.855 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.7", 2024-04-25T08:56:58.855 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAVGipmP7A2MRAAxnFJFosMcbZ6VDbgH8r6wA==", 2024-04-25T08:56:58.855 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:56:58.855 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T08:56:58.855 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T08:56:58.855 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:56:58.855 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:56:58.855 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:58.855 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:56:58.855 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.0", 2024-04-25T08:56:58.856 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpmVOjWGxAAFNmxeGi0h2D//pa6Ne3eQw==", 2024-04-25T08:56:58.856 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:56:58.856 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T08:56:58.856 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow r", 2024-04-25T08:56:58.856 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T08:56:58.856 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:56:58.856 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:58.856 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:56:58.856 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.1", 2024-04-25T08:56:58.856 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm24laJBAA9ijbDlFDTmF/ZZ4IhLfBVQ==", 2024-04-25T08:56:58.856 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:56:58.856 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T08:56:58.856 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-04-25T08:56:58.856 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-04-25T08:56:58.857 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-04-25T08:56:58.857 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:56:58.857 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:58.857 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:56:58.857 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.2", 2024-04-25T08:56:58.857 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm8b5UIRAACcOwDL630oJrXfp7s+fNTA==", 2024-04-25T08:56:58.857 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:56:58.857 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T08:56:58.857 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-04-25T08:56:58.857 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-04-25T08:56:58.857 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-04-25T08:56:58.857 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:56:58.857 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:58.858 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:56:58.858 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.3", 2024-04-25T08:56:58.858 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpmLsbBKRAAIKWtRdpuyCaW1LnKMWTMiw==", 2024-04-25T08:56:58.858 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:56:58.858 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T08:56:58.858 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-04-25T08:56:58.858 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-04-25T08:56:58.858 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-04-25T08:56:58.858 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:56:58.858 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:58.858 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:56:58.858 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.admin", 2024-04-25T08:56:58.858 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD7GSpmE9C8JRAAS263n6a4L2DXzMyS9o42NQ==", 2024-04-25T08:56:58.858 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:56:58.859 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-04-25T08:56:58.859 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T08:56:58.859 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T08:56:58.859 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:56:58.859 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:56:58.859 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:58.859 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:56:58.859 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.x", 2024-04-25T08:56:58.859 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm/uGhAhAAr0FaJ6SwZ8Su1MEqyZwNRA==", 2024-04-25T08:56:58.859 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:56:58.859 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-04-25T08:56:58.859 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2024-04-25T08:56:58.859 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:56:58.859 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:56:58.859 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:56:58.860 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:56:58.860 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.y", 2024-04-25T08:56:58.860 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD7GSpmPwwpOhAALeiQ/LKM2zgjE7yu16RQOg==", 2024-04-25T08:56:58.860 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:56:58.860 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-04-25T08:56:58.860 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2024-04-25T08:56:58.860 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:56:58.860 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:56:58.860 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:56:58.860 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-04-25T08:56:58.860 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T08:56:58.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.860+0000 7fac2c893640 1 -- 172.21.15.43:0/3192869695 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fac0806b4d0 msgr2=0x7fac0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:58.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.860+0000 7fac2c893640 1 --2- 172.21.15.43:0/3192869695 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fac0806b4d0 0x7fac0806d990 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fac18000f60 tx=0x7fac18009040 comp rx=0 tx=0).stop 2024-04-25T08:56:58.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.860+0000 7fac2c893640 1 -- 172.21.15.43:0/3192869695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac2815dd00 msgr2=0x7fac2806cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:58.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.860+0000 7fac2c893640 1 --2- 172.21.15.43:0/3192869695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac2815dd00 0x7fac2806cb20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fac20001560 tx=0x7fac2000cf50 comp rx=0 tx=0).stop 2024-04-25T08:56:58.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.860+0000 7fac2c893640 1 -- 172.21.15.43:0/3192869695 shutdown_connections 2024-04-25T08:56:58.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.860+0000 7fac2c893640 1 --2- 172.21.15.43:0/3192869695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac2815dd00 0x7fac2806cb20 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:58.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.860+0000 7fac2c893640 1 --2- 172.21.15.43:0/3192869695 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fac0806b4d0 0x7fac0806d990 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:58.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.860+0000 7fac2c893640 1 --2- 172.21.15.43:0/3192869695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fac280cdfa0 0x7fac281543c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:58.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.860+0000 7fac2c893640 1 --2- 172.21.15.43:0/3192869695 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fac280cc270 0x7fac2806e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:58.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.860+0000 7fac2c893640 1 -- 172.21.15.43:0/3192869695 >> 172.21.15.43:0/3192869695 conn(0x7fac2806ab30 msgr2=0x7fac280ca4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:58.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.860+0000 7fac2c893640 1 -- 172.21.15.43:0/3192869695 shutdown_connections 2024-04-25T08:56:58.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.860+0000 7fac2c893640 1 -- 172.21.15.43:0/3192869695 wait complete. 2024-04-25T08:56:58.863 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:56:58.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.975+0000 7f4b27f34640 1 Processor -- start 2024-04-25T08:56:58.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.975+0000 7f4b27f34640 1 -- start start 2024-04-25T08:56:58.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.976+0000 7f4b27f34640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f4b2015daa0 0x7f4b20162220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:58.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.976+0000 7f4b27f34640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b20162760 con 0x7f4b200cc8d0 2024-04-25T08:56:58.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.976+0000 7f4b27f34640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b200b4560 con 0x7f4b2015de80 2024-04-25T08:56:58.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.976+0000 7f4b27f34640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b200cd950 con 0x7f4b2015daa0 2024-04-25T08:56:58.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.976+0000 7f4b264d1640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4b200cc8d0 0x7f4b200ce0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49798/0 (socket says 172.21.15.43:49798) 2024-04-25T08:56:58.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.976+0000 7f4b264d1640 1 -- 172.21.15.43:0/652141861 learned_addr learned my addr 172.21.15.43:0/652141861 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:56:58.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.976+0000 7f4b25cd0640 1 --2- 172.21.15.43:0/652141861 >> v2:172.21.15.193:3300/0 conn(0x7f4b2015daa0 0x7f4b20162220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:58.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.976+0000 7f4b24cce640 1 -- 172.21.15.43:0/652141861 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 571780642 0 0) 0x7f4b20162760 con 0x7f4b200cc8d0 2024-04-25T08:56:58.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.976+0000 7f4b24cce640 1 -- 172.21.15.43:0/652141861 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b0c003520 con 0x7f4b200cc8d0 2024-04-25T08:56:58.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.977+0000 7f4b24cce640 1 -- 172.21.15.43:0/652141861 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1529051589 0 0) 0x7f4b0c003520 con 0x7f4b200cc8d0 2024-04-25T08:56:58.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.977+0000 7f4b24cce640 1 -- 172.21.15.43:0/652141861 >> v2:172.21.15.193:3300/0 conn(0x7f4b2015daa0 msgr2=0x7f4b20162220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:58.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.977+0000 7f4b24cce640 1 --2- 172.21.15.43:0/652141861 >> v2:172.21.15.193:3300/0 conn(0x7f4b2015daa0 0x7f4b20162220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:58.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.977+0000 7f4b24cce640 1 -- 172.21.15.43:0/652141861 >> v1:172.21.15.193:6789/0 conn(0x7f4b2015de80 legacy=0x7f4b200cc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:58.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.977+0000 7f4b24cce640 1 -- 172.21.15.43:0/652141861 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b200b4840 con 0x7f4b200cc8d0 2024-04-25T08:56:58.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.977+0000 7f4b24cce640 1 -- 172.21.15.43:0/652141861 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f4b18001a10 con 0x7f4b200cc8d0 2024-04-25T08:56:58.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.977+0000 7f4b24cce640 1 -- 172.21.15.43:0/652141861 >> v1:172.21.15.43:6789/0 conn(0x7f4b200cc8d0 legacy=0x7f4b200ce0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:58.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.977+0000 7f4b24cce640 1 --2- 172.21.15.43:0/652141861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b0c0039b0 0x7f4b0c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:58.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.977+0000 7f4b24cce640 1 --2- 172.21.15.43:0/652141861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b0c004800 0x7f4b0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:58.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.977+0000 7f4b24cce640 1 --2- 172.21.15.43:0/652141861 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4b0c0051c0 0x7f4b0c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:58.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.977+0000 7f4b24cce640 1 -- 172.21.15.43:0/652141861 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b200cd950 con 0x7f4b0c004800 2024-04-25T08:56:58.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.977+0000 7f4b24cce640 1 -- 172.21.15.43:0/652141861 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0c00c060 con 0x7f4b0c0039b0 2024-04-25T08:56:58.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.977+0000 7f4b24cce640 1 -- 172.21.15.43:0/652141861 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0c00c260 con 0x7f4b0c0051c0 2024-04-25T08:56:58.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.977+0000 7f4b254cf640 1 --2- 172.21.15.43:0/652141861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b0c004800 0x7f4b0c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:58.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.977+0000 7f4b25cd0640 1 --2- 172.21.15.43:0/652141861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b0c0039b0 0x7f4b0c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:58.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.977+0000 7f4b264d1640 1 --2- 172.21.15.43:0/652141861 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4b0c0051c0 0x7f4b0c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:58.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.977+0000 7f4b254cf640 1 -- 172.21.15.43:0/652141861 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4b0c0051c0 msgr2=0x7f4b0c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:58.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.977+0000 7f4b254cf640 1 --2- 172.21.15.43:0/652141861 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4b0c0051c0 0x7f4b0c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:58.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.977+0000 7f4b254cf640 1 -- 172.21.15.43:0/652141861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b0c0039b0 msgr2=0x7f4b0c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:58.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.977+0000 7f4b254cf640 1 --2- 172.21.15.43:0/652141861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b0c0039b0 0x7f4b0c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:58.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.977+0000 7f4b254cf640 1 -- 172.21.15.43:0/652141861 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b0c003520 con 0x7f4b0c004800 2024-04-25T08:56:58.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.977+0000 7f4b264d1640 1 -- 172.21.15.43:0/652141861 reap_dead start 2024-04-25T08:56:58.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.978+0000 7f4b254cf640 1 --2- 172.21.15.43:0/652141861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b0c004800 0x7f4b0c004c80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4b10001750 tx=0x7f4b0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a03dd1f3f45c86a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:58.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.978+0000 7f4b24cce640 1 -- 172.21.15.43:0/652141861 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b10001370 con 0x7f4b0c004800 2024-04-25T08:56:58.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.978+0000 7f4b24cce640 1 -- 172.21.15.43:0/652141861 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b100052d0 con 0x7f4b0c004800 2024-04-25T08:56:58.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.978+0000 7f4b24cce640 1 -- 172.21.15.43:0/652141861 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1002abe0 con 0x7f4b0c004800 2024-04-25T08:56:58.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.979+0000 7f4b27f34640 1 -- 172.21.15.43:0/652141861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b0c004800 msgr2=0x7f4b0c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:58.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.979+0000 7f4b27f34640 1 --2- 172.21.15.43:0/652141861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b0c004800 0x7f4b0c004c80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f4b10001750 tx=0x7f4b0c005830 comp rx=0 tx=0).stop 2024-04-25T08:56:58.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.979+0000 7f4b27f34640 1 -- 172.21.15.43:0/652141861 shutdown_connections 2024-04-25T08:56:58.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.979+0000 7f4b27f34640 1 --2- 172.21.15.43:0/652141861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b0c004800 0x7f4b0c004c80 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:58.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.979+0000 7f4b27f34640 1 -- 172.21.15.43:0/652141861 >> 172.21.15.43:0/652141861 conn(0x7f4b200b1660 msgr2=0x7f4b200b1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:58.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.979+0000 7f4b27f34640 1 -- 172.21.15.43:0/652141861 shutdown_connections 2024-04-25T08:56:58.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.979+0000 7f4b27f34640 1 -- 172.21.15.43:0/652141861 wait complete. 2024-04-25T08:56:58.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.980+0000 7f4b27f34640 1 Processor -- start 2024-04-25T08:56:58.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.980+0000 7f4b27f34640 1 -- start start 2024-04-25T08:56:58.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.981+0000 7f4b27f34640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b0c004800 0x7f4b20154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:58.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.981+0000 7f4b27f34640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b2006d6f0 0x7f4b2006db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:58.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.981+0000 7f4b27f34640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4b201576d0 0x7f4b20157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:58.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.981+0000 7f4b27f34640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b200b2960 con 0x7f4b0c004800 2024-04-25T08:56:58.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.981+0000 7f4b27f34640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b200cdba0 con 0x7f4b2006d6f0 2024-04-25T08:56:58.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.981+0000 7f4b27f34640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b200b2ae0 con 0x7f4b201576d0 2024-04-25T08:56:58.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.981+0000 7f4b25cd0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b0c004800 0x7f4b20154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:58.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.981+0000 7f4b25cd0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b0c004800 0x7f4b20154ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57894/0 (socket says 172.21.15.43:57894) 2024-04-25T08:56:58.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.981+0000 7f4b25cd0640 1 -- 172.21.15.43:0/3144220241 learned_addr learned my addr 172.21.15.43:0/3144220241 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:58.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.981+0000 7f4b254cf640 1 --2- 172.21.15.43:0/3144220241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b2006d6f0 0x7f4b2006db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:58.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.981+0000 7f4b264d1640 1 --2- 172.21.15.43:0/3144220241 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4b201576d0 0x7f4b20157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:58.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.981+0000 7f4b25cd0640 1 -- 172.21.15.43:0/3144220241 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4b201576d0 msgr2=0x7f4b20157b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:58.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.981+0000 7f4b25cd0640 1 --2- 172.21.15.43:0/3144220241 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4b201576d0 0x7f4b20157b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:58.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.981+0000 7f4b25cd0640 1 -- 172.21.15.43:0/3144220241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b2006d6f0 msgr2=0x7f4b2006db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:58.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.981+0000 7f4b25cd0640 1 --2- 172.21.15.43:0/3144220241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b2006d6f0 0x7f4b2006db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:58.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.981+0000 7f4b25cd0640 1 -- 172.21.15.43:0/3144220241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b20158440 con 0x7f4b0c004800 2024-04-25T08:56:58.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.981+0000 7f4b25cd0640 1 --2- 172.21.15.43:0/3144220241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b0c004800 0x7f4b20154ea0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c0118c0 tx=0x7f4b1c011d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:58.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.981+0000 7f4b167fc640 1 -- 172.21.15.43:0/3144220241 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c019070 con 0x7f4b0c004800 2024-04-25T08:56:58.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.981+0000 7f4b167fc640 1 -- 172.21.15.43:0/3144220241 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b1c004240 con 0x7f4b0c004800 2024-04-25T08:56:58.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.981+0000 7f4b27f34640 1 -- 172.21.15.43:0/3144220241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b20158720 con 0x7f4b0c004800 2024-04-25T08:56:58.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.981+0000 7f4b167fc640 1 -- 172.21.15.43:0/3144220241 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c00f820 con 0x7f4b0c004800 2024-04-25T08:56:58.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.982+0000 7f4b27f34640 1 -- 172.21.15.43:0/3144220241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b20158ce0 con 0x7f4b0c004800 2024-04-25T08:56:58.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.982+0000 7f4b27f34640 1 -- 172.21.15.43:0/3144220241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b20074ef0 con 0x7f4b0c004800 2024-04-25T08:56:58.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.983+0000 7f4b167fc640 1 -- 172.21.15.43:0/3144220241 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4b1c01c070 con 0x7f4b0c004800 2024-04-25T08:56:58.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.983+0000 7f4b167fc640 1 --2- 172.21.15.43:0/3144220241 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4b0406b500 0x7f4b0406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:58.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.983+0000 7f4b167fc640 1 -- 172.21.15.43:0/3144220241 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4b1c08a260 con 0x7f4b0c004800 2024-04-25T08:56:58.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.983+0000 7f4b254cf640 1 --2- 172.21.15.43:0/3144220241 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4b0406b500 0x7f4b0406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:58.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.984+0000 7f4b254cf640 1 --2- 172.21.15.43:0/3144220241 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4b0406b500 0x7f4b0406d9c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4b10032000 tx=0x7f4b1002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:58.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:58.988+0000 7f4b167fc640 1 -- 172.21.15.43:0/3144220241 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4b1c0594c0 con 0x7f4b0c004800 2024-04-25T08:56:59.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.167+0000 7f4b27f34640 1 -- 172.21.15.43:0/3144220241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4b200b22d0 con 0x7f4b0c004800 2024-04-25T08:56:59.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.167+0000 7f4b167fc640 1 -- 172.21.15.43:0/3144220241 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+12855 (secure 0 0 0) 0x7f4b1c058dc0 con 0x7f4b0c004800 2024-04-25T08:56:59.157 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:56:59.157 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":42,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:56:58.474711+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:56:59.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.170+0000 7f4b27f34640 1 -- 172.21.15.43:0/3144220241 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4b0406b500 msgr2=0x7f4b0406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:59.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.170+0000 7f4b27f34640 1 --2- 172.21.15.43:0/3144220241 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4b0406b500 0x7f4b0406d9c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4b10032000 tx=0x7f4b1002e000 comp rx=0 tx=0).stop 2024-04-25T08:56:59.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.170+0000 7f4b27f34640 1 -- 172.21.15.43:0/3144220241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b0c004800 msgr2=0x7f4b20154ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:59.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.170+0000 7f4b27f34640 1 --2- 172.21.15.43:0/3144220241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b0c004800 0x7f4b20154ea0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c0118c0 tx=0x7f4b1c011d80 comp rx=0 tx=0).stop 2024-04-25T08:56:59.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.170+0000 7f4b27f34640 1 -- 172.21.15.43:0/3144220241 shutdown_connections 2024-04-25T08:56:59.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.170+0000 7f4b27f34640 1 --2- 172.21.15.43:0/3144220241 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4b201576d0 0x7f4b20157b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:59.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.170+0000 7f4b27f34640 1 --2- 172.21.15.43:0/3144220241 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4b0406b500 0x7f4b0406d9c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:59.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.170+0000 7f4b27f34640 1 --2- 172.21.15.43:0/3144220241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b2006d6f0 0x7f4b2006db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:59.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.170+0000 7f4b27f34640 1 --2- 172.21.15.43:0/3144220241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b0c004800 0x7f4b20154ea0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:59.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.170+0000 7f4b27f34640 1 -- 172.21.15.43:0/3144220241 >> 172.21.15.43:0/3144220241 conn(0x7f4b200b1660 msgr2=0x7f4b2015cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:59.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.170+0000 7f4b27f34640 1 -- 172.21.15.43:0/3144220241 shutdown_connections 2024-04-25T08:56:59.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.170+0000 7f4b27f34640 1 -- 172.21.15.43:0/3144220241 wait complete. 2024-04-25T08:56:59.172 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:56:59.172 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:56:59.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.287+0000 7f8d1c14f640 1 Processor -- start 2024-04-25T08:56:59.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.287+0000 7f8d1c14f640 1 -- start start 2024-04-25T08:56:59.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.288+0000 7f8d1c14f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8d140aee80 0x7f8d1406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:59.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.288+0000 7f8d1c14f640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f8d1406d600 0x7f8d140ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:59.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.288+0000 7f8d1c14f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d140cd540 con 0x7f8d140cd120 2024-04-25T08:56:59.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.288+0000 7f8d1c14f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d140af260 con 0x7f8d140aee80 2024-04-25T08:56:59.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.288+0000 7f8d1c14f640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d1415c280 con 0x7f8d1406d600 2024-04-25T08:56:59.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.288+0000 7f8d19eeb640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8d140aee80 0x7f8d1406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:59.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.288+0000 7f8d19eeb640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8d140aee80 0x7f8d1406d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57910/0 (socket says 172.21.15.43:57910) 2024-04-25T08:56:59.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.288+0000 7f8d19eeb640 1 -- 172.21.15.43:0/3225818682 learned_addr learned my addr 172.21.15.43:0/3225818682 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:59.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.288+0000 7f8d19eeb640 1 -- 172.21.15.43:0/3225818682 >> v2:172.21.15.193:3300/0 conn(0x7f8d1406d600 msgr2=0x7f8d140ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:59.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.288+0000 7f8d196ea640 1 --2- 172.21.15.43:0/3225818682 >> v2:172.21.15.193:3300/0 conn(0x7f8d1406d600 0x7f8d140ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:59.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.288+0000 7f8d18ee9640 1 -- 172.21.15.43:0/3225818682 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1089745358 0 0) 0x7f8d140cd540 con 0x7f8d140cd120 2024-04-25T08:56:59.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.288+0000 7f8d19eeb640 1 --2- 172.21.15.43:0/3225818682 >> v2:172.21.15.193:3300/0 conn(0x7f8d1406d600 0x7f8d140ccbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:59.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.288+0000 7f8d19eeb640 1 -- 172.21.15.43:0/3225818682 >> v1:172.21.15.43:6789/0 conn(0x7f8d140cd120 legacy=0x7f8d1415f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:59.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.288+0000 7f8d19eeb640 1 -- 172.21.15.43:0/3225818682 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d1415c560 con 0x7f8d140aee80 2024-04-25T08:56:59.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.289+0000 7f8d19eeb640 1 --2- 172.21.15.43:0/3225818682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d140aee80 0x7f8d1406d0c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f8d14067670 tx=0x7f8d1002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38f6418ffabc615a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:59.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.289+0000 7f8d18ee9640 1 -- 172.21.15.43:0/3225818682 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d10037070 con 0x7f8d140aee80 2024-04-25T08:56:59.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.289+0000 7f8d18ee9640 1 -- 172.21.15.43:0/3225818682 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d10032440 con 0x7f8d140aee80 2024-04-25T08:56:59.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.289+0000 7f8d18ee9640 1 -- 172.21.15.43:0/3225818682 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d10032810 con 0x7f8d140aee80 2024-04-25T08:56:59.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.290+0000 7f8d1c14f640 1 -- 172.21.15.43:0/3225818682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d140aee80 msgr2=0x7f8d1406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:59.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.290+0000 7f8d1c14f640 1 --2- 172.21.15.43:0/3225818682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d140aee80 0x7f8d1406d0c0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f8d14067670 tx=0x7f8d1002a270 comp rx=0 tx=0).stop 2024-04-25T08:56:59.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.290+0000 7f8d1c14f640 1 -- 172.21.15.43:0/3225818682 shutdown_connections 2024-04-25T08:56:59.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.290+0000 7f8d1c14f640 1 --2- 172.21.15.43:0/3225818682 >> v2:172.21.15.193:3300/0 conn(0x7f8d1406d600 0x7f8d140ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:59.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.290+0000 7f8d1c14f640 1 --2- 172.21.15.43:0/3225818682 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d140aee80 0x7f8d1406d0c0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:59.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.290+0000 7f8d1c14f640 1 -- 172.21.15.43:0/3225818682 >> 172.21.15.43:0/3225818682 conn(0x7f8d1406ab30 msgr2=0x7f8d1406af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:59.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.290+0000 7f8d1c14f640 1 -- 172.21.15.43:0/3225818682 shutdown_connections 2024-04-25T08:56:59.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.290+0000 7f8d1c14f640 1 -- 172.21.15.43:0/3225818682 wait complete. 2024-04-25T08:56:59.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.291+0000 7f8d1c14f640 1 Processor -- start 2024-04-25T08:56:59.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.291+0000 7f8d1c14f640 1 -- start start 2024-04-25T08:56:59.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.292+0000 7f8d1c14f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8d1406d600 0x7f8d141559a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:59.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.292+0000 7f8d1c14f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d140aee80 0x7f8d14157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:59.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.292+0000 7f8d1c14f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d140cd120 0x7f8d14144100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:59.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.292+0000 7f8d1c14f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1415e5a0 con 0x7f8d140cd120 2024-04-25T08:56:59.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.292+0000 7f8d1c14f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1415cb20 con 0x7f8d140aee80 2024-04-25T08:56:59.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.292+0000 7f8d1c14f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1415e720 con 0x7f8d1406d600 2024-04-25T08:56:59.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.292+0000 7f8d1a6ec640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d140cd120 0x7f8d14144100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:59.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.292+0000 7f8d1a6ec640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d140cd120 0x7f8d14144100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57914/0 (socket says 172.21.15.43:57914) 2024-04-25T08:56:59.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.292+0000 7f8d1a6ec640 1 -- 172.21.15.43:0/1096343790 learned_addr learned my addr 172.21.15.43:0/1096343790 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:59.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.292+0000 7f8d19eeb640 1 --2- 172.21.15.43:0/1096343790 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8d1406d600 0x7f8d141559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:59.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.292+0000 7f8d196ea640 1 --2- 172.21.15.43:0/1096343790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d140aee80 0x7f8d14157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:59.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.292+0000 7f8d1a6ec640 1 -- 172.21.15.43:0/1096343790 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8d1406d600 msgr2=0x7f8d141559a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:59.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.292+0000 7f8d1a6ec640 1 --2- 172.21.15.43:0/1096343790 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8d1406d600 0x7f8d141559a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:59.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.292+0000 7f8d1a6ec640 1 -- 172.21.15.43:0/1096343790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d140aee80 msgr2=0x7f8d14157020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:59.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.292+0000 7f8d1a6ec640 1 --2- 172.21.15.43:0/1096343790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d140aee80 0x7f8d14157020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:59.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.292+0000 7f8d1a6ec640 1 -- 172.21.15.43:0/1096343790 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d14155f80 con 0x7f8d140cd120 2024-04-25T08:56:59.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.292+0000 7f8d196ea640 1 --2- 172.21.15.43:0/1096343790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d140aee80 0x7f8d14157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:56:59.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.292+0000 7f8d1a6ec640 1 --2- 172.21.15.43:0/1096343790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d140cd120 0x7f8d14144100 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c00c780 tx=0x7f8d0c00cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:59.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.292+0000 7f8d0a7fc640 1 -- 172.21.15.43:0/1096343790 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c003b70 con 0x7f8d140cd120 2024-04-25T08:56:59.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.292+0000 7f8d0a7fc640 1 -- 172.21.15.43:0/1096343790 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d0c003d10 con 0x7f8d140cd120 2024-04-25T08:56:59.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.292+0000 7f8d0a7fc640 1 -- 172.21.15.43:0/1096343790 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c005270 con 0x7f8d140cd120 2024-04-25T08:56:59.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.292+0000 7f8d1c14f640 1 -- 172.21.15.43:0/1096343790 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d14156260 con 0x7f8d140cd120 2024-04-25T08:56:59.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.293+0000 7f8d1c14f640 1 -- 172.21.15.43:0/1096343790 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d14156820 con 0x7f8d140cd120 2024-04-25T08:56:59.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.294+0000 7f8d1c14f640 1 -- 172.21.15.43:0/1096343790 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d14074f60 con 0x7f8d140cd120 2024-04-25T08:56:59.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.294+0000 7f8d0a7fc640 1 -- 172.21.15.43:0/1096343790 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8d0c005410 con 0x7f8d140cd120 2024-04-25T08:56:59.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.294+0000 7f8d0a7fc640 1 --2- 172.21.15.43:0/1096343790 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8cf806b4d0 0x7f8cf806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:59.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.294+0000 7f8d0a7fc640 1 -- 172.21.15.43:0/1096343790 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8d0c089cf0 con 0x7f8d140cd120 2024-04-25T08:56:59.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.295+0000 7f8d19eeb640 1 --2- 172.21.15.43:0/1096343790 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8cf806b4d0 0x7f8cf806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:59.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.295+0000 7f8d19eeb640 1 --2- 172.21.15.43:0/1096343790 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8cf806b4d0 0x7f8cf806d990 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8d10034000 tx=0x7f8d1002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:59.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.299+0000 7f8d0a7fc640 1 -- 172.21.15.43:0/1096343790 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8d0c08e070 con 0x7f8d140cd120 2024-04-25T08:56:59.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.481+0000 7f8d1c14f640 1 -- 172.21.15.43:0/1096343790 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8d1406e1d0 con 0x7f8d140cd120 2024-04-25T08:56:59.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.481+0000 7f8d0a7fc640 1 -- 172.21.15.43:0/1096343790 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8d0c058ec0 con 0x7f8d140cd120 2024-04-25T08:56:59.471 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T08:56:59.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.483+0000 7f8d1c14f640 1 -- 172.21.15.43:0/1096343790 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8cf806b4d0 msgr2=0x7f8cf806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:59.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.483+0000 7f8d1c14f640 1 --2- 172.21.15.43:0/1096343790 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8cf806b4d0 0x7f8cf806d990 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8d10034000 tx=0x7f8d1002e000 comp rx=0 tx=0).stop 2024-04-25T08:56:59.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.483+0000 7f8d1c14f640 1 -- 172.21.15.43:0/1096343790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d140cd120 msgr2=0x7f8d14144100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:59.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.483+0000 7f8d1c14f640 1 --2- 172.21.15.43:0/1096343790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d140cd120 0x7f8d14144100 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c00c780 tx=0x7f8d0c00cb80 comp rx=0 tx=0).stop 2024-04-25T08:56:59.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.483+0000 7f8d1c14f640 1 -- 172.21.15.43:0/1096343790 shutdown_connections 2024-04-25T08:56:59.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.483+0000 7f8d1c14f640 1 --2- 172.21.15.43:0/1096343790 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d140cd120 0x7f8d14144100 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:59.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.483+0000 7f8d1c14f640 1 --2- 172.21.15.43:0/1096343790 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8cf806b4d0 0x7f8cf806d990 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:59.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.483+0000 7f8d1c14f640 1 --2- 172.21.15.43:0/1096343790 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d140aee80 0x7f8d14157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:59.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.483+0000 7f8d1c14f640 1 --2- 172.21.15.43:0/1096343790 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8d1406d600 0x7f8d141559a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:59.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.483+0000 7f8d1c14f640 1 -- 172.21.15.43:0/1096343790 >> 172.21.15.43:0/1096343790 conn(0x7f8d1406ab30 msgr2=0x7f8d140add50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:59.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.483+0000 7f8d1c14f640 1 -- 172.21.15.43:0/1096343790 shutdown_connections 2024-04-25T08:56:59.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.483+0000 7f8d1c14f640 1 -- 172.21.15.43:0/1096343790 wait complete. 2024-04-25T08:56:59.484 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T08:56:59.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.595+0000 7f6aa02e2640 1 Processor -- start 2024-04-25T08:56:59.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.595+0000 7f6aa02e2640 1 -- start start 2024-04-25T08:56:59.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.596+0000 7f6aa02e2640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6a9815dce0 0x7f6a9815ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:59.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.596+0000 7f6aa02e2640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a980ce550 0x7f6a980d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:59.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.596+0000 7f6aa02e2640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6a980ce990 0x7f6a980d0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:59.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.596+0000 7f6aa02e2640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a980cee80 con 0x7f6a980ce990 2024-04-25T08:56:59.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.596+0000 7f6aa02e2640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a980cf160 con 0x7f6a9815dce0 2024-04-25T08:56:59.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.596+0000 7f6aa02e2640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a980cf440 con 0x7f6a980ce550 2024-04-25T08:56:59.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.596+0000 7f6a9e87f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6a980ce990 0x7f6a980d0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:59.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.596+0000 7f6a9e87f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6a980ce990 0x7f6a980d0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57926/0 (socket says 172.21.15.43:57926) 2024-04-25T08:56:59.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.596+0000 7f6a9e87f640 1 -- 172.21.15.43:0/1647242479 learned_addr learned my addr 172.21.15.43:0/1647242479 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:59.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.596+0000 7f6a9e07e640 1 --2- 172.21.15.43:0/1647242479 >> v2:172.21.15.193:3300/0 conn(0x7f6a9815dce0 0x7f6a9815ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:59.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.596+0000 7f6a9d87d640 1 --2- 172.21.15.43:0/1647242479 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a980ce550 0x7f6a980d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:59.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.596+0000 7f6a9e87f640 1 -- 172.21.15.43:0/1647242479 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a980ce550 msgr2=0x7f6a980d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:59.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.596+0000 7f6a9e87f640 1 --2- 172.21.15.43:0/1647242479 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a980ce550 0x7f6a980d03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:59.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.596+0000 7f6a9e87f640 1 -- 172.21.15.43:0/1647242479 >> v2:172.21.15.193:3300/0 conn(0x7f6a9815dce0 msgr2=0x7f6a9815ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:59.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.596+0000 7f6a9e87f640 1 --2- 172.21.15.43:0/1647242479 >> v2:172.21.15.193:3300/0 conn(0x7f6a9815dce0 0x7f6a9815ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:59.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.596+0000 7f6a9e87f640 1 -- 172.21.15.43:0/1647242479 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a980cf720 con 0x7f6a980ce990 2024-04-25T08:56:59.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.596+0000 7f6a9d87d640 1 --2- 172.21.15.43:0/1647242479 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a980ce550 0x7f6a980d03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:56:59.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.597+0000 7f6a9e87f640 1 --2- 172.21.15.43:0/1647242479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a980ce990 0x7f6a980d0920 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f6a9809d3c0 tx=0x7f6a9002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13bf54489d6fcc5d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:59.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.597+0000 7f6a9d07c640 1 -- 172.21.15.43:0/1647242479 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a90036070 con 0x7f6a980ce990 2024-04-25T08:56:59.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.597+0000 7f6a9d07c640 1 -- 172.21.15.43:0/1647242479 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a90031600 con 0x7f6a980ce990 2024-04-25T08:56:59.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.597+0000 7f6a9d07c640 1 -- 172.21.15.43:0/1647242479 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a900319d0 con 0x7f6a980ce990 2024-04-25T08:56:59.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.598+0000 7f6aa02e2640 1 -- 172.21.15.43:0/1647242479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a980ce990 msgr2=0x7f6a980d0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:59.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.598+0000 7f6aa02e2640 1 --2- 172.21.15.43:0/1647242479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a980ce990 0x7f6a980d0920 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f6a9809d3c0 tx=0x7f6a9002a610 comp rx=0 tx=0).stop 2024-04-25T08:56:59.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.598+0000 7f6aa02e2640 1 -- 172.21.15.43:0/1647242479 shutdown_connections 2024-04-25T08:56:59.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.598+0000 7f6aa02e2640 1 --2- 172.21.15.43:0/1647242479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a980ce990 0x7f6a980d0920 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:59.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.598+0000 7f6aa02e2640 1 --2- 172.21.15.43:0/1647242479 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a980ce550 0x7f6a980d03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:59.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.598+0000 7f6aa02e2640 1 --2- 172.21.15.43:0/1647242479 >> v2:172.21.15.193:3300/0 conn(0x7f6a9815dce0 0x7f6a9815ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:59.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.598+0000 7f6aa02e2640 1 -- 172.21.15.43:0/1647242479 >> 172.21.15.43:0/1647242479 conn(0x7f6a9809cd00 msgr2=0x7f6a980d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:59.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.598+0000 7f6aa02e2640 1 -- 172.21.15.43:0/1647242479 shutdown_connections 2024-04-25T08:56:59.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.598+0000 7f6aa02e2640 1 -- 172.21.15.43:0/1647242479 wait complete. 2024-04-25T08:56:59.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.599+0000 7f6aa02e2640 1 Processor -- start 2024-04-25T08:56:59.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.599+0000 7f6aa02e2640 1 -- start start 2024-04-25T08:56:59.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.600+0000 7f6aa02e2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a980ce550 0x7f6a9815a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:59.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.600+0000 7f6aa02e2640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a980ce990 0x7f6a9809dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:59.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.600+0000 7f6aa02e2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a9815dce0 0x7f6a980d1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:59.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.600+0000 7f6aa02e2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a98160980 con 0x7f6a980ce550 2024-04-25T08:56:59.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.600+0000 7f6aa02e2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a9815e0c0 con 0x7f6a9815dce0 2024-04-25T08:56:59.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.600+0000 7f6aa02e2640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a98160b00 con 0x7f6a980ce990 2024-04-25T08:56:59.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.600+0000 7f6a9e07e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a980ce550 0x7f6a9815a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:59.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.600+0000 7f6a9e07e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a980ce550 0x7f6a9815a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57928/0 (socket says 172.21.15.43:57928) 2024-04-25T08:56:59.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.600+0000 7f6a9e07e640 1 -- 172.21.15.43:0/3220507664 learned_addr learned my addr 172.21.15.43:0/3220507664 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:59.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.600+0000 7f6a9e87f640 1 --2- 172.21.15.43:0/3220507664 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a9815dce0 0x7f6a980d1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:59.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.600+0000 7f6a9d87d640 1 --2- 172.21.15.43:0/3220507664 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a980ce990 0x7f6a9809dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:59.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.600+0000 7f6a9e07e640 1 -- 172.21.15.43:0/3220507664 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a980ce990 msgr2=0x7f6a9809dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:59.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.600+0000 7f6a9e07e640 1 --2- 172.21.15.43:0/3220507664 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a980ce990 0x7f6a9809dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:59.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.600+0000 7f6a9e07e640 1 -- 172.21.15.43:0/3220507664 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a9815dce0 msgr2=0x7f6a980d1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:59.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.600+0000 7f6a9e07e640 1 --2- 172.21.15.43:0/3220507664 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a9815dce0 0x7f6a980d1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:59.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.600+0000 7f6a9e07e640 1 -- 172.21.15.43:0/3220507664 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a9815b320 con 0x7f6a980ce550 2024-04-25T08:56:59.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.600+0000 7f6a9e87f640 1 --2- 172.21.15.43:0/3220507664 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a9815dce0 0x7f6a980d1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:59.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.600+0000 7f6a9d87d640 1 --2- 172.21.15.43:0/3220507664 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a980ce990 0x7f6a9809dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:56:59.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.600+0000 7f6a9e07e640 1 --2- 172.21.15.43:0/3220507664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a980ce550 0x7f6a9815a040 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6a9400e870 tx=0x7f6a9400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:59.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.601+0000 7f6a8effd640 1 -- 172.21.15.43:0/3220507664 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a94017070 con 0x7f6a980ce550 2024-04-25T08:56:59.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.601+0000 7f6a8effd640 1 -- 172.21.15.43:0/3220507664 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a9400c940 con 0x7f6a980ce550 2024-04-25T08:56:59.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.601+0000 7f6aa02e2640 1 -- 172.21.15.43:0/3220507664 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a9815b5e0 con 0x7f6a980ce550 2024-04-25T08:56:59.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.601+0000 7f6a8effd640 1 -- 172.21.15.43:0/3220507664 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a9400cd30 con 0x7f6a980ce550 2024-04-25T08:56:59.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.601+0000 7f6aa02e2640 1 -- 172.21.15.43:0/3220507664 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a9815bba0 con 0x7f6a980ce550 2024-04-25T08:56:59.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.602+0000 7f6aa02e2640 1 -- 172.21.15.43:0/3220507664 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a9809fe90 con 0x7f6a980ce550 2024-04-25T08:56:59.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.602+0000 7f6a8effd640 1 -- 172.21.15.43:0/3220507664 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6a9401a070 con 0x7f6a980ce550 2024-04-25T08:56:59.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.603+0000 7f6a8effd640 1 --2- 172.21.15.43:0/3220507664 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6a7c06b4d0 0x7f6a7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:59.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.603+0000 7f6a8effd640 1 -- 172.21.15.43:0/3220507664 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6a94089c70 con 0x7f6a980ce550 2024-04-25T08:56:59.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.603+0000 7f6a9d87d640 1 --2- 172.21.15.43:0/3220507664 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6a7c06b4d0 0x7f6a7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:59.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.603+0000 7f6a9d87d640 1 --2- 172.21.15.43:0/3220507664 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6a7c06b4d0 0x7f6a7c06d990 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6a88006fd0 tx=0x7f6a88008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:59.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.607+0000 7f6a8effd640 1 -- 172.21.15.43:0/3220507664 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6a94058ed0 con 0x7f6a980ce550 2024-04-25T08:56:59.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.791+0000 7f6aa02e2640 1 -- 172.21.15.43:0/3220507664 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f6a980680f0 con 0x7f6a980ce550 2024-04-25T08:56:59.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.792+0000 7f6a8effd640 1 -- 172.21.15.43:0/3220507664 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v42) v1 ==== 84+0+65 (secure 0 0 0) 0x7f6a940587d0 con 0x7f6a980ce550 2024-04-25T08:56:59.781 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:56:59.781 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2024-04-25T08:56:59.781 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:56:59.781 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 1, 2024-04-25T08:56:59.782 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": ".mgr" 2024-04-25T08:56:59.782 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:56:59.782 INFO:teuthology.orchestra.run.smithi043.stdout:] 2024-04-25T08:56:59.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.794+0000 7f6aa02e2640 1 -- 172.21.15.43:0/3220507664 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6a7c06b4d0 msgr2=0x7f6a7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:59.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.794+0000 7f6aa02e2640 1 --2- 172.21.15.43:0/3220507664 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6a7c06b4d0 0x7f6a7c06d990 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6a88006fd0 tx=0x7f6a88008040 comp rx=0 tx=0).stop 2024-04-25T08:56:59.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.794+0000 7f6aa02e2640 1 -- 172.21.15.43:0/3220507664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a980ce550 msgr2=0x7f6a9815a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:59.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.794+0000 7f6aa02e2640 1 --2- 172.21.15.43:0/3220507664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a980ce550 0x7f6a9815a040 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6a9400e870 tx=0x7f6a9400ed30 comp rx=0 tx=0).stop 2024-04-25T08:56:59.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.794+0000 7f6aa02e2640 1 -- 172.21.15.43:0/3220507664 shutdown_connections 2024-04-25T08:56:59.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.794+0000 7f6aa02e2640 1 --2- 172.21.15.43:0/3220507664 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6a7c06b4d0 0x7f6a7c06d990 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:59.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.794+0000 7f6aa02e2640 1 --2- 172.21.15.43:0/3220507664 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a9815dce0 0x7f6a980d1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:59.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.794+0000 7f6aa02e2640 1 --2- 172.21.15.43:0/3220507664 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a980ce990 0x7f6a9809dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:59.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.794+0000 7f6aa02e2640 1 --2- 172.21.15.43:0/3220507664 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a980ce550 0x7f6a9815a040 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:59.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.794+0000 7f6aa02e2640 1 -- 172.21.15.43:0/3220507664 >> 172.21.15.43:0/3220507664 conn(0x7f6a9809cd00 msgr2=0x7f6a9806b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:59.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.794+0000 7f6aa02e2640 1 -- 172.21.15.43:0/3220507664 shutdown_connections 2024-04-25T08:56:59.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.794+0000 7f6aa02e2640 1 -- 172.21.15.43:0/3220507664 wait complete. 2024-04-25T08:56:59.794 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T08:56:59.795 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T08:56:59.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.910+0000 7f55496cb640 1 Processor -- start 2024-04-25T08:56:59.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.910+0000 7f55496cb640 1 -- start start 2024-04-25T08:56:59.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.911+0000 7f55496cb640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f554415db50 0x7f554409ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:59.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.911+0000 7f55496cb640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f554406ccd0 0x7f55440b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:59.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.911+0000 7f55496cb640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55440b5ce0 con 0x7f55440b0cf0 2024-04-25T08:56:59.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.911+0000 7f55496cb640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55440b1540 con 0x7f554415db50 2024-04-25T08:56:59.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.911+0000 7f55496cb640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55440b5f80 con 0x7f554406ccd0 2024-04-25T08:56:59.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.911+0000 7f55427fc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f55440b0cf0 0x7f554406c5c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48064/0 (socket says 172.21.15.43:48064) 2024-04-25T08:56:59.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.911+0000 7f55427fc640 1 -- 172.21.15.43:0/2334802380 learned_addr learned my addr 172.21.15.43:0/2334802380 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:56:59.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.911+0000 7f55437fe640 1 --2- 172.21.15.43:0/2334802380 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f554406ccd0 0x7f55440b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:59.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.911+0000 7f5542ffd640 1 --2- 172.21.15.43:0/2334802380 >> v2:172.21.15.193:3300/0 conn(0x7f554415db50 0x7f554409ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:59.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.911+0000 7f5541ffb640 1 -- 172.21.15.43:0/2334802380 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 169546745 0 0) 0x7f55440b5ce0 con 0x7f55440b0cf0 2024-04-25T08:56:59.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.911+0000 7f5541ffb640 1 -- 172.21.15.43:0/2334802380 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f552c003520 con 0x7f55440b0cf0 2024-04-25T08:56:59.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.912+0000 7f5541ffb640 1 -- 172.21.15.43:0/2334802380 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2756759346 0 0) 0x7f552c003520 con 0x7f55440b0cf0 2024-04-25T08:56:59.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.912+0000 7f5541ffb640 1 -- 172.21.15.43:0/2334802380 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f554406ccd0 msgr2=0x7f55440b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:59.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.912+0000 7f5541ffb640 1 --2- 172.21.15.43:0/2334802380 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f554406ccd0 0x7f55440b19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:59.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.912+0000 7f5541ffb640 1 -- 172.21.15.43:0/2334802380 >> v2:172.21.15.193:3300/0 conn(0x7f554415db50 msgr2=0x7f554409ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:59.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.912+0000 7f5541ffb640 1 --2- 172.21.15.43:0/2334802380 >> v2:172.21.15.193:3300/0 conn(0x7f554415db50 0x7f554409ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:59.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.912+0000 7f5541ffb640 1 -- 172.21.15.43:0/2334802380 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55440b6260 con 0x7f55440b0cf0 2024-04-25T08:56:59.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.912+0000 7f5541ffb640 1 -- 172.21.15.43:0/2334802380 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f5538001690 con 0x7f55440b0cf0 2024-04-25T08:56:59.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.912+0000 7f5541ffb640 1 -- 172.21.15.43:0/2334802380 >> v1:172.21.15.43:6789/0 conn(0x7f55440b0cf0 legacy=0x7f554406c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:59.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.912+0000 7f5541ffb640 1 --2- 172.21.15.43:0/2334802380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f552c0039b0 0x7f552c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:59.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.912+0000 7f5541ffb640 1 --2- 172.21.15.43:0/2334802380 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f554406ccd0 0x7f552c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.912+0000 7f5541ffb640 1 --2- 172.21.15.43:0/2334802380 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f552c004d40 0x7f552c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.912+0000 7f5541ffb640 1 -- 172.21.15.43:0/2334802380 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55440b1540 con 0x7f552c0039b0 2024-04-25T08:56:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.912+0000 7f5541ffb640 1 -- 172.21.15.43:0/2334802380 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55440b5f80 con 0x7f552c004d40 2024-04-25T08:56:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.912+0000 7f5541ffb640 1 -- 172.21.15.43:0/2334802380 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f552c00b9c0 con 0x7f554406ccd0 2024-04-25T08:56:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.912+0000 7f5542ffd640 1 --2- 172.21.15.43:0/2334802380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f552c0039b0 0x7f552c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.912+0000 7f55427fc640 1 --2- 172.21.15.43:0/2334802380 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f554406ccd0 0x7f552c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.912+0000 7f55437fe640 1 --2- 172.21.15.43:0/2334802380 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f552c004d40 0x7f552c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.912+0000 7f5542ffd640 1 -- 172.21.15.43:0/2334802380 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f554406ccd0 msgr2=0x7f552c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.912+0000 7f5542ffd640 1 --2- 172.21.15.43:0/2334802380 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f554406ccd0 0x7f552c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.912+0000 7f5542ffd640 1 -- 172.21.15.43:0/2334802380 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f552c004d40 msgr2=0x7f552c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:56:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.912+0000 7f5542ffd640 1 --2- 172.21.15.43:0/2334802380 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f552c004d40 0x7f552c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.912+0000 7f5542ffd640 1 -- 172.21.15.43:0/2334802380 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f552c003520 con 0x7f552c0039b0 2024-04-25T08:56:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.912+0000 7f55427fc640 1 --2- 172.21.15.43:0/2334802380 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f554406ccd0 0x7f552c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:56:59.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.913+0000 7f5542ffd640 1 --2- 172.21.15.43:0/2334802380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f552c0039b0 0x7f552c003dd0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f5534009af0 tx=0x7f552c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf5579d02c0e0b49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.913+0000 7f5541ffb640 1 -- 172.21.15.43:0/2334802380 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5534036070 con 0x7f552c0039b0 2024-04-25T08:56:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.913+0000 7f5541ffb640 1 -- 172.21.15.43:0/2334802380 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5534031400 con 0x7f552c0039b0 2024-04-25T08:56:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.913+0000 7f5541ffb640 1 -- 172.21.15.43:0/2334802380 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5534009690 con 0x7f552c0039b0 2024-04-25T08:56:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.913+0000 7f55496cb640 1 -- 172.21.15.43:0/2334802380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f552c0039b0 msgr2=0x7f552c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.913+0000 7f55496cb640 1 --2- 172.21.15.43:0/2334802380 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f552c0039b0 0x7f552c003dd0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f5534009af0 tx=0x7f552c005830 comp rx=0 tx=0).stop 2024-04-25T08:56:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.913+0000 7f55437fe640 1 -- 172.21.15.43:0/2334802380 reap_dead start 2024-04-25T08:56:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.913+0000 7f55496cb640 1 -- 172.21.15.43:0/2334802380 shutdown_connections 2024-04-25T08:56:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.913+0000 7f55496cb640 1 -- 172.21.15.43:0/2334802380 >> 172.21.15.43:0/2334802380 conn(0x7f554409aa70 msgr2=0x7f55440b6be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:56:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.914+0000 7f55496cb640 1 -- 172.21.15.43:0/2334802380 shutdown_connections 2024-04-25T08:56:59.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.914+0000 7f55496cb640 1 -- 172.21.15.43:0/2334802380 wait complete. 2024-04-25T08:56:59.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.915+0000 7f55496cb640 1 Processor -- start 2024-04-25T08:56:59.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.915+0000 7f55496cb640 1 -- start start 2024-04-25T08:56:59.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.916+0000 7f55496cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f552c0039b0 0x7f5544144120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:59.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.916+0000 7f55496cb640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5544150580 0x7f55441509e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:59.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.916+0000 7f55496cb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f554406f6f0 0x7f554406fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:59.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.916+0000 7f55496cb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554415df30 con 0x7f552c0039b0 2024-04-25T08:56:59.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.916+0000 7f55496cb640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55440b11d0 con 0x7f554406f6f0 2024-04-25T08:56:59.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.916+0000 7f55496cb640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f554415e0b0 con 0x7f5544150580 2024-04-25T08:56:59.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.916+0000 7f5542ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f552c0039b0 0x7f5544144120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:59.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.916+0000 7f5542ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f552c0039b0 0x7f5544144120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39512/0 (socket says 172.21.15.43:39512) 2024-04-25T08:56:59.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.916+0000 7f5542ffd640 1 -- 172.21.15.43:0/2341658614 learned_addr learned my addr 172.21.15.43:0/2341658614 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:56:59.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.916+0000 7f55427fc640 1 --2- 172.21.15.43:0/2341658614 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5544150580 0x7f55441509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.916+0000 7f55437fe640 1 --2- 172.21.15.43:0/2341658614 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f554406f6f0 0x7f554406fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.916+0000 7f5542ffd640 1 -- 172.21.15.43:0/2341658614 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5544150580 msgr2=0x7f55441509e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.916+0000 7f5542ffd640 1 --2- 172.21.15.43:0/2341658614 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5544150580 0x7f55441509e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.916+0000 7f5542ffd640 1 -- 172.21.15.43:0/2341658614 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f554406f6f0 msgr2=0x7f554406fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:56:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.916+0000 7f5542ffd640 1 --2- 172.21.15.43:0/2341658614 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f554406f6f0 0x7f554406fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:56:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.916+0000 7f5542ffd640 1 -- 172.21.15.43:0/2341658614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55441603f0 con 0x7f552c0039b0 2024-04-25T08:56:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.916+0000 7f5542ffd640 1 --2- 172.21.15.43:0/2341658614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f552c0039b0 0x7f5544144120 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5534034040 tx=0x7f5534004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.916+0000 7f552b7fe640 1 -- 172.21.15.43:0/2341658614 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5534040020 con 0x7f552c0039b0 2024-04-25T08:56:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.916+0000 7f552b7fe640 1 -- 172.21.15.43:0/2341658614 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5534003830 con 0x7f552c0039b0 2024-04-25T08:56:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.916+0000 7f552b7fe640 1 -- 172.21.15.43:0/2341658614 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5534036040 con 0x7f552c0039b0 2024-04-25T08:56:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.916+0000 7f55496cb640 1 -- 172.21.15.43:0/2341658614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55441606d0 con 0x7f552c0039b0 2024-04-25T08:56:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.917+0000 7f55496cb640 1 -- 172.21.15.43:0/2341658614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5544160c90 con 0x7f552c0039b0 2024-04-25T08:56:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.918+0000 7f55496cb640 1 -- 172.21.15.43:0/2341658614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5544068a40 con 0x7f552c0039b0 2024-04-25T08:56:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.918+0000 7f552b7fe640 1 -- 172.21.15.43:0/2341658614 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5534043020 con 0x7f552c0039b0 2024-04-25T08:56:59.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.918+0000 7f552b7fe640 1 --2- 172.21.15.43:0/2341658614 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f553006b4d0 0x7f553006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:56:59.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.919+0000 7f552b7fe640 1 -- 172.21.15.43:0/2341658614 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f55340a9e90 con 0x7f552c0039b0 2024-04-25T08:56:59.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.919+0000 7f55427fc640 1 --2- 172.21.15.43:0/2341658614 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f553006b4d0 0x7f553006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:56:59.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.919+0000 7f55427fc640 1 --2- 172.21.15.43:0/2341658614 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f553006b4d0 0x7f553006d990 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f554414f840 tx=0x7f5538007040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:56:59.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:56:59.923+0000 7f552b7fe640 1 -- 172.21.15.43:0/2341658614 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f553404a070 con 0x7f552c0039b0 2024-04-25T08:57:00.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.104+0000 7f55496cb640 1 -- 172.21.15.43:0/2341658614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f554406dc30 con 0x7f552c0039b0 2024-04-25T08:57:00.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.478+0000 7f552b7fe640 1 -- 172.21.15.43:0/2341658614 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v43) v1 ==== 138+0+0 (secure 0 0 0) 0x7f55340ab0b0 con 0x7f552c0039b0 2024-04-25T08:57:00.467 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' created 2024-04-25T08:57:00.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.480+0000 7f55496cb640 1 -- 172.21.15.43:0/2341658614 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f553006b4d0 msgr2=0x7f553006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:00.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.480+0000 7f55496cb640 1 --2- 172.21.15.43:0/2341658614 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f553006b4d0 0x7f553006d990 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f554414f840 tx=0x7f5538007040 comp rx=0 tx=0).stop 2024-04-25T08:57:00.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.480+0000 7f55496cb640 1 -- 172.21.15.43:0/2341658614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f552c0039b0 msgr2=0x7f5544144120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:00.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.480+0000 7f55496cb640 1 --2- 172.21.15.43:0/2341658614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f552c0039b0 0x7f5544144120 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5534034040 tx=0x7f5534004730 comp rx=0 tx=0).stop 2024-04-25T08:57:00.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.480+0000 7f55496cb640 1 -- 172.21.15.43:0/2341658614 shutdown_connections 2024-04-25T08:57:00.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.480+0000 7f55496cb640 1 --2- 172.21.15.43:0/2341658614 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f553006b4d0 0x7f553006d990 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:00.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.480+0000 7f55496cb640 1 --2- 172.21.15.43:0/2341658614 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f554406f6f0 0x7f554406fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:00.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.480+0000 7f55496cb640 1 --2- 172.21.15.43:0/2341658614 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5544150580 0x7f55441509e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:00.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.480+0000 7f55496cb640 1 --2- 172.21.15.43:0/2341658614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f552c0039b0 0x7f5544144120 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:00.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.480+0000 7f55496cb640 1 -- 172.21.15.43:0/2341658614 >> 172.21.15.43:0/2341658614 conn(0x7f554409aa70 msgr2=0x7f5544158500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:00.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.480+0000 7f55496cb640 1 -- 172.21.15.43:0/2341658614 shutdown_connections 2024-04-25T08:57:00.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.480+0000 7f55496cb640 1 -- 172.21.15.43:0/2341658614 wait complete. 2024-04-25T08:57:00.485 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T08:57:00.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.605+0000 7f94a5c93640 1 Processor -- start 2024-04-25T08:57:00.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.605+0000 7f94a5c93640 1 -- start start 2024-04-25T08:57:00.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.605+0000 7f94a5c93640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f94a00d0190 0x7f94a00d0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:00.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.605+0000 7f94a5c93640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f94a00cf600 0x7f94a006bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:00.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.605+0000 7f94a5c93640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94a00d2310 con 0x7f94a00d0ab0 2024-04-25T08:57:00.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.606+0000 7f94a5c93640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94a00cfa30 con 0x7f94a00d0190 2024-04-25T08:57:00.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.606+0000 7f94a5c93640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94a006b3b0 con 0x7f94a00cf600 2024-04-25T08:57:00.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.606+0000 7f949f7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f94a00d0190 0x7f94a00d0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:00.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.606+0000 7f949f7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f94a00d0190 0x7f94a00d0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39520/0 (socket says 172.21.15.43:39520) 2024-04-25T08:57:00.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.606+0000 7f949f7fe640 1 -- 172.21.15.43:0/1264777832 learned_addr learned my addr 172.21.15.43:0/1264777832 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:00.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.606+0000 7f949ffff640 1 --2- 172.21.15.43:0/1264777832 >> v2:172.21.15.193:3300/0 conn(0x7f94a00cf600 0x7f94a006bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:00.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.606+0000 7f949f7fe640 1 -- 172.21.15.43:0/1264777832 >> v2:172.21.15.193:3300/0 conn(0x7f94a00cf600 msgr2=0x7f94a006bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:00.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.606+0000 7f949f7fe640 1 --2- 172.21.15.43:0/1264777832 >> v2:172.21.15.193:3300/0 conn(0x7f94a00cf600 0x7f94a006bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:00.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.606+0000 7f949f7fe640 1 -- 172.21.15.43:0/1264777832 >> v1:172.21.15.43:6789/0 conn(0x7f94a00d0ab0 legacy=0x7f94a006b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:00.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.606+0000 7f949f7fe640 1 -- 172.21.15.43:0/1264777832 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a00d25f0 con 0x7f94a00d0190 2024-04-25T08:57:00.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.606+0000 7f949e7fc640 1 -- 172.21.15.43:0/1264777832 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1369686252 0 0) 0x7f94a00d2310 con 0x7f94a00d0ab0 2024-04-25T08:57:00.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.607+0000 7f949f7fe640 1 --2- 172.21.15.43:0/1264777832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94a00d0190 0x7f94a00d0570 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f94a008cea0 tx=0x7f949002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ec0c39c27d21259 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:00.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.607+0000 7f949e7fc640 1 -- 172.21.15.43:0/1264777832 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9490037070 con 0x7f94a00d0190 2024-04-25T08:57:00.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.607+0000 7f949e7fc640 1 -- 172.21.15.43:0/1264777832 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9490032440 con 0x7f94a00d0190 2024-04-25T08:57:00.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.607+0000 7f949e7fc640 1 -- 172.21.15.43:0/1264777832 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9490032810 con 0x7f94a00d0190 2024-04-25T08:57:00.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.607+0000 7f94a5c93640 1 -- 172.21.15.43:0/1264777832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94a00d0190 msgr2=0x7f94a00d0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:00.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.607+0000 7f94a5c93640 1 --2- 172.21.15.43:0/1264777832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94a00d0190 0x7f94a00d0570 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f94a008cea0 tx=0x7f949002a1e0 comp rx=0 tx=0).stop 2024-04-25T08:57:00.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.608+0000 7f94a5c93640 1 -- 172.21.15.43:0/1264777832 shutdown_connections 2024-04-25T08:57:00.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.608+0000 7f94a5c93640 1 --2- 172.21.15.43:0/1264777832 >> v2:172.21.15.193:3300/0 conn(0x7f94a00cf600 0x7f94a006bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:00.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.608+0000 7f94a5c93640 1 --2- 172.21.15.43:0/1264777832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94a00d0190 0x7f94a00d0570 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:00.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.608+0000 7f94a5c93640 1 -- 172.21.15.43:0/1264777832 >> 172.21.15.43:0/1264777832 conn(0x7f94a00d39e0 msgr2=0x7f94a00d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:00.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.608+0000 7f94a5c93640 1 -- 172.21.15.43:0/1264777832 shutdown_connections 2024-04-25T08:57:00.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.608+0000 7f94a5c93640 1 -- 172.21.15.43:0/1264777832 wait complete. 2024-04-25T08:57:00.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.609+0000 7f94a5c93640 1 Processor -- start 2024-04-25T08:57:00.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.609+0000 7f94a5c93640 1 -- start start 2024-04-25T08:57:00.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.610+0000 7f94a5c93640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94a00cf600 0x7f94a015a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:00.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.610+0000 7f94a5c93640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f94a00d0190 0x7f94a00cd510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:00.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.610+0000 7f94a5c93640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94a00d0ab0 0x7f94a0157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:00.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.610+0000 7f94a5c93640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a00d0f30 con 0x7f94a00d0ab0 2024-04-25T08:57:00.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.610+0000 7f94a5c93640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a006b040 con 0x7f94a00cf600 2024-04-25T08:57:00.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.610+0000 7f94a5c93640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a00d1f90 con 0x7f94a00d0190 2024-04-25T08:57:00.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.610+0000 7f949ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94a00d0ab0 0x7f94a0157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:00.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.610+0000 7f949ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94a00d0ab0 0x7f94a0157b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39528/0 (socket says 172.21.15.43:39528) 2024-04-25T08:57:00.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.610+0000 7f949ffff640 1 -- 172.21.15.43:0/1974430341 learned_addr learned my addr 172.21.15.43:0/1974430341 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:00.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.610+0000 7f949f7fe640 1 --2- 172.21.15.43:0/1974430341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94a00cf600 0x7f94a015a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:00.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.610+0000 7f949effd640 1 --2- 172.21.15.43:0/1974430341 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f94a00d0190 0x7f94a00cd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:00.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.610+0000 7f949ffff640 1 -- 172.21.15.43:0/1974430341 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f94a00d0190 msgr2=0x7f94a00cd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:00.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.610+0000 7f949ffff640 1 --2- 172.21.15.43:0/1974430341 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f94a00d0190 0x7f94a00cd510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:00.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.610+0000 7f949ffff640 1 -- 172.21.15.43:0/1974430341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94a00cf600 msgr2=0x7f94a015a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:00.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.610+0000 7f949ffff640 1 --2- 172.21.15.43:0/1974430341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94a00cf600 0x7f94a015a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:00.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.610+0000 7f949ffff640 1 -- 172.21.15.43:0/1974430341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a00cb560 con 0x7f94a00d0ab0 2024-04-25T08:57:00.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.610+0000 7f949ffff640 1 --2- 172.21.15.43:0/1974430341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94a00d0ab0 0x7f94a0157b20 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f9498007e40 tx=0x7f949800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:00.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.611+0000 7f9487fff640 1 -- 172.21.15.43:0/1974430341 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9498014620 con 0x7f94a00d0ab0 2024-04-25T08:57:00.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.611+0000 7f9487fff640 1 -- 172.21.15.43:0/1974430341 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9498019070 con 0x7f94a00d0ab0 2024-04-25T08:57:00.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.611+0000 7f94a5c93640 1 -- 172.21.15.43:0/1974430341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a00cb840 con 0x7f94a00d0ab0 2024-04-25T08:57:00.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.611+0000 7f9487fff640 1 -- 172.21.15.43:0/1974430341 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9498014a20 con 0x7f94a00d0ab0 2024-04-25T08:57:00.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.611+0000 7f94a5c93640 1 -- 172.21.15.43:0/1974430341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a00cbe00 con 0x7f94a00d0ab0 2024-04-25T08:57:00.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.612+0000 7f94a5c93640 1 -- 172.21.15.43:0/1974430341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94a0074f60 con 0x7f94a00d0ab0 2024-04-25T08:57:00.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.612+0000 7f9487fff640 1 -- 172.21.15.43:0/1974430341 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9498014bc0 con 0x7f94a00d0ab0 2024-04-25T08:57:00.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.613+0000 7f9487fff640 1 --2- 172.21.15.43:0/1974430341 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f948c06b4d0 0x7f948c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:00.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.613+0000 7f9487fff640 1 -- 172.21.15.43:0/1974430341 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f949808a750 con 0x7f94a00d0ab0 2024-04-25T08:57:00.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.613+0000 7f949f7fe640 1 --2- 172.21.15.43:0/1974430341 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f948c06b4d0 0x7f948c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:00.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.614+0000 7f949f7fe640 1 --2- 172.21.15.43:0/1974430341 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f948c06b4d0 0x7f948c06d990 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9490034000 tx=0x7f949002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:00.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.617+0000 7f9487fff640 1 -- 172.21.15.43:0/1974430341 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f94980591f0 con 0x7f94a00d0ab0 2024-04-25T08:57:00.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:00.798+0000 7f94a5c93640 1 -- 172.21.15.43:0/1974430341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f94a006c650 con 0x7f94a00d0ab0 2024-04-25T08:57:01.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.480+0000 7f9487fff640 1 -- 172.21.15.43:0/1974430341 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v44) v1 ==== 170+0+0 (secure 0 0 0) 0x7f949805f0d0 con 0x7f94a00d0ab0 2024-04-25T08:57:01.470 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' created 2024-04-25T08:57:01.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.484+0000 7f94a5c93640 1 -- 172.21.15.43:0/1974430341 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f948c06b4d0 msgr2=0x7f948c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:01.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.484+0000 7f94a5c93640 1 --2- 172.21.15.43:0/1974430341 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f948c06b4d0 0x7f948c06d990 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9490034000 tx=0x7f949002e000 comp rx=0 tx=0).stop 2024-04-25T08:57:01.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.484+0000 7f94a5c93640 1 -- 172.21.15.43:0/1974430341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94a00d0ab0 msgr2=0x7f94a0157b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:01.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.484+0000 7f94a5c93640 1 --2- 172.21.15.43:0/1974430341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94a00d0ab0 0x7f94a0157b20 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f9498007e40 tx=0x7f949800e4b0 comp rx=0 tx=0).stop 2024-04-25T08:57:01.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.484+0000 7f94a5c93640 1 -- 172.21.15.43:0/1974430341 shutdown_connections 2024-04-25T08:57:01.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.484+0000 7f94a5c93640 1 --2- 172.21.15.43:0/1974430341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94a00d0ab0 0x7f94a0157b20 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:01.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.484+0000 7f94a5c93640 1 --2- 172.21.15.43:0/1974430341 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f94a00d0190 0x7f94a00cd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:01.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.484+0000 7f94a5c93640 1 --2- 172.21.15.43:0/1974430341 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f948c06b4d0 0x7f948c06d990 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:01.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.484+0000 7f94a5c93640 1 --2- 172.21.15.43:0/1974430341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94a00cf600 0x7f94a015a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:01.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.484+0000 7f94a5c93640 1 -- 172.21.15.43:0/1974430341 >> 172.21.15.43:0/1974430341 conn(0x7f94a00d39e0 msgr2=0x7f94a00b25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:01.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.485+0000 7f94a5c93640 1 -- 172.21.15.43:0/1974430341 shutdown_connections 2024-04-25T08:57:01.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.485+0000 7f94a5c93640 1 -- 172.21.15.43:0/1974430341 wait complete. 2024-04-25T08:57:01.501 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T08:57:01.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.620+0000 7f993d441640 1 Processor -- start 2024-04-25T08:57:01.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.620+0000 7f993d441640 1 -- start start 2024-04-25T08:57:01.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.621+0000 7f993d441640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f993815e090 0x7f9938161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:01.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.621+0000 7f993d441640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f99380b5c70 0x7f99380d2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:01.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.621+0000 7f993d441640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9938162610 con 0x7f9938161fe0 2024-04-25T08:57:01.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.621+0000 7f993d441640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99380d1da0 con 0x7f993815e090 2024-04-25T08:57:01.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.621+0000 7f993d441640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99380d1750 con 0x7f99380b5c70 2024-04-25T08:57:01.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.621+0000 7f9936ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f993815e090 0x7f9938161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:01.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.621+0000 7f9936ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f993815e090 0x7f9938161aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39534/0 (socket says 172.21.15.43:39534) 2024-04-25T08:57:01.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.621+0000 7f99367fc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f9938161fe0 0x7f99380b5560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48090/0 (socket says 172.21.15.43:48090) 2024-04-25T08:57:01.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.621+0000 7f9936ffd640 1 -- 172.21.15.43:0/1631014365 learned_addr learned my addr 172.21.15.43:0/1631014365 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:01.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.621+0000 7f99377fe640 1 --2- 172.21.15.43:0/1631014365 >> v2:172.21.15.193:3300/0 conn(0x7f99380b5c70 0x7f99380d2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:01.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.621+0000 7f9936ffd640 1 -- 172.21.15.43:0/1631014365 >> v2:172.21.15.193:3300/0 conn(0x7f99380b5c70 msgr2=0x7f99380d2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:01.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.621+0000 7f9936ffd640 1 --2- 172.21.15.43:0/1631014365 >> v2:172.21.15.193:3300/0 conn(0x7f99380b5c70 0x7f99380d2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:01.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.621+0000 7f9936ffd640 1 -- 172.21.15.43:0/1631014365 >> v1:172.21.15.43:6789/0 conn(0x7f9938161fe0 legacy=0x7f99380b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:01.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.621+0000 7f9936ffd640 1 -- 172.21.15.43:0/1631014365 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99380d1a30 con 0x7f993815e090 2024-04-25T08:57:01.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.621+0000 7f9935ffb640 1 -- 172.21.15.43:0/1631014365 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 581832323 0 0) 0x7f9938162610 con 0x7f9938161fe0 2024-04-25T08:57:01.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.622+0000 7f9936ffd640 1 --2- 172.21.15.43:0/1631014365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f993815e090 0x7f9938161aa0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f99381619e0 tx=0x7f992802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0d01976f932b87d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:01.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.622+0000 7f9935ffb640 1 -- 172.21.15.43:0/1631014365 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9928037070 con 0x7f993815e090 2024-04-25T08:57:01.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.622+0000 7f9935ffb640 1 -- 172.21.15.43:0/1631014365 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992802ae50 con 0x7f993815e090 2024-04-25T08:57:01.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.622+0000 7f9935ffb640 1 -- 172.21.15.43:0/1631014365 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9928032860 con 0x7f993815e090 2024-04-25T08:57:01.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.623+0000 7f993d441640 1 -- 172.21.15.43:0/1631014365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f993815e090 msgr2=0x7f9938161aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:01.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.623+0000 7f993d441640 1 --2- 172.21.15.43:0/1631014365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f993815e090 0x7f9938161aa0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f99381619e0 tx=0x7f992802a300 comp rx=0 tx=0).stop 2024-04-25T08:57:01.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.623+0000 7f993d441640 1 -- 172.21.15.43:0/1631014365 shutdown_connections 2024-04-25T08:57:01.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.623+0000 7f993d441640 1 --2- 172.21.15.43:0/1631014365 >> v2:172.21.15.193:3300/0 conn(0x7f99380b5c70 0x7f99380d2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:01.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.623+0000 7f993d441640 1 --2- 172.21.15.43:0/1631014365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f993815e090 0x7f9938161aa0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:01.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.623+0000 7f993d441640 1 -- 172.21.15.43:0/1631014365 >> 172.21.15.43:0/1631014365 conn(0x7f99380d0560 msgr2=0x7f99380d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:01.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.623+0000 7f993d441640 1 -- 172.21.15.43:0/1631014365 shutdown_connections 2024-04-25T08:57:01.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.623+0000 7f993d441640 1 -- 172.21.15.43:0/1631014365 wait complete. 2024-04-25T08:57:01.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.624+0000 7f993d441640 1 Processor -- start 2024-04-25T08:57:01.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.624+0000 7f993d441640 1 -- start start 2024-04-25T08:57:01.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.625+0000 7f993d441640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99380b5c70 0x7f993815fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:01.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.625+0000 7f993d441640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f993815e090 0x7f99381f3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:01.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.625+0000 7f993d441640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9938161fe0 0x7f99381f4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:01.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.625+0000 7f993d441640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99380d2770 con 0x7f99380b5c70 2024-04-25T08:57:01.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.625+0000 7f993d441640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99380b3280 con 0x7f993815e090 2024-04-25T08:57:01.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.625+0000 7f993d441640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99380d28f0 con 0x7f9938161fe0 2024-04-25T08:57:01.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.625+0000 7f9936ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99380b5c70 0x7f993815fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:01.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.625+0000 7f9936ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99380b5c70 0x7f993815fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39540/0 (socket says 172.21.15.43:39540) 2024-04-25T08:57:01.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.625+0000 7f9936ffd640 1 -- 172.21.15.43:0/4182618126 learned_addr learned my addr 172.21.15.43:0/4182618126 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:01.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.625+0000 7f99377fe640 1 --2- 172.21.15.43:0/4182618126 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9938161fe0 0x7f99381f4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:01.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.625+0000 7f99367fc640 1 --2- 172.21.15.43:0/4182618126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f993815e090 0x7f99381f3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:01.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.625+0000 7f9936ffd640 1 -- 172.21.15.43:0/4182618126 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9938161fe0 msgr2=0x7f99381f4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:01.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.625+0000 7f9936ffd640 1 --2- 172.21.15.43:0/4182618126 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9938161fe0 0x7f99381f4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:01.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.625+0000 7f9936ffd640 1 -- 172.21.15.43:0/4182618126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f993815e090 msgr2=0x7f99381f3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:01.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.625+0000 7f9936ffd640 1 --2- 172.21.15.43:0/4182618126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f993815e090 0x7f99381f3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:01.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.625+0000 7f9936ffd640 1 -- 172.21.15.43:0/4182618126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9938160700 con 0x7f99380b5c70 2024-04-25T08:57:01.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.626+0000 7f9936ffd640 1 --2- 172.21.15.43:0/4182618126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99380b5c70 0x7f993815fed0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f9928000c00 tx=0x7f99280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:01.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.626+0000 7f991f7fe640 1 -- 172.21.15.43:0/4182618126 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9928041020 con 0x7f99380b5c70 2024-04-25T08:57:01.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.626+0000 7f991f7fe640 1 -- 172.21.15.43:0/4182618126 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992803c870 con 0x7f99380b5c70 2024-04-25T08:57:01.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.626+0000 7f993d441640 1 -- 172.21.15.43:0/4182618126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99381609e0 con 0x7f99380b5c70 2024-04-25T08:57:01.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.626+0000 7f991f7fe640 1 -- 172.21.15.43:0/4182618126 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9928037040 con 0x7f99380b5c70 2024-04-25T08:57:01.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.626+0000 7f993d441640 1 -- 172.21.15.43:0/4182618126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9938160410 con 0x7f99380b5c70 2024-04-25T08:57:01.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.627+0000 7f993d441640 1 -- 172.21.15.43:0/4182618126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f993806bf00 con 0x7f99380b5c70 2024-04-25T08:57:01.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.627+0000 7f991f7fe640 1 -- 172.21.15.43:0/4182618126 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9928044020 con 0x7f99380b5c70 2024-04-25T08:57:01.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.628+0000 7f991f7fe640 1 --2- 172.21.15.43:0/4182618126 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f992406b500 0x7f992406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:01.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.628+0000 7f991f7fe640 1 -- 172.21.15.43:0/4182618126 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f99280aabb0 con 0x7f99380b5c70 2024-04-25T08:57:01.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.628+0000 7f99367fc640 1 --2- 172.21.15.43:0/4182618126 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f992406b500 0x7f992406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:01.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.629+0000 7f99367fc640 1 --2- 172.21.15.43:0/4182618126 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f992406b500 0x7f992406d9c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f99381614e0 tx=0x7f992c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:01.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.633+0000 7f991f7fe640 1 -- 172.21.15.43:0/4182618126 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f993806bf00 con 0x7f99380b5c70 2024-04-25T08:57:01.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:01.854+0000 7f993d441640 1 -- 172.21.15.43:0/4182618126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f9938074f60 con 0x7f99380b5c70 2024-04-25T08:57:02.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.500+0000 7f991f7fe640 1 -- 172.21.15.43:0/4182618126 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v19) v1 ==== 172+0+0 (secure 0 0 0) 0x7f992804b030 con 0x7f99380b5c70 2024-04-25T08:57:02.490 INFO:teuthology.orchestra.run.smithi043.stderr:new fs with metadata pool 6 and data pool 7 2024-04-25T08:57:02.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.505+0000 7f993d441640 1 -- 172.21.15.43:0/4182618126 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f992406b500 msgr2=0x7f992406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:02.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.505+0000 7f993d441640 1 --2- 172.21.15.43:0/4182618126 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f992406b500 0x7f992406d9c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f99381614e0 tx=0x7f992c009040 comp rx=0 tx=0).stop 2024-04-25T08:57:02.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.505+0000 7f993d441640 1 -- 172.21.15.43:0/4182618126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99380b5c70 msgr2=0x7f993815fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:02.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.505+0000 7f993d441640 1 --2- 172.21.15.43:0/4182618126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99380b5c70 0x7f993815fed0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f9928000c00 tx=0x7f99280046c0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.514+0000 7f993d441640 1 -- 172.21.15.43:0/4182618126 shutdown_connections 2024-04-25T08:57:02.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.514+0000 7f993d441640 1 --2- 172.21.15.43:0/4182618126 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9938161fe0 0x7f99381f4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.514+0000 7f993d441640 1 --2- 172.21.15.43:0/4182618126 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f992406b500 0x7f992406d9c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.514+0000 7f993d441640 1 --2- 172.21.15.43:0/4182618126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f993815e090 0x7f99381f3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.514+0000 7f993d441640 1 --2- 172.21.15.43:0/4182618126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99380b5c70 0x7f993815fed0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.514+0000 7f993d441640 1 -- 172.21.15.43:0/4182618126 >> 172.21.15.43:0/4182618126 conn(0x7f99380d0560 msgr2=0x7f99380d2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:02.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.515+0000 7f993d441640 1 -- 172.21.15.43:0/4182618126 shutdown_connections 2024-04-25T08:57:02.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.515+0000 7f993d441640 1 -- 172.21.15.43:0/4182618126 wait complete. 2024-04-25T08:57:02.524 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:57:02.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.641+0000 7fcc1149a640 1 Processor -- start 2024-04-25T08:57:02.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.641+0000 7fcc1149a640 1 -- start start 2024-04-25T08:57:02.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.641+0000 7fcc1149a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcc0c161390 0x7fcc0c0b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:02.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.641+0000 7fcc1149a640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fcc0c0cecf0 0x7fcc0c0d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:02.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.641+0000 7fcc1149a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc0c0b4c30 con 0x7fcc0c0d02a0 2024-04-25T08:57:02.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.641+0000 7fcc1149a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc0c0d1910 con 0x7fcc0c161390 2024-04-25T08:57:02.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.641+0000 7fcc1149a640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc0c0cf2c0 con 0x7fcc0c0cecf0 2024-04-25T08:57:02.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.642+0000 7fcc0affd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcc0c161390 0x7fcc0c0b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:02.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.642+0000 7fcc0a7fc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fcc0c0d02a0 0x7fcc0c0ce5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48100/0 (socket says 172.21.15.43:48100) 2024-04-25T08:57:02.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.642+0000 7fcc0affd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcc0c161390 0x7fcc0c0b6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39554/0 (socket says 172.21.15.43:39554) 2024-04-25T08:57:02.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.642+0000 7fcc0a7fc640 1 -- 172.21.15.43:0/3594674938 learned_addr learned my addr 172.21.15.43:0/3594674938 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:02.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.642+0000 7fcc0b7fe640 1 --2- 172.21.15.43:0/3594674938 >> v2:172.21.15.193:3300/0 conn(0x7fcc0c0cecf0 0x7fcc0c0d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:02.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.642+0000 7fcc0affd640 1 -- 172.21.15.43:0/3594674938 >> v2:172.21.15.193:3300/0 conn(0x7fcc0c0cecf0 msgr2=0x7fcc0c0d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:02.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.642+0000 7fcc09ffb640 1 -- 172.21.15.43:0/3594674938 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1240814217 0 0) 0x7fcbf4001110 con 0x7fcc0c0d02a0 2024-04-25T08:57:02.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.642+0000 7fcc0affd640 1 --2- 172.21.15.43:0/3594674938 >> v2:172.21.15.193:3300/0 conn(0x7fcc0c0cecf0 0x7fcc0c0d0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.642+0000 7fcc0affd640 1 -- 172.21.15.43:0/3594674938 >> v1:172.21.15.43:6789/0 conn(0x7fcc0c0d02a0 legacy=0x7fcc0c0ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:02.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.642+0000 7fcc0affd640 1 -- 172.21.15.43:0/3594674938 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc0c0cf5a0 con 0x7fcc0c161390 2024-04-25T08:57:02.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.642+0000 7fcc0b7fe640 1 --2- 172.21.15.43:0/3594674938 >> v2:172.21.15.193:3300/0 conn(0x7fcc0c0cecf0 0x7fcc0c0d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:02.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.643+0000 7fcc0affd640 1 --2- 172.21.15.43:0/3594674938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc0c161390 0x7fcc0c0b6820 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c06d9c0 tx=0x7fcc0402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a4805e573f3bd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:02.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.643+0000 7fcc09ffb640 1 -- 172.21.15.43:0/3594674938 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc04037070 con 0x7fcc0c161390 2024-04-25T08:57:02.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.643+0000 7fcc09ffb640 1 -- 172.21.15.43:0/3594674938 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc04032440 con 0x7fcc0c161390 2024-04-25T08:57:02.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.643+0000 7fcc09ffb640 1 -- 172.21.15.43:0/3594674938 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc04032810 con 0x7fcc0c161390 2024-04-25T08:57:02.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.643+0000 7fcc1149a640 1 -- 172.21.15.43:0/3594674938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc0c161390 msgr2=0x7fcc0c0b6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:02.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.643+0000 7fcc1149a640 1 --2- 172.21.15.43:0/3594674938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc0c161390 0x7fcc0c0b6820 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c06d9c0 tx=0x7fcc0402a1a0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.643+0000 7fcc1149a640 1 -- 172.21.15.43:0/3594674938 shutdown_connections 2024-04-25T08:57:02.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.643+0000 7fcc1149a640 1 --2- 172.21.15.43:0/3594674938 >> v2:172.21.15.193:3300/0 conn(0x7fcc0c0cecf0 0x7fcc0c0d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.643+0000 7fcc1149a640 1 --2- 172.21.15.43:0/3594674938 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc0c161390 0x7fcc0c0b6820 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.643+0000 7fcc1149a640 1 -- 172.21.15.43:0/3594674938 >> 172.21.15.43:0/3594674938 conn(0x7fcc0c09cd20 msgr2=0x7fcc0c0d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:02.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.643+0000 7fcc1149a640 1 -- 172.21.15.43:0/3594674938 shutdown_connections 2024-04-25T08:57:02.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.643+0000 7fcc1149a640 1 -- 172.21.15.43:0/3594674938 wait complete. 2024-04-25T08:57:02.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.645+0000 7fcc1149a640 1 Processor -- start 2024-04-25T08:57:02.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.645+0000 7fcc1149a640 1 -- start start 2024-04-25T08:57:02.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.645+0000 7fcc1149a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc0c0cecf0 0x7fcc0c1f3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:02.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.645+0000 7fcc1149a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc0c0d02a0 0x7fcc0c0c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:02.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.645+0000 7fcc1149a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc0c161390 0x7fcc0c147840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:02.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.645+0000 7fcc1149a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c161770 con 0x7fcc0c161390 2024-04-25T08:57:02.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.645+0000 7fcc1149a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c0d15a0 con 0x7fcc0c0d02a0 2024-04-25T08:57:02.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.645+0000 7fcc1149a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c0d1da0 con 0x7fcc0c0cecf0 2024-04-25T08:57:02.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.646+0000 7fcc0b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc0c161390 0x7fcc0c147840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:02.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.646+0000 7fcc0b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc0c161390 0x7fcc0c147840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39558/0 (socket says 172.21.15.43:39558) 2024-04-25T08:57:02.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.646+0000 7fcc0b7fe640 1 -- 172.21.15.43:0/815659948 learned_addr learned my addr 172.21.15.43:0/815659948 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:02.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.646+0000 7fcc0affd640 1 --2- 172.21.15.43:0/815659948 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc0c0cecf0 0x7fcc0c1f3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:02.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.646+0000 7fcc0a7fc640 1 --2- 172.21.15.43:0/815659948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc0c0d02a0 0x7fcc0c0c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:02.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.646+0000 7fcc0b7fe640 1 -- 172.21.15.43:0/815659948 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc0c0cecf0 msgr2=0x7fcc0c1f3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:02.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.646+0000 7fcc0b7fe640 1 --2- 172.21.15.43:0/815659948 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc0c0cecf0 0x7fcc0c1f3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.646+0000 7fcc0b7fe640 1 -- 172.21.15.43:0/815659948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc0c0d02a0 msgr2=0x7fcc0c0c80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:02.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.646+0000 7fcc0b7fe640 1 --2- 172.21.15.43:0/815659948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc0c0d02a0 0x7fcc0c0c80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.646+0000 7fcc0b7fe640 1 -- 172.21.15.43:0/815659948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc0c0c6660 con 0x7fcc0c161390 2024-04-25T08:57:02.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.646+0000 7fcc0b7fe640 1 --2- 172.21.15.43:0/815659948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc0c161390 0x7fcc0c147840 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc00cef0 tx=0x7fcbfc00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:02.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.646+0000 7fcc037fe640 1 -- 172.21.15.43:0/815659948 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc01a070 con 0x7fcc0c161390 2024-04-25T08:57:02.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.646+0000 7fcc037fe640 1 -- 172.21.15.43:0/815659948 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbfc00d860 con 0x7fcc0c161390 2024-04-25T08:57:02.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.646+0000 7fcc1149a640 1 -- 172.21.15.43:0/815659948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc0c0c6920 con 0x7fcc0c161390 2024-04-25T08:57:02.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.646+0000 7fcc037fe640 1 -- 172.21.15.43:0/815659948 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc00dc50 con 0x7fcc0c161390 2024-04-25T08:57:02.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.646+0000 7fcc1149a640 1 -- 172.21.15.43:0/815659948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc0c15d720 con 0x7fcc0c161390 2024-04-25T08:57:02.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.647+0000 7fcc1149a640 1 -- 172.21.15.43:0/815659948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc0c0680f0 con 0x7fcc0c161390 2024-04-25T08:57:02.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.648+0000 7fcc037fe640 1 -- 172.21.15.43:0/815659948 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcbfc01d070 con 0x7fcc0c161390 2024-04-25T08:57:02.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.648+0000 7fcc037fe640 1 --2- 172.21.15.43:0/815659948 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcbec06b4d0 0x7fcbec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:02.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.648+0000 7fcc037fe640 1 -- 172.21.15.43:0/815659948 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcbfc08a930 con 0x7fcc0c161390 2024-04-25T08:57:02.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.648+0000 7fcc0affd640 1 --2- 172.21.15.43:0/815659948 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcbec06b4d0 0x7fcbec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:02.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.649+0000 7fcc0affd640 1 --2- 172.21.15.43:0/815659948 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcbec06b4d0 0x7fcbec06d990 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcc04034000 tx=0x7fcc0402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:02.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.652+0000 7fcc037fe640 1 -- 172.21.15.43:0/815659948 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcbfc0592b0 con 0x7fcc0c161390 2024-04-25T08:57:02.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.835+0000 7fcc1149a640 1 -- 172.21.15.43:0/815659948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc0c0d1050 con 0x7fcc0c161390 2024-04-25T08:57:02.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.836+0000 7fcc037fe640 1 -- 172.21.15.43:0/815659948 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16590 (secure 0 0 0) 0x7fcbfc05d350 con 0x7fcc0c161390 2024-04-25T08:57:02.826 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:57:02.826 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":45,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:57:01.855808+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:57:00.135489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T08:57:00.830128+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:57:02.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.838+0000 7fcc1149a640 1 -- 172.21.15.43:0/815659948 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcbec06b4d0 msgr2=0x7fcbec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:02.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.838+0000 7fcc1149a640 1 --2- 172.21.15.43:0/815659948 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcbec06b4d0 0x7fcbec06d990 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fcc04034000 tx=0x7fcc0402e000 comp rx=0 tx=0).stop 2024-04-25T08:57:02.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.838+0000 7fcc1149a640 1 -- 172.21.15.43:0/815659948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc0c161390 msgr2=0x7fcc0c147840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:02.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.838+0000 7fcc1149a640 1 --2- 172.21.15.43:0/815659948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc0c161390 0x7fcc0c147840 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc00cef0 tx=0x7fcbfc00a710 comp rx=0 tx=0).stop 2024-04-25T08:57:02.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.838+0000 7fcc1149a640 1 -- 172.21.15.43:0/815659948 shutdown_connections 2024-04-25T08:57:02.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.838+0000 7fcc1149a640 1 --2- 172.21.15.43:0/815659948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc0c161390 0x7fcc0c147840 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.838+0000 7fcc1149a640 1 --2- 172.21.15.43:0/815659948 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcbec06b4d0 0x7fcbec06d990 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.838+0000 7fcc1149a640 1 --2- 172.21.15.43:0/815659948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc0c0d02a0 0x7fcc0c0c80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.838+0000 7fcc1149a640 1 --2- 172.21.15.43:0/815659948 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc0c0cecf0 0x7fcc0c1f3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.838+0000 7fcc1149a640 1 -- 172.21.15.43:0/815659948 >> 172.21.15.43:0/815659948 conn(0x7fcc0c09cd20 msgr2=0x7fcc0c0d2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:02.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.838+0000 7fcc1149a640 1 -- 172.21.15.43:0/815659948 shutdown_connections 2024-04-25T08:57:02.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.838+0000 7fcc1149a640 1 -- 172.21.15.43:0/815659948 wait complete. 2024-04-25T08:57:02.840 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:57:02.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.951+0000 7f6a37aa3640 1 Processor -- start 2024-04-25T08:57:02.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.951+0000 7f6a37aa3640 1 -- start start 2024-04-25T08:57:02.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.952+0000 7f6a37aa3640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6a3015ded0 0x7f6a300ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:02.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.952+0000 7f6a37aa3640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a300cf250 0x7f6a300ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:02.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.952+0000 7f6a37aa3640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a300cf6e0 con 0x7f6a3006ab30 2024-04-25T08:57:02.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.952+0000 7f6a37aa3640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a3015e2b0 con 0x7f6a3015ded0 2024-04-25T08:57:02.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.952+0000 7f6a37aa3640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a300cf9c0 con 0x7f6a300cf250 2024-04-25T08:57:02.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.952+0000 7f6a3583f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6a3015ded0 0x7f6a300ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:02.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.952+0000 7f6a3583f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6a3015ded0 0x7f6a300ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39560/0 (socket says 172.21.15.43:39560) 2024-04-25T08:57:02.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.952+0000 7f6a3583f640 1 -- 172.21.15.43:0/2244179674 learned_addr learned my addr 172.21.15.43:0/2244179674 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:02.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.952+0000 7f6a3503e640 1 --2- 172.21.15.43:0/2244179674 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a300cf250 0x7f6a300ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:02.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.952+0000 7f6a3583f640 1 -- 172.21.15.43:0/2244179674 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a300cf250 msgr2=0x7f6a300ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:02.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.952+0000 7f6a3583f640 1 --2- 172.21.15.43:0/2244179674 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a300cf250 0x7f6a300ad8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.952+0000 7f6a3583f640 1 -- 172.21.15.43:0/2244179674 >> v1:172.21.15.193:6789/0 conn(0x7f6a3006ab30 legacy=0x7f6a300b4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:02.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.952+0000 7f6a3583f640 1 -- 172.21.15.43:0/2244179674 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a300cfca0 con 0x7f6a3015ded0 2024-04-25T08:57:02.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.952+0000 7f6a3503e640 1 --2- 172.21.15.43:0/2244179674 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a300cf250 0x7f6a300ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:02.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.953+0000 7f6a3583f640 1 --2- 172.21.15.43:0/2244179674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a3015ded0 0x7f6a300ce740 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c009870 tx=0x7f6a2c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d52dad27fed2f0b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:02.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.953+0000 7f6a3483d640 1 -- 172.21.15.43:0/2244179674 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c037070 con 0x7f6a3015ded0 2024-04-25T08:57:02.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.953+0000 7f6a3483d640 1 -- 172.21.15.43:0/2244179674 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a2c032440 con 0x7f6a3015ded0 2024-04-25T08:57:02.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.953+0000 7f6a3483d640 1 -- 172.21.15.43:0/2244179674 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c032810 con 0x7f6a3015ded0 2024-04-25T08:57:02.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.954+0000 7f6a37aa3640 1 -- 172.21.15.43:0/2244179674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a3015ded0 msgr2=0x7f6a300ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:02.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.954+0000 7f6a37aa3640 1 --2- 172.21.15.43:0/2244179674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a3015ded0 0x7f6a300ce740 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c009870 tx=0x7f6a2c02a210 comp rx=0 tx=0).stop 2024-04-25T08:57:02.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.954+0000 7f6a37aa3640 1 -- 172.21.15.43:0/2244179674 shutdown_connections 2024-04-25T08:57:02.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.954+0000 7f6a37aa3640 1 --2- 172.21.15.43:0/2244179674 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a300cf250 0x7f6a300ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.954+0000 7f6a37aa3640 1 --2- 172.21.15.43:0/2244179674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a3015ded0 0x7f6a300ce740 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.954+0000 7f6a37aa3640 1 -- 172.21.15.43:0/2244179674 >> 172.21.15.43:0/2244179674 conn(0x7f6a300b2490 msgr2=0x7f6a300b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:02.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.954+0000 7f6a37aa3640 1 -- 172.21.15.43:0/2244179674 shutdown_connections 2024-04-25T08:57:02.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.954+0000 7f6a37aa3640 1 -- 172.21.15.43:0/2244179674 wait complete. 2024-04-25T08:57:02.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.955+0000 7f6a37aa3640 1 Processor -- start 2024-04-25T08:57:02.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.955+0000 7f6a37aa3640 1 -- start start 2024-04-25T08:57:02.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.956+0000 7f6a37aa3640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a3006ab30 0x7f6a301560a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:02.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.956+0000 7f6a37aa3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a3015ded0 0x7f6a301419c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:02.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.956+0000 7f6a37aa3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a301f7630 0x7f6a30153080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:02.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.956+0000 7f6a37aa3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a300cefd0 con 0x7f6a3015ded0 2024-04-25T08:57:02.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.956+0000 7f6a37aa3640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a300cee50 con 0x7f6a301f7630 2024-04-25T08:57:02.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.956+0000 7f6a37aa3640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a3006afc0 con 0x7f6a3006ab30 2024-04-25T08:57:02.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.956+0000 7f6a3503e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a3015ded0 0x7f6a301419c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:02.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.956+0000 7f6a3503e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a3015ded0 0x7f6a301419c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39568/0 (socket says 172.21.15.43:39568) 2024-04-25T08:57:02.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.956+0000 7f6a3503e640 1 -- 172.21.15.43:0/2796527939 learned_addr learned my addr 172.21.15.43:0/2796527939 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:02.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.956+0000 7f6a3583f640 1 --2- 172.21.15.43:0/2796527939 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a3006ab30 0x7f6a301560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:02.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.956+0000 7f6a36040640 1 --2- 172.21.15.43:0/2796527939 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a301f7630 0x7f6a30153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:02.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.956+0000 7f6a3503e640 1 -- 172.21.15.43:0/2796527939 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a3006ab30 msgr2=0x7f6a301560a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:02.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.956+0000 7f6a3503e640 1 --2- 172.21.15.43:0/2796527939 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a3006ab30 0x7f6a301560a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.956+0000 7f6a3503e640 1 -- 172.21.15.43:0/2796527939 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a301f7630 msgr2=0x7f6a30153080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:02.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.956+0000 7f6a3503e640 1 --2- 172.21.15.43:0/2796527939 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a301f7630 0x7f6a30153080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:02.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.956+0000 7f6a3503e640 1 -- 172.21.15.43:0/2796527939 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a30155070 con 0x7f6a3015ded0 2024-04-25T08:57:02.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.956+0000 7f6a36040640 1 --2- 172.21.15.43:0/2796527939 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a301f7630 0x7f6a30153080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:02.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.956+0000 7f6a3503e640 1 --2- 172.21.15.43:0/2796527939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a3015ded0 0x7f6a301419c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f6a2000c7f0 tx=0x7f6a2000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:02.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.956+0000 7f6a267fc640 1 -- 172.21.15.43:0/2796527939 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a20015070 con 0x7f6a3015ded0 2024-04-25T08:57:02.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.956+0000 7f6a267fc640 1 -- 172.21.15.43:0/2796527939 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a20007a30 con 0x7f6a3015ded0 2024-04-25T08:57:02.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.956+0000 7f6a37aa3640 1 -- 172.21.15.43:0/2796527939 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a301552e0 con 0x7f6a3015ded0 2024-04-25T08:57:02.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.956+0000 7f6a267fc640 1 -- 172.21.15.43:0/2796527939 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a20007e00 con 0x7f6a3015ded0 2024-04-25T08:57:02.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.956+0000 7f6a37aa3640 1 -- 172.21.15.43:0/2796527939 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a301558a0 con 0x7f6a3015ded0 2024-04-25T08:57:02.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.957+0000 7f6a37aa3640 1 -- 172.21.15.43:0/2796527939 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a30074ef0 con 0x7f6a3015ded0 2024-04-25T08:57:02.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.958+0000 7f6a267fc640 1 -- 172.21.15.43:0/2796527939 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6a20018070 con 0x7f6a3015ded0 2024-04-25T08:57:02.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.958+0000 7f6a267fc640 1 --2- 172.21.15.43:0/2796527939 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6a1406b500 0x7f6a1406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:02.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.958+0000 7f6a267fc640 1 -- 172.21.15.43:0/2796527939 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a2008a060 con 0x7f6a3015ded0 2024-04-25T08:57:02.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.959+0000 7f6a3583f640 1 --2- 172.21.15.43:0/2796527939 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6a1406b500 0x7f6a1406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:02.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.959+0000 7f6a3583f640 1 --2- 172.21.15.43:0/2796527939 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6a1406b500 0x7f6a1406d9c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c034000 tx=0x7f6a2c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:02.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:02.963+0000 7f6a267fc640 1 -- 172.21.15.43:0/2796527939 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6a20058890 con 0x7f6a3015ded0 2024-04-25T08:57:03.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.142+0000 7f6a37aa3640 1 -- 172.21.15.43:0/2796527939 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a3015c150 con 0x7f6a3015ded0 2024-04-25T08:57:03.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.143+0000 7f6a267fc640 1 -- 172.21.15.43:0/2796527939 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16590 (secure 0 0 0) 0x7f6a2005c930 con 0x7f6a3015ded0 2024-04-25T08:57:03.133 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:57:03.133 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":45,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:57:01.855808+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:57:00.135489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T08:57:00.830128+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:57:03.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.145+0000 7f6a37aa3640 1 -- 172.21.15.43:0/2796527939 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6a1406b500 msgr2=0x7f6a1406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:03.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.145+0000 7f6a37aa3640 1 --2- 172.21.15.43:0/2796527939 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6a1406b500 0x7f6a1406d9c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c034000 tx=0x7f6a2c02e000 comp rx=0 tx=0).stop 2024-04-25T08:57:03.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.145+0000 7f6a37aa3640 1 -- 172.21.15.43:0/2796527939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a3015ded0 msgr2=0x7f6a301419c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:03.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.145+0000 7f6a37aa3640 1 --2- 172.21.15.43:0/2796527939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a3015ded0 0x7f6a301419c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f6a2000c7f0 tx=0x7f6a2000cbf0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.146+0000 7f6a37aa3640 1 -- 172.21.15.43:0/2796527939 shutdown_connections 2024-04-25T08:57:03.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.146+0000 7f6a37aa3640 1 --2- 172.21.15.43:0/2796527939 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6a1406b500 0x7f6a1406d9c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.146+0000 7f6a37aa3640 1 --2- 172.21.15.43:0/2796527939 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a301f7630 0x7f6a30153080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.146+0000 7f6a37aa3640 1 --2- 172.21.15.43:0/2796527939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a3015ded0 0x7f6a301419c0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.146+0000 7f6a37aa3640 1 --2- 172.21.15.43:0/2796527939 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a3006ab30 0x7f6a301560a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.146+0000 7f6a37aa3640 1 -- 172.21.15.43:0/2796527939 >> 172.21.15.43:0/2796527939 conn(0x7f6a300b2490 msgr2=0x7f6a3015c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:03.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.146+0000 7f6a37aa3640 1 -- 172.21.15.43:0/2796527939 shutdown_connections 2024-04-25T08:57:03.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.146+0000 7f6a37aa3640 1 -- 172.21.15.43:0/2796527939 wait complete. 2024-04-25T08:57:03.147 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T08:57:03.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.261+0000 7fc07c44d640 1 Processor -- start 2024-04-25T08:57:03.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.261+0000 7fc07c44d640 1 -- start start 2024-04-25T08:57:03.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.262+0000 7fc07c44d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc0740d19f0 0x7fc0740d1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:03.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.262+0000 7fc07c44d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc0740d2370 0x7fc0740c7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:03.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.262+0000 7fc07c44d640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0740cf030 con 0x7fc074161310 2024-04-25T08:57:03.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.262+0000 7fc07c44d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0740d4870 con 0x7fc0740d2370 2024-04-25T08:57:03.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.262+0000 7fc07c44d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0740cf310 con 0x7fc0740d19f0 2024-04-25T08:57:03.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.262+0000 7fc07a9ea640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc0740d2370 0x7fc0740c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:03.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.262+0000 7fc07a9ea640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc0740d2370 0x7fc0740c7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39574/0 (socket says 172.21.15.43:39574) 2024-04-25T08:57:03.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.262+0000 7fc07a9ea640 1 -- 172.21.15.43:0/1976686162 learned_addr learned my addr 172.21.15.43:0/1976686162 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:03.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.262+0000 7fc0799e8640 1 --2- 172.21.15.43:0/1976686162 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc0740d19f0 0x7fc0740d1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:03.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.262+0000 7fc07a9ea640 1 -- 172.21.15.43:0/1976686162 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc0740d19f0 msgr2=0x7fc0740d1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:03.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.262+0000 7fc07a9ea640 1 --2- 172.21.15.43:0/1976686162 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc0740d19f0 0x7fc0740d1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.262+0000 7fc07a9ea640 1 -- 172.21.15.43:0/1976686162 >> v1:172.21.15.193:6789/0 conn(0x7fc074161310 legacy=0x7fc0740d4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:03.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.262+0000 7fc07a9ea640 1 -- 172.21.15.43:0/1976686162 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0740cf5f0 con 0x7fc0740d2370 2024-04-25T08:57:03.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.263+0000 7fc07a9ea640 1 --2- 172.21.15.43:0/1976686162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0740d2370 0x7fc0740c7860 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fc06c009e50 tx=0x7fc06c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb4af92ba130aa17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:03.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.263+0000 7fc0791e7640 1 -- 172.21.15.43:0/1976686162 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06c036070 con 0x7fc0740d2370 2024-04-25T08:57:03.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.263+0000 7fc0791e7640 1 -- 172.21.15.43:0/1976686162 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc06c031600 con 0x7fc0740d2370 2024-04-25T08:57:03.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.263+0000 7fc0791e7640 1 -- 172.21.15.43:0/1976686162 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06c0319d0 con 0x7fc0740d2370 2024-04-25T08:57:03.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.264+0000 7fc07c44d640 1 -- 172.21.15.43:0/1976686162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0740d2370 msgr2=0x7fc0740c7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:03.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.264+0000 7fc07c44d640 1 --2- 172.21.15.43:0/1976686162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0740d2370 0x7fc0740c7860 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fc06c009e50 tx=0x7fc06c02a6c0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.264+0000 7fc07c44d640 1 -- 172.21.15.43:0/1976686162 shutdown_connections 2024-04-25T08:57:03.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.264+0000 7fc07c44d640 1 --2- 172.21.15.43:0/1976686162 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0740d2370 0x7fc0740c7860 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.264+0000 7fc07c44d640 1 --2- 172.21.15.43:0/1976686162 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc0740d19f0 0x7fc0740d1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.264+0000 7fc07c44d640 1 -- 172.21.15.43:0/1976686162 >> 172.21.15.43:0/1976686162 conn(0x7fc07409cd20 msgr2=0x7fc0740d3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:03.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.264+0000 7fc07c44d640 1 -- 172.21.15.43:0/1976686162 shutdown_connections 2024-04-25T08:57:03.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.264+0000 7fc07c44d640 1 -- 172.21.15.43:0/1976686162 wait complete. 2024-04-25T08:57:03.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.265+0000 7fc07c44d640 1 Processor -- start 2024-04-25T08:57:03.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.265+0000 7fc07c44d640 1 -- start start 2024-04-25T08:57:03.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.266+0000 7fc07c44d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0740d19f0 0x7fc0740b9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:03.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.266+0000 7fc07c44d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc0740d2370 0x7fc07414eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:03.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.266+0000 7fc07c44d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc074161310 0x7fc07415aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:03.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.266+0000 7fc07c44d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0740c7da0 con 0x7fc074161310 2024-04-25T08:57:03.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.266+0000 7fc07c44d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0740d2800 con 0x7fc0740d19f0 2024-04-25T08:57:03.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.266+0000 7fc07c44d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0740c7f20 con 0x7fc0740d2370 2024-04-25T08:57:03.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.266+0000 7fc07a9ea640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc074161310 0x7fc07415aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:03.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.266+0000 7fc07a9ea640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc074161310 0x7fc07415aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39586/0 (socket says 172.21.15.43:39586) 2024-04-25T08:57:03.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.266+0000 7fc07a9ea640 1 -- 172.21.15.43:0/10340365 learned_addr learned my addr 172.21.15.43:0/10340365 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:03.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.266+0000 7fc07a9ea640 1 -- 172.21.15.43:0/10340365 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc0740d2370 msgr2=0x7fc07414eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:03.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.266+0000 7fc07a9ea640 1 --2- 172.21.15.43:0/10340365 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc0740d2370 0x7fc07414eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.266+0000 7fc07a9ea640 1 -- 172.21.15.43:0/10340365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0740d19f0 msgr2=0x7fc0740b9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:03.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.266+0000 7fc07a9ea640 1 --2- 172.21.15.43:0/10340365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0740d19f0 0x7fc0740b9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.266+0000 7fc07a9ea640 1 -- 172.21.15.43:0/10340365 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc074159890 con 0x7fc074161310 2024-04-25T08:57:03.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.266+0000 7fc07a9ea640 1 --2- 172.21.15.43:0/10340365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc074161310 0x7fc07415aad0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fc06c009e50 tx=0x7fc06c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:03.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.267+0000 7fc06affd640 1 -- 172.21.15.43:0/10340365 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06c041020 con 0x7fc074161310 2024-04-25T08:57:03.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.267+0000 7fc06affd640 1 -- 172.21.15.43:0/10340365 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc06c0077a0 con 0x7fc074161310 2024-04-25T08:57:03.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.267+0000 7fc07c44d640 1 -- 172.21.15.43:0/10340365 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc074159b70 con 0x7fc074161310 2024-04-25T08:57:03.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.267+0000 7fc06affd640 1 -- 172.21.15.43:0/10340365 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc06c036040 con 0x7fc074161310 2024-04-25T08:57:03.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.267+0000 7fc07c44d640 1 -- 172.21.15.43:0/10340365 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc07415a130 con 0x7fc074161310 2024-04-25T08:57:03.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.268+0000 7fc07c44d640 1 -- 172.21.15.43:0/10340365 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0740b6550 con 0x7fc074161310 2024-04-25T08:57:03.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.268+0000 7fc06affd640 1 -- 172.21.15.43:0/10340365 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc06c044020 con 0x7fc074161310 2024-04-25T08:57:03.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.269+0000 7fc06affd640 1 --2- 172.21.15.43:0/10340365 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc05806b4d0 0x7fc05806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:03.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.269+0000 7fc06affd640 1 -- 172.21.15.43:0/10340365 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc06c0aafc0 con 0x7fc074161310 2024-04-25T08:57:03.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.269+0000 7fc07a1e9640 1 --2- 172.21.15.43:0/10340365 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc05806b4d0 0x7fc05806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:03.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.269+0000 7fc07a1e9640 1 --2- 172.21.15.43:0/10340365 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc05806b4d0 0x7fc05806d990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc070006fd0 tx=0x7fc070006bc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:03.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.273+0000 7fc06affd640 1 -- 172.21.15.43:0/10340365 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc06c04b030 con 0x7fc074161310 2024-04-25T08:57:03.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.495+0000 7fc07c44d640 1 -- 172.21.15.43:0/10340365 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc074074f60 con 0x7fc074161310 2024-04-25T08:57:03.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.508+0000 7fc06affd640 1 -- 172.21.15.43:0/10340365 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v20) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc06c079ed0 con 0x7fc074161310 2024-04-25T08:57:03.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.510+0000 7fc07c44d640 1 -- 172.21.15.43:0/10340365 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc05806b4d0 msgr2=0x7fc05806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:03.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.510+0000 7fc07c44d640 1 --2- 172.21.15.43:0/10340365 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc05806b4d0 0x7fc05806d990 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc070006fd0 tx=0x7fc070006bc0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.510+0000 7fc07c44d640 1 -- 172.21.15.43:0/10340365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc074161310 msgr2=0x7fc07415aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:03.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.510+0000 7fc07c44d640 1 --2- 172.21.15.43:0/10340365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc074161310 0x7fc07415aad0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fc06c009e50 tx=0x7fc06c004a40 comp rx=0 tx=0).stop 2024-04-25T08:57:03.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.510+0000 7fc07c44d640 1 -- 172.21.15.43:0/10340365 shutdown_connections 2024-04-25T08:57:03.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.510+0000 7fc07c44d640 1 --2- 172.21.15.43:0/10340365 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc074161310 0x7fc07415aad0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.510+0000 7fc07c44d640 1 --2- 172.21.15.43:0/10340365 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc0740d2370 0x7fc07414eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.510+0000 7fc07c44d640 1 --2- 172.21.15.43:0/10340365 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc05806b4d0 0x7fc05806d990 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.510+0000 7fc07c44d640 1 --2- 172.21.15.43:0/10340365 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0740d19f0 0x7fc0740b9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.510+0000 7fc07c44d640 1 -- 172.21.15.43:0/10340365 >> 172.21.15.43:0/10340365 conn(0x7fc07409cd20 msgr2=0x7fc0740cf960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:03.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.510+0000 7fc07c44d640 1 -- 172.21.15.43:0/10340365 shutdown_connections 2024-04-25T08:57:03.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.510+0000 7fc07c44d640 1 -- 172.21.15.43:0/10340365 wait complete. 2024-04-25T08:57:03.511 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T08:57:03.511 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T08:57:03.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.625+0000 7f7ab4f96640 1 Processor -- start 2024-04-25T08:57:03.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.625+0000 7f7ab4f96640 1 -- start start 2024-04-25T08:57:03.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.626+0000 7f7ab4f96640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7ab00ad4b0 0x7f7ab015e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:03.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.626+0000 7f7ab4f96640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ab015e8d0 con 0x7f7ab01618b0 2024-04-25T08:57:03.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.626+0000 7f7ab4f96640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ab015ebb0 con 0x7f7ab006eb20 2024-04-25T08:57:03.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.626+0000 7f7ab4f96640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab015ee90 con 0x7f7ab00ad4b0 2024-04-25T08:57:03.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.626+0000 7f7aae59c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f7ab01618b0 0x7f7ab00acda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48104/0 (socket says 172.21.15.43:48104) 2024-04-25T08:57:03.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.626+0000 7f7aae59c640 1 -- 172.21.15.43:0/4195221210 learned_addr learned my addr 172.21.15.43:0/4195221210 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:03.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.626+0000 7f7aa7fff640 1 --2- 172.21.15.43:0/4195221210 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7ab00ad4b0 0x7f7ab015e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:03.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.626+0000 7f7aadd9b640 1 -- 172.21.15.43:0/4195221210 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3866075281 0 0) 0x7f7ab015e8d0 con 0x7f7ab01618b0 2024-04-25T08:57:03.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.626+0000 7f7aadd9b640 1 -- 172.21.15.43:0/4195221210 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a9c003520 con 0x7f7ab01618b0 2024-04-25T08:57:03.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.626+0000 7f7aadd9b640 1 -- 172.21.15.43:0/4195221210 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2486897638 0 0) 0x7f7ab015ebb0 con 0x7f7ab006eb20 2024-04-25T08:57:03.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.626+0000 7f7aadd9b640 1 -- 172.21.15.43:0/4195221210 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ab015e8d0 con 0x7f7ab006eb20 2024-04-25T08:57:03.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.626+0000 7f7aadd9b640 1 -- 172.21.15.43:0/4195221210 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1284735805 0 0) 0x7f7a9c003520 con 0x7f7ab01618b0 2024-04-25T08:57:03.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.626+0000 7f7aa7fff640 1 -- 172.21.15.43:0/4195221210 >> v1:172.21.15.193:6789/0 conn(0x7f7ab006eb20 legacy=0x7f7ab006ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:03.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.626+0000 7f7aa7fff640 1 -- 172.21.15.43:0/4195221210 >> v1:172.21.15.43:6789/0 conn(0x7f7ab01618b0 legacy=0x7f7ab00acda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:03.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.626+0000 7f7aa7fff640 1 -- 172.21.15.43:0/4195221210 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab015f170 con 0x7f7ab00ad4b0 2024-04-25T08:57:03.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.627+0000 7f7aa7fff640 1 --2- 172.21.15.43:0/4195221210 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7ab00ad4b0 0x7f7ab015e270 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7a980097d0 tx=0x7f7a9802a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5682df3bfa4f265 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:03.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.627+0000 7f7aadd9b640 1 -- 172.21.15.43:0/4195221210 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a98037070 con 0x7f7ab00ad4b0 2024-04-25T08:57:03.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.627+0000 7f7aadd9b640 1 -- 172.21.15.43:0/4195221210 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a9802ac30 con 0x7f7ab00ad4b0 2024-04-25T08:57:03.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.627+0000 7f7aadd9b640 1 -- 172.21.15.43:0/4195221210 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0001670 con 0x7f7ab00ad4b0 2024-04-25T08:57:03.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.628+0000 7f7ab4f96640 1 -- 172.21.15.43:0/4195221210 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7ab00ad4b0 msgr2=0x7f7ab015e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:03.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.628+0000 7f7ab4f96640 1 --2- 172.21.15.43:0/4195221210 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7ab00ad4b0 0x7f7ab015e270 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f7a980097d0 tx=0x7f7a9802a1a0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.628+0000 7f7ab4f96640 1 -- 172.21.15.43:0/4195221210 shutdown_connections 2024-04-25T08:57:03.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.628+0000 7f7ab4f96640 1 --2- 172.21.15.43:0/4195221210 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7ab00ad4b0 0x7f7ab015e270 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.628+0000 7f7ab4f96640 1 -- 172.21.15.43:0/4195221210 >> 172.21.15.43:0/4195221210 conn(0x7f7ab00d0260 msgr2=0x7f7ab00d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:03.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.628+0000 7f7ab4f96640 1 -- 172.21.15.43:0/4195221210 shutdown_connections 2024-04-25T08:57:03.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.628+0000 7f7ab4f96640 1 -- 172.21.15.43:0/4195221210 wait complete. 2024-04-25T08:57:03.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.629+0000 7f7ab4f96640 1 Processor -- start 2024-04-25T08:57:03.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.629+0000 7f7ab4f96640 1 -- start start 2024-04-25T08:57:03.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.630+0000 7f7ab4f96640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ab006eb20 0x7f7ab009d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:03.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.630+0000 7f7ab4f96640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7ab00ad4b0 0x7f7ab00c90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:03.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.630+0000 7f7ab4f96640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ab01618b0 0x7f7ab0153c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:03.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.630+0000 7f7ab4f96640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab0072190 con 0x7f7ab006eb20 2024-04-25T08:57:03.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.630+0000 7f7ab4f96640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab0072010 con 0x7f7ab01618b0 2024-04-25T08:57:03.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.630+0000 7f7ab4f96640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab0072310 con 0x7f7ab00ad4b0 2024-04-25T08:57:03.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.630+0000 7f7aae59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ab006eb20 0x7f7ab009d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:03.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.630+0000 7f7aae59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ab006eb20 0x7f7ab009d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39590/0 (socket says 172.21.15.43:39590) 2024-04-25T08:57:03.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.630+0000 7f7aae59c640 1 -- 172.21.15.43:0/3184745401 learned_addr learned my addr 172.21.15.43:0/3184745401 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:03.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.630+0000 7f7aae59c640 1 -- 172.21.15.43:0/3184745401 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7ab00ad4b0 msgr2=0x7f7ab00c90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:03.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.630+0000 7f7aae59c640 1 --2- 172.21.15.43:0/3184745401 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7ab00ad4b0 0x7f7ab00c90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.630+0000 7f7aae59c640 1 -- 172.21.15.43:0/3184745401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ab01618b0 msgr2=0x7f7ab0153c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:03.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.630+0000 7f7aae59c640 1 --2- 172.21.15.43:0/3184745401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ab01618b0 0x7f7ab0153c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:03.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.630+0000 7f7aae59c640 1 -- 172.21.15.43:0/3184745401 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab00c9840 con 0x7f7ab006eb20 2024-04-25T08:57:03.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.631+0000 7f7aae59c640 1 --2- 172.21.15.43:0/3184745401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ab006eb20 0x7f7ab009d7d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7aa000c5f0 tx=0x7f7aa000cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:03.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.631+0000 7f7aa6ffd640 1 -- 172.21.15.43:0/3184745401 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0017070 con 0x7f7ab006eb20 2024-04-25T08:57:03.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.631+0000 7f7aa6ffd640 1 -- 172.21.15.43:0/3184745401 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aa0005d00 con 0x7f7ab006eb20 2024-04-25T08:57:03.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.631+0000 7f7ab4f96640 1 -- 172.21.15.43:0/3184745401 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ab00c9b20 con 0x7f7ab006eb20 2024-04-25T08:57:03.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.631+0000 7f7aa6ffd640 1 -- 172.21.15.43:0/3184745401 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa00072b0 con 0x7f7ab006eb20 2024-04-25T08:57:03.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.631+0000 7f7ab4f96640 1 -- 172.21.15.43:0/3184745401 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ab00ca0e0 con 0x7f7ab006eb20 2024-04-25T08:57:03.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.632+0000 7f7ab4f96640 1 -- 172.21.15.43:0/3184745401 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ab0074f60 con 0x7f7ab006eb20 2024-04-25T08:57:03.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.632+0000 7f7aa6ffd640 1 -- 172.21.15.43:0/3184745401 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7aa001a070 con 0x7f7ab006eb20 2024-04-25T08:57:03.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.633+0000 7f7aa6ffd640 1 --2- 172.21.15.43:0/3184745401 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7a9006b500 0x7f7a9006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:03.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.633+0000 7f7aa6ffd640 1 -- 172.21.15.43:0/3184745401 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7aa00896d0 con 0x7f7ab006eb20 2024-04-25T08:57:03.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.633+0000 7f7aa7fff640 1 --2- 172.21.15.43:0/3184745401 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7a9006b500 0x7f7a9006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:03.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.633+0000 7f7aa7fff640 1 --2- 172.21.15.43:0/3184745401 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7a9006b500 0x7f7a9006d9c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7a98034000 tx=0x7f7a9802c000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:03.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.637+0000 7f7aa6ffd640 1 -- 172.21.15.43:0/3184745401 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7aa00585e0 con 0x7f7ab006eb20 2024-04-25T08:57:03.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:03.856+0000 7f7ab4f96640 1 -- 172.21.15.43:0/3184745401 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f7ab0072620 con 0x7f7ab006eb20 2024-04-25T08:57:04.465 INFO:tasks.ceph.mds.d.smithi193.stderr:2024-04-25T08:57:04.476+0000 7f52a62c8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:57:04.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.515+0000 7f7aa6ffd640 1 -- 172.21.15.43:0/3184745401 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v21) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7aa0057ee0 con 0x7f7ab006eb20 2024-04-25T08:57:04.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.518+0000 7f7ab4f96640 1 -- 172.21.15.43:0/3184745401 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7a9006b500 msgr2=0x7f7a9006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:04.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.518+0000 7f7ab4f96640 1 --2- 172.21.15.43:0/3184745401 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7a9006b500 0x7f7a9006d9c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7a98034000 tx=0x7f7a9802c000 comp rx=0 tx=0).stop 2024-04-25T08:57:04.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.518+0000 7f7ab4f96640 1 -- 172.21.15.43:0/3184745401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ab006eb20 msgr2=0x7f7ab009d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:04.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.518+0000 7f7ab4f96640 1 --2- 172.21.15.43:0/3184745401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ab006eb20 0x7f7ab009d7d0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7aa000c5f0 tx=0x7f7aa000cab0 comp rx=0 tx=0).stop 2024-04-25T08:57:04.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.518+0000 7f7ab4f96640 1 -- 172.21.15.43:0/3184745401 shutdown_connections 2024-04-25T08:57:04.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.518+0000 7f7ab4f96640 1 --2- 172.21.15.43:0/3184745401 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7a9006b500 0x7f7a9006d9c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:04.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.518+0000 7f7ab4f96640 1 --2- 172.21.15.43:0/3184745401 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ab01618b0 0x7f7ab0153c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:04.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.518+0000 7f7ab4f96640 1 --2- 172.21.15.43:0/3184745401 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7ab00ad4b0 0x7f7ab00c90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:04.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.518+0000 7f7ab4f96640 1 --2- 172.21.15.43:0/3184745401 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ab006eb20 0x7f7ab009d7d0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:04.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.518+0000 7f7ab4f96640 1 -- 172.21.15.43:0/3184745401 >> 172.21.15.43:0/3184745401 conn(0x7f7ab00d0260 msgr2=0x7f7ab006ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:04.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.518+0000 7f7ab4f96640 1 -- 172.21.15.43:0/3184745401 shutdown_connections 2024-04-25T08:57:04.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.518+0000 7f7ab4f96640 1 -- 172.21.15.43:0/3184745401 wait complete. 2024-04-25T08:57:04.522 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T08:57:04.522 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:57:04.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.640+0000 7f9fb5b22640 1 Processor -- start 2024-04-25T08:57:04.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.640+0000 7f9fb5b22640 1 -- start start 2024-04-25T08:57:04.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.640+0000 7f9fb5b22640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9fb015f180 0x7f9fb00cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:04.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.640+0000 7f9fb5b22640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9fb00b23f0 0x7f9fb00cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:04.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.640+0000 7f9fb5b22640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9fb00b2cc0 0x7f9fb015f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:04.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.640+0000 7f9fb5b22640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fb00cccc0 con 0x7f9fb015f180 2024-04-25T08:57:04.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.641+0000 7f9fb5b22640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fb00cd840 con 0x7f9fb00b2cc0 2024-04-25T08:57:04.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.641+0000 7f9fb5b22640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb00ccf10 con 0x7f9fb00b23f0 2024-04-25T08:57:04.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.641+0000 7f9faf7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9fb015f180 0x7f9fb00cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:04.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.641+0000 7f9faf7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9fb015f180 0x7f9fb00cf860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39602/0 (socket says 172.21.15.43:39602) 2024-04-25T08:57:04.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.641+0000 7f9faf7fe640 1 -- 172.21.15.43:0/3403788282 learned_addr learned my addr 172.21.15.43:0/3403788282 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:04.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.641+0000 7f9faf7fe640 1 -- 172.21.15.43:0/3403788282 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9fb00b23f0 msgr2=0x7f9fb00cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:04.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.641+0000 7f9faf7fe640 1 --2- 172.21.15.43:0/3403788282 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9fb00b23f0 0x7f9fb00cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:04.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.641+0000 7f9faf7fe640 1 -- 172.21.15.43:0/3403788282 >> v2:172.21.15.193:3300/0 conn(0x7f9fb00b2cc0 msgr2=0x7f9fb015f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:04.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.641+0000 7f9faf7fe640 1 --2- 172.21.15.43:0/3403788282 >> v2:172.21.15.193:3300/0 conn(0x7f9fb00b2cc0 0x7f9fb015f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:04.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.641+0000 7f9faf7fe640 1 -- 172.21.15.43:0/3403788282 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb00cd1f0 con 0x7f9fb015f180 2024-04-25T08:57:04.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.642+0000 7f9faf7fe640 1 --2- 172.21.15.43:0/3403788282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fb015f180 0x7f9fb00cf860 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f9fb00d28f0 tx=0x7f9fa002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d955ff6fd498265 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:04.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.642+0000 7f9faeffd640 1 -- 172.21.15.43:0/3403788282 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa0037070 con 0x7f9fb015f180 2024-04-25T08:57:04.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.642+0000 7f9faeffd640 1 -- 172.21.15.43:0/3403788282 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa0032440 con 0x7f9fb015f180 2024-04-25T08:57:04.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.642+0000 7f9faeffd640 1 -- 172.21.15.43:0/3403788282 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa0032810 con 0x7f9fb015f180 2024-04-25T08:57:04.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.642+0000 7f9fb5b22640 1 -- 172.21.15.43:0/3403788282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fb015f180 msgr2=0x7f9fb00cf860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:04.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.642+0000 7f9fb5b22640 1 --2- 172.21.15.43:0/3403788282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fb015f180 0x7f9fb00cf860 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f9fb00d28f0 tx=0x7f9fa002a140 comp rx=0 tx=0).stop 2024-04-25T08:57:04.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.642+0000 7f9fb5b22640 1 -- 172.21.15.43:0/3403788282 shutdown_connections 2024-04-25T08:57:04.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.642+0000 7f9fb5b22640 1 --2- 172.21.15.43:0/3403788282 >> v2:172.21.15.193:3300/0 conn(0x7f9fb00b2cc0 0x7f9fb015f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:04.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.642+0000 7f9fb5b22640 1 --2- 172.21.15.43:0/3403788282 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9fb00b23f0 0x7f9fb00cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:04.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.642+0000 7f9fb5b22640 1 --2- 172.21.15.43:0/3403788282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fb015f180 0x7f9fb00cf860 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:04.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.642+0000 7f9fb5b22640 1 -- 172.21.15.43:0/3403788282 >> 172.21.15.43:0/3403788282 conn(0x7f9fb00d17f0 msgr2=0x7f9fb00d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:04.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.642+0000 7f9fb5b22640 1 -- 172.21.15.43:0/3403788282 shutdown_connections 2024-04-25T08:57:04.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.642+0000 7f9fb5b22640 1 -- 172.21.15.43:0/3403788282 wait complete. 2024-04-25T08:57:04.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.644+0000 7f9fb5b22640 1 Processor -- start 2024-04-25T08:57:04.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.644+0000 7f9fb5b22640 1 -- start start 2024-04-25T08:57:04.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.644+0000 7f9fb5b22640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fb00b23f0 0x7f9fb00c6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:04.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.645+0000 7f9fb5b22640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9fb00b2cc0 0x7f9fb00c7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:04.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.645+0000 7f9fb5b22640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9fb015f180 0x7f9fb00c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:04.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.645+0000 7f9fb5b22640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb015f560 con 0x7f9fb00b23f0 2024-04-25T08:57:04.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.645+0000 7f9fb5b22640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb00d23f0 con 0x7f9fb00b2cc0 2024-04-25T08:57:04.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.645+0000 7f9fb5b22640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb00cafb0 con 0x7f9fb015f180 2024-04-25T08:57:04.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.645+0000 7f9faf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fb00b23f0 0x7f9fb00c6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:04.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.645+0000 7f9faf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fb00b23f0 0x7f9fb00c6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39610/0 (socket says 172.21.15.43:39610) 2024-04-25T08:57:04.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.645+0000 7f9faf7fe640 1 -- 172.21.15.43:0/890354168 learned_addr learned my addr 172.21.15.43:0/890354168 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:04.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.645+0000 7f9fa6ffd640 1 --2- 172.21.15.43:0/890354168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9fb00b2cc0 0x7f9fb00c7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:04.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.645+0000 7f9faffff640 1 --2- 172.21.15.43:0/890354168 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9fb015f180 0x7f9fb00c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:04.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.645+0000 7f9faf7fe640 1 -- 172.21.15.43:0/890354168 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9fb015f180 msgr2=0x7f9fb00c75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:04.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.645+0000 7f9faf7fe640 1 --2- 172.21.15.43:0/890354168 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9fb015f180 0x7f9fb00c75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:04.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.645+0000 7f9faf7fe640 1 -- 172.21.15.43:0/890354168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9fb00b2cc0 msgr2=0x7f9fb00c7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:04.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.645+0000 7f9faf7fe640 1 --2- 172.21.15.43:0/890354168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9fb00b2cc0 0x7f9fb00c7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:04.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.645+0000 7f9faf7fe640 1 -- 172.21.15.43:0/890354168 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb00cb290 con 0x7f9fb00b23f0 2024-04-25T08:57:04.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.645+0000 7f9faffff640 1 --2- 172.21.15.43:0/890354168 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9fb015f180 0x7f9fb00c75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:04.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.645+0000 7f9faf7fe640 1 --2- 172.21.15.43:0/890354168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fb00b23f0 0x7f9fb00c6340 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0000c00 tx=0x7f9fa00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:04.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.645+0000 7f9facff9640 1 -- 172.21.15.43:0/890354168 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa0041070 con 0x7f9fb00b23f0 2024-04-25T08:57:04.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.645+0000 7f9facff9640 1 -- 172.21.15.43:0/890354168 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa003c870 con 0x7f9fb00b23f0 2024-04-25T08:57:04.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.645+0000 7f9fb5b22640 1 -- 172.21.15.43:0/890354168 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fb00cb570 con 0x7f9fb00b23f0 2024-04-25T08:57:04.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.645+0000 7f9facff9640 1 -- 172.21.15.43:0/890354168 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa0037040 con 0x7f9fb00b23f0 2024-04-25T08:57:04.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.645+0000 7f9fb5b22640 1 -- 172.21.15.43:0/890354168 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fb00cbb30 con 0x7f9fb00b23f0 2024-04-25T08:57:04.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.647+0000 7f9fb5b22640 1 -- 172.21.15.43:0/890354168 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fb0074ef0 con 0x7f9fb00b23f0 2024-04-25T08:57:04.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.647+0000 7f9facff9640 1 -- 172.21.15.43:0/890354168 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9fa0044070 con 0x7f9fb00b23f0 2024-04-25T08:57:04.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.647+0000 7f9facff9640 1 --2- 172.21.15.43:0/890354168 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9f9006b4d0 0x7f9f9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:04.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.647+0000 7f9facff9640 1 -- 172.21.15.43:0/890354168 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9fa00aafb0 con 0x7f9fb00b23f0 2024-04-25T08:57:04.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.648+0000 7f9fa6ffd640 1 --2- 172.21.15.43:0/890354168 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9f9006b4d0 0x7f9f9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:04.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.648+0000 7f9fa6ffd640 1 --2- 172.21.15.43:0/890354168 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9f9006b4d0 0x7f9f9006d990 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0068080 tx=0x7f9f98009060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:04.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.652+0000 7f9facff9640 1 -- 172.21.15.43:0/890354168 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9fa0079990 con 0x7f9fb00b23f0 2024-04-25T08:57:04.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.872+0000 7f9fb5b22640 1 -- 172.21.15.43:0/890354168 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9fb00cf050 con 0x7f9fb00b23f0 2024-04-25T08:57:04.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.872+0000 7f9facff9640 1 -- 172.21.15.43:0/890354168 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f9fa007da30 con 0x7f9fb00b23f0 2024-04-25T08:57:04.862 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:57:04.862 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1504450503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1504450503},{"type":"v1","addr":"172.21.15.193:6835","nonce":1504450503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5691,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2657133821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2657133821},{"type":"v1","addr":"172.21.15.43:6834","nonce":2657133821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":5694,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/4271431818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":4271431818},{"type":"v1","addr":"172.21.15.43:6835","nonce":4271431818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:57:01.856132+0000","modified":"2024-04-25T08:57:03.857410+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5709":{"gid":5709,"name":"d","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.193:6837/4164634380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4164634380},{"type":"v1","addr":"172.21.15.193:6837","nonce":4164634380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:57:04.862 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 21 2024-04-25T08:57:04.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.874+0000 7f9fb5b22640 1 -- 172.21.15.43:0/890354168 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9f9006b4d0 msgr2=0x7f9f9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:04.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.874+0000 7f9fb5b22640 1 --2- 172.21.15.43:0/890354168 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9f9006b4d0 0x7f9f9006d990 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f9fb0068080 tx=0x7f9f98009060 comp rx=0 tx=0).stop 2024-04-25T08:57:04.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.874+0000 7f9fb5b22640 1 -- 172.21.15.43:0/890354168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fb00b23f0 msgr2=0x7f9fb00c6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:04.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.874+0000 7f9fb5b22640 1 --2- 172.21.15.43:0/890354168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fb00b23f0 0x7f9fb00c6340 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0000c00 tx=0x7f9fa00046c0 comp rx=0 tx=0).stop 2024-04-25T08:57:04.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.874+0000 7f9fb5b22640 1 -- 172.21.15.43:0/890354168 shutdown_connections 2024-04-25T08:57:04.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.874+0000 7f9fb5b22640 1 --2- 172.21.15.43:0/890354168 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9fb015f180 0x7f9fb00c75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:04.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.874+0000 7f9fb5b22640 1 --2- 172.21.15.43:0/890354168 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9f9006b4d0 0x7f9f9006d990 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:04.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.874+0000 7f9fb5b22640 1 --2- 172.21.15.43:0/890354168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9fb00b2cc0 0x7f9fb00c7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:04.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.874+0000 7f9fb5b22640 1 --2- 172.21.15.43:0/890354168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9fb00b23f0 0x7f9fb00c6340 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:04.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.874+0000 7f9fb5b22640 1 -- 172.21.15.43:0/890354168 >> 172.21.15.43:0/890354168 conn(0x7f9fb00d17f0 msgr2=0x7f9fb00b46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:04.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.874+0000 7f9fb5b22640 1 -- 172.21.15.43:0/890354168 shutdown_connections 2024-04-25T08:57:04.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.874+0000 7f9fb5b22640 1 -- 172.21.15.43:0/890354168 wait complete. 2024-04-25T08:57:04.875 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:57:04.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.986+0000 7f6849af5640 1 Processor -- start 2024-04-25T08:57:04.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.986+0000 7f6849af5640 1 -- start start 2024-04-25T08:57:04.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.987+0000 7f6849af5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f684415daa0 0x7f6844162220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:04.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.987+0000 7f6849af5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f684415de80 0x7f68440cc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:04.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.987+0000 7f6849af5640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f68440b2960 0x7f68440cc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:04.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.987+0000 7f6849af5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68440cd950 con 0x7f684415daa0 2024-04-25T08:57:04.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.987+0000 7f6849af5640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6844162760 con 0x7f68440b2960 2024-04-25T08:57:04.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.987+0000 7f6849af5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68440b4560 con 0x7f684415de80 2024-04-25T08:57:04.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.987+0000 7f68437fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f684415daa0 0x7f6844162220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:04.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.987+0000 7f68437fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f684415daa0 0x7f6844162220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39622/0 (socket says 172.21.15.43:39622) 2024-04-25T08:57:04.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.987+0000 7f68437fe640 1 -- 172.21.15.43:0/940484169 learned_addr learned my addr 172.21.15.43:0/940484169 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:04.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.987+0000 7f6842ffd640 1 --2- 172.21.15.43:0/940484169 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f684415de80 0x7f68440cc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:04.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.987+0000 7f6843fff640 1 --2- 172.21.15.43:0/940484169 >> v2:172.21.15.193:3300/0 conn(0x7f68440b2960 0x7f68440cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:04.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.987+0000 7f68437fe640 1 -- 172.21.15.43:0/940484169 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f684415de80 msgr2=0x7f68440cc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:04.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.987+0000 7f68437fe640 1 --2- 172.21.15.43:0/940484169 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f684415de80 0x7f68440cc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:04.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.987+0000 7f68437fe640 1 -- 172.21.15.43:0/940484169 >> v2:172.21.15.193:3300/0 conn(0x7f68440b2960 msgr2=0x7f68440cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:04.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.987+0000 7f68437fe640 1 --2- 172.21.15.43:0/940484169 >> v2:172.21.15.193:3300/0 conn(0x7f68440b2960 0x7f68440cc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:04.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.987+0000 7f68437fe640 1 -- 172.21.15.43:0/940484169 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68440b4840 con 0x7f684415daa0 2024-04-25T08:57:04.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.988+0000 7f68437fe640 1 --2- 172.21.15.43:0/940484169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f684415daa0 0x7f6844162220 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f68440b4e00 tx=0x7f683402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27d5f64200dda13d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:04.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.988+0000 7f68427fc640 1 -- 172.21.15.43:0/940484169 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6834037070 con 0x7f684415daa0 2024-04-25T08:57:04.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.988+0000 7f68427fc640 1 -- 172.21.15.43:0/940484169 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6834032440 con 0x7f684415daa0 2024-04-25T08:57:04.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.988+0000 7f68427fc640 1 -- 172.21.15.43:0/940484169 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6834032810 con 0x7f684415daa0 2024-04-25T08:57:04.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.989+0000 7f6849af5640 1 -- 172.21.15.43:0/940484169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f684415daa0 msgr2=0x7f6844162220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:04.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.989+0000 7f6849af5640 1 --2- 172.21.15.43:0/940484169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f684415daa0 0x7f6844162220 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f68440b4e00 tx=0x7f683402a1e0 comp rx=0 tx=0).stop 2024-04-25T08:57:04.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.989+0000 7f6849af5640 1 -- 172.21.15.43:0/940484169 shutdown_connections 2024-04-25T08:57:04.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.989+0000 7f6849af5640 1 --2- 172.21.15.43:0/940484169 >> v2:172.21.15.193:3300/0 conn(0x7f68440b2960 0x7f68440cc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:04.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.989+0000 7f6849af5640 1 --2- 172.21.15.43:0/940484169 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f684415de80 0x7f68440cc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:04.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.989+0000 7f6849af5640 1 --2- 172.21.15.43:0/940484169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f684415daa0 0x7f6844162220 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:04.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.989+0000 7f6849af5640 1 -- 172.21.15.43:0/940484169 >> 172.21.15.43:0/940484169 conn(0x7f68440b1660 msgr2=0x7f68440b1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:04.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.989+0000 7f6849af5640 1 -- 172.21.15.43:0/940484169 shutdown_connections 2024-04-25T08:57:04.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.989+0000 7f6849af5640 1 -- 172.21.15.43:0/940484169 wait complete. 2024-04-25T08:57:04.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.990+0000 7f6849af5640 1 Processor -- start 2024-04-25T08:57:04.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.990+0000 7f6849af5640 1 -- start start 2024-04-25T08:57:04.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.991+0000 7f6849af5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f68440b2960 0x7f6844145310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:04.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.991+0000 7f6849af5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f684415daa0 0x7f684415e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:04.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.991+0000 7f6849af5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f684415de80 0x7f684415e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:04.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.991+0000 7f6849af5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68440ad8e0 con 0x7f684415daa0 2024-04-25T08:57:04.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.991+0000 7f6849af5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68440cdba0 con 0x7f684415de80 2024-04-25T08:57:04.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.991+0000 7f6849af5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68440ada60 con 0x7f68440b2960 2024-04-25T08:57:04.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.991+0000 7f6842ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f684415daa0 0x7f684415e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:04.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.991+0000 7f6842ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f684415daa0 0x7f684415e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39638/0 (socket says 172.21.15.43:39638) 2024-04-25T08:57:04.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.991+0000 7f6842ffd640 1 -- 172.21.15.43:0/3728300443 learned_addr learned my addr 172.21.15.43:0/3728300443 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:04.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.991+0000 7f6843fff640 1 --2- 172.21.15.43:0/3728300443 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f684415de80 0x7f684415e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:04.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.991+0000 7f68437fe640 1 --2- 172.21.15.43:0/3728300443 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f68440b2960 0x7f6844145310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:04.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.991+0000 7f6842ffd640 1 -- 172.21.15.43:0/3728300443 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f68440b2960 msgr2=0x7f6844145310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:04.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.991+0000 7f6842ffd640 1 --2- 172.21.15.43:0/3728300443 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f68440b2960 0x7f6844145310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:04.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.992+0000 7f6842ffd640 1 -- 172.21.15.43:0/3728300443 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f684415de80 msgr2=0x7f684415e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:04.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.992+0000 7f6842ffd640 1 --2- 172.21.15.43:0/3728300443 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f684415de80 0x7f684415e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:04.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.992+0000 7f6842ffd640 1 -- 172.21.15.43:0/3728300443 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6844145980 con 0x7f684415daa0 2024-04-25T08:57:04.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.992+0000 7f6842ffd640 1 --2- 172.21.15.43:0/3728300443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f684415daa0 0x7f684415e290 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f683800e840 tx=0x7f683800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:04.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.992+0000 7f682bfff640 1 -- 172.21.15.43:0/3728300443 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6838017070 con 0x7f684415daa0 2024-04-25T08:57:04.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.992+0000 7f682bfff640 1 -- 172.21.15.43:0/3728300443 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f683800c940 con 0x7f684415daa0 2024-04-25T08:57:04.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.992+0000 7f682bfff640 1 -- 172.21.15.43:0/3728300443 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683800cd10 con 0x7f684415daa0 2024-04-25T08:57:04.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.992+0000 7f6849af5640 1 -- 172.21.15.43:0/3728300443 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f684406cb50 con 0x7f684415daa0 2024-04-25T08:57:04.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.992+0000 7f6849af5640 1 -- 172.21.15.43:0/3728300443 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f684406d110 con 0x7f684415daa0 2024-04-25T08:57:04.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.993+0000 7f6849af5640 1 -- 172.21.15.43:0/3728300443 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6844074ef0 con 0x7f684415daa0 2024-04-25T08:57:04.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.993+0000 7f682bfff640 1 -- 172.21.15.43:0/3728300443 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f683801a070 con 0x7f684415daa0 2024-04-25T08:57:04.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.994+0000 7f682bfff640 1 --2- 172.21.15.43:0/3728300443 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f683006b4d0 0x7f683006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:04.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.994+0000 7f682bfff640 1 -- 172.21.15.43:0/3728300443 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f683808a380 con 0x7f684415daa0 2024-04-25T08:57:04.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.994+0000 7f68437fe640 1 --2- 172.21.15.43:0/3728300443 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f683006b4d0 0x7f683006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:04.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.995+0000 7f68437fe640 1 --2- 172.21.15.43:0/3728300443 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f683006b4d0 0x7f683006d990 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6834034000 tx=0x7f683402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:04.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:04.998+0000 7f682bfff640 1 -- 172.21.15.43:0/3728300443 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6838059290 con 0x7f684415daa0 2024-04-25T08:57:05.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.179+0000 7f6849af5640 1 -- 172.21.15.43:0/3728300443 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f68440cd4c0 con 0x7f684415daa0 2024-04-25T08:57:05.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.180+0000 7f682bfff640 1 -- 172.21.15.43:0/3728300443 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16577 (secure 0 0 0) 0x7f6838058b90 con 0x7f684415daa0 2024-04-25T08:57:05.170 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:57:05.170 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":46,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:57:03.489946+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:57:00.135489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T08:57:00.830128+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:57:05.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.182+0000 7f6849af5640 1 -- 172.21.15.43:0/3728300443 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f683006b4d0 msgr2=0x7f683006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:05.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.182+0000 7f6849af5640 1 --2- 172.21.15.43:0/3728300443 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f683006b4d0 0x7f683006d990 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6834034000 tx=0x7f683402e000 comp rx=0 tx=0).stop 2024-04-25T08:57:05.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.182+0000 7f6849af5640 1 -- 172.21.15.43:0/3728300443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f684415daa0 msgr2=0x7f684415e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:05.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.182+0000 7f6849af5640 1 --2- 172.21.15.43:0/3728300443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f684415daa0 0x7f684415e290 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f683800e840 tx=0x7f683800ed00 comp rx=0 tx=0).stop 2024-04-25T08:57:05.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.182+0000 7f6849af5640 1 -- 172.21.15.43:0/3728300443 shutdown_connections 2024-04-25T08:57:05.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.182+0000 7f6849af5640 1 --2- 172.21.15.43:0/3728300443 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f683006b4d0 0x7f683006d990 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:05.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.182+0000 7f6849af5640 1 --2- 172.21.15.43:0/3728300443 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f684415de80 0x7f684415e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:05.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.182+0000 7f6849af5640 1 --2- 172.21.15.43:0/3728300443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f684415daa0 0x7f684415e290 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:05.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.182+0000 7f6849af5640 1 --2- 172.21.15.43:0/3728300443 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f68440b2960 0x7f6844145310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:05.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.182+0000 7f6849af5640 1 -- 172.21.15.43:0/3728300443 >> 172.21.15.43:0/3728300443 conn(0x7f68440b1660 msgr2=0x7f684415b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:05.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.182+0000 7f6849af5640 1 -- 172.21.15.43:0/3728300443 shutdown_connections 2024-04-25T08:57:05.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.182+0000 7f6849af5640 1 -- 172.21.15.43:0/3728300443 wait complete. 2024-04-25T08:57:05.183 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T08:57:05.183 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:57:05.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.295+0000 7fc7d3e4c640 1 Processor -- start 2024-04-25T08:57:05.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.295+0000 7fc7d3e4c640 1 -- start start 2024-04-25T08:57:05.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.296+0000 7fc7d3e4c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7cc15be60 0x7fc7cc0ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:05.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.296+0000 7fc7d3e4c640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fc7cc0cedc0 0x7fc7cc0cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:05.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.296+0000 7fc7d3e4c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc7cc0d02b0 0x7fc7cc15f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:05.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.296+0000 7fc7d3e4c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7cc15c240 con 0x7fc7cc0d02b0 2024-04-25T08:57:05.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.296+0000 7fc7d3e4c640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7cc0cc760 con 0x7fc7cc0cedc0 2024-04-25T08:57:05.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.296+0000 7fc7d3e4c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7cc0cca40 con 0x7fc7cc15be60 2024-04-25T08:57:05.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.296+0000 7fc7d23e9640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc7cc0d02b0 0x7fc7cc15f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:05.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.296+0000 7fc7d23e9640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc7cc0d02b0 0x7fc7cc15f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39654/0 (socket says 172.21.15.43:39654) 2024-04-25T08:57:05.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.296+0000 7fc7d23e9640 1 -- 172.21.15.43:0/867121579 learned_addr learned my addr 172.21.15.43:0/867121579 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:05.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.296+0000 7fc7d13e7640 1 --2- 172.21.15.43:0/867121579 >> v2:172.21.15.193:3300/0 conn(0x7fc7cc0cedc0 0x7fc7cc0cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:05.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.296+0000 7fc7d1be8640 1 --2- 172.21.15.43:0/867121579 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7cc15be60 0x7fc7cc0ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:05.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.296+0000 7fc7d23e9640 1 -- 172.21.15.43:0/867121579 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7cc15be60 msgr2=0x7fc7cc0ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:05.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.296+0000 7fc7d23e9640 1 --2- 172.21.15.43:0/867121579 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7cc15be60 0x7fc7cc0ce880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:05.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.296+0000 7fc7d23e9640 1 -- 172.21.15.43:0/867121579 >> v2:172.21.15.193:3300/0 conn(0x7fc7cc0cedc0 msgr2=0x7fc7cc0cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:05.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.296+0000 7fc7d23e9640 1 --2- 172.21.15.43:0/867121579 >> v2:172.21.15.193:3300/0 conn(0x7fc7cc0cedc0 0x7fc7cc0cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:05.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.296+0000 7fc7d23e9640 1 -- 172.21.15.43:0/867121579 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7cc0ccd20 con 0x7fc7cc0d02b0 2024-04-25T08:57:05.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.296+0000 7fc7d1be8640 1 --2- 172.21.15.43:0/867121579 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7cc15be60 0x7fc7cc0ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:05.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.297+0000 7fc7d23e9640 1 --2- 172.21.15.43:0/867121579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7cc0d02b0 0x7fc7cc15f8c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc071e30 tx=0x7fc7c402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b67e1754c66605f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:05.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.297+0000 7fc7d0be6640 1 -- 172.21.15.43:0/867121579 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c4036070 con 0x7fc7cc0d02b0 2024-04-25T08:57:05.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.297+0000 7fc7d0be6640 1 -- 172.21.15.43:0/867121579 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c4031600 con 0x7fc7cc0d02b0 2024-04-25T08:57:05.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.297+0000 7fc7d0be6640 1 -- 172.21.15.43:0/867121579 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c40319d0 con 0x7fc7cc0d02b0 2024-04-25T08:57:05.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.298+0000 7fc7d3e4c640 1 -- 172.21.15.43:0/867121579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7cc0d02b0 msgr2=0x7fc7cc15f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:05.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.298+0000 7fc7d3e4c640 1 --2- 172.21.15.43:0/867121579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7cc0d02b0 0x7fc7cc15f8c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc071e30 tx=0x7fc7c402a6a0 comp rx=0 tx=0).stop 2024-04-25T08:57:05.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.298+0000 7fc7d3e4c640 1 -- 172.21.15.43:0/867121579 shutdown_connections 2024-04-25T08:57:05.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.298+0000 7fc7d3e4c640 1 --2- 172.21.15.43:0/867121579 >> v2:172.21.15.193:3300/0 conn(0x7fc7cc0cedc0 0x7fc7cc0cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:05.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.298+0000 7fc7d3e4c640 1 --2- 172.21.15.43:0/867121579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7cc0d02b0 0x7fc7cc15f8c0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:05.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.298+0000 7fc7d3e4c640 1 --2- 172.21.15.43:0/867121579 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7cc15be60 0x7fc7cc0ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:05.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.298+0000 7fc7d3e4c640 1 -- 172.21.15.43:0/867121579 >> 172.21.15.43:0/867121579 conn(0x7fc7cc0b32e0 msgr2=0x7fc7cc0b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:05.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.298+0000 7fc7d3e4c640 1 -- 172.21.15.43:0/867121579 shutdown_connections 2024-04-25T08:57:05.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.298+0000 7fc7d3e4c640 1 -- 172.21.15.43:0/867121579 wait complete. 2024-04-25T08:57:05.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.299+0000 7fc7d3e4c640 1 Processor -- start 2024-04-25T08:57:05.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.299+0000 7fc7d3e4c640 1 -- start start 2024-04-25T08:57:05.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.300+0000 7fc7d3e4c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7cc0cedc0 0x7fc7cc0c6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:05.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.300+0000 7fc7d3e4c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7cc0d02b0 0x7fc7cc06d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:05.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.300+0000 7fc7d3e4c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7cc15be60 0x7fc7cc0cacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:05.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.300+0000 7fc7d3e4c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7cc0cc4b0 con 0x7fc7cc0d02b0 2024-04-25T08:57:05.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.300+0000 7fc7d3e4c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7cc0cc330 con 0x7fc7cc0cedc0 2024-04-25T08:57:05.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.300+0000 7fc7d3e4c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7cc15fe00 con 0x7fc7cc15be60 2024-04-25T08:57:05.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.300+0000 7fc7d13e7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7cc0d02b0 0x7fc7cc06d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:05.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.300+0000 7fc7d13e7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7cc0d02b0 0x7fc7cc06d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39660/0 (socket says 172.21.15.43:39660) 2024-04-25T08:57:05.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.300+0000 7fc7d13e7640 1 -- 172.21.15.43:0/3754834631 learned_addr learned my addr 172.21.15.43:0/3754834631 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:05.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.300+0000 7fc7d1be8640 1 --2- 172.21.15.43:0/3754834631 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7cc0cedc0 0x7fc7cc0c6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:05.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.300+0000 7fc7d23e9640 1 --2- 172.21.15.43:0/3754834631 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7cc15be60 0x7fc7cc0cacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:05.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.300+0000 7fc7d13e7640 1 -- 172.21.15.43:0/3754834631 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7cc15be60 msgr2=0x7fc7cc0cacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:05.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.300+0000 7fc7d13e7640 1 --2- 172.21.15.43:0/3754834631 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7cc15be60 0x7fc7cc0cacc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:05.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.300+0000 7fc7d13e7640 1 -- 172.21.15.43:0/3754834631 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7cc0cedc0 msgr2=0x7fc7cc0c6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:05.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.300+0000 7fc7d13e7640 1 --2- 172.21.15.43:0/3754834631 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7cc0cedc0 0x7fc7cc0c6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:05.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.300+0000 7fc7d13e7640 1 -- 172.21.15.43:0/3754834631 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7cc0c9c20 con 0x7fc7cc0d02b0 2024-04-25T08:57:05.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.300+0000 7fc7d1be8640 1 --2- 172.21.15.43:0/3754834631 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7cc0cedc0 0x7fc7cc0c6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:05.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.300+0000 7fc7d23e9640 1 --2- 172.21.15.43:0/3754834631 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7cc15be60 0x7fc7cc0cacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:05.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.300+0000 7fc7d13e7640 1 --2- 172.21.15.43:0/3754834631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7cc0d02b0 0x7fc7cc06d750 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc00e910 tx=0x7fc7bc00edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:05.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.300+0000 7fc7c27fc640 1 -- 172.21.15.43:0/3754834631 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc005990 con 0x7fc7cc0d02b0 2024-04-25T08:57:05.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.300+0000 7fc7c27fc640 1 -- 172.21.15.43:0/3754834631 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7bc005b30 con 0x7fc7cc0d02b0 2024-04-25T08:57:05.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.300+0000 7fc7d3e4c640 1 -- 172.21.15.43:0/3754834631 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7cc0c9f00 con 0x7fc7cc0d02b0 2024-04-25T08:57:05.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.300+0000 7fc7c27fc640 1 -- 172.21.15.43:0/3754834631 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc00c9a0 con 0x7fc7cc0d02b0 2024-04-25T08:57:05.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.300+0000 7fc7d3e4c640 1 -- 172.21.15.43:0/3754834631 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7cc0ca4c0 con 0x7fc7cc0d02b0 2024-04-25T08:57:05.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.301+0000 7fc7d3e4c640 1 -- 172.21.15.43:0/3754834631 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7cc074ef0 con 0x7fc7cc0d02b0 2024-04-25T08:57:05.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.302+0000 7fc7c27fc640 1 -- 172.21.15.43:0/3754834631 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc7bc00cb40 con 0x7fc7cc0d02b0 2024-04-25T08:57:05.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.302+0000 7fc7c27fc640 1 --2- 172.21.15.43:0/3754834631 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc7b006b4d0 0x7fc7b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:05.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.302+0000 7fc7c27fc640 1 -- 172.21.15.43:0/3754834631 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc7bc08a6e0 con 0x7fc7cc0d02b0 2024-04-25T08:57:05.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.302+0000 7fc7d1be8640 1 --2- 172.21.15.43:0/3754834631 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc7b006b4d0 0x7fc7b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:05.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.303+0000 7fc7d1be8640 1 --2- 172.21.15.43:0/3754834631 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc7b006b4d0 0x7fc7b006d990 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8006fd0 tx=0x7fc7c8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:05.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.306+0000 7fc7c27fc640 1 -- 172.21.15.43:0/3754834631 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc7bc017020 con 0x7fc7cc0d02b0 2024-04-25T08:57:05.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.486+0000 7fc7d3e4c640 1 -- 172.21.15.43:0/3754834631 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc7cc06dc90 con 0x7fc7b006b4d0 2024-04-25T08:57:05.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.493+0000 7fc7c27fc640 1 -- 172.21.15.43:0/3754834631 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251904 (secure 0 0 0) 0x7fc7c8003e90 con 0x7fc7b006b4d0 2024-04-25T08:57:05.485 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:57:05.487 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-25T08:57:05.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.496+0000 7fc7d3e4c640 1 -- 172.21.15.43:0/3754834631 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc7b006b4d0 msgr2=0x7fc7b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:05.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.496+0000 7fc7d3e4c640 1 --2- 172.21.15.43:0/3754834631 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc7b006b4d0 0x7fc7b006d990 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8006fd0 tx=0x7fc7c8008040 comp rx=0 tx=0).stop 2024-04-25T08:57:05.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.496+0000 7fc7d3e4c640 1 -- 172.21.15.43:0/3754834631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7cc0d02b0 msgr2=0x7fc7cc06d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:05.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.496+0000 7fc7d3e4c640 1 --2- 172.21.15.43:0/3754834631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7cc0d02b0 0x7fc7cc06d750 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc00e910 tx=0x7fc7bc00edd0 comp rx=0 tx=0).stop 2024-04-25T08:57:05.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.496+0000 7fc7d3e4c640 1 -- 172.21.15.43:0/3754834631 shutdown_connections 2024-04-25T08:57:05.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.496+0000 7fc7d3e4c640 1 --2- 172.21.15.43:0/3754834631 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7cc15be60 0x7fc7cc0cacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:05.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.496+0000 7fc7d3e4c640 1 --2- 172.21.15.43:0/3754834631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7cc0d02b0 0x7fc7cc06d750 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:05.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.496+0000 7fc7d3e4c640 1 --2- 172.21.15.43:0/3754834631 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc7b006b4d0 0x7fc7b006d990 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:05.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.496+0000 7fc7d3e4c640 1 --2- 172.21.15.43:0/3754834631 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7cc0cedc0 0x7fc7cc0c6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:05.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.496+0000 7fc7d3e4c640 1 -- 172.21.15.43:0/3754834631 >> 172.21.15.43:0/3754834631 conn(0x7fc7cc0b32e0 msgr2=0x7fc7cc159bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:05.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.496+0000 7fc7d3e4c640 1 -- 172.21.15.43:0/3754834631 shutdown_connections 2024-04-25T08:57:05.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.497+0000 7fc7d3e4c640 1 -- 172.21.15.43:0/3754834631 wait complete. 2024-04-25T08:57:05.497 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":132,"stamp":"2024-04-25T08:57:03.734023+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":85,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":350568,"kb_used_data":5364,"kb_used_omap":12,"kb_used_meta":345075,"kb_avail":754624152,"statfs":{"total":773094113280,"available":772735131648,"internally_reserved":0,"allocated":5492736,"data_stored":2920008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":353357361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000759"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.494359+0000","last_change":"2024-04-25T08:57:02.503942+0000","last_active":"2024-04-25T08:57:03.494359+0000","last_peered":"2024-04-25T08:57:03.494359+0000","last_clean":"2024-04-25T08:57:03.494359+0000","last_became_active":"2024-04-25T08:57:02.503679+0000","last_became_peered":"2024-04-25T08:57:02.503679+0000","last_unstale":"2024-04-25T08:57:03.494359+0000","last_undegraded":"2024-04-25T08:57:03.494359+0000","last_fullsized":"2024-04-25T08:57:03.494359+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:48.776242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.494119+0000","last_change":"2024-04-25T08:57:02.504936+0000","last_active":"2024-04-25T08:57:03.494119+0000","last_peered":"2024-04-25T08:57:03.494119+0000","last_clean":"2024-04-25T08:57:03.494119+0000","last_became_active":"2024-04-25T08:57:02.504644+0000","last_became_peered":"2024-04-25T08:57:02.504644+0000","last_unstale":"2024-04-25T08:57:03.494119+0000","last_undegraded":"2024-04-25T08:57:03.494119+0000","last_fullsized":"2024-04-25T08:57:03.494119+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:55.732881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.500611+0000","last_change":"2024-04-25T08:57:02.497224+0000","last_active":"2024-04-25T08:57:02.500611+0000","last_peered":"2024-04-25T08:57:02.500611+0000","last_clean":"2024-04-25T08:57:02.500611+0000","last_became_active":"2024-04-25T08:57:02.496963+0000","last_became_peered":"2024-04-25T08:57:02.496963+0000","last_unstale":"2024-04-25T08:57:02.500611+0000","last_undegraded":"2024-04-25T08:57:02.500611+0000","last_fullsized":"2024-04-25T08:57:02.500611+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:17.384450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.515346+0000","last_change":"2024-04-25T08:57:02.514066+0000","last_active":"2024-04-25T08:57:02.515346+0000","last_peered":"2024-04-25T08:57:02.515346+0000","last_clean":"2024-04-25T08:57:02.515346+0000","last_became_active":"2024-04-25T08:57:02.513768+0000","last_became_peered":"2024-04-25T08:57:02.513768+0000","last_unstale":"2024-04-25T08:57:02.515346+0000","last_undegraded":"2024-04-25T08:57:02.515346+0000","last_fullsized":"2024-04-25T08:57:02.515346+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:41.134579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.494010+0000","last_change":"2024-04-25T08:57:02.514299+0000","last_active":"2024-04-25T08:57:03.494010+0000","last_peered":"2024-04-25T08:57:03.494010+0000","last_clean":"2024-04-25T08:57:03.494010+0000","last_became_active":"2024-04-25T08:57:02.513896+0000","last_became_peered":"2024-04-25T08:57:02.513896+0000","last_unstale":"2024-04-25T08:57:03.494010+0000","last_undegraded":"2024-04-25T08:57:03.494010+0000","last_fullsized":"2024-04-25T08:57:03.494010+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:49.505425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.512000+0000","last_change":"2024-04-25T08:57:02.510740+0000","last_active":"2024-04-25T08:57:02.512000+0000","last_peered":"2024-04-25T08:57:02.512000+0000","last_clean":"2024-04-25T08:57:02.512000+0000","last_became_active":"2024-04-25T08:57:02.510470+0000","last_became_peered":"2024-04-25T08:57:02.510470+0000","last_unstale":"2024-04-25T08:57:02.512000+0000","last_undegraded":"2024-04-25T08:57:02.512000+0000","last_fullsized":"2024-04-25T08:57:02.512000+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:47.497446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.494728+0000","last_change":"2024-04-25T08:57:02.507150+0000","last_active":"2024-04-25T08:57:03.494728+0000","last_peered":"2024-04-25T08:57:03.494728+0000","last_clean":"2024-04-25T08:57:03.494728+0000","last_became_active":"2024-04-25T08:57:02.506886+0000","last_became_peered":"2024-04-25T08:57:02.506886+0000","last_unstale":"2024-04-25T08:57:03.494728+0000","last_undegraded":"2024-04-25T08:57:03.494728+0000","last_fullsized":"2024-04-25T08:57:03.494728+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:16.311585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.520333+0000","last_change":"2024-04-25T08:57:02.516441+0000","last_active":"2024-04-25T08:57:02.520333+0000","last_peered":"2024-04-25T08:57:02.520333+0000","last_clean":"2024-04-25T08:57:02.520333+0000","last_became_active":"2024-04-25T08:57:02.516142+0000","last_became_peered":"2024-04-25T08:57:02.516142+0000","last_unstale":"2024-04-25T08:57:02.520333+0000","last_undegraded":"2024-04-25T08:57:02.520333+0000","last_fullsized":"2024-04-25T08:57:02.520333+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:53.875005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.511048+0000","last_change":"2024-04-25T08:57:02.509459+0000","last_active":"2024-04-25T08:57:02.511048+0000","last_peered":"2024-04-25T08:57:02.511048+0000","last_clean":"2024-04-25T08:57:02.511048+0000","last_became_active":"2024-04-25T08:57:02.509189+0000","last_became_peered":"2024-04-25T08:57:02.509189+0000","last_unstale":"2024-04-25T08:57:02.511048+0000","last_undegraded":"2024-04-25T08:57:02.511048+0000","last_fullsized":"2024-04-25T08:57:02.511048+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:33.686876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.514999+0000","last_change":"2024-04-25T08:57:02.512888+0000","last_active":"2024-04-25T08:57:02.514999+0000","last_peered":"2024-04-25T08:57:02.514999+0000","last_clean":"2024-04-25T08:57:02.514999+0000","last_became_active":"2024-04-25T08:57:02.512591+0000","last_became_peered":"2024-04-25T08:57:02.512591+0000","last_unstale":"2024-04-25T08:57:02.514999+0000","last_undegraded":"2024-04-25T08:57:02.514999+0000","last_fullsized":"2024-04-25T08:57:02.514999+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:37.204952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.514175+0000","last_change":"2024-04-25T08:57:02.510186+0000","last_active":"2024-04-25T08:57:02.514175+0000","last_peered":"2024-04-25T08:57:02.514175+0000","last_clean":"2024-04-25T08:57:02.514175+0000","last_became_active":"2024-04-25T08:57:02.509863+0000","last_became_peered":"2024-04-25T08:57:02.509863+0000","last_unstale":"2024-04-25T08:57:02.514175+0000","last_undegraded":"2024-04-25T08:57:02.514175+0000","last_fullsized":"2024-04-25T08:57:02.514175+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:17.384450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.494333+0000","last_change":"2024-04-25T08:57:02.499383+0000","last_active":"2024-04-25T08:57:03.494333+0000","last_peered":"2024-04-25T08:57:03.494333+0000","last_clean":"2024-04-25T08:57:03.494333+0000","last_became_active":"2024-04-25T08:57:02.499105+0000","last_became_peered":"2024-04-25T08:57:02.499105+0000","last_unstale":"2024-04-25T08:57:03.494333+0000","last_undegraded":"2024-04-25T08:57:03.494333+0000","last_fullsized":"2024-04-25T08:57:03.494333+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:48.776242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.516236+0000","last_change":"2024-04-25T08:57:02.513293+0000","last_active":"2024-04-25T08:57:02.516236+0000","last_peered":"2024-04-25T08:57:02.516236+0000","last_clean":"2024-04-25T08:57:02.516236+0000","last_became_active":"2024-04-25T08:57:02.512997+0000","last_became_peered":"2024-04-25T08:57:02.512997+0000","last_unstale":"2024-04-25T08:57:02.516236+0000","last_undegraded":"2024-04-25T08:57:02.516236+0000","last_fullsized":"2024-04-25T08:57:02.516236+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:37.589340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.510906+0000","last_change":"2024-04-25T08:57:02.507964+0000","last_active":"2024-04-25T08:57:02.510906+0000","last_peered":"2024-04-25T08:57:02.510906+0000","last_clean":"2024-04-25T08:57:02.510906+0000","last_became_active":"2024-04-25T08:57:02.507666+0000","last_became_peered":"2024-04-25T08:57:02.507666+0000","last_unstale":"2024-04-25T08:57:02.510906+0000","last_undegraded":"2024-04-25T08:57:02.510906+0000","last_fullsized":"2024-04-25T08:57:02.510906+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:08:39.446712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.518197+0000","last_change":"2024-04-25T08:57:02.514617+0000","last_active":"2024-04-25T08:57:02.518197+0000","last_peered":"2024-04-25T08:57:02.518197+0000","last_clean":"2024-04-25T08:57:02.518197+0000","last_became_active":"2024-04-25T08:57:02.513909+0000","last_became_peered":"2024-04-25T08:57:02.513909+0000","last_unstale":"2024-04-25T08:57:02.518197+0000","last_undegraded":"2024-04-25T08:57:02.518197+0000","last_fullsized":"2024-04-25T08:57:02.518197+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:32.273069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.516658+0000","last_change":"2024-04-25T08:57:02.513269+0000","last_active":"2024-04-25T08:57:02.516658+0000","last_peered":"2024-04-25T08:57:02.516658+0000","last_clean":"2024-04-25T08:57:02.516658+0000","last_became_active":"2024-04-25T08:57:02.512693+0000","last_became_peered":"2024-04-25T08:57:02.512693+0000","last_unstale":"2024-04-25T08:57:02.516658+0000","last_undegraded":"2024-04-25T08:57:02.516658+0000","last_fullsized":"2024-04-25T08:57:02.516658+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:08.136531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.504766+0000","last_change":"2024-04-25T08:57:02.502915+0000","last_active":"2024-04-25T08:57:02.504766+0000","last_peered":"2024-04-25T08:57:02.504766+0000","last_clean":"2024-04-25T08:57:02.504766+0000","last_became_active":"2024-04-25T08:57:02.499469+0000","last_became_peered":"2024-04-25T08:57:02.499469+0000","last_unstale":"2024-04-25T08:57:02.504766+0000","last_undegraded":"2024-04-25T08:57:02.504766+0000","last_fullsized":"2024-04-25T08:57:02.504766+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:11.452008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.517534+0000","last_change":"2024-04-25T08:57:02.515325+0000","last_active":"2024-04-25T08:57:02.517534+0000","last_peered":"2024-04-25T08:57:02.517534+0000","last_clean":"2024-04-25T08:57:02.517534+0000","last_became_active":"2024-04-25T08:57:02.515044+0000","last_became_peered":"2024-04-25T08:57:02.515044+0000","last_unstale":"2024-04-25T08:57:02.517534+0000","last_undegraded":"2024-04-25T08:57:02.517534+0000","last_fullsized":"2024-04-25T08:57:02.517534+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:41.134579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.516602+0000","last_change":"2024-04-25T08:57:02.513045+0000","last_active":"2024-04-25T08:57:02.516602+0000","last_peered":"2024-04-25T08:57:02.516602+0000","last_clean":"2024-04-25T08:57:02.516602+0000","last_became_active":"2024-04-25T08:57:02.512688+0000","last_became_peered":"2024-04-25T08:57:02.512688+0000","last_unstale":"2024-04-25T08:57:02.516602+0000","last_undegraded":"2024-04-25T08:57:02.516602+0000","last_fullsized":"2024-04-25T08:57:02.516602+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:20.104362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.502567+0000","last_change":"2024-04-25T08:57:02.500014+0000","last_active":"2024-04-25T08:57:02.502567+0000","last_peered":"2024-04-25T08:57:02.502567+0000","last_clean":"2024-04-25T08:57:02.502567+0000","last_became_active":"2024-04-25T08:57:02.499666+0000","last_became_peered":"2024-04-25T08:57:02.499666+0000","last_unstale":"2024-04-25T08:57:02.502567+0000","last_undegraded":"2024-04-25T08:57:02.502567+0000","last_fullsized":"2024-04-25T08:57:02.502567+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:06:00.871028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.521737+0000","last_change":"2024-04-25T08:57:02.515882+0000","last_active":"2024-04-25T08:57:02.521737+0000","last_peered":"2024-04-25T08:57:02.521737+0000","last_clean":"2024-04-25T08:57:02.521737+0000","last_became_active":"2024-04-25T08:57:02.515600+0000","last_became_peered":"2024-04-25T08:57:02.515600+0000","last_unstale":"2024-04-25T08:57:02.521737+0000","last_undegraded":"2024-04-25T08:57:02.521737+0000","last_fullsized":"2024-04-25T08:57:02.521737+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:16.311585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.514280+0000","last_change":"2024-04-25T08:57:02.512418+0000","last_active":"2024-04-25T08:57:02.514280+0000","last_peered":"2024-04-25T08:57:02.514280+0000","last_clean":"2024-04-25T08:57:02.514280+0000","last_became_active":"2024-04-25T08:57:02.512155+0000","last_became_peered":"2024-04-25T08:57:02.512155+0000","last_unstale":"2024-04-25T08:57:02.514280+0000","last_undegraded":"2024-04-25T08:57:02.514280+0000","last_fullsized":"2024-04-25T08:57:02.514280+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:04.066281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.518509+0000","last_change":"2024-04-25T08:57:02.516698+0000","last_active":"2024-04-25T08:57:02.518509+0000","last_peered":"2024-04-25T08:57:02.518509+0000","last_clean":"2024-04-25T08:57:02.518509+0000","last_became_active":"2024-04-25T08:57:02.515485+0000","last_became_peered":"2024-04-25T08:57:02.515485+0000","last_unstale":"2024-04-25T08:57:02.518509+0000","last_undegraded":"2024-04-25T08:57:02.518509+0000","last_fullsized":"2024-04-25T08:57:02.518509+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:08:39.446712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.516641+0000","last_change":"2024-04-25T08:57:02.511150+0000","last_active":"2024-04-25T08:57:02.516641+0000","last_peered":"2024-04-25T08:57:02.516641+0000","last_clean":"2024-04-25T08:57:02.516641+0000","last_became_active":"2024-04-25T08:57:02.510885+0000","last_became_peered":"2024-04-25T08:57:02.510885+0000","last_unstale":"2024-04-25T08:57:02.516641+0000","last_undegraded":"2024-04-25T08:57:02.516641+0000","last_fullsized":"2024-04-25T08:57:02.516641+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:06:00.871028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.516684+0000","last_change":"2024-04-25T08:57:02.511183+0000","last_active":"2024-04-25T08:57:02.516684+0000","last_peered":"2024-04-25T08:57:02.516684+0000","last_clean":"2024-04-25T08:57:02.516684+0000","last_became_active":"2024-04-25T08:57:02.510589+0000","last_became_peered":"2024-04-25T08:57:02.510589+0000","last_unstale":"2024-04-25T08:57:02.516684+0000","last_undegraded":"2024-04-25T08:57:02.516684+0000","last_fullsized":"2024-04-25T08:57:02.516684+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:33.981688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.517394+0000","last_change":"2024-04-25T08:57:02.515890+0000","last_active":"2024-04-25T08:57:02.517394+0000","last_peered":"2024-04-25T08:57:02.517394+0000","last_clean":"2024-04-25T08:57:02.517394+0000","last_became_active":"2024-04-25T08:57:02.515618+0000","last_became_peered":"2024-04-25T08:57:02.515618+0000","last_unstale":"2024-04-25T08:57:02.517394+0000","last_undegraded":"2024-04-25T08:57:02.517394+0000","last_fullsized":"2024-04-25T08:57:02.517394+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:54.522484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.504607+0000","last_change":"2024-04-25T08:57:02.500245+0000","last_active":"2024-04-25T08:57:02.504607+0000","last_peered":"2024-04-25T08:57:02.504607+0000","last_clean":"2024-04-25T08:57:02.504607+0000","last_became_active":"2024-04-25T08:57:02.499953+0000","last_became_peered":"2024-04-25T08:57:02.499953+0000","last_unstale":"2024-04-25T08:57:02.504607+0000","last_undegraded":"2024-04-25T08:57:02.504607+0000","last_fullsized":"2024-04-25T08:57:02.504607+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:19.291277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.494137+0000","last_change":"2024-04-25T08:57:02.499102+0000","last_active":"2024-04-25T08:57:03.494137+0000","last_peered":"2024-04-25T08:57:03.494137+0000","last_clean":"2024-04-25T08:57:03.494137+0000","last_became_active":"2024-04-25T08:57:02.498735+0000","last_became_peered":"2024-04-25T08:57:02.498735+0000","last_unstale":"2024-04-25T08:57:03.494137+0000","last_undegraded":"2024-04-25T08:57:03.494137+0000","last_fullsized":"2024-04-25T08:57:03.494137+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:28.872188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.510295+0000","last_change":"2024-04-25T08:57:02.508944+0000","last_active":"2024-04-25T08:57:02.510295+0000","last_peered":"2024-04-25T08:57:02.510295+0000","last_clean":"2024-04-25T08:57:02.510295+0000","last_became_active":"2024-04-25T08:57:02.508661+0000","last_became_peered":"2024-04-25T08:57:02.508661+0000","last_unstale":"2024-04-25T08:57:02.510295+0000","last_undegraded":"2024-04-25T08:57:02.510295+0000","last_fullsized":"2024-04-25T08:57:02.510295+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:53.164910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.501469+0000","last_change":"2024-04-25T08:57:02.500327+0000","last_active":"2024-04-25T08:57:02.501469+0000","last_peered":"2024-04-25T08:57:02.501469+0000","last_clean":"2024-04-25T08:57:02.501469+0000","last_became_active":"2024-04-25T08:57:02.500082+0000","last_became_peered":"2024-04-25T08:57:02.500082+0000","last_unstale":"2024-04-25T08:57:02.501469+0000","last_undegraded":"2024-04-25T08:57:02.501469+0000","last_fullsized":"2024-04-25T08:57:02.501469+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:04.066281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.511897+0000","last_change":"2024-04-25T08:57:02.508803+0000","last_active":"2024-04-25T08:57:02.511897+0000","last_peered":"2024-04-25T08:57:02.511897+0000","last_clean":"2024-04-25T08:57:02.511897+0000","last_became_active":"2024-04-25T08:57:02.508530+0000","last_became_peered":"2024-04-25T08:57:02.508530+0000","last_unstale":"2024-04-25T08:57:02.511897+0000","last_undegraded":"2024-04-25T08:57:02.511897+0000","last_fullsized":"2024-04-25T08:57:02.511897+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:48.776242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.502630+0000","last_change":"2024-04-25T08:57:02.500243+0000","last_active":"2024-04-25T08:57:02.502630+0000","last_peered":"2024-04-25T08:57:02.502630+0000","last_clean":"2024-04-25T08:57:02.502630+0000","last_became_active":"2024-04-25T08:57:02.498907+0000","last_became_peered":"2024-04-25T08:57:02.498907+0000","last_unstale":"2024-04-25T08:57:02.502630+0000","last_undegraded":"2024-04-25T08:57:02.502630+0000","last_fullsized":"2024-04-25T08:57:02.502630+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:33.981688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.493024+0000","last_change":"2024-04-25T08:57:02.515096+0000","last_active":"2024-04-25T08:57:03.493024+0000","last_peered":"2024-04-25T08:57:03.493024+0000","last_clean":"2024-04-25T08:57:03.493024+0000","last_became_active":"2024-04-25T08:57:02.514800+0000","last_became_peered":"2024-04-25T08:57:02.514800+0000","last_unstale":"2024-04-25T08:57:03.493024+0000","last_undegraded":"2024-04-25T08:57:03.493024+0000","last_fullsized":"2024-04-25T08:57:03.493024+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:24.167958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"45'4","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.523291+0000","last_change":"2024-04-25T08:57:01.502910+0000","last_active":"2024-04-25T08:57:02.523291+0000","last_peered":"2024-04-25T08:57:02.523291+0000","last_clean":"2024-04-25T08:57:02.523291+0000","last_became_active":"2024-04-25T08:57:01.502588+0000","last_became_peered":"2024-04-25T08:57:01.502588+0000","last_unstale":"2024-04-25T08:57:02.523291+0000","last_undegraded":"2024-04-25T08:57:02.523291+0000","last_fullsized":"2024-04-25T08:57:02.523291+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:48.497715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.516702+0000","last_change":"2024-04-25T08:57:02.511370+0000","last_active":"2024-04-25T08:57:02.516702+0000","last_peered":"2024-04-25T08:57:02.516702+0000","last_clean":"2024-04-25T08:57:02.516702+0000","last_became_active":"2024-04-25T08:57:02.510876+0000","last_became_peered":"2024-04-25T08:57:02.510876+0000","last_unstale":"2024-04-25T08:57:02.516702+0000","last_undegraded":"2024-04-25T08:57:02.516702+0000","last_fullsized":"2024-04-25T08:57:02.516702+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:59.198644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.518690+0000","last_change":"2024-04-25T08:57:01.500492+0000","last_active":"2024-04-25T08:57:02.518690+0000","last_peered":"2024-04-25T08:57:02.518690+0000","last_clean":"2024-04-25T08:57:02.518690+0000","last_became_active":"2024-04-25T08:57:01.500205+0000","last_became_peered":"2024-04-25T08:57:01.500205+0000","last_unstale":"2024-04-25T08:57:02.518690+0000","last_undegraded":"2024-04-25T08:57:02.518690+0000","last_fullsized":"2024-04-25T08:57:02.518690+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:28.006717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.494281+0000","last_change":"2024-04-25T08:57:02.514084+0000","last_active":"2024-04-25T08:57:03.494281+0000","last_peered":"2024-04-25T08:57:03.494281+0000","last_clean":"2024-04-25T08:57:03.494281+0000","last_became_active":"2024-04-25T08:57:02.513586+0000","last_became_peered":"2024-04-25T08:57:02.513586+0000","last_unstale":"2024-04-25T08:57:03.494281+0000","last_undegraded":"2024-04-25T08:57:03.494281+0000","last_fullsized":"2024-04-25T08:57:03.494281+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:25.615662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.493049+0000","last_change":"2024-04-25T08:57:01.502381+0000","last_active":"2024-04-25T08:57:03.493049+0000","last_peered":"2024-04-25T08:57:03.493049+0000","last_clean":"2024-04-25T08:57:03.493049+0000","last_became_active":"2024-04-25T08:57:01.501850+0000","last_became_peered":"2024-04-25T08:57:01.501850+0000","last_unstale":"2024-04-25T08:57:03.493049+0000","last_undegraded":"2024-04-25T08:57:03.493049+0000","last_fullsized":"2024-04-25T08:57:03.493049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:04.894170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.493069+0000","last_change":"2024-04-25T08:57:02.513565+0000","last_active":"2024-04-25T08:57:03.493069+0000","last_peered":"2024-04-25T08:57:03.493069+0000","last_clean":"2024-04-25T08:57:03.493069+0000","last_became_active":"2024-04-25T08:57:02.512682+0000","last_became_peered":"2024-04-25T08:57:02.512682+0000","last_unstale":"2024-04-25T08:57:03.493069+0000","last_undegraded":"2024-04-25T08:57:03.493069+0000","last_fullsized":"2024-04-25T08:57:03.493069+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:31.544490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.494427+0000","last_change":"2024-04-25T08:57:01.491474+0000","last_active":"2024-04-25T08:57:03.494427+0000","last_peered":"2024-04-25T08:57:03.494427+0000","last_clean":"2024-04-25T08:57:03.494427+0000","last_became_active":"2024-04-25T08:57:01.491227+0000","last_became_peered":"2024-04-25T08:57:01.491227+0000","last_unstale":"2024-04-25T08:57:03.494427+0000","last_undegraded":"2024-04-25T08:57:03.494427+0000","last_fullsized":"2024-04-25T08:57:03.494427+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:10.361510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.517300+0000","last_change":"2024-04-25T08:57:02.516299+0000","last_active":"2024-04-25T08:57:02.517300+0000","last_peered":"2024-04-25T08:57:02.517300+0000","last_clean":"2024-04-25T08:57:02.517300+0000","last_became_active":"2024-04-25T08:57:02.516046+0000","last_became_peered":"2024-04-25T08:57:02.516046+0000","last_unstale":"2024-04-25T08:57:02.517300+0000","last_undegraded":"2024-04-25T08:57:02.517300+0000","last_fullsized":"2024-04-25T08:57:02.517300+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:53.875005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.523080+0000","last_change":"2024-04-25T08:57:01.506122+0000","last_active":"2024-04-25T08:57:02.523080+0000","last_peered":"2024-04-25T08:57:02.523080+0000","last_clean":"2024-04-25T08:57:02.523080+0000","last_became_active":"2024-04-25T08:57:01.505760+0000","last_became_peered":"2024-04-25T08:57:01.505760+0000","last_unstale":"2024-04-25T08:57:02.523080+0000","last_undegraded":"2024-04-25T08:57:02.523080+0000","last_fullsized":"2024-04-25T08:57:02.523080+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:49.105790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.514228+0000","last_change":"2024-04-25T08:57:02.511604+0000","last_active":"2024-04-25T08:57:02.514228+0000","last_peered":"2024-04-25T08:57:02.514228+0000","last_clean":"2024-04-25T08:57:02.514228+0000","last_became_active":"2024-04-25T08:57:02.510394+0000","last_became_peered":"2024-04-25T08:57:02.510394+0000","last_unstale":"2024-04-25T08:57:02.514228+0000","last_undegraded":"2024-04-25T08:57:02.514228+0000","last_fullsized":"2024-04-25T08:57:02.514228+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:19.291277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.516621+0000","last_change":"2024-04-25T08:57:01.496439+0000","last_active":"2024-04-25T08:57:02.516621+0000","last_peered":"2024-04-25T08:57:02.516621+0000","last_clean":"2024-04-25T08:57:02.516621+0000","last_became_active":"2024-04-25T08:57:01.496084+0000","last_became_peered":"2024-04-25T08:57:01.496084+0000","last_unstale":"2024-04-25T08:57:02.516621+0000","last_undegraded":"2024-04-25T08:57:02.516621+0000","last_fullsized":"2024-04-25T08:57:02.516621+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:32.679165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.520357+0000","last_change":"2024-04-25T08:57:02.515231+0000","last_active":"2024-04-25T08:57:02.520357+0000","last_peered":"2024-04-25T08:57:02.520357+0000","last_clean":"2024-04-25T08:57:02.520357+0000","last_became_active":"2024-04-25T08:57:02.514945+0000","last_became_peered":"2024-04-25T08:57:02.514945+0000","last_unstale":"2024-04-25T08:57:02.520357+0000","last_undegraded":"2024-04-25T08:57:02.520357+0000","last_fullsized":"2024-04-25T08:57:02.520357+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:30.086945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.494317+0000","last_change":"2024-04-25T08:57:01.496375+0000","last_active":"2024-04-25T08:57:03.494317+0000","last_peered":"2024-04-25T08:57:03.494317+0000","last_clean":"2024-04-25T08:57:03.494317+0000","last_became_active":"2024-04-25T08:57:01.496089+0000","last_became_peered":"2024-04-25T08:57:01.496089+0000","last_unstale":"2024-04-25T08:57:03.494317+0000","last_undegraded":"2024-04-25T08:57:03.494317+0000","last_fullsized":"2024-04-25T08:57:03.494317+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:29.079235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.518066+0000","last_change":"2024-04-25T08:57:02.516253+0000","last_active":"2024-04-25T08:57:02.518066+0000","last_peered":"2024-04-25T08:57:02.518066+0000","last_clean":"2024-04-25T08:57:02.518066+0000","last_became_active":"2024-04-25T08:57:02.515659+0000","last_became_peered":"2024-04-25T08:57:02.515659+0000","last_unstale":"2024-04-25T08:57:02.518066+0000","last_undegraded":"2024-04-25T08:57:02.518066+0000","last_fullsized":"2024-04-25T08:57:02.518066+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:33.381650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.499079+0000","last_change":"2024-04-25T08:57:01.500516+0000","last_active":"2024-04-25T08:57:02.499079+0000","last_peered":"2024-04-25T08:57:02.499079+0000","last_clean":"2024-04-25T08:57:02.499079+0000","last_became_active":"2024-04-25T08:57:01.500223+0000","last_became_peered":"2024-04-25T08:57:01.500223+0000","last_unstale":"2024-04-25T08:57:02.499079+0000","last_undegraded":"2024-04-25T08:57:02.499079+0000","last_fullsized":"2024-04-25T08:57:02.499079+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:36.581630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.510620+0000","last_change":"2024-04-25T08:57:02.509822+0000","last_active":"2024-04-25T08:57:02.510620+0000","last_peered":"2024-04-25T08:57:02.510620+0000","last_clean":"2024-04-25T08:57:02.510620+0000","last_became_active":"2024-04-25T08:57:02.509554+0000","last_became_peered":"2024-04-25T08:57:02.509554+0000","last_unstale":"2024-04-25T08:57:02.510620+0000","last_undegraded":"2024-04-25T08:57:02.510620+0000","last_fullsized":"2024-04-25T08:57:02.510620+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:55.333244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.495566+0000","last_change":"2024-04-25T08:57:01.493058+0000","last_active":"2024-04-25T08:57:02.495566+0000","last_peered":"2024-04-25T08:57:02.495566+0000","last_clean":"2024-04-25T08:57:02.495566+0000","last_became_active":"2024-04-25T08:57:01.492785+0000","last_became_peered":"2024-04-25T08:57:01.492785+0000","last_unstale":"2024-04-25T08:57:02.495566+0000","last_undegraded":"2024-04-25T08:57:02.495566+0000","last_fullsized":"2024-04-25T08:57:02.495566+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:32.373940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.494532+0000","last_change":"2024-04-25T08:57:02.500485+0000","last_active":"2024-04-25T08:57:03.494532+0000","last_peered":"2024-04-25T08:57:03.494532+0000","last_clean":"2024-04-25T08:57:03.494532+0000","last_became_active":"2024-04-25T08:57:02.500201+0000","last_became_peered":"2024-04-25T08:57:02.500201+0000","last_unstale":"2024-04-25T08:57:03.494532+0000","last_undegraded":"2024-04-25T08:57:03.494532+0000","last_fullsized":"2024-04-25T08:57:03.494532+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:47.497446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.503757+0000","last_change":"2024-04-25T08:57:01.497442+0000","last_active":"2024-04-25T08:57:02.503757+0000","last_peered":"2024-04-25T08:57:02.503757+0000","last_clean":"2024-04-25T08:57:02.503757+0000","last_became_active":"2024-04-25T08:57:01.497157+0000","last_became_peered":"2024-04-25T08:57:01.497157+0000","last_unstale":"2024-04-25T08:57:02.503757+0000","last_undegraded":"2024-04-25T08:57:02.503757+0000","last_fullsized":"2024-04-25T08:57:02.503757+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:28.006717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.522875+0000","last_change":"2024-04-25T08:57:02.517863+0000","last_active":"2024-04-25T08:57:02.522875+0000","last_peered":"2024-04-25T08:57:02.522875+0000","last_clean":"2024-04-25T08:57:02.522875+0000","last_became_active":"2024-04-25T08:57:02.517471+0000","last_became_peered":"2024-04-25T08:57:02.517471+0000","last_unstale":"2024-04-25T08:57:02.522875+0000","last_undegraded":"2024-04-25T08:57:02.522875+0000","last_fullsized":"2024-04-25T08:57:02.522875+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:23.745722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"45'3","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.494871+0000","last_change":"2024-04-25T08:57:01.499229+0000","last_active":"2024-04-25T08:57:03.494871+0000","last_peered":"2024-04-25T08:57:03.494871+0000","last_clean":"2024-04-25T08:57:03.494871+0000","last_became_active":"2024-04-25T08:57:01.498950+0000","last_became_peered":"2024-04-25T08:57:01.498950+0000","last_unstale":"2024-04-25T08:57:03.494871+0000","last_undegraded":"2024-04-25T08:57:03.494871+0000","last_fullsized":"2024-04-25T08:57:03.494871+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:30.536780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.494724+0000","last_change":"2024-04-25T08:57:02.507927+0000","last_active":"2024-04-25T08:57:03.494724+0000","last_peered":"2024-04-25T08:57:03.494724+0000","last_clean":"2024-04-25T08:57:03.494724+0000","last_became_active":"2024-04-25T08:57:02.507607+0000","last_became_peered":"2024-04-25T08:57:02.507607+0000","last_unstale":"2024-04-25T08:57:03.494724+0000","last_undegraded":"2024-04-25T08:57:03.494724+0000","last_fullsized":"2024-04-25T08:57:03.494724+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:32.273069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"45'6","reported_seq":18,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.524755+0000","last_change":"2024-04-25T08:57:01.495892+0000","last_active":"2024-04-25T08:57:02.524755+0000","last_peered":"2024-04-25T08:57:02.524755+0000","last_clean":"2024-04-25T08:57:02.524755+0000","last_became_active":"2024-04-25T08:57:01.495572+0000","last_became_peered":"2024-04-25T08:57:01.495572+0000","last_unstale":"2024-04-25T08:57:02.524755+0000","last_undegraded":"2024-04-25T08:57:02.524755+0000","last_fullsized":"2024-04-25T08:57:02.524755+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:52.157200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.516610+0000","last_change":"2024-04-25T08:57:02.512174+0000","last_active":"2024-04-25T08:57:02.516610+0000","last_peered":"2024-04-25T08:57:02.516610+0000","last_clean":"2024-04-25T08:57:02.516610+0000","last_became_active":"2024-04-25T08:57:02.511898+0000","last_became_peered":"2024-04-25T08:57:02.511898+0000","last_unstale":"2024-04-25T08:57:02.516610+0000","last_undegraded":"2024-04-25T08:57:02.516610+0000","last_fullsized":"2024-04-25T08:57:02.516610+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:37.589340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.524047+0000","last_change":"2024-04-25T08:57:01.504685+0000","last_active":"2024-04-25T08:57:02.524047+0000","last_peered":"2024-04-25T08:57:02.524047+0000","last_clean":"2024-04-25T08:57:02.524047+0000","last_became_active":"2024-04-25T08:57:01.504421+0000","last_became_peered":"2024-04-25T08:57:01.504421+0000","last_unstale":"2024-04-25T08:57:02.524047+0000","last_undegraded":"2024-04-25T08:57:02.524047+0000","last_fullsized":"2024-04-25T08:57:02.524047+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:22.222587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.514958+0000","last_change":"2024-04-25T08:57:02.512553+0000","last_active":"2024-04-25T08:57:02.514958+0000","last_peered":"2024-04-25T08:57:02.514958+0000","last_clean":"2024-04-25T08:57:02.514958+0000","last_became_active":"2024-04-25T08:57:02.510690+0000","last_became_peered":"2024-04-25T08:57:02.510690+0000","last_unstale":"2024-04-25T08:57:02.514958+0000","last_undegraded":"2024-04-25T08:57:02.514958+0000","last_fullsized":"2024-04-25T08:57:02.514958+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:11.452008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.494018+0000","last_change":"2024-04-25T08:57:01.490352+0000","last_active":"2024-04-25T08:57:03.494018+0000","last_peered":"2024-04-25T08:57:03.494018+0000","last_clean":"2024-04-25T08:57:03.494018+0000","last_became_active":"2024-04-25T08:57:01.490082+0000","last_became_peered":"2024-04-25T08:57:01.490082+0000","last_unstale":"2024-04-25T08:57:03.494018+0000","last_undegraded":"2024-04-25T08:57:03.494018+0000","last_fullsized":"2024-04-25T08:57:03.494018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:23.160248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.521772+0000","last_change":"2024-04-25T08:57:02.517148+0000","last_active":"2024-04-25T08:57:02.521772+0000","last_peered":"2024-04-25T08:57:02.521772+0000","last_clean":"2024-04-25T08:57:02.521772+0000","last_became_active":"2024-04-25T08:57:02.516844+0000","last_became_peered":"2024-04-25T08:57:02.516844+0000","last_unstale":"2024-04-25T08:57:02.521772+0000","last_undegraded":"2024-04-25T08:57:02.521772+0000","last_fullsized":"2024-04-25T08:57:02.521772+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:19.399992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.502719+0000","last_change":"2024-04-25T08:57:01.504311+0000","last_active":"2024-04-25T08:57:02.502719+0000","last_peered":"2024-04-25T08:57:02.502719+0000","last_clean":"2024-04-25T08:57:02.502719+0000","last_became_active":"2024-04-25T08:57:01.503993+0000","last_became_peered":"2024-04-25T08:57:01.503993+0000","last_unstale":"2024-04-25T08:57:02.502719+0000","last_undegraded":"2024-04-25T08:57:02.502719+0000","last_fullsized":"2024-04-25T08:57:02.502719+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:46.623429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.494798+0000","last_change":"2024-04-25T08:57:02.514973+0000","last_active":"2024-04-25T08:57:03.494798+0000","last_peered":"2024-04-25T08:57:03.494798+0000","last_clean":"2024-04-25T08:57:03.494798+0000","last_became_active":"2024-04-25T08:57:02.514512+0000","last_became_peered":"2024-04-25T08:57:02.514512+0000","last_unstale":"2024-04-25T08:57:03.494798+0000","last_undegraded":"2024-04-25T08:57:03.494798+0000","last_fullsized":"2024-04-25T08:57:03.494798+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:10.240677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"45'8","reported_seq":20,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.524857+0000","last_change":"2024-04-25T08:57:01.493800+0000","last_active":"2024-04-25T08:57:02.524857+0000","last_peered":"2024-04-25T08:57:02.524857+0000","last_clean":"2024-04-25T08:57:02.524857+0000","last_became_active":"2024-04-25T08:57:01.493491+0000","last_became_peered":"2024-04-25T08:57:01.493491+0000","last_unstale":"2024-04-25T08:57:02.524857+0000","last_undegraded":"2024-04-25T08:57:02.524857+0000","last_fullsized":"2024-04-25T08:57:02.524857+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:53.514774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.501569+0000","last_change":"2024-04-25T08:57:02.498331+0000","last_active":"2024-04-25T08:57:02.501569+0000","last_peered":"2024-04-25T08:57:02.501569+0000","last_clean":"2024-04-25T08:57:02.501569+0000","last_became_active":"2024-04-25T08:57:02.497873+0000","last_became_peered":"2024-04-25T08:57:02.497873+0000","last_unstale":"2024-04-25T08:57:02.501569+0000","last_undegraded":"2024-04-25T08:57:02.501569+0000","last_fullsized":"2024-04-25T08:57:02.501569+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:45.000612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.523880+0000","last_change":"2024-04-25T08:57:01.503767+0000","last_active":"2024-04-25T08:57:02.523880+0000","last_peered":"2024-04-25T08:57:02.523880+0000","last_clean":"2024-04-25T08:57:02.523880+0000","last_became_active":"2024-04-25T08:57:01.503446+0000","last_became_peered":"2024-04-25T08:57:01.503446+0000","last_unstale":"2024-04-25T08:57:02.523880+0000","last_undegraded":"2024-04-25T08:57:02.523880+0000","last_fullsized":"2024-04-25T08:57:02.523880+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:26.416915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.514880+0000","last_change":"2024-04-25T08:57:02.512706+0000","last_active":"2024-04-25T08:57:02.514880+0000","last_peered":"2024-04-25T08:57:02.514880+0000","last_clean":"2024-04-25T08:57:02.514880+0000","last_became_active":"2024-04-25T08:57:02.511272+0000","last_became_peered":"2024-04-25T08:57:02.511272+0000","last_unstale":"2024-04-25T08:57:02.514880+0000","last_undegraded":"2024-04-25T08:57:02.514880+0000","last_fullsized":"2024-04-25T08:57:02.514880+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:55.333244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.521747+0000","last_change":"2024-04-25T08:57:01.498697+0000","last_active":"2024-04-25T08:57:02.521747+0000","last_peered":"2024-04-25T08:57:02.521747+0000","last_clean":"2024-04-25T08:57:02.521747+0000","last_became_active":"2024-04-25T08:57:01.497943+0000","last_became_peered":"2024-04-25T08:57:01.497943+0000","last_unstale":"2024-04-25T08:57:02.521747+0000","last_undegraded":"2024-04-25T08:57:02.521747+0000","last_fullsized":"2024-04-25T08:57:02.521747+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:07.905530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.516576+0000","last_change":"2024-04-25T08:57:02.512544+0000","last_active":"2024-04-25T08:57:02.516576+0000","last_peered":"2024-04-25T08:57:02.516576+0000","last_clean":"2024-04-25T08:57:02.516576+0000","last_became_active":"2024-04-25T08:57:02.512246+0000","last_became_peered":"2024-04-25T08:57:02.512246+0000","last_unstale":"2024-04-25T08:57:02.516576+0000","last_undegraded":"2024-04-25T08:57:02.516576+0000","last_fullsized":"2024-04-25T08:57:02.516576+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:54.522484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.494392+0000","last_change":"2024-04-25T08:57:01.502283+0000","last_active":"2024-04-25T08:57:03.494392+0000","last_peered":"2024-04-25T08:57:03.494392+0000","last_clean":"2024-04-25T08:57:03.494392+0000","last_became_active":"2024-04-25T08:57:01.501995+0000","last_became_peered":"2024-04-25T08:57:01.501995+0000","last_unstale":"2024-04-25T08:57:03.494392+0000","last_undegraded":"2024-04-25T08:57:03.494392+0000","last_fullsized":"2024-04-25T08:57:03.494392+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:04.894170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.515512+0000","last_change":"2024-04-25T08:57:02.509526+0000","last_active":"2024-04-25T08:57:02.515512+0000","last_peered":"2024-04-25T08:57:02.515512+0000","last_clean":"2024-04-25T08:57:02.515512+0000","last_became_active":"2024-04-25T08:57:02.509264+0000","last_became_peered":"2024-04-25T08:57:02.509264+0000","last_unstale":"2024-04-25T08:57:02.515512+0000","last_undegraded":"2024-04-25T08:57:02.515512+0000","last_fullsized":"2024-04-25T08:57:02.515512+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:45.000612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.500931+0000","last_change":"2024-04-25T08:57:01.502094+0000","last_active":"2024-04-25T08:57:02.500931+0000","last_peered":"2024-04-25T08:57:02.500931+0000","last_clean":"2024-04-25T08:57:02.500931+0000","last_became_active":"2024-04-25T08:57:01.501522+0000","last_became_peered":"2024-04-25T08:57:01.501522+0000","last_unstale":"2024-04-25T08:57:02.500931+0000","last_undegraded":"2024-04-25T08:57:02.500931+0000","last_fullsized":"2024-04-25T08:57:02.500931+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:30.536780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.511982+0000","last_change":"2024-04-25T08:57:02.510129+0000","last_active":"2024-04-25T08:57:02.511982+0000","last_peered":"2024-04-25T08:57:02.511982+0000","last_clean":"2024-04-25T08:57:02.511982+0000","last_became_active":"2024-04-25T08:57:02.508715+0000","last_became_peered":"2024-04-25T08:57:02.508715+0000","last_unstale":"2024-04-25T08:57:02.511982+0000","last_undegraded":"2024-04-25T08:57:02.511982+0000","last_fullsized":"2024-04-25T08:57:02.511982+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:25.615662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.523169+0000","last_change":"2024-04-25T08:57:01.505370+0000","last_active":"2024-04-25T08:57:02.523169+0000","last_peered":"2024-04-25T08:57:02.523169+0000","last_clean":"2024-04-25T08:57:02.523169+0000","last_became_active":"2024-04-25T08:57:01.505091+0000","last_became_peered":"2024-04-25T08:57:01.505091+0000","last_unstale":"2024-04-25T08:57:02.523169+0000","last_undegraded":"2024-04-25T08:57:02.523169+0000","last_fullsized":"2024-04-25T08:57:02.523169+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:30.144957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.492898+0000","last_change":"2024-04-25T08:57:02.499125+0000","last_active":"2024-04-25T08:57:03.492898+0000","last_peered":"2024-04-25T08:57:03.492898+0000","last_clean":"2024-04-25T08:57:03.492898+0000","last_became_active":"2024-04-25T08:57:02.498822+0000","last_became_peered":"2024-04-25T08:57:02.498822+0000","last_unstale":"2024-04-25T08:57:03.492898+0000","last_undegraded":"2024-04-25T08:57:03.492898+0000","last_fullsized":"2024-04-25T08:57:03.492898+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:20.604306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.494311+0000","last_change":"2024-04-25T08:57:01.497294+0000","last_active":"2024-04-25T08:57:03.494311+0000","last_peered":"2024-04-25T08:57:03.494311+0000","last_clean":"2024-04-25T08:57:03.494311+0000","last_became_active":"2024-04-25T08:57:01.496801+0000","last_became_peered":"2024-04-25T08:57:01.496801+0000","last_unstale":"2024-04-25T08:57:03.494311+0000","last_undegraded":"2024-04-25T08:57:03.494311+0000","last_fullsized":"2024-04-25T08:57:03.494311+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:10.361510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.502651+0000","last_change":"2024-04-25T08:57:02.500965+0000","last_active":"2024-04-25T08:57:02.502651+0000","last_peered":"2024-04-25T08:57:02.502651+0000","last_clean":"2024-04-25T08:57:02.502651+0000","last_became_active":"2024-04-25T08:57:02.500637+0000","last_became_peered":"2024-04-25T08:57:02.500637+0000","last_unstale":"2024-04-25T08:57:02.502651+0000","last_undegraded":"2024-04-25T08:57:02.502651+0000","last_fullsized":"2024-04-25T08:57:02.502651+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:59.198644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.500740+0000","last_change":"2024-04-25T08:57:01.496800+0000","last_active":"2024-04-25T08:57:02.500740+0000","last_peered":"2024-04-25T08:57:02.500740+0000","last_clean":"2024-04-25T08:57:02.500740+0000","last_became_active":"2024-04-25T08:57:01.496508+0000","last_became_peered":"2024-04-25T08:57:01.496508+0000","last_unstale":"2024-04-25T08:57:02.500740+0000","last_undegraded":"2024-04-25T08:57:02.500740+0000","last_fullsized":"2024-04-25T08:57:02.500740+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:58.190934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.510936+0000","last_change":"2024-04-25T08:57:02.508703+0000","last_active":"2024-04-25T08:57:02.510936+0000","last_peered":"2024-04-25T08:57:02.510936+0000","last_clean":"2024-04-25T08:57:02.510936+0000","last_became_active":"2024-04-25T08:57:02.508442+0000","last_became_peered":"2024-04-25T08:57:02.508442+0000","last_unstale":"2024-04-25T08:57:02.510936+0000","last_undegraded":"2024-04-25T08:57:02.510936+0000","last_fullsized":"2024-04-25T08:57:02.510936+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:08.913240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.493910+0000","last_change":"2024-04-25T08:57:01.490725+0000","last_active":"2024-04-25T08:57:03.493910+0000","last_peered":"2024-04-25T08:57:03.493910+0000","last_clean":"2024-04-25T08:57:03.493910+0000","last_became_active":"2024-04-25T08:57:01.490301+0000","last_became_peered":"2024-04-25T08:57:01.490301+0000","last_unstale":"2024-04-25T08:57:03.493910+0000","last_undegraded":"2024-04-25T08:57:03.493910+0000","last_fullsized":"2024-04-25T08:57:03.493910+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:25.368281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.516722+0000","last_change":"2024-04-25T08:57:02.514363+0000","last_active":"2024-04-25T08:57:02.516722+0000","last_peered":"2024-04-25T08:57:02.516722+0000","last_clean":"2024-04-25T08:57:02.516722+0000","last_became_active":"2024-04-25T08:57:02.514084+0000","last_became_peered":"2024-04-25T08:57:02.514084+0000","last_unstale":"2024-04-25T08:57:02.516722+0000","last_undegraded":"2024-04-25T08:57:02.516722+0000","last_fullsized":"2024-04-25T08:57:02.516722+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:29.014427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.508554+0000","last_change":"2024-04-25T08:57:01.503414+0000","last_active":"2024-04-25T08:57:02.508554+0000","last_peered":"2024-04-25T08:57:02.508554+0000","last_clean":"2024-04-25T08:57:02.508554+0000","last_became_active":"2024-04-25T08:57:01.502863+0000","last_became_peered":"2024-04-25T08:57:01.502863+0000","last_unstale":"2024-04-25T08:57:02.508554+0000","last_undegraded":"2024-04-25T08:57:02.508554+0000","last_fullsized":"2024-04-25T08:57:02.508554+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:27.864477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.506650+0000","last_change":"2024-04-25T08:57:02.500273+0000","last_active":"2024-04-25T08:57:02.506650+0000","last_peered":"2024-04-25T08:57:02.506650+0000","last_clean":"2024-04-25T08:57:02.506650+0000","last_became_active":"2024-04-25T08:57:02.499961+0000","last_became_peered":"2024-04-25T08:57:02.499961+0000","last_unstale":"2024-04-25T08:57:02.506650+0000","last_undegraded":"2024-04-25T08:57:02.506650+0000","last_fullsized":"2024-04-25T08:57:02.506650+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:08.136531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.494893+0000","last_change":"2024-04-25T08:57:01.497427+0000","last_active":"2024-04-25T08:57:03.494893+0000","last_peered":"2024-04-25T08:57:03.494893+0000","last_clean":"2024-04-25T08:57:03.494893+0000","last_became_active":"2024-04-25T08:57:01.497129+0000","last_became_peered":"2024-04-25T08:57:01.497129+0000","last_unstale":"2024-04-25T08:57:03.494893+0000","last_undegraded":"2024-04-25T08:57:03.494893+0000","last_fullsized":"2024-04-25T08:57:03.494893+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:46.623429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.494789+0000","last_change":"2024-04-25T08:57:02.503088+0000","last_active":"2024-04-25T08:57:03.494789+0000","last_peered":"2024-04-25T08:57:03.494789+0000","last_clean":"2024-04-25T08:57:03.494789+0000","last_became_active":"2024-04-25T08:57:02.502783+0000","last_became_peered":"2024-04-25T08:57:02.502783+0000","last_unstale":"2024-04-25T08:57:03.494789+0000","last_undegraded":"2024-04-25T08:57:03.494789+0000","last_fullsized":"2024-04-25T08:57:03.494789+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:19.399992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.492871+0000","last_change":"2024-04-25T08:57:01.498134+0000","last_active":"2024-04-25T08:57:03.492871+0000","last_peered":"2024-04-25T08:57:03.492871+0000","last_clean":"2024-04-25T08:57:03.492871+0000","last_became_active":"2024-04-25T08:57:01.497827+0000","last_became_peered":"2024-04-25T08:57:01.497827+0000","last_unstale":"2024-04-25T08:57:03.492871+0000","last_undegraded":"2024-04-25T08:57:03.492871+0000","last_fullsized":"2024-04-25T08:57:03.492871+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:52.867295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.510981+0000","last_change":"2024-04-25T08:57:02.508188+0000","last_active":"2024-04-25T08:57:02.510981+0000","last_peered":"2024-04-25T08:57:02.510981+0000","last_clean":"2024-04-25T08:57:02.510981+0000","last_became_active":"2024-04-25T08:57:02.507906+0000","last_became_peered":"2024-04-25T08:57:02.507906+0000","last_unstale":"2024-04-25T08:57:02.510981+0000","last_undegraded":"2024-04-25T08:57:02.510981+0000","last_fullsized":"2024-04-25T08:57:02.510981+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:55.732881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.494216+0000","last_change":"2024-04-25T08:57:01.497803+0000","last_active":"2024-04-25T08:57:03.494216+0000","last_peered":"2024-04-25T08:57:03.494216+0000","last_clean":"2024-04-25T08:57:03.494216+0000","last_became_active":"2024-04-25T08:57:01.497523+0000","last_became_peered":"2024-04-25T08:57:01.497523+0000","last_unstale":"2024-04-25T08:57:03.494216+0000","last_undegraded":"2024-04-25T08:57:03.494216+0000","last_fullsized":"2024-04-25T08:57:03.494216+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:19.596595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.521763+0000","last_change":"2024-04-25T08:57:02.517285+0000","last_active":"2024-04-25T08:57:02.521763+0000","last_peered":"2024-04-25T08:57:02.521763+0000","last_clean":"2024-04-25T08:57:02.521763+0000","last_became_active":"2024-04-25T08:57:02.516885+0000","last_became_peered":"2024-04-25T08:57:02.516885+0000","last_unstale":"2024-04-25T08:57:02.521763+0000","last_undegraded":"2024-04-25T08:57:02.521763+0000","last_fullsized":"2024-04-25T08:57:02.521763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:10.240677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.496415+0000","last_change":"2024-04-25T08:57:01.503755+0000","last_active":"2024-04-25T08:57:02.496415+0000","last_peered":"2024-04-25T08:57:02.496415+0000","last_clean":"2024-04-25T08:57:02.496415+0000","last_became_active":"2024-04-25T08:57:01.503467+0000","last_became_peered":"2024-04-25T08:57:01.503467+0000","last_unstale":"2024-04-25T08:57:02.496415+0000","last_undegraded":"2024-04-25T08:57:02.496415+0000","last_fullsized":"2024-04-25T08:57:02.496415+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:08:38.439002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.493870+0000","last_change":"2024-04-25T08:57:02.507328+0000","last_active":"2024-04-25T08:57:03.493870+0000","last_peered":"2024-04-25T08:57:03.493870+0000","last_clean":"2024-04-25T08:57:03.493870+0000","last_became_active":"2024-04-25T08:57:02.506606+0000","last_became_peered":"2024-04-25T08:57:02.506606+0000","last_unstale":"2024-04-25T08:57:03.493870+0000","last_undegraded":"2024-04-25T08:57:03.493870+0000","last_fullsized":"2024-04-25T08:57:03.493870+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:31.152667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.492755+0000","last_change":"2024-04-25T08:57:01.500770+0000","last_active":"2024-04-25T08:57:03.492755+0000","last_peered":"2024-04-25T08:57:03.492755+0000","last_clean":"2024-04-25T08:57:03.492755+0000","last_became_active":"2024-04-25T08:57:01.500211+0000","last_became_peered":"2024-04-25T08:57:01.500211+0000","last_unstale":"2024-04-25T08:57:03.492755+0000","last_undegraded":"2024-04-25T08:57:03.492755+0000","last_fullsized":"2024-04-25T08:57:03.492755+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:40.126869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.512963+0000","last_change":"2024-04-25T08:57:02.511188+0000","last_active":"2024-04-25T08:57:02.512963+0000","last_peered":"2024-04-25T08:57:02.512963+0000","last_clean":"2024-04-25T08:57:02.512963+0000","last_became_active":"2024-04-25T08:57:02.510852+0000","last_became_peered":"2024-04-25T08:57:02.510852+0000","last_unstale":"2024-04-25T08:57:02.512963+0000","last_undegraded":"2024-04-25T08:57:02.512963+0000","last_fullsized":"2024-04-25T08:57:02.512963+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:37.204952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.499431+0000","last_change":"2024-04-25T08:57:01.505153+0000","last_active":"2024-04-25T08:57:02.499431+0000","last_peered":"2024-04-25T08:57:02.499431+0000","last_clean":"2024-04-25T08:57:02.499431+0000","last_became_active":"2024-04-25T08:57:01.504404+0000","last_became_peered":"2024-04-25T08:57:01.504404+0000","last_unstale":"2024-04-25T08:57:02.499431+0000","last_undegraded":"2024-04-25T08:57:02.499431+0000","last_fullsized":"2024-04-25T08:57:02.499431+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:19.096651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.510154+0000","last_change":"2024-04-25T08:57:02.504432+0000","last_active":"2024-04-25T08:57:02.510154+0000","last_peered":"2024-04-25T08:57:02.510154+0000","last_clean":"2024-04-25T08:57:02.510154+0000","last_became_active":"2024-04-25T08:57:02.503938+0000","last_became_peered":"2024-04-25T08:57:02.503938+0000","last_unstale":"2024-04-25T08:57:02.510154+0000","last_undegraded":"2024-04-25T08:57:02.510154+0000","last_fullsized":"2024-04-25T08:57:02.510154+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:33.381650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.494526+0000","last_change":"2024-04-25T08:57:01.504290+0000","last_active":"2024-04-25T08:57:03.494526+0000","last_peered":"2024-04-25T08:57:03.494526+0000","last_clean":"2024-04-25T08:57:03.494526+0000","last_became_active":"2024-04-25T08:57:01.503977+0000","last_became_peered":"2024-04-25T08:57:01.503977+0000","last_unstale":"2024-04-25T08:57:03.494526+0000","last_undegraded":"2024-04-25T08:57:03.494526+0000","last_fullsized":"2024-04-25T08:57:03.494526+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:25.368281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":218,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.493964+0000","last_change":"2024-04-25T08:54:00.818044+0000","last_active":"2024-04-25T08:57:03.493964+0000","last_peered":"2024-04-25T08:57:03.493964+0000","last_clean":"2024-04-25T08:57:03.493964+0000","last_became_active":"2024-04-25T08:54:00.817098+0000","last_became_peered":"2024-04-25T08:54:00.817098+0000","last_unstale":"2024-04-25T08:57:03.493964+0000","last_undegraded":"2024-04-25T08:57:03.493964+0000","last_fullsized":"2024-04-25T08:57:03.493964+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_clean_scrub_stamp":"2024-04-25T08:53:59.739677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:48.129513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574887,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40608,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331232,"statfs":{"total":96636764160,"available":96595181568,"internally_reserved":0,"allocated":1126400,"data_stored":806477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40024,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331816,"statfs":{"total":96636764160,"available":96595779584,"internally_reserved":0,"allocated":536576,"data_stored":216691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574890,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40036,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331804,"statfs":{"total":96636764160,"available":96595767296,"internally_reserved":0,"allocated":544768,"data_stored":222389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574888,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45596,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326244,"statfs":{"total":96636764160,"available":96590073856,"internally_reserved":0,"allocated":540672,"data_stored":217631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53916,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317924,"statfs":{"total":96636764160,"available":96581554176,"internally_reserved":0,"allocated":540672,"data_stored":216657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40032,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331808,"statfs":{"total":96636764160,"available":96595771392,"internally_reserved":0,"allocated":544768,"data_stored":217653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574888,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49748,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94322092,"statfs":{"total":96636764160,"available":96585822208,"internally_reserved":0,"allocated":532480,"data_stored":216089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574889,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40608,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331232,"statfs":{"total":96636764160,"available":96595181568,"internally_reserved":0,"allocated":1126400,"data_stored":806421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:57:05.502 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:57:05.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.649+0000 7f91b07bd640 1 Processor -- start 2024-04-25T08:57:05.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.649+0000 7f91b07bd640 1 -- start start 2024-04-25T08:57:05.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.650+0000 7f91b07bd640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f91a8161390 0x7f91a80b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:05.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.650+0000 7f91b07bd640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f91a80d02a0 0x7f91a80d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:05.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.650+0000 7f91b07bd640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91a80b4c30 con 0x7f91a80ce5e0 2024-04-25T08:57:05.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.650+0000 7f91b07bd640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91a80d1910 con 0x7f91a80d02a0 2024-04-25T08:57:05.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.650+0000 7f91b07bd640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91a80cf2c0 con 0x7f91a8161390 2024-04-25T08:57:05.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.650+0000 7f91ae559640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f91a8161390 0x7f91a80b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:05.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.650+0000 7f91add58640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f91a80d02a0 0x7f91a80d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:05.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.650+0000 7f91add58640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f91a80d02a0 0x7f91a80d0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:52044/0 (socket says 172.21.15.43:52044) 2024-04-25T08:57:05.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.650+0000 7f91add58640 1 -- 172.21.15.43:0/282275288 learned_addr learned my addr 172.21.15.43:0/282275288 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:05.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.650+0000 7f91ae559640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f91a8161390 0x7f91a80b6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:58630/0 (socket says 172.21.15.43:58630) 2024-04-25T08:57:05.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.650+0000 7f91aed5a640 1 --1- 172.21.15.43:0/282275288 >> v1:172.21.15.193:6789/0 conn(0x7f91a80ce5e0 0x7f91a80cea70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:52158/0 (socket says 172.21.15.43:52158) 2024-04-25T08:57:05.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.651+0000 7f91ae559640 1 -- 172.21.15.43:0/282275288 >> v2:172.21.15.193:3300/0 conn(0x7f91a80d02a0 msgr2=0x7f91a80d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:05.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.651+0000 7f91ae559640 1 --2- 172.21.15.43:0/282275288 >> v2:172.21.15.193:3300/0 conn(0x7f91a80d02a0 0x7f91a80d0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:05.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.651+0000 7f91ae559640 1 -- 172.21.15.43:0/282275288 >> v1:172.21.15.193:6789/0 conn(0x7f91a80ce5e0 legacy=0x7f91a80cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:05.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.651+0000 7f91ad557640 1 -- 172.21.15.43:0/282275288 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 314075713 0 0) 0x7f91a0001490 con 0x7f91a80ce5e0 2024-04-25T08:57:05.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.651+0000 7f91ae559640 1 -- 172.21.15.43:0/282275288 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a80cf5a0 con 0x7f91a8161390 2024-04-25T08:57:05.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.651+0000 7f91add58640 1 --2- 172.21.15.43:0/282275288 >> v2:172.21.15.193:3300/0 conn(0x7f91a80d02a0 0x7f91a80d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:57:05.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.652+0000 7f91ae559640 1 --2- 172.21.15.43:0/282275288 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f91a8161390 0x7f91a80b6820 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f91a40097a0 tx=0x7f91a402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efedefed1c9169c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:05.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.652+0000 7f91ad557640 1 -- 172.21.15.43:0/282275288 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a4037070 con 0x7f91a8161390 2024-04-25T08:57:05.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.652+0000 7f91ad557640 1 -- 172.21.15.43:0/282275288 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91a4032440 con 0x7f91a8161390 2024-04-25T08:57:05.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.652+0000 7f91ad557640 1 -- 172.21.15.43:0/282275288 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a4032810 con 0x7f91a8161390 2024-04-25T08:57:05.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.652+0000 7f91b07bd640 1 -- 172.21.15.43:0/282275288 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f91a8161390 msgr2=0x7f91a80b6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:05.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.652+0000 7f91b07bd640 1 --2- 172.21.15.43:0/282275288 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f91a8161390 0x7f91a80b6820 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f91a40097a0 tx=0x7f91a402a180 comp rx=0 tx=0).stop 2024-04-25T08:57:05.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.652+0000 7f91b07bd640 1 -- 172.21.15.43:0/282275288 shutdown_connections 2024-04-25T08:57:05.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.652+0000 7f91b07bd640 1 --2- 172.21.15.43:0/282275288 >> v2:172.21.15.193:3300/0 conn(0x7f91a80d02a0 0x7f91a80d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:05.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.652+0000 7f91b07bd640 1 --2- 172.21.15.43:0/282275288 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f91a8161390 0x7f91a80b6820 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:05.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.652+0000 7f91b07bd640 1 -- 172.21.15.43:0/282275288 >> 172.21.15.43:0/282275288 conn(0x7f91a809cd20 msgr2=0x7f91a80d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:05.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.652+0000 7f91b07bd640 1 -- 172.21.15.43:0/282275288 shutdown_connections 2024-04-25T08:57:05.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.652+0000 7f91b07bd640 1 -- 172.21.15.43:0/282275288 wait complete. 2024-04-25T08:57:05.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.654+0000 7f91b07bd640 1 Processor -- start 2024-04-25T08:57:05.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.654+0000 7f91b07bd640 1 -- start start 2024-04-25T08:57:05.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.654+0000 7f91b07bd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91a80ce5e0 0x7f91a815d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:05.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.654+0000 7f91b07bd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91a80d02a0 0x7f91a81557d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:05.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.654+0000 7f91b07bd640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f91a8161390 0x7f91a8159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:05.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.654+0000 7f91b07bd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a8161770 con 0x7f91a80d02a0 2024-04-25T08:57:05.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.654+0000 7f91b07bd640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a80d15a0 con 0x7f91a80ce5e0 2024-04-25T08:57:05.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.654+0000 7f91b07bd640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91a80d1da0 con 0x7f91a8161390 2024-04-25T08:57:05.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.654+0000 7f91add58640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91a80d02a0 0x7f91a81557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:05.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.654+0000 7f91add58640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91a80d02a0 0x7f91a81557d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39668/0 (socket says 172.21.15.43:39668) 2024-04-25T08:57:05.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.654+0000 7f91add58640 1 -- 172.21.15.43:0/817939106 learned_addr learned my addr 172.21.15.43:0/817939106 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:05.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.655+0000 7f91aed5a640 1 --2- 172.21.15.43:0/817939106 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f91a8161390 0x7f91a8159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:05.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.655+0000 7f91ae559640 1 --2- 172.21.15.43:0/817939106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91a80ce5e0 0x7f91a815d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:05.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.655+0000 7f91add58640 1 -- 172.21.15.43:0/817939106 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f91a8161390 msgr2=0x7f91a8159810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:05.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.655+0000 7f91add58640 1 --2- 172.21.15.43:0/817939106 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f91a8161390 0x7f91a8159810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:05.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.655+0000 7f91add58640 1 -- 172.21.15.43:0/817939106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91a80ce5e0 msgr2=0x7f91a815d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:05.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.655+0000 7f91add58640 1 --2- 172.21.15.43:0/817939106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91a80ce5e0 0x7f91a815d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:05.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.655+0000 7f91add58640 1 -- 172.21.15.43:0/817939106 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a8157e40 con 0x7f91a80d02a0 2024-04-25T08:57:05.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.655+0000 7f91add58640 1 --2- 172.21.15.43:0/817939106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91a80d02a0 0x7f91a81557d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f919800e9d0 tx=0x7f919800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:05.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.655+0000 7f919effd640 1 -- 172.21.15.43:0/817939106 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9198017070 con 0x7f91a80d02a0 2024-04-25T08:57:05.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.655+0000 7f919effd640 1 -- 172.21.15.43:0/817939106 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f919800ca60 con 0x7f91a80d02a0 2024-04-25T08:57:05.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.655+0000 7f91b07bd640 1 -- 172.21.15.43:0/817939106 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91a8158120 con 0x7f91a80d02a0 2024-04-25T08:57:05.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.655+0000 7f919effd640 1 -- 172.21.15.43:0/817939106 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919800ce10 con 0x7f91a80d02a0 2024-04-25T08:57:05.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.655+0000 7f91b07bd640 1 -- 172.21.15.43:0/817939106 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91a81577a0 con 0x7f91a80d02a0 2024-04-25T08:57:05.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.656+0000 7f91b07bd640 1 -- 172.21.15.43:0/817939106 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91a80680f0 con 0x7f91a80d02a0 2024-04-25T08:57:05.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.657+0000 7f919effd640 1 -- 172.21.15.43:0/817939106 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f919801a070 con 0x7f91a80d02a0 2024-04-25T08:57:05.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.657+0000 7f919effd640 1 --2- 172.21.15.43:0/817939106 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f918c06b4d0 0x7f918c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:05.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.657+0000 7f919effd640 1 -- 172.21.15.43:0/817939106 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f919808a400 con 0x7f91a80d02a0 2024-04-25T08:57:05.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.657+0000 7f91ae559640 1 --2- 172.21.15.43:0/817939106 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f918c06b4d0 0x7f918c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:05.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.658+0000 7f91ae559640 1 --2- 172.21.15.43:0/817939106 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f918c06b4d0 0x7f918c06d990 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f91a4034000 tx=0x7f91a402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:05.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.661+0000 7f919effd640 1 -- 172.21.15.43:0/817939106 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9198058cc0 con 0x7f91a80d02a0 2024-04-25T08:57:05.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.839+0000 7f91b07bd640 1 -- 172.21.15.43:0/817939106 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f91a8074ef0 con 0x7f918c06b4d0 2024-04-25T08:57:05.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.845+0000 7f919effd640 1 -- 172.21.15.43:0/817939106 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251904 (secure 0 0 0) 0x7f91a8074ef0 con 0x7f918c06b4d0 2024-04-25T08:57:05.837 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:57:05.839 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-25T08:57:05.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.848+0000 7f91b07bd640 1 -- 172.21.15.43:0/817939106 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f918c06b4d0 msgr2=0x7f918c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:05.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.848+0000 7f91b07bd640 1 --2- 172.21.15.43:0/817939106 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f918c06b4d0 0x7f918c06d990 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f91a4034000 tx=0x7f91a402e000 comp rx=0 tx=0).stop 2024-04-25T08:57:05.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.848+0000 7f91b07bd640 1 -- 172.21.15.43:0/817939106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91a80d02a0 msgr2=0x7f91a81557d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:05.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.848+0000 7f91b07bd640 1 --2- 172.21.15.43:0/817939106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91a80d02a0 0x7f91a81557d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f919800e9d0 tx=0x7f919800ee90 comp rx=0 tx=0).stop 2024-04-25T08:57:05.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.848+0000 7f91b07bd640 1 -- 172.21.15.43:0/817939106 shutdown_connections 2024-04-25T08:57:05.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.848+0000 7f91b07bd640 1 --2- 172.21.15.43:0/817939106 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f91a8161390 0x7f91a8159810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:05.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.848+0000 7f91b07bd640 1 --2- 172.21.15.43:0/817939106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91a80d02a0 0x7f91a81557d0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:05.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.848+0000 7f91b07bd640 1 --2- 172.21.15.43:0/817939106 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f918c06b4d0 0x7f918c06d990 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:05.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.848+0000 7f91b07bd640 1 --2- 172.21.15.43:0/817939106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91a80ce5e0 0x7f91a815d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:05.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.848+0000 7f91b07bd640 1 -- 172.21.15.43:0/817939106 >> 172.21.15.43:0/817939106 conn(0x7f91a809cd20 msgr2=0x7f91a80b73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:05.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.848+0000 7f91b07bd640 1 -- 172.21.15.43:0/817939106 shutdown_connections 2024-04-25T08:57:05.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:05.848+0000 7f91b07bd640 1 -- 172.21.15.43:0/817939106 wait complete. 2024-04-25T08:57:05.849 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":133,"stamp":"2024-04-25T08:57:05.734525+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":85,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":350568,"kb_used_data":5364,"kb_used_omap":12,"kb_used_meta":345075,"kb_avail":754624152,"statfs":{"total":773094113280,"available":772735131648,"internally_reserved":0,"allocated":5492736,"data_stored":2920008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":353357361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.252235"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.494359+0000","last_change":"2024-04-25T08:57:02.503942+0000","last_active":"2024-04-25T08:57:03.494359+0000","last_peered":"2024-04-25T08:57:03.494359+0000","last_clean":"2024-04-25T08:57:03.494359+0000","last_became_active":"2024-04-25T08:57:02.503679+0000","last_became_peered":"2024-04-25T08:57:02.503679+0000","last_unstale":"2024-04-25T08:57:03.494359+0000","last_undegraded":"2024-04-25T08:57:03.494359+0000","last_fullsized":"2024-04-25T08:57:03.494359+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:48.776242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.494119+0000","last_change":"2024-04-25T08:57:02.504936+0000","last_active":"2024-04-25T08:57:03.494119+0000","last_peered":"2024-04-25T08:57:03.494119+0000","last_clean":"2024-04-25T08:57:03.494119+0000","last_became_active":"2024-04-25T08:57:02.504644+0000","last_became_peered":"2024-04-25T08:57:02.504644+0000","last_unstale":"2024-04-25T08:57:03.494119+0000","last_undegraded":"2024-04-25T08:57:03.494119+0000","last_fullsized":"2024-04-25T08:57:03.494119+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:55.732881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.500611+0000","last_change":"2024-04-25T08:57:02.497224+0000","last_active":"2024-04-25T08:57:02.500611+0000","last_peered":"2024-04-25T08:57:02.500611+0000","last_clean":"2024-04-25T08:57:02.500611+0000","last_became_active":"2024-04-25T08:57:02.496963+0000","last_became_peered":"2024-04-25T08:57:02.496963+0000","last_unstale":"2024-04-25T08:57:02.500611+0000","last_undegraded":"2024-04-25T08:57:02.500611+0000","last_fullsized":"2024-04-25T08:57:02.500611+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:17.384450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.515346+0000","last_change":"2024-04-25T08:57:02.514066+0000","last_active":"2024-04-25T08:57:02.515346+0000","last_peered":"2024-04-25T08:57:02.515346+0000","last_clean":"2024-04-25T08:57:02.515346+0000","last_became_active":"2024-04-25T08:57:02.513768+0000","last_became_peered":"2024-04-25T08:57:02.513768+0000","last_unstale":"2024-04-25T08:57:02.515346+0000","last_undegraded":"2024-04-25T08:57:02.515346+0000","last_fullsized":"2024-04-25T08:57:02.515346+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:41.134579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.494010+0000","last_change":"2024-04-25T08:57:02.514299+0000","last_active":"2024-04-25T08:57:03.494010+0000","last_peered":"2024-04-25T08:57:03.494010+0000","last_clean":"2024-04-25T08:57:03.494010+0000","last_became_active":"2024-04-25T08:57:02.513896+0000","last_became_peered":"2024-04-25T08:57:02.513896+0000","last_unstale":"2024-04-25T08:57:03.494010+0000","last_undegraded":"2024-04-25T08:57:03.494010+0000","last_fullsized":"2024-04-25T08:57:03.494010+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:49.505425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.512000+0000","last_change":"2024-04-25T08:57:02.510740+0000","last_active":"2024-04-25T08:57:02.512000+0000","last_peered":"2024-04-25T08:57:02.512000+0000","last_clean":"2024-04-25T08:57:02.512000+0000","last_became_active":"2024-04-25T08:57:02.510470+0000","last_became_peered":"2024-04-25T08:57:02.510470+0000","last_unstale":"2024-04-25T08:57:02.512000+0000","last_undegraded":"2024-04-25T08:57:02.512000+0000","last_fullsized":"2024-04-25T08:57:02.512000+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:47.497446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.494728+0000","last_change":"2024-04-25T08:57:02.507150+0000","last_active":"2024-04-25T08:57:03.494728+0000","last_peered":"2024-04-25T08:57:03.494728+0000","last_clean":"2024-04-25T08:57:03.494728+0000","last_became_active":"2024-04-25T08:57:02.506886+0000","last_became_peered":"2024-04-25T08:57:02.506886+0000","last_unstale":"2024-04-25T08:57:03.494728+0000","last_undegraded":"2024-04-25T08:57:03.494728+0000","last_fullsized":"2024-04-25T08:57:03.494728+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:16.311585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.520333+0000","last_change":"2024-04-25T08:57:02.516441+0000","last_active":"2024-04-25T08:57:02.520333+0000","last_peered":"2024-04-25T08:57:02.520333+0000","last_clean":"2024-04-25T08:57:02.520333+0000","last_became_active":"2024-04-25T08:57:02.516142+0000","last_became_peered":"2024-04-25T08:57:02.516142+0000","last_unstale":"2024-04-25T08:57:02.520333+0000","last_undegraded":"2024-04-25T08:57:02.520333+0000","last_fullsized":"2024-04-25T08:57:02.520333+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:53.875005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.511048+0000","last_change":"2024-04-25T08:57:02.509459+0000","last_active":"2024-04-25T08:57:02.511048+0000","last_peered":"2024-04-25T08:57:02.511048+0000","last_clean":"2024-04-25T08:57:02.511048+0000","last_became_active":"2024-04-25T08:57:02.509189+0000","last_became_peered":"2024-04-25T08:57:02.509189+0000","last_unstale":"2024-04-25T08:57:02.511048+0000","last_undegraded":"2024-04-25T08:57:02.511048+0000","last_fullsized":"2024-04-25T08:57:02.511048+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:33.686876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.514999+0000","last_change":"2024-04-25T08:57:02.512888+0000","last_active":"2024-04-25T08:57:02.514999+0000","last_peered":"2024-04-25T08:57:02.514999+0000","last_clean":"2024-04-25T08:57:02.514999+0000","last_became_active":"2024-04-25T08:57:02.512591+0000","last_became_peered":"2024-04-25T08:57:02.512591+0000","last_unstale":"2024-04-25T08:57:02.514999+0000","last_undegraded":"2024-04-25T08:57:02.514999+0000","last_fullsized":"2024-04-25T08:57:02.514999+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:37.204952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.514175+0000","last_change":"2024-04-25T08:57:02.510186+0000","last_active":"2024-04-25T08:57:02.514175+0000","last_peered":"2024-04-25T08:57:02.514175+0000","last_clean":"2024-04-25T08:57:02.514175+0000","last_became_active":"2024-04-25T08:57:02.509863+0000","last_became_peered":"2024-04-25T08:57:02.509863+0000","last_unstale":"2024-04-25T08:57:02.514175+0000","last_undegraded":"2024-04-25T08:57:02.514175+0000","last_fullsized":"2024-04-25T08:57:02.514175+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:17.384450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.494333+0000","last_change":"2024-04-25T08:57:02.499383+0000","last_active":"2024-04-25T08:57:03.494333+0000","last_peered":"2024-04-25T08:57:03.494333+0000","last_clean":"2024-04-25T08:57:03.494333+0000","last_became_active":"2024-04-25T08:57:02.499105+0000","last_became_peered":"2024-04-25T08:57:02.499105+0000","last_unstale":"2024-04-25T08:57:03.494333+0000","last_undegraded":"2024-04-25T08:57:03.494333+0000","last_fullsized":"2024-04-25T08:57:03.494333+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:48.776242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.516236+0000","last_change":"2024-04-25T08:57:02.513293+0000","last_active":"2024-04-25T08:57:02.516236+0000","last_peered":"2024-04-25T08:57:02.516236+0000","last_clean":"2024-04-25T08:57:02.516236+0000","last_became_active":"2024-04-25T08:57:02.512997+0000","last_became_peered":"2024-04-25T08:57:02.512997+0000","last_unstale":"2024-04-25T08:57:02.516236+0000","last_undegraded":"2024-04-25T08:57:02.516236+0000","last_fullsized":"2024-04-25T08:57:02.516236+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:37.589340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.510906+0000","last_change":"2024-04-25T08:57:02.507964+0000","last_active":"2024-04-25T08:57:02.510906+0000","last_peered":"2024-04-25T08:57:02.510906+0000","last_clean":"2024-04-25T08:57:02.510906+0000","last_became_active":"2024-04-25T08:57:02.507666+0000","last_became_peered":"2024-04-25T08:57:02.507666+0000","last_unstale":"2024-04-25T08:57:02.510906+0000","last_undegraded":"2024-04-25T08:57:02.510906+0000","last_fullsized":"2024-04-25T08:57:02.510906+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:08:39.446712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.518197+0000","last_change":"2024-04-25T08:57:02.514617+0000","last_active":"2024-04-25T08:57:02.518197+0000","last_peered":"2024-04-25T08:57:02.518197+0000","last_clean":"2024-04-25T08:57:02.518197+0000","last_became_active":"2024-04-25T08:57:02.513909+0000","last_became_peered":"2024-04-25T08:57:02.513909+0000","last_unstale":"2024-04-25T08:57:02.518197+0000","last_undegraded":"2024-04-25T08:57:02.518197+0000","last_fullsized":"2024-04-25T08:57:02.518197+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:32.273069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.516658+0000","last_change":"2024-04-25T08:57:02.513269+0000","last_active":"2024-04-25T08:57:02.516658+0000","last_peered":"2024-04-25T08:57:02.516658+0000","last_clean":"2024-04-25T08:57:02.516658+0000","last_became_active":"2024-04-25T08:57:02.512693+0000","last_became_peered":"2024-04-25T08:57:02.512693+0000","last_unstale":"2024-04-25T08:57:02.516658+0000","last_undegraded":"2024-04-25T08:57:02.516658+0000","last_fullsized":"2024-04-25T08:57:02.516658+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:08.136531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.504766+0000","last_change":"2024-04-25T08:57:02.502915+0000","last_active":"2024-04-25T08:57:02.504766+0000","last_peered":"2024-04-25T08:57:02.504766+0000","last_clean":"2024-04-25T08:57:02.504766+0000","last_became_active":"2024-04-25T08:57:02.499469+0000","last_became_peered":"2024-04-25T08:57:02.499469+0000","last_unstale":"2024-04-25T08:57:02.504766+0000","last_undegraded":"2024-04-25T08:57:02.504766+0000","last_fullsized":"2024-04-25T08:57:02.504766+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:11.452008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.517534+0000","last_change":"2024-04-25T08:57:02.515325+0000","last_active":"2024-04-25T08:57:02.517534+0000","last_peered":"2024-04-25T08:57:02.517534+0000","last_clean":"2024-04-25T08:57:02.517534+0000","last_became_active":"2024-04-25T08:57:02.515044+0000","last_became_peered":"2024-04-25T08:57:02.515044+0000","last_unstale":"2024-04-25T08:57:02.517534+0000","last_undegraded":"2024-04-25T08:57:02.517534+0000","last_fullsized":"2024-04-25T08:57:02.517534+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:41.134579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.516602+0000","last_change":"2024-04-25T08:57:02.513045+0000","last_active":"2024-04-25T08:57:02.516602+0000","last_peered":"2024-04-25T08:57:02.516602+0000","last_clean":"2024-04-25T08:57:02.516602+0000","last_became_active":"2024-04-25T08:57:02.512688+0000","last_became_peered":"2024-04-25T08:57:02.512688+0000","last_unstale":"2024-04-25T08:57:02.516602+0000","last_undegraded":"2024-04-25T08:57:02.516602+0000","last_fullsized":"2024-04-25T08:57:02.516602+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:20.104362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.502567+0000","last_change":"2024-04-25T08:57:02.500014+0000","last_active":"2024-04-25T08:57:02.502567+0000","last_peered":"2024-04-25T08:57:02.502567+0000","last_clean":"2024-04-25T08:57:02.502567+0000","last_became_active":"2024-04-25T08:57:02.499666+0000","last_became_peered":"2024-04-25T08:57:02.499666+0000","last_unstale":"2024-04-25T08:57:02.502567+0000","last_undegraded":"2024-04-25T08:57:02.502567+0000","last_fullsized":"2024-04-25T08:57:02.502567+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:06:00.871028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.521737+0000","last_change":"2024-04-25T08:57:02.515882+0000","last_active":"2024-04-25T08:57:02.521737+0000","last_peered":"2024-04-25T08:57:02.521737+0000","last_clean":"2024-04-25T08:57:02.521737+0000","last_became_active":"2024-04-25T08:57:02.515600+0000","last_became_peered":"2024-04-25T08:57:02.515600+0000","last_unstale":"2024-04-25T08:57:02.521737+0000","last_undegraded":"2024-04-25T08:57:02.521737+0000","last_fullsized":"2024-04-25T08:57:02.521737+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:31:16.311585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.514280+0000","last_change":"2024-04-25T08:57:02.512418+0000","last_active":"2024-04-25T08:57:02.514280+0000","last_peered":"2024-04-25T08:57:02.514280+0000","last_clean":"2024-04-25T08:57:02.514280+0000","last_became_active":"2024-04-25T08:57:02.512155+0000","last_became_peered":"2024-04-25T08:57:02.512155+0000","last_unstale":"2024-04-25T08:57:02.514280+0000","last_undegraded":"2024-04-25T08:57:02.514280+0000","last_fullsized":"2024-04-25T08:57:02.514280+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:04.066281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.518509+0000","last_change":"2024-04-25T08:57:02.516698+0000","last_active":"2024-04-25T08:57:02.518509+0000","last_peered":"2024-04-25T08:57:02.518509+0000","last_clean":"2024-04-25T08:57:02.518509+0000","last_became_active":"2024-04-25T08:57:02.515485+0000","last_became_peered":"2024-04-25T08:57:02.515485+0000","last_unstale":"2024-04-25T08:57:02.518509+0000","last_undegraded":"2024-04-25T08:57:02.518509+0000","last_fullsized":"2024-04-25T08:57:02.518509+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:08:39.446712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.516641+0000","last_change":"2024-04-25T08:57:02.511150+0000","last_active":"2024-04-25T08:57:02.516641+0000","last_peered":"2024-04-25T08:57:02.516641+0000","last_clean":"2024-04-25T08:57:02.516641+0000","last_became_active":"2024-04-25T08:57:02.510885+0000","last_became_peered":"2024-04-25T08:57:02.510885+0000","last_unstale":"2024-04-25T08:57:02.516641+0000","last_undegraded":"2024-04-25T08:57:02.516641+0000","last_fullsized":"2024-04-25T08:57:02.516641+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:06:00.871028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.516684+0000","last_change":"2024-04-25T08:57:02.511183+0000","last_active":"2024-04-25T08:57:02.516684+0000","last_peered":"2024-04-25T08:57:02.516684+0000","last_clean":"2024-04-25T08:57:02.516684+0000","last_became_active":"2024-04-25T08:57:02.510589+0000","last_became_peered":"2024-04-25T08:57:02.510589+0000","last_unstale":"2024-04-25T08:57:02.516684+0000","last_undegraded":"2024-04-25T08:57:02.516684+0000","last_fullsized":"2024-04-25T08:57:02.516684+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:33.981688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.517394+0000","last_change":"2024-04-25T08:57:02.515890+0000","last_active":"2024-04-25T08:57:02.517394+0000","last_peered":"2024-04-25T08:57:02.517394+0000","last_clean":"2024-04-25T08:57:02.517394+0000","last_became_active":"2024-04-25T08:57:02.515618+0000","last_became_peered":"2024-04-25T08:57:02.515618+0000","last_unstale":"2024-04-25T08:57:02.517394+0000","last_undegraded":"2024-04-25T08:57:02.517394+0000","last_fullsized":"2024-04-25T08:57:02.517394+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:54.522484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.504607+0000","last_change":"2024-04-25T08:57:02.500245+0000","last_active":"2024-04-25T08:57:02.504607+0000","last_peered":"2024-04-25T08:57:02.504607+0000","last_clean":"2024-04-25T08:57:02.504607+0000","last_became_active":"2024-04-25T08:57:02.499953+0000","last_became_peered":"2024-04-25T08:57:02.499953+0000","last_unstale":"2024-04-25T08:57:02.504607+0000","last_undegraded":"2024-04-25T08:57:02.504607+0000","last_fullsized":"2024-04-25T08:57:02.504607+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:19.291277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.494137+0000","last_change":"2024-04-25T08:57:02.499102+0000","last_active":"2024-04-25T08:57:03.494137+0000","last_peered":"2024-04-25T08:57:03.494137+0000","last_clean":"2024-04-25T08:57:03.494137+0000","last_became_active":"2024-04-25T08:57:02.498735+0000","last_became_peered":"2024-04-25T08:57:02.498735+0000","last_unstale":"2024-04-25T08:57:03.494137+0000","last_undegraded":"2024-04-25T08:57:03.494137+0000","last_fullsized":"2024-04-25T08:57:03.494137+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:28.872188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.510295+0000","last_change":"2024-04-25T08:57:02.508944+0000","last_active":"2024-04-25T08:57:02.510295+0000","last_peered":"2024-04-25T08:57:02.510295+0000","last_clean":"2024-04-25T08:57:02.510295+0000","last_became_active":"2024-04-25T08:57:02.508661+0000","last_became_peered":"2024-04-25T08:57:02.508661+0000","last_unstale":"2024-04-25T08:57:02.510295+0000","last_undegraded":"2024-04-25T08:57:02.510295+0000","last_fullsized":"2024-04-25T08:57:02.510295+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:53.164910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.501469+0000","last_change":"2024-04-25T08:57:02.500327+0000","last_active":"2024-04-25T08:57:02.501469+0000","last_peered":"2024-04-25T08:57:02.501469+0000","last_clean":"2024-04-25T08:57:02.501469+0000","last_became_active":"2024-04-25T08:57:02.500082+0000","last_became_peered":"2024-04-25T08:57:02.500082+0000","last_unstale":"2024-04-25T08:57:02.501469+0000","last_undegraded":"2024-04-25T08:57:02.501469+0000","last_fullsized":"2024-04-25T08:57:02.501469+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:04.066281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.511897+0000","last_change":"2024-04-25T08:57:02.508803+0000","last_active":"2024-04-25T08:57:02.511897+0000","last_peered":"2024-04-25T08:57:02.511897+0000","last_clean":"2024-04-25T08:57:02.511897+0000","last_became_active":"2024-04-25T08:57:02.508530+0000","last_became_peered":"2024-04-25T08:57:02.508530+0000","last_unstale":"2024-04-25T08:57:02.511897+0000","last_undegraded":"2024-04-25T08:57:02.511897+0000","last_fullsized":"2024-04-25T08:57:02.511897+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:48.776242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.502630+0000","last_change":"2024-04-25T08:57:02.500243+0000","last_active":"2024-04-25T08:57:02.502630+0000","last_peered":"2024-04-25T08:57:02.502630+0000","last_clean":"2024-04-25T08:57:02.502630+0000","last_became_active":"2024-04-25T08:57:02.498907+0000","last_became_peered":"2024-04-25T08:57:02.498907+0000","last_unstale":"2024-04-25T08:57:02.502630+0000","last_undegraded":"2024-04-25T08:57:02.502630+0000","last_fullsized":"2024-04-25T08:57:02.502630+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:33.981688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.493024+0000","last_change":"2024-04-25T08:57:02.515096+0000","last_active":"2024-04-25T08:57:03.493024+0000","last_peered":"2024-04-25T08:57:03.493024+0000","last_clean":"2024-04-25T08:57:03.493024+0000","last_became_active":"2024-04-25T08:57:02.514800+0000","last_became_peered":"2024-04-25T08:57:02.514800+0000","last_unstale":"2024-04-25T08:57:03.493024+0000","last_undegraded":"2024-04-25T08:57:03.493024+0000","last_fullsized":"2024-04-25T08:57:03.493024+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:24.167958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"45'4","reported_seq":16,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.523291+0000","last_change":"2024-04-25T08:57:01.502910+0000","last_active":"2024-04-25T08:57:02.523291+0000","last_peered":"2024-04-25T08:57:02.523291+0000","last_clean":"2024-04-25T08:57:02.523291+0000","last_became_active":"2024-04-25T08:57:01.502588+0000","last_became_peered":"2024-04-25T08:57:01.502588+0000","last_unstale":"2024-04-25T08:57:02.523291+0000","last_undegraded":"2024-04-25T08:57:02.523291+0000","last_fullsized":"2024-04-25T08:57:02.523291+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:48.497715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.516702+0000","last_change":"2024-04-25T08:57:02.511370+0000","last_active":"2024-04-25T08:57:02.516702+0000","last_peered":"2024-04-25T08:57:02.516702+0000","last_clean":"2024-04-25T08:57:02.516702+0000","last_became_active":"2024-04-25T08:57:02.510876+0000","last_became_peered":"2024-04-25T08:57:02.510876+0000","last_unstale":"2024-04-25T08:57:02.516702+0000","last_undegraded":"2024-04-25T08:57:02.516702+0000","last_fullsized":"2024-04-25T08:57:02.516702+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:59.198644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.518690+0000","last_change":"2024-04-25T08:57:01.500492+0000","last_active":"2024-04-25T08:57:02.518690+0000","last_peered":"2024-04-25T08:57:02.518690+0000","last_clean":"2024-04-25T08:57:02.518690+0000","last_became_active":"2024-04-25T08:57:01.500205+0000","last_became_peered":"2024-04-25T08:57:01.500205+0000","last_unstale":"2024-04-25T08:57:02.518690+0000","last_undegraded":"2024-04-25T08:57:02.518690+0000","last_fullsized":"2024-04-25T08:57:02.518690+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:28.006717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.494281+0000","last_change":"2024-04-25T08:57:02.514084+0000","last_active":"2024-04-25T08:57:03.494281+0000","last_peered":"2024-04-25T08:57:03.494281+0000","last_clean":"2024-04-25T08:57:03.494281+0000","last_became_active":"2024-04-25T08:57:02.513586+0000","last_became_peered":"2024-04-25T08:57:02.513586+0000","last_unstale":"2024-04-25T08:57:03.494281+0000","last_undegraded":"2024-04-25T08:57:03.494281+0000","last_fullsized":"2024-04-25T08:57:03.494281+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:25.615662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.493049+0000","last_change":"2024-04-25T08:57:01.502381+0000","last_active":"2024-04-25T08:57:03.493049+0000","last_peered":"2024-04-25T08:57:03.493049+0000","last_clean":"2024-04-25T08:57:03.493049+0000","last_became_active":"2024-04-25T08:57:01.501850+0000","last_became_peered":"2024-04-25T08:57:01.501850+0000","last_unstale":"2024-04-25T08:57:03.493049+0000","last_undegraded":"2024-04-25T08:57:03.493049+0000","last_fullsized":"2024-04-25T08:57:03.493049+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:04.894170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.493069+0000","last_change":"2024-04-25T08:57:02.513565+0000","last_active":"2024-04-25T08:57:03.493069+0000","last_peered":"2024-04-25T08:57:03.493069+0000","last_clean":"2024-04-25T08:57:03.493069+0000","last_became_active":"2024-04-25T08:57:02.512682+0000","last_became_peered":"2024-04-25T08:57:02.512682+0000","last_unstale":"2024-04-25T08:57:03.493069+0000","last_undegraded":"2024-04-25T08:57:03.493069+0000","last_fullsized":"2024-04-25T08:57:03.493069+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:31.544490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.494427+0000","last_change":"2024-04-25T08:57:01.491474+0000","last_active":"2024-04-25T08:57:03.494427+0000","last_peered":"2024-04-25T08:57:03.494427+0000","last_clean":"2024-04-25T08:57:03.494427+0000","last_became_active":"2024-04-25T08:57:01.491227+0000","last_became_peered":"2024-04-25T08:57:01.491227+0000","last_unstale":"2024-04-25T08:57:03.494427+0000","last_undegraded":"2024-04-25T08:57:03.494427+0000","last_fullsized":"2024-04-25T08:57:03.494427+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:10.361510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.517300+0000","last_change":"2024-04-25T08:57:02.516299+0000","last_active":"2024-04-25T08:57:02.517300+0000","last_peered":"2024-04-25T08:57:02.517300+0000","last_clean":"2024-04-25T08:57:02.517300+0000","last_became_active":"2024-04-25T08:57:02.516046+0000","last_became_peered":"2024-04-25T08:57:02.516046+0000","last_unstale":"2024-04-25T08:57:02.517300+0000","last_undegraded":"2024-04-25T08:57:02.517300+0000","last_fullsized":"2024-04-25T08:57:02.517300+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:53.875005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.523080+0000","last_change":"2024-04-25T08:57:01.506122+0000","last_active":"2024-04-25T08:57:02.523080+0000","last_peered":"2024-04-25T08:57:02.523080+0000","last_clean":"2024-04-25T08:57:02.523080+0000","last_became_active":"2024-04-25T08:57:01.505760+0000","last_became_peered":"2024-04-25T08:57:01.505760+0000","last_unstale":"2024-04-25T08:57:02.523080+0000","last_undegraded":"2024-04-25T08:57:02.523080+0000","last_fullsized":"2024-04-25T08:57:02.523080+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:49.105790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.514228+0000","last_change":"2024-04-25T08:57:02.511604+0000","last_active":"2024-04-25T08:57:02.514228+0000","last_peered":"2024-04-25T08:57:02.514228+0000","last_clean":"2024-04-25T08:57:02.514228+0000","last_became_active":"2024-04-25T08:57:02.510394+0000","last_became_peered":"2024-04-25T08:57:02.510394+0000","last_unstale":"2024-04-25T08:57:02.514228+0000","last_undegraded":"2024-04-25T08:57:02.514228+0000","last_fullsized":"2024-04-25T08:57:02.514228+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:19.291277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.516621+0000","last_change":"2024-04-25T08:57:01.496439+0000","last_active":"2024-04-25T08:57:02.516621+0000","last_peered":"2024-04-25T08:57:02.516621+0000","last_clean":"2024-04-25T08:57:02.516621+0000","last_became_active":"2024-04-25T08:57:01.496084+0000","last_became_peered":"2024-04-25T08:57:01.496084+0000","last_unstale":"2024-04-25T08:57:02.516621+0000","last_undegraded":"2024-04-25T08:57:02.516621+0000","last_fullsized":"2024-04-25T08:57:02.516621+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:32.679165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.520357+0000","last_change":"2024-04-25T08:57:02.515231+0000","last_active":"2024-04-25T08:57:02.520357+0000","last_peered":"2024-04-25T08:57:02.520357+0000","last_clean":"2024-04-25T08:57:02.520357+0000","last_became_active":"2024-04-25T08:57:02.514945+0000","last_became_peered":"2024-04-25T08:57:02.514945+0000","last_unstale":"2024-04-25T08:57:02.520357+0000","last_undegraded":"2024-04-25T08:57:02.520357+0000","last_fullsized":"2024-04-25T08:57:02.520357+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:30.086945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.494317+0000","last_change":"2024-04-25T08:57:01.496375+0000","last_active":"2024-04-25T08:57:03.494317+0000","last_peered":"2024-04-25T08:57:03.494317+0000","last_clean":"2024-04-25T08:57:03.494317+0000","last_became_active":"2024-04-25T08:57:01.496089+0000","last_became_peered":"2024-04-25T08:57:01.496089+0000","last_unstale":"2024-04-25T08:57:03.494317+0000","last_undegraded":"2024-04-25T08:57:03.494317+0000","last_fullsized":"2024-04-25T08:57:03.494317+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:29.079235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.518066+0000","last_change":"2024-04-25T08:57:02.516253+0000","last_active":"2024-04-25T08:57:02.518066+0000","last_peered":"2024-04-25T08:57:02.518066+0000","last_clean":"2024-04-25T08:57:02.518066+0000","last_became_active":"2024-04-25T08:57:02.515659+0000","last_became_peered":"2024-04-25T08:57:02.515659+0000","last_unstale":"2024-04-25T08:57:02.518066+0000","last_undegraded":"2024-04-25T08:57:02.518066+0000","last_fullsized":"2024-04-25T08:57:02.518066+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:33.381650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.499079+0000","last_change":"2024-04-25T08:57:01.500516+0000","last_active":"2024-04-25T08:57:02.499079+0000","last_peered":"2024-04-25T08:57:02.499079+0000","last_clean":"2024-04-25T08:57:02.499079+0000","last_became_active":"2024-04-25T08:57:01.500223+0000","last_became_peered":"2024-04-25T08:57:01.500223+0000","last_unstale":"2024-04-25T08:57:02.499079+0000","last_undegraded":"2024-04-25T08:57:02.499079+0000","last_fullsized":"2024-04-25T08:57:02.499079+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:36.581630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.510620+0000","last_change":"2024-04-25T08:57:02.509822+0000","last_active":"2024-04-25T08:57:02.510620+0000","last_peered":"2024-04-25T08:57:02.510620+0000","last_clean":"2024-04-25T08:57:02.510620+0000","last_became_active":"2024-04-25T08:57:02.509554+0000","last_became_peered":"2024-04-25T08:57:02.509554+0000","last_unstale":"2024-04-25T08:57:02.510620+0000","last_undegraded":"2024-04-25T08:57:02.510620+0000","last_fullsized":"2024-04-25T08:57:02.510620+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:55.333244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.495566+0000","last_change":"2024-04-25T08:57:01.493058+0000","last_active":"2024-04-25T08:57:02.495566+0000","last_peered":"2024-04-25T08:57:02.495566+0000","last_clean":"2024-04-25T08:57:02.495566+0000","last_became_active":"2024-04-25T08:57:01.492785+0000","last_became_peered":"2024-04-25T08:57:01.492785+0000","last_unstale":"2024-04-25T08:57:02.495566+0000","last_undegraded":"2024-04-25T08:57:02.495566+0000","last_fullsized":"2024-04-25T08:57:02.495566+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:32.373940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.494532+0000","last_change":"2024-04-25T08:57:02.500485+0000","last_active":"2024-04-25T08:57:03.494532+0000","last_peered":"2024-04-25T08:57:03.494532+0000","last_clean":"2024-04-25T08:57:03.494532+0000","last_became_active":"2024-04-25T08:57:02.500201+0000","last_became_peered":"2024-04-25T08:57:02.500201+0000","last_unstale":"2024-04-25T08:57:03.494532+0000","last_undegraded":"2024-04-25T08:57:03.494532+0000","last_fullsized":"2024-04-25T08:57:03.494532+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:47.497446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.503757+0000","last_change":"2024-04-25T08:57:01.497442+0000","last_active":"2024-04-25T08:57:02.503757+0000","last_peered":"2024-04-25T08:57:02.503757+0000","last_clean":"2024-04-25T08:57:02.503757+0000","last_became_active":"2024-04-25T08:57:01.497157+0000","last_became_peered":"2024-04-25T08:57:01.497157+0000","last_unstale":"2024-04-25T08:57:02.503757+0000","last_undegraded":"2024-04-25T08:57:02.503757+0000","last_fullsized":"2024-04-25T08:57:02.503757+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:28.006717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.522875+0000","last_change":"2024-04-25T08:57:02.517863+0000","last_active":"2024-04-25T08:57:02.522875+0000","last_peered":"2024-04-25T08:57:02.522875+0000","last_clean":"2024-04-25T08:57:02.522875+0000","last_became_active":"2024-04-25T08:57:02.517471+0000","last_became_peered":"2024-04-25T08:57:02.517471+0000","last_unstale":"2024-04-25T08:57:02.522875+0000","last_undegraded":"2024-04-25T08:57:02.522875+0000","last_fullsized":"2024-04-25T08:57:02.522875+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:23.745722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"45'3","reported_seq":17,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.494871+0000","last_change":"2024-04-25T08:57:01.499229+0000","last_active":"2024-04-25T08:57:03.494871+0000","last_peered":"2024-04-25T08:57:03.494871+0000","last_clean":"2024-04-25T08:57:03.494871+0000","last_became_active":"2024-04-25T08:57:01.498950+0000","last_became_peered":"2024-04-25T08:57:01.498950+0000","last_unstale":"2024-04-25T08:57:03.494871+0000","last_undegraded":"2024-04-25T08:57:03.494871+0000","last_fullsized":"2024-04-25T08:57:03.494871+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:30.536780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.494724+0000","last_change":"2024-04-25T08:57:02.507927+0000","last_active":"2024-04-25T08:57:03.494724+0000","last_peered":"2024-04-25T08:57:03.494724+0000","last_clean":"2024-04-25T08:57:03.494724+0000","last_became_active":"2024-04-25T08:57:02.507607+0000","last_became_peered":"2024-04-25T08:57:02.507607+0000","last_unstale":"2024-04-25T08:57:03.494724+0000","last_undegraded":"2024-04-25T08:57:03.494724+0000","last_fullsized":"2024-04-25T08:57:03.494724+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:32.273069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"45'6","reported_seq":18,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.524755+0000","last_change":"2024-04-25T08:57:01.495892+0000","last_active":"2024-04-25T08:57:02.524755+0000","last_peered":"2024-04-25T08:57:02.524755+0000","last_clean":"2024-04-25T08:57:02.524755+0000","last_became_active":"2024-04-25T08:57:01.495572+0000","last_became_peered":"2024-04-25T08:57:01.495572+0000","last_unstale":"2024-04-25T08:57:02.524755+0000","last_undegraded":"2024-04-25T08:57:02.524755+0000","last_fullsized":"2024-04-25T08:57:02.524755+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:52.157200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.516610+0000","last_change":"2024-04-25T08:57:02.512174+0000","last_active":"2024-04-25T08:57:02.516610+0000","last_peered":"2024-04-25T08:57:02.516610+0000","last_clean":"2024-04-25T08:57:02.516610+0000","last_became_active":"2024-04-25T08:57:02.511898+0000","last_became_peered":"2024-04-25T08:57:02.511898+0000","last_unstale":"2024-04-25T08:57:02.516610+0000","last_undegraded":"2024-04-25T08:57:02.516610+0000","last_fullsized":"2024-04-25T08:57:02.516610+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:37.589340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.524047+0000","last_change":"2024-04-25T08:57:01.504685+0000","last_active":"2024-04-25T08:57:02.524047+0000","last_peered":"2024-04-25T08:57:02.524047+0000","last_clean":"2024-04-25T08:57:02.524047+0000","last_became_active":"2024-04-25T08:57:01.504421+0000","last_became_peered":"2024-04-25T08:57:01.504421+0000","last_unstale":"2024-04-25T08:57:02.524047+0000","last_undegraded":"2024-04-25T08:57:02.524047+0000","last_fullsized":"2024-04-25T08:57:02.524047+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:22.222587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.514958+0000","last_change":"2024-04-25T08:57:02.512553+0000","last_active":"2024-04-25T08:57:02.514958+0000","last_peered":"2024-04-25T08:57:02.514958+0000","last_clean":"2024-04-25T08:57:02.514958+0000","last_became_active":"2024-04-25T08:57:02.510690+0000","last_became_peered":"2024-04-25T08:57:02.510690+0000","last_unstale":"2024-04-25T08:57:02.514958+0000","last_undegraded":"2024-04-25T08:57:02.514958+0000","last_fullsized":"2024-04-25T08:57:02.514958+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:10:11.452008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.494018+0000","last_change":"2024-04-25T08:57:01.490352+0000","last_active":"2024-04-25T08:57:03.494018+0000","last_peered":"2024-04-25T08:57:03.494018+0000","last_clean":"2024-04-25T08:57:03.494018+0000","last_became_active":"2024-04-25T08:57:01.490082+0000","last_became_peered":"2024-04-25T08:57:01.490082+0000","last_unstale":"2024-04-25T08:57:03.494018+0000","last_undegraded":"2024-04-25T08:57:03.494018+0000","last_fullsized":"2024-04-25T08:57:03.494018+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:23.160248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.521772+0000","last_change":"2024-04-25T08:57:02.517148+0000","last_active":"2024-04-25T08:57:02.521772+0000","last_peered":"2024-04-25T08:57:02.521772+0000","last_clean":"2024-04-25T08:57:02.521772+0000","last_became_active":"2024-04-25T08:57:02.516844+0000","last_became_peered":"2024-04-25T08:57:02.516844+0000","last_unstale":"2024-04-25T08:57:02.521772+0000","last_undegraded":"2024-04-25T08:57:02.521772+0000","last_fullsized":"2024-04-25T08:57:02.521772+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:19.399992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.502719+0000","last_change":"2024-04-25T08:57:01.504311+0000","last_active":"2024-04-25T08:57:02.502719+0000","last_peered":"2024-04-25T08:57:02.502719+0000","last_clean":"2024-04-25T08:57:02.502719+0000","last_became_active":"2024-04-25T08:57:01.503993+0000","last_became_peered":"2024-04-25T08:57:01.503993+0000","last_unstale":"2024-04-25T08:57:02.502719+0000","last_undegraded":"2024-04-25T08:57:02.502719+0000","last_fullsized":"2024-04-25T08:57:02.502719+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:46.623429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.494798+0000","last_change":"2024-04-25T08:57:02.514973+0000","last_active":"2024-04-25T08:57:03.494798+0000","last_peered":"2024-04-25T08:57:03.494798+0000","last_clean":"2024-04-25T08:57:03.494798+0000","last_became_active":"2024-04-25T08:57:02.514512+0000","last_became_peered":"2024-04-25T08:57:02.514512+0000","last_unstale":"2024-04-25T08:57:03.494798+0000","last_undegraded":"2024-04-25T08:57:03.494798+0000","last_fullsized":"2024-04-25T08:57:03.494798+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:10.240677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"45'8","reported_seq":20,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.524857+0000","last_change":"2024-04-25T08:57:01.493800+0000","last_active":"2024-04-25T08:57:02.524857+0000","last_peered":"2024-04-25T08:57:02.524857+0000","last_clean":"2024-04-25T08:57:02.524857+0000","last_became_active":"2024-04-25T08:57:01.493491+0000","last_became_peered":"2024-04-25T08:57:01.493491+0000","last_unstale":"2024-04-25T08:57:02.524857+0000","last_undegraded":"2024-04-25T08:57:02.524857+0000","last_fullsized":"2024-04-25T08:57:02.524857+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:53.514774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.501569+0000","last_change":"2024-04-25T08:57:02.498331+0000","last_active":"2024-04-25T08:57:02.501569+0000","last_peered":"2024-04-25T08:57:02.501569+0000","last_clean":"2024-04-25T08:57:02.501569+0000","last_became_active":"2024-04-25T08:57:02.497873+0000","last_became_peered":"2024-04-25T08:57:02.497873+0000","last_unstale":"2024-04-25T08:57:02.501569+0000","last_undegraded":"2024-04-25T08:57:02.501569+0000","last_fullsized":"2024-04-25T08:57:02.501569+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:45.000612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"45'1","reported_seq":13,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.523880+0000","last_change":"2024-04-25T08:57:01.503767+0000","last_active":"2024-04-25T08:57:02.523880+0000","last_peered":"2024-04-25T08:57:02.523880+0000","last_clean":"2024-04-25T08:57:02.523880+0000","last_became_active":"2024-04-25T08:57:01.503446+0000","last_became_peered":"2024-04-25T08:57:01.503446+0000","last_unstale":"2024-04-25T08:57:02.523880+0000","last_undegraded":"2024-04-25T08:57:02.523880+0000","last_fullsized":"2024-04-25T08:57:02.523880+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:00:26.416915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.514880+0000","last_change":"2024-04-25T08:57:02.512706+0000","last_active":"2024-04-25T08:57:02.514880+0000","last_peered":"2024-04-25T08:57:02.514880+0000","last_clean":"2024-04-25T08:57:02.514880+0000","last_became_active":"2024-04-25T08:57:02.511272+0000","last_became_peered":"2024-04-25T08:57:02.511272+0000","last_unstale":"2024-04-25T08:57:02.514880+0000","last_undegraded":"2024-04-25T08:57:02.514880+0000","last_fullsized":"2024-04-25T08:57:02.514880+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:55.333244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.521747+0000","last_change":"2024-04-25T08:57:01.498697+0000","last_active":"2024-04-25T08:57:02.521747+0000","last_peered":"2024-04-25T08:57:02.521747+0000","last_clean":"2024-04-25T08:57:02.521747+0000","last_became_active":"2024-04-25T08:57:01.497943+0000","last_became_peered":"2024-04-25T08:57:01.497943+0000","last_unstale":"2024-04-25T08:57:02.521747+0000","last_undegraded":"2024-04-25T08:57:02.521747+0000","last_fullsized":"2024-04-25T08:57:02.521747+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:07.905530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.516576+0000","last_change":"2024-04-25T08:57:02.512544+0000","last_active":"2024-04-25T08:57:02.516576+0000","last_peered":"2024-04-25T08:57:02.516576+0000","last_clean":"2024-04-25T08:57:02.516576+0000","last_became_active":"2024-04-25T08:57:02.512246+0000","last_became_peered":"2024-04-25T08:57:02.512246+0000","last_unstale":"2024-04-25T08:57:02.516576+0000","last_undegraded":"2024-04-25T08:57:02.516576+0000","last_fullsized":"2024-04-25T08:57:02.516576+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:54.522484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"45'1","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.494392+0000","last_change":"2024-04-25T08:57:01.502283+0000","last_active":"2024-04-25T08:57:03.494392+0000","last_peered":"2024-04-25T08:57:03.494392+0000","last_clean":"2024-04-25T08:57:03.494392+0000","last_became_active":"2024-04-25T08:57:01.501995+0000","last_became_peered":"2024-04-25T08:57:01.501995+0000","last_unstale":"2024-04-25T08:57:03.494392+0000","last_undegraded":"2024-04-25T08:57:03.494392+0000","last_fullsized":"2024-04-25T08:57:03.494392+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:04.894170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.515512+0000","last_change":"2024-04-25T08:57:02.509526+0000","last_active":"2024-04-25T08:57:02.515512+0000","last_peered":"2024-04-25T08:57:02.515512+0000","last_clean":"2024-04-25T08:57:02.515512+0000","last_became_active":"2024-04-25T08:57:02.509264+0000","last_became_peered":"2024-04-25T08:57:02.509264+0000","last_unstale":"2024-04-25T08:57:02.515512+0000","last_undegraded":"2024-04-25T08:57:02.515512+0000","last_fullsized":"2024-04-25T08:57:02.515512+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:45.000612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.500931+0000","last_change":"2024-04-25T08:57:01.502094+0000","last_active":"2024-04-25T08:57:02.500931+0000","last_peered":"2024-04-25T08:57:02.500931+0000","last_clean":"2024-04-25T08:57:02.500931+0000","last_became_active":"2024-04-25T08:57:01.501522+0000","last_became_peered":"2024-04-25T08:57:01.501522+0000","last_unstale":"2024-04-25T08:57:02.500931+0000","last_undegraded":"2024-04-25T08:57:02.500931+0000","last_fullsized":"2024-04-25T08:57:02.500931+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:30.536780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.511982+0000","last_change":"2024-04-25T08:57:02.510129+0000","last_active":"2024-04-25T08:57:02.511982+0000","last_peered":"2024-04-25T08:57:02.511982+0000","last_clean":"2024-04-25T08:57:02.511982+0000","last_became_active":"2024-04-25T08:57:02.508715+0000","last_became_peered":"2024-04-25T08:57:02.508715+0000","last_unstale":"2024-04-25T08:57:02.511982+0000","last_undegraded":"2024-04-25T08:57:02.511982+0000","last_fullsized":"2024-04-25T08:57:02.511982+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:25.615662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"45'2","reported_seq":14,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.523169+0000","last_change":"2024-04-25T08:57:01.505370+0000","last_active":"2024-04-25T08:57:02.523169+0000","last_peered":"2024-04-25T08:57:02.523169+0000","last_clean":"2024-04-25T08:57:02.523169+0000","last_became_active":"2024-04-25T08:57:01.505091+0000","last_became_peered":"2024-04-25T08:57:01.505091+0000","last_unstale":"2024-04-25T08:57:02.523169+0000","last_undegraded":"2024-04-25T08:57:02.523169+0000","last_fullsized":"2024-04-25T08:57:02.523169+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:30.144957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.492898+0000","last_change":"2024-04-25T08:57:02.499125+0000","last_active":"2024-04-25T08:57:03.492898+0000","last_peered":"2024-04-25T08:57:03.492898+0000","last_clean":"2024-04-25T08:57:03.492898+0000","last_became_active":"2024-04-25T08:57:02.498822+0000","last_became_peered":"2024-04-25T08:57:02.498822+0000","last_unstale":"2024-04-25T08:57:03.492898+0000","last_undegraded":"2024-04-25T08:57:03.492898+0000","last_fullsized":"2024-04-25T08:57:03.492898+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:20.604306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.494311+0000","last_change":"2024-04-25T08:57:01.497294+0000","last_active":"2024-04-25T08:57:03.494311+0000","last_peered":"2024-04-25T08:57:03.494311+0000","last_clean":"2024-04-25T08:57:03.494311+0000","last_became_active":"2024-04-25T08:57:01.496801+0000","last_became_peered":"2024-04-25T08:57:01.496801+0000","last_unstale":"2024-04-25T08:57:03.494311+0000","last_undegraded":"2024-04-25T08:57:03.494311+0000","last_fullsized":"2024-04-25T08:57:03.494311+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:10.361510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.502651+0000","last_change":"2024-04-25T08:57:02.500965+0000","last_active":"2024-04-25T08:57:02.502651+0000","last_peered":"2024-04-25T08:57:02.502651+0000","last_clean":"2024-04-25T08:57:02.502651+0000","last_became_active":"2024-04-25T08:57:02.500637+0000","last_became_peered":"2024-04-25T08:57:02.500637+0000","last_unstale":"2024-04-25T08:57:02.502651+0000","last_undegraded":"2024-04-25T08:57:02.502651+0000","last_fullsized":"2024-04-25T08:57:02.502651+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:59.198644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.500740+0000","last_change":"2024-04-25T08:57:01.496800+0000","last_active":"2024-04-25T08:57:02.500740+0000","last_peered":"2024-04-25T08:57:02.500740+0000","last_clean":"2024-04-25T08:57:02.500740+0000","last_became_active":"2024-04-25T08:57:01.496508+0000","last_became_peered":"2024-04-25T08:57:01.496508+0000","last_unstale":"2024-04-25T08:57:02.500740+0000","last_undegraded":"2024-04-25T08:57:02.500740+0000","last_fullsized":"2024-04-25T08:57:02.500740+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:58.190934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.510936+0000","last_change":"2024-04-25T08:57:02.508703+0000","last_active":"2024-04-25T08:57:02.510936+0000","last_peered":"2024-04-25T08:57:02.510936+0000","last_clean":"2024-04-25T08:57:02.510936+0000","last_became_active":"2024-04-25T08:57:02.508442+0000","last_became_peered":"2024-04-25T08:57:02.508442+0000","last_unstale":"2024-04-25T08:57:02.510936+0000","last_undegraded":"2024-04-25T08:57:02.510936+0000","last_fullsized":"2024-04-25T08:57:02.510936+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:14:08.913240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.493910+0000","last_change":"2024-04-25T08:57:01.490725+0000","last_active":"2024-04-25T08:57:03.493910+0000","last_peered":"2024-04-25T08:57:03.493910+0000","last_clean":"2024-04-25T08:57:03.493910+0000","last_became_active":"2024-04-25T08:57:01.490301+0000","last_became_peered":"2024-04-25T08:57:01.490301+0000","last_unstale":"2024-04-25T08:57:03.493910+0000","last_undegraded":"2024-04-25T08:57:03.493910+0000","last_fullsized":"2024-04-25T08:57:03.493910+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:25.368281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.516722+0000","last_change":"2024-04-25T08:57:02.514363+0000","last_active":"2024-04-25T08:57:02.516722+0000","last_peered":"2024-04-25T08:57:02.516722+0000","last_clean":"2024-04-25T08:57:02.516722+0000","last_became_active":"2024-04-25T08:57:02.514084+0000","last_became_peered":"2024-04-25T08:57:02.514084+0000","last_unstale":"2024-04-25T08:57:02.516722+0000","last_undegraded":"2024-04-25T08:57:02.516722+0000","last_fullsized":"2024-04-25T08:57:02.516722+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:29.014427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.508554+0000","last_change":"2024-04-25T08:57:01.503414+0000","last_active":"2024-04-25T08:57:02.508554+0000","last_peered":"2024-04-25T08:57:02.508554+0000","last_clean":"2024-04-25T08:57:02.508554+0000","last_became_active":"2024-04-25T08:57:01.502863+0000","last_became_peered":"2024-04-25T08:57:01.502863+0000","last_unstale":"2024-04-25T08:57:02.508554+0000","last_undegraded":"2024-04-25T08:57:02.508554+0000","last_fullsized":"2024-04-25T08:57:02.508554+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:05:27.864477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.506650+0000","last_change":"2024-04-25T08:57:02.500273+0000","last_active":"2024-04-25T08:57:02.506650+0000","last_peered":"2024-04-25T08:57:02.506650+0000","last_clean":"2024-04-25T08:57:02.506650+0000","last_became_active":"2024-04-25T08:57:02.499961+0000","last_became_peered":"2024-04-25T08:57:02.499961+0000","last_unstale":"2024-04-25T08:57:02.506650+0000","last_undegraded":"2024-04-25T08:57:02.506650+0000","last_fullsized":"2024-04-25T08:57:02.506650+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:08.136531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.494893+0000","last_change":"2024-04-25T08:57:01.497427+0000","last_active":"2024-04-25T08:57:03.494893+0000","last_peered":"2024-04-25T08:57:03.494893+0000","last_clean":"2024-04-25T08:57:03.494893+0000","last_became_active":"2024-04-25T08:57:01.497129+0000","last_became_peered":"2024-04-25T08:57:01.497129+0000","last_unstale":"2024-04-25T08:57:03.494893+0000","last_undegraded":"2024-04-25T08:57:03.494893+0000","last_fullsized":"2024-04-25T08:57:03.494893+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:46.623429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.494789+0000","last_change":"2024-04-25T08:57:02.503088+0000","last_active":"2024-04-25T08:57:03.494789+0000","last_peered":"2024-04-25T08:57:03.494789+0000","last_clean":"2024-04-25T08:57:03.494789+0000","last_became_active":"2024-04-25T08:57:02.502783+0000","last_became_peered":"2024-04-25T08:57:02.502783+0000","last_unstale":"2024-04-25T08:57:03.494789+0000","last_undegraded":"2024-04-25T08:57:03.494789+0000","last_fullsized":"2024-04-25T08:57:03.494789+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:19.399992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.492871+0000","last_change":"2024-04-25T08:57:01.498134+0000","last_active":"2024-04-25T08:57:03.492871+0000","last_peered":"2024-04-25T08:57:03.492871+0000","last_clean":"2024-04-25T08:57:03.492871+0000","last_became_active":"2024-04-25T08:57:01.497827+0000","last_became_peered":"2024-04-25T08:57:01.497827+0000","last_unstale":"2024-04-25T08:57:03.492871+0000","last_undegraded":"2024-04-25T08:57:03.492871+0000","last_fullsized":"2024-04-25T08:57:03.492871+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:52.867295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.510981+0000","last_change":"2024-04-25T08:57:02.508188+0000","last_active":"2024-04-25T08:57:02.510981+0000","last_peered":"2024-04-25T08:57:02.510981+0000","last_clean":"2024-04-25T08:57:02.510981+0000","last_became_active":"2024-04-25T08:57:02.507906+0000","last_became_peered":"2024-04-25T08:57:02.507906+0000","last_unstale":"2024-04-25T08:57:02.510981+0000","last_undegraded":"2024-04-25T08:57:02.510981+0000","last_fullsized":"2024-04-25T08:57:02.510981+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:55.732881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.494216+0000","last_change":"2024-04-25T08:57:01.497803+0000","last_active":"2024-04-25T08:57:03.494216+0000","last_peered":"2024-04-25T08:57:03.494216+0000","last_clean":"2024-04-25T08:57:03.494216+0000","last_became_active":"2024-04-25T08:57:01.497523+0000","last_became_peered":"2024-04-25T08:57:01.497523+0000","last_unstale":"2024-04-25T08:57:03.494216+0000","last_undegraded":"2024-04-25T08:57:03.494216+0000","last_fullsized":"2024-04-25T08:57:03.494216+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:19.596595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.521763+0000","last_change":"2024-04-25T08:57:02.517285+0000","last_active":"2024-04-25T08:57:02.521763+0000","last_peered":"2024-04-25T08:57:02.521763+0000","last_clean":"2024-04-25T08:57:02.521763+0000","last_became_active":"2024-04-25T08:57:02.516885+0000","last_became_peered":"2024-04-25T08:57:02.516885+0000","last_unstale":"2024-04-25T08:57:02.521763+0000","last_undegraded":"2024-04-25T08:57:02.521763+0000","last_fullsized":"2024-04-25T08:57:02.521763+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:10.240677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.496415+0000","last_change":"2024-04-25T08:57:01.503755+0000","last_active":"2024-04-25T08:57:02.496415+0000","last_peered":"2024-04-25T08:57:02.496415+0000","last_clean":"2024-04-25T08:57:02.496415+0000","last_became_active":"2024-04-25T08:57:01.503467+0000","last_became_peered":"2024-04-25T08:57:01.503467+0000","last_unstale":"2024-04-25T08:57:02.496415+0000","last_undegraded":"2024-04-25T08:57:02.496415+0000","last_fullsized":"2024-04-25T08:57:02.496415+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:08:38.439002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":12,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.493870+0000","last_change":"2024-04-25T08:57:02.507328+0000","last_active":"2024-04-25T08:57:03.493870+0000","last_peered":"2024-04-25T08:57:03.493870+0000","last_clean":"2024-04-25T08:57:03.493870+0000","last_became_active":"2024-04-25T08:57:02.506606+0000","last_became_peered":"2024-04-25T08:57:02.506606+0000","last_unstale":"2024-04-25T08:57:03.493870+0000","last_undegraded":"2024-04-25T08:57:03.493870+0000","last_fullsized":"2024-04-25T08:57:03.493870+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:31.152667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":14,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.492755+0000","last_change":"2024-04-25T08:57:01.500770+0000","last_active":"2024-04-25T08:57:03.492755+0000","last_peered":"2024-04-25T08:57:03.492755+0000","last_clean":"2024-04-25T08:57:03.492755+0000","last_became_active":"2024-04-25T08:57:01.500211+0000","last_became_peered":"2024-04-25T08:57:01.500211+0000","last_unstale":"2024-04-25T08:57:03.492755+0000","last_undegraded":"2024-04-25T08:57:03.492755+0000","last_fullsized":"2024-04-25T08:57:03.492755+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:40.126869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.512963+0000","last_change":"2024-04-25T08:57:02.511188+0000","last_active":"2024-04-25T08:57:02.512963+0000","last_peered":"2024-04-25T08:57:02.512963+0000","last_clean":"2024-04-25T08:57:02.512963+0000","last_became_active":"2024-04-25T08:57:02.510852+0000","last_became_peered":"2024-04-25T08:57:02.510852+0000","last_unstale":"2024-04-25T08:57:02.512963+0000","last_undegraded":"2024-04-25T08:57:02.512963+0000","last_fullsized":"2024-04-25T08:57:02.512963+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:37.204952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.499431+0000","last_change":"2024-04-25T08:57:01.505153+0000","last_active":"2024-04-25T08:57:02.499431+0000","last_peered":"2024-04-25T08:57:02.499431+0000","last_clean":"2024-04-25T08:57:02.499431+0000","last_became_active":"2024-04-25T08:57:01.504404+0000","last_became_peered":"2024-04-25T08:57:01.504404+0000","last_unstale":"2024-04-25T08:57:02.499431+0000","last_undegraded":"2024-04-25T08:57:02.499431+0000","last_fullsized":"2024-04-25T08:57:02.499431+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:47:19.096651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-25T08:57:02.510154+0000","last_change":"2024-04-25T08:57:02.504432+0000","last_active":"2024-04-25T08:57:02.510154+0000","last_peered":"2024-04-25T08:57:02.510154+0000","last_clean":"2024-04-25T08:57:02.510154+0000","last_became_active":"2024-04-25T08:57:02.503938+0000","last_became_peered":"2024-04-25T08:57:02.503938+0000","last_unstale":"2024-04-25T08:57:02.510154+0000","last_undegraded":"2024-04-25T08:57:02.510154+0000","last_fullsized":"2024-04-25T08:57:02.510154+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:01.479104+0000","last_clean_scrub_stamp":"2024-04-25T08:57:01.479104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:33.381650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"45'2","reported_seq":16,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.494526+0000","last_change":"2024-04-25T08:57:01.504290+0000","last_active":"2024-04-25T08:57:03.494526+0000","last_peered":"2024-04-25T08:57:03.494526+0000","last_clean":"2024-04-25T08:57:03.494526+0000","last_became_active":"2024-04-25T08:57:01.503977+0000","last_became_peered":"2024-04-25T08:57:01.503977+0000","last_unstale":"2024-04-25T08:57:03.494526+0000","last_undegraded":"2024-04-25T08:57:03.494526+0000","last_fullsized":"2024-04-25T08:57:03.494526+0000","mapping_epoch":43,"log_start":"0'0","ondisk_log_start":"0'0","created":43,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:00.471393+0000","last_clean_scrub_stamp":"2024-04-25T08:57:00.471393+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:49:25.368281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":218,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-25T08:57:03.493964+0000","last_change":"2024-04-25T08:54:00.818044+0000","last_active":"2024-04-25T08:57:03.493964+0000","last_peered":"2024-04-25T08:57:03.493964+0000","last_clean":"2024-04-25T08:57:03.493964+0000","last_became_active":"2024-04-25T08:54:00.817098+0000","last_became_peered":"2024-04-25T08:54:00.817098+0000","last_unstale":"2024-04-25T08:57:03.493964+0000","last_undegraded":"2024-04-25T08:57:03.493964+0000","last_fullsized":"2024-04-25T08:57:03.493964+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_clean_scrub_stamp":"2024-04-25T08:53:59.739677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:48.129513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574887,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40608,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331232,"statfs":{"total":96636764160,"available":96595181568,"internally_reserved":0,"allocated":1126400,"data_stored":806477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40024,"kb_used_data":524,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331816,"statfs":{"total":96636764160,"available":96595779584,"internally_reserved":0,"allocated":536576,"data_stored":216691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574890,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40036,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331804,"statfs":{"total":96636764160,"available":96595767296,"internally_reserved":0,"allocated":544768,"data_stored":222389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574888,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45596,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326244,"statfs":{"total":96636764160,"available":96590073856,"internally_reserved":0,"allocated":540672,"data_stored":217631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574888,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53916,"kb_used_data":528,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317924,"statfs":{"total":96636764160,"available":96581554176,"internally_reserved":0,"allocated":540672,"data_stored":216657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574888,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40032,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331808,"statfs":{"total":96636764160,"available":96595771392,"internally_reserved":0,"allocated":544768,"data_stored":217653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574888,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49748,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94322092,"statfs":{"total":96636764160,"available":96585822208,"internally_reserved":0,"allocated":532480,"data_stored":216089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574889,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40608,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331232,"statfs":{"total":96636764160,"available":96595181568,"internally_reserved":0,"allocated":1126400,"data_stored":806421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:57:05.853 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T08:57:05.853 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T08:57:05.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.007+0000 7fd2495e7640 1 Processor -- start 2024-04-25T08:57:05.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.007+0000 7fd2495e7640 1 -- start start 2024-04-25T08:57:05.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.008+0000 7fd2495e7640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd24415de10 0x7fd2440cf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:05.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.008+0000 7fd2495e7640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd2440cf640 0x7fd2440d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:05.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.008+0000 7fd2495e7640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fd2440d2500 0x7fd244161950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:05.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.008+0000 7fd2495e7640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2440d2960 con 0x7fd2440cf640 2024-04-25T08:57:05.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.008+0000 7fd2495e7640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2440b7490 con 0x7fd2440d2500 2024-04-25T08:57:05.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.008+0000 7fd2495e7640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2440b68c0 con 0x7fd24415de10 2024-04-25T08:57:05.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.008+0000 7fd2427fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd2440cf640 0x7fd2440d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:06.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.008+0000 7fd2427fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd2440cf640 0x7fd2440d1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39676/0 (socket says 172.21.15.43:39676) 2024-04-25T08:57:06.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.008+0000 7fd2427fc640 1 -- 172.21.15.43:0/167565851 learned_addr learned my addr 172.21.15.43:0/167565851 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:06.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.008+0000 7fd242ffd640 1 --2- 172.21.15.43:0/167565851 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd24415de10 0x7fd2440cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:06.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.008+0000 7fd2437fe640 1 --2- 172.21.15.43:0/167565851 >> v2:172.21.15.193:3300/0 conn(0x7fd2440d2500 0x7fd244161950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:06.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.009+0000 7fd2427fc640 1 -- 172.21.15.43:0/167565851 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd24415de10 msgr2=0x7fd2440cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:06.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.009+0000 7fd2427fc640 1 --2- 172.21.15.43:0/167565851 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd24415de10 0x7fd2440cf100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:06.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.009+0000 7fd2427fc640 1 -- 172.21.15.43:0/167565851 >> v2:172.21.15.193:3300/0 conn(0x7fd2440d2500 msgr2=0x7fd244161950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:06.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.009+0000 7fd2427fc640 1 --2- 172.21.15.43:0/167565851 >> v2:172.21.15.193:3300/0 conn(0x7fd2440d2500 0x7fd244161950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:06.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.009+0000 7fd2427fc640 1 -- 172.21.15.43:0/167565851 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2440b6b60 con 0x7fd2440cf640 2024-04-25T08:57:06.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.009+0000 7fd2437fe640 1 --2- 172.21.15.43:0/167565851 >> v2:172.21.15.193:3300/0 conn(0x7fd2440d2500 0x7fd244161950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:06.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.009+0000 7fd2427fc640 1 --2- 172.21.15.43:0/167565851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2440cf640 0x7fd2440d1fc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd2440cfce0 tx=0x7fd22c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=448f169609ec9648 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:06.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.009+0000 7fd241ffb640 1 -- 172.21.15.43:0/167565851 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22c037070 con 0x7fd2440cf640 2024-04-25T08:57:06.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.010+0000 7fd241ffb640 1 -- 172.21.15.43:0/167565851 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd22c032440 con 0x7fd2440cf640 2024-04-25T08:57:06.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.010+0000 7fd241ffb640 1 -- 172.21.15.43:0/167565851 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22c032810 con 0x7fd2440cf640 2024-04-25T08:57:06.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.010+0000 7fd2495e7640 1 -- 172.21.15.43:0/167565851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2440cf640 msgr2=0x7fd2440d1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:06.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.010+0000 7fd2495e7640 1 --2- 172.21.15.43:0/167565851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2440cf640 0x7fd2440d1fc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fd2440cfce0 tx=0x7fd22c02a1e0 comp rx=0 tx=0).stop 2024-04-25T08:57:06.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.010+0000 7fd2495e7640 1 -- 172.21.15.43:0/167565851 shutdown_connections 2024-04-25T08:57:06.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.010+0000 7fd2495e7640 1 --2- 172.21.15.43:0/167565851 >> v2:172.21.15.193:3300/0 conn(0x7fd2440d2500 0x7fd244161950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:06.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.010+0000 7fd2495e7640 1 --2- 172.21.15.43:0/167565851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2440cf640 0x7fd2440d1fc0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:06.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.010+0000 7fd2495e7640 1 --2- 172.21.15.43:0/167565851 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd24415de10 0x7fd2440cf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:06.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.010+0000 7fd2495e7640 1 -- 172.21.15.43:0/167565851 >> 172.21.15.43:0/167565851 conn(0x7fd2440d3a70 msgr2=0x7fd2440d3e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:06.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.010+0000 7fd2495e7640 1 -- 172.21.15.43:0/167565851 shutdown_connections 2024-04-25T08:57:06.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.010+0000 7fd2495e7640 1 -- 172.21.15.43:0/167565851 wait complete. 2024-04-25T08:57:06.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.011+0000 7fd2495e7640 1 Processor -- start 2024-04-25T08:57:06.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.011+0000 7fd2495e7640 1 -- start start 2024-04-25T08:57:06.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.012+0000 7fd2495e7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2440cf640 0x7fd24414efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:06.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.012+0000 7fd2495e7640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd2440d2500 0x7fd24409e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:06.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.012+0000 7fd2495e7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd24415de10 0x7fd24415fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:06.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.012+0000 7fd2495e7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24415e1f0 con 0x7fd2440cf640 2024-04-25T08:57:06.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.012+0000 7fd2495e7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2440b7120 con 0x7fd24415de10 2024-04-25T08:57:06.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.012+0000 7fd2495e7640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2440d19e0 con 0x7fd2440d2500 2024-04-25T08:57:06.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.012+0000 7fd242ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2440cf640 0x7fd24414efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:06.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.012+0000 7fd242ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2440cf640 0x7fd24414efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39692/0 (socket says 172.21.15.43:39692) 2024-04-25T08:57:06.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.012+0000 7fd242ffd640 1 -- 172.21.15.43:0/692626377 learned_addr learned my addr 172.21.15.43:0/692626377 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:06.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.013+0000 7fd2437fe640 1 --2- 172.21.15.43:0/692626377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd24415de10 0x7fd24415fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:06.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.013+0000 7fd2427fc640 1 --2- 172.21.15.43:0/692626377 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd2440d2500 0x7fd24409e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:06.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.013+0000 7fd242ffd640 1 -- 172.21.15.43:0/692626377 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd2440d2500 msgr2=0x7fd24409e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:06.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.013+0000 7fd242ffd640 1 --2- 172.21.15.43:0/692626377 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd2440d2500 0x7fd24409e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:06.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.013+0000 7fd242ffd640 1 -- 172.21.15.43:0/692626377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd24415de10 msgr2=0x7fd24415fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:06.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.013+0000 7fd242ffd640 1 --2- 172.21.15.43:0/692626377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd24415de10 0x7fd24415fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:06.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.013+0000 7fd242ffd640 1 -- 172.21.15.43:0/692626377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd244158ca0 con 0x7fd2440cf640 2024-04-25T08:57:06.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.013+0000 7fd242ffd640 1 --2- 172.21.15.43:0/692626377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2440cf640 0x7fd24414efa0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd23c00e8a0 tx=0x7fd23c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:06.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.013+0000 7fd23b7fe640 1 -- 172.21.15.43:0/692626377 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c017070 con 0x7fd2440cf640 2024-04-25T08:57:06.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.013+0000 7fd23b7fe640 1 -- 172.21.15.43:0/692626377 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd23c00c940 con 0x7fd2440cf640 2024-04-25T08:57:06.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.013+0000 7fd2495e7640 1 -- 172.21.15.43:0/692626377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd244158f80 con 0x7fd2440cf640 2024-04-25T08:57:06.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.013+0000 7fd23b7fe640 1 -- 172.21.15.43:0/692626377 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c00cd30 con 0x7fd2440cf640 2024-04-25T08:57:06.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.013+0000 7fd2495e7640 1 -- 172.21.15.43:0/692626377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd244159540 con 0x7fd2440cf640 2024-04-25T08:57:06.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.014+0000 7fd2495e7640 1 -- 172.21.15.43:0/692626377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2440680f0 con 0x7fd2440cf640 2024-04-25T08:57:06.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.015+0000 7fd23b7fe640 1 -- 172.21.15.43:0/692626377 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd23c01a070 con 0x7fd2440cf640 2024-04-25T08:57:06.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.015+0000 7fd23b7fe640 1 --2- 172.21.15.43:0/692626377 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd22406b4d0 0x7fd22406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:06.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.015+0000 7fd23b7fe640 1 -- 172.21.15.43:0/692626377 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd23c08a430 con 0x7fd2440cf640 2024-04-25T08:57:06.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.015+0000 7fd2427fc640 1 --2- 172.21.15.43:0/692626377 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd22406b4d0 0x7fd22406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:06.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.016+0000 7fd2427fc640 1 --2- 172.21.15.43:0/692626377 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd22406b4d0 0x7fd22406d990 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd22c034000 tx=0x7fd22c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:06.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.019+0000 7fd23b7fe640 1 -- 172.21.15.43:0/692626377 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd23c0592b0 con 0x7fd2440cf640 2024-04-25T08:57:06.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.266+0000 7fd2495e7640 1 -- 172.21.15.43:0/692626377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fd244074f60 con 0x7fd2440cf640 2024-04-25T08:57:06.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.268+0000 7fd23b7fe640 1 -- 172.21.15.43:0/692626377 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+147 (secure 0 0 0) 0x7fd23c058bb0 con 0x7fd2440cf640 2024-04-25T08:57:06.258 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2024-04-25T08:57:06.258 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQD8GSpmVOjWGxAAFNmxeGi0h2D//pa6Ne3eQw== 2024-04-25T08:57:06.258 INFO:teuthology.orchestra.run.smithi043.stdout: caps mds = "allow" 2024-04-25T08:57:06.258 INFO:teuthology.orchestra.run.smithi043.stdout: caps mon = "allow r" 2024-04-25T08:57:06.258 INFO:teuthology.orchestra.run.smithi043.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T08:57:06.259 INFO:teuthology.orchestra.run.smithi043.stderr:updated caps for client.0 2024-04-25T08:57:06.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.270+0000 7fd2495e7640 1 -- 172.21.15.43:0/692626377 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd22406b4d0 msgr2=0x7fd22406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:06.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.270+0000 7fd2495e7640 1 --2- 172.21.15.43:0/692626377 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd22406b4d0 0x7fd22406d990 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fd22c034000 tx=0x7fd22c02e000 comp rx=0 tx=0).stop 2024-04-25T08:57:06.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.270+0000 7fd2495e7640 1 -- 172.21.15.43:0/692626377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2440cf640 msgr2=0x7fd24414efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:06.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.270+0000 7fd2495e7640 1 --2- 172.21.15.43:0/692626377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2440cf640 0x7fd24414efa0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fd23c00e8a0 tx=0x7fd23c00ed60 comp rx=0 tx=0).stop 2024-04-25T08:57:06.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.271+0000 7fd2495e7640 1 -- 172.21.15.43:0/692626377 shutdown_connections 2024-04-25T08:57:06.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.271+0000 7fd2495e7640 1 --2- 172.21.15.43:0/692626377 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd22406b4d0 0x7fd22406d990 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:06.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.271+0000 7fd2495e7640 1 --2- 172.21.15.43:0/692626377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd24415de10 0x7fd24415fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:06.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.271+0000 7fd2495e7640 1 --2- 172.21.15.43:0/692626377 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd2440d2500 0x7fd24409e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:06.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.271+0000 7fd2495e7640 1 --2- 172.21.15.43:0/692626377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd2440cf640 0x7fd24414efa0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:06.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.271+0000 7fd2495e7640 1 -- 172.21.15.43:0/692626377 >> 172.21.15.43:0/692626377 conn(0x7fd2440d3a70 msgr2=0x7fd24406c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:06.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.271+0000 7fd2495e7640 1 -- 172.21.15.43:0/692626377 shutdown_connections 2024-04-25T08:57:06.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.271+0000 7fd2495e7640 1 -- 172.21.15.43:0/692626377 wait complete. 2024-04-25T08:57:06.272 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:57:06.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.383+0000 7fa75cefc640 1 Processor -- start 2024-04-25T08:57:06.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.383+0000 7fa75cefc640 1 -- start start 2024-04-25T08:57:06.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.384+0000 7fa75cefc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa7580d09b0 0x7fa7580b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:06.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.384+0000 7fa75cefc640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fa7580ce5e0 0x7fa7580cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:06.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.384+0000 7fa75cefc640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7580b4c30 con 0x7fa758161390 2024-04-25T08:57:06.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.384+0000 7fa75cefc640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7580d1910 con 0x7fa7580d09b0 2024-04-25T08:57:06.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.384+0000 7fa75cefc640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7580cf2c0 con 0x7fa7580ce5e0 2024-04-25T08:57:06.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.384+0000 7fa74ffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa7580d09b0 0x7fa7580b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:06.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.384+0000 7fa74ffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa7580d09b0 0x7fa7580b6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39706/0 (socket says 172.21.15.43:39706) 2024-04-25T08:57:06.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.384+0000 7fa74ffff640 1 -- 172.21.15.43:0/1298009226 learned_addr learned my addr 172.21.15.43:0/1298009226 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:06.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.384+0000 7fa74ffff640 1 -- 172.21.15.43:0/1298009226 >> v2:172.21.15.193:3300/0 conn(0x7fa7580ce5e0 msgr2=0x7fa7580cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:06.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.384+0000 7fa74ffff640 1 --2- 172.21.15.43:0/1298009226 >> v2:172.21.15.193:3300/0 conn(0x7fa7580ce5e0 0x7fa7580cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:06.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.384+0000 7fa74ffff640 1 -- 172.21.15.43:0/1298009226 >> v1:172.21.15.193:6789/0 conn(0x7fa758161390 legacy=0x7fa7580d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:06.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.384+0000 7fa74ffff640 1 -- 172.21.15.43:0/1298009226 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7580cf5a0 con 0x7fa7580d09b0 2024-04-25T08:57:06.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.385+0000 7fa74ffff640 1 --2- 172.21.15.43:0/1298009226 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7580d09b0 0x7fa7580b6820 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa75808dd30 tx=0x7fa74002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d88dbae89fa0b0f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:06.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.385+0000 7fa755d9b640 1 -- 172.21.15.43:0/1298009226 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa740037070 con 0x7fa7580d09b0 2024-04-25T08:57:06.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.385+0000 7fa755d9b640 1 -- 172.21.15.43:0/1298009226 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa740032440 con 0x7fa7580d09b0 2024-04-25T08:57:06.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.385+0000 7fa755d9b640 1 -- 172.21.15.43:0/1298009226 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa740032810 con 0x7fa7580d09b0 2024-04-25T08:57:06.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.385+0000 7fa75cefc640 1 -- 172.21.15.43:0/1298009226 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7580d09b0 msgr2=0x7fa7580b6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:06.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.385+0000 7fa75cefc640 1 --2- 172.21.15.43:0/1298009226 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7580d09b0 0x7fa7580b6820 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa75808dd30 tx=0x7fa74002a230 comp rx=0 tx=0).stop 2024-04-25T08:57:06.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.385+0000 7fa75cefc640 1 -- 172.21.15.43:0/1298009226 shutdown_connections 2024-04-25T08:57:06.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.385+0000 7fa75cefc640 1 --2- 172.21.15.43:0/1298009226 >> v2:172.21.15.193:3300/0 conn(0x7fa7580ce5e0 0x7fa7580cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:06.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.385+0000 7fa75cefc640 1 --2- 172.21.15.43:0/1298009226 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7580d09b0 0x7fa7580b6820 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:06.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.385+0000 7fa75cefc640 1 -- 172.21.15.43:0/1298009226 >> 172.21.15.43:0/1298009226 conn(0x7fa75809cd20 msgr2=0x7fa7580d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:06.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.386+0000 7fa75cefc640 1 -- 172.21.15.43:0/1298009226 shutdown_connections 2024-04-25T08:57:06.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.386+0000 7fa75cefc640 1 -- 172.21.15.43:0/1298009226 wait complete. 2024-04-25T08:57:06.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.387+0000 7fa75cefc640 1 Processor -- start 2024-04-25T08:57:06.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.387+0000 7fa75cefc640 1 -- start start 2024-04-25T08:57:06.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.387+0000 7fa75cefc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7580ce5e0 0x7fa7581f3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:06.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.388+0000 7fa75cefc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7580d09b0 0x7fa7580ca1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:06.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.388+0000 7fa75cefc640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa758161390 0x7fa758155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:06.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.388+0000 7fa75cefc640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa758161770 con 0x7fa7580d09b0 2024-04-25T08:57:06.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.388+0000 7fa75cefc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7580d15a0 con 0x7fa7580ce5e0 2024-04-25T08:57:06.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.388+0000 7fa75cefc640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7580cef70 con 0x7fa758161390 2024-04-25T08:57:06.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.388+0000 7fa74ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7580d09b0 0x7fa7580ca1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:06.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.388+0000 7fa74ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7580d09b0 0x7fa7580ca1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39712/0 (socket says 172.21.15.43:39712) 2024-04-25T08:57:06.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.388+0000 7fa74ffff640 1 -- 172.21.15.43:0/3534391062 learned_addr learned my addr 172.21.15.43:0/3534391062 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:06.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.388+0000 7fa75659c640 1 --2- 172.21.15.43:0/3534391062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7580ce5e0 0x7fa7581f3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:06.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.388+0000 7fa74ffff640 1 -- 172.21.15.43:0/3534391062 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa758161390 msgr2=0x7fa758155e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:06.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.388+0000 7fa756d9d640 1 --2- 172.21.15.43:0/3534391062 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa758161390 0x7fa758155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:06.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.388+0000 7fa74ffff640 1 --2- 172.21.15.43:0/3534391062 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa758161390 0x7fa758155e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:06.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.388+0000 7fa74ffff640 1 -- 172.21.15.43:0/3534391062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7580ce5e0 msgr2=0x7fa7581f3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:06.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.388+0000 7fa74ffff640 1 --2- 172.21.15.43:0/3534391062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7580ce5e0 0x7fa7581f3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:06.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.388+0000 7fa74ffff640 1 -- 172.21.15.43:0/3534391062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7580c87b0 con 0x7fa7580d09b0 2024-04-25T08:57:06.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.388+0000 7fa75659c640 1 --2- 172.21.15.43:0/3534391062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7580ce5e0 0x7fa7581f3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:06.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.388+0000 7fa74ffff640 1 --2- 172.21.15.43:0/3534391062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7580d09b0 0x7fa7580ca1c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa740000c00 tx=0x7fa7400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:06.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.388+0000 7fa74effd640 1 -- 172.21.15.43:0/3534391062 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa740041070 con 0x7fa7580d09b0 2024-04-25T08:57:06.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.388+0000 7fa74effd640 1 -- 172.21.15.43:0/3534391062 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa74003c870 con 0x7fa7580d09b0 2024-04-25T08:57:06.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.388+0000 7fa75cefc640 1 -- 172.21.15.43:0/3534391062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa758155390 con 0x7fa7580d09b0 2024-04-25T08:57:06.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.388+0000 7fa74effd640 1 -- 172.21.15.43:0/3534391062 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa740037040 con 0x7fa7580d09b0 2024-04-25T08:57:06.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.388+0000 7fa75cefc640 1 -- 172.21.15.43:0/3534391062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7581f3dd0 con 0x7fa7580d09b0 2024-04-25T08:57:06.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.389+0000 7fa75cefc640 1 -- 172.21.15.43:0/3534391062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7580680f0 con 0x7fa7580d09b0 2024-04-25T08:57:06.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.390+0000 7fa74effd640 1 -- 172.21.15.43:0/3534391062 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa740044070 con 0x7fa7580d09b0 2024-04-25T08:57:06.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.390+0000 7fa74effd640 1 --2- 172.21.15.43:0/3534391062 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa73806b4d0 0x7fa73806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:06.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.390+0000 7fa74effd640 1 -- 172.21.15.43:0/3534391062 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa7400aace0 con 0x7fa7580d09b0 2024-04-25T08:57:06.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.391+0000 7fa75659c640 1 --2- 172.21.15.43:0/3534391062 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa73806b4d0 0x7fa73806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:06.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.391+0000 7fa75659c640 1 --2- 172.21.15.43:0/3534391062 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa73806b4d0 0x7fa73806d990 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa74800a810 tx=0x7fa748008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:06.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.394+0000 7fa74effd640 1 -- 172.21.15.43:0/3534391062 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa7580b4c30 con 0x7fa7580d09b0 2024-04-25T08:57:06.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.613+0000 7fa75cefc640 1 -- 172.21.15.43:0/3534391062 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa758074ef0 con 0x7fa7580d09b0 2024-04-25T08:57:06.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.613+0000 7fa74effd640 1 -- 172.21.15.43:0/3534391062 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fa740079d10 con 0x7fa7580d09b0 2024-04-25T08:57:06.603 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:57:06.603 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1504450503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1504450503},{"type":"v1","addr":"172.21.15.193:6835","nonce":1504450503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5691,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2657133821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2657133821},{"type":"v1","addr":"172.21.15.43:6834","nonce":2657133821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":5694,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/4271431818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":4271431818},{"type":"v1","addr":"172.21.15.43:6835","nonce":4271431818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:57:01.856132+0000","modified":"2024-04-25T08:57:03.857410+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5709":{"gid":5709,"name":"d","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.193:6837/4164634380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4164634380},{"type":"v1","addr":"172.21.15.193:6837","nonce":4164634380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:57:06.604 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 21 2024-04-25T08:57:06.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.615+0000 7fa75cefc640 1 -- 172.21.15.43:0/3534391062 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa73806b4d0 msgr2=0x7fa73806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:06.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.615+0000 7fa75cefc640 1 --2- 172.21.15.43:0/3534391062 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa73806b4d0 0x7fa73806d990 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa74800a810 tx=0x7fa748008040 comp rx=0 tx=0).stop 2024-04-25T08:57:06.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.615+0000 7fa75cefc640 1 -- 172.21.15.43:0/3534391062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7580d09b0 msgr2=0x7fa7580ca1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:06.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.615+0000 7fa75cefc640 1 --2- 172.21.15.43:0/3534391062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7580d09b0 0x7fa7580ca1c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fa740000c00 tx=0x7fa7400046c0 comp rx=0 tx=0).stop 2024-04-25T08:57:06.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.615+0000 7fa75cefc640 1 -- 172.21.15.43:0/3534391062 shutdown_connections 2024-04-25T08:57:06.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.615+0000 7fa75cefc640 1 --2- 172.21.15.43:0/3534391062 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa758161390 0x7fa758155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:06.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.615+0000 7fa75cefc640 1 --2- 172.21.15.43:0/3534391062 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7580d09b0 0x7fa7580ca1c0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:06.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.615+0000 7fa75cefc640 1 --2- 172.21.15.43:0/3534391062 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa73806b4d0 0x7fa73806d990 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:06.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.615+0000 7fa75cefc640 1 --2- 172.21.15.43:0/3534391062 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7580ce5e0 0x7fa7581f3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:06.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.616+0000 7fa75cefc640 1 -- 172.21.15.43:0/3534391062 >> 172.21.15.43:0/3534391062 conn(0x7fa75809cd20 msgr2=0x7fa7580b8420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:06.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.616+0000 7fa75cefc640 1 -- 172.21.15.43:0/3534391062 shutdown_connections 2024-04-25T08:57:06.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:06.616+0000 7fa75cefc640 1 -- 172.21.15.43:0/3534391062 wait complete. 2024-04-25T08:57:07.019 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:57:01.856132+0000', 'modified': '2024-04-25T08:57:03.857410+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5709}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5709': {'gid': 5709, 'name': 'd', 'rank': 0, 'incarnation': 19, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.193:6837/4164634380', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6836', 'nonce': 4164634380}, {'type': 'v1', 'addr': '172.21.15.193:6837', 'nonce': 4164634380}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:57:07.019 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:57:07.019 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T08:57:07.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.137+0000 7fdfa429c640 1 Processor -- start 2024-04-25T08:57:07.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.137+0000 7fdfa429c640 1 -- start start 2024-04-25T08:57:07.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.138+0000 7fdfa429c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdf9c06f300 0x7fdf9c06e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:07.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.138+0000 7fdfa429c640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fdf9c06c4e0 0x7fdf9c06c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:07.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.138+0000 7fdfa429c640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf9c0d0d80 con 0x7fdf9c15dcf0 2024-04-25T08:57:07.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.138+0000 7fdfa429c640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf9c06c050 con 0x7fdf9c06c4e0 2024-04-25T08:57:07.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.138+0000 7fdfa429c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c0d04e0 con 0x7fdf9c06f300 2024-04-25T08:57:07.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.138+0000 7fdfa2038640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fdf9c15dcf0 0x7fdf9c06ebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48110/0 (socket says 172.21.15.43:48110) 2024-04-25T08:57:07.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.138+0000 7fdfa2038640 1 -- 172.21.15.43:0/2008090880 learned_addr learned my addr 172.21.15.43:0/2008090880 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:07.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.138+0000 7fdfa1036640 1 -- 172.21.15.43:0/2008090880 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2853456393 0 0) 0x7fdf9c0d0d80 con 0x7fdf9c15dcf0 2024-04-25T08:57:07.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.139+0000 7fdfa1036640 1 -- 172.21.15.43:0/2008090880 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf84003520 con 0x7fdf9c15dcf0 2024-04-25T08:57:07.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.139+0000 7fdfa1837640 1 --2- 172.21.15.43:0/2008090880 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdf9c06f300 0x7fdf9c06e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:07.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.139+0000 7fdfa2839640 1 --2- 172.21.15.43:0/2008090880 >> v2:172.21.15.193:3300/0 conn(0x7fdf9c06c4e0 0x7fdf9c06c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:07.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.139+0000 7fdfa1036640 1 -- 172.21.15.43:0/2008090880 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3665026249 0 0) 0x7fdf84003520 con 0x7fdf9c15dcf0 2024-04-25T08:57:07.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.139+0000 7fdfa1036640 1 -- 172.21.15.43:0/2008090880 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdf9c06f300 msgr2=0x7fdf9c06e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:07.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.139+0000 7fdfa1036640 1 --2- 172.21.15.43:0/2008090880 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdf9c06f300 0x7fdf9c06e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:07.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.139+0000 7fdfa1036640 1 -- 172.21.15.43:0/2008090880 >> v2:172.21.15.193:3300/0 conn(0x7fdf9c06c4e0 msgr2=0x7fdf9c06c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:07.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.139+0000 7fdfa1036640 1 --2- 172.21.15.43:0/2008090880 >> v2:172.21.15.193:3300/0 conn(0x7fdf9c06c4e0 0x7fdf9c06c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:07.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.139+0000 7fdfa1036640 1 -- 172.21.15.43:0/2008090880 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf9c0d07c0 con 0x7fdf9c15dcf0 2024-04-25T08:57:07.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.139+0000 7fdfa1036640 1 -- 172.21.15.43:0/2008090880 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7fdf94001670 con 0x7fdf9c15dcf0 2024-04-25T08:57:07.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.139+0000 7fdfa1036640 1 -- 172.21.15.43:0/2008090880 >> v1:172.21.15.43:6789/0 conn(0x7fdf9c15dcf0 legacy=0x7fdf9c06ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:07.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.139+0000 7fdfa1036640 1 --2- 172.21.15.43:0/2008090880 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf840039b0 0x7fdf84003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:07.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.139+0000 7fdfa1036640 1 --2- 172.21.15.43:0/2008090880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf84004800 0x7fdf84004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:07.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.139+0000 7fdfa1036640 1 --2- 172.21.15.43:0/2008090880 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdf9c06f300 0x7fdf840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:07.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.139+0000 7fdfa1036640 1 -- 172.21.15.43:0/2008090880 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c06c050 con 0x7fdf84004800 2024-04-25T08:57:07.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.139+0000 7fdfa1036640 1 -- 172.21.15.43:0/2008090880 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c0d04e0 con 0x7fdf840039b0 2024-04-25T08:57:07.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.139+0000 7fdfa1036640 1 -- 172.21.15.43:0/2008090880 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8400b9c0 con 0x7fdf9c06f300 2024-04-25T08:57:07.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.139+0000 7fdfa1837640 1 --2- 172.21.15.43:0/2008090880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf84004800 0x7fdf84004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:07.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.139+0000 7fdfa2038640 1 --2- 172.21.15.43:0/2008090880 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf840039b0 0x7fdf84003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:07.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.139+0000 7fdfa2839640 1 --2- 172.21.15.43:0/2008090880 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdf9c06f300 0x7fdf840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:07.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.139+0000 7fdfa1837640 1 -- 172.21.15.43:0/2008090880 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdf9c06f300 msgr2=0x7fdf840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:07.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.139+0000 7fdfa1837640 1 --2- 172.21.15.43:0/2008090880 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdf9c06f300 0x7fdf840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:07.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.139+0000 7fdfa1837640 1 -- 172.21.15.43:0/2008090880 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf840039b0 msgr2=0x7fdf84003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:07.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.139+0000 7fdfa1837640 1 --2- 172.21.15.43:0/2008090880 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf840039b0 0x7fdf84003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:07.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.139+0000 7fdfa1837640 1 -- 172.21.15.43:0/2008090880 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf84003520 con 0x7fdf84004800 2024-04-25T08:57:07.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.140+0000 7fdfa1837640 1 --2- 172.21.15.43:0/2008090880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf84004800 0x7fdf84004c80 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fdf90009af0 tx=0x7fdf84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=235721484d381e5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:07.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.140+0000 7fdfa1036640 1 -- 172.21.15.43:0/2008090880 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf90036070 con 0x7fdf84004800 2024-04-25T08:57:07.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.140+0000 7fdfa1036640 1 -- 172.21.15.43:0/2008090880 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf90031400 con 0x7fdf84004800 2024-04-25T08:57:07.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.140+0000 7fdfa1036640 1 -- 172.21.15.43:0/2008090880 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf90009690 con 0x7fdf84004800 2024-04-25T08:57:07.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.141+0000 7fdfa429c640 1 -- 172.21.15.43:0/2008090880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf84004800 msgr2=0x7fdf84004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:07.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.141+0000 7fdfa429c640 1 --2- 172.21.15.43:0/2008090880 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf84004800 0x7fdf84004c80 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fdf90009af0 tx=0x7fdf84005830 comp rx=0 tx=0).stop 2024-04-25T08:57:07.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.141+0000 7fdfa2839640 1 -- 172.21.15.43:0/2008090880 reap_dead start 2024-04-25T08:57:07.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.141+0000 7fdfa429c640 1 -- 172.21.15.43:0/2008090880 shutdown_connections 2024-04-25T08:57:07.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.141+0000 7fdfa429c640 1 -- 172.21.15.43:0/2008090880 >> 172.21.15.43:0/2008090880 conn(0x7fdf9c0d1700 msgr2=0x7fdf9c0d1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:07.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.141+0000 7fdfa429c640 1 -- 172.21.15.43:0/2008090880 shutdown_connections 2024-04-25T08:57:07.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.141+0000 7fdfa429c640 1 -- 172.21.15.43:0/2008090880 wait complete. 2024-04-25T08:57:07.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.142+0000 7fdfa429c640 1 Processor -- start 2024-04-25T08:57:07.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.142+0000 7fdfa429c640 1 -- start start 2024-04-25T08:57:07.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.143+0000 7fdfa429c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf84004800 0x7fdf9c1564c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:07.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.143+0000 7fdfa429c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdf9c0c5e90 0x7fdf9c0b2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:07.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.143+0000 7fdfa429c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf9c0b29c0 0x7fdf9c158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:07.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.143+0000 7fdfa429c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c0b0d50 con 0x7fdf9c0b29c0 2024-04-25T08:57:07.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.143+0000 7fdfa429c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c06bce0 con 0x7fdf84004800 2024-04-25T08:57:07.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.143+0000 7fdfa429c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c0b0ed0 con 0x7fdf9c0c5e90 2024-04-25T08:57:07.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.143+0000 7fdfa2839640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf9c0b29c0 0x7fdf9c158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:07.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.143+0000 7fdfa2839640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf9c0b29c0 0x7fdf9c158e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39726/0 (socket says 172.21.15.43:39726) 2024-04-25T08:57:07.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.143+0000 7fdfa2839640 1 -- 172.21.15.43:0/1767925764 learned_addr learned my addr 172.21.15.43:0/1767925764 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:07.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.143+0000 7fdfa1837640 1 --2- 172.21.15.43:0/1767925764 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdf9c0c5e90 0x7fdf9c0b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:07.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.143+0000 7fdfa2038640 1 --2- 172.21.15.43:0/1767925764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf84004800 0x7fdf9c1564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:07.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.143+0000 7fdfa2839640 1 -- 172.21.15.43:0/1767925764 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdf9c0c5e90 msgr2=0x7fdf9c0b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:07.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.143+0000 7fdfa2839640 1 --2- 172.21.15.43:0/1767925764 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdf9c0c5e90 0x7fdf9c0b2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:07.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.143+0000 7fdfa2839640 1 -- 172.21.15.43:0/1767925764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf84004800 msgr2=0x7fdf9c1564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:07.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.143+0000 7fdfa2839640 1 --2- 172.21.15.43:0/1767925764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf84004800 0x7fdf9c1564c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:07.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.143+0000 7fdfa2839640 1 -- 172.21.15.43:0/1767925764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf9c156f90 con 0x7fdf9c0b29c0 2024-04-25T08:57:07.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.143+0000 7fdfa2038640 1 --2- 172.21.15.43:0/1767925764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf84004800 0x7fdf9c1564c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:07.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.143+0000 7fdfa2839640 1 --2- 172.21.15.43:0/1767925764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf9c0b29c0 0x7fdf9c158e60 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c072160 tx=0x7fdf8c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:07.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.144+0000 7fdf9affd640 1 -- 172.21.15.43:0/1767925764 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c015430 con 0x7fdf9c0b29c0 2024-04-25T08:57:07.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.144+0000 7fdf9affd640 1 -- 172.21.15.43:0/1767925764 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf8c0155d0 con 0x7fdf9c0b29c0 2024-04-25T08:57:07.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.144+0000 7fdfa429c640 1 -- 172.21.15.43:0/1767925764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf9c156a00 con 0x7fdf9c0b29c0 2024-04-25T08:57:07.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.144+0000 7fdf9affd640 1 -- 172.21.15.43:0/1767925764 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c0159c0 con 0x7fdf9c0b29c0 2024-04-25T08:57:07.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.144+0000 7fdfa429c640 1 -- 172.21.15.43:0/1767925764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf9c0c53b0 con 0x7fdf9c0b29c0 2024-04-25T08:57:07.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.145+0000 7fdf9affd640 1 -- 172.21.15.43:0/1767925764 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdf8c015b60 con 0x7fdf9c0b29c0 2024-04-25T08:57:07.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.146+0000 7fdf9affd640 1 --2- 172.21.15.43:0/1767925764 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdf8806b4d0 0x7fdf8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:07.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.146+0000 7fdf9affd640 1 -- 172.21.15.43:0/1767925764 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdf8c08b070 con 0x7fdf9c0b29c0 2024-04-25T08:57:07.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.146+0000 7fdfa2038640 1 --2- 172.21.15.43:0/1767925764 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdf8806b4d0 0x7fdf8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:07.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.146+0000 7fdfa2038640 1 --2- 172.21.15.43:0/1767925764 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdf8806b4d0 0x7fdf8806d990 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdf940014f0 tx=0x7fdf94009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:07.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.147+0000 7fdfa6f95740 1 Processor -- start 2024-04-25T08:57:07.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.147+0000 7fdfa6f95740 1 -- start start 2024-04-25T08:57:07.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.148+0000 7fdfa6f95740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d73b80230 0x556d73b80610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:07.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.148+0000 7fdfa6f95740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556d73b9dad0 0x556d73b9e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:07.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.148+0000 7fdfa6f95740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d73b55f10 0x556d73b9c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:07.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.148+0000 7fdfa6f95740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d73ab8a10 con 0x556d73b80230 2024-04-25T08:57:07.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.148+0000 7fdfa6f95740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d73a61b30 con 0x556d73b55f10 2024-04-25T08:57:07.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.148+0000 7fdfa6f95740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x556d73ab4c50 con 0x556d73b9dad0 2024-04-25T08:57:07.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.148+0000 7fdfa2038640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d73b80230 0x556d73b80610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:07.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.148+0000 7fdfa2038640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d73b80230 0x556d73b80610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39728/0 (socket says 172.21.15.43:39728) 2024-04-25T08:57:07.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.148+0000 7fdfa2038640 1 -- 172.21.15.43:0/3886464461 learned_addr learned my addr 172.21.15.43:0/3886464461 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:07.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.148+0000 7fdfa2839640 1 --2- 172.21.15.43:0/3886464461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d73b55f10 0x556d73b9c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:07.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.148+0000 7fdfa1837640 1 --2- 172.21.15.43:0/3886464461 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556d73b9dad0 0x556d73b9e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:07.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.148+0000 7fdfa2038640 1 -- 172.21.15.43:0/3886464461 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556d73b9dad0 msgr2=0x556d73b9e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:07.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.148+0000 7fdfa2038640 1 --2- 172.21.15.43:0/3886464461 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556d73b9dad0 0x556d73b9e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:07.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.148+0000 7fdfa2038640 1 -- 172.21.15.43:0/3886464461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d73b55f10 msgr2=0x556d73b9c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:07.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.148+0000 7fdfa2038640 1 --2- 172.21.15.43:0/3886464461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d73b55f10 0x556d73b9c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:07.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.148+0000 7fdfa2038640 1 -- 172.21.15.43:0/3886464461 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d73a76e10 con 0x556d73b80230 2024-04-25T08:57:07.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.148+0000 7fdfa2038640 1 --2- 172.21.15.43:0/3886464461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d73b80230 0x556d73b80610 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fdf94007ef0 tx=0x7fdf94013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f765cc869d7a09 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:07.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.148+0000 7fdf83fff640 1 -- 172.21.15.43:0/3886464461 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf94026070 con 0x556d73b80230 2024-04-25T08:57:07.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.148+0000 7fdf83fff640 1 -- 172.21.15.43:0/3886464461 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf9401c050 con 0x556d73b80230 2024-04-25T08:57:07.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.148+0000 7fdf83fff640 1 -- 172.21.15.43:0/3886464461 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf94021410 con 0x556d73b80230 2024-04-25T08:57:07.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.149+0000 7fdfa6f95740 1 -- 172.21.15.43:0/3886464461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d73b80230 msgr2=0x556d73b80610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:07.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.149+0000 7fdfa6f95740 1 --2- 172.21.15.43:0/3886464461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d73b80230 0x556d73b80610 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fdf94007ef0 tx=0x7fdf94013000 comp rx=0 tx=0).stop 2024-04-25T08:57:07.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.149+0000 7fdfa6f95740 1 -- 172.21.15.43:0/3886464461 shutdown_connections 2024-04-25T08:57:07.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.149+0000 7fdfa6f95740 1 --2- 172.21.15.43:0/3886464461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d73b55f10 0x556d73b9c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:07.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.149+0000 7fdfa6f95740 1 --2- 172.21.15.43:0/3886464461 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556d73b9dad0 0x556d73b9e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:07.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.149+0000 7fdfa6f95740 1 --2- 172.21.15.43:0/3886464461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d73b80230 0x556d73b80610 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:07.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.149+0000 7fdfa6f95740 1 -- 172.21.15.43:0/3886464461 >> 172.21.15.43:0/3886464461 conn(0x556d73b52840 msgr2=0x556d73b9b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:07.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.149+0000 7fdfa6f95740 1 -- 172.21.15.43:0/3886464461 shutdown_connections 2024-04-25T08:57:07.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.149+0000 7fdfa6f95740 1 -- 172.21.15.43:0/3886464461 wait complete. 2024-04-25T08:57:07.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.150+0000 7fdfa6f95740 1 -- start start 2024-04-25T08:57:07.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.150+0000 7fdfa6f95740 1 Processor -- start 2024-04-25T08:57:07.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.151+0000 7fdfa6f95740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556d73b80230 0x556d73b7bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:07.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.151+0000 7fdfa6f95740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d73b9dad0 0x556d73b82810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:07.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.151+0000 7fdfa6f95740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d73bab670 0x556d73b81cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:07.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.151+0000 7fdfa6f95740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d73b56cb0 con 0x556d73b9dad0 2024-04-25T08:57:07.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.151+0000 7fdfa6f95740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d73a750c0 con 0x556d73bab670 2024-04-25T08:57:07.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.151+0000 7fdfa6f95740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x556d73b50770 con 0x556d73b80230 2024-04-25T08:57:07.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.151+0000 7fdfa6f95740 10 client.0 _ll_register_callbacks cb 0x556d73a68ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:07.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.151+0000 7fdfa1837640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d73b9dad0 0x556d73b82810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:07.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.151+0000 7fdfa1837640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d73b9dad0 0x556d73b82810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39730/0 (socket says 172.21.15.43:39730) 2024-04-25T08:57:07.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.151+0000 7fdfa1837640 1 -- 172.21.15.43:0/702454099 learned_addr learned my addr 172.21.15.43:0/702454099 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:07.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.151+0000 7fdfa2839640 1 --2- 172.21.15.43:0/702454099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d73bab670 0x556d73b81cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:07.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.151+0000 7fdfa2038640 1 --2- 172.21.15.43:0/702454099 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556d73b80230 0x556d73b7bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:07.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.151+0000 7fdfa1837640 1 -- 172.21.15.43:0/702454099 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556d73b80230 msgr2=0x556d73b7bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:07.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.151+0000 7fdfa1837640 1 --2- 172.21.15.43:0/702454099 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556d73b80230 0x556d73b7bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:07.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.151+0000 7fdfa1837640 1 -- 172.21.15.43:0/702454099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d73bab670 msgr2=0x556d73b81cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:07.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.151+0000 7fdfa1837640 1 --2- 172.21.15.43:0/702454099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d73bab670 0x556d73b81cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:07.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.151+0000 7fdfa1837640 1 -- 172.21.15.43:0/702454099 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556d737cd2d0 con 0x556d73b9dad0 2024-04-25T08:57:07.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.151+0000 7fdfa1837640 1 --2- 172.21.15.43:0/702454099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d73b9dad0 0x556d73b82810 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fdf90006f80 tx=0x7fdf900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3536cbec6c31b1bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:07.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.151+0000 7fdf817fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:57:07.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.152+0000 7fdf817fa640 1 -- 172.21.15.43:0/702454099 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf90036040 con 0x556d73b9dad0 2024-04-25T08:57:07.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.152+0000 7fdfa6f95740 1 -- 172.21.15.43:0/702454099 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf9401c050 con 0x556d73b9dad0 2024-04-25T08:57:07.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.152+0000 7fdf817fa640 1 -- 172.21.15.43:0/702454099 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdf90009690 con 0x556d73b9dad0 2024-04-25T08:57:07.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.152+0000 7fdf817fa640 1 -- 172.21.15.43:0/702454099 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf90004020 con 0x556d73b9dad0 2024-04-25T08:57:07.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.152+0000 7fdfa6f95740 10 client.5838 fetch_fsmap learned FSMap version 21 2024-04-25T08:57:07.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.152+0000 7fdfa6f95740 1 -- 172.21.15.43:0/702454099 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x556d738976b0 con 0x556d73b9dad0 2024-04-25T08:57:07.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.152+0000 7fdf817fa640 1 -- 172.21.15.43:0/702454099 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf9401c050 con 0x556d73b9dad0 2024-04-25T08:57:07.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.152+0000 7fdf817fa640 1 -- 172.21.15.43:0/702454099 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdf900041c0 con 0x556d73b9dad0 2024-04-25T08:57:07.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.152+0000 7fdfa6f95740 10 client.5838 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T08:57:07.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.152+0000 7fdfa6f95740 10 client.5838 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T08:57:07.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.152+0000 7fdfa6f95740 20 client.5838 populate_metadata read hostname 'smithi043' 2024-04-25T08:57:07.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.152+0000 7fdfa6f95740 1 --2- 172.21.15.43:0/702454099 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x556d73b9ffa0 0x556d73bb9f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:07.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.152+0000 7fdfa6f95740 4 client.5838 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:57:07.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.152+0000 7fdfa6f95740 1 -- 172.21.15.43:0/702454099 --> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556d73b17920 con 0x556d73b9ffa0 2024-04-25T08:57:07.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.152+0000 7fdfa2038640 1 --2- 172.21.15.43:0/702454099 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x556d73b9ffa0 0x556d73bb9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:07.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.153+0000 7fdfa2038640 1 --2- 172.21.15.43:0/702454099 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x556d73b9ffa0 0x556d73bb9f90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a20ec1a0657429f server_cookie=68b2a3c4ad7297f0 in_seq=0 out_seq=0 2024-04-25T08:57:07.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.153+0000 7fdf817fa640 10 client.5838 ms_handle_connect on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:07.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.153+0000 7fdf817fa640 1 -- 172.21.15.43:0/702454099 <== mds.0 v2:172.21.15.193:6836/4164634380 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556d73b17920 con 0x556d73b9ffa0 2024-04-25T08:57:07.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.153+0000 7fdf817fa640 10 client.5838 handle_command_reply: tid=0 2024-04-25T08:57:07.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.153+0000 7fdfa6f95740 1 client.5838 shutdown 2024-04-25T08:57:07.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.154+0000 7fdfa6f95740 1 -- 172.21.15.43:0/702454099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d73b9dad0 msgr2=0x556d73b82810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:07.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.154+0000 7fdfa6f95740 1 --2- 172.21.15.43:0/702454099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d73b9dad0 0x556d73b82810 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fdf90006f80 tx=0x7fdf900046c0 comp rx=0 tx=0).stop 2024-04-25T08:57:07.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.154+0000 7fdfa6f95740 1 -- 172.21.15.43:0/702454099 shutdown_connections 2024-04-25T08:57:07.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.154+0000 7fdfa6f95740 1 --2- 172.21.15.43:0/702454099 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x556d73b9ffa0 0x556d73bb9f90 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:07.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.154+0000 7fdfa6f95740 1 --2- 172.21.15.43:0/702454099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d73bab670 0x556d73b81cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:07.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.154+0000 7fdfa6f95740 1 --2- 172.21.15.43:0/702454099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d73b9dad0 0x556d73b82810 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:07.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.154+0000 7fdfa6f95740 1 --2- 172.21.15.43:0/702454099 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556d73b80230 0x556d73b7bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:07.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.154+0000 7fdf817fa640 0 client.5838 ms_handle_reset on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:07.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.154+0000 7fdfa6f95740 1 -- 172.21.15.43:0/702454099 >> 172.21.15.43:0/702454099 conn(0x556d73b55f10 msgr2=0x556d73b9e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:07.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.154+0000 7fdfa6f95740 1 -- 172.21.15.43:0/702454099 shutdown_connections 2024-04-25T08:57:07.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.154+0000 7fdfa6f95740 1 -- 172.21.15.43:0/702454099 wait complete. 2024-04-25T08:57:07.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.154+0000 7fdfa6f95740 20 client.5838 trim_cache size 0 max 16384 2024-04-25T08:57:07.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.170+0000 7fdfa6f95740 1 Processor -- start 2024-04-25T08:57:07.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.170+0000 7fdfa6f95740 1 -- start start 2024-04-25T08:57:07.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.171+0000 7fdfa6f95740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d73b80230 0x556d73b564e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:07.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.171+0000 7fdfa6f95740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556d73b9dad0 0x556d73c00540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:07.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.171+0000 7fdfa6f95740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d73bab670 0x556d73c00a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:07.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.171+0000 7fdfa6f95740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c158520 con 0x556d73b80230 2024-04-25T08:57:07.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.171+0000 7fdfa6f95740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c1f4f50 con 0x556d73bab670 2024-04-25T08:57:07.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.171+0000 7fdfa6f95740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c157f20 con 0x556d73b9dad0 2024-04-25T08:57:07.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.171+0000 7fdfa2038640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d73b80230 0x556d73b564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:07.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.171+0000 7fdfa2038640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d73b80230 0x556d73b564e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39740/0 (socket says 172.21.15.43:39740) 2024-04-25T08:57:07.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.171+0000 7fdfa2038640 1 -- 172.21.15.43:0/1005101666 learned_addr learned my addr 172.21.15.43:0/1005101666 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:07.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.171+0000 7fdfa2038640 1 -- 172.21.15.43:0/1005101666 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556d73b9dad0 msgr2=0x556d73c00540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:07.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.171+0000 7fdfa2038640 1 --2- 172.21.15.43:0/1005101666 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556d73b9dad0 0x556d73c00540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:07.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.171+0000 7fdfa2038640 1 -- 172.21.15.43:0/1005101666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d73bab670 msgr2=0x556d73c00a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:07.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.171+0000 7fdfa2038640 1 --2- 172.21.15.43:0/1005101666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d73bab670 0x556d73c00a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:07.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.171+0000 7fdfa2038640 1 -- 172.21.15.43:0/1005101666 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556d73b91f20 con 0x556d73b80230 2024-04-25T08:57:07.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.171+0000 7fdfa2038640 1 --2- 172.21.15.43:0/1005101666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d73b80230 0x556d73b564e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fdf94024040 tx=0x7fdf94016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b09dd3f3d4e44e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:07.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.171+0000 7fdf837fe640 1 -- 172.21.15.43:0/1005101666 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf94026040 con 0x556d73b80230 2024-04-25T08:57:07.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.171+0000 7fdf837fe640 1 -- 172.21.15.43:0/1005101666 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf9401c070 con 0x556d73b80230 2024-04-25T08:57:07.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.171+0000 7fdf837fe640 1 -- 172.21.15.43:0/1005101666 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf94031580 con 0x556d73b80230 2024-04-25T08:57:07.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.172+0000 7fdfa6f95740 1 -- 172.21.15.43:0/1005101666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d73b80230 msgr2=0x556d73b564e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:07.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.172+0000 7fdfa6f95740 1 --2- 172.21.15.43:0/1005101666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d73b80230 0x556d73b564e0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fdf94024040 tx=0x7fdf94016000 comp rx=0 tx=0).stop 2024-04-25T08:57:07.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.172+0000 7fdfa6f95740 1 -- 172.21.15.43:0/1005101666 shutdown_connections 2024-04-25T08:57:07.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.172+0000 7fdfa6f95740 1 --2- 172.21.15.43:0/1005101666 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d73bab670 0x556d73c00a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:07.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.172+0000 7fdfa6f95740 1 --2- 172.21.15.43:0/1005101666 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556d73b9dad0 0x556d73c00540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:07.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.172+0000 7fdfa6f95740 1 --2- 172.21.15.43:0/1005101666 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d73b80230 0x556d73b564e0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:07.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.172+0000 7fdfa6f95740 1 -- 172.21.15.43:0/1005101666 >> 172.21.15.43:0/1005101666 conn(0x556d73b55f10 msgr2=0x556d73b9ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:07.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.172+0000 7fdfa6f95740 1 -- 172.21.15.43:0/1005101666 shutdown_connections 2024-04-25T08:57:07.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.172+0000 7fdfa6f95740 1 -- 172.21.15.43:0/1005101666 wait complete. 2024-04-25T08:57:07.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.172+0000 7fdfa6f95740 1 -- start start 2024-04-25T08:57:07.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.172+0000 7fdfa6f95740 1 Processor -- start 2024-04-25T08:57:07.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.173+0000 7fdfa6f95740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556d73b80230 0x556d73b9e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:07.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.173+0000 7fdfa6f95740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d73b9dad0 0x556d73b9e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:07.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.173+0000 7fdfa6f95740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d73bab670 0x556d73b9a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:07.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.173+0000 7fdfa6f95740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d73b9a240 con 0x556d73bab670 2024-04-25T08:57:07.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.173+0000 7fdfa6f95740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x556d73b82db0 con 0x556d73b9dad0 2024-04-25T08:57:07.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.173+0000 7fdfa6f95740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x556d73b7f5d0 con 0x556d73b80230 2024-04-25T08:57:07.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.173+0000 7fdfa6f95740 10 client.0 _ll_register_callbacks cb 0x556d73b50c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:07.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.173+0000 7fdfa2839640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d73bab670 0x556d73b9a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:07.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.173+0000 7fdfa2839640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d73bab670 0x556d73b9a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39754/0 (socket says 172.21.15.43:39754) 2024-04-25T08:57:07.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.173+0000 7fdfa2839640 1 -- 172.21.15.43:0/1820323839 learned_addr learned my addr 172.21.15.43:0/1820323839 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:07.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.173+0000 7fdfa2038640 1 --2- 172.21.15.43:0/1820323839 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556d73b80230 0x556d73b9e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:07.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.173+0000 7fdfa1837640 1 --2- 172.21.15.43:0/1820323839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d73b9dad0 0x556d73b9e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:07.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.174+0000 7fdfa2839640 1 -- 172.21.15.43:0/1820323839 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556d73b80230 msgr2=0x556d73b9e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:07.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.174+0000 7fdfa2839640 1 --2- 172.21.15.43:0/1820323839 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556d73b80230 0x556d73b9e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:07.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.174+0000 7fdfa2839640 1 -- 172.21.15.43:0/1820323839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d73b9dad0 msgr2=0x556d73b9e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:07.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.174+0000 7fdfa2839640 1 --2- 172.21.15.43:0/1820323839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d73b9dad0 0x556d73b9e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:07.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.174+0000 7fdfa2839640 1 -- 172.21.15.43:0/1820323839 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556d73b81d50 con 0x556d73bab670 2024-04-25T08:57:07.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.174+0000 7fdfa2839640 1 --2- 172.21.15.43:0/1820323839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d73bab670 0x556d73b9a3c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c089650 tx=0x7fdf8c059940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e7595817980ea04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:07.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.174+0000 7fdf817fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:57:07.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.174+0000 7fdf817fa640 1 -- 172.21.15.43:0/1820323839 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8c061b00 con 0x556d73bab670 2024-04-25T08:57:07.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.174+0000 7fdf817fa640 1 -- 172.21.15.43:0/1820323839 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdf8c061ca0 con 0x556d73bab670 2024-04-25T08:57:07.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.174+0000 7fdfa6f95740 1 -- 172.21.15.43:0/1820323839 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf9401c070 con 0x556d73bab670 2024-04-25T08:57:07.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.174+0000 7fdf817fa640 1 -- 172.21.15.43:0/1820323839 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf8c05abb0 con 0x556d73bab670 2024-04-25T08:57:07.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.174+0000 7fdfa6f95740 10 client.5844 fetch_fsmap learned FSMap version 21 2024-04-25T08:57:07.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.174+0000 7fdfa6f95740 1 -- 172.21.15.43:0/1820323839 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x556d73b55940 con 0x556d73bab670 2024-04-25T08:57:07.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.174+0000 7fdf817fa640 1 -- 172.21.15.43:0/1820323839 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf9401c070 con 0x556d73bab670 2024-04-25T08:57:07.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.174+0000 7fdf817fa640 1 -- 172.21.15.43:0/1820323839 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdf8c0890d0 con 0x556d73bab670 2024-04-25T08:57:07.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.174+0000 7fdfa6f95740 10 client.5844 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T08:57:07.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.174+0000 7fdfa6f95740 10 client.5844 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T08:57:07.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.174+0000 7fdfa6f95740 20 client.5844 populate_metadata read hostname 'smithi043' 2024-04-25T08:57:07.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.174+0000 7fdfa6f95740 1 --2- 172.21.15.43:0/1820323839 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x556d73b9af00 0x556d73b548c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:07.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.174+0000 7fdfa6f95740 4 client.5844 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:57:07.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.174+0000 7fdfa6f95740 1 -- 172.21.15.43:0/1820323839 --> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556d73ac5f50 con 0x556d73b9af00 2024-04-25T08:57:07.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.175+0000 7fdfa2038640 1 --2- 172.21.15.43:0/1820323839 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x556d73b9af00 0x556d73b548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:07.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.175+0000 7fdfa2038640 1 --2- 172.21.15.43:0/1820323839 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x556d73b9af00 0x556d73b548c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cbe1c1fb758f0cc server_cookie=c7d3bf461e1979b2 in_seq=0 out_seq=0 2024-04-25T08:57:07.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.175+0000 7fdf817fa640 10 client.5844 ms_handle_connect on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:07.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.175+0000 7fdf817fa640 1 -- 172.21.15.43:0/1820323839 <== mds.0 v2:172.21.15.193:6836/4164634380 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x556d73ac5f50 con 0x556d73b9af00 2024-04-25T08:57:07.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.175+0000 7fdf817fa640 10 client.5844 handle_command_reply: tid=0 2024-04-25T08:57:07.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.175+0000 7fdfa6f95740 1 client.5844 shutdown 2024-04-25T08:57:07.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.175+0000 7fdfa6f95740 1 -- 172.21.15.43:0/1820323839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d73bab670 msgr2=0x556d73b9a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:07.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.175+0000 7fdfa6f95740 1 --2- 172.21.15.43:0/1820323839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d73bab670 0x556d73b9a3c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c089650 tx=0x7fdf8c059940 comp rx=0 tx=0).stop 2024-04-25T08:57:07.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.175+0000 7fdfa6f95740 1 -- 172.21.15.43:0/1820323839 shutdown_connections 2024-04-25T08:57:07.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.175+0000 7fdfa6f95740 1 --2- 172.21.15.43:0/1820323839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556d73bab670 0x556d73b9a3c0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:07.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.175+0000 7fdfa6f95740 1 --2- 172.21.15.43:0/1820323839 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x556d73b9af00 0x556d73b548c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:07.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.175+0000 7fdfa6f95740 1 --2- 172.21.15.43:0/1820323839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556d73b9dad0 0x556d73b9e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:07.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.175+0000 7fdf817fa640 0 client.5844 ms_handle_reset on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:07.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.175+0000 7fdfa6f95740 1 --2- 172.21.15.43:0/1820323839 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556d73b80230 0x556d73b9e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:07.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.175+0000 7fdfa6f95740 1 -- 172.21.15.43:0/1820323839 >> 172.21.15.43:0/1820323839 conn(0x556d73b55f10 msgr2=0x556d73baaa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:07.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.176+0000 7fdfa6f95740 1 -- 172.21.15.43:0/1820323839 shutdown_connections 2024-04-25T08:57:07.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.176+0000 7fdfa6f95740 1 -- 172.21.15.43:0/1820323839 wait complete. 2024-04-25T08:57:07.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.176+0000 7fdfa6f95740 20 client.5844 trim_cache size 0 max 16384 2024-04-25T08:57:07.171 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T08:57:07.171 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "6c4bd50e-aa00-4a7f-8d09-352344ef5a4b", 2024-04-25T08:57:07.171 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-04-25T08:57:07.172 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 5709, 2024-04-25T08:57:07.172 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-04-25T08:57:07.172 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-04-25T08:57:07.172 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-04-25T08:57:07.172 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 4.673520431, 2024-04-25T08:57:07.172 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 21, 2024-04-25T08:57:07.172 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 46, 2024-04-25T08:57:07.172 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 46, 2024-04-25T08:57:07.172 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 9.7024413299999992 2024-04-25T08:57:07.172 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T08:57:07.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.176+0000 7fdfa429c640 1 -- 172.21.15.43:0/1767925764 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdf8806b4d0 msgr2=0x7fdf8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:07.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.176+0000 7fdfa429c640 1 --2- 172.21.15.43:0/1767925764 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdf8806b4d0 0x7fdf8806d990 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fdf940014f0 tx=0x7fdf94009040 comp rx=0 tx=0).stop 2024-04-25T08:57:07.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.176+0000 7fdfa429c640 1 -- 172.21.15.43:0/1767925764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf9c0b29c0 msgr2=0x7fdf9c158e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:07.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.176+0000 7fdfa429c640 1 --2- 172.21.15.43:0/1767925764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf9c0b29c0 0x7fdf9c158e60 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c072160 tx=0x7fdf8c00f580 comp rx=0 tx=0).stop 2024-04-25T08:57:07.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.176+0000 7fdfa429c640 1 -- 172.21.15.43:0/1767925764 shutdown_connections 2024-04-25T08:57:07.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.176+0000 7fdfa429c640 1 --2- 172.21.15.43:0/1767925764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf9c0b29c0 0x7fdf9c158e60 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:07.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.176+0000 7fdfa429c640 1 --2- 172.21.15.43:0/1767925764 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdf9c0c5e90 0x7fdf9c0b2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:07.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.176+0000 7fdfa429c640 1 --2- 172.21.15.43:0/1767925764 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdf8806b4d0 0x7fdf8806d990 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:07.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.176+0000 7fdfa429c640 1 --2- 172.21.15.43:0/1767925764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf84004800 0x7fdf9c1564c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:07.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.176+0000 7fdfa429c640 1 -- 172.21.15.43:0/1767925764 >> 172.21.15.43:0/1767925764 conn(0x7fdf9c0d1700 msgr2=0x7fdf9c15cac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:07.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.176+0000 7fdfa429c640 1 -- 172.21.15.43:0/1767925764 shutdown_connections 2024-04-25T08:57:07.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.176+0000 7fdfa429c640 1 -- 172.21.15.43:0/1767925764 wait complete. 2024-04-25T08:57:07.177 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:57:07.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.288+0000 7f146a2f5640 1 Processor -- start 2024-04-25T08:57:07.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.288+0000 7f146a2f5640 1 -- start start 2024-04-25T08:57:07.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.289+0000 7f146a2f5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f146415ef50 0x7f14640d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:07.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.289+0000 7f146a2f5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14640d2920 con 0x7f14640d0920 2024-04-25T08:57:07.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.289+0000 7f146a2f5640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14640cee80 con 0x7f146415dce0 2024-04-25T08:57:07.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.289+0000 7f146a2f5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14640cec60 con 0x7f146415ef50 2024-04-25T08:57:07.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.289+0000 7f1468892640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f14640d0920 0x7f1464160980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48126/0 (socket says 172.21.15.43:48126) 2024-04-25T08:57:07.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.289+0000 7f1468892640 1 -- 172.21.15.43:0/1541390849 learned_addr learned my addr 172.21.15.43:0/1541390849 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:07.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.289+0000 7f14637fe640 1 --2- 172.21.15.43:0/1541390849 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f146415ef50 0x7f14640d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:07.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.289+0000 7f1462ffd640 1 -- 172.21.15.43:0/1541390849 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 7809377 0 0) 0x7f14640d2920 con 0x7f14640d0920 2024-04-25T08:57:07.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.289+0000 7f1462ffd640 1 -- 172.21.15.43:0/1541390849 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f144c003520 con 0x7f14640d0920 2024-04-25T08:57:07.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.290+0000 7f1462ffd640 1 -- 172.21.15.43:0/1541390849 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 504684762 0 0) 0x7f14640cee80 con 0x7f146415dce0 2024-04-25T08:57:07.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.290+0000 7f1462ffd640 1 -- 172.21.15.43:0/1541390849 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14640d2920 con 0x7f146415dce0 2024-04-25T08:57:07.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.290+0000 7f1462ffd640 1 -- 172.21.15.43:0/1541390849 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 959040328 0 0) 0x7f144c003520 con 0x7f14640d0920 2024-04-25T08:57:07.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.290+0000 7f14637fe640 1 -- 172.21.15.43:0/1541390849 >> v1:172.21.15.193:6789/0 conn(0x7f146415dce0 legacy=0x7f14640ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:07.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.290+0000 7f14637fe640 1 -- 172.21.15.43:0/1541390849 >> v1:172.21.15.43:6789/0 conn(0x7f14640d0920 legacy=0x7f1464160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:07.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.290+0000 7f14637fe640 1 -- 172.21.15.43:0/1541390849 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14640cf160 con 0x7f146415ef50 2024-04-25T08:57:07.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.290+0000 7f14637fe640 1 --2- 172.21.15.43:0/1541390849 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f146415ef50 0x7f14640d03e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f14580097d0 tx=0x7f145802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f98387c06e86b8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:07.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.291+0000 7f1462ffd640 1 -- 172.21.15.43:0/1541390849 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1458037070 con 0x7f146415ef50 2024-04-25T08:57:07.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.291+0000 7f1462ffd640 1 -- 172.21.15.43:0/1541390849 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f145802ab90 con 0x7f146415ef50 2024-04-25T08:57:07.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.291+0000 7f1462ffd640 1 -- 172.21.15.43:0/1541390849 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f145c0019f0 con 0x7f146415ef50 2024-04-25T08:57:07.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.291+0000 7f146a2f5640 1 -- 172.21.15.43:0/1541390849 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f146415ef50 msgr2=0x7f14640d03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:07.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.291+0000 7f146a2f5640 1 --2- 172.21.15.43:0/1541390849 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f146415ef50 0x7f14640d03e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f14580097d0 tx=0x7f145802a100 comp rx=0 tx=0).stop 2024-04-25T08:57:07.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.291+0000 7f146a2f5640 1 -- 172.21.15.43:0/1541390849 shutdown_connections 2024-04-25T08:57:07.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.291+0000 7f146a2f5640 1 --2- 172.21.15.43:0/1541390849 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f146415ef50 0x7f14640d03e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:07.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.291+0000 7f146a2f5640 1 -- 172.21.15.43:0/1541390849 >> 172.21.15.43:0/1541390849 conn(0x7f146409cd00 msgr2=0x7f14640d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:07.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.291+0000 7f146a2f5640 1 -- 172.21.15.43:0/1541390849 shutdown_connections 2024-04-25T08:57:07.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.291+0000 7f146a2f5640 1 -- 172.21.15.43:0/1541390849 wait complete. 2024-04-25T08:57:07.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.292+0000 7f146a2f5640 1 Processor -- start 2024-04-25T08:57:07.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.293+0000 7f146a2f5640 1 -- start start 2024-04-25T08:57:07.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.293+0000 7f146a2f5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14640d0920 0x7f1464155f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:07.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.293+0000 7f146a2f5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f146415dce0 0x7f14640cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:07.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.293+0000 7f146a2f5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f146415ef50 0x7f14640cc030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:07.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.293+0000 7f146a2f5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14640cf720 con 0x7f14640d0920 2024-04-25T08:57:07.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.293+0000 7f146a2f5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14640d0d70 con 0x7f146415ef50 2024-04-25T08:57:07.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.293+0000 7f146a2f5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f146415f6b0 con 0x7f146415dce0 2024-04-25T08:57:07.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.293+0000 7f1463fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14640d0920 0x7f1464155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:07.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.294+0000 7f1463fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14640d0920 0x7f1464155f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39768/0 (socket says 172.21.15.43:39768) 2024-04-25T08:57:07.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.294+0000 7f1463fff640 1 -- 172.21.15.43:0/3239793258 learned_addr learned my addr 172.21.15.43:0/3239793258 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:07.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.294+0000 7f1468892640 1 --2- 172.21.15.43:0/3239793258 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f146415ef50 0x7f14640cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:07.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.294+0000 7f14637fe640 1 --2- 172.21.15.43:0/3239793258 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f146415dce0 0x7f14640cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:07.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.294+0000 7f1463fff640 1 -- 172.21.15.43:0/3239793258 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f146415dce0 msgr2=0x7f14640cbaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:07.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.294+0000 7f1463fff640 1 --2- 172.21.15.43:0/3239793258 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f146415dce0 0x7f14640cbaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:07.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.294+0000 7f1463fff640 1 -- 172.21.15.43:0/3239793258 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f146415ef50 msgr2=0x7f14640cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:07.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.294+0000 7f1463fff640 1 --2- 172.21.15.43:0/3239793258 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f146415ef50 0x7f14640cc030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:07.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.294+0000 7f1463fff640 1 -- 172.21.15.43:0/3239793258 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14640a0f50 con 0x7f14640d0920 2024-04-25T08:57:07.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.294+0000 7f1463fff640 1 --2- 172.21.15.43:0/3239793258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14640d0920 0x7f1464155f20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f145400c500 tx=0x7f145400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:07.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.294+0000 7f1460ff9640 1 -- 172.21.15.43:0/3239793258 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14540052d0 con 0x7f14640d0920 2024-04-25T08:57:07.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.294+0000 7f1460ff9640 1 -- 172.21.15.43:0/3239793258 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1454005490 con 0x7f14640d0920 2024-04-25T08:57:07.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.294+0000 7f146a2f5640 1 -- 172.21.15.43:0/3239793258 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14640cc660 con 0x7f14640d0920 2024-04-25T08:57:07.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.294+0000 7f1460ff9640 1 -- 172.21.15.43:0/3239793258 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1454012870 con 0x7f14640d0920 2024-04-25T08:57:07.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.294+0000 7f146a2f5640 1 -- 172.21.15.43:0/3239793258 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14640ccc20 con 0x7f14640d0920 2024-04-25T08:57:07.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.295+0000 7f146a2f5640 1 -- 172.21.15.43:0/3239793258 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14640680f0 con 0x7f14640d0920 2024-04-25T08:57:07.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.296+0000 7f1460ff9640 1 -- 172.21.15.43:0/3239793258 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1454012a10 con 0x7f14640d0920 2024-04-25T08:57:07.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.296+0000 7f1460ff9640 1 --2- 172.21.15.43:0/3239793258 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f145006b4d0 0x7f145006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:07.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.296+0000 7f1460ff9640 1 -- 172.21.15.43:0/3239793258 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1454089da0 con 0x7f14640d0920 2024-04-25T08:57:07.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.297+0000 7f14637fe640 1 --2- 172.21.15.43:0/3239793258 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f145006b4d0 0x7f145006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:07.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.297+0000 7f14637fe640 1 --2- 172.21.15.43:0/3239793258 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f145006b4d0 0x7f145006d990 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1458034000 tx=0x7f145802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:07.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.300+0000 7f1460ff9640 1 -- 172.21.15.43:0/3239793258 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f14540586a0 con 0x7f14640d0920 2024-04-25T08:57:07.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.517+0000 7f146a2f5640 1 -- 172.21.15.43:0/3239793258 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f146409d1a0 con 0x7f14640d0920 2024-04-25T08:57:07.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.517+0000 7f1460ff9640 1 -- 172.21.15.43:0/3239793258 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v21) v1 ==== 53+0+73 (secure 0 0 0) 0x7f145405c740 con 0x7f14640d0920 2024-04-25T08:57:07.507 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:57:07.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.519+0000 7f146a2f5640 1 -- 172.21.15.43:0/3239793258 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f145006b4d0 msgr2=0x7f145006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:07.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.519+0000 7f146a2f5640 1 --2- 172.21.15.43:0/3239793258 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f145006b4d0 0x7f145006d990 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f1458034000 tx=0x7f145802e000 comp rx=0 tx=0).stop 2024-04-25T08:57:07.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.519+0000 7f146a2f5640 1 -- 172.21.15.43:0/3239793258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14640d0920 msgr2=0x7f1464155f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:07.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.519+0000 7f146a2f5640 1 --2- 172.21.15.43:0/3239793258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14640d0920 0x7f1464155f20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f145400c500 tx=0x7f145400c9c0 comp rx=0 tx=0).stop 2024-04-25T08:57:07.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.519+0000 7f146a2f5640 1 -- 172.21.15.43:0/3239793258 shutdown_connections 2024-04-25T08:57:07.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.519+0000 7f146a2f5640 1 --2- 172.21.15.43:0/3239793258 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f145006b4d0 0x7f145006d990 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:07.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.519+0000 7f146a2f5640 1 --2- 172.21.15.43:0/3239793258 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f146415ef50 0x7f14640cc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:07.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.519+0000 7f146a2f5640 1 --2- 172.21.15.43:0/3239793258 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f146415dce0 0x7f14640cbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:07.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.519+0000 7f146a2f5640 1 --2- 172.21.15.43:0/3239793258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f14640d0920 0x7f1464155f20 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:07.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.519+0000 7f146a2f5640 1 -- 172.21.15.43:0/3239793258 >> 172.21.15.43:0/3239793258 conn(0x7f146409cd00 msgr2=0x7f14640c7570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:07.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.520+0000 7f146a2f5640 1 -- 172.21.15.43:0/3239793258 shutdown_connections 2024-04-25T08:57:07.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:07.520+0000 7f146a2f5640 1 -- 172.21.15.43:0/3239793258 wait complete. 2024-04-25T08:57:07.521 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:57:07.521 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:57:07.521 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-04-25T08:57:07.521 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:57:07.521 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:57:07.521 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:57:07.521 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:57:07.521 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:57:07.521 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:57:07.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:07.521 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-04-25T08:57:07.537 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:57:07.538 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:57:07.538 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:57:07.538 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:57:07.538 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-04-25T08:57:07.538 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:57:07.538 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:57:07.538 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-04-25T08:57:07.538 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-04-25T08:57:07.538 INFO:teuthology.orchestra.run.smithi043.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:57:07.538 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:57:07.538 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-04-25T08:57:07.538 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T08:57:07.539 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 42590sec preferred_lft 42590sec 2024-04-25T08:57:07.539 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link noprefixroute 2024-04-25T08:57:07.539 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:57:07.539 INFO:teuthology.orchestra.run.smithi043.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:57:07.539 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-04-25T08:57:07.539 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-04-25T08:57:07.539 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:57:07.539 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:57:07.539 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-04-25T08:57:07.539 INFO:teuthology.orchestra.run.smithi043.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:57:07.539 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 12:2c:7a:72:36:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:57:07.539 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:57:07.539 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:57:07.539 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ccff:4eff:fe9d:c8bf/64 scope link 2024-04-25T08:57:07.540 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:57:07.540 INFO:teuthology.orchestra.run.smithi043.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:57:07.540 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 16:9a:63:37:46:24 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:57:07.540 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::149a:63ff:fe37:4624/64 scope link 2024-04-25T08:57:07.540 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:57:07.540 INFO:teuthology.orchestra.run.smithi043.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:57:07.540 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 12:2c:7a:72:36:14 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:57:07.540 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::102c:7aff:fe72:3614/64 scope link 2024-04-25T08:57:07.540 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:57:07.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:07.540 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:57:07.601 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-04-25T08:57:07.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:07.603 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-04-25T08:57:07.659 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-04-25T08:57:07.659 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:57:07.659 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T08:57:07.659 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T08:57:07.659 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:57:07.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:07.660 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2024-04-25T08:57:07.660 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T08:57:07.660 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:57:07.660 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:57:07.660 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:57:07.660 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T08:57:07.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:07.746 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-04-25T08:57:07.761 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:57:07.761 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:57:07.761 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:57:07.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:07.761 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-04-25T08:57:07.826 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T08:57:07.826 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:57:07.826 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:57:07.881 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:57:07.881 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:57:07.881 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:57:07.936 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-04-25T08:57:08.000 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:57:08.055 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-04-25T08:57:08.056 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-04-25T08:57:08.056 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-04-25T08:57:08.056 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-04-25T08:57:08.056 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-04-25T08:57:08.056 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-04-25T08:57:08.056 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-04-25T08:57:08.056 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-04-25T08:57:08.056 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-04-25T08:57:08.056 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-04-25T08:57:08.056 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-04-25T08:57:08.056 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/selinux 2024-04-25T08:57:08.056 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:57:08.056 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-04-25T08:57:08.057 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-04-25T08:57:08.057 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-04-25T08:57:08.057 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-04-25T08:57:08.057 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:57:08.057 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-04-25T08:57:08.057 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-04-25T08:57:08.057 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:57:08.057 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:57:08.057 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:57:08.057 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:57:08.057 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-04-25T08:57:08.057 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-04-25T08:57:08.057 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:57:08.057 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:57:08.058 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:57:08.058 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:57:08.058 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-04-25T08:57:08.058 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:57:08.058 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:57:08.058 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:57:08.058 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:57:08.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:08.059 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-04-25T08:57:08.114 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T08:57:08.114 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T08:57:08.156 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-04-25T08:57:08.180 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:57:08.237 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-04-25T08:57:08.237 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-04-25T08:57:08.237 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-04-25T08:57:08.237 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-04-25T08:57:08.237 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-04-25T08:57:08.237 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-04-25T08:57:08.237 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-04-25T08:57:08.237 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-04-25T08:57:08.237 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-04-25T08:57:08.237 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-04-25T08:57:08.237 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-04-25T08:57:08.237 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/selinux 2024-04-25T08:57:08.238 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:57:08.238 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-04-25T08:57:08.238 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-04-25T08:57:08.238 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-04-25T08:57:08.238 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-04-25T08:57:08.238 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:57:08.238 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-04-25T08:57:08.238 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-04-25T08:57:08.238 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:57:08.238 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:57:08.238 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:57:08.238 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:57:08.238 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-04-25T08:57:08.239 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-04-25T08:57:08.239 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:57:08.239 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:57:08.239 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:57:08.239 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:57:08.239 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-04-25T08:57:08.239 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:57:08.239 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:57:08.239 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:57:08.239 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:57:08.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:08.239 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-04-25T08:57:08.240 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:2024-04-25T08:57:08.248+0000 7fba3e827480 -1 init, newargv = 0x56082da638d0 newargc=15 2024-04-25T08:57:08.240 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[57915]: starting ceph client 2024-04-25T08:57:08.242 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[57915]: starting fuse 2024-04-25T08:57:08.255 INFO:teuthology.orchestra.run.smithi043.stdout:43 2024-04-25T08:57:08.255 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-25T08:57:08.255 DEBUG:teuthology.orchestra.run.smithi043:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:57:08.255 DEBUG:teuthology.orchestra.run.smithi043:> import glob 2024-04-25T08:57:08.256 DEBUG:teuthology.orchestra.run.smithi043:> import re 2024-04-25T08:57:08.256 DEBUG:teuthology.orchestra.run.smithi043:> import os 2024-04-25T08:57:08.256 DEBUG:teuthology.orchestra.run.smithi043:> import subprocess 2024-04-25T08:57:08.256 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T08:57:08.256 DEBUG:teuthology.orchestra.run.smithi043:> def _find_admin_socket(client_name): 2024-04-25T08:57:08.256 DEBUG:teuthology.orchestra.run.smithi043:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T08:57:08.256 DEBUG:teuthology.orchestra.run.smithi043:> files = glob.glob(asok_path) 2024-04-25T08:57:08.256 DEBUG:teuthology.orchestra.run.smithi043:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T08:57:08.256 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T08:57:08.256 DEBUG:teuthology.orchestra.run.smithi043:> # Given a non-glob path, it better be there 2024-04-25T08:57:08.256 DEBUG:teuthology.orchestra.run.smithi043:> if "*" not in asok_path: 2024-04-25T08:57:08.256 DEBUG:teuthology.orchestra.run.smithi043:> assert(len(files) == 1) 2024-04-25T08:57:08.256 DEBUG:teuthology.orchestra.run.smithi043:> return files[0] 2024-04-25T08:57:08.257 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T08:57:08.257 DEBUG:teuthology.orchestra.run.smithi043:> for f in files: 2024-04-25T08:57:08.257 DEBUG:teuthology.orchestra.run.smithi043:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:57:08.257 DEBUG:teuthology.orchestra.run.smithi043:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:57:08.257 DEBUG:teuthology.orchestra.run.smithi043:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:57:08.257 DEBUG:teuthology.orchestra.run.smithi043:> contents = proc_f.read() 2024-04-25T08:57:08.257 DEBUG:teuthology.orchestra.run.smithi043:> if mountpoint in contents: 2024-04-25T08:57:08.257 DEBUG:teuthology.orchestra.run.smithi043:> return f 2024-04-25T08:57:08.257 DEBUG:teuthology.orchestra.run.smithi043:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:57:08.257 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T08:57:08.257 DEBUG:teuthology.orchestra.run.smithi043:> print(_find_admin_socket("client.0")) 2024-04-25T08:57:08.257 DEBUG:teuthology.orchestra.run.smithi043:> ' 2024-04-25T08:57:08.374 INFO:teuthology.orchestra.run.smithi043.stdout:/var/run/ceph/ceph-client.0.57915.asok 2024-04-25T08:57:08.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:57:08.385 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.57915.asok 2024-04-25T08:57:08.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:08.385 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.57915.asok status 2024-04-25T08:57:08.522 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T08:57:08.522 INFO:teuthology.orchestra.run.smithi043.stdout: "metadata": { 2024-04-25T08:57:08.522 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:57:08.522 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:57:08.522 INFO:teuthology.orchestra.run.smithi043.stdout: "entity_id": "0", 2024-04-25T08:57:08.522 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": "smithi043", 2024-04-25T08:57:08.522 INFO:teuthology.orchestra.run.smithi043.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T08:57:08.522 INFO:teuthology.orchestra.run.smithi043.stdout: "pid": "57915", 2024-04-25T08:57:08.522 INFO:teuthology.orchestra.run.smithi043.stdout: "root": "/" 2024-04-25T08:57:08.522 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:57:08.522 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_count": 0, 2024-04-25T08:57:08.523 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_pinned_count": 0, 2024-04-25T08:57:08.523 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 5856, 2024-04-25T08:57:08.523 INFO:teuthology.orchestra.run.smithi043.stdout: "inst": { 2024-04-25T08:57:08.523 INFO:teuthology.orchestra.run.smithi043.stdout: "name": { 2024-04-25T08:57:08.523 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "client", 2024-04-25T08:57:08.523 INFO:teuthology.orchestra.run.smithi043.stdout: "num": 5856 2024-04-25T08:57:08.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:57:08.523 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-04-25T08:57:08.523 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-04-25T08:57:08.523 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:57:08.523 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 2424235321 2024-04-25T08:57:08.523 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:57:08.523 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:57:08.524 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-04-25T08:57:08.524 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-04-25T08:57:08.524 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:57:08.524 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 2424235321 2024-04-25T08:57:08.524 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:57:08.524 INFO:teuthology.orchestra.run.smithi043.stdout: "inst_str": "client.5856 192.168.0.1:0/2424235321", 2024-04-25T08:57:08.524 INFO:teuthology.orchestra.run.smithi043.stdout: "addr_str": "192.168.0.1:0/2424235321", 2024-04-25T08:57:08.524 INFO:teuthology.orchestra.run.smithi043.stdout: "inode_count": 1, 2024-04-25T08:57:08.524 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_epoch": 21, 2024-04-25T08:57:08.524 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch": 46, 2024-04-25T08:57:08.524 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:57:08.524 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklisted": false, 2024-04-25T08:57:08.524 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs" 2024-04-25T08:57:08.524 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T08:57:08.530 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:08.531 DEBUG:teuthology.orchestra.run.smithi043:> stat --file-system '--printf=%T 2024-04-25T08:57:08.531 DEBUG:teuthology.orchestra.run.smithi043:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T08:57:08.546 INFO:teuthology.orchestra.run.smithi043.stdout:fuseblk 2024-04-25T08:57:08.547 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T08:57:08.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:08.547 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:57:08.612 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2024-04-25T08:57:08.612 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_test_path 2024-04-25T08:57:08.612 INFO:tasks.cephfs.test_scrub_checks:Cloning repo into place 2024-04-25T08:57:08.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:08.613 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p /home/ubuntu/cephtest/mnt.0/scrub_test_path) 2024-04-25T08:57:08.674 DEBUG:teuthology.orchestra.run.smithi043:> stdin-killer -- python3 -c ' 2024-04-25T08:57:08.675 DEBUG:teuthology.orchestra.run.smithi043:> import os 2024-04-25T08:57:08.675 DEBUG:teuthology.orchestra.run.smithi043:> import stat 2024-04-25T08:57:08.675 DEBUG:teuthology.orchestra.run.smithi043:> import json 2024-04-25T08:57:08.675 DEBUG:teuthology.orchestra.run.smithi043:> import sys 2024-04-25T08:57:08.675 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T08:57:08.675 DEBUG:teuthology.orchestra.run.smithi043:> try: 2024-04-25T08:57:08.675 DEBUG:teuthology.orchestra.run.smithi043:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'"'"') 2024-04-25T08:57:08.675 DEBUG:teuthology.orchestra.run.smithi043:> except OSError as e: 2024-04-25T08:57:08.675 DEBUG:teuthology.orchestra.run.smithi043:> sys.exit(e.errno) 2024-04-25T08:57:08.675 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T08:57:08.675 DEBUG:teuthology.orchestra.run.smithi043:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T08:57:08.675 DEBUG:teuthology.orchestra.run.smithi043:> print(json.dumps( 2024-04-25T08:57:08.675 DEBUG:teuthology.orchestra.run.smithi043:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T08:57:08.675 DEBUG:teuthology.orchestra.run.smithi043:> indent=2)) 2024-04-25T08:57:08.675 DEBUG:teuthology.orchestra.run.smithi043:> ' 2024-04-25T08:57:08.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:08 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-04-25T08:57:08.791 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T08:57:08.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:08.792 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone --branch giant http://github.com/ceph/ceph-qa-suite /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite) 2024-04-25T08:57:08.854 INFO:teuthology.orchestra.run.smithi043.stderr:Cloning into '/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'... 2024-04-25T08:57:09.098 INFO:teuthology.orchestra.run.smithi043.stderr:warning: redirecting to https://github.com/ceph/ceph-qa-suite/ 2024-04-25T08:57:14.350 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:57:14.352 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:57:17.215 INFO:teuthology.orchestra.run.smithi043.stderr:Updating files: 20% (288/1398) Updating files: 21% (294/1398) Updating files: 22% (308/1398) Updating files: 23% (322/1398) Updating files: 24% (336/1398) Updating files: 25% (350/1398) Updating files: 26% (364/1398) Updating files: 27% (378/1398) Updating files: 28% (392/1398) Updating files: 29% (406/1398) Updating files: 30% (420/1398) Updating files: 31% (434/1398) Updating files: 32% (448/1398) Updating files: 33% (462/1398) Updating files: 34% (476/1398) Updating files: 35% (490/1398) Updating files: 36% (504/1398) Updating files: 37% (518/1398) Updating files: 38% (532/1398) Updating files: 39% (546/1398) Updating files: 40% (560/1398) Updating files: 41% (574/1398) Updating files: 42% (588/1398) Updating files: 43% (602/1398) Updating files: 43% (610/1398) Updating files: 44% (616/1398) Updating files: 45% (630/1398) Updating files: 46% (644/1398) Updating files: 47% (658/1398) Updating files: 48% (672/1398) Updating files: 49% (686/1398) Updating files: 50% (699/1398) Updating files: 51% (713/1398) Updating files: 52% (727/1398) Updating files: 53% (741/1398) Updating files: 54% (755/1398) Updating files: 55% (769/1398) Updating files: 56% (783/1398) Updating files: 57% (797/1398) Updating files: 58% (811/1398) Updating files: 59% (825/1398) Updating files: 59% (833/1398) Updating files: 60% (839/1398) Updating files: 61% (853/1398) Updating files: 62% (867/1398) Updating files: 63% (881/1398) Updating files: 64% (895/1398) Updating files: 65% (909/1398) Updating files: 66% (923/1398) Updating files: 67% (937/1398) Updating files: 68% (951/1398) Updating files: 69% (965/1398) Updating files: 70% (979/1398) Updating files: 71% (993/1398) Updating files: 72% (1007/1398) Updating files: 73% (1021/1398) Updating files: 74% (1035/1398) Updating files: 75% (1049/1398) Updating files: 76% (1063/1398) Updating files: 76% (1072/1398) Updating files: 77% (1077/1398) Updating files: 78% (1091/1398) Updating files: 79% (1105/1398) Updating files: 80% (1119/1398) Updating files: 81% (1133/1398) Updating files: 82% (1147/1398) Updating files: 83% (1161/1398) Updating files: 84% (1175/1398) Updating files: 85% (1189/1398) Updating files: 86% (1203/1398) Updating files: 87% (1217/1398) Updating files: 88% (1231/1398) Updating files: 89% (1245/1398) Updating files: 90% (1259/1398) Updating files: 91% (1273/1398) Updating files: 92% (1287/1398) Updating files: 92% (1294/1398) Updating files: 93% (1301/1398) Updating files: 94% (1315/1398) Updating files: 95% (1329/1398) Updating files: 96% (1343/1398) Updating files: 97% (1357/1398) Updating files: 98% (1371/1398) Updating files: 99% (1385/1398) Updating files: 100% (1398/1398) Updating files: 100% (1398/1398), done. 2024-04-25T08:57:17.223 INFO:tasks.cephfs.test_scrub_checks:Initiating mds_scrub_checks on mds.0 test_path /scrub_test_path, run_seq 0 2024-04-25T08:57:17.223 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/i/dont/exist' 2024-04-25T08:57:17.223 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:57:17.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.374+0000 7fc86fbd5640 1 Processor -- start 2024-04-25T08:57:17.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.374+0000 7fc86fbd5640 1 -- start start 2024-04-25T08:57:17.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.375+0000 7fc86fbd5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc86815a9d0 0x7fc86806d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:17.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.375+0000 7fc86fbd5640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fc86806d7d0 0x7fc868071a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:17.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.375+0000 7fc86fbd5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc868072400 con 0x7fc868071f90 2024-04-25T08:57:17.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.375+0000 7fc86fbd5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc86806dc30 con 0x7fc86815a9d0 2024-04-25T08:57:17.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.375+0000 7fc86fbd5640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8680704f0 con 0x7fc86806d7d0 2024-04-25T08:57:17.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.375+0000 7fc86d971640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc86815a9d0 0x7fc86806d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:17.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.375+0000 7fc86e172640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc868071f90 0x7fc8680b3990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43300/0 (socket says 172.21.15.43:43300) 2024-04-25T08:57:17.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.375+0000 7fc86d971640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc86815a9d0 0x7fc86806d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44158/0 (socket says 172.21.15.43:44158) 2024-04-25T08:57:17.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.375+0000 7fc86d971640 1 -- 172.21.15.43:0/824323845 learned_addr learned my addr 172.21.15.43:0/824323845 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:17.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.376+0000 7fc86d971640 1 -- 172.21.15.43:0/824323845 >> v2:172.21.15.193:3300/0 conn(0x7fc86806d7d0 msgr2=0x7fc868071a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:17.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.376+0000 7fc86d170640 1 --2- 172.21.15.43:0/824323845 >> v2:172.21.15.193:3300/0 conn(0x7fc86806d7d0 0x7fc868071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:17.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.376+0000 7fc86d971640 1 --2- 172.21.15.43:0/824323845 >> v2:172.21.15.193:3300/0 conn(0x7fc86806d7d0 0x7fc868071a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.376+0000 7fc86d971640 1 -- 172.21.15.43:0/824323845 >> v1:172.21.15.43:6789/0 conn(0x7fc868071f90 legacy=0x7fc8680b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:17.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.376+0000 7fc86c96f640 1 -- 172.21.15.43:0/824323845 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2009363423 0 0) 0x7fc868072400 con 0x7fc868071f90 2024-04-25T08:57:17.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.376+0000 7fc86d971640 1 -- 172.21.15.43:0/824323845 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8680707d0 con 0x7fc86815a9d0 2024-04-25T08:57:17.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.376+0000 7fc86d170640 1 --2- 172.21.15.43:0/824323845 >> v2:172.21.15.193:3300/0 conn(0x7fc86806d7d0 0x7fc868071a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:17.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.376+0000 7fc86d971640 1 --2- 172.21.15.43:0/824323845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc86815a9d0 0x7fc86806d290 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fc86806baf0 tx=0x7fc86002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1de2c179acb88ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:17.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.376+0000 7fc86c96f640 1 -- 172.21.15.43:0/824323845 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc860036070 con 0x7fc86815a9d0 2024-04-25T08:57:17.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.377+0000 7fc86c96f640 1 -- 172.21.15.43:0/824323845 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc860031600 con 0x7fc86815a9d0 2024-04-25T08:57:17.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.377+0000 7fc86c96f640 1 -- 172.21.15.43:0/824323845 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8600319d0 con 0x7fc86815a9d0 2024-04-25T08:57:17.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.377+0000 7fc86fbd5640 1 -- 172.21.15.43:0/824323845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc86815a9d0 msgr2=0x7fc86806d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:17.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.377+0000 7fc86fbd5640 1 --2- 172.21.15.43:0/824323845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc86815a9d0 0x7fc86806d290 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fc86806baf0 tx=0x7fc86002a730 comp rx=0 tx=0).stop 2024-04-25T08:57:17.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.377+0000 7fc86fbd5640 1 -- 172.21.15.43:0/824323845 shutdown_connections 2024-04-25T08:57:17.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.377+0000 7fc86fbd5640 1 --2- 172.21.15.43:0/824323845 >> v2:172.21.15.193:3300/0 conn(0x7fc86806d7d0 0x7fc868071a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.377+0000 7fc86fbd5640 1 --2- 172.21.15.43:0/824323845 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc86815a9d0 0x7fc86806d290 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.377+0000 7fc86fbd5640 1 -- 172.21.15.43:0/824323845 >> 172.21.15.43:0/824323845 conn(0x7fc8680b1480 msgr2=0x7fc8680b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:17.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.377+0000 7fc86fbd5640 1 -- 172.21.15.43:0/824323845 shutdown_connections 2024-04-25T08:57:17.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.377+0000 7fc86fbd5640 1 -- 172.21.15.43:0/824323845 wait complete. 2024-04-25T08:57:17.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.378+0000 7fc86fbd5640 1 Processor -- start 2024-04-25T08:57:17.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.378+0000 7fc86fbd5640 1 -- start start 2024-04-25T08:57:17.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.379+0000 7fc86fbd5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc86806d7d0 0x7fc868144090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:17.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.379+0000 7fc86fbd5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc868071f90 0x7fc8681e7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:17.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.379+0000 7fc86fbd5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8681e7bd0 0x7fc868144930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:17.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.379+0000 7fc86fbd5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8680712b0 con 0x7fc868071f90 2024-04-25T08:57:17.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.379+0000 7fc86fbd5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc868070d90 con 0x7fc8681e7bd0 2024-04-25T08:57:17.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.379+0000 7fc86fbd5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8680ce950 con 0x7fc86806d7d0 2024-04-25T08:57:17.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.379+0000 7fc86d170640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc868071f90 0x7fc8681e7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:17.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.379+0000 7fc86d170640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc868071f90 0x7fc8681e7690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44172/0 (socket says 172.21.15.43:44172) 2024-04-25T08:57:17.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.379+0000 7fc86d170640 1 -- 172.21.15.43:0/4114147290 learned_addr learned my addr 172.21.15.43:0/4114147290 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:17.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.380+0000 7fc86e172640 1 --2- 172.21.15.43:0/4114147290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8681e7bd0 0x7fc868144930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:17.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.380+0000 7fc86d971640 1 --2- 172.21.15.43:0/4114147290 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc86806d7d0 0x7fc868144090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:17.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.380+0000 7fc86d170640 1 -- 172.21.15.43:0/4114147290 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc86806d7d0 msgr2=0x7fc868144090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:17.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.380+0000 7fc86d170640 1 --2- 172.21.15.43:0/4114147290 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc86806d7d0 0x7fc868144090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.380+0000 7fc86d170640 1 -- 172.21.15.43:0/4114147290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8681e7bd0 msgr2=0x7fc868144930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:17.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.380+0000 7fc86d170640 1 --2- 172.21.15.43:0/4114147290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8681e7bd0 0x7fc868144930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.380+0000 7fc86d170640 1 -- 172.21.15.43:0/4114147290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8681e65f0 con 0x7fc868071f90 2024-04-25T08:57:17.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.380+0000 7fc86d170640 1 --2- 172.21.15.43:0/4114147290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc868071f90 0x7fc8681e7690 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc85c00c7f0 tx=0x7fc85c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:17.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.380+0000 7fc8667fc640 1 -- 172.21.15.43:0/4114147290 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c015070 con 0x7fc868071f90 2024-04-25T08:57:17.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.380+0000 7fc8667fc640 1 -- 172.21.15.43:0/4114147290 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc85c007820 con 0x7fc868071f90 2024-04-25T08:57:17.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.380+0000 7fc86fbd5640 1 -- 172.21.15.43:0/4114147290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8681e68d0 con 0x7fc868071f90 2024-04-25T08:57:17.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.380+0000 7fc8667fc640 1 -- 172.21.15.43:0/4114147290 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c007bd0 con 0x7fc868071f90 2024-04-25T08:57:17.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.380+0000 7fc86fbd5640 1 -- 172.21.15.43:0/4114147290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8681e6e90 con 0x7fc868071f90 2024-04-25T08:57:17.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.381+0000 7fc86fbd5640 1 -- 172.21.15.43:0/4114147290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc868067d30 con 0x7fc868071f90 2024-04-25T08:57:17.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.382+0000 7fc8667fc640 1 -- 172.21.15.43:0/4114147290 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc85c018070 con 0x7fc868071f90 2024-04-25T08:57:17.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.382+0000 7fc8667fc640 1 --2- 172.21.15.43:0/4114147290 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc85406b500 0x7fc85406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:17.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.382+0000 7fc8667fc640 1 -- 172.21.15.43:0/4114147290 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc85c08a310 con 0x7fc868071f90 2024-04-25T08:57:17.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.382+0000 7fc86d971640 1 --2- 172.21.15.43:0/4114147290 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc85406b500 0x7fc85406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:17.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.383+0000 7fc86d971640 1 --2- 172.21.15.43:0/4114147290 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc85406b500 0x7fc85406d9c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc860033000 tx=0x7fc860009000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:17.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.386+0000 7fc8667fc640 1 -- 172.21.15.43:0/4114147290 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc868067d30 con 0x7fc868071f90 2024-04-25T08:57:17.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.625+0000 7fc86fbd5640 1 -- 172.21.15.43:0/4114147290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc86806dfd0 con 0x7fc868071f90 2024-04-25T08:57:17.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.626+0000 7fc8667fc640 1 -- 172.21.15.43:0/4114147290 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fc86806dfd0 con 0x7fc868071f90 2024-04-25T08:57:17.615 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:57:17.616 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1504450503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1504450503},{"type":"v1","addr":"172.21.15.193:6835","nonce":1504450503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5691,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2657133821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2657133821},{"type":"v1","addr":"172.21.15.43:6834","nonce":2657133821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":5694,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/4271431818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":4271431818},{"type":"v1","addr":"172.21.15.43:6835","nonce":4271431818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:57:01.856132+0000","modified":"2024-04-25T08:57:03.857410+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5709":{"gid":5709,"name":"d","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.193:6837/4164634380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4164634380},{"type":"v1","addr":"172.21.15.193:6837","nonce":4164634380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:57:17.616 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 21 2024-04-25T08:57:17.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.628+0000 7fc86fbd5640 1 -- 172.21.15.43:0/4114147290 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc85406b500 msgr2=0x7fc85406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:17.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.628+0000 7fc86fbd5640 1 --2- 172.21.15.43:0/4114147290 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc85406b500 0x7fc85406d9c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc860033000 tx=0x7fc860009000 comp rx=0 tx=0).stop 2024-04-25T08:57:17.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.628+0000 7fc86fbd5640 1 -- 172.21.15.43:0/4114147290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc868071f90 msgr2=0x7fc8681e7690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:17.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.628+0000 7fc86fbd5640 1 --2- 172.21.15.43:0/4114147290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc868071f90 0x7fc8681e7690 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc85c00c7f0 tx=0x7fc85c00ccb0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.628+0000 7fc86fbd5640 1 -- 172.21.15.43:0/4114147290 shutdown_connections 2024-04-25T08:57:17.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.628+0000 7fc86fbd5640 1 --2- 172.21.15.43:0/4114147290 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc85406b500 0x7fc85406d9c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.628+0000 7fc86fbd5640 1 --2- 172.21.15.43:0/4114147290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc8681e7bd0 0x7fc868144930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.628+0000 7fc86fbd5640 1 --2- 172.21.15.43:0/4114147290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc868071f90 0x7fc8681e7690 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.628+0000 7fc86fbd5640 1 --2- 172.21.15.43:0/4114147290 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc86806d7d0 0x7fc868144090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.628+0000 7fc86fbd5640 1 -- 172.21.15.43:0/4114147290 >> 172.21.15.43:0/4114147290 conn(0x7fc8680b1480 msgr2=0x7fc8680716b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:17.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.628+0000 7fc86fbd5640 1 -- 172.21.15.43:0/4114147290 shutdown_connections 2024-04-25T08:57:17.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.628+0000 7fc86fbd5640 1 -- 172.21.15.43:0/4114147290 wait complete. 2024-04-25T08:57:17.628 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path /scrub_test_path/i/dont/exist 2024-04-25T08:57:17.731 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-04-25T08:57:17.732 INFO:teuthology.orchestra.run.smithi193.stdout: "return_code": -2 2024-04-25T08:57:17.732 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-04-25T08:57:17.740 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/i/dont/exist' got response code '0' and stdout '{ "return_code": -2 } ' 2024-04-25T08:57:17.741 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/i/dont/exist' 2024-04-25T08:57:17.741 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:57:17.741 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/i/dont/exist 2024-04-25T08:57:17.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.855+0000 7f6248e76640 1 Processor -- start 2024-04-25T08:57:17.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.856+0000 7f6248e76640 1 -- start start 2024-04-25T08:57:17.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.856+0000 7f6248e76640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f624415ddd0 0x7f62440b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:17.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.856+0000 7f6248e76640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f624406d900 con 0x7f624406d4f0 2024-04-25T08:57:17.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.857+0000 7f6248e76640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62440d2620 con 0x7f624406ef90 2024-04-25T08:57:17.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.857+0000 7f6248e76640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62440b0b80 con 0x7f624415ddd0 2024-04-25T08:57:17.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.857+0000 7f6242d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f624406d4f0 0x7f62440ccb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43304/0 (socket says 172.21.15.43:43304) 2024-04-25T08:57:17.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.857+0000 7f6242d9d640 1 -- 172.21.15.43:0/3339784049 learned_addr learned my addr 172.21.15.43:0/3339784049 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:17.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.857+0000 7f624259c640 1 --2- 172.21.15.43:0/3339784049 >> v2:172.21.15.193:3300/0 conn(0x7f624415ddd0 0x7f62440b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:17.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.857+0000 7f624159a640 1 -- 172.21.15.43:0/3339784049 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2323674752 0 0) 0x7f624406d900 con 0x7f624406d4f0 2024-04-25T08:57:17.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.857+0000 7f624159a640 1 -- 172.21.15.43:0/3339784049 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6230003520 con 0x7f624406d4f0 2024-04-25T08:57:17.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.857+0000 7f624159a640 1 -- 172.21.15.43:0/3339784049 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2867225037 0 0) 0x7f6230003520 con 0x7f624406d4f0 2024-04-25T08:57:17.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.857+0000 7f624159a640 1 -- 172.21.15.43:0/3339784049 >> v2:172.21.15.193:3300/0 conn(0x7f624415ddd0 msgr2=0x7f62440b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:17.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.857+0000 7f624159a640 1 --2- 172.21.15.43:0/3339784049 >> v2:172.21.15.193:3300/0 conn(0x7f624415ddd0 0x7f62440b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.857+0000 7f624159a640 1 -- 172.21.15.43:0/3339784049 >> v1:172.21.15.193:6789/0 conn(0x7f624406ef90 legacy=0x7f62440cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:17.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.857+0000 7f624159a640 1 -- 172.21.15.43:0/3339784049 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62440b0e60 con 0x7f624406d4f0 2024-04-25T08:57:17.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.857+0000 7f624159a640 1 -- 172.21.15.43:0/3339784049 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f623c0019f0 con 0x7f624406d4f0 2024-04-25T08:57:17.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.857+0000 7f624159a640 1 -- 172.21.15.43:0/3339784049 >> v1:172.21.15.43:6789/0 conn(0x7f624406d4f0 legacy=0x7f62440ccb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:17.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.857+0000 7f624159a640 1 --2- 172.21.15.43:0/3339784049 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62300039b0 0x7f6230003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:17.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.857+0000 7f624159a640 1 --2- 172.21.15.43:0/3339784049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6230004800 0x7f6230004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:17.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.857+0000 7f624159a640 1 --2- 172.21.15.43:0/3339784049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62300051c0 0x7f623000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:17.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.857+0000 7f624159a640 1 -- 172.21.15.43:0/3339784049 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62440b0b80 con 0x7f6230004800 2024-04-25T08:57:17.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.857+0000 7f624159a640 1 -- 172.21.15.43:0/3339784049 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623000c060 con 0x7f62300051c0 2024-04-25T08:57:17.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.857+0000 7f624159a640 1 -- 172.21.15.43:0/3339784049 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f623000c260 con 0x7f62300039b0 2024-04-25T08:57:17.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.857+0000 7f6241d9b640 1 --2- 172.21.15.43:0/3339784049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6230004800 0x7f6230004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:17.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.857+0000 7f624259c640 1 --2- 172.21.15.43:0/3339784049 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62300039b0 0x7f6230003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:17.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.857+0000 7f6242d9d640 1 --2- 172.21.15.43:0/3339784049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62300051c0 0x7f623000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:17.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.858+0000 7f6241d9b640 1 -- 172.21.15.43:0/3339784049 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62300039b0 msgr2=0x7f6230003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:17.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.858+0000 7f6241d9b640 1 --2- 172.21.15.43:0/3339784049 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62300039b0 0x7f6230003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.858+0000 7f6241d9b640 1 -- 172.21.15.43:0/3339784049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62300051c0 msgr2=0x7f623000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:17.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.858+0000 7f6241d9b640 1 --2- 172.21.15.43:0/3339784049 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62300051c0 0x7f623000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.858+0000 7f6241d9b640 1 -- 172.21.15.43:0/3339784049 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6230003520 con 0x7f6230004800 2024-04-25T08:57:17.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.858+0000 7f6242d9d640 1 -- 172.21.15.43:0/3339784049 reap_dead start 2024-04-25T08:57:17.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.858+0000 7f6241d9b640 1 --2- 172.21.15.43:0/3339784049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6230004800 0x7f6230004c80 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f622c001750 tx=0x7f6230005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=332d48c97756de8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:17.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.859+0000 7f624159a640 1 -- 172.21.15.43:0/3339784049 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c001370 con 0x7f6230004800 2024-04-25T08:57:17.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.859+0000 7f624159a640 1 -- 172.21.15.43:0/3339784049 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f622c0052d0 con 0x7f6230004800 2024-04-25T08:57:17.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.859+0000 7f624159a640 1 -- 172.21.15.43:0/3339784049 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c02abc0 con 0x7f6230004800 2024-04-25T08:57:17.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.859+0000 7f6248e76640 1 -- 172.21.15.43:0/3339784049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6230004800 msgr2=0x7f6230004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:17.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.859+0000 7f6248e76640 1 --2- 172.21.15.43:0/3339784049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6230004800 0x7f6230004c80 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f622c001750 tx=0x7f6230005830 comp rx=0 tx=0).stop 2024-04-25T08:57:17.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.859+0000 7f6248e76640 1 -- 172.21.15.43:0/3339784049 shutdown_connections 2024-04-25T08:57:17.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.859+0000 7f6248e76640 1 --2- 172.21.15.43:0/3339784049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6230004800 0x7f6230004c80 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.859+0000 7f6248e76640 1 -- 172.21.15.43:0/3339784049 >> 172.21.15.43:0/3339784049 conn(0x7f62440d17e0 msgr2=0x7f62440d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:17.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.859+0000 7f6248e76640 1 -- 172.21.15.43:0/3339784049 shutdown_connections 2024-04-25T08:57:17.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.859+0000 7f6248e76640 1 -- 172.21.15.43:0/3339784049 wait complete. 2024-04-25T08:57:17.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.861+0000 7f6248e76640 1 Processor -- start 2024-04-25T08:57:17.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.861+0000 7f6248e76640 1 -- start start 2024-04-25T08:57:17.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.861+0000 7f6248e76640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6230004800 0x7f624414ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:17.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.861+0000 7f6248e76640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62440c7950 0x7f62440c7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:17.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.861+0000 7f6248e76640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62440b59f0 0x7f62440b5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:17.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.861+0000 7f6248e76640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62440b3f50 con 0x7f62440b59f0 2024-04-25T08:57:17.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.861+0000 7f6248e76640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62440b3dd0 con 0x7f62440c7950 2024-04-25T08:57:17.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.861+0000 7f6248e76640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62440b40d0 con 0x7f6230004800 2024-04-25T08:57:17.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.862+0000 7f6242d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62440b59f0 0x7f62440b5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:17.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.862+0000 7f6242d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62440b59f0 0x7f62440b5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44196/0 (socket says 172.21.15.43:44196) 2024-04-25T08:57:17.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.862+0000 7f6242d9d640 1 -- 172.21.15.43:0/3738032640 learned_addr learned my addr 172.21.15.43:0/3738032640 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:17.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.862+0000 7f6241d9b640 1 --2- 172.21.15.43:0/3738032640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62440c7950 0x7f62440c7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:17.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.862+0000 7f624259c640 1 --2- 172.21.15.43:0/3738032640 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6230004800 0x7f624414ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:17.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.862+0000 7f6242d9d640 1 -- 172.21.15.43:0/3738032640 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6230004800 msgr2=0x7f624414ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:17.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.862+0000 7f6242d9d640 1 --2- 172.21.15.43:0/3738032640 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6230004800 0x7f624414ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.862+0000 7f6242d9d640 1 -- 172.21.15.43:0/3738032640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62440c7950 msgr2=0x7f62440c7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:17.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.862+0000 7f6242d9d640 1 --2- 172.21.15.43:0/3738032640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62440c7950 0x7f62440c7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.862+0000 7f6242d9d640 1 -- 172.21.15.43:0/3738032640 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62440b66d0 con 0x7f62440b59f0 2024-04-25T08:57:17.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.862+0000 7f6241d9b640 1 --2- 172.21.15.43:0/3738032640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62440c7950 0x7f62440c7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:17.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.862+0000 7f6242d9d640 1 --2- 172.21.15.43:0/3738032640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62440b59f0 0x7f62440b5e50 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f62440b7930 tx=0x7f623c007b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:17.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.862+0000 7f623affd640 1 -- 172.21.15.43:0/3738032640 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623c00abc0 con 0x7f62440b59f0 2024-04-25T08:57:17.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.862+0000 7f623affd640 1 -- 172.21.15.43:0/3738032640 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f623c019070 con 0x7f62440b59f0 2024-04-25T08:57:17.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.862+0000 7f6248e76640 1 -- 172.21.15.43:0/3738032640 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62440b69b0 con 0x7f62440b59f0 2024-04-25T08:57:17.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.862+0000 7f623affd640 1 -- 172.21.15.43:0/3738032640 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623c0133f0 con 0x7f62440b59f0 2024-04-25T08:57:17.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.862+0000 7f6248e76640 1 -- 172.21.15.43:0/3738032640 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62440b6f70 con 0x7f62440b59f0 2024-04-25T08:57:17.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.864+0000 7f623affd640 1 -- 172.21.15.43:0/3738032640 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f623c013590 con 0x7f62440b59f0 2024-04-25T08:57:17.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.864+0000 7f623affd640 1 --2- 172.21.15.43:0/3738032640 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f622406b500 0x7f622406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:17.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.864+0000 7f623affd640 1 -- 172.21.15.43:0/3738032640 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f623c08a800 con 0x7f62440b59f0 2024-04-25T08:57:17.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.864+0000 7f624259c640 1 --2- 172.21.15.43:0/3738032640 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f622406b500 0x7f622406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:17.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.865+0000 7f624259c640 1 --2- 172.21.15.43:0/3738032640 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f622406b500 0x7f622406d9c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6234009b00 tx=0x7f6234008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:17.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.865+0000 7f624bb80740 1 Processor -- start 2024-04-25T08:57:17.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.865+0000 7f624bb80740 1 -- start start 2024-04-25T08:57:17.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.866+0000 7f624bb80740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5aafd86b0 0x55c5aafd8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:17.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.866+0000 7f624bb80740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5aaff5f50 0x55c5aaff64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:17.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.866+0000 7f624bb80740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c5aafae390 0x55c5aaff4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:17.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.866+0000 7f624bb80740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5aaf10c60 con 0x55c5aaff5f50 2024-04-25T08:57:17.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.866+0000 7f624bb80740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5aaeb9ee0 con 0x55c5aafd86b0 2024-04-25T08:57:17.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.866+0000 7f624bb80740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c5aaf0cd00 con 0x55c5aafae390 2024-04-25T08:57:17.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.866+0000 7f6241d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5aaff5f50 0x55c5aaff64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:17.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.866+0000 7f6241d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5aaff5f50 0x55c5aaff64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44198/0 (socket says 172.21.15.43:44198) 2024-04-25T08:57:17.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.866+0000 7f6241d9b640 1 -- 172.21.15.43:0/3963456476 learned_addr learned my addr 172.21.15.43:0/3963456476 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:17.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.866+0000 7f624259c640 1 --2- 172.21.15.43:0/3963456476 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5aafd86b0 0x55c5aafd8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:17.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.866+0000 7f6242d9d640 1 --2- 172.21.15.43:0/3963456476 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c5aafae390 0x55c5aaff4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:17.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.867+0000 7f6241d9b640 1 -- 172.21.15.43:0/3963456476 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c5aafae390 msgr2=0x55c5aaff4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:17.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.867+0000 7f6241d9b640 1 --2- 172.21.15.43:0/3963456476 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c5aafae390 0x55c5aaff4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.867+0000 7f6241d9b640 1 -- 172.21.15.43:0/3963456476 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5aafd86b0 msgr2=0x55c5aafd8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:17.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.867+0000 7f6241d9b640 1 --2- 172.21.15.43:0/3963456476 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5aafd86b0 0x55c5aafd8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.867+0000 7f6241d9b640 1 -- 172.21.15.43:0/3963456476 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5aacf19f0 con 0x55c5aaff5f50 2024-04-25T08:57:17.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.867+0000 7f624259c640 1 --2- 172.21.15.43:0/3963456476 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5aafd86b0 0x55c5aafd8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:57:17.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.867+0000 7f6241d9b640 1 --2- 172.21.15.43:0/3963456476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5aaff5f50 0x55c5aaff64a0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f622c005040 tx=0x7f622c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acc3a6bd8ee3e9ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:17.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.867+0000 7f6223fff640 1 -- 172.21.15.43:0/3963456476 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c0050d0 con 0x55c5aaff5f50 2024-04-25T08:57:17.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.867+0000 7f6223fff640 1 -- 172.21.15.43:0/3963456476 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f622c042070 con 0x55c5aaff5f50 2024-04-25T08:57:17.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.867+0000 7f6223fff640 1 -- 172.21.15.43:0/3963456476 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c005470 con 0x55c5aaff5f50 2024-04-25T08:57:17.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.867+0000 7f624bb80740 1 -- 172.21.15.43:0/3963456476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5aaff5f50 msgr2=0x55c5aaff64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:17.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.868+0000 7f624bb80740 1 --2- 172.21.15.43:0/3963456476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5aaff5f50 0x55c5aaff64a0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f622c005040 tx=0x7f622c02c000 comp rx=0 tx=0).stop 2024-04-25T08:57:17.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.868+0000 7f624bb80740 1 -- 172.21.15.43:0/3963456476 shutdown_connections 2024-04-25T08:57:17.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.868+0000 7f624bb80740 1 --2- 172.21.15.43:0/3963456476 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c5aafae390 0x55c5aaff4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.868+0000 7f624bb80740 1 --2- 172.21.15.43:0/3963456476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5aaff5f50 0x55c5aaff64a0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.868+0000 7f624bb80740 1 --2- 172.21.15.43:0/3963456476 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5aafd86b0 0x55c5aafd8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.868+0000 7f624bb80740 1 -- 172.21.15.43:0/3963456476 >> 172.21.15.43:0/3963456476 conn(0x55c5aafaacc0 msgr2=0x55c5aaff34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:17.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.868+0000 7f624bb80740 1 -- 172.21.15.43:0/3963456476 shutdown_connections 2024-04-25T08:57:17.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.868+0000 7f624bb80740 1 -- 172.21.15.43:0/3963456476 wait complete. 2024-04-25T08:57:17.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.868+0000 7f624bb80740 1 -- start start 2024-04-25T08:57:17.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.868+0000 7f624bb80740 1 Processor -- start 2024-04-25T08:57:17.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.869+0000 7f624bb80740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5aafd86b0 0x55c5aafc4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:17.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.869+0000 7f624bb80740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c5aaff5f50 0x55c5aafea3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:17.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.869+0000 7f624bb80740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5aafda900 0x55c5aafc58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:17.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.869+0000 7f624bb80740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5aafaf130 con 0x55c5aafd86b0 2024-04-25T08:57:17.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.869+0000 7f624bb80740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5aaecd470 con 0x55c5aafda900 2024-04-25T08:57:17.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.869+0000 7f624bb80740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c5aafa8bf0 con 0x55c5aaff5f50 2024-04-25T08:57:17.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.869+0000 7f624bb80740 10 client.0 _ll_register_callbacks cb 0x55c5aae997d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:17.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.869+0000 7f624259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5aafd86b0 0x55c5aafc4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:17.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.869+0000 7f624259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5aafd86b0 0x55c5aafc4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44204/0 (socket says 172.21.15.43:44204) 2024-04-25T08:57:17.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.869+0000 7f624259c640 1 -- 172.21.15.43:0/2094862719 learned_addr learned my addr 172.21.15.43:0/2094862719 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:17.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.869+0000 7f6242d9d640 1 --2- 172.21.15.43:0/2094862719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5aafda900 0x55c5aafc58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:17.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.869+0000 7f6241d9b640 1 --2- 172.21.15.43:0/2094862719 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c5aaff5f50 0x55c5aafea3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:17.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.870+0000 7f624259c640 1 -- 172.21.15.43:0/2094862719 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c5aaff5f50 msgr2=0x55c5aafea3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:17.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.870+0000 7f624259c640 1 --2- 172.21.15.43:0/2094862719 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c5aaff5f50 0x55c5aafea3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.870+0000 7f624259c640 1 -- 172.21.15.43:0/2094862719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5aafda900 msgr2=0x55c5aafc58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:17.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.870+0000 7f624259c640 1 --2- 172.21.15.43:0/2094862719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5aafda900 0x55c5aafc58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.870+0000 7f624259c640 1 -- 172.21.15.43:0/2094862719 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5aabe52e0 con 0x55c5aafd86b0 2024-04-25T08:57:17.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.870+0000 7f624259c640 1 --2- 172.21.15.43:0/2094862719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5aafd86b0 0x55c5aafc4fa0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f62340038c0 tx=0x7f6234004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c269332e07b0add server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:17.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.870+0000 7f62217fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:57:17.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.870+0000 7f62217fa640 1 -- 172.21.15.43:0/2094862719 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6234029070 con 0x55c5aafd86b0 2024-04-25T08:57:17.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.870+0000 7f624bb80740 1 -- 172.21.15.43:0/2094862719 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f622c042070 con 0x55c5aafd86b0 2024-04-25T08:57:17.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.870+0000 7f62217fa640 1 -- 172.21.15.43:0/2094862719 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62340169b0 con 0x55c5aafd86b0 2024-04-25T08:57:17.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.870+0000 7f62217fa640 1 -- 172.21.15.43:0/2094862719 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6234034050 con 0x55c5aafd86b0 2024-04-25T08:57:17.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.870+0000 7f624bb80740 10 client.5877 fetch_fsmap learned FSMap version 21 2024-04-25T08:57:17.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.870+0000 7f624bb80740 1 -- 172.21.15.43:0/2094862719 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x55c5aad52180 con 0x55c5aafd86b0 2024-04-25T08:57:17.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.870+0000 7f62217fa640 1 -- 172.21.15.43:0/2094862719 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f622c042070 con 0x55c5aafd86b0 2024-04-25T08:57:17.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.870+0000 7f62217fa640 1 -- 172.21.15.43:0/2094862719 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6234037020 con 0x55c5aafd86b0 2024-04-25T08:57:17.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.870+0000 7f624bb80740 10 client.5877 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T08:57:17.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.870+0000 7f624bb80740 10 client.5877 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T08:57:17.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.870+0000 7f624bb80740 20 client.5877 populate_metadata read hostname 'smithi043' 2024-04-25T08:57:17.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.871+0000 7f624bb80740 1 --2- 172.21.15.43:0/2094862719 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55c5aafc5e30 0x55c5aafd6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:17.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.871+0000 7f624bb80740 4 client.5877 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:57:17.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.871+0000 7f624bb80740 1 -- 172.21.15.43:0/2094862719 --> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c5aaf1e1a0 con 0x55c5aafc5e30 2024-04-25T08:57:17.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.871+0000 7f6241d9b640 1 --2- 172.21.15.43:0/2094862719 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55c5aafc5e30 0x55c5aafd6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:17.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.871+0000 7f6241d9b640 1 --2- 172.21.15.43:0/2094862719 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55c5aafc5e30 0x55c5aafd6dd0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d381b1d0817fc98 server_cookie=a6836729dfc8c67c in_seq=0 out_seq=0 2024-04-25T08:57:17.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.871+0000 7f62217fa640 10 client.5877 ms_handle_connect on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:17.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.872+0000 7f62217fa640 1 -- 172.21.15.43:0/2094862719 <== mds.0 v2:172.21.15.193:6836/4164634380 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c5aaf1e1a0 con 0x55c5aafc5e30 2024-04-25T08:57:17.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.872+0000 7f62217fa640 10 client.5877 handle_command_reply: tid=0 2024-04-25T08:57:17.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.872+0000 7f624bb80740 1 client.5877 shutdown 2024-04-25T08:57:17.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.872+0000 7f624bb80740 1 -- 172.21.15.43:0/2094862719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5aafd86b0 msgr2=0x55c5aafc4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:17.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.872+0000 7f624bb80740 1 --2- 172.21.15.43:0/2094862719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5aafd86b0 0x55c5aafc4fa0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f62340038c0 tx=0x7f6234004730 comp rx=0 tx=0).stop 2024-04-25T08:57:17.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.872+0000 7f624bb80740 1 -- 172.21.15.43:0/2094862719 shutdown_connections 2024-04-25T08:57:17.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.872+0000 7f624bb80740 1 --2- 172.21.15.43:0/2094862719 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55c5aafc5e30 0x55c5aafd6dd0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.872+0000 7f624bb80740 1 --2- 172.21.15.43:0/2094862719 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5aafda900 0x55c5aafc58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.872+0000 7f624bb80740 1 --2- 172.21.15.43:0/2094862719 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c5aaff5f50 0x55c5aafea3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.872+0000 7f624bb80740 1 --2- 172.21.15.43:0/2094862719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5aafd86b0 0x55c5aafc4fa0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.872+0000 7f62217fa640 0 client.5877 ms_handle_reset on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:17.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.872+0000 7f624bb80740 1 -- 172.21.15.43:0/2094862719 >> 172.21.15.43:0/2094862719 conn(0x55c5aafae390 msgr2=0x55c5aaff6a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:17.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.872+0000 7f624bb80740 1 -- 172.21.15.43:0/2094862719 shutdown_connections 2024-04-25T08:57:17.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.872+0000 7f624bb80740 1 -- 172.21.15.43:0/2094862719 wait complete. 2024-04-25T08:57:17.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.872+0000 7f624bb80740 20 client.5877 trim_cache size 0 max 16384 2024-04-25T08:57:17.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.887+0000 7f624bb80740 1 Processor -- start 2024-04-25T08:57:17.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.887+0000 7f624bb80740 1 -- start start 2024-04-25T08:57:17.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.888+0000 7f624bb80740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5aafd86b0 0x55c5ab003020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:17.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.888+0000 7f624bb80740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5aafda900 0x55c5aafc4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:17.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.888+0000 7f624bb80740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c5aaff5f50 0x55c5aafc52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:17.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.888+0000 7f624bb80740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62442004e0 con 0x55c5aafd86b0 2024-04-25T08:57:17.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.888+0000 7f624bb80740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62441ff710 con 0x55c5aafda900 2024-04-25T08:57:17.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.888+0000 7f624bb80740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f624414c470 con 0x55c5aaff5f50 2024-04-25T08:57:17.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.888+0000 7f624259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5aafd86b0 0x55c5ab003020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:17.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.888+0000 7f624259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5aafd86b0 0x55c5ab003020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44210/0 (socket says 172.21.15.43:44210) 2024-04-25T08:57:17.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.888+0000 7f624259c640 1 -- 172.21.15.43:0/2075181619 learned_addr learned my addr 172.21.15.43:0/2075181619 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:17.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.888+0000 7f6241d9b640 1 --2- 172.21.15.43:0/2075181619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5aafda900 0x55c5aafc4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:17.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.888+0000 7f6242d9d640 1 --2- 172.21.15.43:0/2075181619 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c5aaff5f50 0x55c5aafc52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:17.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.888+0000 7f624259c640 1 -- 172.21.15.43:0/2075181619 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c5aaff5f50 msgr2=0x55c5aafc52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:17.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.888+0000 7f624259c640 1 --2- 172.21.15.43:0/2075181619 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c5aaff5f50 0x55c5aafc52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.888+0000 7f624259c640 1 -- 172.21.15.43:0/2075181619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5aafda900 msgr2=0x55c5aafc4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:17.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.888+0000 7f624259c640 1 --2- 172.21.15.43:0/2075181619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5aafda900 0x55c5aafc4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.888+0000 7f624259c640 1 -- 172.21.15.43:0/2075181619 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c5aabbe3a0 con 0x55c5aafd86b0 2024-04-25T08:57:17.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.888+0000 7f6241d9b640 1 --2- 172.21.15.43:0/2075181619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5aafda900 0x55c5aafc4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:17.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.888+0000 7f624259c640 1 --2- 172.21.15.43:0/2075181619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5aafd86b0 0x55c5ab003020 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f6234016fd0 tx=0x7f6234026000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae982e8e57ec843 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:17.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.889+0000 7f62237fe640 1 -- 172.21.15.43:0/2075181619 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6234029040 con 0x55c5aafd86b0 2024-04-25T08:57:17.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.889+0000 7f62237fe640 1 -- 172.21.15.43:0/2075181619 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f623403d070 con 0x55c5aafd86b0 2024-04-25T08:57:17.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.889+0000 7f62237fe640 1 -- 172.21.15.43:0/2075181619 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6234016c00 con 0x55c5aafd86b0 2024-04-25T08:57:17.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.889+0000 7f624bb80740 1 -- 172.21.15.43:0/2075181619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5aafd86b0 msgr2=0x55c5ab003020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:17.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.889+0000 7f624bb80740 1 --2- 172.21.15.43:0/2075181619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5aafd86b0 0x55c5ab003020 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f6234016fd0 tx=0x7f6234026000 comp rx=0 tx=0).stop 2024-04-25T08:57:17.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.889+0000 7f624bb80740 1 -- 172.21.15.43:0/2075181619 shutdown_connections 2024-04-25T08:57:17.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.889+0000 7f624bb80740 1 --2- 172.21.15.43:0/2075181619 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c5aaff5f50 0x55c5aafc52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.889+0000 7f624bb80740 1 --2- 172.21.15.43:0/2075181619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5aafda900 0x55c5aafc4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.889+0000 7f624bb80740 1 --2- 172.21.15.43:0/2075181619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5aafd86b0 0x55c5ab003020 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.889+0000 7f624bb80740 1 -- 172.21.15.43:0/2075181619 >> 172.21.15.43:0/2075181619 conn(0x55c5aafae390 msgr2=0x55c5aafeb190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:17.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.889+0000 7f624bb80740 1 -- 172.21.15.43:0/2075181619 shutdown_connections 2024-04-25T08:57:17.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.889+0000 7f624bb80740 1 -- 172.21.15.43:0/2075181619 wait complete. 2024-04-25T08:57:17.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.890+0000 7f624bb80740 1 -- start start 2024-04-25T08:57:17.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.890+0000 7f624bb80740 1 Processor -- start 2024-04-25T08:57:17.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.891+0000 7f624bb80740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5aafd86b0 0x55c5ab005900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:17.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.891+0000 7f624bb80740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5aafda900 0x55c5ab005e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:17.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.891+0000 7f624bb80740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c5aaff5f50 0x55c5aafac260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:17.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.891+0000 7f624bb80740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c5aafa8f20 con 0x55c5aafda900 2024-04-25T08:57:17.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.891+0000 7f624bb80740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f624414ca70 con 0x55c5aafd86b0 2024-04-25T08:57:17.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.891+0000 7f624bb80740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c5aafab860 con 0x55c5aaff5f50 2024-04-25T08:57:17.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.891+0000 7f624bb80740 10 client.0 _ll_register_callbacks cb 0x55c5aafa90d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:17.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.891+0000 7f6241d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5aafda900 0x55c5ab005e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:17.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.891+0000 7f6241d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5aafda900 0x55c5ab005e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44226/0 (socket says 172.21.15.43:44226) 2024-04-25T08:57:17.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.891+0000 7f6241d9b640 1 -- 172.21.15.43:0/2848626420 learned_addr learned my addr 172.21.15.43:0/2848626420 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:17.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.891+0000 7f624259c640 1 --2- 172.21.15.43:0/2848626420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5aafd86b0 0x55c5ab005900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:17.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.891+0000 7f6242d9d640 1 --2- 172.21.15.43:0/2848626420 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c5aaff5f50 0x55c5aafac260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:17.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.891+0000 7f6241d9b640 1 -- 172.21.15.43:0/2848626420 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c5aaff5f50 msgr2=0x55c5aafac260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:17.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.891+0000 7f6241d9b640 1 --2- 172.21.15.43:0/2848626420 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c5aaff5f50 0x55c5aafac260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.891+0000 7f6241d9b640 1 -- 172.21.15.43:0/2848626420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5aafd86b0 msgr2=0x55c5ab005900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:17.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.891+0000 7f6241d9b640 1 --2- 172.21.15.43:0/2848626420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5aafd86b0 0x55c5ab005900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.891+0000 7f6241d9b640 1 -- 172.21.15.43:0/2848626420 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c5aafa8880 con 0x55c5aafda900 2024-04-25T08:57:17.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.891+0000 7f6241d9b640 1 --2- 172.21.15.43:0/2848626420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5aafda900 0x55c5ab005e40 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f622c001720 tx=0x7f622c02c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba82492e951ebf71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:17.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.891+0000 7f62217fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:57:17.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.891+0000 7f62217fa640 1 -- 172.21.15.43:0/2848626420 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c037040 con 0x55c5aafda900 2024-04-25T08:57:17.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.891+0000 7f62217fa640 1 -- 172.21.15.43:0/2848626420 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f622c04fb70 con 0x55c5aafda900 2024-04-25T08:57:17.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.891+0000 7f624bb80740 1 -- 172.21.15.43:0/2848626420 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f623403d070 con 0x55c5aafda900 2024-04-25T08:57:17.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.892+0000 7f62217fa640 1 -- 172.21.15.43:0/2848626420 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f622c058050 con 0x55c5aafda900 2024-04-25T08:57:17.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.892+0000 7f624bb80740 10 client.5883 fetch_fsmap learned FSMap version 21 2024-04-25T08:57:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.892+0000 7f624bb80740 1 -- 172.21.15.43:0/2848626420 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x55c5ab067310 con 0x55c5aafda900 2024-04-25T08:57:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.892+0000 7f62217fa640 1 -- 172.21.15.43:0/2848626420 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f623403d070 con 0x55c5aafda900 2024-04-25T08:57:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.892+0000 7f62217fa640 1 -- 172.21.15.43:0/2848626420 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f622c05b020 con 0x55c5aafda900 2024-04-25T08:57:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.892+0000 7f624bb80740 10 client.5883 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T08:57:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.892+0000 7f624bb80740 10 client.5883 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T08:57:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.892+0000 7f624bb80740 20 client.5883 populate_metadata read hostname 'smithi043' 2024-04-25T08:57:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.892+0000 7f624bb80740 1 --2- 172.21.15.43:0/2848626420 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55c5aafec190 0x55c5aafac7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.892+0000 7f624bb80740 4 client.5883 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}] 2024-04-25T08:57:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.892+0000 7f624bb80740 1 -- 172.21.15.43:0/2848626420 --> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}) v1 -- 0x55c5aaf184f0 con 0x55c5aafec190 2024-04-25T08:57:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.892+0000 7f624259c640 1 --2- 172.21.15.43:0/2848626420 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55c5aafec190 0x55c5aafac7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.893+0000 7f62217fa640 10 client.5883 ms_handle_connect on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.893+0000 7f624259c640 1 --2- 172.21.15.43:0/2848626420 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55c5aafec190 0x55c5aafac7a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ad8a14bfd29d757 server_cookie=2be7e770dc786a00 in_seq=0 out_seq=0 2024-04-25T08:57:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.893+0000 7f62217fa640 1 -- 172.21.15.43:0/2848626420 <== mds.0 v2:172.21.15.193:6836/4164634380 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+26 (crc 0 0 0) 0x55c5aaf184f0 con 0x55c5aafec190 2024-04-25T08:57:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.893+0000 7f62217fa640 10 client.5883 handle_command_reply: tid=0 2024-04-25T08:57:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.893+0000 7f624bb80740 1 client.5883 shutdown 2024-04-25T08:57:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.893+0000 7f624bb80740 1 -- 172.21.15.43:0/2848626420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5aafda900 msgr2=0x55c5ab005e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:17.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.893+0000 7f624bb80740 1 --2- 172.21.15.43:0/2848626420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5aafda900 0x55c5ab005e40 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f622c001720 tx=0x7f622c02c070 comp rx=0 tx=0).stop 2024-04-25T08:57:17.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.893+0000 7f624bb80740 1 -- 172.21.15.43:0/2848626420 shutdown_connections 2024-04-25T08:57:17.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.893+0000 7f624bb80740 1 --2- 172.21.15.43:0/2848626420 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c5aaff5f50 0x55c5aafac260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.893+0000 7f624bb80740 1 --2- 172.21.15.43:0/2848626420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c5aafda900 0x55c5ab005e40 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.893+0000 7f624bb80740 1 --2- 172.21.15.43:0/2848626420 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55c5aafec190 0x55c5aafac7a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.893+0000 7f624bb80740 1 --2- 172.21.15.43:0/2848626420 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c5aafd86b0 0x55c5ab005900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.893+0000 7f62217fa640 0 client.5883 ms_handle_reset on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:17.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.893+0000 7f624bb80740 1 -- 172.21.15.43:0/2848626420 >> 172.21.15.43:0/2848626420 conn(0x55c5aafae390 msgr2=0x55c5aafad9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:17.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.893+0000 7f624bb80740 1 -- 172.21.15.43:0/2848626420 shutdown_connections 2024-04-25T08:57:17.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.893+0000 7f624bb80740 1 -- 172.21.15.43:0/2848626420 wait complete. 2024-04-25T08:57:17.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.893+0000 7f624bb80740 20 client.5883 trim_cache size 0 max 16384 2024-04-25T08:57:17.889 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T08:57:17.890 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": -2 2024-04-25T08:57:17.890 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T08:57:17.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.894+0000 7f6248e76640 1 -- 172.21.15.43:0/3738032640 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f622406b500 msgr2=0x7f622406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:17.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.894+0000 7f6248e76640 1 --2- 172.21.15.43:0/3738032640 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f622406b500 0x7f622406d9c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6234009b00 tx=0x7f6234008040 comp rx=0 tx=0).stop 2024-04-25T08:57:17.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.894+0000 7f6248e76640 1 -- 172.21.15.43:0/3738032640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62440b59f0 msgr2=0x7f62440b5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:17.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.894+0000 7f6248e76640 1 --2- 172.21.15.43:0/3738032640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62440b59f0 0x7f62440b5e50 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f62440b7930 tx=0x7f623c007b00 comp rx=0 tx=0).stop 2024-04-25T08:57:17.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.894+0000 7f6248e76640 1 -- 172.21.15.43:0/3738032640 shutdown_connections 2024-04-25T08:57:17.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.894+0000 7f6248e76640 1 --2- 172.21.15.43:0/3738032640 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62440b59f0 0x7f62440b5e50 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.894+0000 7f6248e76640 1 --2- 172.21.15.43:0/3738032640 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f622406b500 0x7f622406d9c0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.894+0000 7f6248e76640 1 --2- 172.21.15.43:0/3738032640 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62440c7950 0x7f62440c7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.894+0000 7f6248e76640 1 --2- 172.21.15.43:0/3738032640 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6230004800 0x7f624414ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.894+0000 7f6248e76640 1 -- 172.21.15.43:0/3738032640 >> 172.21.15.43:0/3738032640 conn(0x7f62440d17e0 msgr2=0x7f62440b0080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:17.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.894+0000 7f6248e76640 1 -- 172.21.15.43:0/3738032640 shutdown_connections 2024-04-25T08:57:17.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:17.894+0000 7f6248e76640 1 -- 172.21.15.43:0/3738032640 wait complete. 2024-04-25T08:57:17.894 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/i/dont/exist' returned '{'return_code': -2}' 2024-04-25T08:57:17.894 INFO:tasks.cephfs.test_scrub_checks:First run: flushing /scrub_test_path/ceph-qa-suite/suites 2024-04-25T08:57:17.894 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/suites' 2024-04-25T08:57:17.894 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:57:17.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.005+0000 7fa8ee753640 1 Processor -- start 2024-04-25T08:57:17.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.005+0000 7fa8ee753640 1 -- start start 2024-04-25T08:57:17.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.006+0000 7fa8ee753640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa8e815dd00 0x7fa8e80d2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:17.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.006+0000 7fa8ee753640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fa8e80ce970 0x7fa8e80b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:17.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.006+0000 7fa8ee753640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8e815e0e0 con 0x7fa8e80d33f0 2024-04-25T08:57:17.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.006+0000 7fa8ee753640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8e80d3850 con 0x7fa8e815dd00 2024-04-25T08:57:17.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.006+0000 7fa8ee753640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8e80d24e0 con 0x7fa8e80ce970 2024-04-25T08:57:17.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.006+0000 7fa8e7fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa8e815dd00 0x7fa8e80d2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:17.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.006+0000 7fa8e7fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa8e815dd00 0x7fa8e80d2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44228/0 (socket says 172.21.15.43:44228) 2024-04-25T08:57:17.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.006+0000 7fa8e7fff640 1 -- 172.21.15.43:0/1451682292 learned_addr learned my addr 172.21.15.43:0/1451682292 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:17.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.006+0000 7fa8eccf0640 1 --2- 172.21.15.43:0/1451682292 >> v2:172.21.15.193:3300/0 conn(0x7fa8e80ce970 0x7fa8e80b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:17.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.006+0000 7fa8e7fff640 1 -- 172.21.15.43:0/1451682292 >> v2:172.21.15.193:3300/0 conn(0x7fa8e80ce970 msgr2=0x7fa8e80b6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:17.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.006+0000 7fa8e7fff640 1 --2- 172.21.15.43:0/1451682292 >> v2:172.21.15.193:3300/0 conn(0x7fa8e80ce970 0x7fa8e80b6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:17.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.006+0000 7fa8e7fff640 1 -- 172.21.15.43:0/1451682292 >> v1:172.21.15.193:6789/0 conn(0x7fa8e80d33f0 legacy=0x7fa8e80ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:17.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.006+0000 7fa8e6ffd640 1 -- 172.21.15.43:0/1451682292 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1550682953 0 0) 0x7fa8e815e0e0 con 0x7fa8e80d33f0 2024-04-25T08:57:17.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.006+0000 7fa8e7fff640 1 -- 172.21.15.43:0/1451682292 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e80ceeb0 con 0x7fa8e815dd00 2024-04-25T08:57:17.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.006+0000 7fa8eccf0640 1 --2- 172.21.15.43:0/1451682292 >> v2:172.21.15.193:3300/0 conn(0x7fa8e80ce970 0x7fa8e80b6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:57:17.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.007+0000 7fa8e7fff640 1 --2- 172.21.15.43:0/1451682292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8e815dd00 0x7fa8e80d2eb0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa8e80d1180 tx=0x7fa8d802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bbf81c464c56b31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:17.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.007+0000 7fa8e6ffd640 1 -- 172.21.15.43:0/1451682292 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8037070 con 0x7fa8e815dd00 2024-04-25T08:57:17.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.007+0000 7fa8e6ffd640 1 -- 172.21.15.43:0/1451682292 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8d8032440 con 0x7fa8e815dd00 2024-04-25T08:57:17.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.007+0000 7fa8e6ffd640 1 -- 172.21.15.43:0/1451682292 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8032810 con 0x7fa8e815dd00 2024-04-25T08:57:18.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.008+0000 7fa8ee753640 1 -- 172.21.15.43:0/1451682292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8e815dd00 msgr2=0x7fa8e80d2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:18.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.008+0000 7fa8ee753640 1 --2- 172.21.15.43:0/1451682292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8e815dd00 0x7fa8e80d2eb0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa8e80d1180 tx=0x7fa8d802a270 comp rx=0 tx=0).stop 2024-04-25T08:57:18.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.008+0000 7fa8ee753640 1 -- 172.21.15.43:0/1451682292 shutdown_connections 2024-04-25T08:57:18.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.008+0000 7fa8ee753640 1 --2- 172.21.15.43:0/1451682292 >> v2:172.21.15.193:3300/0 conn(0x7fa8e80ce970 0x7fa8e80b6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.008+0000 7fa8ee753640 1 --2- 172.21.15.43:0/1451682292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8e815dd00 0x7fa8e80d2eb0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.008+0000 7fa8ee753640 1 -- 172.21.15.43:0/1451682292 >> 172.21.15.43:0/1451682292 conn(0x7fa8e806cbd0 msgr2=0x7fa8e806cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:18.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.008+0000 7fa8ee753640 1 -- 172.21.15.43:0/1451682292 shutdown_connections 2024-04-25T08:57:18.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.008+0000 7fa8ee753640 1 -- 172.21.15.43:0/1451682292 wait complete. 2024-04-25T08:57:18.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.009+0000 7fa8ee753640 1 Processor -- start 2024-04-25T08:57:18.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.009+0000 7fa8ee753640 1 -- start start 2024-04-25T08:57:18.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.010+0000 7fa8ee753640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa8e80ce970 0x7fa8e8154680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:18.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.010+0000 7fa8ee753640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa8e80d33f0 0x7fa8e81596b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:18.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.010+0000 7fa8ee753640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8e815dd00 0x7fa8e8152c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:18.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.010+0000 7fa8ee753640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e80d2170 con 0x7fa8e815dd00 2024-04-25T08:57:18.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.010+0000 7fa8ee753640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e80d1e90 con 0x7fa8e80ce970 2024-04-25T08:57:18.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.010+0000 7fa8ee753640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e80d2970 con 0x7fa8e80d33f0 2024-04-25T08:57:18.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.010+0000 7fa8eccf0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8e815dd00 0x7fa8e8152c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:18.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.010+0000 7fa8eccf0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8e815dd00 0x7fa8e8152c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44242/0 (socket says 172.21.15.43:44242) 2024-04-25T08:57:18.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.010+0000 7fa8eccf0640 1 -- 172.21.15.43:0/1513263738 learned_addr learned my addr 172.21.15.43:0/1513263738 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:18.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.010+0000 7fa8e7fff640 1 --2- 172.21.15.43:0/1513263738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa8e80ce970 0x7fa8e8154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:18.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.010+0000 7fa8e77fe640 1 --2- 172.21.15.43:0/1513263738 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa8e80d33f0 0x7fa8e81596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:18.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.010+0000 7fa8eccf0640 1 -- 172.21.15.43:0/1513263738 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa8e80d33f0 msgr2=0x7fa8e81596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:18.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.010+0000 7fa8eccf0640 1 --2- 172.21.15.43:0/1513263738 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa8e80d33f0 0x7fa8e81596b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.010+0000 7fa8eccf0640 1 -- 172.21.15.43:0/1513263738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa8e80ce970 msgr2=0x7fa8e8154680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:18.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.010+0000 7fa8eccf0640 1 --2- 172.21.15.43:0/1513263738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa8e80ce970 0x7fa8e8154680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.010+0000 7fa8eccf0640 1 -- 172.21.15.43:0/1513263738 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e80cac50 con 0x7fa8e815dd00 2024-04-25T08:57:18.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.011+0000 7fa8eccf0640 1 --2- 172.21.15.43:0/1513263738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8e815dd00 0x7fa8e8152c70 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa8e000efb0 tx=0x7fa8e000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:18.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.011+0000 7fa8e4ff9640 1 -- 172.21.15.43:0/1513263738 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0018020 con 0x7fa8e815dd00 2024-04-25T08:57:18.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.011+0000 7fa8e4ff9640 1 -- 172.21.15.43:0/1513263738 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8e0014480 con 0x7fa8e815dd00 2024-04-25T08:57:18.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.011+0000 7fa8ee753640 1 -- 172.21.15.43:0/1513263738 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8e80caf30 con 0x7fa8e815dd00 2024-04-25T08:57:18.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.011+0000 7fa8e4ff9640 1 -- 172.21.15.43:0/1513263738 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0014850 con 0x7fa8e815dd00 2024-04-25T08:57:18.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.011+0000 7fa8ee753640 1 -- 172.21.15.43:0/1513263738 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8e80cb4f0 con 0x7fa8e815dd00 2024-04-25T08:57:18.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.012+0000 7fa8ee753640 1 -- 172.21.15.43:0/1513263738 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8e8074ef0 con 0x7fa8e815dd00 2024-04-25T08:57:18.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.012+0000 7fa8e4ff9640 1 -- 172.21.15.43:0/1513263738 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa8e0005d10 con 0x7fa8e815dd00 2024-04-25T08:57:18.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.013+0000 7fa8e4ff9640 1 --2- 172.21.15.43:0/1513263738 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa8d406b500 0x7fa8d406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:18.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.013+0000 7fa8e4ff9640 1 -- 172.21.15.43:0/1513263738 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa8e008ade0 con 0x7fa8e815dd00 2024-04-25T08:57:18.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.013+0000 7fa8e7fff640 1 --2- 172.21.15.43:0/1513263738 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa8d406b500 0x7fa8d406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:18.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.013+0000 7fa8e7fff640 1 --2- 172.21.15.43:0/1513263738 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa8d406b500 0x7fa8d406d9c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8034000 tx=0x7fa8d802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:18.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.017+0000 7fa8e4ff9640 1 -- 172.21.15.43:0/1513263738 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa8e0022030 con 0x7fa8e815dd00 2024-04-25T08:57:18.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.236+0000 7fa8ee753640 1 -- 172.21.15.43:0/1513263738 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8e815c670 con 0x7fa8e815dd00 2024-04-25T08:57:18.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.236+0000 7fa8e4ff9640 1 -- 172.21.15.43:0/1513263738 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fa8e0010040 con 0x7fa8e815dd00 2024-04-25T08:57:18.226 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:57:18.226 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1504450503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1504450503},{"type":"v1","addr":"172.21.15.193:6835","nonce":1504450503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5691,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2657133821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2657133821},{"type":"v1","addr":"172.21.15.43:6834","nonce":2657133821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":5694,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/4271431818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":4271431818},{"type":"v1","addr":"172.21.15.43:6835","nonce":4271431818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:57:01.856132+0000","modified":"2024-04-25T08:57:03.857410+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5709":{"gid":5709,"name":"d","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.193:6837/4164634380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4164634380},{"type":"v1","addr":"172.21.15.193:6837","nonce":4164634380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:57:18.226 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 21 2024-04-25T08:57:18.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.238+0000 7fa8ee753640 1 -- 172.21.15.43:0/1513263738 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa8d406b500 msgr2=0x7fa8d406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:18.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.238+0000 7fa8ee753640 1 --2- 172.21.15.43:0/1513263738 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa8d406b500 0x7fa8d406d9c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8034000 tx=0x7fa8d802e000 comp rx=0 tx=0).stop 2024-04-25T08:57:18.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.238+0000 7fa8ee753640 1 -- 172.21.15.43:0/1513263738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8e815dd00 msgr2=0x7fa8e8152c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:18.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.238+0000 7fa8ee753640 1 --2- 172.21.15.43:0/1513263738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8e815dd00 0x7fa8e8152c70 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fa8e000efb0 tx=0x7fa8e000c490 comp rx=0 tx=0).stop 2024-04-25T08:57:18.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.238+0000 7fa8ee753640 1 -- 172.21.15.43:0/1513263738 shutdown_connections 2024-04-25T08:57:18.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.238+0000 7fa8ee753640 1 --2- 172.21.15.43:0/1513263738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8e815dd00 0x7fa8e8152c70 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.238+0000 7fa8ee753640 1 --2- 172.21.15.43:0/1513263738 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa8e80d33f0 0x7fa8e81596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.238+0000 7fa8ee753640 1 --2- 172.21.15.43:0/1513263738 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa8d406b500 0x7fa8d406d9c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.238+0000 7fa8ee753640 1 --2- 172.21.15.43:0/1513263738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa8e80ce970 0x7fa8e8154680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.238+0000 7fa8ee753640 1 -- 172.21.15.43:0/1513263738 >> 172.21.15.43:0/1513263738 conn(0x7fa8e806cbd0 msgr2=0x7fa8e815d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:18.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.238+0000 7fa8ee753640 1 -- 172.21.15.43:0/1513263738 shutdown_connections 2024-04-25T08:57:18.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.239+0000 7fa8ee753640 1 -- 172.21.15.43:0/1513263738 wait complete. 2024-04-25T08:57:18.239 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path /scrub_test_path/ceph-qa-suite/suites 2024-04-25T08:57:18.348 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-04-25T08:57:18.348 INFO:teuthology.orchestra.run.smithi193.stdout: "return_code": 0 2024-04-25T08:57:18.348 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-04-25T08:57:18.357 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/suites' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T08:57:18.357 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites' 2024-04-25T08:57:18.357 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:57:18.357 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites 2024-04-25T08:57:18.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.472+0000 7fc4cebf9640 1 Processor -- start 2024-04-25T08:57:18.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.472+0000 7fc4cebf9640 1 -- start start 2024-04-25T08:57:18.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.473+0000 7fc4cebf9640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc4c815dd80 0x7fc4c80cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:18.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.473+0000 7fc4cebf9640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc4c80b1cd0 0x7fc4c80d1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:18.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.473+0000 7fc4cebf9640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4c80b2fe0 con 0x7fc4c80d17b0 2024-04-25T08:57:18.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.473+0000 7fc4cebf9640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4c80cfc60 con 0x7fc4c815dd80 2024-04-25T08:57:18.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.473+0000 7fc4cebf9640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c80b32c0 con 0x7fc4c80b1cd0 2024-04-25T08:57:18.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.473+0000 7fc4cc995640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc4c815dd80 0x7fc4c80cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:18.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.473+0000 7fc4cc995640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc4c815dd80 0x7fc4c80cf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44256/0 (socket says 172.21.15.43:44256) 2024-04-25T08:57:18.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.473+0000 7fc4cc995640 1 -- 172.21.15.43:0/954451868 learned_addr learned my addr 172.21.15.43:0/954451868 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:18.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.473+0000 7fc4cd196640 1 --2- 172.21.15.43:0/954451868 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc4c80b1cd0 0x7fc4c80d1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:18.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.473+0000 7fc4cc995640 1 -- 172.21.15.43:0/954451868 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc4c80b1cd0 msgr2=0x7fc4c80d1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:18.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.473+0000 7fc4cc995640 1 --2- 172.21.15.43:0/954451868 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc4c80b1cd0 0x7fc4c80d1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.473+0000 7fc4cc995640 1 -- 172.21.15.43:0/954451868 >> v1:172.21.15.193:6789/0 conn(0x7fc4c80d17b0 legacy=0x7fc4c815e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:18.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.473+0000 7fc4cc995640 1 -- 172.21.15.43:0/954451868 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c80ced70 con 0x7fc4c815dd80 2024-04-25T08:57:18.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.474+0000 7fc4cc995640 1 --2- 172.21.15.43:0/954451868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4c815dd80 0x7fc4c80cf720 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc009870 tx=0x7fc4bc02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7751fc1d71deb64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:18.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.474+0000 7fc4c77fe640 1 -- 172.21.15.43:0/954451868 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc037070 con 0x7fc4c815dd80 2024-04-25T08:57:18.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.474+0000 7fc4c77fe640 1 -- 172.21.15.43:0/954451868 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4bc032440 con 0x7fc4c815dd80 2024-04-25T08:57:18.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.474+0000 7fc4c77fe640 1 -- 172.21.15.43:0/954451868 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc032810 con 0x7fc4c815dd80 2024-04-25T08:57:18.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.474+0000 7fc4cebf9640 1 -- 172.21.15.43:0/954451868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4c815dd80 msgr2=0x7fc4c80cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:18.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.474+0000 7fc4cebf9640 1 --2- 172.21.15.43:0/954451868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4c815dd80 0x7fc4c80cf720 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc009870 tx=0x7fc4bc02a1c0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.475+0000 7fc4cebf9640 1 -- 172.21.15.43:0/954451868 shutdown_connections 2024-04-25T08:57:18.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.475+0000 7fc4cebf9640 1 --2- 172.21.15.43:0/954451868 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc4c80b1cd0 0x7fc4c80d1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.475+0000 7fc4cebf9640 1 --2- 172.21.15.43:0/954451868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4c815dd80 0x7fc4c80cf720 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.475+0000 7fc4cebf9640 1 -- 172.21.15.43:0/954451868 >> 172.21.15.43:0/954451868 conn(0x7fc4c8072aa0 msgr2=0x7fc4c809aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:18.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.475+0000 7fc4cebf9640 1 -- 172.21.15.43:0/954451868 shutdown_connections 2024-04-25T08:57:18.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.475+0000 7fc4cebf9640 1 -- 172.21.15.43:0/954451868 wait complete. 2024-04-25T08:57:18.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.476+0000 7fc4cebf9640 1 Processor -- start 2024-04-25T08:57:18.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.476+0000 7fc4cebf9640 1 -- start start 2024-04-25T08:57:18.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.477+0000 7fc4cebf9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4c80b1cd0 0x7fc4c80b6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:18.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.477+0000 7fc4cebf9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4c80d17b0 0x7fc4c80b01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:18.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.477+0000 7fc4cebf9640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc4c815dd80 0x7fc4c8152590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:18.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.477+0000 7fc4cebf9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c80b2c40 con 0x7fc4c80b1cd0 2024-04-25T08:57:18.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.477+0000 7fc4cebf9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c80cf330 con 0x7fc4c80d17b0 2024-04-25T08:57:18.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.477+0000 7fc4cebf9640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c80b2dc0 con 0x7fc4c815dd80 2024-04-25T08:57:18.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.477+0000 7fc4cc995640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4c80b1cd0 0x7fc4c80b6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:18.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.477+0000 7fc4cc995640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4c80b1cd0 0x7fc4c80b6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44266/0 (socket says 172.21.15.43:44266) 2024-04-25T08:57:18.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.477+0000 7fc4cc995640 1 -- 172.21.15.43:0/895333482 learned_addr learned my addr 172.21.15.43:0/895333482 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:18.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.477+0000 7fc4cd196640 1 --2- 172.21.15.43:0/895333482 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc4c815dd80 0x7fc4c8152590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:18.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.477+0000 7fc4c7fff640 1 --2- 172.21.15.43:0/895333482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4c80d17b0 0x7fc4c80b01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:18.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.477+0000 7fc4cc995640 1 -- 172.21.15.43:0/895333482 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc4c815dd80 msgr2=0x7fc4c8152590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:18.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.477+0000 7fc4cc995640 1 --2- 172.21.15.43:0/895333482 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc4c815dd80 0x7fc4c8152590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.477+0000 7fc4cc995640 1 -- 172.21.15.43:0/895333482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4c80d17b0 msgr2=0x7fc4c80b01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:18.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.477+0000 7fc4cc995640 1 --2- 172.21.15.43:0/895333482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4c80d17b0 0x7fc4c80b01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.477+0000 7fc4cc995640 1 -- 172.21.15.43:0/895333482 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c80cc0f0 con 0x7fc4c80b1cd0 2024-04-25T08:57:18.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.477+0000 7fc4cc995640 1 --2- 172.21.15.43:0/895333482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4c80b1cd0 0x7fc4c80b6720 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc000c00 tx=0x7fc4bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:18.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.477+0000 7fc4c57fa640 1 -- 172.21.15.43:0/895333482 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc037040 con 0x7fc4c80b1cd0 2024-04-25T08:57:18.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.478+0000 7fc4c57fa640 1 -- 172.21.15.43:0/895333482 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4bc042070 con 0x7fc4c80b1cd0 2024-04-25T08:57:18.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.478+0000 7fc4cebf9640 1 -- 172.21.15.43:0/895333482 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4c80cb050 con 0x7fc4c80b1cd0 2024-04-25T08:57:18.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.478+0000 7fc4c57fa640 1 -- 172.21.15.43:0/895333482 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc030880 con 0x7fc4c80b1cd0 2024-04-25T08:57:18.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.478+0000 7fc4cebf9640 1 -- 172.21.15.43:0/895333482 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c80cb610 con 0x7fc4c80b1cd0 2024-04-25T08:57:18.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.479+0000 7fc4c57fa640 1 -- 172.21.15.43:0/895333482 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc4bc02ad10 con 0x7fc4c80b1cd0 2024-04-25T08:57:18.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.479+0000 7fc4c57fa640 1 --2- 172.21.15.43:0/895333482 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc4ac06b500 0x7fc4ac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:18.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.479+0000 7fc4c57fa640 1 -- 172.21.15.43:0/895333482 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc4bc0aa560 con 0x7fc4c80b1cd0 2024-04-25T08:57:18.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.480+0000 7fc4c7fff640 1 --2- 172.21.15.43:0/895333482 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc4ac06b500 0x7fc4ac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:18.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.480+0000 7fc4c7fff640 1 --2- 172.21.15.43:0/895333482 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc4ac06b500 0x7fc4ac06d9c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc4c80cc300 tx=0x7fc4b4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:18.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.480+0000 7fc4d19d8740 1 Processor -- start 2024-04-25T08:57:18.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.480+0000 7fc4d19d8740 1 -- start start 2024-04-25T08:57:18.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.481+0000 7fc4d19d8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584a267ff10 0x5584a2682070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:18.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.481+0000 7fc4d19d8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584a26bc5e0 0x5584a26802f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:18.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.481+0000 7fc4d19d8740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5584a26aa9d0 0x5584a26aae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:18.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.481+0000 7fc4d19d8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584a25b1fe0 con 0x5584a267ff10 2024-04-25T08:57:18.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.481+0000 7fc4d19d8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584a25e3d20 con 0x5584a26bc5e0 2024-04-25T08:57:18.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.481+0000 7fc4d19d8740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x5584a258cfa0 con 0x5584a26aa9d0 2024-04-25T08:57:18.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.481+0000 7fc4cc995640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584a267ff10 0x5584a2682070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:18.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.481+0000 7fc4cc995640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584a267ff10 0x5584a2682070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44270/0 (socket says 172.21.15.43:44270) 2024-04-25T08:57:18.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.481+0000 7fc4cc995640 1 -- 172.21.15.43:0/2163812587 learned_addr learned my addr 172.21.15.43:0/2163812587 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:18.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.481+0000 7fc4c7fff640 1 --2- 172.21.15.43:0/2163812587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584a26bc5e0 0x5584a26802f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:18.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.481+0000 7fc4cd196640 1 --2- 172.21.15.43:0/2163812587 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5584a26aa9d0 0x5584a26aae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:18.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.482+0000 7fc4cc995640 1 -- 172.21.15.43:0/2163812587 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5584a26aa9d0 msgr2=0x5584a26aae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:18.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.482+0000 7fc4cc995640 1 --2- 172.21.15.43:0/2163812587 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5584a26aa9d0 0x5584a26aae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.482+0000 7fc4cc995640 1 -- 172.21.15.43:0/2163812587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584a26bc5e0 msgr2=0x5584a26802f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:18.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.482+0000 7fc4cc995640 1 --2- 172.21.15.43:0/2163812587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584a26bc5e0 0x5584a26802f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.482+0000 7fc4cc995640 1 -- 172.21.15.43:0/2163812587 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584a23c4b00 con 0x5584a267ff10 2024-04-25T08:57:18.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.482+0000 7fc4cd196640 1 --2- 172.21.15.43:0/2163812587 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5584a26aa9d0 0x5584a26aae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:18.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.482+0000 7fc4cc995640 1 --2- 172.21.15.43:0/2163812587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584a267ff10 0x5584a2682070 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc0a8b40 tx=0x7fc4bc009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed2f84a7e0a215fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:18.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.482+0000 7fc4b27fc640 1 -- 172.21.15.43:0/2163812587 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc081480 con 0x5584a267ff10 2024-04-25T08:57:18.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.482+0000 7fc4b27fc640 1 -- 172.21.15.43:0/2163812587 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4bc077120 con 0x5584a267ff10 2024-04-25T08:57:18.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.482+0000 7fc4b27fc640 1 -- 172.21.15.43:0/2163812587 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc075490 con 0x5584a267ff10 2024-04-25T08:57:18.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.482+0000 7fc4d19d8740 1 -- 172.21.15.43:0/2163812587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584a267ff10 msgr2=0x5584a2682070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:18.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.482+0000 7fc4d19d8740 1 --2- 172.21.15.43:0/2163812587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584a267ff10 0x5584a2682070 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc0a8b40 tx=0x7fc4bc009240 comp rx=0 tx=0).stop 2024-04-25T08:57:18.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.482+0000 7fc4d19d8740 1 -- 172.21.15.43:0/2163812587 shutdown_connections 2024-04-25T08:57:18.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.482+0000 7fc4d19d8740 1 --2- 172.21.15.43:0/2163812587 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5584a26aa9d0 0x5584a26aae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.482+0000 7fc4d19d8740 1 --2- 172.21.15.43:0/2163812587 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584a26bc5e0 0x5584a26802f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.482+0000 7fc4d19d8740 1 --2- 172.21.15.43:0/2163812587 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584a267ff10 0x5584a2682070 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.483+0000 7fc4d19d8740 1 -- 172.21.15.43:0/2163812587 >> 172.21.15.43:0/2163812587 conn(0x5584a2648100 msgr2=0x5584a26c59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:18.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.483+0000 7fc4d19d8740 1 -- 172.21.15.43:0/2163812587 shutdown_connections 2024-04-25T08:57:18.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.483+0000 7fc4d19d8740 1 -- 172.21.15.43:0/2163812587 wait complete. 2024-04-25T08:57:18.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.483+0000 7fc4d19d8740 1 -- start start 2024-04-25T08:57:18.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.483+0000 7fc4d19d8740 1 Processor -- start 2024-04-25T08:57:18.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.484+0000 7fc4d19d8740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5584a267ff10 0x5584a26c5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:18.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.484+0000 7fc4d19d8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584a26aa9d0 0x5584a26d8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:18.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.484+0000 7fc4d19d8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584a26bc5e0 0x5584a26c7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:18.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.484+0000 7fc4d19d8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584a2582750 con 0x5584a26bc5e0 2024-04-25T08:57:18.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.484+0000 7fc4d19d8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584a25dfdc0 con 0x5584a26aa9d0 2024-04-25T08:57:18.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.484+0000 7fc4d19d8740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x5584a25b1fe0 con 0x5584a267ff10 2024-04-25T08:57:18.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.484+0000 7fc4d19d8740 10 client.0 _ll_register_callbacks cb 0x5584a259aee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:18.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.484+0000 7fc4cd196640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584a26bc5e0 0x5584a26c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:18.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.484+0000 7fc4cd196640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584a26bc5e0 0x5584a26c7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44284/0 (socket says 172.21.15.43:44284) 2024-04-25T08:57:18.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.484+0000 7fc4cd196640 1 -- 172.21.15.43:0/3132304025 learned_addr learned my addr 172.21.15.43:0/3132304025 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:18.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.484+0000 7fc4cc995640 1 --2- 172.21.15.43:0/3132304025 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5584a267ff10 0x5584a26c5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:18.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.484+0000 7fc4c7fff640 1 --2- 172.21.15.43:0/3132304025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584a26aa9d0 0x5584a26d8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:18.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.485+0000 7fc4cd196640 1 -- 172.21.15.43:0/3132304025 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5584a267ff10 msgr2=0x5584a26c5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:18.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.485+0000 7fc4cd196640 1 --2- 172.21.15.43:0/3132304025 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5584a267ff10 0x5584a26c5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.485+0000 7fc4cd196640 1 -- 172.21.15.43:0/3132304025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584a26aa9d0 msgr2=0x5584a26d8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:18.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.485+0000 7fc4cd196640 1 --2- 172.21.15.43:0/3132304025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584a26aa9d0 0x5584a26d8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.485+0000 7fc4cd196640 1 -- 172.21.15.43:0/3132304025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5584a22b83f0 con 0x5584a26bc5e0 2024-04-25T08:57:18.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.485+0000 7fc4c7fff640 1 --2- 172.21.15.43:0/3132304025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584a26aa9d0 0x5584a26d8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:18.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.485+0000 7fc4cc995640 1 --2- 172.21.15.43:0/3132304025 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5584a267ff10 0x5584a26c5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:18.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.485+0000 7fc4cd196640 1 --2- 172.21.15.43:0/3132304025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584a26bc5e0 0x5584a26c7080 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0007da0 tx=0x7fc4c0012610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7b4c21e59992611 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:18.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.485+0000 7fc4a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:57:18.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.485+0000 7fc4a77fe640 1 -- 172.21.15.43:0/3132304025 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c001c020 con 0x5584a26bc5e0 2024-04-25T08:57:18.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.485+0000 7fc4a77fe640 1 -- 172.21.15.43:0/3132304025 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc4c0018670 con 0x5584a26bc5e0 2024-04-25T08:57:18.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.485+0000 7fc4d19d8740 1 -- 172.21.15.43:0/3132304025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc4bc077120 con 0x5584a26bc5e0 2024-04-25T08:57:18.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.485+0000 7fc4a77fe640 1 -- 172.21.15.43:0/3132304025 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4c0026050 con 0x5584a26bc5e0 2024-04-25T08:57:18.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.485+0000 7fc4d19d8740 10 client.5901 fetch_fsmap learned FSMap version 21 2024-04-25T08:57:18.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.485+0000 7fc4d19d8740 1 -- 172.21.15.43:0/3132304025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x5584a2425290 con 0x5584a26bc5e0 2024-04-25T08:57:18.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.485+0000 7fc4a77fe640 1 -- 172.21.15.43:0/3132304025 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4bc077120 con 0x5584a26bc5e0 2024-04-25T08:57:18.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.485+0000 7fc4a77fe640 1 -- 172.21.15.43:0/3132304025 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc4c0029020 con 0x5584a26bc5e0 2024-04-25T08:57:18.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.485+0000 7fc4d19d8740 10 client.5901 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T08:57:18.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.485+0000 7fc4d19d8740 10 client.5901 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T08:57:18.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.485+0000 7fc4d19d8740 20 client.5901 populate_metadata read hostname 'smithi043' 2024-04-25T08:57:18.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.485+0000 7fc4d19d8740 1 --2- 172.21.15.43:0/3132304025 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5584a26aaea0 0x5584a26ab2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:18.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.485+0000 7fc4d19d8740 4 client.5901 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:57:18.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.485+0000 7fc4d19d8740 1 -- 172.21.15.43:0/3132304025 --> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5584a25f1260 con 0x5584a26aaea0 2024-04-25T08:57:18.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.486+0000 7fc4cc995640 1 --2- 172.21.15.43:0/3132304025 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5584a26aaea0 0x5584a26ab2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:18.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.486+0000 7fc4cc995640 1 --2- 172.21.15.43:0/3132304025 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5584a26aaea0 0x5584a26ab2a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5aa86c80646b978 server_cookie=be6c468bf368d8fc in_seq=0 out_seq=0 2024-04-25T08:57:18.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.486+0000 7fc4a77fe640 10 client.5901 ms_handle_connect on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:18.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.486+0000 7fc4a77fe640 1 -- 172.21.15.43:0/3132304025 <== mds.0 v2:172.21.15.193:6836/4164634380 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5584a25f1260 con 0x5584a26aaea0 2024-04-25T08:57:18.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.486+0000 7fc4a77fe640 10 client.5901 handle_command_reply: tid=0 2024-04-25T08:57:18.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.487+0000 7fc4d19d8740 1 client.5901 shutdown 2024-04-25T08:57:18.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.487+0000 7fc4d19d8740 1 -- 172.21.15.43:0/3132304025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584a26bc5e0 msgr2=0x5584a26c7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:18.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.487+0000 7fc4d19d8740 1 --2- 172.21.15.43:0/3132304025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584a26bc5e0 0x5584a26c7080 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0007da0 tx=0x7fc4c0012610 comp rx=0 tx=0).stop 2024-04-25T08:57:18.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.487+0000 7fc4d19d8740 1 -- 172.21.15.43:0/3132304025 shutdown_connections 2024-04-25T08:57:18.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.487+0000 7fc4d19d8740 1 --2- 172.21.15.43:0/3132304025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584a26bc5e0 0x5584a26c7080 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.487+0000 7fc4d19d8740 1 --2- 172.21.15.43:0/3132304025 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5584a26aaea0 0x5584a26ab2a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.487+0000 7fc4d19d8740 1 --2- 172.21.15.43:0/3132304025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584a26aa9d0 0x5584a26d8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.487+0000 7fc4d19d8740 1 --2- 172.21.15.43:0/3132304025 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5584a267ff10 0x5584a26c5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.487+0000 7fc4a77fe640 0 client.5901 ms_handle_reset on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:18.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.487+0000 7fc4d19d8740 1 -- 172.21.15.43:0/3132304025 >> 172.21.15.43:0/3132304025 conn(0x5584a2648100 msgr2=0x5584a26c8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:18.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.487+0000 7fc4d19d8740 1 -- 172.21.15.43:0/3132304025 shutdown_connections 2024-04-25T08:57:18.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.487+0000 7fc4d19d8740 1 -- 172.21.15.43:0/3132304025 wait complete. 2024-04-25T08:57:18.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.487+0000 7fc4d19d8740 20 client.5901 trim_cache size 0 max 16384 2024-04-25T08:57:18.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.503+0000 7fc4d19d8740 1 Processor -- start 2024-04-25T08:57:18.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.503+0000 7fc4d19d8740 1 -- start start 2024-04-25T08:57:18.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.503+0000 7fc4d19d8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584a267ff10 0x5584a26c8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:18.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.503+0000 7fc4d19d8740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5584a26aa9d0 0x5584a26c92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:18.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.503+0000 7fc4d19d8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584a26bc5e0 0x5584a26a7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:18.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.503+0000 7fc4d19d8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c81fff40 con 0x5584a26bc5e0 2024-04-25T08:57:18.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.503+0000 7fc4d19d8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c81ff140 con 0x5584a267ff10 2024-04-25T08:57:18.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.503+0000 7fc4d19d8740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c815fc60 con 0x5584a26aa9d0 2024-04-25T08:57:18.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.504+0000 7fc4cd196640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584a26bc5e0 0x5584a26a7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:18.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.504+0000 7fc4cc995640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584a267ff10 0x5584a26c8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:18.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.504+0000 7fc4cc995640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584a267ff10 0x5584a26c8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:53470/0 (socket says 172.21.15.43:53470) 2024-04-25T08:57:18.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.504+0000 7fc4cc995640 1 -- 172.21.15.43:0/4020369282 learned_addr learned my addr 172.21.15.43:0/4020369282 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:18.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.504+0000 7fc4cd196640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584a26bc5e0 0x5584a26a7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44300/0 (socket says 172.21.15.43:44300) 2024-04-25T08:57:18.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.504+0000 7fc4c7fff640 1 --2- 172.21.15.43:0/4020369282 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5584a26aa9d0 0x5584a26c92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:18.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.504+0000 7fc4cd196640 1 -- 172.21.15.43:0/4020369282 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5584a26aa9d0 msgr2=0x5584a26c92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:18.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.504+0000 7fc4cd196640 1 --2- 172.21.15.43:0/4020369282 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5584a26aa9d0 0x5584a26c92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.504+0000 7fc4cd196640 1 -- 172.21.15.43:0/4020369282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584a267ff10 msgr2=0x5584a26c8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:18.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.504+0000 7fc4cd196640 1 --2- 172.21.15.43:0/4020369282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584a267ff10 0x5584a26c8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.504+0000 7fc4cd196640 1 -- 172.21.15.43:0/4020369282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584a22914b0 con 0x5584a26bc5e0 2024-04-25T08:57:18.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.504+0000 7fc4cd196640 1 --2- 172.21.15.43:0/4020369282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584a26bc5e0 0x5584a26a7c80 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x5584a26c8d70 tx=0x7fc4c0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1099fe5d99adb235 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:18.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.504+0000 7fc4b1ffb640 1 -- 172.21.15.43:0/4020369282 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c0030070 con 0x5584a26bc5e0 2024-04-25T08:57:18.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.504+0000 7fc4b1ffb640 1 -- 172.21.15.43:0/4020369282 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c0018ca0 con 0x5584a26bc5e0 2024-04-25T08:57:18.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.504+0000 7fc4b1ffb640 1 -- 172.21.15.43:0/4020369282 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c001c050 con 0x5584a26bc5e0 2024-04-25T08:57:18.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.505+0000 7fc4d19d8740 1 -- 172.21.15.43:0/4020369282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584a26bc5e0 msgr2=0x5584a26a7c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:18.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.505+0000 7fc4d19d8740 1 --2- 172.21.15.43:0/4020369282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584a26bc5e0 0x5584a26a7c80 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x5584a26c8d70 tx=0x7fc4c0018000 comp rx=0 tx=0).stop 2024-04-25T08:57:18.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.505+0000 7fc4d19d8740 1 -- 172.21.15.43:0/4020369282 shutdown_connections 2024-04-25T08:57:18.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.505+0000 7fc4d19d8740 1 --2- 172.21.15.43:0/4020369282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584a26bc5e0 0x5584a26a7c80 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.505+0000 7fc4d19d8740 1 --2- 172.21.15.43:0/4020369282 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5584a26aa9d0 0x5584a26c92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.505+0000 7fc4d19d8740 1 --2- 172.21.15.43:0/4020369282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584a267ff10 0x5584a26c8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.505+0000 7fc4d19d8740 1 -- 172.21.15.43:0/4020369282 >> 172.21.15.43:0/4020369282 conn(0x5584a2648100 msgr2=0x5584a26e7a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:18.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.505+0000 7fc4d19d8740 1 -- 172.21.15.43:0/4020369282 shutdown_connections 2024-04-25T08:57:18.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.505+0000 7fc4d19d8740 1 -- 172.21.15.43:0/4020369282 wait complete. 2024-04-25T08:57:18.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.505+0000 7fc4d19d8740 1 -- start start 2024-04-25T08:57:18.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.506+0000 7fc4d19d8740 1 Processor -- start 2024-04-25T08:57:18.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.506+0000 7fc4d19d8740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5584a267ff10 0x5584a272fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:18.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.507+0000 7fc4d19d8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584a26aa9d0 0x5584a26a77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:18.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.507+0000 7fc4d19d8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584a26bc5e0 0x5584a26c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:18.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.507+0000 7fc4d19d8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584a267a420 con 0x5584a26bc5e0 2024-04-25T08:57:18.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.507+0000 7fc4d19d8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c8160260 con 0x5584a26aa9d0 2024-04-25T08:57:18.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.507+0000 7fc4d19d8740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x5584a267c9b0 con 0x5584a267ff10 2024-04-25T08:57:18.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.507+0000 7fc4d19d8740 10 client.0 _ll_register_callbacks cb 0x7fc4bc0094d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:18.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.507+0000 7fc4cd196640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584a26bc5e0 0x5584a26c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:18.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.507+0000 7fc4c7fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584a26aa9d0 0x5584a26a77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:18.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.507+0000 7fc4cc995640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5584a267ff10 0x5584a272fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:18.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.507+0000 7fc4c7fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584a26aa9d0 0x5584a26a77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:53474/0 (socket says 172.21.15.43:53474) 2024-04-25T08:57:18.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.507+0000 7fc4c7fff640 1 -- 172.21.15.43:0/1387710615 learned_addr learned my addr 172.21.15.43:0/1387710615 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:18.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.507+0000 7fc4cd196640 1 -- 172.21.15.43:0/1387710615 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5584a267ff10 msgr2=0x5584a272fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:18.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.507+0000 7fc4cd196640 1 --2- 172.21.15.43:0/1387710615 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5584a267ff10 0x5584a272fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.507+0000 7fc4cd196640 1 -- 172.21.15.43:0/1387710615 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584a26aa9d0 msgr2=0x5584a26a77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:18.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.507+0000 7fc4cd196640 1 --2- 172.21.15.43:0/1387710615 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584a26aa9d0 0x5584a26a77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.507+0000 7fc4cd196640 1 -- 172.21.15.43:0/1387710615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5584a26e7cc0 con 0x5584a26bc5e0 2024-04-25T08:57:18.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.507+0000 7fc4cc995640 1 --2- 172.21.15.43:0/1387710615 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5584a267ff10 0x5584a272fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:57:18.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.507+0000 7fc4c7fff640 1 --2- 172.21.15.43:0/1387710615 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584a26aa9d0 0x5584a26a77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:57:18.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.507+0000 7fc4cd196640 1 --2- 172.21.15.43:0/1387710615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584a26bc5e0 0x5584a26c8cc0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0000f80 tx=0x7fc4c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c16daf3b0a84020 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:18.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.507+0000 7fc4a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:57:18.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.508+0000 7fc4a77fe640 1 -- 172.21.15.43:0/1387710615 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c001c030 con 0x5584a26bc5e0 2024-04-25T08:57:18.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.508+0000 7fc4a77fe640 1 -- 172.21.15.43:0/1387710615 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc4c0018a60 con 0x5584a26bc5e0 2024-04-25T08:57:18.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.508+0000 7fc4d19d8740 1 -- 172.21.15.43:0/1387710615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc4c0018ca0 con 0x5584a26bc5e0 2024-04-25T08:57:18.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.508+0000 7fc4a77fe640 1 -- 172.21.15.43:0/1387710615 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4c0030040 con 0x5584a26bc5e0 2024-04-25T08:57:18.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.508+0000 7fc4d19d8740 10 client.5907 fetch_fsmap learned FSMap version 21 2024-04-25T08:57:18.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.508+0000 7fc4d19d8740 1 -- 172.21.15.43:0/1387710615 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x5584a27301a0 con 0x5584a26bc5e0 2024-04-25T08:57:18.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.508+0000 7fc4a77fe640 1 -- 172.21.15.43:0/1387710615 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c0018ca0 con 0x5584a26bc5e0 2024-04-25T08:57:18.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.508+0000 7fc4a77fe640 1 -- 172.21.15.43:0/1387710615 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc4c0029040 con 0x5584a26bc5e0 2024-04-25T08:57:18.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.508+0000 7fc4d19d8740 10 client.5907 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T08:57:18.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.508+0000 7fc4d19d8740 10 client.5907 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T08:57:18.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.508+0000 7fc4d19d8740 20 client.5907 populate_metadata read hostname 'smithi043' 2024-04-25T08:57:18.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.508+0000 7fc4d19d8740 1 --2- 172.21.15.43:0/1387710615 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5584a2729480 0x5584a26c9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:18.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.508+0000 7fc4d19d8740 4 client.5907 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}] 2024-04-25T08:57:18.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.508+0000 7fc4d19d8740 1 -- 172.21.15.43:0/1387710615 --> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}) v1 -- 0x5584a25eb5b0 con 0x5584a2729480 2024-04-25T08:57:18.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.508+0000 7fc4cc995640 1 --2- 172.21.15.43:0/1387710615 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5584a2729480 0x5584a26c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:18.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.509+0000 7fc4a77fe640 10 client.5907 ms_handle_connect on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:18.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.509+0000 7fc4cc995640 1 --2- 172.21.15.43:0/1387710615 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5584a2729480 0x5584a26c9200 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e1c6dd9406e6de9 server_cookie=aa0304e9306561ee in_seq=0 out_seq=0 2024-04-25T08:57:18.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.509+0000 7fc4a77fe640 1 -- 172.21.15.43:0/1387710615 <== mds.0 v2:172.21.15.193:6836/4164634380 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5584a25eb5b0 con 0x5584a2729480 2024-04-25T08:57:18.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.509+0000 7fc4a77fe640 10 client.5907 handle_command_reply: tid=0 2024-04-25T08:57:18.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.510+0000 7fc4d19d8740 1 client.5907 shutdown 2024-04-25T08:57:18.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.510+0000 7fc4d19d8740 1 -- 172.21.15.43:0/1387710615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584a26bc5e0 msgr2=0x5584a26c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:18.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.510+0000 7fc4d19d8740 1 --2- 172.21.15.43:0/1387710615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584a26bc5e0 0x5584a26c8cc0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0000f80 tx=0x7fc4c0004a40 comp rx=0 tx=0).stop 2024-04-25T08:57:18.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.510+0000 7fc4d19d8740 1 -- 172.21.15.43:0/1387710615 shutdown_connections 2024-04-25T08:57:18.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.510+0000 7fc4d19d8740 1 --2- 172.21.15.43:0/1387710615 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5584a26bc5e0 0x5584a26c8cc0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.510+0000 7fc4d19d8740 1 --2- 172.21.15.43:0/1387710615 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5584a2729480 0x5584a26c9200 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.510+0000 7fc4d19d8740 1 --2- 172.21.15.43:0/1387710615 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5584a26aa9d0 0x5584a26a77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.510+0000 7fc4d19d8740 1 --2- 172.21.15.43:0/1387710615 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5584a267ff10 0x5584a272fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.510+0000 7fc4a77fe640 0 client.5907 ms_handle_reset on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:18.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.510+0000 7fc4d19d8740 1 -- 172.21.15.43:0/1387710615 >> 172.21.15.43:0/1387710615 conn(0x5584a2648100 msgr2=0x5584a26def00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:18.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.510+0000 7fc4d19d8740 1 -- 172.21.15.43:0/1387710615 shutdown_connections 2024-04-25T08:57:18.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.510+0000 7fc4d19d8740 1 -- 172.21.15.43:0/1387710615 wait complete. 2024-04-25T08:57:18.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.510+0000 7fc4d19d8740 20 client.5907 trim_cache size 0 max 16384 2024-04-25T08:57:18.504 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T08:57:18.504 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-25T08:57:18.504 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "1a209b19-b19c-4251-854d-90d58d97192a", 2024-04-25T08:57:18.504 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-25T08:57:18.504 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T08:57:18.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.511+0000 7fc4cebf9640 1 -- 172.21.15.43:0/895333482 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc4ac06b500 msgr2=0x7fc4ac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:18.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.511+0000 7fc4cebf9640 1 --2- 172.21.15.43:0/895333482 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc4ac06b500 0x7fc4ac06d9c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc4c80cc300 tx=0x7fc4b4009040 comp rx=0 tx=0).stop 2024-04-25T08:57:18.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.511+0000 7fc4cebf9640 1 -- 172.21.15.43:0/895333482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4c80b1cd0 msgr2=0x7fc4c80b6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:18.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.511+0000 7fc4cebf9640 1 --2- 172.21.15.43:0/895333482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4c80b1cd0 0x7fc4c80b6720 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc000c00 tx=0x7fc4bc0046c0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.511+0000 7fc4cebf9640 1 -- 172.21.15.43:0/895333482 shutdown_connections 2024-04-25T08:57:18.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.511+0000 7fc4cebf9640 1 --2- 172.21.15.43:0/895333482 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc4c815dd80 0x7fc4c8152590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.511+0000 7fc4cebf9640 1 --2- 172.21.15.43:0/895333482 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc4ac06b500 0x7fc4ac06d9c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.511+0000 7fc4cebf9640 1 --2- 172.21.15.43:0/895333482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4c80d17b0 0x7fc4c80b01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.511+0000 7fc4cebf9640 1 --2- 172.21.15.43:0/895333482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4c80b1cd0 0x7fc4c80b6720 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.511+0000 7fc4cebf9640 1 -- 172.21.15.43:0/895333482 >> 172.21.15.43:0/895333482 conn(0x7fc4c8072aa0 msgr2=0x7fc4c80ae7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:18.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.511+0000 7fc4cebf9640 1 -- 172.21.15.43:0/895333482 shutdown_connections 2024-04-25T08:57:18.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.511+0000 7fc4cebf9640 1 -- 172.21.15.43:0/895333482 wait complete. 2024-04-25T08:57:18.511 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites' returned '{'return_code': 0, 'scrub_tag': '1a209b19-b19c-4251-854d-90d58d97192a', 'mode': 'asynchronous'}' 2024-04-25T08:57:18.511 INFO:tasks.cephfs.test_scrub_checks:First run: flushing /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T08:57:18.512 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2024-04-25T08:57:18.512 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:57:18.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.623+0000 7f597deb3640 1 Processor -- start 2024-04-25T08:57:18.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.623+0000 7f597deb3640 1 -- start start 2024-04-25T08:57:18.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.624+0000 7f597deb3640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f597815a250 0x7f5978070d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:18.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.624+0000 7f597deb3640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f59780b4110 0x7f5978071a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:18.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.624+0000 7f597deb3640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f597806dd30 con 0x7f59780715a0 2024-04-25T08:57:18.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.624+0000 7f597deb3640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5978071f40 con 0x7f59780b4110 2024-04-25T08:57:18.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.624+0000 7f597deb3640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f597806e010 con 0x7f597815a250 2024-04-25T08:57:18.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.624+0000 7f5977fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f59780b4110 0x7f5978071a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:18.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.624+0000 7f59777fe640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f597815a250 0x7f5978070d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:18.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.624+0000 7f5977fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f59780b4110 0x7f5978071a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44330/0 (socket says 172.21.15.43:44330) 2024-04-25T08:57:18.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.624+0000 7f59777fe640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f597815a250 0x7f5978070d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:58566/0 (socket says 172.21.15.43:58566) 2024-04-25T08:57:18.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.624+0000 7f5977fff640 1 -- 172.21.15.43:0/3294003768 learned_addr learned my addr 172.21.15.43:0/3294003768 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:18.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.625+0000 7f5977fff640 1 -- 172.21.15.43:0/3294003768 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f597815a250 msgr2=0x7f5978070d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:18.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.625+0000 7f5977fff640 1 --2- 172.21.15.43:0/3294003768 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f597815a250 0x7f5978070d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.625+0000 7f5977fff640 1 -- 172.21.15.43:0/3294003768 >> v1:172.21.15.193:6789/0 conn(0x7f59780715a0 legacy=0x7f597815dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:18.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.625+0000 7f59767fc640 1 -- 172.21.15.43:0/3294003768 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1038331725 0 0) 0x7f597806dd30 con 0x7f59780715a0 2024-04-25T08:57:18.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.625+0000 7f5977fff640 1 -- 172.21.15.43:0/3294003768 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59780b2900 con 0x7f59780b4110 2024-04-25T08:57:18.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.625+0000 7f5977fff640 1 --2- 172.21.15.43:0/3294003768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59780b4110 0x7f5978071a00 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f5970009e50 tx=0x7f597002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9744e9affdac4d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:18.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.625+0000 7f59767fc640 1 -- 172.21.15.43:0/3294003768 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5970036070 con 0x7f59780b4110 2024-04-25T08:57:18.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.625+0000 7f59767fc640 1 -- 172.21.15.43:0/3294003768 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5970031600 con 0x7f59780b4110 2024-04-25T08:57:18.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.625+0000 7f59767fc640 1 -- 172.21.15.43:0/3294003768 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59700319d0 con 0x7f59780b4110 2024-04-25T08:57:18.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.626+0000 7f597deb3640 1 -- 172.21.15.43:0/3294003768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59780b4110 msgr2=0x7f5978071a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:18.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.626+0000 7f597deb3640 1 --2- 172.21.15.43:0/3294003768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59780b4110 0x7f5978071a00 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f5970009e50 tx=0x7f597002a680 comp rx=0 tx=0).stop 2024-04-25T08:57:18.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.626+0000 7f597deb3640 1 -- 172.21.15.43:0/3294003768 shutdown_connections 2024-04-25T08:57:18.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.626+0000 7f597deb3640 1 --2- 172.21.15.43:0/3294003768 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59780b4110 0x7f5978071a00 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.626+0000 7f597deb3640 1 --2- 172.21.15.43:0/3294003768 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f597815a250 0x7f5978070d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.626+0000 7f597deb3640 1 -- 172.21.15.43:0/3294003768 >> 172.21.15.43:0/3294003768 conn(0x7f59780b1480 msgr2=0x7f59780b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:18.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.626+0000 7f597deb3640 1 -- 172.21.15.43:0/3294003768 shutdown_connections 2024-04-25T08:57:18.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.626+0000 7f597deb3640 1 -- 172.21.15.43:0/3294003768 wait complete. 2024-04-25T08:57:18.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.627+0000 7f597deb3640 1 Processor -- start 2024-04-25T08:57:18.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.627+0000 7f597deb3640 1 -- start start 2024-04-25T08:57:18.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.628+0000 7f597deb3640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f59780715a0 0x7f597815d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:18.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.628+0000 7f597deb3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59780b4110 0x7f59780af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:18.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.628+0000 7f597deb3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f597815a250 0x7f5978152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:18.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.628+0000 7f597deb3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59780aeea0 con 0x7f597815a250 2024-04-25T08:57:18.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.628+0000 7f597deb3640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59780712a0 con 0x7f59780b4110 2024-04-25T08:57:18.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.628+0000 7f597deb3640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59780af020 con 0x7f59780715a0 2024-04-25T08:57:18.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.628+0000 7f5977fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f597815a250 0x7f5978152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:18.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.628+0000 7f5977fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f597815a250 0x7f5978152150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44334/0 (socket says 172.21.15.43:44334) 2024-04-25T08:57:18.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.628+0000 7f5977fff640 1 -- 172.21.15.43:0/2356248381 learned_addr learned my addr 172.21.15.43:0/2356248381 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:18.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.628+0000 7f59777fe640 1 --2- 172.21.15.43:0/2356248381 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f59780715a0 0x7f597815d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:18.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.628+0000 7f5976ffd640 1 --2- 172.21.15.43:0/2356248381 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59780b4110 0x7f59780af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:18.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.629+0000 7f5977fff640 1 -- 172.21.15.43:0/2356248381 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f59780715a0 msgr2=0x7f597815d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:18.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.629+0000 7f5977fff640 1 --2- 172.21.15.43:0/2356248381 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f59780715a0 0x7f597815d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.629+0000 7f5977fff640 1 -- 172.21.15.43:0/2356248381 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59780b4110 msgr2=0x7f59780af1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:18.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.629+0000 7f5977fff640 1 --2- 172.21.15.43:0/2356248381 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59780b4110 0x7f59780af1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.629+0000 7f5977fff640 1 -- 172.21.15.43:0/2356248381 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59780af710 con 0x7f597815a250 2024-04-25T08:57:18.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.629+0000 7f5977fff640 1 --2- 172.21.15.43:0/2356248381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f597815a250 0x7f5978152150 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f5970034040 tx=0x7f5970004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:18.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.629+0000 7f595ffff640 1 -- 172.21.15.43:0/2356248381 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5970036050 con 0x7f597815a250 2024-04-25T08:57:18.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.629+0000 7f595ffff640 1 -- 172.21.15.43:0/2356248381 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5970042070 con 0x7f597815a250 2024-04-25T08:57:18.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.629+0000 7f597deb3640 1 -- 172.21.15.43:0/2356248381 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59781f6f60 con 0x7f597815a250 2024-04-25T08:57:18.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.629+0000 7f595ffff640 1 -- 172.21.15.43:0/2356248381 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5970009510 con 0x7f597815a250 2024-04-25T08:57:18.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.629+0000 7f597deb3640 1 -- 172.21.15.43:0/2356248381 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59781f7520 con 0x7f597815a250 2024-04-25T08:57:18.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.630+0000 7f597deb3640 1 -- 172.21.15.43:0/2356248381 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5978074ef0 con 0x7f597815a250 2024-04-25T08:57:18.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.631+0000 7f595ffff640 1 -- 172.21.15.43:0/2356248381 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5970007bf0 con 0x7f597815a250 2024-04-25T08:57:18.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.631+0000 7f595ffff640 1 --2- 172.21.15.43:0/2356248381 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f596406b4d0 0x7f596406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:18.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.631+0000 7f595ffff640 1 -- 172.21.15.43:0/2356248381 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f59700aad60 con 0x7f597815a250 2024-04-25T08:57:18.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.631+0000 7f59777fe640 1 --2- 172.21.15.43:0/2356248381 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f596406b4d0 0x7f596406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:18.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.632+0000 7f59777fe640 1 --2- 172.21.15.43:0/2356248381 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f596406b4d0 0x7f596406d990 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f59780cc0b0 tx=0x7f5968009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:18.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.635+0000 7f595ffff640 1 -- 172.21.15.43:0/2356248381 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5970079620 con 0x7f597815a250 2024-04-25T08:57:18.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.851+0000 7f597deb3640 1 -- 172.21.15.43:0/2356248381 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59780ae8e0 con 0x7f597815a250 2024-04-25T08:57:18.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.852+0000 7f595ffff640 1 -- 172.21.15.43:0/2356248381 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f597007d6c0 con 0x7f597815a250 2024-04-25T08:57:18.841 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:57:18.841 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1504450503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1504450503},{"type":"v1","addr":"172.21.15.193:6835","nonce":1504450503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5691,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2657133821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2657133821},{"type":"v1","addr":"172.21.15.43:6834","nonce":2657133821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":5694,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/4271431818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":4271431818},{"type":"v1","addr":"172.21.15.43:6835","nonce":4271431818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:57:01.856132+0000","modified":"2024-04-25T08:57:03.857410+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5709":{"gid":5709,"name":"d","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.193:6837/4164634380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4164634380},{"type":"v1","addr":"172.21.15.193:6837","nonce":4164634380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:57:18.842 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 21 2024-04-25T08:57:18.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.853+0000 7f597deb3640 1 -- 172.21.15.43:0/2356248381 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f596406b4d0 msgr2=0x7f596406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:18.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.853+0000 7f597deb3640 1 --2- 172.21.15.43:0/2356248381 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f596406b4d0 0x7f596406d990 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f59780cc0b0 tx=0x7f5968009190 comp rx=0 tx=0).stop 2024-04-25T08:57:18.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.853+0000 7f597deb3640 1 -- 172.21.15.43:0/2356248381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f597815a250 msgr2=0x7f5978152150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:18.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.853+0000 7f597deb3640 1 --2- 172.21.15.43:0/2356248381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f597815a250 0x7f5978152150 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f5970034040 tx=0x7f5970004a40 comp rx=0 tx=0).stop 2024-04-25T08:57:18.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.854+0000 7f597deb3640 1 -- 172.21.15.43:0/2356248381 shutdown_connections 2024-04-25T08:57:18.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.854+0000 7f597deb3640 1 --2- 172.21.15.43:0/2356248381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f597815a250 0x7f5978152150 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.854+0000 7f597deb3640 1 --2- 172.21.15.43:0/2356248381 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f596406b4d0 0x7f596406d990 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.854+0000 7f597deb3640 1 --2- 172.21.15.43:0/2356248381 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59780b4110 0x7f59780af1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.854+0000 7f597deb3640 1 --2- 172.21.15.43:0/2356248381 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f59780715a0 0x7f597815d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:18.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.854+0000 7f597deb3640 1 -- 172.21.15.43:0/2356248381 >> 172.21.15.43:0/2356248381 conn(0x7f59780b1480 msgr2=0x7f59780ce350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:18.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.854+0000 7f597deb3640 1 -- 172.21.15.43:0/2356248381 shutdown_connections 2024-04-25T08:57:18.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:18.854+0000 7f597deb3640 1 -- 172.21.15.43:0/2356248381 wait complete. 2024-04-25T08:57:18.855 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T08:57:18.962 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-04-25T08:57:18.962 INFO:teuthology.orchestra.run.smithi193.stdout: "return_code": 0 2024-04-25T08:57:18.962 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-04-25T08:57:18.971 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T08:57:18.971 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2024-04-25T08:57:18.971 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:57:18.971 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T08:57:19.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.086+0000 7f4d2305d640 1 Processor -- start 2024-04-25T08:57:19.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.086+0000 7f4d2305d640 1 -- start start 2024-04-25T08:57:19.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.087+0000 7f4d2305d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4d1c0aee80 0x7f4d1c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.087+0000 7f4d2305d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4d1c0cd2f0 0x7f4d1c15d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.087+0000 7f4d2305d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d1c15c280 con 0x7f4d1c06d600 2024-04-25T08:57:19.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.087+0000 7f4d2305d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d1c0af260 con 0x7f4d1c0aee80 2024-04-25T08:57:19.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.087+0000 7f4d2305d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c15c560 con 0x7f4d1c0cd2f0 2024-04-25T08:57:19.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.087+0000 7f4d20df9640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4d1c0aee80 0x7f4d1c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.087+0000 7f4d1bfff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4d1c06d600 0x7f4d1c0ccbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43308/0 (socket says 172.21.15.43:43308) 2024-04-25T08:57:19.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.087+0000 7f4d1bfff640 1 -- 172.21.15.43:0/3533855544 learned_addr learned my addr 172.21.15.43:0/3533855544 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:19.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.087+0000 7f4d20df9640 1 -- 172.21.15.43:0/3533855544 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4d1c0cd2f0 msgr2=0x7f4d1c15d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:19.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.087+0000 7f4d20df9640 1 --2- 172.21.15.43:0/3533855544 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4d1c0cd2f0 0x7f4d1c15d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.087+0000 7f4d20df9640 1 -- 172.21.15.43:0/3533855544 >> v1:172.21.15.43:6789/0 conn(0x7f4d1c06d600 legacy=0x7f4d1c0ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:19.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.087+0000 7f4d20df9640 1 -- 172.21.15.43:0/3533855544 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d1c15c840 con 0x7f4d1c0aee80 2024-04-25T08:57:19.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.087+0000 7f4d1b7fe640 1 -- 172.21.15.43:0/3533855544 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3541469142 0 0) 0x7f4d1c15c280 con 0x7f4d1c06d600 2024-04-25T08:57:19.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.088+0000 7f4d20df9640 1 --2- 172.21.15.43:0/3533855544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d1c0aee80 0x7f4d1c06d0c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f4d14009d90 tx=0x7f4d1402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5075dbd391c2a39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:19.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.088+0000 7f4d1b7fe640 1 -- 172.21.15.43:0/3533855544 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d14007740 con 0x7f4d1c0aee80 2024-04-25T08:57:19.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.088+0000 7f4d1b7fe640 1 -- 172.21.15.43:0/3533855544 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d140078e0 con 0x7f4d1c0aee80 2024-04-25T08:57:19.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.088+0000 7f4d1b7fe640 1 -- 172.21.15.43:0/3533855544 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d14007cb0 con 0x7f4d1c0aee80 2024-04-25T08:57:19.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.089+0000 7f4d2305d640 1 -- 172.21.15.43:0/3533855544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d1c0aee80 msgr2=0x7f4d1c06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:19.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.089+0000 7f4d2305d640 1 --2- 172.21.15.43:0/3533855544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d1c0aee80 0x7f4d1c06d0c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f4d14009d90 tx=0x7f4d1402a660 comp rx=0 tx=0).stop 2024-04-25T08:57:19.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.089+0000 7f4d2305d640 1 -- 172.21.15.43:0/3533855544 shutdown_connections 2024-04-25T08:57:19.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.089+0000 7f4d2305d640 1 --2- 172.21.15.43:0/3533855544 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4d1c0cd2f0 0x7f4d1c15d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.089+0000 7f4d2305d640 1 --2- 172.21.15.43:0/3533855544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d1c0aee80 0x7f4d1c06d0c0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.089+0000 7f4d2305d640 1 -- 172.21.15.43:0/3533855544 >> 172.21.15.43:0/3533855544 conn(0x7f4d1c06ab30 msgr2=0x7f4d1c06af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:19.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.089+0000 7f4d2305d640 1 -- 172.21.15.43:0/3533855544 shutdown_connections 2024-04-25T08:57:19.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.089+0000 7f4d2305d640 1 -- 172.21.15.43:0/3533855544 wait complete. 2024-04-25T08:57:19.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.090+0000 7f4d2305d640 1 Processor -- start 2024-04-25T08:57:19.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.090+0000 7f4d2305d640 1 -- start start 2024-04-25T08:57:19.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.091+0000 7f4d2305d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4d1c06d600 0x7f4d1c13ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.091+0000 7f4d2305d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d1c0aee80 0x7f4d1c1e4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.091+0000 7f4d2305d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d1c0cd2f0 0x7f4d1c1e46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.091+0000 7f4d2305d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c15dc50 con 0x7f4d1c0aee80 2024-04-25T08:57:19.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.091+0000 7f4d2305d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c15dad0 con 0x7f4d1c0cd2f0 2024-04-25T08:57:19.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.091+0000 7f4d2305d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c15ddd0 con 0x7f4d1c06d600 2024-04-25T08:57:19.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.091+0000 7f4d1bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d1c0aee80 0x7f4d1c1e4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.091+0000 7f4d1bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d1c0aee80 0x7f4d1c1e4180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44364/0 (socket says 172.21.15.43:44364) 2024-04-25T08:57:19.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.091+0000 7f4d1bfff640 1 -- 172.21.15.43:0/1343438213 learned_addr learned my addr 172.21.15.43:0/1343438213 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:19.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.091+0000 7f4d20df9640 1 --2- 172.21.15.43:0/1343438213 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4d1c06d600 0x7f4d1c13ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.091+0000 7f4d1bfff640 1 -- 172.21.15.43:0/1343438213 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4d1c06d600 msgr2=0x7f4d1c13ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:19.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.091+0000 7f4d215fa640 1 --2- 172.21.15.43:0/1343438213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d1c0cd2f0 0x7f4d1c1e46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.091+0000 7f4d1bfff640 1 --2- 172.21.15.43:0/1343438213 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4d1c06d600 0x7f4d1c13ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.091+0000 7f4d1bfff640 1 -- 172.21.15.43:0/1343438213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d1c0cd2f0 msgr2=0x7f4d1c1e46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:19.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.091+0000 7f4d1bfff640 1 --2- 172.21.15.43:0/1343438213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d1c0cd2f0 0x7f4d1c1e46c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.091+0000 7f4d1bfff640 1 -- 172.21.15.43:0/1343438213 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d1c156670 con 0x7f4d1c0aee80 2024-04-25T08:57:19.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.092+0000 7f4d1bfff640 1 --2- 172.21.15.43:0/1343438213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d1c0aee80 0x7f4d1c1e4180 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f4d1000c460 tx=0x7f4d1000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:19.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.092+0000 7f4d197fa640 1 -- 172.21.15.43:0/1343438213 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d10005ce0 con 0x7f4d1c0aee80 2024-04-25T08:57:19.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.092+0000 7f4d197fa640 1 -- 172.21.15.43:0/1343438213 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d10007070 con 0x7f4d1c0aee80 2024-04-25T08:57:19.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.092+0000 7f4d197fa640 1 -- 172.21.15.43:0/1343438213 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d10007460 con 0x7f4d1c0aee80 2024-04-25T08:57:19.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.092+0000 7f4d2305d640 1 -- 172.21.15.43:0/1343438213 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d1c1552d0 con 0x7f4d1c0aee80 2024-04-25T08:57:19.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.092+0000 7f4d2305d640 1 -- 172.21.15.43:0/1343438213 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d1c13fc00 con 0x7f4d1c0aee80 2024-04-25T08:57:19.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.093+0000 7f4d197fa640 1 -- 172.21.15.43:0/1343438213 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4d10007600 con 0x7f4d1c0aee80 2024-04-25T08:57:19.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.094+0000 7f4d197fa640 1 --2- 172.21.15.43:0/1343438213 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4d0806b4d0 0x7f4d0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.094+0000 7f4d197fa640 1 -- 172.21.15.43:0/1343438213 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4d10089ee0 con 0x7f4d1c0aee80 2024-04-25T08:57:19.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.094+0000 7f4d20df9640 1 --2- 172.21.15.43:0/1343438213 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4d0806b4d0 0x7f4d0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.095+0000 7f4d20df9640 1 --2- 172.21.15.43:0/1343438213 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4d0806b4d0 0x7f4d0806d990 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4d14034000 tx=0x7f4d1402f000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:19.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.095+0000 7f4d259fd740 1 Processor -- start 2024-04-25T08:57:19.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.095+0000 7f4d259fd740 1 -- start start 2024-04-25T08:57:19.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.096+0000 7f4d259fd740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55bfb2e1af60 0x55bfb2e1b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.096+0000 7f4d259fd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfb2dd0050 0x55bfb2dd0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.096+0000 7f4d259fd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bfb2dfba90 0x55bfb2dfbf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.096+0000 7f4d259fd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb2d37e60 con 0x55bfb2dd0050 2024-04-25T08:57:19.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.096+0000 7f4d259fd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb2ce1120 con 0x55bfb2dfba90 2024-04-25T08:57:19.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.096+0000 7f4d259fd740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bfb2d33f00 con 0x55bfb2e1af60 2024-04-25T08:57:19.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.096+0000 7f4d1bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfb2dd0050 0x55bfb2dd0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.096+0000 7f4d1bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfb2dd0050 0x55bfb2dd0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44366/0 (socket says 172.21.15.43:44366) 2024-04-25T08:57:19.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.096+0000 7f4d1bfff640 1 -- 172.21.15.43:0/11302151 learned_addr learned my addr 172.21.15.43:0/11302151 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:19.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.096+0000 7f4d215fa640 1 --2- 172.21.15.43:0/11302151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bfb2dfba90 0x55bfb2dfbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.096+0000 7f4d20df9640 1 --2- 172.21.15.43:0/11302151 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55bfb2e1af60 0x55bfb2e1b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.096+0000 7f4d1bfff640 1 -- 172.21.15.43:0/11302151 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55bfb2e1af60 msgr2=0x55bfb2e1b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:19.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.096+0000 7f4d1bfff640 1 --2- 172.21.15.43:0/11302151 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55bfb2e1af60 0x55bfb2e1b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.096+0000 7f4d1bfff640 1 -- 172.21.15.43:0/11302151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bfb2dfba90 msgr2=0x55bfb2dfbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:19.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.096+0000 7f4d1bfff640 1 --2- 172.21.15.43:0/11302151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bfb2dfba90 0x55bfb2dfbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.096+0000 7f4d1bfff640 1 -- 172.21.15.43:0/11302151 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfb2b18f00 con 0x55bfb2dd0050 2024-04-25T08:57:19.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.096+0000 7f4d1bfff640 1 --2- 172.21.15.43:0/11302151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfb2dd0050 0x55bfb2dd0490 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f4d10000fb0 tx=0x7f4d100580f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a73b5d7737ffa220 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:19.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.096+0000 7f4d027fc640 1 -- 172.21.15.43:0/11302151 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d10060d70 con 0x55bfb2dd0050 2024-04-25T08:57:19.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.097+0000 7f4d027fc640 1 -- 172.21.15.43:0/11302151 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d10056960 con 0x55bfb2dd0050 2024-04-25T08:57:19.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.097+0000 7f4d027fc640 1 -- 172.21.15.43:0/11302151 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d10054cd0 con 0x55bfb2dd0050 2024-04-25T08:57:19.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.097+0000 7f4d259fd740 1 -- 172.21.15.43:0/11302151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfb2dd0050 msgr2=0x55bfb2dd0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:19.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.097+0000 7f4d259fd740 1 --2- 172.21.15.43:0/11302151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfb2dd0050 0x55bfb2dd0490 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f4d10000fb0 tx=0x7f4d100580f0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.097+0000 7f4d259fd740 1 -- 172.21.15.43:0/11302151 shutdown_connections 2024-04-25T08:57:19.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.097+0000 7f4d259fd740 1 --2- 172.21.15.43:0/11302151 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bfb2dfba90 0x55bfb2dfbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.097+0000 7f4d259fd740 1 --2- 172.21.15.43:0/11302151 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfb2dd0050 0x55bfb2dd0490 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.097+0000 7f4d259fd740 1 --2- 172.21.15.43:0/11302151 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55bfb2e1af60 0x55bfb2e1b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.097+0000 7f4d259fd740 1 -- 172.21.15.43:0/11302151 >> 172.21.15.43:0/11302151 conn(0x55bfb2d9bf70 msgr2=0x55bfb2dd2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:19.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.097+0000 7f4d259fd740 1 -- 172.21.15.43:0/11302151 shutdown_connections 2024-04-25T08:57:19.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.097+0000 7f4d259fd740 1 -- 172.21.15.43:0/11302151 wait complete. 2024-04-25T08:57:19.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.098+0000 7f4d259fd740 1 -- start start 2024-04-25T08:57:19.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.098+0000 7f4d259fd740 1 Processor -- start 2024-04-25T08:57:19.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.099+0000 7f4d259fd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bfb2dd0050 0x55bfb2df7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.099+0000 7f4d259fd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfb2dfba90 0x55bfb2e16d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.099+0000 7f4d259fd740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55bfb2e1af60 0x55bfb2e01260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.099+0000 7f4d259fd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb2dd2b50 con 0x55bfb2dfba90 2024-04-25T08:57:19.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.099+0000 7f4d259fd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb2cd6cd0 con 0x55bfb2dd0050 2024-04-25T08:57:19.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.099+0000 7f4d259fd740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bfb2dd2e30 con 0x55bfb2e1af60 2024-04-25T08:57:19.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.099+0000 7f4d1bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfb2dfba90 0x55bfb2e16d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.099+0000 7f4d259fd740 10 client.0 _ll_register_callbacks cb 0x55bfb2a4c2e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:19.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.099+0000 7f4d1bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfb2dfba90 0x55bfb2e16d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44382/0 (socket says 172.21.15.43:44382) 2024-04-25T08:57:19.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.099+0000 7f4d1bfff640 1 -- 172.21.15.43:0/1438438901 learned_addr learned my addr 172.21.15.43:0/1438438901 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:19.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.099+0000 7f4d20df9640 1 --2- 172.21.15.43:0/1438438901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bfb2dd0050 0x55bfb2df7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.099+0000 7f4d215fa640 1 --2- 172.21.15.43:0/1438438901 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55bfb2e1af60 0x55bfb2e01260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.099+0000 7f4d1bfff640 1 -- 172.21.15.43:0/1438438901 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55bfb2e1af60 msgr2=0x55bfb2e01260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:19.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.099+0000 7f4d1bfff640 1 --2- 172.21.15.43:0/1438438901 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55bfb2e1af60 0x55bfb2e01260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.099+0000 7f4d1bfff640 1 -- 172.21.15.43:0/1438438901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bfb2dd0050 msgr2=0x55bfb2df7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:19.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.099+0000 7f4d1bfff640 1 --2- 172.21.15.43:0/1438438901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bfb2dd0050 0x55bfb2df7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.099+0000 7f4d1bfff640 1 -- 172.21.15.43:0/1438438901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bfb2a0c7f0 con 0x55bfb2dfba90 2024-04-25T08:57:19.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.099+0000 7f4d20df9640 1 --2- 172.21.15.43:0/1438438901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bfb2dd0050 0x55bfb2df7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:57:19.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.099+0000 7f4d1bfff640 1 --2- 172.21.15.43:0/1438438901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfb2dfba90 0x55bfb2e16d60 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f4d10054ac0 tx=0x7f4d1005b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d7bb728147d6d93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:19.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.099+0000 7f4cfb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:57:19.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.099+0000 7f4cfb7fe640 1 -- 172.21.15.43:0/1438438901 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d10060d70 con 0x55bfb2dfba90 2024-04-25T08:57:19.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.099+0000 7f4cfb7fe640 1 -- 172.21.15.43:0/1438438901 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4d100608e0 con 0x55bfb2dfba90 2024-04-25T08:57:19.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.099+0000 7f4d259fd740 1 -- 172.21.15.43:0/1438438901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4d10056960 con 0x55bfb2dfba90 2024-04-25T08:57:19.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.100+0000 7f4cfb7fe640 1 -- 172.21.15.43:0/1438438901 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d10060ae0 con 0x55bfb2dfba90 2024-04-25T08:57:19.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.100+0000 7f4d259fd740 10 client.5928 fetch_fsmap learned FSMap version 21 2024-04-25T08:57:19.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.100+0000 7f4d259fd740 1 -- 172.21.15.43:0/1438438901 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x55bfb2b79690 con 0x55bfb2dfba90 2024-04-25T08:57:19.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.100+0000 7f4cfb7fe640 1 -- 172.21.15.43:0/1438438901 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d10056960 con 0x55bfb2dfba90 2024-04-25T08:57:19.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.100+0000 7f4cfb7fe640 1 -- 172.21.15.43:0/1438438901 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4d10070020 con 0x55bfb2dfba90 2024-04-25T08:57:19.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.100+0000 7f4d259fd740 10 client.5928 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T08:57:19.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.100+0000 7f4d259fd740 10 client.5928 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T08:57:19.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.100+0000 7f4d259fd740 20 client.5928 populate_metadata read hostname 'smithi043' 2024-04-25T08:57:19.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.100+0000 7f4d259fd740 1 --2- 172.21.15.43:0/1438438901 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55bfb2e18a90 0x55bfb2e18e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.100+0000 7f4d259fd740 4 client.5928 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:57:19.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.100+0000 7f4d259fd740 1 -- 172.21.15.43:0/1438438901 --> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bfb2d453a0 con 0x55bfb2e18a90 2024-04-25T08:57:19.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.100+0000 7f4d20df9640 1 --2- 172.21.15.43:0/1438438901 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55bfb2e18a90 0x55bfb2e18e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.101+0000 7f4cfb7fe640 10 client.5928 ms_handle_connect on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:19.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.100+0000 7f4d20df9640 1 --2- 172.21.15.43:0/1438438901 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55bfb2e18a90 0x55bfb2e18e70 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b241c8cd30c7b64b server_cookie=9bfa27c618af924 in_seq=0 out_seq=0 2024-04-25T08:57:19.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.101+0000 7f4cfb7fe640 1 -- 172.21.15.43:0/1438438901 <== mds.0 v2:172.21.15.193:6836/4164634380 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bfb2d453a0 con 0x55bfb2e18a90 2024-04-25T08:57:19.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.101+0000 7f4cfb7fe640 10 client.5928 handle_command_reply: tid=0 2024-04-25T08:57:19.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.101+0000 7f4d259fd740 1 client.5928 shutdown 2024-04-25T08:57:19.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.101+0000 7f4d259fd740 1 -- 172.21.15.43:0/1438438901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfb2dfba90 msgr2=0x55bfb2e16d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:19.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.101+0000 7f4d259fd740 1 --2- 172.21.15.43:0/1438438901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfb2dfba90 0x55bfb2e16d60 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f4d10054ac0 tx=0x7f4d1005b040 comp rx=0 tx=0).stop 2024-04-25T08:57:19.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.101+0000 7f4d259fd740 1 -- 172.21.15.43:0/1438438901 shutdown_connections 2024-04-25T08:57:19.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.101+0000 7f4d259fd740 1 --2- 172.21.15.43:0/1438438901 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55bfb2e1af60 0x55bfb2e01260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.101+0000 7f4d259fd740 1 --2- 172.21.15.43:0/1438438901 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfb2dfba90 0x55bfb2e16d60 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.101+0000 7f4d259fd740 1 --2- 172.21.15.43:0/1438438901 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55bfb2e18a90 0x55bfb2e18e70 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.101+0000 7f4d259fd740 1 --2- 172.21.15.43:0/1438438901 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bfb2dd0050 0x55bfb2df7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.101+0000 7f4cfb7fe640 0 client.5928 ms_handle_reset on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:19.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.101+0000 7f4d259fd740 1 -- 172.21.15.43:0/1438438901 >> 172.21.15.43:0/1438438901 conn(0x55bfb2d9bf70 msgr2=0x55bfb2dce000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:19.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.101+0000 7f4d259fd740 1 -- 172.21.15.43:0/1438438901 shutdown_connections 2024-04-25T08:57:19.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.101+0000 7f4d259fd740 1 -- 172.21.15.43:0/1438438901 wait complete. 2024-04-25T08:57:19.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.101+0000 7f4d259fd740 20 client.5928 trim_cache size 0 max 16384 2024-04-25T08:57:19.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.117+0000 7f4d259fd740 1 Processor -- start 2024-04-25T08:57:19.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.117+0000 7f4d259fd740 1 -- start start 2024-04-25T08:57:19.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.117+0000 7f4d259fd740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55bfb2dd0050 0x55bfb2e19060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.117+0000 7f4d259fd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bfb2dfba90 0x55bfb2dfab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.117+0000 7f4d259fd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfb2e1af60 0x55bfb2dfb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.117+0000 7f4d259fd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c071e30 con 0x55bfb2e1af60 2024-04-25T08:57:19.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.117+0000 7f4d259fd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c071050 con 0x55bfb2dfba90 2024-04-25T08:57:19.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.117+0000 7f4d259fd740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c070450 con 0x55bfb2dd0050 2024-04-25T08:57:19.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.117+0000 7f4d215fa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfb2e1af60 0x55bfb2dfb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.117+0000 7f4d215fa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfb2e1af60 0x55bfb2dfb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44390/0 (socket says 172.21.15.43:44390) 2024-04-25T08:57:19.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.117+0000 7f4d215fa640 1 -- 172.21.15.43:0/3633373293 learned_addr learned my addr 172.21.15.43:0/3633373293 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:19.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.118+0000 7f4d20df9640 1 --2- 172.21.15.43:0/3633373293 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55bfb2dd0050 0x55bfb2e19060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.118+0000 7f4d215fa640 1 -- 172.21.15.43:0/3633373293 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55bfb2dd0050 msgr2=0x55bfb2e19060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:19.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.118+0000 7f4d215fa640 1 --2- 172.21.15.43:0/3633373293 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55bfb2dd0050 0x55bfb2e19060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.118+0000 7f4d215fa640 1 -- 172.21.15.43:0/3633373293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bfb2dfba90 msgr2=0x55bfb2dfab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:19.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.118+0000 7f4d215fa640 1 --2- 172.21.15.43:0/3633373293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bfb2dfba90 0x55bfb2dfab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.118+0000 7f4d215fa640 1 -- 172.21.15.43:0/3633373293 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfb2e170d0 con 0x55bfb2e1af60 2024-04-25T08:57:19.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.118+0000 7f4d20df9640 1 --2- 172.21.15.43:0/3633373293 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55bfb2dd0050 0x55bfb2e19060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:19.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.118+0000 7f4d215fa640 1 --2- 172.21.15.43:0/3633373293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfb2e1af60 0x55bfb2dfb0d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c00a890 tx=0x7f4d0c00ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f87b5ee1f53ae458 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:19.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.118+0000 7f4d01ffb640 1 -- 172.21.15.43:0/3633373293 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c012ca0 con 0x55bfb2e1af60 2024-04-25T08:57:19.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.118+0000 7f4d01ffb640 1 -- 172.21.15.43:0/3633373293 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0c012e40 con 0x55bfb2e1af60 2024-04-25T08:57:19.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.118+0000 7f4d01ffb640 1 -- 172.21.15.43:0/3633373293 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c01c660 con 0x55bfb2e1af60 2024-04-25T08:57:19.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.118+0000 7f4d259fd740 1 -- 172.21.15.43:0/3633373293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfb2e1af60 msgr2=0x55bfb2dfb0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:19.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.118+0000 7f4d259fd740 1 --2- 172.21.15.43:0/3633373293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfb2e1af60 0x55bfb2dfb0d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c00a890 tx=0x7f4d0c00ad50 comp rx=0 tx=0).stop 2024-04-25T08:57:19.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.119+0000 7f4d259fd740 1 -- 172.21.15.43:0/3633373293 shutdown_connections 2024-04-25T08:57:19.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.119+0000 7f4d259fd740 1 --2- 172.21.15.43:0/3633373293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfb2e1af60 0x55bfb2dfb0d0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.119+0000 7f4d259fd740 1 --2- 172.21.15.43:0/3633373293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bfb2dfba90 0x55bfb2dfab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.119+0000 7f4d259fd740 1 --2- 172.21.15.43:0/3633373293 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55bfb2dd0050 0x55bfb2e19060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.119+0000 7f4d259fd740 1 -- 172.21.15.43:0/3633373293 >> 172.21.15.43:0/3633373293 conn(0x55bfb2d9bf70 msgr2=0x55bfb2e15fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:19.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.119+0000 7f4d259fd740 1 -- 172.21.15.43:0/3633373293 shutdown_connections 2024-04-25T08:57:19.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.119+0000 7f4d259fd740 1 -- 172.21.15.43:0/3633373293 wait complete. 2024-04-25T08:57:19.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.119+0000 7f4d259fd740 1 -- start start 2024-04-25T08:57:19.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.119+0000 7f4d259fd740 1 Processor -- start 2024-04-25T08:57:19.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.120+0000 7f4d259fd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bfb2dd0050 0x55bfb2e8b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.120+0000 7f4d259fd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfb2dfba90 0x55bfb2e19060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.120+0000 7f4d259fd740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55bfb2e1af60 0x55bfb2e195a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.120+0000 7f4d259fd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb2e16f10 con 0x55bfb2dfba90 2024-04-25T08:57:19.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.120+0000 7f4d259fd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c070a50 con 0x55bfb2dd0050 2024-04-25T08:57:19.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.120+0000 7f4d259fd740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bfb2dd0b10 con 0x55bfb2e1af60 2024-04-25T08:57:19.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.120+0000 7f4d259fd740 10 client.0 _ll_register_callbacks cb 0x7f4d10054900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:19.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.120+0000 7f4d1bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfb2dfba90 0x55bfb2e19060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.120+0000 7f4d1bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfb2dfba90 0x55bfb2e19060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44400/0 (socket says 172.21.15.43:44400) 2024-04-25T08:57:19.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.120+0000 7f4d1bfff640 1 -- 172.21.15.43:0/335094738 learned_addr learned my addr 172.21.15.43:0/335094738 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:19.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.120+0000 7f4d20df9640 1 --2- 172.21.15.43:0/335094738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bfb2dd0050 0x55bfb2e8b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.120+0000 7f4d215fa640 1 --2- 172.21.15.43:0/335094738 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55bfb2e1af60 0x55bfb2e195a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.121+0000 7f4d1bfff640 1 -- 172.21.15.43:0/335094738 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55bfb2e1af60 msgr2=0x55bfb2e195a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:19.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.121+0000 7f4d1bfff640 1 --2- 172.21.15.43:0/335094738 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55bfb2e1af60 0x55bfb2e195a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.121+0000 7f4d1bfff640 1 -- 172.21.15.43:0/335094738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bfb2dd0050 msgr2=0x55bfb2e8b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:19.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.121+0000 7f4d1bfff640 1 --2- 172.21.15.43:0/335094738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bfb2dd0050 0x55bfb2e8b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.121+0000 7f4d1bfff640 1 -- 172.21.15.43:0/335094738 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bfb29e58b0 con 0x55bfb2dfba90 2024-04-25T08:57:19.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.121+0000 7f4d1bfff640 1 --2- 172.21.15.43:0/335094738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfb2dfba90 0x55bfb2e19060 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4d10054fe0 tx=0x7f4d100882e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850fca5c0b1fe6a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:19.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.121+0000 7f4cfb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:57:19.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.121+0000 7f4cfb7fe640 1 -- 172.21.15.43:0/335094738 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d10060d70 con 0x55bfb2dfba90 2024-04-25T08:57:19.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.121+0000 7f4cfb7fe640 1 -- 172.21.15.43:0/335094738 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4d10058ad0 con 0x55bfb2dfba90 2024-04-25T08:57:19.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.121+0000 7f4d259fd740 1 -- 172.21.15.43:0/335094738 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4d0c012e40 con 0x55bfb2dfba90 2024-04-25T08:57:19.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.121+0000 7f4cfb7fe640 1 -- 172.21.15.43:0/335094738 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4d10057fc0 con 0x55bfb2dfba90 2024-04-25T08:57:19.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.121+0000 7f4d259fd740 10 client.5934 fetch_fsmap learned FSMap version 21 2024-04-25T08:57:19.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.121+0000 7f4d259fd740 1 -- 172.21.15.43:0/335094738 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x55bfb2dfb190 con 0x55bfb2dfba90 2024-04-25T08:57:19.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.121+0000 7f4cfb7fe640 1 -- 172.21.15.43:0/335094738 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d0c012e40 con 0x55bfb2dfba90 2024-04-25T08:57:19.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.121+0000 7f4cfb7fe640 1 -- 172.21.15.43:0/335094738 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4d10057520 con 0x55bfb2dfba90 2024-04-25T08:57:19.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.121+0000 7f4d259fd740 10 client.5934 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T08:57:19.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.121+0000 7f4d259fd740 10 client.5934 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T08:57:19.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.121+0000 7f4d259fd740 20 client.5934 populate_metadata read hostname 'smithi043' 2024-04-25T08:57:19.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.121+0000 7f4d259fd740 1 --2- 172.21.15.43:0/335094738 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55bfb2dd25b0 0x55bfb2e2fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.121+0000 7f4d259fd740 4 client.5934 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}] 2024-04-25T08:57:19.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.121+0000 7f4d259fd740 1 -- 172.21.15.43:0/335094738 --> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}) v1 -- 0x55bfb2d3f6f0 con 0x55bfb2dd25b0 2024-04-25T08:57:19.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.122+0000 7f4d20df9640 1 --2- 172.21.15.43:0/335094738 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55bfb2dd25b0 0x55bfb2e2fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.122+0000 7f4cfb7fe640 10 client.5934 ms_handle_connect on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:19.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.122+0000 7f4d20df9640 1 --2- 172.21.15.43:0/335094738 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55bfb2dd25b0 0x55bfb2e2fd20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=186e3dd67982fe37 server_cookie=4396762007f1d12d in_seq=0 out_seq=0 2024-04-25T08:57:19.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.122+0000 7f4cfb7fe640 1 -- 172.21.15.43:0/335094738 <== mds.0 v2:172.21.15.193:6836/4164634380 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55bfb2d3f6f0 con 0x55bfb2dd25b0 2024-04-25T08:57:19.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.122+0000 7f4cfb7fe640 10 client.5934 handle_command_reply: tid=0 2024-04-25T08:57:19.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.122+0000 7f4d259fd740 1 client.5934 shutdown 2024-04-25T08:57:19.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.122+0000 7f4d259fd740 1 -- 172.21.15.43:0/335094738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfb2dfba90 msgr2=0x55bfb2e19060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:19.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.122+0000 7f4d259fd740 1 --2- 172.21.15.43:0/335094738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfb2dfba90 0x55bfb2e19060 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f4d10054fe0 tx=0x7f4d100882e0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.122+0000 7f4d259fd740 1 -- 172.21.15.43:0/335094738 shutdown_connections 2024-04-25T08:57:19.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.122+0000 7f4d259fd740 1 --2- 172.21.15.43:0/335094738 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55bfb2e1af60 0x55bfb2e195a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.122+0000 7f4d259fd740 1 --2- 172.21.15.43:0/335094738 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bfb2dfba90 0x55bfb2e19060 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.122+0000 7f4d259fd740 1 --2- 172.21.15.43:0/335094738 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55bfb2dd25b0 0x55bfb2e2fd20 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.123+0000 7f4d259fd740 1 --2- 172.21.15.43:0/335094738 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bfb2dd0050 0x55bfb2e8b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.123+0000 7f4d259fd740 1 -- 172.21.15.43:0/335094738 >> 172.21.15.43:0/335094738 conn(0x55bfb2d9bf70 msgr2=0x55bfb2dd3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:19.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.123+0000 7f4cfb7fe640 0 client.5934 ms_handle_reset on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:19.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.123+0000 7f4d259fd740 1 -- 172.21.15.43:0/335094738 shutdown_connections 2024-04-25T08:57:19.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.123+0000 7f4d259fd740 1 -- 172.21.15.43:0/335094738 wait complete. 2024-04-25T08:57:19.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.123+0000 7f4d259fd740 20 client.5934 trim_cache size 0 max 16384 2024-04-25T08:57:19.123 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T08:57:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-25T08:57:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "d8defa35-f36f-4955-92b8-791ee432a719", 2024-04-25T08:57:19.123 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-25T08:57:19.123 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T08:57:19.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.123+0000 7f4d2305d640 1 -- 172.21.15.43:0/1343438213 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4d0806b4d0 msgr2=0x7f4d0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:19.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.123+0000 7f4d2305d640 1 --2- 172.21.15.43:0/1343438213 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4d0806b4d0 0x7f4d0806d990 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f4d14034000 tx=0x7f4d1402f000 comp rx=0 tx=0).stop 2024-04-25T08:57:19.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.123+0000 7f4d2305d640 1 -- 172.21.15.43:0/1343438213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d1c0aee80 msgr2=0x7f4d1c1e4180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:19.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.123+0000 7f4d2305d640 1 --2- 172.21.15.43:0/1343438213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d1c0aee80 0x7f4d1c1e4180 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f4d1000c460 tx=0x7f4d1000c920 comp rx=0 tx=0).stop 2024-04-25T08:57:19.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.123+0000 7f4d2305d640 1 -- 172.21.15.43:0/1343438213 shutdown_connections 2024-04-25T08:57:19.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.123+0000 7f4d2305d640 1 --2- 172.21.15.43:0/1343438213 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4d0806b4d0 0x7f4d0806d990 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.123+0000 7f4d2305d640 1 --2- 172.21.15.43:0/1343438213 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4d1c0cd2f0 0x7f4d1c1e46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.123+0000 7f4d2305d640 1 --2- 172.21.15.43:0/1343438213 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4d1c0aee80 0x7f4d1c1e4180 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.123+0000 7f4d2305d640 1 --2- 172.21.15.43:0/1343438213 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4d1c06d600 0x7f4d1c13ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.123+0000 7f4d2305d640 1 -- 172.21.15.43:0/1343438213 >> 172.21.15.43:0/1343438213 conn(0x7f4d1c06ab30 msgr2=0x7f4d1c0b2670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:19.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.123+0000 7f4d2305d640 1 -- 172.21.15.43:0/1343438213 shutdown_connections 2024-04-25T08:57:19.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.123+0000 7f4d2305d640 1 -- 172.21.15.43:0/1343438213 wait complete. 2024-04-25T08:57:19.126 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' returned '{'return_code': 0, 'scrub_tag': 'd8defa35-f36f-4955-92b8-791ee432a719', 'mode': 'asynchronous'}' 2024-04-25T08:57:19.126 INFO:tasks.cephfs.test_scrub_checks:First run: flushing base dir / 2024-04-25T08:57:19.126 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2024-04-25T08:57:19.126 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:57:19.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.237+0000 7ff8d2c9e640 1 Processor -- start 2024-04-25T08:57:19.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.237+0000 7ff8d2c9e640 1 -- start start 2024-04-25T08:57:19.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.238+0000 7ff8d2c9e640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff8cc15df70 0x7ff8cc0cc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.238+0000 7ff8d2c9e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8cc0cbee0 con 0x7ff8cc06c510 2024-04-25T08:57:19.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.238+0000 7ff8d2c9e640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8cc0d0650 con 0x7ff8cc0b0870 2024-04-25T08:57:19.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.238+0000 7ff8d2c9e640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8cc0d0400 con 0x7ff8cc15df70 2024-04-25T08:57:19.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.238+0000 7ff8d0a3a640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff8cc06c510 0x7ff8cc06c8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:43320/0 (socket says 172.21.15.43:43320) 2024-04-25T08:57:19.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.238+0000 7ff8d0a3a640 1 -- 172.21.15.43:0/2218651407 learned_addr learned my addr 172.21.15.43:0/2218651407 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:19.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.239+0000 7ff8cbfff640 1 --2- 172.21.15.43:0/2218651407 >> v2:172.21.15.193:3300/0 conn(0x7ff8cc15df70 0x7ff8cc0cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.239+0000 7ff8cb7fe640 1 -- 172.21.15.43:0/2218651407 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2008923745 0 0) 0x7ff8cc0cbee0 con 0x7ff8cc06c510 2024-04-25T08:57:19.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.239+0000 7ff8cb7fe640 1 -- 172.21.15.43:0/2218651407 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8b8003520 con 0x7ff8cc06c510 2024-04-25T08:57:19.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.239+0000 7ff8cbfff640 1 -- 172.21.15.43:0/2218651407 >> v1:172.21.15.193:6789/0 conn(0x7ff8cc0b0870 legacy=0x7ff8cc0ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:19.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.239+0000 7ff8cbfff640 1 -- 172.21.15.43:0/2218651407 >> v1:172.21.15.43:6789/0 conn(0x7ff8cc06c510 legacy=0x7ff8cc06c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:19.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.239+0000 7ff8cbfff640 1 -- 172.21.15.43:0/2218651407 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8cc0d0930 con 0x7ff8cc15df70 2024-04-25T08:57:19.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.240+0000 7ff8cbfff640 1 --2- 172.21.15.43:0/2218651407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8cc15df70 0x7ff8cc0cc670 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc0097d0 tx=0x7ff8bc02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a16899ef2f070f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:19.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.240+0000 7ff8cb7fe640 1 -- 172.21.15.43:0/2218651407 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc037070 con 0x7ff8cc15df70 2024-04-25T08:57:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.240+0000 7ff8cb7fe640 1 -- 172.21.15.43:0/2218651407 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8bc032440 con 0x7ff8cc15df70 2024-04-25T08:57:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.240+0000 7ff8cb7fe640 1 -- 172.21.15.43:0/2218651407 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc032810 con 0x7ff8cc15df70 2024-04-25T08:57:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.240+0000 7ff8d2c9e640 1 -- 172.21.15.43:0/2218651407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8cc15df70 msgr2=0x7ff8cc0cc670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.240+0000 7ff8d2c9e640 1 --2- 172.21.15.43:0/2218651407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8cc15df70 0x7ff8cc0cc670 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc0097d0 tx=0x7ff8bc02a1c0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.240+0000 7ff8d2c9e640 1 -- 172.21.15.43:0/2218651407 shutdown_connections 2024-04-25T08:57:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.240+0000 7ff8d2c9e640 1 --2- 172.21.15.43:0/2218651407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8cc15df70 0x7ff8cc0cc670 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.240+0000 7ff8d2c9e640 1 -- 172.21.15.43:0/2218651407 >> 172.21.15.43:0/2218651407 conn(0x7ff8cc0d17f0 msgr2=0x7ff8cc0d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.240+0000 7ff8d2c9e640 1 -- 172.21.15.43:0/2218651407 shutdown_connections 2024-04-25T08:57:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.240+0000 7ff8d2c9e640 1 -- 172.21.15.43:0/2218651407 wait complete. 2024-04-25T08:57:19.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.242+0000 7ff8d2c9e640 1 Processor -- start 2024-04-25T08:57:19.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.242+0000 7ff8d2c9e640 1 -- start start 2024-04-25T08:57:19.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.242+0000 7ff8d2c9e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff8cc06c510 0x7ff8cc153ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.242+0000 7ff8d2c9e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8cc0b0870 0x7ff8cc15ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.242+0000 7ff8d2c9e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8cc15df70 0x7ff8cc0cca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.243+0000 7ff8d2c9e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8cc06bd10 con 0x7ff8cc15df70 2024-04-25T08:57:19.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.243+0000 7ff8d2c9e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8cc0d0c10 con 0x7ff8cc0b0870 2024-04-25T08:57:19.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.243+0000 7ff8d2c9e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8cc06be90 con 0x7ff8cc06c510 2024-04-25T08:57:19.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.243+0000 7ff8d123b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8cc15df70 0x7ff8cc0cca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.243+0000 7ff8d123b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8cc15df70 0x7ff8cc0cca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44416/0 (socket says 172.21.15.43:44416) 2024-04-25T08:57:19.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.243+0000 7ff8d123b640 1 -- 172.21.15.43:0/1532204152 learned_addr learned my addr 172.21.15.43:0/1532204152 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:19.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.243+0000 7ff8d0a3a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff8cc06c510 0x7ff8cc153ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.243+0000 7ff8cbfff640 1 --2- 172.21.15.43:0/1532204152 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8cc0b0870 0x7ff8cc15ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.243+0000 7ff8d123b640 1 -- 172.21.15.43:0/1532204152 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff8cc06c510 msgr2=0x7ff8cc153ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:19.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.243+0000 7ff8d123b640 1 --2- 172.21.15.43:0/1532204152 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff8cc06c510 0x7ff8cc153ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.243+0000 7ff8d123b640 1 -- 172.21.15.43:0/1532204152 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8cc0b0870 msgr2=0x7ff8cc15ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:19.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.243+0000 7ff8d123b640 1 --2- 172.21.15.43:0/1532204152 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8cc0b0870 0x7ff8cc15ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.243+0000 7ff8d123b640 1 -- 172.21.15.43:0/1532204152 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8cc154410 con 0x7ff8cc15df70 2024-04-25T08:57:19.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.243+0000 7ff8d123b640 1 --2- 172.21.15.43:0/1532204152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8cc15df70 0x7ff8cc0cca60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff8b400c3c0 tx=0x7ff8b400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:19.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.243+0000 7ff8c97fa640 1 -- 172.21.15.43:0/1532204152 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4017070 con 0x7ff8cc15df70 2024-04-25T08:57:19.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.243+0000 7ff8c97fa640 1 -- 172.21.15.43:0/1532204152 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8b40052d0 con 0x7ff8cc15df70 2024-04-25T08:57:19.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.243+0000 7ff8c97fa640 1 -- 172.21.15.43:0/1532204152 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4003810 con 0x7ff8cc15df70 2024-04-25T08:57:19.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.243+0000 7ff8d2c9e640 1 -- 172.21.15.43:0/1532204152 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8cc0c88c0 con 0x7ff8cc15df70 2024-04-25T08:57:19.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.243+0000 7ff8d2c9e640 1 -- 172.21.15.43:0/1532204152 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8cc0c8e80 con 0x7ff8cc15df70 2024-04-25T08:57:19.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.245+0000 7ff8d2c9e640 1 -- 172.21.15.43:0/1532204152 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8cc074ef0 con 0x7ff8cc15df70 2024-04-25T08:57:19.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.245+0000 7ff8c97fa640 1 -- 172.21.15.43:0/1532204152 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff8b401a070 con 0x7ff8cc15df70 2024-04-25T08:57:19.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.245+0000 7ff8c97fa640 1 --2- 172.21.15.43:0/1532204152 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff8ac06b500 0x7ff8ac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.246+0000 7ff8c97fa640 1 -- 172.21.15.43:0/1532204152 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff8b4089400 con 0x7ff8cc15df70 2024-04-25T08:57:19.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.246+0000 7ff8d0a3a640 1 --2- 172.21.15.43:0/1532204152 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff8ac06b500 0x7ff8ac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.246+0000 7ff8d0a3a640 1 --2- 172.21.15.43:0/1532204152 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff8ac06b500 0x7ff8ac06d9c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4001560 tx=0x7ff8c40019f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:19.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.250+0000 7ff8c97fa640 1 -- 172.21.15.43:0/1532204152 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff8b4057cc0 con 0x7ff8cc15df70 2024-04-25T08:57:19.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.469+0000 7ff8d2c9e640 1 -- 172.21.15.43:0/1532204152 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8cc06cc70 con 0x7ff8cc15df70 2024-04-25T08:57:19.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.469+0000 7ff8c97fa640 1 -- 172.21.15.43:0/1532204152 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4944 (secure 0 0 0) 0x7ff8b405bd60 con 0x7ff8cc15df70 2024-04-25T08:57:19.459 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:57:19.459 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1504450503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1504450503},{"type":"v1","addr":"172.21.15.193:6835","nonce":1504450503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5691,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2657133821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2657133821},{"type":"v1","addr":"172.21.15.43:6834","nonce":2657133821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":5694,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/4271431818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":4271431818},{"type":"v1","addr":"172.21.15.43:6835","nonce":4271431818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:57:01.856132+0000","modified":"2024-04-25T08:57:03.857410+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5709":{"gid":5709,"name":"d","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.193:6837/4164634380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4164634380},{"type":"v1","addr":"172.21.15.193:6837","nonce":4164634380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:57:19.459 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 21 2024-04-25T08:57:19.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.471+0000 7ff8d2c9e640 1 -- 172.21.15.43:0/1532204152 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff8ac06b500 msgr2=0x7ff8ac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:19.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.471+0000 7ff8d2c9e640 1 --2- 172.21.15.43:0/1532204152 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff8ac06b500 0x7ff8ac06d9c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4001560 tx=0x7ff8c40019f0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.471+0000 7ff8d2c9e640 1 -- 172.21.15.43:0/1532204152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8cc15df70 msgr2=0x7ff8cc0cca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:19.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.471+0000 7ff8d2c9e640 1 --2- 172.21.15.43:0/1532204152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8cc15df70 0x7ff8cc0cca60 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff8b400c3c0 tx=0x7ff8b400c880 comp rx=0 tx=0).stop 2024-04-25T08:57:19.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.472+0000 7ff8d2c9e640 1 -- 172.21.15.43:0/1532204152 shutdown_connections 2024-04-25T08:57:19.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.472+0000 7ff8d2c9e640 1 --2- 172.21.15.43:0/1532204152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8cc15df70 0x7ff8cc0cca60 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.472+0000 7ff8d2c9e640 1 --2- 172.21.15.43:0/1532204152 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff8ac06b500 0x7ff8ac06d9c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.472+0000 7ff8d2c9e640 1 --2- 172.21.15.43:0/1532204152 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff8cc0b0870 0x7ff8cc15ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.472+0000 7ff8d2c9e640 1 --2- 172.21.15.43:0/1532204152 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff8cc06c510 0x7ff8cc153ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.472+0000 7ff8d2c9e640 1 -- 172.21.15.43:0/1532204152 >> 172.21.15.43:0/1532204152 conn(0x7ff8cc0d17f0 msgr2=0x7ff8cc06d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:19.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.472+0000 7ff8d2c9e640 1 -- 172.21.15.43:0/1532204152 shutdown_connections 2024-04-25T08:57:19.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.472+0000 7ff8d2c9e640 1 -- 172.21.15.43:0/1532204152 wait complete. 2024-04-25T08:57:19.472 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path / 2024-04-25T08:57:19.577 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-04-25T08:57:19.577 INFO:teuthology.orchestra.run.smithi193.stdout: "return_code": 0 2024-04-25T08:57:19.577 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-04-25T08:57:19.585 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T08:57:19.586 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /' 2024-04-25T08:57:19.586 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:57:19.586 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / 2024-04-25T08:57:19.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.700+0000 7f25b413b640 1 Processor -- start 2024-04-25T08:57:19.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.701+0000 7f25b413b640 1 -- start start 2024-04-25T08:57:19.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.701+0000 7f25b413b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f25ac0d04a0 0x7f25ac15ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.701+0000 7f25b413b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25ac0cf110 con 0x7f25ac15dec0 2024-04-25T08:57:19.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.701+0000 7f25b413b640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25ac0cf3f0 con 0x7f25ac0ce8c0 2024-04-25T08:57:19.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.701+0000 7f25b413b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25ac0cf6d0 con 0x7f25ac0d04a0 2024-04-25T08:57:19.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.702+0000 7f25b1ed7640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f25ac0d04a0 0x7f25ac15ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.702+0000 7f25b16d6640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f25ac15dec0 0x7f25ac0d0880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49612/0 (socket says 172.21.15.43:49612) 2024-04-25T08:57:19.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.702+0000 7f25b16d6640 1 -- 172.21.15.43:0/1508633537 learned_addr learned my addr 172.21.15.43:0/1508633537 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:19.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.702+0000 7f25b1ed7640 1 -- 172.21.15.43:0/1508633537 >> v1:172.21.15.193:6789/0 conn(0x7f25ac0ce8c0 legacy=0x7f25ac0d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:19.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.702+0000 7f25b1ed7640 1 -- 172.21.15.43:0/1508633537 >> v1:172.21.15.43:6789/0 conn(0x7f25ac15dec0 legacy=0x7f25ac0d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:19.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.702+0000 7f25b0ed5640 1 -- 172.21.15.43:0/1508633537 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2587864725 0 0) 0x7f25ac0cf110 con 0x7f25ac15dec0 2024-04-25T08:57:19.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.702+0000 7f25b1ed7640 1 -- 172.21.15.43:0/1508633537 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ac0cf9b0 con 0x7f25ac0d04a0 2024-04-25T08:57:19.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.703+0000 7f25b1ed7640 1 --2- 172.21.15.43:0/1508633537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25ac0d04a0 0x7f25ac15ed90 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f25a80097a0 tx=0x7f25a802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=900dcc40b3846b73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:19.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.703+0000 7f25b0ed5640 1 -- 172.21.15.43:0/1508633537 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a8037070 con 0x7f25ac0d04a0 2024-04-25T08:57:19.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.703+0000 7f25b0ed5640 1 -- 172.21.15.43:0/1508633537 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a8032440 con 0x7f25ac0d04a0 2024-04-25T08:57:19.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.703+0000 7f25b0ed5640 1 -- 172.21.15.43:0/1508633537 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a8032810 con 0x7f25ac0d04a0 2024-04-25T08:57:19.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.703+0000 7f25b413b640 1 -- 172.21.15.43:0/1508633537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25ac0d04a0 msgr2=0x7f25ac15ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:19.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.703+0000 7f25b413b640 1 --2- 172.21.15.43:0/1508633537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25ac0d04a0 0x7f25ac15ed90 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f25a80097a0 tx=0x7f25a802a1e0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.703+0000 7f25b413b640 1 -- 172.21.15.43:0/1508633537 shutdown_connections 2024-04-25T08:57:19.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.703+0000 7f25b413b640 1 --2- 172.21.15.43:0/1508633537 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25ac0d04a0 0x7f25ac15ed90 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.703+0000 7f25b413b640 1 -- 172.21.15.43:0/1508633537 >> 172.21.15.43:0/1508633537 conn(0x7f25ac09cd40 msgr2=0x7f25ac0d3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:19.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.703+0000 7f25b413b640 1 -- 172.21.15.43:0/1508633537 shutdown_connections 2024-04-25T08:57:19.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.703+0000 7f25b413b640 1 -- 172.21.15.43:0/1508633537 wait complete. 2024-04-25T08:57:19.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.704+0000 7f25b413b640 1 Processor -- start 2024-04-25T08:57:19.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.705+0000 7f25b413b640 1 -- start start 2024-04-25T08:57:19.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.705+0000 7f25b413b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25ac0ce8c0 0x7f25ac09e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.705+0000 7f25b413b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f25ac0d04a0 0x7f25ac0a0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.705+0000 7f25b413b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25ac15dec0 0x7f25ac155080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.705+0000 7f25b413b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac0cee50 con 0x7f25ac0ce8c0 2024-04-25T08:57:19.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.705+0000 7f25b413b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac0cecd0 con 0x7f25ac15dec0 2024-04-25T08:57:19.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.705+0000 7f25b413b640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac0d1eb0 con 0x7f25ac0d04a0 2024-04-25T08:57:19.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.705+0000 7f25b1ed7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25ac0ce8c0 0x7f25ac09e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.705+0000 7f25b1ed7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25ac0ce8c0 0x7f25ac09e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41928/0 (socket says 172.21.15.43:41928) 2024-04-25T08:57:19.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.705+0000 7f25b1ed7640 1 -- 172.21.15.43:0/618275453 learned_addr learned my addr 172.21.15.43:0/618275453 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:19.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.706+0000 7f25b26d8640 1 --2- 172.21.15.43:0/618275453 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25ac15dec0 0x7f25ac155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.706+0000 7f25b16d6640 1 --2- 172.21.15.43:0/618275453 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f25ac0d04a0 0x7f25ac0a0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.706+0000 7f25b1ed7640 1 -- 172.21.15.43:0/618275453 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f25ac0d04a0 msgr2=0x7f25ac0a0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:19.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.706+0000 7f25b1ed7640 1 --2- 172.21.15.43:0/618275453 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f25ac0d04a0 0x7f25ac0a0890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.706+0000 7f25b1ed7640 1 -- 172.21.15.43:0/618275453 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25ac15dec0 msgr2=0x7f25ac155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:19.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.706+0000 7f25b1ed7640 1 --2- 172.21.15.43:0/618275453 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25ac15dec0 0x7f25ac155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.706+0000 7f25b1ed7640 1 -- 172.21.15.43:0/618275453 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ac0a0dd0 con 0x7f25ac0ce8c0 2024-04-25T08:57:19.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.706+0000 7f25b26d8640 1 --2- 172.21.15.43:0/618275453 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25ac15dec0 0x7f25ac155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:19.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.706+0000 7f25b1ed7640 1 --2- 172.21.15.43:0/618275453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25ac0ce8c0 0x7f25ac09e4a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f25a80097a0 tx=0x7f25a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:19.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.706+0000 7f25a27fc640 1 -- 172.21.15.43:0/618275453 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a8041070 con 0x7f25ac0ce8c0 2024-04-25T08:57:19.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.706+0000 7f25a27fc640 1 -- 172.21.15.43:0/618275453 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a803c870 con 0x7f25ac0ce8c0 2024-04-25T08:57:19.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.706+0000 7f25b413b640 1 -- 172.21.15.43:0/618275453 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25ac0b3dd0 con 0x7f25ac0ce8c0 2024-04-25T08:57:19.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.706+0000 7f25a27fc640 1 -- 172.21.15.43:0/618275453 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a8037040 con 0x7f25ac0ce8c0 2024-04-25T08:57:19.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.706+0000 7f25b413b640 1 -- 172.21.15.43:0/618275453 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25ac0a1630 con 0x7f25ac0ce8c0 2024-04-25T08:57:19.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.708+0000 7f25a27fc640 1 -- 172.21.15.43:0/618275453 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f25a8044070 con 0x7f25ac0ce8c0 2024-04-25T08:57:19.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.708+0000 7f25a27fc640 1 --2- 172.21.15.43:0/618275453 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f259006b4d0 0x7f259006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.708+0000 7f25a27fc640 1 -- 172.21.15.43:0/618275453 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f25a80aad10 con 0x7f25ac0ce8c0 2024-04-25T08:57:19.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.708+0000 7f25b16d6640 1 --2- 172.21.15.43:0/618275453 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f259006b4d0 0x7f259006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.709+0000 7f25b16d6640 1 --2- 172.21.15.43:0/618275453 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f259006b4d0 0x7f259006d990 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f25ac0b2320 tx=0x7f259c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:19.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.709+0000 7f25b6e40740 1 Processor -- start 2024-04-25T08:57:19.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.709+0000 7f25b6e40740 1 -- start start 2024-04-25T08:57:19.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.710+0000 7f25b6e40740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a6a5742fe0 0x55a6a57457f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.710+0000 7f25b6e40740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6a57433c0 0x55a6a5745d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.710+0000 7f25b6e40740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6a5715c40 0x55a6a5719450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.710+0000 7f25b6e40740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6a567ea50 con 0x55a6a5715c40 2024-04-25T08:57:19.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.710+0000 7f25b6e40740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6a5627d40 con 0x55a6a57433c0 2024-04-25T08:57:19.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.710+0000 7f25b6e40740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a6a567aaf0 con 0x55a6a5742fe0 2024-04-25T08:57:19.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.710+0000 7f25b26d8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6a5715c40 0x55a6a5719450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.710+0000 7f25b26d8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6a5715c40 0x55a6a5719450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41934/0 (socket says 172.21.15.43:41934) 2024-04-25T08:57:19.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.710+0000 7f25b26d8640 1 -- 172.21.15.43:0/3048977684 learned_addr learned my addr 172.21.15.43:0/3048977684 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:19.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.710+0000 7f25b16d6640 1 --2- 172.21.15.43:0/3048977684 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6a57433c0 0x55a6a5745d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.710+0000 7f25b1ed7640 1 --2- 172.21.15.43:0/3048977684 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a6a5742fe0 0x55a6a57457f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.710+0000 7f25b26d8640 1 -- 172.21.15.43:0/3048977684 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a6a5742fe0 msgr2=0x55a6a57457f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:19.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.710+0000 7f25b26d8640 1 --2- 172.21.15.43:0/3048977684 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a6a5742fe0 0x55a6a57457f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.710+0000 7f25b26d8640 1 -- 172.21.15.43:0/3048977684 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6a57433c0 msgr2=0x55a6a5745d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:19.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.710+0000 7f25b26d8640 1 --2- 172.21.15.43:0/3048977684 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6a57433c0 0x55a6a5745d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.710+0000 7f25b26d8640 1 -- 172.21.15.43:0/3048977684 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6a545f690 con 0x55a6a5715c40 2024-04-25T08:57:19.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.710+0000 7f25b26d8640 1 --2- 172.21.15.43:0/3048977684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6a5715c40 0x55a6a5719450 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f25a4004550 tx=0x7f25a400d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b43ab79aa4efe838 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:19.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.710+0000 7f258f7fe640 1 -- 172.21.15.43:0/3048977684 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a4016070 con 0x55a6a5715c40 2024-04-25T08:57:19.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.711+0000 7f258f7fe640 1 -- 172.21.15.43:0/3048977684 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a400ddf0 con 0x55a6a5715c40 2024-04-25T08:57:19.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.711+0000 7f258f7fe640 1 -- 172.21.15.43:0/3048977684 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a4006d90 con 0x55a6a5715c40 2024-04-25T08:57:19.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.711+0000 7f25b6e40740 1 -- 172.21.15.43:0/3048977684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6a5715c40 msgr2=0x55a6a5719450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:19.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.711+0000 7f25b6e40740 1 --2- 172.21.15.43:0/3048977684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6a5715c40 0x55a6a5719450 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f25a4004550 tx=0x7f25a400d520 comp rx=0 tx=0).stop 2024-04-25T08:57:19.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.711+0000 7f25b6e40740 1 -- 172.21.15.43:0/3048977684 shutdown_connections 2024-04-25T08:57:19.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.711+0000 7f25b6e40740 1 --2- 172.21.15.43:0/3048977684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6a5715c40 0x55a6a5719450 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.711+0000 7f25b6e40740 1 --2- 172.21.15.43:0/3048977684 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6a57433c0 0x55a6a5745d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.711+0000 7f25b6e40740 1 --2- 172.21.15.43:0/3048977684 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a6a5742fe0 0x55a6a57457f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.711+0000 7f25b6e40740 1 -- 172.21.15.43:0/3048977684 >> 172.21.15.43:0/3048977684 conn(0x55a6a5717eb0 msgr2=0x55a6a57607f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:19.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.711+0000 7f25b6e40740 1 -- 172.21.15.43:0/3048977684 shutdown_connections 2024-04-25T08:57:19.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.711+0000 7f25b6e40740 1 -- 172.21.15.43:0/3048977684 wait complete. 2024-04-25T08:57:19.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.712+0000 7f25b6e40740 1 -- start start 2024-04-25T08:57:19.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.712+0000 7f25b6e40740 1 Processor -- start 2024-04-25T08:57:19.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.713+0000 7f25b6e40740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6a5742fe0 0x55a6a5762530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.713+0000 7f25b6e40740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a6a57433c0 0x55a6a573fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.713+0000 7f25b6e40740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6a57400d0 0x55a6a5741660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.713+0000 7f25b6e40740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6a571b060 con 0x55a6a5742fe0 2024-04-25T08:57:19.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.713+0000 7f25b6e40740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6a561d340 con 0x55a6a57400d0 2024-04-25T08:57:19.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.713+0000 7f25b6e40740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a6a571b340 con 0x55a6a57433c0 2024-04-25T08:57:19.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.713+0000 7f25b6e40740 10 client.0 _ll_register_callbacks cb 0x55a6a563acb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:19.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.713+0000 7f25b1ed7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6a5742fe0 0x55a6a5762530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.713+0000 7f25b1ed7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6a5742fe0 0x55a6a5762530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41948/0 (socket says 172.21.15.43:41948) 2024-04-25T08:57:19.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.713+0000 7f25b1ed7640 1 -- 172.21.15.43:0/2040552652 learned_addr learned my addr 172.21.15.43:0/2040552652 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:19.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.713+0000 7f25b16d6640 1 --2- 172.21.15.43:0/2040552652 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a6a57433c0 0x55a6a573fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.713+0000 7f25b26d8640 1 --2- 172.21.15.43:0/2040552652 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6a57400d0 0x55a6a5741660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.713+0000 7f25b1ed7640 1 -- 172.21.15.43:0/2040552652 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a6a57433c0 msgr2=0x55a6a573fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:19.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.713+0000 7f25b1ed7640 1 --2- 172.21.15.43:0/2040552652 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a6a57433c0 0x55a6a573fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.713+0000 7f25b1ed7640 1 -- 172.21.15.43:0/2040552652 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6a57400d0 msgr2=0x55a6a5741660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:19.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.713+0000 7f25b1ed7640 1 --2- 172.21.15.43:0/2040552652 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6a57400d0 0x55a6a5741660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.713+0000 7f25b1ed7640 1 -- 172.21.15.43:0/2040552652 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a6a54bfe20 con 0x55a6a5742fe0 2024-04-25T08:57:19.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.713+0000 7f25b26d8640 1 --2- 172.21.15.43:0/2040552652 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6a57400d0 0x55a6a5741660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:19.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.713+0000 7f25b1ed7640 1 --2- 172.21.15.43:0/2040552652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6a5742fe0 0x55a6a5762530 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f25a8079200 tx=0x7f25a8079640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b977180b8dd994f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:19.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.713+0000 7f258cff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:57:19.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.713+0000 7f258cff9640 1 -- 172.21.15.43:0/2040552652 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a804b040 con 0x55a6a5742fe0 2024-04-25T08:57:19.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.713+0000 7f258cff9640 1 -- 172.21.15.43:0/2040552652 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f25a8079760 con 0x55a6a5742fe0 2024-04-25T08:57:19.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.713+0000 7f25b6e40740 1 -- 172.21.15.43:0/2040552652 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f25a400ddf0 con 0x55a6a5742fe0 2024-04-25T08:57:19.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.714+0000 7f258cff9640 1 -- 172.21.15.43:0/2040552652 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25a8079960 con 0x55a6a5742fe0 2024-04-25T08:57:19.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.714+0000 7f25b6e40740 10 client.5955 fetch_fsmap learned FSMap version 21 2024-04-25T08:57:19.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.714+0000 7f25b6e40740 1 -- 172.21.15.43:0/2040552652 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x55a6a5352f80 con 0x55a6a5742fe0 2024-04-25T08:57:19.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.714+0000 7f258cff9640 1 -- 172.21.15.43:0/2040552652 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a400ddf0 con 0x55a6a5742fe0 2024-04-25T08:57:19.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.714+0000 7f258cff9640 1 -- 172.21.15.43:0/2040552652 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f25a8081780 con 0x55a6a5742fe0 2024-04-25T08:57:19.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.714+0000 7f25b6e40740 10 client.5955 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T08:57:19.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.714+0000 7f25b6e40740 10 client.5955 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T08:57:19.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.714+0000 7f25b6e40740 20 client.5955 populate_metadata read hostname 'smithi043' 2024-04-25T08:57:19.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.714+0000 7f25b6e40740 1 --2- 172.21.15.43:0/2040552652 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55a6a577b1b0 0x55a6a577d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.714+0000 7f25b6e40740 4 client.5955 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:57:19.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.714+0000 7f25b6e40740 1 -- 172.21.15.43:0/2040552652 --> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a6a568bf90 con 0x55a6a577b1b0 2024-04-25T08:57:19.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.714+0000 7f25b16d6640 1 --2- 172.21.15.43:0/2040552652 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55a6a577b1b0 0x55a6a577d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.714+0000 7f258cff9640 10 client.5955 ms_handle_connect on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:19.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.714+0000 7f25b16d6640 1 --2- 172.21.15.43:0/2040552652 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55a6a577b1b0 0x55a6a577d5c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e9626652d0d6ade server_cookie=8748cfc95d624169 in_seq=0 out_seq=0 2024-04-25T08:57:19.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.715+0000 7f258cff9640 1 -- 172.21.15.43:0/2040552652 <== mds.0 v2:172.21.15.193:6836/4164634380 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a6a568bf90 con 0x55a6a577b1b0 2024-04-25T08:57:19.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.715+0000 7f258cff9640 10 client.5955 handle_command_reply: tid=0 2024-04-25T08:57:19.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.715+0000 7f25b6e40740 1 client.5955 shutdown 2024-04-25T08:57:19.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.715+0000 7f25b6e40740 1 -- 172.21.15.43:0/2040552652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6a5742fe0 msgr2=0x55a6a5762530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:19.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.715+0000 7f25b6e40740 1 --2- 172.21.15.43:0/2040552652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6a5742fe0 0x55a6a5762530 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f25a8079200 tx=0x7f25a8079640 comp rx=0 tx=0).stop 2024-04-25T08:57:19.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.715+0000 7f25b6e40740 1 -- 172.21.15.43:0/2040552652 shutdown_connections 2024-04-25T08:57:19.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.715+0000 7f25b6e40740 1 --2- 172.21.15.43:0/2040552652 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55a6a577b1b0 0x55a6a577d5c0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.715+0000 7f25b6e40740 1 --2- 172.21.15.43:0/2040552652 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6a57400d0 0x55a6a5741660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.715+0000 7f25b6e40740 1 --2- 172.21.15.43:0/2040552652 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a6a57433c0 0x55a6a573fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.715+0000 7f258cff9640 0 client.5955 ms_handle_reset on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:19.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.715+0000 7f25b6e40740 1 --2- 172.21.15.43:0/2040552652 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6a5742fe0 0x55a6a5762530 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.715+0000 7f25b6e40740 1 -- 172.21.15.43:0/2040552652 >> 172.21.15.43:0/2040552652 conn(0x55a6a5715c40 msgr2=0x55a6a57168f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:19.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.715+0000 7f25b6e40740 1 -- 172.21.15.43:0/2040552652 shutdown_connections 2024-04-25T08:57:19.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.715+0000 7f25b6e40740 1 -- 172.21.15.43:0/2040552652 wait complete. 2024-04-25T08:57:19.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.715+0000 7f25b6e40740 20 client.5955 trim_cache size 0 max 16384 2024-04-25T08:57:19.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.731+0000 7f25b6e40740 1 Processor -- start 2024-04-25T08:57:19.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.731+0000 7f25b6e40740 1 -- start start 2024-04-25T08:57:19.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.731+0000 7f25b6e40740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6a5742fe0 0x55a6a57821a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.731+0000 7f25b6e40740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a6a57433c0 0x55a6a5714a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.731+0000 7f25b6e40740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6a5717960 0x55a6a5714fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.731+0000 7f25b6e40740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac1ffea0 con 0x55a6a5742fe0 2024-04-25T08:57:19.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.731+0000 7f25b6e40740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac1ff040 con 0x55a6a5717960 2024-04-25T08:57:19.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.731+0000 7f25b6e40740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac0b2e40 con 0x55a6a57433c0 2024-04-25T08:57:19.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.731+0000 7f25b1ed7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6a5742fe0 0x55a6a57821a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.731+0000 7f25b1ed7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6a5742fe0 0x55a6a57821a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41962/0 (socket says 172.21.15.43:41962) 2024-04-25T08:57:19.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.731+0000 7f25b1ed7640 1 -- 172.21.15.43:0/1152678131 learned_addr learned my addr 172.21.15.43:0/1152678131 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:19.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.732+0000 7f25b1ed7640 1 -- 172.21.15.43:0/1152678131 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a6a57433c0 msgr2=0x55a6a5714a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:19.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.732+0000 7f25b1ed7640 1 --2- 172.21.15.43:0/1152678131 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a6a57433c0 0x55a6a5714a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.732+0000 7f25b1ed7640 1 -- 172.21.15.43:0/1152678131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6a5717960 msgr2=0x55a6a5714fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:19.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.732+0000 7f25b1ed7640 1 --2- 172.21.15.43:0/1152678131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6a5717960 0x55a6a5714fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.732+0000 7f25b1ed7640 1 -- 172.21.15.43:0/1152678131 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6a5762d90 con 0x55a6a5742fe0 2024-04-25T08:57:19.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.732+0000 7f25b1ed7640 1 --2- 172.21.15.43:0/1152678131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6a5742fe0 0x55a6a57821a0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f25a8008010 tx=0x7f25a803f050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb1f68e209c3baa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:19.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.732+0000 7f258effd640 1 -- 172.21.15.43:0/1152678131 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a804b070 con 0x55a6a5742fe0 2024-04-25T08:57:19.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.732+0000 7f258effd640 1 -- 172.21.15.43:0/1152678131 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a8081780 con 0x55a6a5742fe0 2024-04-25T08:57:19.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.732+0000 7f258effd640 1 -- 172.21.15.43:0/1152678131 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a807d170 con 0x55a6a5742fe0 2024-04-25T08:57:19.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.732+0000 7f25b6e40740 1 -- 172.21.15.43:0/1152678131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6a5742fe0 msgr2=0x55a6a57821a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:19.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.732+0000 7f25b6e40740 1 --2- 172.21.15.43:0/1152678131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6a5742fe0 0x55a6a57821a0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f25a8008010 tx=0x7f25a803f050 comp rx=0 tx=0).stop 2024-04-25T08:57:19.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.732+0000 7f25b6e40740 1 -- 172.21.15.43:0/1152678131 shutdown_connections 2024-04-25T08:57:19.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.732+0000 7f25b6e40740 1 --2- 172.21.15.43:0/1152678131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6a5717960 0x55a6a5714fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.732+0000 7f25b6e40740 1 --2- 172.21.15.43:0/1152678131 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a6a57433c0 0x55a6a5714a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.732+0000 7f25b6e40740 1 --2- 172.21.15.43:0/1152678131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6a5742fe0 0x55a6a57821a0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.732+0000 7f25b6e40740 1 -- 172.21.15.43:0/1152678131 >> 172.21.15.43:0/1152678131 conn(0x55a6a5715c40 msgr2=0x55a6a571a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:19.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.733+0000 7f25b6e40740 1 -- 172.21.15.43:0/1152678131 shutdown_connections 2024-04-25T08:57:19.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.733+0000 7f25b6e40740 1 -- 172.21.15.43:0/1152678131 wait complete. 2024-04-25T08:57:19.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.733+0000 7f25b6e40740 1 -- start start 2024-04-25T08:57:19.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.733+0000 7f25b6e40740 1 Processor -- start 2024-04-25T08:57:19.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.734+0000 7f25b6e40740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a6a5717960 0x55a6a57c2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.734+0000 7f25b6e40740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6a5742fe0 0x55a6a5760090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.734+0000 7f25b6e40740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6a57433c0 0x55a6a5766ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.734+0000 7f25b6e40740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25ac1fea40 con 0x55a6a57433c0 2024-04-25T08:57:19.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.734+0000 7f25b6e40740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6a573f4a0 con 0x55a6a5742fe0 2024-04-25T08:57:19.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.734+0000 7f25b6e40740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a6a57ccae0 con 0x55a6a5717960 2024-04-25T08:57:19.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.734+0000 7f25b6e40740 10 client.0 _ll_register_callbacks cb 0x7f25a8079040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:19.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.734+0000 7f25b26d8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6a57433c0 0x55a6a5766ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.734+0000 7f25b26d8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6a57433c0 0x55a6a5766ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41974/0 (socket says 172.21.15.43:41974) 2024-04-25T08:57:19.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.734+0000 7f25b26d8640 1 -- 172.21.15.43:0/59682416 learned_addr learned my addr 172.21.15.43:0/59682416 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:19.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.734+0000 7f25b16d6640 1 --2- 172.21.15.43:0/59682416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6a5742fe0 0x55a6a5760090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.734+0000 7f25b1ed7640 1 --2- 172.21.15.43:0/59682416 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a6a5717960 0x55a6a57c2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.734+0000 7f25b26d8640 1 -- 172.21.15.43:0/59682416 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a6a5717960 msgr2=0x55a6a57c2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:19.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.734+0000 7f25b26d8640 1 --2- 172.21.15.43:0/59682416 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a6a5717960 0x55a6a57c2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.734+0000 7f25b26d8640 1 -- 172.21.15.43:0/59682416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6a5742fe0 msgr2=0x55a6a5760090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:19.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.734+0000 7f25b26d8640 1 --2- 172.21.15.43:0/59682416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6a5742fe0 0x55a6a5760090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.734+0000 7f25b26d8640 1 -- 172.21.15.43:0/59682416 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a6a5773300 con 0x55a6a57433c0 2024-04-25T08:57:19.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.735+0000 7f25b26d8640 1 --2- 172.21.15.43:0/59682416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6a57433c0 0x55a6a5766ca0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f25a400da20 tx=0x7f25a4003ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb5223a10e307c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:19.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.735+0000 7f258cff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:57:19.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.735+0000 7f258cff9640 1 -- 172.21.15.43:0/59682416 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a4016040 con 0x55a6a57433c0 2024-04-25T08:57:19.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.735+0000 7f258cff9640 1 -- 172.21.15.43:0/59682416 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f25a4004970 con 0x55a6a57433c0 2024-04-25T08:57:19.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.735+0000 7f25b6e40740 1 -- 172.21.15.43:0/59682416 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f25a8081780 con 0x55a6a57433c0 2024-04-25T08:57:19.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.735+0000 7f258cff9640 1 -- 172.21.15.43:0/59682416 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25a402c050 con 0x55a6a57433c0 2024-04-25T08:57:19.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.735+0000 7f25b6e40740 10 client.5961 fetch_fsmap learned FSMap version 21 2024-04-25T08:57:19.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.735+0000 7f25b6e40740 1 -- 172.21.15.43:0/59682416 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x55a6a57495a0 con 0x55a6a57433c0 2024-04-25T08:57:19.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.735+0000 7f258cff9640 1 -- 172.21.15.43:0/59682416 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a8081780 con 0x55a6a57433c0 2024-04-25T08:57:19.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.735+0000 7f258cff9640 1 -- 172.21.15.43:0/59682416 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f25a402f020 con 0x55a6a57433c0 2024-04-25T08:57:19.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.735+0000 7f25b6e40740 10 client.5961 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T08:57:19.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.735+0000 7f25b6e40740 10 client.5961 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T08:57:19.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.735+0000 7f25b6e40740 20 client.5961 populate_metadata read hostname 'smithi043' 2024-04-25T08:57:19.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.735+0000 7f25b6e40740 1 --2- 172.21.15.43:0/59682416 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55a6a57cc6a0 0x55a6a5761bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.735+0000 7f25b6e40740 4 client.5961 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/"}] 2024-04-25T08:57:19.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.735+0000 7f25b6e40740 1 -- 172.21.15.43:0/59682416 --> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] -- command(tid 0: {"prefix": "scrub start", "path": "/"}) v1 -- 0x55a6a56862e0 con 0x55a6a57cc6a0 2024-04-25T08:57:19.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.735+0000 7f25b1ed7640 1 --2- 172.21.15.43:0/59682416 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55a6a57cc6a0 0x55a6a5761bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.736+0000 7f258cff9640 10 client.5961 ms_handle_connect on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:19.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.736+0000 7f25b1ed7640 1 --2- 172.21.15.43:0/59682416 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55a6a57cc6a0 0x55a6a5761bb0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f8d8c370dac5728 server_cookie=f9bdc6f34dadef32 in_seq=0 out_seq=0 2024-04-25T08:57:19.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.736+0000 7f258cff9640 1 -- 172.21.15.43:0/59682416 <== mds.0 v2:172.21.15.193:6836/4164634380 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a6a56862e0 con 0x55a6a57cc6a0 2024-04-25T08:57:19.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.736+0000 7f258cff9640 10 client.5961 handle_command_reply: tid=0 2024-04-25T08:57:19.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.736+0000 7f25b6e40740 1 client.5961 shutdown 2024-04-25T08:57:19.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.736+0000 7f25b6e40740 1 -- 172.21.15.43:0/59682416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6a57433c0 msgr2=0x55a6a5766ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:19.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.736+0000 7f25b6e40740 1 --2- 172.21.15.43:0/59682416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6a57433c0 0x55a6a5766ca0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f25a400da20 tx=0x7f25a4003ae0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.737+0000 7f25b6e40740 1 -- 172.21.15.43:0/59682416 shutdown_connections 2024-04-25T08:57:19.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.737+0000 7f25b6e40740 1 --2- 172.21.15.43:0/59682416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a6a57433c0 0x55a6a5766ca0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.737+0000 7f25b6e40740 1 --2- 172.21.15.43:0/59682416 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55a6a57cc6a0 0x55a6a5761bb0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.737+0000 7f25b6e40740 1 --2- 172.21.15.43:0/59682416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a6a5742fe0 0x55a6a5760090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.737+0000 7f25b6e40740 1 --2- 172.21.15.43:0/59682416 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a6a5717960 0x55a6a57c2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.737+0000 7f25b6e40740 1 -- 172.21.15.43:0/59682416 >> 172.21.15.43:0/59682416 conn(0x55a6a5715c40 msgr2=0x55a6a5714a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:19.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.737+0000 7f258cff9640 0 client.5961 ms_handle_reset on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:19.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.737+0000 7f25b6e40740 1 -- 172.21.15.43:0/59682416 shutdown_connections 2024-04-25T08:57:19.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.737+0000 7f25b6e40740 1 -- 172.21.15.43:0/59682416 wait complete. 2024-04-25T08:57:19.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.737+0000 7f25b6e40740 20 client.5961 trim_cache size 0 max 16384 2024-04-25T08:57:19.732 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T08:57:19.732 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-25T08:57:19.732 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "399e9625-dd76-4969-bb4f-d3a666b763cb", 2024-04-25T08:57:19.733 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-25T08:57:19.733 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T08:57:19.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.738+0000 7f25b413b640 1 -- 172.21.15.43:0/618275453 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f259006b4d0 msgr2=0x7f259006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:19.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.738+0000 7f25b413b640 1 --2- 172.21.15.43:0/618275453 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f259006b4d0 0x7f259006d990 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f25ac0b2320 tx=0x7f259c009040 comp rx=0 tx=0).stop 2024-04-25T08:57:19.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.738+0000 7f25b413b640 1 -- 172.21.15.43:0/618275453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25ac0ce8c0 msgr2=0x7f25ac09e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:19.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.738+0000 7f25b413b640 1 --2- 172.21.15.43:0/618275453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25ac0ce8c0 0x7f25ac09e4a0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f25a80097a0 tx=0x7f25a80046c0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.738+0000 7f25b413b640 1 -- 172.21.15.43:0/618275453 shutdown_connections 2024-04-25T08:57:19.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.738+0000 7f25b413b640 1 --2- 172.21.15.43:0/618275453 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f259006b4d0 0x7f259006d990 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.738+0000 7f25b413b640 1 --2- 172.21.15.43:0/618275453 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25ac15dec0 0x7f25ac155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.738+0000 7f25b413b640 1 --2- 172.21.15.43:0/618275453 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f25ac0d04a0 0x7f25ac0a0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.738+0000 7f25b413b640 1 --2- 172.21.15.43:0/618275453 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25ac0ce8c0 0x7f25ac09e4a0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.738+0000 7f25b413b640 1 -- 172.21.15.43:0/618275453 >> 172.21.15.43:0/618275453 conn(0x7f25ac09cd40 msgr2=0x7f25ac15d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:19.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.738+0000 7f25b413b640 1 -- 172.21.15.43:0/618275453 shutdown_connections 2024-04-25T08:57:19.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.738+0000 7f25b413b640 1 -- 172.21.15.43:0/618275453 wait complete. 2024-04-25T08:57:19.738 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /' returned '{'return_code': 0, 'scrub_tag': '399e9625-dd76-4969-bb4f-d3a666b763cb', 'mode': 'asynchronous'}' 2024-04-25T08:57:19.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:19.739 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_dir_0) 2024-04-25T08:57:19.757 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_0' 2024-04-25T08:57:19.757 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:57:19.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.908+0000 7f83646e1640 1 Processor -- start 2024-04-25T08:57:19.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.908+0000 7f83646e1640 1 -- start start 2024-04-25T08:57:19.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.909+0000 7f83646e1640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f835c0ce2a0 0x7f835c0d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.909+0000 7f83646e1640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f835c0b6c50 con 0x7f835c0d0d90 2024-04-25T08:57:19.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.909+0000 7f83646e1640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f835c15e010 con 0x7f835c0d39d0 2024-04-25T08:57:19.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.909+0000 7f83646e1640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f835c0d2020 con 0x7f835c0ce2a0 2024-04-25T08:57:19.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.909+0000 7f8361c7c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f835c0d0d90 0x7f835c0d2500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49626/0 (socket says 172.21.15.43:49626) 2024-04-25T08:57:19.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.909+0000 7f8361c7c640 1 -- 172.21.15.43:0/602100039 learned_addr learned my addr 172.21.15.43:0/602100039 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:19.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.910+0000 7f8362c7e640 1 --2- 172.21.15.43:0/602100039 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f835c0ce2a0 0x7f835c0d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.910+0000 7f836147b640 1 -- 172.21.15.43:0/602100039 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1769024081 0 0) 0x7f8350001110 con 0x7f835c0d0d90 2024-04-25T08:57:19.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.910+0000 7f836147b640 1 -- 172.21.15.43:0/602100039 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8344003520 con 0x7f835c0d0d90 2024-04-25T08:57:19.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.910+0000 7f836147b640 1 -- 172.21.15.43:0/602100039 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 688692547 0 0) 0x7f835c15e010 con 0x7f835c0d39d0 2024-04-25T08:57:19.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.910+0000 7f836147b640 1 -- 172.21.15.43:0/602100039 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8350001110 con 0x7f835c0d39d0 2024-04-25T08:57:19.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.910+0000 7f836147b640 1 -- 172.21.15.43:0/602100039 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3277162288 0 0) 0x7f8344003520 con 0x7f835c0d0d90 2024-04-25T08:57:19.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.910+0000 7f8362c7e640 1 -- 172.21.15.43:0/602100039 >> v1:172.21.15.193:6789/0 conn(0x7f835c0d39d0 legacy=0x7f835c0d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:19.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.910+0000 7f8362c7e640 1 -- 172.21.15.43:0/602100039 >> v1:172.21.15.43:6789/0 conn(0x7f835c0d0d90 legacy=0x7f835c0d2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:19.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.910+0000 7f8362c7e640 1 -- 172.21.15.43:0/602100039 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f835c0cec40 con 0x7f835c0ce2a0 2024-04-25T08:57:19.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.911+0000 7f8362c7e640 1 --2- 172.21.15.43:0/602100039 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f835c0ce2a0 0x7f835c0d2c10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8354009dc0 tx=0x7f835402a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcea0096d9f43f9e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:19.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.911+0000 7f836147b640 1 -- 172.21.15.43:0/602100039 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8350001840 con 0x7f835c0ce2a0 2024-04-25T08:57:19.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.911+0000 7f836147b640 1 -- 172.21.15.43:0/602100039 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8354007540 con 0x7f835c0ce2a0 2024-04-25T08:57:19.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.911+0000 7f836147b640 1 -- 172.21.15.43:0/602100039 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8354007910 con 0x7f835c0ce2a0 2024-04-25T08:57:19.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.911+0000 7f83646e1640 1 -- 172.21.15.43:0/602100039 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f835c0ce2a0 msgr2=0x7f835c0d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:19.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.911+0000 7f83646e1640 1 --2- 172.21.15.43:0/602100039 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f835c0ce2a0 0x7f835c0d2c10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8354009dc0 tx=0x7f835402a5d0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.912+0000 7f83646e1640 1 -- 172.21.15.43:0/602100039 shutdown_connections 2024-04-25T08:57:19.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.912+0000 7f83646e1640 1 --2- 172.21.15.43:0/602100039 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f835c0ce2a0 0x7f835c0d2c10 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.912+0000 7f83646e1640 1 -- 172.21.15.43:0/602100039 >> 172.21.15.43:0/602100039 conn(0x7f835c06b000 msgr2=0x7f835c06b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:19.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.912+0000 7f83646e1640 1 -- 172.21.15.43:0/602100039 shutdown_connections 2024-04-25T08:57:19.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.912+0000 7f83646e1640 1 -- 172.21.15.43:0/602100039 wait complete. 2024-04-25T08:57:19.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.913+0000 7f83646e1640 1 Processor -- start 2024-04-25T08:57:19.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.913+0000 7f83646e1640 1 -- start start 2024-04-25T08:57:19.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.913+0000 7f83646e1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f835c0ce2a0 0x7f835c0c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.913+0000 7f83646e1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f835c0d0d90 0x7f835c1434f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.913+0000 7f83646e1640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f835c0d39d0 0x7f835c143a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.913+0000 7f83646e1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835c0cf200 con 0x7f835c0ce2a0 2024-04-25T08:57:19.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.913+0000 7f83646e1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f835c0b2f00 con 0x7f835c0d0d90 2024-04-25T08:57:19.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.913+0000 7f83646e1640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f835c0d3150 con 0x7f835c0d39d0 2024-04-25T08:57:19.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.914+0000 7f836247d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f835c0ce2a0 0x7f835c0c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.914+0000 7f836247d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f835c0ce2a0 0x7f835c0c9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41982/0 (socket says 172.21.15.43:41982) 2024-04-25T08:57:19.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.914+0000 7f836247d640 1 -- 172.21.15.43:0/451911126 learned_addr learned my addr 172.21.15.43:0/451911126 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:19.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.914+0000 7f8362c7e640 1 --2- 172.21.15.43:0/451911126 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f835c0d39d0 0x7f835c143a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.914+0000 7f8361c7c640 1 --2- 172.21.15.43:0/451911126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f835c0d0d90 0x7f835c1434f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.914+0000 7f836247d640 1 -- 172.21.15.43:0/451911126 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f835c0d39d0 msgr2=0x7f835c143a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:19.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.914+0000 7f836247d640 1 --2- 172.21.15.43:0/451911126 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f835c0d39d0 0x7f835c143a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.914+0000 7f836247d640 1 -- 172.21.15.43:0/451911126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f835c0d0d90 msgr2=0x7f835c1434f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:19.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.914+0000 7f836247d640 1 --2- 172.21.15.43:0/451911126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f835c0d0d90 0x7f835c1434f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:19.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.914+0000 7f836247d640 1 -- 172.21.15.43:0/451911126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f835c0b8d00 con 0x7f835c0ce2a0 2024-04-25T08:57:19.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.914+0000 7f836247d640 1 --2- 172.21.15.43:0/451911126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f835c0ce2a0 0x7f835c0c9e60 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f834c00c460 tx=0x7f834c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:19.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.914+0000 7f835affd640 1 -- 172.21.15.43:0/451911126 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834c0052d0 con 0x7f835c0ce2a0 2024-04-25T08:57:19.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.914+0000 7f835affd640 1 -- 172.21.15.43:0/451911126 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f834c005490 con 0x7f835c0ce2a0 2024-04-25T08:57:19.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.914+0000 7f835affd640 1 -- 172.21.15.43:0/451911126 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834c012870 con 0x7f835c0ce2a0 2024-04-25T08:57:19.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.914+0000 7f83646e1640 1 -- 172.21.15.43:0/451911126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f835c0ca3a0 con 0x7f835c0ce2a0 2024-04-25T08:57:19.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.914+0000 7f83646e1640 1 -- 172.21.15.43:0/451911126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f835c1468e0 con 0x7f835c0ce2a0 2024-04-25T08:57:19.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.915+0000 7f83646e1640 1 -- 172.21.15.43:0/451911126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f835c074ef0 con 0x7f835c0ce2a0 2024-04-25T08:57:19.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.916+0000 7f835affd640 1 -- 172.21.15.43:0/451911126 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f834c012a10 con 0x7f835c0ce2a0 2024-04-25T08:57:19.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.916+0000 7f835affd640 1 --2- 172.21.15.43:0/451911126 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f834806b4d0 0x7f834806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:19.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.916+0000 7f835affd640 1 -- 172.21.15.43:0/451911126 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f834c089d20 con 0x7f835c0ce2a0 2024-04-25T08:57:19.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.916+0000 7f8361c7c640 1 --2- 172.21.15.43:0/451911126 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f834806b4d0 0x7f834806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:19.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.917+0000 7f8361c7c640 1 --2- 172.21.15.43:0/451911126 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f834806b4d0 0x7f834806d990 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f835c154970 tx=0x7f8350001df0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:19.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:19.920+0000 7f835affd640 1 -- 172.21.15.43:0/451911126 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f835c0ca3a0 con 0x7f835c0ce2a0 2024-04-25T08:57:20.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.136+0000 7f83646e1640 1 -- 172.21.15.43:0/451911126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f835c0ce880 con 0x7f835c0ce2a0 2024-04-25T08:57:20.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.136+0000 7f835affd640 1 -- 172.21.15.43:0/451911126 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f835c0b8d00 con 0x7f835c0ce2a0 2024-04-25T08:57:20.126 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:57:20.126 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1504450503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1504450503},{"type":"v1","addr":"172.21.15.193:6835","nonce":1504450503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5691,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2657133821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2657133821},{"type":"v1","addr":"172.21.15.43:6834","nonce":2657133821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":5694,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/4271431818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":4271431818},{"type":"v1","addr":"172.21.15.43:6835","nonce":4271431818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:57:01.856132+0000","modified":"2024-04-25T08:57:03.857410+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5709":{"gid":5709,"name":"d","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.193:6837/4164634380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4164634380},{"type":"v1","addr":"172.21.15.193:6837","nonce":4164634380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:57:20.127 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 21 2024-04-25T08:57:20.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.139+0000 7f83646e1640 1 -- 172.21.15.43:0/451911126 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f834806b4d0 msgr2=0x7f834806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:20.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.139+0000 7f83646e1640 1 --2- 172.21.15.43:0/451911126 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f834806b4d0 0x7f834806d990 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f835c154970 tx=0x7f8350001df0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.139+0000 7f83646e1640 1 -- 172.21.15.43:0/451911126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f835c0ce2a0 msgr2=0x7f835c0c9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:20.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.139+0000 7f83646e1640 1 --2- 172.21.15.43:0/451911126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f835c0ce2a0 0x7f835c0c9e60 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f834c00c460 tx=0x7f834c00c920 comp rx=0 tx=0).stop 2024-04-25T08:57:20.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.139+0000 7f83646e1640 1 -- 172.21.15.43:0/451911126 shutdown_connections 2024-04-25T08:57:20.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.139+0000 7f83646e1640 1 --2- 172.21.15.43:0/451911126 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f835c0d39d0 0x7f835c143a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.139+0000 7f83646e1640 1 --2- 172.21.15.43:0/451911126 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f834806b4d0 0x7f834806d990 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.139+0000 7f83646e1640 1 --2- 172.21.15.43:0/451911126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f835c0d0d90 0x7f835c1434f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.139+0000 7f83646e1640 1 --2- 172.21.15.43:0/451911126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f835c0ce2a0 0x7f835c0c9e60 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.139+0000 7f83646e1640 1 -- 172.21.15.43:0/451911126 >> 172.21.15.43:0/451911126 conn(0x7f835c06b000 msgr2=0x7f835c15cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:20.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.139+0000 7f83646e1640 1 -- 172.21.15.43:0/451911126 shutdown_connections 2024-04-25T08:57:20.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.139+0000 7f83646e1640 1 -- 172.21.15.43:0/451911126 wait complete. 2024-04-25T08:57:20.139 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path /scrub_test_path/ceph-qa-suite/new_dir_0 2024-04-25T08:57:20.247 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-04-25T08:57:20.248 INFO:teuthology.orchestra.run.smithi193.stdout: "return_code": 0 2024-04-25T08:57:20.248 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-04-25T08:57:20.256 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_0' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T08:57:20.257 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:57:20.257 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:57:20.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:20.273 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_0 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T08:57:20.381 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T08:57:20.381 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T08:57:20.381 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00878769 s, 119 MB/s 2024-04-25T08:57:20.382 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_0' 2024-04-25T08:57:20.382 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:57:20.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.494+0000 7f08e0590640 1 Processor -- start 2024-04-25T08:57:20.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.494+0000 7f08e0590640 1 -- start start 2024-04-25T08:57:20.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.495+0000 7f08e0590640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f08d80cbfc0 0x7f08d806dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:20.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.495+0000 7f08e0590640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08d80cd010 0x7f08d815fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:20.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.495+0000 7f08e0590640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08d80b2780 con 0x7f08d80ccbb0 2024-04-25T08:57:20.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.495+0000 7f08e0590640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08d80b2a60 con 0x7f08d80cbfc0 2024-04-25T08:57:20.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.495+0000 7f08e0590640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08d80b2d40 con 0x7f08d80cd010 2024-04-25T08:57:20.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.495+0000 7f08de32c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f08d80cbfc0 0x7f08d806dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:20.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.495+0000 7f08de32c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f08d80cbfc0 0x7f08d806dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41988/0 (socket says 172.21.15.43:41988) 2024-04-25T08:57:20.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.495+0000 7f08de32c640 1 -- 172.21.15.43:0/3557884567 learned_addr learned my addr 172.21.15.43:0/3557884567 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:20.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.495+0000 7f08deb2d640 1 --2- 172.21.15.43:0/3557884567 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08d80cd010 0x7f08d815fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:20.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.495+0000 7f08de32c640 1 -- 172.21.15.43:0/3557884567 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08d80cd010 msgr2=0x7f08d815fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:20.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.495+0000 7f08de32c640 1 --2- 172.21.15.43:0/3557884567 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08d80cd010 0x7f08d815fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.495+0000 7f08de32c640 1 -- 172.21.15.43:0/3557884567 >> v1:172.21.15.193:6789/0 conn(0x7f08d80ccbb0 legacy=0x7f08d815f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:20.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.495+0000 7f08de32c640 1 -- 172.21.15.43:0/3557884567 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d80b3020 con 0x7f08d80cbfc0 2024-04-25T08:57:20.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.496+0000 7f08de32c640 1 --2- 172.21.15.43:0/3557884567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08d80cbfc0 0x7f08d806dc60 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f08d4009870 tx=0x7f08d402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7da67b36836a4c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:20.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.496+0000 7f08dd32a640 1 -- 172.21.15.43:0/3557884567 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d4037070 con 0x7f08d80cbfc0 2024-04-25T08:57:20.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.496+0000 7f08dd32a640 1 -- 172.21.15.43:0/3557884567 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08d4032440 con 0x7f08d80cbfc0 2024-04-25T08:57:20.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.496+0000 7f08dd32a640 1 -- 172.21.15.43:0/3557884567 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d4032810 con 0x7f08d80cbfc0 2024-04-25T08:57:20.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.496+0000 7f08e0590640 1 -- 172.21.15.43:0/3557884567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08d80cbfc0 msgr2=0x7f08d806dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:20.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.496+0000 7f08e0590640 1 --2- 172.21.15.43:0/3557884567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08d80cbfc0 0x7f08d806dc60 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f08d4009870 tx=0x7f08d402a210 comp rx=0 tx=0).stop 2024-04-25T08:57:20.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.497+0000 7f08e0590640 1 -- 172.21.15.43:0/3557884567 shutdown_connections 2024-04-25T08:57:20.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.497+0000 7f08e0590640 1 --2- 172.21.15.43:0/3557884567 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08d80cd010 0x7f08d815fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.497+0000 7f08e0590640 1 --2- 172.21.15.43:0/3557884567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08d80cbfc0 0x7f08d806dc60 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.497+0000 7f08e0590640 1 -- 172.21.15.43:0/3557884567 >> 172.21.15.43:0/3557884567 conn(0x7f08d80b1480 msgr2=0x7f08d80b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:20.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.497+0000 7f08e0590640 1 -- 172.21.15.43:0/3557884567 shutdown_connections 2024-04-25T08:57:20.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.497+0000 7f08e0590640 1 -- 172.21.15.43:0/3557884567 wait complete. 2024-04-25T08:57:20.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.498+0000 7f08e0590640 1 Processor -- start 2024-04-25T08:57:20.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.498+0000 7f08e0590640 1 -- start start 2024-04-25T08:57:20.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.499+0000 7f08e0590640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08d80cbfc0 0x7f08d8071dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:20.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.499+0000 7f08e0590640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08d80ccbb0 0x7f08d8141140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:20.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.499+0000 7f08e0590640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08d80cd010 0x7f08d80ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:20.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.499+0000 7f08e0590640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d80cd4a0 con 0x7f08d80ccbb0 2024-04-25T08:57:20.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.499+0000 7f08e0590640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d80b35e0 con 0x7f08d80cbfc0 2024-04-25T08:57:20.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.499+0000 7f08e0590640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08d806e1a0 con 0x7f08d80cd010 2024-04-25T08:57:20.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.499+0000 7f08ddb2b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08d80ccbb0 0x7f08d8141140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:20.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.499+0000 7f08ddb2b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08d80ccbb0 0x7f08d8141140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41998/0 (socket says 172.21.15.43:41998) 2024-04-25T08:57:20.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.499+0000 7f08ddb2b640 1 -- 172.21.15.43:0/2152415934 learned_addr learned my addr 172.21.15.43:0/2152415934 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:20.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.499+0000 7f08de32c640 1 --2- 172.21.15.43:0/2152415934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08d80cbfc0 0x7f08d8071dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:20.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.499+0000 7f08deb2d640 1 --2- 172.21.15.43:0/2152415934 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08d80cd010 0x7f08d80ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:20.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.499+0000 7f08ddb2b640 1 -- 172.21.15.43:0/2152415934 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08d80cd010 msgr2=0x7f08d80ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:20.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.499+0000 7f08ddb2b640 1 --2- 172.21.15.43:0/2152415934 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08d80cd010 0x7f08d80ad8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.499+0000 7f08ddb2b640 1 -- 172.21.15.43:0/2152415934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08d80cbfc0 msgr2=0x7f08d8071dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:20.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.499+0000 7f08ddb2b640 1 --2- 172.21.15.43:0/2152415934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08d80cbfc0 0x7f08d8071dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.499+0000 7f08ddb2b640 1 -- 172.21.15.43:0/2152415934 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d81e4710 con 0x7f08d80ccbb0 2024-04-25T08:57:20.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.499+0000 7f08ddb2b640 1 --2- 172.21.15.43:0/2152415934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08d80ccbb0 0x7f08d8141140 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f08c800b3f0 tx=0x7f08c800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:20.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.499+0000 7f08ceffd640 1 -- 172.21.15.43:0/2152415934 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c8015020 con 0x7f08d80ccbb0 2024-04-25T08:57:20.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.499+0000 7f08ceffd640 1 -- 172.21.15.43:0/2152415934 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08c8011830 con 0x7f08d80ccbb0 2024-04-25T08:57:20.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.500+0000 7f08ceffd640 1 -- 172.21.15.43:0/2152415934 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c8011c20 con 0x7f08d80ccbb0 2024-04-25T08:57:20.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.500+0000 7f08e0590640 1 -- 172.21.15.43:0/2152415934 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08d8140660 con 0x7f08d80ccbb0 2024-04-25T08:57:20.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.500+0000 7f08e0590640 1 -- 172.21.15.43:0/2152415934 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d8140c20 con 0x7f08d80ccbb0 2024-04-25T08:57:20.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.500+0000 7f08e0590640 1 -- 172.21.15.43:0/2152415934 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08d8140e20 con 0x7f08d80ccbb0 2024-04-25T08:57:20.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.501+0000 7f08ceffd640 1 -- 172.21.15.43:0/2152415934 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f08c8018020 con 0x7f08d80ccbb0 2024-04-25T08:57:20.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.501+0000 7f08ceffd640 1 --2- 172.21.15.43:0/2152415934 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f08bc06b4d0 0x7f08bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:20.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.502+0000 7f08ceffd640 1 -- 172.21.15.43:0/2152415934 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f08c8089dc0 con 0x7f08d80ccbb0 2024-04-25T08:57:20.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.502+0000 7f08de32c640 1 --2- 172.21.15.43:0/2152415934 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f08bc06b4d0 0x7f08bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:20.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.502+0000 7f08de32c640 1 --2- 172.21.15.43:0/2152415934 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f08bc06b4d0 0x7f08bc06d990 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f08d4034000 tx=0x7f08d402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:20.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.506+0000 7f08ceffd640 1 -- 172.21.15.43:0/2152415934 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f08d8140e20 con 0x7f08d80ccbb0 2024-04-25T08:57:20.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.722+0000 7f08e0590640 1 -- 172.21.15.43:0/2152415934 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08d80af0d0 con 0x7f08d80ccbb0 2024-04-25T08:57:20.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.723+0000 7f08ceffd640 1 -- 172.21.15.43:0/2152415934 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f08c801f030 con 0x7f08d80ccbb0 2024-04-25T08:57:20.713 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:57:20.713 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":21,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1504450503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1504450503},{"type":"v1","addr":"172.21.15.193:6835","nonce":1504450503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5691,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2657133821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2657133821},{"type":"v1","addr":"172.21.15.43:6834","nonce":2657133821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":5694,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/4271431818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":4271431818},{"type":"v1","addr":"172.21.15.43:6835","nonce":4271431818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:57:01.856132+0000","modified":"2024-04-25T08:57:03.857410+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5709":{"gid":5709,"name":"d","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.193:6837/4164634380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4164634380},{"type":"v1","addr":"172.21.15.193:6837","nonce":4164634380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:57:20.713 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 21 2024-04-25T08:57:20.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.725+0000 7f08e0590640 1 -- 172.21.15.43:0/2152415934 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f08bc06b4d0 msgr2=0x7f08bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:20.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.725+0000 7f08e0590640 1 --2- 172.21.15.43:0/2152415934 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f08bc06b4d0 0x7f08bc06d990 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f08d4034000 tx=0x7f08d402e000 comp rx=0 tx=0).stop 2024-04-25T08:57:20.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.725+0000 7f08e0590640 1 -- 172.21.15.43:0/2152415934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08d80ccbb0 msgr2=0x7f08d8141140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:20.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.725+0000 7f08e0590640 1 --2- 172.21.15.43:0/2152415934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08d80ccbb0 0x7f08d8141140 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f08c800b3f0 tx=0x7f08c800b8b0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.725+0000 7f08e0590640 1 -- 172.21.15.43:0/2152415934 shutdown_connections 2024-04-25T08:57:20.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.725+0000 7f08e0590640 1 --2- 172.21.15.43:0/2152415934 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08d80cd010 0x7f08d80ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.725+0000 7f08e0590640 1 --2- 172.21.15.43:0/2152415934 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08d80ccbb0 0x7f08d8141140 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.725+0000 7f08e0590640 1 --2- 172.21.15.43:0/2152415934 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f08bc06b4d0 0x7f08bc06d990 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.725+0000 7f08e0590640 1 --2- 172.21.15.43:0/2152415934 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08d80cbfc0 0x7f08d8071dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:20.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.725+0000 7f08e0590640 1 -- 172.21.15.43:0/2152415934 >> 172.21.15.43:0/2152415934 conn(0x7f08d80b1480 msgr2=0x7f08d815c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:20.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.725+0000 7f08e0590640 1 -- 172.21.15.43:0/2152415934 shutdown_connections 2024-04-25T08:57:20.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20.725+0000 7f08e0590640 1 -- 172.21.15.43:0/2152415934 wait complete. 2024-04-25T08:57:20.726 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path /scrub_test_path/ceph-qa-suite/new_file_0 2024-04-25T08:57:20.869 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-04-25T08:57:20.869 INFO:teuthology.orchestra.run.smithi193.stdout: "return_code": 0 2024-04-25T08:57:20.869 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-04-25T08:57:20.878 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_0' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T08:57:20.879 DEBUG:teuthology.orchestra.run.smithi043:> stdin-killer -- python3 -c ' 2024-04-25T08:57:20.879 DEBUG:teuthology.orchestra.run.smithi043:> import os 2024-04-25T08:57:20.879 DEBUG:teuthology.orchestra.run.smithi043:> import stat 2024-04-25T08:57:20.879 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T08:57:20.879 DEBUG:teuthology.orchestra.run.smithi043:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_0").st_ino) 2024-04-25T08:57:20.879 DEBUG:teuthology.orchestra.run.smithi043:> ' 2024-04-25T08:57:20.947 INFO:teuthology.orchestra.run.smithi043.stdout:1099511630412 2024-04-25T08:57:20.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:57:20.956 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:57:20.957 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T08:57:21.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.109+0000 7fe7278cb640 1 Processor -- start 2024-04-25T08:57:21.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.109+0000 7fe7278cb640 1 -- start start 2024-04-25T08:57:21.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.110+0000 7fe7278cb640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe72015b9f0 0x7fe72015e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.110+0000 7fe7278cb640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fe7200cda20 0x7fe7201621d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.110+0000 7fe7278cb640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe72015bdd0 con 0x7fe720162710 2024-04-25T08:57:21.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.110+0000 7fe7278cb640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe72015ec80 con 0x7fe7200cda20 2024-04-25T08:57:21.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.110+0000 7fe7278cb640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe72015ef60 con 0x7fe72015b9f0 2024-04-25T08:57:21.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.110+0000 7fe725e68640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe720162710 0x7fe7200cfa80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49636/0 (socket says 172.21.15.43:49636) 2024-04-25T08:57:21.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.110+0000 7fe725e68640 1 -- 172.21.15.43:0/3162867416 learned_addr learned my addr 172.21.15.43:0/3162867416 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:21.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.110+0000 7fe71ffff640 1 -- 172.21.15.43:0/3162867416 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4260397604 0 0) 0x7fe72015bdd0 con 0x7fe720162710 2024-04-25T08:57:21.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.110+0000 7fe725667640 1 --2- 172.21.15.43:0/3162867416 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe72015b9f0 0x7fe72015e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.110+0000 7fe724e66640 1 --2- 172.21.15.43:0/3162867416 >> v2:172.21.15.193:3300/0 conn(0x7fe7200cda20 0x7fe7201621d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.110+0000 7fe71ffff640 1 -- 172.21.15.43:0/3162867416 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe708003520 con 0x7fe720162710 2024-04-25T08:57:21.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.111+0000 7fe71ffff640 1 -- 172.21.15.43:0/3162867416 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1658574089 0 0) 0x7fe708003520 con 0x7fe720162710 2024-04-25T08:57:21.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.111+0000 7fe71ffff640 1 -- 172.21.15.43:0/3162867416 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe72015b9f0 msgr2=0x7fe72015e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:21.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.111+0000 7fe71ffff640 1 --2- 172.21.15.43:0/3162867416 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe72015b9f0 0x7fe72015e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.111+0000 7fe71ffff640 1 -- 172.21.15.43:0/3162867416 >> v2:172.21.15.193:3300/0 conn(0x7fe7200cda20 msgr2=0x7fe7201621d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:21.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.111+0000 7fe71ffff640 1 --2- 172.21.15.43:0/3162867416 >> v2:172.21.15.193:3300/0 conn(0x7fe7200cda20 0x7fe7201621d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.111+0000 7fe71ffff640 1 -- 172.21.15.43:0/3162867416 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe72015f240 con 0x7fe720162710 2024-04-25T08:57:21.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.111+0000 7fe71ffff640 1 -- 172.21.15.43:0/3162867416 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7fe7100019f0 con 0x7fe720162710 2024-04-25T08:57:21.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.111+0000 7fe71ffff640 1 -- 172.21.15.43:0/3162867416 >> v1:172.21.15.43:6789/0 conn(0x7fe720162710 legacy=0x7fe7200cfa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.111+0000 7fe71ffff640 1 --2- 172.21.15.43:0/3162867416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7080039b0 0x7fe708003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.111+0000 7fe71ffff640 1 --2- 172.21.15.43:0/3162867416 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe72015b9f0 0x7fe708004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.111+0000 7fe71ffff640 1 --2- 172.21.15.43:0/3162867416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe708004d40 0x7fe7080051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.111+0000 7fe71ffff640 1 -- 172.21.15.43:0/3162867416 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe72015ec80 con 0x7fe708004d40 2024-04-25T08:57:21.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.111+0000 7fe71ffff640 1 -- 172.21.15.43:0/3162867416 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe72015ef60 con 0x7fe7080039b0 2024-04-25T08:57:21.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.111+0000 7fe71ffff640 1 -- 172.21.15.43:0/3162867416 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe70800b9c0 con 0x7fe72015b9f0 2024-04-25T08:57:21.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.111+0000 7fe725e68640 1 --2- 172.21.15.43:0/3162867416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe708004d40 0x7fe7080051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.111+0000 7fe724e66640 1 --2- 172.21.15.43:0/3162867416 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe72015b9f0 0x7fe708004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.111+0000 7fe725667640 1 --2- 172.21.15.43:0/3162867416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7080039b0 0x7fe708003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.111+0000 7fe725e68640 1 -- 172.21.15.43:0/3162867416 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe72015b9f0 msgr2=0x7fe708004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:21.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.111+0000 7fe725e68640 1 --2- 172.21.15.43:0/3162867416 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe72015b9f0 0x7fe708004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.111+0000 7fe725e68640 1 -- 172.21.15.43:0/3162867416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7080039b0 msgr2=0x7fe708003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:21.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.111+0000 7fe725e68640 1 --2- 172.21.15.43:0/3162867416 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe7080039b0 0x7fe708003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.111+0000 7fe725e68640 1 -- 172.21.15.43:0/3162867416 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe708003520 con 0x7fe708004d40 2024-04-25T08:57:21.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.112+0000 7fe725e68640 1 --2- 172.21.15.43:0/3162867416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe708004d40 0x7fe7080051c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe7100013d0 tx=0x7fe708005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc747d7c55e66e1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:21.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.112+0000 7fe71ffff640 1 -- 172.21.15.43:0/3162867416 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71000a4e0 con 0x7fe708004d40 2024-04-25T08:57:21.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.112+0000 7fe71ffff640 1 -- 172.21.15.43:0/3162867416 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe71002f070 con 0x7fe708004d40 2024-04-25T08:57:21.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.112+0000 7fe71ffff640 1 -- 172.21.15.43:0/3162867416 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71002bcb0 con 0x7fe708004d40 2024-04-25T08:57:21.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.113+0000 7fe7278cb640 1 -- 172.21.15.43:0/3162867416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe708004d40 msgr2=0x7fe7080051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.113+0000 7fe7278cb640 1 --2- 172.21.15.43:0/3162867416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe708004d40 0x7fe7080051c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe7100013d0 tx=0x7fe708005830 comp rx=0 tx=0).stop 2024-04-25T08:57:21.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.113+0000 7fe725e68640 1 -- 172.21.15.43:0/3162867416 reap_dead start 2024-04-25T08:57:21.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.113+0000 7fe7278cb640 1 -- 172.21.15.43:0/3162867416 shutdown_connections 2024-04-25T08:57:21.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.113+0000 7fe7278cb640 1 -- 172.21.15.43:0/3162867416 >> 172.21.15.43:0/3162867416 conn(0x7fe72006ab30 msgr2=0x7fe72006af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:21.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.113+0000 7fe7278cb640 1 -- 172.21.15.43:0/3162867416 shutdown_connections 2024-04-25T08:57:21.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.113+0000 7fe7278cb640 1 -- 172.21.15.43:0/3162867416 wait complete. 2024-04-25T08:57:21.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.114+0000 7fe7278cb640 1 Processor -- start 2024-04-25T08:57:21.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.114+0000 7fe7278cb640 1 -- start start 2024-04-25T08:57:21.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.115+0000 7fe7278cb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe708004d40 0x7fe720145610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.115+0000 7fe7278cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe720155f90 0x7fe7200ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.115+0000 7fe7278cb640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe720156470 0x7fe7200ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.115+0000 7fe7278cb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7200d0310 con 0x7fe720155f90 2024-04-25T08:57:21.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.115+0000 7fe7278cb640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7200d0190 con 0x7fe708004d40 2024-04-25T08:57:21.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.115+0000 7fe7278cb640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe72015e360 con 0x7fe720156470 2024-04-25T08:57:21.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.115+0000 7fe724e66640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe720155f90 0x7fe7200ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.115+0000 7fe724e66640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe720155f90 0x7fe7200ae3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42014/0 (socket says 172.21.15.43:42014) 2024-04-25T08:57:21.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.115+0000 7fe724e66640 1 -- 172.21.15.43:0/2991917737 learned_addr learned my addr 172.21.15.43:0/2991917737 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:21.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.115+0000 7fe725e68640 1 --2- 172.21.15.43:0/2991917737 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe720156470 0x7fe7200ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.115+0000 7fe725667640 1 --2- 172.21.15.43:0/2991917737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe708004d40 0x7fe720145610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.115+0000 7fe724e66640 1 -- 172.21.15.43:0/2991917737 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe720156470 msgr2=0x7fe7200ae8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.115+0000 7fe724e66640 1 --2- 172.21.15.43:0/2991917737 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe720156470 0x7fe7200ae8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.115+0000 7fe724e66640 1 -- 172.21.15.43:0/2991917737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe708004d40 msgr2=0x7fe720145610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.115+0000 7fe724e66640 1 --2- 172.21.15.43:0/2991917737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe708004d40 0x7fe720145610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.115+0000 7fe724e66640 1 -- 172.21.15.43:0/2991917737 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe720145b50 con 0x7fe720155f90 2024-04-25T08:57:21.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.115+0000 7fe724e66640 1 --2- 172.21.15.43:0/2991917737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe720155f90 0x7fe7200ae3b0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fe714011a70 tx=0x7fe714011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:21.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.115+0000 7fe71dffb640 1 -- 172.21.15.43:0/2991917737 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71400fa90 con 0x7fe720155f90 2024-04-25T08:57:21.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.116+0000 7fe71dffb640 1 -- 172.21.15.43:0/2991917737 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe71400fc30 con 0x7fe720155f90 2024-04-25T08:57:21.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.116+0000 7fe7278cb640 1 -- 172.21.15.43:0/2991917737 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7201482a0 con 0x7fe720155f90 2024-04-25T08:57:21.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.116+0000 7fe71dffb640 1 -- 172.21.15.43:0/2991917737 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe714014440 con 0x7fe720155f90 2024-04-25T08:57:21.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.116+0000 7fe7278cb640 1 -- 172.21.15.43:0/2991917737 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe720148860 con 0x7fe720155f90 2024-04-25T08:57:21.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.117+0000 7fe71dffb640 1 -- 172.21.15.43:0/2991917737 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe714012040 con 0x7fe720155f90 2024-04-25T08:57:21.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.117+0000 7fe71dffb640 1 --2- 172.21.15.43:0/2991917737 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe70c06b500 0x7fe70c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.118+0000 7fe71dffb640 1 -- 172.21.15.43:0/2991917737 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe71408a730 con 0x7fe720155f90 2024-04-25T08:57:21.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.118+0000 7fe725667640 1 --2- 172.21.15.43:0/2991917737 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe70c06b500 0x7fe70c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.118+0000 7fe725667640 1 --2- 172.21.15.43:0/2991917737 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe70c06b500 0x7fe70c06d9c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe718009b40 tx=0x7fe718009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:21.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.119+0000 7fe72a5d1740 1 Processor -- start 2024-04-25T08:57:21.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.119+0000 7fe72a5d1740 1 -- start start 2024-04-25T08:57:21.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.119+0000 7fe72a5d1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd82a0aaa0 0x55dd82a0d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.119+0000 7fe72a5d1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd82a0ae80 0x55dd82a0d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.119+0000 7fe72a5d1740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55dd829dd700 0x55dd829e0f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.119+0000 7fe72a5d1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd82946e00 con 0x55dd82a0ae80 2024-04-25T08:57:21.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.119+0000 7fe72a5d1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd828efd20 con 0x55dd82a0aaa0 2024-04-25T08:57:21.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.119+0000 7fe72a5d1740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dd82942ea0 con 0x55dd829dd700 2024-04-25T08:57:21.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.119+0000 7fe724e66640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd82a0ae80 0x55dd82a0d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.119+0000 7fe724e66640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd82a0ae80 0x55dd82a0d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42030/0 (socket says 172.21.15.43:42030) 2024-04-25T08:57:21.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.119+0000 7fe724e66640 1 -- 172.21.15.43:0/3178187819 learned_addr learned my addr 172.21.15.43:0/3178187819 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:21.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.119+0000 7fe725667640 1 --2- 172.21.15.43:0/3178187819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd82a0aaa0 0x55dd82a0d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.119+0000 7fe725e68640 1 --2- 172.21.15.43:0/3178187819 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55dd829dd700 0x55dd829e0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.120+0000 7fe724e66640 1 -- 172.21.15.43:0/3178187819 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55dd829dd700 msgr2=0x55dd829e0f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:21.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.120+0000 7fe724e66640 1 --2- 172.21.15.43:0/3178187819 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55dd829dd700 0x55dd829e0f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.120+0000 7fe724e66640 1 -- 172.21.15.43:0/3178187819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd82a0aaa0 msgr2=0x55dd82a0d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:21.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.120+0000 7fe724e66640 1 --2- 172.21.15.43:0/3178187819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd82a0aaa0 0x55dd82a0d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.120+0000 7fe724e66640 1 -- 172.21.15.43:0/3178187819 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd82911df0 con 0x55dd82a0ae80 2024-04-25T08:57:21.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.120+0000 7fe724e66640 1 --2- 172.21.15.43:0/3178187819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd82a0ae80 0x55dd82a0d7f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fe714061120 tx=0x7fe714059640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fa970f4fbc9c09a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:21.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.120+0000 7fe706ffd640 1 -- 172.21.15.43:0/3178187819 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe714070020 con 0x55dd82a0ae80 2024-04-25T08:57:21.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.120+0000 7fe706ffd640 1 -- 172.21.15.43:0/3178187819 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe71405d090 con 0x55dd82a0ae80 2024-04-25T08:57:21.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.120+0000 7fe706ffd640 1 -- 172.21.15.43:0/3178187819 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe714061650 con 0x55dd82a0ae80 2024-04-25T08:57:21.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.121+0000 7fe72a5d1740 1 -- 172.21.15.43:0/3178187819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd82a0ae80 msgr2=0x55dd82a0d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.121+0000 7fe72a5d1740 1 --2- 172.21.15.43:0/3178187819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd82a0ae80 0x55dd82a0d7f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fe714061120 tx=0x7fe714059640 comp rx=0 tx=0).stop 2024-04-25T08:57:21.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.121+0000 7fe72a5d1740 1 -- 172.21.15.43:0/3178187819 shutdown_connections 2024-04-25T08:57:21.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.121+0000 7fe72a5d1740 1 --2- 172.21.15.43:0/3178187819 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55dd829dd700 0x55dd829e0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.121+0000 7fe72a5d1740 1 --2- 172.21.15.43:0/3178187819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd82a0ae80 0x55dd82a0d7f0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.121+0000 7fe72a5d1740 1 --2- 172.21.15.43:0/3178187819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd82a0aaa0 0x55dd82a0d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.121+0000 7fe72a5d1740 1 -- 172.21.15.43:0/3178187819 >> 172.21.15.43:0/3178187819 conn(0x55dd829df970 msgr2=0x55dd82a282b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:21.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.121+0000 7fe72a5d1740 1 -- 172.21.15.43:0/3178187819 shutdown_connections 2024-04-25T08:57:21.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.121+0000 7fe72a5d1740 1 -- 172.21.15.43:0/3178187819 wait complete. 2024-04-25T08:57:21.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.121+0000 7fe72a5d1740 1 -- start start 2024-04-25T08:57:21.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.121+0000 7fe72a5d1740 1 Processor -- start 2024-04-25T08:57:21.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.122+0000 7fe72a5d1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd82a0aaa0 0x55dd82a2a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.122+0000 7fe72a5d1740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55dd82a0ae80 0x55dd82a242f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.122+0000 7fe72a5d1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd82a01c30 0x55dd82a2c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.122+0000 7fe72a5d1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd829e2b20 con 0x55dd82a01c30 2024-04-25T08:57:21.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.122+0000 7fe72a5d1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd828e5590 con 0x55dd82a0aaa0 2024-04-25T08:57:21.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.122+0000 7fe72a5d1740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dd829e2e00 con 0x55dd82a0ae80 2024-04-25T08:57:21.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.122+0000 7fe72a5d1740 10 client.0 _ll_register_callbacks cb 0x55dd828f1e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:21.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.122+0000 7fe725e68640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd82a01c30 0x55dd82a2c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.122+0000 7fe725e68640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd82a01c30 0x55dd82a2c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42044/0 (socket says 172.21.15.43:42044) 2024-04-25T08:57:21.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.122+0000 7fe725e68640 1 -- 172.21.15.43:0/1818695264 learned_addr learned my addr 172.21.15.43:0/1818695264 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:21.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.122+0000 7fe724e66640 1 --2- 172.21.15.43:0/1818695264 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55dd82a0ae80 0x55dd82a242f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.122+0000 7fe725667640 1 --2- 172.21.15.43:0/1818695264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd82a0aaa0 0x55dd82a2a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.123+0000 7fe725e68640 1 -- 172.21.15.43:0/1818695264 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55dd82a0ae80 msgr2=0x55dd82a242f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.123+0000 7fe725e68640 1 --2- 172.21.15.43:0/1818695264 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55dd82a0ae80 0x55dd82a242f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.123+0000 7fe725e68640 1 -- 172.21.15.43:0/1818695264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd82a0aaa0 msgr2=0x55dd82a2a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.123+0000 7fe725e68640 1 --2- 172.21.15.43:0/1818695264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd82a0aaa0 0x55dd82a2a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.123+0000 7fe725e68640 1 -- 172.21.15.43:0/1818695264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dd8265b3f0 con 0x55dd82a01c30 2024-04-25T08:57:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.123+0000 7fe725e68640 1 --2- 172.21.15.43:0/1818695264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd82a01c30 0x55dd82a2c090 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fe71000a950 tx=0x7fe71002d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c675b5b71a0687f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.123+0000 7fe6fffff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:57:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.123+0000 7fe6fffff640 1 -- 172.21.15.43:0/1818695264 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe71002bc60 con 0x55dd82a01c30 2024-04-25T08:57:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.123+0000 7fe6fffff640 1 -- 172.21.15.43:0/1818695264 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe71002be00 con 0x55dd82a01c30 2024-04-25T08:57:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.123+0000 7fe72a5d1740 1 -- 172.21.15.43:0/1818695264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe71405d090 con 0x55dd82a01c30 2024-04-25T08:57:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.123+0000 7fe6fffff640 1 -- 172.21.15.43:0/1818695264 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe71004b050 con 0x55dd82a01c30 2024-04-25T08:57:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.123+0000 7fe72a5d1740 10 client.5985 fetch_fsmap learned FSMap version 21 2024-04-25T08:57:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.123+0000 7fe72a5d1740 1 -- 172.21.15.43:0/1818695264 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x55dd827257d0 con 0x55dd82a01c30 2024-04-25T08:57:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.123+0000 7fe6fffff640 1 -- 172.21.15.43:0/1818695264 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe71405d090 con 0x55dd82a01c30 2024-04-25T08:57:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.123+0000 7fe6fffff640 1 -- 172.21.15.43:0/1818695264 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe71004e020 con 0x55dd82a01c30 2024-04-25T08:57:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.123+0000 7fe72a5d1740 10 client.5985 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T08:57:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.123+0000 7fe72a5d1740 10 client.5985 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T08:57:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.123+0000 7fe72a5d1740 20 client.5985 populate_metadata read hostname 'smithi043' 2024-04-25T08:57:21.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.123+0000 7fe72a5d1740 1 --2- 172.21.15.43:0/1818695264 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55dd829e23f0 0x55dd82a228e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.123+0000 7fe72a5d1740 4 client.5985 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:57:21.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.123+0000 7fe72a5d1740 1 -- 172.21.15.43:0/1818695264 --> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dd82954340 con 0x55dd829e23f0 2024-04-25T08:57:21.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.124+0000 7fe725667640 1 --2- 172.21.15.43:0/1818695264 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55dd829e23f0 0x55dd82a228e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.124+0000 7fe6fffff640 10 client.5985 ms_handle_connect on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:21.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.124+0000 7fe725667640 1 --2- 172.21.15.43:0/1818695264 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55dd829e23f0 0x55dd82a228e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b623b52cec213bd server_cookie=8c40a8d9a11a76c7 in_seq=0 out_seq=0 2024-04-25T08:57:21.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.124+0000 7fe6fffff640 1 -- 172.21.15.43:0/1818695264 <== mds.0 v2:172.21.15.193:6836/4164634380 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dd82954340 con 0x55dd829e23f0 2024-04-25T08:57:21.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.124+0000 7fe6fffff640 10 client.5985 handle_command_reply: tid=0 2024-04-25T08:57:21.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.125+0000 7fe72a5d1740 1 client.5985 shutdown 2024-04-25T08:57:21.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.125+0000 7fe72a5d1740 1 -- 172.21.15.43:0/1818695264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd82a01c30 msgr2=0x55dd82a2c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.125+0000 7fe72a5d1740 1 --2- 172.21.15.43:0/1818695264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd82a01c30 0x55dd82a2c090 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fe71000a950 tx=0x7fe71002d070 comp rx=0 tx=0).stop 2024-04-25T08:57:21.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.125+0000 7fe72a5d1740 1 -- 172.21.15.43:0/1818695264 shutdown_connections 2024-04-25T08:57:21.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.125+0000 7fe72a5d1740 1 --2- 172.21.15.43:0/1818695264 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd82a01c30 0x55dd82a2c090 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.125+0000 7fe72a5d1740 1 --2- 172.21.15.43:0/1818695264 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55dd82a0ae80 0x55dd82a242f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.125+0000 7fe72a5d1740 1 --2- 172.21.15.43:0/1818695264 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55dd829e23f0 0x55dd82a228e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.125+0000 7fe72a5d1740 1 --2- 172.21.15.43:0/1818695264 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd82a0aaa0 0x55dd82a2a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.125+0000 7fe72a5d1740 1 -- 172.21.15.43:0/1818695264 >> 172.21.15.43:0/1818695264 conn(0x55dd829dd700 msgr2=0x55dd82a09ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:21.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.125+0000 7fe6fffff640 0 client.5985 ms_handle_reset on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:21.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.125+0000 7fe72a5d1740 1 -- 172.21.15.43:0/1818695264 shutdown_connections 2024-04-25T08:57:21.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.125+0000 7fe72a5d1740 1 -- 172.21.15.43:0/1818695264 wait complete. 2024-04-25T08:57:21.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.125+0000 7fe72a5d1740 20 client.5985 trim_cache size 0 max 16384 2024-04-25T08:57:21.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.137+0000 7fe72a5d1740 1 Processor -- start 2024-04-25T08:57:21.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.137+0000 7fe72a5d1740 1 -- start start 2024-04-25T08:57:21.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.138+0000 7fe72a5d1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd82a01c30 0x55dd829dc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.138+0000 7fe72a5d1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd82a0aaa0 0x55dd82a09d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.138+0000 7fe72a5d1740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55dd82a0ae80 0x55dd82a0a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.138+0000 7fe72a5d1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe720200680 con 0x55dd82a0aaa0 2024-04-25T08:57:21.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.138+0000 7fe72a5d1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7201ff870 con 0x55dd82a01c30 2024-04-25T08:57:21.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.138+0000 7fe72a5d1740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe720146e70 con 0x55dd82a0ae80 2024-04-25T08:57:21.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.138+0000 7fe724e66640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd82a0aaa0 0x55dd82a09d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.138+0000 7fe724e66640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd82a0aaa0 0x55dd82a09d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42054/0 (socket says 172.21.15.43:42054) 2024-04-25T08:57:21.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.138+0000 7fe724e66640 1 -- 172.21.15.43:0/2997176545 learned_addr learned my addr 172.21.15.43:0/2997176545 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:21.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.138+0000 7fe725667640 1 --2- 172.21.15.43:0/2997176545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd82a01c30 0x55dd829dc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.138+0000 7fe724e66640 1 -- 172.21.15.43:0/2997176545 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55dd82a0ae80 msgr2=0x55dd82a0a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:21.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.138+0000 7fe724e66640 1 --2- 172.21.15.43:0/2997176545 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55dd82a0ae80 0x55dd82a0a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.138+0000 7fe724e66640 1 -- 172.21.15.43:0/2997176545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd82a01c30 msgr2=0x55dd829dc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:21.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.138+0000 7fe724e66640 1 --2- 172.21.15.43:0/2997176545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd82a01c30 0x55dd829dc8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.138+0000 7fe724e66640 1 -- 172.21.15.43:0/2997176545 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd82a2b510 con 0x55dd82a0aaa0 2024-04-25T08:57:21.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.138+0000 7fe724e66640 1 --2- 172.21.15.43:0/2997176545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd82a0aaa0 0x55dd82a09d80 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fe7140597f0 tx=0x7fe71405f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c4e3130b1c6a9da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:21.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.138+0000 7fe7067fc640 1 -- 172.21.15.43:0/2997176545 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe714070040 con 0x55dd82a0aaa0 2024-04-25T08:57:21.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.138+0000 7fe7067fc640 1 -- 172.21.15.43:0/2997176545 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe714061650 con 0x55dd82a0aaa0 2024-04-25T08:57:21.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.138+0000 7fe7067fc640 1 -- 172.21.15.43:0/2997176545 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7140555b0 con 0x55dd82a0aaa0 2024-04-25T08:57:21.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.139+0000 7fe72a5d1740 1 -- 172.21.15.43:0/2997176545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd82a0aaa0 msgr2=0x55dd82a09d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.139+0000 7fe72a5d1740 1 --2- 172.21.15.43:0/2997176545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd82a0aaa0 0x55dd82a09d80 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fe7140597f0 tx=0x7fe71405f000 comp rx=0 tx=0).stop 2024-04-25T08:57:21.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.139+0000 7fe72a5d1740 1 -- 172.21.15.43:0/2997176545 shutdown_connections 2024-04-25T08:57:21.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.139+0000 7fe72a5d1740 1 --2- 172.21.15.43:0/2997176545 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55dd82a0ae80 0x55dd82a0a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.139+0000 7fe72a5d1740 1 --2- 172.21.15.43:0/2997176545 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd82a0aaa0 0x55dd82a09d80 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.139+0000 7fe72a5d1740 1 --2- 172.21.15.43:0/2997176545 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd82a01c30 0x55dd829dc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.139+0000 7fe72a5d1740 1 -- 172.21.15.43:0/2997176545 >> 172.21.15.43:0/2997176545 conn(0x55dd829dd700 msgr2=0x55dd82a2c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:21.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.139+0000 7fe72a5d1740 1 -- 172.21.15.43:0/2997176545 shutdown_connections 2024-04-25T08:57:21.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.139+0000 7fe72a5d1740 1 -- 172.21.15.43:0/2997176545 wait complete. 2024-04-25T08:57:21.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.139+0000 7fe72a5d1740 1 -- start start 2024-04-25T08:57:21.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.139+0000 7fe72a5d1740 1 Processor -- start 2024-04-25T08:57:21.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.140+0000 7fe72a5d1740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd82a01c30 0x55dd82a92910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.140+0000 7fe72a5d1740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55dd82a0aaa0 0x55dd82a09cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.140+0000 7fe72a5d1740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd82a0ae80 0x55dd82a0a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.140+0000 7fe72a5d1740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe720147470 con 0x55dd82a0ae80 2024-04-25T08:57:21.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.140+0000 7fe72a5d1740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd829e4590 con 0x55dd82a01c30 2024-04-25T08:57:21.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.140+0000 7fe72a5d1740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dd82a228e0 con 0x55dd82a0aaa0 2024-04-25T08:57:21.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.140+0000 7fe72a5d1740 10 client.0 _ll_register_callbacks cb 0x7fe71408a040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:21.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.141+0000 7fe725e68640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd82a0ae80 0x55dd82a0a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.141+0000 7fe725667640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd82a01c30 0x55dd82a92910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.141+0000 7fe725667640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd82a01c30 0x55dd82a92910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:44726/0 (socket says 172.21.15.43:44726) 2024-04-25T08:57:21.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.141+0000 7fe725667640 1 -- 172.21.15.43:0/4016731476 learned_addr learned my addr 172.21.15.43:0/4016731476 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:21.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.141+0000 7fe724e66640 1 --2- 172.21.15.43:0/4016731476 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55dd82a0aaa0 0x55dd82a09cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.141+0000 7fe725e68640 1 -- 172.21.15.43:0/4016731476 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55dd82a0aaa0 msgr2=0x55dd82a09cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:21.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.141+0000 7fe725e68640 1 --2- 172.21.15.43:0/4016731476 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55dd82a0aaa0 0x55dd82a09cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.141+0000 7fe725e68640 1 -- 172.21.15.43:0/4016731476 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd82a01c30 msgr2=0x55dd82a92910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:21.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.141+0000 7fe725e68640 1 --2- 172.21.15.43:0/4016731476 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd82a01c30 0x55dd82a92910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.141+0000 7fe725e68640 1 -- 172.21.15.43:0/4016731476 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dd82a103b0 con 0x55dd82a0ae80 2024-04-25T08:57:21.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.141+0000 7fe725e68640 1 --2- 172.21.15.43:0/4016731476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd82a0ae80 0x55dd82a0a230 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fe7100013d0 tx=0x7fe710001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4694735a5df1b502 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:21.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.141+0000 7fe6fffff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:57:21.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.141+0000 7fe6fffff640 1 -- 172.21.15.43:0/4016731476 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe710004d60 con 0x55dd82a0ae80 2024-04-25T08:57:21.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.141+0000 7fe6fffff640 1 -- 172.21.15.43:0/4016731476 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe71002ba60 con 0x55dd82a0ae80 2024-04-25T08:57:21.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.141+0000 7fe72a5d1740 1 -- 172.21.15.43:0/4016731476 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe714061650 con 0x55dd82a0ae80 2024-04-25T08:57:21.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.141+0000 7fe6fffff640 1 -- 172.21.15.43:0/4016731476 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe71005c050 con 0x55dd82a0ae80 2024-04-25T08:57:21.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.141+0000 7fe72a5d1740 10 client.5991 fetch_fsmap learned FSMap version 21 2024-04-25T08:57:21.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.141+0000 7fe72a5d1740 1 -- 172.21.15.43:0/4016731476 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x55dd82a086c0 con 0x55dd82a0ae80 2024-04-25T08:57:21.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.141+0000 7fe6fffff640 1 -- 172.21.15.43:0/4016731476 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe714061650 con 0x55dd82a0ae80 2024-04-25T08:57:21.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.142+0000 7fe6fffff640 1 -- 172.21.15.43:0/4016731476 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe71005f020 con 0x55dd82a0ae80 2024-04-25T08:57:21.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.142+0000 7fe72a5d1740 10 client.5991 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T08:57:21.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.142+0000 7fe72a5d1740 10 client.5991 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T08:57:21.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.142+0000 7fe72a5d1740 20 client.5991 populate_metadata read hostname 'smithi043' 2024-04-25T08:57:21.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.142+0000 7fe72a5d1740 1 --2- 172.21.15.43:0/4016731476 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55dd82a20f50 0x55dd82a21360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.142+0000 7fe72a5d1740 4 client.5991 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T08:57:21.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.142+0000 7fe72a5d1740 1 -- 172.21.15.43:0/4016731476 --> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dd8294e690 con 0x55dd82a20f50 2024-04-25T08:57:21.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.142+0000 7fe725667640 1 --2- 172.21.15.43:0/4016731476 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55dd82a20f50 0x55dd82a21360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.142+0000 7fe6fffff640 10 client.5991 ms_handle_connect on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:21.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.142+0000 7fe725667640 1 --2- 172.21.15.43:0/4016731476 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55dd82a20f50 0x55dd82a21360 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2c5f981bbeb0e45 server_cookie=2918ec33a2151a7 in_seq=0 out_seq=0 2024-04-25T08:57:21.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.142+0000 7fe6fffff640 1 -- 172.21.15.43:0/4016731476 <== mds.0 v2:172.21.15.193:6836/4164634380 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dd8294e690 con 0x55dd82a20f50 2024-04-25T08:57:21.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.142+0000 7fe6fffff640 10 client.5991 handle_command_reply: tid=0 2024-04-25T08:57:21.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.142+0000 7fe72a5d1740 1 client.5991 shutdown 2024-04-25T08:57:21.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.142+0000 7fe72a5d1740 1 -- 172.21.15.43:0/4016731476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd82a0ae80 msgr2=0x55dd82a0a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.142+0000 7fe72a5d1740 1 --2- 172.21.15.43:0/4016731476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd82a0ae80 0x55dd82a0a230 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fe7100013d0 tx=0x7fe710001dc0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.142+0000 7fe72a5d1740 1 -- 172.21.15.43:0/4016731476 shutdown_connections 2024-04-25T08:57:21.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.142+0000 7fe72a5d1740 1 --2- 172.21.15.43:0/4016731476 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dd82a0ae80 0x55dd82a0a230 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.142+0000 7fe72a5d1740 1 --2- 172.21.15.43:0/4016731476 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55dd82a0aaa0 0x55dd82a09cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.142+0000 7fe72a5d1740 1 --2- 172.21.15.43:0/4016731476 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55dd82a20f50 0x55dd82a21360 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.142+0000 7fe72a5d1740 1 --2- 172.21.15.43:0/4016731476 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dd82a01c30 0x55dd82a92910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.142+0000 7fe72a5d1740 1 -- 172.21.15.43:0/4016731476 >> 172.21.15.43:0/4016731476 conn(0x55dd829dd700 msgr2=0x55dd829dc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:21.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.142+0000 7fe6fffff640 0 client.5991 ms_handle_reset on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:21.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.143+0000 7fe72a5d1740 1 -- 172.21.15.43:0/4016731476 shutdown_connections 2024-04-25T08:57:21.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.143+0000 7fe72a5d1740 1 -- 172.21.15.43:0/4016731476 wait complete. 2024-04-25T08:57:21.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.143+0000 7fe72a5d1740 20 client.5991 trim_cache size 0 max 16384 2024-04-25T08:57:21.138 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T08:57:21.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.143+0000 7fe7278cb640 1 -- 172.21.15.43:0/2991917737 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe70c06b500 msgr2=0x7fe70c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.143+0000 7fe7278cb640 1 --2- 172.21.15.43:0/2991917737 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe70c06b500 0x7fe70c06d9c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe718009b40 tx=0x7fe718009190 comp rx=0 tx=0).stop 2024-04-25T08:57:21.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.143+0000 7fe7278cb640 1 -- 172.21.15.43:0/2991917737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe720155f90 msgr2=0x7fe7200ae3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.143+0000 7fe7278cb640 1 --2- 172.21.15.43:0/2991917737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe720155f90 0x7fe7200ae3b0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fe714011a70 tx=0x7fe714011e70 comp rx=0 tx=0).stop 2024-04-25T08:57:21.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.143+0000 7fe7278cb640 1 -- 172.21.15.43:0/2991917737 shutdown_connections 2024-04-25T08:57:21.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.143+0000 7fe7278cb640 1 --2- 172.21.15.43:0/2991917737 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe720156470 0x7fe7200ae8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.143+0000 7fe7278cb640 1 --2- 172.21.15.43:0/2991917737 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe720155f90 0x7fe7200ae3b0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.143+0000 7fe7278cb640 1 --2- 172.21.15.43:0/2991917737 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe70c06b500 0x7fe70c06d9c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.143+0000 7fe7278cb640 1 --2- 172.21.15.43:0/2991917737 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe708004d40 0x7fe720145610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.143+0000 7fe7278cb640 1 -- 172.21.15.43:0/2991917737 >> 172.21.15.43:0/2991917737 conn(0x7fe72006ab30 msgr2=0x7fe72006af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:21.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.143+0000 7fe7278cb640 1 -- 172.21.15.43:0/2991917737 shutdown_connections 2024-04-25T08:57:21.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.143+0000 7fe7278cb640 1 -- 172.21.15.43:0/2991917737 wait complete. 2024-04-25T08:57:21.143 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data rmxattr 10000000a4c.00000000 parent 2024-04-25T08:57:21.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.197+0000 7f52ce8aff40 1 Processor -- start 2024-04-25T08:57:21.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.197+0000 7f52ce8aff40 1 -- start start 2024-04-25T08:57:21.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.197+0000 7f52ce8aff40 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563a340dadc0 0x563a341a6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.197+0000 7f52ce8aff40 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a34126290 0x563a34123f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.197+0000 7f52ce8aff40 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a341b3f00 0x563a34124440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.197+0000 7f52ce8aff40 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a340db1c0 con 0x563a34126290 2024-04-25T08:57:21.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.197+0000 7f52ce8aff40 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a34083df0 con 0x563a341b3f00 2024-04-25T08:57:21.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.197+0000 7f52ce8aff40 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x563a341267d0 con 0x563a340dadc0 2024-04-25T08:57:21.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.197+0000 7f52cbbaf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a34126290 0x563a34123f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.198+0000 7f52cbbaf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a34126290 0x563a34123f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42080/0 (socket says 172.21.15.43:42080) 2024-04-25T08:57:21.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.198+0000 7f52cbbaf640 1 -- 172.21.15.43:0/840017521 learned_addr learned my addr 172.21.15.43:0/840017521 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:21.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.198+0000 7f52cc3b0640 1 --2- 172.21.15.43:0/840017521 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563a340dadc0 0x563a341a6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.198+0000 7f52ccbb1640 1 --2- 172.21.15.43:0/840017521 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a341b3f00 0x563a34124440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.198+0000 7f52cbbaf640 1 -- 172.21.15.43:0/840017521 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563a340dadc0 msgr2=0x563a341a6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:21.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.198+0000 7f52cbbaf640 1 --2- 172.21.15.43:0/840017521 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563a340dadc0 0x563a341a6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.198+0000 7f52cbbaf640 1 -- 172.21.15.43:0/840017521 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a341b3f00 msgr2=0x563a34124440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:21.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.198+0000 7f52cbbaf640 1 --2- 172.21.15.43:0/840017521 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a341b3f00 0x563a34124440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.198+0000 7f52cbbaf640 1 -- 172.21.15.43:0/840017521 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a34009c50 con 0x563a34126290 2024-04-25T08:57:21.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.198+0000 7f52cbbaf640 1 --2- 172.21.15.43:0/840017521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a34126290 0x563a34123f00 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f52b4025aa0 tx=0x7f52b404e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a44a02b95ccdb42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:21.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.198+0000 7f52cabad640 1 -- 172.21.15.43:0/840017521 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b4065070 con 0x563a34126290 2024-04-25T08:57:21.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.198+0000 7f52cabad640 1 -- 172.21.15.43:0/840017521 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b40549f0 con 0x563a34126290 2024-04-25T08:57:21.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.198+0000 7f52cabad640 1 -- 172.21.15.43:0/840017521 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b4059040 con 0x563a34126290 2024-04-25T08:57:21.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.199+0000 7f52ce8aff40 1 -- 172.21.15.43:0/840017521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a34126290 msgr2=0x563a34123f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.199+0000 7f52ce8aff40 1 --2- 172.21.15.43:0/840017521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a34126290 0x563a34123f00 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f52b4025aa0 tx=0x7f52b404e000 comp rx=0 tx=0).stop 2024-04-25T08:57:21.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.199+0000 7f52ce8aff40 1 -- 172.21.15.43:0/840017521 shutdown_connections 2024-04-25T08:57:21.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.199+0000 7f52ce8aff40 1 --2- 172.21.15.43:0/840017521 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a341b3f00 0x563a34124440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.199+0000 7f52ce8aff40 1 --2- 172.21.15.43:0/840017521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a34126290 0x563a34123f00 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.199+0000 7f52ce8aff40 1 --2- 172.21.15.43:0/840017521 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563a340dadc0 0x563a341a6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.199+0000 7f52ce8aff40 1 -- 172.21.15.43:0/840017521 >> 172.21.15.43:0/840017521 conn(0x563a341074a0 msgr2=0x563a34106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:21.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.199+0000 7f52ce8aff40 1 -- 172.21.15.43:0/840017521 shutdown_connections 2024-04-25T08:57:21.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.199+0000 7f52ce8aff40 1 -- 172.21.15.43:0/840017521 wait complete. 2024-04-25T08:57:21.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.199+0000 7f52ce8aff40 1 Processor -- start 2024-04-25T08:57:21.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.199+0000 7f52ce8aff40 1 -- start start 2024-04-25T08:57:21.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.200+0000 7f52ce8aff40 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563a340dadc0 0x563a340deb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.200+0000 7f52ce8aff40 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a34126290 0x563a341a81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.200+0000 7f52ce8aff40 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a341b3f00 0x563a341a33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.200+0000 7f52ce8aff40 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a34120b80 con 0x563a34126290 2024-04-25T08:57:21.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.200+0000 7f52ce8aff40 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a341213f0 con 0x563a341b3f00 2024-04-25T08:57:21.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.200+0000 7f52ce8aff40 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x563a341a7400 con 0x563a340dadc0 2024-04-25T08:57:21.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.200+0000 7f52cbbaf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a34126290 0x563a341a81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.200+0000 7f52cbbaf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a34126290 0x563a341a81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42084/0 (socket says 172.21.15.43:42084) 2024-04-25T08:57:21.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.200+0000 7f52cbbaf640 1 -- 172.21.15.43:0/3098147039 learned_addr learned my addr 172.21.15.43:0/3098147039 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:21.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.200+0000 7f52cc3b0640 1 --2- 172.21.15.43:0/3098147039 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563a340dadc0 0x563a340deb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.200+0000 7f52ccbb1640 1 --2- 172.21.15.43:0/3098147039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a341b3f00 0x563a341a33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.200+0000 7f52cbbaf640 1 -- 172.21.15.43:0/3098147039 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563a340dadc0 msgr2=0x563a340deb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.200+0000 7f52cbbaf640 1 --2- 172.21.15.43:0/3098147039 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563a340dadc0 0x563a340deb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.200+0000 7f52cbbaf640 1 -- 172.21.15.43:0/3098147039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a341b3f00 msgr2=0x563a341a33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.200+0000 7f52cbbaf640 1 --2- 172.21.15.43:0/3098147039 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a341b3f00 0x563a341a33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.200+0000 7f52cbbaf640 1 -- 172.21.15.43:0/3098147039 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a341a9c50 con 0x563a34126290 2024-04-25T08:57:21.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.201+0000 7f52cbbaf640 1 --2- 172.21.15.43:0/3098147039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a34126290 0x563a341a81a0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f52b4025aa0 tx=0x7f52b4002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:21.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.201+0000 7f52c9bab640 1 -- 172.21.15.43:0/3098147039 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b4070070 con 0x563a34126290 2024-04-25T08:57:21.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.201+0000 7f52c9bab640 1 -- 172.21.15.43:0/3098147039 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52b4054da0 con 0x563a34126290 2024-04-25T08:57:21.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.201+0000 7f52ce8aff40 1 -- 172.21.15.43:0/3098147039 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563a341a9f30 con 0x563a34126290 2024-04-25T08:57:21.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.201+0000 7f52c9bab640 1 -- 172.21.15.43:0/3098147039 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52b4065040 con 0x563a34126290 2024-04-25T08:57:21.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.201+0000 7f52ce8aff40 1 -- 172.21.15.43:0/3098147039 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563a341a7c80 con 0x563a34126290 2024-04-25T08:57:21.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.202+0000 7f52c9bab640 1 -- 172.21.15.43:0/3098147039 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f52b4073070 con 0x563a34126290 2024-04-25T08:57:21.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.203+0000 7f52c9bab640 1 --2- 172.21.15.43:0/3098147039 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f52ac069ee0 0x7f52ac06c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.203+0000 7f52c9bab640 1 -- 172.21.15.43:0/3098147039 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f52b40cd7a0 con 0x563a34126290 2024-04-25T08:57:21.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.203+0000 7f52cc3b0640 1 --2- 172.21.15.43:0/3098147039 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f52ac069ee0 0x7f52ac06c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.203+0000 7f52ce8aff40 1 --2- 172.21.15.43:0/3098147039 >> [v2:172.21.15.43:6800/1101549916,v1:172.21.15.43:6801/1101549916] conn(0x563a34121e50 0x563a341a3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.203+0000 7f52ce8aff40 1 -- 172.21.15.43:0/3098147039 --> [v2:172.21.15.43:6800/1101549916,v1:172.21.15.43:6801/1101549916] -- osd_op(unknown.0.0:1 7.3a 7:5fa3b718:::10000000a4c.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46) v9 -- 0x563a341a41b0 con 0x563a34121e50 2024-04-25T08:57:21.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.203+0000 7f52ccbb1640 1 --2- 172.21.15.43:0/3098147039 >> [v2:172.21.15.43:6800/1101549916,v1:172.21.15.43:6801/1101549916] conn(0x563a34121e50 0x563a341a3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.203+0000 7f52cc3b0640 1 --2- 172.21.15.43:0/3098147039 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f52ac069ee0 0x7f52ac06c3a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x563a34121780 tx=0x7f52bc005d80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:21.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.203+0000 7f52ccbb1640 1 --2- 172.21.15.43:0/3098147039 >> [v2:172.21.15.43:6800/1101549916,v1:172.21.15.43:6801/1101549916] conn(0x563a34121e50 0x563a341a3c70 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:21.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.205+0000 7f52ccbb1640 1 -- 172.21.15.43:0/3098147039 <== osd.3 v2:172.21.15.43:6800/1101549916 1 ==== osd_op_reply(1 10000000a4c.00000000 [rmxattr] v46'24 uv24 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f52c4009040 con 0x563a34121e50 2024-04-25T08:57:21.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.205+0000 7f52ce8aff40 1 -- 172.21.15.43:0/3098147039 >> [v2:172.21.15.43:6800/1101549916,v1:172.21.15.43:6801/1101549916] conn(0x563a34121e50 msgr2=0x563a341a3c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.205+0000 7f52ce8aff40 1 --2- 172.21.15.43:0/3098147039 >> [v2:172.21.15.43:6800/1101549916,v1:172.21.15.43:6801/1101549916] conn(0x563a34121e50 0x563a341a3c70 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.205+0000 7f52ce8aff40 1 -- 172.21.15.43:0/3098147039 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f52ac069ee0 msgr2=0x7f52ac06c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.205+0000 7f52ce8aff40 1 --2- 172.21.15.43:0/3098147039 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f52ac069ee0 0x7f52ac06c3a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x563a34121780 tx=0x7f52bc005d80 comp rx=0 tx=0).stop 2024-04-25T08:57:21.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.205+0000 7f52ce8aff40 1 -- 172.21.15.43:0/3098147039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a34126290 msgr2=0x563a341a81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.205+0000 7f52ce8aff40 1 --2- 172.21.15.43:0/3098147039 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a34126290 0x563a341a81a0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f52b4025aa0 tx=0x7f52b4002fb0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.205+0000 7f52ccbb1640 1 -- 172.21.15.43:0/3098147039 reap_dead start 2024-04-25T08:57:21.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.205+0000 7f52ce8aff40 1 -- 172.21.15.43:0/3098147039 shutdown_connections 2024-04-25T08:57:21.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.205+0000 7f52ce8aff40 1 -- 172.21.15.43:0/3098147039 >> 172.21.15.43:0/3098147039 conn(0x563a341074a0 msgr2=0x563a34106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:21.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.205+0000 7f52ce8aff40 1 -- 172.21.15.43:0/3098147039 shutdown_connections 2024-04-25T08:57:21.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.205+0000 7f52ce8aff40 1 -- 172.21.15.43:0/3098147039 wait complete. 2024-04-25T08:57:21.198 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_0' 2024-04-25T08:57:21.198 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:57:21.198 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/new_file_0 2024-04-25T08:57:21.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.352+0000 7fe870316640 1 Processor -- start 2024-04-25T08:57:21.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.352+0000 7fe870316640 1 -- start start 2024-04-25T08:57:21.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.353+0000 7fe870316640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fe8680b6110 0x7fe8680d1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.353+0000 7fe870316640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe8680d2340 0x7fe8681656d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.353+0000 7fe870316640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8680b32a0 con 0x7fe8680d1a20 2024-04-25T08:57:21.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.353+0000 7fe870316640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8680cf430 con 0x7fe8680d2340 2024-04-25T08:57:21.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.353+0000 7fe870316640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe86815bf90 con 0x7fe8680b6110 2024-04-25T08:57:21.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.353+0000 7fe86e8b3640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe8680d2340 0x7fe8681656d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.353+0000 7fe86e8b3640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe8680d2340 0x7fe8681656d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42100/0 (socket says 172.21.15.43:42100) 2024-04-25T08:57:21.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.353+0000 7fe86e8b3640 1 -- 172.21.15.43:0/7750113 learned_addr learned my addr 172.21.15.43:0/7750113 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:21.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.354+0000 7fe86d8b1640 1 --2- 172.21.15.43:0/7750113 >> v2:172.21.15.193:3300/0 conn(0x7fe8680b6110 0x7fe8680d1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.354+0000 7fe86e8b3640 1 -- 172.21.15.43:0/7750113 >> v2:172.21.15.193:3300/0 conn(0x7fe8680b6110 msgr2=0x7fe8680d1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:21.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.354+0000 7fe86e8b3640 1 --2- 172.21.15.43:0/7750113 >> v2:172.21.15.193:3300/0 conn(0x7fe8680b6110 0x7fe8680d1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.354+0000 7fe86e8b3640 1 -- 172.21.15.43:0/7750113 >> v1:172.21.15.193:6789/0 conn(0x7fe8680d1a20 legacy=0x7fe8680b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.354+0000 7fe86e8b3640 1 -- 172.21.15.43:0/7750113 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8680cee70 con 0x7fe8680d2340 2024-04-25T08:57:21.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.354+0000 7fe86e8b3640 1 --2- 172.21.15.43:0/7750113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8680d2340 0x7fe8681656d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fe86815f600 tx=0x7fe85802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67a9af35cbee1ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:21.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.355+0000 7fe86d0b0640 1 -- 172.21.15.43:0/7750113 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe858036070 con 0x7fe8680d2340 2024-04-25T08:57:21.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.355+0000 7fe86d0b0640 1 -- 172.21.15.43:0/7750113 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe858031600 con 0x7fe8680d2340 2024-04-25T08:57:21.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.355+0000 7fe86d0b0640 1 -- 172.21.15.43:0/7750113 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8580319d0 con 0x7fe8680d2340 2024-04-25T08:57:21.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.355+0000 7fe870316640 1 -- 172.21.15.43:0/7750113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8680d2340 msgr2=0x7fe8681656d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.355+0000 7fe870316640 1 --2- 172.21.15.43:0/7750113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8680d2340 0x7fe8681656d0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fe86815f600 tx=0x7fe85802a610 comp rx=0 tx=0).stop 2024-04-25T08:57:21.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.355+0000 7fe870316640 1 -- 172.21.15.43:0/7750113 shutdown_connections 2024-04-25T08:57:21.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.355+0000 7fe870316640 1 --2- 172.21.15.43:0/7750113 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8680d2340 0x7fe8681656d0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.355+0000 7fe870316640 1 --2- 172.21.15.43:0/7750113 >> v2:172.21.15.193:3300/0 conn(0x7fe8680b6110 0x7fe8680d1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.355+0000 7fe870316640 1 -- 172.21.15.43:0/7750113 >> 172.21.15.43:0/7750113 conn(0x7fe868072aa0 msgr2=0x7fe86809aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:21.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.355+0000 7fe870316640 1 -- 172.21.15.43:0/7750113 shutdown_connections 2024-04-25T08:57:21.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.355+0000 7fe870316640 1 -- 172.21.15.43:0/7750113 wait complete. 2024-04-25T08:57:21.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.356+0000 7fe870316640 1 Processor -- start 2024-04-25T08:57:21.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.357+0000 7fe870316640 1 -- start start 2024-04-25T08:57:21.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.357+0000 7fe870316640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8680b6110 0x7fe86806d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.357+0000 7fe870316640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8680d1a20 0x7fe86806c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.357+0000 7fe870316640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe8680d2340 0x7fe8681f76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.357+0000 7fe870316640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8680b4be0 con 0x7fe8680b6110 2024-04-25T08:57:21.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.357+0000 7fe870316640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8680b4900 con 0x7fe8680d1a20 2024-04-25T08:57:21.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.357+0000 7fe870316640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8680cea60 con 0x7fe8680d2340 2024-04-25T08:57:21.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.357+0000 7fe86e0b2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8680b6110 0x7fe86806d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.357+0000 7fe86e0b2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8680b6110 0x7fe86806d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42112/0 (socket says 172.21.15.43:42112) 2024-04-25T08:57:21.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.357+0000 7fe86e0b2640 1 -- 172.21.15.43:0/3582652030 learned_addr learned my addr 172.21.15.43:0/3582652030 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:21.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.357+0000 7fe86d8b1640 1 --2- 172.21.15.43:0/3582652030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8680d1a20 0x7fe86806c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.357+0000 7fe86e8b3640 1 --2- 172.21.15.43:0/3582652030 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe8680d2340 0x7fe8681f76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.358+0000 7fe86e0b2640 1 -- 172.21.15.43:0/3582652030 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe8680d2340 msgr2=0x7fe8681f76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.358+0000 7fe86e0b2640 1 --2- 172.21.15.43:0/3582652030 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe8680d2340 0x7fe8681f76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.358+0000 7fe86e0b2640 1 -- 172.21.15.43:0/3582652030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8680d1a20 msgr2=0x7fe86806c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.358+0000 7fe86e0b2640 1 --2- 172.21.15.43:0/3582652030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8680d1a20 0x7fe86806c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.358+0000 7fe86e0b2640 1 -- 172.21.15.43:0/3582652030 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe86815db00 con 0x7fe8680b6110 2024-04-25T08:57:21.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.358+0000 7fe86e0b2640 1 --2- 172.21.15.43:0/3582652030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8680b6110 0x7fe86806d8d0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fe86000c430 tx=0x7fe86000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:21.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.358+0000 7fe866ffd640 1 -- 172.21.15.43:0/3582652030 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe860017070 con 0x7fe8680b6110 2024-04-25T08:57:21.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.358+0000 7fe866ffd640 1 -- 172.21.15.43:0/3582652030 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8600052d0 con 0x7fe8680b6110 2024-04-25T08:57:21.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.358+0000 7fe870316640 1 -- 172.21.15.43:0/3582652030 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe86815dda0 con 0x7fe8680b6110 2024-04-25T08:57:21.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.358+0000 7fe866ffd640 1 -- 172.21.15.43:0/3582652030 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe860003850 con 0x7fe8680b6110 2024-04-25T08:57:21.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.358+0000 7fe870316640 1 -- 172.21.15.43:0/3582652030 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe86815e640 con 0x7fe8680b6110 2024-04-25T08:57:21.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.360+0000 7fe866ffd640 1 -- 172.21.15.43:0/3582652030 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe86001a070 con 0x7fe8680b6110 2024-04-25T08:57:21.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.360+0000 7fe866ffd640 1 --2- 172.21.15.43:0/3582652030 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe85406b4d0 0x7fe85406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.360+0000 7fe866ffd640 1 -- 172.21.15.43:0/3582652030 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe8600894b0 con 0x7fe8680b6110 2024-04-25T08:57:21.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.360+0000 7fe86d8b1640 1 --2- 172.21.15.43:0/3582652030 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe85406b4d0 0x7fe85406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.361+0000 7fe86d8b1640 1 --2- 172.21.15.43:0/3582652030 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe85406b4d0 0x7fe85406d990 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe8680cd780 tx=0x7fe85c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:21.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.361+0000 7fe873017740 1 Processor -- start 2024-04-25T08:57:21.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.361+0000 7fe873017740 1 -- start start 2024-04-25T08:57:21.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.362+0000 7fe873017740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616f760d360 0x5616f762d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.362+0000 7fe873017740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616f75ec360 0x5616f7614c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.362+0000 7fe873017740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5616f7615140 0x5616f75ea7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.362+0000 7fe873017740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616f754dee0 con 0x5616f75ec360 2024-04-25T08:57:21.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.362+0000 7fe873017740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616f74f6c50 con 0x5616f760d360 2024-04-25T08:57:21.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.362+0000 7fe873017740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x5616f7549f80 con 0x5616f7615140 2024-04-25T08:57:21.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.362+0000 7fe86e0b2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616f760d360 0x5616f762d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.362+0000 7fe86d8b1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616f75ec360 0x5616f7614c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.362+0000 7fe86e8b3640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5616f7615140 0x5616f75ea7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.362+0000 7fe86e0b2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616f760d360 0x5616f762d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:44776/0 (socket says 172.21.15.43:44776) 2024-04-25T08:57:21.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.362+0000 7fe86e0b2640 1 -- 172.21.15.43:0/2957197734 learned_addr learned my addr 172.21.15.43:0/2957197734 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:21.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.362+0000 7fe86e0b2640 1 -- 172.21.15.43:0/2957197734 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5616f7615140 msgr2=0x5616f75ea7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:21.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.362+0000 7fe86e0b2640 1 --2- 172.21.15.43:0/2957197734 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5616f7615140 0x5616f75ea7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.362+0000 7fe86e0b2640 1 -- 172.21.15.43:0/2957197734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616f75ec360 msgr2=0x5616f7614c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:21.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.362+0000 7fe86e0b2640 1 --2- 172.21.15.43:0/2957197734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616f75ec360 0x5616f7614c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.362+0000 7fe86e0b2640 1 -- 172.21.15.43:0/2957197734 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616f732ec00 con 0x5616f760d360 2024-04-25T08:57:21.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.362+0000 7fe86d8b1640 1 --2- 172.21.15.43:0/2957197734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616f75ec360 0x5616f7614c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:57:21.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.363+0000 7fe86e0b2640 1 --2- 172.21.15.43:0/2957197734 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616f760d360 0x5616f762d7d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe860087a90 tx=0x7fe8600055b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfe352164aba1006 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:21.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.363+0000 7fe84ffff640 1 -- 172.21.15.43:0/2957197734 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86005e0c0 con 0x5616f760d360 2024-04-25T08:57:21.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.363+0000 7fe84ffff640 1 -- 172.21.15.43:0/2957197734 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe860055e10 con 0x5616f760d360 2024-04-25T08:57:21.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.363+0000 7fe84ffff640 1 -- 172.21.15.43:0/2957197734 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8600543e0 con 0x5616f760d360 2024-04-25T08:57:21.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.363+0000 7fe873017740 1 -- 172.21.15.43:0/2957197734 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616f760d360 msgr2=0x5616f762d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.363+0000 7fe873017740 1 --2- 172.21.15.43:0/2957197734 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616f760d360 0x5616f762d7d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe860087a90 tx=0x7fe8600055b0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.363+0000 7fe873017740 1 -- 172.21.15.43:0/2957197734 shutdown_connections 2024-04-25T08:57:21.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.363+0000 7fe873017740 1 --2- 172.21.15.43:0/2957197734 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5616f7615140 0x5616f75ea7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.363+0000 7fe873017740 1 --2- 172.21.15.43:0/2957197734 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616f75ec360 0x5616f7614c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.363+0000 7fe873017740 1 --2- 172.21.15.43:0/2957197734 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616f760d360 0x5616f762d7d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.363+0000 7fe873017740 1 -- 172.21.15.43:0/2957197734 >> 172.21.15.43:0/2957197734 conn(0x5616f75b1810 msgr2=0x5616f762fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:21.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.364+0000 7fe873017740 1 -- 172.21.15.43:0/2957197734 shutdown_connections 2024-04-25T08:57:21.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.364+0000 7fe873017740 1 -- 172.21.15.43:0/2957197734 wait complete. 2024-04-25T08:57:21.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.364+0000 7fe873017740 1 -- start start 2024-04-25T08:57:21.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.364+0000 7fe873017740 1 Processor -- start 2024-04-25T08:57:21.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.365+0000 7fe873017740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616f75ec360 0x5616f76281d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.365+0000 7fe873017740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616f760d360 0x5616f7616770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.365+0000 7fe873017740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5616f7615140 0x5616f7626c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.365+0000 7fe873017740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616f75ea330 con 0x5616f75ec360 2024-04-25T08:57:21.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.365+0000 7fe873017740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616f74ec340 con 0x5616f760d360 2024-04-25T08:57:21.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.365+0000 7fe873017740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x5616f75e51a0 con 0x5616f7615140 2024-04-25T08:57:21.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.365+0000 7fe873017740 10 client.0 _ll_register_callbacks cb 0x5616f7261fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:21.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.365+0000 7fe86e0b2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616f75ec360 0x5616f76281d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.365+0000 7fe86e0b2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616f75ec360 0x5616f76281d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42124/0 (socket says 172.21.15.43:42124) 2024-04-25T08:57:21.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.365+0000 7fe86e0b2640 1 -- 172.21.15.43:0/32250169 learned_addr learned my addr 172.21.15.43:0/32250169 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:21.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.365+0000 7fe86d8b1640 1 --2- 172.21.15.43:0/32250169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616f760d360 0x5616f7616770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.365+0000 7fe86e8b3640 1 --2- 172.21.15.43:0/32250169 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5616f7615140 0x5616f7626c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.365+0000 7fe86e0b2640 1 -- 172.21.15.43:0/32250169 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5616f7615140 msgr2=0x5616f7626c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:21.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.365+0000 7fe86e0b2640 1 --2- 172.21.15.43:0/32250169 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5616f7615140 0x5616f7626c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.365+0000 7fe86e0b2640 1 -- 172.21.15.43:0/32250169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616f760d360 msgr2=0x5616f7616770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:21.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.365+0000 7fe86e0b2640 1 --2- 172.21.15.43:0/32250169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616f760d360 0x5616f7616770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.366+0000 7fe86e0b2640 1 -- 172.21.15.43:0/32250169 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5616f72224f0 con 0x5616f75ec360 2024-04-25T08:57:21.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.366+0000 7fe86e0b2640 1 --2- 172.21.15.43:0/32250169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616f75ec360 0x5616f76281d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fe860000fb0 tx=0x7fe860000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a25b6491b301349 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:21.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.366+0000 7fe84d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:57:21.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.366+0000 7fe84d7fa640 1 -- 172.21.15.43:0/32250169 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86005e0c0 con 0x5616f75ec360 2024-04-25T08:57:21.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.366+0000 7fe84d7fa640 1 -- 172.21.15.43:0/32250169 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe8600890e0 con 0x5616f75ec360 2024-04-25T08:57:21.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.366+0000 7fe873017740 1 -- 172.21.15.43:0/32250169 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe860055e10 con 0x5616f75ec360 2024-04-25T08:57:21.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.366+0000 7fe84d7fa640 1 -- 172.21.15.43:0/32250169 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe860089300 con 0x5616f75ec360 2024-04-25T08:57:21.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.366+0000 7fe873017740 10 client.6015 fetch_fsmap learned FSMap version 21 2024-04-25T08:57:21.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.366+0000 7fe873017740 1 -- 172.21.15.43:0/32250169 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x5616f738f390 con 0x5616f75ec360 2024-04-25T08:57:21.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.366+0000 7fe84d7fa640 1 -- 172.21.15.43:0/32250169 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe860055e10 con 0x5616f75ec360 2024-04-25T08:57:21.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.366+0000 7fe84d7fa640 1 -- 172.21.15.43:0/32250169 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe860076b90 con 0x5616f75ec360 2024-04-25T08:57:21.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.366+0000 7fe873017740 10 client.6015 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T08:57:21.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.366+0000 7fe873017740 10 client.6015 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T08:57:21.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.366+0000 7fe873017740 20 client.6015 populate_metadata read hostname 'smithi043' 2024-04-25T08:57:21.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.366+0000 7fe873017740 1 --2- 172.21.15.43:0/32250169 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5616f76271b0 0x5616f763fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.366+0000 7fe873017740 4 client.6015 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:57:21.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.366+0000 7fe873017740 1 -- 172.21.15.43:0/32250169 --> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5616f755b420 con 0x5616f76271b0 2024-04-25T08:57:21.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.366+0000 7fe86d8b1640 1 --2- 172.21.15.43:0/32250169 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5616f76271b0 0x5616f763fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.367+0000 7fe84d7fa640 10 client.6015 ms_handle_connect on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:21.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.367+0000 7fe86d8b1640 1 --2- 172.21.15.43:0/32250169 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5616f76271b0 0x5616f763fb70 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e064642d3c4bb450 server_cookie=c6f1552c005b7f46 in_seq=0 out_seq=0 2024-04-25T08:57:21.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.368+0000 7fe84d7fa640 1 -- 172.21.15.43:0/32250169 <== mds.0 v2:172.21.15.193:6836/4164634380 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5616f755b420 con 0x5616f76271b0 2024-04-25T08:57:21.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.368+0000 7fe84d7fa640 10 client.6015 handle_command_reply: tid=0 2024-04-25T08:57:21.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.368+0000 7fe873017740 1 client.6015 shutdown 2024-04-25T08:57:21.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.368+0000 7fe873017740 1 -- 172.21.15.43:0/32250169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616f75ec360 msgr2=0x5616f76281d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.368+0000 7fe873017740 1 --2- 172.21.15.43:0/32250169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616f75ec360 0x5616f76281d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fe860000fb0 tx=0x7fe860000bc0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.368+0000 7fe873017740 1 -- 172.21.15.43:0/32250169 shutdown_connections 2024-04-25T08:57:21.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.368+0000 7fe873017740 1 --2- 172.21.15.43:0/32250169 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5616f7615140 0x5616f7626c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.368+0000 7fe873017740 1 --2- 172.21.15.43:0/32250169 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5616f76271b0 0x5616f763fb70 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.368+0000 7fe873017740 1 --2- 172.21.15.43:0/32250169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616f760d360 0x5616f7616770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.368+0000 7fe873017740 1 --2- 172.21.15.43:0/32250169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616f75ec360 0x5616f76281d0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.368+0000 7fe84d7fa640 0 client.6015 ms_handle_reset on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:21.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.368+0000 7fe873017740 1 -- 172.21.15.43:0/32250169 >> 172.21.15.43:0/32250169 conn(0x5616f75b1810 msgr2=0x5616f75eaed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:21.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.368+0000 7fe873017740 1 -- 172.21.15.43:0/32250169 shutdown_connections 2024-04-25T08:57:21.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.368+0000 7fe873017740 1 -- 172.21.15.43:0/32250169 wait complete. 2024-04-25T08:57:21.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.368+0000 7fe873017740 20 client.6015 trim_cache size 0 max 16384 2024-04-25T08:57:21.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.383+0000 7fe873017740 1 Processor -- start 2024-04-25T08:57:21.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.384+0000 7fe873017740 1 -- start start 2024-04-25T08:57:21.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.384+0000 7fe873017740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5616f75ec360 0x5616f75e0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.384+0000 7fe873017740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616f760d360 0x5616f7633370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.384+0000 7fe873017740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616f7615140 0x5616f7610b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.384+0000 7fe873017740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8681fff90 con 0x5616f7615140 2024-04-25T08:57:21.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.384+0000 7fe873017740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8681f4af0 con 0x5616f760d360 2024-04-25T08:57:21.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.384+0000 7fe873017740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8681f3ef0 con 0x5616f75ec360 2024-04-25T08:57:21.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.384+0000 7fe86e8b3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616f7615140 0x5616f7610b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.384+0000 7fe86e8b3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616f7615140 0x5616f7610b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42136/0 (socket says 172.21.15.43:42136) 2024-04-25T08:57:21.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.384+0000 7fe86e8b3640 1 -- 172.21.15.43:0/1250874764 learned_addr learned my addr 172.21.15.43:0/1250874764 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:21.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.384+0000 7fe86e0b2640 1 --2- 172.21.15.43:0/1250874764 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5616f75ec360 0x5616f75e0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.384+0000 7fe86d8b1640 1 --2- 172.21.15.43:0/1250874764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616f760d360 0x5616f7633370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.385+0000 7fe86e8b3640 1 -- 172.21.15.43:0/1250874764 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5616f75ec360 msgr2=0x5616f75e0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:21.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.385+0000 7fe86e8b3640 1 --2- 172.21.15.43:0/1250874764 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5616f75ec360 0x5616f75e0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.385+0000 7fe86e8b3640 1 -- 172.21.15.43:0/1250874764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616f760d360 msgr2=0x5616f7633370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:21.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.385+0000 7fe86e8b3640 1 --2- 172.21.15.43:0/1250874764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616f760d360 0x5616f7633370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.385+0000 7fe86e8b3640 1 -- 172.21.15.43:0/1250874764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616f71fb670 con 0x5616f7615140 2024-04-25T08:57:21.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.385+0000 7fe86d8b1640 1 --2- 172.21.15.43:0/1250874764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616f760d360 0x5616f7633370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:57:21.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.385+0000 7fe86e8b3640 1 --2- 172.21.15.43:0/1250874764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616f7615140 0x5616f7610b00 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fe85802a650 tx=0x7fe85802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe8d0be3c1ff014 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:21.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.385+0000 7fe84f7fe640 1 -- 172.21.15.43:0/1250874764 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe858036040 con 0x5616f7615140 2024-04-25T08:57:21.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.385+0000 7fe84f7fe640 1 -- 172.21.15.43:0/1250874764 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe858037040 con 0x5616f7615140 2024-04-25T08:57:21.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.385+0000 7fe84f7fe640 1 -- 172.21.15.43:0/1250874764 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe858031b40 con 0x5616f7615140 2024-04-25T08:57:21.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.386+0000 7fe873017740 1 -- 172.21.15.43:0/1250874764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616f7615140 msgr2=0x5616f7610b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.386+0000 7fe873017740 1 --2- 172.21.15.43:0/1250874764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616f7615140 0x5616f7610b00 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fe85802a650 tx=0x7fe85802e000 comp rx=0 tx=0).stop 2024-04-25T08:57:21.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.386+0000 7fe873017740 1 -- 172.21.15.43:0/1250874764 shutdown_connections 2024-04-25T08:57:21.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.386+0000 7fe873017740 1 --2- 172.21.15.43:0/1250874764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616f7615140 0x5616f7610b00 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.386+0000 7fe873017740 1 --2- 172.21.15.43:0/1250874764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616f760d360 0x5616f7633370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.386+0000 7fe873017740 1 --2- 172.21.15.43:0/1250874764 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5616f75ec360 0x5616f75e0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.386+0000 7fe873017740 1 -- 172.21.15.43:0/1250874764 >> 172.21.15.43:0/1250874764 conn(0x5616f75b1810 msgr2=0x5616f7617220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:21.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.386+0000 7fe873017740 1 -- 172.21.15.43:0/1250874764 shutdown_connections 2024-04-25T08:57:21.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.386+0000 7fe873017740 1 -- 172.21.15.43:0/1250874764 wait complete. 2024-04-25T08:57:21.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.386+0000 7fe873017740 1 -- start start 2024-04-25T08:57:21.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.386+0000 7fe873017740 1 Processor -- start 2024-04-25T08:57:21.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.387+0000 7fe873017740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616f75ec360 0x5616f7625590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.387+0000 7fe873017740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5616f760d360 0x5616f7625ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.387+0000 7fe873017740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616f7615140 0x5616f762c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.387+0000 7fe873017740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8681f44f0 con 0x5616f75ec360 2024-04-25T08:57:21.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.387+0000 7fe873017740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616f75e6be0 con 0x5616f7615140 2024-04-25T08:57:21.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.387+0000 7fe873017740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x5616f762d810 con 0x5616f760d360 2024-04-25T08:57:21.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.387+0000 7fe873017740 10 client.0 _ll_register_callbacks cb 0x7fe858034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:21.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.387+0000 7fe86e0b2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616f75ec360 0x5616f7625590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.387+0000 7fe86e0b2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616f75ec360 0x5616f7625590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42150/0 (socket says 172.21.15.43:42150) 2024-04-25T08:57:21.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.387+0000 7fe86e0b2640 1 -- 172.21.15.43:0/3316196257 learned_addr learned my addr 172.21.15.43:0/3316196257 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:21.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.387+0000 7fe86e8b3640 1 --2- 172.21.15.43:0/3316196257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616f7615140 0x5616f762c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.387+0000 7fe86d8b1640 1 --2- 172.21.15.43:0/3316196257 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5616f760d360 0x5616f7625ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.387+0000 7fe86e0b2640 1 -- 172.21.15.43:0/3316196257 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5616f760d360 msgr2=0x5616f7625ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:21.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.388+0000 7fe86e0b2640 1 --2- 172.21.15.43:0/3316196257 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5616f760d360 0x5616f7625ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.388+0000 7fe86e0b2640 1 -- 172.21.15.43:0/3316196257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616f7615140 msgr2=0x5616f762c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:21.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.388+0000 7fe86e0b2640 1 --2- 172.21.15.43:0/3316196257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616f7615140 0x5616f762c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.388+0000 7fe86e0b2640 1 -- 172.21.15.43:0/3316196257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5616f7628cd0 con 0x5616f75ec360 2024-04-25T08:57:21.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.388+0000 7fe86e8b3640 1 --2- 172.21.15.43:0/3316196257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616f7615140 0x5616f762c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:21.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.388+0000 7fe86e0b2640 1 --2- 172.21.15.43:0/3316196257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616f75ec360 0x5616f7625590 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fe8600562c0 tx=0x7fe86001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90015cfdd2fd3aae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:21.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.388+0000 7fe84d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:57:21.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.388+0000 7fe84d7fa640 1 -- 172.21.15.43:0/3316196257 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86007a070 con 0x5616f75ec360 2024-04-25T08:57:21.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.388+0000 7fe84d7fa640 1 -- 172.21.15.43:0/3316196257 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe8600571c0 con 0x5616f75ec360 2024-04-25T08:57:21.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.388+0000 7fe873017740 1 -- 172.21.15.43:0/3316196257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe858037040 con 0x5616f75ec360 2024-04-25T08:57:21.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.388+0000 7fe84d7fa640 1 -- 172.21.15.43:0/3316196257 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe860054010 con 0x5616f75ec360 2024-04-25T08:57:21.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.388+0000 7fe873017740 10 client.6021 fetch_fsmap learned FSMap version 21 2024-04-25T08:57:21.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.388+0000 7fe873017740 1 -- 172.21.15.43:0/3316196257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x5616f7628cd0 con 0x5616f75ec360 2024-04-25T08:57:21.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.388+0000 7fe84d7fa640 1 -- 172.21.15.43:0/3316196257 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe858037040 con 0x5616f75ec360 2024-04-25T08:57:21.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.388+0000 7fe84d7fa640 1 -- 172.21.15.43:0/3316196257 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe860081020 con 0x5616f75ec360 2024-04-25T08:57:21.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.388+0000 7fe873017740 10 client.6021 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T08:57:21.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.388+0000 7fe873017740 10 client.6021 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T08:57:21.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.388+0000 7fe873017740 20 client.6021 populate_metadata read hostname 'smithi043' 2024-04-25T08:57:21.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.388+0000 7fe873017740 1 --2- 172.21.15.43:0/3316196257 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5616f767f910 0x5616f762cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.388+0000 7fe873017740 4 client.6021 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_0"}] 2024-04-25T08:57:21.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.388+0000 7fe873017740 1 -- 172.21.15.43:0/3316196257 --> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_0"}) v1 -- 0x5616f7555770 con 0x5616f767f910 2024-04-25T08:57:21.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.388+0000 7fe86d8b1640 1 --2- 172.21.15.43:0/3316196257 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5616f767f910 0x5616f762cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.389+0000 7fe86d8b1640 1 --2- 172.21.15.43:0/3316196257 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5616f767f910 0x5616f762cc40 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c12429cf55df6d4 server_cookie=64279c1938559934 in_seq=0 out_seq=0 2024-04-25T08:57:21.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.389+0000 7fe84d7fa640 10 client.6021 ms_handle_connect on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:21.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.389+0000 7fe84d7fa640 1 -- 172.21.15.43:0/3316196257 <== mds.0 v2:172.21.15.193:6836/4164634380 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5616f7555770 con 0x5616f767f910 2024-04-25T08:57:21.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.389+0000 7fe84d7fa640 10 client.6021 handle_command_reply: tid=0 2024-04-25T08:57:21.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.389+0000 7fe873017740 1 client.6021 shutdown 2024-04-25T08:57:21.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.389+0000 7fe873017740 1 -- 172.21.15.43:0/3316196257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616f75ec360 msgr2=0x5616f7625590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.389+0000 7fe873017740 1 --2- 172.21.15.43:0/3316196257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616f75ec360 0x5616f7625590 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fe8600562c0 tx=0x7fe86001d000 comp rx=0 tx=0).stop 2024-04-25T08:57:21.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.390+0000 7fe873017740 1 -- 172.21.15.43:0/3316196257 shutdown_connections 2024-04-25T08:57:21.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.390+0000 7fe873017740 1 --2- 172.21.15.43:0/3316196257 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5616f767f910 0x5616f762cc40 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.390+0000 7fe873017740 1 --2- 172.21.15.43:0/3316196257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5616f7615140 0x5616f762c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.390+0000 7fe873017740 1 --2- 172.21.15.43:0/3316196257 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5616f760d360 0x5616f7625ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.390+0000 7fe84d7fa640 0 client.6021 ms_handle_reset on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:21.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.390+0000 7fe873017740 1 --2- 172.21.15.43:0/3316196257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5616f75ec360 0x5616f7625590 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.390+0000 7fe873017740 1 -- 172.21.15.43:0/3316196257 >> 172.21.15.43:0/3316196257 conn(0x5616f75b1810 msgr2=0x5616f7617040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:21.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.390+0000 7fe873017740 1 -- 172.21.15.43:0/3316196257 shutdown_connections 2024-04-25T08:57:21.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.390+0000 7fe873017740 1 -- 172.21.15.43:0/3316196257 wait complete. 2024-04-25T08:57:21.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.390+0000 7fe873017740 20 client.6021 trim_cache size 0 max 16384 2024-04-25T08:57:21.386 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T08:57:21.386 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-25T08:57:21.386 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "0807b0e7-b681-432e-993f-ad04fc5ced81", 2024-04-25T08:57:21.386 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-25T08:57:21.386 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T08:57:21.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.390+0000 7fe870316640 1 -- 172.21.15.43:0/3582652030 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe85406b4d0 msgr2=0x7fe85406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.390+0000 7fe870316640 1 --2- 172.21.15.43:0/3582652030 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe85406b4d0 0x7fe85406d990 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe8680cd780 tx=0x7fe85c009190 comp rx=0 tx=0).stop 2024-04-25T08:57:21.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.390+0000 7fe870316640 1 -- 172.21.15.43:0/3582652030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8680b6110 msgr2=0x7fe86806d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.390+0000 7fe870316640 1 --2- 172.21.15.43:0/3582652030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8680b6110 0x7fe86806d8d0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fe86000c430 tx=0x7fe86000c8f0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.390+0000 7fe870316640 1 -- 172.21.15.43:0/3582652030 shutdown_connections 2024-04-25T08:57:21.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.390+0000 7fe870316640 1 --2- 172.21.15.43:0/3582652030 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe8680d2340 0x7fe8681f76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.390+0000 7fe870316640 1 --2- 172.21.15.43:0/3582652030 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe85406b4d0 0x7fe85406d990 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.390+0000 7fe870316640 1 --2- 172.21.15.43:0/3582652030 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8680d1a20 0x7fe86806c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.390+0000 7fe870316640 1 --2- 172.21.15.43:0/3582652030 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8680b6110 0x7fe86806d8d0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.390+0000 7fe870316640 1 -- 172.21.15.43:0/3582652030 >> 172.21.15.43:0/3582652030 conn(0x7fe868072aa0 msgr2=0x7fe86815b260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:21.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.390+0000 7fe870316640 1 -- 172.21.15.43:0/3582652030 shutdown_connections 2024-04-25T08:57:21.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.390+0000 7fe870316640 1 -- 172.21.15.43:0/3582652030 wait complete. 2024-04-25T08:57:21.388 INFO:tasks.ceph.mds.d.smithi193.stderr:2024-04-25T08:57:21.390+0000 7f52a22c0640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000a4c [2,head] /scrub_test_path/ceph-qa-suite/new_file_0 auth v546 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5856=0-4194304@1} caps={5856=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@3},l=5856 | request=0 lock=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x556abacbd180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000a4c:[<0x10000000001/new_file_0 v546>,<0x10000000000/ceph-qa-suite v61>,<0x1/scrub_test_path v28>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":-61} 2024-04-25T08:57:21.390 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_0' returned '{'return_code': 0, 'scrub_tag': '0807b0e7-b681-432e-993f-ad04fc5ced81', 'mode': 'asynchronous'}' 2024-04-25T08:57:21.390 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T08:57:21.391 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T08:57:21.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.504+0000 7f221df0a640 1 Processor -- start 2024-04-25T08:57:21.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.504+0000 7f221df0a640 1 -- start start 2024-04-25T08:57:21.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.505+0000 7f221df0a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f221815dc40 0x7f22180ce5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.505+0000 7f221df0a640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f22180d1fa0 0x7f22180ceb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.505+0000 7f221df0a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f22180cf050 0x7f22180d03b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.505+0000 7f221df0a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22180d47f0 con 0x7f22180cf050 2024-04-25T08:57:21.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.505+0000 7f221df0a640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22180d24b0 con 0x7f22180d1fa0 2024-04-25T08:57:21.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.505+0000 7f221df0a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22180d2750 con 0x7f221815dc40 2024-04-25T08:57:21.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.505+0000 7f2217fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f22180cf050 0x7f22180d03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.505+0000 7f2217fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f22180cf050 0x7f22180d03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42166/0 (socket says 172.21.15.43:42166) 2024-04-25T08:57:21.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.505+0000 7f22177fe640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f221815dc40 0x7f22180ce5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.505+0000 7f2217fff640 1 -- 172.21.15.43:0/3311464121 learned_addr learned my addr 172.21.15.43:0/3311464121 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:21.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.505+0000 7f2216ffd640 1 --2- 172.21.15.43:0/3311464121 >> v2:172.21.15.193:3300/0 conn(0x7f22180d1fa0 0x7f22180ceb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.506+0000 7f2217fff640 1 -- 172.21.15.43:0/3311464121 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f221815dc40 msgr2=0x7f22180ce5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:21.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.506+0000 7f2217fff640 1 --2- 172.21.15.43:0/3311464121 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f221815dc40 0x7f22180ce5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.506+0000 7f2217fff640 1 -- 172.21.15.43:0/3311464121 >> v2:172.21.15.193:3300/0 conn(0x7f22180d1fa0 msgr2=0x7f22180ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:21.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.506+0000 7f2217fff640 1 --2- 172.21.15.43:0/3311464121 >> v2:172.21.15.193:3300/0 conn(0x7f22180d1fa0 0x7f22180ceb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.506+0000 7f2217fff640 1 -- 172.21.15.43:0/3311464121 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f221806b170 con 0x7f22180cf050 2024-04-25T08:57:21.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.506+0000 7f2217fff640 1 --2- 172.21.15.43:0/3311464121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22180cf050 0x7f22180d03b0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f221808d3c0 tx=0x7f221002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a96056e5425284e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:21.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.506+0000 7f22167fc640 1 -- 172.21.15.43:0/3311464121 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2210036070 con 0x7f22180cf050 2024-04-25T08:57:21.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.507+0000 7f22167fc640 1 -- 172.21.15.43:0/3311464121 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2210031600 con 0x7f22180cf050 2024-04-25T08:57:21.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.507+0000 7f22167fc640 1 -- 172.21.15.43:0/3311464121 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22100319d0 con 0x7f22180cf050 2024-04-25T08:57:21.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.507+0000 7f221df0a640 1 -- 172.21.15.43:0/3311464121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22180cf050 msgr2=0x7f22180d03b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.507+0000 7f221df0a640 1 --2- 172.21.15.43:0/3311464121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22180cf050 0x7f22180d03b0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f221808d3c0 tx=0x7f221002a610 comp rx=0 tx=0).stop 2024-04-25T08:57:21.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.507+0000 7f221df0a640 1 -- 172.21.15.43:0/3311464121 shutdown_connections 2024-04-25T08:57:21.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.507+0000 7f221df0a640 1 --2- 172.21.15.43:0/3311464121 >> v2:172.21.15.193:3300/0 conn(0x7f22180d1fa0 0x7f22180ceb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.507+0000 7f221df0a640 1 --2- 172.21.15.43:0/3311464121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22180cf050 0x7f22180d03b0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.507+0000 7f221df0a640 1 --2- 172.21.15.43:0/3311464121 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f221815dc40 0x7f22180ce5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.507+0000 7f221df0a640 1 -- 172.21.15.43:0/3311464121 >> 172.21.15.43:0/3311464121 conn(0x7f22180d39c0 msgr2=0x7f22180d3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:21.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.507+0000 7f221df0a640 1 -- 172.21.15.43:0/3311464121 shutdown_connections 2024-04-25T08:57:21.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.507+0000 7f221df0a640 1 -- 172.21.15.43:0/3311464121 wait complete. 2024-04-25T08:57:21.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.508+0000 7f221df0a640 1 Processor -- start 2024-04-25T08:57:21.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.508+0000 7f221df0a640 1 -- start start 2024-04-25T08:57:21.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.509+0000 7f221df0a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22180cf050 0x7f221815c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.509+0000 7f221df0a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f22180d1fa0 0x7f22180cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.509+0000 7f221df0a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f221815dc40 0x7f22181e61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.509+0000 7f221df0a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22180d2a30 con 0x7f221815dc40 2024-04-25T08:57:21.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.509+0000 7f221df0a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22180cf4a0 con 0x7f22180cf050 2024-04-25T08:57:21.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.509+0000 7f221df0a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f221815e020 con 0x7f22180d1fa0 2024-04-25T08:57:21.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.509+0000 7f2217fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f221815dc40 0x7f22181e61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.509+0000 7f2217fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f221815dc40 0x7f22181e61f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42168/0 (socket says 172.21.15.43:42168) 2024-04-25T08:57:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.509+0000 7f2217fff640 1 -- 172.21.15.43:0/3380755277 learned_addr learned my addr 172.21.15.43:0/3380755277 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.509+0000 7f22177fe640 1 --2- 172.21.15.43:0/3380755277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22180cf050 0x7f221815c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.509+0000 7f2216ffd640 1 --2- 172.21.15.43:0/3380755277 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f22180d1fa0 0x7f22180cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.510+0000 7f2217fff640 1 -- 172.21.15.43:0/3380755277 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f22180d1fa0 msgr2=0x7f22180cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.510+0000 7f2217fff640 1 --2- 172.21.15.43:0/3380755277 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f22180d1fa0 0x7f22180cc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.510+0000 7f2217fff640 1 -- 172.21.15.43:0/3380755277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22180cf050 msgr2=0x7f221815c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.510+0000 7f2217fff640 1 --2- 172.21.15.43:0/3380755277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22180cf050 0x7f221815c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.510+0000 7f2217fff640 1 -- 172.21.15.43:0/3380755277 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22180ccf70 con 0x7f221815dc40 2024-04-25T08:57:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.510+0000 7f22177fe640 1 --2- 172.21.15.43:0/3380755277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22180cf050 0x7f221815c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.510+0000 7f2217fff640 1 --2- 172.21.15.43:0/3380755277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f221815dc40 0x7f22181e61f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f2210034040 tx=0x7f2210004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.510+0000 7f21fffff640 1 -- 172.21.15.43:0/3380755277 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2210036050 con 0x7f221815dc40 2024-04-25T08:57:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.510+0000 7f21fffff640 1 -- 172.21.15.43:0/3380755277 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2210042070 con 0x7f221815dc40 2024-04-25T08:57:21.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.510+0000 7f221df0a640 1 -- 172.21.15.43:0/3380755277 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22180cd250 con 0x7f221815dc40 2024-04-25T08:57:21.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.510+0000 7f21fffff640 1 -- 172.21.15.43:0/3380755277 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2210009510 con 0x7f221815dc40 2024-04-25T08:57:21.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.510+0000 7f221df0a640 1 -- 172.21.15.43:0/3380755277 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22180b7680 con 0x7f221815dc40 2024-04-25T08:57:21.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.512+0000 7f21fffff640 1 -- 172.21.15.43:0/3380755277 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2210007bf0 con 0x7f221815dc40 2024-04-25T08:57:21.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.512+0000 7f21fffff640 1 --2- 172.21.15.43:0/3380755277 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f220406b4d0 0x7f220406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.512+0000 7f21fffff640 1 -- 172.21.15.43:0/3380755277 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f22100ab000 con 0x7f221815dc40 2024-04-25T08:57:21.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.512+0000 7f22177fe640 1 --2- 172.21.15.43:0/3380755277 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f220406b4d0 0x7f220406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.513+0000 7f22177fe640 1 --2- 172.21.15.43:0/3380755277 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f220406b4d0 0x7f220406d990 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2208009a70 tx=0x7f2208008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:21.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.513+0000 7f2220c0a740 1 Processor -- start 2024-04-25T08:57:21.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.513+0000 7f2220c0a740 1 -- start start 2024-04-25T08:57:21.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.514+0000 7f2220c0a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5579239f0b10 0x5579239f3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.514+0000 7f2220c0a740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5579239f0ef0 0x5579239f3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.514+0000 7f2220c0a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579239c3770 0x5579239c6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.514+0000 7f2220c0a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55792392cda0 con 0x5579239c3770 2024-04-25T08:57:21.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.514+0000 7f2220c0a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579238d5b80 con 0x5579239f0b10 2024-04-25T08:57:21.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.514+0000 7f2220c0a740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x557923928e40 con 0x5579239f0ef0 2024-04-25T08:57:21.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.514+0000 7f2217fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579239c3770 0x5579239c6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.514+0000 7f22177fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5579239f0b10 0x5579239f3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.514+0000 7f2217fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579239c3770 0x5579239c6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42172/0 (socket says 172.21.15.43:42172) 2024-04-25T08:57:21.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.514+0000 7f22177fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5579239f0b10 0x5579239f3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:44812/0 (socket says 172.21.15.43:44812) 2024-04-25T08:57:21.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.514+0000 7f2217fff640 1 -- 172.21.15.43:0/966490960 learned_addr learned my addr 172.21.15.43:0/966490960 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:21.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.514+0000 7f2216ffd640 1 --2- 172.21.15.43:0/966490960 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5579239f0ef0 0x5579239f3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.514+0000 7f2217fff640 1 -- 172.21.15.43:0/966490960 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5579239f0ef0 msgr2=0x5579239f3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:21.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.514+0000 7f2217fff640 1 --2- 172.21.15.43:0/966490960 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5579239f0ef0 0x5579239f3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.514+0000 7f2217fff640 1 -- 172.21.15.43:0/966490960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5579239f0b10 msgr2=0x5579239f3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:21.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.514+0000 7f2217fff640 1 --2- 172.21.15.43:0/966490960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5579239f0b10 0x5579239f3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.514+0000 7f2217fff640 1 -- 172.21.15.43:0/966490960 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579238dff10 con 0x5579239c3770 2024-04-25T08:57:21.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.514+0000 7f22177fe640 1 --2- 172.21.15.43:0/966490960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5579239f0b10 0x5579239f3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:57:21.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.514+0000 7f2217fff640 1 --2- 172.21.15.43:0/966490960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579239c3770 0x5579239c6f80 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f2210079830 tx=0x7f2210009c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e571dc7d96c5496 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:21.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.514+0000 7f21fd7fa640 1 -- 172.21.15.43:0/966490960 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2210091070 con 0x5579239c3770 2024-04-25T08:57:21.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.514+0000 7f21fd7fa640 1 -- 172.21.15.43:0/966490960 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2210037040 con 0x5579239c3770 2024-04-25T08:57:21.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.514+0000 7f21fd7fa640 1 -- 172.21.15.43:0/966490960 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2210031600 con 0x5579239c3770 2024-04-25T08:57:21.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.515+0000 7f2220c0a740 1 -- 172.21.15.43:0/966490960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579239c3770 msgr2=0x5579239c6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.515+0000 7f2220c0a740 1 --2- 172.21.15.43:0/966490960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579239c3770 0x5579239c6f80 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f2210079830 tx=0x7f2210009c50 comp rx=0 tx=0).stop 2024-04-25T08:57:21.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.515+0000 7f2220c0a740 1 -- 172.21.15.43:0/966490960 shutdown_connections 2024-04-25T08:57:21.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.515+0000 7f2220c0a740 1 --2- 172.21.15.43:0/966490960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579239c3770 0x5579239c6f80 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.515+0000 7f2220c0a740 1 --2- 172.21.15.43:0/966490960 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5579239f0ef0 0x5579239f3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.515+0000 7f2220c0a740 1 --2- 172.21.15.43:0/966490960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5579239f0b10 0x5579239f3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.515+0000 7f2220c0a740 1 -- 172.21.15.43:0/966490960 >> 172.21.15.43:0/966490960 conn(0x5579239c59e0 msgr2=0x557923a0e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:21.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.515+0000 7f2220c0a740 1 -- 172.21.15.43:0/966490960 shutdown_connections 2024-04-25T08:57:21.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.515+0000 7f2220c0a740 1 -- 172.21.15.43:0/966490960 wait complete. 2024-04-25T08:57:21.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.516+0000 7f2220c0a740 1 -- start start 2024-04-25T08:57:21.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.516+0000 7f2220c0a740 1 Processor -- start 2024-04-25T08:57:21.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.517+0000 7f2220c0a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5579239f0b10 0x5579239e7690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.517+0000 7f2220c0a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579239f0ef0 0x5579239e90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.517+0000 7f2220c0a740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5579239e9b00 0x5579239e9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.517+0000 7f2220c0a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579238d5b80 con 0x5579239f0ef0 2024-04-25T08:57:21.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.517+0000 7f2220c0a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5579238cb4b0 con 0x5579239f0b10 2024-04-25T08:57:21.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.517+0000 7f2220c0a740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55792392cda0 con 0x5579239e9b00 2024-04-25T08:57:21.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.517+0000 7f2220c0a740 10 client.0 _ll_register_callbacks cb 0x55792390b180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:21.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.517+0000 7f2216ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579239f0ef0 0x5579239e90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.517+0000 7f2216ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579239f0ef0 0x5579239e90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42184/0 (socket says 172.21.15.43:42184) 2024-04-25T08:57:21.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.517+0000 7f2216ffd640 1 -- 172.21.15.43:0/1542271928 learned_addr learned my addr 172.21.15.43:0/1542271928 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:21.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.517+0000 7f22177fe640 1 --2- 172.21.15.43:0/1542271928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5579239f0b10 0x5579239e7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.517+0000 7f2217fff640 1 --2- 172.21.15.43:0/1542271928 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5579239e9b00 0x5579239e9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.517+0000 7f2216ffd640 1 -- 172.21.15.43:0/1542271928 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5579239e9b00 msgr2=0x5579239e9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:21.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.517+0000 7f2216ffd640 1 --2- 172.21.15.43:0/1542271928 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5579239e9b00 0x5579239e9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.517+0000 7f2216ffd640 1 -- 172.21.15.43:0/1542271928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5579239f0b10 msgr2=0x5579239e7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:21.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.517+0000 7f2216ffd640 1 --2- 172.21.15.43:0/1542271928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5579239f0b10 0x5579239e7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.517+0000 7f2216ffd640 1 -- 172.21.15.43:0/1542271928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55792370b7d0 con 0x5579239f0ef0 2024-04-25T08:57:21.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.517+0000 7f2216ffd640 1 --2- 172.21.15.43:0/1542271928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579239f0ef0 0x5579239e90a0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f220c0147a0 tx=0x7f220c014c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de587a088c09562a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:21.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.517+0000 7f21f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:57:21.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.517+0000 7f21f67fc640 1 -- 172.21.15.43:0/1542271928 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220c012ac0 con 0x5579239f0ef0 2024-04-25T08:57:21.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.517+0000 7f21f67fc640 1 -- 172.21.15.43:0/1542271928 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f220c012c60 con 0x5579239f0ef0 2024-04-25T08:57:21.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.517+0000 7f2220c0a740 1 -- 172.21.15.43:0/1542271928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2210037040 con 0x5579239f0ef0 2024-04-25T08:57:21.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.518+0000 7f21f67fc640 1 -- 172.21.15.43:0/1542271928 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f220c012e60 con 0x5579239f0ef0 2024-04-25T08:57:21.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.518+0000 7f2220c0a740 10 client.6033 fetch_fsmap learned FSMap version 21 2024-04-25T08:57:21.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.518+0000 7f2220c0a740 1 -- 172.21.15.43:0/1542271928 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x5579236413f0 con 0x5579239f0ef0 2024-04-25T08:57:21.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.518+0000 7f21f67fc640 1 -- 172.21.15.43:0/1542271928 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2210037040 con 0x5579239f0ef0 2024-04-25T08:57:21.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.518+0000 7f21f67fc640 1 -- 172.21.15.43:0/1542271928 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f220c02a080 con 0x5579239f0ef0 2024-04-25T08:57:21.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.518+0000 7f2220c0a740 10 client.6033 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T08:57:21.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.518+0000 7f2220c0a740 10 client.6033 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T08:57:21.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.518+0000 7f2220c0a740 20 client.6033 populate_metadata read hostname 'smithi043' 2024-04-25T08:57:21.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.518+0000 7f2220c0a740 1 --2- 172.21.15.43:0/1542271928 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x557923a11990 0x557923a11db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.518+0000 7f2220c0a740 4 client.6033 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:57:21.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.518+0000 7f2220c0a740 1 -- 172.21.15.43:0/1542271928 --> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55792393a2e0 con 0x557923a11990 2024-04-25T08:57:21.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.518+0000 7f22177fe640 1 --2- 172.21.15.43:0/1542271928 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x557923a11990 0x557923a11db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.518+0000 7f22177fe640 1 --2- 172.21.15.43:0/1542271928 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x557923a11990 0x557923a11db0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c64b30ad4ba7f4e5 server_cookie=35cbba434d351c7e in_seq=0 out_seq=0 2024-04-25T08:57:21.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.518+0000 7f21f67fc640 10 client.6033 ms_handle_connect on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:21.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.519+0000 7f21f67fc640 1 -- 172.21.15.43:0/1542271928 <== mds.0 v2:172.21.15.193:6836/4164634380 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55792393a2e0 con 0x557923a11990 2024-04-25T08:57:21.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.519+0000 7f21f67fc640 10 client.6033 handle_command_reply: tid=0 2024-04-25T08:57:21.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.519+0000 7f2220c0a740 1 client.6033 shutdown 2024-04-25T08:57:21.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.519+0000 7f2220c0a740 1 -- 172.21.15.43:0/1542271928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579239f0ef0 msgr2=0x5579239e90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.519+0000 7f2220c0a740 1 --2- 172.21.15.43:0/1542271928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579239f0ef0 0x5579239e90a0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f220c0147a0 tx=0x7f220c014c60 comp rx=0 tx=0).stop 2024-04-25T08:57:21.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.519+0000 7f2220c0a740 1 -- 172.21.15.43:0/1542271928 shutdown_connections 2024-04-25T08:57:21.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.519+0000 7f2220c0a740 1 --2- 172.21.15.43:0/1542271928 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5579239e9b00 0x5579239e9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.519+0000 7f2220c0a740 1 --2- 172.21.15.43:0/1542271928 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579239f0ef0 0x5579239e90a0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.519+0000 7f2220c0a740 1 --2- 172.21.15.43:0/1542271928 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x557923a11990 0x557923a11db0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.519+0000 7f2220c0a740 1 --2- 172.21.15.43:0/1542271928 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5579239f0b10 0x5579239e7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.519+0000 7f21f67fc640 0 client.6033 ms_handle_reset on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:21.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.519+0000 7f2220c0a740 1 -- 172.21.15.43:0/1542271928 >> 172.21.15.43:0/1542271928 conn(0x5579239c3770 msgr2=0x557923a0daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:21.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.519+0000 7f2220c0a740 1 -- 172.21.15.43:0/1542271928 shutdown_connections 2024-04-25T08:57:21.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.519+0000 7f2220c0a740 1 -- 172.21.15.43:0/1542271928 wait complete. 2024-04-25T08:57:21.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.519+0000 7f2220c0a740 20 client.6033 trim_cache size 0 max 16384 2024-04-25T08:57:21.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.534+0000 7f2220c0a740 1 Processor -- start 2024-04-25T08:57:21.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.535+0000 7f2220c0a740 1 -- start start 2024-04-25T08:57:21.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.535+0000 7f2220c0a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5579239e9b00 0x557923a0e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.535+0000 7f2220c0a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579239f0b10 0x5579239f6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.535+0000 7f2220c0a740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5579239f0ef0 0x5579239f7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.535+0000 7f2220c0a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22181fffc0 con 0x5579239f0b10 2024-04-25T08:57:21.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.535+0000 7f2220c0a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22180b8960 con 0x5579239e9b00 2024-04-25T08:57:21.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.535+0000 7f2220c0a740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22180b7d60 con 0x5579239f0ef0 2024-04-25T08:57:21.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.535+0000 7f2216ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579239f0b10 0x5579239f6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.535+0000 7f2216ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579239f0b10 0x5579239f6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42188/0 (socket says 172.21.15.43:42188) 2024-04-25T08:57:21.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.535+0000 7f2216ffd640 1 -- 172.21.15.43:0/1625938348 learned_addr learned my addr 172.21.15.43:0/1625938348 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:21.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.536+0000 7f2216ffd640 1 -- 172.21.15.43:0/1625938348 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5579239f0ef0 msgr2=0x5579239f7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:21.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.536+0000 7f2216ffd640 1 --2- 172.21.15.43:0/1625938348 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5579239f0ef0 0x5579239f7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.536+0000 7f2216ffd640 1 -- 172.21.15.43:0/1625938348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5579239e9b00 msgr2=0x557923a0e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:21.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.536+0000 7f2216ffd640 1 --2- 172.21.15.43:0/1625938348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5579239e9b00 0x557923a0e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.536+0000 7f2216ffd640 1 -- 172.21.15.43:0/1625938348 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5579236010f0 con 0x5579239f0b10 2024-04-25T08:57:21.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.536+0000 7f2216ffd640 1 --2- 172.21.15.43:0/1625938348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579239f0b10 0x5579239f6e50 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f220c012730 tx=0x7f220c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f18ae584e59d3158 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:21.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.536+0000 7f21fcff9640 1 -- 172.21.15.43:0/1625938348 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220c012cc0 con 0x5579239f0b10 2024-04-25T08:57:21.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.536+0000 7f21fcff9640 1 -- 172.21.15.43:0/1625938348 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f220c02d070 con 0x5579239f0b10 2024-04-25T08:57:21.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.536+0000 7f21fcff9640 1 -- 172.21.15.43:0/1625938348 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220c014290 con 0x5579239f0b10 2024-04-25T08:57:21.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.537+0000 7f2220c0a740 1 -- 172.21.15.43:0/1625938348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579239f0b10 msgr2=0x5579239f6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.537+0000 7f2220c0a740 1 --2- 172.21.15.43:0/1625938348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579239f0b10 0x5579239f6e50 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f220c012730 tx=0x7f220c017000 comp rx=0 tx=0).stop 2024-04-25T08:57:21.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.537+0000 7f2220c0a740 1 -- 172.21.15.43:0/1625938348 shutdown_connections 2024-04-25T08:57:21.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.537+0000 7f2220c0a740 1 --2- 172.21.15.43:0/1625938348 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5579239f0ef0 0x5579239f7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.537+0000 7f2220c0a740 1 --2- 172.21.15.43:0/1625938348 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579239f0b10 0x5579239f6e50 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.537+0000 7f2220c0a740 1 --2- 172.21.15.43:0/1625938348 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5579239e9b00 0x557923a0e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.537+0000 7f2220c0a740 1 -- 172.21.15.43:0/1625938348 >> 172.21.15.43:0/1625938348 conn(0x5579239c3770 msgr2=0x5579239bee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:21.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.537+0000 7f2220c0a740 1 -- 172.21.15.43:0/1625938348 shutdown_connections 2024-04-25T08:57:21.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.537+0000 7f2220c0a740 1 -- 172.21.15.43:0/1625938348 wait complete. 2024-04-25T08:57:21.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.537+0000 7f2220c0a740 1 -- start start 2024-04-25T08:57:21.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.537+0000 7f2220c0a740 1 Processor -- start 2024-04-25T08:57:21.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.538+0000 7f2220c0a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5579239e9b00 0x557923a78a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.538+0000 7f2220c0a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579239f0b10 0x557923a7ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.538+0000 7f2220c0a740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5579239f0ef0 0x5579239e4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.538+0000 7f2220c0a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55792392f2b0 con 0x5579239f0b10 2024-04-25T08:57:21.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.538+0000 7f2220c0a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22180b8360 con 0x5579239e9b00 2024-04-25T08:57:21.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.538+0000 7f2220c0a740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x557923a0fe20 con 0x5579239f0ef0 2024-04-25T08:57:21.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.538+0000 7f2220c0a740 10 client.0 _ll_register_callbacks cb 0x7f22100092e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:21.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.538+0000 7f2216ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579239f0b10 0x557923a7ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.538+0000 7f2216ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579239f0b10 0x557923a7ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42190/0 (socket says 172.21.15.43:42190) 2024-04-25T08:57:21.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.538+0000 7f2216ffd640 1 -- 172.21.15.43:0/92307638 learned_addr learned my addr 172.21.15.43:0/92307638 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:21.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.538+0000 7f2216ffd640 1 -- 172.21.15.43:0/92307638 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5579239f0ef0 msgr2=0x5579239e4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:21.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.538+0000 7f2216ffd640 1 --2- 172.21.15.43:0/92307638 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5579239f0ef0 0x5579239e4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.538+0000 7f2216ffd640 1 -- 172.21.15.43:0/92307638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5579239e9b00 msgr2=0x557923a78a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:21.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.538+0000 7f2216ffd640 1 --2- 172.21.15.43:0/92307638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5579239e9b00 0x557923a78a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.538+0000 7f2216ffd640 1 -- 172.21.15.43:0/92307638 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557923a10c70 con 0x5579239f0b10 2024-04-25T08:57:21.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.539+0000 7f2216ffd640 1 --2- 172.21.15.43:0/92307638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579239f0b10 0x557923a7ca50 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f220c009520 tx=0x7f220c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f81d96bfdcb5a424 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:21.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.539+0000 7f21f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:57:21.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.539+0000 7f21f67fc640 1 -- 172.21.15.43:0/92307638 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220c037060 con 0x5579239f0b10 2024-04-25T08:57:21.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.539+0000 7f21f67fc640 1 -- 172.21.15.43:0/92307638 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f220c004230 con 0x5579239f0b10 2024-04-25T08:57:21.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.539+0000 7f2220c0a740 1 -- 172.21.15.43:0/92307638 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f220c02d070 con 0x5579239f0b10 2024-04-25T08:57:21.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.539+0000 7f21f67fc640 1 -- 172.21.15.43:0/92307638 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f220c014320 con 0x5579239f0b10 2024-04-25T08:57:21.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.539+0000 7f2220c0a740 10 client.6039 fetch_fsmap learned FSMap version 21 2024-04-25T08:57:21.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.539+0000 7f2220c0a740 1 -- 172.21.15.43:0/92307638 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=21,monmap=2+}) v3 -- 0x557923a78fe0 con 0x5579239f0b10 2024-04-25T08:57:21.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.539+0000 7f21f67fc640 1 -- 172.21.15.43:0/92307638 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f220c02d070 con 0x5579239f0b10 2024-04-25T08:57:21.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.539+0000 7f21f67fc640 1 -- 172.21.15.43:0/92307638 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 21) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f220c02a070 con 0x5579239f0b10 2024-04-25T08:57:21.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.539+0000 7f2220c0a740 10 client.6039 fetch_fsmap finished waiting for FSMap version 21 2024-04-25T08:57:21.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.539+0000 7f2220c0a740 10 client.6039 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T08:57:21.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.539+0000 7f2220c0a740 20 client.6039 populate_metadata read hostname 'smithi043' 2024-04-25T08:57:21.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.539+0000 7f2220c0a740 1 --2- 172.21.15.43:0/92307638 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5579239e7af0 0x5579239e4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:21.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.539+0000 7f2220c0a740 4 client.6039 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:57:21.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.539+0000 7f2220c0a740 1 -- 172.21.15.43:0/92307638 --> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557923934630 con 0x5579239e7af0 2024-04-25T08:57:21.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.539+0000 7f22177fe640 1 --2- 172.21.15.43:0/92307638 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5579239e7af0 0x5579239e4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:21.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.540+0000 7f21f67fc640 10 client.6039 ms_handle_connect on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:21.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.540+0000 7f22177fe640 1 --2- 172.21.15.43:0/92307638 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5579239e7af0 0x5579239e4ad0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf86cea7d8ba8636 server_cookie=7a7467a9f6b3bafe in_seq=0 out_seq=0 2024-04-25T08:57:21.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.540+0000 7f21f67fc640 1 -- 172.21.15.43:0/92307638 <== mds.0 v2:172.21.15.193:6836/4164634380 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+903 (crc 0 0 0) 0x557923934630 con 0x5579239e7af0 2024-04-25T08:57:21.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.540+0000 7f21f67fc640 10 client.6039 handle_command_reply: tid=0 2024-04-25T08:57:21.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.540+0000 7f2220c0a740 1 client.6039 shutdown 2024-04-25T08:57:21.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.540+0000 7f2220c0a740 1 -- 172.21.15.43:0/92307638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579239f0b10 msgr2=0x557923a7ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.540+0000 7f2220c0a740 1 --2- 172.21.15.43:0/92307638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579239f0b10 0x557923a7ca50 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f220c009520 tx=0x7f220c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.540+0000 7f2220c0a740 1 -- 172.21.15.43:0/92307638 shutdown_connections 2024-04-25T08:57:21.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.540+0000 7f2220c0a740 1 --2- 172.21.15.43:0/92307638 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5579239f0ef0 0x5579239e4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.540+0000 7f2220c0a740 1 --2- 172.21.15.43:0/92307638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5579239f0b10 0x557923a7ca50 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.540+0000 7f2220c0a740 1 --2- 172.21.15.43:0/92307638 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5579239e7af0 0x5579239e4ad0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.540+0000 7f2220c0a740 1 --2- 172.21.15.43:0/92307638 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5579239e9b00 0x557923a78a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.540+0000 7f21f67fc640 0 client.6039 ms_handle_reset on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:21.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.540+0000 7f2220c0a740 1 -- 172.21.15.43:0/92307638 >> 172.21.15.43:0/92307638 conn(0x5579239c3770 msgr2=0x557923a0e510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:21.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.540+0000 7f2220c0a740 1 -- 172.21.15.43:0/92307638 shutdown_connections 2024-04-25T08:57:21.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.540+0000 7f2220c0a740 1 -- 172.21.15.43:0/92307638 wait complete. 2024-04-25T08:57:21.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.540+0000 7f2220c0a740 20 client.6039 trim_cache size 0 max 16384 2024-04-25T08:57:21.535 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T08:57:21.536 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T08:57:21.536 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-25T08:57:21.536 INFO:teuthology.orchestra.run.smithi043.stdout: "0807b0e7-b681-432e-993f-ad04fc5ced81": { 2024-04-25T08:57:21.536 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_0", 2024-04-25T08:57:21.536 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "0807b0e7-b681-432e-993f-ad04fc5ced81", 2024-04-25T08:57:21.536 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "" 2024-04-25T08:57:21.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:57:21.536 INFO:teuthology.orchestra.run.smithi043.stdout: "1a209b19-b19c-4251-854d-90d58d97192a": { 2024-04-25T08:57:21.536 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/scrub_test_path/ceph-qa-suite/suites", 2024-04-25T08:57:21.536 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "1a209b19-b19c-4251-854d-90d58d97192a", 2024-04-25T08:57:21.536 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "" 2024-04-25T08:57:21.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:57:21.536 INFO:teuthology.orchestra.run.smithi043.stdout: "399e9625-dd76-4969-bb4f-d3a666b763cb": { 2024-04-25T08:57:21.536 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-25T08:57:21.537 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "399e9625-dd76-4969-bb4f-d3a666b763cb", 2024-04-25T08:57:21.537 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "" 2024-04-25T08:57:21.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:57:21.537 INFO:teuthology.orchestra.run.smithi043.stdout: "d8defa35-f36f-4955-92b8-791ee432a719": { 2024-04-25T08:57:21.537 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml", 2024-04-25T08:57:21.537 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "d8defa35-f36f-4955-92b8-791ee432a719", 2024-04-25T08:57:21.537 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "" 2024-04-25T08:57:21.537 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:57:21.537 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:57:21.537 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T08:57:21.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.541+0000 7f221df0a640 1 -- 172.21.15.43:0/3380755277 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f220406b4d0 msgr2=0x7f220406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.541+0000 7f221df0a640 1 --2- 172.21.15.43:0/3380755277 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f220406b4d0 0x7f220406d990 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2208009a70 tx=0x7f2208008040 comp rx=0 tx=0).stop 2024-04-25T08:57:21.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.541+0000 7f221df0a640 1 -- 172.21.15.43:0/3380755277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f221815dc40 msgr2=0x7f22181e61f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:21.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.541+0000 7f221df0a640 1 --2- 172.21.15.43:0/3380755277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f221815dc40 0x7f22181e61f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f2210034040 tx=0x7f2210004a40 comp rx=0 tx=0).stop 2024-04-25T08:57:21.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.541+0000 7f221df0a640 1 -- 172.21.15.43:0/3380755277 shutdown_connections 2024-04-25T08:57:21.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.541+0000 7f221df0a640 1 --2- 172.21.15.43:0/3380755277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f221815dc40 0x7f22181e61f0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.541+0000 7f221df0a640 1 --2- 172.21.15.43:0/3380755277 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f22180d1fa0 0x7f22180cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.541+0000 7f221df0a640 1 --2- 172.21.15.43:0/3380755277 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f220406b4d0 0x7f220406d990 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.541+0000 7f221df0a640 1 --2- 172.21.15.43:0/3380755277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22180cf050 0x7f221815c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:21.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.541+0000 7f221df0a640 1 -- 172.21.15.43:0/3380755277 >> 172.21.15.43:0/3380755277 conn(0x7f22180d39c0 msgr2=0x7f22180b4790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:21.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.541+0000 7f221df0a640 1 -- 172.21.15.43:0/3380755277 shutdown_connections 2024-04-25T08:57:21.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:21.541+0000 7f221df0a640 1 -- 172.21.15.43:0/3380755277 wait complete. 2024-04-25T08:57:22.531 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-04-25T08:57:22.542+0000 7fe4fd99d640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T08:57:26.543 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T08:57:26.543 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T08:57:26.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.670+0000 7f9a2b438640 1 Processor -- start 2024-04-25T08:57:26.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.670+0000 7f9a2b438640 1 -- start start 2024-04-25T08:57:26.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.671+0000 7f9a2b438640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9a24161310 0x7f9a240d4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.671+0000 7f9a2b438640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9a240d0120 0x7f9a240d0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.671+0000 7f9a2b438640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9a240d0aa0 0x7f9a240d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.671+0000 7f9a2b438640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a240d4870 con 0x7f9a240d0aa0 2024-04-25T08:57:26.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.671+0000 7f9a2b438640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a240d0f30 con 0x7f9a24161310 2024-04-25T08:57:26.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.671+0000 7f9a2b438640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a240d45c0 con 0x7f9a240d0120 2024-04-25T08:57:26.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.671+0000 7f9a299d5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9a240d0aa0 0x7f9a240d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.671+0000 7f9a299d5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9a240d0aa0 0x7f9a240d19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42206/0 (socket says 172.21.15.43:42206) 2024-04-25T08:57:26.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.671+0000 7f9a299d5640 1 -- 172.21.15.43:0/3028137202 learned_addr learned my addr 172.21.15.43:0/3028137202 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:26.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.671+0000 7f9a291d4640 1 --2- 172.21.15.43:0/3028137202 >> v2:172.21.15.193:3300/0 conn(0x7f9a24161310 0x7f9a240d4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.671+0000 7f9a289d3640 1 --2- 172.21.15.43:0/3028137202 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9a240d0120 0x7f9a240d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.671+0000 7f9a299d5640 1 -- 172.21.15.43:0/3028137202 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9a240d0120 msgr2=0x7f9a240d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:26.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.671+0000 7f9a299d5640 1 --2- 172.21.15.43:0/3028137202 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9a240d0120 0x7f9a240d0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.671+0000 7f9a299d5640 1 -- 172.21.15.43:0/3028137202 >> v2:172.21.15.193:3300/0 conn(0x7f9a24161310 msgr2=0x7f9a240d4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:26.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.671+0000 7f9a299d5640 1 --2- 172.21.15.43:0/3028137202 >> v2:172.21.15.193:3300/0 conn(0x7f9a24161310 0x7f9a240d4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.671+0000 7f9a299d5640 1 -- 172.21.15.43:0/3028137202 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a240cf030 con 0x7f9a240d0aa0 2024-04-25T08:57:26.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.672+0000 7f9a299d5640 1 --2- 172.21.15.43:0/3028137202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a240d0aa0 0x7f9a240d19f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f9a2408d2f0 tx=0x7f9a1c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23cc3805d9b3aa1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:26.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.672+0000 7f9a1bfff640 1 -- 172.21.15.43:0/3028137202 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c036070 con 0x7f9a240d0aa0 2024-04-25T08:57:26.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.672+0000 7f9a1bfff640 1 -- 172.21.15.43:0/3028137202 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a1c031600 con 0x7f9a240d0aa0 2024-04-25T08:57:26.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.672+0000 7f9a1bfff640 1 -- 172.21.15.43:0/3028137202 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c0319d0 con 0x7f9a240d0aa0 2024-04-25T08:57:26.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.673+0000 7f9a2b438640 1 -- 172.21.15.43:0/3028137202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a240d0aa0 msgr2=0x7f9a240d19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:26.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.673+0000 7f9a2b438640 1 --2- 172.21.15.43:0/3028137202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a240d0aa0 0x7f9a240d19f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f9a2408d2f0 tx=0x7f9a1c02a6a0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.673+0000 7f9a2b438640 1 -- 172.21.15.43:0/3028137202 shutdown_connections 2024-04-25T08:57:26.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.673+0000 7f9a2b438640 1 --2- 172.21.15.43:0/3028137202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a240d0aa0 0x7f9a240d19f0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.673+0000 7f9a2b438640 1 --2- 172.21.15.43:0/3028137202 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9a240d0120 0x7f9a240d0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.673+0000 7f9a2b438640 1 --2- 172.21.15.43:0/3028137202 >> v2:172.21.15.193:3300/0 conn(0x7f9a24161310 0x7f9a240d4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.673+0000 7f9a2b438640 1 -- 172.21.15.43:0/3028137202 >> 172.21.15.43:0/3028137202 conn(0x7f9a2409cd20 msgr2=0x7f9a240d3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:26.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.673+0000 7f9a2b438640 1 -- 172.21.15.43:0/3028137202 shutdown_connections 2024-04-25T08:57:26.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.673+0000 7f9a2b438640 1 -- 172.21.15.43:0/3028137202 wait complete. 2024-04-25T08:57:26.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.674+0000 7f9a2b438640 1 Processor -- start 2024-04-25T08:57:26.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.674+0000 7f9a2b438640 1 -- start start 2024-04-25T08:57:26.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.675+0000 7f9a2b438640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a240d0120 0x7f9a2415a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.675+0000 7f9a2b438640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a240d0aa0 0x7f9a2415b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.675+0000 7f9a2b438640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9a24161310 0x7f9a2415bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.675+0000 7f9a2b438640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a241616f0 con 0x7f9a240d0120 2024-04-25T08:57:26.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.675+0000 7f9a2b438640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a240cf5f0 con 0x7f9a240d0aa0 2024-04-25T08:57:26.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.675+0000 7f9a2b438640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a240d1f30 con 0x7f9a24161310 2024-04-25T08:57:26.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.675+0000 7f9a291d4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a240d0120 0x7f9a2415a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.675+0000 7f9a291d4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a240d0120 0x7f9a2415a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42218/0 (socket says 172.21.15.43:42218) 2024-04-25T08:57:26.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.675+0000 7f9a291d4640 1 -- 172.21.15.43:0/2315612853 learned_addr learned my addr 172.21.15.43:0/2315612853 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:26.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.675+0000 7f9a289d3640 1 --2- 172.21.15.43:0/2315612853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a240d0aa0 0x7f9a2415b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.675+0000 7f9a299d5640 1 --2- 172.21.15.43:0/2315612853 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9a24161310 0x7f9a2415bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.675+0000 7f9a291d4640 1 -- 172.21.15.43:0/2315612853 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9a24161310 msgr2=0x7f9a2415bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:26.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.675+0000 7f9a291d4640 1 --2- 172.21.15.43:0/2315612853 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9a24161310 0x7f9a2415bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.675+0000 7f9a291d4640 1 -- 172.21.15.43:0/2315612853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a240d0aa0 msgr2=0x7f9a2415b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:26.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.675+0000 7f9a291d4640 1 --2- 172.21.15.43:0/2315612853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a240d0aa0 0x7f9a2415b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.675+0000 7f9a291d4640 1 -- 172.21.15.43:0/2315612853 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a2415a6a0 con 0x7f9a240d0120 2024-04-25T08:57:26.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.675+0000 7f9a289d3640 1 --2- 172.21.15.43:0/2315612853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a240d0aa0 0x7f9a2415b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:26.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.676+0000 7f9a291d4640 1 --2- 172.21.15.43:0/2315612853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a240d0120 0x7f9a2415a160 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f9a2000e8a0 tx=0x7f9a2000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:26.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.676+0000 7f9a19ffb640 1 -- 172.21.15.43:0/2315612853 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a20017070 con 0x7f9a240d0120 2024-04-25T08:57:26.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.676+0000 7f9a19ffb640 1 -- 172.21.15.43:0/2315612853 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a2000c940 con 0x7f9a240d0120 2024-04-25T08:57:26.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.676+0000 7f9a2b438640 1 -- 172.21.15.43:0/2315612853 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a240cb7c0 con 0x7f9a240d0120 2024-04-25T08:57:26.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.676+0000 7f9a19ffb640 1 -- 172.21.15.43:0/2315612853 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2000cd30 con 0x7f9a240d0120 2024-04-25T08:57:26.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.676+0000 7f9a2b438640 1 -- 172.21.15.43:0/2315612853 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a240cbd80 con 0x7f9a240d0120 2024-04-25T08:57:26.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.677+0000 7f9a19ffb640 1 -- 172.21.15.43:0/2315612853 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9a2001a070 con 0x7f9a240d0120 2024-04-25T08:57:26.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.678+0000 7f9a19ffb640 1 --2- 172.21.15.43:0/2315612853 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9a0806b4d0 0x7f9a0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.678+0000 7f9a19ffb640 1 -- 172.21.15.43:0/2315612853 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9a2008a460 con 0x7f9a240d0120 2024-04-25T08:57:26.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.678+0000 7f9a289d3640 1 --2- 172.21.15.43:0/2315612853 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9a0806b4d0 0x7f9a0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.679+0000 7f9a289d3640 1 --2- 172.21.15.43:0/2315612853 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9a0806b4d0 0x7f9a0806d990 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9a2415b0e0 tx=0x7f9a14008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:26.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.679+0000 7f9a2e21a740 1 Processor -- start 2024-04-25T08:57:26.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.679+0000 7f9a2e21a740 1 -- start start 2024-04-25T08:57:26.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.679+0000 7f9a2e21a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dd063ab10 0x561dd063d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.679+0000 7f9a2e21a740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561dd063aef0 0x561dd063d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.680+0000 7f9a2e21a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd060d770 0x561dd0610f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.680+0000 7f9a2e21a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dd0576da0 con 0x561dd060d770 2024-04-25T08:57:26.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.680+0000 7f9a2e21a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dd051fb80 con 0x561dd063ab10 2024-04-25T08:57:26.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.680+0000 7f9a2e21a740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x561dd0572e40 con 0x561dd063aef0 2024-04-25T08:57:26.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.680+0000 7f9a299d5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd060d770 0x561dd0610f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.680+0000 7f9a299d5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd060d770 0x561dd0610f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42228/0 (socket says 172.21.15.43:42228) 2024-04-25T08:57:26.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.680+0000 7f9a299d5640 1 -- 172.21.15.43:0/3104384515 learned_addr learned my addr 172.21.15.43:0/3104384515 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:26.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.680+0000 7f9a291d4640 1 --2- 172.21.15.43:0/3104384515 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dd063ab10 0x561dd063d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.680+0000 7f9a289d3640 1 --2- 172.21.15.43:0/3104384515 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561dd063aef0 0x561dd063d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.680+0000 7f9a299d5640 1 -- 172.21.15.43:0/3104384515 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561dd063aef0 msgr2=0x561dd063d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:26.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.680+0000 7f9a299d5640 1 --2- 172.21.15.43:0/3104384515 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561dd063aef0 0x561dd063d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.680+0000 7f9a299d5640 1 -- 172.21.15.43:0/3104384515 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dd063ab10 msgr2=0x561dd063d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:26.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.680+0000 7f9a299d5640 1 --2- 172.21.15.43:0/3104384515 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dd063ab10 0x561dd063d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.680+0000 7f9a299d5640 1 -- 172.21.15.43:0/3104384515 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561dd0529f10 con 0x561dd060d770 2024-04-25T08:57:26.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.680+0000 7f9a299d5640 1 --2- 172.21.15.43:0/3104384515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd060d770 0x561dd0610f80 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c009f40 tx=0x7f9a1c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=406caee43cea8296 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:26.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.680+0000 7f9a0effd640 1 -- 172.21.15.43:0/3104384515 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c036040 con 0x561dd060d770 2024-04-25T08:57:26.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.680+0000 7f9a0effd640 1 -- 172.21.15.43:0/3104384515 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a1c042070 con 0x561dd060d770 2024-04-25T08:57:26.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.680+0000 7f9a0effd640 1 -- 172.21.15.43:0/3104384515 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c03c8d0 con 0x561dd060d770 2024-04-25T08:57:26.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.681+0000 7f9a2e21a740 1 -- 172.21.15.43:0/3104384515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd060d770 msgr2=0x561dd0610f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:26.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.681+0000 7f9a2e21a740 1 --2- 172.21.15.43:0/3104384515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd060d770 0x561dd0610f80 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c009f40 tx=0x7f9a1c02e000 comp rx=0 tx=0).stop 2024-04-25T08:57:26.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.681+0000 7f9a2e21a740 1 -- 172.21.15.43:0/3104384515 shutdown_connections 2024-04-25T08:57:26.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.681+0000 7f9a2e21a740 1 --2- 172.21.15.43:0/3104384515 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd060d770 0x561dd0610f80 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.681+0000 7f9a2e21a740 1 --2- 172.21.15.43:0/3104384515 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561dd063aef0 0x561dd063d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.681+0000 7f9a2e21a740 1 --2- 172.21.15.43:0/3104384515 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dd063ab10 0x561dd063d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.681+0000 7f9a2e21a740 1 -- 172.21.15.43:0/3104384515 >> 172.21.15.43:0/3104384515 conn(0x561dd060f9e0 msgr2=0x561dd0658320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:26.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.681+0000 7f9a2e21a740 1 -- 172.21.15.43:0/3104384515 shutdown_connections 2024-04-25T08:57:26.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.681+0000 7f9a2e21a740 1 -- 172.21.15.43:0/3104384515 wait complete. 2024-04-25T08:57:26.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.682+0000 7f9a2e21a740 1 -- start start 2024-04-25T08:57:26.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.682+0000 7f9a2e21a740 1 Processor -- start 2024-04-25T08:57:26.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.683+0000 7f9a2e21a740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561dd063ab10 0x561dd062cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.683+0000 7f9a2e21a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dd063aef0 0x561dd060e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.683+0000 7f9a2e21a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd06309e0 0x561dd060e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.683+0000 7f9a2e21a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dd0612b90 con 0x561dd06309e0 2024-04-25T08:57:26.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.683+0000 7f9a2e21a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dd05154b0 con 0x561dd063aef0 2024-04-25T08:57:26.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.683+0000 7f9a2e21a740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x561dd0612e70 con 0x561dd063ab10 2024-04-25T08:57:26.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.683+0000 7f9a2e21a740 10 client.0 _ll_register_callbacks cb 0x561dd0555180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:26.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.683+0000 7f9a299d5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd06309e0 0x561dd060e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.683+0000 7f9a299d5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd06309e0 0x561dd060e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42238/0 (socket says 172.21.15.43:42238) 2024-04-25T08:57:26.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.683+0000 7f9a299d5640 1 -- 172.21.15.43:0/3498260284 learned_addr learned my addr 172.21.15.43:0/3498260284 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:26.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.683+0000 7f9a289d3640 1 --2- 172.21.15.43:0/3498260284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dd063aef0 0x561dd060e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.683+0000 7f9a291d4640 1 --2- 172.21.15.43:0/3498260284 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561dd063ab10 0x561dd062cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.683+0000 7f9a299d5640 1 -- 172.21.15.43:0/3498260284 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561dd063ab10 msgr2=0x561dd062cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:26.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.683+0000 7f9a299d5640 1 --2- 172.21.15.43:0/3498260284 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561dd063ab10 0x561dd062cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.683+0000 7f9a299d5640 1 -- 172.21.15.43:0/3498260284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dd063aef0 msgr2=0x561dd060e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:26.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.683+0000 7f9a299d5640 1 --2- 172.21.15.43:0/3498260284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dd063aef0 0x561dd060e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.683+0000 7f9a299d5640 1 -- 172.21.15.43:0/3498260284 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561dd03557d0 con 0x561dd06309e0 2024-04-25T08:57:26.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.683+0000 7f9a289d3640 1 --2- 172.21.15.43:0/3498260284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dd063aef0 0x561dd060e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:26.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.683+0000 7f9a291d4640 1 --2- 172.21.15.43:0/3498260284 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561dd063ab10 0x561dd062cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:26.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.683+0000 7f9a299d5640 1 --2- 172.21.15.43:0/3498260284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd06309e0 0x561dd060e960 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c0072a0 tx=0x7f9a1c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b157b923271b3037 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:26.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.683+0000 7f9a03fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:57:26.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.684+0000 7f9a03fff640 1 -- 172.21.15.43:0/3498260284 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1c04a070 con 0x561dd06309e0 2024-04-25T08:57:26.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.684+0000 7f9a03fff640 1 -- 172.21.15.43:0/3498260284 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9a1c004450 con 0x561dd06309e0 2024-04-25T08:57:26.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.684+0000 7f9a2e21a740 1 -- 172.21.15.43:0/3498260284 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9a1c042070 con 0x561dd06309e0 2024-04-25T08:57:26.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.684+0000 7f9a03fff640 1 -- 172.21.15.43:0/3498260284 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a1c056050 con 0x561dd06309e0 2024-04-25T08:57:26.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.684+0000 7f9a2e21a740 10 client.6051 fetch_fsmap learned FSMap version 22 2024-04-25T08:57:26.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.684+0000 7f9a2e21a740 1 -- 172.21.15.43:0/3498260284 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x561dd028b3f0 con 0x561dd06309e0 2024-04-25T08:57:26.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.684+0000 7f9a03fff640 1 -- 172.21.15.43:0/3498260284 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a1c042070 con 0x561dd06309e0 2024-04-25T08:57:26.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.684+0000 7f9a03fff640 1 -- 172.21.15.43:0/3498260284 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9a1c059020 con 0x561dd06309e0 2024-04-25T08:57:26.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.684+0000 7f9a2e21a740 10 client.6051 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T08:57:26.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.684+0000 7f9a2e21a740 10 client.6051 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T08:57:26.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.684+0000 7f9a2e21a740 20 client.6051 populate_metadata read hostname 'smithi043' 2024-04-25T08:57:26.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.684+0000 7f9a2e21a740 1 --2- 172.21.15.43:0/3498260284 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x561dd063f0f0 0x561dd063f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.684+0000 7f9a2e21a740 4 client.6051 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:57:26.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.684+0000 7f9a2e21a740 1 -- 172.21.15.43:0/3498260284 --> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561dd05842e0 con 0x561dd063f0f0 2024-04-25T08:57:26.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.684+0000 7f9a291d4640 1 --2- 172.21.15.43:0/3498260284 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x561dd063f0f0 0x561dd063f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.685+0000 7f9a291d4640 1 --2- 172.21.15.43:0/3498260284 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x561dd063f0f0 0x561dd063f520 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcdbc60dcde3dd15 server_cookie=eb2a4cbb9f81e40e in_seq=0 out_seq=0 2024-04-25T08:57:26.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.685+0000 7f9a03fff640 10 client.6051 ms_handle_connect on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:26.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.685+0000 7f9a03fff640 1 -- 172.21.15.43:0/3498260284 <== mds.0 v2:172.21.15.193:6836/4164634380 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561dd05842e0 con 0x561dd063f0f0 2024-04-25T08:57:26.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.685+0000 7f9a03fff640 10 client.6051 handle_command_reply: tid=0 2024-04-25T08:57:26.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.685+0000 7f9a2e21a740 1 client.6051 shutdown 2024-04-25T08:57:26.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.686+0000 7f9a2e21a740 1 -- 172.21.15.43:0/3498260284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd06309e0 msgr2=0x561dd060e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:26.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.686+0000 7f9a2e21a740 1 --2- 172.21.15.43:0/3498260284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd06309e0 0x561dd060e960 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c0072a0 tx=0x7f9a1c004ab0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.686+0000 7f9a2e21a740 1 -- 172.21.15.43:0/3498260284 shutdown_connections 2024-04-25T08:57:26.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.686+0000 7f9a2e21a740 1 --2- 172.21.15.43:0/3498260284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd06309e0 0x561dd060e960 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.686+0000 7f9a2e21a740 1 --2- 172.21.15.43:0/3498260284 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x561dd063f0f0 0x561dd063f520 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.686+0000 7f9a2e21a740 1 --2- 172.21.15.43:0/3498260284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dd063aef0 0x561dd060e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.686+0000 7f9a2e21a740 1 --2- 172.21.15.43:0/3498260284 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561dd063ab10 0x561dd062cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.686+0000 7f9a03fff640 0 client.6051 ms_handle_reset on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:26.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.686+0000 7f9a2e21a740 1 -- 172.21.15.43:0/3498260284 >> 172.21.15.43:0/3498260284 conn(0x561dd060d770 msgr2=0x561dd0639dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:26.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.686+0000 7f9a2e21a740 1 -- 172.21.15.43:0/3498260284 shutdown_connections 2024-04-25T08:57:26.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.686+0000 7f9a2e21a740 1 -- 172.21.15.43:0/3498260284 wait complete. 2024-04-25T08:57:26.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.686+0000 7f9a2e21a740 20 client.6051 trim_cache size 0 max 16384 2024-04-25T08:57:26.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.701+0000 7f9a2e21a740 1 Processor -- start 2024-04-25T08:57:26.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.701+0000 7f9a2e21a740 1 -- start start 2024-04-25T08:57:26.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.702+0000 7f9a2e21a740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561dd06309e0 0x561dd060e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.702+0000 7f9a2e21a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd063ab10 0x561dd0655210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.702+0000 7f9a2e21a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dd063aef0 0x561dd060c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.702+0000 7f9a2e21a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a241ffd50 con 0x561dd063ab10 2024-04-25T08:57:26.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.702+0000 7f9a2e21a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a241ff090 con 0x561dd063aef0 2024-04-25T08:57:26.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.702+0000 7f9a2e21a740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a240cce30 con 0x561dd06309e0 2024-04-25T08:57:26.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.702+0000 7f9a289d3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd063ab10 0x561dd0655210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.702+0000 7f9a289d3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd063ab10 0x561dd0655210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42252/0 (socket says 172.21.15.43:42252) 2024-04-25T08:57:26.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.702+0000 7f9a289d3640 1 -- 172.21.15.43:0/1508070179 learned_addr learned my addr 172.21.15.43:0/1508070179 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:26.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.702+0000 7f9a299d5640 1 --2- 172.21.15.43:0/1508070179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dd063aef0 0x561dd060c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.702+0000 7f9a289d3640 1 -- 172.21.15.43:0/1508070179 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561dd06309e0 msgr2=0x561dd060e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:26.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.702+0000 7f9a289d3640 1 --2- 172.21.15.43:0/1508070179 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561dd06309e0 0x561dd060e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.702+0000 7f9a289d3640 1 -- 172.21.15.43:0/1508070179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dd063aef0 msgr2=0x561dd060c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:26.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.702+0000 7f9a289d3640 1 --2- 172.21.15.43:0/1508070179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dd063aef0 0x561dd060c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.702+0000 7f9a289d3640 1 -- 172.21.15.43:0/1508070179 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561dd024b0f0 con 0x561dd063ab10 2024-04-25T08:57:26.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.702+0000 7f9a299d5640 1 --2- 172.21.15.43:0/1508070179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dd063aef0 0x561dd060c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:26.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.702+0000 7f9a289d3640 1 --2- 172.21.15.43:0/1508070179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd063ab10 0x561dd0655210 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f9a14010c50 tx=0x7f9a14016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db9490437ec7afbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:26.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.702+0000 7f9a0e7fc640 1 -- 172.21.15.43:0/1508070179 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1401c040 con 0x561dd063ab10 2024-04-25T08:57:26.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.702+0000 7f9a0e7fc640 1 -- 172.21.15.43:0/1508070179 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a1401d040 con 0x561dd063ab10 2024-04-25T08:57:26.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.702+0000 7f9a0e7fc640 1 -- 172.21.15.43:0/1508070179 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1400f040 con 0x561dd063ab10 2024-04-25T08:57:26.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.703+0000 7f9a2e21a740 1 -- 172.21.15.43:0/1508070179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd063ab10 msgr2=0x561dd0655210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:26.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.703+0000 7f9a2e21a740 1 --2- 172.21.15.43:0/1508070179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd063ab10 0x561dd0655210 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f9a14010c50 tx=0x7f9a14016000 comp rx=0 tx=0).stop 2024-04-25T08:57:26.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.703+0000 7f9a2e21a740 1 -- 172.21.15.43:0/1508070179 shutdown_connections 2024-04-25T08:57:26.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.703+0000 7f9a2e21a740 1 --2- 172.21.15.43:0/1508070179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dd063aef0 0x561dd060c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.703+0000 7f9a2e21a740 1 --2- 172.21.15.43:0/1508070179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd063ab10 0x561dd0655210 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.703+0000 7f9a2e21a740 1 --2- 172.21.15.43:0/1508070179 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561dd06309e0 0x561dd060e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.703+0000 7f9a2e21a740 1 -- 172.21.15.43:0/1508070179 >> 172.21.15.43:0/1508070179 conn(0x561dd060d770 msgr2=0x561dd063f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:26.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.703+0000 7f9a2e21a740 1 -- 172.21.15.43:0/1508070179 shutdown_connections 2024-04-25T08:57:26.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.703+0000 7f9a2e21a740 1 -- 172.21.15.43:0/1508070179 wait complete. 2024-04-25T08:57:26.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.703+0000 7f9a2e21a740 1 -- start start 2024-04-25T08:57:26.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.704+0000 7f9a2e21a740 1 Processor -- start 2024-04-25T08:57:26.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.704+0000 7f9a2e21a740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561dd06309e0 0x561dd062b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.705+0000 7f9a2e21a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd063ab10 0x561dd062ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.705+0000 7f9a2e21a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dd063aef0 0x561dd06c1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.705+0000 7f9a2e21a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x561dd060e2c0 con 0x561dd063ab10 2024-04-25T08:57:26.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.705+0000 7f9a2e21a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a240cd430 con 0x561dd063aef0 2024-04-25T08:57:26.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.705+0000 7f9a2e21a740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x561dd063a390 con 0x561dd06309e0 2024-04-25T08:57:26.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.705+0000 7f9a2e21a740 10 client.0 _ll_register_callbacks cb 0x7f9a1c034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:26.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.705+0000 7f9a289d3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd063ab10 0x561dd062ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.705+0000 7f9a289d3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd063ab10 0x561dd062ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42260/0 (socket says 172.21.15.43:42260) 2024-04-25T08:57:26.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.705+0000 7f9a289d3640 1 -- 172.21.15.43:0/2018555625 learned_addr learned my addr 172.21.15.43:0/2018555625 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:26.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.705+0000 7f9a291d4640 1 --2- 172.21.15.43:0/2018555625 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561dd06309e0 0x561dd062b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.705+0000 7f9a299d5640 1 --2- 172.21.15.43:0/2018555625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dd063aef0 0x561dd06c1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.705+0000 7f9a289d3640 1 -- 172.21.15.43:0/2018555625 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561dd06309e0 msgr2=0x561dd062b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:26.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.705+0000 7f9a289d3640 1 --2- 172.21.15.43:0/2018555625 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561dd06309e0 0x561dd062b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.705+0000 7f9a289d3640 1 -- 172.21.15.43:0/2018555625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dd063aef0 msgr2=0x561dd06c1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:26.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.705+0000 7f9a289d3640 1 --2- 172.21.15.43:0/2018555625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dd063aef0 0x561dd06c1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.705+0000 7f9a289d3640 1 -- 172.21.15.43:0/2018555625 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561dd0612fe0 con 0x561dd063ab10 2024-04-25T08:57:26.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.705+0000 7f9a291d4640 1 --2- 172.21.15.43:0/2018555625 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561dd06309e0 0x561dd062b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:26.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.705+0000 7f9a299d5640 1 --2- 172.21.15.43:0/2018555625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dd063aef0 0x561dd06c1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:26.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.705+0000 7f9a289d3640 1 --2- 172.21.15.43:0/2018555625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd063ab10 0x561dd062ba80 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f9a14010c20 tx=0x7f9a140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb5b77abd6b09a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:26.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.705+0000 7f9a03fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:57:26.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.705+0000 7f9a03fff640 1 -- 172.21.15.43:0/2018555625 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1400f040 con 0x561dd063ab10 2024-04-25T08:57:26.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.705+0000 7f9a03fff640 1 -- 172.21.15.43:0/2018555625 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9a14035ae0 con 0x561dd063ab10 2024-04-25T08:57:26.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.705+0000 7f9a2e21a740 1 -- 172.21.15.43:0/2018555625 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9a1401d040 con 0x561dd063ab10 2024-04-25T08:57:26.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.705+0000 7f9a03fff640 1 -- 172.21.15.43:0/2018555625 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9a1401c040 con 0x561dd063ab10 2024-04-25T08:57:26.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.705+0000 7f9a2e21a740 10 client.6057 fetch_fsmap learned FSMap version 22 2024-04-25T08:57:26.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.705+0000 7f9a2e21a740 1 -- 172.21.15.43:0/2018555625 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x561dd0610580 con 0x561dd063ab10 2024-04-25T08:57:26.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.706+0000 7f9a03fff640 1 -- 172.21.15.43:0/2018555625 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a1401d040 con 0x561dd063ab10 2024-04-25T08:57:26.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.706+0000 7f9a03fff640 1 -- 172.21.15.43:0/2018555625 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9a1403c050 con 0x561dd063ab10 2024-04-25T08:57:26.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.706+0000 7f9a2e21a740 10 client.6057 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T08:57:26.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.706+0000 7f9a2e21a740 10 client.6057 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T08:57:26.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.706+0000 7f9a2e21a740 20 client.6057 populate_metadata read hostname 'smithi043' 2024-04-25T08:57:26.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.706+0000 7f9a2e21a740 1 --2- 172.21.15.43:0/2018555625 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x561dd06c2090 0x561dd0667080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.706+0000 7f9a2e21a740 4 client.6057 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:57:26.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.706+0000 7f9a2e21a740 1 -- 172.21.15.43:0/2018555625 --> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561dd057e630 con 0x561dd06c2090 2024-04-25T08:57:26.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.706+0000 7f9a291d4640 1 --2- 172.21.15.43:0/2018555625 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x561dd06c2090 0x561dd0667080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.707+0000 7f9a03fff640 10 client.6057 ms_handle_connect on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:26.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.707+0000 7f9a291d4640 1 --2- 172.21.15.43:0/2018555625 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x561dd06c2090 0x561dd0667080 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dd66b1998cb0cd7 server_cookie=7214e9312d61386d in_seq=0 out_seq=0 2024-04-25T08:57:26.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.707+0000 7f9a03fff640 1 -- 172.21.15.43:0/2018555625 <== mds.0 v2:172.21.15.193:6836/4164634380 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x561dd057e630 con 0x561dd06c2090 2024-04-25T08:57:26.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.707+0000 7f9a03fff640 10 client.6057 handle_command_reply: tid=0 2024-04-25T08:57:26.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.707+0000 7f9a2e21a740 1 client.6057 shutdown 2024-04-25T08:57:26.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.707+0000 7f9a2e21a740 1 -- 172.21.15.43:0/2018555625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd063ab10 msgr2=0x561dd062ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:26.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.707+0000 7f9a2e21a740 1 --2- 172.21.15.43:0/2018555625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd063ab10 0x561dd062ba80 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f9a14010c20 tx=0x7f9a140046c0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.707+0000 7f9a2e21a740 1 -- 172.21.15.43:0/2018555625 shutdown_connections 2024-04-25T08:57:26.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.707+0000 7f9a2e21a740 1 --2- 172.21.15.43:0/2018555625 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x561dd06c2090 0x561dd0667080 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.707+0000 7f9a2e21a740 1 --2- 172.21.15.43:0/2018555625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561dd063aef0 0x561dd06c1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.707+0000 7f9a2e21a740 1 --2- 172.21.15.43:0/2018555625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561dd063ab10 0x561dd062ba80 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.707+0000 7f9a03fff640 0 client.6057 ms_handle_reset on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:26.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.707+0000 7f9a2e21a740 1 --2- 172.21.15.43:0/2018555625 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561dd06309e0 0x561dd062b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.707+0000 7f9a2e21a740 1 -- 172.21.15.43:0/2018555625 >> 172.21.15.43:0/2018555625 conn(0x561dd060d770 msgr2=0x561dd060e470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:26.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.707+0000 7f9a2e21a740 1 -- 172.21.15.43:0/2018555625 shutdown_connections 2024-04-25T08:57:26.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.707+0000 7f9a2e21a740 1 -- 172.21.15.43:0/2018555625 wait complete. 2024-04-25T08:57:26.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.707+0000 7f9a2e21a740 20 client.6057 trim_cache size 0 max 16384 2024-04-25T08:57:26.703 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T08:57:26.703 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-25T08:57:26.703 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-25T08:57:26.703 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T08:57:26.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.708+0000 7f9a2b438640 1 -- 172.21.15.43:0/2315612853 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9a0806b4d0 msgr2=0x7f9a0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:26.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.708+0000 7f9a2b438640 1 --2- 172.21.15.43:0/2315612853 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9a0806b4d0 0x7f9a0806d990 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9a2415b0e0 tx=0x7f9a14008040 comp rx=0 tx=0).stop 2024-04-25T08:57:26.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.708+0000 7f9a2b438640 1 -- 172.21.15.43:0/2315612853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a240d0120 msgr2=0x7f9a2415a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:26.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.708+0000 7f9a2b438640 1 --2- 172.21.15.43:0/2315612853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a240d0120 0x7f9a2415a160 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f9a2000e8a0 tx=0x7f9a2000ed60 comp rx=0 tx=0).stop 2024-04-25T08:57:26.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.708+0000 7f9a2b438640 1 -- 172.21.15.43:0/2315612853 shutdown_connections 2024-04-25T08:57:26.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.708+0000 7f9a2b438640 1 --2- 172.21.15.43:0/2315612853 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9a24161310 0x7f9a2415bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.708+0000 7f9a2b438640 1 --2- 172.21.15.43:0/2315612853 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9a0806b4d0 0x7f9a0806d990 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.708+0000 7f9a2b438640 1 --2- 172.21.15.43:0/2315612853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a240d0aa0 0x7f9a2415b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.708+0000 7f9a2b438640 1 --2- 172.21.15.43:0/2315612853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a240d0120 0x7f9a2415a160 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.708+0000 7f9a2b438640 1 -- 172.21.15.43:0/2315612853 >> 172.21.15.43:0/2315612853 conn(0x7f9a2409cd20 msgr2=0x7f9a240d3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:26.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.708+0000 7f9a2b438640 1 -- 172.21.15.43:0/2315612853 shutdown_connections 2024-04-25T08:57:26.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.708+0000 7f9a2b438640 1 -- 172.21.15.43:0/2315612853 wait complete. 2024-04-25T08:57:26.708 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T08:57:26.708 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:57:26.708 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T08:57:26.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.821+0000 7f23dea5d640 1 Processor -- start 2024-04-25T08:57:26.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.822+0000 7f23dea5d640 1 -- start start 2024-04-25T08:57:26.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.822+0000 7f23dea5d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f23d815dd60 0x7f23d80d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.822+0000 7f23dea5d640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f23d80d0aa0 0x7f23d806bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.822+0000 7f23dea5d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23d815e140 con 0x7f23d80d1fa0 2024-04-25T08:57:26.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.822+0000 7f23dea5d640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23d80d27d0 con 0x7f23d80d0aa0 2024-04-25T08:57:26.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.822+0000 7f23dea5d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23d806b1e0 con 0x7f23d815dd60 2024-04-25T08:57:26.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.822+0000 7f23d77fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f23d80d1fa0 0x7f23d80d0390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49660/0 (socket says 172.21.15.43:49660) 2024-04-25T08:57:26.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.823+0000 7f23d77fe640 1 -- 172.21.15.43:0/4134310067 learned_addr learned my addr 172.21.15.43:0/4134310067 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:26.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.823+0000 7f23d6ffd640 1 -- 172.21.15.43:0/4134310067 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1791874486 0 0) 0x7f23d815e140 con 0x7f23d80d1fa0 2024-04-25T08:57:26.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.823+0000 7f23d7fff640 1 --2- 172.21.15.43:0/4134310067 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f23d815dd60 0x7f23d80d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.823+0000 7f23d6ffd640 1 -- 172.21.15.43:0/4134310067 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23c0003520 con 0x7f23d80d1fa0 2024-04-25T08:57:26.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.823+0000 7f23dcffa640 1 --2- 172.21.15.43:0/4134310067 >> v2:172.21.15.193:3300/0 conn(0x7f23d80d0aa0 0x7f23d806bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.823+0000 7f23d6ffd640 1 -- 172.21.15.43:0/4134310067 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2756335336 0 0) 0x7f23c0003520 con 0x7f23d80d1fa0 2024-04-25T08:57:26.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.823+0000 7f23d6ffd640 1 -- 172.21.15.43:0/4134310067 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f23d815dd60 msgr2=0x7f23d80d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:26.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.823+0000 7f23d6ffd640 1 --2- 172.21.15.43:0/4134310067 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f23d815dd60 0x7f23d80d1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.823+0000 7f23d6ffd640 1 -- 172.21.15.43:0/4134310067 >> v2:172.21.15.193:3300/0 conn(0x7f23d80d0aa0 msgr2=0x7f23d806bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:26.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.823+0000 7f23d6ffd640 1 --2- 172.21.15.43:0/4134310067 >> v2:172.21.15.193:3300/0 conn(0x7f23d80d0aa0 0x7f23d806bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.823+0000 7f23d6ffd640 1 -- 172.21.15.43:0/4134310067 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d806b4c0 con 0x7f23d80d1fa0 2024-04-25T08:57:26.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.823+0000 7f23d6ffd640 1 -- 172.21.15.43:0/4134310067 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f23cc001670 con 0x7f23d80d1fa0 2024-04-25T08:57:26.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.823+0000 7f23d6ffd640 1 -- 172.21.15.43:0/4134310067 >> v1:172.21.15.43:6789/0 conn(0x7f23d80d1fa0 legacy=0x7f23d80d0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:26.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.823+0000 7f23d6ffd640 1 --2- 172.21.15.43:0/4134310067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23c00039b0 0x7f23c0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.823+0000 7f23d6ffd640 1 --2- 172.21.15.43:0/4134310067 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f23d815dd60 0x7f23c0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.823+0000 7f23d6ffd640 1 --2- 172.21.15.43:0/4134310067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23c0004d40 0x7f23c00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.823+0000 7f23d6ffd640 1 -- 172.21.15.43:0/4134310067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d80d27d0 con 0x7f23c0004d40 2024-04-25T08:57:26.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.823+0000 7f23d6ffd640 1 -- 172.21.15.43:0/4134310067 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d806b1e0 con 0x7f23c00039b0 2024-04-25T08:57:26.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.823+0000 7f23d6ffd640 1 -- 172.21.15.43:0/4134310067 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23c000b9c0 con 0x7f23d815dd60 2024-04-25T08:57:26.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.823+0000 7f23dcffa640 1 --2- 172.21.15.43:0/4134310067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23c0004d40 0x7f23c00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.823+0000 7f23d7fff640 1 --2- 172.21.15.43:0/4134310067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23c00039b0 0x7f23c0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.823+0000 7f23d77fe640 1 --2- 172.21.15.43:0/4134310067 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f23d815dd60 0x7f23c0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.824+0000 7f23dcffa640 1 -- 172.21.15.43:0/4134310067 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f23d815dd60 msgr2=0x7f23c0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:26.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.824+0000 7f23dcffa640 1 --2- 172.21.15.43:0/4134310067 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f23d815dd60 0x7f23c0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.824+0000 7f23dcffa640 1 -- 172.21.15.43:0/4134310067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23c00039b0 msgr2=0x7f23c0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:26.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.824+0000 7f23dcffa640 1 --2- 172.21.15.43:0/4134310067 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23c00039b0 0x7f23c0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.824+0000 7f23dcffa640 1 -- 172.21.15.43:0/4134310067 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c0003520 con 0x7f23c0004d40 2024-04-25T08:57:26.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.824+0000 7f23dcffa640 1 --2- 172.21.15.43:0/4134310067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23c0004d40 0x7f23c00051c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f23d808e180 tx=0x7f23c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e97942828a3574e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:26.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.825+0000 7f23d6ffd640 1 -- 172.21.15.43:0/4134310067 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d0009c10 con 0x7f23c0004d40 2024-04-25T08:57:26.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.825+0000 7f23d6ffd640 1 -- 172.21.15.43:0/4134310067 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d0036070 con 0x7f23c0004d40 2024-04-25T08:57:26.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.825+0000 7f23d6ffd640 1 -- 172.21.15.43:0/4134310067 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d0031810 con 0x7f23c0004d40 2024-04-25T08:57:26.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.825+0000 7f23dea5d640 1 -- 172.21.15.43:0/4134310067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23c0004d40 msgr2=0x7f23c00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:26.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.825+0000 7f23dea5d640 1 --2- 172.21.15.43:0/4134310067 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23c0004d40 0x7f23c00051c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f23d808e180 tx=0x7f23c0005830 comp rx=0 tx=0).stop 2024-04-25T08:57:26.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.825+0000 7f23dcffa640 1 -- 172.21.15.43:0/4134310067 reap_dead start 2024-04-25T08:57:26.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.825+0000 7f23dea5d640 1 -- 172.21.15.43:0/4134310067 shutdown_connections 2024-04-25T08:57:26.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.825+0000 7f23dea5d640 1 -- 172.21.15.43:0/4134310067 >> 172.21.15.43:0/4134310067 conn(0x7f23d80d39c0 msgr2=0x7f23d80d3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:26.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.825+0000 7f23dea5d640 1 -- 172.21.15.43:0/4134310067 shutdown_connections 2024-04-25T08:57:26.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.825+0000 7f23dea5d640 1 -- 172.21.15.43:0/4134310067 wait complete. 2024-04-25T08:57:26.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.826+0000 7f23dea5d640 1 Processor -- start 2024-04-25T08:57:26.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.827+0000 7f23dea5d640 1 -- start start 2024-04-25T08:57:26.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.827+0000 7f23dea5d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23d81f3f80 0x7f23d81f49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.827+0000 7f23dea5d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23d81606b0 0x7f23d8160b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.827+0000 7f23dea5d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f23d815e7a0 0x7f23d815ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.827+0000 7f23dea5d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d80d2580 con 0x7f23d81f3f80 2024-04-25T08:57:26.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.827+0000 7f23dea5d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d80d2400 con 0x7f23d81606b0 2024-04-25T08:57:26.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.827+0000 7f23dea5d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23d80d1000 con 0x7f23d815e7a0 2024-04-25T08:57:26.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.827+0000 7f23d7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23d81f3f80 0x7f23d81f49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.827+0000 7f23d7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23d81f3f80 0x7f23d81f49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42278/0 (socket says 172.21.15.43:42278) 2024-04-25T08:57:26.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.827+0000 7f23d7fff640 1 -- 172.21.15.43:0/2496322065 learned_addr learned my addr 172.21.15.43:0/2496322065 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:26.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.827+0000 7f23d77fe640 1 --2- 172.21.15.43:0/2496322065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23d81606b0 0x7f23d8160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.828+0000 7f23dcffa640 1 --2- 172.21.15.43:0/2496322065 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f23d815e7a0 0x7f23d815ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.828+0000 7f23d7fff640 1 -- 172.21.15.43:0/2496322065 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f23d815e7a0 msgr2=0x7f23d815ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:26.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.828+0000 7f23d7fff640 1 --2- 172.21.15.43:0/2496322065 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f23d815e7a0 0x7f23d815ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.828+0000 7f23d7fff640 1 -- 172.21.15.43:0/2496322065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23d81606b0 msgr2=0x7f23d8160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:26.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.828+0000 7f23d7fff640 1 --2- 172.21.15.43:0/2496322065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23d81606b0 0x7f23d8160b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.828+0000 7f23d7fff640 1 -- 172.21.15.43:0/2496322065 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d8161250 con 0x7f23d81f3f80 2024-04-25T08:57:26.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.828+0000 7f23d7fff640 1 --2- 172.21.15.43:0/2496322065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23d81f3f80 0x7f23d81f49f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f23c8011940 tx=0x7f23c8011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:26.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.828+0000 7f23d4ff9640 1 -- 172.21.15.43:0/2496322065 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c800fa90 con 0x7f23d81f3f80 2024-04-25T08:57:26.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.828+0000 7f23d4ff9640 1 -- 172.21.15.43:0/2496322065 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c800fc30 con 0x7f23d81f3f80 2024-04-25T08:57:26.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.828+0000 7f23dea5d640 1 -- 172.21.15.43:0/2496322065 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23d80b79e0 con 0x7f23d81f3f80 2024-04-25T08:57:26.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.828+0000 7f23d4ff9640 1 -- 172.21.15.43:0/2496322065 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c8014440 con 0x7f23d81f3f80 2024-04-25T08:57:26.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.828+0000 7f23dea5d640 1 -- 172.21.15.43:0/2496322065 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d80b7fa0 con 0x7f23d81f3f80 2024-04-25T08:57:26.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.830+0000 7f23d4ff9640 1 -- 172.21.15.43:0/2496322065 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f23c8012040 con 0x7f23d81f3f80 2024-04-25T08:57:26.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.830+0000 7f23d4ff9640 1 --2- 172.21.15.43:0/2496322065 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f23c406b500 0x7f23c406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.830+0000 7f23d4ff9640 1 -- 172.21.15.43:0/2496322065 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f23c808a7c0 con 0x7f23d81f3f80 2024-04-25T08:57:26.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.830+0000 7f23d77fe640 1 --2- 172.21.15.43:0/2496322065 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f23c406b500 0x7f23c406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.831+0000 7f23d77fe640 1 --2- 172.21.15.43:0/2496322065 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f23c406b500 0x7f23c406d9c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f23cc004840 tx=0x7f23cc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:26.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.831+0000 7f23e13fd740 1 Processor -- start 2024-04-25T08:57:26.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.831+0000 7f23e13fd740 1 -- start start 2024-04-25T08:57:26.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.832+0000 7f23e13fd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c872fbbaa0 0x55c872fbe2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.832+0000 7f23e13fd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c872fbbe80 0x55c872fbe7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.832+0000 7f23e13fd740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c872f8e700 0x55c872f91f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.832+0000 7f23e13fd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c872ef7e00 con 0x55c872fbbaa0 2024-04-25T08:57:26.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.832+0000 7f23e13fd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c872ea0d20 con 0x55c872fbbe80 2024-04-25T08:57:26.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.832+0000 7f23e13fd740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c872ef3ea0 con 0x55c872f8e700 2024-04-25T08:57:26.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.832+0000 7f23d7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c872fbbaa0 0x55c872fbe2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.832+0000 7f23d7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c872fbbaa0 0x55c872fbe2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42282/0 (socket says 172.21.15.43:42282) 2024-04-25T08:57:26.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.832+0000 7f23d7fff640 1 -- 172.21.15.43:0/3262863159 learned_addr learned my addr 172.21.15.43:0/3262863159 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:26.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.832+0000 7f23d77fe640 1 --2- 172.21.15.43:0/3262863159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c872fbbe80 0x55c872fbe7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.832+0000 7f23dcffa640 1 --2- 172.21.15.43:0/3262863159 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c872f8e700 0x55c872f91f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.832+0000 7f23d7fff640 1 -- 172.21.15.43:0/3262863159 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c872f8e700 msgr2=0x55c872f91f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:26.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.832+0000 7f23d7fff640 1 --2- 172.21.15.43:0/3262863159 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c872f8e700 0x55c872f91f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.832+0000 7f23d7fff640 1 -- 172.21.15.43:0/3262863159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c872fbbe80 msgr2=0x55c872fbe7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:26.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.832+0000 7f23d7fff640 1 --2- 172.21.15.43:0/3262863159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c872fbbe80 0x55c872fbe7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.832+0000 7f23d7fff640 1 -- 172.21.15.43:0/3262863159 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c872ec2df0 con 0x55c872fbbaa0 2024-04-25T08:57:26.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.832+0000 7f23d77fe640 1 --2- 172.21.15.43:0/3262863159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c872fbbe80 0x55c872fbe7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:26.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.832+0000 7f23d7fff640 1 --2- 172.21.15.43:0/3262863159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c872fbbaa0 0x55c872fbe2b0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f23c8088d10 tx=0x7f23c8055290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c7f05043c4b2cdf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:26.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.832+0000 7f23bdffb640 1 -- 172.21.15.43:0/3262863159 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c8057240 con 0x55c872fbbaa0 2024-04-25T08:57:26.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.832+0000 7f23bdffb640 1 -- 172.21.15.43:0/3262863159 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c80555b0 con 0x55c872fbbaa0 2024-04-25T08:57:26.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.832+0000 7f23bdffb640 1 -- 172.21.15.43:0/3262863159 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c80593e0 con 0x55c872fbbaa0 2024-04-25T08:57:26.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.833+0000 7f23e13fd740 1 -- 172.21.15.43:0/3262863159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c872fbbaa0 msgr2=0x55c872fbe2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:26.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.833+0000 7f23e13fd740 1 --2- 172.21.15.43:0/3262863159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c872fbbaa0 0x55c872fbe2b0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f23c8088d10 tx=0x7f23c8055290 comp rx=0 tx=0).stop 2024-04-25T08:57:26.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.833+0000 7f23e13fd740 1 -- 172.21.15.43:0/3262863159 shutdown_connections 2024-04-25T08:57:26.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.833+0000 7f23e13fd740 1 --2- 172.21.15.43:0/3262863159 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c872f8e700 0x55c872f91f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.833+0000 7f23e13fd740 1 --2- 172.21.15.43:0/3262863159 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c872fbbe80 0x55c872fbe7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.833+0000 7f23e13fd740 1 --2- 172.21.15.43:0/3262863159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c872fbbaa0 0x55c872fbe2b0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.833+0000 7f23e13fd740 1 -- 172.21.15.43:0/3262863159 >> 172.21.15.43:0/3262863159 conn(0x55c872f90970 msgr2=0x55c872fd92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:26.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.833+0000 7f23e13fd740 1 -- 172.21.15.43:0/3262863159 shutdown_connections 2024-04-25T08:57:26.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.833+0000 7f23e13fd740 1 -- 172.21.15.43:0/3262863159 wait complete. 2024-04-25T08:57:26.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.833+0000 7f23e13fd740 1 -- start start 2024-04-25T08:57:26.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.834+0000 7f23e13fd740 1 Processor -- start 2024-04-25T08:57:26.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.834+0000 7f23e13fd740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c872fbbaa0 0x55c872fd07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.834+0000 7f23e13fd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c872fbbe80 0x55c872fb2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.834+0000 7f23e13fd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c872fdd190 0x55c872fd4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.834+0000 7f23e13fd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c872f93b20 con 0x55c872fbbe80 2024-04-25T08:57:26.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.834+0000 7f23e13fd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c872e96590 con 0x55c872fdd190 2024-04-25T08:57:26.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.834+0000 7f23e13fd740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c872f93e00 con 0x55c872fbbaa0 2024-04-25T08:57:26.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.835+0000 7f23e13fd740 10 client.0 _ll_register_callbacks cb 0x55c872ea2e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:26.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.835+0000 7f23d77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c872fbbe80 0x55c872fb2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.835+0000 7f23d77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c872fbbe80 0x55c872fb2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42288/0 (socket says 172.21.15.43:42288) 2024-04-25T08:57:26.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.835+0000 7f23d77fe640 1 -- 172.21.15.43:0/3247162017 learned_addr learned my addr 172.21.15.43:0/3247162017 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:26.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.835+0000 7f23d7fff640 1 --2- 172.21.15.43:0/3247162017 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c872fbbaa0 0x55c872fd07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.835+0000 7f23dcffa640 1 --2- 172.21.15.43:0/3247162017 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c872fdd190 0x55c872fd4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.835+0000 7f23d77fe640 1 -- 172.21.15.43:0/3247162017 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c872fbbaa0 msgr2=0x55c872fd07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:26.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.835+0000 7f23d77fe640 1 --2- 172.21.15.43:0/3247162017 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c872fbbaa0 0x55c872fd07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.835+0000 7f23d77fe640 1 -- 172.21.15.43:0/3247162017 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c872fdd190 msgr2=0x55c872fd4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:26.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.835+0000 7f23d77fe640 1 --2- 172.21.15.43:0/3247162017 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c872fdd190 0x55c872fd4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.835+0000 7f23d77fe640 1 -- 172.21.15.43:0/3247162017 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c872c0c3f0 con 0x55c872fbbe80 2024-04-25T08:57:26.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.835+0000 7f23d77fe640 1 --2- 172.21.15.43:0/3247162017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c872fbbe80 0x55c872fb2ca0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f23cc015e80 tx=0x7f23d806b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ccf9bc1b0b7bbb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:26.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.835+0000 7f23b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:57:26.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.835+0000 7f23b6ffd640 1 -- 172.21.15.43:0/3247162017 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23cc028060 con 0x55c872fbbe80 2024-04-25T08:57:26.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.835+0000 7f23b6ffd640 1 -- 172.21.15.43:0/3247162017 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f23cc007db0 con 0x55c872fbbe80 2024-04-25T08:57:26.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.835+0000 7f23e13fd740 1 -- 172.21.15.43:0/3247162017 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23c80555b0 con 0x55c872fbbe80 2024-04-25T08:57:26.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.835+0000 7f23b6ffd640 1 -- 172.21.15.43:0/3247162017 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23cc02c3a0 con 0x55c872fbbe80 2024-04-25T08:57:26.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.835+0000 7f23e13fd740 10 client.6069 fetch_fsmap learned FSMap version 22 2024-04-25T08:57:26.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.835+0000 7f23e13fd740 1 -- 172.21.15.43:0/3247162017 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x55c872cd67d0 con 0x55c872fbbe80 2024-04-25T08:57:26.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.836+0000 7f23b6ffd640 1 -- 172.21.15.43:0/3247162017 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c80555b0 con 0x55c872fbbe80 2024-04-25T08:57:26.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.836+0000 7f23b6ffd640 1 -- 172.21.15.43:0/3247162017 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f23cc02c810 con 0x55c872fbbe80 2024-04-25T08:57:26.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.836+0000 7f23e13fd740 10 client.6069 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T08:57:26.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.836+0000 7f23e13fd740 10 client.6069 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T08:57:26.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.836+0000 7f23e13fd740 20 client.6069 populate_metadata read hostname 'smithi043' 2024-04-25T08:57:26.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.836+0000 7f23e13fd740 1 --2- 172.21.15.43:0/3247162017 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55c872f933f0 0x55c872fda3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.836+0000 7f23e13fd740 4 client.6069 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:57:26.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.836+0000 7f23e13fd740 1 -- 172.21.15.43:0/3247162017 --> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c872f05340 con 0x55c872f933f0 2024-04-25T08:57:26.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.836+0000 7f23d7fff640 1 --2- 172.21.15.43:0/3247162017 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55c872f933f0 0x55c872fda3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.836+0000 7f23b6ffd640 10 client.6069 ms_handle_connect on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:26.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.836+0000 7f23d7fff640 1 --2- 172.21.15.43:0/3247162017 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55c872f933f0 0x55c872fda3e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e04787756f25643 server_cookie=9a6c297cfc815f13 in_seq=0 out_seq=0 2024-04-25T08:57:26.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.837+0000 7f23b6ffd640 1 -- 172.21.15.43:0/3247162017 <== mds.0 v2:172.21.15.193:6836/4164634380 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c872f05340 con 0x55c872f933f0 2024-04-25T08:57:26.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.837+0000 7f23b6ffd640 10 client.6069 handle_command_reply: tid=0 2024-04-25T08:57:26.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.837+0000 7f23e13fd740 1 client.6069 shutdown 2024-04-25T08:57:26.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.837+0000 7f23e13fd740 1 -- 172.21.15.43:0/3247162017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c872fbbe80 msgr2=0x55c872fb2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:26.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.837+0000 7f23e13fd740 1 --2- 172.21.15.43:0/3247162017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c872fbbe80 0x55c872fb2ca0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f23cc015e80 tx=0x7f23d806b690 comp rx=0 tx=0).stop 2024-04-25T08:57:26.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.837+0000 7f23e13fd740 1 -- 172.21.15.43:0/3247162017 shutdown_connections 2024-04-25T08:57:26.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.837+0000 7f23e13fd740 1 --2- 172.21.15.43:0/3247162017 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55c872f933f0 0x55c872fda3e0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.837+0000 7f23e13fd740 1 --2- 172.21.15.43:0/3247162017 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c872fdd190 0x55c872fd4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.837+0000 7f23e13fd740 1 --2- 172.21.15.43:0/3247162017 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c872fbbe80 0x55c872fb2ca0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.837+0000 7f23b6ffd640 0 client.6069 ms_handle_reset on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:26.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.837+0000 7f23e13fd740 1 --2- 172.21.15.43:0/3247162017 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c872fbbaa0 0x55c872fd07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.837+0000 7f23e13fd740 1 -- 172.21.15.43:0/3247162017 >> 172.21.15.43:0/3247162017 conn(0x55c872f8e700 msgr2=0x55c872fbace0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:26.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.837+0000 7f23e13fd740 1 -- 172.21.15.43:0/3247162017 shutdown_connections 2024-04-25T08:57:26.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.837+0000 7f23e13fd740 1 -- 172.21.15.43:0/3247162017 wait complete. 2024-04-25T08:57:26.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.837+0000 7f23e13fd740 20 client.6069 trim_cache size 0 max 16384 2024-04-25T08:57:26.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.849+0000 7f23e13fd740 1 Processor -- start 2024-04-25T08:57:26.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.849+0000 7f23e13fd740 1 -- start start 2024-04-25T08:57:26.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.850+0000 7f23e13fd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c872f933f0 0x55c872fd3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.850+0000 7f23e13fd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c872fbbaa0 0x55c872fda3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.850+0000 7f23e13fd740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c872fbbe80 0x55c872fda920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.850+0000 7f23e13fd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c872fe1960 con 0x55c872fbbaa0 2024-04-25T08:57:26.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.850+0000 7f23e13fd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d8200b10 con 0x55c872f933f0 2024-04-25T08:57:26.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.850+0000 7f23e13fd740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23d81ffd10 con 0x55c872fbbe80 2024-04-25T08:57:26.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.850+0000 7f23d77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c872fbbaa0 0x55c872fda3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.850+0000 7f23d77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c872fbbaa0 0x55c872fda3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42300/0 (socket says 172.21.15.43:42300) 2024-04-25T08:57:26.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.850+0000 7f23d77fe640 1 -- 172.21.15.43:0/3261395750 learned_addr learned my addr 172.21.15.43:0/3261395750 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:26.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.850+0000 7f23d7fff640 1 --2- 172.21.15.43:0/3261395750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c872f933f0 0x55c872fd3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.850+0000 7f23dcffa640 1 --2- 172.21.15.43:0/3261395750 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c872fbbe80 0x55c872fda920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.850+0000 7f23d77fe640 1 -- 172.21.15.43:0/3261395750 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c872fbbe80 msgr2=0x55c872fda920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:26.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.850+0000 7f23d77fe640 1 --2- 172.21.15.43:0/3261395750 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c872fbbe80 0x55c872fda920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.850+0000 7f23d77fe640 1 -- 172.21.15.43:0/3261395750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c872f933f0 msgr2=0x55c872fd3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:26.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.850+0000 7f23d77fe640 1 --2- 172.21.15.43:0/3261395750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c872f933f0 0x55c872fd3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.850+0000 7f23d77fe640 1 -- 172.21.15.43:0/3261395750 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c872fd52b0 con 0x55c872fbbaa0 2024-04-25T08:57:26.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.850+0000 7f23d7fff640 1 --2- 172.21.15.43:0/3261395750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c872f933f0 0x55c872fd3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:26.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.850+0000 7f23dcffa640 1 --2- 172.21.15.43:0/3261395750 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c872fbbe80 0x55c872fda920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:26.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.850+0000 7f23d77fe640 1 --2- 172.21.15.43:0/3261395750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c872fbbaa0 0x55c872fda3e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f23cc001050 tx=0x7f23cc022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cde8d0b9a82fd45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:26.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.851+0000 7f23bd7fa640 1 -- 172.21.15.43:0/3261395750 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23cc007480 con 0x55c872fbbaa0 2024-04-25T08:57:26.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.851+0000 7f23bd7fa640 1 -- 172.21.15.43:0/3261395750 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23cc02cd60 con 0x55c872fbbaa0 2024-04-25T08:57:26.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.851+0000 7f23bd7fa640 1 -- 172.21.15.43:0/3261395750 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23cc022430 con 0x55c872fbbaa0 2024-04-25T08:57:26.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.851+0000 7f23e13fd740 1 -- 172.21.15.43:0/3261395750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c872fbbaa0 msgr2=0x55c872fda3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:26.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.851+0000 7f23e13fd740 1 --2- 172.21.15.43:0/3261395750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c872fbbaa0 0x55c872fda3e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f23cc001050 tx=0x7f23cc022000 comp rx=0 tx=0).stop 2024-04-25T08:57:26.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.851+0000 7f23e13fd740 1 -- 172.21.15.43:0/3261395750 shutdown_connections 2024-04-25T08:57:26.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.851+0000 7f23e13fd740 1 --2- 172.21.15.43:0/3261395750 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c872fbbe80 0x55c872fda920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.851+0000 7f23e13fd740 1 --2- 172.21.15.43:0/3261395750 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c872fbbaa0 0x55c872fda3e0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.851+0000 7f23e13fd740 1 --2- 172.21.15.43:0/3261395750 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c872f933f0 0x55c872fd3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.851+0000 7f23e13fd740 1 -- 172.21.15.43:0/3261395750 >> 172.21.15.43:0/3261395750 conn(0x55c872f8e700 msgr2=0x55c872f8d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:26.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.851+0000 7f23e13fd740 1 -- 172.21.15.43:0/3261395750 shutdown_connections 2024-04-25T08:57:26.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.851+0000 7f23e13fd740 1 -- 172.21.15.43:0/3261395750 wait complete. 2024-04-25T08:57:26.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.852+0000 7f23e13fd740 1 -- start start 2024-04-25T08:57:26.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.852+0000 7f23e13fd740 1 Processor -- start 2024-04-25T08:57:26.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.853+0000 7f23e13fd740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c872f933f0 0x55c872fcf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.853+0000 7f23e13fd740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c872fbbaa0 0x55c872fcf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.853+0000 7f23e13fd740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c872fbbe80 0x55c872fc12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.853+0000 7f23e13fd740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d81ff710 con 0x55c872f933f0 2024-04-25T08:57:26.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.853+0000 7f23e13fd740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d81ff110 con 0x55c872fbbaa0 2024-04-25T08:57:26.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.853+0000 7f23e13fd740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c872fd3950 con 0x55c872fbbe80 2024-04-25T08:57:26.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.853+0000 7f23e13fd740 10 client.0 _ll_register_callbacks cb 0x7f23c80551e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:26.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.853+0000 7f23d7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c872f933f0 0x55c872fcf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.853+0000 7f23d7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c872f933f0 0x55c872fcf310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42302/0 (socket says 172.21.15.43:42302) 2024-04-25T08:57:26.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.853+0000 7f23d7fff640 1 -- 172.21.15.43:0/3237583556 learned_addr learned my addr 172.21.15.43:0/3237583556 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:26.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.853+0000 7f23d7fff640 1 -- 172.21.15.43:0/3237583556 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c872fbbe80 msgr2=0x55c872fc12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:26.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.853+0000 7f23d7fff640 1 --2- 172.21.15.43:0/3237583556 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c872fbbe80 0x55c872fc12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.853+0000 7f23d7fff640 1 -- 172.21.15.43:0/3237583556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c872fbbaa0 msgr2=0x55c872fcf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:26.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.853+0000 7f23d7fff640 1 --2- 172.21.15.43:0/3237583556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c872fbbaa0 0x55c872fcf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.853+0000 7f23d7fff640 1 -- 172.21.15.43:0/3237583556 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c873043aa0 con 0x55c872f933f0 2024-04-25T08:57:26.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.853+0000 7f23d7fff640 1 --2- 172.21.15.43:0/3237583556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c872f933f0 0x55c872fcf310 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f23c8011910 tx=0x7f23c805c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681173630318b69c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:26.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.853+0000 7f23b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:57:26.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.853+0000 7f23b6ffd640 1 -- 172.21.15.43:0/3237583556 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c807f020 con 0x55c872f933f0 2024-04-25T08:57:26.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.853+0000 7f23b6ffd640 1 -- 172.21.15.43:0/3237583556 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f23c8082310 con 0x55c872f933f0 2024-04-25T08:57:26.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.853+0000 7f23e13fd740 1 -- 172.21.15.43:0/3237583556 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f23cc02cd60 con 0x55c872f933f0 2024-04-25T08:57:26.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.854+0000 7f23b6ffd640 1 -- 172.21.15.43:0/3237583556 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f23c8068050 con 0x55c872f933f0 2024-04-25T08:57:26.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.854+0000 7f23e13fd740 10 client.6075 fetch_fsmap learned FSMap version 22 2024-04-25T08:57:26.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.854+0000 7f23e13fd740 1 -- 172.21.15.43:0/3237583556 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x55c873043620 con 0x55c872f933f0 2024-04-25T08:57:26.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.854+0000 7f23b6ffd640 1 -- 172.21.15.43:0/3237583556 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23cc02cd60 con 0x55c872f933f0 2024-04-25T08:57:26.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.854+0000 7f23b6ffd640 1 -- 172.21.15.43:0/3237583556 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f23c806b020 con 0x55c872f933f0 2024-04-25T08:57:26.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.854+0000 7f23e13fd740 10 client.6075 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T08:57:26.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.854+0000 7f23e13fd740 10 client.6075 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T08:57:26.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.854+0000 7f23e13fd740 20 client.6075 populate_metadata read hostname 'smithi043' 2024-04-25T08:57:26.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.854+0000 7f23e13fd740 1 --2- 172.21.15.43:0/3237583556 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55c872fdd190 0x55c872fc17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.854+0000 7f23e13fd740 4 client.6075 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T08:57:26.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.854+0000 7f23e13fd740 1 -- 172.21.15.43:0/3237583556 --> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c872eff690 con 0x55c872fdd190 2024-04-25T08:57:26.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.854+0000 7f23d77fe640 1 --2- 172.21.15.43:0/3237583556 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55c872fdd190 0x55c872fc17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.854+0000 7f23b6ffd640 10 client.6075 ms_handle_connect on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:26.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.854+0000 7f23d77fe640 1 --2- 172.21.15.43:0/3237583556 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55c872fdd190 0x55c872fc17f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e265a34b740476f server_cookie=e11bba31424e9e34 in_seq=0 out_seq=0 2024-04-25T08:57:26.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.855+0000 7f23b6ffd640 1 -- 172.21.15.43:0/3237583556 <== mds.0 v2:172.21.15.193:6836/4164634380 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+168 (crc 0 0 0) 0x55c872eff690 con 0x55c872fdd190 2024-04-25T08:57:26.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.855+0000 7f23b6ffd640 10 client.6075 handle_command_reply: tid=0 2024-04-25T08:57:26.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.855+0000 7f23e13fd740 1 client.6075 shutdown 2024-04-25T08:57:26.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.855+0000 7f23e13fd740 1 -- 172.21.15.43:0/3237583556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c872f933f0 msgr2=0x55c872fcf310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:26.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.855+0000 7f23e13fd740 1 --2- 172.21.15.43:0/3237583556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c872f933f0 0x55c872fcf310 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f23c8011910 tx=0x7f23c805c000 comp rx=0 tx=0).stop 2024-04-25T08:57:26.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.855+0000 7f23e13fd740 1 -- 172.21.15.43:0/3237583556 shutdown_connections 2024-04-25T08:57:26.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.855+0000 7f23e13fd740 1 --2- 172.21.15.43:0/3237583556 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c872fbbe80 0x55c872fc12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.855+0000 7f23e13fd740 1 --2- 172.21.15.43:0/3237583556 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55c872fdd190 0x55c872fc17f0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.855+0000 7f23e13fd740 1 --2- 172.21.15.43:0/3237583556 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c872fbbaa0 0x55c872fcf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.855+0000 7f23e13fd740 1 --2- 172.21.15.43:0/3237583556 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c872f933f0 0x55c872fcf310 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.855+0000 7f23b6ffd640 0 client.6075 ms_handle_reset on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:26.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.855+0000 7f23e13fd740 1 -- 172.21.15.43:0/3237583556 >> 172.21.15.43:0/3237583556 conn(0x55c872f8e700 msgr2=0x55c872f94730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:26.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.855+0000 7f23e13fd740 1 -- 172.21.15.43:0/3237583556 shutdown_connections 2024-04-25T08:57:26.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.855+0000 7f23e13fd740 1 -- 172.21.15.43:0/3237583556 wait complete. 2024-04-25T08:57:26.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.855+0000 7f23e13fd740 20 client.6075 trim_cache size 0 max 16384 2024-04-25T08:57:26.863 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2024-04-25T08:57:26.863 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:57:26.863 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_type": "backtrace", 2024-04-25T08:57:26.863 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 1330060164, 2024-04-25T08:57:26.863 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511630412, 2024-04-25T08:57:26.863 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_0" 2024-04-25T08:57:26.863 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:57:26.863 INFO:teuthology.orchestra.run.smithi043.stdout:] 2024-04-25T08:57:26.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.856+0000 7f23dea5d640 1 -- 172.21.15.43:0/2496322065 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f23c406b500 msgr2=0x7f23c406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:26.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.856+0000 7f23dea5d640 1 --2- 172.21.15.43:0/2496322065 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f23c406b500 0x7f23c406d9c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f23cc004840 tx=0x7f23cc009040 comp rx=0 tx=0).stop 2024-04-25T08:57:26.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.856+0000 7f23dea5d640 1 -- 172.21.15.43:0/2496322065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23d81f3f80 msgr2=0x7f23d81f49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:26.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.856+0000 7f23dea5d640 1 --2- 172.21.15.43:0/2496322065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23d81f3f80 0x7f23d81f49f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f23c8011940 tx=0x7f23c8011e00 comp rx=0 tx=0).stop 2024-04-25T08:57:26.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.856+0000 7f23dea5d640 1 -- 172.21.15.43:0/2496322065 shutdown_connections 2024-04-25T08:57:26.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.856+0000 7f23dea5d640 1 --2- 172.21.15.43:0/2496322065 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f23d815e7a0 0x7f23d815ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.856+0000 7f23dea5d640 1 --2- 172.21.15.43:0/2496322065 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f23c406b500 0x7f23c406d9c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.856+0000 7f23dea5d640 1 --2- 172.21.15.43:0/2496322065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23d81606b0 0x7f23d8160b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.856+0000 7f23dea5d640 1 --2- 172.21.15.43:0/2496322065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23d81f3f80 0x7f23d81f49f0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.856+0000 7f23dea5d640 1 -- 172.21.15.43:0/2496322065 >> 172.21.15.43:0/2496322065 conn(0x7f23d80d39c0 msgr2=0x7f23d815a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:26.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.856+0000 7f23dea5d640 1 -- 172.21.15.43:0/2496322065 shutdown_connections 2024-04-25T08:57:26.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.856+0000 7f23dea5d640 1 -- 172.21.15.43:0/2496322065 wait complete. 2024-04-25T08:57:26.866 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:57:26.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.977+0000 7f588e8ac640 1 Processor -- start 2024-04-25T08:57:26.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.977+0000 7f588e8ac640 1 -- start start 2024-04-25T08:57:26.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.978+0000 7f588e8ac640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f588815ef50 0x7f58880d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.978+0000 7f588e8ac640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f58880d0920 0x7f5888160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.978+0000 7f588e8ac640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58880d0d30 con 0x7f588815dce0 2024-04-25T08:57:26.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.978+0000 7f588e8ac640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58880cec60 con 0x7f588815ef50 2024-04-25T08:57:26.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.978+0000 7f588e8ac640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58880d2920 con 0x7f58880d0920 2024-04-25T08:57:26.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.978+0000 7f58877fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f588815ef50 0x7f58880d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.978+0000 7f5887fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f588815dce0 0x7f58880ce550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49676/0 (socket says 172.21.15.43:49676) 2024-04-25T08:57:26.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.978+0000 7f5887fff640 1 -- 172.21.15.43:0/1589622225 learned_addr learned my addr 172.21.15.43:0/1589622225 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:26.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.978+0000 7f58877fe640 1 -- 172.21.15.43:0/1589622225 >> v2:172.21.15.193:3300/0 conn(0x7f58880d0920 msgr2=0x7f5888160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:26.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.978+0000 7f58877fe640 1 --2- 172.21.15.43:0/1589622225 >> v2:172.21.15.193:3300/0 conn(0x7f58880d0920 0x7f5888160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.978+0000 7f58877fe640 1 -- 172.21.15.43:0/1589622225 >> v1:172.21.15.43:6789/0 conn(0x7f588815dce0 legacy=0x7f58880ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:26.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.978+0000 7f5886ffd640 1 -- 172.21.15.43:0/1589622225 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1344607016 0 0) 0x7f587c001110 con 0x7f588815dce0 2024-04-25T08:57:26.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.979+0000 7f58877fe640 1 -- 172.21.15.43:0/1589622225 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58880cee80 con 0x7f588815ef50 2024-04-25T08:57:26.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.979+0000 7f58877fe640 1 --2- 172.21.15.43:0/1589622225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f588815ef50 0x7f58880d03e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f58880b5470 tx=0x7f587802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c663d246715f90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:26.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.979+0000 7f5886ffd640 1 -- 172.21.15.43:0/1589622225 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5878037070 con 0x7f588815ef50 2024-04-25T08:57:26.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.979+0000 7f5886ffd640 1 -- 172.21.15.43:0/1589622225 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5878032440 con 0x7f588815ef50 2024-04-25T08:57:26.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.979+0000 7f5886ffd640 1 -- 172.21.15.43:0/1589622225 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587802ae60 con 0x7f588815ef50 2024-04-25T08:57:26.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.980+0000 7f588e8ac640 1 -- 172.21.15.43:0/1589622225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f588815ef50 msgr2=0x7f58880d03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:26.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.980+0000 7f588e8ac640 1 --2- 172.21.15.43:0/1589622225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f588815ef50 0x7f58880d03e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f58880b5470 tx=0x7f587802a110 comp rx=0 tx=0).stop 2024-04-25T08:57:26.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.980+0000 7f588e8ac640 1 -- 172.21.15.43:0/1589622225 shutdown_connections 2024-04-25T08:57:26.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.980+0000 7f588e8ac640 1 --2- 172.21.15.43:0/1589622225 >> v2:172.21.15.193:3300/0 conn(0x7f58880d0920 0x7f5888160980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.980+0000 7f588e8ac640 1 --2- 172.21.15.43:0/1589622225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f588815ef50 0x7f58880d03e0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.980+0000 7f588e8ac640 1 -- 172.21.15.43:0/1589622225 >> 172.21.15.43:0/1589622225 conn(0x7f588809cd00 msgr2=0x7f58880d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:26.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.980+0000 7f588e8ac640 1 -- 172.21.15.43:0/1589622225 shutdown_connections 2024-04-25T08:57:26.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.980+0000 7f588e8ac640 1 -- 172.21.15.43:0/1589622225 wait complete. 2024-04-25T08:57:26.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.981+0000 7f588e8ac640 1 Processor -- start 2024-04-25T08:57:26.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.981+0000 7f588e8ac640 1 -- start start 2024-04-25T08:57:26.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.982+0000 7f588e8ac640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58880d0920 0x7f58881f72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.982+0000 7f588e8ac640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f588815dce0 0x7f58880cd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.982+0000 7f588e8ac640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f588815ef50 0x7f58881f43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.982+0000 7f588e8ac640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58880cf720 con 0x7f588815ef50 2024-04-25T08:57:26.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.982+0000 7f588e8ac640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58880cf440 con 0x7f58880d0920 2024-04-25T08:57:26.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.982+0000 7f588e8ac640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f588815f6b0 con 0x7f588815dce0 2024-04-25T08:57:26.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.982+0000 7f588ce49640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f588815ef50 0x7f58881f43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.982+0000 7f588ce49640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f588815ef50 0x7f58881f43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42324/0 (socket says 172.21.15.43:42324) 2024-04-25T08:57:26.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.982+0000 7f588ce49640 1 -- 172.21.15.43:0/381934396 learned_addr learned my addr 172.21.15.43:0/381934396 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:26.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.982+0000 7f5887fff640 1 --2- 172.21.15.43:0/381934396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58880d0920 0x7f58881f72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.982+0000 7f58877fe640 1 --2- 172.21.15.43:0/381934396 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f588815dce0 0x7f58880cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.982+0000 7f588ce49640 1 -- 172.21.15.43:0/381934396 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f588815dce0 msgr2=0x7f58880cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:26.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.982+0000 7f588ce49640 1 --2- 172.21.15.43:0/381934396 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f588815dce0 0x7f58880cd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.982+0000 7f588ce49640 1 -- 172.21.15.43:0/381934396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58880d0920 msgr2=0x7f58881f72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:26.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.982+0000 7f588ce49640 1 --2- 172.21.15.43:0/381934396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58880d0920 0x7f58881f72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:26.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.982+0000 7f588ce49640 1 -- 172.21.15.43:0/381934396 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58880cb770 con 0x7f588815ef50 2024-04-25T08:57:26.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.982+0000 7f5887fff640 1 --2- 172.21.15.43:0/381934396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58880d0920 0x7f58881f72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:26.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.983+0000 7f588ce49640 1 --2- 172.21.15.43:0/381934396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f588815ef50 0x7f58881f43f0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f588000b9a0 tx=0x7f588000be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:26.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.983+0000 7f5884ff9640 1 -- 172.21.15.43:0/381934396 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5880019070 con 0x7f588815ef50 2024-04-25T08:57:26.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.983+0000 7f5884ff9640 1 -- 172.21.15.43:0/381934396 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f588000cbc0 con 0x7f588815ef50 2024-04-25T08:57:26.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.983+0000 7f588e8ac640 1 -- 172.21.15.43:0/381934396 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58880cba50 con 0x7f588815ef50 2024-04-25T08:57:26.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.983+0000 7f5884ff9640 1 -- 172.21.15.43:0/381934396 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5880014440 con 0x7f588815ef50 2024-04-25T08:57:26.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.983+0000 7f588e8ac640 1 -- 172.21.15.43:0/381934396 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58880cc010 con 0x7f588815ef50 2024-04-25T08:57:26.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.984+0000 7f588e8ac640 1 -- 172.21.15.43:0/381934396 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58880680f0 con 0x7f588815ef50 2024-04-25T08:57:26.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.984+0000 7f5884ff9640 1 -- 172.21.15.43:0/381934396 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f588001c070 con 0x7f588815ef50 2024-04-25T08:57:26.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.985+0000 7f5884ff9640 1 --2- 172.21.15.43:0/381934396 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f587406b4d0 0x7f587406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:26.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.985+0000 7f5884ff9640 1 -- 172.21.15.43:0/381934396 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5880089e50 con 0x7f588815ef50 2024-04-25T08:57:26.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.985+0000 7f5887fff640 1 --2- 172.21.15.43:0/381934396 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f587406b4d0 0x7f587406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:26.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.985+0000 7f5887fff640 1 --2- 172.21.15.43:0/381934396 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f587406b4d0 0x7f587406d990 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f587c000f60 tx=0x7f587c007a10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:26.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:26.989+0000 7f5884ff9640 1 -- 172.21.15.43:0/381934396 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5880058710 con 0x7f588815ef50 2024-04-25T08:57:27.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.207+0000 7f588e8ac640 1 -- 172.21.15.43:0/381934396 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58880d01a0 con 0x7f588815ef50 2024-04-25T08:57:27.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.207+0000 7f5884ff9640 1 -- 172.21.15.43:0/381934396 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f58880d01a0 con 0x7f588815ef50 2024-04-25T08:57:27.198 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:57:27.198 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1504450503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1504450503},{"type":"v1","addr":"172.21.15.193:6835","nonce":1504450503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5691,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2657133821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2657133821},{"type":"v1","addr":"172.21.15.43:6834","nonce":2657133821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":5694,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/4271431818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":4271431818},{"type":"v1","addr":"172.21.15.43:6835","nonce":4271431818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:57:01.856132+0000","modified":"2024-04-25T08:57:22.526624+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5709":{"gid":5709,"name":"d","rank":0,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6837/4164634380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4164634380},{"type":"v1","addr":"172.21.15.193:6837","nonce":4164634380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:57:27.198 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 22 2024-04-25T08:57:27.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.209+0000 7f588e8ac640 1 -- 172.21.15.43:0/381934396 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f587406b4d0 msgr2=0x7f587406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:27.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.209+0000 7f588e8ac640 1 --2- 172.21.15.43:0/381934396 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f587406b4d0 0x7f587406d990 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f587c000f60 tx=0x7f587c007a10 comp rx=0 tx=0).stop 2024-04-25T08:57:27.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.209+0000 7f588e8ac640 1 -- 172.21.15.43:0/381934396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f588815ef50 msgr2=0x7f58881f43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:27.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.209+0000 7f588e8ac640 1 --2- 172.21.15.43:0/381934396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f588815ef50 0x7f58881f43f0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f588000b9a0 tx=0x7f588000be60 comp rx=0 tx=0).stop 2024-04-25T08:57:27.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.209+0000 7f588e8ac640 1 -- 172.21.15.43:0/381934396 shutdown_connections 2024-04-25T08:57:27.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.209+0000 7f588e8ac640 1 --2- 172.21.15.43:0/381934396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f588815ef50 0x7f58881f43f0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.209+0000 7f588e8ac640 1 --2- 172.21.15.43:0/381934396 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f588815dce0 0x7f58880cd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.209+0000 7f588e8ac640 1 --2- 172.21.15.43:0/381934396 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f587406b4d0 0x7f587406d990 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.209+0000 7f588e8ac640 1 --2- 172.21.15.43:0/381934396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58880d0920 0x7f58881f72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.210+0000 7f588e8ac640 1 -- 172.21.15.43:0/381934396 >> 172.21.15.43:0/381934396 conn(0x7f588809cd00 msgr2=0x7f58880cfa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:27.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.210+0000 7f588e8ac640 1 -- 172.21.15.43:0/381934396 shutdown_connections 2024-04-25T08:57:27.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.210+0000 7f588e8ac640 1 -- 172.21.15.43:0/381934396 wait complete. 2024-04-25T08:57:27.210 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d damage rm 1330060164 2024-04-25T08:57:27.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.324+0000 7fbf36728640 1 Processor -- start 2024-04-25T08:57:27.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.324+0000 7fbf36728640 1 -- start start 2024-04-25T08:57:27.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.324+0000 7fbf36728640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf3015a750 0x7fbf300cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:27.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.324+0000 7fbf36728640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fbf300d12f0 0x7fbf3006d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:27.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.324+0000 7fbf36728640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf300ce7e0 con 0x7fbf30071fa0 2024-04-25T08:57:27.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.324+0000 7fbf36728640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf300b4350 con 0x7fbf300d12f0 2024-04-25T08:57:27.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.324+0000 7fbf36728640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf300afda0 con 0x7fbf3015a750 2024-04-25T08:57:27.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.325+0000 7fbf2f7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fbf30071fa0 0x7fbf300d0be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49688/0 (socket says 172.21.15.43:49688) 2024-04-25T08:57:27.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.325+0000 7fbf2f7fe640 1 -- 172.21.15.43:0/1053633976 learned_addr learned my addr 172.21.15.43:0/1053633976 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:27.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.325+0000 7fbf34cc5640 1 --2- 172.21.15.43:0/1053633976 >> v2:172.21.15.193:3300/0 conn(0x7fbf300d12f0 0x7fbf3006d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:27.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.325+0000 7fbf2ffff640 1 --2- 172.21.15.43:0/1053633976 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf3015a750 0x7fbf300cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:27.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.325+0000 7fbf2effd640 1 -- 172.21.15.43:0/1053633976 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1592640068 0 0) 0x7fbf300ce7e0 con 0x7fbf30071fa0 2024-04-25T08:57:27.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.325+0000 7fbf2effd640 1 -- 172.21.15.43:0/1053633976 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf18003520 con 0x7fbf30071fa0 2024-04-25T08:57:27.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.325+0000 7fbf2effd640 1 -- 172.21.15.43:0/1053633976 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1865735601 0 0) 0x7fbf18003520 con 0x7fbf30071fa0 2024-04-25T08:57:27.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.325+0000 7fbf2ffff640 1 -- 172.21.15.43:0/1053633976 >> v2:172.21.15.193:3300/0 conn(0x7fbf300d12f0 msgr2=0x7fbf3006d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:27.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.325+0000 7fbf2ffff640 1 --2- 172.21.15.43:0/1053633976 >> v2:172.21.15.193:3300/0 conn(0x7fbf300d12f0 0x7fbf3006d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.325+0000 7fbf2ffff640 1 -- 172.21.15.43:0/1053633976 >> v1:172.21.15.43:6789/0 conn(0x7fbf30071fa0 legacy=0x7fbf300d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:27.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.325+0000 7fbf2ffff640 1 -- 172.21.15.43:0/1053633976 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf300afff0 con 0x7fbf3015a750 2024-04-25T08:57:27.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.325+0000 7fbf34cc5640 1 --2- 172.21.15.43:0/1053633976 >> v2:172.21.15.193:3300/0 conn(0x7fbf300d12f0 0x7fbf3006d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:57:27.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.326+0000 7fbf2ffff640 1 --2- 172.21.15.43:0/1053633976 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf3015a750 0x7fbf300cf580 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbf200097a0 tx=0x7fbf2002a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69736aaf8af659a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:27.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.326+0000 7fbf2effd640 1 -- 172.21.15.43:0/1053633976 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf20037070 con 0x7fbf3015a750 2024-04-25T08:57:27.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.326+0000 7fbf2effd640 1 -- 172.21.15.43:0/1053633976 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf2002aba0 con 0x7fbf3015a750 2024-04-25T08:57:27.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.326+0000 7fbf2effd640 1 -- 172.21.15.43:0/1053633976 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf24001670 con 0x7fbf3015a750 2024-04-25T08:57:27.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.327+0000 7fbf36728640 1 -- 172.21.15.43:0/1053633976 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf3015a750 msgr2=0x7fbf300cf580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:27.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.327+0000 7fbf36728640 1 --2- 172.21.15.43:0/1053633976 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf3015a750 0x7fbf300cf580 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbf200097a0 tx=0x7fbf2002a110 comp rx=0 tx=0).stop 2024-04-25T08:57:27.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.327+0000 7fbf36728640 1 -- 172.21.15.43:0/1053633976 shutdown_connections 2024-04-25T08:57:27.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.327+0000 7fbf36728640 1 --2- 172.21.15.43:0/1053633976 >> v2:172.21.15.193:3300/0 conn(0x7fbf300d12f0 0x7fbf3006d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.327+0000 7fbf36728640 1 --2- 172.21.15.43:0/1053633976 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf3015a750 0x7fbf300cf580 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.327+0000 7fbf36728640 1 -- 172.21.15.43:0/1053633976 >> 172.21.15.43:0/1053633976 conn(0x7fbf3006c060 msgr2=0x7fbf3006c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:27.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.327+0000 7fbf36728640 1 -- 172.21.15.43:0/1053633976 shutdown_connections 2024-04-25T08:57:27.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.327+0000 7fbf36728640 1 -- 172.21.15.43:0/1053633976 wait complete. 2024-04-25T08:57:27.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.328+0000 7fbf36728640 1 Processor -- start 2024-04-25T08:57:27.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.328+0000 7fbf36728640 1 -- start start 2024-04-25T08:57:27.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.329+0000 7fbf36728640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf30071fa0 0x7fbf3006e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:27.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.329+0000 7fbf36728640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf300d12f0 0x7fbf301554a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:27.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.329+0000 7fbf36728640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf3015a750 0x7fbf301f3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:27.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.329+0000 7fbf36728640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf300ced80 con 0x7fbf300d12f0 2024-04-25T08:57:27.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.329+0000 7fbf36728640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf300cf0f0 con 0x7fbf3015a750 2024-04-25T08:57:27.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.329+0000 7fbf36728640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3015ab30 con 0x7fbf30071fa0 2024-04-25T08:57:27.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.329+0000 7fbf2f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf300d12f0 0x7fbf301554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:27.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.329+0000 7fbf2f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf300d12f0 0x7fbf301554a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42338/0 (socket says 172.21.15.43:42338) 2024-04-25T08:57:27.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.329+0000 7fbf2f7fe640 1 -- 172.21.15.43:0/714337674 learned_addr learned my addr 172.21.15.43:0/714337674 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:27.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.329+0000 7fbf2f7fe640 1 -- 172.21.15.43:0/714337674 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf30071fa0 msgr2=0x7fbf3006e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:27.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.329+0000 7fbf2f7fe640 1 --2- 172.21.15.43:0/714337674 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf30071fa0 0x7fbf3006e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.329+0000 7fbf2f7fe640 1 -- 172.21.15.43:0/714337674 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf3015a750 msgr2=0x7fbf301f3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:27.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.329+0000 7fbf34cc5640 1 --2- 172.21.15.43:0/714337674 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf3015a750 0x7fbf301f3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:27.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.329+0000 7fbf2f7fe640 1 --2- 172.21.15.43:0/714337674 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf3015a750 0x7fbf301f3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.329+0000 7fbf2f7fe640 1 -- 172.21.15.43:0/714337674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf3006d9e0 con 0x7fbf300d12f0 2024-04-25T08:57:27.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.330+0000 7fbf2f7fe640 1 --2- 172.21.15.43:0/714337674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf300d12f0 0x7fbf301554a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fbf2400c750 tx=0x7fbf2400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:27.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.330+0000 7fbf2cff9640 1 -- 172.21.15.43:0/714337674 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf24017070 con 0x7fbf300d12f0 2024-04-25T08:57:27.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.330+0000 7fbf2cff9640 1 -- 172.21.15.43:0/714337674 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf24005ad0 con 0x7fbf300d12f0 2024-04-25T08:57:27.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.330+0000 7fbf36728640 1 -- 172.21.15.43:0/714337674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf3006dcc0 con 0x7fbf300d12f0 2024-04-25T08:57:27.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.330+0000 7fbf2cff9640 1 -- 172.21.15.43:0/714337674 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf24007070 con 0x7fbf300d12f0 2024-04-25T08:57:27.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.330+0000 7fbf36728640 1 -- 172.21.15.43:0/714337674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf300adaf0 con 0x7fbf300d12f0 2024-04-25T08:57:27.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.331+0000 7fbf2cff9640 1 -- 172.21.15.43:0/714337674 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbf2401a070 con 0x7fbf300d12f0 2024-04-25T08:57:27.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.332+0000 7fbf2cff9640 1 --2- 172.21.15.43:0/714337674 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbf1c06b4d0 0x7fbf1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:27.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.332+0000 7fbf2cff9640 1 -- 172.21.15.43:0/714337674 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbf24089590 con 0x7fbf300d12f0 2024-04-25T08:57:27.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.332+0000 7fbf2ffff640 1 --2- 172.21.15.43:0/714337674 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbf1c06b4d0 0x7fbf1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:27.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.332+0000 7fbf2ffff640 1 --2- 172.21.15.43:0/714337674 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbf1c06b4d0 0x7fbf1c06d990 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbf20034000 tx=0x7fbf2002c000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:27.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.333+0000 7fbf39420740 1 Processor -- start 2024-04-25T08:57:27.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.333+0000 7fbf39420740 1 -- start start 2024-04-25T08:57:27.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.333+0000 7fbf39420740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55e3df3f8f80 0x55e3df3fb790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:27.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.333+0000 7fbf39420740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3df3f9360 0x55e3df3fbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:27.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.333+0000 7fbf39420740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3df3cbbe0 0x55e3df3cf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:27.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.333+0000 7fbf39420740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3df334ac0 con 0x55e3df3cbbe0 2024-04-25T08:57:27.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.333+0000 7fbf39420740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3df2dddb0 con 0x55e3df3f9360 2024-04-25T08:57:27.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.333+0000 7fbf39420740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e3df330b60 con 0x55e3df3f8f80 2024-04-25T08:57:27.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.334+0000 7fbf34cc5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3df3cbbe0 0x55e3df3cf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:27.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.334+0000 7fbf34cc5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3df3cbbe0 0x55e3df3cf3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42342/0 (socket says 172.21.15.43:42342) 2024-04-25T08:57:27.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.334+0000 7fbf34cc5640 1 -- 172.21.15.43:0/723008868 learned_addr learned my addr 172.21.15.43:0/723008868 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:27.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.334+0000 7fbf2ffff640 1 --2- 172.21.15.43:0/723008868 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55e3df3f8f80 0x55e3df3fb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:27.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.334+0000 7fbf2f7fe640 1 --2- 172.21.15.43:0/723008868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3df3f9360 0x55e3df3fbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:27.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.334+0000 7fbf34cc5640 1 -- 172.21.15.43:0/723008868 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55e3df3f8f80 msgr2=0x55e3df3fb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:27.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.334+0000 7fbf34cc5640 1 --2- 172.21.15.43:0/723008868 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55e3df3f8f80 0x55e3df3fb790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.334+0000 7fbf34cc5640 1 -- 172.21.15.43:0/723008868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3df3f9360 msgr2=0x55e3df3fbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:27.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.334+0000 7fbf34cc5640 1 --2- 172.21.15.43:0/723008868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3df3f9360 0x55e3df3fbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.334+0000 7fbf34cc5640 1 -- 172.21.15.43:0/723008868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3df115710 con 0x55e3df3cbbe0 2024-04-25T08:57:27.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.334+0000 7fbf2f7fe640 1 --2- 172.21.15.43:0/723008868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3df3f9360 0x55e3df3fbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:27.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.334+0000 7fbf34cc5640 1 --2- 172.21.15.43:0/723008868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3df3cbbe0 0x55e3df3cf3f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fbf28007430 tx=0x7fbf28009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d9cee059d2ba8d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:27.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.334+0000 7fbf15ffb640 1 -- 172.21.15.43:0/723008868 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf280147e0 con 0x55e3df3cbbe0 2024-04-25T08:57:27.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.334+0000 7fbf15ffb640 1 -- 172.21.15.43:0/723008868 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf2801a070 con 0x55e3df3cbbe0 2024-04-25T08:57:27.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.334+0000 7fbf15ffb640 1 -- 172.21.15.43:0/723008868 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf28014ba0 con 0x55e3df3cbbe0 2024-04-25T08:57:27.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.335+0000 7fbf39420740 1 -- 172.21.15.43:0/723008868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3df3cbbe0 msgr2=0x55e3df3cf3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:27.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.335+0000 7fbf39420740 1 --2- 172.21.15.43:0/723008868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3df3cbbe0 0x55e3df3cf3f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fbf28007430 tx=0x7fbf28009750 comp rx=0 tx=0).stop 2024-04-25T08:57:27.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.335+0000 7fbf39420740 1 -- 172.21.15.43:0/723008868 shutdown_connections 2024-04-25T08:57:27.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.335+0000 7fbf39420740 1 --2- 172.21.15.43:0/723008868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3df3cbbe0 0x55e3df3cf3f0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.335+0000 7fbf39420740 1 --2- 172.21.15.43:0/723008868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3df3f9360 0x55e3df3fbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.335+0000 7fbf39420740 1 --2- 172.21.15.43:0/723008868 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55e3df3f8f80 0x55e3df3fb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.335+0000 7fbf39420740 1 -- 172.21.15.43:0/723008868 >> 172.21.15.43:0/723008868 conn(0x55e3df3cde50 msgr2=0x55e3df416790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:27.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.335+0000 7fbf39420740 1 -- 172.21.15.43:0/723008868 shutdown_connections 2024-04-25T08:57:27.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.335+0000 7fbf39420740 1 -- 172.21.15.43:0/723008868 wait complete. 2024-04-25T08:57:27.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.335+0000 7fbf39420740 1 -- start start 2024-04-25T08:57:27.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.336+0000 7fbf39420740 1 Processor -- start 2024-04-25T08:57:27.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.336+0000 7fbf39420740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3df3f8f80 0x55e3df3f3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:27.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.336+0000 7fbf39420740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3df3f9360 0x55e3df418cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:27.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.336+0000 7fbf39420740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55e3df3f5960 0x55e3df3f6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:27.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.336+0000 7fbf39420740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3df3d1000 con 0x55e3df3f9360 2024-04-25T08:57:27.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.336+0000 7fbf39420740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3df2d34e0 con 0x55e3df3f8f80 2024-04-25T08:57:27.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.336+0000 7fbf39420740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e3df3d12e0 con 0x55e3df3f5960 2024-04-25T08:57:27.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.336+0000 7fbf39420740 10 client.0 _ll_register_callbacks cb 0x55e3df048af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:27.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.336+0000 7fbf2f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3df3f9360 0x55e3df418cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:27.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.337+0000 7fbf2f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3df3f9360 0x55e3df418cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42358/0 (socket says 172.21.15.43:42358) 2024-04-25T08:57:27.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.337+0000 7fbf2f7fe640 1 -- 172.21.15.43:0/785482942 learned_addr learned my addr 172.21.15.43:0/785482942 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:27.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.337+0000 7fbf2ffff640 1 --2- 172.21.15.43:0/785482942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3df3f8f80 0x55e3df3f3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:27.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.337+0000 7fbf34cc5640 1 --2- 172.21.15.43:0/785482942 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55e3df3f5960 0x55e3df3f6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:27.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.337+0000 7fbf2f7fe640 1 -- 172.21.15.43:0/785482942 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55e3df3f5960 msgr2=0x55e3df3f6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:27.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.337+0000 7fbf2f7fe640 1 --2- 172.21.15.43:0/785482942 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55e3df3f5960 0x55e3df3f6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.337+0000 7fbf2f7fe640 1 -- 172.21.15.43:0/785482942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3df3f8f80 msgr2=0x55e3df3f3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:27.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.337+0000 7fbf2f7fe640 1 --2- 172.21.15.43:0/785482942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3df3f8f80 0x55e3df3f3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.337+0000 7fbf2f7fe640 1 -- 172.21.15.43:0/785482942 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e3df175ea0 con 0x55e3df3f9360 2024-04-25T08:57:27.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.337+0000 7fbf2ffff640 1 --2- 172.21.15.43:0/785482942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3df3f8f80 0x55e3df3f3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:27.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.337+0000 7fbf34cc5640 1 --2- 172.21.15.43:0/785482942 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55e3df3f5960 0x55e3df3f6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:27.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.337+0000 7fbf2f7fe640 1 --2- 172.21.15.43:0/785482942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3df3f9360 0x55e3df418cb0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fbf24087af0 tx=0x7fbf2405bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6117000955ac496f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:27.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.337+0000 7fbf0effd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:57:27.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.337+0000 7fbf0effd640 1 -- 172.21.15.43:0/785482942 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf24086060 con 0x55e3df3f9360 2024-04-25T08:57:27.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.337+0000 7fbf0effd640 1 -- 172.21.15.43:0/785482942 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbf24060220 con 0x55e3df3f9360 2024-04-25T08:57:27.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.337+0000 7fbf39420740 1 -- 172.21.15.43:0/785482942 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf2801a070 con 0x55e3df3f9360 2024-04-25T08:57:27.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.337+0000 7fbf0effd640 1 -- 172.21.15.43:0/785482942 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf24074050 con 0x55e3df3f9360 2024-04-25T08:57:27.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.337+0000 7fbf39420740 10 client.6096 fetch_fsmap learned FSMap version 22 2024-04-25T08:57:27.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.337+0000 7fbf39420740 1 -- 172.21.15.43:0/785482942 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x55e3df009000 con 0x55e3df3f9360 2024-04-25T08:57:27.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.337+0000 7fbf0effd640 1 -- 172.21.15.43:0/785482942 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf2801a070 con 0x55e3df3f9360 2024-04-25T08:57:27.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.337+0000 7fbf0effd640 1 -- 172.21.15.43:0/785482942 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbf240890c0 con 0x55e3df3f9360 2024-04-25T08:57:27.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.338+0000 7fbf39420740 10 client.6096 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T08:57:27.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.338+0000 7fbf39420740 10 client.6096 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T08:57:27.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.338+0000 7fbf39420740 20 client.6096 populate_metadata read hostname 'smithi043' 2024-04-25T08:57:27.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.338+0000 7fbf39420740 1 --2- 172.21.15.43:0/785482942 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55e3df3f5d80 0x55e3df3f4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:27.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.338+0000 7fbf39420740 4 client.6096 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:57:27.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.338+0000 7fbf39420740 1 -- 172.21.15.43:0/785482942 --> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e3df342000 con 0x55e3df3f5d80 2024-04-25T08:57:27.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.338+0000 7fbf2ffff640 1 --2- 172.21.15.43:0/785482942 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55e3df3f5d80 0x55e3df3f4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:27.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.338+0000 7fbf0effd640 10 client.6096 ms_handle_connect on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:27.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.338+0000 7fbf2ffff640 1 --2- 172.21.15.43:0/785482942 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55e3df3f5d80 0x55e3df3f4560 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=370ea50468af04e server_cookie=a8aac776a57205f4 in_seq=0 out_seq=0 2024-04-25T08:57:27.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.339+0000 7fbf0effd640 1 -- 172.21.15.43:0/785482942 <== mds.0 v2:172.21.15.193:6836/4164634380 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e3df342000 con 0x55e3df3f5d80 2024-04-25T08:57:27.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.339+0000 7fbf0effd640 10 client.6096 handle_command_reply: tid=0 2024-04-25T08:57:27.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.339+0000 7fbf39420740 1 client.6096 shutdown 2024-04-25T08:57:27.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.339+0000 7fbf39420740 1 -- 172.21.15.43:0/785482942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3df3f9360 msgr2=0x55e3df418cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:27.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.339+0000 7fbf39420740 1 --2- 172.21.15.43:0/785482942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3df3f9360 0x55e3df418cb0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fbf24087af0 tx=0x7fbf2405bfb0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.339+0000 7fbf39420740 1 -- 172.21.15.43:0/785482942 shutdown_connections 2024-04-25T08:57:27.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.339+0000 7fbf39420740 1 --2- 172.21.15.43:0/785482942 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55e3df3f5960 0x55e3df3f6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.339+0000 7fbf39420740 1 --2- 172.21.15.43:0/785482942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3df3f9360 0x55e3df418cb0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.339+0000 7fbf39420740 1 --2- 172.21.15.43:0/785482942 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55e3df3f5d80 0x55e3df3f4560 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.339+0000 7fbf39420740 1 --2- 172.21.15.43:0/785482942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3df3f8f80 0x55e3df3f3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.339+0000 7fbf39420740 1 -- 172.21.15.43:0/785482942 >> 172.21.15.43:0/785482942 conn(0x55e3df3cbbe0 msgr2=0x55e3df3f8410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:27.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.339+0000 7fbf0effd640 0 client.6096 ms_handle_reset on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:27.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.339+0000 7fbf39420740 1 -- 172.21.15.43:0/785482942 shutdown_connections 2024-04-25T08:57:27.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.339+0000 7fbf39420740 1 -- 172.21.15.43:0/785482942 wait complete. 2024-04-25T08:57:27.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.339+0000 7fbf39420740 20 client.6096 trim_cache size 0 max 16384 2024-04-25T08:57:27.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.351+0000 7fbf39420740 1 Processor -- start 2024-04-25T08:57:27.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.351+0000 7fbf39420740 1 -- start start 2024-04-25T08:57:27.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.352+0000 7fbf39420740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3df3f5960 0x55e3df3cac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:27.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.352+0000 7fbf39420740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55e3df3f8f80 0x55e3df3cb170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:27.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.352+0000 7fbf39420740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3df3f9360 0x55e3df3f18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:27.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.352+0000 7fbf39420740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf301fedf0 con 0x55e3df3f9360 2024-04-25T08:57:27.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.352+0000 7fbf39420740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf301f6e60 con 0x55e3df3f5960 2024-04-25T08:57:27.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.352+0000 7fbf39420740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf301f7460 con 0x55e3df3f8f80 2024-04-25T08:57:27.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.352+0000 7fbf34cc5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3df3f9360 0x55e3df3f18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:27.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.352+0000 7fbf34cc5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3df3f9360 0x55e3df3f18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42360/0 (socket says 172.21.15.43:42360) 2024-04-25T08:57:27.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.352+0000 7fbf34cc5640 1 -- 172.21.15.43:0/1959706341 learned_addr learned my addr 172.21.15.43:0/1959706341 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:27.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.352+0000 7fbf2f7fe640 1 --2- 172.21.15.43:0/1959706341 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55e3df3f8f80 0x55e3df3cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:27.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.352+0000 7fbf2ffff640 1 --2- 172.21.15.43:0/1959706341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3df3f5960 0x55e3df3cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:27.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.352+0000 7fbf34cc5640 1 -- 172.21.15.43:0/1959706341 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55e3df3f8f80 msgr2=0x55e3df3cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:27.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.352+0000 7fbf34cc5640 1 --2- 172.21.15.43:0/1959706341 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55e3df3f8f80 0x55e3df3cb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.352+0000 7fbf34cc5640 1 -- 172.21.15.43:0/1959706341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3df3f5960 msgr2=0x55e3df3cac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:27.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.352+0000 7fbf34cc5640 1 --2- 172.21.15.43:0/1959706341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3df3f5960 0x55e3df3cac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.352+0000 7fbf34cc5640 1 -- 172.21.15.43:0/1959706341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e3df3cd6f0 con 0x55e3df3f9360 2024-04-25T08:57:27.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.353+0000 7fbf34cc5640 1 --2- 172.21.15.43:0/1959706341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3df3f9360 0x55e3df3f18d0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fbf28017040 tx=0x7fbf28014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=230141c3c304b437 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:27.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.353+0000 7fbf157fa640 1 -- 172.21.15.43:0/1959706341 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2802d070 con 0x55e3df3f9360 2024-04-25T08:57:27.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.353+0000 7fbf157fa640 1 -- 172.21.15.43:0/1959706341 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf2801a040 con 0x55e3df3f9360 2024-04-25T08:57:27.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.353+0000 7fbf157fa640 1 -- 172.21.15.43:0/1959706341 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf28014880 con 0x55e3df3f9360 2024-04-25T08:57:27.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.353+0000 7fbf39420740 1 -- 172.21.15.43:0/1959706341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3df3f9360 msgr2=0x55e3df3f18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:27.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.353+0000 7fbf39420740 1 --2- 172.21.15.43:0/1959706341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3df3f9360 0x55e3df3f18d0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fbf28017040 tx=0x7fbf28014000 comp rx=0 tx=0).stop 2024-04-25T08:57:27.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.353+0000 7fbf39420740 1 -- 172.21.15.43:0/1959706341 shutdown_connections 2024-04-25T08:57:27.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.353+0000 7fbf39420740 1 --2- 172.21.15.43:0/1959706341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3df3f9360 0x55e3df3f18d0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.353+0000 7fbf39420740 1 --2- 172.21.15.43:0/1959706341 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55e3df3f8f80 0x55e3df3cb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.353+0000 7fbf39420740 1 --2- 172.21.15.43:0/1959706341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3df3f5960 0x55e3df3cac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.353+0000 7fbf39420740 1 -- 172.21.15.43:0/1959706341 >> 172.21.15.43:0/1959706341 conn(0x55e3df3cbbe0 msgr2=0x55e3df3d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:27.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.353+0000 7fbf39420740 1 -- 172.21.15.43:0/1959706341 shutdown_connections 2024-04-25T08:57:27.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.353+0000 7fbf39420740 1 -- 172.21.15.43:0/1959706341 wait complete. 2024-04-25T08:57:27.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.354+0000 7fbf39420740 1 -- start start 2024-04-25T08:57:27.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.354+0000 7fbf39420740 1 Processor -- start 2024-04-25T08:57:27.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.355+0000 7fbf39420740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3df3f5960 0x55e3df4809e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:27.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.355+0000 7fbf39420740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3df3f8f80 0x55e3df3fe010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:27.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.355+0000 7fbf39420740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55e3df3f9360 0x55e3df3fe550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:27.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.355+0000 7fbf39420740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3df4187e0 con 0x55e3df3f5960 2024-04-25T08:57:27.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.355+0000 7fbf39420740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e3df418290 con 0x55e3df3f8f80 2024-04-25T08:57:27.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.355+0000 7fbf39420740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e3df3fd470 con 0x55e3df3f9360 2024-04-25T08:57:27.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.355+0000 7fbf39420740 10 client.0 _ll_register_callbacks cb 0x7fbf28009ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:27.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.355+0000 7fbf2ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3df3f5960 0x55e3df4809e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:27.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.355+0000 7fbf2ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3df3f5960 0x55e3df4809e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42370/0 (socket says 172.21.15.43:42370) 2024-04-25T08:57:27.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.355+0000 7fbf2ffff640 1 -- 172.21.15.43:0/509119054 learned_addr learned my addr 172.21.15.43:0/509119054 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:27.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.355+0000 7fbf2f7fe640 1 --2- 172.21.15.43:0/509119054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3df3f8f80 0x55e3df3fe010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:27.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.355+0000 7fbf34cc5640 1 --2- 172.21.15.43:0/509119054 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55e3df3f9360 0x55e3df3fe550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:27.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.355+0000 7fbf2ffff640 1 -- 172.21.15.43:0/509119054 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55e3df3f9360 msgr2=0x55e3df3fe550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:27.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.355+0000 7fbf2ffff640 1 --2- 172.21.15.43:0/509119054 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55e3df3f9360 0x55e3df3fe550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.355+0000 7fbf2ffff640 1 -- 172.21.15.43:0/509119054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3df3f8f80 msgr2=0x55e3df3fe010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:27.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.355+0000 7fbf2ffff640 1 --2- 172.21.15.43:0/509119054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3df3f8f80 0x55e3df3fe010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.355+0000 7fbf2ffff640 1 -- 172.21.15.43:0/509119054 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e3df3f85a0 con 0x55e3df3f5960 2024-04-25T08:57:27.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.355+0000 7fbf2f7fe640 1 --2- 172.21.15.43:0/509119054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3df3f8f80 0x55e3df3fe010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:27.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.355+0000 7fbf34cc5640 1 --2- 172.21.15.43:0/509119054 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55e3df3f9360 0x55e3df3fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:27.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.355+0000 7fbf2ffff640 1 --2- 172.21.15.43:0/509119054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3df3f5960 0x55e3df4809e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fbf2002a700 tx=0x7fbf20004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e91ed46a2b06ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:27.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.355+0000 7fbf0effd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:57:27.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.356+0000 7fbf0effd640 1 -- 172.21.15.43:0/509119054 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf20037040 con 0x55e3df3f5960 2024-04-25T08:57:27.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.356+0000 7fbf0effd640 1 -- 172.21.15.43:0/509119054 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbf200501b0 con 0x55e3df3f5960 2024-04-25T08:57:27.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.356+0000 7fbf39420740 1 -- 172.21.15.43:0/509119054 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf2801a040 con 0x55e3df3f5960 2024-04-25T08:57:27.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.356+0000 7fbf0effd640 1 -- 172.21.15.43:0/509119054 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf20061050 con 0x55e3df3f5960 2024-04-25T08:57:27.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.356+0000 7fbf39420740 10 client.6102 fetch_fsmap learned FSMap version 22 2024-04-25T08:57:27.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.356+0000 7fbf39420740 1 -- 172.21.15.43:0/509119054 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x55e3df40cf20 con 0x55e3df3f5960 2024-04-25T08:57:27.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.356+0000 7fbf0effd640 1 -- 172.21.15.43:0/509119054 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf2801a040 con 0x55e3df3f5960 2024-04-25T08:57:27.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.356+0000 7fbf0effd640 1 -- 172.21.15.43:0/509119054 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbf20064020 con 0x55e3df3f5960 2024-04-25T08:57:27.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.356+0000 7fbf39420740 10 client.6102 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T08:57:27.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.356+0000 7fbf39420740 10 client.6102 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T08:57:27.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.356+0000 7fbf39420740 20 client.6102 populate_metadata read hostname 'smithi043' 2024-04-25T08:57:27.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.356+0000 7fbf39420740 1 --2- 172.21.15.43:0/509119054 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55e3df3cb680 0x55e3df419e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:27.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.356+0000 7fbf39420740 4 client.6102 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 1330060164}] 2024-04-25T08:57:27.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.356+0000 7fbf39420740 1 -- 172.21.15.43:0/509119054 --> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] -- command(tid 0: {"prefix": "damage rm", "damage_id": 1330060164}) v1 -- 0x55e3df33c350 con 0x55e3df3cb680 2024-04-25T08:57:27.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.356+0000 7fbf2f7fe640 1 --2- 172.21.15.43:0/509119054 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55e3df3cb680 0x55e3df419e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:27.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.357+0000 7fbf0effd640 10 client.6102 ms_handle_connect on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:27.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.357+0000 7fbf2f7fe640 1 --2- 172.21.15.43:0/509119054 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55e3df3cb680 0x55e3df419e30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e21bd5bf1a704047 server_cookie=da09ade1facadcd in_seq=0 out_seq=0 2024-04-25T08:57:27.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.357+0000 7fbf0effd640 1 -- 172.21.15.43:0/509119054 <== mds.0 v2:172.21.15.193:6836/4164634380 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x55e3df33c350 con 0x55e3df3cb680 2024-04-25T08:57:27.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.357+0000 7fbf0effd640 10 client.6102 handle_command_reply: tid=0 2024-04-25T08:57:27.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.357+0000 7fbf39420740 1 client.6102 shutdown 2024-04-25T08:57:27.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.357+0000 7fbf39420740 1 -- 172.21.15.43:0/509119054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3df3f5960 msgr2=0x55e3df4809e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:27.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.357+0000 7fbf39420740 1 --2- 172.21.15.43:0/509119054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3df3f5960 0x55e3df4809e0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fbf2002a700 tx=0x7fbf20004730 comp rx=0 tx=0).stop 2024-04-25T08:57:27.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.357+0000 7fbf39420740 1 -- 172.21.15.43:0/509119054 shutdown_connections 2024-04-25T08:57:27.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.357+0000 7fbf39420740 1 --2- 172.21.15.43:0/509119054 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55e3df3f9360 0x55e3df3fe550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.357+0000 7fbf39420740 1 --2- 172.21.15.43:0/509119054 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55e3df3cb680 0x55e3df419e30 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.357+0000 7fbf39420740 1 --2- 172.21.15.43:0/509119054 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e3df3f8f80 0x55e3df3fe010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.357+0000 7fbf39420740 1 --2- 172.21.15.43:0/509119054 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e3df3f5960 0x55e3df4809e0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.357+0000 7fbf0effd640 0 client.6102 ms_handle_reset on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:27.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.357+0000 7fbf39420740 1 -- 172.21.15.43:0/509119054 >> 172.21.15.43:0/509119054 conn(0x55e3df3cbbe0 msgr2=0x55e3df3fc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:27.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.357+0000 7fbf39420740 1 -- 172.21.15.43:0/509119054 shutdown_connections 2024-04-25T08:57:27.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.357+0000 7fbf39420740 1 -- 172.21.15.43:0/509119054 wait complete. 2024-04-25T08:57:27.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.357+0000 7fbf39420740 20 client.6102 trim_cache size 0 max 16384 2024-04-25T08:57:27.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.358+0000 7fbf36728640 1 -- 172.21.15.43:0/714337674 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbf1c06b4d0 msgr2=0x7fbf1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:27.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.358+0000 7fbf36728640 1 --2- 172.21.15.43:0/714337674 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbf1c06b4d0 0x7fbf1c06d990 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbf20034000 tx=0x7fbf2002c000 comp rx=0 tx=0).stop 2024-04-25T08:57:27.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.358+0000 7fbf36728640 1 -- 172.21.15.43:0/714337674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf300d12f0 msgr2=0x7fbf301554a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:27.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.358+0000 7fbf36728640 1 --2- 172.21.15.43:0/714337674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf300d12f0 0x7fbf301554a0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fbf2400c750 tx=0x7fbf2400cc10 comp rx=0 tx=0).stop 2024-04-25T08:57:27.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.358+0000 7fbf36728640 1 -- 172.21.15.43:0/714337674 shutdown_connections 2024-04-25T08:57:27.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.358+0000 7fbf36728640 1 --2- 172.21.15.43:0/714337674 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbf1c06b4d0 0x7fbf1c06d990 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.358+0000 7fbf36728640 1 --2- 172.21.15.43:0/714337674 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf3015a750 0x7fbf301f3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.358+0000 7fbf36728640 1 --2- 172.21.15.43:0/714337674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf300d12f0 0x7fbf301554a0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.358+0000 7fbf36728640 1 --2- 172.21.15.43:0/714337674 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf30071fa0 0x7fbf3006e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.358+0000 7fbf36728640 1 -- 172.21.15.43:0/714337674 >> 172.21.15.43:0/714337674 conn(0x7fbf3006c060 msgr2=0x7fbf300cfc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:27.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.358+0000 7fbf36728640 1 -- 172.21.15.43:0/714337674 shutdown_connections 2024-04-25T08:57:27.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.358+0000 7fbf36728640 1 -- 172.21.15.43:0/714337674 wait complete. 2024-04-25T08:57:27.358 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2024-04-25T08:57:27.358 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:57:27.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.469+0000 7f6ad0c3a640 1 Processor -- start 2024-04-25T08:57:27.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.469+0000 7f6ad0c3a640 1 -- start start 2024-04-25T08:57:27.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.470+0000 7f6ad0c3a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6acc15df50 0x7f6acc1619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:27.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.470+0000 7f6ad0c3a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6acc0d24a0 con 0x7f6acc161f30 2024-04-25T08:57:27.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.470+0000 7f6ad0c3a640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6acc0d2740 con 0x7f6acc0cf670 2024-04-25T08:57:27.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.470+0000 7f6ad0c3a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc0cfa80 con 0x7f6acc15df50 2024-04-25T08:57:27.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.470+0000 7f6ac9d9b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f6acc161f30 0x7f6acc0c7cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49696/0 (socket says 172.21.15.43:49696) 2024-04-25T08:57:27.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.470+0000 7f6ac9d9b640 1 -- 172.21.15.43:0/129287270 learned_addr learned my addr 172.21.15.43:0/129287270 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:27.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.471+0000 7f6aca59c640 1 --2- 172.21.15.43:0/129287270 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6acc15df50 0x7f6acc1619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:27.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.471+0000 7f6ac959a640 1 -- 172.21.15.43:0/129287270 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 938537509 0 0) 0x7f6acc0d24a0 con 0x7f6acc161f30 2024-04-25T08:57:27.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.471+0000 7f6ac959a640 1 -- 172.21.15.43:0/129287270 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ab8003520 con 0x7f6acc161f30 2024-04-25T08:57:27.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.471+0000 7f6ac959a640 1 -- 172.21.15.43:0/129287270 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3663616462 0 0) 0x7f6acc0d2740 con 0x7f6acc0cf670 2024-04-25T08:57:27.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.471+0000 7f6ac959a640 1 -- 172.21.15.43:0/129287270 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6acc0d24a0 con 0x7f6acc0cf670 2024-04-25T08:57:27.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.471+0000 7f6aca59c640 1 -- 172.21.15.43:0/129287270 >> v1:172.21.15.193:6789/0 conn(0x7f6acc0cf670 legacy=0x7f6acc0c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:27.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.471+0000 7f6aca59c640 1 -- 172.21.15.43:0/129287270 >> v1:172.21.15.43:6789/0 conn(0x7f6acc161f30 legacy=0x7f6acc0c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:27.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.471+0000 7f6aca59c640 1 -- 172.21.15.43:0/129287270 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6acc0d2a20 con 0x7f6acc15df50 2024-04-25T08:57:27.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.472+0000 7f6aca59c640 1 --2- 172.21.15.43:0/129287270 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6acc15df50 0x7f6acc1619f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6abc009dc0 tx=0x7f6abc02a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71429d08539e7a61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:27.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.472+0000 7f6ac959a640 1 -- 172.21.15.43:0/129287270 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6abc007740 con 0x7f6acc15df50 2024-04-25T08:57:27.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.472+0000 7f6ac959a640 1 -- 172.21.15.43:0/129287270 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6abc0078e0 con 0x7f6acc15df50 2024-04-25T08:57:27.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.472+0000 7f6ac959a640 1 -- 172.21.15.43:0/129287270 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6abc007c90 con 0x7f6acc15df50 2024-04-25T08:57:27.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.472+0000 7f6ad0c3a640 1 -- 172.21.15.43:0/129287270 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6acc15df50 msgr2=0x7f6acc1619f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:27.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.472+0000 7f6ad0c3a640 1 --2- 172.21.15.43:0/129287270 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6acc15df50 0x7f6acc1619f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6abc009dc0 tx=0x7f6abc02a6d0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.473+0000 7f6ad0c3a640 1 -- 172.21.15.43:0/129287270 shutdown_connections 2024-04-25T08:57:27.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.473+0000 7f6ad0c3a640 1 --2- 172.21.15.43:0/129287270 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6acc15df50 0x7f6acc1619f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.473+0000 7f6ad0c3a640 1 -- 172.21.15.43:0/129287270 >> 172.21.15.43:0/129287270 conn(0x7f6acc0d3a30 msgr2=0x7f6acc0d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:27.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.473+0000 7f6ad0c3a640 1 -- 172.21.15.43:0/129287270 shutdown_connections 2024-04-25T08:57:27.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.473+0000 7f6ad0c3a640 1 -- 172.21.15.43:0/129287270 wait complete. 2024-04-25T08:57:27.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.474+0000 7f6ad0c3a640 1 Processor -- start 2024-04-25T08:57:27.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.474+0000 7f6ad0c3a640 1 -- start start 2024-04-25T08:57:27.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.475+0000 7f6ad0c3a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6acc0cf670 0x7f6acc1f65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:27.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.475+0000 7f6ad0c3a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6acc15df50 0x7f6acc0caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:27.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.475+0000 7f6ad0c3a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6acc161f30 0x7f6acc0cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:27.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.475+0000 7f6ad0c3a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc0c77d0 con 0x7f6acc15df50 2024-04-25T08:57:27.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.475+0000 7f6ad0c3a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc15e330 con 0x7f6acc0cf670 2024-04-25T08:57:27.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.475+0000 7f6ad0c3a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6acc0d2190 con 0x7f6acc161f30 2024-04-25T08:57:27.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.475+0000 7f6ac9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6acc15df50 0x7f6acc0caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:27.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.475+0000 7f6ac9d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6acc15df50 0x7f6acc0caeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42380/0 (socket says 172.21.15.43:42380) 2024-04-25T08:57:27.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.475+0000 7f6ac9d9b640 1 -- 172.21.15.43:0/3325294543 learned_addr learned my addr 172.21.15.43:0/3325294543 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:27.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.475+0000 7f6acad9d640 1 --2- 172.21.15.43:0/3325294543 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6acc161f30 0x7f6acc0cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:27.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.475+0000 7f6aca59c640 1 --2- 172.21.15.43:0/3325294543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6acc0cf670 0x7f6acc1f65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:27.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.475+0000 7f6ac9d9b640 1 -- 172.21.15.43:0/3325294543 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6acc161f30 msgr2=0x7f6acc0cb3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:27.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.475+0000 7f6ac9d9b640 1 --2- 172.21.15.43:0/3325294543 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6acc161f30 0x7f6acc0cb3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.475+0000 7f6ac9d9b640 1 -- 172.21.15.43:0/3325294543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6acc0cf670 msgr2=0x7f6acc1f65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:27.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.475+0000 7f6ac9d9b640 1 --2- 172.21.15.43:0/3325294543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6acc0cf670 0x7f6acc1f65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.475+0000 7f6ac9d9b640 1 -- 172.21.15.43:0/3325294543 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6acc1f6b30 con 0x7f6acc15df50 2024-04-25T08:57:27.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.475+0000 7f6acad9d640 1 --2- 172.21.15.43:0/3325294543 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6acc161f30 0x7f6acc0cb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:27.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.475+0000 7f6aca59c640 1 --2- 172.21.15.43:0/3325294543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6acc0cf670 0x7f6acc1f65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:27.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.475+0000 7f6ac9d9b640 1 --2- 172.21.15.43:0/3325294543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6acc15df50 0x7f6acc0caeb0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f6ab400c410 tx=0x7f6ab400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:27.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.475+0000 7f6ac2ffd640 1 -- 172.21.15.43:0/3325294543 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab4017070 con 0x7f6acc15df50 2024-04-25T08:57:27.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.475+0000 7f6ac2ffd640 1 -- 172.21.15.43:0/3325294543 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ab4005ce0 con 0x7f6acc15df50 2024-04-25T08:57:27.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.476+0000 7f6ad0c3a640 1 -- 172.21.15.43:0/3325294543 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6acc15c190 con 0x7f6acc15df50 2024-04-25T08:57:27.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.476+0000 7f6ac2ffd640 1 -- 172.21.15.43:0/3325294543 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab40072d0 con 0x7f6acc15df50 2024-04-25T08:57:27.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.476+0000 7f6ad0c3a640 1 -- 172.21.15.43:0/3325294543 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6acc0cbce0 con 0x7f6acc15df50 2024-04-25T08:57:27.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.477+0000 7f6ad0c3a640 1 -- 172.21.15.43:0/3325294543 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6acc0680f0 con 0x7f6acc15df50 2024-04-25T08:57:27.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.477+0000 7f6ac2ffd640 1 -- 172.21.15.43:0/3325294543 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6ab401a070 con 0x7f6acc15df50 2024-04-25T08:57:27.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.478+0000 7f6ac2ffd640 1 --2- 172.21.15.43:0/3325294543 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6aac06b500 0x7f6aac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:27.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.478+0000 7f6ac2ffd640 1 -- 172.21.15.43:0/3325294543 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6ab4089430 con 0x7f6acc15df50 2024-04-25T08:57:27.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.478+0000 7f6aca59c640 1 --2- 172.21.15.43:0/3325294543 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6aac06b500 0x7f6aac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:27.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.479+0000 7f6aca59c640 1 --2- 172.21.15.43:0/3325294543 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6aac06b500 0x7f6aac06d9c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6abc034000 tx=0x7f6abc02f000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:27.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.483+0000 7f6ac2ffd640 1 -- 172.21.15.43:0/3325294543 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6ab4057cf0 con 0x7f6acc15df50 2024-04-25T08:57:27.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.701+0000 7f6ad0c3a640 1 -- 172.21.15.43:0/3325294543 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6acc0672d0 con 0x7f6acc15df50 2024-04-25T08:57:27.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.701+0000 7f6ac2ffd640 1 -- 172.21.15.43:0/3325294543 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f6ab405bd90 con 0x7f6acc15df50 2024-04-25T08:57:27.691 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:57:27.691 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1504450503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1504450503},{"type":"v1","addr":"172.21.15.193:6835","nonce":1504450503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5691,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2657133821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2657133821},{"type":"v1","addr":"172.21.15.43:6834","nonce":2657133821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":5694,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/4271431818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":4271431818},{"type":"v1","addr":"172.21.15.43:6835","nonce":4271431818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:57:01.856132+0000","modified":"2024-04-25T08:57:22.526624+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5709":{"gid":5709,"name":"d","rank":0,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6837/4164634380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4164634380},{"type":"v1","addr":"172.21.15.193:6837","nonce":4164634380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:57:27.691 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 22 2024-04-25T08:57:27.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.703+0000 7f6ad0c3a640 1 -- 172.21.15.43:0/3325294543 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6aac06b500 msgr2=0x7f6aac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:27.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.703+0000 7f6ad0c3a640 1 --2- 172.21.15.43:0/3325294543 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6aac06b500 0x7f6aac06d9c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f6abc034000 tx=0x7f6abc02f000 comp rx=0 tx=0).stop 2024-04-25T08:57:27.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.703+0000 7f6ad0c3a640 1 -- 172.21.15.43:0/3325294543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6acc15df50 msgr2=0x7f6acc0caeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:27.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.703+0000 7f6ad0c3a640 1 --2- 172.21.15.43:0/3325294543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6acc15df50 0x7f6acc0caeb0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f6ab400c410 tx=0x7f6ab400c8d0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.703+0000 7f6ad0c3a640 1 -- 172.21.15.43:0/3325294543 shutdown_connections 2024-04-25T08:57:27.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.703+0000 7f6ad0c3a640 1 --2- 172.21.15.43:0/3325294543 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6acc161f30 0x7f6acc0cb3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.703+0000 7f6ad0c3a640 1 --2- 172.21.15.43:0/3325294543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6acc15df50 0x7f6acc0caeb0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.703+0000 7f6ad0c3a640 1 --2- 172.21.15.43:0/3325294543 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6aac06b500 0x7f6aac06d9c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.703+0000 7f6ad0c3a640 1 --2- 172.21.15.43:0/3325294543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6acc0cf670 0x7f6acc1f65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:27.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.703+0000 7f6ad0c3a640 1 -- 172.21.15.43:0/3325294543 >> 172.21.15.43:0/3325294543 conn(0x7f6acc0d3a30 msgr2=0x7f6acc1560c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:27.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.703+0000 7f6ad0c3a640 1 -- 172.21.15.43:0/3325294543 shutdown_connections 2024-04-25T08:57:27.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27.703+0000 7f6ad0c3a640 1 -- 172.21.15.43:0/3325294543 wait complete. 2024-04-25T08:57:27.703 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path / 2024-04-25T08:57:27.819 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-04-25T08:57:27.819 INFO:teuthology.orchestra.run.smithi193.stdout: "return_code": 0 2024-04-25T08:57:27.819 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-04-25T08:57:27.828 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T08:57:27.828 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2024-04-25T08:57:27.828 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_test_path 2024-04-25T08:57:27.829 INFO:tasks.cephfs.test_scrub_checks:Cloning repo into place 2024-04-25T08:57:27.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:27.829 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p /home/ubuntu/cephtest/mnt.0/scrub_test_path) 2024-04-25T08:57:27.845 DEBUG:teuthology.orchestra.run.smithi043:> stdin-killer -- python3 -c ' 2024-04-25T08:57:27.845 DEBUG:teuthology.orchestra.run.smithi043:> import os 2024-04-25T08:57:27.845 DEBUG:teuthology.orchestra.run.smithi043:> import stat 2024-04-25T08:57:27.845 DEBUG:teuthology.orchestra.run.smithi043:> import json 2024-04-25T08:57:27.845 DEBUG:teuthology.orchestra.run.smithi043:> import sys 2024-04-25T08:57:27.845 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T08:57:27.846 DEBUG:teuthology.orchestra.run.smithi043:> try: 2024-04-25T08:57:27.846 DEBUG:teuthology.orchestra.run.smithi043:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'"'"') 2024-04-25T08:57:27.846 DEBUG:teuthology.orchestra.run.smithi043:> except OSError as e: 2024-04-25T08:57:27.846 DEBUG:teuthology.orchestra.run.smithi043:> sys.exit(e.errno) 2024-04-25T08:57:27.846 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T08:57:27.846 DEBUG:teuthology.orchestra.run.smithi043:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T08:57:27.846 DEBUG:teuthology.orchestra.run.smithi043:> print(json.dumps( 2024-04-25T08:57:27.846 DEBUG:teuthology.orchestra.run.smithi043:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T08:57:27.846 DEBUG:teuthology.orchestra.run.smithi043:> indent=2)) 2024-04-25T08:57:27.846 DEBUG:teuthology.orchestra.run.smithi043:> ' 2024-04-25T08:57:27.954 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T08:57:27.954 INFO:teuthology.orchestra.run.smithi043.stdout: "st_mode": 16877, 2024-04-25T08:57:27.954 INFO:teuthology.orchestra.run.smithi043.stdout: "st_ino": 1099511627777, 2024-04-25T08:57:27.954 INFO:teuthology.orchestra.run.smithi043.stdout: "st_dev": 43, 2024-04-25T08:57:27.955 INFO:teuthology.orchestra.run.smithi043.stdout: "st_nlink": 16, 2024-04-25T08:57:27.955 INFO:teuthology.orchestra.run.smithi043.stdout: "st_uid": 1000, 2024-04-25T08:57:27.955 INFO:teuthology.orchestra.run.smithi043.stdout: "st_gid": 1316, 2024-04-25T08:57:27.955 INFO:teuthology.orchestra.run.smithi043.stdout: "st_size": 9370617, 2024-04-25T08:57:27.955 INFO:teuthology.orchestra.run.smithi043.stdout: "st_atime": 1714035428.8639314, 2024-04-25T08:57:27.955 INFO:teuthology.orchestra.run.smithi043.stdout: "st_mtime": 1714035440.340908, 2024-04-25T08:57:27.956 INFO:teuthology.orchestra.run.smithi043.stdout: "st_ctime": 1714035440.340908 2024-04-25T08:57:27.956 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T08:57:27.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:57:27.963 INFO:tasks.cephfs.test_scrub_checks:Initiating mds_scrub_checks on mds.0 test_path /scrub_test_path, run_seq 1 2024-04-25T08:57:27.963 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/i/dont/exist' 2024-04-25T08:57:27.963 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:57:28.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.115+0000 7fb70fd09640 1 Processor -- start 2024-04-25T08:57:28.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.115+0000 7fb70fd09640 1 -- start start 2024-04-25T08:57:28.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.116+0000 7fb70fd09640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb70815dce0 0x7fb70815ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.116+0000 7fb70fd09640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb7080ce550 0x7fb7080d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.116+0000 7fb70fd09640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb7080ce9c0 0x7fb7080d0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.116+0000 7fb70fd09640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7080cee80 con 0x7fb70815dce0 2024-04-25T08:57:28.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.116+0000 7fb70fd09640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7080cf160 con 0x7fb7080ce550 2024-04-25T08:57:28.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.116+0000 7fb70fd09640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7080cf440 con 0x7fb7080ce9c0 2024-04-25T08:57:28.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.116+0000 7fb70daa5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb70815dce0 0x7fb70815ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.116+0000 7fb70daa5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb70815dce0 0x7fb70815ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42392/0 (socket says 172.21.15.43:42392) 2024-04-25T08:57:28.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.116+0000 7fb70daa5640 1 -- 172.21.15.43:0/1595855145 learned_addr learned my addr 172.21.15.43:0/1595855145 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:28.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.116+0000 7fb70daa5640 1 -- 172.21.15.43:0/1595855145 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb7080ce9c0 msgr2=0x7fb7080d0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:28.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.116+0000 7fb70daa5640 1 --2- 172.21.15.43:0/1595855145 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb7080ce9c0 0x7fb7080d0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.116+0000 7fb70daa5640 1 -- 172.21.15.43:0/1595855145 >> v2:172.21.15.193:3300/0 conn(0x7fb7080ce550 msgr2=0x7fb7080d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:28.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.116+0000 7fb70daa5640 1 --2- 172.21.15.43:0/1595855145 >> v2:172.21.15.193:3300/0 conn(0x7fb7080ce550 0x7fb7080d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.116+0000 7fb70daa5640 1 -- 172.21.15.43:0/1595855145 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7080cf720 con 0x7fb70815dce0 2024-04-25T08:57:28.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.117+0000 7fb70daa5640 1 --2- 172.21.15.43:0/1595855145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb70815dce0 0x7fb70815ef50 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb70806afa0 tx=0x7fb70002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b932b3947a8fb195 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:28.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.117+0000 7fb70caa3640 1 -- 172.21.15.43:0/1595855145 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb700036070 con 0x7fb70815dce0 2024-04-25T08:57:28.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.117+0000 7fb70caa3640 1 -- 172.21.15.43:0/1595855145 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb700031600 con 0x7fb70815dce0 2024-04-25T08:57:28.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.117+0000 7fb70caa3640 1 -- 172.21.15.43:0/1595855145 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7000319d0 con 0x7fb70815dce0 2024-04-25T08:57:28.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.118+0000 7fb70fd09640 1 -- 172.21.15.43:0/1595855145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb70815dce0 msgr2=0x7fb70815ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:28.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.118+0000 7fb70fd09640 1 --2- 172.21.15.43:0/1595855145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb70815dce0 0x7fb70815ef50 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb70806afa0 tx=0x7fb70002a610 comp rx=0 tx=0).stop 2024-04-25T08:57:28.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.118+0000 7fb70fd09640 1 -- 172.21.15.43:0/1595855145 shutdown_connections 2024-04-25T08:57:28.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.118+0000 7fb70fd09640 1 --2- 172.21.15.43:0/1595855145 >> v2:172.21.15.193:3300/0 conn(0x7fb7080ce550 0x7fb7080d03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.118+0000 7fb70fd09640 1 --2- 172.21.15.43:0/1595855145 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb7080ce9c0 0x7fb7080d0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.118+0000 7fb70fd09640 1 --2- 172.21.15.43:0/1595855145 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb70815dce0 0x7fb70815ef50 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.118+0000 7fb70fd09640 1 -- 172.21.15.43:0/1595855145 >> 172.21.15.43:0/1595855145 conn(0x7fb70809cd00 msgr2=0x7fb7080d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:28.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.118+0000 7fb70fd09640 1 -- 172.21.15.43:0/1595855145 shutdown_connections 2024-04-25T08:57:28.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.118+0000 7fb70fd09640 1 -- 172.21.15.43:0/1595855145 wait complete. 2024-04-25T08:57:28.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.119+0000 7fb70fd09640 1 Processor -- start 2024-04-25T08:57:28.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.119+0000 7fb70fd09640 1 -- start start 2024-04-25T08:57:28.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.120+0000 7fb70fd09640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7080ce550 0x7fb7081492a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.120+0000 7fb70fd09640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7080ce9c0 0x7fb70815a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.120+0000 7fb70fd09640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb70815dce0 0x7fb7080ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.120+0000 7fb70fd09640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb708160980 con 0x7fb7080ce9c0 2024-04-25T08:57:28.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.120+0000 7fb70fd09640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70815e0c0 con 0x7fb7080ce550 2024-04-25T08:57:28.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.120+0000 7fb70fd09640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb708160b00 con 0x7fb70815dce0 2024-04-25T08:57:28.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.120+0000 7fb70d2a4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7080ce9c0 0x7fb70815a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.120+0000 7fb70d2a4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7080ce9c0 0x7fb70815a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42408/0 (socket says 172.21.15.43:42408) 2024-04-25T08:57:28.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.120+0000 7fb70d2a4640 1 -- 172.21.15.43:0/3227916660 learned_addr learned my addr 172.21.15.43:0/3227916660 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:28.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.120+0000 7fb70daa5640 1 --2- 172.21.15.43:0/3227916660 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7080ce550 0x7fb7081492a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.120+0000 7fb70e2a6640 1 --2- 172.21.15.43:0/3227916660 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb70815dce0 0x7fb7080ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.120+0000 7fb70d2a4640 1 -- 172.21.15.43:0/3227916660 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb70815dce0 msgr2=0x7fb7080ccc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:28.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.120+0000 7fb70d2a4640 1 --2- 172.21.15.43:0/3227916660 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb70815dce0 0x7fb7080ccc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.120+0000 7fb70d2a4640 1 -- 172.21.15.43:0/3227916660 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7080ce550 msgr2=0x7fb7081492a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:28.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.120+0000 7fb70d2a4640 1 --2- 172.21.15.43:0/3227916660 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7080ce550 0x7fb7081492a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.120+0000 7fb70d2a4640 1 -- 172.21.15.43:0/3227916660 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb70815ae80 con 0x7fb7080ce9c0 2024-04-25T08:57:28.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.120+0000 7fb70e2a6640 1 --2- 172.21.15.43:0/3227916660 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb70815dce0 0x7fb7080ccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:28.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.121+0000 7fb70d2a4640 1 --2- 172.21.15.43:0/3227916660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7080ce9c0 0x7fb70815a480 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc00b4a0 tx=0x7fb6fc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:28.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.121+0000 7fb7067fc640 1 -- 172.21.15.43:0/3227916660 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc014070 con 0x7fb7080ce9c0 2024-04-25T08:57:28.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.121+0000 7fb7067fc640 1 -- 172.21.15.43:0/3227916660 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6fc010610 con 0x7fb7080ce9c0 2024-04-25T08:57:28.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.121+0000 7fb7067fc640 1 -- 172.21.15.43:0/3227916660 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6fc010a00 con 0x7fb7080ce9c0 2024-04-25T08:57:28.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.121+0000 7fb70fd09640 1 -- 172.21.15.43:0/3227916660 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb70815b160 con 0x7fb7080ce9c0 2024-04-25T08:57:28.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.121+0000 7fb70fd09640 1 -- 172.21.15.43:0/3227916660 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb70815ba00 con 0x7fb7080ce9c0 2024-04-25T08:57:28.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.122+0000 7fb70fd09640 1 -- 172.21.15.43:0/3227916660 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7080680f0 con 0x7fb7080ce9c0 2024-04-25T08:57:28.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.122+0000 7fb7067fc640 1 -- 172.21.15.43:0/3227916660 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb6fc017070 con 0x7fb7080ce9c0 2024-04-25T08:57:28.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.123+0000 7fb7067fc640 1 --2- 172.21.15.43:0/3227916660 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb6f406b4d0 0x7fb6f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.123+0000 7fb7067fc640 1 -- 172.21.15.43:0/3227916660 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb6fc089b90 con 0x7fb7080ce9c0 2024-04-25T08:57:28.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.123+0000 7fb70daa5640 1 --2- 172.21.15.43:0/3227916660 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb6f406b4d0 0x7fb6f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.124+0000 7fb70daa5640 1 --2- 172.21.15.43:0/3227916660 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb6f406b4d0 0x7fb6f406d990 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb700033000 tx=0x7fb700009000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:28.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.127+0000 7fb7067fc640 1 -- 172.21.15.43:0/3227916660 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb6fc0583c0 con 0x7fb7080ce9c0 2024-04-25T08:57:28.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.343+0000 7fb70fd09640 1 -- 172.21.15.43:0/3227916660 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb708149840 con 0x7fb7080ce9c0 2024-04-25T08:57:28.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.344+0000 7fb7067fc640 1 -- 172.21.15.43:0/3227916660 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fb6fc05c460 con 0x7fb7080ce9c0 2024-04-25T08:57:28.334 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:57:28.334 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1504450503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1504450503},{"type":"v1","addr":"172.21.15.193:6835","nonce":1504450503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5691,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2657133821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2657133821},{"type":"v1","addr":"172.21.15.43:6834","nonce":2657133821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":5694,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/4271431818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":4271431818},{"type":"v1","addr":"172.21.15.43:6835","nonce":4271431818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:57:01.856132+0000","modified":"2024-04-25T08:57:22.526624+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5709":{"gid":5709,"name":"d","rank":0,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6837/4164634380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4164634380},{"type":"v1","addr":"172.21.15.193:6837","nonce":4164634380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:57:28.334 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 22 2024-04-25T08:57:28.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.346+0000 7fb70fd09640 1 -- 172.21.15.43:0/3227916660 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb6f406b4d0 msgr2=0x7fb6f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:28.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.346+0000 7fb70fd09640 1 --2- 172.21.15.43:0/3227916660 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb6f406b4d0 0x7fb6f406d990 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb700033000 tx=0x7fb700009000 comp rx=0 tx=0).stop 2024-04-25T08:57:28.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.346+0000 7fb70fd09640 1 -- 172.21.15.43:0/3227916660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7080ce9c0 msgr2=0x7fb70815a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:28.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.346+0000 7fb70fd09640 1 --2- 172.21.15.43:0/3227916660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7080ce9c0 0x7fb70815a480 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb6fc00b4a0 tx=0x7fb6fc00b960 comp rx=0 tx=0).stop 2024-04-25T08:57:28.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.346+0000 7fb70fd09640 1 -- 172.21.15.43:0/3227916660 shutdown_connections 2024-04-25T08:57:28.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.346+0000 7fb70fd09640 1 --2- 172.21.15.43:0/3227916660 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb70815dce0 0x7fb7080ccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.346+0000 7fb70fd09640 1 --2- 172.21.15.43:0/3227916660 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7080ce9c0 0x7fb70815a480 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.346+0000 7fb70fd09640 1 --2- 172.21.15.43:0/3227916660 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb6f406b4d0 0x7fb6f406d990 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.346+0000 7fb70fd09640 1 --2- 172.21.15.43:0/3227916660 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7080ce550 0x7fb7081492a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.346+0000 7fb70fd09640 1 -- 172.21.15.43:0/3227916660 >> 172.21.15.43:0/3227916660 conn(0x7fb70809cd00 msgr2=0x7fb70806b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:28.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.346+0000 7fb70fd09640 1 -- 172.21.15.43:0/3227916660 shutdown_connections 2024-04-25T08:57:28.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.346+0000 7fb70fd09640 1 -- 172.21.15.43:0/3227916660 wait complete. 2024-04-25T08:57:28.346 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path /scrub_test_path/i/dont/exist 2024-04-25T08:57:28.460 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-04-25T08:57:28.460 INFO:teuthology.orchestra.run.smithi193.stdout: "return_code": -2 2024-04-25T08:57:28.460 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-04-25T08:57:28.469 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/i/dont/exist' got response code '0' and stdout '{ "return_code": -2 } ' 2024-04-25T08:57:28.469 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/i/dont/exist' 2024-04-25T08:57:28.470 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:57:28.470 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/i/dont/exist 2024-04-25T08:57:28.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.585+0000 7fa7e50c6640 1 Processor -- start 2024-04-25T08:57:28.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.585+0000 7fa7e50c6640 1 -- start start 2024-04-25T08:57:28.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.586+0000 7fa7e50c6640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa7e00d27b0 0x7fa7e015ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.586+0000 7fa7e50c6640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7e00cf110 con 0x7fa7e00d04a0 2024-04-25T08:57:28.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.586+0000 7fa7e50c6640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7e00cf3f0 con 0x7fa7e015dec0 2024-04-25T08:57:28.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.586+0000 7fa7e50c6640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7e00cf6d0 con 0x7fa7e00d27b0 2024-04-25T08:57:28.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.586+0000 7fa7df59e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa7e00d27b0 0x7fa7e015ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.586+0000 7fa7df59e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa7e00d27b0 0x7fa7e015ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42422/0 (socket says 172.21.15.43:42422) 2024-04-25T08:57:28.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.586+0000 7fa7ded9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa7e00d04a0 0x7fa7e00d0880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49710/0 (socket says 172.21.15.43:49710) 2024-04-25T08:57:28.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.586+0000 7fa7df59e640 1 -- 172.21.15.43:0/3840453159 learned_addr learned my addr 172.21.15.43:0/3840453159 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:28.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.586+0000 7fa7df59e640 1 -- 172.21.15.43:0/3840453159 >> v1:172.21.15.193:6789/0 conn(0x7fa7e015dec0 legacy=0x7fa7e00ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:28.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.586+0000 7fa7df59e640 1 -- 172.21.15.43:0/3840453159 >> v1:172.21.15.43:6789/0 conn(0x7fa7e00d04a0 legacy=0x7fa7e00d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:28.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.586+0000 7fa7df59e640 1 -- 172.21.15.43:0/3840453159 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e00cf9b0 con 0x7fa7e00d27b0 2024-04-25T08:57:28.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.587+0000 7fa7df59e640 1 --2- 172.21.15.43:0/3840453159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e00d27b0 0x7fa7e015ed90 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8009e50 tx=0x7fa7d802a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb6c5f7da234e875 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:28.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.587+0000 7fa7de59c640 1 -- 172.21.15.43:0/3840453159 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8036070 con 0x7fa7e00d27b0 2024-04-25T08:57:28.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.587+0000 7fa7de59c640 1 -- 172.21.15.43:0/3840453159 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d8031600 con 0x7fa7e00d27b0 2024-04-25T08:57:28.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.587+0000 7fa7de59c640 1 -- 172.21.15.43:0/3840453159 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d80319d0 con 0x7fa7e00d27b0 2024-04-25T08:57:28.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.587+0000 7fa7e50c6640 1 -- 172.21.15.43:0/3840453159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e00d27b0 msgr2=0x7fa7e015ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:28.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.587+0000 7fa7e50c6640 1 --2- 172.21.15.43:0/3840453159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e00d27b0 0x7fa7e015ed90 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8009e50 tx=0x7fa7d802a6d0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.588+0000 7fa7e50c6640 1 -- 172.21.15.43:0/3840453159 shutdown_connections 2024-04-25T08:57:28.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.588+0000 7fa7e50c6640 1 --2- 172.21.15.43:0/3840453159 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e00d27b0 0x7fa7e015ed90 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.588+0000 7fa7e50c6640 1 -- 172.21.15.43:0/3840453159 >> 172.21.15.43:0/3840453159 conn(0x7fa7e009cd40 msgr2=0x7fa7e00d3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:28.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.588+0000 7fa7e50c6640 1 -- 172.21.15.43:0/3840453159 shutdown_connections 2024-04-25T08:57:28.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.588+0000 7fa7e50c6640 1 -- 172.21.15.43:0/3840453159 wait complete. 2024-04-25T08:57:28.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.589+0000 7fa7e50c6640 1 Processor -- start 2024-04-25T08:57:28.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.589+0000 7fa7e50c6640 1 -- start start 2024-04-25T08:57:28.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.590+0000 7fa7e50c6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7e00d04a0 0x7fa7e00cd640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.590+0000 7fa7e50c6640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7e00d27b0 0x7fa7e00b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.590+0000 7fa7e50c6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e015dec0 0x7fa7e00b3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.590+0000 7fa7e50c6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e0168c50 con 0x7fa7e015dec0 2024-04-25T08:57:28.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.590+0000 7fa7e50c6640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e0168ad0 con 0x7fa7e00d04a0 2024-04-25T08:57:28.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.590+0000 7fa7e50c6640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e0168dd0 con 0x7fa7e00d27b0 2024-04-25T08:57:28.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.590+0000 7fa7df59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e015dec0 0x7fa7e00b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.590+0000 7fa7df59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e015dec0 0x7fa7e00b3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42424/0 (socket says 172.21.15.43:42424) 2024-04-25T08:57:28.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.590+0000 7fa7df59e640 1 -- 172.21.15.43:0/433571809 learned_addr learned my addr 172.21.15.43:0/433571809 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:28.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.590+0000 7fa7ded9d640 1 --2- 172.21.15.43:0/433571809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7e00d04a0 0x7fa7e00cd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.590+0000 7fa7d7fff640 1 --2- 172.21.15.43:0/433571809 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7e00d27b0 0x7fa7e00b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.590+0000 7fa7df59e640 1 -- 172.21.15.43:0/433571809 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7e00d27b0 msgr2=0x7fa7e00b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:28.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.590+0000 7fa7df59e640 1 --2- 172.21.15.43:0/433571809 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7e00d27b0 0x7fa7e00b2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.590+0000 7fa7df59e640 1 -- 172.21.15.43:0/433571809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7e00d04a0 msgr2=0x7fa7e00cd640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:28.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.590+0000 7fa7df59e640 1 --2- 172.21.15.43:0/433571809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7e00d04a0 0x7fa7e00cd640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.590+0000 7fa7df59e640 1 -- 172.21.15.43:0/433571809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e0155d80 con 0x7fa7e015dec0 2024-04-25T08:57:28.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.590+0000 7fa7d7fff640 1 --2- 172.21.15.43:0/433571809 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7e00d27b0 0x7fa7e00b2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:28.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.590+0000 7fa7df59e640 1 --2- 172.21.15.43:0/433571809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e015dec0 0x7fa7e00b3990 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8009e50 tx=0x7fa7d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:28.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.590+0000 7fa7d77fe640 1 -- 172.21.15.43:0/433571809 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8041020 con 0x7fa7e015dec0 2024-04-25T08:57:28.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.590+0000 7fa7d77fe640 1 -- 172.21.15.43:0/433571809 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d8007820 con 0x7fa7e015dec0 2024-04-25T08:57:28.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.590+0000 7fa7d77fe640 1 -- 172.21.15.43:0/433571809 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8036040 con 0x7fa7e015dec0 2024-04-25T08:57:28.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.591+0000 7fa7e50c6640 1 -- 172.21.15.43:0/433571809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7e0156060 con 0x7fa7e015dec0 2024-04-25T08:57:28.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.591+0000 7fa7e50c6640 1 -- 172.21.15.43:0/433571809 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e0156620 con 0x7fa7e015dec0 2024-04-25T08:57:28.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.592+0000 7fa7d77fe640 1 -- 172.21.15.43:0/433571809 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa7d8044020 con 0x7fa7e015dec0 2024-04-25T08:57:28.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.593+0000 7fa7d77fe640 1 --2- 172.21.15.43:0/433571809 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa7c006b4d0 0x7fa7c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.593+0000 7fa7d77fe640 1 -- 172.21.15.43:0/433571809 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa7d80ab010 con 0x7fa7e015dec0 2024-04-25T08:57:28.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.593+0000 7fa7ded9d640 1 --2- 172.21.15.43:0/433571809 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa7c006b4d0 0x7fa7c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.593+0000 7fa7ded9d640 1 --2- 172.21.15.43:0/433571809 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa7c006b4d0 0x7fa7c006d990 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0007fd0 tx=0x7fa7d0007e80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:28.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.594+0000 7fa7e7dc5740 1 Processor -- start 2024-04-25T08:57:28.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.594+0000 7fa7e7dc5740 1 -- start start 2024-04-25T08:57:28.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.594+0000 7fa7e7dc5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563704c726b0 0x563704c72a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.594+0000 7fa7e7dc5740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563704c8ff50 0x563704c904a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.595+0000 7fa7e7dc5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563704c48390 0x563704c8e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.595+0000 7fa7e7dc5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x563704baac60 con 0x563704c48390 2024-04-25T08:57:28.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.595+0000 7fa7e7dc5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x563704b53ee0 con 0x563704c726b0 2024-04-25T08:57:28.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.595+0000 7fa7e7dc5740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x563704ba6d00 con 0x563704c8ff50 2024-04-25T08:57:28.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.595+0000 7fa7df59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563704c48390 0x563704c8e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.595+0000 7fa7df59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563704c48390 0x563704c8e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42432/0 (socket says 172.21.15.43:42432) 2024-04-25T08:57:28.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.595+0000 7fa7df59e640 1 -- 172.21.15.43:0/2745280576 learned_addr learned my addr 172.21.15.43:0/2745280576 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:28.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.595+0000 7fa7ded9d640 1 --2- 172.21.15.43:0/2745280576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563704c726b0 0x563704c72a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.595+0000 7fa7d7fff640 1 --2- 172.21.15.43:0/2745280576 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563704c8ff50 0x563704c904a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.595+0000 7fa7df59e640 1 -- 172.21.15.43:0/2745280576 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563704c8ff50 msgr2=0x563704c904a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:28.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.595+0000 7fa7df59e640 1 --2- 172.21.15.43:0/2745280576 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563704c8ff50 0x563704c904a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.595+0000 7fa7df59e640 1 -- 172.21.15.43:0/2745280576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563704c726b0 msgr2=0x563704c72a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:28.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.595+0000 7fa7df59e640 1 --2- 172.21.15.43:0/2745280576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563704c726b0 0x563704c72a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.595+0000 7fa7df59e640 1 -- 172.21.15.43:0/2745280576 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56370498b9f0 con 0x563704c48390 2024-04-25T08:57:28.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.595+0000 7fa7df59e640 1 --2- 172.21.15.43:0/2745280576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563704c48390 0x563704c8e610 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8034040 tx=0x7fa7d80095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=748320a7479ee6ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:28.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.595+0000 7fa7d4ff9640 1 -- 172.21.15.43:0/2745280576 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d807d8e0 con 0x563704c48390 2024-04-25T08:57:28.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.595+0000 7fa7d4ff9640 1 -- 172.21.15.43:0/2745280576 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d8081ea0 con 0x563704c48390 2024-04-25T08:57:28.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.595+0000 7fa7d4ff9640 1 -- 172.21.15.43:0/2745280576 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d80911c0 con 0x563704c48390 2024-04-25T08:57:28.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.596+0000 7fa7e7dc5740 1 -- 172.21.15.43:0/2745280576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563704c48390 msgr2=0x563704c8e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:28.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.596+0000 7fa7e7dc5740 1 --2- 172.21.15.43:0/2745280576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563704c48390 0x563704c8e610 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8034040 tx=0x7fa7d80095d0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.596+0000 7fa7e7dc5740 1 -- 172.21.15.43:0/2745280576 shutdown_connections 2024-04-25T08:57:28.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.596+0000 7fa7e7dc5740 1 --2- 172.21.15.43:0/2745280576 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563704c48390 0x563704c8e610 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.596+0000 7fa7e7dc5740 1 --2- 172.21.15.43:0/2745280576 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563704c8ff50 0x563704c904a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.596+0000 7fa7e7dc5740 1 --2- 172.21.15.43:0/2745280576 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563704c726b0 0x563704c72a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.596+0000 7fa7e7dc5740 1 -- 172.21.15.43:0/2745280576 >> 172.21.15.43:0/2745280576 conn(0x563704c44cc0 msgr2=0x563704c8d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:28.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.596+0000 7fa7e7dc5740 1 -- 172.21.15.43:0/2745280576 shutdown_connections 2024-04-25T08:57:28.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.596+0000 7fa7e7dc5740 1 -- 172.21.15.43:0/2745280576 wait complete. 2024-04-25T08:57:28.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.597+0000 7fa7e7dc5740 1 -- start start 2024-04-25T08:57:28.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.597+0000 7fa7e7dc5740 1 Processor -- start 2024-04-25T08:57:28.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.597+0000 7fa7e7dc5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563704c726b0 0x563704c6efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.597+0000 7fa7e7dc5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563704c8ff50 0x563704c75010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.597+0000 7fa7e7dc5740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563704c41920 0x563704c41da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.597+0000 7fa7e7dc5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x563704c49130 con 0x563704c8ff50 2024-04-25T08:57:28.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.597+0000 7fa7e7dc5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x563704b67470 con 0x563704c726b0 2024-04-25T08:57:28.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.597+0000 7fa7e7dc5740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x563704c42bf0 con 0x563704c41920 2024-04-25T08:57:28.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.597+0000 7fa7e7dc5740 10 client.0 _ll_register_callbacks cb 0x563704b337d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:28.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.597+0000 7fa7d7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563704c8ff50 0x563704c75010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.598+0000 7fa7d7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563704c8ff50 0x563704c75010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42440/0 (socket says 172.21.15.43:42440) 2024-04-25T08:57:28.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.598+0000 7fa7d7fff640 1 -- 172.21.15.43:0/3935452468 learned_addr learned my addr 172.21.15.43:0/3935452468 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:28.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.598+0000 7fa7ded9d640 1 --2- 172.21.15.43:0/3935452468 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563704c726b0 0x563704c6efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.598+0000 7fa7df59e640 1 --2- 172.21.15.43:0/3935452468 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563704c41920 0x563704c41da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.598+0000 7fa7d7fff640 1 -- 172.21.15.43:0/3935452468 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563704c41920 msgr2=0x563704c41da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:28.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.598+0000 7fa7d7fff640 1 --2- 172.21.15.43:0/3935452468 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563704c41920 0x563704c41da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.598+0000 7fa7d7fff640 1 -- 172.21.15.43:0/3935452468 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563704c726b0 msgr2=0x563704c6efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:28.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.598+0000 7fa7d7fff640 1 --2- 172.21.15.43:0/3935452468 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563704c726b0 0x563704c6efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.598+0000 7fa7d7fff640 1 -- 172.21.15.43:0/3935452468 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56370487f2e0 con 0x563704c8ff50 2024-04-25T08:57:28.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.598+0000 7fa7d7fff640 1 --2- 172.21.15.43:0/3935452468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563704c8ff50 0x563704c75010 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fa7c80114d0 tx=0x7fa7c8011990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9dc09d2b2a449c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:28.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.598+0000 7fa7bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:57:28.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.598+0000 7fa7bdffb640 1 -- 172.21.15.43:0/3935452468 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8003a00 con 0x563704c8ff50 2024-04-25T08:57:28.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.598+0000 7fa7bdffb640 1 -- 172.21.15.43:0/3935452468 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa7c8003ba0 con 0x563704c8ff50 2024-04-25T08:57:28.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.598+0000 7fa7e7dc5740 1 -- 172.21.15.43:0/3935452468 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa7d8081ea0 con 0x563704c8ff50 2024-04-25T08:57:28.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.598+0000 7fa7bdffb640 1 -- 172.21.15.43:0/3935452468 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7c8022050 con 0x563704c8ff50 2024-04-25T08:57:28.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.598+0000 7fa7e7dc5740 10 client.6129 fetch_fsmap learned FSMap version 22 2024-04-25T08:57:28.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.598+0000 7fa7e7dc5740 1 -- 172.21.15.43:0/3935452468 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x5637049ec180 con 0x563704c8ff50 2024-04-25T08:57:28.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.598+0000 7fa7bdffb640 1 -- 172.21.15.43:0/3935452468 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d8081ea0 con 0x563704c8ff50 2024-04-25T08:57:28.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.599+0000 7fa7bdffb640 1 -- 172.21.15.43:0/3935452468 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa7c8025020 con 0x563704c8ff50 2024-04-25T08:57:28.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.599+0000 7fa7e7dc5740 10 client.6129 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T08:57:28.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.599+0000 7fa7e7dc5740 10 client.6129 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T08:57:28.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.599+0000 7fa7e7dc5740 20 client.6129 populate_metadata read hostname 'smithi043' 2024-04-25T08:57:28.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.599+0000 7fa7e7dc5740 1 --2- 172.21.15.43:0/3935452468 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x563704c86600 0x563704c869e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.599+0000 7fa7e7dc5740 4 client.6129 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:57:28.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.599+0000 7fa7e7dc5740 1 -- 172.21.15.43:0/3935452468 --> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563704bb81a0 con 0x563704c86600 2024-04-25T08:57:28.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.599+0000 7fa7ded9d640 1 --2- 172.21.15.43:0/3935452468 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x563704c86600 0x563704c869e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.599+0000 7fa7ded9d640 1 --2- 172.21.15.43:0/3935452468 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x563704c86600 0x563704c869e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36482aa59eafbf51 server_cookie=648a46c2d22fab18 in_seq=0 out_seq=0 2024-04-25T08:57:28.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.599+0000 7fa7bdffb640 10 client.6129 ms_handle_connect on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:28.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.600+0000 7fa7bdffb640 1 -- 172.21.15.43:0/3935452468 <== mds.0 v2:172.21.15.193:6836/4164634380 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563704bb81a0 con 0x563704c86600 2024-04-25T08:57:28.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.600+0000 7fa7bdffb640 10 client.6129 handle_command_reply: tid=0 2024-04-25T08:57:28.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.600+0000 7fa7e7dc5740 1 client.6129 shutdown 2024-04-25T08:57:28.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.600+0000 7fa7e7dc5740 1 -- 172.21.15.43:0/3935452468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563704c8ff50 msgr2=0x563704c75010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:28.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.600+0000 7fa7e7dc5740 1 --2- 172.21.15.43:0/3935452468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563704c8ff50 0x563704c75010 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fa7c80114d0 tx=0x7fa7c8011990 comp rx=0 tx=0).stop 2024-04-25T08:57:28.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.600+0000 7fa7e7dc5740 1 -- 172.21.15.43:0/3935452468 shutdown_connections 2024-04-25T08:57:28.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.600+0000 7fa7e7dc5740 1 --2- 172.21.15.43:0/3935452468 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563704c41920 0x563704c41da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.600+0000 7fa7e7dc5740 1 --2- 172.21.15.43:0/3935452468 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563704c8ff50 0x563704c75010 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.600+0000 7fa7e7dc5740 1 --2- 172.21.15.43:0/3935452468 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x563704c86600 0x563704c869e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.600+0000 7fa7e7dc5740 1 --2- 172.21.15.43:0/3935452468 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563704c726b0 0x563704c6efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.600+0000 7fa7bdffb640 0 client.6129 ms_handle_reset on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:28.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.600+0000 7fa7e7dc5740 1 -- 172.21.15.43:0/3935452468 >> 172.21.15.43:0/3935452468 conn(0x563704c48390 msgr2=0x563704c71940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:28.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.600+0000 7fa7e7dc5740 1 -- 172.21.15.43:0/3935452468 shutdown_connections 2024-04-25T08:57:28.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.600+0000 7fa7e7dc5740 1 -- 172.21.15.43:0/3935452468 wait complete. 2024-04-25T08:57:28.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.600+0000 7fa7e7dc5740 20 client.6129 trim_cache size 0 max 16384 2024-04-25T08:57:28.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.616+0000 7fa7e7dc5740 1 Processor -- start 2024-04-25T08:57:28.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.616+0000 7fa7e7dc5740 1 -- start start 2024-04-25T08:57:28.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.617+0000 7fa7e7dc5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563704c41920 0x563704cef080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.617+0000 7fa7e7dc5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563704c726b0 0x563704c75da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.617+0000 7fa7e7dc5740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563704c8ff50 0x563704c876c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.617+0000 7fa7e7dc5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e02000d0 con 0x563704c41920 2024-04-25T08:57:28.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.617+0000 7fa7e7dc5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e01ff2d0 con 0x563704c726b0 2024-04-25T08:57:28.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.617+0000 7fa7e7dc5740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e01f85a0 con 0x563704c8ff50 2024-04-25T08:57:28.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.617+0000 7fa7ded9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563704c41920 0x563704cef080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.617+0000 7fa7ded9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563704c41920 0x563704cef080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42448/0 (socket says 172.21.15.43:42448) 2024-04-25T08:57:28.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.617+0000 7fa7ded9d640 1 -- 172.21.15.43:0/1620749185 learned_addr learned my addr 172.21.15.43:0/1620749185 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:28.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.617+0000 7fa7d7fff640 1 --2- 172.21.15.43:0/1620749185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563704c726b0 0x563704c75da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.617+0000 7fa7df59e640 1 --2- 172.21.15.43:0/1620749185 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563704c8ff50 0x563704c876c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.617+0000 7fa7ded9d640 1 -- 172.21.15.43:0/1620749185 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563704c8ff50 msgr2=0x563704c876c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:28.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.617+0000 7fa7ded9d640 1 --2- 172.21.15.43:0/1620749185 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563704c8ff50 0x563704c876c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.617+0000 7fa7ded9d640 1 -- 172.21.15.43:0/1620749185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563704c726b0 msgr2=0x563704c75da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:28.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.617+0000 7fa7ded9d640 1 --2- 172.21.15.43:0/1620749185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563704c726b0 0x563704c75da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.617+0000 7fa7ded9d640 1 -- 172.21.15.43:0/1620749185 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563704c86d50 con 0x563704c41920 2024-04-25T08:57:28.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.617+0000 7fa7ded9d640 1 --2- 172.21.15.43:0/1620749185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563704c41920 0x563704cef080 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x563704c852e0 tx=0x7fa7d0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de7ccc8b2cc1f788 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:28.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.617+0000 7fa7bffff640 1 -- 172.21.15.43:0/1620749185 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d00287a0 con 0x563704c41920 2024-04-25T08:57:28.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.618+0000 7fa7bffff640 1 -- 172.21.15.43:0/1620749185 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d002c020 con 0x563704c41920 2024-04-25T08:57:28.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.618+0000 7fa7bffff640 1 -- 172.21.15.43:0/1620749185 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0028b00 con 0x563704c41920 2024-04-25T08:57:28.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.618+0000 7fa7e7dc5740 1 -- 172.21.15.43:0/1620749185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563704c41920 msgr2=0x563704cef080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:28.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.618+0000 7fa7e7dc5740 1 --2- 172.21.15.43:0/1620749185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563704c41920 0x563704cef080 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x563704c852e0 tx=0x7fa7d0013000 comp rx=0 tx=0).stop 2024-04-25T08:57:28.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.618+0000 7fa7e7dc5740 1 -- 172.21.15.43:0/1620749185 shutdown_connections 2024-04-25T08:57:28.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.618+0000 7fa7e7dc5740 1 --2- 172.21.15.43:0/1620749185 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563704c8ff50 0x563704c876c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.618+0000 7fa7e7dc5740 1 --2- 172.21.15.43:0/1620749185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563704c726b0 0x563704c75da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.618+0000 7fa7e7dc5740 1 --2- 172.21.15.43:0/1620749185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563704c41920 0x563704cef080 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.618+0000 7fa7e7dc5740 1 -- 172.21.15.43:0/1620749185 >> 172.21.15.43:0/1620749185 conn(0x563704c48390 msgr2=0x563704c9d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:28.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.618+0000 7fa7e7dc5740 1 -- 172.21.15.43:0/1620749185 shutdown_connections 2024-04-25T08:57:28.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.618+0000 7fa7e7dc5740 1 -- 172.21.15.43:0/1620749185 wait complete. 2024-04-25T08:57:28.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.619+0000 7fa7e7dc5740 1 -- start start 2024-04-25T08:57:28.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.619+0000 7fa7e7dc5740 1 Processor -- start 2024-04-25T08:57:28.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.620+0000 7fa7e7dc5740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563704c41920 0x563704c882f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.620+0000 7fa7e7dc5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563704c726b0 0x563704c88830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.620+0000 7fa7e7dc5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563704c8ff50 0x563704cef080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.620+0000 7fa7e7dc5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x563704c431e0 con 0x563704c8ff50 2024-04-25T08:57:28.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.620+0000 7fa7e7dc5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e01fecd0 con 0x563704c726b0 2024-04-25T08:57:28.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.620+0000 7fa7e7dc5740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x563704c8f9c0 con 0x563704c41920 2024-04-25T08:57:28.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.620+0000 7fa7e7dc5740 10 client.0 _ll_register_callbacks cb 0x563704c430d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:28.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.620+0000 7fa7df59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563704c8ff50 0x563704cef080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.620+0000 7fa7df59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563704c8ff50 0x563704cef080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42460/0 (socket says 172.21.15.43:42460) 2024-04-25T08:57:28.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.620+0000 7fa7df59e640 1 -- 172.21.15.43:0/2083134658 learned_addr learned my addr 172.21.15.43:0/2083134658 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:28.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.620+0000 7fa7d7fff640 1 --2- 172.21.15.43:0/2083134658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563704c726b0 0x563704c88830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.620+0000 7fa7ded9d640 1 --2- 172.21.15.43:0/2083134658 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563704c41920 0x563704c882f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.620+0000 7fa7df59e640 1 -- 172.21.15.43:0/2083134658 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563704c41920 msgr2=0x563704c882f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:28.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.620+0000 7fa7df59e640 1 --2- 172.21.15.43:0/2083134658 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563704c41920 0x563704c882f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.620+0000 7fa7df59e640 1 -- 172.21.15.43:0/2083134658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563704c726b0 msgr2=0x563704c88830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:28.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.620+0000 7fa7df59e640 1 --2- 172.21.15.43:0/2083134658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563704c726b0 0x563704c88830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.620+0000 7fa7df59e640 1 -- 172.21.15.43:0/2083134658 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563704c9dc60 con 0x563704c8ff50 2024-04-25T08:57:28.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.620+0000 7fa7ded9d640 1 --2- 172.21.15.43:0/2083134658 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563704c41920 0x563704c882f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:28.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.620+0000 7fa7d7fff640 1 --2- 172.21.15.43:0/2083134658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563704c726b0 0x563704c88830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:28.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.620+0000 7fa7df59e640 1 --2- 172.21.15.43:0/2083134658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563704c8ff50 0x563704cef080 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8075ba0 tx=0x7fa7d8009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e0e6339ecf0b36e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:28.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.620+0000 7fa7bdffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:57:28.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.620+0000 7fa7bdffb640 1 -- 172.21.15.43:0/2083134658 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8095070 con 0x563704c8ff50 2024-04-25T08:57:28.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.620+0000 7fa7bdffb640 1 -- 172.21.15.43:0/2083134658 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa7d80911c0 con 0x563704c8ff50 2024-04-25T08:57:28.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.620+0000 7fa7e7dc5740 1 -- 172.21.15.43:0/2083134658 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa7d002c020 con 0x563704c8ff50 2024-04-25T08:57:28.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.621+0000 7fa7bdffb640 1 -- 172.21.15.43:0/2083134658 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7d80a94b0 con 0x563704c8ff50 2024-04-25T08:57:28.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.621+0000 7fa7e7dc5740 10 client.6135 fetch_fsmap learned FSMap version 22 2024-04-25T08:57:28.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.621+0000 7fa7e7dc5740 1 -- 172.21.15.43:0/2083134658 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x5637048583a0 con 0x563704c8ff50 2024-04-25T08:57:28.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.621+0000 7fa7bdffb640 1 -- 172.21.15.43:0/2083134658 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d002c020 con 0x563704c8ff50 2024-04-25T08:57:28.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.621+0000 7fa7bdffb640 1 -- 172.21.15.43:0/2083134658 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa7d8081a10 con 0x563704c8ff50 2024-04-25T08:57:28.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.621+0000 7fa7e7dc5740 10 client.6135 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T08:57:28.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.621+0000 7fa7e7dc5740 10 client.6135 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T08:57:28.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.621+0000 7fa7e7dc5740 20 client.6135 populate_metadata read hostname 'smithi043' 2024-04-25T08:57:28.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.621+0000 7fa7e7dc5740 1 --2- 172.21.15.43:0/2083134658 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x563704c833f0 0x563704cef5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.621+0000 7fa7e7dc5740 4 client.6135 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}] 2024-04-25T08:57:28.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.621+0000 7fa7e7dc5740 1 -- 172.21.15.43:0/2083134658 --> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}) v1 -- 0x563704bb24f0 con 0x563704c833f0 2024-04-25T08:57:28.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.621+0000 7fa7ded9d640 1 --2- 172.21.15.43:0/2083134658 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x563704c833f0 0x563704cef5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.621+0000 7fa7ded9d640 1 --2- 172.21.15.43:0/2083134658 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x563704c833f0 0x563704cef5c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84a2d8c32c00c4f6 server_cookie=238d538598981cdb in_seq=0 out_seq=0 2024-04-25T08:57:28.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.621+0000 7fa7bdffb640 10 client.6135 ms_handle_connect on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:28.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.622+0000 7fa7bdffb640 1 -- 172.21.15.43:0/2083134658 <== mds.0 v2:172.21.15.193:6836/4164634380 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+26 (crc 0 0 0) 0x563704bb24f0 con 0x563704c833f0 2024-04-25T08:57:28.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.622+0000 7fa7bdffb640 10 client.6135 handle_command_reply: tid=0 2024-04-25T08:57:28.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.622+0000 7fa7e7dc5740 1 client.6135 shutdown 2024-04-25T08:57:28.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.622+0000 7fa7e7dc5740 1 -- 172.21.15.43:0/2083134658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563704c8ff50 msgr2=0x563704cef080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:28.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.622+0000 7fa7e7dc5740 1 --2- 172.21.15.43:0/2083134658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563704c8ff50 0x563704cef080 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8075ba0 tx=0x7fa7d8009810 comp rx=0 tx=0).stop 2024-04-25T08:57:28.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.622+0000 7fa7e7dc5740 1 -- 172.21.15.43:0/2083134658 shutdown_connections 2024-04-25T08:57:28.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.622+0000 7fa7e7dc5740 1 --2- 172.21.15.43:0/2083134658 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563704c8ff50 0x563704cef080 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.622+0000 7fa7e7dc5740 1 --2- 172.21.15.43:0/2083134658 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x563704c833f0 0x563704cef5c0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.622+0000 7fa7e7dc5740 1 --2- 172.21.15.43:0/2083134658 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563704c726b0 0x563704c88830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.622+0000 7fa7e7dc5740 1 --2- 172.21.15.43:0/2083134658 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563704c41920 0x563704c882f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.622+0000 7fa7bdffb640 0 client.6135 ms_handle_reset on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:28.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.622+0000 7fa7e7dc5740 1 -- 172.21.15.43:0/2083134658 >> 172.21.15.43:0/2083134658 conn(0x563704c48390 msgr2=0x563704c9d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:28.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.622+0000 7fa7e7dc5740 1 -- 172.21.15.43:0/2083134658 shutdown_connections 2024-04-25T08:57:28.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.622+0000 7fa7e7dc5740 1 -- 172.21.15.43:0/2083134658 wait complete. 2024-04-25T08:57:28.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.622+0000 7fa7e7dc5740 20 client.6135 trim_cache size 0 max 16384 2024-04-25T08:57:28.618 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T08:57:28.619 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": -2 2024-04-25T08:57:28.619 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T08:57:28.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.622+0000 7fa7e50c6640 1 -- 172.21.15.43:0/433571809 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa7c006b4d0 msgr2=0x7fa7c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:28.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.623+0000 7fa7e50c6640 1 --2- 172.21.15.43:0/433571809 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa7c006b4d0 0x7fa7c006d990 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0007fd0 tx=0x7fa7d0007e80 comp rx=0 tx=0).stop 2024-04-25T08:57:28.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.623+0000 7fa7e50c6640 1 -- 172.21.15.43:0/433571809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e015dec0 msgr2=0x7fa7e00b3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:28.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.623+0000 7fa7e50c6640 1 --2- 172.21.15.43:0/433571809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e015dec0 0x7fa7e00b3990 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8009e50 tx=0x7fa7d8004a40 comp rx=0 tx=0).stop 2024-04-25T08:57:28.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.623+0000 7fa7e50c6640 1 -- 172.21.15.43:0/433571809 shutdown_connections 2024-04-25T08:57:28.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.623+0000 7fa7e50c6640 1 --2- 172.21.15.43:0/433571809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7e015dec0 0x7fa7e00b3990 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.623+0000 7fa7e50c6640 1 --2- 172.21.15.43:0/433571809 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7e00d27b0 0x7fa7e00b2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.623+0000 7fa7e50c6640 1 --2- 172.21.15.43:0/433571809 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa7c006b4d0 0x7fa7c006d990 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.623+0000 7fa7e50c6640 1 --2- 172.21.15.43:0/433571809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7e00d04a0 0x7fa7e00cd640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.623+0000 7fa7e50c6640 1 -- 172.21.15.43:0/433571809 >> 172.21.15.43:0/433571809 conn(0x7fa7e009cd40 msgr2=0x7fa7e006ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:28.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.623+0000 7fa7e50c6640 1 -- 172.21.15.43:0/433571809 shutdown_connections 2024-04-25T08:57:28.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.623+0000 7fa7e50c6640 1 -- 172.21.15.43:0/433571809 wait complete. 2024-04-25T08:57:28.623 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/i/dont/exist' returned '{'return_code': -2}' 2024-04-25T08:57:28.623 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites' 2024-04-25T08:57:28.623 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:57:28.623 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites 2024-04-25T08:57:28.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.737+0000 7fa34d45a640 1 Processor -- start 2024-04-25T08:57:28.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.737+0000 7fa34d45a640 1 -- start start 2024-04-25T08:57:28.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.738+0000 7fa34d45a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa34815e090 0x7fa348161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.738+0000 7fa34d45a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa348162610 con 0x7fa3480b5c70 2024-04-25T08:57:28.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.738+0000 7fa34d45a640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3480d1750 con 0x7fa348161fe0 2024-04-25T08:57:28.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.738+0000 7fa34d45a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3480d1da0 con 0x7fa34815e090 2024-04-25T08:57:28.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.738+0000 7fa346ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa34815e090 0x7fa348161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.738+0000 7fa3477fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa3480b5c70 0x7fa3480d2230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49726/0 (socket says 172.21.15.43:49726) 2024-04-25T08:57:28.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.738+0000 7fa3477fe640 1 -- 172.21.15.43:0/4132776863 learned_addr learned my addr 172.21.15.43:0/4132776863 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:28.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.738+0000 7fa346ffd640 1 -- 172.21.15.43:0/4132776863 >> v1:172.21.15.193:6789/0 conn(0x7fa348161fe0 legacy=0x7fa3480b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:28.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.738+0000 7fa346ffd640 1 -- 172.21.15.43:0/4132776863 >> v1:172.21.15.43:6789/0 conn(0x7fa3480b5c70 legacy=0x7fa3480d2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:28.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.738+0000 7fa345ffb640 1 -- 172.21.15.43:0/4132776863 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 307838236 0 0) 0x7fa348162610 con 0x7fa3480b5c70 2024-04-25T08:57:28.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.738+0000 7fa346ffd640 1 -- 172.21.15.43:0/4132776863 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3480d1a30 con 0x7fa34815e090 2024-04-25T08:57:28.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.739+0000 7fa346ffd640 1 --2- 172.21.15.43:0/4132776863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa34815e090 0x7fa348161aa0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa3380097a0 tx=0x7fa33802a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7386892a3d162488 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:28.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.739+0000 7fa345ffb640 1 -- 172.21.15.43:0/4132776863 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa338037070 con 0x7fa34815e090 2024-04-25T08:57:28.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.739+0000 7fa345ffb640 1 -- 172.21.15.43:0/4132776863 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa338032640 con 0x7fa34815e090 2024-04-25T08:57:28.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.739+0000 7fa345ffb640 1 -- 172.21.15.43:0/4132776863 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa338032a10 con 0x7fa34815e090 2024-04-25T08:57:28.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.740+0000 7fa34d45a640 1 -- 172.21.15.43:0/4132776863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa34815e090 msgr2=0x7fa348161aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:28.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.740+0000 7fa34d45a640 1 --2- 172.21.15.43:0/4132776863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa34815e090 0x7fa348161aa0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa3380097a0 tx=0x7fa33802a2c0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.740+0000 7fa34d45a640 1 -- 172.21.15.43:0/4132776863 shutdown_connections 2024-04-25T08:57:28.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.740+0000 7fa34d45a640 1 --2- 172.21.15.43:0/4132776863 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa34815e090 0x7fa348161aa0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.740+0000 7fa34d45a640 1 -- 172.21.15.43:0/4132776863 >> 172.21.15.43:0/4132776863 conn(0x7fa3480d0560 msgr2=0x7fa3480d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:28.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.740+0000 7fa34d45a640 1 -- 172.21.15.43:0/4132776863 shutdown_connections 2024-04-25T08:57:28.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.740+0000 7fa34d45a640 1 -- 172.21.15.43:0/4132776863 wait complete. 2024-04-25T08:57:28.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.741+0000 7fa34d45a640 1 Processor -- start 2024-04-25T08:57:28.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.741+0000 7fa34d45a640 1 -- start start 2024-04-25T08:57:28.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.742+0000 7fa34d45a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3480b5c70 0x7fa3480b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.742+0000 7fa34d45a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa34815e090 0x7fa3480cde30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.742+0000 7fa34d45a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa348161fe0 0x7fa3480ce370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.742+0000 7fa34d45a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3480d2940 con 0x7fa348161fe0 2024-04-25T08:57:28.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.742+0000 7fa34d45a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3480b3280 con 0x7fa3480b5c70 2024-04-25T08:57:28.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.742+0000 7fa34d45a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3480d2ac0 con 0x7fa34815e090 2024-04-25T08:57:28.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.742+0000 7fa3477fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa348161fe0 0x7fa3480ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.742+0000 7fa3477fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa348161fe0 0x7fa3480ce370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42490/0 (socket says 172.21.15.43:42490) 2024-04-25T08:57:28.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.742+0000 7fa3477fe640 1 -- 172.21.15.43:0/177062436 learned_addr learned my addr 172.21.15.43:0/177062436 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:28.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.742+0000 7fa346ffd640 1 --2- 172.21.15.43:0/177062436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3480b5c70 0x7fa3480b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.742+0000 7fa3467fc640 1 --2- 172.21.15.43:0/177062436 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa34815e090 0x7fa3480cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.742+0000 7fa3477fe640 1 -- 172.21.15.43:0/177062436 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa34815e090 msgr2=0x7fa3480cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:28.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.742+0000 7fa3477fe640 1 --2- 172.21.15.43:0/177062436 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa34815e090 0x7fa3480cde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.742+0000 7fa3477fe640 1 -- 172.21.15.43:0/177062436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3480b5c70 msgr2=0x7fa3480b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:28.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.742+0000 7fa3477fe640 1 --2- 172.21.15.43:0/177062436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3480b5c70 0x7fa3480b1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.742+0000 7fa3477fe640 1 -- 172.21.15.43:0/177062436 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa348160c00 con 0x7fa348161fe0 2024-04-25T08:57:28.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.743+0000 7fa3477fe640 1 --2- 172.21.15.43:0/177062436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa348161fe0 0x7fa3480ce370 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa34000c7a0 tx=0x7fa34000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:28.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.743+0000 7fa32f7fe640 1 -- 172.21.15.43:0/177062436 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa340018070 con 0x7fa348161fe0 2024-04-25T08:57:28.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.743+0000 7fa32f7fe640 1 -- 172.21.15.43:0/177062436 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa34000a640 con 0x7fa348161fe0 2024-04-25T08:57:28.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.743+0000 7fa34d45a640 1 -- 172.21.15.43:0/177062436 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa348149120 con 0x7fa348161fe0 2024-04-25T08:57:28.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.743+0000 7fa32f7fe640 1 -- 172.21.15.43:0/177062436 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34000aab0 con 0x7fa348161fe0 2024-04-25T08:57:28.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.743+0000 7fa34d45a640 1 -- 172.21.15.43:0/177062436 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3481496e0 con 0x7fa348161fe0 2024-04-25T08:57:28.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.744+0000 7fa32f7fe640 1 -- 172.21.15.43:0/177062436 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa34001b070 con 0x7fa348161fe0 2024-04-25T08:57:28.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.745+0000 7fa32f7fe640 1 --2- 172.21.15.43:0/177062436 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa33406b4d0 0x7fa33406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.745+0000 7fa32f7fe640 1 -- 172.21.15.43:0/177062436 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa3400897f0 con 0x7fa348161fe0 2024-04-25T08:57:28.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.745+0000 7fa346ffd640 1 --2- 172.21.15.43:0/177062436 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa33406b4d0 0x7fa33406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.745+0000 7fa346ffd640 1 --2- 172.21.15.43:0/177062436 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa33406b4d0 0x7fa33406d990 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa338034000 tx=0x7fa33802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:28.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.746+0000 7fa350234740 1 Processor -- start 2024-04-25T08:57:28.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.746+0000 7fa350234740 1 -- start start 2024-04-25T08:57:28.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.746+0000 7fa350234740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb55f7f10 0x55fcb55fa070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.746+0000 7fa350234740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55fcb56345e0 0x55fcb55f82f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.747+0000 7fa350234740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcb56229d0 0x55fcb5622e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.747+0000 7fa350234740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcb5529fe0 con 0x55fcb55f7f10 2024-04-25T08:57:28.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.747+0000 7fa350234740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcb555bd20 con 0x55fcb56229d0 2024-04-25T08:57:28.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.747+0000 7fa350234740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fcb5504fa0 con 0x55fcb56345e0 2024-04-25T08:57:28.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.747+0000 7fa346ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb55f7f10 0x55fcb55fa070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.747+0000 7fa346ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb55f7f10 0x55fcb55fa070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42496/0 (socket says 172.21.15.43:42496) 2024-04-25T08:57:28.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.747+0000 7fa346ffd640 1 -- 172.21.15.43:0/659967036 learned_addr learned my addr 172.21.15.43:0/659967036 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:28.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.747+0000 7fa3477fe640 1 --2- 172.21.15.43:0/659967036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcb56229d0 0x55fcb5622e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.747+0000 7fa3467fc640 1 --2- 172.21.15.43:0/659967036 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55fcb56345e0 0x55fcb55f82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.747+0000 7fa346ffd640 1 -- 172.21.15.43:0/659967036 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55fcb56345e0 msgr2=0x55fcb55f82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:28.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.747+0000 7fa346ffd640 1 --2- 172.21.15.43:0/659967036 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55fcb56345e0 0x55fcb55f82f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.747+0000 7fa346ffd640 1 -- 172.21.15.43:0/659967036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcb56229d0 msgr2=0x55fcb5622e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:28.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.747+0000 7fa346ffd640 1 --2- 172.21.15.43:0/659967036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcb56229d0 0x55fcb5622e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.747+0000 7fa346ffd640 1 -- 172.21.15.43:0/659967036 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fcb533cb00 con 0x55fcb55f7f10 2024-04-25T08:57:28.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.747+0000 7fa3477fe640 1 --2- 172.21.15.43:0/659967036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcb56229d0 0x55fcb5622e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:57:28.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.747+0000 7fa346ffd640 1 --2- 172.21.15.43:0/659967036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb55f7f10 0x55fcb55fa070 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa338000c00 tx=0x7fa33802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c801011a53eeea0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:28.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.747+0000 7fa32cff9640 1 -- 172.21.15.43:0/659967036 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa338037040 con 0x55fcb55f7f10 2024-04-25T08:57:28.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.747+0000 7fa32cff9640 1 -- 172.21.15.43:0/659967036 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa33804e070 con 0x55fcb55f7f10 2024-04-25T08:57:28.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.747+0000 7fa32cff9640 1 -- 172.21.15.43:0/659967036 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33803c680 con 0x55fcb55f7f10 2024-04-25T08:57:28.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.748+0000 7fa350234740 1 -- 172.21.15.43:0/659967036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb55f7f10 msgr2=0x55fcb55fa070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:28.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.748+0000 7fa350234740 1 --2- 172.21.15.43:0/659967036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb55f7f10 0x55fcb55fa070 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa338000c00 tx=0x7fa33802c000 comp rx=0 tx=0).stop 2024-04-25T08:57:28.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.748+0000 7fa350234740 1 -- 172.21.15.43:0/659967036 shutdown_connections 2024-04-25T08:57:28.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.748+0000 7fa350234740 1 --2- 172.21.15.43:0/659967036 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcb56229d0 0x55fcb5622e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.748+0000 7fa350234740 1 --2- 172.21.15.43:0/659967036 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55fcb56345e0 0x55fcb55f82f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.748+0000 7fa350234740 1 --2- 172.21.15.43:0/659967036 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb55f7f10 0x55fcb55fa070 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.748+0000 7fa350234740 1 -- 172.21.15.43:0/659967036 >> 172.21.15.43:0/659967036 conn(0x55fcb55c0100 msgr2=0x55fcb563d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:28.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.748+0000 7fa350234740 1 -- 172.21.15.43:0/659967036 shutdown_connections 2024-04-25T08:57:28.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.748+0000 7fa350234740 1 -- 172.21.15.43:0/659967036 wait complete. 2024-04-25T08:57:28.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.749+0000 7fa350234740 1 -- start start 2024-04-25T08:57:28.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.749+0000 7fa350234740 1 Processor -- start 2024-04-25T08:57:28.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.750+0000 7fa350234740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55fcb55f7f10 0x55fcb5615290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.750+0000 7fa350234740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcb56229d0 0x55fcb56249b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.750+0000 7fa350234740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb56345e0 0x55fcb563a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.750+0000 7fa350234740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcb54fa750 con 0x55fcb56345e0 2024-04-25T08:57:28.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.750+0000 7fa350234740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcb5557dc0 con 0x55fcb56229d0 2024-04-25T08:57:28.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.750+0000 7fa350234740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fcb564dbb0 con 0x55fcb55f7f10 2024-04-25T08:57:28.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.750+0000 7fa350234740 10 client.0 _ll_register_callbacks cb 0x55fcb5512ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:28.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.750+0000 7fa3477fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb56345e0 0x55fcb563a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.750+0000 7fa3477fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb56345e0 0x55fcb563a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42504/0 (socket says 172.21.15.43:42504) 2024-04-25T08:57:28.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.750+0000 7fa3477fe640 1 -- 172.21.15.43:0/1880738985 learned_addr learned my addr 172.21.15.43:0/1880738985 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:28.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.750+0000 7fa3467fc640 1 --2- 172.21.15.43:0/1880738985 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcb56229d0 0x55fcb56249b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.750+0000 7fa346ffd640 1 --2- 172.21.15.43:0/1880738985 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55fcb55f7f10 0x55fcb5615290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.750+0000 7fa3477fe640 1 -- 172.21.15.43:0/1880738985 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55fcb55f7f10 msgr2=0x55fcb5615290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:28.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.750+0000 7fa3477fe640 1 --2- 172.21.15.43:0/1880738985 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55fcb55f7f10 0x55fcb5615290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.750+0000 7fa3477fe640 1 -- 172.21.15.43:0/1880738985 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcb56229d0 msgr2=0x55fcb56249b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:28.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.750+0000 7fa3477fe640 1 --2- 172.21.15.43:0/1880738985 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcb56229d0 0x55fcb56249b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.750+0000 7fa3477fe640 1 -- 172.21.15.43:0/1880738985 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fcb52303f0 con 0x55fcb56345e0 2024-04-25T08:57:28.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.750+0000 7fa3467fc640 1 --2- 172.21.15.43:0/1880738985 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcb56229d0 0x55fcb56249b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:28.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.750+0000 7fa3477fe640 1 --2- 172.21.15.43:0/1880738985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb56345e0 0x55fcb563a660 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa340057d00 tx=0x7fa340060660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=515658abb256be1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:28.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.750+0000 7fa325ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:57:28.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.750+0000 7fa325ffb640 1 -- 172.21.15.43:0/1880738985 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34000ae60 con 0x55fcb56345e0 2024-04-25T08:57:28.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.750+0000 7fa325ffb640 1 -- 172.21.15.43:0/1880738985 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa34005e270 con 0x55fcb56345e0 2024-04-25T08:57:28.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.750+0000 7fa350234740 1 -- 172.21.15.43:0/1880738985 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa33804e070 con 0x55fcb56345e0 2024-04-25T08:57:28.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.750+0000 7fa325ffb640 1 -- 172.21.15.43:0/1880738985 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa340057450 con 0x55fcb56345e0 2024-04-25T08:57:28.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.750+0000 7fa350234740 10 client.6150 fetch_fsmap learned FSMap version 22 2024-04-25T08:57:28.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.750+0000 7fa350234740 1 -- 172.21.15.43:0/1880738985 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x55fcb539d290 con 0x55fcb56345e0 2024-04-25T08:57:28.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.751+0000 7fa325ffb640 1 -- 172.21.15.43:0/1880738985 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa33804e070 con 0x55fcb56345e0 2024-04-25T08:57:28.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.751+0000 7fa325ffb640 1 -- 172.21.15.43:0/1880738985 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa34006fce0 con 0x55fcb56345e0 2024-04-25T08:57:28.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.751+0000 7fa350234740 10 client.6150 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T08:57:28.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.751+0000 7fa350234740 10 client.6150 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T08:57:28.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.751+0000 7fa350234740 20 client.6150 populate_metadata read hostname 'smithi043' 2024-04-25T08:57:28.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.751+0000 7fa350234740 1 --2- 172.21.15.43:0/1880738985 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55fcb5650ef0 0x55fcb5651300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.751+0000 7fa350234740 4 client.6150 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:57:28.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.751+0000 7fa350234740 1 -- 172.21.15.43:0/1880738985 --> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fcb5569260 con 0x55fcb5650ef0 2024-04-25T08:57:28.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.751+0000 7fa346ffd640 1 --2- 172.21.15.43:0/1880738985 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55fcb5650ef0 0x55fcb5651300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.751+0000 7fa325ffb640 10 client.6150 ms_handle_connect on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:28.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.751+0000 7fa346ffd640 1 --2- 172.21.15.43:0/1880738985 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55fcb5650ef0 0x55fcb5651300 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9df3348f9c354f30 server_cookie=1e28f33dac4f6a83 in_seq=0 out_seq=0 2024-04-25T08:57:28.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.752+0000 7fa325ffb640 1 -- 172.21.15.43:0/1880738985 <== mds.0 v2:172.21.15.193:6836/4164634380 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fcb5569260 con 0x55fcb5650ef0 2024-04-25T08:57:28.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.752+0000 7fa325ffb640 10 client.6150 handle_command_reply: tid=0 2024-04-25T08:57:28.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.752+0000 7fa350234740 1 client.6150 shutdown 2024-04-25T08:57:28.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.752+0000 7fa350234740 1 -- 172.21.15.43:0/1880738985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb56345e0 msgr2=0x55fcb563a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:28.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.752+0000 7fa350234740 1 --2- 172.21.15.43:0/1880738985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb56345e0 0x55fcb563a660 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa340057d00 tx=0x7fa340060660 comp rx=0 tx=0).stop 2024-04-25T08:57:28.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.752+0000 7fa350234740 1 -- 172.21.15.43:0/1880738985 shutdown_connections 2024-04-25T08:57:28.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.752+0000 7fa350234740 1 --2- 172.21.15.43:0/1880738985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb56345e0 0x55fcb563a660 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.752+0000 7fa350234740 1 --2- 172.21.15.43:0/1880738985 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55fcb5650ef0 0x55fcb5651300 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.752+0000 7fa350234740 1 --2- 172.21.15.43:0/1880738985 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcb56229d0 0x55fcb56249b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.752+0000 7fa350234740 1 --2- 172.21.15.43:0/1880738985 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55fcb55f7f10 0x55fcb5615290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.752+0000 7fa325ffb640 0 client.6150 ms_handle_reset on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:28.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.752+0000 7fa350234740 1 -- 172.21.15.43:0/1880738985 >> 172.21.15.43:0/1880738985 conn(0x55fcb55c0100 msgr2=0x55fcb5640ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:28.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.752+0000 7fa350234740 1 -- 172.21.15.43:0/1880738985 shutdown_connections 2024-04-25T08:57:28.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.752+0000 7fa350234740 1 -- 172.21.15.43:0/1880738985 wait complete. 2024-04-25T08:57:28.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.752+0000 7fa350234740 20 client.6150 trim_cache size 0 max 16384 2024-04-25T08:57:28.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.768+0000 7fa350234740 1 Processor -- start 2024-04-25T08:57:28.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.768+0000 7fa350234740 1 -- start start 2024-04-25T08:57:28.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.768+0000 7fa350234740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb55f7f10 0x55fcb561ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.768+0000 7fa350234740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcb56229d0 0x55fcb56a2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.768+0000 7fa350234740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55fcb56345e0 0x55fcb56a33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.768+0000 7fa350234740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3481ffcb0 con 0x55fcb55f7f10 2024-04-25T08:57:28.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.768+0000 7fa350234740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3481fee30 con 0x55fcb56229d0 2024-04-25T08:57:28.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.768+0000 7fa350234740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3481fe830 con 0x55fcb56345e0 2024-04-25T08:57:28.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.768+0000 7fa346ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb55f7f10 0x55fcb561ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.768+0000 7fa346ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb55f7f10 0x55fcb561ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42516/0 (socket says 172.21.15.43:42516) 2024-04-25T08:57:28.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.768+0000 7fa346ffd640 1 -- 172.21.15.43:0/364280975 learned_addr learned my addr 172.21.15.43:0/364280975 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:28.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.769+0000 7fa346ffd640 1 -- 172.21.15.43:0/364280975 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55fcb56345e0 msgr2=0x55fcb56a33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:28.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.769+0000 7fa346ffd640 1 --2- 172.21.15.43:0/364280975 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55fcb56345e0 0x55fcb56a33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.769+0000 7fa346ffd640 1 -- 172.21.15.43:0/364280975 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcb56229d0 msgr2=0x55fcb56a2e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:28.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.769+0000 7fa346ffd640 1 --2- 172.21.15.43:0/364280975 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcb56229d0 0x55fcb56a2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.769+0000 7fa346ffd640 1 -- 172.21.15.43:0/364280975 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fcb52094b0 con 0x55fcb55f7f10 2024-04-25T08:57:28.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.769+0000 7fa346ffd640 1 --2- 172.21.15.43:0/364280975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb55f7f10 0x55fcb561ff50 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa338000c00 tx=0x7fa33803c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d15717dc1682f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:28.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.769+0000 7fa327fff640 1 -- 172.21.15.43:0/364280975 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa338037040 con 0x55fcb55f7f10 2024-04-25T08:57:28.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.769+0000 7fa327fff640 1 -- 172.21.15.43:0/364280975 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa33805f030 con 0x55fcb55f7f10 2024-04-25T08:57:28.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.769+0000 7fa327fff640 1 -- 172.21.15.43:0/364280975 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa33805b340 con 0x55fcb55f7f10 2024-04-25T08:57:28.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.769+0000 7fa350234740 1 -- 172.21.15.43:0/364280975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb55f7f10 msgr2=0x55fcb561ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:28.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.769+0000 7fa350234740 1 --2- 172.21.15.43:0/364280975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb55f7f10 0x55fcb561ff50 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa338000c00 tx=0x7fa33803c000 comp rx=0 tx=0).stop 2024-04-25T08:57:28.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.769+0000 7fa350234740 1 -- 172.21.15.43:0/364280975 shutdown_connections 2024-04-25T08:57:28.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.769+0000 7fa350234740 1 --2- 172.21.15.43:0/364280975 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55fcb56345e0 0x55fcb56a33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.769+0000 7fa350234740 1 --2- 172.21.15.43:0/364280975 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcb56229d0 0x55fcb56a2e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.769+0000 7fa350234740 1 --2- 172.21.15.43:0/364280975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb55f7f10 0x55fcb561ff50 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.769+0000 7fa350234740 1 -- 172.21.15.43:0/364280975 >> 172.21.15.43:0/364280975 conn(0x55fcb55c0100 msgr2=0x55fcb55f2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:28.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.770+0000 7fa350234740 1 -- 172.21.15.43:0/364280975 shutdown_connections 2024-04-25T08:57:28.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.770+0000 7fa350234740 1 -- 172.21.15.43:0/364280975 wait complete. 2024-04-25T08:57:28.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.770+0000 7fa350234740 1 -- start start 2024-04-25T08:57:28.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.770+0000 7fa350234740 1 Processor -- start 2024-04-25T08:57:28.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.771+0000 7fa350234740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb55f7f10 0x55fcb5689e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.771+0000 7fa350234740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55fcb56229d0 0x55fcb5633360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.771+0000 7fa350234740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcb56345e0 0x55fcb563ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.771+0000 7fa350234740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcb55f2d00 con 0x55fcb55f7f10 2024-04-25T08:57:28.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.771+0000 7fa350234740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fcb55f7800 con 0x55fcb56345e0 2024-04-25T08:57:28.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.771+0000 7fa350234740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fcb5633c90 con 0x55fcb56229d0 2024-04-25T08:57:28.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.771+0000 7fa350234740 10 client.0 _ll_register_callbacks cb 0x7fa338009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:28.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.771+0000 7fa346ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb55f7f10 0x55fcb5689e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.771+0000 7fa346ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb55f7f10 0x55fcb5689e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42518/0 (socket says 172.21.15.43:42518) 2024-04-25T08:57:28.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.771+0000 7fa346ffd640 1 -- 172.21.15.43:0/3793717543 learned_addr learned my addr 172.21.15.43:0/3793717543 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:28.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.771+0000 7fa3477fe640 1 --2- 172.21.15.43:0/3793717543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcb56345e0 0x55fcb563ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.771+0000 7fa3467fc640 1 --2- 172.21.15.43:0/3793717543 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55fcb56229d0 0x55fcb5633360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.771+0000 7fa346ffd640 1 -- 172.21.15.43:0/3793717543 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55fcb56229d0 msgr2=0x55fcb5633360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:28.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.771+0000 7fa346ffd640 1 --2- 172.21.15.43:0/3793717543 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55fcb56229d0 0x55fcb5633360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.771+0000 7fa346ffd640 1 -- 172.21.15.43:0/3793717543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcb56345e0 msgr2=0x55fcb563ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:28.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.771+0000 7fa346ffd640 1 --2- 172.21.15.43:0/3793717543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcb56345e0 0x55fcb563ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.771+0000 7fa346ffd640 1 -- 172.21.15.43:0/3793717543 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fcb563bf50 con 0x55fcb55f7f10 2024-04-25T08:57:28.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.771+0000 7fa346ffd640 1 --2- 172.21.15.43:0/3793717543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb55f7f10 0x55fcb5689e30 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa338035040 tx=0x7fa338004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c28fbc1dcad2d116 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:28.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.771+0000 7fa325ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:57:28.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.772+0000 7fa325ffb640 1 -- 172.21.15.43:0/3793717543 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa338037040 con 0x55fcb55f7f10 2024-04-25T08:57:28.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.772+0000 7fa325ffb640 1 -- 172.21.15.43:0/3793717543 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa33805bb80 con 0x55fcb55f7f10 2024-04-25T08:57:28.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.772+0000 7fa350234740 1 -- 172.21.15.43:0/3793717543 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa33805f030 con 0x55fcb55f7f10 2024-04-25T08:57:28.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.772+0000 7fa325ffb640 1 -- 172.21.15.43:0/3793717543 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa33805bd80 con 0x55fcb55f7f10 2024-04-25T08:57:28.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.772+0000 7fa350234740 10 client.6156 fetch_fsmap learned FSMap version 22 2024-04-25T08:57:28.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.772+0000 7fa350234740 1 -- 172.21.15.43:0/3793717543 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x55fcb56339d0 con 0x55fcb55f7f10 2024-04-25T08:57:28.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.772+0000 7fa325ffb640 1 -- 172.21.15.43:0/3793717543 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa33805f030 con 0x55fcb55f7f10 2024-04-25T08:57:28.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.772+0000 7fa325ffb640 1 -- 172.21.15.43:0/3793717543 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa33805b340 con 0x55fcb55f7f10 2024-04-25T08:57:28.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.772+0000 7fa350234740 10 client.6156 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T08:57:28.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.772+0000 7fa350234740 10 client.6156 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T08:57:28.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.772+0000 7fa350234740 20 client.6156 populate_metadata read hostname 'smithi043' 2024-04-25T08:57:28.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.772+0000 7fa350234740 1 --2- 172.21.15.43:0/3793717543 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55fcb5623520 0x55fcb5623930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.772+0000 7fa350234740 4 client.6156 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}] 2024-04-25T08:57:28.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.772+0000 7fa350234740 1 -- 172.21.15.43:0/3793717543 --> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}) v1 -- 0x55fcb55635b0 con 0x55fcb5623520 2024-04-25T08:57:28.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.772+0000 7fa3467fc640 1 --2- 172.21.15.43:0/3793717543 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55fcb5623520 0x55fcb5623930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.773+0000 7fa3467fc640 1 --2- 172.21.15.43:0/3793717543 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55fcb5623520 0x55fcb5623930 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d26f4865c63e1ef server_cookie=fc9baba7adda6bd0 in_seq=0 out_seq=0 2024-04-25T08:57:28.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.773+0000 7fa325ffb640 10 client.6156 ms_handle_connect on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:28.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.773+0000 7fa325ffb640 1 -- 172.21.15.43:0/3793717543 <== mds.0 v2:172.21.15.193:6836/4164634380 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55fcb55635b0 con 0x55fcb5623520 2024-04-25T08:57:28.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.773+0000 7fa325ffb640 10 client.6156 handle_command_reply: tid=0 2024-04-25T08:57:28.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.773+0000 7fa350234740 1 client.6156 shutdown 2024-04-25T08:57:28.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.773+0000 7fa350234740 1 -- 172.21.15.43:0/3793717543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb55f7f10 msgr2=0x55fcb5689e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:28.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.773+0000 7fa350234740 1 --2- 172.21.15.43:0/3793717543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb55f7f10 0x55fcb5689e30 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa338035040 tx=0x7fa338004730 comp rx=0 tx=0).stop 2024-04-25T08:57:28.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.773+0000 7fa350234740 1 -- 172.21.15.43:0/3793717543 shutdown_connections 2024-04-25T08:57:28.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.773+0000 7fa350234740 1 --2- 172.21.15.43:0/3793717543 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55fcb5623520 0x55fcb5623930 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.773+0000 7fa350234740 1 --2- 172.21.15.43:0/3793717543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fcb56345e0 0x55fcb563ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.773+0000 7fa350234740 1 --2- 172.21.15.43:0/3793717543 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55fcb56229d0 0x55fcb5633360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.773+0000 7fa325ffb640 0 client.6156 ms_handle_reset on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:28.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.773+0000 7fa350234740 1 --2- 172.21.15.43:0/3793717543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fcb55f7f10 0x55fcb5689e30 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.774+0000 7fa350234740 1 -- 172.21.15.43:0/3793717543 >> 172.21.15.43:0/3793717543 conn(0x55fcb55c0100 msgr2=0x55fcb55f9ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:28.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.774+0000 7fa350234740 1 -- 172.21.15.43:0/3793717543 shutdown_connections 2024-04-25T08:57:28.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.774+0000 7fa350234740 1 -- 172.21.15.43:0/3793717543 wait complete. 2024-04-25T08:57:28.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.774+0000 7fa350234740 20 client.6156 trim_cache size 0 max 16384 2024-04-25T08:57:28.769 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T08:57:28.769 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-25T08:57:28.769 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "4e9f1257-348e-464d-8ed3-b6c639b655ec", 2024-04-25T08:57:28.769 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-25T08:57:28.769 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T08:57:28.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.774+0000 7fa34d45a640 1 -- 172.21.15.43:0/177062436 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa33406b4d0 msgr2=0x7fa33406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:28.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.774+0000 7fa34d45a640 1 --2- 172.21.15.43:0/177062436 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa33406b4d0 0x7fa33406d990 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa338034000 tx=0x7fa33802e000 comp rx=0 tx=0).stop 2024-04-25T08:57:28.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.774+0000 7fa34d45a640 1 -- 172.21.15.43:0/177062436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa348161fe0 msgr2=0x7fa3480ce370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:28.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.774+0000 7fa34d45a640 1 --2- 172.21.15.43:0/177062436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa348161fe0 0x7fa3480ce370 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa34000c7a0 tx=0x7fa34000cc60 comp rx=0 tx=0).stop 2024-04-25T08:57:28.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.774+0000 7fa34d45a640 1 -- 172.21.15.43:0/177062436 shutdown_connections 2024-04-25T08:57:28.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.774+0000 7fa34d45a640 1 --2- 172.21.15.43:0/177062436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa348161fe0 0x7fa3480ce370 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.774+0000 7fa34d45a640 1 --2- 172.21.15.43:0/177062436 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa34815e090 0x7fa3480cde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.774+0000 7fa34d45a640 1 --2- 172.21.15.43:0/177062436 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa33406b4d0 0x7fa33406d990 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.774+0000 7fa34d45a640 1 --2- 172.21.15.43:0/177062436 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3480b5c70 0x7fa3480b1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.774+0000 7fa34d45a640 1 -- 172.21.15.43:0/177062436 >> 172.21.15.43:0/177062436 conn(0x7fa3480d0560 msgr2=0x7fa3480cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:28.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.774+0000 7fa34d45a640 1 -- 172.21.15.43:0/177062436 shutdown_connections 2024-04-25T08:57:28.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.774+0000 7fa34d45a640 1 -- 172.21.15.43:0/177062436 wait complete. 2024-04-25T08:57:28.774 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites' returned '{'return_code': 0, 'scrub_tag': '4e9f1257-348e-464d-8ed3-b6c639b655ec', 'mode': 'asynchronous'}' 2024-04-25T08:57:28.774 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2024-04-25T08:57:28.774 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:57:28.775 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T08:57:28.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.889+0000 7f09d14b3640 1 Processor -- start 2024-04-25T08:57:28.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.889+0000 7f09d14b3640 1 -- start start 2024-04-25T08:57:28.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.890+0000 7f09d14b3640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f09cc0d04a0 0x7f09cc15ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.890+0000 7f09d14b3640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09cc0cf110 con 0x7f09cc0ce8c0 2024-04-25T08:57:28.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.890+0000 7f09d14b3640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09cc0cf3f0 con 0x7f09cc15dec0 2024-04-25T08:57:28.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.890+0000 7f09d14b3640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09cc0cf6d0 con 0x7f09cc0d04a0 2024-04-25T08:57:28.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.890+0000 7f09cb7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f09cc0ce8c0 0x7f09cc0d17a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49738/0 (socket says 172.21.15.43:49738) 2024-04-25T08:57:28.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.890+0000 7f09cb7fe640 1 -- 172.21.15.43:0/412125363 learned_addr learned my addr 172.21.15.43:0/412125363 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:28.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.890+0000 7f09caffd640 1 --2- 172.21.15.43:0/412125363 >> v2:172.21.15.193:3300/0 conn(0x7f09cc0d04a0 0x7f09cc15ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.890+0000 7f09c9ffb640 1 -- 172.21.15.43:0/412125363 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2929657819 0 0) 0x7f09cc0cf110 con 0x7f09cc0ce8c0 2024-04-25T08:57:28.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.890+0000 7f09c9ffb640 1 -- 172.21.15.43:0/412125363 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09b4003520 con 0x7f09cc0ce8c0 2024-04-25T08:57:28.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.890+0000 7f09c9ffb640 1 -- 172.21.15.43:0/412125363 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 483217449 0 0) 0x7f09b4003520 con 0x7f09cc0ce8c0 2024-04-25T08:57:28.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.890+0000 7f09c9ffb640 1 -- 172.21.15.43:0/412125363 >> v2:172.21.15.193:3300/0 conn(0x7f09cc0d04a0 msgr2=0x7f09cc15ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:28.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.890+0000 7f09c9ffb640 1 --2- 172.21.15.43:0/412125363 >> v2:172.21.15.193:3300/0 conn(0x7f09cc0d04a0 0x7f09cc15ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.890+0000 7f09c9ffb640 1 -- 172.21.15.43:0/412125363 >> v1:172.21.15.193:6789/0 conn(0x7f09cc15dec0 legacy=0x7f09cc0d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:28.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.890+0000 7f09c9ffb640 1 -- 172.21.15.43:0/412125363 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09cc0cf9b0 con 0x7f09cc0ce8c0 2024-04-25T08:57:28.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.890+0000 7f09c9ffb640 1 -- 172.21.15.43:0/412125363 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f09c40019f0 con 0x7f09cc0ce8c0 2024-04-25T08:57:28.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.891+0000 7f09c9ffb640 1 -- 172.21.15.43:0/412125363 >> v1:172.21.15.43:6789/0 conn(0x7f09cc0ce8c0 legacy=0x7f09cc0d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:28.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.891+0000 7f09c9ffb640 1 --2- 172.21.15.43:0/412125363 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09b40039b0 0x7f09b4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.891+0000 7f09c9ffb640 1 --2- 172.21.15.43:0/412125363 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f09b4004800 0x7f09b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.891+0000 7f09c9ffb640 1 --2- 172.21.15.43:0/412125363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09b40051c0 0x7f09b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.891+0000 7f09c9ffb640 1 -- 172.21.15.43:0/412125363 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc0cf6d0 con 0x7f09b40051c0 2024-04-25T08:57:28.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.891+0000 7f09c9ffb640 1 -- 172.21.15.43:0/412125363 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b400c060 con 0x7f09b40039b0 2024-04-25T08:57:28.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.891+0000 7f09c9ffb640 1 -- 172.21.15.43:0/412125363 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b400c260 con 0x7f09b4004800 2024-04-25T08:57:28.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.891+0000 7f09cb7fe640 1 --2- 172.21.15.43:0/412125363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09b40051c0 0x7f09b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.891+0000 7f09caffd640 1 --2- 172.21.15.43:0/412125363 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09b40039b0 0x7f09b4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.891+0000 7f09ca7fc640 1 --2- 172.21.15.43:0/412125363 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f09b4004800 0x7f09b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.891+0000 7f09cb7fe640 1 -- 172.21.15.43:0/412125363 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f09b4004800 msgr2=0x7f09b4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:28.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.891+0000 7f09cb7fe640 1 --2- 172.21.15.43:0/412125363 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f09b4004800 0x7f09b4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.891+0000 7f09cb7fe640 1 -- 172.21.15.43:0/412125363 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09b40039b0 msgr2=0x7f09b4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:28.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.891+0000 7f09cb7fe640 1 --2- 172.21.15.43:0/412125363 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09b40039b0 0x7f09b4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.891+0000 7f09cb7fe640 1 -- 172.21.15.43:0/412125363 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b4003520 con 0x7f09b40051c0 2024-04-25T08:57:28.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.892+0000 7f09cb7fe640 1 -- 172.21.15.43:0/412125363 reap_dead start 2024-04-25T08:57:28.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.892+0000 7f09cb7fe640 1 --2- 172.21.15.43:0/412125363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09b40051c0 0x7f09b400b9c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f09c400a860 tx=0x7f09b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9724c7ba82d891fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:28.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.892+0000 7f09c9ffb640 1 -- 172.21.15.43:0/412125363 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c400a480 con 0x7f09b40051c0 2024-04-25T08:57:28.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.892+0000 7f09c9ffb640 1 -- 172.21.15.43:0/412125363 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c402f070 con 0x7f09b40051c0 2024-04-25T08:57:28.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.892+0000 7f09c9ffb640 1 -- 172.21.15.43:0/412125363 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c402bcc0 con 0x7f09b40051c0 2024-04-25T08:57:28.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.892+0000 7f09d14b3640 1 -- 172.21.15.43:0/412125363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09b40051c0 msgr2=0x7f09b400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:28.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.892+0000 7f09d14b3640 1 --2- 172.21.15.43:0/412125363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09b40051c0 0x7f09b400b9c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f09c400a860 tx=0x7f09b4005830 comp rx=0 tx=0).stop 2024-04-25T08:57:28.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.892+0000 7f09d14b3640 1 -- 172.21.15.43:0/412125363 shutdown_connections 2024-04-25T08:57:28.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.892+0000 7f09d14b3640 1 --2- 172.21.15.43:0/412125363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09b40051c0 0x7f09b400b9c0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.892+0000 7f09d14b3640 1 -- 172.21.15.43:0/412125363 >> 172.21.15.43:0/412125363 conn(0x7f09cc09cd40 msgr2=0x7f09cc0d3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:28.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.892+0000 7f09d14b3640 1 -- 172.21.15.43:0/412125363 shutdown_connections 2024-04-25T08:57:28.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.892+0000 7f09d14b3640 1 -- 172.21.15.43:0/412125363 wait complete. 2024-04-25T08:57:28.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.894+0000 7f09d14b3640 1 Processor -- start 2024-04-25T08:57:28.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.894+0000 7f09d14b3640 1 -- start start 2024-04-25T08:57:28.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.894+0000 7f09d14b3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09b40051c0 0x7f09cc0b8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.894+0000 7f09d14b3640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f09cc06c920 0x7f09cc0b68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.894+0000 7f09d14b3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09cc1612f0 0x7f09cc161770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.894+0000 7f09d14b3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc0cee50 con 0x7f09b40051c0 2024-04-25T08:57:28.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.895+0000 7f09d14b3640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc0cecd0 con 0x7f09cc1612f0 2024-04-25T08:57:28.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.895+0000 7f09d14b3640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc15ed90 con 0x7f09cc06c920 2024-04-25T08:57:28.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.895+0000 7f09caffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09b40051c0 0x7f09cc0b8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.895+0000 7f09caffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09b40051c0 0x7f09cc0b8450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42528/0 (socket says 172.21.15.43:42528) 2024-04-25T08:57:28.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.895+0000 7f09caffd640 1 -- 172.21.15.43:0/4163444025 learned_addr learned my addr 172.21.15.43:0/4163444025 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:28.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.895+0000 7f09cb7fe640 1 --2- 172.21.15.43:0/4163444025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09cc1612f0 0x7f09cc161770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.895+0000 7f09ca7fc640 1 --2- 172.21.15.43:0/4163444025 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f09cc06c920 0x7f09cc0b68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.895+0000 7f09caffd640 1 -- 172.21.15.43:0/4163444025 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f09cc06c920 msgr2=0x7f09cc0b68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:28.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.895+0000 7f09caffd640 1 --2- 172.21.15.43:0/4163444025 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f09cc06c920 0x7f09cc0b68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.895+0000 7f09caffd640 1 -- 172.21.15.43:0/4163444025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09cc1612f0 msgr2=0x7f09cc161770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:28.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.895+0000 7f09caffd640 1 --2- 172.21.15.43:0/4163444025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09cc1612f0 0x7f09cc161770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.895+0000 7f09caffd640 1 -- 172.21.15.43:0/4163444025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09cc0b9740 con 0x7f09b40051c0 2024-04-25T08:57:28.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.895+0000 7f09caffd640 1 --2- 172.21.15.43:0/4163444025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09b40051c0 0x7f09cc0b8450 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f09bc011910 tx=0x7f09bc011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:28.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.895+0000 7f09b37fe640 1 -- 172.21.15.43:0/4163444025 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc00fa90 con 0x7f09b40051c0 2024-04-25T08:57:28.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.895+0000 7f09b37fe640 1 -- 172.21.15.43:0/4163444025 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09bc00fc30 con 0x7f09b40051c0 2024-04-25T08:57:28.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.895+0000 7f09d14b3640 1 -- 172.21.15.43:0/4163444025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09cc0b8b30 con 0x7f09b40051c0 2024-04-25T08:57:28.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.895+0000 7f09b37fe640 1 -- 172.21.15.43:0/4163444025 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc014440 con 0x7f09b40051c0 2024-04-25T08:57:28.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.895+0000 7f09d14b3640 1 -- 172.21.15.43:0/4163444025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09cc0b90f0 con 0x7f09b40051c0 2024-04-25T08:57:28.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.897+0000 7f09b37fe640 1 -- 172.21.15.43:0/4163444025 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f09bc012040 con 0x7f09b40051c0 2024-04-25T08:57:28.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.897+0000 7f09b37fe640 1 --2- 172.21.15.43:0/4163444025 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f09b806b4d0 0x7f09b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.897+0000 7f09b37fe640 1 -- 172.21.15.43:0/4163444025 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f09bc08a710 con 0x7f09b40051c0 2024-04-25T08:57:28.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.897+0000 7f09ca7fc640 1 --2- 172.21.15.43:0/4163444025 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f09b806b4d0 0x7f09b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.898+0000 7f09ca7fc640 1 --2- 172.21.15.43:0/4163444025 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f09b806b4d0 0x7f09b806d990 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f09c0000c00 tx=0x7f09c0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:28.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.898+0000 7f09d41bb740 1 Processor -- start 2024-04-25T08:57:28.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.898+0000 7f09d41bb740 1 -- start start 2024-04-25T08:57:28.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.899+0000 7f09d41bb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587c2bd4d70 0x5587c2bd5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.899+0000 7f09d41bb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c2bca210 0x5587c2b89170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.899+0000 7f09d41bb740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5587c2bb88f0 0x5587c2b896b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.899+0000 7f09d41bb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587c2af1e60 con 0x5587c2bd4d70 2024-04-25T08:57:28.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.899+0000 7f09d41bb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587c2a9b120 con 0x5587c2bca210 2024-04-25T08:57:28.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.899+0000 7f09d41bb740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x5587c2aedf00 con 0x5587c2bb88f0 2024-04-25T08:57:28.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.899+0000 7f09caffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587c2bd4d70 0x5587c2bd5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.899+0000 7f09caffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587c2bd4d70 0x5587c2bd5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42538/0 (socket says 172.21.15.43:42538) 2024-04-25T08:57:28.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.899+0000 7f09caffd640 1 -- 172.21.15.43:0/2710908520 learned_addr learned my addr 172.21.15.43:0/2710908520 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:28.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.899+0000 7f09caffd640 1 -- 172.21.15.43:0/2710908520 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5587c2bb88f0 msgr2=0x5587c2b896b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:28.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.899+0000 7f09cb7fe640 1 --2- 172.21.15.43:0/2710908520 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5587c2bb88f0 0x5587c2b896b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.899+0000 7f09ca7fc640 1 --2- 172.21.15.43:0/2710908520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c2bca210 0x5587c2b89170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.899+0000 7f09caffd640 1 --2- 172.21.15.43:0/2710908520 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5587c2bb88f0 0x5587c2b896b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.899+0000 7f09caffd640 1 -- 172.21.15.43:0/2710908520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c2bca210 msgr2=0x5587c2b89170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:28.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.899+0000 7f09caffd640 1 --2- 172.21.15.43:0/2710908520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c2bca210 0x5587c2b89170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.899+0000 7f09caffd640 1 -- 172.21.15.43:0/2710908520 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587c28d2f00 con 0x5587c2bd4d70 2024-04-25T08:57:28.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.900+0000 7f09caffd640 1 --2- 172.21.15.43:0/2710908520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587c2bd4d70 0x5587c2bd5150 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f09bc061070 tx=0x7f09bc0596d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b72c9deac7bd545 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:28.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.900+0000 7f09b0ff9640 1 -- 172.21.15.43:0/2710908520 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc087070 con 0x5587c2bd4d70 2024-04-25T08:57:28.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.900+0000 7f09b0ff9640 1 -- 172.21.15.43:0/2710908520 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09bc06ce50 con 0x5587c2bd4d70 2024-04-25T08:57:28.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.900+0000 7f09b0ff9640 1 -- 172.21.15.43:0/2710908520 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc055500 con 0x5587c2bd4d70 2024-04-25T08:57:28.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.900+0000 7f09d41bb740 1 -- 172.21.15.43:0/2710908520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587c2bd4d70 msgr2=0x5587c2bd5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:28.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.900+0000 7f09d41bb740 1 --2- 172.21.15.43:0/2710908520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587c2bd4d70 0x5587c2bd5150 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f09bc061070 tx=0x7f09bc0596d0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.900+0000 7f09d41bb740 1 -- 172.21.15.43:0/2710908520 shutdown_connections 2024-04-25T08:57:28.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.900+0000 7f09d41bb740 1 --2- 172.21.15.43:0/2710908520 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5587c2bb88f0 0x5587c2b896b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.900+0000 7f09d41bb740 1 --2- 172.21.15.43:0/2710908520 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c2bca210 0x5587c2b89170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.900+0000 7f09d41bb740 1 --2- 172.21.15.43:0/2710908520 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587c2bd4d70 0x5587c2bd5150 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.900+0000 7f09d41bb740 1 -- 172.21.15.43:0/2710908520 >> 172.21.15.43:0/2710908520 conn(0x5587c2b55f70 msgr2=0x5587c2b8c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:28.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.901+0000 7f09d41bb740 1 -- 172.21.15.43:0/2710908520 shutdown_connections 2024-04-25T08:57:28.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.901+0000 7f09d41bb740 1 -- 172.21.15.43:0/2710908520 wait complete. 2024-04-25T08:57:28.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.901+0000 7f09d41bb740 1 -- start start 2024-04-25T08:57:28.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.901+0000 7f09d41bb740 1 Processor -- start 2024-04-25T08:57:28.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.902+0000 7f09d41bb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c2bb88f0 0x5587c2bd2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.902+0000 7f09d41bb740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5587c2bca210 0x5587c2bcad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.902+0000 7f09d41bb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587c2bd4d70 0x5587c2bd08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.902+0000 7f09d41bb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587c2bd5690 con 0x5587c2bd4d70 2024-04-25T08:57:28.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.902+0000 7f09d41bb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587c2a90cd0 con 0x5587c2bb88f0 2024-04-25T08:57:28.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.902+0000 7f09d41bb740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x5587c2bb8eb0 con 0x5587c2bca210 2024-04-25T08:57:28.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.902+0000 7f09d41bb740 10 client.0 _ll_register_callbacks cb 0x5587c28062e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:28.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.902+0000 7f09cb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587c2bd4d70 0x5587c2bd08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.902+0000 7f09cb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587c2bd4d70 0x5587c2bd08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42554/0 (socket says 172.21.15.43:42554) 2024-04-25T08:57:28.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.902+0000 7f09cb7fe640 1 -- 172.21.15.43:0/825067807 learned_addr learned my addr 172.21.15.43:0/825067807 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:28.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.902+0000 7f09caffd640 1 --2- 172.21.15.43:0/825067807 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c2bb88f0 0x5587c2bd2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.902+0000 7f09ca7fc640 1 --2- 172.21.15.43:0/825067807 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5587c2bca210 0x5587c2bcad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.902+0000 7f09cb7fe640 1 -- 172.21.15.43:0/825067807 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5587c2bca210 msgr2=0x5587c2bcad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:28.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.902+0000 7f09cb7fe640 1 --2- 172.21.15.43:0/825067807 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5587c2bca210 0x5587c2bcad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.902+0000 7f09cb7fe640 1 -- 172.21.15.43:0/825067807 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c2bb88f0 msgr2=0x5587c2bd2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:28.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.902+0000 7f09cb7fe640 1 --2- 172.21.15.43:0/825067807 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c2bb88f0 0x5587c2bd2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.902+0000 7f09cb7fe640 1 -- 172.21.15.43:0/825067807 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5587c27c67f0 con 0x5587c2bd4d70 2024-04-25T08:57:28.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.902+0000 7f09ca7fc640 1 --2- 172.21.15.43:0/825067807 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5587c2bca210 0x5587c2bcad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:28.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.903+0000 7f09cb7fe640 1 --2- 172.21.15.43:0/825067807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587c2bd4d70 0x5587c2bd08c0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f09c400a980 tx=0x7f09c402d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e35206892f11b9e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:28.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.903+0000 7f09a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:57:28.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.903+0000 7f09a9ffb640 1 -- 172.21.15.43:0/825067807 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c402b880 con 0x5587c2bd4d70 2024-04-25T08:57:28.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.903+0000 7f09a9ffb640 1 -- 172.21.15.43:0/825067807 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f09c402ba20 con 0x5587c2bd4d70 2024-04-25T08:57:28.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.903+0000 7f09d41bb740 1 -- 172.21.15.43:0/825067807 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f09bc06ce50 con 0x5587c2bd4d70 2024-04-25T08:57:28.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.903+0000 7f09a9ffb640 1 -- 172.21.15.43:0/825067807 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09c402bc20 con 0x5587c2bd4d70 2024-04-25T08:57:28.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.903+0000 7f09d41bb740 10 client.6168 fetch_fsmap learned FSMap version 22 2024-04-25T08:57:28.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.903+0000 7f09d41bb740 1 -- 172.21.15.43:0/825067807 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x5587c2933690 con 0x5587c2bd4d70 2024-04-25T08:57:28.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.903+0000 7f09a9ffb640 1 -- 172.21.15.43:0/825067807 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09bc06ce50 con 0x5587c2bd4d70 2024-04-25T08:57:28.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.903+0000 7f09a9ffb640 1 -- 172.21.15.43:0/825067807 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f09c404c020 con 0x5587c2bd4d70 2024-04-25T08:57:28.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.903+0000 7f09d41bb740 10 client.6168 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T08:57:28.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.903+0000 7f09d41bb740 10 client.6168 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T08:57:28.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.903+0000 7f09d41bb740 20 client.6168 populate_metadata read hostname 'smithi043' 2024-04-25T08:57:28.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.903+0000 7f09d41bb740 1 --2- 172.21.15.43:0/825067807 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5587c2beb0f0 0x5587c2beb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.903+0000 7f09d41bb740 4 client.6168 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:57:28.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.903+0000 7f09d41bb740 1 -- 172.21.15.43:0/825067807 --> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5587c2aff3a0 con 0x5587c2beb0f0 2024-04-25T08:57:28.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.903+0000 7f09caffd640 1 --2- 172.21.15.43:0/825067807 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5587c2beb0f0 0x5587c2beb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.904+0000 7f09a9ffb640 10 client.6168 ms_handle_connect on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:28.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.904+0000 7f09caffd640 1 --2- 172.21.15.43:0/825067807 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5587c2beb0f0 0x5587c2beb4d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64d59e0d75a24814 server_cookie=7f32d47e476ff2c9 in_seq=0 out_seq=0 2024-04-25T08:57:28.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.904+0000 7f09a9ffb640 1 -- 172.21.15.43:0/825067807 <== mds.0 v2:172.21.15.193:6836/4164634380 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5587c2aff3a0 con 0x5587c2beb0f0 2024-04-25T08:57:28.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.904+0000 7f09a9ffb640 10 client.6168 handle_command_reply: tid=0 2024-04-25T08:57:28.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.904+0000 7f09d41bb740 1 client.6168 shutdown 2024-04-25T08:57:28.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.904+0000 7f09d41bb740 1 -- 172.21.15.43:0/825067807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587c2bd4d70 msgr2=0x5587c2bd08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:28.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.904+0000 7f09d41bb740 1 --2- 172.21.15.43:0/825067807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587c2bd4d70 0x5587c2bd08c0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f09c400a980 tx=0x7f09c402d070 comp rx=0 tx=0).stop 2024-04-25T08:57:28.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.904+0000 7f09d41bb740 1 -- 172.21.15.43:0/825067807 shutdown_connections 2024-04-25T08:57:28.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.905+0000 7f09d41bb740 1 --2- 172.21.15.43:0/825067807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587c2bd4d70 0x5587c2bd08c0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.905+0000 7f09d41bb740 1 --2- 172.21.15.43:0/825067807 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5587c2bca210 0x5587c2bcad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.905+0000 7f09d41bb740 1 --2- 172.21.15.43:0/825067807 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5587c2beb0f0 0x5587c2beb4d0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.905+0000 7f09d41bb740 1 --2- 172.21.15.43:0/825067807 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c2bb88f0 0x5587c2bd2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.905+0000 7f09a9ffb640 0 client.6168 ms_handle_reset on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:28.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.905+0000 7f09d41bb740 1 -- 172.21.15.43:0/825067807 >> 172.21.15.43:0/825067807 conn(0x5587c2b55f70 msgr2=0x5587c2b89d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:28.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.905+0000 7f09d41bb740 1 -- 172.21.15.43:0/825067807 shutdown_connections 2024-04-25T08:57:28.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.905+0000 7f09d41bb740 1 -- 172.21.15.43:0/825067807 wait complete. 2024-04-25T08:57:28.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.905+0000 7f09d41bb740 20 client.6168 trim_cache size 0 max 16384 2024-04-25T08:57:28.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.920+0000 7f09d41bb740 1 Processor -- start 2024-04-25T08:57:28.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.920+0000 7f09d41bb740 1 -- start start 2024-04-25T08:57:28.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.921+0000 7f09d41bb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c2bb88f0 0x5587c2bb5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.921+0000 7f09d41bb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587c2bca210 0x5587c2b89bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.921+0000 7f09d41bb740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5587c2bd4d70 0x5587c2b8a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.921+0000 7f09d41bb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587c2bbce90 con 0x5587c2bca210 2024-04-25T08:57:28.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.921+0000 7f09d41bb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc070480 con 0x5587c2bb88f0 2024-04-25T08:57:28.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.921+0000 7f09d41bb740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc06f660 con 0x5587c2bd4d70 2024-04-25T08:57:28.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.921+0000 7f09ca7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587c2bca210 0x5587c2b89bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.921+0000 7f09ca7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587c2bca210 0x5587c2b89bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42560/0 (socket says 172.21.15.43:42560) 2024-04-25T08:57:28.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.921+0000 7f09ca7fc640 1 -- 172.21.15.43:0/2878780411 learned_addr learned my addr 172.21.15.43:0/2878780411 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:28.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.921+0000 7f09caffd640 1 --2- 172.21.15.43:0/2878780411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c2bb88f0 0x5587c2bb5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.921+0000 7f09ca7fc640 1 -- 172.21.15.43:0/2878780411 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5587c2bd4d70 msgr2=0x5587c2b8a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:28.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.921+0000 7f09ca7fc640 1 --2- 172.21.15.43:0/2878780411 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5587c2bd4d70 0x5587c2b8a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.921+0000 7f09ca7fc640 1 -- 172.21.15.43:0/2878780411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c2bb88f0 msgr2=0x5587c2bb5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:28.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.921+0000 7f09ca7fc640 1 --2- 172.21.15.43:0/2878780411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c2bb88f0 0x5587c2bb5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.921+0000 7f09ca7fc640 1 -- 172.21.15.43:0/2878780411 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587c279f8b0 con 0x5587c2bca210 2024-04-25T08:57:28.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.921+0000 7f09ca7fc640 1 --2- 172.21.15.43:0/2878780411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587c2bca210 0x5587c2b89bc0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f09c0001770 tx=0x7f09c0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2dc97c85572e23d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:28.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.921+0000 7f09abfff640 1 -- 172.21.15.43:0/2878780411 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c001b070 con 0x5587c2bca210 2024-04-25T08:57:28.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.921+0000 7f09abfff640 1 -- 172.21.15.43:0/2878780411 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c0028070 con 0x5587c2bca210 2024-04-25T08:57:28.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.921+0000 7f09abfff640 1 -- 172.21.15.43:0/2878780411 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c0015bf0 con 0x5587c2bca210 2024-04-25T08:57:28.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.922+0000 7f09d41bb740 1 -- 172.21.15.43:0/2878780411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587c2bca210 msgr2=0x5587c2b89bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:28.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.922+0000 7f09d41bb740 1 --2- 172.21.15.43:0/2878780411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587c2bca210 0x5587c2b89bc0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f09c0001770 tx=0x7f09c0014000 comp rx=0 tx=0).stop 2024-04-25T08:57:28.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.922+0000 7f09d41bb740 1 -- 172.21.15.43:0/2878780411 shutdown_connections 2024-04-25T08:57:28.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.922+0000 7f09d41bb740 1 --2- 172.21.15.43:0/2878780411 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5587c2bd4d70 0x5587c2b8a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.922+0000 7f09d41bb740 1 --2- 172.21.15.43:0/2878780411 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587c2bca210 0x5587c2b89bc0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.922+0000 7f09d41bb740 1 --2- 172.21.15.43:0/2878780411 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c2bb88f0 0x5587c2bb5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.922+0000 7f09d41bb740 1 -- 172.21.15.43:0/2878780411 >> 172.21.15.43:0/2878780411 conn(0x5587c2b55f70 msgr2=0x5587c2bb81a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:28.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.922+0000 7f09d41bb740 1 -- 172.21.15.43:0/2878780411 shutdown_connections 2024-04-25T08:57:28.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.922+0000 7f09d41bb740 1 -- 172.21.15.43:0/2878780411 wait complete. 2024-04-25T08:57:28.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.922+0000 7f09d41bb740 1 -- start start 2024-04-25T08:57:28.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.922+0000 7f09d41bb740 1 Processor -- start 2024-04-25T08:57:28.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.923+0000 7f09d41bb740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5587c2bb88f0 0x5587c2c459f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.923+0000 7f09d41bb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587c2bca210 0x5587c2b89bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.923+0000 7f09d41bb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c2bd4d70 0x5587c2b8a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.923+0000 7f09d41bb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc06f060 con 0x5587c2bca210 2024-04-25T08:57:28.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.923+0000 7f09d41bb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc14bdf0 con 0x5587c2bd4d70 2024-04-25T08:57:28.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.923+0000 7f09d41bb740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x5587c2b84ff0 con 0x5587c2bb88f0 2024-04-25T08:57:28.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.923+0000 7f09d41bb740 10 client.0 _ll_register_callbacks cb 0x7f09b40036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:28.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.923+0000 7f09ca7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587c2bca210 0x5587c2b89bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.923+0000 7f09ca7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587c2bca210 0x5587c2b89bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42574/0 (socket says 172.21.15.43:42574) 2024-04-25T08:57:28.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.923+0000 7f09ca7fc640 1 -- 172.21.15.43:0/1536716135 learned_addr learned my addr 172.21.15.43:0/1536716135 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:28.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.923+0000 7f09cb7fe640 1 --2- 172.21.15.43:0/1536716135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c2bd4d70 0x5587c2b8a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.923+0000 7f09caffd640 1 --2- 172.21.15.43:0/1536716135 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5587c2bb88f0 0x5587c2c459f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.924+0000 7f09ca7fc640 1 -- 172.21.15.43:0/1536716135 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5587c2bb88f0 msgr2=0x5587c2c459f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:28.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.924+0000 7f09ca7fc640 1 --2- 172.21.15.43:0/1536716135 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5587c2bb88f0 0x5587c2c459f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.924+0000 7f09ca7fc640 1 -- 172.21.15.43:0/1536716135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c2bd4d70 msgr2=0x5587c2b8a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:28.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.924+0000 7f09ca7fc640 1 --2- 172.21.15.43:0/1536716135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c2bd4d70 0x5587c2b8a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.924+0000 7f09ca7fc640 1 -- 172.21.15.43:0/1536716135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5587c2bb5b70 con 0x5587c2bca210 2024-04-25T08:57:28.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.924+0000 7f09ca7fc640 1 --2- 172.21.15.43:0/1536716135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587c2bca210 0x5587c2b89bc0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f09c0009010 tx=0x7f09c0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c98237efe2594e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:28.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.924+0000 7f09a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:57:28.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.924+0000 7f09a9ffb640 1 -- 172.21.15.43:0/1536716135 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c001b040 con 0x5587c2bca210 2024-04-25T08:57:28.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.924+0000 7f09a9ffb640 1 -- 172.21.15.43:0/1536716135 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f09c0023a50 con 0x5587c2bca210 2024-04-25T08:57:28.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.924+0000 7f09d41bb740 1 -- 172.21.15.43:0/1536716135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f09c0028070 con 0x5587c2bca210 2024-04-25T08:57:28.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.924+0000 7f09a9ffb640 1 -- 172.21.15.43:0/1536716135 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09c003c050 con 0x5587c2bca210 2024-04-25T08:57:28.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.924+0000 7f09d41bb740 10 client.6174 fetch_fsmap learned FSMap version 22 2024-04-25T08:57:28.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.924+0000 7f09d41bb740 1 -- 172.21.15.43:0/1536716135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x5587c2beece0 con 0x5587c2bca210 2024-04-25T08:57:28.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.924+0000 7f09a9ffb640 1 -- 172.21.15.43:0/1536716135 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c0028070 con 0x5587c2bca210 2024-04-25T08:57:28.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.924+0000 7f09a9ffb640 1 -- 172.21.15.43:0/1536716135 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f09c003f020 con 0x5587c2bca210 2024-04-25T08:57:28.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.924+0000 7f09d41bb740 10 client.6174 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T08:57:28.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.924+0000 7f09d41bb740 10 client.6174 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T08:57:28.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.924+0000 7f09d41bb740 20 client.6174 populate_metadata read hostname 'smithi043' 2024-04-25T08:57:28.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.924+0000 7f09d41bb740 1 --2- 172.21.15.43:0/1536716135 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5587c2b8d7d0 0x5587c2b8dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:28.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.924+0000 7f09d41bb740 4 client.6174 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}] 2024-04-25T08:57:28.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.924+0000 7f09d41bb740 1 -- 172.21.15.43:0/1536716135 --> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}) v1 -- 0x5587c2af96f0 con 0x5587c2b8d7d0 2024-04-25T08:57:28.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.925+0000 7f09caffd640 1 --2- 172.21.15.43:0/1536716135 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5587c2b8d7d0 0x5587c2b8dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:28.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.925+0000 7f09a9ffb640 10 client.6174 ms_handle_connect on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:28.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.925+0000 7f09caffd640 1 --2- 172.21.15.43:0/1536716135 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5587c2b8d7d0 0x5587c2b8dbe0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9f13d6ab22c4dda server_cookie=2130732962efea31 in_seq=0 out_seq=0 2024-04-25T08:57:28.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.925+0000 7f09a9ffb640 1 -- 172.21.15.43:0/1536716135 <== mds.0 v2:172.21.15.193:6836/4164634380 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5587c2af96f0 con 0x5587c2b8d7d0 2024-04-25T08:57:28.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.925+0000 7f09a9ffb640 10 client.6174 handle_command_reply: tid=0 2024-04-25T08:57:28.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.926+0000 7f09d41bb740 1 client.6174 shutdown 2024-04-25T08:57:28.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.926+0000 7f09d41bb740 1 -- 172.21.15.43:0/1536716135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587c2bca210 msgr2=0x5587c2b89bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:28.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.926+0000 7f09d41bb740 1 --2- 172.21.15.43:0/1536716135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587c2bca210 0x5587c2b89bc0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f09c0009010 tx=0x7f09c0001120 comp rx=0 tx=0).stop 2024-04-25T08:57:28.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.926+0000 7f09d41bb740 1 -- 172.21.15.43:0/1536716135 shutdown_connections 2024-04-25T08:57:28.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.926+0000 7f09d41bb740 1 --2- 172.21.15.43:0/1536716135 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5587c2b8d7d0 0x5587c2b8dbe0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.926+0000 7f09d41bb740 1 --2- 172.21.15.43:0/1536716135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5587c2bd4d70 0x5587c2b8a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.926+0000 7f09d41bb740 1 --2- 172.21.15.43:0/1536716135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5587c2bca210 0x5587c2b89bc0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.926+0000 7f09a9ffb640 0 client.6174 ms_handle_reset on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:28.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.926+0000 7f09d41bb740 1 --2- 172.21.15.43:0/1536716135 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5587c2bb88f0 0x5587c2c459f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.926+0000 7f09d41bb740 1 -- 172.21.15.43:0/1536716135 >> 172.21.15.43:0/1536716135 conn(0x5587c2b55f70 msgr2=0x5587c2bcab20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:28.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.926+0000 7f09d41bb740 1 -- 172.21.15.43:0/1536716135 shutdown_connections 2024-04-25T08:57:28.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.926+0000 7f09d41bb740 1 -- 172.21.15.43:0/1536716135 wait complete. 2024-04-25T08:57:28.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.926+0000 7f09d41bb740 20 client.6174 trim_cache size 0 max 16384 2024-04-25T08:57:28.922 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T08:57:28.922 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-25T08:57:28.923 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "a5c661af-ce28-48d1-a6c6-c0ae561df41d", 2024-04-25T08:57:28.923 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-25T08:57:28.923 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T08:57:28.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.926+0000 7f09d14b3640 1 -- 172.21.15.43:0/4163444025 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f09b806b4d0 msgr2=0x7f09b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:28.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.926+0000 7f09d14b3640 1 --2- 172.21.15.43:0/4163444025 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f09b806b4d0 0x7f09b806d990 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f09c0000c00 tx=0x7f09c0009040 comp rx=0 tx=0).stop 2024-04-25T08:57:28.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.926+0000 7f09d14b3640 1 -- 172.21.15.43:0/4163444025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09b40051c0 msgr2=0x7f09cc0b8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:28.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.926+0000 7f09d14b3640 1 --2- 172.21.15.43:0/4163444025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09b40051c0 0x7f09cc0b8450 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f09bc011910 tx=0x7f09bc011dd0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.926+0000 7f09d14b3640 1 -- 172.21.15.43:0/4163444025 shutdown_connections 2024-04-25T08:57:28.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.926+0000 7f09d14b3640 1 --2- 172.21.15.43:0/4163444025 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f09b806b4d0 0x7f09b806d990 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.926+0000 7f09d14b3640 1 --2- 172.21.15.43:0/4163444025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09cc1612f0 0x7f09cc161770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.926+0000 7f09d14b3640 1 --2- 172.21.15.43:0/4163444025 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f09cc06c920 0x7f09cc0b68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.926+0000 7f09d14b3640 1 --2- 172.21.15.43:0/4163444025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09b40051c0 0x7f09cc0b8450 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:28.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.926+0000 7f09d14b3640 1 -- 172.21.15.43:0/4163444025 >> 172.21.15.43:0/4163444025 conn(0x7f09cc09cd40 msgr2=0x7f09cc15d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:28.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.927+0000 7f09d14b3640 1 -- 172.21.15.43:0/4163444025 shutdown_connections 2024-04-25T08:57:28.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:28.927+0000 7f09d14b3640 1 -- 172.21.15.43:0/4163444025 wait complete. 2024-04-25T08:57:28.927 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' returned '{'return_code': 0, 'scrub_tag': 'a5c661af-ce28-48d1-a6c6-c0ae561df41d', 'mode': 'asynchronous'}' 2024-04-25T08:57:28.927 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /' 2024-04-25T08:57:28.927 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:57:28.927 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / 2024-04-25T08:57:29.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.041+0000 7f632d44c640 1 Processor -- start 2024-04-25T08:57:29.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.041+0000 7f632d44c640 1 -- start start 2024-04-25T08:57:29.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.042+0000 7f632d44c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f632815e090 0x7f6328161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.042+0000 7f632d44c640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6328162610 con 0x7f6328161fe0 2024-04-25T08:57:29.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.042+0000 7f632d44c640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63280d1750 con 0x7f63280b5c70 2024-04-25T08:57:29.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.042+0000 7f632d44c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63280d1da0 con 0x7f632815e090 2024-04-25T08:57:29.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.042+0000 7f6326ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f632815e090 0x7f6328161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.042+0000 7f63267fc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f6328161fe0 0x7f63280b5560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49748/0 (socket says 172.21.15.43:49748) 2024-04-25T08:57:29.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.042+0000 7f63267fc640 1 -- 172.21.15.43:0/3889039080 learned_addr learned my addr 172.21.15.43:0/3889039080 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:29.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.042+0000 7f6326ffd640 1 -- 172.21.15.43:0/3889039080 >> v1:172.21.15.193:6789/0 conn(0x7f63280b5c70 legacy=0x7f63280d2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:29.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.042+0000 7f6326ffd640 1 -- 172.21.15.43:0/3889039080 >> v1:172.21.15.43:6789/0 conn(0x7f6328161fe0 legacy=0x7f63280b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:29.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.043+0000 7f6326ffd640 1 -- 172.21.15.43:0/3889039080 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63280d1a30 con 0x7f632815e090 2024-04-25T08:57:29.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.042+0000 7f6325ffb640 1 -- 172.21.15.43:0/3889039080 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3448005798 0 0) 0x7f6328162610 con 0x7f6328161fe0 2024-04-25T08:57:29.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.043+0000 7f6326ffd640 1 --2- 172.21.15.43:0/3889039080 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f632815e090 0x7f6328161aa0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f63200097a0 tx=0x7f632002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87a500efa52f2a69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:29.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.043+0000 7f6325ffb640 1 -- 172.21.15.43:0/3889039080 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6320037070 con 0x7f632815e090 2024-04-25T08:57:29.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.043+0000 7f6325ffb640 1 -- 172.21.15.43:0/3889039080 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6320032640 con 0x7f632815e090 2024-04-25T08:57:29.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.043+0000 7f6325ffb640 1 -- 172.21.15.43:0/3889039080 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6320032a10 con 0x7f632815e090 2024-04-25T08:57:29.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.044+0000 7f632d44c640 1 -- 172.21.15.43:0/3889039080 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f632815e090 msgr2=0x7f6328161aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:29.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.044+0000 7f632d44c640 1 --2- 172.21.15.43:0/3889039080 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f632815e090 0x7f6328161aa0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f63200097a0 tx=0x7f632002a2c0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.044+0000 7f632d44c640 1 -- 172.21.15.43:0/3889039080 shutdown_connections 2024-04-25T08:57:29.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.044+0000 7f632d44c640 1 --2- 172.21.15.43:0/3889039080 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f632815e090 0x7f6328161aa0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.044+0000 7f632d44c640 1 -- 172.21.15.43:0/3889039080 >> 172.21.15.43:0/3889039080 conn(0x7f63280d0560 msgr2=0x7f63280d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:29.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.044+0000 7f632d44c640 1 -- 172.21.15.43:0/3889039080 shutdown_connections 2024-04-25T08:57:29.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.044+0000 7f632d44c640 1 -- 172.21.15.43:0/3889039080 wait complete. 2024-04-25T08:57:29.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.045+0000 7f632d44c640 1 Processor -- start 2024-04-25T08:57:29.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.045+0000 7f632d44c640 1 -- start start 2024-04-25T08:57:29.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.046+0000 7f632d44c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f63280b5c70 0x7f632815b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.046+0000 7f632d44c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f632815e090 0x7f63280cec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.046+0000 7f632d44c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6328161fe0 0x7f63280b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.046+0000 7f632d44c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63280d2940 con 0x7f632815e090 2024-04-25T08:57:29.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.046+0000 7f632d44c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63280b3280 con 0x7f6328161fe0 2024-04-25T08:57:29.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.046+0000 7f632d44c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63280d2ac0 con 0x7f63280b5c70 2024-04-25T08:57:29.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.046+0000 7f63267fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f632815e090 0x7f63280cec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.046+0000 7f63267fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f632815e090 0x7f63280cec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42590/0 (socket says 172.21.15.43:42590) 2024-04-25T08:57:29.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.046+0000 7f63267fc640 1 -- 172.21.15.43:0/1585692846 learned_addr learned my addr 172.21.15.43:0/1585692846 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:29.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.046+0000 7f6326ffd640 1 --2- 172.21.15.43:0/1585692846 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f63280b5c70 0x7f632815b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.046+0000 7f63267fc640 1 -- 172.21.15.43:0/1585692846 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f63280b5c70 msgr2=0x7f632815b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:29.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.046+0000 7f63277fe640 1 --2- 172.21.15.43:0/1585692846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6328161fe0 0x7f63280b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.046+0000 7f63267fc640 1 --2- 172.21.15.43:0/1585692846 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f63280b5c70 0x7f632815b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.046+0000 7f63267fc640 1 -- 172.21.15.43:0/1585692846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6328161fe0 msgr2=0x7f63280b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:29.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.046+0000 7f63267fc640 1 --2- 172.21.15.43:0/1585692846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6328161fe0 0x7f63280b3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.046+0000 7f63267fc640 1 -- 172.21.15.43:0/1585692846 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63280aa4f0 con 0x7f632815e090 2024-04-25T08:57:29.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.047+0000 7f63267fc640 1 --2- 172.21.15.43:0/1585692846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f632815e090 0x7f63280cec40 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f631000c3f0 tx=0x7f631000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:29.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.047+0000 7f631f7fe640 1 -- 172.21.15.43:0/1585692846 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63100074e0 con 0x7f632815e090 2024-04-25T08:57:29.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.047+0000 7f631f7fe640 1 -- 172.21.15.43:0/1585692846 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63100047e0 con 0x7f632815e090 2024-04-25T08:57:29.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.047+0000 7f632d44c640 1 -- 172.21.15.43:0/1585692846 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63280aa7d0 con 0x7f632815e090 2024-04-25T08:57:29.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.047+0000 7f631f7fe640 1 -- 172.21.15.43:0/1585692846 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6310004c10 con 0x7f632815e090 2024-04-25T08:57:29.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.047+0000 7f632d44c640 1 -- 172.21.15.43:0/1585692846 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63280aad90 con 0x7f632815e090 2024-04-25T08:57:29.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.048+0000 7f631f7fe640 1 -- 172.21.15.43:0/1585692846 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6310011070 con 0x7f632815e090 2024-04-25T08:57:29.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.049+0000 7f631f7fe640 1 --2- 172.21.15.43:0/1585692846 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f630806b4d0 0x7f630806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.049+0000 7f631f7fe640 1 -- 172.21.15.43:0/1585692846 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6310089d90 con 0x7f632815e090 2024-04-25T08:57:29.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.049+0000 7f6326ffd640 1 --2- 172.21.15.43:0/1585692846 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f630806b4d0 0x7f630806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.049+0000 7f6326ffd640 1 --2- 172.21.15.43:0/1585692846 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f630806b4d0 0x7f630806d990 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6320034000 tx=0x7f632002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:29.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.050+0000 7f6330220740 1 Processor -- start 2024-04-25T08:57:29.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.050+0000 7f6330220740 1 -- start start 2024-04-25T08:57:29.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.051+0000 7f6330220740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f7defb8fe0 0x55f7defbb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.051+0000 7f6330220740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7defb93c0 0x55f7defbbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.051+0000 7f6330220740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7def8bc40 0x55f7def8f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.051+0000 7f6330220740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7deef4a50 con 0x55f7def8bc40 2024-04-25T08:57:29.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.051+0000 7f6330220740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7dee9dd40 con 0x55f7defb93c0 2024-04-25T08:57:29.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.051+0000 7f6330220740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f7deef0af0 con 0x55f7defb8fe0 2024-04-25T08:57:29.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.051+0000 7f63277fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7def8bc40 0x55f7def8f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.051+0000 7f63277fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7def8bc40 0x55f7def8f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42598/0 (socket says 172.21.15.43:42598) 2024-04-25T08:57:29.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.051+0000 7f63277fe640 1 -- 172.21.15.43:0/3611175455 learned_addr learned my addr 172.21.15.43:0/3611175455 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:29.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.051+0000 7f63267fc640 1 --2- 172.21.15.43:0/3611175455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7defb93c0 0x55f7defbbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.051+0000 7f6326ffd640 1 --2- 172.21.15.43:0/3611175455 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f7defb8fe0 0x55f7defbb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.051+0000 7f63277fe640 1 -- 172.21.15.43:0/3611175455 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f7defb8fe0 msgr2=0x55f7defbb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:29.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.051+0000 7f63277fe640 1 --2- 172.21.15.43:0/3611175455 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f7defb8fe0 0x55f7defbb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.051+0000 7f63277fe640 1 -- 172.21.15.43:0/3611175455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7defb93c0 msgr2=0x55f7defbbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:29.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.051+0000 7f63277fe640 1 --2- 172.21.15.43:0/3611175455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7defb93c0 0x55f7defbbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.051+0000 7f63277fe640 1 -- 172.21.15.43:0/3611175455 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7decd5690 con 0x55f7def8bc40 2024-04-25T08:57:29.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.051+0000 7f63277fe640 1 --2- 172.21.15.43:0/3611175455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7def8bc40 0x55f7def8f450 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f6318001d90 tx=0x7f631800d150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c8b733bdc47c4bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:29.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.051+0000 7f631cff9640 1 -- 172.21.15.43:0/3611175455 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6318016020 con 0x55f7def8bc40 2024-04-25T08:57:29.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.051+0000 7f631cff9640 1 -- 172.21.15.43:0/3611175455 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f631800dd30 con 0x55f7def8bc40 2024-04-25T08:57:29.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.051+0000 7f631cff9640 1 -- 172.21.15.43:0/3611175455 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6318012610 con 0x55f7def8bc40 2024-04-25T08:57:29.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.052+0000 7f6330220740 1 -- 172.21.15.43:0/3611175455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7def8bc40 msgr2=0x55f7def8f450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:29.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.052+0000 7f6330220740 1 --2- 172.21.15.43:0/3611175455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7def8bc40 0x55f7def8f450 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f6318001d90 tx=0x7f631800d150 comp rx=0 tx=0).stop 2024-04-25T08:57:29.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.052+0000 7f6330220740 1 -- 172.21.15.43:0/3611175455 shutdown_connections 2024-04-25T08:57:29.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.052+0000 7f6330220740 1 --2- 172.21.15.43:0/3611175455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7def8bc40 0x55f7def8f450 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.052+0000 7f6330220740 1 --2- 172.21.15.43:0/3611175455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7defb93c0 0x55f7defbbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.052+0000 7f6330220740 1 --2- 172.21.15.43:0/3611175455 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f7defb8fe0 0x55f7defbb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.052+0000 7f6330220740 1 -- 172.21.15.43:0/3611175455 >> 172.21.15.43:0/3611175455 conn(0x55f7def8deb0 msgr2=0x55f7defd67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:29.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.052+0000 7f6330220740 1 -- 172.21.15.43:0/3611175455 shutdown_connections 2024-04-25T08:57:29.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.052+0000 7f6330220740 1 -- 172.21.15.43:0/3611175455 wait complete. 2024-04-25T08:57:29.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.053+0000 7f6330220740 1 -- start start 2024-04-25T08:57:29.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.053+0000 7f6330220740 1 Processor -- start 2024-04-25T08:57:29.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.054+0000 7f6330220740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7defb8fe0 0x55f7defb3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.054+0000 7f6330220740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f7defb93c0 0x55f7defb5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.054+0000 7f6330220740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7defbd160 0x55f7defb6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.054+0000 7f6330220740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7def91060 con 0x55f7defb8fe0 2024-04-25T08:57:29.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.054+0000 7f6330220740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7dee93340 con 0x55f7defbd160 2024-04-25T08:57:29.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.054+0000 7f6330220740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f7def91340 con 0x55f7defb93c0 2024-04-25T08:57:29.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.054+0000 7f6330220740 10 client.0 _ll_register_callbacks cb 0x55f7deeb0cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:29.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.054+0000 7f6326ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7defb8fe0 0x55f7defb3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.054+0000 7f6326ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7defb8fe0 0x55f7defb3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42606/0 (socket says 172.21.15.43:42606) 2024-04-25T08:57:29.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.054+0000 7f6326ffd640 1 -- 172.21.15.43:0/1860605979 learned_addr learned my addr 172.21.15.43:0/1860605979 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:29.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.054+0000 7f63277fe640 1 --2- 172.21.15.43:0/1860605979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7defbd160 0x55f7defb6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.054+0000 7f63267fc640 1 --2- 172.21.15.43:0/1860605979 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f7defb93c0 0x55f7defb5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.054+0000 7f6326ffd640 1 -- 172.21.15.43:0/1860605979 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f7defb93c0 msgr2=0x55f7defb5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:29.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.054+0000 7f6326ffd640 1 --2- 172.21.15.43:0/1860605979 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f7defb93c0 0x55f7defb5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.054+0000 7f6326ffd640 1 -- 172.21.15.43:0/1860605979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7defbd160 msgr2=0x55f7defb6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:29.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.054+0000 7f6326ffd640 1 --2- 172.21.15.43:0/1860605979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7defbd160 0x55f7defb6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.054+0000 7f6326ffd640 1 -- 172.21.15.43:0/1860605979 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f7ded35e20 con 0x55f7defb8fe0 2024-04-25T08:57:29.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.054+0000 7f63277fe640 1 --2- 172.21.15.43:0/1860605979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7defbd160 0x55f7defb6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:29.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.054+0000 7f6326ffd640 1 --2- 172.21.15.43:0/1860605979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7defb8fe0 0x55f7defb3620 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f6320000c00 tx=0x7f632002c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6ff4aff6962f1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:29.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.054+0000 7f6305ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:57:29.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.054+0000 7f6305ffb640 1 -- 172.21.15.43:0/1860605979 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632004e060 con 0x55f7defb8fe0 2024-04-25T08:57:29.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.054+0000 7f6305ffb640 1 -- 172.21.15.43:0/1860605979 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f632003cd40 con 0x55f7defb8fe0 2024-04-25T08:57:29.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.054+0000 7f6330220740 1 -- 172.21.15.43:0/1860605979 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f631800dd30 con 0x55f7defb8fe0 2024-04-25T08:57:29.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.054+0000 7f6305ffb640 1 -- 172.21.15.43:0/1860605979 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6320054050 con 0x55f7defb8fe0 2024-04-25T08:57:29.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.054+0000 7f6330220740 10 client.6189 fetch_fsmap learned FSMap version 22 2024-04-25T08:57:29.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.055+0000 7f6330220740 1 -- 172.21.15.43:0/1860605979 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x55f7debc8f80 con 0x55f7defb8fe0 2024-04-25T08:57:29.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.055+0000 7f6305ffb640 1 -- 172.21.15.43:0/1860605979 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f631800dd30 con 0x55f7defb8fe0 2024-04-25T08:57:29.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.055+0000 7f6305ffb640 1 -- 172.21.15.43:0/1860605979 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6320057020 con 0x55f7defb8fe0 2024-04-25T08:57:29.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.055+0000 7f6330220740 10 client.6189 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T08:57:29.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.055+0000 7f6330220740 10 client.6189 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T08:57:29.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.055+0000 7f6330220740 20 client.6189 populate_metadata read hostname 'smithi043' 2024-04-25T08:57:29.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.055+0000 7f6330220740 1 --2- 172.21.15.43:0/1860605979 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55f7defda2e0 0x55f7defd8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.055+0000 7f6330220740 4 client.6189 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:57:29.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.055+0000 7f6330220740 1 -- 172.21.15.43:0/1860605979 --> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f7def01f90 con 0x55f7defda2e0 2024-04-25T08:57:29.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.055+0000 7f63267fc640 1 --2- 172.21.15.43:0/1860605979 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55f7defda2e0 0x55f7defd8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.055+0000 7f6305ffb640 10 client.6189 ms_handle_connect on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:29.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.055+0000 7f63267fc640 1 --2- 172.21.15.43:0/1860605979 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55f7defda2e0 0x55f7defd8e00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4342947d833c705e server_cookie=9b8bcd0b00585146 in_seq=0 out_seq=0 2024-04-25T08:57:29.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.056+0000 7f6305ffb640 1 -- 172.21.15.43:0/1860605979 <== mds.0 v2:172.21.15.193:6836/4164634380 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f7def01f90 con 0x55f7defda2e0 2024-04-25T08:57:29.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.056+0000 7f6305ffb640 10 client.6189 handle_command_reply: tid=0 2024-04-25T08:57:29.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.056+0000 7f6330220740 1 client.6189 shutdown 2024-04-25T08:57:29.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.056+0000 7f6330220740 1 -- 172.21.15.43:0/1860605979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7defb8fe0 msgr2=0x55f7defb3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:29.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.056+0000 7f6330220740 1 --2- 172.21.15.43:0/1860605979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7defb8fe0 0x55f7defb3620 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f6320000c00 tx=0x7f632002c070 comp rx=0 tx=0).stop 2024-04-25T08:57:29.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.056+0000 7f6330220740 1 -- 172.21.15.43:0/1860605979 shutdown_connections 2024-04-25T08:57:29.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.056+0000 7f6330220740 1 --2- 172.21.15.43:0/1860605979 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55f7defda2e0 0x55f7defd8e00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.056+0000 7f6330220740 1 --2- 172.21.15.43:0/1860605979 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7defbd160 0x55f7defb6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.056+0000 7f6330220740 1 --2- 172.21.15.43:0/1860605979 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f7defb93c0 0x55f7defb5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.056+0000 7f6305ffb640 0 client.6189 ms_handle_reset on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:29.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.056+0000 7f6330220740 1 --2- 172.21.15.43:0/1860605979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7defb8fe0 0x55f7defb3620 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.056+0000 7f6330220740 1 -- 172.21.15.43:0/1860605979 >> 172.21.15.43:0/1860605979 conn(0x55f7def8bc40 msgr2=0x55f7def90820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:29.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.056+0000 7f6330220740 1 -- 172.21.15.43:0/1860605979 shutdown_connections 2024-04-25T08:57:29.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.056+0000 7f6330220740 1 -- 172.21.15.43:0/1860605979 wait complete. 2024-04-25T08:57:29.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.056+0000 7f6330220740 20 client.6189 trim_cache size 0 max 16384 2024-04-25T08:57:29.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.071+0000 7f6330220740 1 Processor -- start 2024-04-25T08:57:29.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.071+0000 7f6330220740 1 -- start start 2024-04-25T08:57:29.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.072+0000 7f6330220740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7defb8fe0 0x55f7deff9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.072+0000 7f6330220740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7defb93c0 0x55f7defbc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.072+0000 7f6330220740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f7defb74f0 0x55f7defb6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.072+0000 7f6330220740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63281ffdf0 con 0x55f7defb93c0 2024-04-25T08:57:29.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.072+0000 7f6330220740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63281f4930 con 0x55f7defb8fe0 2024-04-25T08:57:29.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.072+0000 7f6330220740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63281f3d30 con 0x55f7defb74f0 2024-04-25T08:57:29.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.072+0000 7f63267fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7defb93c0 0x55f7defbc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.072+0000 7f63267fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7defb93c0 0x55f7defbc740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42614/0 (socket says 172.21.15.43:42614) 2024-04-25T08:57:29.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.072+0000 7f63267fc640 1 -- 172.21.15.43:0/1946093490 learned_addr learned my addr 172.21.15.43:0/1946093490 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:29.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.072+0000 7f63267fc640 1 -- 172.21.15.43:0/1946093490 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f7defb74f0 msgr2=0x55f7defb6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:29.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.072+0000 7f63267fc640 1 --2- 172.21.15.43:0/1946093490 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f7defb74f0 0x55f7defb6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.072+0000 7f63267fc640 1 -- 172.21.15.43:0/1946093490 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7defb8fe0 msgr2=0x55f7deff9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:29.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.072+0000 7f6326ffd640 1 --2- 172.21.15.43:0/1946093490 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7defb8fe0 0x55f7deff9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.072+0000 7f63267fc640 1 --2- 172.21.15.43:0/1946093490 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7defb8fe0 0x55f7deff9390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.072+0000 7f63267fc640 1 -- 172.21.15.43:0/1946093490 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7def90f90 con 0x55f7defb93c0 2024-04-25T08:57:29.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.073+0000 7f63267fc640 1 --2- 172.21.15.43:0/1946093490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7defb93c0 0x55f7defbc740 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f631000c3c0 tx=0x7f6310088220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa2fc125c1a819c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:29.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.073+0000 7f6307fff640 1 -- 172.21.15.43:0/1946093490 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6310060c30 con 0x55f7defb93c0 2024-04-25T08:57:29.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.073+0000 7f6307fff640 1 -- 172.21.15.43:0/1946093490 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63100565c0 con 0x55f7defb93c0 2024-04-25T08:57:29.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.073+0000 7f6307fff640 1 -- 172.21.15.43:0/1946093490 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6310056820 con 0x55f7defb93c0 2024-04-25T08:57:29.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.073+0000 7f6330220740 1 -- 172.21.15.43:0/1946093490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7defb93c0 msgr2=0x55f7defbc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:29.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.073+0000 7f6330220740 1 --2- 172.21.15.43:0/1946093490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7defb93c0 0x55f7defbc740 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f631000c3c0 tx=0x7f6310088220 comp rx=0 tx=0).stop 2024-04-25T08:57:29.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.073+0000 7f6330220740 1 -- 172.21.15.43:0/1946093490 shutdown_connections 2024-04-25T08:57:29.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.073+0000 7f6330220740 1 --2- 172.21.15.43:0/1946093490 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f7defb74f0 0x55f7defb6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.073+0000 7f6330220740 1 --2- 172.21.15.43:0/1946093490 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7defb93c0 0x55f7defbc740 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.073+0000 7f6330220740 1 --2- 172.21.15.43:0/1946093490 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7defb8fe0 0x55f7deff9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.073+0000 7f6330220740 1 -- 172.21.15.43:0/1946093490 >> 172.21.15.43:0/1946093490 conn(0x55f7def8bc40 msgr2=0x55f7defd67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:29.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.073+0000 7f6330220740 1 -- 172.21.15.43:0/1946093490 shutdown_connections 2024-04-25T08:57:29.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.073+0000 7f6330220740 1 -- 172.21.15.43:0/1946093490 wait complete. 2024-04-25T08:57:29.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.074+0000 7f6330220740 1 -- start start 2024-04-25T08:57:29.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.074+0000 7f6330220740 1 Processor -- start 2024-04-25T08:57:29.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.075+0000 7f6330220740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7defb74f0 0x55f7defb4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.075+0000 7f6330220740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f7defb8fe0 0x55f7defb8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.075+0000 7f6330220740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7defb93c0 0x55f7defb8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.075+0000 7f6330220740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7defb68b0 con 0x55f7defb74f0 2024-04-25T08:57:29.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.075+0000 7f6330220740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63281f4330 con 0x55f7defb93c0 2024-04-25T08:57:29.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.075+0000 7f6330220740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f7defd6100 con 0x55f7defb8fe0 2024-04-25T08:57:29.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.075+0000 7f6330220740 10 client.0 _ll_register_callbacks cb 0x7f6320009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:29.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.075+0000 7f6326ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7defb74f0 0x55f7defb4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.075+0000 7f6326ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7defb74f0 0x55f7defb4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42630/0 (socket says 172.21.15.43:42630) 2024-04-25T08:57:29.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.075+0000 7f6326ffd640 1 -- 172.21.15.43:0/4197540609 learned_addr learned my addr 172.21.15.43:0/4197540609 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:29.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.075+0000 7f63277fe640 1 --2- 172.21.15.43:0/4197540609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7defb93c0 0x55f7defb8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.075+0000 7f63267fc640 1 --2- 172.21.15.43:0/4197540609 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f7defb8fe0 0x55f7defb8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.075+0000 7f6326ffd640 1 -- 172.21.15.43:0/4197540609 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f7defb8fe0 msgr2=0x55f7defb8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:29.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.075+0000 7f6326ffd640 1 --2- 172.21.15.43:0/4197540609 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f7defb8fe0 0x55f7defb8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.075+0000 7f6326ffd640 1 -- 172.21.15.43:0/4197540609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7defb93c0 msgr2=0x55f7defb8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:29.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.075+0000 7f6326ffd640 1 --2- 172.21.15.43:0/4197540609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7defb93c0 0x55f7defb8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.075+0000 7f6326ffd640 1 -- 172.21.15.43:0/4197540609 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f7defb55e0 con 0x55f7defb74f0 2024-04-25T08:57:29.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.075+0000 7f6326ffd640 1 --2- 172.21.15.43:0/4197540609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7defb74f0 0x55f7defb4c20 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f632002a830 tx=0x7f6320004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42bcc567e3965178 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:29.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.075+0000 7f6305ffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:57:29.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.075+0000 7f6305ffb640 1 -- 172.21.15.43:0/4197540609 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632005d060 con 0x55f7defb74f0 2024-04-25T08:57:29.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.075+0000 7f6305ffb640 1 -- 172.21.15.43:0/4197540609 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6320004450 con 0x55f7defb74f0 2024-04-25T08:57:29.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.075+0000 7f6330220740 1 -- 172.21.15.43:0/4197540609 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63100565c0 con 0x55f7defb74f0 2024-04-25T08:57:29.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.075+0000 7f6305ffb640 1 -- 172.21.15.43:0/4197540609 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6320063050 con 0x55f7defb74f0 2024-04-25T08:57:29.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.075+0000 7f6330220740 10 client.6195 fetch_fsmap learned FSMap version 22 2024-04-25T08:57:29.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.075+0000 7f6330220740 1 -- 172.21.15.43:0/4197540609 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x55f7df046450 con 0x55f7defb74f0 2024-04-25T08:57:29.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.076+0000 7f6305ffb640 1 -- 172.21.15.43:0/4197540609 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63100565c0 con 0x55f7defb74f0 2024-04-25T08:57:29.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.076+0000 7f6305ffb640 1 -- 172.21.15.43:0/4197540609 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6320066020 con 0x55f7defb74f0 2024-04-25T08:57:29.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.076+0000 7f6330220740 10 client.6195 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T08:57:29.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.076+0000 7f6330220740 10 client.6195 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T08:57:29.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.076+0000 7f6330220740 20 client.6195 populate_metadata read hostname 'smithi043' 2024-04-25T08:57:29.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.076+0000 7f6330220740 1 --2- 172.21.15.43:0/4197540609 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55f7def92d00 0x55f7defefd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.076+0000 7f6330220740 4 client.6195 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/"}] 2024-04-25T08:57:29.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.076+0000 7f6330220740 1 -- 172.21.15.43:0/4197540609 --> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] -- command(tid 0: {"prefix": "scrub start", "path": "/"}) v1 -- 0x55f7deefc2e0 con 0x55f7def92d00 2024-04-25T08:57:29.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.076+0000 7f63267fc640 1 --2- 172.21.15.43:0/4197540609 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55f7def92d00 0x55f7defefd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.076+0000 7f6305ffb640 10 client.6195 ms_handle_connect on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:29.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.076+0000 7f63267fc640 1 --2- 172.21.15.43:0/4197540609 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55f7def92d00 0x55f7defefd90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb674ad9a72c5080 server_cookie=22c1d00b397e786 in_seq=0 out_seq=0 2024-04-25T08:57:29.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.077+0000 7f6305ffb640 1 -- 172.21.15.43:0/4197540609 <== mds.0 v2:172.21.15.193:6836/4164634380 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f7deefc2e0 con 0x55f7def92d00 2024-04-25T08:57:29.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.077+0000 7f6305ffb640 10 client.6195 handle_command_reply: tid=0 2024-04-25T08:57:29.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.077+0000 7f6330220740 1 client.6195 shutdown 2024-04-25T08:57:29.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.077+0000 7f6330220740 1 -- 172.21.15.43:0/4197540609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7defb74f0 msgr2=0x55f7defb4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:29.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.077+0000 7f6330220740 1 --2- 172.21.15.43:0/4197540609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7defb74f0 0x55f7defb4c20 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f632002a830 tx=0x7f6320004730 comp rx=0 tx=0).stop 2024-04-25T08:57:29.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.077+0000 7f6330220740 1 -- 172.21.15.43:0/4197540609 shutdown_connections 2024-04-25T08:57:29.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.077+0000 7f6330220740 1 --2- 172.21.15.43:0/4197540609 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55f7def92d00 0x55f7defefd90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.077+0000 7f6330220740 1 --2- 172.21.15.43:0/4197540609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f7defb93c0 0x55f7defb8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.077+0000 7f6330220740 1 --2- 172.21.15.43:0/4197540609 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f7defb8fe0 0x55f7defb8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.077+0000 7f6305ffb640 0 client.6195 ms_handle_reset on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:29.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.077+0000 7f6330220740 1 --2- 172.21.15.43:0/4197540609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f7defb74f0 0x55f7defb4c20 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.077+0000 7f6330220740 1 -- 172.21.15.43:0/4197540609 >> 172.21.15.43:0/4197540609 conn(0x55f7def8bc40 msgr2=0x55f7defd67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:29.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.077+0000 7f6330220740 1 -- 172.21.15.43:0/4197540609 shutdown_connections 2024-04-25T08:57:29.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.077+0000 7f6330220740 1 -- 172.21.15.43:0/4197540609 wait complete. 2024-04-25T08:57:29.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.077+0000 7f6330220740 20 client.6195 trim_cache size 0 max 16384 2024-04-25T08:57:29.072 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T08:57:29.073 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-25T08:57:29.073 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "d5d9ac94-69a1-4dee-8264-b2cdf26267f9", 2024-04-25T08:57:29.073 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-25T08:57:29.073 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T08:57:29.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.078+0000 7f632d44c640 1 -- 172.21.15.43:0/1585692846 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f630806b4d0 msgr2=0x7f630806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:29.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.078+0000 7f632d44c640 1 --2- 172.21.15.43:0/1585692846 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f630806b4d0 0x7f630806d990 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6320034000 tx=0x7f632002e000 comp rx=0 tx=0).stop 2024-04-25T08:57:29.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.078+0000 7f632d44c640 1 -- 172.21.15.43:0/1585692846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f632815e090 msgr2=0x7f63280cec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:29.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.078+0000 7f632d44c640 1 --2- 172.21.15.43:0/1585692846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f632815e090 0x7f63280cec40 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f631000c3f0 tx=0x7f631000c8b0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.078+0000 7f632d44c640 1 -- 172.21.15.43:0/1585692846 shutdown_connections 2024-04-25T08:57:29.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.078+0000 7f632d44c640 1 --2- 172.21.15.43:0/1585692846 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f630806b4d0 0x7f630806d990 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.078+0000 7f632d44c640 1 --2- 172.21.15.43:0/1585692846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6328161fe0 0x7f63280b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.078+0000 7f632d44c640 1 --2- 172.21.15.43:0/1585692846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f632815e090 0x7f63280cec40 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.078+0000 7f632d44c640 1 --2- 172.21.15.43:0/1585692846 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f63280b5c70 0x7f632815b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.078+0000 7f632d44c640 1 -- 172.21.15.43:0/1585692846 >> 172.21.15.43:0/1585692846 conn(0x7f63280d0560 msgr2=0x7f6328160580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:29.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.078+0000 7f632d44c640 1 -- 172.21.15.43:0/1585692846 shutdown_connections 2024-04-25T08:57:29.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.078+0000 7f632d44c640 1 -- 172.21.15.43:0/1585692846 wait complete. 2024-04-25T08:57:29.078 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /' returned '{'return_code': 0, 'scrub_tag': 'd5d9ac94-69a1-4dee-8264-b2cdf26267f9', 'mode': 'asynchronous'}' 2024-04-25T08:57:29.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:29.078 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_dir_1) 2024-04-25T08:57:29.096 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_1' 2024-04-25T08:57:29.096 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:57:29.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.247+0000 7fd0433f9640 1 Processor -- start 2024-04-25T08:57:29.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.247+0000 7fd0433f9640 1 -- start start 2024-04-25T08:57:29.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.248+0000 7fd0433f9640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd03c06f300 0x7fd03c06e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.248+0000 7fd0433f9640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fd03c06c4e0 0x7fd03c06c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.248+0000 7fd0433f9640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd03c0d0d80 con 0x7fd03c15dcf0 2024-04-25T08:57:29.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.248+0000 7fd0433f9640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd03c06c050 con 0x7fd03c06c4e0 2024-04-25T08:57:29.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.248+0000 7fd0433f9640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c0d04e0 con 0x7fd03c06f300 2024-04-25T08:57:29.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.248+0000 7fd041195640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fd03c15dcf0 0x7fd03c06ebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49750/0 (socket says 172.21.15.43:49750) 2024-04-25T08:57:29.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.248+0000 7fd041195640 1 -- 172.21.15.43:0/108731695 learned_addr learned my addr 172.21.15.43:0/108731695 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:29.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.248+0000 7fd033fff640 1 -- 172.21.15.43:0/108731695 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 414108378 0 0) 0x7fd03c0d0d80 con 0x7fd03c15dcf0 2024-04-25T08:57:29.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.249+0000 7fd033fff640 1 -- 172.21.15.43:0/108731695 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd028003520 con 0x7fd03c15dcf0 2024-04-25T08:57:29.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.249+0000 7fd041996640 1 --2- 172.21.15.43:0/108731695 >> v2:172.21.15.193:3300/0 conn(0x7fd03c06c4e0 0x7fd03c06c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.249+0000 7fd040994640 1 --2- 172.21.15.43:0/108731695 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd03c06f300 0x7fd03c06e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.249+0000 7fd033fff640 1 -- 172.21.15.43:0/108731695 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3944463265 0 0) 0x7fd028003520 con 0x7fd03c15dcf0 2024-04-25T08:57:29.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.249+0000 7fd033fff640 1 -- 172.21.15.43:0/108731695 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd03c06f300 msgr2=0x7fd03c06e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:29.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.249+0000 7fd033fff640 1 --2- 172.21.15.43:0/108731695 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd03c06f300 0x7fd03c06e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.249+0000 7fd033fff640 1 -- 172.21.15.43:0/108731695 >> v2:172.21.15.193:3300/0 conn(0x7fd03c06c4e0 msgr2=0x7fd03c06c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:29.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.249+0000 7fd033fff640 1 --2- 172.21.15.43:0/108731695 >> v2:172.21.15.193:3300/0 conn(0x7fd03c06c4e0 0x7fd03c06c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.249+0000 7fd033fff640 1 -- 172.21.15.43:0/108731695 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd03c0d07c0 con 0x7fd03c15dcf0 2024-04-25T08:57:29.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.249+0000 7fd033fff640 1 -- 172.21.15.43:0/108731695 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7fd038001670 con 0x7fd03c15dcf0 2024-04-25T08:57:29.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.249+0000 7fd033fff640 1 -- 172.21.15.43:0/108731695 >> v1:172.21.15.43:6789/0 conn(0x7fd03c15dcf0 legacy=0x7fd03c06ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:29.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.249+0000 7fd033fff640 1 --2- 172.21.15.43:0/108731695 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd03c06f300 0x7fd0280039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.249+0000 7fd033fff640 1 --2- 172.21.15.43:0/108731695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd028003f30 0x7fd028004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.249+0000 7fd033fff640 1 --2- 172.21.15.43:0/108731695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd028004d40 0x7fd0280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.249+0000 7fd033fff640 1 -- 172.21.15.43:0/108731695 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c06c050 con 0x7fd028003f30 2024-04-25T08:57:29.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.249+0000 7fd033fff640 1 -- 172.21.15.43:0/108731695 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c0d04e0 con 0x7fd028004d40 2024-04-25T08:57:29.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.249+0000 7fd033fff640 1 -- 172.21.15.43:0/108731695 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd02800ba10 con 0x7fd03c06f300 2024-04-25T08:57:29.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.249+0000 7fd040994640 1 --2- 172.21.15.43:0/108731695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd028003f30 0x7fd028004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.249+0000 7fd041195640 1 --2- 172.21.15.43:0/108731695 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd03c06f300 0x7fd0280039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.249+0000 7fd041996640 1 --2- 172.21.15.43:0/108731695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd028004d40 0x7fd0280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.249+0000 7fd040994640 1 -- 172.21.15.43:0/108731695 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd03c06f300 msgr2=0x7fd0280039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:29.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.249+0000 7fd040994640 1 --2- 172.21.15.43:0/108731695 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd03c06f300 0x7fd0280039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.249+0000 7fd040994640 1 -- 172.21.15.43:0/108731695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd028004d40 msgr2=0x7fd0280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:29.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.249+0000 7fd040994640 1 --2- 172.21.15.43:0/108731695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd028004d40 0x7fd0280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.249+0000 7fd040994640 1 -- 172.21.15.43:0/108731695 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd028003520 con 0x7fd028003f30 2024-04-25T08:57:29.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.250+0000 7fd040994640 1 --2- 172.21.15.43:0/108731695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd028003f30 0x7fd028004800 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fd02c009af0 tx=0x7fd028005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7593c5881acd7229 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:29.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.250+0000 7fd033fff640 1 -- 172.21.15.43:0/108731695 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd02c036070 con 0x7fd028003f30 2024-04-25T08:57:29.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.250+0000 7fd033fff640 1 -- 172.21.15.43:0/108731695 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd02c031400 con 0x7fd028003f30 2024-04-25T08:57:29.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.250+0000 7fd033fff640 1 -- 172.21.15.43:0/108731695 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd02c009690 con 0x7fd028003f30 2024-04-25T08:57:29.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.251+0000 7fd0433f9640 1 -- 172.21.15.43:0/108731695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd028003f30 msgr2=0x7fd028004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:29.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.251+0000 7fd0433f9640 1 --2- 172.21.15.43:0/108731695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd028003f30 0x7fd028004800 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fd02c009af0 tx=0x7fd028005830 comp rx=0 tx=0).stop 2024-04-25T08:57:29.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.251+0000 7fd041996640 1 -- 172.21.15.43:0/108731695 reap_dead start 2024-04-25T08:57:29.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.251+0000 7fd0433f9640 1 -- 172.21.15.43:0/108731695 shutdown_connections 2024-04-25T08:57:29.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.251+0000 7fd0433f9640 1 -- 172.21.15.43:0/108731695 >> 172.21.15.43:0/108731695 conn(0x7fd03c0d1700 msgr2=0x7fd03c0d1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:29.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.251+0000 7fd0433f9640 1 -- 172.21.15.43:0/108731695 shutdown_connections 2024-04-25T08:57:29.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.251+0000 7fd0433f9640 1 -- 172.21.15.43:0/108731695 wait complete. 2024-04-25T08:57:29.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.252+0000 7fd0433f9640 1 Processor -- start 2024-04-25T08:57:29.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.252+0000 7fd0433f9640 1 -- start start 2024-04-25T08:57:29.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.253+0000 7fd0433f9640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd028003f30 0x7fd03c1f7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.253+0000 7fd0433f9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd03c0c7d80 0x7fd03c1f7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.253+0000 7fd0433f9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd03c1f7c80 0x7fd03c09cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.253+0000 7fd0433f9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c0b0d50 con 0x7fd03c1f7c80 2024-04-25T08:57:29.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.253+0000 7fd0433f9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c06bce0 con 0x7fd03c0c7d80 2024-04-25T08:57:29.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.253+0000 7fd0433f9640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd03c0b0ed0 con 0x7fd028003f30 2024-04-25T08:57:29.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.253+0000 7fd041996640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd03c1f7c80 0x7fd03c09cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.253+0000 7fd041996640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd03c1f7c80 0x7fd03c09cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42644/0 (socket says 172.21.15.43:42644) 2024-04-25T08:57:29.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.253+0000 7fd041996640 1 -- 172.21.15.43:0/1883845377 learned_addr learned my addr 172.21.15.43:0/1883845377 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:29.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.253+0000 7fd040994640 1 --2- 172.21.15.43:0/1883845377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd03c0c7d80 0x7fd03c1f7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.253+0000 7fd041195640 1 --2- 172.21.15.43:0/1883845377 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd028003f30 0x7fd03c1f7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.253+0000 7fd041996640 1 -- 172.21.15.43:0/1883845377 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd028003f30 msgr2=0x7fd03c1f7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:29.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.253+0000 7fd041996640 1 --2- 172.21.15.43:0/1883845377 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd028003f30 0x7fd03c1f7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.253+0000 7fd041996640 1 -- 172.21.15.43:0/1883845377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd03c0c7d80 msgr2=0x7fd03c1f7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:29.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.253+0000 7fd041996640 1 --2- 172.21.15.43:0/1883845377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd03c0c7d80 0x7fd03c1f7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.253+0000 7fd041996640 1 -- 172.21.15.43:0/1883845377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd03c0c8280 con 0x7fd03c1f7c80 2024-04-25T08:57:29.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.253+0000 7fd041195640 1 --2- 172.21.15.43:0/1883845377 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd028003f30 0x7fd03c1f7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:29.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.253+0000 7fd041996640 1 --2- 172.21.15.43:0/1883845377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd03c1f7c80 0x7fd03c09cdd0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fd03c0706d0 tx=0x7fd03400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:29.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.253+0000 7fd031ffb640 1 -- 172.21.15.43:0/1883845377 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03400fde0 con 0x7fd03c1f7c80 2024-04-25T08:57:29.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.254+0000 7fd031ffb640 1 -- 172.21.15.43:0/1883845377 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd034015430 con 0x7fd03c1f7c80 2024-04-25T08:57:29.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.254+0000 7fd0433f9640 1 -- 172.21.15.43:0/1883845377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd03c15fb30 con 0x7fd03c1f7c80 2024-04-25T08:57:29.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.254+0000 7fd031ffb640 1 -- 172.21.15.43:0/1883845377 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd034015820 con 0x7fd03c1f7c80 2024-04-25T08:57:29.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.254+0000 7fd0433f9640 1 -- 172.21.15.43:0/1883845377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd03c1600f0 con 0x7fd03c1f7c80 2024-04-25T08:57:29.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.255+0000 7fd0433f9640 1 -- 172.21.15.43:0/1883845377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd03c074ef0 con 0x7fd03c1f7c80 2024-04-25T08:57:29.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.255+0000 7fd031ffb640 1 -- 172.21.15.43:0/1883845377 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd0340159c0 con 0x7fd03c1f7c80 2024-04-25T08:57:29.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.256+0000 7fd031ffb640 1 --2- 172.21.15.43:0/1883845377 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd02006b4d0 0x7fd02006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.256+0000 7fd031ffb640 1 -- 172.21.15.43:0/1883845377 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd03408aff0 con 0x7fd03c1f7c80 2024-04-25T08:57:29.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.256+0000 7fd041195640 1 --2- 172.21.15.43:0/1883845377 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd02006b4d0 0x7fd02006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.256+0000 7fd041195640 1 --2- 172.21.15.43:0/1883845377 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd02006b4d0 0x7fd02006d990 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd038001590 tx=0x7fd038009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:29.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.260+0000 7fd031ffb640 1 -- 172.21.15.43:0/1883845377 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd034019030 con 0x7fd03c1f7c80 2024-04-25T08:57:29.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.480+0000 7fd0433f9640 1 -- 172.21.15.43:0/1883845377 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd03c0b7680 con 0x7fd03c1f7c80 2024-04-25T08:57:29.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.480+0000 7fd031ffb640 1 -- 172.21.15.43:0/1883845377 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fd034010040 con 0x7fd03c1f7c80 2024-04-25T08:57:29.470 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:57:29.470 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1504450503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1504450503},{"type":"v1","addr":"172.21.15.193:6835","nonce":1504450503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5691,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2657133821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2657133821},{"type":"v1","addr":"172.21.15.43:6834","nonce":2657133821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":5694,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/4271431818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":4271431818},{"type":"v1","addr":"172.21.15.43:6835","nonce":4271431818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:57:01.856132+0000","modified":"2024-04-25T08:57:22.526624+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5709":{"gid":5709,"name":"d","rank":0,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6837/4164634380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4164634380},{"type":"v1","addr":"172.21.15.193:6837","nonce":4164634380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:57:29.471 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 22 2024-04-25T08:57:29.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.483+0000 7fd0433f9640 1 -- 172.21.15.43:0/1883845377 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd02006b4d0 msgr2=0x7fd02006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:29.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.483+0000 7fd0433f9640 1 --2- 172.21.15.43:0/1883845377 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd02006b4d0 0x7fd02006d990 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd038001590 tx=0x7fd038009040 comp rx=0 tx=0).stop 2024-04-25T08:57:29.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.483+0000 7fd0433f9640 1 -- 172.21.15.43:0/1883845377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd03c1f7c80 msgr2=0x7fd03c09cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:29.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.483+0000 7fd0433f9640 1 --2- 172.21.15.43:0/1883845377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd03c1f7c80 0x7fd03c09cdd0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fd03c0706d0 tx=0x7fd03400f580 comp rx=0 tx=0).stop 2024-04-25T08:57:29.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.483+0000 7fd0433f9640 1 -- 172.21.15.43:0/1883845377 shutdown_connections 2024-04-25T08:57:29.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.483+0000 7fd0433f9640 1 --2- 172.21.15.43:0/1883845377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd03c1f7c80 0x7fd03c09cdd0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.483+0000 7fd0433f9640 1 --2- 172.21.15.43:0/1883845377 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd02006b4d0 0x7fd02006d990 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.483+0000 7fd0433f9640 1 --2- 172.21.15.43:0/1883845377 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd03c0c7d80 0x7fd03c1f7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.483+0000 7fd0433f9640 1 --2- 172.21.15.43:0/1883845377 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd028003f30 0x7fd03c1f7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.483+0000 7fd0433f9640 1 -- 172.21.15.43:0/1883845377 >> 172.21.15.43:0/1883845377 conn(0x7fd03c0d1700 msgr2=0x7fd03c0b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:29.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.483+0000 7fd0433f9640 1 -- 172.21.15.43:0/1883845377 shutdown_connections 2024-04-25T08:57:29.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.483+0000 7fd0433f9640 1 -- 172.21.15.43:0/1883845377 wait complete. 2024-04-25T08:57:29.483 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path /scrub_test_path/ceph-qa-suite/new_dir_1 2024-04-25T08:57:29.600 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-04-25T08:57:29.601 INFO:teuthology.orchestra.run.smithi193.stdout: "return_code": 0 2024-04-25T08:57:29.601 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-04-25T08:57:29.610 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_1' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T08:57:29.610 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:57:29.610 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:57:29.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:29.626 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_1 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T08:57:29.691 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T08:57:29.691 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T08:57:29.691 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773738 s, 136 MB/s 2024-04-25T08:57:29.692 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_1' 2024-04-25T08:57:29.692 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:57:29.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.843+0000 7f9dc5732640 1 Processor -- start 2024-04-25T08:57:29.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.843+0000 7f9dc5732640 1 -- start start 2024-04-25T08:57:29.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.844+0000 7f9dc5732640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9dc006f930 0x7f9dc006db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.844+0000 7f9dc5732640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc006f4c0 con 0x7f9dc006f0e0 2024-04-25T08:57:29.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.844+0000 7f9dc5732640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc00b3360 con 0x7f9dc00b3ea0 2024-04-25T08:57:29.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.844+0000 7f9dc5732640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dc0072130 con 0x7f9dc006f930 2024-04-25T08:57:29.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.844+0000 7f9dbeffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f9dc006f0e0 0x7f9dc00b3790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:52680/0 (socket says 172.21.15.43:52680) 2024-04-25T08:57:29.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.844+0000 7f9dbeffd640 1 -- 172.21.15.43:0/2348665884 learned_addr learned my addr 172.21.15.43:0/2348665884 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:29.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.845+0000 7f9dbf7fe640 1 --2- 172.21.15.43:0/2348665884 >> v2:172.21.15.193:3300/0 conn(0x7f9dc006f930 0x7f9dc006db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.845+0000 7f9dbdffb640 1 -- 172.21.15.43:0/2348665884 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1263916425 0 0) 0x7f9dc006f4c0 con 0x7f9dc006f0e0 2024-04-25T08:57:29.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.845+0000 7f9dbdffb640 1 -- 172.21.15.43:0/2348665884 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9dac003520 con 0x7f9dc006f0e0 2024-04-25T08:57:29.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.845+0000 7f9dbf7fe640 1 -- 172.21.15.43:0/2348665884 >> v1:172.21.15.193:6789/0 conn(0x7f9dc00b3ea0 legacy=0x7f9dc006d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:29.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.845+0000 7f9dbdffb640 1 -- 172.21.15.43:0/2348665884 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4082274637 0 0) 0x7f9dc00b3360 con 0x7f9dc00b3ea0 2024-04-25T08:57:29.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.845+0000 7f9dbf7fe640 1 -- 172.21.15.43:0/2348665884 >> v1:172.21.15.43:6789/0 conn(0x7f9dc006f0e0 legacy=0x7f9dc00b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:29.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.845+0000 7f9dbf7fe640 1 -- 172.21.15.43:0/2348665884 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc0071520 con 0x7f9dc006f930 2024-04-25T08:57:29.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.846+0000 7f9dbf7fe640 1 --2- 172.21.15.43:0/2348665884 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9dc006f930 0x7f9dc006db60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9dc00722b0 tx=0x7f9db002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f8c44b6a5b74cfa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:29.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.846+0000 7f9dbdffb640 1 -- 172.21.15.43:0/2348665884 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db0007740 con 0x7f9dc006f930 2024-04-25T08:57:29.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.846+0000 7f9dbdffb640 1 -- 172.21.15.43:0/2348665884 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db00078e0 con 0x7f9dc006f930 2024-04-25T08:57:29.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.846+0000 7f9dbdffb640 1 -- 172.21.15.43:0/2348665884 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db0007cb0 con 0x7f9dc006f930 2024-04-25T08:57:29.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.846+0000 7f9dc5732640 1 -- 172.21.15.43:0/2348665884 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9dc006f930 msgr2=0x7f9dc006db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:29.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.846+0000 7f9dc5732640 1 --2- 172.21.15.43:0/2348665884 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9dc006f930 0x7f9dc006db60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9dc00722b0 tx=0x7f9db002a660 comp rx=0 tx=0).stop 2024-04-25T08:57:29.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.847+0000 7f9dc5732640 1 -- 172.21.15.43:0/2348665884 shutdown_connections 2024-04-25T08:57:29.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.847+0000 7f9dc5732640 1 --2- 172.21.15.43:0/2348665884 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9dc006f930 0x7f9dc006db60 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.847+0000 7f9dc5732640 1 -- 172.21.15.43:0/2348665884 >> 172.21.15.43:0/2348665884 conn(0x7f9dc00b2490 msgr2=0x7f9dc00b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:29.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.847+0000 7f9dc5732640 1 -- 172.21.15.43:0/2348665884 shutdown_connections 2024-04-25T08:57:29.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.847+0000 7f9dc5732640 1 -- 172.21.15.43:0/2348665884 wait complete. 2024-04-25T08:57:29.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.848+0000 7f9dc5732640 1 Processor -- start 2024-04-25T08:57:29.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.848+0000 7f9dc5732640 1 -- start start 2024-04-25T08:57:29.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.849+0000 7f9dc5732640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9dc006f0e0 0x7f9dc00c6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.849+0000 7f9dc5732640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9dc006f930 0x7f9dc00d01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.849+0000 7f9dc5732640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9dc00b3ea0 0x7f9dc01576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.849+0000 7f9dc5732640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc0071dc0 con 0x7f9dc006f0e0 2024-04-25T08:57:29.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.849+0000 7f9dc5732640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc0071ae0 con 0x7f9dc006f930 2024-04-25T08:57:29.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.849+0000 7f9dc5732640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc015aee0 con 0x7f9dc00b3ea0 2024-04-25T08:57:29.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.849+0000 7f9dbeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9dc006f0e0 0x7f9dc00c6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.849+0000 7f9dbeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9dc006f0e0 0x7f9dc00c6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57812/0 (socket says 172.21.15.43:57812) 2024-04-25T08:57:29.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.849+0000 7f9dbeffd640 1 -- 172.21.15.43:0/1447882242 learned_addr learned my addr 172.21.15.43:0/1447882242 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:29.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.849+0000 7f9dbe7fc640 1 --2- 172.21.15.43:0/1447882242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9dc006f930 0x7f9dc00d01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.849+0000 7f9dbf7fe640 1 --2- 172.21.15.43:0/1447882242 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9dc00b3ea0 0x7f9dc01576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.849+0000 7f9dbeffd640 1 -- 172.21.15.43:0/1447882242 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9dc00b3ea0 msgr2=0x7f9dc01576f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:29.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.849+0000 7f9dbeffd640 1 --2- 172.21.15.43:0/1447882242 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9dc00b3ea0 0x7f9dc01576f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.849+0000 7f9dbeffd640 1 -- 172.21.15.43:0/1447882242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9dc006f930 msgr2=0x7f9dc00d01f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:29.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.849+0000 7f9dbeffd640 1 --2- 172.21.15.43:0/1447882242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9dc006f930 0x7f9dc00d01f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:29.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.849+0000 7f9dbeffd640 1 -- 172.21.15.43:0/1447882242 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc01f79d0 con 0x7f9dc006f0e0 2024-04-25T08:57:29.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.849+0000 7f9dbeffd640 1 --2- 172.21.15.43:0/1447882242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9dc006f0e0 0x7f9dc00c6110 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f9db800c430 tx=0x7f9db800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:29.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.849+0000 7f9db77fe640 1 -- 172.21.15.43:0/1447882242 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db8017070 con 0x7f9dc006f0e0 2024-04-25T08:57:29.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.849+0000 7f9db77fe640 1 -- 172.21.15.43:0/1447882242 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db8005ce0 con 0x7f9dc006f0e0 2024-04-25T08:57:29.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.849+0000 7f9db77fe640 1 -- 172.21.15.43:0/1447882242 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db80072f0 con 0x7f9dc006f0e0 2024-04-25T08:57:29.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.849+0000 7f9dc5732640 1 -- 172.21.15.43:0/1447882242 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dc00c5950 con 0x7f9dc006f0e0 2024-04-25T08:57:29.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.850+0000 7f9dc5732640 1 -- 172.21.15.43:0/1447882242 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc01f6b70 con 0x7f9dc006f0e0 2024-04-25T08:57:29.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.850+0000 7f9dc5732640 1 -- 172.21.15.43:0/1447882242 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dc0074ef0 con 0x7f9dc006f0e0 2024-04-25T08:57:29.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.851+0000 7f9db77fe640 1 -- 172.21.15.43:0/1447882242 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9db801a070 con 0x7f9dc006f0e0 2024-04-25T08:57:29.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.851+0000 7f9db77fe640 1 --2- 172.21.15.43:0/1447882242 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9da006b4d0 0x7f9da006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:29.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.851+0000 7f9db77fe640 1 -- 172.21.15.43:0/1447882242 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9db80895a0 con 0x7f9dc006f0e0 2024-04-25T08:57:29.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.851+0000 7f9dbe7fc640 1 --2- 172.21.15.43:0/1447882242 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9da006b4d0 0x7f9da006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:29.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.852+0000 7f9dbe7fc640 1 --2- 172.21.15.43:0/1447882242 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9da006b4d0 0x7f9da006d990 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9dc015d8e0 tx=0x7f9da8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:29.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:29.855+0000 7f9db77fe640 1 -- 172.21.15.43:0/1447882242 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9dc01f79d0 con 0x7f9dc006f0e0 2024-04-25T08:57:30.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.073+0000 7f9dc5732640 1 -- 172.21.15.43:0/1447882242 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9dc00b3760 con 0x7f9dc006f0e0 2024-04-25T08:57:30.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.074+0000 7f9db77fe640 1 -- 172.21.15.43:0/1447882242 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f9db805ce20 con 0x7f9dc006f0e0 2024-04-25T08:57:30.063 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:57:30.063 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":22,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1504450503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1504450503},{"type":"v1","addr":"172.21.15.193:6835","nonce":1504450503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5691,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2657133821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2657133821},{"type":"v1","addr":"172.21.15.43:6834","nonce":2657133821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":5694,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/4271431818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":4271431818},{"type":"v1","addr":"172.21.15.43:6835","nonce":4271431818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":22,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:57:01.856132+0000","modified":"2024-04-25T08:57:22.526624+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5709":{"gid":5709,"name":"d","rank":0,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.193:6837/4164634380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4164634380},{"type":"v1","addr":"172.21.15.193:6837","nonce":4164634380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:57:30.063 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 22 2024-04-25T08:57:30.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.075+0000 7f9dc5732640 1 -- 172.21.15.43:0/1447882242 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9da006b4d0 msgr2=0x7f9da006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:30.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.075+0000 7f9dc5732640 1 --2- 172.21.15.43:0/1447882242 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9da006b4d0 0x7f9da006d990 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f9dc015d8e0 tx=0x7f9da8009040 comp rx=0 tx=0).stop 2024-04-25T08:57:30.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.075+0000 7f9dc5732640 1 -- 172.21.15.43:0/1447882242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9dc006f0e0 msgr2=0x7f9dc00c6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:30.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.075+0000 7f9dc5732640 1 --2- 172.21.15.43:0/1447882242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9dc006f0e0 0x7f9dc00c6110 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f9db800c430 tx=0x7f9db800c8f0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.075+0000 7f9dc5732640 1 -- 172.21.15.43:0/1447882242 shutdown_connections 2024-04-25T08:57:30.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.076+0000 7f9dc5732640 1 --2- 172.21.15.43:0/1447882242 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9dc00b3ea0 0x7f9dc01576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.076+0000 7f9dc5732640 1 --2- 172.21.15.43:0/1447882242 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9da006b4d0 0x7f9da006d990 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.076+0000 7f9dc5732640 1 --2- 172.21.15.43:0/1447882242 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9dc006f930 0x7f9dc00d01f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.076+0000 7f9dc5732640 1 --2- 172.21.15.43:0/1447882242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9dc006f0e0 0x7f9dc00c6110 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.076+0000 7f9dc5732640 1 -- 172.21.15.43:0/1447882242 >> 172.21.15.43:0/1447882242 conn(0x7f9dc00b2490 msgr2=0x7f9dc006ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:30.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.076+0000 7f9dc5732640 1 -- 172.21.15.43:0/1447882242 shutdown_connections 2024-04-25T08:57:30.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.076+0000 7f9dc5732640 1 -- 172.21.15.43:0/1447882242 wait complete. 2024-04-25T08:57:30.076 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path /scrub_test_path/ceph-qa-suite/new_file_1 2024-04-25T08:57:30.187 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-04-25T08:57:30.187 INFO:teuthology.orchestra.run.smithi193.stdout: "return_code": 0 2024-04-25T08:57:30.188 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-04-25T08:57:30.196 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_1' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T08:57:30.196 DEBUG:teuthology.orchestra.run.smithi043:> stdin-killer -- python3 -c ' 2024-04-25T08:57:30.197 DEBUG:teuthology.orchestra.run.smithi043:> import os 2024-04-25T08:57:30.197 DEBUG:teuthology.orchestra.run.smithi043:> import stat 2024-04-25T08:57:30.197 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T08:57:30.197 DEBUG:teuthology.orchestra.run.smithi043:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_1").st_ino) 2024-04-25T08:57:30.197 DEBUG:teuthology.orchestra.run.smithi043:> ' 2024-04-25T08:57:30.266 INFO:teuthology.orchestra.run.smithi043.stdout:1099511630414 2024-04-25T08:57:30.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:57:30.275 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:57:30.275 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T08:57:30.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.428+0000 7f89ecaf5640 1 Processor -- start 2024-04-25T08:57:30.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.428+0000 7f89ecaf5640 1 -- start start 2024-04-25T08:57:30.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.429+0000 7f89ecaf5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f89e815e090 0x7f89e8161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.429+0000 7f89ecaf5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f89e80b5c70 0x7f89e80d2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.429+0000 7f89ecaf5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89e8162610 con 0x7f89e8161fe0 2024-04-25T08:57:30.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.429+0000 7f89ecaf5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89e80d1da0 con 0x7f89e80b5c70 2024-04-25T08:57:30.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.429+0000 7f89ecaf5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e80d1750 con 0x7f89e815e090 2024-04-25T08:57:30.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.429+0000 7f89e6d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f89e80b5c70 0x7f89e80d2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.429+0000 7f89e5d9b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f89e8161fe0 0x7f89e80b5560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:52692/0 (socket says 172.21.15.43:52692) 2024-04-25T08:57:30.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.429+0000 7f89e5d9b640 1 -- 172.21.15.43:0/1637809634 learned_addr learned my addr 172.21.15.43:0/1637809634 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:30.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.430+0000 7f89e659c640 1 --2- 172.21.15.43:0/1637809634 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f89e815e090 0x7f89e8161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.430+0000 7f89e6d9d640 1 -- 172.21.15.43:0/1637809634 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f89e815e090 msgr2=0x7f89e8161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:30.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.430+0000 7f89e6d9d640 1 --2- 172.21.15.43:0/1637809634 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f89e815e090 0x7f89e8161aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.430+0000 7f89e6d9d640 1 -- 172.21.15.43:0/1637809634 >> v1:172.21.15.43:6789/0 conn(0x7f89e8161fe0 legacy=0x7f89e80b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:30.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.430+0000 7f89e6d9d640 1 -- 172.21.15.43:0/1637809634 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e80d1a30 con 0x7f89e80b5c70 2024-04-25T08:57:30.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.430+0000 7f89e559a640 1 -- 172.21.15.43:0/1637809634 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1807546380 0 0) 0x7f89e8162610 con 0x7f89e8161fe0 2024-04-25T08:57:30.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.430+0000 7f89e6d9d640 1 --2- 172.21.15.43:0/1637809634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89e80b5c70 0x7f89e80d2230 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f89e00097a0 tx=0x7f89e002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bedd7c6e82ceb7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:30.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.431+0000 7f89e559a640 1 -- 172.21.15.43:0/1637809634 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e0037070 con 0x7f89e80b5c70 2024-04-25T08:57:30.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.431+0000 7f89e559a640 1 -- 172.21.15.43:0/1637809634 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e0032440 con 0x7f89e80b5c70 2024-04-25T08:57:30.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.431+0000 7f89e559a640 1 -- 172.21.15.43:0/1637809634 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e0032810 con 0x7f89e80b5c70 2024-04-25T08:57:30.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.431+0000 7f89ecaf5640 1 -- 172.21.15.43:0/1637809634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89e80b5c70 msgr2=0x7f89e80d2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:30.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.431+0000 7f89ecaf5640 1 --2- 172.21.15.43:0/1637809634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89e80b5c70 0x7f89e80d2230 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f89e00097a0 tx=0x7f89e002a210 comp rx=0 tx=0).stop 2024-04-25T08:57:30.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.431+0000 7f89ecaf5640 1 -- 172.21.15.43:0/1637809634 shutdown_connections 2024-04-25T08:57:30.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.431+0000 7f89ecaf5640 1 --2- 172.21.15.43:0/1637809634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89e80b5c70 0x7f89e80d2230 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.431+0000 7f89ecaf5640 1 --2- 172.21.15.43:0/1637809634 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f89e815e090 0x7f89e8161aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.431+0000 7f89ecaf5640 1 -- 172.21.15.43:0/1637809634 >> 172.21.15.43:0/1637809634 conn(0x7f89e80d0560 msgr2=0x7f89e80d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:30.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.431+0000 7f89ecaf5640 1 -- 172.21.15.43:0/1637809634 shutdown_connections 2024-04-25T08:57:30.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.431+0000 7f89ecaf5640 1 -- 172.21.15.43:0/1637809634 wait complete. 2024-04-25T08:57:30.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.433+0000 7f89ecaf5640 1 Processor -- start 2024-04-25T08:57:30.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.433+0000 7f89ecaf5640 1 -- start start 2024-04-25T08:57:30.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.433+0000 7f89ecaf5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89e80b5c70 0x7f89e80ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.433+0000 7f89ecaf5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f89e815e090 0x7f89e80b7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.433+0000 7f89ecaf5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89e8161fe0 0x7f89e80b8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.433+0000 7f89ecaf5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e80d2770 con 0x7f89e8161fe0 2024-04-25T08:57:30.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.433+0000 7f89ecaf5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e80b3280 con 0x7f89e80b5c70 2024-04-25T08:57:30.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.433+0000 7f89ecaf5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e80d28f0 con 0x7f89e815e090 2024-04-25T08:57:30.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.433+0000 7f89e6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89e8161fe0 0x7f89e80b8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.434+0000 7f89e6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89e8161fe0 0x7f89e80b8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57824/0 (socket says 172.21.15.43:57824) 2024-04-25T08:57:30.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.434+0000 7f89e6d9d640 1 -- 172.21.15.43:0/4087653935 learned_addr learned my addr 172.21.15.43:0/4087653935 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:30.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.434+0000 7f89e5d9b640 1 --2- 172.21.15.43:0/4087653935 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f89e815e090 0x7f89e80b7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.434+0000 7f89e659c640 1 --2- 172.21.15.43:0/4087653935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89e80b5c70 0x7f89e80ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.434+0000 7f89e6d9d640 1 -- 172.21.15.43:0/4087653935 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f89e815e090 msgr2=0x7f89e80b7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:30.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.434+0000 7f89e6d9d640 1 --2- 172.21.15.43:0/4087653935 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f89e815e090 0x7f89e80b7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.434+0000 7f89e6d9d640 1 -- 172.21.15.43:0/4087653935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89e80b5c70 msgr2=0x7f89e80ca050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:30.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.434+0000 7f89e6d9d640 1 --2- 172.21.15.43:0/4087653935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89e80b5c70 0x7f89e80ca050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.434+0000 7f89e6d9d640 1 -- 172.21.15.43:0/4087653935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e81f4290 con 0x7f89e8161fe0 2024-04-25T08:57:30.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.434+0000 7f89e659c640 1 --2- 172.21.15.43:0/4087653935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89e80b5c70 0x7f89e80ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:30.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.434+0000 7f89e6d9d640 1 --2- 172.21.15.43:0/4087653935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89e8161fe0 0x7f89e80b8390 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f89e0035040 tx=0x7f89e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:30.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.434+0000 7f89deffd640 1 -- 172.21.15.43:0/4087653935 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e0037040 con 0x7f89e8161fe0 2024-04-25T08:57:30.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.434+0000 7f89deffd640 1 -- 172.21.15.43:0/4087653935 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e0042070 con 0x7f89e8161fe0 2024-04-25T08:57:30.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.434+0000 7f89deffd640 1 -- 172.21.15.43:0/4087653935 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e00308a0 con 0x7f89e8161fe0 2024-04-25T08:57:30.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.434+0000 7f89ecaf5640 1 -- 172.21.15.43:0/4087653935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e8144f30 con 0x7f89e8161fe0 2024-04-25T08:57:30.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.434+0000 7f89ecaf5640 1 -- 172.21.15.43:0/4087653935 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e8145a10 con 0x7f89e8161fe0 2024-04-25T08:57:30.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.436+0000 7f89deffd640 1 -- 172.21.15.43:0/4087653935 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f89e0030a40 con 0x7f89e8161fe0 2024-04-25T08:57:30.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.436+0000 7f89deffd640 1 --2- 172.21.15.43:0/4087653935 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f89c806b500 0x7f89c806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.437+0000 7f89deffd640 1 -- 172.21.15.43:0/4087653935 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f89e00aa9e0 con 0x7f89e8161fe0 2024-04-25T08:57:30.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.437+0000 7f89e659c640 1 --2- 172.21.15.43:0/4087653935 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f89c806b500 0x7f89c806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.437+0000 7f89e659c640 1 --2- 172.21.15.43:0/4087653935 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f89c806b500 0x7f89c806d9c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f89d8007520 tx=0x7f89d800b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:30.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.437+0000 7f89ef7f8740 1 Processor -- start 2024-04-25T08:57:30.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.438+0000 7f89ef7f8740 1 -- start start 2024-04-25T08:57:30.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.438+0000 7f89ef7f8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c62fe84aa0 0x55c62fe872b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.438+0000 7f89ef7f8740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c62fe84e80 0x55c62fe877f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.438+0000 7f89ef7f8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62fe57700 0x55c62fe5af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.438+0000 7f89ef7f8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c62fdc0e00 con 0x55c62fe57700 2024-04-25T08:57:30.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.438+0000 7f89ef7f8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c62fd69d20 con 0x55c62fe84aa0 2024-04-25T08:57:30.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.438+0000 7f89ef7f8740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c62fdbcea0 con 0x55c62fe84e80 2024-04-25T08:57:30.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.438+0000 7f89e6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62fe57700 0x55c62fe5af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.438+0000 7f89e5d9b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c62fe84e80 0x55c62fe877f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.438+0000 7f89e659c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c62fe84aa0 0x55c62fe872b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.438+0000 7f89e6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62fe57700 0x55c62fe5af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57840/0 (socket says 172.21.15.43:57840) 2024-04-25T08:57:30.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.438+0000 7f89e6d9d640 1 -- 172.21.15.43:0/4083639260 learned_addr learned my addr 172.21.15.43:0/4083639260 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:30.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.439+0000 7f89e6d9d640 1 -- 172.21.15.43:0/4083639260 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c62fe84e80 msgr2=0x55c62fe877f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:30.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.439+0000 7f89e6d9d640 1 --2- 172.21.15.43:0/4083639260 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c62fe84e80 0x55c62fe877f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.439+0000 7f89e6d9d640 1 -- 172.21.15.43:0/4083639260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c62fe84aa0 msgr2=0x55c62fe872b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:30.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.439+0000 7f89e6d9d640 1 --2- 172.21.15.43:0/4083639260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c62fe84aa0 0x55c62fe872b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.439+0000 7f89e6d9d640 1 -- 172.21.15.43:0/4083639260 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c62fd8bdf0 con 0x55c62fe57700 2024-04-25T08:57:30.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.439+0000 7f89e6d9d640 1 --2- 172.21.15.43:0/4083639260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62fe57700 0x55c62fe5af10 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f89e00a8fc0 tx=0x7f89e0078e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c211e56b419b911 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:30.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.439+0000 7f89c7fff640 1 -- 172.21.15.43:0/4083639260 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e00a7070 con 0x55c62fe57700 2024-04-25T08:57:30.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.439+0000 7f89c7fff640 1 -- 172.21.15.43:0/4083639260 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e00791f0 con 0x55c62fe57700 2024-04-25T08:57:30.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.439+0000 7f89c7fff640 1 -- 172.21.15.43:0/4083639260 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e007d340 con 0x55c62fe57700 2024-04-25T08:57:30.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.440+0000 7f89ef7f8740 1 -- 172.21.15.43:0/4083639260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62fe57700 msgr2=0x55c62fe5af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:30.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.440+0000 7f89ef7f8740 1 --2- 172.21.15.43:0/4083639260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62fe57700 0x55c62fe5af10 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f89e00a8fc0 tx=0x7f89e0078e00 comp rx=0 tx=0).stop 2024-04-25T08:57:30.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.440+0000 7f89ef7f8740 1 -- 172.21.15.43:0/4083639260 shutdown_connections 2024-04-25T08:57:30.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.440+0000 7f89ef7f8740 1 --2- 172.21.15.43:0/4083639260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62fe57700 0x55c62fe5af10 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.440+0000 7f89ef7f8740 1 --2- 172.21.15.43:0/4083639260 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c62fe84e80 0x55c62fe877f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.440+0000 7f89ef7f8740 1 --2- 172.21.15.43:0/4083639260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c62fe84aa0 0x55c62fe872b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.440+0000 7f89ef7f8740 1 -- 172.21.15.43:0/4083639260 >> 172.21.15.43:0/4083639260 conn(0x55c62fe59970 msgr2=0x55c62fea22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:30.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.440+0000 7f89ef7f8740 1 -- 172.21.15.43:0/4083639260 shutdown_connections 2024-04-25T08:57:30.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.440+0000 7f89ef7f8740 1 -- 172.21.15.43:0/4083639260 wait complete. 2024-04-25T08:57:30.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.440+0000 7f89ef7f8740 1 -- start start 2024-04-25T08:57:30.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.440+0000 7f89ef7f8740 1 Processor -- start 2024-04-25T08:57:30.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.441+0000 7f89ef7f8740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c62fe84aa0 0x55c62feb35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.441+0000 7f89ef7f8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c62fe84e80 0x55c62fe81ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.441+0000 7f89ef7f8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62feb5560 0x55c62fe82220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.441+0000 7f89ef7f8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c62fdc0e00 con 0x55c62feb5560 2024-04-25T08:57:30.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.441+0000 7f89ef7f8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c62fd5f590 con 0x55c62fe84e80 2024-04-25T08:57:30.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.441+0000 7f89ef7f8740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c62fe5cb20 con 0x55c62fe84aa0 2024-04-25T08:57:30.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.441+0000 7f89ef7f8740 10 client.0 _ll_register_callbacks cb 0x55c62fd6be30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:30.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.441+0000 7f89e6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62feb5560 0x55c62fe82220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.441+0000 7f89e6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62feb5560 0x55c62fe82220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57856/0 (socket says 172.21.15.43:57856) 2024-04-25T08:57:30.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.441+0000 7f89e6d9d640 1 -- 172.21.15.43:0/3202649327 learned_addr learned my addr 172.21.15.43:0/3202649327 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:30.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.442+0000 7f89e5d9b640 1 --2- 172.21.15.43:0/3202649327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c62fe84e80 0x55c62fe81ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.442+0000 7f89e659c640 1 --2- 172.21.15.43:0/3202649327 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c62fe84aa0 0x55c62feb35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.442+0000 7f89e6d9d640 1 -- 172.21.15.43:0/3202649327 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c62fe84aa0 msgr2=0x55c62feb35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:30.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.442+0000 7f89e6d9d640 1 --2- 172.21.15.43:0/3202649327 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c62fe84aa0 0x55c62feb35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.442+0000 7f89e6d9d640 1 -- 172.21.15.43:0/3202649327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c62fe84e80 msgr2=0x55c62fe81ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:30.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.442+0000 7f89e6d9d640 1 --2- 172.21.15.43:0/3202649327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c62fe84e80 0x55c62fe81ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.442+0000 7f89e6d9d640 1 -- 172.21.15.43:0/3202649327 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c62fad53f0 con 0x55c62feb5560 2024-04-25T08:57:30.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.442+0000 7f89e6d9d640 1 --2- 172.21.15.43:0/3202649327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62feb5560 0x55c62fe82220 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f89e0078ef0 tx=0x7f89e0006c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a27e56579eb9c52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:30.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.442+0000 7f89c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:57:30.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.442+0000 7f89c57fa640 1 -- 172.21.15.43:0/3202649327 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e00a7030 con 0x55c62feb5560 2024-04-25T08:57:30.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.442+0000 7f89c57fa640 1 -- 172.21.15.43:0/3202649327 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f89e00774f0 con 0x55c62feb5560 2024-04-25T08:57:30.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.442+0000 7f89ef7f8740 1 -- 172.21.15.43:0/3202649327 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f89e00791f0 con 0x55c62feb5560 2024-04-25T08:57:30.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.442+0000 7f89c57fa640 1 -- 172.21.15.43:0/3202649327 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89e009f050 con 0x55c62feb5560 2024-04-25T08:57:30.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.442+0000 7f89ef7f8740 10 client.6222 fetch_fsmap learned FSMap version 22 2024-04-25T08:57:30.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.442+0000 7f89ef7f8740 1 -- 172.21.15.43:0/3202649327 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x55c62fb9f7d0 con 0x55c62feb5560 2024-04-25T08:57:30.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.442+0000 7f89c57fa640 1 -- 172.21.15.43:0/3202649327 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e00791f0 con 0x55c62feb5560 2024-04-25T08:57:30.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.442+0000 7f89c57fa640 1 -- 172.21.15.43:0/3202649327 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f89e00a2020 con 0x55c62feb5560 2024-04-25T08:57:30.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.442+0000 7f89ef7f8740 10 client.6222 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T08:57:30.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.442+0000 7f89ef7f8740 10 client.6222 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T08:57:30.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.442+0000 7f89ef7f8740 20 client.6222 populate_metadata read hostname 'smithi043' 2024-04-25T08:57:30.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.442+0000 7f89ef7f8740 1 --2- 172.21.15.43:0/3202649327 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55c62fe80070 0x55c62fe80480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.442+0000 7f89ef7f8740 4 client.6222 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:57:30.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.442+0000 7f89ef7f8740 1 -- 172.21.15.43:0/3202649327 --> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c62fdce340 con 0x55c62fe80070 2024-04-25T08:57:30.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.443+0000 7f89e659c640 1 --2- 172.21.15.43:0/3202649327 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55c62fe80070 0x55c62fe80480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.443+0000 7f89c57fa640 10 client.6222 ms_handle_connect on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:30.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.443+0000 7f89e659c640 1 --2- 172.21.15.43:0/3202649327 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55c62fe80070 0x55c62fe80480 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1eb0b2b34a85e385 server_cookie=583c759771a7e566 in_seq=0 out_seq=0 2024-04-25T08:57:30.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.444+0000 7f89c57fa640 1 -- 172.21.15.43:0/3202649327 <== mds.0 v2:172.21.15.193:6836/4164634380 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c62fdce340 con 0x55c62fe80070 2024-04-25T08:57:30.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.444+0000 7f89c57fa640 10 client.6222 handle_command_reply: tid=0 2024-04-25T08:57:30.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.444+0000 7f89ef7f8740 1 client.6222 shutdown 2024-04-25T08:57:30.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.444+0000 7f89ef7f8740 1 -- 172.21.15.43:0/3202649327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62feb5560 msgr2=0x55c62fe82220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:30.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.444+0000 7f89ef7f8740 1 --2- 172.21.15.43:0/3202649327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62feb5560 0x55c62fe82220 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f89e0078ef0 tx=0x7f89e0006c30 comp rx=0 tx=0).stop 2024-04-25T08:57:30.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.444+0000 7f89ef7f8740 1 -- 172.21.15.43:0/3202649327 shutdown_connections 2024-04-25T08:57:30.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.444+0000 7f89ef7f8740 1 --2- 172.21.15.43:0/3202649327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62feb5560 0x55c62fe82220 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.444+0000 7f89ef7f8740 1 --2- 172.21.15.43:0/3202649327 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55c62fe80070 0x55c62fe80480 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.444+0000 7f89ef7f8740 1 --2- 172.21.15.43:0/3202649327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c62fe84e80 0x55c62fe81ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.444+0000 7f89ef7f8740 1 --2- 172.21.15.43:0/3202649327 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c62fe84aa0 0x55c62feb35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.444+0000 7f89c57fa640 0 client.6222 ms_handle_reset on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:30.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.444+0000 7f89ef7f8740 1 -- 172.21.15.43:0/3202649327 >> 172.21.15.43:0/3202649327 conn(0x55c62fe57700 msgr2=0x55c62fe83120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:30.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.444+0000 7f89ef7f8740 1 -- 172.21.15.43:0/3202649327 shutdown_connections 2024-04-25T08:57:30.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.444+0000 7f89ef7f8740 1 -- 172.21.15.43:0/3202649327 wait complete. 2024-04-25T08:57:30.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.444+0000 7f89ef7f8740 20 client.6222 trim_cache size 0 max 16384 2024-04-25T08:57:30.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.456+0000 7f89ef7f8740 1 Processor -- start 2024-04-25T08:57:30.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.456+0000 7f89ef7f8740 1 -- start start 2024-04-25T08:57:30.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.457+0000 7f89ef7f8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62fe84aa0 0x55c62fe566d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.457+0000 7f89ef7f8740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c62fe84e80 0x55c62fe56c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.457+0000 7f89ef7f8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c62feb5560 0x55c62feb2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.457+0000 7f89ef7f8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e81ffc80 con 0x55c62fe84aa0 2024-04-25T08:57:30.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.457+0000 7f89ef7f8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e81feee0 con 0x55c62feb5560 2024-04-25T08:57:30.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.457+0000 7f89ef7f8740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e8159ff0 con 0x55c62fe84e80 2024-04-25T08:57:30.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.457+0000 7f89e659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62fe84aa0 0x55c62fe566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.457+0000 7f89e659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62fe84aa0 0x55c62fe566d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57872/0 (socket says 172.21.15.43:57872) 2024-04-25T08:57:30.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.457+0000 7f89e659c640 1 -- 172.21.15.43:0/202573614 learned_addr learned my addr 172.21.15.43:0/202573614 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:30.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.457+0000 7f89e5d9b640 1 --2- 172.21.15.43:0/202573614 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c62fe84e80 0x55c62fe56c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.457+0000 7f89e6d9d640 1 --2- 172.21.15.43:0/202573614 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c62feb5560 0x55c62feb2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.457+0000 7f89e659c640 1 -- 172.21.15.43:0/202573614 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c62fe84e80 msgr2=0x55c62fe56c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:30.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.457+0000 7f89e659c640 1 --2- 172.21.15.43:0/202573614 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c62fe84e80 0x55c62fe56c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.457+0000 7f89e659c640 1 -- 172.21.15.43:0/202573614 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c62feb5560 msgr2=0x55c62feb2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:30.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.457+0000 7f89e659c640 1 --2- 172.21.15.43:0/202573614 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c62feb5560 0x55c62feb2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.457+0000 7f89e659c640 1 -- 172.21.15.43:0/202573614 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c62fe80520 con 0x55c62fe84aa0 2024-04-25T08:57:30.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.457+0000 7f89e5d9b640 1 --2- 172.21.15.43:0/202573614 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c62fe84e80 0x55c62fe56c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:30.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.457+0000 7f89e659c640 1 --2- 172.21.15.43:0/202573614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62fe84aa0 0x55c62fe566d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x55c62fe82c40 tx=0x7f89d8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9173452987b89b3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:30.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.457+0000 7f89c77fe640 1 -- 172.21.15.43:0/202573614 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d8016a20 con 0x55c62fe84aa0 2024-04-25T08:57:30.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.457+0000 7f89c77fe640 1 -- 172.21.15.43:0/202573614 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d8016bc0 con 0x55c62fe84aa0 2024-04-25T08:57:30.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.457+0000 7f89c77fe640 1 -- 172.21.15.43:0/202573614 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d8003990 con 0x55c62fe84aa0 2024-04-25T08:57:30.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.458+0000 7f89ef7f8740 1 -- 172.21.15.43:0/202573614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62fe84aa0 msgr2=0x55c62fe566d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:30.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.458+0000 7f89ef7f8740 1 --2- 172.21.15.43:0/202573614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62fe84aa0 0x55c62fe566d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x55c62fe82c40 tx=0x7f89d8012000 comp rx=0 tx=0).stop 2024-04-25T08:57:30.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.458+0000 7f89ef7f8740 1 -- 172.21.15.43:0/202573614 shutdown_connections 2024-04-25T08:57:30.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.458+0000 7f89ef7f8740 1 --2- 172.21.15.43:0/202573614 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c62feb5560 0x55c62feb2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.458+0000 7f89ef7f8740 1 --2- 172.21.15.43:0/202573614 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c62fe84e80 0x55c62fe56c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.458+0000 7f89ef7f8740 1 --2- 172.21.15.43:0/202573614 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62fe84aa0 0x55c62fe566d0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.458+0000 7f89ef7f8740 1 -- 172.21.15.43:0/202573614 >> 172.21.15.43:0/202573614 conn(0x55c62fe57700 msgr2=0x55c62fea5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:30.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.458+0000 7f89ef7f8740 1 -- 172.21.15.43:0/202573614 shutdown_connections 2024-04-25T08:57:30.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.458+0000 7f89ef7f8740 1 -- 172.21.15.43:0/202573614 wait complete. 2024-04-25T08:57:30.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.458+0000 7f89ef7f8740 1 -- start start 2024-04-25T08:57:30.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.458+0000 7f89ef7f8740 1 Processor -- start 2024-04-25T08:57:30.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.459+0000 7f89ef7f8740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62fe84aa0 0x55c62ff0bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.459+0000 7f89ef7f8740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c62fe84e80 0x55c62febb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.459+0000 7f89ef7f8740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c62feb5560 0x55c62feb3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.459+0000 7f89ef7f8740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e81f74a0 con 0x55c62fe84aa0 2024-04-25T08:57:30.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.459+0000 7f89ef7f8740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c62fe816f0 con 0x55c62fe84e80 2024-04-25T08:57:30.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.459+0000 7f89ef7f8740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c62fea4d30 con 0x55c62feb5560 2024-04-25T08:57:30.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.459+0000 7f89ef7f8740 10 client.0 _ll_register_callbacks cb 0x7f89e0009400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:30.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.459+0000 7f89e659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62fe84aa0 0x55c62ff0bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.459+0000 7f89e659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62fe84aa0 0x55c62ff0bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57882/0 (socket says 172.21.15.43:57882) 2024-04-25T08:57:30.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.459+0000 7f89e659c640 1 -- 172.21.15.43:0/2783585027 learned_addr learned my addr 172.21.15.43:0/2783585027 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:30.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.460+0000 7f89e6d9d640 1 --2- 172.21.15.43:0/2783585027 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c62feb5560 0x55c62feb3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.460+0000 7f89e5d9b640 1 --2- 172.21.15.43:0/2783585027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c62fe84e80 0x55c62febb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.460+0000 7f89e659c640 1 -- 172.21.15.43:0/2783585027 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c62feb5560 msgr2=0x55c62feb3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:30.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.460+0000 7f89e659c640 1 --2- 172.21.15.43:0/2783585027 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c62feb5560 0x55c62feb3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.460+0000 7f89e659c640 1 -- 172.21.15.43:0/2783585027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c62fe84e80 msgr2=0x55c62febb380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:30.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.460+0000 7f89e659c640 1 --2- 172.21.15.43:0/2783585027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c62fe84e80 0x55c62febb380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.460+0000 7f89e659c640 1 -- 172.21.15.43:0/2783585027 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c62fe7bfb0 con 0x55c62fe84aa0 2024-04-25T08:57:30.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.460+0000 7f89e5d9b640 1 --2- 172.21.15.43:0/2783585027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c62fe84e80 0x55c62febb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:30.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.460+0000 7f89e659c640 1 --2- 172.21.15.43:0/2783585027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62fe84aa0 0x55c62ff0bf70 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f89d800b010 tx=0x7f89d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35dd5e4b593e9ee0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:30.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.460+0000 7f89c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:57:30.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.460+0000 7f89c57fa640 1 -- 172.21.15.43:0/2783585027 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d8039060 con 0x55c62fe84aa0 2024-04-25T08:57:30.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.460+0000 7f89c57fa640 1 -- 172.21.15.43:0/2783585027 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f89d80186e0 con 0x55c62fe84aa0 2024-04-25T08:57:30.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.460+0000 7f89ef7f8740 1 -- 172.21.15.43:0/2783585027 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f89d8016bc0 con 0x55c62fe84aa0 2024-04-25T08:57:30.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.460+0000 7f89c57fa640 1 -- 172.21.15.43:0/2783585027 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89d8004040 con 0x55c62fe84aa0 2024-04-25T08:57:30.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.460+0000 7f89ef7f8740 10 client.6228 fetch_fsmap learned FSMap version 22 2024-04-25T08:57:30.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.460+0000 7f89ef7f8740 1 -- 172.21.15.43:0/2783585027 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=22,monmap=2+}) v3 -- 0x55c62fea5400 con 0x55c62fe84aa0 2024-04-25T08:57:30.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.460+0000 7f89c57fa640 1 -- 172.21.15.43:0/2783585027 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d8016bc0 con 0x55c62fe84aa0 2024-04-25T08:57:30.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.460+0000 7f89c57fa640 1 -- 172.21.15.43:0/2783585027 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 22) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f89d8033070 con 0x55c62fe84aa0 2024-04-25T08:57:30.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.460+0000 7f89ef7f8740 10 client.6228 fetch_fsmap finished waiting for FSMap version 22 2024-04-25T08:57:30.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.460+0000 7f89ef7f8740 10 client.6228 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T08:57:30.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.460+0000 7f89ef7f8740 20 client.6228 populate_metadata read hostname 'smithi043' 2024-04-25T08:57:30.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.460+0000 7f89ef7f8740 1 --2- 172.21.15.43:0/2783585027 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55c62feb3260 0x55c62fe566d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.460+0000 7f89ef7f8740 4 client.6228 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T08:57:30.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.460+0000 7f89ef7f8740 1 -- 172.21.15.43:0/2783585027 --> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c62fdc8690 con 0x55c62feb3260 2024-04-25T08:57:30.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.461+0000 7f89e5d9b640 1 --2- 172.21.15.43:0/2783585027 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55c62feb3260 0x55c62fe566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.461+0000 7f89c57fa640 10 client.6228 ms_handle_connect on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:30.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.461+0000 7f89e5d9b640 1 --2- 172.21.15.43:0/2783585027 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55c62feb3260 0x55c62fe566d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d816232b4a68db37 server_cookie=be28568c1bf5ba5c in_seq=0 out_seq=0 2024-04-25T08:57:30.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.461+0000 7f89c57fa640 1 -- 172.21.15.43:0/2783585027 <== mds.0 v2:172.21.15.193:6836/4164634380 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c62fdc8690 con 0x55c62feb3260 2024-04-25T08:57:30.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.461+0000 7f89c57fa640 10 client.6228 handle_command_reply: tid=0 2024-04-25T08:57:30.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.461+0000 7f89ef7f8740 1 client.6228 shutdown 2024-04-25T08:57:30.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.461+0000 7f89ef7f8740 1 -- 172.21.15.43:0/2783585027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62fe84aa0 msgr2=0x55c62ff0bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:30.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.461+0000 7f89ef7f8740 1 --2- 172.21.15.43:0/2783585027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62fe84aa0 0x55c62ff0bf70 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f89d800b010 tx=0x7f89d8004a40 comp rx=0 tx=0).stop 2024-04-25T08:57:30.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.461+0000 7f89ef7f8740 1 -- 172.21.15.43:0/2783585027 shutdown_connections 2024-04-25T08:57:30.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.461+0000 7f89ef7f8740 1 --2- 172.21.15.43:0/2783585027 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c62feb5560 0x55c62feb3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.461+0000 7f89ef7f8740 1 --2- 172.21.15.43:0/2783585027 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55c62feb3260 0x55c62fe566d0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.461+0000 7f89ef7f8740 1 --2- 172.21.15.43:0/2783585027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c62fe84e80 0x55c62febb380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.461+0000 7f89c57fa640 0 client.6228 ms_handle_reset on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:30.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.461+0000 7f89ef7f8740 1 --2- 172.21.15.43:0/2783585027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c62fe84aa0 0x55c62ff0bf70 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.461+0000 7f89ef7f8740 1 -- 172.21.15.43:0/2783585027 >> 172.21.15.43:0/2783585027 conn(0x55c62fe57700 msgr2=0x55c62fec3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:30.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.461+0000 7f89ef7f8740 1 -- 172.21.15.43:0/2783585027 shutdown_connections 2024-04-25T08:57:30.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.462+0000 7f89ef7f8740 1 -- 172.21.15.43:0/2783585027 wait complete. 2024-04-25T08:57:30.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.462+0000 7f89ef7f8740 20 client.6228 trim_cache size 0 max 16384 2024-04-25T08:57:30.460 INFO:teuthology.orchestra.run.smithi043.stdout:[] 2024-04-25T08:57:30.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.462+0000 7f89ecaf5640 1 -- 172.21.15.43:0/4087653935 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f89c806b500 msgr2=0x7f89c806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:30.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.462+0000 7f89ecaf5640 1 --2- 172.21.15.43:0/4087653935 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f89c806b500 0x7f89c806d9c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f89d8007520 tx=0x7f89d800b040 comp rx=0 tx=0).stop 2024-04-25T08:57:30.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.462+0000 7f89ecaf5640 1 -- 172.21.15.43:0/4087653935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89e8161fe0 msgr2=0x7f89e80b8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:30.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.462+0000 7f89ecaf5640 1 --2- 172.21.15.43:0/4087653935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89e8161fe0 0x7f89e80b8390 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f89e0035040 tx=0x7f89e00046c0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.462+0000 7f89ecaf5640 1 -- 172.21.15.43:0/4087653935 shutdown_connections 2024-04-25T08:57:30.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.462+0000 7f89ecaf5640 1 --2- 172.21.15.43:0/4087653935 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89e8161fe0 0x7f89e80b8390 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.462+0000 7f89ecaf5640 1 --2- 172.21.15.43:0/4087653935 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f89e815e090 0x7f89e80b7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.462+0000 7f89ecaf5640 1 --2- 172.21.15.43:0/4087653935 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f89c806b500 0x7f89c806d9c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.462+0000 7f89ecaf5640 1 --2- 172.21.15.43:0/4087653935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89e80b5c70 0x7f89e80ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.462+0000 7f89ecaf5640 1 -- 172.21.15.43:0/4087653935 >> 172.21.15.43:0/4087653935 conn(0x7f89e80d0560 msgr2=0x7f89e80b47c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:30.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.462+0000 7f89ecaf5640 1 -- 172.21.15.43:0/4087653935 shutdown_connections 2024-04-25T08:57:30.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.462+0000 7f89ecaf5640 1 -- 172.21.15.43:0/4087653935 wait complete. 2024-04-25T08:57:30.463 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data rmxattr 10000000a4e.00000000 parent 2024-04-25T08:57:30.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.516+0000 7f6fa8f21f40 1 Processor -- start 2024-04-25T08:57:30.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.516+0000 7f6fa8f21f40 1 -- start start 2024-04-25T08:57:30.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.516+0000 7f6fa8f21f40 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aeb8f44f90 0x55aeb8f76450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.516+0000 7f6fa8f21f40 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55aeb8f90290 0x55aeb8f8e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.516+0000 7f6fa8f21f40 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aeb901df00 0x55aeb90141c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.516+0000 7f6fa8f21f40 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb8eeddf0 con 0x55aeb8f44f90 2024-04-25T08:57:30.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.516+0000 7f6fa8f21f40 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb8ef1d50 con 0x55aeb901df00 2024-04-25T08:57:30.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.516+0000 7f6fa8f21f40 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aeb8f907d0 con 0x55aeb8f90290 2024-04-25T08:57:30.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.516+0000 7f6fa5fd9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aeb8f44f90 0x55aeb8f76450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.517+0000 7f6fa5fd9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aeb8f44f90 0x55aeb8f76450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57900/0 (socket says 172.21.15.43:57900) 2024-04-25T08:57:30.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.517+0000 7f6fa5fd9640 1 -- 172.21.15.43:0/1740561554 learned_addr learned my addr 172.21.15.43:0/1740561554 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:30.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.517+0000 7f6fa67da640 1 --2- 172.21.15.43:0/1740561554 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aeb901df00 0x55aeb90141c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.517+0000 7f6fa57d8640 1 --2- 172.21.15.43:0/1740561554 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55aeb8f90290 0x55aeb8f8e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.517+0000 7f6fa5fd9640 1 -- 172.21.15.43:0/1740561554 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55aeb8f90290 msgr2=0x55aeb8f8e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:30.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.517+0000 7f6fa5fd9640 1 --2- 172.21.15.43:0/1740561554 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55aeb8f90290 0x55aeb8f8e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.517+0000 7f6fa5fd9640 1 -- 172.21.15.43:0/1740561554 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aeb901df00 msgr2=0x55aeb90141c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:30.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.517+0000 7f6fa5fd9640 1 --2- 172.21.15.43:0/1740561554 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aeb901df00 0x55aeb90141c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.517+0000 7f6fa5fd9640 1 -- 172.21.15.43:0/1740561554 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aeb901d430 con 0x55aeb8f44f90 2024-04-25T08:57:30.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.517+0000 7f6fa5fd9640 1 --2- 172.21.15.43:0/1740561554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aeb8f44f90 0x55aeb8f76450 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f6fa000a4f0 tx=0x7f6fa0051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f716ba9c0e2960d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:30.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.517+0000 7f6f9ffff640 1 -- 172.21.15.43:0/1740561554 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0062070 con 0x55aeb8f44f90 2024-04-25T08:57:30.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.517+0000 7f6f9ffff640 1 -- 172.21.15.43:0/1740561554 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa0054b70 con 0x55aeb8f44f90 2024-04-25T08:57:30.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.517+0000 7f6f9ffff640 1 -- 172.21.15.43:0/1740561554 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa00545e0 con 0x55aeb8f44f90 2024-04-25T08:57:30.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.518+0000 7f6fa8f21f40 1 -- 172.21.15.43:0/1740561554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aeb8f44f90 msgr2=0x55aeb8f76450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:30.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.518+0000 7f6fa8f21f40 1 --2- 172.21.15.43:0/1740561554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aeb8f44f90 0x55aeb8f76450 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f6fa000a4f0 tx=0x7f6fa0051000 comp rx=0 tx=0).stop 2024-04-25T08:57:30.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.518+0000 7f6fa8f21f40 1 -- 172.21.15.43:0/1740561554 shutdown_connections 2024-04-25T08:57:30.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.518+0000 7f6fa8f21f40 1 --2- 172.21.15.43:0/1740561554 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aeb901df00 0x55aeb90141c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.518+0000 7f6fa8f21f40 1 --2- 172.21.15.43:0/1740561554 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55aeb8f90290 0x55aeb8f8e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.518+0000 7f6fa8f21f40 1 --2- 172.21.15.43:0/1740561554 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aeb8f44f90 0x55aeb8f76450 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.518+0000 7f6fa8f21f40 1 -- 172.21.15.43:0/1740561554 >> 172.21.15.43:0/1740561554 conn(0x55aeb8f714a0 msgr2=0x55aeb8f48aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:30.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.518+0000 7f6fa8f21f40 1 -- 172.21.15.43:0/1740561554 shutdown_connections 2024-04-25T08:57:30.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.518+0000 7f6fa8f21f40 1 -- 172.21.15.43:0/1740561554 wait complete. 2024-04-25T08:57:30.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.518+0000 7f6fa8f21f40 1 Processor -- start 2024-04-25T08:57:30.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.518+0000 7f6fa8f21f40 1 -- start start 2024-04-25T08:57:30.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.519+0000 7f6fa8f21f40 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aeb8f44f90 0x55aeb9009530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.519+0000 7f6fa8f21f40 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55aeb8f90290 0x55aeb900ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.519+0000 7f6fa8f21f40 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aeb901df00 0x55aeb8f74b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.519+0000 7f6fa8f21f40 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb8f77420 con 0x55aeb8f44f90 2024-04-25T08:57:30.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.519+0000 7f6fa8f21f40 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aeb901d710 con 0x55aeb901df00 2024-04-25T08:57:30.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.519+0000 7f6fa8f21f40 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aeb8f75f30 con 0x55aeb8f90290 2024-04-25T08:57:30.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.519+0000 7f6fa5fd9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aeb8f44f90 0x55aeb9009530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.519+0000 7f6fa5fd9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aeb8f44f90 0x55aeb9009530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57908/0 (socket says 172.21.15.43:57908) 2024-04-25T08:57:30.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.519+0000 7f6fa5fd9640 1 -- 172.21.15.43:0/594478870 learned_addr learned my addr 172.21.15.43:0/594478870 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:30.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.519+0000 7f6fa67da640 1 --2- 172.21.15.43:0/594478870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aeb901df00 0x55aeb8f74b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.519+0000 7f6fa57d8640 1 --2- 172.21.15.43:0/594478870 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55aeb8f90290 0x55aeb900ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.519+0000 7f6fa5fd9640 1 -- 172.21.15.43:0/594478870 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55aeb8f90290 msgr2=0x55aeb900ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:30.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.519+0000 7f6fa5fd9640 1 --2- 172.21.15.43:0/594478870 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55aeb8f90290 0x55aeb900ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.519+0000 7f6fa5fd9640 1 -- 172.21.15.43:0/594478870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aeb901df00 msgr2=0x55aeb8f74b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:30.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.519+0000 7f6fa5fd9640 1 --2- 172.21.15.43:0/594478870 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aeb901df00 0x55aeb8f74b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.519+0000 7f6fa5fd9640 1 -- 172.21.15.43:0/594478870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aeb9010090 con 0x55aeb8f44f90 2024-04-25T08:57:30.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.520+0000 7f6fa5fd9640 1 --2- 172.21.15.43:0/594478870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aeb8f44f90 0x55aeb9009530 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0025ba0 tx=0x7f6fa0002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:30.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.520+0000 7f6f9effd640 1 -- 172.21.15.43:0/594478870 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa0062040 con 0x55aeb8f44f90 2024-04-25T08:57:30.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.520+0000 7f6f9effd640 1 -- 172.21.15.43:0/594478870 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa0054d20 con 0x55aeb8f44f90 2024-04-25T08:57:30.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.520+0000 7f6f9effd640 1 -- 172.21.15.43:0/594478870 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa00676a0 con 0x55aeb8f44f90 2024-04-25T08:57:30.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.520+0000 7f6fa8f21f40 1 -- 172.21.15.43:0/594478870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55aeb9010370 con 0x55aeb8f44f90 2024-04-25T08:57:30.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.520+0000 7f6fa8f21f40 1 -- 172.21.15.43:0/594478870 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aeb900fcd0 con 0x55aeb8f44f90 2024-04-25T08:57:30.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.521+0000 7f6f9effd640 1 -- 172.21.15.43:0/594478870 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6fa0067840 con 0x55aeb8f44f90 2024-04-25T08:57:30.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.522+0000 7f6f9effd640 1 --2- 172.21.15.43:0/594478870 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6f88069f30 0x7f6f8806c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.522+0000 7f6f9effd640 1 -- 172.21.15.43:0/594478870 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6fa00ce5e0 con 0x55aeb8f44f90 2024-04-25T08:57:30.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.522+0000 7f6fa57d8640 1 --2- 172.21.15.43:0/594478870 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6f88069f30 0x7f6f8806c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.522+0000 7f6fa8f21f40 1 --2- 172.21.15.43:0/594478870 >> [v2:172.21.15.43:6800/1101549916,v1:172.21.15.43:6801/1101549916] conn(0x55aeb900e310 0x55aeb9020080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.522+0000 7f6fa8f21f40 1 -- 172.21.15.43:0/594478870 --> [v2:172.21.15.43:6800/1101549916,v1:172.21.15.43:6801/1101549916] -- osd_op(unknown.0.0:1 7.2 7:43adcc81:::10000000a4e.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46) v9 -- 0x55aeb9009aa0 con 0x55aeb900e310 2024-04-25T08:57:30.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.522+0000 7f6fa67da640 1 --2- 172.21.15.43:0/594478870 >> [v2:172.21.15.43:6800/1101549916,v1:172.21.15.43:6801/1101549916] conn(0x55aeb900e310 0x55aeb9020080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.522+0000 7f6fa57d8640 1 --2- 172.21.15.43:0/594478870 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6f88069f30 0x7f6f8806c3f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55aeb900d710 tx=0x7f6f900017f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:30.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.522+0000 7f6fa67da640 1 --2- 172.21.15.43:0/594478870 >> [v2:172.21.15.43:6800/1101549916,v1:172.21.15.43:6801/1101549916] conn(0x55aeb900e310 0x55aeb9020080 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:30.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.523+0000 7f6fa67da640 1 -- 172.21.15.43:0/594478870 <== osd.3 v2:172.21.15.43:6800/1101549916 1 ==== osd_op_reply(1 10000000a4e.00000000 [rmxattr] v46'18 uv18 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f6f9800f730 con 0x55aeb900e310 2024-04-25T08:57:30.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.524+0000 7f6fa8f21f40 1 -- 172.21.15.43:0/594478870 >> [v2:172.21.15.43:6800/1101549916,v1:172.21.15.43:6801/1101549916] conn(0x55aeb900e310 msgr2=0x55aeb9020080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:30.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.524+0000 7f6fa8f21f40 1 --2- 172.21.15.43:0/594478870 >> [v2:172.21.15.43:6800/1101549916,v1:172.21.15.43:6801/1101549916] conn(0x55aeb900e310 0x55aeb9020080 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.524+0000 7f6fa8f21f40 1 -- 172.21.15.43:0/594478870 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6f88069f30 msgr2=0x7f6f8806c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:30.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.524+0000 7f6fa8f21f40 1 --2- 172.21.15.43:0/594478870 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6f88069f30 0x7f6f8806c3f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55aeb900d710 tx=0x7f6f900017f0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.524+0000 7f6fa8f21f40 1 -- 172.21.15.43:0/594478870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aeb8f44f90 msgr2=0x55aeb9009530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:30.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.524+0000 7f6fa8f21f40 1 --2- 172.21.15.43:0/594478870 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aeb8f44f90 0x55aeb9009530 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f6fa0025ba0 tx=0x7f6fa0002fb0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.524+0000 7f6fa67da640 1 -- 172.21.15.43:0/594478870 reap_dead start 2024-04-25T08:57:30.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.524+0000 7f6fa8f21f40 1 -- 172.21.15.43:0/594478870 shutdown_connections 2024-04-25T08:57:30.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.524+0000 7f6fa8f21f40 1 -- 172.21.15.43:0/594478870 >> 172.21.15.43:0/594478870 conn(0x55aeb8f714a0 msgr2=0x55aeb8f727e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:30.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.524+0000 7f6fa8f21f40 1 -- 172.21.15.43:0/594478870 shutdown_connections 2024-04-25T08:57:30.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.524+0000 7f6fa8f21f40 1 -- 172.21.15.43:0/594478870 wait complete. 2024-04-25T08:57:30.517 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_1' 2024-04-25T08:57:30.517 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:57:30.517 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/new_file_1 2024-04-25T08:57:30.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.671+0000 7fede3e91640 1 Processor -- start 2024-04-25T08:57:30.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.671+0000 7fede3e91640 1 -- start start 2024-04-25T08:57:30.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.672+0000 7fede3e91640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feddc0b5560 0x7feddc0b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.672+0000 7fede3e91640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7feddc0d2230 0x7feddc0d2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.672+0000 7fede3e91640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feddc162610 con 0x7feddc15e090 2024-04-25T08:57:30.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.672+0000 7fede3e91640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feddc0d1da0 con 0x7feddc0d2230 2024-04-25T08:57:30.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.672+0000 7fede3e91640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feddc0d1750 con 0x7feddc0b5560 2024-04-25T08:57:30.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.672+0000 7fede242e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7feddc0d2230 0x7feddc0d2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.672+0000 7fede1c2d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7feddc15e090 0x7feddc161aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:52706/0 (socket says 172.21.15.43:52706) 2024-04-25T08:57:30.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.672+0000 7fede1c2d640 1 -- 172.21.15.43:0/3932176914 learned_addr learned my addr 172.21.15.43:0/3932176914 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:30.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.673+0000 7fede142c640 1 --2- 172.21.15.43:0/3932176914 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feddc0b5560 0x7feddc0b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.673+0000 7fede242e640 1 -- 172.21.15.43:0/3932176914 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feddc0b5560 msgr2=0x7feddc0b59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:30.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.673+0000 7fede242e640 1 --2- 172.21.15.43:0/3932176914 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feddc0b5560 0x7feddc0b59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.673+0000 7fede242e640 1 -- 172.21.15.43:0/3932176914 >> v1:172.21.15.43:6789/0 conn(0x7feddc15e090 legacy=0x7feddc161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:30.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.673+0000 7fede242e640 1 -- 172.21.15.43:0/3932176914 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feddc0d1a30 con 0x7feddc0d2230 2024-04-25T08:57:30.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.673+0000 7fede0c2b640 1 -- 172.21.15.43:0/3932176914 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3526056619 0 0) 0x7feddc162610 con 0x7feddc15e090 2024-04-25T08:57:30.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.673+0000 7fede242e640 1 --2- 172.21.15.43:0/3932176914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feddc0d2230 0x7feddc0d2690 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fedd80097a0 tx=0x7fedd802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7107e42db60cf9d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:30.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.674+0000 7fede0c2b640 1 -- 172.21.15.43:0/3932176914 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd8037070 con 0x7feddc0d2230 2024-04-25T08:57:30.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.674+0000 7fede0c2b640 1 -- 172.21.15.43:0/3932176914 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedd8032440 con 0x7feddc0d2230 2024-04-25T08:57:30.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.674+0000 7fede0c2b640 1 -- 172.21.15.43:0/3932176914 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd8032810 con 0x7feddc0d2230 2024-04-25T08:57:30.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.674+0000 7fede3e91640 1 -- 172.21.15.43:0/3932176914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feddc0d2230 msgr2=0x7feddc0d2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:30.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.674+0000 7fede3e91640 1 --2- 172.21.15.43:0/3932176914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feddc0d2230 0x7feddc0d2690 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fedd80097a0 tx=0x7fedd802a210 comp rx=0 tx=0).stop 2024-04-25T08:57:30.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.674+0000 7fede3e91640 1 -- 172.21.15.43:0/3932176914 shutdown_connections 2024-04-25T08:57:30.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.674+0000 7fede3e91640 1 --2- 172.21.15.43:0/3932176914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feddc0d2230 0x7feddc0d2690 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.674+0000 7fede3e91640 1 --2- 172.21.15.43:0/3932176914 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feddc0b5560 0x7feddc0b59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.674+0000 7fede3e91640 1 -- 172.21.15.43:0/3932176914 >> 172.21.15.43:0/3932176914 conn(0x7feddc0d0560 msgr2=0x7feddc0d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:30.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.674+0000 7fede3e91640 1 -- 172.21.15.43:0/3932176914 shutdown_connections 2024-04-25T08:57:30.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.674+0000 7fede3e91640 1 -- 172.21.15.43:0/3932176914 wait complete. 2024-04-25T08:57:30.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.675+0000 7fede3e91640 1 Processor -- start 2024-04-25T08:57:30.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.675+0000 7fede3e91640 1 -- start start 2024-04-25T08:57:30.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.676+0000 7fede3e91640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feddc0b5560 0x7feddc0c9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.676+0000 7fede3e91640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feddc0d2230 0x7feddc160e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.676+0000 7fede3e91640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feddc15e090 0x7feddc161360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.676+0000 7fede3e91640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feddc1621b0 con 0x7feddc15e090 2024-04-25T08:57:30.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.676+0000 7fede3e91640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feddc0b3280 con 0x7feddc0d2230 2024-04-25T08:57:30.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.676+0000 7fede3e91640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feddc162330 con 0x7feddc0b5560 2024-04-25T08:57:30.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.676+0000 7fede242e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feddc15e090 0x7feddc161360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.676+0000 7fede242e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feddc15e090 0x7feddc161360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57930/0 (socket says 172.21.15.43:57930) 2024-04-25T08:57:30.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.676+0000 7fede242e640 1 -- 172.21.15.43:0/4216750783 learned_addr learned my addr 172.21.15.43:0/4216750783 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:30.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.676+0000 7fede1c2d640 1 --2- 172.21.15.43:0/4216750783 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feddc0b5560 0x7feddc0c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.677+0000 7fede142c640 1 --2- 172.21.15.43:0/4216750783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feddc0d2230 0x7feddc160e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.677+0000 7fede242e640 1 -- 172.21.15.43:0/4216750783 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feddc0b5560 msgr2=0x7feddc0c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:30.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.677+0000 7fede242e640 1 --2- 172.21.15.43:0/4216750783 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feddc0b5560 0x7feddc0c9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.677+0000 7fede242e640 1 -- 172.21.15.43:0/4216750783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feddc0d2230 msgr2=0x7feddc160e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:30.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.677+0000 7fede242e640 1 --2- 172.21.15.43:0/4216750783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feddc0d2230 0x7feddc160e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.677+0000 7fede242e640 1 -- 172.21.15.43:0/4216750783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feddc0c9ca0 con 0x7feddc15e090 2024-04-25T08:57:30.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.677+0000 7fede142c640 1 --2- 172.21.15.43:0/4216750783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feddc0d2230 0x7feddc160e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:30.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.677+0000 7fede1c2d640 1 --2- 172.21.15.43:0/4216750783 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feddc0b5560 0x7feddc0c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:30.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.677+0000 7fede242e640 1 --2- 172.21.15.43:0/4216750783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feddc15e090 0x7feddc161360 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fedd80096f0 tx=0x7fedd80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:30.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.677+0000 7fedd27fc640 1 -- 172.21.15.43:0/4216750783 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd8041030 con 0x7feddc15e090 2024-04-25T08:57:30.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.677+0000 7fedd27fc640 1 -- 172.21.15.43:0/4216750783 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedd803d4e0 con 0x7feddc15e090 2024-04-25T08:57:30.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.677+0000 7fedd27fc640 1 -- 172.21.15.43:0/4216750783 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd8037040 con 0x7feddc15e090 2024-04-25T08:57:30.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.677+0000 7fede3e91640 1 -- 172.21.15.43:0/4216750783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feddc1f5380 con 0x7feddc15e090 2024-04-25T08:57:30.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.677+0000 7fede3e91640 1 -- 172.21.15.43:0/4216750783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feddc1f5c20 con 0x7feddc15e090 2024-04-25T08:57:30.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.679+0000 7fedd27fc640 1 -- 172.21.15.43:0/4216750783 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fedd8044070 con 0x7feddc15e090 2024-04-25T08:57:30.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.679+0000 7fedd27fc640 1 --2- 172.21.15.43:0/4216750783 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fedc006b4d0 0x7fedc006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.679+0000 7fedd27fc640 1 -- 172.21.15.43:0/4216750783 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fedd80aa880 con 0x7feddc15e090 2024-04-25T08:57:30.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.679+0000 7fede1c2d640 1 --2- 172.21.15.43:0/4216750783 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fedc006b4d0 0x7fedc006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.680+0000 7fede1c2d640 1 --2- 172.21.15.43:0/4216750783 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fedc006b4d0 0x7fedc006d990 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fedd40012e0 tx=0x7fedd4005b90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:30.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.680+0000 7fede6b8b740 1 Processor -- start 2024-04-25T08:57:30.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.680+0000 7fede6b8b740 1 -- start start 2024-04-25T08:57:30.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.681+0000 7fede6b8b740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558c86735360 0x558c867557d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.681+0000 7fede6b8b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c86714360 0x558c8673cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.681+0000 7fede6b8b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558c8673d140 0x558c867127c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.681+0000 7fede6b8b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c86675ee0 con 0x558c8673d140 2024-04-25T08:57:30.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.681+0000 7fede6b8b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c8661ec50 con 0x558c86714360 2024-04-25T08:57:30.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.681+0000 7fede6b8b740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x558c86671f80 con 0x558c86735360 2024-04-25T08:57:30.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.681+0000 7fede242e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558c8673d140 0x558c867127c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.681+0000 7fede242e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558c8673d140 0x558c867127c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57938/0 (socket says 172.21.15.43:57938) 2024-04-25T08:57:30.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.681+0000 7fede242e640 1 -- 172.21.15.43:0/702339942 learned_addr learned my addr 172.21.15.43:0/702339942 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:30.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.681+0000 7fede1c2d640 1 --2- 172.21.15.43:0/702339942 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558c86735360 0x558c867557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.681+0000 7fede142c640 1 --2- 172.21.15.43:0/702339942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c86714360 0x558c8673cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.681+0000 7fede242e640 1 -- 172.21.15.43:0/702339942 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558c86735360 msgr2=0x558c867557d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:30.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.681+0000 7fede242e640 1 --2- 172.21.15.43:0/702339942 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558c86735360 0x558c867557d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.681+0000 7fede242e640 1 -- 172.21.15.43:0/702339942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c86714360 msgr2=0x558c8673cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:30.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.681+0000 7fede242e640 1 --2- 172.21.15.43:0/702339942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c86714360 0x558c8673cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.681+0000 7fede242e640 1 -- 172.21.15.43:0/702339942 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c86456c00 con 0x558c8673d140 2024-04-25T08:57:30.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.681+0000 7fede242e640 1 --2- 172.21.15.43:0/702339942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558c8673d140 0x558c867127c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fedd80a8dd0 tx=0x7fedd8078a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a4f5f4f78bbd36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:30.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.681+0000 7fedbf7fe640 1 -- 172.21.15.43:0/702339942 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd807d270 con 0x558c8673d140 2024-04-25T08:57:30.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.681+0000 7fedbf7fe640 1 -- 172.21.15.43:0/702339942 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedd808f030 con 0x558c8673d140 2024-04-25T08:57:30.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.681+0000 7fedbf7fe640 1 -- 172.21.15.43:0/702339942 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd8081830 con 0x558c8673d140 2024-04-25T08:57:30.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.682+0000 7fede6b8b740 1 -- 172.21.15.43:0/702339942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558c8673d140 msgr2=0x558c867127c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:30.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.682+0000 7fede6b8b740 1 --2- 172.21.15.43:0/702339942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558c8673d140 0x558c867127c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fedd80a8dd0 tx=0x7fedd8078a80 comp rx=0 tx=0).stop 2024-04-25T08:57:30.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.682+0000 7fede6b8b740 1 -- 172.21.15.43:0/702339942 shutdown_connections 2024-04-25T08:57:30.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.682+0000 7fede6b8b740 1 --2- 172.21.15.43:0/702339942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558c8673d140 0x558c867127c0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.682+0000 7fede6b8b740 1 --2- 172.21.15.43:0/702339942 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c86714360 0x558c8673cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.682+0000 7fede6b8b740 1 --2- 172.21.15.43:0/702339942 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558c86735360 0x558c867557d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.682+0000 7fede6b8b740 1 -- 172.21.15.43:0/702339942 >> 172.21.15.43:0/702339942 conn(0x558c866d9810 msgr2=0x558c86757c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:30.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.682+0000 7fede6b8b740 1 -- 172.21.15.43:0/702339942 shutdown_connections 2024-04-25T08:57:30.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.682+0000 7fede6b8b740 1 -- 172.21.15.43:0/702339942 wait complete. 2024-04-25T08:57:30.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.683+0000 7fede6b8b740 1 -- start start 2024-04-25T08:57:30.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.683+0000 7fede6b8b740 1 Processor -- start 2024-04-25T08:57:30.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.684+0000 7fede6b8b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558c86714360 0x558c86734260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.684+0000 7fede6b8b740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558c86735360 0x558c8676b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.684+0000 7fede6b8b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c8673d140 0x558c8673dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.684+0000 7fede6b8b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c8661ec50 con 0x558c86714360 2024-04-25T08:57:30.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.684+0000 7fede6b8b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c86614340 con 0x558c8673d140 2024-04-25T08:57:30.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.684+0000 7fede6b8b740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x558c86675ee0 con 0x558c86735360 2024-04-25T08:57:30.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.684+0000 7fede6b8b740 10 client.0 _ll_register_callbacks cb 0x558c86389fe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:30.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.684+0000 7fede1c2d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558c86714360 0x558c86734260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.684+0000 7fede1c2d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558c86714360 0x558c86734260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57954/0 (socket says 172.21.15.43:57954) 2024-04-25T08:57:30.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.684+0000 7fede1c2d640 1 -- 172.21.15.43:0/1785141135 learned_addr learned my addr 172.21.15.43:0/1785141135 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:30.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.684+0000 7fede242e640 1 --2- 172.21.15.43:0/1785141135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c8673d140 0x558c8673dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.684+0000 7fede142c640 1 --2- 172.21.15.43:0/1785141135 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558c86735360 0x558c8676b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.684+0000 7fede1c2d640 1 -- 172.21.15.43:0/1785141135 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558c86735360 msgr2=0x558c8676b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:30.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.684+0000 7fede1c2d640 1 --2- 172.21.15.43:0/1785141135 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558c86735360 0x558c8676b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.684+0000 7fede1c2d640 1 -- 172.21.15.43:0/1785141135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c8673d140 msgr2=0x558c8673dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:30.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.684+0000 7fede1c2d640 1 --2- 172.21.15.43:0/1785141135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c8673d140 0x558c8673dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.684+0000 7fede1c2d640 1 -- 172.21.15.43:0/1785141135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558c8634a4f0 con 0x558c86714360 2024-04-25T08:57:30.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.684+0000 7fede242e640 1 --2- 172.21.15.43:0/1785141135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c8673d140 0x558c8673dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:30.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.684+0000 7fede1c2d640 1 --2- 172.21.15.43:0/1785141135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558c86714360 0x558c86734260 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fedd4014710 tx=0x7fedd4001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e65f2c6b4b4c5d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:30.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.684+0000 7fedbcff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:57:30.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.684+0000 7fedbcff9640 1 -- 172.21.15.43:0/1785141135 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd40126d0 con 0x558c86714360 2024-04-25T08:57:30.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.684+0000 7fedbcff9640 1 -- 172.21.15.43:0/1785141135 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fedd4010040 con 0x558c86714360 2024-04-25T08:57:30.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.684+0000 7fede6b8b740 1 -- 172.21.15.43:0/1785141135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fedd808f030 con 0x558c86714360 2024-04-25T08:57:30.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.685+0000 7fedbcff9640 1 -- 172.21.15.43:0/1785141135 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fedd4012870 con 0x558c86714360 2024-04-25T08:57:30.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.685+0000 7fede6b8b740 10 client.6255 fetch_fsmap learned FSMap version 23 2024-04-25T08:57:30.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.685+0000 7fede6b8b740 1 -- 172.21.15.43:0/1785141135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x558c864b7390 con 0x558c86714360 2024-04-25T08:57:30.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.685+0000 7fedbcff9640 1 -- 172.21.15.43:0/1785141135 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedd808f030 con 0x558c86714360 2024-04-25T08:57:30.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.685+0000 7fedbcff9640 1 -- 172.21.15.43:0/1785141135 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fedd4012a10 con 0x558c86714360 2024-04-25T08:57:30.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.685+0000 7fede6b8b740 10 client.6255 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T08:57:30.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.685+0000 7fede6b8b740 10 client.6255 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T08:57:30.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.685+0000 7fede6b8b740 20 client.6255 populate_metadata read hostname 'smithi043' 2024-04-25T08:57:30.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.685+0000 7fede6b8b740 1 --2- 172.21.15.43:0/1785141135 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x558c86750380 0x558c8675a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.685+0000 7fede6b8b740 4 client.6255 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:57:30.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.685+0000 7fede6b8b740 1 -- 172.21.15.43:0/1785141135 --> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558c86683420 con 0x558c86750380 2024-04-25T08:57:30.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.685+0000 7fede142c640 1 --2- 172.21.15.43:0/1785141135 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x558c86750380 0x558c8675a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.686+0000 7fede142c640 1 --2- 172.21.15.43:0/1785141135 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x558c86750380 0x558c8675a100 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff389d1dbec79c9a server_cookie=ab5515d910c918a9 in_seq=0 out_seq=0 2024-04-25T08:57:30.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.686+0000 7fedbcff9640 10 client.6255 ms_handle_connect on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:30.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.686+0000 7fedbcff9640 1 -- 172.21.15.43:0/1785141135 <== mds.0 v2:172.21.15.193:6836/4164634380 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558c86683420 con 0x558c86750380 2024-04-25T08:57:30.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.686+0000 7fedbcff9640 10 client.6255 handle_command_reply: tid=0 2024-04-25T08:57:30.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.686+0000 7fede6b8b740 1 client.6255 shutdown 2024-04-25T08:57:30.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.686+0000 7fede6b8b740 1 -- 172.21.15.43:0/1785141135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558c86714360 msgr2=0x558c86734260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:30.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.686+0000 7fede6b8b740 1 --2- 172.21.15.43:0/1785141135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558c86714360 0x558c86734260 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fedd4014710 tx=0x7fedd4001250 comp rx=0 tx=0).stop 2024-04-25T08:57:30.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.686+0000 7fede6b8b740 1 -- 172.21.15.43:0/1785141135 shutdown_connections 2024-04-25T08:57:30.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.686+0000 7fede6b8b740 1 --2- 172.21.15.43:0/1785141135 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x558c86750380 0x558c8675a100 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.686+0000 7fede6b8b740 1 --2- 172.21.15.43:0/1785141135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c8673d140 0x558c8673dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.686+0000 7fede6b8b740 1 --2- 172.21.15.43:0/1785141135 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558c86735360 0x558c8676b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.686+0000 7fede6b8b740 1 --2- 172.21.15.43:0/1785141135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558c86714360 0x558c86734260 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.686+0000 7fedbcff9640 0 client.6255 ms_handle_reset on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:30.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.686+0000 7fede6b8b740 1 -- 172.21.15.43:0/1785141135 >> 172.21.15.43:0/1785141135 conn(0x558c866d9810 msgr2=0x558c86711380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:30.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.686+0000 7fede6b8b740 1 -- 172.21.15.43:0/1785141135 shutdown_connections 2024-04-25T08:57:30.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.686+0000 7fede6b8b740 1 -- 172.21.15.43:0/1785141135 wait complete. 2024-04-25T08:57:30.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.687+0000 7fede6b8b740 20 client.6255 trim_cache size 0 max 16384 2024-04-25T08:57:30.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.702+0000 7fede6b8b740 1 Processor -- start 2024-04-25T08:57:30.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.702+0000 7fede6b8b740 1 -- start start 2024-04-25T08:57:30.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.702+0000 7fede6b8b740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558c86714360 0x558c86759b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.702+0000 7fede6b8b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558c86735360 0x558c8675a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.702+0000 7fede6b8b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c8673d140 0x558c86712310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.702+0000 7fede6b8b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feddc1ffca0 con 0x558c86735360 2024-04-25T08:57:30.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.702+0000 7fede6b8b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feddc1fee60 con 0x558c8673d140 2024-04-25T08:57:30.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.702+0000 7fede6b8b740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feddc1f7260 con 0x558c86714360 2024-04-25T08:57:30.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.702+0000 7fede142c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558c86735360 0x558c8675a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.703+0000 7fede142c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558c86735360 0x558c8675a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57970/0 (socket says 172.21.15.43:57970) 2024-04-25T08:57:30.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.703+0000 7fede142c640 1 -- 172.21.15.43:0/2022178789 learned_addr learned my addr 172.21.15.43:0/2022178789 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:30.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.703+0000 7fede242e640 1 --2- 172.21.15.43:0/2022178789 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c8673d140 0x558c86712310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.703+0000 7fede1c2d640 1 --2- 172.21.15.43:0/2022178789 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558c86714360 0x558c86759b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.703+0000 7fede142c640 1 -- 172.21.15.43:0/2022178789 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558c86714360 msgr2=0x558c86759b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:30.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.703+0000 7fede142c640 1 --2- 172.21.15.43:0/2022178789 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558c86714360 0x558c86759b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.703+0000 7fede142c640 1 -- 172.21.15.43:0/2022178789 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c8673d140 msgr2=0x558c86712310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:30.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.703+0000 7fede142c640 1 --2- 172.21.15.43:0/2022178789 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c8673d140 0x558c86712310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.703+0000 7fede142c640 1 -- 172.21.15.43:0/2022178789 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c86323670 con 0x558c86735360 2024-04-25T08:57:30.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.703+0000 7fede242e640 1 --2- 172.21.15.43:0/2022178789 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c8673d140 0x558c86712310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:30.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.703+0000 7fede142c640 1 --2- 172.21.15.43:0/2022178789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558c86735360 0x558c8675a0a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7feddc15d300 tx=0x7fedcc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c20bf9d83ad6c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:30.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.703+0000 7fedbeffd640 1 -- 172.21.15.43:0/2022178789 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedcc021020 con 0x558c86735360 2024-04-25T08:57:30.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.703+0000 7fedbeffd640 1 -- 172.21.15.43:0/2022178789 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedcc01ddc0 con 0x558c86735360 2024-04-25T08:57:30.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.703+0000 7fedbeffd640 1 -- 172.21.15.43:0/2022178789 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedcc003c90 con 0x558c86735360 2024-04-25T08:57:30.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.704+0000 7fede6b8b740 1 -- 172.21.15.43:0/2022178789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558c86735360 msgr2=0x558c8675a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:30.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.704+0000 7fede6b8b740 1 --2- 172.21.15.43:0/2022178789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558c86735360 0x558c8675a0a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7feddc15d300 tx=0x7fedcc008040 comp rx=0 tx=0).stop 2024-04-25T08:57:30.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.704+0000 7fede6b8b740 1 -- 172.21.15.43:0/2022178789 shutdown_connections 2024-04-25T08:57:30.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.704+0000 7fede6b8b740 1 --2- 172.21.15.43:0/2022178789 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c8673d140 0x558c86712310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.704+0000 7fede6b8b740 1 --2- 172.21.15.43:0/2022178789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558c86735360 0x558c8675a0a0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.704+0000 7fede6b8b740 1 --2- 172.21.15.43:0/2022178789 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558c86714360 0x558c86759b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.704+0000 7fede6b8b740 1 -- 172.21.15.43:0/2022178789 >> 172.21.15.43:0/2022178789 conn(0x558c866d9810 msgr2=0x558c86708090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:30.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.704+0000 7fede6b8b740 1 -- 172.21.15.43:0/2022178789 shutdown_connections 2024-04-25T08:57:30.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.704+0000 7fede6b8b740 1 -- 172.21.15.43:0/2022178789 wait complete. 2024-04-25T08:57:30.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.704+0000 7fede6b8b740 1 -- start start 2024-04-25T08:57:30.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.704+0000 7fede6b8b740 1 Processor -- start 2024-04-25T08:57:30.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.705+0000 7fede6b8b740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558c86714360 0x558c86712310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.705+0000 7fede6b8b740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558c86735360 0x558c86712850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.705+0000 7fede6b8b740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c8673d140 0x558c86766300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.705+0000 7fede6b8b740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c86758ee0 con 0x558c86735360 2024-04-25T08:57:30.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.705+0000 7fede6b8b740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feddc1fe890 con 0x558c8673d140 2024-04-25T08:57:30.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.705+0000 7fede6b8b740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x558c86767330 con 0x558c86714360 2024-04-25T08:57:30.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.705+0000 7fede6b8b740 10 client.0 _ll_register_callbacks cb 0x7fedd8009400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:30.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.705+0000 7fede142c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558c86735360 0x558c86712850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.705+0000 7fede142c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558c86735360 0x558c86712850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57986/0 (socket says 172.21.15.43:57986) 2024-04-25T08:57:30.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.705+0000 7fede142c640 1 -- 172.21.15.43:0/186076398 learned_addr learned my addr 172.21.15.43:0/186076398 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:30.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.706+0000 7fede242e640 1 --2- 172.21.15.43:0/186076398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c8673d140 0x558c86766300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.706+0000 7fede1c2d640 1 --2- 172.21.15.43:0/186076398 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558c86714360 0x558c86712310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.706+0000 7fede142c640 1 -- 172.21.15.43:0/186076398 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558c86714360 msgr2=0x558c86712310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:30.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.706+0000 7fede142c640 1 --2- 172.21.15.43:0/186076398 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558c86714360 0x558c86712310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.706+0000 7fede142c640 1 -- 172.21.15.43:0/186076398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c8673d140 msgr2=0x558c86766300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:30.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.706+0000 7fede142c640 1 --2- 172.21.15.43:0/186076398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c8673d140 0x558c86766300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.706+0000 7fede142c640 1 -- 172.21.15.43:0/186076398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558c8675ba60 con 0x558c86735360 2024-04-25T08:57:30.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.706+0000 7fede142c640 1 --2- 172.21.15.43:0/186076398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558c86735360 0x558c86712850 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fedcc0039e0 tx=0x7fedd4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b6366f2a86b2d86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:30.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.706+0000 7fedbcff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:57:30.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.706+0000 7fedbcff9640 1 -- 172.21.15.43:0/186076398 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedcc021030 con 0x558c86735360 2024-04-25T08:57:30.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.706+0000 7fedbcff9640 1 -- 172.21.15.43:0/186076398 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fedcc01d130 con 0x558c86735360 2024-04-25T08:57:30.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.706+0000 7fede6b8b740 1 -- 172.21.15.43:0/186076398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fedcc01ddc0 con 0x558c86735360 2024-04-25T08:57:30.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.706+0000 7fedbcff9640 1 -- 172.21.15.43:0/186076398 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fedcc036050 con 0x558c86735360 2024-04-25T08:57:30.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.706+0000 7fede6b8b740 10 client.6261 fetch_fsmap learned FSMap version 23 2024-04-25T08:57:30.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.706+0000 7fede6b8b740 1 -- 172.21.15.43:0/186076398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x558c86767130 con 0x558c86735360 2024-04-25T08:57:30.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.706+0000 7fedbcff9640 1 -- 172.21.15.43:0/186076398 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedcc01ddc0 con 0x558c86735360 2024-04-25T08:57:30.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.706+0000 7fedbcff9640 1 -- 172.21.15.43:0/186076398 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fedcc039020 con 0x558c86735360 2024-04-25T08:57:30.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.706+0000 7fede6b8b740 10 client.6261 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T08:57:30.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.706+0000 7fede6b8b740 10 client.6261 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T08:57:30.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.706+0000 7fede6b8b740 20 client.6261 populate_metadata read hostname 'smithi043' 2024-04-25T08:57:30.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.706+0000 7fede6b8b740 1 --2- 172.21.15.43:0/186076398 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x558c86734580 0x558c86766840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.706+0000 7fede6b8b740 4 client.6261 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_1"}] 2024-04-25T08:57:30.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.706+0000 7fede6b8b740 1 -- 172.21.15.43:0/186076398 --> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_1"}) v1 -- 0x558c86735ec0 con 0x558c86734580 2024-04-25T08:57:30.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.707+0000 7fede1c2d640 1 --2- 172.21.15.43:0/186076398 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x558c86734580 0x558c86766840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.707+0000 7fedbcff9640 10 client.6261 ms_handle_connect on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:30.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.707+0000 7fede1c2d640 1 --2- 172.21.15.43:0/186076398 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x558c86734580 0x558c86766840 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fa72b9f31a6a352 server_cookie=d26ad5057de6cc6a in_seq=0 out_seq=0 2024-04-25T08:57:30.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.707+0000 7fedbcff9640 1 -- 172.21.15.43:0/186076398 <== mds.0 v2:172.21.15.193:6836/4164634380 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x558c86735ec0 con 0x558c86734580 2024-04-25T08:57:30.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.707+0000 7fedbcff9640 10 client.6261 handle_command_reply: tid=0 2024-04-25T08:57:30.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.707+0000 7fede6b8b740 1 client.6261 shutdown 2024-04-25T08:57:30.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.707+0000 7fede6b8b740 1 -- 172.21.15.43:0/186076398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558c86735360 msgr2=0x558c86712850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:30.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.707+0000 7fede6b8b740 1 --2- 172.21.15.43:0/186076398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558c86735360 0x558c86712850 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fedcc0039e0 tx=0x7fedd4012000 comp rx=0 tx=0).stop 2024-04-25T08:57:30.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.708+0000 7fede6b8b740 1 -- 172.21.15.43:0/186076398 shutdown_connections 2024-04-25T08:57:30.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.708+0000 7fede6b8b740 1 --2- 172.21.15.43:0/186076398 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x558c86734580 0x558c86766840 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.708+0000 7fede6b8b740 1 --2- 172.21.15.43:0/186076398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558c8673d140 0x558c86766300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.708+0000 7fede6b8b740 1 --2- 172.21.15.43:0/186076398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558c86735360 0x558c86712850 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.708+0000 7fede6b8b740 1 --2- 172.21.15.43:0/186076398 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558c86714360 0x558c86712310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.708+0000 7fede6b8b740 1 -- 172.21.15.43:0/186076398 >> 172.21.15.43:0/186076398 conn(0x558c866d9810 msgr2=0x558c8677a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:30.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.708+0000 7fedbcff9640 0 client.6261 ms_handle_reset on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:30.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.708+0000 7fede6b8b740 1 -- 172.21.15.43:0/186076398 shutdown_connections 2024-04-25T08:57:30.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.708+0000 7fede6b8b740 1 -- 172.21.15.43:0/186076398 wait complete. 2024-04-25T08:57:30.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.708+0000 7fede6b8b740 20 client.6261 trim_cache size 0 max 16384 2024-04-25T08:57:30.703 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T08:57:30.703 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-25T08:57:30.703 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "67aa3ec7-a1b6-4d7f-ae17-b32c916823aa", 2024-04-25T08:57:30.703 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-25T08:57:30.703 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T08:57:30.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.708+0000 7fede3e91640 1 -- 172.21.15.43:0/4216750783 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fedc006b4d0 msgr2=0x7fedc006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:30.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.708+0000 7fede3e91640 1 --2- 172.21.15.43:0/4216750783 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fedc006b4d0 0x7fedc006d990 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fedd40012e0 tx=0x7fedd4005b90 comp rx=0 tx=0).stop 2024-04-25T08:57:30.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.708+0000 7fede3e91640 1 -- 172.21.15.43:0/4216750783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feddc15e090 msgr2=0x7feddc161360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:30.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.708+0000 7fede3e91640 1 --2- 172.21.15.43:0/4216750783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feddc15e090 0x7feddc161360 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fedd80096f0 tx=0x7fedd80046c0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.708+0000 7fede3e91640 1 -- 172.21.15.43:0/4216750783 shutdown_connections 2024-04-25T08:57:30.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.708+0000 7fede3e91640 1 --2- 172.21.15.43:0/4216750783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feddc15e090 0x7feddc161360 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.708+0000 7fede3e91640 1 --2- 172.21.15.43:0/4216750783 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fedc006b4d0 0x7fedc006d990 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.708+0000 7fede3e91640 1 --2- 172.21.15.43:0/4216750783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feddc0d2230 0x7feddc160e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.708+0000 7fede3e91640 1 --2- 172.21.15.43:0/4216750783 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feddc0b5560 0x7feddc0c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.708+0000 7fede3e91640 1 -- 172.21.15.43:0/4216750783 >> 172.21.15.43:0/4216750783 conn(0x7feddc0d0560 msgr2=0x7feddc0cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:30.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.708+0000 7fede3e91640 1 -- 172.21.15.43:0/4216750783 shutdown_connections 2024-04-25T08:57:30.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.708+0000 7fede3e91640 1 -- 172.21.15.43:0/4216750783 wait complete. 2024-04-25T08:57:30.705 INFO:tasks.ceph.mds.d.smithi193.stderr:2024-04-25T08:57:30.707+0000 7f52a22c0640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000a4e [2,head] /scrub_test_path/ceph-qa-suite/new_file_1 auth v787 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5856=0-4194304@1} caps={5856=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@3},l=5856 | request=0 lock=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x556abacbd700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000a4e:[<0x10000000001/new_file_1 v787>,<0x10000000000/ceph-qa-suite v84>,<0x1/scrub_test_path v40>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":-61} 2024-04-25T08:57:30.708 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_1' returned '{'return_code': 0, 'scrub_tag': '67aa3ec7-a1b6-4d7f-ae17-b32c916823aa', 'mode': 'asynchronous'}' 2024-04-25T08:57:30.708 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T08:57:30.708 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T08:57:30.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.821+0000 7fe80a5b0640 1 Processor -- start 2024-04-25T08:57:30.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.821+0000 7fe80a5b0640 1 -- start start 2024-04-25T08:57:30.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.821+0000 7fe80a5b0640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe80415e090 0x7fe804161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.821+0000 7fe80a5b0640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe804161fe0 0x7fe8040b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.821+0000 7fe80a5b0640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe804162610 con 0x7fe8040b5aa0 2024-04-25T08:57:30.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.821+0000 7fe80a5b0640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8040b5f00 con 0x7fe804161fe0 2024-04-25T08:57:30.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.821+0000 7fe80a5b0640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8040d1da0 con 0x7fe80415e090 2024-04-25T08:57:30.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.822+0000 7fe8037fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe804161fe0 0x7fe8040b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.822+0000 7fe808b4d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe8040b5aa0 0x7fe8040ce560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:52718/0 (socket says 172.21.15.43:52718) 2024-04-25T08:57:30.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.822+0000 7fe808b4d640 1 -- 172.21.15.43:0/162994460 learned_addr learned my addr 172.21.15.43:0/162994460 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:30.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.822+0000 7fe803fff640 1 --2- 172.21.15.43:0/162994460 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe80415e090 0x7fe804161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.822+0000 7fe8037fe640 1 -- 172.21.15.43:0/162994460 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe80415e090 msgr2=0x7fe804161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:30.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.822+0000 7fe8037fe640 1 --2- 172.21.15.43:0/162994460 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe80415e090 0x7fe804161aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.822+0000 7fe8037fe640 1 -- 172.21.15.43:0/162994460 >> v1:172.21.15.43:6789/0 conn(0x7fe8040b5aa0 legacy=0x7fe8040ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:30.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.822+0000 7fe8037fe640 1 -- 172.21.15.43:0/162994460 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8040d1750 con 0x7fe804161fe0 2024-04-25T08:57:30.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.822+0000 7fe802ffd640 1 -- 172.21.15.43:0/162994460 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1293899939 0 0) 0x7fe804162610 con 0x7fe8040b5aa0 2024-04-25T08:57:30.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.822+0000 7fe803fff640 1 --2- 172.21.15.43:0/162994460 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe80415e090 0x7fe804161aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:30.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.823+0000 7fe8037fe640 1 --2- 172.21.15.43:0/162994460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe804161fe0 0x7fe8040b5560 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fe7f40097a0 tx=0x7fe7f402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f85adbd848cd4e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:30.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.823+0000 7fe802ffd640 1 -- 172.21.15.43:0/162994460 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4037070 con 0x7fe804161fe0 2024-04-25T08:57:30.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.823+0000 7fe802ffd640 1 -- 172.21.15.43:0/162994460 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f4032440 con 0x7fe804161fe0 2024-04-25T08:57:30.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.823+0000 7fe802ffd640 1 -- 172.21.15.43:0/162994460 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4032810 con 0x7fe804161fe0 2024-04-25T08:57:30.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.823+0000 7fe80a5b0640 1 -- 172.21.15.43:0/162994460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe804161fe0 msgr2=0x7fe8040b5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:30.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.823+0000 7fe80a5b0640 1 --2- 172.21.15.43:0/162994460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe804161fe0 0x7fe8040b5560 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fe7f40097a0 tx=0x7fe7f402a210 comp rx=0 tx=0).stop 2024-04-25T08:57:30.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.823+0000 7fe80a5b0640 1 -- 172.21.15.43:0/162994460 shutdown_connections 2024-04-25T08:57:30.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.823+0000 7fe80a5b0640 1 --2- 172.21.15.43:0/162994460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe804161fe0 0x7fe8040b5560 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.823+0000 7fe80a5b0640 1 --2- 172.21.15.43:0/162994460 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe80415e090 0x7fe804161aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.823+0000 7fe80a5b0640 1 -- 172.21.15.43:0/162994460 >> 172.21.15.43:0/162994460 conn(0x7fe8040d0560 msgr2=0x7fe8040d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:30.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.823+0000 7fe80a5b0640 1 -- 172.21.15.43:0/162994460 shutdown_connections 2024-04-25T08:57:30.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.823+0000 7fe80a5b0640 1 -- 172.21.15.43:0/162994460 wait complete. 2024-04-25T08:57:30.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.824+0000 7fe80a5b0640 1 Processor -- start 2024-04-25T08:57:30.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.825+0000 7fe80a5b0640 1 -- start start 2024-04-25T08:57:30.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.825+0000 7fe80a5b0640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe8040b5aa0 0x7fe804156b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.825+0000 7fe80a5b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe80415e090 0x7fe8041584c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.825+0000 7fe80a5b0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe804161fe0 0x7fe80415fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.825+0000 7fe80a5b0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8040b3280 con 0x7fe80415e090 2024-04-25T08:57:30.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.825+0000 7fe80a5b0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8041628f0 con 0x7fe804161fe0 2024-04-25T08:57:30.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.825+0000 7fe80a5b0640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8040d3240 con 0x7fe8040b5aa0 2024-04-25T08:57:30.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.826+0000 7fe8037fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe80415e090 0x7fe8041584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.826+0000 7fe8037fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe80415e090 0x7fe8041584c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58002/0 (socket says 172.21.15.43:58002) 2024-04-25T08:57:30.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.826+0000 7fe8037fe640 1 -- 172.21.15.43:0/4146089832 learned_addr learned my addr 172.21.15.43:0/4146089832 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:30.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.826+0000 7fe803fff640 1 --2- 172.21.15.43:0/4146089832 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe8040b5aa0 0x7fe804156b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.826+0000 7fe808b4d640 1 --2- 172.21.15.43:0/4146089832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe804161fe0 0x7fe80415fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.826+0000 7fe8037fe640 1 -- 172.21.15.43:0/4146089832 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe8040b5aa0 msgr2=0x7fe804156b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:30.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.826+0000 7fe8037fe640 1 --2- 172.21.15.43:0/4146089832 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe8040b5aa0 0x7fe804156b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.826+0000 7fe8037fe640 1 -- 172.21.15.43:0/4146089832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe804161fe0 msgr2=0x7fe80415fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:30.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.826+0000 7fe8037fe640 1 --2- 172.21.15.43:0/4146089832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe804161fe0 0x7fe80415fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.826+0000 7fe8037fe640 1 -- 172.21.15.43:0/4146089832 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe804158d30 con 0x7fe80415e090 2024-04-25T08:57:30.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.826+0000 7fe8037fe640 1 --2- 172.21.15.43:0/4146089832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe80415e090 0x7fe8041584c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fe7f40097a0 tx=0x7fe7f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:30.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.826+0000 7fe800ff9640 1 -- 172.21.15.43:0/4146089832 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4037050 con 0x7fe80415e090 2024-04-25T08:57:30.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.826+0000 7fe800ff9640 1 -- 172.21.15.43:0/4146089832 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f4038040 con 0x7fe80415e090 2024-04-25T08:57:30.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.826+0000 7fe80a5b0640 1 -- 172.21.15.43:0/4146089832 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe804159010 con 0x7fe80415e090 2024-04-25T08:57:30.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.826+0000 7fe800ff9640 1 -- 172.21.15.43:0/4146089832 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4004490 con 0x7fe80415e090 2024-04-25T08:57:30.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.826+0000 7fe80a5b0640 1 -- 172.21.15.43:0/4146089832 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8041595d0 con 0x7fe80415e090 2024-04-25T08:57:30.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.828+0000 7fe800ff9640 1 -- 172.21.15.43:0/4146089832 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe7f4030980 con 0x7fe80415e090 2024-04-25T08:57:30.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.828+0000 7fe800ff9640 1 --2- 172.21.15.43:0/4146089832 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe7e406b4d0 0x7fe7e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.828+0000 7fe800ff9640 1 -- 172.21.15.43:0/4146089832 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe7f40aabf0 con 0x7fe80415e090 2024-04-25T08:57:30.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.829+0000 7fe803fff640 1 --2- 172.21.15.43:0/4146089832 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe7e406b4d0 0x7fe7e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.829+0000 7fe803fff640 1 --2- 172.21.15.43:0/4146089832 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe7e406b4d0 0x7fe7e406d990 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec00ab60 tx=0x7fe7ec008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:30.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.829+0000 7fe80d38f740 1 Processor -- start 2024-04-25T08:57:30.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.829+0000 7fe80d38f740 1 -- start start 2024-04-25T08:57:30.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.830+0000 7fe80d38f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600ef93ab10 0x5600ef93d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.830+0000 7fe80d38f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ef93aef0 0x5600ef93d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.830+0000 7fe80d38f740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5600ef90d770 0x5600ef910f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.830+0000 7fe80d38f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600ef876da0 con 0x5600ef93aef0 2024-04-25T08:57:30.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.830+0000 7fe80d38f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600ef81fb80 con 0x5600ef93ab10 2024-04-25T08:57:30.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.830+0000 7fe80d38f740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x5600ef872e40 con 0x5600ef90d770 2024-04-25T08:57:30.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.830+0000 7fe8037fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ef93aef0 0x5600ef93d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.830+0000 7fe8037fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ef93aef0 0x5600ef93d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58010/0 (socket says 172.21.15.43:58010) 2024-04-25T08:57:30.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.830+0000 7fe8037fe640 1 -- 172.21.15.43:0/1306650551 learned_addr learned my addr 172.21.15.43:0/1306650551 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:30.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.830+0000 7fe808b4d640 1 --2- 172.21.15.43:0/1306650551 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5600ef90d770 0x5600ef910f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.830+0000 7fe803fff640 1 --2- 172.21.15.43:0/1306650551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600ef93ab10 0x5600ef93d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.830+0000 7fe8037fe640 1 -- 172.21.15.43:0/1306650551 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5600ef90d770 msgr2=0x5600ef910f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:30.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.830+0000 7fe8037fe640 1 --2- 172.21.15.43:0/1306650551 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5600ef90d770 0x5600ef910f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.830+0000 7fe8037fe640 1 -- 172.21.15.43:0/1306650551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600ef93ab10 msgr2=0x5600ef93d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:30.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.830+0000 7fe8037fe640 1 --2- 172.21.15.43:0/1306650551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600ef93ab10 0x5600ef93d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.830+0000 7fe8037fe640 1 -- 172.21.15.43:0/1306650551 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600ef829f10 con 0x5600ef93aef0 2024-04-25T08:57:30.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.831+0000 7fe8037fe640 1 --2- 172.21.15.43:0/1306650551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ef93aef0 0x5600ef93d860 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4035040 tx=0x7fe7f4079430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e022ca589afb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:30.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.831+0000 7fe7f9ffb640 1 -- 172.21.15.43:0/1306650551 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f40a7070 con 0x5600ef93aef0 2024-04-25T08:57:30.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.831+0000 7fe7f9ffb640 1 -- 172.21.15.43:0/1306650551 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f4032e40 con 0x5600ef93aef0 2024-04-25T08:57:30.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.831+0000 7fe7f9ffb640 1 -- 172.21.15.43:0/1306650551 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4079530 con 0x5600ef93aef0 2024-04-25T08:57:30.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.831+0000 7fe80d38f740 1 -- 172.21.15.43:0/1306650551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ef93aef0 msgr2=0x5600ef93d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:30.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.831+0000 7fe80d38f740 1 --2- 172.21.15.43:0/1306650551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ef93aef0 0x5600ef93d860 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4035040 tx=0x7fe7f4079430 comp rx=0 tx=0).stop 2024-04-25T08:57:30.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.831+0000 7fe80d38f740 1 -- 172.21.15.43:0/1306650551 shutdown_connections 2024-04-25T08:57:30.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.831+0000 7fe80d38f740 1 --2- 172.21.15.43:0/1306650551 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5600ef90d770 0x5600ef910f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.831+0000 7fe80d38f740 1 --2- 172.21.15.43:0/1306650551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ef93aef0 0x5600ef93d860 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.831+0000 7fe80d38f740 1 --2- 172.21.15.43:0/1306650551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600ef93ab10 0x5600ef93d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.831+0000 7fe80d38f740 1 -- 172.21.15.43:0/1306650551 >> 172.21.15.43:0/1306650551 conn(0x5600ef90f9e0 msgr2=0x5600ef958320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:30.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.831+0000 7fe80d38f740 1 -- 172.21.15.43:0/1306650551 shutdown_connections 2024-04-25T08:57:30.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.831+0000 7fe80d38f740 1 -- 172.21.15.43:0/1306650551 wait complete. 2024-04-25T08:57:30.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.832+0000 7fe80d38f740 1 -- start start 2024-04-25T08:57:30.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.832+0000 7fe80d38f740 1 Processor -- start 2024-04-25T08:57:30.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.833+0000 7fe80d38f740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5600ef93ab10 0x5600ef954890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.833+0000 7fe80d38f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ef93aef0 0x5600ef939590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.833+0000 7fe80d38f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600ef9539a0 0x5600ef931800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.833+0000 7fe80d38f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600ef912b90 con 0x5600ef93aef0 2024-04-25T08:57:30.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.833+0000 7fe80d38f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600ef8154b0 con 0x5600ef9539a0 2024-04-25T08:57:30.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.833+0000 7fe80d38f740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x5600ef912e70 con 0x5600ef93ab10 2024-04-25T08:57:30.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.833+0000 7fe80d38f740 10 client.0 _ll_register_callbacks cb 0x5600ef855180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:30.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.833+0000 7fe8037fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ef93aef0 0x5600ef939590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.833+0000 7fe8037fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ef93aef0 0x5600ef939590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58026/0 (socket says 172.21.15.43:58026) 2024-04-25T08:57:30.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.833+0000 7fe8037fe640 1 -- 172.21.15.43:0/2208943513 learned_addr learned my addr 172.21.15.43:0/2208943513 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.833+0000 7fe808b4d640 1 --2- 172.21.15.43:0/2208943513 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600ef9539a0 0x5600ef931800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.833+0000 7fe803fff640 1 --2- 172.21.15.43:0/2208943513 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5600ef93ab10 0x5600ef954890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.833+0000 7fe8037fe640 1 -- 172.21.15.43:0/2208943513 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5600ef93ab10 msgr2=0x5600ef954890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.833+0000 7fe8037fe640 1 --2- 172.21.15.43:0/2208943513 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5600ef93ab10 0x5600ef954890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.833+0000 7fe8037fe640 1 -- 172.21.15.43:0/2208943513 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600ef9539a0 msgr2=0x5600ef931800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.833+0000 7fe8037fe640 1 --2- 172.21.15.43:0/2208943513 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600ef9539a0 0x5600ef931800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.833+0000 7fe8037fe640 1 -- 172.21.15.43:0/2208943513 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5600ef6557d0 con 0x5600ef93aef0 2024-04-25T08:57:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.834+0000 7fe8037fe640 1 --2- 172.21.15.43:0/2208943513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ef93aef0 0x5600ef939590 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4091180 tx=0x7fe7f4009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7270cf1e4e9a37f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.834+0000 7fe7e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:57:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.834+0000 7fe7e2ffd640 1 -- 172.21.15.43:0/2208943513 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f40a7070 con 0x5600ef93aef0 2024-04-25T08:57:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.834+0000 7fe7e2ffd640 1 -- 172.21.15.43:0/2208943513 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe7f407d160 con 0x5600ef93aef0 2024-04-25T08:57:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.834+0000 7fe80d38f740 1 -- 172.21.15.43:0/2208943513 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5600ef8842e0 con 0x5600ef93aef0 2024-04-25T08:57:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.834+0000 7fe7e2ffd640 1 -- 172.21.15.43:0/2208943513 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7f4081b90 con 0x5600ef93aef0 2024-04-25T08:57:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.834+0000 7fe80d38f740 10 client.6276 fetch_fsmap learned FSMap version 23 2024-04-25T08:57:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.834+0000 7fe80d38f740 1 -- 172.21.15.43:0/2208943513 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x5600ef58b3f0 con 0x5600ef93aef0 2024-04-25T08:57:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.834+0000 7fe7e2ffd640 1 -- 172.21.15.43:0/2208943513 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5600ef8842e0 con 0x5600ef93aef0 2024-04-25T08:57:30.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.834+0000 7fe7e2ffd640 1 -- 172.21.15.43:0/2208943513 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe7f4094070 con 0x5600ef93aef0 2024-04-25T08:57:30.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.834+0000 7fe80d38f740 10 client.6276 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T08:57:30.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.834+0000 7fe80d38f740 10 client.6276 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T08:57:30.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.834+0000 7fe80d38f740 20 client.6276 populate_metadata read hostname 'smithi043' 2024-04-25T08:57:30.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.834+0000 7fe80d38f740 1 --2- 172.21.15.43:0/2208943513 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5600ef96bc20 0x5600ef96c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.834+0000 7fe80d38f740 4 client.6276 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:57:30.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.834+0000 7fe80d38f740 1 -- 172.21.15.43:0/2208943513 --> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5600ef87e630 con 0x5600ef96bc20 2024-04-25T08:57:30.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.834+0000 7fe803fff640 1 --2- 172.21.15.43:0/2208943513 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5600ef96bc20 0x5600ef96c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.835+0000 7fe803fff640 1 --2- 172.21.15.43:0/2208943513 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5600ef96bc20 0x5600ef96c030 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7b2adc788972261 server_cookie=bd31348ade1f206f in_seq=0 out_seq=0 2024-04-25T08:57:30.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.835+0000 7fe7e2ffd640 10 client.6276 ms_handle_connect on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:30.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.835+0000 7fe7e2ffd640 1 -- 172.21.15.43:0/2208943513 <== mds.0 v2:172.21.15.193:6836/4164634380 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5600ef87e630 con 0x5600ef96bc20 2024-04-25T08:57:30.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.835+0000 7fe7e2ffd640 10 client.6276 handle_command_reply: tid=0 2024-04-25T08:57:30.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.835+0000 7fe80d38f740 1 client.6276 shutdown 2024-04-25T08:57:30.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.836+0000 7fe80d38f740 1 -- 172.21.15.43:0/2208943513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ef93aef0 msgr2=0x5600ef939590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:30.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.836+0000 7fe80d38f740 1 --2- 172.21.15.43:0/2208943513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ef93aef0 0x5600ef939590 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4091180 tx=0x7fe7f4009190 comp rx=0 tx=0).stop 2024-04-25T08:57:30.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.836+0000 7fe80d38f740 1 -- 172.21.15.43:0/2208943513 shutdown_connections 2024-04-25T08:57:30.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.836+0000 7fe80d38f740 1 --2- 172.21.15.43:0/2208943513 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5600ef96bc20 0x5600ef96c030 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.836+0000 7fe80d38f740 1 --2- 172.21.15.43:0/2208943513 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600ef9539a0 0x5600ef931800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.836+0000 7fe7e2ffd640 0 client.6276 ms_handle_reset on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:30.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.836+0000 7fe80d38f740 1 --2- 172.21.15.43:0/2208943513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ef93aef0 0x5600ef939590 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.836+0000 7fe80d38f740 1 --2- 172.21.15.43:0/2208943513 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5600ef93ab10 0x5600ef954890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.836+0000 7fe80d38f740 1 -- 172.21.15.43:0/2208943513 >> 172.21.15.43:0/2208943513 conn(0x5600ef90d770 msgr2=0x5600ef90e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:30.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.836+0000 7fe80d38f740 1 -- 172.21.15.43:0/2208943513 shutdown_connections 2024-04-25T08:57:30.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.836+0000 7fe80d38f740 1 -- 172.21.15.43:0/2208943513 wait complete. 2024-04-25T08:57:30.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.836+0000 7fe80d38f740 20 client.6276 trim_cache size 0 max 16384 2024-04-25T08:57:30.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.851+0000 7fe80d38f740 1 Processor -- start 2024-04-25T08:57:30.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.851+0000 7fe80d38f740 1 -- start start 2024-04-25T08:57:30.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.852+0000 7fe80d38f740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5600ef93ab10 0x5600ef940da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.852+0000 7fe80d38f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600ef93aef0 0x5600ef9412e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.852+0000 7fe80d38f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ef9539a0 0x5600ef90c600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.852+0000 7fe80d38f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8041ffa10 con 0x5600ef9539a0 2024-04-25T08:57:30.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.852+0000 7fe80d38f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8041f36d0 con 0x5600ef93aef0 2024-04-25T08:57:30.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.852+0000 7fe80d38f740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8041446d0 con 0x5600ef93ab10 2024-04-25T08:57:30.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.852+0000 7fe808b4d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ef9539a0 0x5600ef90c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.852+0000 7fe808b4d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ef9539a0 0x5600ef90c600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58030/0 (socket says 172.21.15.43:58030) 2024-04-25T08:57:30.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.852+0000 7fe808b4d640 1 -- 172.21.15.43:0/585230230 learned_addr learned my addr 172.21.15.43:0/585230230 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:30.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.852+0000 7fe8037fe640 1 --2- 172.21.15.43:0/585230230 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600ef93aef0 0x5600ef9412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.852+0000 7fe803fff640 1 --2- 172.21.15.43:0/585230230 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5600ef93ab10 0x5600ef940da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.852+0000 7fe808b4d640 1 -- 172.21.15.43:0/585230230 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5600ef93ab10 msgr2=0x5600ef940da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:30.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.852+0000 7fe808b4d640 1 --2- 172.21.15.43:0/585230230 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5600ef93ab10 0x5600ef940da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.852+0000 7fe808b4d640 1 -- 172.21.15.43:0/585230230 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600ef93aef0 msgr2=0x5600ef9412e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:30.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.852+0000 7fe808b4d640 1 --2- 172.21.15.43:0/585230230 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600ef93aef0 0x5600ef9412e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.852+0000 7fe808b4d640 1 -- 172.21.15.43:0/585230230 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600ef54b0f0 con 0x5600ef9539a0 2024-04-25T08:57:30.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.852+0000 7fe8037fe640 1 --2- 172.21.15.43:0/585230230 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600ef93aef0 0x5600ef9412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:30.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.852+0000 7fe808b4d640 1 --2- 172.21.15.43:0/585230230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ef9539a0 0x5600ef90c600 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc001cc0 tx=0x7fe7fc004c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c90d3472dfab60f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:30.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.853+0000 7fe7f97fa640 1 -- 172.21.15.43:0/585230230 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc007990 con 0x5600ef9539a0 2024-04-25T08:57:30.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.853+0000 7fe7f97fa640 1 -- 172.21.15.43:0/585230230 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7fc016420 con 0x5600ef9539a0 2024-04-25T08:57:30.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.853+0000 7fe7f97fa640 1 -- 172.21.15.43:0/585230230 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc0167d0 con 0x5600ef9539a0 2024-04-25T08:57:30.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.853+0000 7fe80d38f740 1 -- 172.21.15.43:0/585230230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ef9539a0 msgr2=0x5600ef90c600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:30.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.853+0000 7fe80d38f740 1 --2- 172.21.15.43:0/585230230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ef9539a0 0x5600ef90c600 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc001cc0 tx=0x7fe7fc004c50 comp rx=0 tx=0).stop 2024-04-25T08:57:30.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.853+0000 7fe80d38f740 1 -- 172.21.15.43:0/585230230 shutdown_connections 2024-04-25T08:57:30.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.853+0000 7fe80d38f740 1 --2- 172.21.15.43:0/585230230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ef9539a0 0x5600ef90c600 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.853+0000 7fe80d38f740 1 --2- 172.21.15.43:0/585230230 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600ef93aef0 0x5600ef9412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.853+0000 7fe80d38f740 1 --2- 172.21.15.43:0/585230230 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5600ef93ab10 0x5600ef940da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.853+0000 7fe80d38f740 1 -- 172.21.15.43:0/585230230 >> 172.21.15.43:0/585230230 conn(0x5600ef90d770 msgr2=0x5600ef97ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:30.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.853+0000 7fe80d38f740 1 -- 172.21.15.43:0/585230230 shutdown_connections 2024-04-25T08:57:30.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.853+0000 7fe80d38f740 1 -- 172.21.15.43:0/585230230 wait complete. 2024-04-25T08:57:30.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.854+0000 7fe80d38f740 1 -- start start 2024-04-25T08:57:30.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.854+0000 7fe80d38f740 1 Processor -- start 2024-04-25T08:57:30.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.855+0000 7fe80d38f740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5600ef93ab10 0x5600ef940cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.855+0000 7fe80d38f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600ef93aef0 0x5600ef941200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.855+0000 7fe80d38f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ef9539a0 0x5600ef94dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.855+0000 7fe80d38f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600ef913480 con 0x5600ef9539a0 2024-04-25T08:57:30.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.855+0000 7fe80d38f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8041f30d0 con 0x5600ef93aef0 2024-04-25T08:57:30.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.855+0000 7fe80d38f740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x5600ef96c150 con 0x5600ef93ab10 2024-04-25T08:57:30.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.855+0000 7fe80d38f740 10 client.0 _ll_register_callbacks cb 0x7fe7f4009400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:30.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.855+0000 7fe808b4d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ef9539a0 0x5600ef94dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.855+0000 7fe808b4d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ef9539a0 0x5600ef94dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58032/0 (socket says 172.21.15.43:58032) 2024-04-25T08:57:30.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.855+0000 7fe808b4d640 1 -- 172.21.15.43:0/2546410644 learned_addr learned my addr 172.21.15.43:0/2546410644 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:30.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.855+0000 7fe8037fe640 1 --2- 172.21.15.43:0/2546410644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600ef93aef0 0x5600ef941200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.855+0000 7fe803fff640 1 --2- 172.21.15.43:0/2546410644 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5600ef93ab10 0x5600ef940cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.855+0000 7fe808b4d640 1 -- 172.21.15.43:0/2546410644 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5600ef93ab10 msgr2=0x5600ef940cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:30.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.855+0000 7fe808b4d640 1 --2- 172.21.15.43:0/2546410644 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5600ef93ab10 0x5600ef940cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.855+0000 7fe808b4d640 1 -- 172.21.15.43:0/2546410644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600ef93aef0 msgr2=0x5600ef941200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:30.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.855+0000 7fe808b4d640 1 --2- 172.21.15.43:0/2546410644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600ef93aef0 0x5600ef941200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.855+0000 7fe808b4d640 1 -- 172.21.15.43:0/2546410644 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5600ef9c3940 con 0x5600ef9539a0 2024-04-25T08:57:30.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.855+0000 7fe808b4d640 1 --2- 172.21.15.43:0/2546410644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ef9539a0 0x5600ef94dcb0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc006010 tx=0x7fe7fc007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce968e874277ac6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:30.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.855+0000 7fe7e2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:57:30.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.855+0000 7fe7e2ffd640 1 -- 172.21.15.43:0/2546410644 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc004b60 con 0x5600ef9539a0 2024-04-25T08:57:30.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.856+0000 7fe7e2ffd640 1 -- 172.21.15.43:0/2546410644 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe7fc007990 con 0x5600ef9539a0 2024-04-25T08:57:30.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.856+0000 7fe80d38f740 1 -- 172.21.15.43:0/2546410644 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe7fc016420 con 0x5600ef9539a0 2024-04-25T08:57:30.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.856+0000 7fe7e2ffd640 1 -- 172.21.15.43:0/2546410644 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7fc032050 con 0x5600ef9539a0 2024-04-25T08:57:30.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.856+0000 7fe80d38f740 10 client.6282 fetch_fsmap learned FSMap version 23 2024-04-25T08:57:30.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.856+0000 7fe80d38f740 1 -- 172.21.15.43:0/2546410644 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x5600ef9c3140 con 0x5600ef9539a0 2024-04-25T08:57:30.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.856+0000 7fe7e2ffd640 1 -- 172.21.15.43:0/2546410644 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7fc016420 con 0x5600ef9539a0 2024-04-25T08:57:30.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.856+0000 7fe7e2ffd640 1 -- 172.21.15.43:0/2546410644 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 23) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe7fc035020 con 0x5600ef9539a0 2024-04-25T08:57:30.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.856+0000 7fe80d38f740 10 client.6282 fetch_fsmap finished waiting for FSMap version 23 2024-04-25T08:57:30.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.856+0000 7fe80d38f740 10 client.6282 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T08:57:30.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.856+0000 7fe80d38f740 20 client.6282 populate_metadata read hostname 'smithi043' 2024-04-25T08:57:30.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.856+0000 7fe80d38f740 1 --2- 172.21.15.43:0/2546410644 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5600ef93d2e0 0x5600ef94e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:30.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.856+0000 7fe80d38f740 4 client.6282 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:57:30.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.856+0000 7fe80d38f740 1 -- 172.21.15.43:0/2546410644 --> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5600ef7fc690 con 0x5600ef93d2e0 2024-04-25T08:57:30.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.856+0000 7fe803fff640 1 --2- 172.21.15.43:0/2546410644 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5600ef93d2e0 0x5600ef94e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:30.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.857+0000 7fe803fff640 1 --2- 172.21.15.43:0/2546410644 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5600ef93d2e0 0x5600ef94e1f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89ecec299b57d984 server_cookie=3ac8830119006288 in_seq=0 out_seq=0 2024-04-25T08:57:30.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.857+0000 7fe7e2ffd640 10 client.6282 ms_handle_connect on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:30.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.857+0000 7fe7e2ffd640 1 -- 172.21.15.43:0/2546410644 <== mds.0 v2:172.21.15.193:6836/4164634380 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+903 (crc 0 0 0) 0x5600ef7fc690 con 0x5600ef93d2e0 2024-04-25T08:57:30.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.857+0000 7fe7e2ffd640 10 client.6282 handle_command_reply: tid=0 2024-04-25T08:57:30.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.857+0000 7fe80d38f740 1 client.6282 shutdown 2024-04-25T08:57:30.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.857+0000 7fe80d38f740 1 -- 172.21.15.43:0/2546410644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ef9539a0 msgr2=0x5600ef94dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:30.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.857+0000 7fe80d38f740 1 --2- 172.21.15.43:0/2546410644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ef9539a0 0x5600ef94dcb0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc006010 tx=0x7fe7fc007f90 comp rx=0 tx=0).stop 2024-04-25T08:57:30.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.857+0000 7fe80d38f740 1 -- 172.21.15.43:0/2546410644 shutdown_connections 2024-04-25T08:57:30.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.857+0000 7fe80d38f740 1 --2- 172.21.15.43:0/2546410644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5600ef9539a0 0x5600ef94dcb0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.857+0000 7fe80d38f740 1 --2- 172.21.15.43:0/2546410644 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5600ef93d2e0 0x5600ef94e1f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.857+0000 7fe80d38f740 1 --2- 172.21.15.43:0/2546410644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5600ef93aef0 0x5600ef941200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.857+0000 7fe80d38f740 1 --2- 172.21.15.43:0/2546410644 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5600ef93ab10 0x5600ef940cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.857+0000 7fe7e2ffd640 0 client.6282 ms_handle_reset on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:30.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.857+0000 7fe80d38f740 1 -- 172.21.15.43:0/2546410644 >> 172.21.15.43:0/2546410644 conn(0x5600ef90d770 msgr2=0x5600ef97ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:30.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.857+0000 7fe80d38f740 1 -- 172.21.15.43:0/2546410644 shutdown_connections 2024-04-25T08:57:30.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.857+0000 7fe80d38f740 1 -- 172.21.15.43:0/2546410644 wait complete. 2024-04-25T08:57:30.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.857+0000 7fe80d38f740 20 client.6282 trim_cache size 0 max 16384 2024-04-25T08:57:30.853 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T08:57:30.853 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T08:57:30.853 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-25T08:57:30.853 INFO:teuthology.orchestra.run.smithi043.stdout: "4e9f1257-348e-464d-8ed3-b6c639b655ec": { 2024-04-25T08:57:30.853 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/scrub_test_path/ceph-qa-suite/suites", 2024-04-25T08:57:30.853 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "4e9f1257-348e-464d-8ed3-b6c639b655ec", 2024-04-25T08:57:30.853 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "" 2024-04-25T08:57:30.853 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:57:30.854 INFO:teuthology.orchestra.run.smithi043.stdout: "67aa3ec7-a1b6-4d7f-ae17-b32c916823aa": { 2024-04-25T08:57:30.854 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_1", 2024-04-25T08:57:30.854 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "67aa3ec7-a1b6-4d7f-ae17-b32c916823aa", 2024-04-25T08:57:30.854 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "" 2024-04-25T08:57:30.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:57:30.854 INFO:teuthology.orchestra.run.smithi043.stdout: "a5c661af-ce28-48d1-a6c6-c0ae561df41d": { 2024-04-25T08:57:30.854 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml", 2024-04-25T08:57:30.854 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "a5c661af-ce28-48d1-a6c6-c0ae561df41d", 2024-04-25T08:57:30.854 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "" 2024-04-25T08:57:30.854 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:57:30.854 INFO:teuthology.orchestra.run.smithi043.stdout: "d5d9ac94-69a1-4dee-8264-b2cdf26267f9": { 2024-04-25T08:57:30.854 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/", 2024-04-25T08:57:30.854 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "d5d9ac94-69a1-4dee-8264-b2cdf26267f9", 2024-04-25T08:57:30.854 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "" 2024-04-25T08:57:30.855 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:57:30.855 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:57:30.855 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T08:57:30.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.858+0000 7fe80a5b0640 1 -- 172.21.15.43:0/4146089832 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe7e406b4d0 msgr2=0x7fe7e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:30.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.858+0000 7fe80a5b0640 1 --2- 172.21.15.43:0/4146089832 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe7e406b4d0 0x7fe7e406d990 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe7ec00ab60 tx=0x7fe7ec008040 comp rx=0 tx=0).stop 2024-04-25T08:57:30.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.858+0000 7fe80a5b0640 1 -- 172.21.15.43:0/4146089832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe80415e090 msgr2=0x7fe8041584c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:30.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.858+0000 7fe80a5b0640 1 --2- 172.21.15.43:0/4146089832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe80415e090 0x7fe8041584c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fe7f40097a0 tx=0x7fe7f40046c0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.858+0000 7fe80a5b0640 1 -- 172.21.15.43:0/4146089832 shutdown_connections 2024-04-25T08:57:30.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.858+0000 7fe80a5b0640 1 --2- 172.21.15.43:0/4146089832 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe7e406b4d0 0x7fe7e406d990 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.858+0000 7fe80a5b0640 1 --2- 172.21.15.43:0/4146089832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe804161fe0 0x7fe80415fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.858+0000 7fe80a5b0640 1 --2- 172.21.15.43:0/4146089832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe80415e090 0x7fe8041584c0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.858+0000 7fe80a5b0640 1 --2- 172.21.15.43:0/4146089832 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe8040b5aa0 0x7fe804156b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:30.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.858+0000 7fe80a5b0640 1 -- 172.21.15.43:0/4146089832 >> 172.21.15.43:0/4146089832 conn(0x7fe8040d0560 msgr2=0x7fe80415bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:30.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.858+0000 7fe80a5b0640 1 -- 172.21.15.43:0/4146089832 shutdown_connections 2024-04-25T08:57:30.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:30.858+0000 7fe80a5b0640 1 -- 172.21.15.43:0/4146089832 wait complete. 2024-04-25T08:57:34.609 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-04-25T08:57:34.619+0000 7fe4fd99d640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T08:57:35.859 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T08:57:35.860 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T08:57:35.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.986+0000 7fbd25522640 1 Processor -- start 2024-04-25T08:57:35.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.986+0000 7fbd25522640 1 -- start start 2024-04-25T08:57:35.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.987+0000 7fbd25522640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd200b5560 0x7fbd200b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:35.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.987+0000 7fbd25522640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbd200d2230 0x7fbd200d2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:35.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.987+0000 7fbd25522640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd20162610 con 0x7fbd2015e090 2024-04-25T08:57:35.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.987+0000 7fbd25522640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd200d1da0 con 0x7fbd200d2230 2024-04-25T08:57:35.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.987+0000 7fbd25522640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd200d1750 con 0x7fbd200b5560 2024-04-25T08:57:35.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.987+0000 7fbd1f7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbd200d2230 0x7fbd200d2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:35.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.987+0000 7fbd1f7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbd200d2230 0x7fbd200d2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58044/0 (socket says 172.21.15.43:58044) 2024-04-25T08:57:35.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.987+0000 7fbd1f7fe640 1 -- 172.21.15.43:0/2091345143 learned_addr learned my addr 172.21.15.43:0/2091345143 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:35.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.987+0000 7fbd1e7fc640 1 --2- 172.21.15.43:0/2091345143 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd200b5560 0x7fbd200b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:35.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.988+0000 7fbd1f7fe640 1 -- 172.21.15.43:0/2091345143 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd200b5560 msgr2=0x7fbd200b59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:35.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.988+0000 7fbd1f7fe640 1 --2- 172.21.15.43:0/2091345143 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd200b5560 0x7fbd200b59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:35.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.988+0000 7fbd1f7fe640 1 -- 172.21.15.43:0/2091345143 >> v1:172.21.15.193:6789/0 conn(0x7fbd2015e090 legacy=0x7fbd20161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:35.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.988+0000 7fbd1f7fe640 1 -- 172.21.15.43:0/2091345143 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd200d1a30 con 0x7fbd200d2230 2024-04-25T08:57:35.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.988+0000 7fbd1e7fc640 1 --2- 172.21.15.43:0/2091345143 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd200b5560 0x7fbd200b59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:35.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.988+0000 7fbd1f7fe640 1 --2- 172.21.15.43:0/2091345143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd200d2230 0x7fbd200d2690 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fbd18009870 tx=0x7fbd1802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37cf6ef6c7bf34f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:35.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.988+0000 7fbd1dffb640 1 -- 172.21.15.43:0/2091345143 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd18037070 con 0x7fbd200d2230 2024-04-25T08:57:35.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.989+0000 7fbd1dffb640 1 -- 172.21.15.43:0/2091345143 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd1802ae50 con 0x7fbd200d2230 2024-04-25T08:57:35.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.989+0000 7fbd1dffb640 1 -- 172.21.15.43:0/2091345143 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd18032860 con 0x7fbd200d2230 2024-04-25T08:57:35.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.989+0000 7fbd25522640 1 -- 172.21.15.43:0/2091345143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd200d2230 msgr2=0x7fbd200d2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:35.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.989+0000 7fbd25522640 1 --2- 172.21.15.43:0/2091345143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd200d2230 0x7fbd200d2690 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fbd18009870 tx=0x7fbd1802a300 comp rx=0 tx=0).stop 2024-04-25T08:57:35.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.989+0000 7fbd25522640 1 -- 172.21.15.43:0/2091345143 shutdown_connections 2024-04-25T08:57:35.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.989+0000 7fbd25522640 1 --2- 172.21.15.43:0/2091345143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd200d2230 0x7fbd200d2690 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:35.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.989+0000 7fbd25522640 1 --2- 172.21.15.43:0/2091345143 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd200b5560 0x7fbd200b59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:35.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.989+0000 7fbd25522640 1 -- 172.21.15.43:0/2091345143 >> 172.21.15.43:0/2091345143 conn(0x7fbd200d0560 msgr2=0x7fbd200d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:35.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.989+0000 7fbd25522640 1 -- 172.21.15.43:0/2091345143 shutdown_connections 2024-04-25T08:57:35.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.989+0000 7fbd25522640 1 -- 172.21.15.43:0/2091345143 wait complete. 2024-04-25T08:57:35.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.990+0000 7fbd25522640 1 Processor -- start 2024-04-25T08:57:35.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.991+0000 7fbd25522640 1 -- start start 2024-04-25T08:57:35.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.991+0000 7fbd25522640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd200b5560 0x7fbd2015eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:35.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.991+0000 7fbd25522640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd200d2230 0x7fbd20157c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:35.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.991+0000 7fbd25522640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd2015e090 0x7fbd201582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:35.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.991+0000 7fbd25522640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd201621b0 con 0x7fbd200d2230 2024-04-25T08:57:35.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.991+0000 7fbd25522640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd200b3280 con 0x7fbd200b5560 2024-04-25T08:57:35.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.991+0000 7fbd25522640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd20162330 con 0x7fbd2015e090 2024-04-25T08:57:35.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.991+0000 7fbd1e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd200d2230 0x7fbd20157c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:35.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.991+0000 7fbd1e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd200d2230 0x7fbd20157c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58048/0 (socket says 172.21.15.43:58048) 2024-04-25T08:57:35.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.991+0000 7fbd1e7fc640 1 -- 172.21.15.43:0/2188411104 learned_addr learned my addr 172.21.15.43:0/2188411104 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:35.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.992+0000 7fbd1e7fc640 1 -- 172.21.15.43:0/2188411104 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd2015e090 msgr2=0x7fbd201582c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:35.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.992+0000 7fbd1f7fe640 1 --2- 172.21.15.43:0/2188411104 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd2015e090 0x7fbd201582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:35.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.992+0000 7fbd1effd640 1 --2- 172.21.15.43:0/2188411104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd200b5560 0x7fbd2015eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:35.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.992+0000 7fbd1e7fc640 1 --2- 172.21.15.43:0/2188411104 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd2015e090 0x7fbd201582c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:35.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.992+0000 7fbd1e7fc640 1 -- 172.21.15.43:0/2188411104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd200b5560 msgr2=0x7fbd2015eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:35.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.992+0000 7fbd1e7fc640 1 --2- 172.21.15.43:0/2188411104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd200b5560 0x7fbd2015eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:35.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.992+0000 7fbd1e7fc640 1 -- 172.21.15.43:0/2188411104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd20158a10 con 0x7fbd200d2230 2024-04-25T08:57:35.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.992+0000 7fbd1e7fc640 1 --2- 172.21.15.43:0/2188411104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd200d2230 0x7fbd20157c70 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fbd1400e7e0 tx=0x7fbd1400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:35.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.992+0000 7fbd077fe640 1 -- 172.21.15.43:0/2188411104 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1400f040 con 0x7fbd200d2230 2024-04-25T08:57:35.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.992+0000 7fbd077fe640 1 -- 172.21.15.43:0/2188411104 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd14004030 con 0x7fbd200d2230 2024-04-25T08:57:35.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.992+0000 7fbd25522640 1 -- 172.21.15.43:0/2188411104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd2015e510 con 0x7fbd200d2230 2024-04-25T08:57:35.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.992+0000 7fbd077fe640 1 -- 172.21.15.43:0/2188411104 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd14004400 con 0x7fbd200d2230 2024-04-25T08:57:35.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.992+0000 7fbd25522640 1 -- 172.21.15.43:0/2188411104 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd20158cf0 con 0x7fbd200d2230 2024-04-25T08:57:35.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.994+0000 7fbd077fe640 1 -- 172.21.15.43:0/2188411104 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbd1400c6d0 con 0x7fbd200d2230 2024-04-25T08:57:35.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.994+0000 7fbd077fe640 1 --2- 172.21.15.43:0/2188411104 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbd0c06b500 0x7fbd0c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:35.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.994+0000 7fbd077fe640 1 -- 172.21.15.43:0/2188411104 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd14089fb0 con 0x7fbd200d2230 2024-04-25T08:57:35.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.995+0000 7fbd1effd640 1 --2- 172.21.15.43:0/2188411104 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbd0c06b500 0x7fbd0c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:35.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.995+0000 7fbd1effd640 1 --2- 172.21.15.43:0/2188411104 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbd0c06b500 0x7fbd0c06d9c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbd10007fd0 tx=0x7fbd10009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:35.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.995+0000 7fbd28228740 1 Processor -- start 2024-04-25T08:57:35.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.996+0000 7fbd28228740 1 -- start start 2024-04-25T08:57:35.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.996+0000 7fbd28228740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5567c9031b10 0x5567c9034320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:35.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.996+0000 7fbd28228740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c9031ef0 0x5567c9034860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:35.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.996+0000 7fbd28228740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567c9004770 0x5567c9007f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:35.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.996+0000 7fbd28228740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567c8f6dda0 con 0x5567c9004770 2024-04-25T08:57:35.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.996+0000 7fbd28228740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567c8f16b80 con 0x5567c9031ef0 2024-04-25T08:57:35.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.996+0000 7fbd28228740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x5567c8f69e40 con 0x5567c9031b10 2024-04-25T08:57:35.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.996+0000 7fbd1f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567c9004770 0x5567c9007f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:35.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.996+0000 7fbd1f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567c9004770 0x5567c9007f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58050/0 (socket says 172.21.15.43:58050) 2024-04-25T08:57:35.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.997+0000 7fbd1f7fe640 1 -- 172.21.15.43:0/673212243 learned_addr learned my addr 172.21.15.43:0/673212243 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:35.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.997+0000 7fbd1effd640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5567c9031b10 0x5567c9034320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:35.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.997+0000 7fbd1e7fc640 1 --2- 172.21.15.43:0/673212243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c9031ef0 0x5567c9034860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:35.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.997+0000 7fbd1f7fe640 1 -- 172.21.15.43:0/673212243 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5567c9031b10 msgr2=0x5567c9034320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:35.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.997+0000 7fbd1f7fe640 1 --2- 172.21.15.43:0/673212243 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5567c9031b10 0x5567c9034320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:35.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.997+0000 7fbd1f7fe640 1 -- 172.21.15.43:0/673212243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c9031ef0 msgr2=0x5567c9034860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:35.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.997+0000 7fbd1f7fe640 1 --2- 172.21.15.43:0/673212243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c9031ef0 0x5567c9034860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:35.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.997+0000 7fbd1f7fe640 1 -- 172.21.15.43:0/673212243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567c8f20f10 con 0x5567c9004770 2024-04-25T08:57:35.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.997+0000 7fbd1f7fe640 1 --2- 172.21.15.43:0/673212243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567c9004770 0x5567c9007f80 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fbd1802a340 tx=0x7fbd1802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84963f1fb5bdb126 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:35.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.997+0000 7fbd04ff9640 1 -- 172.21.15.43:0/673212243 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd18037040 con 0x5567c9004770 2024-04-25T08:57:35.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.997+0000 7fbd04ff9640 1 -- 172.21.15.43:0/673212243 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd1802ae50 con 0x5567c9004770 2024-04-25T08:57:35.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.997+0000 7fbd04ff9640 1 -- 172.21.15.43:0/673212243 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1803c880 con 0x5567c9004770 2024-04-25T08:57:35.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.998+0000 7fbd28228740 1 -- 172.21.15.43:0/673212243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567c9004770 msgr2=0x5567c9007f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:35.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.998+0000 7fbd28228740 1 --2- 172.21.15.43:0/673212243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567c9004770 0x5567c9007f80 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fbd1802a340 tx=0x7fbd1802c000 comp rx=0 tx=0).stop 2024-04-25T08:57:35.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.998+0000 7fbd28228740 1 -- 172.21.15.43:0/673212243 shutdown_connections 2024-04-25T08:57:35.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.998+0000 7fbd28228740 1 --2- 172.21.15.43:0/673212243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567c9004770 0x5567c9007f80 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:35.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.998+0000 7fbd28228740 1 --2- 172.21.15.43:0/673212243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c9031ef0 0x5567c9034860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:35.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.998+0000 7fbd28228740 1 --2- 172.21.15.43:0/673212243 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5567c9031b10 0x5567c9034320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:35.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.998+0000 7fbd28228740 1 -- 172.21.15.43:0/673212243 >> 172.21.15.43:0/673212243 conn(0x5567c90069e0 msgr2=0x5567c904f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:35.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.998+0000 7fbd28228740 1 -- 172.21.15.43:0/673212243 shutdown_connections 2024-04-25T08:57:35.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.998+0000 7fbd28228740 1 -- 172.21.15.43:0/673212243 wait complete. 2024-04-25T08:57:35.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.999+0000 7fbd28228740 1 -- start start 2024-04-25T08:57:35.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.999+0000 7fbd28228740 1 Processor -- start 2024-04-25T08:57:35.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.999+0000 7fbd28228740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c9031b10 0x5567c902e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:35.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.999+0000 7fbd28228740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5567c9031ef0 0x5567c9053180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:35.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.999+0000 7fbd28228740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567c902b860 0x5567c905ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:35.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.999+0000 7fbd28228740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567c9009b90 con 0x5567c902b860 2024-04-25T08:57:35.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.999+0000 7fbd28228740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567c8f0c4b0 con 0x5567c9031b10 2024-04-25T08:57:35.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.999+0000 7fbd28228740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x5567c9009e70 con 0x5567c9031ef0 2024-04-25T08:57:35.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:35.999+0000 7fbd28228740 10 client.0 _ll_register_callbacks cb 0x5567c8f4c180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:35.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.000+0000 7fbd1f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567c902b860 0x5567c905ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:35.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.000+0000 7fbd1e7fc640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5567c9031ef0 0x5567c9053180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:35.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.000+0000 7fbd1effd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c9031b10 0x5567c902e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:35.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.000+0000 7fbd1f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567c902b860 0x5567c905ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58066/0 (socket says 172.21.15.43:58066) 2024-04-25T08:57:35.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.000+0000 7fbd1f7fe640 1 -- 172.21.15.43:0/2146687785 learned_addr learned my addr 172.21.15.43:0/2146687785 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:35.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.000+0000 7fbd1f7fe640 1 -- 172.21.15.43:0/2146687785 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5567c9031ef0 msgr2=0x5567c9053180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:35.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.000+0000 7fbd1f7fe640 1 --2- 172.21.15.43:0/2146687785 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5567c9031ef0 0x5567c9053180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:35.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.000+0000 7fbd1f7fe640 1 -- 172.21.15.43:0/2146687785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c9031b10 msgr2=0x5567c902e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:35.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.000+0000 7fbd1f7fe640 1 --2- 172.21.15.43:0/2146687785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c9031b10 0x5567c902e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:35.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.000+0000 7fbd1f7fe640 1 -- 172.21.15.43:0/2146687785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5567c8d4c7d0 con 0x5567c902b860 2024-04-25T08:57:35.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.000+0000 7fbd1effd640 1 --2- 172.21.15.43:0/2146687785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c9031b10 0x5567c902e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:57:35.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.000+0000 7fbd1f7fe640 1 --2- 172.21.15.43:0/2146687785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567c902b860 0x5567c905ee60 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fbd18009990 tx=0x7fbd18004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ff18cc47709569 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:35.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.000+0000 7fbcfdffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:57:35.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.000+0000 7fbcfdffb640 1 -- 172.21.15.43:0/2146687785 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd18049030 con 0x5567c902b860 2024-04-25T08:57:35.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.000+0000 7fbcfdffb640 1 -- 172.21.15.43:0/2146687785 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd18003820 con 0x5567c902b860 2024-04-25T08:57:35.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.000+0000 7fbd28228740 1 -- 172.21.15.43:0/2146687785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbd1802ae50 con 0x5567c902b860 2024-04-25T08:57:35.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.000+0000 7fbcfdffb640 1 -- 172.21.15.43:0/2146687785 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd18055050 con 0x5567c902b860 2024-04-25T08:57:35.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.001+0000 7fbd28228740 10 client.6294 fetch_fsmap learned FSMap version 24 2024-04-25T08:57:35.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.001+0000 7fbd28228740 1 -- 172.21.15.43:0/2146687785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x5567c8c823f0 con 0x5567c902b860 2024-04-25T08:57:35.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.001+0000 7fbcfdffb640 1 -- 172.21.15.43:0/2146687785 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd1802ae50 con 0x5567c902b860 2024-04-25T08:57:35.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.001+0000 7fbcfdffb640 1 -- 172.21.15.43:0/2146687785 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbd18058020 con 0x5567c902b860 2024-04-25T08:57:35.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.001+0000 7fbd28228740 10 client.6294 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:57:35.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.001+0000 7fbd28228740 10 client.6294 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T08:57:35.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.001+0000 7fbd28228740 20 client.6294 populate_metadata read hostname 'smithi043' 2024-04-25T08:57:35.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.001+0000 7fbd28228740 1 --2- 172.21.15.43:0/2146687785 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5567c90306b0 0x5567c905f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:35.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.001+0000 7fbd28228740 4 client.6294 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:57:35.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.001+0000 7fbd28228740 1 -- 172.21.15.43:0/2146687785 --> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5567c8f7b2e0 con 0x5567c90306b0 2024-04-25T08:57:35.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.001+0000 7fbd1effd640 1 --2- 172.21.15.43:0/2146687785 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5567c90306b0 0x5567c905f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:35.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.002+0000 7fbcfdffb640 10 client.6294 ms_handle_connect on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:35.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.002+0000 7fbd1effd640 1 --2- 172.21.15.43:0/2146687785 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5567c90306b0 0x5567c905f3a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2940898bf30b1415 server_cookie=4c452b6471344d62 in_seq=0 out_seq=0 2024-04-25T08:57:35.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.002+0000 7fbcfdffb640 1 -- 172.21.15.43:0/2146687785 <== mds.0 v2:172.21.15.193:6836/4164634380 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5567c8f7b2e0 con 0x5567c90306b0 2024-04-25T08:57:35.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.002+0000 7fbcfdffb640 10 client.6294 handle_command_reply: tid=0 2024-04-25T08:57:35.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.002+0000 7fbd28228740 1 client.6294 shutdown 2024-04-25T08:57:35.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.002+0000 7fbd28228740 1 -- 172.21.15.43:0/2146687785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567c902b860 msgr2=0x5567c905ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:35.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.002+0000 7fbd28228740 1 --2- 172.21.15.43:0/2146687785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567c902b860 0x5567c905ee60 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fbd18009990 tx=0x7fbd18004730 comp rx=0 tx=0).stop 2024-04-25T08:57:35.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.002+0000 7fbd28228740 1 -- 172.21.15.43:0/2146687785 shutdown_connections 2024-04-25T08:57:35.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.002+0000 7fbd28228740 1 --2- 172.21.15.43:0/2146687785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567c902b860 0x5567c905ee60 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:35.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.002+0000 7fbd28228740 1 --2- 172.21.15.43:0/2146687785 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5567c9031ef0 0x5567c9053180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:35.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.002+0000 7fbd28228740 1 --2- 172.21.15.43:0/2146687785 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5567c90306b0 0x5567c905f3a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:35.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.002+0000 7fbcfdffb640 0 client.6294 ms_handle_reset on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:35.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.002+0000 7fbd28228740 1 --2- 172.21.15.43:0/2146687785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c9031b10 0x5567c902e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:35.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.002+0000 7fbd28228740 1 -- 172.21.15.43:0/2146687785 >> 172.21.15.43:0/2146687785 conn(0x5567c9004770 msgr2=0x5567c902c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:35.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.002+0000 7fbd28228740 1 -- 172.21.15.43:0/2146687785 shutdown_connections 2024-04-25T08:57:35.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.003+0000 7fbd28228740 1 -- 172.21.15.43:0/2146687785 wait complete. 2024-04-25T08:57:35.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.003+0000 7fbd28228740 20 client.6294 trim_cache size 0 max 16384 2024-04-25T08:57:36.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.018+0000 7fbd28228740 1 Processor -- start 2024-04-25T08:57:36.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.018+0000 7fbd28228740 1 -- start start 2024-04-25T08:57:36.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.018+0000 7fbd28228740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5567c902b860 0x5567c90052f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.018+0000 7fbd28228740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567c90306b0 0x5567c902d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.018+0000 7fbd28228740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c9031b10 0x5567c902d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.018+0000 7fbd28228740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd201ffde0 con 0x5567c90306b0 2024-04-25T08:57:36.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.018+0000 7fbd28228740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd201f7c10 con 0x5567c9031b10 2024-04-25T08:57:36.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.018+0000 7fbd28228740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd20160110 con 0x5567c902b860 2024-04-25T08:57:36.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.019+0000 7fbd1e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567c90306b0 0x5567c902d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.019+0000 7fbd1e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567c90306b0 0x5567c902d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58080/0 (socket says 172.21.15.43:58080) 2024-04-25T08:57:36.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.019+0000 7fbd1e7fc640 1 -- 172.21.15.43:0/3991297457 learned_addr learned my addr 172.21.15.43:0/3991297457 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:36.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.019+0000 7fbd1f7fe640 1 --2- 172.21.15.43:0/3991297457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c9031b10 0x5567c902d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.019+0000 7fbd1effd640 1 --2- 172.21.15.43:0/3991297457 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5567c902b860 0x5567c90052f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.019+0000 7fbd1e7fc640 1 -- 172.21.15.43:0/3991297457 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5567c902b860 msgr2=0x5567c90052f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:36.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.019+0000 7fbd1e7fc640 1 --2- 172.21.15.43:0/3991297457 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5567c902b860 0x5567c90052f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.019+0000 7fbd1e7fc640 1 -- 172.21.15.43:0/3991297457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c9031b10 msgr2=0x5567c902d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:36.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.019+0000 7fbd1e7fc640 1 --2- 172.21.15.43:0/3991297457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c9031b10 0x5567c902d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.019+0000 7fbd1e7fc640 1 -- 172.21.15.43:0/3991297457 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567c8c420f0 con 0x5567c90306b0 2024-04-25T08:57:36.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.019+0000 7fbd1effd640 1 --2- 172.21.15.43:0/3991297457 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5567c902b860 0x5567c90052f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:36.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.019+0000 7fbd1e7fc640 1 --2- 172.21.15.43:0/3991297457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567c90306b0 0x5567c902d1b0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fbd140586c0 tx=0x7fbd1405c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce9c0abf92c2237 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:36.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.019+0000 7fbcfffff640 1 -- 172.21.15.43:0/3991297457 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd14020040 con 0x5567c90306b0 2024-04-25T08:57:36.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.019+0000 7fbcfffff640 1 -- 172.21.15.43:0/3991297457 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd14070030 con 0x5567c90306b0 2024-04-25T08:57:36.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.019+0000 7fbcfffff640 1 -- 172.21.15.43:0/3991297457 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd14054da0 con 0x5567c90306b0 2024-04-25T08:57:36.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.020+0000 7fbd28228740 1 -- 172.21.15.43:0/3991297457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567c90306b0 msgr2=0x5567c902d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:36.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.020+0000 7fbd28228740 1 --2- 172.21.15.43:0/3991297457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567c90306b0 0x5567c902d1b0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fbd140586c0 tx=0x7fbd1405c9c0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.020+0000 7fbd28228740 1 -- 172.21.15.43:0/3991297457 shutdown_connections 2024-04-25T08:57:36.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.020+0000 7fbd28228740 1 --2- 172.21.15.43:0/3991297457 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c9031b10 0x5567c902d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.020+0000 7fbd28228740 1 --2- 172.21.15.43:0/3991297457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567c90306b0 0x5567c902d1b0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.020+0000 7fbd28228740 1 --2- 172.21.15.43:0/3991297457 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5567c902b860 0x5567c90052f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.020+0000 7fbd28228740 1 -- 172.21.15.43:0/3991297457 >> 172.21.15.43:0/3991297457 conn(0x5567c9004770 msgr2=0x5567c8fffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:36.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.020+0000 7fbd28228740 1 -- 172.21.15.43:0/3991297457 shutdown_connections 2024-04-25T08:57:36.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.020+0000 7fbd28228740 1 -- 172.21.15.43:0/3991297457 wait complete. 2024-04-25T08:57:36.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.020+0000 7fbd28228740 1 -- start start 2024-04-25T08:57:36.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.020+0000 7fbd28228740 1 Processor -- start 2024-04-25T08:57:36.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.021+0000 7fbd28228740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c902b860 0x5567c9068590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.021+0000 7fbd28228740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5567c90306b0 0x5567c905f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.021+0000 7fbd28228740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567c9031b10 0x5567c9030a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.021+0000 7fbd28228740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd201f7610 con 0x5567c9031b10 2024-04-25T08:57:36.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.021+0000 7fbd28228740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567c9069030 con 0x5567c902b860 2024-04-25T08:57:36.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.021+0000 7fbd28228740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x5567c90450a0 con 0x5567c90306b0 2024-04-25T08:57:36.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.021+0000 7fbd28228740 10 client.0 _ll_register_callbacks cb 0x7fbd18035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:36.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.021+0000 7fbd1f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567c9031b10 0x5567c9030a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.021+0000 7fbd1f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567c9031b10 0x5567c9030a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58096/0 (socket says 172.21.15.43:58096) 2024-04-25T08:57:36.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.021+0000 7fbd1f7fe640 1 -- 172.21.15.43:0/3518913833 learned_addr learned my addr 172.21.15.43:0/3518913833 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:36.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.021+0000 7fbd1effd640 1 --2- 172.21.15.43:0/3518913833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c902b860 0x5567c9068590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.021+0000 7fbd1e7fc640 1 --2- 172.21.15.43:0/3518913833 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5567c90306b0 0x5567c905f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.022+0000 7fbd1f7fe640 1 -- 172.21.15.43:0/3518913833 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5567c90306b0 msgr2=0x5567c905f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:36.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.022+0000 7fbd1f7fe640 1 --2- 172.21.15.43:0/3518913833 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5567c90306b0 0x5567c905f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.022+0000 7fbd1f7fe640 1 -- 172.21.15.43:0/3518913833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c902b860 msgr2=0x5567c9068590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:36.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.022+0000 7fbd1f7fe640 1 --2- 172.21.15.43:0/3518913833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c902b860 0x5567c9068590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.022+0000 7fbd1f7fe640 1 -- 172.21.15.43:0/3518913833 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5567c9068ad0 con 0x5567c9031b10 2024-04-25T08:57:36.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.022+0000 7fbd1f7fe640 1 --2- 172.21.15.43:0/3518913833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567c9031b10 0x5567c9030a90 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fbd1802aaf0 tx=0x7fbd1802e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2bd1e93b188d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:36.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.022+0000 7fbcfdffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:57:36.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.022+0000 7fbcfdffb640 1 -- 172.21.15.43:0/3518913833 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd18049040 con 0x5567c9031b10 2024-04-25T08:57:36.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.022+0000 7fbcfdffb640 1 -- 172.21.15.43:0/3518913833 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd18003e30 con 0x5567c9031b10 2024-04-25T08:57:36.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.022+0000 7fbd28228740 1 -- 172.21.15.43:0/3518913833 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbd14070030 con 0x5567c9031b10 2024-04-25T08:57:36.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.022+0000 7fbcfdffb640 1 -- 172.21.15.43:0/3518913833 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd18037040 con 0x5567c9031b10 2024-04-25T08:57:36.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.022+0000 7fbd28228740 10 client.6300 fetch_fsmap learned FSMap version 24 2024-04-25T08:57:36.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.022+0000 7fbd28228740 1 -- 172.21.15.43:0/3518913833 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x5567c902d9f0 con 0x5567c9031b10 2024-04-25T08:57:36.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.022+0000 7fbcfdffb640 1 -- 172.21.15.43:0/3518913833 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd14070030 con 0x5567c9031b10 2024-04-25T08:57:36.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.022+0000 7fbcfdffb640 1 -- 172.21.15.43:0/3518913833 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbd18058040 con 0x5567c9031b10 2024-04-25T08:57:36.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.022+0000 7fbd28228740 10 client.6300 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:57:36.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.022+0000 7fbd28228740 10 client.6300 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T08:57:36.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.022+0000 7fbd28228740 20 client.6300 populate_metadata read hostname 'smithi043' 2024-04-25T08:57:36.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.022+0000 7fbd28228740 1 --2- 172.21.15.43:0/3518913833 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5567c9031ef0 0x5567c9030fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.022+0000 7fbd28228740 4 client.6300 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:57:36.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.022+0000 7fbd28228740 1 -- 172.21.15.43:0/3518913833 --> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5567c8f75630 con 0x5567c9031ef0 2024-04-25T08:57:36.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.023+0000 7fbd1effd640 1 --2- 172.21.15.43:0/3518913833 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5567c9031ef0 0x5567c9030fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.023+0000 7fbd1effd640 1 --2- 172.21.15.43:0/3518913833 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5567c9031ef0 0x5567c9030fd0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dde3dbc0bd54bce2 server_cookie=cfb17a4b5ae561db in_seq=0 out_seq=0 2024-04-25T08:57:36.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.023+0000 7fbcfdffb640 10 client.6300 ms_handle_connect on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:36.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.023+0000 7fbcfdffb640 1 -- 172.21.15.43:0/3518913833 <== mds.0 v2:172.21.15.193:6836/4164634380 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5567c8f75630 con 0x5567c9031ef0 2024-04-25T08:57:36.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.023+0000 7fbcfdffb640 10 client.6300 handle_command_reply: tid=0 2024-04-25T08:57:36.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.023+0000 7fbd28228740 1 client.6300 shutdown 2024-04-25T08:57:36.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.023+0000 7fbd28228740 1 -- 172.21.15.43:0/3518913833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567c9031b10 msgr2=0x5567c9030a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:36.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.023+0000 7fbd28228740 1 --2- 172.21.15.43:0/3518913833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567c9031b10 0x5567c9030a90 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fbd1802aaf0 tx=0x7fbd1802e000 comp rx=0 tx=0).stop 2024-04-25T08:57:36.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.023+0000 7fbd28228740 1 -- 172.21.15.43:0/3518913833 shutdown_connections 2024-04-25T08:57:36.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.023+0000 7fbd28228740 1 --2- 172.21.15.43:0/3518913833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567c9031b10 0x5567c9030a90 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.023+0000 7fbd28228740 1 --2- 172.21.15.43:0/3518913833 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5567c90306b0 0x5567c905f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.023+0000 7fbd28228740 1 --2- 172.21.15.43:0/3518913833 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x5567c9031ef0 0x5567c9030fd0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.023+0000 7fbd28228740 1 --2- 172.21.15.43:0/3518913833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567c902b860 0x5567c9068590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.023+0000 7fbcfdffb640 0 client.6300 ms_handle_reset on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:36.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.023+0000 7fbd28228740 1 -- 172.21.15.43:0/3518913833 >> 172.21.15.43:0/3518913833 conn(0x5567c9004770 msgr2=0x5567c90342e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:36.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.023+0000 7fbd28228740 1 -- 172.21.15.43:0/3518913833 shutdown_connections 2024-04-25T08:57:36.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.023+0000 7fbd28228740 1 -- 172.21.15.43:0/3518913833 wait complete. 2024-04-25T08:57:36.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.023+0000 7fbd28228740 20 client.6300 trim_cache size 0 max 16384 2024-04-25T08:57:36.019 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T08:57:36.019 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-25T08:57:36.020 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-25T08:57:36.020 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T08:57:36.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.024+0000 7fbd25522640 1 -- 172.21.15.43:0/2188411104 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbd0c06b500 msgr2=0x7fbd0c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:36.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.024+0000 7fbd25522640 1 --2- 172.21.15.43:0/2188411104 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbd0c06b500 0x7fbd0c06d9c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fbd10007fd0 tx=0x7fbd10009040 comp rx=0 tx=0).stop 2024-04-25T08:57:36.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.024+0000 7fbd25522640 1 -- 172.21.15.43:0/2188411104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd200d2230 msgr2=0x7fbd20157c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:36.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.024+0000 7fbd25522640 1 --2- 172.21.15.43:0/2188411104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd200d2230 0x7fbd20157c70 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fbd1400e7e0 tx=0x7fbd1400eca0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.024+0000 7fbd25522640 1 -- 172.21.15.43:0/2188411104 shutdown_connections 2024-04-25T08:57:36.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.024+0000 7fbd25522640 1 --2- 172.21.15.43:0/2188411104 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd2015e090 0x7fbd201582c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.024+0000 7fbd25522640 1 --2- 172.21.15.43:0/2188411104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd200d2230 0x7fbd20157c70 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.024+0000 7fbd25522640 1 --2- 172.21.15.43:0/2188411104 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbd0c06b500 0x7fbd0c06d9c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.024+0000 7fbd25522640 1 --2- 172.21.15.43:0/2188411104 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd200b5560 0x7fbd2015eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.024+0000 7fbd25522640 1 -- 172.21.15.43:0/2188411104 >> 172.21.15.43:0/2188411104 conn(0x7fbd200d0560 msgr2=0x7fbd200cee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:36.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.024+0000 7fbd25522640 1 -- 172.21.15.43:0/2188411104 shutdown_connections 2024-04-25T08:57:36.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.024+0000 7fbd25522640 1 -- 172.21.15.43:0/2188411104 wait complete. 2024-04-25T08:57:36.024 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T08:57:36.024 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:57:36.025 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T08:57:36.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.138+0000 7f839561c640 1 Processor -- start 2024-04-25T08:57:36.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.138+0000 7f839561c640 1 -- start start 2024-04-25T08:57:36.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.139+0000 7f839561c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f83900d1a20 0x7f83900b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.139+0000 7f839561c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f83900d2260 0x7f83900b5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.139+0000 7f839561c640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f839015bf90 con 0x7f83900d1e00 2024-04-25T08:57:36.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.139+0000 7f839561c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83900cf430 con 0x7f83900d1a20 2024-04-25T08:57:36.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.139+0000 7f839561c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83900cee70 con 0x7f83900d2260 2024-04-25T08:57:36.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.139+0000 7f838effd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f83900d1a20 0x7f83900b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.139+0000 7f838e7fc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f83900d1e00 0x7f83900b5100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:52722/0 (socket says 172.21.15.43:52722) 2024-04-25T08:57:36.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.139+0000 7f838e7fc640 1 -- 172.21.15.43:0/1587616082 learned_addr learned my addr 172.21.15.43:0/1587616082 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:36.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.140+0000 7f838effd640 1 -- 172.21.15.43:0/1587616082 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f83900d2260 msgr2=0x7f83900b5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:36.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.140+0000 7f838f7fe640 1 --2- 172.21.15.43:0/1587616082 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f83900d2260 0x7f83900b5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.140+0000 7f838effd640 1 --2- 172.21.15.43:0/1587616082 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f83900d2260 0x7f83900b5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.140+0000 7f838dffb640 1 -- 172.21.15.43:0/1587616082 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1121816506 0 0) 0x7f839015bf90 con 0x7f83900d1e00 2024-04-25T08:57:36.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.140+0000 7f838effd640 1 -- 172.21.15.43:0/1587616082 >> v1:172.21.15.43:6789/0 conn(0x7f83900d1e00 legacy=0x7f83900b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:36.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.140+0000 7f838effd640 1 -- 172.21.15.43:0/1587616082 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83900cf150 con 0x7f83900d1a20 2024-04-25T08:57:36.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.140+0000 7f838f7fe640 1 --2- 172.21.15.43:0/1587616082 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f83900d2260 0x7f83900b5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:36.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.140+0000 7f838effd640 1 --2- 172.21.15.43:0/1587616082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83900d1a20 0x7f83900b2b90 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f83880097a0 tx=0x7f838802a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54a337c37cb8866d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:36.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.141+0000 7f838dffb640 1 -- 172.21.15.43:0/1587616082 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8388037070 con 0x7f83900d1a20 2024-04-25T08:57:36.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.141+0000 7f838dffb640 1 -- 172.21.15.43:0/1587616082 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f838802adb0 con 0x7f83900d1a20 2024-04-25T08:57:36.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.141+0000 7f838dffb640 1 -- 172.21.15.43:0/1587616082 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8388032660 con 0x7f83900d1a20 2024-04-25T08:57:36.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.141+0000 7f839561c640 1 -- 172.21.15.43:0/1587616082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83900d1a20 msgr2=0x7f83900b2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:36.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.141+0000 7f839561c640 1 --2- 172.21.15.43:0/1587616082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83900d1a20 0x7f83900b2b90 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f83880097a0 tx=0x7f838802a060 comp rx=0 tx=0).stop 2024-04-25T08:57:36.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.141+0000 7f839561c640 1 -- 172.21.15.43:0/1587616082 shutdown_connections 2024-04-25T08:57:36.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.141+0000 7f839561c640 1 --2- 172.21.15.43:0/1587616082 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f83900d2260 0x7f83900b5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.141+0000 7f839561c640 1 --2- 172.21.15.43:0/1587616082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83900d1a20 0x7f83900b2b90 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.141+0000 7f839561c640 1 -- 172.21.15.43:0/1587616082 >> 172.21.15.43:0/1587616082 conn(0x7f8390072aa0 msgr2=0x7f839009aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:36.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.141+0000 7f839561c640 1 -- 172.21.15.43:0/1587616082 shutdown_connections 2024-04-25T08:57:36.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.141+0000 7f839561c640 1 -- 172.21.15.43:0/1587616082 wait complete. 2024-04-25T08:57:36.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.142+0000 7f839561c640 1 Processor -- start 2024-04-25T08:57:36.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.143+0000 7f839561c640 1 -- start start 2024-04-25T08:57:36.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.143+0000 7f839561c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f83900d1a20 0x7f839009f380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.143+0000 7f839561c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83900d1e00 0x7f83901f1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.143+0000 7f839561c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83900d2260 0x7f839006f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.143+0000 7f839561c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83900cea60 con 0x7f83900d1e00 2024-04-25T08:57:36.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.143+0000 7f839561c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83900b4be0 con 0x7f83900d2260 2024-04-25T08:57:36.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.143+0000 7f839561c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83900cebe0 con 0x7f83900d1a20 2024-04-25T08:57:36.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.143+0000 7f838e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83900d1e00 0x7f83901f1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.143+0000 7f838e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83900d1e00 0x7f83901f1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58120/0 (socket says 172.21.15.43:58120) 2024-04-25T08:57:36.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.143+0000 7f838e7fc640 1 -- 172.21.15.43:0/1623809799 learned_addr learned my addr 172.21.15.43:0/1623809799 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:36.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.144+0000 7f838effd640 1 --2- 172.21.15.43:0/1623809799 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f83900d1a20 0x7f839009f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.144+0000 7f838f7fe640 1 --2- 172.21.15.43:0/1623809799 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83900d2260 0x7f839006f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.144+0000 7f838e7fc640 1 -- 172.21.15.43:0/1623809799 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f83900d1a20 msgr2=0x7f839009f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:36.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.144+0000 7f838e7fc640 1 --2- 172.21.15.43:0/1623809799 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f83900d1a20 0x7f839009f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.144+0000 7f838e7fc640 1 -- 172.21.15.43:0/1623809799 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83900d2260 msgr2=0x7f839006f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:36.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.144+0000 7f838e7fc640 1 --2- 172.21.15.43:0/1623809799 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83900d2260 0x7f839006f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.144+0000 7f838e7fc640 1 -- 172.21.15.43:0/1623809799 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f839009f8c0 con 0x7f83900d1e00 2024-04-25T08:57:36.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.144+0000 7f838e7fc640 1 --2- 172.21.15.43:0/1623809799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83900d1e00 0x7f83901f1440 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f837800c4d0 tx=0x7f837800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:36.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.144+0000 7f83877fe640 1 -- 172.21.15.43:0/1623809799 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8378018070 con 0x7f83900d1e00 2024-04-25T08:57:36.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.144+0000 7f83877fe640 1 -- 172.21.15.43:0/1623809799 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8378007500 con 0x7f83900d1e00 2024-04-25T08:57:36.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.144+0000 7f839561c640 1 -- 172.21.15.43:0/1623809799 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83901f1f00 con 0x7f83900d1e00 2024-04-25T08:57:36.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.144+0000 7f83877fe640 1 -- 172.21.15.43:0/1623809799 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8378004a20 con 0x7f83900d1e00 2024-04-25T08:57:36.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.144+0000 7f839561c640 1 -- 172.21.15.43:0/1623809799 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83901f27c0 con 0x7f83900d1e00 2024-04-25T08:57:36.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.146+0000 7f83877fe640 1 -- 172.21.15.43:0/1623809799 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f837801b070 con 0x7f83900d1e00 2024-04-25T08:57:36.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.146+0000 7f83877fe640 1 --2- 172.21.15.43:0/1623809799 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f837006b500 0x7f837006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.146+0000 7f83877fe640 1 -- 172.21.15.43:0/1623809799 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f83780894d0 con 0x7f83900d1e00 2024-04-25T08:57:36.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.146+0000 7f838effd640 1 --2- 172.21.15.43:0/1623809799 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f837006b500 0x7f837006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.147+0000 7f838effd640 1 --2- 172.21.15.43:0/1623809799 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f837006b500 0x7f837006d9c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8388034000 tx=0x7f838802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:36.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.147+0000 7f83983f6740 1 Processor -- start 2024-04-25T08:57:36.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.147+0000 7f83983f6740 1 -- start start 2024-04-25T08:57:36.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.148+0000 7f83983f6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c42d015aa0 0x55c42d0182b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.148+0000 7f83983f6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c42d015e80 0x55c42d0187f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.148+0000 7f83983f6740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c42cfe8700 0x55c42cfebf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.148+0000 7f83983f6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42cf51e00 con 0x55c42d015e80 2024-04-25T08:57:36.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.148+0000 7f83983f6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42cefad20 con 0x55c42d015aa0 2024-04-25T08:57:36.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.148+0000 7f83983f6740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c42cf4dea0 con 0x55c42cfe8700 2024-04-25T08:57:36.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.148+0000 7f838e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c42d015e80 0x55c42d0187f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.148+0000 7f838e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c42d015e80 0x55c42d0187f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58126/0 (socket says 172.21.15.43:58126) 2024-04-25T08:57:36.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.148+0000 7f838e7fc640 1 -- 172.21.15.43:0/2663824957 learned_addr learned my addr 172.21.15.43:0/2663824957 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:36.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.148+0000 7f838f7fe640 1 --2- 172.21.15.43:0/2663824957 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c42cfe8700 0x55c42cfebf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.148+0000 7f838effd640 1 --2- 172.21.15.43:0/2663824957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c42d015aa0 0x55c42d0182b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.148+0000 7f838e7fc640 1 -- 172.21.15.43:0/2663824957 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c42cfe8700 msgr2=0x55c42cfebf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:36.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.148+0000 7f838e7fc640 1 --2- 172.21.15.43:0/2663824957 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c42cfe8700 0x55c42cfebf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.148+0000 7f838e7fc640 1 -- 172.21.15.43:0/2663824957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c42d015aa0 msgr2=0x55c42d0182b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:36.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.148+0000 7f838e7fc640 1 --2- 172.21.15.43:0/2663824957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c42d015aa0 0x55c42d0182b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.148+0000 7f838e7fc640 1 -- 172.21.15.43:0/2663824957 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c42cf1cdf0 con 0x55c42d015e80 2024-04-25T08:57:36.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.149+0000 7f838e7fc640 1 --2- 172.21.15.43:0/2663824957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c42d015e80 0x55c42d0187f0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f83780077e0 tx=0x7f8378058480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=255d104fec4050c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:36.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.149+0000 7f8384ff9640 1 -- 172.21.15.43:0/2663824957 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8378060350 con 0x55c42d015e80 2024-04-25T08:57:36.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.149+0000 7f8384ff9640 1 -- 172.21.15.43:0/2663824957 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8378055f40 con 0x55c42d015e80 2024-04-25T08:57:36.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.149+0000 7f8384ff9640 1 -- 172.21.15.43:0/2663824957 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83780890c0 con 0x55c42d015e80 2024-04-25T08:57:36.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.149+0000 7f83983f6740 1 -- 172.21.15.43:0/2663824957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c42d015e80 msgr2=0x55c42d0187f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:36.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.149+0000 7f83983f6740 1 --2- 172.21.15.43:0/2663824957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c42d015e80 0x55c42d0187f0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f83780077e0 tx=0x7f8378058480 comp rx=0 tx=0).stop 2024-04-25T08:57:36.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.149+0000 7f83983f6740 1 -- 172.21.15.43:0/2663824957 shutdown_connections 2024-04-25T08:57:36.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.149+0000 7f83983f6740 1 --2- 172.21.15.43:0/2663824957 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c42cfe8700 0x55c42cfebf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.149+0000 7f83983f6740 1 --2- 172.21.15.43:0/2663824957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c42d015e80 0x55c42d0187f0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.149+0000 7f83983f6740 1 --2- 172.21.15.43:0/2663824957 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c42d015aa0 0x55c42d0182b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.149+0000 7f83983f6740 1 -- 172.21.15.43:0/2663824957 >> 172.21.15.43:0/2663824957 conn(0x55c42cfea970 msgr2=0x55c42d0332b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:36.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.149+0000 7f83983f6740 1 -- 172.21.15.43:0/2663824957 shutdown_connections 2024-04-25T08:57:36.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.149+0000 7f83983f6740 1 -- 172.21.15.43:0/2663824957 wait complete. 2024-04-25T08:57:36.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.150+0000 7f83983f6740 1 -- start start 2024-04-25T08:57:36.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.150+0000 7f83983f6740 1 Processor -- start 2024-04-25T08:57:36.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.151+0000 7f83983f6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c42d015aa0 0x55c42d01ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.151+0000 7f83983f6740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c42d015e80 0x55c42d0471c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.151+0000 7f83983f6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c42d00c640 0x55c42d047700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.151+0000 7f83983f6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42cfedb20 con 0x55c42d00c640 2024-04-25T08:57:36.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.151+0000 7f83983f6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42cef0590 con 0x55c42d015aa0 2024-04-25T08:57:36.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.151+0000 7f83983f6740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c42cfede00 con 0x55c42d015e80 2024-04-25T08:57:36.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.151+0000 7f83983f6740 10 client.0 _ll_register_callbacks cb 0x55c42cefce30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:36.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.151+0000 7f838f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c42d00c640 0x55c42d047700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.151+0000 7f838f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c42d00c640 0x55c42d047700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58134/0 (socket says 172.21.15.43:58134) 2024-04-25T08:57:36.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.151+0000 7f838f7fe640 1 -- 172.21.15.43:0/642037276 learned_addr learned my addr 172.21.15.43:0/642037276 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:36.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.151+0000 7f838e7fc640 1 --2- 172.21.15.43:0/642037276 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c42d015e80 0x55c42d0471c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.151+0000 7f838effd640 1 --2- 172.21.15.43:0/642037276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c42d015aa0 0x55c42d01ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.151+0000 7f838f7fe640 1 -- 172.21.15.43:0/642037276 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c42d015e80 msgr2=0x55c42d0471c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:36.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.151+0000 7f838f7fe640 1 --2- 172.21.15.43:0/642037276 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c42d015e80 0x55c42d0471c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.151+0000 7f838f7fe640 1 -- 172.21.15.43:0/642037276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c42d015aa0 msgr2=0x55c42d01ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:36.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.151+0000 7f838f7fe640 1 --2- 172.21.15.43:0/642037276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c42d015aa0 0x55c42d01ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.151+0000 7f838f7fe640 1 -- 172.21.15.43:0/642037276 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c42cc663f0 con 0x55c42d00c640 2024-04-25T08:57:36.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.151+0000 7f838effd640 1 --2- 172.21.15.43:0/642037276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c42d015aa0 0x55c42d01ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:36.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.151+0000 7f838f7fe640 1 --2- 172.21.15.43:0/642037276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c42d00c640 0x55c42d047700 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f838000fed0 tx=0x7f838000e7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35cb8b84b53c1ded server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:36.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.151+0000 7f836dffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:57:36.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.152+0000 7f836dffb640 1 -- 172.21.15.43:0/642037276 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8380019420 con 0x55c42d00c640 2024-04-25T08:57:36.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.152+0000 7f836dffb640 1 -- 172.21.15.43:0/642037276 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f838001ac70 con 0x55c42d00c640 2024-04-25T08:57:36.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.152+0000 7f83983f6740 1 -- 172.21.15.43:0/642037276 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8378055f40 con 0x55c42d00c640 2024-04-25T08:57:36.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.152+0000 7f836dffb640 1 -- 172.21.15.43:0/642037276 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f838001ae70 con 0x55c42d00c640 2024-04-25T08:57:36.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.152+0000 7f83983f6740 10 client.6315 fetch_fsmap learned FSMap version 24 2024-04-25T08:57:36.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.152+0000 7f83983f6740 1 -- 172.21.15.43:0/642037276 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55c42cd307d0 con 0x55c42d00c640 2024-04-25T08:57:36.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.152+0000 7f836dffb640 1 -- 172.21.15.43:0/642037276 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8378055f40 con 0x55c42d00c640 2024-04-25T08:57:36.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.152+0000 7f836dffb640 1 -- 172.21.15.43:0/642037276 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8380021050 con 0x55c42d00c640 2024-04-25T08:57:36.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.152+0000 7f83983f6740 10 client.6315 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:57:36.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.152+0000 7f83983f6740 10 client.6315 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T08:57:36.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.152+0000 7f83983f6740 20 client.6315 populate_metadata read hostname 'smithi043' 2024-04-25T08:57:36.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.152+0000 7f83983f6740 1 --2- 172.21.15.43:0/642037276 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55c42d04e5c0 0x55c42d0509b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.152+0000 7f83983f6740 4 client.6315 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:57:36.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.152+0000 7f83983f6740 1 -- 172.21.15.43:0/642037276 --> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c42cf5f340 con 0x55c42d04e5c0 2024-04-25T08:57:36.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.152+0000 7f838effd640 1 --2- 172.21.15.43:0/642037276 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55c42d04e5c0 0x55c42d0509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.153+0000 7f838effd640 1 --2- 172.21.15.43:0/642037276 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55c42d04e5c0 0x55c42d0509b0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed07e3840db8156d server_cookie=8819682957a76dff in_seq=0 out_seq=0 2024-04-25T08:57:36.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.153+0000 7f836dffb640 10 client.6315 ms_handle_connect on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:36.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.153+0000 7f836dffb640 1 -- 172.21.15.43:0/642037276 <== mds.0 v2:172.21.15.193:6836/4164634380 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c42cf5f340 con 0x55c42d04e5c0 2024-04-25T08:57:36.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.153+0000 7f836dffb640 10 client.6315 handle_command_reply: tid=0 2024-04-25T08:57:36.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.153+0000 7f83983f6740 1 client.6315 shutdown 2024-04-25T08:57:36.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.153+0000 7f83983f6740 1 -- 172.21.15.43:0/642037276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c42d00c640 msgr2=0x55c42d047700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:36.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.153+0000 7f83983f6740 1 --2- 172.21.15.43:0/642037276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c42d00c640 0x55c42d047700 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f838000fed0 tx=0x7f838000e7b0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.153+0000 7f83983f6740 1 -- 172.21.15.43:0/642037276 shutdown_connections 2024-04-25T08:57:36.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.153+0000 7f83983f6740 1 --2- 172.21.15.43:0/642037276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c42d00c640 0x55c42d047700 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.153+0000 7f83983f6740 1 --2- 172.21.15.43:0/642037276 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c42d015e80 0x55c42d0471c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.153+0000 7f83983f6740 1 --2- 172.21.15.43:0/642037276 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55c42d04e5c0 0x55c42d0509b0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.153+0000 7f83983f6740 1 --2- 172.21.15.43:0/642037276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c42d015aa0 0x55c42d01ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.153+0000 7f836dffb640 0 client.6315 ms_handle_reset on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:36.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.153+0000 7f83983f6740 1 -- 172.21.15.43:0/642037276 >> 172.21.15.43:0/642037276 conn(0x55c42cfe8700 msgr2=0x55c42d014f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:36.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.154+0000 7f83983f6740 1 -- 172.21.15.43:0/642037276 shutdown_connections 2024-04-25T08:57:36.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.154+0000 7f83983f6740 1 -- 172.21.15.43:0/642037276 wait complete. 2024-04-25T08:57:36.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.154+0000 7f83983f6740 20 client.6315 trim_cache size 0 max 16384 2024-04-25T08:57:36.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.165+0000 7f83983f6740 1 Processor -- start 2024-04-25T08:57:36.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.165+0000 7f83983f6740 1 -- start start 2024-04-25T08:57:36.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.166+0000 7f83983f6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c42d00c640 0x55c42cfe8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.166+0000 7f83983f6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c42d015aa0 0x55c42d014d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.166+0000 7f83983f6740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c42d015e80 0x55c42d0152c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.166+0000 7f83983f6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83901fda50 con 0x55c42d015aa0 2024-04-25T08:57:36.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.166+0000 7f83983f6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83901fcd80 con 0x55c42d00c640 2024-04-25T08:57:36.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.166+0000 7f83983f6740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83901f3d30 con 0x55c42d015e80 2024-04-25T08:57:36.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.166+0000 7f838e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c42d015aa0 0x55c42d014d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.166+0000 7f838e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c42d015aa0 0x55c42d014d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58144/0 (socket says 172.21.15.43:58144) 2024-04-25T08:57:36.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.166+0000 7f838e7fc640 1 -- 172.21.15.43:0/2228220770 learned_addr learned my addr 172.21.15.43:0/2228220770 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:36.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.166+0000 7f838f7fe640 1 --2- 172.21.15.43:0/2228220770 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c42d015e80 0x55c42d0152c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.167+0000 7f838effd640 1 --2- 172.21.15.43:0/2228220770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c42d00c640 0x55c42cfe8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.167+0000 7f838e7fc640 1 -- 172.21.15.43:0/2228220770 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c42d015e80 msgr2=0x55c42d0152c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:36.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.167+0000 7f838e7fc640 1 --2- 172.21.15.43:0/2228220770 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c42d015e80 0x55c42d0152c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.167+0000 7f838e7fc640 1 -- 172.21.15.43:0/2228220770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c42d00c640 msgr2=0x55c42cfe8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:36.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.167+0000 7f838e7fc640 1 --2- 172.21.15.43:0/2228220770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c42d00c640 0x55c42cfe8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.167+0000 7f838e7fc640 1 -- 172.21.15.43:0/2228220770 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c42cfe4430 con 0x55c42d015aa0 2024-04-25T08:57:36.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.167+0000 7f838f7fe640 1 --2- 172.21.15.43:0/2228220770 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c42d015e80 0x55c42d0152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:36.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.167+0000 7f838effd640 1 --2- 172.21.15.43:0/2228220770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c42d00c640 0x55c42cfe8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:36.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.167+0000 7f838e7fc640 1 --2- 172.21.15.43:0/2228220770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c42d015aa0 0x55c42d014d80 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f837805a010 tx=0x7f837805b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e73c00d98bcb4ab2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:36.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.167+0000 7f836ffff640 1 -- 172.21.15.43:0/2228220770 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8378086020 con 0x55c42d015aa0 2024-04-25T08:57:36.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.167+0000 7f836ffff640 1 -- 172.21.15.43:0/2228220770 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8378055f40 con 0x55c42d015aa0 2024-04-25T08:57:36.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.167+0000 7f836ffff640 1 -- 172.21.15.43:0/2228220770 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8378057a10 con 0x55c42d015aa0 2024-04-25T08:57:36.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.167+0000 7f83983f6740 1 -- 172.21.15.43:0/2228220770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c42d015aa0 msgr2=0x55c42d014d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:36.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.167+0000 7f83983f6740 1 --2- 172.21.15.43:0/2228220770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c42d015aa0 0x55c42d014d80 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f837805a010 tx=0x7f837805b000 comp rx=0 tx=0).stop 2024-04-25T08:57:36.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.167+0000 7f83983f6740 1 -- 172.21.15.43:0/2228220770 shutdown_connections 2024-04-25T08:57:36.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.167+0000 7f83983f6740 1 --2- 172.21.15.43:0/2228220770 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c42d015e80 0x55c42d0152c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.167+0000 7f83983f6740 1 --2- 172.21.15.43:0/2228220770 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c42d015aa0 0x55c42d014d80 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.167+0000 7f83983f6740 1 --2- 172.21.15.43:0/2228220770 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c42d00c640 0x55c42cfe8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.167+0000 7f83983f6740 1 -- 172.21.15.43:0/2228220770 >> 172.21.15.43:0/2228220770 conn(0x55c42cfe8700 msgr2=0x55c42d0184a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:36.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.167+0000 7f83983f6740 1 -- 172.21.15.43:0/2228220770 shutdown_connections 2024-04-25T08:57:36.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.167+0000 7f83983f6740 1 -- 172.21.15.43:0/2228220770 wait complete. 2024-04-25T08:57:36.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.168+0000 7f83983f6740 1 -- start start 2024-04-25T08:57:36.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.168+0000 7f83983f6740 1 Processor -- start 2024-04-25T08:57:36.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.169+0000 7f83983f6740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c42d00c640 0x55c42d014670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.169+0000 7f83983f6740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c42d015aa0 0x55c42d04dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.169+0000 7f83983f6740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c42d015e80 0x55c42cfec8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.169+0000 7f83983f6740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42d02d3f0 con 0x55c42d015aa0 2024-04-25T08:57:36.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.169+0000 7f83983f6740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c42cfe7f70 con 0x55c42d00c640 2024-04-25T08:57:36.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.169+0000 7f83983f6740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c42d015330 con 0x55c42d015e80 2024-04-25T08:57:36.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.169+0000 7f83983f6740 10 client.0 _ll_register_callbacks cb 0x7f83780076b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:36.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.169+0000 7f838e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c42d015aa0 0x55c42d04dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.169+0000 7f838e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c42d015aa0 0x55c42d04dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58158/0 (socket says 172.21.15.43:58158) 2024-04-25T08:57:36.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.169+0000 7f838e7fc640 1 -- 172.21.15.43:0/4105520669 learned_addr learned my addr 172.21.15.43:0/4105520669 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:36.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.169+0000 7f838e7fc640 1 -- 172.21.15.43:0/4105520669 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c42d015e80 msgr2=0x55c42cfec8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:36.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.169+0000 7f838effd640 1 --2- 172.21.15.43:0/4105520669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c42d00c640 0x55c42d014670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.169+0000 7f838f7fe640 1 --2- 172.21.15.43:0/4105520669 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c42d015e80 0x55c42cfec8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.169+0000 7f838e7fc640 1 --2- 172.21.15.43:0/4105520669 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c42d015e80 0x55c42cfec8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.169+0000 7f838e7fc640 1 -- 172.21.15.43:0/4105520669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c42d00c640 msgr2=0x55c42d014670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:36.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.169+0000 7f838e7fc640 1 --2- 172.21.15.43:0/4105520669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c42d00c640 0x55c42d014670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.169+0000 7f838e7fc640 1 -- 172.21.15.43:0/4105520669 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c42d037340 con 0x55c42d015aa0 2024-04-25T08:57:36.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.169+0000 7f838effd640 1 --2- 172.21.15.43:0/4105520669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c42d00c640 0x55c42d014670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:36.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.169+0000 7f838e7fc640 1 --2- 172.21.15.43:0/4105520669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c42d015aa0 0x55c42d04dcc0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f8378057940 tx=0x7f8378000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db7a166deac8376a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:36.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.169+0000 7f836dffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:57:36.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.170+0000 7f836dffb640 1 -- 172.21.15.43:0/4105520669 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837807b060 con 0x55c42d015aa0 2024-04-25T08:57:36.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.170+0000 7f836dffb640 1 -- 172.21.15.43:0/4105520669 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8378004bc0 con 0x55c42d015aa0 2024-04-25T08:57:36.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.170+0000 7f83983f6740 1 -- 172.21.15.43:0/4105520669 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8378055f40 con 0x55c42d015aa0 2024-04-25T08:57:36.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.170+0000 7f836dffb640 1 -- 172.21.15.43:0/4105520669 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8378086030 con 0x55c42d015aa0 2024-04-25T08:57:36.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.170+0000 7f83983f6740 10 client.6321 fetch_fsmap learned FSMap version 24 2024-04-25T08:57:36.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.170+0000 7f83983f6740 1 -- 172.21.15.43:0/4105520669 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55c42d04e200 con 0x55c42d015aa0 2024-04-25T08:57:36.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.170+0000 7f836dffb640 1 -- 172.21.15.43:0/4105520669 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8378055f40 con 0x55c42d015aa0 2024-04-25T08:57:36.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.170+0000 7f836dffb640 1 -- 172.21.15.43:0/4105520669 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8378057090 con 0x55c42d015aa0 2024-04-25T08:57:36.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.170+0000 7f83983f6740 10 client.6321 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:57:36.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.170+0000 7f83983f6740 10 client.6321 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T08:57:36.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.170+0000 7f83983f6740 20 client.6321 populate_metadata read hostname 'smithi043' 2024-04-25T08:57:36.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.170+0000 7f83983f6740 1 --2- 172.21.15.43:0/4105520669 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55c42d0551d0 0x55c42d0a1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.170+0000 7f83983f6740 4 client.6321 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T08:57:36.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.170+0000 7f83983f6740 1 -- 172.21.15.43:0/4105520669 --> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c42cf59690 con 0x55c42d0551d0 2024-04-25T08:57:36.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.170+0000 7f838effd640 1 --2- 172.21.15.43:0/4105520669 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55c42d0551d0 0x55c42d0a1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.171+0000 7f838effd640 1 --2- 172.21.15.43:0/4105520669 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55c42d0551d0 0x55c42d0a1c20 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2f4cd8cdff50474 server_cookie=7e2d0df3072b2ae9 in_seq=0 out_seq=0 2024-04-25T08:57:36.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.171+0000 7f836dffb640 10 client.6321 ms_handle_connect on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:36.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.171+0000 7f836dffb640 1 -- 172.21.15.43:0/4105520669 <== mds.0 v2:172.21.15.193:6836/4164634380 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+168 (crc 0 0 0) 0x55c42cf59690 con 0x55c42d0551d0 2024-04-25T08:57:36.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.171+0000 7f836dffb640 10 client.6321 handle_command_reply: tid=0 2024-04-25T08:57:36.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.171+0000 7f83983f6740 1 client.6321 shutdown 2024-04-25T08:57:36.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.171+0000 7f83983f6740 1 -- 172.21.15.43:0/4105520669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c42d015aa0 msgr2=0x55c42d04dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:36.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.171+0000 7f83983f6740 1 --2- 172.21.15.43:0/4105520669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c42d015aa0 0x55c42d04dcc0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f8378057940 tx=0x7f8378000ed0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.171+0000 7f83983f6740 1 -- 172.21.15.43:0/4105520669 shutdown_connections 2024-04-25T08:57:36.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.171+0000 7f83983f6740 1 --2- 172.21.15.43:0/4105520669 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c42d015e80 0x55c42cfec8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.171+0000 7f83983f6740 1 --2- 172.21.15.43:0/4105520669 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c42d015aa0 0x55c42d04dcc0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.171+0000 7f83983f6740 1 --2- 172.21.15.43:0/4105520669 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55c42d0551d0 0x55c42d0a1c20 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.171+0000 7f83983f6740 1 --2- 172.21.15.43:0/4105520669 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c42d00c640 0x55c42d014670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.171+0000 7f836dffb640 0 client.6321 ms_handle_reset on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:36.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.171+0000 7f83983f6740 1 -- 172.21.15.43:0/4105520669 >> 172.21.15.43:0/4105520669 conn(0x55c42cfe8700 msgr2=0x55c42cfe8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:36.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.171+0000 7f83983f6740 1 -- 172.21.15.43:0/4105520669 shutdown_connections 2024-04-25T08:57:36.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.171+0000 7f83983f6740 1 -- 172.21.15.43:0/4105520669 wait complete. 2024-04-25T08:57:36.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.171+0000 7f83983f6740 20 client.6321 trim_cache size 0 max 16384 2024-04-25T08:57:36.169 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2024-04-25T08:57:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:57:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: "damage_type": "backtrace", 2024-04-25T08:57:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 2580061446, 2024-04-25T08:57:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 1099511630414, 2024-04-25T08:57:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_1" 2024-04-25T08:57:36.169 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:57:36.169 INFO:teuthology.orchestra.run.smithi043.stdout:] 2024-04-25T08:57:36.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.172+0000 7f839561c640 1 -- 172.21.15.43:0/1623809799 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f837006b500 msgr2=0x7f837006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:36.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.172+0000 7f839561c640 1 --2- 172.21.15.43:0/1623809799 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f837006b500 0x7f837006d9c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8388034000 tx=0x7f838802e000 comp rx=0 tx=0).stop 2024-04-25T08:57:36.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.172+0000 7f839561c640 1 -- 172.21.15.43:0/1623809799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83900d1e00 msgr2=0x7f83901f1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:36.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.172+0000 7f839561c640 1 --2- 172.21.15.43:0/1623809799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83900d1e00 0x7f83901f1440 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f837800c4d0 tx=0x7f837800c990 comp rx=0 tx=0).stop 2024-04-25T08:57:36.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.172+0000 7f839561c640 1 -- 172.21.15.43:0/1623809799 shutdown_connections 2024-04-25T08:57:36.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.172+0000 7f839561c640 1 --2- 172.21.15.43:0/1623809799 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f837006b500 0x7f837006d9c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.172+0000 7f839561c640 1 --2- 172.21.15.43:0/1623809799 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83900d2260 0x7f839006f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.172+0000 7f839561c640 1 --2- 172.21.15.43:0/1623809799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83900d1e00 0x7f83901f1440 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.172+0000 7f839561c640 1 --2- 172.21.15.43:0/1623809799 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f83900d1a20 0x7f839009f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.172+0000 7f839561c640 1 -- 172.21.15.43:0/1623809799 >> 172.21.15.43:0/1623809799 conn(0x7f8390072aa0 msgr2=0x7f839015c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:36.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.172+0000 7f839561c640 1 -- 172.21.15.43:0/1623809799 shutdown_connections 2024-04-25T08:57:36.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.172+0000 7f839561c640 1 -- 172.21.15.43:0/1623809799 wait complete. 2024-04-25T08:57:36.172 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:57:36.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.283+0000 7f4fd333b640 1 Processor -- start 2024-04-25T08:57:36.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.284+0000 7f4fd333b640 1 -- start start 2024-04-25T08:57:36.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.284+0000 7f4fd333b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4fcc161390 0x7f4fcc0b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.284+0000 7f4fd333b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4fcc0cecf0 0x7f4fcc0d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.284+0000 7f4fd333b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fcc0b4c30 con 0x7f4fcc0d02a0 2024-04-25T08:57:36.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.284+0000 7f4fd333b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fcc0d1910 con 0x7f4fcc0cecf0 2024-04-25T08:57:36.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.284+0000 7f4fd333b640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc0cf2c0 con 0x7f4fcc161390 2024-04-25T08:57:36.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.285+0000 7f4fd18d8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4fcc0cecf0 0x7f4fcc0d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.285+0000 7f4fd08d6640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4fcc0d02a0 0x7f4fcc0ce5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:52726/0 (socket says 172.21.15.43:52726) 2024-04-25T08:57:36.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.285+0000 7f4fd08d6640 1 -- 172.21.15.43:0/3589333796 learned_addr learned my addr 172.21.15.43:0/3589333796 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:36.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.285+0000 7f4fd10d7640 1 --2- 172.21.15.43:0/3589333796 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4fcc161390 0x7f4fcc0b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.285+0000 7f4fd18d8640 1 -- 172.21.15.43:0/3589333796 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4fcc161390 msgr2=0x7f4fcc0b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:36.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.285+0000 7f4fd18d8640 1 --2- 172.21.15.43:0/3589333796 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4fcc161390 0x7f4fcc0b6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.285+0000 7f4fd18d8640 1 -- 172.21.15.43:0/3589333796 >> v1:172.21.15.43:6789/0 conn(0x7f4fcc0d02a0 legacy=0x7f4fcc0ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:36.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.285+0000 7f4fc3fff640 1 -- 172.21.15.43:0/3589333796 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4201139266 0 0) 0x7f4fbc001110 con 0x7f4fcc0d02a0 2024-04-25T08:57:36.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.285+0000 7f4fd18d8640 1 -- 172.21.15.43:0/3589333796 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fcc0cf5a0 con 0x7f4fcc0cecf0 2024-04-25T08:57:36.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.285+0000 7f4fd10d7640 1 --2- 172.21.15.43:0/3589333796 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4fcc161390 0x7f4fcc0b6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:36.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.286+0000 7f4fd18d8640 1 --2- 172.21.15.43:0/3589333796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fcc0cecf0 0x7f4fcc0d0720 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4009d90 tx=0x7f4fc402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa265551ecd21da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:36.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.286+0000 7f4fc3fff640 1 -- 172.21.15.43:0/3589333796 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4007740 con 0x7f4fcc0cecf0 2024-04-25T08:57:36.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.286+0000 7f4fc3fff640 1 -- 172.21.15.43:0/3589333796 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc40078e0 con 0x7f4fcc0cecf0 2024-04-25T08:57:36.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.286+0000 7f4fc3fff640 1 -- 172.21.15.43:0/3589333796 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4007cb0 con 0x7f4fcc0cecf0 2024-04-25T08:57:36.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.286+0000 7f4fd333b640 1 -- 172.21.15.43:0/3589333796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fcc0cecf0 msgr2=0x7f4fcc0d0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:36.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.286+0000 7f4fd333b640 1 --2- 172.21.15.43:0/3589333796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fcc0cecf0 0x7f4fcc0d0720 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4009d90 tx=0x7f4fc402a660 comp rx=0 tx=0).stop 2024-04-25T08:57:36.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.286+0000 7f4fd333b640 1 -- 172.21.15.43:0/3589333796 shutdown_connections 2024-04-25T08:57:36.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.286+0000 7f4fd333b640 1 --2- 172.21.15.43:0/3589333796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fcc0cecf0 0x7f4fcc0d0720 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.286+0000 7f4fd333b640 1 --2- 172.21.15.43:0/3589333796 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4fcc161390 0x7f4fcc0b6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.286+0000 7f4fd333b640 1 -- 172.21.15.43:0/3589333796 >> 172.21.15.43:0/3589333796 conn(0x7f4fcc09cd20 msgr2=0x7f4fcc0d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:36.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.287+0000 7f4fd333b640 1 -- 172.21.15.43:0/3589333796 shutdown_connections 2024-04-25T08:57:36.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.287+0000 7f4fd333b640 1 -- 172.21.15.43:0/3589333796 wait complete. 2024-04-25T08:57:36.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.288+0000 7f4fd333b640 1 Processor -- start 2024-04-25T08:57:36.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.288+0000 7f4fd333b640 1 -- start start 2024-04-25T08:57:36.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.288+0000 7f4fd333b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4fcc0cecf0 0x7f4fcc14d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.288+0000 7f4fd333b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4fcc0d02a0 0x7f4fcc0b26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.288+0000 7f4fd333b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fcc161390 0x7f4fcc0b0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.288+0000 7f4fd333b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc161770 con 0x7f4fcc161390 2024-04-25T08:57:36.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.288+0000 7f4fd333b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc0d15a0 con 0x7f4fcc0d02a0 2024-04-25T08:57:36.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.288+0000 7f4fd333b640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc0d1da0 con 0x7f4fcc0cecf0 2024-04-25T08:57:36.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.289+0000 7f4fd18d8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fcc161390 0x7f4fcc0b0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.289+0000 7f4fd18d8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fcc161390 0x7f4fcc0b0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58170/0 (socket says 172.21.15.43:58170) 2024-04-25T08:57:36.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.289+0000 7f4fd18d8640 1 -- 172.21.15.43:0/2969790941 learned_addr learned my addr 172.21.15.43:0/2969790941 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:36.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.289+0000 7f4fd10d7640 1 --2- 172.21.15.43:0/2969790941 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4fcc0cecf0 0x7f4fcc14d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.289+0000 7f4fd08d6640 1 --2- 172.21.15.43:0/2969790941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4fcc0d02a0 0x7f4fcc0b26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.289+0000 7f4fd18d8640 1 -- 172.21.15.43:0/2969790941 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4fcc0cecf0 msgr2=0x7f4fcc14d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:36.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.289+0000 7f4fd18d8640 1 --2- 172.21.15.43:0/2969790941 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4fcc0cecf0 0x7f4fcc14d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.289+0000 7f4fd18d8640 1 -- 172.21.15.43:0/2969790941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4fcc0d02a0 msgr2=0x7f4fcc0b26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:36.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.289+0000 7f4fd18d8640 1 --2- 172.21.15.43:0/2969790941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4fcc0d02a0 0x7f4fcc0b26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.289+0000 7f4fd18d8640 1 -- 172.21.15.43:0/2969790941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fcc14db30 con 0x7f4fcc161390 2024-04-25T08:57:36.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.289+0000 7f4fd10d7640 1 --2- 172.21.15.43:0/2969790941 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4fcc0cecf0 0x7f4fcc14d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:36.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.289+0000 7f4fd18d8640 1 --2- 172.21.15.43:0/2969790941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fcc161390 0x7f4fcc0b0680 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4009d90 tx=0x7f4fc4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:36.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.289+0000 7f4fc1ffb640 1 -- 172.21.15.43:0/2969790941 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4040070 con 0x7f4fcc161390 2024-04-25T08:57:36.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.289+0000 7f4fc1ffb640 1 -- 172.21.15.43:0/2969790941 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc4031c40 con 0x7f4fcc161390 2024-04-25T08:57:36.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.289+0000 7f4fd333b640 1 -- 172.21.15.43:0/2969790941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fcc14de10 con 0x7f4fcc161390 2024-04-25T08:57:36.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.289+0000 7f4fc1ffb640 1 -- 172.21.15.43:0/2969790941 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4003990 con 0x7f4fcc161390 2024-04-25T08:57:36.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.289+0000 7f4fd333b640 1 -- 172.21.15.43:0/2969790941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fcc14e3d0 con 0x7f4fcc161390 2024-04-25T08:57:36.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.290+0000 7f4fd333b640 1 -- 172.21.15.43:0/2969790941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fcc0680f0 con 0x7f4fcc161390 2024-04-25T08:57:36.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.291+0000 7f4fc1ffb640 1 -- 172.21.15.43:0/2969790941 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4fc4039040 con 0x7f4fcc161390 2024-04-25T08:57:36.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.291+0000 7f4fc1ffb640 1 --2- 172.21.15.43:0/2969790941 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4fb006b4d0 0x7f4fb006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.291+0000 7f4fc1ffb640 1 -- 172.21.15.43:0/2969790941 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4fc40ab270 con 0x7f4fcc161390 2024-04-25T08:57:36.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.292+0000 7f4fd10d7640 1 --2- 172.21.15.43:0/2969790941 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4fb006b4d0 0x7f4fb006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.292+0000 7f4fd10d7640 1 --2- 172.21.15.43:0/2969790941 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4fb006b4d0 0x7f4fb006d990 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8006fd0 tx=0x7f4fc8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:36.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.296+0000 7f4fc1ffb640 1 -- 172.21.15.43:0/2969790941 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4fc407a050 con 0x7f4fcc161390 2024-04-25T08:57:36.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.514+0000 7f4fd333b640 1 -- 172.21.15.43:0/2969790941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4fcc074ef0 con 0x7f4fcc161390 2024-04-25T08:57:36.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.514+0000 7f4fc1ffb640 1 -- 172.21.15.43:0/2969790941 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4945 (secure 0 0 0) 0x7f4fc4079950 con 0x7f4fcc161390 2024-04-25T08:57:36.504 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:57:36.504 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1504450503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1504450503},{"type":"v1","addr":"172.21.15.193:6835","nonce":1504450503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5691,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2657133821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2657133821},{"type":"v1","addr":"172.21.15.43:6834","nonce":2657133821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":5694,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/4271431818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":4271431818},{"type":"v1","addr":"172.21.15.43:6835","nonce":4271431818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:57:01.856132+0000","modified":"2024-04-25T08:57:34.526463+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5709":{"gid":5709,"name":"d","rank":0,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6837/4164634380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4164634380},{"type":"v1","addr":"172.21.15.193:6837","nonce":4164634380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:57:36.504 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 24 2024-04-25T08:57:36.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.516+0000 7f4fd333b640 1 -- 172.21.15.43:0/2969790941 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4fb006b4d0 msgr2=0x7f4fb006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:36.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.516+0000 7f4fd333b640 1 --2- 172.21.15.43:0/2969790941 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4fb006b4d0 0x7f4fb006d990 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8006fd0 tx=0x7f4fc8008040 comp rx=0 tx=0).stop 2024-04-25T08:57:36.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.516+0000 7f4fd333b640 1 -- 172.21.15.43:0/2969790941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fcc161390 msgr2=0x7f4fcc0b0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:36.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.516+0000 7f4fd333b640 1 --2- 172.21.15.43:0/2969790941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fcc161390 0x7f4fcc0b0680 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4009d90 tx=0x7f4fc4004a40 comp rx=0 tx=0).stop 2024-04-25T08:57:36.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.516+0000 7f4fd333b640 1 -- 172.21.15.43:0/2969790941 shutdown_connections 2024-04-25T08:57:36.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.516+0000 7f4fd333b640 1 --2- 172.21.15.43:0/2969790941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fcc161390 0x7f4fcc0b0680 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.516+0000 7f4fd333b640 1 --2- 172.21.15.43:0/2969790941 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4fb006b4d0 0x7f4fb006d990 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.516+0000 7f4fd333b640 1 --2- 172.21.15.43:0/2969790941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4fcc0d02a0 0x7f4fcc0b26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.516+0000 7f4fd333b640 1 --2- 172.21.15.43:0/2969790941 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4fcc0cecf0 0x7f4fcc14d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.516+0000 7f4fd333b640 1 -- 172.21.15.43:0/2969790941 >> 172.21.15.43:0/2969790941 conn(0x7f4fcc09cd20 msgr2=0x7f4fcc1605c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:36.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.516+0000 7f4fd333b640 1 -- 172.21.15.43:0/2969790941 shutdown_connections 2024-04-25T08:57:36.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.516+0000 7f4fd333b640 1 -- 172.21.15.43:0/2969790941 wait complete. 2024-04-25T08:57:36.516 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d damage rm 2580061446 2024-04-25T08:57:36.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.631+0000 7fd0ab2e1640 1 Processor -- start 2024-04-25T08:57:36.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.631+0000 7fd0ab2e1640 1 -- start start 2024-04-25T08:57:36.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.632+0000 7fd0ab2e1640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd0a406f9a0 0x7fd0a415cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.632+0000 7fd0ab2e1640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fd0a40d0af0 0x7fd0a40d0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.632+0000 7fd0ab2e1640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0a40cf7d0 con 0x7fd0a406b6d0 2024-04-25T08:57:36.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.632+0000 7fd0ab2e1640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0a415c070 con 0x7fd0a40d0af0 2024-04-25T08:57:36.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.632+0000 7fd0ab2e1640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a40cfa40 con 0x7fd0a406f9a0 2024-04-25T08:57:36.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.632+0000 7fd0a987e640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fd0a40d0af0 0x7fd0a40d0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.633+0000 7fd0a907d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd0a406f9a0 0x7fd0a415cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.633+0000 7fd0a987e640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fd0a40d0af0 0x7fd0a40d0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:50352/0 (socket says 172.21.15.43:50352) 2024-04-25T08:57:36.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.633+0000 7fd0a987e640 1 -- 172.21.15.43:0/516328892 learned_addr learned my addr 172.21.15.43:0/516328892 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:36.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.633+0000 7fd0a907d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd0a406f9a0 0x7fd0a415cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:52268/0 (socket says 172.21.15.43:52268) 2024-04-25T08:57:36.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.633+0000 7fd0a987e640 1 -- 172.21.15.43:0/516328892 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd0a406f9a0 msgr2=0x7fd0a415cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:36.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.633+0000 7fd0a987e640 1 --2- 172.21.15.43:0/516328892 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd0a406f9a0 0x7fd0a415cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.633+0000 7fd0a987e640 1 -- 172.21.15.43:0/516328892 >> v1:172.21.15.193:6789/0 conn(0x7fd0a406b6d0 legacy=0x7fd0a40cf0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:36.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.633+0000 7fd0a987e640 1 -- 172.21.15.43:0/516328892 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a415c310 con 0x7fd0a40d0af0 2024-04-25T08:57:36.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.633+0000 7fd0a907d640 1 --2- 172.21.15.43:0/516328892 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd0a406f9a0 0x7fd0a415cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:57:36.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.634+0000 7fd0a987e640 1 --2- 172.21.15.43:0/516328892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0a40d0af0 0x7fd0a40d0fa0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd094009e50 tx=0x7fd09402a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f797736811f2441 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:36.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.634+0000 7fd0a3fff640 1 -- 172.21.15.43:0/516328892 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd094036070 con 0x7fd0a40d0af0 2024-04-25T08:57:36.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.634+0000 7fd0a3fff640 1 -- 172.21.15.43:0/516328892 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd094031600 con 0x7fd0a40d0af0 2024-04-25T08:57:36.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.634+0000 7fd0a3fff640 1 -- 172.21.15.43:0/516328892 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0940319d0 con 0x7fd0a40d0af0 2024-04-25T08:57:36.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.634+0000 7fd0ab2e1640 1 -- 172.21.15.43:0/516328892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0a40d0af0 msgr2=0x7fd0a40d0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:36.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.634+0000 7fd0ab2e1640 1 --2- 172.21.15.43:0/516328892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0a40d0af0 0x7fd0a40d0fa0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd094009e50 tx=0x7fd09402a730 comp rx=0 tx=0).stop 2024-04-25T08:57:36.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.634+0000 7fd0ab2e1640 1 -- 172.21.15.43:0/516328892 shutdown_connections 2024-04-25T08:57:36.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.634+0000 7fd0ab2e1640 1 --2- 172.21.15.43:0/516328892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0a40d0af0 0x7fd0a40d0fa0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.634+0000 7fd0ab2e1640 1 --2- 172.21.15.43:0/516328892 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd0a406f9a0 0x7fd0a415cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.634+0000 7fd0ab2e1640 1 -- 172.21.15.43:0/516328892 >> 172.21.15.43:0/516328892 conn(0x7fd0a406c060 msgr2=0x7fd0a406c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:36.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.634+0000 7fd0ab2e1640 1 -- 172.21.15.43:0/516328892 shutdown_connections 2024-04-25T08:57:36.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.634+0000 7fd0ab2e1640 1 -- 172.21.15.43:0/516328892 wait complete. 2024-04-25T08:57:36.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.636+0000 7fd0ab2e1640 1 Processor -- start 2024-04-25T08:57:36.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.636+0000 7fd0ab2e1640 1 -- start start 2024-04-25T08:57:36.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.636+0000 7fd0ab2e1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0a406b6d0 0x7fd0a4157f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.636+0000 7fd0ab2e1640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd0a406f9a0 0x7fd0a4157930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.636+0000 7fd0ab2e1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0a40d0af0 0x7fd0a40aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.636+0000 7fd0ab2e1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a40cfd20 con 0x7fd0a406b6d0 2024-04-25T08:57:36.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.636+0000 7fd0ab2e1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a406fd80 con 0x7fd0a40d0af0 2024-04-25T08:57:36.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.636+0000 7fd0ab2e1640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a40cfea0 con 0x7fd0a406f9a0 2024-04-25T08:57:36.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.637+0000 7fd0a907d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0a406b6d0 0x7fd0a4157f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.637+0000 7fd0a907d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0a406b6d0 0x7fd0a4157f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58184/0 (socket says 172.21.15.43:58184) 2024-04-25T08:57:36.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.637+0000 7fd0a907d640 1 -- 172.21.15.43:0/43460568 learned_addr learned my addr 172.21.15.43:0/43460568 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:36.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.637+0000 7fd0a987e640 1 --2- 172.21.15.43:0/43460568 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0a40d0af0 0x7fd0a40aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.637+0000 7fd0a887c640 1 --2- 172.21.15.43:0/43460568 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd0a406f9a0 0x7fd0a4157930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.637+0000 7fd0a907d640 1 -- 172.21.15.43:0/43460568 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd0a406f9a0 msgr2=0x7fd0a4157930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:36.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.637+0000 7fd0a907d640 1 --2- 172.21.15.43:0/43460568 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd0a406f9a0 0x7fd0a4157930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.637+0000 7fd0a907d640 1 -- 172.21.15.43:0/43460568 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0a40d0af0 msgr2=0x7fd0a40aecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:36.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.637+0000 7fd0a907d640 1 --2- 172.21.15.43:0/43460568 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0a40d0af0 0x7fd0a40aecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.637+0000 7fd0a907d640 1 -- 172.21.15.43:0/43460568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a41f3550 con 0x7fd0a406b6d0 2024-04-25T08:57:36.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.637+0000 7fd0a987e640 1 --2- 172.21.15.43:0/43460568 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0a40d0af0 0x7fd0a40aecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:57:36.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.637+0000 7fd0a907d640 1 --2- 172.21.15.43:0/43460568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0a406b6d0 0x7fd0a4157f20 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00ebe0 tx=0x7fd09c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:36.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.637+0000 7fd0a1ffb640 1 -- 172.21.15.43:0/43460568 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09c017070 con 0x7fd0a406b6d0 2024-04-25T08:57:36.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.637+0000 7fd0a1ffb640 1 -- 172.21.15.43:0/43460568 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd09c00cc30 con 0x7fd0a406b6d0 2024-04-25T08:57:36.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.637+0000 7fd0a1ffb640 1 -- 172.21.15.43:0/43460568 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09c013470 con 0x7fd0a406b6d0 2024-04-25T08:57:36.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.637+0000 7fd0ab2e1640 1 -- 172.21.15.43:0/43460568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0a40af350 con 0x7fd0a406b6d0 2024-04-25T08:57:36.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.638+0000 7fd0ab2e1640 1 -- 172.21.15.43:0/43460568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0a40b3a60 con 0x7fd0a406b6d0 2024-04-25T08:57:36.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.639+0000 7fd0a1ffb640 1 -- 172.21.15.43:0/43460568 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd09c01a070 con 0x7fd0a406b6d0 2024-04-25T08:57:36.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.640+0000 7fd0a1ffb640 1 --2- 172.21.15.43:0/43460568 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd09006b4d0 0x7fd09006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.640+0000 7fd0a1ffb640 1 -- 172.21.15.43:0/43460568 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd09c08a450 con 0x7fd0a406b6d0 2024-04-25T08:57:36.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.640+0000 7fd0a887c640 1 --2- 172.21.15.43:0/43460568 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd09006b4d0 0x7fd09006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.640+0000 7fd0a887c640 1 --2- 172.21.15.43:0/43460568 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd09006b4d0 0x7fd09006d990 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd0a40aec50 tx=0x7fd098009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:36.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.640+0000 7fd0adfe9740 1 Processor -- start 2024-04-25T08:57:36.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.641+0000 7fd0adfe9740 1 -- start start 2024-04-25T08:57:36.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.641+0000 7fd0adfe9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f1fc1e7f80 0x55f1fc1ea790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.641+0000 7fd0adfe9740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f1fc1e8360 0x55f1fc1eacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.641+0000 7fd0adfe9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1fc1babe0 0x55f1fc1be3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.641+0000 7fd0adfe9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1fc123ac0 con 0x55f1fc1babe0 2024-04-25T08:57:36.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.641+0000 7fd0adfe9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1fc0ccdb0 con 0x55f1fc1e7f80 2024-04-25T08:57:36.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.641+0000 7fd0adfe9740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f1fc11fb60 con 0x55f1fc1e8360 2024-04-25T08:57:36.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.641+0000 7fd0a987e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1fc1babe0 0x55f1fc1be3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.641+0000 7fd0a987e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1fc1babe0 0x55f1fc1be3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58196/0 (socket says 172.21.15.43:58196) 2024-04-25T08:57:36.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.641+0000 7fd0a987e640 1 -- 172.21.15.43:0/689849214 learned_addr learned my addr 172.21.15.43:0/689849214 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:36.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.641+0000 7fd0a907d640 1 --2- 172.21.15.43:0/689849214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f1fc1e7f80 0x55f1fc1ea790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.641+0000 7fd0a887c640 1 --2- 172.21.15.43:0/689849214 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f1fc1e8360 0x55f1fc1eacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.642+0000 7fd0a987e640 1 -- 172.21.15.43:0/689849214 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f1fc1e8360 msgr2=0x55f1fc1eacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:36.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.642+0000 7fd0a987e640 1 --2- 172.21.15.43:0/689849214 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f1fc1e8360 0x55f1fc1eacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.642+0000 7fd0a987e640 1 -- 172.21.15.43:0/689849214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f1fc1e7f80 msgr2=0x55f1fc1ea790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:36.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.642+0000 7fd0a987e640 1 --2- 172.21.15.43:0/689849214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f1fc1e7f80 0x55f1fc1ea790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.642+0000 7fd0a987e640 1 -- 172.21.15.43:0/689849214 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1fbf04710 con 0x55f1fc1babe0 2024-04-25T08:57:36.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.642+0000 7fd0a887c640 1 --2- 172.21.15.43:0/689849214 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f1fc1e8360 0x55f1fc1eacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:36.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.642+0000 7fd0a987e640 1 --2- 172.21.15.43:0/689849214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1fc1babe0 0x55f1fc1be3f0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fd094034040 tx=0x7fd09402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=916076aae29e2f18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:36.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.642+0000 7fd08affd640 1 -- 172.21.15.43:0/689849214 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd094036040 con 0x55f1fc1babe0 2024-04-25T08:57:36.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.642+0000 7fd08affd640 1 -- 172.21.15.43:0/689849214 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd094037040 con 0x55f1fc1babe0 2024-04-25T08:57:36.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.642+0000 7fd08affd640 1 -- 172.21.15.43:0/689849214 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd094031d60 con 0x55f1fc1babe0 2024-04-25T08:57:36.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.643+0000 7fd0adfe9740 1 -- 172.21.15.43:0/689849214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1fc1babe0 msgr2=0x55f1fc1be3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:36.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.643+0000 7fd0adfe9740 1 --2- 172.21.15.43:0/689849214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1fc1babe0 0x55f1fc1be3f0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fd094034040 tx=0x7fd09402e000 comp rx=0 tx=0).stop 2024-04-25T08:57:36.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.643+0000 7fd0adfe9740 1 -- 172.21.15.43:0/689849214 shutdown_connections 2024-04-25T08:57:36.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.643+0000 7fd0adfe9740 1 --2- 172.21.15.43:0/689849214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1fc1babe0 0x55f1fc1be3f0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.643+0000 7fd0adfe9740 1 --2- 172.21.15.43:0/689849214 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f1fc1e8360 0x55f1fc1eacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.643+0000 7fd0adfe9740 1 --2- 172.21.15.43:0/689849214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f1fc1e7f80 0x55f1fc1ea790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.643+0000 7fd0adfe9740 1 -- 172.21.15.43:0/689849214 >> 172.21.15.43:0/689849214 conn(0x55f1fc1bce50 msgr2=0x55f1fc205790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:36.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.643+0000 7fd0adfe9740 1 -- 172.21.15.43:0/689849214 shutdown_connections 2024-04-25T08:57:36.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.643+0000 7fd0adfe9740 1 -- 172.21.15.43:0/689849214 wait complete. 2024-04-25T08:57:36.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.643+0000 7fd0adfe9740 1 -- start start 2024-04-25T08:57:36.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.643+0000 7fd0adfe9740 1 Processor -- start 2024-04-25T08:57:36.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.644+0000 7fd0adfe9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1fc1e7f80 0x55f1fc1e1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.644+0000 7fd0adfe9740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f1fc1e8360 0x55f1fc1e3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.644+0000 7fd0adfe9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f1fc1e3fd0 0x55f1fc204580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.644+0000 7fd0adfe9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1fc1c0000 con 0x55f1fc1e7f80 2024-04-25T08:57:36.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.644+0000 7fd0adfe9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1fc0c24e0 con 0x55f1fc1e3fd0 2024-04-25T08:57:36.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.644+0000 7fd0adfe9740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f1fc1c02e0 con 0x55f1fc1e8360 2024-04-25T08:57:36.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.644+0000 7fd0adfe9740 10 client.0 _ll_register_callbacks cb 0x55f1fbe37af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:36.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.644+0000 7fd0a907d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1fc1e7f80 0x55f1fc1e1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.644+0000 7fd0a907d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1fc1e7f80 0x55f1fc1e1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58198/0 (socket says 172.21.15.43:58198) 2024-04-25T08:57:36.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.644+0000 7fd0a907d640 1 -- 172.21.15.43:0/2885454516 learned_addr learned my addr 172.21.15.43:0/2885454516 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:36.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.644+0000 7fd0a987e640 1 --2- 172.21.15.43:0/2885454516 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f1fc1e3fd0 0x55f1fc204580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.644+0000 7fd0a887c640 1 --2- 172.21.15.43:0/2885454516 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f1fc1e8360 0x55f1fc1e3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.645+0000 7fd0a907d640 1 -- 172.21.15.43:0/2885454516 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f1fc1e8360 msgr2=0x55f1fc1e3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:36.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.645+0000 7fd0a907d640 1 --2- 172.21.15.43:0/2885454516 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f1fc1e8360 0x55f1fc1e3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.645+0000 7fd0a907d640 1 -- 172.21.15.43:0/2885454516 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f1fc1e3fd0 msgr2=0x55f1fc204580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:36.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.645+0000 7fd0a907d640 1 --2- 172.21.15.43:0/2885454516 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f1fc1e3fd0 0x55f1fc204580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.645+0000 7fd0a907d640 1 -- 172.21.15.43:0/2885454516 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f1fbf64ea0 con 0x55f1fc1e7f80 2024-04-25T08:57:36.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.645+0000 7fd0a887c640 1 --2- 172.21.15.43:0/2885454516 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f1fc1e8360 0x55f1fc1e3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:36.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.645+0000 7fd0a907d640 1 --2- 172.21.15.43:0/2885454516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1fc1e7f80 0x55f1fc1e1d60 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fd09c088910 tx=0x7fd09c0585d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04e30c21990b3d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:36.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.645+0000 7fd07ffff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:57:36.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.645+0000 7fd07ffff640 1 -- 172.21.15.43:0/2885454516 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09c070020 con 0x55f1fc1e7f80 2024-04-25T08:57:36.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.645+0000 7fd07ffff640 1 -- 172.21.15.43:0/2885454516 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd09c060e40 con 0x55f1fc1e7f80 2024-04-25T08:57:36.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.645+0000 7fd0adfe9740 1 -- 172.21.15.43:0/2885454516 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd094037040 con 0x55f1fc1e7f80 2024-04-25T08:57:36.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.645+0000 7fd07ffff640 1 -- 172.21.15.43:0/2885454516 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd09c021040 con 0x55f1fc1e7f80 2024-04-25T08:57:36.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.645+0000 7fd0adfe9740 10 client.6339 fetch_fsmap learned FSMap version 24 2024-04-25T08:57:36.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.645+0000 7fd0adfe9740 1 -- 172.21.15.43:0/2885454516 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55f1fbdf8000 con 0x55f1fc1e7f80 2024-04-25T08:57:36.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.645+0000 7fd07ffff640 1 -- 172.21.15.43:0/2885454516 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd094037040 con 0x55f1fc1e7f80 2024-04-25T08:57:36.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.645+0000 7fd07ffff640 1 -- 172.21.15.43:0/2885454516 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd09c06cb80 con 0x55f1fc1e7f80 2024-04-25T08:57:36.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.645+0000 7fd0adfe9740 10 client.6339 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:57:36.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.645+0000 7fd0adfe9740 10 client.6339 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T08:57:36.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.645+0000 7fd0adfe9740 20 client.6339 populate_metadata read hostname 'smithi043' 2024-04-25T08:57:36.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.645+0000 7fd0adfe9740 1 --2- 172.21.15.43:0/2885454516 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55f1fc1cf120 0x55f1fc208690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.645+0000 7fd0adfe9740 4 client.6339 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:57:36.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.645+0000 7fd0adfe9740 1 -- 172.21.15.43:0/2885454516 --> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f1fc131000 con 0x55f1fc1cf120 2024-04-25T08:57:36.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.646+0000 7fd0a887c640 1 --2- 172.21.15.43:0/2885454516 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55f1fc1cf120 0x55f1fc208690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.646+0000 7fd07ffff640 10 client.6339 ms_handle_connect on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:36.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.646+0000 7fd0a887c640 1 --2- 172.21.15.43:0/2885454516 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55f1fc1cf120 0x55f1fc208690 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a234ff52022271e server_cookie=6367205b815fdb28 in_seq=0 out_seq=0 2024-04-25T08:57:36.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.646+0000 7fd07ffff640 1 -- 172.21.15.43:0/2885454516 <== mds.0 v2:172.21.15.193:6836/4164634380 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f1fc131000 con 0x55f1fc1cf120 2024-04-25T08:57:36.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.646+0000 7fd07ffff640 10 client.6339 handle_command_reply: tid=0 2024-04-25T08:57:36.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.646+0000 7fd0adfe9740 1 client.6339 shutdown 2024-04-25T08:57:36.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.647+0000 7fd0adfe9740 1 -- 172.21.15.43:0/2885454516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1fc1e7f80 msgr2=0x55f1fc1e1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:36.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.647+0000 7fd0adfe9740 1 --2- 172.21.15.43:0/2885454516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1fc1e7f80 0x55f1fc1e1d60 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fd09c088910 tx=0x7fd09c0585d0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.647+0000 7fd0adfe9740 1 -- 172.21.15.43:0/2885454516 shutdown_connections 2024-04-25T08:57:36.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.647+0000 7fd0adfe9740 1 --2- 172.21.15.43:0/2885454516 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55f1fc1cf120 0x55f1fc208690 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.647+0000 7fd0adfe9740 1 --2- 172.21.15.43:0/2885454516 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f1fc1e3fd0 0x55f1fc204580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.647+0000 7fd07ffff640 0 client.6339 ms_handle_reset on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:36.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.647+0000 7fd0adfe9740 1 --2- 172.21.15.43:0/2885454516 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f1fc1e8360 0x55f1fc1e3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.647+0000 7fd0adfe9740 1 --2- 172.21.15.43:0/2885454516 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1fc1e7f80 0x55f1fc1e1d60 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.647+0000 7fd0adfe9740 1 -- 172.21.15.43:0/2885454516 >> 172.21.15.43:0/2885454516 conn(0x55f1fc1babe0 msgr2=0x55f1fc1bb890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:36.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.647+0000 7fd0adfe9740 1 -- 172.21.15.43:0/2885454516 shutdown_connections 2024-04-25T08:57:36.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.647+0000 7fd0adfe9740 1 -- 172.21.15.43:0/2885454516 wait complete. 2024-04-25T08:57:36.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.647+0000 7fd0adfe9740 20 client.6339 trim_cache size 0 max 16384 2024-04-25T08:57:36.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.659+0000 7fd0adfe9740 1 Processor -- start 2024-04-25T08:57:36.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.659+0000 7fd0adfe9740 1 -- start start 2024-04-25T08:57:36.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.660+0000 7fd0adfe9740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f1fc1e3fd0 0x55f1fc1e43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.660+0000 7fd0adfe9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f1fc1e7f80 0x55f1fc1e48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.660+0000 7fd0adfe9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1fc1e8360 0x55f1fc208610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.660+0000 7fd0adfe9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a41ff120 con 0x55f1fc1e8360 2024-04-25T08:57:36.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.660+0000 7fd0adfe9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a40b5090 con 0x55f1fc1e7f80 2024-04-25T08:57:36.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.660+0000 7fd0adfe9740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a40b5690 con 0x55f1fc1e3fd0 2024-04-25T08:57:36.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.660+0000 7fd0a987e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1fc1e8360 0x55f1fc208610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.660+0000 7fd0a987e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1fc1e8360 0x55f1fc208610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58208/0 (socket says 172.21.15.43:58208) 2024-04-25T08:57:36.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.660+0000 7fd0a987e640 1 -- 172.21.15.43:0/4143859859 learned_addr learned my addr 172.21.15.43:0/4143859859 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:36.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.660+0000 7fd0a887c640 1 --2- 172.21.15.43:0/4143859859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f1fc1e7f80 0x55f1fc1e48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.660+0000 7fd0a987e640 1 -- 172.21.15.43:0/4143859859 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f1fc1e3fd0 msgr2=0x55f1fc1e43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:36.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.660+0000 7fd0a907d640 1 --2- 172.21.15.43:0/4143859859 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f1fc1e3fd0 0x55f1fc1e43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.660+0000 7fd0a987e640 1 --2- 172.21.15.43:0/4143859859 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f1fc1e3fd0 0x55f1fc1e43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.660+0000 7fd0a987e640 1 -- 172.21.15.43:0/4143859859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f1fc1e7f80 msgr2=0x55f1fc1e48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:36.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.660+0000 7fd0a987e640 1 --2- 172.21.15.43:0/4143859859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f1fc1e7f80 0x55f1fc1e48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.660+0000 7fd0a987e640 1 -- 172.21.15.43:0/4143859859 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f1fc205e00 con 0x55f1fc1e8360 2024-04-25T08:57:36.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.660+0000 7fd0a887c640 1 --2- 172.21.15.43:0/4143859859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f1fc1e7f80 0x55f1fc1e48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:36.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.660+0000 7fd0a907d640 1 --2- 172.21.15.43:0/4143859859 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f1fc1e3fd0 0x55f1fc1e43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:36.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.660+0000 7fd0a987e640 1 --2- 172.21.15.43:0/4143859859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1fc1e8360 0x55f1fc208610 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fd09402b060 tx=0x7fd09402f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6593443db8bb67c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:36.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.660+0000 7fd08a7fc640 1 -- 172.21.15.43:0/4143859859 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd09404a070 con 0x55f1fc1e8360 2024-04-25T08:57:36.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.660+0000 7fd08a7fc640 1 -- 172.21.15.43:0/4143859859 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd094031600 con 0x55f1fc1e8360 2024-04-25T08:57:36.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.660+0000 7fd08a7fc640 1 -- 172.21.15.43:0/4143859859 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd094036040 con 0x55f1fc1e8360 2024-04-25T08:57:36.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.661+0000 7fd0adfe9740 1 -- 172.21.15.43:0/4143859859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1fc1e8360 msgr2=0x55f1fc208610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:36.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.661+0000 7fd0adfe9740 1 --2- 172.21.15.43:0/4143859859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1fc1e8360 0x55f1fc208610 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fd09402b060 tx=0x7fd09402f000 comp rx=0 tx=0).stop 2024-04-25T08:57:36.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.661+0000 7fd0adfe9740 1 -- 172.21.15.43:0/4143859859 shutdown_connections 2024-04-25T08:57:36.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.661+0000 7fd0adfe9740 1 --2- 172.21.15.43:0/4143859859 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1fc1e8360 0x55f1fc208610 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.661+0000 7fd0adfe9740 1 --2- 172.21.15.43:0/4143859859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f1fc1e7f80 0x55f1fc1e48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.661+0000 7fd0adfe9740 1 --2- 172.21.15.43:0/4143859859 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f1fc1e3fd0 0x55f1fc1e43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.661+0000 7fd0adfe9740 1 -- 172.21.15.43:0/4143859859 >> 172.21.15.43:0/4143859859 conn(0x55f1fc1babe0 msgr2=0x55f1fc1ce5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:36.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.661+0000 7fd0adfe9740 1 -- 172.21.15.43:0/4143859859 shutdown_connections 2024-04-25T08:57:36.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.661+0000 7fd0adfe9740 1 -- 172.21.15.43:0/4143859859 wait complete. 2024-04-25T08:57:36.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.661+0000 7fd0adfe9740 1 -- start start 2024-04-25T08:57:36.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.661+0000 7fd0adfe9740 1 Processor -- start 2024-04-25T08:57:36.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.662+0000 7fd0adfe9740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f1fc1e3fd0 0x55f1fc1fb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.662+0000 7fd0adfe9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f1fc1e7f80 0x55f1fc1fbd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.662+0000 7fd0adfe9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1fc1e8360 0x55f1fc1ed670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.662+0000 7fd0adfe9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1fc1cefa0 con 0x55f1fc1e8360 2024-04-25T08:57:36.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.662+0000 7fd0adfe9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f1fc2095f0 con 0x55f1fc1e7f80 2024-04-25T08:57:36.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.662+0000 7fd0adfe9740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f1fc1cf510 con 0x55f1fc1e3fd0 2024-04-25T08:57:36.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.662+0000 7fd0adfe9740 10 client.0 _ll_register_callbacks cb 0x7fd09402af50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:57:36.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.662+0000 7fd0a987e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1fc1e8360 0x55f1fc1ed670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.662+0000 7fd0a987e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1fc1e8360 0x55f1fc1ed670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58210/0 (socket says 172.21.15.43:58210) 2024-04-25T08:57:36.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.662+0000 7fd0a987e640 1 -- 172.21.15.43:0/1291760180 learned_addr learned my addr 172.21.15.43:0/1291760180 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:36.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.663+0000 7fd0a887c640 1 --2- 172.21.15.43:0/1291760180 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f1fc1e7f80 0x55f1fc1fbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.663+0000 7fd0a907d640 1 --2- 172.21.15.43:0/1291760180 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f1fc1e3fd0 0x55f1fc1fb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.663+0000 7fd0a987e640 1 -- 172.21.15.43:0/1291760180 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f1fc1e3fd0 msgr2=0x55f1fc1fb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:36.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.663+0000 7fd0a987e640 1 --2- 172.21.15.43:0/1291760180 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f1fc1e3fd0 0x55f1fc1fb810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.663+0000 7fd0a987e640 1 -- 172.21.15.43:0/1291760180 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f1fc1e7f80 msgr2=0x55f1fc1fbd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:36.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.663+0000 7fd0a987e640 1 --2- 172.21.15.43:0/1291760180 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f1fc1e7f80 0x55f1fc1fbd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.663+0000 7fd0a987e640 1 -- 172.21.15.43:0/1291760180 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f1fc205410 con 0x55f1fc1e8360 2024-04-25T08:57:36.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.663+0000 7fd0a987e640 1 --2- 172.21.15.43:0/1291760180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1fc1e8360 0x55f1fc1ed670 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fd094009e50 tx=0x7fd094004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e12514a911f35c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:36.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.663+0000 7fd07ffff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:57:36.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.663+0000 7fd07ffff640 1 -- 172.21.15.43:0/1291760180 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd094053020 con 0x55f1fc1e8360 2024-04-25T08:57:36.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.663+0000 7fd07ffff640 1 -- 172.21.15.43:0/1291760180 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd094003b90 con 0x55f1fc1e8360 2024-04-25T08:57:36.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.663+0000 7fd0adfe9740 1 -- 172.21.15.43:0/1291760180 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd094031600 con 0x55f1fc1e8360 2024-04-25T08:57:36.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.663+0000 7fd07ffff640 1 -- 172.21.15.43:0/1291760180 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd09404a050 con 0x55f1fc1e8360 2024-04-25T08:57:36.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.663+0000 7fd0adfe9740 10 client.6345 fetch_fsmap learned FSMap version 24 2024-04-25T08:57:36.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.663+0000 7fd0adfe9740 1 -- 172.21.15.43:0/1291760180 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x55f1fc208d70 con 0x55f1fc1e8360 2024-04-25T08:57:36.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.663+0000 7fd07ffff640 1 -- 172.21.15.43:0/1291760180 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd094031600 con 0x55f1fc1e8360 2024-04-25T08:57:36.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.663+0000 7fd07ffff640 1 -- 172.21.15.43:0/1291760180 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 24) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd094008030 con 0x55f1fc1e8360 2024-04-25T08:57:36.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.663+0000 7fd0adfe9740 10 client.6345 fetch_fsmap finished waiting for FSMap version 24 2024-04-25T08:57:36.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.663+0000 7fd0adfe9740 10 client.6345 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T08:57:36.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.663+0000 7fd0adfe9740 20 client.6345 populate_metadata read hostname 'smithi043' 2024-04-25T08:57:36.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.663+0000 7fd0adfe9740 1 --2- 172.21.15.43:0/1291760180 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55f1fc1e4950 0x55f1fc1edbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.663+0000 7fd0adfe9740 4 client.6345 mds_command: new command op to 5709 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 2580061446}] 2024-04-25T08:57:36.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.663+0000 7fd0adfe9740 1 -- 172.21.15.43:0/1291760180 --> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] -- command(tid 0: {"prefix": "damage rm", "damage_id": 2580061446}) v1 -- 0x55f1fc20de80 con 0x55f1fc1e4950 2024-04-25T08:57:36.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.664+0000 7fd0a907d640 1 --2- 172.21.15.43:0/1291760180 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55f1fc1e4950 0x55f1fc1edbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.664+0000 7fd07ffff640 10 client.6345 ms_handle_connect on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:36.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.664+0000 7fd0a907d640 1 --2- 172.21.15.43:0/1291760180 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55f1fc1e4950 0x55f1fc1edbb0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a9ca4bb5994f43e server_cookie=fa779398971bc848 in_seq=0 out_seq=0 2024-04-25T08:57:36.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.664+0000 7fd07ffff640 1 -- 172.21.15.43:0/1291760180 <== mds.0 v2:172.21.15.193:6836/4164634380 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x55f1fc20de80 con 0x55f1fc1e4950 2024-04-25T08:57:36.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.664+0000 7fd07ffff640 10 client.6345 handle_command_reply: tid=0 2024-04-25T08:57:36.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.664+0000 7fd0adfe9740 1 client.6345 shutdown 2024-04-25T08:57:36.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.664+0000 7fd0adfe9740 1 -- 172.21.15.43:0/1291760180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1fc1e8360 msgr2=0x55f1fc1ed670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:36.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.664+0000 7fd0adfe9740 1 --2- 172.21.15.43:0/1291760180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1fc1e8360 0x55f1fc1ed670 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fd094009e50 tx=0x7fd094004ab0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.664+0000 7fd0adfe9740 1 -- 172.21.15.43:0/1291760180 shutdown_connections 2024-04-25T08:57:36.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.664+0000 7fd0adfe9740 1 --2- 172.21.15.43:0/1291760180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f1fc1e8360 0x55f1fc1ed670 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.664+0000 7fd0adfe9740 1 --2- 172.21.15.43:0/1291760180 >> [v2:172.21.15.193:6836/4164634380,v1:172.21.15.193:6837/4164634380] conn(0x55f1fc1e4950 0x55f1fc1edbb0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.664+0000 7fd0adfe9740 1 --2- 172.21.15.43:0/1291760180 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f1fc1e7f80 0x55f1fc1fbd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.664+0000 7fd0adfe9740 1 --2- 172.21.15.43:0/1291760180 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f1fc1e3fd0 0x55f1fc1fb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.664+0000 7fd07ffff640 0 client.6345 ms_handle_reset on v2:172.21.15.193:6836/4164634380 2024-04-25T08:57:36.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.664+0000 7fd0adfe9740 1 -- 172.21.15.43:0/1291760180 >> 172.21.15.43:0/1291760180 conn(0x55f1fc1babe0 msgr2=0x55f1fc228340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:36.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.664+0000 7fd0adfe9740 1 -- 172.21.15.43:0/1291760180 shutdown_connections 2024-04-25T08:57:36.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.664+0000 7fd0adfe9740 1 -- 172.21.15.43:0/1291760180 wait complete. 2024-04-25T08:57:36.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.665+0000 7fd0adfe9740 20 client.6345 trim_cache size 0 max 16384 2024-04-25T08:57:36.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.665+0000 7fd0ab2e1640 1 -- 172.21.15.43:0/43460568 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd09006b4d0 msgr2=0x7fd09006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:36.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.665+0000 7fd0ab2e1640 1 --2- 172.21.15.43:0/43460568 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd09006b4d0 0x7fd09006d990 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fd0a40aec50 tx=0x7fd098009040 comp rx=0 tx=0).stop 2024-04-25T08:57:36.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.665+0000 7fd0ab2e1640 1 -- 172.21.15.43:0/43460568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0a406b6d0 msgr2=0x7fd0a4157f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:36.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.665+0000 7fd0ab2e1640 1 --2- 172.21.15.43:0/43460568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0a406b6d0 0x7fd0a4157f20 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fd09c00ebe0 tx=0x7fd09c00c520 comp rx=0 tx=0).stop 2024-04-25T08:57:36.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.665+0000 7fd0ab2e1640 1 -- 172.21.15.43:0/43460568 shutdown_connections 2024-04-25T08:57:36.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.665+0000 7fd0ab2e1640 1 --2- 172.21.15.43:0/43460568 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd09006b4d0 0x7fd09006d990 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.665+0000 7fd0ab2e1640 1 --2- 172.21.15.43:0/43460568 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd0a40d0af0 0x7fd0a40aecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.665+0000 7fd0ab2e1640 1 --2- 172.21.15.43:0/43460568 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd0a406f9a0 0x7fd0a4157930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.665+0000 7fd0ab2e1640 1 --2- 172.21.15.43:0/43460568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd0a406b6d0 0x7fd0a4157f20 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.665+0000 7fd0ab2e1640 1 -- 172.21.15.43:0/43460568 >> 172.21.15.43:0/43460568 conn(0x7fd0a406c060 msgr2=0x7fd0a406c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:36.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.665+0000 7fd0ab2e1640 1 -- 172.21.15.43:0/43460568 shutdown_connections 2024-04-25T08:57:36.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.665+0000 7fd0ab2e1640 1 -- 172.21.15.43:0/43460568 wait complete. 2024-04-25T08:57:36.665 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2024-04-25T08:57:36.666 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:57:36.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.776+0000 7fe6ecda9640 1 Processor -- start 2024-04-25T08:57:36.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.776+0000 7fe6ecda9640 1 -- start start 2024-04-25T08:57:36.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.777+0000 7fe6ecda9640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe6e815def0 0x7fe6e806be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.777+0000 7fe6ecda9640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fe6e81619f0 0x7fe6e8161e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.777+0000 7fe6ecda9640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6e8162820 con 0x7fe6e8162390 2024-04-25T08:57:36.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.777+0000 7fe6ecda9640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6e80b30d0 con 0x7fe6e81619f0 2024-04-25T08:57:36.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.777+0000 7fe6ecda9640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e80d0e90 con 0x7fe6e815def0 2024-04-25T08:57:36.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.777+0000 7fe6e5d9b640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fe6e81619f0 0x7fe6e8161e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.777+0000 7fe6e659c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe6e815def0 0x7fe6e806be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.777+0000 7fe6e5d9b640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fe6e81619f0 0x7fe6e8161e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:50410/0 (socket says 172.21.15.43:50410) 2024-04-25T08:57:36.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.777+0000 7fe6e659c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe6e815def0 0x7fe6e806be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:52320/0 (socket says 172.21.15.43:52320) 2024-04-25T08:57:36.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.777+0000 7fe6e5d9b640 1 -- 172.21.15.43:0/596931398 learned_addr learned my addr 172.21.15.43:0/596931398 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:36.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.777+0000 7fe6e6d9d640 1 --1- 172.21.15.43:0/596931398 >> v1:172.21.15.193:6789/0 conn(0x7fe6e8162390 0x7fe6e80cfa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:37656/0 (socket says 172.21.15.43:37656) 2024-04-25T08:57:36.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.778+0000 7fe6e659c640 1 -- 172.21.15.43:0/596931398 >> v2:172.21.15.193:3300/0 conn(0x7fe6e81619f0 msgr2=0x7fe6e8161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:36.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.778+0000 7fe6e659c640 1 --2- 172.21.15.43:0/596931398 >> v2:172.21.15.193:3300/0 conn(0x7fe6e81619f0 0x7fe6e8161e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.778+0000 7fe6e659c640 1 -- 172.21.15.43:0/596931398 >> v1:172.21.15.193:6789/0 conn(0x7fe6e8162390 legacy=0x7fe6e80cfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:36.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.778+0000 7fe6e659c640 1 -- 172.21.15.43:0/596931398 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e815e2d0 con 0x7fe6e815def0 2024-04-25T08:57:36.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.778+0000 7fe6e5d9b640 1 --2- 172.21.15.43:0/596931398 >> v2:172.21.15.193:3300/0 conn(0x7fe6e81619f0 0x7fe6e8161e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:57:36.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.779+0000 7fe6e659c640 1 --2- 172.21.15.43:0/596931398 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe6e815def0 0x7fe6e806be50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe6e00097d0 tx=0x7fe6e002a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ae34abf14f894d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:36.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.779+0000 7fe6e559a640 1 -- 172.21.15.43:0/596931398 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e0037070 con 0x7fe6e815def0 2024-04-25T08:57:36.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.779+0000 7fe6e559a640 1 -- 172.21.15.43:0/596931398 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e0032440 con 0x7fe6e815def0 2024-04-25T08:57:36.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.779+0000 7fe6e559a640 1 -- 172.21.15.43:0/596931398 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e0032810 con 0x7fe6e815def0 2024-04-25T08:57:36.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.779+0000 7fe6ecda9640 1 -- 172.21.15.43:0/596931398 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe6e815def0 msgr2=0x7fe6e806be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:36.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.779+0000 7fe6ecda9640 1 --2- 172.21.15.43:0/596931398 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe6e815def0 0x7fe6e806be50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe6e00097d0 tx=0x7fe6e002a1c0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.779+0000 7fe6ecda9640 1 -- 172.21.15.43:0/596931398 shutdown_connections 2024-04-25T08:57:36.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.779+0000 7fe6ecda9640 1 --2- 172.21.15.43:0/596931398 >> v2:172.21.15.193:3300/0 conn(0x7fe6e81619f0 0x7fe6e8161e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.779+0000 7fe6ecda9640 1 --2- 172.21.15.43:0/596931398 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe6e815def0 0x7fe6e806be50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.779+0000 7fe6ecda9640 1 -- 172.21.15.43:0/596931398 >> 172.21.15.43:0/596931398 conn(0x7fe6e80b32e0 msgr2=0x7fe6e80b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:36.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.779+0000 7fe6ecda9640 1 -- 172.21.15.43:0/596931398 shutdown_connections 2024-04-25T08:57:36.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.779+0000 7fe6ecda9640 1 -- 172.21.15.43:0/596931398 wait complete. 2024-04-25T08:57:36.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.781+0000 7fe6ecda9640 1 Processor -- start 2024-04-25T08:57:36.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.781+0000 7fe6ecda9640 1 -- start start 2024-04-25T08:57:36.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.782+0000 7fe6ecda9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6e815def0 0x7fe6e809f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.782+0000 7fe6ecda9640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe6e81619f0 0x7fe6e80b5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.782+0000 7fe6ecda9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6e8162390 0x7fe6e80c7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.782+0000 7fe6ecda9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e80b1300 con 0x7fe6e8162390 2024-04-25T08:57:36.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.782+0000 7fe6ecda9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e806c390 con 0x7fe6e815def0 2024-04-25T08:57:36.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.782+0000 7fe6ecda9640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e80b15e0 con 0x7fe6e81619f0 2024-04-25T08:57:36.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.782+0000 7fe6e6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6e8162390 0x7fe6e80c7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.782+0000 7fe6e6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6e8162390 0x7fe6e80c7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58212/0 (socket says 172.21.15.43:58212) 2024-04-25T08:57:36.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.782+0000 7fe6e6d9d640 1 -- 172.21.15.43:0/1676088825 learned_addr learned my addr 172.21.15.43:0/1676088825 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:36.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.782+0000 7fe6e659c640 1 --2- 172.21.15.43:0/1676088825 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6e815def0 0x7fe6e809f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.782+0000 7fe6e5d9b640 1 --2- 172.21.15.43:0/1676088825 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe6e81619f0 0x7fe6e80b5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.782+0000 7fe6e659c640 1 -- 172.21.15.43:0/1676088825 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe6e81619f0 msgr2=0x7fe6e80b5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:36.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.782+0000 7fe6e659c640 1 --2- 172.21.15.43:0/1676088825 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe6e81619f0 0x7fe6e80b5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.782+0000 7fe6e659c640 1 -- 172.21.15.43:0/1676088825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6e8162390 msgr2=0x7fe6e80c7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:36.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.782+0000 7fe6e659c640 1 --2- 172.21.15.43:0/1676088825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6e8162390 0x7fe6e80c7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:36.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.782+0000 7fe6e659c640 1 -- 172.21.15.43:0/1676088825 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e814c830 con 0x7fe6e815def0 2024-04-25T08:57:36.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.782+0000 7fe6e6d9d640 1 --2- 172.21.15.43:0/1676088825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6e8162390 0x7fe6e80c7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:57:36.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.782+0000 7fe6e659c640 1 --2- 172.21.15.43:0/1676088825 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6e815def0 0x7fe6e809f0b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0031000 tx=0x7fe6e00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:36.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.783+0000 7fe6deffd640 1 -- 172.21.15.43:0/1676088825 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e0041030 con 0x7fe6e815def0 2024-04-25T08:57:36.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.783+0000 7fe6deffd640 1 -- 172.21.15.43:0/1676088825 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e003d4e0 con 0x7fe6e815def0 2024-04-25T08:57:36.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.783+0000 7fe6ecda9640 1 -- 172.21.15.43:0/1676088825 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6e814cb10 con 0x7fe6e815def0 2024-04-25T08:57:36.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.783+0000 7fe6deffd640 1 -- 172.21.15.43:0/1676088825 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e0037040 con 0x7fe6e815def0 2024-04-25T08:57:36.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.783+0000 7fe6ecda9640 1 -- 172.21.15.43:0/1676088825 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6e814d0d0 con 0x7fe6e815def0 2024-04-25T08:57:36.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.784+0000 7fe6ecda9640 1 -- 172.21.15.43:0/1676088825 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6e80b7710 con 0x7fe6e815def0 2024-04-25T08:57:36.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.784+0000 7fe6deffd640 1 -- 172.21.15.43:0/1676088825 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe6e0044070 con 0x7fe6e815def0 2024-04-25T08:57:36.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.785+0000 7fe6deffd640 1 --2- 172.21.15.43:0/1676088825 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe6c806b500 0x7fe6c806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:36.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.785+0000 7fe6deffd640 1 -- 172.21.15.43:0/1676088825 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe6e00ab310 con 0x7fe6e815def0 2024-04-25T08:57:36.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.785+0000 7fe6e5d9b640 1 --2- 172.21.15.43:0/1676088825 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe6c806b500 0x7fe6c806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:36.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.785+0000 7fe6e5d9b640 1 --2- 172.21.15.43:0/1676088825 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe6c806b500 0x7fe6c806d9c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe6d00095c0 tx=0x7fe6d0009210 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:36.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:36.789+0000 7fe6deffd640 1 -- 172.21.15.43:0/1676088825 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe6e0079bb0 con 0x7fe6e815def0 2024-04-25T08:57:37.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.013+0000 7fe6ecda9640 1 -- 172.21.15.43:0/1676088825 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe6e8074ed0 con 0x7fe6e815def0 2024-04-25T08:57:37.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.014+0000 7fe6deffd640 1 -- 172.21.15.43:0/1676088825 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4945 (secure 0 0 0) 0x7fe6e007dc50 con 0x7fe6e815def0 2024-04-25T08:57:37.003 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:57:37.003 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1504450503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1504450503},{"type":"v1","addr":"172.21.15.193:6835","nonce":1504450503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5691,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2657133821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2657133821},{"type":"v1","addr":"172.21.15.43:6834","nonce":2657133821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":5694,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/4271431818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":4271431818},{"type":"v1","addr":"172.21.15.43:6835","nonce":4271431818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:57:01.856132+0000","modified":"2024-04-25T08:57:34.526463+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5709":{"gid":5709,"name":"d","rank":0,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.193:6837/4164634380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4164634380},{"type":"v1","addr":"172.21.15.193:6837","nonce":4164634380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:57:37.003 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 24 2024-04-25T08:57:37.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.016+0000 7fe6ecda9640 1 -- 172.21.15.43:0/1676088825 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe6c806b500 msgr2=0x7fe6c806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:37.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.016+0000 7fe6ecda9640 1 --2- 172.21.15.43:0/1676088825 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe6c806b500 0x7fe6c806d9c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe6d00095c0 tx=0x7fe6d0009210 comp rx=0 tx=0).stop 2024-04-25T08:57:37.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.016+0000 7fe6ecda9640 1 -- 172.21.15.43:0/1676088825 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6e815def0 msgr2=0x7fe6e809f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:37.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.016+0000 7fe6ecda9640 1 --2- 172.21.15.43:0/1676088825 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6e815def0 0x7fe6e809f0b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0031000 tx=0x7fe6e00046c0 comp rx=0 tx=0).stop 2024-04-25T08:57:37.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.016+0000 7fe6ecda9640 1 -- 172.21.15.43:0/1676088825 shutdown_connections 2024-04-25T08:57:37.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.016+0000 7fe6ecda9640 1 --2- 172.21.15.43:0/1676088825 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6e8162390 0x7fe6e80c7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:37.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.016+0000 7fe6ecda9640 1 --2- 172.21.15.43:0/1676088825 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe6e81619f0 0x7fe6e80b5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:37.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.016+0000 7fe6ecda9640 1 --2- 172.21.15.43:0/1676088825 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe6c806b500 0x7fe6c806d9c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:37.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.016+0000 7fe6ecda9640 1 --2- 172.21.15.43:0/1676088825 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6e815def0 0x7fe6e809f0b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:37.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.016+0000 7fe6ecda9640 1 -- 172.21.15.43:0/1676088825 >> 172.21.15.43:0/1676088825 conn(0x7fe6e80b32e0 msgr2=0x7fe6e806e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:37.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.016+0000 7fe6ecda9640 1 -- 172.21.15.43:0/1676088825 shutdown_connections 2024-04-25T08:57:37.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.016+0000 7fe6ecda9640 1 -- 172.21.15.43:0/1676088825 wait complete. 2024-04-25T08:57:37.016 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path / 2024-04-25T08:57:37.131 INFO:teuthology.orchestra.run.smithi193.stdout:{ 2024-04-25T08:57:37.131 INFO:teuthology.orchestra.run.smithi193.stdout: "return_code": 0 2024-04-25T08:57:37.131 INFO:teuthology.orchestra.run.smithi193.stdout:} 2024-04-25T08:57:37.140 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T08:57:37.141 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:57:37.141 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:57:37.188 DEBUG:teuthology.orchestra.run.smithi193:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:57:37.221 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:57:37.221 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:57:37.221 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:57:37.237 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:57:37.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.388+0000 7f017aa3f640 1 Processor -- start 2024-04-25T08:57:37.378 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.388+0000 7f017aa3f640 1 -- start start 2024-04-25T08:57:37.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.389+0000 7f017aa3f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f01740cf670 0x7f0174168d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:37.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.389+0000 7f017aa3f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0174162810 con 0x7f01740c8d00 2024-04-25T08:57:37.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.389+0000 7f017aa3f640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01740d24a0 con 0x7f017415df50 2024-04-25T08:57:37.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.389+0000 7f017aa3f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01740cfa80 con 0x7f01740cf670 2024-04-25T08:57:37.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.389+0000 7f0178fdc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f01740cf670 0x7f0174168d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:37.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.389+0000 7f01737fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f01740c8d00 0x7f0174162100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:52732/0 (socket says 172.21.15.43:52732) 2024-04-25T08:57:37.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.389+0000 7f01737fe640 1 -- 172.21.15.43:0/670100688 learned_addr learned my addr 172.21.15.43:0/670100688 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:37.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.390+0000 7f0178fdc640 1 -- 172.21.15.43:0/670100688 >> v1:172.21.15.193:6789/0 conn(0x7f017415df50 legacy=0x7f01741619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:37.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.390+0000 7f0178fdc640 1 -- 172.21.15.43:0/670100688 >> v1:172.21.15.43:6789/0 conn(0x7f01740c8d00 legacy=0x7f0174162100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:37.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.390+0000 7f0178fdc640 1 -- 172.21.15.43:0/670100688 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01740d2740 con 0x7f01740cf670 2024-04-25T08:57:37.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.390+0000 7f0172ffd640 1 -- 172.21.15.43:0/670100688 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 114207953 0 0) 0x7f0174162810 con 0x7f01740c8d00 2024-04-25T08:57:37.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.390+0000 7f0178fdc640 1 --2- 172.21.15.43:0/670100688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01740cf670 0x7f0174168d20 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f016c009d90 tx=0x7f016c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=194f60d9ef81e8a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:37.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.391+0000 7f0172ffd640 1 -- 172.21.15.43:0/670100688 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016c007740 con 0x7f01740cf670 2024-04-25T08:57:37.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.391+0000 7f0172ffd640 1 -- 172.21.15.43:0/670100688 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f016c0078e0 con 0x7f01740cf670 2024-04-25T08:57:37.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.391+0000 7f0172ffd640 1 -- 172.21.15.43:0/670100688 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f016c007cb0 con 0x7f01740cf670 2024-04-25T08:57:37.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.391+0000 7f017aa3f640 1 -- 172.21.15.43:0/670100688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01740cf670 msgr2=0x7f0174168d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:37.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.391+0000 7f017aa3f640 1 --2- 172.21.15.43:0/670100688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01740cf670 0x7f0174168d20 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f016c009d90 tx=0x7f016c02a6f0 comp rx=0 tx=0).stop 2024-04-25T08:57:37.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.391+0000 7f017aa3f640 1 -- 172.21.15.43:0/670100688 shutdown_connections 2024-04-25T08:57:37.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.391+0000 7f017aa3f640 1 --2- 172.21.15.43:0/670100688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01740cf670 0x7f0174168d20 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:37.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.391+0000 7f017aa3f640 1 -- 172.21.15.43:0/670100688 >> 172.21.15.43:0/670100688 conn(0x7f01740d3a30 msgr2=0x7f01740d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:37.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.391+0000 7f017aa3f640 1 -- 172.21.15.43:0/670100688 shutdown_connections 2024-04-25T08:57:37.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.391+0000 7f017aa3f640 1 -- 172.21.15.43:0/670100688 wait complete. 2024-04-25T08:57:37.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.392+0000 7f017aa3f640 1 Processor -- start 2024-04-25T08:57:37.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.393+0000 7f017aa3f640 1 -- start start 2024-04-25T08:57:37.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.393+0000 7f017aa3f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01740c8d00 0x7f01740c6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:37.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.393+0000 7f017aa3f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01740cf670 0x7f0174156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:37.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.393+0000 7f017aa3f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f017415df50 0x7f0174159910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:37.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.393+0000 7f017aa3f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01740c74f0 con 0x7f01740c8d00 2024-04-25T08:57:37.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.393+0000 7f017aa3f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f017415e330 con 0x7f017415df50 2024-04-25T08:57:37.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.393+0000 7f017aa3f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01740c77d0 con 0x7f01740cf670 2024-04-25T08:57:37.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.393+0000 7f0173fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01740c8d00 0x7f01740c6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:37.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.393+0000 7f0173fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01740c8d00 0x7f01740c6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58238/0 (socket says 172.21.15.43:58238) 2024-04-25T08:57:37.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.393+0000 7f0173fff640 1 -- 172.21.15.43:0/3754108072 learned_addr learned my addr 172.21.15.43:0/3754108072 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:37.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.394+0000 7f01737fe640 1 --2- 172.21.15.43:0/3754108072 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01740cf670 0x7f0174156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:37.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.394+0000 7f0178fdc640 1 --2- 172.21.15.43:0/3754108072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f017415df50 0x7f0174159910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:37.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.394+0000 7f0173fff640 1 -- 172.21.15.43:0/3754108072 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01740cf670 msgr2=0x7f0174156de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:37.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.394+0000 7f0173fff640 1 --2- 172.21.15.43:0/3754108072 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01740cf670 0x7f0174156de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:37.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.394+0000 7f0173fff640 1 -- 172.21.15.43:0/3754108072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f017415df50 msgr2=0x7f0174159910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:37.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.394+0000 7f0173fff640 1 --2- 172.21.15.43:0/3754108072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f017415df50 0x7f0174159910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:37.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.394+0000 7f0173fff640 1 -- 172.21.15.43:0/3754108072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0174155d40 con 0x7f01740c8d00 2024-04-25T08:57:37.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.394+0000 7f0178fdc640 1 --2- 172.21.15.43:0/3754108072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f017415df50 0x7f0174159910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:37.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.394+0000 7f01737fe640 1 --2- 172.21.15.43:0/3754108072 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01740cf670 0x7f0174156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:37.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.394+0000 7f0173fff640 1 --2- 172.21.15.43:0/3754108072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01740c8d00 0x7f01740c6810 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f016400c400 tx=0x7f016400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:37.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.394+0000 7f0170ff9640 1 -- 172.21.15.43:0/3754108072 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0164017070 con 0x7f01740c8d00 2024-04-25T08:57:37.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.394+0000 7f0170ff9640 1 -- 172.21.15.43:0/3754108072 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01640052d0 con 0x7f01740c8d00 2024-04-25T08:57:37.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.394+0000 7f017aa3f640 1 -- 172.21.15.43:0/3754108072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0174156020 con 0x7f01740c8d00 2024-04-25T08:57:37.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.394+0000 7f0170ff9640 1 -- 172.21.15.43:0/3754108072 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0164003830 con 0x7f01740c8d00 2024-04-25T08:57:37.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.394+0000 7f017aa3f640 1 -- 172.21.15.43:0/3754108072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01741565e0 con 0x7f01740c8d00 2024-04-25T08:57:37.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.395+0000 7f017aa3f640 1 -- 172.21.15.43:0/3754108072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01740680f0 con 0x7f01740c8d00 2024-04-25T08:57:37.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.396+0000 7f0170ff9640 1 -- 172.21.15.43:0/3754108072 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f016401a070 con 0x7f01740c8d00 2024-04-25T08:57:37.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.396+0000 7f0170ff9640 1 --2- 172.21.15.43:0/3754108072 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f016006b4d0 0x7f016006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:37.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.396+0000 7f0170ff9640 1 -- 172.21.15.43:0/3754108072 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0164089590 con 0x7f01740c8d00 2024-04-25T08:57:37.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.396+0000 7f01737fe640 1 --2- 172.21.15.43:0/3754108072 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f016006b4d0 0x7f016006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:37.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.397+0000 7f01737fe640 1 --2- 172.21.15.43:0/3754108072 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f016006b4d0 0x7f016006d990 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f01741579c0 tx=0x7f0168009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:37.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.401+0000 7f0170ff9640 1 -- 172.21.15.43:0/3754108072 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0164058410 con 0x7f01740c8d00 2024-04-25T08:57:37.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.583+0000 7f017aa3f640 1 -- 172.21.15.43:0/3754108072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0174074ef0 con 0x7f01740c8d00 2024-04-25T08:57:37.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.584+0000 7f0170ff9640 1 -- 172.21.15.43:0/3754108072 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16577 (secure 0 0 0) 0x7f0164057d10 con 0x7f01740c8d00 2024-04-25T08:57:37.573 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:57:37.574 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":46,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:57:03.489946+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:57:00.135489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T08:57:00.830128+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:57:37.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.586+0000 7f017aa3f640 1 -- 172.21.15.43:0/3754108072 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f016006b4d0 msgr2=0x7f016006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:37.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.586+0000 7f017aa3f640 1 --2- 172.21.15.43:0/3754108072 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f016006b4d0 0x7f016006d990 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f01741579c0 tx=0x7f0168009040 comp rx=0 tx=0).stop 2024-04-25T08:57:37.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.586+0000 7f017aa3f640 1 -- 172.21.15.43:0/3754108072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01740c8d00 msgr2=0x7f01740c6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:37.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.586+0000 7f017aa3f640 1 --2- 172.21.15.43:0/3754108072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01740c8d00 0x7f01740c6810 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f016400c400 tx=0x7f016400c8c0 comp rx=0 tx=0).stop 2024-04-25T08:57:37.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.586+0000 7f017aa3f640 1 -- 172.21.15.43:0/3754108072 shutdown_connections 2024-04-25T08:57:37.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.586+0000 7f017aa3f640 1 --2- 172.21.15.43:0/3754108072 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f016006b4d0 0x7f016006d990 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:37.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.586+0000 7f017aa3f640 1 --2- 172.21.15.43:0/3754108072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f017415df50 0x7f0174159910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:37.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.586+0000 7f017aa3f640 1 --2- 172.21.15.43:0/3754108072 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01740cf670 0x7f0174156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:37.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.586+0000 7f017aa3f640 1 --2- 172.21.15.43:0/3754108072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01740c8d00 0x7f01740c6810 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:37.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.586+0000 7f017aa3f640 1 -- 172.21.15.43:0/3754108072 >> 172.21.15.43:0/3754108072 conn(0x7f01740d3a30 msgr2=0x7f0174152ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:37.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.586+0000 7f017aa3f640 1 -- 172.21.15.43:0/3754108072 shutdown_connections 2024-04-25T08:57:37.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.586+0000 7f017aa3f640 1 -- 172.21.15.43:0/3754108072 wait complete. 2024-04-25T08:57:37.586 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:57:00.135489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:57:00.830128+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:57:37.587 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:57:37.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.701+0000 7f430228f640 1 Processor -- start 2024-04-25T08:57:37.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.701+0000 7f430228f640 1 -- start start 2024-04-25T08:57:37.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.702+0000 7f430228f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f42fc15ddd0 0x7f42fc0b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:37.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.702+0000 7f430228f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f42fc06ef90 0x7f42fc06f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:37.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.702+0000 7f430228f640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42fc0d2620 con 0x7f42fc0cc460 2024-04-25T08:57:37.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.702+0000 7f430228f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42fc06f930 con 0x7f42fc15ddd0 2024-04-25T08:57:37.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.702+0000 7f430228f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42fc0b0b80 con 0x7f42fc06ef90 2024-04-25T08:57:37.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.702+0000 7f42fbfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f42fc15ddd0 0x7f42fc0b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:37.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.702+0000 7f42fbfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f42fc15ddd0 0x7f42fc0b3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58242/0 (socket says 172.21.15.43:58242) 2024-04-25T08:57:37.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.702+0000 7f42fbfff640 1 -- 172.21.15.43:0/2257755589 learned_addr learned my addr 172.21.15.43:0/2257755589 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:37.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.702+0000 7f42fb7fe640 1 --2- 172.21.15.43:0/2257755589 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f42fc06ef90 0x7f42fc06f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:37.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.702+0000 7f42fbfff640 1 -- 172.21.15.43:0/2257755589 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f42fc06ef90 msgr2=0x7f42fc06f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:37.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.702+0000 7f42fbfff640 1 --2- 172.21.15.43:0/2257755589 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f42fc06ef90 0x7f42fc06f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:37.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.702+0000 7f42fbfff640 1 -- 172.21.15.43:0/2257755589 >> v1:172.21.15.193:6789/0 conn(0x7f42fc0cc460 legacy=0x7f42fc0cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:37.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.702+0000 7f42fbfff640 1 -- 172.21.15.43:0/2257755589 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42fc0b0e60 con 0x7f42fc15ddd0 2024-04-25T08:57:37.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.703+0000 7f42fbfff640 1 --2- 172.21.15.43:0/2257755589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42fc15ddd0 0x7f42fc0b3dd0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f42f0009870 tx=0x7f42f002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d1d395c7ecea268 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:37.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.703+0000 7f42faffd640 1 -- 172.21.15.43:0/2257755589 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f0037070 con 0x7f42fc15ddd0 2024-04-25T08:57:37.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.703+0000 7f42faffd640 1 -- 172.21.15.43:0/2257755589 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42f0032440 con 0x7f42fc15ddd0 2024-04-25T08:57:37.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.703+0000 7f42faffd640 1 -- 172.21.15.43:0/2257755589 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f0032810 con 0x7f42fc15ddd0 2024-04-25T08:57:37.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.704+0000 7f430228f640 1 -- 172.21.15.43:0/2257755589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42fc15ddd0 msgr2=0x7f42fc0b3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:37.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.704+0000 7f430228f640 1 --2- 172.21.15.43:0/2257755589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42fc15ddd0 0x7f42fc0b3dd0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f42f0009870 tx=0x7f42f002a210 comp rx=0 tx=0).stop 2024-04-25T08:57:37.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.704+0000 7f430228f640 1 -- 172.21.15.43:0/2257755589 shutdown_connections 2024-04-25T08:57:37.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.704+0000 7f430228f640 1 --2- 172.21.15.43:0/2257755589 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f42fc06ef90 0x7f42fc06f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:37.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.704+0000 7f430228f640 1 --2- 172.21.15.43:0/2257755589 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42fc15ddd0 0x7f42fc0b3dd0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:37.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.704+0000 7f430228f640 1 -- 172.21.15.43:0/2257755589 >> 172.21.15.43:0/2257755589 conn(0x7f42fc0d17e0 msgr2=0x7f42fc0d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:37.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.704+0000 7f430228f640 1 -- 172.21.15.43:0/2257755589 shutdown_connections 2024-04-25T08:57:37.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.704+0000 7f430228f640 1 -- 172.21.15.43:0/2257755589 wait complete. 2024-04-25T08:57:37.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.705+0000 7f430228f640 1 Processor -- start 2024-04-25T08:57:37.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.705+0000 7f430228f640 1 -- start start 2024-04-25T08:57:37.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.706+0000 7f430228f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42fc06ef90 0x7f42fc0ca120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:37.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.706+0000 7f430228f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42fc0cc460 0x7f42fc0ca660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:37.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.706+0000 7f430228f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f42fc15ddd0 0x7f42fc0caba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:37.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.706+0000 7f430228f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42fc06d670 con 0x7f42fc06ef90 2024-04-25T08:57:37.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.706+0000 7f430228f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42fc06d4f0 con 0x7f42fc0cc460 2024-04-25T08:57:37.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.706+0000 7f430228f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42fc06d7f0 con 0x7f42fc15ddd0 2024-04-25T08:57:37.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.706+0000 7f42fbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42fc06ef90 0x7f42fc0ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:37.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.706+0000 7f42fbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42fc06ef90 0x7f42fc0ca120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58250/0 (socket says 172.21.15.43:58250) 2024-04-25T08:57:37.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.706+0000 7f42fbfff640 1 -- 172.21.15.43:0/4288881444 learned_addr learned my addr 172.21.15.43:0/4288881444 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:37.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.706+0000 7f430082c640 1 --2- 172.21.15.43:0/4288881444 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f42fc15ddd0 0x7f42fc0caba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:37.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.706+0000 7f42fb7fe640 1 --2- 172.21.15.43:0/4288881444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42fc0cc460 0x7f42fc0ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:37.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.706+0000 7f42fbfff640 1 -- 172.21.15.43:0/4288881444 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f42fc15ddd0 msgr2=0x7f42fc0caba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:37.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.706+0000 7f42fbfff640 1 --2- 172.21.15.43:0/4288881444 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f42fc15ddd0 0x7f42fc0caba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:37.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.706+0000 7f42fbfff640 1 -- 172.21.15.43:0/4288881444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42fc0cc460 msgr2=0x7f42fc0ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:37.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.706+0000 7f42fbfff640 1 --2- 172.21.15.43:0/4288881444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42fc0cc460 0x7f42fc0ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:37.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.706+0000 7f42fbfff640 1 -- 172.21.15.43:0/4288881444 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42fc0cb610 con 0x7f42fc06ef90 2024-04-25T08:57:37.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.706+0000 7f42fb7fe640 1 --2- 172.21.15.43:0/4288881444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42fc0cc460 0x7f42fc0ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:37.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.706+0000 7f42fbfff640 1 --2- 172.21.15.43:0/4288881444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42fc06ef90 0x7f42fc0ca120 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f42f0000c00 tx=0x7f42f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:37.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.707+0000 7f42f8ff9640 1 -- 172.21.15.43:0/4288881444 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f0041020 con 0x7f42fc06ef90 2024-04-25T08:57:37.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.707+0000 7f42f8ff9640 1 -- 172.21.15.43:0/4288881444 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42f00306c0 con 0x7f42fc06ef90 2024-04-25T08:57:37.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.707+0000 7f430228f640 1 -- 172.21.15.43:0/4288881444 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42fc0b45c0 con 0x7f42fc06ef90 2024-04-25T08:57:37.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.707+0000 7f42f8ff9640 1 -- 172.21.15.43:0/4288881444 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f0037040 con 0x7f42fc06ef90 2024-04-25T08:57:37.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.707+0000 7f430228f640 1 -- 172.21.15.43:0/4288881444 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42fc0b4b80 con 0x7f42fc06ef90 2024-04-25T08:57:37.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.708+0000 7f430228f640 1 -- 172.21.15.43:0/4288881444 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42fc074f60 con 0x7f42fc06ef90 2024-04-25T08:57:37.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.708+0000 7f42f8ff9640 1 -- 172.21.15.43:0/4288881444 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f42f0044020 con 0x7f42fc06ef90 2024-04-25T08:57:37.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.709+0000 7f42f8ff9640 1 --2- 172.21.15.43:0/4288881444 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f42e806b500 0x7f42e806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:37.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.709+0000 7f42f8ff9640 1 -- 172.21.15.43:0/4288881444 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f42f00aa8e0 con 0x7f42fc06ef90 2024-04-25T08:57:37.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.709+0000 7f42fb7fe640 1 --2- 172.21.15.43:0/4288881444 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f42e806b500 0x7f42e806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:37.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.710+0000 7f42fb7fe640 1 --2- 172.21.15.43:0/4288881444 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f42e806b500 0x7f42e806d9c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f42ec006fd0 tx=0x7f42ec008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:37.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.713+0000 7f42f8ff9640 1 -- 172.21.15.43:0/4288881444 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f42f004b030 con 0x7f42fc06ef90 2024-04-25T08:57:37.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.893+0000 7f430228f640 1 -- 172.21.15.43:0/4288881444 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f42fc0cef60 con 0x7f42fc06ef90 2024-04-25T08:57:37.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.894+0000 7f42f8ff9640 1 -- 172.21.15.43:0/4288881444 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7f42f0079760 con 0x7f42fc06ef90 2024-04-25T08:57:37.883 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T08:57:37.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.896+0000 7f430228f640 1 -- 172.21.15.43:0/4288881444 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f42e806b500 msgr2=0x7f42e806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:37.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.896+0000 7f430228f640 1 --2- 172.21.15.43:0/4288881444 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f42e806b500 0x7f42e806d9c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f42ec006fd0 tx=0x7f42ec008040 comp rx=0 tx=0).stop 2024-04-25T08:57:37.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.896+0000 7f430228f640 1 -- 172.21.15.43:0/4288881444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42fc06ef90 msgr2=0x7f42fc0ca120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:37.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.896+0000 7f430228f640 1 --2- 172.21.15.43:0/4288881444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42fc06ef90 0x7f42fc0ca120 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f42f0000c00 tx=0x7f42f00046c0 comp rx=0 tx=0).stop 2024-04-25T08:57:37.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.896+0000 7f430228f640 1 -- 172.21.15.43:0/4288881444 shutdown_connections 2024-04-25T08:57:37.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.896+0000 7f430228f640 1 --2- 172.21.15.43:0/4288881444 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f42fc15ddd0 0x7f42fc0caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:37.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.896+0000 7f430228f640 1 --2- 172.21.15.43:0/4288881444 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f42e806b500 0x7f42e806d9c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:37.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.896+0000 7f430228f640 1 --2- 172.21.15.43:0/4288881444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f42fc0cc460 0x7f42fc0ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:37.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.896+0000 7f430228f640 1 --2- 172.21.15.43:0/4288881444 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f42fc06ef90 0x7f42fc0ca120 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:37.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.896+0000 7f430228f640 1 -- 172.21.15.43:0/4288881444 >> 172.21.15.43:0/4288881444 conn(0x7f42fc0d17e0 msgr2=0x7f42fc15a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:37.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.896+0000 7f430228f640 1 -- 172.21.15.43:0/4288881444 shutdown_connections 2024-04-25T08:57:37.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:37.896+0000 7f430228f640 1 -- 172.21.15.43:0/4288881444 wait complete. 2024-04-25T08:57:37.897 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:57:38.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.011+0000 7f03f4438640 1 Processor -- start 2024-04-25T08:57:38.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.011+0000 7f03f4438640 1 -- start start 2024-04-25T08:57:38.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.012+0000 7f03f4438640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f03ec06d7d0 0x7f03ec0ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:38.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.012+0000 7f03f4438640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03ec0cd570 con 0x7f03ec0cd120 2024-04-25T08:57:38.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.012+0000 7f03f4438640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03ec15c280 con 0x7f03ec0aee80 2024-04-25T08:57:38.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.012+0000 7f03f4438640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03ec0af260 con 0x7f03ec06d7d0 2024-04-25T08:57:38.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.012+0000 7f03f29d5640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f03ec0cd120 0x7f03ec15d590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:52740/0 (socket says 172.21.15.43:52740) 2024-04-25T08:57:38.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.012+0000 7f03f29d5640 1 -- 172.21.15.43:0/1733902339 learned_addr learned my addr 172.21.15.43:0/1733902339 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:38.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.012+0000 7f03f19d3640 1 --2- 172.21.15.43:0/1733902339 >> v2:172.21.15.193:3300/0 conn(0x7f03ec06d7d0 0x7f03ec0ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:38.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.013+0000 7f03f11d2640 1 -- 172.21.15.43:0/1733902339 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2416884101 0 0) 0x7f03ec0cd570 con 0x7f03ec0cd120 2024-04-25T08:57:38.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.013+0000 7f03f11d2640 1 -- 172.21.15.43:0/1733902339 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03d4003520 con 0x7f03ec0cd120 2024-04-25T08:57:38.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.013+0000 7f03f11d2640 1 -- 172.21.15.43:0/1733902339 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3701749809 0 0) 0x7f03ec15c280 con 0x7f03ec0aee80 2024-04-25T08:57:38.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.013+0000 7f03f11d2640 1 -- 172.21.15.43:0/1733902339 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03ec0cd570 con 0x7f03ec0aee80 2024-04-25T08:57:38.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.013+0000 7f03f11d2640 1 -- 172.21.15.43:0/1733902339 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4062261916 0 0) 0x7f03d4003520 con 0x7f03ec0cd120 2024-04-25T08:57:38.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.013+0000 7f03f11d2640 1 -- 172.21.15.43:0/1733902339 >> v2:172.21.15.193:3300/0 conn(0x7f03ec06d7d0 msgr2=0x7f03ec0ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:38.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.013+0000 7f03f11d2640 1 --2- 172.21.15.43:0/1733902339 >> v2:172.21.15.193:3300/0 conn(0x7f03ec06d7d0 0x7f03ec0ccbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:38.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.013+0000 7f03f11d2640 1 -- 172.21.15.43:0/1733902339 >> v1:172.21.15.193:6789/0 conn(0x7f03ec0aee80 legacy=0x7f03ec06d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:38.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.013+0000 7f03f11d2640 1 -- 172.21.15.43:0/1733902339 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03ec15c560 con 0x7f03ec0cd120 2024-04-25T08:57:38.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.013+0000 7f03f11d2640 1 -- 172.21.15.43:0/1733902339 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f03dc0019f0 con 0x7f03ec0cd120 2024-04-25T08:57:38.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.013+0000 7f03f11d2640 1 -- 172.21.15.43:0/1733902339 >> v1:172.21.15.43:6789/0 conn(0x7f03ec0cd120 legacy=0x7f03ec15d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:38.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.013+0000 7f03f11d2640 1 --2- 172.21.15.43:0/1733902339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03d4003b40 0x7f03d4003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:38.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.013+0000 7f03f11d2640 1 --2- 172.21.15.43:0/1733902339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03d4004800 0x7f03d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:38.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.013+0000 7f03f11d2640 1 --2- 172.21.15.43:0/1733902339 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03d40051c0 0x7f03d400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:38.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.013+0000 7f03f11d2640 1 -- 172.21.15.43:0/1733902339 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d400c010 con 0x7f03d4004800 2024-04-25T08:57:38.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.013+0000 7f03f11d2640 1 -- 172.21.15.43:0/1733902339 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d400c1c0 con 0x7f03d4003b40 2024-04-25T08:57:38.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.013+0000 7f03f11d2640 1 -- 172.21.15.43:0/1733902339 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d400c3c0 con 0x7f03d40051c0 2024-04-25T08:57:38.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.013+0000 7f03f19d3640 1 --2- 172.21.15.43:0/1733902339 >> v2:172.21.15.193:3300/0 conn(0x7f03ec06d7d0 0x7f03ec0ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:57:38.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.013+0000 7f03f19d3640 1 --2- 172.21.15.43:0/1733902339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03d4004800 0x7f03d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:38.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.013+0000 7f03f21d4640 1 --2- 172.21.15.43:0/1733902339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03d4003b40 0x7f03d4003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:38.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.013+0000 7f03f29d5640 1 --2- 172.21.15.43:0/1733902339 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03d40051c0 0x7f03d400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:38.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.013+0000 7f03f19d3640 1 -- 172.21.15.43:0/1733902339 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03d40051c0 msgr2=0x7f03d400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:38.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.013+0000 7f03f19d3640 1 --2- 172.21.15.43:0/1733902339 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03d40051c0 0x7f03d400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:38.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.013+0000 7f03f19d3640 1 -- 172.21.15.43:0/1733902339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03d4003b40 msgr2=0x7f03d4003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:38.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.013+0000 7f03f19d3640 1 --2- 172.21.15.43:0/1733902339 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03d4003b40 0x7f03d4003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:38.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.013+0000 7f03f19d3640 1 -- 172.21.15.43:0/1733902339 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d4003520 con 0x7f03d4004800 2024-04-25T08:57:38.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.013+0000 7f03f29d5640 1 --2- 172.21.15.43:0/1733902339 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03d40051c0 0x7f03d400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:38.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.013+0000 7f03f29d5640 1 -- 172.21.15.43:0/1733902339 reap_dead start 2024-04-25T08:57:38.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.014+0000 7f03f19d3640 1 --2- 172.21.15.43:0/1733902339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03d4004800 0x7f03d4004c80 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f03e0009c70 tx=0x7f03d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a83a1d715de00f69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:38.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.014+0000 7f03f11d2640 1 -- 172.21.15.43:0/1733902339 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e0009840 con 0x7f03d4004800 2024-04-25T08:57:38.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.014+0000 7f03f11d2640 1 -- 172.21.15.43:0/1733902339 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e0004020 con 0x7f03d4004800 2024-04-25T08:57:38.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.014+0000 7f03f11d2640 1 -- 172.21.15.43:0/1733902339 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e00043d0 con 0x7f03d4004800 2024-04-25T08:57:38.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.015+0000 7f03f4438640 1 -- 172.21.15.43:0/1733902339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03d4004800 msgr2=0x7f03d4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:38.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.015+0000 7f03f4438640 1 --2- 172.21.15.43:0/1733902339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03d4004800 0x7f03d4004c80 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f03e0009c70 tx=0x7f03d4005830 comp rx=0 tx=0).stop 2024-04-25T08:57:38.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.015+0000 7f03f4438640 1 -- 172.21.15.43:0/1733902339 shutdown_connections 2024-04-25T08:57:38.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.015+0000 7f03f4438640 1 --2- 172.21.15.43:0/1733902339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03d4004800 0x7f03d4004c80 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:38.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.015+0000 7f03f4438640 1 -- 172.21.15.43:0/1733902339 >> 172.21.15.43:0/1733902339 conn(0x7f03ec06ab30 msgr2=0x7f03ec06af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:38.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.015+0000 7f03f4438640 1 -- 172.21.15.43:0/1733902339 shutdown_connections 2024-04-25T08:57:38.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.015+0000 7f03f4438640 1 -- 172.21.15.43:0/1733902339 wait complete. 2024-04-25T08:57:38.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.016+0000 7f03f4438640 1 Processor -- start 2024-04-25T08:57:38.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.016+0000 7f03f4438640 1 -- start start 2024-04-25T08:57:38.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.017+0000 7f03f4438640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03d4004800 0x7f03ec154890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:38.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.017+0000 7f03f4438640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03ec1416f0 0x7f03ec141b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:38.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.017+0000 7f03f4438640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03ec142090 0x7f03ec1533e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:38.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.017+0000 7f03f4438640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec0ccbe0 con 0x7f03d4004800 2024-04-25T08:57:38.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.017+0000 7f03f4438640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec15cb20 con 0x7f03ec142090 2024-04-25T08:57:38.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.017+0000 7f03f4438640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03ec0ccd60 con 0x7f03ec1416f0 2024-04-25T08:57:38.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.017+0000 7f03f21d4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03d4004800 0x7f03ec154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:38.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.017+0000 7f03f21d4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03d4004800 0x7f03ec154890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58268/0 (socket says 172.21.15.43:58268) 2024-04-25T08:57:38.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.017+0000 7f03f21d4640 1 -- 172.21.15.43:0/1630394078 learned_addr learned my addr 172.21.15.43:0/1630394078 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:38.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.017+0000 7f03f29d5640 1 --2- 172.21.15.43:0/1630394078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03ec142090 0x7f03ec1533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:38.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.017+0000 7f03f21d4640 1 -- 172.21.15.43:0/1630394078 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03ec1416f0 msgr2=0x7f03ec141b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:38.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.017+0000 7f03f21d4640 1 --2- 172.21.15.43:0/1630394078 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03ec1416f0 0x7f03ec141b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:38.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.017+0000 7f03f21d4640 1 -- 172.21.15.43:0/1630394078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03ec142090 msgr2=0x7f03ec1533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:38.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.017+0000 7f03f21d4640 1 --2- 172.21.15.43:0/1630394078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03ec142090 0x7f03ec1533e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:38.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.017+0000 7f03f21d4640 1 -- 172.21.15.43:0/1630394078 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03ec154dd0 con 0x7f03d4004800 2024-04-25T08:57:38.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.018+0000 7f03f21d4640 1 --2- 172.21.15.43:0/1630394078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03d4004800 0x7f03ec154890 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f03e400f820 tx=0x7f03e400fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:38.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.018+0000 7f03eaffd640 1 -- 172.21.15.43:0/1630394078 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e40052e0 con 0x7f03d4004800 2024-04-25T08:57:38.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.018+0000 7f03eaffd640 1 -- 172.21.15.43:0/1630394078 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e4018070 con 0x7f03d4004800 2024-04-25T08:57:38.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.018+0000 7f03f4438640 1 -- 172.21.15.43:0/1630394078 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03ec1428c0 con 0x7f03d4004800 2024-04-25T08:57:38.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.018+0000 7f03eaffd640 1 -- 172.21.15.43:0/1630394078 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e400d970 con 0x7f03d4004800 2024-04-25T08:57:38.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.018+0000 7f03f4438640 1 -- 172.21.15.43:0/1630394078 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03ec0a98b0 con 0x7f03d4004800 2024-04-25T08:57:38.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.019+0000 7f03f4438640 1 -- 172.21.15.43:0/1630394078 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03ec074f60 con 0x7f03d4004800 2024-04-25T08:57:38.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.019+0000 7f03eaffd640 1 -- 172.21.15.43:0/1630394078 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f03e400db10 con 0x7f03d4004800 2024-04-25T08:57:38.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.020+0000 7f03eaffd640 1 --2- 172.21.15.43:0/1630394078 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f03d806b4d0 0x7f03d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:38.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.020+0000 7f03eaffd640 1 -- 172.21.15.43:0/1630394078 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f03e408a1c0 con 0x7f03d4004800 2024-04-25T08:57:38.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.020+0000 7f03f19d3640 1 --2- 172.21.15.43:0/1630394078 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f03d806b4d0 0x7f03d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:38.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.020+0000 7f03f19d3640 1 --2- 172.21.15.43:0/1630394078 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f03d806b4d0 0x7f03d806d990 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f03e0032000 tx=0x7f03e002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:38.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.024+0000 7f03eaffd640 1 -- 172.21.15.43:0/1630394078 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f03e401a310 con 0x7f03d4004800 2024-04-25T08:57:38.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.202+0000 7f03f4438640 1 -- 172.21.15.43:0/1630394078 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f03ec0725f0 con 0x7f03d4004800 2024-04-25T08:57:38.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.202+0000 7f03eaffd640 1 -- 172.21.15.43:0/1630394078 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v46) v1 ==== 104+0+11 (secure 0 0 0) 0x7f03e4059040 con 0x7f03d4004800 2024-04-25T08:57:38.192 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T08:57:38.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.204+0000 7f03f4438640 1 -- 172.21.15.43:0/1630394078 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f03d806b4d0 msgr2=0x7f03d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:38.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.204+0000 7f03f4438640 1 --2- 172.21.15.43:0/1630394078 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f03d806b4d0 0x7f03d806d990 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f03e0032000 tx=0x7f03e002e000 comp rx=0 tx=0).stop 2024-04-25T08:57:38.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.204+0000 7f03f4438640 1 -- 172.21.15.43:0/1630394078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03d4004800 msgr2=0x7f03ec154890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:38.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.204+0000 7f03f4438640 1 --2- 172.21.15.43:0/1630394078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03d4004800 0x7f03ec154890 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f03e400f820 tx=0x7f03e400fce0 comp rx=0 tx=0).stop 2024-04-25T08:57:38.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.204+0000 7f03f4438640 1 -- 172.21.15.43:0/1630394078 shutdown_connections 2024-04-25T08:57:38.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.204+0000 7f03f4438640 1 --2- 172.21.15.43:0/1630394078 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f03d806b4d0 0x7f03d806d990 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:38.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.204+0000 7f03f4438640 1 --2- 172.21.15.43:0/1630394078 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03ec142090 0x7f03ec1533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:38.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.204+0000 7f03f4438640 1 --2- 172.21.15.43:0/1630394078 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03ec1416f0 0x7f03ec141b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:38.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.204+0000 7f03f4438640 1 --2- 172.21.15.43:0/1630394078 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03d4004800 0x7f03ec154890 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:38.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.204+0000 7f03f4438640 1 -- 172.21.15.43:0/1630394078 >> 172.21.15.43:0/1630394078 conn(0x7f03ec06ab30 msgr2=0x7f03ec0b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:38.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.205+0000 7f03f4438640 1 -- 172.21.15.43:0/1630394078 shutdown_connections 2024-04-25T08:57:38.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.205+0000 7f03f4438640 1 -- 172.21.15.43:0/1630394078 wait complete. 2024-04-25T08:57:38.205 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:57:38.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.319+0000 7f776fccb640 1 Processor -- start 2024-04-25T08:57:38.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.319+0000 7f776fccb640 1 -- start start 2024-04-25T08:57:38.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.320+0000 7f776fccb640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77680d19f0 0x7f77680d1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:38.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.320+0000 7f776fccb640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f77680d2370 0x7f77680c7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:38.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.320+0000 7f776fccb640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77680cf030 con 0x7f7768161310 2024-04-25T08:57:38.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.320+0000 7f776fccb640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77680d4870 con 0x7f77680d2370 2024-04-25T08:57:38.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.320+0000 7f776fccb640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77680cf310 con 0x7f77680d19f0 2024-04-25T08:57:38.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.320+0000 7f776e268640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f77680d2370 0x7f77680c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:38.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.320+0000 7f776da67640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f7768161310 0x7f77680d4080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:52746/0 (socket says 172.21.15.43:52746) 2024-04-25T08:57:38.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.320+0000 7f776da67640 1 -- 172.21.15.43:0/2950043117 learned_addr learned my addr 172.21.15.43:0/2950043117 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:38.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.320+0000 7f776d266640 1 --2- 172.21.15.43:0/2950043117 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77680d19f0 0x7f77680d1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:38.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.320+0000 7f776e268640 1 -- 172.21.15.43:0/2950043117 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77680d19f0 msgr2=0x7f77680d1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:38.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.320+0000 7f776e268640 1 --2- 172.21.15.43:0/2950043117 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77680d19f0 0x7f77680d1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:38.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.320+0000 7f776e268640 1 -- 172.21.15.43:0/2950043117 >> v1:172.21.15.43:6789/0 conn(0x7f7768161310 legacy=0x7f77680d4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:38.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.320+0000 7f776e268640 1 -- 172.21.15.43:0/2950043117 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77680cf5f0 con 0x7f77680d2370 2024-04-25T08:57:38.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.320+0000 7f776ca65640 1 -- 172.21.15.43:0/2950043117 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2779921256 0 0) 0x7f77680cf030 con 0x7f7768161310 2024-04-25T08:57:38.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.321+0000 7f776e268640 1 --2- 172.21.15.43:0/2950043117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77680d2370 0x7f77680c7860 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f7760009d90 tx=0x7f776002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43217e188b36480a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:38.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.321+0000 7f776ca65640 1 -- 172.21.15.43:0/2950043117 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7760007740 con 0x7f77680d2370 2024-04-25T08:57:38.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.321+0000 7f776ca65640 1 -- 172.21.15.43:0/2950043117 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77600078e0 con 0x7f77680d2370 2024-04-25T08:57:38.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.321+0000 7f776ca65640 1 -- 172.21.15.43:0/2950043117 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7760007cb0 con 0x7f77680d2370 2024-04-25T08:57:38.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.322+0000 7f776fccb640 1 -- 172.21.15.43:0/2950043117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77680d2370 msgr2=0x7f77680c7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:38.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.322+0000 7f776fccb640 1 --2- 172.21.15.43:0/2950043117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77680d2370 0x7f77680c7860 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f7760009d90 tx=0x7f776002a660 comp rx=0 tx=0).stop 2024-04-25T08:57:38.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.322+0000 7f776fccb640 1 -- 172.21.15.43:0/2950043117 shutdown_connections 2024-04-25T08:57:38.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.322+0000 7f776fccb640 1 --2- 172.21.15.43:0/2950043117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77680d2370 0x7f77680c7860 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:38.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.322+0000 7f776fccb640 1 --2- 172.21.15.43:0/2950043117 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77680d19f0 0x7f77680d1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:38.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.322+0000 7f776fccb640 1 -- 172.21.15.43:0/2950043117 >> 172.21.15.43:0/2950043117 conn(0x7f776809cd20 msgr2=0x7f77680d3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:38.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.322+0000 7f776fccb640 1 -- 172.21.15.43:0/2950043117 shutdown_connections 2024-04-25T08:57:38.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.322+0000 7f776fccb640 1 -- 172.21.15.43:0/2950043117 wait complete. 2024-04-25T08:57:38.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.323+0000 7f776fccb640 1 Processor -- start 2024-04-25T08:57:38.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.323+0000 7f776fccb640 1 -- start start 2024-04-25T08:57:38.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.324+0000 7f776fccb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77680d19f0 0x7f77680b8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:38.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.324+0000 7f776fccb640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77680d2370 0x7f77681f4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:38.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.324+0000 7f776fccb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7768161310 0x7f77680d3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:38.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.324+0000 7f776fccb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77680c7da0 con 0x7f7768161310 2024-04-25T08:57:38.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.324+0000 7f776fccb640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77680d2800 con 0x7f77680d19f0 2024-04-25T08:57:38.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.324+0000 7f776fccb640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77680c7f20 con 0x7f77680d2370 2024-04-25T08:57:38.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.324+0000 7f776e268640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7768161310 0x7f77680d3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:38.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.324+0000 7f776e268640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7768161310 0x7f77680d3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58284/0 (socket says 172.21.15.43:58284) 2024-04-25T08:57:38.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.324+0000 7f776e268640 1 -- 172.21.15.43:0/1809815981 learned_addr learned my addr 172.21.15.43:0/1809815981 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:38.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.324+0000 7f776d266640 1 --2- 172.21.15.43:0/1809815981 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77680d2370 0x7f77681f4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:38.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.324+0000 7f776da67640 1 --2- 172.21.15.43:0/1809815981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77680d19f0 0x7f77680b8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:38.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.324+0000 7f776e268640 1 -- 172.21.15.43:0/1809815981 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77680d2370 msgr2=0x7f77681f4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:38.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.324+0000 7f776e268640 1 --2- 172.21.15.43:0/1809815981 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77680d2370 0x7f77681f4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:38.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.324+0000 7f776e268640 1 -- 172.21.15.43:0/1809815981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77680d19f0 msgr2=0x7f77680b8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:38.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.324+0000 7f776e268640 1 --2- 172.21.15.43:0/1809815981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77680d19f0 0x7f77680b8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:38.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.324+0000 7f776e268640 1 -- 172.21.15.43:0/1809815981 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77680a02b0 con 0x7f7768161310 2024-04-25T08:57:38.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.324+0000 7f776d266640 1 --2- 172.21.15.43:0/1809815981 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77680d2370 0x7f77681f4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:38.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.324+0000 7f776e268640 1 --2- 172.21.15.43:0/1809815981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7768161310 0x7f77680d3850 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f7760009d90 tx=0x7f7760004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:38.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.324+0000 7f775e7fc640 1 -- 172.21.15.43:0/1809815981 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7760040070 con 0x7f7768161310 2024-04-25T08:57:38.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.324+0000 7f775e7fc640 1 -- 172.21.15.43:0/1809815981 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7760031c40 con 0x7f7768161310 2024-04-25T08:57:38.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.324+0000 7f776fccb640 1 -- 172.21.15.43:0/1809815981 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77680a0590 con 0x7f7768161310 2024-04-25T08:57:38.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.324+0000 7f775e7fc640 1 -- 172.21.15.43:0/1809815981 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7760009450 con 0x7f7768161310 2024-04-25T08:57:38.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.325+0000 7f776fccb640 1 -- 172.21.15.43:0/1809815981 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77680a0b50 con 0x7f7768161310 2024-04-25T08:57:38.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.325+0000 7f776fccb640 1 -- 172.21.15.43:0/1809815981 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7768074f60 con 0x7f7768161310 2024-04-25T08:57:38.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.326+0000 7f775e7fc640 1 -- 172.21.15.43:0/1809815981 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7760039040 con 0x7f7768161310 2024-04-25T08:57:38.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.326+0000 7f775e7fc640 1 --2- 172.21.15.43:0/1809815981 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f774c06b4d0 0x7f774c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:38.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.326+0000 7f775e7fc640 1 -- 172.21.15.43:0/1809815981 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f77600ab260 con 0x7f7768161310 2024-04-25T08:57:38.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.327+0000 7f776da67640 1 --2- 172.21.15.43:0/1809815981 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f774c06b4d0 0x7f774c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:38.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.327+0000 7f776da67640 1 --2- 172.21.15.43:0/1809815981 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f774c06b4d0 0x7f774c06d990 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7764000f60 tx=0x7f7764007c00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:38.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.331+0000 7f775e7fc640 1 -- 172.21.15.43:0/1809815981 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f776007a040 con 0x7f7768161310 2024-04-25T08:57:38.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.510+0000 7f776fccb640 1 -- 172.21.15.43:0/1809815981 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f776809de80 con 0x7f7768161310 2024-04-25T08:57:38.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.510+0000 7f775e7fc640 1 -- 172.21.15.43:0/1809815981 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v46) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7760079940 con 0x7f7768161310 2024-04-25T08:57:38.500 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T08:57:38.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.512+0000 7f776fccb640 1 -- 172.21.15.43:0/1809815981 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f774c06b4d0 msgr2=0x7f774c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:38.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.512+0000 7f776fccb640 1 --2- 172.21.15.43:0/1809815981 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f774c06b4d0 0x7f774c06d990 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7764000f60 tx=0x7f7764007c00 comp rx=0 tx=0).stop 2024-04-25T08:57:38.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.512+0000 7f776fccb640 1 -- 172.21.15.43:0/1809815981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7768161310 msgr2=0x7f77680d3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:38.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.512+0000 7f776fccb640 1 --2- 172.21.15.43:0/1809815981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7768161310 0x7f77680d3850 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f7760009d90 tx=0x7f7760004a40 comp rx=0 tx=0).stop 2024-04-25T08:57:38.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.512+0000 7f776fccb640 1 -- 172.21.15.43:0/1809815981 shutdown_connections 2024-04-25T08:57:38.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.512+0000 7f776fccb640 1 --2- 172.21.15.43:0/1809815981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7768161310 0x7f77680d3850 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:38.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.512+0000 7f776fccb640 1 --2- 172.21.15.43:0/1809815981 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77680d2370 0x7f77681f4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:38.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.512+0000 7f776fccb640 1 --2- 172.21.15.43:0/1809815981 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f774c06b4d0 0x7f774c06d990 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:38.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.512+0000 7f776fccb640 1 --2- 172.21.15.43:0/1809815981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77680d19f0 0x7f77680b8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:38.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.512+0000 7f776fccb640 1 -- 172.21.15.43:0/1809815981 >> 172.21.15.43:0/1809815981 conn(0x7f776809cd20 msgr2=0x7f776815f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:38.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.512+0000 7f776fccb640 1 -- 172.21.15.43:0/1809815981 shutdown_connections 2024-04-25T08:57:38.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.512+0000 7f776fccb640 1 -- 172.21.15.43:0/1809815981 wait complete. 2024-04-25T08:57:38.512 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:57:38.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.624+0000 7f204a51d640 1 Processor -- start 2024-04-25T08:57:38.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.624+0000 7f204a51d640 1 -- start start 2024-04-25T08:57:38.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.625+0000 7f204a51d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f204415df50 0x7f20441619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:38.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.625+0000 7f204a51d640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f2044161f30 0x7f2044162380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:38.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.625+0000 7f204a51d640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20440d24a0 con 0x7f20440c7cf0 2024-04-25T08:57:38.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.625+0000 7f204a51d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f204415e330 con 0x7f204415df50 2024-04-25T08:57:38.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.625+0000 7f204a51d640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20440d2740 con 0x7f2044161f30 2024-04-25T08:57:38.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.625+0000 7f2043fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f204415df50 0x7f20441619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:38.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.625+0000 7f2043fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f204415df50 0x7f20441619f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58298/0 (socket says 172.21.15.43:58298) 2024-04-25T08:57:38.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.625+0000 7f2043fff640 1 -- 172.21.15.43:0/742119143 learned_addr learned my addr 172.21.15.43:0/742119143 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:38.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.625+0000 7f20437fe640 1 --2- 172.21.15.43:0/742119143 >> v2:172.21.15.193:3300/0 conn(0x7f2044161f30 0x7f2044162380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:38.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.625+0000 7f2043fff640 1 -- 172.21.15.43:0/742119143 >> v2:172.21.15.193:3300/0 conn(0x7f2044161f30 msgr2=0x7f2044162380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:38.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.625+0000 7f2043fff640 1 --2- 172.21.15.43:0/742119143 >> v2:172.21.15.193:3300/0 conn(0x7f2044161f30 0x7f2044162380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:38.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.625+0000 7f2043fff640 1 -- 172.21.15.43:0/742119143 >> v1:172.21.15.193:6789/0 conn(0x7f20440c7cf0 legacy=0x7f20440c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:38.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.625+0000 7f2043fff640 1 -- 172.21.15.43:0/742119143 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20440d2a20 con 0x7f204415df50 2024-04-25T08:57:38.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.625+0000 7f20437fe640 1 --2- 172.21.15.43:0/742119143 >> v2:172.21.15.193:3300/0 conn(0x7f2044161f30 0x7f2044162380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:38.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.626+0000 7f2043fff640 1 --2- 172.21.15.43:0/742119143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f204415df50 0x7f20441619f0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f204408c880 tx=0x7f203402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a9a60628ff562b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:38.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.626+0000 7f2042ffd640 1 -- 172.21.15.43:0/742119143 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2034037070 con 0x7f204415df50 2024-04-25T08:57:38.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.626+0000 7f2042ffd640 1 -- 172.21.15.43:0/742119143 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2034032440 con 0x7f204415df50 2024-04-25T08:57:38.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.626+0000 7f2042ffd640 1 -- 172.21.15.43:0/742119143 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2034032810 con 0x7f204415df50 2024-04-25T08:57:38.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.627+0000 7f204a51d640 1 -- 172.21.15.43:0/742119143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f204415df50 msgr2=0x7f20441619f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:38.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.627+0000 7f204a51d640 1 --2- 172.21.15.43:0/742119143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f204415df50 0x7f20441619f0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f204408c880 tx=0x7f203402a200 comp rx=0 tx=0).stop 2024-04-25T08:57:38.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.627+0000 7f204a51d640 1 -- 172.21.15.43:0/742119143 shutdown_connections 2024-04-25T08:57:38.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.627+0000 7f204a51d640 1 --2- 172.21.15.43:0/742119143 >> v2:172.21.15.193:3300/0 conn(0x7f2044161f30 0x7f2044162380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:38.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.627+0000 7f204a51d640 1 --2- 172.21.15.43:0/742119143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f204415df50 0x7f20441619f0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:38.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.627+0000 7f204a51d640 1 -- 172.21.15.43:0/742119143 >> 172.21.15.43:0/742119143 conn(0x7f20440d3a30 msgr2=0x7f20440d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:38.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.627+0000 7f204a51d640 1 -- 172.21.15.43:0/742119143 shutdown_connections 2024-04-25T08:57:38.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.627+0000 7f204a51d640 1 -- 172.21.15.43:0/742119143 wait complete. 2024-04-25T08:57:38.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.628+0000 7f204a51d640 1 Processor -- start 2024-04-25T08:57:38.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.628+0000 7f204a51d640 1 -- start start 2024-04-25T08:57:38.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.629+0000 7f204a51d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20440c7cf0 0x7f2044151f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:38.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.629+0000 7f204a51d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f204415df50 0x7f204409d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:38.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.629+0000 7f204a51d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2044161f30 0x7f204409dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:38.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.629+0000 7f204a51d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20440d2190 con 0x7f2044161f30 2024-04-25T08:57:38.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.629+0000 7f204a51d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20440c77d0 con 0x7f20440c7cf0 2024-04-25T08:57:38.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.629+0000 7f204a51d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20440cf670 con 0x7f204415df50 2024-04-25T08:57:38.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.629+0000 7f2048aba640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2044161f30 0x7f204409dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:38.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.629+0000 7f2048aba640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2044161f30 0x7f204409dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58308/0 (socket says 172.21.15.43:58308) 2024-04-25T08:57:38.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.629+0000 7f2048aba640 1 -- 172.21.15.43:0/1808091888 learned_addr learned my addr 172.21.15.43:0/1808091888 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:38.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.629+0000 7f20437fe640 1 --2- 172.21.15.43:0/1808091888 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f204415df50 0x7f204409d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:38.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.629+0000 7f2048aba640 1 -- 172.21.15.43:0/1808091888 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f204415df50 msgr2=0x7f204409d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:38.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.629+0000 7f2043fff640 1 --2- 172.21.15.43:0/1808091888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20440c7cf0 0x7f2044151f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:38.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.629+0000 7f2048aba640 1 --2- 172.21.15.43:0/1808091888 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f204415df50 0x7f204409d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:38.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.629+0000 7f2048aba640 1 -- 172.21.15.43:0/1808091888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20440c7cf0 msgr2=0x7f2044151f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:38.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.629+0000 7f2048aba640 1 --2- 172.21.15.43:0/1808091888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20440c7cf0 0x7f2044151f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:38.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.629+0000 7f2048aba640 1 -- 172.21.15.43:0/1808091888 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20441f4c10 con 0x7f2044161f30 2024-04-25T08:57:38.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.629+0000 7f20437fe640 1 --2- 172.21.15.43:0/1808091888 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f204415df50 0x7f204409d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:38.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.629+0000 7f2043fff640 1 --2- 172.21.15.43:0/1808091888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20440c7cf0 0x7f2044151f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:38.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.630+0000 7f2048aba640 1 --2- 172.21.15.43:0/1808091888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2044161f30 0x7f204409dcb0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f203c00c780 tx=0x7f203c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:38.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.630+0000 7f2040ff9640 1 -- 172.21.15.43:0/1808091888 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203c004b60 con 0x7f2044161f30 2024-04-25T08:57:38.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.630+0000 7f2040ff9640 1 -- 172.21.15.43:0/1808091888 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f203c004d00 con 0x7f2044161f30 2024-04-25T08:57:38.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.630+0000 7f204a51d640 1 -- 172.21.15.43:0/1808091888 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20441f4ef0 con 0x7f2044161f30 2024-04-25T08:57:38.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.630+0000 7f2040ff9640 1 -- 172.21.15.43:0/1808091888 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203c00aa90 con 0x7f2044161f30 2024-04-25T08:57:38.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.630+0000 7f204a51d640 1 -- 172.21.15.43:0/1808091888 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2044152460 con 0x7f2044161f30 2024-04-25T08:57:38.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.631+0000 7f204a51d640 1 -- 172.21.15.43:0/1808091888 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20440680f0 con 0x7f2044161f30 2024-04-25T08:57:38.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.631+0000 7f2040ff9640 1 -- 172.21.15.43:0/1808091888 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f203c00ac30 con 0x7f2044161f30 2024-04-25T08:57:38.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.632+0000 7f2040ff9640 1 --2- 172.21.15.43:0/1808091888 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f203006b4d0 0x7f203006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:38.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.632+0000 7f2040ff9640 1 -- 172.21.15.43:0/1808091888 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f203c08a4f0 con 0x7f2044161f30 2024-04-25T08:57:38.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.632+0000 7f2043fff640 1 --2- 172.21.15.43:0/1808091888 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f203006b4d0 0x7f203006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:38.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.633+0000 7f2043fff640 1 --2- 172.21.15.43:0/1808091888 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f203006b4d0 0x7f203006d990 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2034034000 tx=0x7f203402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:38.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.636+0000 7f2040ff9640 1 -- 172.21.15.43:0/1808091888 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f203c020070 con 0x7f2044161f30 2024-04-25T08:57:38.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.828+0000 7f204a51d640 1 -- 172.21.15.43:0/1808091888 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2044074ef0 con 0x7f2044161f30 2024-04-25T08:57:38.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.828+0000 7f2040ff9640 1 -- 172.21.15.43:0/1808091888 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v46) v1 ==== 80+0+0 (secure 0 0 0) 0x7f203c059360 con 0x7f2044161f30 2024-04-25T08:57:38.818 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-04-25T08:57:38.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.830+0000 7f204a51d640 1 -- 172.21.15.43:0/1808091888 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f203006b4d0 msgr2=0x7f203006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:38.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.830+0000 7f204a51d640 1 --2- 172.21.15.43:0/1808091888 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f203006b4d0 0x7f203006d990 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2034034000 tx=0x7f203402e000 comp rx=0 tx=0).stop 2024-04-25T08:57:38.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.830+0000 7f204a51d640 1 -- 172.21.15.43:0/1808091888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2044161f30 msgr2=0x7f204409dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:38.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.830+0000 7f204a51d640 1 --2- 172.21.15.43:0/1808091888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2044161f30 0x7f204409dcb0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f203c00c780 tx=0x7f203c00cc40 comp rx=0 tx=0).stop 2024-04-25T08:57:38.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.830+0000 7f204a51d640 1 -- 172.21.15.43:0/1808091888 shutdown_connections 2024-04-25T08:57:38.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.830+0000 7f204a51d640 1 --2- 172.21.15.43:0/1808091888 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2044161f30 0x7f204409dcb0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:38.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.830+0000 7f204a51d640 1 --2- 172.21.15.43:0/1808091888 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f204415df50 0x7f204409d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:38.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.830+0000 7f204a51d640 1 --2- 172.21.15.43:0/1808091888 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f203006b4d0 0x7f203006d990 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:38.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.830+0000 7f204a51d640 1 --2- 172.21.15.43:0/1808091888 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20440c7cf0 0x7f2044151f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:38.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.830+0000 7f204a51d640 1 -- 172.21.15.43:0/1808091888 >> 172.21.15.43:0/1808091888 conn(0x7f20440d3a30 msgr2=0x7f2044153e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:38.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.831+0000 7f204a51d640 1 -- 172.21.15.43:0/1808091888 shutdown_connections 2024-04-25T08:57:38.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:38.831+0000 7f204a51d640 1 -- 172.21.15.43:0/1808091888 wait complete. 2024-04-25T08:57:38.831 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi043.front.sepia.ceph.com... 2024-04-25T08:57:38.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:38.831 DEBUG:teuthology.orchestra.run.smithi043:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T08:57:38.854 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[57915]: fuse finished with error 0 and tester_r 0 2024-04-25T08:57:38.859 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-04-25T08:57:38.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:38.860 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:57:39.028 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T08:57:39.029 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:57:39.047 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19508 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:57:39.047 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi193.front.sepia.ceph.com 2024-04-25T08:57:39.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:39.048 DEBUG:teuthology.orchestra.run.smithi193:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T08:57:39.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:57:39.106 INFO:teuthology.orchestra.run.smithi193.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T08:57:39.106 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:57:39.106 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:57:39.122 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2374 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:57:39.122 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-04-25T08:57:39.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:39.122 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T08:57:39.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:57:39.178 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T08:57:39.178 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T08:57:39.178 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:57:39.196 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19508 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:57:39.197 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi193.front.sepia.ceph.com 2024-04-25T08:57:39.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:57:39.197 DEBUG:teuthology.orchestra.run.smithi193:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T08:57:39.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:57:39.256 INFO:teuthology.orchestra.run.smithi193.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T08:57:39.256 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:57:39.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.368+0000 7f9c4145a640 1 Processor -- start 2024-04-25T08:57:39.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.368+0000 7f9c4145a640 1 -- start start 2024-04-25T08:57:39.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.369+0000 7f9c4145a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9c3c0d39d0 0x7f9c3c0d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:39.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.369+0000 7f9c4145a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9c3c0d2500 0x7f9c3c0d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:39.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.369+0000 7f9c4145a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c3c0b6c50 con 0x7f9c3c0ce2a0 2024-04-25T08:57:39.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.369+0000 7f9c4145a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c3c15e010 con 0x7f9c3c0d2500 2024-04-25T08:57:39.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.369+0000 7f9c4145a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c0d2020 con 0x7f9c3c0d39d0 2024-04-25T08:57:39.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.369+0000 7f9c3b7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9c3c0d2500 0x7f9c3c0d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:39.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.369+0000 7f9c3a7fc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f9c3c0ce2a0 0x7f9c3c0d3db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:52750/0 (socket says 172.21.15.43:52750) 2024-04-25T08:57:39.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.369+0000 7f9c3a7fc640 1 -- 172.21.15.43:0/3431546205 learned_addr learned my addr 172.21.15.43:0/3431546205 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:39.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.369+0000 7f9c3affd640 1 --2- 172.21.15.43:0/3431546205 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9c3c0d39d0 0x7f9c3c0d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:39.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.369+0000 7f9c3b7fe640 1 -- 172.21.15.43:0/3431546205 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9c3c0d39d0 msgr2=0x7f9c3c0d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:39.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.369+0000 7f9c3b7fe640 1 --2- 172.21.15.43:0/3431546205 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9c3c0d39d0 0x7f9c3c0d0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:39.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.369+0000 7f9c3b7fe640 1 -- 172.21.15.43:0/3431546205 >> v1:172.21.15.43:6789/0 conn(0x7f9c3c0ce2a0 legacy=0x7f9c3c0d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:39.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.369+0000 7f9c3b7fe640 1 -- 172.21.15.43:0/3431546205 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3c0cec40 con 0x7f9c3c0d2500 2024-04-25T08:57:39.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.369+0000 7f9c39ffb640 1 -- 172.21.15.43:0/3431546205 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3061447588 0 0) 0x7f9c30001110 con 0x7f9c3c0ce2a0 2024-04-25T08:57:39.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.370+0000 7f9c3b7fe640 1 --2- 172.21.15.43:0/3431546205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c3c0d2500 0x7f9c3c0d2990 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f9c34009d90 tx=0x7f9c3402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f87c3c5566d6c0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:39.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.370+0000 7f9c39ffb640 1 -- 172.21.15.43:0/3431546205 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c34031600 con 0x7f9c3c0d2500 2024-04-25T08:57:39.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.370+0000 7f9c39ffb640 1 -- 172.21.15.43:0/3431546205 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c340317a0 con 0x7f9c3c0d2500 2024-04-25T08:57:39.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.370+0000 7f9c39ffb640 1 -- 172.21.15.43:0/3431546205 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c34031b70 con 0x7f9c3c0d2500 2024-04-25T08:57:39.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.371+0000 7f9c4145a640 1 -- 172.21.15.43:0/3431546205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c3c0d2500 msgr2=0x7f9c3c0d2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:39.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.371+0000 7f9c4145a640 1 --2- 172.21.15.43:0/3431546205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c3c0d2500 0x7f9c3c0d2990 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f9c34009d90 tx=0x7f9c3402a5d0 comp rx=0 tx=0).stop 2024-04-25T08:57:39.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.371+0000 7f9c4145a640 1 -- 172.21.15.43:0/3431546205 shutdown_connections 2024-04-25T08:57:39.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.371+0000 7f9c4145a640 1 --2- 172.21.15.43:0/3431546205 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c3c0d2500 0x7f9c3c0d2990 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:39.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.371+0000 7f9c4145a640 1 --2- 172.21.15.43:0/3431546205 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9c3c0d39d0 0x7f9c3c0d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:39.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.371+0000 7f9c4145a640 1 -- 172.21.15.43:0/3431546205 >> 172.21.15.43:0/3431546205 conn(0x7f9c3c06b000 msgr2=0x7f9c3c06b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:39.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.371+0000 7f9c4145a640 1 -- 172.21.15.43:0/3431546205 shutdown_connections 2024-04-25T08:57:39.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.371+0000 7f9c4145a640 1 -- 172.21.15.43:0/3431546205 wait complete. 2024-04-25T08:57:39.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.372+0000 7f9c4145a640 1 Processor -- start 2024-04-25T08:57:39.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.372+0000 7f9c4145a640 1 -- start start 2024-04-25T08:57:39.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.373+0000 7f9c4145a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c3c0ce2a0 0x7f9c3c14ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:39.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.373+0000 7f9c4145a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c3c0d2500 0x7f9c3c0cab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:39.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.373+0000 7f9c4145a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9c3c0d39d0 0x7f9c3c0cb050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:39.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.373+0000 7f9c4145a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c0cf200 con 0x7f9c3c0d2500 2024-04-25T08:57:39.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.373+0000 7f9c4145a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c0b2f00 con 0x7f9c3c0ce2a0 2024-04-25T08:57:39.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.373+0000 7f9c4145a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3c0d2ed0 con 0x7f9c3c0d39d0 2024-04-25T08:57:39.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.373+0000 7f9c3a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c3c0d2500 0x7f9c3c0cab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:39.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.373+0000 7f9c3a7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c3c0d2500 0x7f9c3c0cab10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58330/0 (socket says 172.21.15.43:58330) 2024-04-25T08:57:39.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.373+0000 7f9c3a7fc640 1 -- 172.21.15.43:0/740791446 learned_addr learned my addr 172.21.15.43:0/740791446 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:39.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.373+0000 7f9c3b7fe640 1 --2- 172.21.15.43:0/740791446 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9c3c0d39d0 0x7f9c3c0cb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:39.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.373+0000 7f9c3affd640 1 --2- 172.21.15.43:0/740791446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c3c0ce2a0 0x7f9c3c14ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:39.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.373+0000 7f9c3a7fc640 1 -- 172.21.15.43:0/740791446 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9c3c0d39d0 msgr2=0x7f9c3c0cb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:39.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.373+0000 7f9c3a7fc640 1 --2- 172.21.15.43:0/740791446 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9c3c0d39d0 0x7f9c3c0cb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:39.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.373+0000 7f9c3a7fc640 1 -- 172.21.15.43:0/740791446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c3c0ce2a0 msgr2=0x7f9c3c14ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:39.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.373+0000 7f9c3a7fc640 1 --2- 172.21.15.43:0/740791446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c3c0ce2a0 0x7f9c3c14ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:39.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.373+0000 7f9c3a7fc640 1 -- 172.21.15.43:0/740791446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3c0cb680 con 0x7f9c3c0d2500 2024-04-25T08:57:39.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.373+0000 7f9c3a7fc640 1 --2- 172.21.15.43:0/740791446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c3c0d2500 0x7f9c3c0cab10 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f9c3000c780 tx=0x7f9c3000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:39.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.374+0000 7f9c237fe640 1 -- 172.21.15.43:0/740791446 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c30018070 con 0x7f9c3c0d2500 2024-04-25T08:57:39.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.374+0000 7f9c237fe640 1 -- 172.21.15.43:0/740791446 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c300072d0 con 0x7f9c3c0d2500 2024-04-25T08:57:39.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.374+0000 7f9c4145a640 1 -- 172.21.15.43:0/740791446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c3c0cb960 con 0x7f9c3c0d2500 2024-04-25T08:57:39.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.374+0000 7f9c237fe640 1 -- 172.21.15.43:0/740791446 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c300047e0 con 0x7f9c3c0d2500 2024-04-25T08:57:39.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.374+0000 7f9c4145a640 1 -- 172.21.15.43:0/740791446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c3c0cbf20 con 0x7f9c3c0d2500 2024-04-25T08:57:39.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.375+0000 7f9c4145a640 1 -- 172.21.15.43:0/740791446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c3c15db50 con 0x7f9c3c0d2500 2024-04-25T08:57:39.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.375+0000 7f9c237fe640 1 -- 172.21.15.43:0/740791446 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9c3001b070 con 0x7f9c3c0d2500 2024-04-25T08:57:39.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.375+0000 7f9c237fe640 1 --2- 172.21.15.43:0/740791446 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9c2806b4d0 0x7f9c2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:39.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.376+0000 7f9c237fe640 1 -- 172.21.15.43:0/740791446 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9c30089550 con 0x7f9c3c0d2500 2024-04-25T08:57:39.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.376+0000 7f9c3affd640 1 --2- 172.21.15.43:0/740791446 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9c2806b4d0 0x7f9c2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:39.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.376+0000 7f9c3affd640 1 --2- 172.21.15.43:0/740791446 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9c2806b4d0 0x7f9c2806d990 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c006fd0 tx=0x7f9c2c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:39.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.380+0000 7f9c237fe640 1 -- 172.21.15.43:0/740791446 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9c3c0b6c50 con 0x7f9c3c0d2500 2024-04-25T08:57:39.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.595+0000 7f9c4145a640 1 -- 172.21.15.43:0/740791446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9c3c074ef0 con 0x7f9c3c0d2500 2024-04-25T08:57:39.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.596+0000 7f9c237fe640 1 -- 172.21.15.43:0/740791446 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4945 (secure 0 0 0) 0x7f9c300583d0 con 0x7f9c3c0d2500 2024-04-25T08:57:39.586 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:57:39.586 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1504450503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1504450503},{"type":"v1","addr":"172.21.15.193:6835","nonce":1504450503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5691,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2657133821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2657133821},{"type":"v1","addr":"172.21.15.43:6834","nonce":2657133821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":5694,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/4271431818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":4271431818},{"type":"v1","addr":"172.21.15.43:6835","nonce":4271431818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:57:01.856132+0000","modified":"2024-04-25T08:57:38.526599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5709":{"gid":5709,"name":"d","rank":0,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.193:6837/4164634380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4164634380},{"type":"v1","addr":"172.21.15.193:6837","nonce":4164634380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:57:39.586 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 25 2024-04-25T08:57:39.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.598+0000 7f9c4145a640 1 -- 172.21.15.43:0/740791446 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9c2806b4d0 msgr2=0x7f9c2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:39.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.598+0000 7f9c4145a640 1 --2- 172.21.15.43:0/740791446 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9c2806b4d0 0x7f9c2806d990 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9c2c006fd0 tx=0x7f9c2c008040 comp rx=0 tx=0).stop 2024-04-25T08:57:39.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.598+0000 7f9c4145a640 1 -- 172.21.15.43:0/740791446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c3c0d2500 msgr2=0x7f9c3c0cab10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:39.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.598+0000 7f9c4145a640 1 --2- 172.21.15.43:0/740791446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c3c0d2500 0x7f9c3c0cab10 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f9c3000c780 tx=0x7f9c3000cc40 comp rx=0 tx=0).stop 2024-04-25T08:57:39.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.598+0000 7f9c4145a640 1 -- 172.21.15.43:0/740791446 shutdown_connections 2024-04-25T08:57:39.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.598+0000 7f9c4145a640 1 --2- 172.21.15.43:0/740791446 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9c3c0d39d0 0x7f9c3c0cb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:39.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.598+0000 7f9c4145a640 1 --2- 172.21.15.43:0/740791446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9c3c0d2500 0x7f9c3c0cab10 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:39.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.598+0000 7f9c4145a640 1 --2- 172.21.15.43:0/740791446 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9c2806b4d0 0x7f9c2806d990 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:39.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.598+0000 7f9c4145a640 1 --2- 172.21.15.43:0/740791446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9c3c0ce2a0 0x7f9c3c14ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:39.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.598+0000 7f9c4145a640 1 -- 172.21.15.43:0/740791446 >> 172.21.15.43:0/740791446 conn(0x7f9c3c06b000 msgr2=0x7f9c3c15e740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:39.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.598+0000 7f9c4145a640 1 -- 172.21.15.43:0/740791446 shutdown_connections 2024-04-25T08:57:39.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.598+0000 7f9c4145a640 1 -- 172.21.15.43:0/740791446 wait complete. 2024-04-25T08:57:39.599 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:57:39.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.710+0000 7fe6613d5640 1 Processor -- start 2024-04-25T08:57:39.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.711+0000 7fe6613d5640 1 -- start start 2024-04-25T08:57:39.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.711+0000 7fe6613d5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe65c0cbfc0 0x7fe65c06dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:39.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.711+0000 7fe6613d5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe65c0cd010 0x7fe65c15fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:39.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.711+0000 7fe6613d5640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe65c0b2780 con 0x7fe65c0ccbb0 2024-04-25T08:57:39.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.711+0000 7fe6613d5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe65c0b2a60 con 0x7fe65c0cbfc0 2024-04-25T08:57:39.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.711+0000 7fe6613d5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c0b2d40 con 0x7fe65c0cd010 2024-04-25T08:57:39.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.711+0000 7fe65affd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe65c0cbfc0 0x7fe65c06dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:39.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.712+0000 7fe65affd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe65c0cbfc0 0x7fe65c06dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45616/0 (socket says 172.21.15.43:45616) 2024-04-25T08:57:39.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.712+0000 7fe65affd640 1 -- 172.21.15.43:0/2006721041 learned_addr learned my addr 172.21.15.43:0/2006721041 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:39.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.712+0000 7fe65b7fe640 1 --2- 172.21.15.43:0/2006721041 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe65c0cd010 0x7fe65c15fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:39.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.712+0000 7fe65affd640 1 -- 172.21.15.43:0/2006721041 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe65c0cd010 msgr2=0x7fe65c15fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:39.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.712+0000 7fe65affd640 1 --2- 172.21.15.43:0/2006721041 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe65c0cd010 0x7fe65c15fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:39.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.712+0000 7fe65affd640 1 -- 172.21.15.43:0/2006721041 >> v1:172.21.15.193:6789/0 conn(0x7fe65c0ccbb0 legacy=0x7fe65c15f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:39.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.712+0000 7fe65affd640 1 -- 172.21.15.43:0/2006721041 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe65c0b3020 con 0x7fe65c0cbfc0 2024-04-25T08:57:39.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.712+0000 7fe65b7fe640 1 --2- 172.21.15.43:0/2006721041 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe65c0cd010 0x7fe65c15fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:39.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.713+0000 7fe65affd640 1 --2- 172.21.15.43:0/2006721041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe65c0cbfc0 0x7fe65c06dc60 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe64c009e50 tx=0x7fe64c02a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d164f3552fcac0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:39.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.713+0000 7fe659ffb640 1 -- 172.21.15.43:0/2006721041 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64c036070 con 0x7fe65c0cbfc0 2024-04-25T08:57:39.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.713+0000 7fe659ffb640 1 -- 172.21.15.43:0/2006721041 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe64c031600 con 0x7fe65c0cbfc0 2024-04-25T08:57:39.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.713+0000 7fe659ffb640 1 -- 172.21.15.43:0/2006721041 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64c0319d0 con 0x7fe65c0cbfc0 2024-04-25T08:57:39.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.713+0000 7fe6613d5640 1 -- 172.21.15.43:0/2006721041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe65c0cbfc0 msgr2=0x7fe65c06dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:39.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.713+0000 7fe6613d5640 1 --2- 172.21.15.43:0/2006721041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe65c0cbfc0 0x7fe65c06dc60 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe64c009e50 tx=0x7fe64c02a6d0 comp rx=0 tx=0).stop 2024-04-25T08:57:39.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.713+0000 7fe6613d5640 1 -- 172.21.15.43:0/2006721041 shutdown_connections 2024-04-25T08:57:39.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.713+0000 7fe6613d5640 1 --2- 172.21.15.43:0/2006721041 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe65c0cd010 0x7fe65c15fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:39.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.713+0000 7fe6613d5640 1 --2- 172.21.15.43:0/2006721041 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe65c0cbfc0 0x7fe65c06dc60 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:39.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.713+0000 7fe6613d5640 1 -- 172.21.15.43:0/2006721041 >> 172.21.15.43:0/2006721041 conn(0x7fe65c0b1480 msgr2=0x7fe65c0b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:39.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.713+0000 7fe6613d5640 1 -- 172.21.15.43:0/2006721041 shutdown_connections 2024-04-25T08:57:39.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.713+0000 7fe6613d5640 1 -- 172.21.15.43:0/2006721041 wait complete. 2024-04-25T08:57:39.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.715+0000 7fe6613d5640 1 Processor -- start 2024-04-25T08:57:39.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.715+0000 7fe6613d5640 1 -- start start 2024-04-25T08:57:39.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.716+0000 7fe6613d5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe65c0cbfc0 0x7fe65c141130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:39.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.716+0000 7fe6613d5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe65c0ccbb0 0x7fe65c1e6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:39.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.716+0000 7fe6613d5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe65c0cd010 0x7fe65c1e4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:39.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.716+0000 7fe6613d5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c0cd4a0 con 0x7fe65c0cbfc0 2024-04-25T08:57:39.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.716+0000 7fe6613d5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c0b35e0 con 0x7fe65c0cd010 2024-04-25T08:57:39.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.716+0000 7fe6613d5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe65c06e1a0 con 0x7fe65c0ccbb0 2024-04-25T08:57:39.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.716+0000 7fe65affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe65c0cbfc0 0x7fe65c141130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:39.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.716+0000 7fe65affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe65c0cbfc0 0x7fe65c141130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45620/0 (socket says 172.21.15.43:45620) 2024-04-25T08:57:39.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.716+0000 7fe65affd640 1 -- 172.21.15.43:0/2543274040 learned_addr learned my addr 172.21.15.43:0/2543274040 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:39.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.716+0000 7fe65a7fc640 1 --2- 172.21.15.43:0/2543274040 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe65c0ccbb0 0x7fe65c1e6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:39.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.716+0000 7fe65b7fe640 1 --2- 172.21.15.43:0/2543274040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe65c0cd010 0x7fe65c1e4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:39.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.716+0000 7fe65affd640 1 -- 172.21.15.43:0/2543274040 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe65c0ccbb0 msgr2=0x7fe65c1e6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:39.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.716+0000 7fe65affd640 1 --2- 172.21.15.43:0/2543274040 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe65c0ccbb0 0x7fe65c1e6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:39.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.716+0000 7fe65affd640 1 -- 172.21.15.43:0/2543274040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe65c0cd010 msgr2=0x7fe65c1e4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:39.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.716+0000 7fe65affd640 1 --2- 172.21.15.43:0/2543274040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe65c0cd010 0x7fe65c1e4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:39.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.716+0000 7fe65affd640 1 -- 172.21.15.43:0/2543274040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe65c1e56c0 con 0x7fe65c0cbfc0 2024-04-25T08:57:39.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.716+0000 7fe65a7fc640 1 --2- 172.21.15.43:0/2543274040 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe65c0ccbb0 0x7fe65c1e6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:39.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.716+0000 7fe65b7fe640 1 --2- 172.21.15.43:0/2543274040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe65c0cd010 0x7fe65c1e4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:39.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.716+0000 7fe65affd640 1 --2- 172.21.15.43:0/2543274040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe65c0cbfc0 0x7fe65c141130 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fe64c034040 tx=0x7fe64c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:39.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.716+0000 7fe6437fe640 1 -- 172.21.15.43:0/2543274040 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64c036050 con 0x7fe65c0cbfc0 2024-04-25T08:57:39.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.716+0000 7fe6437fe640 1 -- 172.21.15.43:0/2543274040 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe64c042070 con 0x7fe65c0cbfc0 2024-04-25T08:57:39.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.716+0000 7fe6613d5640 1 -- 172.21.15.43:0/2543274040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe65c1e59a0 con 0x7fe65c0cbfc0 2024-04-25T08:57:39.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.716+0000 7fe6437fe640 1 -- 172.21.15.43:0/2543274040 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64c009510 con 0x7fe65c0cbfc0 2024-04-25T08:57:39.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.716+0000 7fe6613d5640 1 -- 172.21.15.43:0/2543274040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe65c1e5f60 con 0x7fe65c0cbfc0 2024-04-25T08:57:39.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.717+0000 7fe6613d5640 1 -- 172.21.15.43:0/2543274040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe65c074ef0 con 0x7fe65c0cbfc0 2024-04-25T08:57:39.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.718+0000 7fe6437fe640 1 -- 172.21.15.43:0/2543274040 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe64c007c70 con 0x7fe65c0cbfc0 2024-04-25T08:57:39.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.718+0000 7fe6437fe640 1 --2- 172.21.15.43:0/2543274040 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe64806b4d0 0x7fe64806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:39.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.718+0000 7fe6437fe640 1 -- 172.21.15.43:0/2543274040 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe64c0ab350 con 0x7fe65c0cbfc0 2024-04-25T08:57:39.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.719+0000 7fe65a7fc640 1 --2- 172.21.15.43:0/2543274040 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe64806b4d0 0x7fe64806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:39.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.719+0000 7fe65a7fc640 1 --2- 172.21.15.43:0/2543274040 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe64806b4d0 0x7fe64806d990 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe65000a830 tx=0x7fe650008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:39.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.723+0000 7fe6437fe640 1 -- 172.21.15.43:0/2543274040 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe64c07a1d0 con 0x7fe65c0cbfc0 2024-04-25T08:57:39.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.907+0000 7fe6613d5640 1 -- 172.21.15.43:0/2543274040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe65c0af0d0 con 0x7fe65c0cbfc0 2024-04-25T08:57:39.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.908+0000 7fe6437fe640 1 -- 172.21.15.43:0/2543274040 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16577 (secure 0 0 0) 0x7fe64c079ad0 con 0x7fe65c0cbfc0 2024-04-25T08:57:39.898 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:57:39.898 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":46,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:57:03.489946+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:57:00.135489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T08:57:00.830128+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:57:39.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.910+0000 7fe6613d5640 1 -- 172.21.15.43:0/2543274040 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe64806b4d0 msgr2=0x7fe64806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:39.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.910+0000 7fe6613d5640 1 --2- 172.21.15.43:0/2543274040 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe64806b4d0 0x7fe64806d990 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe65000a830 tx=0x7fe650008040 comp rx=0 tx=0).stop 2024-04-25T08:57:39.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.910+0000 7fe6613d5640 1 -- 172.21.15.43:0/2543274040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe65c0cbfc0 msgr2=0x7fe65c141130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:39.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.910+0000 7fe6613d5640 1 --2- 172.21.15.43:0/2543274040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe65c0cbfc0 0x7fe65c141130 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fe64c034040 tx=0x7fe64c004a40 comp rx=0 tx=0).stop 2024-04-25T08:57:39.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.910+0000 7fe6613d5640 1 -- 172.21.15.43:0/2543274040 shutdown_connections 2024-04-25T08:57:39.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.910+0000 7fe6613d5640 1 --2- 172.21.15.43:0/2543274040 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe64806b4d0 0x7fe64806d990 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:39.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.910+0000 7fe6613d5640 1 --2- 172.21.15.43:0/2543274040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe65c0cd010 0x7fe65c1e4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:39.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.910+0000 7fe6613d5640 1 --2- 172.21.15.43:0/2543274040 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe65c0ccbb0 0x7fe65c1e6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:39.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.910+0000 7fe6613d5640 1 --2- 172.21.15.43:0/2543274040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe65c0cbfc0 0x7fe65c141130 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:39.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.910+0000 7fe6613d5640 1 -- 172.21.15.43:0/2543274040 >> 172.21.15.43:0/2543274040 conn(0x7fe65c0b1480 msgr2=0x7fe65c073bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:39.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.910+0000 7fe6613d5640 1 -- 172.21.15.43:0/2543274040 shutdown_connections 2024-04-25T08:57:39.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:39.910+0000 7fe6613d5640 1 -- 172.21.15.43:0/2543274040 wait complete. 2024-04-25T08:57:39.911 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:57:00.135489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:57:00.830128+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '46', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:57:39.912 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:57:40.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.026+0000 7ffb1531d640 1 Processor -- start 2024-04-25T08:57:40.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.026+0000 7ffb1531d640 1 -- start start 2024-04-25T08:57:40.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.027+0000 7ffb1531d640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ffb100b5560 0x7ffb100b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:40.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.027+0000 7ffb1531d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffb100d2230 0x7ffb100d26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:40.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.027+0000 7ffb1531d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb10162610 con 0x7ffb1015e090 2024-04-25T08:57:40.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.027+0000 7ffb1531d640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb100d1da0 con 0x7ffb100b5560 2024-04-25T08:57:40.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.027+0000 7ffb1531d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb100d1750 con 0x7ffb100d2230 2024-04-25T08:57:40.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.027+0000 7ffb0effd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ffb1015e090 0x7ffb10161aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:57582/0 (socket says 172.21.15.43:57582) 2024-04-25T08:57:40.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.027+0000 7ffb0effd640 1 -- 172.21.15.43:0/593656397 learned_addr learned my addr 172.21.15.43:0/593656397 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:40.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.027+0000 7ffb0e7fc640 1 --2- 172.21.15.43:0/593656397 >> v2:172.21.15.193:3300/0 conn(0x7ffb100b5560 0x7ffb100b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:40.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.027+0000 7ffb0dffb640 1 -- 172.21.15.43:0/593656397 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 524326191 0 0) 0x7ffb10162610 con 0x7ffb1015e090 2024-04-25T08:57:40.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.027+0000 7ffb0f7fe640 1 --2- 172.21.15.43:0/593656397 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffb100d2230 0x7ffb100d26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:40.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.027+0000 7ffb0dffb640 1 -- 172.21.15.43:0/593656397 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffafc003520 con 0x7ffb1015e090 2024-04-25T08:57:40.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.028+0000 7ffb0dffb640 1 -- 172.21.15.43:0/593656397 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3029856650 0 0) 0x7ffafc003520 con 0x7ffb1015e090 2024-04-25T08:57:40.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.028+0000 7ffb0dffb640 1 -- 172.21.15.43:0/593656397 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffb100d2230 msgr2=0x7ffb100d26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:40.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.028+0000 7ffb0dffb640 1 --2- 172.21.15.43:0/593656397 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffb100d2230 0x7ffb100d26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:40.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.028+0000 7ffb0dffb640 1 -- 172.21.15.43:0/593656397 >> v2:172.21.15.193:3300/0 conn(0x7ffb100b5560 msgr2=0x7ffb100b59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:40.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.028+0000 7ffb0dffb640 1 --2- 172.21.15.43:0/593656397 >> v2:172.21.15.193:3300/0 conn(0x7ffb100b5560 0x7ffb100b59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:40.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.028+0000 7ffb0dffb640 1 -- 172.21.15.43:0/593656397 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb100d1a30 con 0x7ffb1015e090 2024-04-25T08:57:40.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.028+0000 7ffb0dffb640 1 -- 172.21.15.43:0/593656397 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7ffb080019f0 con 0x7ffb1015e090 2024-04-25T08:57:40.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.028+0000 7ffb0dffb640 1 -- 172.21.15.43:0/593656397 >> v1:172.21.15.43:6789/0 conn(0x7ffb1015e090 legacy=0x7ffb10161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:40.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.028+0000 7ffb0dffb640 1 --2- 172.21.15.43:0/593656397 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffb100d2230 0x7ffafc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:40.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.028+0000 7ffb0dffb640 1 --2- 172.21.15.43:0/593656397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffafc003f30 0x7ffafc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:40.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.028+0000 7ffb0dffb640 1 --2- 172.21.15.43:0/593656397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffafc004d40 0x7ffafc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:40.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.028+0000 7ffb0dffb640 1 -- 172.21.15.43:0/593656397 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb100d1da0 con 0x7ffafc004d40 2024-04-25T08:57:40.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.028+0000 7ffb0dffb640 1 -- 172.21.15.43:0/593656397 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb100d1750 con 0x7ffafc003f30 2024-04-25T08:57:40.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.028+0000 7ffb0dffb640 1 -- 172.21.15.43:0/593656397 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffafc00b9f0 con 0x7ffb100d2230 2024-04-25T08:57:40.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.028+0000 7ffb0f7fe640 1 --2- 172.21.15.43:0/593656397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffafc004d40 0x7ffafc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:40.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.028+0000 7ffb0effd640 1 --2- 172.21.15.43:0/593656397 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffb100d2230 0x7ffafc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:40.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.028+0000 7ffb0e7fc640 1 --2- 172.21.15.43:0/593656397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffafc003f30 0x7ffafc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:40.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.028+0000 7ffb0f7fe640 1 -- 172.21.15.43:0/593656397 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffb100d2230 msgr2=0x7ffafc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:40.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.028+0000 7ffb0f7fe640 1 --2- 172.21.15.43:0/593656397 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffb100d2230 0x7ffafc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:40.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.028+0000 7ffb0f7fe640 1 -- 172.21.15.43:0/593656397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffafc003f30 msgr2=0x7ffafc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:40.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.028+0000 7ffb0f7fe640 1 --2- 172.21.15.43:0/593656397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffafc003f30 0x7ffafc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:40.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.028+0000 7ffb0f7fe640 1 -- 172.21.15.43:0/593656397 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffafc003520 con 0x7ffafc004d40 2024-04-25T08:57:40.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.028+0000 7ffb0e7fc640 1 --2- 172.21.15.43:0/593656397 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffafc003f30 0x7ffafc004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:57:40.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.029+0000 7ffb0f7fe640 1 --2- 172.21.15.43:0/593656397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffafc004d40 0x7ffafc0051c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7ffb00009ac0 tx=0x7ffafc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b279f4afff7b9c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:40.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.029+0000 7ffb0dffb640 1 -- 172.21.15.43:0/593656397 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb00036070 con 0x7ffafc004d40 2024-04-25T08:57:40.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.029+0000 7ffb0dffb640 1 -- 172.21.15.43:0/593656397 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb00031400 con 0x7ffafc004d40 2024-04-25T08:57:40.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.029+0000 7ffb0dffb640 1 -- 172.21.15.43:0/593656397 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb00009690 con 0x7ffafc004d40 2024-04-25T08:57:40.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.030+0000 7ffb1531d640 1 -- 172.21.15.43:0/593656397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffafc004d40 msgr2=0x7ffafc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:40.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.030+0000 7ffb1531d640 1 --2- 172.21.15.43:0/593656397 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffafc004d40 0x7ffafc0051c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7ffb00009ac0 tx=0x7ffafc005830 comp rx=0 tx=0).stop 2024-04-25T08:57:40.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.030+0000 7ffb0f7fe640 1 -- 172.21.15.43:0/593656397 reap_dead start 2024-04-25T08:57:40.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.030+0000 7ffb1531d640 1 -- 172.21.15.43:0/593656397 shutdown_connections 2024-04-25T08:57:40.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.030+0000 7ffb1531d640 1 -- 172.21.15.43:0/593656397 >> 172.21.15.43:0/593656397 conn(0x7ffb100d0560 msgr2=0x7ffb100d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:40.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.030+0000 7ffb1531d640 1 -- 172.21.15.43:0/593656397 shutdown_connections 2024-04-25T08:57:40.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.030+0000 7ffb1531d640 1 -- 172.21.15.43:0/593656397 wait complete. 2024-04-25T08:57:40.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.031+0000 7ffb1531d640 1 Processor -- start 2024-04-25T08:57:40.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.031+0000 7ffb1531d640 1 -- start start 2024-04-25T08:57:40.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.032+0000 7ffb1531d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffafc004d40 0x7ffb100cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:40.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.032+0000 7ffb1531d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb100cabb0 0x7ffb100ce0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:40.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.032+0000 7ffb1531d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffb101f2ac0 0x7ffb101f2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:40.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.032+0000 7ffb1531d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb101621b0 con 0x7ffb100cabb0 2024-04-25T08:57:40.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.032+0000 7ffb1531d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb100b3280 con 0x7ffafc004d40 2024-04-25T08:57:40.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.032+0000 7ffb1531d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb10162330 con 0x7ffb101f2ac0 2024-04-25T08:57:40.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.032+0000 7ffb0e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb100cabb0 0x7ffb100ce0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:40.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.032+0000 7ffb0e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb100cabb0 0x7ffb100ce0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45646/0 (socket says 172.21.15.43:45646) 2024-04-25T08:57:40.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.032+0000 7ffb0e7fc640 1 -- 172.21.15.43:0/3653975621 learned_addr learned my addr 172.21.15.43:0/3653975621 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:40.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.032+0000 7ffb0effd640 1 --2- 172.21.15.43:0/3653975621 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffafc004d40 0x7ffb100cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:40.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.032+0000 7ffb0f7fe640 1 --2- 172.21.15.43:0/3653975621 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffb101f2ac0 0x7ffb101f2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:40.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.032+0000 7ffb0e7fc640 1 -- 172.21.15.43:0/3653975621 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffb101f2ac0 msgr2=0x7ffb101f2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:40.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.032+0000 7ffb0e7fc640 1 --2- 172.21.15.43:0/3653975621 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffb101f2ac0 0x7ffb101f2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:40.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.032+0000 7ffb0e7fc640 1 -- 172.21.15.43:0/3653975621 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffafc004d40 msgr2=0x7ffb100cbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:40.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.032+0000 7ffb0e7fc640 1 --2- 172.21.15.43:0/3653975621 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffafc004d40 0x7ffb100cbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:40.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.032+0000 7ffb0e7fc640 1 -- 172.21.15.43:0/3653975621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb100cc220 con 0x7ffb100cabb0 2024-04-25T08:57:40.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.032+0000 7ffb0f7fe640 1 --2- 172.21.15.43:0/3653975621 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffb101f2ac0 0x7ffb101f2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:40.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.032+0000 7ffb0e7fc640 1 --2- 172.21.15.43:0/3653975621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb100cabb0 0x7ffb100ce0a0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8011cd0 tx=0x7ffaf800f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:40.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.033+0000 7ffb077fe640 1 -- 172.21.15.43:0/3653975621 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf800fd20 con 0x7ffb100cabb0 2024-04-25T08:57:40.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.033+0000 7ffb077fe640 1 -- 172.21.15.43:0/3653975621 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaf8014440 con 0x7ffb100cabb0 2024-04-25T08:57:40.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.033+0000 7ffb1531d640 1 -- 172.21.15.43:0/3653975621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb100cc500 con 0x7ffb100cabb0 2024-04-25T08:57:40.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.033+0000 7ffb077fe640 1 -- 172.21.15.43:0/3653975621 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8014810 con 0x7ffb100cabb0 2024-04-25T08:57:40.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.033+0000 7ffb1531d640 1 -- 172.21.15.43:0/3653975621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb1015e7f0 con 0x7ffb100cabb0 2024-04-25T08:57:40.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.034+0000 7ffb1531d640 1 -- 172.21.15.43:0/3653975621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb1006bf00 con 0x7ffb100cabb0 2024-04-25T08:57:40.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.034+0000 7ffb077fe640 1 -- 172.21.15.43:0/3653975621 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffaf8012040 con 0x7ffb100cabb0 2024-04-25T08:57:40.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.034+0000 7ffb077fe640 1 --2- 172.21.15.43:0/3653975621 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ffaf006b500 0x7ffaf006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:40.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.035+0000 7ffb077fe640 1 -- 172.21.15.43:0/3653975621 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffaf808ab40 con 0x7ffb100cabb0 2024-04-25T08:57:40.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.035+0000 7ffb0effd640 1 --2- 172.21.15.43:0/3653975621 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ffaf006b500 0x7ffaf006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:40.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.035+0000 7ffb0effd640 1 --2- 172.21.15.43:0/3653975621 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ffaf006b500 0x7ffaf006d9c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ffb080013d0 tx=0x7ffb08006040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:40.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.039+0000 7ffb077fe640 1 -- 172.21.15.43:0/3653975621 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffb1006bf00 con 0x7ffb100cabb0 2024-04-25T08:57:40.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.222+0000 7ffb1531d640 1 -- 172.21.15.43:0/3653975621 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffb10074f60 con 0x7ffb100cabb0 2024-04-25T08:57:40.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.222+0000 7ffb077fe640 1 -- 172.21.15.43:0/3653975621 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffaf8019030 con 0x7ffb100cabb0 2024-04-25T08:57:40.212 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T08:57:40.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.225+0000 7ffb1531d640 1 -- 172.21.15.43:0/3653975621 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ffaf006b500 msgr2=0x7ffaf006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:40.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.225+0000 7ffb1531d640 1 --2- 172.21.15.43:0/3653975621 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ffaf006b500 0x7ffaf006d9c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ffb080013d0 tx=0x7ffb08006040 comp rx=0 tx=0).stop 2024-04-25T08:57:40.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.225+0000 7ffb1531d640 1 -- 172.21.15.43:0/3653975621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb100cabb0 msgr2=0x7ffb100ce0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:40.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.225+0000 7ffb1531d640 1 --2- 172.21.15.43:0/3653975621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb100cabb0 0x7ffb100ce0a0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8011cd0 tx=0x7ffaf800f610 comp rx=0 tx=0).stop 2024-04-25T08:57:40.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.225+0000 7ffb1531d640 1 -- 172.21.15.43:0/3653975621 shutdown_connections 2024-04-25T08:57:40.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.225+0000 7ffb1531d640 1 --2- 172.21.15.43:0/3653975621 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffb101f2ac0 0x7ffb101f2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:40.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.225+0000 7ffb1531d640 1 --2- 172.21.15.43:0/3653975621 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffb100cabb0 0x7ffb100ce0a0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:40.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.225+0000 7ffb1531d640 1 --2- 172.21.15.43:0/3653975621 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ffaf006b500 0x7ffaf006d9c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:40.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.225+0000 7ffb1531d640 1 --2- 172.21.15.43:0/3653975621 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffafc004d40 0x7ffb100cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:40.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.225+0000 7ffb1531d640 1 -- 172.21.15.43:0/3653975621 >> 172.21.15.43:0/3653975621 conn(0x7ffb100d0560 msgr2=0x7ffb100d2640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:40.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.225+0000 7ffb1531d640 1 -- 172.21.15.43:0/3653975621 shutdown_connections 2024-04-25T08:57:40.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.225+0000 7ffb1531d640 1 -- 172.21.15.43:0/3653975621 wait complete. 2024-04-25T08:57:40.226 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:57:40.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.340+0000 7fbc642cb640 1 Processor -- start 2024-04-25T08:57:40.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.340+0000 7fbc642cb640 1 -- start start 2024-04-25T08:57:40.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.341+0000 7fbc642cb640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc5c0b6110 0x7fbc5c0d1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:40.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.341+0000 7fbc642cb640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbc5c0d2340 0x7fbc5c1656d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:40.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.341+0000 7fbc642cb640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc5c0b32a0 con 0x7fbc5c0d1a20 2024-04-25T08:57:40.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.341+0000 7fbc642cb640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc5c0cf430 con 0x7fbc5c0d2340 2024-04-25T08:57:40.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.341+0000 7fbc642cb640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c15bf90 con 0x7fbc5c0b6110 2024-04-25T08:57:40.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.341+0000 7fbc62868640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbc5c0d2340 0x7fbc5c1656d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:40.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.341+0000 7fbc62067640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fbc5c0d1a20 0x7fbc5c0b2b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:57592/0 (socket says 172.21.15.43:57592) 2024-04-25T08:57:40.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.341+0000 7fbc62067640 1 -- 172.21.15.43:0/2696300606 learned_addr learned my addr 172.21.15.43:0/2696300606 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:40.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.341+0000 7fbc62868640 1 --2- 172.21.15.43:0/2696300606 >> v2:172.21.15.43:3300/0 conn(0x7fbc5c0d2340 0x7fbc5c1656d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45660/0 (socket says 172.21.15.43:45660) 2024-04-25T08:57:40.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.342+0000 7fbc61866640 1 --2- 172.21.15.43:0/2696300606 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc5c0b6110 0x7fbc5c0d1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:40.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.342+0000 7fbc62868640 1 -- 172.21.15.43:0/2696300606 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc5c0b6110 msgr2=0x7fbc5c0d1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:40.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.342+0000 7fbc62868640 1 --2- 172.21.15.43:0/2696300606 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc5c0b6110 0x7fbc5c0d1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:40.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.342+0000 7fbc62868640 1 -- 172.21.15.43:0/2696300606 >> v1:172.21.15.43:6789/0 conn(0x7fbc5c0d1a20 legacy=0x7fbc5c0b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:40.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.342+0000 7fbc61065640 1 -- 172.21.15.43:0/2696300606 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1039886792 0 0) 0x7fbc5c0b32a0 con 0x7fbc5c0d1a20 2024-04-25T08:57:40.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.342+0000 7fbc62868640 1 -- 172.21.15.43:0/2696300606 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc5c0cee70 con 0x7fbc5c0d2340 2024-04-25T08:57:40.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.342+0000 7fbc61866640 1 --2- 172.21.15.43:0/2696300606 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc5c0b6110 0x7fbc5c0d1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:40.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.342+0000 7fbc62868640 1 --2- 172.21.15.43:0/2696300606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5c0d2340 0x7fbc5c1656d0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fbc58009d90 tx=0x7fbc5802a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de4b9c56f2f8cf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:40.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.343+0000 7fbc61065640 1 -- 172.21.15.43:0/2696300606 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc58007740 con 0x7fbc5c0d2340 2024-04-25T08:57:40.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.343+0000 7fbc61065640 1 -- 172.21.15.43:0/2696300606 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc580078e0 con 0x7fbc5c0d2340 2024-04-25T08:57:40.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.343+0000 7fbc61065640 1 -- 172.21.15.43:0/2696300606 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc58007cb0 con 0x7fbc5c0d2340 2024-04-25T08:57:40.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.343+0000 7fbc642cb640 1 -- 172.21.15.43:0/2696300606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5c0d2340 msgr2=0x7fbc5c1656d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:40.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.343+0000 7fbc642cb640 1 --2- 172.21.15.43:0/2696300606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5c0d2340 0x7fbc5c1656d0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fbc58009d90 tx=0x7fbc5802a530 comp rx=0 tx=0).stop 2024-04-25T08:57:40.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.343+0000 7fbc642cb640 1 -- 172.21.15.43:0/2696300606 shutdown_connections 2024-04-25T08:57:40.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.343+0000 7fbc642cb640 1 --2- 172.21.15.43:0/2696300606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5c0d2340 0x7fbc5c1656d0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:40.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.343+0000 7fbc642cb640 1 --2- 172.21.15.43:0/2696300606 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc5c0b6110 0x7fbc5c0d1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:40.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.343+0000 7fbc642cb640 1 -- 172.21.15.43:0/2696300606 >> 172.21.15.43:0/2696300606 conn(0x7fbc5c072aa0 msgr2=0x7fbc5c09aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:40.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.343+0000 7fbc642cb640 1 -- 172.21.15.43:0/2696300606 shutdown_connections 2024-04-25T08:57:40.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.343+0000 7fbc642cb640 1 -- 172.21.15.43:0/2696300606 wait complete. 2024-04-25T08:57:40.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.344+0000 7fbc642cb640 1 Processor -- start 2024-04-25T08:57:40.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.345+0000 7fbc642cb640 1 -- start start 2024-04-25T08:57:40.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.345+0000 7fbc642cb640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc5c0b6110 0x7fbc5c06c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:40.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.345+0000 7fbc642cb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc5c0d1a20 0x7fbc5c158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:40.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.345+0000 7fbc642cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5c0d2340 0x7fbc5c14a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:40.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.345+0000 7fbc642cb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c0b4be0 con 0x7fbc5c0d2340 2024-04-25T08:57:40.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.345+0000 7fbc642cb640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c0b4900 con 0x7fbc5c0d1a20 2024-04-25T08:57:40.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.345+0000 7fbc642cb640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5c0cea60 con 0x7fbc5c0b6110 2024-04-25T08:57:40.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.345+0000 7fbc62868640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5c0d2340 0x7fbc5c14a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:40.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.345+0000 7fbc62868640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5c0d2340 0x7fbc5c14a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45666/0 (socket says 172.21.15.43:45666) 2024-04-25T08:57:40.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.345+0000 7fbc62868640 1 -- 172.21.15.43:0/1079082892 learned_addr learned my addr 172.21.15.43:0/1079082892 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:40.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.346+0000 7fbc61866640 1 --2- 172.21.15.43:0/1079082892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc5c0d1a20 0x7fbc5c158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:40.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.346+0000 7fbc62067640 1 --2- 172.21.15.43:0/1079082892 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc5c0b6110 0x7fbc5c06c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:40.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.346+0000 7fbc62868640 1 -- 172.21.15.43:0/1079082892 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc5c0b6110 msgr2=0x7fbc5c06c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:40.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.346+0000 7fbc62868640 1 --2- 172.21.15.43:0/1079082892 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc5c0b6110 0x7fbc5c06c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:40.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.346+0000 7fbc62868640 1 -- 172.21.15.43:0/1079082892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc5c0d1a20 msgr2=0x7fbc5c158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:40.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.346+0000 7fbc62868640 1 --2- 172.21.15.43:0/1079082892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc5c0d1a20 0x7fbc5c158d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:40.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.346+0000 7fbc62868640 1 -- 172.21.15.43:0/1079082892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc5c06d590 con 0x7fbc5c0d2340 2024-04-25T08:57:40.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.346+0000 7fbc62868640 1 --2- 172.21.15.43:0/1079082892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5c0d2340 0x7fbc5c14a450 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fbc58009eb0 tx=0x7fbc58004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:40.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.346+0000 7fbc52ffd640 1 -- 172.21.15.43:0/1079082892 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc58040070 con 0x7fbc5c0d2340 2024-04-25T08:57:40.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.346+0000 7fbc52ffd640 1 -- 172.21.15.43:0/1079082892 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc58031c90 con 0x7fbc5c0d2340 2024-04-25T08:57:40.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.346+0000 7fbc52ffd640 1 -- 172.21.15.43:0/1079082892 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc58045680 con 0x7fbc5c0d2340 2024-04-25T08:57:40.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.346+0000 7fbc642cb640 1 -- 172.21.15.43:0/1079082892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc5c06d870 con 0x7fbc5c0d2340 2024-04-25T08:57:40.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.346+0000 7fbc642cb640 1 -- 172.21.15.43:0/1079082892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc5c06db50 con 0x7fbc5c0d2340 2024-04-25T08:57:40.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.347+0000 7fbc642cb640 1 -- 172.21.15.43:0/1079082892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc5c074f60 con 0x7fbc5c0d2340 2024-04-25T08:57:40.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.348+0000 7fbc52ffd640 1 -- 172.21.15.43:0/1079082892 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbc58039040 con 0x7fbc5c0d2340 2024-04-25T08:57:40.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.348+0000 7fbc52ffd640 1 --2- 172.21.15.43:0/1079082892 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbc4006b4d0 0x7fbc4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:40.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.348+0000 7fbc52ffd640 1 -- 172.21.15.43:0/1079082892 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbc580aba90 con 0x7fbc5c0d2340 2024-04-25T08:57:40.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.348+0000 7fbc62067640 1 --2- 172.21.15.43:0/1079082892 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbc4006b4d0 0x7fbc4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:40.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.349+0000 7fbc62067640 1 --2- 172.21.15.43:0/1079082892 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbc4006b4d0 0x7fbc4006d990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbc54000f60 tx=0x7fbc54009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:40.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.352+0000 7fbc52ffd640 1 -- 172.21.15.43:0/1079082892 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbc5807a410 con 0x7fbc5c0d2340 2024-04-25T08:57:40.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.533+0000 7fbc642cb640 1 -- 172.21.15.43:0/1079082892 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbc5c158580 con 0x7fbc5c0d2340 2024-04-25T08:57:40.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.534+0000 7fbc52ffd640 1 -- 172.21.15.43:0/1079082892 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v46) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbc5807e4b0 con 0x7fbc5c0d2340 2024-04-25T08:57:40.523 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T08:57:40.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.535+0000 7fbc642cb640 1 -- 172.21.15.43:0/1079082892 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbc4006b4d0 msgr2=0x7fbc4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:40.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.535+0000 7fbc642cb640 1 --2- 172.21.15.43:0/1079082892 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbc4006b4d0 0x7fbc4006d990 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbc54000f60 tx=0x7fbc54009040 comp rx=0 tx=0).stop 2024-04-25T08:57:40.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.535+0000 7fbc642cb640 1 -- 172.21.15.43:0/1079082892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5c0d2340 msgr2=0x7fbc5c14a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:40.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.535+0000 7fbc642cb640 1 --2- 172.21.15.43:0/1079082892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5c0d2340 0x7fbc5c14a450 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fbc58009eb0 tx=0x7fbc58004a40 comp rx=0 tx=0).stop 2024-04-25T08:57:40.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.535+0000 7fbc642cb640 1 -- 172.21.15.43:0/1079082892 shutdown_connections 2024-04-25T08:57:40.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.535+0000 7fbc642cb640 1 --2- 172.21.15.43:0/1079082892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc5c0d2340 0x7fbc5c14a450 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:40.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.536+0000 7fbc642cb640 1 --2- 172.21.15.43:0/1079082892 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbc4006b4d0 0x7fbc4006d990 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:40.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.536+0000 7fbc642cb640 1 --2- 172.21.15.43:0/1079082892 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc5c0d1a20 0x7fbc5c158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:40.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.536+0000 7fbc642cb640 1 --2- 172.21.15.43:0/1079082892 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc5c0b6110 0x7fbc5c06c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:40.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.536+0000 7fbc642cb640 1 -- 172.21.15.43:0/1079082892 >> 172.21.15.43:0/1079082892 conn(0x7fbc5c072aa0 msgr2=0x7fbc5c0b2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:40.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.536+0000 7fbc642cb640 1 -- 172.21.15.43:0/1079082892 shutdown_connections 2024-04-25T08:57:40.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.536+0000 7fbc642cb640 1 -- 172.21.15.43:0/1079082892 wait complete. 2024-04-25T08:57:40.536 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:57:40.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.651+0000 7fdc2c130640 1 Processor -- start 2024-04-25T08:57:40.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.651+0000 7fdc2c130640 1 -- start start 2024-04-25T08:57:40.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.652+0000 7fdc2c130640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdc2406f6a0 0x7fdc240b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:40.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.652+0000 7fdc2c130640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fdc240cd470 0x7fdc2406c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:40.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.652+0000 7fdc2c130640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc240d2620 con 0x7fdc2415ddd0 2024-04-25T08:57:40.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.652+0000 7fdc2c130640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc240b0b80 con 0x7fdc2406f6a0 2024-04-25T08:57:40.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.652+0000 7fdc2c130640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc240b0e60 con 0x7fdc240cd470 2024-04-25T08:57:40.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.652+0000 7fdc296cb640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdc2406f6a0 0x7fdc240b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:40.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.652+0000 7fdc296cb640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdc2406f6a0 0x7fdc240b3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45680/0 (socket says 172.21.15.43:45680) 2024-04-25T08:57:40.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.652+0000 7fdc296cb640 1 -- 172.21.15.43:0/1635631239 learned_addr learned my addr 172.21.15.43:0/1635631239 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:40.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.652+0000 7fdc2a6cd640 1 --2- 172.21.15.43:0/1635631239 >> v2:172.21.15.193:3300/0 conn(0x7fdc240cd470 0x7fdc2406c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:40.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.652+0000 7fdc296cb640 1 -- 172.21.15.43:0/1635631239 >> v2:172.21.15.193:3300/0 conn(0x7fdc240cd470 msgr2=0x7fdc2406c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:40.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.652+0000 7fdc296cb640 1 --2- 172.21.15.43:0/1635631239 >> v2:172.21.15.193:3300/0 conn(0x7fdc240cd470 0x7fdc2406c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:40.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.652+0000 7fdc296cb640 1 -- 172.21.15.43:0/1635631239 >> v1:172.21.15.193:6789/0 conn(0x7fdc2415ddd0 legacy=0x7fdc2406ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:40.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.652+0000 7fdc296cb640 1 -- 172.21.15.43:0/1635631239 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc240cbf40 con 0x7fdc2406f6a0 2024-04-25T08:57:40.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.653+0000 7fdc296cb640 1 --2- 172.21.15.43:0/1635631239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc2406f6a0 0x7fdc240b3dd0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fdc2406ee30 tx=0x7fdc1402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc5091b5a9dd0ef7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:40.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.653+0000 7fdc28eca640 1 -- 172.21.15.43:0/1635631239 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc14037070 con 0x7fdc2406f6a0 2024-04-25T08:57:40.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.653+0000 7fdc28eca640 1 -- 172.21.15.43:0/1635631239 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc14032440 con 0x7fdc2406f6a0 2024-04-25T08:57:40.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.653+0000 7fdc28eca640 1 -- 172.21.15.43:0/1635631239 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc14032810 con 0x7fdc2406f6a0 2024-04-25T08:57:40.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.654+0000 7fdc2c130640 1 -- 172.21.15.43:0/1635631239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc2406f6a0 msgr2=0x7fdc240b3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:40.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.654+0000 7fdc2c130640 1 --2- 172.21.15.43:0/1635631239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc2406f6a0 0x7fdc240b3dd0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fdc2406ee30 tx=0x7fdc1402a230 comp rx=0 tx=0).stop 2024-04-25T08:57:40.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.654+0000 7fdc2c130640 1 -- 172.21.15.43:0/1635631239 shutdown_connections 2024-04-25T08:57:40.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.654+0000 7fdc2c130640 1 --2- 172.21.15.43:0/1635631239 >> v2:172.21.15.193:3300/0 conn(0x7fdc240cd470 0x7fdc2406c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:40.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.654+0000 7fdc2c130640 1 --2- 172.21.15.43:0/1635631239 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc2406f6a0 0x7fdc240b3dd0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:40.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.654+0000 7fdc2c130640 1 -- 172.21.15.43:0/1635631239 >> 172.21.15.43:0/1635631239 conn(0x7fdc240d17e0 msgr2=0x7fdc240d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:40.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.654+0000 7fdc2c130640 1 -- 172.21.15.43:0/1635631239 shutdown_connections 2024-04-25T08:57:40.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.654+0000 7fdc2c130640 1 -- 172.21.15.43:0/1635631239 wait complete. 2024-04-25T08:57:40.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.655+0000 7fdc2c130640 1 Processor -- start 2024-04-25T08:57:40.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.655+0000 7fdc2c130640 1 -- start start 2024-04-25T08:57:40.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.656+0000 7fdc2c130640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc2406f6a0 0x7fdc240b4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:40.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.656+0000 7fdc2c130640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc240cd470 0x7fdc240cb020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:40.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.656+0000 7fdc2c130640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdc2415ddd0 0x7fdc2414de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:40.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.656+0000 7fdc2c130640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2406cdb0 con 0x7fdc2406f6a0 2024-04-25T08:57:40.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.656+0000 7fdc2c130640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2406cc30 con 0x7fdc240cd470 2024-04-25T08:57:40.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.656+0000 7fdc2c130640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2406cf30 con 0x7fdc2415ddd0 2024-04-25T08:57:40.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.656+0000 7fdc29ecc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc2406f6a0 0x7fdc240b4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:40.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.656+0000 7fdc29ecc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc2406f6a0 0x7fdc240b4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45690/0 (socket says 172.21.15.43:45690) 2024-04-25T08:57:40.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.656+0000 7fdc29ecc640 1 -- 172.21.15.43:0/5962636 learned_addr learned my addr 172.21.15.43:0/5962636 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:40.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.656+0000 7fdc29ecc640 1 -- 172.21.15.43:0/5962636 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdc2415ddd0 msgr2=0x7fdc2414de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:40.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.656+0000 7fdc29ecc640 1 --2- 172.21.15.43:0/5962636 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdc2415ddd0 0x7fdc2414de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:40.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.656+0000 7fdc296cb640 1 --2- 172.21.15.43:0/5962636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc240cd470 0x7fdc240cb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:40.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.656+0000 7fdc29ecc640 1 -- 172.21.15.43:0/5962636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc240cd470 msgr2=0x7fdc240cb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:40.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.656+0000 7fdc29ecc640 1 --2- 172.21.15.43:0/5962636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc240cd470 0x7fdc240cb020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:40.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.656+0000 7fdc29ecc640 1 -- 172.21.15.43:0/5962636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc240b1fb0 con 0x7fdc2406f6a0 2024-04-25T08:57:40.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.656+0000 7fdc29ecc640 1 --2- 172.21.15.43:0/5962636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc2406f6a0 0x7fdc240b4670 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fdc2000b3f0 tx=0x7fdc2000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:40.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.656+0000 7fdc1a7fc640 1 -- 172.21.15.43:0/5962636 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc20015020 con 0x7fdc2406f6a0 2024-04-25T08:57:40.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.657+0000 7fdc1a7fc640 1 -- 172.21.15.43:0/5962636 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc20011830 con 0x7fdc2406f6a0 2024-04-25T08:57:40.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.657+0000 7fdc2c130640 1 -- 172.21.15.43:0/5962636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc240b2290 con 0x7fdc2406f6a0 2024-04-25T08:57:40.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.657+0000 7fdc1a7fc640 1 -- 172.21.15.43:0/5962636 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc20011c20 con 0x7fdc2406f6a0 2024-04-25T08:57:40.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.657+0000 7fdc2c130640 1 -- 172.21.15.43:0/5962636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc240b2850 con 0x7fdc2406f6a0 2024-04-25T08:57:40.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.657+0000 7fdc2c130640 1 -- 172.21.15.43:0/5962636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc24074f60 con 0x7fdc2406f6a0 2024-04-25T08:57:40.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.658+0000 7fdc1a7fc640 1 -- 172.21.15.43:0/5962636 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdc20018020 con 0x7fdc2406f6a0 2024-04-25T08:57:40.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.658+0000 7fdc1a7fc640 1 --2- 172.21.15.43:0/5962636 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdc0806b4d0 0x7fdc0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:40.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.659+0000 7fdc1a7fc640 1 -- 172.21.15.43:0/5962636 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdc20089e20 con 0x7fdc2406f6a0 2024-04-25T08:57:40.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.659+0000 7fdc296cb640 1 --2- 172.21.15.43:0/5962636 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdc0806b4d0 0x7fdc0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:40.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.659+0000 7fdc296cb640 1 --2- 172.21.15.43:0/5962636 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdc0806b4d0 0x7fdc0806d990 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fdc14034000 tx=0x7fdc1402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:40.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.663+0000 7fdc1a7fc640 1 -- 172.21.15.43:0/5962636 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdc2001f030 con 0x7fdc2406f6a0 2024-04-25T08:57:40.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.844+0000 7fdc2c130640 1 -- 172.21.15.43:0/5962636 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdc2406f3a0 con 0x7fdc2406f6a0 2024-04-25T08:57:40.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.844+0000 7fdc1a7fc640 1 -- 172.21.15.43:0/5962636 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v46) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdc200172c0 con 0x7fdc2406f6a0 2024-04-25T08:57:40.834 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T08:57:40.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.847+0000 7fdc2c130640 1 -- 172.21.15.43:0/5962636 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdc0806b4d0 msgr2=0x7fdc0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:40.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.847+0000 7fdc2c130640 1 --2- 172.21.15.43:0/5962636 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdc0806b4d0 0x7fdc0806d990 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fdc14034000 tx=0x7fdc1402e000 comp rx=0 tx=0).stop 2024-04-25T08:57:40.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.847+0000 7fdc2c130640 1 -- 172.21.15.43:0/5962636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc2406f6a0 msgr2=0x7fdc240b4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:40.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.847+0000 7fdc2c130640 1 --2- 172.21.15.43:0/5962636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc2406f6a0 0x7fdc240b4670 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fdc2000b3f0 tx=0x7fdc2000b8b0 comp rx=0 tx=0).stop 2024-04-25T08:57:40.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.847+0000 7fdc2c130640 1 -- 172.21.15.43:0/5962636 shutdown_connections 2024-04-25T08:57:40.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.847+0000 7fdc2c130640 1 --2- 172.21.15.43:0/5962636 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdc2415ddd0 0x7fdc2414de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:40.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.847+0000 7fdc2c130640 1 --2- 172.21.15.43:0/5962636 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdc0806b4d0 0x7fdc0806d990 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:40.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.847+0000 7fdc2c130640 1 --2- 172.21.15.43:0/5962636 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdc240cd470 0x7fdc240cb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:40.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.847+0000 7fdc2c130640 1 --2- 172.21.15.43:0/5962636 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdc2406f6a0 0x7fdc240b4670 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:40.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.847+0000 7fdc2c130640 1 -- 172.21.15.43:0/5962636 >> 172.21.15.43:0/5962636 conn(0x7fdc240d17e0 msgr2=0x7fdc2415bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:40.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.847+0000 7fdc2c130640 1 -- 172.21.15.43:0/5962636 shutdown_connections 2024-04-25T08:57:40.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.847+0000 7fdc2c130640 1 -- 172.21.15.43:0/5962636 wait complete. 2024-04-25T08:57:40.847 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:57:40.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.959+0000 7f2c05441640 1 Processor -- start 2024-04-25T08:57:40.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.959+0000 7f2c05441640 1 -- start start 2024-04-25T08:57:40.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.960+0000 7f2c05441640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f2c000d35c0 0x7f2c000ce260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:40.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.960+0000 7f2c05441640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c000ce7a0 0x7f2c000b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:40.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.960+0000 7f2c05441640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c000cec30 con 0x7f2c0015dd00 2024-04-25T08:57:40.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.960+0000 7f2c05441640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c000d24e0 con 0x7f2c000d35c0 2024-04-25T08:57:40.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.960+0000 7f2c05441640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c0015e0e0 con 0x7f2c000ce7a0 2024-04-25T08:57:40.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.960+0000 7f2bfeffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f2c0015dd00 0x7f2c000d2eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:57606/0 (socket says 172.21.15.43:57606) 2024-04-25T08:57:40.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.960+0000 7f2bfeffd640 1 -- 172.21.15.43:0/4245559809 learned_addr learned my addr 172.21.15.43:0/4245559809 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:40.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.960+0000 7f2bfe7fc640 1 --2- 172.21.15.43:0/4245559809 >> v2:172.21.15.193:3300/0 conn(0x7f2c000d35c0 0x7f2c000ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:40.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.960+0000 7f2bff7fe640 1 --2- 172.21.15.43:0/4245559809 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c000ce7a0 0x7f2c000b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:40.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.960+0000 7f2bfdffb640 1 -- 172.21.15.43:0/4245559809 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 925577924 0 0) 0x7f2bf0001110 con 0x7f2c0015dd00 2024-04-25T08:57:40.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.960+0000 7f2bfdffb640 1 -- 172.21.15.43:0/4245559809 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2be8003520 con 0x7f2c0015dd00 2024-04-25T08:57:40.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.960+0000 7f2bfe7fc640 1 -- 172.21.15.43:0/4245559809 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c000ce7a0 msgr2=0x7f2c000b6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:40.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.960+0000 7f2bfdffb640 1 -- 172.21.15.43:0/4245559809 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1129381895 0 0) 0x7f2be8003520 con 0x7f2c0015dd00 2024-04-25T08:57:40.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.960+0000 7f2bfe7fc640 1 --2- 172.21.15.43:0/4245559809 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c000ce7a0 0x7f2c000b6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:40.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.960+0000 7f2bfe7fc640 1 -- 172.21.15.43:0/4245559809 >> v1:172.21.15.43:6789/0 conn(0x7f2c0015dd00 legacy=0x7f2c000d2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:40.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.960+0000 7f2bfe7fc640 1 -- 172.21.15.43:0/4245559809 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c000ceeb0 con 0x7f2c000d35c0 2024-04-25T08:57:40.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.960+0000 7f2bff7fe640 1 --2- 172.21.15.43:0/4245559809 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c000ce7a0 0x7f2c000b6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:57:40.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.961+0000 7f2bfe7fc640 1 --2- 172.21.15.43:0/4245559809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c000d35c0 0x7f2c000ce260 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2bf40097a0 tx=0x7f2bf402a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2c6db5a6fbd7c3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:40.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.961+0000 7f2bfdffb640 1 -- 172.21.15.43:0/4245559809 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf4037070 con 0x7f2c000d35c0 2024-04-25T08:57:40.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.962+0000 7f2bfdffb640 1 -- 172.21.15.43:0/4245559809 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf402aba0 con 0x7f2c000d35c0 2024-04-25T08:57:40.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.962+0000 7f2bfdffb640 1 -- 172.21.15.43:0/4245559809 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0001840 con 0x7f2c000d35c0 2024-04-25T08:57:40.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.962+0000 7f2c05441640 1 -- 172.21.15.43:0/4245559809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c000d35c0 msgr2=0x7f2c000ce260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:40.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.962+0000 7f2c05441640 1 --2- 172.21.15.43:0/4245559809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c000d35c0 0x7f2c000ce260 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f2bf40097a0 tx=0x7f2bf402a110 comp rx=0 tx=0).stop 2024-04-25T08:57:40.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.962+0000 7f2c05441640 1 -- 172.21.15.43:0/4245559809 shutdown_connections 2024-04-25T08:57:40.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.962+0000 7f2c05441640 1 --2- 172.21.15.43:0/4245559809 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c000ce7a0 0x7f2c000b6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:40.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.962+0000 7f2c05441640 1 --2- 172.21.15.43:0/4245559809 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c000d35c0 0x7f2c000ce260 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:40.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.962+0000 7f2c05441640 1 -- 172.21.15.43:0/4245559809 >> 172.21.15.43:0/4245559809 conn(0x7f2c0006cbd0 msgr2=0x7f2c0006cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:40.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.962+0000 7f2c05441640 1 -- 172.21.15.43:0/4245559809 shutdown_connections 2024-04-25T08:57:40.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.962+0000 7f2c05441640 1 -- 172.21.15.43:0/4245559809 wait complete. 2024-04-25T08:57:40.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.964+0000 7f2c05441640 1 Processor -- start 2024-04-25T08:57:40.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.964+0000 7f2c05441640 1 -- start start 2024-04-25T08:57:40.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.965+0000 7f2c05441640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c000ce7a0 0x7f2c00159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:40.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.965+0000 7f2c05441640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c000d35c0 0x7f2c0015f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:40.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.965+0000 7f2c05441640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c0015dd00 0x7f2c000b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:40.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.965+0000 7f2c05441640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c000d2170 con 0x7f2c000ce7a0 2024-04-25T08:57:40.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.965+0000 7f2c05441640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c000d1e90 con 0x7f2c000d35c0 2024-04-25T08:57:40.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.965+0000 7f2c05441640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c000d2970 con 0x7f2c0015dd00 2024-04-25T08:57:40.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.965+0000 7f2bfeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c000ce7a0 0x7f2c00159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:40.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.965+0000 7f2bfeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c000ce7a0 0x7f2c00159470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45706/0 (socket says 172.21.15.43:45706) 2024-04-25T08:57:40.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.965+0000 7f2bfeffd640 1 -- 172.21.15.43:0/93735291 learned_addr learned my addr 172.21.15.43:0/93735291 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:40.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.965+0000 7f2bfe7fc640 1 --2- 172.21.15.43:0/93735291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c000d35c0 0x7f2c0015f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:40.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.965+0000 7f2bff7fe640 1 --2- 172.21.15.43:0/93735291 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c0015dd00 0x7f2c000b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:40.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.965+0000 7f2bfeffd640 1 -- 172.21.15.43:0/93735291 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c0015dd00 msgr2=0x7f2c000b6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:40.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.965+0000 7f2bfeffd640 1 --2- 172.21.15.43:0/93735291 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c0015dd00 0x7f2c000b6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:40.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.965+0000 7f2bfeffd640 1 -- 172.21.15.43:0/93735291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c000d35c0 msgr2=0x7f2c0015f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:40.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.965+0000 7f2bfeffd640 1 --2- 172.21.15.43:0/93735291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c000d35c0 0x7f2c0015f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:40.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.965+0000 7f2bfeffd640 1 -- 172.21.15.43:0/93735291 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c0015e460 con 0x7f2c000ce7a0 2024-04-25T08:57:40.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.965+0000 7f2bfeffd640 1 --2- 172.21.15.43:0/93735291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c000ce7a0 0x7f2c00159470 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0001eb0 tx=0x7f2bf000cad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:40.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.965+0000 7f2be77fe640 1 -- 172.21.15.43:0/93735291 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf0017070 con 0x7f2c000ce7a0 2024-04-25T08:57:40.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.965+0000 7f2be77fe640 1 -- 172.21.15.43:0/93735291 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bf0005ce0 con 0x7f2c000ce7a0 2024-04-25T08:57:40.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.965+0000 7f2c05441640 1 -- 172.21.15.43:0/93735291 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c0015e740 con 0x7f2c000ce7a0 2024-04-25T08:57:40.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.965+0000 7f2be77fe640 1 -- 172.21.15.43:0/93735291 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bf00072b0 con 0x7f2c000ce7a0 2024-04-25T08:57:40.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.965+0000 7f2c05441640 1 -- 172.21.15.43:0/93735291 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c0015ed00 con 0x7f2c000ce7a0 2024-04-25T08:57:40.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.967+0000 7f2c05441640 1 -- 172.21.15.43:0/93735291 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c00074ef0 con 0x7f2c000ce7a0 2024-04-25T08:57:40.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.967+0000 7f2be77fe640 1 -- 172.21.15.43:0/93735291 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2bf001a070 con 0x7f2c000ce7a0 2024-04-25T08:57:40.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.967+0000 7f2be77fe640 1 --2- 172.21.15.43:0/93735291 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2bec06b500 0x7f2bec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:40.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.968+0000 7f2be77fe640 1 -- 172.21.15.43:0/93735291 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2bf0089510 con 0x7f2c000ce7a0 2024-04-25T08:57:40.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.968+0000 7f2bfe7fc640 1 --2- 172.21.15.43:0/93735291 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2bec06b500 0x7f2bec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:40.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.968+0000 7f2bfe7fc640 1 --2- 172.21.15.43:0/93735291 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2bec06b500 0x7f2bec06d9c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4034000 tx=0x7f2bf402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:40.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:40.972+0000 7f2be77fe640 1 -- 172.21.15.43:0/93735291 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2c000cec30 con 0x7f2c000ce7a0 2024-04-25T08:57:41.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.188+0000 7f2c05441640 1 -- 172.21.15.43:0/93735291 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c000d0610 con 0x7f2c000ce7a0 2024-04-25T08:57:41.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.188+0000 7f2be77fe640 1 -- 172.21.15.43:0/93735291 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4945 (secure 0 0 0) 0x7f2bf0057dd0 con 0x7f2c000ce7a0 2024-04-25T08:57:41.178 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:57:41.178 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1504450503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1504450503},{"type":"v1","addr":"172.21.15.193:6835","nonce":1504450503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5691,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2657133821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2657133821},{"type":"v1","addr":"172.21.15.43:6834","nonce":2657133821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":5694,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/4271431818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":4271431818},{"type":"v1","addr":"172.21.15.43:6835","nonce":4271431818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:57:01.856132+0000","modified":"2024-04-25T08:57:38.526599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5709":{"gid":5709,"name":"d","rank":0,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.193:6837/4164634380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4164634380},{"type":"v1","addr":"172.21.15.193:6837","nonce":4164634380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:57:41.178 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 25 2024-04-25T08:57:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.190+0000 7f2c05441640 1 -- 172.21.15.43:0/93735291 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2bec06b500 msgr2=0x7f2bec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.190+0000 7f2c05441640 1 --2- 172.21.15.43:0/93735291 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2bec06b500 0x7f2bec06d9c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4034000 tx=0x7f2bf402e000 comp rx=0 tx=0).stop 2024-04-25T08:57:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.190+0000 7f2c05441640 1 -- 172.21.15.43:0/93735291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c000ce7a0 msgr2=0x7f2c00159470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.190+0000 7f2c05441640 1 --2- 172.21.15.43:0/93735291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c000ce7a0 0x7f2c00159470 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0001eb0 tx=0x7f2bf000cad0 comp rx=0 tx=0).stop 2024-04-25T08:57:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.190+0000 7f2c05441640 1 -- 172.21.15.43:0/93735291 shutdown_connections 2024-04-25T08:57:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.190+0000 7f2c05441640 1 --2- 172.21.15.43:0/93735291 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2c0015dd00 0x7f2c000b6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.190+0000 7f2c05441640 1 --2- 172.21.15.43:0/93735291 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2bec06b500 0x7f2bec06d9c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.190+0000 7f2c05441640 1 --2- 172.21.15.43:0/93735291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2c000d35c0 0x7f2c0015f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.190+0000 7f2c05441640 1 --2- 172.21.15.43:0/93735291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2c000ce7a0 0x7f2c00159470 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.190+0000 7f2c05441640 1 -- 172.21.15.43:0/93735291 >> 172.21.15.43:0/93735291 conn(0x7f2c0006cbd0 msgr2=0x7f2c000b5250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.190+0000 7f2c05441640 1 -- 172.21.15.43:0/93735291 shutdown_connections 2024-04-25T08:57:41.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.190+0000 7f2c05441640 1 -- 172.21.15.43:0/93735291 wait complete. 2024-04-25T08:57:41.191 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:57:41.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.302+0000 7f8deb75c640 1 Processor -- start 2024-04-25T08:57:41.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.302+0000 7f8deb75c640 1 -- start start 2024-04-25T08:57:41.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.303+0000 7f8deb75c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8de415dcf0 0x7f8de406e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:41.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.303+0000 7f8deb75c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8de406d4f0 0x7f8de406f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:41.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.303+0000 7f8deb75c640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8de40d0d80 con 0x7f8de406ebf0 2024-04-25T08:57:41.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.303+0000 7f8deb75c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8de406f590 con 0x7f8de415dcf0 2024-04-25T08:57:41.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.303+0000 7f8deb75c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8de406c050 con 0x7f8de406d4f0 2024-04-25T08:57:41.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.303+0000 7f8de94f8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8de415dcf0 0x7f8de406e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:41.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.303+0000 7f8de94f8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8de415dcf0 0x7f8de406e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45710/0 (socket says 172.21.15.43:45710) 2024-04-25T08:57:41.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.303+0000 7f8de8cf7640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8de406ebf0 0x7f8de41616e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:57620/0 (socket says 172.21.15.43:57620) 2024-04-25T08:57:41.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.303+0000 7f8de94f8640 1 -- 172.21.15.43:0/3074132741 learned_addr learned my addr 172.21.15.43:0/3074132741 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:41.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.303+0000 7f8de9cf9640 1 --2- 172.21.15.43:0/3074132741 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8de406d4f0 0x7f8de406f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:41.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.303+0000 7f8de94f8640 1 -- 172.21.15.43:0/3074132741 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8de406d4f0 msgr2=0x7f8de406f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:41.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.303+0000 7f8de94f8640 1 --2- 172.21.15.43:0/3074132741 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8de406d4f0 0x7f8de406f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:41.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.303+0000 7f8de94f8640 1 -- 172.21.15.43:0/3074132741 >> v1:172.21.15.43:6789/0 conn(0x7f8de406ebf0 legacy=0x7f8de41616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:41.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.303+0000 7f8de94f8640 1 -- 172.21.15.43:0/3074132741 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de40d04e0 con 0x7f8de415dcf0 2024-04-25T08:57:41.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.304+0000 7f8de94f8640 1 --2- 172.21.15.43:0/3074132741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8de415dcf0 0x7f8de406e530 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f8de0009870 tx=0x7f8de002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=342520f6129aa54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:41.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.304+0000 7f8ddbfff640 1 -- 172.21.15.43:0/3074132741 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de0037070 con 0x7f8de415dcf0 2024-04-25T08:57:41.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.304+0000 7f8ddbfff640 1 -- 172.21.15.43:0/3074132741 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de0032440 con 0x7f8de415dcf0 2024-04-25T08:57:41.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.304+0000 7f8ddbfff640 1 -- 172.21.15.43:0/3074132741 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de0032810 con 0x7f8de415dcf0 2024-04-25T08:57:41.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.305+0000 7f8deb75c640 1 -- 172.21.15.43:0/3074132741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8de415dcf0 msgr2=0x7f8de406e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:41.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.305+0000 7f8deb75c640 1 --2- 172.21.15.43:0/3074132741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8de415dcf0 0x7f8de406e530 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f8de0009870 tx=0x7f8de002a1c0 comp rx=0 tx=0).stop 2024-04-25T08:57:41.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.305+0000 7f8deb75c640 1 -- 172.21.15.43:0/3074132741 shutdown_connections 2024-04-25T08:57:41.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.305+0000 7f8deb75c640 1 --2- 172.21.15.43:0/3074132741 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8de406d4f0 0x7f8de406f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:41.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.305+0000 7f8deb75c640 1 --2- 172.21.15.43:0/3074132741 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8de415dcf0 0x7f8de406e530 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:41.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.305+0000 7f8deb75c640 1 -- 172.21.15.43:0/3074132741 >> 172.21.15.43:0/3074132741 conn(0x7f8de40d1700 msgr2=0x7f8de40d1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:41.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.305+0000 7f8deb75c640 1 -- 172.21.15.43:0/3074132741 shutdown_connections 2024-04-25T08:57:41.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.305+0000 7f8deb75c640 1 -- 172.21.15.43:0/3074132741 wait complete. 2024-04-25T08:57:41.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.306+0000 7f8deb75c640 1 Processor -- start 2024-04-25T08:57:41.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.306+0000 7f8deb75c640 1 -- start start 2024-04-25T08:57:41.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.307+0000 7f8deb75c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8de406d4f0 0x7f8de40c8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:41.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.307+0000 7f8deb75c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8de406ebf0 0x7f8de4151a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:41.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.307+0000 7f8deb75c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8de415dcf0 0x7f8de4157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:41.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.307+0000 7f8deb75c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de406bce0 con 0x7f8de415dcf0 2024-04-25T08:57:41.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.307+0000 7f8deb75c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de40d0aa0 con 0x7f8de406ebf0 2024-04-25T08:57:41.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.307+0000 7f8deb75c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8de40b0d50 con 0x7f8de406d4f0 2024-04-25T08:57:41.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.307+0000 7f8de9cf9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8de415dcf0 0x7f8de4157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:41.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.307+0000 7f8de9cf9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8de415dcf0 0x7f8de4157190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45716/0 (socket says 172.21.15.43:45716) 2024-04-25T08:57:41.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.307+0000 7f8de9cf9640 1 -- 172.21.15.43:0/3698815083 learned_addr learned my addr 172.21.15.43:0/3698815083 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:41.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.307+0000 7f8de8cf7640 1 --2- 172.21.15.43:0/3698815083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8de406ebf0 0x7f8de4151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:41.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.307+0000 7f8de94f8640 1 --2- 172.21.15.43:0/3698815083 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8de406d4f0 0x7f8de40c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:41.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.307+0000 7f8de9cf9640 1 -- 172.21.15.43:0/3698815083 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8de406d4f0 msgr2=0x7f8de40c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:41.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.307+0000 7f8de9cf9640 1 --2- 172.21.15.43:0/3698815083 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8de406d4f0 0x7f8de40c8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:41.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.307+0000 7f8de9cf9640 1 -- 172.21.15.43:0/3698815083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8de406ebf0 msgr2=0x7f8de4151a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:41.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.307+0000 7f8de9cf9640 1 --2- 172.21.15.43:0/3698815083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8de406ebf0 0x7f8de4151a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:41.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.307+0000 7f8de9cf9640 1 -- 172.21.15.43:0/3698815083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de4151fb0 con 0x7f8de415dcf0 2024-04-25T08:57:41.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.308+0000 7f8de9cf9640 1 --2- 172.21.15.43:0/3698815083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8de415dcf0 0x7f8de4157190 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc00cd60 tx=0x7f8ddc00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:41.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.308+0000 7f8dd9ffb640 1 -- 172.21.15.43:0/3698815083 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc003b90 con 0x7f8de415dcf0 2024-04-25T08:57:41.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.308+0000 7f8dd9ffb640 1 -- 172.21.15.43:0/3698815083 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ddc003d50 con 0x7f8de415dcf0 2024-04-25T08:57:41.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.308+0000 7f8deb75c640 1 -- 172.21.15.43:0/3698815083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8de41576d0 con 0x7f8de415dcf0 2024-04-25T08:57:41.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.308+0000 7f8dd9ffb640 1 -- 172.21.15.43:0/3698815083 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ddc004260 con 0x7f8de415dcf0 2024-04-25T08:57:41.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.308+0000 7f8deb75c640 1 -- 172.21.15.43:0/3698815083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8de40c54d0 con 0x7f8de415dcf0 2024-04-25T08:57:41.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.309+0000 7f8deb75c640 1 -- 172.21.15.43:0/3698815083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8de4074ef0 con 0x7f8de415dcf0 2024-04-25T08:57:41.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.309+0000 7f8dd9ffb640 1 -- 172.21.15.43:0/3698815083 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8ddc004420 con 0x7f8de415dcf0 2024-04-25T08:57:41.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.309+0000 7f8dd9ffb640 1 --2- 172.21.15.43:0/3698815083 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8dc806b4d0 0x7f8dc806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:41.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.310+0000 7f8dd9ffb640 1 -- 172.21.15.43:0/3698815083 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8ddc08ac30 con 0x7f8de415dcf0 2024-04-25T08:57:41.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.310+0000 7f8de94f8640 1 --2- 172.21.15.43:0/3698815083 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8dc806b4d0 0x7f8dc806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:41.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.310+0000 7f8de94f8640 1 --2- 172.21.15.43:0/3698815083 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8dc806b4d0 0x7f8dc806d990 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8de0034000 tx=0x7f8de002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:41.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.314+0000 7f8dd9ffb640 1 -- 172.21.15.43:0/3698815083 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8ddc059b40 con 0x7f8de415dcf0 2024-04-25T08:57:41.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.497+0000 7f8deb75c640 1 -- 172.21.15.43:0/3698815083 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8de40b7680 con 0x7f8de415dcf0 2024-04-25T08:57:41.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.498+0000 7f8dd9ffb640 1 -- 172.21.15.43:0/3698815083 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16577 (secure 0 0 0) 0x7f8ddc059440 con 0x7f8de415dcf0 2024-04-25T08:57:41.488 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:57:41.488 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":46,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:57:03.489946+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:57:00.135489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T08:57:00.830128+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:57:41.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.500+0000 7f8deb75c640 1 -- 172.21.15.43:0/3698815083 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8dc806b4d0 msgr2=0x7f8dc806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:41.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.500+0000 7f8deb75c640 1 --2- 172.21.15.43:0/3698815083 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8dc806b4d0 0x7f8dc806d990 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8de0034000 tx=0x7f8de002e000 comp rx=0 tx=0).stop 2024-04-25T08:57:41.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.500+0000 7f8deb75c640 1 -- 172.21.15.43:0/3698815083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8de415dcf0 msgr2=0x7f8de4157190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:41.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.500+0000 7f8deb75c640 1 --2- 172.21.15.43:0/3698815083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8de415dcf0 0x7f8de4157190 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f8ddc00cd60 tx=0x7f8ddc00a660 comp rx=0 tx=0).stop 2024-04-25T08:57:41.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.500+0000 7f8deb75c640 1 -- 172.21.15.43:0/3698815083 shutdown_connections 2024-04-25T08:57:41.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.500+0000 7f8deb75c640 1 --2- 172.21.15.43:0/3698815083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8de415dcf0 0x7f8de4157190 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:41.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.500+0000 7f8deb75c640 1 --2- 172.21.15.43:0/3698815083 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8dc806b4d0 0x7f8dc806d990 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:41.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.500+0000 7f8deb75c640 1 --2- 172.21.15.43:0/3698815083 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8de406ebf0 0x7f8de4151a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:41.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.500+0000 7f8deb75c640 1 --2- 172.21.15.43:0/3698815083 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8de406d4f0 0x7f8de40c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:41.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.500+0000 7f8deb75c640 1 -- 172.21.15.43:0/3698815083 >> 172.21.15.43:0/3698815083 conn(0x7f8de40d1700 msgr2=0x7f8de40af130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:41.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.500+0000 7f8deb75c640 1 -- 172.21.15.43:0/3698815083 shutdown_connections 2024-04-25T08:57:41.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.501+0000 7f8deb75c640 1 -- 172.21.15.43:0/3698815083 wait complete. 2024-04-25T08:57:41.502 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T08:57:41.502 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:57:41.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.613+0000 7f77b2d5c640 1 Processor -- start 2024-04-25T08:57:41.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.613+0000 7f77b2d5c640 1 -- start start 2024-04-25T08:57:41.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.614+0000 7f77b2d5c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77ac06c510 0x7f77ac06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:41.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.614+0000 7f77b2d5c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f77ac15df70 0x7f77ac0ce070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:41.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.614+0000 7f77b2d5c640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77ac0cbee0 con 0x7f77ac0b0870 2024-04-25T08:57:41.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.614+0000 7f77b2d5c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77ac0d0400 con 0x7f77ac15df70 2024-04-25T08:57:41.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.614+0000 7f77b2d5c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77ac06ce30 con 0x7f77ac06c510 2024-04-25T08:57:41.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.614+0000 7f77abfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f77ac15df70 0x7f77ac0ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:41.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.614+0000 7f77abfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f77ac15df70 0x7f77ac0ce070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45718/0 (socket says 172.21.15.43:45718) 2024-04-25T08:57:41.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.614+0000 7f77abfff640 1 -- 172.21.15.43:0/790719809 learned_addr learned my addr 172.21.15.43:0/790719809 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:41.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.615+0000 7f77b0af8640 1 --2- 172.21.15.43:0/790719809 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77ac06c510 0x7f77ac06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:41.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.615+0000 7f77abfff640 1 -- 172.21.15.43:0/790719809 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77ac06c510 msgr2=0x7f77ac06c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:41.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.615+0000 7f77abfff640 1 --2- 172.21.15.43:0/790719809 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77ac06c510 0x7f77ac06c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:41.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.615+0000 7f77abfff640 1 -- 172.21.15.43:0/790719809 >> v1:172.21.15.193:6789/0 conn(0x7f77ac0b0870 legacy=0x7f77ac0ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:41.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.615+0000 7f77abfff640 1 -- 172.21.15.43:0/790719809 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77ac0d0650 con 0x7f77ac15df70 2024-04-25T08:57:41.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.615+0000 7f77abfff640 1 --2- 172.21.15.43:0/790719809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77ac15df70 0x7f77ac0ce070 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f77a0009870 tx=0x7f77a002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee733b4d64ab93a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:41.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.616+0000 7f77ab7fe640 1 -- 172.21.15.43:0/790719809 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a0037070 con 0x7f77ac15df70 2024-04-25T08:57:41.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.616+0000 7f77ab7fe640 1 -- 172.21.15.43:0/790719809 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77a0032440 con 0x7f77ac15df70 2024-04-25T08:57:41.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.616+0000 7f77ab7fe640 1 -- 172.21.15.43:0/790719809 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a0032810 con 0x7f77ac15df70 2024-04-25T08:57:41.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.616+0000 7f77b2d5c640 1 -- 172.21.15.43:0/790719809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77ac15df70 msgr2=0x7f77ac0ce070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:41.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.616+0000 7f77b2d5c640 1 --2- 172.21.15.43:0/790719809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77ac15df70 0x7f77ac0ce070 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f77a0009870 tx=0x7f77a002a210 comp rx=0 tx=0).stop 2024-04-25T08:57:41.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.616+0000 7f77b2d5c640 1 -- 172.21.15.43:0/790719809 shutdown_connections 2024-04-25T08:57:41.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.616+0000 7f77b2d5c640 1 --2- 172.21.15.43:0/790719809 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77ac15df70 0x7f77ac0ce070 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:41.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.616+0000 7f77b2d5c640 1 --2- 172.21.15.43:0/790719809 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77ac06c510 0x7f77ac06c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:41.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.616+0000 7f77b2d5c640 1 -- 172.21.15.43:0/790719809 >> 172.21.15.43:0/790719809 conn(0x7f77ac0d17f0 msgr2=0x7f77ac0d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:41.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.616+0000 7f77b2d5c640 1 -- 172.21.15.43:0/790719809 shutdown_connections 2024-04-25T08:57:41.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.616+0000 7f77b2d5c640 1 -- 172.21.15.43:0/790719809 wait complete. 2024-04-25T08:57:41.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.617+0000 7f77b2d5c640 1 Processor -- start 2024-04-25T08:57:41.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.617+0000 7f77b2d5c640 1 -- start start 2024-04-25T08:57:41.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.618+0000 7f77b2d5c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77ac06c510 0x7f77ac157f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:41.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.618+0000 7f77b2d5c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77ac0b0870 0x7f77ac06f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:41.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.618+0000 7f77b2d5c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77ac15df70 0x7f77ac06fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:41.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.618+0000 7f77b2d5c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77ac0d0c10 con 0x7f77ac06c510 2024-04-25T08:57:41.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.618+0000 7f77b2d5c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77ac0cc150 con 0x7f77ac15df70 2024-04-25T08:57:41.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.618+0000 7f77b2d5c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77ac0cecc0 con 0x7f77ac0b0870 2024-04-25T08:57:41.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.618+0000 7f77b0af8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77ac06c510 0x7f77ac157f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:41.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.618+0000 7f77b0af8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77ac06c510 0x7f77ac157f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45724/0 (socket says 172.21.15.43:45724) 2024-04-25T08:57:41.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.618+0000 7f77b0af8640 1 -- 172.21.15.43:0/3316083940 learned_addr learned my addr 172.21.15.43:0/3316083940 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:41.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.619+0000 7f77b12f9640 1 --2- 172.21.15.43:0/3316083940 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77ac15df70 0x7f77ac06fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:41.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.619+0000 7f77abfff640 1 --2- 172.21.15.43:0/3316083940 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77ac0b0870 0x7f77ac06f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:41.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.619+0000 7f77b0af8640 1 -- 172.21.15.43:0/3316083940 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77ac0b0870 msgr2=0x7f77ac06f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:41.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.619+0000 7f77b0af8640 1 --2- 172.21.15.43:0/3316083940 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77ac0b0870 0x7f77ac06f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:41.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.619+0000 7f77b0af8640 1 -- 172.21.15.43:0/3316083940 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77ac15df70 msgr2=0x7f77ac06fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:41.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.619+0000 7f77b0af8640 1 --2- 172.21.15.43:0/3316083940 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77ac15df70 0x7f77ac06fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:41.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.619+0000 7f77b0af8640 1 -- 172.21.15.43:0/3316083940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77ac0b55e0 con 0x7f77ac06c510 2024-04-25T08:57:41.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.619+0000 7f77b12f9640 1 --2- 172.21.15.43:0/3316083940 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77ac15df70 0x7f77ac06fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:41.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.619+0000 7f77b0af8640 1 --2- 172.21.15.43:0/3316083940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77ac06c510 0x7f77ac157f80 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f77a400ce10 tx=0x7f77a400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:41.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.619+0000 7f77a97fa640 1 -- 172.21.15.43:0/3316083940 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a4003b90 con 0x7f77ac06c510 2024-04-25T08:57:41.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.619+0000 7f77a97fa640 1 -- 172.21.15.43:0/3316083940 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77a4003d30 con 0x7f77ac06c510 2024-04-25T08:57:41.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.619+0000 7f77b2d5c640 1 -- 172.21.15.43:0/3316083940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77ac0b58c0 con 0x7f77ac06c510 2024-04-25T08:57:41.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.619+0000 7f77a97fa640 1 -- 172.21.15.43:0/3316083940 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a4004260 con 0x7f77ac06c510 2024-04-25T08:57:41.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.619+0000 7f77b2d5c640 1 -- 172.21.15.43:0/3316083940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77ac0b63a0 con 0x7f77ac06c510 2024-04-25T08:57:41.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.620+0000 7f77b2d5c640 1 -- 172.21.15.43:0/3316083940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77ac074ef0 con 0x7f77ac06c510 2024-04-25T08:57:41.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.621+0000 7f77a97fa640 1 -- 172.21.15.43:0/3316083940 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f77a4004400 con 0x7f77ac06c510 2024-04-25T08:57:41.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.621+0000 7f77a97fa640 1 --2- 172.21.15.43:0/3316083940 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f779806b4d0 0x7f779806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:41.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.621+0000 7f77a97fa640 1 -- 172.21.15.43:0/3316083940 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f77a408acd0 con 0x7f77ac06c510 2024-04-25T08:57:41.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.621+0000 7f77abfff640 1 --2- 172.21.15.43:0/3316083940 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f779806b4d0 0x7f779806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:41.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.622+0000 7f77abfff640 1 --2- 172.21.15.43:0/3316083940 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f779806b4d0 0x7f779806d990 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f77a0034000 tx=0x7f77a002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:41.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.625+0000 7f77a97fa640 1 -- 172.21.15.43:0/3316083940 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f77a4059b50 con 0x7f77ac06c510 2024-04-25T08:57:41.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.841+0000 7f77b2d5c640 1 -- 172.21.15.43:0/3316083940 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f77ac160ab0 con 0x7f77ac06c510 2024-04-25T08:57:41.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.842+0000 7f77a97fa640 1 -- 172.21.15.43:0/3316083940 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4945 (secure 0 0 0) 0x7f77a4059450 con 0x7f77ac06c510 2024-04-25T08:57:41.832 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:57:41.832 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1504450503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1504450503},{"type":"v1","addr":"172.21.15.193:6835","nonce":1504450503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5691,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2657133821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2657133821},{"type":"v1","addr":"172.21.15.43:6834","nonce":2657133821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":5694,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/4271431818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":4271431818},{"type":"v1","addr":"172.21.15.43:6835","nonce":4271431818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:57:01.856132+0000","modified":"2024-04-25T08:57:38.526599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5709":{"gid":5709,"name":"d","rank":0,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.193:6837/4164634380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4164634380},{"type":"v1","addr":"172.21.15.193:6837","nonce":4164634380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:57:41.832 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 25 2024-04-25T08:57:41.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.844+0000 7f77b2d5c640 1 -- 172.21.15.43:0/3316083940 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f779806b4d0 msgr2=0x7f779806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:41.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.844+0000 7f77b2d5c640 1 --2- 172.21.15.43:0/3316083940 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f779806b4d0 0x7f779806d990 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f77a0034000 tx=0x7f77a002e000 comp rx=0 tx=0).stop 2024-04-25T08:57:41.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.844+0000 7f77b2d5c640 1 -- 172.21.15.43:0/3316083940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77ac06c510 msgr2=0x7f77ac157f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:41.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.844+0000 7f77b2d5c640 1 --2- 172.21.15.43:0/3316083940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77ac06c510 0x7f77ac157f80 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f77a400ce10 tx=0x7f77a400a660 comp rx=0 tx=0).stop 2024-04-25T08:57:41.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.844+0000 7f77b2d5c640 1 -- 172.21.15.43:0/3316083940 shutdown_connections 2024-04-25T08:57:41.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.844+0000 7f77b2d5c640 1 --2- 172.21.15.43:0/3316083940 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f779806b4d0 0x7f779806d990 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:41.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.844+0000 7f77b2d5c640 1 --2- 172.21.15.43:0/3316083940 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77ac15df70 0x7f77ac06fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:41.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.844+0000 7f77b2d5c640 1 --2- 172.21.15.43:0/3316083940 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77ac0b0870 0x7f77ac06f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:41.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.844+0000 7f77b2d5c640 1 --2- 172.21.15.43:0/3316083940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f77ac06c510 0x7f77ac157f80 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:41.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.844+0000 7f77b2d5c640 1 -- 172.21.15.43:0/3316083940 >> 172.21.15.43:0/3316083940 conn(0x7f77ac0d17f0 msgr2=0x7f77ac1603a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:41.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.844+0000 7f77b2d5c640 1 -- 172.21.15.43:0/3316083940 shutdown_connections 2024-04-25T08:57:41.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.844+0000 7f77b2d5c640 1 -- 172.21.15.43:0/3316083940 wait complete. 2024-04-25T08:57:41.846 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:57:41.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.956+0000 7f6bb40fc640 1 Processor -- start 2024-04-25T08:57:41.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.957+0000 7f6bb40fc640 1 -- start start 2024-04-25T08:57:41.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.957+0000 7f6bb40fc640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bac15df50 0x7f6bac1619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:41.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.957+0000 7f6bb40fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6bac161f30 0x7f6bac162370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:41.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.957+0000 7f6bb40fc640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bac0d24a0 con 0x7f6bac0c7cf0 2024-04-25T08:57:41.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.957+0000 7f6bb40fc640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bac15e330 con 0x7f6bac161f30 2024-04-25T08:57:41.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.957+0000 7f6bb40fc640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac0d2740 con 0x7f6bac15df50 2024-04-25T08:57:41.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.958+0000 7f6bb1697640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6bac161f30 0x7f6bac162370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:41.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.958+0000 7f6bb1697640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6bac161f30 0x7f6bac162370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45732/0 (socket says 172.21.15.43:45732) 2024-04-25T08:57:41.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.958+0000 7f6bb1697640 1 -- 172.21.15.43:0/233410298 learned_addr learned my addr 172.21.15.43:0/233410298 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:41.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.958+0000 7f6bb1e98640 1 --2- 172.21.15.43:0/233410298 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bac15df50 0x7f6bac1619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:41.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.958+0000 7f6bb1697640 1 -- 172.21.15.43:0/233410298 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bac15df50 msgr2=0x7f6bac1619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:41.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.958+0000 7f6bb1697640 1 --2- 172.21.15.43:0/233410298 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bac15df50 0x7f6bac1619f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:41.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.958+0000 7f6bb1697640 1 -- 172.21.15.43:0/233410298 >> v1:172.21.15.193:6789/0 conn(0x7f6bac0c7cf0 legacy=0x7f6bac0c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:41.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.958+0000 7f6bb1697640 1 -- 172.21.15.43:0/233410298 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bac0d2a20 con 0x7f6bac161f30 2024-04-25T08:57:41.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.959+0000 7f6bb1697640 1 --2- 172.21.15.43:0/233410298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6bac161f30 0x7f6bac162370 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0009870 tx=0x7f6ba002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e306eea02cb69a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:41.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.959+0000 7f6bb0e96640 1 -- 172.21.15.43:0/233410298 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0037070 con 0x7f6bac161f30 2024-04-25T08:57:41.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.959+0000 7f6bb0e96640 1 -- 172.21.15.43:0/233410298 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ba0032440 con 0x7f6bac161f30 2024-04-25T08:57:41.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.959+0000 7f6bb0e96640 1 -- 172.21.15.43:0/233410298 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ba0032810 con 0x7f6bac161f30 2024-04-25T08:57:41.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.959+0000 7f6bb40fc640 1 -- 172.21.15.43:0/233410298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6bac161f30 msgr2=0x7f6bac162370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:41.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.959+0000 7f6bb40fc640 1 --2- 172.21.15.43:0/233410298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6bac161f30 0x7f6bac162370 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f6ba0009870 tx=0x7f6ba002a270 comp rx=0 tx=0).stop 2024-04-25T08:57:41.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.959+0000 7f6bb40fc640 1 -- 172.21.15.43:0/233410298 shutdown_connections 2024-04-25T08:57:41.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.959+0000 7f6bb40fc640 1 --2- 172.21.15.43:0/233410298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6bac161f30 0x7f6bac162370 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:41.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.959+0000 7f6bb40fc640 1 --2- 172.21.15.43:0/233410298 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bac15df50 0x7f6bac1619f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:41.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.959+0000 7f6bb40fc640 1 -- 172.21.15.43:0/233410298 >> 172.21.15.43:0/233410298 conn(0x7f6bac0d3a30 msgr2=0x7f6bac0d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:41.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.959+0000 7f6bb40fc640 1 -- 172.21.15.43:0/233410298 shutdown_connections 2024-04-25T08:57:41.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.959+0000 7f6bb40fc640 1 -- 172.21.15.43:0/233410298 wait complete. 2024-04-25T08:57:41.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.961+0000 7f6bb40fc640 1 Processor -- start 2024-04-25T08:57:41.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.961+0000 7f6bb40fc640 1 -- start start 2024-04-25T08:57:41.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.961+0000 7f6bb40fc640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bac0c7cf0 0x7f6bac15ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:41.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.961+0000 7f6bb40fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bac15df50 0x7f6bac151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:41.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.961+0000 7f6bb40fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6bac161f30 0x7f6bac15f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:41.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.961+0000 7f6bb40fc640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac0d2190 con 0x7f6bac161f30 2024-04-25T08:57:41.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.961+0000 7f6bb40fc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac0c77d0 con 0x7f6bac15df50 2024-04-25T08:57:41.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.961+0000 7f6bb40fc640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bac0cf670 con 0x7f6bac0c7cf0 2024-04-25T08:57:41.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.961+0000 7f6bb2699640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6bac161f30 0x7f6bac15f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:41.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.961+0000 7f6bb2699640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6bac161f30 0x7f6bac15f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45736/0 (socket says 172.21.15.43:45736) 2024-04-25T08:57:41.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.962+0000 7f6bb2699640 1 -- 172.21.15.43:0/1911897793 learned_addr learned my addr 172.21.15.43:0/1911897793 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:41.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.962+0000 7f6bb1e98640 1 --2- 172.21.15.43:0/1911897793 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bac0c7cf0 0x7f6bac15ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:41.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.962+0000 7f6bb1697640 1 --2- 172.21.15.43:0/1911897793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bac15df50 0x7f6bac151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:41.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.962+0000 7f6bb2699640 1 -- 172.21.15.43:0/1911897793 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bac0c7cf0 msgr2=0x7f6bac15ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:41.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.962+0000 7f6bb2699640 1 --2- 172.21.15.43:0/1911897793 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bac0c7cf0 0x7f6bac15ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:41.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.962+0000 7f6bb2699640 1 -- 172.21.15.43:0/1911897793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bac15df50 msgr2=0x7f6bac151130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:41.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.962+0000 7f6bb2699640 1 --2- 172.21.15.43:0/1911897793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bac15df50 0x7f6bac151130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:41.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.962+0000 7f6bb2699640 1 -- 172.21.15.43:0/1911897793 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bac1605c0 con 0x7f6bac161f30 2024-04-25T08:57:41.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.962+0000 7f6bb2699640 1 --2- 172.21.15.43:0/1911897793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6bac161f30 0x7f6bac15f330 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c00c7d0 tx=0x7f6b9c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:41.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.962+0000 7f6baa7fc640 1 -- 172.21.15.43:0/1911897793 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c003b70 con 0x7f6bac161f30 2024-04-25T08:57:41.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.962+0000 7f6baa7fc640 1 -- 172.21.15.43:0/1911897793 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b9c003d30 con 0x7f6bac161f30 2024-04-25T08:57:41.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.962+0000 7f6bb40fc640 1 -- 172.21.15.43:0/1911897793 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bac1608a0 con 0x7f6bac161f30 2024-04-25T08:57:41.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.962+0000 7f6baa7fc640 1 -- 172.21.15.43:0/1911897793 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9c00aa90 con 0x7f6bac161f30 2024-04-25T08:57:41.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.962+0000 7f6bb40fc640 1 -- 172.21.15.43:0/1911897793 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bac160e60 con 0x7f6bac161f30 2024-04-25T08:57:41.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.963+0000 7f6bb40fc640 1 -- 172.21.15.43:0/1911897793 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bac0680f0 con 0x7f6bac161f30 2024-04-25T08:57:41.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.964+0000 7f6baa7fc640 1 -- 172.21.15.43:0/1911897793 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6b9c00ac30 con 0x7f6bac161f30 2024-04-25T08:57:41.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.964+0000 7f6baa7fc640 1 --2- 172.21.15.43:0/1911897793 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6b9806b4d0 0x7f6b9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:41.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.964+0000 7f6baa7fc640 1 -- 172.21.15.43:0/1911897793 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6b9c08a010 con 0x7f6bac161f30 2024-04-25T08:57:41.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.964+0000 7f6bb1e98640 1 --2- 172.21.15.43:0/1911897793 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6b9806b4d0 0x7f6b9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:41.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.965+0000 7f6bb1e98640 1 --2- 172.21.15.43:0/1911897793 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6b9806b4d0 0x7f6b9806d990 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4009b40 tx=0x7f6ba4009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:41.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:41.968+0000 7f6baa7fc640 1 -- 172.21.15.43:0/1911897793 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6b9c058840 con 0x7f6bac161f30 2024-04-25T08:57:42.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.187+0000 7f6bb40fc640 1 -- 172.21.15.43:0/1911897793 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6bac074ef0 con 0x7f6bac161f30 2024-04-25T08:57:42.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.188+0000 7f6baa7fc640 1 -- 172.21.15.43:0/1911897793 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4945 (secure 0 0 0) 0x7f6b9c05c8e0 con 0x7f6bac161f30 2024-04-25T08:57:42.177 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:57:42.177 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1504450503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1504450503},{"type":"v1","addr":"172.21.15.193:6835","nonce":1504450503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5691,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2657133821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2657133821},{"type":"v1","addr":"172.21.15.43:6834","nonce":2657133821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":5694,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/4271431818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":4271431818},{"type":"v1","addr":"172.21.15.43:6835","nonce":4271431818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:57:01.856132+0000","modified":"2024-04-25T08:57:38.526599+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5709},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5709":{"gid":5709,"name":"d","rank":0,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.193:6837/4164634380","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":4164634380},{"type":"v1","addr":"172.21.15.193:6837","nonce":4164634380}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T08:57:42.178 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 25 2024-04-25T08:57:42.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.189+0000 7f6bb40fc640 1 -- 172.21.15.43:0/1911897793 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6b9806b4d0 msgr2=0x7f6b9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:42.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.189+0000 7f6bb40fc640 1 --2- 172.21.15.43:0/1911897793 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6b9806b4d0 0x7f6b9806d990 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4009b40 tx=0x7f6ba4009190 comp rx=0 tx=0).stop 2024-04-25T08:57:42.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.189+0000 7f6bb40fc640 1 -- 172.21.15.43:0/1911897793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6bac161f30 msgr2=0x7f6bac15f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:42.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.189+0000 7f6bb40fc640 1 --2- 172.21.15.43:0/1911897793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6bac161f30 0x7f6bac15f330 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c00c7d0 tx=0x7f6b9c00cc90 comp rx=0 tx=0).stop 2024-04-25T08:57:42.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.190+0000 7f6bb40fc640 1 -- 172.21.15.43:0/1911897793 shutdown_connections 2024-04-25T08:57:42.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.190+0000 7f6bb40fc640 1 --2- 172.21.15.43:0/1911897793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6bac161f30 0x7f6bac15f330 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:42.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.190+0000 7f6bb40fc640 1 --2- 172.21.15.43:0/1911897793 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6b9806b4d0 0x7f6b9806d990 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:42.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.190+0000 7f6bb40fc640 1 --2- 172.21.15.43:0/1911897793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bac15df50 0x7f6bac151130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:42.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.190+0000 7f6bb40fc640 1 --2- 172.21.15.43:0/1911897793 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bac0c7cf0 0x7f6bac15ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:42.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.190+0000 7f6bb40fc640 1 -- 172.21.15.43:0/1911897793 >> 172.21.15.43:0/1911897793 conn(0x7f6bac0d3a30 msgr2=0x7f6bac152f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:42.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.190+0000 7f6bb40fc640 1 -- 172.21.15.43:0/1911897793 shutdown_connections 2024-04-25T08:57:42.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.190+0000 7f6bb40fc640 1 -- 172.21.15.43:0/1911897793 wait complete. 2024-04-25T08:57:42.190 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:57:42.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.301+0000 7f8387eb6640 1 Processor -- start 2024-04-25T08:57:42.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.302+0000 7f8387eb6640 1 -- start start 2024-04-25T08:57:42.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.302+0000 7f8387eb6640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f838006c510 0x7f838006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:42.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.302+0000 7f8387eb6640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f83800cd680 0x7f83800ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:42.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.302+0000 7f8387eb6640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83800cbee0 con 0x7f838015df70 2024-04-25T08:57:42.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.302+0000 7f8387eb6640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83800d0400 con 0x7f83800cd680 2024-04-25T08:57:42.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.302+0000 7f8387eb6640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f838006ce30 con 0x7f838006c510 2024-04-25T08:57:42.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.303+0000 7f8386453640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f83800cd680 0x7f83800ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:42.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.303+0000 7f8385c52640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f838006c510 0x7f838006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:42.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.303+0000 7f8386453640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f83800cd680 0x7f83800ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:46022/0 (socket says 172.21.15.43:46022) 2024-04-25T08:57:42.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.303+0000 7f8386453640 1 -- 172.21.15.43:0/4064498628 learned_addr learned my addr 172.21.15.43:0/4064498628 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:42.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.303+0000 7f8385451640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f838015df70 0x7f83800ce070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:39464/0 (socket says 172.21.15.43:39464) 2024-04-25T08:57:42.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.303+0000 7f8385c52640 1 -- 172.21.15.43:0/4064498628 >> v2:172.21.15.193:3300/0 conn(0x7f83800cd680 msgr2=0x7f83800ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:42.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.303+0000 7f8385c52640 1 --2- 172.21.15.43:0/4064498628 >> v2:172.21.15.193:3300/0 conn(0x7f83800cd680 0x7f83800ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:42.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.303+0000 7f8385c52640 1 -- 172.21.15.43:0/4064498628 >> v1:172.21.15.193:6789/0 conn(0x7f838015df70 legacy=0x7f83800ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:42.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.303+0000 7f8385c52640 1 -- 172.21.15.43:0/4064498628 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83800d0650 con 0x7f838006c510 2024-04-25T08:57:42.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.303+0000 7f8386453640 1 --2- 172.21.15.43:0/4064498628 >> v2:172.21.15.193:3300/0 conn(0x7f83800cd680 0x7f83800ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:57:42.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.304+0000 7f8385c52640 1 --2- 172.21.15.43:0/4064498628 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f838006c510 0x7f838006c8f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f83780097a0 tx=0x7f837802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=836a3444d5747ba4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:42.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.304+0000 7f8384c50640 1 -- 172.21.15.43:0/4064498628 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8378037070 con 0x7f838006c510 2024-04-25T08:57:42.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.304+0000 7f8384c50640 1 -- 172.21.15.43:0/4064498628 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8378032440 con 0x7f838006c510 2024-04-25T08:57:42.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.304+0000 7f8384c50640 1 -- 172.21.15.43:0/4064498628 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8378032810 con 0x7f838006c510 2024-04-25T08:57:42.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.305+0000 7f8387eb6640 1 -- 172.21.15.43:0/4064498628 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f838006c510 msgr2=0x7f838006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:42.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.305+0000 7f8387eb6640 1 --2- 172.21.15.43:0/4064498628 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f838006c510 0x7f838006c8f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f83780097a0 tx=0x7f837802a180 comp rx=0 tx=0).stop 2024-04-25T08:57:42.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.305+0000 7f8387eb6640 1 -- 172.21.15.43:0/4064498628 shutdown_connections 2024-04-25T08:57:42.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.305+0000 7f8387eb6640 1 --2- 172.21.15.43:0/4064498628 >> v2:172.21.15.193:3300/0 conn(0x7f83800cd680 0x7f83800ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:42.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.305+0000 7f8387eb6640 1 --2- 172.21.15.43:0/4064498628 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f838006c510 0x7f838006c8f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:42.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.305+0000 7f8387eb6640 1 -- 172.21.15.43:0/4064498628 >> 172.21.15.43:0/4064498628 conn(0x7f83800d17f0 msgr2=0x7f83800d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:42.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.305+0000 7f8387eb6640 1 -- 172.21.15.43:0/4064498628 shutdown_connections 2024-04-25T08:57:42.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.305+0000 7f8387eb6640 1 -- 172.21.15.43:0/4064498628 wait complete. 2024-04-25T08:57:42.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.306+0000 7f8387eb6640 1 Processor -- start 2024-04-25T08:57:42.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.306+0000 7f8387eb6640 1 -- start start 2024-04-25T08:57:42.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.307+0000 7f8387eb6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f838006c510 0x7f83800c6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:42.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.307+0000 7f8387eb6640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f83800cd680 0x7f83800c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:42.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.307+0000 7f8387eb6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f838015df70 0x7f838014dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:42.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.307+0000 7f8387eb6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83800d0c10 con 0x7f838015df70 2024-04-25T08:57:42.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.307+0000 7f8387eb6640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83800cc150 con 0x7f838006c510 2024-04-25T08:57:42.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.307+0000 7f8387eb6640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83800cecc0 con 0x7f83800cd680 2024-04-25T08:57:42.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.307+0000 7f8386453640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f838015df70 0x7f838014dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:42.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.307+0000 7f8386453640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f838015df70 0x7f838014dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45744/0 (socket says 172.21.15.43:45744) 2024-04-25T08:57:42.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.307+0000 7f8386453640 1 -- 172.21.15.43:0/2800204079 learned_addr learned my addr 172.21.15.43:0/2800204079 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:42.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.307+0000 7f8385c52640 1 --2- 172.21.15.43:0/2800204079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f838006c510 0x7f83800c6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:42.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.307+0000 7f8385451640 1 --2- 172.21.15.43:0/2800204079 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f83800cd680 0x7f83800c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:42.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.307+0000 7f8386453640 1 -- 172.21.15.43:0/2800204079 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f83800cd680 msgr2=0x7f83800c88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:42.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.307+0000 7f8386453640 1 --2- 172.21.15.43:0/2800204079 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f83800cd680 0x7f83800c88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:42.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.307+0000 7f8386453640 1 -- 172.21.15.43:0/2800204079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f838006c510 msgr2=0x7f83800c6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:42.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.307+0000 7f8386453640 1 --2- 172.21.15.43:0/2800204079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f838006c510 0x7f83800c6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:42.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.307+0000 7f8386453640 1 -- 172.21.15.43:0/2800204079 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f838014e190 con 0x7f838015df70 2024-04-25T08:57:42.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.307+0000 7f8386453640 1 --2- 172.21.15.43:0/2800204079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f838015df70 0x7f838014dc50 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f837000ef90 tx=0x7f837000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:42.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.307+0000 7f837e7fc640 1 -- 172.21.15.43:0/2800204079 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837000cbe0 con 0x7f838015df70 2024-04-25T08:57:42.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.307+0000 7f837e7fc640 1 -- 172.21.15.43:0/2800204079 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f837000cd80 con 0x7f838015df70 2024-04-25T08:57:42.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.308+0000 7f8387eb6640 1 -- 172.21.15.43:0/2800204079 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f838015a3c0 con 0x7f838015df70 2024-04-25T08:57:42.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.308+0000 7f837e7fc640 1 -- 172.21.15.43:0/2800204079 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8370003bd0 con 0x7f838015df70 2024-04-25T08:57:42.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.308+0000 7f8387eb6640 1 -- 172.21.15.43:0/2800204079 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83800c6ac0 con 0x7f838015df70 2024-04-25T08:57:42.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.308+0000 7f8387eb6640 1 -- 172.21.15.43:0/2800204079 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f838006ea10 con 0x7f838015df70 2024-04-25T08:57:42.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.309+0000 7f837e7fc640 1 -- 172.21.15.43:0/2800204079 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f837001b030 con 0x7f838015df70 2024-04-25T08:57:42.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.309+0000 7f837e7fc640 1 --2- 172.21.15.43:0/2800204079 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f836c06b4d0 0x7f836c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:42.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.309+0000 7f837e7fc640 1 -- 172.21.15.43:0/2800204079 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f837008a3d0 con 0x7f838015df70 2024-04-25T08:57:42.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.310+0000 7f8385c52640 1 --2- 172.21.15.43:0/2800204079 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f836c06b4d0 0x7f836c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:42.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.310+0000 7f8385c52640 1 --2- 172.21.15.43:0/2800204079 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f836c06b4d0 0x7f836c06d990 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8378034000 tx=0x7f837802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:42.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.313+0000 7f837e7fc640 1 -- 172.21.15.43:0/2800204079 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f838006ea10 con 0x7f838015df70 2024-04-25T08:57:42.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.495+0000 7f8387eb6640 1 -- 172.21.15.43:0/2800204079 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8380074ef0 con 0x7f838015df70 2024-04-25T08:57:42.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.496+0000 7f837e7fc640 1 -- 172.21.15.43:0/2800204079 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+16577 (secure 0 0 0) 0x7f8370022020 con 0x7f838015df70 2024-04-25T08:57:42.486 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:57:42.486 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":46,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:57:03.489946+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:57:00.135489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-04-25T08:57:00.830128+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:57:42.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.499+0000 7f8387eb6640 1 -- 172.21.15.43:0/2800204079 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f836c06b4d0 msgr2=0x7f836c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:42.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.499+0000 7f8387eb6640 1 --2- 172.21.15.43:0/2800204079 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f836c06b4d0 0x7f836c06d990 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f8378034000 tx=0x7f837802e000 comp rx=0 tx=0).stop 2024-04-25T08:57:42.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.499+0000 7f8387eb6640 1 -- 172.21.15.43:0/2800204079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f838015df70 msgr2=0x7f838014dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:42.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.499+0000 7f8387eb6640 1 --2- 172.21.15.43:0/2800204079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f838015df70 0x7f838014dc50 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f837000ef90 tx=0x7f837000c490 comp rx=0 tx=0).stop 2024-04-25T08:57:42.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.499+0000 7f8387eb6640 1 -- 172.21.15.43:0/2800204079 shutdown_connections 2024-04-25T08:57:42.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.499+0000 7f8387eb6640 1 --2- 172.21.15.43:0/2800204079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f838015df70 0x7f838014dc50 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:42.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.499+0000 7f8387eb6640 1 --2- 172.21.15.43:0/2800204079 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f83800cd680 0x7f83800c88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:42.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.499+0000 7f8387eb6640 1 --2- 172.21.15.43:0/2800204079 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f836c06b4d0 0x7f836c06d990 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:42.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.499+0000 7f8387eb6640 1 --2- 172.21.15.43:0/2800204079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f838006c510 0x7f83800c6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:42.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.499+0000 7f8387eb6640 1 -- 172.21.15.43:0/2800204079 >> 172.21.15.43:0/2800204079 conn(0x7f83800d17f0 msgr2=0x7f838006ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:42.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.499+0000 7f8387eb6640 1 -- 172.21.15.43:0/2800204079 shutdown_connections 2024-04-25T08:57:42.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.499+0000 7f8387eb6640 1 -- 172.21.15.43:0/2800204079 wait complete. 2024-04-25T08:57:42.501 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:57:42.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.613+0000 7f262e9d5640 1 Processor -- start 2024-04-25T08:57:42.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.613+0000 7f262e9d5640 1 -- start start 2024-04-25T08:57:42.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.614+0000 7f262e9d5640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f26280c8d00 0x7f2628162100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:42.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.614+0000 7f262e9d5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26280d24a0 con 0x7f262815df50 2024-04-25T08:57:42.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.614+0000 7f262e9d5640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26280d2740 con 0x7f26280ce660 2024-04-25T08:57:42.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.614+0000 7f262e9d5640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f262815e330 con 0x7f26280c8d00 2024-04-25T08:57:42.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.614+0000 7f2627fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f262815df50 0x7f26281619f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:57626/0 (socket says 172.21.15.43:57626) 2024-04-25T08:57:42.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.614+0000 7f2627fff640 1 -- 172.21.15.43:0/1085308667 learned_addr learned my addr 172.21.15.43:0/1085308667 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:42.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.614+0000 7f26277fe640 1 --2- 172.21.15.43:0/1085308667 >> v2:172.21.15.193:3300/0 conn(0x7f26280c8d00 0x7f2628162100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:42.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.614+0000 7f2626ffd640 1 -- 172.21.15.43:0/1085308667 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2127067078 0 0) 0x7f26280d24a0 con 0x7f262815df50 2024-04-25T08:57:42.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.614+0000 7f2626ffd640 1 -- 172.21.15.43:0/1085308667 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2614003520 con 0x7f262815df50 2024-04-25T08:57:42.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.614+0000 7f2626ffd640 1 -- 172.21.15.43:0/1085308667 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3723380732 0 0) 0x7f26280d2740 con 0x7f26280ce660 2024-04-25T08:57:42.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.614+0000 7f2626ffd640 1 -- 172.21.15.43:0/1085308667 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26280d24a0 con 0x7f26280ce660 2024-04-25T08:57:42.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.614+0000 7f26277fe640 1 -- 172.21.15.43:0/1085308667 >> v1:172.21.15.193:6789/0 conn(0x7f26280ce660 legacy=0x7f26280cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:42.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.614+0000 7f2626ffd640 1 -- 172.21.15.43:0/1085308667 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1388502932 0 0) 0x7f2614003520 con 0x7f262815df50 2024-04-25T08:57:42.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.614+0000 7f26277fe640 1 -- 172.21.15.43:0/1085308667 >> v1:172.21.15.43:6789/0 conn(0x7f262815df50 legacy=0x7f26281619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:42.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.614+0000 7f26277fe640 1 -- 172.21.15.43:0/1085308667 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26280d2a20 con 0x7f26280c8d00 2024-04-25T08:57:42.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.615+0000 7f26277fe640 1 --2- 172.21.15.43:0/1085308667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26280c8d00 0x7f2628162100 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f26100097a0 tx=0x7f261002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2a78a19601cc2e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:42.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.615+0000 7f2626ffd640 1 -- 172.21.15.43:0/1085308667 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2610037070 con 0x7f26280c8d00 2024-04-25T08:57:42.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.615+0000 7f2626ffd640 1 -- 172.21.15.43:0/1085308667 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f261002ac50 con 0x7f26280c8d00 2024-04-25T08:57:42.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.616+0000 7f2626ffd640 1 -- 172.21.15.43:0/1085308667 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2618001690 con 0x7f26280c8d00 2024-04-25T08:57:42.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.616+0000 7f262e9d5640 1 -- 172.21.15.43:0/1085308667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26280c8d00 msgr2=0x7f2628162100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:42.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.616+0000 7f262e9d5640 1 --2- 172.21.15.43:0/1085308667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26280c8d00 0x7f2628162100 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f26100097a0 tx=0x7f261002a1c0 comp rx=0 tx=0).stop 2024-04-25T08:57:42.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.616+0000 7f262e9d5640 1 -- 172.21.15.43:0/1085308667 shutdown_connections 2024-04-25T08:57:42.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.616+0000 7f262e9d5640 1 --2- 172.21.15.43:0/1085308667 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26280c8d00 0x7f2628162100 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:42.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.616+0000 7f262e9d5640 1 -- 172.21.15.43:0/1085308667 >> 172.21.15.43:0/1085308667 conn(0x7f26280d3a30 msgr2=0x7f26280d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:42.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.616+0000 7f262e9d5640 1 -- 172.21.15.43:0/1085308667 shutdown_connections 2024-04-25T08:57:42.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.616+0000 7f262e9d5640 1 -- 172.21.15.43:0/1085308667 wait complete. 2024-04-25T08:57:42.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.617+0000 7f262e9d5640 1 Processor -- start 2024-04-25T08:57:42.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.618+0000 7f262e9d5640 1 -- start start 2024-04-25T08:57:42.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.618+0000 7f262e9d5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26280c8d00 0x7f26281f0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:42.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.618+0000 7f262e9d5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26280ce660 0x7f26281f4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:42.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.618+0000 7f262e9d5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f262815df50 0x7f26280cb430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:42.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.618+0000 7f262e9d5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26280d2190 con 0x7f26280c8d00 2024-04-25T08:57:42.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.618+0000 7f262e9d5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26280c77d0 con 0x7f26280ce660 2024-04-25T08:57:42.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.618+0000 7f262e9d5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2628162640 con 0x7f262815df50 2024-04-25T08:57:42.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.618+0000 7f2627fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26280c8d00 0x7f26281f0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:42.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.618+0000 7f2627fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26280c8d00 0x7f26281f0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45760/0 (socket says 172.21.15.43:45760) 2024-04-25T08:57:42.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.618+0000 7f2627fff640 1 -- 172.21.15.43:0/1022127266 learned_addr learned my addr 172.21.15.43:0/1022127266 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:42.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.618+0000 7f26277fe640 1 --2- 172.21.15.43:0/1022127266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26280ce660 0x7f26281f4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:42.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.618+0000 7f262cf72640 1 --2- 172.21.15.43:0/1022127266 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f262815df50 0x7f26280cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:42.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.619+0000 7f2627fff640 1 -- 172.21.15.43:0/1022127266 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f262815df50 msgr2=0x7f26280cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:42.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.619+0000 7f2627fff640 1 --2- 172.21.15.43:0/1022127266 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f262815df50 0x7f26280cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:42.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.619+0000 7f2627fff640 1 -- 172.21.15.43:0/1022127266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26280ce660 msgr2=0x7f26281f4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:42.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.619+0000 7f2627fff640 1 --2- 172.21.15.43:0/1022127266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26280ce660 0x7f26281f4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:42.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.619+0000 7f2627fff640 1 -- 172.21.15.43:0/1022127266 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f262815b6e0 con 0x7f26280c8d00 2024-04-25T08:57:42.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.619+0000 7f2627fff640 1 --2- 172.21.15.43:0/1022127266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26280c8d00 0x7f26281f0e30 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f2618001200 tx=0x7f261800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:42.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.619+0000 7f2624ff9640 1 -- 172.21.15.43:0/1022127266 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2618018070 con 0x7f26280c8d00 2024-04-25T08:57:42.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.619+0000 7f2624ff9640 1 -- 172.21.15.43:0/1022127266 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26180074e0 con 0x7f26280c8d00 2024-04-25T08:57:42.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.619+0000 7f2624ff9640 1 -- 172.21.15.43:0/1022127266 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2618004a20 con 0x7f26280c8d00 2024-04-25T08:57:42.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.619+0000 7f262e9d5640 1 -- 172.21.15.43:0/1022127266 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f262815b9c0 con 0x7f26280c8d00 2024-04-25T08:57:42.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.619+0000 7f262e9d5640 1 -- 172.21.15.43:0/1022127266 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f262815bf80 con 0x7f26280c8d00 2024-04-25T08:57:42.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.620+0000 7f262e9d5640 1 -- 172.21.15.43:0/1022127266 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26280680f0 con 0x7f26280c8d00 2024-04-25T08:57:42.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.621+0000 7f2624ff9640 1 -- 172.21.15.43:0/1022127266 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f261801b070 con 0x7f26280c8d00 2024-04-25T08:57:42.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.621+0000 7f2624ff9640 1 --2- 172.21.15.43:0/1022127266 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f260806b4d0 0x7f260806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:42.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.621+0000 7f2624ff9640 1 -- 172.21.15.43:0/1022127266 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2618089440 con 0x7f26280c8d00 2024-04-25T08:57:42.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.621+0000 7f26277fe640 1 --2- 172.21.15.43:0/1022127266 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f260806b4d0 0x7f260806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:42.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.622+0000 7f26277fe640 1 --2- 172.21.15.43:0/1022127266 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f260806b4d0 0x7f260806d990 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2610034000 tx=0x7f261002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:42.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.625+0000 7f2624ff9640 1 -- 172.21.15.43:0/1022127266 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2618057d00 con 0x7f26280c8d00 2024-04-25T08:57:42.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.846+0000 7f262e9d5640 1 -- 172.21.15.43:0/1022127266 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f2628074ef0 con 0x7f26280c8d00 2024-04-25T08:57:42.836 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-04-25T08:57:42.847+0000 7fe4fb198640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:57:42.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.848+0000 7f2624ff9640 1 -- 172.21.15.43:0/1022127266 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) v1 ==== 161+0+0 (secure 0 0 0) 0x7f261805bda0 con 0x7f26280c8d00 2024-04-25T08:57:42.837 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:57:42.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.851+0000 7f262e9d5640 1 -- 172.21.15.43:0/1022127266 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f260806b4d0 msgr2=0x7f260806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:42.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.851+0000 7f262e9d5640 1 --2- 172.21.15.43:0/1022127266 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f260806b4d0 0x7f260806d990 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f2610034000 tx=0x7f261002e000 comp rx=0 tx=0).stop 2024-04-25T08:57:42.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.851+0000 7f262e9d5640 1 -- 172.21.15.43:0/1022127266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26280c8d00 msgr2=0x7f26281f0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:42.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.851+0000 7f262e9d5640 1 --2- 172.21.15.43:0/1022127266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26280c8d00 0x7f26281f0e30 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f2618001200 tx=0x7f261800c9b0 comp rx=0 tx=0).stop 2024-04-25T08:57:42.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.851+0000 7f262e9d5640 1 -- 172.21.15.43:0/1022127266 shutdown_connections 2024-04-25T08:57:42.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.851+0000 7f262e9d5640 1 --2- 172.21.15.43:0/1022127266 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f262815df50 0x7f26280cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:42.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.851+0000 7f262e9d5640 1 --2- 172.21.15.43:0/1022127266 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f260806b4d0 0x7f260806d990 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:42.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.851+0000 7f262e9d5640 1 --2- 172.21.15.43:0/1022127266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26280ce660 0x7f26281f4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:42.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.851+0000 7f262e9d5640 1 --2- 172.21.15.43:0/1022127266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26280c8d00 0x7f26281f0e30 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:42.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.851+0000 7f262e9d5640 1 -- 172.21.15.43:0/1022127266 >> 172.21.15.43:0/1022127266 conn(0x7f26280d3a30 msgr2=0x7f26280d03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:42.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.851+0000 7f262e9d5640 1 -- 172.21.15.43:0/1022127266 shutdown_connections 2024-04-25T08:57:42.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.851+0000 7f262e9d5640 1 -- 172.21.15.43:0/1022127266 wait complete. 2024-04-25T08:57:42.851 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T08:57:42.852 INFO:tasks.ceph.mds.d.smithi193.stderr: -2237> 2024-04-25T08:57:30.707+0000 7f52a22c0640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000a4e [2,head] /scrub_test_path/ceph-qa-suite/new_file_1 auth v787 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={5856=0-4194304@1} caps={5856=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@3},l=5856 | request=0 lock=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x556abacbd700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000a4e:[<0x10000000001/new_file_1 v787>,<0x10000000000/ceph-qa-suite v84>,<0x1/scrub_test_path v40>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":-61} 2024-04-25T08:57:42.881 INFO:tasks.ceph.mds.d.smithi193.stdout:starting mds.d at 2024-04-25T08:57:42.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.964+0000 7f303c163640 1 Processor -- start 2024-04-25T08:57:42.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.965+0000 7f303c163640 1 -- start start 2024-04-25T08:57:42.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.965+0000 7f303c163640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f303415a960 0x7f30340b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:42.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.965+0000 7f303c163640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f30340b3ed0 0x7f30340ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:42.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.966+0000 7f303c163640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30340af0a0 con 0x7f30340aec70 2024-04-25T08:57:42.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.966+0000 7f303c163640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3034071340 con 0x7f30340b3ed0 2024-04-25T08:57:42.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.966+0000 7f303c163640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f303415b110 con 0x7f303415a960 2024-04-25T08:57:42.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.966+0000 7f30396fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f30340b3ed0 0x7f30340ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:42.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.966+0000 7f30396fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f30340b3ed0 0x7f30340ae730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45766/0 (socket says 172.21.15.43:45766) 2024-04-25T08:57:42.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.966+0000 7f30396fe640 1 -- 172.21.15.43:0/692890507 learned_addr learned my addr 172.21.15.43:0/692890507 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:42.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.966+0000 7f3039eff640 1 --2- 172.21.15.43:0/692890507 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f303415a960 0x7f30340b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:42.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.966+0000 7f30396fe640 1 -- 172.21.15.43:0/692890507 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f303415a960 msgr2=0x7f30340b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:42.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.966+0000 7f30396fe640 1 --2- 172.21.15.43:0/692890507 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f303415a960 0x7f30340b3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:42.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.966+0000 7f30396fe640 1 -- 172.21.15.43:0/692890507 >> v1:172.21.15.193:6789/0 conn(0x7f30340aec70 legacy=0x7f303406d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:42.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.966+0000 7f30396fe640 1 -- 172.21.15.43:0/692890507 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f303415b3f0 con 0x7f30340b3ed0 2024-04-25T08:57:42.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.966+0000 7f3039eff640 1 --2- 172.21.15.43:0/692890507 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f303415a960 0x7f30340b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:42.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.967+0000 7f30396fe640 1 --2- 172.21.15.43:0/692890507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30340b3ed0 0x7f30340ae730 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f302400bb10 tx=0x7f302402c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b131c313af8031 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:42.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.967+0000 7f3038efd640 1 -- 172.21.15.43:0/692890507 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3024036020 con 0x7f30340b3ed0 2024-04-25T08:57:42.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.967+0000 7f3038efd640 1 -- 172.21.15.43:0/692890507 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3024032820 con 0x7f30340b3ed0 2024-04-25T08:57:42.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.967+0000 7f3038efd640 1 -- 172.21.15.43:0/692890507 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3024032bf0 con 0x7f30340b3ed0 2024-04-25T08:57:42.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.967+0000 7f303c163640 1 -- 172.21.15.43:0/692890507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30340b3ed0 msgr2=0x7f30340ae730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:42.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.967+0000 7f303c163640 1 --2- 172.21.15.43:0/692890507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30340b3ed0 0x7f30340ae730 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f302400bb10 tx=0x7f302402c920 comp rx=0 tx=0).stop 2024-04-25T08:57:42.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.967+0000 7f303c163640 1 -- 172.21.15.43:0/692890507 shutdown_connections 2024-04-25T08:57:42.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.967+0000 7f303c163640 1 --2- 172.21.15.43:0/692890507 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30340b3ed0 0x7f30340ae730 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:42.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.967+0000 7f303c163640 1 --2- 172.21.15.43:0/692890507 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f303415a960 0x7f30340b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:42.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.967+0000 7f303c163640 1 -- 172.21.15.43:0/692890507 >> 172.21.15.43:0/692890507 conn(0x7f303406ab30 msgr2=0x7f303406af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:42.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.968+0000 7f303c163640 1 -- 172.21.15.43:0/692890507 shutdown_connections 2024-04-25T08:57:42.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.968+0000 7f303c163640 1 -- 172.21.15.43:0/692890507 wait complete. 2024-04-25T08:57:42.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.969+0000 7f303c163640 1 Processor -- start 2024-04-25T08:57:42.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.969+0000 7f303c163640 1 -- start start 2024-04-25T08:57:42.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.969+0000 7f303c163640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30340aec70 0x7f303414b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:42.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.969+0000 7f303c163640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30340b3ed0 0x7f303415d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:42.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.969+0000 7f303c163640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f303415a960 0x7f30341596b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:42.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.969+0000 7f303c163640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30340cc440 con 0x7f30340b3ed0 2024-04-25T08:57:42.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.969+0000 7f303c163640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f303415b9b0 con 0x7f30340aec70 2024-04-25T08:57:42.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.969+0000 7f303c163640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30340cd940 con 0x7f303415a960 2024-04-25T08:57:42.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.969+0000 7f30396fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30340b3ed0 0x7f303415d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:42.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.969+0000 7f30396fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30340b3ed0 0x7f303415d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45776/0 (socket says 172.21.15.43:45776) 2024-04-25T08:57:42.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.969+0000 7f30396fe640 1 -- 172.21.15.43:0/2741057324 learned_addr learned my addr 172.21.15.43:0/2741057324 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:42.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.970+0000 7f3039eff640 1 --2- 172.21.15.43:0/2741057324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30340aec70 0x7f303414b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:42.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.970+0000 7f303a700640 1 --2- 172.21.15.43:0/2741057324 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f303415a960 0x7f30341596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:42.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.970+0000 7f30396fe640 1 -- 172.21.15.43:0/2741057324 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f303415a960 msgr2=0x7f30341596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:42.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.970+0000 7f30396fe640 1 --2- 172.21.15.43:0/2741057324 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f303415a960 0x7f30341596b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:42.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.970+0000 7f30396fe640 1 -- 172.21.15.43:0/2741057324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30340aec70 msgr2=0x7f303414b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:42.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.970+0000 7f30396fe640 1 --2- 172.21.15.43:0/2741057324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30340aec70 0x7f303414b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:42.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.970+0000 7f30396fe640 1 -- 172.21.15.43:0/2741057324 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3034150700 con 0x7f30340b3ed0 2024-04-25T08:57:42.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.970+0000 7f30396fe640 1 --2- 172.21.15.43:0/2741057324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30340b3ed0 0x7f303415d0a0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f302400b2c0 tx=0x7f30240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:42.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.970+0000 7f302a7fc640 1 -- 172.21.15.43:0/2741057324 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3024036040 con 0x7f30340b3ed0 2024-04-25T08:57:42.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.970+0000 7f302a7fc640 1 -- 172.21.15.43:0/2741057324 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f302403cd70 con 0x7f30340b3ed0 2024-04-25T08:57:42.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.970+0000 7f303c163640 1 -- 172.21.15.43:0/2741057324 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30341509e0 con 0x7f30340b3ed0 2024-04-25T08:57:42.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.970+0000 7f302a7fc640 1 -- 172.21.15.43:0/2741057324 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3024046630 con 0x7f30340b3ed0 2024-04-25T08:57:42.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.970+0000 7f303c163640 1 -- 172.21.15.43:0/2741057324 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3034150fa0 con 0x7f30340b3ed0 2024-04-25T08:57:42.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.971+0000 7f303c163640 1 -- 172.21.15.43:0/2741057324 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3034074f60 con 0x7f30340b3ed0 2024-04-25T08:57:42.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.972+0000 7f302a7fc640 1 -- 172.21.15.43:0/2741057324 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f302404d060 con 0x7f30340b3ed0 2024-04-25T08:57:42.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.972+0000 7f302a7fc640 1 --2- 172.21.15.43:0/2741057324 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f301806b4d0 0x7f301806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:42.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.972+0000 7f302a7fc640 1 -- 172.21.15.43:0/2741057324 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f30240ad240 con 0x7f30340b3ed0 2024-04-25T08:57:42.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.972+0000 7f3039eff640 1 --2- 172.21.15.43:0/2741057324 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f301806b4d0 0x7f301806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:42.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.973+0000 7f3039eff640 1 --2- 172.21.15.43:0/2741057324 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f301806b4d0 0x7f301806d990 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3030006fd0 tx=0x7f3030008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:42.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:42.977+0000 7f302a7fc640 1 -- 172.21.15.43:0/2741057324 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3024041070 con 0x7f30340b3ed0 2024-04-25T08:57:43.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.196+0000 7f303c163640 1 -- 172.21.15.43:0/2741057324 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f303414ba80 con 0x7f30340b3ed0 2024-04-25T08:57:43.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.852+0000 7f302a7fc640 1 -- 172.21.15.43:0/2741057324 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7f302402f070 con 0x7f30340b3ed0 2024-04-25T08:57:43.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.855+0000 7f303c163640 1 -- 172.21.15.43:0/2741057324 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f301806b4d0 msgr2=0x7f301806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:43.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.855+0000 7f303c163640 1 --2- 172.21.15.43:0/2741057324 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f301806b4d0 0x7f301806d990 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f3030006fd0 tx=0x7f3030008040 comp rx=0 tx=0).stop 2024-04-25T08:57:43.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.855+0000 7f303c163640 1 -- 172.21.15.43:0/2741057324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30340b3ed0 msgr2=0x7f303415d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:43.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.855+0000 7f303c163640 1 --2- 172.21.15.43:0/2741057324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30340b3ed0 0x7f303415d0a0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f302400b2c0 tx=0x7f30240046c0 comp rx=0 tx=0).stop 2024-04-25T08:57:43.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.855+0000 7f303c163640 1 -- 172.21.15.43:0/2741057324 shutdown_connections 2024-04-25T08:57:43.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.855+0000 7f303c163640 1 --2- 172.21.15.43:0/2741057324 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f303415a960 0x7f30341596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:43.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.855+0000 7f303c163640 1 --2- 172.21.15.43:0/2741057324 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30340b3ed0 0x7f303415d0a0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:43.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.855+0000 7f303c163640 1 --2- 172.21.15.43:0/2741057324 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f301806b4d0 0x7f301806d990 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:43.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.855+0000 7f303c163640 1 --2- 172.21.15.43:0/2741057324 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30340aec70 0x7f303414b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:43.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.855+0000 7f303c163640 1 -- 172.21.15.43:0/2741057324 >> 172.21.15.43:0/2741057324 conn(0x7f303406ab30 msgr2=0x7f3034158f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:43.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.855+0000 7f303c163640 1 -- 172.21.15.43:0/2741057324 shutdown_connections 2024-04-25T08:57:43.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.855+0000 7f303c163640 1 -- 172.21.15.43:0/2741057324 wait complete. 2024-04-25T08:57:43.858 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T08:57:43.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.980+0000 7f450bc76640 1 Processor -- start 2024-04-25T08:57:43.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.981+0000 7f450bc76640 1 -- start start 2024-04-25T08:57:43.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.981+0000 7f450bc76640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f450415def0 0x7f45040d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:43.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.981+0000 7f450bc76640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45040d4670 0x7f45040d03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:43.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.981+0000 7f450bc76640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45040d2370 con 0x7f45040d0930 2024-04-25T08:57:43.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.982+0000 7f450bc76640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45040b37b0 con 0x7f450415def0 2024-04-25T08:57:43.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.982+0000 7f450bc76640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45040d2650 con 0x7f45040d4670 2024-04-25T08:57:43.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.982+0000 7f4509a12640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f450415def0 0x7f45040d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:43.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.982+0000 7f4509a12640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f450415def0 0x7f45040d4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45778/0 (socket says 172.21.15.43:45778) 2024-04-25T08:57:43.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.982+0000 7f4509a12640 1 -- 172.21.15.43:0/448701747 learned_addr learned my addr 172.21.15.43:0/448701747 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:43.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.982+0000 7f4509211640 1 --2- 172.21.15.43:0/448701747 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45040d4670 0x7f45040d03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:43.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.982+0000 7f4509a12640 1 -- 172.21.15.43:0/448701747 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45040d4670 msgr2=0x7f45040d03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:43.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.982+0000 7f4509a12640 1 --2- 172.21.15.43:0/448701747 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45040d4670 0x7f45040d03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:43.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.982+0000 7f4509a12640 1 -- 172.21.15.43:0/448701747 >> v1:172.21.15.193:6789/0 conn(0x7f45040d0930 legacy=0x7f45040ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:43.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.982+0000 7f4509a12640 1 -- 172.21.15.43:0/448701747 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45040b3440 con 0x7f450415def0 2024-04-25T08:57:43.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.983+0000 7f4509a12640 1 --2- 172.21.15.43:0/448701747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f450415def0 0x7f45040d4130 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f4500009870 tx=0x7f450002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f518fa2ca32d41 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:43.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.983+0000 7f4508a10640 1 -- 172.21.15.43:0/448701747 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4500037070 con 0x7f450415def0 2024-04-25T08:57:43.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.983+0000 7f4508a10640 1 -- 172.21.15.43:0/448701747 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4500032440 con 0x7f450415def0 2024-04-25T08:57:43.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.983+0000 7f4508a10640 1 -- 172.21.15.43:0/448701747 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4500032810 con 0x7f450415def0 2024-04-25T08:57:43.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.983+0000 7f450bc76640 1 -- 172.21.15.43:0/448701747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f450415def0 msgr2=0x7f45040d4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:43.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.983+0000 7f450bc76640 1 --2- 172.21.15.43:0/448701747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f450415def0 0x7f45040d4130 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f4500009870 tx=0x7f450002a1c0 comp rx=0 tx=0).stop 2024-04-25T08:57:43.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.984+0000 7f450bc76640 1 -- 172.21.15.43:0/448701747 shutdown_connections 2024-04-25T08:57:43.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.984+0000 7f450bc76640 1 --2- 172.21.15.43:0/448701747 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45040d4670 0x7f45040d03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:43.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.984+0000 7f450bc76640 1 --2- 172.21.15.43:0/448701747 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f450415def0 0x7f45040d4130 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:43.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.984+0000 7f450bc76640 1 -- 172.21.15.43:0/448701747 >> 172.21.15.43:0/448701747 conn(0x7f450409cd60 msgr2=0x7f45040d3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:43.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.984+0000 7f450bc76640 1 -- 172.21.15.43:0/448701747 shutdown_connections 2024-04-25T08:57:43.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.984+0000 7f450bc76640 1 -- 172.21.15.43:0/448701747 wait complete. 2024-04-25T08:57:43.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.985+0000 7f450bc76640 1 Processor -- start 2024-04-25T08:57:43.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.985+0000 7f450bc76640 1 -- start start 2024-04-25T08:57:43.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.986+0000 7f450bc76640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45040d0930 0x7f45040b7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:43.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.986+0000 7f450bc76640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45040d4670 0x7f450414a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:43.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.986+0000 7f450bc76640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f450415def0 0x7f450406bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:43.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.986+0000 7f450bc76640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45040d2150 con 0x7f45040d0930 2024-04-25T08:57:43.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.986+0000 7f450bc76640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45040d1fd0 con 0x7f450415def0 2024-04-25T08:57:43.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.986+0000 7f450bc76640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45040b4c50 con 0x7f45040d4670 2024-04-25T08:57:43.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.986+0000 7f4509a12640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45040d0930 0x7f45040b7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:43.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.986+0000 7f4509a12640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45040d0930 0x7f45040b7650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45780/0 (socket says 172.21.15.43:45780) 2024-04-25T08:57:43.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.986+0000 7f4509a12640 1 -- 172.21.15.43:0/1192618075 learned_addr learned my addr 172.21.15.43:0/1192618075 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:43.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.986+0000 7f4509211640 1 --2- 172.21.15.43:0/1192618075 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45040d4670 0x7f450414a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:43.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.986+0000 7f450a213640 1 --2- 172.21.15.43:0/1192618075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f450415def0 0x7f450406bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:43.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.986+0000 7f4509a12640 1 -- 172.21.15.43:0/1192618075 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45040d4670 msgr2=0x7f450414a0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:43.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.986+0000 7f4509a12640 1 --2- 172.21.15.43:0/1192618075 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45040d4670 0x7f450414a0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:43.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.986+0000 7f4509a12640 1 -- 172.21.15.43:0/1192618075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f450415def0 msgr2=0x7f450406bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:43.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.986+0000 7f4509a12640 1 --2- 172.21.15.43:0/1192618075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f450415def0 0x7f450406bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:43.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.986+0000 7f4509a12640 1 -- 172.21.15.43:0/1192618075 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45040b7dc0 con 0x7f45040d0930 2024-04-25T08:57:43.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.986+0000 7f4509a12640 1 --2- 172.21.15.43:0/1192618075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45040d0930 0x7f45040b7650 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f4500000c00 tx=0x7f45000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:43.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.986+0000 7f44fa7fc640 1 -- 172.21.15.43:0/1192618075 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4500041020 con 0x7f45040d0930 2024-04-25T08:57:43.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.986+0000 7f44fa7fc640 1 -- 172.21.15.43:0/1192618075 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45000306c0 con 0x7f45040d0930 2024-04-25T08:57:43.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.986+0000 7f44fa7fc640 1 -- 172.21.15.43:0/1192618075 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4500037040 con 0x7f45040d0930 2024-04-25T08:57:43.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.986+0000 7f450bc76640 1 -- 172.21.15.43:0/1192618075 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45040b80a0 con 0x7f45040d0930 2024-04-25T08:57:43.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.986+0000 7f450bc76640 1 -- 172.21.15.43:0/1192618075 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f450414a5f0 con 0x7f45040d0930 2024-04-25T08:57:43.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.987+0000 7f450bc76640 1 -- 172.21.15.43:0/1192618075 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f450415d100 con 0x7f45040d0930 2024-04-25T08:57:43.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.988+0000 7f44fa7fc640 1 -- 172.21.15.43:0/1192618075 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4500044020 con 0x7f45040d0930 2024-04-25T08:57:43.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.988+0000 7f44fa7fc640 1 --2- 172.21.15.43:0/1192618075 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f44e806b500 0x7f44e806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:43.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.988+0000 7f44fa7fc640 1 -- 172.21.15.43:0/1192618075 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f45000aa9d0 con 0x7f45040d0930 2024-04-25T08:57:43.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.988+0000 7f4509211640 1 --2- 172.21.15.43:0/1192618075 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f44e806b500 0x7f44e806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:43.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.989+0000 7f4509211640 1 --2- 172.21.15.43:0/1192618075 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f44e806b500 0x7f44e806d9c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f45041f6f50 tx=0x7f44f4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:43.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:43.992+0000 7f44fa7fc640 1 -- 172.21.15.43:0/1192618075 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f450415d100 con 0x7f45040d0930 2024-04-25T08:57:44.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.173+0000 7f450bc76640 1 -- 172.21.15.43:0/1192618075 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4504068a40 con 0x7f45040d0930 2024-04-25T08:57:44.469 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:57:44.472 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:57:44.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.853+0000 7f44fa7fc640 1 -- 172.21.15.43:0/1192618075 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v48) v1 ==== 181+0+0 (secure 0 0 0) 0x7f450004b030 con 0x7f45040d0930 2024-04-25T08:57:44.843 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' removed 2024-04-25T08:57:44.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.856+0000 7f450bc76640 1 -- 172.21.15.43:0/1192618075 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f44e806b500 msgr2=0x7f44e806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:44.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.856+0000 7f450bc76640 1 --2- 172.21.15.43:0/1192618075 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f44e806b500 0x7f44e806d9c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f45041f6f50 tx=0x7f44f4008040 comp rx=0 tx=0).stop 2024-04-25T08:57:44.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.856+0000 7f450bc76640 1 -- 172.21.15.43:0/1192618075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45040d0930 msgr2=0x7f45040b7650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:44.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.856+0000 7f450bc76640 1 --2- 172.21.15.43:0/1192618075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45040d0930 0x7f45040b7650 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f4500000c00 tx=0x7f45000046c0 comp rx=0 tx=0).stop 2024-04-25T08:57:44.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.857+0000 7f450bc76640 1 -- 172.21.15.43:0/1192618075 shutdown_connections 2024-04-25T08:57:44.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.857+0000 7f450bc76640 1 --2- 172.21.15.43:0/1192618075 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f44e806b500 0x7f44e806d9c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:44.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.857+0000 7f450bc76640 1 --2- 172.21.15.43:0/1192618075 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f450415def0 0x7f450406bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:44.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.857+0000 7f450bc76640 1 --2- 172.21.15.43:0/1192618075 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45040d4670 0x7f450414a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:44.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.857+0000 7f450bc76640 1 --2- 172.21.15.43:0/1192618075 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45040d0930 0x7f45040b7650 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:44.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.857+0000 7f450bc76640 1 -- 172.21.15.43:0/1192618075 >> 172.21.15.43:0/1192618075 conn(0x7f450409cd60 msgr2=0x7f45040b09c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:44.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.857+0000 7f450bc76640 1 -- 172.21.15.43:0/1192618075 shutdown_connections 2024-04-25T08:57:44.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.857+0000 7f450bc76640 1 -- 172.21.15.43:0/1192618075 wait complete. 2024-04-25T08:57:44.860 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T08:57:44.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.980+0000 7f624a831640 1 Processor -- start 2024-04-25T08:57:44.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.980+0000 7f624a831640 1 -- start start 2024-04-25T08:57:44.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.981+0000 7f624a831640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f62440d0850 0x7f62440b4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:44.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.981+0000 7f624a831640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62440d2370 con 0x7f62440d03f0 2024-04-25T08:57:44.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.981+0000 7f624a831640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62440d2650 con 0x7f624415def0 2024-04-25T08:57:44.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.981+0000 7f624a831640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62440b37b0 con 0x7f62440d0850 2024-04-25T08:57:44.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.981+0000 7f62437fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f62440d03f0 0x7f62440b3c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:57628/0 (socket says 172.21.15.43:57628) 2024-04-25T08:57:44.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.981+0000 7f62437fe640 1 -- 172.21.15.43:0/992054095 learned_addr learned my addr 172.21.15.43:0/992054095 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:44.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.981+0000 7f6248dce640 1 --2- 172.21.15.43:0/992054095 >> v2:172.21.15.43:3300/0 conn(0x7f62440d0850 0x7f62440b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:44.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.981+0000 7f6242ffd640 1 -- 172.21.15.43:0/992054095 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 797938080 0 0) 0x7f62440d2370 con 0x7f62440d03f0 2024-04-25T08:57:44.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.981+0000 7f6242ffd640 1 -- 172.21.15.43:0/992054095 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f622c003520 con 0x7f62440d03f0 2024-04-25T08:57:44.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.981+0000 7f6248dce640 1 -- 172.21.15.43:0/992054095 >> v1:172.21.15.193:6789/0 conn(0x7f624415def0 legacy=0x7f62440d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:44.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.981+0000 7f6248dce640 1 -- 172.21.15.43:0/992054095 >> v1:172.21.15.43:6789/0 conn(0x7f62440d03f0 legacy=0x7f62440b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:44.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.981+0000 7f6248dce640 1 -- 172.21.15.43:0/992054095 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62440b3440 con 0x7f62440d0850 2024-04-25T08:57:44.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.982+0000 7f6248dce640 1 --2- 172.21.15.43:0/992054095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62440d0850 0x7f62440b4350 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f623c009d90 tx=0x7f623c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f84219e3165e7567 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:44.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.982+0000 7f6242ffd640 1 -- 172.21.15.43:0/992054095 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623c031600 con 0x7f62440d0850 2024-04-25T08:57:44.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.982+0000 7f6242ffd640 1 -- 172.21.15.43:0/992054095 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f623c0317a0 con 0x7f62440d0850 2024-04-25T08:57:44.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.982+0000 7f6242ffd640 1 -- 172.21.15.43:0/992054095 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623c031b70 con 0x7f62440d0850 2024-04-25T08:57:44.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.983+0000 7f624a831640 1 -- 172.21.15.43:0/992054095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62440d0850 msgr2=0x7f62440b4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:44.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.983+0000 7f624a831640 1 --2- 172.21.15.43:0/992054095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62440d0850 0x7f62440b4350 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f623c009d90 tx=0x7f623c02a610 comp rx=0 tx=0).stop 2024-04-25T08:57:44.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.983+0000 7f624a831640 1 -- 172.21.15.43:0/992054095 shutdown_connections 2024-04-25T08:57:44.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.983+0000 7f624a831640 1 --2- 172.21.15.43:0/992054095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62440d0850 0x7f62440b4350 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:44.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.983+0000 7f624a831640 1 -- 172.21.15.43:0/992054095 >> 172.21.15.43:0/992054095 conn(0x7f624409cd60 msgr2=0x7f62440d3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:44.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.983+0000 7f624a831640 1 -- 172.21.15.43:0/992054095 shutdown_connections 2024-04-25T08:57:44.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.983+0000 7f624a831640 1 -- 172.21.15.43:0/992054095 wait complete. 2024-04-25T08:57:44.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.984+0000 7f624a831640 1 Processor -- start 2024-04-25T08:57:44.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.984+0000 7f624a831640 1 -- start start 2024-04-25T08:57:44.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.985+0000 7f624a831640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62440d03f0 0x7f6244155060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:44.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.985+0000 7f624a831640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62440d0850 0x7f62440cb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:44.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.985+0000 7f624a831640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f624415def0 0x7f62441f3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:44.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.985+0000 7f624a831640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62440d2930 con 0x7f624415def0 2024-04-25T08:57:44.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.985+0000 7f624a831640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62440d4840 con 0x7f62440d0850 2024-04-25T08:57:44.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.985+0000 7f624a831640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62440d1fd0 con 0x7f62440d03f0 2024-04-25T08:57:44.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.985+0000 7f6248dce640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f624415def0 0x7f62441f3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:44.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.985+0000 7f6248dce640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f624415def0 0x7f62441f3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45800/0 (socket says 172.21.15.43:45800) 2024-04-25T08:57:44.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.985+0000 7f6248dce640 1 -- 172.21.15.43:0/3240343646 learned_addr learned my addr 172.21.15.43:0/3240343646 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:44.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.986+0000 7f62437fe640 1 --2- 172.21.15.43:0/3240343646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62440d0850 0x7f62440cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:44.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.986+0000 7f6243fff640 1 --2- 172.21.15.43:0/3240343646 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62440d03f0 0x7f6244155060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:44.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.986+0000 7f6248dce640 1 -- 172.21.15.43:0/3240343646 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62440d03f0 msgr2=0x7f6244155060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:44.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.986+0000 7f6248dce640 1 --2- 172.21.15.43:0/3240343646 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62440d03f0 0x7f6244155060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:44.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.986+0000 7f6248dce640 1 -- 172.21.15.43:0/3240343646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62440d0850 msgr2=0x7f62440cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:44.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.986+0000 7f6248dce640 1 --2- 172.21.15.43:0/3240343646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62440d0850 0x7f62440cb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:44.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.986+0000 7f6248dce640 1 -- 172.21.15.43:0/3240343646 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62440cac70 con 0x7f624415def0 2024-04-25T08:57:44.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.986+0000 7f62437fe640 1 --2- 172.21.15.43:0/3240343646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62440d0850 0x7f62440cb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:44.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.986+0000 7f6243fff640 1 --2- 172.21.15.43:0/3240343646 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62440d03f0 0x7f6244155060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:44.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.986+0000 7f6248dce640 1 --2- 172.21.15.43:0/3240343646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f624415def0 0x7f62441f3260 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f623c009d90 tx=0x7f623c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:44.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.986+0000 7f6240ff9640 1 -- 172.21.15.43:0/3240343646 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623c040070 con 0x7f624415def0 2024-04-25T08:57:44.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.986+0000 7f6240ff9640 1 -- 172.21.15.43:0/3240343646 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f623c030b60 con 0x7f624415def0 2024-04-25T08:57:44.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.986+0000 7f6240ff9640 1 -- 172.21.15.43:0/3240343646 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623c009450 con 0x7f624415def0 2024-04-25T08:57:44.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.986+0000 7f624a831640 1 -- 172.21.15.43:0/3240343646 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62441f3cc0 con 0x7f624415def0 2024-04-25T08:57:44.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.986+0000 7f624a831640 1 -- 172.21.15.43:0/3240343646 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62440cb990 con 0x7f624415def0 2024-04-25T08:57:44.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.987+0000 7f624a831640 1 -- 172.21.15.43:0/3240343646 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62441f41e0 con 0x7f624415def0 2024-04-25T08:57:44.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.987+0000 7f6240ff9640 1 -- 172.21.15.43:0/3240343646 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f623c039040 con 0x7f624415def0 2024-04-25T08:57:44.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.988+0000 7f6240ff9640 1 --2- 172.21.15.43:0/3240343646 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f623006b4d0 0x7f623006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:44.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.988+0000 7f6240ff9640 1 -- 172.21.15.43:0/3240343646 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f623c0ab340 con 0x7f624415def0 2024-04-25T08:57:44.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.988+0000 7f6243fff640 1 --2- 172.21.15.43:0/3240343646 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f623006b4d0 0x7f623006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:44.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.989+0000 7f6243fff640 1 --2- 172.21.15.43:0/3240343646 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f623006b4d0 0x7f623006d990 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6244160c00 tx=0x7f6234008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:44.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:44.992+0000 7f6240ff9640 1 -- 172.21.15.43:0/3240343646 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f62441f41e0 con 0x7f624415def0 2024-04-25T08:57:45.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.180+0000 7f624a831640 1 -- 172.21.15.43:0/3240343646 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6244068a40 con 0x7f624415def0 2024-04-25T08:57:45.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.859+0000 7f6240ff9640 1 -- 172.21.15.43:0/3240343646 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v49) v1 ==== 169+0+0 (secure 0 0 0) 0x7f623c079d60 con 0x7f624415def0 2024-04-25T08:57:45.848 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' removed 2024-04-25T08:57:45.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.865+0000 7f624a831640 1 -- 172.21.15.43:0/3240343646 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f623006b4d0 msgr2=0x7f623006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:45.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.865+0000 7f624a831640 1 --2- 172.21.15.43:0/3240343646 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f623006b4d0 0x7f623006d990 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6244160c00 tx=0x7f6234008040 comp rx=0 tx=0).stop 2024-04-25T08:57:45.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.865+0000 7f624a831640 1 -- 172.21.15.43:0/3240343646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f624415def0 msgr2=0x7f62441f3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:45.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.865+0000 7f624a831640 1 --2- 172.21.15.43:0/3240343646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f624415def0 0x7f62441f3260 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f623c009d90 tx=0x7f623c004a40 comp rx=0 tx=0).stop 2024-04-25T08:57:45.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.865+0000 7f624a831640 1 -- 172.21.15.43:0/3240343646 shutdown_connections 2024-04-25T08:57:45.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.865+0000 7f624a831640 1 --2- 172.21.15.43:0/3240343646 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f624415def0 0x7f62441f3260 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:45.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.865+0000 7f624a831640 1 --2- 172.21.15.43:0/3240343646 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f623006b4d0 0x7f623006d990 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:45.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.865+0000 7f624a831640 1 --2- 172.21.15.43:0/3240343646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62440d0850 0x7f62440cb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:45.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.865+0000 7f624a831640 1 --2- 172.21.15.43:0/3240343646 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62440d03f0 0x7f6244155060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:45.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.865+0000 7f624a831640 1 -- 172.21.15.43:0/3240343646 >> 172.21.15.43:0/3240343646 conn(0x7f624409cd60 msgr2=0x7f62440ce4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:45.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.866+0000 7f624a831640 1 -- 172.21.15.43:0/3240343646 shutdown_connections 2024-04-25T08:57:45.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.866+0000 7f624a831640 1 -- 172.21.15.43:0/3240343646 wait complete. 2024-04-25T08:57:45.875 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T08:57:45.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.993+0000 7fa71ed90640 1 Processor -- start 2024-04-25T08:57:45.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.993+0000 7fa71ed90640 1 -- start start 2024-04-25T08:57:45.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.994+0000 7fa71ed90640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fa7180d19f0 0x7fa7180d1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:45.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.994+0000 7fa71ed90640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa7180d2370 0x7fa7180c7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:45.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.994+0000 7fa71ed90640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7180cf030 con 0x7fa718161310 2024-04-25T08:57:45.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.994+0000 7fa71ed90640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7180d4870 con 0x7fa7180d2370 2024-04-25T08:57:45.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.994+0000 7fa71ed90640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7180cf310 con 0x7fa7180d19f0 2024-04-25T08:57:45.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.994+0000 7fa71d32d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa7180d2370 0x7fa7180c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:45.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.994+0000 7fa71d32d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa7180d2370 0x7fa7180c7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45812/0 (socket says 172.21.15.43:45812) 2024-04-25T08:57:45.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.994+0000 7fa71d32d640 1 -- 172.21.15.43:0/2560171081 learned_addr learned my addr 172.21.15.43:0/2560171081 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:45.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.994+0000 7fa717fff640 1 --2- 172.21.15.43:0/2560171081 >> v2:172.21.15.193:3300/0 conn(0x7fa7180d19f0 0x7fa7180d1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:45.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.995+0000 7fa71d32d640 1 -- 172.21.15.43:0/2560171081 >> v2:172.21.15.193:3300/0 conn(0x7fa7180d19f0 msgr2=0x7fa7180d1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:45.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.995+0000 7fa71d32d640 1 --2- 172.21.15.43:0/2560171081 >> v2:172.21.15.193:3300/0 conn(0x7fa7180d19f0 0x7fa7180d1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:45.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.995+0000 7fa71d32d640 1 -- 172.21.15.43:0/2560171081 >> v1:172.21.15.193:6789/0 conn(0x7fa718161310 legacy=0x7fa7180d4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:45.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.995+0000 7fa71d32d640 1 -- 172.21.15.43:0/2560171081 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7180cf5f0 con 0x7fa7180d2370 2024-04-25T08:57:45.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.995+0000 7fa71d32d640 1 --2- 172.21.15.43:0/2560171081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7180d2370 0x7fa7180c7860 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fa71808d2c0 tx=0x7fa70802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c413181e768dd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:45.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.995+0000 7fa7177fe640 1 -- 172.21.15.43:0/2560171081 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa708037070 con 0x7fa7180d2370 2024-04-25T08:57:45.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.996+0000 7fa7177fe640 1 -- 172.21.15.43:0/2560171081 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa708032440 con 0x7fa7180d2370 2024-04-25T08:57:45.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.996+0000 7fa7177fe640 1 -- 172.21.15.43:0/2560171081 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa708032810 con 0x7fa7180d2370 2024-04-25T08:57:45.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.996+0000 7fa71ed90640 1 -- 172.21.15.43:0/2560171081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7180d2370 msgr2=0x7fa7180c7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:45.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.996+0000 7fa71ed90640 1 --2- 172.21.15.43:0/2560171081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7180d2370 0x7fa7180c7860 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fa71808d2c0 tx=0x7fa70802a270 comp rx=0 tx=0).stop 2024-04-25T08:57:45.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.996+0000 7fa71ed90640 1 -- 172.21.15.43:0/2560171081 shutdown_connections 2024-04-25T08:57:45.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.996+0000 7fa71ed90640 1 --2- 172.21.15.43:0/2560171081 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7180d2370 0x7fa7180c7860 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:45.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.996+0000 7fa71ed90640 1 --2- 172.21.15.43:0/2560171081 >> v2:172.21.15.193:3300/0 conn(0x7fa7180d19f0 0x7fa7180d1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:45.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.996+0000 7fa71ed90640 1 -- 172.21.15.43:0/2560171081 >> 172.21.15.43:0/2560171081 conn(0x7fa71809cd20 msgr2=0x7fa7180d3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:45.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.996+0000 7fa71ed90640 1 -- 172.21.15.43:0/2560171081 shutdown_connections 2024-04-25T08:57:45.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.996+0000 7fa71ed90640 1 -- 172.21.15.43:0/2560171081 wait complete. 2024-04-25T08:57:45.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.997+0000 7fa71ed90640 1 Processor -- start 2024-04-25T08:57:45.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.997+0000 7fa71ed90640 1 -- start start 2024-04-25T08:57:45.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.998+0000 7fa71ed90640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7180d19f0 0x7fa7180b7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:45.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.998+0000 7fa71ed90640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7180d2370 0x7fa718157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:45.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.998+0000 7fa71ed90640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa718161310 0x7fa7180b7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:45.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.998+0000 7fa71ed90640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7180c7da0 con 0x7fa718161310 2024-04-25T08:57:45.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.998+0000 7fa71ed90640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7180d27e0 con 0x7fa7180d19f0 2024-04-25T08:57:45.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.998+0000 7fa71ed90640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7180c7f20 con 0x7fa7180d2370 2024-04-25T08:57:45.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.998+0000 7fa71d32d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa718161310 0x7fa7180b7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:45.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.998+0000 7fa71d32d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa718161310 0x7fa7180b7ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45824/0 (socket says 172.21.15.43:45824) 2024-04-25T08:57:45.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.998+0000 7fa71d32d640 1 -- 172.21.15.43:0/3244826793 learned_addr learned my addr 172.21.15.43:0/3244826793 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:45.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.999+0000 7fa71d32d640 1 -- 172.21.15.43:0/3244826793 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7180d2370 msgr2=0x7fa718157f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:45.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.999+0000 7fa71cb2c640 1 --2- 172.21.15.43:0/3244826793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7180d19f0 0x7fa7180b7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:45.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.999+0000 7fa71d32d640 1 --2- 172.21.15.43:0/3244826793 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7180d2370 0x7fa718157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:45.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.999+0000 7fa71d32d640 1 -- 172.21.15.43:0/3244826793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7180d19f0 msgr2=0x7fa7180b7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:45.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.999+0000 7fa71d32d640 1 --2- 172.21.15.43:0/3244826793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7180d19f0 0x7fa7180b7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:45.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.999+0000 7fa71d32d640 1 -- 172.21.15.43:0/3244826793 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7180b67a0 con 0x7fa718161310 2024-04-25T08:57:45.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.999+0000 7fa71d32d640 1 --2- 172.21.15.43:0/3244826793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa718161310 0x7fa7180b7ee0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fa70802a2b0 tx=0x7fa7080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:45.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.999+0000 7fa7157fa640 1 -- 172.21.15.43:0/3244826793 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa708037040 con 0x7fa718161310 2024-04-25T08:57:45.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.999+0000 7fa7157fa640 1 -- 172.21.15.43:0/3244826793 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa708042070 con 0x7fa718161310 2024-04-25T08:57:45.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.999+0000 7fa71ed90640 1 -- 172.21.15.43:0/3244826793 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7180b6a80 con 0x7fa718161310 2024-04-25T08:57:45.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.999+0000 7fa7157fa640 1 -- 172.21.15.43:0/3244826793 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa708030920 con 0x7fa718161310 2024-04-25T08:57:45.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:45.999+0000 7fa71ed90640 1 -- 172.21.15.43:0/3244826793 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7180b7aa0 con 0x7fa718161310 2024-04-25T08:57:45.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.000+0000 7fa71ed90640 1 -- 172.21.15.43:0/3244826793 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa718074f60 con 0x7fa718161310 2024-04-25T08:57:45.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.000+0000 7fa7157fa640 1 -- 172.21.15.43:0/3244826793 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa708030ac0 con 0x7fa718161310 2024-04-25T08:57:45.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.001+0000 7fa7157fa640 1 --2- 172.21.15.43:0/3244826793 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa6f806b4d0 0x7fa6f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:45.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.001+0000 7fa7157fa640 1 -- 172.21.15.43:0/3244826793 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa7080aa790 con 0x7fa718161310 2024-04-25T08:57:45.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.001+0000 7fa71cb2c640 1 --2- 172.21.15.43:0/3244826793 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa6f806b4d0 0x7fa6f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:45.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.002+0000 7fa71cb2c640 1 --2- 172.21.15.43:0/3244826793 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa6f806b4d0 0x7fa6f806d990 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa710007fd0 tx=0x7fa710009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:45.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.005+0000 7fa7157fa640 1 -- 172.21.15.43:0/3244826793 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa708079350 con 0x7fa718161310 2024-04-25T08:57:46.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.186+0000 7fa71ed90640 1 -- 172.21.15.43:0/3244826793 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fa71806b040 con 0x7fa718161310 2024-04-25T08:57:46.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.189+0000 7fa7157fa640 1 -- 172.21.15.43:0/3244826793 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa70807d3f0 con 0x7fa718161310 2024-04-25T08:57:46.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.191+0000 7fa71ed90640 1 -- 172.21.15.43:0/3244826793 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa6f806b4d0 msgr2=0x7fa6f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:46.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.191+0000 7fa71ed90640 1 --2- 172.21.15.43:0/3244826793 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa6f806b4d0 0x7fa6f806d990 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa710007fd0 tx=0x7fa710009040 comp rx=0 tx=0).stop 2024-04-25T08:57:46.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.191+0000 7fa71ed90640 1 -- 172.21.15.43:0/3244826793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa718161310 msgr2=0x7fa7180b7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:46.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.191+0000 7fa71ed90640 1 --2- 172.21.15.43:0/3244826793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa718161310 0x7fa7180b7ee0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fa70802a2b0 tx=0x7fa7080046c0 comp rx=0 tx=0).stop 2024-04-25T08:57:46.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.191+0000 7fa71ed90640 1 -- 172.21.15.43:0/3244826793 shutdown_connections 2024-04-25T08:57:46.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.191+0000 7fa71ed90640 1 --2- 172.21.15.43:0/3244826793 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa718161310 0x7fa7180b7ee0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:46.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.191+0000 7fa71ed90640 1 --2- 172.21.15.43:0/3244826793 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7180d2370 0x7fa718157f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:46.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.191+0000 7fa71ed90640 1 --2- 172.21.15.43:0/3244826793 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa6f806b4d0 0x7fa6f806d990 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:46.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.191+0000 7fa71ed90640 1 --2- 172.21.15.43:0/3244826793 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7180d19f0 0x7fa7180b7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:46.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.191+0000 7fa71ed90640 1 -- 172.21.15.43:0/3244826793 >> 172.21.15.43:0/3244826793 conn(0x7fa71809cd20 msgr2=0x7fa718160520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:46.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.191+0000 7fa71ed90640 1 -- 172.21.15.43:0/3244826793 shutdown_connections 2024-04-25T08:57:46.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.191+0000 7fa71ed90640 1 -- 172.21.15.43:0/3244826793 wait complete. 2024-04-25T08:57:46.192 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks' 2024-04-25T08:57:46.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.308+0000 7f200f61f640 1 Processor -- start 2024-04-25T08:57:46.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.308+0000 7f200f61f640 1 -- start start 2024-04-25T08:57:46.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.309+0000 7f200f61f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f200815dce0 0x7f200815ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:46.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.309+0000 7f200f61f640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f20080ce550 0x7f20080d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:46.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.309+0000 7f200f61f640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20080cee80 con 0x7f20080ce990 2024-04-25T08:57:46.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.309+0000 7f200f61f640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20080d2920 con 0x7f20080ce550 2024-04-25T08:57:46.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.309+0000 7f200f61f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20080cf160 con 0x7f200815dce0 2024-04-25T08:57:46.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.309+0000 7f200cbba640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f20080ce550 0x7f20080d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:46.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.309+0000 7f200d3bb640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f200815dce0 0x7f200815ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:46.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.309+0000 7f200dbbc640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f20080ce990 0x7f2008160980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:39530/0 (socket says 172.21.15.43:39530) 2024-04-25T08:57:46.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.309+0000 7f200dbbc640 1 -- 172.21.15.43:0/1673119141 learned_addr learned my addr 172.21.15.43:0/1673119141 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:46.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.310+0000 7f1ffffff640 1 -- 172.21.15.43:0/1673119141 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2605186091 0 0) 0x7f20080cee80 con 0x7f20080ce990 2024-04-25T08:57:46.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.310+0000 7f1ffffff640 1 -- 172.21.15.43:0/1673119141 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ff4003520 con 0x7f20080ce990 2024-04-25T08:57:46.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.310+0000 7f200d3bb640 1 -- 172.21.15.43:0/1673119141 >> v2:172.21.15.193:3300/0 conn(0x7f20080ce550 msgr2=0x7f20080d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:46.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.310+0000 7f200d3bb640 1 --2- 172.21.15.43:0/1673119141 >> v2:172.21.15.193:3300/0 conn(0x7f20080ce550 0x7f20080d03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:46.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.310+0000 7f200d3bb640 1 -- 172.21.15.43:0/1673119141 >> v1:172.21.15.193:6789/0 conn(0x7f20080ce990 legacy=0x7f2008160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:46.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.310+0000 7f200d3bb640 1 -- 172.21.15.43:0/1673119141 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20080cf440 con 0x7f200815dce0 2024-04-25T08:57:46.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.310+0000 7f200cbba640 1 --2- 172.21.15.43:0/1673119141 >> v2:172.21.15.193:3300/0 conn(0x7f20080ce550 0x7f20080d03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:57:46.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.310+0000 7f200d3bb640 1 --2- 172.21.15.43:0/1673119141 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f200815dce0 0x7f200815ef50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f20040097a0 tx=0x7f200402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=424b2dae221c88e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:46.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.311+0000 7f1ffffff640 1 -- 172.21.15.43:0/1673119141 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2004037070 con 0x7f200815dce0 2024-04-25T08:57:46.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.311+0000 7f1ffffff640 1 -- 172.21.15.43:0/1673119141 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f200402ae50 con 0x7f200815dce0 2024-04-25T08:57:46.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.311+0000 7f1ffffff640 1 -- 172.21.15.43:0/1673119141 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2004032640 con 0x7f200815dce0 2024-04-25T08:57:46.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.311+0000 7f200f61f640 1 -- 172.21.15.43:0/1673119141 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f200815dce0 msgr2=0x7f200815ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:46.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.311+0000 7f200f61f640 1 --2- 172.21.15.43:0/1673119141 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f200815dce0 0x7f200815ef50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f20040097a0 tx=0x7f200402a100 comp rx=0 tx=0).stop 2024-04-25T08:57:46.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.311+0000 7f200f61f640 1 -- 172.21.15.43:0/1673119141 shutdown_connections 2024-04-25T08:57:46.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.311+0000 7f200f61f640 1 --2- 172.21.15.43:0/1673119141 >> v2:172.21.15.193:3300/0 conn(0x7f20080ce550 0x7f20080d03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:46.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.311+0000 7f200f61f640 1 --2- 172.21.15.43:0/1673119141 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f200815dce0 0x7f200815ef50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:46.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.311+0000 7f200f61f640 1 -- 172.21.15.43:0/1673119141 >> 172.21.15.43:0/1673119141 conn(0x7f200809cd00 msgr2=0x7f20080d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:46.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.311+0000 7f200f61f640 1 -- 172.21.15.43:0/1673119141 shutdown_connections 2024-04-25T08:57:46.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.311+0000 7f200f61f640 1 -- 172.21.15.43:0/1673119141 wait complete. 2024-04-25T08:57:46.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.312+0000 7f200f61f640 1 Processor -- start 2024-04-25T08:57:46.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.312+0000 7f200f61f640 1 -- start start 2024-04-25T08:57:46.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.313+0000 7f200f61f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20080ce550 0x7f20081f3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:46.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.313+0000 7f200f61f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20080ce990 0x7f20081f7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:46.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.313+0000 7f200f61f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f200815dce0 0x7f20081f6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:46.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.313+0000 7f200f61f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200815e0c0 con 0x7f20080ce990 2024-04-25T08:57:46.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.313+0000 7f200f61f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200815f6b0 con 0x7f20080ce550 2024-04-25T08:57:46.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.313+0000 7f200f61f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20080d0920 con 0x7f200815dce0 2024-04-25T08:57:46.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.313+0000 7f200cbba640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20080ce990 0x7f20081f7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:46.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.313+0000 7f200cbba640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20080ce990 0x7f20081f7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45836/0 (socket says 172.21.15.43:45836) 2024-04-25T08:57:46.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.313+0000 7f200cbba640 1 -- 172.21.15.43:0/3039523026 learned_addr learned my addr 172.21.15.43:0/3039523026 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:46.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.313+0000 7f200d3bb640 1 --2- 172.21.15.43:0/3039523026 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20080ce550 0x7f20081f3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:46.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.313+0000 7f200dbbc640 1 --2- 172.21.15.43:0/3039523026 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f200815dce0 0x7f20081f6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:46.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.314+0000 7f200cbba640 1 -- 172.21.15.43:0/3039523026 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f200815dce0 msgr2=0x7f20081f6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:46.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.314+0000 7f200cbba640 1 --2- 172.21.15.43:0/3039523026 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f200815dce0 0x7f20081f6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:46.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.314+0000 7f200cbba640 1 -- 172.21.15.43:0/3039523026 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20080ce550 msgr2=0x7f20081f3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:46.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.314+0000 7f200cbba640 1 --2- 172.21.15.43:0/3039523026 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20080ce550 0x7f20081f3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:46.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.314+0000 7f200cbba640 1 -- 172.21.15.43:0/3039523026 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20080cae20 con 0x7f20080ce990 2024-04-25T08:57:46.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.314+0000 7f200cbba640 1 --2- 172.21.15.43:0/3039523026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20080ce990 0x7f20081f7e50 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f1ff800ebe0 tx=0x7f1ff800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:46.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.314+0000 7f1ffdffb640 1 -- 172.21.15.43:0/3039523026 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff8017070 con 0x7f20080ce990 2024-04-25T08:57:46.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.314+0000 7f1ffdffb640 1 -- 172.21.15.43:0/3039523026 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff800cc30 con 0x7f20080ce990 2024-04-25T08:57:46.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.314+0000 7f200f61f640 1 -- 172.21.15.43:0/3039523026 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20080cb100 con 0x7f20080ce990 2024-04-25T08:57:46.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.314+0000 7f1ffdffb640 1 -- 172.21.15.43:0/3039523026 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff8013470 con 0x7f20080ce990 2024-04-25T08:57:46.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.314+0000 7f200f61f640 1 -- 172.21.15.43:0/3039523026 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20080b7650 con 0x7f20080ce990 2024-04-25T08:57:46.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.315+0000 7f200f61f640 1 -- 172.21.15.43:0/3039523026 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20080680f0 con 0x7f20080ce990 2024-04-25T08:57:46.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.316+0000 7f1ffdffb640 1 -- 172.21.15.43:0/3039523026 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1ff801a070 con 0x7f20080ce990 2024-04-25T08:57:46.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.316+0000 7f1ffdffb640 1 --2- 172.21.15.43:0/3039523026 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1fec06b4d0 0x7f1fec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:46.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.317+0000 7f1ffdffb640 1 -- 172.21.15.43:0/3039523026 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1ff808a200 con 0x7f20080ce990 2024-04-25T08:57:46.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.317+0000 7f200d3bb640 1 --2- 172.21.15.43:0/3039523026 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1fec06b4d0 0x7f1fec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:46.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.317+0000 7f200d3bb640 1 --2- 172.21.15.43:0/3039523026 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1fec06b4d0 0x7f1fec06d990 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2004034000 tx=0x7f200402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:46.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.321+0000 7f1ffdffb640 1 -- 172.21.15.43:0/3039523026 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1ff8058c90 con 0x7f20080ce990 2024-04-25T08:57:46.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.569+0000 7f200f61f640 1 -- 172.21.15.43:0/3039523026 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]} v 0) v1 -- 0x7f200806c650 con 0x7f20080ce990 2024-04-25T08:57:46.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.860+0000 7f1ffdffb640 1 -- 172.21.15.43:0/3039523026 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]}]=0 v185) v1 ==== 143+0+0 (secure 0 0 0) 0x7f1ff805cd30 con 0x7f20080ce990 2024-04-25T08:57:46.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.862+0000 7f200f61f640 1 -- 172.21.15.43:0/3039523026 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1fec06b4d0 msgr2=0x7f1fec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:46.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.862+0000 7f200f61f640 1 --2- 172.21.15.43:0/3039523026 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1fec06b4d0 0x7f1fec06d990 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2004034000 tx=0x7f200402e000 comp rx=0 tx=0).stop 2024-04-25T08:57:46.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.862+0000 7f200f61f640 1 -- 172.21.15.43:0/3039523026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20080ce990 msgr2=0x7f20081f7e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:46.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.862+0000 7f200f61f640 1 --2- 172.21.15.43:0/3039523026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20080ce990 0x7f20081f7e50 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f1ff800ebe0 tx=0x7f1ff800c520 comp rx=0 tx=0).stop 2024-04-25T08:57:46.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.862+0000 7f200f61f640 1 -- 172.21.15.43:0/3039523026 shutdown_connections 2024-04-25T08:57:46.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.862+0000 7f200f61f640 1 --2- 172.21.15.43:0/3039523026 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f200815dce0 0x7f20081f6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:46.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.862+0000 7f200f61f640 1 --2- 172.21.15.43:0/3039523026 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20080ce990 0x7f20081f7e50 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:46.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.862+0000 7f200f61f640 1 --2- 172.21.15.43:0/3039523026 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1fec06b4d0 0x7f1fec06d990 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:46.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.862+0000 7f200f61f640 1 --2- 172.21.15.43:0/3039523026 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20080ce550 0x7f20081f3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:46.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.862+0000 7f200f61f640 1 -- 172.21.15.43:0/3039523026 >> 172.21.15.43:0/3039523026 conn(0x7f200809cd00 msgr2=0x7f20080d1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:46.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.862+0000 7f200f61f640 1 -- 172.21.15.43:0/3039523026 shutdown_connections 2024-04-25T08:57:46.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.862+0000 7f200f61f640 1 -- 172.21.15.43:0/3039523026 wait complete. 2024-04-25T08:57:46.863 INFO:tasks.cephfs_test_runner:test_scrub_checks (tasks.cephfs.test_scrub_checks.TestScrubChecks) ... ok 2024-04-25T08:57:46.863 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_repair (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T08:57:46.864 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:57:46.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.983+0000 7fae332e4640 1 Processor -- start 2024-04-25T08:57:46.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.983+0000 7fae332e4640 1 -- start start 2024-04-25T08:57:46.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.984+0000 7fae332e4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fae2c0c8d00 0x7fae2c162100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:46.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.984+0000 7fae332e4640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae2c0d24a0 con 0x7fae2c15df50 2024-04-25T08:57:46.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.984+0000 7fae332e4640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae2c0d2740 con 0x7fae2c0ce660 2024-04-25T08:57:46.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.984+0000 7fae332e4640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae2c15e330 con 0x7fae2c0c8d00 2024-04-25T08:57:46.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.984+0000 7fae3087f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fae2c0c8d00 0x7fae2c162100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:46.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.984+0000 7fae31080640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fae2c15df50 0x7fae2c1619f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:57644/0 (socket says 172.21.15.43:57644) 2024-04-25T08:57:46.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.984+0000 7fae31080640 1 -- 172.21.15.43:0/2868269085 learned_addr learned my addr 172.21.15.43:0/2868269085 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:46.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.984+0000 7fae3087f640 1 -- 172.21.15.43:0/2868269085 >> v1:172.21.15.193:6789/0 conn(0x7fae2c0ce660 legacy=0x7fae2c0cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:46.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.984+0000 7fae3087f640 1 -- 172.21.15.43:0/2868269085 >> v1:172.21.15.43:6789/0 conn(0x7fae2c15df50 legacy=0x7fae2c1619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:46.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.985+0000 7fae2bfff640 1 -- 172.21.15.43:0/2868269085 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 272748604 0 0) 0x7fae2c0d24a0 con 0x7fae2c15df50 2024-04-25T08:57:46.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.985+0000 7fae3087f640 1 -- 172.21.15.43:0/2868269085 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae2c0d2a20 con 0x7fae2c0c8d00 2024-04-25T08:57:46.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.985+0000 7fae3087f640 1 --2- 172.21.15.43:0/2868269085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae2c0c8d00 0x7fae2c162100 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fae200097a0 tx=0x7fae2002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ffcdcd6a4308e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:46.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.985+0000 7fae2bfff640 1 -- 172.21.15.43:0/2868269085 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae20037070 con 0x7fae2c0c8d00 2024-04-25T08:57:46.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.985+0000 7fae2bfff640 1 -- 172.21.15.43:0/2868269085 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae20032440 con 0x7fae2c0c8d00 2024-04-25T08:57:46.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.985+0000 7fae2bfff640 1 -- 172.21.15.43:0/2868269085 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae20032810 con 0x7fae2c0c8d00 2024-04-25T08:57:46.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.986+0000 7fae332e4640 1 -- 172.21.15.43:0/2868269085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae2c0c8d00 msgr2=0x7fae2c162100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:46.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.986+0000 7fae332e4640 1 --2- 172.21.15.43:0/2868269085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae2c0c8d00 0x7fae2c162100 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fae200097a0 tx=0x7fae2002a1e0 comp rx=0 tx=0).stop 2024-04-25T08:57:46.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.986+0000 7fae332e4640 1 -- 172.21.15.43:0/2868269085 shutdown_connections 2024-04-25T08:57:46.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.986+0000 7fae332e4640 1 --2- 172.21.15.43:0/2868269085 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae2c0c8d00 0x7fae2c162100 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:46.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.986+0000 7fae332e4640 1 -- 172.21.15.43:0/2868269085 >> 172.21.15.43:0/2868269085 conn(0x7fae2c0d3a30 msgr2=0x7fae2c0d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:46.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.986+0000 7fae332e4640 1 -- 172.21.15.43:0/2868269085 shutdown_connections 2024-04-25T08:57:46.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.986+0000 7fae332e4640 1 -- 172.21.15.43:0/2868269085 wait complete. 2024-04-25T08:57:46.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.987+0000 7fae332e4640 1 Processor -- start 2024-04-25T08:57:46.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.987+0000 7fae332e4640 1 -- start start 2024-04-25T08:57:46.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.988+0000 7fae332e4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae2c0c8d00 0x7fae2c0c64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:46.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.988+0000 7fae332e4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae2c0ce660 0x7fae2c0cce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:46.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.988+0000 7fae332e4640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fae2c15df50 0x7fae2c0cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:46.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.988+0000 7fae332e4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2c0d2190 con 0x7fae2c0c8d00 2024-04-25T08:57:46.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.988+0000 7fae332e4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae2c0c77d0 con 0x7fae2c0ce660 2024-04-25T08:57:46.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.988+0000 7fae332e4640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae2c162640 con 0x7fae2c15df50 2024-04-25T08:57:46.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.988+0000 7fae31080640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae2c0c8d00 0x7fae2c0c64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:46.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.988+0000 7fae31080640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae2c0c8d00 0x7fae2c0c64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45844/0 (socket says 172.21.15.43:45844) 2024-04-25T08:57:46.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.988+0000 7fae31080640 1 -- 172.21.15.43:0/3917527492 learned_addr learned my addr 172.21.15.43:0/3917527492 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:46.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.988+0000 7fae3087f640 1 --2- 172.21.15.43:0/3917527492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae2c0ce660 0x7fae2c0cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:46.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.988+0000 7fae31881640 1 --2- 172.21.15.43:0/3917527492 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fae2c15df50 0x7fae2c0cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:46.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.988+0000 7fae31080640 1 -- 172.21.15.43:0/3917527492 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fae2c15df50 msgr2=0x7fae2c0cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:46.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.988+0000 7fae31080640 1 --2- 172.21.15.43:0/3917527492 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fae2c15df50 0x7fae2c0cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:46.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.988+0000 7fae31080640 1 -- 172.21.15.43:0/3917527492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae2c0ce660 msgr2=0x7fae2c0cce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:46.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.988+0000 7fae31080640 1 --2- 172.21.15.43:0/3917527492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae2c0ce660 0x7fae2c0cce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:46.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.988+0000 7fae31080640 1 -- 172.21.15.43:0/3917527492 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae2c15e4b0 con 0x7fae2c0c8d00 2024-04-25T08:57:46.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.988+0000 7fae31881640 1 --2- 172.21.15.43:0/3917527492 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fae2c15df50 0x7fae2c0cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:46.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.989+0000 7fae31080640 1 --2- 172.21.15.43:0/3917527492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae2c0c8d00 0x7fae2c0c64f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fae2400c430 tx=0x7fae2400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:46.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.989+0000 7fae29ffb640 1 -- 172.21.15.43:0/3917527492 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae24017070 con 0x7fae2c0c8d00 2024-04-25T08:57:46.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.989+0000 7fae29ffb640 1 -- 172.21.15.43:0/3917527492 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae24005ce0 con 0x7fae2c0c8d00 2024-04-25T08:57:46.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.989+0000 7fae29ffb640 1 -- 172.21.15.43:0/3917527492 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae24007310 con 0x7fae2c0c8d00 2024-04-25T08:57:46.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.989+0000 7fae332e4640 1 -- 172.21.15.43:0/3917527492 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae2c0a1080 con 0x7fae2c0c8d00 2024-04-25T08:57:46.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.989+0000 7fae332e4640 1 -- 172.21.15.43:0/3917527492 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae2c0a1640 con 0x7fae2c0c8d00 2024-04-25T08:57:46.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.990+0000 7fae332e4640 1 -- 172.21.15.43:0/3917527492 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae2c0680f0 con 0x7fae2c0c8d00 2024-04-25T08:57:46.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.990+0000 7fae29ffb640 1 -- 172.21.15.43:0/3917527492 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae2401a070 con 0x7fae2c0c8d00 2024-04-25T08:57:46.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.991+0000 7fae29ffb640 1 --2- 172.21.15.43:0/3917527492 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fae1806b500 0x7fae1806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:46.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.991+0000 7fae29ffb640 1 -- 172.21.15.43:0/3917527492 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fae240892f0 con 0x7fae2c0c8d00 2024-04-25T08:57:46.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.991+0000 7fae3087f640 1 --2- 172.21.15.43:0/3917527492 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fae1806b500 0x7fae1806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:46.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.991+0000 7fae3087f640 1 --2- 172.21.15.43:0/3917527492 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fae1806b500 0x7fae1806d9c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fae20034000 tx=0x7fae2002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:46.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:46.995+0000 7fae29ffb640 1 -- 172.21.15.43:0/3917527492 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fae24058470 con 0x7fae2c0c8d00 2024-04-25T08:57:47.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.182+0000 7fae332e4640 1 -- 172.21.15.43:0/3917527492 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fae2c068f20 con 0x7fae2c0c8d00 2024-04-25T08:57:47.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.183+0000 7fae29ffb640 1 -- 172.21.15.43:0/3917527492 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+12986 (secure 0 0 0) 0x7fae2c068f20 con 0x7fae2c0c8d00 2024-04-25T08:57:47.172 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:57:47.173 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":49,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:57:45.855084+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.193:6837/4164634380":"2024-04-26T08:57:42.848098+0000","172.21.15.193:6836/4164634380":"2024-04-26T08:57:42.848098+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:57:47.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.185+0000 7fae332e4640 1 -- 172.21.15.43:0/3917527492 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fae1806b500 msgr2=0x7fae1806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:47.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.185+0000 7fae332e4640 1 --2- 172.21.15.43:0/3917527492 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fae1806b500 0x7fae1806d9c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fae20034000 tx=0x7fae2002e000 comp rx=0 tx=0).stop 2024-04-25T08:57:47.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.185+0000 7fae332e4640 1 -- 172.21.15.43:0/3917527492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae2c0c8d00 msgr2=0x7fae2c0c64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:47.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.185+0000 7fae332e4640 1 --2- 172.21.15.43:0/3917527492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae2c0c8d00 0x7fae2c0c64f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fae2400c430 tx=0x7fae2400c8f0 comp rx=0 tx=0).stop 2024-04-25T08:57:47.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.185+0000 7fae332e4640 1 -- 172.21.15.43:0/3917527492 shutdown_connections 2024-04-25T08:57:47.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.185+0000 7fae332e4640 1 --2- 172.21.15.43:0/3917527492 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fae2c15df50 0x7fae2c0cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:47.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.185+0000 7fae332e4640 1 --2- 172.21.15.43:0/3917527492 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fae1806b500 0x7fae1806d9c0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:47.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.185+0000 7fae332e4640 1 --2- 172.21.15.43:0/3917527492 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae2c0ce660 0x7fae2c0cce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:47.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.185+0000 7fae332e4640 1 --2- 172.21.15.43:0/3917527492 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae2c0c8d00 0x7fae2c0c64f0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:47.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.185+0000 7fae332e4640 1 -- 172.21.15.43:0/3917527492 >> 172.21.15.43:0/3917527492 conn(0x7fae2c0d3a30 msgr2=0x7fae2c06b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:47.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.185+0000 7fae332e4640 1 -- 172.21.15.43:0/3917527492 shutdown_connections 2024-04-25T08:57:47.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.185+0000 7fae332e4640 1 -- 172.21.15.43:0/3917527492 wait complete. 2024-04-25T08:57:47.185 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:57:47.186 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:57:47.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.300+0000 7fb4a6a15640 1 Processor -- start 2024-04-25T08:57:47.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.300+0000 7fb4a6a15640 1 -- start start 2024-04-25T08:57:47.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.301+0000 7fb4a6a15640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb4a006ed70 0x7fb4a006f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:47.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.301+0000 7fb4a6a15640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb4a00cbda0 0x7fb4a00d0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:47.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.301+0000 7fb4a6a15640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4a015bf00 con 0x7fb4a00d0ec0 2024-04-25T08:57:47.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.301+0000 7fb4a6a15640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4a00cfdb0 con 0x7fb4a00cbda0 2024-04-25T08:57:47.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.301+0000 7fb4a6a15640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a00d1350 con 0x7fb4a006ed70 2024-04-25T08:57:47.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.301+0000 7fb49f7fe640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb4a00cbda0 0x7fb4a00d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:47.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.301+0000 7fb49ffff640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb4a006ed70 0x7fb4a006f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:47.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.301+0000 7fb4a4fb2640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7fb4a00d0ec0 0x7fb4a00b33c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:39544/0 (socket says 172.21.15.43:39544) 2024-04-25T08:57:47.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.301+0000 7fb4a4fb2640 1 -- 172.21.15.43:0/2754381635 learned_addr learned my addr 172.21.15.43:0/2754381635 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:47.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.302+0000 7fb49f7fe640 1 -- 172.21.15.43:0/2754381635 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb4a006ed70 msgr2=0x7fb4a006f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:47.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.302+0000 7fb49f7fe640 1 --2- 172.21.15.43:0/2754381635 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb4a006ed70 0x7fb4a006f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:47.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.302+0000 7fb49f7fe640 1 -- 172.21.15.43:0/2754381635 >> v1:172.21.15.193:6789/0 conn(0x7fb4a00d0ec0 legacy=0x7fb4a00b33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:47.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.302+0000 7fb49f7fe640 1 -- 172.21.15.43:0/2754381635 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a006f690 con 0x7fb4a00cbda0 2024-04-25T08:57:47.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.302+0000 7fb49effd640 1 -- 172.21.15.43:0/2754381635 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3042329366 0 0) 0x7fb4a015bf00 con 0x7fb4a00d0ec0 2024-04-25T08:57:47.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.302+0000 7fb49f7fe640 1 --2- 172.21.15.43:0/2754381635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4a00cbda0 0x7fb4a00d0980 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb4900097a0 tx=0x7fb49002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd38b07134b0eb30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:47.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.303+0000 7fb49effd640 1 -- 172.21.15.43:0/2754381635 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb490037070 con 0x7fb4a00cbda0 2024-04-25T08:57:47.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.303+0000 7fb49effd640 1 -- 172.21.15.43:0/2754381635 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb490032440 con 0x7fb4a00cbda0 2024-04-25T08:57:47.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.303+0000 7fb49effd640 1 -- 172.21.15.43:0/2754381635 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb490032810 con 0x7fb4a00cbda0 2024-04-25T08:57:47.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.303+0000 7fb4a6a15640 1 -- 172.21.15.43:0/2754381635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4a00cbda0 msgr2=0x7fb4a00d0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:47.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.303+0000 7fb4a6a15640 1 --2- 172.21.15.43:0/2754381635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4a00cbda0 0x7fb4a00d0980 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb4900097a0 tx=0x7fb49002a180 comp rx=0 tx=0).stop 2024-04-25T08:57:47.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.303+0000 7fb4a6a15640 1 -- 172.21.15.43:0/2754381635 shutdown_connections 2024-04-25T08:57:47.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.303+0000 7fb4a6a15640 1 --2- 172.21.15.43:0/2754381635 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4a00cbda0 0x7fb4a00d0980 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:47.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.303+0000 7fb4a6a15640 1 --2- 172.21.15.43:0/2754381635 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb4a006ed70 0x7fb4a006f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:47.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.303+0000 7fb4a6a15640 1 -- 172.21.15.43:0/2754381635 >> 172.21.15.43:0/2754381635 conn(0x7fb4a00d17e0 msgr2=0x7fb4a00d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:47.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.303+0000 7fb4a6a15640 1 -- 172.21.15.43:0/2754381635 shutdown_connections 2024-04-25T08:57:47.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.303+0000 7fb4a6a15640 1 -- 172.21.15.43:0/2754381635 wait complete. 2024-04-25T08:57:47.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.304+0000 7fb4a6a15640 1 Processor -- start 2024-04-25T08:57:47.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.305+0000 7fb4a6a15640 1 -- start start 2024-04-25T08:57:47.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.305+0000 7fb4a6a15640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4a006ed70 0x7fb4a014e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:47.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.305+0000 7fb4a6a15640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4a00cbda0 0x7fb4a0151520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:47.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.305+0000 7fb4a6a15640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb4a00d0ec0 0x7fb4a01f6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:47.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.305+0000 7fb4a6a15640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a00c6d00 con 0x7fb4a006ed70 2024-04-25T08:57:47.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.305+0000 7fb4a6a15640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a00c6a20 con 0x7fb4a00cbda0 2024-04-25T08:57:47.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.305+0000 7fb4a6a15640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a00c6fe0 con 0x7fb4a00d0ec0 2024-04-25T08:57:47.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.305+0000 7fb49ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4a006ed70 0x7fb4a014e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:47.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.305+0000 7fb49ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4a006ed70 0x7fb4a014e350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45860/0 (socket says 172.21.15.43:45860) 2024-04-25T08:57:47.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.305+0000 7fb49ffff640 1 -- 172.21.15.43:0/919504352 learned_addr learned my addr 172.21.15.43:0/919504352 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:47.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.306+0000 7fb4a4fb2640 1 --2- 172.21.15.43:0/919504352 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb4a00d0ec0 0x7fb4a01f6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:47.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.306+0000 7fb49f7fe640 1 --2- 172.21.15.43:0/919504352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4a00cbda0 0x7fb4a0151520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:47.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.306+0000 7fb49ffff640 1 -- 172.21.15.43:0/919504352 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb4a00d0ec0 msgr2=0x7fb4a01f6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:47.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.306+0000 7fb49ffff640 1 --2- 172.21.15.43:0/919504352 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb4a00d0ec0 0x7fb4a01f6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:47.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.306+0000 7fb49ffff640 1 -- 172.21.15.43:0/919504352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4a00cbda0 msgr2=0x7fb4a0151520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:47.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.306+0000 7fb49ffff640 1 --2- 172.21.15.43:0/919504352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4a00cbda0 0x7fb4a0151520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:47.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.306+0000 7fb49ffff640 1 -- 172.21.15.43:0/919504352 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a01f7af0 con 0x7fb4a006ed70 2024-04-25T08:57:47.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.306+0000 7fb49ffff640 1 --2- 172.21.15.43:0/919504352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4a006ed70 0x7fb4a014e350 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fb48800e870 tx=0x7fb48800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:47.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.306+0000 7fb49cff9640 1 -- 172.21.15.43:0/919504352 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb488017070 con 0x7fb4a006ed70 2024-04-25T08:57:47.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.306+0000 7fb49cff9640 1 -- 172.21.15.43:0/919504352 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb48800c940 con 0x7fb4a006ed70 2024-04-25T08:57:47.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.306+0000 7fb4a6a15640 1 -- 172.21.15.43:0/919504352 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4a01f7530 con 0x7fb4a006ed70 2024-04-25T08:57:47.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.306+0000 7fb49cff9640 1 -- 172.21.15.43:0/919504352 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48800cd30 con 0x7fb4a006ed70 2024-04-25T08:57:47.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.306+0000 7fb4a6a15640 1 -- 172.21.15.43:0/919504352 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4a0150d20 con 0x7fb4a006ed70 2024-04-25T08:57:47.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.307+0000 7fb4a6a15640 1 -- 172.21.15.43:0/919504352 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4a00ccf40 con 0x7fb4a006ed70 2024-04-25T08:57:47.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.308+0000 7fb49cff9640 1 -- 172.21.15.43:0/919504352 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb48801a070 con 0x7fb4a006ed70 2024-04-25T08:57:47.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.308+0000 7fb49cff9640 1 --2- 172.21.15.43:0/919504352 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb48006b4d0 0x7fb48006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:47.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.308+0000 7fb49cff9640 1 -- 172.21.15.43:0/919504352 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb488089d50 con 0x7fb4a006ed70 2024-04-25T08:57:47.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.308+0000 7fb49f7fe640 1 --2- 172.21.15.43:0/919504352 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb48006b4d0 0x7fb48006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:47.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.309+0000 7fb49f7fe640 1 --2- 172.21.15.43:0/919504352 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb48006b4d0 0x7fb48006d990 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb490034000 tx=0x7fb49002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:47.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.312+0000 7fb49cff9640 1 -- 172.21.15.43:0/919504352 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb4a00ccf40 con 0x7fb4a006ed70 2024-04-25T08:57:47.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.490+0000 7fb4a6a15640 1 -- 172.21.15.43:0/919504352 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb4a0074f60 con 0x7fb4a006ed70 2024-04-25T08:57:47.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.490+0000 7fb49cff9640 1 -- 172.21.15.43:0/919504352 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb48808d030 con 0x7fb4a006ed70 2024-04-25T08:57:47.480 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T08:57:47.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.492+0000 7fb4a6a15640 1 -- 172.21.15.43:0/919504352 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb48006b4d0 msgr2=0x7fb48006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:47.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.492+0000 7fb4a6a15640 1 --2- 172.21.15.43:0/919504352 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb48006b4d0 0x7fb48006d990 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb490034000 tx=0x7fb49002e000 comp rx=0 tx=0).stop 2024-04-25T08:57:47.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.492+0000 7fb4a6a15640 1 -- 172.21.15.43:0/919504352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4a006ed70 msgr2=0x7fb4a014e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:47.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.492+0000 7fb4a6a15640 1 --2- 172.21.15.43:0/919504352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4a006ed70 0x7fb4a014e350 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fb48800e870 tx=0x7fb48800ed30 comp rx=0 tx=0).stop 2024-04-25T08:57:47.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.493+0000 7fb4a6a15640 1 -- 172.21.15.43:0/919504352 shutdown_connections 2024-04-25T08:57:47.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.493+0000 7fb4a6a15640 1 --2- 172.21.15.43:0/919504352 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb4a00d0ec0 0x7fb4a01f6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:47.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.493+0000 7fb4a6a15640 1 --2- 172.21.15.43:0/919504352 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb48006b4d0 0x7fb48006d990 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:47.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.493+0000 7fb4a6a15640 1 --2- 172.21.15.43:0/919504352 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4a00cbda0 0x7fb4a0151520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:47.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.493+0000 7fb4a6a15640 1 --2- 172.21.15.43:0/919504352 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4a006ed70 0x7fb4a014e350 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:47.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.493+0000 7fb4a6a15640 1 -- 172.21.15.43:0/919504352 >> 172.21.15.43:0/919504352 conn(0x7fb4a00d17e0 msgr2=0x7fb4a006c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:47.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.493+0000 7fb4a6a15640 1 -- 172.21.15.43:0/919504352 shutdown_connections 2024-04-25T08:57:47.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.493+0000 7fb4a6a15640 1 -- 172.21.15.43:0/919504352 wait complete. 2024-04-25T08:57:47.494 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair' 2024-04-25T08:57:47.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.606+0000 7f70d74c4640 1 Processor -- start 2024-04-25T08:57:47.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.606+0000 7f70d74c4640 1 -- start start 2024-04-25T08:57:47.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.607+0000 7f70d74c4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f70d015dcf0 0x7f70d006e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:47.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.607+0000 7f70d74c4640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70d0162500 con 0x7f70d006d4f0 2024-04-25T08:57:47.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.607+0000 7f70d74c4640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70d00d0d80 con 0x7f70d006ebf0 2024-04-25T08:57:47.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.607+0000 7f70d74c4640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70d006c050 con 0x7f70d015dcf0 2024-04-25T08:57:47.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.607+0000 7f70d5260640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f70d015dcf0 0x7f70d006e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:47.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.607+0000 7f70d5a61640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f70d006d4f0 0x7f70d0161df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:57660/0 (socket says 172.21.15.43:57660) 2024-04-25T08:57:47.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.607+0000 7f70d5260640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f70d015dcf0 0x7f70d006e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45876/0 (socket says 172.21.15.43:45876) 2024-04-25T08:57:47.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.607+0000 7f70d5a61640 1 -- 172.21.15.43:0/3805584261 learned_addr learned my addr 172.21.15.43:0/3805584261 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:47.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.607+0000 7f70d5260640 1 -- 172.21.15.43:0/3805584261 >> v1:172.21.15.193:6789/0 conn(0x7f70d006ebf0 legacy=0x7f70d01616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:47.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.607+0000 7f70d5260640 1 -- 172.21.15.43:0/3805584261 >> v1:172.21.15.43:6789/0 conn(0x7f70d006d4f0 legacy=0x7f70d0161df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:47.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.607+0000 7f70d5260640 1 -- 172.21.15.43:0/3805584261 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d00d04e0 con 0x7f70d015dcf0 2024-04-25T08:57:47.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.607+0000 7f70c7fff640 1 -- 172.21.15.43:0/3805584261 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 709658458 0 0) 0x7f70d0162500 con 0x7f70d006d4f0 2024-04-25T08:57:47.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.608+0000 7f70d5260640 1 --2- 172.21.15.43:0/3805584261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70d015dcf0 0x7f70d006e530 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f70cc0097a0 tx=0x7f70cc02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca558a0d4261eeca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:47.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.608+0000 7f70c7fff640 1 -- 172.21.15.43:0/3805584261 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70cc037070 con 0x7f70d015dcf0 2024-04-25T08:57:47.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.608+0000 7f70c7fff640 1 -- 172.21.15.43:0/3805584261 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70cc02ae50 con 0x7f70d015dcf0 2024-04-25T08:57:47.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.608+0000 7f70c7fff640 1 -- 172.21.15.43:0/3805584261 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70cc032660 con 0x7f70d015dcf0 2024-04-25T08:57:47.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.609+0000 7f70d74c4640 1 -- 172.21.15.43:0/3805584261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70d015dcf0 msgr2=0x7f70d006e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:47.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.609+0000 7f70d74c4640 1 --2- 172.21.15.43:0/3805584261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70d015dcf0 0x7f70d006e530 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f70cc0097a0 tx=0x7f70cc02a140 comp rx=0 tx=0).stop 2024-04-25T08:57:47.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.609+0000 7f70d74c4640 1 -- 172.21.15.43:0/3805584261 shutdown_connections 2024-04-25T08:57:47.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.609+0000 7f70d74c4640 1 --2- 172.21.15.43:0/3805584261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70d015dcf0 0x7f70d006e530 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:47.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.609+0000 7f70d74c4640 1 -- 172.21.15.43:0/3805584261 >> 172.21.15.43:0/3805584261 conn(0x7f70d00d1700 msgr2=0x7f70d00d1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:47.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.609+0000 7f70d74c4640 1 -- 172.21.15.43:0/3805584261 shutdown_connections 2024-04-25T08:57:47.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.609+0000 7f70d74c4640 1 -- 172.21.15.43:0/3805584261 wait complete. 2024-04-25T08:57:47.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.610+0000 7f70d74c4640 1 Processor -- start 2024-04-25T08:57:47.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.610+0000 7f70d74c4640 1 -- start start 2024-04-25T08:57:47.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.611+0000 7f70d74c4640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f70d006d4f0 0x7f70d0160d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:47.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.611+0000 7f70d74c4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70d006ebf0 0x7f70d015e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:47.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.611+0000 7f70d74c4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70d015dcf0 0x7f70d015e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:47.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.611+0000 7f70d74c4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d006bce0 con 0x7f70d015dcf0 2024-04-25T08:57:47.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.611+0000 7f70d74c4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d00d0aa0 con 0x7f70d006ebf0 2024-04-25T08:57:47.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.611+0000 7f70d74c4640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70d00b0d50 con 0x7f70d006d4f0 2024-04-25T08:57:47.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.611+0000 7f70d5a61640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70d015dcf0 0x7f70d015e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:47.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.611+0000 7f70d5a61640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70d015dcf0 0x7f70d015e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45888/0 (socket says 172.21.15.43:45888) 2024-04-25T08:57:47.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.611+0000 7f70d5a61640 1 -- 172.21.15.43:0/1809159073 learned_addr learned my addr 172.21.15.43:0/1809159073 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:47.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.611+0000 7f70d5260640 1 --2- 172.21.15.43:0/1809159073 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f70d006d4f0 0x7f70d0160d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:47.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.611+0000 7f70d4a5f640 1 --2- 172.21.15.43:0/1809159073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70d006ebf0 0x7f70d015e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:47.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.611+0000 7f70d5a61640 1 -- 172.21.15.43:0/1809159073 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f70d006d4f0 msgr2=0x7f70d0160d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:47.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.611+0000 7f70d5a61640 1 --2- 172.21.15.43:0/1809159073 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f70d006d4f0 0x7f70d0160d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:47.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.611+0000 7f70d5a61640 1 -- 172.21.15.43:0/1809159073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70d006ebf0 msgr2=0x7f70d015e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:47.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.611+0000 7f70d5a61640 1 --2- 172.21.15.43:0/1809159073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70d006ebf0 0x7f70d015e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:47.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.611+0000 7f70d5a61640 1 -- 172.21.15.43:0/1809159073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d00b6520 con 0x7f70d015dcf0 2024-04-25T08:57:47.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.611+0000 7f70d5260640 1 --2- 172.21.15.43:0/1809159073 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f70d006d4f0 0x7f70d0160d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:47.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.611+0000 7f70d4a5f640 1 --2- 172.21.15.43:0/1809159073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70d006ebf0 0x7f70d015e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:47.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.611+0000 7f70d5a61640 1 --2- 172.21.15.43:0/1809159073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70d015dcf0 0x7f70d015e610 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f70c800c780 tx=0x7f70c800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:47.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.611+0000 7f70c5ffb640 1 -- 172.21.15.43:0/1809159073 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c8018070 con 0x7f70d015dcf0 2024-04-25T08:57:47.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.611+0000 7f70c5ffb640 1 -- 172.21.15.43:0/1809159073 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c800a640 con 0x7f70d015dcf0 2024-04-25T08:57:47.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.611+0000 7f70d74c4640 1 -- 172.21.15.43:0/1809159073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70d00b6800 con 0x7f70d015dcf0 2024-04-25T08:57:47.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.611+0000 7f70c5ffb640 1 -- 172.21.15.43:0/1809159073 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c800aab0 con 0x7f70d015dcf0 2024-04-25T08:57:47.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.612+0000 7f70d74c4640 1 -- 172.21.15.43:0/1809159073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d00b57d0 con 0x7f70d015dcf0 2024-04-25T08:57:47.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.613+0000 7f70d74c4640 1 -- 172.21.15.43:0/1809159073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70d0143620 con 0x7f70d015dcf0 2024-04-25T08:57:47.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.613+0000 7f70c5ffb640 1 -- 172.21.15.43:0/1809159073 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f70c801b070 con 0x7f70d015dcf0 2024-04-25T08:57:47.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.613+0000 7f70c5ffb640 1 --2- 172.21.15.43:0/1809159073 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f70b406b4d0 0x7f70b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:47.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.613+0000 7f70c5ffb640 1 -- 172.21.15.43:0/1809159073 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f70c8089490 con 0x7f70d015dcf0 2024-04-25T08:57:47.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.614+0000 7f70d5260640 1 --2- 172.21.15.43:0/1809159073 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f70b406b4d0 0x7f70b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:47.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.614+0000 7f70d5260640 1 --2- 172.21.15.43:0/1809159073 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f70b406b4d0 0x7f70b406d990 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f70cc034000 tx=0x7f70cc02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:47.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.617+0000 7f70c5ffb640 1 -- 172.21.15.43:0/1809159073 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f70c80586a0 con 0x7f70d015dcf0 2024-04-25T08:57:47.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:47.865+0000 7f70d74c4640 1 -- 172.21.15.43:0/1809159073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]} v 0) v1 -- 0x7f70d0074ef0 con 0x7f70d015dcf0 2024-04-25T08:57:48.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:48.871+0000 7f70c5ffb640 1 -- 172.21.15.43:0/1809159073 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]}]=0 v187) v1 ==== 146+0+0 (secure 0 0 0) 0x7f70c8057fa0 con 0x7f70d015dcf0 2024-04-25T08:57:48.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:48.873+0000 7f70d74c4640 1 -- 172.21.15.43:0/1809159073 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f70b406b4d0 msgr2=0x7f70b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:48.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:48.873+0000 7f70d74c4640 1 --2- 172.21.15.43:0/1809159073 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f70b406b4d0 0x7f70b406d990 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f70cc034000 tx=0x7f70cc02e000 comp rx=0 tx=0).stop 2024-04-25T08:57:48.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:48.873+0000 7f70d74c4640 1 -- 172.21.15.43:0/1809159073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70d015dcf0 msgr2=0x7f70d015e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:48.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:48.873+0000 7f70d74c4640 1 --2- 172.21.15.43:0/1809159073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70d015dcf0 0x7f70d015e610 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f70c800c780 tx=0x7f70c800cc40 comp rx=0 tx=0).stop 2024-04-25T08:57:48.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:48.873+0000 7f70d74c4640 1 -- 172.21.15.43:0/1809159073 shutdown_connections 2024-04-25T08:57:48.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:48.873+0000 7f70d74c4640 1 --2- 172.21.15.43:0/1809159073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70d015dcf0 0x7f70d015e610 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:48.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:48.873+0000 7f70d74c4640 1 --2- 172.21.15.43:0/1809159073 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f70b406b4d0 0x7f70b406d990 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:48.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:48.873+0000 7f70d74c4640 1 --2- 172.21.15.43:0/1809159073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f70d006ebf0 0x7f70d015e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:48.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:48.873+0000 7f70d74c4640 1 --2- 172.21.15.43:0/1809159073 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f70d006d4f0 0x7f70d0160d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:48.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:48.873+0000 7f70d74c4640 1 -- 172.21.15.43:0/1809159073 >> 172.21.15.43:0/1809159073 conn(0x7f70d00d1700 msgr2=0x7f70d00cf990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:48.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:48.873+0000 7f70d74c4640 1 -- 172.21.15.43:0/1809159073 shutdown_connections 2024-04-25T08:57:48.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:48.873+0000 7f70d74c4640 1 -- 172.21.15.43:0/1809159073 wait complete. 2024-04-25T08:57:48.876 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T08:57:48.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:48.998+0000 7feea541f640 1 Processor -- start 2024-04-25T08:57:48.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:48.998+0000 7feea541f640 1 -- start start 2024-04-25T08:57:48.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:48.999+0000 7feea541f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7feea00b5560 0x7feea00b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:48.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:48.999+0000 7feea541f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feea0162610 con 0x7feea015e090 2024-04-25T08:57:48.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:48.999+0000 7feea541f640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feea00d1750 con 0x7feea00d2230 2024-04-25T08:57:48.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:48.999+0000 7feea541f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feea00d1da0 con 0x7feea00b5560 2024-04-25T08:57:48.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:48.999+0000 7fee9effd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7feea015e090 0x7feea0161aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:57662/0 (socket says 172.21.15.43:57662) 2024-04-25T08:57:48.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:48.999+0000 7fee9effd640 1 -- 172.21.15.43:0/2407807521 learned_addr learned my addr 172.21.15.43:0/2407807521 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:48.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:48.999+0000 7fee9e7fc640 1 --2- 172.21.15.43:0/2407807521 >> v2:172.21.15.43:3300/0 conn(0x7feea00b5560 0x7feea00b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:48.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:48.999+0000 7fee9e7fc640 1 -- 172.21.15.43:0/2407807521 >> v1:172.21.15.193:6789/0 conn(0x7feea00d2230 legacy=0x7feea00d26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:48.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:48.999+0000 7fee9e7fc640 1 -- 172.21.15.43:0/2407807521 >> v1:172.21.15.43:6789/0 conn(0x7feea015e090 legacy=0x7feea0161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:48.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:48.999+0000 7fee9e7fc640 1 -- 172.21.15.43:0/2407807521 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea00d1a30 con 0x7feea00b5560 2024-04-25T08:57:48.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:48.999+0000 7fee9dffb640 1 -- 172.21.15.43:0/2407807521 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1768497127 0 0) 0x7feea0162610 con 0x7feea015e090 2024-04-25T08:57:48.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.000+0000 7fee9e7fc640 1 --2- 172.21.15.43:0/2407807521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feea00b5560 0x7feea00b59e0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fee940097a0 tx=0x7fee9402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d36001633a900a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:48.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.000+0000 7fee9dffb640 1 -- 172.21.15.43:0/2407807521 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee94037070 con 0x7feea00b5560 2024-04-25T08:57:48.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.000+0000 7fee9dffb640 1 -- 172.21.15.43:0/2407807521 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee94032440 con 0x7feea00b5560 2024-04-25T08:57:48.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.000+0000 7fee9dffb640 1 -- 172.21.15.43:0/2407807521 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee94032810 con 0x7feea00b5560 2024-04-25T08:57:48.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.001+0000 7feea541f640 1 -- 172.21.15.43:0/2407807521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feea00b5560 msgr2=0x7feea00b59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:48.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.001+0000 7feea541f640 1 --2- 172.21.15.43:0/2407807521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feea00b5560 0x7feea00b59e0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fee940097a0 tx=0x7fee9402a270 comp rx=0 tx=0).stop 2024-04-25T08:57:48.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.001+0000 7feea541f640 1 -- 172.21.15.43:0/2407807521 shutdown_connections 2024-04-25T08:57:48.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.001+0000 7feea541f640 1 --2- 172.21.15.43:0/2407807521 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feea00b5560 0x7feea00b59e0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:48.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.001+0000 7feea541f640 1 -- 172.21.15.43:0/2407807521 >> 172.21.15.43:0/2407807521 conn(0x7feea00d0560 msgr2=0x7feea00d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:48.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.001+0000 7feea541f640 1 -- 172.21.15.43:0/2407807521 shutdown_connections 2024-04-25T08:57:48.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.001+0000 7feea541f640 1 -- 172.21.15.43:0/2407807521 wait complete. 2024-04-25T08:57:48.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.002+0000 7feea541f640 1 Processor -- start 2024-04-25T08:57:48.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.002+0000 7feea541f640 1 -- start start 2024-04-25T08:57:48.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.003+0000 7feea541f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feea00b5560 0x7feea015afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:48.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.003+0000 7feea541f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feea00d2230 0x7feea00d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:48.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.003+0000 7feea541f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feea015e090 0x7feea00d2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:48.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.003+0000 7feea541f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea01621b0 con 0x7feea015e090 2024-04-25T08:57:48.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.003+0000 7feea541f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea00b3280 con 0x7feea00b5560 2024-04-25T08:57:48.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.003+0000 7feea541f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feea0162330 con 0x7feea00d2230 2024-04-25T08:57:48.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.003+0000 7fee9f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feea015e090 0x7feea00d2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:48.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.003+0000 7fee9f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feea015e090 0x7feea00d2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45896/0 (socket says 172.21.15.43:45896) 2024-04-25T08:57:48.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.003+0000 7fee9f7fe640 1 -- 172.21.15.43:0/3053750243 learned_addr learned my addr 172.21.15.43:0/3053750243 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:48.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.003+0000 7fee9f7fe640 1 -- 172.21.15.43:0/3053750243 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feea00d2230 msgr2=0x7feea00d26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:48.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.003+0000 7fee9f7fe640 1 --2- 172.21.15.43:0/3053750243 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feea00d2230 0x7feea00d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:48.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.003+0000 7fee9f7fe640 1 -- 172.21.15.43:0/3053750243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feea00b5560 msgr2=0x7feea015afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:48.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.003+0000 7fee9effd640 1 --2- 172.21.15.43:0/3053750243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feea00b5560 0x7feea015afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:48.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.003+0000 7fee9f7fe640 1 --2- 172.21.15.43:0/3053750243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feea00b5560 0x7feea015afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:48.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.003+0000 7fee9f7fe640 1 -- 172.21.15.43:0/3053750243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea01f3d40 con 0x7feea015e090 2024-04-25T08:57:48.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.003+0000 7fee9effd640 1 --2- 172.21.15.43:0/3053750243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feea00b5560 0x7feea015afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:48.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.004+0000 7fee9f7fe640 1 --2- 172.21.15.43:0/3053750243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feea015e090 0x7feea00d2be0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fee9800c780 tx=0x7fee9800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:48.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.004+0000 7fee877fe640 1 -- 172.21.15.43:0/3053750243 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee98017070 con 0x7feea015e090 2024-04-25T08:57:48.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.004+0000 7fee877fe640 1 -- 172.21.15.43:0/3053750243 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee98003da0 con 0x7feea015e090 2024-04-25T08:57:48.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.004+0000 7fee877fe640 1 -- 172.21.15.43:0/3053750243 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee980052b0 con 0x7feea015e090 2024-04-25T08:57:48.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.004+0000 7feea541f640 1 -- 172.21.15.43:0/3053750243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feea01f4020 con 0x7feea015e090 2024-04-25T08:57:48.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.004+0000 7feea541f640 1 -- 172.21.15.43:0/3053750243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feea01f45e0 con 0x7feea015e090 2024-04-25T08:57:48.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.005+0000 7feea541f640 1 -- 172.21.15.43:0/3053750243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feea006bf00 con 0x7feea015e090 2024-04-25T08:57:48.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.005+0000 7fee877fe640 1 -- 172.21.15.43:0/3053750243 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fee9801a070 con 0x7feea015e090 2024-04-25T08:57:48.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.006+0000 7fee877fe640 1 --2- 172.21.15.43:0/3053750243 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fee8c06b4d0 0x7fee8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:48.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.006+0000 7fee877fe640 1 -- 172.21.15.43:0/3053750243 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fee98089620 con 0x7feea015e090 2024-04-25T08:57:48.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.006+0000 7fee9effd640 1 --2- 172.21.15.43:0/3053750243 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fee8c06b4d0 0x7fee8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:48.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.007+0000 7fee9effd640 1 --2- 172.21.15.43:0/3053750243 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fee8c06b4d0 0x7fee8c06d990 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fee90000f60 tx=0x7fee90009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:48.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.010+0000 7fee877fe640 1 -- 172.21.15.43:0/3053750243 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feea006bf00 con 0x7feea015e090 2024-04-25T08:57:49.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.190+0000 7feea541f640 1 -- 172.21.15.43:0/3053750243 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7feea0074f60 con 0x7feea015e090 2024-04-25T08:57:49.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.193+0000 7fee877fe640 1 -- 172.21.15.43:0/3053750243 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7fee98058700 con 0x7feea015e090 2024-04-25T08:57:49.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.195+0000 7feea541f640 1 -- 172.21.15.43:0/3053750243 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fee8c06b4d0 msgr2=0x7fee8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:49.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.195+0000 7feea541f640 1 --2- 172.21.15.43:0/3053750243 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fee8c06b4d0 0x7fee8c06d990 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fee90000f60 tx=0x7fee90009040 comp rx=0 tx=0).stop 2024-04-25T08:57:49.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.195+0000 7feea541f640 1 -- 172.21.15.43:0/3053750243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feea015e090 msgr2=0x7feea00d2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:49.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.195+0000 7feea541f640 1 --2- 172.21.15.43:0/3053750243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feea015e090 0x7feea00d2be0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fee9800c780 tx=0x7fee9800cc40 comp rx=0 tx=0).stop 2024-04-25T08:57:49.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.195+0000 7feea541f640 1 -- 172.21.15.43:0/3053750243 shutdown_connections 2024-04-25T08:57:49.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.195+0000 7feea541f640 1 --2- 172.21.15.43:0/3053750243 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feea015e090 0x7feea00d2be0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:49.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.195+0000 7feea541f640 1 --2- 172.21.15.43:0/3053750243 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feea00d2230 0x7feea00d26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:49.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.195+0000 7feea541f640 1 --2- 172.21.15.43:0/3053750243 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fee8c06b4d0 0x7fee8c06d990 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:49.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.195+0000 7feea541f640 1 --2- 172.21.15.43:0/3053750243 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feea00b5560 0x7feea015afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:49.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.195+0000 7feea541f640 1 -- 172.21.15.43:0/3053750243 >> 172.21.15.43:0/3053750243 conn(0x7feea00d0560 msgr2=0x7feea00b8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:49.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.195+0000 7feea541f640 1 -- 172.21.15.43:0/3053750243 shutdown_connections 2024-04-25T08:57:49.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.195+0000 7feea541f640 1 -- 172.21.15.43:0/3053750243 wait complete. 2024-04-25T08:57:49.196 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:57:49.196 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:57:49.234 DEBUG:teuthology.orchestra.run.smithi193:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:57:49.277 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:57:49.278 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:57:49.278 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:57:49.294 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2374 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:57:49.294 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T08:57:49.294 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:57:49.335 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19508 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:57:49.335 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:57:49.335 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:57:49.351 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2374 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:57:49.351 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T08:57:49.351 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:57:49.392 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19508 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:57:49.392 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T08:57:49.393 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:57:49.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.504+0000 7f96e6d3b640 1 Processor -- start 2024-04-25T08:57:49.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.504+0000 7f96e6d3b640 1 -- start start 2024-04-25T08:57:49.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.505+0000 7f96e6d3b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f96e006f0e0 0x7f96e00b3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:49.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.505+0000 7f96e6d3b640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f96e006f930 0x7f96e00b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:49.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.505+0000 7f96e6d3b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f96e015aee0 0x7f96e015e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:49.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.505+0000 7f96e6d3b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96e0072130 con 0x7f96e015aee0 2024-04-25T08:57:49.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.505+0000 7f96e6d3b640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96e006f4c0 con 0x7f96e006f930 2024-04-25T08:57:49.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.505+0000 7f96e6d3b640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96e00b3360 con 0x7f96e006f0e0 2024-04-25T08:57:49.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.505+0000 7f96e52d8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f96e015aee0 0x7f96e015e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:49.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.505+0000 7f96e52d8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f96e015aee0 0x7f96e015e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45902/0 (socket says 172.21.15.43:45902) 2024-04-25T08:57:49.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.505+0000 7f96e52d8640 1 -- 172.21.15.43:0/758095248 learned_addr learned my addr 172.21.15.43:0/758095248 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:49.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.505+0000 7f96dffff640 1 --2- 172.21.15.43:0/758095248 >> v2:172.21.15.193:3300/0 conn(0x7f96e006f930 0x7f96e00b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:49.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.505+0000 7f96e4ad7640 1 --2- 172.21.15.43:0/758095248 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f96e006f0e0 0x7f96e00b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:49.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.505+0000 7f96e52d8640 1 -- 172.21.15.43:0/758095248 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f96e006f0e0 msgr2=0x7f96e00b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:49.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.505+0000 7f96e52d8640 1 --2- 172.21.15.43:0/758095248 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f96e006f0e0 0x7f96e00b3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:49.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.505+0000 7f96e52d8640 1 -- 172.21.15.43:0/758095248 >> v2:172.21.15.193:3300/0 conn(0x7f96e006f930 msgr2=0x7f96e00b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:49.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.505+0000 7f96e52d8640 1 --2- 172.21.15.43:0/758095248 >> v2:172.21.15.193:3300/0 conn(0x7f96e006f930 0x7f96e00b3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:49.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.505+0000 7f96e52d8640 1 -- 172.21.15.43:0/758095248 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e0071520 con 0x7f96e015aee0 2024-04-25T08:57:49.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.505+0000 7f96e4ad7640 1 --2- 172.21.15.43:0/758095248 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f96e006f0e0 0x7f96e00b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:49.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.506+0000 7f96e52d8640 1 --2- 172.21.15.43:0/758095248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96e015aee0 0x7f96e015e4d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f96e006d330 tx=0x7f96d802c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e8daa46d317fbb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:49.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.506+0000 7f96df7fe640 1 -- 172.21.15.43:0/758095248 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d8033400 con 0x7f96e015aee0 2024-04-25T08:57:49.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.506+0000 7f96df7fe640 1 -- 172.21.15.43:0/758095248 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d80335a0 con 0x7f96e015aee0 2024-04-25T08:57:49.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.506+0000 7f96df7fe640 1 -- 172.21.15.43:0/758095248 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d8033970 con 0x7f96e015aee0 2024-04-25T08:57:49.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.507+0000 7f96e6d3b640 1 -- 172.21.15.43:0/758095248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96e015aee0 msgr2=0x7f96e015e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:49.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.507+0000 7f96e6d3b640 1 --2- 172.21.15.43:0/758095248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96e015aee0 0x7f96e015e4d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f96e006d330 tx=0x7f96d802c8b0 comp rx=0 tx=0).stop 2024-04-25T08:57:49.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.507+0000 7f96e6d3b640 1 -- 172.21.15.43:0/758095248 shutdown_connections 2024-04-25T08:57:49.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.507+0000 7f96e6d3b640 1 --2- 172.21.15.43:0/758095248 >> v2:172.21.15.193:3300/0 conn(0x7f96e006f930 0x7f96e00b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:49.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.507+0000 7f96e6d3b640 1 --2- 172.21.15.43:0/758095248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96e015aee0 0x7f96e015e4d0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:49.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.507+0000 7f96e6d3b640 1 --2- 172.21.15.43:0/758095248 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f96e006f0e0 0x7f96e00b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:49.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.507+0000 7f96e6d3b640 1 -- 172.21.15.43:0/758095248 >> 172.21.15.43:0/758095248 conn(0x7f96e00b2490 msgr2=0x7f96e00b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:49.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.507+0000 7f96e6d3b640 1 -- 172.21.15.43:0/758095248 shutdown_connections 2024-04-25T08:57:49.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.507+0000 7f96e6d3b640 1 -- 172.21.15.43:0/758095248 wait complete. 2024-04-25T08:57:49.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.508+0000 7f96e6d3b640 1 Processor -- start 2024-04-25T08:57:49.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.508+0000 7f96e6d3b640 1 -- start start 2024-04-25T08:57:49.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.509+0000 7f96e6d3b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f96e006f0e0 0x7f96e01f2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:49.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.509+0000 7f96e6d3b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96e006f930 0x7f96e0154730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:49.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.509+0000 7f96e6d3b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96e015aee0 0x7f96e015cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:49.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.509+0000 7f96e6d3b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e0071dc0 con 0x7f96e015aee0 2024-04-25T08:57:49.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.509+0000 7f96e6d3b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e0071ae0 con 0x7f96e006f930 2024-04-25T08:57:49.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.509+0000 7f96e6d3b640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96e015ea10 con 0x7f96e006f0e0 2024-04-25T08:57:49.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.509+0000 7f96e52d8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96e015aee0 0x7f96e015cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:49.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.509+0000 7f96e52d8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96e015aee0 0x7f96e015cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45916/0 (socket says 172.21.15.43:45916) 2024-04-25T08:57:49.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.509+0000 7f96e52d8640 1 -- 172.21.15.43:0/781594504 learned_addr learned my addr 172.21.15.43:0/781594504 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:49.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.509+0000 7f96dffff640 1 --2- 172.21.15.43:0/781594504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96e006f930 0x7f96e0154730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:49.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.509+0000 7f96e4ad7640 1 --2- 172.21.15.43:0/781594504 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f96e006f0e0 0x7f96e01f2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:49.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.509+0000 7f96e52d8640 1 -- 172.21.15.43:0/781594504 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f96e006f0e0 msgr2=0x7f96e01f2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:49.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.509+0000 7f96e52d8640 1 --2- 172.21.15.43:0/781594504 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f96e006f0e0 0x7f96e01f2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:49.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.509+0000 7f96e52d8640 1 -- 172.21.15.43:0/781594504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96e006f930 msgr2=0x7f96e0154730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:49.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.509+0000 7f96e52d8640 1 --2- 172.21.15.43:0/781594504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96e006f930 0x7f96e0154730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:49.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.509+0000 7f96e52d8640 1 -- 172.21.15.43:0/781594504 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e0145ec0 con 0x7f96e015aee0 2024-04-25T08:57:49.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.509+0000 7f96e52d8640 1 --2- 172.21.15.43:0/781594504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96e015aee0 0x7f96e015cf60 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f96d8009f40 tx=0x7f96d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:49.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.510+0000 7f96dd7fa640 1 -- 172.21.15.43:0/781594504 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d8041070 con 0x7f96e015aee0 2024-04-25T08:57:49.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.510+0000 7f96dd7fa640 1 -- 172.21.15.43:0/781594504 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d803d820 con 0x7f96e015aee0 2024-04-25T08:57:49.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.510+0000 7f96e6d3b640 1 -- 172.21.15.43:0/781594504 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96e01461a0 con 0x7f96e015aee0 2024-04-25T08:57:49.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.510+0000 7f96dd7fa640 1 -- 172.21.15.43:0/781594504 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d8033400 con 0x7f96e015aee0 2024-04-25T08:57:49.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.510+0000 7f96e6d3b640 1 -- 172.21.15.43:0/781594504 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96e0146760 con 0x7f96e015aee0 2024-04-25T08:57:49.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.511+0000 7f96e6d3b640 1 -- 172.21.15.43:0/781594504 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96e0074ef0 con 0x7f96e015aee0 2024-04-25T08:57:49.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.511+0000 7f96dd7fa640 1 -- 172.21.15.43:0/781594504 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f96d803dbd0 con 0x7f96e015aee0 2024-04-25T08:57:49.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.512+0000 7f96dd7fa640 1 --2- 172.21.15.43:0/781594504 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f96c406b4d0 0x7f96c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:49.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.512+0000 7f96dd7fa640 1 -- 172.21.15.43:0/781594504 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f96d80ad1f0 con 0x7f96e015aee0 2024-04-25T08:57:49.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.512+0000 7f96e4ad7640 1 --2- 172.21.15.43:0/781594504 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f96c406b4d0 0x7f96c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:49.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.512+0000 7f96e4ad7640 1 --2- 172.21.15.43:0/781594504 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f96c406b4d0 0x7f96c406d990 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f96d4006fd0 tx=0x7f96d4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:49.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.516+0000 7f96dd7fa640 1 -- 172.21.15.43:0/781594504 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f96d807c380 con 0x7f96e015aee0 2024-04-25T08:57:49.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.735+0000 7f96e6d3b640 1 -- 172.21.15.43:0/781594504 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96e00b44d0 con 0x7f96e015aee0 2024-04-25T08:57:49.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.736+0000 7f96dd7fa640 1 -- 172.21.15.43:0/781594504 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f96d807bc80 con 0x7f96e015aee0 2024-04-25T08:57:49.725 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:57:49.725 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":27,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5359,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/1504450503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1504450503},{"type":"v1","addr":"172.21.15.193:6835","nonce":1504450503}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5691,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2657133821","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2657133821},{"type":"v1","addr":"172.21.15.43:6834","nonce":2657133821}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17},{"gid":5694,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/4271431818","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":4271431818},{"type":"v1","addr":"172.21.15.43:6835","nonce":4271431818}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":5893,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/307572958","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":307572958},{"type":"v1","addr":"172.21.15.193:6837","nonce":307572958}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":27}],"filesystems":[]} 2024-04-25T08:57:49.726 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 27 2024-04-25T08:57:49.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.738+0000 7f96e6d3b640 1 -- 172.21.15.43:0/781594504 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f96c406b4d0 msgr2=0x7f96c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:49.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.738+0000 7f96e6d3b640 1 --2- 172.21.15.43:0/781594504 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f96c406b4d0 0x7f96c406d990 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f96d4006fd0 tx=0x7f96d4008040 comp rx=0 tx=0).stop 2024-04-25T08:57:49.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.738+0000 7f96e6d3b640 1 -- 172.21.15.43:0/781594504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96e015aee0 msgr2=0x7f96e015cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:49.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.738+0000 7f96e6d3b640 1 --2- 172.21.15.43:0/781594504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96e015aee0 0x7f96e015cf60 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f96d8009f40 tx=0x7f96d8004a40 comp rx=0 tx=0).stop 2024-04-25T08:57:49.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.738+0000 7f96e6d3b640 1 -- 172.21.15.43:0/781594504 shutdown_connections 2024-04-25T08:57:49.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.738+0000 7f96e6d3b640 1 --2- 172.21.15.43:0/781594504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96e015aee0 0x7f96e015cf60 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:49.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.738+0000 7f96e6d3b640 1 --2- 172.21.15.43:0/781594504 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f96c406b4d0 0x7f96c406d990 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:49.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.738+0000 7f96e6d3b640 1 --2- 172.21.15.43:0/781594504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96e006f930 0x7f96e0154730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:49.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.738+0000 7f96e6d3b640 1 --2- 172.21.15.43:0/781594504 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f96e006f0e0 0x7f96e01f2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:49.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.738+0000 7f96e6d3b640 1 -- 172.21.15.43:0/781594504 >> 172.21.15.43:0/781594504 conn(0x7f96e00b2490 msgr2=0x7f96e015b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:49.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.738+0000 7f96e6d3b640 1 -- 172.21.15.43:0/781594504 shutdown_connections 2024-04-25T08:57:49.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.738+0000 7f96e6d3b640 1 -- 172.21.15.43:0/781594504 wait complete. 2024-04-25T08:57:49.739 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T08:57:49.739 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T08:57:49.739 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T08:57:49.739 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:57:49.739 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T08:57:49.739 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T08:57:49.740 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T08:57:49.740 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:57:49.740 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T08:57:49.740 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T08:57:49.740 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T08:57:49.740 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:57:49.740 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T08:57:49.741 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T08:57:49.741 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T08:57:49.741 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:57:49.749 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T08:57:49.749 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T08:57:49.750 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T08:57:49.750 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T08:57:49.755 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T08:57:49.756 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T08:57:49.756 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T08:57:49.756 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T08:57:49.792 INFO:tasks.ceph.mds.a:Started 2024-04-25T08:57:49.792 DEBUG:teuthology.parallel:result is None 2024-04-25T08:57:49.793 INFO:tasks.ceph.mds.c:Started 2024-04-25T08:57:49.793 DEBUG:teuthology.parallel:result is None 2024-04-25T08:57:49.799 INFO:tasks.ceph.mds.b:Started 2024-04-25T08:57:49.799 DEBUG:teuthology.parallel:result is None 2024-04-25T08:57:49.799 INFO:tasks.ceph.mds.d:Started 2024-04-25T08:57:49.799 DEBUG:teuthology.parallel:result is None 2024-04-25T08:57:49.800 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T08:57:49.868 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2024-04-25T08:57:49.868 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-04-25T08:57:49.880 INFO:tasks.ceph.mds.b.smithi193.stdout:starting mds.b at 2024-04-25T08:57:49.880 INFO:tasks.ceph.mds.d.smithi193.stdout:starting mds.d at 2024-04-25T08:57:49.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.915+0000 7fcc6c9d5640 1 Processor -- start 2024-04-25T08:57:49.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.915+0000 7fcc6c9d5640 1 -- start start 2024-04-25T08:57:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.916+0000 7fcc6c9d5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcc6815df50 0x7fcc681619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.916+0000 7fcc6c9d5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc680d24a0 con 0x7fcc680cf670 2024-04-25T08:57:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.916+0000 7fcc6c9d5640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc680d2740 con 0x7fcc68161f30 2024-04-25T08:57:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.916+0000 7fcc6c9d5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc680cfa80 con 0x7fcc6815df50 2024-04-25T08:57:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.916+0000 7fcc6659c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcc6815df50 0x7fcc681619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.916+0000 7fcc66d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fcc680cf670 0x7fcc680c8400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45292/0 (socket says 172.21.15.43:45292) 2024-04-25T08:57:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.916+0000 7fcc66d9d640 1 -- 172.21.15.43:0/1092289957 learned_addr learned my addr 172.21.15.43:0/1092289957 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.916+0000 7fcc6659c640 1 -- 172.21.15.43:0/1092289957 >> v1:172.21.15.193:6789/0 conn(0x7fcc68161f30 legacy=0x7fcc680c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:49.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.916+0000 7fcc6659c640 1 -- 172.21.15.43:0/1092289957 >> v1:172.21.15.43:6789/0 conn(0x7fcc680cf670 legacy=0x7fcc680c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:49.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.916+0000 7fcc6659c640 1 -- 172.21.15.43:0/1092289957 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc680d2a20 con 0x7fcc6815df50 2024-04-25T08:57:49.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.917+0000 7fcc6659c640 1 --2- 172.21.15.43:0/1092289957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc6815df50 0x7fcc681619f0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fcc580097a0 tx=0x7fcc5802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44bd95f8e28fa2a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:49.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.917+0000 7fcc65d9b640 1 -- 172.21.15.43:0/1092289957 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc58037070 con 0x7fcc6815df50 2024-04-25T08:57:49.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.917+0000 7fcc65d9b640 1 -- 172.21.15.43:0/1092289957 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc58032440 con 0x7fcc6815df50 2024-04-25T08:57:49.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.917+0000 7fcc65d9b640 1 -- 172.21.15.43:0/1092289957 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc58032810 con 0x7fcc6815df50 2024-04-25T08:57:49.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.918+0000 7fcc6c9d5640 1 -- 172.21.15.43:0/1092289957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc6815df50 msgr2=0x7fcc681619f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:49.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.918+0000 7fcc6c9d5640 1 --2- 172.21.15.43:0/1092289957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc6815df50 0x7fcc681619f0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fcc580097a0 tx=0x7fcc5802a230 comp rx=0 tx=0).stop 2024-04-25T08:57:49.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.918+0000 7fcc6c9d5640 1 -- 172.21.15.43:0/1092289957 shutdown_connections 2024-04-25T08:57:49.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.918+0000 7fcc6c9d5640 1 --2- 172.21.15.43:0/1092289957 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc6815df50 0x7fcc681619f0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:49.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.918+0000 7fcc6c9d5640 1 -- 172.21.15.43:0/1092289957 >> 172.21.15.43:0/1092289957 conn(0x7fcc680d3a30 msgr2=0x7fcc680d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:49.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.918+0000 7fcc6c9d5640 1 -- 172.21.15.43:0/1092289957 shutdown_connections 2024-04-25T08:57:49.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.918+0000 7fcc6c9d5640 1 -- 172.21.15.43:0/1092289957 wait complete. 2024-04-25T08:57:49.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.919+0000 7fcc6c9d5640 1 Processor -- start 2024-04-25T08:57:49.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.919+0000 7fcc6c9d5640 1 -- start start 2024-04-25T08:57:49.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.920+0000 7fcc6c9d5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc680cf670 0x7fcc681607e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:49.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.920+0000 7fcc6c9d5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc6815df50 0x7fcc6809d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:49.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.920+0000 7fcc6c9d5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc68161f30 0x7fcc68144b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:49.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.920+0000 7fcc6c9d5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc680c77d0 con 0x7fcc68161f30 2024-04-25T08:57:49.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.920+0000 7fcc6c9d5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6815e330 con 0x7fcc6815df50 2024-04-25T08:57:49.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.920+0000 7fcc6c9d5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc680d2190 con 0x7fcc680cf670 2024-04-25T08:57:49.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.920+0000 7fcc66d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc68161f30 0x7fcc68144b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:49.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.920+0000 7fcc66d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc68161f30 0x7fcc68144b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49894/0 (socket says 172.21.15.43:49894) 2024-04-25T08:57:49.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.920+0000 7fcc66d9d640 1 -- 172.21.15.43:0/2863306033 learned_addr learned my addr 172.21.15.43:0/2863306033 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:49.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.920+0000 7fcc5ffff640 1 --2- 172.21.15.43:0/2863306033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc6815df50 0x7fcc6809d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:49.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.920+0000 7fcc6659c640 1 --2- 172.21.15.43:0/2863306033 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc680cf670 0x7fcc681607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:49.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.920+0000 7fcc66d9d640 1 -- 172.21.15.43:0/2863306033 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc680cf670 msgr2=0x7fcc681607e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:49.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.920+0000 7fcc66d9d640 1 --2- 172.21.15.43:0/2863306033 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc680cf670 0x7fcc681607e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:49.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.920+0000 7fcc66d9d640 1 -- 172.21.15.43:0/2863306033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc6815df50 msgr2=0x7fcc6809d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:49.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.920+0000 7fcc66d9d640 1 --2- 172.21.15.43:0/2863306033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc6815df50 0x7fcc6809d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:49.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.920+0000 7fcc66d9d640 1 -- 172.21.15.43:0/2863306033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc68143a80 con 0x7fcc68161f30 2024-04-25T08:57:49.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.920+0000 7fcc5ffff640 1 --2- 172.21.15.43:0/2863306033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc6815df50 0x7fcc6809d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:49.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.921+0000 7fcc66d9d640 1 --2- 172.21.15.43:0/2863306033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc68161f30 0x7fcc68144b20 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fcc6000cb10 tx=0x7fcc6000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:49.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.921+0000 7fcc5effd640 1 -- 172.21.15.43:0/2863306033 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc60017070 con 0x7fcc68161f30 2024-04-25T08:57:49.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.921+0000 7fcc5effd640 1 -- 172.21.15.43:0/2863306033 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc6000ac70 con 0x7fcc68161f30 2024-04-25T08:57:49.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.921+0000 7fcc6c9d5640 1 -- 172.21.15.43:0/2863306033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc68143d60 con 0x7fcc68161f30 2024-04-25T08:57:49.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.921+0000 7fcc5effd640 1 -- 172.21.15.43:0/2863306033 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc60012620 con 0x7fcc68161f30 2024-04-25T08:57:49.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.921+0000 7fcc6c9d5640 1 -- 172.21.15.43:0/2863306033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc68144320 con 0x7fcc68161f30 2024-04-25T08:57:49.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.922+0000 7fcc6c9d5640 1 -- 172.21.15.43:0/2863306033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc680680f0 con 0x7fcc68161f30 2024-04-25T08:57:49.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.922+0000 7fcc5effd640 1 -- 172.21.15.43:0/2863306033 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcc6001a070 con 0x7fcc68161f30 2024-04-25T08:57:49.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.923+0000 7fcc5effd640 1 --2- 172.21.15.43:0/2863306033 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcc4806b4d0 0x7fcc4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:49.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.923+0000 7fcc5effd640 1 -- 172.21.15.43:0/2863306033 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fcc60089c40 con 0x7fcc68161f30 2024-04-25T08:57:49.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.923+0000 7fcc6659c640 1 --2- 172.21.15.43:0/2863306033 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcc4806b4d0 0x7fcc4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:49.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.923+0000 7fcc6659c640 1 --2- 172.21.15.43:0/2863306033 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcc4806b4d0 0x7fcc4806d990 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcc58034000 tx=0x7fcc5802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:49.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:49.927+0000 7fcc5effd640 1 -- 172.21.15.43:0/2863306033 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcc6000ae20 con 0x7fcc68161f30 2024-04-25T08:57:50.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:50.120+0000 7fcc6c9d5640 1 -- 172.21.15.43:0/2863306033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fcc680ca400 con 0x7fcc68161f30 2024-04-25T08:57:51.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.207+0000 7fcc5effd640 1 -- 172.21.15.43:0/2863306033 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v54) v1 ==== 97+0+0 (secure 0 0 0) 0x7fcc60058dc0 con 0x7fcc68161f30 2024-04-25T08:57:51.196 INFO:teuthology.orchestra.run.smithi043.stderr: removed all blocklist entries 2024-04-25T08:57:51.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.210+0000 7fcc6c9d5640 1 -- 172.21.15.43:0/2863306033 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcc4806b4d0 msgr2=0x7fcc4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:51.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.210+0000 7fcc6c9d5640 1 --2- 172.21.15.43:0/2863306033 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcc4806b4d0 0x7fcc4806d990 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fcc58034000 tx=0x7fcc5802e000 comp rx=0 tx=0).stop 2024-04-25T08:57:51.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.210+0000 7fcc6c9d5640 1 -- 172.21.15.43:0/2863306033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc68161f30 msgr2=0x7fcc68144b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:51.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.210+0000 7fcc6c9d5640 1 --2- 172.21.15.43:0/2863306033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc68161f30 0x7fcc68144b20 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fcc6000cb10 tx=0x7fcc6000a060 comp rx=0 tx=0).stop 2024-04-25T08:57:51.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.210+0000 7fcc6c9d5640 1 -- 172.21.15.43:0/2863306033 shutdown_connections 2024-04-25T08:57:51.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.210+0000 7fcc6c9d5640 1 --2- 172.21.15.43:0/2863306033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc68161f30 0x7fcc68144b20 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:51.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.210+0000 7fcc6c9d5640 1 --2- 172.21.15.43:0/2863306033 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcc4806b4d0 0x7fcc4806d990 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:51.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.210+0000 7fcc6c9d5640 1 --2- 172.21.15.43:0/2863306033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc6815df50 0x7fcc6809d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:51.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.210+0000 7fcc6c9d5640 1 --2- 172.21.15.43:0/2863306033 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc680cf670 0x7fcc681607e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:51.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.210+0000 7fcc6c9d5640 1 -- 172.21.15.43:0/2863306033 >> 172.21.15.43:0/2863306033 conn(0x7fcc680d3a30 msgr2=0x7fcc68153c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:51.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.210+0000 7fcc6c9d5640 1 -- 172.21.15.43:0/2863306033 shutdown_connections 2024-04-25T08:57:51.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.210+0000 7fcc6c9d5640 1 -- 172.21.15.43:0/2863306033 wait complete. 2024-04-25T08:57:51.213 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T08:57:51.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.331+0000 7f6912730640 1 Processor -- start 2024-04-25T08:57:51.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.331+0000 7f6912730640 1 -- start start 2024-04-25T08:57:51.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.331+0000 7f6912730640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f690c161390 0x7f690c0b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:51.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.331+0000 7f6912730640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f690c0d02a0 0x7f690c0d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:51.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.331+0000 7f6912730640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f690c0b4c30 con 0x7f690c0ce5e0 2024-04-25T08:57:51.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.331+0000 7f6912730640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f690c0d1910 con 0x7f690c0d02a0 2024-04-25T08:57:51.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.331+0000 7f6912730640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f690c0cf2c0 con 0x7f690c161390 2024-04-25T08:57:51.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.332+0000 7f690b7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f690c0d02a0 0x7f690c0d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:51.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.332+0000 7f6910ccd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f690c0ce5e0 0x7f690c0cea70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45296/0 (socket says 172.21.15.43:45296) 2024-04-25T08:57:51.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.332+0000 7f6910ccd640 1 -- 172.21.15.43:0/805047221 learned_addr learned my addr 172.21.15.43:0/805047221 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:51.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.332+0000 7f690b7fe640 1 -- 172.21.15.43:0/805047221 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f690c161390 msgr2=0x7f690c0b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:51.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.332+0000 7f690b7fe640 1 --2- 172.21.15.43:0/805047221 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f690c161390 0x7f690c0b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:51.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.332+0000 7f690b7fe640 1 -- 172.21.15.43:0/805047221 >> v1:172.21.15.43:6789/0 conn(0x7f690c0ce5e0 legacy=0x7f690c0cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:51.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.332+0000 7f690b7fe640 1 -- 172.21.15.43:0/805047221 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f690c0cf5a0 con 0x7f690c0d02a0 2024-04-25T08:57:51.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.333+0000 7f690b7fe640 1 --2- 172.21.15.43:0/805047221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f690c0d02a0 0x7f690c0d0720 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f69000097a0 tx=0x7f690002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39315a1cc961a159 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:51.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.333+0000 7f690affd640 1 -- 172.21.15.43:0/805047221 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6900037070 con 0x7f690c0d02a0 2024-04-25T08:57:51.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.333+0000 7f690affd640 1 -- 172.21.15.43:0/805047221 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6900032440 con 0x7f690c0d02a0 2024-04-25T08:57:51.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.333+0000 7f690affd640 1 -- 172.21.15.43:0/805047221 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6900032810 con 0x7f690c0d02a0 2024-04-25T08:57:51.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.333+0000 7f6912730640 1 -- 172.21.15.43:0/805047221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f690c0d02a0 msgr2=0x7f690c0d0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:51.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.333+0000 7f6912730640 1 --2- 172.21.15.43:0/805047221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f690c0d02a0 0x7f690c0d0720 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f69000097a0 tx=0x7f690002a180 comp rx=0 tx=0).stop 2024-04-25T08:57:51.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.333+0000 7f6912730640 1 -- 172.21.15.43:0/805047221 shutdown_connections 2024-04-25T08:57:51.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.333+0000 7f6912730640 1 --2- 172.21.15.43:0/805047221 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f690c0d02a0 0x7f690c0d0720 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:51.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.333+0000 7f6912730640 1 --2- 172.21.15.43:0/805047221 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f690c161390 0x7f690c0b6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:51.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.333+0000 7f6912730640 1 -- 172.21.15.43:0/805047221 >> 172.21.15.43:0/805047221 conn(0x7f690c09cd20 msgr2=0x7f690c0d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:51.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.333+0000 7f6912730640 1 -- 172.21.15.43:0/805047221 shutdown_connections 2024-04-25T08:57:51.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.333+0000 7f6912730640 1 -- 172.21.15.43:0/805047221 wait complete. 2024-04-25T08:57:51.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.335+0000 7f6912730640 1 Processor -- start 2024-04-25T08:57:51.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.335+0000 7f6912730640 1 -- start start 2024-04-25T08:57:51.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.335+0000 7f6912730640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f690c0ce5e0 0x7f690c155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:51.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.336+0000 7f6912730640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f690c0d02a0 0x7f690c15a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:51.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.336+0000 7f6912730640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f690c161390 0x7f690c15ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:51.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.336+0000 7f6912730640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690c161770 con 0x7f690c161390 2024-04-25T08:57:51.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.336+0000 7f6912730640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690c0d15a0 con 0x7f690c0ce5e0 2024-04-25T08:57:51.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.336+0000 7f6912730640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f690c0d1da0 con 0x7f690c0d02a0 2024-04-25T08:57:51.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.336+0000 7f6910ccd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f690c161390 0x7f690c15ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:51.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.336+0000 7f6910ccd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f690c161390 0x7f690c15ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49900/0 (socket says 172.21.15.43:49900) 2024-04-25T08:57:51.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.336+0000 7f6910ccd640 1 -- 172.21.15.43:0/2684070875 learned_addr learned my addr 172.21.15.43:0/2684070875 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:51.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.336+0000 7f690bfff640 1 --2- 172.21.15.43:0/2684070875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f690c0ce5e0 0x7f690c155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:51.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.336+0000 7f690b7fe640 1 --2- 172.21.15.43:0/2684070875 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f690c0d02a0 0x7f690c15a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:51.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.336+0000 7f6910ccd640 1 -- 172.21.15.43:0/2684070875 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f690c0d02a0 msgr2=0x7f690c15a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:51.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.336+0000 7f6910ccd640 1 --2- 172.21.15.43:0/2684070875 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f690c0d02a0 0x7f690c15a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:51.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.336+0000 7f6910ccd640 1 -- 172.21.15.43:0/2684070875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f690c0ce5e0 msgr2=0x7f690c155cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:51.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.336+0000 7f6910ccd640 1 --2- 172.21.15.43:0/2684070875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f690c0ce5e0 0x7f690c155cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:51.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.336+0000 7f6910ccd640 1 -- 172.21.15.43:0/2684070875 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f690c0c8b90 con 0x7f690c161390 2024-04-25T08:57:51.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.336+0000 7f690bfff640 1 --2- 172.21.15.43:0/2684070875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f690c0ce5e0 0x7f690c155cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:51.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.336+0000 7f690b7fe640 1 --2- 172.21.15.43:0/2684070875 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f690c0d02a0 0x7f690c15a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:51.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.336+0000 7f6910ccd640 1 --2- 172.21.15.43:0/2684070875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f690c161390 0x7f690c15ce60 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f690400cbc0 tx=0x7f690400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:51.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.336+0000 7f6908ff9640 1 -- 172.21.15.43:0/2684070875 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6904017070 con 0x7f690c161390 2024-04-25T08:57:51.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.336+0000 7f6912730640 1 -- 172.21.15.43:0/2684070875 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f690c0c8e70 con 0x7f690c161390 2024-04-25T08:57:51.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.336+0000 7f6908ff9640 1 -- 172.21.15.43:0/2684070875 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f690400ad40 con 0x7f690c161390 2024-04-25T08:57:51.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.336+0000 7f6908ff9640 1 -- 172.21.15.43:0/2684070875 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6904012600 con 0x7f690c161390 2024-04-25T08:57:51.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.336+0000 7f6912730640 1 -- 172.21.15.43:0/2684070875 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f690c0c9430 con 0x7f690c161390 2024-04-25T08:57:51.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.337+0000 7f6912730640 1 -- 172.21.15.43:0/2684070875 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f690c0680f0 con 0x7f690c161390 2024-04-25T08:57:51.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.338+0000 7f6908ff9640 1 -- 172.21.15.43:0/2684070875 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f690401a070 con 0x7f690c161390 2024-04-25T08:57:51.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.338+0000 7f6908ff9640 1 --2- 172.21.15.43:0/2684070875 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f68f806b4d0 0x7f68f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:51.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.338+0000 7f6908ff9640 1 -- 172.21.15.43:0/2684070875 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6904089b40 con 0x7f690c161390 2024-04-25T08:57:51.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.338+0000 7f690bfff640 1 --2- 172.21.15.43:0/2684070875 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f68f806b4d0 0x7f68f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:51.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.339+0000 7f690bfff640 1 --2- 172.21.15.43:0/2684070875 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f68f806b4d0 0x7f68f806d990 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f68fc009890 tx=0x7f68fc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:51.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.342+0000 7f6908ff9640 1 -- 172.21.15.43:0/2684070875 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f690c0b4c30 con 0x7f690c161390 2024-04-25T08:57:51.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.593+0000 7f6912730640 1 -- 172.21.15.43:0/2684070875 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f690c074ef0 con 0x7f690c161390 2024-04-25T08:57:51.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.594+0000 7f6908ff9640 1 -- 172.21.15.43:0/2684070875 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f69040586d0 con 0x7f690c161390 2024-04-25T08:57:51.584 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:57:51.584 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T08:57:51.584 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_dump": [ 2024-04-25T08:57:51.584 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:57:51.585 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.a", 2024-04-25T08:57:51.585 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpmIjzEBRAAs2L10XrFO3mThjZevna5wA==", 2024-04-25T08:57:51.585 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:57:51.585 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T08:57:51.585 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T08:57:51.585 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T08:57:51.586 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:57:51.586 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:57:51.586 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:57:51.586 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:57:51.586 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.b", 2024-04-25T08:57:51.586 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm51cmERAAij4vy8xyihh8u/4dkmcllw==", 2024-04-25T08:57:51.587 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:57:51.587 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T08:57:51.587 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T08:57:51.587 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T08:57:51.587 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:57:51.587 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:57:51.588 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:57:51.588 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:57:51.588 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.c", 2024-04-25T08:57:51.588 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm1OOtDBAAlUYbCh1mBEEka9CWRnUz9A==", 2024-04-25T08:57:51.588 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:57:51.588 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T08:57:51.589 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T08:57:51.589 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T08:57:51.589 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:57:51.589 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:57:51.589 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:57:51.589 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:57:51.590 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.d", 2024-04-25T08:57:51.590 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpmpEvsFxAAXX1h37UWdFkyJ3Iw3xd8kw==", 2024-04-25T08:57:51.590 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:57:51.590 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T08:57:51.590 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T08:57:51.590 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T08:57:51.591 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:57:51.591 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:57:51.591 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:57:51.591 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:57:51.591 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.0", 2024-04-25T08:57:51.591 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD9GSpmi7qkORAAwSNgwzcVMitrPPf3nDyqvQ==", 2024-04-25T08:57:51.592 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:57:51.592 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T08:57:51.592 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T08:57:51.592 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:57:51.592 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:57:51.592 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:57:51.592 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:57:51.593 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.1", 2024-04-25T08:57:51.593 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAAGipmJDT3NxAAcWBCd9nt7Au+Xftjqcuc5w==", 2024-04-25T08:57:51.593 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:57:51.593 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T08:57:51.593 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T08:57:51.593 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:57:51.594 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:57:51.594 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:57:51.594 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:57:51.594 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.2", 2024-04-25T08:57:51.594 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQADGipm68kzNhAA0ZOnlr4edWDjx6RfoEHYIw==", 2024-04-25T08:57:51.595 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:57:51.595 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T08:57:51.595 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T08:57:51.595 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:57:51.595 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:57:51.595 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:57:51.596 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:57:51.596 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.3", 2024-04-25T08:57:51.596 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAGGipmT1AeNBAAoNPiEtkznSxxhVJ1EmZ1ZA==", 2024-04-25T08:57:51.596 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:57:51.596 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T08:57:51.596 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T08:57:51.597 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:57:51.597 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:57:51.597 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:57:51.597 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:57:51.597 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.4", 2024-04-25T08:57:51.597 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAMGipmBjHHNxAAH/SCzKHus8fKxxI3UN4wWQ==", 2024-04-25T08:57:51.598 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:57:51.598 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T08:57:51.598 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T08:57:51.598 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:57:51.598 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:57:51.598 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:57:51.599 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:57:51.599 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.5", 2024-04-25T08:57:51.599 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPGipm3FXKNRAAL97YYWONg4QHr4ajC5D7fA==", 2024-04-25T08:57:51.599 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:57:51.599 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T08:57:51.599 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T08:57:51.600 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:57:51.600 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:57:51.600 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:57:51.600 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:57:51.600 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.6", 2024-04-25T08:57:51.600 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQASGipmCwKaMxAAcM0p9Q2sbNQviGkwSLg8AA==", 2024-04-25T08:57:51.601 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:57:51.601 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T08:57:51.601 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T08:57:51.601 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:57:51.601 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:57:51.601 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:57:51.602 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:57:51.602 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.7", 2024-04-25T08:57:51.602 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAVGipmP7A2MRAAxnFJFosMcbZ6VDbgH8r6wA==", 2024-04-25T08:57:51.602 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:57:51.602 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T08:57:51.603 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T08:57:51.603 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:57:51.603 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:57:51.603 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:57:51.603 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:57:51.603 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.0", 2024-04-25T08:57:51.603 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpmVOjWGxAAFNmxeGi0h2D//pa6Ne3eQw==", 2024-04-25T08:57:51.603 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:57:51.604 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T08:57:51.604 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow r", 2024-04-25T08:57:51.604 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T08:57:51.604 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:57:51.604 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:57:51.604 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:57:51.604 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.1", 2024-04-25T08:57:51.604 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm24laJBAA9ijbDlFDTmF/ZZ4IhLfBVQ==", 2024-04-25T08:57:51.605 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:57:51.605 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T08:57:51.605 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-04-25T08:57:51.605 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-04-25T08:57:51.605 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-04-25T08:57:51.605 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:57:51.605 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:57:51.605 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:57:51.606 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.2", 2024-04-25T08:57:51.606 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm8b5UIRAACcOwDL630oJrXfp7s+fNTA==", 2024-04-25T08:57:51.606 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:57:51.606 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T08:57:51.606 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-04-25T08:57:51.606 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-04-25T08:57:51.606 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-04-25T08:57:51.606 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:57:51.607 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:57:51.607 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:57:51.607 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.3", 2024-04-25T08:57:51.607 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpmLsbBKRAAIKWtRdpuyCaW1LnKMWTMiw==", 2024-04-25T08:57:51.607 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:57:51.607 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T08:57:51.607 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-04-25T08:57:51.607 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-04-25T08:57:51.607 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-04-25T08:57:51.608 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:57:51.608 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:57:51.608 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:57:51.608 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.admin", 2024-04-25T08:57:51.608 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD7GSpmE9C8JRAAS263n6a4L2DXzMyS9o42NQ==", 2024-04-25T08:57:51.608 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:57:51.608 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-04-25T08:57:51.608 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T08:57:51.609 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T08:57:51.609 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:57:51.609 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:57:51.609 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:57:51.609 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:57:51.609 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.x", 2024-04-25T08:57:51.609 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm/uGhAhAAr0FaJ6SwZ8Su1MEqyZwNRA==", 2024-04-25T08:57:51.609 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:57:51.610 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-04-25T08:57:51.610 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2024-04-25T08:57:51.610 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:57:51.610 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:57:51.610 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:57:51.610 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:57:51.610 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.y", 2024-04-25T08:57:51.610 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD7GSpmPwwpOhAALeiQ/LKM2zgjE7yu16RQOg==", 2024-04-25T08:57:51.610 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:57:51.610 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-04-25T08:57:51.610 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2024-04-25T08:57:51.610 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:57:51.610 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:57:51.610 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:57:51.611 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-04-25T08:57:51.611 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T08:57:51.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.596+0000 7f6912730640 1 -- 172.21.15.43:0/2684070875 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f68f806b4d0 msgr2=0x7f68f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:51.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.596+0000 7f6912730640 1 --2- 172.21.15.43:0/2684070875 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f68f806b4d0 0x7f68f806d990 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f68fc009890 tx=0x7f68fc008040 comp rx=0 tx=0).stop 2024-04-25T08:57:51.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.596+0000 7f6912730640 1 -- 172.21.15.43:0/2684070875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f690c161390 msgr2=0x7f690c15ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:51.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.596+0000 7f6912730640 1 --2- 172.21.15.43:0/2684070875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f690c161390 0x7f690c15ce60 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f690400cbc0 tx=0x7f690400a430 comp rx=0 tx=0).stop 2024-04-25T08:57:51.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.596+0000 7f6912730640 1 -- 172.21.15.43:0/2684070875 shutdown_connections 2024-04-25T08:57:51.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.596+0000 7f6912730640 1 --2- 172.21.15.43:0/2684070875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f690c161390 0x7f690c15ce60 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:51.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.596+0000 7f6912730640 1 --2- 172.21.15.43:0/2684070875 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f690c0d02a0 0x7f690c15a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:51.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.596+0000 7f6912730640 1 --2- 172.21.15.43:0/2684070875 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f68f806b4d0 0x7f68f806d990 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:51.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.596+0000 7f6912730640 1 --2- 172.21.15.43:0/2684070875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f690c0ce5e0 0x7f690c155cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:51.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.596+0000 7f6912730640 1 -- 172.21.15.43:0/2684070875 >> 172.21.15.43:0/2684070875 conn(0x7f690c09cd20 msgr2=0x7f690c0d06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:51.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.596+0000 7f6912730640 1 -- 172.21.15.43:0/2684070875 shutdown_connections 2024-04-25T08:57:51.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.596+0000 7f6912730640 1 -- 172.21.15.43:0/2684070875 wait complete. 2024-04-25T08:57:51.614 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:57:51.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.726+0000 7f2be54cd640 1 Processor -- start 2024-04-25T08:57:51.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.726+0000 7f2be54cd640 1 -- start start 2024-04-25T08:57:51.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.727+0000 7f2be54cd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2be00cf250 0x7f2be00ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:51.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.727+0000 7f2be54cd640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2be00cf6e0 con 0x7f2be015ded0 2024-04-25T08:57:51.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.727+0000 7f2be54cd640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2be00cf9c0 con 0x7f2be006b240 2024-04-25T08:57:51.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.727+0000 7f2be54cd640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2be015e2b0 con 0x7f2be00cf250 2024-04-25T08:57:51.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.727+0000 7f2bdeffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f2be015ded0 0x7f2be006ab30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45308/0 (socket says 172.21.15.43:45308) 2024-04-25T08:57:51.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.727+0000 7f2bdeffd640 1 -- 172.21.15.43:0/370026424 learned_addr learned my addr 172.21.15.43:0/370026424 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:51.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.727+0000 7f2bd7fff640 1 --2- 172.21.15.43:0/370026424 >> v2:172.21.15.43:3300/0 conn(0x7f2be00cf250 0x7f2be00ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:51.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.727+0000 7f2bde7fc640 1 -- 172.21.15.43:0/370026424 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4190703116 0 0) 0x7f2be00cf6e0 con 0x7f2be015ded0 2024-04-25T08:57:51.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.727+0000 7f2bde7fc640 1 -- 172.21.15.43:0/370026424 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bcc003520 con 0x7f2be015ded0 2024-04-25T08:57:51.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.727+0000 7f2bd7fff640 1 -- 172.21.15.43:0/370026424 >> v1:172.21.15.193:6789/0 conn(0x7f2be006b240 legacy=0x7f2be00b4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:51.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.727+0000 7f2bd7fff640 1 -- 172.21.15.43:0/370026424 >> v1:172.21.15.43:6789/0 conn(0x7f2be015ded0 legacy=0x7f2be006ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:51.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.727+0000 7f2bd7fff640 1 -- 172.21.15.43:0/370026424 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be00cfca0 con 0x7f2be00cf250 2024-04-25T08:57:51.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.728+0000 7f2bd7fff640 1 --2- 172.21.15.43:0/370026424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2be00cf250 0x7f2be00ce740 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f2bc80097a0 tx=0x7f2bc802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6292be8196120b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:51.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.728+0000 7f2bde7fc640 1 -- 172.21.15.43:0/370026424 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8037070 con 0x7f2be00cf250 2024-04-25T08:57:51.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.728+0000 7f2bde7fc640 1 -- 172.21.15.43:0/370026424 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc8032440 con 0x7f2be00cf250 2024-04-25T08:57:51.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.728+0000 7f2bde7fc640 1 -- 172.21.15.43:0/370026424 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8032810 con 0x7f2be00cf250 2024-04-25T08:57:51.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.729+0000 7f2be54cd640 1 -- 172.21.15.43:0/370026424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2be00cf250 msgr2=0x7f2be00ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:51.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.729+0000 7f2be54cd640 1 --2- 172.21.15.43:0/370026424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2be00cf250 0x7f2be00ce740 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f2bc80097a0 tx=0x7f2bc802a230 comp rx=0 tx=0).stop 2024-04-25T08:57:51.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.729+0000 7f2be54cd640 1 -- 172.21.15.43:0/370026424 shutdown_connections 2024-04-25T08:57:51.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.729+0000 7f2be54cd640 1 --2- 172.21.15.43:0/370026424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2be00cf250 0x7f2be00ce740 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:51.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.729+0000 7f2be54cd640 1 -- 172.21.15.43:0/370026424 >> 172.21.15.43:0/370026424 conn(0x7f2be00b2490 msgr2=0x7f2be00b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:51.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.729+0000 7f2be54cd640 1 -- 172.21.15.43:0/370026424 shutdown_connections 2024-04-25T08:57:51.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.729+0000 7f2be54cd640 1 -- 172.21.15.43:0/370026424 wait complete. 2024-04-25T08:57:51.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.730+0000 7f2be54cd640 1 Processor -- start 2024-04-25T08:57:51.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.730+0000 7f2be54cd640 1 -- start start 2024-04-25T08:57:51.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.731+0000 7f2be54cd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2be006b240 0x7f2be015b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:51.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.731+0000 7f2be54cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2be015ded0 0x7f2be01562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:51.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.731+0000 7f2be54cd640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2be015e440 0x7f2be014c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:51.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.731+0000 7f2be54cd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be00cefd0 con 0x7f2be015ded0 2024-04-25T08:57:51.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.731+0000 7f2be54cd640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be00cee50 con 0x7f2be006b240 2024-04-25T08:57:51.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.731+0000 7f2be54cd640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2be00ad8e0 con 0x7f2be015e440 2024-04-25T08:57:51.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.731+0000 7f2bd7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2be015ded0 0x7f2be01562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:51.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.731+0000 7f2bd7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2be015ded0 0x7f2be01562a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49920/0 (socket says 172.21.15.43:49920) 2024-04-25T08:57:51.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.731+0000 7f2bd7fff640 1 -- 172.21.15.43:0/1083033530 learned_addr learned my addr 172.21.15.43:0/1083033530 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:51.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.731+0000 7f2bdeffd640 1 --2- 172.21.15.43:0/1083033530 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2be006b240 0x7f2be015b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:51.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.731+0000 7f2bd7fff640 1 -- 172.21.15.43:0/1083033530 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2be015e440 msgr2=0x7f2be014c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:51.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.731+0000 7f2bdf7fe640 1 --2- 172.21.15.43:0/1083033530 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2be015e440 0x7f2be014c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:51.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.731+0000 7f2bd7fff640 1 --2- 172.21.15.43:0/1083033530 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2be015e440 0x7f2be014c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:51.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.731+0000 7f2bd7fff640 1 -- 172.21.15.43:0/1083033530 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2be006b240 msgr2=0x7f2be015b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:51.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.731+0000 7f2bd7fff640 1 --2- 172.21.15.43:0/1083033530 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2be006b240 0x7f2be015b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:51.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.731+0000 7f2bd7fff640 1 -- 172.21.15.43:0/1083033530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2be015f170 con 0x7f2be015ded0 2024-04-25T08:57:51.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.731+0000 7f2bdeffd640 1 --2- 172.21.15.43:0/1083033530 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2be006b240 0x7f2be015b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:51.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.731+0000 7f2bd7fff640 1 --2- 172.21.15.43:0/1083033530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2be015ded0 0x7f2be01562a0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8035040 tx=0x7f2bc80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:51.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.732+0000 7f2bd77fe640 1 -- 172.21.15.43:0/1083033530 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8037040 con 0x7f2be015ded0 2024-04-25T08:57:51.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.732+0000 7f2bd77fe640 1 -- 172.21.15.43:0/1083033530 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc8030500 con 0x7f2be015ded0 2024-04-25T08:57:51.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.732+0000 7f2be54cd640 1 -- 172.21.15.43:0/1083033530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2be015e8d0 con 0x7f2be015ded0 2024-04-25T08:57:51.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.732+0000 7f2bd77fe640 1 -- 172.21.15.43:0/1083033530 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc8030910 con 0x7f2be015ded0 2024-04-25T08:57:51.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.732+0000 7f2be54cd640 1 -- 172.21.15.43:0/1083033530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be015ee90 con 0x7f2be015ded0 2024-04-25T08:57:51.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.733+0000 7f2be54cd640 1 -- 172.21.15.43:0/1083033530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2be0074ef0 con 0x7f2be015ded0 2024-04-25T08:57:51.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.733+0000 7f2bd77fe640 1 -- 172.21.15.43:0/1083033530 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2bc8044020 con 0x7f2be015ded0 2024-04-25T08:57:51.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.734+0000 7f2bd77fe640 1 --2- 172.21.15.43:0/1083033530 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2bc006b4d0 0x7f2bc006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:51.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.734+0000 7f2bd77fe640 1 -- 172.21.15.43:0/1083033530 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2bc80aa160 con 0x7f2be015ded0 2024-04-25T08:57:51.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.734+0000 7f2bdeffd640 1 --2- 172.21.15.43:0/1083033530 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2bc006b4d0 0x7f2bc006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:51.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.734+0000 7f2bdeffd640 1 --2- 172.21.15.43:0/1083033530 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2bc006b4d0 0x7f2bc006d990 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2bd00011e0 tx=0x7f2bd0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:51.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.738+0000 7f2bd77fe640 1 -- 172.21.15.43:0/1083033530 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2bc8079320 con 0x7f2be015ded0 2024-04-25T08:57:51.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.920+0000 7f2be54cd640 1 -- 172.21.15.43:0/1083033530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2be00b31e0 con 0x7f2be015ded0 2024-04-25T08:57:51.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.921+0000 7f2bd77fe640 1 -- 172.21.15.43:0/1083033530 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+12855 (secure 0 0 0) 0x7f2bc8078c20 con 0x7f2be015ded0 2024-04-25T08:57:51.911 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:57:51.911 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":54,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:57:51.206224+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":44,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:57:51.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.923+0000 7f2be54cd640 1 -- 172.21.15.43:0/1083033530 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2bc006b4d0 msgr2=0x7f2bc006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:51.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.923+0000 7f2be54cd640 1 --2- 172.21.15.43:0/1083033530 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2bc006b4d0 0x7f2bc006d990 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f2bd00011e0 tx=0x7f2bd0009040 comp rx=0 tx=0).stop 2024-04-25T08:57:51.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.923+0000 7f2be54cd640 1 -- 172.21.15.43:0/1083033530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2be015ded0 msgr2=0x7f2be01562a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:51.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.923+0000 7f2be54cd640 1 --2- 172.21.15.43:0/1083033530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2be015ded0 0x7f2be01562a0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8035040 tx=0x7f2bc80046c0 comp rx=0 tx=0).stop 2024-04-25T08:57:51.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.923+0000 7f2be54cd640 1 -- 172.21.15.43:0/1083033530 shutdown_connections 2024-04-25T08:57:51.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.923+0000 7f2be54cd640 1 --2- 172.21.15.43:0/1083033530 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2be015e440 0x7f2be014c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:51.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.923+0000 7f2be54cd640 1 --2- 172.21.15.43:0/1083033530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2be015ded0 0x7f2be01562a0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:51.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.923+0000 7f2be54cd640 1 --2- 172.21.15.43:0/1083033530 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2bc006b4d0 0x7f2bc006d990 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:51.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.923+0000 7f2be54cd640 1 --2- 172.21.15.43:0/1083033530 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2be006b240 0x7f2be015b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:51.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.923+0000 7f2be54cd640 1 -- 172.21.15.43:0/1083033530 >> 172.21.15.43:0/1083033530 conn(0x7f2be00b2490 msgr2=0x7f2be00aeeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:51.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.923+0000 7f2be54cd640 1 -- 172.21.15.43:0/1083033530 shutdown_connections 2024-04-25T08:57:51.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:51.923+0000 7f2be54cd640 1 -- 172.21.15.43:0/1083033530 wait complete. 2024-04-25T08:57:51.924 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:57:51.924 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:57:52.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.038+0000 7f5f9841f640 1 Processor -- start 2024-04-25T08:57:52.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.038+0000 7f5f9841f640 1 -- start start 2024-04-25T08:57:52.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.039+0000 7f5f9841f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5f900d04a0 0x7f5f9015ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:52.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.039+0000 7f5f9841f640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f5f9015dec0 0x7f5f900d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:52.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.039+0000 7f5f9841f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5f900ce8c0 0x7f5f901617a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:52.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.039+0000 7f5f9841f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f900d0dc0 con 0x7f5f900ce8c0 2024-04-25T08:57:52.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.039+0000 7f5f9841f640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f900cf110 con 0x7f5f9015dec0 2024-04-25T08:57:52.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.039+0000 7f5f9841f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f900cf3f0 con 0x7f5f900d04a0 2024-04-25T08:57:52.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.039+0000 7f5f969bc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5f900ce8c0 0x7f5f901617a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:52.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.039+0000 7f5f969bc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5f900ce8c0 0x7f5f901617a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49922/0 (socket says 172.21.15.43:49922) 2024-04-25T08:57:52.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.039+0000 7f5f969bc640 1 -- 172.21.15.43:0/1723414685 learned_addr learned my addr 172.21.15.43:0/1723414685 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:52.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.040+0000 7f5f959ba640 1 --2- 172.21.15.43:0/1723414685 >> v2:172.21.15.193:3300/0 conn(0x7f5f9015dec0 0x7f5f900d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:52.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.040+0000 7f5f961bb640 1 --2- 172.21.15.43:0/1723414685 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5f900d04a0 0x7f5f9015ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:52.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.040+0000 7f5f969bc640 1 -- 172.21.15.43:0/1723414685 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5f900d04a0 msgr2=0x7f5f9015ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:52.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.040+0000 7f5f969bc640 1 --2- 172.21.15.43:0/1723414685 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5f900d04a0 0x7f5f9015ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:52.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.040+0000 7f5f969bc640 1 -- 172.21.15.43:0/1723414685 >> v2:172.21.15.193:3300/0 conn(0x7f5f9015dec0 msgr2=0x7f5f900d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:52.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.040+0000 7f5f969bc640 1 --2- 172.21.15.43:0/1723414685 >> v2:172.21.15.193:3300/0 conn(0x7f5f9015dec0 0x7f5f900d0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:52.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.040+0000 7f5f969bc640 1 -- 172.21.15.43:0/1723414685 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f900cf6d0 con 0x7f5f900ce8c0 2024-04-25T08:57:52.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.040+0000 7f5f969bc640 1 --2- 172.21.15.43:0/1723414685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f900ce8c0 0x7f5f901617a0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f5f900b66c0 tx=0x7f5f8c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9ee5557f4cd117 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:52.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.041+0000 7f5f951b9640 1 -- 172.21.15.43:0/1723414685 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c036070 con 0x7f5f900ce8c0 2024-04-25T08:57:52.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.041+0000 7f5f951b9640 1 -- 172.21.15.43:0/1723414685 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f8c031600 con 0x7f5f900ce8c0 2024-04-25T08:57:52.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.041+0000 7f5f951b9640 1 -- 172.21.15.43:0/1723414685 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c0319d0 con 0x7f5f900ce8c0 2024-04-25T08:57:52.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.041+0000 7f5f9841f640 1 -- 172.21.15.43:0/1723414685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f900ce8c0 msgr2=0x7f5f901617a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:52.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.041+0000 7f5f9841f640 1 --2- 172.21.15.43:0/1723414685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f900ce8c0 0x7f5f901617a0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f5f900b66c0 tx=0x7f5f8c02a6a0 comp rx=0 tx=0).stop 2024-04-25T08:57:52.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.041+0000 7f5f9841f640 1 -- 172.21.15.43:0/1723414685 shutdown_connections 2024-04-25T08:57:52.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.041+0000 7f5f9841f640 1 --2- 172.21.15.43:0/1723414685 >> v2:172.21.15.193:3300/0 conn(0x7f5f9015dec0 0x7f5f900d0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:52.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.041+0000 7f5f9841f640 1 --2- 172.21.15.43:0/1723414685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f900ce8c0 0x7f5f901617a0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:52.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.041+0000 7f5f9841f640 1 --2- 172.21.15.43:0/1723414685 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5f900d04a0 0x7f5f9015ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:52.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.041+0000 7f5f9841f640 1 -- 172.21.15.43:0/1723414685 >> 172.21.15.43:0/1723414685 conn(0x7f5f9009cd40 msgr2=0x7f5f900d3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:52.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.041+0000 7f5f9841f640 1 -- 172.21.15.43:0/1723414685 shutdown_connections 2024-04-25T08:57:52.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.041+0000 7f5f9841f640 1 -- 172.21.15.43:0/1723414685 wait complete. 2024-04-25T08:57:52.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.042+0000 7f5f9841f640 1 Processor -- start 2024-04-25T08:57:52.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.043+0000 7f5f9841f640 1 -- start start 2024-04-25T08:57:52.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.043+0000 7f5f9841f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f900ce8c0 0x7f5f9009d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:52.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.043+0000 7f5f9841f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f900d04a0 0x7f5f9009f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:52.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.043+0000 7f5f9841f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5f9015dec0 0x7f5f9009f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:52.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.043+0000 7f5f9841f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f900cecf0 con 0x7f5f900d04a0 2024-04-25T08:57:52.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.043+0000 7f5f9841f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f900cfc90 con 0x7f5f900ce8c0 2024-04-25T08:57:52.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.043+0000 7f5f9841f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f900cee70 con 0x7f5f9015dec0 2024-04-25T08:57:52.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.043+0000 7f5f959ba640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f900d04a0 0x7f5f9009f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:52.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.043+0000 7f5f959ba640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f900d04a0 0x7f5f9009f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49934/0 (socket says 172.21.15.43:49934) 2024-04-25T08:57:52.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.043+0000 7f5f959ba640 1 -- 172.21.15.43:0/2294072454 learned_addr learned my addr 172.21.15.43:0/2294072454 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:52.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.043+0000 7f5f961bb640 1 --2- 172.21.15.43:0/2294072454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f900ce8c0 0x7f5f9009d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:52.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.043+0000 7f5f969bc640 1 --2- 172.21.15.43:0/2294072454 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5f9015dec0 0x7f5f9009f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:52.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.044+0000 7f5f959ba640 1 -- 172.21.15.43:0/2294072454 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5f9015dec0 msgr2=0x7f5f9009f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:52.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.044+0000 7f5f959ba640 1 --2- 172.21.15.43:0/2294072454 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5f9015dec0 0x7f5f9009f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:52.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.044+0000 7f5f959ba640 1 -- 172.21.15.43:0/2294072454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f900ce8c0 msgr2=0x7f5f9009d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:52.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.044+0000 7f5f959ba640 1 --2- 172.21.15.43:0/2294072454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f900ce8c0 0x7f5f9009d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:52.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.044+0000 7f5f959ba640 1 -- 172.21.15.43:0/2294072454 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f9009dcf0 con 0x7f5f900d04a0 2024-04-25T08:57:52.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.044+0000 7f5f959ba640 1 --2- 172.21.15.43:0/2294072454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f900d04a0 0x7f5f9009f420 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f5f8000d7d0 tx=0x7f5f8000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:52.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.044+0000 7f5f86ffd640 1 -- 172.21.15.43:0/2294072454 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f80004020 con 0x7f5f900d04a0 2024-04-25T08:57:52.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.044+0000 7f5f86ffd640 1 -- 172.21.15.43:0/2294072454 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f800041e0 con 0x7f5f900d04a0 2024-04-25T08:57:52.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.044+0000 7f5f86ffd640 1 -- 172.21.15.43:0/2294072454 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f800043d0 con 0x7f5f900d04a0 2024-04-25T08:57:52.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.044+0000 7f5f9841f640 1 -- 172.21.15.43:0/2294072454 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f900a01f0 con 0x7f5f900d04a0 2024-04-25T08:57:52.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.044+0000 7f5f9841f640 1 -- 172.21.15.43:0/2294072454 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f900a07b0 con 0x7f5f900d04a0 2024-04-25T08:57:52.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.045+0000 7f5f9841f640 1 -- 172.21.15.43:0/2294072454 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f90074f60 con 0x7f5f900d04a0 2024-04-25T08:57:52.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.046+0000 7f5f86ffd640 1 -- 172.21.15.43:0/2294072454 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5f8001b020 con 0x7f5f900d04a0 2024-04-25T08:57:52.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.046+0000 7f5f86ffd640 1 --2- 172.21.15.43:0/2294072454 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5f7406b4d0 0x7f5f7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:52.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.046+0000 7f5f86ffd640 1 -- 172.21.15.43:0/2294072454 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5f80089d50 con 0x7f5f900d04a0 2024-04-25T08:57:52.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.046+0000 7f5f961bb640 1 --2- 172.21.15.43:0/2294072454 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5f7406b4d0 0x7f5f7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:52.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.047+0000 7f5f961bb640 1 --2- 172.21.15.43:0/2294072454 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5f7406b4d0 0x7f5f7406d990 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5f88006fd0 tx=0x7f5f88008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:52.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.050+0000 7f5f86ffd640 1 -- 172.21.15.43:0/2294072454 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5f8008d030 con 0x7f5f900d04a0 2024-04-25T08:57:52.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.232+0000 7f5f9841f640 1 -- 172.21.15.43:0/2294072454 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5f9008d7e0 con 0x7f5f900d04a0 2024-04-25T08:57:52.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.232+0000 7f5f86ffd640 1 -- 172.21.15.43:0/2294072454 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5f80058e90 con 0x7f5f900d04a0 2024-04-25T08:57:52.222 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T08:57:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.234+0000 7f5f9841f640 1 -- 172.21.15.43:0/2294072454 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5f7406b4d0 msgr2=0x7f5f7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:52.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.234+0000 7f5f9841f640 1 --2- 172.21.15.43:0/2294072454 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5f7406b4d0 0x7f5f7406d990 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5f88006fd0 tx=0x7f5f88008040 comp rx=0 tx=0).stop 2024-04-25T08:57:52.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.234+0000 7f5f9841f640 1 -- 172.21.15.43:0/2294072454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f900d04a0 msgr2=0x7f5f9009f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:52.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.234+0000 7f5f9841f640 1 --2- 172.21.15.43:0/2294072454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f900d04a0 0x7f5f9009f420 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f5f8000d7d0 tx=0x7f5f8000dc90 comp rx=0 tx=0).stop 2024-04-25T08:57:52.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.234+0000 7f5f9841f640 1 -- 172.21.15.43:0/2294072454 shutdown_connections 2024-04-25T08:57:52.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.234+0000 7f5f9841f640 1 --2- 172.21.15.43:0/2294072454 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5f9015dec0 0x7f5f9009f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:52.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.234+0000 7f5f9841f640 1 --2- 172.21.15.43:0/2294072454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5f900d04a0 0x7f5f9009f420 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:52.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.234+0000 7f5f9841f640 1 --2- 172.21.15.43:0/2294072454 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5f7406b4d0 0x7f5f7406d990 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:52.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.234+0000 7f5f9841f640 1 --2- 172.21.15.43:0/2294072454 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5f900ce8c0 0x7f5f9009d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:52.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.234+0000 7f5f9841f640 1 -- 172.21.15.43:0/2294072454 >> 172.21.15.43:0/2294072454 conn(0x7f5f9009cd40 msgr2=0x7f5f90160ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:52.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.234+0000 7f5f9841f640 1 -- 172.21.15.43:0/2294072454 shutdown_connections 2024-04-25T08:57:52.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.234+0000 7f5f9841f640 1 -- 172.21.15.43:0/2294072454 wait complete. 2024-04-25T08:57:52.234 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T08:57:52.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.346+0000 7fbd37012640 1 Processor -- start 2024-04-25T08:57:52.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.346+0000 7fbd37012640 1 -- start start 2024-04-25T08:57:52.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.346+0000 7fbd37012640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fbd300cff70 0x7fbd300cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:52.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.346+0000 7fbd37012640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd300ccf10 con 0x7fbd3015f180 2024-04-25T08:57:52.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.346+0000 7fbd37012640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd300cd1f0 con 0x7fbd300b23f0 2024-04-25T08:57:52.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.346+0000 7fbd37012640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd300cccc0 con 0x7fbd300cff70 2024-04-25T08:57:52.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.347+0000 7fbd34dae640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fbd3015f180 0x7fbd300cf860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45316/0 (socket says 172.21.15.43:45316) 2024-04-25T08:57:52.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.347+0000 7fbd34dae640 1 -- 172.21.15.43:0/3679015891 learned_addr learned my addr 172.21.15.43:0/3679015891 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:52.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.347+0000 7fbd2ffff640 1 --2- 172.21.15.43:0/3679015891 >> v2:172.21.15.193:3300/0 conn(0x7fbd300cff70 0x7fbd300cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:52.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.347+0000 7fbd2f7fe640 1 -- 172.21.15.43:0/3679015891 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 491246421 0 0) 0x7fbd300ccf10 con 0x7fbd3015f180 2024-04-25T08:57:52.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.347+0000 7fbd2f7fe640 1 -- 172.21.15.43:0/3679015891 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd20003520 con 0x7fbd3015f180 2024-04-25T08:57:52.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.347+0000 7fbd2f7fe640 1 -- 172.21.15.43:0/3679015891 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3670867542 0 0) 0x7fbd300cd1f0 con 0x7fbd300b23f0 2024-04-25T08:57:52.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.347+0000 7fbd2f7fe640 1 -- 172.21.15.43:0/3679015891 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd300ccf10 con 0x7fbd300b23f0 2024-04-25T08:57:52.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.347+0000 7fbd2ffff640 1 -- 172.21.15.43:0/3679015891 >> v1:172.21.15.193:6789/0 conn(0x7fbd300b23f0 legacy=0x7fbd300ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:52.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.347+0000 7fbd2ffff640 1 -- 172.21.15.43:0/3679015891 >> v1:172.21.15.43:6789/0 conn(0x7fbd3015f180 legacy=0x7fbd300cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:52.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.347+0000 7fbd2f7fe640 1 -- 172.21.15.43:0/3679015891 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1342859019 0 0) 0x7fbd20003520 con 0x7fbd3015f180 2024-04-25T08:57:52.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.347+0000 7fbd2ffff640 1 -- 172.21.15.43:0/3679015891 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd300cd4d0 con 0x7fbd300cff70 2024-04-25T08:57:52.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.348+0000 7fbd2ffff640 1 --2- 172.21.15.43:0/3679015891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd300cff70 0x7fbd300cdcd0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c0097a0 tx=0x7fbd1c02a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=85adc63010a82e8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:52.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.348+0000 7fbd2f7fe640 1 -- 172.21.15.43:0/3679015891 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c037070 con 0x7fbd300cff70 2024-04-25T08:57:52.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.348+0000 7fbd2f7fe640 1 -- 172.21.15.43:0/3679015891 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd1c02abc0 con 0x7fbd300cff70 2024-04-25T08:57:52.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.348+0000 7fbd2f7fe640 1 -- 172.21.15.43:0/3679015891 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd24001670 con 0x7fbd300cff70 2024-04-25T08:57:52.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.349+0000 7fbd37012640 1 -- 172.21.15.43:0/3679015891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd300cff70 msgr2=0x7fbd300cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:52.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.349+0000 7fbd37012640 1 --2- 172.21.15.43:0/3679015891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd300cff70 0x7fbd300cdcd0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c0097a0 tx=0x7fbd1c02a130 comp rx=0 tx=0).stop 2024-04-25T08:57:52.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.349+0000 7fbd37012640 1 -- 172.21.15.43:0/3679015891 shutdown_connections 2024-04-25T08:57:52.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.349+0000 7fbd37012640 1 --2- 172.21.15.43:0/3679015891 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd300cff70 0x7fbd300cdcd0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:52.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.349+0000 7fbd37012640 1 -- 172.21.15.43:0/3679015891 >> 172.21.15.43:0/3679015891 conn(0x7fbd300d17f0 msgr2=0x7fbd300d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:52.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.349+0000 7fbd37012640 1 -- 172.21.15.43:0/3679015891 shutdown_connections 2024-04-25T08:57:52.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.349+0000 7fbd37012640 1 -- 172.21.15.43:0/3679015891 wait complete. 2024-04-25T08:57:52.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.350+0000 7fbd37012640 1 Processor -- start 2024-04-25T08:57:52.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.350+0000 7fbd37012640 1 -- start start 2024-04-25T08:57:52.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.351+0000 7fbd37012640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd300b23f0 0x7fbd300c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:52.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.351+0000 7fbd37012640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd300cff70 0x7fbd300c7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:52.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.351+0000 7fbd37012640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd3015f180 0x7fbd300c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:52.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.351+0000 7fbd37012640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd300d23f0 con 0x7fbd3015f180 2024-04-25T08:57:52.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.351+0000 7fbd37012640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd300cd840 con 0x7fbd300cff70 2024-04-25T08:57:52.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.351+0000 7fbd37012640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd300d2570 con 0x7fbd300b23f0 2024-04-25T08:57:52.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.351+0000 7fbd355af640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd3015f180 0x7fbd300c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:52.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.351+0000 7fbd355af640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd3015f180 0x7fbd300c6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49936/0 (socket says 172.21.15.43:49936) 2024-04-25T08:57:52.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.351+0000 7fbd355af640 1 -- 172.21.15.43:0/1267052060 learned_addr learned my addr 172.21.15.43:0/1267052060 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:52.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.351+0000 7fbd34dae640 1 --2- 172.21.15.43:0/1267052060 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd300b23f0 0x7fbd300c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:52.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.351+0000 7fbd2ffff640 1 --2- 172.21.15.43:0/1267052060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd300cff70 0x7fbd300c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:52.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.351+0000 7fbd355af640 1 -- 172.21.15.43:0/1267052060 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd300b23f0 msgr2=0x7fbd300c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:52.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.351+0000 7fbd355af640 1 --2- 172.21.15.43:0/1267052060 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd300b23f0 0x7fbd300c52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:52.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.351+0000 7fbd355af640 1 -- 172.21.15.43:0/1267052060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd300cff70 msgr2=0x7fbd300c7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:52.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.351+0000 7fbd355af640 1 --2- 172.21.15.43:0/1267052060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd300cff70 0x7fbd300c7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:52.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.351+0000 7fbd355af640 1 -- 172.21.15.43:0/1267052060 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd3006d310 con 0x7fbd3015f180 2024-04-25T08:57:52.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.351+0000 7fbd34dae640 1 --2- 172.21.15.43:0/1267052060 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd300b23f0 0x7fbd300c52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:52.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.351+0000 7fbd2ffff640 1 --2- 172.21.15.43:0/1267052060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd300cff70 0x7fbd300c7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:52.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.351+0000 7fbd355af640 1 --2- 172.21.15.43:0/1267052060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd3015f180 0x7fbd300c6a70 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fbd2800c790 tx=0x7fbd2800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:52.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.352+0000 7fbd2d7fa640 1 -- 172.21.15.43:0/1267052060 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd28003b70 con 0x7fbd3015f180 2024-04-25T08:57:52.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.352+0000 7fbd2d7fa640 1 -- 172.21.15.43:0/1267052060 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd28003d10 con 0x7fbd3015f180 2024-04-25T08:57:52.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.352+0000 7fbd37012640 1 -- 172.21.15.43:0/1267052060 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd3006d5f0 con 0x7fbd3015f180 2024-04-25T08:57:52.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.352+0000 7fbd2d7fa640 1 -- 172.21.15.43:0/1267052060 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2800aa50 con 0x7fbd3015f180 2024-04-25T08:57:52.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.352+0000 7fbd37012640 1 -- 172.21.15.43:0/1267052060 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd300c6fb0 con 0x7fbd3015f180 2024-04-25T08:57:52.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.353+0000 7fbd37012640 1 -- 172.21.15.43:0/1267052060 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd30074ef0 con 0x7fbd3015f180 2024-04-25T08:57:52.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.353+0000 7fbd2d7fa640 1 -- 172.21.15.43:0/1267052060 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbd2800abf0 con 0x7fbd3015f180 2024-04-25T08:57:52.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.354+0000 7fbd2d7fa640 1 --2- 172.21.15.43:0/1267052060 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbd1406b4d0 0x7fbd1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:52.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.354+0000 7fbd2d7fa640 1 -- 172.21.15.43:0/1267052060 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbd28089e10 con 0x7fbd3015f180 2024-04-25T08:57:52.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.354+0000 7fbd34dae640 1 --2- 172.21.15.43:0/1267052060 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbd1406b4d0 0x7fbd1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:52.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.355+0000 7fbd34dae640 1 --2- 172.21.15.43:0/1267052060 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbd1406b4d0 0x7fbd1406d990 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbd240017e0 tx=0x7fbd24009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:52.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.358+0000 7fbd2d7fa640 1 -- 172.21.15.43:0/1267052060 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbd28059070 con 0x7fbd3015f180 2024-04-25T08:57:52.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.542+0000 7fbd37012640 1 -- 172.21.15.43:0/1267052060 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fbd300cf050 con 0x7fbd3015f180 2024-04-25T08:57:52.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.542+0000 7fbd2d7fa640 1 -- 172.21.15.43:0/1267052060 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v54) v1 ==== 84+0+65 (secure 0 0 0) 0x7fbd28058970 con 0x7fbd3015f180 2024-04-25T08:57:52.531 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:57:52.532 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2024-04-25T08:57:52.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:57:52.532 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 1, 2024-04-25T08:57:52.532 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": ".mgr" 2024-04-25T08:57:52.532 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:57:52.532 INFO:teuthology.orchestra.run.smithi043.stdout:] 2024-04-25T08:57:52.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.544+0000 7fbd37012640 1 -- 172.21.15.43:0/1267052060 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbd1406b4d0 msgr2=0x7fbd1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:52.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.544+0000 7fbd37012640 1 --2- 172.21.15.43:0/1267052060 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbd1406b4d0 0x7fbd1406d990 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fbd240017e0 tx=0x7fbd24009040 comp rx=0 tx=0).stop 2024-04-25T08:57:52.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.544+0000 7fbd37012640 1 -- 172.21.15.43:0/1267052060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd3015f180 msgr2=0x7fbd300c6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:52.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.544+0000 7fbd37012640 1 --2- 172.21.15.43:0/1267052060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd3015f180 0x7fbd300c6a70 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fbd2800c790 tx=0x7fbd2800cc50 comp rx=0 tx=0).stop 2024-04-25T08:57:52.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.544+0000 7fbd37012640 1 -- 172.21.15.43:0/1267052060 shutdown_connections 2024-04-25T08:57:52.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.544+0000 7fbd37012640 1 --2- 172.21.15.43:0/1267052060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd3015f180 0x7fbd300c6a70 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:52.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.544+0000 7fbd37012640 1 --2- 172.21.15.43:0/1267052060 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbd1406b4d0 0x7fbd1406d990 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:52.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.544+0000 7fbd37012640 1 --2- 172.21.15.43:0/1267052060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd300cff70 0x7fbd300c7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:52.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.544+0000 7fbd37012640 1 --2- 172.21.15.43:0/1267052060 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd300b23f0 0x7fbd300c52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:52.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.544+0000 7fbd37012640 1 -- 172.21.15.43:0/1267052060 >> 172.21.15.43:0/1267052060 conn(0x7fbd300d17f0 msgr2=0x7fbd300b4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:52.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.544+0000 7fbd37012640 1 -- 172.21.15.43:0/1267052060 shutdown_connections 2024-04-25T08:57:52.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.544+0000 7fbd37012640 1 -- 172.21.15.43:0/1267052060 wait complete. 2024-04-25T08:57:52.544 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T08:57:52.545 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T08:57:52.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.661+0000 7fcb0ea15640 1 Processor -- start 2024-04-25T08:57:52.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.661+0000 7fcb0ea15640 1 -- start start 2024-04-25T08:57:52.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.661+0000 7fcb0ea15640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fcb080d3760 0x7fcb080d3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:52.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.661+0000 7fcb0ea15640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcb080b06c0 0x7fcb080b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:52.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.661+0000 7fcb0ea15640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb080d4130 con 0x7fcb0815de80 2024-04-25T08:57:52.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.661+0000 7fcb0ea15640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb0815e260 con 0x7fcb080d3760 2024-04-25T08:57:52.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.661+0000 7fcb0ea15640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb080b52a0 con 0x7fcb080b06c0 2024-04-25T08:57:52.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.661+0000 7fcb077fe640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fcb080d3760 0x7fcb080d3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:52.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.661+0000 7fcb0cfb2640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcb080b06c0 0x7fcb080b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:52.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.661+0000 7fcb07fff640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7fcb0815de80 0x7fcb080b4b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:40800/0 (socket says 172.21.15.43:40800) 2024-04-25T08:57:52.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.661+0000 7fcb07fff640 1 -- 172.21.15.43:0/2237585789 learned_addr learned my addr 172.21.15.43:0/2237585789 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:52.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.661+0000 7fcb0cfb2640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcb080b06c0 0x7fcb080b0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:37020/0 (socket says 172.21.15.43:37020) 2024-04-25T08:57:52.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.661+0000 7fcb0cfb2640 1 -- 172.21.15.43:0/2237585789 >> v2:172.21.15.193:3300/0 conn(0x7fcb080d3760 msgr2=0x7fcb080d3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:52.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.661+0000 7fcb0cfb2640 1 --2- 172.21.15.43:0/2237585789 >> v2:172.21.15.193:3300/0 conn(0x7fcb080d3760 0x7fcb080d3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:52.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.662+0000 7fcb0cfb2640 1 -- 172.21.15.43:0/2237585789 >> v1:172.21.15.193:6789/0 conn(0x7fcb0815de80 legacy=0x7fcb080b4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:52.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.662+0000 7fcb0cfb2640 1 -- 172.21.15.43:0/2237585789 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb08161970 con 0x7fcb080b06c0 2024-04-25T08:57:52.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.662+0000 7fcb06ffd640 1 -- 172.21.15.43:0/2237585789 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 118247181 0 0) 0x7fcb080d4130 con 0x7fcb0815de80 2024-04-25T08:57:52.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.662+0000 7fcb0cfb2640 1 --2- 172.21.15.43:0/2237585789 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcb080b06c0 0x7fcb080b0b50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcb00009dc0 tx=0x7fcb0002a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae99b71da125c750 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:52.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.662+0000 7fcb06ffd640 1 -- 172.21.15.43:0/2237585789 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb00007740 con 0x7fcb080b06c0 2024-04-25T08:57:52.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.663+0000 7fcb06ffd640 1 -- 172.21.15.43:0/2237585789 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb000078e0 con 0x7fcb080b06c0 2024-04-25T08:57:52.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.663+0000 7fcb06ffd640 1 -- 172.21.15.43:0/2237585789 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb00007cb0 con 0x7fcb080b06c0 2024-04-25T08:57:52.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.663+0000 7fcb0ea15640 1 -- 172.21.15.43:0/2237585789 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcb080b06c0 msgr2=0x7fcb080b0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:52.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.663+0000 7fcb0ea15640 1 --2- 172.21.15.43:0/2237585789 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcb080b06c0 0x7fcb080b0b50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcb00009dc0 tx=0x7fcb0002a5f0 comp rx=0 tx=0).stop 2024-04-25T08:57:52.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.663+0000 7fcb0ea15640 1 -- 172.21.15.43:0/2237585789 shutdown_connections 2024-04-25T08:57:52.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.663+0000 7fcb0ea15640 1 --2- 172.21.15.43:0/2237585789 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcb080b06c0 0x7fcb080b0b50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:52.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.663+0000 7fcb0ea15640 1 --2- 172.21.15.43:0/2237585789 >> v2:172.21.15.193:3300/0 conn(0x7fcb080d3760 0x7fcb080d3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:52.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.663+0000 7fcb0ea15640 1 -- 172.21.15.43:0/2237585789 >> 172.21.15.43:0/2237585789 conn(0x7fcb0808d460 msgr2=0x7fcb0809cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:52.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.663+0000 7fcb0ea15640 1 -- 172.21.15.43:0/2237585789 shutdown_connections 2024-04-25T08:57:52.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.663+0000 7fcb0ea15640 1 -- 172.21.15.43:0/2237585789 wait complete. 2024-04-25T08:57:52.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.664+0000 7fcb0ea15640 1 Processor -- start 2024-04-25T08:57:52.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.664+0000 7fcb0ea15640 1 -- start start 2024-04-25T08:57:52.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.665+0000 7fcb0ea15640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcb080b06c0 0x7fcb080c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:52.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.665+0000 7fcb0ea15640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb080d3760 0x7fcb080cf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:52.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.665+0000 7fcb0ea15640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb0815de80 0x7fcb080cff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:52.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.665+0000 7fcb0ea15640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb08162210 con 0x7fcb080d3760 2024-04-25T08:57:52.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.665+0000 7fcb0ea15640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb08161f30 con 0x7fcb0815de80 2024-04-25T08:57:52.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.665+0000 7fcb0ea15640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb081624f0 con 0x7fcb080b06c0 2024-04-25T08:57:52.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.665+0000 7fcb077fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb080d3760 0x7fcb080cf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:52.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.665+0000 7fcb077fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb080d3760 0x7fcb080cf9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49950/0 (socket says 172.21.15.43:49950) 2024-04-25T08:57:52.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.665+0000 7fcb077fe640 1 -- 172.21.15.43:0/1031478093 learned_addr learned my addr 172.21.15.43:0/1031478093 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:52.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.665+0000 7fcb0cfb2640 1 --2- 172.21.15.43:0/1031478093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb0815de80 0x7fcb080cff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:52.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.665+0000 7fcb07fff640 1 --2- 172.21.15.43:0/1031478093 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcb080b06c0 0x7fcb080c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:52.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.666+0000 7fcb077fe640 1 -- 172.21.15.43:0/1031478093 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcb080b06c0 msgr2=0x7fcb080c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:52.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.666+0000 7fcb077fe640 1 --2- 172.21.15.43:0/1031478093 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcb080b06c0 0x7fcb080c9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:52.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.666+0000 7fcb077fe640 1 -- 172.21.15.43:0/1031478093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb0815de80 msgr2=0x7fcb080cff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:52.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.666+0000 7fcb077fe640 1 --2- 172.21.15.43:0/1031478093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb0815de80 0x7fcb080cff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:52.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.666+0000 7fcb077fe640 1 -- 172.21.15.43:0/1031478093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb08161140 con 0x7fcb080d3760 2024-04-25T08:57:52.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.666+0000 7fcb077fe640 1 --2- 172.21.15.43:0/1031478093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb080d3760 0x7fcb080cf9c0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fcaf000e930 tx=0x7fcaf000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:52.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.666+0000 7fcb04ff9640 1 -- 172.21.15.43:0/1031478093 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf0017070 con 0x7fcb080d3760 2024-04-25T08:57:52.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.666+0000 7fcb04ff9640 1 -- 172.21.15.43:0/1031478093 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaf000ca00 con 0x7fcb080d3760 2024-04-25T08:57:52.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.666+0000 7fcb0ea15640 1 -- 172.21.15.43:0/1031478093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb081600a0 con 0x7fcb080d3760 2024-04-25T08:57:52.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.666+0000 7fcb04ff9640 1 -- 172.21.15.43:0/1031478093 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaf000cdf0 con 0x7fcb080d3760 2024-04-25T08:57:52.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.666+0000 7fcb0ea15640 1 -- 172.21.15.43:0/1031478093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb080cc160 con 0x7fcb080d3760 2024-04-25T08:57:52.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.667+0000 7fcb0ea15640 1 -- 172.21.15.43:0/1031478093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb08068a40 con 0x7fcb080d3760 2024-04-25T08:57:52.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.667+0000 7fcb04ff9640 1 -- 172.21.15.43:0/1031478093 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcaf001a070 con 0x7fcb080d3760 2024-04-25T08:57:52.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.668+0000 7fcb04ff9640 1 --2- 172.21.15.43:0/1031478093 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcae806b4d0 0x7fcae806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:52.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.668+0000 7fcb04ff9640 1 -- 172.21.15.43:0/1031478093 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcaf0089d90 con 0x7fcb080d3760 2024-04-25T08:57:52.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.668+0000 7fcb07fff640 1 --2- 172.21.15.43:0/1031478093 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcae806b4d0 0x7fcae806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:52.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.669+0000 7fcb07fff640 1 --2- 172.21.15.43:0/1031478093 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcae806b4d0 0x7fcae806d990 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8000f60 tx=0x7fcaf8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:52.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.673+0000 7fcb04ff9640 1 -- 172.21.15.43:0/1031478093 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcb08161140 con 0x7fcb080d3760 2024-04-25T08:57:52.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:52.852+0000 7fcb0ea15640 1 -- 172.21.15.43:0/1031478093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fcb080b3730 con 0x7fcb080d3760 2024-04-25T08:57:53.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.210+0000 7fcb04ff9640 1 -- 172.21.15.43:0/1031478093 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v55) v1 ==== 138+0+0 (secure 0 0 0) 0x7fcaf0058a60 con 0x7fcb080d3760 2024-04-25T08:57:53.199 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' created 2024-04-25T08:57:53.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.212+0000 7fcb0ea15640 1 -- 172.21.15.43:0/1031478093 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcae806b4d0 msgr2=0x7fcae806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:53.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.212+0000 7fcb0ea15640 1 --2- 172.21.15.43:0/1031478093 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcae806b4d0 0x7fcae806d990 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8000f60 tx=0x7fcaf8009040 comp rx=0 tx=0).stop 2024-04-25T08:57:53.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.212+0000 7fcb0ea15640 1 -- 172.21.15.43:0/1031478093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb080d3760 msgr2=0x7fcb080cf9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:53.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.212+0000 7fcb0ea15640 1 --2- 172.21.15.43:0/1031478093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb080d3760 0x7fcb080cf9c0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fcaf000e930 tx=0x7fcaf000edf0 comp rx=0 tx=0).stop 2024-04-25T08:57:53.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.212+0000 7fcb0ea15640 1 -- 172.21.15.43:0/1031478093 shutdown_connections 2024-04-25T08:57:53.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.212+0000 7fcb0ea15640 1 --2- 172.21.15.43:0/1031478093 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcae806b4d0 0x7fcae806d990 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:53.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.212+0000 7fcb0ea15640 1 --2- 172.21.15.43:0/1031478093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb0815de80 0x7fcb080cff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:53.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.212+0000 7fcb0ea15640 1 --2- 172.21.15.43:0/1031478093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb080d3760 0x7fcb080cf9c0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:53.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.212+0000 7fcb0ea15640 1 --2- 172.21.15.43:0/1031478093 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcb080b06c0 0x7fcb080c9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:53.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.212+0000 7fcb0ea15640 1 -- 172.21.15.43:0/1031478093 >> 172.21.15.43:0/1031478093 conn(0x7fcb0808d460 msgr2=0x7fcb080b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:53.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.212+0000 7fcb0ea15640 1 -- 172.21.15.43:0/1031478093 shutdown_connections 2024-04-25T08:57:53.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.212+0000 7fcb0ea15640 1 -- 172.21.15.43:0/1031478093 wait complete. 2024-04-25T08:57:53.217 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T08:57:53.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.337+0000 7f566975a640 1 Processor -- start 2024-04-25T08:57:53.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.337+0000 7f566975a640 1 -- start start 2024-04-25T08:57:53.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.338+0000 7f566975a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56640b6d60 0x7f56640d0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:53.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.338+0000 7f566975a640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f566415ded0 0x7f56640d2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:53.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.338+0000 7f566975a640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56640d0d50 con 0x7f56640d0430 2024-04-25T08:57:53.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.338+0000 7f566975a640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56640d1ed0 con 0x7f566415ded0 2024-04-25T08:57:53.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.338+0000 7f566975a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56640ce820 con 0x7f56640b6d60 2024-04-25T08:57:53.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.338+0000 7f56637fe640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f566415ded0 0x7f56640d2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:53.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.338+0000 7f56627fc640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56640b6d60 0x7f56640d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:53.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.338+0000 7f5662ffd640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f56640d0430 0x7f56641621c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:40808/0 (socket says 172.21.15.43:40808) 2024-04-25T08:57:53.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.338+0000 7f56637fe640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f566415ded0 0x7f56640d2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:37344/0 (socket says 172.21.15.43:37344) 2024-04-25T08:57:53.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.338+0000 7f5662ffd640 1 -- 172.21.15.43:0/348386914 learned_addr learned my addr 172.21.15.43:0/348386914 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:53.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.338+0000 7f56627fc640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56640b6d60 0x7f56640d0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:37046/0 (socket says 172.21.15.43:37046) 2024-04-25T08:57:53.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.339+0000 7f56637fe640 1 -- 172.21.15.43:0/348386914 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56640b6d60 msgr2=0x7f56640d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:53.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.339+0000 7f56637fe640 1 --2- 172.21.15.43:0/348386914 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56640b6d60 0x7f56640d0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:53.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.339+0000 7f56637fe640 1 -- 172.21.15.43:0/348386914 >> v1:172.21.15.193:6789/0 conn(0x7f56640d0430 legacy=0x7f56641621c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:53.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.339+0000 7f56637fe640 1 -- 172.21.15.43:0/348386914 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56640ceb00 con 0x7f566415ded0 2024-04-25T08:57:53.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.339+0000 7f56627fc640 1 --2- 172.21.15.43:0/348386914 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56640b6d60 0x7f56640d0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:57:53.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.340+0000 7f56637fe640 1 --2- 172.21.15.43:0/348386914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f566415ded0 0x7f56640d2360 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5654009d90 tx=0x7f565402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4eb1a059a491cdb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:53.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.340+0000 7f5661ffb640 1 -- 172.21.15.43:0/348386914 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5654007740 con 0x7f566415ded0 2024-04-25T08:57:53.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.340+0000 7f5661ffb640 1 -- 172.21.15.43:0/348386914 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56540078e0 con 0x7f566415ded0 2024-04-25T08:57:53.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.340+0000 7f5661ffb640 1 -- 172.21.15.43:0/348386914 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5654007cb0 con 0x7f566415ded0 2024-04-25T08:57:53.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.340+0000 7f566975a640 1 -- 172.21.15.43:0/348386914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f566415ded0 msgr2=0x7f56640d2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:53.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.340+0000 7f566975a640 1 --2- 172.21.15.43:0/348386914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f566415ded0 0x7f56640d2360 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f5654009d90 tx=0x7f565402a660 comp rx=0 tx=0).stop 2024-04-25T08:57:53.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.340+0000 7f566975a640 1 -- 172.21.15.43:0/348386914 shutdown_connections 2024-04-25T08:57:53.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.340+0000 7f566975a640 1 --2- 172.21.15.43:0/348386914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f566415ded0 0x7f56640d2360 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:53.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.340+0000 7f566975a640 1 --2- 172.21.15.43:0/348386914 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56640b6d60 0x7f56640d0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:53.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.340+0000 7f566975a640 1 -- 172.21.15.43:0/348386914 >> 172.21.15.43:0/348386914 conn(0x7f566406b1e0 msgr2=0x7f566406b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:53.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.340+0000 7f566975a640 1 -- 172.21.15.43:0/348386914 shutdown_connections 2024-04-25T08:57:53.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.340+0000 7f566975a640 1 -- 172.21.15.43:0/348386914 wait complete. 2024-04-25T08:57:53.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.343+0000 7f566975a640 1 Processor -- start 2024-04-25T08:57:53.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.343+0000 7f566975a640 1 -- start start 2024-04-25T08:57:53.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.343+0000 7f566975a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56640b6d60 0x7f56640c9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:53.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.343+0000 7f566975a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56640d0430 0x7f56641549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:53.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.343+0000 7f566975a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f566415ded0 0x7f5664157c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:53.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.343+0000 7f566975a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56640ce280 con 0x7f56640b6d60 2024-04-25T08:57:53.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.343+0000 7f566975a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56640cf0c0 con 0x7f56640d0430 2024-04-25T08:57:53.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.343+0000 7f566975a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56640ce400 con 0x7f566415ded0 2024-04-25T08:57:53.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.343+0000 7f5662ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56640b6d60 0x7f56640c9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:53.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.343+0000 7f5662ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56640b6d60 0x7f56640c9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49960/0 (socket says 172.21.15.43:49960) 2024-04-25T08:57:53.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.343+0000 7f5662ffd640 1 -- 172.21.15.43:0/1815590287 learned_addr learned my addr 172.21.15.43:0/1815590287 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:53.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.343+0000 7f56627fc640 1 --2- 172.21.15.43:0/1815590287 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56640d0430 0x7f56641549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:53.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.343+0000 7f56637fe640 1 --2- 172.21.15.43:0/1815590287 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f566415ded0 0x7f5664157c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:53.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.343+0000 7f5662ffd640 1 -- 172.21.15.43:0/1815590287 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f566415ded0 msgr2=0x7f5664157c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:53.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.343+0000 7f5662ffd640 1 --2- 172.21.15.43:0/1815590287 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f566415ded0 0x7f5664157c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:53.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.343+0000 7f5662ffd640 1 -- 172.21.15.43:0/1815590287 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56640d0430 msgr2=0x7f56641549f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:53.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.343+0000 7f5662ffd640 1 --2- 172.21.15.43:0/1815590287 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56640d0430 0x7f56641549f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:53.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.343+0000 7f5662ffd640 1 -- 172.21.15.43:0/1815590287 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56640cbc50 con 0x7f56640b6d60 2024-04-25T08:57:53.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.343+0000 7f5662ffd640 1 --2- 172.21.15.43:0/1815590287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56640b6d60 0x7f56640c9f70 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f565c00c400 tx=0x7f565c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:53.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.343+0000 7f565b7fe640 1 -- 172.21.15.43:0/1815590287 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565c017070 con 0x7f56640b6d60 2024-04-25T08:57:53.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.343+0000 7f565b7fe640 1 -- 172.21.15.43:0/1815590287 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f565c005ce0 con 0x7f56640b6d60 2024-04-25T08:57:53.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.343+0000 7f566975a640 1 -- 172.21.15.43:0/1815590287 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56640cbf30 con 0x7f56640b6d60 2024-04-25T08:57:53.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.343+0000 7f565b7fe640 1 -- 172.21.15.43:0/1815590287 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565c0072f0 con 0x7f56640b6d60 2024-04-25T08:57:53.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.343+0000 7f566975a640 1 -- 172.21.15.43:0/1815590287 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5664156a70 con 0x7f56640b6d60 2024-04-25T08:57:53.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.344+0000 7f566975a640 1 -- 172.21.15.43:0/1815590287 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56640680f0 con 0x7f56640b6d60 2024-04-25T08:57:53.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.345+0000 7f565b7fe640 1 -- 172.21.15.43:0/1815590287 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f565c01a070 con 0x7f56640b6d60 2024-04-25T08:57:53.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.346+0000 7f565b7fe640 1 --2- 172.21.15.43:0/1815590287 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f564406b4d0 0x7f564406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:53.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.346+0000 7f565b7fe640 1 -- 172.21.15.43:0/1815590287 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f565c089320 con 0x7f56640b6d60 2024-04-25T08:57:53.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.346+0000 7f56627fc640 1 --2- 172.21.15.43:0/1815590287 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f564406b4d0 0x7f564406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:53.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.346+0000 7f56627fc640 1 --2- 172.21.15.43:0/1815590287 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f564406b4d0 0x7f564406d990 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f564c009560 tx=0x7f564c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:53.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.350+0000 7f565b7fe640 1 -- 172.21.15.43:0/1815590287 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f565c057dc0 con 0x7f56640b6d60 2024-04-25T08:57:53.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:53.534+0000 7f566975a640 1 -- 172.21.15.43:0/1815590287 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f56640cc450 con 0x7f56640b6d60 2024-04-25T08:57:54.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.215+0000 7f565b7fe640 1 -- 172.21.15.43:0/1815590287 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v56) v1 ==== 170+0+0 (secure 0 0 0) 0x7f56640cc450 con 0x7f56640b6d60 2024-04-25T08:57:54.204 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' created 2024-04-25T08:57:54.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.219+0000 7f566975a640 1 -- 172.21.15.43:0/1815590287 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f564406b4d0 msgr2=0x7f564406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:54.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.219+0000 7f566975a640 1 --2- 172.21.15.43:0/1815590287 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f564406b4d0 0x7f564406d990 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f564c009560 tx=0x7f564c009190 comp rx=0 tx=0).stop 2024-04-25T08:57:54.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.220+0000 7f566975a640 1 -- 172.21.15.43:0/1815590287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56640b6d60 msgr2=0x7f56640c9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:54.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.220+0000 7f566975a640 1 --2- 172.21.15.43:0/1815590287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56640b6d60 0x7f56640c9f70 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f565c00c400 tx=0x7f565c00c8c0 comp rx=0 tx=0).stop 2024-04-25T08:57:54.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.229+0000 7f566975a640 1 -- 172.21.15.43:0/1815590287 shutdown_connections 2024-04-25T08:57:54.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.229+0000 7f566975a640 1 --2- 172.21.15.43:0/1815590287 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f566415ded0 0x7f5664157c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:54.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.229+0000 7f566975a640 1 --2- 172.21.15.43:0/1815590287 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f564406b4d0 0x7f564406d990 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:54.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.229+0000 7f566975a640 1 --2- 172.21.15.43:0/1815590287 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56640d0430 0x7f56641549f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:54.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.229+0000 7f566975a640 1 --2- 172.21.15.43:0/1815590287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f56640b6d60 0x7f56640c9f70 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:54.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.230+0000 7f566975a640 1 -- 172.21.15.43:0/1815590287 >> 172.21.15.43:0/1815590287 conn(0x7f566406b1e0 msgr2=0x7f56640d3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:54.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.231+0000 7f566975a640 1 -- 172.21.15.43:0/1815590287 shutdown_connections 2024-04-25T08:57:54.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.231+0000 7f566975a640 1 -- 172.21.15.43:0/1815590287 wait complete. 2024-04-25T08:57:54.238 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T08:57:54.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.358+0000 7fdf44a9a640 1 Processor -- start 2024-04-25T08:57:54.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.358+0000 7fdf44a9a640 1 -- start start 2024-04-25T08:57:54.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.359+0000 7fdf44a9a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdf400d1a20 0x7fdf400b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:54.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.359+0000 7fdf44a9a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf4015bf90 con 0x7fdf400d1e00 2024-04-25T08:57:54.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.359+0000 7fdf44a9a640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf400cee70 con 0x7fdf400d2260 2024-04-25T08:57:54.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.359+0000 7fdf44a9a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf400cf430 con 0x7fdf400d1a20 2024-04-25T08:57:54.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.359+0000 7fdf3dd9b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fdf400d1e00 0x7fdf400b5100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45326/0 (socket says 172.21.15.43:45326) 2024-04-25T08:57:54.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.359+0000 7fdf3dd9b640 1 -- 172.21.15.43:0/1981977146 learned_addr learned my addr 172.21.15.43:0/1981977146 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:54.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.360+0000 7fdf3d59a640 1 -- 172.21.15.43:0/1981977146 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 36702972 0 0) 0x7fdf4015bf90 con 0x7fdf400d1e00 2024-04-25T08:57:54.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.360+0000 7fdf3e59c640 1 --2- 172.21.15.43:0/1981977146 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdf400d1a20 0x7fdf400b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:54.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.360+0000 7fdf3d59a640 1 -- 172.21.15.43:0/1981977146 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf2c003520 con 0x7fdf400d1e00 2024-04-25T08:57:54.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.360+0000 7fdf3d59a640 1 -- 172.21.15.43:0/1981977146 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 623560028 0 0) 0x7fdf2c003520 con 0x7fdf400d1e00 2024-04-25T08:57:54.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.360+0000 7fdf3d59a640 1 -- 172.21.15.43:0/1981977146 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdf400d1a20 msgr2=0x7fdf400b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:54.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.360+0000 7fdf3d59a640 1 --2- 172.21.15.43:0/1981977146 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdf400d1a20 0x7fdf400b2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:54.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.360+0000 7fdf3d59a640 1 -- 172.21.15.43:0/1981977146 >> v1:172.21.15.193:6789/0 conn(0x7fdf400d2260 legacy=0x7fdf400b5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:54.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.360+0000 7fdf3d59a640 1 -- 172.21.15.43:0/1981977146 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf400cf150 con 0x7fdf400d1e00 2024-04-25T08:57:54.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.360+0000 7fdf3d59a640 1 -- 172.21.15.43:0/1981977146 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7fdf28001670 con 0x7fdf400d1e00 2024-04-25T08:57:54.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.360+0000 7fdf3d59a640 1 -- 172.21.15.43:0/1981977146 >> v1:172.21.15.43:6789/0 conn(0x7fdf400d1e00 legacy=0x7fdf400b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:54.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.360+0000 7fdf3d59a640 1 --2- 172.21.15.43:0/1981977146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf2c0039e0 0x7fdf2c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:54.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.360+0000 7fdf3d59a640 1 --2- 172.21.15.43:0/1981977146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf2c004800 0x7fdf2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:54.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.360+0000 7fdf3d59a640 1 --2- 172.21.15.43:0/1981977146 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdf400d1a20 0x7fdf2c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:54.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.360+0000 7fdf3d59a640 1 -- 172.21.15.43:0/1981977146 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf400cf430 con 0x7fdf2c004800 2024-04-25T08:57:54.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.360+0000 7fdf3d59a640 1 -- 172.21.15.43:0/1981977146 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c005540 con 0x7fdf2c0039e0 2024-04-25T08:57:54.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.360+0000 7fdf3d59a640 1 -- 172.21.15.43:0/1981977146 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c00bf00 con 0x7fdf400d1a20 2024-04-25T08:57:54.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.360+0000 7fdf3dd9b640 1 --2- 172.21.15.43:0/1981977146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf2c004800 0x7fdf2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:54.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.360+0000 7fdf3e59c640 1 --2- 172.21.15.43:0/1981977146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf2c0039e0 0x7fdf2c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:54.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.360+0000 7fdf3ed9d640 1 --2- 172.21.15.43:0/1981977146 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdf400d1a20 0x7fdf2c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:54.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.361+0000 7fdf3dd9b640 1 -- 172.21.15.43:0/1981977146 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdf400d1a20 msgr2=0x7fdf2c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:54.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.361+0000 7fdf3dd9b640 1 --2- 172.21.15.43:0/1981977146 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdf400d1a20 0x7fdf2c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:54.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.361+0000 7fdf3dd9b640 1 -- 172.21.15.43:0/1981977146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf2c0039e0 msgr2=0x7fdf2c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:54.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.361+0000 7fdf3dd9b640 1 --2- 172.21.15.43:0/1981977146 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf2c0039e0 0x7fdf2c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:54.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.361+0000 7fdf3dd9b640 1 -- 172.21.15.43:0/1981977146 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2c003520 con 0x7fdf2c004800 2024-04-25T08:57:54.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.361+0000 7fdf3dd9b640 1 --2- 172.21.15.43:0/1981977146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf2c004800 0x7fdf2c004c80 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fdf2800c210 tx=0x7fdf2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a46e5769fea3387f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:54.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.361+0000 7fdf3d59a640 1 -- 172.21.15.43:0/1981977146 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf28007d30 con 0x7fdf2c004800 2024-04-25T08:57:54.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.361+0000 7fdf3d59a640 1 -- 172.21.15.43:0/1981977146 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf28005ad0 con 0x7fdf2c004800 2024-04-25T08:57:54.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.361+0000 7fdf3d59a640 1 -- 172.21.15.43:0/1981977146 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf28007070 con 0x7fdf2c004800 2024-04-25T08:57:54.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.362+0000 7fdf44a9a640 1 -- 172.21.15.43:0/1981977146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf2c004800 msgr2=0x7fdf2c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:54.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.362+0000 7fdf44a9a640 1 --2- 172.21.15.43:0/1981977146 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf2c004800 0x7fdf2c004c80 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fdf2800c210 tx=0x7fdf2c005830 comp rx=0 tx=0).stop 2024-04-25T08:57:54.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.362+0000 7fdf3ed9d640 1 -- 172.21.15.43:0/1981977146 reap_dead start 2024-04-25T08:57:54.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.362+0000 7fdf44a9a640 1 -- 172.21.15.43:0/1981977146 shutdown_connections 2024-04-25T08:57:54.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.362+0000 7fdf44a9a640 1 -- 172.21.15.43:0/1981977146 >> 172.21.15.43:0/1981977146 conn(0x7fdf40072aa0 msgr2=0x7fdf4009aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:54.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.362+0000 7fdf44a9a640 1 -- 172.21.15.43:0/1981977146 shutdown_connections 2024-04-25T08:57:54.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.362+0000 7fdf44a9a640 1 -- 172.21.15.43:0/1981977146 wait complete. 2024-04-25T08:57:54.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.364+0000 7fdf44a9a640 1 Processor -- start 2024-04-25T08:57:54.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.364+0000 7fdf44a9a640 1 -- start start 2024-04-25T08:57:54.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.364+0000 7fdf44a9a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdf2c004800 0x7fdf40148210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:54.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.364+0000 7fdf44a9a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf401f6f50 0x7fdf4006d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:54.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.364+0000 7fdf44a9a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf40154d90 0x7fdf40155210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:54.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.364+0000 7fdf44a9a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf400cea60 con 0x7fdf40154d90 2024-04-25T08:57:54.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.364+0000 7fdf44a9a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf400b4be0 con 0x7fdf401f6f50 2024-04-25T08:57:54.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.364+0000 7fdf44a9a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf400cebe0 con 0x7fdf2c004800 2024-04-25T08:57:54.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.364+0000 7fdf3ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf40154d90 0x7fdf40155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:54.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.364+0000 7fdf3ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf40154d90 0x7fdf40155210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49974/0 (socket says 172.21.15.43:49974) 2024-04-25T08:57:54.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.364+0000 7fdf3ed9d640 1 -- 172.21.15.43:0/1020724256 learned_addr learned my addr 172.21.15.43:0/1020724256 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:54.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.365+0000 7fdf3e59c640 1 --2- 172.21.15.43:0/1020724256 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdf2c004800 0x7fdf40148210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:54.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.365+0000 7fdf3dd9b640 1 --2- 172.21.15.43:0/1020724256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf401f6f50 0x7fdf4006d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:54.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.365+0000 7fdf3ed9d640 1 -- 172.21.15.43:0/1020724256 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdf2c004800 msgr2=0x7fdf40148210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:54.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.365+0000 7fdf3ed9d640 1 --2- 172.21.15.43:0/1020724256 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdf2c004800 0x7fdf40148210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:54.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.365+0000 7fdf3ed9d640 1 -- 172.21.15.43:0/1020724256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf401f6f50 msgr2=0x7fdf4006d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:54.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.365+0000 7fdf3ed9d640 1 --2- 172.21.15.43:0/1020724256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf401f6f50 0x7fdf4006d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:54.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.365+0000 7fdf3ed9d640 1 -- 172.21.15.43:0/1020724256 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf401f74c0 con 0x7fdf40154d90 2024-04-25T08:57:54.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.365+0000 7fdf3ed9d640 1 --2- 172.21.15.43:0/1020724256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf40154d90 0x7fdf40155210 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fdf400d2640 tx=0x7fdf3000fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:54.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.365+0000 7fdf36ffd640 1 -- 172.21.15.43:0/1020724256 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3001a070 con 0x7fdf40154d90 2024-04-25T08:57:54.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.365+0000 7fdf36ffd640 1 -- 172.21.15.43:0/1020724256 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf30006d60 con 0x7fdf40154d90 2024-04-25T08:57:54.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.365+0000 7fdf44a9a640 1 -- 172.21.15.43:0/1020724256 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf401f77a0 con 0x7fdf40154d90 2024-04-25T08:57:54.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.365+0000 7fdf36ffd640 1 -- 172.21.15.43:0/1020724256 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf30003b90 con 0x7fdf40154d90 2024-04-25T08:57:54.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.365+0000 7fdf44a9a640 1 -- 172.21.15.43:0/1020724256 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf40148750 con 0x7fdf40154d90 2024-04-25T08:57:54.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.366+0000 7fdf44a9a640 1 -- 172.21.15.43:0/1020724256 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf40074f60 con 0x7fdf40154d90 2024-04-25T08:57:54.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.367+0000 7fdf36ffd640 1 -- 172.21.15.43:0/1020724256 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdf30005070 con 0x7fdf40154d90 2024-04-25T08:57:54.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.367+0000 7fdf36ffd640 1 --2- 172.21.15.43:0/1020724256 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdf2006b4d0 0x7fdf2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:54.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.367+0000 7fdf36ffd640 1 -- 172.21.15.43:0/1020724256 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fdf300897d0 con 0x7fdf40154d90 2024-04-25T08:57:54.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.367+0000 7fdf3e59c640 1 --2- 172.21.15.43:0/1020724256 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdf2006b4d0 0x7fdf2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:54.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.368+0000 7fdf3e59c640 1 --2- 172.21.15.43:0/1020724256 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdf2006b4d0 0x7fdf2006d990 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fdf38006fd0 tx=0x7fdf38008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:54.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.372+0000 7fdf36ffd640 1 -- 172.21.15.43:0/1020724256 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdf30058630 con 0x7fdf40154d90 2024-04-25T08:57:54.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:54.589+0000 7fdf44a9a640 1 -- 172.21.15.43:0/1020724256 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fdf4006fb30 con 0x7fdf40154d90 2024-04-25T08:57:55.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.216+0000 7fdf36ffd640 1 -- 172.21.15.43:0/1020724256 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v29) v1 ==== 172+0+0 (secure 0 0 0) 0x7fdf30057f30 con 0x7fdf40154d90 2024-04-25T08:57:55.205 INFO:teuthology.orchestra.run.smithi043.stderr:new fs with metadata pool 8 and data pool 9 2024-04-25T08:57:55.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.219+0000 7fdf44a9a640 1 -- 172.21.15.43:0/1020724256 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdf2006b4d0 msgr2=0x7fdf2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:55.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.219+0000 7fdf44a9a640 1 --2- 172.21.15.43:0/1020724256 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdf2006b4d0 0x7fdf2006d990 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fdf38006fd0 tx=0x7fdf38008040 comp rx=0 tx=0).stop 2024-04-25T08:57:55.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.220+0000 7fdf44a9a640 1 -- 172.21.15.43:0/1020724256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf40154d90 msgr2=0x7fdf40155210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:55.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.220+0000 7fdf44a9a640 1 --2- 172.21.15.43:0/1020724256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf40154d90 0x7fdf40155210 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fdf400d2640 tx=0x7fdf3000fd40 comp rx=0 tx=0).stop 2024-04-25T08:57:55.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.220+0000 7fdf44a9a640 1 -- 172.21.15.43:0/1020724256 shutdown_connections 2024-04-25T08:57:55.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.220+0000 7fdf44a9a640 1 --2- 172.21.15.43:0/1020724256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf40154d90 0x7fdf40155210 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:55.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.220+0000 7fdf44a9a640 1 --2- 172.21.15.43:0/1020724256 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdf2006b4d0 0x7fdf2006d990 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:55.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.220+0000 7fdf44a9a640 1 --2- 172.21.15.43:0/1020724256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf401f6f50 0x7fdf4006d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:55.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.220+0000 7fdf44a9a640 1 --2- 172.21.15.43:0/1020724256 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdf2c004800 0x7fdf40148210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:55.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.220+0000 7fdf44a9a640 1 -- 172.21.15.43:0/1020724256 >> 172.21.15.43:0/1020724256 conn(0x7fdf40072aa0 msgr2=0x7fdf4006fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:55.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.220+0000 7fdf44a9a640 1 -- 172.21.15.43:0/1020724256 shutdown_connections 2024-04-25T08:57:55.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.220+0000 7fdf44a9a640 1 -- 172.21.15.43:0/1020724256 wait complete. 2024-04-25T08:57:55.225 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:57:55.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.342+0000 7fe870963640 1 Processor -- start 2024-04-25T08:57:55.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.342+0000 7fe870963640 1 -- start start 2024-04-25T08:57:55.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.343+0000 7fe870963640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe86c161390 0x7fe86c0b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:55.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.343+0000 7fe870963640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe86c0cecf0 0x7fe86c0d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:55.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.343+0000 7fe870963640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe86c0b4c30 con 0x7fe86c0d02a0 2024-04-25T08:57:55.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.343+0000 7fe870963640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe86c0d1910 con 0x7fe86c161390 2024-04-25T08:57:55.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.343+0000 7fe870963640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe86c0cf2c0 con 0x7fe86c0cecf0 2024-04-25T08:57:55.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.343+0000 7fe86a59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe86c161390 0x7fe86c0b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:55.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.343+0000 7fe869d9b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe86c0d02a0 0x7fe86c0ce5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45328/0 (socket says 172.21.15.43:45328) 2024-04-25T08:57:55.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.343+0000 7fe869d9b640 1 -- 172.21.15.43:0/120521785 learned_addr learned my addr 172.21.15.43:0/120521785 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:55.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.344+0000 7fe86ad9d640 1 --2- 172.21.15.43:0/120521785 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe86c0cecf0 0x7fe86c0d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:55.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.344+0000 7fe86a59c640 1 -- 172.21.15.43:0/120521785 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe86c0cecf0 msgr2=0x7fe86c0d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:55.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.344+0000 7fe86a59c640 1 --2- 172.21.15.43:0/120521785 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe86c0cecf0 0x7fe86c0d0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:55.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.344+0000 7fe86a59c640 1 -- 172.21.15.43:0/120521785 >> v1:172.21.15.43:6789/0 conn(0x7fe86c0d02a0 legacy=0x7fe86c0ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:55.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.344+0000 7fe86959a640 1 -- 172.21.15.43:0/120521785 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2446914696 0 0) 0x7fe860001110 con 0x7fe86c0d02a0 2024-04-25T08:57:55.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.344+0000 7fe86a59c640 1 -- 172.21.15.43:0/120521785 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe86c0cf5a0 con 0x7fe86c161390 2024-04-25T08:57:55.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.344+0000 7fe86a59c640 1 --2- 172.21.15.43:0/120521785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe86c161390 0x7fe86c0b6820 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fe85c0097a0 tx=0x7fe85c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcbd0504883833b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:55.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.345+0000 7fe86959a640 1 -- 172.21.15.43:0/120521785 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85c037070 con 0x7fe86c161390 2024-04-25T08:57:55.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.345+0000 7fe86959a640 1 -- 172.21.15.43:0/120521785 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe85c032440 con 0x7fe86c161390 2024-04-25T08:57:55.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.345+0000 7fe86959a640 1 -- 172.21.15.43:0/120521785 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe85c032810 con 0x7fe86c161390 2024-04-25T08:57:55.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.345+0000 7fe870963640 1 -- 172.21.15.43:0/120521785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe86c161390 msgr2=0x7fe86c0b6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:55.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.345+0000 7fe870963640 1 --2- 172.21.15.43:0/120521785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe86c161390 0x7fe86c0b6820 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fe85c0097a0 tx=0x7fe85c02a180 comp rx=0 tx=0).stop 2024-04-25T08:57:55.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.345+0000 7fe870963640 1 -- 172.21.15.43:0/120521785 shutdown_connections 2024-04-25T08:57:55.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.345+0000 7fe870963640 1 --2- 172.21.15.43:0/120521785 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe86c0cecf0 0x7fe86c0d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:55.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.345+0000 7fe870963640 1 --2- 172.21.15.43:0/120521785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe86c161390 0x7fe86c0b6820 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:55.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.345+0000 7fe870963640 1 -- 172.21.15.43:0/120521785 >> 172.21.15.43:0/120521785 conn(0x7fe86c09cd20 msgr2=0x7fe86c0d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:55.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.345+0000 7fe870963640 1 -- 172.21.15.43:0/120521785 shutdown_connections 2024-04-25T08:57:55.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.345+0000 7fe870963640 1 -- 172.21.15.43:0/120521785 wait complete. 2024-04-25T08:57:55.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.347+0000 7fe870963640 1 Processor -- start 2024-04-25T08:57:55.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.347+0000 7fe870963640 1 -- start start 2024-04-25T08:57:55.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.347+0000 7fe870963640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe86c0cecf0 0x7fe86c14bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:55.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.347+0000 7fe870963640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe86c0d02a0 0x7fe86c0cd090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:55.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.347+0000 7fe870963640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe86c161390 0x7fe86c1e6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:55.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.347+0000 7fe870963640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe86c161770 con 0x7fe86c161390 2024-04-25T08:57:55.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.347+0000 7fe870963640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe86c0d15a0 con 0x7fe86c0d02a0 2024-04-25T08:57:55.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.347+0000 7fe870963640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe86c0d1da0 con 0x7fe86c0cecf0 2024-04-25T08:57:55.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.347+0000 7fe86ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe86c161390 0x7fe86c1e6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:55.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.347+0000 7fe86ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe86c161390 0x7fe86c1e6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49988/0 (socket says 172.21.15.43:49988) 2024-04-25T08:57:55.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.347+0000 7fe86ad9d640 1 -- 172.21.15.43:0/1305607821 learned_addr learned my addr 172.21.15.43:0/1305607821 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:55.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.348+0000 7fe86a59c640 1 --2- 172.21.15.43:0/1305607821 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe86c0cecf0 0x7fe86c14bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:55.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.348+0000 7fe869d9b640 1 --2- 172.21.15.43:0/1305607821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe86c0d02a0 0x7fe86c0cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:55.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.348+0000 7fe86ad9d640 1 -- 172.21.15.43:0/1305607821 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe86c0cecf0 msgr2=0x7fe86c14bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:55.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.348+0000 7fe86ad9d640 1 --2- 172.21.15.43:0/1305607821 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe86c0cecf0 0x7fe86c14bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:55.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.348+0000 7fe86ad9d640 1 -- 172.21.15.43:0/1305607821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe86c0d02a0 msgr2=0x7fe86c0cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:55.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.348+0000 7fe86ad9d640 1 --2- 172.21.15.43:0/1305607821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe86c0d02a0 0x7fe86c0cd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:55.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.348+0000 7fe86ad9d640 1 -- 172.21.15.43:0/1305607821 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe86c14ccd0 con 0x7fe86c161390 2024-04-25T08:57:55.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.348+0000 7fe86a59c640 1 --2- 172.21.15.43:0/1305607821 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe86c0cecf0 0x7fe86c14bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:55.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.348+0000 7fe86ad9d640 1 --2- 172.21.15.43:0/1305607821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe86c161390 0x7fe86c1e6720 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fe864007e40 tx=0x7fe86400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:55.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.348+0000 7fe85affd640 1 -- 172.21.15.43:0/1305607821 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe864014620 con 0x7fe86c161390 2024-04-25T08:57:55.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.348+0000 7fe85affd640 1 -- 172.21.15.43:0/1305607821 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe864019070 con 0x7fe86c161390 2024-04-25T08:57:55.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.348+0000 7fe85affd640 1 -- 172.21.15.43:0/1305607821 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe864014a20 con 0x7fe86c161390 2024-04-25T08:57:55.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.348+0000 7fe870963640 1 -- 172.21.15.43:0/1305607821 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe86c14cfb0 con 0x7fe86c161390 2024-04-25T08:57:55.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.348+0000 7fe870963640 1 -- 172.21.15.43:0/1305607821 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe86c14d570 con 0x7fe86c161390 2024-04-25T08:57:55.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.349+0000 7fe870963640 1 -- 172.21.15.43:0/1305607821 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe86c0680f0 con 0x7fe86c161390 2024-04-25T08:57:55.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.350+0000 7fe85affd640 1 -- 172.21.15.43:0/1305607821 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe864014bc0 con 0x7fe86c161390 2024-04-25T08:57:55.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.350+0000 7fe85affd640 1 --2- 172.21.15.43:0/1305607821 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe84c06b4d0 0x7fe84c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:55.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.350+0000 7fe85affd640 1 -- 172.21.15.43:0/1305607821 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe86408aab0 con 0x7fe86c161390 2024-04-25T08:57:55.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.350+0000 7fe86a59c640 1 --2- 172.21.15.43:0/1305607821 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe84c06b4d0 0x7fe84c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:55.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.351+0000 7fe86a59c640 1 --2- 172.21.15.43:0/1305607821 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe84c06b4d0 0x7fe84c06d990 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe85c034000 tx=0x7fe85c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:55.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.354+0000 7fe85affd640 1 -- 172.21.15.43:0/1305607821 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe864059930 con 0x7fe86c161390 2024-04-25T08:57:55.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.534+0000 7fe870963640 1 -- 172.21.15.43:0/1305607821 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe86c0d1050 con 0x7fe86c161390 2024-04-25T08:57:55.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.535+0000 7fe85affd640 1 -- 172.21.15.43:0/1305607821 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16642 (secure 0 0 0) 0x7fe864059230 con 0x7fe86c161390 2024-04-25T08:57:55.524 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:57:55.524 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":57,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:57:54.590043+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:57:52.884445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T08:57:53.565716+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:57:55.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.537+0000 7fe870963640 1 -- 172.21.15.43:0/1305607821 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe84c06b4d0 msgr2=0x7fe84c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:55.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.537+0000 7fe870963640 1 --2- 172.21.15.43:0/1305607821 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe84c06b4d0 0x7fe84c06d990 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fe85c034000 tx=0x7fe85c02e000 comp rx=0 tx=0).stop 2024-04-25T08:57:55.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.537+0000 7fe870963640 1 -- 172.21.15.43:0/1305607821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe86c161390 msgr2=0x7fe86c1e6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:55.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.537+0000 7fe870963640 1 --2- 172.21.15.43:0/1305607821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe86c161390 0x7fe86c1e6720 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fe864007e40 tx=0x7fe86400e4b0 comp rx=0 tx=0).stop 2024-04-25T08:57:55.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.537+0000 7fe870963640 1 -- 172.21.15.43:0/1305607821 shutdown_connections 2024-04-25T08:57:55.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.537+0000 7fe870963640 1 --2- 172.21.15.43:0/1305607821 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe86c161390 0x7fe86c1e6720 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:55.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.537+0000 7fe870963640 1 --2- 172.21.15.43:0/1305607821 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe84c06b4d0 0x7fe84c06d990 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:55.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.537+0000 7fe870963640 1 --2- 172.21.15.43:0/1305607821 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe86c0d02a0 0x7fe86c0cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:55.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.537+0000 7fe870963640 1 --2- 172.21.15.43:0/1305607821 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe86c0cecf0 0x7fe86c14bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:55.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.537+0000 7fe870963640 1 -- 172.21.15.43:0/1305607821 >> 172.21.15.43:0/1305607821 conn(0x7fe86c09cd20 msgr2=0x7fe86c0d3770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:55.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.537+0000 7fe870963640 1 -- 172.21.15.43:0/1305607821 shutdown_connections 2024-04-25T08:57:55.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.537+0000 7fe870963640 1 -- 172.21.15.43:0/1305607821 wait complete. 2024-04-25T08:57:55.537 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:57:55.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.650+0000 7fcdfc1b0640 1 Processor -- start 2024-04-25T08:57:55.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.650+0000 7fcdfc1b0640 1 -- start start 2024-04-25T08:57:55.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.651+0000 7fcdfc1b0640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fcdf4161390 0x7fcdf40b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:55.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.651+0000 7fcdfc1b0640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcdf40d02a0 0x7fcdf40d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:55.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.651+0000 7fcdfc1b0640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf40b4c30 con 0x7fcdf40ce5e0 2024-04-25T08:57:55.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.651+0000 7fcdfc1b0640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdf40d1910 con 0x7fcdf40d02a0 2024-04-25T08:57:55.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.651+0000 7fcdfc1b0640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdf40cf2c0 con 0x7fcdf4161390 2024-04-25T08:57:55.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.651+0000 7fcdf974b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcdf40d02a0 0x7fcdf40d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:55.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.651+0000 7fcdfa74d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fcdf40ce5e0 0x7fcdf40cea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45330/0 (socket says 172.21.15.43:45330) 2024-04-25T08:57:55.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.651+0000 7fcdfa74d640 1 -- 172.21.15.43:0/3646458903 learned_addr learned my addr 172.21.15.43:0/3646458903 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:55.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.651+0000 7fcdf9f4c640 1 --2- 172.21.15.43:0/3646458903 >> v2:172.21.15.193:3300/0 conn(0x7fcdf4161390 0x7fcdf40b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:55.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.651+0000 7fcdf974b640 1 -- 172.21.15.43:0/3646458903 >> v2:172.21.15.193:3300/0 conn(0x7fcdf4161390 msgr2=0x7fcdf40b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:55.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.652+0000 7fcdf974b640 1 --2- 172.21.15.43:0/3646458903 >> v2:172.21.15.193:3300/0 conn(0x7fcdf4161390 0x7fcdf40b6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:55.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.652+0000 7fcdf974b640 1 -- 172.21.15.43:0/3646458903 >> v1:172.21.15.43:6789/0 conn(0x7fcdf40ce5e0 legacy=0x7fcdf40cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:55.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.652+0000 7fcdf974b640 1 -- 172.21.15.43:0/3646458903 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf40cf5a0 con 0x7fcdf40d02a0 2024-04-25T08:57:55.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.652+0000 7fcdf9f4c640 1 --2- 172.21.15.43:0/3646458903 >> v2:172.21.15.193:3300/0 conn(0x7fcdf4161390 0x7fcdf40b6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:55.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.652+0000 7fcdf974b640 1 --2- 172.21.15.43:0/3646458903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdf40d02a0 0x7fcdf40d0720 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fcdf40c74d0 tx=0x7fcde802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f94906f56daf754e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:55.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.652+0000 7fcdf8f4a640 1 -- 172.21.15.43:0/3646458903 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde8037070 con 0x7fcdf40d02a0 2024-04-25T08:57:55.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.652+0000 7fcdf8f4a640 1 -- 172.21.15.43:0/3646458903 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde8032440 con 0x7fcdf40d02a0 2024-04-25T08:57:55.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.652+0000 7fcdf8f4a640 1 -- 172.21.15.43:0/3646458903 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde8032810 con 0x7fcdf40d02a0 2024-04-25T08:57:55.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.653+0000 7fcdfc1b0640 1 -- 172.21.15.43:0/3646458903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdf40d02a0 msgr2=0x7fcdf40d0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:55.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.653+0000 7fcdfc1b0640 1 --2- 172.21.15.43:0/3646458903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdf40d02a0 0x7fcdf40d0720 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fcdf40c74d0 tx=0x7fcde802a1a0 comp rx=0 tx=0).stop 2024-04-25T08:57:55.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.653+0000 7fcdfc1b0640 1 -- 172.21.15.43:0/3646458903 shutdown_connections 2024-04-25T08:57:55.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.653+0000 7fcdfc1b0640 1 --2- 172.21.15.43:0/3646458903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdf40d02a0 0x7fcdf40d0720 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:55.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.653+0000 7fcdfc1b0640 1 --2- 172.21.15.43:0/3646458903 >> v2:172.21.15.193:3300/0 conn(0x7fcdf4161390 0x7fcdf40b6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:55.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.653+0000 7fcdfc1b0640 1 -- 172.21.15.43:0/3646458903 >> 172.21.15.43:0/3646458903 conn(0x7fcdf409cd20 msgr2=0x7fcdf40d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:55.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.653+0000 7fcdfc1b0640 1 -- 172.21.15.43:0/3646458903 shutdown_connections 2024-04-25T08:57:55.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.653+0000 7fcdfc1b0640 1 -- 172.21.15.43:0/3646458903 wait complete. 2024-04-25T08:57:55.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.654+0000 7fcdfc1b0640 1 Processor -- start 2024-04-25T08:57:55.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.654+0000 7fcdfc1b0640 1 -- start start 2024-04-25T08:57:55.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.655+0000 7fcdfc1b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdf40ce5e0 0x7fcdf40c9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:55.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.655+0000 7fcdfc1b0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcdf40d02a0 0x7fcdf4158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:55.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.655+0000 7fcdfc1b0640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcdf4161390 0x7fcdf41593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:55.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.655+0000 7fcdfc1b0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf4161770 con 0x7fcdf40ce5e0 2024-04-25T08:57:55.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.655+0000 7fcdfc1b0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf40d15a0 con 0x7fcdf40d02a0 2024-04-25T08:57:55.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.655+0000 7fcdfc1b0640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf40d1da0 con 0x7fcdf4161390 2024-04-25T08:57:55.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.655+0000 7fcdf9f4c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdf40ce5e0 0x7fcdf40c9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:55.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.655+0000 7fcdf9f4c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdf40ce5e0 0x7fcdf40c9b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50014/0 (socket says 172.21.15.43:50014) 2024-04-25T08:57:55.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.655+0000 7fcdf9f4c640 1 -- 172.21.15.43:0/2295650784 learned_addr learned my addr 172.21.15.43:0/2295650784 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:55.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.655+0000 7fcdfa74d640 1 --2- 172.21.15.43:0/2295650784 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcdf4161390 0x7fcdf41593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:55.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.655+0000 7fcdf974b640 1 --2- 172.21.15.43:0/2295650784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcdf40d02a0 0x7fcdf4158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:55.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.655+0000 7fcdf9f4c640 1 -- 172.21.15.43:0/2295650784 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcdf4161390 msgr2=0x7fcdf41593e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:55.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.655+0000 7fcdf9f4c640 1 --2- 172.21.15.43:0/2295650784 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcdf4161390 0x7fcdf41593e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:55.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.655+0000 7fcdf9f4c640 1 -- 172.21.15.43:0/2295650784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcdf40d02a0 msgr2=0x7fcdf4158ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:55.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.655+0000 7fcdf9f4c640 1 --2- 172.21.15.43:0/2295650784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcdf40d02a0 0x7fcdf4158ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:55.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.655+0000 7fcdf9f4c640 1 -- 172.21.15.43:0/2295650784 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf4159920 con 0x7fcdf40ce5e0 2024-04-25T08:57:55.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.656+0000 7fcdf9f4c640 1 --2- 172.21.15.43:0/2295650784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdf40ce5e0 0x7fcdf40c9b50 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fcdec00d7d0 tx=0x7fcdec00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:55.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.656+0000 7fcdf27fc640 1 -- 172.21.15.43:0/2295650784 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdec004030 con 0x7fcdf40ce5e0 2024-04-25T08:57:55.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.656+0000 7fcdf27fc640 1 -- 172.21.15.43:0/2295650784 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdec0041d0 con 0x7fcdf40ce5e0 2024-04-25T08:57:55.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.656+0000 7fcdfc1b0640 1 -- 172.21.15.43:0/2295650784 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf4159bd0 con 0x7fcdf40ce5e0 2024-04-25T08:57:55.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.656+0000 7fcdf27fc640 1 -- 172.21.15.43:0/2295650784 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdec0096a0 con 0x7fcdf40ce5e0 2024-04-25T08:57:55.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.656+0000 7fcdfc1b0640 1 -- 172.21.15.43:0/2295650784 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf40c74d0 con 0x7fcdf40ce5e0 2024-04-25T08:57:55.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.657+0000 7fcdfc1b0640 1 -- 172.21.15.43:0/2295650784 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdf40680f0 con 0x7fcdf40ce5e0 2024-04-25T08:57:55.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.657+0000 7fcdf27fc640 1 -- 172.21.15.43:0/2295650784 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcdec009840 con 0x7fcdf40ce5e0 2024-04-25T08:57:55.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.658+0000 7fcdf27fc640 1 --2- 172.21.15.43:0/2295650784 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcde006b4d0 0x7fcde006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:55.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.658+0000 7fcdf27fc640 1 -- 172.21.15.43:0/2295650784 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcdec08a470 con 0x7fcdf40ce5e0 2024-04-25T08:57:55.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.658+0000 7fcdf974b640 1 --2- 172.21.15.43:0/2295650784 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcde006b4d0 0x7fcde006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:55.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.658+0000 7fcdf974b640 1 --2- 172.21.15.43:0/2295650784 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcde006b4d0 0x7fcde006d990 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fcde8034000 tx=0x7fcde802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:55.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.662+0000 7fcdf27fc640 1 -- 172.21.15.43:0/2295650784 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcdec059380 con 0x7fcdf40ce5e0 2024-04-25T08:57:55.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.842+0000 7fcdfc1b0640 1 -- 172.21.15.43:0/2295650784 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcdf4074ef0 con 0x7fcdf40ce5e0 2024-04-25T08:57:55.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.843+0000 7fcdf27fc640 1 -- 172.21.15.43:0/2295650784 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+16642 (secure 0 0 0) 0x7fcdec058c80 con 0x7fcdf40ce5e0 2024-04-25T08:57:55.831 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:57:55.832 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":57,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:57:54.590043+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:57:52.884445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T08:57:53.565716+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:57:55.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.845+0000 7fcdfc1b0640 1 -- 172.21.15.43:0/2295650784 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcde006b4d0 msgr2=0x7fcde006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:55.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.845+0000 7fcdfc1b0640 1 --2- 172.21.15.43:0/2295650784 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcde006b4d0 0x7fcde006d990 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fcde8034000 tx=0x7fcde802e000 comp rx=0 tx=0).stop 2024-04-25T08:57:55.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.845+0000 7fcdfc1b0640 1 -- 172.21.15.43:0/2295650784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdf40ce5e0 msgr2=0x7fcdf40c9b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:55.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.845+0000 7fcdfc1b0640 1 --2- 172.21.15.43:0/2295650784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdf40ce5e0 0x7fcdf40c9b50 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fcdec00d7d0 tx=0x7fcdec00dc90 comp rx=0 tx=0).stop 2024-04-25T08:57:55.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.845+0000 7fcdfc1b0640 1 -- 172.21.15.43:0/2295650784 shutdown_connections 2024-04-25T08:57:55.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.845+0000 7fcdfc1b0640 1 --2- 172.21.15.43:0/2295650784 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcdf4161390 0x7fcdf41593e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:55.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.845+0000 7fcdfc1b0640 1 --2- 172.21.15.43:0/2295650784 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcde006b4d0 0x7fcde006d990 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:55.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.845+0000 7fcdfc1b0640 1 --2- 172.21.15.43:0/2295650784 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcdf40d02a0 0x7fcdf4158ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:55.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.845+0000 7fcdfc1b0640 1 --2- 172.21.15.43:0/2295650784 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcdf40ce5e0 0x7fcdf40c9b50 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:55.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.845+0000 7fcdfc1b0640 1 -- 172.21.15.43:0/2295650784 >> 172.21.15.43:0/2295650784 conn(0x7fcdf409cd20 msgr2=0x7fcdf40b7a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:55.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.845+0000 7fcdfc1b0640 1 -- 172.21.15.43:0/2295650784 shutdown_connections 2024-04-25T08:57:55.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.845+0000 7fcdfc1b0640 1 -- 172.21.15.43:0/2295650784 wait complete. 2024-04-25T08:57:55.845 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T08:57:55.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.961+0000 7fd977641640 1 Processor -- start 2024-04-25T08:57:55.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.961+0000 7fd977641640 1 -- start start 2024-04-25T08:57:55.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.962+0000 7fd977641640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd97015ddd0 0x7fd9700b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:55.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.962+0000 7fd977641640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fd97006d4f0 0x7fd97006f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:55.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.962+0000 7fd977641640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd97006d940 con 0x7fd97006ef90 2024-04-25T08:57:55.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.962+0000 7fd977641640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd97006f930 con 0x7fd97006d4f0 2024-04-25T08:57:55.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.962+0000 7fd977641640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9700d2620 con 0x7fd97015ddd0 2024-04-25T08:57:55.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.962+0000 7fd974bdc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fd97006ef90 0x7fd9700cc460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45346/0 (socket says 172.21.15.43:45346) 2024-04-25T08:57:55.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.962+0000 7fd974bdc640 1 -- 172.21.15.43:0/4175684840 learned_addr learned my addr 172.21.15.43:0/4175684840 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:55.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.962+0000 7fd975bde640 1 --2- 172.21.15.43:0/4175684840 >> v2:172.21.15.193:3300/0 conn(0x7fd97006d4f0 0x7fd97006f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:55.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.962+0000 7fd9753dd640 1 --2- 172.21.15.43:0/4175684840 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd97015ddd0 0x7fd9700b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:55.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.962+0000 7fd96ffff640 1 -- 172.21.15.43:0/4175684840 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2716447183 0 0) 0x7fd97006d940 con 0x7fd97006ef90 2024-04-25T08:57:55.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.962+0000 7fd96ffff640 1 -- 172.21.15.43:0/4175684840 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd958003520 con 0x7fd97006ef90 2024-04-25T08:57:55.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.963+0000 7fd96ffff640 1 -- 172.21.15.43:0/4175684840 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1467062348 0 0) 0x7fd958003520 con 0x7fd97006ef90 2024-04-25T08:57:55.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.963+0000 7fd96ffff640 1 -- 172.21.15.43:0/4175684840 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd97015ddd0 msgr2=0x7fd9700b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:55.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.963+0000 7fd96ffff640 1 --2- 172.21.15.43:0/4175684840 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd97015ddd0 0x7fd9700b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:55.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.963+0000 7fd96ffff640 1 -- 172.21.15.43:0/4175684840 >> v2:172.21.15.193:3300/0 conn(0x7fd97006d4f0 msgr2=0x7fd97006f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:55.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.963+0000 7fd96ffff640 1 --2- 172.21.15.43:0/4175684840 >> v2:172.21.15.193:3300/0 conn(0x7fd97006d4f0 0x7fd97006f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:55.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.963+0000 7fd96ffff640 1 -- 172.21.15.43:0/4175684840 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9700b0b80 con 0x7fd97006ef90 2024-04-25T08:57:55.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.963+0000 7fd96ffff640 1 -- 172.21.15.43:0/4175684840 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7fd964001670 con 0x7fd97006ef90 2024-04-25T08:57:55.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.963+0000 7fd96ffff640 1 -- 172.21.15.43:0/4175684840 >> v1:172.21.15.43:6789/0 conn(0x7fd97006ef90 legacy=0x7fd9700cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:55.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.963+0000 7fd96ffff640 1 --2- 172.21.15.43:0/4175684840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9580039b0 0x7fd958003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:55.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.963+0000 7fd96ffff640 1 --2- 172.21.15.43:0/4175684840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd958004800 0x7fd958004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:55.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.963+0000 7fd96ffff640 1 --2- 172.21.15.43:0/4175684840 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd97015ddd0 0x7fd9580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:55.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.963+0000 7fd96ffff640 1 -- 172.21.15.43:0/4175684840 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9700d2620 con 0x7fd958004800 2024-04-25T08:57:55.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.963+0000 7fd96ffff640 1 -- 172.21.15.43:0/4175684840 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9580141d0 con 0x7fd9580039b0 2024-04-25T08:57:55.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.963+0000 7fd96ffff640 1 -- 172.21.15.43:0/4175684840 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd958014350 con 0x7fd97015ddd0 2024-04-25T08:57:55.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.963+0000 7fd974bdc640 1 --2- 172.21.15.43:0/4175684840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd958004800 0x7fd958004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:55.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.963+0000 7fd9753dd640 1 --2- 172.21.15.43:0/4175684840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9580039b0 0x7fd958003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:55.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.963+0000 7fd975bde640 1 --2- 172.21.15.43:0/4175684840 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd97015ddd0 0x7fd9580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:55.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.963+0000 7fd975bde640 1 -- 172.21.15.43:0/4175684840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9580039b0 msgr2=0x7fd958003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:55.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.963+0000 7fd975bde640 1 --2- 172.21.15.43:0/4175684840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9580039b0 0x7fd958003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:55.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.963+0000 7fd975bde640 1 -- 172.21.15.43:0/4175684840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd958004800 msgr2=0x7fd958004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:55.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.963+0000 7fd975bde640 1 --2- 172.21.15.43:0/4175684840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd958004800 0x7fd958004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:55.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.963+0000 7fd975bde640 1 -- 172.21.15.43:0/4175684840 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd958003520 con 0x7fd97015ddd0 2024-04-25T08:57:55.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.963+0000 7fd974bdc640 1 --2- 172.21.15.43:0/4175684840 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd958004800 0x7fd958004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:57:55.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.963+0000 7fd9753dd640 1 --2- 172.21.15.43:0/4175684840 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd9580039b0 0x7fd958003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:57:55.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.964+0000 7fd975bde640 1 --2- 172.21.15.43:0/4175684840 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd97015ddd0 0x7fd9580051c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd968009ac0 tx=0x7fd958005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78dbf1b7fc46cb67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:55.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.964+0000 7fd96ffff640 1 -- 172.21.15.43:0/4175684840 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd968036070 con 0x7fd97015ddd0 2024-04-25T08:57:55.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.964+0000 7fd96ffff640 1 -- 172.21.15.43:0/4175684840 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd968031400 con 0x7fd97015ddd0 2024-04-25T08:57:55.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.964+0000 7fd96ffff640 1 -- 172.21.15.43:0/4175684840 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd968009690 con 0x7fd97015ddd0 2024-04-25T08:57:55.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.965+0000 7fd977641640 1 -- 172.21.15.43:0/4175684840 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd97015ddd0 msgr2=0x7fd9580051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:55.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.965+0000 7fd977641640 1 --2- 172.21.15.43:0/4175684840 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd97015ddd0 0x7fd9580051c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fd968009ac0 tx=0x7fd958005830 comp rx=0 tx=0).stop 2024-04-25T08:57:55.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.965+0000 7fd975bde640 1 -- 172.21.15.43:0/4175684840 reap_dead start 2024-04-25T08:57:55.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.965+0000 7fd977641640 1 -- 172.21.15.43:0/4175684840 shutdown_connections 2024-04-25T08:57:55.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.965+0000 7fd977641640 1 -- 172.21.15.43:0/4175684840 >> 172.21.15.43:0/4175684840 conn(0x7fd9700d17e0 msgr2=0x7fd9700d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:55.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.965+0000 7fd977641640 1 -- 172.21.15.43:0/4175684840 shutdown_connections 2024-04-25T08:57:55.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.965+0000 7fd977641640 1 -- 172.21.15.43:0/4175684840 wait complete. 2024-04-25T08:57:55.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.966+0000 7fd977641640 1 Processor -- start 2024-04-25T08:57:55.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.966+0000 7fd977641640 1 -- start start 2024-04-25T08:57:55.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.967+0000 7fd977641640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd97014ce60 0x7fd9700c8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:55.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.967+0000 7fd977641640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd9700b10a0 0x7fd9700b1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:55.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.967+0000 7fd977641640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9700b1a60 0x7fd9700ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:55.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.967+0000 7fd977641640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9700b3dd0 con 0x7fd9700b1a60 2024-04-25T08:57:55.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.967+0000 7fd977641640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9700cbf40 con 0x7fd97014ce60 2024-04-25T08:57:55.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.967+0000 7fd977641640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9700b3f50 con 0x7fd9700b10a0 2024-04-25T08:57:55.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.967+0000 7fd975bde640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9700b1a60 0x7fd9700ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:55.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.967+0000 7fd975bde640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9700b1a60 0x7fd9700ca160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50028/0 (socket says 172.21.15.43:50028) 2024-04-25T08:57:55.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.967+0000 7fd975bde640 1 -- 172.21.15.43:0/356294404 learned_addr learned my addr 172.21.15.43:0/356294404 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:55.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.967+0000 7fd974bdc640 1 --2- 172.21.15.43:0/356294404 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd9700b10a0 0x7fd9700b1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:55.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.967+0000 7fd9753dd640 1 --2- 172.21.15.43:0/356294404 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd97014ce60 0x7fd9700c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:55.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.967+0000 7fd975bde640 1 -- 172.21.15.43:0/356294404 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd9700b10a0 msgr2=0x7fd9700b1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:55.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.967+0000 7fd975bde640 1 --2- 172.21.15.43:0/356294404 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd9700b10a0 0x7fd9700b1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:55.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.967+0000 7fd975bde640 1 -- 172.21.15.43:0/356294404 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd97014ce60 msgr2=0x7fd9700c8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:55.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.967+0000 7fd975bde640 1 --2- 172.21.15.43:0/356294404 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd97014ce60 0x7fd9700c8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:55.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.967+0000 7fd975bde640 1 -- 172.21.15.43:0/356294404 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9701ef750 con 0x7fd9700b1a60 2024-04-25T08:57:55.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.967+0000 7fd975bde640 1 --2- 172.21.15.43:0/356294404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9700b1a60 0x7fd9700ca160 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fd968034040 tx=0x7fd968004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:55.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.968+0000 7fd96dffb640 1 -- 172.21.15.43:0/356294404 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd968036070 con 0x7fd9700b1a60 2024-04-25T08:57:55.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.968+0000 7fd96dffb640 1 -- 172.21.15.43:0/356294404 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd968037040 con 0x7fd9700b1a60 2024-04-25T08:57:55.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.968+0000 7fd977641640 1 -- 172.21.15.43:0/356294404 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9700b1ee0 con 0x7fd9700b1a60 2024-04-25T08:57:55.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.968+0000 7fd96dffb640 1 -- 172.21.15.43:0/356294404 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd960009c10 con 0x7fd9700b1a60 2024-04-25T08:57:55.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.968+0000 7fd977641640 1 -- 172.21.15.43:0/356294404 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9700cb830 con 0x7fd9700b1a60 2024-04-25T08:57:55.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.969+0000 7fd977641640 1 -- 172.21.15.43:0/356294404 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd970074f60 con 0x7fd9700b1a60 2024-04-25T08:57:55.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.969+0000 7fd96dffb640 1 -- 172.21.15.43:0/356294404 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd968042020 con 0x7fd9700b1a60 2024-04-25T08:57:55.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.970+0000 7fd96dffb640 1 --2- 172.21.15.43:0/356294404 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd95c06b4d0 0x7fd95c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:55.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.970+0000 7fd96dffb640 1 -- 172.21.15.43:0/356294404 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd9680a9690 con 0x7fd9700b1a60 2024-04-25T08:57:55.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.970+0000 7fd9753dd640 1 --2- 172.21.15.43:0/356294404 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd95c06b4d0 0x7fd95c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:55.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.970+0000 7fd9753dd640 1 --2- 172.21.15.43:0/356294404 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd95c06b4d0 0x7fd95c06d990 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd9600075c0 tx=0x7fd96000b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:55.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:55.974+0000 7fd96dffb640 1 -- 172.21.15.43:0/356294404 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd968049020 con 0x7fd9700b1a60 2024-04-25T08:57:56.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.193+0000 7fd977641640 1 -- 172.21.15.43:0/356294404 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd9700c77c0 con 0x7fd9700b1a60 2024-04-25T08:57:56.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.220+0000 7fd96dffb640 1 -- 172.21.15.43:0/356294404 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v30) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd9680783e0 con 0x7fd9700b1a60 2024-04-25T08:57:56.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.222+0000 7fd977641640 1 -- 172.21.15.43:0/356294404 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd95c06b4d0 msgr2=0x7fd95c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:56.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.222+0000 7fd977641640 1 --2- 172.21.15.43:0/356294404 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd95c06b4d0 0x7fd95c06d990 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd9600075c0 tx=0x7fd96000b040 comp rx=0 tx=0).stop 2024-04-25T08:57:56.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.222+0000 7fd977641640 1 -- 172.21.15.43:0/356294404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9700b1a60 msgr2=0x7fd9700ca160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:56.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.222+0000 7fd977641640 1 --2- 172.21.15.43:0/356294404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9700b1a60 0x7fd9700ca160 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fd968034040 tx=0x7fd968004730 comp rx=0 tx=0).stop 2024-04-25T08:57:56.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.222+0000 7fd977641640 1 -- 172.21.15.43:0/356294404 shutdown_connections 2024-04-25T08:57:56.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.222+0000 7fd977641640 1 --2- 172.21.15.43:0/356294404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd9700b1a60 0x7fd9700ca160 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:56.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.222+0000 7fd977641640 1 --2- 172.21.15.43:0/356294404 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd9700b10a0 0x7fd9700b1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:56.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.222+0000 7fd977641640 1 --2- 172.21.15.43:0/356294404 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd95c06b4d0 0x7fd95c06d990 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:56.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.222+0000 7fd977641640 1 --2- 172.21.15.43:0/356294404 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd97014ce60 0x7fd9700c8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:56.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.222+0000 7fd977641640 1 -- 172.21.15.43:0/356294404 >> 172.21.15.43:0/356294404 conn(0x7fd9700d17e0 msgr2=0x7fd9700c7fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:56.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.223+0000 7fd977641640 1 -- 172.21.15.43:0/356294404 shutdown_connections 2024-04-25T08:57:56.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.223+0000 7fd977641640 1 -- 172.21.15.43:0/356294404 wait complete. 2024-04-25T08:57:56.223 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T08:57:56.223 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T08:57:56.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.337+0000 7f4820e38640 1 Processor -- start 2024-04-25T08:57:56.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.337+0000 7f4820e38640 1 -- start start 2024-04-25T08:57:56.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.338+0000 7f4820e38640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f481c0d19f0 0x7f481c0d1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:56.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.338+0000 7f4820e38640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f481c0d2370 0x7f481c0c7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:56.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.338+0000 7f4820e38640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f481c0cf030 con 0x7f481c161310 2024-04-25T08:57:56.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.338+0000 7f4820e38640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f481c0d4870 con 0x7f481c0d2370 2024-04-25T08:57:56.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.338+0000 7f4820e38640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f481c0cf310 con 0x7f481c0d19f0 2024-04-25T08:57:56.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.338+0000 7f481ad9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f481c0d2370 0x7f481c0c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:56.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.338+0000 7f481ad9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f481c0d2370 0x7f481c0c7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50038/0 (socket says 172.21.15.43:50038) 2024-04-25T08:57:56.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.338+0000 7f481ad9d640 1 -- 172.21.15.43:0/2239704124 learned_addr learned my addr 172.21.15.43:0/2239704124 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:56.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.338+0000 7f4819d9b640 1 --2- 172.21.15.43:0/2239704124 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f481c0d19f0 0x7f481c0d1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:56.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.339+0000 7f481ad9d640 1 -- 172.21.15.43:0/2239704124 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f481c0d19f0 msgr2=0x7f481c0d1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:56.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.339+0000 7f481ad9d640 1 --2- 172.21.15.43:0/2239704124 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f481c0d19f0 0x7f481c0d1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:56.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.339+0000 7f481ad9d640 1 -- 172.21.15.43:0/2239704124 >> v1:172.21.15.193:6789/0 conn(0x7f481c161310 legacy=0x7f481c0d4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:56.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.339+0000 7f481ad9d640 1 -- 172.21.15.43:0/2239704124 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f481c0cf5f0 con 0x7f481c0d2370 2024-04-25T08:57:56.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.339+0000 7f481ad9d640 1 --2- 172.21.15.43:0/2239704124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f481c0d2370 0x7f481c0c7860 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f480c009e50 tx=0x7f480c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6069556fc55780d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:56.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.340+0000 7f481959a640 1 -- 172.21.15.43:0/2239704124 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c036070 con 0x7f481c0d2370 2024-04-25T08:57:56.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.340+0000 7f481959a640 1 -- 172.21.15.43:0/2239704124 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480c031600 con 0x7f481c0d2370 2024-04-25T08:57:56.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.340+0000 7f481959a640 1 -- 172.21.15.43:0/2239704124 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c0319d0 con 0x7f481c0d2370 2024-04-25T08:57:56.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.340+0000 7f4820e38640 1 -- 172.21.15.43:0/2239704124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f481c0d2370 msgr2=0x7f481c0c7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:56.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.340+0000 7f4820e38640 1 --2- 172.21.15.43:0/2239704124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f481c0d2370 0x7f481c0c7860 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f480c009e50 tx=0x7f480c02a710 comp rx=0 tx=0).stop 2024-04-25T08:57:56.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.340+0000 7f4820e38640 1 -- 172.21.15.43:0/2239704124 shutdown_connections 2024-04-25T08:57:56.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.340+0000 7f4820e38640 1 --2- 172.21.15.43:0/2239704124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f481c0d2370 0x7f481c0c7860 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:56.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.340+0000 7f4820e38640 1 --2- 172.21.15.43:0/2239704124 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f481c0d19f0 0x7f481c0d1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:56.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.340+0000 7f4820e38640 1 -- 172.21.15.43:0/2239704124 >> 172.21.15.43:0/2239704124 conn(0x7f481c09cd20 msgr2=0x7f481c0d3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:56.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.340+0000 7f4820e38640 1 -- 172.21.15.43:0/2239704124 shutdown_connections 2024-04-25T08:57:56.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.340+0000 7f4820e38640 1 -- 172.21.15.43:0/2239704124 wait complete. 2024-04-25T08:57:56.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.341+0000 7f4820e38640 1 Processor -- start 2024-04-25T08:57:56.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.342+0000 7f4820e38640 1 -- start start 2024-04-25T08:57:56.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.342+0000 7f4820e38640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f481c0d19f0 0x7f481c0caa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:56.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.342+0000 7f4820e38640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f481c0d2370 0x7f481c1f3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:56.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.342+0000 7f4820e38640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f481c161310 0x7f481c0cc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:56.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.342+0000 7f4820e38640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481c0c7da0 con 0x7f481c0d19f0 2024-04-25T08:57:56.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.342+0000 7f4820e38640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481c0d2800 con 0x7f481c0d2370 2024-04-25T08:57:56.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.342+0000 7f4820e38640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f481c0c7f20 con 0x7f481c161310 2024-04-25T08:57:56.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.342+0000 7f481a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f481c0d19f0 0x7f481c0caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:56.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.342+0000 7f481a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f481c0d19f0 0x7f481c0caa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50052/0 (socket says 172.21.15.43:50052) 2024-04-25T08:57:56.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.342+0000 7f481a59c640 1 -- 172.21.15.43:0/2082519449 learned_addr learned my addr 172.21.15.43:0/2082519449 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:56.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.342+0000 7f481ad9d640 1 --2- 172.21.15.43:0/2082519449 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f481c161310 0x7f481c0cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:56.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.342+0000 7f4819d9b640 1 --2- 172.21.15.43:0/2082519449 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f481c0d2370 0x7f481c1f3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:56.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.343+0000 7f481a59c640 1 -- 172.21.15.43:0/2082519449 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f481c161310 msgr2=0x7f481c0cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:56.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.343+0000 7f481a59c640 1 --2- 172.21.15.43:0/2082519449 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f481c161310 0x7f481c0cc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:56.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.343+0000 7f481a59c640 1 -- 172.21.15.43:0/2082519449 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f481c0d2370 msgr2=0x7f481c1f3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:56.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.343+0000 7f481a59c640 1 --2- 172.21.15.43:0/2082519449 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f481c0d2370 0x7f481c1f3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:56.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.343+0000 7f481a59c640 1 -- 172.21.15.43:0/2082519449 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f481c0b8170 con 0x7f481c0d19f0 2024-04-25T08:57:56.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.343+0000 7f481a59c640 1 --2- 172.21.15.43:0/2082519449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f481c0d19f0 0x7f481c0caa40 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f481400c3c0 tx=0x7f481400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:56.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.343+0000 7f4812ffd640 1 -- 172.21.15.43:0/2082519449 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48140052d0 con 0x7f481c0d19f0 2024-04-25T08:57:56.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.343+0000 7f4812ffd640 1 -- 172.21.15.43:0/2082519449 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4814005490 con 0x7f481c0d19f0 2024-04-25T08:57:56.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.343+0000 7f4812ffd640 1 -- 172.21.15.43:0/2082519449 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4814007260 con 0x7f481c0d19f0 2024-04-25T08:57:56.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.343+0000 7f4820e38640 1 -- 172.21.15.43:0/2082519449 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f481c0cb060 con 0x7f481c0d19f0 2024-04-25T08:57:56.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.343+0000 7f4820e38640 1 -- 172.21.15.43:0/2082519449 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f481c0cb620 con 0x7f481c0d19f0 2024-04-25T08:57:56.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.344+0000 7f4820e38640 1 -- 172.21.15.43:0/2082519449 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f481c074f60 con 0x7f481c0d19f0 2024-04-25T08:57:56.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.344+0000 7f4812ffd640 1 -- 172.21.15.43:0/2082519449 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4814007400 con 0x7f481c0d19f0 2024-04-25T08:57:56.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.345+0000 7f4812ffd640 1 --2- 172.21.15.43:0/2082519449 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f47fc06b4d0 0x7f47fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:56.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.345+0000 7f4812ffd640 1 -- 172.21.15.43:0/2082519449 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4814089650 con 0x7f481c0d19f0 2024-04-25T08:57:56.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.345+0000 7f4819d9b640 1 --2- 172.21.15.43:0/2082519449 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f47fc06b4d0 0x7f47fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:56.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.346+0000 7f4819d9b640 1 --2- 172.21.15.43:0/2082519449 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f47fc06b4d0 0x7f47fc06d990 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f481c06b740 tx=0x7f4804008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:56.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.349+0000 7f4812ffd640 1 -- 172.21.15.43:0/2082519449 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4814058560 con 0x7f481c0d19f0 2024-04-25T08:57:56.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:56.567+0000 7f4820e38640 1 -- 172.21.15.43:0/2082519449 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f481c1f37d0 con 0x7f481c0d19f0 2024-04-25T08:57:57.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.221+0000 7f4812ffd640 1 -- 172.21.15.43:0/2082519449 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v31) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4814057e60 con 0x7f481c0d19f0 2024-04-25T08:57:57.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.224+0000 7f4820e38640 1 -- 172.21.15.43:0/2082519449 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f47fc06b4d0 msgr2=0x7f47fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:57.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.224+0000 7f4820e38640 1 --2- 172.21.15.43:0/2082519449 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f47fc06b4d0 0x7f47fc06d990 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f481c06b740 tx=0x7f4804008040 comp rx=0 tx=0).stop 2024-04-25T08:57:57.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.225+0000 7f4820e38640 1 -- 172.21.15.43:0/2082519449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f481c0d19f0 msgr2=0x7f481c0caa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:57.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.225+0000 7f4820e38640 1 --2- 172.21.15.43:0/2082519449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f481c0d19f0 0x7f481c0caa40 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f481400c3c0 tx=0x7f481400c880 comp rx=0 tx=0).stop 2024-04-25T08:57:57.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.225+0000 7f4820e38640 1 -- 172.21.15.43:0/2082519449 shutdown_connections 2024-04-25T08:57:57.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.225+0000 7f4820e38640 1 --2- 172.21.15.43:0/2082519449 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f481c161310 0x7f481c0cc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:57.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.225+0000 7f4820e38640 1 --2- 172.21.15.43:0/2082519449 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f47fc06b4d0 0x7f47fc06d990 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:57.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.225+0000 7f4820e38640 1 --2- 172.21.15.43:0/2082519449 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f481c0d2370 0x7f481c1f3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:57.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.225+0000 7f4820e38640 1 --2- 172.21.15.43:0/2082519449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f481c0d19f0 0x7f481c0caa40 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:57.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.225+0000 7f4820e38640 1 -- 172.21.15.43:0/2082519449 >> 172.21.15.43:0/2082519449 conn(0x7f481c09cd20 msgr2=0x7f481c160520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:57.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.225+0000 7f4820e38640 1 -- 172.21.15.43:0/2082519449 shutdown_connections 2024-04-25T08:57:57.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.225+0000 7f4820e38640 1 -- 172.21.15.43:0/2082519449 wait complete. 2024-04-25T08:57:57.228 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T08:57:57.228 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:57:57.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.348+0000 7f714a088640 1 Processor -- start 2024-04-25T08:57:57.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.348+0000 7f714a088640 1 -- start start 2024-04-25T08:57:57.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.349+0000 7f714a088640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f714415df50 0x7f71441619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:57.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.349+0000 7f714a088640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f71440cf670 0x7f7144162380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:57.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.349+0000 7f714a088640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71440d24a0 con 0x7f7144161f30 2024-04-25T08:57:57.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.349+0000 7f714a088640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f714415e330 con 0x7f714415df50 2024-04-25T08:57:57.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.349+0000 7f714a088640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71440d2740 con 0x7f71440cf670 2024-04-25T08:57:57.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.349+0000 7f71437fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f714415df50 0x7f71441619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:57.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.349+0000 7f7142ffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f7144161f30 0x7f71440c7cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45350/0 (socket says 172.21.15.43:45350) 2024-04-25T08:57:57.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.349+0000 7f7142ffd640 1 -- 172.21.15.43:0/2246419104 learned_addr learned my addr 172.21.15.43:0/2246419104 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:57.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.350+0000 7f71437fe640 1 -- 172.21.15.43:0/2246419104 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f71440cf670 msgr2=0x7f7144162380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:57.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.350+0000 7f7143fff640 1 --2- 172.21.15.43:0/2246419104 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f71440cf670 0x7f7144162380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:57.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.350+0000 7f71427fc640 1 -- 172.21.15.43:0/2246419104 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2818148567 0 0) 0x7f71440d24a0 con 0x7f7144161f30 2024-04-25T08:57:57.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.350+0000 7f71437fe640 1 --2- 172.21.15.43:0/2246419104 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f71440cf670 0x7f7144162380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:57.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.350+0000 7f71437fe640 1 -- 172.21.15.43:0/2246419104 >> v1:172.21.15.43:6789/0 conn(0x7f7144161f30 legacy=0x7f71440c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:57.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.350+0000 7f71437fe640 1 -- 172.21.15.43:0/2246419104 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71440d2a20 con 0x7f714415df50 2024-04-25T08:57:57.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.350+0000 7f7143fff640 1 --2- 172.21.15.43:0/2246419104 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f71440cf670 0x7f7144162380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:57.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.350+0000 7f71437fe640 1 --2- 172.21.15.43:0/2246419104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f714415df50 0x7f71441619f0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f71340097a0 tx=0x7f713402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eda958fa5b7fadd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:57.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.351+0000 7f71427fc640 1 -- 172.21.15.43:0/2246419104 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7134037070 con 0x7f714415df50 2024-04-25T08:57:57.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.351+0000 7f71427fc640 1 -- 172.21.15.43:0/2246419104 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7134032440 con 0x7f714415df50 2024-04-25T08:57:57.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.351+0000 7f71427fc640 1 -- 172.21.15.43:0/2246419104 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7134032810 con 0x7f714415df50 2024-04-25T08:57:57.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.351+0000 7f714a088640 1 -- 172.21.15.43:0/2246419104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f714415df50 msgr2=0x7f71441619f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:57.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.351+0000 7f714a088640 1 --2- 172.21.15.43:0/2246419104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f714415df50 0x7f71441619f0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f71340097a0 tx=0x7f713402a180 comp rx=0 tx=0).stop 2024-04-25T08:57:57.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.351+0000 7f714a088640 1 -- 172.21.15.43:0/2246419104 shutdown_connections 2024-04-25T08:57:57.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.351+0000 7f714a088640 1 --2- 172.21.15.43:0/2246419104 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f71440cf670 0x7f7144162380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:57.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.351+0000 7f714a088640 1 --2- 172.21.15.43:0/2246419104 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f714415df50 0x7f71441619f0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:57.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.351+0000 7f714a088640 1 -- 172.21.15.43:0/2246419104 >> 172.21.15.43:0/2246419104 conn(0x7f71440d3a30 msgr2=0x7f71440d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:57.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.351+0000 7f714a088640 1 -- 172.21.15.43:0/2246419104 shutdown_connections 2024-04-25T08:57:57.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.351+0000 7f714a088640 1 -- 172.21.15.43:0/2246419104 wait complete. 2024-04-25T08:57:57.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.352+0000 7f714a088640 1 Processor -- start 2024-04-25T08:57:57.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.353+0000 7f714a088640 1 -- start start 2024-04-25T08:57:57.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.353+0000 7f714a088640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f71440cf670 0x7f7144153c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:57.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.353+0000 7f714a088640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f714415df50 0x7f714409f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:57.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.353+0000 7f714a088640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7144161f30 0x7f714409fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:57.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.353+0000 7f714a088640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71440d2190 con 0x7f7144161f30 2024-04-25T08:57:57.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.353+0000 7f714a088640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71440c77d0 con 0x7f714415df50 2024-04-25T08:57:57.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.353+0000 7f714a088640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71440c8400 con 0x7f71440cf670 2024-04-25T08:57:57.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.353+0000 7f7143fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7144161f30 0x7f714409fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:57.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.354+0000 7f7143fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7144161f30 0x7f714409fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50066/0 (socket says 172.21.15.43:50066) 2024-04-25T08:57:57.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.354+0000 7f7143fff640 1 -- 172.21.15.43:0/3242053374 learned_addr learned my addr 172.21.15.43:0/3242053374 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:57.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.354+0000 7f7142ffd640 1 --2- 172.21.15.43:0/3242053374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f714415df50 0x7f714409f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:57.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.354+0000 7f71437fe640 1 --2- 172.21.15.43:0/3242053374 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f71440cf670 0x7f7144153c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:57.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.354+0000 7f7143fff640 1 -- 172.21.15.43:0/3242053374 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f71440cf670 msgr2=0x7f7144153c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:57.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.354+0000 7f7143fff640 1 --2- 172.21.15.43:0/3242053374 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f71440cf670 0x7f7144153c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:57.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.354+0000 7f7143fff640 1 -- 172.21.15.43:0/3242053374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f714415df50 msgr2=0x7f714409f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:57.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.354+0000 7f7143fff640 1 --2- 172.21.15.43:0/3242053374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f714415df50 0x7f714409f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:57.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.354+0000 7f7143fff640 1 -- 172.21.15.43:0/3242053374 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71440a0f10 con 0x7f7144161f30 2024-04-25T08:57:57.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.354+0000 7f7143fff640 1 --2- 172.21.15.43:0/3242053374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7144161f30 0x7f714409fd50 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f713c00cf20 tx=0x7f713c00a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:57.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.354+0000 7f712bfff640 1 -- 172.21.15.43:0/3242053374 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713c01a070 con 0x7f7144161f30 2024-04-25T08:57:57.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.354+0000 7f712bfff640 1 -- 172.21.15.43:0/3242053374 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f713c00d840 con 0x7f7144161f30 2024-04-25T08:57:57.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.354+0000 7f714a088640 1 -- 172.21.15.43:0/3242053374 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71440a0390 con 0x7f7144161f30 2024-04-25T08:57:57.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.354+0000 7f712bfff640 1 -- 172.21.15.43:0/3242053374 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f713c00dc30 con 0x7f7144161f30 2024-04-25T08:57:57.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.354+0000 7f714a088640 1 -- 172.21.15.43:0/3242053374 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71440a0950 con 0x7f7144161f30 2024-04-25T08:57:57.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.355+0000 7f714a088640 1 -- 172.21.15.43:0/3242053374 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71440680f0 con 0x7f7144161f30 2024-04-25T08:57:57.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.356+0000 7f712bfff640 1 -- 172.21.15.43:0/3242053374 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f713c01d070 con 0x7f7144161f30 2024-04-25T08:57:57.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.356+0000 7f712bfff640 1 --2- 172.21.15.43:0/3242053374 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f713006b4d0 0x7f713006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:57.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.356+0000 7f712bfff640 1 -- 172.21.15.43:0/3242053374 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f713c08aa10 con 0x7f7144161f30 2024-04-25T08:57:57.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.356+0000 7f71437fe640 1 --2- 172.21.15.43:0/3242053374 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f713006b4d0 0x7f713006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:57.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.357+0000 7f71437fe640 1 --2- 172.21.15.43:0/3242053374 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f713006b4d0 0x7f713006d990 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7134034000 tx=0x7f713402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:57.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.360+0000 7f712bfff640 1 -- 172.21.15.43:0/3242053374 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f713c059240 con 0x7f7144161f30 2024-04-25T08:57:57.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.582+0000 7f714a088640 1 -- 172.21.15.43:0/3242053374 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f714415f630 con 0x7f7144161f30 2024-04-25T08:57:57.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.583+0000 7f712bfff640 1 -- 172.21.15.43:0/3242053374 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4944 (secure 0 0 0) 0x7f713c05d2e0 con 0x7f7144161f30 2024-04-25T08:57:57.572 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:57:57.572 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5950,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/2226672650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2226672650},{"type":"v1","addr":"172.21.15.193:6836","nonce":2226672650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5953,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2666301750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2666301750},{"type":"v1","addr":"172.21.15.193:6837","nonce":2666301750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6537,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/3574570511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3574570511},{"type":"v1","addr":"172.21.15.43:6834","nonce":3574570511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:57:54.590369+0000","modified":"2024-04-25T08:57:56.568622+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6540},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6540":{"gid":6540,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/2268368384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2268368384},{"type":"v1","addr":"172.21.15.43:6835","nonce":2268368384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:57:57.573 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 31 2024-04-25T08:57:57.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.585+0000 7f714a088640 1 -- 172.21.15.43:0/3242053374 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f713006b4d0 msgr2=0x7f713006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:57.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.585+0000 7f714a088640 1 --2- 172.21.15.43:0/3242053374 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f713006b4d0 0x7f713006d990 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f7134034000 tx=0x7f713402e000 comp rx=0 tx=0).stop 2024-04-25T08:57:57.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.585+0000 7f714a088640 1 -- 172.21.15.43:0/3242053374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7144161f30 msgr2=0x7f714409fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:57.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.585+0000 7f714a088640 1 --2- 172.21.15.43:0/3242053374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7144161f30 0x7f714409fd50 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f713c00cf20 tx=0x7f713c00a7d0 comp rx=0 tx=0).stop 2024-04-25T08:57:57.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.585+0000 7f714a088640 1 -- 172.21.15.43:0/3242053374 shutdown_connections 2024-04-25T08:57:57.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.585+0000 7f714a088640 1 --2- 172.21.15.43:0/3242053374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7144161f30 0x7f714409fd50 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:57.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.585+0000 7f714a088640 1 --2- 172.21.15.43:0/3242053374 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f713006b4d0 0x7f713006d990 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:57.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.585+0000 7f714a088640 1 --2- 172.21.15.43:0/3242053374 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f714415df50 0x7f714409f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:57.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.585+0000 7f714a088640 1 --2- 172.21.15.43:0/3242053374 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f71440cf670 0x7f7144153c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:57.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.585+0000 7f714a088640 1 -- 172.21.15.43:0/3242053374 >> 172.21.15.43:0/3242053374 conn(0x7f71440d3a30 msgr2=0x7f71440d1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:57.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.585+0000 7f714a088640 1 -- 172.21.15.43:0/3242053374 shutdown_connections 2024-04-25T08:57:57.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.585+0000 7f714a088640 1 -- 172.21.15.43:0/3242053374 wait complete. 2024-04-25T08:57:57.585 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:57:57.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.697+0000 7fa32461f640 1 Processor -- start 2024-04-25T08:57:57.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.697+0000 7fa32461f640 1 -- start start 2024-04-25T08:57:57.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.698+0000 7fa32461f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa31c0d09b0 0x7fa31c0b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:57.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.698+0000 7fa32461f640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fa31c0ce5e0 0x7fa31c0cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:57.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.698+0000 7fa32461f640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa31c0b4c30 con 0x7fa31c161390 2024-04-25T08:57:57.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.698+0000 7fa32461f640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa31c0d1910 con 0x7fa31c0ce5e0 2024-04-25T08:57:57.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.698+0000 7fa32461f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c0cf2c0 con 0x7fa31c0d09b0 2024-04-25T08:57:57.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.698+0000 7fa321bba640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa31c0d09b0 0x7fa31c0b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:57.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.698+0000 7fa321bba640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa31c0d09b0 0x7fa31c0b6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:37146/0 (socket says 172.21.15.43:37146) 2024-04-25T08:57:57.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.698+0000 7fa322bbc640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fa31c0ce5e0 0x7fa31c0cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:57.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.698+0000 7fa321bba640 1 -- 172.21.15.43:0/3868491500 learned_addr learned my addr 172.21.15.43:0/3868491500 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:57.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.699+0000 7fa321bba640 1 -- 172.21.15.43:0/3868491500 >> v2:172.21.15.193:3300/0 conn(0x7fa31c0ce5e0 msgr2=0x7fa31c0cea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:57.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.699+0000 7fa321bba640 1 --2- 172.21.15.43:0/3868491500 >> v2:172.21.15.193:3300/0 conn(0x7fa31c0ce5e0 0x7fa31c0cea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:57.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.699+0000 7fa321bba640 1 -- 172.21.15.43:0/3868491500 >> v1:172.21.15.193:6789/0 conn(0x7fa31c161390 legacy=0x7fa31c0d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:57.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.699+0000 7fa321bba640 1 -- 172.21.15.43:0/3868491500 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa31c0cf5a0 con 0x7fa31c0d09b0 2024-04-25T08:57:57.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.699+0000 7fa321bba640 1 --2- 172.21.15.43:0/3868491500 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa31c0d09b0 0x7fa31c0b6820 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa310009870 tx=0x7fa31002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e89de9e80c5b15ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:57.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.700+0000 7fa3213b9640 1 -- 172.21.15.43:0/3868491500 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa310037070 con 0x7fa31c0d09b0 2024-04-25T08:57:57.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.700+0000 7fa3213b9640 1 -- 172.21.15.43:0/3868491500 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa310032440 con 0x7fa31c0d09b0 2024-04-25T08:57:57.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.700+0000 7fa3213b9640 1 -- 172.21.15.43:0/3868491500 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa310032810 con 0x7fa31c0d09b0 2024-04-25T08:57:57.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.700+0000 7fa32461f640 1 -- 172.21.15.43:0/3868491500 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa31c0d09b0 msgr2=0x7fa31c0b6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:57.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.700+0000 7fa32461f640 1 --2- 172.21.15.43:0/3868491500 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa31c0d09b0 0x7fa31c0b6820 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa310009870 tx=0x7fa31002a270 comp rx=0 tx=0).stop 2024-04-25T08:57:57.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.700+0000 7fa32461f640 1 -- 172.21.15.43:0/3868491500 shutdown_connections 2024-04-25T08:57:57.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.700+0000 7fa32461f640 1 --2- 172.21.15.43:0/3868491500 >> v2:172.21.15.193:3300/0 conn(0x7fa31c0ce5e0 0x7fa31c0cea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:57.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.700+0000 7fa32461f640 1 --2- 172.21.15.43:0/3868491500 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa31c0d09b0 0x7fa31c0b6820 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:57.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.700+0000 7fa32461f640 1 -- 172.21.15.43:0/3868491500 >> 172.21.15.43:0/3868491500 conn(0x7fa31c09cd20 msgr2=0x7fa31c0d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:57.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.700+0000 7fa32461f640 1 -- 172.21.15.43:0/3868491500 shutdown_connections 2024-04-25T08:57:57.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.700+0000 7fa32461f640 1 -- 172.21.15.43:0/3868491500 wait complete. 2024-04-25T08:57:57.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.701+0000 7fa32461f640 1 Processor -- start 2024-04-25T08:57:57.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.701+0000 7fa32461f640 1 -- start start 2024-04-25T08:57:57.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.702+0000 7fa32461f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa31c0ce5e0 0x7fa31c1564f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:57.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.702+0000 7fa32461f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa31c0d09b0 0x7fa31c0caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:57.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.702+0000 7fa32461f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa31c161390 0x7fa31c0cafe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:57.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.702+0000 7fa32461f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c0cefd0 con 0x7fa31c0ce5e0 2024-04-25T08:57:57.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.702+0000 7fa32461f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c0d15a0 con 0x7fa31c161390 2024-04-25T08:57:57.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.702+0000 7fa32461f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c161770 con 0x7fa31c0d09b0 2024-04-25T08:57:57.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.702+0000 7fa3223bb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa31c0ce5e0 0x7fa31c1564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:57.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.702+0000 7fa3223bb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa31c0ce5e0 0x7fa31c1564f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50072/0 (socket says 172.21.15.43:50072) 2024-04-25T08:57:57.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.702+0000 7fa3223bb640 1 -- 172.21.15.43:0/2671096208 learned_addr learned my addr 172.21.15.43:0/2671096208 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:57.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.702+0000 7fa321bba640 1 --2- 172.21.15.43:0/2671096208 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa31c0d09b0 0x7fa31c0caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:57.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.702+0000 7fa322bbc640 1 --2- 172.21.15.43:0/2671096208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa31c161390 0x7fa31c0cafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:57.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.703+0000 7fa3223bb640 1 -- 172.21.15.43:0/2671096208 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa31c0d09b0 msgr2=0x7fa31c0caaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:57.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.703+0000 7fa3223bb640 1 --2- 172.21.15.43:0/2671096208 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa31c0d09b0 0x7fa31c0caaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:57.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.703+0000 7fa3223bb640 1 -- 172.21.15.43:0/2671096208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa31c161390 msgr2=0x7fa31c0cafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:57.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.703+0000 7fa3223bb640 1 --2- 172.21.15.43:0/2671096208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa31c161390 0x7fa31c0cafe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:57.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.703+0000 7fa3223bb640 1 -- 172.21.15.43:0/2671096208 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa31c0ccd30 con 0x7fa31c0ce5e0 2024-04-25T08:57:57.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.703+0000 7fa3223bb640 1 --2- 172.21.15.43:0/2671096208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa31c0ce5e0 0x7fa31c1564f0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fa30c00c730 tx=0x7fa30c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:57.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.703+0000 7fa31affd640 1 -- 172.21.15.43:0/2671096208 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30c016020 con 0x7fa31c0ce5e0 2024-04-25T08:57:57.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.703+0000 7fa31affd640 1 -- 172.21.15.43:0/2671096208 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa30c006d50 con 0x7fa31c0ce5e0 2024-04-25T08:57:57.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.703+0000 7fa32461f640 1 -- 172.21.15.43:0/2671096208 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa31c1f3c10 con 0x7fa31c0ce5e0 2024-04-25T08:57:57.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.703+0000 7fa31affd640 1 -- 172.21.15.43:0/2671096208 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30c003bb0 con 0x7fa31c0ce5e0 2024-04-25T08:57:57.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.703+0000 7fa32461f640 1 -- 172.21.15.43:0/2671096208 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa31c0ccfb0 con 0x7fa31c0ce5e0 2024-04-25T08:57:57.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.704+0000 7fa32461f640 1 -- 172.21.15.43:0/2671096208 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa31c0680f0 con 0x7fa31c0ce5e0 2024-04-25T08:57:57.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.705+0000 7fa31affd640 1 -- 172.21.15.43:0/2671096208 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa30c00a850 con 0x7fa31c0ce5e0 2024-04-25T08:57:57.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.705+0000 7fa31affd640 1 --2- 172.21.15.43:0/2671096208 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa30806b4d0 0x7fa30806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:57.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.705+0000 7fa31affd640 1 -- 172.21.15.43:0/2671096208 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa30c089d10 con 0x7fa31c0ce5e0 2024-04-25T08:57:57.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.705+0000 7fa321bba640 1 --2- 172.21.15.43:0/2671096208 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa30806b4d0 0x7fa30806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:57.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.706+0000 7fa321bba640 1 --2- 172.21.15.43:0/2671096208 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa30806b4d0 0x7fa30806d990 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa310034000 tx=0x7fa31002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:57.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.710+0000 7fa31affd640 1 -- 172.21.15.43:0/2671096208 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa30c020020 con 0x7fa31c0ce5e0 2024-04-25T08:57:57.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.892+0000 7fa32461f640 1 -- 172.21.15.43:0/2671096208 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa31c074ef0 con 0x7fa31c0ce5e0 2024-04-25T08:57:57.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.893+0000 7fa31affd640 1 -- 172.21.15.43:0/2671096208 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16629 (secure 0 0 0) 0x7fa31c0ccd30 con 0x7fa31c0ce5e0 2024-04-25T08:57:57.883 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:57:57.883 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":58,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:57:55.889262+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:57:52.884445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T08:57:53.565716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:57:57.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.895+0000 7fa32461f640 1 -- 172.21.15.43:0/2671096208 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa30806b4d0 msgr2=0x7fa30806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:57.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.895+0000 7fa32461f640 1 --2- 172.21.15.43:0/2671096208 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa30806b4d0 0x7fa30806d990 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa310034000 tx=0x7fa31002e000 comp rx=0 tx=0).stop 2024-04-25T08:57:57.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.895+0000 7fa32461f640 1 -- 172.21.15.43:0/2671096208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa31c0ce5e0 msgr2=0x7fa31c1564f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:57.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.895+0000 7fa32461f640 1 --2- 172.21.15.43:0/2671096208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa31c0ce5e0 0x7fa31c1564f0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fa30c00c730 tx=0x7fa30c00cbf0 comp rx=0 tx=0).stop 2024-04-25T08:57:57.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.895+0000 7fa32461f640 1 -- 172.21.15.43:0/2671096208 shutdown_connections 2024-04-25T08:57:57.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.895+0000 7fa32461f640 1 --2- 172.21.15.43:0/2671096208 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa30806b4d0 0x7fa30806d990 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:57.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.895+0000 7fa32461f640 1 --2- 172.21.15.43:0/2671096208 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa31c161390 0x7fa31c0cafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:57.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.895+0000 7fa32461f640 1 --2- 172.21.15.43:0/2671096208 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa31c0d09b0 0x7fa31c0caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:57.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.895+0000 7fa32461f640 1 --2- 172.21.15.43:0/2671096208 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa31c0ce5e0 0x7fa31c1564f0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:57.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.895+0000 7fa32461f640 1 -- 172.21.15.43:0/2671096208 >> 172.21.15.43:0/2671096208 conn(0x7fa31c09cd20 msgr2=0x7fa31c0b7140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:57.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.895+0000 7fa32461f640 1 -- 172.21.15.43:0/2671096208 shutdown_connections 2024-04-25T08:57:57.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:57.895+0000 7fa32461f640 1 -- 172.21.15.43:0/2671096208 wait complete. 2024-04-25T08:57:57.896 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T08:57:57.896 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:57:57.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.008+0000 7fa774f89640 1 Processor -- start 2024-04-25T08:57:57.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.008+0000 7fa774f89640 1 -- start start 2024-04-25T08:57:57.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.009+0000 7fa774f89640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fa77015dd00 0x7fa7700cc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:57.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.009+0000 7fa774f89640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa7700ce170 0x7fa770161760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:57.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.009+0000 7fa774f89640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7700c4ed0 con 0x7fa7700c5f70 2024-04-25T08:57:57.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.009+0000 7fa774f89640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7700c51b0 con 0x7fa7700ce170 2024-04-25T08:57:57.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.009+0000 7fa774f89640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7700c5490 con 0x7fa77015dd00 2024-04-25T08:57:57.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.009+0000 7fa76ed9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa7700ce170 0x7fa770161760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:57.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.009+0000 7fa76dd9b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa7700c5f70 0x7fa7700cda60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45364/0 (socket says 172.21.15.43:45364) 2024-04-25T08:57:57.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.009+0000 7fa76dd9b640 1 -- 172.21.15.43:0/3280995550 learned_addr learned my addr 172.21.15.43:0/3280995550 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:57.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.009+0000 7fa76e59c640 1 --2- 172.21.15.43:0/3280995550 >> v2:172.21.15.193:3300/0 conn(0x7fa77015dd00 0x7fa7700cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:57.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.009+0000 7fa76ed9d640 1 -- 172.21.15.43:0/3280995550 >> v2:172.21.15.193:3300/0 conn(0x7fa77015dd00 msgr2=0x7fa7700cc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:58.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.009+0000 7fa76ed9d640 1 --2- 172.21.15.43:0/3280995550 >> v2:172.21.15.193:3300/0 conn(0x7fa77015dd00 0x7fa7700cc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:58.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.009+0000 7fa76ed9d640 1 -- 172.21.15.43:0/3280995550 >> v1:172.21.15.43:6789/0 conn(0x7fa7700c5f70 legacy=0x7fa7700cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:58.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.009+0000 7fa76ed9d640 1 -- 172.21.15.43:0/3280995550 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7700c5770 con 0x7fa7700ce170 2024-04-25T08:57:58.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.009+0000 7fa76e59c640 1 --2- 172.21.15.43:0/3280995550 >> v2:172.21.15.193:3300/0 conn(0x7fa77015dd00 0x7fa7700cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:58.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.010+0000 7fa76ed9d640 1 --2- 172.21.15.43:0/3280995550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7700ce170 0x7fa770161760 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa7700676a0 tx=0x7fa76802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4590119ed4aa8144 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:58.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.010+0000 7fa76d59a640 1 -- 172.21.15.43:0/3280995550 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa768007740 con 0x7fa7700ce170 2024-04-25T08:57:58.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.010+0000 7fa76d59a640 1 -- 172.21.15.43:0/3280995550 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7680078e0 con 0x7fa7700ce170 2024-04-25T08:57:58.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.010+0000 7fa76d59a640 1 -- 172.21.15.43:0/3280995550 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa768007cb0 con 0x7fa7700ce170 2024-04-25T08:57:58.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.010+0000 7fa774f89640 1 -- 172.21.15.43:0/3280995550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7700ce170 msgr2=0x7fa770161760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:58.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.010+0000 7fa774f89640 1 --2- 172.21.15.43:0/3280995550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7700ce170 0x7fa770161760 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fa7700676a0 tx=0x7fa76802a680 comp rx=0 tx=0).stop 2024-04-25T08:57:58.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.011+0000 7fa774f89640 1 -- 172.21.15.43:0/3280995550 shutdown_connections 2024-04-25T08:57:58.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.011+0000 7fa774f89640 1 --2- 172.21.15.43:0/3280995550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7700ce170 0x7fa770161760 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:58.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.011+0000 7fa774f89640 1 --2- 172.21.15.43:0/3280995550 >> v2:172.21.15.193:3300/0 conn(0x7fa77015dd00 0x7fa7700cc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:58.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.011+0000 7fa774f89640 1 -- 172.21.15.43:0/3280995550 >> 172.21.15.43:0/3280995550 conn(0x7fa77006ab30 msgr2=0x7fa77006af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:58.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.011+0000 7fa774f89640 1 -- 172.21.15.43:0/3280995550 shutdown_connections 2024-04-25T08:57:58.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.011+0000 7fa774f89640 1 -- 172.21.15.43:0/3280995550 wait complete. 2024-04-25T08:57:58.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.012+0000 7fa774f89640 1 Processor -- start 2024-04-25T08:57:58.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.012+0000 7fa774f89640 1 -- start start 2024-04-25T08:57:58.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.013+0000 7fa774f89640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7700c5f70 0x7fa770154ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:58.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.013+0000 7fa774f89640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7700ce170 0x7fa770158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:58.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.013+0000 7fa774f89640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa77015dd00 0x7fa7700ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:58.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.013+0000 7fa774f89640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7700c6580 con 0x7fa77015dd00 2024-04-25T08:57:58.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.013+0000 7fa774f89640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7700c6400 con 0x7fa7700c5f70 2024-04-25T08:57:58.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.013+0000 7fa774f89640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7700c6700 con 0x7fa7700ce170 2024-04-25T08:57:58.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.013+0000 7fa76ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa77015dd00 0x7fa7700ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:58.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.013+0000 7fa76ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa77015dd00 0x7fa7700ce5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50094/0 (socket says 172.21.15.43:50094) 2024-04-25T08:57:58.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.013+0000 7fa76ed9d640 1 -- 172.21.15.43:0/308404782 learned_addr learned my addr 172.21.15.43:0/308404782 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:58.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.013+0000 7fa76ed9d640 1 -- 172.21.15.43:0/308404782 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7700ce170 msgr2=0x7fa770158c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:58.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.013+0000 7fa76ed9d640 1 --2- 172.21.15.43:0/308404782 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7700ce170 0x7fa770158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:58.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.013+0000 7fa76ed9d640 1 -- 172.21.15.43:0/308404782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7700c5f70 msgr2=0x7fa770154ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:58.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.013+0000 7fa76ed9d640 1 --2- 172.21.15.43:0/308404782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7700c5f70 0x7fa770154ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:58.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.013+0000 7fa76ed9d640 1 -- 172.21.15.43:0/308404782 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7700b3620 con 0x7fa77015dd00 2024-04-25T08:57:58.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.013+0000 7fa76ed9d640 1 --2- 172.21.15.43:0/308404782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa77015dd00 0x7fa7700ce5c0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fa768009e50 tx=0x7fa768004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:58.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.013+0000 7fa75effd640 1 -- 172.21.15.43:0/308404782 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa768040070 con 0x7fa77015dd00 2024-04-25T08:57:58.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.013+0000 7fa75effd640 1 -- 172.21.15.43:0/308404782 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa768031cc0 con 0x7fa77015dd00 2024-04-25T08:57:58.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.013+0000 7fa774f89640 1 -- 172.21.15.43:0/308404782 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7700b3900 con 0x7fa77015dd00 2024-04-25T08:57:58.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.013+0000 7fa75effd640 1 -- 172.21.15.43:0/308404782 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa768045680 con 0x7fa77015dd00 2024-04-25T08:57:58.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.014+0000 7fa774f89640 1 -- 172.21.15.43:0/308404782 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7700b3ec0 con 0x7fa77015dd00 2024-04-25T08:57:58.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.014+0000 7fa774f89640 1 -- 172.21.15.43:0/308404782 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa770074ef0 con 0x7fa77015dd00 2024-04-25T08:57:58.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.015+0000 7fa75effd640 1 -- 172.21.15.43:0/308404782 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa768039040 con 0x7fa77015dd00 2024-04-25T08:57:58.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.015+0000 7fa75effd640 1 --2- 172.21.15.43:0/308404782 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa75006b4d0 0x7fa75006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:58.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.015+0000 7fa75effd640 1 -- 172.21.15.43:0/308404782 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa7680abb30 con 0x7fa77015dd00 2024-04-25T08:57:58.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.016+0000 7fa76e59c640 1 --2- 172.21.15.43:0/308404782 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa75006b4d0 0x7fa75006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:58.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.016+0000 7fa76e59c640 1 --2- 172.21.15.43:0/308404782 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa75006b4d0 0x7fa75006d990 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa760006fd0 tx=0x7fa760008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:58.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.020+0000 7fa75effd640 1 -- 172.21.15.43:0/308404782 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa76807ab00 con 0x7fa77015dd00 2024-04-25T08:57:58.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.195+0000 7fa774f89640 1 -- 172.21.15.43:0/308404782 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa7700669c0 con 0x7fa75006b4d0 2024-04-25T08:57:58.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.201+0000 7fa75effd640 1 -- 172.21.15.43:0/308404782 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+239771 (secure 0 0 0) 0x7fa7700669c0 con 0x7fa75006b4d0 2024-04-25T08:57:58.193 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:57:58.194 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-25T08:57:58.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.204+0000 7fa774f89640 1 -- 172.21.15.43:0/308404782 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa75006b4d0 msgr2=0x7fa75006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:58.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.204+0000 7fa774f89640 1 --2- 172.21.15.43:0/308404782 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa75006b4d0 0x7fa75006d990 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fa760006fd0 tx=0x7fa760008040 comp rx=0 tx=0).stop 2024-04-25T08:57:58.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.204+0000 7fa774f89640 1 -- 172.21.15.43:0/308404782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa77015dd00 msgr2=0x7fa7700ce5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:58.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.204+0000 7fa774f89640 1 --2- 172.21.15.43:0/308404782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa77015dd00 0x7fa7700ce5c0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fa768009e50 tx=0x7fa768004a40 comp rx=0 tx=0).stop 2024-04-25T08:57:58.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.204+0000 7fa774f89640 1 -- 172.21.15.43:0/308404782 shutdown_connections 2024-04-25T08:57:58.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.204+0000 7fa774f89640 1 --2- 172.21.15.43:0/308404782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa77015dd00 0x7fa7700ce5c0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:58.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.204+0000 7fa774f89640 1 --2- 172.21.15.43:0/308404782 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7700ce170 0x7fa770158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:58.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.204+0000 7fa774f89640 1 --2- 172.21.15.43:0/308404782 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa75006b4d0 0x7fa75006d990 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:58.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.204+0000 7fa774f89640 1 --2- 172.21.15.43:0/308404782 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7700c5f70 0x7fa770154ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:58.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.204+0000 7fa774f89640 1 -- 172.21.15.43:0/308404782 >> 172.21.15.43:0/308404782 conn(0x7fa77006ab30 msgr2=0x7fa7700b1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:58.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.204+0000 7fa774f89640 1 -- 172.21.15.43:0/308404782 shutdown_connections 2024-04-25T08:57:58.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.204+0000 7fa774f89640 1 -- 172.21.15.43:0/308404782 wait complete. 2024-04-25T08:57:58.204 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":171,"stamp":"2024-04-25T08:57:57.747195+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":338988,"kb_used_data":6252,"kb_used_omap":12,"kb_used_meta":332595,"kb_avail":754635732,"statfs":{"total":773094113280,"available":772746989568,"internally_reserved":0,"allocated":6402048,"data_stored":3323523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":340577841},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.535207"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.216743+0000","last_change":"2024-04-25T08:57:53.216743+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.216743+0000","last_undegraded":"2024-04-25T08:57:53.216743+0000","last_fullsized":"2024-04-25T08:57:53.216743+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.215243+0000","last_change":"2024-04-25T08:57:53.215243+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.215243+0000","last_undegraded":"2024-04-25T08:57:53.215243+0000","last_fullsized":"2024-04-25T08:57:53.215243+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.216709+0000","last_change":"2024-04-25T08:57:53.216709+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.216709+0000","last_undegraded":"2024-04-25T08:57:53.216709+0000","last_fullsized":"2024-04-25T08:57:53.216709+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.214788+0000","last_change":"2024-04-25T08:57:53.214788+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.214788+0000","last_undegraded":"2024-04-25T08:57:53.214788+0000","last_fullsized":"2024-04-25T08:57:53.214788+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.217098+0000","last_change":"2024-04-25T08:57:53.217098+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.217098+0000","last_undegraded":"2024-04-25T08:57:53.217098+0000","last_fullsized":"2024-04-25T08:57:53.217098+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.216664+0000","last_change":"2024-04-25T08:57:53.216664+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.216664+0000","last_undegraded":"2024-04-25T08:57:53.216664+0000","last_fullsized":"2024-04-25T08:57:53.216664+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.214289+0000","last_change":"2024-04-25T08:57:53.214289+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.214289+0000","last_undegraded":"2024-04-25T08:57:53.214289+0000","last_fullsized":"2024-04-25T08:57:53.214289+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.216716+0000","last_change":"2024-04-25T08:57:53.216716+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.216716+0000","last_undegraded":"2024-04-25T08:57:53.216716+0000","last_fullsized":"2024-04-25T08:57:53.216716+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.213648+0000","last_change":"2024-04-25T08:57:53.213648+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.213648+0000","last_undegraded":"2024-04-25T08:57:53.213648+0000","last_fullsized":"2024-04-25T08:57:53.213648+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.214791+0000","last_change":"2024-04-25T08:57:53.214791+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.214791+0000","last_undegraded":"2024-04-25T08:57:53.214791+0000","last_fullsized":"2024-04-25T08:57:53.214791+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.214000+0000","last_change":"2024-04-25T08:57:53.214000+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.214000+0000","last_undegraded":"2024-04-25T08:57:53.214000+0000","last_fullsized":"2024-04-25T08:57:53.214000+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.216775+0000","last_change":"2024-04-25T08:57:53.216775+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.216775+0000","last_undegraded":"2024-04-25T08:57:53.216775+0000","last_fullsized":"2024-04-25T08:57:53.216775+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.214777+0000","last_change":"2024-04-25T08:57:53.214777+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.214777+0000","last_undegraded":"2024-04-25T08:57:53.214777+0000","last_fullsized":"2024-04-25T08:57:53.214777+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.219307+0000","last_change":"2024-04-25T08:57:53.219307+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.219307+0000","last_undegraded":"2024-04-25T08:57:53.219307+0000","last_fullsized":"2024-04-25T08:57:53.219307+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.213691+0000","last_change":"2024-04-25T08:57:53.213691+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.213691+0000","last_undegraded":"2024-04-25T08:57:53.213691+0000","last_fullsized":"2024-04-25T08:57:53.213691+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.213640+0000","last_change":"2024-04-25T08:57:53.213640+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.213640+0000","last_undegraded":"2024-04-25T08:57:53.213640+0000","last_fullsized":"2024-04-25T08:57:53.213640+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.214281+0000","last_change":"2024-04-25T08:57:53.214281+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.214281+0000","last_undegraded":"2024-04-25T08:57:53.214281+0000","last_fullsized":"2024-04-25T08:57:53.214281+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.218465+0000","last_change":"2024-04-25T08:57:53.218465+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.218465+0000","last_undegraded":"2024-04-25T08:57:53.218465+0000","last_fullsized":"2024-04-25T08:57:53.218465+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.213640+0000","last_change":"2024-04-25T08:57:53.213640+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.213640+0000","last_undegraded":"2024-04-25T08:57:53.213640+0000","last_fullsized":"2024-04-25T08:57:53.213640+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.216771+0000","last_change":"2024-04-25T08:57:53.216771+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.216771+0000","last_undegraded":"2024-04-25T08:57:53.216771+0000","last_fullsized":"2024-04-25T08:57:53.216771+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.214220+0000","last_change":"2024-04-25T08:57:53.214220+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.214220+0000","last_undegraded":"2024-04-25T08:57:53.214220+0000","last_fullsized":"2024-04-25T08:57:53.214220+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.215905+0000","last_change":"2024-04-25T08:57:53.215905+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.215905+0000","last_undegraded":"2024-04-25T08:57:53.215905+0000","last_fullsized":"2024-04-25T08:57:53.215905+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.213640+0000","last_change":"2024-04-25T08:57:53.213640+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.213640+0000","last_undegraded":"2024-04-25T08:57:53.213640+0000","last_fullsized":"2024-04-25T08:57:53.213640+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.215838+0000","last_change":"2024-04-25T08:57:53.215838+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.215838+0000","last_undegraded":"2024-04-25T08:57:53.215838+0000","last_fullsized":"2024-04-25T08:57:53.215838+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.217997+0000","last_change":"2024-04-25T08:57:53.217997+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.217997+0000","last_undegraded":"2024-04-25T08:57:53.217997+0000","last_fullsized":"2024-04-25T08:57:53.217997+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.219296+0000","last_change":"2024-04-25T08:57:53.219296+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.219296+0000","last_undegraded":"2024-04-25T08:57:53.219296+0000","last_fullsized":"2024-04-25T08:57:53.219296+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.214302+0000","last_change":"2024-04-25T08:57:53.214302+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.214302+0000","last_undegraded":"2024-04-25T08:57:53.214302+0000","last_fullsized":"2024-04-25T08:57:53.214302+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.216286+0000","last_change":"2024-04-25T08:57:53.216286+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.216286+0000","last_undegraded":"2024-04-25T08:57:53.216286+0000","last_fullsized":"2024-04-25T08:57:53.216286+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.216724+0000","last_change":"2024-04-25T08:57:53.216724+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.216724+0000","last_undegraded":"2024-04-25T08:57:53.216724+0000","last_fullsized":"2024-04-25T08:57:53.216724+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.215814+0000","last_change":"2024-04-25T08:57:53.215814+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.215814+0000","last_undegraded":"2024-04-25T08:57:53.215814+0000","last_fullsized":"2024-04-25T08:57:53.215814+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.214076+0000","last_change":"2024-04-25T08:57:53.214076+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.214076+0000","last_undegraded":"2024-04-25T08:57:53.214076+0000","last_fullsized":"2024-04-25T08:57:53.214076+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.219243+0000","last_change":"2024-04-25T08:57:53.219243+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.219243+0000","last_undegraded":"2024-04-25T08:57:53.219243+0000","last_fullsized":"2024-04-25T08:57:53.219243+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":234,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:57:53.214527+0000","last_change":"2024-04-25T08:54:00.818044+0000","last_active":"2024-04-25T08:57:53.214527+0000","last_peered":"2024-04-25T08:57:53.214527+0000","last_clean":"2024-04-25T08:57:53.214527+0000","last_became_active":"2024-04-25T08:54:00.817098+0000","last_became_peered":"2024-04-25T08:54:00.817098+0000","last_unstale":"2024-04-25T08:57:53.214527+0000","last_undegraded":"2024-04-25T08:57:53.214527+0000","last_fullsized":"2024-04-25T08:57:53.214527+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_clean_scrub_stamp":"2024-04-25T08:53:59.739677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:48.129513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40720,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331120,"statfs":{"total":96636764160,"available":96595066880,"internally_reserved":0,"allocated":1241088,"data_stored":856035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40136,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331704,"statfs":{"total":96636764160,"available":96595664896,"internally_reserved":0,"allocated":647168,"data_stored":265667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574900,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40148,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331692,"statfs":{"total":96636764160,"available":96595652608,"internally_reserved":0,"allocated":659456,"data_stored":271484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45720,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326120,"statfs":{"total":96636764160,"available":96589946880,"internally_reserved":0,"allocated":659456,"data_stored":271484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54020,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317820,"statfs":{"total":96636764160,"available":96581447680,"internally_reserved":0,"allocated":647168,"data_stored":265667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":40152,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331688,"statfs":{"total":96636764160,"available":96595648512,"internally_reserved":0,"allocated":659456,"data_stored":271484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49860,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321980,"statfs":{"total":96636764160,"available":96585707520,"internally_reserved":0,"allocated":647168,"data_stored":265667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574899,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28232,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343608,"statfs":{"total":96636764160,"available":96607854592,"internally_reserved":0,"allocated":1241088,"data_stored":856035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:57:58.208 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:57:58.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.356+0000 7f4e6e4cb640 1 Processor -- start 2024-04-25T08:57:58.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.356+0000 7f4e6e4cb640 1 -- start start 2024-04-25T08:57:58.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.356+0000 7f4e6e4cb640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4e6815dce0 0x7f4e6815ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:58.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.357+0000 7f4e6e4cb640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e680d2920 con 0x7f4e680ce550 2024-04-25T08:57:58.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.357+0000 7f4e6e4cb640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e680cee80 con 0x7f4e680d0af0 2024-04-25T08:57:58.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.357+0000 7f4e6e4cb640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e680cf160 con 0x7f4e6815dce0 2024-04-25T08:57:58.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.357+0000 7f4e67fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4e6815dce0 0x7f4e6815ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:58.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.357+0000 7f4e677fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4e680ce550 0x7f4e680d03e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45380/0 (socket says 172.21.15.43:45380) 2024-04-25T08:57:58.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.357+0000 7f4e677fe640 1 -- 172.21.15.43:0/1885758690 learned_addr learned my addr 172.21.15.43:0/1885758690 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:58.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.357+0000 7f4e67fff640 1 -- 172.21.15.43:0/1885758690 >> v1:172.21.15.193:6789/0 conn(0x7f4e680d0af0 legacy=0x7f4e68160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:58.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.357+0000 7f4e67fff640 1 -- 172.21.15.43:0/1885758690 >> v1:172.21.15.43:6789/0 conn(0x7f4e680ce550 legacy=0x7f4e680d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:58.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.357+0000 7f4e67fff640 1 -- 172.21.15.43:0/1885758690 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e680cf440 con 0x7f4e6815dce0 2024-04-25T08:57:58.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.357+0000 7f4e66ffd640 1 -- 172.21.15.43:0/1885758690 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3501319286 0 0) 0x7f4e680d2920 con 0x7f4e680ce550 2024-04-25T08:57:58.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.358+0000 7f4e67fff640 1 --2- 172.21.15.43:0/1885758690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e6815dce0 0x7f4e6815ef50 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f4e580097a0 tx=0x7f4e5802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e63196adea76a57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:58.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.358+0000 7f4e66ffd640 1 -- 172.21.15.43:0/1885758690 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e58037070 con 0x7f4e6815dce0 2024-04-25T08:57:58.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.358+0000 7f4e66ffd640 1 -- 172.21.15.43:0/1885758690 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e5802ae50 con 0x7f4e6815dce0 2024-04-25T08:57:58.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.358+0000 7f4e66ffd640 1 -- 172.21.15.43:0/1885758690 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e58032660 con 0x7f4e6815dce0 2024-04-25T08:57:58.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.358+0000 7f4e6e4cb640 1 -- 172.21.15.43:0/1885758690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e6815dce0 msgr2=0x7f4e6815ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:58.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.358+0000 7f4e6e4cb640 1 --2- 172.21.15.43:0/1885758690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e6815dce0 0x7f4e6815ef50 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f4e580097a0 tx=0x7f4e5802a140 comp rx=0 tx=0).stop 2024-04-25T08:57:58.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.358+0000 7f4e6e4cb640 1 -- 172.21.15.43:0/1885758690 shutdown_connections 2024-04-25T08:57:58.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.358+0000 7f4e6e4cb640 1 --2- 172.21.15.43:0/1885758690 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e6815dce0 0x7f4e6815ef50 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:58.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.358+0000 7f4e6e4cb640 1 -- 172.21.15.43:0/1885758690 >> 172.21.15.43:0/1885758690 conn(0x7f4e6809cd00 msgr2=0x7f4e680d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:58.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.358+0000 7f4e6e4cb640 1 -- 172.21.15.43:0/1885758690 shutdown_connections 2024-04-25T08:57:58.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.359+0000 7f4e6e4cb640 1 -- 172.21.15.43:0/1885758690 wait complete. 2024-04-25T08:57:58.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.360+0000 7f4e6e4cb640 1 Processor -- start 2024-04-25T08:57:58.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.360+0000 7f4e6e4cb640 1 -- start start 2024-04-25T08:57:58.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.360+0000 7f4e6e4cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e680ce550 0x7f4e680a10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:58.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.360+0000 7f4e6e4cb640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4e680d0af0 0x7f4e6809f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:58.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.360+0000 7f4e6e4cb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e6815dce0 0x7f4e68154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:58.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.361+0000 7f4e6e4cb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6815e0c0 con 0x7f4e680ce550 2024-04-25T08:57:58.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.361+0000 7f4e6e4cb640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6815f6b0 con 0x7f4e6815dce0 2024-04-25T08:57:58.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.361+0000 7f4e6e4cb640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e680ce9c0 con 0x7f4e680d0af0 2024-04-25T08:57:58.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.361+0000 7f4e67fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e680ce550 0x7f4e680a10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:58.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.361+0000 7f4e67fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e680ce550 0x7f4e680a10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50110/0 (socket says 172.21.15.43:50110) 2024-04-25T08:57:58.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.361+0000 7f4e67fff640 1 -- 172.21.15.43:0/2160157775 learned_addr learned my addr 172.21.15.43:0/2160157775 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:58.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.361+0000 7f4e6ca68640 1 --2- 172.21.15.43:0/2160157775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e6815dce0 0x7f4e68154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:58.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.361+0000 7f4e677fe640 1 --2- 172.21.15.43:0/2160157775 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4e680d0af0 0x7f4e6809f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:58.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.361+0000 7f4e67fff640 1 -- 172.21.15.43:0/2160157775 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4e680d0af0 msgr2=0x7f4e6809f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:58.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.361+0000 7f4e67fff640 1 --2- 172.21.15.43:0/2160157775 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4e680d0af0 0x7f4e6809f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:58.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.361+0000 7f4e67fff640 1 -- 172.21.15.43:0/2160157775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e6815dce0 msgr2=0x7f4e68154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:58.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.361+0000 7f4e67fff640 1 --2- 172.21.15.43:0/2160157775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e6815dce0 0x7f4e68154af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:58.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.361+0000 7f4e67fff640 1 -- 172.21.15.43:0/2160157775 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e6809d770 con 0x7f4e680ce550 2024-04-25T08:57:58.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.361+0000 7f4e6ca68640 1 --2- 172.21.15.43:0/2160157775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e6815dce0 0x7f4e68154af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:58.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.361+0000 7f4e67fff640 1 --2- 172.21.15.43:0/2160157775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e680ce550 0x7f4e680a10f0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f4e58000c00 tx=0x7f4e580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:58.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.361+0000 7f4e64ff9640 1 -- 172.21.15.43:0/2160157775 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e58037040 con 0x7f4e680ce550 2024-04-25T08:57:58.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.361+0000 7f4e64ff9640 1 -- 172.21.15.43:0/2160157775 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e58004030 con 0x7f4e680ce550 2024-04-25T08:57:58.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.361+0000 7f4e6e4cb640 1 -- 172.21.15.43:0/2160157775 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e6809da50 con 0x7f4e680ce550 2024-04-25T08:57:58.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.361+0000 7f4e64ff9640 1 -- 172.21.15.43:0/2160157775 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e58006e10 con 0x7f4e680ce550 2024-04-25T08:57:58.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.361+0000 7f4e6e4cb640 1 -- 172.21.15.43:0/2160157775 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e6809e010 con 0x7f4e680ce550 2024-04-25T08:57:58.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.363+0000 7f4e6e4cb640 1 -- 172.21.15.43:0/2160157775 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e680680f0 con 0x7f4e680ce550 2024-04-25T08:57:58.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.363+0000 7f4e64ff9640 1 -- 172.21.15.43:0/2160157775 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4e5803d650 con 0x7f4e680ce550 2024-04-25T08:57:58.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.363+0000 7f4e64ff9640 1 --2- 172.21.15.43:0/2160157775 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4e5406b500 0x7f4e5406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:58.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.363+0000 7f4e64ff9640 1 -- 172.21.15.43:0/2160157775 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4e580aa970 con 0x7f4e680ce550 2024-04-25T08:57:58.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.363+0000 7f4e677fe640 1 --2- 172.21.15.43:0/2160157775 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4e5406b500 0x7f4e5406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:58.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.364+0000 7f4e677fe640 1 --2- 172.21.15.43:0/2160157775 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4e5406b500 0x7f4e5406d9c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4e680ca520 tx=0x7f4e5c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:58.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.367+0000 7f4e64ff9640 1 -- 172.21.15.43:0/2160157775 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4e58042020 con 0x7f4e680ce550 2024-04-25T08:57:58.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.546+0000 7f4e6e4cb640 1 -- 172.21.15.43:0/2160157775 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4e68074ef0 con 0x7f4e5406b500 2024-04-25T08:57:58.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.552+0000 7f4e64ff9640 1 -- 172.21.15.43:0/2160157775 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+239771 (secure 0 0 0) 0x7f4e68074ef0 con 0x7f4e5406b500 2024-04-25T08:57:58.544 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:57:58.545 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-25T08:57:58.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.555+0000 7f4e6e4cb640 1 -- 172.21.15.43:0/2160157775 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4e5406b500 msgr2=0x7f4e5406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:58.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.555+0000 7f4e6e4cb640 1 --2- 172.21.15.43:0/2160157775 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4e5406b500 0x7f4e5406d9c0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4e680ca520 tx=0x7f4e5c009040 comp rx=0 tx=0).stop 2024-04-25T08:57:58.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.555+0000 7f4e6e4cb640 1 -- 172.21.15.43:0/2160157775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e680ce550 msgr2=0x7f4e680a10f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:58.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.555+0000 7f4e6e4cb640 1 --2- 172.21.15.43:0/2160157775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e680ce550 0x7f4e680a10f0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f4e58000c00 tx=0x7f4e580046c0 comp rx=0 tx=0).stop 2024-04-25T08:57:58.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.555+0000 7f4e6e4cb640 1 -- 172.21.15.43:0/2160157775 shutdown_connections 2024-04-25T08:57:58.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.555+0000 7f4e6e4cb640 1 --2- 172.21.15.43:0/2160157775 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4e5406b500 0x7f4e5406d9c0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:58.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.555+0000 7f4e6e4cb640 1 --2- 172.21.15.43:0/2160157775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4e6815dce0 0x7f4e68154af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:58.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.555+0000 7f4e6e4cb640 1 --2- 172.21.15.43:0/2160157775 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4e680d0af0 0x7f4e6809f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:58.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.555+0000 7f4e6e4cb640 1 --2- 172.21.15.43:0/2160157775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4e680ce550 0x7f4e680a10f0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:58.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.555+0000 7f4e6e4cb640 1 -- 172.21.15.43:0/2160157775 >> 172.21.15.43:0/2160157775 conn(0x7f4e6809cd00 msgr2=0x7f4e680c8060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:58.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.555+0000 7f4e6e4cb640 1 -- 172.21.15.43:0/2160157775 shutdown_connections 2024-04-25T08:57:58.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.555+0000 7f4e6e4cb640 1 -- 172.21.15.43:0/2160157775 wait complete. 2024-04-25T08:57:58.555 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":171,"stamp":"2024-04-25T08:57:57.747195+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":338988,"kb_used_data":6252,"kb_used_omap":12,"kb_used_meta":332595,"kb_avail":754635732,"statfs":{"total":773094113280,"available":772746989568,"internally_reserved":0,"allocated":6402048,"data_stored":3323523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":340577841},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.535207"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.216743+0000","last_change":"2024-04-25T08:57:53.216743+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.216743+0000","last_undegraded":"2024-04-25T08:57:53.216743+0000","last_fullsized":"2024-04-25T08:57:53.216743+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.215243+0000","last_change":"2024-04-25T08:57:53.215243+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.215243+0000","last_undegraded":"2024-04-25T08:57:53.215243+0000","last_fullsized":"2024-04-25T08:57:53.215243+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.216709+0000","last_change":"2024-04-25T08:57:53.216709+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.216709+0000","last_undegraded":"2024-04-25T08:57:53.216709+0000","last_fullsized":"2024-04-25T08:57:53.216709+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.214788+0000","last_change":"2024-04-25T08:57:53.214788+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.214788+0000","last_undegraded":"2024-04-25T08:57:53.214788+0000","last_fullsized":"2024-04-25T08:57:53.214788+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.217098+0000","last_change":"2024-04-25T08:57:53.217098+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.217098+0000","last_undegraded":"2024-04-25T08:57:53.217098+0000","last_fullsized":"2024-04-25T08:57:53.217098+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.216664+0000","last_change":"2024-04-25T08:57:53.216664+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.216664+0000","last_undegraded":"2024-04-25T08:57:53.216664+0000","last_fullsized":"2024-04-25T08:57:53.216664+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.214289+0000","last_change":"2024-04-25T08:57:53.214289+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.214289+0000","last_undegraded":"2024-04-25T08:57:53.214289+0000","last_fullsized":"2024-04-25T08:57:53.214289+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.216716+0000","last_change":"2024-04-25T08:57:53.216716+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.216716+0000","last_undegraded":"2024-04-25T08:57:53.216716+0000","last_fullsized":"2024-04-25T08:57:53.216716+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.213648+0000","last_change":"2024-04-25T08:57:53.213648+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.213648+0000","last_undegraded":"2024-04-25T08:57:53.213648+0000","last_fullsized":"2024-04-25T08:57:53.213648+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.214791+0000","last_change":"2024-04-25T08:57:53.214791+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.214791+0000","last_undegraded":"2024-04-25T08:57:53.214791+0000","last_fullsized":"2024-04-25T08:57:53.214791+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.214000+0000","last_change":"2024-04-25T08:57:53.214000+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.214000+0000","last_undegraded":"2024-04-25T08:57:53.214000+0000","last_fullsized":"2024-04-25T08:57:53.214000+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.216775+0000","last_change":"2024-04-25T08:57:53.216775+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.216775+0000","last_undegraded":"2024-04-25T08:57:53.216775+0000","last_fullsized":"2024-04-25T08:57:53.216775+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.214777+0000","last_change":"2024-04-25T08:57:53.214777+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.214777+0000","last_undegraded":"2024-04-25T08:57:53.214777+0000","last_fullsized":"2024-04-25T08:57:53.214777+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.219307+0000","last_change":"2024-04-25T08:57:53.219307+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.219307+0000","last_undegraded":"2024-04-25T08:57:53.219307+0000","last_fullsized":"2024-04-25T08:57:53.219307+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.213691+0000","last_change":"2024-04-25T08:57:53.213691+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.213691+0000","last_undegraded":"2024-04-25T08:57:53.213691+0000","last_fullsized":"2024-04-25T08:57:53.213691+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.213640+0000","last_change":"2024-04-25T08:57:53.213640+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.213640+0000","last_undegraded":"2024-04-25T08:57:53.213640+0000","last_fullsized":"2024-04-25T08:57:53.213640+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.214281+0000","last_change":"2024-04-25T08:57:53.214281+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.214281+0000","last_undegraded":"2024-04-25T08:57:53.214281+0000","last_fullsized":"2024-04-25T08:57:53.214281+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.218465+0000","last_change":"2024-04-25T08:57:53.218465+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.218465+0000","last_undegraded":"2024-04-25T08:57:53.218465+0000","last_fullsized":"2024-04-25T08:57:53.218465+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.213640+0000","last_change":"2024-04-25T08:57:53.213640+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.213640+0000","last_undegraded":"2024-04-25T08:57:53.213640+0000","last_fullsized":"2024-04-25T08:57:53.213640+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.216771+0000","last_change":"2024-04-25T08:57:53.216771+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.216771+0000","last_undegraded":"2024-04-25T08:57:53.216771+0000","last_fullsized":"2024-04-25T08:57:53.216771+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.214220+0000","last_change":"2024-04-25T08:57:53.214220+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.214220+0000","last_undegraded":"2024-04-25T08:57:53.214220+0000","last_fullsized":"2024-04-25T08:57:53.214220+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.215905+0000","last_change":"2024-04-25T08:57:53.215905+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.215905+0000","last_undegraded":"2024-04-25T08:57:53.215905+0000","last_fullsized":"2024-04-25T08:57:53.215905+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.213640+0000","last_change":"2024-04-25T08:57:53.213640+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.213640+0000","last_undegraded":"2024-04-25T08:57:53.213640+0000","last_fullsized":"2024-04-25T08:57:53.213640+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.215838+0000","last_change":"2024-04-25T08:57:53.215838+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.215838+0000","last_undegraded":"2024-04-25T08:57:53.215838+0000","last_fullsized":"2024-04-25T08:57:53.215838+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.217997+0000","last_change":"2024-04-25T08:57:53.217997+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.217997+0000","last_undegraded":"2024-04-25T08:57:53.217997+0000","last_fullsized":"2024-04-25T08:57:53.217997+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.219296+0000","last_change":"2024-04-25T08:57:53.219296+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.219296+0000","last_undegraded":"2024-04-25T08:57:53.219296+0000","last_fullsized":"2024-04-25T08:57:53.219296+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.214302+0000","last_change":"2024-04-25T08:57:53.214302+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.214302+0000","last_undegraded":"2024-04-25T08:57:53.214302+0000","last_fullsized":"2024-04-25T08:57:53.214302+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.216286+0000","last_change":"2024-04-25T08:57:53.216286+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.216286+0000","last_undegraded":"2024-04-25T08:57:53.216286+0000","last_fullsized":"2024-04-25T08:57:53.216286+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.216724+0000","last_change":"2024-04-25T08:57:53.216724+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.216724+0000","last_undegraded":"2024-04-25T08:57:53.216724+0000","last_fullsized":"2024-04-25T08:57:53.216724+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.215814+0000","last_change":"2024-04-25T08:57:53.215814+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.215814+0000","last_undegraded":"2024-04-25T08:57:53.215814+0000","last_fullsized":"2024-04-25T08:57:53.215814+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.214076+0000","last_change":"2024-04-25T08:57:53.214076+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.214076+0000","last_undegraded":"2024-04-25T08:57:53.214076+0000","last_fullsized":"2024-04-25T08:57:53.214076+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.219243+0000","last_change":"2024-04-25T08:57:53.219243+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.219243+0000","last_undegraded":"2024-04-25T08:57:53.219243+0000","last_fullsized":"2024-04-25T08:57:53.219243+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":234,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:57:53.214527+0000","last_change":"2024-04-25T08:54:00.818044+0000","last_active":"2024-04-25T08:57:53.214527+0000","last_peered":"2024-04-25T08:57:53.214527+0000","last_clean":"2024-04-25T08:57:53.214527+0000","last_became_active":"2024-04-25T08:54:00.817098+0000","last_became_peered":"2024-04-25T08:54:00.817098+0000","last_unstale":"2024-04-25T08:57:53.214527+0000","last_undegraded":"2024-04-25T08:57:53.214527+0000","last_fullsized":"2024-04-25T08:57:53.214527+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_clean_scrub_stamp":"2024-04-25T08:53:59.739677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:48.129513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40720,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331120,"statfs":{"total":96636764160,"available":96595066880,"internally_reserved":0,"allocated":1241088,"data_stored":856035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40136,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331704,"statfs":{"total":96636764160,"available":96595664896,"internally_reserved":0,"allocated":647168,"data_stored":265667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574900,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40148,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331692,"statfs":{"total":96636764160,"available":96595652608,"internally_reserved":0,"allocated":659456,"data_stored":271484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45720,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326120,"statfs":{"total":96636764160,"available":96589946880,"internally_reserved":0,"allocated":659456,"data_stored":271484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54020,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317820,"statfs":{"total":96636764160,"available":96581447680,"internally_reserved":0,"allocated":647168,"data_stored":265667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":40152,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331688,"statfs":{"total":96636764160,"available":96595648512,"internally_reserved":0,"allocated":659456,"data_stored":271484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49860,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321980,"statfs":{"total":96636764160,"available":96585707520,"internally_reserved":0,"allocated":647168,"data_stored":265667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574899,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28232,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343608,"statfs":{"total":96636764160,"available":96607854592,"internally_reserved":0,"allocated":1241088,"data_stored":856035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:57:58.559 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:57:58.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.707+0000 7f8adaa4d640 1 Processor -- start 2024-04-25T08:57:58.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.708+0000 7f8adaa4d640 1 -- start start 2024-04-25T08:57:58.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.708+0000 7f8adaa4d640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f8ad415df70 0x7f8ad40cc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:58.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.708+0000 7f8adaa4d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8ad40b0870 0x7f8ad40ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:58.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.708+0000 7f8adaa4d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ad40cbee0 con 0x7f8ad406c510 2024-04-25T08:57:58.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.708+0000 7f8adaa4d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ad40d0400 con 0x7f8ad40b0870 2024-04-25T08:57:58.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.708+0000 7f8adaa4d640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ad40d0650 con 0x7f8ad415df70 2024-04-25T08:57:58.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.708+0000 7f8ad8fea640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8ad40b0870 0x7f8ad40ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:58.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.708+0000 7f8ad3fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8ad406c510 0x7f8ad406c8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45384/0 (socket says 172.21.15.43:45384) 2024-04-25T08:57:58.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.709+0000 7f8ad3fff640 1 -- 172.21.15.43:0/2467305842 learned_addr learned my addr 172.21.15.43:0/2467305842 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:58.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.709+0000 7f8ad37fe640 1 --2- 172.21.15.43:0/2467305842 >> v2:172.21.15.193:3300/0 conn(0x7f8ad415df70 0x7f8ad40cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:58.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.709+0000 7f8ad8fea640 1 -- 172.21.15.43:0/2467305842 >> v2:172.21.15.193:3300/0 conn(0x7f8ad415df70 msgr2=0x7f8ad40cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:58.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.709+0000 7f8ad8fea640 1 --2- 172.21.15.43:0/2467305842 >> v2:172.21.15.193:3300/0 conn(0x7f8ad415df70 0x7f8ad40cc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:58.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.709+0000 7f8ad8fea640 1 -- 172.21.15.43:0/2467305842 >> v1:172.21.15.43:6789/0 conn(0x7f8ad406c510 legacy=0x7f8ad406c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:58.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.709+0000 7f8ad8fea640 1 -- 172.21.15.43:0/2467305842 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad40d0930 con 0x7f8ad40b0870 2024-04-25T08:57:58.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.709+0000 7f8ad37fe640 1 --2- 172.21.15.43:0/2467305842 >> v2:172.21.15.193:3300/0 conn(0x7f8ad415df70 0x7f8ad40cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:58.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.709+0000 7f8ad8fea640 1 --2- 172.21.15.43:0/2467305842 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ad40b0870 0x7f8ad40ccbb0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f8acc009d90 tx=0x7f8acc02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=281e9b6e4262e43f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:58.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.710+0000 7f8ad2ffd640 1 -- 172.21.15.43:0/2467305842 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc031600 con 0x7f8ad40b0870 2024-04-25T08:57:58.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.710+0000 7f8ad2ffd640 1 -- 172.21.15.43:0/2467305842 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8acc0317a0 con 0x7f8ad40b0870 2024-04-25T08:57:58.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.710+0000 7f8ad2ffd640 1 -- 172.21.15.43:0/2467305842 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc031b70 con 0x7f8ad40b0870 2024-04-25T08:57:58.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.710+0000 7f8adaa4d640 1 -- 172.21.15.43:0/2467305842 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ad40b0870 msgr2=0x7f8ad40ccbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:58.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.710+0000 7f8adaa4d640 1 --2- 172.21.15.43:0/2467305842 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ad40b0870 0x7f8ad40ccbb0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f8acc009d90 tx=0x7f8acc02a610 comp rx=0 tx=0).stop 2024-04-25T08:57:58.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.710+0000 7f8adaa4d640 1 -- 172.21.15.43:0/2467305842 shutdown_connections 2024-04-25T08:57:58.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.710+0000 7f8adaa4d640 1 --2- 172.21.15.43:0/2467305842 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ad40b0870 0x7f8ad40ccbb0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:58.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.710+0000 7f8adaa4d640 1 --2- 172.21.15.43:0/2467305842 >> v2:172.21.15.193:3300/0 conn(0x7f8ad415df70 0x7f8ad40cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:58.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.710+0000 7f8adaa4d640 1 -- 172.21.15.43:0/2467305842 >> 172.21.15.43:0/2467305842 conn(0x7f8ad40d17f0 msgr2=0x7f8ad40d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:58.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.710+0000 7f8adaa4d640 1 -- 172.21.15.43:0/2467305842 shutdown_connections 2024-04-25T08:57:58.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.710+0000 7f8adaa4d640 1 -- 172.21.15.43:0/2467305842 wait complete. 2024-04-25T08:57:58.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.711+0000 7f8adaa4d640 1 Processor -- start 2024-04-25T08:57:58.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.712+0000 7f8adaa4d640 1 -- start start 2024-04-25T08:57:58.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.712+0000 7f8adaa4d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ad406c510 0x7f8ad40cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:58.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.712+0000 7f8adaa4d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ad40b0870 0x7f8ad415f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:58.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.712+0000 7f8adaa4d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ad415df70 0x7f8ad415fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:58.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.712+0000 7f8adaa4d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad406bd10 con 0x7f8ad415df70 2024-04-25T08:57:58.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.712+0000 7f8adaa4d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad40d0c10 con 0x7f8ad40b0870 2024-04-25T08:57:58.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.712+0000 7f8adaa4d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad406be90 con 0x7f8ad406c510 2024-04-25T08:57:58.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.713+0000 7f8ad8fea640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ad415df70 0x7f8ad415fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:58.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.713+0000 7f8ad8fea640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ad415df70 0x7f8ad415fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50138/0 (socket says 172.21.15.43:50138) 2024-04-25T08:57:58.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.713+0000 7f8ad8fea640 1 -- 172.21.15.43:0/878021222 learned_addr learned my addr 172.21.15.43:0/878021222 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:58.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.713+0000 7f8ad37fe640 1 --2- 172.21.15.43:0/878021222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ad40b0870 0x7f8ad415f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:58.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.713+0000 7f8ad3fff640 1 --2- 172.21.15.43:0/878021222 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ad406c510 0x7f8ad40cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:58.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.713+0000 7f8ad8fea640 1 -- 172.21.15.43:0/878021222 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ad406c510 msgr2=0x7f8ad40cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:58.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.713+0000 7f8ad8fea640 1 --2- 172.21.15.43:0/878021222 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ad406c510 0x7f8ad40cabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:58.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.713+0000 7f8ad8fea640 1 -- 172.21.15.43:0/878021222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ad40b0870 msgr2=0x7f8ad415f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:58.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.713+0000 7f8ad8fea640 1 --2- 172.21.15.43:0/878021222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ad40b0870 0x7f8ad415f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:58.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.713+0000 7f8ad8fea640 1 -- 172.21.15.43:0/878021222 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad4160370 con 0x7f8ad415df70 2024-04-25T08:57:58.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.713+0000 7f8ad8fea640 1 --2- 172.21.15.43:0/878021222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ad415df70 0x7f8ad415fca0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f8acc009eb0 tx=0x7f8acc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:58.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.713+0000 7f8ad0ff9640 1 -- 172.21.15.43:0/878021222 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc03f070 con 0x7f8ad415df70 2024-04-25T08:57:58.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.713+0000 7f8ad0ff9640 1 -- 172.21.15.43:0/878021222 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8acc030b60 con 0x7f8ad415df70 2024-04-25T08:57:58.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.713+0000 7f8adaa4d640 1 -- 172.21.15.43:0/878021222 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ad4160650 con 0x7f8ad415df70 2024-04-25T08:57:58.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.713+0000 7f8ad0ff9640 1 -- 172.21.15.43:0/878021222 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc044460 con 0x7f8ad415df70 2024-04-25T08:57:58.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.713+0000 7f8adaa4d640 1 -- 172.21.15.43:0/878021222 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ad40c8df0 con 0x7f8ad415df70 2024-04-25T08:57:58.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.714+0000 7f8adaa4d640 1 -- 172.21.15.43:0/878021222 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ad4074ef0 con 0x7f8ad415df70 2024-04-25T08:57:58.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.715+0000 7f8ad0ff9640 1 -- 172.21.15.43:0/878021222 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8acc03b410 con 0x7f8ad415df70 2024-04-25T08:57:58.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.715+0000 7f8ad0ff9640 1 --2- 172.21.15.43:0/878021222 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8ac006b4d0 0x7f8ac006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:58.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.716+0000 7f8ad0ff9640 1 -- 172.21.15.43:0/878021222 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8acc0ab770 con 0x7f8ad415df70 2024-04-25T08:57:58.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.716+0000 7f8ad3fff640 1 --2- 172.21.15.43:0/878021222 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8ac006b4d0 0x7f8ac006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:58.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.716+0000 7f8ad3fff640 1 --2- 172.21.15.43:0/878021222 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8ac006b4d0 0x7f8ac006d990 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4007fd0 tx=0x7f8ac4007e80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:58.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.720+0000 7f8ad0ff9640 1 -- 172.21.15.43:0/878021222 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8acc07a0f0 con 0x7f8ad415df70 2024-04-25T08:57:58.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.896+0000 7f8adaa4d640 1 -- 172.21.15.43:0/878021222 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8ad415c0f0 con 0x7f8ac006b4d0 2024-04-25T08:57:58.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.902+0000 7f8ad0ff9640 1 -- 172.21.15.43:0/878021222 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+239771 (secure 0 0 0) 0x7f8ad415c0f0 con 0x7f8ac006b4d0 2024-04-25T08:57:58.894 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:57:58.895 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-25T08:57:58.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.905+0000 7f8adaa4d640 1 -- 172.21.15.43:0/878021222 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8ac006b4d0 msgr2=0x7f8ac006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:58.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.905+0000 7f8adaa4d640 1 --2- 172.21.15.43:0/878021222 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8ac006b4d0 0x7f8ac006d990 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4007fd0 tx=0x7f8ac4007e80 comp rx=0 tx=0).stop 2024-04-25T08:57:58.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.905+0000 7f8adaa4d640 1 -- 172.21.15.43:0/878021222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ad415df70 msgr2=0x7f8ad415fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:58.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.905+0000 7f8adaa4d640 1 --2- 172.21.15.43:0/878021222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ad415df70 0x7f8ad415fca0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f8acc009eb0 tx=0x7f8acc004a40 comp rx=0 tx=0).stop 2024-04-25T08:57:58.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.905+0000 7f8adaa4d640 1 -- 172.21.15.43:0/878021222 shutdown_connections 2024-04-25T08:57:58.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.905+0000 7f8adaa4d640 1 --2- 172.21.15.43:0/878021222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ad415df70 0x7f8ad415fca0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:58.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.905+0000 7f8adaa4d640 1 --2- 172.21.15.43:0/878021222 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8ac006b4d0 0x7f8ac006d990 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:58.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.905+0000 7f8adaa4d640 1 --2- 172.21.15.43:0/878021222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ad40b0870 0x7f8ad415f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:58.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.905+0000 7f8adaa4d640 1 --2- 172.21.15.43:0/878021222 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ad406c510 0x7f8ad40cabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:58.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.905+0000 7f8adaa4d640 1 -- 172.21.15.43:0/878021222 >> 172.21.15.43:0/878021222 conn(0x7f8ad40d17f0 msgr2=0x7f8ad40d1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:58.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.905+0000 7f8adaa4d640 1 -- 172.21.15.43:0/878021222 shutdown_connections 2024-04-25T08:57:58.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:58.905+0000 7f8adaa4d640 1 -- 172.21.15.43:0/878021222 wait complete. 2024-04-25T08:57:58.906 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":171,"stamp":"2024-04-25T08:57:57.747195+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":338988,"kb_used_data":6252,"kb_used_omap":12,"kb_used_meta":332595,"kb_avail":754635732,"statfs":{"total":773094113280,"available":772746989568,"internally_reserved":0,"allocated":6402048,"data_stored":3323523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":340577841},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.535207"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.216743+0000","last_change":"2024-04-25T08:57:53.216743+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.216743+0000","last_undegraded":"2024-04-25T08:57:53.216743+0000","last_fullsized":"2024-04-25T08:57:53.216743+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.215243+0000","last_change":"2024-04-25T08:57:53.215243+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.215243+0000","last_undegraded":"2024-04-25T08:57:53.215243+0000","last_fullsized":"2024-04-25T08:57:53.215243+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.216709+0000","last_change":"2024-04-25T08:57:53.216709+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.216709+0000","last_undegraded":"2024-04-25T08:57:53.216709+0000","last_fullsized":"2024-04-25T08:57:53.216709+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.214788+0000","last_change":"2024-04-25T08:57:53.214788+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.214788+0000","last_undegraded":"2024-04-25T08:57:53.214788+0000","last_fullsized":"2024-04-25T08:57:53.214788+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.217098+0000","last_change":"2024-04-25T08:57:53.217098+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.217098+0000","last_undegraded":"2024-04-25T08:57:53.217098+0000","last_fullsized":"2024-04-25T08:57:53.217098+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.216664+0000","last_change":"2024-04-25T08:57:53.216664+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.216664+0000","last_undegraded":"2024-04-25T08:57:53.216664+0000","last_fullsized":"2024-04-25T08:57:53.216664+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.214289+0000","last_change":"2024-04-25T08:57:53.214289+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.214289+0000","last_undegraded":"2024-04-25T08:57:53.214289+0000","last_fullsized":"2024-04-25T08:57:53.214289+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.216716+0000","last_change":"2024-04-25T08:57:53.216716+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.216716+0000","last_undegraded":"2024-04-25T08:57:53.216716+0000","last_fullsized":"2024-04-25T08:57:53.216716+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.213648+0000","last_change":"2024-04-25T08:57:53.213648+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.213648+0000","last_undegraded":"2024-04-25T08:57:53.213648+0000","last_fullsized":"2024-04-25T08:57:53.213648+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.214791+0000","last_change":"2024-04-25T08:57:53.214791+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.214791+0000","last_undegraded":"2024-04-25T08:57:53.214791+0000","last_fullsized":"2024-04-25T08:57:53.214791+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.214000+0000","last_change":"2024-04-25T08:57:53.214000+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.214000+0000","last_undegraded":"2024-04-25T08:57:53.214000+0000","last_fullsized":"2024-04-25T08:57:53.214000+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.216775+0000","last_change":"2024-04-25T08:57:53.216775+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.216775+0000","last_undegraded":"2024-04-25T08:57:53.216775+0000","last_fullsized":"2024-04-25T08:57:53.216775+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.214777+0000","last_change":"2024-04-25T08:57:53.214777+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.214777+0000","last_undegraded":"2024-04-25T08:57:53.214777+0000","last_fullsized":"2024-04-25T08:57:53.214777+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.219307+0000","last_change":"2024-04-25T08:57:53.219307+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.219307+0000","last_undegraded":"2024-04-25T08:57:53.219307+0000","last_fullsized":"2024-04-25T08:57:53.219307+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.213691+0000","last_change":"2024-04-25T08:57:53.213691+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.213691+0000","last_undegraded":"2024-04-25T08:57:53.213691+0000","last_fullsized":"2024-04-25T08:57:53.213691+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.213640+0000","last_change":"2024-04-25T08:57:53.213640+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.213640+0000","last_undegraded":"2024-04-25T08:57:53.213640+0000","last_fullsized":"2024-04-25T08:57:53.213640+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.214281+0000","last_change":"2024-04-25T08:57:53.214281+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.214281+0000","last_undegraded":"2024-04-25T08:57:53.214281+0000","last_fullsized":"2024-04-25T08:57:53.214281+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.218465+0000","last_change":"2024-04-25T08:57:53.218465+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.218465+0000","last_undegraded":"2024-04-25T08:57:53.218465+0000","last_fullsized":"2024-04-25T08:57:53.218465+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.213640+0000","last_change":"2024-04-25T08:57:53.213640+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.213640+0000","last_undegraded":"2024-04-25T08:57:53.213640+0000","last_fullsized":"2024-04-25T08:57:53.213640+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.216771+0000","last_change":"2024-04-25T08:57:53.216771+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.216771+0000","last_undegraded":"2024-04-25T08:57:53.216771+0000","last_fullsized":"2024-04-25T08:57:53.216771+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.214220+0000","last_change":"2024-04-25T08:57:53.214220+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.214220+0000","last_undegraded":"2024-04-25T08:57:53.214220+0000","last_fullsized":"2024-04-25T08:57:53.214220+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.215905+0000","last_change":"2024-04-25T08:57:53.215905+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.215905+0000","last_undegraded":"2024-04-25T08:57:53.215905+0000","last_fullsized":"2024-04-25T08:57:53.215905+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.213640+0000","last_change":"2024-04-25T08:57:53.213640+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.213640+0000","last_undegraded":"2024-04-25T08:57:53.213640+0000","last_fullsized":"2024-04-25T08:57:53.213640+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.215838+0000","last_change":"2024-04-25T08:57:53.215838+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.215838+0000","last_undegraded":"2024-04-25T08:57:53.215838+0000","last_fullsized":"2024-04-25T08:57:53.215838+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.217997+0000","last_change":"2024-04-25T08:57:53.217997+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.217997+0000","last_undegraded":"2024-04-25T08:57:53.217997+0000","last_fullsized":"2024-04-25T08:57:53.217997+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.219296+0000","last_change":"2024-04-25T08:57:53.219296+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.219296+0000","last_undegraded":"2024-04-25T08:57:53.219296+0000","last_fullsized":"2024-04-25T08:57:53.219296+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.214302+0000","last_change":"2024-04-25T08:57:53.214302+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.214302+0000","last_undegraded":"2024-04-25T08:57:53.214302+0000","last_fullsized":"2024-04-25T08:57:53.214302+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.216286+0000","last_change":"2024-04-25T08:57:53.216286+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.216286+0000","last_undegraded":"2024-04-25T08:57:53.216286+0000","last_fullsized":"2024-04-25T08:57:53.216286+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.216724+0000","last_change":"2024-04-25T08:57:53.216724+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.216724+0000","last_undegraded":"2024-04-25T08:57:53.216724+0000","last_fullsized":"2024-04-25T08:57:53.216724+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.215814+0000","last_change":"2024-04-25T08:57:53.215814+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.215814+0000","last_undegraded":"2024-04-25T08:57:53.215814+0000","last_fullsized":"2024-04-25T08:57:53.215814+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.214076+0000","last_change":"2024-04-25T08:57:53.214076+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.214076+0000","last_undegraded":"2024-04-25T08:57:53.214076+0000","last_fullsized":"2024-04-25T08:57:53.214076+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.219243+0000","last_change":"2024-04-25T08:57:53.219243+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.219243+0000","last_undegraded":"2024-04-25T08:57:53.219243+0000","last_fullsized":"2024-04-25T08:57:53.219243+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":234,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:57:53.214527+0000","last_change":"2024-04-25T08:54:00.818044+0000","last_active":"2024-04-25T08:57:53.214527+0000","last_peered":"2024-04-25T08:57:53.214527+0000","last_clean":"2024-04-25T08:57:53.214527+0000","last_became_active":"2024-04-25T08:54:00.817098+0000","last_became_peered":"2024-04-25T08:54:00.817098+0000","last_unstale":"2024-04-25T08:57:53.214527+0000","last_undegraded":"2024-04-25T08:57:53.214527+0000","last_fullsized":"2024-04-25T08:57:53.214527+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_clean_scrub_stamp":"2024-04-25T08:53:59.739677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:48.129513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40720,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331120,"statfs":{"total":96636764160,"available":96595066880,"internally_reserved":0,"allocated":1241088,"data_stored":856035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40136,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331704,"statfs":{"total":96636764160,"available":96595664896,"internally_reserved":0,"allocated":647168,"data_stored":265667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574900,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40148,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331692,"statfs":{"total":96636764160,"available":96595652608,"internally_reserved":0,"allocated":659456,"data_stored":271484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45720,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326120,"statfs":{"total":96636764160,"available":96589946880,"internally_reserved":0,"allocated":659456,"data_stored":271484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54020,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317820,"statfs":{"total":96636764160,"available":96581447680,"internally_reserved":0,"allocated":647168,"data_stored":265667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":40152,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331688,"statfs":{"total":96636764160,"available":96595648512,"internally_reserved":0,"allocated":659456,"data_stored":271484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49860,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321980,"statfs":{"total":96636764160,"available":96585707520,"internally_reserved":0,"allocated":647168,"data_stored":265667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574899,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28232,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343608,"statfs":{"total":96636764160,"available":96607854592,"internally_reserved":0,"allocated":1241088,"data_stored":856035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:57:58.909 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3f is not active+clean 2024-04-25T08:57:58.910 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.910 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3e is not active+clean 2024-04-25T08:57:58.910 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.910 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3d is not active+clean 2024-04-25T08:57:58.910 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.910 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3c is not active+clean 2024-04-25T08:57:58.910 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.910 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3b is not active+clean 2024-04-25T08:57:58.910 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.910 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3a is not active+clean 2024-04-25T08:57:58.911 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.911 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.39 is not active+clean 2024-04-25T08:57:58.911 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.911 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.38 is not active+clean 2024-04-25T08:57:58.911 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.911 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.37 is not active+clean 2024-04-25T08:57:58.911 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.911 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.36 is not active+clean 2024-04-25T08:57:58.911 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.911 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.35 is not active+clean 2024-04-25T08:57:58.911 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.911 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.34 is not active+clean 2024-04-25T08:57:58.912 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.912 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.33 is not active+clean 2024-04-25T08:57:58.912 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.912 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.32 is not active+clean 2024-04-25T08:57:58.912 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.912 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.31 is not active+clean 2024-04-25T08:57:58.912 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.912 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.30 is not active+clean 2024-04-25T08:57:58.912 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.912 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2f is not active+clean 2024-04-25T08:57:58.912 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.912 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2e is not active+clean 2024-04-25T08:57:58.913 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.913 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2d is not active+clean 2024-04-25T08:57:58.913 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.913 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2c is not active+clean 2024-04-25T08:57:58.913 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.913 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2b is not active+clean 2024-04-25T08:57:58.913 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.913 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2a is not active+clean 2024-04-25T08:57:58.913 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.913 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.29 is not active+clean 2024-04-25T08:57:58.913 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.914 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.28 is not active+clean 2024-04-25T08:57:58.914 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.914 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.27 is not active+clean 2024-04-25T08:57:58.914 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.914 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.26 is not active+clean 2024-04-25T08:57:58.914 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.914 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.25 is not active+clean 2024-04-25T08:57:58.914 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.914 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.24 is not active+clean 2024-04-25T08:57:58.914 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.914 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.23 is not active+clean 2024-04-25T08:57:58.914 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.915 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.22 is not active+clean 2024-04-25T08:57:58.915 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.915 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.21 is not active+clean 2024-04-25T08:57:58.915 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.915 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.20 is not active+clean 2024-04-25T08:57:58.915 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.915 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1e is not active+clean 2024-04-25T08:57:58.915 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.915 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1f is not active+clean 2024-04-25T08:57:58.915 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:57:53.216743+0000', 'last_change': '2024-04-25T08:57:53.216743+0000', 'last_active': '2024-04-25T08:57:53.209247+0000', 'last_peered': '2024-04-25T08:57:53.209247+0000', 'last_clean': '2024-04-25T08:57:53.209247+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:53.216743+0000', 'last_undegraded': '2024-04-25T08:57:53.216743+0000', 'last_fullsized': '2024-04-25T08:57:53.216743+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:57:58.916 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1f is not active+clean 2024-04-25T08:57:58.916 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.916 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1e is not active+clean 2024-04-25T08:57:58.916 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:57:53.215243+0000', 'last_change': '2024-04-25T08:57:53.215243+0000', 'last_active': '2024-04-25T08:57:53.209247+0000', 'last_peered': '2024-04-25T08:57:53.209247+0000', 'last_clean': '2024-04-25T08:57:53.209247+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:53.215243+0000', 'last_undegraded': '2024-04-25T08:57:53.215243+0000', 'last_fullsized': '2024-04-25T08:57:53.215243+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:57:58.916 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1c is not active+clean 2024-04-25T08:57:58.916 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.916 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1d is not active+clean 2024-04-25T08:57:58.916 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:57:53.216709+0000', 'last_change': '2024-04-25T08:57:53.216709+0000', 'last_active': '2024-04-25T08:57:53.209247+0000', 'last_peered': '2024-04-25T08:57:53.209247+0000', 'last_clean': '2024-04-25T08:57:53.209247+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:53.216709+0000', 'last_undegraded': '2024-04-25T08:57:53.216709+0000', 'last_fullsized': '2024-04-25T08:57:53.216709+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:57:58.916 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1d is not active+clean 2024-04-25T08:57:58.916 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.916 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1c is not active+clean 2024-04-25T08:57:58.916 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:57:53.214788+0000', 'last_change': '2024-04-25T08:57:53.214788+0000', 'last_active': '2024-04-25T08:57:53.209247+0000', 'last_peered': '2024-04-25T08:57:53.209247+0000', 'last_clean': '2024-04-25T08:57:53.209247+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:53.214788+0000', 'last_undegraded': '2024-04-25T08:57:53.214788+0000', 'last_fullsized': '2024-04-25T08:57:53.214788+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:57:58.917 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1a is not active+clean 2024-04-25T08:57:58.917 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.917 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1b is not active+clean 2024-04-25T08:57:58.917 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:57:53.217098+0000', 'last_change': '2024-04-25T08:57:53.217098+0000', 'last_active': '2024-04-25T08:57:53.209247+0000', 'last_peered': '2024-04-25T08:57:53.209247+0000', 'last_clean': '2024-04-25T08:57:53.209247+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:53.217098+0000', 'last_undegraded': '2024-04-25T08:57:53.217098+0000', 'last_fullsized': '2024-04-25T08:57:53.217098+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:57:58.917 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1b is not active+clean 2024-04-25T08:57:58.917 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.917 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1a is not active+clean 2024-04-25T08:57:58.917 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:57:53.216664+0000', 'last_change': '2024-04-25T08:57:53.216664+0000', 'last_active': '2024-04-25T08:57:53.209247+0000', 'last_peered': '2024-04-25T08:57:53.209247+0000', 'last_clean': '2024-04-25T08:57:53.209247+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:53.216664+0000', 'last_undegraded': '2024-04-25T08:57:53.216664+0000', 'last_fullsized': '2024-04-25T08:57:53.216664+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:57:58.917 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.18 is not active+clean 2024-04-25T08:57:58.917 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.917 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.19 is not active+clean 2024-04-25T08:57:58.917 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:57:53.214289+0000', 'last_change': '2024-04-25T08:57:53.214289+0000', 'last_active': '2024-04-25T08:57:53.209247+0000', 'last_peered': '2024-04-25T08:57:53.209247+0000', 'last_clean': '2024-04-25T08:57:53.209247+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:53.214289+0000', 'last_undegraded': '2024-04-25T08:57:53.214289+0000', 'last_fullsized': '2024-04-25T08:57:53.214289+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:57:58.918 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.19 is not active+clean 2024-04-25T08:57:58.918 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.918 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.18 is not active+clean 2024-04-25T08:57:58.918 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:57:53.216716+0000', 'last_change': '2024-04-25T08:57:53.216716+0000', 'last_active': '2024-04-25T08:57:53.209247+0000', 'last_peered': '2024-04-25T08:57:53.209247+0000', 'last_clean': '2024-04-25T08:57:53.209247+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:53.216716+0000', 'last_undegraded': '2024-04-25T08:57:53.216716+0000', 'last_fullsized': '2024-04-25T08:57:53.216716+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:57:58.918 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.16 is not active+clean 2024-04-25T08:57:58.918 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.918 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.17 is not active+clean 2024-04-25T08:57:58.918 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:57:53.213648+0000', 'last_change': '2024-04-25T08:57:53.213648+0000', 'last_active': '2024-04-25T08:57:53.209247+0000', 'last_peered': '2024-04-25T08:57:53.209247+0000', 'last_clean': '2024-04-25T08:57:53.209247+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:53.213648+0000', 'last_undegraded': '2024-04-25T08:57:53.213648+0000', 'last_fullsized': '2024-04-25T08:57:53.213648+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:57:58.918 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.17 is not active+clean 2024-04-25T08:57:58.918 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.919 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.16 is not active+clean 2024-04-25T08:57:58.919 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:57:53.214791+0000', 'last_change': '2024-04-25T08:57:53.214791+0000', 'last_active': '2024-04-25T08:57:53.209247+0000', 'last_peered': '2024-04-25T08:57:53.209247+0000', 'last_clean': '2024-04-25T08:57:53.209247+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:53.214791+0000', 'last_undegraded': '2024-04-25T08:57:53.214791+0000', 'last_fullsized': '2024-04-25T08:57:53.214791+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:57:58.919 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.14 is not active+clean 2024-04-25T08:57:58.919 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.919 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.15 is not active+clean 2024-04-25T08:57:58.919 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:57:53.214000+0000', 'last_change': '2024-04-25T08:57:53.214000+0000', 'last_active': '2024-04-25T08:57:53.209247+0000', 'last_peered': '2024-04-25T08:57:53.209247+0000', 'last_clean': '2024-04-25T08:57:53.209247+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:53.214000+0000', 'last_undegraded': '2024-04-25T08:57:53.214000+0000', 'last_fullsized': '2024-04-25T08:57:53.214000+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:57:58.919 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.15 is not active+clean 2024-04-25T08:57:58.919 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.919 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.14 is not active+clean 2024-04-25T08:57:58.919 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:57:53.216775+0000', 'last_change': '2024-04-25T08:57:53.216775+0000', 'last_active': '2024-04-25T08:57:53.209247+0000', 'last_peered': '2024-04-25T08:57:53.209247+0000', 'last_clean': '2024-04-25T08:57:53.209247+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:53.216775+0000', 'last_undegraded': '2024-04-25T08:57:53.216775+0000', 'last_fullsized': '2024-04-25T08:57:53.216775+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:57:58.920 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.12 is not active+clean 2024-04-25T08:57:58.920 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.920 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.13 is not active+clean 2024-04-25T08:57:58.920 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:57:53.214777+0000', 'last_change': '2024-04-25T08:57:53.214777+0000', 'last_active': '2024-04-25T08:57:53.209247+0000', 'last_peered': '2024-04-25T08:57:53.209247+0000', 'last_clean': '2024-04-25T08:57:53.209247+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:53.214777+0000', 'last_undegraded': '2024-04-25T08:57:53.214777+0000', 'last_fullsized': '2024-04-25T08:57:53.214777+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:57:58.920 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.13 is not active+clean 2024-04-25T08:57:58.920 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.920 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.12 is not active+clean 2024-04-25T08:57:58.920 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:57:53.219307+0000', 'last_change': '2024-04-25T08:57:53.219307+0000', 'last_active': '2024-04-25T08:57:53.209247+0000', 'last_peered': '2024-04-25T08:57:53.209247+0000', 'last_clean': '2024-04-25T08:57:53.209247+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:53.219307+0000', 'last_undegraded': '2024-04-25T08:57:53.219307+0000', 'last_fullsized': '2024-04-25T08:57:53.219307+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:57:58.920 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.10 is not active+clean 2024-04-25T08:57:58.920 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.920 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.11 is not active+clean 2024-04-25T08:57:58.921 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:57:53.213691+0000', 'last_change': '2024-04-25T08:57:53.213691+0000', 'last_active': '2024-04-25T08:57:53.209247+0000', 'last_peered': '2024-04-25T08:57:53.209247+0000', 'last_clean': '2024-04-25T08:57:53.209247+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:53.213691+0000', 'last_undegraded': '2024-04-25T08:57:53.213691+0000', 'last_fullsized': '2024-04-25T08:57:53.213691+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:57:58.921 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.11 is not active+clean 2024-04-25T08:57:58.921 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.921 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.10 is not active+clean 2024-04-25T08:57:58.921 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:57:53.213640+0000', 'last_change': '2024-04-25T08:57:53.213640+0000', 'last_active': '2024-04-25T08:57:53.209247+0000', 'last_peered': '2024-04-25T08:57:53.209247+0000', 'last_clean': '2024-04-25T08:57:53.209247+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:53.213640+0000', 'last_undegraded': '2024-04-25T08:57:53.213640+0000', 'last_fullsized': '2024-04-25T08:57:53.213640+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T08:57:58.921 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.e is not active+clean 2024-04-25T08:57:58.921 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.921 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.f is not active+clean 2024-04-25T08:57:58.921 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:57:53.214281+0000', 'last_change': '2024-04-25T08:57:53.214281+0000', 'last_active': '2024-04-25T08:57:53.209247+0000', 'last_peered': '2024-04-25T08:57:53.209247+0000', 'last_clean': '2024-04-25T08:57:53.209247+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:53.214281+0000', 'last_undegraded': '2024-04-25T08:57:53.214281+0000', 'last_fullsized': '2024-04-25T08:57:53.214281+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:57:58.921 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.f is not active+clean 2024-04-25T08:57:58.921 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.922 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.e is not active+clean 2024-04-25T08:57:58.922 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:57:53.218465+0000', 'last_change': '2024-04-25T08:57:53.218465+0000', 'last_active': '2024-04-25T08:57:53.209247+0000', 'last_peered': '2024-04-25T08:57:53.209247+0000', 'last_clean': '2024-04-25T08:57:53.209247+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:53.218465+0000', 'last_undegraded': '2024-04-25T08:57:53.218465+0000', 'last_fullsized': '2024-04-25T08:57:53.218465+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:57:58.922 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.c is not active+clean 2024-04-25T08:57:58.922 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.922 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.d is not active+clean 2024-04-25T08:57:58.922 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:57:53.213640+0000', 'last_change': '2024-04-25T08:57:53.213640+0000', 'last_active': '2024-04-25T08:57:53.209247+0000', 'last_peered': '2024-04-25T08:57:53.209247+0000', 'last_clean': '2024-04-25T08:57:53.209247+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:53.213640+0000', 'last_undegraded': '2024-04-25T08:57:53.213640+0000', 'last_fullsized': '2024-04-25T08:57:53.213640+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:57:58.922 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.d is not active+clean 2024-04-25T08:57:58.922 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.922 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.c is not active+clean 2024-04-25T08:57:58.922 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:57:53.216771+0000', 'last_change': '2024-04-25T08:57:53.216771+0000', 'last_active': '2024-04-25T08:57:53.209247+0000', 'last_peered': '2024-04-25T08:57:53.209247+0000', 'last_clean': '2024-04-25T08:57:53.209247+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:53.216771+0000', 'last_undegraded': '2024-04-25T08:57:53.216771+0000', 'last_fullsized': '2024-04-25T08:57:53.216771+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:57:58.922 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.a is not active+clean 2024-04-25T08:57:58.923 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.923 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.b is not active+clean 2024-04-25T08:57:58.923 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:57:53.214220+0000', 'last_change': '2024-04-25T08:57:53.214220+0000', 'last_active': '2024-04-25T08:57:53.209247+0000', 'last_peered': '2024-04-25T08:57:53.209247+0000', 'last_clean': '2024-04-25T08:57:53.209247+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:53.214220+0000', 'last_undegraded': '2024-04-25T08:57:53.214220+0000', 'last_fullsized': '2024-04-25T08:57:53.214220+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:57:58.923 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.b is not active+clean 2024-04-25T08:57:58.923 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.923 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.a is not active+clean 2024-04-25T08:57:58.923 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:57:53.215905+0000', 'last_change': '2024-04-25T08:57:53.215905+0000', 'last_active': '2024-04-25T08:57:53.209247+0000', 'last_peered': '2024-04-25T08:57:53.209247+0000', 'last_clean': '2024-04-25T08:57:53.209247+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:53.215905+0000', 'last_undegraded': '2024-04-25T08:57:53.215905+0000', 'last_fullsized': '2024-04-25T08:57:53.215905+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:57:58.923 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.9 is not active+clean 2024-04-25T08:57:58.923 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.923 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.8 is not active+clean 2024-04-25T08:57:58.923 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:57:53.213640+0000', 'last_change': '2024-04-25T08:57:53.213640+0000', 'last_active': '2024-04-25T08:57:53.209247+0000', 'last_peered': '2024-04-25T08:57:53.209247+0000', 'last_clean': '2024-04-25T08:57:53.209247+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:53.213640+0000', 'last_undegraded': '2024-04-25T08:57:53.213640+0000', 'last_fullsized': '2024-04-25T08:57:53.213640+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:57:58.924 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.6 is not active+clean 2024-04-25T08:57:58.924 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.924 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.7 is not active+clean 2024-04-25T08:57:58.924 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:57:53.215838+0000', 'last_change': '2024-04-25T08:57:53.215838+0000', 'last_active': '2024-04-25T08:57:53.209247+0000', 'last_peered': '2024-04-25T08:57:53.209247+0000', 'last_clean': '2024-04-25T08:57:53.209247+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:53.215838+0000', 'last_undegraded': '2024-04-25T08:57:53.215838+0000', 'last_fullsized': '2024-04-25T08:57:53.215838+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:57:58.924 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.7 is not active+clean 2024-04-25T08:57:58.924 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.924 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.6 is not active+clean 2024-04-25T08:57:58.924 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:57:53.217997+0000', 'last_change': '2024-04-25T08:57:53.217997+0000', 'last_active': '2024-04-25T08:57:53.209247+0000', 'last_peered': '2024-04-25T08:57:53.209247+0000', 'last_clean': '2024-04-25T08:57:53.209247+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:53.217997+0000', 'last_undegraded': '2024-04-25T08:57:53.217997+0000', 'last_fullsized': '2024-04-25T08:57:53.217997+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T08:57:58.924 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.4 is not active+clean 2024-04-25T08:57:58.924 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.924 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.5 is not active+clean 2024-04-25T08:57:58.925 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:57:53.219296+0000', 'last_change': '2024-04-25T08:57:53.219296+0000', 'last_active': '2024-04-25T08:57:53.209247+0000', 'last_peered': '2024-04-25T08:57:53.209247+0000', 'last_clean': '2024-04-25T08:57:53.209247+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:53.219296+0000', 'last_undegraded': '2024-04-25T08:57:53.219296+0000', 'last_fullsized': '2024-04-25T08:57:53.219296+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:57:58.925 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.5 is not active+clean 2024-04-25T08:57:58.925 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.925 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.4 is not active+clean 2024-04-25T08:57:58.925 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:57:53.214302+0000', 'last_change': '2024-04-25T08:57:53.214302+0000', 'last_active': '2024-04-25T08:57:53.209247+0000', 'last_peered': '2024-04-25T08:57:53.209247+0000', 'last_clean': '2024-04-25T08:57:53.209247+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:53.214302+0000', 'last_undegraded': '2024-04-25T08:57:53.214302+0000', 'last_fullsized': '2024-04-25T08:57:53.214302+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T08:57:58.925 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2 is not active+clean 2024-04-25T08:57:58.925 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.925 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3 is not active+clean 2024-04-25T08:57:58.925 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:57:53.216286+0000', 'last_change': '2024-04-25T08:57:53.216286+0000', 'last_active': '2024-04-25T08:57:53.209247+0000', 'last_peered': '2024-04-25T08:57:53.209247+0000', 'last_clean': '2024-04-25T08:57:53.209247+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:53.216286+0000', 'last_undegraded': '2024-04-25T08:57:53.216286+0000', 'last_fullsized': '2024-04-25T08:57:53.216286+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:57:58.925 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3 is not active+clean 2024-04-25T08:57:58.925 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.926 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2 is not active+clean 2024-04-25T08:57:58.926 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:57:53.216724+0000', 'last_change': '2024-04-25T08:57:53.216724+0000', 'last_active': '2024-04-25T08:57:53.209247+0000', 'last_peered': '2024-04-25T08:57:53.209247+0000', 'last_clean': '2024-04-25T08:57:53.209247+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:53.216724+0000', 'last_undegraded': '2024-04-25T08:57:53.216724+0000', 'last_fullsized': '2024-04-25T08:57:53.216724+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T08:57:58.926 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.0 is not active+clean 2024-04-25T08:57:58.926 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.926 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1 is not active+clean 2024-04-25T08:57:58.926 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:57:53.215814+0000', 'last_change': '2024-04-25T08:57:53.215814+0000', 'last_active': '2024-04-25T08:57:53.209247+0000', 'last_peered': '2024-04-25T08:57:53.209247+0000', 'last_clean': '2024-04-25T08:57:53.209247+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:53.215814+0000', 'last_undegraded': '2024-04-25T08:57:53.215814+0000', 'last_fullsized': '2024-04-25T08:57:53.215814+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T08:57:58.926 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1 is not active+clean 2024-04-25T08:57:58.926 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.926 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.0 is not active+clean 2024-04-25T08:57:58.926 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:57:53.214076+0000', 'last_change': '2024-04-25T08:57:53.214076+0000', 'last_active': '2024-04-25T08:57:53.209247+0000', 'last_peered': '2024-04-25T08:57:53.209247+0000', 'last_clean': '2024-04-25T08:57:53.209247+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:53.214076+0000', 'last_undegraded': '2024-04-25T08:57:53.214076+0000', 'last_fullsized': '2024-04-25T08:57:53.214076+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T08:57:58.926 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.8 is not active+clean 2024-04-25T08:57:58.927 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2024-04-25T08:57:54.211413+0000', 'last_change': '2024-04-25T08:57:54.211413+0000', 'last_active': '2024-04-25T08:57:54.211413+0000', 'last_peered': '2024-04-25T08:57:54.211413+0000', 'last_clean': '2024-04-25T08:57:54.211413+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:54.211413+0000', 'last_undegraded': '2024-04-25T08:57:54.211413+0000', 'last_fullsized': '2024-04-25T08:57:54.211413+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:54.211413+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2024-04-25T08:57:58.927 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.9 is not active+clean 2024-04-25T08:57:58.927 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 55, 'state': 'creating+peering', 'last_fresh': '2024-04-25T08:57:53.219243+0000', 'last_change': '2024-04-25T08:57:53.219243+0000', 'last_active': '2024-04-25T08:57:53.209247+0000', 'last_peered': '2024-04-25T08:57:53.209247+0000', 'last_clean': '2024-04-25T08:57:53.209247+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T08:57:53.219243+0000', 'last_undegraded': '2024-04-25T08:57:53.219243+0000', 'last_fullsized': '2024-04-25T08:57:53.219243+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 55, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'last_clean_scrub_stamp': '2024-04-25T08:57:53.209247+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T08:57:58.927 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T08:57:59.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.057+0000 7f1df7aac640 1 Processor -- start 2024-04-25T08:57:59.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.057+0000 7f1df7aac640 1 -- start start 2024-04-25T08:57:59.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.058+0000 7f1df7aac640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1df015a9d0 0x7f1df006d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:59.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.058+0000 7f1df7aac640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1df006d7d0 0x7f1df0071a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:59.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.058+0000 7f1df7aac640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f1df0071f90 0x7f1df00b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:59.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.058+0000 7f1df7aac640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1df006dc30 con 0x7f1df015a9d0 2024-04-25T08:57:59.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.058+0000 7f1df7aac640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1df0072420 con 0x7f1df0071f90 2024-04-25T08:57:59.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.058+0000 7f1df7aac640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df00704f0 con 0x7f1df006d7d0 2024-04-25T08:57:59.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.058+0000 7f1df5848640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1df015a9d0 0x7f1df006d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:59.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.058+0000 7f1df5848640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1df015a9d0 0x7f1df006d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50154/0 (socket says 172.21.15.43:50154) 2024-04-25T08:57:59.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.058+0000 7f1df5848640 1 -- 172.21.15.43:0/1919166436 learned_addr learned my addr 172.21.15.43:0/1919166436 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:59.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.058+0000 7f1df5047640 1 --2- 172.21.15.43:0/1919166436 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1df006d7d0 0x7f1df0071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:59.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.058+0000 7f1df5848640 1 -- 172.21.15.43:0/1919166436 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1df006d7d0 msgr2=0x7f1df0071a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:59.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.058+0000 7f1df6049640 1 --2- 172.21.15.43:0/1919166436 >> v2:172.21.15.193:3300/0 conn(0x7f1df0071f90 0x7f1df00b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:59.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.058+0000 7f1df5848640 1 --2- 172.21.15.43:0/1919166436 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1df006d7d0 0x7f1df0071a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.058+0000 7f1df5848640 1 -- 172.21.15.43:0/1919166436 >> v2:172.21.15.193:3300/0 conn(0x7f1df0071f90 msgr2=0x7f1df00b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:59.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.058+0000 7f1df5848640 1 --2- 172.21.15.43:0/1919166436 >> v2:172.21.15.193:3300/0 conn(0x7f1df0071f90 0x7f1df00b3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.058+0000 7f1df5848640 1 -- 172.21.15.43:0/1919166436 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df00707d0 con 0x7f1df015a9d0 2024-04-25T08:57:59.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.058+0000 7f1df5047640 1 --2- 172.21.15.43:0/1919166436 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1df006d7d0 0x7f1df0071a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:59.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.058+0000 7f1df6049640 1 --2- 172.21.15.43:0/1919166436 >> v2:172.21.15.193:3300/0 conn(0x7f1df0071f90 0x7f1df00b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:57:59.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.059+0000 7f1df5848640 1 --2- 172.21.15.43:0/1919166436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1df015a9d0 0x7f1df006d290 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f1df00702e0 tx=0x7f1dec02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccccdfab89b0cca3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:59.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.059+0000 7f1df4846640 1 -- 172.21.15.43:0/1919166436 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dec037070 con 0x7f1df015a9d0 2024-04-25T08:57:59.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.059+0000 7f1df4846640 1 -- 172.21.15.43:0/1919166436 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dec032440 con 0x7f1df015a9d0 2024-04-25T08:57:59.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.059+0000 7f1df4846640 1 -- 172.21.15.43:0/1919166436 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dec032810 con 0x7f1df015a9d0 2024-04-25T08:57:59.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.060+0000 7f1df7aac640 1 -- 172.21.15.43:0/1919166436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1df015a9d0 msgr2=0x7f1df006d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:59.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.060+0000 7f1df7aac640 1 --2- 172.21.15.43:0/1919166436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1df015a9d0 0x7f1df006d290 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f1df00702e0 tx=0x7f1dec02a1e0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.060+0000 7f1df7aac640 1 -- 172.21.15.43:0/1919166436 shutdown_connections 2024-04-25T08:57:59.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.060+0000 7f1df7aac640 1 --2- 172.21.15.43:0/1919166436 >> v2:172.21.15.193:3300/0 conn(0x7f1df0071f90 0x7f1df00b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.060+0000 7f1df7aac640 1 --2- 172.21.15.43:0/1919166436 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1df006d7d0 0x7f1df0071a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.060+0000 7f1df7aac640 1 --2- 172.21.15.43:0/1919166436 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1df015a9d0 0x7f1df006d290 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.060+0000 7f1df7aac640 1 -- 172.21.15.43:0/1919166436 >> 172.21.15.43:0/1919166436 conn(0x7f1df00b1480 msgr2=0x7f1df00b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:59.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.060+0000 7f1df7aac640 1 -- 172.21.15.43:0/1919166436 shutdown_connections 2024-04-25T08:57:59.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.060+0000 7f1df7aac640 1 -- 172.21.15.43:0/1919166436 wait complete. 2024-04-25T08:57:59.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.061+0000 7f1df7aac640 1 Processor -- start 2024-04-25T08:57:59.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.061+0000 7f1df7aac640 1 -- start start 2024-04-25T08:57:59.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.062+0000 7f1df7aac640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1df006d7d0 0x7f1df0162bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:59.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.062+0000 7f1df7aac640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1df0071f90 0x7f1df01612b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:59.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.062+0000 7f1df7aac640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1df0159660 0x7f1df0159ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:59.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.062+0000 7f1df7aac640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df00712b0 con 0x7f1df0071f90 2024-04-25T08:57:59.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.062+0000 7f1df7aac640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1df0070d90 con 0x7f1df0159660 2024-04-25T08:57:59.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.062+0000 7f1df7aac640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1df00b3ed0 con 0x7f1df006d7d0 2024-04-25T08:57:59.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.062+0000 7f1df5047640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1df0071f90 0x7f1df01612b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:59.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.062+0000 7f1df5047640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1df0071f90 0x7f1df01612b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50156/0 (socket says 172.21.15.43:50156) 2024-04-25T08:57:59.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.062+0000 7f1df5047640 1 -- 172.21.15.43:0/2999614668 learned_addr learned my addr 172.21.15.43:0/2999614668 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:59.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.062+0000 7f1df5848640 1 --2- 172.21.15.43:0/2999614668 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1df006d7d0 0x7f1df0162bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:59.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.062+0000 7f1df6049640 1 --2- 172.21.15.43:0/2999614668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1df0159660 0x7f1df0159ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:59.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.062+0000 7f1df5047640 1 -- 172.21.15.43:0/2999614668 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1df006d7d0 msgr2=0x7f1df0162bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:59.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.062+0000 7f1df5047640 1 --2- 172.21.15.43:0/2999614668 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1df006d7d0 0x7f1df0162bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.062+0000 7f1df5047640 1 -- 172.21.15.43:0/2999614668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1df0159660 msgr2=0x7f1df0159ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:59.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.062+0000 7f1df5047640 1 --2- 172.21.15.43:0/2999614668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1df0159660 0x7f1df0159ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.062+0000 7f1df5047640 1 -- 172.21.15.43:0/2999614668 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df013bd30 con 0x7f1df0071f90 2024-04-25T08:57:59.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.062+0000 7f1df6049640 1 --2- 172.21.15.43:0/2999614668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1df0159660 0x7f1df0159ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:59.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.062+0000 7f1df5047640 1 --2- 172.21.15.43:0/2999614668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1df0071f90 0x7f1df01612b0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f1de000c7f0 tx=0x7f1de000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:59.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.062+0000 7f1de67fc640 1 -- 172.21.15.43:0/2999614668 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de0015070 con 0x7f1df0071f90 2024-04-25T08:57:59.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.062+0000 7f1de67fc640 1 -- 172.21.15.43:0/2999614668 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1de0007a30 con 0x7f1df0071f90 2024-04-25T08:57:59.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.062+0000 7f1de67fc640 1 -- 172.21.15.43:0/2999614668 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de0007e00 con 0x7f1df0071f90 2024-04-25T08:57:59.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.062+0000 7f1df7aac640 1 -- 172.21.15.43:0/2999614668 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1df013c010 con 0x7f1df0071f90 2024-04-25T08:57:59.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.063+0000 7f1df7aac640 1 -- 172.21.15.43:0/2999614668 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1df013c5d0 con 0x7f1df0071f90 2024-04-25T08:57:59.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.063+0000 7f1df7aac640 1 -- 172.21.15.43:0/2999614668 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1df015a020 con 0x7f1df0071f90 2024-04-25T08:57:59.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.064+0000 7f1de67fc640 1 -- 172.21.15.43:0/2999614668 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1de0018070 con 0x7f1df0071f90 2024-04-25T08:57:59.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.064+0000 7f1de67fc640 1 --2- 172.21.15.43:0/2999614668 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1dd406b4d0 0x7f1dd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:59.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.065+0000 7f1de67fc640 1 -- 172.21.15.43:0/2999614668 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1de008a0c0 con 0x7f1df0071f90 2024-04-25T08:57:59.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.065+0000 7f1df5848640 1 --2- 172.21.15.43:0/2999614668 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1dd406b4d0 0x7f1dd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:59.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.065+0000 7f1df5848640 1 --2- 172.21.15.43:0/2999614668 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1dd406b4d0 0x7f1dd406d990 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1dec034000 tx=0x7f1dec02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:59.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.069+0000 7f1de67fc640 1 -- 172.21.15.43:0/2999614668 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1de00588f0 con 0x7f1df0071f90 2024-04-25T08:57:59.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.318+0000 7f1df7aac640 1 -- 172.21.15.43:0/2999614668 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f1df0074ef0 con 0x7f1df0071f90 2024-04-25T08:57:59.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.319+0000 7f1de67fc640 1 -- 172.21.15.43:0/2999614668 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1528 (secure 0 0 0) 0x7f1de005c990 con 0x7f1df0071f90 2024-04-25T08:57:59.309 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:57:59.309 INFO:teuthology.orchestra.run.smithi043.stdout:{"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":251,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":58,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035238,"num_in_osds":8,"osd_in_since":1714035229,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":347123712,"bytes_avail":772746989568,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":31,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"c","status":"up:active","gid":6540}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-04-25T08:57:43.743983+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T08:57:59.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.321+0000 7f1df7aac640 1 -- 172.21.15.43:0/2999614668 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1dd406b4d0 msgr2=0x7f1dd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:59.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.321+0000 7f1df7aac640 1 --2- 172.21.15.43:0/2999614668 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1dd406b4d0 0x7f1dd406d990 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1dec034000 tx=0x7f1dec02e000 comp rx=0 tx=0).stop 2024-04-25T08:57:59.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.321+0000 7f1df7aac640 1 -- 172.21.15.43:0/2999614668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1df0071f90 msgr2=0x7f1df01612b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:59.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.321+0000 7f1df7aac640 1 --2- 172.21.15.43:0/2999614668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1df0071f90 0x7f1df01612b0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f1de000c7f0 tx=0x7f1de000cbf0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.321+0000 7f1df7aac640 1 -- 172.21.15.43:0/2999614668 shutdown_connections 2024-04-25T08:57:59.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.321+0000 7f1df7aac640 1 --2- 172.21.15.43:0/2999614668 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1dd406b4d0 0x7f1dd406d990 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.321+0000 7f1df7aac640 1 --2- 172.21.15.43:0/2999614668 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1df0159660 0x7f1df0159ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.321+0000 7f1df7aac640 1 --2- 172.21.15.43:0/2999614668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1df0071f90 0x7f1df01612b0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.321+0000 7f1df7aac640 1 --2- 172.21.15.43:0/2999614668 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1df006d7d0 0x7f1df0162bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.321+0000 7f1df7aac640 1 -- 172.21.15.43:0/2999614668 >> 172.21.15.43:0/2999614668 conn(0x7f1df00b1480 msgr2=0x7f1df00715a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:59.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.321+0000 7f1df7aac640 1 -- 172.21.15.43:0/2999614668 shutdown_connections 2024-04-25T08:57:59.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.321+0000 7f1df7aac640 1 -- 172.21.15.43:0/2999614668 wait complete. 2024-04-25T08:57:59.321 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T08:57:59.322 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:57:59.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.433+0000 7f34d2891640 1 Processor -- start 2024-04-25T08:57:59.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.433+0000 7f34d2891640 1 -- start start 2024-04-25T08:57:59.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.434+0000 7f34d2891640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f34cc0b0870 0x7f34cc0ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:59.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.434+0000 7f34d2891640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34cc0cbee0 con 0x7f34cc06c510 2024-04-25T08:57:59.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.434+0000 7f34d2891640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34cc0d0650 con 0x7f34cc15df70 2024-04-25T08:57:59.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.434+0000 7f34d2891640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34cc0d0400 con 0x7f34cc0b0870 2024-04-25T08:57:59.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.434+0000 7f34cbfff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f34cc06c510 0x7f34cc06c8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45390/0 (socket says 172.21.15.43:45390) 2024-04-25T08:57:59.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.434+0000 7f34cbfff640 1 -- 172.21.15.43:0/308519278 learned_addr learned my addr 172.21.15.43:0/308519278 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:57:59.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.434+0000 7f34d0e2e640 1 --2- 172.21.15.43:0/308519278 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f34cc0b0870 0x7f34cc0ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:59.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.434+0000 7f34caffd640 1 -- 172.21.15.43:0/308519278 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 388511003 0 0) 0x7f34cc0cbee0 con 0x7f34cc06c510 2024-04-25T08:57:59.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.435+0000 7f34caffd640 1 -- 172.21.15.43:0/308519278 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34b8003520 con 0x7f34cc06c510 2024-04-25T08:57:59.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.435+0000 7f34caffd640 1 -- 172.21.15.43:0/308519278 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2635238549 0 0) 0x7f34b8003520 con 0x7f34cc06c510 2024-04-25T08:57:59.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.435+0000 7f34caffd640 1 -- 172.21.15.43:0/308519278 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f34cc0b0870 msgr2=0x7f34cc0ccd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:59.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.435+0000 7f34caffd640 1 --2- 172.21.15.43:0/308519278 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f34cc0b0870 0x7f34cc0ccd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.435+0000 7f34caffd640 1 -- 172.21.15.43:0/308519278 >> v1:172.21.15.193:6789/0 conn(0x7f34cc15df70 legacy=0x7f34cc0cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:59.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.435+0000 7f34caffd640 1 -- 172.21.15.43:0/308519278 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34cc0d0930 con 0x7f34cc06c510 2024-04-25T08:57:59.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.435+0000 7f34caffd640 1 -- 172.21.15.43:0/308519278 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f34b4001670 con 0x7f34cc06c510 2024-04-25T08:57:59.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.435+0000 7f34caffd640 1 -- 172.21.15.43:0/308519278 >> v1:172.21.15.43:6789/0 conn(0x7f34cc06c510 legacy=0x7f34cc06c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:59.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.435+0000 7f34caffd640 1 --2- 172.21.15.43:0/308519278 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f34cc0b0870 0x7f34b80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:59.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.435+0000 7f34caffd640 1 --2- 172.21.15.43:0/308519278 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34b8003f60 0x7f34b8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:59.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.435+0000 7f34caffd640 1 --2- 172.21.15.43:0/308519278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34b8004d40 0x7f34b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:59.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.435+0000 7f34caffd640 1 -- 172.21.15.43:0/308519278 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34cc0d0400 con 0x7f34b8004d40 2024-04-25T08:57:59.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.435+0000 7f34caffd640 1 -- 172.21.15.43:0/308519278 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b800baa0 con 0x7f34b8003f60 2024-04-25T08:57:59.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.435+0000 7f34caffd640 1 -- 172.21.15.43:0/308519278 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34b800bca0 con 0x7f34cc0b0870 2024-04-25T08:57:59.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.435+0000 7f34d0e2e640 1 --2- 172.21.15.43:0/308519278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34b8004d40 0x7f34b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:59.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.435+0000 7f34cbfff640 1 --2- 172.21.15.43:0/308519278 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f34cc0b0870 0x7f34b80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:59.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.435+0000 7f34cb7fe640 1 --2- 172.21.15.43:0/308519278 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34b8003f60 0x7f34b8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:59.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.435+0000 7f34d0e2e640 1 -- 172.21.15.43:0/308519278 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f34cc0b0870 msgr2=0x7f34b80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:59.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.435+0000 7f34d0e2e640 1 --2- 172.21.15.43:0/308519278 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f34cc0b0870 0x7f34b80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.435+0000 7f34d0e2e640 1 -- 172.21.15.43:0/308519278 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34b8003f60 msgr2=0x7f34b8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:57:59.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.435+0000 7f34d0e2e640 1 --2- 172.21.15.43:0/308519278 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34b8003f60 0x7f34b8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.435+0000 7f34d0e2e640 1 -- 172.21.15.43:0/308519278 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b8003520 con 0x7f34b8004d40 2024-04-25T08:57:59.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.436+0000 7f34d0e2e640 1 --2- 172.21.15.43:0/308519278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34b8004d40 0x7f34b80051c0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f34c4009fc0 tx=0x7f34b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3535069d83ef64f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:59.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.436+0000 7f34caffd640 1 -- 172.21.15.43:0/308519278 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c4009c10 con 0x7f34b8004d40 2024-04-25T08:57:59.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.436+0000 7f34caffd640 1 -- 172.21.15.43:0/308519278 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34c4036070 con 0x7f34b8004d40 2024-04-25T08:57:59.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.436+0000 7f34caffd640 1 -- 172.21.15.43:0/308519278 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c4031880 con 0x7f34b8004d40 2024-04-25T08:57:59.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.437+0000 7f34d2891640 1 -- 172.21.15.43:0/308519278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34b8004d40 msgr2=0x7f34b80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:59.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.437+0000 7f34d2891640 1 --2- 172.21.15.43:0/308519278 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34b8004d40 0x7f34b80051c0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f34c4009fc0 tx=0x7f34b8005830 comp rx=0 tx=0).stop 2024-04-25T08:57:59.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.437+0000 7f34d0e2e640 1 -- 172.21.15.43:0/308519278 reap_dead start 2024-04-25T08:57:59.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.437+0000 7f34d2891640 1 -- 172.21.15.43:0/308519278 shutdown_connections 2024-04-25T08:57:59.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.437+0000 7f34d2891640 1 -- 172.21.15.43:0/308519278 >> 172.21.15.43:0/308519278 conn(0x7f34cc0d17f0 msgr2=0x7f34cc0d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:59.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.437+0000 7f34d2891640 1 -- 172.21.15.43:0/308519278 shutdown_connections 2024-04-25T08:57:59.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.437+0000 7f34d2891640 1 -- 172.21.15.43:0/308519278 wait complete. 2024-04-25T08:57:59.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.438+0000 7f34d2891640 1 Processor -- start 2024-04-25T08:57:59.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.438+0000 7f34d2891640 1 -- start start 2024-04-25T08:57:59.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.439+0000 7f34d2891640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f34cc15afa0 0x7f34cc0c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:59.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.439+0000 7f34d2891640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34cc0c9810 0x7f34cc0cce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:59.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.439+0000 7f34d2891640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34cc0cd3a0 0x7f34cc0b4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:59.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.439+0000 7f34d2891640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34cc06bd10 con 0x7f34cc0c9810 2024-04-25T08:57:59.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.439+0000 7f34d2891640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34cc0d0c10 con 0x7f34cc0cd3a0 2024-04-25T08:57:59.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.439+0000 7f34d2891640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34cc06be90 con 0x7f34cc15afa0 2024-04-25T08:57:59.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.439+0000 7f34cb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34cc0c9810 0x7f34cc0cce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:59.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.439+0000 7f34cb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34cc0c9810 0x7f34cc0cce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50168/0 (socket says 172.21.15.43:50168) 2024-04-25T08:57:59.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.439+0000 7f34cb7fe640 1 -- 172.21.15.43:0/497901841 learned_addr learned my addr 172.21.15.43:0/497901841 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:57:59.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.439+0000 7f34d0e2e640 1 --2- 172.21.15.43:0/497901841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34cc0cd3a0 0x7f34cc0b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:59.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.439+0000 7f34cbfff640 1 --2- 172.21.15.43:0/497901841 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f34cc15afa0 0x7f34cc0c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:59.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.439+0000 7f34cb7fe640 1 -- 172.21.15.43:0/497901841 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f34cc15afa0 msgr2=0x7f34cc0c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:59.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.439+0000 7f34cb7fe640 1 --2- 172.21.15.43:0/497901841 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f34cc15afa0 0x7f34cc0c92d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.439+0000 7f34cb7fe640 1 -- 172.21.15.43:0/497901841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34cc0cd3a0 msgr2=0x7f34cc0b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:59.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.439+0000 7f34cb7fe640 1 --2- 172.21.15.43:0/497901841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34cc0cd3a0 0x7f34cc0b4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.439+0000 7f34cb7fe640 1 -- 172.21.15.43:0/497901841 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34cc15b380 con 0x7f34cc0c9810 2024-04-25T08:57:59.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.439+0000 7f34d0e2e640 1 --2- 172.21.15.43:0/497901841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34cc0cd3a0 0x7f34cc0b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:59.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.439+0000 7f34cbfff640 1 --2- 172.21.15.43:0/497901841 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f34cc15afa0 0x7f34cc0c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:57:59.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.440+0000 7f34cb7fe640 1 --2- 172.21.15.43:0/497901841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34cc0c9810 0x7f34cc0cce60 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f34bc00f7f0 tx=0x7f34bc00fcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:59.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.440+0000 7f34c8ff9640 1 -- 172.21.15.43:0/497901841 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34bc0050d0 con 0x7f34cc0c9810 2024-04-25T08:57:59.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.440+0000 7f34c8ff9640 1 -- 172.21.15.43:0/497901841 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34bc018070 con 0x7f34cc0c9810 2024-04-25T08:57:59.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.440+0000 7f34d2891640 1 -- 172.21.15.43:0/497901841 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34cc0c9d20 con 0x7f34cc0c9810 2024-04-25T08:57:59.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.440+0000 7f34c8ff9640 1 -- 172.21.15.43:0/497901841 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34bc00d730 con 0x7f34cc0c9810 2024-04-25T08:57:59.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.440+0000 7f34d2891640 1 -- 172.21.15.43:0/497901841 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34cc0ca2e0 con 0x7f34cc0c9810 2024-04-25T08:57:59.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.441+0000 7f34d2891640 1 -- 172.21.15.43:0/497901841 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34cc074ef0 con 0x7f34cc0c9810 2024-04-25T08:57:59.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.441+0000 7f34c8ff9640 1 -- 172.21.15.43:0/497901841 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f34bc00d8d0 con 0x7f34cc0c9810 2024-04-25T08:57:59.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.442+0000 7f34c8ff9640 1 --2- 172.21.15.43:0/497901841 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f34ac06b4d0 0x7f34ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:57:59.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.442+0000 7f34c8ff9640 1 -- 172.21.15.43:0/497901841 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f34bc08a030 con 0x7f34cc0c9810 2024-04-25T08:57:59.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.442+0000 7f34cbfff640 1 --2- 172.21.15.43:0/497901841 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f34ac06b4d0 0x7f34ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:57:59.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.443+0000 7f34cbfff640 1 --2- 172.21.15.43:0/497901841 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f34ac06b4d0 0x7f34ac06d990 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f34b4004840 tx=0x7f34b4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:57:59.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.446+0000 7f34c8ff9640 1 -- 172.21.15.43:0/497901841 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f34bc058eb0 con 0x7f34cc0c9810 2024-04-25T08:57:59.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.627+0000 7f34d2891640 1 -- 172.21.15.43:0/497901841 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f34cc06d5f0 con 0x7f34ac06b4d0 2024-04-25T08:57:59.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.633+0000 7f34c8ff9640 1 -- 172.21.15.43:0/497901841 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+239771 (secure 0 0 0) 0x7f34cc06d5f0 con 0x7f34ac06b4d0 2024-04-25T08:57:59.625 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:57:59.626 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-25T08:57:59.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.636+0000 7f34d2891640 1 -- 172.21.15.43:0/497901841 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f34ac06b4d0 msgr2=0x7f34ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:59.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.636+0000 7f34d2891640 1 --2- 172.21.15.43:0/497901841 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f34ac06b4d0 0x7f34ac06d990 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f34b4004840 tx=0x7f34b4009040 comp rx=0 tx=0).stop 2024-04-25T08:57:59.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.636+0000 7f34d2891640 1 -- 172.21.15.43:0/497901841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34cc0c9810 msgr2=0x7f34cc0cce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:57:59.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.636+0000 7f34d2891640 1 --2- 172.21.15.43:0/497901841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34cc0c9810 0x7f34cc0cce60 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f34bc00f7f0 tx=0x7f34bc00fcb0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.636+0000 7f34d2891640 1 -- 172.21.15.43:0/497901841 shutdown_connections 2024-04-25T08:57:59.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.636+0000 7f34d2891640 1 --2- 172.21.15.43:0/497901841 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f34ac06b4d0 0x7f34ac06d990 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.636+0000 7f34d2891640 1 --2- 172.21.15.43:0/497901841 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f34cc0cd3a0 0x7f34cc0b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.636+0000 7f34d2891640 1 --2- 172.21.15.43:0/497901841 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f34cc0c9810 0x7f34cc0cce60 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.636+0000 7f34d2891640 1 --2- 172.21.15.43:0/497901841 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f34cc15afa0 0x7f34cc0c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:57:59.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.636+0000 7f34d2891640 1 -- 172.21.15.43:0/497901841 >> 172.21.15.43:0/497901841 conn(0x7f34cc0d17f0 msgr2=0x7f34cc0d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:57:59.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.636+0000 7f34d2891640 1 -- 172.21.15.43:0/497901841 shutdown_connections 2024-04-25T08:57:59.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:57:59.636+0000 7f34d2891640 1 -- 172.21.15.43:0/497901841 wait complete. 2024-04-25T08:57:59.636 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":171,"stamp":"2024-04-25T08:57:57.747195+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":338988,"kb_used_data":6252,"kb_used_omap":12,"kb_used_meta":332595,"kb_avail":754635732,"statfs":{"total":773094113280,"available":772746989568,"internally_reserved":0,"allocated":6402048,"data_stored":3323523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":340577841},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.535207"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.216743+0000","last_change":"2024-04-25T08:57:53.216743+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.216743+0000","last_undegraded":"2024-04-25T08:57:53.216743+0000","last_fullsized":"2024-04-25T08:57:53.216743+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.215243+0000","last_change":"2024-04-25T08:57:53.215243+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.215243+0000","last_undegraded":"2024-04-25T08:57:53.215243+0000","last_fullsized":"2024-04-25T08:57:53.215243+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.216709+0000","last_change":"2024-04-25T08:57:53.216709+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.216709+0000","last_undegraded":"2024-04-25T08:57:53.216709+0000","last_fullsized":"2024-04-25T08:57:53.216709+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.214788+0000","last_change":"2024-04-25T08:57:53.214788+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.214788+0000","last_undegraded":"2024-04-25T08:57:53.214788+0000","last_fullsized":"2024-04-25T08:57:53.214788+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.217098+0000","last_change":"2024-04-25T08:57:53.217098+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.217098+0000","last_undegraded":"2024-04-25T08:57:53.217098+0000","last_fullsized":"2024-04-25T08:57:53.217098+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.216664+0000","last_change":"2024-04-25T08:57:53.216664+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.216664+0000","last_undegraded":"2024-04-25T08:57:53.216664+0000","last_fullsized":"2024-04-25T08:57:53.216664+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.214289+0000","last_change":"2024-04-25T08:57:53.214289+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.214289+0000","last_undegraded":"2024-04-25T08:57:53.214289+0000","last_fullsized":"2024-04-25T08:57:53.214289+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.216716+0000","last_change":"2024-04-25T08:57:53.216716+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.216716+0000","last_undegraded":"2024-04-25T08:57:53.216716+0000","last_fullsized":"2024-04-25T08:57:53.216716+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.213648+0000","last_change":"2024-04-25T08:57:53.213648+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.213648+0000","last_undegraded":"2024-04-25T08:57:53.213648+0000","last_fullsized":"2024-04-25T08:57:53.213648+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.214791+0000","last_change":"2024-04-25T08:57:53.214791+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.214791+0000","last_undegraded":"2024-04-25T08:57:53.214791+0000","last_fullsized":"2024-04-25T08:57:53.214791+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.214000+0000","last_change":"2024-04-25T08:57:53.214000+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.214000+0000","last_undegraded":"2024-04-25T08:57:53.214000+0000","last_fullsized":"2024-04-25T08:57:53.214000+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.216775+0000","last_change":"2024-04-25T08:57:53.216775+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.216775+0000","last_undegraded":"2024-04-25T08:57:53.216775+0000","last_fullsized":"2024-04-25T08:57:53.216775+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.214777+0000","last_change":"2024-04-25T08:57:53.214777+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.214777+0000","last_undegraded":"2024-04-25T08:57:53.214777+0000","last_fullsized":"2024-04-25T08:57:53.214777+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.219307+0000","last_change":"2024-04-25T08:57:53.219307+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.219307+0000","last_undegraded":"2024-04-25T08:57:53.219307+0000","last_fullsized":"2024-04-25T08:57:53.219307+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.213691+0000","last_change":"2024-04-25T08:57:53.213691+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.213691+0000","last_undegraded":"2024-04-25T08:57:53.213691+0000","last_fullsized":"2024-04-25T08:57:53.213691+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.213640+0000","last_change":"2024-04-25T08:57:53.213640+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.213640+0000","last_undegraded":"2024-04-25T08:57:53.213640+0000","last_fullsized":"2024-04-25T08:57:53.213640+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.214281+0000","last_change":"2024-04-25T08:57:53.214281+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.214281+0000","last_undegraded":"2024-04-25T08:57:53.214281+0000","last_fullsized":"2024-04-25T08:57:53.214281+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.218465+0000","last_change":"2024-04-25T08:57:53.218465+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.218465+0000","last_undegraded":"2024-04-25T08:57:53.218465+0000","last_fullsized":"2024-04-25T08:57:53.218465+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.213640+0000","last_change":"2024-04-25T08:57:53.213640+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.213640+0000","last_undegraded":"2024-04-25T08:57:53.213640+0000","last_fullsized":"2024-04-25T08:57:53.213640+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.216771+0000","last_change":"2024-04-25T08:57:53.216771+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.216771+0000","last_undegraded":"2024-04-25T08:57:53.216771+0000","last_fullsized":"2024-04-25T08:57:53.216771+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.214220+0000","last_change":"2024-04-25T08:57:53.214220+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.214220+0000","last_undegraded":"2024-04-25T08:57:53.214220+0000","last_fullsized":"2024-04-25T08:57:53.214220+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.215905+0000","last_change":"2024-04-25T08:57:53.215905+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.215905+0000","last_undegraded":"2024-04-25T08:57:53.215905+0000","last_fullsized":"2024-04-25T08:57:53.215905+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.213640+0000","last_change":"2024-04-25T08:57:53.213640+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.213640+0000","last_undegraded":"2024-04-25T08:57:53.213640+0000","last_fullsized":"2024-04-25T08:57:53.213640+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.215838+0000","last_change":"2024-04-25T08:57:53.215838+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.215838+0000","last_undegraded":"2024-04-25T08:57:53.215838+0000","last_fullsized":"2024-04-25T08:57:53.215838+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.217997+0000","last_change":"2024-04-25T08:57:53.217997+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.217997+0000","last_undegraded":"2024-04-25T08:57:53.217997+0000","last_fullsized":"2024-04-25T08:57:53.217997+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.219296+0000","last_change":"2024-04-25T08:57:53.219296+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.219296+0000","last_undegraded":"2024-04-25T08:57:53.219296+0000","last_fullsized":"2024-04-25T08:57:53.219296+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.214302+0000","last_change":"2024-04-25T08:57:53.214302+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.214302+0000","last_undegraded":"2024-04-25T08:57:53.214302+0000","last_fullsized":"2024-04-25T08:57:53.214302+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.216286+0000","last_change":"2024-04-25T08:57:53.216286+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.216286+0000","last_undegraded":"2024-04-25T08:57:53.216286+0000","last_fullsized":"2024-04-25T08:57:53.216286+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.216724+0000","last_change":"2024-04-25T08:57:53.216724+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.216724+0000","last_undegraded":"2024-04-25T08:57:53.216724+0000","last_fullsized":"2024-04-25T08:57:53.216724+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.215814+0000","last_change":"2024-04-25T08:57:53.215814+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.215814+0000","last_undegraded":"2024-04-25T08:57:53.215814+0000","last_fullsized":"2024-04-25T08:57:53.215814+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.214076+0000","last_change":"2024-04-25T08:57:53.214076+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.214076+0000","last_undegraded":"2024-04-25T08:57:53.214076+0000","last_fullsized":"2024-04-25T08:57:53.214076+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T08:57:54.211413+0000","last_change":"2024-04-25T08:57:54.211413+0000","last_active":"2024-04-25T08:57:54.211413+0000","last_peered":"2024-04-25T08:57:54.211413+0000","last_clean":"2024-04-25T08:57:54.211413+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:54.211413+0000","last_undegraded":"2024-04-25T08:57:54.211413+0000","last_fullsized":"2024-04-25T08:57:54.211413+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":1,"reported_epoch":55,"state":"creating+peering","last_fresh":"2024-04-25T08:57:53.219243+0000","last_change":"2024-04-25T08:57:53.219243+0000","last_active":"2024-04-25T08:57:53.209247+0000","last_peered":"2024-04-25T08:57:53.209247+0000","last_clean":"2024-04-25T08:57:53.209247+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T08:57:53.219243+0000","last_undegraded":"2024-04-25T08:57:53.219243+0000","last_fullsized":"2024-04-25T08:57:53.219243+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":234,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-04-25T08:57:53.214527+0000","last_change":"2024-04-25T08:54:00.818044+0000","last_active":"2024-04-25T08:57:53.214527+0000","last_peered":"2024-04-25T08:57:53.214527+0000","last_clean":"2024-04-25T08:57:53.214527+0000","last_became_active":"2024-04-25T08:54:00.817098+0000","last_became_peered":"2024-04-25T08:54:00.817098+0000","last_unstale":"2024-04-25T08:57:53.214527+0000","last_undegraded":"2024-04-25T08:57:53.214527+0000","last_fullsized":"2024-04-25T08:57:53.214527+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_clean_scrub_stamp":"2024-04-25T08:53:59.739677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:48.129513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574897,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40720,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331120,"statfs":{"total":96636764160,"available":96595066880,"internally_reserved":0,"allocated":1241088,"data_stored":856035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40136,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331704,"statfs":{"total":96636764160,"available":96595664896,"internally_reserved":0,"allocated":647168,"data_stored":265667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574900,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40148,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331692,"statfs":{"total":96636764160,"available":96595652608,"internally_reserved":0,"allocated":659456,"data_stored":271484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45720,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326120,"statfs":{"total":96636764160,"available":96589946880,"internally_reserved":0,"allocated":659456,"data_stored":271484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54020,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317820,"statfs":{"total":96636764160,"available":96581447680,"internally_reserved":0,"allocated":647168,"data_stored":265667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":40152,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331688,"statfs":{"total":96636764160,"available":96595648512,"internally_reserved":0,"allocated":659456,"data_stored":271484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49860,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321980,"statfs":{"total":96636764160,"available":96585707520,"internally_reserved":0,"allocated":647168,"data_stored":265667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574899,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28232,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343608,"statfs":{"total":96636764160,"available":96607854592,"internally_reserved":0,"allocated":1241088,"data_stored":856035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:58:00.872 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-04-25T08:58:00.883+0000 7f7e30a49640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:58:02.641 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:58:02.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.766+0000 7f3117170640 1 Processor -- start 2024-04-25T08:58:02.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.766+0000 7f3117170640 1 -- start start 2024-04-25T08:58:02.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.766+0000 7f3117170640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f31100cf670 0x7f3110168d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:02.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.766+0000 7f3117170640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3110162810 con 0x7f311015df50 2024-04-25T08:58:02.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.766+0000 7f3117170640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31100d24a0 con 0x7f31100c8d00 2024-04-25T08:58:02.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.766+0000 7f3117170640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31100cfa80 con 0x7f31100cf670 2024-04-25T08:58:02.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.767+0000 7f3114f0c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f311015df50 0x7f31101619f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35674/0 (socket says 172.21.15.43:35674) 2024-04-25T08:58:02.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.767+0000 7f3114f0c640 1 -- 172.21.15.43:0/4192680971 learned_addr learned my addr 172.21.15.43:0/4192680971 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:02.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.767+0000 7f311570d640 1 --2- 172.21.15.43:0/4192680971 >> v2:172.21.15.193:3300/0 conn(0x7f31100cf670 0x7f3110168d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:02.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.767+0000 7f310f7fe640 1 -- 172.21.15.43:0/4192680971 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2461042306 0 0) 0x7f3110162810 con 0x7f311015df50 2024-04-25T08:58:02.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.767+0000 7f310f7fe640 1 -- 172.21.15.43:0/4192680971 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30fc003520 con 0x7f311015df50 2024-04-25T08:58:02.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.767+0000 7f310f7fe640 1 -- 172.21.15.43:0/4192680971 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2151529644 0 0) 0x7f30fc003520 con 0x7f311015df50 2024-04-25T08:58:02.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.767+0000 7f310f7fe640 1 -- 172.21.15.43:0/4192680971 >> v2:172.21.15.193:3300/0 conn(0x7f31100cf670 msgr2=0x7f3110168d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:02.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.767+0000 7f310f7fe640 1 --2- 172.21.15.43:0/4192680971 >> v2:172.21.15.193:3300/0 conn(0x7f31100cf670 0x7f3110168d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:02.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.767+0000 7f310f7fe640 1 -- 172.21.15.43:0/4192680971 >> v1:172.21.15.193:6789/0 conn(0x7f31100c8d00 legacy=0x7f3110162100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:02.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.767+0000 7f310f7fe640 1 -- 172.21.15.43:0/4192680971 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31100d2740 con 0x7f311015df50 2024-04-25T08:58:02.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.767+0000 7f310f7fe640 1 -- 172.21.15.43:0/4192680971 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f31080019f0 con 0x7f311015df50 2024-04-25T08:58:02.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.767+0000 7f310f7fe640 1 -- 172.21.15.43:0/4192680971 >> v1:172.21.15.43:6789/0 conn(0x7f311015df50 legacy=0x7f31101619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:02.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.767+0000 7f310f7fe640 1 --2- 172.21.15.43:0/4192680971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30fc0039b0 0x7f30fc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:02.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.767+0000 7f310f7fe640 1 --2- 172.21.15.43:0/4192680971 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f30fc004800 0x7f30fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:02.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.767+0000 7f310f7fe640 1 --2- 172.21.15.43:0/4192680971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30fc0051c0 0x7f30fc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:02.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.767+0000 7f310f7fe640 1 -- 172.21.15.43:0/4192680971 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31100cfa80 con 0x7f30fc0039b0 2024-04-25T08:58:02.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.767+0000 7f310f7fe640 1 -- 172.21.15.43:0/4192680971 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc00c060 con 0x7f30fc0051c0 2024-04-25T08:58:02.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.767+0000 7f310f7fe640 1 -- 172.21.15.43:0/4192680971 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc00c260 con 0x7f30fc004800 2024-04-25T08:58:02.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.767+0000 7f3114f0c640 1 --2- 172.21.15.43:0/4192680971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30fc0039b0 0x7f30fc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:02.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.768+0000 7f310ffff640 1 --2- 172.21.15.43:0/4192680971 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f30fc004800 0x7f30fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:02.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.768+0000 7f311570d640 1 --2- 172.21.15.43:0/4192680971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30fc0051c0 0x7f30fc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:02.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.768+0000 7f3114f0c640 1 -- 172.21.15.43:0/4192680971 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f30fc004800 msgr2=0x7f30fc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:02.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.768+0000 7f3114f0c640 1 --2- 172.21.15.43:0/4192680971 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f30fc004800 0x7f30fc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:02.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.768+0000 7f3114f0c640 1 -- 172.21.15.43:0/4192680971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30fc0051c0 msgr2=0x7f30fc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:02.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.768+0000 7f3114f0c640 1 --2- 172.21.15.43:0/4192680971 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f30fc0051c0 0x7f30fc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:02.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.768+0000 7f3114f0c640 1 -- 172.21.15.43:0/4192680971 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30fc003520 con 0x7f30fc0039b0 2024-04-25T08:58:02.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.768+0000 7f311570d640 1 -- 172.21.15.43:0/4192680971 reap_dead start 2024-04-25T08:58:02.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.768+0000 7f3114f0c640 1 --2- 172.21.15.43:0/4192680971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30fc0039b0 0x7f30fc003dd0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f310800a850 tx=0x7f30fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fb4f27582613cc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:02.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.769+0000 7f310f7fe640 1 -- 172.21.15.43:0/4192680971 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310800a4a0 con 0x7f30fc0039b0 2024-04-25T08:58:02.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.769+0000 7f310f7fe640 1 -- 172.21.15.43:0/4192680971 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f310802f070 con 0x7f30fc0039b0 2024-04-25T08:58:02.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.769+0000 7f310f7fe640 1 -- 172.21.15.43:0/4192680971 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310802bcb0 con 0x7f30fc0039b0 2024-04-25T08:58:02.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.769+0000 7f3117170640 1 -- 172.21.15.43:0/4192680971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30fc0039b0 msgr2=0x7f30fc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:02.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.769+0000 7f3117170640 1 --2- 172.21.15.43:0/4192680971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30fc0039b0 0x7f30fc003dd0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f310800a850 tx=0x7f30fc005830 comp rx=0 tx=0).stop 2024-04-25T08:58:02.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.769+0000 7f3117170640 1 -- 172.21.15.43:0/4192680971 shutdown_connections 2024-04-25T08:58:02.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.769+0000 7f3117170640 1 --2- 172.21.15.43:0/4192680971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30fc0039b0 0x7f30fc003dd0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:02.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.769+0000 7f3117170640 1 -- 172.21.15.43:0/4192680971 >> 172.21.15.43:0/4192680971 conn(0x7f31100d3a30 msgr2=0x7f31100d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:02.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.769+0000 7f3117170640 1 -- 172.21.15.43:0/4192680971 shutdown_connections 2024-04-25T08:58:02.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.769+0000 7f3117170640 1 -- 172.21.15.43:0/4192680971 wait complete. 2024-04-25T08:58:02.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.770+0000 7f3117170640 1 Processor -- start 2024-04-25T08:58:02.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.771+0000 7f3117170640 1 -- start start 2024-04-25T08:58:02.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.771+0000 7f3117170640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30fc0039b0 0x7f311014fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:02.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.771+0000 7f3117170640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f31101e83d0 0x7f31101f3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:02.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.771+0000 7f3117170640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31101f4280 0x7f31100c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:02.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.771+0000 7f3117170640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31100c74f0 con 0x7f30fc0039b0 2024-04-25T08:58:02.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.771+0000 7f3117170640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311015e330 con 0x7f31101f4280 2024-04-25T08:58:02.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.771+0000 7f3117170640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31100c77d0 con 0x7f31101e83d0 2024-04-25T08:58:02.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.772+0000 7f3114f0c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30fc0039b0 0x7f311014fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:02.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.772+0000 7f3114f0c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30fc0039b0 0x7f311014fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43998/0 (socket says 172.21.15.43:43998) 2024-04-25T08:58:02.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.772+0000 7f3114f0c640 1 -- 172.21.15.43:0/1999346953 learned_addr learned my addr 172.21.15.43:0/1999346953 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:02.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.772+0000 7f310ffff640 1 --2- 172.21.15.43:0/1999346953 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f31101e83d0 0x7f31101f3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:02.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.772+0000 7f311570d640 1 --2- 172.21.15.43:0/1999346953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31101f4280 0x7f31100c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:02.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.772+0000 7f3114f0c640 1 -- 172.21.15.43:0/1999346953 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f31101e83d0 msgr2=0x7f31101f3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:02.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.772+0000 7f3114f0c640 1 --2- 172.21.15.43:0/1999346953 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f31101e83d0 0x7f31101f3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:02.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.772+0000 7f3114f0c640 1 -- 172.21.15.43:0/1999346953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31101f4280 msgr2=0x7f31100c7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:02.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.772+0000 7f3114f0c640 1 --2- 172.21.15.43:0/1999346953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31101f4280 0x7f31100c7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:02.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.772+0000 7f3114f0c640 1 -- 172.21.15.43:0/1999346953 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31101f47f0 con 0x7f30fc0039b0 2024-04-25T08:58:02.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.772+0000 7f3114f0c640 1 --2- 172.21.15.43:0/1999346953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30fc0039b0 0x7f311014fe70 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f310800a970 tx=0x7f310802d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:02.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.772+0000 7f310d7fa640 1 -- 172.21.15.43:0/1999346953 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310802bc20 con 0x7f30fc0039b0 2024-04-25T08:58:02.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.772+0000 7f310d7fa640 1 -- 172.21.15.43:0/1999346953 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f310802bdc0 con 0x7f30fc0039b0 2024-04-25T08:58:02.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.772+0000 7f3117170640 1 -- 172.21.15.43:0/1999346953 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31101f4ad0 con 0x7f30fc0039b0 2024-04-25T08:58:02.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.772+0000 7f310d7fa640 1 -- 172.21.15.43:0/1999346953 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3108034690 con 0x7f30fc0039b0 2024-04-25T08:58:02.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.772+0000 7f3117170640 1 -- 172.21.15.43:0/1999346953 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31101f5090 con 0x7f30fc0039b0 2024-04-25T08:58:02.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.774+0000 7f3117170640 1 -- 172.21.15.43:0/1999346953 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31100680f0 con 0x7f30fc0039b0 2024-04-25T08:58:02.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.774+0000 7f310d7fa640 1 -- 172.21.15.43:0/1999346953 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3108034830 con 0x7f30fc0039b0 2024-04-25T08:58:02.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.774+0000 7f310d7fa640 1 --2- 172.21.15.43:0/1999346953 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f30f006b4d0 0x7f30f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:02.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.774+0000 7f310d7fa640 1 -- 172.21.15.43:0/1999346953 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f31080aa570 con 0x7f30fc0039b0 2024-04-25T08:58:02.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.775+0000 7f310ffff640 1 --2- 172.21.15.43:0/1999346953 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f30f006b4d0 0x7f30f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:02.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.775+0000 7f310ffff640 1 --2- 172.21.15.43:0/1999346953 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f30f006b4d0 0x7f30f006d990 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f31101e87f0 tx=0x7f3100009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:02.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.779+0000 7f310d7fa640 1 -- 172.21.15.43:0/1999346953 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3108078f50 con 0x7f30fc0039b0 2024-04-25T08:58:02.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.957+0000 7f3117170640 1 -- 172.21.15.43:0/1999346953 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3110074ef0 con 0x7f30f006b4d0 2024-04-25T08:58:02.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.963+0000 7f310d7fa640 1 -- 172.21.15.43:0/1999346953 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251952 (secure 0 0 0) 0x7f3110074ef0 con 0x7f30f006b4d0 2024-04-25T08:58:02.956 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:58:02.957 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-25T08:58:02.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.966+0000 7f3117170640 1 -- 172.21.15.43:0/1999346953 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f30f006b4d0 msgr2=0x7f30f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:02.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.966+0000 7f3117170640 1 --2- 172.21.15.43:0/1999346953 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f30f006b4d0 0x7f30f006d990 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f31101e87f0 tx=0x7f3100009040 comp rx=0 tx=0).stop 2024-04-25T08:58:02.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.966+0000 7f3117170640 1 -- 172.21.15.43:0/1999346953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30fc0039b0 msgr2=0x7f311014fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:02.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.966+0000 7f3117170640 1 --2- 172.21.15.43:0/1999346953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30fc0039b0 0x7f311014fe70 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f310800a970 tx=0x7f310802d070 comp rx=0 tx=0).stop 2024-04-25T08:58:02.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.967+0000 7f3117170640 1 -- 172.21.15.43:0/1999346953 shutdown_connections 2024-04-25T08:58:02.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.967+0000 7f3117170640 1 --2- 172.21.15.43:0/1999346953 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f30f006b4d0 0x7f30f006d990 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:02.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.967+0000 7f3117170640 1 --2- 172.21.15.43:0/1999346953 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31101f4280 0x7f31100c7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:02.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.967+0000 7f3117170640 1 --2- 172.21.15.43:0/1999346953 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f31101e83d0 0x7f31101f3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:02.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.967+0000 7f3117170640 1 --2- 172.21.15.43:0/1999346953 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f30fc0039b0 0x7f311014fe70 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:02.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.967+0000 7f3117170640 1 -- 172.21.15.43:0/1999346953 >> 172.21.15.43:0/1999346953 conn(0x7f31100d3a30 msgr2=0x7f3110152d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:02.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.967+0000 7f3117170640 1 -- 172.21.15.43:0/1999346953 shutdown_connections 2024-04-25T08:58:02.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:02.967+0000 7f3117170640 1 -- 172.21.15.43:0/1999346953 wait complete. 2024-04-25T08:58:02.967 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":173,"stamp":"2024-04-25T08:58:01.748418+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":339412,"kb_used_data":6664,"kb_used_omap":12,"kb_used_meta":332595,"kb_avail":754635308,"statfs":{"total":773094113280,"available":772746555392,"internally_reserved":0,"allocated":6823936,"data_stored":3512164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":340577841},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.531381"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:55.895912+0000","last_change":"2024-04-25T08:57:54.907738+0000","last_active":"2024-04-25T08:57:55.895912+0000","last_peered":"2024-04-25T08:57:55.895912+0000","last_clean":"2024-04-25T08:57:55.895912+0000","last_became_active":"2024-04-25T08:57:54.907427+0000","last_became_peered":"2024-04-25T08:57:54.907427+0000","last_unstale":"2024-04-25T08:57:55.895912+0000","last_undegraded":"2024-04-25T08:57:55.895912+0000","last_fullsized":"2024-04-25T08:57:55.895912+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:34:25.005378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.031113+0000","last_change":"2024-04-25T08:57:54.900026+0000","last_active":"2024-04-25T08:57:56.031113+0000","last_peered":"2024-04-25T08:57:56.031113+0000","last_clean":"2024-04-25T08:57:56.031113+0000","last_became_active":"2024-04-25T08:57:54.899756+0000","last_became_peered":"2024-04-25T08:57:54.899756+0000","last_unstale":"2024-04-25T08:57:56.031113+0000","last_undegraded":"2024-04-25T08:57:56.031113+0000","last_fullsized":"2024-04-25T08:57:56.031113+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:14.666866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:55.897036+0000","last_change":"2024-04-25T08:57:54.910940+0000","last_active":"2024-04-25T08:57:55.897036+0000","last_peered":"2024-04-25T08:57:55.897036+0000","last_clean":"2024-04-25T08:57:55.897036+0000","last_became_active":"2024-04-25T08:57:54.908806+0000","last_became_peered":"2024-04-25T08:57:54.908806+0000","last_unstale":"2024-04-25T08:57:55.897036+0000","last_undegraded":"2024-04-25T08:57:55.897036+0000","last_fullsized":"2024-04-25T08:57:55.897036+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:06.964882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.030605+0000","last_change":"2024-04-25T08:57:54.900465+0000","last_active":"2024-04-25T08:57:56.030605+0000","last_peered":"2024-04-25T08:57:56.030605+0000","last_clean":"2024-04-25T08:57:56.030605+0000","last_became_active":"2024-04-25T08:57:54.900132+0000","last_became_peered":"2024-04-25T08:57:54.900132+0000","last_unstale":"2024-04-25T08:57:56.030605+0000","last_undegraded":"2024-04-25T08:57:56.030605+0000","last_fullsized":"2024-04-25T08:57:56.030605+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:58.138893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:55.895909+0000","last_change":"2024-04-25T08:57:54.902044+0000","last_active":"2024-04-25T08:57:55.895909+0000","last_peered":"2024-04-25T08:57:55.895909+0000","last_clean":"2024-04-25T08:57:55.895909+0000","last_became_active":"2024-04-25T08:57:54.901653+0000","last_became_peered":"2024-04-25T08:57:54.901653+0000","last_unstale":"2024-04-25T08:57:55.895909+0000","last_undegraded":"2024-04-25T08:57:55.895909+0000","last_fullsized":"2024-04-25T08:57:55.895909+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:40.229755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.029986+0000","last_change":"2024-04-25T08:57:54.900940+0000","last_active":"2024-04-25T08:57:56.029986+0000","last_peered":"2024-04-25T08:57:56.029986+0000","last_clean":"2024-04-25T08:57:56.029986+0000","last_became_active":"2024-04-25T08:57:54.899847+0000","last_became_peered":"2024-04-25T08:57:54.899847+0000","last_unstale":"2024-04-25T08:57:56.029986+0000","last_undegraded":"2024-04-25T08:57:56.029986+0000","last_fullsized":"2024-04-25T08:57:56.029986+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:09.341246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.029961+0000","last_change":"2024-04-25T08:57:54.905637+0000","last_active":"2024-04-25T08:57:56.029961+0000","last_peered":"2024-04-25T08:57:56.029961+0000","last_clean":"2024-04-25T08:57:56.029961+0000","last_became_active":"2024-04-25T08:57:54.905374+0000","last_became_peered":"2024-04-25T08:57:54.905374+0000","last_unstale":"2024-04-25T08:57:56.029961+0000","last_undegraded":"2024-04-25T08:57:56.029961+0000","last_fullsized":"2024-04-25T08:57:56.029961+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:18.206708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.030479+0000","last_change":"2024-04-25T08:57:54.905386+0000","last_active":"2024-04-25T08:57:56.030479+0000","last_peered":"2024-04-25T08:57:56.030479+0000","last_clean":"2024-04-25T08:57:56.030479+0000","last_became_active":"2024-04-25T08:57:54.905131+0000","last_became_peered":"2024-04-25T08:57:54.905131+0000","last_unstale":"2024-04-25T08:57:56.030479+0000","last_undegraded":"2024-04-25T08:57:56.030479+0000","last_fullsized":"2024-04-25T08:57:56.030479+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:22.361272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.030439+0000","last_change":"2024-04-25T08:57:54.906951+0000","last_active":"2024-04-25T08:57:56.030439+0000","last_peered":"2024-04-25T08:57:56.030439+0000","last_clean":"2024-04-25T08:57:56.030439+0000","last_became_active":"2024-04-25T08:57:54.906629+0000","last_became_peered":"2024-04-25T08:57:54.906629+0000","last_unstale":"2024-04-25T08:57:56.030439+0000","last_undegraded":"2024-04-25T08:57:56.030439+0000","last_fullsized":"2024-04-25T08:57:56.030439+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:29.779916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.029679+0000","last_change":"2024-04-25T08:57:54.909918+0000","last_active":"2024-04-25T08:57:56.029679+0000","last_peered":"2024-04-25T08:57:56.029679+0000","last_clean":"2024-04-25T08:57:56.029679+0000","last_became_active":"2024-04-25T08:57:54.909615+0000","last_became_peered":"2024-04-25T08:57:54.909615+0000","last_unstale":"2024-04-25T08:57:56.029679+0000","last_undegraded":"2024-04-25T08:57:56.029679+0000","last_fullsized":"2024-04-25T08:57:56.029679+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:18.206708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.029981+0000","last_change":"2024-04-25T08:57:54.901959+0000","last_active":"2024-04-25T08:57:56.029981+0000","last_peered":"2024-04-25T08:57:56.029981+0000","last_clean":"2024-04-25T08:57:56.029981+0000","last_became_active":"2024-04-25T08:57:54.901614+0000","last_became_peered":"2024-04-25T08:57:54.901614+0000","last_unstale":"2024-04-25T08:57:56.029981+0000","last_undegraded":"2024-04-25T08:57:56.029981+0000","last_fullsized":"2024-04-25T08:57:56.029981+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:47.880251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.030770+0000","last_change":"2024-04-25T08:57:54.909805+0000","last_active":"2024-04-25T08:57:56.030770+0000","last_peered":"2024-04-25T08:57:56.030770+0000","last_clean":"2024-04-25T08:57:56.030770+0000","last_became_active":"2024-04-25T08:57:54.909528+0000","last_became_peered":"2024-04-25T08:57:54.909528+0000","last_unstale":"2024-04-25T08:57:56.030770+0000","last_undegraded":"2024-04-25T08:57:56.030770+0000","last_fullsized":"2024-04-25T08:57:56.030770+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:40.636048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:55.896724+0000","last_change":"2024-04-25T08:57:54.907472+0000","last_active":"2024-04-25T08:57:55.896724+0000","last_peered":"2024-04-25T08:57:55.896724+0000","last_clean":"2024-04-25T08:57:55.896724+0000","last_became_active":"2024-04-25T08:57:54.907161+0000","last_became_peered":"2024-04-25T08:57:54.907161+0000","last_unstale":"2024-04-25T08:57:55.896724+0000","last_undegraded":"2024-04-25T08:57:55.896724+0000","last_fullsized":"2024-04-25T08:57:55.896724+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:51.260921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.029988+0000","last_change":"2024-04-25T08:57:54.905010+0000","last_active":"2024-04-25T08:57:56.029988+0000","last_peered":"2024-04-25T08:57:56.029988+0000","last_clean":"2024-04-25T08:57:56.029988+0000","last_became_active":"2024-04-25T08:57:54.904668+0000","last_became_peered":"2024-04-25T08:57:54.904668+0000","last_unstale":"2024-04-25T08:57:56.029988+0000","last_undegraded":"2024-04-25T08:57:56.029988+0000","last_fullsized":"2024-04-25T08:57:56.029988+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:41.753594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.030258+0000","last_change":"2024-04-25T08:57:54.904801+0000","last_active":"2024-04-25T08:57:56.030258+0000","last_peered":"2024-04-25T08:57:56.030258+0000","last_clean":"2024-04-25T08:57:56.030258+0000","last_became_active":"2024-04-25T08:57:54.904505+0000","last_became_peered":"2024-04-25T08:57:54.904505+0000","last_unstale":"2024-04-25T08:57:56.030258+0000","last_undegraded":"2024-04-25T08:57:56.030258+0000","last_fullsized":"2024-04-25T08:57:56.030258+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:47.880251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.030262+0000","last_change":"2024-04-25T08:57:54.910935+0000","last_active":"2024-04-25T08:57:56.030262+0000","last_peered":"2024-04-25T08:57:56.030262+0000","last_clean":"2024-04-25T08:57:56.030262+0000","last_became_active":"2024-04-25T08:57:54.910601+0000","last_became_peered":"2024-04-25T08:57:54.910601+0000","last_unstale":"2024-04-25T08:57:56.030262+0000","last_undegraded":"2024-04-25T08:57:56.030262+0000","last_fullsized":"2024-04-25T08:57:56.030262+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:32.778719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.031177+0000","last_change":"2024-04-25T08:57:54.907100+0000","last_active":"2024-04-25T08:57:56.031177+0000","last_peered":"2024-04-25T08:57:56.031177+0000","last_clean":"2024-04-25T08:57:56.031177+0000","last_became_active":"2024-04-25T08:57:54.906781+0000","last_became_peered":"2024-04-25T08:57:54.906781+0000","last_unstale":"2024-04-25T08:57:56.031177+0000","last_undegraded":"2024-04-25T08:57:56.031177+0000","last_fullsized":"2024-04-25T08:57:56.031177+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:09.572541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:55.896772+0000","last_change":"2024-04-25T08:57:54.909924+0000","last_active":"2024-04-25T08:57:55.896772+0000","last_peered":"2024-04-25T08:57:55.896772+0000","last_clean":"2024-04-25T08:57:55.896772+0000","last_became_active":"2024-04-25T08:57:54.909640+0000","last_became_peered":"2024-04-25T08:57:54.909640+0000","last_unstale":"2024-04-25T08:57:55.896772+0000","last_undegraded":"2024-04-25T08:57:55.896772+0000","last_fullsized":"2024-04-25T08:57:55.896772+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:40.636048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:55.895810+0000","last_change":"2024-04-25T08:57:54.904677+0000","last_active":"2024-04-25T08:57:55.895810+0000","last_peered":"2024-04-25T08:57:55.895810+0000","last_clean":"2024-04-25T08:57:55.895810+0000","last_became_active":"2024-04-25T08:57:54.904405+0000","last_became_peered":"2024-04-25T08:57:54.904405+0000","last_unstale":"2024-04-25T08:57:55.895810+0000","last_undegraded":"2024-04-25T08:57:55.895810+0000","last_fullsized":"2024-04-25T08:57:55.895810+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:18.347971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.030310+0000","last_change":"2024-04-25T08:57:54.906589+0000","last_active":"2024-04-25T08:57:56.030310+0000","last_peered":"2024-04-25T08:57:56.030310+0000","last_clean":"2024-04-25T08:57:56.030310+0000","last_became_active":"2024-04-25T08:57:54.906321+0000","last_became_peered":"2024-04-25T08:57:54.906321+0000","last_unstale":"2024-04-25T08:57:56.030310+0000","last_undegraded":"2024-04-25T08:57:56.030310+0000","last_fullsized":"2024-04-25T08:57:56.030310+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:05.202280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:55.896967+0000","last_change":"2024-04-25T08:57:54.912981+0000","last_active":"2024-04-25T08:57:55.896967+0000","last_peered":"2024-04-25T08:57:55.896967+0000","last_clean":"2024-04-25T08:57:55.896967+0000","last_became_active":"2024-04-25T08:57:54.912651+0000","last_became_peered":"2024-04-25T08:57:54.912651+0000","last_unstale":"2024-04-25T08:57:55.896967+0000","last_undegraded":"2024-04-25T08:57:55.896967+0000","last_fullsized":"2024-04-25T08:57:55.896967+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:47.073980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.030609+0000","last_change":"2024-04-25T08:57:54.902808+0000","last_active":"2024-04-25T08:57:56.030609+0000","last_peered":"2024-04-25T08:57:56.030609+0000","last_clean":"2024-04-25T08:57:56.030609+0000","last_became_active":"2024-04-25T08:57:54.900559+0000","last_became_peered":"2024-04-25T08:57:54.900559+0000","last_unstale":"2024-04-25T08:57:56.030609+0000","last_undegraded":"2024-04-25T08:57:56.030609+0000","last_fullsized":"2024-04-25T08:57:56.030609+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:04.907321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:55.896569+0000","last_change":"2024-04-25T08:57:54.911218+0000","last_active":"2024-04-25T08:57:55.896569+0000","last_peered":"2024-04-25T08:57:55.896569+0000","last_clean":"2024-04-25T08:57:55.896569+0000","last_became_active":"2024-04-25T08:57:54.909357+0000","last_became_peered":"2024-04-25T08:57:54.909357+0000","last_unstale":"2024-04-25T08:57:55.896569+0000","last_undegraded":"2024-04-25T08:57:55.896569+0000","last_fullsized":"2024-04-25T08:57:55.896569+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:02.057084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:55.896668+0000","last_change":"2024-04-25T08:57:54.908821+0000","last_active":"2024-04-25T08:57:55.896668+0000","last_peered":"2024-04-25T08:57:55.896668+0000","last_clean":"2024-04-25T08:57:55.896668+0000","last_became_active":"2024-04-25T08:57:54.906730+0000","last_became_peered":"2024-04-25T08:57:54.906730+0000","last_unstale":"2024-04-25T08:57:55.896668+0000","last_undegraded":"2024-04-25T08:57:55.896668+0000","last_fullsized":"2024-04-25T08:57:55.896668+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:20.540246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:55.896062+0000","last_change":"2024-04-25T08:57:54.901463+0000","last_active":"2024-04-25T08:57:55.896062+0000","last_peered":"2024-04-25T08:57:55.896062+0000","last_clean":"2024-04-25T08:57:55.896062+0000","last_became_active":"2024-04-25T08:57:54.901094+0000","last_became_peered":"2024-04-25T08:57:54.901094+0000","last_unstale":"2024-04-25T08:57:55.896062+0000","last_undegraded":"2024-04-25T08:57:55.896062+0000","last_fullsized":"2024-04-25T08:57:55.896062+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:09.043895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.030487+0000","last_change":"2024-04-25T08:57:54.907582+0000","last_active":"2024-04-25T08:57:56.030487+0000","last_peered":"2024-04-25T08:57:56.030487+0000","last_clean":"2024-04-25T08:57:56.030487+0000","last_became_active":"2024-04-25T08:57:54.907317+0000","last_became_peered":"2024-04-25T08:57:54.907317+0000","last_unstale":"2024-04-25T08:57:56.030487+0000","last_undegraded":"2024-04-25T08:57:56.030487+0000","last_fullsized":"2024-04-25T08:57:56.030487+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:09.341246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.030127+0000","last_change":"2024-04-25T08:57:54.907886+0000","last_active":"2024-04-25T08:57:56.030127+0000","last_peered":"2024-04-25T08:57:56.030127+0000","last_clean":"2024-04-25T08:57:56.030127+0000","last_became_active":"2024-04-25T08:57:54.907587+0000","last_became_peered":"2024-04-25T08:57:54.907587+0000","last_unstale":"2024-04-25T08:57:56.030127+0000","last_undegraded":"2024-04-25T08:57:56.030127+0000","last_fullsized":"2024-04-25T08:57:56.030127+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:06.703136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:55.895817+0000","last_change":"2024-04-25T08:57:54.902308+0000","last_active":"2024-04-25T08:57:55.895817+0000","last_peered":"2024-04-25T08:57:55.895817+0000","last_clean":"2024-04-25T08:57:55.895817+0000","last_became_active":"2024-04-25T08:57:54.902033+0000","last_became_peered":"2024-04-25T08:57:54.902033+0000","last_unstale":"2024-04-25T08:57:55.895817+0000","last_undegraded":"2024-04-25T08:57:55.895817+0000","last_fullsized":"2024-04-25T08:57:55.895817+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:41.508551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.031257+0000","last_change":"2024-04-25T08:57:54.900364+0000","last_active":"2024-04-25T08:57:56.031257+0000","last_peered":"2024-04-25T08:57:56.031257+0000","last_clean":"2024-04-25T08:57:56.031257+0000","last_became_active":"2024-04-25T08:57:54.899797+0000","last_became_peered":"2024-04-25T08:57:54.899797+0000","last_unstale":"2024-04-25T08:57:56.031257+0000","last_undegraded":"2024-04-25T08:57:56.031257+0000","last_fullsized":"2024-04-25T08:57:56.031257+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:18.594366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:55.896384+0000","last_change":"2024-04-25T08:57:54.910100+0000","last_active":"2024-04-25T08:57:55.896384+0000","last_peered":"2024-04-25T08:57:55.896384+0000","last_clean":"2024-04-25T08:57:55.896384+0000","last_became_active":"2024-04-25T08:57:54.909789+0000","last_became_peered":"2024-04-25T08:57:54.909789+0000","last_unstale":"2024-04-25T08:57:55.896384+0000","last_undegraded":"2024-04-25T08:57:55.896384+0000","last_fullsized":"2024-04-25T08:57:55.896384+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:06.703136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:55.895733+0000","last_change":"2024-04-25T08:57:54.899195+0000","last_active":"2024-04-25T08:57:55.895733+0000","last_peered":"2024-04-25T08:57:55.895733+0000","last_clean":"2024-04-25T08:57:55.895733+0000","last_became_active":"2024-04-25T08:57:54.898920+0000","last_became_peered":"2024-04-25T08:57:54.898920+0000","last_unstale":"2024-04-25T08:57:55.895733+0000","last_undegraded":"2024-04-25T08:57:55.895733+0000","last_fullsized":"2024-04-25T08:57:55.895733+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:06:21.604497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.030511+0000","last_change":"2024-04-25T08:57:54.904664+0000","last_active":"2024-04-25T08:57:56.030511+0000","last_peered":"2024-04-25T08:57:56.030511+0000","last_clean":"2024-04-25T08:57:56.030511+0000","last_became_active":"2024-04-25T08:57:54.904407+0000","last_became_peered":"2024-04-25T08:57:54.904407+0000","last_unstale":"2024-04-25T08:57:56.030511+0000","last_undegraded":"2024-04-25T08:57:56.030511+0000","last_fullsized":"2024-04-25T08:57:56.030511+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:30.498771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.030249+0000","last_change":"2024-04-25T08:57:54.909911+0000","last_active":"2024-04-25T08:57:56.030249+0000","last_peered":"2024-04-25T08:57:56.030249+0000","last_clean":"2024-04-25T08:57:56.030249+0000","last_became_active":"2024-04-25T08:57:54.909567+0000","last_became_peered":"2024-04-25T08:57:54.909567+0000","last_unstale":"2024-04-25T08:57:56.030249+0000","last_undegraded":"2024-04-25T08:57:56.030249+0000","last_fullsized":"2024-04-25T08:57:56.030249+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:02.057084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"57'4","reported_seq":18,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.030233+0000","last_change":"2024-04-25T08:57:54.227970+0000","last_active":"2024-04-25T08:57:56.030233+0000","last_peered":"2024-04-25T08:57:56.030233+0000","last_clean":"2024-04-25T08:57:56.030233+0000","last_became_active":"2024-04-25T08:57:54.227683+0000","last_became_peered":"2024-04-25T08:57:54.227683+0000","last_unstale":"2024-04-25T08:57:56.030233+0000","last_undegraded":"2024-04-25T08:57:56.030233+0000","last_fullsized":"2024-04-25T08:57:56.030233+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:40.751428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.029853+0000","last_change":"2024-04-25T08:57:54.910603+0000","last_active":"2024-04-25T08:57:56.029853+0000","last_peered":"2024-04-25T08:57:56.029853+0000","last_clean":"2024-04-25T08:57:56.029853+0000","last_became_active":"2024-04-25T08:57:54.910341+0000","last_became_peered":"2024-04-25T08:57:54.910341+0000","last_unstale":"2024-04-25T08:57:56.029853+0000","last_undegraded":"2024-04-25T08:57:56.029853+0000","last_fullsized":"2024-04-25T08:57:56.029853+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:07.974369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"57'1","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.031167+0000","last_change":"2024-04-25T08:57:54.236301+0000","last_active":"2024-04-25T08:57:56.031167+0000","last_peered":"2024-04-25T08:57:56.031167+0000","last_clean":"2024-04-25T08:57:56.031167+0000","last_became_active":"2024-04-25T08:57:54.235755+0000","last_became_peered":"2024-04-25T08:57:54.235755+0000","last_unstale":"2024-04-25T08:57:56.031167+0000","last_undegraded":"2024-04-25T08:57:56.031167+0000","last_fullsized":"2024-04-25T08:57:56.031167+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:57.136727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.029906+0000","last_change":"2024-04-25T08:57:54.901397+0000","last_active":"2024-04-25T08:57:56.029906+0000","last_peered":"2024-04-25T08:57:56.029906+0000","last_clean":"2024-04-25T08:57:56.029906+0000","last_became_active":"2024-04-25T08:57:54.901140+0000","last_became_peered":"2024-04-25T08:57:54.901140+0000","last_unstale":"2024-04-25T08:57:56.029906+0000","last_undegraded":"2024-04-25T08:57:56.029906+0000","last_fullsized":"2024-04-25T08:57:56.029906+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:26.624364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"57'2","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.030352+0000","last_change":"2024-04-25T08:57:54.229173+0000","last_active":"2024-04-25T08:57:56.030352+0000","last_peered":"2024-04-25T08:57:56.030352+0000","last_clean":"2024-04-25T08:57:56.030352+0000","last_became_active":"2024-04-25T08:57:54.227629+0000","last_became_peered":"2024-04-25T08:57:54.227629+0000","last_unstale":"2024-04-25T08:57:56.030352+0000","last_undegraded":"2024-04-25T08:57:56.030352+0000","last_fullsized":"2024-04-25T08:57:56.030352+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:45.221029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.030535+0000","last_change":"2024-04-25T08:57:54.904490+0000","last_active":"2024-04-25T08:57:56.030535+0000","last_peered":"2024-04-25T08:57:56.030535+0000","last_clean":"2024-04-25T08:57:56.030535+0000","last_became_active":"2024-04-25T08:57:54.903876+0000","last_became_peered":"2024-04-25T08:57:54.903876+0000","last_unstale":"2024-04-25T08:57:56.030535+0000","last_undegraded":"2024-04-25T08:57:56.030535+0000","last_fullsized":"2024-04-25T08:57:56.030535+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:47.073980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"57'2","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.030704+0000","last_change":"2024-04-25T08:57:54.234168+0000","last_active":"2024-04-25T08:57:56.030704+0000","last_peered":"2024-04-25T08:57:56.030704+0000","last_clean":"2024-04-25T08:57:56.030704+0000","last_became_active":"2024-04-25T08:57:54.233853+0000","last_became_peered":"2024-04-25T08:57:54.233853+0000","last_unstale":"2024-04-25T08:57:56.030704+0000","last_undegraded":"2024-04-25T08:57:56.030704+0000","last_fullsized":"2024-04-25T08:57:56.030704+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:17.204542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.030752+0000","last_change":"2024-04-25T08:57:54.900804+0000","last_active":"2024-04-25T08:57:56.030752+0000","last_peered":"2024-04-25T08:57:56.030752+0000","last_clean":"2024-04-25T08:57:56.030752+0000","last_became_active":"2024-04-25T08:57:54.900021+0000","last_became_peered":"2024-04-25T08:57:54.900021+0000","last_unstale":"2024-04-25T08:57:56.030752+0000","last_undegraded":"2024-04-25T08:57:56.030752+0000","last_fullsized":"2024-04-25T08:57:56.030752+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:29.441556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"57'1","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.030287+0000","last_change":"2024-04-25T08:57:54.232795+0000","last_active":"2024-04-25T08:57:56.030287+0000","last_peered":"2024-04-25T08:57:56.030287+0000","last_clean":"2024-04-25T08:57:56.030287+0000","last_became_active":"2024-04-25T08:57:54.230376+0000","last_became_peered":"2024-04-25T08:57:54.230376+0000","last_unstale":"2024-04-25T08:57:56.030287+0000","last_undegraded":"2024-04-25T08:57:56.030287+0000","last_fullsized":"2024-04-25T08:57:56.030287+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:27.142179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.031125+0000","last_change":"2024-04-25T08:57:54.902890+0000","last_active":"2024-04-25T08:57:56.031125+0000","last_peered":"2024-04-25T08:57:56.031125+0000","last_clean":"2024-04-25T08:57:56.031125+0000","last_became_active":"2024-04-25T08:57:54.900098+0000","last_became_peered":"2024-04-25T08:57:54.900098+0000","last_unstale":"2024-04-25T08:57:56.031125+0000","last_undegraded":"2024-04-25T08:57:56.031125+0000","last_fullsized":"2024-04-25T08:57:56.031125+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:41.159282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"57'1","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.029922+0000","last_change":"2024-04-25T08:57:54.225811+0000","last_active":"2024-04-25T08:57:56.029922+0000","last_peered":"2024-04-25T08:57:56.029922+0000","last_clean":"2024-04-25T08:57:56.029922+0000","last_became_active":"2024-04-25T08:57:54.225484+0000","last_became_peered":"2024-04-25T08:57:54.225484+0000","last_unstale":"2024-04-25T08:57:56.029922+0000","last_undegraded":"2024-04-25T08:57:56.029922+0000","last_fullsized":"2024-04-25T08:57:56.029922+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:52.117052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:55.896555+0000","last_change":"2024-04-25T08:57:54.910284+0000","last_active":"2024-04-25T08:57:55.896555+0000","last_peered":"2024-04-25T08:57:55.896555+0000","last_clean":"2024-04-25T08:57:55.896555+0000","last_became_active":"2024-04-25T08:57:54.909973+0000","last_became_peered":"2024-04-25T08:57:54.909973+0000","last_unstale":"2024-04-25T08:57:55.896555+0000","last_undegraded":"2024-04-25T08:57:55.896555+0000","last_fullsized":"2024-04-25T08:57:55.896555+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:08.506694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.030121+0000","last_change":"2024-04-25T08:57:54.232453+0000","last_active":"2024-04-25T08:57:56.030121+0000","last_peered":"2024-04-25T08:57:56.030121+0000","last_clean":"2024-04-25T08:57:56.030121+0000","last_became_active":"2024-04-25T08:57:54.232143+0000","last_became_peered":"2024-04-25T08:57:54.232143+0000","last_unstale":"2024-04-25T08:57:56.030121+0000","last_undegraded":"2024-04-25T08:57:56.030121+0000","last_fullsized":"2024-04-25T08:57:56.030121+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:45.221029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.030919+0000","last_change":"2024-04-25T08:57:54.908305+0000","last_active":"2024-04-25T08:57:56.030919+0000","last_peered":"2024-04-25T08:57:56.030919+0000","last_clean":"2024-04-25T08:57:56.030919+0000","last_became_active":"2024-04-25T08:57:54.906935+0000","last_became_peered":"2024-04-25T08:57:54.906935+0000","last_unstale":"2024-04-25T08:57:56.030919+0000","last_undegraded":"2024-04-25T08:57:56.030919+0000","last_fullsized":"2024-04-25T08:57:56.030919+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:08.746330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.029827+0000","last_change":"2024-04-25T08:57:54.231170+0000","last_active":"2024-04-25T08:57:56.029827+0000","last_peered":"2024-04-25T08:57:56.029827+0000","last_clean":"2024-04-25T08:57:56.029827+0000","last_became_active":"2024-04-25T08:57:54.230891+0000","last_became_peered":"2024-04-25T08:57:54.230891+0000","last_unstale":"2024-04-25T08:57:56.029827+0000","last_undegraded":"2024-04-25T08:57:56.029827+0000","last_fullsized":"2024-04-25T08:57:56.029827+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:16:19.538080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.030157+0000","last_change":"2024-04-25T08:57:54.905758+0000","last_active":"2024-04-25T08:57:56.030157+0000","last_peered":"2024-04-25T08:57:56.030157+0000","last_clean":"2024-04-25T08:57:56.030157+0000","last_became_active":"2024-04-25T08:57:54.905485+0000","last_became_peered":"2024-04-25T08:57:54.905485+0000","last_unstale":"2024-04-25T08:57:56.030157+0000","last_undegraded":"2024-04-25T08:57:56.030157+0000","last_fullsized":"2024-04-25T08:57:56.030157+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:07.974369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:55.896745+0000","last_change":"2024-04-25T08:57:54.234732+0000","last_active":"2024-04-25T08:57:55.896745+0000","last_peered":"2024-04-25T08:57:55.896745+0000","last_clean":"2024-04-25T08:57:55.896745+0000","last_became_active":"2024-04-25T08:57:54.234419+0000","last_became_peered":"2024-04-25T08:57:54.234419+0000","last_unstale":"2024-04-25T08:57:55.896745+0000","last_undegraded":"2024-04-25T08:57:55.896745+0000","last_fullsized":"2024-04-25T08:57:55.896745+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:46.878085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:55.896888+0000","last_change":"2024-04-25T08:57:54.912012+0000","last_active":"2024-04-25T08:57:55.896888+0000","last_peered":"2024-04-25T08:57:55.896888+0000","last_clean":"2024-04-25T08:57:55.896888+0000","last_became_active":"2024-04-25T08:57:54.911483+0000","last_became_peered":"2024-04-25T08:57:54.911483+0000","last_unstale":"2024-04-25T08:57:55.896888+0000","last_undegraded":"2024-04-25T08:57:55.896888+0000","last_fullsized":"2024-04-25T08:57:55.896888+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:30.498771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.031246+0000","last_change":"2024-04-25T08:57:54.237444+0000","last_active":"2024-04-25T08:57:56.031246+0000","last_peered":"2024-04-25T08:57:56.031246+0000","last_clean":"2024-04-25T08:57:56.031246+0000","last_became_active":"2024-04-25T08:57:54.236899+0000","last_became_peered":"2024-04-25T08:57:54.236899+0000","last_unstale":"2024-04-25T08:57:56.031246+0000","last_undegraded":"2024-04-25T08:57:56.031246+0000","last_fullsized":"2024-04-25T08:57:56.031246+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:01.054918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.029609+0000","last_change":"2024-04-25T08:57:54.908905+0000","last_active":"2024-04-25T08:57:56.029609+0000","last_peered":"2024-04-25T08:57:56.029609+0000","last_clean":"2024-04-25T08:57:56.029609+0000","last_became_active":"2024-04-25T08:57:54.908647+0000","last_became_peered":"2024-04-25T08:57:54.908647+0000","last_unstale":"2024-04-25T08:57:56.029609+0000","last_undegraded":"2024-04-25T08:57:56.029609+0000","last_fullsized":"2024-04-25T08:57:56.029609+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:46.223195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"57'3","reported_seq":17,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:55.897299+0000","last_change":"2024-04-25T08:57:54.232433+0000","last_active":"2024-04-25T08:57:55.897299+0000","last_peered":"2024-04-25T08:57:55.897299+0000","last_clean":"2024-04-25T08:57:55.897299+0000","last_became_active":"2024-04-25T08:57:54.232110+0000","last_became_peered":"2024-04-25T08:57:54.232110+0000","last_unstale":"2024-04-25T08:57:55.897299+0000","last_undegraded":"2024-04-25T08:57:55.897299+0000","last_fullsized":"2024-04-25T08:57:55.897299+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:28.777749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:55.897247+0000","last_change":"2024-04-25T08:57:54.912568+0000","last_active":"2024-04-25T08:57:55.897247+0000","last_peered":"2024-04-25T08:57:55.897247+0000","last_clean":"2024-04-25T08:57:55.897247+0000","last_became_active":"2024-04-25T08:57:54.911748+0000","last_became_peered":"2024-04-25T08:57:54.911748+0000","last_unstale":"2024-04-25T08:57:55.897247+0000","last_undegraded":"2024-04-25T08:57:55.897247+0000","last_fullsized":"2024-04-25T08:57:55.897247+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:49.494654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"57'6","reported_seq":20,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.030226+0000","last_change":"2024-04-25T08:57:54.224711+0000","last_active":"2024-04-25T08:57:56.030226+0000","last_peered":"2024-04-25T08:57:56.030226+0000","last_clean":"2024-04-25T08:57:56.030226+0000","last_became_active":"2024-04-25T08:57:54.224436+0000","last_became_peered":"2024-04-25T08:57:54.224436+0000","last_unstale":"2024-04-25T08:57:56.030226+0000","last_undegraded":"2024-04-25T08:57:56.030226+0000","last_fullsized":"2024-04-25T08:57:56.030226+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:07.504528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.030017+0000","last_change":"2024-04-25T08:57:54.902896+0000","last_active":"2024-04-25T08:57:56.030017+0000","last_peered":"2024-04-25T08:57:56.030017+0000","last_clean":"2024-04-25T08:57:56.030017+0000","last_became_active":"2024-04-25T08:57:54.902609+0000","last_became_peered":"2024-04-25T08:57:54.902609+0000","last_unstale":"2024-04-25T08:57:56.030017+0000","last_undegraded":"2024-04-25T08:57:56.030017+0000","last_fullsized":"2024-04-25T08:57:56.030017+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:33.471973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"57'2","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.031060+0000","last_change":"2024-04-25T08:57:54.237409+0000","last_active":"2024-04-25T08:57:56.031060+0000","last_peered":"2024-04-25T08:57:56.031060+0000","last_clean":"2024-04-25T08:57:56.031060+0000","last_became_active":"2024-04-25T08:57:54.236853+0000","last_became_peered":"2024-04-25T08:57:54.236853+0000","last_unstale":"2024-04-25T08:57:56.031060+0000","last_undegraded":"2024-04-25T08:57:56.031060+0000","last_fullsized":"2024-04-25T08:57:56.031060+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:05.700970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:55.897201+0000","last_change":"2024-04-25T08:57:54.914441+0000","last_active":"2024-04-25T08:57:55.897201+0000","last_peered":"2024-04-25T08:57:55.897201+0000","last_clean":"2024-04-25T08:57:55.897201+0000","last_became_active":"2024-04-25T08:57:54.914160+0000","last_became_peered":"2024-04-25T08:57:54.914160+0000","last_unstale":"2024-04-25T08:57:55.897201+0000","last_undegraded":"2024-04-25T08:57:55.897201+0000","last_fullsized":"2024-04-25T08:57:55.897201+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:43.183812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:55.896306+0000","last_change":"2024-04-25T08:57:54.227609+0000","last_active":"2024-04-25T08:57:55.896306+0000","last_peered":"2024-04-25T08:57:55.896306+0000","last_clean":"2024-04-25T08:57:55.896306+0000","last_became_active":"2024-04-25T08:57:54.227331+0000","last_became_peered":"2024-04-25T08:57:54.227331+0000","last_unstale":"2024-04-25T08:57:55.896306+0000","last_undegraded":"2024-04-25T08:57:55.896306+0000","last_fullsized":"2024-04-25T08:57:55.896306+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:33.288018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.029791+0000","last_change":"2024-04-25T08:57:54.912318+0000","last_active":"2024-04-25T08:57:56.029791+0000","last_peered":"2024-04-25T08:57:56.029791+0000","last_clean":"2024-04-25T08:57:56.029791+0000","last_became_active":"2024-04-25T08:57:54.911080+0000","last_became_peered":"2024-04-25T08:57:54.911080+0000","last_unstale":"2024-04-25T08:57:56.029791+0000","last_undegraded":"2024-04-25T08:57:56.029791+0000","last_fullsized":"2024-04-25T08:57:56.029791+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:46.223195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.030234+0000","last_change":"2024-04-25T08:57:54.231825+0000","last_active":"2024-04-25T08:57:56.030234+0000","last_peered":"2024-04-25T08:57:56.030234+0000","last_clean":"2024-04-25T08:57:56.030234+0000","last_became_active":"2024-04-25T08:57:54.231326+0000","last_became_peered":"2024-04-25T08:57:54.231326+0000","last_unstale":"2024-04-25T08:57:56.030234+0000","last_undegraded":"2024-04-25T08:57:56.030234+0000","last_fullsized":"2024-04-25T08:57:56.030234+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:52.117052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:55.896751+0000","last_change":"2024-04-25T08:57:54.912432+0000","last_active":"2024-04-25T08:57:55.896751+0000","last_peered":"2024-04-25T08:57:55.896751+0000","last_clean":"2024-04-25T08:57:55.896751+0000","last_became_active":"2024-04-25T08:57:54.912160+0000","last_became_peered":"2024-04-25T08:57:54.912160+0000","last_unstale":"2024-04-25T08:57:55.896751+0000","last_undegraded":"2024-04-25T08:57:55.896751+0000","last_fullsized":"2024-04-25T08:57:55.896751+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:32.778719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"57'8","reported_seq":22,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:55.897141+0000","last_change":"2024-04-25T08:57:54.234461+0000","last_active":"2024-04-25T08:57:55.897141+0000","last_peered":"2024-04-25T08:57:55.897141+0000","last_clean":"2024-04-25T08:57:55.897141+0000","last_became_active":"2024-04-25T08:57:54.234149+0000","last_became_peered":"2024-04-25T08:57:54.234149+0000","last_unstale":"2024-04-25T08:57:55.897141+0000","last_undegraded":"2024-04-25T08:57:55.897141+0000","last_fullsized":"2024-04-25T08:57:55.897141+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:06.972203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:55.897065+0000","last_change":"2024-04-25T08:57:54.914760+0000","last_active":"2024-04-25T08:57:55.897065+0000","last_peered":"2024-04-25T08:57:55.897065+0000","last_clean":"2024-04-25T08:57:55.897065+0000","last_became_active":"2024-04-25T08:57:54.914461+0000","last_became_peered":"2024-04-25T08:57:54.914461+0000","last_unstale":"2024-04-25T08:57:55.897065+0000","last_undegraded":"2024-04-25T08:57:55.897065+0000","last_fullsized":"2024-04-25T08:57:55.897065+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:27.548279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"57'1","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.031017+0000","last_change":"2024-04-25T08:57:54.231942+0000","last_active":"2024-04-25T08:57:56.031017+0000","last_peered":"2024-04-25T08:57:56.031017+0000","last_clean":"2024-04-25T08:57:56.031017+0000","last_became_active":"2024-04-25T08:57:54.231632+0000","last_became_peered":"2024-04-25T08:57:54.231632+0000","last_unstale":"2024-04-25T08:57:56.031017+0000","last_undegraded":"2024-04-25T08:57:56.031017+0000","last_fullsized":"2024-04-25T08:57:56.031017+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:39.633882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.030659+0000","last_change":"2024-04-25T08:57:54.900506+0000","last_active":"2024-04-25T08:57:56.030659+0000","last_peered":"2024-04-25T08:57:56.030659+0000","last_clean":"2024-04-25T08:57:56.030659+0000","last_became_active":"2024-04-25T08:57:54.900214+0000","last_became_peered":"2024-04-25T08:57:54.900214+0000","last_unstale":"2024-04-25T08:57:56.030659+0000","last_undegraded":"2024-04-25T08:57:56.030659+0000","last_fullsized":"2024-04-25T08:57:56.030659+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:06.703136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"57'2","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.030302+0000","last_change":"2024-04-25T08:57:54.229048+0000","last_active":"2024-04-25T08:57:56.030302+0000","last_peered":"2024-04-25T08:57:56.030302+0000","last_clean":"2024-04-25T08:57:56.030302+0000","last_became_active":"2024-04-25T08:57:54.228292+0000","last_became_peered":"2024-04-25T08:57:54.228292+0000","last_unstale":"2024-04-25T08:57:56.030302+0000","last_undegraded":"2024-04-25T08:57:56.030302+0000","last_fullsized":"2024-04-25T08:57:56.030302+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:17.204542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.030465+0000","last_change":"2024-04-25T08:57:54.909203+0000","last_active":"2024-04-25T08:57:56.030465+0000","last_peered":"2024-04-25T08:57:56.030465+0000","last_clean":"2024-04-25T08:57:56.030465+0000","last_became_active":"2024-04-25T08:57:54.908916+0000","last_became_peered":"2024-04-25T08:57:54.908916+0000","last_unstale":"2024-04-25T08:57:56.030465+0000","last_undegraded":"2024-04-25T08:57:56.030465+0000","last_fullsized":"2024-04-25T08:57:56.030465+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:03.286126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"57'1","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.030484+0000","last_change":"2024-04-25T08:57:54.234206+0000","last_active":"2024-04-25T08:57:56.030484+0000","last_peered":"2024-04-25T08:57:56.030484+0000","last_clean":"2024-04-25T08:57:56.030484+0000","last_became_active":"2024-04-25T08:57:54.233883+0000","last_became_peered":"2024-04-25T08:57:54.233883+0000","last_unstale":"2024-04-25T08:57:56.030484+0000","last_undegraded":"2024-04-25T08:57:56.030484+0000","last_fullsized":"2024-04-25T08:57:56.030484+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:06.972203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:55.897269+0000","last_change":"2024-04-25T08:57:54.912358+0000","last_active":"2024-04-25T08:57:55.897269+0000","last_peered":"2024-04-25T08:57:55.897269+0000","last_clean":"2024-04-25T08:57:55.897269+0000","last_became_active":"2024-04-25T08:57:54.911997+0000","last_became_peered":"2024-04-25T08:57:54.911997+0000","last_unstale":"2024-04-25T08:57:55.897269+0000","last_undegraded":"2024-04-25T08:57:55.897269+0000","last_fullsized":"2024-04-25T08:57:55.897269+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:48:43.635237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.029957+0000","last_change":"2024-04-25T08:57:54.232620+0000","last_active":"2024-04-25T08:57:56.029957+0000","last_peered":"2024-04-25T08:57:56.029957+0000","last_clean":"2024-04-25T08:57:56.029957+0000","last_became_active":"2024-04-25T08:57:54.232285+0000","last_became_peered":"2024-04-25T08:57:54.232285+0000","last_unstale":"2024-04-25T08:57:56.029957+0000","last_undegraded":"2024-04-25T08:57:56.029957+0000","last_fullsized":"2024-04-25T08:57:56.029957+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:31.052486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.030176+0000","last_change":"2024-04-25T08:57:54.899652+0000","last_active":"2024-04-25T08:57:56.030176+0000","last_peered":"2024-04-25T08:57:56.030176+0000","last_clean":"2024-04-25T08:57:56.030176+0000","last_became_active":"2024-04-25T08:57:54.899372+0000","last_became_peered":"2024-04-25T08:57:54.899372+0000","last_unstale":"2024-04-25T08:57:56.030176+0000","last_undegraded":"2024-04-25T08:57:56.030176+0000","last_fullsized":"2024-04-25T08:57:56.030176+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:26.624364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"57'2","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.031172+0000","last_change":"2024-04-25T08:57:54.234330+0000","last_active":"2024-04-25T08:57:56.031172+0000","last_peered":"2024-04-25T08:57:56.031172+0000","last_clean":"2024-04-25T08:57:56.031172+0000","last_became_active":"2024-04-25T08:57:54.234011+0000","last_became_peered":"2024-04-25T08:57:54.234011+0000","last_unstale":"2024-04-25T08:57:56.031172+0000","last_undegraded":"2024-04-25T08:57:56.031172+0000","last_fullsized":"2024-04-25T08:57:56.031172+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:31.776553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.029989+0000","last_change":"2024-04-25T08:57:54.910287+0000","last_active":"2024-04-25T08:57:56.029989+0000","last_peered":"2024-04-25T08:57:56.029989+0000","last_clean":"2024-04-25T08:57:56.029989+0000","last_became_active":"2024-04-25T08:57:54.910013+0000","last_became_peered":"2024-04-25T08:57:54.910013+0000","last_unstale":"2024-04-25T08:57:56.029989+0000","last_undegraded":"2024-04-25T08:57:56.029989+0000","last_fullsized":"2024-04-25T08:57:56.029989+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:58.138893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.029601+0000","last_change":"2024-04-25T08:57:54.228592+0000","last_active":"2024-04-25T08:57:56.029601+0000","last_peered":"2024-04-25T08:57:56.029601+0000","last_clean":"2024-04-25T08:57:56.029601+0000","last_became_active":"2024-04-25T08:57:54.228258+0000","last_became_peered":"2024-04-25T08:57:54.228258+0000","last_unstale":"2024-04-25T08:57:56.029601+0000","last_undegraded":"2024-04-25T08:57:56.029601+0000","last_fullsized":"2024-04-25T08:57:56.029601+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:27.142179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.029749+0000","last_change":"2024-04-25T08:57:54.909860+0000","last_active":"2024-04-25T08:57:56.029749+0000","last_peered":"2024-04-25T08:57:56.029749+0000","last_clean":"2024-04-25T08:57:56.029749+0000","last_became_active":"2024-04-25T08:57:54.909559+0000","last_became_peered":"2024-04-25T08:57:54.909559+0000","last_unstale":"2024-04-25T08:57:56.029749+0000","last_undegraded":"2024-04-25T08:57:56.029749+0000","last_fullsized":"2024-04-25T08:57:56.029749+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:58:33.471973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.029708+0000","last_change":"2024-04-25T08:57:54.235934+0000","last_active":"2024-04-25T08:57:56.029708+0000","last_peered":"2024-04-25T08:57:56.029708+0000","last_clean":"2024-04-25T08:57:56.029708+0000","last_became_active":"2024-04-25T08:57:54.235710+0000","last_became_peered":"2024-04-25T08:57:54.235710+0000","last_unstale":"2024-04-25T08:57:56.029708+0000","last_undegraded":"2024-04-25T08:57:56.029708+0000","last_fullsized":"2024-04-25T08:57:56.029708+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:31.052486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.030665+0000","last_change":"2024-04-25T08:57:54.902584+0000","last_active":"2024-04-25T08:57:56.030665+0000","last_peered":"2024-04-25T08:57:56.030665+0000","last_clean":"2024-04-25T08:57:56.030665+0000","last_became_active":"2024-04-25T08:57:54.902285+0000","last_became_peered":"2024-04-25T08:57:54.902285+0000","last_unstale":"2024-04-25T08:57:56.030665+0000","last_undegraded":"2024-04-25T08:57:56.030665+0000","last_fullsized":"2024-04-25T08:57:56.030665+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:02.057084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"57'2","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.029795+0000","last_change":"2024-04-25T08:57:54.235230+0000","last_active":"2024-04-25T08:57:56.029795+0000","last_peered":"2024-04-25T08:57:56.029795+0000","last_clean":"2024-04-25T08:57:56.029795+0000","last_became_active":"2024-04-25T08:57:54.234909+0000","last_became_peered":"2024-04-25T08:57:54.234909+0000","last_unstale":"2024-04-25T08:57:56.029795+0000","last_undegraded":"2024-04-25T08:57:56.029795+0000","last_fullsized":"2024-04-25T08:57:56.029795+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:07.504528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.030707+0000","last_change":"2024-04-25T08:57:54.908955+0000","last_active":"2024-04-25T08:57:56.030707+0000","last_peered":"2024-04-25T08:57:56.030707+0000","last_clean":"2024-04-25T08:57:56.030707+0000","last_became_active":"2024-04-25T08:57:54.908657+0000","last_became_peered":"2024-04-25T08:57:54.908657+0000","last_unstale":"2024-04-25T08:57:56.030707+0000","last_undegraded":"2024-04-25T08:57:56.030707+0000","last_fullsized":"2024-04-25T08:57:56.030707+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:06:32.778719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"58'1","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.030412+0000","last_change":"2024-04-25T08:57:54.225552+0000","last_active":"2024-04-25T08:57:56.030412+0000","last_peered":"2024-04-25T08:57:56.030412+0000","last_clean":"2024-04-25T08:57:56.030412+0000","last_became_active":"2024-04-25T08:57:54.225193+0000","last_became_peered":"2024-04-25T08:57:54.225193+0000","last_unstale":"2024-04-25T08:57:56.030412+0000","last_undegraded":"2024-04-25T08:57:56.030412+0000","last_fullsized":"2024-04-25T08:57:56.030412+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:06.972203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.029765+0000","last_change":"2024-04-25T08:57:54.909174+0000","last_active":"2024-04-25T08:57:56.029765+0000","last_peered":"2024-04-25T08:57:56.029765+0000","last_clean":"2024-04-25T08:57:56.029765+0000","last_became_active":"2024-04-25T08:57:54.908869+0000","last_became_peered":"2024-04-25T08:57:54.908869+0000","last_unstale":"2024-04-25T08:57:56.029765+0000","last_undegraded":"2024-04-25T08:57:56.029765+0000","last_fullsized":"2024-04-25T08:57:56.029765+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:41.753594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:55.896884+0000","last_change":"2024-04-25T08:57:54.230273+0000","last_active":"2024-04-25T08:57:55.896884+0000","last_peered":"2024-04-25T08:57:55.896884+0000","last_clean":"2024-04-25T08:57:55.896884+0000","last_became_active":"2024-04-25T08:57:54.229991+0000","last_became_peered":"2024-04-25T08:57:54.229991+0000","last_unstale":"2024-04-25T08:57:55.896884+0000","last_undegraded":"2024-04-25T08:57:55.896884+0000","last_fullsized":"2024-04-25T08:57:55.896884+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:45.925105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.030294+0000","last_change":"2024-04-25T08:57:54.904082+0000","last_active":"2024-04-25T08:57:56.030294+0000","last_peered":"2024-04-25T08:57:56.030294+0000","last_clean":"2024-04-25T08:57:56.030294+0000","last_became_active":"2024-04-25T08:57:54.903808+0000","last_became_peered":"2024-04-25T08:57:54.903808+0000","last_unstale":"2024-04-25T08:57:56.030294+0000","last_undegraded":"2024-04-25T08:57:56.030294+0000","last_fullsized":"2024-04-25T08:57:56.030294+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:10:40.636048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.030803+0000","last_change":"2024-04-25T08:57:54.231405+0000","last_active":"2024-04-25T08:57:56.030803+0000","last_peered":"2024-04-25T08:57:56.030803+0000","last_clean":"2024-04-25T08:57:56.030803+0000","last_became_active":"2024-04-25T08:57:54.231101+0000","last_became_peered":"2024-04-25T08:57:54.231101+0000","last_unstale":"2024-04-25T08:57:56.030803+0000","last_undegraded":"2024-04-25T08:57:56.030803+0000","last_fullsized":"2024-04-25T08:57:56.030803+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:40.751428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.030909+0000","last_change":"2024-04-25T08:57:54.900311+0000","last_active":"2024-04-25T08:57:56.030909+0000","last_peered":"2024-04-25T08:57:56.030909+0000","last_clean":"2024-04-25T08:57:56.030909+0000","last_became_active":"2024-04-25T08:57:54.899702+0000","last_became_peered":"2024-04-25T08:57:54.899702+0000","last_unstale":"2024-04-25T08:57:56.030909+0000","last_undegraded":"2024-04-25T08:57:56.030909+0000","last_fullsized":"2024-04-25T08:57:56.030909+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:35.406306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.029308+0000","last_change":"2024-04-25T08:57:54.236073+0000","last_active":"2024-04-25T08:57:56.029308+0000","last_peered":"2024-04-25T08:57:56.029308+0000","last_clean":"2024-04-25T08:57:56.029308+0000","last_became_active":"2024-04-25T08:57:54.235642+0000","last_became_peered":"2024-04-25T08:57:54.235642+0000","last_unstale":"2024-04-25T08:57:56.029308+0000","last_undegraded":"2024-04-25T08:57:56.029308+0000","last_fullsized":"2024-04-25T08:57:56.029308+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:21.650933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:55.897228+0000","last_change":"2024-04-25T08:57:54.913680+0000","last_active":"2024-04-25T08:57:55.897228+0000","last_peered":"2024-04-25T08:57:55.897228+0000","last_clean":"2024-04-25T08:57:55.897228+0000","last_became_active":"2024-04-25T08:57:54.913020+0000","last_became_peered":"2024-04-25T08:57:54.913020+0000","last_unstale":"2024-04-25T08:57:55.897228+0000","last_undegraded":"2024-04-25T08:57:55.897228+0000","last_fullsized":"2024-04-25T08:57:55.897228+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:05.792017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.030043+0000","last_change":"2024-04-25T08:57:54.228066+0000","last_active":"2024-04-25T08:57:56.030043+0000","last_peered":"2024-04-25T08:57:56.030043+0000","last_clean":"2024-04-25T08:57:56.030043+0000","last_became_active":"2024-04-25T08:57:54.227753+0000","last_became_peered":"2024-04-25T08:57:54.227753+0000","last_unstale":"2024-04-25T08:57:56.030043+0000","last_undegraded":"2024-04-25T08:57:56.030043+0000","last_fullsized":"2024-04-25T08:57:56.030043+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:21.650933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.029392+0000","last_change":"2024-04-25T08:57:54.909856+0000","last_active":"2024-04-25T08:57:56.029392+0000","last_peered":"2024-04-25T08:57:56.029392+0000","last_clean":"2024-04-25T08:57:56.029392+0000","last_became_active":"2024-04-25T08:57:54.909575+0000","last_became_peered":"2024-04-25T08:57:54.909575+0000","last_unstale":"2024-04-25T08:57:56.029392+0000","last_undegraded":"2024-04-25T08:57:56.029392+0000","last_fullsized":"2024-04-25T08:57:56.029392+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:53.119218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.029609+0000","last_change":"2024-04-25T08:57:54.234093+0000","last_active":"2024-04-25T08:57:56.029609+0000","last_peered":"2024-04-25T08:57:56.029609+0000","last_clean":"2024-04-25T08:57:56.029609+0000","last_became_active":"2024-04-25T08:57:54.233813+0000","last_became_peered":"2024-04-25T08:57:54.233813+0000","last_unstale":"2024-04-25T08:57:56.029609+0000","last_undegraded":"2024-04-25T08:57:56.029609+0000","last_fullsized":"2024-04-25T08:57:56.029609+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:27.849052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.030951+0000","last_change":"2024-04-25T08:57:54.907051+0000","last_active":"2024-04-25T08:57:56.030951+0000","last_peered":"2024-04-25T08:57:56.030951+0000","last_clean":"2024-04-25T08:57:56.030951+0000","last_became_active":"2024-04-25T08:57:54.906772+0000","last_became_peered":"2024-04-25T08:57:54.906772+0000","last_unstale":"2024-04-25T08:57:56.030951+0000","last_undegraded":"2024-04-25T08:57:56.030951+0000","last_fullsized":"2024-04-25T08:57:56.030951+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:54.442729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.030668+0000","last_change":"2024-04-25T08:57:54.232831+0000","last_active":"2024-04-25T08:57:56.030668+0000","last_peered":"2024-04-25T08:57:56.030668+0000","last_clean":"2024-04-25T08:57:56.030668+0000","last_became_active":"2024-04-25T08:57:54.232557+0000","last_became_peered":"2024-04-25T08:57:54.232557+0000","last_unstale":"2024-04-25T08:57:56.030668+0000","last_undegraded":"2024-04-25T08:57:56.030668+0000","last_fullsized":"2024-04-25T08:57:56.030668+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:08.339080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":12,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:56.030621+0000","last_change":"2024-04-25T08:57:54.904750+0000","last_active":"2024-04-25T08:57:56.030621+0000","last_peered":"2024-04-25T08:57:56.030621+0000","last_clean":"2024-04-25T08:57:56.030621+0000","last_became_active":"2024-04-25T08:57:54.904473+0000","last_became_peered":"2024-04-25T08:57:54.904473+0000","last_unstale":"2024-04-25T08:57:56.030621+0000","last_undegraded":"2024-04-25T08:57:56.030621+0000","last_fullsized":"2024-04-25T08:57:56.030621+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:54.211413+0000","last_clean_scrub_stamp":"2024-04-25T08:57:54.211413+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:58.599997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":14,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:55.897069+0000","last_change":"2024-04-25T08:57:54.231118+0000","last_active":"2024-04-25T08:57:55.897069+0000","last_peered":"2024-04-25T08:57:55.897069+0000","last_clean":"2024-04-25T08:57:55.897069+0000","last_became_active":"2024-04-25T08:57:54.230780+0000","last_became_peered":"2024-04-25T08:57:54.230780+0000","last_unstale":"2024-04-25T08:57:55.897069+0000","last_undegraded":"2024-04-25T08:57:55.897069+0000","last_fullsized":"2024-04-25T08:57:55.897069+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:57:53.209247+0000","last_clean_scrub_stamp":"2024-04-25T08:57:53.209247+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:17.876168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":240,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-25T08:57:55.897173+0000","last_change":"2024-04-25T08:54:00.818044+0000","last_active":"2024-04-25T08:57:55.897173+0000","last_peered":"2024-04-25T08:57:55.897173+0000","last_clean":"2024-04-25T08:57:55.897173+0000","last_became_active":"2024-04-25T08:54:00.817098+0000","last_became_peered":"2024-04-25T08:54:00.817098+0000","last_unstale":"2024-04-25T08:57:55.897173+0000","last_undegraded":"2024-04-25T08:57:55.897173+0000","last_fullsized":"2024-04-25T08:57:55.897173+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_clean_scrub_stamp":"2024-04-25T08:53:59.739677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:48.129513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574898,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40768,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331072,"statfs":{"total":96636764160,"available":96595017728,"internally_reserved":0,"allocated":1298432,"data_stored":881354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574899,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40188,"kb_used_data":680,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331652,"statfs":{"total":96636764160,"available":96595611648,"internally_reserved":0,"allocated":696320,"data_stored":290826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574901,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40196,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331644,"statfs":{"total":96636764160,"available":96595603456,"internally_reserved":0,"allocated":704512,"data_stored":291478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574899,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45768,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94326072,"statfs":{"total":96636764160,"available":96589897728,"internally_reserved":0,"allocated":708608,"data_stored":292438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574899,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54076,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317764,"statfs":{"total":96636764160,"available":96581390336,"internally_reserved":0,"allocated":704512,"data_stored":290882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574899,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40196,"kb_used_data":688,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331644,"statfs":{"total":96636764160,"available":96595603456,"internally_reserved":0,"allocated":704512,"data_stored":290882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574899,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49920,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321920,"statfs":{"total":96636764160,"available":96585646080,"internally_reserved":0,"allocated":700416,"data_stored":290916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574900,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28300,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343540,"statfs":{"total":96636764160,"available":96607784960,"internally_reserved":0,"allocated":1306624,"data_stored":883388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:58:02.971 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T08:58:02.972 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T08:58:03.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.126+0000 7f79da4d8640 1 Processor -- start 2024-04-25T08:58:03.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.126+0000 7f79da4d8640 1 -- start start 2024-04-25T08:58:03.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.127+0000 7f79da4d8640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79d40d04a0 0x7f79d415ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:03.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.127+0000 7f79da4d8640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f79d415dec0 0x7f79d40d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:03.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.127+0000 7f79da4d8640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79d40d0dc0 con 0x7f79d40ce8c0 2024-04-25T08:58:03.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.127+0000 7f79da4d8640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79d40cf110 con 0x7f79d415dec0 2024-04-25T08:58:03.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.127+0000 7f79da4d8640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79d40cf3f0 con 0x7f79d40d04a0 2024-04-25T08:58:03.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.127+0000 7f79d8a75640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f79d40ce8c0 0x7f79d41617a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35686/0 (socket says 172.21.15.43:35686) 2024-04-25T08:58:03.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.127+0000 7f79d8a75640 1 -- 172.21.15.43:0/3810362130 learned_addr learned my addr 172.21.15.43:0/3810362130 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:03.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.127+0000 7f79d3fff640 1 --2- 172.21.15.43:0/3810362130 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79d40d04a0 0x7f79d415ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:03.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.127+0000 7f79d37fe640 1 --2- 172.21.15.43:0/3810362130 >> v2:172.21.15.193:3300/0 conn(0x7f79d415dec0 0x7f79d40d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:03.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.127+0000 7f79d2ffd640 1 -- 172.21.15.43:0/3810362130 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3249291070 0 0) 0x7f79d40d0dc0 con 0x7f79d40ce8c0 2024-04-25T08:58:03.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.127+0000 7f79d2ffd640 1 -- 172.21.15.43:0/3810362130 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79c0003520 con 0x7f79d40ce8c0 2024-04-25T08:58:03.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.127+0000 7f79d3fff640 1 -- 172.21.15.43:0/3810362130 >> v2:172.21.15.193:3300/0 conn(0x7f79d415dec0 msgr2=0x7f79d40d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:03.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.127+0000 7f79d3fff640 1 --2- 172.21.15.43:0/3810362130 >> v2:172.21.15.193:3300/0 conn(0x7f79d415dec0 0x7f79d40d0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:03.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.128+0000 7f79d3fff640 1 -- 172.21.15.43:0/3810362130 >> v1:172.21.15.43:6789/0 conn(0x7f79d40ce8c0 legacy=0x7f79d41617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:03.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.128+0000 7f79d2ffd640 1 -- 172.21.15.43:0/3810362130 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1068063101 0 0) 0x7f79c0003520 con 0x7f79d40ce8c0 2024-04-25T08:58:03.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.128+0000 7f79d3fff640 1 -- 172.21.15.43:0/3810362130 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d40cf6d0 con 0x7f79d40d04a0 2024-04-25T08:58:03.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.128+0000 7f79d3fff640 1 --2- 172.21.15.43:0/3810362130 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79d40d04a0 0x7f79d415ed90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f79bc0097a0 tx=0x7f79bc02a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=943b7af8c9215a92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:03.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.128+0000 7f79d2ffd640 1 -- 172.21.15.43:0/3810362130 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79bc037070 con 0x7f79d40d04a0 2024-04-25T08:58:03.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.129+0000 7f79d2ffd640 1 -- 172.21.15.43:0/3810362130 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79bc02aba0 con 0x7f79d40d04a0 2024-04-25T08:58:03.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.129+0000 7f79d2ffd640 1 -- 172.21.15.43:0/3810362130 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79cc0019f0 con 0x7f79d40d04a0 2024-04-25T08:58:03.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.129+0000 7f79da4d8640 1 -- 172.21.15.43:0/3810362130 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79d40d04a0 msgr2=0x7f79d415ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:03.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.129+0000 7f79da4d8640 1 --2- 172.21.15.43:0/3810362130 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79d40d04a0 0x7f79d415ed90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f79bc0097a0 tx=0x7f79bc02a110 comp rx=0 tx=0).stop 2024-04-25T08:58:03.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.129+0000 7f79da4d8640 1 -- 172.21.15.43:0/3810362130 shutdown_connections 2024-04-25T08:58:03.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.129+0000 7f79da4d8640 1 --2- 172.21.15.43:0/3810362130 >> v2:172.21.15.193:3300/0 conn(0x7f79d415dec0 0x7f79d40d0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:03.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.129+0000 7f79da4d8640 1 --2- 172.21.15.43:0/3810362130 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79d40d04a0 0x7f79d415ed90 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:03.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.129+0000 7f79da4d8640 1 -- 172.21.15.43:0/3810362130 >> 172.21.15.43:0/3810362130 conn(0x7f79d409cd40 msgr2=0x7f79d40d3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:03.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.129+0000 7f79da4d8640 1 -- 172.21.15.43:0/3810362130 shutdown_connections 2024-04-25T08:58:03.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.129+0000 7f79da4d8640 1 -- 172.21.15.43:0/3810362130 wait complete. 2024-04-25T08:58:03.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.130+0000 7f79da4d8640 1 Processor -- start 2024-04-25T08:58:03.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.131+0000 7f79da4d8640 1 -- start start 2024-04-25T08:58:03.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.131+0000 7f79da4d8640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79d40ce8c0 0x7f79d40b8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:03.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.131+0000 7f79da4d8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79d40d04a0 0x7f79d40cd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:03.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.131+0000 7f79da4d8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79d415dec0 0x7f79d40b6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:03.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.131+0000 7f79da4d8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d40cecf0 con 0x7f79d415dec0 2024-04-25T08:58:03.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.131+0000 7f79da4d8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d40cfc90 con 0x7f79d40d04a0 2024-04-25T08:58:03.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.131+0000 7f79da4d8640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79d40cee70 con 0x7f79d40ce8c0 2024-04-25T08:58:03.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.131+0000 7f79d8a75640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79d415dec0 0x7f79d40b6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:03.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.131+0000 7f79d8a75640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79d415dec0 0x7f79d40b6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44004/0 (socket says 172.21.15.43:44004) 2024-04-25T08:58:03.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.131+0000 7f79d8a75640 1 -- 172.21.15.43:0/1351123337 learned_addr learned my addr 172.21.15.43:0/1351123337 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:03.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.132+0000 7f79d37fe640 1 --2- 172.21.15.43:0/1351123337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79d40d04a0 0x7f79d40cd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:03.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.132+0000 7f79d3fff640 1 --2- 172.21.15.43:0/1351123337 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79d40ce8c0 0x7f79d40b8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:03.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.132+0000 7f79d8a75640 1 -- 172.21.15.43:0/1351123337 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79d40ce8c0 msgr2=0x7f79d40b8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:03.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.132+0000 7f79d8a75640 1 --2- 172.21.15.43:0/1351123337 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79d40ce8c0 0x7f79d40b8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:03.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.132+0000 7f79d8a75640 1 -- 172.21.15.43:0/1351123337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79d40d04a0 msgr2=0x7f79d40cd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:03.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.132+0000 7f79d8a75640 1 --2- 172.21.15.43:0/1351123337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79d40d04a0 0x7f79d40cd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:03.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.132+0000 7f79d8a75640 1 -- 172.21.15.43:0/1351123337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d40cbb00 con 0x7f79d415dec0 2024-04-25T08:58:03.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.132+0000 7f79d37fe640 1 --2- 172.21.15.43:0/1351123337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79d40d04a0 0x7f79d40cd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:03.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.132+0000 7f79d8a75640 1 --2- 172.21.15.43:0/1351123337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79d415dec0 0x7f79d40b6c90 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f79cc00cc30 tx=0x7f79cc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:03.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.132+0000 7f79d0ff9640 1 -- 172.21.15.43:0/1351123337 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79cc017070 con 0x7f79d415dec0 2024-04-25T08:58:03.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.132+0000 7f79d0ff9640 1 -- 172.21.15.43:0/1351123337 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79cc00ce40 con 0x7f79d415dec0 2024-04-25T08:58:03.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.132+0000 7f79da4d8640 1 -- 172.21.15.43:0/1351123337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79d40cbde0 con 0x7f79d415dec0 2024-04-25T08:58:03.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.132+0000 7f79d0ff9640 1 -- 172.21.15.43:0/1351123337 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79cc012600 con 0x7f79d415dec0 2024-04-25T08:58:03.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.132+0000 7f79da4d8640 1 -- 172.21.15.43:0/1351123337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79d40cc3a0 con 0x7f79d415dec0 2024-04-25T08:58:03.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.133+0000 7f79da4d8640 1 -- 172.21.15.43:0/1351123337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79d4074f60 con 0x7f79d415dec0 2024-04-25T08:58:03.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.134+0000 7f79d0ff9640 1 -- 172.21.15.43:0/1351123337 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f79cc01a070 con 0x7f79d415dec0 2024-04-25T08:58:03.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.134+0000 7f79d0ff9640 1 --2- 172.21.15.43:0/1351123337 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f79b406b4d0 0x7f79b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:03.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.134+0000 7f79d0ff9640 1 -- 172.21.15.43:0/1351123337 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f79cc089f80 con 0x7f79d415dec0 2024-04-25T08:58:03.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.135+0000 7f79d3fff640 1 --2- 172.21.15.43:0/1351123337 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f79b406b4d0 0x7f79b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:03.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.135+0000 7f79d3fff640 1 --2- 172.21.15.43:0/1351123337 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f79b406b4d0 0x7f79b406d990 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f79bc034000 tx=0x7f79bc02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:03.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.138+0000 7f79d0ff9640 1 -- 172.21.15.43:0/1351123337 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f79cc058840 con 0x7f79d415dec0 2024-04-25T08:58:03.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.388+0000 7f79da4d8640 1 -- 172.21.15.43:0/1351123337 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f79d408d7e0 con 0x7f79d415dec0 2024-04-25T08:58:03.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.390+0000 7f79d0ff9640 1 -- 172.21.15.43:0/1351123337 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7f79cc05c8e0 con 0x7f79d415dec0 2024-04-25T08:58:03.380 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2024-04-25T08:58:03.380 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQD8GSpmVOjWGxAAFNmxeGi0h2D//pa6Ne3eQw== 2024-04-25T08:58:03.380 INFO:teuthology.orchestra.run.smithi043.stdout: caps mds = "allow" 2024-04-25T08:58:03.380 INFO:teuthology.orchestra.run.smithi043.stdout: caps mon = "allow r" 2024-04-25T08:58:03.380 INFO:teuthology.orchestra.run.smithi043.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T08:58:03.381 INFO:teuthology.orchestra.run.smithi043.stderr:updated caps for client.0 2024-04-25T08:58:03.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.392+0000 7f79da4d8640 1 -- 172.21.15.43:0/1351123337 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f79b406b4d0 msgr2=0x7f79b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:03.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.392+0000 7f79da4d8640 1 --2- 172.21.15.43:0/1351123337 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f79b406b4d0 0x7f79b406d990 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f79bc034000 tx=0x7f79bc02e000 comp rx=0 tx=0).stop 2024-04-25T08:58:03.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.392+0000 7f79da4d8640 1 -- 172.21.15.43:0/1351123337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79d415dec0 msgr2=0x7f79d40b6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:03.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.392+0000 7f79da4d8640 1 --2- 172.21.15.43:0/1351123337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79d415dec0 0x7f79d40b6c90 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f79cc00cc30 tx=0x7f79cc00a430 comp rx=0 tx=0).stop 2024-04-25T08:58:03.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.392+0000 7f79da4d8640 1 -- 172.21.15.43:0/1351123337 shutdown_connections 2024-04-25T08:58:03.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.392+0000 7f79da4d8640 1 --2- 172.21.15.43:0/1351123337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f79d415dec0 0x7f79d40b6c90 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:03.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.392+0000 7f79da4d8640 1 --2- 172.21.15.43:0/1351123337 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f79b406b4d0 0x7f79b406d990 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:03.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.392+0000 7f79da4d8640 1 --2- 172.21.15.43:0/1351123337 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f79d40d04a0 0x7f79d40cd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:03.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.392+0000 7f79da4d8640 1 --2- 172.21.15.43:0/1351123337 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f79d40ce8c0 0x7f79d40b8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:03.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.392+0000 7f79da4d8640 1 -- 172.21.15.43:0/1351123337 >> 172.21.15.43:0/1351123337 conn(0x7f79d409cd40 msgr2=0x7f79d40d0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:03.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.393+0000 7f79da4d8640 1 -- 172.21.15.43:0/1351123337 shutdown_connections 2024-04-25T08:58:03.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.393+0000 7f79da4d8640 1 -- 172.21.15.43:0/1351123337 wait complete. 2024-04-25T08:58:03.393 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:58:03.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.504+0000 7fc7fc5f5640 1 Processor -- start 2024-04-25T08:58:03.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.504+0000 7fc7fc5f5640 1 -- start start 2024-04-25T08:58:03.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.505+0000 7fc7fc5f5640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fc7f40d09b0 0x7fc7f40b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:03.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.505+0000 7fc7fc5f5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7f40ce5e0 0x7fc7f40cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:03.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.505+0000 7fc7fc5f5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7f40b4c30 con 0x7fc7f4161390 2024-04-25T08:58:03.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.505+0000 7fc7fc5f5640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7f40d1910 con 0x7fc7f40d09b0 2024-04-25T08:58:03.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.505+0000 7fc7fc5f5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f40cf2c0 con 0x7fc7f40ce5e0 2024-04-25T08:58:03.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.505+0000 7fc7fa391640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc7f4161390 0x7fc7f40d02a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35688/0 (socket says 172.21.15.43:35688) 2024-04-25T08:58:03.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.505+0000 7fc7fa391640 1 -- 172.21.15.43:0/3309228820 learned_addr learned my addr 172.21.15.43:0/3309228820 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:03.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.505+0000 7fc7fab92640 1 --2- 172.21.15.43:0/3309228820 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7f40ce5e0 0x7fc7f40cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:03.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.505+0000 7fc7f938f640 1 -- 172.21.15.43:0/3309228820 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4215969133 0 0) 0x7fc7f0001490 con 0x7fc7f4161390 2024-04-25T08:58:03.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.505+0000 7fc7f9b90640 1 --2- 172.21.15.43:0/3309228820 >> v2:172.21.15.193:3300/0 conn(0x7fc7f40d09b0 0x7fc7f40b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:03.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.506+0000 7fc7f938f640 1 -- 172.21.15.43:0/3309228820 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7e0003520 con 0x7fc7f4161390 2024-04-25T08:58:03.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.506+0000 7fc7f938f640 1 -- 172.21.15.43:0/3309228820 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2706680406 0 0) 0x7fc7e0003520 con 0x7fc7f4161390 2024-04-25T08:58:03.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.506+0000 7fc7f938f640 1 -- 172.21.15.43:0/3309228820 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7f40ce5e0 msgr2=0x7fc7f40cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:03.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.506+0000 7fc7f938f640 1 --2- 172.21.15.43:0/3309228820 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7f40ce5e0 0x7fc7f40cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:03.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.506+0000 7fc7f938f640 1 -- 172.21.15.43:0/3309228820 >> v2:172.21.15.193:3300/0 conn(0x7fc7f40d09b0 msgr2=0x7fc7f40b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:03.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.506+0000 7fc7f938f640 1 --2- 172.21.15.43:0/3309228820 >> v2:172.21.15.193:3300/0 conn(0x7fc7f40d09b0 0x7fc7f40b6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:03.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.506+0000 7fc7f938f640 1 -- 172.21.15.43:0/3309228820 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f40cf5a0 con 0x7fc7f4161390 2024-04-25T08:58:03.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.506+0000 7fc7f938f640 1 -- 172.21.15.43:0/3309228820 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7fc7f0001bc0 con 0x7fc7f4161390 2024-04-25T08:58:03.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.506+0000 7fc7f938f640 1 -- 172.21.15.43:0/3309228820 >> v1:172.21.15.43:6789/0 conn(0x7fc7f4161390 legacy=0x7fc7f40d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:03.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.506+0000 7fc7f938f640 1 --2- 172.21.15.43:0/3309228820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7e00039b0 0x7fc7e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:03.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.506+0000 7fc7f938f640 1 --2- 172.21.15.43:0/3309228820 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7f40ce5e0 0x7fc7e0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:03.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.506+0000 7fc7f938f640 1 --2- 172.21.15.43:0/3309228820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7e0004d40 0x7fc7e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:03.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.506+0000 7fc7f938f640 1 -- 172.21.15.43:0/3309228820 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f40d1910 con 0x7fc7e00039b0 2024-04-25T08:58:03.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.506+0000 7fc7f938f640 1 -- 172.21.15.43:0/3309228820 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f40cf2c0 con 0x7fc7e0004d40 2024-04-25T08:58:03.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.506+0000 7fc7f938f640 1 -- 172.21.15.43:0/3309228820 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e000b9c0 con 0x7fc7f40ce5e0 2024-04-25T08:58:03.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.506+0000 7fc7fa391640 1 --2- 172.21.15.43:0/3309228820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7e00039b0 0x7fc7e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:03.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.506+0000 7fc7fab92640 1 --2- 172.21.15.43:0/3309228820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7e0004d40 0x7fc7e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:03.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.506+0000 7fc7f9b90640 1 --2- 172.21.15.43:0/3309228820 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7f40ce5e0 0x7fc7e0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:03.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.506+0000 7fc7fa391640 1 -- 172.21.15.43:0/3309228820 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7f40ce5e0 msgr2=0x7fc7e0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:03.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.506+0000 7fc7fa391640 1 --2- 172.21.15.43:0/3309228820 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7f40ce5e0 0x7fc7e0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:03.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.506+0000 7fc7fa391640 1 -- 172.21.15.43:0/3309228820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7e0004d40 msgr2=0x7fc7e00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:03.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.506+0000 7fc7fa391640 1 --2- 172.21.15.43:0/3309228820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7e0004d40 0x7fc7e00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:03.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.506+0000 7fc7fa391640 1 -- 172.21.15.43:0/3309228820 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e0003520 con 0x7fc7e00039b0 2024-04-25T08:58:03.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.506+0000 7fc7fab92640 1 --2- 172.21.15.43:0/3309228820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7e0004d40 0x7fc7e00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:58:03.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.506+0000 7fc7f9b90640 1 --2- 172.21.15.43:0/3309228820 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7f40ce5e0 0x7fc7e0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:58:03.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.507+0000 7fc7fa391640 1 --2- 172.21.15.43:0/3309228820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7e00039b0 0x7fc7e0003dd0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fc7f000aa40 tx=0x7fc7e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae1a3449bc58c1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:03.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.507+0000 7fc7f938f640 1 -- 172.21.15.43:0/3309228820 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f000a690 con 0x7fc7e00039b0 2024-04-25T08:58:03.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.507+0000 7fc7f938f640 1 -- 172.21.15.43:0/3309228820 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7f002f070 con 0x7fc7e00039b0 2024-04-25T08:58:03.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.507+0000 7fc7f938f640 1 -- 172.21.15.43:0/3309228820 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f0034470 con 0x7fc7e00039b0 2024-04-25T08:58:03.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.508+0000 7fc7fc5f5640 1 -- 172.21.15.43:0/3309228820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7e00039b0 msgr2=0x7fc7e0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:03.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.508+0000 7fc7fc5f5640 1 --2- 172.21.15.43:0/3309228820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7e00039b0 0x7fc7e0003dd0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fc7f000aa40 tx=0x7fc7e0005830 comp rx=0 tx=0).stop 2024-04-25T08:58:03.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.508+0000 7fc7fab92640 1 -- 172.21.15.43:0/3309228820 reap_dead start 2024-04-25T08:58:03.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.508+0000 7fc7fc5f5640 1 -- 172.21.15.43:0/3309228820 shutdown_connections 2024-04-25T08:58:03.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.508+0000 7fc7fc5f5640 1 -- 172.21.15.43:0/3309228820 >> 172.21.15.43:0/3309228820 conn(0x7fc7f409cd20 msgr2=0x7fc7f40d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:03.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.508+0000 7fc7fc5f5640 1 -- 172.21.15.43:0/3309228820 shutdown_connections 2024-04-25T08:58:03.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.508+0000 7fc7fc5f5640 1 -- 172.21.15.43:0/3309228820 wait complete. 2024-04-25T08:58:03.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.509+0000 7fc7fc5f5640 1 Processor -- start 2024-04-25T08:58:03.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.509+0000 7fc7fc5f5640 1 -- start start 2024-04-25T08:58:03.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.510+0000 7fc7fc5f5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7e00039b0 0x7fc7f41f47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:03.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.510+0000 7fc7fc5f5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7f40ca340 0x7fc7f40c74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:03.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.510+0000 7fc7fc5f5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7f40c7a10 0x7fc7f4147cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:03.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.510+0000 7fc7fc5f5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f4161770 con 0x7fc7f40ca340 2024-04-25T08:58:03.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.510+0000 7fc7fc5f5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f40d15a0 con 0x7fc7e00039b0 2024-04-25T08:58:03.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.510+0000 7fc7fc5f5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f40cea70 con 0x7fc7f40c7a10 2024-04-25T08:58:03.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.510+0000 7fc7f9b90640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7f40ca340 0x7fc7f40c74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:03.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.510+0000 7fc7f9b90640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7f40ca340 0x7fc7f40c74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44026/0 (socket says 172.21.15.43:44026) 2024-04-25T08:58:03.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.510+0000 7fc7f9b90640 1 -- 172.21.15.43:0/948180758 learned_addr learned my addr 172.21.15.43:0/948180758 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:03.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.510+0000 7fc7fab92640 1 --2- 172.21.15.43:0/948180758 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7f40c7a10 0x7fc7f4147cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:03.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.510+0000 7fc7fa391640 1 --2- 172.21.15.43:0/948180758 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7e00039b0 0x7fc7f41f47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:03.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.510+0000 7fc7f9b90640 1 -- 172.21.15.43:0/948180758 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7f40c7a10 msgr2=0x7fc7f4147cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:03.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.510+0000 7fc7f9b90640 1 --2- 172.21.15.43:0/948180758 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7f40c7a10 0x7fc7f4147cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:03.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.510+0000 7fc7f9b90640 1 -- 172.21.15.43:0/948180758 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7e00039b0 msgr2=0x7fc7f41f47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:03.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.510+0000 7fc7f9b90640 1 --2- 172.21.15.43:0/948180758 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7e00039b0 0x7fc7f41f47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:03.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.510+0000 7fc7f9b90640 1 -- 172.21.15.43:0/948180758 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f40c7eb0 con 0x7fc7f40ca340 2024-04-25T08:58:03.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.510+0000 7fc7f9b90640 1 --2- 172.21.15.43:0/948180758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7f40ca340 0x7fc7f40c74d0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fc7e40116d0 tx=0x7fc7e4011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:03.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.510+0000 7fc7eaffd640 1 -- 172.21.15.43:0/948180758 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e400f960 con 0x7fc7f40ca340 2024-04-25T08:58:03.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.510+0000 7fc7eaffd640 1 -- 172.21.15.43:0/948180758 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e400fb00 con 0x7fc7f40ca340 2024-04-25T08:58:03.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.510+0000 7fc7fc5f5640 1 -- 172.21.15.43:0/948180758 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7f40c8190 con 0x7fc7f40ca340 2024-04-25T08:58:03.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.510+0000 7fc7eaffd640 1 -- 172.21.15.43:0/948180758 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e4015460 con 0x7fc7f40ca340 2024-04-25T08:58:03.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.510+0000 7fc7fc5f5640 1 -- 172.21.15.43:0/948180758 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7f41f42a0 con 0x7fc7f40ca340 2024-04-25T08:58:03.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.511+0000 7fc7fc5f5640 1 -- 172.21.15.43:0/948180758 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7f40680f0 con 0x7fc7f40ca340 2024-04-25T08:58:03.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.512+0000 7fc7eaffd640 1 -- 172.21.15.43:0/948180758 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc7e4015600 con 0x7fc7f40ca340 2024-04-25T08:58:03.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.512+0000 7fc7eaffd640 1 --2- 172.21.15.43:0/948180758 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc7d806b4d0 0x7fc7d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:03.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.512+0000 7fc7eaffd640 1 -- 172.21.15.43:0/948180758 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc7e408ab30 con 0x7fc7f40ca340 2024-04-25T08:58:03.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.512+0000 7fc7fa391640 1 --2- 172.21.15.43:0/948180758 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc7d806b4d0 0x7fc7d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:03.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.513+0000 7fc7fa391640 1 --2- 172.21.15.43:0/948180758 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc7d806b4d0 0x7fc7d806d990 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0033000 tx=0x7fc7f002d000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:03.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.516+0000 7fc7eaffd640 1 -- 172.21.15.43:0/948180758 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc7e4019030 con 0x7fc7f40ca340 2024-04-25T08:58:03.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.731+0000 7fc7fc5f5640 1 -- 172.21.15.43:0/948180758 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7f4074ef0 con 0x7fc7f40ca340 2024-04-25T08:58:03.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.732+0000 7fc7eaffd640 1 -- 172.21.15.43:0/948180758 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4944 (secure 0 0 0) 0x7fc7e40599b0 con 0x7fc7f40ca340 2024-04-25T08:58:03.721 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:58:03.721 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5950,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/2226672650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2226672650},{"type":"v1","addr":"172.21.15.193:6836","nonce":2226672650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5953,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2666301750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2666301750},{"type":"v1","addr":"172.21.15.193:6837","nonce":2666301750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6537,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/3574570511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3574570511},{"type":"v1","addr":"172.21.15.43:6834","nonce":3574570511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:57:54.590369+0000","modified":"2024-04-25T08:57:56.568622+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6540},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6540":{"gid":6540,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/2268368384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2268368384},{"type":"v1","addr":"172.21.15.43:6835","nonce":2268368384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:58:03.722 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 31 2024-04-25T08:58:03.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.734+0000 7fc7fc5f5640 1 -- 172.21.15.43:0/948180758 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc7d806b4d0 msgr2=0x7fc7d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:03.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.734+0000 7fc7fc5f5640 1 --2- 172.21.15.43:0/948180758 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc7d806b4d0 0x7fc7d806d990 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0033000 tx=0x7fc7f002d000 comp rx=0 tx=0).stop 2024-04-25T08:58:03.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.734+0000 7fc7fc5f5640 1 -- 172.21.15.43:0/948180758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7f40ca340 msgr2=0x7fc7f40c74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:03.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.734+0000 7fc7fc5f5640 1 --2- 172.21.15.43:0/948180758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7f40ca340 0x7fc7f40c74d0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fc7e40116d0 tx=0x7fc7e4011b90 comp rx=0 tx=0).stop 2024-04-25T08:58:03.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.734+0000 7fc7fc5f5640 1 -- 172.21.15.43:0/948180758 shutdown_connections 2024-04-25T08:58:03.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.734+0000 7fc7fc5f5640 1 --2- 172.21.15.43:0/948180758 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7f40c7a10 0x7fc7f4147cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:03.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.734+0000 7fc7fc5f5640 1 --2- 172.21.15.43:0/948180758 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7f40ca340 0x7fc7f40c74d0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:03.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.734+0000 7fc7fc5f5640 1 --2- 172.21.15.43:0/948180758 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc7d806b4d0 0x7fc7d806d990 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:03.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.734+0000 7fc7fc5f5640 1 --2- 172.21.15.43:0/948180758 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7e00039b0 0x7fc7f41f47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:03.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.734+0000 7fc7fc5f5640 1 -- 172.21.15.43:0/948180758 >> 172.21.15.43:0/948180758 conn(0x7fc7f409cd20 msgr2=0x7fc7f40b7970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:03.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.734+0000 7fc7fc5f5640 1 -- 172.21.15.43:0/948180758 shutdown_connections 2024-04-25T08:58:03.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:03.734+0000 7fc7fc5f5640 1 -- 172.21.15.43:0/948180758 wait complete. 2024-04-25T08:58:04.137 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 31, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:57:54.590369+0000', 'modified': '2024-04-25T08:57:56.568622+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6540}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6540': {'gid': 6540, 'name': 'c', 'rank': 0, 'incarnation': 29, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.43:6835/2268368384', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6833', 'nonce': 2268368384}, {'type': 'v1', 'addr': '172.21.15.43:6835', 'nonce': 2268368384}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:58:04.137 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:58:04.137 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T08:58:04.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.258+0000 7fb8d6807640 1 Processor -- start 2024-04-25T08:58:04.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.258+0000 7fb8d6807640 1 -- start start 2024-04-25T08:58:04.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.259+0000 7fb8d6807640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb8d015dcf0 0x7fb8d006e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:04.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.259+0000 7fb8d6807640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb8d006d4f0 0x7fb8d006f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:04.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.259+0000 7fb8d6807640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8d00d0d80 con 0x7fb8d006ebf0 2024-04-25T08:58:04.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.259+0000 7fb8d6807640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8d006f590 con 0x7fb8d015dcf0 2024-04-25T08:58:04.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.259+0000 7fb8d6807640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d006c050 con 0x7fb8d006d4f0 2024-04-25T08:58:04.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.259+0000 7fb8cf7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb8d006ebf0 0x7fb8d01616e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35694/0 (socket says 172.21.15.43:35694) 2024-04-25T08:58:04.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.259+0000 7fb8cf7fe640 1 -- 172.21.15.43:0/868028760 learned_addr learned my addr 172.21.15.43:0/868028760 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:04.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.259+0000 7fb8cffff640 1 --2- 172.21.15.43:0/868028760 >> v2:172.21.15.193:3300/0 conn(0x7fb8d015dcf0 0x7fb8d006e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:04.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.259+0000 7fb8ceffd640 1 -- 172.21.15.43:0/868028760 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3970795279 0 0) 0x7fb8d00d0d80 con 0x7fb8d006ebf0 2024-04-25T08:58:04.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.259+0000 7fb8d4da4640 1 --2- 172.21.15.43:0/868028760 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb8d006d4f0 0x7fb8d006f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:04.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.259+0000 7fb8ceffd640 1 -- 172.21.15.43:0/868028760 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8b8003520 con 0x7fb8d006ebf0 2024-04-25T08:58:04.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.259+0000 7fb8ceffd640 1 -- 172.21.15.43:0/868028760 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 189436602 0 0) 0x7fb8b8003520 con 0x7fb8d006ebf0 2024-04-25T08:58:04.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.259+0000 7fb8ceffd640 1 -- 172.21.15.43:0/868028760 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb8d006d4f0 msgr2=0x7fb8d006f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:04.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.259+0000 7fb8ceffd640 1 --2- 172.21.15.43:0/868028760 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb8d006d4f0 0x7fb8d006f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.259+0000 7fb8ceffd640 1 -- 172.21.15.43:0/868028760 >> v2:172.21.15.193:3300/0 conn(0x7fb8d015dcf0 msgr2=0x7fb8d006e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:04.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.259+0000 7fb8ceffd640 1 --2- 172.21.15.43:0/868028760 >> v2:172.21.15.193:3300/0 conn(0x7fb8d015dcf0 0x7fb8d006e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.259+0000 7fb8ceffd640 1 -- 172.21.15.43:0/868028760 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d00d04e0 con 0x7fb8d006ebf0 2024-04-25T08:58:04.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.259+0000 7fb8ceffd640 1 -- 172.21.15.43:0/868028760 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7fb8c0001670 con 0x7fb8d006ebf0 2024-04-25T08:58:04.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.259+0000 7fb8ceffd640 1 -- 172.21.15.43:0/868028760 >> v1:172.21.15.43:6789/0 conn(0x7fb8d006ebf0 legacy=0x7fb8d01616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:04.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.260+0000 7fb8ceffd640 1 --2- 172.21.15.43:0/868028760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8b80039b0 0x7fb8b8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:04.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.260+0000 7fb8ceffd640 1 --2- 172.21.15.43:0/868028760 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8b8004800 0x7fb8b8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:04.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.260+0000 7fb8ceffd640 1 --2- 172.21.15.43:0/868028760 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb8d006d4f0 0x7fb8b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:04.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.260+0000 7fb8ceffd640 1 -- 172.21.15.43:0/868028760 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d006c050 con 0x7fb8b80039b0 2024-04-25T08:58:04.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.260+0000 7fb8ceffd640 1 -- 172.21.15.43:0/868028760 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b800b9c0 con 0x7fb8b8004800 2024-04-25T08:58:04.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.260+0000 7fb8ceffd640 1 -- 172.21.15.43:0/868028760 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b800bb70 con 0x7fb8d006d4f0 2024-04-25T08:58:04.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.260+0000 7fb8cffff640 1 --2- 172.21.15.43:0/868028760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8b80039b0 0x7fb8b8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:04.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.260+0000 7fb8cf7fe640 1 --2- 172.21.15.43:0/868028760 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8b8004800 0x7fb8b8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:04.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.260+0000 7fb8cffff640 1 -- 172.21.15.43:0/868028760 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb8d006d4f0 msgr2=0x7fb8b80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:04.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.260+0000 7fb8d4da4640 1 --2- 172.21.15.43:0/868028760 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb8d006d4f0 0x7fb8b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:04.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.260+0000 7fb8cffff640 1 --2- 172.21.15.43:0/868028760 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb8d006d4f0 0x7fb8b80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.260+0000 7fb8cffff640 1 -- 172.21.15.43:0/868028760 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8b8004800 msgr2=0x7fb8b8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:04.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.260+0000 7fb8cffff640 1 --2- 172.21.15.43:0/868028760 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8b8004800 0x7fb8b8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.260+0000 7fb8cffff640 1 -- 172.21.15.43:0/868028760 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b8003520 con 0x7fb8b80039b0 2024-04-25T08:58:04.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.260+0000 7fb8d4da4640 1 --2- 172.21.15.43:0/868028760 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb8d006d4f0 0x7fb8b80051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:04.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.261+0000 7fb8cffff640 1 --2- 172.21.15.43:0/868028760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8b80039b0 0x7fb8b8003dd0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4009ac0 tx=0x7fb8b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36cd50e667c011a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:04.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.261+0000 7fb8ceffd640 1 -- 172.21.15.43:0/868028760 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4036070 con 0x7fb8b80039b0 2024-04-25T08:58:04.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.261+0000 7fb8ceffd640 1 -- 172.21.15.43:0/868028760 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c4031400 con 0x7fb8b80039b0 2024-04-25T08:58:04.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.261+0000 7fb8ceffd640 1 -- 172.21.15.43:0/868028760 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c4009690 con 0x7fb8b80039b0 2024-04-25T08:58:04.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.261+0000 7fb8d6807640 1 -- 172.21.15.43:0/868028760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8b80039b0 msgr2=0x7fb8b8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:04.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.261+0000 7fb8d6807640 1 --2- 172.21.15.43:0/868028760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8b80039b0 0x7fb8b8003dd0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4009ac0 tx=0x7fb8b8005830 comp rx=0 tx=0).stop 2024-04-25T08:58:04.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.261+0000 7fb8d4da4640 1 -- 172.21.15.43:0/868028760 reap_dead start 2024-04-25T08:58:04.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.261+0000 7fb8d6807640 1 -- 172.21.15.43:0/868028760 shutdown_connections 2024-04-25T08:58:04.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.261+0000 7fb8d6807640 1 -- 172.21.15.43:0/868028760 >> 172.21.15.43:0/868028760 conn(0x7fb8d00d1700 msgr2=0x7fb8d00d1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:04.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.261+0000 7fb8d6807640 1 -- 172.21.15.43:0/868028760 shutdown_connections 2024-04-25T08:58:04.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.261+0000 7fb8d6807640 1 -- 172.21.15.43:0/868028760 wait complete. 2024-04-25T08:58:04.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.263+0000 7fb8d6807640 1 Processor -- start 2024-04-25T08:58:04.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.263+0000 7fb8d6807640 1 -- start start 2024-04-25T08:58:04.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.263+0000 7fb8d6807640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8b80039b0 0x7fb8d009b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:04.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.264+0000 7fb8d6807640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb8d01e6aa0 0x7fb8d015c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:04.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.264+0000 7fb8d6807640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8d01e9010 0x7fb8d01e9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:04.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.264+0000 7fb8d6807640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d006bce0 con 0x7fb8d01e9010 2024-04-25T08:58:04.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.264+0000 7fb8d6807640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d00d0aa0 con 0x7fb8b80039b0 2024-04-25T08:58:04.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.264+0000 7fb8d6807640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d00b0d50 con 0x7fb8d01e6aa0 2024-04-25T08:58:04.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.264+0000 7fb8d4da4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8d01e9010 0x7fb8d01e9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:04.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.264+0000 7fb8d4da4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8d01e9010 0x7fb8d01e9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44044/0 (socket says 172.21.15.43:44044) 2024-04-25T08:58:04.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.264+0000 7fb8d4da4640 1 -- 172.21.15.43:0/3152510929 learned_addr learned my addr 172.21.15.43:0/3152510929 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:04.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.264+0000 7fb8cffff640 1 --2- 172.21.15.43:0/3152510929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8b80039b0 0x7fb8d009b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:04.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.264+0000 7fb8cf7fe640 1 --2- 172.21.15.43:0/3152510929 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb8d01e6aa0 0x7fb8d015c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:04.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.264+0000 7fb8d4da4640 1 -- 172.21.15.43:0/3152510929 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb8d01e6aa0 msgr2=0x7fb8d015c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:04.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.264+0000 7fb8d4da4640 1 --2- 172.21.15.43:0/3152510929 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb8d01e6aa0 0x7fb8d015c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.264+0000 7fb8d4da4640 1 -- 172.21.15.43:0/3152510929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8b80039b0 msgr2=0x7fb8d009b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:04.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.264+0000 7fb8d4da4640 1 --2- 172.21.15.43:0/3152510929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8b80039b0 0x7fb8d009b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.264+0000 7fb8d4da4640 1 -- 172.21.15.43:0/3152510929 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d01f4000 con 0x7fb8d01e9010 2024-04-25T08:58:04.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.264+0000 7fb8d4da4640 1 --2- 172.21.15.43:0/3152510929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8d01e9010 0x7fb8d01e9490 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fb8d006d980 tx=0x7fb8c800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:04.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.264+0000 7fb8ccff9640 1 -- 172.21.15.43:0/3152510929 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c801c070 con 0x7fb8d01e9010 2024-04-25T08:58:04.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.264+0000 7fb8ccff9640 1 -- 172.21.15.43:0/3152510929 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c8017400 con 0x7fb8d01e9010 2024-04-25T08:58:04.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.264+0000 7fb8ccff9640 1 -- 172.21.15.43:0/3152510929 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c80177f0 con 0x7fb8d01e9010 2024-04-25T08:58:04.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.264+0000 7fb8d6807640 1 -- 172.21.15.43:0/3152510929 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8d01f42c0 con 0x7fb8d01e9010 2024-04-25T08:58:04.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.264+0000 7fb8d6807640 1 -- 172.21.15.43:0/3152510929 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8d01f4880 con 0x7fb8d01e9010 2024-04-25T08:58:04.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.266+0000 7fb8ccff9640 1 -- 172.21.15.43:0/3152510929 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb8c8005d10 con 0x7fb8d01e9010 2024-04-25T08:58:04.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.266+0000 7fb8ccff9640 1 --2- 172.21.15.43:0/3152510929 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb8bc06b500 0x7fb8bc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:04.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.266+0000 7fb8ccff9640 1 -- 172.21.15.43:0/3152510929 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb8c808a3a0 con 0x7fb8d01e9010 2024-04-25T08:58:04.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.267+0000 7fb8cffff640 1 --2- 172.21.15.43:0/3152510929 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb8bc06b500 0x7fb8bc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:04.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.267+0000 7fb8cffff640 1 --2- 172.21.15.43:0/3152510929 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb8bc06b500 0x7fb8bc06d9c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4033000 tx=0x7fb8c4008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:04.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.267+0000 7fb8d95e9740 1 Processor -- start 2024-04-25T08:58:04.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.267+0000 7fb8d95e9740 1 -- start start 2024-04-25T08:58:04.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.268+0000 7fb8d95e9740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x562e00d26230 0x562e00d26610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:04.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.268+0000 7fb8d95e9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e00d43ad0 0x562e00d44020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:04.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.268+0000 7fb8d95e9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562e00cfbf10 0x562e00d42190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:04.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.268+0000 7fb8d95e9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e00c5ea10 con 0x562e00cfbf10 2024-04-25T08:58:04.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.268+0000 7fb8d95e9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e00c07b30 con 0x562e00d43ad0 2024-04-25T08:58:04.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.268+0000 7fb8d95e9740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e00c5ac50 con 0x562e00d26230 2024-04-25T08:58:04.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.268+0000 7fb8d4da4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562e00cfbf10 0x562e00d42190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:04.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.268+0000 7fb8d4da4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562e00cfbf10 0x562e00d42190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44054/0 (socket says 172.21.15.43:44054) 2024-04-25T08:58:04.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.268+0000 7fb8d4da4640 1 -- 172.21.15.43:0/2222532398 learned_addr learned my addr 172.21.15.43:0/2222532398 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:04.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.268+0000 7fb8cf7fe640 1 --2- 172.21.15.43:0/2222532398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e00d43ad0 0x562e00d44020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:04.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.268+0000 7fb8cffff640 1 --2- 172.21.15.43:0/2222532398 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x562e00d26230 0x562e00d26610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:04.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.268+0000 7fb8d4da4640 1 -- 172.21.15.43:0/2222532398 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x562e00d26230 msgr2=0x562e00d26610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:04.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.268+0000 7fb8d4da4640 1 --2- 172.21.15.43:0/2222532398 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x562e00d26230 0x562e00d26610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.268+0000 7fb8d4da4640 1 -- 172.21.15.43:0/2222532398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e00d43ad0 msgr2=0x562e00d44020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:04.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.268+0000 7fb8d4da4640 1 --2- 172.21.15.43:0/2222532398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e00d43ad0 0x562e00d44020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.268+0000 7fb8d4da4640 1 -- 172.21.15.43:0/2222532398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e00c1ce10 con 0x562e00cfbf10 2024-04-25T08:58:04.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.268+0000 7fb8d4da4640 1 --2- 172.21.15.43:0/2222532398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562e00cfbf10 0x562e00d42190 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8058830 tx=0x7fb8c8058a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a84c7fbfcb915e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:04.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.269+0000 7fb8b5ffb640 1 -- 172.21.15.43:0/2222532398 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c808a0c0 con 0x562e00cfbf10 2024-04-25T08:58:04.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.269+0000 7fb8b5ffb640 1 -- 172.21.15.43:0/2222532398 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c80551a0 con 0x562e00cfbf10 2024-04-25T08:58:04.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.269+0000 7fb8b5ffb640 1 -- 172.21.15.43:0/2222532398 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c8058fb0 con 0x562e00cfbf10 2024-04-25T08:58:04.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.269+0000 7fb8d95e9740 1 -- 172.21.15.43:0/2222532398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562e00cfbf10 msgr2=0x562e00d42190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:04.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.269+0000 7fb8d95e9740 1 --2- 172.21.15.43:0/2222532398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562e00cfbf10 0x562e00d42190 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8058830 tx=0x7fb8c8058a30 comp rx=0 tx=0).stop 2024-04-25T08:58:04.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.269+0000 7fb8d95e9740 1 -- 172.21.15.43:0/2222532398 shutdown_connections 2024-04-25T08:58:04.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.269+0000 7fb8d95e9740 1 --2- 172.21.15.43:0/2222532398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562e00cfbf10 0x562e00d42190 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.269+0000 7fb8d95e9740 1 --2- 172.21.15.43:0/2222532398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e00d43ad0 0x562e00d44020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.269+0000 7fb8d95e9740 1 --2- 172.21.15.43:0/2222532398 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x562e00d26230 0x562e00d26610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.269+0000 7fb8d95e9740 1 -- 172.21.15.43:0/2222532398 >> 172.21.15.43:0/2222532398 conn(0x562e00cf8840 msgr2=0x562e00d41040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:04.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.269+0000 7fb8d95e9740 1 -- 172.21.15.43:0/2222532398 shutdown_connections 2024-04-25T08:58:04.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.269+0000 7fb8d95e9740 1 -- 172.21.15.43:0/2222532398 wait complete. 2024-04-25T08:58:04.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.270+0000 7fb8d95e9740 1 -- start start 2024-04-25T08:58:04.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.270+0000 7fb8d95e9740 1 Processor -- start 2024-04-25T08:58:04.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.271+0000 7fb8d95e9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562e00d26230 0x562e00d26ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:04.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.271+0000 7fb8d95e9740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x562e00d43ad0 0x562e00d54ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:04.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.271+0000 7fb8d95e9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e00d55400 0x562e00d143d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:04.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.271+0000 7fb8d95e9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e00cfccb0 con 0x562e00d26230 2024-04-25T08:58:04.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.271+0000 7fb8d95e9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e00c1b0c0 con 0x562e00d55400 2024-04-25T08:58:04.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.271+0000 7fb8d95e9740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e00cf6770 con 0x562e00d43ad0 2024-04-25T08:58:04.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.271+0000 7fb8d95e9740 10 client.0 _ll_register_callbacks cb 0x562e00c0eec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:58:04.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.271+0000 7fb8cffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562e00d26230 0x562e00d26ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:04.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.271+0000 7fb8cffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562e00d26230 0x562e00d26ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44060/0 (socket says 172.21.15.43:44060) 2024-04-25T08:58:04.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.271+0000 7fb8cffff640 1 -- 172.21.15.43:0/3317648240 learned_addr learned my addr 172.21.15.43:0/3317648240 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:04.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.271+0000 7fb8d4da4640 1 --2- 172.21.15.43:0/3317648240 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e00d55400 0x562e00d143d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:04.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.271+0000 7fb8cf7fe640 1 --2- 172.21.15.43:0/3317648240 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x562e00d43ad0 0x562e00d54ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:04.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.271+0000 7fb8cffff640 1 -- 172.21.15.43:0/3317648240 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x562e00d43ad0 msgr2=0x562e00d54ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:04.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.271+0000 7fb8cffff640 1 --2- 172.21.15.43:0/3317648240 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x562e00d43ad0 0x562e00d54ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.271+0000 7fb8cffff640 1 -- 172.21.15.43:0/3317648240 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e00d55400 msgr2=0x562e00d143d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:04.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.271+0000 7fb8cffff640 1 --2- 172.21.15.43:0/3317648240 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e00d55400 0x562e00d143d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.271+0000 7fb8cffff640 1 -- 172.21.15.43:0/3317648240 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562e009732d0 con 0x562e00d26230 2024-04-25T08:58:04.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.271+0000 7fb8cffff640 1 --2- 172.21.15.43:0/3317648240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562e00d26230 0x562e00d26ef0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fb8c402abc0 tx=0x7fb8c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47067cc459b9ac1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:04.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.271+0000 7fb8aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:58:04.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.272+0000 7fb8aeffd640 1 -- 172.21.15.43:0/3317648240 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c404c070 con 0x562e00d26230 2024-04-25T08:58:04.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.272+0000 7fb8aeffd640 1 -- 172.21.15.43:0/3317648240 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb8c4003a10 con 0x562e00d26230 2024-04-25T08:58:04.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.272+0000 7fb8d95e9740 1 -- 172.21.15.43:0/3317648240 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb8c80551a0 con 0x562e00d26230 2024-04-25T08:58:04.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.272+0000 7fb8aeffd640 1 -- 172.21.15.43:0/3317648240 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8c4057050 con 0x562e00d26230 2024-04-25T08:58:04.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.272+0000 7fb8d95e9740 10 client.6717 fetch_fsmap learned FSMap version 31 2024-04-25T08:58:04.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.272+0000 7fb8d95e9740 1 -- 172.21.15.43:0/3317648240 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x562e00a3d6b0 con 0x562e00d26230 2024-04-25T08:58:04.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.272+0000 7fb8aeffd640 1 -- 172.21.15.43:0/3317648240 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c80551a0 con 0x562e00d26230 2024-04-25T08:58:04.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.272+0000 7fb8aeffd640 1 -- 172.21.15.43:0/3317648240 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 31) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb8c405a020 con 0x562e00d26230 2024-04-25T08:58:04.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.272+0000 7fb8d95e9740 10 client.6717 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T08:58:04.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.272+0000 7fb8d95e9740 10 client.6717 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:58:04.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.272+0000 7fb8d95e9740 20 client.6717 populate_metadata read hostname 'smithi043' 2024-04-25T08:58:04.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.272+0000 7fb8d95e9740 1 --2- 172.21.15.43:0/3317648240 >> [v2:172.21.15.43:6833/2268368384,v1:172.21.15.43:6835/2268368384] conn(0x562e00d3ad10 0x562e00d5f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:04.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.272+0000 7fb8d95e9740 4 client.6717 mds_command: new command op to 6540 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:58:04.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.272+0000 7fb8d95e9740 1 -- 172.21.15.43:0/3317648240 --> [v2:172.21.15.43:6833/2268368384,v1:172.21.15.43:6835/2268368384] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e00cbd920 con 0x562e00d3ad10 2024-04-25T08:58:04.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.272+0000 7fb8cf7fe640 1 --2- 172.21.15.43:0/3317648240 >> [v2:172.21.15.43:6833/2268368384,v1:172.21.15.43:6835/2268368384] conn(0x562e00d3ad10 0x562e00d5f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:04.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.273+0000 7fb8aeffd640 10 client.6717 ms_handle_connect on v2:172.21.15.43:6833/2268368384 2024-04-25T08:58:04.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.273+0000 7fb8cf7fe640 1 --2- 172.21.15.43:0/3317648240 >> [v2:172.21.15.43:6833/2268368384,v1:172.21.15.43:6835/2268368384] conn(0x562e00d3ad10 0x562e00d5f9d0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e92e620ca3987e6f server_cookie=475e919a05db419 in_seq=0 out_seq=0 2024-04-25T08:58:04.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.273+0000 7fb8aeffd640 1 -- 172.21.15.43:0/3317648240 <== mds.0 v2:172.21.15.43:6833/2268368384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562e00cbd920 con 0x562e00d3ad10 2024-04-25T08:58:04.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.273+0000 7fb8aeffd640 10 client.6717 handle_command_reply: tid=0 2024-04-25T08:58:04.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.273+0000 7fb8d95e9740 1 client.6717 shutdown 2024-04-25T08:58:04.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.273+0000 7fb8d95e9740 1 -- 172.21.15.43:0/3317648240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562e00d26230 msgr2=0x562e00d26ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:04.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.273+0000 7fb8d95e9740 1 --2- 172.21.15.43:0/3317648240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562e00d26230 0x562e00d26ef0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fb8c402abc0 tx=0x7fb8c40046c0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.273+0000 7fb8d95e9740 1 -- 172.21.15.43:0/3317648240 shutdown_connections 2024-04-25T08:58:04.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.273+0000 7fb8d95e9740 1 --2- 172.21.15.43:0/3317648240 >> [v2:172.21.15.43:6833/2268368384,v1:172.21.15.43:6835/2268368384] conn(0x562e00d3ad10 0x562e00d5f9d0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.273+0000 7fb8d95e9740 1 --2- 172.21.15.43:0/3317648240 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e00d55400 0x562e00d143d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.273+0000 7fb8aeffd640 0 client.6717 ms_handle_reset on v2:172.21.15.43:6833/2268368384 2024-04-25T08:58:04.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.273+0000 7fb8d95e9740 1 --2- 172.21.15.43:0/3317648240 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x562e00d43ad0 0x562e00d54ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.273+0000 7fb8d95e9740 1 --2- 172.21.15.43:0/3317648240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562e00d26230 0x562e00d26ef0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.273+0000 7fb8d95e9740 1 -- 172.21.15.43:0/3317648240 >> 172.21.15.43:0/3317648240 conn(0x562e00cfbf10 msgr2=0x562e00d44e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:04.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.273+0000 7fb8d95e9740 1 -- 172.21.15.43:0/3317648240 shutdown_connections 2024-04-25T08:58:04.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.274+0000 7fb8d95e9740 1 -- 172.21.15.43:0/3317648240 wait complete. 2024-04-25T08:58:04.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.274+0000 7fb8d95e9740 20 client.6717 trim_cache size 0 max 16384 2024-04-25T08:58:04.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.290+0000 7fb8d95e9740 1 Processor -- start 2024-04-25T08:58:04.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.290+0000 7fb8d95e9740 1 -- start start 2024-04-25T08:58:04.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.290+0000 7fb8d95e9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e00d26230 0x562e00d51960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:04.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.290+0000 7fb8d95e9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562e00d43ad0 0x562e00da6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:04.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.290+0000 7fb8d95e9740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x562e00d55400 0x562e00cf54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:04.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.290+0000 7fb8d95e9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d009f3d0 con 0x562e00d43ad0 2024-04-25T08:58:04.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.290+0000 7fb8d95e9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d01f8160 con 0x562e00d26230 2024-04-25T08:58:04.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.290+0000 7fb8d95e9740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d009edd0 con 0x562e00d55400 2024-04-25T08:58:04.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.290+0000 7fb8cf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562e00d43ad0 0x562e00da6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:04.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.290+0000 7fb8cf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562e00d43ad0 0x562e00da6a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44068/0 (socket says 172.21.15.43:44068) 2024-04-25T08:58:04.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.290+0000 7fb8cf7fe640 1 -- 172.21.15.43:0/3867134252 learned_addr learned my addr 172.21.15.43:0/3867134252 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:04.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.291+0000 7fb8cf7fe640 1 -- 172.21.15.43:0/3867134252 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x562e00d55400 msgr2=0x562e00cf54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:04.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.291+0000 7fb8cf7fe640 1 --2- 172.21.15.43:0/3867134252 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x562e00d55400 0x562e00cf54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.291+0000 7fb8cf7fe640 1 -- 172.21.15.43:0/3867134252 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e00d26230 msgr2=0x562e00d51960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:04.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.291+0000 7fb8cf7fe640 1 --2- 172.21.15.43:0/3867134252 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e00d26230 0x562e00d51960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.291+0000 7fb8cf7fe640 1 -- 172.21.15.43:0/3867134252 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e00d647c0 con 0x562e00d43ad0 2024-04-25T08:58:04.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.291+0000 7fb8cf7fe640 1 --2- 172.21.15.43:0/3867134252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562e00d43ad0 0x562e00da6a00 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0007f10 tx=0x7fb8c00078f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a33a943b28cbbead server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:04.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.291+0000 7fb8b57fa640 1 -- 172.21.15.43:0/3867134252 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c001f330 con 0x562e00d43ad0 2024-04-25T08:58:04.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.291+0000 7fb8b57fa640 1 -- 172.21.15.43:0/3867134252 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c001f4d0 con 0x562e00d43ad0 2024-04-25T08:58:04.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.291+0000 7fb8b57fa640 1 -- 172.21.15.43:0/3867134252 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c001f8c0 con 0x562e00d43ad0 2024-04-25T08:58:04.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.291+0000 7fb8d95e9740 1 -- 172.21.15.43:0/3867134252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562e00d43ad0 msgr2=0x562e00da6a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:04.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.291+0000 7fb8d95e9740 1 --2- 172.21.15.43:0/3867134252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562e00d43ad0 0x562e00da6a00 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0007f10 tx=0x7fb8c00078f0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.291+0000 7fb8d95e9740 1 -- 172.21.15.43:0/3867134252 shutdown_connections 2024-04-25T08:58:04.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.291+0000 7fb8d95e9740 1 --2- 172.21.15.43:0/3867134252 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x562e00d55400 0x562e00cf54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.291+0000 7fb8d95e9740 1 --2- 172.21.15.43:0/3867134252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562e00d43ad0 0x562e00da6a00 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.292+0000 7fb8d95e9740 1 --2- 172.21.15.43:0/3867134252 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e00d26230 0x562e00d51960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.292+0000 7fb8d95e9740 1 -- 172.21.15.43:0/3867134252 >> 172.21.15.43:0/3867134252 conn(0x562e00cfbf10 msgr2=0x562e00d11c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:04.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.292+0000 7fb8d95e9740 1 -- 172.21.15.43:0/3867134252 shutdown_connections 2024-04-25T08:58:04.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.292+0000 7fb8d95e9740 1 -- 172.21.15.43:0/3867134252 wait complete. 2024-04-25T08:58:04.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.292+0000 7fb8d95e9740 1 -- start start 2024-04-25T08:58:04.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.292+0000 7fb8d95e9740 1 Processor -- start 2024-04-25T08:58:04.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.293+0000 7fb8d95e9740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x562e00d26230 0x562e00d92260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:04.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.293+0000 7fb8d95e9740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562e00d43ad0 0x562e00d61010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:04.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.293+0000 7fb8d95e9740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e00d55400 0x562e00d61550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:04.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.293+0000 7fb8d95e9740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e00d14ed0 con 0x562e00d43ad0 2024-04-25T08:58:04.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.293+0000 7fb8d95e9740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e00d283e0 con 0x562e00d55400 2024-04-25T08:58:04.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.293+0000 7fb8d95e9740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x562e00d3a160 con 0x562e00d26230 2024-04-25T08:58:04.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.293+0000 7fb8d95e9740 10 client.0 _ll_register_callbacks cb 0x562e00cf6c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:58:04.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.293+0000 7fb8cf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562e00d43ad0 0x562e00d61010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:04.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.293+0000 7fb8cf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562e00d43ad0 0x562e00d61010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44070/0 (socket says 172.21.15.43:44070) 2024-04-25T08:58:04.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.293+0000 7fb8cf7fe640 1 -- 172.21.15.43:0/782164361 learned_addr learned my addr 172.21.15.43:0/782164361 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:04.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.293+0000 7fb8d4da4640 1 --2- 172.21.15.43:0/782164361 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e00d55400 0x562e00d61550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:04.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.293+0000 7fb8cffff640 1 --2- 172.21.15.43:0/782164361 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x562e00d26230 0x562e00d92260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:04.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.293+0000 7fb8cf7fe640 1 -- 172.21.15.43:0/782164361 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x562e00d26230 msgr2=0x562e00d92260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:04.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.293+0000 7fb8cf7fe640 1 --2- 172.21.15.43:0/782164361 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x562e00d26230 0x562e00d92260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.293+0000 7fb8cf7fe640 1 -- 172.21.15.43:0/782164361 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e00d55400 msgr2=0x562e00d61550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:04.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.293+0000 7fb8cf7fe640 1 --2- 172.21.15.43:0/782164361 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e00d55400 0x562e00d61550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.294+0000 7fb8cf7fe640 1 -- 172.21.15.43:0/782164361 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562e00d60830 con 0x562e00d43ad0 2024-04-25T08:58:04.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.294+0000 7fb8cf7fe640 1 --2- 172.21.15.43:0/782164361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562e00d43ad0 0x562e00d61010 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0009000 tx=0x7fb8d006bfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b87fbca2b276ba8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:04.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.294+0000 7fb8aeffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:58:04.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.294+0000 7fb8aeffd640 1 -- 172.21.15.43:0/782164361 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c001f330 con 0x562e00d43ad0 2024-04-25T08:58:04.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.294+0000 7fb8aeffd640 1 -- 172.21.15.43:0/782164361 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb8c00074d0 con 0x562e00d43ad0 2024-04-25T08:58:04.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.294+0000 7fb8d95e9740 1 -- 172.21.15.43:0/782164361 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb8c001f4d0 con 0x562e00d43ad0 2024-04-25T08:58:04.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.294+0000 7fb8aeffd640 1 -- 172.21.15.43:0/782164361 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8c0038050 con 0x562e00d43ad0 2024-04-25T08:58:04.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.294+0000 7fb8d95e9740 10 client.6723 fetch_fsmap learned FSMap version 31 2024-04-25T08:58:04.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.294+0000 7fb8d95e9740 1 -- 172.21.15.43:0/782164361 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x562e00d42af0 con 0x562e00d43ad0 2024-04-25T08:58:04.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.294+0000 7fb8aeffd640 1 -- 172.21.15.43:0/782164361 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c001f4d0 con 0x562e00d43ad0 2024-04-25T08:58:04.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.294+0000 7fb8aeffd640 1 -- 172.21.15.43:0/782164361 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 31) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb8c003b020 con 0x562e00d43ad0 2024-04-25T08:58:04.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.294+0000 7fb8d95e9740 10 client.6723 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T08:58:04.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.294+0000 7fb8d95e9740 10 client.6723 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:58:04.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.294+0000 7fb8d95e9740 20 client.6723 populate_metadata read hostname 'smithi043' 2024-04-25T08:58:04.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.294+0000 7fb8d95e9740 1 --2- 172.21.15.43:0/782164361 >> [v2:172.21.15.43:6833/2268368384,v1:172.21.15.43:6835/2268368384] conn(0x562e00d5bc10 0x562e00d4f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:04.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.294+0000 7fb8d95e9740 4 client.6723 mds_command: new command op to 6540 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:58:04.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.294+0000 7fb8d95e9740 1 -- 172.21.15.43:0/782164361 --> [v2:172.21.15.43:6833/2268368384,v1:172.21.15.43:6835/2268368384] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562e00c6bf50 con 0x562e00d5bc10 2024-04-25T08:58:04.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.294+0000 7fb8cffff640 1 --2- 172.21.15.43:0/782164361 >> [v2:172.21.15.43:6833/2268368384,v1:172.21.15.43:6835/2268368384] conn(0x562e00d5bc10 0x562e00d4f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:04.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.295+0000 7fb8cffff640 1 --2- 172.21.15.43:0/782164361 >> [v2:172.21.15.43:6833/2268368384,v1:172.21.15.43:6835/2268368384] conn(0x562e00d5bc10 0x562e00d4f760 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37899498f4cffb7e server_cookie=a584e471d952d447 in_seq=0 out_seq=0 2024-04-25T08:58:04.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.295+0000 7fb8aeffd640 10 client.6723 ms_handle_connect on v2:172.21.15.43:6833/2268368384 2024-04-25T08:58:04.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.295+0000 7fb8aeffd640 1 -- 172.21.15.43:0/782164361 <== mds.0 v2:172.21.15.43:6833/2268368384 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x562e00c6bf50 con 0x562e00d5bc10 2024-04-25T08:58:04.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.295+0000 7fb8aeffd640 10 client.6723 handle_command_reply: tid=0 2024-04-25T08:58:04.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.295+0000 7fb8d95e9740 1 client.6723 shutdown 2024-04-25T08:58:04.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.295+0000 7fb8d95e9740 1 -- 172.21.15.43:0/782164361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562e00d43ad0 msgr2=0x562e00d61010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:04.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.295+0000 7fb8d95e9740 1 --2- 172.21.15.43:0/782164361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562e00d43ad0 0x562e00d61010 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0009000 tx=0x7fb8d006bfc0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.295+0000 7fb8d95e9740 1 -- 172.21.15.43:0/782164361 shutdown_connections 2024-04-25T08:58:04.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.295+0000 7fb8d95e9740 1 --2- 172.21.15.43:0/782164361 >> [v2:172.21.15.43:6833/2268368384,v1:172.21.15.43:6835/2268368384] conn(0x562e00d5bc10 0x562e00d4f760 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.295+0000 7fb8d95e9740 1 --2- 172.21.15.43:0/782164361 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562e00d55400 0x562e00d61550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.295+0000 7fb8d95e9740 1 --2- 172.21.15.43:0/782164361 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562e00d43ad0 0x562e00d61010 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.295+0000 7fb8aeffd640 0 client.6723 ms_handle_reset on v2:172.21.15.43:6833/2268368384 2024-04-25T08:58:04.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.295+0000 7fb8d95e9740 1 --2- 172.21.15.43:0/782164361 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x562e00d26230 0x562e00d92260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.295+0000 7fb8d95e9740 1 -- 172.21.15.43:0/782164361 >> 172.21.15.43:0/782164361 conn(0x562e00cfbf10 msgr2=0x562e00cfc310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:04.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.295+0000 7fb8d95e9740 1 -- 172.21.15.43:0/782164361 shutdown_connections 2024-04-25T08:58:04.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.295+0000 7fb8d95e9740 1 -- 172.21.15.43:0/782164361 wait complete. 2024-04-25T08:58:04.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.295+0000 7fb8d95e9740 20 client.6723 trim_cache size 0 max 16384 2024-04-25T08:58:04.291 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T08:58:04.292 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "6c4bd50e-aa00-4a7f-8d09-352344ef5a4b", 2024-04-25T08:58:04.292 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-04-25T08:58:04.292 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 6540, 2024-04-25T08:58:04.292 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-04-25T08:58:04.292 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-04-25T08:58:04.292 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-04-25T08:58:04.292 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 9.0788533999999999, 2024-04-25T08:58:04.292 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 31, 2024-04-25T08:58:04.292 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 58, 2024-04-25T08:58:04.292 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 58, 2024-04-25T08:58:04.292 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 14.414398378 2024-04-25T08:58:04.292 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T08:58:04.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.296+0000 7fb8d6807640 1 -- 172.21.15.43:0/3152510929 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb8bc06b500 msgr2=0x7fb8bc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:04.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.296+0000 7fb8d6807640 1 --2- 172.21.15.43:0/3152510929 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb8bc06b500 0x7fb8bc06d9c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4033000 tx=0x7fb8c4008030 comp rx=0 tx=0).stop 2024-04-25T08:58:04.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.296+0000 7fb8d6807640 1 -- 172.21.15.43:0/3152510929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8d01e9010 msgr2=0x7fb8d01e9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:04.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.296+0000 7fb8d6807640 1 --2- 172.21.15.43:0/3152510929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8d01e9010 0x7fb8d01e9490 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fb8d006d980 tx=0x7fb8c800f4a0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.296+0000 7fb8d6807640 1 -- 172.21.15.43:0/3152510929 shutdown_connections 2024-04-25T08:58:04.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.296+0000 7fb8d6807640 1 --2- 172.21.15.43:0/3152510929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8d01e9010 0x7fb8d01e9490 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.296+0000 7fb8d6807640 1 --2- 172.21.15.43:0/3152510929 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb8d01e6aa0 0x7fb8d015c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.296+0000 7fb8d6807640 1 --2- 172.21.15.43:0/3152510929 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb8bc06b500 0x7fb8bc06d9c0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.296+0000 7fb8d6807640 1 --2- 172.21.15.43:0/3152510929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8b80039b0 0x7fb8d009b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.296+0000 7fb8d6807640 1 -- 172.21.15.43:0/3152510929 >> 172.21.15.43:0/3152510929 conn(0x7fb8d00d1700 msgr2=0x7fb8d00aebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:04.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.296+0000 7fb8d6807640 1 -- 172.21.15.43:0/3152510929 shutdown_connections 2024-04-25T08:58:04.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.296+0000 7fb8d6807640 1 -- 172.21.15.43:0/3152510929 wait complete. 2024-04-25T08:58:04.297 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:58:04.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.409+0000 7fe843c88640 1 Processor -- start 2024-04-25T08:58:04.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.409+0000 7fe843c88640 1 -- start start 2024-04-25T08:58:04.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.410+0000 7fe843c88640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe83c15def0 0x7fe83c06be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:04.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.410+0000 7fe843c88640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fe83c1619f0 0x7fe83c161e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:04.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.410+0000 7fe843c88640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe83c162820 con 0x7fe83c162390 2024-04-25T08:58:04.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.410+0000 7fe843c88640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe83c0b30d0 con 0x7fe83c1619f0 2024-04-25T08:58:04.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.410+0000 7fe843c88640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c0d0e90 con 0x7fe83c15def0 2024-04-25T08:58:04.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.410+0000 7fe841223640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fe83c1619f0 0x7fe83c161e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:04.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.410+0000 7fe841a24640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe83c15def0 0x7fe83c06be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:04.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.410+0000 7fe842225640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7fe83c162390 0x7fe83c0cfa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:43566/0 (socket says 172.21.15.43:43566) 2024-04-25T08:58:04.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.410+0000 7fe841a24640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe83c15def0 0x7fe83c06be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:40110/0 (socket says 172.21.15.43:40110) 2024-04-25T08:58:04.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.410+0000 7fe841223640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fe83c1619f0 0x7fe83c161e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:49554/0 (socket says 172.21.15.43:49554) 2024-04-25T08:58:04.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.410+0000 7fe842225640 1 -- 172.21.15.43:0/2097940956 learned_addr learned my addr 172.21.15.43:0/2097940956 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:04.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.411+0000 7fe841a24640 1 -- 172.21.15.43:0/2097940956 >> v2:172.21.15.193:3300/0 conn(0x7fe83c1619f0 msgr2=0x7fe83c161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:04.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.411+0000 7fe841a24640 1 --2- 172.21.15.43:0/2097940956 >> v2:172.21.15.193:3300/0 conn(0x7fe83c1619f0 0x7fe83c161e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.411+0000 7fe841a24640 1 -- 172.21.15.43:0/2097940956 >> v1:172.21.15.193:6789/0 conn(0x7fe83c162390 legacy=0x7fe83c0cfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:04.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.411+0000 7fe841a24640 1 -- 172.21.15.43:0/2097940956 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe83c15e2d0 con 0x7fe83c15def0 2024-04-25T08:58:04.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.411+0000 7fe841a24640 1 --2- 172.21.15.43:0/2097940956 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe83c15def0 0x7fe83c06be50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe8380097d0 tx=0x7fe83802a1c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6e6a96d87635eaa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:04.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.411+0000 7fe840a22640 1 -- 172.21.15.43:0/2097940956 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe838037070 con 0x7fe83c15def0 2024-04-25T08:58:04.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.411+0000 7fe840a22640 1 -- 172.21.15.43:0/2097940956 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe838032440 con 0x7fe83c15def0 2024-04-25T08:58:04.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.412+0000 7fe840a22640 1 -- 172.21.15.43:0/2097940956 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe838032810 con 0x7fe83c15def0 2024-04-25T08:58:04.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.412+0000 7fe843c88640 1 -- 172.21.15.43:0/2097940956 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe83c15def0 msgr2=0x7fe83c06be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:04.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.412+0000 7fe843c88640 1 --2- 172.21.15.43:0/2097940956 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe83c15def0 0x7fe83c06be50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe8380097d0 tx=0x7fe83802a1c0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.412+0000 7fe843c88640 1 -- 172.21.15.43:0/2097940956 shutdown_connections 2024-04-25T08:58:04.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.412+0000 7fe843c88640 1 --2- 172.21.15.43:0/2097940956 >> v2:172.21.15.193:3300/0 conn(0x7fe83c1619f0 0x7fe83c161e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.412+0000 7fe843c88640 1 --2- 172.21.15.43:0/2097940956 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe83c15def0 0x7fe83c06be50 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.412+0000 7fe843c88640 1 -- 172.21.15.43:0/2097940956 >> 172.21.15.43:0/2097940956 conn(0x7fe83c0b32e0 msgr2=0x7fe83c0b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:04.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.412+0000 7fe843c88640 1 -- 172.21.15.43:0/2097940956 shutdown_connections 2024-04-25T08:58:04.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.412+0000 7fe843c88640 1 -- 172.21.15.43:0/2097940956 wait complete. 2024-04-25T08:58:04.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.413+0000 7fe843c88640 1 Processor -- start 2024-04-25T08:58:04.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.413+0000 7fe843c88640 1 -- start start 2024-04-25T08:58:04.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.414+0000 7fe843c88640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe83c15def0 0x7fe83c15f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:04.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.414+0000 7fe843c88640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe83c1619f0 0x7fe83c0c51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:04.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.414+0000 7fe843c88640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe83c162390 0x7fe83c0c5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:04.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.414+0000 7fe843c88640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c0b1300 con 0x7fe83c15def0 2024-04-25T08:58:04.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.414+0000 7fe843c88640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c06c390 con 0x7fe83c162390 2024-04-25T08:58:04.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.414+0000 7fe843c88640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c0b15e0 con 0x7fe83c1619f0 2024-04-25T08:58:04.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.414+0000 7fe841a24640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe83c15def0 0x7fe83c15f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:04.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.414+0000 7fe841a24640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe83c15def0 0x7fe83c15f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44072/0 (socket says 172.21.15.43:44072) 2024-04-25T08:58:04.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.414+0000 7fe841a24640 1 -- 172.21.15.43:0/3635426718 learned_addr learned my addr 172.21.15.43:0/3635426718 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:04.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.414+0000 7fe841223640 1 --2- 172.21.15.43:0/3635426718 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe83c1619f0 0x7fe83c0c51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:04.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.414+0000 7fe842225640 1 --2- 172.21.15.43:0/3635426718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe83c162390 0x7fe83c0c5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:04.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.414+0000 7fe841a24640 1 -- 172.21.15.43:0/3635426718 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe83c1619f0 msgr2=0x7fe83c0c51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:04.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.414+0000 7fe841a24640 1 --2- 172.21.15.43:0/3635426718 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe83c1619f0 0x7fe83c0c51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.414+0000 7fe841a24640 1 -- 172.21.15.43:0/3635426718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe83c162390 msgr2=0x7fe83c0c5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:04.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.414+0000 7fe841a24640 1 --2- 172.21.15.43:0/3635426718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe83c162390 0x7fe83c0c5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.415+0000 7fe841a24640 1 -- 172.21.15.43:0/3635426718 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe83c160150 con 0x7fe83c15def0 2024-04-25T08:58:04.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.415+0000 7fe841a24640 1 --2- 172.21.15.43:0/3635426718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe83c15def0 0x7fe83c15f700 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fe8380098f0 tx=0x7fe8380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:04.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.415+0000 7fe8327fc640 1 -- 172.21.15.43:0/3635426718 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe838037040 con 0x7fe83c15def0 2024-04-25T08:58:04.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.415+0000 7fe8327fc640 1 -- 172.21.15.43:0/3635426718 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe838042070 con 0x7fe83c15def0 2024-04-25T08:58:04.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.415+0000 7fe843c88640 1 -- 172.21.15.43:0/3635426718 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe83c160430 con 0x7fe83c15def0 2024-04-25T08:58:04.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.415+0000 7fe8327fc640 1 -- 172.21.15.43:0/3635426718 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe838030940 con 0x7fe83c15def0 2024-04-25T08:58:04.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.415+0000 7fe843c88640 1 -- 172.21.15.43:0/3635426718 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe83c1609f0 con 0x7fe83c15def0 2024-04-25T08:58:04.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.416+0000 7fe843c88640 1 -- 172.21.15.43:0/3635426718 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe83c074ef0 con 0x7fe83c15def0 2024-04-25T08:58:04.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.416+0000 7fe8327fc640 1 -- 172.21.15.43:0/3635426718 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe838030ae0 con 0x7fe83c15def0 2024-04-25T08:58:04.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.417+0000 7fe8327fc640 1 --2- 172.21.15.43:0/3635426718 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe82006b4d0 0x7fe82006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:04.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.417+0000 7fe8327fc640 1 -- 172.21.15.43:0/3635426718 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe8380aaa70 con 0x7fe83c15def0 2024-04-25T08:58:04.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.417+0000 7fe841223640 1 --2- 172.21.15.43:0/3635426718 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe82006b4d0 0x7fe82006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:04.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.418+0000 7fe841223640 1 --2- 172.21.15.43:0/3635426718 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe82006b4d0 0x7fe82006d990 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe83c0c4f00 tx=0x7fe82c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:04.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.421+0000 7fe8327fc640 1 -- 172.21.15.43:0/3635426718 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe8380798f0 con 0x7fe83c15def0 2024-04-25T08:58:04.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.642+0000 7fe843c88640 1 -- 172.21.15.43:0/3635426718 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe83c0b36c0 con 0x7fe83c15def0 2024-04-25T08:58:04.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.642+0000 7fe8327fc640 1 -- 172.21.15.43:0/3635426718 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v31) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe8380791f0 con 0x7fe83c15def0 2024-04-25T08:58:04.632 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:58:04.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.644+0000 7fe843c88640 1 -- 172.21.15.43:0/3635426718 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe82006b4d0 msgr2=0x7fe82006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:04.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.644+0000 7fe843c88640 1 --2- 172.21.15.43:0/3635426718 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe82006b4d0 0x7fe82006d990 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe83c0c4f00 tx=0x7fe82c008040 comp rx=0 tx=0).stop 2024-04-25T08:58:04.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.644+0000 7fe843c88640 1 -- 172.21.15.43:0/3635426718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe83c15def0 msgr2=0x7fe83c15f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:04.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.644+0000 7fe843c88640 1 --2- 172.21.15.43:0/3635426718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe83c15def0 0x7fe83c15f700 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fe8380098f0 tx=0x7fe8380046c0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.644+0000 7fe843c88640 1 -- 172.21.15.43:0/3635426718 shutdown_connections 2024-04-25T08:58:04.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.644+0000 7fe843c88640 1 --2- 172.21.15.43:0/3635426718 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe82006b4d0 0x7fe82006d990 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.644+0000 7fe843c88640 1 --2- 172.21.15.43:0/3635426718 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe83c162390 0x7fe83c0c5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.644+0000 7fe843c88640 1 --2- 172.21.15.43:0/3635426718 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe83c1619f0 0x7fe83c0c51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.644+0000 7fe843c88640 1 --2- 172.21.15.43:0/3635426718 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe83c15def0 0x7fe83c15f700 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:04.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.644+0000 7fe843c88640 1 -- 172.21.15.43:0/3635426718 >> 172.21.15.43:0/3635426718 conn(0x7fe83c0b32e0 msgr2=0x7fe83c0d1380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:04.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.645+0000 7fe843c88640 1 -- 172.21.15.43:0/3635426718 shutdown_connections 2024-04-25T08:58:04.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:04.645+0000 7fe843c88640 1 -- 172.21.15.43:0/3635426718 wait complete. 2024-04-25T08:58:04.645 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:58:04.645 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:58:04.645 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-04-25T08:58:04.645 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:58:04.645 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:58:04.645 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:58:04.645 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:58:04.645 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:58:04.645 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:58:04.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:04.646 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-04-25T08:58:04.661 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:58:04.662 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:58:04.662 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:58:04.662 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:58:04.662 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-04-25T08:58:04.662 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:58:04.662 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:58:04.662 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-04-25T08:58:04.662 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-04-25T08:58:04.662 INFO:teuthology.orchestra.run.smithi043.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:58:04.662 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:58:04.662 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-04-25T08:58:04.662 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T08:58:04.662 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 42533sec preferred_lft 42533sec 2024-04-25T08:58:04.662 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link noprefixroute 2024-04-25T08:58:04.663 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:58:04.663 INFO:teuthology.orchestra.run.smithi043.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:58:04.663 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-04-25T08:58:04.663 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-04-25T08:58:04.663 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:58:04.663 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:58:04.663 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-04-25T08:58:04.663 INFO:teuthology.orchestra.run.smithi043.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:58:04.663 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 12:2c:7a:72:36:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:58:04.663 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:58:04.663 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:58:04.663 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ccff:4eff:fe9d:c8bf/64 scope link 2024-04-25T08:58:04.663 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:58:04.664 INFO:teuthology.orchestra.run.smithi043.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:58:04.664 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 16:9a:63:37:46:24 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:58:04.664 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::149a:63ff:fe37:4624/64 scope link 2024-04-25T08:58:04.664 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:58:04.664 INFO:teuthology.orchestra.run.smithi043.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:58:04.664 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 12:2c:7a:72:36:14 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:58:04.664 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::102c:7aff:fe72:3614/64 scope link 2024-04-25T08:58:04.664 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:58:04.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:04.664 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:58:04.725 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-04-25T08:58:04.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:04.727 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-04-25T08:58:04.783 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-04-25T08:58:04.783 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:58:04.783 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T08:58:04.783 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T08:58:04.783 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:58:04.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:04.784 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2024-04-25T08:58:04.784 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T08:58:04.784 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:58:04.784 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:58:04.784 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:58:04.784 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T08:58:04.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:04.868 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-04-25T08:58:04.923 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:58:04.923 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:58:04.924 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:58:04.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:04.924 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-04-25T08:58:04.989 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T08:58:04.989 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:58:04.989 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:58:05.044 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:58:05.045 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:58:05.045 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:58:05.100 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-04-25T08:58:05.165 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:58:05.221 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-04-25T08:58:05.221 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-04-25T08:58:05.221 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-04-25T08:58:05.222 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-04-25T08:58:05.222 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-04-25T08:58:05.222 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-04-25T08:58:05.222 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-04-25T08:58:05.222 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-04-25T08:58:05.222 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-04-25T08:58:05.222 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-04-25T08:58:05.222 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-04-25T08:58:05.222 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/selinux 2024-04-25T08:58:05.222 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:58:05.222 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-04-25T08:58:05.222 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-04-25T08:58:05.222 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-04-25T08:58:05.222 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-04-25T08:58:05.223 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:58:05.223 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-04-25T08:58:05.223 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-04-25T08:58:05.223 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:58:05.223 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:58:05.223 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:58:05.223 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:58:05.223 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-04-25T08:58:05.223 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-04-25T08:58:05.223 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:58:05.223 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:58:05.223 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:58:05.223 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:58:05.223 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-04-25T08:58:05.223 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:58:05.224 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:58:05.224 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:58:05.224 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:58:05.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:05.224 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-04-25T08:58:05.276 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T08:58:05.276 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T08:58:05.319 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-04-25T08:58:05.343 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:58:05.400 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-04-25T08:58:05.400 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-04-25T08:58:05.400 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-04-25T08:58:05.400 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-04-25T08:58:05.400 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-04-25T08:58:05.400 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-04-25T08:58:05.400 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-04-25T08:58:05.400 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-04-25T08:58:05.401 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-04-25T08:58:05.401 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-04-25T08:58:05.401 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-04-25T08:58:05.401 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/selinux 2024-04-25T08:58:05.401 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:58:05.401 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-04-25T08:58:05.401 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-04-25T08:58:05.401 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-04-25T08:58:05.401 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-04-25T08:58:05.401 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:58:05.401 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-04-25T08:58:05.401 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-04-25T08:58:05.402 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:58:05.402 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:58:05.402 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:58:05.402 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:58:05.402 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-04-25T08:58:05.402 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-04-25T08:58:05.402 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:58:05.402 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:58:05.402 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:58:05.402 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:58:05.402 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-04-25T08:58:05.403 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:58:05.403 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:58:05.403 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:58:05.403 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:58:05.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:05.403 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-04-25T08:58:05.404 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:2024-04-25T08:58:05.410+0000 7f389f21d480 -1 init, newargv = 0x55b96ee56750 newargc=15 2024-04-25T08:58:05.404 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[62874]: starting ceph client 2024-04-25T08:58:05.404 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[62874]: starting fuse 2024-04-25T08:58:05.419 INFO:teuthology.orchestra.run.smithi043.stdout:43 2024-04-25T08:58:05.419 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-25T08:58:05.419 DEBUG:teuthology.orchestra.run.smithi043:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:58:05.419 DEBUG:teuthology.orchestra.run.smithi043:> import glob 2024-04-25T08:58:05.419 DEBUG:teuthology.orchestra.run.smithi043:> import re 2024-04-25T08:58:05.419 DEBUG:teuthology.orchestra.run.smithi043:> import os 2024-04-25T08:58:05.419 DEBUG:teuthology.orchestra.run.smithi043:> import subprocess 2024-04-25T08:58:05.419 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T08:58:05.419 DEBUG:teuthology.orchestra.run.smithi043:> def _find_admin_socket(client_name): 2024-04-25T08:58:05.419 DEBUG:teuthology.orchestra.run.smithi043:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T08:58:05.419 DEBUG:teuthology.orchestra.run.smithi043:> files = glob.glob(asok_path) 2024-04-25T08:58:05.420 DEBUG:teuthology.orchestra.run.smithi043:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T08:58:05.420 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T08:58:05.420 DEBUG:teuthology.orchestra.run.smithi043:> # Given a non-glob path, it better be there 2024-04-25T08:58:05.420 DEBUG:teuthology.orchestra.run.smithi043:> if "*" not in asok_path: 2024-04-25T08:58:05.420 DEBUG:teuthology.orchestra.run.smithi043:> assert(len(files) == 1) 2024-04-25T08:58:05.420 DEBUG:teuthology.orchestra.run.smithi043:> return files[0] 2024-04-25T08:58:05.420 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T08:58:05.420 DEBUG:teuthology.orchestra.run.smithi043:> for f in files: 2024-04-25T08:58:05.420 DEBUG:teuthology.orchestra.run.smithi043:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:58:05.420 DEBUG:teuthology.orchestra.run.smithi043:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:58:05.420 DEBUG:teuthology.orchestra.run.smithi043:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:58:05.420 DEBUG:teuthology.orchestra.run.smithi043:> contents = proc_f.read() 2024-04-25T08:58:05.420 DEBUG:teuthology.orchestra.run.smithi043:> if mountpoint in contents: 2024-04-25T08:58:05.420 DEBUG:teuthology.orchestra.run.smithi043:> return f 2024-04-25T08:58:05.420 DEBUG:teuthology.orchestra.run.smithi043:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:58:05.421 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T08:58:05.421 DEBUG:teuthology.orchestra.run.smithi043:> print(_find_admin_socket("client.0")) 2024-04-25T08:58:05.421 DEBUG:teuthology.orchestra.run.smithi043:> ' 2024-04-25T08:58:05.538 INFO:teuthology.orchestra.run.smithi043.stdout:/var/run/ceph/ceph-client.0.62874.asok 2024-04-25T08:58:05.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:58:05.549 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.62874.asok 2024-04-25T08:58:05.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:05.549 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.62874.asok status 2024-04-25T08:58:05.685 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T08:58:05.686 INFO:teuthology.orchestra.run.smithi043.stdout: "metadata": { 2024-04-25T08:58:05.686 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:58:05.686 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:58:05.686 INFO:teuthology.orchestra.run.smithi043.stdout: "entity_id": "0", 2024-04-25T08:58:05.686 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": "smithi043", 2024-04-25T08:58:05.686 INFO:teuthology.orchestra.run.smithi043.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T08:58:05.686 INFO:teuthology.orchestra.run.smithi043.stdout: "pid": "62874", 2024-04-25T08:58:05.686 INFO:teuthology.orchestra.run.smithi043.stdout: "root": "/" 2024-04-25T08:58:05.686 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:58:05.686 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_count": 0, 2024-04-25T08:58:05.686 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_pinned_count": 0, 2024-04-25T08:58:05.687 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 6735, 2024-04-25T08:58:05.687 INFO:teuthology.orchestra.run.smithi043.stdout: "inst": { 2024-04-25T08:58:05.687 INFO:teuthology.orchestra.run.smithi043.stdout: "name": { 2024-04-25T08:58:05.687 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "client", 2024-04-25T08:58:05.687 INFO:teuthology.orchestra.run.smithi043.stdout: "num": 6735 2024-04-25T08:58:05.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:58:05.687 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-04-25T08:58:05.687 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-04-25T08:58:05.687 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:58:05.687 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3122689170 2024-04-25T08:58:05.687 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:58:05.687 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:58:05.687 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-04-25T08:58:05.687 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-04-25T08:58:05.687 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:58:05.688 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3122689170 2024-04-25T08:58:05.688 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:58:05.688 INFO:teuthology.orchestra.run.smithi043.stdout: "inst_str": "client.6735 192.168.0.1:0/3122689170", 2024-04-25T08:58:05.688 INFO:teuthology.orchestra.run.smithi043.stdout: "addr_str": "192.168.0.1:0/3122689170", 2024-04-25T08:58:05.688 INFO:teuthology.orchestra.run.smithi043.stdout: "inode_count": 1, 2024-04-25T08:58:05.688 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_epoch": 31, 2024-04-25T08:58:05.688 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch": 58, 2024-04-25T08:58:05.688 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:58:05.688 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklisted": false, 2024-04-25T08:58:05.688 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs" 2024-04-25T08:58:05.688 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T08:58:05.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:05.694 DEBUG:teuthology.orchestra.run.smithi043:> stat --file-system '--printf=%T 2024-04-25T08:58:05.694 DEBUG:teuthology.orchestra.run.smithi043:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T08:58:05.709 INFO:teuthology.orchestra.run.smithi043.stdout:fuseblk 2024-04-25T08:58:05.710 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T08:58:05.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:05.710 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:58:05.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:05.775 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir scrub_repair_path) 2024-04-25T08:58:05.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:05.836 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch scrub_repair_path/file) 2024-04-25T08:58:05.894 DEBUG:teuthology.orchestra.run.smithi043:> stdin-killer -- python3 -c ' 2024-04-25T08:58:05.894 DEBUG:teuthology.orchestra.run.smithi043:> import os 2024-04-25T08:58:05.894 DEBUG:teuthology.orchestra.run.smithi043:> import stat 2024-04-25T08:58:05.894 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T08:58:05.895 DEBUG:teuthology.orchestra.run.smithi043:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_repair_path").st_ino) 2024-04-25T08:58:05.895 DEBUG:teuthology.orchestra.run.smithi043:> ' 2024-04-25T08:58:06.001 INFO:teuthology.orchestra.run.smithi043.stdout:1099511627776 2024-04-25T08:58:06.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:58:06.010 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:58:06.010 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:58:06.066 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:58:06.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.217+0000 7f2338b3b640 1 Processor -- start 2024-04-25T08:58:06.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.217+0000 7f2338b3b640 1 -- start start 2024-04-25T08:58:06.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.218+0000 7f2338b3b640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f23340cbfc0 0x7f233406dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:06.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.218+0000 7f2338b3b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f23340ccbb0 0x7f23340cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:06.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.218+0000 7f2338b3b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23340b2780 con 0x7f233415f5e0 2024-04-25T08:58:06.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.218+0000 7f2338b3b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23340cd550 con 0x7f23340ccbb0 2024-04-25T08:58:06.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.218+0000 7f2338b3b640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23340b2a60 con 0x7f23340cbfc0 2024-04-25T08:58:06.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.218+0000 7f2331d9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f23340ccbb0 0x7f23340cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:06.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.218+0000 7f2331d9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f23340ccbb0 0x7f23340cd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44086/0 (socket says 172.21.15.43:44086) 2024-04-25T08:58:06.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.218+0000 7f2332d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f233415f5e0 0x7f233415fa50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35708/0 (socket says 172.21.15.43:35708) 2024-04-25T08:58:06.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.218+0000 7f2331d9b640 1 -- 172.21.15.43:0/3725901330 learned_addr learned my addr 172.21.15.43:0/3725901330 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:06.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.219+0000 7f233259c640 1 --2- 172.21.15.43:0/3725901330 >> v2:172.21.15.193:3300/0 conn(0x7f23340cbfc0 0x7f233406dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:06.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.219+0000 7f2331d9b640 1 -- 172.21.15.43:0/3725901330 >> v2:172.21.15.193:3300/0 conn(0x7f23340cbfc0 msgr2=0x7f233406dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:06.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.219+0000 7f2331d9b640 1 --2- 172.21.15.43:0/3725901330 >> v2:172.21.15.193:3300/0 conn(0x7f23340cbfc0 0x7f233406dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:06.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.219+0000 7f2331d9b640 1 -- 172.21.15.43:0/3725901330 >> v1:172.21.15.43:6789/0 conn(0x7f233415f5e0 legacy=0x7f233415fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:06.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.219+0000 7f233159a640 1 -- 172.21.15.43:0/3725901330 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3262998860 0 0) 0x7f23340b2780 con 0x7f233415f5e0 2024-04-25T08:58:06.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.219+0000 7f2331d9b640 1 -- 172.21.15.43:0/3725901330 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23340b2d40 con 0x7f23340ccbb0 2024-04-25T08:58:06.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.219+0000 7f2331d9b640 1 --2- 172.21.15.43:0/3725901330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23340ccbb0 0x7f23340cd010 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f23340cf630 tx=0x7f231c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=961a377c4dfb2500 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:06.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.219+0000 7f233159a640 1 -- 172.21.15.43:0/3725901330 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231c037070 con 0x7f23340ccbb0 2024-04-25T08:58:06.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.219+0000 7f233159a640 1 -- 172.21.15.43:0/3725901330 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f231c032440 con 0x7f23340ccbb0 2024-04-25T08:58:06.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.219+0000 7f233159a640 1 -- 172.21.15.43:0/3725901330 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231c032810 con 0x7f23340ccbb0 2024-04-25T08:58:06.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.220+0000 7f2338b3b640 1 -- 172.21.15.43:0/3725901330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23340ccbb0 msgr2=0x7f23340cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:06.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.220+0000 7f2338b3b640 1 --2- 172.21.15.43:0/3725901330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23340ccbb0 0x7f23340cd010 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f23340cf630 tx=0x7f231c02a270 comp rx=0 tx=0).stop 2024-04-25T08:58:06.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.220+0000 7f2338b3b640 1 -- 172.21.15.43:0/3725901330 shutdown_connections 2024-04-25T08:58:06.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.220+0000 7f2338b3b640 1 --2- 172.21.15.43:0/3725901330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23340ccbb0 0x7f23340cd010 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:06.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.220+0000 7f2338b3b640 1 --2- 172.21.15.43:0/3725901330 >> v2:172.21.15.193:3300/0 conn(0x7f23340cbfc0 0x7f233406dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:06.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.220+0000 7f2338b3b640 1 -- 172.21.15.43:0/3725901330 >> 172.21.15.43:0/3725901330 conn(0x7f23340b1480 msgr2=0x7f23340b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:06.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.220+0000 7f2338b3b640 1 -- 172.21.15.43:0/3725901330 shutdown_connections 2024-04-25T08:58:06.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.220+0000 7f2338b3b640 1 -- 172.21.15.43:0/3725901330 wait complete. 2024-04-25T08:58:06.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.221+0000 7f2338b3b640 1 Processor -- start 2024-04-25T08:58:06.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.222+0000 7f2338b3b640 1 -- start start 2024-04-25T08:58:06.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.222+0000 7f2338b3b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23340cbfc0 0x7f233415b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:06.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.222+0000 7f2338b3b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23340ccbb0 0x7f233413f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:06.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.222+0000 7f2338b3b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f233415f5e0 0x7f23340ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:06.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.222+0000 7f2338b3b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23340b35e0 con 0x7f23340ccbb0 2024-04-25T08:58:06.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.222+0000 7f2338b3b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23340b3300 con 0x7f23340cbfc0 2024-04-25T08:58:06.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.222+0000 7f2338b3b640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f233406e1a0 con 0x7f233415f5e0 2024-04-25T08:58:06.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.222+0000 7f2331d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23340ccbb0 0x7f233413f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:06.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.222+0000 7f2331d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23340ccbb0 0x7f233413f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44102/0 (socket says 172.21.15.43:44102) 2024-04-25T08:58:06.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.222+0000 7f2331d9b640 1 -- 172.21.15.43:0/1882720924 learned_addr learned my addr 172.21.15.43:0/1882720924 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:06.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.223+0000 7f233259c640 1 --2- 172.21.15.43:0/1882720924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23340cbfc0 0x7f233415b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:06.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.223+0000 7f2332d9d640 1 --2- 172.21.15.43:0/1882720924 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f233415f5e0 0x7f23340ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:06.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.223+0000 7f2331d9b640 1 -- 172.21.15.43:0/1882720924 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f233415f5e0 msgr2=0x7f23340ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:06.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.223+0000 7f2331d9b640 1 --2- 172.21.15.43:0/1882720924 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f233415f5e0 0x7f23340ad8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:06.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.223+0000 7f2331d9b640 1 -- 172.21.15.43:0/1882720924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23340cbfc0 msgr2=0x7f233415b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:06.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.223+0000 7f2331d9b640 1 --2- 172.21.15.43:0/1882720924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23340cbfc0 0x7f233415b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:06.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.223+0000 7f2331d9b640 1 -- 172.21.15.43:0/1882720924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23341404d0 con 0x7f23340ccbb0 2024-04-25T08:58:06.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.223+0000 7f2331d9b640 1 --2- 172.21.15.43:0/1882720924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23340ccbb0 0x7f233413f930 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f231c000c00 tx=0x7f231c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:06.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.223+0000 7f232affd640 1 -- 172.21.15.43:0/1882720924 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231c041020 con 0x7f23340ccbb0 2024-04-25T08:58:06.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.223+0000 7f232affd640 1 -- 172.21.15.43:0/1882720924 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f231c0306c0 con 0x7f23340ccbb0 2024-04-25T08:58:06.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.223+0000 7f2338b3b640 1 -- 172.21.15.43:0/1882720924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23341407b0 con 0x7f23340ccbb0 2024-04-25T08:58:06.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.223+0000 7f232affd640 1 -- 172.21.15.43:0/1882720924 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231c037040 con 0x7f23340ccbb0 2024-04-25T08:58:06.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.223+0000 7f2338b3b640 1 -- 172.21.15.43:0/1882720924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2334140d70 con 0x7f23340ccbb0 2024-04-25T08:58:06.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.224+0000 7f2338b3b640 1 -- 172.21.15.43:0/1882720924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2334074ef0 con 0x7f23340ccbb0 2024-04-25T08:58:06.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.225+0000 7f232affd640 1 -- 172.21.15.43:0/1882720924 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f231c044020 con 0x7f23340ccbb0 2024-04-25T08:58:06.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.225+0000 7f232affd640 1 --2- 172.21.15.43:0/1882720924 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f231406b4d0 0x7f231406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:06.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.225+0000 7f232affd640 1 -- 172.21.15.43:0/1882720924 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f231c0aa8d0 con 0x7f23340ccbb0 2024-04-25T08:58:06.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.226+0000 7f233259c640 1 --2- 172.21.15.43:0/1882720924 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f231406b4d0 0x7f231406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:06.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.226+0000 7f233259c640 1 --2- 172.21.15.43:0/1882720924 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f231406b4d0 0x7f231406d990 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f232c006fd0 tx=0x7f232c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:06.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.229+0000 7f232affd640 1 -- 172.21.15.43:0/1882720924 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f231c04b030 con 0x7f23340ccbb0 2024-04-25T08:58:06.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.409+0000 7f2338b3b640 1 -- 172.21.15.43:0/1882720924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23340afba0 con 0x7f23340ccbb0 2024-04-25T08:58:06.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.410+0000 7f232affd640 1 -- 172.21.15.43:0/1882720924 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16629 (secure 0 0 0) 0x7f231c0790f0 con 0x7f23340ccbb0 2024-04-25T08:58:06.400 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:58:06.400 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":58,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:57:55.889262+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:57:52.884445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T08:57:53.565716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:58:06.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.413+0000 7f2338b3b640 1 -- 172.21.15.43:0/1882720924 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f231406b4d0 msgr2=0x7f231406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:06.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.413+0000 7f2338b3b640 1 --2- 172.21.15.43:0/1882720924 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f231406b4d0 0x7f231406d990 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f232c006fd0 tx=0x7f232c008040 comp rx=0 tx=0).stop 2024-04-25T08:58:06.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.413+0000 7f2338b3b640 1 -- 172.21.15.43:0/1882720924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23340ccbb0 msgr2=0x7f233413f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:06.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.413+0000 7f2338b3b640 1 --2- 172.21.15.43:0/1882720924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23340ccbb0 0x7f233413f930 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f231c000c00 tx=0x7f231c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:58:06.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.413+0000 7f2338b3b640 1 -- 172.21.15.43:0/1882720924 shutdown_connections 2024-04-25T08:58:06.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.413+0000 7f2338b3b640 1 --2- 172.21.15.43:0/1882720924 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f233415f5e0 0x7f23340ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:06.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.413+0000 7f2338b3b640 1 --2- 172.21.15.43:0/1882720924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23340ccbb0 0x7f233413f930 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:06.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.413+0000 7f2338b3b640 1 --2- 172.21.15.43:0/1882720924 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f231406b4d0 0x7f231406d990 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:06.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.413+0000 7f2338b3b640 1 --2- 172.21.15.43:0/1882720924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f23340cbfc0 0x7f233415b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:06.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.413+0000 7f2338b3b640 1 -- 172.21.15.43:0/1882720924 >> 172.21.15.43:0/1882720924 conn(0x7f23340b1480 msgr2=0x7f233415eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:06.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.413+0000 7f2338b3b640 1 -- 172.21.15.43:0/1882720924 shutdown_connections 2024-04-25T08:58:06.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.413+0000 7f2338b3b640 1 -- 172.21.15.43:0/1882720924 wait complete. 2024-04-25T08:58:06.413 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:57:52.884445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:57:53.565716+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:58:06.414 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:58:06.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.528+0000 7f25b4ed8640 1 Processor -- start 2024-04-25T08:58:06.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.528+0000 7f25b4ed8640 1 -- start start 2024-04-25T08:58:06.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.529+0000 7f25b4ed8640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f25b006f6a0 0x7f25b00b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:06.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.529+0000 7f25b4ed8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f25b00cd470 0x7f25b006c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:06.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.529+0000 7f25b4ed8640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25b00d2620 con 0x7f25b015ddd0 2024-04-25T08:58:06.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.529+0000 7f25b4ed8640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25b00b0b80 con 0x7f25b00cd470 2024-04-25T08:58:06.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.529+0000 7f25b4ed8640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25b00b0e60 con 0x7f25b006f6a0 2024-04-25T08:58:06.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.529+0000 7f25aed9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f25b00cd470 0x7f25b006c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:06.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.529+0000 7f25aed9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f25b00cd470 0x7f25b006c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44118/0 (socket says 172.21.15.43:44118) 2024-04-25T08:58:06.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.529+0000 7f25aed9d640 1 -- 172.21.15.43:0/510220005 learned_addr learned my addr 172.21.15.43:0/510220005 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:06.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.529+0000 7f25a5d9b640 1 --2- 172.21.15.43:0/510220005 >> v2:172.21.15.193:3300/0 conn(0x7f25b006f6a0 0x7f25b00b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:06.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.529+0000 7f25aed9d640 1 -- 172.21.15.43:0/510220005 >> v2:172.21.15.193:3300/0 conn(0x7f25b006f6a0 msgr2=0x7f25b00b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:06.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.529+0000 7f25aed9d640 1 --2- 172.21.15.43:0/510220005 >> v2:172.21.15.193:3300/0 conn(0x7f25b006f6a0 0x7f25b00b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:06.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.529+0000 7f25aed9d640 1 -- 172.21.15.43:0/510220005 >> v1:172.21.15.193:6789/0 conn(0x7f25b015ddd0 legacy=0x7f25b006ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:06.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.529+0000 7f25aed9d640 1 -- 172.21.15.43:0/510220005 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b00cbf40 con 0x7f25b00cd470 2024-04-25T08:58:06.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.529+0000 7f25a5d9b640 1 --2- 172.21.15.43:0/510220005 >> v2:172.21.15.193:3300/0 conn(0x7f25b006f6a0 0x7f25b00b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:06.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.530+0000 7f25aed9d640 1 --2- 172.21.15.43:0/510220005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25b00cd470 0x7f25b006c4e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f25b00d1790 tx=0x7f25a802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861ce5ed0e05129e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:06.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.530+0000 7f25add9b640 1 -- 172.21.15.43:0/510220005 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a8036070 con 0x7f25b00cd470 2024-04-25T08:58:06.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.530+0000 7f25add9b640 1 -- 172.21.15.43:0/510220005 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a8031600 con 0x7f25b00cd470 2024-04-25T08:58:06.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.530+0000 7f25add9b640 1 -- 172.21.15.43:0/510220005 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a80319d0 con 0x7f25b00cd470 2024-04-25T08:58:06.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.530+0000 7f25b4ed8640 1 -- 172.21.15.43:0/510220005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25b00cd470 msgr2=0x7f25b006c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:06.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.530+0000 7f25b4ed8640 1 --2- 172.21.15.43:0/510220005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25b00cd470 0x7f25b006c4e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f25b00d1790 tx=0x7f25a802a6f0 comp rx=0 tx=0).stop 2024-04-25T08:58:06.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.531+0000 7f25b4ed8640 1 -- 172.21.15.43:0/510220005 shutdown_connections 2024-04-25T08:58:06.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.531+0000 7f25b4ed8640 1 --2- 172.21.15.43:0/510220005 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25b00cd470 0x7f25b006c4e0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:06.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.531+0000 7f25b4ed8640 1 --2- 172.21.15.43:0/510220005 >> v2:172.21.15.193:3300/0 conn(0x7f25b006f6a0 0x7f25b00b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:06.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.531+0000 7f25b4ed8640 1 -- 172.21.15.43:0/510220005 >> 172.21.15.43:0/510220005 conn(0x7f25b00d17e0 msgr2=0x7f25b00d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:06.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.531+0000 7f25b4ed8640 1 -- 172.21.15.43:0/510220005 shutdown_connections 2024-04-25T08:58:06.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.531+0000 7f25b4ed8640 1 -- 172.21.15.43:0/510220005 wait complete. 2024-04-25T08:58:06.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.532+0000 7f25b4ed8640 1 Processor -- start 2024-04-25T08:58:06.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.532+0000 7f25b4ed8640 1 -- start start 2024-04-25T08:58:06.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.533+0000 7f25b4ed8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25b006f6a0 0x7f25b0160a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:06.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.533+0000 7f25b4ed8640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f25b00cd470 0x7f25b0151590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:06.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.533+0000 7f25b4ed8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25b015ddd0 0x7f25b00c7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:06.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.533+0000 7f25b4ed8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b006cdb0 con 0x7f25b006f6a0 2024-04-25T08:58:06.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.533+0000 7f25b4ed8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b006cc30 con 0x7f25b015ddd0 2024-04-25T08:58:06.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.533+0000 7f25b4ed8640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25b006cf30 con 0x7f25b00cd470 2024-04-25T08:58:06.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.533+0000 7f25ae59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25b006f6a0 0x7f25b0160a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:06.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.533+0000 7f25ae59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25b006f6a0 0x7f25b0160a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44132/0 (socket says 172.21.15.43:44132) 2024-04-25T08:58:06.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.533+0000 7f25ae59c640 1 -- 172.21.15.43:0/3677186168 learned_addr learned my addr 172.21.15.43:0/3677186168 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:06.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.533+0000 7f25aed9d640 1 --2- 172.21.15.43:0/3677186168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25b015ddd0 0x7f25b00c7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:06.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.533+0000 7f25a5d9b640 1 --2- 172.21.15.43:0/3677186168 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f25b00cd470 0x7f25b0151590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:06.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.533+0000 7f25ae59c640 1 -- 172.21.15.43:0/3677186168 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f25b00cd470 msgr2=0x7f25b0151590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:06.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.533+0000 7f25ae59c640 1 --2- 172.21.15.43:0/3677186168 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f25b00cd470 0x7f25b0151590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:06.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.533+0000 7f25ae59c640 1 -- 172.21.15.43:0/3677186168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25b015ddd0 msgr2=0x7f25b00c7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:06.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.533+0000 7f25ae59c640 1 --2- 172.21.15.43:0/3677186168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25b015ddd0 0x7f25b00c7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:06.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.533+0000 7f25ae59c640 1 -- 172.21.15.43:0/3677186168 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b015f9d0 con 0x7f25b006f6a0 2024-04-25T08:58:06.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.533+0000 7f25ae59c640 1 --2- 172.21.15.43:0/3677186168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25b006f6a0 0x7f25b0160a70 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f25a000b420 tx=0x7f25a000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:06.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.534+0000 7f25a77fe640 1 -- 172.21.15.43:0/3677186168 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a0011830 con 0x7f25b006f6a0 2024-04-25T08:58:06.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.534+0000 7f25a77fe640 1 -- 172.21.15.43:0/3677186168 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25a0016070 con 0x7f25b006f6a0 2024-04-25T08:58:06.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.534+0000 7f25b4ed8640 1 -- 172.21.15.43:0/3677186168 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b015fcb0 con 0x7f25b006f6a0 2024-04-25T08:58:06.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.534+0000 7f25a77fe640 1 -- 172.21.15.43:0/3677186168 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25a0011c10 con 0x7f25b006f6a0 2024-04-25T08:58:06.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.534+0000 7f25b4ed8640 1 -- 172.21.15.43:0/3677186168 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b0160270 con 0x7f25b006f6a0 2024-04-25T08:58:06.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.535+0000 7f25b4ed8640 1 -- 172.21.15.43:0/3677186168 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25b0074f60 con 0x7f25b006f6a0 2024-04-25T08:58:06.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.535+0000 7f25a77fe640 1 -- 172.21.15.43:0/3677186168 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f25a001b460 con 0x7f25b006f6a0 2024-04-25T08:58:06.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.536+0000 7f25a77fe640 1 --2- 172.21.15.43:0/3677186168 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f259006b500 0x7f259006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:06.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.536+0000 7f25a77fe640 1 -- 172.21.15.43:0/3677186168 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f25a0089b20 con 0x7f25b006f6a0 2024-04-25T08:58:06.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.536+0000 7f25a5d9b640 1 --2- 172.21.15.43:0/3677186168 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f259006b500 0x7f259006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:06.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.537+0000 7f25a5d9b640 1 --2- 172.21.15.43:0/3677186168 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f259006b500 0x7f259006d9c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f25b015afb0 tx=0x7f2598008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:06.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.540+0000 7f25a77fe640 1 -- 172.21.15.43:0/3677186168 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f25a0020020 con 0x7f25b006f6a0 2024-04-25T08:58:06.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.719+0000 7f25b4ed8640 1 -- 172.21.15.43:0/3677186168 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f25b00743d0 con 0x7f25b006f6a0 2024-04-25T08:58:06.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.719+0000 7f25a77fe640 1 -- 172.21.15.43:0/3677186168 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7f25a00583e0 con 0x7f25b006f6a0 2024-04-25T08:58:06.709 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T08:58:06.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.721+0000 7f25b4ed8640 1 -- 172.21.15.43:0/3677186168 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f259006b500 msgr2=0x7f259006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:06.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.721+0000 7f25b4ed8640 1 --2- 172.21.15.43:0/3677186168 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f259006b500 0x7f259006d9c0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f25b015afb0 tx=0x7f2598008040 comp rx=0 tx=0).stop 2024-04-25T08:58:06.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.721+0000 7f25b4ed8640 1 -- 172.21.15.43:0/3677186168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25b006f6a0 msgr2=0x7f25b0160a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:06.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.721+0000 7f25b4ed8640 1 --2- 172.21.15.43:0/3677186168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25b006f6a0 0x7f25b0160a70 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f25a000b420 tx=0x7f25a000b8e0 comp rx=0 tx=0).stop 2024-04-25T08:58:06.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.721+0000 7f25b4ed8640 1 -- 172.21.15.43:0/3677186168 shutdown_connections 2024-04-25T08:58:06.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.721+0000 7f25b4ed8640 1 --2- 172.21.15.43:0/3677186168 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f259006b500 0x7f259006d9c0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:06.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.721+0000 7f25b4ed8640 1 --2- 172.21.15.43:0/3677186168 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25b015ddd0 0x7f25b00c7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:06.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.721+0000 7f25b4ed8640 1 --2- 172.21.15.43:0/3677186168 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f25b00cd470 0x7f25b0151590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:06.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.721+0000 7f25b4ed8640 1 --2- 172.21.15.43:0/3677186168 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25b006f6a0 0x7f25b0160a70 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:06.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.721+0000 7f25b4ed8640 1 -- 172.21.15.43:0/3677186168 >> 172.21.15.43:0/3677186168 conn(0x7f25b00d17e0 msgr2=0x7f25b00aec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:06.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.721+0000 7f25b4ed8640 1 -- 172.21.15.43:0/3677186168 shutdown_connections 2024-04-25T08:58:06.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.721+0000 7f25b4ed8640 1 -- 172.21.15.43:0/3677186168 wait complete. 2024-04-25T08:58:06.721 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:58:06.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.836+0000 7f3a9c7a5640 1 Processor -- start 2024-04-25T08:58:06.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.836+0000 7f3a9c7a5640 1 -- start start 2024-04-25T08:58:06.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.837+0000 7f3a9c7a5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3a940cd470 0x7f3a940b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:06.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.837+0000 7f3a9c7a5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a940d2620 con 0x7f3a9415ddd0 2024-04-25T08:58:06.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.837+0000 7f3a9c7a5640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a940b0b80 con 0x7f3a9406f6a0 2024-04-25T08:58:06.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.837+0000 7f3a9c7a5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a940b0e60 con 0x7f3a940cd470 2024-04-25T08:58:06.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.837+0000 7f3a9ad42640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3a940cd470 0x7f3a940b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:06.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.837+0000 7f3a9a541640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f3a9415ddd0 0x7f3a9406ef90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35724/0 (socket says 172.21.15.43:35724) 2024-04-25T08:58:06.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.837+0000 7f3a9ad42640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3a940cd470 0x7f3a940b3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44138/0 (socket says 172.21.15.43:44138) 2024-04-25T08:58:06.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.837+0000 7f3a9a541640 1 -- 172.21.15.43:0/2051767979 learned_addr learned my addr 172.21.15.43:0/2051767979 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:06.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.838+0000 7f3a9ad42640 1 -- 172.21.15.43:0/2051767979 >> v1:172.21.15.193:6789/0 conn(0x7f3a9406f6a0 legacy=0x7f3a9406c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:06.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.838+0000 7f3a9ad42640 1 -- 172.21.15.43:0/2051767979 >> v1:172.21.15.43:6789/0 conn(0x7f3a9415ddd0 legacy=0x7f3a9406ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:06.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.838+0000 7f3a9ad42640 1 -- 172.21.15.43:0/2051767979 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a940cbf40 con 0x7f3a940cd470 2024-04-25T08:58:06.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.838+0000 7f3a9953f640 1 -- 172.21.15.43:0/2051767979 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3937475606 0 0) 0x7f3a940d2620 con 0x7f3a9415ddd0 2024-04-25T08:58:06.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.838+0000 7f3a9ad42640 1 --2- 172.21.15.43:0/2051767979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a940cd470 0x7f3a940b3dd0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009d90 tx=0x7f3a8402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c66e93dbf1a3ada0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:06.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.838+0000 7f3a9953f640 1 -- 172.21.15.43:0/2051767979 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a84007740 con 0x7f3a940cd470 2024-04-25T08:58:06.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.839+0000 7f3a9953f640 1 -- 172.21.15.43:0/2051767979 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a840078e0 con 0x7f3a940cd470 2024-04-25T08:58:06.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.839+0000 7f3a9953f640 1 -- 172.21.15.43:0/2051767979 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a84007cb0 con 0x7f3a940cd470 2024-04-25T08:58:06.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.839+0000 7f3a9c7a5640 1 -- 172.21.15.43:0/2051767979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a940cd470 msgr2=0x7f3a940b3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:06.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.839+0000 7f3a9c7a5640 1 --2- 172.21.15.43:0/2051767979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a940cd470 0x7f3a940b3dd0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009d90 tx=0x7f3a8402a6a0 comp rx=0 tx=0).stop 2024-04-25T08:58:06.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.839+0000 7f3a9c7a5640 1 -- 172.21.15.43:0/2051767979 shutdown_connections 2024-04-25T08:58:06.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.839+0000 7f3a9c7a5640 1 --2- 172.21.15.43:0/2051767979 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a940cd470 0x7f3a940b3dd0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:06.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.839+0000 7f3a9c7a5640 1 -- 172.21.15.43:0/2051767979 >> 172.21.15.43:0/2051767979 conn(0x7f3a940d17e0 msgr2=0x7f3a940d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:06.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.839+0000 7f3a9c7a5640 1 -- 172.21.15.43:0/2051767979 shutdown_connections 2024-04-25T08:58:06.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.839+0000 7f3a9c7a5640 1 -- 172.21.15.43:0/2051767979 wait complete. 2024-04-25T08:58:06.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.840+0000 7f3a9c7a5640 1 Processor -- start 2024-04-25T08:58:06.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.840+0000 7f3a9c7a5640 1 -- start start 2024-04-25T08:58:06.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.841+0000 7f3a9c7a5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a9406f6a0 0x7f3a940c8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:06.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.841+0000 7f3a9c7a5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a940cd470 0x7f3a941516c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:06.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.841+0000 7f3a9c7a5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3a9415ddd0 0x7f3a940cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:06.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.841+0000 7f3a9c7a5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9406d000 con 0x7f3a940cd470 2024-04-25T08:58:06.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.841+0000 7f3a9c7a5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9406ce80 con 0x7f3a9406f6a0 2024-04-25T08:58:06.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.841+0000 7f3a9c7a5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9406d180 con 0x7f3a9415ddd0 2024-04-25T08:58:06.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.841+0000 7f3a99d40640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a940cd470 0x7f3a941516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:06.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.841+0000 7f3a99d40640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a940cd470 0x7f3a941516c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44148/0 (socket says 172.21.15.43:44148) 2024-04-25T08:58:06.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.841+0000 7f3a99d40640 1 -- 172.21.15.43:0/3285840314 learned_addr learned my addr 172.21.15.43:0/3285840314 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:06.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.841+0000 7f3a99d40640 1 -- 172.21.15.43:0/3285840314 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3a9415ddd0 msgr2=0x7f3a940cabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:06.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.841+0000 7f3a99d40640 1 --2- 172.21.15.43:0/3285840314 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3a9415ddd0 0x7f3a940cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:06.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.841+0000 7f3a99d40640 1 -- 172.21.15.43:0/3285840314 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a9406f6a0 msgr2=0x7f3a940c8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:06.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.841+0000 7f3a99d40640 1 --2- 172.21.15.43:0/3285840314 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a9406f6a0 0x7f3a940c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:06.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.841+0000 7f3a99d40640 1 -- 172.21.15.43:0/3285840314 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a94151c00 con 0x7f3a940cd470 2024-04-25T08:58:06.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.842+0000 7f3a99d40640 1 --2- 172.21.15.43:0/3285840314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a940cd470 0x7f3a941516c0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f3a8800c460 tx=0x7f3a8800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:06.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.842+0000 7f3a92ffd640 1 -- 172.21.15.43:0/3285840314 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a88016020 con 0x7f3a940cd470 2024-04-25T08:58:06.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.842+0000 7f3a92ffd640 1 -- 172.21.15.43:0/3285840314 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a88012870 con 0x7f3a940cd470 2024-04-25T08:58:06.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.842+0000 7f3a9c7a5640 1 -- 172.21.15.43:0/3285840314 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a941604e0 con 0x7f3a940cd470 2024-04-25T08:58:06.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.842+0000 7f3a92ffd640 1 -- 172.21.15.43:0/3285840314 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a88012c60 con 0x7f3a940cd470 2024-04-25T08:58:06.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.842+0000 7f3a9c7a5640 1 -- 172.21.15.43:0/3285840314 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a94160aa0 con 0x7f3a940cd470 2024-04-25T08:58:06.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.843+0000 7f3a9c7a5640 1 -- 172.21.15.43:0/3285840314 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a94074f60 con 0x7f3a940cd470 2024-04-25T08:58:06.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.843+0000 7f3a92ffd640 1 -- 172.21.15.43:0/3285840314 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3a88019020 con 0x7f3a940cd470 2024-04-25T08:58:06.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.844+0000 7f3a92ffd640 1 --2- 172.21.15.43:0/3285840314 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3a8006b4d0 0x7f3a8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:06.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.844+0000 7f3a92ffd640 1 -- 172.21.15.43:0/3285840314 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a88089ce0 con 0x7f3a940cd470 2024-04-25T08:58:06.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.844+0000 7f3a9a541640 1 --2- 172.21.15.43:0/3285840314 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3a8006b4d0 0x7f3a8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:06.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.844+0000 7f3a9a541640 1 --2- 172.21.15.43:0/3285840314 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3a8006b4d0 0x7f3a8006d990 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c007fd0 tx=0x7f3a8c007960 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:06.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:06.848+0000 7f3a92ffd640 1 -- 172.21.15.43:0/3285840314 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3a88020070 con 0x7f3a940cd470 2024-04-25T08:58:07.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.029+0000 7f3a9c7a5640 1 -- 172.21.15.43:0/3285840314 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3a9406f310 con 0x7f3a940cd470 2024-04-25T08:58:07.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.030+0000 7f3a92ffd640 1 -- 172.21.15.43:0/3285840314 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v58) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3a94151c00 con 0x7f3a940cd470 2024-04-25T08:58:07.019 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T08:58:07.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.031+0000 7f3a9c7a5640 1 -- 172.21.15.43:0/3285840314 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3a8006b4d0 msgr2=0x7f3a8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:07.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.031+0000 7f3a9c7a5640 1 --2- 172.21.15.43:0/3285840314 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3a8006b4d0 0x7f3a8006d990 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c007fd0 tx=0x7f3a8c007960 comp rx=0 tx=0).stop 2024-04-25T08:58:07.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.031+0000 7f3a9c7a5640 1 -- 172.21.15.43:0/3285840314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a940cd470 msgr2=0x7f3a941516c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:07.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.031+0000 7f3a9c7a5640 1 --2- 172.21.15.43:0/3285840314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a940cd470 0x7f3a941516c0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f3a8800c460 tx=0x7f3a8800c920 comp rx=0 tx=0).stop 2024-04-25T08:58:07.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.032+0000 7f3a9c7a5640 1 -- 172.21.15.43:0/3285840314 shutdown_connections 2024-04-25T08:58:07.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.032+0000 7f3a9c7a5640 1 --2- 172.21.15.43:0/3285840314 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3a9415ddd0 0x7f3a940cabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:07.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.032+0000 7f3a9c7a5640 1 --2- 172.21.15.43:0/3285840314 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3a940cd470 0x7f3a941516c0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:07.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.032+0000 7f3a9c7a5640 1 --2- 172.21.15.43:0/3285840314 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3a8006b4d0 0x7f3a8006d990 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:07.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.032+0000 7f3a9c7a5640 1 --2- 172.21.15.43:0/3285840314 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3a9406f6a0 0x7f3a940c8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:07.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.032+0000 7f3a9c7a5640 1 -- 172.21.15.43:0/3285840314 >> 172.21.15.43:0/3285840314 conn(0x7f3a940d17e0 msgr2=0x7f3a9415d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:07.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.032+0000 7f3a9c7a5640 1 -- 172.21.15.43:0/3285840314 shutdown_connections 2024-04-25T08:58:07.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.032+0000 7f3a9c7a5640 1 -- 172.21.15.43:0/3285840314 wait complete. 2024-04-25T08:58:07.032 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:58:07.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.146+0000 7f310bd10640 1 Processor -- start 2024-04-25T08:58:07.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.147+0000 7f310bd10640 1 -- start start 2024-04-25T08:58:07.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.147+0000 7f310bd10640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f31040d04a0 0x7f310415ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:07.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.147+0000 7f310bd10640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f310415dec0 0x7f31040d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:07.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.147+0000 7f310bd10640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31040d0dc0 con 0x7f31040ce8c0 2024-04-25T08:58:07.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.147+0000 7f310bd10640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31040cf110 con 0x7f31040d04a0 2024-04-25T08:58:07.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.147+0000 7f310bd10640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31040cf3f0 con 0x7f310415dec0 2024-04-25T08:58:07.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.148+0000 7f3109aac640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f31040d04a0 0x7f310415ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:07.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.148+0000 7f310a2ad640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f31040ce8c0 0x7f31041617a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35726/0 (socket says 172.21.15.43:35726) 2024-04-25T08:58:07.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.148+0000 7f310a2ad640 1 -- 172.21.15.43:0/2219916715 learned_addr learned my addr 172.21.15.43:0/2219916715 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:07.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.148+0000 7f31092ab640 1 --2- 172.21.15.43:0/2219916715 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f310415dec0 0x7f31040d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:07.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.148+0000 7f3109aac640 1 -- 172.21.15.43:0/2219916715 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f310415dec0 msgr2=0x7f31040d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:07.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.148+0000 7f3109aac640 1 --2- 172.21.15.43:0/2219916715 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f310415dec0 0x7f31040d0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:07.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.148+0000 7f3109aac640 1 -- 172.21.15.43:0/2219916715 >> v1:172.21.15.43:6789/0 conn(0x7f31040ce8c0 legacy=0x7f31041617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:07.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.148+0000 7f3109aac640 1 -- 172.21.15.43:0/2219916715 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31040cf6d0 con 0x7f31040d04a0 2024-04-25T08:58:07.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.149+0000 7f3109aac640 1 --2- 172.21.15.43:0/2219916715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31040d04a0 0x7f310415ed90 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f31000097a0 tx=0x7f310002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca6de2e5bee266f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:07.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.149+0000 7f3108aaa640 1 -- 172.21.15.43:0/2219916715 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3100037070 con 0x7f31040d04a0 2024-04-25T08:58:07.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.149+0000 7f3108aaa640 1 -- 172.21.15.43:0/2219916715 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3100032440 con 0x7f31040d04a0 2024-04-25T08:58:07.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.149+0000 7f3108aaa640 1 -- 172.21.15.43:0/2219916715 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3100032810 con 0x7f31040d04a0 2024-04-25T08:58:07.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.149+0000 7f310bd10640 1 -- 172.21.15.43:0/2219916715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31040d04a0 msgr2=0x7f310415ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:07.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.149+0000 7f310bd10640 1 --2- 172.21.15.43:0/2219916715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31040d04a0 0x7f310415ed90 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f31000097a0 tx=0x7f310002a180 comp rx=0 tx=0).stop 2024-04-25T08:58:07.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.149+0000 7f310bd10640 1 -- 172.21.15.43:0/2219916715 shutdown_connections 2024-04-25T08:58:07.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.149+0000 7f310bd10640 1 --2- 172.21.15.43:0/2219916715 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f310415dec0 0x7f31040d0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:07.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.149+0000 7f310bd10640 1 --2- 172.21.15.43:0/2219916715 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31040d04a0 0x7f310415ed90 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:07.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.149+0000 7f310bd10640 1 -- 172.21.15.43:0/2219916715 >> 172.21.15.43:0/2219916715 conn(0x7f310409cd40 msgr2=0x7f31040d3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:07.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.149+0000 7f310bd10640 1 -- 172.21.15.43:0/2219916715 shutdown_connections 2024-04-25T08:58:07.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.150+0000 7f310bd10640 1 -- 172.21.15.43:0/2219916715 wait complete. 2024-04-25T08:58:07.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.151+0000 7f310bd10640 1 Processor -- start 2024-04-25T08:58:07.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.151+0000 7f310bd10640 1 -- start start 2024-04-25T08:58:07.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.151+0000 7f310bd10640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f31040ce8c0 0x7f31040c9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:07.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.151+0000 7f310bd10640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31040d04a0 0x7f31041f7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:07.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.151+0000 7f310bd10640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f310415dec0 0x7f31041f83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:07.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.151+0000 7f310bd10640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31040cecf0 con 0x7f31040d04a0 2024-04-25T08:58:07.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.151+0000 7f310bd10640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31040cfc90 con 0x7f310415dec0 2024-04-25T08:58:07.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.151+0000 7f310bd10640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31040cee70 con 0x7f31040ce8c0 2024-04-25T08:58:07.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.151+0000 7f31092ab640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31040d04a0 0x7f31041f7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:07.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.151+0000 7f31092ab640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31040d04a0 0x7f31041f7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44170/0 (socket says 172.21.15.43:44170) 2024-04-25T08:58:07.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.151+0000 7f31092ab640 1 -- 172.21.15.43:0/2734841688 learned_addr learned my addr 172.21.15.43:0/2734841688 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:07.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.152+0000 7f3109aac640 1 --2- 172.21.15.43:0/2734841688 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f31040ce8c0 0x7f31040c9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:07.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.152+0000 7f310a2ad640 1 --2- 172.21.15.43:0/2734841688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f310415dec0 0x7f31041f83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:07.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.152+0000 7f31092ab640 1 -- 172.21.15.43:0/2734841688 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f31040ce8c0 msgr2=0x7f31040c9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:07.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.152+0000 7f31092ab640 1 --2- 172.21.15.43:0/2734841688 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f31040ce8c0 0x7f31040c9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:07.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.152+0000 7f31092ab640 1 -- 172.21.15.43:0/2734841688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f310415dec0 msgr2=0x7f31041f83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:07.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.152+0000 7f31092ab640 1 --2- 172.21.15.43:0/2734841688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f310415dec0 0x7f31041f83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:07.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.152+0000 7f31092ab640 1 -- 172.21.15.43:0/2734841688 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31041f5150 con 0x7f31040d04a0 2024-04-25T08:58:07.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.152+0000 7f31092ab640 1 --2- 172.21.15.43:0/2734841688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31040d04a0 0x7f31041f7e80 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f30f400d7d0 tx=0x7f30f400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:07.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.152+0000 7f30fa7fc640 1 -- 172.21.15.43:0/2734841688 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f4004020 con 0x7f31040d04a0 2024-04-25T08:58:07.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.152+0000 7f30fa7fc640 1 -- 172.21.15.43:0/2734841688 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f40041e0 con 0x7f31040d04a0 2024-04-25T08:58:07.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.152+0000 7f310bd10640 1 -- 172.21.15.43:0/2734841688 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31041f4390 con 0x7f31040d04a0 2024-04-25T08:58:07.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.152+0000 7f30fa7fc640 1 -- 172.21.15.43:0/2734841688 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f40043d0 con 0x7f31040d04a0 2024-04-25T08:58:07.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.152+0000 7f310bd10640 1 -- 172.21.15.43:0/2734841688 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31041f4950 con 0x7f31040d04a0 2024-04-25T08:58:07.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.153+0000 7f310bd10640 1 -- 172.21.15.43:0/2734841688 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3104074f60 con 0x7f31040d04a0 2024-04-25T08:58:07.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.154+0000 7f30fa7fc640 1 -- 172.21.15.43:0/2734841688 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f30f401b020 con 0x7f31040d04a0 2024-04-25T08:58:07.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.154+0000 7f30fa7fc640 1 --2- 172.21.15.43:0/2734841688 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f30e806b500 0x7f30e806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:07.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.154+0000 7f30fa7fc640 1 -- 172.21.15.43:0/2734841688 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f30f4089da0 con 0x7f31040d04a0 2024-04-25T08:58:07.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.154+0000 7f3109aac640 1 --2- 172.21.15.43:0/2734841688 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f30e806b500 0x7f30e806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:07.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.155+0000 7f3109aac640 1 --2- 172.21.15.43:0/2734841688 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f30e806b500 0x7f30e806d9c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3100034000 tx=0x7f310002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:07.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.158+0000 7f30fa7fc640 1 -- 172.21.15.43:0/2734841688 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f30f4058660 con 0x7f31040d04a0 2024-04-25T08:58:07.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.336+0000 7f310bd10640 1 -- 172.21.15.43:0/2734841688 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f310408daf0 con 0x7f31040d04a0 2024-04-25T08:58:07.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.336+0000 7f30fa7fc640 1 -- 172.21.15.43:0/2734841688 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v58) v1 ==== 100+0+11 (secure 0 0 0) 0x7f30f405c700 con 0x7f31040d04a0 2024-04-25T08:58:07.326 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T08:58:07.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.338+0000 7f310bd10640 1 -- 172.21.15.43:0/2734841688 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f30e806b500 msgr2=0x7f30e806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:07.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.338+0000 7f310bd10640 1 --2- 172.21.15.43:0/2734841688 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f30e806b500 0x7f30e806d9c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3100034000 tx=0x7f310002e000 comp rx=0 tx=0).stop 2024-04-25T08:58:07.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.338+0000 7f310bd10640 1 -- 172.21.15.43:0/2734841688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31040d04a0 msgr2=0x7f31041f7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:07.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.338+0000 7f310bd10640 1 --2- 172.21.15.43:0/2734841688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31040d04a0 0x7f31041f7e80 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f30f400d7d0 tx=0x7f30f400dc90 comp rx=0 tx=0).stop 2024-04-25T08:58:07.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.338+0000 7f310bd10640 1 -- 172.21.15.43:0/2734841688 shutdown_connections 2024-04-25T08:58:07.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.338+0000 7f310bd10640 1 --2- 172.21.15.43:0/2734841688 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f30e806b500 0x7f30e806d9c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:07.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.338+0000 7f310bd10640 1 --2- 172.21.15.43:0/2734841688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f310415dec0 0x7f31041f83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:07.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.338+0000 7f310bd10640 1 --2- 172.21.15.43:0/2734841688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31040d04a0 0x7f31041f7e80 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:07.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.338+0000 7f310bd10640 1 --2- 172.21.15.43:0/2734841688 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f31040ce8c0 0x7f31040c9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:07.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.338+0000 7f310bd10640 1 -- 172.21.15.43:0/2734841688 >> 172.21.15.43:0/2734841688 conn(0x7f310409cd40 msgr2=0x7f31040b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:07.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.338+0000 7f310bd10640 1 -- 172.21.15.43:0/2734841688 shutdown_connections 2024-04-25T08:58:07.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.338+0000 7f310bd10640 1 -- 172.21.15.43:0/2734841688 wait complete. 2024-04-25T08:58:07.339 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:58:07.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.451+0000 7fa4d7309640 1 Processor -- start 2024-04-25T08:58:07.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.451+0000 7fa4d7309640 1 -- start start 2024-04-25T08:58:07.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.452+0000 7fa4d7309640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa4d015a9d0 0x7fa4d006d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:07.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.452+0000 7fa4d7309640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fa4d0072160 0x7fa4d00cd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:07.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.452+0000 7fa4d7309640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4d00704f0 con 0x7fa4d006d7d0 2024-04-25T08:58:07.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.452+0000 7fa4d7309640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4d006dc30 con 0x7fa4d015a9d0 2024-04-25T08:58:07.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.452+0000 7fa4d7309640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4d00707d0 con 0x7fa4d0072160 2024-04-25T08:58:07.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.452+0000 7fa4d50a5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa4d015a9d0 0x7fa4d006d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:07.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.452+0000 7fa4d48a4640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa4d006d7d0 0x7fa4d0071a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35736/0 (socket says 172.21.15.43:35736) 2024-04-25T08:58:07.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.452+0000 7fa4d48a4640 1 -- 172.21.15.43:0/4224966765 learned_addr learned my addr 172.21.15.43:0/4224966765 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:07.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.452+0000 7fa4d58a6640 1 --2- 172.21.15.43:0/4224966765 >> v2:172.21.15.193:3300/0 conn(0x7fa4d0072160 0x7fa4d00cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:07.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.452+0000 7fa4d50a5640 1 -- 172.21.15.43:0/4224966765 >> v2:172.21.15.193:3300/0 conn(0x7fa4d0072160 msgr2=0x7fa4d00cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:07.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.452+0000 7fa4d50a5640 1 --2- 172.21.15.43:0/4224966765 >> v2:172.21.15.193:3300/0 conn(0x7fa4d0072160 0x7fa4d00cd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:07.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.453+0000 7fa4d50a5640 1 -- 172.21.15.43:0/4224966765 >> v1:172.21.15.43:6789/0 conn(0x7fa4d006d7d0 legacy=0x7fa4d0071a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:07.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.453+0000 7fa4cffff640 1 -- 172.21.15.43:0/4224966765 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 668391662 0 0) 0x7fa4d00704f0 con 0x7fa4d006d7d0 2024-04-25T08:58:07.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.453+0000 7fa4d50a5640 1 -- 172.21.15.43:0/4224966765 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d0070ab0 con 0x7fa4d015a9d0 2024-04-25T08:58:07.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.453+0000 7fa4d50a5640 1 --2- 172.21.15.43:0/4224966765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4d015a9d0 0x7fa4d006d290 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0067530 tx=0x7fa4c802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f76f8cab9ddd98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:07.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.453+0000 7fa4cffff640 1 -- 172.21.15.43:0/4224966765 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c8037070 con 0x7fa4d015a9d0 2024-04-25T08:58:07.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.453+0000 7fa4cffff640 1 -- 172.21.15.43:0/4224966765 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c8032440 con 0x7fa4d015a9d0 2024-04-25T08:58:07.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.453+0000 7fa4cffff640 1 -- 172.21.15.43:0/4224966765 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c8032810 con 0x7fa4d015a9d0 2024-04-25T08:58:07.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.454+0000 7fa4d7309640 1 -- 172.21.15.43:0/4224966765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4d015a9d0 msgr2=0x7fa4d006d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:07.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.454+0000 7fa4d7309640 1 --2- 172.21.15.43:0/4224966765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4d015a9d0 0x7fa4d006d290 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fa4d0067530 tx=0x7fa4c802a1a0 comp rx=0 tx=0).stop 2024-04-25T08:58:07.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.454+0000 7fa4d7309640 1 -- 172.21.15.43:0/4224966765 shutdown_connections 2024-04-25T08:58:07.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.454+0000 7fa4d7309640 1 --2- 172.21.15.43:0/4224966765 >> v2:172.21.15.193:3300/0 conn(0x7fa4d0072160 0x7fa4d00cd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:07.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.454+0000 7fa4d7309640 1 --2- 172.21.15.43:0/4224966765 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4d015a9d0 0x7fa4d006d290 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:07.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.454+0000 7fa4d7309640 1 -- 172.21.15.43:0/4224966765 >> 172.21.15.43:0/4224966765 conn(0x7fa4d00b1480 msgr2=0x7fa4d00b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:07.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.454+0000 7fa4d7309640 1 -- 172.21.15.43:0/4224966765 shutdown_connections 2024-04-25T08:58:07.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.454+0000 7fa4d7309640 1 -- 172.21.15.43:0/4224966765 wait complete. 2024-04-25T08:58:07.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.455+0000 7fa4d7309640 1 Processor -- start 2024-04-25T08:58:07.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.455+0000 7fa4d7309640 1 -- start start 2024-04-25T08:58:07.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.456+0000 7fa4d7309640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4d006d7d0 0x7fa4d0149fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:07.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.456+0000 7fa4d7309640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4d0072160 0x7fa4d006c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:07.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.456+0000 7fa4d7309640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa4d006cdd0 0x7fa4d00af450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:07.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.456+0000 7fa4d7309640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d00cde80 con 0x7fa4d0072160 2024-04-25T08:58:07.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.456+0000 7fa4d7309640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d00712b0 con 0x7fa4d006d7d0 2024-04-25T08:58:07.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.456+0000 7fa4d7309640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4d00ce000 con 0x7fa4d006cdd0 2024-04-25T08:58:07.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.456+0000 7fa4d48a4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4d0072160 0x7fa4d006c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:07.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.456+0000 7fa4d48a4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4d0072160 0x7fa4d006c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44194/0 (socket says 172.21.15.43:44194) 2024-04-25T08:58:07.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.456+0000 7fa4d48a4640 1 -- 172.21.15.43:0/3096445900 learned_addr learned my addr 172.21.15.43:0/3096445900 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:07.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.456+0000 7fa4d48a4640 1 -- 172.21.15.43:0/3096445900 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa4d006cdd0 msgr2=0x7fa4d00af450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:07.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.456+0000 7fa4d48a4640 1 --2- 172.21.15.43:0/3096445900 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa4d006cdd0 0x7fa4d00af450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:07.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.456+0000 7fa4d48a4640 1 -- 172.21.15.43:0/3096445900 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4d006d7d0 msgr2=0x7fa4d0149fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:07.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.456+0000 7fa4d48a4640 1 --2- 172.21.15.43:0/3096445900 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4d006d7d0 0x7fa4d0149fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:07.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.456+0000 7fa4d48a4640 1 -- 172.21.15.43:0/3096445900 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4d00ad8e0 con 0x7fa4d0072160 2024-04-25T08:58:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.457+0000 7fa4d48a4640 1 --2- 172.21.15.43:0/3096445900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4d0072160 0x7fa4d006c890 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fa4c400c4d0 tx=0x7fa4c400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.457+0000 7fa4cdffb640 1 -- 172.21.15.43:0/3096445900 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4018070 con 0x7fa4d0072160 2024-04-25T08:58:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.457+0000 7fa4cdffb640 1 -- 172.21.15.43:0/3096445900 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c4007500 con 0x7fa4d0072160 2024-04-25T08:58:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.457+0000 7fa4d7309640 1 -- 172.21.15.43:0/3096445900 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4d00adb80 con 0x7fa4d0072160 2024-04-25T08:58:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.457+0000 7fa4cdffb640 1 -- 172.21.15.43:0/3096445900 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4004a20 con 0x7fa4d0072160 2024-04-25T08:58:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.457+0000 7fa4d7309640 1 -- 172.21.15.43:0/3096445900 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4d00ae140 con 0x7fa4d0072160 2024-04-25T08:58:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.458+0000 7fa4d7309640 1 -- 172.21.15.43:0/3096445900 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4d0074ef0 con 0x7fa4d0072160 2024-04-25T08:58:07.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.458+0000 7fa4cdffb640 1 -- 172.21.15.43:0/3096445900 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa4c401b070 con 0x7fa4d0072160 2024-04-25T08:58:07.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.459+0000 7fa4cdffb640 1 --2- 172.21.15.43:0/3096445900 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa4bc06b500 0x7fa4bc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:07.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.459+0000 7fa4cdffb640 1 -- 172.21.15.43:0/3096445900 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa4c4089500 con 0x7fa4d0072160 2024-04-25T08:58:07.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.459+0000 7fa4d50a5640 1 --2- 172.21.15.43:0/3096445900 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa4bc06b500 0x7fa4bc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:07.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.460+0000 7fa4d50a5640 1 --2- 172.21.15.43:0/3096445900 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa4bc06b500 0x7fa4bc06d9c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8034000 tx=0x7fa4c802c000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:07.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.463+0000 7fa4cdffb640 1 -- 172.21.15.43:0/3096445900 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa4c4058380 con 0x7fa4d0072160 2024-04-25T08:58:07.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.656+0000 7fa4d7309640 1 -- 172.21.15.43:0/3096445900 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa4d00b0970 con 0x7fa4d0072160 2024-04-25T08:58:07.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.656+0000 7fa4cdffb640 1 -- 172.21.15.43:0/3096445900 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v58) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa4c4057c80 con 0x7fa4d0072160 2024-04-25T08:58:07.645 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-04-25T08:58:07.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.658+0000 7fa4d7309640 1 -- 172.21.15.43:0/3096445900 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa4bc06b500 msgr2=0x7fa4bc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:07.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.658+0000 7fa4d7309640 1 --2- 172.21.15.43:0/3096445900 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa4bc06b500 0x7fa4bc06d9c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa4c8034000 tx=0x7fa4c802c000 comp rx=0 tx=0).stop 2024-04-25T08:58:07.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.658+0000 7fa4d7309640 1 -- 172.21.15.43:0/3096445900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4d0072160 msgr2=0x7fa4d006c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:07.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.658+0000 7fa4d7309640 1 --2- 172.21.15.43:0/3096445900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4d0072160 0x7fa4d006c890 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fa4c400c4d0 tx=0x7fa4c400c990 comp rx=0 tx=0).stop 2024-04-25T08:58:07.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.658+0000 7fa4d7309640 1 -- 172.21.15.43:0/3096445900 shutdown_connections 2024-04-25T08:58:07.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.658+0000 7fa4d7309640 1 --2- 172.21.15.43:0/3096445900 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa4d006cdd0 0x7fa4d00af450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:07.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.658+0000 7fa4d7309640 1 --2- 172.21.15.43:0/3096445900 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4d0072160 0x7fa4d006c890 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:07.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.658+0000 7fa4d7309640 1 --2- 172.21.15.43:0/3096445900 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa4bc06b500 0x7fa4bc06d9c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:07.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.658+0000 7fa4d7309640 1 --2- 172.21.15.43:0/3096445900 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4d006d7d0 0x7fa4d0149fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:07.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.658+0000 7fa4d7309640 1 -- 172.21.15.43:0/3096445900 >> 172.21.15.43:0/3096445900 conn(0x7fa4d00b1480 msgr2=0x7fa4d006c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:07.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.658+0000 7fa4d7309640 1 -- 172.21.15.43:0/3096445900 shutdown_connections 2024-04-25T08:58:07.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.658+0000 7fa4d7309640 1 -- 172.21.15.43:0/3096445900 wait complete. 2024-04-25T08:58:07.658 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:58:07.658 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:58:07.674 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:58:07.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.826+0000 7fbe918db640 1 Processor -- start 2024-04-25T08:58:07.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.826+0000 7fbe918db640 1 -- start start 2024-04-25T08:58:07.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.827+0000 7fbe918db640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbe8c06d9a0 0x7fbe8c071a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:07.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.827+0000 7fbe918db640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbe8c071f90 0x7fbe8c0cd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:07.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.827+0000 7fbe918db640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe8c072420 con 0x7fbe8c15a9d0 2024-04-25T08:58:07.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.827+0000 7fbe918db640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe8c0704f0 con 0x7fbe8c06d9a0 2024-04-25T08:58:07.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.827+0000 7fbe918db640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8c0707d0 con 0x7fbe8c071f90 2024-04-25T08:58:07.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.827+0000 7fbe8a7fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbe8c06d9a0 0x7fbe8c071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:07.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.827+0000 7fbe8a7fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbe8c06d9a0 0x7fbe8c071a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44202/0 (socket says 172.21.15.43:44202) 2024-04-25T08:58:07.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.827+0000 7fbe8a7fc640 1 -- 172.21.15.43:0/1731354503 learned_addr learned my addr 172.21.15.43:0/1731354503 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:07.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.827+0000 7fbe8b7fe640 1 --2- 172.21.15.43:0/1731354503 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbe8c071f90 0x7fbe8c0cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:07.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.827+0000 7fbe8a7fc640 1 -- 172.21.15.43:0/1731354503 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbe8c071f90 msgr2=0x7fbe8c0cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:07.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.827+0000 7fbe8a7fc640 1 --2- 172.21.15.43:0/1731354503 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbe8c071f90 0x7fbe8c0cd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:07.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.827+0000 7fbe8a7fc640 1 -- 172.21.15.43:0/1731354503 >> v1:172.21.15.193:6789/0 conn(0x7fbe8c15a9d0 legacy=0x7fbe8c06d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:07.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.827+0000 7fbe8a7fc640 1 -- 172.21.15.43:0/1731354503 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe8c070ab0 con 0x7fbe8c06d9a0 2024-04-25T08:58:07.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.828+0000 7fbe8a7fc640 1 --2- 172.21.15.43:0/1731354503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe8c06d9a0 0x7fbe8c071a50 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fbe74009870 tx=0x7fbe7402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec426991473b7b64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:07.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.828+0000 7fbe89ffb640 1 -- 172.21.15.43:0/1731354503 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe74037070 con 0x7fbe8c06d9a0 2024-04-25T08:58:07.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.828+0000 7fbe89ffb640 1 -- 172.21.15.43:0/1731354503 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe74032440 con 0x7fbe8c06d9a0 2024-04-25T08:58:07.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.828+0000 7fbe89ffb640 1 -- 172.21.15.43:0/1731354503 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe74032810 con 0x7fbe8c06d9a0 2024-04-25T08:58:07.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.829+0000 7fbe918db640 1 -- 172.21.15.43:0/1731354503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe8c06d9a0 msgr2=0x7fbe8c071a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:07.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.829+0000 7fbe918db640 1 --2- 172.21.15.43:0/1731354503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe8c06d9a0 0x7fbe8c071a50 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fbe74009870 tx=0x7fbe7402a210 comp rx=0 tx=0).stop 2024-04-25T08:58:07.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.829+0000 7fbe918db640 1 -- 172.21.15.43:0/1731354503 shutdown_connections 2024-04-25T08:58:07.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.829+0000 7fbe918db640 1 --2- 172.21.15.43:0/1731354503 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbe8c071f90 0x7fbe8c0cd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:07.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.829+0000 7fbe918db640 1 --2- 172.21.15.43:0/1731354503 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe8c06d9a0 0x7fbe8c071a50 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:07.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.829+0000 7fbe918db640 1 -- 172.21.15.43:0/1731354503 >> 172.21.15.43:0/1731354503 conn(0x7fbe8c0b1480 msgr2=0x7fbe8c0b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:07.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.829+0000 7fbe918db640 1 -- 172.21.15.43:0/1731354503 shutdown_connections 2024-04-25T08:58:07.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.829+0000 7fbe918db640 1 -- 172.21.15.43:0/1731354503 wait complete. 2024-04-25T08:58:07.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.830+0000 7fbe918db640 1 Processor -- start 2024-04-25T08:58:07.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.830+0000 7fbe918db640 1 -- start start 2024-04-25T08:58:07.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.831+0000 7fbe918db640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbe8c06d9a0 0x7fbe8c154f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:07.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.831+0000 7fbe918db640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe8c071f90 0x7fbe8c1f2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:07.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.831+0000 7fbe918db640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe8c0c6230 0x7fbe8c0c66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:07.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.831+0000 7fbe918db640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8c0cde80 con 0x7fbe8c0c6230 2024-04-25T08:58:07.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.831+0000 7fbe918db640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8c0712b0 con 0x7fbe8c071f90 2024-04-25T08:58:07.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.831+0000 7fbe918db640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8c0ce000 con 0x7fbe8c06d9a0 2024-04-25T08:58:07.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.831+0000 7fbe8b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe8c0c6230 0x7fbe8c0c66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:07.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.831+0000 7fbe8b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe8c0c6230 0x7fbe8c0c66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44206/0 (socket says 172.21.15.43:44206) 2024-04-25T08:58:07.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.831+0000 7fbe8b7fe640 1 -- 172.21.15.43:0/1731560432 learned_addr learned my addr 172.21.15.43:0/1731560432 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:07.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.831+0000 7fbe8affd640 1 --2- 172.21.15.43:0/1731560432 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbe8c06d9a0 0x7fbe8c154f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:07.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.831+0000 7fbe8a7fc640 1 --2- 172.21.15.43:0/1731560432 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe8c071f90 0x7fbe8c1f2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:07.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.831+0000 7fbe8b7fe640 1 -- 172.21.15.43:0/1731560432 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbe8c06d9a0 msgr2=0x7fbe8c154f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:07.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.831+0000 7fbe8b7fe640 1 --2- 172.21.15.43:0/1731560432 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbe8c06d9a0 0x7fbe8c154f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:07.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.831+0000 7fbe8b7fe640 1 -- 172.21.15.43:0/1731560432 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe8c071f90 msgr2=0x7fbe8c1f2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:07.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.831+0000 7fbe8b7fe640 1 --2- 172.21.15.43:0/1731560432 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe8c071f90 0x7fbe8c1f2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:07.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.831+0000 7fbe8b7fe640 1 -- 172.21.15.43:0/1731560432 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe8c1f29a0 con 0x7fbe8c0c6230 2024-04-25T08:58:07.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.831+0000 7fbe8b7fe640 1 --2- 172.21.15.43:0/1731560432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe8c0c6230 0x7fbe8c0c66c0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c007dc0 tx=0x7fbe7c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:07.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.832+0000 7fbe83fff640 1 -- 172.21.15.43:0/1731560432 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c014420 con 0x7fbe8c0c6230 2024-04-25T08:58:07.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.832+0000 7fbe83fff640 1 -- 172.21.15.43:0/1731560432 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe7c019070 con 0x7fbe8c0c6230 2024-04-25T08:58:07.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.832+0000 7fbe918db640 1 -- 172.21.15.43:0/1731560432 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe8c0c4a70 con 0x7fbe8c0c6230 2024-04-25T08:58:07.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.832+0000 7fbe83fff640 1 -- 172.21.15.43:0/1731560432 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c014820 con 0x7fbe8c0c6230 2024-04-25T08:58:07.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.832+0000 7fbe918db640 1 -- 172.21.15.43:0/1731560432 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe8c0c5030 con 0x7fbe8c0c6230 2024-04-25T08:58:07.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.833+0000 7fbe918db640 1 -- 172.21.15.43:0/1731560432 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe8c067d30 con 0x7fbe8c0c6230 2024-04-25T08:58:07.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.833+0000 7fbe83fff640 1 -- 172.21.15.43:0/1731560432 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbe7c0149c0 con 0x7fbe8c0c6230 2024-04-25T08:58:07.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.833+0000 7fbe83fff640 1 --2- 172.21.15.43:0/1731560432 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbe6c06b4d0 0x7fbe6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:07.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.833+0000 7fbe83fff640 1 -- 172.21.15.43:0/1731560432 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbe7c08a8d0 con 0x7fbe8c0c6230 2024-04-25T08:58:07.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.834+0000 7fbe8affd640 1 --2- 172.21.15.43:0/1731560432 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbe6c06b4d0 0x7fbe6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:07.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.834+0000 7fbe8affd640 1 --2- 172.21.15.43:0/1731560432 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbe6c06b4d0 0x7fbe6c06d990 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fbe84006fd0 tx=0x7fbe84009060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:07.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:07.837+0000 7fbe83fff640 1 -- 172.21.15.43:0/1731560432 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbe8c067d30 con 0x7fbe8c0c6230 2024-04-25T08:58:08.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.021+0000 7fbe918db640 1 -- 172.21.15.43:0/1731560432 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe8c074ef0 con 0x7fbe8c0c6230 2024-04-25T08:58:08.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.022+0000 7fbe83fff640 1 -- 172.21.15.43:0/1731560432 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+16629 (secure 0 0 0) 0x7fbe7c059100 con 0x7fbe8c0c6230 2024-04-25T08:58:08.012 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:58:08.012 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":58,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:57:55.889262+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:57:52.884445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T08:57:53.565716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:58:08.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.024+0000 7fbe918db640 1 -- 172.21.15.43:0/1731560432 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbe6c06b4d0 msgr2=0x7fbe6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:08.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.024+0000 7fbe918db640 1 --2- 172.21.15.43:0/1731560432 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbe6c06b4d0 0x7fbe6c06d990 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fbe84006fd0 tx=0x7fbe84009060 comp rx=0 tx=0).stop 2024-04-25T08:58:08.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.024+0000 7fbe918db640 1 -- 172.21.15.43:0/1731560432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe8c0c6230 msgr2=0x7fbe8c0c66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:08.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.024+0000 7fbe918db640 1 --2- 172.21.15.43:0/1731560432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe8c0c6230 0x7fbe8c0c66c0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c007dc0 tx=0x7fbe7c00e4b0 comp rx=0 tx=0).stop 2024-04-25T08:58:08.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.024+0000 7fbe918db640 1 -- 172.21.15.43:0/1731560432 shutdown_connections 2024-04-25T08:58:08.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.024+0000 7fbe918db640 1 --2- 172.21.15.43:0/1731560432 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe8c0c6230 0x7fbe8c0c66c0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:08.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.024+0000 7fbe918db640 1 --2- 172.21.15.43:0/1731560432 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbe6c06b4d0 0x7fbe6c06d990 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:08.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.024+0000 7fbe918db640 1 --2- 172.21.15.43:0/1731560432 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe8c071f90 0x7fbe8c1f2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:08.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.024+0000 7fbe918db640 1 --2- 172.21.15.43:0/1731560432 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbe8c06d9a0 0x7fbe8c154f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:08.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.024+0000 7fbe918db640 1 -- 172.21.15.43:0/1731560432 >> 172.21.15.43:0/1731560432 conn(0x7fbe8c0b1480 msgr2=0x7fbe8c15d410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:08.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.024+0000 7fbe918db640 1 -- 172.21.15.43:0/1731560432 shutdown_connections 2024-04-25T08:58:08.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.024+0000 7fbe918db640 1 -- 172.21.15.43:0/1731560432 wait complete. 2024-04-25T08:58:08.025 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:57:52.884445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:57:53.565716+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:58:08.025 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:58:08.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.139+0000 7f83f983a640 1 Processor -- start 2024-04-25T08:58:08.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.139+0000 7f83f983a640 1 -- start start 2024-04-25T08:58:08.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.140+0000 7f83f983a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f83f40d1ec0 0x7f83f40cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:08.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.140+0000 7f83f983a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f415f170 con 0x7f83f40b0cc0 2024-04-25T08:58:08.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.140+0000 7f83f983a640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f40b2fe0 con 0x7f83f415dd80 2024-04-25T08:58:08.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.140+0000 7f83f983a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83f40cfc60 con 0x7f83f40d1ec0 2024-04-25T08:58:08.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.141+0000 7f83f37fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f83f40b0cc0 0x7f83f40b1130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35742/0 (socket says 172.21.15.43:35742) 2024-04-25T08:58:08.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.141+0000 7f83f27fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f83f40d1ec0 0x7f83f40cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:08.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.141+0000 7f83f37fe640 1 -- 172.21.15.43:0/355070074 learned_addr learned my addr 172.21.15.43:0/355070074 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:08.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.141+0000 7f83f27fc640 1 -- 172.21.15.43:0/355070074 >> v1:172.21.15.193:6789/0 conn(0x7f83f415dd80 legacy=0x7f83f40d17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:08.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.141+0000 7f83f27fc640 1 -- 172.21.15.43:0/355070074 >> v1:172.21.15.43:6789/0 conn(0x7f83f40b0cc0 legacy=0x7f83f40b1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:08.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.141+0000 7f83f1ffb640 1 -- 172.21.15.43:0/355070074 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1014782186 0 0) 0x7f83f415f170 con 0x7f83f40b0cc0 2024-04-25T08:58:08.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.141+0000 7f83f27fc640 1 -- 172.21.15.43:0/355070074 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f40ced70 con 0x7f83f40d1ec0 2024-04-25T08:58:08.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.142+0000 7f83f27fc640 1 --2- 172.21.15.43:0/355070074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83f40d1ec0 0x7f83f40cf720 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f83e8009720 tx=0x7f83e802a030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6917314c3eefd802 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:08.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.142+0000 7f83f1ffb640 1 -- 172.21.15.43:0/355070074 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e8037070 con 0x7f83f40d1ec0 2024-04-25T08:58:08.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.142+0000 7f83f1ffb640 1 -- 172.21.15.43:0/355070074 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e802ad40 con 0x7f83f40d1ec0 2024-04-25T08:58:08.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.142+0000 7f83f1ffb640 1 -- 172.21.15.43:0/355070074 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e8032660 con 0x7f83f40d1ec0 2024-04-25T08:58:08.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.142+0000 7f83f983a640 1 -- 172.21.15.43:0/355070074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83f40d1ec0 msgr2=0x7f83f40cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:08.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.142+0000 7f83f983a640 1 --2- 172.21.15.43:0/355070074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83f40d1ec0 0x7f83f40cf720 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f83e8009720 tx=0x7f83e802a030 comp rx=0 tx=0).stop 2024-04-25T08:58:08.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.143+0000 7f83f983a640 1 -- 172.21.15.43:0/355070074 shutdown_connections 2024-04-25T08:58:08.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.143+0000 7f83f983a640 1 --2- 172.21.15.43:0/355070074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83f40d1ec0 0x7f83f40cf720 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:08.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.143+0000 7f83f983a640 1 -- 172.21.15.43:0/355070074 >> 172.21.15.43:0/355070074 conn(0x7f83f4072aa0 msgr2=0x7f83f409aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:08.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.143+0000 7f83f983a640 1 -- 172.21.15.43:0/355070074 shutdown_connections 2024-04-25T08:58:08.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.143+0000 7f83f983a640 1 -- 172.21.15.43:0/355070074 wait complete. 2024-04-25T08:58:08.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.144+0000 7f83f983a640 1 Processor -- start 2024-04-25T08:58:08.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.144+0000 7f83f983a640 1 -- start start 2024-04-25T08:58:08.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.145+0000 7f83f983a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83f40b0cc0 0x7f83f40b60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:08.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.145+0000 7f83f983a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f83f40d1ec0 0x7f83f414d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:08.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.145+0000 7f83f983a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83f415dd80 0x7f83f414eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:08.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.145+0000 7f83f983a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f40cf330 con 0x7f83f40b0cc0 2024-04-25T08:58:08.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.145+0000 7f83f983a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f40b32c0 con 0x7f83f415dd80 2024-04-25T08:58:08.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.145+0000 7f83f983a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f40b2c40 con 0x7f83f40d1ec0 2024-04-25T08:58:08.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.145+0000 7f83f2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83f40b0cc0 0x7f83f40b60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:08.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.145+0000 7f83f2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83f40b0cc0 0x7f83f40b60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44230/0 (socket says 172.21.15.43:44230) 2024-04-25T08:58:08.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.145+0000 7f83f2ffd640 1 -- 172.21.15.43:0/1811864390 learned_addr learned my addr 172.21.15.43:0/1811864390 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:08.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.145+0000 7f83f37fe640 1 --2- 172.21.15.43:0/1811864390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83f415dd80 0x7f83f414eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:08.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.145+0000 7f83f27fc640 1 --2- 172.21.15.43:0/1811864390 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f83f40d1ec0 0x7f83f414d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:08.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.145+0000 7f83f2ffd640 1 -- 172.21.15.43:0/1811864390 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f83f40d1ec0 msgr2=0x7f83f414d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:08.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.145+0000 7f83f2ffd640 1 --2- 172.21.15.43:0/1811864390 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f83f40d1ec0 0x7f83f414d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:08.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.145+0000 7f83f2ffd640 1 -- 172.21.15.43:0/1811864390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83f415dd80 msgr2=0x7f83f414eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:08.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.145+0000 7f83f2ffd640 1 --2- 172.21.15.43:0/1811864390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83f415dd80 0x7f83f414eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:08.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.145+0000 7f83f2ffd640 1 -- 172.21.15.43:0/1811864390 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f414da60 con 0x7f83f40b0cc0 2024-04-25T08:58:08.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.145+0000 7f83f2ffd640 1 --2- 172.21.15.43:0/1811864390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83f40b0cc0 0x7f83f40b60f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f83ec00c550 tx=0x7f83ec00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:08.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.146+0000 7f83dbfff640 1 -- 172.21.15.43:0/1811864390 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec005480 con 0x7f83f40b0cc0 2024-04-25T08:58:08.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.146+0000 7f83dbfff640 1 -- 172.21.15.43:0/1811864390 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83ec0035f0 con 0x7f83f40b0cc0 2024-04-25T08:58:08.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.146+0000 7f83f983a640 1 -- 172.21.15.43:0/1811864390 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83f414dd40 con 0x7f83f40b0cc0 2024-04-25T08:58:08.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.146+0000 7f83dbfff640 1 -- 172.21.15.43:0/1811864390 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec0039c0 con 0x7f83f40b0cc0 2024-04-25T08:58:08.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.146+0000 7f83f983a640 1 -- 172.21.15.43:0/1811864390 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83f414e300 con 0x7f83f40b0cc0 2024-04-25T08:58:08.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.147+0000 7f83f983a640 1 -- 172.21.15.43:0/1811864390 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83f4074f60 con 0x7f83f40b0cc0 2024-04-25T08:58:08.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.147+0000 7f83dbfff640 1 -- 172.21.15.43:0/1811864390 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f83ec003b60 con 0x7f83f40b0cc0 2024-04-25T08:58:08.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.148+0000 7f83dbfff640 1 --2- 172.21.15.43:0/1811864390 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f83e006b670 0x7f83e006db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:08.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.148+0000 7f83dbfff640 1 -- 172.21.15.43:0/1811864390 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f83ec089ec0 con 0x7f83f40b0cc0 2024-04-25T08:58:08.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.148+0000 7f83f27fc640 1 --2- 172.21.15.43:0/1811864390 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f83e006b670 0x7f83e006db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:08.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.148+0000 7f83f27fc640 1 --2- 172.21.15.43:0/1811864390 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f83e006b670 0x7f83e006db30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f83e8034000 tx=0x7f83e802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:08.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.152+0000 7f83dbfff640 1 -- 172.21.15.43:0/1811864390 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f83ec058650 con 0x7f83f40b0cc0 2024-04-25T08:58:08.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.330+0000 7f83f983a640 1 -- 172.21.15.43:0/1811864390 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f83f40cdfb0 con 0x7f83f40b0cc0 2024-04-25T08:58:08.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.331+0000 7f83dbfff640 1 -- 172.21.15.43:0/1811864390 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7f83ec05c6f0 con 0x7f83f40b0cc0 2024-04-25T08:58:08.320 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T08:58:08.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.333+0000 7f83f983a640 1 -- 172.21.15.43:0/1811864390 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f83e006b670 msgr2=0x7f83e006db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:08.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.333+0000 7f83f983a640 1 --2- 172.21.15.43:0/1811864390 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f83e006b670 0x7f83e006db30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f83e8034000 tx=0x7f83e802e000 comp rx=0 tx=0).stop 2024-04-25T08:58:08.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.333+0000 7f83f983a640 1 -- 172.21.15.43:0/1811864390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83f40b0cc0 msgr2=0x7f83f40b60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:08.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.333+0000 7f83f983a640 1 --2- 172.21.15.43:0/1811864390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83f40b0cc0 0x7f83f40b60f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f83ec00c550 tx=0x7f83ec00ca10 comp rx=0 tx=0).stop 2024-04-25T08:58:08.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.333+0000 7f83f983a640 1 -- 172.21.15.43:0/1811864390 shutdown_connections 2024-04-25T08:58:08.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.333+0000 7f83f983a640 1 --2- 172.21.15.43:0/1811864390 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f83e006b670 0x7f83e006db30 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:08.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.333+0000 7f83f983a640 1 --2- 172.21.15.43:0/1811864390 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83f415dd80 0x7f83f414eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:08.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.333+0000 7f83f983a640 1 --2- 172.21.15.43:0/1811864390 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f83f40d1ec0 0x7f83f414d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:08.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.333+0000 7f83f983a640 1 --2- 172.21.15.43:0/1811864390 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83f40b0cc0 0x7f83f40b60f0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:08.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.333+0000 7f83f983a640 1 -- 172.21.15.43:0/1811864390 >> 172.21.15.43:0/1811864390 conn(0x7f83f4072aa0 msgr2=0x7f83f40d17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:08.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.333+0000 7f83f983a640 1 -- 172.21.15.43:0/1811864390 shutdown_connections 2024-04-25T08:58:08.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.333+0000 7f83f983a640 1 -- 172.21.15.43:0/1811864390 wait complete. 2024-04-25T08:58:08.333 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:58:08.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.450+0000 7fb509929640 1 Processor -- start 2024-04-25T08:58:08.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.450+0000 7fb509929640 1 -- start start 2024-04-25T08:58:08.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.451+0000 7fb509929640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5040d04a0 0x7fb50415ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:08.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.451+0000 7fb509929640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5040cf110 con 0x7fb5040ce8c0 2024-04-25T08:58:08.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.451+0000 7fb509929640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5040cf3f0 con 0x7fb50415dec0 2024-04-25T08:58:08.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.451+0000 7fb509929640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5040cf6d0 con 0x7fb5040d04a0 2024-04-25T08:58:08.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.451+0000 7fb5037fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb5040ce8c0 0x7fb5040d17a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35748/0 (socket says 172.21.15.43:35748) 2024-04-25T08:58:08.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.451+0000 7fb5037fe640 1 -- 172.21.15.43:0/886174755 learned_addr learned my addr 172.21.15.43:0/886174755 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:08.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.451+0000 7fb5027fc640 1 -- 172.21.15.43:0/886174755 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 943065988 0 0) 0x7fb5040cf110 con 0x7fb5040ce8c0 2024-04-25T08:58:08.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.451+0000 7fb502ffd640 1 --2- 172.21.15.43:0/886174755 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5040d04a0 0x7fb50415ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:08.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.451+0000 7fb5027fc640 1 -- 172.21.15.43:0/886174755 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4f0003520 con 0x7fb5040ce8c0 2024-04-25T08:58:08.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.451+0000 7fb5027fc640 1 -- 172.21.15.43:0/886174755 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4286509168 0 0) 0x7fb4f0003520 con 0x7fb5040ce8c0 2024-04-25T08:58:08.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.451+0000 7fb5027fc640 1 -- 172.21.15.43:0/886174755 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5040d04a0 msgr2=0x7fb50415ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:08.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.451+0000 7fb5027fc640 1 --2- 172.21.15.43:0/886174755 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5040d04a0 0x7fb50415ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:08.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.451+0000 7fb5027fc640 1 -- 172.21.15.43:0/886174755 >> v1:172.21.15.193:6789/0 conn(0x7fb50415dec0 legacy=0x7fb5040d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:08.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.451+0000 7fb5027fc640 1 -- 172.21.15.43:0/886174755 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5040cf9b0 con 0x7fb5040ce8c0 2024-04-25T08:58:08.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.451+0000 7fb5027fc640 1 -- 172.21.15.43:0/886174755 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7fb4fc0019f0 con 0x7fb5040ce8c0 2024-04-25T08:58:08.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.451+0000 7fb5027fc640 1 -- 172.21.15.43:0/886174755 >> v1:172.21.15.43:6789/0 conn(0x7fb5040ce8c0 legacy=0x7fb5040d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:08.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.451+0000 7fb5027fc640 1 --2- 172.21.15.43:0/886174755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4f00039e0 0x7fb4f0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:08.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.452+0000 7fb5027fc640 1 --2- 172.21.15.43:0/886174755 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5040d04a0 0x7fb4f0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:08.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.452+0000 7fb5027fc640 1 --2- 172.21.15.43:0/886174755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4f0004d40 0x7fb4f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:08.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.452+0000 7fb5027fc640 1 -- 172.21.15.43:0/886174755 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5040cf6d0 con 0x7fb4f0004d40 2024-04-25T08:58:08.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.452+0000 7fb5027fc640 1 -- 172.21.15.43:0/886174755 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f000ba70 con 0x7fb4f00039e0 2024-04-25T08:58:08.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.452+0000 7fb5027fc640 1 -- 172.21.15.43:0/886174755 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f000bc70 con 0x7fb5040d04a0 2024-04-25T08:58:08.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.452+0000 7fb5037fe640 1 --2- 172.21.15.43:0/886174755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4f0004d40 0x7fb4f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:08.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.452+0000 7fb502ffd640 1 --2- 172.21.15.43:0/886174755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4f00039e0 0x7fb4f0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:08.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.452+0000 7fb4fbfff640 1 --2- 172.21.15.43:0/886174755 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5040d04a0 0x7fb4f0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:08.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.452+0000 7fb5037fe640 1 -- 172.21.15.43:0/886174755 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5040d04a0 msgr2=0x7fb4f0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:08.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.452+0000 7fb5037fe640 1 --2- 172.21.15.43:0/886174755 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5040d04a0 0x7fb4f0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:08.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.452+0000 7fb5037fe640 1 -- 172.21.15.43:0/886174755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4f00039e0 msgr2=0x7fb4f0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:08.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.452+0000 7fb5037fe640 1 --2- 172.21.15.43:0/886174755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb4f00039e0 0x7fb4f0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:08.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.452+0000 7fb5037fe640 1 -- 172.21.15.43:0/886174755 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f0003520 con 0x7fb4f0004d40 2024-04-25T08:58:08.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.453+0000 7fb5037fe640 1 --2- 172.21.15.43:0/886174755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4f0004d40 0x7fb4f00051c0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc002230 tx=0x7fb4f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=807b31f72e3025bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:08.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.453+0000 7fb5027fc640 1 -- 172.21.15.43:0/886174755 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc00a4a0 con 0x7fb4f0004d40 2024-04-25T08:58:08.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.453+0000 7fb5027fc640 1 -- 172.21.15.43:0/886174755 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4fc02f070 con 0x7fb4f0004d40 2024-04-25T08:58:08.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.453+0000 7fb5027fc640 1 -- 172.21.15.43:0/886174755 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4fc02bcb0 con 0x7fb4f0004d40 2024-04-25T08:58:08.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.453+0000 7fb509929640 1 -- 172.21.15.43:0/886174755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4f0004d40 msgr2=0x7fb4f00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:08.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.453+0000 7fb509929640 1 --2- 172.21.15.43:0/886174755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb4f0004d40 0x7fb4f00051c0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fb4fc002230 tx=0x7fb4f0005830 comp rx=0 tx=0).stop 2024-04-25T08:58:08.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.453+0000 7fb5037fe640 1 -- 172.21.15.43:0/886174755 reap_dead start 2024-04-25T08:58:08.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.453+0000 7fb509929640 1 -- 172.21.15.43:0/886174755 shutdown_connections 2024-04-25T08:58:08.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.453+0000 7fb509929640 1 -- 172.21.15.43:0/886174755 >> 172.21.15.43:0/886174755 conn(0x7fb50409cd40 msgr2=0x7fb5040d3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:08.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.453+0000 7fb509929640 1 -- 172.21.15.43:0/886174755 shutdown_connections 2024-04-25T08:58:08.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.453+0000 7fb509929640 1 -- 172.21.15.43:0/886174755 wait complete. 2024-04-25T08:58:08.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.455+0000 7fb509929640 1 Processor -- start 2024-04-25T08:58:08.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.455+0000 7fb509929640 1 -- start start 2024-04-25T08:58:08.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.455+0000 7fb509929640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb4f0004d40 0x7fb5041f6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:08.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.455+0000 7fb509929640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5041610d0 0x7fb50409d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:08.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.455+0000 7fb509929640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb504161550 0x7fb50415c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:08.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.455+0000 7fb509929640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5040cee90 con 0x7fb5041610d0 2024-04-25T08:58:08.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.455+0000 7fb509929640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5040ced10 con 0x7fb504161550 2024-04-25T08:58:08.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.455+0000 7fb509929640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb50415ed90 con 0x7fb4f0004d40 2024-04-25T08:58:08.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.455+0000 7fb4fbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5041610d0 0x7fb50409d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:08.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.455+0000 7fb4fbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5041610d0 0x7fb50409d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44250/0 (socket says 172.21.15.43:44250) 2024-04-25T08:58:08.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.455+0000 7fb4fbfff640 1 -- 172.21.15.43:0/2016057227 learned_addr learned my addr 172.21.15.43:0/2016057227 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:08.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.456+0000 7fb5037fe640 1 --2- 172.21.15.43:0/2016057227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb504161550 0x7fb50415c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:08.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.456+0000 7fb502ffd640 1 --2- 172.21.15.43:0/2016057227 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb4f0004d40 0x7fb5041f6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:08.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.456+0000 7fb4fbfff640 1 -- 172.21.15.43:0/2016057227 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb4f0004d40 msgr2=0x7fb5041f6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:08.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.456+0000 7fb4fbfff640 1 --2- 172.21.15.43:0/2016057227 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb4f0004d40 0x7fb5041f6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:08.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.456+0000 7fb4fbfff640 1 -- 172.21.15.43:0/2016057227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb504161550 msgr2=0x7fb50415c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:08.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.456+0000 7fb4fbfff640 1 --2- 172.21.15.43:0/2016057227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb504161550 0x7fb50415c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:08.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.456+0000 7fb4fbfff640 1 -- 172.21.15.43:0/2016057227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5040a0a50 con 0x7fb5041610d0 2024-04-25T08:58:08.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.456+0000 7fb5037fe640 1 --2- 172.21.15.43:0/2016057227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb504161550 0x7fb50415c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:58:08.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.456+0000 7fb4fbfff640 1 --2- 172.21.15.43:0/2016057227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5041610d0 0x7fb50409d7b0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec00f820 tx=0x7fb4ec00fce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:08.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.456+0000 7fb500ff9640 1 -- 172.21.15.43:0/2016057227 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec0050d0 con 0x7fb5041610d0 2024-04-25T08:58:08.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.456+0000 7fb500ff9640 1 -- 172.21.15.43:0/2016057227 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4ec018070 con 0x7fb5041610d0 2024-04-25T08:58:08.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.456+0000 7fb509929640 1 -- 172.21.15.43:0/2016057227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5040a0cd0 con 0x7fb5041610d0 2024-04-25T08:58:08.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.456+0000 7fb500ff9640 1 -- 172.21.15.43:0/2016057227 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4ec005490 con 0x7fb5041610d0 2024-04-25T08:58:08.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.456+0000 7fb509929640 1 -- 172.21.15.43:0/2016057227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5040a1250 con 0x7fb5041610d0 2024-04-25T08:58:08.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.457+0000 7fb509929640 1 -- 172.21.15.43:0/2016057227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb504074f60 con 0x7fb5041610d0 2024-04-25T08:58:08.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.458+0000 7fb500ff9640 1 -- 172.21.15.43:0/2016057227 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb4ec00d730 con 0x7fb5041610d0 2024-04-25T08:58:08.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.458+0000 7fb500ff9640 1 --2- 172.21.15.43:0/2016057227 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb4e406b500 0x7fb4e406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:08.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.458+0000 7fb500ff9640 1 -- 172.21.15.43:0/2016057227 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb4ec08a080 con 0x7fb5041610d0 2024-04-25T08:58:08.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.458+0000 7fb502ffd640 1 --2- 172.21.15.43:0/2016057227 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb4e406b500 0x7fb4e406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:08.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.459+0000 7fb502ffd640 1 --2- 172.21.15.43:0/2016057227 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb4e406b500 0x7fb4e406d9c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4009b00 tx=0x7fb4f4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:08.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.462+0000 7fb500ff9640 1 -- 172.21.15.43:0/2016057227 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb5040a0a50 con 0x7fb5041610d0 2024-04-25T08:58:08.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.641+0000 7fb509929640 1 -- 172.21.15.43:0/2016057227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb50408daf0 con 0x7fb5041610d0 2024-04-25T08:58:08.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.641+0000 7fb500ff9640 1 -- 172.21.15.43:0/2016057227 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v58) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb4ec058f00 con 0x7fb5041610d0 2024-04-25T08:58:08.631 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T08:58:08.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.643+0000 7fb509929640 1 -- 172.21.15.43:0/2016057227 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb4e406b500 msgr2=0x7fb4e406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:08.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.643+0000 7fb509929640 1 --2- 172.21.15.43:0/2016057227 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb4e406b500 0x7fb4e406d9c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fb4f4009b00 tx=0x7fb4f4008040 comp rx=0 tx=0).stop 2024-04-25T08:58:08.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.643+0000 7fb509929640 1 -- 172.21.15.43:0/2016057227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5041610d0 msgr2=0x7fb50409d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:08.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.643+0000 7fb509929640 1 --2- 172.21.15.43:0/2016057227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5041610d0 0x7fb50409d7b0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec00f820 tx=0x7fb4ec00fce0 comp rx=0 tx=0).stop 2024-04-25T08:58:08.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.643+0000 7fb509929640 1 -- 172.21.15.43:0/2016057227 shutdown_connections 2024-04-25T08:58:08.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.643+0000 7fb509929640 1 --2- 172.21.15.43:0/2016057227 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb4e406b500 0x7fb4e406d9c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:08.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.643+0000 7fb509929640 1 --2- 172.21.15.43:0/2016057227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb504161550 0x7fb50415c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:08.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.643+0000 7fb509929640 1 --2- 172.21.15.43:0/2016057227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5041610d0 0x7fb50409d7b0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:08.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.643+0000 7fb509929640 1 --2- 172.21.15.43:0/2016057227 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb4f0004d40 0x7fb5041f6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:08.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.643+0000 7fb509929640 1 -- 172.21.15.43:0/2016057227 >> 172.21.15.43:0/2016057227 conn(0x7fb50409cd40 msgr2=0x7fb5040b2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:08.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.643+0000 7fb509929640 1 -- 172.21.15.43:0/2016057227 shutdown_connections 2024-04-25T08:58:08.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.643+0000 7fb509929640 1 -- 172.21.15.43:0/2016057227 wait complete. 2024-04-25T08:58:08.644 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:58:08.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.759+0000 7fdcd365d640 1 Processor -- start 2024-04-25T08:58:08.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.759+0000 7fdcd365d640 1 -- start start 2024-04-25T08:58:08.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.760+0000 7fdcd365d640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fdccc0d19f0 0x7fdccc0d1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:08.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.760+0000 7fdcd365d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdccc0d2370 0x7fdccc0c7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:08.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.760+0000 7fdcd365d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdccc0cf030 con 0x7fdccc161310 2024-04-25T08:58:08.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.760+0000 7fdcd365d640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdccc0d4870 con 0x7fdccc0d19f0 2024-04-25T08:58:08.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.760+0000 7fdcd365d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc0cf310 con 0x7fdccc0d2370 2024-04-25T08:58:08.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.760+0000 7fdcd13f9640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fdccc161310 0x7fdccc0d4080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35762/0 (socket says 172.21.15.43:35762) 2024-04-25T08:58:08.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.760+0000 7fdcd13f9640 1 -- 172.21.15.43:0/3263277475 learned_addr learned my addr 172.21.15.43:0/3263277475 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:08.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.760+0000 7fdcd1bfa640 1 --2- 172.21.15.43:0/3263277475 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdccc0d2370 0x7fdccc0c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:08.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.760+0000 7fdcd0bf8640 1 --2- 172.21.15.43:0/3263277475 >> v2:172.21.15.193:3300/0 conn(0x7fdccc0d19f0 0x7fdccc0d1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:08.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.760+0000 7fdcbffff640 1 -- 172.21.15.43:0/3263277475 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 644980040 0 0) 0x7fdccc0cf030 con 0x7fdccc161310 2024-04-25T08:58:08.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.760+0000 7fdcbffff640 1 -- 172.21.15.43:0/3263277475 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcb8003520 con 0x7fdccc161310 2024-04-25T08:58:08.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.761+0000 7fdcbffff640 1 -- 172.21.15.43:0/3263277475 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1289039615 0 0) 0x7fdcb8003520 con 0x7fdccc161310 2024-04-25T08:58:08.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.761+0000 7fdcd1bfa640 1 -- 172.21.15.43:0/3263277475 >> v2:172.21.15.193:3300/0 conn(0x7fdccc0d19f0 msgr2=0x7fdccc0d1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:08.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.761+0000 7fdcd1bfa640 1 --2- 172.21.15.43:0/3263277475 >> v2:172.21.15.193:3300/0 conn(0x7fdccc0d19f0 0x7fdccc0d1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:08.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.761+0000 7fdcd1bfa640 1 -- 172.21.15.43:0/3263277475 >> v1:172.21.15.43:6789/0 conn(0x7fdccc161310 legacy=0x7fdccc0d4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:08.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.761+0000 7fdcd1bfa640 1 -- 172.21.15.43:0/3263277475 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdccc0cf5f0 con 0x7fdccc0d2370 2024-04-25T08:58:08.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.761+0000 7fdcd0bf8640 1 --2- 172.21.15.43:0/3263277475 >> v2:172.21.15.193:3300/0 conn(0x7fdccc0d19f0 0x7fdccc0d1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:58:08.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.762+0000 7fdcd1bfa640 1 --2- 172.21.15.43:0/3263277475 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdccc0d2370 0x7fdccc0c7860 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4009d90 tx=0x7fdcc402a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3075292575ecca58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:08.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.762+0000 7fdcbffff640 1 -- 172.21.15.43:0/3263277475 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc0001670 con 0x7fdccc0d2370 2024-04-25T08:58:08.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.762+0000 7fdcbffff640 1 -- 172.21.15.43:0/3263277475 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc4007540 con 0x7fdccc0d2370 2024-04-25T08:58:08.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.762+0000 7fdcbffff640 1 -- 172.21.15.43:0/3263277475 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc40078f0 con 0x7fdccc0d2370 2024-04-25T08:58:08.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.762+0000 7fdcd365d640 1 -- 172.21.15.43:0/3263277475 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdccc0d2370 msgr2=0x7fdccc0c7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:08.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.762+0000 7fdcd365d640 1 --2- 172.21.15.43:0/3263277475 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdccc0d2370 0x7fdccc0c7860 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4009d90 tx=0x7fdcc402a5f0 comp rx=0 tx=0).stop 2024-04-25T08:58:08.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.762+0000 7fdcd365d640 1 -- 172.21.15.43:0/3263277475 shutdown_connections 2024-04-25T08:58:08.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.762+0000 7fdcd365d640 1 --2- 172.21.15.43:0/3263277475 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdccc0d2370 0x7fdccc0c7860 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:08.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.762+0000 7fdcd365d640 1 --2- 172.21.15.43:0/3263277475 >> v2:172.21.15.193:3300/0 conn(0x7fdccc0d19f0 0x7fdccc0d1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:08.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.762+0000 7fdcd365d640 1 -- 172.21.15.43:0/3263277475 >> 172.21.15.43:0/3263277475 conn(0x7fdccc09cd20 msgr2=0x7fdccc0d3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:08.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.762+0000 7fdcd365d640 1 -- 172.21.15.43:0/3263277475 shutdown_connections 2024-04-25T08:58:08.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.762+0000 7fdcd365d640 1 -- 172.21.15.43:0/3263277475 wait complete. 2024-04-25T08:58:08.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.764+0000 7fdcd365d640 1 Processor -- start 2024-04-25T08:58:08.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.764+0000 7fdcd365d640 1 -- start start 2024-04-25T08:58:08.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.764+0000 7fdcd365d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdccc0d19f0 0x7fdccc0cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:08.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.764+0000 7fdcd365d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdccc0d2370 0x7fdccc15d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:08.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.764+0000 7fdcd365d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdccc161310 0x7fdccc0cbe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:08.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.764+0000 7fdcd365d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc0c7da0 con 0x7fdccc0d2370 2024-04-25T08:58:08.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.764+0000 7fdcd365d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc0d2800 con 0x7fdccc161310 2024-04-25T08:58:08.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.764+0000 7fdcd365d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdccc0c7f20 con 0x7fdccc0d19f0 2024-04-25T08:58:08.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.764+0000 7fdcd0bf8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdccc0d2370 0x7fdccc15d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:08.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.764+0000 7fdcd0bf8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdccc0d2370 0x7fdccc15d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44268/0 (socket says 172.21.15.43:44268) 2024-04-25T08:58:08.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.764+0000 7fdcd0bf8640 1 -- 172.21.15.43:0/1283144985 learned_addr learned my addr 172.21.15.43:0/1283144985 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:08.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.765+0000 7fdcd13f9640 1 --2- 172.21.15.43:0/1283144985 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdccc0d19f0 0x7fdccc0cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:08.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.765+0000 7fdcd1bfa640 1 --2- 172.21.15.43:0/1283144985 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdccc161310 0x7fdccc0cbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:08.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.765+0000 7fdcd0bf8640 1 -- 172.21.15.43:0/1283144985 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdccc0d19f0 msgr2=0x7fdccc0cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:08.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.765+0000 7fdcd0bf8640 1 --2- 172.21.15.43:0/1283144985 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdccc0d19f0 0x7fdccc0cd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:08.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.765+0000 7fdcd0bf8640 1 -- 172.21.15.43:0/1283144985 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdccc161310 msgr2=0x7fdccc0cbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:08.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.765+0000 7fdcd0bf8640 1 --2- 172.21.15.43:0/1283144985 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdccc161310 0x7fdccc0cbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:08.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.765+0000 7fdcd0bf8640 1 -- 172.21.15.43:0/1283144985 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdccc0cade0 con 0x7fdccc0d2370 2024-04-25T08:58:08.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.765+0000 7fdcd0bf8640 1 --2- 172.21.15.43:0/1283144985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdccc0d2370 0x7fdccc15d4f0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fdcc800ebe0 tx=0x7fdcc800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:08.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.765+0000 7fdcbdffb640 1 -- 172.21.15.43:0/1283144985 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc8017070 con 0x7fdccc0d2370 2024-04-25T08:58:08.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.765+0000 7fdcd365d640 1 -- 172.21.15.43:0/1283144985 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdccc0cb0c0 con 0x7fdccc0d2370 2024-04-25T08:58:08.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.765+0000 7fdcbdffb640 1 -- 172.21.15.43:0/1283144985 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcc800cc30 con 0x7fdccc0d2370 2024-04-25T08:58:08.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.765+0000 7fdcbdffb640 1 -- 172.21.15.43:0/1283144985 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcc8013470 con 0x7fdccc0d2370 2024-04-25T08:58:08.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.765+0000 7fdcd365d640 1 -- 172.21.15.43:0/1283144985 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdccc0cb680 con 0x7fdccc0d2370 2024-04-25T08:58:08.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.766+0000 7fdcd365d640 1 -- 172.21.15.43:0/1283144985 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdccc074f60 con 0x7fdccc0d2370 2024-04-25T08:58:08.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.767+0000 7fdcbdffb640 1 -- 172.21.15.43:0/1283144985 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdcc801a070 con 0x7fdccc0d2370 2024-04-25T08:58:08.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.767+0000 7fdcbdffb640 1 --2- 172.21.15.43:0/1283144985 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdcb006b4d0 0x7fdcb006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:08.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.767+0000 7fdcbdffb640 1 -- 172.21.15.43:0/1283144985 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdcc808a130 con 0x7fdccc0d2370 2024-04-25T08:58:08.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.768+0000 7fdcd13f9640 1 --2- 172.21.15.43:0/1283144985 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdcb006b4d0 0x7fdcb006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:08.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.768+0000 7fdcd13f9640 1 --2- 172.21.15.43:0/1283144985 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdcb006b4d0 0x7fdcb006d990 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0005880 tx=0x7fdcc00057f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:08.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.771+0000 7fdcbdffb640 1 -- 172.21.15.43:0/1283144985 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdcc80588d0 con 0x7fdccc0d2370 2024-04-25T08:58:08.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.950+0000 7fdcd365d640 1 -- 172.21.15.43:0/1283144985 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdccc0cd620 con 0x7fdccc0d2370 2024-04-25T08:58:08.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.951+0000 7fdcbdffb640 1 -- 172.21.15.43:0/1283144985 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v58) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdcc805c970 con 0x7fdccc0d2370 2024-04-25T08:58:08.940 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T08:58:08.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.952+0000 7fdcd365d640 1 -- 172.21.15.43:0/1283144985 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdcb006b4d0 msgr2=0x7fdcb006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:08.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.952+0000 7fdcd365d640 1 --2- 172.21.15.43:0/1283144985 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdcb006b4d0 0x7fdcb006d990 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fdcc0005880 tx=0x7fdcc00057f0 comp rx=0 tx=0).stop 2024-04-25T08:58:08.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.952+0000 7fdcd365d640 1 -- 172.21.15.43:0/1283144985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdccc0d2370 msgr2=0x7fdccc15d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:08.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.952+0000 7fdcd365d640 1 --2- 172.21.15.43:0/1283144985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdccc0d2370 0x7fdccc15d4f0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fdcc800ebe0 tx=0x7fdcc800c520 comp rx=0 tx=0).stop 2024-04-25T08:58:08.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.952+0000 7fdcd365d640 1 -- 172.21.15.43:0/1283144985 shutdown_connections 2024-04-25T08:58:08.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.952+0000 7fdcd365d640 1 --2- 172.21.15.43:0/1283144985 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdcb006b4d0 0x7fdcb006d990 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:08.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.952+0000 7fdcd365d640 1 --2- 172.21.15.43:0/1283144985 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdccc161310 0x7fdccc0cbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:08.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.952+0000 7fdcd365d640 1 --2- 172.21.15.43:0/1283144985 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdccc0d2370 0x7fdccc15d4f0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:08.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.952+0000 7fdcd365d640 1 --2- 172.21.15.43:0/1283144985 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdccc0d19f0 0x7fdccc0cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:08.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.952+0000 7fdcd365d640 1 -- 172.21.15.43:0/1283144985 >> 172.21.15.43:0/1283144985 conn(0x7fdccc09cd20 msgr2=0x7fdccc15f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:08.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.952+0000 7fdcd365d640 1 -- 172.21.15.43:0/1283144985 shutdown_connections 2024-04-25T08:58:08.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:08.952+0000 7fdcd365d640 1 -- 172.21.15.43:0/1283144985 wait complete. 2024-04-25T08:58:08.953 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:58:09.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.065+0000 7ff68d713640 1 Processor -- start 2024-04-25T08:58:09.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.065+0000 7ff68d713640 1 -- start start 2024-04-25T08:58:09.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.066+0000 7ff68d713640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff68815df50 0x7ff6881619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:09.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.066+0000 7ff68d713640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff6880cf670 0x7ff688162380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:09.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.066+0000 7ff68d713640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6880d24a0 con 0x7ff688161f30 2024-04-25T08:58:09.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.066+0000 7ff68d713640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff68815e330 con 0x7ff68815df50 2024-04-25T08:58:09.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.066+0000 7ff68d713640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6880d2740 con 0x7ff6880cf670 2024-04-25T08:58:09.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.066+0000 7ff686ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff68815df50 0x7ff6881619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:09.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.066+0000 7ff686ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff68815df50 0x7ff6881619f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44282/0 (socket says 172.21.15.43:44282) 2024-04-25T08:58:09.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.066+0000 7ff686ffd640 1 -- 172.21.15.43:0/2153895844 learned_addr learned my addr 172.21.15.43:0/2153895844 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:09.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.066+0000 7ff6877fe640 1 --2- 172.21.15.43:0/2153895844 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff6880cf670 0x7ff688162380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:09.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.066+0000 7ff686ffd640 1 -- 172.21.15.43:0/2153895844 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff6880cf670 msgr2=0x7ff688162380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:09.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.066+0000 7ff686ffd640 1 --2- 172.21.15.43:0/2153895844 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff6880cf670 0x7ff688162380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:09.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.066+0000 7ff686ffd640 1 -- 172.21.15.43:0/2153895844 >> v1:172.21.15.193:6789/0 conn(0x7ff688161f30 legacy=0x7ff6880c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:09.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.066+0000 7ff686ffd640 1 -- 172.21.15.43:0/2153895844 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6880d2a20 con 0x7ff68815df50 2024-04-25T08:58:09.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.067+0000 7ff686ffd640 1 --2- 172.21.15.43:0/2153895844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff68815df50 0x7ff6881619f0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ff678009870 tx=0x7ff67802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cb0e5dc90c45108 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:09.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.067+0000 7ff6867fc640 1 -- 172.21.15.43:0/2153895844 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff678037070 con 0x7ff68815df50 2024-04-25T08:58:09.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.067+0000 7ff6867fc640 1 -- 172.21.15.43:0/2153895844 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff678032440 con 0x7ff68815df50 2024-04-25T08:58:09.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.067+0000 7ff6867fc640 1 -- 172.21.15.43:0/2153895844 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff678032810 con 0x7ff68815df50 2024-04-25T08:58:09.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.068+0000 7ff68d713640 1 -- 172.21.15.43:0/2153895844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff68815df50 msgr2=0x7ff6881619f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:09.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.068+0000 7ff68d713640 1 --2- 172.21.15.43:0/2153895844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff68815df50 0x7ff6881619f0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ff678009870 tx=0x7ff67802a210 comp rx=0 tx=0).stop 2024-04-25T08:58:09.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.068+0000 7ff68d713640 1 -- 172.21.15.43:0/2153895844 shutdown_connections 2024-04-25T08:58:09.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.068+0000 7ff68d713640 1 --2- 172.21.15.43:0/2153895844 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff6880cf670 0x7ff688162380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:09.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.068+0000 7ff68d713640 1 --2- 172.21.15.43:0/2153895844 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff68815df50 0x7ff6881619f0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:09.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.068+0000 7ff68d713640 1 -- 172.21.15.43:0/2153895844 >> 172.21.15.43:0/2153895844 conn(0x7ff6880d3a30 msgr2=0x7ff6880d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:09.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.068+0000 7ff68d713640 1 -- 172.21.15.43:0/2153895844 shutdown_connections 2024-04-25T08:58:09.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.068+0000 7ff68d713640 1 -- 172.21.15.43:0/2153895844 wait complete. 2024-04-25T08:58:09.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.069+0000 7ff68d713640 1 Processor -- start 2024-04-25T08:58:09.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.069+0000 7ff68d713640 1 -- start start 2024-04-25T08:58:09.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.070+0000 7ff68d713640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff6880cf670 0x7ff68815ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:09.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.070+0000 7ff68d713640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff68815df50 0x7ff6881601c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:09.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.070+0000 7ff68d713640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff688161f30 0x7ff688160700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:09.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.070+0000 7ff68d713640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6880d2190 con 0x7ff68815df50 2024-04-25T08:58:09.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.070+0000 7ff68d713640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6880c77d0 con 0x7ff688161f30 2024-04-25T08:58:09.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.070+0000 7ff68d713640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6880c8400 con 0x7ff6880cf670 2024-04-25T08:58:09.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.070+0000 7ff67ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff68815df50 0x7ff6881601c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:09.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.070+0000 7ff67ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff68815df50 0x7ff6881601c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44288/0 (socket says 172.21.15.43:44288) 2024-04-25T08:58:09.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.070+0000 7ff67ffff640 1 -- 172.21.15.43:0/3487749256 learned_addr learned my addr 172.21.15.43:0/3487749256 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:09.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.070+0000 7ff6877fe640 1 --2- 172.21.15.43:0/3487749256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff688161f30 0x7ff688160700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:09.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.070+0000 7ff686ffd640 1 --2- 172.21.15.43:0/3487749256 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff6880cf670 0x7ff68815ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:09.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.070+0000 7ff67ffff640 1 -- 172.21.15.43:0/3487749256 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff6880cf670 msgr2=0x7ff68815ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:09.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.070+0000 7ff67ffff640 1 --2- 172.21.15.43:0/3487749256 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff6880cf670 0x7ff68815ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:09.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.070+0000 7ff67ffff640 1 -- 172.21.15.43:0/3487749256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff688161f30 msgr2=0x7ff688160700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:09.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.070+0000 7ff67ffff640 1 --2- 172.21.15.43:0/3487749256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff688161f30 0x7ff688160700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:09.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.070+0000 7ff67ffff640 1 -- 172.21.15.43:0/3487749256 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff68815f580 con 0x7ff68815df50 2024-04-25T08:58:09.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.070+0000 7ff6877fe640 1 --2- 172.21.15.43:0/3487749256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff688161f30 0x7ff688160700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:58:09.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.070+0000 7ff686ffd640 1 --2- 172.21.15.43:0/3487749256 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff6880cf670 0x7ff68815ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:58:09.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.070+0000 7ff67ffff640 1 --2- 172.21.15.43:0/3487749256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff68815df50 0x7ff6881601c0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7ff67000b420 tx=0x7ff67000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:09.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.070+0000 7ff67f7fe640 1 -- 172.21.15.43:0/3487749256 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff670011830 con 0x7ff68815df50 2024-04-25T08:58:09.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.071+0000 7ff67f7fe640 1 -- 172.21.15.43:0/3487749256 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff670016070 con 0x7ff68815df50 2024-04-25T08:58:09.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.071+0000 7ff68d713640 1 -- 172.21.15.43:0/3487749256 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6880cac80 con 0x7ff68815df50 2024-04-25T08:58:09.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.071+0000 7ff67f7fe640 1 -- 172.21.15.43:0/3487749256 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff670011c10 con 0x7ff68815df50 2024-04-25T08:58:09.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.071+0000 7ff68d713640 1 -- 172.21.15.43:0/3487749256 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6880cb240 con 0x7ff68815df50 2024-04-25T08:58:09.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.072+0000 7ff68d713640 1 -- 172.21.15.43:0/3487749256 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6880680f0 con 0x7ff68815df50 2024-04-25T08:58:09.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.072+0000 7ff67f7fe640 1 -- 172.21.15.43:0/3487749256 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff67001b460 con 0x7ff68815df50 2024-04-25T08:58:09.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.072+0000 7ff67f7fe640 1 --2- 172.21.15.43:0/3487749256 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff66806b4d0 0x7ff66806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:09.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.072+0000 7ff67f7fe640 1 -- 172.21.15.43:0/3487749256 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff670089c90 con 0x7ff68815df50 2024-04-25T08:58:09.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.073+0000 7ff686ffd640 1 --2- 172.21.15.43:0/3487749256 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff66806b4d0 0x7ff66806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:09.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.073+0000 7ff686ffd640 1 --2- 172.21.15.43:0/3487749256 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff66806b4d0 0x7ff66806d990 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff678034000 tx=0x7ff67802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:09.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.077+0000 7ff67f7fe640 1 -- 172.21.15.43:0/3487749256 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff670020020 con 0x7ff68815df50 2024-04-25T08:58:09.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.270+0000 7ff68d713640 1 -- 172.21.15.43:0/3487749256 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff688154810 con 0x7ff68815df50 2024-04-25T08:58:09.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.270+0000 7ff67f7fe640 1 -- 172.21.15.43:0/3487749256 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v58) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff688154810 con 0x7ff68815df50 2024-04-25T08:58:09.260 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-04-25T08:58:09.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.272+0000 7ff68d713640 1 -- 172.21.15.43:0/3487749256 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff66806b4d0 msgr2=0x7ff66806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:09.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.272+0000 7ff68d713640 1 --2- 172.21.15.43:0/3487749256 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff66806b4d0 0x7ff66806d990 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7ff678034000 tx=0x7ff67802e000 comp rx=0 tx=0).stop 2024-04-25T08:58:09.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.272+0000 7ff68d713640 1 -- 172.21.15.43:0/3487749256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff68815df50 msgr2=0x7ff6881601c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:09.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.272+0000 7ff68d713640 1 --2- 172.21.15.43:0/3487749256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff68815df50 0x7ff6881601c0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7ff67000b420 tx=0x7ff67000b8e0 comp rx=0 tx=0).stop 2024-04-25T08:58:09.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.272+0000 7ff68d713640 1 -- 172.21.15.43:0/3487749256 shutdown_connections 2024-04-25T08:58:09.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.272+0000 7ff68d713640 1 --2- 172.21.15.43:0/3487749256 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff66806b4d0 0x7ff66806d990 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:09.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.272+0000 7ff68d713640 1 --2- 172.21.15.43:0/3487749256 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff688161f30 0x7ff688160700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:09.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.272+0000 7ff68d713640 1 --2- 172.21.15.43:0/3487749256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff68815df50 0x7ff6881601c0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:09.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.272+0000 7ff68d713640 1 --2- 172.21.15.43:0/3487749256 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff6880cf670 0x7ff68815ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:09.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.272+0000 7ff68d713640 1 -- 172.21.15.43:0/3487749256 >> 172.21.15.43:0/3487749256 conn(0x7ff6880d3a30 msgr2=0x7ff6881536e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:09.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.272+0000 7ff68d713640 1 -- 172.21.15.43:0/3487749256 shutdown_connections 2024-04-25T08:58:09.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.273+0000 7ff68d713640 1 -- 172.21.15.43:0/3487749256 wait complete. 2024-04-25T08:58:09.273 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi043.front.sepia.ceph.com... 2024-04-25T08:58:09.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:09.273 DEBUG:teuthology.orchestra.run.smithi043:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T08:58:09.296 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[62874]: fuse finished with error 0 and tester_r 0 2024-04-25T08:58:09.297 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:58:09.392 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-04-25T08:58:09.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:09.393 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:58:09.408 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:58:09.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.559+0000 7ff5590cb640 1 Processor -- start 2024-04-25T08:58:09.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.559+0000 7ff5590cb640 1 -- start start 2024-04-25T08:58:09.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.560+0000 7ff5590cb640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff5540ce170 0x7ff554168af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:09.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.560+0000 7ff5590cb640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5540cc700 con 0x7ff55415dd00 2024-04-25T08:58:09.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.560+0000 7ff5590cb640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5540c4ed0 con 0x7ff5540cc270 2024-04-25T08:58:09.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.560+0000 7ff5590cb640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5540c51b0 con 0x7ff5540ce170 2024-04-25T08:58:09.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.560+0000 7ff552d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff55415dd00 0x7ff5540c5f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:35778/0 (socket says 172.21.15.43:35778) 2024-04-25T08:58:09.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.560+0000 7ff552d9d640 1 -- 172.21.15.43:0/1426270038 learned_addr learned my addr 172.21.15.43:0/1426270038 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:09.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.560+0000 7ff551d9b640 1 -- 172.21.15.43:0/1426270038 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4284135889 0 0) 0x7ff5540cc700 con 0x7ff55415dd00 2024-04-25T08:58:09.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.560+0000 7ff551d9b640 1 -- 172.21.15.43:0/1426270038 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff53c003520 con 0x7ff55415dd00 2024-04-25T08:58:09.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.560+0000 7ff55359e640 1 --2- 172.21.15.43:0/1426270038 >> v2:172.21.15.193:3300/0 conn(0x7ff5540ce170 0x7ff554168af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:09.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.560+0000 7ff551d9b640 1 -- 172.21.15.43:0/1426270038 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3090488463 0 0) 0x7ff53c003520 con 0x7ff55415dd00 2024-04-25T08:58:09.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.560+0000 7ff551d9b640 1 -- 172.21.15.43:0/1426270038 >> v2:172.21.15.193:3300/0 conn(0x7ff5540ce170 msgr2=0x7ff554168af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:09.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.560+0000 7ff551d9b640 1 --2- 172.21.15.43:0/1426270038 >> v2:172.21.15.193:3300/0 conn(0x7ff5540ce170 0x7ff554168af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:09.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.561+0000 7ff551d9b640 1 -- 172.21.15.43:0/1426270038 >> v1:172.21.15.193:6789/0 conn(0x7ff5540cc270 legacy=0x7ff5540cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:09.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.561+0000 7ff551d9b640 1 -- 172.21.15.43:0/1426270038 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5540c5490 con 0x7ff55415dd00 2024-04-25T08:58:09.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.561+0000 7ff551d9b640 1 -- 172.21.15.43:0/1426270038 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7ff54c001690 con 0x7ff55415dd00 2024-04-25T08:58:09.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.561+0000 7ff551d9b640 1 -- 172.21.15.43:0/1426270038 >> v1:172.21.15.43:6789/0 conn(0x7ff55415dd00 legacy=0x7ff5540c5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:09.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.561+0000 7ff551d9b640 1 --2- 172.21.15.43:0/1426270038 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff53c0039b0 0x7ff53c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:09.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.561+0000 7ff551d9b640 1 --2- 172.21.15.43:0/1426270038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff53c004800 0x7ff53c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:09.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.561+0000 7ff551d9b640 1 --2- 172.21.15.43:0/1426270038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff53c0051c0 0x7ff53c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:09.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.561+0000 7ff551d9b640 1 -- 172.21.15.43:0/1426270038 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5540c51b0 con 0x7ff53c0051c0 2024-04-25T08:58:09.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.561+0000 7ff551d9b640 1 -- 172.21.15.43:0/1426270038 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53c00c060 con 0x7ff53c004800 2024-04-25T08:58:09.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.561+0000 7ff551d9b640 1 -- 172.21.15.43:0/1426270038 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff53c00c260 con 0x7ff53c0039b0 2024-04-25T08:58:09.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.561+0000 7ff55359e640 1 --2- 172.21.15.43:0/1426270038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff53c0051c0 0x7ff53c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:09.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.561+0000 7ff552d9d640 1 --2- 172.21.15.43:0/1426270038 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff53c0039b0 0x7ff53c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:09.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.561+0000 7ff55259c640 1 --2- 172.21.15.43:0/1426270038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff53c004800 0x7ff53c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:09.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.561+0000 7ff55359e640 1 -- 172.21.15.43:0/1426270038 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff53c0039b0 msgr2=0x7ff53c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:09.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.561+0000 7ff55359e640 1 --2- 172.21.15.43:0/1426270038 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff53c0039b0 0x7ff53c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:09.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.561+0000 7ff55359e640 1 -- 172.21.15.43:0/1426270038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff53c004800 msgr2=0x7ff53c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:09.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.561+0000 7ff55359e640 1 --2- 172.21.15.43:0/1426270038 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff53c004800 0x7ff53c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:09.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.561+0000 7ff55359e640 1 -- 172.21.15.43:0/1426270038 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff53c003520 con 0x7ff53c0051c0 2024-04-25T08:58:09.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.562+0000 7ff55359e640 1 -- 172.21.15.43:0/1426270038 reap_dead start 2024-04-25T08:58:09.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.562+0000 7ff55359e640 1 --2- 172.21.15.43:0/1426270038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff53c0051c0 0x7ff53c00b9c0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7ff544009fc0 tx=0x7ff53c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ed5e072fbaf742d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:09.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.562+0000 7ff551d9b640 1 -- 172.21.15.43:0/1426270038 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff544035020 con 0x7ff53c0051c0 2024-04-25T08:58:09.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.562+0000 7ff551d9b640 1 -- 172.21.15.43:0/1426270038 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff544003bd0 con 0x7ff53c0051c0 2024-04-25T08:58:09.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.562+0000 7ff551d9b640 1 -- 172.21.15.43:0/1426270038 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff544009c10 con 0x7ff53c0051c0 2024-04-25T08:58:09.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.562+0000 7ff5590cb640 1 -- 172.21.15.43:0/1426270038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff53c0051c0 msgr2=0x7ff53c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:09.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.562+0000 7ff5590cb640 1 --2- 172.21.15.43:0/1426270038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff53c0051c0 0x7ff53c00b9c0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7ff544009fc0 tx=0x7ff53c005830 comp rx=0 tx=0).stop 2024-04-25T08:58:09.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.562+0000 7ff5590cb640 1 -- 172.21.15.43:0/1426270038 shutdown_connections 2024-04-25T08:58:09.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.562+0000 7ff5590cb640 1 --2- 172.21.15.43:0/1426270038 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff53c0051c0 0x7ff53c00b9c0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:09.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.562+0000 7ff5590cb640 1 -- 172.21.15.43:0/1426270038 >> 172.21.15.43:0/1426270038 conn(0x7ff55406ab30 msgr2=0x7ff55406af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:09.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.562+0000 7ff5590cb640 1 -- 172.21.15.43:0/1426270038 shutdown_connections 2024-04-25T08:58:09.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.562+0000 7ff5590cb640 1 -- 172.21.15.43:0/1426270038 wait complete. 2024-04-25T08:58:09.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.564+0000 7ff5590cb640 1 Processor -- start 2024-04-25T08:58:09.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.564+0000 7ff5590cb640 1 -- start start 2024-04-25T08:58:09.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.565+0000 7ff5590cb640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff53c0051c0 0x7ff5540b58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:09.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.565+0000 7ff5590cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff554156a00 0x7ff554156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:09.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.565+0000 7ff5590cb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff55409aa70 0x7ff55415bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:09.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.565+0000 7ff5590cb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5540cda60 con 0x7ff554156a00 2024-04-25T08:58:09.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.565+0000 7ff5590cb640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5540c5a50 con 0x7ff55409aa70 2024-04-25T08:58:09.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.565+0000 7ff5590cb640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5540cdbe0 con 0x7ff53c0051c0 2024-04-25T08:58:09.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.565+0000 7ff55259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff554156a00 0x7ff554156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:09.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.565+0000 7ff55259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff554156a00 0x7ff554156290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44302/0 (socket says 172.21.15.43:44302) 2024-04-25T08:58:09.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.565+0000 7ff55259c640 1 -- 172.21.15.43:0/619449703 learned_addr learned my addr 172.21.15.43:0/619449703 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:09.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.565+0000 7ff55359e640 1 --2- 172.21.15.43:0/619449703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff55409aa70 0x7ff55415bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:09.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.565+0000 7ff552d9d640 1 --2- 172.21.15.43:0/619449703 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff53c0051c0 0x7ff5540b58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:09.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.565+0000 7ff55259c640 1 -- 172.21.15.43:0/619449703 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff53c0051c0 msgr2=0x7ff5540b58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:09.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.565+0000 7ff55259c640 1 --2- 172.21.15.43:0/619449703 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff53c0051c0 0x7ff5540b58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:09.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.565+0000 7ff55259c640 1 -- 172.21.15.43:0/619449703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff55409aa70 msgr2=0x7ff55415bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:09.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.565+0000 7ff55259c640 1 --2- 172.21.15.43:0/619449703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff55409aa70 0x7ff55415bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:09.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.565+0000 7ff55259c640 1 -- 172.21.15.43:0/619449703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5541567d0 con 0x7ff554156a00 2024-04-25T08:58:09.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.565+0000 7ff55259c640 1 --2- 172.21.15.43:0/619449703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff554156a00 0x7ff554156290 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7ff54800f620 tx=0x7ff54800fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:09.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.565+0000 7ff53b7fe640 1 -- 172.21.15.43:0/619449703 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5480052e0 con 0x7ff554156a00 2024-04-25T08:58:09.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.565+0000 7ff53b7fe640 1 -- 172.21.15.43:0/619449703 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff548018070 con 0x7ff554156a00 2024-04-25T08:58:09.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.565+0000 7ff5590cb640 1 -- 172.21.15.43:0/619449703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5540c84c0 con 0x7ff554156a00 2024-04-25T08:58:09.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.565+0000 7ff53b7fe640 1 -- 172.21.15.43:0/619449703 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff548003810 con 0x7ff554156a00 2024-04-25T08:58:09.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.566+0000 7ff5590cb640 1 -- 172.21.15.43:0/619449703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5540c8a80 con 0x7ff554156a00 2024-04-25T08:58:09.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.566+0000 7ff5590cb640 1 -- 172.21.15.43:0/619449703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff554074ef0 con 0x7ff554156a00 2024-04-25T08:58:09.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.567+0000 7ff53b7fe640 1 -- 172.21.15.43:0/619449703 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff5480039b0 con 0x7ff554156a00 2024-04-25T08:58:09.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.567+0000 7ff53b7fe640 1 --2- 172.21.15.43:0/619449703 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff54006b4d0 0x7ff54006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:09.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.567+0000 7ff53b7fe640 1 -- 172.21.15.43:0/619449703 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff548089f20 con 0x7ff554156a00 2024-04-25T08:58:09.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.567+0000 7ff552d9d640 1 --2- 172.21.15.43:0/619449703 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff54006b4d0 0x7ff54006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:09.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.568+0000 7ff552d9d640 1 --2- 172.21.15.43:0/619449703 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff54006b4d0 0x7ff54006d990 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff5541f7910 tx=0x7ff54c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:09.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.571+0000 7ff53b7fe640 1 -- 172.21.15.43:0/619449703 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff548058e30 con 0x7ff554156a00 2024-04-25T08:58:09.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.797+0000 7ff5590cb640 1 -- 172.21.15.43:0/619449703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff55406c450 con 0x7ff554156a00 2024-04-25T08:58:09.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.797+0000 7ff53b7fe640 1 -- 172.21.15.43:0/619449703 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4944 (secure 0 0 0) 0x7ff548058730 con 0x7ff554156a00 2024-04-25T08:58:09.787 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:58:09.787 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5950,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/2226672650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2226672650},{"type":"v1","addr":"172.21.15.193:6836","nonce":2226672650}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":5953,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2666301750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2666301750},{"type":"v1","addr":"172.21.15.193:6837","nonce":2666301750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28},{"gid":6537,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/3574570511","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":3574570511},{"type":"v1","addr":"172.21.15.43:6834","nonce":3574570511}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:57:54.590369+0000","modified":"2024-04-25T08:57:56.568622+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6540},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6540":{"gid":6540,"name":"c","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/2268368384","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2268368384},{"type":"v1","addr":"172.21.15.43:6835","nonce":2268368384}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:58:09.787 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 31 2024-04-25T08:58:09.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.799+0000 7ff5590cb640 1 -- 172.21.15.43:0/619449703 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff54006b4d0 msgr2=0x7ff54006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:09.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.799+0000 7ff5590cb640 1 --2- 172.21.15.43:0/619449703 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff54006b4d0 0x7ff54006d990 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7ff5541f7910 tx=0x7ff54c009040 comp rx=0 tx=0).stop 2024-04-25T08:58:09.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.799+0000 7ff5590cb640 1 -- 172.21.15.43:0/619449703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff554156a00 msgr2=0x7ff554156290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:09.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.799+0000 7ff5590cb640 1 --2- 172.21.15.43:0/619449703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff554156a00 0x7ff554156290 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7ff54800f620 tx=0x7ff54800fae0 comp rx=0 tx=0).stop 2024-04-25T08:58:09.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.799+0000 7ff5590cb640 1 -- 172.21.15.43:0/619449703 shutdown_connections 2024-04-25T08:58:09.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.799+0000 7ff5590cb640 1 --2- 172.21.15.43:0/619449703 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff54006b4d0 0x7ff54006d990 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:09.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.799+0000 7ff5590cb640 1 --2- 172.21.15.43:0/619449703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff55409aa70 0x7ff55415bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:09.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.799+0000 7ff5590cb640 1 --2- 172.21.15.43:0/619449703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff554156a00 0x7ff554156290 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:09.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.799+0000 7ff5590cb640 1 --2- 172.21.15.43:0/619449703 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff53c0051c0 0x7ff5540b58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:09.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.799+0000 7ff5590cb640 1 -- 172.21.15.43:0/619449703 >> 172.21.15.43:0/619449703 conn(0x7ff55406ab30 msgr2=0x7ff554071470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:09.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.799+0000 7ff5590cb640 1 -- 172.21.15.43:0/619449703 shutdown_connections 2024-04-25T08:58:09.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:09.799+0000 7ff5590cb640 1 -- 172.21.15.43:0/619449703 wait complete. 2024-04-25T08:58:09.799 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T08:58:09.921 INFO:teuthology.orchestra.run.smithi043.stdout:{"message":"","return_code":0} 2024-04-25T08:58:09.922 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T08:58:09.922 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T08:58:09.922 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:58:09.923 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T08:58:09.923 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:58:09.923 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T08:58:09.923 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:58:09.923 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T08:58:09.924 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:58:09.932 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T08:58:09.933 DEBUG:teuthology.parallel:result is None 2024-04-25T08:58:09.933 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T08:58:09.933 DEBUG:teuthology.parallel:result is None 2024-04-25T08:58:09.935 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T08:58:09.935 DEBUG:teuthology.parallel:result is None 2024-04-25T08:58:09.945 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T08:58:09.946 DEBUG:teuthology.parallel:result is None 2024-04-25T08:58:09.946 DEBUG:teuthology.orchestra.run.smithi043:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000000.00000000 file_head 2024-04-25T08:58:10.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.026+0000 7f3ee3378f40 1 Processor -- start 2024-04-25T08:58:10.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.026+0000 7f3ee3378f40 1 -- start start 2024-04-25T08:58:10.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.027+0000 7f3ee3378f40 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ecc0022a0 0x5583024c0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:10.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.027+0000 7f3ee3378f40 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558302565db0 0x55830254cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:10.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.027+0000 7f3ee3378f40 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55830254b420 0x558302492980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:10.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.027+0000 7f3ee3378f40 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x558302438df0 con 0x55830254b420 2024-04-25T08:58:10.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.027+0000 7f3ee3378f40 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55830243cd50 con 0x558302565db0 2024-04-25T08:58:10.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.027+0000 7f3ee3378f40 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55830254d750 con 0x7f3ecc0022a0 2024-04-25T08:58:10.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.027+0000 7f3ee167c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55830254b420 0x558302492980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:10.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.027+0000 7f3ee167c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55830254b420 0x558302492980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45582/0 (socket says 172.21.15.43:45582) 2024-04-25T08:58:10.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.027+0000 7f3ee167c640 1 -- 172.21.15.43:0/34618539 learned_addr learned my addr 172.21.15.43:0/34618539 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:10.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.027+0000 7f3ee167c640 1 -- 172.21.15.43:0/34618539 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ecc0022a0 msgr2=0x5583024c0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:10.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.027+0000 7f3ee167c640 1 --2- 172.21.15.43:0/34618539 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ecc0022a0 0x5583024c0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:10.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.027+0000 7f3ee167c640 1 -- 172.21.15.43:0/34618539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558302565db0 msgr2=0x55830254cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:10.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.027+0000 7f3ee167c640 1 --2- 172.21.15.43:0/34618539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558302565db0 0x55830254cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:10.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.027+0000 7f3ee167c640 1 -- 172.21.15.43:0/34618539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583024da730 con 0x55830254b420 2024-04-25T08:58:10.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.028+0000 7f3ee167c640 1 --2- 172.21.15.43:0/34618539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55830254b420 0x558302492980 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0006b50 tx=0x7f3ed0003a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=247b80453e570baa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:10.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.028+0000 7f3edf678640 1 -- 172.21.15.43:0/34618539 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed0018070 con 0x55830254b420 2024-04-25T08:58:10.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.028+0000 7f3edf678640 1 -- 172.21.15.43:0/34618539 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ed0006e50 con 0x55830254b420 2024-04-25T08:58:10.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.028+0000 7f3edf678640 1 -- 172.21.15.43:0/34618539 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed0003330 con 0x55830254b420 2024-04-25T08:58:10.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.028+0000 7f3ee3378f40 1 -- 172.21.15.43:0/34618539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55830254b420 msgr2=0x558302492980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:10.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.028+0000 7f3ee3378f40 1 --2- 172.21.15.43:0/34618539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55830254b420 0x558302492980 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0006b50 tx=0x7f3ed0003a00 comp rx=0 tx=0).stop 2024-04-25T08:58:10.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.028+0000 7f3ee3378f40 1 -- 172.21.15.43:0/34618539 shutdown_connections 2024-04-25T08:58:10.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.028+0000 7f3ee3378f40 1 --2- 172.21.15.43:0/34618539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55830254b420 0x558302492980 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:10.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.028+0000 7f3ee3378f40 1 --2- 172.21.15.43:0/34618539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558302565db0 0x55830254cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:10.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.028+0000 7f3ee3378f40 1 --2- 172.21.15.43:0/34618539 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ecc0022a0 0x5583024c0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:10.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.028+0000 7f3ee3378f40 1 -- 172.21.15.43:0/34618539 >> 172.21.15.43:0/34618539 conn(0x5583024bc4a0 msgr2=0x558302493b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:10.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.028+0000 7f3ee3378f40 1 -- 172.21.15.43:0/34618539 shutdown_connections 2024-04-25T08:58:10.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.028+0000 7f3ee3378f40 1 -- 172.21.15.43:0/34618539 wait complete. 2024-04-25T08:58:10.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.029+0000 7f3ee3378f40 1 Processor -- start 2024-04-25T08:58:10.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.029+0000 7f3ee3378f40 1 -- start start 2024-04-25T08:58:10.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.030+0000 7f3ee3378f40 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55830254b420 0x5583024bdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:10.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.030+0000 7f3ee3378f40 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558302565db0 0x5583025ff850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:10.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.030+0000 7f3ee3378f40 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ecc0022a0 0x5583024d4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:10.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.030+0000 7f3ee3378f40 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x5583024dacd0 con 0x7f3ecc0022a0 2024-04-25T08:58:10.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.030+0000 7f3ee3378f40 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55830254bb30 con 0x558302565db0 2024-04-25T08:58:10.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.030+0000 7f3ee3378f40 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x5583024db860 con 0x55830254b420 2024-04-25T08:58:10.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.030+0000 7f3ee167c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ecc0022a0 0x5583024d4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:10.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.030+0000 7f3ee167c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ecc0022a0 0x5583024d4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45598/0 (socket says 172.21.15.43:45598) 2024-04-25T08:58:10.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.030+0000 7f3ee167c640 1 -- 172.21.15.43:0/3300660276 learned_addr learned my addr 172.21.15.43:0/3300660276 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:10.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.030+0000 7f3ee067a640 1 --2- 172.21.15.43:0/3300660276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558302565db0 0x5583025ff850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:10.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.030+0000 7f3ee0e7b640 1 --2- 172.21.15.43:0/3300660276 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55830254b420 0x5583024bdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:10.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.030+0000 7f3ee167c640 1 -- 172.21.15.43:0/3300660276 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55830254b420 msgr2=0x5583024bdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:10.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.030+0000 7f3ee167c640 1 --2- 172.21.15.43:0/3300660276 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55830254b420 0x5583024bdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:10.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.030+0000 7f3ee167c640 1 -- 172.21.15.43:0/3300660276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558302565db0 msgr2=0x5583025ff850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:10.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.030+0000 7f3ee167c640 1 --2- 172.21.15.43:0/3300660276 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558302565db0 0x5583025ff850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:10.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.030+0000 7f3ee167c640 1 -- 172.21.15.43:0/3300660276 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5583024da3a0 con 0x7f3ecc0022a0 2024-04-25T08:58:10.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.030+0000 7f3ee167c640 1 --2- 172.21.15.43:0/3300660276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ecc0022a0 0x5583024d4f90 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0003a40 tx=0x7f3ed0001510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:10.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.030+0000 7f3ede676640 1 -- 172.21.15.43:0/3300660276 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed0018040 con 0x7f3ecc0022a0 2024-04-25T08:58:10.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.030+0000 7f3ede676640 1 -- 172.21.15.43:0/3300660276 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ed0023070 con 0x7f3ecc0022a0 2024-04-25T08:58:10.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.030+0000 7f3ee3378f40 1 -- 172.21.15.43:0/3300660276 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5583024da0c0 con 0x7f3ecc0022a0 2024-04-25T08:58:10.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.031+0000 7f3ede676640 1 -- 172.21.15.43:0/3300660276 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed0013450 con 0x7f3ecc0022a0 2024-04-25T08:58:10.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.031+0000 7f3ee3378f40 1 -- 172.21.15.43:0/3300660276 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5583024d2d70 con 0x7f3ecc0022a0 2024-04-25T08:58:10.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.032+0000 7f3ede676640 1 -- 172.21.15.43:0/3300660276 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ed0028460 con 0x7f3ecc0022a0 2024-04-25T08:58:10.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.032+0000 7f3ede676640 1 --2- 172.21.15.43:0/3300660276 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3ec0069eb0 0x7f3ec006c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:10.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.032+0000 7f3ede676640 1 -- 172.21.15.43:0/3300660276 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3ed0089d40 con 0x7f3ecc0022a0 2024-04-25T08:58:10.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.033+0000 7f3ee3378f40 1 --2- 172.21.15.43:0/3300660276 >> [v2:172.21.15.193:6808/3086001447,v1:172.21.15.193:6809/3086001447] conn(0x5583024d9bc0 0x5583024d7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:10.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.033+0000 7f3ee3378f40 1 -- 172.21.15.43:0/3300660276 --> [v2:172.21.15.193:6808/3086001447,v1:172.21.15.193:6809/3086001447] -- osd_op(unknown.0.0:1 8.10 8:0cf6ad0f:::10000000000.00000000:head [omap-rm-keys in=17b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58) v9 -- 0x5583024d7a40 con 0x5583024d9bc0 2024-04-25T08:58:10.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.033+0000 7f3ee0e7b640 1 --2- 172.21.15.43:0/3300660276 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3ec0069eb0 0x7f3ec006c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:10.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.033+0000 7f3ee067a640 1 --2- 172.21.15.43:0/3300660276 >> [v2:172.21.15.193:6808/3086001447,v1:172.21.15.193:6809/3086001447] conn(0x5583024d9bc0 0x5583024d7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:10.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.033+0000 7f3ee0e7b640 1 --2- 172.21.15.43:0/3300660276 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3ec0069eb0 0x7f3ec006c370 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8056000 tx=0x7f3ed805d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:10.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.033+0000 7f3ee067a640 1 --2- 172.21.15.43:0/3300660276 >> [v2:172.21.15.193:6808/3086001447,v1:172.21.15.193:6809/3086001447] conn(0x5583024d9bc0 0x5583024d7500 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:10.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.035+0000 7f3ee067a640 1 -- 172.21.15.43:0/3300660276 <== osd.7 v2:172.21.15.193:6808/3086001447 1 ==== osd_op_reply(1 10000000000.00000000 [omap-rm-keys] v58'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f3ec80096f0 con 0x5583024d9bc0 2024-04-25T08:58:10.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.035+0000 7f3ee3378f40 1 -- 172.21.15.43:0/3300660276 >> [v2:172.21.15.193:6808/3086001447,v1:172.21.15.193:6809/3086001447] conn(0x5583024d9bc0 msgr2=0x5583024d7500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:10.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.035+0000 7f3ee3378f40 1 --2- 172.21.15.43:0/3300660276 >> [v2:172.21.15.193:6808/3086001447,v1:172.21.15.193:6809/3086001447] conn(0x5583024d9bc0 0x5583024d7500 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:10.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.035+0000 7f3ee3378f40 1 -- 172.21.15.43:0/3300660276 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3ec0069eb0 msgr2=0x7f3ec006c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:10.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.035+0000 7f3ee3378f40 1 --2- 172.21.15.43:0/3300660276 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3ec0069eb0 0x7f3ec006c370 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8056000 tx=0x7f3ed805d040 comp rx=0 tx=0).stop 2024-04-25T08:58:10.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.035+0000 7f3ee3378f40 1 -- 172.21.15.43:0/3300660276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ecc0022a0 msgr2=0x5583024d4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:10.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.035+0000 7f3ee3378f40 1 --2- 172.21.15.43:0/3300660276 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ecc0022a0 0x5583024d4f90 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0003a40 tx=0x7f3ed0001510 comp rx=0 tx=0).stop 2024-04-25T08:58:10.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.035+0000 7f3ee167c640 1 -- 172.21.15.43:0/3300660276 reap_dead start 2024-04-25T08:58:10.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.035+0000 7f3ee3378f40 1 -- 172.21.15.43:0/3300660276 shutdown_connections 2024-04-25T08:58:10.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.035+0000 7f3ee3378f40 1 -- 172.21.15.43:0/3300660276 >> 172.21.15.43:0/3300660276 conn(0x5583024bc4a0 msgr2=0x55830254ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:10.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.035+0000 7f3ee3378f40 1 -- 172.21.15.43:0/3300660276 shutdown_connections 2024-04-25T08:58:10.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.035+0000 7f3ee3378f40 1 -- 172.21.15.43:0/3300660276 wait complete. 2024-04-25T08:58:10.028 ERROR:tasks.ceph.mds.a:tried to stop a non-running daemon 2024-04-25T08:58:10.028 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail a 2024-04-25T08:58:10.029 ERROR:tasks.ceph.mds.c:tried to stop a non-running daemon 2024-04-25T08:58:10.029 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail c 2024-04-25T08:58:10.029 ERROR:tasks.ceph.mds.b:tried to stop a non-running daemon 2024-04-25T08:58:10.029 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail b 2024-04-25T08:58:10.030 ERROR:tasks.ceph.mds.d:tried to stop a non-running daemon 2024-04-25T08:58:10.030 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail d 2024-04-25T08:58:10.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.186+0000 7fc757a1c640 1 Processor -- start 2024-04-25T08:58:10.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.186+0000 7f29d1b9e640 1 Processor -- start 2024-04-25T08:58:10.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.186+0000 7fca25bce640 1 Processor -- start 2024-04-25T08:58:10.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.186+0000 7f821b23a640 1 Processor -- start 2024-04-25T08:58:10.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.187+0000 7f29d1b9e640 1 -- start start 2024-04-25T08:58:10.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.187+0000 7fca25bce640 1 -- start start 2024-04-25T08:58:10.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.187+0000 7fc757a1c640 1 -- start start 2024-04-25T08:58:10.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.187+0000 7f821b23a640 1 -- start start 2024-04-25T08:58:10.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.187+0000 7f29d1b9e640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f29cc0cbfc0 0x7f29cc06dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:10.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.187+0000 7f29d1b9e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f29cc0ccbb0 0x7f29cc0cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:10.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.187+0000 7fc757a1c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc75015df50 0x7fc7501619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:10.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.187+0000 7f29d1b9e640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29cc0b2780 con 0x7f29cc15f5e0 2024-04-25T08:58:10.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7f29d1b9e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29cc0cd550 con 0x7f29cc0ccbb0 2024-04-25T08:58:10.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.187+0000 7f821b23a640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f821415a840 0x7f821406d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:10.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7f821b23a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f821406d7d0 0x7f8214071a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:10.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7f29d1b9e640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29cc0b2a60 con 0x7f29cc0cbfc0 2024-04-25T08:58:10.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7f821b23a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8214071f90 0x7f82140b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:10.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7f29caffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f29cc0ccbb0 0x7f29cc0cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:10.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7fc7557b8640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc75015df50 0x7fc7501619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:10.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7fc754fb7640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc750161f30 0x7fc7500c7cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47144/0 (socket says 172.21.15.43:47144) 2024-04-25T08:58:10.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7fc754fb7640 1 -- 172.21.15.43:0/1769979171 learned_addr learned my addr 172.21.15.43:0/1769979171 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:10.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7fc757a1c640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7500d24a0 con 0x7fc750161f30 2024-04-25T08:58:10.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7f29caffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f29cc0ccbb0 0x7f29cc0cd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45600/0 (socket says 172.21.15.43:45600) 2024-04-25T08:58:10.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7f29caffd640 1 -- 172.21.15.43:0/8203479 learned_addr learned my addr 172.21.15.43:0/8203479 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:10.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7f29cb7fe640 1 --2- 172.21.15.43:0/8203479 >> v2:172.21.15.193:3300/0 conn(0x7f29cc0cbfc0 0x7f29cc06dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:10.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7fc757a1c640 1 -- 172.21.15.43:0/1769979171 --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7500d2740 con 0x7fc7500cf670 2024-04-25T08:58:10.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7fc757a1c640 1 -- 172.21.15.43:0/1769979171 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7500cfa80 con 0x7fc75015df50 2024-04-25T08:58:10.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7f821b23a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f821415ac20 con 0x7f821406d7d0 2024-04-25T08:58:10.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7f821b23a640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82140704d0 con 0x7f821415a840 2024-04-25T08:58:10.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7f821b23a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82140707b0 con 0x7f8214071f90 2024-04-25T08:58:10.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7f8218fd6640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f821415a840 0x7f821406d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:10.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7f8218fd6640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f821415a840 0x7f821406d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:50496/0 (socket says 172.21.15.43:50496) 2024-04-25T08:58:10.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7f8218fd6640 1 -- 172.21.15.43:0/3978080069 learned_addr learned my addr 172.21.15.43:0/3978080069 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:10.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7f8213fff640 1 --2- 172.21.15.43:0/3978080069 >> v2:172.21.15.43:3300/0 conn(0x7f821406d7d0 0x7f8214071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:10.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7f82197d7640 1 --2- 172.21.15.43:0/3978080069 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8214071f90 0x7f82140b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:10.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7fca25bce640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fca20071a30 0x7fca200b3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:10.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7fca1effd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fca200715a0 0x7fca200b3100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47140/0 (socket says 172.21.15.43:47140) 2024-04-25T08:58:10.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7f8218fd6640 1 -- 172.21.15.43:0/3978080069 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8214071f90 msgr2=0x7f82140b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:10.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7fca1effd640 1 -- 172.21.15.43:0/3607296222 learned_addr learned my addr 172.21.15.43:0/3607296222 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:10.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7fca1ffff640 1 --2- 172.21.15.43:0/3607296222 >> v2:172.21.15.193:3300/0 conn(0x7fca20071a30 0x7fca200b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:10.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7fca25bce640 1 -- 172.21.15.43:0/3607296222 --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca2006dd30 con 0x7fca200715a0 2024-04-25T08:58:10.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7fc747fff640 1 -- 172.21.15.43:0/1769979171 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 777639630 0 0) 0x7fc7500d2740 con 0x7fc7500cf670 2024-04-25T08:58:10.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7f8218fd6640 1 --2- 172.21.15.43:0/3978080069 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8214071f90 0x7f82140b3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:10.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7f8218fd6640 1 -- 172.21.15.43:0/3978080069 >> v2:172.21.15.43:3300/0 conn(0x7f821406d7d0 msgr2=0x7f8214071a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:10.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7f8218fd6640 1 --2- 172.21.15.43:0/3978080069 >> v2:172.21.15.43:3300/0 conn(0x7f821406d7d0 0x7f8214071a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:10.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7fc747fff640 1 -- 172.21.15.43:0/1769979171 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc73c003520 con 0x7fc7500cf670 2024-04-25T08:58:10.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7f8218fd6640 1 -- 172.21.15.43:0/3978080069 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8214070a90 con 0x7f821415a840 2024-04-25T08:58:10.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7fc747fff640 1 -- 172.21.15.43:0/1769979171 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1132017756 0 0) 0x7fc7500d24a0 con 0x7fc750161f30 2024-04-25T08:58:10.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7fc747fff640 1 -- 172.21.15.43:0/1769979171 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7500d2740 con 0x7fc750161f30 2024-04-25T08:58:10.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7fca25bce640 1 -- 172.21.15.43:0/3607296222 --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca2006e010 con 0x7fca2015a250 2024-04-25T08:58:10.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7fca25bce640 1 -- 172.21.15.43:0/3607296222 --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca200b2900 con 0x7fca20071a30 2024-04-25T08:58:10.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7f29cb7fe640 1 -- 172.21.15.43:0/8203479 >> v2:172.21.15.43:3300/0 conn(0x7f29cc0ccbb0 msgr2=0x7f29cc0cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:10.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7f29cb7fe640 1 --2- 172.21.15.43:0/8203479 >> v2:172.21.15.43:3300/0 conn(0x7f29cc0ccbb0 0x7f29cc0cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:10.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7f29cb7fe640 1 -- 172.21.15.43:0/8203479 >> v1:172.21.15.193:6789/0 conn(0x7f29cc15f5e0 legacy=0x7f29cc15fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:10.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7f29cb7fe640 1 -- 172.21.15.43:0/8203479 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29cc0b2d40 con 0x7f29cc0cbfc0 2024-04-25T08:58:10.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7fc7557b8640 1 -- 172.21.15.43:0/1769979171 >> v1:172.21.15.193:6789/0 conn(0x7fc7500cf670 legacy=0x7fc7500c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:10.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7fc7557b8640 1 -- 172.21.15.43:0/1769979171 >> v1:172.21.15.43:6789/0 conn(0x7fc750161f30 legacy=0x7fc7500c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:10.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.189+0000 7fc7557b8640 1 -- 172.21.15.43:0/1769979171 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7500d2a20 con 0x7fc75015df50 2024-04-25T08:58:10.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.188+0000 7fca1e7fc640 1 -- 172.21.15.43:0/3607296222 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1084748795 0 0) 0x7fca2006dd30 con 0x7fca200715a0 2024-04-25T08:58:10.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.189+0000 7fca1e7fc640 1 -- 172.21.15.43:0/3607296222 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca08003520 con 0x7fca200715a0 2024-04-25T08:58:10.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.189+0000 7f29caffd640 1 --2- 172.21.15.43:0/8203479 >> v2:172.21.15.43:3300/0 conn(0x7f29cc0ccbb0 0x7f29cc0cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:58:10.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.189+0000 7f82197d7640 1 --2- 172.21.15.43:0/3978080069 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8214071f90 0x7f82140b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:10.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.189+0000 7fca1ffff640 1 -- 172.21.15.43:0/3607296222 >> v1:172.21.15.193:6789/0 conn(0x7fca2015a250 legacy=0x7fca20070d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:10.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.189+0000 7fca1ffff640 1 -- 172.21.15.43:0/3607296222 >> v1:172.21.15.43:6789/0 conn(0x7fca200715a0 legacy=0x7fca200b3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:10.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.189+0000 7fca1e7fc640 1 -- 172.21.15.43:0/3607296222 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1777762102 0 0) 0x7fca2006e010 con 0x7fca2015a250 2024-04-25T08:58:10.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.189+0000 7fca1ffff640 1 -- 172.21.15.43:0/3607296222 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca200b2be0 con 0x7fca20071a30 2024-04-25T08:58:10.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.189+0000 7f8218fd6640 1 --2- 172.21.15.43:0/3978080069 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f821415a840 0x7f821406d290 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f821406c190 tx=0x7f820802e780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28eb3224aea5d952 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.189+0000 7f29cb7fe640 1 --2- 172.21.15.43:0/8203479 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29cc0cbfc0 0x7f29cc06dc60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f29cc0cf630 tx=0x7f29bc02a0b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e806d4d490f0450 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.189+0000 7f82137fe640 1 -- 172.21.15.43:0/3978080069 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820803b070 con 0x7f821415a840 2024-04-25T08:58:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.189+0000 7fc7557b8640 1 --2- 172.21.15.43:0/1769979171 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc75015df50 0x7fc7501619f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc74c0097d0 tx=0x7fc74c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c84a44b266a29109 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:10.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.190+0000 7f82137fe640 1 -- 172.21.15.43:0/3978080069 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82080336b0 con 0x7f821415a840 2024-04-25T08:58:10.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.190+0000 7f82137fe640 1 -- 172.21.15.43:0/3978080069 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8208033a80 con 0x7f821415a840 2024-04-25T08:58:10.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.190+0000 7f29ca7fc640 1 -- 172.21.15.43:0/8203479 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc037070 con 0x7f29cc0cbfc0 2024-04-25T08:58:10.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.190+0000 7f29ca7fc640 1 -- 172.21.15.43:0/8203479 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29bc02ad40 con 0x7f29cc0cbfc0 2024-04-25T08:58:10.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.190+0000 7f29ca7fc640 1 -- 172.21.15.43:0/8203479 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29bc032660 con 0x7f29cc0cbfc0 2024-04-25T08:58:10.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.190+0000 7fca1ffff640 1 --2- 172.21.15.43:0/3607296222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca20071a30 0x7fca200b3810 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fca1800f370 tx=0x7fca1802ea90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a6f42a03342d7ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:10.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.190+0000 7fc747fff640 1 -- 172.21.15.43:0/1769979171 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74c037070 con 0x7fc75015df50 2024-04-25T08:58:10.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.190+0000 7fc747fff640 1 -- 172.21.15.43:0/1769979171 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74c032440 con 0x7fc75015df50 2024-04-25T08:58:10.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.190+0000 7fc747fff640 1 -- 172.21.15.43:0/1769979171 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74c032810 con 0x7fc75015df50 2024-04-25T08:58:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.190+0000 7fca1e7fc640 1 -- 172.21.15.43:0/3607296222 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca18037030 con 0x7fca20071a30 2024-04-25T08:58:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.190+0000 7fca1e7fc640 1 -- 172.21.15.43:0/3607296222 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca1800d2e0 con 0x7fca20071a30 2024-04-25T08:58:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.190+0000 7fca1e7fc640 1 -- 172.21.15.43:0/3607296222 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca140015f0 con 0x7fca20071a30 2024-04-25T08:58:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.190+0000 7f821b23a640 1 -- 172.21.15.43:0/3978080069 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f821415a840 msgr2=0x7f821406d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.190+0000 7f821b23a640 1 --2- 172.21.15.43:0/3978080069 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f821415a840 0x7f821406d290 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f821406c190 tx=0x7f820802e780 comp rx=0 tx=0).stop 2024-04-25T08:58:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.190+0000 7f821b23a640 1 -- 172.21.15.43:0/3978080069 shutdown_connections 2024-04-25T08:58:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.190+0000 7f821b23a640 1 --2- 172.21.15.43:0/3978080069 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8214071f90 0x7f82140b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.191+0000 7f29d1b9e640 1 -- 172.21.15.43:0/8203479 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29cc0cbfc0 msgr2=0x7f29cc06dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.191+0000 7fc757a1c640 1 -- 172.21.15.43:0/1769979171 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc75015df50 msgr2=0x7fc7501619f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:10.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.191+0000 7f29d1b9e640 1 --2- 172.21.15.43:0/8203479 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29cc0cbfc0 0x7f29cc06dc60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f29cc0cf630 tx=0x7f29bc02a0b0 comp rx=0 tx=0).stop 2024-04-25T08:58:10.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.191+0000 7fc757a1c640 1 --2- 172.21.15.43:0/1769979171 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc75015df50 0x7fc7501619f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc74c0097d0 tx=0x7fc74c02a210 comp rx=0 tx=0).stop 2024-04-25T08:58:10.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.191+0000 7f821b23a640 1 --2- 172.21.15.43:0/3978080069 >> v2:172.21.15.43:3300/0 conn(0x7f821406d7d0 0x7f8214071a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:10.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.191+0000 7f821b23a640 1 --2- 172.21.15.43:0/3978080069 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f821415a840 0x7f821406d290 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:10.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.191+0000 7f821b23a640 1 -- 172.21.15.43:0/3978080069 >> 172.21.15.43:0/3978080069 conn(0x7f82140b1480 msgr2=0x7f82140b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:10.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.191+0000 7f821b23a640 1 -- 172.21.15.43:0/3978080069 shutdown_connections 2024-04-25T08:58:10.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.191+0000 7f821b23a640 1 -- 172.21.15.43:0/3978080069 wait complete. 2024-04-25T08:58:10.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.191+0000 7fca25bce640 1 -- 172.21.15.43:0/3607296222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca20071a30 msgr2=0x7fca200b3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:10.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.191+0000 7fca25bce640 1 --2- 172.21.15.43:0/3607296222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca20071a30 0x7fca200b3810 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fca1800f370 tx=0x7fca1802ea90 comp rx=0 tx=0).stop 2024-04-25T08:58:10.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.191+0000 7fc757a1c640 1 -- 172.21.15.43:0/1769979171 shutdown_connections 2024-04-25T08:58:10.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.191+0000 7fc757a1c640 1 --2- 172.21.15.43:0/1769979171 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc75015df50 0x7fc7501619f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:10.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.191+0000 7f29d1b9e640 1 -- 172.21.15.43:0/8203479 shutdown_connections 2024-04-25T08:58:10.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.191+0000 7fc757a1c640 1 -- 172.21.15.43:0/1769979171 >> 172.21.15.43:0/1769979171 conn(0x7fc7500d3a30 msgr2=0x7fc7500d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:10.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.191+0000 7fca25bce640 1 -- 172.21.15.43:0/3607296222 shutdown_connections 2024-04-25T08:58:10.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.191+0000 7f29d1b9e640 1 --2- 172.21.15.43:0/8203479 >> v2:172.21.15.43:3300/0 conn(0x7f29cc0ccbb0 0x7f29cc0cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:10.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.191+0000 7f29d1b9e640 1 --2- 172.21.15.43:0/8203479 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29cc0cbfc0 0x7f29cc06dc60 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:10.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.191+0000 7fca25bce640 1 --2- 172.21.15.43:0/3607296222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca20071a30 0x7fca200b3810 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:10.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.191+0000 7fca25bce640 1 -- 172.21.15.43:0/3607296222 >> 172.21.15.43:0/3607296222 conn(0x7fca200b1480 msgr2=0x7fca200b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:10.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.191+0000 7f29d1b9e640 1 -- 172.21.15.43:0/8203479 >> 172.21.15.43:0/8203479 conn(0x7f29cc0b1480 msgr2=0x7f29cc0b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:10.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.191+0000 7fca25bce640 1 -- 172.21.15.43:0/3607296222 shutdown_connections 2024-04-25T08:58:10.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.191+0000 7fc757a1c640 1 -- 172.21.15.43:0/1769979171 shutdown_connections 2024-04-25T08:58:10.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.191+0000 7fca25bce640 1 -- 172.21.15.43:0/3607296222 wait complete. 2024-04-25T08:58:10.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.191+0000 7fc757a1c640 1 -- 172.21.15.43:0/1769979171 wait complete. 2024-04-25T08:58:10.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.191+0000 7f29d1b9e640 1 -- 172.21.15.43:0/8203479 shutdown_connections 2024-04-25T08:58:10.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.191+0000 7f29d1b9e640 1 -- 172.21.15.43:0/8203479 wait complete. 2024-04-25T08:58:10.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.192+0000 7f821b23a640 1 Processor -- start 2024-04-25T08:58:10.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.192+0000 7f821b23a640 1 -- start start 2024-04-25T08:58:10.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.192+0000 7fc757a1c640 1 Processor -- start 2024-04-25T08:58:10.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.192+0000 7fc757a1c640 1 -- start start 2024-04-25T08:58:10.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.192+0000 7fca25bce640 1 Processor -- start 2024-04-25T08:58:10.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.193+0000 7f29d1b9e640 1 Processor -- start 2024-04-25T08:58:10.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.193+0000 7fca25bce640 1 -- start start 2024-04-25T08:58:10.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.193+0000 7f29d1b9e640 1 -- start start 2024-04-25T08:58:10.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.193+0000 7fc757a1c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7500cf670 0x7fc7501f6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:10.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.193+0000 7f821b23a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f821406d7d0 0x7f82141532e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:10.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.193+0000 7f821b23a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8214071f90 0x7f821413ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:10.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.193+0000 7fc757a1c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc75015df50 0x7fc7500c8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:10.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.193+0000 7fc757a1c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc750161f30 0x7fc7500c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:10.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.193+0000 7fc757a1c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7500c77d0 con 0x7fc7500cf670 2024-04-25T08:58:10.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.193+0000 7f821b23a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f821415a840 0x7f8214151860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:10.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.193+0000 7fca25bce640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fca200715a0 0x7fca200adfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:10.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.193+0000 7fca25bce640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca20071a30 0x7fca201f2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:10.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.193+0000 7fca25bce640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca2015a250 0x7fca200b0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:10.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.193+0000 7f8213fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8214071f90 0x7f821413ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:10.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.193+0000 7fc757a1c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc75015e330 con 0x7fc75015df50 2024-04-25T08:58:10.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.193+0000 7fc757a1c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7500d2190 con 0x7fc750161f30 2024-04-25T08:58:10.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.193+0000 7fc7557b8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7500cf670 0x7fc7501f6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:10.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.193+0000 7fc7557b8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7500cf670 0x7fc7501f6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45614/0 (socket says 172.21.15.43:45614) 2024-04-25T08:58:10.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.193+0000 7fc7557b8640 1 -- 172.21.15.43:0/1106247965 learned_addr learned my addr 172.21.15.43:0/1106247965 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:10.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7f821b23a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8214071290 con 0x7f8214071f90 2024-04-25T08:58:10.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7f821b23a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82140723a0 con 0x7f821406d7d0 2024-04-25T08:58:10.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7fc755fb9640 1 --2- 172.21.15.43:0/1106247965 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc750161f30 0x7fc7500c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:10.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7f821b23a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8214071410 con 0x7f821415a840 2024-04-25T08:58:10.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7f8218fd6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f821406d7d0 0x7f82141532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:10.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.193+0000 7f8213fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8214071f90 0x7f821413ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45618/0 (socket says 172.21.15.43:45618) 2024-04-25T08:58:10.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7fc754fb7640 1 --2- 172.21.15.43:0/1106247965 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc75015df50 0x7fc7500c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:10.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7f8213fff640 1 -- 172.21.15.43:0/449988952 learned_addr learned my addr 172.21.15.43:0/449988952 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:10.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7f82197d7640 1 --2- 172.21.15.43:0/449988952 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f821415a840 0x7f8214151860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:10.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7fca1effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca20071a30 0x7fca201f2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:10.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7fca25bce640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca200af020 con 0x7fca20071a30 2024-04-25T08:58:10.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7fca25bce640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca200aeea0 con 0x7fca2015a250 2024-04-25T08:58:10.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7fca25bce640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca200b3d50 con 0x7fca200715a0 2024-04-25T08:58:10.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7fca1effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca20071a30 0x7fca201f2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45624/0 (socket says 172.21.15.43:45624) 2024-04-25T08:58:10.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7fca1effd640 1 -- 172.21.15.43:0/1692214651 learned_addr learned my addr 172.21.15.43:0/1692214651 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:10.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7fca1f7fe640 1 --2- 172.21.15.43:0/1692214651 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fca200715a0 0x7fca200adfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:10.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7fca1ffff640 1 --2- 172.21.15.43:0/1692214651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca2015a250 0x7fca200b0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:10.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7f29d1b9e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29cc0cbfc0 0x7f29cc1f18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:10.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7f29d1b9e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f29cc0ccbb0 0x7f29cc0c46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:10.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7fc7557b8640 1 -- 172.21.15.43:0/1106247965 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc750161f30 msgr2=0x7fc7500c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:10.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7fc7557b8640 1 --2- 172.21.15.43:0/1106247965 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc750161f30 0x7fc7500c8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:10.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7fc7557b8640 1 -- 172.21.15.43:0/1106247965 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc75015df50 msgr2=0x7fc7500c8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:10.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7fc7557b8640 1 --2- 172.21.15.43:0/1106247965 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc75015df50 0x7fc7500c8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:10.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7f29d1b9e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29cc15f5e0 0x7f29cc0c4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:10.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7f8218fd6640 1 -- 172.21.15.43:0/449988952 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f821415a840 msgr2=0x7f8214151860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:10.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7f8218fd6640 1 --2- 172.21.15.43:0/449988952 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f821415a840 0x7f8214151860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:10.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7f8218fd6640 1 -- 172.21.15.43:0/449988952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8214071f90 msgr2=0x7f821413ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:10.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7f8218fd6640 1 --2- 172.21.15.43:0/449988952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8214071f90 0x7f821413ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:10.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7f8218fd6640 1 -- 172.21.15.43:0/449988952 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8214152240 con 0x7f821406d7d0 2024-04-25T08:58:10.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7fc7557b8640 1 -- 172.21.15.43:0/1106247965 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7501f70c0 con 0x7fc7500cf670 2024-04-25T08:58:10.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7f29cb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29cc0cbfc0 0x7f29cc1f18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:10.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7f82197d7640 1 --2- 172.21.15.43:0/449988952 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f821415a840 0x7f8214151860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:58:10.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7f29d1b9e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29cc0b35e0 con 0x7f29cc0cbfc0 2024-04-25T08:58:10.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7f29cb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29cc0cbfc0 0x7f29cc1f18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45638/0 (socket says 172.21.15.43:45638) 2024-04-25T08:58:10.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7f29cb7fe640 1 -- 172.21.15.43:0/3371648123 learned_addr learned my addr 172.21.15.43:0/3371648123 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:10.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7f29d1b9e640 1 -- 172.21.15.43:0/3371648123 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29cc0b3300 con 0x7f29cc15f5e0 2024-04-25T08:58:10.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7f29d1b9e640 1 -- 172.21.15.43:0/3371648123 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29cc06e1a0 con 0x7f29cc0ccbb0 2024-04-25T08:58:10.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7f29cbfff640 1 --2- 172.21.15.43:0/3371648123 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29cc15f5e0 0x7f29cc0c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:10.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7fc7557b8640 1 --2- 172.21.15.43:0/1106247965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7500cf670 0x7fc7501f6b80 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fc74c0098f0 tx=0x7fc74c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:10.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7fca1f7fe640 1 -- 172.21.15.43:0/1692214651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca2015a250 msgr2=0x7fca200b0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:10.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7fca1f7fe640 1 --2- 172.21.15.43:0/1692214651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca2015a250 0x7fca200b0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:10.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7fca1f7fe640 1 -- 172.21.15.43:0/1692214651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca20071a30 msgr2=0x7fca201f2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:10.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7fca1f7fe640 1 --2- 172.21.15.43:0/1692214651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca20071a30 0x7fca201f2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:10.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7fca1f7fe640 1 -- 172.21.15.43:0/1692214651 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca201f3540 con 0x7fca200715a0 2024-04-25T08:58:10.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7fca1effd640 1 --2- 172.21.15.43:0/1692214651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca20071a30 0x7fca201f2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:58:10.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7f8218fd6640 1 --2- 172.21.15.43:0/449988952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f821406d7d0 0x7f82141532e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8208035000 tx=0x7f82080046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:10.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.194+0000 7f29caffd640 1 --2- 172.21.15.43:0/3371648123 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f29cc0ccbb0 0x7f29cc0c46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:10.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.195+0000 7f82117fa640 1 -- 172.21.15.43:0/449988952 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8208045030 con 0x7f821406d7d0 2024-04-25T08:58:10.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.195+0000 7f82117fa640 1 -- 172.21.15.43:0/449988952 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82080334b0 con 0x7f821406d7d0 2024-04-25T08:58:10.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.195+0000 7f821b23a640 1 -- 172.21.15.43:0/449988952 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8214152520 con 0x7f821406d7d0 2024-04-25T08:58:10.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.195+0000 7fca1f7fe640 1 --2- 172.21.15.43:0/1692214651 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fca200715a0 0x7fca200adfd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fca1000c3e0 tx=0x7fca1000c7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:10.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.195+0000 7f29cbfff640 1 -- 172.21.15.43:0/3371648123 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f29cc0ccbb0 msgr2=0x7f29cc0c46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:10.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.195+0000 7f29cbfff640 1 --2- 172.21.15.43:0/3371648123 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f29cc0ccbb0 0x7f29cc0c46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:10.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.195+0000 7f29cbfff640 1 -- 172.21.15.43:0/3371648123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29cc0cbfc0 msgr2=0x7f29cc1f18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:10.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.195+0000 7f29cbfff640 1 --2- 172.21.15.43:0/3371648123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29cc0cbfc0 0x7f29cc1f18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:10.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.195+0000 7f29cbfff640 1 -- 172.21.15.43:0/3371648123 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29cc1f2150 con 0x7f29cc15f5e0 2024-04-25T08:58:10.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.195+0000 7fc745ffb640 1 -- 172.21.15.43:0/1106247965 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74c037050 con 0x7fc7500cf670 2024-04-25T08:58:10.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.195+0000 7fc745ffb640 1 -- 172.21.15.43:0/1106247965 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74c043070 con 0x7fc7500cf670 2024-04-25T08:58:10.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.195+0000 7f82117fa640 1 -- 172.21.15.43:0/449988952 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f820803b040 con 0x7f821406d7d0 2024-04-25T08:58:10.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.195+0000 7fc745ffb640 1 -- 172.21.15.43:0/1106247965 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74c03cb30 con 0x7fc7500cf670 2024-04-25T08:58:10.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.195+0000 7f29cb7fe640 1 --2- 172.21.15.43:0/3371648123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29cc0cbfc0 0x7f29cc1f18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:58:10.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.195+0000 7f821b23a640 1 -- 172.21.15.43:0/449988952 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8214152ae0 con 0x7f821406d7d0 2024-04-25T08:58:10.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.195+0000 7fc757a1c640 1 -- 172.21.15.43:0/1106247965 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc75015bca0 con 0x7fc7500cf670 2024-04-25T08:58:10.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.195+0000 7f29cbfff640 1 --2- 172.21.15.43:0/3371648123 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29cc15f5e0 0x7f29cc0c4c30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f29c400c780 tx=0x7f29c400cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:10.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.195+0000 7fca07fff640 1 -- 172.21.15.43:0/1692214651 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca10017070 con 0x7fca200715a0 2024-04-25T08:58:10.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.195+0000 7fca25bce640 1 -- 172.21.15.43:0/1692214651 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca201f3820 con 0x7fca200715a0 2024-04-25T08:58:10.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.195+0000 7fca07fff640 1 -- 172.21.15.43:0/1692214651 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca100035f0 con 0x7fca200715a0 2024-04-25T08:58:10.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.195+0000 7fca25bce640 1 -- 172.21.15.43:0/1692214651 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca201f3230 con 0x7fca200715a0 2024-04-25T08:58:10.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.195+0000 7fca07fff640 1 -- 172.21.15.43:0/1692214651 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca10017070 con 0x7fca200715a0 2024-04-25T08:58:10.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.196+0000 7f29b3fff640 1 -- 172.21.15.43:0/3371648123 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c4017070 con 0x7f29cc15f5e0 2024-04-25T08:58:10.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.196+0000 7f29b3fff640 1 -- 172.21.15.43:0/3371648123 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29c4003da0 con 0x7f29cc15f5e0 2024-04-25T08:58:10.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.196+0000 7f29b3fff640 1 -- 172.21.15.43:0/3371648123 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c40052b0 con 0x7f29cc15f5e0 2024-04-25T08:58:10.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.196+0000 7f29d1b9e640 1 -- 172.21.15.43:0/3371648123 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29cc1f2430 con 0x7f29cc15f5e0 2024-04-25T08:58:10.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.196+0000 7f29d1b9e640 1 -- 172.21.15.43:0/3371648123 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29cc1f1de0 con 0x7f29cc15f5e0 2024-04-25T08:58:10.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.196+0000 7f821b23a640 1 -- 172.21.15.43:0/449988952 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8214074ef0 con 0x7f821406d7d0 2024-04-25T08:58:10.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.196+0000 7fc757a1c640 1 -- 172.21.15.43:0/1106247965 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc75015c260 con 0x7fc7500cf670 2024-04-25T08:58:10.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.197+0000 7fca25bce640 1 -- 172.21.15.43:0/1692214651 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca201f39f0 con 0x7fca200715a0 2024-04-25T08:58:10.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.197+0000 7f29d1b9e640 1 -- 172.21.15.43:0/3371648123 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29cc131990 con 0x7f29cc15f5e0 2024-04-25T08:58:10.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.197+0000 7f82117fa640 1 -- 172.21.15.43:0/449988952 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8208048070 con 0x7f821406d7d0 2024-04-25T08:58:10.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.197+0000 7fc757a1c640 1 -- 172.21.15.43:0/1106247965 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7500680f0 con 0x7fc7500cf670 2024-04-25T08:58:10.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.198+0000 7fc745ffb640 1 -- 172.21.15.43:0/1106247965 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc74c03ccd0 con 0x7fc7500cf670 2024-04-25T08:58:10.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.198+0000 7f82117fa640 1 --2- 172.21.15.43:0/449988952 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f81f806b4d0 0x7f81f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:10.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.198+0000 7f82117fa640 1 -- 172.21.15.43:0/449988952 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82080aeb20 con 0x7f821406d7d0 2024-04-25T08:58:10.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.198+0000 7f8213fff640 1 --2- 172.21.15.43:0/449988952 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f81f806b4d0 0x7f81f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:10.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.198+0000 7fc745ffb640 1 --2- 172.21.15.43:0/1106247965 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc73406b520 0x7fc73406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:10.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.198+0000 7fc745ffb640 1 -- 172.21.15.43:0/1106247965 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc74c0aac90 con 0x7fc7500cf670 2024-04-25T08:58:10.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.198+0000 7fc754fb7640 1 --2- 172.21.15.43:0/1106247965 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc73406b520 0x7fc73406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:10.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.198+0000 7fca07fff640 1 -- 172.21.15.43:0/1692214651 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fca1001a070 con 0x7fca200715a0 2024-04-25T08:58:10.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.198+0000 7f29b3fff640 1 -- 172.21.15.43:0/3371648123 <== mon.1 v2:172.21.15.193:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f29c401a070 con 0x7f29cc15f5e0 2024-04-25T08:58:10.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.199+0000 7fca07fff640 1 --2- 172.21.15.43:0/1692214651 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fca0c06b6d0 0x7fca0c06db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:10.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.199+0000 7f8213fff640 1 --2- 172.21.15.43:0/449988952 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f81f806b4d0 0x7f81f806d990 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f82141f2870 tx=0x7f8200008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:10.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.199+0000 7f29b3fff640 1 --2- 172.21.15.43:0/3371648123 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f29b806f960 0x7f29b8071e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:10.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.199+0000 7fc754fb7640 1 --2- 172.21.15.43:0/1106247965 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc73406b520 0x7fc73406d9e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc7500c88a0 tx=0x7fc740009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:10.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.199+0000 7fca07fff640 1 -- 172.21.15.43:0/1692214651 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fca100890e0 con 0x7fca200715a0 2024-04-25T08:58:10.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.199+0000 7fca1effd640 1 --2- 172.21.15.43:0/1692214651 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fca0c06b6d0 0x7fca0c06db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:10.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.199+0000 7f29b3fff640 1 -- 172.21.15.43:0/3371648123 <== mon.1 v2:172.21.15.193:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f29c4089710 con 0x7f29cc15f5e0 2024-04-25T08:58:10.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.199+0000 7f29cb7fe640 1 --2- 172.21.15.43:0/3371648123 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f29b806f960 0x7f29b8071e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:10.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.200+0000 7fca1effd640 1 --2- 172.21.15.43:0/1692214651 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fca0c06b6d0 0x7fca0c06db90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fca14001660 tx=0x7fca140015b0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:10.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.200+0000 7f29cb7fe640 1 --2- 172.21.15.43:0/3371648123 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f29b806f960 0x7f29b8071e20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f29bc02a0f0 tx=0x7f29bc02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:10.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.203+0000 7fc745ffb640 1 -- 172.21.15.43:0/1106247965 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc74c079c80 con 0x7fc7500cf670 2024-04-25T08:58:10.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.203+0000 7f82117fa640 1 -- 172.21.15.43:0/449988952 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f820804f070 con 0x7f821406d7d0 2024-04-25T08:58:10.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.203+0000 7fca07fff640 1 -- 172.21.15.43:0/1692214651 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fca100577f0 con 0x7fca200715a0 2024-04-25T08:58:10.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.207+0000 7f29b3fff640 1 -- 172.21.15.43:0/3371648123 <== mon.1 v2:172.21.15.193:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f29c4057fa0 con 0x7f29cc15f5e0 2024-04-25T08:58:10.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.460+0000 7fc757a1c640 1 -- 172.21.15.43:0/1106247965 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "b"} v 0) v1 -- 0x7fc7500ca400 con 0x7fc7500cf670 2024-04-25T08:58:10.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.467+0000 7f821b23a640 1 -- 172.21.15.43:0/449988952 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "c"} v 0) v1 -- 0x7f82140b49b0 con 0x7f821406d7d0 2024-04-25T08:58:10.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.471+0000 7fca25bce640 1 -- 172.21.15.43:0/1692214651 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "d"} v 0) v1 -- 0x7fca20074ef0 con 0x7fca200715a0 2024-04-25T08:58:10.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:10.474+0000 7f29d1b9e640 1 -- 172.21.15.43:0/3371648123 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "a"} v 0) v1 -- 0x7f29cc074ef0 con 0x7f29cc15f5e0 2024-04-25T08:58:11.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:11.413+0000 7fc745ffb640 1 -- 172.21.15.43:0/1106247965 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "b"}]=0 failed mds gid 5950 v32) v1 ==== 95+0+0 (secure 0 0 0) 0x7fc74c079580 con 0x7fc7500cf670 2024-04-25T08:58:11.403 INFO:teuthology.orchestra.run.smithi043.stderr:failed mds gid 5950 2024-04-25T08:58:11.404 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-04-25T08:58:11.415+0000 7fe4f8993640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:58:11.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:11.415+0000 7fc757a1c640 1 -- 172.21.15.43:0/1106247965 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc73406b520 msgr2=0x7fc73406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:11.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:11.415+0000 7fc757a1c640 1 --2- 172.21.15.43:0/1106247965 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc73406b520 0x7fc73406d9e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc7500c88a0 tx=0x7fc740009040 comp rx=0 tx=0).stop 2024-04-25T08:58:11.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:11.415+0000 7fc757a1c640 1 -- 172.21.15.43:0/1106247965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7500cf670 msgr2=0x7fc7501f6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:11.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:11.415+0000 7fc757a1c640 1 --2- 172.21.15.43:0/1106247965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7500cf670 0x7fc7501f6b80 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fc74c0098f0 tx=0x7fc74c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:58:11.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:11.415+0000 7fc757a1c640 1 -- 172.21.15.43:0/1106247965 shutdown_connections 2024-04-25T08:58:11.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:11.415+0000 7fc757a1c640 1 --2- 172.21.15.43:0/1106247965 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc750161f30 0x7fc7500c8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:11.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:11.415+0000 7fc757a1c640 1 --2- 172.21.15.43:0/1106247965 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc73406b520 0x7fc73406d9e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:11.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:11.415+0000 7fc757a1c640 1 --2- 172.21.15.43:0/1106247965 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc75015df50 0x7fc7500c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:11.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:11.415+0000 7fc757a1c640 1 --2- 172.21.15.43:0/1106247965 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7500cf670 0x7fc7501f6b80 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:11.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:11.415+0000 7fc757a1c640 1 -- 172.21.15.43:0/1106247965 >> 172.21.15.43:0/1106247965 conn(0x7fc7500d3a30 msgr2=0x7fc7500c94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:11.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:11.415+0000 7fc757a1c640 1 -- 172.21.15.43:0/1106247965 shutdown_connections 2024-04-25T08:58:11.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:11.415+0000 7fc757a1c640 1 -- 172.21.15.43:0/1106247965 wait complete. 2024-04-25T08:58:11.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:11.419+0000 7f82117fa640 1 -- 172.21.15.43:0/449988952 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "c"}]=0 failed mds gid 6540 v33) v1 ==== 95+0+0 (secure 0 0 0) 0x7f8208033680 con 0x7f821406d7d0 2024-04-25T08:58:11.409 INFO:teuthology.orchestra.run.smithi043.stderr:failed mds gid 6540 2024-04-25T08:58:11.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:11.422+0000 7f821b23a640 1 -- 172.21.15.43:0/449988952 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f81f806b4d0 msgr2=0x7f81f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:11.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:11.422+0000 7f821b23a640 1 --2- 172.21.15.43:0/449988952 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f81f806b4d0 0x7f81f806d990 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f82141f2870 tx=0x7f8200008040 comp rx=0 tx=0).stop 2024-04-25T08:58:11.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:11.422+0000 7f821b23a640 1 -- 172.21.15.43:0/449988952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f821406d7d0 msgr2=0x7f82141532e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:11.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:11.422+0000 7f821b23a640 1 --2- 172.21.15.43:0/449988952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f821406d7d0 0x7f82141532e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f8208035000 tx=0x7f82080046c0 comp rx=0 tx=0).stop 2024-04-25T08:58:11.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:11.422+0000 7f821b23a640 1 -- 172.21.15.43:0/449988952 shutdown_connections 2024-04-25T08:58:11.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:11.422+0000 7f821b23a640 1 --2- 172.21.15.43:0/449988952 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f821415a840 0x7f8214151860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:11.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:11.422+0000 7f821b23a640 1 --2- 172.21.15.43:0/449988952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8214071f90 0x7f821413ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:11.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:11.422+0000 7f821b23a640 1 --2- 172.21.15.43:0/449988952 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f81f806b4d0 0x7f81f806d990 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:11.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:11.422+0000 7f821b23a640 1 --2- 172.21.15.43:0/449988952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f821406d7d0 0x7f82141532e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:11.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:11.422+0000 7f821b23a640 1 -- 172.21.15.43:0/449988952 >> 172.21.15.43:0/449988952 conn(0x7f82140b1480 msgr2=0x7f8214157f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:11.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:11.422+0000 7f821b23a640 1 -- 172.21.15.43:0/449988952 shutdown_connections 2024-04-25T08:58:11.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:11.422+0000 7f821b23a640 1 -- 172.21.15.43:0/449988952 wait complete. 2024-04-25T08:58:11.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:11.423+0000 7fca07fff640 1 -- 172.21.15.43:0/1692214651 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "d"}]=0 failed mds gid 5953 v34) v1 ==== 95+0+0 (secure 0 0 0) 0x7fca1005b890 con 0x7fca200715a0 2024-04-25T08:58:11.413 INFO:teuthology.orchestra.run.smithi043.stderr:failed mds gid 5953 2024-04-25T08:58:11.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:11.427+0000 7fca25bce640 1 -- 172.21.15.43:0/1692214651 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fca0c06b6d0 msgr2=0x7fca0c06db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:11.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:11.427+0000 7fca25bce640 1 --2- 172.21.15.43:0/1692214651 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fca0c06b6d0 0x7fca0c06db90 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fca14001660 tx=0x7fca140015b0 comp rx=0 tx=0).stop 2024-04-25T08:58:11.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:11.427+0000 7fca25bce640 1 -- 172.21.15.43:0/1692214651 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fca200715a0 msgr2=0x7fca200adfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:11.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:11.427+0000 7fca25bce640 1 --2- 172.21.15.43:0/1692214651 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fca200715a0 0x7fca200adfd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fca1000c3e0 tx=0x7fca1000c7e0 comp rx=0 tx=0).stop 2024-04-25T08:58:11.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:11.427+0000 7fca25bce640 1 -- 172.21.15.43:0/1692214651 shutdown_connections 2024-04-25T08:58:11.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:11.427+0000 7fca25bce640 1 --2- 172.21.15.43:0/1692214651 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fca0c06b6d0 0x7fca0c06db90 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:11.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:11.427+0000 7fca25bce640 1 --2- 172.21.15.43:0/1692214651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fca2015a250 0x7fca200b0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:11.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:11.427+0000 7fca25bce640 1 --2- 172.21.15.43:0/1692214651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fca20071a30 0x7fca201f2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:11.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:11.427+0000 7fca25bce640 1 --2- 172.21.15.43:0/1692214651 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fca200715a0 0x7fca200adfd0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:11.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:11.427+0000 7fca25bce640 1 -- 172.21.15.43:0/1692214651 >> 172.21.15.43:0/1692214651 conn(0x7fca200b1480 msgr2=0x7fca200b3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:11.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:11.427+0000 7fca25bce640 1 -- 172.21.15.43:0/1692214651 shutdown_connections 2024-04-25T08:58:11.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:11.427+0000 7fca25bce640 1 -- 172.21.15.43:0/1692214651 wait complete. 2024-04-25T08:58:11.420 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T08:58:11.420 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T08:58:11.422 INFO:tasks.ceph.mds.b:Started 2024-04-25T08:58:11.423 DEBUG:teuthology.parallel:result is None 2024-04-25T08:58:11.429 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T08:58:11.429 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T08:58:11.434 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T08:58:11.434 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T08:58:11.437 INFO:tasks.ceph.mds.d:Started 2024-04-25T08:58:11.437 DEBUG:teuthology.parallel:result is None 2024-04-25T08:58:11.476 INFO:tasks.ceph.mds.c:Started 2024-04-25T08:58:11.477 DEBUG:teuthology.parallel:result is None 2024-04-25T08:58:11.501 INFO:tasks.ceph.mds.b.smithi193.stdout:starting mds.b at 2024-04-25T08:58:11.512 INFO:tasks.ceph.mds.d.smithi193.stdout:starting mds.d at 2024-04-25T08:58:11.551 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-04-25T08:58:12.402 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-04-25T08:58:12.412+0000 7fe4f8993640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:58:12.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.417+0000 7f29b3fff640 1 -- 172.21.15.43:0/3371648123 <== mon.1 v2:172.21.15.193:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "a"}]=0 failed mds gid 6537 v35) v1 ==== 95+0+0 (secure 0 0 0) 0x7f29c405c040 con 0x7f29cc15f5e0 2024-04-25T08:58:12.406 INFO:teuthology.orchestra.run.smithi043.stderr:failed mds gid 6537 2024-04-25T08:58:12.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.421+0000 7f29d1b9e640 1 -- 172.21.15.43:0/3371648123 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f29b806f960 msgr2=0x7f29b8071e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:12.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.421+0000 7f29d1b9e640 1 --2- 172.21.15.43:0/3371648123 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f29b806f960 0x7f29b8071e20 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f29bc02a0f0 tx=0x7f29bc02e000 comp rx=0 tx=0).stop 2024-04-25T08:58:12.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.421+0000 7f29d1b9e640 1 -- 172.21.15.43:0/3371648123 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29cc15f5e0 msgr2=0x7f29cc0c4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:12.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.421+0000 7f29d1b9e640 1 --2- 172.21.15.43:0/3371648123 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29cc15f5e0 0x7f29cc0c4c30 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f29c400c780 tx=0x7f29c400cc40 comp rx=0 tx=0).stop 2024-04-25T08:58:12.410 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.421+0000 7f29d1b9e640 1 -- 172.21.15.43:0/3371648123 shutdown_connections 2024-04-25T08:58:12.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.421+0000 7f29d1b9e640 1 --2- 172.21.15.43:0/3371648123 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f29b806f960 0x7f29b8071e20 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:12.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.421+0000 7f29d1b9e640 1 --2- 172.21.15.43:0/3371648123 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f29cc15f5e0 0x7f29cc0c4c30 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:12.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.421+0000 7f29d1b9e640 1 --2- 172.21.15.43:0/3371648123 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f29cc0ccbb0 0x7f29cc0c46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:12.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.421+0000 7f29d1b9e640 1 --2- 172.21.15.43:0/3371648123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f29cc0cbfc0 0x7f29cc1f18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:12.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.421+0000 7f29d1b9e640 1 -- 172.21.15.43:0/3371648123 >> 172.21.15.43:0/3371648123 conn(0x7f29cc0b1480 msgr2=0x7f29cc1445a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:12.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.421+0000 7f29d1b9e640 1 -- 172.21.15.43:0/3371648123 shutdown_connections 2024-04-25T08:58:12.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.421+0000 7f29d1b9e640 1 -- 172.21.15.43:0/3371648123 wait complete. 2024-04-25T08:58:12.428 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T08:58:12.429 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T08:58:12.472 INFO:tasks.ceph.mds.a:Started 2024-04-25T08:58:12.472 DEBUG:teuthology.parallel:result is None 2024-04-25T08:58:12.472 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:58:12.552 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2024-04-25T08:58:12.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.592+0000 7fe21a9b7640 1 Processor -- start 2024-04-25T08:58:12.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.592+0000 7fe21a9b7640 1 -- start start 2024-04-25T08:58:12.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.593+0000 7fe21a9b7640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe21415dd00 0x7fe2140d2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:12.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.593+0000 7fe21a9b7640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe2140d33f0 0x7fe2140ce260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:12.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.593+0000 7fe21a9b7640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2140cec30 con 0x7fe2140ce7a0 2024-04-25T08:58:12.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.593+0000 7fe21a9b7640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2140d3850 con 0x7fe2140d33f0 2024-04-25T08:58:12.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.593+0000 7fe21a9b7640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2140d24e0 con 0x7fe21415dd00 2024-04-25T08:58:12.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.593+0000 7fe2137fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe2140d33f0 0x7fe2140ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:12.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.593+0000 7fe218f54640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe2140ce7a0 0x7fe2140b5480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47166/0 (socket says 172.21.15.43:47166) 2024-04-25T08:58:12.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.593+0000 7fe218f54640 1 -- 172.21.15.43:0/1315232170 learned_addr learned my addr 172.21.15.43:0/1315232170 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:12.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.593+0000 7fe213fff640 1 --2- 172.21.15.43:0/1315232170 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe21415dd00 0x7fe2140d2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:12.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.593+0000 7fe2137fe640 1 -- 172.21.15.43:0/1315232170 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe21415dd00 msgr2=0x7fe2140d2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:12.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.593+0000 7fe2137fe640 1 --2- 172.21.15.43:0/1315232170 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe21415dd00 0x7fe2140d2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:12.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.593+0000 7fe2137fe640 1 -- 172.21.15.43:0/1315232170 >> v1:172.21.15.43:6789/0 conn(0x7fe2140ce7a0 legacy=0x7fe2140b5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:12.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.593+0000 7fe2137fe640 1 -- 172.21.15.43:0/1315232170 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe21415e0e0 con 0x7fe2140d33f0 2024-04-25T08:58:12.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.593+0000 7fe212ffd640 1 -- 172.21.15.43:0/1315232170 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2085205703 0 0) 0x7fe20c001490 con 0x7fe2140ce7a0 2024-04-25T08:58:12.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.594+0000 7fe2137fe640 1 --2- 172.21.15.43:0/1315232170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2140d33f0 0x7fe2140ce260 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fe2040097a0 tx=0x7fe20402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2866ea7657f7dd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:12.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.594+0000 7fe212ffd640 1 -- 172.21.15.43:0/1315232170 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe204037070 con 0x7fe2140d33f0 2024-04-25T08:58:12.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.594+0000 7fe212ffd640 1 -- 172.21.15.43:0/1315232170 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe204032440 con 0x7fe2140d33f0 2024-04-25T08:58:12.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.594+0000 7fe212ffd640 1 -- 172.21.15.43:0/1315232170 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe204032810 con 0x7fe2140d33f0 2024-04-25T08:58:12.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.595+0000 7fe21a9b7640 1 -- 172.21.15.43:0/1315232170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2140d33f0 msgr2=0x7fe2140ce260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:12.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.595+0000 7fe21a9b7640 1 --2- 172.21.15.43:0/1315232170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2140d33f0 0x7fe2140ce260 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fe2040097a0 tx=0x7fe20402a180 comp rx=0 tx=0).stop 2024-04-25T08:58:12.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.595+0000 7fe21a9b7640 1 -- 172.21.15.43:0/1315232170 shutdown_connections 2024-04-25T08:58:12.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.595+0000 7fe21a9b7640 1 --2- 172.21.15.43:0/1315232170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2140d33f0 0x7fe2140ce260 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:12.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.595+0000 7fe21a9b7640 1 --2- 172.21.15.43:0/1315232170 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe21415dd00 0x7fe2140d2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:12.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.595+0000 7fe21a9b7640 1 -- 172.21.15.43:0/1315232170 >> 172.21.15.43:0/1315232170 conn(0x7fe21406cbd0 msgr2=0x7fe21406cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:12.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.595+0000 7fe21a9b7640 1 -- 172.21.15.43:0/1315232170 shutdown_connections 2024-04-25T08:58:12.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.595+0000 7fe21a9b7640 1 -- 172.21.15.43:0/1315232170 wait complete. 2024-04-25T08:58:12.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.596+0000 7fe21a9b7640 1 Processor -- start 2024-04-25T08:58:12.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.596+0000 7fe21a9b7640 1 -- start start 2024-04-25T08:58:12.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.597+0000 7fe21a9b7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2140ce7a0 0x7fe214148db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:12.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.597+0000 7fe21a9b7640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2140d33f0 0x7fe214157f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:12.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.597+0000 7fe21a9b7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe21415dd00 0x7fe214158450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:12.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.597+0000 7fe21a9b7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2140d1e90 con 0x7fe21415dd00 2024-04-25T08:58:12.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.597+0000 7fe21a9b7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2140d1bb0 con 0x7fe2140ce7a0 2024-04-25T08:58:12.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.597+0000 7fe21a9b7640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2140d2170 con 0x7fe2140d33f0 2024-04-25T08:58:12.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.597+0000 7fe218f54640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe21415dd00 0x7fe214158450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:12.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.597+0000 7fe218f54640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe21415dd00 0x7fe214158450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45702/0 (socket says 172.21.15.43:45702) 2024-04-25T08:58:12.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.597+0000 7fe218f54640 1 -- 172.21.15.43:0/2282586291 learned_addr learned my addr 172.21.15.43:0/2282586291 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:12.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.597+0000 7fe213fff640 1 --2- 172.21.15.43:0/2282586291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2140ce7a0 0x7fe214148db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:12.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.597+0000 7fe2137fe640 1 --2- 172.21.15.43:0/2282586291 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2140d33f0 0x7fe214157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:12.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.597+0000 7fe218f54640 1 -- 172.21.15.43:0/2282586291 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2140d33f0 msgr2=0x7fe214157f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:12.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.597+0000 7fe218f54640 1 --2- 172.21.15.43:0/2282586291 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2140d33f0 0x7fe214157f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:12.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.597+0000 7fe218f54640 1 -- 172.21.15.43:0/2282586291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2140ce7a0 msgr2=0x7fe214148db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:12.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.597+0000 7fe218f54640 1 --2- 172.21.15.43:0/2282586291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2140ce7a0 0x7fe214148db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:12.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.597+0000 7fe218f54640 1 -- 172.21.15.43:0/2282586291 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2140b69c0 con 0x7fe21415dd00 2024-04-25T08:58:12.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.598+0000 7fe2137fe640 1 --2- 172.21.15.43:0/2282586291 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2140d33f0 0x7fe214157f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:12.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.598+0000 7fe218f54640 1 --2- 172.21.15.43:0/2282586291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe21415dd00 0x7fe214158450 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fe20c00c7f0 tx=0x7fe20c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:12.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.598+0000 7fe210ff9640 1 -- 172.21.15.43:0/2282586291 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20c018070 con 0x7fe21415dd00 2024-04-25T08:58:12.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.598+0000 7fe210ff9640 1 -- 172.21.15.43:0/2282586291 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe20c00a5e0 con 0x7fe21415dd00 2024-04-25T08:58:12.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.598+0000 7fe21a9b7640 1 -- 172.21.15.43:0/2282586291 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2140b6ce0 con 0x7fe21415dd00 2024-04-25T08:58:12.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.598+0000 7fe210ff9640 1 -- 172.21.15.43:0/2282586291 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20c00aa10 con 0x7fe21415dd00 2024-04-25T08:58:12.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.598+0000 7fe21a9b7640 1 -- 172.21.15.43:0/2282586291 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2140b7520 con 0x7fe21415dd00 2024-04-25T08:58:12.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.599+0000 7fe21a9b7640 1 -- 172.21.15.43:0/2282586291 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe214074ef0 con 0x7fe21415dd00 2024-04-25T08:58:12.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.599+0000 7fe210ff9640 1 -- 172.21.15.43:0/2282586291 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe20c01b070 con 0x7fe21415dd00 2024-04-25T08:58:12.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.600+0000 7fe210ff9640 1 --2- 172.21.15.43:0/2282586291 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe20006b500 0x7fe20006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:12.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.600+0000 7fe210ff9640 1 -- 172.21.15.43:0/2282586291 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fe20c0898a0 con 0x7fe21415dd00 2024-04-25T08:58:12.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.600+0000 7fe213fff640 1 --2- 172.21.15.43:0/2282586291 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe20006b500 0x7fe20006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:12.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.601+0000 7fe213fff640 1 --2- 172.21.15.43:0/2282586291 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe20006b500 0x7fe20006d9c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe208006fd0 tx=0x7fe208008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:12.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.604+0000 7fe210ff9640 1 -- 172.21.15.43:0/2282586291 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe2140b6ce0 con 0x7fe21415dd00 2024-04-25T08:58:12.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.821+0000 7fe21a9b7640 1 -- 172.21.15.43:0/2282586291 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe214068080 con 0x7fe21415dd00 2024-04-25T08:58:12.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.822+0000 7fe210ff9640 1 -- 172.21.15.43:0/2282586291 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+4148 (secure 0 0 0) 0x7fe2140cec30 con 0x7fe21415dd00 2024-04-25T08:58:12.811 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:58:12.811 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":36,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5765,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/594587590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":594587590},{"type":"v1","addr":"172.21.15.193:6835","nonce":594587590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6187,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/681300720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":681300720},{"type":"v1","addr":"172.21.15.193:6837","nonce":681300720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:57:54.590369+0000","modified":"2024-04-25T08:58:12.417098+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6870},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6870":{"gid":6870,"name":"c","rank":0,"incarnation":36,"state":"up:replay","state_seq":1,"addr":"172.21.15.43:6833/1866869111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1866869111},{"type":"v1","addr":"172.21.15.43:6833","nonce":1866869111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:58:12.812 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 36 2024-04-25T08:58:12.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.824+0000 7fe21a9b7640 1 -- 172.21.15.43:0/2282586291 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe20006b500 msgr2=0x7fe20006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:12.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.824+0000 7fe21a9b7640 1 --2- 172.21.15.43:0/2282586291 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe20006b500 0x7fe20006d9c0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fe208006fd0 tx=0x7fe208008040 comp rx=0 tx=0).stop 2024-04-25T08:58:12.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.824+0000 7fe21a9b7640 1 -- 172.21.15.43:0/2282586291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe21415dd00 msgr2=0x7fe214158450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:12.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.824+0000 7fe21a9b7640 1 --2- 172.21.15.43:0/2282586291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe21415dd00 0x7fe214158450 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fe20c00c7f0 tx=0x7fe20c00ccb0 comp rx=0 tx=0).stop 2024-04-25T08:58:12.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.824+0000 7fe21a9b7640 1 -- 172.21.15.43:0/2282586291 shutdown_connections 2024-04-25T08:58:12.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.824+0000 7fe21a9b7640 1 --2- 172.21.15.43:0/2282586291 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe21415dd00 0x7fe214158450 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:12.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.824+0000 7fe21a9b7640 1 --2- 172.21.15.43:0/2282586291 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe2140d33f0 0x7fe214157f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:12.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.824+0000 7fe21a9b7640 1 --2- 172.21.15.43:0/2282586291 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe20006b500 0x7fe20006d9c0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:12.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.824+0000 7fe21a9b7640 1 --2- 172.21.15.43:0/2282586291 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe2140ce7a0 0x7fe214148db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:12.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.824+0000 7fe21a9b7640 1 -- 172.21.15.43:0/2282586291 >> 172.21.15.43:0/2282586291 conn(0x7fe21406cbd0 msgr2=0x7fe21415c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:12.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.824+0000 7fe21a9b7640 1 -- 172.21.15.43:0/2282586291 shutdown_connections 2024-04-25T08:58:12.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:12.824+0000 7fe21a9b7640 1 -- 172.21.15.43:0/2282586291 wait complete. 2024-04-25T08:58:13.227 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 36, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:57:54.590369+0000', 'modified': '2024-04-25T08:58:12.417098+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 62, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6870}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6870': {'gid': 6870, 'name': 'c', 'rank': 0, 'incarnation': 36, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.43:6833/1866869111', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6832', 'nonce': 1866869111}, {'type': 'v1', 'addr': '172.21.15.43:6833', 'nonce': 1866869111}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:58:13.227 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6870:up:replay 2024-04-25T08:58:14.227 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:58:14.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.353+0000 7f82eec83640 1 Processor -- start 2024-04-25T08:58:14.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.353+0000 7f82eec83640 1 -- start start 2024-04-25T08:58:14.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.353+0000 7f82eec83640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f82e80cbfc0 0x7f82e806dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:14.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.353+0000 7f82eec83640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82e80cd010 0x7f82e815fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:14.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.353+0000 7f82eec83640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82e80b2780 con 0x7f82e80ccbb0 2024-04-25T08:58:14.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.353+0000 7f82eec83640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82e80b2a60 con 0x7f82e80cbfc0 2024-04-25T08:58:14.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.353+0000 7f82eec83640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82e80b2d40 con 0x7f82e80cd010 2024-04-25T08:58:14.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.354+0000 7f82eca1f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f82e80cbfc0 0x7f82e806dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:14.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.354+0000 7f82eca1f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f82e80cbfc0 0x7f82e806dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45712/0 (socket says 172.21.15.43:45712) 2024-04-25T08:58:14.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.354+0000 7f82eca1f640 1 -- 172.21.15.43:0/3901318334 learned_addr learned my addr 172.21.15.43:0/3901318334 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:14.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.354+0000 7f82ed220640 1 --2- 172.21.15.43:0/3901318334 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82e80cd010 0x7f82e815fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:14.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.354+0000 7f82eca1f640 1 -- 172.21.15.43:0/3901318334 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82e80cd010 msgr2=0x7f82e815fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:14.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.354+0000 7f82eca1f640 1 --2- 172.21.15.43:0/3901318334 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82e80cd010 0x7f82e815fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:14.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.354+0000 7f82eca1f640 1 -- 172.21.15.43:0/3901318334 >> v1:172.21.15.193:6789/0 conn(0x7f82e80ccbb0 legacy=0x7f82e815f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:14.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.354+0000 7f82eca1f640 1 -- 172.21.15.43:0/3901318334 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e80b3020 con 0x7f82e80cbfc0 2024-04-25T08:58:14.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.355+0000 7f82eca1f640 1 --2- 172.21.15.43:0/3901318334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82e80cbfc0 0x7f82e806dc60 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f82e0009e50 tx=0x7f82e002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a47b9da4cd6974f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:14.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.355+0000 7f82e77fe640 1 -- 172.21.15.43:0/3901318334 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e0036070 con 0x7f82e80cbfc0 2024-04-25T08:58:14.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.355+0000 7f82e77fe640 1 -- 172.21.15.43:0/3901318334 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e0031600 con 0x7f82e80cbfc0 2024-04-25T08:58:14.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.355+0000 7f82e77fe640 1 -- 172.21.15.43:0/3901318334 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e00319d0 con 0x7f82e80cbfc0 2024-04-25T08:58:14.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.355+0000 7f82eec83640 1 -- 172.21.15.43:0/3901318334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82e80cbfc0 msgr2=0x7f82e806dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:14.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.355+0000 7f82eec83640 1 --2- 172.21.15.43:0/3901318334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82e80cbfc0 0x7f82e806dc60 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f82e0009e50 tx=0x7f82e002a710 comp rx=0 tx=0).stop 2024-04-25T08:58:14.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.355+0000 7f82eec83640 1 -- 172.21.15.43:0/3901318334 shutdown_connections 2024-04-25T08:58:14.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.355+0000 7f82eec83640 1 --2- 172.21.15.43:0/3901318334 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82e80cd010 0x7f82e815fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:14.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.355+0000 7f82eec83640 1 --2- 172.21.15.43:0/3901318334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82e80cbfc0 0x7f82e806dc60 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:14.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.355+0000 7f82eec83640 1 -- 172.21.15.43:0/3901318334 >> 172.21.15.43:0/3901318334 conn(0x7f82e80b1480 msgr2=0x7f82e80b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:14.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.355+0000 7f82eec83640 1 -- 172.21.15.43:0/3901318334 shutdown_connections 2024-04-25T08:58:14.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.355+0000 7f82eec83640 1 -- 172.21.15.43:0/3901318334 wait complete. 2024-04-25T08:58:14.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.357+0000 7f82eec83640 1 Processor -- start 2024-04-25T08:58:14.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.357+0000 7f82eec83640 1 -- start start 2024-04-25T08:58:14.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.358+0000 7f82eec83640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82e80cbfc0 0x7f82e815a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:14.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.358+0000 7f82eec83640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82e80ccbb0 0x7f82e81e60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:14.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.358+0000 7f82eec83640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82e80cd010 0x7f82e812eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:14.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.358+0000 7f82eec83640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e80cd4a0 con 0x7f82e80cd010 2024-04-25T08:58:14.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.358+0000 7f82eec83640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82e80b35e0 con 0x7f82e80cbfc0 2024-04-25T08:58:14.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.358+0000 7f82eec83640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82e806e1a0 con 0x7f82e80ccbb0 2024-04-25T08:58:14.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.358+0000 7f82ed220640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82e80cd010 0x7f82e812eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:14.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.358+0000 7f82ed220640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82e80cd010 0x7f82e812eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45720/0 (socket says 172.21.15.43:45720) 2024-04-25T08:58:14.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.358+0000 7f82ed220640 1 -- 172.21.15.43:0/3179659395 learned_addr learned my addr 172.21.15.43:0/3179659395 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:14.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.358+0000 7f82eca1f640 1 --2- 172.21.15.43:0/3179659395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82e80cbfc0 0x7f82e815a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:14.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.358+0000 7f82e7fff640 1 --2- 172.21.15.43:0/3179659395 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82e80ccbb0 0x7f82e81e60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:14.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.358+0000 7f82ed220640 1 -- 172.21.15.43:0/3179659395 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82e80ccbb0 msgr2=0x7f82e81e60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:14.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.358+0000 7f82ed220640 1 --2- 172.21.15.43:0/3179659395 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82e80ccbb0 0x7f82e81e60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:14.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.358+0000 7f82ed220640 1 -- 172.21.15.43:0/3179659395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82e80cbfc0 msgr2=0x7f82e815a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:14.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.358+0000 7f82ed220640 1 --2- 172.21.15.43:0/3179659395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82e80cbfc0 0x7f82e815a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:14.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.358+0000 7f82ed220640 1 -- 172.21.15.43:0/3179659395 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e81e6a60 con 0x7f82e80cd010 2024-04-25T08:58:14.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.358+0000 7f82ed220640 1 --2- 172.21.15.43:0/3179659395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82e80cd010 0x7f82e812eb70 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f82d800e860 tx=0x7f82d800ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:14.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.358+0000 7f82e57fa640 1 -- 172.21.15.43:0/3179659395 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d8017070 con 0x7f82e80cd010 2024-04-25T08:58:14.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.359+0000 7f82e57fa640 1 -- 172.21.15.43:0/3179659395 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82d800c6d0 con 0x7f82e80cd010 2024-04-25T08:58:14.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.359+0000 7f82eec83640 1 -- 172.21.15.43:0/3179659395 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82e81e6d40 con 0x7f82e80cd010 2024-04-25T08:58:14.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.359+0000 7f82e57fa640 1 -- 172.21.15.43:0/3179659395 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82d800cac0 con 0x7f82e80cd010 2024-04-25T08:58:14.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.359+0000 7f82eec83640 1 -- 172.21.15.43:0/3179659395 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82e81e7300 con 0x7f82e80cd010 2024-04-25T08:58:14.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.360+0000 7f82eec83640 1 -- 172.21.15.43:0/3179659395 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82e8074ef0 con 0x7f82e80cd010 2024-04-25T08:58:14.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.360+0000 7f82e57fa640 1 -- 172.21.15.43:0/3179659395 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f82d801a070 con 0x7f82e80cd010 2024-04-25T08:58:14.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.361+0000 7f82e57fa640 1 --2- 172.21.15.43:0/3179659395 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f82c806b4d0 0x7f82c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:14.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.361+0000 7f82e57fa640 1 -- 172.21.15.43:0/3179659395 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f82d808a970 con 0x7f82e80cd010 2024-04-25T08:58:14.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.361+0000 7f82eca1f640 1 --2- 172.21.15.43:0/3179659395 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f82c806b4d0 0x7f82c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:14.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.361+0000 7f82eca1f640 1 --2- 172.21.15.43:0/3179659395 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f82c806b4d0 0x7f82c806d990 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f82e0033000 tx=0x7f82e0009000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:14.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.365+0000 7f82e57fa640 1 -- 172.21.15.43:0/3179659395 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f82d80596a0 con 0x7f82e80cd010 2024-04-25T08:58:14.532 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:58:14.535 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:58:14.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.587+0000 7f82eec83640 1 -- 172.21.15.43:0/3179659395 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82e815dba0 con 0x7f82e80cd010 2024-04-25T08:58:14.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.587+0000 7f82e57fa640 1 -- 172.21.15.43:0/3179659395 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4936 (secure 0 0 0) 0x7f82d8058fa0 con 0x7f82e80cd010 2024-04-25T08:58:14.577 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:58:14.577 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":38,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5765,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/594587590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":594587590},{"type":"v1","addr":"172.21.15.193:6835","nonce":594587590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6187,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/681300720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":681300720},{"type":"v1","addr":"172.21.15.193:6837","nonce":681300720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6876,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/432022866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":432022866},{"type":"v1","addr":"172.21.15.43:6835","nonce":432022866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:57:54.590369+0000","modified":"2024-04-25T08:58:13.427804+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6870},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6870":{"gid":6870,"name":"c","rank":0,"incarnation":36,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.43:6833/1866869111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1866869111},{"type":"v1","addr":"172.21.15.43:6833","nonce":1866869111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:58:14.577 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 38 2024-04-25T08:58:14.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.589+0000 7f82eec83640 1 -- 172.21.15.43:0/3179659395 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f82c806b4d0 msgr2=0x7f82c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:14.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.589+0000 7f82eec83640 1 --2- 172.21.15.43:0/3179659395 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f82c806b4d0 0x7f82c806d990 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f82e0033000 tx=0x7f82e0009000 comp rx=0 tx=0).stop 2024-04-25T08:58:14.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.589+0000 7f82eec83640 1 -- 172.21.15.43:0/3179659395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82e80cd010 msgr2=0x7f82e812eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:14.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.589+0000 7f82eec83640 1 --2- 172.21.15.43:0/3179659395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82e80cd010 0x7f82e812eb70 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f82d800e860 tx=0x7f82d800ed20 comp rx=0 tx=0).stop 2024-04-25T08:58:14.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.590+0000 7f82eec83640 1 -- 172.21.15.43:0/3179659395 shutdown_connections 2024-04-25T08:58:14.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.590+0000 7f82eec83640 1 --2- 172.21.15.43:0/3179659395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82e80cd010 0x7f82e812eb70 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:14.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.590+0000 7f82eec83640 1 --2- 172.21.15.43:0/3179659395 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82e80ccbb0 0x7f82e81e60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:14.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.590+0000 7f82eec83640 1 --2- 172.21.15.43:0/3179659395 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f82c806b4d0 0x7f82c806d990 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:14.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.590+0000 7f82eec83640 1 --2- 172.21.15.43:0/3179659395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82e80cbfc0 0x7f82e815a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:14.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.590+0000 7f82eec83640 1 -- 172.21.15.43:0/3179659395 >> 172.21.15.43:0/3179659395 conn(0x7f82e80b1480 msgr2=0x7f82e815c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:14.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.590+0000 7f82eec83640 1 -- 172.21.15.43:0/3179659395 shutdown_connections 2024-04-25T08:58:14.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:14.590+0000 7f82eec83640 1 -- 172.21.15.43:0/3179659395 wait complete. 2024-04-25T08:58:14.992 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 38, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:57:54.590369+0000', 'modified': '2024-04-25T08:58:13.427804+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 62, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6870}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6870': {'gid': 6870, 'name': 'c', 'rank': 0, 'incarnation': 36, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.43:6833/1866869111', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6832', 'nonce': 1866869111}, {'type': 'v1', 'addr': '172.21.15.43:6833', 'nonce': 1866869111}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:58:14.992 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6870:up:rejoin 2024-04-25T08:58:15.994 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:58:16.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.117+0000 7f7177122640 1 Processor -- start 2024-04-25T08:58:16.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.118+0000 7f7177122640 1 -- start start 2024-04-25T08:58:16.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.118+0000 7f7177122640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f71700b0870 0x7f71700ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:16.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.118+0000 7f7177122640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71700cbee0 con 0x7f717015df70 2024-04-25T08:58:16.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.118+0000 7f7177122640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71700d0650 con 0x7f717006c510 2024-04-25T08:58:16.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.118+0000 7f7177122640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71700d0400 con 0x7f71700b0870 2024-04-25T08:58:16.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.118+0000 7f71756bf640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f71700b0870 0x7f71700ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:16.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.119+0000 7f716ffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f717015df70 0x7f71700cc670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47176/0 (socket says 172.21.15.43:47176) 2024-04-25T08:58:16.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.119+0000 7f716ffff640 1 -- 172.21.15.43:0/757785421 learned_addr learned my addr 172.21.15.43:0/757785421 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:16.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.119+0000 7f71756bf640 1 -- 172.21.15.43:0/757785421 >> v1:172.21.15.193:6789/0 conn(0x7f717006c510 legacy=0x7f717006c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:16.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.119+0000 7f716f7fe640 1 -- 172.21.15.43:0/757785421 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4231010000 0 0) 0x7f71700cbee0 con 0x7f717015df70 2024-04-25T08:58:16.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.119+0000 7f71756bf640 1 -- 172.21.15.43:0/757785421 >> v1:172.21.15.43:6789/0 conn(0x7f717015df70 legacy=0x7f71700cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:16.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.119+0000 7f71756bf640 1 -- 172.21.15.43:0/757785421 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71700d0930 con 0x7f71700b0870 2024-04-25T08:58:16.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.120+0000 7f71756bf640 1 --2- 172.21.15.43:0/757785421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71700b0870 0x7f71700ccd80 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f71600097d0 tx=0x7f716002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a28dfdcfa602ff6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:16.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.120+0000 7f716f7fe640 1 -- 172.21.15.43:0/757785421 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7160037070 con 0x7f71700b0870 2024-04-25T08:58:16.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.120+0000 7f716f7fe640 1 -- 172.21.15.43:0/757785421 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7160032440 con 0x7f71700b0870 2024-04-25T08:58:16.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.120+0000 7f716f7fe640 1 -- 172.21.15.43:0/757785421 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7160032810 con 0x7f71700b0870 2024-04-25T08:58:16.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.120+0000 7f7177122640 1 -- 172.21.15.43:0/757785421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71700b0870 msgr2=0x7f71700ccd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:16.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.120+0000 7f7177122640 1 --2- 172.21.15.43:0/757785421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71700b0870 0x7f71700ccd80 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f71600097d0 tx=0x7f716002a1c0 comp rx=0 tx=0).stop 2024-04-25T08:58:16.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.120+0000 7f7177122640 1 -- 172.21.15.43:0/757785421 shutdown_connections 2024-04-25T08:58:16.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.120+0000 7f7177122640 1 --2- 172.21.15.43:0/757785421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71700b0870 0x7f71700ccd80 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:16.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.120+0000 7f7177122640 1 -- 172.21.15.43:0/757785421 >> 172.21.15.43:0/757785421 conn(0x7f71700d17f0 msgr2=0x7f71700d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:16.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.120+0000 7f7177122640 1 -- 172.21.15.43:0/757785421 shutdown_connections 2024-04-25T08:58:16.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.120+0000 7f7177122640 1 -- 172.21.15.43:0/757785421 wait complete. 2024-04-25T08:58:16.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.122+0000 7f7177122640 1 Processor -- start 2024-04-25T08:58:16.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.122+0000 7f7177122640 1 -- start start 2024-04-25T08:58:16.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.123+0000 7f7177122640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f717006c510 0x7f71700b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:16.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.123+0000 7f7177122640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71700b0870 0x7f7170155b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:16.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.123+0000 7f7177122640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f717015df70 0x7f71701f3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:16.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.123+0000 7f7177122640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717006bd10 con 0x7f717006c510 2024-04-25T08:58:16.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.123+0000 7f7177122640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71700d0c10 con 0x7f71700b0870 2024-04-25T08:58:16.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.123+0000 7f7177122640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f717006be90 con 0x7f717015df70 2024-04-25T08:58:16.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.123+0000 7f7174ebe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f717006c510 0x7f71700b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:16.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.123+0000 7f7174ebe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f717006c510 0x7f71700b50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45740/0 (socket says 172.21.15.43:45740) 2024-04-25T08:58:16.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.123+0000 7f7174ebe640 1 -- 172.21.15.43:0/1884943808 learned_addr learned my addr 172.21.15.43:0/1884943808 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:16.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.123+0000 7f71756bf640 1 --2- 172.21.15.43:0/1884943808 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f717015df70 0x7f71701f3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:16.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.123+0000 7f716ffff640 1 --2- 172.21.15.43:0/1884943808 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71700b0870 0x7f7170155b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:16.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.123+0000 7f7174ebe640 1 -- 172.21.15.43:0/1884943808 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f717015df70 msgr2=0x7f71701f3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:16.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.123+0000 7f7174ebe640 1 --2- 172.21.15.43:0/1884943808 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f717015df70 0x7f71701f3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:16.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.123+0000 7f7174ebe640 1 -- 172.21.15.43:0/1884943808 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71700b0870 msgr2=0x7f7170155b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:16.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.123+0000 7f7174ebe640 1 --2- 172.21.15.43:0/1884943808 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71700b0870 0x7f7170155b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:16.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.123+0000 7f7174ebe640 1 -- 172.21.15.43:0/1884943808 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71700b55e0 con 0x7f717006c510 2024-04-25T08:58:16.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.123+0000 7f71756bf640 1 --2- 172.21.15.43:0/1884943808 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f717015df70 0x7f71701f3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:58:16.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.123+0000 7f7174ebe640 1 --2- 172.21.15.43:0/1884943808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f717006c510 0x7f71700b50a0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f716800c730 tx=0x7f716800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:16.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.123+0000 7f716d7fa640 1 -- 172.21.15.43:0/1884943808 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7168003b70 con 0x7f717006c510 2024-04-25T08:58:16.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.123+0000 7f716d7fa640 1 -- 172.21.15.43:0/1884943808 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7168003d30 con 0x7f717006c510 2024-04-25T08:58:16.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.123+0000 7f7177122640 1 -- 172.21.15.43:0/1884943808 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f717015faa0 con 0x7f717006c510 2024-04-25T08:58:16.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.123+0000 7f716d7fa640 1 -- 172.21.15.43:0/1884943808 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716800aa90 con 0x7f717006c510 2024-04-25T08:58:16.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.123+0000 7f7177122640 1 -- 172.21.15.43:0/1884943808 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7170160060 con 0x7f717006c510 2024-04-25T08:58:16.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.124+0000 7f7177122640 1 -- 172.21.15.43:0/1884943808 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f717006dc70 con 0x7f717006c510 2024-04-25T08:58:16.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.125+0000 7f716d7fa640 1 -- 172.21.15.43:0/1884943808 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f716800ac30 con 0x7f717006c510 2024-04-25T08:58:16.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.125+0000 7f716d7fa640 1 --2- 172.21.15.43:0/1884943808 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f715c06b4d0 0x7f715c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:16.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.126+0000 7f716d7fa640 1 -- 172.21.15.43:0/1884943808 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f716808a100 con 0x7f717006c510 2024-04-25T08:58:16.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.126+0000 7f716ffff640 1 --2- 172.21.15.43:0/1884943808 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f715c06b4d0 0x7f715c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:16.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.126+0000 7f716ffff640 1 --2- 172.21.15.43:0/1884943808 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f715c06b4d0 0x7f715c06d990 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f71700b4630 tx=0x7f7164007b90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:16.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.130+0000 7f716d7fa640 1 -- 172.21.15.43:0/1884943808 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f717006dc70 con 0x7f717006c510 2024-04-25T08:58:16.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.350+0000 7f7177122640 1 -- 172.21.15.43:0/1884943808 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7170074ef0 con 0x7f717006c510 2024-04-25T08:58:16.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.351+0000 7f716d7fa640 1 -- 172.21.15.43:0/1884943808 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4936 (secure 0 0 0) 0x7f7168058f60 con 0x7f717006c510 2024-04-25T08:58:16.341 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:58:16.341 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":39,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5765,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/594587590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":594587590},{"type":"v1","addr":"172.21.15.193:6835","nonce":594587590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6187,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/681300720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":681300720},{"type":"v1","addr":"172.21.15.193:6837","nonce":681300720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6876,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/432022866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":432022866},{"type":"v1","addr":"172.21.15.43:6835","nonce":432022866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:57:54.590369+0000","modified":"2024-04-25T08:58:14.432383+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6870},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6870":{"gid":6870,"name":"c","rank":0,"incarnation":36,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6833/1866869111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1866869111},{"type":"v1","addr":"172.21.15.43:6833","nonce":1866869111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:58:16.341 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 39 2024-04-25T08:58:16.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.353+0000 7f7177122640 1 -- 172.21.15.43:0/1884943808 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f715c06b4d0 msgr2=0x7f715c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:16.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.353+0000 7f7177122640 1 --2- 172.21.15.43:0/1884943808 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f715c06b4d0 0x7f715c06d990 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f71700b4630 tx=0x7f7164007b90 comp rx=0 tx=0).stop 2024-04-25T08:58:16.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.353+0000 7f7177122640 1 -- 172.21.15.43:0/1884943808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f717006c510 msgr2=0x7f71700b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:16.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.353+0000 7f7177122640 1 --2- 172.21.15.43:0/1884943808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f717006c510 0x7f71700b50a0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f716800c730 tx=0x7f716800cbf0 comp rx=0 tx=0).stop 2024-04-25T08:58:16.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.353+0000 7f7177122640 1 -- 172.21.15.43:0/1884943808 shutdown_connections 2024-04-25T08:58:16.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.353+0000 7f7177122640 1 --2- 172.21.15.43:0/1884943808 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f717015df70 0x7f71701f3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:16.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.353+0000 7f7177122640 1 --2- 172.21.15.43:0/1884943808 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f715c06b4d0 0x7f715c06d990 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:16.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.353+0000 7f7177122640 1 --2- 172.21.15.43:0/1884943808 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71700b0870 0x7f7170155b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:16.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.353+0000 7f7177122640 1 --2- 172.21.15.43:0/1884943808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f717006c510 0x7f71700b50a0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:16.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.353+0000 7f7177122640 1 -- 172.21.15.43:0/1884943808 >> 172.21.15.43:0/1884943808 conn(0x7f71700d17f0 msgr2=0x7f717015bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:16.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.353+0000 7f7177122640 1 -- 172.21.15.43:0/1884943808 shutdown_connections 2024-04-25T08:58:16.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.353+0000 7f7177122640 1 -- 172.21.15.43:0/1884943808 wait complete. 2024-04-25T08:58:16.756 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 39, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:57:54.590369+0000', 'modified': '2024-04-25T08:58:14.432383+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 62, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6870}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6870': {'gid': 6870, 'name': 'c', 'rank': 0, 'incarnation': 36, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.43:6833/1866869111', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6832', 'nonce': 1866869111}, {'type': 'v1', 'addr': '172.21.15.43:6833', 'nonce': 1866869111}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:58:16.756 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:58:16.756 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T08:58:16.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.872+0000 7f953529e640 1 Processor -- start 2024-04-25T08:58:16.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.872+0000 7f953529e640 1 -- start start 2024-04-25T08:58:16.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.873+0000 7f953529e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f95300cff70 0x7f95300cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:16.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.873+0000 7f953529e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95300ccf10 con 0x7f953015f180 2024-04-25T08:58:16.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.873+0000 7f953529e640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95300cd1f0 con 0x7f95300b23f0 2024-04-25T08:58:16.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.873+0000 7f953529e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95300cccc0 con 0x7f95300cff70 2024-04-25T08:58:16.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.873+0000 7f952effd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f953015f180 0x7f95300cf860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47178/0 (socket says 172.21.15.43:47178) 2024-04-25T08:58:16.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.873+0000 7f952effd640 1 -- 172.21.15.43:0/2644851203 learned_addr learned my addr 172.21.15.43:0/2644851203 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:16.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.873+0000 7f952dffb640 1 -- 172.21.15.43:0/2644851203 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2578205270 0 0) 0x7f95300ccf10 con 0x7f953015f180 2024-04-25T08:58:16.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.873+0000 7f952dffb640 1 -- 172.21.15.43:0/2644851203 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9518003520 con 0x7f953015f180 2024-04-25T08:58:16.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.874+0000 7f952e7fc640 1 --2- 172.21.15.43:0/2644851203 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f95300cff70 0x7f95300cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:16.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.874+0000 7f952dffb640 1 -- 172.21.15.43:0/2644851203 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1518645928 0 0) 0x7f9518003520 con 0x7f953015f180 2024-04-25T08:58:16.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.874+0000 7f952dffb640 1 -- 172.21.15.43:0/2644851203 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f95300cff70 msgr2=0x7f95300cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:16.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.874+0000 7f952dffb640 1 --2- 172.21.15.43:0/2644851203 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f95300cff70 0x7f95300cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:16.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.874+0000 7f952dffb640 1 -- 172.21.15.43:0/2644851203 >> v1:172.21.15.193:6789/0 conn(0x7f95300b23f0 legacy=0x7f95300ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:16.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.874+0000 7f952dffb640 1 -- 172.21.15.43:0/2644851203 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95300cd4d0 con 0x7f953015f180 2024-04-25T08:58:16.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.874+0000 7f952dffb640 1 -- 172.21.15.43:0/2644851203 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f9528001670 con 0x7f953015f180 2024-04-25T08:58:16.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.874+0000 7f952e7fc640 1 --2- 172.21.15.43:0/2644851203 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f95300cff70 0x7f95300cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:58:16.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.874+0000 7f952dffb640 1 -- 172.21.15.43:0/2644851203 >> v1:172.21.15.43:6789/0 conn(0x7f953015f180 legacy=0x7f95300cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:16.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.874+0000 7f952dffb640 1 --2- 172.21.15.43:0/2644851203 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95180039e0 0x7f9518003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:16.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.874+0000 7f952dffb640 1 --2- 172.21.15.43:0/2644851203 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f95300cff70 0x7f9518004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:16.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.874+0000 7f952dffb640 1 --2- 172.21.15.43:0/2644851203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9518004d40 0x7f95180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:16.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.874+0000 7f952dffb640 1 -- 172.21.15.43:0/2644851203 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95300cccc0 con 0x7f9518004d40 2024-04-25T08:58:16.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.874+0000 7f952dffb640 1 -- 172.21.15.43:0/2644851203 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951800bb50 con 0x7f95180039e0 2024-04-25T08:58:16.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.874+0000 7f952dffb640 1 -- 172.21.15.43:0/2644851203 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f951800bd50 con 0x7f95300cff70 2024-04-25T08:58:16.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.874+0000 7f952f7fe640 1 --2- 172.21.15.43:0/2644851203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9518004d40 0x7f95180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:16.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.874+0000 7f952effd640 1 --2- 172.21.15.43:0/2644851203 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95180039e0 0x7f9518003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:16.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.874+0000 7f952e7fc640 1 --2- 172.21.15.43:0/2644851203 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f95300cff70 0x7f9518004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:16.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.874+0000 7f952f7fe640 1 -- 172.21.15.43:0/2644851203 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f95300cff70 msgr2=0x7f9518004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:16.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.874+0000 7f952f7fe640 1 --2- 172.21.15.43:0/2644851203 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f95300cff70 0x7f9518004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:16.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.874+0000 7f952f7fe640 1 -- 172.21.15.43:0/2644851203 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95180039e0 msgr2=0x7f9518003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:16.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.874+0000 7f952f7fe640 1 --2- 172.21.15.43:0/2644851203 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95180039e0 0x7f9518003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:16.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.874+0000 7f952f7fe640 1 -- 172.21.15.43:0/2644851203 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9518003520 con 0x7f9518004d40 2024-04-25T08:58:16.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.875+0000 7f952f7fe640 1 --2- 172.21.15.43:0/2644851203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9518004d40 0x7f95180051c0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f9520001e50 tx=0x7f9518005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d18e3b1e5b7b004 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:16.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.875+0000 7f952dffb640 1 -- 172.21.15.43:0/2644851203 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9520001a70 con 0x7f9518004d40 2024-04-25T08:58:16.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.875+0000 7f952dffb640 1 -- 172.21.15.43:0/2644851203 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9520003970 con 0x7f9518004d40 2024-04-25T08:58:16.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.875+0000 7f952dffb640 1 -- 172.21.15.43:0/2644851203 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9520003d60 con 0x7f9518004d40 2024-04-25T08:58:16.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.876+0000 7f953529e640 1 -- 172.21.15.43:0/2644851203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9518004d40 msgr2=0x7f95180051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:16.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.876+0000 7f953529e640 1 --2- 172.21.15.43:0/2644851203 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9518004d40 0x7f95180051c0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f9520001e50 tx=0x7f9518005830 comp rx=0 tx=0).stop 2024-04-25T08:58:16.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.876+0000 7f952f7fe640 1 -- 172.21.15.43:0/2644851203 reap_dead start 2024-04-25T08:58:16.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.876+0000 7f953529e640 1 -- 172.21.15.43:0/2644851203 shutdown_connections 2024-04-25T08:58:16.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.876+0000 7f953529e640 1 -- 172.21.15.43:0/2644851203 >> 172.21.15.43:0/2644851203 conn(0x7f95300d17f0 msgr2=0x7f95300d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:16.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.876+0000 7f953529e640 1 -- 172.21.15.43:0/2644851203 shutdown_connections 2024-04-25T08:58:16.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.876+0000 7f953529e640 1 -- 172.21.15.43:0/2644851203 wait complete. 2024-04-25T08:58:16.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.877+0000 7f953529e640 1 Processor -- start 2024-04-25T08:58:16.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.877+0000 7f953529e640 1 -- start start 2024-04-25T08:58:16.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.878+0000 7f953529e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9518004d40 0x7f95301567c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:16.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.878+0000 7f953529e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95300c4c70 0x7f95300c6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:16.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.878+0000 7f953529e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f95300b66a0 0x7f95300b6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:16.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.878+0000 7f953529e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95300d23f0 con 0x7f9518004d40 2024-04-25T08:58:16.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.878+0000 7f953529e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95300cd840 con 0x7f95300c4c70 2024-04-25T08:58:16.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.878+0000 7f953529e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95300d2570 con 0x7f95300b66a0 2024-04-25T08:58:16.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.878+0000 7f952effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9518004d40 0x7f95301567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:16.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.878+0000 7f952effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9518004d40 0x7f95301567c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45758/0 (socket says 172.21.15.43:45758) 2024-04-25T08:58:16.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.878+0000 7f952effd640 1 -- 172.21.15.43:0/2435075240 learned_addr learned my addr 172.21.15.43:0/2435075240 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:16.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.878+0000 7f952effd640 1 -- 172.21.15.43:0/2435075240 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f95300b66a0 msgr2=0x7f95300b6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:16.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.878+0000 7f952effd640 1 --2- 172.21.15.43:0/2435075240 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f95300b66a0 0x7f95300b6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:16.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.878+0000 7f952effd640 1 -- 172.21.15.43:0/2435075240 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95300c4c70 msgr2=0x7f95300c6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:16.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.878+0000 7f952e7fc640 1 --2- 172.21.15.43:0/2435075240 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95300c4c70 0x7f95300c6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:16.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.878+0000 7f952effd640 1 --2- 172.21.15.43:0/2435075240 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95300c4c70 0x7f95300c6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:16.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.878+0000 7f952effd640 1 -- 172.21.15.43:0/2435075240 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95300c51e0 con 0x7f9518004d40 2024-04-25T08:58:16.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.878+0000 7f952e7fc640 1 --2- 172.21.15.43:0/2435075240 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95300c4c70 0x7f95300c6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:16.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.878+0000 7f952effd640 1 --2- 172.21.15.43:0/2435075240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9518004d40 0x7f95301567c0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f9528007320 tx=0x7f95280077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:16.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.878+0000 7f95177fe640 1 -- 172.21.15.43:0/2435075240 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952801a070 con 0x7f9518004d40 2024-04-25T08:58:16.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.878+0000 7f95177fe640 1 -- 172.21.15.43:0/2435075240 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f952800f9c0 con 0x7f9518004d40 2024-04-25T08:58:16.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.879+0000 7f953529e640 1 -- 172.21.15.43:0/2435075240 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95300c54c0 con 0x7f9518004d40 2024-04-25T08:58:16.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.879+0000 7f95177fe640 1 -- 172.21.15.43:0/2435075240 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952800fdb0 con 0x7f9518004d40 2024-04-25T08:58:16.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.879+0000 7f953529e640 1 -- 172.21.15.43:0/2435075240 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95300c5a80 con 0x7f9518004d40 2024-04-25T08:58:16.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.880+0000 7f95177fe640 1 -- 172.21.15.43:0/2435075240 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f952801d070 con 0x7f9518004d40 2024-04-25T08:58:16.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.880+0000 7f95177fe640 1 --2- 172.21.15.43:0/2435075240 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f951c06b4d0 0x7f951c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:16.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.880+0000 7f95177fe640 1 -- 172.21.15.43:0/2435075240 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f9528089c00 con 0x7f9518004d40 2024-04-25T08:58:16.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.881+0000 7f952e7fc640 1 --2- 172.21.15.43:0/2435075240 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f951c06b4d0 0x7f951c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:16.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.881+0000 7f952e7fc640 1 --2- 172.21.15.43:0/2435075240 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f951c06b4d0 0x7f951c06d990 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f95301f6200 tx=0x7f9524008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:16.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.881+0000 7f9537fa5740 1 Processor -- start 2024-04-25T08:58:16.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.882+0000 7f9537fa5740 1 -- start start 2024-04-25T08:58:16.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.882+0000 7f9537fa5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aedebad230 0x55aedebad610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:16.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.882+0000 7f9537fa5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aedebcaad0 0x55aedebcb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:16.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.882+0000 7f9537fa5740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55aedeb82f10 0x55aedebc9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:16.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.882+0000 7f9537fa5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aedeae5a10 con 0x55aedebcaad0 2024-04-25T08:58:16.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.882+0000 7f9537fa5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aedea8eb30 con 0x55aedebad230 2024-04-25T08:58:16.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.882+0000 7f9537fa5740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aedeae1c50 con 0x55aedeb82f10 2024-04-25T08:58:16.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.883+0000 7f952e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aedebcaad0 0x55aedebcb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:16.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.883+0000 7f952e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aedebcaad0 0x55aedebcb020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45772/0 (socket says 172.21.15.43:45772) 2024-04-25T08:58:16.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.883+0000 7f952e7fc640 1 -- 172.21.15.43:0/3180708999 learned_addr learned my addr 172.21.15.43:0/3180708999 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:16.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.883+0000 7f952f7fe640 1 --2- 172.21.15.43:0/3180708999 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55aedeb82f10 0x55aedebc9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:16.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.883+0000 7f952effd640 1 --2- 172.21.15.43:0/3180708999 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aedebad230 0x55aedebad610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:16.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.883+0000 7f952e7fc640 1 -- 172.21.15.43:0/3180708999 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55aedeb82f10 msgr2=0x55aedebc9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:16.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.883+0000 7f952e7fc640 1 --2- 172.21.15.43:0/3180708999 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55aedeb82f10 0x55aedebc9190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:16.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.883+0000 7f952e7fc640 1 -- 172.21.15.43:0/3180708999 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aedebad230 msgr2=0x55aedebad610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:16.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.883+0000 7f952e7fc640 1 --2- 172.21.15.43:0/3180708999 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aedebad230 0x55aedebad610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:16.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.883+0000 7f952e7fc640 1 -- 172.21.15.43:0/3180708999 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aedeaa3e10 con 0x55aedebcaad0 2024-04-25T08:58:16.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.883+0000 7f952e7fc640 1 --2- 172.21.15.43:0/3180708999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aedebcaad0 0x55aedebcb020 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f9524012a50 tx=0x7f9524015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a75daadd504dbd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:16.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.883+0000 7f9514ff9640 1 -- 172.21.15.43:0/3180708999 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f952400f040 con 0x55aedebcaad0 2024-04-25T08:58:16.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.883+0000 7f9514ff9640 1 -- 172.21.15.43:0/3180708999 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9524027070 con 0x55aedebcaad0 2024-04-25T08:58:16.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.883+0000 7f9514ff9640 1 -- 172.21.15.43:0/3180708999 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9524015cf0 con 0x55aedebcaad0 2024-04-25T08:58:16.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.884+0000 7f9537fa5740 1 -- 172.21.15.43:0/3180708999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aedebcaad0 msgr2=0x55aedebcb020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:16.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.884+0000 7f9537fa5740 1 --2- 172.21.15.43:0/3180708999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aedebcaad0 0x55aedebcb020 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f9524012a50 tx=0x7f9524015000 comp rx=0 tx=0).stop 2024-04-25T08:58:16.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.884+0000 7f9537fa5740 1 -- 172.21.15.43:0/3180708999 shutdown_connections 2024-04-25T08:58:16.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.884+0000 7f9537fa5740 1 --2- 172.21.15.43:0/3180708999 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55aedeb82f10 0x55aedebc9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:16.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.884+0000 7f9537fa5740 1 --2- 172.21.15.43:0/3180708999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aedebcaad0 0x55aedebcb020 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:16.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.884+0000 7f9537fa5740 1 --2- 172.21.15.43:0/3180708999 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aedebad230 0x55aedebad610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:16.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.884+0000 7f9537fa5740 1 -- 172.21.15.43:0/3180708999 >> 172.21.15.43:0/3180708999 conn(0x55aedeb7f840 msgr2=0x55aedebc8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:16.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.884+0000 7f9537fa5740 1 -- 172.21.15.43:0/3180708999 shutdown_connections 2024-04-25T08:58:16.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.884+0000 7f9537fa5740 1 -- 172.21.15.43:0/3180708999 wait complete. 2024-04-25T08:58:16.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.885+0000 7f9537fa5740 1 -- start start 2024-04-25T08:58:16.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.885+0000 7f9537fa5740 1 Processor -- start 2024-04-25T08:58:16.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.885+0000 7f9537fa5740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55aedebad230 0x55aedeb7c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:16.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.885+0000 7f9537fa5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aedebcaad0 0x55aedebafe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:16.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.885+0000 7f9537fa5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aedebae1e0 0x55aedebb0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:16.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.885+0000 7f9537fa5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aedeb83cb0 con 0x55aedebcaad0 2024-04-25T08:58:16.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.885+0000 7f9537fa5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aedeaa20c0 con 0x55aedebae1e0 2024-04-25T08:58:16.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.885+0000 7f9537fa5740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aedeb7d770 con 0x55aedebad230 2024-04-25T08:58:16.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.885+0000 7f9537fa5740 10 client.0 _ll_register_callbacks cb 0x55aedea95ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:58:16.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.886+0000 7f952e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aedebcaad0 0x55aedebafe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:16.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.886+0000 7f952e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aedebcaad0 0x55aedebafe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45774/0 (socket says 172.21.15.43:45774) 2024-04-25T08:58:16.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.886+0000 7f952e7fc640 1 -- 172.21.15.43:0/1570284395 learned_addr learned my addr 172.21.15.43:0/1570284395 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:16.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.886+0000 7f952effd640 1 --2- 172.21.15.43:0/1570284395 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55aedebad230 0x55aedeb7c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:16.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.886+0000 7f952f7fe640 1 --2- 172.21.15.43:0/1570284395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aedebae1e0 0x55aedebb0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:16.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.886+0000 7f952e7fc640 1 -- 172.21.15.43:0/1570284395 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55aedebad230 msgr2=0x55aedeb7c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:16.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.886+0000 7f952e7fc640 1 --2- 172.21.15.43:0/1570284395 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55aedebad230 0x55aedeb7c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:16.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.886+0000 7f952e7fc640 1 -- 172.21.15.43:0/1570284395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aedebae1e0 msgr2=0x55aedebb0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:16.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.886+0000 7f952e7fc640 1 --2- 172.21.15.43:0/1570284395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aedebae1e0 0x55aedebb0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:16.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.886+0000 7f952e7fc640 1 -- 172.21.15.43:0/1570284395 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aede7fa2d0 con 0x55aedebcaad0 2024-04-25T08:58:16.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.886+0000 7f952e7fc640 1 --2- 172.21.15.43:0/1570284395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aedebcaad0 0x55aedebafe50 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f9524023000 tx=0x7f95240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b61514afebb0d2d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:16.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.886+0000 7f950dffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:58:16.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.886+0000 7f950dffb640 1 -- 172.21.15.43:0/1570284395 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9524030030 con 0x55aedebcaad0 2024-04-25T08:58:16.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.886+0000 7f950dffb640 1 -- 172.21.15.43:0/1570284395 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(62..62 src has 1..62) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f9524005c60 con 0x55aedebcaad0 2024-04-25T08:58:16.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.886+0000 7f9537fa5740 1 -- 172.21.15.43:0/1570284395 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9524027070 con 0x55aedebcaad0 2024-04-25T08:58:16.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.886+0000 7f950dffb640 1 -- 172.21.15.43:0/1570284395 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9524039050 con 0x55aedebcaad0 2024-04-25T08:58:16.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.886+0000 7f9537fa5740 10 client.6912 fetch_fsmap learned FSMap version 39 2024-04-25T08:58:16.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.886+0000 7f9537fa5740 1 -- 172.21.15.43:0/1570284395 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x55aede8c46b0 con 0x55aedebcaad0 2024-04-25T08:58:16.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.887+0000 7f950dffb640 1 -- 172.21.15.43:0/1570284395 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9524027070 con 0x55aedebcaad0 2024-04-25T08:58:16.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.887+0000 7f950dffb640 1 -- 172.21.15.43:0/1570284395 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 39) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f952403c020 con 0x55aedebcaad0 2024-04-25T08:58:16.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.887+0000 7f9537fa5740 10 client.6912 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T08:58:16.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.887+0000 7f9537fa5740 10 client.6912 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:58:16.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.887+0000 7f9537fa5740 20 client.6912 populate_metadata read hostname 'smithi043' 2024-04-25T08:58:16.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.887+0000 7f9537fa5740 1 --2- 172.21.15.43:0/1570284395 >> [v2:172.21.15.43:6832/1866869111,v1:172.21.15.43:6833/1866869111] conn(0x55aedebd8b70 0x55aedebe5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:16.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.887+0000 7f9537fa5740 4 client.6912 mds_command: new command op to 6870 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:58:16.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.887+0000 7f9537fa5740 1 -- 172.21.15.43:0/1570284395 --> [v2:172.21.15.43:6832/1866869111,v1:172.21.15.43:6833/1866869111] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aedeb44920 con 0x55aedebd8b70 2024-04-25T08:58:16.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.887+0000 7f952effd640 1 --2- 172.21.15.43:0/1570284395 >> [v2:172.21.15.43:6832/1866869111,v1:172.21.15.43:6833/1866869111] conn(0x55aedebd8b70 0x55aedebe5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:16.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.887+0000 7f950dffb640 10 client.6912 ms_handle_connect on v2:172.21.15.43:6832/1866869111 2024-04-25T08:58:16.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.887+0000 7f952effd640 1 --2- 172.21.15.43:0/1570284395 >> [v2:172.21.15.43:6832/1866869111,v1:172.21.15.43:6833/1866869111] conn(0x55aedebd8b70 0x55aedebe5ef0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=186da0eda90aa982 server_cookie=9d683ea9d836d07 in_seq=0 out_seq=0 2024-04-25T08:58:16.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.888+0000 7f950dffb640 1 -- 172.21.15.43:0/1570284395 <== mds.0 v2:172.21.15.43:6832/1866869111 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55aedeb44920 con 0x55aedebd8b70 2024-04-25T08:58:16.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.888+0000 7f950dffb640 10 client.6912 handle_command_reply: tid=0 2024-04-25T08:58:16.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.888+0000 7f9537fa5740 1 client.6912 shutdown 2024-04-25T08:58:16.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.888+0000 7f9537fa5740 1 -- 172.21.15.43:0/1570284395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aedebcaad0 msgr2=0x55aedebafe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:16.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.888+0000 7f9537fa5740 1 --2- 172.21.15.43:0/1570284395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aedebcaad0 0x55aedebafe50 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f9524023000 tx=0x7f95240046c0 comp rx=0 tx=0).stop 2024-04-25T08:58:16.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.888+0000 7f9537fa5740 1 -- 172.21.15.43:0/1570284395 shutdown_connections 2024-04-25T08:58:16.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.888+0000 7f9537fa5740 1 --2- 172.21.15.43:0/1570284395 >> [v2:172.21.15.43:6832/1866869111,v1:172.21.15.43:6833/1866869111] conn(0x55aedebd8b70 0x55aedebe5ef0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:16.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.888+0000 7f9537fa5740 1 --2- 172.21.15.43:0/1570284395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aedebae1e0 0x55aedebb0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:16.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.888+0000 7f9537fa5740 1 --2- 172.21.15.43:0/1570284395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aedebcaad0 0x55aedebafe50 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:16.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.888+0000 7f950dffb640 0 client.6912 ms_handle_reset on v2:172.21.15.43:6832/1866869111 2024-04-25T08:58:16.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.888+0000 7f9537fa5740 1 --2- 172.21.15.43:0/1570284395 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55aedebad230 0x55aedeb7c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:16.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.888+0000 7f9537fa5740 1 -- 172.21.15.43:0/1570284395 >> 172.21.15.43:0/1570284395 conn(0x55aedeb82f10 msgr2=0x55aedebc7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:16.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.888+0000 7f9537fa5740 1 -- 172.21.15.43:0/1570284395 shutdown_connections 2024-04-25T08:58:16.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.888+0000 7f9537fa5740 1 -- 172.21.15.43:0/1570284395 wait complete. 2024-04-25T08:58:16.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.888+0000 7f9537fa5740 20 client.6912 trim_cache size 0 max 16384 2024-04-25T08:58:16.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.904+0000 7f9537fa5740 1 Processor -- start 2024-04-25T08:58:16.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.904+0000 7f9537fa5740 1 -- start start 2024-04-25T08:58:16.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.904+0000 7f9537fa5740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55aedebad230 0x55aedebc6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:16.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.904+0000 7f9537fa5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aedebae1e0 0x55aedeb832f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:16.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.904+0000 7f9537fa5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aedebcaad0 0x55aedeb83830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:16.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.905+0000 7f9537fa5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95301fd930 con 0x55aedebae1e0 2024-04-25T08:58:16.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.905+0000 7f9537fa5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95300b54b0 con 0x55aedebcaad0 2024-04-25T08:58:16.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.905+0000 7f9537fa5740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95301fd4b0 con 0x55aedebad230 2024-04-25T08:58:16.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.905+0000 7f952e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aedebae1e0 0x55aedeb832f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:16.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.905+0000 7f952e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aedebae1e0 0x55aedeb832f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45782/0 (socket says 172.21.15.43:45782) 2024-04-25T08:58:16.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.905+0000 7f952e7fc640 1 -- 172.21.15.43:0/1525995832 learned_addr learned my addr 172.21.15.43:0/1525995832 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:16.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.905+0000 7f952f7fe640 1 --2- 172.21.15.43:0/1525995832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aedebcaad0 0x55aedeb83830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:16.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.905+0000 7f952effd640 1 --2- 172.21.15.43:0/1525995832 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55aedebad230 0x55aedebc6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:16.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.905+0000 7f952e7fc640 1 -- 172.21.15.43:0/1525995832 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55aedebad230 msgr2=0x55aedebc6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:16.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.905+0000 7f952e7fc640 1 --2- 172.21.15.43:0/1525995832 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55aedebad230 0x55aedebc6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:16.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.905+0000 7f952e7fc640 1 -- 172.21.15.43:0/1525995832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aedebcaad0 msgr2=0x55aedeb83830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:16.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.905+0000 7f952e7fc640 1 --2- 172.21.15.43:0/1525995832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aedebcaad0 0x55aedeb83830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:16.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.905+0000 7f952e7fc640 1 -- 172.21.15.43:0/1525995832 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aedebbee60 con 0x55aedebae1e0 2024-04-25T08:58:16.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.905+0000 7f952f7fe640 1 --2- 172.21.15.43:0/1525995832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aedebcaad0 0x55aedeb83830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:16.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.905+0000 7f952effd640 1 --2- 172.21.15.43:0/1525995832 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55aedebad230 0x55aedebc6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:16.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.905+0000 7f952e7fc640 1 --2- 172.21.15.43:0/1525995832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aedebae1e0 0x55aedeb832f0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f9524005fb0 tx=0x7f952402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c82fd156d5f52455 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:16.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.905+0000 7f950ffff640 1 -- 172.21.15.43:0/1525995832 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9524041070 con 0x55aedebae1e0 2024-04-25T08:58:16.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.905+0000 7f950ffff640 1 -- 172.21.15.43:0/1525995832 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9524027050 con 0x55aedebae1e0 2024-04-25T08:58:16.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.905+0000 7f950ffff640 1 -- 172.21.15.43:0/1525995832 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9524030040 con 0x55aedebae1e0 2024-04-25T08:58:16.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.906+0000 7f9537fa5740 1 -- 172.21.15.43:0/1525995832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aedebae1e0 msgr2=0x55aedeb832f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:16.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.906+0000 7f9537fa5740 1 --2- 172.21.15.43:0/1525995832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aedebae1e0 0x55aedeb832f0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f9524005fb0 tx=0x7f952402c000 comp rx=0 tx=0).stop 2024-04-25T08:58:16.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.906+0000 7f9537fa5740 1 -- 172.21.15.43:0/1525995832 shutdown_connections 2024-04-25T08:58:16.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.906+0000 7f9537fa5740 1 --2- 172.21.15.43:0/1525995832 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aedebcaad0 0x55aedeb83830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:16.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.906+0000 7f9537fa5740 1 --2- 172.21.15.43:0/1525995832 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aedebae1e0 0x55aedeb832f0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:16.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.906+0000 7f9537fa5740 1 --2- 172.21.15.43:0/1525995832 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55aedebad230 0x55aedebc6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:16.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.906+0000 7f9537fa5740 1 -- 172.21.15.43:0/1525995832 >> 172.21.15.43:0/1525995832 conn(0x55aedeb82f10 msgr2=0x55aedebca080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:16.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.906+0000 7f9537fa5740 1 -- 172.21.15.43:0/1525995832 shutdown_connections 2024-04-25T08:58:16.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.906+0000 7f9537fa5740 1 -- 172.21.15.43:0/1525995832 wait complete. 2024-04-25T08:58:16.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.906+0000 7f9537fa5740 1 -- start start 2024-04-25T08:58:16.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.906+0000 7f9537fa5740 1 Processor -- start 2024-04-25T08:58:16.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.907+0000 7f9537fa5740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55aedebad230 0x55aedebbe610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:16.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.907+0000 7f9537fa5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aedebae1e0 0x55aedebafd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:16.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.907+0000 7f9537fa5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aedebcaad0 0x55aedeb832f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:16.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.907+0000 7f9537fa5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aedebd89f0 con 0x55aedebcaad0 2024-04-25T08:58:16.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.907+0000 7f9537fa5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aedebaf260 con 0x55aedebae1e0 2024-04-25T08:58:16.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.907+0000 7f9537fa5740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aedebe28b0 con 0x55aedebad230 2024-04-25T08:58:16.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.907+0000 7f9537fa5740 10 client.0 _ll_register_callbacks cb 0x55aedeb7dc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:58:16.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.907+0000 7f952f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aedebcaad0 0x55aedeb832f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:16.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.907+0000 7f952f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aedebcaad0 0x55aedeb832f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45792/0 (socket says 172.21.15.43:45792) 2024-04-25T08:58:16.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.907+0000 7f952f7fe640 1 -- 172.21.15.43:0/2136701674 learned_addr learned my addr 172.21.15.43:0/2136701674 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:16.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.907+0000 7f952effd640 1 --2- 172.21.15.43:0/2136701674 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55aedebad230 0x55aedebbe610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:16.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.907+0000 7f952e7fc640 1 --2- 172.21.15.43:0/2136701674 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aedebae1e0 0x55aedebafd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:16.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.908+0000 7f952f7fe640 1 -- 172.21.15.43:0/2136701674 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55aedebad230 msgr2=0x55aedebbe610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:16.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.908+0000 7f952f7fe640 1 --2- 172.21.15.43:0/2136701674 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55aedebad230 0x55aedebbe610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:16.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.908+0000 7f952f7fe640 1 -- 172.21.15.43:0/2136701674 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aedebae1e0 msgr2=0x55aedebafd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:16.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.908+0000 7f952f7fe640 1 --2- 172.21.15.43:0/2136701674 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aedebae1e0 0x55aedebafd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:16.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.908+0000 7f952f7fe640 1 -- 172.21.15.43:0/2136701674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aedec3a330 con 0x55aedebcaad0 2024-04-25T08:58:16.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.908+0000 7f952f7fe640 1 --2- 172.21.15.43:0/2136701674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aedebcaad0 0x55aedeb832f0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f9520005040 tx=0x7f95200014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b40a2965be21ef5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:16.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.908+0000 7f950dffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:58:16.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.908+0000 7f950dffb640 1 -- 172.21.15.43:0/2136701674 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9520007db0 con 0x55aedebcaad0 2024-04-25T08:58:16.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.908+0000 7f950dffb640 1 -- 172.21.15.43:0/2136701674 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(62..62 src has 1..62) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f9520006b50 con 0x55aedebcaad0 2024-04-25T08:58:16.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.908+0000 7f9537fa5740 1 -- 172.21.15.43:0/2136701674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9524027050 con 0x55aedebcaad0 2024-04-25T08:58:16.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.908+0000 7f950dffb640 1 -- 172.21.15.43:0/2136701674 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f952004e050 con 0x55aedebcaad0 2024-04-25T08:58:16.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.908+0000 7f9537fa5740 10 client.6918 fetch_fsmap learned FSMap version 39 2024-04-25T08:58:16.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.908+0000 7f9537fa5740 1 -- 172.21.15.43:0/2136701674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x55aedebbda60 con 0x55aedebcaad0 2024-04-25T08:58:16.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.908+0000 7f950dffb640 1 -- 172.21.15.43:0/2136701674 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9524027050 con 0x55aedebcaad0 2024-04-25T08:58:16.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.908+0000 7f950dffb640 1 -- 172.21.15.43:0/2136701674 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 39) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9520051020 con 0x55aedebcaad0 2024-04-25T08:58:16.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.908+0000 7f9537fa5740 10 client.6918 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T08:58:16.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.908+0000 7f9537fa5740 10 client.6918 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T08:58:16.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.909+0000 7f9537fa5740 20 client.6918 populate_metadata read hostname 'smithi043' 2024-04-25T08:58:16.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.909+0000 7f9537fa5740 1 --2- 172.21.15.43:0/2136701674 >> [v2:172.21.15.43:6832/1866869111,v1:172.21.15.43:6833/1866869111] conn(0x55aedeb81700 0x55aedec36d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:16.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.909+0000 7f9537fa5740 4 client.6918 mds_command: new command op to 6870 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:58:16.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.909+0000 7f9537fa5740 1 -- 172.21.15.43:0/2136701674 --> [v2:172.21.15.43:6832/1866869111,v1:172.21.15.43:6833/1866869111] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55aedeaf2f50 con 0x55aedeb81700 2024-04-25T08:58:16.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.909+0000 7f952effd640 1 --2- 172.21.15.43:0/2136701674 >> [v2:172.21.15.43:6832/1866869111,v1:172.21.15.43:6833/1866869111] conn(0x55aedeb81700 0x55aedec36d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:16.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.909+0000 7f952effd640 1 --2- 172.21.15.43:0/2136701674 >> [v2:172.21.15.43:6832/1866869111,v1:172.21.15.43:6833/1866869111] conn(0x55aedeb81700 0x55aedec36d80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc5bed45fd013d3a server_cookie=6be3b719b686b0c8 in_seq=0 out_seq=0 2024-04-25T08:58:16.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.909+0000 7f950dffb640 10 client.6918 ms_handle_connect on v2:172.21.15.43:6832/1866869111 2024-04-25T08:58:16.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.909+0000 7f950dffb640 1 -- 172.21.15.43:0/2136701674 <== mds.0 v2:172.21.15.43:6832/1866869111 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55aedeaf2f50 con 0x55aedeb81700 2024-04-25T08:58:16.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.909+0000 7f950dffb640 10 client.6918 handle_command_reply: tid=0 2024-04-25T08:58:16.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.909+0000 7f9537fa5740 1 client.6918 shutdown 2024-04-25T08:58:16.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.909+0000 7f9537fa5740 1 -- 172.21.15.43:0/2136701674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aedebcaad0 msgr2=0x55aedeb832f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:16.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.909+0000 7f9537fa5740 1 --2- 172.21.15.43:0/2136701674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aedebcaad0 0x55aedeb832f0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f9520005040 tx=0x7f95200014a0 comp rx=0 tx=0).stop 2024-04-25T08:58:16.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.909+0000 7f9537fa5740 1 -- 172.21.15.43:0/2136701674 shutdown_connections 2024-04-25T08:58:16.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.909+0000 7f9537fa5740 1 --2- 172.21.15.43:0/2136701674 >> [v2:172.21.15.43:6832/1866869111,v1:172.21.15.43:6833/1866869111] conn(0x55aedeb81700 0x55aedec36d80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:16.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.909+0000 7f9537fa5740 1 --2- 172.21.15.43:0/2136701674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55aedebcaad0 0x55aedeb832f0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:16.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.909+0000 7f9537fa5740 1 --2- 172.21.15.43:0/2136701674 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55aedebae1e0 0x55aedebafd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:16.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.909+0000 7f950dffb640 0 client.6918 ms_handle_reset on v2:172.21.15.43:6832/1866869111 2024-04-25T08:58:16.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.909+0000 7f9537fa5740 1 --2- 172.21.15.43:0/2136701674 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55aedebad230 0x55aedebbe610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:16.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.909+0000 7f9537fa5740 1 -- 172.21.15.43:0/2136701674 >> 172.21.15.43:0/2136701674 conn(0x55aedeb82f10 msgr2=0x55aedebcd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:16.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.909+0000 7f9537fa5740 1 -- 172.21.15.43:0/2136701674 shutdown_connections 2024-04-25T08:58:16.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.909+0000 7f9537fa5740 1 -- 172.21.15.43:0/2136701674 wait complete. 2024-04-25T08:58:16.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.910+0000 7f9537fa5740 20 client.6918 trim_cache size 0 max 16384 2024-04-25T08:58:16.906 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T08:58:16.906 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "6c4bd50e-aa00-4a7f-8d09-352344ef5a4b", 2024-04-25T08:58:16.906 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-04-25T08:58:16.906 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 6870, 2024-04-25T08:58:16.906 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-04-25T08:58:16.906 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-04-25T08:58:16.906 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-04-25T08:58:16.906 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 4.488502285, 2024-04-25T08:58:16.906 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 39, 2024-04-25T08:58:16.906 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 62, 2024-04-25T08:58:16.906 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 62, 2024-04-25T08:58:16.906 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 5.3458910990000001 2024-04-25T08:58:16.907 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T08:58:16.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.910+0000 7f953529e640 1 -- 172.21.15.43:0/2435075240 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f951c06b4d0 msgr2=0x7f951c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:16.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.910+0000 7f953529e640 1 --2- 172.21.15.43:0/2435075240 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f951c06b4d0 0x7f951c06d990 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f95301f6200 tx=0x7f9524008040 comp rx=0 tx=0).stop 2024-04-25T08:58:16.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.910+0000 7f953529e640 1 -- 172.21.15.43:0/2435075240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9518004d40 msgr2=0x7f95301567c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:16.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.910+0000 7f953529e640 1 --2- 172.21.15.43:0/2435075240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9518004d40 0x7f95301567c0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f9528007320 tx=0x7f95280077e0 comp rx=0 tx=0).stop 2024-04-25T08:58:16.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.910+0000 7f953529e640 1 -- 172.21.15.43:0/2435075240 shutdown_connections 2024-04-25T08:58:16.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.910+0000 7f953529e640 1 --2- 172.21.15.43:0/2435075240 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f95300b66a0 0x7f95300b6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:16.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.910+0000 7f953529e640 1 --2- 172.21.15.43:0/2435075240 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f951c06b4d0 0x7f951c06d990 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:16.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.910+0000 7f953529e640 1 --2- 172.21.15.43:0/2435075240 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f95300c4c70 0x7f95300c6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:16.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.910+0000 7f953529e640 1 --2- 172.21.15.43:0/2435075240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9518004d40 0x7f95301567c0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:16.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.910+0000 7f953529e640 1 -- 172.21.15.43:0/2435075240 >> 172.21.15.43:0/2435075240 conn(0x7f95300d17f0 msgr2=0x7f9530159a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:16.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.910+0000 7f953529e640 1 -- 172.21.15.43:0/2435075240 shutdown_connections 2024-04-25T08:58:16.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:16.910+0000 7f953529e640 1 -- 172.21.15.43:0/2435075240 wait complete. 2024-04-25T08:58:16.910 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:58:17.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.022+0000 7f5325363640 1 Processor -- start 2024-04-25T08:58:17.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.022+0000 7f5325363640 1 -- start start 2024-04-25T08:58:17.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.023+0000 7f5325363640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f5320161390 0x7f53200b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:17.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.023+0000 7f5325363640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f53200d02a0 0x7f53200d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:17.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.023+0000 7f5325363640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53200b4c30 con 0x7f53200ce5e0 2024-04-25T08:58:17.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.023+0000 7f5325363640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53200d1910 con 0x7f53200d02a0 2024-04-25T08:58:17.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.023+0000 7f5325363640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53200cf2c0 con 0x7f5320161390 2024-04-25T08:58:17.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.023+0000 7f531e7fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f53200d02a0 0x7f53200d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:17.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.023+0000 7f531e7fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f53200d02a0 0x7f53200d0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45796/0 (socket says 172.21.15.43:45796) 2024-04-25T08:58:17.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.023+0000 7f531e7fc640 1 -- 172.21.15.43:0/337709109 learned_addr learned my addr 172.21.15.43:0/337709109 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:17.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.023+0000 7f531e7fc640 1 -- 172.21.15.43:0/337709109 >> v2:172.21.15.193:3300/0 conn(0x7f5320161390 msgr2=0x7f53200b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:17.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.023+0000 7f531e7fc640 1 --2- 172.21.15.43:0/337709109 >> v2:172.21.15.193:3300/0 conn(0x7f5320161390 0x7f53200b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:17.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.023+0000 7f531e7fc640 1 -- 172.21.15.43:0/337709109 >> v1:172.21.15.193:6789/0 conn(0x7f53200ce5e0 legacy=0x7f53200cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:17.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.023+0000 7f531e7fc640 1 -- 172.21.15.43:0/337709109 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53200cf5a0 con 0x7f53200d02a0 2024-04-25T08:58:17.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.024+0000 7f531e7fc640 1 --2- 172.21.15.43:0/337709109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53200d02a0 0x7f53200d0720 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f53200c74d0 tx=0x7f531402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a9e9d6c7c68b5e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:17.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.024+0000 7f531dffb640 1 -- 172.21.15.43:0/337709109 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5314037070 con 0x7f53200d02a0 2024-04-25T08:58:17.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.024+0000 7f531dffb640 1 -- 172.21.15.43:0/337709109 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5314032440 con 0x7f53200d02a0 2024-04-25T08:58:17.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.024+0000 7f531dffb640 1 -- 172.21.15.43:0/337709109 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5314032810 con 0x7f53200d02a0 2024-04-25T08:58:17.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.025+0000 7f5325363640 1 -- 172.21.15.43:0/337709109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53200d02a0 msgr2=0x7f53200d0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:17.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.025+0000 7f5325363640 1 --2- 172.21.15.43:0/337709109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53200d02a0 0x7f53200d0720 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f53200c74d0 tx=0x7f531402a230 comp rx=0 tx=0).stop 2024-04-25T08:58:17.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.025+0000 7f5325363640 1 -- 172.21.15.43:0/337709109 shutdown_connections 2024-04-25T08:58:17.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.025+0000 7f5325363640 1 --2- 172.21.15.43:0/337709109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53200d02a0 0x7f53200d0720 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:17.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.025+0000 7f5325363640 1 --2- 172.21.15.43:0/337709109 >> v2:172.21.15.193:3300/0 conn(0x7f5320161390 0x7f53200b6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:17.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.025+0000 7f5325363640 1 -- 172.21.15.43:0/337709109 >> 172.21.15.43:0/337709109 conn(0x7f532009cd20 msgr2=0x7f53200d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:17.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.025+0000 7f5325363640 1 -- 172.21.15.43:0/337709109 shutdown_connections 2024-04-25T08:58:17.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.025+0000 7f5325363640 1 -- 172.21.15.43:0/337709109 wait complete. 2024-04-25T08:58:17.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.026+0000 7f5325363640 1 Processor -- start 2024-04-25T08:58:17.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.026+0000 7f5325363640 1 -- start start 2024-04-25T08:58:17.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.027+0000 7f5325363640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53200ce5e0 0x7f53200c8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:17.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.027+0000 7f5325363640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53200d02a0 0x7f53200cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:17.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.027+0000 7f5325363640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5320161390 0x7f53200ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:17.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.027+0000 7f5325363640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5320161770 con 0x7f53200ce5e0 2024-04-25T08:58:17.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.027+0000 7f5325363640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53200d15a0 con 0x7f53200d02a0 2024-04-25T08:58:17.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.027+0000 7f5325363640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53200d1da0 con 0x7f5320161390 2024-04-25T08:58:17.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.027+0000 7f531effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53200ce5e0 0x7f53200c8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:17.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.027+0000 7f531effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53200ce5e0 0x7f53200c8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45810/0 (socket says 172.21.15.43:45810) 2024-04-25T08:58:17.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.027+0000 7f531effd640 1 -- 172.21.15.43:0/2821137878 learned_addr learned my addr 172.21.15.43:0/2821137878 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:17.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.027+0000 7f531e7fc640 1 --2- 172.21.15.43:0/2821137878 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53200d02a0 0x7f53200cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:17.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.027+0000 7f531f7fe640 1 --2- 172.21.15.43:0/2821137878 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5320161390 0x7f53200ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:17.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.027+0000 7f531effd640 1 -- 172.21.15.43:0/2821137878 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5320161390 msgr2=0x7f53200ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:17.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.027+0000 7f531effd640 1 --2- 172.21.15.43:0/2821137878 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5320161390 0x7f53200ccd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:17.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.027+0000 7f531effd640 1 -- 172.21.15.43:0/2821137878 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53200d02a0 msgr2=0x7f53200cc7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:17.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.027+0000 7f531effd640 1 --2- 172.21.15.43:0/2821137878 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53200d02a0 0x7f53200cc7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:17.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.027+0000 7f531effd640 1 -- 172.21.15.43:0/2821137878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53200cd980 con 0x7f53200ce5e0 2024-04-25T08:58:17.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.027+0000 7f531f7fe640 1 --2- 172.21.15.43:0/2821137878 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5320161390 0x7f53200ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:17.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.027+0000 7f531e7fc640 1 --2- 172.21.15.43:0/2821137878 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53200d02a0 0x7f53200cc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:58:17.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.027+0000 7f531effd640 1 --2- 172.21.15.43:0/2821137878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53200ce5e0 0x7f53200c8d20 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f531000b410 tx=0x7f531000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:17.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.028+0000 7f53077fe640 1 -- 172.21.15.43:0/2821137878 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5310015020 con 0x7f53200ce5e0 2024-04-25T08:58:17.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.028+0000 7f53077fe640 1 -- 172.21.15.43:0/2821137878 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5310011850 con 0x7f53200ce5e0 2024-04-25T08:58:17.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.028+0000 7f53077fe640 1 -- 172.21.15.43:0/2821137878 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5310011c40 con 0x7f53200ce5e0 2024-04-25T08:58:17.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.028+0000 7f5325363640 1 -- 172.21.15.43:0/2821137878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5320154ae0 con 0x7f53200ce5e0 2024-04-25T08:58:17.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.028+0000 7f5325363640 1 -- 172.21.15.43:0/2821137878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53201550a0 con 0x7f53200ce5e0 2024-04-25T08:58:17.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.029+0000 7f5325363640 1 -- 172.21.15.43:0/2821137878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53200680f0 con 0x7f53200ce5e0 2024-04-25T08:58:17.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.029+0000 7f53077fe640 1 -- 172.21.15.43:0/2821137878 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5310018020 con 0x7f53200ce5e0 2024-04-25T08:58:17.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.029+0000 7f53077fe640 1 --2- 172.21.15.43:0/2821137878 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f530c06b4d0 0x7f530c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:17.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.030+0000 7f53077fe640 1 -- 172.21.15.43:0/2821137878 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f531008a180 con 0x7f53200ce5e0 2024-04-25T08:58:17.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.030+0000 7f531e7fc640 1 --2- 172.21.15.43:0/2821137878 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f530c06b4d0 0x7f530c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:17.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.030+0000 7f531e7fc640 1 --2- 172.21.15.43:0/2821137878 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f530c06b4d0 0x7f530c06d990 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5314034000 tx=0x7f531402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:17.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.034+0000 7f53077fe640 1 -- 172.21.15.43:0/2821137878 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f531001f020 con 0x7f53200ce5e0 2024-04-25T08:58:17.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.252+0000 7f5325363640 1 -- 172.21.15.43:0/2821137878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5320074ef0 con 0x7f53200ce5e0 2024-04-25T08:58:17.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.252+0000 7f53077fe640 1 -- 172.21.15.43:0/2821137878 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v39) v1 ==== 53+0+73 (secure 0 0 0) 0x7f53100172c0 con 0x7f53200ce5e0 2024-04-25T08:58:17.242 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:58:17.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.254+0000 7f5325363640 1 -- 172.21.15.43:0/2821137878 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f530c06b4d0 msgr2=0x7f530c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:17.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.254+0000 7f5325363640 1 --2- 172.21.15.43:0/2821137878 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f530c06b4d0 0x7f530c06d990 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5314034000 tx=0x7f531402e000 comp rx=0 tx=0).stop 2024-04-25T08:58:17.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.254+0000 7f5325363640 1 -- 172.21.15.43:0/2821137878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53200ce5e0 msgr2=0x7f53200c8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:17.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.254+0000 7f5325363640 1 --2- 172.21.15.43:0/2821137878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53200ce5e0 0x7f53200c8d20 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f531000b410 tx=0x7f531000b8d0 comp rx=0 tx=0).stop 2024-04-25T08:58:17.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.254+0000 7f5325363640 1 -- 172.21.15.43:0/2821137878 shutdown_connections 2024-04-25T08:58:17.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.254+0000 7f5325363640 1 --2- 172.21.15.43:0/2821137878 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5320161390 0x7f53200ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:17.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.254+0000 7f5325363640 1 --2- 172.21.15.43:0/2821137878 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f530c06b4d0 0x7f530c06d990 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:17.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.254+0000 7f5325363640 1 --2- 172.21.15.43:0/2821137878 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53200d02a0 0x7f53200cc7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:17.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.254+0000 7f5325363640 1 --2- 172.21.15.43:0/2821137878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53200ce5e0 0x7f53200c8d20 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:17.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.254+0000 7f5325363640 1 -- 172.21.15.43:0/2821137878 >> 172.21.15.43:0/2821137878 conn(0x7f532009cd20 msgr2=0x7f53200d38a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:17.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.254+0000 7f5325363640 1 -- 172.21.15.43:0/2821137878 shutdown_connections 2024-04-25T08:58:17.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:17.254+0000 7f5325363640 1 -- 172.21.15.43:0/2821137878 wait complete. 2024-04-25T08:58:17.255 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:58:17.255 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:58:17.255 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-04-25T08:58:17.255 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:58:17.255 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:58:17.255 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:58:17.255 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:58:17.255 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:58:17.255 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:58:17.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:17.255 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-04-25T08:58:17.271 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:58:17.271 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:58:17.272 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:58:17.272 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:58:17.272 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-04-25T08:58:17.272 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:58:17.272 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:58:17.272 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-04-25T08:58:17.272 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-04-25T08:58:17.272 INFO:teuthology.orchestra.run.smithi043.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:58:17.272 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:58:17.272 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-04-25T08:58:17.272 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T08:58:17.272 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 42521sec preferred_lft 42521sec 2024-04-25T08:58:17.272 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link noprefixroute 2024-04-25T08:58:17.272 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:58:17.273 INFO:teuthology.orchestra.run.smithi043.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:58:17.273 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-04-25T08:58:17.273 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-04-25T08:58:17.273 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:58:17.273 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:58:17.273 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-04-25T08:58:17.273 INFO:teuthology.orchestra.run.smithi043.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:58:17.273 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 12:2c:7a:72:36:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:58:17.273 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:58:17.273 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:58:17.273 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ccff:4eff:fe9d:c8bf/64 scope link 2024-04-25T08:58:17.273 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:58:17.273 INFO:teuthology.orchestra.run.smithi043.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:58:17.273 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 16:9a:63:37:46:24 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:58:17.274 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::149a:63ff:fe37:4624/64 scope link 2024-04-25T08:58:17.274 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:58:17.274 INFO:teuthology.orchestra.run.smithi043.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:58:17.274 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 12:2c:7a:72:36:14 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:58:17.274 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::102c:7aff:fe72:3614/64 scope link 2024-04-25T08:58:17.274 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:58:17.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:17.274 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:58:17.335 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-04-25T08:58:17.337 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:17.337 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-04-25T08:58:17.393 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-04-25T08:58:17.393 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:58:17.393 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T08:58:17.393 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T08:58:17.393 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:58:17.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:17.393 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2024-04-25T08:58:17.393 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T08:58:17.394 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:58:17.394 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:58:17.394 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:58:17.394 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T08:58:17.413 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-04-25T08:58:17.423+0000 7f1586a1e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:58:17.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:17.459 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-04-25T08:58:17.473 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:58:17.474 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:58:17.474 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:58:17.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:17.474 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-04-25T08:58:17.539 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T08:58:17.539 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:58:17.539 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:58:17.595 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:58:17.595 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:58:17.595 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:58:17.650 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-04-25T08:58:17.714 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:58:17.771 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-04-25T08:58:17.771 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-04-25T08:58:17.771 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-04-25T08:58:17.771 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-04-25T08:58:17.771 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-04-25T08:58:17.771 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-04-25T08:58:17.771 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-04-25T08:58:17.771 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-04-25T08:58:17.772 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-04-25T08:58:17.772 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-04-25T08:58:17.772 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-04-25T08:58:17.772 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/selinux 2024-04-25T08:58:17.772 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:58:17.772 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-04-25T08:58:17.772 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-04-25T08:58:17.772 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-04-25T08:58:17.772 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-04-25T08:58:17.772 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:58:17.772 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-04-25T08:58:17.772 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-04-25T08:58:17.772 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:58:17.772 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:58:17.773 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:58:17.773 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:58:17.773 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-04-25T08:58:17.773 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-04-25T08:58:17.773 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:58:17.773 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:58:17.773 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:58:17.773 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:58:17.773 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-04-25T08:58:17.773 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:58:17.773 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:58:17.773 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:58:17.773 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:58:17.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:17.774 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-04-25T08:58:17.826 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T08:58:17.826 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T08:58:17.868 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-04-25T08:58:17.892 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:58:17.948 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-04-25T08:58:17.949 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-04-25T08:58:17.949 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-04-25T08:58:17.949 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-04-25T08:58:17.949 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-04-25T08:58:17.949 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-04-25T08:58:17.949 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-04-25T08:58:17.949 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-04-25T08:58:17.949 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-04-25T08:58:17.949 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-04-25T08:58:17.949 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-04-25T08:58:17.949 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/selinux 2024-04-25T08:58:17.949 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:58:17.949 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-04-25T08:58:17.949 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-04-25T08:58:17.950 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-04-25T08:58:17.950 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-04-25T08:58:17.950 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:58:17.950 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-04-25T08:58:17.950 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-04-25T08:58:17.950 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:58:17.950 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:58:17.950 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:58:17.950 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:58:17.950 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-04-25T08:58:17.950 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-04-25T08:58:17.950 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:58:17.950 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:58:17.950 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:58:17.951 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:58:17.951 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-04-25T08:58:17.951 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:58:17.951 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:58:17.951 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:58:17.951 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:58:17.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:17.952 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-04-25T08:58:17.952 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:2024-04-25T08:58:17.961+0000 7fab5dd22480 -1 init, newargv = 0x55b892c97bf0 newargc=15 2024-04-25T08:58:17.953 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[64398]: starting ceph client 2024-04-25T08:58:17.955 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[64398]: starting fuse 2024-04-25T08:58:17.969 INFO:teuthology.orchestra.run.smithi043.stdout:43 2024-04-25T08:58:17.970 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-25T08:58:17.970 DEBUG:teuthology.orchestra.run.smithi043:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:58:17.970 DEBUG:teuthology.orchestra.run.smithi043:> import glob 2024-04-25T08:58:17.970 DEBUG:teuthology.orchestra.run.smithi043:> import re 2024-04-25T08:58:17.970 DEBUG:teuthology.orchestra.run.smithi043:> import os 2024-04-25T08:58:17.970 DEBUG:teuthology.orchestra.run.smithi043:> import subprocess 2024-04-25T08:58:17.970 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T08:58:17.970 DEBUG:teuthology.orchestra.run.smithi043:> def _find_admin_socket(client_name): 2024-04-25T08:58:17.970 DEBUG:teuthology.orchestra.run.smithi043:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T08:58:17.970 DEBUG:teuthology.orchestra.run.smithi043:> files = glob.glob(asok_path) 2024-04-25T08:58:17.970 DEBUG:teuthology.orchestra.run.smithi043:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T08:58:17.970 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T08:58:17.971 DEBUG:teuthology.orchestra.run.smithi043:> # Given a non-glob path, it better be there 2024-04-25T08:58:17.971 DEBUG:teuthology.orchestra.run.smithi043:> if "*" not in asok_path: 2024-04-25T08:58:17.971 DEBUG:teuthology.orchestra.run.smithi043:> assert(len(files) == 1) 2024-04-25T08:58:17.971 DEBUG:teuthology.orchestra.run.smithi043:> return files[0] 2024-04-25T08:58:17.971 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T08:58:17.971 DEBUG:teuthology.orchestra.run.smithi043:> for f in files: 2024-04-25T08:58:17.971 DEBUG:teuthology.orchestra.run.smithi043:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:58:17.971 DEBUG:teuthology.orchestra.run.smithi043:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:58:17.971 DEBUG:teuthology.orchestra.run.smithi043:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:58:17.971 DEBUG:teuthology.orchestra.run.smithi043:> contents = proc_f.read() 2024-04-25T08:58:17.971 DEBUG:teuthology.orchestra.run.smithi043:> if mountpoint in contents: 2024-04-25T08:58:17.971 DEBUG:teuthology.orchestra.run.smithi043:> return f 2024-04-25T08:58:17.971 DEBUG:teuthology.orchestra.run.smithi043:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:58:17.971 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T08:58:17.971 DEBUG:teuthology.orchestra.run.smithi043:> print(_find_admin_socket("client.0")) 2024-04-25T08:58:17.971 DEBUG:teuthology.orchestra.run.smithi043:> ' 2024-04-25T08:58:18.089 INFO:teuthology.orchestra.run.smithi043.stdout:/var/run/ceph/ceph-client.0.64398.asok 2024-04-25T08:58:18.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:58:18.100 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.64398.asok 2024-04-25T08:58:18.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:18.100 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.64398.asok status 2024-04-25T08:58:18.238 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T08:58:18.238 INFO:teuthology.orchestra.run.smithi043.stdout: "metadata": { 2024-04-25T08:58:18.238 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:58:18.238 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:58:18.238 INFO:teuthology.orchestra.run.smithi043.stdout: "entity_id": "0", 2024-04-25T08:58:18.238 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": "smithi043", 2024-04-25T08:58:18.238 INFO:teuthology.orchestra.run.smithi043.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T08:58:18.238 INFO:teuthology.orchestra.run.smithi043.stdout: "pid": "64398", 2024-04-25T08:58:18.238 INFO:teuthology.orchestra.run.smithi043.stdout: "root": "/" 2024-04-25T08:58:18.238 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:58:18.239 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_count": 0, 2024-04-25T08:58:18.239 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_pinned_count": 0, 2024-04-25T08:58:18.239 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 6933, 2024-04-25T08:58:18.239 INFO:teuthology.orchestra.run.smithi043.stdout: "inst": { 2024-04-25T08:58:18.239 INFO:teuthology.orchestra.run.smithi043.stdout: "name": { 2024-04-25T08:58:18.239 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "client", 2024-04-25T08:58:18.239 INFO:teuthology.orchestra.run.smithi043.stdout: "num": 6933 2024-04-25T08:58:18.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:58:18.239 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-04-25T08:58:18.239 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-04-25T08:58:18.239 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:58:18.239 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3108900474 2024-04-25T08:58:18.239 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:58:18.239 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:58:18.240 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-04-25T08:58:18.240 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-04-25T08:58:18.240 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:58:18.240 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 3108900474 2024-04-25T08:58:18.240 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:58:18.240 INFO:teuthology.orchestra.run.smithi043.stdout: "inst_str": "client.6933 192.168.0.1:0/3108900474", 2024-04-25T08:58:18.240 INFO:teuthology.orchestra.run.smithi043.stdout: "addr_str": "192.168.0.1:0/3108900474", 2024-04-25T08:58:18.240 INFO:teuthology.orchestra.run.smithi043.stdout: "inode_count": 1, 2024-04-25T08:58:18.240 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_epoch": 39, 2024-04-25T08:58:18.240 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch": 62, 2024-04-25T08:58:18.240 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:58:18.240 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklisted": false, 2024-04-25T08:58:18.240 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs" 2024-04-25T08:58:18.240 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T08:58:18.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:18.246 DEBUG:teuthology.orchestra.run.smithi043:> stat --file-system '--printf=%T 2024-04-25T08:58:18.246 DEBUG:teuthology.orchestra.run.smithi043:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T08:58:18.262 INFO:teuthology.orchestra.run.smithi043.stdout:fuseblk 2024-04-25T08:58:18.262 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T08:58:18.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:18.262 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:58:18.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:18.327 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir scrub_repair_path) 2024-04-25T08:58:18.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:58:18.385 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove 'scrub_repair_path': Directory not empty 2024-04-25T08:58:18.385 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_repair_path repair' 2024-04-25T08:58:18.386 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:58:18.386 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub start /scrub_repair_path repair 2024-04-25T08:58:18.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.542+0000 7f63b398b640 1 Processor -- start 2024-04-25T08:58:18.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.542+0000 7f63b398b640 1 -- start start 2024-04-25T08:58:18.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.543+0000 7f63b398b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f63ac0aeea0 0x7f63ac15ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:18.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.543+0000 7f63b398b640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f63ac0b4300 0x7f63ac0cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:18.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.543+0000 7f63b398b640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63ac15f390 con 0x7f63ac0cf2e0 2024-04-25T08:58:18.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.543+0000 7f63b398b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63ac0b2710 con 0x7f63ac0aeea0 2024-04-25T08:58:18.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.543+0000 7f63b398b640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63ac15ae50 con 0x7f63ac0b4300 2024-04-25T08:58:18.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.543+0000 7f63b1727640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f63ac0aeea0 0x7f63ac15ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:18.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.543+0000 7f63b1727640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f63ac0aeea0 0x7f63ac15ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45826/0 (socket says 172.21.15.43:45826) 2024-04-25T08:58:18.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.543+0000 7f63b1727640 1 -- 172.21.15.43:0/931200389 learned_addr learned my addr 172.21.15.43:0/931200389 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:18.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.543+0000 7f63b1727640 1 -- 172.21.15.43:0/931200389 >> v2:172.21.15.193:3300/0 conn(0x7f63ac0b4300 msgr2=0x7f63ac0cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:18.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.543+0000 7f63b1727640 1 --2- 172.21.15.43:0/931200389 >> v2:172.21.15.193:3300/0 conn(0x7f63ac0b4300 0x7f63ac0cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:18.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.543+0000 7f63b1727640 1 -- 172.21.15.43:0/931200389 >> v1:172.21.15.193:6789/0 conn(0x7f63ac0cf2e0 legacy=0x7f63ac15b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:18.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.543+0000 7f63b1727640 1 -- 172.21.15.43:0/931200389 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63ac06dfe0 con 0x7f63ac0aeea0 2024-04-25T08:58:18.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.544+0000 7f63b1727640 1 --2- 172.21.15.43:0/931200389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63ac0aeea0 0x7f63ac15ee50 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f63ac0cd430 tx=0x7f63a802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aa450a3c8c5a593 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:18.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.544+0000 7f63a3fff640 1 -- 172.21.15.43:0/931200389 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a8037070 con 0x7f63ac0aeea0 2024-04-25T08:58:18.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.544+0000 7f63a3fff640 1 -- 172.21.15.43:0/931200389 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a8032440 con 0x7f63ac0aeea0 2024-04-25T08:58:18.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.544+0000 7f63a3fff640 1 -- 172.21.15.43:0/931200389 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a8032810 con 0x7f63ac0aeea0 2024-04-25T08:58:18.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.545+0000 7f63b398b640 1 -- 172.21.15.43:0/931200389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63ac0aeea0 msgr2=0x7f63ac15ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:18.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.545+0000 7f63b398b640 1 --2- 172.21.15.43:0/931200389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63ac0aeea0 0x7f63ac15ee50 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f63ac0cd430 tx=0x7f63a802a1a0 comp rx=0 tx=0).stop 2024-04-25T08:58:18.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.545+0000 7f63b398b640 1 -- 172.21.15.43:0/931200389 shutdown_connections 2024-04-25T08:58:18.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.545+0000 7f63b398b640 1 --2- 172.21.15.43:0/931200389 >> v2:172.21.15.193:3300/0 conn(0x7f63ac0b4300 0x7f63ac0cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:18.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.545+0000 7f63b398b640 1 --2- 172.21.15.43:0/931200389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63ac0aeea0 0x7f63ac15ee50 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:18.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.545+0000 7f63b398b640 1 -- 172.21.15.43:0/931200389 >> 172.21.15.43:0/931200389 conn(0x7f63ac06ab30 msgr2=0x7f63ac06af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:18.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.545+0000 7f63b398b640 1 -- 172.21.15.43:0/931200389 shutdown_connections 2024-04-25T08:58:18.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.545+0000 7f63b398b640 1 -- 172.21.15.43:0/931200389 wait complete. 2024-04-25T08:58:18.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.546+0000 7f63b398b640 1 Processor -- start 2024-04-25T08:58:18.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.546+0000 7f63b398b640 1 -- start start 2024-04-25T08:58:18.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.547+0000 7f63b398b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63ac0aeea0 0x7f63ac147f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:18.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.547+0000 7f63b398b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f63ac0b4300 0x7f63ac15d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:18.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.547+0000 7f63b398b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63ac0cf2e0 0x7f63ac15da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:18.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.547+0000 7f63b398b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac0b2c40 con 0x7f63ac0aeea0 2024-04-25T08:58:18.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.547+0000 7f63b398b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac0b2960 con 0x7f63ac0cf2e0 2024-04-25T08:58:18.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.547+0000 7f63b398b640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac0cdaa0 con 0x7f63ac0b4300 2024-04-25T08:58:18.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.547+0000 7f63b1727640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63ac0aeea0 0x7f63ac147f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:18.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.547+0000 7f63b1727640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63ac0aeea0 0x7f63ac147f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45830/0 (socket says 172.21.15.43:45830) 2024-04-25T08:58:18.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.547+0000 7f63b1727640 1 -- 172.21.15.43:0/1437199193 learned_addr learned my addr 172.21.15.43:0/1437199193 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:18.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.547+0000 7f63b0f26640 1 --2- 172.21.15.43:0/1437199193 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f63ac0b4300 0x7f63ac15d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:18.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.547+0000 7f63b1f28640 1 --2- 172.21.15.43:0/1437199193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63ac0cf2e0 0x7f63ac15da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:18.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.547+0000 7f63b1727640 1 -- 172.21.15.43:0/1437199193 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f63ac0b4300 msgr2=0x7f63ac15d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:18.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.547+0000 7f63b1727640 1 --2- 172.21.15.43:0/1437199193 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f63ac0b4300 0x7f63ac15d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:18.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.547+0000 7f63b1727640 1 -- 172.21.15.43:0/1437199193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63ac0cf2e0 msgr2=0x7f63ac15da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:18.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.547+0000 7f63b1727640 1 --2- 172.21.15.43:0/1437199193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63ac0cf2e0 0x7f63ac15da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:18.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.547+0000 7f63b1727640 1 -- 172.21.15.43:0/1437199193 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63ac158250 con 0x7f63ac0aeea0 2024-04-25T08:58:18.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.547+0000 7f63b0f26640 1 --2- 172.21.15.43:0/1437199193 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f63ac0b4300 0x7f63ac15d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:18.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.547+0000 7f63b1f28640 1 --2- 172.21.15.43:0/1437199193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63ac0cf2e0 0x7f63ac15da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:18.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.547+0000 7f63b1727640 1 --2- 172.21.15.43:0/1437199193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63ac0aeea0 0x7f63ac147f70 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f63a8000c00 tx=0x7f63a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:18.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.547+0000 7f63a1ffb640 1 -- 172.21.15.43:0/1437199193 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a8041070 con 0x7f63ac0aeea0 2024-04-25T08:58:18.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.548+0000 7f63a1ffb640 1 -- 172.21.15.43:0/1437199193 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a803c870 con 0x7f63ac0aeea0 2024-04-25T08:58:18.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.548+0000 7f63b398b640 1 -- 172.21.15.43:0/1437199193 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63ac158530 con 0x7f63ac0aeea0 2024-04-25T08:58:18.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.548+0000 7f63a1ffb640 1 -- 172.21.15.43:0/1437199193 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a8037040 con 0x7f63ac0aeea0 2024-04-25T08:58:18.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.548+0000 7f63b398b640 1 -- 172.21.15.43:0/1437199193 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63ac158af0 con 0x7f63ac0aeea0 2024-04-25T08:58:18.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.549+0000 7f63a1ffb640 1 -- 172.21.15.43:0/1437199193 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f63a8044070 con 0x7f63ac0aeea0 2024-04-25T08:58:18.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.550+0000 7f63a1ffb640 1 --2- 172.21.15.43:0/1437199193 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f639006b4d0 0x7f639006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:18.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.550+0000 7f63a1ffb640 1 -- 172.21.15.43:0/1437199193 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f63a80aaeb0 con 0x7f63ac0aeea0 2024-04-25T08:58:18.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.550+0000 7f63b0f26640 1 --2- 172.21.15.43:0/1437199193 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f639006b4d0 0x7f639006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:18.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.550+0000 7f63b0f26640 1 --2- 172.21.15.43:0/1437199193 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f639006b4d0 0x7f639006d990 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f639c006fd0 tx=0x7f639c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:18.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.551+0000 7f63b6768740 1 Processor -- start 2024-04-25T08:58:18.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.551+0000 7f63b6768740 1 -- start start 2024-04-25T08:58:18.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.551+0000 7f63b6768740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a8940c5d20 0x55a8940786b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:18.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.551+0000 7f63b6768740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a8940a8e40 0x55a8940a81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:18.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.551+0000 7f63b6768740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8940a8700 0x55a89407d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:18.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.551+0000 7f63b6768740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a893fe0f40 con 0x55a8940a8e40 2024-04-25T08:58:18.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.551+0000 7f63b6768740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a893f800d0 con 0x55a8940a8700 2024-04-25T08:58:18.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.551+0000 7f63b6768740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a893fdcfe0 con 0x55a8940c5d20 2024-04-25T08:58:18.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.551+0000 7f63b0f26640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a8940a8e40 0x55a8940a81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:18.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.552+0000 7f63b0f26640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a8940a8e40 0x55a8940a81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45838/0 (socket says 172.21.15.43:45838) 2024-04-25T08:58:18.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.552+0000 7f63b0f26640 1 -- 172.21.15.43:0/3500183052 learned_addr learned my addr 172.21.15.43:0/3500183052 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:18.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.552+0000 7f63b1727640 1 --2- 172.21.15.43:0/3500183052 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a8940c5d20 0x55a8940786b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:18.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.552+0000 7f63b1f28640 1 --2- 172.21.15.43:0/3500183052 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8940a8700 0x55a89407d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:18.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.552+0000 7f63b0f26640 1 -- 172.21.15.43:0/3500183052 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a8940c5d20 msgr2=0x55a8940786b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:18.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.552+0000 7f63b0f26640 1 --2- 172.21.15.43:0/3500183052 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a8940c5d20 0x55a8940786b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:18.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.552+0000 7f63b0f26640 1 -- 172.21.15.43:0/3500183052 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8940a8700 msgr2=0x55a89407d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:18.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.552+0000 7f63b0f26640 1 --2- 172.21.15.43:0/3500183052 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8940a8700 0x55a89407d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:18.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.552+0000 7f63b0f26640 1 -- 172.21.15.43:0/3500183052 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a893dc1af0 con 0x55a8940a8e40 2024-04-25T08:58:18.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.552+0000 7f63b0f26640 1 --2- 172.21.15.43:0/3500183052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a8940a8e40 0x55a8940a81c0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f639c00cae0 tx=0x7f639c00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda3da5caa1e8c31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:18.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.552+0000 7f6396ffd640 1 -- 172.21.15.43:0/3500183052 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639c023070 con 0x55a8940a8e40 2024-04-25T08:58:18.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.552+0000 7f6396ffd640 1 -- 172.21.15.43:0/3500183052 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f639c01e400 con 0x55a8940a8e40 2024-04-25T08:58:18.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.552+0000 7f6396ffd640 1 -- 172.21.15.43:0/3500183052 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639c01e7d0 con 0x55a8940a8e40 2024-04-25T08:58:18.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.553+0000 7f63b6768740 1 -- 172.21.15.43:0/3500183052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a8940a8e40 msgr2=0x55a8940a81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:18.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.553+0000 7f63b6768740 1 --2- 172.21.15.43:0/3500183052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a8940a8e40 0x55a8940a81c0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f639c00cae0 tx=0x7f639c00f000 comp rx=0 tx=0).stop 2024-04-25T08:58:18.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.553+0000 7f63b6768740 1 -- 172.21.15.43:0/3500183052 shutdown_connections 2024-04-25T08:58:18.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.553+0000 7f63b6768740 1 --2- 172.21.15.43:0/3500183052 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8940a8700 0x55a89407d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:18.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.553+0000 7f63b6768740 1 --2- 172.21.15.43:0/3500183052 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a8940a8e40 0x55a8940a81c0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:18.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.553+0000 7f63b6768740 1 --2- 172.21.15.43:0/3500183052 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a8940c5d20 0x55a8940786b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:18.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.553+0000 7f63b6768740 1 -- 172.21.15.43:0/3500183052 >> 172.21.15.43:0/3500183052 conn(0x55a89407a920 msgr2=0x55a8940c31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:18.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.553+0000 7f63b6768740 1 -- 172.21.15.43:0/3500183052 shutdown_connections 2024-04-25T08:58:18.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.553+0000 7f63b6768740 1 -- 172.21.15.43:0/3500183052 wait complete. 2024-04-25T08:58:18.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.553+0000 7f63b6768740 1 -- start start 2024-04-25T08:58:18.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.553+0000 7f63b6768740 1 Processor -- start 2024-04-25T08:58:18.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.554+0000 7f63b6768740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8940a8e40 0x55a8940a3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:18.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.554+0000 7f63b6768740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a8940c5d20 0x55a8940ab490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:18.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.554+0000 7f63b6768740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a8940ab9d0 0x55a8940c6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:18.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.554+0000 7f63b6768740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a89407cad0 con 0x55a8940c5d20 2024-04-25T08:58:18.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.554+0000 7f63b6768740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a893f7dba0 con 0x55a8940a8e40 2024-04-25T08:58:18.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.554+0000 7f63b6768740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a89407bec0 con 0x55a8940ab9d0 2024-04-25T08:58:18.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.554+0000 7f63b6768740 10 client.0 _ll_register_callbacks cb 0x55a893f7d570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:58:18.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.554+0000 7f63b0f26640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a8940c5d20 0x55a8940ab490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:18.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.554+0000 7f63b0f26640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a8940c5d20 0x55a8940ab490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45846/0 (socket says 172.21.15.43:45846) 2024-04-25T08:58:18.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.554+0000 7f63b0f26640 1 -- 172.21.15.43:0/1510136633 learned_addr learned my addr 172.21.15.43:0/1510136633 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:18.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.555+0000 7f63b1727640 1 --2- 172.21.15.43:0/1510136633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8940a8e40 0x55a8940a3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:18.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.555+0000 7f63b1f28640 1 --2- 172.21.15.43:0/1510136633 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a8940ab9d0 0x55a8940c6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:18.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.555+0000 7f63b0f26640 1 -- 172.21.15.43:0/1510136633 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a8940ab9d0 msgr2=0x55a8940c6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:18.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.555+0000 7f63b0f26640 1 --2- 172.21.15.43:0/1510136633 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a8940ab9d0 0x55a8940c6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:18.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.555+0000 7f63b0f26640 1 -- 172.21.15.43:0/1510136633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8940a8e40 msgr2=0x55a8940a3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:18.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.555+0000 7f63b0f26640 1 --2- 172.21.15.43:0/1510136633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8940a8e40 0x55a8940a3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:18.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.555+0000 7f63b0f26640 1 -- 172.21.15.43:0/1510136633 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a893cb53e0 con 0x55a8940c5d20 2024-04-25T08:58:18.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.555+0000 7f63b0f26640 1 --2- 172.21.15.43:0/1510136633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a8940c5d20 0x55a8940ab490 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f639c0137f0 tx=0x7f639c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375f911de8e308c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:18.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.555+0000 7f638bfff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:58:18.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.555+0000 7f638bfff640 1 -- 172.21.15.43:0/1510136633 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639c02d020 con 0x55a8940c5d20 2024-04-25T08:58:18.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.555+0000 7f638bfff640 1 -- 172.21.15.43:0/1510136633 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(62..62 src has 1..62) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f639c01ec10 con 0x55a8940c5d20 2024-04-25T08:58:18.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.555+0000 7f63b6768740 1 -- 172.21.15.43:0/1510136633 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f639c01e400 con 0x55a8940c5d20 2024-04-25T08:58:18.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.555+0000 7f638bfff640 1 -- 172.21.15.43:0/1510136633 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f639c037050 con 0x55a8940c5d20 2024-04-25T08:58:18.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.555+0000 7f63b6768740 10 client.6945 fetch_fsmap learned FSMap version 39 2024-04-25T08:58:18.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.555+0000 7f63b6768740 1 -- 172.21.15.43:0/1510136633 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x55a893e22280 con 0x55a8940c5d20 2024-04-25T08:58:18.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.555+0000 7f638bfff640 1 -- 172.21.15.43:0/1510136633 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f639c01e400 con 0x55a8940c5d20 2024-04-25T08:58:18.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.555+0000 7f638bfff640 1 -- 172.21.15.43:0/1510136633 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 39) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f639c03a020 con 0x55a8940c5d20 2024-04-25T08:58:18.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.555+0000 7f63b6768740 10 client.6945 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T08:58:18.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.555+0000 7f63b6768740 10 client.6945 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T08:58:18.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.555+0000 7f63b6768740 20 client.6945 populate_metadata read hostname 'smithi043' 2024-04-25T08:58:18.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.556+0000 7f63b6768740 1 --2- 172.21.15.43:0/1510136633 >> [v2:172.21.15.43:6832/1866869111,v1:172.21.15.43:6833/1866869111] conn(0x55a8940a9800 0x55a8940a9c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:18.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.556+0000 7f63b6768740 4 client.6945 mds_command: new command op to 6870 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:58:18.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.556+0000 7f63b6768740 1 -- 172.21.15.43:0/1510136633 --> [v2:172.21.15.43:6832/1866869111,v1:172.21.15.43:6833/1866869111] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a893fee480 con 0x55a8940a9800 2024-04-25T08:58:18.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.556+0000 7f63b1727640 1 --2- 172.21.15.43:0/1510136633 >> [v2:172.21.15.43:6832/1866869111,v1:172.21.15.43:6833/1866869111] conn(0x55a8940a9800 0x55a8940a9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:18.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.556+0000 7f638bfff640 10 client.6945 ms_handle_connect on v2:172.21.15.43:6832/1866869111 2024-04-25T08:58:18.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.556+0000 7f63b1727640 1 --2- 172.21.15.43:0/1510136633 >> [v2:172.21.15.43:6832/1866869111,v1:172.21.15.43:6833/1866869111] conn(0x55a8940a9800 0x55a8940a9c00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=daa68ae66d3b8ae6 server_cookie=a61a7b9a90ea81bb in_seq=0 out_seq=0 2024-04-25T08:58:18.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.556+0000 7f638bfff640 1 -- 172.21.15.43:0/1510136633 <== mds.0 v2:172.21.15.43:6832/1866869111 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a893fee480 con 0x55a8940a9800 2024-04-25T08:58:18.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.556+0000 7f638bfff640 10 client.6945 handle_command_reply: tid=0 2024-04-25T08:58:18.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.556+0000 7f63b6768740 1 client.6945 shutdown 2024-04-25T08:58:18.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.557+0000 7f63b6768740 1 -- 172.21.15.43:0/1510136633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a8940c5d20 msgr2=0x55a8940ab490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:18.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.557+0000 7f63b6768740 1 --2- 172.21.15.43:0/1510136633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a8940c5d20 0x55a8940ab490 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f639c0137f0 tx=0x7f639c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:58:18.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.557+0000 7f63b6768740 1 -- 172.21.15.43:0/1510136633 shutdown_connections 2024-04-25T08:58:18.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.557+0000 7f63b6768740 1 --2- 172.21.15.43:0/1510136633 >> [v2:172.21.15.43:6832/1866869111,v1:172.21.15.43:6833/1866869111] conn(0x55a8940a9800 0x55a8940a9c00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:18.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.557+0000 7f63b6768740 1 --2- 172.21.15.43:0/1510136633 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a8940ab9d0 0x55a8940c6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:18.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.557+0000 7f63b6768740 1 --2- 172.21.15.43:0/1510136633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a8940c5d20 0x55a8940ab490 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:18.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.557+0000 7f638bfff640 0 client.6945 ms_handle_reset on v2:172.21.15.43:6832/1866869111 2024-04-25T08:58:18.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.557+0000 7f63b6768740 1 --2- 172.21.15.43:0/1510136633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8940a8e40 0x55a8940a3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:18.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.557+0000 7f63b6768740 1 -- 172.21.15.43:0/1510136633 >> 172.21.15.43:0/1510136633 conn(0x55a8940a8700 msgr2=0x55a8940c31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:18.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.557+0000 7f63b6768740 1 -- 172.21.15.43:0/1510136633 shutdown_connections 2024-04-25T08:58:18.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.557+0000 7f63b6768740 1 -- 172.21.15.43:0/1510136633 wait complete. 2024-04-25T08:58:18.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.557+0000 7f63b6768740 20 client.6945 trim_cache size 0 max 16384 2024-04-25T08:58:18.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.572+0000 7f63b6768740 1 Processor -- start 2024-04-25T08:58:18.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.572+0000 7f63b6768740 1 -- start start 2024-04-25T08:58:18.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.573+0000 7f63b6768740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a8940a8e40 0x55a8940c2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:18.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.573+0000 7f63b6768740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8940ab9d0 0x55a8940c3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:18.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.573+0000 7f63b6768740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a8940c5d20 0x55a8940c1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:18.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.573+0000 7f63b6768740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac1fbb20 con 0x55a8940a8e40 2024-04-25T08:58:18.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.573+0000 7f63b6768740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac15c4b0 con 0x55a8940ab9d0 2024-04-25T08:58:18.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.573+0000 7f63b6768740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac1f3890 con 0x55a8940c5d20 2024-04-25T08:58:18.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.573+0000 7f63b1727640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a8940a8e40 0x55a8940c2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:18.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.573+0000 7f63b1727640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a8940a8e40 0x55a8940c2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45862/0 (socket says 172.21.15.43:45862) 2024-04-25T08:58:18.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.573+0000 7f63b1727640 1 -- 172.21.15.43:0/2820974172 learned_addr learned my addr 172.21.15.43:0/2820974172 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:18.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.573+0000 7f63b1727640 1 -- 172.21.15.43:0/2820974172 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a8940c5d20 msgr2=0x55a8940c1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:18.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.573+0000 7f63b1727640 1 --2- 172.21.15.43:0/2820974172 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a8940c5d20 0x55a8940c1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:18.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.573+0000 7f63b1727640 1 -- 172.21.15.43:0/2820974172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8940ab9d0 msgr2=0x55a8940c3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:18.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.573+0000 7f63b1727640 1 --2- 172.21.15.43:0/2820974172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8940ab9d0 0x55a8940c3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:18.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.573+0000 7f63b1727640 1 -- 172.21.15.43:0/2820974172 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a893c8e560 con 0x55a8940a8e40 2024-04-25T08:58:18.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.573+0000 7f63b1727640 1 --2- 172.21.15.43:0/2820974172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a8940a8e40 0x55a8940c2fc0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f63a80794b0 tx=0x7f63a80758c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ce36e0e7b46de4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:18.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.574+0000 7f63967fc640 1 -- 172.21.15.43:0/2820974172 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a80a7070 con 0x55a8940a8e40 2024-04-25T08:58:18.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.574+0000 7f63967fc640 1 -- 172.21.15.43:0/2820974172 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a80a93a0 con 0x55a8940a8e40 2024-04-25T08:58:18.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.574+0000 7f63967fc640 1 -- 172.21.15.43:0/2820974172 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a8091070 con 0x55a8940a8e40 2024-04-25T08:58:18.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.574+0000 7f63b6768740 1 -- 172.21.15.43:0/2820974172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a8940a8e40 msgr2=0x55a8940c2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:18.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.574+0000 7f63b6768740 1 --2- 172.21.15.43:0/2820974172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a8940a8e40 0x55a8940c2fc0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f63a80794b0 tx=0x7f63a80758c0 comp rx=0 tx=0).stop 2024-04-25T08:58:18.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.574+0000 7f63b6768740 1 -- 172.21.15.43:0/2820974172 shutdown_connections 2024-04-25T08:58:18.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.574+0000 7f63b6768740 1 --2- 172.21.15.43:0/2820974172 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a8940c5d20 0x55a8940c1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:18.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.574+0000 7f63b6768740 1 --2- 172.21.15.43:0/2820974172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8940ab9d0 0x55a8940c3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:18.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.574+0000 7f63b6768740 1 --2- 172.21.15.43:0/2820974172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a8940a8e40 0x55a8940c2fc0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:18.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.574+0000 7f63b6768740 1 -- 172.21.15.43:0/2820974172 >> 172.21.15.43:0/2820974172 conn(0x55a8940a8700 msgr2=0x55a8940bc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:18.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.574+0000 7f63b6768740 1 -- 172.21.15.43:0/2820974172 shutdown_connections 2024-04-25T08:58:18.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.574+0000 7f63b6768740 1 -- 172.21.15.43:0/2820974172 wait complete. 2024-04-25T08:58:18.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.575+0000 7f63b6768740 1 -- start start 2024-04-25T08:58:18.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.575+0000 7f63b6768740 1 Processor -- start 2024-04-25T08:58:18.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.576+0000 7f63b6768740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a8940a8e40 0x55a894132a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:18.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.576+0000 7f63b6768740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8940ab9d0 0x55a89407cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:18.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.576+0000 7f63b6768740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a8940c5d20 0x55a8940a2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:18.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.576+0000 7f63b6768740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8940d6e40 con 0x55a8940a8e40 2024-04-25T08:58:18.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.576+0000 7f63b6768740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ac15beb0 con 0x55a8940ab9d0 2024-04-25T08:58:18.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.576+0000 7f63b6768740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a89407e9e0 con 0x55a8940c5d20 2024-04-25T08:58:18.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.576+0000 7f63b6768740 10 client.0 _ll_register_callbacks cb 0x7f639c021070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:58:18.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.576+0000 7f63b1727640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a8940a8e40 0x55a894132a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:18.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.576+0000 7f63b1727640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a8940a8e40 0x55a894132a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45870/0 (socket says 172.21.15.43:45870) 2024-04-25T08:58:18.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.576+0000 7f63b1727640 1 -- 172.21.15.43:0/3348898902 learned_addr learned my addr 172.21.15.43:0/3348898902 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:18.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.576+0000 7f63b1727640 1 -- 172.21.15.43:0/3348898902 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a8940c5d20 msgr2=0x55a8940a2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:18.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.576+0000 7f63b0f26640 1 --2- 172.21.15.43:0/3348898902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8940ab9d0 0x55a89407cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:18.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.576+0000 7f63b1f28640 1 --2- 172.21.15.43:0/3348898902 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a8940c5d20 0x55a8940a2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:18.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.576+0000 7f63b1727640 1 --2- 172.21.15.43:0/3348898902 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a8940c5d20 0x55a8940a2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:18.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.576+0000 7f63b1727640 1 -- 172.21.15.43:0/3348898902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8940ab9d0 msgr2=0x55a89407cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:18.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.576+0000 7f63b1727640 1 --2- 172.21.15.43:0/3348898902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8940ab9d0 0x55a89407cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:18.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.576+0000 7f63b1727640 1 -- 172.21.15.43:0/3348898902 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a89407d060 con 0x55a8940a8e40 2024-04-25T08:58:18.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.576+0000 7f63b0f26640 1 --2- 172.21.15.43:0/3348898902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8940ab9d0 0x55a89407cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:18.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.576+0000 7f63b1f28640 1 --2- 172.21.15.43:0/3348898902 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a8940c5d20 0x55a8940a2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:18.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.576+0000 7f63b1727640 1 --2- 172.21.15.43:0/3348898902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a8940a8e40 0x55a894132a30 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f63a80a9230 tx=0x7f639c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=547f6a372030362b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:18.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.576+0000 7f638bfff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:58:18.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.576+0000 7f638bfff640 1 -- 172.21.15.43:0/3348898902 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a8098060 con 0x55a8940a8e40 2024-04-25T08:58:18.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.576+0000 7f638bfff640 1 -- 172.21.15.43:0/3348898902 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(62..62 src has 1..62) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f63a803cc60 con 0x55a8940a8e40 2024-04-25T08:58:18.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.576+0000 7f63b6768740 1 -- 172.21.15.43:0/3348898902 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63a80a93a0 con 0x55a8940a8e40 2024-04-25T08:58:18.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.576+0000 7f638bfff640 1 -- 172.21.15.43:0/3348898902 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63a80a7050 con 0x55a8940a8e40 2024-04-25T08:58:18.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.576+0000 7f63b6768740 10 client.6951 fetch_fsmap learned FSMap version 39 2024-04-25T08:58:18.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.576+0000 7f63b6768740 1 -- 172.21.15.43:0/3348898902 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x55a89407e4e0 con 0x55a8940a8e40 2024-04-25T08:58:18.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.577+0000 7f638bfff640 1 -- 172.21.15.43:0/3348898902 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a80a93a0 con 0x55a8940a8e40 2024-04-25T08:58:18.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.577+0000 7f638bfff640 1 -- 172.21.15.43:0/3348898902 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 39) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f63a80817f0 con 0x55a8940a8e40 2024-04-25T08:58:18.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.577+0000 7f63b6768740 10 client.6951 fetch_fsmap finished waiting for FSMap version 39 2024-04-25T08:58:18.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.577+0000 7f63b6768740 10 client.6951 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T08:58:18.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.577+0000 7f63b6768740 20 client.6951 populate_metadata read hostname 'smithi043' 2024-04-25T08:58:18.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.577+0000 7f63b6768740 1 --2- 172.21.15.43:0/3348898902 >> [v2:172.21.15.43:6832/1866869111,v1:172.21.15.43:6833/1866869111] conn(0x55a8940c5430 0x55a8940cf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:18.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.577+0000 7f63b6768740 4 client.6951 mds_command: new command op to 6870 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_repair_path", "scrubops": ["repair"]}] 2024-04-25T08:58:18.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.577+0000 7f63b6768740 1 -- 172.21.15.43:0/3348898902 --> [v2:172.21.15.43:6832/1866869111,v1:172.21.15.43:6833/1866869111] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_repair_path", "scrubops": ["repair"]}) v1 -- 0x55a893fe87d0 con 0x55a8940c5430 2024-04-25T08:58:18.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.577+0000 7f63b0f26640 1 --2- 172.21.15.43:0/3348898902 >> [v2:172.21.15.43:6832/1866869111,v1:172.21.15.43:6833/1866869111] conn(0x55a8940c5430 0x55a8940cf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:18.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.577+0000 7f638bfff640 10 client.6951 ms_handle_connect on v2:172.21.15.43:6832/1866869111 2024-04-25T08:58:18.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.577+0000 7f63b0f26640 1 --2- 172.21.15.43:0/3348898902 >> [v2:172.21.15.43:6832/1866869111,v1:172.21.15.43:6833/1866869111] conn(0x55a8940c5430 0x55a8940cf490 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c82f982e38be06b server_cookie=a3a629ead57dd256 in_seq=0 out_seq=0 2024-04-25T08:58:18.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.578+0000 7f638bfff640 1 -- 172.21.15.43:0/3348898902 <== mds.0 v2:172.21.15.43:6832/1866869111 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a893fe87d0 con 0x55a8940c5430 2024-04-25T08:58:18.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.578+0000 7f638bfff640 10 client.6951 handle_command_reply: tid=0 2024-04-25T08:58:18.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.578+0000 7f63b6768740 1 client.6951 shutdown 2024-04-25T08:58:18.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.578+0000 7f63b6768740 1 -- 172.21.15.43:0/3348898902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a8940a8e40 msgr2=0x55a894132a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:18.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.578+0000 7f63b6768740 1 --2- 172.21.15.43:0/3348898902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a8940a8e40 0x55a894132a30 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f63a80a9230 tx=0x7f639c011000 comp rx=0 tx=0).stop 2024-04-25T08:58:18.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.578+0000 7f63b6768740 1 -- 172.21.15.43:0/3348898902 shutdown_connections 2024-04-25T08:58:18.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.578+0000 7f63b6768740 1 --2- 172.21.15.43:0/3348898902 >> [v2:172.21.15.43:6832/1866869111,v1:172.21.15.43:6833/1866869111] conn(0x55a8940c5430 0x55a8940cf490 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:18.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.578+0000 7f63b6768740 1 --2- 172.21.15.43:0/3348898902 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a8940c5d20 0x55a8940a2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:18.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.578+0000 7f638bfff640 0 client.6951 ms_handle_reset on v2:172.21.15.43:6832/1866869111 2024-04-25T08:58:18.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.578+0000 7f63b6768740 1 --2- 172.21.15.43:0/3348898902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a8940ab9d0 0x55a89407cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:18.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.578+0000 7f63b6768740 1 --2- 172.21.15.43:0/3348898902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a8940a8e40 0x55a894132a30 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:18.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.578+0000 7f63b6768740 1 -- 172.21.15.43:0/3348898902 >> 172.21.15.43:0/3348898902 conn(0x55a8940a8700 msgr2=0x55a8940c2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:18.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.578+0000 7f63b6768740 1 -- 172.21.15.43:0/3348898902 shutdown_connections 2024-04-25T08:58:18.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.578+0000 7f63b6768740 1 -- 172.21.15.43:0/3348898902 wait complete. 2024-04-25T08:58:18.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.578+0000 7f63b6768740 20 client.6951 trim_cache size 0 max 16384 2024-04-25T08:58:18.573 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T08:58:18.573 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-25T08:58:18.573 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "103e1c8e-a044-4bda-abbb-105a22daebc4", 2024-04-25T08:58:18.573 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-25T08:58:18.574 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T08:58:18.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.578+0000 7f63b398b640 1 -- 172.21.15.43:0/1437199193 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f639006b4d0 msgr2=0x7f639006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:18.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.578+0000 7f63b398b640 1 --2- 172.21.15.43:0/1437199193 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f639006b4d0 0x7f639006d990 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f639c006fd0 tx=0x7f639c008040 comp rx=0 tx=0).stop 2024-04-25T08:58:18.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.578+0000 7f63b398b640 1 -- 172.21.15.43:0/1437199193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63ac0aeea0 msgr2=0x7f63ac147f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:18.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.578+0000 7f63b398b640 1 --2- 172.21.15.43:0/1437199193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63ac0aeea0 0x7f63ac147f70 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f63a8000c00 tx=0x7f63a80046c0 comp rx=0 tx=0).stop 2024-04-25T08:58:18.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.578+0000 7f63b398b640 1 -- 172.21.15.43:0/1437199193 shutdown_connections 2024-04-25T08:58:18.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.578+0000 7f63b398b640 1 --2- 172.21.15.43:0/1437199193 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f639006b4d0 0x7f639006d990 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:18.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.578+0000 7f63b398b640 1 --2- 172.21.15.43:0/1437199193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63ac0cf2e0 0x7f63ac15da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:18.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.578+0000 7f63b398b640 1 --2- 172.21.15.43:0/1437199193 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f63ac0b4300 0x7f63ac15d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:18.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.578+0000 7f63b398b640 1 --2- 172.21.15.43:0/1437199193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63ac0aeea0 0x7f63ac147f70 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:18.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.578+0000 7f63b398b640 1 -- 172.21.15.43:0/1437199193 >> 172.21.15.43:0/1437199193 conn(0x7f63ac06ab30 msgr2=0x7f63ac0cf730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:18.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.579+0000 7f63b398b640 1 -- 172.21.15.43:0/1437199193 shutdown_connections 2024-04-25T08:58:18.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:18.579+0000 7f63b398b640 1 -- 172.21.15.43:0/1437199193 wait complete. 2024-04-25T08:58:18.578 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_repair_path repair' returned '{'return_code': 0, 'scrub_tag': '103e1c8e-a044-4bda-abbb-105a22daebc4', 'mode': 'asynchronous'}' 2024-04-25T08:58:28.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:28.580 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir scrub_repair_path) 2024-04-25T08:58:28.602 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:58:28.602 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:58:28.684 DEBUG:teuthology.orchestra.run.smithi193:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:58:28.730 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:58:28.730 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:58:28.730 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:58:28.748 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:58:28.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.906+0000 7fed9f9d5640 1 Processor -- start 2024-04-25T08:58:28.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.907+0000 7fed9f9d5640 1 -- start start 2024-04-25T08:58:28.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.907+0000 7fed9f9d5640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fed9815df50 0x7fed981619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:28.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.907+0000 7fed9f9d5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed980d24a0 con 0x7fed98161f30 2024-04-25T08:58:28.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.907+0000 7fed9f9d5640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed980d2740 con 0x7fed980cf670 2024-04-25T08:58:28.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.907+0000 7fed9f9d5640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed980cfa80 con 0x7fed9815df50 2024-04-25T08:58:28.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.908+0000 7fed9cf70640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fed98161f30 0x7fed980c7cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:52424/0 (socket says 172.21.15.43:52424) 2024-04-25T08:58:28.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.908+0000 7fed9cf70640 1 -- 172.21.15.43:0/2949005193 learned_addr learned my addr 172.21.15.43:0/2949005193 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:28.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.908+0000 7fed9d771640 1 --2- 172.21.15.43:0/2949005193 >> v2:172.21.15.193:3300/0 conn(0x7fed9815df50 0x7fed981619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:28.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.908+0000 7fed8ffff640 1 -- 172.21.15.43:0/2949005193 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1363336493 0 0) 0x7fed980d24a0 con 0x7fed98161f30 2024-04-25T08:58:28.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.908+0000 7fed8ffff640 1 -- 172.21.15.43:0/2949005193 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed84003520 con 0x7fed98161f30 2024-04-25T08:58:28.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.908+0000 7fed8ffff640 1 -- 172.21.15.43:0/2949005193 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1530541334 0 0) 0x7fed84003520 con 0x7fed98161f30 2024-04-25T08:58:28.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.908+0000 7fed8ffff640 1 -- 172.21.15.43:0/2949005193 >> v2:172.21.15.193:3300/0 conn(0x7fed9815df50 msgr2=0x7fed981619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:28.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.908+0000 7fed8ffff640 1 --2- 172.21.15.43:0/2949005193 >> v2:172.21.15.193:3300/0 conn(0x7fed9815df50 0x7fed981619f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:28.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.908+0000 7fed8ffff640 1 -- 172.21.15.43:0/2949005193 >> v1:172.21.15.193:6789/0 conn(0x7fed980cf670 legacy=0x7fed980c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:28.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.908+0000 7fed8ffff640 1 -- 172.21.15.43:0/2949005193 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed980d2a20 con 0x7fed98161f30 2024-04-25T08:58:28.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.908+0000 7fed8ffff640 1 -- 172.21.15.43:0/2949005193 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7fed88001690 con 0x7fed98161f30 2024-04-25T08:58:28.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.908+0000 7fed8ffff640 1 -- 172.21.15.43:0/2949005193 >> v1:172.21.15.43:6789/0 conn(0x7fed98161f30 legacy=0x7fed980c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:28.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.908+0000 7fed8ffff640 1 --2- 172.21.15.43:0/2949005193 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fed840039b0 0x7fed84003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:28.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.908+0000 7fed8ffff640 1 --2- 172.21.15.43:0/2949005193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed84004800 0x7fed84004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:28.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.908+0000 7fed8ffff640 1 --2- 172.21.15.43:0/2949005193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed840051c0 0x7fed8400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:28.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.908+0000 7fed8ffff640 1 -- 172.21.15.43:0/2949005193 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed980cfa80 con 0x7fed840051c0 2024-04-25T08:58:28.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.908+0000 7fed8ffff640 1 -- 172.21.15.43:0/2949005193 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed8400c060 con 0x7fed84004800 2024-04-25T08:58:28.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.908+0000 7fed8ffff640 1 -- 172.21.15.43:0/2949005193 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed8400c260 con 0x7fed840039b0 2024-04-25T08:58:28.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.908+0000 7fed9df72640 1 --2- 172.21.15.43:0/2949005193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed840051c0 0x7fed8400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:28.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.908+0000 7fed9cf70640 1 --2- 172.21.15.43:0/2949005193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed84004800 0x7fed84004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:28.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.909+0000 7fed9d771640 1 --2- 172.21.15.43:0/2949005193 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fed840039b0 0x7fed84003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:28.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.909+0000 7fed9df72640 1 -- 172.21.15.43:0/2949005193 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fed840039b0 msgr2=0x7fed84003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:28.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.909+0000 7fed9df72640 1 --2- 172.21.15.43:0/2949005193 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fed840039b0 0x7fed84003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:28.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.909+0000 7fed9df72640 1 -- 172.21.15.43:0/2949005193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed84004800 msgr2=0x7fed84004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:28.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.909+0000 7fed9df72640 1 --2- 172.21.15.43:0/2949005193 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed84004800 0x7fed84004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:28.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.909+0000 7fed9df72640 1 -- 172.21.15.43:0/2949005193 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed84003520 con 0x7fed840051c0 2024-04-25T08:58:28.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.909+0000 7fed9df72640 1 -- 172.21.15.43:0/2949005193 reap_dead start 2024-04-25T08:58:28.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.909+0000 7fed9df72640 1 --2- 172.21.15.43:0/2949005193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed840051c0 0x7fed8400b9c0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fed90001e50 tx=0x7fed84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e286599e3dfdd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:28.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.910+0000 7fed8ffff640 1 -- 172.21.15.43:0/2949005193 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed90001a70 con 0x7fed840051c0 2024-04-25T08:58:28.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.910+0000 7fed8ffff640 1 -- 172.21.15.43:0/2949005193 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed90003970 con 0x7fed840051c0 2024-04-25T08:58:28.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.910+0000 7fed8ffff640 1 -- 172.21.15.43:0/2949005193 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed90003d60 con 0x7fed840051c0 2024-04-25T08:58:28.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.910+0000 7fed9f9d5640 1 -- 172.21.15.43:0/2949005193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed840051c0 msgr2=0x7fed8400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:28.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.910+0000 7fed9f9d5640 1 --2- 172.21.15.43:0/2949005193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed840051c0 0x7fed8400b9c0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fed90001e50 tx=0x7fed84005830 comp rx=0 tx=0).stop 2024-04-25T08:58:28.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.910+0000 7fed9f9d5640 1 -- 172.21.15.43:0/2949005193 shutdown_connections 2024-04-25T08:58:28.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.910+0000 7fed9f9d5640 1 --2- 172.21.15.43:0/2949005193 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed840051c0 0x7fed8400b9c0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:28.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.910+0000 7fed9f9d5640 1 -- 172.21.15.43:0/2949005193 >> 172.21.15.43:0/2949005193 conn(0x7fed980d3a30 msgr2=0x7fed980d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:28.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.910+0000 7fed9f9d5640 1 -- 172.21.15.43:0/2949005193 shutdown_connections 2024-04-25T08:58:28.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.910+0000 7fed9f9d5640 1 -- 172.21.15.43:0/2949005193 wait complete. 2024-04-25T08:58:28.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.912+0000 7fed9f9d5640 1 Processor -- start 2024-04-25T08:58:28.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.912+0000 7fed9f9d5640 1 -- start start 2024-04-25T08:58:28.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.912+0000 7fed9f9d5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed840051c0 0x7fed981553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:28.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.912+0000 7fed9f9d5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fed9815e540 0x7fed98160b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:28.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.912+0000 7fed9f9d5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed98161070 0x7fed980cbac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:28.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.912+0000 7fed9f9d5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed980c77d0 con 0x7fed840051c0 2024-04-25T08:58:28.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.912+0000 7fed9f9d5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed9815e330 con 0x7fed98161070 2024-04-25T08:58:28.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.912+0000 7fed9f9d5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed980d2190 con 0x7fed9815e540 2024-04-25T08:58:28.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.912+0000 7fed9d771640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed840051c0 0x7fed981553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:28.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.912+0000 7fed9d771640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed840051c0 0x7fed981553a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55536/0 (socket says 172.21.15.43:55536) 2024-04-25T08:58:28.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.912+0000 7fed9d771640 1 -- 172.21.15.43:0/3698726820 learned_addr learned my addr 172.21.15.43:0/3698726820 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:28.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.913+0000 7fed9df72640 1 --2- 172.21.15.43:0/3698726820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed98161070 0x7fed980cbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:28.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.913+0000 7fed9cf70640 1 --2- 172.21.15.43:0/3698726820 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fed9815e540 0x7fed98160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:28.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.913+0000 7fed9d771640 1 -- 172.21.15.43:0/3698726820 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fed9815e540 msgr2=0x7fed98160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:28.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.913+0000 7fed9d771640 1 --2- 172.21.15.43:0/3698726820 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fed9815e540 0x7fed98160b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:28.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.913+0000 7fed9d771640 1 -- 172.21.15.43:0/3698726820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed98161070 msgr2=0x7fed980cbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:28.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.913+0000 7fed9d771640 1 --2- 172.21.15.43:0/3698726820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed98161070 0x7fed980cbac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:28.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.913+0000 7fed9d771640 1 -- 172.21.15.43:0/3698726820 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed98161860 con 0x7fed840051c0 2024-04-25T08:58:28.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.913+0000 7fed9cf70640 1 --2- 172.21.15.43:0/3698726820 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fed9815e540 0x7fed98160b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:58:28.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.913+0000 7fed9d771640 1 --2- 172.21.15.43:0/3698726820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed840051c0 0x7fed981553a0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fed940118e0 tx=0x7fed94011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:28.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.913+0000 7fed8dffb640 1 -- 172.21.15.43:0/3698726820 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9400f9d0 con 0x7fed840051c0 2024-04-25T08:58:28.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.913+0000 7fed8dffb640 1 -- 172.21.15.43:0/3698726820 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed9400fb70 con 0x7fed840051c0 2024-04-25T08:58:28.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.913+0000 7fed9f9d5640 1 -- 172.21.15.43:0/3698726820 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed98161580 con 0x7fed840051c0 2024-04-25T08:58:28.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.913+0000 7fed8dffb640 1 -- 172.21.15.43:0/3698726820 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed94014440 con 0x7fed840051c0 2024-04-25T08:58:28.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.913+0000 7fed9f9d5640 1 -- 172.21.15.43:0/3698726820 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed9815b820 con 0x7fed840051c0 2024-04-25T08:58:28.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.914+0000 7fed9f9d5640 1 -- 172.21.15.43:0/3698726820 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed980680f0 con 0x7fed840051c0 2024-04-25T08:58:28.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.915+0000 7fed8dffb640 1 -- 172.21.15.43:0/3698726820 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fed94012040 con 0x7fed840051c0 2024-04-25T08:58:28.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.915+0000 7fed8dffb640 1 --2- 172.21.15.43:0/3698726820 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fed7c06b4d0 0x7fed7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:28.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.915+0000 7fed8dffb640 1 -- 172.21.15.43:0/3698726820 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fed9408a7b0 con 0x7fed840051c0 2024-04-25T08:58:28.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.916+0000 7fed9cf70640 1 --2- 172.21.15.43:0/3698726820 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fed7c06b4d0 0x7fed7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:28.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.916+0000 7fed9cf70640 1 --2- 172.21.15.43:0/3698726820 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fed7c06b4d0 0x7fed7c06d990 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fed98159550 tx=0x7fed88009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:28.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:28.920+0000 7fed8dffb640 1 -- 172.21.15.43:0/3698726820 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fed94019030 con 0x7fed840051c0 2024-04-25T08:58:29.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.102+0000 7fed9f9d5640 1 -- 172.21.15.43:0/3698726820 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed98074ef0 con 0x7fed840051c0 2024-04-25T08:58:29.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.104+0000 7fed8dffb640 1 -- 172.21.15.43:0/3698726820 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+16888 (secure 0 0 0) 0x7fed94059570 con 0x7fed840051c0 2024-04-25T08:58:29.093 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:58:29.093 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":62,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:58:12.413565+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:57:52.884445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T08:57:53.565716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6834/3574570511":"2024-04-26T08:58:12.413536+0000","172.21.15.43:6832/3574570511":"2024-04-26T08:58:12.413536+0000","172.21.15.43:6835/2268368384":"2024-04-26T08:58:11.416075+0000","172.21.15.43:6833/2268368384":"2024-04-26T08:58:11.416075+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:58:29.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.106+0000 7fed9f9d5640 1 -- 172.21.15.43:0/3698726820 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fed7c06b4d0 msgr2=0x7fed7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:29.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.106+0000 7fed9f9d5640 1 --2- 172.21.15.43:0/3698726820 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fed7c06b4d0 0x7fed7c06d990 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fed98159550 tx=0x7fed88009040 comp rx=0 tx=0).stop 2024-04-25T08:58:29.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.106+0000 7fed9f9d5640 1 -- 172.21.15.43:0/3698726820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed840051c0 msgr2=0x7fed981553a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:29.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.106+0000 7fed9f9d5640 1 --2- 172.21.15.43:0/3698726820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed840051c0 0x7fed981553a0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fed940118e0 tx=0x7fed94011da0 comp rx=0 tx=0).stop 2024-04-25T08:58:29.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.106+0000 7fed9f9d5640 1 -- 172.21.15.43:0/3698726820 shutdown_connections 2024-04-25T08:58:29.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.106+0000 7fed9f9d5640 1 --2- 172.21.15.43:0/3698726820 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fed7c06b4d0 0x7fed7c06d990 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:29.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.106+0000 7fed9f9d5640 1 --2- 172.21.15.43:0/3698726820 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed98161070 0x7fed980cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:29.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.106+0000 7fed9f9d5640 1 --2- 172.21.15.43:0/3698726820 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fed9815e540 0x7fed98160b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:29.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.106+0000 7fed9f9d5640 1 --2- 172.21.15.43:0/3698726820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed840051c0 0x7fed981553a0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:29.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.106+0000 7fed9f9d5640 1 -- 172.21.15.43:0/3698726820 >> 172.21.15.43:0/3698726820 conn(0x7fed980d3a30 msgr2=0x7fed981560a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:29.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.106+0000 7fed9f9d5640 1 -- 172.21.15.43:0/3698726820 shutdown_connections 2024-04-25T08:58:29.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.106+0000 7fed9f9d5640 1 -- 172.21.15.43:0/3698726820 wait complete. 2024-04-25T08:58:29.106 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:57:52.884445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:57:53.565716+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:58:29.107 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:58:29.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.221+0000 7f0333cb3640 1 Processor -- start 2024-04-25T08:58:29.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.221+0000 7f0333cb3640 1 -- start start 2024-04-25T08:58:29.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.222+0000 7f0333cb3640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f032c15dec0 0x7f032c15ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:29.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.222+0000 7f0333cb3640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f032c0cf110 con 0x7f032c0ce8c0 2024-04-25T08:58:29.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.222+0000 7f0333cb3640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f032c0cf3f0 con 0x7f032c0d04a0 2024-04-25T08:58:29.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.222+0000 7f0333cb3640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f032c0cf6d0 con 0x7f032c15dec0 2024-04-25T08:58:29.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.222+0000 7f0332250640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f032c0ce8c0 0x7f032c0d17a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:52436/0 (socket says 172.21.15.43:52436) 2024-04-25T08:58:29.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.222+0000 7f0332250640 1 -- 172.21.15.43:0/2938855461 learned_addr learned my addr 172.21.15.43:0/2938855461 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:29.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.222+0000 7f033124e640 1 --2- 172.21.15.43:0/2938855461 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f032c15dec0 0x7f032c15ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:29.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.222+0000 7f0330a4d640 1 -- 172.21.15.43:0/2938855461 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 207131009 0 0) 0x7f032c0cf110 con 0x7f032c0ce8c0 2024-04-25T08:58:29.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.222+0000 7f0330a4d640 1 -- 172.21.15.43:0/2938855461 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0318003520 con 0x7f032c0ce8c0 2024-04-25T08:58:29.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.222+0000 7f0330a4d640 1 -- 172.21.15.43:0/2938855461 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2284069046 0 0) 0x7f0318003520 con 0x7f032c0ce8c0 2024-04-25T08:58:29.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.222+0000 7f0330a4d640 1 -- 172.21.15.43:0/2938855461 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f032c15dec0 msgr2=0x7f032c15ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:29.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.222+0000 7f0330a4d640 1 --2- 172.21.15.43:0/2938855461 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f032c15dec0 0x7f032c15ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:29.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.222+0000 7f0330a4d640 1 -- 172.21.15.43:0/2938855461 >> v1:172.21.15.193:6789/0 conn(0x7f032c0d04a0 legacy=0x7f032c0d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:29.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.222+0000 7f0330a4d640 1 -- 172.21.15.43:0/2938855461 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f032c0cf9b0 con 0x7f032c0ce8c0 2024-04-25T08:58:29.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.222+0000 7f0330a4d640 1 -- 172.21.15.43:0/2938855461 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f03240019f0 con 0x7f032c0ce8c0 2024-04-25T08:58:29.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.222+0000 7f0330a4d640 1 -- 172.21.15.43:0/2938855461 >> v1:172.21.15.43:6789/0 conn(0x7f032c0ce8c0 legacy=0x7f032c0d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:29.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.222+0000 7f0330a4d640 1 --2- 172.21.15.43:0/2938855461 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f032c15dec0 0x7f03180039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:29.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.222+0000 7f0330a4d640 1 --2- 172.21.15.43:0/2938855461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0318003f20 0x7f0318004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:29.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.223+0000 7f0330a4d640 1 --2- 172.21.15.43:0/2938855461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0318004d40 0x7f03180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:29.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.223+0000 7f0330a4d640 1 -- 172.21.15.43:0/2938855461 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032c0cf6d0 con 0x7f0318004d40 2024-04-25T08:58:29.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.223+0000 7f0330a4d640 1 -- 172.21.15.43:0/2938855461 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031800ba20 con 0x7f0318003f20 2024-04-25T08:58:29.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.223+0000 7f0330a4d640 1 -- 172.21.15.43:0/2938855461 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f031800bc20 con 0x7f032c15dec0 2024-04-25T08:58:29.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.223+0000 7f0332250640 1 --2- 172.21.15.43:0/2938855461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0318004d40 0x7f03180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:29.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.223+0000 7f0331a4f640 1 --2- 172.21.15.43:0/2938855461 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f032c15dec0 0x7f03180039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:29.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.223+0000 7f033124e640 1 --2- 172.21.15.43:0/2938855461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0318003f20 0x7f0318004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:29.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.223+0000 7f0332250640 1 -- 172.21.15.43:0/2938855461 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f032c15dec0 msgr2=0x7f03180039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:29.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.223+0000 7f0332250640 1 --2- 172.21.15.43:0/2938855461 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f032c15dec0 0x7f03180039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:29.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.223+0000 7f0332250640 1 -- 172.21.15.43:0/2938855461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0318003f20 msgr2=0x7f0318004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:29.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.223+0000 7f0332250640 1 --2- 172.21.15.43:0/2938855461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0318003f20 0x7f0318004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:29.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.223+0000 7f0332250640 1 -- 172.21.15.43:0/2938855461 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0318003520 con 0x7f0318004d40 2024-04-25T08:58:29.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.224+0000 7f0332250640 1 --2- 172.21.15.43:0/2938855461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0318004d40 0x7f03180051c0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f032400a920 tx=0x7f0318005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a971877f1329e2b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:29.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.224+0000 7f0330a4d640 1 -- 172.21.15.43:0/2938855461 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032400a4c0 con 0x7f0318004d40 2024-04-25T08:58:29.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.224+0000 7f0330a4d640 1 -- 172.21.15.43:0/2938855461 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f032402f070 con 0x7f0318004d40 2024-04-25T08:58:29.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.224+0000 7f0330a4d640 1 -- 172.21.15.43:0/2938855461 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032402bd20 con 0x7f0318004d40 2024-04-25T08:58:29.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.224+0000 7f0333cb3640 1 -- 172.21.15.43:0/2938855461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0318004d40 msgr2=0x7f03180051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:29.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.224+0000 7f0333cb3640 1 --2- 172.21.15.43:0/2938855461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0318004d40 0x7f03180051c0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f032400a920 tx=0x7f0318005830 comp rx=0 tx=0).stop 2024-04-25T08:58:29.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.224+0000 7f0332250640 1 -- 172.21.15.43:0/2938855461 reap_dead start 2024-04-25T08:58:29.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.224+0000 7f0333cb3640 1 -- 172.21.15.43:0/2938855461 shutdown_connections 2024-04-25T08:58:29.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.224+0000 7f0333cb3640 1 -- 172.21.15.43:0/2938855461 >> 172.21.15.43:0/2938855461 conn(0x7f032c09cd40 msgr2=0x7f032c0d3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:29.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.224+0000 7f0333cb3640 1 -- 172.21.15.43:0/2938855461 shutdown_connections 2024-04-25T08:58:29.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.224+0000 7f0333cb3640 1 -- 172.21.15.43:0/2938855461 wait complete. 2024-04-25T08:58:29.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.226+0000 7f0333cb3640 1 Processor -- start 2024-04-25T08:58:29.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.226+0000 7f0333cb3640 1 -- start start 2024-04-25T08:58:29.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.226+0000 7f0333cb3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f032c158b70 0x7f032c160d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:29.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.226+0000 7f0333cb3640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f032c161250 0x7f032c0cc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:29.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.226+0000 7f0333cb3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f032c0cc950 0x7f032c15f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:29.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.226+0000 7f0333cb3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032c0cee70 con 0x7f032c0cc950 2024-04-25T08:58:29.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.226+0000 7f0333cb3640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032c0cecf0 con 0x7f032c158b70 2024-04-25T08:58:29.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.226+0000 7f0333cb3640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f032c15ed90 con 0x7f032c161250 2024-04-25T08:58:29.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.227+0000 7f0332250640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f032c0cc950 0x7f032c15f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:29.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.227+0000 7f0332250640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f032c0cc950 0x7f032c15f1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55548/0 (socket says 172.21.15.43:55548) 2024-04-25T08:58:29.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.227+0000 7f0332250640 1 -- 172.21.15.43:0/2966246923 learned_addr learned my addr 172.21.15.43:0/2966246923 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:29.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.227+0000 7f0331a4f640 1 --2- 172.21.15.43:0/2966246923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f032c158b70 0x7f032c160d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:29.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.227+0000 7f033124e640 1 --2- 172.21.15.43:0/2966246923 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f032c161250 0x7f032c0cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:29.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.227+0000 7f0332250640 1 -- 172.21.15.43:0/2966246923 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f032c161250 msgr2=0x7f032c0cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:29.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.227+0000 7f0332250640 1 --2- 172.21.15.43:0/2966246923 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f032c161250 0x7f032c0cc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:29.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.227+0000 7f0332250640 1 -- 172.21.15.43:0/2966246923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f032c158b70 msgr2=0x7f032c160d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:29.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.227+0000 7f0332250640 1 --2- 172.21.15.43:0/2966246923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f032c158b70 0x7f032c160d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:29.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.227+0000 7f0332250640 1 -- 172.21.15.43:0/2966246923 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f032c161880 con 0x7f032c0cc950 2024-04-25T08:58:29.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.227+0000 7f0332250640 1 --2- 172.21.15.43:0/2966246923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f032c0cc950 0x7f032c15f1c0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f032400aa40 tx=0x7f032402d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:29.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.227+0000 7f03227fc640 1 -- 172.21.15.43:0/2966246923 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032400a4c0 con 0x7f032c0cc950 2024-04-25T08:58:29.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.227+0000 7f03227fc640 1 -- 172.21.15.43:0/2966246923 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0324034470 con 0x7f032c0cc950 2024-04-25T08:58:29.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.227+0000 7f0333cb3640 1 -- 172.21.15.43:0/2966246923 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f032c0c76e0 con 0x7f032c0cc950 2024-04-25T08:58:29.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.227+0000 7f03227fc640 1 -- 172.21.15.43:0/2966246923 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0324034860 con 0x7f032c0cc950 2024-04-25T08:58:29.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.227+0000 7f0333cb3640 1 -- 172.21.15.43:0/2966246923 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f032c0c7ca0 con 0x7f032c0cc950 2024-04-25T08:58:29.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.228+0000 7f0333cb3640 1 -- 172.21.15.43:0/2966246923 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f032c074f60 con 0x7f032c0cc950 2024-04-25T08:58:29.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.229+0000 7f03227fc640 1 -- 172.21.15.43:0/2966246923 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0324034a00 con 0x7f032c0cc950 2024-04-25T08:58:29.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.229+0000 7f03227fc640 1 --2- 172.21.15.43:0/2966246923 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f031006b4d0 0x7f031006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:29.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.229+0000 7f03227fc640 1 -- 172.21.15.43:0/2966246923 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f03240a9c90 con 0x7f032c0cc950 2024-04-25T08:58:29.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.229+0000 7f0331a4f640 1 --2- 172.21.15.43:0/2966246923 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f031006b4d0 0x7f031006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:29.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.230+0000 7f0331a4f640 1 --2- 172.21.15.43:0/2966246923 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f031006b4d0 0x7f031006d990 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0328001910 tx=0x7f0328009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:29.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.233+0000 7f03227fc640 1 -- 172.21.15.43:0/2966246923 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0324046060 con 0x7f032c0cc950 2024-04-25T08:58:29.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.412+0000 7f0333cb3640 1 -- 172.21.15.43:0/2966246923 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f032c08daf0 con 0x7f032c0cc950 2024-04-25T08:58:29.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.412+0000 7f03227fc640 1 -- 172.21.15.43:0/2966246923 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v62) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0324078b50 con 0x7f032c0cc950 2024-04-25T08:58:29.402 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T08:58:29.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.414+0000 7f0333cb3640 1 -- 172.21.15.43:0/2966246923 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f031006b4d0 msgr2=0x7f031006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:29.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.414+0000 7f0333cb3640 1 --2- 172.21.15.43:0/2966246923 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f031006b4d0 0x7f031006d990 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0328001910 tx=0x7f0328009040 comp rx=0 tx=0).stop 2024-04-25T08:58:29.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.414+0000 7f0333cb3640 1 -- 172.21.15.43:0/2966246923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f032c0cc950 msgr2=0x7f032c15f1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:29.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.414+0000 7f0333cb3640 1 --2- 172.21.15.43:0/2966246923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f032c0cc950 0x7f032c15f1c0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f032400aa40 tx=0x7f032402d070 comp rx=0 tx=0).stop 2024-04-25T08:58:29.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.414+0000 7f0333cb3640 1 -- 172.21.15.43:0/2966246923 shutdown_connections 2024-04-25T08:58:29.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.414+0000 7f0333cb3640 1 --2- 172.21.15.43:0/2966246923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f032c0cc950 0x7f032c15f1c0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:29.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.414+0000 7f0333cb3640 1 --2- 172.21.15.43:0/2966246923 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f032c161250 0x7f032c0cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:29.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.414+0000 7f0333cb3640 1 --2- 172.21.15.43:0/2966246923 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f031006b4d0 0x7f031006d990 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:29.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.414+0000 7f0333cb3640 1 --2- 172.21.15.43:0/2966246923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f032c158b70 0x7f032c160d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:29.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.414+0000 7f0333cb3640 1 -- 172.21.15.43:0/2966246923 >> 172.21.15.43:0/2966246923 conn(0x7f032c09cd40 msgr2=0x7f032c0b1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:29.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.414+0000 7f0333cb3640 1 -- 172.21.15.43:0/2966246923 shutdown_connections 2024-04-25T08:58:29.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.414+0000 7f0333cb3640 1 -- 172.21.15.43:0/2966246923 wait complete. 2024-04-25T08:58:29.414 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:58:29.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.529+0000 7f0ec89b7640 1 Processor -- start 2024-04-25T08:58:29.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.529+0000 7f0ec89b7640 1 -- start start 2024-04-25T08:58:29.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.530+0000 7f0ec89b7640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0ec40b5560 0x7f0ec40b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:29.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.530+0000 7f0ec89b7640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0ec40d2230 0x7f0ec40d26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:29.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.530+0000 7f0ec89b7640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ec4162610 con 0x7f0ec415e090 2024-04-25T08:58:29.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.530+0000 7f0ec89b7640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ec40d1da0 con 0x7f0ec40b5560 2024-04-25T08:58:29.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.530+0000 7f0ec89b7640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec40d1750 con 0x7f0ec40d2230 2024-04-25T08:58:29.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.531+0000 7f0ec1d9b640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0ec40b5560 0x7f0ec40b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:29.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.531+0000 7f0ec2d9d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0ec40d2230 0x7f0ec40d26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:29.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.531+0000 7f0ec259c640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f0ec415e090 0x7f0ec4161aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:46486/0 (socket says 172.21.15.43:46486) 2024-04-25T08:58:29.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.531+0000 7f0ec259c640 1 -- 172.21.15.43:0/381101643 learned_addr learned my addr 172.21.15.43:0/381101643 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:29.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.531+0000 7f0ec159a640 1 -- 172.21.15.43:0/381101643 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4137408703 0 0) 0x7f0ec4162610 con 0x7f0ec415e090 2024-04-25T08:58:29.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.531+0000 7f0ec159a640 1 -- 172.21.15.43:0/381101643 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0eb0003520 con 0x7f0ec415e090 2024-04-25T08:58:29.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.531+0000 7f0ec2d9d640 1 -- 172.21.15.43:0/381101643 >> v2:172.21.15.193:3300/0 conn(0x7f0ec40b5560 msgr2=0x7f0ec40b59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:29.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.531+0000 7f0ec2d9d640 1 --2- 172.21.15.43:0/381101643 >> v2:172.21.15.193:3300/0 conn(0x7f0ec40b5560 0x7f0ec40b59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:29.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.531+0000 7f0ec2d9d640 1 -- 172.21.15.43:0/381101643 >> v1:172.21.15.193:6789/0 conn(0x7f0ec415e090 legacy=0x7f0ec4161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:29.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.531+0000 7f0ec2d9d640 1 -- 172.21.15.43:0/381101643 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec40d1a30 con 0x7f0ec40d2230 2024-04-25T08:58:29.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.532+0000 7f0ec2d9d640 1 --2- 172.21.15.43:0/381101643 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0ec40d2230 0x7f0ec40d26c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4009d90 tx=0x7f0eb402a6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e66f374985745640 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:29.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.532+0000 7f0ec159a640 1 -- 172.21.15.43:0/381101643 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb4007740 con 0x7f0ec40d2230 2024-04-25T08:58:29.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.532+0000 7f0ec159a640 1 -- 172.21.15.43:0/381101643 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb40078e0 con 0x7f0ec40d2230 2024-04-25T08:58:29.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.532+0000 7f0ec159a640 1 -- 172.21.15.43:0/381101643 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb4007cb0 con 0x7f0ec40d2230 2024-04-25T08:58:29.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.533+0000 7f0ec89b7640 1 -- 172.21.15.43:0/381101643 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0ec40d2230 msgr2=0x7f0ec40d26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:29.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.533+0000 7f0ec89b7640 1 --2- 172.21.15.43:0/381101643 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0ec40d2230 0x7f0ec40d26c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f0eb4009d90 tx=0x7f0eb402a6f0 comp rx=0 tx=0).stop 2024-04-25T08:58:29.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.533+0000 7f0ec89b7640 1 -- 172.21.15.43:0/381101643 shutdown_connections 2024-04-25T08:58:29.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.533+0000 7f0ec89b7640 1 --2- 172.21.15.43:0/381101643 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0ec40d2230 0x7f0ec40d26c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:29.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.533+0000 7f0ec89b7640 1 --2- 172.21.15.43:0/381101643 >> v2:172.21.15.193:3300/0 conn(0x7f0ec40b5560 0x7f0ec40b59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:29.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.533+0000 7f0ec89b7640 1 -- 172.21.15.43:0/381101643 >> 172.21.15.43:0/381101643 conn(0x7f0ec40d0560 msgr2=0x7f0ec40d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:29.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.533+0000 7f0ec89b7640 1 -- 172.21.15.43:0/381101643 shutdown_connections 2024-04-25T08:58:29.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.533+0000 7f0ec89b7640 1 -- 172.21.15.43:0/381101643 wait complete. 2024-04-25T08:58:29.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.534+0000 7f0ec89b7640 1 Processor -- start 2024-04-25T08:58:29.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.534+0000 7f0ec89b7640 1 -- start start 2024-04-25T08:58:29.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.535+0000 7f0ec89b7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ec40b5560 0x7f0ec41f5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:29.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.535+0000 7f0ec89b7640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0ec40d2230 0x7f0ec41f6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:29.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.535+0000 7f0ec89b7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0ec415e090 0x7f0ec41605e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:29.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.535+0000 7f0ec89b7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec41621b0 con 0x7f0ec40b5560 2024-04-25T08:58:29.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.535+0000 7f0ec89b7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec40b3280 con 0x7f0ec415e090 2024-04-25T08:58:29.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.535+0000 7f0ec89b7640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec4162330 con 0x7f0ec40d2230 2024-04-25T08:58:29.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.535+0000 7f0ec259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ec40b5560 0x7f0ec41f5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:29.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.535+0000 7f0ec259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ec40b5560 0x7f0ec41f5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55562/0 (socket says 172.21.15.43:55562) 2024-04-25T08:58:29.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.535+0000 7f0ec259c640 1 -- 172.21.15.43:0/943939058 learned_addr learned my addr 172.21.15.43:0/943939058 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:29.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.535+0000 7f0ec2d9d640 1 --2- 172.21.15.43:0/943939058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0ec415e090 0x7f0ec41605e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:29.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.535+0000 7f0ec259c640 1 -- 172.21.15.43:0/943939058 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0ec40d2230 msgr2=0x7f0ec41f6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:29.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.535+0000 7f0ec1d9b640 1 --2- 172.21.15.43:0/943939058 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0ec40d2230 0x7f0ec41f6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:29.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.535+0000 7f0ec259c640 1 --2- 172.21.15.43:0/943939058 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0ec40d2230 0x7f0ec41f6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:29.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.535+0000 7f0ec259c640 1 -- 172.21.15.43:0/943939058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0ec415e090 msgr2=0x7f0ec41605e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:29.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.535+0000 7f0ec259c640 1 --2- 172.21.15.43:0/943939058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0ec415e090 0x7f0ec41605e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:29.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.535+0000 7f0ec259c640 1 -- 172.21.15.43:0/943939058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec4155db0 con 0x7f0ec40b5560 2024-04-25T08:58:29.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.535+0000 7f0ec259c640 1 --2- 172.21.15.43:0/943939058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ec40b5560 0x7f0ec41f5060 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc00c390 tx=0x7f0ebc00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:29.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.535+0000 7f0ebaffd640 1 -- 172.21.15.43:0/943939058 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc017070 con 0x7f0ec40b5560 2024-04-25T08:58:29.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.536+0000 7f0ebaffd640 1 -- 172.21.15.43:0/943939058 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ebc005ce0 con 0x7f0ec40b5560 2024-04-25T08:58:29.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.536+0000 7f0ec89b7640 1 -- 172.21.15.43:0/943939058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ec4156090 con 0x7f0ec40b5560 2024-04-25T08:58:29.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.536+0000 7f0ebaffd640 1 -- 172.21.15.43:0/943939058 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc0072f0 con 0x7f0ec40b5560 2024-04-25T08:58:29.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.536+0000 7f0ec89b7640 1 -- 172.21.15.43:0/943939058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ec4156650 con 0x7f0ec40b5560 2024-04-25T08:58:29.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.537+0000 7f0ec89b7640 1 -- 172.21.15.43:0/943939058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ec406bf00 con 0x7f0ec40b5560 2024-04-25T08:58:29.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.537+0000 7f0ebaffd640 1 -- 172.21.15.43:0/943939058 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0ebc01a070 con 0x7f0ec40b5560 2024-04-25T08:58:29.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.537+0000 7f0ebaffd640 1 --2- 172.21.15.43:0/943939058 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0ea406b4d0 0x7f0ea406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:29.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.537+0000 7f0ebaffd640 1 -- 172.21.15.43:0/943939058 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f0ebc089630 con 0x7f0ec40b5560 2024-04-25T08:58:29.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.538+0000 7f0ec1d9b640 1 --2- 172.21.15.43:0/943939058 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0ea406b4d0 0x7f0ea406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:29.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.538+0000 7f0ec1d9b640 1 --2- 172.21.15.43:0/943939058 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0ea406b4d0 0x7f0ea406d990 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0ec40742e0 tx=0x7f0eac008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:29.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.542+0000 7f0ebaffd640 1 -- 172.21.15.43:0/943939058 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0ec406bf00 con 0x7f0ec40b5560 2024-04-25T08:58:29.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.721+0000 7f0ec89b7640 1 -- 172.21.15.43:0/943939058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0ec4074f60 con 0x7f0ec40b5560 2024-04-25T08:58:29.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.721+0000 7f0ebaffd640 1 -- 172.21.15.43:0/943939058 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v62) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0ebc058490 con 0x7f0ec40b5560 2024-04-25T08:58:29.711 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T08:58:29.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.723+0000 7f0ec89b7640 1 -- 172.21.15.43:0/943939058 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0ea406b4d0 msgr2=0x7f0ea406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:29.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.723+0000 7f0ec89b7640 1 --2- 172.21.15.43:0/943939058 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0ea406b4d0 0x7f0ea406d990 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0ec40742e0 tx=0x7f0eac008040 comp rx=0 tx=0).stop 2024-04-25T08:58:29.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.723+0000 7f0ec89b7640 1 -- 172.21.15.43:0/943939058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ec40b5560 msgr2=0x7f0ec41f5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:29.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.723+0000 7f0ec89b7640 1 --2- 172.21.15.43:0/943939058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ec40b5560 0x7f0ec41f5060 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f0ebc00c390 tx=0x7f0ebc00c850 comp rx=0 tx=0).stop 2024-04-25T08:58:29.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.723+0000 7f0ec89b7640 1 -- 172.21.15.43:0/943939058 shutdown_connections 2024-04-25T08:58:29.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.723+0000 7f0ec89b7640 1 --2- 172.21.15.43:0/943939058 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0ea406b4d0 0x7f0ea406d990 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:29.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.723+0000 7f0ec89b7640 1 --2- 172.21.15.43:0/943939058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0ec415e090 0x7f0ec41605e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:29.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.723+0000 7f0ec89b7640 1 --2- 172.21.15.43:0/943939058 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0ec40d2230 0x7f0ec41f6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:29.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.723+0000 7f0ec89b7640 1 --2- 172.21.15.43:0/943939058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0ec40b5560 0x7f0ec41f5060 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:29.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.723+0000 7f0ec89b7640 1 -- 172.21.15.43:0/943939058 >> 172.21.15.43:0/943939058 conn(0x7f0ec40d0560 msgr2=0x7f0ec40b6e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:29.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.723+0000 7f0ec89b7640 1 -- 172.21.15.43:0/943939058 shutdown_connections 2024-04-25T08:58:29.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.723+0000 7f0ec89b7640 1 -- 172.21.15.43:0/943939058 wait complete. 2024-04-25T08:58:29.723 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:58:29.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.838+0000 7fce84a5d640 1 Processor -- start 2024-04-25T08:58:29.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.838+0000 7fce84a5d640 1 -- start start 2024-04-25T08:58:29.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.839+0000 7fce84a5d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fce8006f6a0 0x7fce800b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:29.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.839+0000 7fce84a5d640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fce800cd470 0x7fce8006c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:29.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.839+0000 7fce84a5d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce800d2620 con 0x7fce8015ddd0 2024-04-25T08:58:29.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.839+0000 7fce84a5d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce800b0b80 con 0x7fce8006f6a0 2024-04-25T08:58:29.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.839+0000 7fce84a5d640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce800b0e60 con 0x7fce800cd470 2024-04-25T08:58:29.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.839+0000 7fce7e59c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fce8015ddd0 0x7fce8006ef90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54888/0 (socket says 172.21.15.43:54888) 2024-04-25T08:58:29.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.839+0000 7fce77fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fce8006f6a0 0x7fce800b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:29.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.839+0000 7fce7e59c640 1 -- 172.21.15.43:0/3454073123 learned_addr learned my addr 172.21.15.43:0/3454073123 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:29.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.839+0000 7fce77fff640 1 -- 172.21.15.43:0/3454073123 >> v2:172.21.15.193:3300/0 conn(0x7fce800cd470 msgr2=0x7fce8006c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:29.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.839+0000 7fce7ed9d640 1 --2- 172.21.15.43:0/3454073123 >> v2:172.21.15.193:3300/0 conn(0x7fce800cd470 0x7fce8006c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:29.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.840+0000 7fce77fff640 1 --2- 172.21.15.43:0/3454073123 >> v2:172.21.15.193:3300/0 conn(0x7fce800cd470 0x7fce8006c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:29.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.840+0000 7fce77fff640 1 -- 172.21.15.43:0/3454073123 >> v1:172.21.15.43:6789/0 conn(0x7fce8015ddd0 legacy=0x7fce8006ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:29.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.840+0000 7fce7dd9b640 1 -- 172.21.15.43:0/3454073123 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1443004875 0 0) 0x7fce800d2620 con 0x7fce8015ddd0 2024-04-25T08:58:29.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.840+0000 7fce77fff640 1 -- 172.21.15.43:0/3454073123 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce800cbf40 con 0x7fce8006f6a0 2024-04-25T08:58:29.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.840+0000 7fce7ed9d640 1 --2- 172.21.15.43:0/3454073123 >> v2:172.21.15.193:3300/0 conn(0x7fce800cd470 0x7fce8006c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:29.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.840+0000 7fce77fff640 1 --2- 172.21.15.43:0/3454073123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce8006f6a0 0x7fce800b3dd0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fce8006ee30 tx=0x7fce6802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6461b2d902ff267 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:29.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.840+0000 7fce7dd9b640 1 -- 172.21.15.43:0/3454073123 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce68037070 con 0x7fce8006f6a0 2024-04-25T08:58:29.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.840+0000 7fce7dd9b640 1 -- 172.21.15.43:0/3454073123 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce68032440 con 0x7fce8006f6a0 2024-04-25T08:58:29.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.840+0000 7fce7dd9b640 1 -- 172.21.15.43:0/3454073123 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce68032810 con 0x7fce8006f6a0 2024-04-25T08:58:29.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.841+0000 7fce84a5d640 1 -- 172.21.15.43:0/3454073123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce8006f6a0 msgr2=0x7fce800b3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:29.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.841+0000 7fce84a5d640 1 --2- 172.21.15.43:0/3454073123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce8006f6a0 0x7fce800b3dd0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fce8006ee30 tx=0x7fce6802a1a0 comp rx=0 tx=0).stop 2024-04-25T08:58:29.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.841+0000 7fce84a5d640 1 -- 172.21.15.43:0/3454073123 shutdown_connections 2024-04-25T08:58:29.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.841+0000 7fce84a5d640 1 --2- 172.21.15.43:0/3454073123 >> v2:172.21.15.193:3300/0 conn(0x7fce800cd470 0x7fce8006c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:29.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.841+0000 7fce84a5d640 1 --2- 172.21.15.43:0/3454073123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce8006f6a0 0x7fce800b3dd0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:29.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.841+0000 7fce84a5d640 1 -- 172.21.15.43:0/3454073123 >> 172.21.15.43:0/3454073123 conn(0x7fce800d17e0 msgr2=0x7fce800d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:29.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.841+0000 7fce84a5d640 1 -- 172.21.15.43:0/3454073123 shutdown_connections 2024-04-25T08:58:29.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.841+0000 7fce84a5d640 1 -- 172.21.15.43:0/3454073123 wait complete. 2024-04-25T08:58:29.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.842+0000 7fce84a5d640 1 Processor -- start 2024-04-25T08:58:29.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.842+0000 7fce84a5d640 1 -- start start 2024-04-25T08:58:29.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.843+0000 7fce84a5d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce8006f6a0 0x7fce8014b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:29.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.843+0000 7fce84a5d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce800cd470 0x7fce8015e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:29.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.843+0000 7fce84a5d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fce8015ddd0 0x7fce8015c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:29.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.843+0000 7fce84a5d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8006cdb0 con 0x7fce8006f6a0 2024-04-25T08:58:29.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.843+0000 7fce84a5d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8006cc30 con 0x7fce800cd470 2024-04-25T08:58:29.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.843+0000 7fce84a5d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce8006cf30 con 0x7fce8015ddd0 2024-04-25T08:58:29.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.843+0000 7fce7e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce8006f6a0 0x7fce8014b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:29.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.843+0000 7fce7e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce8006f6a0 0x7fce8014b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36424/0 (socket says 172.21.15.43:36424) 2024-04-25T08:58:29.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.843+0000 7fce7e59c640 1 -- 172.21.15.43:0/4176376930 learned_addr learned my addr 172.21.15.43:0/4176376930 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:29.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.844+0000 7fce7e59c640 1 -- 172.21.15.43:0/4176376930 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fce8015ddd0 msgr2=0x7fce8015c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:29.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.844+0000 7fce7ed9d640 1 --2- 172.21.15.43:0/4176376930 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fce8015ddd0 0x7fce8015c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:29.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.844+0000 7fce77fff640 1 --2- 172.21.15.43:0/4176376930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce800cd470 0x7fce8015e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:29.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.844+0000 7fce7e59c640 1 --2- 172.21.15.43:0/4176376930 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fce8015ddd0 0x7fce8015c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:29.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.844+0000 7fce7e59c640 1 -- 172.21.15.43:0/4176376930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce800cd470 msgr2=0x7fce8015e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:29.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.844+0000 7fce7e59c640 1 --2- 172.21.15.43:0/4176376930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce800cd470 0x7fce8015e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:29.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.844+0000 7fce7e59c640 1 -- 172.21.15.43:0/4176376930 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce801f40c0 con 0x7fce8006f6a0 2024-04-25T08:58:29.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.844+0000 7fce7e59c640 1 --2- 172.21.15.43:0/4176376930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce8006f6a0 0x7fce8014b1f0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fce7000c430 tx=0x7fce7000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:29.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.844+0000 7fce76ffd640 1 -- 172.21.15.43:0/4176376930 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce700074e0 con 0x7fce8006f6a0 2024-04-25T08:58:29.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.844+0000 7fce76ffd640 1 -- 172.21.15.43:0/4176376930 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce700047e0 con 0x7fce8006f6a0 2024-04-25T08:58:29.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.844+0000 7fce84a5d640 1 -- 172.21.15.43:0/4176376930 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce801f4360 con 0x7fce8006f6a0 2024-04-25T08:58:29.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.844+0000 7fce76ffd640 1 -- 172.21.15.43:0/4176376930 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce70004bd0 con 0x7fce8006f6a0 2024-04-25T08:58:29.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.844+0000 7fce84a5d640 1 -- 172.21.15.43:0/4176376930 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce801f4920 con 0x7fce8006f6a0 2024-04-25T08:58:29.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.845+0000 7fce84a5d640 1 -- 172.21.15.43:0/4176376930 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce80074f60 con 0x7fce8006f6a0 2024-04-25T08:58:29.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.846+0000 7fce76ffd640 1 -- 172.21.15.43:0/4176376930 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fce70011070 con 0x7fce8006f6a0 2024-04-25T08:58:29.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.846+0000 7fce76ffd640 1 --2- 172.21.15.43:0/4176376930 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fce6006b4d0 0x7fce6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:29.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.846+0000 7fce76ffd640 1 -- 172.21.15.43:0/4176376930 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fce70089fc0 con 0x7fce8006f6a0 2024-04-25T08:58:29.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.846+0000 7fce77fff640 1 --2- 172.21.15.43:0/4176376930 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fce6006b4d0 0x7fce6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:29.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.847+0000 7fce77fff640 1 --2- 172.21.15.43:0/4176376930 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fce6006b4d0 0x7fce6006d990 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fce68034000 tx=0x7fce6802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:29.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:29.850+0000 7fce76ffd640 1 -- 172.21.15.43:0/4176376930 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fce70058cf0 con 0x7fce8006f6a0 2024-04-25T08:58:30.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.030+0000 7fce84a5d640 1 -- 172.21.15.43:0/4176376930 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fce801f4d40 con 0x7fce8006f6a0 2024-04-25T08:58:30.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.030+0000 7fce76ffd640 1 -- 172.21.15.43:0/4176376930 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v62) v1 ==== 100+0+11 (secure 0 0 0) 0x7fce700585f0 con 0x7fce8006f6a0 2024-04-25T08:58:30.020 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T08:58:30.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.032+0000 7fce84a5d640 1 -- 172.21.15.43:0/4176376930 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fce6006b4d0 msgr2=0x7fce6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:30.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.032+0000 7fce84a5d640 1 --2- 172.21.15.43:0/4176376930 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fce6006b4d0 0x7fce6006d990 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fce68034000 tx=0x7fce6802e000 comp rx=0 tx=0).stop 2024-04-25T08:58:30.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.032+0000 7fce84a5d640 1 -- 172.21.15.43:0/4176376930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce8006f6a0 msgr2=0x7fce8014b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:30.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.032+0000 7fce84a5d640 1 --2- 172.21.15.43:0/4176376930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce8006f6a0 0x7fce8014b1f0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fce7000c430 tx=0x7fce7000c8f0 comp rx=0 tx=0).stop 2024-04-25T08:58:30.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.032+0000 7fce84a5d640 1 -- 172.21.15.43:0/4176376930 shutdown_connections 2024-04-25T08:58:30.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.032+0000 7fce84a5d640 1 --2- 172.21.15.43:0/4176376930 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fce8015ddd0 0x7fce8015c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:30.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.032+0000 7fce84a5d640 1 --2- 172.21.15.43:0/4176376930 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fce6006b4d0 0x7fce6006d990 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:30.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.032+0000 7fce84a5d640 1 --2- 172.21.15.43:0/4176376930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fce800cd470 0x7fce8015e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:30.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.032+0000 7fce84a5d640 1 --2- 172.21.15.43:0/4176376930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fce8006f6a0 0x7fce8014b1f0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:30.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.032+0000 7fce84a5d640 1 -- 172.21.15.43:0/4176376930 >> 172.21.15.43:0/4176376930 conn(0x7fce800d17e0 msgr2=0x7fce800ce6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:30.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.032+0000 7fce84a5d640 1 -- 172.21.15.43:0/4176376930 shutdown_connections 2024-04-25T08:58:30.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.032+0000 7fce84a5d640 1 -- 172.21.15.43:0/4176376930 wait complete. 2024-04-25T08:58:30.033 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T08:58:30.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.145+0000 7f18fc683640 1 Processor -- start 2024-04-25T08:58:30.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.145+0000 7f18fc683640 1 -- start start 2024-04-25T08:58:30.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.146+0000 7f18fc683640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f18f415df50 0x7f18f41619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:30.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.146+0000 7f18fc683640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f18f4161f30 0x7f18f41623a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:30.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.146+0000 7f18fc683640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18f40d24a0 con 0x7f18f40c7cf0 2024-04-25T08:58:30.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.146+0000 7f18fc683640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18f415e330 con 0x7f18f415df50 2024-04-25T08:58:30.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.146+0000 7f18fc683640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f40d2740 con 0x7f18f4161f30 2024-04-25T08:58:30.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.146+0000 7f18fa41f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f18f415df50 0x7f18f41619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:30.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.146+0000 7f18fac20640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f18f40c7cf0 0x7f18f40c8100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54902/0 (socket says 172.21.15.43:54902) 2024-04-25T08:58:30.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.146+0000 7f18fa41f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f18f415df50 0x7f18f41619f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36430/0 (socket says 172.21.15.43:36430) 2024-04-25T08:58:30.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.146+0000 7f18fac20640 1 -- 172.21.15.43:0/1364476230 learned_addr learned my addr 172.21.15.43:0/1364476230 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:30.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.146+0000 7f18f9c1e640 1 --2- 172.21.15.43:0/1364476230 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f18f4161f30 0x7f18f41623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:30.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.146+0000 7f18fa41f640 1 -- 172.21.15.43:0/1364476230 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f18f4161f30 msgr2=0x7f18f41623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:30.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.146+0000 7f18fa41f640 1 --2- 172.21.15.43:0/1364476230 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f18f4161f30 0x7f18f41623a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:30.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.146+0000 7f18fa41f640 1 -- 172.21.15.43:0/1364476230 >> v1:172.21.15.43:6789/0 conn(0x7f18f40c7cf0 legacy=0x7f18f40c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:30.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.146+0000 7f18fa41f640 1 -- 172.21.15.43:0/1364476230 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f40d2a20 con 0x7f18f415df50 2024-04-25T08:58:30.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.146+0000 7f18f941d640 1 -- 172.21.15.43:0/1364476230 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2935567631 0 0) 0x7f18f40d24a0 con 0x7f18f40c7cf0 2024-04-25T08:58:30.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.147+0000 7f18fa41f640 1 --2- 172.21.15.43:0/1364476230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18f415df50 0x7f18f41619f0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f18f00097a0 tx=0x7f18f002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=938c1f12da6a1762 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:30.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.147+0000 7f18f941d640 1 -- 172.21.15.43:0/1364476230 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f0037070 con 0x7f18f415df50 2024-04-25T08:58:30.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.147+0000 7f18f941d640 1 -- 172.21.15.43:0/1364476230 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18f0032440 con 0x7f18f415df50 2024-04-25T08:58:30.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.147+0000 7f18f941d640 1 -- 172.21.15.43:0/1364476230 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18f0032810 con 0x7f18f415df50 2024-04-25T08:58:30.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.147+0000 7f18fc683640 1 -- 172.21.15.43:0/1364476230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18f415df50 msgr2=0x7f18f41619f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:30.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.147+0000 7f18fc683640 1 --2- 172.21.15.43:0/1364476230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18f415df50 0x7f18f41619f0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f18f00097a0 tx=0x7f18f002a180 comp rx=0 tx=0).stop 2024-04-25T08:58:30.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.148+0000 7f18fc683640 1 -- 172.21.15.43:0/1364476230 shutdown_connections 2024-04-25T08:58:30.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.148+0000 7f18fc683640 1 --2- 172.21.15.43:0/1364476230 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f18f4161f30 0x7f18f41623a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:30.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.148+0000 7f18fc683640 1 --2- 172.21.15.43:0/1364476230 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18f415df50 0x7f18f41619f0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:30.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.148+0000 7f18fc683640 1 -- 172.21.15.43:0/1364476230 >> 172.21.15.43:0/1364476230 conn(0x7f18f40d3a30 msgr2=0x7f18f40d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:30.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.148+0000 7f18fc683640 1 -- 172.21.15.43:0/1364476230 shutdown_connections 2024-04-25T08:58:30.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.148+0000 7f18fc683640 1 -- 172.21.15.43:0/1364476230 wait complete. 2024-04-25T08:58:30.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.149+0000 7f18fc683640 1 Processor -- start 2024-04-25T08:58:30.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.149+0000 7f18fc683640 1 -- start start 2024-04-25T08:58:30.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.150+0000 7f18fc683640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f18f40c7cf0 0x7f18f414ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:30.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.150+0000 7f18fc683640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18f415df50 0x7f18f406c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:30.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.150+0000 7f18fc683640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18f4161f30 0x7f18f406c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:30.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.150+0000 7f18fc683640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f40d2190 con 0x7f18f415df50 2024-04-25T08:58:30.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.150+0000 7f18fc683640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f40c77d0 con 0x7f18f4161f30 2024-04-25T08:58:30.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.150+0000 7f18fc683640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18f40cf670 con 0x7f18f40c7cf0 2024-04-25T08:58:30.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.150+0000 7f18f9c1e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18f415df50 0x7f18f406c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:30.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.150+0000 7f18f9c1e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18f415df50 0x7f18f406c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36434/0 (socket says 172.21.15.43:36434) 2024-04-25T08:58:30.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.150+0000 7f18f9c1e640 1 -- 172.21.15.43:0/134291674 learned_addr learned my addr 172.21.15.43:0/134291674 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:30.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.150+0000 7f18fa41f640 1 --2- 172.21.15.43:0/134291674 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f18f40c7cf0 0x7f18f414ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:30.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.150+0000 7f18fac20640 1 --2- 172.21.15.43:0/134291674 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18f4161f30 0x7f18f406c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:30.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.150+0000 7f18f9c1e640 1 -- 172.21.15.43:0/134291674 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f18f40c7cf0 msgr2=0x7f18f414ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:30.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.150+0000 7f18f9c1e640 1 --2- 172.21.15.43:0/134291674 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f18f40c7cf0 0x7f18f414ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:30.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.150+0000 7f18f9c1e640 1 -- 172.21.15.43:0/134291674 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18f4161f30 msgr2=0x7f18f406c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:30.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.150+0000 7f18f9c1e640 1 --2- 172.21.15.43:0/134291674 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18f4161f30 0x7f18f406c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:30.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.150+0000 7f18f9c1e640 1 -- 172.21.15.43:0/134291674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18f415eb60 con 0x7f18f415df50 2024-04-25T08:58:30.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.150+0000 7f18f9c1e640 1 --2- 172.21.15.43:0/134291674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18f415df50 0x7f18f406c150 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f18e400d620 tx=0x7f18e400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:30.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.150+0000 7f18eaffd640 1 -- 172.21.15.43:0/134291674 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e400b620 con 0x7f18f415df50 2024-04-25T08:58:30.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.150+0000 7f18eaffd640 1 -- 172.21.15.43:0/134291674 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18e4019070 con 0x7f18f415df50 2024-04-25T08:58:30.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.150+0000 7f18fc683640 1 -- 172.21.15.43:0/134291674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18f415ee40 con 0x7f18f415df50 2024-04-25T08:58:30.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.150+0000 7f18eaffd640 1 -- 172.21.15.43:0/134291674 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18e400ba00 con 0x7f18f415df50 2024-04-25T08:58:30.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.151+0000 7f18fc683640 1 -- 172.21.15.43:0/134291674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18f414d9d0 con 0x7f18f415df50 2024-04-25T08:58:30.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.152+0000 7f18fc683640 1 -- 172.21.15.43:0/134291674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18f40680f0 con 0x7f18f415df50 2024-04-25T08:58:30.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.152+0000 7f18eaffd640 1 -- 172.21.15.43:0/134291674 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f18e400bba0 con 0x7f18f415df50 2024-04-25T08:58:30.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.152+0000 7f18eaffd640 1 --2- 172.21.15.43:0/134291674 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f18d806b4d0 0x7f18d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:30.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.152+0000 7f18eaffd640 1 -- 172.21.15.43:0/134291674 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f18e408a4e0 con 0x7f18f415df50 2024-04-25T08:58:30.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.153+0000 7f18fa41f640 1 --2- 172.21.15.43:0/134291674 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f18d806b4d0 0x7f18d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:30.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.153+0000 7f18fa41f640 1 --2- 172.21.15.43:0/134291674 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f18d806b4d0 0x7f18d806d990 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f18f0034000 tx=0x7f18f002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:30.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.157+0000 7f18eaffd640 1 -- 172.21.15.43:0/134291674 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f18e4058cf0 con 0x7f18f415df50 2024-04-25T08:58:30.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.353+0000 7f18fc683640 1 -- 172.21.15.43:0/134291674 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f18f4074ef0 con 0x7f18f415df50 2024-04-25T08:58:30.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.353+0000 7f18eaffd640 1 -- 172.21.15.43:0/134291674 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v62) v1 ==== 80+0+244 (secure 0 0 0) 0x7f18e405cd90 con 0x7f18f415df50 2024-04-25T08:58:30.342 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6834/3574570511 2024-04-26T08:58:12.413536+0000 2024-04-25T08:58:30.343 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6832/3574570511 2024-04-26T08:58:12.413536+0000 2024-04-25T08:58:30.343 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6835/2268368384 2024-04-26T08:58:11.416075+0000 2024-04-25T08:58:30.343 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6833/2268368384 2024-04-26T08:58:11.416075+0000 2024-04-25T08:58:30.343 INFO:teuthology.orchestra.run.smithi043.stderr:listed 4 entries 2024-04-25T08:58:30.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.355+0000 7f18fc683640 1 -- 172.21.15.43:0/134291674 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f18d806b4d0 msgr2=0x7f18d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:30.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.355+0000 7f18fc683640 1 --2- 172.21.15.43:0/134291674 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f18d806b4d0 0x7f18d806d990 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f18f0034000 tx=0x7f18f002e000 comp rx=0 tx=0).stop 2024-04-25T08:58:30.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.355+0000 7f18fc683640 1 -- 172.21.15.43:0/134291674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18f415df50 msgr2=0x7f18f406c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:30.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.355+0000 7f18fc683640 1 --2- 172.21.15.43:0/134291674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18f415df50 0x7f18f406c150 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f18e400d620 tx=0x7f18e400dae0 comp rx=0 tx=0).stop 2024-04-25T08:58:30.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.355+0000 7f18fc683640 1 -- 172.21.15.43:0/134291674 shutdown_connections 2024-04-25T08:58:30.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.355+0000 7f18fc683640 1 --2- 172.21.15.43:0/134291674 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f18d806b4d0 0x7f18d806d990 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:30.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.355+0000 7f18fc683640 1 --2- 172.21.15.43:0/134291674 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f18f4161f30 0x7f18f406c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:30.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.355+0000 7f18fc683640 1 --2- 172.21.15.43:0/134291674 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f18f415df50 0x7f18f406c150 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:30.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.355+0000 7f18fc683640 1 --2- 172.21.15.43:0/134291674 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f18f40c7cf0 0x7f18f414ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:30.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.355+0000 7f18fc683640 1 -- 172.21.15.43:0/134291674 >> 172.21.15.43:0/134291674 conn(0x7f18f40d3a30 msgr2=0x7f18f4161720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:30.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.355+0000 7f18fc683640 1 -- 172.21.15.43:0/134291674 shutdown_connections 2024-04-25T08:58:30.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.355+0000 7f18fc683640 1 -- 172.21.15.43:0/134291674 wait complete. 2024-04-25T08:58:30.355 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi043.front.sepia.ceph.com... 2024-04-25T08:58:30.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:30.355 DEBUG:teuthology.orchestra.run.smithi043:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T08:58:30.378 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[64398]: fuse finished with error 0 and tester_r 0 2024-04-25T08:58:30.379 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-04-25T08:58:30.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:30.379 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T08:58:30.544 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T08:58:30.544 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:58:30.563 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19508 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:58:30.563 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi193.front.sepia.ceph.com 2024-04-25T08:58:30.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:30.563 DEBUG:teuthology.orchestra.run.smithi193:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T08:58:30.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:58:30.622 INFO:teuthology.orchestra.run.smithi193.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T08:58:30.622 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:58:30.622 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:58:30.638 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2374 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:58:30.638 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-04-25T08:58:30.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:30.638 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T08:58:30.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:58:30.694 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T08:58:30.694 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T08:58:30.694 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:58:30.712 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19508 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:58:30.712 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi193.front.sepia.ceph.com 2024-04-25T08:58:30.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:30.712 DEBUG:teuthology.orchestra.run.smithi193:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T08:58:30.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T08:58:30.769 INFO:teuthology.orchestra.run.smithi193.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T08:58:30.769 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:58:30.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.881+0000 7f7ca3ee1640 1 Processor -- start 2024-04-25T08:58:30.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.881+0000 7f7ca3ee1640 1 -- start start 2024-04-25T08:58:30.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.882+0000 7f7ca3ee1640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7c9c15dcf0 0x7f7c9c06e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:30.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.882+0000 7f7ca3ee1640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f7c9c06ebf0 0x7f7c9c06f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:30.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.882+0000 7f7ca3ee1640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7c9c1616e0 0x7f7c9c161b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:30.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.882+0000 7f7ca3ee1640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c9c06f590 con 0x7f7c9c1616e0 2024-04-25T08:58:30.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.882+0000 7f7ca3ee1640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c9c06c050 con 0x7f7c9c06ebf0 2024-04-25T08:58:30.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.882+0000 7f7ca3ee1640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9c0d0d80 con 0x7f7c9c15dcf0 2024-04-25T08:58:30.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.882+0000 7f7ca247e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7c9c1616e0 0x7f7c9c161b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:30.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.882+0000 7f7ca247e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7c9c1616e0 0x7f7c9c161b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36436/0 (socket says 172.21.15.43:36436) 2024-04-25T08:58:30.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.882+0000 7f7ca247e640 1 -- 172.21.15.43:0/1185136917 learned_addr learned my addr 172.21.15.43:0/1185136917 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:30.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.882+0000 7f7ca247e640 1 -- 172.21.15.43:0/1185136917 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7c9c15dcf0 msgr2=0x7f7c9c06e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:30.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.882+0000 7f7ca247e640 1 --2- 172.21.15.43:0/1185136917 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7c9c15dcf0 0x7f7c9c06e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:30.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.882+0000 7f7ca247e640 1 -- 172.21.15.43:0/1185136917 >> v2:172.21.15.193:3300/0 conn(0x7f7c9c06ebf0 msgr2=0x7f7c9c06f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:30.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.882+0000 7f7ca247e640 1 --2- 172.21.15.43:0/1185136917 >> v2:172.21.15.193:3300/0 conn(0x7f7c9c06ebf0 0x7f7c9c06f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:30.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.883+0000 7f7ca247e640 1 -- 172.21.15.43:0/1185136917 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c9c0d04e0 con 0x7f7c9c1616e0 2024-04-25T08:58:30.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.883+0000 7f7ca247e640 1 --2- 172.21.15.43:0/1185136917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c9c1616e0 0x7f7c9c161b70 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c0678d0 tx=0x7f7c8c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3235a497f6bc12bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:30.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.883+0000 7f7ca0c7b640 1 -- 172.21.15.43:0/1185136917 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c036070 con 0x7f7c9c1616e0 2024-04-25T08:58:30.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.883+0000 7f7ca0c7b640 1 -- 172.21.15.43:0/1185136917 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c8c031600 con 0x7f7c9c1616e0 2024-04-25T08:58:30.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.883+0000 7f7ca0c7b640 1 -- 172.21.15.43:0/1185136917 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c0319d0 con 0x7f7c9c1616e0 2024-04-25T08:58:30.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.884+0000 7f7ca3ee1640 1 -- 172.21.15.43:0/1185136917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c9c1616e0 msgr2=0x7f7c9c161b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:30.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.884+0000 7f7ca3ee1640 1 --2- 172.21.15.43:0/1185136917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c9c1616e0 0x7f7c9c161b70 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f7c9c0678d0 tx=0x7f7c8c02a610 comp rx=0 tx=0).stop 2024-04-25T08:58:30.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.884+0000 7f7ca3ee1640 1 -- 172.21.15.43:0/1185136917 shutdown_connections 2024-04-25T08:58:30.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.884+0000 7f7ca3ee1640 1 --2- 172.21.15.43:0/1185136917 >> v2:172.21.15.193:3300/0 conn(0x7f7c9c06ebf0 0x7f7c9c06f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:30.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.884+0000 7f7ca3ee1640 1 --2- 172.21.15.43:0/1185136917 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c9c1616e0 0x7f7c9c161b70 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:30.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.884+0000 7f7ca3ee1640 1 --2- 172.21.15.43:0/1185136917 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7c9c15dcf0 0x7f7c9c06e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:30.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.884+0000 7f7ca3ee1640 1 -- 172.21.15.43:0/1185136917 >> 172.21.15.43:0/1185136917 conn(0x7f7c9c0d1700 msgr2=0x7f7c9c0d1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:30.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.884+0000 7f7ca3ee1640 1 -- 172.21.15.43:0/1185136917 shutdown_connections 2024-04-25T08:58:30.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.884+0000 7f7ca3ee1640 1 -- 172.21.15.43:0/1185136917 wait complete. 2024-04-25T08:58:30.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.885+0000 7f7ca3ee1640 1 Processor -- start 2024-04-25T08:58:30.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.885+0000 7f7ca3ee1640 1 -- start start 2024-04-25T08:58:30.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.886+0000 7f7ca3ee1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c9c06ebf0 0x7f7c9c152ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:30.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.886+0000 7f7ca3ee1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c9c15dcf0 0x7f7c9c15ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:30.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.886+0000 7f7ca3ee1640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7c9c1616e0 0x7f7c9c15f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:30.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.886+0000 7f7ca3ee1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9c06bce0 con 0x7f7c9c15dcf0 2024-04-25T08:58:30.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.886+0000 7f7ca3ee1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9c0d0aa0 con 0x7f7c9c06ebf0 2024-04-25T08:58:30.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.886+0000 7f7ca3ee1640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9c0b0d50 con 0x7f7c9c1616e0 2024-04-25T08:58:30.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.886+0000 7f7ca147c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c9c15dcf0 0x7f7c9c15ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:30.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.886+0000 7f7ca147c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c9c15dcf0 0x7f7c9c15ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36450/0 (socket says 172.21.15.43:36450) 2024-04-25T08:58:30.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.886+0000 7f7ca147c640 1 -- 172.21.15.43:0/985609378 learned_addr learned my addr 172.21.15.43:0/985609378 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:30.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.886+0000 7f7ca247e640 1 --2- 172.21.15.43:0/985609378 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7c9c1616e0 0x7f7c9c15f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:30.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.886+0000 7f7ca1c7d640 1 --2- 172.21.15.43:0/985609378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c9c06ebf0 0x7f7c9c152ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:30.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.886+0000 7f7ca147c640 1 -- 172.21.15.43:0/985609378 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7c9c1616e0 msgr2=0x7f7c9c15f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:30.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.886+0000 7f7ca147c640 1 --2- 172.21.15.43:0/985609378 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7c9c1616e0 0x7f7c9c15f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:30.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.886+0000 7f7ca147c640 1 -- 172.21.15.43:0/985609378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c9c06ebf0 msgr2=0x7f7c9c152ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:30.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.886+0000 7f7ca147c640 1 --2- 172.21.15.43:0/985609378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c9c06ebf0 0x7f7c9c152ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:30.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.886+0000 7f7ca147c640 1 -- 172.21.15.43:0/985609378 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c9c1604b0 con 0x7f7c9c15dcf0 2024-04-25T08:58:30.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.886+0000 7f7ca247e640 1 --2- 172.21.15.43:0/985609378 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7c9c1616e0 0x7f7c9c15f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:30.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.886+0000 7f7ca1c7d640 1 --2- 172.21.15.43:0/985609378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c9c06ebf0 0x7f7c9c152ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:30.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.887+0000 7f7ca147c640 1 --2- 172.21.15.43:0/985609378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c9c15dcf0 0x7f7c9c15ee10 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000b4d0 tx=0x7f7c9000b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:30.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.887+0000 7f7c9a7fc640 1 -- 172.21.15.43:0/985609378 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c90014070 con 0x7f7c9c15dcf0 2024-04-25T08:58:30.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.887+0000 7f7c9a7fc640 1 -- 172.21.15.43:0/985609378 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c90010870 con 0x7f7c9c15dcf0 2024-04-25T08:58:30.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.887+0000 7f7c9a7fc640 1 -- 172.21.15.43:0/985609378 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c90010c60 con 0x7f7c9c15dcf0 2024-04-25T08:58:30.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.887+0000 7f7ca3ee1640 1 -- 172.21.15.43:0/985609378 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c9c160790 con 0x7f7c9c15dcf0 2024-04-25T08:58:30.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.887+0000 7f7ca3ee1640 1 -- 172.21.15.43:0/985609378 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c9c160110 con 0x7f7c9c15dcf0 2024-04-25T08:58:30.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.888+0000 7f7ca3ee1640 1 -- 172.21.15.43:0/985609378 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c9c074ef0 con 0x7f7c9c15dcf0 2024-04-25T08:58:30.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.888+0000 7f7c9a7fc640 1 -- 172.21.15.43:0/985609378 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7c90017070 con 0x7f7c9c15dcf0 2024-04-25T08:58:30.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.889+0000 7f7c9a7fc640 1 --2- 172.21.15.43:0/985609378 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7c8806b4d0 0x7f7c8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:30.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.889+0000 7f7c9a7fc640 1 -- 172.21.15.43:0/985609378 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f7c9008a0e0 con 0x7f7c9c15dcf0 2024-04-25T08:58:30.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.889+0000 7f7ca1c7d640 1 --2- 172.21.15.43:0/985609378 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7c8806b4d0 0x7f7c8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:30.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.890+0000 7f7ca1c7d640 1 --2- 172.21.15.43:0/985609378 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7c8806b4d0 0x7f7c8806d990 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7c94006fd0 tx=0x7f7c94008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:30.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:30.893+0000 7f7c9a7fc640 1 -- 172.21.15.43:0/985609378 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7c90058ea0 con 0x7f7c9c15dcf0 2024-04-25T08:58:31.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.109+0000 7f7ca3ee1640 1 -- 172.21.15.43:0/985609378 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c9c0b7430 con 0x7f7c9c15dcf0 2024-04-25T08:58:31.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.110+0000 7f7c9a7fc640 1 -- 172.21.15.43:0/985609378 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4936 (secure 0 0 0) 0x7f7c900587a0 con 0x7f7c9c15dcf0 2024-04-25T08:58:31.099 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:58:31.099 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":39,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5765,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/594587590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":594587590},{"type":"v1","addr":"172.21.15.193:6835","nonce":594587590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6187,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/681300720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":681300720},{"type":"v1","addr":"172.21.15.193:6837","nonce":681300720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6876,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/432022866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":432022866},{"type":"v1","addr":"172.21.15.43:6835","nonce":432022866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:57:54.590369+0000","modified":"2024-04-25T08:58:14.432383+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6870},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6870":{"gid":6870,"name":"c","rank":0,"incarnation":36,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6833/1866869111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1866869111},{"type":"v1","addr":"172.21.15.43:6833","nonce":1866869111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:58:31.100 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 39 2024-04-25T08:58:31.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.112+0000 7f7ca3ee1640 1 -- 172.21.15.43:0/985609378 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7c8806b4d0 msgr2=0x7f7c8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.112+0000 7f7ca3ee1640 1 --2- 172.21.15.43:0/985609378 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7c8806b4d0 0x7f7c8806d990 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7c94006fd0 tx=0x7f7c94008040 comp rx=0 tx=0).stop 2024-04-25T08:58:31.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.112+0000 7f7ca3ee1640 1 -- 172.21.15.43:0/985609378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c9c15dcf0 msgr2=0x7f7c9c15ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.112+0000 7f7ca3ee1640 1 --2- 172.21.15.43:0/985609378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c9c15dcf0 0x7f7c9c15ee10 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000b4d0 tx=0x7f7c9000b990 comp rx=0 tx=0).stop 2024-04-25T08:58:31.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.112+0000 7f7ca3ee1640 1 -- 172.21.15.43:0/985609378 shutdown_connections 2024-04-25T08:58:31.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.112+0000 7f7ca3ee1640 1 --2- 172.21.15.43:0/985609378 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7c9c1616e0 0x7f7c9c15f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.112+0000 7f7ca3ee1640 1 --2- 172.21.15.43:0/985609378 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c9c15dcf0 0x7f7c9c15ee10 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.112+0000 7f7ca3ee1640 1 --2- 172.21.15.43:0/985609378 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7c8806b4d0 0x7f7c8806d990 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.112+0000 7f7ca3ee1640 1 --2- 172.21.15.43:0/985609378 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c9c06ebf0 0x7f7c9c152ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.112+0000 7f7ca3ee1640 1 -- 172.21.15.43:0/985609378 >> 172.21.15.43:0/985609378 conn(0x7f7c9c0d1700 msgr2=0x7f7c9c0ae490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:31.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.112+0000 7f7ca3ee1640 1 -- 172.21.15.43:0/985609378 shutdown_connections 2024-04-25T08:58:31.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.112+0000 7f7ca3ee1640 1 -- 172.21.15.43:0/985609378 wait complete. 2024-04-25T08:58:31.112 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:58:31.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.224+0000 7fac8a3b7640 1 Processor -- start 2024-04-25T08:58:31.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.224+0000 7fac8a3b7640 1 -- start start 2024-04-25T08:58:31.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.224+0000 7fac8a3b7640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fac840d39d0 0x7fac840d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.224+0000 7fac8a3b7640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fac840d2500 0x7fac840d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.224+0000 7fac8a3b7640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac840b6c50 con 0x7fac840ce2a0 2024-04-25T08:58:31.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.224+0000 7fac8a3b7640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac8415e010 con 0x7fac840d2500 2024-04-25T08:58:31.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.224+0000 7fac8a3b7640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac840d2020 con 0x7fac840d39d0 2024-04-25T08:58:31.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.225+0000 7fac88954640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fac840d2500 0x7fac840d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.225+0000 7fac88954640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fac840d2500 0x7fac840d2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36464/0 (socket says 172.21.15.43:36464) 2024-04-25T08:58:31.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.225+0000 7fac88954640 1 -- 172.21.15.43:0/3091069892 learned_addr learned my addr 172.21.15.43:0/3091069892 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:31.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.225+0000 7fac83fff640 1 --2- 172.21.15.43:0/3091069892 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fac840d39d0 0x7fac840d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.225+0000 7fac88954640 1 -- 172.21.15.43:0/3091069892 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fac840d39d0 msgr2=0x7fac840d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:31.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.225+0000 7fac88954640 1 --2- 172.21.15.43:0/3091069892 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fac840d39d0 0x7fac840d0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.225+0000 7fac88954640 1 -- 172.21.15.43:0/3091069892 >> v1:172.21.15.193:6789/0 conn(0x7fac840ce2a0 legacy=0x7fac840d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:31.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.225+0000 7fac88954640 1 -- 172.21.15.43:0/3091069892 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac840cec40 con 0x7fac840d2500 2024-04-25T08:58:31.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.226+0000 7fac88954640 1 --2- 172.21.15.43:0/3091069892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac840d2500 0x7fac840d2990 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fac7c009e50 tx=0x7fac7c02a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2fb64af813441b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:31.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.226+0000 7fac82ffd640 1 -- 172.21.15.43:0/3091069892 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7c036070 con 0x7fac840d2500 2024-04-25T08:58:31.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.226+0000 7fac82ffd640 1 -- 172.21.15.43:0/3091069892 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac7c031600 con 0x7fac840d2500 2024-04-25T08:58:31.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.226+0000 7fac82ffd640 1 -- 172.21.15.43:0/3091069892 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7c0319d0 con 0x7fac840d2500 2024-04-25T08:58:31.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.226+0000 7fac8a3b7640 1 -- 172.21.15.43:0/3091069892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac840d2500 msgr2=0x7fac840d2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.226+0000 7fac8a3b7640 1 --2- 172.21.15.43:0/3091069892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac840d2500 0x7fac840d2990 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fac7c009e50 tx=0x7fac7c02a630 comp rx=0 tx=0).stop 2024-04-25T08:58:31.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.226+0000 7fac8a3b7640 1 -- 172.21.15.43:0/3091069892 shutdown_connections 2024-04-25T08:58:31.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.226+0000 7fac8a3b7640 1 --2- 172.21.15.43:0/3091069892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac840d2500 0x7fac840d2990 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.226+0000 7fac8a3b7640 1 --2- 172.21.15.43:0/3091069892 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fac840d39d0 0x7fac840d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.226+0000 7fac8a3b7640 1 -- 172.21.15.43:0/3091069892 >> 172.21.15.43:0/3091069892 conn(0x7fac8406b000 msgr2=0x7fac8406b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:31.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.226+0000 7fac8a3b7640 1 -- 172.21.15.43:0/3091069892 shutdown_connections 2024-04-25T08:58:31.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.226+0000 7fac8a3b7640 1 -- 172.21.15.43:0/3091069892 wait complete. 2024-04-25T08:58:31.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.228+0000 7fac8a3b7640 1 Processor -- start 2024-04-25T08:58:31.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.228+0000 7fac8a3b7640 1 -- start start 2024-04-25T08:58:31.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.229+0000 7fac8a3b7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fac840ce2a0 0x7fac840c8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.229+0000 7fac8a3b7640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fac840d2500 0x7fac840b5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.229+0000 7fac8a3b7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac840d39d0 0x7fac84153f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.229+0000 7fac8a3b7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac840cf200 con 0x7fac840d39d0 2024-04-25T08:58:31.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.229+0000 7fac8a3b7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac840b2f00 con 0x7fac840ce2a0 2024-04-25T08:58:31.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.229+0000 7fac8a3b7640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac840d2ed0 con 0x7fac840d2500 2024-04-25T08:58:31.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.229+0000 7fac88954640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac840d39d0 0x7fac84153f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.229+0000 7fac88954640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac840d39d0 0x7fac84153f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36472/0 (socket says 172.21.15.43:36472) 2024-04-25T08:58:31.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.229+0000 7fac88954640 1 -- 172.21.15.43:0/345349552 learned_addr learned my addr 172.21.15.43:0/345349552 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:31.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.229+0000 7fac837fe640 1 --2- 172.21.15.43:0/345349552 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fac840d2500 0x7fac840b5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.229+0000 7fac83fff640 1 --2- 172.21.15.43:0/345349552 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fac840ce2a0 0x7fac840c8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.229+0000 7fac88954640 1 -- 172.21.15.43:0/345349552 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fac840d2500 msgr2=0x7fac840b5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.229+0000 7fac88954640 1 --2- 172.21.15.43:0/345349552 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fac840d2500 0x7fac840b5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.229+0000 7fac88954640 1 -- 172.21.15.43:0/345349552 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fac840ce2a0 msgr2=0x7fac840c8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.229+0000 7fac88954640 1 --2- 172.21.15.43:0/345349552 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fac840ce2a0 0x7fac840c8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.229+0000 7fac88954640 1 -- 172.21.15.43:0/345349552 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac840b80e0 con 0x7fac840d39d0 2024-04-25T08:58:31.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.229+0000 7fac837fe640 1 --2- 172.21.15.43:0/345349552 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fac840d2500 0x7fac840b5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:31.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.229+0000 7fac83fff640 1 --2- 172.21.15.43:0/345349552 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fac840ce2a0 0x7fac840c8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:31.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.229+0000 7fac88954640 1 --2- 172.21.15.43:0/345349552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac840d39d0 0x7fac84153f60 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fac7c009e50 tx=0x7fac7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:31.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.229+0000 7fac80ff9640 1 -- 172.21.15.43:0/345349552 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7c036050 con 0x7fac840d39d0 2024-04-25T08:58:31.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.229+0000 7fac80ff9640 1 -- 172.21.15.43:0/345349552 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac7c042070 con 0x7fac840d39d0 2024-04-25T08:58:31.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.229+0000 7fac8a3b7640 1 -- 172.21.15.43:0/345349552 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac840b83c0 con 0x7fac840d39d0 2024-04-25T08:58:31.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.229+0000 7fac80ff9640 1 -- 172.21.15.43:0/345349552 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac7c009510 con 0x7fac840d39d0 2024-04-25T08:58:31.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.230+0000 7fac8a3b7640 1 -- 172.21.15.43:0/345349552 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac840b8980 con 0x7fac840d39d0 2024-04-25T08:58:31.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.230+0000 7fac8a3b7640 1 -- 172.21.15.43:0/345349552 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac84074ef0 con 0x7fac840d39d0 2024-04-25T08:58:31.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.231+0000 7fac80ff9640 1 -- 172.21.15.43:0/345349552 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fac7c007bd0 con 0x7fac840d39d0 2024-04-25T08:58:31.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.231+0000 7fac80ff9640 1 --2- 172.21.15.43:0/345349552 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fac7006b500 0x7fac7006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.231+0000 7fac80ff9640 1 -- 172.21.15.43:0/345349552 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fac7c0ab140 con 0x7fac840d39d0 2024-04-25T08:58:31.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.231+0000 7fac83fff640 1 --2- 172.21.15.43:0/345349552 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fac7006b500 0x7fac7006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.232+0000 7fac83fff640 1 --2- 172.21.15.43:0/345349552 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fac7006b500 0x7fac7006d9c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fac7400b820 tx=0x7fac74008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:31.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.235+0000 7fac80ff9640 1 -- 172.21.15.43:0/345349552 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fac840b6c50 con 0x7fac840d39d0 2024-04-25T08:58:31.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.417+0000 7fac8a3b7640 1 -- 172.21.15.43:0/345349552 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fac840b6850 con 0x7fac840d39d0 2024-04-25T08:58:31.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.418+0000 7fac80ff9640 1 -- 172.21.15.43:0/345349552 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+16888 (secure 0 0 0) 0x7fac7c079fa0 con 0x7fac840d39d0 2024-04-25T08:58:31.408 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:58:31.408 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":62,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:58:12.413565+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:57:52.884445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T08:57:53.565716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6834/3574570511":"2024-04-26T08:58:12.413536+0000","172.21.15.43:6832/3574570511":"2024-04-26T08:58:12.413536+0000","172.21.15.43:6835/2268368384":"2024-04-26T08:58:11.416075+0000","172.21.15.43:6833/2268368384":"2024-04-26T08:58:11.416075+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:58:31.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.421+0000 7fac8a3b7640 1 -- 172.21.15.43:0/345349552 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fac7006b500 msgr2=0x7fac7006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.421+0000 7fac8a3b7640 1 --2- 172.21.15.43:0/345349552 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fac7006b500 0x7fac7006d9c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fac7400b820 tx=0x7fac74008040 comp rx=0 tx=0).stop 2024-04-25T08:58:31.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.421+0000 7fac8a3b7640 1 -- 172.21.15.43:0/345349552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac840d39d0 msgr2=0x7fac84153f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.421+0000 7fac8a3b7640 1 --2- 172.21.15.43:0/345349552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac840d39d0 0x7fac84153f60 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fac7c009e50 tx=0x7fac7c004a40 comp rx=0 tx=0).stop 2024-04-25T08:58:31.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.421+0000 7fac8a3b7640 1 -- 172.21.15.43:0/345349552 shutdown_connections 2024-04-25T08:58:31.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.421+0000 7fac8a3b7640 1 --2- 172.21.15.43:0/345349552 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fac840d39d0 0x7fac84153f60 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.421+0000 7fac8a3b7640 1 --2- 172.21.15.43:0/345349552 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fac840d2500 0x7fac840b5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.421+0000 7fac8a3b7640 1 --2- 172.21.15.43:0/345349552 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fac7006b500 0x7fac7006d9c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.421+0000 7fac8a3b7640 1 --2- 172.21.15.43:0/345349552 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fac840ce2a0 0x7fac840c8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.421+0000 7fac8a3b7640 1 -- 172.21.15.43:0/345349552 >> 172.21.15.43:0/345349552 conn(0x7fac8406b000 msgr2=0x7fac8406b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:31.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.421+0000 7fac8a3b7640 1 -- 172.21.15.43:0/345349552 shutdown_connections 2024-04-25T08:58:31.412 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.421+0000 7fac8a3b7640 1 -- 172.21.15.43:0/345349552 wait complete. 2024-04-25T08:58:31.421 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:57:52.884445+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:57:53.565716+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '58', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:58:31.421 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:58:31.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.537+0000 7fae40b5c640 1 Processor -- start 2024-04-25T08:58:31.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.537+0000 7fae40b5c640 1 -- start start 2024-04-25T08:58:31.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.538+0000 7fae40b5c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fae3c0c8870 0x7fae3c0d2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.538+0000 7fae40b5c640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae3c0cf030 con 0x7fae3c0d19f0 2024-04-25T08:58:31.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.538+0000 7fae40b5c640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae3c0cf310 con 0x7fae3c161310 2024-04-25T08:58:31.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.538+0000 7fae40b5c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae3c0d4870 con 0x7fae3c0c8870 2024-04-25T08:58:31.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.538+0000 7fae3ad9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fae3c0c8870 0x7fae3c0d2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.538+0000 7fae39d9b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fae3c0d19f0 0x7fae3c0d1e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54914/0 (socket says 172.21.15.43:54914) 2024-04-25T08:58:31.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.538+0000 7fae39d9b640 1 -- 172.21.15.43:0/974933777 learned_addr learned my addr 172.21.15.43:0/974933777 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:31.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.538+0000 7fae3ad9d640 1 -- 172.21.15.43:0/974933777 >> v1:172.21.15.193:6789/0 conn(0x7fae3c161310 legacy=0x7fae3c0d4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:31.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.538+0000 7fae3ad9d640 1 -- 172.21.15.43:0/974933777 >> v1:172.21.15.43:6789/0 conn(0x7fae3c0d19f0 legacy=0x7fae3c0d1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.538+0000 7fae3ad9d640 1 -- 172.21.15.43:0/974933777 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae3c0cf5f0 con 0x7fae3c0c8870 2024-04-25T08:58:31.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.539+0000 7fae3ad9d640 1 --2- 172.21.15.43:0/974933777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae3c0c8870 0x7fae3c0d2540 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fae34009d90 tx=0x7fae3402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8369d1c422f040ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:31.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.539+0000 7fae3959a640 1 -- 172.21.15.43:0/974933777 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae34007740 con 0x7fae3c0c8870 2024-04-25T08:58:31.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.539+0000 7fae3959a640 1 -- 172.21.15.43:0/974933777 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae340078e0 con 0x7fae3c0c8870 2024-04-25T08:58:31.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.539+0000 7fae3959a640 1 -- 172.21.15.43:0/974933777 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae34007cb0 con 0x7fae3c0c8870 2024-04-25T08:58:31.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.539+0000 7fae40b5c640 1 -- 172.21.15.43:0/974933777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae3c0c8870 msgr2=0x7fae3c0d2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.539+0000 7fae40b5c640 1 --2- 172.21.15.43:0/974933777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae3c0c8870 0x7fae3c0d2540 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fae34009d90 tx=0x7fae3402a6a0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.539+0000 7fae40b5c640 1 -- 172.21.15.43:0/974933777 shutdown_connections 2024-04-25T08:58:31.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.539+0000 7fae40b5c640 1 --2- 172.21.15.43:0/974933777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae3c0c8870 0x7fae3c0d2540 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.539+0000 7fae40b5c640 1 -- 172.21.15.43:0/974933777 >> 172.21.15.43:0/974933777 conn(0x7fae3c09cd20 msgr2=0x7fae3c0d3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:31.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.539+0000 7fae40b5c640 1 -- 172.21.15.43:0/974933777 shutdown_connections 2024-04-25T08:58:31.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.539+0000 7fae40b5c640 1 -- 172.21.15.43:0/974933777 wait complete. 2024-04-25T08:58:31.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.541+0000 7fae40b5c640 1 Processor -- start 2024-04-25T08:58:31.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.541+0000 7fae40b5c640 1 -- start start 2024-04-25T08:58:31.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.541+0000 7fae40b5c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae3c0c8870 0x7fae3c157cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.541+0000 7fae40b5c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae3c0d19f0 0x7fae3c0b6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.541+0000 7fae40b5c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fae3c161310 0x7fae3c153050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.541+0000 7fae40b5c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c168bb0 con 0x7fae3c0c8870 2024-04-25T08:58:31.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.541+0000 7fae40b5c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c168a30 con 0x7fae3c0d19f0 2024-04-25T08:58:31.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.541+0000 7fae40b5c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c168d30 con 0x7fae3c161310 2024-04-25T08:58:31.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.542+0000 7fae3a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae3c0c8870 0x7fae3c157cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.542+0000 7fae3a59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae3c0c8870 0x7fae3c157cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36492/0 (socket says 172.21.15.43:36492) 2024-04-25T08:58:31.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.542+0000 7fae3a59c640 1 -- 172.21.15.43:0/1907948549 learned_addr learned my addr 172.21.15.43:0/1907948549 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:31.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.542+0000 7fae39d9b640 1 --2- 172.21.15.43:0/1907948549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae3c0d19f0 0x7fae3c0b6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.542+0000 7fae3ad9d640 1 --2- 172.21.15.43:0/1907948549 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fae3c161310 0x7fae3c153050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.542+0000 7fae3a59c640 1 -- 172.21.15.43:0/1907948549 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fae3c161310 msgr2=0x7fae3c153050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.542+0000 7fae3a59c640 1 --2- 172.21.15.43:0/1907948549 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fae3c161310 0x7fae3c153050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.542+0000 7fae3a59c640 1 -- 172.21.15.43:0/1907948549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae3c0d19f0 msgr2=0x7fae3c0b6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.542+0000 7fae3a59c640 1 --2- 172.21.15.43:0/1907948549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae3c0d19f0 0x7fae3c0b6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.542+0000 7fae3a59c640 1 -- 172.21.15.43:0/1907948549 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae3c0cb740 con 0x7fae3c0c8870 2024-04-25T08:58:31.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.542+0000 7fae3a59c640 1 --2- 172.21.15.43:0/1907948549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae3c0c8870 0x7fae3c157cb0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00c460 tx=0x7fae2c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:31.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.542+0000 7fae2affd640 1 -- 172.21.15.43:0/1907948549 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c016020 con 0x7fae3c0c8870 2024-04-25T08:58:31.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.542+0000 7fae2affd640 1 -- 172.21.15.43:0/1907948549 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae2c012870 con 0x7fae3c0c8870 2024-04-25T08:58:31.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.542+0000 7fae40b5c640 1 -- 172.21.15.43:0/1907948549 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae3c0cba20 con 0x7fae3c0c8870 2024-04-25T08:58:31.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.542+0000 7fae2affd640 1 -- 172.21.15.43:0/1907948549 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae2c012c60 con 0x7fae3c0c8870 2024-04-25T08:58:31.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.542+0000 7fae40b5c640 1 -- 172.21.15.43:0/1907948549 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae3c0cbfe0 con 0x7fae3c0c8870 2024-04-25T08:58:31.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.543+0000 7fae40b5c640 1 -- 172.21.15.43:0/1907948549 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae3c074f60 con 0x7fae3c0c8870 2024-04-25T08:58:31.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.544+0000 7fae2affd640 1 -- 172.21.15.43:0/1907948549 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae2c019020 con 0x7fae3c0c8870 2024-04-25T08:58:31.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.544+0000 7fae2affd640 1 --2- 172.21.15.43:0/1907948549 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fae1c06b4d0 0x7fae1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.544+0000 7fae2affd640 1 -- 172.21.15.43:0/1907948549 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fae2c089ea0 con 0x7fae3c0c8870 2024-04-25T08:58:31.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.545+0000 7fae39d9b640 1 --2- 172.21.15.43:0/1907948549 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fae1c06b4d0 0x7fae1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.545+0000 7fae39d9b640 1 --2- 172.21.15.43:0/1907948549 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fae1c06b4d0 0x7fae1c06d990 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fae3c159960 tx=0x7fae30007e80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:31.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.548+0000 7fae2affd640 1 -- 172.21.15.43:0/1907948549 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fae2c020030 con 0x7fae3c0c8870 2024-04-25T08:58:31.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.728+0000 7fae40b5c640 1 -- 172.21.15.43:0/1907948549 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fae3c0cda70 con 0x7fae3c0c8870 2024-04-25T08:58:31.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.728+0000 7fae2affd640 1 -- 172.21.15.43:0/1907948549 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v62) v1 ==== 93+0+10 (secure 0 0 0) 0x7fae2c058620 con 0x7fae3c0c8870 2024-04-25T08:58:31.718 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T08:58:31.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.730+0000 7fae40b5c640 1 -- 172.21.15.43:0/1907948549 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fae1c06b4d0 msgr2=0x7fae1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.730+0000 7fae40b5c640 1 --2- 172.21.15.43:0/1907948549 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fae1c06b4d0 0x7fae1c06d990 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fae3c159960 tx=0x7fae30007e80 comp rx=0 tx=0).stop 2024-04-25T08:58:31.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.730+0000 7fae40b5c640 1 -- 172.21.15.43:0/1907948549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae3c0c8870 msgr2=0x7fae3c157cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.730+0000 7fae40b5c640 1 --2- 172.21.15.43:0/1907948549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae3c0c8870 0x7fae3c157cb0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00c460 tx=0x7fae2c00c920 comp rx=0 tx=0).stop 2024-04-25T08:58:31.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.730+0000 7fae40b5c640 1 -- 172.21.15.43:0/1907948549 shutdown_connections 2024-04-25T08:58:31.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.730+0000 7fae40b5c640 1 --2- 172.21.15.43:0/1907948549 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fae3c161310 0x7fae3c153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.730+0000 7fae40b5c640 1 --2- 172.21.15.43:0/1907948549 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fae1c06b4d0 0x7fae1c06d990 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.730+0000 7fae40b5c640 1 --2- 172.21.15.43:0/1907948549 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae3c0d19f0 0x7fae3c0b6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.730+0000 7fae40b5c640 1 --2- 172.21.15.43:0/1907948549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae3c0c8870 0x7fae3c157cb0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.730+0000 7fae40b5c640 1 -- 172.21.15.43:0/1907948549 >> 172.21.15.43:0/1907948549 conn(0x7fae3c09cd20 msgr2=0x7fae3c0d3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:31.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.730+0000 7fae40b5c640 1 -- 172.21.15.43:0/1907948549 shutdown_connections 2024-04-25T08:58:31.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.730+0000 7fae40b5c640 1 -- 172.21.15.43:0/1907948549 wait complete. 2024-04-25T08:58:31.731 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T08:58:31.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.846+0000 7f9b81b22640 1 Processor -- start 2024-04-25T08:58:31.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.846+0000 7f9b81b22640 1 -- start start 2024-04-25T08:58:31.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.846+0000 7f9b81b22640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b7c06f6a0 0x7f9b7c0b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.846+0000 7f9b81b22640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9b7c0cd470 0x7f9b7c06c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.846+0000 7f9b81b22640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b7c0d2620 con 0x7f9b7c15ddd0 2024-04-25T08:58:31.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.846+0000 7f9b81b22640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b7c0b0b80 con 0x7f9b7c0cd470 2024-04-25T08:58:31.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.846+0000 7f9b81b22640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7c0b0e60 con 0x7f9b7c06f6a0 2024-04-25T08:58:31.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.847+0000 7f9b7b7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f9b7c15ddd0 0x7f9b7c06ef90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54918/0 (socket says 172.21.15.43:54918) 2024-04-25T08:58:31.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.847+0000 7f9b7b7fe640 1 -- 172.21.15.43:0/1022908235 learned_addr learned my addr 172.21.15.43:0/1022908235 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:31.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.847+0000 7f9b7affd640 1 --2- 172.21.15.43:0/1022908235 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b7c06f6a0 0x7f9b7c0b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.847+0000 7f9b7bfff640 1 --2- 172.21.15.43:0/1022908235 >> v2:172.21.15.193:3300/0 conn(0x7f9b7c0cd470 0x7f9b7c06c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.847+0000 7f9b7a7fc640 1 -- 172.21.15.43:0/1022908235 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 432685001 0 0) 0x7f9b7c0d2620 con 0x7f9b7c15ddd0 2024-04-25T08:58:31.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.847+0000 7f9b7a7fc640 1 -- 172.21.15.43:0/1022908235 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b64003520 con 0x7f9b7c15ddd0 2024-04-25T08:58:31.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.847+0000 7f9b7affd640 1 -- 172.21.15.43:0/1022908235 >> v2:172.21.15.193:3300/0 conn(0x7f9b7c0cd470 msgr2=0x7f9b7c06c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:31.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.847+0000 7f9b7affd640 1 --2- 172.21.15.43:0/1022908235 >> v2:172.21.15.193:3300/0 conn(0x7f9b7c0cd470 0x7f9b7c06c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.847+0000 7f9b7affd640 1 -- 172.21.15.43:0/1022908235 >> v1:172.21.15.43:6789/0 conn(0x7f9b7c15ddd0 legacy=0x7f9b7c06ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.847+0000 7f9b7a7fc640 1 -- 172.21.15.43:0/1022908235 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1928327518 0 0) 0x7f9b64003520 con 0x7f9b7c15ddd0 2024-04-25T08:58:31.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.847+0000 7f9b7affd640 1 -- 172.21.15.43:0/1022908235 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b7c0cbf40 con 0x7f9b7c06f6a0 2024-04-25T08:58:31.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.848+0000 7f9b7affd640 1 --2- 172.21.15.43:0/1022908235 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b7c06f6a0 0x7f9b7c0b3dd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9b700097a0 tx=0x7f9b7002a080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a61990fa9896cf7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:31.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.848+0000 7f9b7a7fc640 1 -- 172.21.15.43:0/1022908235 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b70037070 con 0x7f9b7c06f6a0 2024-04-25T08:58:31.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.848+0000 7f9b7a7fc640 1 -- 172.21.15.43:0/1022908235 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b7002ab10 con 0x7f9b7c06f6a0 2024-04-25T08:58:31.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.848+0000 7f9b7a7fc640 1 -- 172.21.15.43:0/1022908235 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6c001670 con 0x7f9b7c06f6a0 2024-04-25T08:58:31.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.848+0000 7f9b81b22640 1 -- 172.21.15.43:0/1022908235 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b7c06f6a0 msgr2=0x7f9b7c0b3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.848+0000 7f9b81b22640 1 --2- 172.21.15.43:0/1022908235 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b7c06f6a0 0x7f9b7c0b3dd0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9b700097a0 tx=0x7f9b7002a080 comp rx=0 tx=0).stop 2024-04-25T08:58:31.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.849+0000 7f9b81b22640 1 -- 172.21.15.43:0/1022908235 shutdown_connections 2024-04-25T08:58:31.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.849+0000 7f9b81b22640 1 --2- 172.21.15.43:0/1022908235 >> v2:172.21.15.193:3300/0 conn(0x7f9b7c0cd470 0x7f9b7c06c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.849+0000 7f9b81b22640 1 --2- 172.21.15.43:0/1022908235 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b7c06f6a0 0x7f9b7c0b3dd0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.849+0000 7f9b81b22640 1 -- 172.21.15.43:0/1022908235 >> 172.21.15.43:0/1022908235 conn(0x7f9b7c0d17e0 msgr2=0x7f9b7c0d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:31.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.849+0000 7f9b81b22640 1 -- 172.21.15.43:0/1022908235 shutdown_connections 2024-04-25T08:58:31.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.849+0000 7f9b81b22640 1 -- 172.21.15.43:0/1022908235 wait complete. 2024-04-25T08:58:31.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.850+0000 7f9b81b22640 1 Processor -- start 2024-04-25T08:58:31.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.850+0000 7f9b81b22640 1 -- start start 2024-04-25T08:58:31.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.851+0000 7f9b81b22640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b7c06f6a0 0x7f9b7c0c8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.851+0000 7f9b81b22640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b7c0cd470 0x7f9b7c14e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.851+0000 7f9b81b22640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b7c15ddd0 0x7f9b7c0ca2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.851+0000 7f9b81b22640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7c06ce30 con 0x7f9b7c15ddd0 2024-04-25T08:58:31.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.851+0000 7f9b81b22640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7c06ccb0 con 0x7f9b7c0cd470 2024-04-25T08:58:31.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.851+0000 7f9b81b22640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b7c06cfb0 con 0x7f9b7c06f6a0 2024-04-25T08:58:31.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.851+0000 7f9b7bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b7c15ddd0 0x7f9b7c0ca2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.851+0000 7f9b7bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b7c15ddd0 0x7f9b7c0ca2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36498/0 (socket says 172.21.15.43:36498) 2024-04-25T08:58:31.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.851+0000 7f9b7bfff640 1 -- 172.21.15.43:0/623853795 learned_addr learned my addr 172.21.15.43:0/623853795 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:31.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.851+0000 7f9b7b7fe640 1 --2- 172.21.15.43:0/623853795 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b7c06f6a0 0x7f9b7c0c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.851+0000 7f9b7affd640 1 --2- 172.21.15.43:0/623853795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b7c0cd470 0x7f9b7c14e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.851+0000 7f9b7bfff640 1 -- 172.21.15.43:0/623853795 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b7c06f6a0 msgr2=0x7f9b7c0c8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.851+0000 7f9b7bfff640 1 --2- 172.21.15.43:0/623853795 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b7c06f6a0 0x7f9b7c0c8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.851+0000 7f9b7bfff640 1 -- 172.21.15.43:0/623853795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b7c0cd470 msgr2=0x7f9b7c14e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:31.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.851+0000 7f9b7bfff640 1 --2- 172.21.15.43:0/623853795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b7c0cd470 0x7f9b7c14e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:31.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.851+0000 7f9b7bfff640 1 -- 172.21.15.43:0/623853795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b7c0b45d0 con 0x7f9b7c15ddd0 2024-04-25T08:58:31.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.851+0000 7f9b7affd640 1 --2- 172.21.15.43:0/623853795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b7c0cd470 0x7f9b7c14e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:58:31.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.851+0000 7f9b7bfff640 1 --2- 172.21.15.43:0/623853795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b7c15ddd0 0x7f9b7c0ca2a0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f9b7400efc0 tx=0x7f9b7400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:31.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.851+0000 7f9b63fff640 1 -- 172.21.15.43:0/623853795 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b74018020 con 0x7f9b7c15ddd0 2024-04-25T08:58:31.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.851+0000 7f9b63fff640 1 -- 172.21.15.43:0/623853795 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b74014480 con 0x7f9b7c15ddd0 2024-04-25T08:58:31.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.851+0000 7f9b81b22640 1 -- 172.21.15.43:0/623853795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b7c0b2600 con 0x7f9b7c15ddd0 2024-04-25T08:58:31.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.851+0000 7f9b63fff640 1 -- 172.21.15.43:0/623853795 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b74014870 con 0x7f9b7c15ddd0 2024-04-25T08:58:31.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.852+0000 7f9b81b22640 1 -- 172.21.15.43:0/623853795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b7c0b4890 con 0x7f9b7c15ddd0 2024-04-25T08:58:31.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.852+0000 7f9b81b22640 1 -- 172.21.15.43:0/623853795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b7c074f60 con 0x7f9b7c15ddd0 2024-04-25T08:58:31.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.853+0000 7f9b63fff640 1 -- 172.21.15.43:0/623853795 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9b74005d10 con 0x7f9b7c15ddd0 2024-04-25T08:58:31.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.853+0000 7f9b63fff640 1 --2- 172.21.15.43:0/623853795 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9b6806b4d0 0x7f9b6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:31.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.854+0000 7f9b63fff640 1 -- 172.21.15.43:0/623853795 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f9b7408acc0 con 0x7f9b7c15ddd0 2024-04-25T08:58:31.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.854+0000 7f9b7b7fe640 1 --2- 172.21.15.43:0/623853795 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9b6806b4d0 0x7f9b6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:31.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.854+0000 7f9b7b7fe640 1 --2- 172.21.15.43:0/623853795 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9b6806b4d0 0x7f9b6806d990 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c005880 tx=0x7f9b6c0057f0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:31.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:31.858+0000 7f9b63fff640 1 -- 172.21.15.43:0/623853795 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9b74022070 con 0x7f9b7c15ddd0 2024-04-25T08:58:32.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.039+0000 7f9b81b22640 1 -- 172.21.15.43:0/623853795 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9b7c0b4d30 con 0x7f9b7c15ddd0 2024-04-25T08:58:32.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.039+0000 7f9b63fff640 1 -- 172.21.15.43:0/623853795 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v62) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9b74010040 con 0x7f9b7c15ddd0 2024-04-25T08:58:32.029 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T08:58:32.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.041+0000 7f9b81b22640 1 -- 172.21.15.43:0/623853795 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9b6806b4d0 msgr2=0x7f9b6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:32.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.041+0000 7f9b81b22640 1 --2- 172.21.15.43:0/623853795 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9b6806b4d0 0x7f9b6806d990 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9b6c005880 tx=0x7f9b6c0057f0 comp rx=0 tx=0).stop 2024-04-25T08:58:32.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.041+0000 7f9b81b22640 1 -- 172.21.15.43:0/623853795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b7c15ddd0 msgr2=0x7f9b7c0ca2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:32.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.041+0000 7f9b81b22640 1 --2- 172.21.15.43:0/623853795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b7c15ddd0 0x7f9b7c0ca2a0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f9b7400efc0 tx=0x7f9b7400c490 comp rx=0 tx=0).stop 2024-04-25T08:58:32.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.041+0000 7f9b81b22640 1 -- 172.21.15.43:0/623853795 shutdown_connections 2024-04-25T08:58:32.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.041+0000 7f9b81b22640 1 --2- 172.21.15.43:0/623853795 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9b7c15ddd0 0x7f9b7c0ca2a0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:32.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.041+0000 7f9b81b22640 1 --2- 172.21.15.43:0/623853795 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9b6806b4d0 0x7f9b6806d990 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:32.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.041+0000 7f9b81b22640 1 --2- 172.21.15.43:0/623853795 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9b7c0cd470 0x7f9b7c14e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:32.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.041+0000 7f9b81b22640 1 --2- 172.21.15.43:0/623853795 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9b7c06f6a0 0x7f9b7c0c8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:32.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.041+0000 7f9b81b22640 1 -- 172.21.15.43:0/623853795 >> 172.21.15.43:0/623853795 conn(0x7f9b7c0d17e0 msgr2=0x7f9b7c15c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:32.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.042+0000 7f9b81b22640 1 -- 172.21.15.43:0/623853795 shutdown_connections 2024-04-25T08:58:32.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.042+0000 7f9b81b22640 1 -- 172.21.15.43:0/623853795 wait complete. 2024-04-25T08:58:32.042 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T08:58:32.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.156+0000 7f4ac60d8640 1 Processor -- start 2024-04-25T08:58:32.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.156+0000 7f4ac60d8640 1 -- start start 2024-04-25T08:58:32.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.157+0000 7f4ac60d8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4ac00cd470 0x7f4ac00b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:32.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.157+0000 7f4ac60d8640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ac00d2620 con 0x7f4ac015ddd0 2024-04-25T08:58:32.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.157+0000 7f4ac60d8640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ac00b0b80 con 0x7f4ac006f6a0 2024-04-25T08:58:32.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.157+0000 7f4ac60d8640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ac00b0e60 con 0x7f4ac00cd470 2024-04-25T08:58:32.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.157+0000 7f4abffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4ac00cd470 0x7f4ac00b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:32.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.157+0000 7f4abf7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4ac015ddd0 0x7f4ac006ef90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54926/0 (socket says 172.21.15.43:54926) 2024-04-25T08:58:32.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.157+0000 7f4abf7fe640 1 -- 172.21.15.43:0/369140149 learned_addr learned my addr 172.21.15.43:0/369140149 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:32.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.157+0000 7f4abffff640 1 -- 172.21.15.43:0/369140149 >> v1:172.21.15.193:6789/0 conn(0x7f4ac006f6a0 legacy=0x7f4ac006c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:32.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.157+0000 7f4abffff640 1 -- 172.21.15.43:0/369140149 >> v1:172.21.15.43:6789/0 conn(0x7f4ac015ddd0 legacy=0x7f4ac006ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:32.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.157+0000 7f4abffff640 1 -- 172.21.15.43:0/369140149 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac00cbf40 con 0x7f4ac00cd470 2024-04-25T08:58:32.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.158+0000 7f4abffff640 1 --2- 172.21.15.43:0/369140149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ac00cd470 0x7f4ac00b3dd0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8009d90 tx=0x7f4ab802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c0312116b67a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:32.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.158+0000 7f4abe7fc640 1 -- 172.21.15.43:0/369140149 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab8007740 con 0x7f4ac00cd470 2024-04-25T08:58:32.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.158+0000 7f4abe7fc640 1 -- 172.21.15.43:0/369140149 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ab80078e0 con 0x7f4ac00cd470 2024-04-25T08:58:32.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.158+0000 7f4abe7fc640 1 -- 172.21.15.43:0/369140149 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab8007cb0 con 0x7f4ac00cd470 2024-04-25T08:58:32.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.158+0000 7f4ac60d8640 1 -- 172.21.15.43:0/369140149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ac00cd470 msgr2=0x7f4ac00b3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:32.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.158+0000 7f4ac60d8640 1 --2- 172.21.15.43:0/369140149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ac00cd470 0x7f4ac00b3dd0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f4ab8009d90 tx=0x7f4ab802a6f0 comp rx=0 tx=0).stop 2024-04-25T08:58:32.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.159+0000 7f4ac60d8640 1 -- 172.21.15.43:0/369140149 shutdown_connections 2024-04-25T08:58:32.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.159+0000 7f4ac60d8640 1 --2- 172.21.15.43:0/369140149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ac00cd470 0x7f4ac00b3dd0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:32.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.159+0000 7f4ac60d8640 1 -- 172.21.15.43:0/369140149 >> 172.21.15.43:0/369140149 conn(0x7f4ac00d17e0 msgr2=0x7f4ac00d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:32.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.159+0000 7f4ac60d8640 1 -- 172.21.15.43:0/369140149 shutdown_connections 2024-04-25T08:58:32.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.159+0000 7f4ac60d8640 1 -- 172.21.15.43:0/369140149 wait complete. 2024-04-25T08:58:32.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.160+0000 7f4ac60d8640 1 Processor -- start 2024-04-25T08:58:32.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.160+0000 7f4ac60d8640 1 -- start start 2024-04-25T08:58:32.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.161+0000 7f4ac60d8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ac006f6a0 0x7f4ac01f6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:32.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.161+0000 7f4ac60d8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ac00cd470 0x7f4ac00cb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:32.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.161+0000 7f4ac60d8640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4ac015ddd0 0x7f4ac0158300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:32.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.161+0000 7f4ac60d8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac006d000 con 0x7f4ac00cd470 2024-04-25T08:58:32.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.161+0000 7f4ac60d8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac006ce80 con 0x7f4ac006f6a0 2024-04-25T08:58:32.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.161+0000 7f4ac60d8640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac006d180 con 0x7f4ac015ddd0 2024-04-25T08:58:32.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.161+0000 7f4abeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ac00cd470 0x7f4ac00cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:32.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.161+0000 7f4abeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ac00cd470 0x7f4ac00cb080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36516/0 (socket says 172.21.15.43:36516) 2024-04-25T08:58:32.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.161+0000 7f4abeffd640 1 -- 172.21.15.43:0/3622022461 learned_addr learned my addr 172.21.15.43:0/3622022461 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:32.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.161+0000 7f4abf7fe640 1 --2- 172.21.15.43:0/3622022461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ac006f6a0 0x7f4ac01f6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:32.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.161+0000 7f4abffff640 1 --2- 172.21.15.43:0/3622022461 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4ac015ddd0 0x7f4ac0158300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:32.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.161+0000 7f4abeffd640 1 -- 172.21.15.43:0/3622022461 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4ac015ddd0 msgr2=0x7f4ac0158300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:32.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.161+0000 7f4abeffd640 1 --2- 172.21.15.43:0/3622022461 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4ac015ddd0 0x7f4ac0158300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:32.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.161+0000 7f4abeffd640 1 -- 172.21.15.43:0/3622022461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ac006f6a0 msgr2=0x7f4ac01f6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:32.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.161+0000 7f4abeffd640 1 --2- 172.21.15.43:0/3622022461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ac006f6a0 0x7f4ac01f6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:32.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.161+0000 7f4abeffd640 1 -- 172.21.15.43:0/3622022461 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac01602d0 con 0x7f4ac00cd470 2024-04-25T08:58:32.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.161+0000 7f4abeffd640 1 --2- 172.21.15.43:0/3622022461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ac00cd470 0x7f4ac00cb080 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f4ab000c430 tx=0x7f4ab000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:32.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.161+0000 7f4aa7fff640 1 -- 172.21.15.43:0/3622022461 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab00052d0 con 0x7f4ac00cd470 2024-04-25T08:58:32.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.161+0000 7f4aa7fff640 1 -- 172.21.15.43:0/3622022461 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ab0005490 con 0x7f4ac00cd470 2024-04-25T08:58:32.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.161+0000 7f4ac60d8640 1 -- 172.21.15.43:0/3622022461 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ac01605b0 con 0x7f4ac00cd470 2024-04-25T08:58:32.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.161+0000 7f4aa7fff640 1 -- 172.21.15.43:0/3622022461 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab0007260 con 0x7f4ac00cd470 2024-04-25T08:58:32.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.162+0000 7f4ac60d8640 1 -- 172.21.15.43:0/3622022461 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ac00c9680 con 0x7f4ac00cd470 2024-04-25T08:58:32.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.163+0000 7f4ac60d8640 1 -- 172.21.15.43:0/3622022461 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ac0074f60 con 0x7f4ac00cd470 2024-04-25T08:58:32.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.163+0000 7f4aa7fff640 1 -- 172.21.15.43:0/3622022461 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4ab0007400 con 0x7f4ac00cd470 2024-04-25T08:58:32.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.163+0000 7f4aa7fff640 1 --2- 172.21.15.43:0/3622022461 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4aac06b500 0x7f4aac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:32.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.163+0000 7f4aa7fff640 1 -- 172.21.15.43:0/3622022461 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f4ab0089660 con 0x7f4ac00cd470 2024-04-25T08:58:32.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.164+0000 7f4abf7fe640 1 --2- 172.21.15.43:0/3622022461 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4aac06b500 0x7f4aac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:32.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.164+0000 7f4abf7fe640 1 --2- 172.21.15.43:0/3622022461 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4aac06b500 0x7f4aac06d9c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4005880 tx=0x7f4ab4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:32.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.167+0000 7f4aa7fff640 1 -- 172.21.15.43:0/3622022461 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4ab0057e70 con 0x7f4ac00cd470 2024-04-25T08:58:32.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.346+0000 7f4ac60d8640 1 -- 172.21.15.43:0/3622022461 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4ac00b06a0 con 0x7f4ac00cd470 2024-04-25T08:58:32.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.346+0000 7f4aa7fff640 1 -- 172.21.15.43:0/3622022461 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v62) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4ab005bf10 con 0x7f4ac00cd470 2024-04-25T08:58:32.336 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T08:58:32.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.348+0000 7f4ac60d8640 1 -- 172.21.15.43:0/3622022461 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4aac06b500 msgr2=0x7f4aac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:32.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.348+0000 7f4ac60d8640 1 --2- 172.21.15.43:0/3622022461 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4aac06b500 0x7f4aac06d9c0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4005880 tx=0x7f4ab4009040 comp rx=0 tx=0).stop 2024-04-25T08:58:32.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.348+0000 7f4ac60d8640 1 -- 172.21.15.43:0/3622022461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ac00cd470 msgr2=0x7f4ac00cb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:32.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.348+0000 7f4ac60d8640 1 --2- 172.21.15.43:0/3622022461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ac00cd470 0x7f4ac00cb080 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f4ab000c430 tx=0x7f4ab000c8f0 comp rx=0 tx=0).stop 2024-04-25T08:58:32.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.348+0000 7f4ac60d8640 1 -- 172.21.15.43:0/3622022461 shutdown_connections 2024-04-25T08:58:32.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.348+0000 7f4ac60d8640 1 --2- 172.21.15.43:0/3622022461 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4ac015ddd0 0x7f4ac0158300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:32.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.348+0000 7f4ac60d8640 1 --2- 172.21.15.43:0/3622022461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ac00cd470 0x7f4ac00cb080 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:32.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.348+0000 7f4ac60d8640 1 --2- 172.21.15.43:0/3622022461 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4aac06b500 0x7f4aac06d9c0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:32.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.348+0000 7f4ac60d8640 1 --2- 172.21.15.43:0/3622022461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4ac006f6a0 0x7f4ac01f6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:32.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.348+0000 7f4ac60d8640 1 -- 172.21.15.43:0/3622022461 >> 172.21.15.43:0/3622022461 conn(0x7f4ac00d17e0 msgr2=0x7f4ac006ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:32.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.348+0000 7f4ac60d8640 1 -- 172.21.15.43:0/3622022461 shutdown_connections 2024-04-25T08:58:32.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.348+0000 7f4ac60d8640 1 -- 172.21.15.43:0/3622022461 wait complete. 2024-04-25T08:58:32.349 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:58:32.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.460+0000 7fc544177640 1 Processor -- start 2024-04-25T08:58:32.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.461+0000 7fc544177640 1 -- start start 2024-04-25T08:58:32.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.461+0000 7fc544177640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fc53c15a9d0 0x7fc53c06d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:32.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.461+0000 7fc544177640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc53c072160 0x7fc53c0cd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:32.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.461+0000 7fc544177640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc53c0704f0 con 0x7fc53c06d7d0 2024-04-25T08:58:32.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.461+0000 7fc544177640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc53c06dc30 con 0x7fc53c072160 2024-04-25T08:58:32.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.461+0000 7fc544177640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc53c0707d0 con 0x7fc53c15a9d0 2024-04-25T08:58:32.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.462+0000 7fc542714640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc53c072160 0x7fc53c0cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:32.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.462+0000 7fc541712640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc53c06d7d0 0x7fc53c071a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54936/0 (socket says 172.21.15.43:54936) 2024-04-25T08:58:32.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.462+0000 7fc541712640 1 -- 172.21.15.43:0/2004864644 learned_addr learned my addr 172.21.15.43:0/2004864644 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:32.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.462+0000 7fc541f13640 1 --2- 172.21.15.43:0/2004864644 >> v2:172.21.15.193:3300/0 conn(0x7fc53c15a9d0 0x7fc53c06d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:32.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.462+0000 7fc542714640 1 -- 172.21.15.43:0/2004864644 >> v2:172.21.15.193:3300/0 conn(0x7fc53c15a9d0 msgr2=0x7fc53c06d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:32.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.462+0000 7fc542714640 1 --2- 172.21.15.43:0/2004864644 >> v2:172.21.15.193:3300/0 conn(0x7fc53c15a9d0 0x7fc53c06d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:32.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.462+0000 7fc542714640 1 -- 172.21.15.43:0/2004864644 >> v1:172.21.15.43:6789/0 conn(0x7fc53c06d7d0 legacy=0x7fc53c071a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:32.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.462+0000 7fc540f11640 1 -- 172.21.15.43:0/2004864644 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 205286935 0 0) 0x7fc53c0704f0 con 0x7fc53c06d7d0 2024-04-25T08:58:32.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.462+0000 7fc542714640 1 -- 172.21.15.43:0/2004864644 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc53c070ab0 con 0x7fc53c072160 2024-04-25T08:58:32.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.463+0000 7fc542714640 1 --2- 172.21.15.43:0/2004864644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc53c072160 0x7fc53c0cd940 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fc53c06c190 tx=0x7fc52c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb9bfaacec94cf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:32.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.463+0000 7fc540f11640 1 -- 172.21.15.43:0/2004864644 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c007740 con 0x7fc53c072160 2024-04-25T08:58:32.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.463+0000 7fc540f11640 1 -- 172.21.15.43:0/2004864644 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52c0078e0 con 0x7fc53c072160 2024-04-25T08:58:32.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.463+0000 7fc540f11640 1 -- 172.21.15.43:0/2004864644 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c007cb0 con 0x7fc53c072160 2024-04-25T08:58:32.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.463+0000 7fc544177640 1 -- 172.21.15.43:0/2004864644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc53c072160 msgr2=0x7fc53c0cd940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:32.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.463+0000 7fc544177640 1 --2- 172.21.15.43:0/2004864644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc53c072160 0x7fc53c0cd940 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fc53c06c190 tx=0x7fc52c02a680 comp rx=0 tx=0).stop 2024-04-25T08:58:32.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.463+0000 7fc544177640 1 -- 172.21.15.43:0/2004864644 shutdown_connections 2024-04-25T08:58:32.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.463+0000 7fc544177640 1 --2- 172.21.15.43:0/2004864644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc53c072160 0x7fc53c0cd940 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:32.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.463+0000 7fc544177640 1 --2- 172.21.15.43:0/2004864644 >> v2:172.21.15.193:3300/0 conn(0x7fc53c15a9d0 0x7fc53c06d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:32.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.463+0000 7fc544177640 1 -- 172.21.15.43:0/2004864644 >> 172.21.15.43:0/2004864644 conn(0x7fc53c0b1480 msgr2=0x7fc53c0b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:32.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.463+0000 7fc544177640 1 -- 172.21.15.43:0/2004864644 shutdown_connections 2024-04-25T08:58:32.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.463+0000 7fc544177640 1 -- 172.21.15.43:0/2004864644 wait complete. 2024-04-25T08:58:32.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.464+0000 7fc544177640 1 Processor -- start 2024-04-25T08:58:32.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.464+0000 7fc544177640 1 -- start start 2024-04-25T08:58:32.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.465+0000 7fc544177640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc53c06d7d0 0x7fc53c1f4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:32.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.465+0000 7fc544177640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc53c072160 0x7fc53c14faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:32.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.465+0000 7fc544177640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc53c06c0d0 0x7fc53c06c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:32.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.465+0000 7fc544177640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53c0cde80 con 0x7fc53c06c0d0 2024-04-25T08:58:32.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.465+0000 7fc544177640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53c0712b0 con 0x7fc53c072160 2024-04-25T08:58:32.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.465+0000 7fc544177640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc53c0ce000 con 0x7fc53c06d7d0 2024-04-25T08:58:32.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.465+0000 7fc542714640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc53c06c0d0 0x7fc53c06c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:32.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.465+0000 7fc542714640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc53c06c0d0 0x7fc53c06c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36530/0 (socket says 172.21.15.43:36530) 2024-04-25T08:58:32.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.465+0000 7fc542714640 1 -- 172.21.15.43:0/4290271469 learned_addr learned my addr 172.21.15.43:0/4290271469 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:32.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.465+0000 7fc541712640 1 --2- 172.21.15.43:0/4290271469 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc53c072160 0x7fc53c14faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:32.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.465+0000 7fc541f13640 1 --2- 172.21.15.43:0/4290271469 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc53c06d7d0 0x7fc53c1f4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:32.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.466+0000 7fc542714640 1 -- 172.21.15.43:0/4290271469 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc53c06d7d0 msgr2=0x7fc53c1f4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:32.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.466+0000 7fc542714640 1 --2- 172.21.15.43:0/4290271469 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc53c06d7d0 0x7fc53c1f4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:32.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.466+0000 7fc542714640 1 -- 172.21.15.43:0/4290271469 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc53c072160 msgr2=0x7fc53c14faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:32.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.466+0000 7fc542714640 1 --2- 172.21.15.43:0/4290271469 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc53c072160 0x7fc53c14faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:32.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.466+0000 7fc542714640 1 -- 172.21.15.43:0/4290271469 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc53c150260 con 0x7fc53c06c0d0 2024-04-25T08:58:32.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.466+0000 7fc542714640 1 --2- 172.21.15.43:0/4290271469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc53c06c0d0 0x7fc53c06c560 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fc52c009e80 tx=0x7fc52c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:32.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.466+0000 7fc53a7fc640 1 -- 172.21.15.43:0/4290271469 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c040070 con 0x7fc53c06c0d0 2024-04-25T08:58:32.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.466+0000 7fc53a7fc640 1 -- 172.21.15.43:0/4290271469 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52c031c40 con 0x7fc53c06c0d0 2024-04-25T08:58:32.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.466+0000 7fc544177640 1 -- 172.21.15.43:0/4290271469 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc53c150540 con 0x7fc53c06c0d0 2024-04-25T08:58:32.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.466+0000 7fc53a7fc640 1 -- 172.21.15.43:0/4290271469 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c009450 con 0x7fc53c06c0d0 2024-04-25T08:58:32.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.466+0000 7fc544177640 1 -- 172.21.15.43:0/4290271469 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc53c150de0 con 0x7fc53c06c0d0 2024-04-25T08:58:32.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.467+0000 7fc544177640 1 -- 172.21.15.43:0/4290271469 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc53c074ef0 con 0x7fc53c06c0d0 2024-04-25T08:58:32.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.468+0000 7fc53a7fc640 1 -- 172.21.15.43:0/4290271469 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc52c039040 con 0x7fc53c06c0d0 2024-04-25T08:58:32.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.468+0000 7fc53a7fc640 1 --2- 172.21.15.43:0/4290271469 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc52806b500 0x7fc52806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:32.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.468+0000 7fc53a7fc640 1 -- 172.21.15.43:0/4290271469 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fc52c0ab370 con 0x7fc53c06c0d0 2024-04-25T08:58:32.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.468+0000 7fc541f13640 1 --2- 172.21.15.43:0/4290271469 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc52806b500 0x7fc52806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:32.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.469+0000 7fc541f13640 1 --2- 172.21.15.43:0/4290271469 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc52806b500 0x7fc52806d9c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc5340079a0 tx=0x7fc534008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:32.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.473+0000 7fc53a7fc640 1 -- 172.21.15.43:0/4290271469 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc52c079af0 con 0x7fc53c06c0d0 2024-04-25T08:58:32.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.690+0000 7fc544177640 1 -- 172.21.15.43:0/4290271469 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc53c0743d0 con 0x7fc53c06c0d0 2024-04-25T08:58:32.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.690+0000 7fc53a7fc640 1 -- 172.21.15.43:0/4290271469 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4936 (secure 0 0 0) 0x7fc52c07db90 con 0x7fc53c06c0d0 2024-04-25T08:58:32.680 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:58:32.680 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":39,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5765,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/594587590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":594587590},{"type":"v1","addr":"172.21.15.193:6835","nonce":594587590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6187,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/681300720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":681300720},{"type":"v1","addr":"172.21.15.193:6837","nonce":681300720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6876,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/432022866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":432022866},{"type":"v1","addr":"172.21.15.43:6835","nonce":432022866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:57:54.590369+0000","modified":"2024-04-25T08:58:14.432383+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6870},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6870":{"gid":6870,"name":"c","rank":0,"incarnation":36,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6833/1866869111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1866869111},{"type":"v1","addr":"172.21.15.43:6833","nonce":1866869111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:58:32.680 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 39 2024-04-25T08:58:32.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.692+0000 7fc544177640 1 -- 172.21.15.43:0/4290271469 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc52806b500 msgr2=0x7fc52806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:32.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.692+0000 7fc544177640 1 --2- 172.21.15.43:0/4290271469 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc52806b500 0x7fc52806d9c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc5340079a0 tx=0x7fc534008040 comp rx=0 tx=0).stop 2024-04-25T08:58:32.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.692+0000 7fc544177640 1 -- 172.21.15.43:0/4290271469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc53c06c0d0 msgr2=0x7fc53c06c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:32.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.692+0000 7fc544177640 1 --2- 172.21.15.43:0/4290271469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc53c06c0d0 0x7fc53c06c560 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fc52c009e80 tx=0x7fc52c004a40 comp rx=0 tx=0).stop 2024-04-25T08:58:32.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.692+0000 7fc544177640 1 -- 172.21.15.43:0/4290271469 shutdown_connections 2024-04-25T08:58:32.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.692+0000 7fc544177640 1 --2- 172.21.15.43:0/4290271469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc53c06c0d0 0x7fc53c06c560 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:32.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.692+0000 7fc544177640 1 --2- 172.21.15.43:0/4290271469 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc52806b500 0x7fc52806d9c0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:32.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.692+0000 7fc544177640 1 --2- 172.21.15.43:0/4290271469 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc53c072160 0x7fc53c14faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:32.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.692+0000 7fc544177640 1 --2- 172.21.15.43:0/4290271469 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc53c06d7d0 0x7fc53c1f4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:32.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.692+0000 7fc544177640 1 -- 172.21.15.43:0/4290271469 >> 172.21.15.43:0/4290271469 conn(0x7fc53c0b1480 msgr2=0x7fc53c158fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:32.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.693+0000 7fc544177640 1 -- 172.21.15.43:0/4290271469 shutdown_connections 2024-04-25T08:58:32.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.693+0000 7fc544177640 1 -- 172.21.15.43:0/4290271469 wait complete. 2024-04-25T08:58:32.693 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:58:32.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.804+0000 7fa87e49e640 1 Processor -- start 2024-04-25T08:58:32.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.804+0000 7fa87e49e640 1 -- start start 2024-04-25T08:58:32.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.805+0000 7fa87e49e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa87815dd00 0x7fa8780cc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:32.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.805+0000 7fa87e49e640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fa8780c5f70 0x7fa8780cda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:32.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.805+0000 7fa87e49e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8780c4ed0 con 0x7fa8780c6400 2024-04-25T08:58:32.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.805+0000 7fa87e49e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8780c51b0 con 0x7fa87815dd00 2024-04-25T08:58:32.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.805+0000 7fa87e49e640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8780c5490 con 0x7fa8780c5f70 2024-04-25T08:58:32.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.805+0000 7fa877fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa87815dd00 0x7fa8780cc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:32.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.805+0000 7fa87ca3b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa8780c6400 0x7fa87815e0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54942/0 (socket says 172.21.15.43:54942) 2024-04-25T08:58:32.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.805+0000 7fa87ca3b640 1 -- 172.21.15.43:0/3755694188 learned_addr learned my addr 172.21.15.43:0/3755694188 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:32.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.805+0000 7fa877fff640 1 -- 172.21.15.43:0/3755694188 >> v2:172.21.15.193:3300/0 conn(0x7fa8780c5f70 msgr2=0x7fa8780cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:32.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.805+0000 7fa8777fe640 1 --2- 172.21.15.43:0/3755694188 >> v2:172.21.15.193:3300/0 conn(0x7fa8780c5f70 0x7fa8780cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:32.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.805+0000 7fa877fff640 1 --2- 172.21.15.43:0/3755694188 >> v2:172.21.15.193:3300/0 conn(0x7fa8780c5f70 0x7fa8780cda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:32.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.805+0000 7fa876ffd640 1 -- 172.21.15.43:0/3755694188 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2225634089 0 0) 0x7fa8780c4ed0 con 0x7fa8780c6400 2024-04-25T08:58:32.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.805+0000 7fa877fff640 1 -- 172.21.15.43:0/3755694188 >> v1:172.21.15.43:6789/0 conn(0x7fa8780c6400 legacy=0x7fa87815e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:32.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.805+0000 7fa877fff640 1 -- 172.21.15.43:0/3755694188 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8780c5770 con 0x7fa87815dd00 2024-04-25T08:58:32.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.805+0000 7fa8777fe640 1 --2- 172.21.15.43:0/3755694188 >> v2:172.21.15.193:3300/0 conn(0x7fa8780c5f70 0x7fa8780cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:32.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.806+0000 7fa877fff640 1 --2- 172.21.15.43:0/3755694188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa87815dd00 0x7fa8780cc270 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fa8780cfe30 tx=0x7fa86c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73c7803f8e4d4fdc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:32.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.806+0000 7fa876ffd640 1 -- 172.21.15.43:0/3755694188 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c037070 con 0x7fa87815dd00 2024-04-25T08:58:32.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.806+0000 7fa876ffd640 1 -- 172.21.15.43:0/3755694188 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa86c032440 con 0x7fa87815dd00 2024-04-25T08:58:32.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.806+0000 7fa876ffd640 1 -- 172.21.15.43:0/3755694188 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c032810 con 0x7fa87815dd00 2024-04-25T08:58:32.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.807+0000 7fa87e49e640 1 -- 172.21.15.43:0/3755694188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa87815dd00 msgr2=0x7fa8780cc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:32.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.807+0000 7fa87e49e640 1 --2- 172.21.15.43:0/3755694188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa87815dd00 0x7fa8780cc270 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fa8780cfe30 tx=0x7fa86c02a1a0 comp rx=0 tx=0).stop 2024-04-25T08:58:32.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.807+0000 7fa87e49e640 1 -- 172.21.15.43:0/3755694188 shutdown_connections 2024-04-25T08:58:32.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.807+0000 7fa87e49e640 1 --2- 172.21.15.43:0/3755694188 >> v2:172.21.15.193:3300/0 conn(0x7fa8780c5f70 0x7fa8780cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:32.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.807+0000 7fa87e49e640 1 --2- 172.21.15.43:0/3755694188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa87815dd00 0x7fa8780cc270 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:32.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.807+0000 7fa87e49e640 1 -- 172.21.15.43:0/3755694188 >> 172.21.15.43:0/3755694188 conn(0x7fa87806ab30 msgr2=0x7fa87806af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:32.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.807+0000 7fa87e49e640 1 -- 172.21.15.43:0/3755694188 shutdown_connections 2024-04-25T08:58:32.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.807+0000 7fa87e49e640 1 -- 172.21.15.43:0/3755694188 wait complete. 2024-04-25T08:58:32.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.808+0000 7fa87e49e640 1 Processor -- start 2024-04-25T08:58:32.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.808+0000 7fa87e49e640 1 -- start start 2024-04-25T08:58:32.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.809+0000 7fa87e49e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8780c5f70 0x7fa8780c8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:32.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.809+0000 7fa87e49e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa8780c6400 0x7fa8780cec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:32.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.809+0000 7fa87e49e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa87815dd00 0x7fa8780cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:32.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.809+0000 7fa87e49e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8780ce120 con 0x7fa8780c5f70 2024-04-25T08:58:32.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.809+0000 7fa87e49e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8780cdfa0 con 0x7fa8780c6400 2024-04-25T08:58:32.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.809+0000 7fa87e49e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8780ce2a0 con 0x7fa87815dd00 2024-04-25T08:58:32.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.809+0000 7fa877fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8780c5f70 0x7fa8780c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:32.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.809+0000 7fa877fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8780c5f70 0x7fa8780c8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36546/0 (socket says 172.21.15.43:36546) 2024-04-25T08:58:32.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.809+0000 7fa877fff640 1 -- 172.21.15.43:0/3555532547 learned_addr learned my addr 172.21.15.43:0/3555532547 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:32.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.809+0000 7fa8777fe640 1 --2- 172.21.15.43:0/3555532547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa8780c6400 0x7fa8780cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:32.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.809+0000 7fa87ca3b640 1 --2- 172.21.15.43:0/3555532547 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa87815dd00 0x7fa8780cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:32.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.809+0000 7fa877fff640 1 -- 172.21.15.43:0/3555532547 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa87815dd00 msgr2=0x7fa8780cf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:32.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.809+0000 7fa877fff640 1 --2- 172.21.15.43:0/3555532547 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa87815dd00 0x7fa8780cf1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:32.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.809+0000 7fa877fff640 1 -- 172.21.15.43:0/3555532547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa8780c6400 msgr2=0x7fa8780cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:32.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.809+0000 7fa877fff640 1 --2- 172.21.15.43:0/3555532547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa8780c6400 0x7fa8780cec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:32.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.809+0000 7fa877fff640 1 -- 172.21.15.43:0/3555532547 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8781f4b60 con 0x7fa8780c5f70 2024-04-25T08:58:32.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.809+0000 7fa8777fe640 1 --2- 172.21.15.43:0/3555532547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa8780c6400 0x7fa8780cec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:32.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.809+0000 7fa87ca3b640 1 --2- 172.21.15.43:0/3555532547 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa87815dd00 0x7fa8780cf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:32.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.809+0000 7fa877fff640 1 --2- 172.21.15.43:0/3555532547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8780c5f70 0x7fa8780c8760 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fa86c000c00 tx=0x7fa86c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:32.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.810+0000 7fa874ff9640 1 -- 172.21.15.43:0/3555532547 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c037040 con 0x7fa8780c5f70 2024-04-25T08:58:32.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.810+0000 7fa874ff9640 1 -- 172.21.15.43:0/3555532547 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa86c042070 con 0x7fa8780c5f70 2024-04-25T08:58:32.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.810+0000 7fa87e49e640 1 -- 172.21.15.43:0/3555532547 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8781f4880 con 0x7fa8780c5f70 2024-04-25T08:58:32.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.810+0000 7fa874ff9640 1 -- 172.21.15.43:0/3555532547 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86c030a50 con 0x7fa8780c5f70 2024-04-25T08:58:32.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.810+0000 7fa87e49e640 1 -- 172.21.15.43:0/3555532547 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8781f75c0 con 0x7fa8780c5f70 2024-04-25T08:58:32.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.811+0000 7fa87e49e640 1 -- 172.21.15.43:0/3555532547 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa878074ef0 con 0x7fa8780c5f70 2024-04-25T08:58:32.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.811+0000 7fa874ff9640 1 -- 172.21.15.43:0/3555532547 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa86c030bf0 con 0x7fa8780c5f70 2024-04-25T08:58:32.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.811+0000 7fa874ff9640 1 --2- 172.21.15.43:0/3555532547 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa86406b4d0 0x7fa86406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:32.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.811+0000 7fa874ff9640 1 -- 172.21.15.43:0/3555532547 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fa86c0aad30 con 0x7fa8780c5f70 2024-04-25T08:58:32.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.812+0000 7fa8777fe640 1 --2- 172.21.15.43:0/3555532547 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa86406b4d0 0x7fa86406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:32.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.812+0000 7fa8777fe640 1 --2- 172.21.15.43:0/3555532547 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa86406b4d0 0x7fa86406d990 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa868006fd0 tx=0x7fa868008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:32.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.816+0000 7fa874ff9640 1 -- 172.21.15.43:0/3555532547 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa86c079b90 con 0x7fa8780c5f70 2024-04-25T08:58:32.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.995+0000 7fa87e49e640 1 -- 172.21.15.43:0/3555532547 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa8780cad30 con 0x7fa8780c5f70 2024-04-25T08:58:32.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.996+0000 7fa874ff9640 1 -- 172.21.15.43:0/3555532547 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+16888 (secure 0 0 0) 0x7fa86c079490 con 0x7fa8780c5f70 2024-04-25T08:58:32.986 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:58:32.986 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":62,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:58:12.413565+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:57:52.884445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T08:57:53.565716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6834/3574570511":"2024-04-26T08:58:12.413536+0000","172.21.15.43:6832/3574570511":"2024-04-26T08:58:12.413536+0000","172.21.15.43:6835/2268368384":"2024-04-26T08:58:11.416075+0000","172.21.15.43:6833/2268368384":"2024-04-26T08:58:11.416075+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:58:32.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.998+0000 7fa87e49e640 1 -- 172.21.15.43:0/3555532547 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa86406b4d0 msgr2=0x7fa86406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:32.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.998+0000 7fa87e49e640 1 --2- 172.21.15.43:0/3555532547 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa86406b4d0 0x7fa86406d990 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa868006fd0 tx=0x7fa868008040 comp rx=0 tx=0).stop 2024-04-25T08:58:32.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.998+0000 7fa87e49e640 1 -- 172.21.15.43:0/3555532547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8780c5f70 msgr2=0x7fa8780c8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:32.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.998+0000 7fa87e49e640 1 --2- 172.21.15.43:0/3555532547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8780c5f70 0x7fa8780c8760 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fa86c000c00 tx=0x7fa86c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:58:32.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.998+0000 7fa87e49e640 1 -- 172.21.15.43:0/3555532547 shutdown_connections 2024-04-25T08:58:32.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.998+0000 7fa87e49e640 1 --2- 172.21.15.43:0/3555532547 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa87815dd00 0x7fa8780cf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:32.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.998+0000 7fa87e49e640 1 --2- 172.21.15.43:0/3555532547 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa86406b4d0 0x7fa86406d990 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:32.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.998+0000 7fa87e49e640 1 --2- 172.21.15.43:0/3555532547 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa8780c6400 0x7fa8780cec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:32.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.998+0000 7fa87e49e640 1 --2- 172.21.15.43:0/3555532547 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa8780c5f70 0x7fa8780c8760 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:32.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.998+0000 7fa87e49e640 1 -- 172.21.15.43:0/3555532547 >> 172.21.15.43:0/3555532547 conn(0x7fa87806ab30 msgr2=0x7fa878071210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:32.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.998+0000 7fa87e49e640 1 -- 172.21.15.43:0/3555532547 shutdown_connections 2024-04-25T08:58:32.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:32.998+0000 7fa87e49e640 1 -- 172.21.15.43:0/3555532547 wait complete. 2024-04-25T08:58:32.999 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T08:58:32.999 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:58:33.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.110+0000 7f869b4d8640 1 Processor -- start 2024-04-25T08:58:33.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.110+0000 7f869b4d8640 1 -- start start 2024-04-25T08:58:33.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.111+0000 7f869b4d8640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f869415dce0 0x7f869415ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:33.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.111+0000 7f869b4d8640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86940d2920 con 0x7f86940d0af0 2024-04-25T08:58:33.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.111+0000 7f869b4d8640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86940cee80 con 0x7f86940ce550 2024-04-25T08:58:33.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.111+0000 7f869b4d8640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86940cf160 con 0x7f869415dce0 2024-04-25T08:58:33.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.111+0000 7f8699a75640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f86940d0af0 0x7f8694160980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54952/0 (socket says 172.21.15.43:54952) 2024-04-25T08:58:33.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.111+0000 7f8699a75640 1 -- 172.21.15.43:0/3866312939 learned_addr learned my addr 172.21.15.43:0/3866312939 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:33.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.111+0000 7f8699274640 1 --2- 172.21.15.43:0/3866312939 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f869415dce0 0x7f869415ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:33.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.111+0000 7f868bfff640 1 -- 172.21.15.43:0/3866312939 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 885481515 0 0) 0x7f86940d2920 con 0x7f86940d0af0 2024-04-25T08:58:33.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.111+0000 7f868bfff640 1 -- 172.21.15.43:0/3866312939 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8680003520 con 0x7f86940d0af0 2024-04-25T08:58:33.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.111+0000 7f868bfff640 1 -- 172.21.15.43:0/3866312939 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3377105797 0 0) 0x7f8680003520 con 0x7f86940d0af0 2024-04-25T08:58:33.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.111+0000 7f868bfff640 1 -- 172.21.15.43:0/3866312939 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f869415dce0 msgr2=0x7f869415ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:33.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.111+0000 7f868bfff640 1 --2- 172.21.15.43:0/3866312939 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f869415dce0 0x7f869415ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:33.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.111+0000 7f868bfff640 1 -- 172.21.15.43:0/3866312939 >> v1:172.21.15.193:6789/0 conn(0x7f86940ce550 legacy=0x7f86940d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:33.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.111+0000 7f868bfff640 1 -- 172.21.15.43:0/3866312939 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86940cf440 con 0x7f86940d0af0 2024-04-25T08:58:33.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.111+0000 7f868bfff640 1 -- 172.21.15.43:0/3866312939 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f868c001a10 con 0x7f86940d0af0 2024-04-25T08:58:33.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.111+0000 7f868bfff640 1 -- 172.21.15.43:0/3866312939 >> v1:172.21.15.43:6789/0 conn(0x7f86940d0af0 legacy=0x7f8694160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:33.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.112+0000 7f868bfff640 1 --2- 172.21.15.43:0/3866312939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86800039e0 0x7f8680003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:33.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.112+0000 7f868bfff640 1 --2- 172.21.15.43:0/3866312939 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f869415dce0 0x7f8680004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:33.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.112+0000 7f868bfff640 1 --2- 172.21.15.43:0/3866312939 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8680004d40 0x7f86800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:33.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.112+0000 7f868bfff640 1 -- 172.21.15.43:0/3866312939 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86940cf160 con 0x7f86800039e0 2024-04-25T08:58:33.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.112+0000 7f868bfff640 1 -- 172.21.15.43:0/3866312939 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f868000baf0 con 0x7f8680004d40 2024-04-25T08:58:33.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.112+0000 7f868bfff640 1 -- 172.21.15.43:0/3866312939 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f868000bcf0 con 0x7f869415dce0 2024-04-25T08:58:33.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.112+0000 7f8699274640 1 --2- 172.21.15.43:0/3866312939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86800039e0 0x7f8680003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:33.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.112+0000 7f8699a75640 1 --2- 172.21.15.43:0/3866312939 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8680004d40 0x7f86800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:33.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.112+0000 7f8698a73640 1 --2- 172.21.15.43:0/3866312939 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f869415dce0 0x7f8680004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:33.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.112+0000 7f8699274640 1 -- 172.21.15.43:0/3866312939 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f869415dce0 msgr2=0x7f8680004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:33.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.112+0000 7f8699274640 1 --2- 172.21.15.43:0/3866312939 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f869415dce0 0x7f8680004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:33.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.112+0000 7f8699274640 1 -- 172.21.15.43:0/3866312939 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8680004d40 msgr2=0x7f86800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:33.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.112+0000 7f8699274640 1 --2- 172.21.15.43:0/3866312939 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8680004d40 0x7f86800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:33.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.112+0000 7f8699274640 1 -- 172.21.15.43:0/3866312939 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8680003520 con 0x7f86800039e0 2024-04-25T08:58:33.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.113+0000 7f8699274640 1 --2- 172.21.15.43:0/3866312939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86800039e0 0x7f8680003e00 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f8690009ac0 tx=0x7f8680005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68e5bb8ce163119a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:33.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.113+0000 7f868bfff640 1 -- 172.21.15.43:0/3866312939 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8690036070 con 0x7f86800039e0 2024-04-25T08:58:33.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.113+0000 7f868bfff640 1 -- 172.21.15.43:0/3866312939 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8690031400 con 0x7f86800039e0 2024-04-25T08:58:33.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.113+0000 7f868bfff640 1 -- 172.21.15.43:0/3866312939 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8690009690 con 0x7f86800039e0 2024-04-25T08:58:33.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.113+0000 7f869b4d8640 1 -- 172.21.15.43:0/3866312939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86800039e0 msgr2=0x7f8680003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:33.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.113+0000 7f869b4d8640 1 --2- 172.21.15.43:0/3866312939 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86800039e0 0x7f8680003e00 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f8690009ac0 tx=0x7f8680005830 comp rx=0 tx=0).stop 2024-04-25T08:58:33.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.113+0000 7f8699a75640 1 -- 172.21.15.43:0/3866312939 reap_dead start 2024-04-25T08:58:33.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.113+0000 7f869b4d8640 1 -- 172.21.15.43:0/3866312939 shutdown_connections 2024-04-25T08:58:33.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.113+0000 7f869b4d8640 1 -- 172.21.15.43:0/3866312939 >> 172.21.15.43:0/3866312939 conn(0x7f869409cd00 msgr2=0x7f86940d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:33.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.114+0000 7f869b4d8640 1 -- 172.21.15.43:0/3866312939 shutdown_connections 2024-04-25T08:58:33.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.114+0000 7f869b4d8640 1 -- 172.21.15.43:0/3866312939 wait complete. 2024-04-25T08:58:33.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.115+0000 7f869b4d8640 1 Processor -- start 2024-04-25T08:58:33.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.115+0000 7f869b4d8640 1 -- start start 2024-04-25T08:58:33.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.115+0000 7f869b4d8640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f86800039e0 0x7f8694155410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:33.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.115+0000 7f869b4d8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8694156f30 0x7f8694158660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:33.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.115+0000 7f869b4d8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86941547c0 0x7f869406c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:33.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.115+0000 7f869b4d8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869415e0c0 con 0x7f86941547c0 2024-04-25T08:58:33.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.115+0000 7f869b4d8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869415f6b0 con 0x7f8694156f30 2024-04-25T08:58:33.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.115+0000 7f869b4d8640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86940ce990 con 0x7f86800039e0 2024-04-25T08:58:33.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.116+0000 7f8699a75640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86941547c0 0x7f869406c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:33.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.116+0000 7f8699a75640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86941547c0 0x7f869406c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36560/0 (socket says 172.21.15.43:36560) 2024-04-25T08:58:33.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.116+0000 7f8699a75640 1 -- 172.21.15.43:0/2056214656 learned_addr learned my addr 172.21.15.43:0/2056214656 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:33.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.116+0000 7f8698a73640 1 --2- 172.21.15.43:0/2056214656 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8694156f30 0x7f8694158660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:33.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.116+0000 7f8699274640 1 --2- 172.21.15.43:0/2056214656 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f86800039e0 0x7f8694155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:33.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.116+0000 7f8699a75640 1 -- 172.21.15.43:0/2056214656 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f86800039e0 msgr2=0x7f8694155410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:33.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.116+0000 7f8699a75640 1 --2- 172.21.15.43:0/2056214656 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f86800039e0 0x7f8694155410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:33.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.116+0000 7f8699a75640 1 -- 172.21.15.43:0/2056214656 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8694156f30 msgr2=0x7f8694158660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:33.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.116+0000 7f8699a75640 1 --2- 172.21.15.43:0/2056214656 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8694156f30 0x7f8694158660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:33.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.116+0000 7f8699a75640 1 -- 172.21.15.43:0/2056214656 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8694158ba0 con 0x7f86941547c0 2024-04-25T08:58:33.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.116+0000 7f8699a75640 1 --2- 172.21.15.43:0/2056214656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86941547c0 0x7f869406c400 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f86940c7290 tx=0x7f868c007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:33.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.116+0000 7f8689ffb640 1 -- 172.21.15.43:0/2056214656 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c017020 con 0x7f86941547c0 2024-04-25T08:58:33.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.116+0000 7f8689ffb640 1 -- 172.21.15.43:0/2056214656 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868c00ac20 con 0x7f86941547c0 2024-04-25T08:58:33.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.116+0000 7f869b4d8640 1 -- 172.21.15.43:0/2056214656 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8694156170 con 0x7f86941547c0 2024-04-25T08:58:33.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.116+0000 7f8689ffb640 1 -- 172.21.15.43:0/2056214656 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868c0133f0 con 0x7f86941547c0 2024-04-25T08:58:33.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.116+0000 7f869b4d8640 1 -- 172.21.15.43:0/2056214656 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8694156730 con 0x7f86941547c0 2024-04-25T08:58:33.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.117+0000 7f869b4d8640 1 -- 172.21.15.43:0/2056214656 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86940680f0 con 0x7f86941547c0 2024-04-25T08:58:33.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.118+0000 7f8689ffb640 1 -- 172.21.15.43:0/2056214656 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f868c01a020 con 0x7f86941547c0 2024-04-25T08:58:33.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.118+0000 7f8689ffb640 1 --2- 172.21.15.43:0/2056214656 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f867806b500 0x7f867806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:33.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.118+0000 7f8689ffb640 1 -- 172.21.15.43:0/2056214656 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f868c08aab0 con 0x7f86941547c0 2024-04-25T08:58:33.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.119+0000 7f8699274640 1 --2- 172.21.15.43:0/2056214656 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f867806b500 0x7f867806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:33.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.119+0000 7f8699274640 1 --2- 172.21.15.43:0/2056214656 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f867806b500 0x7f867806d9c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8690033000 tx=0x7f8690008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:33.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.122+0000 7f8689ffb640 1 -- 172.21.15.43:0/2056214656 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f868c021020 con 0x7f86941547c0 2024-04-25T08:58:33.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.338+0000 7f869b4d8640 1 -- 172.21.15.43:0/2056214656 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8694074ef0 con 0x7f86941547c0 2024-04-25T08:58:33.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.338+0000 7f8689ffb640 1 -- 172.21.15.43:0/2056214656 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4936 (secure 0 0 0) 0x7f868c059910 con 0x7f86941547c0 2024-04-25T08:58:33.328 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:58:33.328 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":39,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5765,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/594587590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":594587590},{"type":"v1","addr":"172.21.15.193:6835","nonce":594587590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6187,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/681300720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":681300720},{"type":"v1","addr":"172.21.15.193:6837","nonce":681300720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6876,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/432022866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":432022866},{"type":"v1","addr":"172.21.15.43:6835","nonce":432022866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:57:54.590369+0000","modified":"2024-04-25T08:58:14.432383+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6870},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6870":{"gid":6870,"name":"c","rank":0,"incarnation":36,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6833/1866869111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1866869111},{"type":"v1","addr":"172.21.15.43:6833","nonce":1866869111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:58:33.328 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 39 2024-04-25T08:58:33.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.340+0000 7f869b4d8640 1 -- 172.21.15.43:0/2056214656 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f867806b500 msgr2=0x7f867806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:33.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.340+0000 7f869b4d8640 1 --2- 172.21.15.43:0/2056214656 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f867806b500 0x7f867806d9c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8690033000 tx=0x7f8690008030 comp rx=0 tx=0).stop 2024-04-25T08:58:33.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.340+0000 7f869b4d8640 1 -- 172.21.15.43:0/2056214656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86941547c0 msgr2=0x7f869406c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:33.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.340+0000 7f869b4d8640 1 --2- 172.21.15.43:0/2056214656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86941547c0 0x7f869406c400 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f86940c7290 tx=0x7f868c007bd0 comp rx=0 tx=0).stop 2024-04-25T08:58:33.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.340+0000 7f869b4d8640 1 -- 172.21.15.43:0/2056214656 shutdown_connections 2024-04-25T08:58:33.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.340+0000 7f869b4d8640 1 --2- 172.21.15.43:0/2056214656 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f86941547c0 0x7f869406c400 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:33.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.340+0000 7f869b4d8640 1 --2- 172.21.15.43:0/2056214656 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f867806b500 0x7f867806d9c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:33.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.340+0000 7f869b4d8640 1 --2- 172.21.15.43:0/2056214656 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8694156f30 0x7f8694158660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:33.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.340+0000 7f869b4d8640 1 --2- 172.21.15.43:0/2056214656 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f86800039e0 0x7f8694155410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:33.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.340+0000 7f869b4d8640 1 -- 172.21.15.43:0/2056214656 >> 172.21.15.43:0/2056214656 conn(0x7f869409cd00 msgr2=0x7f86940c86a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:33.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.340+0000 7f869b4d8640 1 -- 172.21.15.43:0/2056214656 shutdown_connections 2024-04-25T08:58:33.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.340+0000 7f869b4d8640 1 -- 172.21.15.43:0/2056214656 wait complete. 2024-04-25T08:58:33.341 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:58:33.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.453+0000 7f7f9556a640 1 Processor -- start 2024-04-25T08:58:33.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.453+0000 7f7f9556a640 1 -- start start 2024-04-25T08:58:33.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.454+0000 7f7f9556a640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f7f900b0870 0x7f7f900ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:33.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.454+0000 7f7f9556a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f900cbee0 con 0x7f7f9015df70 2024-04-25T08:58:33.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.454+0000 7f7f9556a640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f900d0650 con 0x7f7f9006c510 2024-04-25T08:58:33.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.454+0000 7f7f9556a640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f900d0400 con 0x7f7f900b0870 2024-04-25T08:58:33.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.454+0000 7f7f87fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f7f9015df70 0x7f7f900cc670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54966/0 (socket says 172.21.15.43:54966) 2024-04-25T08:58:33.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.454+0000 7f7f87fff640 1 -- 172.21.15.43:0/2090994760 learned_addr learned my addr 172.21.15.43:0/2090994760 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:33.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.454+0000 7f7f8f7fe640 1 --2- 172.21.15.43:0/2090994760 >> v2:172.21.15.193:3300/0 conn(0x7f7f900b0870 0x7f7f900ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:33.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.454+0000 7f7f8e7fc640 1 -- 172.21.15.43:0/2090994760 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3022251339 0 0) 0x7f7f900cbee0 con 0x7f7f9015df70 2024-04-25T08:58:33.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.454+0000 7f7f8e7fc640 1 -- 172.21.15.43:0/2090994760 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f7c003520 con 0x7f7f9015df70 2024-04-25T08:58:33.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.454+0000 7f7f8e7fc640 1 -- 172.21.15.43:0/2090994760 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3122348695 0 0) 0x7f7f900d0650 con 0x7f7f9006c510 2024-04-25T08:58:33.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.454+0000 7f7f8e7fc640 1 -- 172.21.15.43:0/2090994760 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f900cbee0 con 0x7f7f9006c510 2024-04-25T08:58:33.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.455+0000 7f7f8f7fe640 1 -- 172.21.15.43:0/2090994760 >> v1:172.21.15.193:6789/0 conn(0x7f7f9006c510 legacy=0x7f7f9006c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:33.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.455+0000 7f7f8f7fe640 1 -- 172.21.15.43:0/2090994760 >> v1:172.21.15.43:6789/0 conn(0x7f7f9015df70 legacy=0x7f7f900cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:33.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.455+0000 7f7f8f7fe640 1 -- 172.21.15.43:0/2090994760 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f900d0930 con 0x7f7f900b0870 2024-04-25T08:58:33.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.455+0000 7f7f8f7fe640 1 --2- 172.21.15.43:0/2090994760 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f900b0870 0x7f7f900ccd80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7f88009dc0 tx=0x7f7f8802a6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f7266c801ac6c7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:33.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.456+0000 7f7f8e7fc640 1 -- 172.21.15.43:0/2090994760 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f88007740 con 0x7f7f900b0870 2024-04-25T08:58:33.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.456+0000 7f7f8e7fc640 1 -- 172.21.15.43:0/2090994760 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f880078e0 con 0x7f7f900b0870 2024-04-25T08:58:33.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.456+0000 7f7f8e7fc640 1 -- 172.21.15.43:0/2090994760 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f88007cb0 con 0x7f7f900b0870 2024-04-25T08:58:33.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.456+0000 7f7f9556a640 1 -- 172.21.15.43:0/2090994760 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f900b0870 msgr2=0x7f7f900ccd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:33.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.456+0000 7f7f9556a640 1 --2- 172.21.15.43:0/2090994760 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f900b0870 0x7f7f900ccd80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7f88009dc0 tx=0x7f7f8802a6a0 comp rx=0 tx=0).stop 2024-04-25T08:58:33.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.456+0000 7f7f9556a640 1 -- 172.21.15.43:0/2090994760 shutdown_connections 2024-04-25T08:58:33.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.456+0000 7f7f9556a640 1 --2- 172.21.15.43:0/2090994760 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f900b0870 0x7f7f900ccd80 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:33.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.456+0000 7f7f9556a640 1 -- 172.21.15.43:0/2090994760 >> 172.21.15.43:0/2090994760 conn(0x7f7f900d17f0 msgr2=0x7f7f900d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:33.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.456+0000 7f7f9556a640 1 -- 172.21.15.43:0/2090994760 shutdown_connections 2024-04-25T08:58:33.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.456+0000 7f7f9556a640 1 -- 172.21.15.43:0/2090994760 wait complete. 2024-04-25T08:58:33.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.457+0000 7f7f9556a640 1 Processor -- start 2024-04-25T08:58:33.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.457+0000 7f7f9556a640 1 -- start start 2024-04-25T08:58:33.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.458+0000 7f7f9556a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f9006c510 0x7f7f901f2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:33.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.458+0000 7f7f9556a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7f900b0870 0x7f7f900ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:33.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.458+0000 7f7f9556a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f9015df70 0x7f7f900c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:33.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.458+0000 7f7f9556a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9006bd10 con 0x7f7f9015df70 2024-04-25T08:58:33.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.458+0000 7f7f9556a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f900d0c10 con 0x7f7f9006c510 2024-04-25T08:58:33.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.458+0000 7f7f9556a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9006be90 con 0x7f7f900b0870 2024-04-25T08:58:33.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.458+0000 7f7f8f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f9015df70 0x7f7f900c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:33.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.458+0000 7f7f8f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f9015df70 0x7f7f900c88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36574/0 (socket says 172.21.15.43:36574) 2024-04-25T08:58:33.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.458+0000 7f7f8f7fe640 1 -- 172.21.15.43:0/1679059398 learned_addr learned my addr 172.21.15.43:0/1679059398 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:33.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.458+0000 7f7f87fff640 1 --2- 172.21.15.43:0/1679059398 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7f900b0870 0x7f7f900ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:33.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.458+0000 7f7f8effd640 1 --2- 172.21.15.43:0/1679059398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f9006c510 0x7f7f901f2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:33.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.459+0000 7f7f8f7fe640 1 -- 172.21.15.43:0/1679059398 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7f900b0870 msgr2=0x7f7f900ca3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:33.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.459+0000 7f7f8f7fe640 1 --2- 172.21.15.43:0/1679059398 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7f900b0870 0x7f7f900ca3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:33.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.459+0000 7f7f8f7fe640 1 -- 172.21.15.43:0/1679059398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f9006c510 msgr2=0x7f7f901f2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:33.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.459+0000 7f7f8f7fe640 1 --2- 172.21.15.43:0/1679059398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f9006c510 0x7f7f901f2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:33.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.459+0000 7f7f8f7fe640 1 -- 172.21.15.43:0/1679059398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f900c9340 con 0x7f7f9015df70 2024-04-25T08:58:33.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.459+0000 7f7f8f7fe640 1 --2- 172.21.15.43:0/1679059398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f9015df70 0x7f7f900c88c0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f7f8802ac90 tx=0x7f7f88004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:33.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.459+0000 7f7f877fe640 1 -- 172.21.15.43:0/1679059398 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f8803f070 con 0x7f7f9015df70 2024-04-25T08:58:33.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.459+0000 7f7f877fe640 1 -- 172.21.15.43:0/1679059398 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f88031bc0 con 0x7f7f9015df70 2024-04-25T08:58:33.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.459+0000 7f7f9556a640 1 -- 172.21.15.43:0/1679059398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f900c9620 con 0x7f7f9015df70 2024-04-25T08:58:33.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.459+0000 7f7f877fe640 1 -- 172.21.15.43:0/1679059398 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f88009450 con 0x7f7f9015df70 2024-04-25T08:58:33.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.459+0000 7f7f9556a640 1 -- 172.21.15.43:0/1679059398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f900c9be0 con 0x7f7f9015df70 2024-04-25T08:58:33.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.460+0000 7f7f9556a640 1 -- 172.21.15.43:0/1679059398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f90074ef0 con 0x7f7f9015df70 2024-04-25T08:58:33.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.461+0000 7f7f877fe640 1 -- 172.21.15.43:0/1679059398 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7f8803b410 con 0x7f7f9015df70 2024-04-25T08:58:33.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.461+0000 7f7f877fe640 1 --2- 172.21.15.43:0/1679059398 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7f7006b4d0 0x7f7f7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:33.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.461+0000 7f7f877fe640 1 -- 172.21.15.43:0/1679059398 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f7f880ab830 con 0x7f7f9015df70 2024-04-25T08:58:33.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.461+0000 7f7f8effd640 1 --2- 172.21.15.43:0/1679059398 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7f7006b4d0 0x7f7f7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:33.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.462+0000 7f7f8effd640 1 --2- 172.21.15.43:0/1679059398 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7f7006b4d0 0x7f7f7006d990 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7f80007fd0 tx=0x7f7f80009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:33.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.465+0000 7f7f877fe640 1 -- 172.21.15.43:0/1679059398 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7f88079fb0 con 0x7f7f9015df70 2024-04-25T08:58:33.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.682+0000 7f7f9556a640 1 -- 172.21.15.43:0/1679059398 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f900743d0 con 0x7f7f9015df70 2024-04-25T08:58:33.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.682+0000 7f7f877fe640 1 -- 172.21.15.43:0/1679059398 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4936 (secure 0 0 0) 0x7f7f8807e050 con 0x7f7f9015df70 2024-04-25T08:58:33.672 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:58:33.672 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":39,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5765,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/594587590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":594587590},{"type":"v1","addr":"172.21.15.193:6835","nonce":594587590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6187,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/681300720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":681300720},{"type":"v1","addr":"172.21.15.193:6837","nonce":681300720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6876,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/432022866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":432022866},{"type":"v1","addr":"172.21.15.43:6835","nonce":432022866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:57:54.590369+0000","modified":"2024-04-25T08:58:14.432383+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":62,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6870},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6870":{"gid":6870,"name":"c","rank":0,"incarnation":36,"state":"up:active","state_seq":4,"addr":"172.21.15.43:6833/1866869111","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":1866869111},{"type":"v1","addr":"172.21.15.43:6833","nonce":1866869111}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T08:58:33.672 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 39 2024-04-25T08:58:33.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.684+0000 7f7f9556a640 1 -- 172.21.15.43:0/1679059398 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7f7006b4d0 msgr2=0x7f7f7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:33.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.684+0000 7f7f9556a640 1 --2- 172.21.15.43:0/1679059398 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7f7006b4d0 0x7f7f7006d990 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7f80007fd0 tx=0x7f7f80009040 comp rx=0 tx=0).stop 2024-04-25T08:58:33.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.684+0000 7f7f9556a640 1 -- 172.21.15.43:0/1679059398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f9015df70 msgr2=0x7f7f900c88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:33.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.684+0000 7f7f9556a640 1 --2- 172.21.15.43:0/1679059398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f9015df70 0x7f7f900c88c0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f7f8802ac90 tx=0x7f7f88004a40 comp rx=0 tx=0).stop 2024-04-25T08:58:33.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.684+0000 7f7f9556a640 1 -- 172.21.15.43:0/1679059398 shutdown_connections 2024-04-25T08:58:33.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.684+0000 7f7f9556a640 1 --2- 172.21.15.43:0/1679059398 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7f9015df70 0x7f7f900c88c0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:33.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.684+0000 7f7f9556a640 1 --2- 172.21.15.43:0/1679059398 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7f900b0870 0x7f7f900ca3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:33.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.684+0000 7f7f9556a640 1 --2- 172.21.15.43:0/1679059398 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7f7006b4d0 0x7f7f7006d990 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:33.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.684+0000 7f7f9556a640 1 --2- 172.21.15.43:0/1679059398 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7f9006c510 0x7f7f901f2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:33.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.684+0000 7f7f9556a640 1 -- 172.21.15.43:0/1679059398 >> 172.21.15.43:0/1679059398 conn(0x7f7f900d17f0 msgr2=0x7f7f9015bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:33.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.684+0000 7f7f9556a640 1 -- 172.21.15.43:0/1679059398 shutdown_connections 2024-04-25T08:58:33.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.684+0000 7f7f9556a640 1 -- 172.21.15.43:0/1679059398 wait complete. 2024-04-25T08:58:33.685 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:58:33.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.797+0000 7fc64ddf5640 1 Processor -- start 2024-04-25T08:58:33.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.797+0000 7fc64ddf5640 1 -- start start 2024-04-25T08:58:33.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.798+0000 7fc64ddf5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc64815dce0 0x7fc64815ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:33.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.798+0000 7fc64ddf5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6480d2920 con 0x7fc6480d0af0 2024-04-25T08:58:33.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.798+0000 7fc64ddf5640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6480cee80 con 0x7fc6480ce550 2024-04-25T08:58:33.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.798+0000 7fc64ddf5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6480cf160 con 0x7fc64815dce0 2024-04-25T08:58:33.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.798+0000 7fc6477fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc64815dce0 0x7fc64815ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:33.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.798+0000 7fc647fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc6480d0af0 0x7fc648160980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54976/0 (socket says 172.21.15.43:54976) 2024-04-25T08:58:33.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.798+0000 7fc647fff640 1 -- 172.21.15.43:0/3409203004 learned_addr learned my addr 172.21.15.43:0/3409203004 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:33.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.798+0000 7fc6477fe640 1 -- 172.21.15.43:0/3409203004 >> v1:172.21.15.193:6789/0 conn(0x7fc6480ce550 legacy=0x7fc6480d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:33.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.798+0000 7fc6477fe640 1 -- 172.21.15.43:0/3409203004 >> v1:172.21.15.43:6789/0 conn(0x7fc6480d0af0 legacy=0x7fc648160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:33.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.798+0000 7fc6477fe640 1 -- 172.21.15.43:0/3409203004 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6480cf440 con 0x7fc64815dce0 2024-04-25T08:58:33.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.799+0000 7fc6477fe640 1 --2- 172.21.15.43:0/3409203004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc64815dce0 0x7fc64815ef50 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fc6380097a0 tx=0x7fc63802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15589fad85544833 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:33.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.799+0000 7fc6467fc640 1 -- 172.21.15.43:0/3409203004 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc638037070 con 0x7fc64815dce0 2024-04-25T08:58:33.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.799+0000 7fc6467fc640 1 -- 172.21.15.43:0/3409203004 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc63802ae50 con 0x7fc64815dce0 2024-04-25T08:58:33.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.799+0000 7fc6467fc640 1 -- 172.21.15.43:0/3409203004 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc638032660 con 0x7fc64815dce0 2024-04-25T08:58:33.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.800+0000 7fc64ddf5640 1 -- 172.21.15.43:0/3409203004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc64815dce0 msgr2=0x7fc64815ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:33.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.800+0000 7fc64ddf5640 1 --2- 172.21.15.43:0/3409203004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc64815dce0 0x7fc64815ef50 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fc6380097a0 tx=0x7fc63802a140 comp rx=0 tx=0).stop 2024-04-25T08:58:33.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.800+0000 7fc64ddf5640 1 -- 172.21.15.43:0/3409203004 shutdown_connections 2024-04-25T08:58:33.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.800+0000 7fc64ddf5640 1 --2- 172.21.15.43:0/3409203004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc64815dce0 0x7fc64815ef50 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:33.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.800+0000 7fc64ddf5640 1 -- 172.21.15.43:0/3409203004 >> 172.21.15.43:0/3409203004 conn(0x7fc64809cd00 msgr2=0x7fc6480d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:33.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.800+0000 7fc64ddf5640 1 -- 172.21.15.43:0/3409203004 shutdown_connections 2024-04-25T08:58:33.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.800+0000 7fc64ddf5640 1 -- 172.21.15.43:0/3409203004 wait complete. 2024-04-25T08:58:33.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.801+0000 7fc64ddf5640 1 Processor -- start 2024-04-25T08:58:33.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.801+0000 7fc64ddf5640 1 -- start start 2024-04-25T08:58:33.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.802+0000 7fc64ddf5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6480ce550 0x7fc6480b6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:33.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.802+0000 7fc64ddf5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc6480d0af0 0x7fc64806c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:33.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.802+0000 7fc64ddf5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc64815dce0 0x7fc64806c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:33.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.802+0000 7fc64ddf5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64815e0c0 con 0x7fc64815dce0 2024-04-25T08:58:33.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.802+0000 7fc64ddf5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64815f6b0 con 0x7fc6480ce550 2024-04-25T08:58:33.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.802+0000 7fc64ddf5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6480ce9c0 con 0x7fc6480d0af0 2024-04-25T08:58:33.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.802+0000 7fc647fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc64815dce0 0x7fc64806c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:33.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.802+0000 7fc647fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc64815dce0 0x7fc64806c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36588/0 (socket says 172.21.15.43:36588) 2024-04-25T08:58:33.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.802+0000 7fc647fff640 1 -- 172.21.15.43:0/4148112326 learned_addr learned my addr 172.21.15.43:0/4148112326 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:33.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.802+0000 7fc646ffd640 1 --2- 172.21.15.43:0/4148112326 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc6480d0af0 0x7fc64806c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:33.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.802+0000 7fc6477fe640 1 --2- 172.21.15.43:0/4148112326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6480ce550 0x7fc6480b6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:33.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.802+0000 7fc647fff640 1 -- 172.21.15.43:0/4148112326 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc6480d0af0 msgr2=0x7fc64806c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:33.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.802+0000 7fc647fff640 1 --2- 172.21.15.43:0/4148112326 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc6480d0af0 0x7fc64806c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:33.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.802+0000 7fc647fff640 1 -- 172.21.15.43:0/4148112326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6480ce550 msgr2=0x7fc6480b6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:33.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.802+0000 7fc647fff640 1 --2- 172.21.15.43:0/4148112326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6480ce550 0x7fc6480b6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:33.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.802+0000 7fc647fff640 1 -- 172.21.15.43:0/4148112326 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6480cce80 con 0x7fc64815dce0 2024-04-25T08:58:33.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.802+0000 7fc646ffd640 1 --2- 172.21.15.43:0/4148112326 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc6480d0af0 0x7fc64806c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:33.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.802+0000 7fc6477fe640 1 --2- 172.21.15.43:0/4148112326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6480ce550 0x7fc6480b6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:33.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.803+0000 7fc647fff640 1 --2- 172.21.15.43:0/4148112326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc64815dce0 0x7fc64806c810 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fc64000cb10 tx=0x7fc64000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:33.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.803+0000 7fc62ffff640 1 -- 172.21.15.43:0/4148112326 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc640017070 con 0x7fc64815dce0 2024-04-25T08:58:33.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.803+0000 7fc62ffff640 1 -- 172.21.15.43:0/4148112326 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc64000ac70 con 0x7fc64815dce0 2024-04-25T08:58:33.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.803+0000 7fc64ddf5640 1 -- 172.21.15.43:0/4148112326 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6480b74b0 con 0x7fc64815dce0 2024-04-25T08:58:33.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.803+0000 7fc62ffff640 1 -- 172.21.15.43:0/4148112326 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc640012620 con 0x7fc64815dce0 2024-04-25T08:58:33.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.803+0000 7fc64ddf5640 1 -- 172.21.15.43:0/4148112326 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6480b8040 con 0x7fc64815dce0 2024-04-25T08:58:33.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.804+0000 7fc64ddf5640 1 -- 172.21.15.43:0/4148112326 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6480680f0 con 0x7fc64815dce0 2024-04-25T08:58:33.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.804+0000 7fc62ffff640 1 -- 172.21.15.43:0/4148112326 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc64001a070 con 0x7fc64815dce0 2024-04-25T08:58:33.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.805+0000 7fc62ffff640 1 --2- 172.21.15.43:0/4148112326 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc63406b4d0 0x7fc63406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:33.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.805+0000 7fc62ffff640 1 -- 172.21.15.43:0/4148112326 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5881+0+0 (secure 0 0 0) 0x7fc640089f70 con 0x7fc64815dce0 2024-04-25T08:58:33.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.805+0000 7fc6477fe640 1 --2- 172.21.15.43:0/4148112326 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc63406b4d0 0x7fc63406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:33.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.805+0000 7fc6477fe640 1 --2- 172.21.15.43:0/4148112326 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc63406b4d0 0x7fc63406d990 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc638034000 tx=0x7fc63802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:33.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.809+0000 7fc62ffff640 1 -- 172.21.15.43:0/4148112326 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc6480b74b0 con 0x7fc64815dce0 2024-04-25T08:58:33.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.990+0000 7fc64ddf5640 1 -- 172.21.15.43:0/4148112326 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc648074ef0 con 0x7fc64815dce0 2024-04-25T08:58:33.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.992+0000 7fc62ffff640 1 -- 172.21.15.43:0/4148112326 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+16888 (secure 0 0 0) 0x7fc6480cce80 con 0x7fc64815dce0 2024-04-25T08:58:33.981 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:58:33.981 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":62,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:58:12.413565+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:57:52.884445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-04-25T08:57:53.565716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6834/3574570511":"2024-04-26T08:58:12.413536+0000","172.21.15.43:6832/3574570511":"2024-04-26T08:58:12.413536+0000","172.21.15.43:6835/2268368384":"2024-04-26T08:58:11.416075+0000","172.21.15.43:6833/2268368384":"2024-04-26T08:58:11.416075+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:58:33.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.994+0000 7fc64ddf5640 1 -- 172.21.15.43:0/4148112326 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc63406b4d0 msgr2=0x7fc63406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:33.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.994+0000 7fc64ddf5640 1 --2- 172.21.15.43:0/4148112326 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc63406b4d0 0x7fc63406d990 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc638034000 tx=0x7fc63802e000 comp rx=0 tx=0).stop 2024-04-25T08:58:33.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.994+0000 7fc64ddf5640 1 -- 172.21.15.43:0/4148112326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc64815dce0 msgr2=0x7fc64806c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:33.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.994+0000 7fc64ddf5640 1 --2- 172.21.15.43:0/4148112326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc64815dce0 0x7fc64806c810 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fc64000cb10 tx=0x7fc64000a060 comp rx=0 tx=0).stop 2024-04-25T08:58:33.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.994+0000 7fc64ddf5640 1 -- 172.21.15.43:0/4148112326 shutdown_connections 2024-04-25T08:58:33.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.994+0000 7fc64ddf5640 1 --2- 172.21.15.43:0/4148112326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc64815dce0 0x7fc64806c810 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:33.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.994+0000 7fc64ddf5640 1 --2- 172.21.15.43:0/4148112326 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc6480d0af0 0x7fc64806c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:33.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.994+0000 7fc64ddf5640 1 --2- 172.21.15.43:0/4148112326 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc63406b4d0 0x7fc63406d990 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:33.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.994+0000 7fc64ddf5640 1 --2- 172.21.15.43:0/4148112326 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6480ce550 0x7fc6480b6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:33.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.994+0000 7fc64ddf5640 1 -- 172.21.15.43:0/4148112326 >> 172.21.15.43:0/4148112326 conn(0x7fc64809cd00 msgr2=0x7fc6480c97c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:33.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.994+0000 7fc64ddf5640 1 -- 172.21.15.43:0/4148112326 shutdown_connections 2024-04-25T08:58:33.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:33.994+0000 7fc64ddf5640 1 -- 172.21.15.43:0/4148112326 wait complete. 2024-04-25T08:58:33.994 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T08:58:34.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.106+0000 7f3b67f70640 1 Processor -- start 2024-04-25T08:58:34.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.106+0000 7f3b67f70640 1 -- start start 2024-04-25T08:58:34.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.107+0000 7f3b67f70640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f3b6015dd00 0x7f3b600d2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:34.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.107+0000 7f3b67f70640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b600d33f0 0x7f3b600ce260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:34.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.107+0000 7f3b67f70640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b600cec30 con 0x7f3b600ce7a0 2024-04-25T08:58:34.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.107+0000 7f3b67f70640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b600d3850 con 0x7f3b6015dd00 2024-04-25T08:58:34.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.107+0000 7f3b67f70640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b600d24e0 con 0x7f3b600d33f0 2024-04-25T08:58:34.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.107+0000 7f3b6650d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f3b600ce7a0 0x7f3b600b5480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54982/0 (socket says 172.21.15.43:54982) 2024-04-25T08:58:34.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.107+0000 7f3b6650d640 1 -- 172.21.15.43:0/3068211040 learned_addr learned my addr 172.21.15.43:0/3068211040 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:34.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.108+0000 7f3b6550b640 1 --2- 172.21.15.43:0/3068211040 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b600d33f0 0x7f3b600ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:34.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.108+0000 7f3b65d0c640 1 --2- 172.21.15.43:0/3068211040 >> v2:172.21.15.193:3300/0 conn(0x7f3b6015dd00 0x7f3b600d2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:34.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.108+0000 7f3b64d0a640 1 -- 172.21.15.43:0/3068211040 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1468102623 0 0) 0x7f3b50001490 con 0x7f3b600ce7a0 2024-04-25T08:58:34.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.108+0000 7f3b64d0a640 1 -- 172.21.15.43:0/3068211040 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b48003520 con 0x7f3b600ce7a0 2024-04-25T08:58:34.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.108+0000 7f3b64d0a640 1 -- 172.21.15.43:0/3068211040 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2154514981 0 0) 0x7f3b48003520 con 0x7f3b600ce7a0 2024-04-25T08:58:34.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.108+0000 7f3b64d0a640 1 -- 172.21.15.43:0/3068211040 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b600d33f0 msgr2=0x7f3b600ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:34.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.108+0000 7f3b64d0a640 1 --2- 172.21.15.43:0/3068211040 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b600d33f0 0x7f3b600ce260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:34.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.108+0000 7f3b64d0a640 1 -- 172.21.15.43:0/3068211040 >> v2:172.21.15.193:3300/0 conn(0x7f3b6015dd00 msgr2=0x7f3b600d2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:34.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.108+0000 7f3b64d0a640 1 --2- 172.21.15.43:0/3068211040 >> v2:172.21.15.193:3300/0 conn(0x7f3b6015dd00 0x7f3b600d2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:34.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.108+0000 7f3b64d0a640 1 -- 172.21.15.43:0/3068211040 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b6015e0e0 con 0x7f3b600ce7a0 2024-04-25T08:58:34.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.108+0000 7f3b64d0a640 1 -- 172.21.15.43:0/3068211040 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f3b50001bc0 con 0x7f3b600ce7a0 2024-04-25T08:58:34.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.108+0000 7f3b6550b640 1 --2- 172.21.15.43:0/3068211040 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b600d33f0 0x7f3b600ce260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:58:34.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.108+0000 7f3b64d0a640 1 -- 172.21.15.43:0/3068211040 >> v1:172.21.15.43:6789/0 conn(0x7f3b600ce7a0 legacy=0x7f3b600b5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:34.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.108+0000 7f3b64d0a640 1 --2- 172.21.15.43:0/3068211040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b480039b0 0x7f3b48003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:34.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.108+0000 7f3b64d0a640 1 --2- 172.21.15.43:0/3068211040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b48004800 0x7f3b48004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:34.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.108+0000 7f3b64d0a640 1 --2- 172.21.15.43:0/3068211040 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b600d33f0 0x7f3b480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:34.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.108+0000 7f3b64d0a640 1 -- 172.21.15.43:0/3068211040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b600d24e0 con 0x7f3b480039b0 2024-04-25T08:58:34.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.108+0000 7f3b64d0a640 1 -- 172.21.15.43:0/3068211040 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4800b9c0 con 0x7f3b48004800 2024-04-25T08:58:34.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.108+0000 7f3b64d0a640 1 -- 172.21.15.43:0/3068211040 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4800bb40 con 0x7f3b600d33f0 2024-04-25T08:58:34.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.108+0000 7f3b65d0c640 1 --2- 172.21.15.43:0/3068211040 >> v2:172.21.15.193:3300/0 conn(0x7f3b6015dd00 0x7f3b600d2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:58:34.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.108+0000 7f3b65d0c640 1 --2- 172.21.15.43:0/3068211040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b480039b0 0x7f3b48003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:34.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.108+0000 7f3b6650d640 1 --2- 172.21.15.43:0/3068211040 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b600d33f0 0x7f3b480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:34.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.108+0000 7f3b6550b640 1 --2- 172.21.15.43:0/3068211040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b48004800 0x7f3b48004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:34.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.108+0000 7f3b65d0c640 1 -- 172.21.15.43:0/3068211040 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b600d33f0 msgr2=0x7f3b480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:34.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.108+0000 7f3b65d0c640 1 --2- 172.21.15.43:0/3068211040 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b600d33f0 0x7f3b480051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:34.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.108+0000 7f3b65d0c640 1 -- 172.21.15.43:0/3068211040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b48004800 msgr2=0x7f3b48004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:34.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.108+0000 7f3b65d0c640 1 --2- 172.21.15.43:0/3068211040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b48004800 0x7f3b48004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:34.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.108+0000 7f3b65d0c640 1 -- 172.21.15.43:0/3068211040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b48003520 con 0x7f3b480039b0 2024-04-25T08:58:34.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.109+0000 7f3b65d0c640 1 --2- 172.21.15.43:0/3068211040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b480039b0 0x7f3b48003dd0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f3b58009c70 tx=0x7f3b48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bf2dedbdc406482 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:34.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.109+0000 7f3b64d0a640 1 -- 172.21.15.43:0/3068211040 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b58009840 con 0x7f3b480039b0 2024-04-25T08:58:34.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.109+0000 7f3b64d0a640 1 -- 172.21.15.43:0/3068211040 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b58004020 con 0x7f3b480039b0 2024-04-25T08:58:34.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.109+0000 7f3b64d0a640 1 -- 172.21.15.43:0/3068211040 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b580043d0 con 0x7f3b480039b0 2024-04-25T08:58:34.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.110+0000 7f3b67f70640 1 -- 172.21.15.43:0/3068211040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b480039b0 msgr2=0x7f3b48003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:34.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.110+0000 7f3b67f70640 1 --2- 172.21.15.43:0/3068211040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b480039b0 0x7f3b48003dd0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f3b58009c70 tx=0x7f3b48005830 comp rx=0 tx=0).stop 2024-04-25T08:58:34.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.110+0000 7f3b6650d640 1 -- 172.21.15.43:0/3068211040 reap_dead start 2024-04-25T08:58:34.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.110+0000 7f3b67f70640 1 -- 172.21.15.43:0/3068211040 shutdown_connections 2024-04-25T08:58:34.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.110+0000 7f3b67f70640 1 -- 172.21.15.43:0/3068211040 >> 172.21.15.43:0/3068211040 conn(0x7f3b6006cbd0 msgr2=0x7f3b6006cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:34.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.110+0000 7f3b67f70640 1 -- 172.21.15.43:0/3068211040 shutdown_connections 2024-04-25T08:58:34.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.110+0000 7f3b67f70640 1 -- 172.21.15.43:0/3068211040 wait complete. 2024-04-25T08:58:34.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.111+0000 7f3b67f70640 1 Processor -- start 2024-04-25T08:58:34.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.111+0000 7f3b67f70640 1 -- start start 2024-04-25T08:58:34.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.112+0000 7f3b67f70640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b480039b0 0x7f3b600adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:34.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.112+0000 7f3b67f70640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b6015b260 0x7f3b6015b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:34.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.112+0000 7f3b67f70640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b600cc970 0x7f3b600ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:34.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.112+0000 7f3b67f70640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b600d1e90 con 0x7f3b600cc970 2024-04-25T08:58:34.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.112+0000 7f3b67f70640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b600d1bb0 con 0x7f3b480039b0 2024-04-25T08:58:34.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.112+0000 7f3b67f70640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b600d2170 con 0x7f3b6015b260 2024-04-25T08:58:34.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.112+0000 7f3b6650d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b600cc970 0x7f3b600ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:34.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.112+0000 7f3b6650d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b600cc970 0x7f3b600ccdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36606/0 (socket says 172.21.15.43:36606) 2024-04-25T08:58:34.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.112+0000 7f3b6650d640 1 -- 172.21.15.43:0/3407648686 learned_addr learned my addr 172.21.15.43:0/3407648686 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:34.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.112+0000 7f3b65d0c640 1 --2- 172.21.15.43:0/3407648686 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b480039b0 0x7f3b600adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:34.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.112+0000 7f3b6550b640 1 --2- 172.21.15.43:0/3407648686 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b6015b260 0x7f3b6015b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:34.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.112+0000 7f3b6650d640 1 -- 172.21.15.43:0/3407648686 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b6015b260 msgr2=0x7f3b6015b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:34.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.112+0000 7f3b6650d640 1 --2- 172.21.15.43:0/3407648686 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b6015b260 0x7f3b6015b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:34.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.112+0000 7f3b6650d640 1 -- 172.21.15.43:0/3407648686 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b480039b0 msgr2=0x7f3b600adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:34.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.112+0000 7f3b6650d640 1 --2- 172.21.15.43:0/3407648686 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b480039b0 0x7f3b600adfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:34.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.112+0000 7f3b6650d640 1 -- 172.21.15.43:0/3407648686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b600cd960 con 0x7f3b600cc970 2024-04-25T08:58:34.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.112+0000 7f3b65d0c640 1 --2- 172.21.15.43:0/3407648686 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b480039b0 0x7f3b600adfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:58:34.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.112+0000 7f3b6650d640 1 --2- 172.21.15.43:0/3407648686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b600cc970 0x7f3b600ccdd0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f3b50002230 tx=0x7f3b5000aec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:34.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.113+0000 7f3b5e7fc640 1 -- 172.21.15.43:0/3407648686 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b50007980 con 0x7f3b600cc970 2024-04-25T08:58:34.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.113+0000 7f3b5e7fc640 1 -- 172.21.15.43:0/3407648686 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b50018070 con 0x7f3b600cc970 2024-04-25T08:58:34.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.113+0000 7f3b67f70640 1 -- 172.21.15.43:0/3407648686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b600cd680 con 0x7f3b600cc970 2024-04-25T08:58:34.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.113+0000 7f3b5e7fc640 1 -- 172.21.15.43:0/3407648686 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b50013630 con 0x7f3b600cc970 2024-04-25T08:58:34.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.113+0000 7f3b67f70640 1 -- 172.21.15.43:0/3407648686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b600ab730 con 0x7f3b600cc970 2024-04-25T08:58:34.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.114+0000 7f3b67f70640 1 -- 172.21.15.43:0/3407648686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b60074ef0 con 0x7f3b600cc970 2024-04-25T08:58:34.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.114+0000 7f3b5e7fc640 1 -- 172.21.15.43:0/3407648686 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3b500137d0 con 0x7f3b600cc970 2024-04-25T08:58:34.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.115+0000 7f3b5e7fc640 1 --2- 172.21.15.43:0/3407648686 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3b4c06b4d0 0x7f3b4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:34.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.115+0000 7f3b5e7fc640 1 -- 172.21.15.43:0/3407648686 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5881+0+0 (secure 0 0 0) 0x7f3b5008a8b0 con 0x7f3b600cc970 2024-04-25T08:58:34.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.115+0000 7f3b65d0c640 1 --2- 172.21.15.43:0/3407648686 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3b4c06b4d0 0x7f3b4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:34.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.115+0000 7f3b65d0c640 1 --2- 172.21.15.43:0/3407648686 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3b4c06b4d0 0x7f3b4c06d990 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3b58032000 tx=0x7f3b58008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:34.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.119+0000 7f3b5e7fc640 1 -- 172.21.15.43:0/3407648686 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3b600cec30 con 0x7f3b600cc970 2024-04-25T08:58:34.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.337+0000 7f3b67f70640 1 -- 172.21.15.43:0/3407648686 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3b600d16e0 con 0x7f3b600cc970 2024-04-25T08:58:34.327 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-04-25T08:58:34.338+0000 7fe4fb198640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T08:58:34.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.339+0000 7f3b5e7fc640 1 -- 172.21.15.43:0/3407648686 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v40) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3b50059680 con 0x7f3b600cc970 2024-04-25T08:58:34.329 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T08:58:34.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.342+0000 7f3b67f70640 1 -- 172.21.15.43:0/3407648686 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3b4c06b4d0 msgr2=0x7f3b4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:34.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.342+0000 7f3b67f70640 1 --2- 172.21.15.43:0/3407648686 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3b4c06b4d0 0x7f3b4c06d990 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3b58032000 tx=0x7f3b58008030 comp rx=0 tx=0).stop 2024-04-25T08:58:34.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.342+0000 7f3b67f70640 1 -- 172.21.15.43:0/3407648686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b600cc970 msgr2=0x7f3b600ccdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:34.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.342+0000 7f3b67f70640 1 --2- 172.21.15.43:0/3407648686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b600cc970 0x7f3b600ccdd0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f3b50002230 tx=0x7f3b5000aec0 comp rx=0 tx=0).stop 2024-04-25T08:58:34.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.342+0000 7f3b67f70640 1 -- 172.21.15.43:0/3407648686 shutdown_connections 2024-04-25T08:58:34.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.342+0000 7f3b67f70640 1 --2- 172.21.15.43:0/3407648686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b600cc970 0x7f3b600ccdd0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:34.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.342+0000 7f3b67f70640 1 --2- 172.21.15.43:0/3407648686 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b6015b260 0x7f3b6015b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:34.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.342+0000 7f3b67f70640 1 --2- 172.21.15.43:0/3407648686 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3b4c06b4d0 0x7f3b4c06d990 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:34.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.342+0000 7f3b67f70640 1 --2- 172.21.15.43:0/3407648686 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b480039b0 0x7f3b600adfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:34.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.342+0000 7f3b67f70640 1 -- 172.21.15.43:0/3407648686 >> 172.21.15.43:0/3407648686 conn(0x7f3b6006cbd0 msgr2=0x7f3b60160600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:34.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.342+0000 7f3b67f70640 1 -- 172.21.15.43:0/3407648686 shutdown_connections 2024-04-25T08:58:34.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.342+0000 7f3b67f70640 1 -- 172.21.15.43:0/3407648686 wait complete. 2024-04-25T08:58:34.333 INFO:tasks.ceph.mds.c.smithi043.stderr: -2259> 2024-04-25T08:58:17.423+0000 7f1586a1e640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:58:34.343 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T08:58:34.361 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-04-25T08:58:34.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.457+0000 7f0358f17640 1 Processor -- start 2024-04-25T08:58:34.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.457+0000 7f0358f17640 1 -- start start 2024-04-25T08:58:34.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.458+0000 7f0358f17640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f035415a960 0x7f03540b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:34.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.458+0000 7f0358f17640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f03540b3ed0 0x7f03540ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:34.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.458+0000 7f0358f17640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03540aec70 0x7f035406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:34.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.458+0000 7f0358f17640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03540af080 con 0x7f035415a960 2024-04-25T08:58:34.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.458+0000 7f0358f17640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f035415b110 con 0x7f03540b3ed0 2024-04-25T08:58:34.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.458+0000 7f0358f17640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f035415b3f0 con 0x7f03540aec70 2024-04-25T08:58:34.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.458+0000 7f035259c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f035415a960 0x7f03540b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:34.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.458+0000 7f035259c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f035415a960 0x7f03540b3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36622/0 (socket says 172.21.15.43:36622) 2024-04-25T08:58:34.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.458+0000 7f035259c640 1 -- 172.21.15.43:0/4132681111 learned_addr learned my addr 172.21.15.43:0/4132681111 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:34.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.458+0000 7f0352d9d640 1 --2- 172.21.15.43:0/4132681111 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03540aec70 0x7f035406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:34.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.458+0000 7f0351d9b640 1 --2- 172.21.15.43:0/4132681111 >> v2:172.21.15.193:3300/0 conn(0x7f03540b3ed0 0x7f03540ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:34.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.458+0000 7f035259c640 1 -- 172.21.15.43:0/4132681111 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03540aec70 msgr2=0x7f035406d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:34.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.458+0000 7f035259c640 1 --2- 172.21.15.43:0/4132681111 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03540aec70 0x7f035406d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:34.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.458+0000 7f035259c640 1 -- 172.21.15.43:0/4132681111 >> v2:172.21.15.193:3300/0 conn(0x7f03540b3ed0 msgr2=0x7f03540ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:34.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.458+0000 7f035259c640 1 --2- 172.21.15.43:0/4132681111 >> v2:172.21.15.193:3300/0 conn(0x7f03540b3ed0 0x7f03540ae730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:34.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.458+0000 7f035259c640 1 -- 172.21.15.43:0/4132681111 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f035415b6d0 con 0x7f035415a960 2024-04-25T08:58:34.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.459+0000 7f035259c640 1 --2- 172.21.15.43:0/4132681111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f035415a960 0x7f03540b3990 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f03540702c0 tx=0x7f034c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ac8f6513436d2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:34.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.459+0000 7f035159a640 1 -- 172.21.15.43:0/4132681111 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034c037070 con 0x7f035415a960 2024-04-25T08:58:34.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.459+0000 7f035159a640 1 -- 172.21.15.43:0/4132681111 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f034c032440 con 0x7f035415a960 2024-04-25T08:58:34.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.459+0000 7f035159a640 1 -- 172.21.15.43:0/4132681111 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034c032810 con 0x7f035415a960 2024-04-25T08:58:34.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.460+0000 7f0358f17640 1 -- 172.21.15.43:0/4132681111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f035415a960 msgr2=0x7f03540b3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:34.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.460+0000 7f0358f17640 1 --2- 172.21.15.43:0/4132681111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f035415a960 0x7f03540b3990 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f03540702c0 tx=0x7f034c02a1e0 comp rx=0 tx=0).stop 2024-04-25T08:58:34.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.460+0000 7f0358f17640 1 -- 172.21.15.43:0/4132681111 shutdown_connections 2024-04-25T08:58:34.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.460+0000 7f0358f17640 1 --2- 172.21.15.43:0/4132681111 >> v2:172.21.15.193:3300/0 conn(0x7f03540b3ed0 0x7f03540ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:34.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.460+0000 7f0358f17640 1 --2- 172.21.15.43:0/4132681111 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03540aec70 0x7f035406d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:34.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.460+0000 7f0358f17640 1 --2- 172.21.15.43:0/4132681111 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f035415a960 0x7f03540b3990 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:34.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.460+0000 7f0358f17640 1 -- 172.21.15.43:0/4132681111 >> 172.21.15.43:0/4132681111 conn(0x7f035406ab30 msgr2=0x7f035406af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:34.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.460+0000 7f0358f17640 1 -- 172.21.15.43:0/4132681111 shutdown_connections 2024-04-25T08:58:34.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.460+0000 7f0358f17640 1 -- 172.21.15.43:0/4132681111 wait complete. 2024-04-25T08:58:34.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.461+0000 7f0358f17640 1 Processor -- start 2024-04-25T08:58:34.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.461+0000 7f0358f17640 1 -- start start 2024-04-25T08:58:34.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.462+0000 7f0358f17640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03540aec70 0x7f03540aa470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:34.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.462+0000 7f0358f17640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03540b3ed0 0x7f03540abc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:34.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.462+0000 7f0358f17640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f035415a960 0x7f03540a9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:34.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.462+0000 7f0358f17640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03540cd940 con 0x7f035415a960 2024-04-25T08:58:34.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.462+0000 7f0358f17640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03540cc440 con 0x7f03540b3ed0 2024-04-25T08:58:34.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.462+0000 7f0358f17640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f035415ad40 con 0x7f03540aec70 2024-04-25T08:58:34.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.462+0000 7f0352d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f035415a960 0x7f03540a9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:34.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.462+0000 7f0352d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f035415a960 0x7f03540a9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36632/0 (socket says 172.21.15.43:36632) 2024-04-25T08:58:34.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.462+0000 7f0352d9d640 1 -- 172.21.15.43:0/155484360 learned_addr learned my addr 172.21.15.43:0/155484360 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:34.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.462+0000 7f035259c640 1 --2- 172.21.15.43:0/155484360 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03540aec70 0x7f03540aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:34.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.462+0000 7f0351d9b640 1 --2- 172.21.15.43:0/155484360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03540b3ed0 0x7f03540abc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:34.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.462+0000 7f0352d9d640 1 -- 172.21.15.43:0/155484360 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03540aec70 msgr2=0x7f03540aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:34.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.462+0000 7f0352d9d640 1 --2- 172.21.15.43:0/155484360 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03540aec70 0x7f03540aa470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:34.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.462+0000 7f0352d9d640 1 -- 172.21.15.43:0/155484360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03540b3ed0 msgr2=0x7f03540abc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:34.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.462+0000 7f0352d9d640 1 --2- 172.21.15.43:0/155484360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03540b3ed0 0x7f03540abc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:34.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.462+0000 7f0352d9d640 1 -- 172.21.15.43:0/155484360 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03540ac5c0 con 0x7f035415a960 2024-04-25T08:58:34.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.463+0000 7f035259c640 1 --2- 172.21.15.43:0/155484360 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03540aec70 0x7f03540aa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:58:34.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.463+0000 7f0351d9b640 1 --2- 172.21.15.43:0/155484360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03540b3ed0 0x7f03540abc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:58:34.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.463+0000 7f0352d9d640 1 --2- 172.21.15.43:0/155484360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f035415a960 0x7f03540a9740 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f0344007e60 tx=0x7f034400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:34.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.463+0000 7f034affd640 1 -- 172.21.15.43:0/155484360 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0344014620 con 0x7f035415a960 2024-04-25T08:58:34.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.463+0000 7f034affd640 1 -- 172.21.15.43:0/155484360 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0344019070 con 0x7f035415a960 2024-04-25T08:58:34.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.463+0000 7f034affd640 1 -- 172.21.15.43:0/155484360 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0344014a20 con 0x7f035415a960 2024-04-25T08:58:34.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.463+0000 7f0358f17640 1 -- 172.21.15.43:0/155484360 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03540ac8a0 con 0x7f035415a960 2024-04-25T08:58:34.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.463+0000 7f0358f17640 1 -- 172.21.15.43:0/155484360 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03540ace60 con 0x7f035415a960 2024-04-25T08:58:34.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.464+0000 7f0358f17640 1 -- 172.21.15.43:0/155484360 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0354072560 con 0x7f035415a960 2024-04-25T08:58:34.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.464+0000 7f034affd640 1 -- 172.21.15.43:0/155484360 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0344014bc0 con 0x7f035415a960 2024-04-25T08:58:34.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.465+0000 7f034affd640 1 --2- 172.21.15.43:0/155484360 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f033406b4d0 0x7f033406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:34.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.465+0000 7f034affd640 1 -- 172.21.15.43:0/155484360 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f034408ab30 con 0x7f035415a960 2024-04-25T08:58:34.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.465+0000 7f035259c640 1 --2- 172.21.15.43:0/155484360 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f033406b4d0 0x7f033406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:34.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.466+0000 7f035259c640 1 --2- 172.21.15.43:0/155484360 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f033406b4d0 0x7f033406d990 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f034c034000 tx=0x7f034c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:34.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.469+0000 7f034affd640 1 -- 172.21.15.43:0/155484360 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0354072560 con 0x7f035415a960 2024-04-25T08:58:34.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:34.686+0000 7f0358f17640 1 -- 172.21.15.43:0/155484360 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0354074f60 con 0x7f035415a960 2024-04-25T08:58:35.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.341+0000 7f034affd640 1 -- 172.21.15.43:0/155484360 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v41) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0344059940 con 0x7f035415a960 2024-04-25T08:58:35.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.343+0000 7f0358f17640 1 -- 172.21.15.43:0/155484360 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f033406b4d0 msgr2=0x7f033406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:35.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.343+0000 7f0358f17640 1 --2- 172.21.15.43:0/155484360 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f033406b4d0 0x7f033406d990 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f034c034000 tx=0x7f034c02e000 comp rx=0 tx=0).stop 2024-04-25T08:58:35.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.343+0000 7f0358f17640 1 -- 172.21.15.43:0/155484360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f035415a960 msgr2=0x7f03540a9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:35.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.343+0000 7f0358f17640 1 --2- 172.21.15.43:0/155484360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f035415a960 0x7f03540a9740 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f0344007e60 tx=0x7f034400e4b0 comp rx=0 tx=0).stop 2024-04-25T08:58:35.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.343+0000 7f0358f17640 1 -- 172.21.15.43:0/155484360 shutdown_connections 2024-04-25T08:58:35.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.343+0000 7f0358f17640 1 --2- 172.21.15.43:0/155484360 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f035415a960 0x7f03540a9740 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:35.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.343+0000 7f0358f17640 1 --2- 172.21.15.43:0/155484360 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f033406b4d0 0x7f033406d990 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:35.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.343+0000 7f0358f17640 1 --2- 172.21.15.43:0/155484360 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03540b3ed0 0x7f03540abc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:35.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.343+0000 7f0358f17640 1 --2- 172.21.15.43:0/155484360 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03540aec70 0x7f03540aa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:35.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.343+0000 7f0358f17640 1 -- 172.21.15.43:0/155484360 >> 172.21.15.43:0/155484360 conn(0x7f035406ab30 msgr2=0x7f0354159840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:35.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.344+0000 7f0358f17640 1 -- 172.21.15.43:0/155484360 shutdown_connections 2024-04-25T08:58:35.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.344+0000 7f0358f17640 1 -- 172.21.15.43:0/155484360 wait complete. 2024-04-25T08:58:35.346 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T08:58:35.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.508+0000 7f28df44a640 1 Processor -- start 2024-04-25T08:58:35.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.508+0000 7f28df44a640 1 -- start start 2024-04-25T08:58:35.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.509+0000 7f28df44a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f28d815def0 0x7f28d80d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:35.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.509+0000 7f28df44a640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f28d80d4670 0x7f28d80d03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:35.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.509+0000 7f28df44a640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28d80d2370 con 0x7f28d80d0930 2024-04-25T08:58:35.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.509+0000 7f28df44a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28d80b37b0 con 0x7f28d815def0 2024-04-25T08:58:35.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.509+0000 7f28df44a640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28d80d2650 con 0x7f28d80d4670 2024-04-25T08:58:35.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.509+0000 7f28dd1e6640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f28d815def0 0x7f28d80d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:35.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.509+0000 7f28dd1e6640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f28d815def0 0x7f28d80d4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36642/0 (socket says 172.21.15.43:36642) 2024-04-25T08:58:35.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.509+0000 7f28dd1e6640 1 -- 172.21.15.43:0/3708178749 learned_addr learned my addr 172.21.15.43:0/3708178749 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:35.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.509+0000 7f28dc9e5640 1 --2- 172.21.15.43:0/3708178749 >> v2:172.21.15.193:3300/0 conn(0x7f28d80d4670 0x7f28d80d03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:35.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.509+0000 7f28dd1e6640 1 -- 172.21.15.43:0/3708178749 >> v2:172.21.15.193:3300/0 conn(0x7f28d80d4670 msgr2=0x7f28d80d03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:35.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.509+0000 7f28dd1e6640 1 --2- 172.21.15.43:0/3708178749 >> v2:172.21.15.193:3300/0 conn(0x7f28d80d4670 0x7f28d80d03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:35.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.509+0000 7f28dd1e6640 1 -- 172.21.15.43:0/3708178749 >> v1:172.21.15.193:6789/0 conn(0x7f28d80d0930 legacy=0x7f28d80ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:35.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.509+0000 7f28dd1e6640 1 -- 172.21.15.43:0/3708178749 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d80b3440 con 0x7f28d815def0 2024-04-25T08:58:35.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.510+0000 7f28dd1e6640 1 --2- 172.21.15.43:0/3708178749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28d815def0 0x7f28d80d4130 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f28d80cfbe0 tx=0x7f28d002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dabdb5b14ed3caac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:35.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.510+0000 7f28d7fff640 1 -- 172.21.15.43:0/3708178749 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d0037070 con 0x7f28d815def0 2024-04-25T08:58:35.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.510+0000 7f28d7fff640 1 -- 172.21.15.43:0/3708178749 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28d0032440 con 0x7f28d815def0 2024-04-25T08:58:35.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.510+0000 7f28d7fff640 1 -- 172.21.15.43:0/3708178749 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d0032810 con 0x7f28d815def0 2024-04-25T08:58:35.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.511+0000 7f28df44a640 1 -- 172.21.15.43:0/3708178749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28d815def0 msgr2=0x7f28d80d4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:35.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.511+0000 7f28df44a640 1 --2- 172.21.15.43:0/3708178749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28d815def0 0x7f28d80d4130 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f28d80cfbe0 tx=0x7f28d002a1e0 comp rx=0 tx=0).stop 2024-04-25T08:58:35.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.511+0000 7f28df44a640 1 -- 172.21.15.43:0/3708178749 shutdown_connections 2024-04-25T08:58:35.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.511+0000 7f28df44a640 1 --2- 172.21.15.43:0/3708178749 >> v2:172.21.15.193:3300/0 conn(0x7f28d80d4670 0x7f28d80d03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:35.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.511+0000 7f28df44a640 1 --2- 172.21.15.43:0/3708178749 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28d815def0 0x7f28d80d4130 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:35.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.511+0000 7f28df44a640 1 -- 172.21.15.43:0/3708178749 >> 172.21.15.43:0/3708178749 conn(0x7f28d809cd60 msgr2=0x7f28d80d3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:35.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.511+0000 7f28df44a640 1 -- 172.21.15.43:0/3708178749 shutdown_connections 2024-04-25T08:58:35.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.511+0000 7f28df44a640 1 -- 172.21.15.43:0/3708178749 wait complete. 2024-04-25T08:58:35.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.512+0000 7f28df44a640 1 Processor -- start 2024-04-25T08:58:35.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.512+0000 7f28df44a640 1 -- start start 2024-04-25T08:58:35.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.513+0000 7f28df44a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28d80d0930 0x7f28d8160d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:35.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.513+0000 7f28df44a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28d80d4670 0x7f28d80b6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:35.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.513+0000 7f28df44a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f28d815def0 0x7f28d81593c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:35.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.513+0000 7f28df44a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d80d2150 con 0x7f28d80d0930 2024-04-25T08:58:35.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.513+0000 7f28df44a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28d80d1fd0 con 0x7f28d80d4670 2024-04-25T08:58:35.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.513+0000 7f28df44a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28d80b4c50 con 0x7f28d815def0 2024-04-25T08:58:35.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.513+0000 7f28dd1e6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28d80d0930 0x7f28d8160d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:35.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.513+0000 7f28dd1e6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28d80d0930 0x7f28d8160d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36648/0 (socket says 172.21.15.43:36648) 2024-04-25T08:58:35.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.513+0000 7f28dd1e6640 1 -- 172.21.15.43:0/226952508 learned_addr learned my addr 172.21.15.43:0/226952508 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:35.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.513+0000 7f28dd1e6640 1 -- 172.21.15.43:0/226952508 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f28d815def0 msgr2=0x7f28d81593c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:35.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.513+0000 7f28dd1e6640 1 --2- 172.21.15.43:0/226952508 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f28d815def0 0x7f28d81593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:35.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.513+0000 7f28dd1e6640 1 -- 172.21.15.43:0/226952508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28d80d4670 msgr2=0x7f28d80b6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:35.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.513+0000 7f28dd1e6640 1 --2- 172.21.15.43:0/226952508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28d80d4670 0x7f28d80b6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:35.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.513+0000 7f28dd1e6640 1 -- 172.21.15.43:0/226952508 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28d80b7760 con 0x7f28d80d0930 2024-04-25T08:58:35.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.513+0000 7f28dd1e6640 1 --2- 172.21.15.43:0/226952508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28d80d0930 0x7f28d8160d50 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f28d002a220 tx=0x7f28d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:35.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.513+0000 7f28d5ffb640 1 -- 172.21.15.43:0/226952508 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d0037040 con 0x7f28d80d0930 2024-04-25T08:58:35.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.513+0000 7f28d5ffb640 1 -- 172.21.15.43:0/226952508 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28d0042070 con 0x7f28d80d0930 2024-04-25T08:58:35.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.513+0000 7f28df44a640 1 -- 172.21.15.43:0/226952508 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28d8154c90 con 0x7f28d80d0930 2024-04-25T08:58:35.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.513+0000 7f28d5ffb640 1 -- 172.21.15.43:0/226952508 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28d0030880 con 0x7f28d80d0930 2024-04-25T08:58:35.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.514+0000 7f28df44a640 1 -- 172.21.15.43:0/226952508 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28d8155250 con 0x7f28d80d0930 2024-04-25T08:58:35.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.515+0000 7f28df44a640 1 -- 172.21.15.43:0/226952508 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28d8068a40 con 0x7f28d80d0930 2024-04-25T08:58:35.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.515+0000 7f28d5ffb640 1 -- 172.21.15.43:0/226952508 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f28d0030a20 con 0x7f28d80d0930 2024-04-25T08:58:35.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.515+0000 7f28d5ffb640 1 --2- 172.21.15.43:0/226952508 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f28c406b500 0x7f28c406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:35.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.515+0000 7f28d5ffb640 1 -- 172.21.15.43:0/226952508 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f28d00aab10 con 0x7f28d80d0930 2024-04-25T08:58:35.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.516+0000 7f28dc9e5640 1 --2- 172.21.15.43:0/226952508 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f28c406b500 0x7f28c406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:35.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.516+0000 7f28dc9e5640 1 --2- 172.21.15.43:0/226952508 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f28c406b500 0x7f28c406d9c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f28cc006a90 tx=0x7f28cc006a20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:35.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.520+0000 7f28d5ffb640 1 -- 172.21.15.43:0/226952508 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f28d0079890 con 0x7f28d80d0930 2024-04-25T08:58:35.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:35.700+0000 7f28df44a640 1 -- 172.21.15.43:0/226952508 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f28d80cdd30 con 0x7f28d80d0930 2024-04-25T08:58:36.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.491+0000 7f28d5ffb640 1 -- 172.21.15.43:0/226952508 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v64) v1 ==== 181+0+0 (secure 0 0 0) 0x7f28d0079190 con 0x7f28d80d0930 2024-04-25T08:58:36.480 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' removed 2024-04-25T08:58:36.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.495+0000 7f28df44a640 1 -- 172.21.15.43:0/226952508 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f28c406b500 msgr2=0x7f28c406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:36.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.495+0000 7f28df44a640 1 --2- 172.21.15.43:0/226952508 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f28c406b500 0x7f28c406d9c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f28cc006a90 tx=0x7f28cc006a20 comp rx=0 tx=0).stop 2024-04-25T08:58:36.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.495+0000 7f28df44a640 1 -- 172.21.15.43:0/226952508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28d80d0930 msgr2=0x7f28d8160d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:36.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.495+0000 7f28df44a640 1 --2- 172.21.15.43:0/226952508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28d80d0930 0x7f28d8160d50 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f28d002a220 tx=0x7f28d00046c0 comp rx=0 tx=0).stop 2024-04-25T08:58:36.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.495+0000 7f28df44a640 1 -- 172.21.15.43:0/226952508 shutdown_connections 2024-04-25T08:58:36.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.495+0000 7f28df44a640 1 --2- 172.21.15.43:0/226952508 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f28d815def0 0x7f28d81593c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:36.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.495+0000 7f28df44a640 1 --2- 172.21.15.43:0/226952508 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f28c406b500 0x7f28c406d9c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:36.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.495+0000 7f28df44a640 1 --2- 172.21.15.43:0/226952508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f28d80d4670 0x7f28d80b6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:36.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.495+0000 7f28df44a640 1 --2- 172.21.15.43:0/226952508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f28d80d0930 0x7f28d8160d50 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:36.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.495+0000 7f28df44a640 1 -- 172.21.15.43:0/226952508 >> 172.21.15.43:0/226952508 conn(0x7f28d809cd60 msgr2=0x7f28d80b4fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:36.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.495+0000 7f28df44a640 1 -- 172.21.15.43:0/226952508 shutdown_connections 2024-04-25T08:58:36.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.495+0000 7f28df44a640 1 -- 172.21.15.43:0/226952508 wait complete. 2024-04-25T08:58:36.499 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T08:58:36.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.621+0000 7f607630a640 1 Processor -- start 2024-04-25T08:58:36.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.621+0000 7f607630a640 1 -- start start 2024-04-25T08:58:36.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.622+0000 7f607630a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f607015b6c0 0x7f607015baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:36.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.622+0000 7f607630a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f60700b4ea0 0x7f60700d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:36.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.622+0000 7f607630a640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60700b37a0 con 0x7f607015e410 2024-04-25T08:58:36.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.622+0000 7f607630a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f607015fce0 con 0x7f60700b4ea0 2024-04-25T08:58:36.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.622+0000 7f607630a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f607015e870 con 0x7f607015b6c0 2024-04-25T08:58:36.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.622+0000 7f60748a7640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f60700b4ea0 0x7f60700d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:36.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.622+0000 7f60748a7640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f60700b4ea0 0x7f60700d3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36650/0 (socket says 172.21.15.43:36650) 2024-04-25T08:58:36.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.622+0000 7f60748a7640 1 -- 172.21.15.43:0/3862437807 learned_addr learned my addr 172.21.15.43:0/3862437807 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:36.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.622+0000 7f606ffff640 1 --2- 172.21.15.43:0/3862437807 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f607015b6c0 0x7f607015baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:36.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.623+0000 7f60748a7640 1 -- 172.21.15.43:0/3862437807 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f607015b6c0 msgr2=0x7f607015baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:36.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.623+0000 7f60748a7640 1 --2- 172.21.15.43:0/3862437807 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f607015b6c0 0x7f607015baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:36.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.623+0000 7f60748a7640 1 -- 172.21.15.43:0/3862437807 >> v1:172.21.15.193:6789/0 conn(0x7f607015e410 legacy=0x7f607015f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:36.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.623+0000 7f60748a7640 1 -- 172.21.15.43:0/3862437807 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f607015eab0 con 0x7f60700b4ea0 2024-04-25T08:58:36.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.623+0000 7f60748a7640 1 --2- 172.21.15.43:0/3862437807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60700b4ea0 0x7f60700d3760 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f6068009e50 tx=0x7f606802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b325b97716d24659 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:36.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.623+0000 7f606effd640 1 -- 172.21.15.43:0/3862437807 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6068036070 con 0x7f60700b4ea0 2024-04-25T08:58:36.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.623+0000 7f606effd640 1 -- 172.21.15.43:0/3862437807 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6068031600 con 0x7f60700b4ea0 2024-04-25T08:58:36.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.623+0000 7f606effd640 1 -- 172.21.15.43:0/3862437807 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60680319d0 con 0x7f60700b4ea0 2024-04-25T08:58:36.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.624+0000 7f607630a640 1 -- 172.21.15.43:0/3862437807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60700b4ea0 msgr2=0x7f60700d3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:36.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.624+0000 7f607630a640 1 --2- 172.21.15.43:0/3862437807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60700b4ea0 0x7f60700d3760 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f6068009e50 tx=0x7f606802a710 comp rx=0 tx=0).stop 2024-04-25T08:58:36.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.624+0000 7f607630a640 1 -- 172.21.15.43:0/3862437807 shutdown_connections 2024-04-25T08:58:36.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.624+0000 7f607630a640 1 --2- 172.21.15.43:0/3862437807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60700b4ea0 0x7f60700d3760 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:36.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.624+0000 7f607630a640 1 --2- 172.21.15.43:0/3862437807 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f607015b6c0 0x7f607015baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:36.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.624+0000 7f607630a640 1 -- 172.21.15.43:0/3862437807 >> 172.21.15.43:0/3862437807 conn(0x7f607008d460 msgr2=0x7f607009cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:36.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.624+0000 7f607630a640 1 -- 172.21.15.43:0/3862437807 shutdown_connections 2024-04-25T08:58:36.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.624+0000 7f607630a640 1 -- 172.21.15.43:0/3862437807 wait complete. 2024-04-25T08:58:36.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.625+0000 7f607630a640 1 Processor -- start 2024-04-25T08:58:36.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.626+0000 7f607630a640 1 -- start start 2024-04-25T08:58:36.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.626+0000 7f607630a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f60700b4ea0 0x7f60700c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:36.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.626+0000 7f607630a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f607015b6c0 0x7f6070158a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:36.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.626+0000 7f607630a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f607015e410 0x7f60700d0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:36.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.626+0000 7f607630a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60700b5310 con 0x7f607015b6c0 2024-04-25T08:58:36.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.626+0000 7f607630a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f607015f350 con 0x7f607015e410 2024-04-25T08:58:36.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.626+0000 7f607630a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60700d3ca0 con 0x7f60700b4ea0 2024-04-25T08:58:36.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.626+0000 7f606f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f607015b6c0 0x7f6070158a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:36.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.626+0000 7f606f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f607015b6c0 0x7f6070158a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36654/0 (socket says 172.21.15.43:36654) 2024-04-25T08:58:36.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.626+0000 7f606f7fe640 1 -- 172.21.15.43:0/654669028 learned_addr learned my addr 172.21.15.43:0/654669028 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:36.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.626+0000 7f606ffff640 1 --2- 172.21.15.43:0/654669028 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f60700b4ea0 0x7f60700c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:36.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.626+0000 7f60748a7640 1 --2- 172.21.15.43:0/654669028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f607015e410 0x7f60700d0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:36.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.627+0000 7f606f7fe640 1 -- 172.21.15.43:0/654669028 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f60700b4ea0 msgr2=0x7f60700c8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:36.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.627+0000 7f606f7fe640 1 --2- 172.21.15.43:0/654669028 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f60700b4ea0 0x7f60700c8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:36.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.627+0000 7f606f7fe640 1 -- 172.21.15.43:0/654669028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f607015e410 msgr2=0x7f60700d0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:36.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.627+0000 7f606f7fe640 1 --2- 172.21.15.43:0/654669028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f607015e410 0x7f60700d0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:36.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.627+0000 7f606f7fe640 1 -- 172.21.15.43:0/654669028 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60700c9fd0 con 0x7f607015b6c0 2024-04-25T08:58:36.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.627+0000 7f606f7fe640 1 --2- 172.21.15.43:0/654669028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f607015b6c0 0x7f6070158a80 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f606400c340 tx=0x7f606400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:36.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.627+0000 7f606cff9640 1 -- 172.21.15.43:0/654669028 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6064017070 con 0x7f607015b6c0 2024-04-25T08:58:36.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.627+0000 7f606cff9640 1 -- 172.21.15.43:0/654669028 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60640052d0 con 0x7f607015b6c0 2024-04-25T08:58:36.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.627+0000 7f607630a640 1 -- 172.21.15.43:0/654669028 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60700ca2b0 con 0x7f607015b6c0 2024-04-25T08:58:36.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.627+0000 7f606cff9640 1 -- 172.21.15.43:0/654669028 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6064003850 con 0x7f607015b6c0 2024-04-25T08:58:36.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.627+0000 7f607630a640 1 -- 172.21.15.43:0/654669028 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60700c8a80 con 0x7f607015b6c0 2024-04-25T08:58:36.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.628+0000 7f607630a640 1 -- 172.21.15.43:0/654669028 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6070068a40 con 0x7f607015b6c0 2024-04-25T08:58:36.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.629+0000 7f606cff9640 1 -- 172.21.15.43:0/654669028 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f606401a070 con 0x7f607015b6c0 2024-04-25T08:58:36.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.629+0000 7f606cff9640 1 --2- 172.21.15.43:0/654669028 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f605c06b500 0x7f605c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:36.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.629+0000 7f606cff9640 1 -- 172.21.15.43:0/654669028 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5529+0+0 (secure 0 0 0) 0x7f60640893b0 con 0x7f607015b6c0 2024-04-25T08:58:36.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.629+0000 7f606ffff640 1 --2- 172.21.15.43:0/654669028 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f605c06b500 0x7f605c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:36.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.630+0000 7f606ffff640 1 --2- 172.21.15.43:0/654669028 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f605c06b500 0x7f605c06d9c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6060006fd0 tx=0x7f6060008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:36.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.633+0000 7f606cff9640 1 -- 172.21.15.43:0/654669028 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6064057c90 con 0x7f607015b6c0 2024-04-25T08:58:36.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:36.815+0000 7f607630a640 1 -- 172.21.15.43:0/654669028 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f60700d1130 con 0x7f607015b6c0 2024-04-25T08:58:37.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.492+0000 7f606cff9640 1 -- 172.21.15.43:0/654669028 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v65) v1 ==== 169+0+0 (secure 0 0 0) 0x7f606405bd30 con 0x7f607015b6c0 2024-04-25T08:58:37.482 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' removed 2024-04-25T08:58:37.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.496+0000 7f607630a640 1 -- 172.21.15.43:0/654669028 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f605c06b500 msgr2=0x7f605c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:37.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.496+0000 7f607630a640 1 --2- 172.21.15.43:0/654669028 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f605c06b500 0x7f605c06d9c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6060006fd0 tx=0x7f6060008040 comp rx=0 tx=0).stop 2024-04-25T08:58:37.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.496+0000 7f607630a640 1 -- 172.21.15.43:0/654669028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f607015b6c0 msgr2=0x7f6070158a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:37.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.496+0000 7f607630a640 1 --2- 172.21.15.43:0/654669028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f607015b6c0 0x7f6070158a80 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f606400c340 tx=0x7f606400c800 comp rx=0 tx=0).stop 2024-04-25T08:58:37.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.496+0000 7f607630a640 1 -- 172.21.15.43:0/654669028 shutdown_connections 2024-04-25T08:58:37.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.496+0000 7f607630a640 1 --2- 172.21.15.43:0/654669028 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f605c06b500 0x7f605c06d9c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.496+0000 7f607630a640 1 --2- 172.21.15.43:0/654669028 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f607015e410 0x7f60700d0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.496+0000 7f607630a640 1 --2- 172.21.15.43:0/654669028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f607015b6c0 0x7f6070158a80 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.496+0000 7f607630a640 1 --2- 172.21.15.43:0/654669028 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f60700b4ea0 0x7f60700c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.496+0000 7f607630a640 1 -- 172.21.15.43:0/654669028 >> 172.21.15.43:0/654669028 conn(0x7f607008d460 msgr2=0x7f607015af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:37.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.497+0000 7f607630a640 1 -- 172.21.15.43:0/654669028 shutdown_connections 2024-04-25T08:58:37.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.497+0000 7f607630a640 1 -- 172.21.15.43:0/654669028 wait complete. 2024-04-25T08:58:37.502 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T08:58:37.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.621+0000 7fcba214c640 1 Processor -- start 2024-04-25T08:58:37.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.621+0000 7fcba214c640 1 -- start start 2024-04-25T08:58:37.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.622+0000 7fcba214c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcb9c161310 0x7fcb9c0d4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:37.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.622+0000 7fcba214c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcb9c0d0120 0x7fcb9c0d0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:37.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.622+0000 7fcba214c640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb9c0d0f30 con 0x7fcb9c0d0aa0 2024-04-25T08:58:37.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.622+0000 7fcba214c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb9c0d4870 con 0x7fcb9c0d0120 2024-04-25T08:58:37.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.622+0000 7fcba214c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9c0d45c0 con 0x7fcb9c161310 2024-04-25T08:58:37.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.622+0000 7fcb9affd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcb9c0d0120 0x7fcb9c0d0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:37.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.622+0000 7fcb9affd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcb9c0d0120 0x7fcb9c0d0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36660/0 (socket says 172.21.15.43:36660) 2024-04-25T08:58:37.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.622+0000 7fcb9affd640 1 -- 172.21.15.43:0/2303466034 learned_addr learned my addr 172.21.15.43:0/2303466034 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:37.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.623+0000 7fcb9b7fe640 1 --2- 172.21.15.43:0/2303466034 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcb9c161310 0x7fcb9c0d4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:37.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.623+0000 7fcb9affd640 1 -- 172.21.15.43:0/2303466034 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcb9c161310 msgr2=0x7fcb9c0d4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:37.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.623+0000 7fcb9affd640 1 --2- 172.21.15.43:0/2303466034 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcb9c161310 0x7fcb9c0d4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.623+0000 7fcb9affd640 1 -- 172.21.15.43:0/2303466034 >> v1:172.21.15.193:6789/0 conn(0x7fcb9c0d0aa0 legacy=0x7fcb9c0d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:37.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.623+0000 7fcb9affd640 1 -- 172.21.15.43:0/2303466034 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb9c0cf030 con 0x7fcb9c0d0120 2024-04-25T08:58:37.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.623+0000 7fcb9affd640 1 --2- 172.21.15.43:0/2303466034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb9c0d0120 0x7fcb9c0d0560 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fcb90009870 tx=0x7fcb9002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=975b4c63f87da7b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:37.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.624+0000 7fcb9a7fc640 1 -- 172.21.15.43:0/2303466034 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb90037070 con 0x7fcb9c0d0120 2024-04-25T08:58:37.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.624+0000 7fcb9a7fc640 1 -- 172.21.15.43:0/2303466034 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb90032440 con 0x7fcb9c0d0120 2024-04-25T08:58:37.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.624+0000 7fcb9a7fc640 1 -- 172.21.15.43:0/2303466034 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb90032810 con 0x7fcb9c0d0120 2024-04-25T08:58:37.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.624+0000 7fcba214c640 1 -- 172.21.15.43:0/2303466034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb9c0d0120 msgr2=0x7fcb9c0d0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:37.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.624+0000 7fcba214c640 1 --2- 172.21.15.43:0/2303466034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb9c0d0120 0x7fcb9c0d0560 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fcb90009870 tx=0x7fcb9002a270 comp rx=0 tx=0).stop 2024-04-25T08:58:37.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.624+0000 7fcba214c640 1 -- 172.21.15.43:0/2303466034 shutdown_connections 2024-04-25T08:58:37.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.624+0000 7fcba214c640 1 --2- 172.21.15.43:0/2303466034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb9c0d0120 0x7fcb9c0d0560 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.624+0000 7fcba214c640 1 --2- 172.21.15.43:0/2303466034 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcb9c161310 0x7fcb9c0d4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.624+0000 7fcba214c640 1 -- 172.21.15.43:0/2303466034 >> 172.21.15.43:0/2303466034 conn(0x7fcb9c09cd20 msgr2=0x7fcb9c0d3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:37.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.624+0000 7fcba214c640 1 -- 172.21.15.43:0/2303466034 shutdown_connections 2024-04-25T08:58:37.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.624+0000 7fcba214c640 1 -- 172.21.15.43:0/2303466034 wait complete. 2024-04-25T08:58:37.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.626+0000 7fcba214c640 1 Processor -- start 2024-04-25T08:58:37.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.626+0000 7fcba214c640 1 -- start start 2024-04-25T08:58:37.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.626+0000 7fcba214c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb9c0d0120 0x7fcb9c149800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:37.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.626+0000 7fcba214c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb9c0d0aa0 0x7fcb9c15a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:37.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.627+0000 7fcba214c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcb9c161310 0x7fcb9c15aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:37.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.627+0000 7fcba214c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9c1616f0 con 0x7fcb9c0d0aa0 2024-04-25T08:58:37.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.627+0000 7fcba214c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9c0cf5f0 con 0x7fcb9c0d0120 2024-04-25T08:58:37.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.627+0000 7fcba214c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9c0c8870 con 0x7fcb9c161310 2024-04-25T08:58:37.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.627+0000 7fcb9affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb9c0d0aa0 0x7fcb9c15a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:37.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.627+0000 7fcb9affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb9c0d0aa0 0x7fcb9c15a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36676/0 (socket says 172.21.15.43:36676) 2024-04-25T08:58:37.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.627+0000 7fcb9affd640 1 -- 172.21.15.43:0/2737343126 learned_addr learned my addr 172.21.15.43:0/2737343126 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:37.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.627+0000 7fcb9b7fe640 1 --2- 172.21.15.43:0/2737343126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb9c0d0120 0x7fcb9c149800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:37.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.627+0000 7fcb9bfff640 1 --2- 172.21.15.43:0/2737343126 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcb9c161310 0x7fcb9c15aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:37.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.627+0000 7fcb9affd640 1 -- 172.21.15.43:0/2737343126 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcb9c161310 msgr2=0x7fcb9c15aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:37.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.627+0000 7fcb9affd640 1 --2- 172.21.15.43:0/2737343126 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcb9c161310 0x7fcb9c15aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.627+0000 7fcb9affd640 1 -- 172.21.15.43:0/2737343126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb9c0d0120 msgr2=0x7fcb9c149800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:37.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.627+0000 7fcb9affd640 1 --2- 172.21.15.43:0/2737343126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb9c0d0120 0x7fcb9c149800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.627+0000 7fcb9affd640 1 -- 172.21.15.43:0/2737343126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb9c15c4a0 con 0x7fcb9c0d0aa0 2024-04-25T08:58:37.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.627+0000 7fcb9affd640 1 --2- 172.21.15.43:0/2737343126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb9c0d0aa0 0x7fcb9c15a4f0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fcb90000c00 tx=0x7fcb900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:37.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.627+0000 7fcb83fff640 1 -- 172.21.15.43:0/2737343126 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb90041070 con 0x7fcb9c0d0aa0 2024-04-25T08:58:37.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.627+0000 7fcb83fff640 1 -- 172.21.15.43:0/2737343126 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb9003c870 con 0x7fcb9c0d0aa0 2024-04-25T08:58:37.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.627+0000 7fcba214c640 1 -- 172.21.15.43:0/2737343126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb9c15c740 con 0x7fcb9c0d0aa0 2024-04-25T08:58:37.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.627+0000 7fcb83fff640 1 -- 172.21.15.43:0/2737343126 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb90037040 con 0x7fcb9c0d0aa0 2024-04-25T08:58:37.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.628+0000 7fcba214c640 1 -- 172.21.15.43:0/2737343126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb9c15cd00 con 0x7fcb9c0d0aa0 2024-04-25T08:58:37.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.629+0000 7fcba214c640 1 -- 172.21.15.43:0/2737343126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb9c074f60 con 0x7fcb9c0d0aa0 2024-04-25T08:58:37.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.629+0000 7fcb83fff640 1 -- 172.21.15.43:0/2737343126 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcb90044070 con 0x7fcb9c0d0aa0 2024-04-25T08:58:37.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.629+0000 7fcb83fff640 1 --2- 172.21.15.43:0/2737343126 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcb8806b500 0x7fcb8806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:37.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.630+0000 7fcb83fff640 1 -- 172.21.15.43:0/2737343126 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fcb900aab70 con 0x7fcb9c0d0aa0 2024-04-25T08:58:37.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.630+0000 7fcb9b7fe640 1 --2- 172.21.15.43:0/2737343126 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcb8806b500 0x7fcb8806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:37.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.631+0000 7fcb9b7fe640 1 --2- 172.21.15.43:0/2737343126 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcb8806b500 0x7fcb8806d9c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c0079a0 tx=0x7fcb8c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:37.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.636+0000 7fcb83fff640 1 -- 172.21.15.43:0/2737343126 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcb90079670 con 0x7fcb9c0d0aa0 2024-04-25T08:58:37.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.817+0000 7fcba214c640 1 -- 172.21.15.43:0/2737343126 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fcb9c15d7c0 con 0x7fcb9c0d0aa0 2024-04-25T08:58:37.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.820+0000 7fcb83fff640 1 -- 172.21.15.43:0/2737343126 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcb9007d710 con 0x7fcb9c0d0aa0 2024-04-25T08:58:37.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.822+0000 7fcba214c640 1 -- 172.21.15.43:0/2737343126 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcb8806b500 msgr2=0x7fcb8806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:37.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.822+0000 7fcba214c640 1 --2- 172.21.15.43:0/2737343126 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcb8806b500 0x7fcb8806d9c0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c0079a0 tx=0x7fcb8c008040 comp rx=0 tx=0).stop 2024-04-25T08:58:37.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.822+0000 7fcba214c640 1 -- 172.21.15.43:0/2737343126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb9c0d0aa0 msgr2=0x7fcb9c15a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:37.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.822+0000 7fcba214c640 1 --2- 172.21.15.43:0/2737343126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb9c0d0aa0 0x7fcb9c15a4f0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fcb90000c00 tx=0x7fcb900046c0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.822+0000 7fcba214c640 1 -- 172.21.15.43:0/2737343126 shutdown_connections 2024-04-25T08:58:37.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.822+0000 7fcba214c640 1 --2- 172.21.15.43:0/2737343126 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcb9c161310 0x7fcb9c15aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.822+0000 7fcba214c640 1 --2- 172.21.15.43:0/2737343126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcb9c0d0aa0 0x7fcb9c15a4f0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.822+0000 7fcba214c640 1 --2- 172.21.15.43:0/2737343126 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcb8806b500 0x7fcb8806d9c0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.822+0000 7fcba214c640 1 --2- 172.21.15.43:0/2737343126 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcb9c0d0120 0x7fcb9c149800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.822+0000 7fcba214c640 1 -- 172.21.15.43:0/2737343126 >> 172.21.15.43:0/2737343126 conn(0x7fcb9c09cd20 msgr2=0x7fcb9c0c9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:37.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.822+0000 7fcba214c640 1 -- 172.21.15.43:0/2737343126 shutdown_connections 2024-04-25T08:58:37.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.822+0000 7fcba214c640 1 -- 172.21.15.43:0/2737343126 wait complete. 2024-04-25T08:58:37.823 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair' 2024-04-25T08:58:37.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.934+0000 7f8ec0b28640 1 Processor -- start 2024-04-25T08:58:37.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.934+0000 7f8ec0b28640 1 -- start start 2024-04-25T08:58:37.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.935+0000 7f8ec0b28640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8ebc0cff70 0x7f8ebc0cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:37.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.935+0000 7f8ec0b28640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ebc0b23f0 0x7f8ebc0ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:37.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.935+0000 7f8ec0b28640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ebc0ccf10 con 0x7f8ebc15f180 2024-04-25T08:58:37.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.935+0000 7f8ec0b28640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ebc0cccc0 con 0x7f8ebc0cff70 2024-04-25T08:58:37.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.935+0000 7f8ec0b28640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc0cd840 con 0x7f8ebc0b23f0 2024-04-25T08:58:37.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.935+0000 7f8eba59c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8ebc15f180 0x7f8ebc0cf860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54994/0 (socket says 172.21.15.43:54994) 2024-04-25T08:58:37.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.935+0000 7f8eba59c640 1 -- 172.21.15.43:0/2596514961 learned_addr learned my addr 172.21.15.43:0/2596514961 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:37.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.936+0000 7f8eb9d9b640 1 --2- 172.21.15.43:0/2596514961 >> v2:172.21.15.43:3300/0 conn(0x7f8ebc0cff70 0x7f8ebc0cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:37.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.936+0000 7f8ebad9d640 1 --2- 172.21.15.43:0/2596514961 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ebc0b23f0 0x7f8ebc0ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:37.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.936+0000 7f8eb959a640 1 -- 172.21.15.43:0/2596514961 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 410530537 0 0) 0x7f8ebc0ccf10 con 0x7f8ebc15f180 2024-04-25T08:58:37.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.936+0000 7f8eb959a640 1 -- 172.21.15.43:0/2596514961 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ea8003520 con 0x7f8ebc15f180 2024-04-25T08:58:37.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.936+0000 7f8eb9d9b640 1 -- 172.21.15.43:0/2596514961 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ebc0b23f0 msgr2=0x7f8ebc0ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:37.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.936+0000 7f8eb9d9b640 1 --2- 172.21.15.43:0/2596514961 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ebc0b23f0 0x7f8ebc0ce210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.936+0000 7f8eb9d9b640 1 -- 172.21.15.43:0/2596514961 >> v1:172.21.15.43:6789/0 conn(0x7f8ebc15f180 legacy=0x7f8ebc0cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:37.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.936+0000 7f8eb9d9b640 1 -- 172.21.15.43:0/2596514961 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ebc0cd1f0 con 0x7f8ebc0cff70 2024-04-25T08:58:37.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.937+0000 7f8eb9d9b640 1 --2- 172.21.15.43:0/2596514961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ebc0cff70 0x7f8ebc0cdcd0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f8ea40097a0 tx=0x7f8ea402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e00e56a41a947b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:37.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.937+0000 7f8eb959a640 1 -- 172.21.15.43:0/2596514961 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4037070 con 0x7f8ebc0cff70 2024-04-25T08:58:37.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.937+0000 7f8eb959a640 1 -- 172.21.15.43:0/2596514961 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea402ae50 con 0x7f8ebc0cff70 2024-04-25T08:58:37.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.937+0000 7f8eb959a640 1 -- 172.21.15.43:0/2596514961 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea4032660 con 0x7f8ebc0cff70 2024-04-25T08:58:37.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.937+0000 7f8ec0b28640 1 -- 172.21.15.43:0/2596514961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ebc0cff70 msgr2=0x7f8ebc0cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:37.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.937+0000 7f8ec0b28640 1 --2- 172.21.15.43:0/2596514961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ebc0cff70 0x7f8ebc0cdcd0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f8ea40097a0 tx=0x7f8ea402a100 comp rx=0 tx=0).stop 2024-04-25T08:58:37.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.937+0000 7f8ec0b28640 1 -- 172.21.15.43:0/2596514961 shutdown_connections 2024-04-25T08:58:37.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.937+0000 7f8ec0b28640 1 --2- 172.21.15.43:0/2596514961 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ebc0b23f0 0x7f8ebc0ce210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.937+0000 7f8ec0b28640 1 --2- 172.21.15.43:0/2596514961 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ebc0cff70 0x7f8ebc0cdcd0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.937+0000 7f8ec0b28640 1 -- 172.21.15.43:0/2596514961 >> 172.21.15.43:0/2596514961 conn(0x7f8ebc0d17f0 msgr2=0x7f8ebc0d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:37.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.937+0000 7f8ec0b28640 1 -- 172.21.15.43:0/2596514961 shutdown_connections 2024-04-25T08:58:37.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.937+0000 7f8ec0b28640 1 -- 172.21.15.43:0/2596514961 wait complete. 2024-04-25T08:58:37.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.939+0000 7f8ec0b28640 1 Processor -- start 2024-04-25T08:58:37.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.939+0000 7f8ec0b28640 1 -- start start 2024-04-25T08:58:37.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.939+0000 7f8ec0b28640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ebc0b23f0 0x7f8ebc157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:37.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.940+0000 7f8ec0b28640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ebc0cff70 0x7f8ebc06caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:37.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.940+0000 7f8ec0b28640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ebc15f180 0x7f8ebc158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:37.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.940+0000 7f8ec0b28640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc15f560 con 0x7f8ebc0b23f0 2024-04-25T08:58:37.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.940+0000 7f8ec0b28640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc0d23f0 con 0x7f8ebc15f180 2024-04-25T08:58:37.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.940+0000 7f8ec0b28640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ebc06d4d0 con 0x7f8ebc0cff70 2024-04-25T08:58:37.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.940+0000 7f8eba59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ebc0b23f0 0x7f8ebc157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:37.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.940+0000 7f8eba59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ebc0b23f0 0x7f8ebc157bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36704/0 (socket says 172.21.15.43:36704) 2024-04-25T08:58:37.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.940+0000 7f8eba59c640 1 -- 172.21.15.43:0/2022855816 learned_addr learned my addr 172.21.15.43:0/2022855816 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:37.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.940+0000 7f8eb9d9b640 1 --2- 172.21.15.43:0/2022855816 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ebc0cff70 0x7f8ebc06caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:37.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.940+0000 7f8ebad9d640 1 --2- 172.21.15.43:0/2022855816 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ebc15f180 0x7f8ebc158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:37.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.940+0000 7f8eba59c640 1 -- 172.21.15.43:0/2022855816 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ebc0cff70 msgr2=0x7f8ebc06caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:37.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.940+0000 7f8eba59c640 1 --2- 172.21.15.43:0/2022855816 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ebc0cff70 0x7f8ebc06caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.940+0000 7f8eba59c640 1 -- 172.21.15.43:0/2022855816 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ebc15f180 msgr2=0x7f8ebc158730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:37.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.940+0000 7f8eba59c640 1 --2- 172.21.15.43:0/2022855816 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ebc15f180 0x7f8ebc158730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:37.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.940+0000 7f8eba59c640 1 -- 172.21.15.43:0/2022855816 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ebc0c94e0 con 0x7f8ebc0b23f0 2024-04-25T08:58:37.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.940+0000 7f8eb9d9b640 1 --2- 172.21.15.43:0/2022855816 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ebc0cff70 0x7f8ebc06caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:37.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.940+0000 7f8eba59c640 1 --2- 172.21.15.43:0/2022855816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ebc0b23f0 0x7f8ebc157bd0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f8eb400c390 tx=0x7f8eb400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:37.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.940+0000 7f8eb2ffd640 1 -- 172.21.15.43:0/2022855816 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb4017070 con 0x7f8ebc0b23f0 2024-04-25T08:58:37.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.940+0000 7f8eb2ffd640 1 -- 172.21.15.43:0/2022855816 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8eb4005ce0 con 0x7f8ebc0b23f0 2024-04-25T08:58:37.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.940+0000 7f8eb2ffd640 1 -- 172.21.15.43:0/2022855816 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8eb40072f0 con 0x7f8ebc0b23f0 2024-04-25T08:58:37.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.940+0000 7f8ec0b28640 1 -- 172.21.15.43:0/2022855816 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ebc0c97c0 con 0x7f8ebc0b23f0 2024-04-25T08:58:37.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.940+0000 7f8ec0b28640 1 -- 172.21.15.43:0/2022855816 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ebc158110 con 0x7f8ebc0b23f0 2024-04-25T08:58:37.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.941+0000 7f8ec0b28640 1 -- 172.21.15.43:0/2022855816 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ebc074ef0 con 0x7f8ebc0b23f0 2024-04-25T08:58:37.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.942+0000 7f8eb2ffd640 1 -- 172.21.15.43:0/2022855816 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8eb401a070 con 0x7f8ebc0b23f0 2024-04-25T08:58:37.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.942+0000 7f8eb2ffd640 1 --2- 172.21.15.43:0/2022855816 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8e9c06b4d0 0x7f8e9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:37.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.942+0000 7f8eb2ffd640 1 -- 172.21.15.43:0/2022855816 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f8eb4089330 con 0x7f8ebc0b23f0 2024-04-25T08:58:37.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.943+0000 7f8eb9d9b640 1 --2- 172.21.15.43:0/2022855816 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8e9c06b4d0 0x7f8e9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:37.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.943+0000 7f8eb9d9b640 1 --2- 172.21.15.43:0/2022855816 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8e9c06b4d0 0x7f8e9c06d990 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4034000 tx=0x7f8ea402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:37.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:37.947+0000 7f8eb2ffd640 1 -- 172.21.15.43:0/2022855816 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8eb4058490 con 0x7f8ebc0b23f0 2024-04-25T08:58:38.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.196+0000 7f8ec0b28640 1 -- 172.21.15.43:0/2022855816 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]} v 0) v1 -- 0x7f8ebc0cf050 con 0x7f8ebc0b23f0 2024-04-25T08:58:38.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.502+0000 7f8eb2ffd640 1 -- 172.21.15.43:0/2022855816 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]}]=0 v230) v1 ==== 143+0+0 (secure 0 0 0) 0x7f8eb4057d90 con 0x7f8ebc0b23f0 2024-04-25T08:58:38.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.504+0000 7f8ec0b28640 1 -- 172.21.15.43:0/2022855816 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8e9c06b4d0 msgr2=0x7f8e9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:38.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.504+0000 7f8ec0b28640 1 --2- 172.21.15.43:0/2022855816 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8e9c06b4d0 0x7f8e9c06d990 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f8ea4034000 tx=0x7f8ea402e000 comp rx=0 tx=0).stop 2024-04-25T08:58:38.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.504+0000 7f8ec0b28640 1 -- 172.21.15.43:0/2022855816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ebc0b23f0 msgr2=0x7f8ebc157bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:38.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.504+0000 7f8ec0b28640 1 --2- 172.21.15.43:0/2022855816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ebc0b23f0 0x7f8ebc157bd0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f8eb400c390 tx=0x7f8eb400c850 comp rx=0 tx=0).stop 2024-04-25T08:58:38.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.505+0000 7f8ec0b28640 1 -- 172.21.15.43:0/2022855816 shutdown_connections 2024-04-25T08:58:38.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.505+0000 7f8ec0b28640 1 --2- 172.21.15.43:0/2022855816 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8e9c06b4d0 0x7f8e9c06d990 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.505+0000 7f8ec0b28640 1 --2- 172.21.15.43:0/2022855816 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ebc15f180 0x7f8ebc158730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.505+0000 7f8ec0b28640 1 --2- 172.21.15.43:0/2022855816 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ebc0cff70 0x7f8ebc06caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.505+0000 7f8ec0b28640 1 --2- 172.21.15.43:0/2022855816 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ebc0b23f0 0x7f8ebc157bd0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.505+0000 7f8ec0b28640 1 -- 172.21.15.43:0/2022855816 >> 172.21.15.43:0/2022855816 conn(0x7f8ebc0d17f0 msgr2=0x7f8ebc0cc380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:38.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.505+0000 7f8ec0b28640 1 -- 172.21.15.43:0/2022855816 shutdown_connections 2024-04-25T08:58:38.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.505+0000 7f8ec0b28640 1 -- 172.21.15.43:0/2022855816 wait complete. 2024-04-25T08:58:38.506 INFO:tasks.cephfs_test_runner:test_scrub_repair (tasks.cephfs.test_scrub_checks.TestScrubChecks) ... ok 2024-04-25T08:58:38.506 INFO:tasks.cephfs_test_runner:Starting test: test_stray_evaluation_with_scrub (tasks.cephfs.test_scrub_checks.TestScrubChecks) test that scrub can iterate over ~mdsdir and evaluate strays 2024-04-25T08:58:38.507 INFO:tasks.cephfs_test_runner:test_stray_evaluation_with_scrub (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T08:58:38.507 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:58:38.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.620+0000 7fc094d9e640 1 Processor -- start 2024-04-25T08:58:38.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.620+0000 7fc094d9e640 1 -- start start 2024-04-25T08:58:38.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.620+0000 7fc094d9e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc09015df50 0x7fc0901619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:38.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.620+0000 7fc094d9e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc0900cf670 0x7fc090162380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:38.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.620+0000 7fc094d9e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0900d24a0 con 0x7fc090161f30 2024-04-25T08:58:38.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.620+0000 7fc094d9e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc09015e330 con 0x7fc09015df50 2024-04-25T08:58:38.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.620+0000 7fc094d9e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0900d2740 con 0x7fc0900cf670 2024-04-25T08:58:38.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.621+0000 7fc08e59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc09015df50 0x7fc0901619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:38.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.621+0000 7fc087fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc090161f30 0x7fc0900c7cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:54996/0 (socket says 172.21.15.43:54996) 2024-04-25T08:58:38.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.621+0000 7fc087fff640 1 -- 172.21.15.43:0/4204771772 learned_addr learned my addr 172.21.15.43:0/4204771772 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:38.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.621+0000 7fc08e59c640 1 -- 172.21.15.43:0/4204771772 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc0900cf670 msgr2=0x7fc090162380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:38.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.621+0000 7fc08ed9d640 1 --2- 172.21.15.43:0/4204771772 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc0900cf670 0x7fc090162380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:38.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.621+0000 7fc08e59c640 1 --2- 172.21.15.43:0/4204771772 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc0900cf670 0x7fc090162380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.621+0000 7fc08e59c640 1 -- 172.21.15.43:0/4204771772 >> v1:172.21.15.43:6789/0 conn(0x7fc090161f30 legacy=0x7fc0900c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:38.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.621+0000 7fc08dd9b640 1 -- 172.21.15.43:0/4204771772 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2586426840 0 0) 0x7fc0900d24a0 con 0x7fc090161f30 2024-04-25T08:58:38.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.621+0000 7fc08e59c640 1 -- 172.21.15.43:0/4204771772 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0900d2a20 con 0x7fc09015df50 2024-04-25T08:58:38.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.621+0000 7fc08ed9d640 1 --2- 172.21.15.43:0/4204771772 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc0900cf670 0x7fc090162380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:38.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.622+0000 7fc08e59c640 1 --2- 172.21.15.43:0/4204771772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc09015df50 0x7fc0901619f0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fc0800097a0 tx=0x7fc08002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c94074962f1c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:38.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.622+0000 7fc08dd9b640 1 -- 172.21.15.43:0/4204771772 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc080037070 con 0x7fc09015df50 2024-04-25T08:58:38.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.622+0000 7fc08dd9b640 1 -- 172.21.15.43:0/4204771772 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc080032440 con 0x7fc09015df50 2024-04-25T08:58:38.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.622+0000 7fc08dd9b640 1 -- 172.21.15.43:0/4204771772 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc080032810 con 0x7fc09015df50 2024-04-25T08:58:38.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.622+0000 7fc094d9e640 1 -- 172.21.15.43:0/4204771772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc09015df50 msgr2=0x7fc0901619f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:38.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.622+0000 7fc094d9e640 1 --2- 172.21.15.43:0/4204771772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc09015df50 0x7fc0901619f0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fc0800097a0 tx=0x7fc08002a180 comp rx=0 tx=0).stop 2024-04-25T08:58:38.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.622+0000 7fc094d9e640 1 -- 172.21.15.43:0/4204771772 shutdown_connections 2024-04-25T08:58:38.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.622+0000 7fc094d9e640 1 --2- 172.21.15.43:0/4204771772 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc0900cf670 0x7fc090162380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.622+0000 7fc094d9e640 1 --2- 172.21.15.43:0/4204771772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc09015df50 0x7fc0901619f0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.622+0000 7fc094d9e640 1 -- 172.21.15.43:0/4204771772 >> 172.21.15.43:0/4204771772 conn(0x7fc0900d3a30 msgr2=0x7fc0900d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:38.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.622+0000 7fc094d9e640 1 -- 172.21.15.43:0/4204771772 shutdown_connections 2024-04-25T08:58:38.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.622+0000 7fc094d9e640 1 -- 172.21.15.43:0/4204771772 wait complete. 2024-04-25T08:58:38.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.624+0000 7fc094d9e640 1 Processor -- start 2024-04-25T08:58:38.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.624+0000 7fc094d9e640 1 -- start start 2024-04-25T08:58:38.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.624+0000 7fc094d9e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0900cf670 0x7fc090153c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:38.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.624+0000 7fc094d9e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc09015df50 0x7fc09009f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:38.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.624+0000 7fc094d9e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc090161f30 0x7fc09009fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:38.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.624+0000 7fc094d9e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0900d2190 con 0x7fc090161f30 2024-04-25T08:58:38.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.624+0000 7fc094d9e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0900c77d0 con 0x7fc0900cf670 2024-04-25T08:58:38.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.624+0000 7fc094d9e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0900c8400 con 0x7fc09015df50 2024-04-25T08:58:38.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.625+0000 7fc08ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc090161f30 0x7fc09009fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:38.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.625+0000 7fc08ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc090161f30 0x7fc09009fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36724/0 (socket says 172.21.15.43:36724) 2024-04-25T08:58:38.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.625+0000 7fc08ed9d640 1 -- 172.21.15.43:0/3159757894 learned_addr learned my addr 172.21.15.43:0/3159757894 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:38.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.625+0000 7fc08e59c640 1 --2- 172.21.15.43:0/3159757894 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0900cf670 0x7fc090153c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:38.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.625+0000 7fc087fff640 1 --2- 172.21.15.43:0/3159757894 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc09015df50 0x7fc09009f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:38.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.625+0000 7fc08ed9d640 1 -- 172.21.15.43:0/3159757894 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc09015df50 msgr2=0x7fc09009f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:38.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.625+0000 7fc08ed9d640 1 --2- 172.21.15.43:0/3159757894 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc09015df50 0x7fc09009f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.625+0000 7fc08ed9d640 1 -- 172.21.15.43:0/3159757894 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0900cf670 msgr2=0x7fc090153c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:38.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.625+0000 7fc08ed9d640 1 --2- 172.21.15.43:0/3159757894 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0900cf670 0x7fc090153c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.625+0000 7fc08ed9d640 1 -- 172.21.15.43:0/3159757894 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0900a0f10 con 0x7fc090161f30 2024-04-25T08:58:38.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.625+0000 7fc08ed9d640 1 --2- 172.21.15.43:0/3159757894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc090161f30 0x7fc09009fd50 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fc08800cf20 tx=0x7fc08800a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:38.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.625+0000 7fc086ffd640 1 -- 172.21.15.43:0/3159757894 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08801a070 con 0x7fc090161f30 2024-04-25T08:58:38.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.625+0000 7fc086ffd640 1 -- 172.21.15.43:0/3159757894 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc08800d840 con 0x7fc090161f30 2024-04-25T08:58:38.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.625+0000 7fc094d9e640 1 -- 172.21.15.43:0/3159757894 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0900a0390 con 0x7fc090161f30 2024-04-25T08:58:38.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.625+0000 7fc086ffd640 1 -- 172.21.15.43:0/3159757894 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08800dc30 con 0x7fc090161f30 2024-04-25T08:58:38.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.625+0000 7fc094d9e640 1 -- 172.21.15.43:0/3159757894 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0900a0950 con 0x7fc090161f30 2024-04-25T08:58:38.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.627+0000 7fc094d9e640 1 -- 172.21.15.43:0/3159757894 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0900680f0 con 0x7fc090161f30 2024-04-25T08:58:38.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.627+0000 7fc086ffd640 1 -- 172.21.15.43:0/3159757894 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc08801d070 con 0x7fc090161f30 2024-04-25T08:58:38.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.627+0000 7fc086ffd640 1 --2- 172.21.15.43:0/3159757894 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc07006b4d0 0x7fc07006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:38.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.628+0000 7fc086ffd640 1 -- 172.21.15.43:0/3159757894 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fc08808a7f0 con 0x7fc090161f30 2024-04-25T08:58:38.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.628+0000 7fc08e59c640 1 --2- 172.21.15.43:0/3159757894 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc07006b4d0 0x7fc07006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:38.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.628+0000 7fc08e59c640 1 --2- 172.21.15.43:0/3159757894 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc07006b4d0 0x7fc07006d990 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc080034000 tx=0x7fc08002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:38.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.632+0000 7fc086ffd640 1 -- 172.21.15.43:0/3159757894 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc088059270 con 0x7fc090161f30 2024-04-25T08:58:38.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.814+0000 7fc094d9e640 1 -- 172.21.15.43:0/3159757894 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc09015f630 con 0x7fc090161f30 2024-04-25T08:58:38.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.815+0000 7fc086ffd640 1 -- 172.21.15.43:0/3159757894 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+13244 (secure 0 0 0) 0x7fc08805d310 con 0x7fc090161f30 2024-04-25T08:58:38.805 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:58:38.805 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":65,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:58:37.491976+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6833/1866869111":"2024-04-26T08:58:34.339307+0000","172.21.15.43:6832/1866869111":"2024-04-26T08:58:34.339307+0000","172.21.15.43:6834/3574570511":"2024-04-26T08:58:12.413536+0000","172.21.15.43:6832/3574570511":"2024-04-26T08:58:12.413536+0000","172.21.15.43:6835/2268368384":"2024-04-26T08:58:11.416075+0000","172.21.15.43:6833/2268368384":"2024-04-26T08:58:11.416075+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:58:38.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.817+0000 7fc094d9e640 1 -- 172.21.15.43:0/3159757894 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc07006b4d0 msgr2=0x7fc07006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:38.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.817+0000 7fc094d9e640 1 --2- 172.21.15.43:0/3159757894 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc07006b4d0 0x7fc07006d990 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc080034000 tx=0x7fc08002e000 comp rx=0 tx=0).stop 2024-04-25T08:58:38.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.817+0000 7fc094d9e640 1 -- 172.21.15.43:0/3159757894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc090161f30 msgr2=0x7fc09009fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:38.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.817+0000 7fc094d9e640 1 --2- 172.21.15.43:0/3159757894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc090161f30 0x7fc09009fd50 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fc08800cf20 tx=0x7fc08800a7d0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.817+0000 7fc094d9e640 1 -- 172.21.15.43:0/3159757894 shutdown_connections 2024-04-25T08:58:38.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.817+0000 7fc094d9e640 1 --2- 172.21.15.43:0/3159757894 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc090161f30 0x7fc09009fd50 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.817+0000 7fc094d9e640 1 --2- 172.21.15.43:0/3159757894 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc09015df50 0x7fc09009f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.817+0000 7fc094d9e640 1 --2- 172.21.15.43:0/3159757894 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc07006b4d0 0x7fc07006d990 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.817+0000 7fc094d9e640 1 --2- 172.21.15.43:0/3159757894 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0900cf670 0x7fc090153c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.817+0000 7fc094d9e640 1 -- 172.21.15.43:0/3159757894 >> 172.21.15.43:0/3159757894 conn(0x7fc0900d3a30 msgr2=0x7fc0900d1570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:38.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.817+0000 7fc094d9e640 1 -- 172.21.15.43:0/3159757894 shutdown_connections 2024-04-25T08:58:38.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.817+0000 7fc094d9e640 1 -- 172.21.15.43:0/3159757894 wait complete. 2024-04-25T08:58:38.818 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:58:38.818 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:58:38.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.935+0000 7f36b12b3640 1 Processor -- start 2024-04-25T08:58:38.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.935+0000 7f36b12b3640 1 -- start start 2024-04-25T08:58:38.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.935+0000 7f36b12b3640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f36ac06ab30 0x7f36ac06af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:38.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.935+0000 7f36b12b3640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f36ac0b42a0 0x7f36ac0b4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:38.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.935+0000 7f36b12b3640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36ac15e160 con 0x7f36ac071440 2024-04-25T08:58:38.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.935+0000 7f36b12b3640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36ac06b450 con 0x7f36ac06ab30 2024-04-25T08:58:38.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.935+0000 7f36b12b3640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36ac15e400 con 0x7f36ac0b42a0 2024-04-25T08:58:38.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.936+0000 7f36a3fff640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f36ac0b42a0 0x7f36ac0b4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:38.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.936+0000 7f36aaffd640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f36ac06ab30 0x7f36ac06af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:38.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.936+0000 7f36a3fff640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f36ac0b42a0 0x7f36ac0b4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:40512/0 (socket says 172.21.15.43:40512) 2024-04-25T08:58:38.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.936+0000 7f36a3fff640 1 -- 172.21.15.43:0/2411429501 learned_addr learned my addr 172.21.15.43:0/2411429501 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:38.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.936+0000 7f36a3fff640 1 -- 172.21.15.43:0/2411429501 >> v2:172.21.15.193:3300/0 conn(0x7f36ac06ab30 msgr2=0x7f36ac06af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:38.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.936+0000 7f36a3fff640 1 --2- 172.21.15.43:0/2411429501 >> v2:172.21.15.193:3300/0 conn(0x7f36ac06ab30 0x7f36ac06af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.936+0000 7f36a3fff640 1 -- 172.21.15.43:0/2411429501 >> v1:172.21.15.193:6789/0 conn(0x7f36ac071440 legacy=0x7f36ac0718d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:38.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.936+0000 7f36a3fff640 1 -- 172.21.15.43:0/2411429501 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ac15e6e0 con 0x7f36ac0b42a0 2024-04-25T08:58:38.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.937+0000 7f36a3fff640 1 --2- 172.21.15.43:0/2411429501 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f36ac0b42a0 0x7f36ac0b4720 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3694009870 tx=0x7f369402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93cbdab809dbc455 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:38.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.937+0000 7f36aa7fc640 1 -- 172.21.15.43:0/2411429501 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3694037070 con 0x7f36ac0b42a0 2024-04-25T08:58:38.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.937+0000 7f36aa7fc640 1 -- 172.21.15.43:0/2411429501 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3694032440 con 0x7f36ac0b42a0 2024-04-25T08:58:38.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.937+0000 7f36aa7fc640 1 -- 172.21.15.43:0/2411429501 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3694032810 con 0x7f36ac0b42a0 2024-04-25T08:58:38.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.938+0000 7f36b12b3640 1 -- 172.21.15.43:0/2411429501 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f36ac0b42a0 msgr2=0x7f36ac0b4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:38.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.938+0000 7f36b12b3640 1 --2- 172.21.15.43:0/2411429501 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f36ac0b42a0 0x7f36ac0b4720 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3694009870 tx=0x7f369402a270 comp rx=0 tx=0).stop 2024-04-25T08:58:38.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.938+0000 7f36b12b3640 1 -- 172.21.15.43:0/2411429501 shutdown_connections 2024-04-25T08:58:38.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.938+0000 7f36b12b3640 1 --2- 172.21.15.43:0/2411429501 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f36ac0b42a0 0x7f36ac0b4720 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.938+0000 7f36b12b3640 1 --2- 172.21.15.43:0/2411429501 >> v2:172.21.15.193:3300/0 conn(0x7f36ac06ab30 0x7f36ac06af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.938+0000 7f36b12b3640 1 -- 172.21.15.43:0/2411429501 >> 172.21.15.43:0/2411429501 conn(0x7f36ac0b2590 msgr2=0x7f36ac0b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:38.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.938+0000 7f36b12b3640 1 -- 172.21.15.43:0/2411429501 shutdown_connections 2024-04-25T08:58:38.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.938+0000 7f36b12b3640 1 -- 172.21.15.43:0/2411429501 wait complete. 2024-04-25T08:58:38.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.939+0000 7f36b12b3640 1 Processor -- start 2024-04-25T08:58:38.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.939+0000 7f36b12b3640 1 -- start start 2024-04-25T08:58:38.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.940+0000 7f36b12b3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36ac06ab30 0x7f36ac1e69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:38.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.940+0000 7f36b12b3640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f36ac071440 0x7f36ac141bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:38.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.940+0000 7f36b12b3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36ac0b42a0 0x7f36ac142110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:38.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.940+0000 7f36b12b3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ac15b950 con 0x7f36ac06ab30 2024-04-25T08:58:38.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.940+0000 7f36b12b3640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ac15eca0 con 0x7f36ac0b42a0 2024-04-25T08:58:38.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.940+0000 7f36b12b3640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36ac15bad0 con 0x7f36ac071440 2024-04-25T08:58:38.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.940+0000 7f36aaffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36ac06ab30 0x7f36ac1e69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:38.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.940+0000 7f36aaffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36ac06ab30 0x7f36ac1e69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36738/0 (socket says 172.21.15.43:36738) 2024-04-25T08:58:38.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.940+0000 7f36aaffd640 1 -- 172.21.15.43:0/2295163754 learned_addr learned my addr 172.21.15.43:0/2295163754 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:38.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.940+0000 7f36a3fff640 1 --2- 172.21.15.43:0/2295163754 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f36ac071440 0x7f36ac141bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:38.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.940+0000 7f36ab7fe640 1 --2- 172.21.15.43:0/2295163754 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36ac0b42a0 0x7f36ac142110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:38.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.940+0000 7f36aaffd640 1 -- 172.21.15.43:0/2295163754 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f36ac071440 msgr2=0x7f36ac141bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:38.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.940+0000 7f36aaffd640 1 --2- 172.21.15.43:0/2295163754 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f36ac071440 0x7f36ac141bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.940+0000 7f36aaffd640 1 -- 172.21.15.43:0/2295163754 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36ac0b42a0 msgr2=0x7f36ac142110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:38.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.940+0000 7f36aaffd640 1 --2- 172.21.15.43:0/2295163754 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36ac0b42a0 0x7f36ac142110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:38.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.940+0000 7f36aaffd640 1 -- 172.21.15.43:0/2295163754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36ac1593f0 con 0x7f36ac06ab30 2024-04-25T08:58:38.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.940+0000 7f36aaffd640 1 --2- 172.21.15.43:0/2295163754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36ac06ab30 0x7f36ac1e69b0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f369c00e9b0 tx=0x7f369c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:38.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.941+0000 7f36a37fe640 1 -- 172.21.15.43:0/2295163754 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369c017070 con 0x7f36ac06ab30 2024-04-25T08:58:38.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.941+0000 7f36a37fe640 1 -- 172.21.15.43:0/2295163754 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f369c00c9a0 con 0x7f36ac06ab30 2024-04-25T08:58:38.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.941+0000 7f36b12b3640 1 -- 172.21.15.43:0/2295163754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36ac158870 con 0x7f36ac06ab30 2024-04-25T08:58:38.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.941+0000 7f36a37fe640 1 -- 172.21.15.43:0/2295163754 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369c00cd90 con 0x7f36ac06ab30 2024-04-25T08:58:38.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.941+0000 7f36b12b3640 1 -- 172.21.15.43:0/2295163754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36ac159110 con 0x7f36ac06ab30 2024-04-25T08:58:38.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.942+0000 7f36b12b3640 1 -- 172.21.15.43:0/2295163754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36ac074f60 con 0x7f36ac06ab30 2024-04-25T08:58:38.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.942+0000 7f36a37fe640 1 -- 172.21.15.43:0/2295163754 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f369c01a070 con 0x7f36ac06ab30 2024-04-25T08:58:38.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.943+0000 7f36a37fe640 1 --2- 172.21.15.43:0/2295163754 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f368c06b4d0 0x7f368c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:38.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.943+0000 7f36a37fe640 1 -- 172.21.15.43:0/2295163754 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f369c089f40 con 0x7f36ac06ab30 2024-04-25T08:58:38.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.943+0000 7f36a3fff640 1 --2- 172.21.15.43:0/2295163754 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f368c06b4d0 0x7f368c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:38.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.943+0000 7f36a3fff640 1 --2- 172.21.15.43:0/2295163754 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f368c06b4d0 0x7f368c06d990 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3694034000 tx=0x7f369402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:38.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:38.950+0000 7f36a37fe640 1 -- 172.21.15.43:0/2295163754 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f369c0589c0 con 0x7f36ac06ab30 2024-04-25T08:58:39.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.133+0000 7f36b12b3640 1 -- 172.21.15.43:0/2295163754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f36ac0ae2b0 con 0x7f36ac06ab30 2024-04-25T08:58:39.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.134+0000 7f36a37fe640 1 -- 172.21.15.43:0/2295163754 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) v1 ==== 93+0+10 (secure 0 0 0) 0x7f369c05ca60 con 0x7f36ac06ab30 2024-04-25T08:58:39.123 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T08:58:39.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.136+0000 7f36b12b3640 1 -- 172.21.15.43:0/2295163754 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f368c06b4d0 msgr2=0x7f368c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:39.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.136+0000 7f36b12b3640 1 --2- 172.21.15.43:0/2295163754 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f368c06b4d0 0x7f368c06d990 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f3694034000 tx=0x7f369402e000 comp rx=0 tx=0).stop 2024-04-25T08:58:39.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.136+0000 7f36b12b3640 1 -- 172.21.15.43:0/2295163754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36ac06ab30 msgr2=0x7f36ac1e69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:39.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.136+0000 7f36b12b3640 1 --2- 172.21.15.43:0/2295163754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36ac06ab30 0x7f36ac1e69b0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f369c00e9b0 tx=0x7f369c00ee70 comp rx=0 tx=0).stop 2024-04-25T08:58:39.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.136+0000 7f36b12b3640 1 -- 172.21.15.43:0/2295163754 shutdown_connections 2024-04-25T08:58:39.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.136+0000 7f36b12b3640 1 --2- 172.21.15.43:0/2295163754 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f368c06b4d0 0x7f368c06d990 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:39.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.136+0000 7f36b12b3640 1 --2- 172.21.15.43:0/2295163754 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36ac0b42a0 0x7f36ac142110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:39.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.136+0000 7f36b12b3640 1 --2- 172.21.15.43:0/2295163754 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f36ac071440 0x7f36ac141bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:39.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.136+0000 7f36b12b3640 1 --2- 172.21.15.43:0/2295163754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36ac06ab30 0x7f36ac1e69b0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:39.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.136+0000 7f36b12b3640 1 -- 172.21.15.43:0/2295163754 >> 172.21.15.43:0/2295163754 conn(0x7f36ac0b2590 msgr2=0x7f36ac0b1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:39.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.136+0000 7f36b12b3640 1 -- 172.21.15.43:0/2295163754 shutdown_connections 2024-04-25T08:58:39.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.136+0000 7f36b12b3640 1 -- 172.21.15.43:0/2295163754 wait complete. 2024-04-25T08:58:39.138 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub' 2024-04-25T08:58:39.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.250+0000 7f66a2db0640 1 Processor -- start 2024-04-25T08:58:39.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.250+0000 7f66a2db0640 1 -- start start 2024-04-25T08:58:39.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.251+0000 7f66a2db0640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f669c15ef50 0x7f669c0d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:39.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.251+0000 7f66a2db0640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f669c0d0d30 con 0x7f669c15dce0 2024-04-25T08:58:39.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.251+0000 7f66a2db0640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f669c0d2920 con 0x7f669c0d0920 2024-04-25T08:58:39.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.251+0000 7f66a2db0640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f669c0cec60 con 0x7f669c15ef50 2024-04-25T08:58:39.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.251+0000 7f669bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f669c15ef50 0x7f669c0d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:39.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.251+0000 7f66a0b4c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f669c15dce0 0x7f669c0ce550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:55006/0 (socket says 172.21.15.43:55006) 2024-04-25T08:58:39.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.251+0000 7f66a0b4c640 1 -- 172.21.15.43:0/657301697 learned_addr learned my addr 172.21.15.43:0/657301697 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:39.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.251+0000 7f669bfff640 1 -- 172.21.15.43:0/657301697 >> v1:172.21.15.193:6789/0 conn(0x7f669c0d0920 legacy=0x7f669c160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:39.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.251+0000 7f669bfff640 1 -- 172.21.15.43:0/657301697 >> v1:172.21.15.43:6789/0 conn(0x7f669c15dce0 legacy=0x7f669c0ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:39.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.251+0000 7f669bfff640 1 -- 172.21.15.43:0/657301697 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f669c0cee80 con 0x7f669c15ef50 2024-04-25T08:58:39.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.251+0000 7f669b7fe640 1 -- 172.21.15.43:0/657301697 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4214196849 0 0) 0x7f6690001110 con 0x7f669c15dce0 2024-04-25T08:58:39.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.252+0000 7f669bfff640 1 --2- 172.21.15.43:0/657301697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f669c15ef50 0x7f669c0d03e0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f66880097a0 tx=0x7f668802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58490043603dff2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:39.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.252+0000 7f669b7fe640 1 -- 172.21.15.43:0/657301697 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6688037070 con 0x7f669c15ef50 2024-04-25T08:58:39.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.252+0000 7f669b7fe640 1 -- 172.21.15.43:0/657301697 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6688032440 con 0x7f669c15ef50 2024-04-25T08:58:39.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.252+0000 7f669b7fe640 1 -- 172.21.15.43:0/657301697 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6688032810 con 0x7f669c15ef50 2024-04-25T08:58:39.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.252+0000 7f66a2db0640 1 -- 172.21.15.43:0/657301697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f669c15ef50 msgr2=0x7f669c0d03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:39.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.252+0000 7f66a2db0640 1 --2- 172.21.15.43:0/657301697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f669c15ef50 0x7f669c0d03e0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f66880097a0 tx=0x7f668802a1a0 comp rx=0 tx=0).stop 2024-04-25T08:58:39.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.253+0000 7f66a2db0640 1 -- 172.21.15.43:0/657301697 shutdown_connections 2024-04-25T08:58:39.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.253+0000 7f66a2db0640 1 --2- 172.21.15.43:0/657301697 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f669c15ef50 0x7f669c0d03e0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:39.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.253+0000 7f66a2db0640 1 -- 172.21.15.43:0/657301697 >> 172.21.15.43:0/657301697 conn(0x7f669c09cd00 msgr2=0x7f669c0d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:39.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.253+0000 7f66a2db0640 1 -- 172.21.15.43:0/657301697 shutdown_connections 2024-04-25T08:58:39.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.253+0000 7f66a2db0640 1 -- 172.21.15.43:0/657301697 wait complete. 2024-04-25T08:58:39.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.254+0000 7f66a2db0640 1 Processor -- start 2024-04-25T08:58:39.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.254+0000 7f66a2db0640 1 -- start start 2024-04-25T08:58:39.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.255+0000 7f66a2db0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f669c0d0920 0x7f669c1582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:39.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.255+0000 7f66a2db0640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f669c15dce0 0x7f669c14d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:39.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.255+0000 7f66a2db0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f669c15ef50 0x7f669c14deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:39.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.255+0000 7f66a2db0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669c0cf720 con 0x7f669c0d0920 2024-04-25T08:58:39.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.255+0000 7f66a2db0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669c0cf440 con 0x7f669c15ef50 2024-04-25T08:58:39.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.255+0000 7f66a2db0640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f669c15f6b0 con 0x7f669c15dce0 2024-04-25T08:58:39.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.255+0000 7f66a0b4c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f669c0d0920 0x7f669c1582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:39.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.255+0000 7f66a0b4c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f669c0d0920 0x7f669c1582c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:36756/0 (socket says 172.21.15.43:36756) 2024-04-25T08:58:39.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.255+0000 7f66a0b4c640 1 -- 172.21.15.43:0/3440374651 learned_addr learned my addr 172.21.15.43:0/3440374651 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:39.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.255+0000 7f669bfff640 1 --2- 172.21.15.43:0/3440374651 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f669c15dce0 0x7f669c14d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:39.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.255+0000 7f66a134d640 1 --2- 172.21.15.43:0/3440374651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f669c15ef50 0x7f669c14deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:39.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.255+0000 7f66a0b4c640 1 -- 172.21.15.43:0/3440374651 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f669c15dce0 msgr2=0x7f669c14d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:39.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.255+0000 7f66a0b4c640 1 --2- 172.21.15.43:0/3440374651 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f669c15dce0 0x7f669c14d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:39.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.255+0000 7f66a0b4c640 1 -- 172.21.15.43:0/3440374651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f669c15ef50 msgr2=0x7f669c14deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:39.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.255+0000 7f66a0b4c640 1 --2- 172.21.15.43:0/3440374651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f669c15ef50 0x7f669c14deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:39.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.255+0000 7f66a0b4c640 1 -- 172.21.15.43:0/3440374651 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f669c0cd690 con 0x7f669c0d0920 2024-04-25T08:58:39.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.255+0000 7f66a134d640 1 --2- 172.21.15.43:0/3440374651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f669c15ef50 0x7f669c14deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:39.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.255+0000 7f66a0b4c640 1 --2- 172.21.15.43:0/3440374651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f669c0d0920 0x7f669c1582c0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f669000c780 tx=0x7f669000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:39.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.255+0000 7f66997fa640 1 -- 172.21.15.43:0/3440374651 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6690018070 con 0x7f669c0d0920 2024-04-25T08:58:39.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.255+0000 7f66997fa640 1 -- 172.21.15.43:0/3440374651 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66900072d0 con 0x7f669c0d0920 2024-04-25T08:58:39.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.255+0000 7f66a2db0640 1 -- 172.21.15.43:0/3440374651 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f669c0cd970 con 0x7f669c0d0920 2024-04-25T08:58:39.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.255+0000 7f66997fa640 1 -- 172.21.15.43:0/3440374651 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66900047e0 con 0x7f669c0d0920 2024-04-25T08:58:39.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.255+0000 7f66a2db0640 1 -- 172.21.15.43:0/3440374651 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f669c15a0b0 con 0x7f669c0d0920 2024-04-25T08:58:39.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.257+0000 7f66a2db0640 1 -- 172.21.15.43:0/3440374651 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f669c0680f0 con 0x7f669c0d0920 2024-04-25T08:58:39.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.257+0000 7f66997fa640 1 -- 172.21.15.43:0/3440374651 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f669001b070 con 0x7f669c0d0920 2024-04-25T08:58:39.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.257+0000 7f66997fa640 1 --2- 172.21.15.43:0/3440374651 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f668006b4d0 0x7f668006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:39.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.257+0000 7f66997fa640 1 -- 172.21.15.43:0/3440374651 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f6690089260 con 0x7f669c0d0920 2024-04-25T08:58:39.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.258+0000 7f669bfff640 1 --2- 172.21.15.43:0/3440374651 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f668006b4d0 0x7f668006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:39.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.258+0000 7f669bfff640 1 --2- 172.21.15.43:0/3440374651 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f668006b4d0 0x7f668006d990 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6688034000 tx=0x7f668802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:39.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.261+0000 7f66997fa640 1 -- 172.21.15.43:0/3440374651 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f669c0d0d30 con 0x7f669c0d0920 2024-04-25T08:58:39.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:39.519+0000 7f66a2db0640 1 -- 172.21.15.43:0/3440374651 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]} v 0) v1 -- 0x7f669c09d1a0 con 0x7f669c0d0920 2024-04-25T08:58:40.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.511+0000 7f66997fa640 1 -- 172.21.15.43:0/3440374651 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]}]=0 v232) v1 ==== 161+0+0 (secure 0 0 0) 0x7f66900583c0 con 0x7f669c0d0920 2024-04-25T08:58:40.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.514+0000 7f66a2db0640 1 -- 172.21.15.43:0/3440374651 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f668006b4d0 msgr2=0x7f668006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:40.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.514+0000 7f66a2db0640 1 --2- 172.21.15.43:0/3440374651 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f668006b4d0 0x7f668006d990 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6688034000 tx=0x7f668802e000 comp rx=0 tx=0).stop 2024-04-25T08:58:40.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.514+0000 7f66a2db0640 1 -- 172.21.15.43:0/3440374651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f669c0d0920 msgr2=0x7f669c1582c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:40.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.514+0000 7f66a2db0640 1 --2- 172.21.15.43:0/3440374651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f669c0d0920 0x7f669c1582c0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f669000c780 tx=0x7f669000cc40 comp rx=0 tx=0).stop 2024-04-25T08:58:40.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.514+0000 7f66a2db0640 1 -- 172.21.15.43:0/3440374651 shutdown_connections 2024-04-25T08:58:40.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.514+0000 7f66a2db0640 1 --2- 172.21.15.43:0/3440374651 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f668006b4d0 0x7f668006d990 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:40.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.514+0000 7f66a2db0640 1 --2- 172.21.15.43:0/3440374651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f669c15ef50 0x7f669c14deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:40.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.514+0000 7f66a2db0640 1 --2- 172.21.15.43:0/3440374651 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f669c15dce0 0x7f669c14d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:40.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.514+0000 7f66a2db0640 1 --2- 172.21.15.43:0/3440374651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f669c0d0920 0x7f669c1582c0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:40.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.514+0000 7f66a2db0640 1 -- 172.21.15.43:0/3440374651 >> 172.21.15.43:0/3440374651 conn(0x7f669c09cd00 msgr2=0x7f669c155830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:40.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.514+0000 7f66a2db0640 1 -- 172.21.15.43:0/3440374651 shutdown_connections 2024-04-25T08:58:40.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.514+0000 7f66a2db0640 1 -- 172.21.15.43:0/3440374651 wait complete. 2024-04-25T08:58:40.517 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T08:58:40.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.639+0000 7f1b93189640 1 Processor -- start 2024-04-25T08:58:40.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.639+0000 7f1b93189640 1 -- start start 2024-04-25T08:58:40.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.640+0000 7f1b93189640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1b8c0d27b0 0x7f1b8c15ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:40.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.640+0000 7f1b93189640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b8c0cf110 con 0x7f1b8c15dec0 2024-04-25T08:58:40.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.640+0000 7f1b93189640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b8c0cf3f0 con 0x7f1b8c0d04a0 2024-04-25T08:58:40.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.640+0000 7f1b93189640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8c0cf6d0 con 0x7f1b8c0d27b0 2024-04-25T08:58:40.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.640+0000 7f1b8bfff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f1b8c15dec0 0x7f1b8c0ce8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39126/0 (socket says 172.21.15.43:39126) 2024-04-25T08:58:40.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.640+0000 7f1b8bfff640 1 -- 172.21.15.43:0/2816552565 learned_addr learned my addr 172.21.15.43:0/2816552565 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:40.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.640+0000 7f1b91726640 1 --2- 172.21.15.43:0/2816552565 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1b8c0d27b0 0x7f1b8c15ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:40.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.640+0000 7f1b8b7fe640 1 -- 172.21.15.43:0/2816552565 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3250246517 0 0) 0x7f1b8c0cf110 con 0x7f1b8c15dec0 2024-04-25T08:58:40.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.640+0000 7f1b8b7fe640 1 -- 172.21.15.43:0/2816552565 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b7c003520 con 0x7f1b8c15dec0 2024-04-25T08:58:40.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.640+0000 7f1b8b7fe640 1 -- 172.21.15.43:0/2816552565 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1815614269 0 0) 0x7f1b7c003520 con 0x7f1b8c15dec0 2024-04-25T08:58:40.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.640+0000 7f1b8b7fe640 1 -- 172.21.15.43:0/2816552565 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1b8c0d27b0 msgr2=0x7f1b8c15ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:40.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.640+0000 7f1b8b7fe640 1 --2- 172.21.15.43:0/2816552565 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1b8c0d27b0 0x7f1b8c15ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:40.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.640+0000 7f1b8b7fe640 1 -- 172.21.15.43:0/2816552565 >> v1:172.21.15.193:6789/0 conn(0x7f1b8c0d04a0 legacy=0x7f1b8c0d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:40.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.640+0000 7f1b8b7fe640 1 -- 172.21.15.43:0/2816552565 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b8c0cf9b0 con 0x7f1b8c15dec0 2024-04-25T08:58:40.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.640+0000 7f1b8b7fe640 1 -- 172.21.15.43:0/2816552565 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f1b78001690 con 0x7f1b8c15dec0 2024-04-25T08:58:40.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.640+0000 7f1b8b7fe640 1 -- 172.21.15.43:0/2816552565 >> v1:172.21.15.43:6789/0 conn(0x7f1b8c15dec0 legacy=0x7f1b8c0ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:40.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.640+0000 7f1b8b7fe640 1 --2- 172.21.15.43:0/2816552565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b7c0039e0 0x7f1b7c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:40.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.640+0000 7f1b8b7fe640 1 --2- 172.21.15.43:0/2816552565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b7c004800 0x7f1b7c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:40.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.640+0000 7f1b8b7fe640 1 --2- 172.21.15.43:0/2816552565 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1b8c0d27b0 0x7f1b7c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:40.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.640+0000 7f1b8b7fe640 1 -- 172.21.15.43:0/2816552565 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8c0cf6d0 con 0x7f1b7c004800 2024-04-25T08:58:40.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.640+0000 7f1b8b7fe640 1 -- 172.21.15.43:0/2816552565 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7c00bb20 con 0x7f1b7c0039e0 2024-04-25T08:58:40.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.640+0000 7f1b8b7fe640 1 -- 172.21.15.43:0/2816552565 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b7c00bd20 con 0x7f1b8c0d27b0 2024-04-25T08:58:40.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.640+0000 7f1b8bfff640 1 --2- 172.21.15.43:0/2816552565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b7c004800 0x7f1b7c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:40.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.641+0000 7f1b90f25640 1 --2- 172.21.15.43:0/2816552565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b7c0039e0 0x7f1b7c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:40.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.641+0000 7f1b91726640 1 --2- 172.21.15.43:0/2816552565 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1b8c0d27b0 0x7f1b7c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:40.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.641+0000 7f1b8bfff640 1 -- 172.21.15.43:0/2816552565 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1b8c0d27b0 msgr2=0x7f1b7c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:40.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.641+0000 7f1b8bfff640 1 --2- 172.21.15.43:0/2816552565 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1b8c0d27b0 0x7f1b7c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:40.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.641+0000 7f1b8bfff640 1 -- 172.21.15.43:0/2816552565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b7c0039e0 msgr2=0x7f1b7c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:40.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.641+0000 7f1b8bfff640 1 --2- 172.21.15.43:0/2816552565 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b7c0039e0 0x7f1b7c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:40.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.641+0000 7f1b8bfff640 1 -- 172.21.15.43:0/2816552565 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b7c003520 con 0x7f1b7c004800 2024-04-25T08:58:40.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.641+0000 7f1b8bfff640 1 --2- 172.21.15.43:0/2816552565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b7c004800 0x7f1b7c004c80 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c0d01e0 tx=0x7f1b7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4662f6b8fcdd4b46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:40.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.642+0000 7f1b8b7fe640 1 -- 172.21.15.43:0/2816552565 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b78007d30 con 0x7f1b7c004800 2024-04-25T08:58:40.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.642+0000 7f1b8b7fe640 1 -- 172.21.15.43:0/2816552565 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b78005af0 con 0x7f1b7c004800 2024-04-25T08:58:40.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.642+0000 7f1b8b7fe640 1 -- 172.21.15.43:0/2816552565 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b78007070 con 0x7f1b7c004800 2024-04-25T08:58:40.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.642+0000 7f1b93189640 1 -- 172.21.15.43:0/2816552565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b7c004800 msgr2=0x7f1b7c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:40.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.642+0000 7f1b93189640 1 --2- 172.21.15.43:0/2816552565 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b7c004800 0x7f1b7c004c80 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c0d01e0 tx=0x7f1b7c005830 comp rx=0 tx=0).stop 2024-04-25T08:58:40.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.642+0000 7f1b91726640 1 -- 172.21.15.43:0/2816552565 reap_dead start 2024-04-25T08:58:40.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.642+0000 7f1b93189640 1 -- 172.21.15.43:0/2816552565 shutdown_connections 2024-04-25T08:58:40.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.642+0000 7f1b93189640 1 -- 172.21.15.43:0/2816552565 >> 172.21.15.43:0/2816552565 conn(0x7f1b8c09cd40 msgr2=0x7f1b8c0d3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:40.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.642+0000 7f1b93189640 1 -- 172.21.15.43:0/2816552565 shutdown_connections 2024-04-25T08:58:40.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.642+0000 7f1b93189640 1 -- 172.21.15.43:0/2816552565 wait complete. 2024-04-25T08:58:40.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.643+0000 7f1b93189640 1 Processor -- start 2024-04-25T08:58:40.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.644+0000 7f1b93189640 1 -- start start 2024-04-25T08:58:40.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.644+0000 7f1b93189640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b7c004800 0x7f1b8c151c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:40.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.644+0000 7f1b93189640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1b8c0b6690 0x7f1b8c0b9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:40.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.644+0000 7f1b93189640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b8c1f48c0 0x7f1b8c0cc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:40.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.644+0000 7f1b93189640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8c15ef10 con 0x7f1b7c004800 2024-04-25T08:58:40.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.644+0000 7f1b93189640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8c15ed90 con 0x7f1b8c1f48c0 2024-04-25T08:58:40.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.644+0000 7f1b93189640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8c15f090 con 0x7f1b8c0b6690 2024-04-25T08:58:40.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.644+0000 7f1b90f25640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b7c004800 0x7f1b8c151c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:40.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.645+0000 7f1b90f25640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b7c004800 0x7f1b8c151c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50284/0 (socket says 172.21.15.43:50284) 2024-04-25T08:58:40.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.645+0000 7f1b90f25640 1 -- 172.21.15.43:0/3865511473 learned_addr learned my addr 172.21.15.43:0/3865511473 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:40.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.645+0000 7f1b8bfff640 1 --2- 172.21.15.43:0/3865511473 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1b8c0b6690 0x7f1b8c0b9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:40.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.645+0000 7f1b91726640 1 --2- 172.21.15.43:0/3865511473 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b8c1f48c0 0x7f1b8c0cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:40.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.645+0000 7f1b90f25640 1 -- 172.21.15.43:0/3865511473 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1b8c0b6690 msgr2=0x7f1b8c0b9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:40.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.645+0000 7f1b90f25640 1 --2- 172.21.15.43:0/3865511473 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1b8c0b6690 0x7f1b8c0b9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:40.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.645+0000 7f1b90f25640 1 -- 172.21.15.43:0/3865511473 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b8c1f48c0 msgr2=0x7f1b8c0cc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:40.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.645+0000 7f1b90f25640 1 --2- 172.21.15.43:0/3865511473 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b8c1f48c0 0x7f1b8c0cc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:40.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.645+0000 7f1b90f25640 1 -- 172.21.15.43:0/3865511473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b8c152490 con 0x7f1b7c004800 2024-04-25T08:58:40.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.645+0000 7f1b90f25640 1 --2- 172.21.15.43:0/3865511473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b7c004800 0x7f1b8c151c20 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f1b8000f5f0 tx=0x7f1b8000fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:40.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.645+0000 7f1b897fa640 1 -- 172.21.15.43:0/3865511473 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b800052e0 con 0x7f1b7c004800 2024-04-25T08:58:40.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.645+0000 7f1b897fa640 1 -- 172.21.15.43:0/3865511473 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b80018070 con 0x7f1b7c004800 2024-04-25T08:58:40.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.645+0000 7f1b93189640 1 -- 172.21.15.43:0/3865511473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b8c0b6e50 con 0x7f1b7c004800 2024-04-25T08:58:40.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.645+0000 7f1b897fa640 1 -- 172.21.15.43:0/3865511473 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b80003810 con 0x7f1b7c004800 2024-04-25T08:58:40.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.645+0000 7f1b93189640 1 -- 172.21.15.43:0/3865511473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b8c1f3de0 con 0x7f1b7c004800 2024-04-25T08:58:40.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.646+0000 7f1b93189640 1 -- 172.21.15.43:0/3865511473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b8c074f60 con 0x7f1b7c004800 2024-04-25T08:58:40.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.647+0000 7f1b897fa640 1 -- 172.21.15.43:0/3865511473 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1b800039b0 con 0x7f1b7c004800 2024-04-25T08:58:40.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.647+0000 7f1b897fa640 1 --2- 172.21.15.43:0/3865511473 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1b7006b500 0x7f1b7006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:40.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.647+0000 7f1b897fa640 1 -- 172.21.15.43:0/3865511473 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f1b80089dd0 con 0x7f1b7c004800 2024-04-25T08:58:40.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.647+0000 7f1b8bfff640 1 --2- 172.21.15.43:0/3865511473 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1b7006b500 0x7f1b7006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:40.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.648+0000 7f1b8bfff640 1 --2- 172.21.15.43:0/3865511473 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1b7006b500 0x7f1b7006d9c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1b78032000 tx=0x7f1b7802d000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:40.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.652+0000 7f1b897fa640 1 -- 172.21.15.43:0/3865511473 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1b8c152490 con 0x7f1b7c004800 2024-04-25T08:58:40.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.830+0000 7f1b93189640 1 -- 172.21.15.43:0/3865511473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f1b8c08d7e0 con 0x7f1b7c004800 2024-04-25T08:58:40.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.833+0000 7f1b897fa640 1 -- 172.21.15.43:0/3865511473 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1b80058ea0 con 0x7f1b7c004800 2024-04-25T08:58:40.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.835+0000 7f1b93189640 1 -- 172.21.15.43:0/3865511473 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1b7006b500 msgr2=0x7f1b7006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:40.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.835+0000 7f1b93189640 1 --2- 172.21.15.43:0/3865511473 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1b7006b500 0x7f1b7006d9c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f1b78032000 tx=0x7f1b7802d000 comp rx=0 tx=0).stop 2024-04-25T08:58:40.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.835+0000 7f1b93189640 1 -- 172.21.15.43:0/3865511473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b7c004800 msgr2=0x7f1b8c151c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:40.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.835+0000 7f1b93189640 1 --2- 172.21.15.43:0/3865511473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b7c004800 0x7f1b8c151c20 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f1b8000f5f0 tx=0x7f1b8000fab0 comp rx=0 tx=0).stop 2024-04-25T08:58:40.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.835+0000 7f1b93189640 1 -- 172.21.15.43:0/3865511473 shutdown_connections 2024-04-25T08:58:40.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.835+0000 7f1b93189640 1 --2- 172.21.15.43:0/3865511473 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1b7006b500 0x7f1b7006d9c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:40.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.835+0000 7f1b93189640 1 --2- 172.21.15.43:0/3865511473 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1b8c1f48c0 0x7f1b8c0cc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:40.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.835+0000 7f1b93189640 1 --2- 172.21.15.43:0/3865511473 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1b8c0b6690 0x7f1b8c0b9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:40.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.835+0000 7f1b93189640 1 --2- 172.21.15.43:0/3865511473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1b7c004800 0x7f1b8c151c20 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:40.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.835+0000 7f1b93189640 1 -- 172.21.15.43:0/3865511473 >> 172.21.15.43:0/3865511473 conn(0x7f1b8c09cd40 msgr2=0x7f1b8c1606e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:40.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.835+0000 7f1b93189640 1 -- 172.21.15.43:0/3865511473 shutdown_connections 2024-04-25T08:58:40.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:40.835+0000 7f1b93189640 1 -- 172.21.15.43:0/3865511473 wait complete. 2024-04-25T08:58:40.837 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T08:58:40.837 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:58:40.882 DEBUG:teuthology.orchestra.run.smithi193:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T08:58:40.935 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T08:58:40.935 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:58:40.935 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:58:40.952 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2374 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:58:40.952 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T08:58:40.952 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:58:40.970 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19508 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:58:40.970 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:58:40.970 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:58:41.007 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2374 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T08:58:41.007 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T08:58:41.008 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:58:41.028 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19508 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T08:58:41.028 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T08:58:41.028 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:58:41.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.159+0000 7fe1e8e5a640 1 Processor -- start 2024-04-25T08:58:41.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.160+0000 7fe1e8e5a640 1 -- start start 2024-04-25T08:58:41.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.160+0000 7fe1e8e5a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe1e415dd00 0x7fe1e40d2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:41.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.160+0000 7fe1e8e5a640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fe1e40d33f0 0x7fe1e40ce260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:41.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.160+0000 7fe1e8e5a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe1e40ce7a0 0x7fe1e40b5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:41.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.160+0000 7fe1e8e5a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1e40d3850 con 0x7fe1e415dd00 2024-04-25T08:58:41.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.161+0000 7fe1e8e5a640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1e40d24e0 con 0x7fe1e40d33f0 2024-04-25T08:58:41.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.161+0000 7fe1e8e5a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e40cec30 con 0x7fe1e40ce7a0 2024-04-25T08:58:41.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.161+0000 7fe1e259c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe1e415dd00 0x7fe1e40d2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:41.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.161+0000 7fe1e259c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe1e415dd00 0x7fe1e40d2eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50296/0 (socket says 172.21.15.43:50296) 2024-04-25T08:58:41.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.161+0000 7fe1e259c640 1 -- 172.21.15.43:0/1367688272 learned_addr learned my addr 172.21.15.43:0/1367688272 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:41.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.161+0000 7fe1e1d9b640 1 --2- 172.21.15.43:0/1367688272 >> v2:172.21.15.193:3300/0 conn(0x7fe1e40d33f0 0x7fe1e40ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:41.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.161+0000 7fe1e2d9d640 1 --2- 172.21.15.43:0/1367688272 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe1e40ce7a0 0x7fe1e40b5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:41.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.161+0000 7fe1e259c640 1 -- 172.21.15.43:0/1367688272 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe1e40ce7a0 msgr2=0x7fe1e40b5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:41.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.161+0000 7fe1e259c640 1 --2- 172.21.15.43:0/1367688272 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe1e40ce7a0 0x7fe1e40b5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:41.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.161+0000 7fe1e259c640 1 -- 172.21.15.43:0/1367688272 >> v2:172.21.15.193:3300/0 conn(0x7fe1e40d33f0 msgr2=0x7fe1e40ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:41.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.161+0000 7fe1e259c640 1 --2- 172.21.15.43:0/1367688272 >> v2:172.21.15.193:3300/0 conn(0x7fe1e40d33f0 0x7fe1e40ce260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:41.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.161+0000 7fe1e259c640 1 -- 172.21.15.43:0/1367688272 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e415e0e0 con 0x7fe1e415dd00 2024-04-25T08:58:41.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.162+0000 7fe1e259c640 1 --2- 172.21.15.43:0/1367688272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1e415dd00 0x7fe1e40d2eb0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fe1e408cbb0 tx=0x7fe1dc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8df18a7521ae41f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:41.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.162+0000 7fe1e159a640 1 -- 172.21.15.43:0/1367688272 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc037070 con 0x7fe1e415dd00 2024-04-25T08:58:41.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.162+0000 7fe1e159a640 1 -- 172.21.15.43:0/1367688272 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1dc032440 con 0x7fe1e415dd00 2024-04-25T08:58:41.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.162+0000 7fe1e159a640 1 -- 172.21.15.43:0/1367688272 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1dc032810 con 0x7fe1e415dd00 2024-04-25T08:58:41.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.162+0000 7fe1e8e5a640 1 -- 172.21.15.43:0/1367688272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1e415dd00 msgr2=0x7fe1e40d2eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:41.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.162+0000 7fe1e8e5a640 1 --2- 172.21.15.43:0/1367688272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1e415dd00 0x7fe1e40d2eb0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fe1e408cbb0 tx=0x7fe1dc02a1e0 comp rx=0 tx=0).stop 2024-04-25T08:58:41.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.162+0000 7fe1e8e5a640 1 -- 172.21.15.43:0/1367688272 shutdown_connections 2024-04-25T08:58:41.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.162+0000 7fe1e8e5a640 1 --2- 172.21.15.43:0/1367688272 >> v2:172.21.15.193:3300/0 conn(0x7fe1e40d33f0 0x7fe1e40ce260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:41.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.162+0000 7fe1e8e5a640 1 --2- 172.21.15.43:0/1367688272 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe1e40ce7a0 0x7fe1e40b5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:41.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.162+0000 7fe1e8e5a640 1 --2- 172.21.15.43:0/1367688272 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1e415dd00 0x7fe1e40d2eb0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:41.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.162+0000 7fe1e8e5a640 1 -- 172.21.15.43:0/1367688272 >> 172.21.15.43:0/1367688272 conn(0x7fe1e406cbd0 msgr2=0x7fe1e406cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:41.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.163+0000 7fe1e8e5a640 1 -- 172.21.15.43:0/1367688272 shutdown_connections 2024-04-25T08:58:41.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.163+0000 7fe1e8e5a640 1 -- 172.21.15.43:0/1367688272 wait complete. 2024-04-25T08:58:41.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.164+0000 7fe1e8e5a640 1 Processor -- start 2024-04-25T08:58:41.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.164+0000 7fe1e8e5a640 1 -- start start 2024-04-25T08:58:41.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.165+0000 7fe1e8e5a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1e40ce7a0 0x7fe1e40b9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:41.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.165+0000 7fe1e8e5a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1e40d33f0 0x7fe1e414cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:41.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.165+0000 7fe1e8e5a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe1e415dd00 0x7fe1e40b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:41.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.165+0000 7fe1e8e5a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e40d1e90 con 0x7fe1e40d33f0 2024-04-25T08:58:41.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.165+0000 7fe1e8e5a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e40d1bb0 con 0x7fe1e40ce7a0 2024-04-25T08:58:41.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.165+0000 7fe1e8e5a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e40d2170 con 0x7fe1e415dd00 2024-04-25T08:58:41.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.165+0000 7fe1e1d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1e40d33f0 0x7fe1e414cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:41.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.165+0000 7fe1e1d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1e40d33f0 0x7fe1e414cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50310/0 (socket says 172.21.15.43:50310) 2024-04-25T08:58:41.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.165+0000 7fe1e1d9b640 1 -- 172.21.15.43:0/1870189877 learned_addr learned my addr 172.21.15.43:0/1870189877 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:41.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.165+0000 7fe1e259c640 1 --2- 172.21.15.43:0/1870189877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1e40ce7a0 0x7fe1e40b9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:41.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.165+0000 7fe1e2d9d640 1 --2- 172.21.15.43:0/1870189877 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe1e415dd00 0x7fe1e40b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:41.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.165+0000 7fe1e1d9b640 1 -- 172.21.15.43:0/1870189877 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe1e415dd00 msgr2=0x7fe1e40b3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:41.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.165+0000 7fe1e1d9b640 1 --2- 172.21.15.43:0/1870189877 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe1e415dd00 0x7fe1e40b3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:41.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.165+0000 7fe1e1d9b640 1 -- 172.21.15.43:0/1870189877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1e40ce7a0 msgr2=0x7fe1e40b9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:41.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.165+0000 7fe1e1d9b640 1 --2- 172.21.15.43:0/1870189877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1e40ce7a0 0x7fe1e40b9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:41.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.165+0000 7fe1e1d9b640 1 -- 172.21.15.43:0/1870189877 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e41f6c90 con 0x7fe1e40d33f0 2024-04-25T08:58:41.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.165+0000 7fe1e1d9b640 1 --2- 172.21.15.43:0/1870189877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1e40d33f0 0x7fe1e414cfd0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc00d800 tx=0x7fe1cc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:41.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.166+0000 7fe1daffd640 1 -- 172.21.15.43:0/1870189877 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc00b6c0 con 0x7fe1e40d33f0 2024-04-25T08:58:41.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.166+0000 7fe1daffd640 1 -- 172.21.15.43:0/1870189877 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1cc019070 con 0x7fe1e40d33f0 2024-04-25T08:58:41.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.166+0000 7fe1daffd640 1 -- 172.21.15.43:0/1870189877 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc00ba80 con 0x7fe1e40d33f0 2024-04-25T08:58:41.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.166+0000 7fe1e8e5a640 1 -- 172.21.15.43:0/1870189877 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1e40b6d00 con 0x7fe1e40d33f0 2024-04-25T08:58:41.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.166+0000 7fe1e8e5a640 1 -- 172.21.15.43:0/1870189877 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1e414d510 con 0x7fe1e40d33f0 2024-04-25T08:58:41.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.167+0000 7fe1e8e5a640 1 -- 172.21.15.43:0/1870189877 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1e40c72f0 con 0x7fe1e40d33f0 2024-04-25T08:58:41.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.167+0000 7fe1daffd640 1 -- 172.21.15.43:0/1870189877 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe1cc00bc20 con 0x7fe1e40d33f0 2024-04-25T08:58:41.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.168+0000 7fe1daffd640 1 --2- 172.21.15.43:0/1870189877 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe1c406b500 0x7fe1c406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:41.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.168+0000 7fe1daffd640 1 -- 172.21.15.43:0/1870189877 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5115+0+0 (secure 0 0 0) 0x7fe1cc08a2d0 con 0x7fe1e40d33f0 2024-04-25T08:58:41.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.168+0000 7fe1e259c640 1 --2- 172.21.15.43:0/1870189877 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe1c406b500 0x7fe1c406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:41.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.169+0000 7fe1e259c640 1 --2- 172.21.15.43:0/1870189877 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe1c406b500 0x7fe1c406d9c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc034000 tx=0x7fe1dc02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:41.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.172+0000 7fe1daffd640 1 -- 172.21.15.43:0/1870189877 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe1e40c72f0 con 0x7fe1e40d33f0 2024-04-25T08:58:41.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.387+0000 7fe1e8e5a640 1 -- 172.21.15.43:0/1870189877 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1e4074ef0 con 0x7fe1e40d33f0 2024-04-25T08:58:41.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.387+0000 7fe1daffd640 1 -- 172.21.15.43:0/1870189877 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+3696 (secure 0 0 0) 0x7fe1cc08a0d0 con 0x7fe1e40d33f0 2024-04-25T08:58:41.377 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:58:41.377 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":41,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5765,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6835/594587590","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":594587590},{"type":"v1","addr":"172.21.15.193:6835","nonce":594587590}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6187,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/681300720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6836","nonce":681300720},{"type":"v1","addr":"172.21.15.193:6837","nonce":681300720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35},{"gid":6876,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/432022866","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":432022866},{"type":"v1","addr":"172.21.15.43:6835","nonce":432022866}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":7071,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/2970409621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2970409621},{"type":"v1","addr":"172.21.15.43:6833","nonce":2970409621}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":41}],"filesystems":[]} 2024-04-25T08:58:41.377 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 41 2024-04-25T08:58:41.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.389+0000 7fe1e8e5a640 1 -- 172.21.15.43:0/1870189877 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe1c406b500 msgr2=0x7fe1c406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:41.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.389+0000 7fe1e8e5a640 1 --2- 172.21.15.43:0/1870189877 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe1c406b500 0x7fe1c406d9c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc034000 tx=0x7fe1dc02e000 comp rx=0 tx=0).stop 2024-04-25T08:58:41.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.389+0000 7fe1e8e5a640 1 -- 172.21.15.43:0/1870189877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1e40d33f0 msgr2=0x7fe1e414cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:41.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.389+0000 7fe1e8e5a640 1 --2- 172.21.15.43:0/1870189877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1e40d33f0 0x7fe1e414cfd0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc00d800 tx=0x7fe1cc00dcc0 comp rx=0 tx=0).stop 2024-04-25T08:58:41.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.389+0000 7fe1e8e5a640 1 -- 172.21.15.43:0/1870189877 shutdown_connections 2024-04-25T08:58:41.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.389+0000 7fe1e8e5a640 1 --2- 172.21.15.43:0/1870189877 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe1e415dd00 0x7fe1e40b3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:41.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.389+0000 7fe1e8e5a640 1 --2- 172.21.15.43:0/1870189877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe1e40d33f0 0x7fe1e414cfd0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:41.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.389+0000 7fe1e8e5a640 1 --2- 172.21.15.43:0/1870189877 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe1c406b500 0x7fe1c406d9c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:41.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.389+0000 7fe1e8e5a640 1 --2- 172.21.15.43:0/1870189877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe1e40ce7a0 0x7fe1e40b9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:41.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.389+0000 7fe1e8e5a640 1 -- 172.21.15.43:0/1870189877 >> 172.21.15.43:0/1870189877 conn(0x7fe1e406cbd0 msgr2=0x7fe1e40b26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:41.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.389+0000 7fe1e8e5a640 1 -- 172.21.15.43:0/1870189877 shutdown_connections 2024-04-25T08:58:41.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.389+0000 7fe1e8e5a640 1 -- 172.21.15.43:0/1870189877 wait complete. 2024-04-25T08:58:41.390 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T08:58:41.390 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T08:58:41.390 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T08:58:41.390 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:58:41.390 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T08:58:41.390 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T08:58:41.391 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T08:58:41.391 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:58:41.391 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T08:58:41.391 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T08:58:41.391 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T08:58:41.391 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:58:41.392 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T08:58:41.392 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T08:58:41.392 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T08:58:41.392 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T08:58:41.402 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T08:58:41.402 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T08:58:41.403 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T08:58:41.403 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T08:58:41.404 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T08:58:41.404 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T08:58:41.405 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T08:58:41.405 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T08:58:41.445 INFO:tasks.ceph.mds.a:Started 2024-04-25T08:58:41.445 DEBUG:teuthology.parallel:result is None 2024-04-25T08:58:41.446 INFO:tasks.ceph.mds.c:Started 2024-04-25T08:58:41.447 DEBUG:teuthology.parallel:result is None 2024-04-25T08:58:41.448 INFO:tasks.ceph.mds.b:Started 2024-04-25T08:58:41.448 DEBUG:teuthology.parallel:result is None 2024-04-25T08:58:41.448 INFO:tasks.ceph.mds.d:Started 2024-04-25T08:58:41.449 DEBUG:teuthology.parallel:result is None 2024-04-25T08:58:41.449 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T08:58:41.521 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2024-04-25T08:58:41.521 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-04-25T08:58:41.528 INFO:tasks.ceph.mds.b.smithi193.stdout:starting mds.b at 2024-04-25T08:58:41.528 INFO:tasks.ceph.mds.d.smithi193.stdout:starting mds.d at 2024-04-25T08:58:41.529 INFO:tasks.ceph.mds.b.smithi193.stderr:2024-04-25T08:58:41.540+0000 7fdac2528640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6836/0: (98) Address already in use 2024-04-25T08:58:41.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.567+0000 7f67e43f4640 1 Processor -- start 2024-04-25T08:58:41.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.568+0000 7f67e43f4640 1 -- start start 2024-04-25T08:58:41.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.568+0000 7f67e43f4640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f67dc15f180 0x7f67dc0cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:41.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.568+0000 7f67e43f4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f67dc0b2cc0 0x7f67dc0cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:41.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.568+0000 7f67e43f4640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67dc0ccf10 con 0x7f67dc0b23f0 2024-04-25T08:58:41.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.568+0000 7f67e43f4640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67dc0cccc0 con 0x7f67dc0b2cc0 2024-04-25T08:58:41.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.568+0000 7f67e43f4640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67dc0cd840 con 0x7f67dc15f180 2024-04-25T08:58:41.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.569+0000 7f67e2991640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f67dc0b2cc0 0x7f67dc0cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:41.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.569+0000 7f67e198f640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f67dc0b23f0 0x7f67dc15f8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39148/0 (socket says 172.21.15.43:39148) 2024-04-25T08:58:41.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.569+0000 7f67e198f640 1 -- 172.21.15.43:0/967679219 learned_addr learned my addr 172.21.15.43:0/967679219 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:41.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.569+0000 7f67e2991640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f67dc0b2cc0 0x7f67dc0cfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50360/0 (socket says 172.21.15.43:50360) 2024-04-25T08:58:41.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.569+0000 7f67e2190640 1 --2- 172.21.15.43:0/967679219 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f67dc15f180 0x7f67dc0cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:41.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.569+0000 7f67e2991640 1 -- 172.21.15.43:0/967679219 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f67dc15f180 msgr2=0x7f67dc0cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:41.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.569+0000 7f67e2991640 1 --2- 172.21.15.43:0/967679219 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f67dc15f180 0x7f67dc0cf860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:41.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.569+0000 7f67e2991640 1 -- 172.21.15.43:0/967679219 >> v1:172.21.15.43:6789/0 conn(0x7f67dc0b23f0 legacy=0x7f67dc15f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:41.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.569+0000 7f67e2991640 1 -- 172.21.15.43:0/967679219 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67dc0cd1f0 con 0x7f67dc0b2cc0 2024-04-25T08:58:41.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.569+0000 7f67e118e640 1 -- 172.21.15.43:0/967679219 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 871803273 0 0) 0x7f67dc0ccf10 con 0x7f67dc0b23f0 2024-04-25T08:58:41.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.569+0000 7f67e2190640 1 --2- 172.21.15.43:0/967679219 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f67dc15f180 0x7f67dc0cf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:41.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.570+0000 7f67e2991640 1 --2- 172.21.15.43:0/967679219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67dc0b2cc0 0x7f67dc0cfda0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f67d4009d90 tx=0x7f67d402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=988cabb7148aafeb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:41.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.570+0000 7f67e118e640 1 -- 172.21.15.43:0/967679219 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d4031600 con 0x7f67dc0b2cc0 2024-04-25T08:58:41.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.570+0000 7f67e118e640 1 -- 172.21.15.43:0/967679219 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d40317a0 con 0x7f67dc0b2cc0 2024-04-25T08:58:41.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.570+0000 7f67e118e640 1 -- 172.21.15.43:0/967679219 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d4031b70 con 0x7f67dc0b2cc0 2024-04-25T08:58:41.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.570+0000 7f67e43f4640 1 -- 172.21.15.43:0/967679219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67dc0b2cc0 msgr2=0x7f67dc0cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:41.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.570+0000 7f67e43f4640 1 --2- 172.21.15.43:0/967679219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67dc0b2cc0 0x7f67dc0cfda0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f67d4009d90 tx=0x7f67d402a5d0 comp rx=0 tx=0).stop 2024-04-25T08:58:41.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.570+0000 7f67e43f4640 1 -- 172.21.15.43:0/967679219 shutdown_connections 2024-04-25T08:58:41.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.570+0000 7f67e43f4640 1 --2- 172.21.15.43:0/967679219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67dc0b2cc0 0x7f67dc0cfda0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:41.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.570+0000 7f67e43f4640 1 --2- 172.21.15.43:0/967679219 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f67dc15f180 0x7f67dc0cf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:41.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.570+0000 7f67e43f4640 1 -- 172.21.15.43:0/967679219 >> 172.21.15.43:0/967679219 conn(0x7f67dc0d17f0 msgr2=0x7f67dc0d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:41.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.570+0000 7f67e43f4640 1 -- 172.21.15.43:0/967679219 shutdown_connections 2024-04-25T08:58:41.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.570+0000 7f67e43f4640 1 -- 172.21.15.43:0/967679219 wait complete. 2024-04-25T08:58:41.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.572+0000 7f67e43f4640 1 Processor -- start 2024-04-25T08:58:41.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.572+0000 7f67e43f4640 1 -- start start 2024-04-25T08:58:41.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.572+0000 7f67e43f4640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f67dc0b23f0 0x7f67dc158c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:41.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.572+0000 7f67e43f4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67dc0b2cc0 0x7f67dc15bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:41.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.572+0000 7f67e43f4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f67dc15f180 0x7f67dc15c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:41.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.572+0000 7f67e43f4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67dc15f560 con 0x7f67dc0b2cc0 2024-04-25T08:58:41.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.572+0000 7f67e43f4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67dc0cece0 con 0x7f67dc15f180 2024-04-25T08:58:41.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.572+0000 7f67e43f4640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67dc15ae40 con 0x7f67dc0b23f0 2024-04-25T08:58:41.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.572+0000 7f67e198f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67dc0b2cc0 0x7f67dc15bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:41.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.573+0000 7f67e198f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67dc0b2cc0 0x7f67dc15bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50370/0 (socket says 172.21.15.43:50370) 2024-04-25T08:58:41.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.573+0000 7f67e198f640 1 -- 172.21.15.43:0/1890050018 learned_addr learned my addr 172.21.15.43:0/1890050018 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:41.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.573+0000 7f67e2190640 1 --2- 172.21.15.43:0/1890050018 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f67dc0b23f0 0x7f67dc158c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:41.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.573+0000 7f67e2991640 1 --2- 172.21.15.43:0/1890050018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f67dc15f180 0x7f67dc15c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:41.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.573+0000 7f67e198f640 1 -- 172.21.15.43:0/1890050018 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f67dc0b23f0 msgr2=0x7f67dc158c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:41.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.573+0000 7f67e198f640 1 --2- 172.21.15.43:0/1890050018 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f67dc0b23f0 0x7f67dc158c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:41.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.573+0000 7f67e198f640 1 -- 172.21.15.43:0/1890050018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f67dc15f180 msgr2=0x7f67dc15c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:41.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.573+0000 7f67e198f640 1 --2- 172.21.15.43:0/1890050018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f67dc15f180 0x7f67dc15c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:41.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.573+0000 7f67e198f640 1 -- 172.21.15.43:0/1890050018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67dc1f5a50 con 0x7f67dc0b2cc0 2024-04-25T08:58:41.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.573+0000 7f67e198f640 1 --2- 172.21.15.43:0/1890050018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67dc0b2cc0 0x7f67dc15bb70 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f67d800c4d0 tx=0x7f67d800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:41.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.573+0000 7f67d2ffd640 1 -- 172.21.15.43:0/1890050018 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d8018070 con 0x7f67dc0b2cc0 2024-04-25T08:58:41.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.573+0000 7f67d2ffd640 1 -- 172.21.15.43:0/1890050018 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d8007500 con 0x7f67dc0b2cc0 2024-04-25T08:58:41.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.573+0000 7f67e43f4640 1 -- 172.21.15.43:0/1890050018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67dc15b090 con 0x7f67dc0b2cc0 2024-04-25T08:58:41.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.573+0000 7f67d2ffd640 1 -- 172.21.15.43:0/1890050018 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d8004a20 con 0x7f67dc0b2cc0 2024-04-25T08:58:41.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.573+0000 7f67e43f4640 1 -- 172.21.15.43:0/1890050018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67dc15b650 con 0x7f67dc0b2cc0 2024-04-25T08:58:41.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.574+0000 7f67e43f4640 1 -- 172.21.15.43:0/1890050018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67dc074ef0 con 0x7f67dc0b2cc0 2024-04-25T08:58:41.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.575+0000 7f67d2ffd640 1 -- 172.21.15.43:0/1890050018 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f67d801b070 con 0x7f67dc0b2cc0 2024-04-25T08:58:41.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.575+0000 7f67d2ffd640 1 --2- 172.21.15.43:0/1890050018 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f67c006b4d0 0x7f67c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:41.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.575+0000 7f67d2ffd640 1 -- 172.21.15.43:0/1890050018 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5115+0+0 (secure 0 0 0) 0x7f67d8089200 con 0x7f67dc0b2cc0 2024-04-25T08:58:41.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.575+0000 7f67e2190640 1 --2- 172.21.15.43:0/1890050018 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f67c006b4d0 0x7f67c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:41.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.576+0000 7f67e2190640 1 --2- 172.21.15.43:0/1890050018 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f67c006b4d0 0x7f67c006d990 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f67dc09f8f0 tx=0x7f67cc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:41.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.579+0000 7f67d2ffd640 1 -- 172.21.15.43:0/1890050018 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f67d8057c80 con 0x7f67dc0b2cc0 2024-04-25T08:58:41.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:41.772+0000 7f67e43f4640 1 -- 172.21.15.43:0/1890050018 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f67dc15b820 con 0x7f67dc0b2cc0 2024-04-25T08:58:42.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.523+0000 7f67d2ffd640 1 -- 172.21.15.43:0/1890050018 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v67) v1 ==== 97+0+0 (secure 0 0 0) 0x7f67d805bd20 con 0x7f67dc0b2cc0 2024-04-25T08:58:42.513 INFO:teuthology.orchestra.run.smithi043.stderr: removed all blocklist entries 2024-04-25T08:58:42.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.526+0000 7f67e43f4640 1 -- 172.21.15.43:0/1890050018 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f67c006b4d0 msgr2=0x7f67c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:42.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.526+0000 7f67e43f4640 1 --2- 172.21.15.43:0/1890050018 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f67c006b4d0 0x7f67c006d990 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f67dc09f8f0 tx=0x7f67cc008040 comp rx=0 tx=0).stop 2024-04-25T08:58:42.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.526+0000 7f67e43f4640 1 -- 172.21.15.43:0/1890050018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67dc0b2cc0 msgr2=0x7f67dc15bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:42.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.526+0000 7f67e43f4640 1 --2- 172.21.15.43:0/1890050018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67dc0b2cc0 0x7f67dc15bb70 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f67d800c4d0 tx=0x7f67d800c990 comp rx=0 tx=0).stop 2024-04-25T08:58:42.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.526+0000 7f67e43f4640 1 -- 172.21.15.43:0/1890050018 shutdown_connections 2024-04-25T08:58:42.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.526+0000 7f67e43f4640 1 --2- 172.21.15.43:0/1890050018 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f67c006b4d0 0x7f67c006d990 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:42.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.526+0000 7f67e43f4640 1 --2- 172.21.15.43:0/1890050018 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f67dc15f180 0x7f67dc15c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:42.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.526+0000 7f67e43f4640 1 --2- 172.21.15.43:0/1890050018 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f67dc0b2cc0 0x7f67dc15bb70 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:42.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.526+0000 7f67e43f4640 1 --2- 172.21.15.43:0/1890050018 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f67dc0b23f0 0x7f67dc158c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:42.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.526+0000 7f67e43f4640 1 -- 172.21.15.43:0/1890050018 >> 172.21.15.43:0/1890050018 conn(0x7f67dc0d17f0 msgr2=0x7f67dc0b7030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:42.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.526+0000 7f67e43f4640 1 -- 172.21.15.43:0/1890050018 shutdown_connections 2024-04-25T08:58:42.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.526+0000 7f67e43f4640 1 -- 172.21.15.43:0/1890050018 wait complete. 2024-04-25T08:58:42.530 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T08:58:42.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.648+0000 7fec49c58640 1 Processor -- start 2024-04-25T08:58:42.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.649+0000 7fec49c58640 1 -- start start 2024-04-25T08:58:42.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.649+0000 7fec49c58640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fec440d39d0 0x7fec440d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:42.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.649+0000 7fec49c58640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec440b6c50 con 0x7fec440ce2a0 2024-04-25T08:58:42.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.649+0000 7fec49c58640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec4415e010 con 0x7fec440d2500 2024-04-25T08:58:42.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.649+0000 7fec49c58640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec440d2020 con 0x7fec440d39d0 2024-04-25T08:58:42.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.649+0000 7fec42ffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fec440ce2a0 0x7fec440d3db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39164/0 (socket says 172.21.15.43:39164) 2024-04-25T08:58:42.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.650+0000 7fec42ffd640 1 -- 172.21.15.43:0/228324485 learned_addr learned my addr 172.21.15.43:0/228324485 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:42.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.650+0000 7fec437fe640 1 --2- 172.21.15.43:0/228324485 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fec440d39d0 0x7fec440d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:42.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.650+0000 7fec427fc640 1 -- 172.21.15.43:0/228324485 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2483552118 0 0) 0x7fec38001110 con 0x7fec440ce2a0 2024-04-25T08:58:42.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.650+0000 7fec427fc640 1 -- 172.21.15.43:0/228324485 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec2c003520 con 0x7fec440ce2a0 2024-04-25T08:58:42.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.650+0000 7fec427fc640 1 -- 172.21.15.43:0/228324485 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3458578106 0 0) 0x7fec4415e010 con 0x7fec440d2500 2024-04-25T08:58:42.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.650+0000 7fec427fc640 1 -- 172.21.15.43:0/228324485 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec38001110 con 0x7fec440d2500 2024-04-25T08:58:42.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.650+0000 7fec437fe640 1 -- 172.21.15.43:0/228324485 >> v1:172.21.15.193:6789/0 conn(0x7fec440d2500 legacy=0x7fec440d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:42.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.650+0000 7fec427fc640 1 -- 172.21.15.43:0/228324485 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 885376791 0 0) 0x7fec2c003520 con 0x7fec440ce2a0 2024-04-25T08:58:42.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.650+0000 7fec437fe640 1 -- 172.21.15.43:0/228324485 >> v1:172.21.15.43:6789/0 conn(0x7fec440ce2a0 legacy=0x7fec440d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:42.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.650+0000 7fec437fe640 1 -- 172.21.15.43:0/228324485 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec440cec40 con 0x7fec440d39d0 2024-04-25T08:58:42.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.651+0000 7fec437fe640 1 --2- 172.21.15.43:0/228324485 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fec440d39d0 0x7fec440d0d90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fec340097d0 tx=0x7fec3402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49e7d6f780061c73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:42.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.651+0000 7fec427fc640 1 -- 172.21.15.43:0/228324485 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec34037070 con 0x7fec440d39d0 2024-04-25T08:58:42.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.651+0000 7fec427fc640 1 -- 172.21.15.43:0/228324485 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec3402ab90 con 0x7fec440d39d0 2024-04-25T08:58:42.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.651+0000 7fec427fc640 1 -- 172.21.15.43:0/228324485 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec38001840 con 0x7fec440d39d0 2024-04-25T08:58:42.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.652+0000 7fec49c58640 1 -- 172.21.15.43:0/228324485 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fec440d39d0 msgr2=0x7fec440d0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:42.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.652+0000 7fec49c58640 1 --2- 172.21.15.43:0/228324485 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fec440d39d0 0x7fec440d0d90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fec340097d0 tx=0x7fec3402a100 comp rx=0 tx=0).stop 2024-04-25T08:58:42.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.652+0000 7fec49c58640 1 -- 172.21.15.43:0/228324485 shutdown_connections 2024-04-25T08:58:42.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.652+0000 7fec49c58640 1 --2- 172.21.15.43:0/228324485 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fec440d39d0 0x7fec440d0d90 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:42.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.652+0000 7fec49c58640 1 -- 172.21.15.43:0/228324485 >> 172.21.15.43:0/228324485 conn(0x7fec4406b000 msgr2=0x7fec4406b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:42.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.652+0000 7fec49c58640 1 -- 172.21.15.43:0/228324485 shutdown_connections 2024-04-25T08:58:42.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.652+0000 7fec49c58640 1 -- 172.21.15.43:0/228324485 wait complete. 2024-04-25T08:58:42.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.653+0000 7fec49c58640 1 Processor -- start 2024-04-25T08:58:42.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.653+0000 7fec49c58640 1 -- start start 2024-04-25T08:58:42.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.654+0000 7fec49c58640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fec440ce2a0 0x7fec441f4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:42.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.654+0000 7fec49c58640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec440d2500 0x7fec440c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:42.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.654+0000 7fec49c58640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec440d39d0 0x7fec440c7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:42.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.654+0000 7fec49c58640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec440cf200 con 0x7fec440d2500 2024-04-25T08:58:42.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.654+0000 7fec49c58640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec440b2f00 con 0x7fec440d39d0 2024-04-25T08:58:42.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.654+0000 7fec49c58640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec440d30a0 con 0x7fec440ce2a0 2024-04-25T08:58:42.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.654+0000 7fec42ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec440d2500 0x7fec440c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:42.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.654+0000 7fec42ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec440d2500 0x7fec440c9ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50384/0 (socket says 172.21.15.43:50384) 2024-04-25T08:58:42.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.654+0000 7fec42ffd640 1 -- 172.21.15.43:0/1116279396 learned_addr learned my addr 172.21.15.43:0/1116279396 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:42.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.654+0000 7fec437fe640 1 --2- 172.21.15.43:0/1116279396 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fec440ce2a0 0x7fec441f4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:42.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.654+0000 7fec43fff640 1 --2- 172.21.15.43:0/1116279396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec440d39d0 0x7fec440c7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:42.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.654+0000 7fec42ffd640 1 -- 172.21.15.43:0/1116279396 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fec440ce2a0 msgr2=0x7fec441f4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:42.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.654+0000 7fec42ffd640 1 --2- 172.21.15.43:0/1116279396 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fec440ce2a0 0x7fec441f4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:42.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.654+0000 7fec42ffd640 1 -- 172.21.15.43:0/1116279396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec440d39d0 msgr2=0x7fec440c7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:42.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.654+0000 7fec42ffd640 1 --2- 172.21.15.43:0/1116279396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec440d39d0 0x7fec440c7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:42.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.654+0000 7fec42ffd640 1 -- 172.21.15.43:0/1116279396 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec440ca010 con 0x7fec440d2500 2024-04-25T08:58:42.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.654+0000 7fec43fff640 1 --2- 172.21.15.43:0/1116279396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec440d39d0 0x7fec440c7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:58:42.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.654+0000 7fec42ffd640 1 --2- 172.21.15.43:0/1116279396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec440d2500 0x7fec440c9ad0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fec38000fd0 tx=0x7fec3800cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:42.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.655+0000 7fec2bfff640 1 -- 172.21.15.43:0/1116279396 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec38017070 con 0x7fec440d2500 2024-04-25T08:58:42.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.655+0000 7fec2bfff640 1 -- 172.21.15.43:0/1116279396 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec38005d00 con 0x7fec440d2500 2024-04-25T08:58:42.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.655+0000 7fec2bfff640 1 -- 172.21.15.43:0/1116279396 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec380072b0 con 0x7fec440d2500 2024-04-25T08:58:42.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.655+0000 7fec49c58640 1 -- 172.21.15.43:0/1116279396 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec440ca2c0 con 0x7fec440d2500 2024-04-25T08:58:42.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.655+0000 7fec49c58640 1 -- 172.21.15.43:0/1116279396 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec440b4b80 con 0x7fec440d2500 2024-04-25T08:58:42.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.656+0000 7fec49c58640 1 -- 172.21.15.43:0/1116279396 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec440b46d0 con 0x7fec440d2500 2024-04-25T08:58:42.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.656+0000 7fec2bfff640 1 -- 172.21.15.43:0/1116279396 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fec3801a070 con 0x7fec440d2500 2024-04-25T08:58:42.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.657+0000 7fec2bfff640 1 --2- 172.21.15.43:0/1116279396 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fec3006b4d0 0x7fec3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:42.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.657+0000 7fec2bfff640 1 -- 172.21.15.43:0/1116279396 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fec38089450 con 0x7fec440d2500 2024-04-25T08:58:42.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.657+0000 7fec437fe640 1 --2- 172.21.15.43:0/1116279396 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fec3006b4d0 0x7fec3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:42.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.658+0000 7fec437fe640 1 --2- 172.21.15.43:0/1116279396 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fec3006b4d0 0x7fec3006d990 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fec34034000 tx=0x7fec3402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:42.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.661+0000 7fec2bfff640 1 -- 172.21.15.43:0/1116279396 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fec440b6c50 con 0x7fec440d2500 2024-04-25T08:58:42.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.908+0000 7fec49c58640 1 -- 172.21.15.43:0/1116279396 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fec44074ef0 con 0x7fec440d2500 2024-04-25T08:58:42.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.909+0000 7fec2bfff640 1 -- 172.21.15.43:0/1116279396 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fec38058060 con 0x7fec440d2500 2024-04-25T08:58:42.898 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:58:42.898 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T08:58:42.898 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_dump": [ 2024-04-25T08:58:42.898 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:58:42.898 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.a", 2024-04-25T08:58:42.898 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpmIjzEBRAAs2L10XrFO3mThjZevna5wA==", 2024-04-25T08:58:42.899 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:58:42.899 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T08:58:42.899 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T08:58:42.899 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T08:58:42.899 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:58:42.899 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:58:42.899 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:58:42.899 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:58:42.899 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.b", 2024-04-25T08:58:42.899 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm51cmERAAij4vy8xyihh8u/4dkmcllw==", 2024-04-25T08:58:42.899 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:58:42.899 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T08:58:42.899 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T08:58:42.899 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T08:58:42.900 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:58:42.900 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:58:42.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:58:42.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:58:42.900 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.c", 2024-04-25T08:58:42.900 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm1OOtDBAAlUYbCh1mBEEka9CWRnUz9A==", 2024-04-25T08:58:42.900 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:58:42.900 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T08:58:42.900 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T08:58:42.900 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T08:58:42.900 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:58:42.900 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:58:42.900 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:58:42.900 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:58:42.900 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.d", 2024-04-25T08:58:42.901 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpmpEvsFxAAXX1h37UWdFkyJ3Iw3xd8kw==", 2024-04-25T08:58:42.901 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:58:42.901 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T08:58:42.901 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T08:58:42.901 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T08:58:42.901 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:58:42.901 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:58:42.901 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:58:42.901 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:58:42.901 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.0", 2024-04-25T08:58:42.901 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD9GSpmi7qkORAAwSNgwzcVMitrPPf3nDyqvQ==", 2024-04-25T08:58:42.901 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:58:42.901 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T08:58:42.902 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T08:58:42.902 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:58:42.902 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:58:42.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:58:42.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:58:42.902 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.1", 2024-04-25T08:58:42.902 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAAGipmJDT3NxAAcWBCd9nt7Au+Xftjqcuc5w==", 2024-04-25T08:58:42.902 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:58:42.902 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T08:58:42.902 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T08:58:42.902 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:58:42.902 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:58:42.902 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:58:42.902 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:58:42.903 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.2", 2024-04-25T08:58:42.903 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQADGipm68kzNhAA0ZOnlr4edWDjx6RfoEHYIw==", 2024-04-25T08:58:42.903 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:58:42.903 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T08:58:42.903 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T08:58:42.903 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:58:42.903 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:58:42.903 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:58:42.903 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:58:42.903 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.3", 2024-04-25T08:58:42.903 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAGGipmT1AeNBAAoNPiEtkznSxxhVJ1EmZ1ZA==", 2024-04-25T08:58:42.903 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:58:42.903 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T08:58:42.903 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T08:58:42.904 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:58:42.904 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:58:42.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:58:42.904 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:58:42.904 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.4", 2024-04-25T08:58:42.904 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAMGipmBjHHNxAAH/SCzKHus8fKxxI3UN4wWQ==", 2024-04-25T08:58:42.904 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:58:42.904 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T08:58:42.904 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T08:58:42.904 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:58:42.904 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:58:42.904 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:58:42.904 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:58:42.904 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.5", 2024-04-25T08:58:42.904 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPGipm3FXKNRAAL97YYWONg4QHr4ajC5D7fA==", 2024-04-25T08:58:42.905 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:58:42.905 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T08:58:42.905 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T08:58:42.905 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:58:42.905 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:58:42.905 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:58:42.905 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:58:42.905 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.6", 2024-04-25T08:58:42.905 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQASGipmCwKaMxAAcM0p9Q2sbNQviGkwSLg8AA==", 2024-04-25T08:58:42.905 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:58:42.905 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T08:58:42.905 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T08:58:42.905 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:58:42.905 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:58:42.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:58:42.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:58:42.906 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.7", 2024-04-25T08:58:42.906 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAVGipmP7A2MRAAxnFJFosMcbZ6VDbgH8r6wA==", 2024-04-25T08:58:42.906 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:58:42.906 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T08:58:42.906 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T08:58:42.906 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:58:42.906 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:58:42.906 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:58:42.906 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:58:42.906 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.0", 2024-04-25T08:58:42.906 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpmVOjWGxAAFNmxeGi0h2D//pa6Ne3eQw==", 2024-04-25T08:58:42.906 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:58:42.907 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T08:58:42.907 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow r", 2024-04-25T08:58:42.907 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T08:58:42.907 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:58:42.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:58:42.907 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:58:42.907 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.1", 2024-04-25T08:58:42.907 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm24laJBAA9ijbDlFDTmF/ZZ4IhLfBVQ==", 2024-04-25T08:58:42.907 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:58:42.907 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T08:58:42.907 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-04-25T08:58:42.907 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-04-25T08:58:42.907 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-04-25T08:58:42.907 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:58:42.907 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:58:42.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:58:42.908 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.2", 2024-04-25T08:58:42.908 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm8b5UIRAACcOwDL630oJrXfp7s+fNTA==", 2024-04-25T08:58:42.908 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:58:42.908 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T08:58:42.908 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-04-25T08:58:42.908 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-04-25T08:58:42.908 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-04-25T08:58:42.908 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:58:42.908 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:58:42.908 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:58:42.908 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.3", 2024-04-25T08:58:42.908 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpmLsbBKRAAIKWtRdpuyCaW1LnKMWTMiw==", 2024-04-25T08:58:42.908 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:58:42.908 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T08:58:42.909 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-04-25T08:58:42.909 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-04-25T08:58:42.909 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-04-25T08:58:42.909 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:58:42.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:58:42.909 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:58:42.909 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.admin", 2024-04-25T08:58:42.909 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD7GSpmE9C8JRAAS263n6a4L2DXzMyS9o42NQ==", 2024-04-25T08:58:42.909 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:58:42.909 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-04-25T08:58:42.909 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T08:58:42.909 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T08:58:42.909 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:58:42.909 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:58:42.909 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:58:42.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:58:42.910 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.x", 2024-04-25T08:58:42.910 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm/uGhAhAAr0FaJ6SwZ8Su1MEqyZwNRA==", 2024-04-25T08:58:42.910 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:58:42.910 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-04-25T08:58:42.910 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2024-04-25T08:58:42.910 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:58:42.910 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:58:42.910 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:58:42.910 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:58:42.910 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.y", 2024-04-25T08:58:42.910 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD7GSpmPwwpOhAALeiQ/LKM2zgjE7yu16RQOg==", 2024-04-25T08:58:42.910 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T08:58:42.910 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-04-25T08:58:42.911 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2024-04-25T08:58:42.911 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T08:58:42.911 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:58:42.911 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:58:42.911 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-04-25T08:58:42.911 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T08:58:42.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.910+0000 7fec49c58640 1 -- 172.21.15.43:0/1116279396 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fec3006b4d0 msgr2=0x7fec3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:42.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.910+0000 7fec49c58640 1 --2- 172.21.15.43:0/1116279396 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fec3006b4d0 0x7fec3006d990 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fec34034000 tx=0x7fec3402e000 comp rx=0 tx=0).stop 2024-04-25T08:58:42.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.910+0000 7fec49c58640 1 -- 172.21.15.43:0/1116279396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec440d2500 msgr2=0x7fec440c9ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:42.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.911+0000 7fec49c58640 1 --2- 172.21.15.43:0/1116279396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec440d2500 0x7fec440c9ad0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fec38000fd0 tx=0x7fec3800cf50 comp rx=0 tx=0).stop 2024-04-25T08:58:42.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.911+0000 7fec49c58640 1 -- 172.21.15.43:0/1116279396 shutdown_connections 2024-04-25T08:58:42.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.911+0000 7fec49c58640 1 --2- 172.21.15.43:0/1116279396 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fec3006b4d0 0x7fec3006d990 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:42.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.911+0000 7fec49c58640 1 --2- 172.21.15.43:0/1116279396 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fec440d39d0 0x7fec440c7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:42.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.911+0000 7fec49c58640 1 --2- 172.21.15.43:0/1116279396 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fec440d2500 0x7fec440c9ad0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:42.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.911+0000 7fec49c58640 1 --2- 172.21.15.43:0/1116279396 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fec440ce2a0 0x7fec441f4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:42.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.911+0000 7fec49c58640 1 -- 172.21.15.43:0/1116279396 >> 172.21.15.43:0/1116279396 conn(0x7fec4406b000 msgr2=0x7fec4406b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:42.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.911+0000 7fec49c58640 1 -- 172.21.15.43:0/1116279396 shutdown_connections 2024-04-25T08:58:42.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:42.911+0000 7fec49c58640 1 -- 172.21.15.43:0/1116279396 wait complete. 2024-04-25T08:58:42.914 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:58:43.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.025+0000 7fd417b53640 1 Processor -- start 2024-04-25T08:58:43.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.025+0000 7fd417b53640 1 -- start start 2024-04-25T08:58:43.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.026+0000 7fd417b53640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fd41015dce0 0x7fd41015ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.026+0000 7fd417b53640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd4100ce550 0x7fd4100d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.026+0000 7fd417b53640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd4100ce990 0x7fd4100d0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.026+0000 7fd417b53640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4100cee80 con 0x7fd4100ce990 2024-04-25T08:58:43.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.026+0000 7fd417b53640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4100cf160 con 0x7fd41015dce0 2024-04-25T08:58:43.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.026+0000 7fd417b53640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4100cf440 con 0x7fd4100ce550 2024-04-25T08:58:43.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.026+0000 7fd4160f0640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd4100ce990 0x7fd4100d0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:43.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.026+0000 7fd4160f0640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd4100ce990 0x7fd4100d0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50390/0 (socket says 172.21.15.43:50390) 2024-04-25T08:58:43.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.026+0000 7fd4160f0640 1 -- 172.21.15.43:0/3125705356 learned_addr learned my addr 172.21.15.43:0/3125705356 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:43.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.026+0000 7fd4158ef640 1 --2- 172.21.15.43:0/3125705356 >> v2:172.21.15.193:3300/0 conn(0x7fd41015dce0 0x7fd41015ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:43.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.026+0000 7fd4150ee640 1 --2- 172.21.15.43:0/3125705356 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd4100ce550 0x7fd4100d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:43.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.027+0000 7fd4160f0640 1 -- 172.21.15.43:0/3125705356 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd4100ce550 msgr2=0x7fd4100d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:43.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.027+0000 7fd4160f0640 1 --2- 172.21.15.43:0/3125705356 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd4100ce550 0x7fd4100d03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.027+0000 7fd4160f0640 1 -- 172.21.15.43:0/3125705356 >> v2:172.21.15.193:3300/0 conn(0x7fd41015dce0 msgr2=0x7fd41015ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:43.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.027+0000 7fd4160f0640 1 --2- 172.21.15.43:0/3125705356 >> v2:172.21.15.193:3300/0 conn(0x7fd41015dce0 0x7fd41015ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.027+0000 7fd4160f0640 1 -- 172.21.15.43:0/3125705356 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4100cf720 con 0x7fd4100ce990 2024-04-25T08:58:43.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.027+0000 7fd4160f0640 1 --2- 172.21.15.43:0/3125705356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4100ce990 0x7fd4100d0920 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fd41009d3c0 tx=0x7fd40002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f61545bc04e8cf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:43.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.027+0000 7fd4148ed640 1 -- 172.21.15.43:0/3125705356 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd400036070 con 0x7fd4100ce990 2024-04-25T08:58:43.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.027+0000 7fd4148ed640 1 -- 172.21.15.43:0/3125705356 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd400031600 con 0x7fd4100ce990 2024-04-25T08:58:43.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.027+0000 7fd4148ed640 1 -- 172.21.15.43:0/3125705356 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4000319d0 con 0x7fd4100ce990 2024-04-25T08:58:43.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.028+0000 7fd417b53640 1 -- 172.21.15.43:0/3125705356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4100ce990 msgr2=0x7fd4100d0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:43.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.028+0000 7fd417b53640 1 --2- 172.21.15.43:0/3125705356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4100ce990 0x7fd4100d0920 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fd41009d3c0 tx=0x7fd40002a610 comp rx=0 tx=0).stop 2024-04-25T08:58:43.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.028+0000 7fd417b53640 1 -- 172.21.15.43:0/3125705356 shutdown_connections 2024-04-25T08:58:43.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.028+0000 7fd417b53640 1 --2- 172.21.15.43:0/3125705356 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4100ce990 0x7fd4100d0920 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.028+0000 7fd417b53640 1 --2- 172.21.15.43:0/3125705356 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd4100ce550 0x7fd4100d03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.028+0000 7fd417b53640 1 --2- 172.21.15.43:0/3125705356 >> v2:172.21.15.193:3300/0 conn(0x7fd41015dce0 0x7fd41015ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.028+0000 7fd417b53640 1 -- 172.21.15.43:0/3125705356 >> 172.21.15.43:0/3125705356 conn(0x7fd41009cd00 msgr2=0x7fd4100d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:43.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.028+0000 7fd417b53640 1 -- 172.21.15.43:0/3125705356 shutdown_connections 2024-04-25T08:58:43.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.028+0000 7fd417b53640 1 -- 172.21.15.43:0/3125705356 wait complete. 2024-04-25T08:58:43.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.029+0000 7fd417b53640 1 Processor -- start 2024-04-25T08:58:43.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.029+0000 7fd417b53640 1 -- start start 2024-04-25T08:58:43.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.030+0000 7fd417b53640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4100ce550 0x7fd41009d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.030+0000 7fd417b53640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd4100ce990 0x7fd4100b81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.030+0000 7fd417b53640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd41015dce0 0x7fd4100d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.030+0000 7fd417b53640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd410160980 con 0x7fd4100ce550 2024-04-25T08:58:43.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.030+0000 7fd417b53640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd41015e0c0 con 0x7fd41015dce0 2024-04-25T08:58:43.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.030+0000 7fd417b53640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd410160b00 con 0x7fd4100ce990 2024-04-25T08:58:43.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.030+0000 7fd4158ef640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4100ce550 0x7fd41009d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:43.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.030+0000 7fd4158ef640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4100ce550 0x7fd41009d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50400/0 (socket says 172.21.15.43:50400) 2024-04-25T08:58:43.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.030+0000 7fd4158ef640 1 -- 172.21.15.43:0/2830133875 learned_addr learned my addr 172.21.15.43:0/2830133875 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:43.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.030+0000 7fd4160f0640 1 --2- 172.21.15.43:0/2830133875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd41015dce0 0x7fd4100d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:43.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.030+0000 7fd4150ee640 1 --2- 172.21.15.43:0/2830133875 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd4100ce990 0x7fd4100b81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:43.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.030+0000 7fd4158ef640 1 -- 172.21.15.43:0/2830133875 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd4100ce990 msgr2=0x7fd4100b81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:43.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.030+0000 7fd4158ef640 1 --2- 172.21.15.43:0/2830133875 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd4100ce990 0x7fd4100b81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.030+0000 7fd4158ef640 1 -- 172.21.15.43:0/2830133875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd41015dce0 msgr2=0x7fd4100d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:43.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.030+0000 7fd4158ef640 1 --2- 172.21.15.43:0/2830133875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd41015dce0 0x7fd4100d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.030+0000 7fd4158ef640 1 -- 172.21.15.43:0/2830133875 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4100b9390 con 0x7fd4100ce550 2024-04-25T08:58:43.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.031+0000 7fd4160f0640 1 --2- 172.21.15.43:0/2830133875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd41015dce0 0x7fd4100d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:58:43.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.031+0000 7fd4158ef640 1 --2- 172.21.15.43:0/2830133875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4100ce550 0x7fd41009d770 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fd40800d620 tx=0x7fd40800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:43.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.031+0000 7fd40e7fc640 1 -- 172.21.15.43:0/2830133875 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40800b620 con 0x7fd4100ce550 2024-04-25T08:58:43.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.031+0000 7fd40e7fc640 1 -- 172.21.15.43:0/2830133875 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd408019070 con 0x7fd4100ce550 2024-04-25T08:58:43.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.031+0000 7fd417b53640 1 -- 172.21.15.43:0/2830133875 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4100b76f0 con 0x7fd4100ce550 2024-04-25T08:58:43.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.031+0000 7fd40e7fc640 1 -- 172.21.15.43:0/2830133875 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40800ba00 con 0x7fd4100ce550 2024-04-25T08:58:43.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.031+0000 7fd417b53640 1 -- 172.21.15.43:0/2830133875 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd410153280 con 0x7fd4100ce550 2024-04-25T08:58:43.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.032+0000 7fd417b53640 1 -- 172.21.15.43:0/2830133875 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4100680f0 con 0x7fd4100ce550 2024-04-25T08:58:43.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.032+0000 7fd40e7fc640 1 -- 172.21.15.43:0/2830133875 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd40800bba0 con 0x7fd4100ce550 2024-04-25T08:58:43.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.033+0000 7fd40e7fc640 1 --2- 172.21.15.43:0/2830133875 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd3fc06b4d0 0x7fd3fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.033+0000 7fd40e7fc640 1 -- 172.21.15.43:0/2830133875 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd40808a360 con 0x7fd4100ce550 2024-04-25T08:58:43.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.033+0000 7fd4150ee640 1 --2- 172.21.15.43:0/2830133875 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd3fc06b4d0 0x7fd3fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:43.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.034+0000 7fd4150ee640 1 --2- 172.21.15.43:0/2830133875 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd3fc06b4d0 0x7fd3fc06d990 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd4100c9d60 tx=0x7fd404008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:43.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.037+0000 7fd40e7fc640 1 -- 172.21.15.43:0/2830133875 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd408059400 con 0x7fd4100ce550 2024-04-25T08:58:43.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.223+0000 7fd417b53640 1 -- 172.21.15.43:0/2830133875 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd410074ef0 con 0x7fd4100ce550 2024-04-25T08:58:43.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.224+0000 7fd40e7fc640 1 -- 172.21.15.43:0/2830133875 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+12855 (secure 0 0 0) 0x7fd408058d00 con 0x7fd4100ce550 2024-04-25T08:58:43.214 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:58:43.214 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":69,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:58:42.532685+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:58:43.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.226+0000 7fd417b53640 1 -- 172.21.15.43:0/2830133875 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd3fc06b4d0 msgr2=0x7fd3fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:43.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.226+0000 7fd417b53640 1 --2- 172.21.15.43:0/2830133875 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd3fc06b4d0 0x7fd3fc06d990 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fd4100c9d60 tx=0x7fd404008040 comp rx=0 tx=0).stop 2024-04-25T08:58:43.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.226+0000 7fd417b53640 1 -- 172.21.15.43:0/2830133875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4100ce550 msgr2=0x7fd41009d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:43.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.226+0000 7fd417b53640 1 --2- 172.21.15.43:0/2830133875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4100ce550 0x7fd41009d770 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fd40800d620 tx=0x7fd40800dae0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.226+0000 7fd417b53640 1 -- 172.21.15.43:0/2830133875 shutdown_connections 2024-04-25T08:58:43.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.226+0000 7fd417b53640 1 --2- 172.21.15.43:0/2830133875 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd3fc06b4d0 0x7fd3fc06d990 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.226+0000 7fd417b53640 1 --2- 172.21.15.43:0/2830133875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd41015dce0 0x7fd4100d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.226+0000 7fd417b53640 1 --2- 172.21.15.43:0/2830133875 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd4100ce990 0x7fd4100b81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.226+0000 7fd417b53640 1 --2- 172.21.15.43:0/2830133875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4100ce550 0x7fd41009d770 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.226+0000 7fd417b53640 1 -- 172.21.15.43:0/2830133875 >> 172.21.15.43:0/2830133875 conn(0x7fd41009cd00 msgr2=0x7fd4100d2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:43.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.226+0000 7fd417b53640 1 -- 172.21.15.43:0/2830133875 shutdown_connections 2024-04-25T08:58:43.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.226+0000 7fd417b53640 1 -- 172.21.15.43:0/2830133875 wait complete. 2024-04-25T08:58:43.227 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T08:58:43.228 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T08:58:43.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.343+0000 7f3867722640 1 Processor -- start 2024-04-25T08:58:43.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.343+0000 7f3867722640 1 -- start start 2024-04-25T08:58:43.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.344+0000 7f3867722640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3860161310 0x7f38600d4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.344+0000 7f3867722640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f38600d0c70 0x7f38600c7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.344+0000 7f3867722640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38600d45c0 con 0x7f38600d0120 2024-04-25T08:58:43.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.344+0000 7f3867722640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38600d4870 con 0x7f3860161310 2024-04-25T08:58:43.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.344+0000 7f3867722640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38600cf030 con 0x7f38600d0c70 2024-04-25T08:58:43.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.344+0000 7f38654be640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3860161310 0x7f38600d4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:43.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.344+0000 7f38654be640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3860161310 0x7f38600d4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50414/0 (socket says 172.21.15.43:50414) 2024-04-25T08:58:43.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.344+0000 7f38654be640 1 -- 172.21.15.43:0/3188290820 learned_addr learned my addr 172.21.15.43:0/3188290820 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:43.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.344+0000 7f38654be640 1 -- 172.21.15.43:0/3188290820 >> v2:172.21.15.193:3300/0 conn(0x7f38600d0c70 msgr2=0x7f38600c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:43.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.344+0000 7f38654be640 1 --2- 172.21.15.43:0/3188290820 >> v2:172.21.15.193:3300/0 conn(0x7f38600d0c70 0x7f38600c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.344+0000 7f38654be640 1 -- 172.21.15.43:0/3188290820 >> v1:172.21.15.193:6789/0 conn(0x7f38600d0120 legacy=0x7f38600d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:43.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.345+0000 7f38654be640 1 -- 172.21.15.43:0/3188290820 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38600cf310 con 0x7f3860161310 2024-04-25T08:58:43.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.345+0000 7f38654be640 1 --2- 172.21.15.43:0/3188290820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3860161310 0x7f38600d4080 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f386006ae90 tx=0x7f385002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e311718d1029b4cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:43.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.345+0000 7f385ffff640 1 -- 172.21.15.43:0/3188290820 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3850036070 con 0x7f3860161310 2024-04-25T08:58:43.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.345+0000 7f385ffff640 1 -- 172.21.15.43:0/3188290820 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3850031600 con 0x7f3860161310 2024-04-25T08:58:43.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.345+0000 7f385ffff640 1 -- 172.21.15.43:0/3188290820 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38500319d0 con 0x7f3860161310 2024-04-25T08:58:43.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.346+0000 7f3867722640 1 -- 172.21.15.43:0/3188290820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3860161310 msgr2=0x7f38600d4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:43.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.346+0000 7f3867722640 1 --2- 172.21.15.43:0/3188290820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3860161310 0x7f38600d4080 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f386006ae90 tx=0x7f385002a6f0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.346+0000 7f3867722640 1 -- 172.21.15.43:0/3188290820 shutdown_connections 2024-04-25T08:58:43.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.346+0000 7f3867722640 1 --2- 172.21.15.43:0/3188290820 >> v2:172.21.15.193:3300/0 conn(0x7f38600d0c70 0x7f38600c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.346+0000 7f3867722640 1 --2- 172.21.15.43:0/3188290820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3860161310 0x7f38600d4080 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.346+0000 7f3867722640 1 -- 172.21.15.43:0/3188290820 >> 172.21.15.43:0/3188290820 conn(0x7f386009cd20 msgr2=0x7f38600d3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:43.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.346+0000 7f3867722640 1 -- 172.21.15.43:0/3188290820 shutdown_connections 2024-04-25T08:58:43.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.346+0000 7f3867722640 1 -- 172.21.15.43:0/3188290820 wait complete. 2024-04-25T08:58:43.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.347+0000 7f3867722640 1 Processor -- start 2024-04-25T08:58:43.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.347+0000 7f3867722640 1 -- start start 2024-04-25T08:58:43.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.348+0000 7f3867722640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38600d0120 0x7f38600b8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.348+0000 7f3867722640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f38600d0c70 0x7f386014f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.348+0000 7f3867722640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3860161310 0x7f386015c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.348+0000 7f3867722640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38600c7da0 con 0x7f38600d0120 2024-04-25T08:58:43.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.348+0000 7f3867722640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38601616f0 con 0x7f3860161310 2024-04-25T08:58:43.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.348+0000 7f3867722640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38600c7f20 con 0x7f38600d0c70 2024-04-25T08:58:43.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.348+0000 7f38654be640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38600d0120 0x7f38600b8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:43.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.348+0000 7f38654be640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38600d0120 0x7f38600b8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50420/0 (socket says 172.21.15.43:50420) 2024-04-25T08:58:43.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.348+0000 7f38654be640 1 -- 172.21.15.43:0/1532667421 learned_addr learned my addr 172.21.15.43:0/1532667421 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:43.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.348+0000 7f3865cbf640 1 --2- 172.21.15.43:0/1532667421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3860161310 0x7f386015c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:43.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.348+0000 7f3864cbd640 1 --2- 172.21.15.43:0/1532667421 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f38600d0c70 0x7f386014f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:43.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.348+0000 7f38654be640 1 -- 172.21.15.43:0/1532667421 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f38600d0c70 msgr2=0x7f386014f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:43.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.348+0000 7f38654be640 1 --2- 172.21.15.43:0/1532667421 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f38600d0c70 0x7f386014f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.348+0000 7f38654be640 1 -- 172.21.15.43:0/1532667421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3860161310 msgr2=0x7f386015c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:43.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.348+0000 7f38654be640 1 --2- 172.21.15.43:0/1532667421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3860161310 0x7f386015c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.348+0000 7f38654be640 1 -- 172.21.15.43:0/1532667421 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38600b7040 con 0x7f38600d0120 2024-04-25T08:58:43.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.349+0000 7f38654be640 1 --2- 172.21.15.43:0/1532667421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38600d0120 0x7f38600b8eb0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f3850034040 tx=0x7f3850004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:43.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.349+0000 7f385dffb640 1 -- 172.21.15.43:0/1532667421 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3850041020 con 0x7f38600d0120 2024-04-25T08:58:43.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.349+0000 7f385dffb640 1 -- 172.21.15.43:0/1532667421 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3850007820 con 0x7f38600d0120 2024-04-25T08:58:43.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.349+0000 7f3867722640 1 -- 172.21.15.43:0/1532667421 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38600b7320 con 0x7f38600d0120 2024-04-25T08:58:43.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.349+0000 7f385dffb640 1 -- 172.21.15.43:0/1532667421 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3850036040 con 0x7f38600d0120 2024-04-25T08:58:43.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.349+0000 7f3867722640 1 -- 172.21.15.43:0/1532667421 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38600b7e00 con 0x7f38600d0120 2024-04-25T08:58:43.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.350+0000 7f3867722640 1 -- 172.21.15.43:0/1532667421 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3860074f60 con 0x7f38600d0120 2024-04-25T08:58:43.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.350+0000 7f385dffb640 1 -- 172.21.15.43:0/1532667421 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3850044020 con 0x7f38600d0120 2024-04-25T08:58:43.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.351+0000 7f385dffb640 1 --2- 172.21.15.43:0/1532667421 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f384c06b500 0x7f384c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.351+0000 7f385dffb640 1 -- 172.21.15.43:0/1532667421 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f38500aac60 con 0x7f38600d0120 2024-04-25T08:58:43.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.351+0000 7f3864cbd640 1 --2- 172.21.15.43:0/1532667421 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f384c06b500 0x7f384c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:43.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.352+0000 7f3864cbd640 1 --2- 172.21.15.43:0/1532667421 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f384c06b500 0x7f384c06d9c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3854007fd0 tx=0x7f3854009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:43.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.355+0000 7f385dffb640 1 -- 172.21.15.43:0/1532667421 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f385004b030 con 0x7f38600d0120 2024-04-25T08:58:43.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.537+0000 7f3867722640 1 -- 172.21.15.43:0/1532667421 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f38600c9ba0 con 0x7f38600d0120 2024-04-25T08:58:43.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.537+0000 7f385dffb640 1 -- 172.21.15.43:0/1532667421 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v69) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3850079ec0 con 0x7f38600d0120 2024-04-25T08:58:43.526 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T08:58:43.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.539+0000 7f3867722640 1 -- 172.21.15.43:0/1532667421 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f384c06b500 msgr2=0x7f384c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:43.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.539+0000 7f3867722640 1 --2- 172.21.15.43:0/1532667421 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f384c06b500 0x7f384c06d9c0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3854007fd0 tx=0x7f3854009040 comp rx=0 tx=0).stop 2024-04-25T08:58:43.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.539+0000 7f3867722640 1 -- 172.21.15.43:0/1532667421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38600d0120 msgr2=0x7f38600b8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:43.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.539+0000 7f3867722640 1 --2- 172.21.15.43:0/1532667421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38600d0120 0x7f38600b8eb0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f3850034040 tx=0x7f3850004a40 comp rx=0 tx=0).stop 2024-04-25T08:58:43.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.539+0000 7f3867722640 1 -- 172.21.15.43:0/1532667421 shutdown_connections 2024-04-25T08:58:43.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.539+0000 7f3867722640 1 --2- 172.21.15.43:0/1532667421 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f384c06b500 0x7f384c06d9c0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.539+0000 7f3867722640 1 --2- 172.21.15.43:0/1532667421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3860161310 0x7f386015c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.539+0000 7f3867722640 1 --2- 172.21.15.43:0/1532667421 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f38600d0c70 0x7f386014f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.539+0000 7f3867722640 1 --2- 172.21.15.43:0/1532667421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38600d0120 0x7f38600b8eb0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.539+0000 7f3867722640 1 -- 172.21.15.43:0/1532667421 >> 172.21.15.43:0/1532667421 conn(0x7f386009cd20 msgr2=0x7f38600c8350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:43.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.539+0000 7f3867722640 1 -- 172.21.15.43:0/1532667421 shutdown_connections 2024-04-25T08:58:43.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.539+0000 7f3867722640 1 -- 172.21.15.43:0/1532667421 wait complete. 2024-04-25T08:58:43.540 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T08:58:43.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.651+0000 7ff9c939e640 1 Processor -- start 2024-04-25T08:58:43.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.651+0000 7ff9c939e640 1 -- start start 2024-04-25T08:58:43.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.652+0000 7ff9c939e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff9c40ccbb0 0x7ff9c40cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.652+0000 7ff9c939e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff9c40b3b00 0x7ff9c40b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.652+0000 7ff9c939e640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9c40b2780 con 0x7ff9c40cbfc0 2024-04-25T08:58:43.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.652+0000 7ff9c939e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9c40cd550 con 0x7ff9c40b3b00 2024-04-25T08:58:43.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.652+0000 7ff9c939e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c40b2a60 con 0x7ff9c40ccbb0 2024-04-25T08:58:43.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.652+0000 7ff9c37fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff9c40b3b00 0x7ff9c40b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:43.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.652+0000 7ff9c37fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff9c40b3b00 0x7ff9c40b3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50430/0 (socket says 172.21.15.43:50430) 2024-04-25T08:58:43.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.652+0000 7ff9c37fe640 1 -- 172.21.15.43:0/4221445192 learned_addr learned my addr 172.21.15.43:0/4221445192 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:43.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.652+0000 7ff9c27fc640 1 --2- 172.21.15.43:0/4221445192 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff9c40ccbb0 0x7ff9c40cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:43.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.652+0000 7ff9c37fe640 1 -- 172.21.15.43:0/4221445192 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff9c40ccbb0 msgr2=0x7ff9c40cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:43.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.652+0000 7ff9c37fe640 1 --2- 172.21.15.43:0/4221445192 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff9c40ccbb0 0x7ff9c40cd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.652+0000 7ff9c37fe640 1 -- 172.21.15.43:0/4221445192 >> v1:172.21.15.193:6789/0 conn(0x7ff9c40cbfc0 legacy=0x7ff9c406dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:43.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.652+0000 7ff9c37fe640 1 -- 172.21.15.43:0/4221445192 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c40b2d40 con 0x7ff9c40b3b00 2024-04-25T08:58:43.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.653+0000 7ff9c37fe640 1 --2- 172.21.15.43:0/4221445192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9c40b3b00 0x7ff9c40b3f90 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4009e50 tx=0x7ff9b402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9830ff7390e4598a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:43.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.653+0000 7ff9c1ffb640 1 -- 172.21.15.43:0/4221445192 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4036070 con 0x7ff9c40b3b00 2024-04-25T08:58:43.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.653+0000 7ff9c1ffb640 1 -- 172.21.15.43:0/4221445192 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9b4031600 con 0x7ff9c40b3b00 2024-04-25T08:58:43.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.653+0000 7ff9c1ffb640 1 -- 172.21.15.43:0/4221445192 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b40319d0 con 0x7ff9c40b3b00 2024-04-25T08:58:43.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.654+0000 7ff9c939e640 1 -- 172.21.15.43:0/4221445192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9c40b3b00 msgr2=0x7ff9c40b3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:43.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.654+0000 7ff9c939e640 1 --2- 172.21.15.43:0/4221445192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9c40b3b00 0x7ff9c40b3f90 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4009e50 tx=0x7ff9b402a6c0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.654+0000 7ff9c939e640 1 -- 172.21.15.43:0/4221445192 shutdown_connections 2024-04-25T08:58:43.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.654+0000 7ff9c939e640 1 --2- 172.21.15.43:0/4221445192 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9c40b3b00 0x7ff9c40b3f90 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.654+0000 7ff9c939e640 1 --2- 172.21.15.43:0/4221445192 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff9c40ccbb0 0x7ff9c40cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.654+0000 7ff9c939e640 1 -- 172.21.15.43:0/4221445192 >> 172.21.15.43:0/4221445192 conn(0x7ff9c40b1480 msgr2=0x7ff9c40b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:43.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.654+0000 7ff9c939e640 1 -- 172.21.15.43:0/4221445192 shutdown_connections 2024-04-25T08:58:43.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.654+0000 7ff9c939e640 1 -- 172.21.15.43:0/4221445192 wait complete. 2024-04-25T08:58:43.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.655+0000 7ff9c939e640 1 Processor -- start 2024-04-25T08:58:43.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.655+0000 7ff9c939e640 1 -- start start 2024-04-25T08:58:43.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.656+0000 7ff9c939e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff9c40b3b00 0x7ff9c415b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.656+0000 7ff9c939e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9c40cbfc0 0x7ff9c41e73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.656+0000 7ff9c939e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9c40ccbb0 0x7ff9c41e7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.656+0000 7ff9c939e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c40b35e0 con 0x7ff9c40ccbb0 2024-04-25T08:58:43.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.656+0000 7ff9c939e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c40b3300 con 0x7ff9c40cbfc0 2024-04-25T08:58:43.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.656+0000 7ff9c939e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c40cc3a0 con 0x7ff9c40b3b00 2024-04-25T08:58:43.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.656+0000 7ff9c37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9c40ccbb0 0x7ff9c41e7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:43.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.656+0000 7ff9c37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9c40ccbb0 0x7ff9c41e7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50432/0 (socket says 172.21.15.43:50432) 2024-04-25T08:58:43.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.656+0000 7ff9c37fe640 1 -- 172.21.15.43:0/2926096878 learned_addr learned my addr 172.21.15.43:0/2926096878 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:43.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.656+0000 7ff9c27fc640 1 --2- 172.21.15.43:0/2926096878 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9c40cbfc0 0x7ff9c41e73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:43.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.656+0000 7ff9c2ffd640 1 --2- 172.21.15.43:0/2926096878 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff9c40b3b00 0x7ff9c415b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:43.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.656+0000 7ff9c37fe640 1 -- 172.21.15.43:0/2926096878 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff9c40b3b00 msgr2=0x7ff9c415b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:43.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.656+0000 7ff9c37fe640 1 --2- 172.21.15.43:0/2926096878 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff9c40b3b00 0x7ff9c415b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.656+0000 7ff9c37fe640 1 -- 172.21.15.43:0/2926096878 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9c40cbfc0 msgr2=0x7ff9c41e73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:43.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.656+0000 7ff9c37fe640 1 --2- 172.21.15.43:0/2926096878 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9c40cbfc0 0x7ff9c41e73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.656+0000 7ff9c37fe640 1 -- 172.21.15.43:0/2926096878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c413e7e0 con 0x7ff9c40ccbb0 2024-04-25T08:58:43.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.656+0000 7ff9c37fe640 1 --2- 172.21.15.43:0/2926096878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9c40ccbb0 0x7ff9c41e7930 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4034040 tx=0x7ff9b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:43.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.657+0000 7ff9ab7fe640 1 -- 172.21.15.43:0/2926096878 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b403f020 con 0x7ff9c40ccbb0 2024-04-25T08:58:43.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.657+0000 7ff9ab7fe640 1 -- 172.21.15.43:0/2926096878 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9b4037040 con 0x7ff9c40ccbb0 2024-04-25T08:58:43.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.657+0000 7ff9c939e640 1 -- 172.21.15.43:0/2926096878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9c413eaa0 con 0x7ff9c40ccbb0 2024-04-25T08:58:43.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.657+0000 7ff9ab7fe640 1 -- 172.21.15.43:0/2926096878 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4036070 con 0x7ff9c40ccbb0 2024-04-25T08:58:43.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.657+0000 7ff9c939e640 1 -- 172.21.15.43:0/2926096878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9c413f060 con 0x7ff9c40ccbb0 2024-04-25T08:58:43.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.658+0000 7ff9c939e640 1 -- 172.21.15.43:0/2926096878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9c4074ef0 con 0x7ff9c40ccbb0 2024-04-25T08:58:43.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.658+0000 7ff9ab7fe640 1 -- 172.21.15.43:0/2926096878 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff9b4042020 con 0x7ff9c40ccbb0 2024-04-25T08:58:43.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.659+0000 7ff9ab7fe640 1 --2- 172.21.15.43:0/2926096878 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff9b006b4d0 0x7ff9b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.659+0000 7ff9ab7fe640 1 -- 172.21.15.43:0/2926096878 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff9b40aa2f0 con 0x7ff9c40ccbb0 2024-04-25T08:58:43.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.659+0000 7ff9c2ffd640 1 --2- 172.21.15.43:0/2926096878 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff9b006b4d0 0x7ff9b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:43.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.659+0000 7ff9c2ffd640 1 --2- 172.21.15.43:0/2926096878 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff9b006b4d0 0x7ff9b006d990 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc004f60 tx=0x7ff9bc007e30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:43.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.663+0000 7ff9ab7fe640 1 -- 172.21.15.43:0/2926096878 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff9b4049030 con 0x7ff9c40ccbb0 2024-04-25T08:58:43.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.844+0000 7ff9c939e640 1 -- 172.21.15.43:0/2926096878 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff9c40af0d0 con 0x7ff9c40ccbb0 2024-04-25T08:58:43.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.844+0000 7ff9ab7fe640 1 -- 172.21.15.43:0/2926096878 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v69) v1 ==== 84+0+65 (secure 0 0 0) 0x7ff9b407f120 con 0x7ff9c40ccbb0 2024-04-25T08:58:43.834 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:58:43.834 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2024-04-25T08:58:43.834 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T08:58:43.834 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 1, 2024-04-25T08:58:43.834 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": ".mgr" 2024-04-25T08:58:43.835 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:58:43.835 INFO:teuthology.orchestra.run.smithi043.stdout:] 2024-04-25T08:58:43.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.846+0000 7ff9c939e640 1 -- 172.21.15.43:0/2926096878 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff9b006b4d0 msgr2=0x7ff9b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:43.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.846+0000 7ff9c939e640 1 --2- 172.21.15.43:0/2926096878 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff9b006b4d0 0x7ff9b006d990 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc004f60 tx=0x7ff9bc007e30 comp rx=0 tx=0).stop 2024-04-25T08:58:43.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.846+0000 7ff9c939e640 1 -- 172.21.15.43:0/2926096878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9c40ccbb0 msgr2=0x7ff9c41e7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:43.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.846+0000 7ff9c939e640 1 --2- 172.21.15.43:0/2926096878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9c40ccbb0 0x7ff9c41e7930 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7ff9b4034040 tx=0x7ff9b4004a40 comp rx=0 tx=0).stop 2024-04-25T08:58:43.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.846+0000 7ff9c939e640 1 -- 172.21.15.43:0/2926096878 shutdown_connections 2024-04-25T08:58:43.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.846+0000 7ff9c939e640 1 --2- 172.21.15.43:0/2926096878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff9c40ccbb0 0x7ff9c41e7930 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.846+0000 7ff9c939e640 1 --2- 172.21.15.43:0/2926096878 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff9b006b4d0 0x7ff9b006d990 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.846+0000 7ff9c939e640 1 --2- 172.21.15.43:0/2926096878 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9c40cbfc0 0x7ff9c41e73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.846+0000 7ff9c939e640 1 --2- 172.21.15.43:0/2926096878 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff9c40b3b00 0x7ff9c415b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.846+0000 7ff9c939e640 1 -- 172.21.15.43:0/2926096878 >> 172.21.15.43:0/2926096878 conn(0x7ff9c40b1480 msgr2=0x7ff9c40cea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:43.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.846+0000 7ff9c939e640 1 -- 172.21.15.43:0/2926096878 shutdown_connections 2024-04-25T08:58:43.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.846+0000 7ff9c939e640 1 -- 172.21.15.43:0/2926096878 wait complete. 2024-04-25T08:58:43.847 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T08:58:43.847 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T08:58:43.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.963+0000 7fcf92b90640 1 Processor -- start 2024-04-25T08:58:43.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.963+0000 7fcf92b90640 1 -- start start 2024-04-25T08:58:43.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.964+0000 7fcf92b90640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fcf8c15def0 0x7fcf8c0d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.964+0000 7fcf92b90640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcf8c0d4670 0x7fcf8c0d03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.964+0000 7fcf92b90640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf8c0d2370 con 0x7fcf8c0d0930 2024-04-25T08:58:43.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.964+0000 7fcf92b90640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf8c0b37b0 con 0x7fcf8c15def0 2024-04-25T08:58:43.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.964+0000 7fcf92b90640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8c0d2650 con 0x7fcf8c0d4670 2024-04-25T08:58:43.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.964+0000 7fcf9112d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fcf8c0d0930 0x7fcf8c0ce4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39170/0 (socket says 172.21.15.43:39170) 2024-04-25T08:58:43.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.964+0000 7fcf9112d640 1 -- 172.21.15.43:0/2459290854 learned_addr learned my addr 172.21.15.43:0/2459290854 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:43.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.964+0000 7fcf8bfff640 1 --2- 172.21.15.43:0/2459290854 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcf8c0d4670 0x7fcf8c0d03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:43.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.964+0000 7fcf9092c640 1 --2- 172.21.15.43:0/2459290854 >> v2:172.21.15.193:3300/0 conn(0x7fcf8c15def0 0x7fcf8c0d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:43.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.964+0000 7fcf8b7fe640 1 -- 172.21.15.43:0/2459290854 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3737389368 0 0) 0x7fcf8c0d2370 con 0x7fcf8c0d0930 2024-04-25T08:58:43.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.964+0000 7fcf8b7fe640 1 -- 172.21.15.43:0/2459290854 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf7c003520 con 0x7fcf8c0d0930 2024-04-25T08:58:43.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.964+0000 7fcf8b7fe640 1 -- 172.21.15.43:0/2459290854 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4243223546 0 0) 0x7fcf7c003520 con 0x7fcf8c0d0930 2024-04-25T08:58:43.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.964+0000 7fcf8b7fe640 1 -- 172.21.15.43:0/2459290854 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcf8c0d4670 msgr2=0x7fcf8c0d03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:43.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.964+0000 7fcf8b7fe640 1 --2- 172.21.15.43:0/2459290854 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcf8c0d4670 0x7fcf8c0d03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.964+0000 7fcf8b7fe640 1 -- 172.21.15.43:0/2459290854 >> v2:172.21.15.193:3300/0 conn(0x7fcf8c15def0 msgr2=0x7fcf8c0d4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:43.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.964+0000 7fcf8b7fe640 1 --2- 172.21.15.43:0/2459290854 >> v2:172.21.15.193:3300/0 conn(0x7fcf8c15def0 0x7fcf8c0d4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.964+0000 7fcf8b7fe640 1 -- 172.21.15.43:0/2459290854 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf8c0b3440 con 0x7fcf8c0d0930 2024-04-25T08:58:43.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.964+0000 7fcf8b7fe640 1 -- 172.21.15.43:0/2459290854 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7fcf840019f0 con 0x7fcf8c0d0930 2024-04-25T08:58:43.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.964+0000 7fcf8b7fe640 1 -- 172.21.15.43:0/2459290854 >> v1:172.21.15.43:6789/0 conn(0x7fcf8c0d0930 legacy=0x7fcf8c0ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:43.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.964+0000 7fcf8b7fe640 1 --2- 172.21.15.43:0/2459290854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf7c0039b0 0x7fcf7c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.964+0000 7fcf8b7fe640 1 --2- 172.21.15.43:0/2459290854 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcf8c0d4670 0x7fcf7c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.964+0000 7fcf8b7fe640 1 --2- 172.21.15.43:0/2459290854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf7c004d40 0x7fcf7c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.964+0000 7fcf8b7fe640 1 -- 172.21.15.43:0/2459290854 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8c0b37b0 con 0x7fcf7c0039b0 2024-04-25T08:58:43.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.964+0000 7fcf8b7fe640 1 -- 172.21.15.43:0/2459290854 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8c0d2650 con 0x7fcf7c004d40 2024-04-25T08:58:43.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.964+0000 7fcf8b7fe640 1 -- 172.21.15.43:0/2459290854 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7c00b9c0 con 0x7fcf8c0d4670 2024-04-25T08:58:43.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.965+0000 7fcf9092c640 1 --2- 172.21.15.43:0/2459290854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf7c0039b0 0x7fcf7c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:43.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.965+0000 7fcf8bfff640 1 --2- 172.21.15.43:0/2459290854 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcf8c0d4670 0x7fcf7c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:43.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.965+0000 7fcf9112d640 1 --2- 172.21.15.43:0/2459290854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf7c004d40 0x7fcf7c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:43.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.965+0000 7fcf9092c640 1 -- 172.21.15.43:0/2459290854 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcf8c0d4670 msgr2=0x7fcf7c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:43.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.965+0000 7fcf9092c640 1 --2- 172.21.15.43:0/2459290854 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcf8c0d4670 0x7fcf7c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.965+0000 7fcf9092c640 1 -- 172.21.15.43:0/2459290854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf7c004d40 msgr2=0x7fcf7c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:43.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.965+0000 7fcf9092c640 1 --2- 172.21.15.43:0/2459290854 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf7c004d40 0x7fcf7c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.965+0000 7fcf9092c640 1 -- 172.21.15.43:0/2459290854 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf7c003520 con 0x7fcf7c0039b0 2024-04-25T08:58:43.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.966+0000 7fcf9092c640 1 --2- 172.21.15.43:0/2459290854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf7c0039b0 0x7fcf7c003dd0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fcf80009ac0 tx=0x7fcf7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b225414a0f308150 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:43.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.966+0000 7fcf8b7fe640 1 -- 172.21.15.43:0/2459290854 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf80036070 con 0x7fcf7c0039b0 2024-04-25T08:58:43.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.966+0000 7fcf8b7fe640 1 -- 172.21.15.43:0/2459290854 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf80031400 con 0x7fcf7c0039b0 2024-04-25T08:58:43.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.966+0000 7fcf8b7fe640 1 -- 172.21.15.43:0/2459290854 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf80009690 con 0x7fcf7c0039b0 2024-04-25T08:58:43.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.966+0000 7fcf92b90640 1 -- 172.21.15.43:0/2459290854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf7c0039b0 msgr2=0x7fcf7c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:43.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.966+0000 7fcf92b90640 1 --2- 172.21.15.43:0/2459290854 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf7c0039b0 0x7fcf7c003dd0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fcf80009ac0 tx=0x7fcf7c005830 comp rx=0 tx=0).stop 2024-04-25T08:58:43.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.966+0000 7fcf9112d640 1 -- 172.21.15.43:0/2459290854 reap_dead start 2024-04-25T08:58:43.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.966+0000 7fcf92b90640 1 -- 172.21.15.43:0/2459290854 shutdown_connections 2024-04-25T08:58:43.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.966+0000 7fcf92b90640 1 -- 172.21.15.43:0/2459290854 >> 172.21.15.43:0/2459290854 conn(0x7fcf8c09cd60 msgr2=0x7fcf8c0d3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:43.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.966+0000 7fcf92b90640 1 -- 172.21.15.43:0/2459290854 shutdown_connections 2024-04-25T08:58:43.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.966+0000 7fcf92b90640 1 -- 172.21.15.43:0/2459290854 wait complete. 2024-04-25T08:58:43.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.967+0000 7fcf92b90640 1 Processor -- start 2024-04-25T08:58:43.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.968+0000 7fcf92b90640 1 -- start start 2024-04-25T08:58:43.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.968+0000 7fcf92b90640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf7c0039b0 0x7fcf8c152d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.968+0000 7fcf92b90640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcf8c09ffc0 0x7fcf8c09d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.968+0000 7fcf92b90640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf8c0cb5c0 0x7fcf8c0cba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.968+0000 7fcf92b90640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8c0d2150 con 0x7fcf8c0cb5c0 2024-04-25T08:58:43.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.968+0000 7fcf92b90640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8c0d1fd0 con 0x7fcf7c0039b0 2024-04-25T08:58:43.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.968+0000 7fcf92b90640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8c0b4c50 con 0x7fcf8c09ffc0 2024-04-25T08:58:43.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.968+0000 7fcf9112d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf8c0cb5c0 0x7fcf8c0cba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:43.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.969+0000 7fcf9112d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf8c0cb5c0 0x7fcf8c0cba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50456/0 (socket says 172.21.15.43:50456) 2024-04-25T08:58:43.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.969+0000 7fcf9112d640 1 -- 172.21.15.43:0/4189573143 learned_addr learned my addr 172.21.15.43:0/4189573143 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:43.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.969+0000 7fcf8bfff640 1 --2- 172.21.15.43:0/4189573143 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcf8c09ffc0 0x7fcf8c09d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:43.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.969+0000 7fcf9092c640 1 --2- 172.21.15.43:0/4189573143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf7c0039b0 0x7fcf8c152d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:43.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.969+0000 7fcf9112d640 1 -- 172.21.15.43:0/4189573143 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcf8c09ffc0 msgr2=0x7fcf8c09d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:43.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.969+0000 7fcf9112d640 1 --2- 172.21.15.43:0/4189573143 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcf8c09ffc0 0x7fcf8c09d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.969+0000 7fcf9112d640 1 -- 172.21.15.43:0/4189573143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf7c0039b0 msgr2=0x7fcf8c152d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:43.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.969+0000 7fcf9112d640 1 --2- 172.21.15.43:0/4189573143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf7c0039b0 0x7fcf8c152d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:43.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.969+0000 7fcf9112d640 1 -- 172.21.15.43:0/4189573143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf8c0a0480 con 0x7fcf8c0cb5c0 2024-04-25T08:58:43.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.969+0000 7fcf8bfff640 1 --2- 172.21.15.43:0/4189573143 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcf8c09ffc0 0x7fcf8c09d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:58:43.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.969+0000 7fcf9112d640 1 --2- 172.21.15.43:0/4189573143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf8c0cb5c0 0x7fcf8c0cba50 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c06b030 tx=0x7fcf84007b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:43.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.969+0000 7fcf897fa640 1 -- 172.21.15.43:0/4189573143 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8400aca0 con 0x7fcf8c0cb5c0 2024-04-25T08:58:43.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.969+0000 7fcf897fa640 1 -- 172.21.15.43:0/4189573143 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf84018070 con 0x7fcf8c0cb5c0 2024-04-25T08:58:43.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.969+0000 7fcf92b90640 1 -- 172.21.15.43:0/4189573143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf8c0a0760 con 0x7fcf8c0cb5c0 2024-04-25T08:58:43.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.969+0000 7fcf897fa640 1 -- 172.21.15.43:0/4189573143 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf84013610 con 0x7fcf8c0cb5c0 2024-04-25T08:58:43.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.969+0000 7fcf92b90640 1 -- 172.21.15.43:0/4189573143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf8c0a0d20 con 0x7fcf8c0cb5c0 2024-04-25T08:58:43.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.970+0000 7fcf92b90640 1 -- 172.21.15.43:0/4189573143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf8c068a40 con 0x7fcf8c0cb5c0 2024-04-25T08:58:43.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.971+0000 7fcf897fa640 1 -- 172.21.15.43:0/4189573143 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcf840137b0 con 0x7fcf8c0cb5c0 2024-04-25T08:58:43.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.971+0000 7fcf897fa640 1 --2- 172.21.15.43:0/4189573143 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcf7006b500 0x7fcf7006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:43.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.971+0000 7fcf897fa640 1 -- 172.21.15.43:0/4189573143 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcf8408a490 con 0x7fcf8c0cb5c0 2024-04-25T08:58:43.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.971+0000 7fcf9092c640 1 --2- 172.21.15.43:0/4189573143 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcf7006b500 0x7fcf7006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:43.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.972+0000 7fcf9092c640 1 --2- 172.21.15.43:0/4189573143 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcf7006b500 0x7fcf7006d9c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fcf80033000 tx=0x7fcf80008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:43.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:43.976+0000 7fcf897fa640 1 -- 172.21.15.43:0/4189573143 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcf84059660 con 0x7fcf8c0cb5c0 2024-04-25T08:58:44.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.157+0000 7fcf92b90640 1 -- 172.21.15.43:0/4189573143 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fcf8c066cc0 con 0x7fcf8c0cb5c0 2024-04-25T08:58:44.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.525+0000 7fcf897fa640 1 -- 172.21.15.43:0/4189573143 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v70) v1 ==== 138+0+0 (secure 0 0 0) 0x7fcf84058f60 con 0x7fcf8c0cb5c0 2024-04-25T08:58:44.514 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' created 2024-04-25T08:58:44.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.527+0000 7fcf92b90640 1 -- 172.21.15.43:0/4189573143 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcf7006b500 msgr2=0x7fcf7006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:44.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.527+0000 7fcf92b90640 1 --2- 172.21.15.43:0/4189573143 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcf7006b500 0x7fcf7006d9c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fcf80033000 tx=0x7fcf80008030 comp rx=0 tx=0).stop 2024-04-25T08:58:44.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.527+0000 7fcf92b90640 1 -- 172.21.15.43:0/4189573143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf8c0cb5c0 msgr2=0x7fcf8c0cba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:44.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.527+0000 7fcf92b90640 1 --2- 172.21.15.43:0/4189573143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf8c0cb5c0 0x7fcf8c0cba50 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c06b030 tx=0x7fcf84007b20 comp rx=0 tx=0).stop 2024-04-25T08:58:44.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.527+0000 7fcf92b90640 1 -- 172.21.15.43:0/4189573143 shutdown_connections 2024-04-25T08:58:44.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.527+0000 7fcf92b90640 1 --2- 172.21.15.43:0/4189573143 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf8c0cb5c0 0x7fcf8c0cba50 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:44.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.527+0000 7fcf92b90640 1 --2- 172.21.15.43:0/4189573143 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcf8c09ffc0 0x7fcf8c09d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:44.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.527+0000 7fcf92b90640 1 --2- 172.21.15.43:0/4189573143 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcf7006b500 0x7fcf7006d9c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:44.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.527+0000 7fcf92b90640 1 --2- 172.21.15.43:0/4189573143 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf7c0039b0 0x7fcf8c152d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:44.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.527+0000 7fcf92b90640 1 -- 172.21.15.43:0/4189573143 >> 172.21.15.43:0/4189573143 conn(0x7fcf8c09cd60 msgr2=0x7fcf8c06b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:44.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.527+0000 7fcf92b90640 1 -- 172.21.15.43:0/4189573143 shutdown_connections 2024-04-25T08:58:44.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.528+0000 7fcf92b90640 1 -- 172.21.15.43:0/4189573143 wait complete. 2024-04-25T08:58:44.531 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T08:58:44.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.651+0000 7fb20a86f640 1 Processor -- start 2024-04-25T08:58:44.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.651+0000 7fb20a86f640 1 -- start start 2024-04-25T08:58:44.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.652+0000 7fb20a86f640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb2040b6d60 0x7fb2040d0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:44.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.652+0000 7fb20a86f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb20415ded0 0x7fb2040d2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:44.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.652+0000 7fb20a86f640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2040d0d50 con 0x7fb2040d0430 2024-04-25T08:58:44.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.652+0000 7fb20a86f640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2040d1ed0 con 0x7fb2040b6d60 2024-04-25T08:58:44.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.652+0000 7fb20a86f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2040ce820 con 0x7fb20415ded0 2024-04-25T08:58:44.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.652+0000 7fb2037fe640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb2040b6d60 0x7fb2040d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:44.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.652+0000 7fb208e0c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb20415ded0 0x7fb2040d2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:44.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.652+0000 7fb2037fe640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb2040b6d60 0x7fb2040d0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:51694/0 (socket says 172.21.15.43:51694) 2024-04-25T08:58:44.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.652+0000 7fb208e0c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb20415ded0 0x7fb2040d2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:46776/0 (socket says 172.21.15.43:46776) 2024-04-25T08:58:44.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.652+0000 7fb2037fe640 1 -- 172.21.15.43:0/823320849 learned_addr learned my addr 172.21.15.43:0/823320849 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:44.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.653+0000 7fb2037fe640 1 -- 172.21.15.43:0/823320849 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb20415ded0 msgr2=0x7fb2040d2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:44.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.653+0000 7fb2037fe640 1 --2- 172.21.15.43:0/823320849 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb20415ded0 0x7fb2040d2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:44.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.653+0000 7fb2037fe640 1 -- 172.21.15.43:0/823320849 >> v1:172.21.15.193:6789/0 conn(0x7fb2040d0430 legacy=0x7fb2041621c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:44.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.653+0000 7fb2037fe640 1 -- 172.21.15.43:0/823320849 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2040ceb00 con 0x7fb2040b6d60 2024-04-25T08:58:44.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.653+0000 7fb2037fe640 1 --2- 172.21.15.43:0/823320849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2040b6d60 0x7fb2040d0810 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4009870 tx=0x7fb1f402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=594b3038496a3c06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:44.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.654+0000 7fb202ffd640 1 -- 172.21.15.43:0/823320849 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f4037070 con 0x7fb2040b6d60 2024-04-25T08:58:44.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.654+0000 7fb202ffd640 1 -- 172.21.15.43:0/823320849 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f4032440 con 0x7fb2040b6d60 2024-04-25T08:58:44.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.654+0000 7fb202ffd640 1 -- 172.21.15.43:0/823320849 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f4032810 con 0x7fb2040b6d60 2024-04-25T08:58:44.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.654+0000 7fb20a86f640 1 -- 172.21.15.43:0/823320849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2040b6d60 msgr2=0x7fb2040d0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:44.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.654+0000 7fb20a86f640 1 --2- 172.21.15.43:0/823320849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2040b6d60 0x7fb2040d0810 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4009870 tx=0x7fb1f402a270 comp rx=0 tx=0).stop 2024-04-25T08:58:44.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.654+0000 7fb20a86f640 1 -- 172.21.15.43:0/823320849 shutdown_connections 2024-04-25T08:58:44.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.654+0000 7fb20a86f640 1 --2- 172.21.15.43:0/823320849 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb20415ded0 0x7fb2040d2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:44.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.654+0000 7fb20a86f640 1 --2- 172.21.15.43:0/823320849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2040b6d60 0x7fb2040d0810 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:44.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.654+0000 7fb20a86f640 1 -- 172.21.15.43:0/823320849 >> 172.21.15.43:0/823320849 conn(0x7fb20406b1e0 msgr2=0x7fb20406b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:44.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.654+0000 7fb20a86f640 1 -- 172.21.15.43:0/823320849 shutdown_connections 2024-04-25T08:58:44.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.654+0000 7fb20a86f640 1 -- 172.21.15.43:0/823320849 wait complete. 2024-04-25T08:58:44.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.655+0000 7fb20a86f640 1 Processor -- start 2024-04-25T08:58:44.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.656+0000 7fb20a86f640 1 -- start start 2024-04-25T08:58:44.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.656+0000 7fb20a86f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb2040b6d60 0x7fb204159890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:44.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.656+0000 7fb20a86f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2040d0430 0x7fb2040c9d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:44.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.656+0000 7fb20a86f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb20415ded0 0x7fb20415fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:44.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.656+0000 7fb20a86f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2040ce280 con 0x7fb2040d0430 2024-04-25T08:58:44.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.656+0000 7fb20a86f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2040cf0c0 con 0x7fb20415ded0 2024-04-25T08:58:44.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.656+0000 7fb20a86f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2040ce400 con 0x7fb2040b6d60 2024-04-25T08:58:44.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.657+0000 7fb2037fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2040d0430 0x7fb2040c9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:44.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.657+0000 7fb2037fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2040d0430 0x7fb2040c9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50464/0 (socket says 172.21.15.43:50464) 2024-04-25T08:58:44.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.657+0000 7fb2037fe640 1 -- 172.21.15.43:0/2339245458 learned_addr learned my addr 172.21.15.43:0/2339245458 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:44.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.657+0000 7fb208e0c640 1 --2- 172.21.15.43:0/2339245458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb20415ded0 0x7fb20415fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:44.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.657+0000 7fb203fff640 1 --2- 172.21.15.43:0/2339245458 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb2040b6d60 0x7fb204159890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:44.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.657+0000 7fb2037fe640 1 -- 172.21.15.43:0/2339245458 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb2040b6d60 msgr2=0x7fb204159890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:44.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.657+0000 7fb2037fe640 1 --2- 172.21.15.43:0/2339245458 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb2040b6d60 0x7fb204159890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:44.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.657+0000 7fb2037fe640 1 -- 172.21.15.43:0/2339245458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb20415ded0 msgr2=0x7fb20415fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:44.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.657+0000 7fb2037fe640 1 --2- 172.21.15.43:0/2339245458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb20415ded0 0x7fb20415fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:44.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.657+0000 7fb2037fe640 1 -- 172.21.15.43:0/2339245458 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2041f4fa0 con 0x7fb2040d0430 2024-04-25T08:58:44.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.657+0000 7fb208e0c640 1 --2- 172.21.15.43:0/2339245458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb20415ded0 0x7fb20415fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:58:44.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.657+0000 7fb2037fe640 1 --2- 172.21.15.43:0/2339245458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2040d0430 0x7fb2040c9d90 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4000c00 tx=0x7fb1f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:44.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.657+0000 7fb200ff9640 1 -- 172.21.15.43:0/2339245458 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f4037040 con 0x7fb2040d0430 2024-04-25T08:58:44.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.657+0000 7fb200ff9640 1 -- 172.21.15.43:0/2339245458 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f4042070 con 0x7fb2040d0430 2024-04-25T08:58:44.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.657+0000 7fb200ff9640 1 -- 172.21.15.43:0/2339245458 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f4030880 con 0x7fb2040d0430 2024-04-25T08:58:44.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.657+0000 7fb20a86f640 1 -- 172.21.15.43:0/2339245458 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb204158db0 con 0x7fb2040d0430 2024-04-25T08:58:44.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.657+0000 7fb20a86f640 1 -- 172.21.15.43:0/2339245458 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2041600f0 con 0x7fb2040d0430 2024-04-25T08:58:44.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.658+0000 7fb20a86f640 1 -- 172.21.15.43:0/2339245458 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2040680f0 con 0x7fb2040d0430 2024-04-25T08:58:44.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.659+0000 7fb200ff9640 1 -- 172.21.15.43:0/2339245458 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb1f4030a20 con 0x7fb2040d0430 2024-04-25T08:58:44.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.659+0000 7fb200ff9640 1 --2- 172.21.15.43:0/2339245458 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb1f006b4d0 0x7fb1f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:44.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.660+0000 7fb200ff9640 1 -- 172.21.15.43:0/2339245458 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fb1f40aa860 con 0x7fb2040d0430 2024-04-25T08:58:44.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.660+0000 7fb203fff640 1 --2- 172.21.15.43:0/2339245458 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb1f006b4d0 0x7fb1f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:44.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.660+0000 7fb203fff640 1 --2- 172.21.15.43:0/2339245458 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb1f006b4d0 0x7fb1f006d990 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8001640 tx=0x7fb1f8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:44.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.664+0000 7fb200ff9640 1 -- 172.21.15.43:0/2339245458 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb1f40798d0 con 0x7fb2040d0430 2024-04-25T08:58:44.663 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:58:44.665 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:58:44.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:44.845+0000 7fb20a86f640 1 -- 172.21.15.43:0/2339245458 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb204074f60 con 0x7fb2040d0430 2024-04-25T08:58:45.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.531+0000 7fb200ff9640 1 -- 172.21.15.43:0/2339245458 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v71) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb1f40791d0 con 0x7fb2040d0430 2024-04-25T08:58:45.521 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' created 2024-04-25T08:58:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.535+0000 7fb20a86f640 1 -- 172.21.15.43:0/2339245458 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb1f006b4d0 msgr2=0x7fb1f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.535+0000 7fb20a86f640 1 --2- 172.21.15.43:0/2339245458 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb1f006b4d0 0x7fb1f006d990 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8001640 tx=0x7fb1f8009040 comp rx=0 tx=0).stop 2024-04-25T08:58:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.535+0000 7fb20a86f640 1 -- 172.21.15.43:0/2339245458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2040d0430 msgr2=0x7fb2040c9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.535+0000 7fb20a86f640 1 --2- 172.21.15.43:0/2339245458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2040d0430 0x7fb2040c9d90 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fb1f4000c00 tx=0x7fb1f40046c0 comp rx=0 tx=0).stop 2024-04-25T08:58:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.535+0000 7fb20a86f640 1 -- 172.21.15.43:0/2339245458 shutdown_connections 2024-04-25T08:58:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.535+0000 7fb20a86f640 1 --2- 172.21.15.43:0/2339245458 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb1f006b4d0 0x7fb1f006d990 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.535+0000 7fb20a86f640 1 --2- 172.21.15.43:0/2339245458 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb20415ded0 0x7fb20415fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.535+0000 7fb20a86f640 1 --2- 172.21.15.43:0/2339245458 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2040d0430 0x7fb2040c9d90 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.535+0000 7fb20a86f640 1 --2- 172.21.15.43:0/2339245458 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb2040b6d60 0x7fb204159890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:45.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.535+0000 7fb20a86f640 1 -- 172.21.15.43:0/2339245458 >> 172.21.15.43:0/2339245458 conn(0x7fb20406b1e0 msgr2=0x7fb2040b4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:45.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.543+0000 7fb20a86f640 1 -- 172.21.15.43:0/2339245458 shutdown_connections 2024-04-25T08:58:45.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.545+0000 7fb20a86f640 1 -- 172.21.15.43:0/2339245458 wait complete. 2024-04-25T08:58:45.554 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T08:58:45.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.673+0000 7fc11685d640 1 Processor -- start 2024-04-25T08:58:45.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.673+0000 7fc11685d640 1 -- start start 2024-04-25T08:58:45.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.674+0000 7fc11685d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc1100aee80 0x7fc11006d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:45.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.674+0000 7fc11685d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc1100cd2f0 0x7fc11015d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:45.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.674+0000 7fc11685d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc11015c280 con 0x7fc11006d600 2024-04-25T08:58:45.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.674+0000 7fc11685d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1100af260 con 0x7fc1100aee80 2024-04-25T08:58:45.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.674+0000 7fc11685d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc11015c560 con 0x7fc1100cd2f0 2024-04-25T08:58:45.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.674+0000 7fc10ffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc1100aee80 0x7fc11006d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:45.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.674+0000 7fc10f7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc11006d600 0x7fc1100ccbe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39178/0 (socket says 172.21.15.43:39178) 2024-04-25T08:58:45.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.674+0000 7fc10ffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc1100aee80 0x7fc11006d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50466/0 (socket says 172.21.15.43:50466) 2024-04-25T08:58:45.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.674+0000 7fc10f7fe640 1 -- 172.21.15.43:0/3223978489 learned_addr learned my addr 172.21.15.43:0/3223978489 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:45.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.675+0000 7fc114dfa640 1 --2- 172.21.15.43:0/3223978489 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc1100cd2f0 0x7fc11015d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:45.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.675+0000 7fc10ffff640 1 -- 172.21.15.43:0/3223978489 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc1100cd2f0 msgr2=0x7fc11015d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:45.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.675+0000 7fc10ffff640 1 --2- 172.21.15.43:0/3223978489 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc1100cd2f0 0x7fc11015d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:45.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.675+0000 7fc10ffff640 1 -- 172.21.15.43:0/3223978489 >> v1:172.21.15.43:6789/0 conn(0x7fc11006d600 legacy=0x7fc1100ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:45.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.675+0000 7fc10effd640 1 -- 172.21.15.43:0/3223978489 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 82885723 0 0) 0x7fc11015c280 con 0x7fc11006d600 2024-04-25T08:58:45.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.675+0000 7fc10ffff640 1 -- 172.21.15.43:0/3223978489 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc11015c840 con 0x7fc1100aee80 2024-04-25T08:58:45.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.675+0000 7fc10ffff640 1 --2- 172.21.15.43:0/3223978489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1100aee80 0x7fc11006d0c0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fc1000097a0 tx=0x7fc10002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3588d1b0fcdaea60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:45.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.676+0000 7fc10effd640 1 -- 172.21.15.43:0/3223978489 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc100037070 con 0x7fc1100aee80 2024-04-25T08:58:45.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.676+0000 7fc10effd640 1 -- 172.21.15.43:0/3223978489 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc100032440 con 0x7fc1100aee80 2024-04-25T08:58:45.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.676+0000 7fc10effd640 1 -- 172.21.15.43:0/3223978489 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc100032810 con 0x7fc1100aee80 2024-04-25T08:58:45.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.676+0000 7fc11685d640 1 -- 172.21.15.43:0/3223978489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1100aee80 msgr2=0x7fc11006d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:45.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.676+0000 7fc11685d640 1 --2- 172.21.15.43:0/3223978489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1100aee80 0x7fc11006d0c0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fc1000097a0 tx=0x7fc10002a180 comp rx=0 tx=0).stop 2024-04-25T08:58:45.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.676+0000 7fc11685d640 1 -- 172.21.15.43:0/3223978489 shutdown_connections 2024-04-25T08:58:45.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.676+0000 7fc11685d640 1 --2- 172.21.15.43:0/3223978489 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc1100cd2f0 0x7fc11015d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:45.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.676+0000 7fc11685d640 1 --2- 172.21.15.43:0/3223978489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1100aee80 0x7fc11006d0c0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:45.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.676+0000 7fc11685d640 1 -- 172.21.15.43:0/3223978489 >> 172.21.15.43:0/3223978489 conn(0x7fc11006ab30 msgr2=0x7fc11006af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:45.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.676+0000 7fc11685d640 1 -- 172.21.15.43:0/3223978489 shutdown_connections 2024-04-25T08:58:45.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.676+0000 7fc11685d640 1 -- 172.21.15.43:0/3223978489 wait complete. 2024-04-25T08:58:45.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.677+0000 7fc11685d640 1 Processor -- start 2024-04-25T08:58:45.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.678+0000 7fc11685d640 1 -- start start 2024-04-25T08:58:45.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.678+0000 7fc11685d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc11006d600 0x7fc11015a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:45.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.678+0000 7fc11685d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1100aee80 0x7fc110155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:45.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.678+0000 7fc11685d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1100cd2f0 0x7fc110155d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:45.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.678+0000 7fc11685d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11015dc50 con 0x7fc1100aee80 2024-04-25T08:58:45.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.678+0000 7fc11685d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc11015dad0 con 0x7fc1100cd2f0 2024-04-25T08:58:45.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.678+0000 7fc11685d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc11015ddd0 con 0x7fc11006d600 2024-04-25T08:58:45.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.678+0000 7fc10f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1100aee80 0x7fc110155810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:45.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.678+0000 7fc10f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1100aee80 0x7fc110155810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50480/0 (socket says 172.21.15.43:50480) 2024-04-25T08:58:45.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.678+0000 7fc10f7fe640 1 -- 172.21.15.43:0/1902367310 learned_addr learned my addr 172.21.15.43:0/1902367310 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:45.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.679+0000 7fc10ffff640 1 --2- 172.21.15.43:0/1902367310 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc11006d600 0x7fc11015a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:45.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.679+0000 7fc114dfa640 1 --2- 172.21.15.43:0/1902367310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1100cd2f0 0x7fc110155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:45.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.679+0000 7fc10f7fe640 1 -- 172.21.15.43:0/1902367310 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc11006d600 msgr2=0x7fc11015a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:45.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.679+0000 7fc10f7fe640 1 --2- 172.21.15.43:0/1902367310 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc11006d600 0x7fc11015a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:45.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.679+0000 7fc10f7fe640 1 -- 172.21.15.43:0/1902367310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1100cd2f0 msgr2=0x7fc110155d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:45.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.679+0000 7fc10f7fe640 1 --2- 172.21.15.43:0/1902367310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1100cd2f0 0x7fc110155d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:45.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.679+0000 7fc10f7fe640 1 -- 172.21.15.43:0/1902367310 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc110157e80 con 0x7fc1100aee80 2024-04-25T08:58:45.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.679+0000 7fc10f7fe640 1 --2- 172.21.15.43:0/1902367310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1100aee80 0x7fc110155810 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fc10400c4a0 tx=0x7fc10400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:45.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.679+0000 7fc10cff9640 1 -- 172.21.15.43:0/1902367310 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc104017070 con 0x7fc1100aee80 2024-04-25T08:58:45.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.679+0000 7fc10cff9640 1 -- 172.21.15.43:0/1902367310 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc104005ce0 con 0x7fc1100aee80 2024-04-25T08:58:45.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.679+0000 7fc11685d640 1 -- 172.21.15.43:0/1902367310 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc110158160 con 0x7fc1100aee80 2024-04-25T08:58:45.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.679+0000 7fc10cff9640 1 -- 172.21.15.43:0/1902367310 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1040072b0 con 0x7fc1100aee80 2024-04-25T08:58:45.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.679+0000 7fc11685d640 1 -- 172.21.15.43:0/1902367310 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc110158720 con 0x7fc1100aee80 2024-04-25T08:58:45.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.680+0000 7fc11685d640 1 -- 172.21.15.43:0/1902367310 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc110074f60 con 0x7fc1100aee80 2024-04-25T08:58:45.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.681+0000 7fc10cff9640 1 -- 172.21.15.43:0/1902367310 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc10401a070 con 0x7fc1100aee80 2024-04-25T08:58:45.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.681+0000 7fc10cff9640 1 --2- 172.21.15.43:0/1902367310 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc0fc06b4d0 0x7fc0fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:45.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.681+0000 7fc10cff9640 1 -- 172.21.15.43:0/1902367310 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fc1040894e0 con 0x7fc1100aee80 2024-04-25T08:58:45.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.682+0000 7fc10ffff640 1 --2- 172.21.15.43:0/1902367310 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc0fc06b4d0 0x7fc0fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:45.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.682+0000 7fc10ffff640 1 --2- 172.21.15.43:0/1902367310 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc0fc06b4d0 0x7fc0fc06d990 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc100034000 tx=0x7fc10002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:45.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.686+0000 7fc10cff9640 1 -- 172.21.15.43:0/1902367310 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc104057d70 con 0x7fc1100aee80 2024-04-25T08:58:45.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:45.905+0000 7fc11685d640 1 -- 172.21.15.43:0/1902367310 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc1100725f0 con 0x7fc1100aee80 2024-04-25T08:58:46.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.542+0000 7fc10cff9640 1 -- 172.21.15.43:0/1902367310 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v44) v1 ==== 174+0+0 (secure 0 0 0) 0x7fc10405e0c0 con 0x7fc1100aee80 2024-04-25T08:58:46.537 INFO:teuthology.orchestra.run.smithi043.stderr:new fs with metadata pool 10 and data pool 11 2024-04-25T08:58:46.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.551+0000 7fc11685d640 1 -- 172.21.15.43:0/1902367310 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc0fc06b4d0 msgr2=0x7fc0fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:46.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.551+0000 7fc11685d640 1 --2- 172.21.15.43:0/1902367310 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc0fc06b4d0 0x7fc0fc06d990 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc100034000 tx=0x7fc10002e000 comp rx=0 tx=0).stop 2024-04-25T08:58:46.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.551+0000 7fc11685d640 1 -- 172.21.15.43:0/1902367310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1100aee80 msgr2=0x7fc110155810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:46.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.551+0000 7fc11685d640 1 --2- 172.21.15.43:0/1902367310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1100aee80 0x7fc110155810 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fc10400c4a0 tx=0x7fc10400c960 comp rx=0 tx=0).stop 2024-04-25T08:58:46.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.554+0000 7fc11685d640 1 -- 172.21.15.43:0/1902367310 shutdown_connections 2024-04-25T08:58:46.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.554+0000 7fc11685d640 1 --2- 172.21.15.43:0/1902367310 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc0fc06b4d0 0x7fc0fc06d990 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:46.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.554+0000 7fc11685d640 1 --2- 172.21.15.43:0/1902367310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1100cd2f0 0x7fc110155d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:46.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.554+0000 7fc11685d640 1 --2- 172.21.15.43:0/1902367310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1100aee80 0x7fc110155810 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:46.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.554+0000 7fc11685d640 1 --2- 172.21.15.43:0/1902367310 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc11006d600 0x7fc11015a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:46.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.554+0000 7fc11685d640 1 -- 172.21.15.43:0/1902367310 >> 172.21.15.43:0/1902367310 conn(0x7fc11006ab30 msgr2=0x7fc1100b0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:46.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.556+0000 7fc11685d640 1 -- 172.21.15.43:0/1902367310 shutdown_connections 2024-04-25T08:58:46.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.558+0000 7fc11685d640 1 -- 172.21.15.43:0/1902367310 wait complete. 2024-04-25T08:58:46.565 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:58:46.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.682+0000 7fadae949640 1 Processor -- start 2024-04-25T08:58:46.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.682+0000 7fadae949640 1 -- start start 2024-04-25T08:58:46.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.683+0000 7fadae949640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fada80d0af0 0x7fada8160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:46.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.683+0000 7fadae949640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fada80d2920 con 0x7fada815dce0 2024-04-25T08:58:46.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.683+0000 7fadae949640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fada80cee80 con 0x7fada815ef50 2024-04-25T08:58:46.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.683+0000 7fadae949640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fada80cec60 con 0x7fada80d0af0 2024-04-25T08:58:46.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.683+0000 7fada7fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fada815dce0 0x7fada80ce550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39182/0 (socket says 172.21.15.43:39182) 2024-04-25T08:58:46.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.683+0000 7fada7fff640 1 -- 172.21.15.43:0/1805628970 learned_addr learned my addr 172.21.15.43:0/1805628970 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:46.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.683+0000 7fadacee6640 1 --2- 172.21.15.43:0/1805628970 >> v2:172.21.15.193:3300/0 conn(0x7fada80d0af0 0x7fada8160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:46.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.683+0000 7fada6ffd640 1 -- 172.21.15.43:0/1805628970 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 374273736 0 0) 0x7fada80d2920 con 0x7fada815dce0 2024-04-25T08:58:46.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.683+0000 7fada6ffd640 1 -- 172.21.15.43:0/1805628970 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad90003520 con 0x7fada815dce0 2024-04-25T08:58:46.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.684+0000 7fada6ffd640 1 -- 172.21.15.43:0/1805628970 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3613947634 0 0) 0x7fada80cee80 con 0x7fada815ef50 2024-04-25T08:58:46.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.684+0000 7fada6ffd640 1 -- 172.21.15.43:0/1805628970 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fada80d2920 con 0x7fada815ef50 2024-04-25T08:58:46.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.684+0000 7fadacee6640 1 -- 172.21.15.43:0/1805628970 >> v1:172.21.15.193:6789/0 conn(0x7fada815ef50 legacy=0x7fada80d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:46.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.684+0000 7fadacee6640 1 -- 172.21.15.43:0/1805628970 >> v1:172.21.15.43:6789/0 conn(0x7fada815dce0 legacy=0x7fada80ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:46.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.684+0000 7fadacee6640 1 -- 172.21.15.43:0/1805628970 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada80cf160 con 0x7fada80d0af0 2024-04-25T08:58:46.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.684+0000 7fadacee6640 1 --2- 172.21.15.43:0/1805628970 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fada80d0af0 0x7fada8160980 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fada0009d90 tx=0x7fada002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15da8c4d26f84d53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:46.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.685+0000 7fada6ffd640 1 -- 172.21.15.43:0/1805628970 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada0007740 con 0x7fada80d0af0 2024-04-25T08:58:46.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.685+0000 7fada6ffd640 1 -- 172.21.15.43:0/1805628970 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fada00078e0 con 0x7fada80d0af0 2024-04-25T08:58:46.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.685+0000 7fada6ffd640 1 -- 172.21.15.43:0/1805628970 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada0007cb0 con 0x7fada80d0af0 2024-04-25T08:58:46.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.685+0000 7fadae949640 1 -- 172.21.15.43:0/1805628970 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fada80d0af0 msgr2=0x7fada8160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:46.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.685+0000 7fadae949640 1 --2- 172.21.15.43:0/1805628970 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fada80d0af0 0x7fada8160980 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fada0009d90 tx=0x7fada002a660 comp rx=0 tx=0).stop 2024-04-25T08:58:46.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.685+0000 7fadae949640 1 -- 172.21.15.43:0/1805628970 shutdown_connections 2024-04-25T08:58:46.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.685+0000 7fadae949640 1 --2- 172.21.15.43:0/1805628970 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fada80d0af0 0x7fada8160980 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:46.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.685+0000 7fadae949640 1 -- 172.21.15.43:0/1805628970 >> 172.21.15.43:0/1805628970 conn(0x7fada809cd00 msgr2=0x7fada80d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:46.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.685+0000 7fadae949640 1 -- 172.21.15.43:0/1805628970 shutdown_connections 2024-04-25T08:58:46.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.685+0000 7fadae949640 1 -- 172.21.15.43:0/1805628970 wait complete. 2024-04-25T08:58:46.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.686+0000 7fadae949640 1 Processor -- start 2024-04-25T08:58:46.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.687+0000 7fadae949640 1 -- start start 2024-04-25T08:58:46.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.687+0000 7fadae949640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fada80d0af0 0x7fada809d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:46.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.687+0000 7fadae949640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fada815dce0 0x7fada80ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:46.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.687+0000 7fadae949640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fada815ef50 0x7fada80ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:46.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.687+0000 7fadae949640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada815f6b0 con 0x7fada815ef50 2024-04-25T08:58:46.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.687+0000 7fadae949640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada80cf720 con 0x7fada815dce0 2024-04-25T08:58:46.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.687+0000 7fadae949640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fada815e0c0 con 0x7fada80d0af0 2024-04-25T08:58:46.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.687+0000 7fadacee6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fada815ef50 0x7fada80ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:46.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.687+0000 7fadacee6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fada815ef50 0x7fada80ccff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50482/0 (socket says 172.21.15.43:50482) 2024-04-25T08:58:46.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.687+0000 7fadacee6640 1 -- 172.21.15.43:0/279871543 learned_addr learned my addr 172.21.15.43:0/279871543 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:46.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.688+0000 7fada7fff640 1 --2- 172.21.15.43:0/279871543 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fada80d0af0 0x7fada809d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:46.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.688+0000 7fada77fe640 1 --2- 172.21.15.43:0/279871543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fada815dce0 0x7fada80ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:46.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.688+0000 7fadacee6640 1 -- 172.21.15.43:0/279871543 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fada80d0af0 msgr2=0x7fada809d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:46.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.688+0000 7fadacee6640 1 --2- 172.21.15.43:0/279871543 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fada80d0af0 0x7fada809d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:46.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.688+0000 7fadacee6640 1 -- 172.21.15.43:0/279871543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fada815dce0 msgr2=0x7fada80ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:46.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.688+0000 7fadacee6640 1 --2- 172.21.15.43:0/279871543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fada815dce0 0x7fada80ccab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:46.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.688+0000 7fadacee6640 1 -- 172.21.15.43:0/279871543 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada809eb20 con 0x7fada815ef50 2024-04-25T08:58:46.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.688+0000 7fadacee6640 1 --2- 172.21.15.43:0/279871543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fada815ef50 0x7fada80ccff0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fada0009eb0 tx=0x7fada0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:46.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.688+0000 7fada4ff9640 1 -- 172.21.15.43:0/279871543 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada0040070 con 0x7fada815ef50 2024-04-25T08:58:46.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.688+0000 7fada4ff9640 1 -- 172.21.15.43:0/279871543 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fada0031bc0 con 0x7fada815ef50 2024-04-25T08:58:46.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.688+0000 7fadae949640 1 -- 172.21.15.43:0/279871543 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fada809df10 con 0x7fada815ef50 2024-04-25T08:58:46.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.688+0000 7fada4ff9640 1 -- 172.21.15.43:0/279871543 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada0009450 con 0x7fada815ef50 2024-04-25T08:58:46.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.688+0000 7fadae949640 1 -- 172.21.15.43:0/279871543 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fada809e4d0 con 0x7fada815ef50 2024-04-25T08:58:46.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.689+0000 7fadae949640 1 -- 172.21.15.43:0/279871543 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fada80680f0 con 0x7fada815ef50 2024-04-25T08:58:46.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.690+0000 7fada4ff9640 1 -- 172.21.15.43:0/279871543 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fada0039040 con 0x7fada815ef50 2024-04-25T08:58:46.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.690+0000 7fada4ff9640 1 --2- 172.21.15.43:0/279871543 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fad9406b4d0 0x7fad9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:46.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.690+0000 7fada4ff9640 1 -- 172.21.15.43:0/279871543 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fada00aba90 con 0x7fada815ef50 2024-04-25T08:58:46.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.691+0000 7fada7fff640 1 --2- 172.21.15.43:0/279871543 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fad9406b4d0 0x7fad9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:46.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.691+0000 7fada7fff640 1 --2- 172.21.15.43:0/279871543 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fad9406b4d0 0x7fad9406d990 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fad98007fd0 tx=0x7fad98009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:46.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.695+0000 7fada4ff9640 1 -- 172.21.15.43:0/279871543 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fada007aa30 con 0x7fada815ef50 2024-04-25T08:58:46.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.877+0000 7fadae949640 1 -- 172.21.15.43:0/279871543 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fada8074ef0 con 0x7fada815ef50 2024-04-25T08:58:46.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.878+0000 7fada4ff9640 1 -- 172.21.15.43:0/279871543 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+16645 (secure 0 0 0) 0x7fada007a330 con 0x7fada815ef50 2024-04-25T08:58:46.867 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:58:46.868 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":72,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:58:45.906377+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:58:44.188924+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T08:58:44.877113+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:58:46.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.880+0000 7fadae949640 1 -- 172.21.15.43:0/279871543 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fad9406b4d0 msgr2=0x7fad9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:46.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.880+0000 7fadae949640 1 --2- 172.21.15.43:0/279871543 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fad9406b4d0 0x7fad9406d990 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fad98007fd0 tx=0x7fad98009040 comp rx=0 tx=0).stop 2024-04-25T08:58:46.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.880+0000 7fadae949640 1 -- 172.21.15.43:0/279871543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fada815ef50 msgr2=0x7fada80ccff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:46.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.880+0000 7fadae949640 1 --2- 172.21.15.43:0/279871543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fada815ef50 0x7fada80ccff0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fada0009eb0 tx=0x7fada0004a40 comp rx=0 tx=0).stop 2024-04-25T08:58:46.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.880+0000 7fadae949640 1 -- 172.21.15.43:0/279871543 shutdown_connections 2024-04-25T08:58:46.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.880+0000 7fadae949640 1 --2- 172.21.15.43:0/279871543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fada815ef50 0x7fada80ccff0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:46.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.880+0000 7fadae949640 1 --2- 172.21.15.43:0/279871543 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fad9406b4d0 0x7fad9406d990 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:46.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.880+0000 7fadae949640 1 --2- 172.21.15.43:0/279871543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fada815dce0 0x7fada80ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:46.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.880+0000 7fadae949640 1 --2- 172.21.15.43:0/279871543 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fada80d0af0 0x7fada809d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:46.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.880+0000 7fadae949640 1 -- 172.21.15.43:0/279871543 >> 172.21.15.43:0/279871543 conn(0x7fada809cd00 msgr2=0x7fada80c9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:46.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.880+0000 7fadae949640 1 -- 172.21.15.43:0/279871543 shutdown_connections 2024-04-25T08:58:46.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.880+0000 7fadae949640 1 -- 172.21.15.43:0/279871543 wait complete. 2024-04-25T08:58:46.881 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:58:46.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.992+0000 7f1fe1c76640 1 Processor -- start 2024-04-25T08:58:46.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.992+0000 7f1fe1c76640 1 -- start start 2024-04-25T08:58:46.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.993+0000 7f1fe1c76640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1fdc15f180 0x7f1fdc0cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:46.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.993+0000 7f1fe1c76640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1fdc0b23f0 0x7f1fdc0cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:46.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.993+0000 7f1fe1c76640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fdc0ccf10 con 0x7f1fdc0b2cc0 2024-04-25T08:58:46.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.993+0000 7f1fe1c76640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fdc0cccc0 con 0x7f1fdc0b23f0 2024-04-25T08:58:46.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.993+0000 7f1fe1c76640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fdc0cd840 con 0x7f1fdc15f180 2024-04-25T08:58:46.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.993+0000 7f1fdaffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1fdc0b23f0 0x7f1fdc0cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:46.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.993+0000 7f1fdaffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1fdc0b23f0 0x7f1fdc0cfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50496/0 (socket says 172.21.15.43:50496) 2024-04-25T08:58:46.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.993+0000 7f1fdaffd640 1 -- 172.21.15.43:0/2183161413 learned_addr learned my addr 172.21.15.43:0/2183161413 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:46.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.993+0000 7f1fdbfff640 1 --1- 172.21.15.43:0/2183161413 >> v1:172.21.15.43:6789/0 conn(0x7f1fdc0b2cc0 0x7f1fdc15f8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39190/0 (socket says 172.21.15.43:39190) 2024-04-25T08:58:46.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.994+0000 7f1fdb7fe640 1 --2- 172.21.15.43:0/2183161413 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1fdc15f180 0x7f1fdc0cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:46.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.994+0000 7f1fdaffd640 1 -- 172.21.15.43:0/2183161413 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1fdc15f180 msgr2=0x7f1fdc0cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:46.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.994+0000 7f1fdaffd640 1 --2- 172.21.15.43:0/2183161413 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1fdc15f180 0x7f1fdc0cf860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:46.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.994+0000 7f1fdaffd640 1 -- 172.21.15.43:0/2183161413 >> v1:172.21.15.43:6789/0 conn(0x7f1fdc0b2cc0 legacy=0x7f1fdc15f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:46.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.994+0000 7f1fda7fc640 1 -- 172.21.15.43:0/2183161413 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3561399215 0 0) 0x7f1fdc0ccf10 con 0x7f1fdc0b2cc0 2024-04-25T08:58:46.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.994+0000 7f1fdaffd640 1 -- 172.21.15.43:0/2183161413 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fdc0cd1f0 con 0x7f1fdc0b23f0 2024-04-25T08:58:46.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.994+0000 7f1fdaffd640 1 --2- 172.21.15.43:0/2183161413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fdc0b23f0 0x7f1fdc0cfda0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0009870 tx=0x7f1fd002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b92f5f40c5848a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:46.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.995+0000 7f1fda7fc640 1 -- 172.21.15.43:0/2183161413 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0037070 con 0x7f1fdc0b23f0 2024-04-25T08:58:46.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.995+0000 7f1fda7fc640 1 -- 172.21.15.43:0/2183161413 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd0032440 con 0x7f1fdc0b23f0 2024-04-25T08:58:46.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.995+0000 7f1fda7fc640 1 -- 172.21.15.43:0/2183161413 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0032810 con 0x7f1fdc0b23f0 2024-04-25T08:58:46.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.995+0000 7f1fe1c76640 1 -- 172.21.15.43:0/2183161413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fdc0b23f0 msgr2=0x7f1fdc0cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:46.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.995+0000 7f1fe1c76640 1 --2- 172.21.15.43:0/2183161413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fdc0b23f0 0x7f1fdc0cfda0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0009870 tx=0x7f1fd002a1c0 comp rx=0 tx=0).stop 2024-04-25T08:58:46.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.995+0000 7f1fe1c76640 1 -- 172.21.15.43:0/2183161413 shutdown_connections 2024-04-25T08:58:46.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.995+0000 7f1fe1c76640 1 --2- 172.21.15.43:0/2183161413 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fdc0b23f0 0x7f1fdc0cfda0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:46.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.995+0000 7f1fe1c76640 1 --2- 172.21.15.43:0/2183161413 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1fdc15f180 0x7f1fdc0cf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:46.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.995+0000 7f1fe1c76640 1 -- 172.21.15.43:0/2183161413 >> 172.21.15.43:0/2183161413 conn(0x7f1fdc0d17f0 msgr2=0x7f1fdc0d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:46.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.995+0000 7f1fe1c76640 1 -- 172.21.15.43:0/2183161413 shutdown_connections 2024-04-25T08:58:46.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.995+0000 7f1fe1c76640 1 -- 172.21.15.43:0/2183161413 wait complete. 2024-04-25T08:58:46.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.996+0000 7f1fe1c76640 1 Processor -- start 2024-04-25T08:58:46.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.996+0000 7f1fe1c76640 1 -- start start 2024-04-25T08:58:46.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.997+0000 7f1fe1c76640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fdc0b23f0 0x7f1fdc0b07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:46.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.997+0000 7f1fe1c76640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1fdc0b2cc0 0x7f1fdc0aea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:46.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.997+0000 7f1fe1c76640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1fdc15f180 0x7f1fdc0c9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:46.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.997+0000 7f1fe1c76640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fdc15f560 con 0x7f1fdc0b23f0 2024-04-25T08:58:46.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.997+0000 7f1fe1c76640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fdc0cece0 con 0x7f1fdc0b2cc0 2024-04-25T08:58:46.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.997+0000 7f1fe1c76640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fdc0b0cf0 con 0x7f1fdc15f180 2024-04-25T08:58:46.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.997+0000 7f1fdb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fdc0b23f0 0x7f1fdc0b07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:46.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.997+0000 7f1fdb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fdc0b23f0 0x7f1fdc0b07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50510/0 (socket says 172.21.15.43:50510) 2024-04-25T08:58:46.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.997+0000 7f1fdb7fe640 1 -- 172.21.15.43:0/794819127 learned_addr learned my addr 172.21.15.43:0/794819127 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:46.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.997+0000 7f1fdbfff640 1 --2- 172.21.15.43:0/794819127 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1fdc15f180 0x7f1fdc0c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:46.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.997+0000 7f1fdaffd640 1 --2- 172.21.15.43:0/794819127 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1fdc0b2cc0 0x7f1fdc0aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:46.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.998+0000 7f1fdb7fe640 1 -- 172.21.15.43:0/794819127 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1fdc15f180 msgr2=0x7f1fdc0c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:46.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.998+0000 7f1fdb7fe640 1 --2- 172.21.15.43:0/794819127 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1fdc15f180 0x7f1fdc0c9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:46.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.998+0000 7f1fdb7fe640 1 -- 172.21.15.43:0/794819127 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1fdc0b2cc0 msgr2=0x7f1fdc0aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:46.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.998+0000 7f1fdb7fe640 1 --2- 172.21.15.43:0/794819127 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1fdc0b2cc0 0x7f1fdc0aea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:46.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.998+0000 7f1fdb7fe640 1 -- 172.21.15.43:0/794819127 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fdc15dce0 con 0x7f1fdc0b23f0 2024-04-25T08:58:46.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.998+0000 7f1fdbfff640 1 --2- 172.21.15.43:0/794819127 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1fdc15f180 0x7f1fdc0c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:58:46.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.998+0000 7f1fdb7fe640 1 --2- 172.21.15.43:0/794819127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fdc0b23f0 0x7f1fdc0b07b0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc00e8a0 tx=0x7f1fcc00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:46.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.998+0000 7f1fc3fff640 1 -- 172.21.15.43:0/794819127 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc017070 con 0x7f1fdc0b23f0 2024-04-25T08:58:46.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.998+0000 7f1fc3fff640 1 -- 172.21.15.43:0/794819127 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fcc00c940 con 0x7f1fdc0b23f0 2024-04-25T08:58:46.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.998+0000 7f1fc3fff640 1 -- 172.21.15.43:0/794819127 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fcc00cd30 con 0x7f1fdc0b23f0 2024-04-25T08:58:46.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.998+0000 7f1fe1c76640 1 -- 172.21.15.43:0/794819127 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fdc15dfc0 con 0x7f1fdc0b23f0 2024-04-25T08:58:46.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.998+0000 7f1fe1c76640 1 -- 172.21.15.43:0/794819127 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fdc0b02b0 con 0x7f1fdc0b23f0 2024-04-25T08:58:46.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.999+0000 7f1fe1c76640 1 -- 172.21.15.43:0/794819127 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fdc0afdf0 con 0x7f1fdc0b23f0 2024-04-25T08:58:46.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:46.999+0000 7f1fc3fff640 1 -- 172.21.15.43:0/794819127 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1fcc01a070 con 0x7f1fdc0b23f0 2024-04-25T08:58:46.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.000+0000 7f1fc3fff640 1 --2- 172.21.15.43:0/794819127 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1fc806b4d0 0x7f1fc806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:46.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.000+0000 7f1fc3fff640 1 -- 172.21.15.43:0/794819127 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1fcc08a380 con 0x7f1fdc0b23f0 2024-04-25T08:58:46.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.000+0000 7f1fdaffd640 1 --2- 172.21.15.43:0/794819127 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1fc806b4d0 0x7f1fc806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:46.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.000+0000 7f1fdaffd640 1 --2- 172.21.15.43:0/794819127 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1fc806b4d0 0x7f1fc806d990 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0034000 tx=0x7f1fd002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:46.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.004+0000 7f1fc3fff640 1 -- 172.21.15.43:0/794819127 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1fdc0afdf0 con 0x7f1fdc0b23f0 2024-04-25T08:58:47.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.184+0000 7f1fe1c76640 1 -- 172.21.15.43:0/794819127 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1fdc074ef0 con 0x7f1fdc0b23f0 2024-04-25T08:58:47.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.185+0000 7f1fc3fff640 1 -- 172.21.15.43:0/794819127 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) v1 ==== 74+0+16645 (secure 0 0 0) 0x7f1fcc058bb0 con 0x7f1fdc0b23f0 2024-04-25T08:58:47.175 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:58:47.175 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":72,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:58:45.906377+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:58:44.188924+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T08:58:44.877113+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:58:47.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.187+0000 7f1fe1c76640 1 -- 172.21.15.43:0/794819127 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1fc806b4d0 msgr2=0x7f1fc806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:47.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.187+0000 7f1fe1c76640 1 --2- 172.21.15.43:0/794819127 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1fc806b4d0 0x7f1fc806d990 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0034000 tx=0x7f1fd002e000 comp rx=0 tx=0).stop 2024-04-25T08:58:47.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.187+0000 7f1fe1c76640 1 -- 172.21.15.43:0/794819127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fdc0b23f0 msgr2=0x7f1fdc0b07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:47.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.187+0000 7f1fe1c76640 1 --2- 172.21.15.43:0/794819127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fdc0b23f0 0x7f1fdc0b07b0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc00e8a0 tx=0x7f1fcc00ed60 comp rx=0 tx=0).stop 2024-04-25T08:58:47.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.187+0000 7f1fe1c76640 1 -- 172.21.15.43:0/794819127 shutdown_connections 2024-04-25T08:58:47.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.187+0000 7f1fe1c76640 1 --2- 172.21.15.43:0/794819127 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1fdc15f180 0x7f1fdc0c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:47.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.187+0000 7f1fe1c76640 1 --2- 172.21.15.43:0/794819127 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1fc806b4d0 0x7f1fc806d990 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:47.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.187+0000 7f1fe1c76640 1 --2- 172.21.15.43:0/794819127 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1fdc0b2cc0 0x7f1fdc0aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:47.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.187+0000 7f1fe1c76640 1 --2- 172.21.15.43:0/794819127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1fdc0b23f0 0x7f1fdc0b07b0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:47.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.187+0000 7f1fe1c76640 1 -- 172.21.15.43:0/794819127 >> 172.21.15.43:0/794819127 conn(0x7f1fdc0d17f0 msgr2=0x7f1fdc0b70e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:47.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.188+0000 7f1fe1c76640 1 -- 172.21.15.43:0/794819127 shutdown_connections 2024-04-25T08:58:47.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.188+0000 7f1fe1c76640 1 -- 172.21.15.43:0/794819127 wait complete. 2024-04-25T08:58:47.188 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T08:58:47.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.302+0000 7f4db4e74640 1 Processor -- start 2024-04-25T08:58:47.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.302+0000 7f4db4e74640 1 -- start start 2024-04-25T08:58:47.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.303+0000 7f4db4e74640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f4db00d27b0 0x7f4db015ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:47.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.303+0000 7f4db4e74640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4db00cf110 con 0x7f4db015dec0 2024-04-25T08:58:47.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.303+0000 7f4db4e74640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4db00cf3f0 con 0x7f4db00d04a0 2024-04-25T08:58:47.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.303+0000 7f4db4e74640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4db00cf6d0 con 0x7f4db00d27b0 2024-04-25T08:58:47.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.303+0000 7f4dadd9b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4db015dec0 0x7f4db00ce8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:39200/0 (socket says 172.21.15.43:39200) 2024-04-25T08:58:47.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.303+0000 7f4dadd9b640 1 -- 172.21.15.43:0/2858731133 learned_addr learned my addr 172.21.15.43:0/2858731133 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:47.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.303+0000 7f4daed9d640 1 --2- 172.21.15.43:0/2858731133 >> v2:172.21.15.193:3300/0 conn(0x7f4db00d27b0 0x7f4db015ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:47.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.304+0000 7f4dad59a640 1 -- 172.21.15.43:0/2858731133 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2770149447 0 0) 0x7f4db00cf110 con 0x7f4db015dec0 2024-04-25T08:58:47.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.304+0000 7f4dad59a640 1 -- 172.21.15.43:0/2858731133 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d9c003520 con 0x7f4db015dec0 2024-04-25T08:58:47.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.304+0000 7f4dad59a640 1 -- 172.21.15.43:0/2858731133 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2868732002 0 0) 0x7f4db00cf3f0 con 0x7f4db00d04a0 2024-04-25T08:58:47.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.304+0000 7f4dad59a640 1 -- 172.21.15.43:0/2858731133 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4db00cf110 con 0x7f4db00d04a0 2024-04-25T08:58:47.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.304+0000 7f4daed9d640 1 -- 172.21.15.43:0/2858731133 >> v1:172.21.15.193:6789/0 conn(0x7f4db00d04a0 legacy=0x7f4db00d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:47.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.304+0000 7f4daed9d640 1 -- 172.21.15.43:0/2858731133 >> v1:172.21.15.43:6789/0 conn(0x7f4db015dec0 legacy=0x7f4db00ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:47.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.304+0000 7f4daed9d640 1 -- 172.21.15.43:0/2858731133 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db00cf9b0 con 0x7f4db00d27b0 2024-04-25T08:58:47.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.305+0000 7f4daed9d640 1 --2- 172.21.15.43:0/2858731133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4db00d27b0 0x7f4db015ed90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4da0009d90 tx=0x7f4da002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9576f6721481f7cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:47.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.305+0000 7f4dad59a640 1 -- 172.21.15.43:0/2858731133 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da0007740 con 0x7f4db00d27b0 2024-04-25T08:58:47.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.305+0000 7f4dad59a640 1 -- 172.21.15.43:0/2858731133 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4da00078e0 con 0x7f4db00d27b0 2024-04-25T08:58:47.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.305+0000 7f4dad59a640 1 -- 172.21.15.43:0/2858731133 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da0007c90 con 0x7f4db00d27b0 2024-04-25T08:58:47.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.305+0000 7f4db4e74640 1 -- 172.21.15.43:0/2858731133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4db00d27b0 msgr2=0x7f4db015ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:47.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.305+0000 7f4db4e74640 1 --2- 172.21.15.43:0/2858731133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4db00d27b0 0x7f4db015ed90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4da0009d90 tx=0x7f4da002a660 comp rx=0 tx=0).stop 2024-04-25T08:58:47.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.305+0000 7f4db4e74640 1 -- 172.21.15.43:0/2858731133 shutdown_connections 2024-04-25T08:58:47.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.305+0000 7f4db4e74640 1 --2- 172.21.15.43:0/2858731133 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4db00d27b0 0x7f4db015ed90 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:47.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.305+0000 7f4db4e74640 1 -- 172.21.15.43:0/2858731133 >> 172.21.15.43:0/2858731133 conn(0x7f4db009cd40 msgr2=0x7f4db00d3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:47.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.305+0000 7f4db4e74640 1 -- 172.21.15.43:0/2858731133 shutdown_connections 2024-04-25T08:58:47.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.305+0000 7f4db4e74640 1 -- 172.21.15.43:0/2858731133 wait complete. 2024-04-25T08:58:47.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.307+0000 7f4db4e74640 1 Processor -- start 2024-04-25T08:58:47.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.307+0000 7f4db4e74640 1 -- start start 2024-04-25T08:58:47.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.307+0000 7f4db4e74640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4db00d04a0 0x7f4db00cae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:47.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.307+0000 7f4db4e74640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4db00d27b0 0x7f4db00c9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:47.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.307+0000 7f4db4e74640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4db015dec0 0x7f4db00ca4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:47.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.307+0000 7f4db4e74640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db0168c50 con 0x7f4db00d04a0 2024-04-25T08:58:47.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.307+0000 7f4db4e74640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4db0168ad0 con 0x7f4db015dec0 2024-04-25T08:58:47.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.307+0000 7f4db4e74640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4db0168dd0 con 0x7f4db00d27b0 2024-04-25T08:58:47.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.308+0000 7f4dae59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4db00d04a0 0x7f4db00cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:47.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.308+0000 7f4dae59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4db00d04a0 0x7f4db00cae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50514/0 (socket says 172.21.15.43:50514) 2024-04-25T08:58:47.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.308+0000 7f4dae59c640 1 -- 172.21.15.43:0/2664947485 learned_addr learned my addr 172.21.15.43:0/2664947485 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:47.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.308+0000 7f4daed9d640 1 --2- 172.21.15.43:0/2664947485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4db015dec0 0x7f4db00ca4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:47.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.308+0000 7f4dadd9b640 1 --2- 172.21.15.43:0/2664947485 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4db00d27b0 0x7f4db00c9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:47.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.308+0000 7f4dae59c640 1 -- 172.21.15.43:0/2664947485 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4db00d27b0 msgr2=0x7f4db00c9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:47.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.308+0000 7f4dae59c640 1 --2- 172.21.15.43:0/2664947485 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4db00d27b0 0x7f4db00c9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:47.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.308+0000 7f4dae59c640 1 -- 172.21.15.43:0/2664947485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4db015dec0 msgr2=0x7f4db00ca4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:47.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.308+0000 7f4dae59c640 1 --2- 172.21.15.43:0/2664947485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4db015dec0 0x7f4db00ca4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:47.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.308+0000 7f4dae59c640 1 -- 172.21.15.43:0/2664947485 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db00b8f00 con 0x7f4db00d04a0 2024-04-25T08:58:47.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.308+0000 7f4dadd9b640 1 --2- 172.21.15.43:0/2664947485 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4db00d27b0 0x7f4db00c9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:58:47.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.308+0000 7f4dae59c640 1 --2- 172.21.15.43:0/2664947485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4db00d04a0 0x7f4db00cae10 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f4da800c430 tx=0x7f4da800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:47.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.308+0000 7f4da6ffd640 1 -- 172.21.15.43:0/2664947485 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da8017070 con 0x7f4db00d04a0 2024-04-25T08:58:47.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.308+0000 7f4da6ffd640 1 -- 172.21.15.43:0/2664947485 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4da80052f0 con 0x7f4db00d04a0 2024-04-25T08:58:47.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.308+0000 7f4da6ffd640 1 -- 172.21.15.43:0/2664947485 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4da8003830 con 0x7f4db00d04a0 2024-04-25T08:58:47.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.308+0000 7f4db4e74640 1 -- 172.21.15.43:0/2664947485 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4db00b7f90 con 0x7f4db00d04a0 2024-04-25T08:58:47.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.308+0000 7f4db4e74640 1 -- 172.21.15.43:0/2664947485 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4db00b8550 con 0x7f4db00d04a0 2024-04-25T08:58:47.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.309+0000 7f4db4e74640 1 -- 172.21.15.43:0/2664947485 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4db00b8750 con 0x7f4db00d04a0 2024-04-25T08:58:47.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.310+0000 7f4da6ffd640 1 -- 172.21.15.43:0/2664947485 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4da801a070 con 0x7f4db00d04a0 2024-04-25T08:58:47.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.310+0000 7f4da6ffd640 1 --2- 172.21.15.43:0/2664947485 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4d9006b4d0 0x7f4d9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:47.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.310+0000 7f4da6ffd640 1 -- 172.21.15.43:0/2664947485 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4da8089470 con 0x7f4db00d04a0 2024-04-25T08:58:47.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.311+0000 7f4dadd9b640 1 --2- 172.21.15.43:0/2664947485 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4d9006b4d0 0x7f4d9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:47.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.311+0000 7f4dadd9b640 1 --2- 172.21.15.43:0/2664947485 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4d9006b4d0 0x7f4d9006d990 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4db00b9700 tx=0x7f4d98009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:47.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.315+0000 7f4da6ffd640 1 -- 172.21.15.43:0/2664947485 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4db00b8750 con 0x7f4db00d04a0 2024-04-25T08:58:47.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:47.534+0000 7f4db4e74640 1 -- 172.21.15.43:0/2664947485 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4db0074f60 con 0x7f4db00d04a0 2024-04-25T08:58:48.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.401+0000 7f4da6ffd640 1 -- 172.21.15.43:0/2664947485 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v46) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4da805e080 con 0x7f4db00d04a0 2024-04-25T08:58:48.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.404+0000 7f4db4e74640 1 -- 172.21.15.43:0/2664947485 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4d9006b4d0 msgr2=0x7f4d9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:48.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.404+0000 7f4db4e74640 1 --2- 172.21.15.43:0/2664947485 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4d9006b4d0 0x7f4d9006d990 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f4db00b9700 tx=0x7f4d98009040 comp rx=0 tx=0).stop 2024-04-25T08:58:48.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.404+0000 7f4db4e74640 1 -- 172.21.15.43:0/2664947485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4db00d04a0 msgr2=0x7f4db00cae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:48.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.404+0000 7f4db4e74640 1 --2- 172.21.15.43:0/2664947485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4db00d04a0 0x7f4db00cae10 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f4da800c430 tx=0x7f4da800c8f0 comp rx=0 tx=0).stop 2024-04-25T08:58:48.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.404+0000 7f4db4e74640 1 -- 172.21.15.43:0/2664947485 shutdown_connections 2024-04-25T08:58:48.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.404+0000 7f4db4e74640 1 --2- 172.21.15.43:0/2664947485 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4d9006b4d0 0x7f4d9006d990 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:48.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.404+0000 7f4db4e74640 1 --2- 172.21.15.43:0/2664947485 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4db015dec0 0x7f4db00ca4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:48.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.404+0000 7f4db4e74640 1 --2- 172.21.15.43:0/2664947485 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4db00d27b0 0x7f4db00c9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:48.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.404+0000 7f4db4e74640 1 --2- 172.21.15.43:0/2664947485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4db00d04a0 0x7f4db00cae10 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:48.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.404+0000 7f4db4e74640 1 -- 172.21.15.43:0/2664947485 >> 172.21.15.43:0/2664947485 conn(0x7f4db009cd40 msgr2=0x7f4db0155780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:48.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.404+0000 7f4db4e74640 1 -- 172.21.15.43:0/2664947485 shutdown_connections 2024-04-25T08:58:48.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.404+0000 7f4db4e74640 1 -- 172.21.15.43:0/2664947485 wait complete. 2024-04-25T08:58:48.407 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T08:58:48.407 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T08:58:48.516 INFO:tasks.ceph.mds.d.smithi193.stderr:2024-04-25T08:58:48.527+0000 7f904305a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T08:58:48.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.529+0000 7f9e11c83640 1 Processor -- start 2024-04-25T08:58:48.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.529+0000 7f9e11c83640 1 -- start start 2024-04-25T08:58:48.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.530+0000 7f9e11c83640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9e0c15e090 0x7f9e0c161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:48.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.530+0000 7f9e11c83640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9e0c161fe0 0x7f9e0c0b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:48.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.530+0000 7f9e11c83640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e0c162610 con 0x7f9e0c0b5aa0 2024-04-25T08:58:48.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.530+0000 7f9e11c83640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e0c0d1da0 con 0x7f9e0c15e090 2024-04-25T08:58:48.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.530+0000 7f9e11c83640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e0c0d1750 con 0x7f9e0c161fe0 2024-04-25T08:58:48.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.530+0000 7f9e0b7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9e0c15e090 0x7f9e0c161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:48.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.530+0000 7f9e0b7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9e0c15e090 0x7f9e0c161aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50520/0 (socket says 172.21.15.43:50520) 2024-04-25T08:58:48.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.530+0000 7f9e0b7fe640 1 -- 172.21.15.43:0/1621422152 learned_addr learned my addr 172.21.15.43:0/1621422152 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:48.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.530+0000 7f9e0b7fe640 1 -- 172.21.15.43:0/1621422152 >> v2:172.21.15.193:3300/0 conn(0x7f9e0c161fe0 msgr2=0x7f9e0c0b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:48.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.530+0000 7f9e0b7fe640 1 --2- 172.21.15.43:0/1621422152 >> v2:172.21.15.193:3300/0 conn(0x7f9e0c161fe0 0x7f9e0c0b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:48.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.530+0000 7f9e0b7fe640 1 -- 172.21.15.43:0/1621422152 >> v1:172.21.15.193:6789/0 conn(0x7f9e0c0b5aa0 legacy=0x7f9e0c0ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:48.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.530+0000 7f9e0b7fe640 1 -- 172.21.15.43:0/1621422152 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e0c0d1a30 con 0x7f9e0c15e090 2024-04-25T08:58:48.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.531+0000 7f9e0b7fe640 1 --2- 172.21.15.43:0/1621422152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e0c15e090 0x7f9e0c161aa0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c0cfe10 tx=0x7f9dfc02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d011f003530f985 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:48.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.531+0000 7f9e0a7fc640 1 -- 172.21.15.43:0/1621422152 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc037070 con 0x7f9e0c15e090 2024-04-25T08:58:48.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.531+0000 7f9e0a7fc640 1 -- 172.21.15.43:0/1621422152 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dfc032440 con 0x7f9e0c15e090 2024-04-25T08:58:48.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.531+0000 7f9e0a7fc640 1 -- 172.21.15.43:0/1621422152 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc032810 con 0x7f9e0c15e090 2024-04-25T08:58:48.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.532+0000 7f9e11c83640 1 -- 172.21.15.43:0/1621422152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e0c15e090 msgr2=0x7f9e0c161aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:48.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.532+0000 7f9e11c83640 1 --2- 172.21.15.43:0/1621422152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e0c15e090 0x7f9e0c161aa0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c0cfe10 tx=0x7f9dfc02a2c0 comp rx=0 tx=0).stop 2024-04-25T08:58:48.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.532+0000 7f9e11c83640 1 -- 172.21.15.43:0/1621422152 shutdown_connections 2024-04-25T08:58:48.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.532+0000 7f9e11c83640 1 --2- 172.21.15.43:0/1621422152 >> v2:172.21.15.193:3300/0 conn(0x7f9e0c161fe0 0x7f9e0c0b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:48.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.532+0000 7f9e11c83640 1 --2- 172.21.15.43:0/1621422152 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e0c15e090 0x7f9e0c161aa0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:48.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.532+0000 7f9e11c83640 1 -- 172.21.15.43:0/1621422152 >> 172.21.15.43:0/1621422152 conn(0x7f9e0c0d0560 msgr2=0x7f9e0c0d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:48.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.532+0000 7f9e11c83640 1 -- 172.21.15.43:0/1621422152 shutdown_connections 2024-04-25T08:58:48.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.532+0000 7f9e11c83640 1 -- 172.21.15.43:0/1621422152 wait complete. 2024-04-25T08:58:48.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.533+0000 7f9e11c83640 1 Processor -- start 2024-04-25T08:58:48.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.534+0000 7f9e11c83640 1 -- start start 2024-04-25T08:58:48.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.534+0000 7f9e11c83640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e0c0b5aa0 0x7f9e0c148850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:48.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.534+0000 7f9e11c83640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e0c15e090 0x7f9e0c0c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:48.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.534+0000 7f9e11c83640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9e0c161fe0 0x7f9e0c0c9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:48.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.534+0000 7f9e11c83640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0c0d3240 con 0x7f9e0c15e090 2024-04-25T08:58:48.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.534+0000 7f9e11c83640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0c0b3280 con 0x7f9e0c0b5aa0 2024-04-25T08:58:48.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.534+0000 7f9e11c83640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0c0d33c0 con 0x7f9e0c161fe0 2024-04-25T08:58:48.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.535+0000 7f9e0affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e0c15e090 0x7f9e0c0c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:48.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.535+0000 7f9e0affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e0c15e090 0x7f9e0c0c8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50522/0 (socket says 172.21.15.43:50522) 2024-04-25T08:58:48.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.535+0000 7f9e0affd640 1 -- 172.21.15.43:0/519786194 learned_addr learned my addr 172.21.15.43:0/519786194 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:48.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.535+0000 7f9e0b7fe640 1 --2- 172.21.15.43:0/519786194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e0c0b5aa0 0x7f9e0c148850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:48.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.535+0000 7f9e0bfff640 1 --2- 172.21.15.43:0/519786194 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9e0c161fe0 0x7f9e0c0c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:48.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.535+0000 7f9e0affd640 1 -- 172.21.15.43:0/519786194 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9e0c161fe0 msgr2=0x7f9e0c0c9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:48.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.535+0000 7f9e0affd640 1 --2- 172.21.15.43:0/519786194 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9e0c161fe0 0x7f9e0c0c9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:48.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.535+0000 7f9e0affd640 1 -- 172.21.15.43:0/519786194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e0c0b5aa0 msgr2=0x7f9e0c148850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:48.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.535+0000 7f9e0affd640 1 --2- 172.21.15.43:0/519786194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e0c0b5aa0 0x7f9e0c148850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:48.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.535+0000 7f9e0affd640 1 -- 172.21.15.43:0/519786194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e0c148d90 con 0x7f9e0c15e090 2024-04-25T08:58:48.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.535+0000 7f9e0affd640 1 --2- 172.21.15.43:0/519786194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e0c15e090 0x7f9e0c0c8f50 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f9e0000b410 tx=0x7f9e0000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:48.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.535+0000 7f9df3fff640 1 -- 172.21.15.43:0/519786194 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e00015020 con 0x7f9e0c15e090 2024-04-25T08:58:48.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.535+0000 7f9df3fff640 1 -- 172.21.15.43:0/519786194 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e00011850 con 0x7f9e0c15e090 2024-04-25T08:58:48.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.535+0000 7f9e11c83640 1 -- 172.21.15.43:0/519786194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e0c0c9aa0 con 0x7f9e0c15e090 2024-04-25T08:58:48.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.535+0000 7f9df3fff640 1 -- 172.21.15.43:0/519786194 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e00011c40 con 0x7f9e0c15e090 2024-04-25T08:58:48.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.535+0000 7f9e11c83640 1 -- 172.21.15.43:0/519786194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e0c149260 con 0x7f9e0c15e090 2024-04-25T08:58:48.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.536+0000 7f9e11c83640 1 -- 172.21.15.43:0/519786194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e0c06bf00 con 0x7f9e0c15e090 2024-04-25T08:58:48.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.537+0000 7f9df3fff640 1 -- 172.21.15.43:0/519786194 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9e00018020 con 0x7f9e0c15e090 2024-04-25T08:58:48.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.537+0000 7f9df3fff640 1 --2- 172.21.15.43:0/519786194 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9df806b4d0 0x7f9df806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:48.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.537+0000 7f9df3fff640 1 -- 172.21.15.43:0/519786194 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e0008a010 con 0x7f9e0c15e090 2024-04-25T08:58:48.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.537+0000 7f9e0b7fe640 1 --2- 172.21.15.43:0/519786194 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9df806b4d0 0x7f9df806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:48.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.538+0000 7f9e0b7fe640 1 --2- 172.21.15.43:0/519786194 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9df806b4d0 0x7f9df806d990 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc034000 tx=0x7f9dfc02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:48.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.541+0000 7f9df3fff640 1 -- 172.21.15.43:0/519786194 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9e0c06bf00 con 0x7f9e0c15e090 2024-04-25T08:58:48.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:48.766+0000 7f9e11c83640 1 -- 172.21.15.43:0/519786194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9e0c074f60 con 0x7f9e0c15e090 2024-04-25T08:58:49.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.558+0000 7f9df3fff640 1 -- 172.21.15.43:0/519786194 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v47) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9e0001f020 con 0x7f9e0c15e090 2024-04-25T08:58:49.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.560+0000 7f9e11c83640 1 -- 172.21.15.43:0/519786194 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9df806b4d0 msgr2=0x7f9df806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:49.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.560+0000 7f9e11c83640 1 --2- 172.21.15.43:0/519786194 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9df806b4d0 0x7f9df806d990 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc034000 tx=0x7f9dfc02e000 comp rx=0 tx=0).stop 2024-04-25T08:58:49.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.561+0000 7f9e11c83640 1 -- 172.21.15.43:0/519786194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e0c15e090 msgr2=0x7f9e0c0c8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:49.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.561+0000 7f9e11c83640 1 --2- 172.21.15.43:0/519786194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e0c15e090 0x7f9e0c0c8f50 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f9e0000b410 tx=0x7f9e0000b8d0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.561+0000 7f9e11c83640 1 -- 172.21.15.43:0/519786194 shutdown_connections 2024-04-25T08:58:49.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.561+0000 7f9e11c83640 1 --2- 172.21.15.43:0/519786194 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9e0c161fe0 0x7f9e0c0c9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.561+0000 7f9e11c83640 1 --2- 172.21.15.43:0/519786194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e0c15e090 0x7f9e0c0c8f50 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.561+0000 7f9e11c83640 1 --2- 172.21.15.43:0/519786194 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9df806b4d0 0x7f9df806d990 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.561+0000 7f9e11c83640 1 --2- 172.21.15.43:0/519786194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e0c0b5aa0 0x7f9e0c148850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.561+0000 7f9e11c83640 1 -- 172.21.15.43:0/519786194 >> 172.21.15.43:0/519786194 conn(0x7f9e0c0d0560 msgr2=0x7f9e0c0d2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:49.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.561+0000 7f9e11c83640 1 -- 172.21.15.43:0/519786194 shutdown_connections 2024-04-25T08:58:49.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.561+0000 7f9e11c83640 1 -- 172.21.15.43:0/519786194 wait complete. 2024-04-25T08:58:49.563 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T08:58:49.564 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:58:49.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.682+0000 7f988759e640 1 Processor -- start 2024-04-25T08:58:49.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.682+0000 7f988759e640 1 -- start start 2024-04-25T08:58:49.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.683+0000 7f988759e640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f98800cbfc0 0x7f988006dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:49.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.683+0000 7f988759e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f98800ccbb0 0x7f98800cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:49.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.683+0000 7f988759e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f988015f5e0 0x7f988015fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:49.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.683+0000 7f988759e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98800cd550 con 0x7f988015f5e0 2024-04-25T08:58:49.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.683+0000 7f988759e640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98800b2780 con 0x7f98800cbfc0 2024-04-25T08:58:49.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.683+0000 7f988759e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98800b2a60 con 0x7f98800ccbb0 2024-04-25T08:58:49.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.683+0000 7f9885b3b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f988015f5e0 0x7f988015fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:49.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.683+0000 7f9885b3b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f988015f5e0 0x7f988015fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49034/0 (socket says 172.21.15.43:49034) 2024-04-25T08:58:49.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.683+0000 7f9885b3b640 1 -- 172.21.15.43:0/1968251657 learned_addr learned my addr 172.21.15.43:0/1968251657 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:49.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.683+0000 7f988533a640 1 --2- 172.21.15.43:0/1968251657 >> v2:172.21.15.193:3300/0 conn(0x7f98800cbfc0 0x7f988006dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:49.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.683+0000 7f9884b39640 1 --2- 172.21.15.43:0/1968251657 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f98800ccbb0 0x7f98800cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:49.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.683+0000 7f9885b3b640 1 -- 172.21.15.43:0/1968251657 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f98800ccbb0 msgr2=0x7f98800cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:49.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.683+0000 7f9885b3b640 1 --2- 172.21.15.43:0/1968251657 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f98800ccbb0 0x7f98800cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.683+0000 7f9885b3b640 1 -- 172.21.15.43:0/1968251657 >> v2:172.21.15.193:3300/0 conn(0x7f98800cbfc0 msgr2=0x7f988006dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:49.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.683+0000 7f9885b3b640 1 --2- 172.21.15.43:0/1968251657 >> v2:172.21.15.193:3300/0 conn(0x7f98800cbfc0 0x7f988006dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.683+0000 7f9885b3b640 1 -- 172.21.15.43:0/1968251657 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98800b2d40 con 0x7f988015f5e0 2024-04-25T08:58:49.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.683+0000 7f988533a640 1 --2- 172.21.15.43:0/1968251657 >> v2:172.21.15.193:3300/0 conn(0x7f98800cbfc0 0x7f988006dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:58:49.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.684+0000 7f9885b3b640 1 --2- 172.21.15.43:0/1968251657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f988015f5e0 0x7f988015fa70 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f98800cf630 tx=0x7f987002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cf4d5c08828a0ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:49.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.684+0000 7f987ffff640 1 -- 172.21.15.43:0/1968251657 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9870036070 con 0x7f988015f5e0 2024-04-25T08:58:49.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.684+0000 7f987ffff640 1 -- 172.21.15.43:0/1968251657 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9870031600 con 0x7f988015f5e0 2024-04-25T08:58:49.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.684+0000 7f987ffff640 1 -- 172.21.15.43:0/1968251657 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98700319d0 con 0x7f988015f5e0 2024-04-25T08:58:49.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.685+0000 7f988759e640 1 -- 172.21.15.43:0/1968251657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f988015f5e0 msgr2=0x7f988015fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:49.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.685+0000 7f988759e640 1 --2- 172.21.15.43:0/1968251657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f988015f5e0 0x7f988015fa70 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f98800cf630 tx=0x7f987002a6a0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.685+0000 7f988759e640 1 -- 172.21.15.43:0/1968251657 shutdown_connections 2024-04-25T08:58:49.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.685+0000 7f988759e640 1 --2- 172.21.15.43:0/1968251657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f988015f5e0 0x7f988015fa70 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.685+0000 7f988759e640 1 --2- 172.21.15.43:0/1968251657 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f98800ccbb0 0x7f98800cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.685+0000 7f988759e640 1 --2- 172.21.15.43:0/1968251657 >> v2:172.21.15.193:3300/0 conn(0x7f98800cbfc0 0x7f988006dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.685+0000 7f988759e640 1 -- 172.21.15.43:0/1968251657 >> 172.21.15.43:0/1968251657 conn(0x7f98800b1480 msgr2=0x7f98800b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:49.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.685+0000 7f988759e640 1 -- 172.21.15.43:0/1968251657 shutdown_connections 2024-04-25T08:58:49.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.685+0000 7f988759e640 1 -- 172.21.15.43:0/1968251657 wait complete. 2024-04-25T08:58:49.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.686+0000 7f988759e640 1 Processor -- start 2024-04-25T08:58:49.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.686+0000 7f988759e640 1 -- start start 2024-04-25T08:58:49.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.687+0000 7f988759e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98800cbfc0 0x7f988013e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:49.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.687+0000 7f988759e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98800ccbb0 0x7f98800a9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:49.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.687+0000 7f988759e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f988015f5e0 0x7f98801f1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:49.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.687+0000 7f988759e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98800b35e0 con 0x7f98800ccbb0 2024-04-25T08:58:49.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.687+0000 7f988759e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98800b3300 con 0x7f98800cbfc0 2024-04-25T08:58:49.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.687+0000 7f988759e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988006e1a0 con 0x7f988015f5e0 2024-04-25T08:58:49.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.687+0000 7f9884b39640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98800ccbb0 0x7f98800a9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:49.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.687+0000 7f9884b39640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98800ccbb0 0x7f98800a9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49040/0 (socket says 172.21.15.43:49040) 2024-04-25T08:58:49.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.687+0000 7f9884b39640 1 -- 172.21.15.43:0/4015619441 learned_addr learned my addr 172.21.15.43:0/4015619441 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:49.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.687+0000 7f9885b3b640 1 --2- 172.21.15.43:0/4015619441 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f988015f5e0 0x7f98801f1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:49.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.687+0000 7f988533a640 1 --2- 172.21.15.43:0/4015619441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98800cbfc0 0x7f988013e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:49.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.687+0000 7f9884b39640 1 -- 172.21.15.43:0/4015619441 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f988015f5e0 msgr2=0x7f98801f1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:49.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.687+0000 7f9884b39640 1 --2- 172.21.15.43:0/4015619441 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f988015f5e0 0x7f98801f1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.687+0000 7f9884b39640 1 -- 172.21.15.43:0/4015619441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98800cbfc0 msgr2=0x7f988013e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:49.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.687+0000 7f9884b39640 1 --2- 172.21.15.43:0/4015619441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98800cbfc0 0x7f988013e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.687+0000 7f9884b39640 1 -- 172.21.15.43:0/4015619441 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98801f21b0 con 0x7f98800ccbb0 2024-04-25T08:58:49.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.688+0000 7f9884b39640 1 --2- 172.21.15.43:0/4015619441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98800ccbb0 0x7f98800a9fe0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f987400d800 tx=0x7f987400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:49.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.688+0000 7f987dffb640 1 -- 172.21.15.43:0/4015619441 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987400b6c0 con 0x7f98800ccbb0 2024-04-25T08:58:49.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.688+0000 7f987dffb640 1 -- 172.21.15.43:0/4015619441 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9874019070 con 0x7f98800ccbb0 2024-04-25T08:58:49.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.688+0000 7f988759e640 1 -- 172.21.15.43:0/4015619441 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98801f2490 con 0x7f98800ccbb0 2024-04-25T08:58:49.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.688+0000 7f987dffb640 1 -- 172.21.15.43:0/4015619441 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987400ba80 con 0x7f98800ccbb0 2024-04-25T08:58:49.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.688+0000 7f988759e640 1 -- 172.21.15.43:0/4015619441 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f988013e6b0 con 0x7f98800ccbb0 2024-04-25T08:58:49.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.689+0000 7f988759e640 1 -- 172.21.15.43:0/4015619441 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98800cde10 con 0x7f98800ccbb0 2024-04-25T08:58:49.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.690+0000 7f987dffb640 1 -- 172.21.15.43:0/4015619441 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f987400bc20 con 0x7f98800ccbb0 2024-04-25T08:58:49.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.690+0000 7f987dffb640 1 --2- 172.21.15.43:0/4015619441 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f986c06b4d0 0x7f986c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:49.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.690+0000 7f987dffb640 1 -- 172.21.15.43:0/4015619441 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f987408a4d0 con 0x7f98800ccbb0 2024-04-25T08:58:49.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.690+0000 7f988533a640 1 --2- 172.21.15.43:0/4015619441 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f986c06b4d0 0x7f986c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:49.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.691+0000 7f988533a640 1 --2- 172.21.15.43:0/4015619441 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f986c06b4d0 0x7f986c06d990 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9878003610 tx=0x7f9878008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:49.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.694+0000 7f987dffb640 1 -- 172.21.15.43:0/4015619441 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f98800cde10 con 0x7f98800ccbb0 2024-04-25T08:58:49.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.916+0000 7f988759e640 1 -- 172.21.15.43:0/4015619441 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9880074ef0 con 0x7f98800ccbb0 2024-04-25T08:58:49.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.916+0000 7f987dffb640 1 -- 172.21.15.43:0/4015619441 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f9874088e20 con 0x7f98800ccbb0 2024-04-25T08:58:49.906 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:58:49.906 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6373,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3648859409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":3648859409},{"type":"v1","addr":"172.21.15.193:6837","nonce":3648859409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":7149,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/356516962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":356516962},{"type":"v1","addr":"172.21.15.43:6834","nonce":356516962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":7152,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3032637719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":3032637719},{"type":"v1","addr":"172.21.15.43:6835","nonce":3032637719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:58:45.906679+0000","modified":"2024-04-25T08:58:48.767705+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7164},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7164":{"gid":7164,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.193:6836/3458098162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":3458098162},{"type":"v1","addr":"172.21.15.193:6836","nonce":3458098162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T08:58:49.906 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 47 2024-04-25T08:58:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.918+0000 7f988759e640 1 -- 172.21.15.43:0/4015619441 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f986c06b4d0 msgr2=0x7f986c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.918+0000 7f988759e640 1 --2- 172.21.15.43:0/4015619441 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f986c06b4d0 0x7f986c06d990 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9878003610 tx=0x7f9878008040 comp rx=0 tx=0).stop 2024-04-25T08:58:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.918+0000 7f988759e640 1 -- 172.21.15.43:0/4015619441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98800ccbb0 msgr2=0x7f98800a9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.918+0000 7f988759e640 1 --2- 172.21.15.43:0/4015619441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98800ccbb0 0x7f98800a9fe0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f987400d800 tx=0x7f987400dcc0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.918+0000 7f988759e640 1 -- 172.21.15.43:0/4015619441 shutdown_connections 2024-04-25T08:58:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.918+0000 7f988759e640 1 --2- 172.21.15.43:0/4015619441 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f988015f5e0 0x7f98801f1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.918+0000 7f988759e640 1 --2- 172.21.15.43:0/4015619441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98800ccbb0 0x7f98800a9fe0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.918+0000 7f988759e640 1 --2- 172.21.15.43:0/4015619441 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f986c06b4d0 0x7f986c06d990 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.918+0000 7f988759e640 1 --2- 172.21.15.43:0/4015619441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f98800cbfc0 0x7f988013e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:49.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.918+0000 7f988759e640 1 -- 172.21.15.43:0/4015619441 >> 172.21.15.43:0/4015619441 conn(0x7f98800b1480 msgr2=0x7f988015f9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:49.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.918+0000 7f988759e640 1 -- 172.21.15.43:0/4015619441 shutdown_connections 2024-04-25T08:58:49.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:49.918+0000 7f988759e640 1 -- 172.21.15.43:0/4015619441 wait complete. 2024-04-25T08:58:49.919 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T08:58:50.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.030+0000 7f11819b7640 1 Processor -- start 2024-04-25T08:58:50.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.030+0000 7f11819b7640 1 -- start start 2024-04-25T08:58:50.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.031+0000 7f11819b7640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f117c0d09b0 0x7f117c0b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:50.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.031+0000 7f11819b7640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f117c0b4c30 con 0x7f117c161390 2024-04-25T08:58:50.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.031+0000 7f11819b7640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f117c0cf2c0 con 0x7f117c0ce5e0 2024-04-25T08:58:50.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.031+0000 7f11819b7640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f117c0d1910 con 0x7f117c0d09b0 2024-04-25T08:58:50.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.031+0000 7f117affd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f117c161390 0x7f117c0d02a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45284/0 (socket says 172.21.15.43:45284) 2024-04-25T08:58:50.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.031+0000 7f117affd640 1 -- 172.21.15.43:0/3164660986 learned_addr learned my addr 172.21.15.43:0/3164660986 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:50.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.031+0000 7f1179ffb640 1 -- 172.21.15.43:0/3164660986 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1787594861 0 0) 0x7f1174001110 con 0x7f117c161390 2024-04-25T08:58:50.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.031+0000 7f1179ffb640 1 -- 172.21.15.43:0/3164660986 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1168003520 con 0x7f117c161390 2024-04-25T08:58:50.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.031+0000 7f117a7fc640 1 --2- 172.21.15.43:0/3164660986 >> v2:172.21.15.193:3300/0 conn(0x7f117c0d09b0 0x7f117c0b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:50.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.032+0000 7f1179ffb640 1 -- 172.21.15.43:0/3164660986 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3133057036 0 0) 0x7f1168003520 con 0x7f117c161390 2024-04-25T08:58:50.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.032+0000 7f1179ffb640 1 -- 172.21.15.43:0/3164660986 >> v2:172.21.15.193:3300/0 conn(0x7f117c0d09b0 msgr2=0x7f117c0b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:50.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.032+0000 7f1179ffb640 1 --2- 172.21.15.43:0/3164660986 >> v2:172.21.15.193:3300/0 conn(0x7f117c0d09b0 0x7f117c0b6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.032+0000 7f1179ffb640 1 -- 172.21.15.43:0/3164660986 >> v1:172.21.15.193:6789/0 conn(0x7f117c0ce5e0 legacy=0x7f117c0cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:50.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.032+0000 7f1179ffb640 1 -- 172.21.15.43:0/3164660986 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117c0cf5a0 con 0x7f117c161390 2024-04-25T08:58:50.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.032+0000 7f1179ffb640 1 -- 172.21.15.43:0/3164660986 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f1174001840 con 0x7f117c161390 2024-04-25T08:58:50.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.032+0000 7f1179ffb640 1 -- 172.21.15.43:0/3164660986 >> v1:172.21.15.43:6789/0 conn(0x7f117c161390 legacy=0x7f117c0d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:50.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.032+0000 7f1179ffb640 1 --2- 172.21.15.43:0/3164660986 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f11680039b0 0x7f1168003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:50.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.032+0000 7f1179ffb640 1 --2- 172.21.15.43:0/3164660986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1168004800 0x7f1168004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:50.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.032+0000 7f1179ffb640 1 --2- 172.21.15.43:0/3164660986 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11680051c0 0x7f116800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:50.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.032+0000 7f1179ffb640 1 -- 172.21.15.43:0/3164660986 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f117c0d1910 con 0x7f1168004800 2024-04-25T08:58:50.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.032+0000 7f1179ffb640 1 -- 172.21.15.43:0/3164660986 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116800c060 con 0x7f11680051c0 2024-04-25T08:58:50.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.032+0000 7f1179ffb640 1 -- 172.21.15.43:0/3164660986 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f116800c260 con 0x7f11680039b0 2024-04-25T08:58:50.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.032+0000 7f117a7fc640 1 --2- 172.21.15.43:0/3164660986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1168004800 0x7f1168004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:50.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.032+0000 7f117affd640 1 --2- 172.21.15.43:0/3164660986 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f11680039b0 0x7f1168003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:50.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.032+0000 7f117b7fe640 1 --2- 172.21.15.43:0/3164660986 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11680051c0 0x7f116800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:50.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.032+0000 7f117a7fc640 1 -- 172.21.15.43:0/3164660986 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f11680039b0 msgr2=0x7f1168003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:50.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.032+0000 7f117a7fc640 1 --2- 172.21.15.43:0/3164660986 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f11680039b0 0x7f1168003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.032+0000 7f117a7fc640 1 -- 172.21.15.43:0/3164660986 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11680051c0 msgr2=0x7f116800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:50.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.032+0000 7f117a7fc640 1 --2- 172.21.15.43:0/3164660986 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11680051c0 0x7f116800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.032+0000 7f117a7fc640 1 -- 172.21.15.43:0/3164660986 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1168003520 con 0x7f1168004800 2024-04-25T08:58:50.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.032+0000 7f117b7fe640 1 --2- 172.21.15.43:0/3164660986 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f11680051c0 0x7f116800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:58:50.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.032+0000 7f117b7fe640 1 -- 172.21.15.43:0/3164660986 reap_dead start 2024-04-25T08:58:50.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.033+0000 7f117a7fc640 1 --2- 172.21.15.43:0/3164660986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1168004800 0x7f1168004c80 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f1164009ac0 tx=0x7f1168005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8ebebd04de0f8b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:50.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.033+0000 7f1179ffb640 1 -- 172.21.15.43:0/3164660986 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1164036070 con 0x7f1168004800 2024-04-25T08:58:50.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.033+0000 7f1179ffb640 1 -- 172.21.15.43:0/3164660986 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1164031400 con 0x7f1168004800 2024-04-25T08:58:50.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.033+0000 7f1179ffb640 1 -- 172.21.15.43:0/3164660986 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1164009690 con 0x7f1168004800 2024-04-25T08:58:50.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.034+0000 7f11819b7640 1 -- 172.21.15.43:0/3164660986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1168004800 msgr2=0x7f1168004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:50.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.034+0000 7f11819b7640 1 --2- 172.21.15.43:0/3164660986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1168004800 0x7f1168004c80 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f1164009ac0 tx=0x7f1168005830 comp rx=0 tx=0).stop 2024-04-25T08:58:50.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.034+0000 7f11819b7640 1 -- 172.21.15.43:0/3164660986 shutdown_connections 2024-04-25T08:58:50.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.034+0000 7f11819b7640 1 --2- 172.21.15.43:0/3164660986 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1168004800 0x7f1168004c80 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.034+0000 7f11819b7640 1 -- 172.21.15.43:0/3164660986 >> 172.21.15.43:0/3164660986 conn(0x7f117c09cd20 msgr2=0x7f117c0d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:50.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.034+0000 7f11819b7640 1 -- 172.21.15.43:0/3164660986 shutdown_connections 2024-04-25T08:58:50.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.034+0000 7f11819b7640 1 -- 172.21.15.43:0/3164660986 wait complete. 2024-04-25T08:58:50.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.035+0000 7f11819b7640 1 Processor -- start 2024-04-25T08:58:50.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.035+0000 7f11819b7640 1 -- start start 2024-04-25T08:58:50.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.036+0000 7f11819b7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1168004800 0x7f117c14aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:50.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.036+0000 7f11819b7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f117c15f340 0x7f117c15f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:50.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.036+0000 7f11819b7640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f117c15c040 0x7f117c15c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:50.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.036+0000 7f11819b7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f117c161770 con 0x7f1168004800 2024-04-25T08:58:50.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.036+0000 7f11819b7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f117c0d15a0 con 0x7f117c15f340 2024-04-25T08:58:50.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.036+0000 7f11819b7640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f117c0b6820 con 0x7f117c15c040 2024-04-25T08:58:50.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.036+0000 7f117affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1168004800 0x7f117c14aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:50.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.036+0000 7f117affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1168004800 0x7f117c14aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49060/0 (socket says 172.21.15.43:49060) 2024-04-25T08:58:50.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.036+0000 7f117affd640 1 -- 172.21.15.43:0/3549625109 learned_addr learned my addr 172.21.15.43:0/3549625109 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:50.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.036+0000 7f117affd640 1 -- 172.21.15.43:0/3549625109 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f117c15c040 msgr2=0x7f117c15c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:50.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.036+0000 7f117affd640 1 --2- 172.21.15.43:0/3549625109 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f117c15c040 0x7f117c15c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.036+0000 7f117affd640 1 -- 172.21.15.43:0/3549625109 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f117c15f340 msgr2=0x7f117c15f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:50.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.036+0000 7f117affd640 1 --2- 172.21.15.43:0/3549625109 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f117c15f340 0x7f117c15f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.036+0000 7f117affd640 1 -- 172.21.15.43:0/3549625109 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117c15ace0 con 0x7f1168004800 2024-04-25T08:58:50.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.036+0000 7f117affd640 1 --2- 172.21.15.43:0/3549625109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1168004800 0x7f117c14aec0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f11740072c0 tx=0x7f1174007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:50.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.037+0000 7f1173fff640 1 -- 172.21.15.43:0/3549625109 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117401a070 con 0x7f1168004800 2024-04-25T08:58:50.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.037+0000 7f1173fff640 1 -- 172.21.15.43:0/3549625109 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117400f980 con 0x7f1168004800 2024-04-25T08:58:50.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.037+0000 7f1173fff640 1 -- 172.21.15.43:0/3549625109 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117400fd70 con 0x7f1168004800 2024-04-25T08:58:50.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.037+0000 7f11819b7640 1 -- 172.21.15.43:0/3549625109 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f117c15af80 con 0x7f1168004800 2024-04-25T08:58:50.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.037+0000 7f11819b7640 1 -- 172.21.15.43:0/3549625109 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f117c15b540 con 0x7f1168004800 2024-04-25T08:58:50.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.038+0000 7f11819b7640 1 -- 172.21.15.43:0/3549625109 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f117c0680f0 con 0x7f1168004800 2024-04-25T08:58:50.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.038+0000 7f1173fff640 1 -- 172.21.15.43:0/3549625109 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f117401d070 con 0x7f1168004800 2024-04-25T08:58:50.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.039+0000 7f1173fff640 1 --2- 172.21.15.43:0/3549625109 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f115c06b4d0 0x7f115c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:50.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.039+0000 7f1173fff640 1 -- 172.21.15.43:0/3549625109 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1174089c40 con 0x7f1168004800 2024-04-25T08:58:50.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.039+0000 7f117a7fc640 1 --2- 172.21.15.43:0/3549625109 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f115c06b4d0 0x7f115c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:50.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.039+0000 7f117a7fc640 1 --2- 172.21.15.43:0/3549625109 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f115c06b4d0 0x7f115c06d990 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1164033000 tx=0x7f116402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:50.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.043+0000 7f1173fff640 1 -- 172.21.15.43:0/3549625109 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f117c0b4c30 con 0x7f1168004800 2024-04-25T08:58:50.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.223+0000 7f11819b7640 1 -- 172.21.15.43:0/3549625109 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f117c074ef0 con 0x7f1168004800 2024-04-25T08:58:50.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.224+0000 7f1173fff640 1 -- 172.21.15.43:0/3549625109 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) v1 ==== 74+0+16632 (secure 0 0 0) 0x7f1174058ab0 con 0x7f1168004800 2024-04-25T08:58:50.214 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:58:50.214 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":73,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T08:58:47.543316+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:58:44.188924+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T08:58:44.877113+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T08:58:50.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.226+0000 7f11819b7640 1 -- 172.21.15.43:0/3549625109 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f115c06b4d0 msgr2=0x7f115c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:50.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.226+0000 7f11819b7640 1 --2- 172.21.15.43:0/3549625109 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f115c06b4d0 0x7f115c06d990 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1164033000 tx=0x7f116402e000 comp rx=0 tx=0).stop 2024-04-25T08:58:50.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.226+0000 7f11819b7640 1 -- 172.21.15.43:0/3549625109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1168004800 msgr2=0x7f117c14aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:50.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.226+0000 7f11819b7640 1 --2- 172.21.15.43:0/3549625109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1168004800 0x7f117c14aec0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f11740072c0 tx=0x7f1174007780 comp rx=0 tx=0).stop 2024-04-25T08:58:50.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.226+0000 7f11819b7640 1 -- 172.21.15.43:0/3549625109 shutdown_connections 2024-04-25T08:58:50.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.226+0000 7f11819b7640 1 --2- 172.21.15.43:0/3549625109 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f117c15c040 0x7f117c15c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.226+0000 7f11819b7640 1 --2- 172.21.15.43:0/3549625109 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f115c06b4d0 0x7f115c06d990 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.226+0000 7f11819b7640 1 --2- 172.21.15.43:0/3549625109 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f117c15f340 0x7f117c15f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.226+0000 7f11819b7640 1 --2- 172.21.15.43:0/3549625109 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1168004800 0x7f117c14aec0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.226+0000 7f11819b7640 1 -- 172.21.15.43:0/3549625109 >> 172.21.15.43:0/3549625109 conn(0x7f117c09cd20 msgr2=0x7f117c0b6c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:50.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.226+0000 7f11819b7640 1 -- 172.21.15.43:0/3549625109 shutdown_connections 2024-04-25T08:58:50.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.226+0000 7f11819b7640 1 -- 172.21.15.43:0/3549625109 wait complete. 2024-04-25T08:58:50.227 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T08:58:50.227 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:58:50.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.338+0000 7f6cbaba5640 1 Processor -- start 2024-04-25T08:58:50.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.338+0000 7f6cbaba5640 1 -- start start 2024-04-25T08:58:50.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.339+0000 7f6cbaba5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6cb415ded0 0x7f6cb40ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:50.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.339+0000 7f6cbaba5640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6cb40cf250 0x7f6cb40ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:50.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.339+0000 7f6cbaba5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6cb406ab30 0x7f6cb406afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:50.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.339+0000 7f6cbaba5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cb415e2b0 con 0x7f6cb406ab30 2024-04-25T08:58:50.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.339+0000 7f6cbaba5640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cb40cf6e0 con 0x7f6cb40cf250 2024-04-25T08:58:50.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.339+0000 7f6cbaba5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb40cf9c0 con 0x7f6cb415ded0 2024-04-25T08:58:50.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.339+0000 7f6cb9142640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6cb406ab30 0x7f6cb406afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:50.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.339+0000 7f6cb9142640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6cb406ab30 0x7f6cb406afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49062/0 (socket says 172.21.15.43:49062) 2024-04-25T08:58:50.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.339+0000 7f6cb9142640 1 -- 172.21.15.43:0/2183468936 learned_addr learned my addr 172.21.15.43:0/2183468936 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:50.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.340+0000 7f6cb8941640 1 --2- 172.21.15.43:0/2183468936 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6cb415ded0 0x7f6cb40ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:50.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.340+0000 7f6cb9142640 1 -- 172.21.15.43:0/2183468936 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6cb415ded0 msgr2=0x7f6cb40ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:50.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.340+0000 7f6cb9142640 1 --2- 172.21.15.43:0/2183468936 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6cb415ded0 0x7f6cb40ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.340+0000 7f6cb3fff640 1 --2- 172.21.15.43:0/2183468936 >> v2:172.21.15.193:3300/0 conn(0x7f6cb40cf250 0x7f6cb40ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:50.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.340+0000 7f6cb9142640 1 -- 172.21.15.43:0/2183468936 >> v2:172.21.15.193:3300/0 conn(0x7f6cb40cf250 msgr2=0x7f6cb40ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:50.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.340+0000 7f6cb9142640 1 --2- 172.21.15.43:0/2183468936 >> v2:172.21.15.193:3300/0 conn(0x7f6cb40cf250 0x7f6cb40ad8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.340+0000 7f6cb9142640 1 -- 172.21.15.43:0/2183468936 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb40cfca0 con 0x7f6cb406ab30 2024-04-25T08:58:50.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.340+0000 7f6cb8941640 1 --2- 172.21.15.43:0/2183468936 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6cb415ded0 0x7f6cb40ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:50.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.340+0000 7f6cb9142640 1 --2- 172.21.15.43:0/2183468936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb406ab30 0x7f6cb406afc0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f6cb406bc10 tx=0x7f6cac02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58e10e835b7cde47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:50.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.340+0000 7f6cb37fe640 1 -- 172.21.15.43:0/2183468936 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cac036070 con 0x7f6cb406ab30 2024-04-25T08:58:50.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.341+0000 7f6cb37fe640 1 -- 172.21.15.43:0/2183468936 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cac031600 con 0x7f6cb406ab30 2024-04-25T08:58:50.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.341+0000 7f6cb37fe640 1 -- 172.21.15.43:0/2183468936 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cac0319d0 con 0x7f6cb406ab30 2024-04-25T08:58:50.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.341+0000 7f6cbaba5640 1 -- 172.21.15.43:0/2183468936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb406ab30 msgr2=0x7f6cb406afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:50.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.341+0000 7f6cbaba5640 1 --2- 172.21.15.43:0/2183468936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb406ab30 0x7f6cb406afc0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f6cb406bc10 tx=0x7f6cac02a6a0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.341+0000 7f6cbaba5640 1 -- 172.21.15.43:0/2183468936 shutdown_connections 2024-04-25T08:58:50.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.341+0000 7f6cbaba5640 1 --2- 172.21.15.43:0/2183468936 >> v2:172.21.15.193:3300/0 conn(0x7f6cb40cf250 0x7f6cb40ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.341+0000 7f6cbaba5640 1 --2- 172.21.15.43:0/2183468936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb406ab30 0x7f6cb406afc0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.341+0000 7f6cbaba5640 1 --2- 172.21.15.43:0/2183468936 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6cb415ded0 0x7f6cb40ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.341+0000 7f6cbaba5640 1 -- 172.21.15.43:0/2183468936 >> 172.21.15.43:0/2183468936 conn(0x7f6cb40b2490 msgr2=0x7f6cb40b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:50.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.341+0000 7f6cbaba5640 1 -- 172.21.15.43:0/2183468936 shutdown_connections 2024-04-25T08:58:50.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.341+0000 7f6cbaba5640 1 -- 172.21.15.43:0/2183468936 wait complete. 2024-04-25T08:58:50.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.342+0000 7f6cbaba5640 1 Processor -- start 2024-04-25T08:58:50.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.342+0000 7f6cbaba5640 1 -- start start 2024-04-25T08:58:50.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.343+0000 7f6cbaba5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cb406ab30 0x7f6cb4155940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:50.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.343+0000 7f6cbaba5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6cb415ded0 0x7f6cb4160880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:50.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.343+0000 7f6cbaba5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb406e510 0x7f6cb40b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:50.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.343+0000 7f6cbaba5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb40cefd0 con 0x7f6cb406e510 2024-04-25T08:58:50.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.343+0000 7f6cbaba5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb40cee50 con 0x7f6cb406ab30 2024-04-25T08:58:50.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.343+0000 7f6cbaba5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb40b4db0 con 0x7f6cb415ded0 2024-04-25T08:58:50.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.343+0000 7f6cb9142640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb406e510 0x7f6cb40b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:50.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.343+0000 7f6cb9142640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb406e510 0x7f6cb40b0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49066/0 (socket says 172.21.15.43:49066) 2024-04-25T08:58:50.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.343+0000 7f6cb9142640 1 -- 172.21.15.43:0/2910902185 learned_addr learned my addr 172.21.15.43:0/2910902185 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:50.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.344+0000 7f6cb8941640 1 --2- 172.21.15.43:0/2910902185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cb406ab30 0x7f6cb4155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:50.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.344+0000 7f6cb3fff640 1 --2- 172.21.15.43:0/2910902185 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6cb415ded0 0x7f6cb4160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:50.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.344+0000 7f6cb9142640 1 -- 172.21.15.43:0/2910902185 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6cb415ded0 msgr2=0x7f6cb4160880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:50.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.344+0000 7f6cb9142640 1 --2- 172.21.15.43:0/2910902185 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6cb415ded0 0x7f6cb4160880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.344+0000 7f6cb9142640 1 -- 172.21.15.43:0/2910902185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cb406ab30 msgr2=0x7f6cb4155940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:50.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.344+0000 7f6cb9142640 1 --2- 172.21.15.43:0/2910902185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cb406ab30 0x7f6cb4155940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.344+0000 7f6cb9142640 1 -- 172.21.15.43:0/2910902185 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb406ead0 con 0x7f6cb406e510 2024-04-25T08:58:50.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.344+0000 7f6cb9142640 1 --2- 172.21.15.43:0/2910902185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb406e510 0x7f6cb40b0bd0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f6cac009f40 tx=0x7f6cac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:50.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.344+0000 7f6cb17fa640 1 -- 172.21.15.43:0/2910902185 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cac041020 con 0x7f6cb406e510 2024-04-25T08:58:50.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.344+0000 7f6cb17fa640 1 -- 172.21.15.43:0/2910902185 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cac007780 con 0x7f6cb406e510 2024-04-25T08:58:50.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.344+0000 7f6cbaba5640 1 -- 172.21.15.43:0/2910902185 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cb406edb0 con 0x7f6cb406e510 2024-04-25T08:58:50.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.344+0000 7f6cb17fa640 1 -- 172.21.15.43:0/2910902185 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cac036040 con 0x7f6cb406e510 2024-04-25T08:58:50.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.344+0000 7f6cbaba5640 1 -- 172.21.15.43:0/2910902185 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cb406f370 con 0x7f6cb406e510 2024-04-25T08:58:50.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.345+0000 7f6cbaba5640 1 -- 172.21.15.43:0/2910902185 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cb4074ef0 con 0x7f6cb406e510 2024-04-25T08:58:50.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.345+0000 7f6cb17fa640 1 -- 172.21.15.43:0/2910902185 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6cac044020 con 0x7f6cb406e510 2024-04-25T08:58:50.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.346+0000 7f6cb17fa640 1 --2- 172.21.15.43:0/2910902185 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6c9806b4d0 0x7f6c9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:50.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.346+0000 7f6cb17fa640 1 -- 172.21.15.43:0/2910902185 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6cac0ab2d0 con 0x7f6cb406e510 2024-04-25T08:58:50.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.346+0000 7f6cb8941640 1 --2- 172.21.15.43:0/2910902185 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6c9806b4d0 0x7f6c9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:50.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.347+0000 7f6cb8941640 1 --2- 172.21.15.43:0/2910902185 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6c9806b4d0 0x7f6c9806d990 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6ca800b7c0 tx=0x7f6ca8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:50.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.350+0000 7f6cb17fa640 1 -- 172.21.15.43:0/2910902185 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6cac04b020 con 0x7f6cb406e510 2024-04-25T08:58:50.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.528+0000 7f6cbaba5640 1 -- 172.21.15.43:0/2910902185 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6cb40b31e0 con 0x7f6c9806b4d0 2024-04-25T08:58:50.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.535+0000 7f6cb17fa640 1 -- 172.21.15.43:0/2910902185 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252067 (secure 0 0 0) 0x7f6cb40b31e0 con 0x7f6c9806b4d0 2024-04-25T08:58:50.528 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:58:50.529 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-25T08:58:50.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.538+0000 7f6cbaba5640 1 -- 172.21.15.43:0/2910902185 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6c9806b4d0 msgr2=0x7f6c9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:50.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.538+0000 7f6cbaba5640 1 --2- 172.21.15.43:0/2910902185 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6c9806b4d0 0x7f6c9806d990 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6ca800b7c0 tx=0x7f6ca8008040 comp rx=0 tx=0).stop 2024-04-25T08:58:50.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.538+0000 7f6cbaba5640 1 -- 172.21.15.43:0/2910902185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb406e510 msgr2=0x7f6cb40b0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:50.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.538+0000 7f6cbaba5640 1 --2- 172.21.15.43:0/2910902185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb406e510 0x7f6cb40b0bd0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f6cac009f40 tx=0x7f6cac004a40 comp rx=0 tx=0).stop 2024-04-25T08:58:50.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.538+0000 7f6cbaba5640 1 -- 172.21.15.43:0/2910902185 shutdown_connections 2024-04-25T08:58:50.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.538+0000 7f6cbaba5640 1 --2- 172.21.15.43:0/2910902185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cb406e510 0x7f6cb40b0bd0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.538+0000 7f6cbaba5640 1 --2- 172.21.15.43:0/2910902185 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6cb415ded0 0x7f6cb4160880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.538+0000 7f6cbaba5640 1 --2- 172.21.15.43:0/2910902185 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6c9806b4d0 0x7f6c9806d990 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.538+0000 7f6cbaba5640 1 --2- 172.21.15.43:0/2910902185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cb406ab30 0x7f6cb4155940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.538+0000 7f6cbaba5640 1 -- 172.21.15.43:0/2910902185 >> 172.21.15.43:0/2910902185 conn(0x7f6cb40b2490 msgr2=0x7f6cb40b3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:50.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.538+0000 7f6cbaba5640 1 -- 172.21.15.43:0/2910902185 shutdown_connections 2024-04-25T08:58:50.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.538+0000 7f6cbaba5640 1 -- 172.21.15.43:0/2910902185 wait complete. 2024-04-25T08:58:50.539 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2024-04-25T08:58:49.760135+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":162,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":345036,"kb_used_data":8156,"kb_used_omap":12,"kb_used_meta":336755,"kb_avail":754629684,"statfs":{"total":773094113280,"available":772740796416,"internally_reserved":0,"allocated":8351744,"data_stored":4211304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":344837681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.232420"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.548312+0000","last_change":"2024-04-25T08:58:46.560577+0000","last_active":"2024-04-25T08:58:47.548312+0000","last_peered":"2024-04-25T08:58:47.548312+0000","last_clean":"2024-04-25T08:58:47.548312+0000","last_became_active":"2024-04-25T08:58:46.560305+0000","last_became_peered":"2024-04-25T08:58:46.560305+0000","last_unstale":"2024-04-25T08:58:47.548312+0000","last_undegraded":"2024-04-25T08:58:47.548312+0000","last_fullsized":"2024-04-25T08:58:47.548312+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:55.762916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.549167+0000","last_change":"2024-04-25T08:58:46.551585+0000","last_active":"2024-04-25T08:58:47.549167+0000","last_peered":"2024-04-25T08:58:47.549167+0000","last_clean":"2024-04-25T08:58:47.549167+0000","last_became_active":"2024-04-25T08:58:46.551290+0000","last_became_peered":"2024-04-25T08:58:46.551290+0000","last_unstale":"2024-04-25T08:58:47.549167+0000","last_undegraded":"2024-04-25T08:58:47.549167+0000","last_fullsized":"2024-04-25T08:58:47.549167+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:00.887733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921429+0000","last_change":"2024-04-25T08:58:46.555701+0000","last_active":"2024-04-25T08:58:47.921429+0000","last_peered":"2024-04-25T08:58:47.921429+0000","last_clean":"2024-04-25T08:58:47.921429+0000","last_became_active":"2024-04-25T08:58:46.555335+0000","last_became_peered":"2024-04-25T08:58:46.555335+0000","last_unstale":"2024-04-25T08:58:47.921429+0000","last_undegraded":"2024-04-25T08:58:47.921429+0000","last_fullsized":"2024-04-25T08:58:47.921429+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:21.823279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.546632+0000","last_change":"2024-04-25T08:58:46.562436+0000","last_active":"2024-04-25T08:58:47.546632+0000","last_peered":"2024-04-25T08:58:47.546632+0000","last_clean":"2024-04-25T08:58:47.546632+0000","last_became_active":"2024-04-25T08:58:46.561139+0000","last_became_peered":"2024-04-25T08:58:46.561139+0000","last_unstale":"2024-04-25T08:58:47.546632+0000","last_undegraded":"2024-04-25T08:58:47.546632+0000","last_fullsized":"2024-04-25T08:58:47.546632+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:46.584584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921286+0000","last_change":"2024-04-25T08:58:46.553792+0000","last_active":"2024-04-25T08:58:47.921286+0000","last_peered":"2024-04-25T08:58:47.921286+0000","last_clean":"2024-04-25T08:58:47.921286+0000","last_became_active":"2024-04-25T08:58:46.553422+0000","last_became_peered":"2024-04-25T08:58:46.553422+0000","last_unstale":"2024-04-25T08:58:47.921286+0000","last_undegraded":"2024-04-25T08:58:47.921286+0000","last_fullsized":"2024-04-25T08:58:47.921286+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:49.454085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.548920+0000","last_change":"2024-04-25T08:58:46.552544+0000","last_active":"2024-04-25T08:58:47.548920+0000","last_peered":"2024-04-25T08:58:47.548920+0000","last_clean":"2024-04-25T08:58:47.548920+0000","last_became_active":"2024-04-25T08:58:46.551662+0000","last_became_peered":"2024-04-25T08:58:46.551662+0000","last_unstale":"2024-04-25T08:58:47.548920+0000","last_undegraded":"2024-04-25T08:58:47.548920+0000","last_fullsized":"2024-04-25T08:58:47.548920+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:32.474473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.554161+0000","last_change":"2024-04-25T08:58:46.550264+0000","last_active":"2024-04-25T08:58:47.554161+0000","last_peered":"2024-04-25T08:58:47.554161+0000","last_clean":"2024-04-25T08:58:47.554161+0000","last_became_active":"2024-04-25T08:58:46.549996+0000","last_became_peered":"2024-04-25T08:58:46.549996+0000","last_unstale":"2024-04-25T08:58:47.554161+0000","last_undegraded":"2024-04-25T08:58:47.554161+0000","last_fullsized":"2024-04-25T08:58:47.554161+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:31.390831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.547613+0000","last_change":"2024-04-25T08:58:46.556863+0000","last_active":"2024-04-25T08:58:47.547613+0000","last_peered":"2024-04-25T08:58:47.547613+0000","last_clean":"2024-04-25T08:58:47.547613+0000","last_became_active":"2024-04-25T08:58:46.556554+0000","last_became_peered":"2024-04-25T08:58:46.556554+0000","last_unstale":"2024-04-25T08:58:47.547613+0000","last_undegraded":"2024-04-25T08:58:47.547613+0000","last_fullsized":"2024-04-25T08:58:47.547613+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:56.222512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921237+0000","last_change":"2024-04-25T08:58:46.559636+0000","last_active":"2024-04-25T08:58:47.921237+0000","last_peered":"2024-04-25T08:58:47.921237+0000","last_clean":"2024-04-25T08:58:47.921237+0000","last_became_active":"2024-04-25T08:58:46.559095+0000","last_became_peered":"2024-04-25T08:58:46.559095+0000","last_unstale":"2024-04-25T08:58:47.921237+0000","last_undegraded":"2024-04-25T08:58:47.921237+0000","last_fullsized":"2024-04-25T08:58:47.921237+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:00.656437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921218+0000","last_change":"2024-04-25T08:58:46.555239+0000","last_active":"2024-04-25T08:58:47.921218+0000","last_peered":"2024-04-25T08:58:47.921218+0000","last_clean":"2024-04-25T08:58:47.921218+0000","last_became_active":"2024-04-25T08:58:46.554653+0000","last_became_peered":"2024-04-25T08:58:46.554653+0000","last_unstale":"2024-04-25T08:58:47.921218+0000","last_undegraded":"2024-04-25T08:58:47.921218+0000","last_fullsized":"2024-04-25T08:58:47.921218+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:05.982057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.548719+0000","last_change":"2024-04-25T08:58:46.555570+0000","last_active":"2024-04-25T08:58:47.548719+0000","last_peered":"2024-04-25T08:58:47.548719+0000","last_clean":"2024-04-25T08:58:47.548719+0000","last_became_active":"2024-04-25T08:58:46.555301+0000","last_became_peered":"2024-04-25T08:58:46.555301+0000","last_unstale":"2024-04-25T08:58:47.548719+0000","last_undegraded":"2024-04-25T08:58:47.548719+0000","last_fullsized":"2024-04-25T08:58:47.548719+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:01.993830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.546801+0000","last_change":"2024-04-25T08:58:46.552337+0000","last_active":"2024-04-25T08:58:47.546801+0000","last_peered":"2024-04-25T08:58:47.546801+0000","last_clean":"2024-04-25T08:58:47.546801+0000","last_became_active":"2024-04-25T08:58:46.552004+0000","last_became_peered":"2024-04-25T08:58:46.552004+0000","last_unstale":"2024-04-25T08:58:47.546801+0000","last_undegraded":"2024-04-25T08:58:47.546801+0000","last_fullsized":"2024-04-25T08:58:47.546801+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:54.601317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.556458+0000","last_change":"2024-04-25T08:58:46.560175+0000","last_active":"2024-04-25T08:58:47.556458+0000","last_peered":"2024-04-25T08:58:47.556458+0000","last_clean":"2024-04-25T08:58:47.556458+0000","last_became_active":"2024-04-25T08:58:46.559340+0000","last_became_peered":"2024-04-25T08:58:46.559340+0000","last_unstale":"2024-04-25T08:58:47.556458+0000","last_undegraded":"2024-04-25T08:58:47.556458+0000","last_fullsized":"2024-04-25T08:58:47.556458+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:52.919994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921053+0000","last_change":"2024-04-25T08:58:46.551530+0000","last_active":"2024-04-25T08:58:47.921053+0000","last_peered":"2024-04-25T08:58:47.921053+0000","last_clean":"2024-04-25T08:58:47.921053+0000","last_became_active":"2024-04-25T08:58:46.551083+0000","last_became_peered":"2024-04-25T08:58:46.551083+0000","last_unstale":"2024-04-25T08:58:47.921053+0000","last_undegraded":"2024-04-25T08:58:47.921053+0000","last_fullsized":"2024-04-25T08:58:47.921053+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:24.787164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.556337+0000","last_change":"2024-04-25T08:58:46.549615+0000","last_active":"2024-04-25T08:58:47.556337+0000","last_peered":"2024-04-25T08:58:47.556337+0000","last_clean":"2024-04-25T08:58:47.556337+0000","last_became_active":"2024-04-25T08:58:46.549293+0000","last_became_peered":"2024-04-25T08:58:46.549293+0000","last_unstale":"2024-04-25T08:58:47.556337+0000","last_undegraded":"2024-04-25T08:58:47.556337+0000","last_fullsized":"2024-04-25T08:58:47.556337+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:44.144300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.920741+0000","last_change":"2024-04-25T08:58:46.558943+0000","last_active":"2024-04-25T08:58:47.920741+0000","last_peered":"2024-04-25T08:58:47.920741+0000","last_clean":"2024-04-25T08:58:47.920741+0000","last_became_active":"2024-04-25T08:58:46.558650+0000","last_became_peered":"2024-04-25T08:58:46.558650+0000","last_unstale":"2024-04-25T08:58:47.920741+0000","last_undegraded":"2024-04-25T08:58:47.920741+0000","last_fullsized":"2024-04-25T08:58:47.920741+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:13.676463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921388+0000","last_change":"2024-04-25T08:58:46.561747+0000","last_active":"2024-04-25T08:58:47.921388+0000","last_peered":"2024-04-25T08:58:47.921388+0000","last_clean":"2024-04-25T08:58:47.921388+0000","last_became_active":"2024-04-25T08:58:46.558281+0000","last_became_peered":"2024-04-25T08:58:46.558281+0000","last_unstale":"2024-04-25T08:58:47.921388+0000","last_undegraded":"2024-04-25T08:58:47.921388+0000","last_fullsized":"2024-04-25T08:58:47.921388+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:53.372275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.546758+0000","last_change":"2024-04-25T08:58:46.553711+0000","last_active":"2024-04-25T08:58:47.546758+0000","last_peered":"2024-04-25T08:58:47.546758+0000","last_clean":"2024-04-25T08:58:47.546758+0000","last_became_active":"2024-04-25T08:58:46.553355+0000","last_became_peered":"2024-04-25T08:58:46.553355+0000","last_unstale":"2024-04-25T08:58:47.546758+0000","last_undegraded":"2024-04-25T08:58:47.546758+0000","last_fullsized":"2024-04-25T08:58:47.546758+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:56.517471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.548585+0000","last_change":"2024-04-25T08:58:46.557573+0000","last_active":"2024-04-25T08:58:47.548585+0000","last_peered":"2024-04-25T08:58:47.548585+0000","last_clean":"2024-04-25T08:58:47.548585+0000","last_became_active":"2024-04-25T08:58:46.557020+0000","last_became_peered":"2024-04-25T08:58:46.557020+0000","last_unstale":"2024-04-25T08:58:47.548585+0000","last_undegraded":"2024-04-25T08:58:47.548585+0000","last_fullsized":"2024-04-25T08:58:47.548585+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:45.757920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.556459+0000","last_change":"2024-04-25T08:58:46.560977+0000","last_active":"2024-04-25T08:58:47.556459+0000","last_peered":"2024-04-25T08:58:47.556459+0000","last_clean":"2024-04-25T08:58:47.556459+0000","last_became_active":"2024-04-25T08:58:46.560672+0000","last_became_peered":"2024-04-25T08:58:46.560672+0000","last_unstale":"2024-04-25T08:58:47.556459+0000","last_undegraded":"2024-04-25T08:58:47.556459+0000","last_fullsized":"2024-04-25T08:58:47.556459+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:33.116719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.920983+0000","last_change":"2024-04-25T08:58:46.555160+0000","last_active":"2024-04-25T08:58:47.920983+0000","last_peered":"2024-04-25T08:58:47.920983+0000","last_clean":"2024-04-25T08:58:47.920983+0000","last_became_active":"2024-04-25T08:58:46.554563+0000","last_became_peered":"2024-04-25T08:58:46.554563+0000","last_unstale":"2024-04-25T08:58:47.920983+0000","last_undegraded":"2024-04-25T08:58:47.920983+0000","last_fullsized":"2024-04-25T08:58:47.920983+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:49.915188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.920829+0000","last_change":"2024-04-25T08:58:46.550878+0000","last_active":"2024-04-25T08:58:47.920829+0000","last_peered":"2024-04-25T08:58:47.920829+0000","last_clean":"2024-04-25T08:58:47.920829+0000","last_became_active":"2024-04-25T08:58:46.550530+0000","last_became_peered":"2024-04-25T08:58:46.550530+0000","last_unstale":"2024-04-25T08:58:47.920829+0000","last_undegraded":"2024-04-25T08:58:47.920829+0000","last_fullsized":"2024-04-25T08:58:47.920829+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:56.222512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.547762+0000","last_change":"2024-04-25T08:58:46.546951+0000","last_active":"2024-04-25T08:58:47.547762+0000","last_peered":"2024-04-25T08:58:47.547762+0000","last_clean":"2024-04-25T08:58:47.547762+0000","last_became_active":"2024-04-25T08:58:46.546687+0000","last_became_peered":"2024-04-25T08:58:46.546687+0000","last_unstale":"2024-04-25T08:58:47.547762+0000","last_undegraded":"2024-04-25T08:58:47.547762+0000","last_fullsized":"2024-04-25T08:58:47.547762+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:20.996902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.547512+0000","last_change":"2024-04-25T08:58:46.561556+0000","last_active":"2024-04-25T08:58:47.547512+0000","last_peered":"2024-04-25T08:58:47.547512+0000","last_clean":"2024-04-25T08:58:47.547512+0000","last_became_active":"2024-04-25T08:58:46.561244+0000","last_became_peered":"2024-04-25T08:58:46.561244+0000","last_unstale":"2024-04-25T08:58:47.547512+0000","last_undegraded":"2024-04-25T08:58:47.547512+0000","last_fullsized":"2024-04-25T08:58:47.547512+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:13.640828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921166+0000","last_change":"2024-04-25T08:58:46.554691+0000","last_active":"2024-04-25T08:58:47.921166+0000","last_peered":"2024-04-25T08:58:47.921166+0000","last_clean":"2024-04-25T08:58:47.921166+0000","last_became_active":"2024-04-25T08:58:46.554410+0000","last_became_peered":"2024-04-25T08:58:46.554410+0000","last_unstale":"2024-04-25T08:58:47.921166+0000","last_undegraded":"2024-04-25T08:58:47.921166+0000","last_fullsized":"2024-04-25T08:58:47.921166+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:00.061522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.554524+0000","last_change":"2024-04-25T08:58:46.559256+0000","last_active":"2024-04-25T08:58:47.554524+0000","last_peered":"2024-04-25T08:58:47.554524+0000","last_clean":"2024-04-25T08:58:47.554524+0000","last_became_active":"2024-04-25T08:58:46.558803+0000","last_became_peered":"2024-04-25T08:58:46.558803+0000","last_unstale":"2024-04-25T08:58:47.554524+0000","last_undegraded":"2024-04-25T08:58:47.554524+0000","last_fullsized":"2024-04-25T08:58:47.554524+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:33.116719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921567+0000","last_change":"2024-04-25T08:58:46.548248+0000","last_active":"2024-04-25T08:58:47.921567+0000","last_peered":"2024-04-25T08:58:47.921567+0000","last_clean":"2024-04-25T08:58:47.921567+0000","last_became_active":"2024-04-25T08:58:46.547960+0000","last_became_peered":"2024-04-25T08:58:46.547960+0000","last_unstale":"2024-04-25T08:58:47.921567+0000","last_undegraded":"2024-04-25T08:58:47.921567+0000","last_fullsized":"2024-04-25T08:58:47.921567+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:11:44.139340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921020+0000","last_change":"2024-04-25T08:58:46.550792+0000","last_active":"2024-04-25T08:58:47.921020+0000","last_peered":"2024-04-25T08:58:47.921020+0000","last_clean":"2024-04-25T08:58:47.921020+0000","last_became_active":"2024-04-25T08:58:46.550508+0000","last_became_peered":"2024-04-25T08:58:46.550508+0000","last_unstale":"2024-04-25T08:58:47.921020+0000","last_undegraded":"2024-04-25T08:58:47.921020+0000","last_fullsized":"2024-04-25T08:58:47.921020+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:20.756748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921078+0000","last_change":"2024-04-25T08:58:46.555307+0000","last_active":"2024-04-25T08:58:47.921078+0000","last_peered":"2024-04-25T08:58:47.921078+0000","last_clean":"2024-04-25T08:58:47.921078+0000","last_became_active":"2024-04-25T08:58:46.554992+0000","last_became_peered":"2024-04-25T08:58:46.554992+0000","last_unstale":"2024-04-25T08:58:47.921078+0000","last_undegraded":"2024-04-25T08:58:47.921078+0000","last_fullsized":"2024-04-25T08:58:47.921078+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:19.756407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.554290+0000","last_change":"2024-04-25T08:58:46.554809+0000","last_active":"2024-04-25T08:58:47.554290+0000","last_peered":"2024-04-25T08:58:47.554290+0000","last_clean":"2024-04-25T08:58:47.554290+0000","last_became_active":"2024-04-25T08:58:46.554526+0000","last_became_peered":"2024-04-25T08:58:46.554526+0000","last_unstale":"2024-04-25T08:58:47.554290+0000","last_undegraded":"2024-04-25T08:58:47.554290+0000","last_fullsized":"2024-04-25T08:58:47.554290+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:45.020761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.554464+0000","last_change":"2024-04-25T08:58:46.551042+0000","last_active":"2024-04-25T08:58:47.554464+0000","last_peered":"2024-04-25T08:58:47.554464+0000","last_clean":"2024-04-25T08:58:47.554464+0000","last_became_active":"2024-04-25T08:58:46.550738+0000","last_became_peered":"2024-04-25T08:58:46.550738+0000","last_unstale":"2024-04-25T08:58:47.554464+0000","last_undegraded":"2024-04-25T08:58:47.554464+0000","last_fullsized":"2024-04-25T08:58:47.554464+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:18.357607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.547669+0000","last_change":"2024-04-25T08:58:46.552873+0000","last_active":"2024-04-25T08:58:47.547669+0000","last_peered":"2024-04-25T08:58:47.547669+0000","last_clean":"2024-04-25T08:58:47.547669+0000","last_became_active":"2024-04-25T08:58:46.551313+0000","last_became_peered":"2024-04-25T08:58:46.551313+0000","last_unstale":"2024-04-25T08:58:47.547669+0000","last_undegraded":"2024-04-25T08:58:47.547669+0000","last_fullsized":"2024-04-25T08:58:47.547669+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:49.915188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.548456+0000","last_change":"2024-04-25T08:58:46.561672+0000","last_active":"2024-04-25T08:58:47.548456+0000","last_peered":"2024-04-25T08:58:47.548456+0000","last_clean":"2024-04-25T08:58:47.548456+0000","last_became_active":"2024-04-25T08:58:46.561384+0000","last_became_peered":"2024-04-25T08:58:46.561384+0000","last_unstale":"2024-04-25T08:58:47.548456+0000","last_undegraded":"2024-04-25T08:58:47.548456+0000","last_fullsized":"2024-04-25T08:58:47.548456+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:56.419286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"72'4","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.549389+0000","last_change":"2024-04-25T08:58:45.552209+0000","last_active":"2024-04-25T08:58:47.549389+0000","last_peered":"2024-04-25T08:58:47.549389+0000","last_clean":"2024-04-25T08:58:47.549389+0000","last_became_active":"2024-04-25T08:58:45.551711+0000","last_became_peered":"2024-04-25T08:58:45.551711+0000","last_unstale":"2024-04-25T08:58:47.549389+0000","last_undegraded":"2024-04-25T08:58:47.549389+0000","last_fullsized":"2024-04-25T08:58:47.549389+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:08.904655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921563+0000","last_change":"2024-04-25T08:58:46.549376+0000","last_active":"2024-04-25T08:58:47.921563+0000","last_peered":"2024-04-25T08:58:47.921563+0000","last_clean":"2024-04-25T08:58:47.921563+0000","last_became_active":"2024-04-25T08:58:46.549051+0000","last_became_peered":"2024-04-25T08:58:46.549051+0000","last_unstale":"2024-04-25T08:58:47.921563+0000","last_undegraded":"2024-04-25T08:58:47.921563+0000","last_fullsized":"2024-04-25T08:58:47.921563+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:31.951240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"72'1","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.556551+0000","last_change":"2024-04-25T08:58:45.548533+0000","last_active":"2024-04-25T08:58:47.556551+0000","last_peered":"2024-04-25T08:58:47.556551+0000","last_clean":"2024-04-25T08:58:47.556551+0000","last_became_active":"2024-04-25T08:58:45.547726+0000","last_became_peered":"2024-04-25T08:58:45.547726+0000","last_unstale":"2024-04-25T08:58:47.556551+0000","last_undegraded":"2024-04-25T08:58:47.556551+0000","last_fullsized":"2024-04-25T08:58:47.556551+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:31.469571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.548195+0000","last_change":"2024-04-25T08:58:46.557827+0000","last_active":"2024-04-25T08:58:47.548195+0000","last_peered":"2024-04-25T08:58:47.548195+0000","last_clean":"2024-04-25T08:58:47.548195+0000","last_became_active":"2024-04-25T08:58:46.557443+0000","last_became_peered":"2024-04-25T08:58:46.557443+0000","last_unstale":"2024-04-25T08:58:47.548195+0000","last_undegraded":"2024-04-25T08:58:47.548195+0000","last_fullsized":"2024-04-25T08:58:47.548195+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:26.721498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"72'2","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921076+0000","last_change":"2024-04-25T08:58:45.551223+0000","last_active":"2024-04-25T08:58:47.921076+0000","last_peered":"2024-04-25T08:58:47.921076+0000","last_clean":"2024-04-25T08:58:47.921076+0000","last_became_active":"2024-04-25T08:58:45.550909+0000","last_became_peered":"2024-04-25T08:58:45.550909+0000","last_unstale":"2024-04-25T08:58:47.921076+0000","last_undegraded":"2024-04-25T08:58:47.921076+0000","last_fullsized":"2024-04-25T08:58:47.921076+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:39.804943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.548886+0000","last_change":"2024-04-25T08:58:46.552878+0000","last_active":"2024-04-25T08:58:47.548886+0000","last_peered":"2024-04-25T08:58:47.548886+0000","last_clean":"2024-04-25T08:58:47.548886+0000","last_became_active":"2024-04-25T08:58:46.552616+0000","last_became_peered":"2024-04-25T08:58:46.552616+0000","last_unstale":"2024-04-25T08:58:47.548886+0000","last_undegraded":"2024-04-25T08:58:47.548886+0000","last_fullsized":"2024-04-25T08:58:47.548886+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:02.850873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"72'2","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.548695+0000","last_change":"2024-04-25T08:58:45.553283+0000","last_active":"2024-04-25T08:58:47.548695+0000","last_peered":"2024-04-25T08:58:47.548695+0000","last_clean":"2024-04-25T08:58:47.548695+0000","last_became_active":"2024-04-25T08:58:45.553012+0000","last_became_peered":"2024-04-25T08:58:45.553012+0000","last_unstale":"2024-04-25T08:58:47.548695+0000","last_undegraded":"2024-04-25T08:58:47.548695+0000","last_fullsized":"2024-04-25T08:58:47.548695+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:17.858568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.554350+0000","last_change":"2024-04-25T08:58:46.546837+0000","last_active":"2024-04-25T08:58:47.554350+0000","last_peered":"2024-04-25T08:58:47.554350+0000","last_clean":"2024-04-25T08:58:47.554350+0000","last_became_active":"2024-04-25T08:58:46.546582+0000","last_became_peered":"2024-04-25T08:58:46.546582+0000","last_unstale":"2024-04-25T08:58:47.554350+0000","last_undegraded":"2024-04-25T08:58:47.554350+0000","last_fullsized":"2024-04-25T08:58:47.554350+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:41.339390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"72'1","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921209+0000","last_change":"2024-04-25T08:58:45.550618+0000","last_active":"2024-04-25T08:58:47.921209+0000","last_peered":"2024-04-25T08:58:47.921209+0000","last_clean":"2024-04-25T08:58:47.921209+0000","last_became_active":"2024-04-25T08:58:45.550332+0000","last_became_peered":"2024-04-25T08:58:45.550332+0000","last_unstale":"2024-04-25T08:58:47.921209+0000","last_undegraded":"2024-04-25T08:58:47.921209+0000","last_fullsized":"2024-04-25T08:58:47.921209+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:01.845971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921079+0000","last_change":"2024-04-25T08:58:46.555264+0000","last_active":"2024-04-25T08:58:47.921079+0000","last_peered":"2024-04-25T08:58:47.921079+0000","last_clean":"2024-04-25T08:58:47.921079+0000","last_became_active":"2024-04-25T08:58:46.554909+0000","last_became_peered":"2024-04-25T08:58:46.554909+0000","last_unstale":"2024-04-25T08:58:47.921079+0000","last_undegraded":"2024-04-25T08:58:47.921079+0000","last_fullsized":"2024-04-25T08:58:47.921079+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:26.721498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"72'1","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.548211+0000","last_change":"2024-04-25T08:58:45.554390+0000","last_active":"2024-04-25T08:58:47.548211+0000","last_peered":"2024-04-25T08:58:47.548211+0000","last_clean":"2024-04-25T08:58:47.548211+0000","last_became_active":"2024-04-25T08:58:45.554067+0000","last_became_peered":"2024-04-25T08:58:45.554067+0000","last_unstale":"2024-04-25T08:58:47.548211+0000","last_undegraded":"2024-04-25T08:58:47.548211+0000","last_fullsized":"2024-04-25T08:58:47.548211+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:23.089008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.920981+0000","last_change":"2024-04-25T08:58:46.546861+0000","last_active":"2024-04-25T08:58:47.920981+0000","last_peered":"2024-04-25T08:58:47.920981+0000","last_clean":"2024-04-25T08:58:47.920981+0000","last_became_active":"2024-04-25T08:58:46.546555+0000","last_became_peered":"2024-04-25T08:58:46.546555+0000","last_unstale":"2024-04-25T08:58:47.920981+0000","last_undegraded":"2024-04-25T08:58:47.920981+0000","last_fullsized":"2024-04-25T08:58:47.920981+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:10.527777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.546418+0000","last_change":"2024-04-25T08:58:45.542798+0000","last_active":"2024-04-25T08:58:47.546418+0000","last_peered":"2024-04-25T08:58:47.546418+0000","last_clean":"2024-04-25T08:58:47.546418+0000","last_became_active":"2024-04-25T08:58:45.542020+0000","last_became_peered":"2024-04-25T08:58:45.542020+0000","last_unstale":"2024-04-25T08:58:47.546418+0000","last_undegraded":"2024-04-25T08:58:47.546418+0000","last_fullsized":"2024-04-25T08:58:47.546418+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:17.858568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.556386+0000","last_change":"2024-04-25T08:58:46.550466+0000","last_active":"2024-04-25T08:58:47.556386+0000","last_peered":"2024-04-25T08:58:47.556386+0000","last_clean":"2024-04-25T08:58:47.556386+0000","last_became_active":"2024-04-25T08:58:46.550099+0000","last_became_peered":"2024-04-25T08:58:46.550099+0000","last_unstale":"2024-04-25T08:58:47.556386+0000","last_undegraded":"2024-04-25T08:58:47.556386+0000","last_fullsized":"2024-04-25T08:58:47.556386+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:18.357607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.920901+0000","last_change":"2024-04-25T08:58:45.550842+0000","last_active":"2024-04-25T08:58:47.920901+0000","last_peered":"2024-04-25T08:58:47.920901+0000","last_clean":"2024-04-25T08:58:47.920901+0000","last_became_active":"2024-04-25T08:58:45.550579+0000","last_became_peered":"2024-04-25T08:58:45.550579+0000","last_unstale":"2024-04-25T08:58:47.920901+0000","last_undegraded":"2024-04-25T08:58:47.920901+0000","last_fullsized":"2024-04-25T08:58:47.920901+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:36.533485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.554550+0000","last_change":"2024-04-25T08:58:46.550247+0000","last_active":"2024-04-25T08:58:47.554550+0000","last_peered":"2024-04-25T08:58:47.554550+0000","last_clean":"2024-04-25T08:58:47.554550+0000","last_became_active":"2024-04-25T08:58:46.549966+0000","last_became_peered":"2024-04-25T08:58:46.549966+0000","last_unstale":"2024-04-25T08:58:47.554550+0000","last_undegraded":"2024-04-25T08:58:47.554550+0000","last_fullsized":"2024-04-25T08:58:47.554550+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:13.640828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921160+0000","last_change":"2024-04-25T08:58:45.547081+0000","last_active":"2024-04-25T08:58:47.921160+0000","last_peered":"2024-04-25T08:58:47.921160+0000","last_clean":"2024-04-25T08:58:47.921160+0000","last_became_active":"2024-04-25T08:58:45.546672+0000","last_became_peered":"2024-04-25T08:58:45.546672+0000","last_unstale":"2024-04-25T08:58:47.921160+0000","last_undegraded":"2024-04-25T08:58:47.921160+0000","last_fullsized":"2024-04-25T08:58:47.921160+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:33.945526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.549255+0000","last_change":"2024-04-25T08:58:46.560124+0000","last_active":"2024-04-25T08:58:47.549255+0000","last_peered":"2024-04-25T08:58:47.549255+0000","last_clean":"2024-04-25T08:58:47.549255+0000","last_became_active":"2024-04-25T08:58:46.559836+0000","last_became_peered":"2024-04-25T08:58:46.559836+0000","last_unstale":"2024-04-25T08:58:47.549255+0000","last_undegraded":"2024-04-25T08:58:47.549255+0000","last_fullsized":"2024-04-25T08:58:47.549255+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:11.068522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.549167+0000","last_change":"2024-04-25T08:58:45.553198+0000","last_active":"2024-04-25T08:58:47.549167+0000","last_peered":"2024-04-25T08:58:47.549167+0000","last_clean":"2024-04-25T08:58:47.549167+0000","last_became_active":"2024-04-25T08:58:45.552506+0000","last_became_peered":"2024-04-25T08:58:45.552506+0000","last_unstale":"2024-04-25T08:58:47.549167+0000","last_undegraded":"2024-04-25T08:58:47.549167+0000","last_fullsized":"2024-04-25T08:58:47.549167+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:56.102307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.548585+0000","last_change":"2024-04-25T08:58:46.557705+0000","last_active":"2024-04-25T08:58:47.548585+0000","last_peered":"2024-04-25T08:58:47.548585+0000","last_clean":"2024-04-25T08:58:47.548585+0000","last_became_active":"2024-04-25T08:58:46.557081+0000","last_became_peered":"2024-04-25T08:58:46.557081+0000","last_unstale":"2024-04-25T08:58:47.548585+0000","last_undegraded":"2024-04-25T08:58:47.548585+0000","last_fullsized":"2024-04-25T08:58:47.548585+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:05.982057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"72'3","reported_seq":17,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921707+0000","last_change":"2024-04-25T08:58:45.548063+0000","last_active":"2024-04-25T08:58:47.921707+0000","last_peered":"2024-04-25T08:58:47.921707+0000","last_clean":"2024-04-25T08:58:47.921707+0000","last_became_active":"2024-04-25T08:58:45.546207+0000","last_became_peered":"2024-04-25T08:58:45.546207+0000","last_unstale":"2024-04-25T08:58:47.921707+0000","last_undegraded":"2024-04-25T08:58:47.921707+0000","last_fullsized":"2024-04-25T08:58:47.921707+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:00.988928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921381+0000","last_change":"2024-04-25T08:58:46.551175+0000","last_active":"2024-04-25T08:58:47.921381+0000","last_peered":"2024-04-25T08:58:47.921381+0000","last_clean":"2024-04-25T08:58:47.921381+0000","last_became_active":"2024-04-25T08:58:46.550799+0000","last_became_peered":"2024-04-25T08:58:46.550799+0000","last_unstale":"2024-04-25T08:58:47.921381+0000","last_undegraded":"2024-04-25T08:58:47.921381+0000","last_fullsized":"2024-04-25T08:58:47.921381+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:58.018327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"72'6","reported_seq":20,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.546805+0000","last_change":"2024-04-25T08:58:45.547581+0000","last_active":"2024-04-25T08:58:47.546805+0000","last_peered":"2024-04-25T08:58:47.546805+0000","last_clean":"2024-04-25T08:58:47.546805+0000","last_became_active":"2024-04-25T08:58:45.547280+0000","last_became_peered":"2024-04-25T08:58:45.547280+0000","last_unstale":"2024-04-25T08:58:47.546805+0000","last_undegraded":"2024-04-25T08:58:47.546805+0000","last_fullsized":"2024-04-25T08:58:47.546805+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:33.494101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.547956+0000","last_change":"2024-04-25T08:58:46.554406+0000","last_active":"2024-04-25T08:58:47.547956+0000","last_peered":"2024-04-25T08:58:47.547956+0000","last_clean":"2024-04-25T08:58:47.547956+0000","last_became_active":"2024-04-25T08:58:46.554096+0000","last_became_peered":"2024-04-25T08:58:46.554096+0000","last_unstale":"2024-04-25T08:58:47.547956+0000","last_undegraded":"2024-04-25T08:58:47.547956+0000","last_fullsized":"2024-04-25T08:58:47.547956+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:00.935851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"72'2","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.546518+0000","last_change":"2024-04-25T08:58:45.547352+0000","last_active":"2024-04-25T08:58:47.546518+0000","last_peered":"2024-04-25T08:58:47.546518+0000","last_clean":"2024-04-25T08:58:47.546518+0000","last_became_active":"2024-04-25T08:58:45.546900+0000","last_became_peered":"2024-04-25T08:58:45.546900+0000","last_unstale":"2024-04-25T08:58:47.546518+0000","last_undegraded":"2024-04-25T08:58:47.546518+0000","last_fullsized":"2024-04-25T08:58:47.546518+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:56.102307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.554453+0000","last_change":"2024-04-25T08:58:46.542814+0000","last_active":"2024-04-25T08:58:47.554453+0000","last_peered":"2024-04-25T08:58:47.554453+0000","last_clean":"2024-04-25T08:58:47.554453+0000","last_became_active":"2024-04-25T08:58:46.542530+0000","last_became_peered":"2024-04-25T08:58:46.542530+0000","last_unstale":"2024-04-25T08:58:47.554453+0000","last_undegraded":"2024-04-25T08:58:47.554453+0000","last_fullsized":"2024-04-25T08:58:47.554453+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:44.144300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.548678+0000","last_change":"2024-04-25T08:58:45.553000+0000","last_active":"2024-04-25T08:58:47.548678+0000","last_peered":"2024-04-25T08:58:47.548678+0000","last_clean":"2024-04-25T08:58:47.548678+0000","last_became_active":"2024-04-25T08:58:45.552593+0000","last_became_peered":"2024-04-25T08:58:45.552593+0000","last_unstale":"2024-04-25T08:58:47.548678+0000","last_undegraded":"2024-04-25T08:58:47.548678+0000","last_fullsized":"2024-04-25T08:58:47.548678+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:30.946338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.556297+0000","last_change":"2024-04-25T08:58:46.559888+0000","last_active":"2024-04-25T08:58:47.556297+0000","last_peered":"2024-04-25T08:58:47.556297+0000","last_clean":"2024-04-25T08:58:47.556297+0000","last_became_active":"2024-04-25T08:58:46.559566+0000","last_became_peered":"2024-04-25T08:58:46.559566+0000","last_unstale":"2024-04-25T08:58:47.556297+0000","last_undegraded":"2024-04-25T08:58:47.556297+0000","last_fullsized":"2024-04-25T08:58:47.556297+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:41.339390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.554171+0000","last_change":"2024-04-25T08:58:45.542358+0000","last_active":"2024-04-25T08:58:47.554171+0000","last_peered":"2024-04-25T08:58:47.554171+0000","last_clean":"2024-04-25T08:58:47.554171+0000","last_became_active":"2024-04-25T08:58:45.542071+0000","last_became_peered":"2024-04-25T08:58:45.542071+0000","last_unstale":"2024-04-25T08:58:47.554171+0000","last_undegraded":"2024-04-25T08:58:47.554171+0000","last_fullsized":"2024-04-25T08:58:47.554171+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:48.910286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921560+0000","last_change":"2024-04-25T08:58:46.547715+0000","last_active":"2024-04-25T08:58:47.921560+0000","last_peered":"2024-04-25T08:58:47.921560+0000","last_clean":"2024-04-25T08:58:47.921560+0000","last_became_active":"2024-04-25T08:58:46.547408+0000","last_became_peered":"2024-04-25T08:58:46.547408+0000","last_unstale":"2024-04-25T08:58:47.921560+0000","last_undegraded":"2024-04-25T08:58:47.921560+0000","last_fullsized":"2024-04-25T08:58:47.921560+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:21.813962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"72'8","reported_seq":22,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.920768+0000","last_change":"2024-04-25T08:58:45.546702+0000","last_active":"2024-04-25T08:58:47.920768+0000","last_peered":"2024-04-25T08:58:47.920768+0000","last_clean":"2024-04-25T08:58:47.920768+0000","last_became_active":"2024-04-25T08:58:45.546346+0000","last_became_peered":"2024-04-25T08:58:45.546346+0000","last_unstale":"2024-04-25T08:58:47.920768+0000","last_undegraded":"2024-04-25T08:58:47.920768+0000","last_fullsized":"2024-04-25T08:58:47.920768+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:17.858568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921198+0000","last_change":"2024-04-25T08:58:46.555032+0000","last_active":"2024-04-25T08:58:47.921198+0000","last_peered":"2024-04-25T08:58:47.921198+0000","last_clean":"2024-04-25T08:58:47.921198+0000","last_became_active":"2024-04-25T08:58:46.554239+0000","last_became_peered":"2024-04-25T08:58:46.554239+0000","last_unstale":"2024-04-25T08:58:47.921198+0000","last_undegraded":"2024-04-25T08:58:47.921198+0000","last_fullsized":"2024-04-25T08:58:47.921198+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:45.757920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"72'1","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.548912+0000","last_change":"2024-04-25T08:58:45.551995+0000","last_active":"2024-04-25T08:58:47.548912+0000","last_peered":"2024-04-25T08:58:47.548912+0000","last_clean":"2024-04-25T08:58:47.548912+0000","last_became_active":"2024-04-25T08:58:45.551744+0000","last_became_peered":"2024-04-25T08:58:45.551744+0000","last_unstale":"2024-04-25T08:58:47.548912+0000","last_undegraded":"2024-04-25T08:58:47.548912+0000","last_fullsized":"2024-04-25T08:58:47.548912+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:33.494101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.549337+0000","last_change":"2024-04-25T08:58:46.565290+0000","last_active":"2024-04-25T08:58:47.549337+0000","last_peered":"2024-04-25T08:58:47.549337+0000","last_clean":"2024-04-25T08:58:47.549337+0000","last_became_active":"2024-04-25T08:58:46.564451+0000","last_became_peered":"2024-04-25T08:58:46.564451+0000","last_unstale":"2024-04-25T08:58:47.549337+0000","last_undegraded":"2024-04-25T08:58:47.549337+0000","last_fullsized":"2024-04-25T08:58:47.549337+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:31.000732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"72'2","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921262+0000","last_change":"2024-04-25T08:58:45.551495+0000","last_active":"2024-04-25T08:58:47.921262+0000","last_peered":"2024-04-25T08:58:47.921262+0000","last_clean":"2024-04-25T08:58:47.921262+0000","last_became_active":"2024-04-25T08:58:45.551222+0000","last_became_peered":"2024-04-25T08:58:45.551222+0000","last_unstale":"2024-04-25T08:58:47.921262+0000","last_undegraded":"2024-04-25T08:58:47.921262+0000","last_fullsized":"2024-04-25T08:58:47.921262+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:33.494101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.546739+0000","last_change":"2024-04-25T08:58:46.552609+0000","last_active":"2024-04-25T08:58:47.546739+0000","last_peered":"2024-04-25T08:58:47.546739+0000","last_clean":"2024-04-25T08:58:47.546739+0000","last_became_active":"2024-04-25T08:58:46.551554+0000","last_became_peered":"2024-04-25T08:58:46.551554+0000","last_unstale":"2024-04-25T08:58:47.546739+0000","last_undegraded":"2024-04-25T08:58:47.546739+0000","last_fullsized":"2024-04-25T08:58:47.546739+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:52.614541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"72'1","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.548679+0000","last_change":"2024-04-25T08:58:45.549628+0000","last_active":"2024-04-25T08:58:47.548679+0000","last_peered":"2024-04-25T08:58:47.548679+0000","last_clean":"2024-04-25T08:58:47.548679+0000","last_became_active":"2024-04-25T08:58:45.549330+0000","last_became_peered":"2024-04-25T08:58:45.549330+0000","last_unstale":"2024-04-25T08:58:47.548679+0000","last_undegraded":"2024-04-25T08:58:47.548679+0000","last_fullsized":"2024-04-25T08:58:47.548679+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:57.275171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921547+0000","last_change":"2024-04-25T08:58:46.556781+0000","last_active":"2024-04-25T08:58:47.921547+0000","last_peered":"2024-04-25T08:58:47.921547+0000","last_clean":"2024-04-25T08:58:47.921547+0000","last_became_active":"2024-04-25T08:58:46.556485+0000","last_became_peered":"2024-04-25T08:58:46.556485+0000","last_unstale":"2024-04-25T08:58:47.921547+0000","last_undegraded":"2024-04-25T08:58:47.921547+0000","last_fullsized":"2024-04-25T08:58:47.921547+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:24.093910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.548306+0000","last_change":"2024-04-25T08:58:45.550699+0000","last_active":"2024-04-25T08:58:47.548306+0000","last_peered":"2024-04-25T08:58:47.548306+0000","last_clean":"2024-04-25T08:58:47.548306+0000","last_became_active":"2024-04-25T08:58:45.550408+0000","last_became_peered":"2024-04-25T08:58:45.550408+0000","last_unstale":"2024-04-25T08:58:47.548306+0000","last_undegraded":"2024-04-25T08:58:47.548306+0000","last_fullsized":"2024-04-25T08:58:47.548306+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:31.469571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921673+0000","last_change":"2024-04-25T08:58:46.543314+0000","last_active":"2024-04-25T08:58:47.921673+0000","last_peered":"2024-04-25T08:58:47.921673+0000","last_clean":"2024-04-25T08:58:47.921673+0000","last_became_active":"2024-04-25T08:58:46.543011+0000","last_became_peered":"2024-04-25T08:58:46.543011+0000","last_unstale":"2024-04-25T08:58:47.921673+0000","last_undegraded":"2024-04-25T08:58:47.921673+0000","last_fullsized":"2024-04-25T08:58:47.921673+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:16.379484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.548773+0000","last_change":"2024-04-25T08:58:45.554464+0000","last_active":"2024-04-25T08:58:47.548773+0000","last_peered":"2024-04-25T08:58:47.548773+0000","last_clean":"2024-04-25T08:58:47.548773+0000","last_became_active":"2024-04-25T08:58:45.554162+0000","last_became_peered":"2024-04-25T08:58:45.554162+0000","last_unstale":"2024-04-25T08:58:47.548773+0000","last_undegraded":"2024-04-25T08:58:47.548773+0000","last_fullsized":"2024-04-25T08:58:47.548773+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:20.809060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.920947+0000","last_change":"2024-04-25T08:58:46.559421+0000","last_active":"2024-04-25T08:58:47.920947+0000","last_peered":"2024-04-25T08:58:47.920947+0000","last_clean":"2024-04-25T08:58:47.920947+0000","last_became_active":"2024-04-25T08:58:46.559130+0000","last_became_peered":"2024-04-25T08:58:46.559130+0000","last_unstale":"2024-04-25T08:58:47.920947+0000","last_undegraded":"2024-04-25T08:58:47.920947+0000","last_fullsized":"2024-04-25T08:58:47.920947+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:29.992024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.554395+0000","last_change":"2024-04-25T08:58:45.547534+0000","last_active":"2024-04-25T08:58:47.554395+0000","last_peered":"2024-04-25T08:58:47.554395+0000","last_clean":"2024-04-25T08:58:47.554395+0000","last_became_active":"2024-04-25T08:58:45.547251+0000","last_became_peered":"2024-04-25T08:58:45.547251+0000","last_unstale":"2024-04-25T08:58:47.554395+0000","last_undegraded":"2024-04-25T08:58:47.554395+0000","last_fullsized":"2024-04-25T08:58:47.554395+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:19.751845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921074+0000","last_change":"2024-04-25T08:58:46.558692+0000","last_active":"2024-04-25T08:58:47.921074+0000","last_peered":"2024-04-25T08:58:47.921074+0000","last_clean":"2024-04-25T08:58:47.921074+0000","last_became_active":"2024-04-25T08:58:46.558034+0000","last_became_peered":"2024-04-25T08:58:46.558034+0000","last_unstale":"2024-04-25T08:58:47.921074+0000","last_undegraded":"2024-04-25T08:58:47.921074+0000","last_fullsized":"2024-04-25T08:58:47.921074+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:06.462171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.556239+0000","last_change":"2024-04-25T08:58:45.548289+0000","last_active":"2024-04-25T08:58:47.556239+0000","last_peered":"2024-04-25T08:58:47.556239+0000","last_clean":"2024-04-25T08:58:47.556239+0000","last_became_active":"2024-04-25T08:58:45.547912+0000","last_became_peered":"2024-04-25T08:58:45.547912+0000","last_unstale":"2024-04-25T08:58:47.556239+0000","last_undegraded":"2024-04-25T08:58:47.556239+0000","last_fullsized":"2024-04-25T08:58:47.556239+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:59.882831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.556524+0000","last_change":"2024-04-25T08:58:46.560297+0000","last_active":"2024-04-25T08:58:47.556524+0000","last_peered":"2024-04-25T08:58:47.556524+0000","last_clean":"2024-04-25T08:58:47.556524+0000","last_became_active":"2024-04-25T08:58:46.559975+0000","last_became_peered":"2024-04-25T08:58:46.559975+0000","last_unstale":"2024-04-25T08:58:47.556524+0000","last_undegraded":"2024-04-25T08:58:47.556524+0000","last_fullsized":"2024-04-25T08:58:47.556524+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:13.640828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"72'2","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.549012+0000","last_change":"2024-04-25T08:58:45.551966+0000","last_active":"2024-04-25T08:58:47.549012+0000","last_peered":"2024-04-25T08:58:47.549012+0000","last_clean":"2024-04-25T08:58:47.549012+0000","last_became_active":"2024-04-25T08:58:45.551405+0000","last_became_peered":"2024-04-25T08:58:45.551405+0000","last_unstale":"2024-04-25T08:58:47.549012+0000","last_undegraded":"2024-04-25T08:58:47.549012+0000","last_fullsized":"2024-04-25T08:58:47.549012+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:39.804943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.554514+0000","last_change":"2024-04-25T08:58:46.548415+0000","last_active":"2024-04-25T08:58:47.554514+0000","last_peered":"2024-04-25T08:58:47.554514+0000","last_clean":"2024-04-25T08:58:47.554514+0000","last_became_active":"2024-04-25T08:58:46.548134+0000","last_became_peered":"2024-04-25T08:58:46.548134+0000","last_unstale":"2024-04-25T08:58:47.554514+0000","last_undegraded":"2024-04-25T08:58:47.554514+0000","last_fullsized":"2024-04-25T08:58:47.554514+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:52.919994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"73'1","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.555891+0000","last_change":"2024-04-25T08:58:45.543388+0000","last_active":"2024-04-25T08:58:47.555891+0000","last_peered":"2024-04-25T08:58:47.555891+0000","last_clean":"2024-04-25T08:58:47.555891+0000","last_became_active":"2024-04-25T08:58:45.543130+0000","last_became_peered":"2024-04-25T08:58:45.543130+0000","last_unstale":"2024-04-25T08:58:47.555891+0000","last_undegraded":"2024-04-25T08:58:47.555891+0000","last_fullsized":"2024-04-25T08:58:47.555891+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:25.716596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.547710+0000","last_change":"2024-04-25T08:58:46.561203+0000","last_active":"2024-04-25T08:58:47.547710+0000","last_peered":"2024-04-25T08:58:47.547710+0000","last_clean":"2024-04-25T08:58:47.547710+0000","last_became_active":"2024-04-25T08:58:46.560912+0000","last_became_peered":"2024-04-25T08:58:46.560912+0000","last_unstale":"2024-04-25T08:58:47.547710+0000","last_undegraded":"2024-04-25T08:58:47.547710+0000","last_fullsized":"2024-04-25T08:58:47.547710+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:42.343818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921491+0000","last_change":"2024-04-25T08:58:45.548036+0000","last_active":"2024-04-25T08:58:47.921491+0000","last_peered":"2024-04-25T08:58:47.921491+0000","last_clean":"2024-04-25T08:58:47.921491+0000","last_became_active":"2024-04-25T08:58:45.547730+0000","last_became_peered":"2024-04-25T08:58:45.547730+0000","last_unstale":"2024-04-25T08:58:47.921491+0000","last_undegraded":"2024-04-25T08:58:47.921491+0000","last_fullsized":"2024-04-25T08:58:47.921491+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:32.063884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.549040+0000","last_change":"2024-04-25T08:58:46.552300+0000","last_active":"2024-04-25T08:58:47.549040+0000","last_peered":"2024-04-25T08:58:47.549040+0000","last_clean":"2024-04-25T08:58:47.549040+0000","last_became_active":"2024-04-25T08:58:46.552006+0000","last_became_peered":"2024-04-25T08:58:46.552006+0000","last_unstale":"2024-04-25T08:58:47.549040+0000","last_undegraded":"2024-04-25T08:58:47.549040+0000","last_fullsized":"2024-04-25T08:58:47.549040+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:07.410153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.556434+0000","last_change":"2024-04-25T08:58:45.546323+0000","last_active":"2024-04-25T08:58:47.556434+0000","last_peered":"2024-04-25T08:58:47.556434+0000","last_clean":"2024-04-25T08:58:47.556434+0000","last_became_active":"2024-04-25T08:58:45.545900+0000","last_became_peered":"2024-04-25T08:58:45.545900+0000","last_unstale":"2024-04-25T08:58:47.556434+0000","last_undegraded":"2024-04-25T08:58:47.556434+0000","last_fullsized":"2024-04-25T08:58:47.556434+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:04.977155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921216+0000","last_change":"2024-04-25T08:58:46.552672+0000","last_active":"2024-04-25T08:58:47.921216+0000","last_peered":"2024-04-25T08:58:47.921216+0000","last_clean":"2024-04-25T08:58:47.921216+0000","last_became_active":"2024-04-25T08:58:46.552265+0000","last_became_peered":"2024-04-25T08:58:46.552265+0000","last_unstale":"2024-04-25T08:58:47.921216+0000","last_undegraded":"2024-04-25T08:58:47.921216+0000","last_fullsized":"2024-04-25T08:58:47.921216+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:17.939556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.920999+0000","last_change":"2024-04-25T08:58:45.548287+0000","last_active":"2024-04-25T08:58:47.920999+0000","last_peered":"2024-04-25T08:58:47.920999+0000","last_clean":"2024-04-25T08:58:47.920999+0000","last_became_active":"2024-04-25T08:58:45.547386+0000","last_became_peered":"2024-04-25T08:58:45.547386+0000","last_unstale":"2024-04-25T08:58:47.920999+0000","last_undegraded":"2024-04-25T08:58:47.920999+0000","last_fullsized":"2024-04-25T08:58:47.920999+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:08.516997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.548447+0000","last_change":"2024-04-25T08:58:46.556933+0000","last_active":"2024-04-25T08:58:47.548447+0000","last_peered":"2024-04-25T08:58:47.548447+0000","last_clean":"2024-04-25T08:58:47.548447+0000","last_became_active":"2024-04-25T08:58:46.556606+0000","last_became_peered":"2024-04-25T08:58:46.556606+0000","last_unstale":"2024-04-25T08:58:47.548447+0000","last_undegraded":"2024-04-25T08:58:47.548447+0000","last_fullsized":"2024-04-25T08:58:47.548447+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:00.061522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.546411+0000","last_change":"2024-04-25T08:58:45.541082+0000","last_active":"2024-04-25T08:58:47.546411+0000","last_peered":"2024-04-25T08:58:47.546411+0000","last_clean":"2024-04-25T08:58:47.546411+0000","last_became_active":"2024-04-25T08:58:45.540543+0000","last_became_peered":"2024-04-25T08:58:45.540543+0000","last_unstale":"2024-04-25T08:58:47.546411+0000","last_undegraded":"2024-04-25T08:58:47.546411+0000","last_fullsized":"2024-04-25T08:58:47.546411+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:57.275171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.546506+0000","last_change":"2024-04-25T08:58:46.543187+0000","last_active":"2024-04-25T08:58:47.546506+0000","last_peered":"2024-04-25T08:58:47.546506+0000","last_clean":"2024-04-25T08:58:47.546506+0000","last_became_active":"2024-04-25T08:58:46.542866+0000","last_became_peered":"2024-04-25T08:58:46.542866+0000","last_unstale":"2024-04-25T08:58:47.546506+0000","last_undegraded":"2024-04-25T08:58:47.546506+0000","last_fullsized":"2024-04-25T08:58:47.546506+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:20.948989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.920717+0000","last_change":"2024-04-25T08:58:45.547880+0000","last_active":"2024-04-25T08:58:47.920717+0000","last_peered":"2024-04-25T08:58:47.920717+0000","last_clean":"2024-04-25T08:58:47.920717+0000","last_became_active":"2024-04-25T08:58:45.547619+0000","last_became_peered":"2024-04-25T08:58:45.547619+0000","last_unstale":"2024-04-25T08:58:47.920717+0000","last_undegraded":"2024-04-25T08:58:47.920717+0000","last_fullsized":"2024-04-25T08:58:47.920717+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:56.102307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.547751+0000","last_change":"2024-04-25T08:58:46.556812+0000","last_active":"2024-04-25T08:58:47.547751+0000","last_peered":"2024-04-25T08:58:47.547751+0000","last_clean":"2024-04-25T08:58:47.547751+0000","last_became_active":"2024-04-25T08:58:46.556518+0000","last_became_peered":"2024-04-25T08:58:46.556518+0000","last_unstale":"2024-04-25T08:58:47.547751+0000","last_undegraded":"2024-04-25T08:58:47.547751+0000","last_fullsized":"2024-04-25T08:58:47.547751+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:20.756748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.547756+0000","last_change":"2024-04-25T08:58:45.553821+0000","last_active":"2024-04-25T08:58:47.547756+0000","last_peered":"2024-04-25T08:58:47.547756+0000","last_clean":"2024-04-25T08:58:47.547756+0000","last_became_active":"2024-04-25T08:58:45.553473+0000","last_became_peered":"2024-04-25T08:58:45.553473+0000","last_unstale":"2024-04-25T08:58:47.547756+0000","last_undegraded":"2024-04-25T08:58:47.547756+0000","last_fullsized":"2024-04-25T08:58:47.547756+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:37.384269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921249+0000","last_change":"2024-04-25T08:58:46.557743+0000","last_active":"2024-04-25T08:58:47.921249+0000","last_peered":"2024-04-25T08:58:47.921249+0000","last_clean":"2024-04-25T08:58:47.921249+0000","last_became_active":"2024-04-25T08:58:46.557454+0000","last_became_peered":"2024-04-25T08:58:46.557454+0000","last_unstale":"2024-04-25T08:58:47.921249+0000","last_undegraded":"2024-04-25T08:58:47.921249+0000","last_fullsized":"2024-04-25T08:58:47.921249+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:16.939892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"72'2","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.548552+0000","last_change":"2024-04-25T08:58:45.549705+0000","last_active":"2024-04-25T08:58:47.548552+0000","last_peered":"2024-04-25T08:58:47.548552+0000","last_clean":"2024-04-25T08:58:47.548552+0000","last_became_active":"2024-04-25T08:58:45.547577+0000","last_became_peered":"2024-04-25T08:58:45.547577+0000","last_unstale":"2024-04-25T08:58:47.548552+0000","last_undegraded":"2024-04-25T08:58:47.548552+0000","last_fullsized":"2024-04-25T08:58:47.548552+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:33.945526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":267,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.547576+0000","last_change":"2024-04-25T08:54:00.818044+0000","last_active":"2024-04-25T08:58:47.547576+0000","last_peered":"2024-04-25T08:58:47.547576+0000","last_clean":"2024-04-25T08:58:47.547576+0000","last_became_active":"2024-04-25T08:54:00.817098+0000","last_became_peered":"2024-04-25T08:54:00.817098+0000","last_unstale":"2024-04-25T08:58:47.547576+0000","last_undegraded":"2024-04-25T08:58:47.547576+0000","last_fullsized":"2024-04-25T08:58:47.547576+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_clean_scrub_stamp":"2024-04-25T08:53:59.739677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:48.129513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574908,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40956,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330884,"statfs":{"total":96636764160,"available":96594825216,"internally_reserved":0,"allocated":1490944,"data_stored":969990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574909,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40384,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331456,"statfs":{"total":96636764160,"available":96595410944,"internally_reserved":0,"allocated":897024,"data_stored":379098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574911,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40372,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331468,"statfs":{"total":96636764160,"available":96595423232,"internally_reserved":0,"allocated":892928,"data_stored":379096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574909,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45940,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325900,"statfs":{"total":96636764160,"available":96589721600,"internally_reserved":0,"allocated":892928,"data_stored":379076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574909,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58412,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313428,"statfs":{"total":96636764160,"available":96576950272,"internally_reserved":0,"allocated":884736,"data_stored":372818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574909,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40380,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331460,"statfs":{"total":96636764160,"available":96595415040,"internally_reserved":0,"allocated":901120,"data_stored":380618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574909,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50116,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321724,"statfs":{"total":96636764160,"available":96585445376,"internally_reserved":0,"allocated":909312,"data_stored":381234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574910,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28476,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343364,"statfs":{"total":96636764160,"available":96607604736,"internally_reserved":0,"allocated":1482752,"data_stored":969374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:58:50.543 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T08:58:50.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.690+0000 7fc72b7d5640 1 Processor -- start 2024-04-25T08:58:50.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.691+0000 7fc72b7d5640 1 -- start start 2024-04-25T08:58:50.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.691+0000 7fc72b7d5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7240cff70 0x7fc7240cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:50.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.691+0000 7fc72b7d5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7240ccf10 con 0x7fc72415f180 2024-04-25T08:58:50.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.691+0000 7fc72b7d5640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7240cd1f0 con 0x7fc7240b23f0 2024-04-25T08:58:50.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.691+0000 7fc72b7d5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7240cccc0 con 0x7fc7240cff70 2024-04-25T08:58:50.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.691+0000 7fc729571640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc72415f180 0x7fc7240cf860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45290/0 (socket says 172.21.15.43:45290) 2024-04-25T08:58:50.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.691+0000 7fc729571640 1 -- 172.21.15.43:0/3707554613 learned_addr learned my addr 172.21.15.43:0/3707554613 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:50.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.692+0000 7fc728d70640 1 --2- 172.21.15.43:0/3707554613 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7240cff70 0x7fc7240cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:50.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.692+0000 7fc71bfff640 1 -- 172.21.15.43:0/3707554613 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 273685071 0 0) 0x7fc7240ccf10 con 0x7fc72415f180 2024-04-25T08:58:50.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.692+0000 7fc71bfff640 1 -- 172.21.15.43:0/3707554613 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc710003520 con 0x7fc72415f180 2024-04-25T08:58:50.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.692+0000 7fc71bfff640 1 -- 172.21.15.43:0/3707554613 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3172824578 0 0) 0x7fc710003520 con 0x7fc72415f180 2024-04-25T08:58:50.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.692+0000 7fc71bfff640 1 -- 172.21.15.43:0/3707554613 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7240cff70 msgr2=0x7fc7240cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:50.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.692+0000 7fc71bfff640 1 --2- 172.21.15.43:0/3707554613 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7240cff70 0x7fc7240cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.692+0000 7fc71bfff640 1 -- 172.21.15.43:0/3707554613 >> v1:172.21.15.193:6789/0 conn(0x7fc7240b23f0 legacy=0x7fc7240ce210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:50.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.692+0000 7fc71bfff640 1 -- 172.21.15.43:0/3707554613 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7240cd4d0 con 0x7fc72415f180 2024-04-25T08:58:50.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.692+0000 7fc71bfff640 1 -- 172.21.15.43:0/3707554613 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7fc71c001670 con 0x7fc72415f180 2024-04-25T08:58:50.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.692+0000 7fc71bfff640 1 -- 172.21.15.43:0/3707554613 >> v1:172.21.15.43:6789/0 conn(0x7fc72415f180 legacy=0x7fc7240cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:50.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.692+0000 7fc71bfff640 1 --2- 172.21.15.43:0/3707554613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7100039e0 0x7fc710003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:50.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.692+0000 7fc71bfff640 1 --2- 172.21.15.43:0/3707554613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc710004800 0x7fc710004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:50.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.692+0000 7fc71bfff640 1 --2- 172.21.15.43:0/3707554613 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7240cff70 0x7fc710005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:50.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.692+0000 7fc71bfff640 1 -- 172.21.15.43:0/3707554613 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7240cccc0 con 0x7fc7100039e0 2024-04-25T08:58:50.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.692+0000 7fc71bfff640 1 -- 172.21.15.43:0/3707554613 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc71000baf0 con 0x7fc710004800 2024-04-25T08:58:50.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.692+0000 7fc71bfff640 1 -- 172.21.15.43:0/3707554613 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc71000bcf0 con 0x7fc7240cff70 2024-04-25T08:58:50.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.692+0000 7fc729571640 1 --2- 172.21.15.43:0/3707554613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7100039e0 0x7fc710003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:50.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.692+0000 7fc728d70640 1 --2- 172.21.15.43:0/3707554613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc710004800 0x7fc710004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:50.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.692+0000 7fc729d72640 1 --2- 172.21.15.43:0/3707554613 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7240cff70 0x7fc710005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:50.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.693+0000 7fc729571640 1 -- 172.21.15.43:0/3707554613 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7240cff70 msgr2=0x7fc710005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:50.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.693+0000 7fc729571640 1 --2- 172.21.15.43:0/3707554613 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7240cff70 0x7fc710005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.693+0000 7fc729571640 1 -- 172.21.15.43:0/3707554613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc710004800 msgr2=0x7fc710004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:50.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.693+0000 7fc729571640 1 --2- 172.21.15.43:0/3707554613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc710004800 0x7fc710004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.693+0000 7fc729571640 1 -- 172.21.15.43:0/3707554613 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc710003520 con 0x7fc7100039e0 2024-04-25T08:58:50.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.693+0000 7fc729571640 1 --2- 172.21.15.43:0/3707554613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7100039e0 0x7fc710003e00 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fc71c00c240 tx=0x7fc710005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dbe0c5fd94513e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:50.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.693+0000 7fc71bfff640 1 -- 172.21.15.43:0/3707554613 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71c007cf0 con 0x7fc7100039e0 2024-04-25T08:58:50.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.693+0000 7fc71bfff640 1 -- 172.21.15.43:0/3707554613 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc71c0058d0 con 0x7fc7100039e0 2024-04-25T08:58:50.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.693+0000 7fc71bfff640 1 -- 172.21.15.43:0/3707554613 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71c005ce0 con 0x7fc7100039e0 2024-04-25T08:58:50.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.694+0000 7fc72b7d5640 1 -- 172.21.15.43:0/3707554613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7100039e0 msgr2=0x7fc710003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:50.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.694+0000 7fc72b7d5640 1 --2- 172.21.15.43:0/3707554613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7100039e0 0x7fc710003e00 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fc71c00c240 tx=0x7fc710005830 comp rx=0 tx=0).stop 2024-04-25T08:58:50.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.694+0000 7fc729d72640 1 -- 172.21.15.43:0/3707554613 reap_dead start 2024-04-25T08:58:50.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.694+0000 7fc72b7d5640 1 -- 172.21.15.43:0/3707554613 shutdown_connections 2024-04-25T08:58:50.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.694+0000 7fc72b7d5640 1 -- 172.21.15.43:0/3707554613 >> 172.21.15.43:0/3707554613 conn(0x7fc7240d17f0 msgr2=0x7fc7240d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:50.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.694+0000 7fc72b7d5640 1 -- 172.21.15.43:0/3707554613 shutdown_connections 2024-04-25T08:58:50.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.694+0000 7fc72b7d5640 1 -- 172.21.15.43:0/3707554613 wait complete. 2024-04-25T08:58:50.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.695+0000 7fc72b7d5640 1 Processor -- start 2024-04-25T08:58:50.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.695+0000 7fc72b7d5640 1 -- start start 2024-04-25T08:58:50.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.696+0000 7fc72b7d5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7100039e0 0x7fc72414cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:50.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.696+0000 7fc72b7d5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7240c9790 0x7fc72406f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:50.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.696+0000 7fc72b7d5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc72406f5a0 0x7fc72415d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:50.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.696+0000 7fc72b7d5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7240d23f0 con 0x7fc72406f5a0 2024-04-25T08:58:50.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.696+0000 7fc72b7d5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7240cd840 con 0x7fc7100039e0 2024-04-25T08:58:50.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.696+0000 7fc72b7d5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7240d2570 con 0x7fc7240c9790 2024-04-25T08:58:50.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.696+0000 7fc729d72640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc72406f5a0 0x7fc72415d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:50.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.696+0000 7fc729d72640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc72406f5a0 0x7fc72415d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49082/0 (socket says 172.21.15.43:49082) 2024-04-25T08:58:50.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.696+0000 7fc729d72640 1 -- 172.21.15.43:0/3435733722 learned_addr learned my addr 172.21.15.43:0/3435733722 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:50.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.696+0000 7fc729d72640 1 -- 172.21.15.43:0/3435733722 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7240c9790 msgr2=0x7fc72406f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:50.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.696+0000 7fc728d70640 1 --2- 172.21.15.43:0/3435733722 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7240c9790 0x7fc72406f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:50.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.696+0000 7fc729d72640 1 --2- 172.21.15.43:0/3435733722 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7240c9790 0x7fc72406f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.696+0000 7fc729d72640 1 -- 172.21.15.43:0/3435733722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7100039e0 msgr2=0x7fc72414cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:50.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.696+0000 7fc729571640 1 --2- 172.21.15.43:0/3435733722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7100039e0 0x7fc72414cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:50.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.696+0000 7fc729d72640 1 --2- 172.21.15.43:0/3435733722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7100039e0 0x7fc72414cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.696+0000 7fc729d72640 1 -- 172.21.15.43:0/3435733722 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc72415e190 con 0x7fc72406f5a0 2024-04-25T08:58:50.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.697+0000 7fc729d72640 1 --2- 172.21.15.43:0/3435733722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc72406f5a0 0x7fc72415d510 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fc7240cbf80 tx=0x7fc72000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:50.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.697+0000 7fc719ffb640 1 -- 172.21.15.43:0/3435733722 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc720003970 con 0x7fc72406f5a0 2024-04-25T08:58:50.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.697+0000 7fc719ffb640 1 -- 172.21.15.43:0/3435733722 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc720018070 con 0x7fc72406f5a0 2024-04-25T08:58:50.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.697+0000 7fc72b7d5640 1 -- 172.21.15.43:0/3435733722 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7240c89d0 con 0x7fc72406f5a0 2024-04-25T08:58:50.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.697+0000 7fc719ffb640 1 -- 172.21.15.43:0/3435733722 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc720003d50 con 0x7fc72406f5a0 2024-04-25T08:58:50.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.697+0000 7fc72b7d5640 1 -- 172.21.15.43:0/3435733722 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7240c8f90 con 0x7fc72406f5a0 2024-04-25T08:58:50.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.698+0000 7fc72b7d5640 1 -- 172.21.15.43:0/3435733722 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7240cf050 con 0x7fc72406f5a0 2024-04-25T08:58:50.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.698+0000 7fc719ffb640 1 -- 172.21.15.43:0/3435733722 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc7200050d0 con 0x7fc72406f5a0 2024-04-25T08:58:50.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.699+0000 7fc719ffb640 1 --2- 172.21.15.43:0/3435733722 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc70806b4d0 0x7fc70806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:50.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.699+0000 7fc719ffb640 1 -- 172.21.15.43:0/3435733722 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc720089fb0 con 0x7fc72406f5a0 2024-04-25T08:58:50.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.699+0000 7fc729571640 1 --2- 172.21.15.43:0/3435733722 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc70806b4d0 0x7fc70806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:50.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.700+0000 7fc729571640 1 --2- 172.21.15.43:0/3435733722 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc70806b4d0 0x7fc70806d990 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc71c033000 tx=0x7fc71c009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:50.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.703+0000 7fc719ffb640 1 -- 172.21.15.43:0/3435733722 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc720058d90 con 0x7fc72406f5a0 2024-04-25T08:58:50.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.880+0000 7fc72b7d5640 1 -- 172.21.15.43:0/3435733722 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc724073010 con 0x7fc70806b4d0 2024-04-25T08:58:50.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.887+0000 7fc719ffb640 1 -- 172.21.15.43:0/3435733722 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252067 (secure 0 0 0) 0x7fc724073010 con 0x7fc70806b4d0 2024-04-25T08:58:50.879 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:58:50.880 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-25T08:58:50.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.890+0000 7fc72b7d5640 1 -- 172.21.15.43:0/3435733722 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc70806b4d0 msgr2=0x7fc70806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:50.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.890+0000 7fc72b7d5640 1 --2- 172.21.15.43:0/3435733722 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc70806b4d0 0x7fc70806d990 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc71c033000 tx=0x7fc71c009030 comp rx=0 tx=0).stop 2024-04-25T08:58:50.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.890+0000 7fc72b7d5640 1 -- 172.21.15.43:0/3435733722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc72406f5a0 msgr2=0x7fc72415d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:50.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.890+0000 7fc72b7d5640 1 --2- 172.21.15.43:0/3435733722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc72406f5a0 0x7fc72415d510 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fc7240cbf80 tx=0x7fc72000ec10 comp rx=0 tx=0).stop 2024-04-25T08:58:50.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.890+0000 7fc72b7d5640 1 -- 172.21.15.43:0/3435733722 shutdown_connections 2024-04-25T08:58:50.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.890+0000 7fc72b7d5640 1 --2- 172.21.15.43:0/3435733722 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc72406f5a0 0x7fc72415d510 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.890+0000 7fc72b7d5640 1 --2- 172.21.15.43:0/3435733722 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7240c9790 0x7fc72406f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.890+0000 7fc72b7d5640 1 --2- 172.21.15.43:0/3435733722 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc70806b4d0 0x7fc70806d990 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.890+0000 7fc72b7d5640 1 --2- 172.21.15.43:0/3435733722 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7100039e0 0x7fc72414cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:50.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.890+0000 7fc72b7d5640 1 -- 172.21.15.43:0/3435733722 >> 172.21.15.43:0/3435733722 conn(0x7fc7240d17f0 msgr2=0x7fc7240b43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:50.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.890+0000 7fc72b7d5640 1 -- 172.21.15.43:0/3435733722 shutdown_connections 2024-04-25T08:58:50.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:50.890+0000 7fc72b7d5640 1 -- 172.21.15.43:0/3435733722 wait complete. 2024-04-25T08:58:50.890 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2024-04-25T08:58:49.760135+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":162,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":345036,"kb_used_data":8156,"kb_used_omap":12,"kb_used_meta":336755,"kb_avail":754629684,"statfs":{"total":773094113280,"available":772740796416,"internally_reserved":0,"allocated":8351744,"data_stored":4211304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":344837681},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.232420"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.548312+0000","last_change":"2024-04-25T08:58:46.560577+0000","last_active":"2024-04-25T08:58:47.548312+0000","last_peered":"2024-04-25T08:58:47.548312+0000","last_clean":"2024-04-25T08:58:47.548312+0000","last_became_active":"2024-04-25T08:58:46.560305+0000","last_became_peered":"2024-04-25T08:58:46.560305+0000","last_unstale":"2024-04-25T08:58:47.548312+0000","last_undegraded":"2024-04-25T08:58:47.548312+0000","last_fullsized":"2024-04-25T08:58:47.548312+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:55.762916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.549167+0000","last_change":"2024-04-25T08:58:46.551585+0000","last_active":"2024-04-25T08:58:47.549167+0000","last_peered":"2024-04-25T08:58:47.549167+0000","last_clean":"2024-04-25T08:58:47.549167+0000","last_became_active":"2024-04-25T08:58:46.551290+0000","last_became_peered":"2024-04-25T08:58:46.551290+0000","last_unstale":"2024-04-25T08:58:47.549167+0000","last_undegraded":"2024-04-25T08:58:47.549167+0000","last_fullsized":"2024-04-25T08:58:47.549167+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:00.887733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921429+0000","last_change":"2024-04-25T08:58:46.555701+0000","last_active":"2024-04-25T08:58:47.921429+0000","last_peered":"2024-04-25T08:58:47.921429+0000","last_clean":"2024-04-25T08:58:47.921429+0000","last_became_active":"2024-04-25T08:58:46.555335+0000","last_became_peered":"2024-04-25T08:58:46.555335+0000","last_unstale":"2024-04-25T08:58:47.921429+0000","last_undegraded":"2024-04-25T08:58:47.921429+0000","last_fullsized":"2024-04-25T08:58:47.921429+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:10:21.823279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.546632+0000","last_change":"2024-04-25T08:58:46.562436+0000","last_active":"2024-04-25T08:58:47.546632+0000","last_peered":"2024-04-25T08:58:47.546632+0000","last_clean":"2024-04-25T08:58:47.546632+0000","last_became_active":"2024-04-25T08:58:46.561139+0000","last_became_peered":"2024-04-25T08:58:46.561139+0000","last_unstale":"2024-04-25T08:58:47.546632+0000","last_undegraded":"2024-04-25T08:58:47.546632+0000","last_fullsized":"2024-04-25T08:58:47.546632+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:46.584584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921286+0000","last_change":"2024-04-25T08:58:46.553792+0000","last_active":"2024-04-25T08:58:47.921286+0000","last_peered":"2024-04-25T08:58:47.921286+0000","last_clean":"2024-04-25T08:58:47.921286+0000","last_became_active":"2024-04-25T08:58:46.553422+0000","last_became_peered":"2024-04-25T08:58:46.553422+0000","last_unstale":"2024-04-25T08:58:47.921286+0000","last_undegraded":"2024-04-25T08:58:47.921286+0000","last_fullsized":"2024-04-25T08:58:47.921286+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:56:49.454085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.548920+0000","last_change":"2024-04-25T08:58:46.552544+0000","last_active":"2024-04-25T08:58:47.548920+0000","last_peered":"2024-04-25T08:58:47.548920+0000","last_clean":"2024-04-25T08:58:47.548920+0000","last_became_active":"2024-04-25T08:58:46.551662+0000","last_became_peered":"2024-04-25T08:58:46.551662+0000","last_unstale":"2024-04-25T08:58:47.548920+0000","last_undegraded":"2024-04-25T08:58:47.548920+0000","last_fullsized":"2024-04-25T08:58:47.548920+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:32.474473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.554161+0000","last_change":"2024-04-25T08:58:46.550264+0000","last_active":"2024-04-25T08:58:47.554161+0000","last_peered":"2024-04-25T08:58:47.554161+0000","last_clean":"2024-04-25T08:58:47.554161+0000","last_became_active":"2024-04-25T08:58:46.549996+0000","last_became_peered":"2024-04-25T08:58:46.549996+0000","last_unstale":"2024-04-25T08:58:47.554161+0000","last_undegraded":"2024-04-25T08:58:47.554161+0000","last_fullsized":"2024-04-25T08:58:47.554161+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:51:31.390831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.547613+0000","last_change":"2024-04-25T08:58:46.556863+0000","last_active":"2024-04-25T08:58:47.547613+0000","last_peered":"2024-04-25T08:58:47.547613+0000","last_clean":"2024-04-25T08:58:47.547613+0000","last_became_active":"2024-04-25T08:58:46.556554+0000","last_became_peered":"2024-04-25T08:58:46.556554+0000","last_unstale":"2024-04-25T08:58:47.547613+0000","last_undegraded":"2024-04-25T08:58:47.547613+0000","last_fullsized":"2024-04-25T08:58:47.547613+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:56.222512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921237+0000","last_change":"2024-04-25T08:58:46.559636+0000","last_active":"2024-04-25T08:58:47.921237+0000","last_peered":"2024-04-25T08:58:47.921237+0000","last_clean":"2024-04-25T08:58:47.921237+0000","last_became_active":"2024-04-25T08:58:46.559095+0000","last_became_peered":"2024-04-25T08:58:46.559095+0000","last_unstale":"2024-04-25T08:58:47.921237+0000","last_undegraded":"2024-04-25T08:58:47.921237+0000","last_fullsized":"2024-04-25T08:58:47.921237+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:00.656437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921218+0000","last_change":"2024-04-25T08:58:46.555239+0000","last_active":"2024-04-25T08:58:47.921218+0000","last_peered":"2024-04-25T08:58:47.921218+0000","last_clean":"2024-04-25T08:58:47.921218+0000","last_became_active":"2024-04-25T08:58:46.554653+0000","last_became_peered":"2024-04-25T08:58:46.554653+0000","last_unstale":"2024-04-25T08:58:47.921218+0000","last_undegraded":"2024-04-25T08:58:47.921218+0000","last_fullsized":"2024-04-25T08:58:47.921218+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:05.982057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.548719+0000","last_change":"2024-04-25T08:58:46.555570+0000","last_active":"2024-04-25T08:58:47.548719+0000","last_peered":"2024-04-25T08:58:47.548719+0000","last_clean":"2024-04-25T08:58:47.548719+0000","last_became_active":"2024-04-25T08:58:46.555301+0000","last_became_peered":"2024-04-25T08:58:46.555301+0000","last_unstale":"2024-04-25T08:58:47.548719+0000","last_undegraded":"2024-04-25T08:58:47.548719+0000","last_fullsized":"2024-04-25T08:58:47.548719+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:01.993830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.546801+0000","last_change":"2024-04-25T08:58:46.552337+0000","last_active":"2024-04-25T08:58:47.546801+0000","last_peered":"2024-04-25T08:58:47.546801+0000","last_clean":"2024-04-25T08:58:47.546801+0000","last_became_active":"2024-04-25T08:58:46.552004+0000","last_became_peered":"2024-04-25T08:58:46.552004+0000","last_unstale":"2024-04-25T08:58:47.546801+0000","last_undegraded":"2024-04-25T08:58:47.546801+0000","last_fullsized":"2024-04-25T08:58:47.546801+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:54.601317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.556458+0000","last_change":"2024-04-25T08:58:46.560175+0000","last_active":"2024-04-25T08:58:47.556458+0000","last_peered":"2024-04-25T08:58:47.556458+0000","last_clean":"2024-04-25T08:58:47.556458+0000","last_became_active":"2024-04-25T08:58:46.559340+0000","last_became_peered":"2024-04-25T08:58:46.559340+0000","last_unstale":"2024-04-25T08:58:47.556458+0000","last_undegraded":"2024-04-25T08:58:47.556458+0000","last_fullsized":"2024-04-25T08:58:47.556458+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:52.919994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921053+0000","last_change":"2024-04-25T08:58:46.551530+0000","last_active":"2024-04-25T08:58:47.921053+0000","last_peered":"2024-04-25T08:58:47.921053+0000","last_clean":"2024-04-25T08:58:47.921053+0000","last_became_active":"2024-04-25T08:58:46.551083+0000","last_became_peered":"2024-04-25T08:58:46.551083+0000","last_unstale":"2024-04-25T08:58:47.921053+0000","last_undegraded":"2024-04-25T08:58:47.921053+0000","last_fullsized":"2024-04-25T08:58:47.921053+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T08:59:24.787164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.556337+0000","last_change":"2024-04-25T08:58:46.549615+0000","last_active":"2024-04-25T08:58:47.556337+0000","last_peered":"2024-04-25T08:58:47.556337+0000","last_clean":"2024-04-25T08:58:47.556337+0000","last_became_active":"2024-04-25T08:58:46.549293+0000","last_became_peered":"2024-04-25T08:58:46.549293+0000","last_unstale":"2024-04-25T08:58:47.556337+0000","last_undegraded":"2024-04-25T08:58:47.556337+0000","last_fullsized":"2024-04-25T08:58:47.556337+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:44.144300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.920741+0000","last_change":"2024-04-25T08:58:46.558943+0000","last_active":"2024-04-25T08:58:47.920741+0000","last_peered":"2024-04-25T08:58:47.920741+0000","last_clean":"2024-04-25T08:58:47.920741+0000","last_became_active":"2024-04-25T08:58:46.558650+0000","last_became_peered":"2024-04-25T08:58:46.558650+0000","last_unstale":"2024-04-25T08:58:47.920741+0000","last_undegraded":"2024-04-25T08:58:47.920741+0000","last_fullsized":"2024-04-25T08:58:47.920741+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:13.676463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921388+0000","last_change":"2024-04-25T08:58:46.561747+0000","last_active":"2024-04-25T08:58:47.921388+0000","last_peered":"2024-04-25T08:58:47.921388+0000","last_clean":"2024-04-25T08:58:47.921388+0000","last_became_active":"2024-04-25T08:58:46.558281+0000","last_became_peered":"2024-04-25T08:58:46.558281+0000","last_unstale":"2024-04-25T08:58:47.921388+0000","last_undegraded":"2024-04-25T08:58:47.921388+0000","last_fullsized":"2024-04-25T08:58:47.921388+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:53.372275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.546758+0000","last_change":"2024-04-25T08:58:46.553711+0000","last_active":"2024-04-25T08:58:47.546758+0000","last_peered":"2024-04-25T08:58:47.546758+0000","last_clean":"2024-04-25T08:58:47.546758+0000","last_became_active":"2024-04-25T08:58:46.553355+0000","last_became_peered":"2024-04-25T08:58:46.553355+0000","last_unstale":"2024-04-25T08:58:47.546758+0000","last_undegraded":"2024-04-25T08:58:47.546758+0000","last_fullsized":"2024-04-25T08:58:47.546758+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:56.517471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.548585+0000","last_change":"2024-04-25T08:58:46.557573+0000","last_active":"2024-04-25T08:58:47.548585+0000","last_peered":"2024-04-25T08:58:47.548585+0000","last_clean":"2024-04-25T08:58:47.548585+0000","last_became_active":"2024-04-25T08:58:46.557020+0000","last_became_peered":"2024-04-25T08:58:46.557020+0000","last_unstale":"2024-04-25T08:58:47.548585+0000","last_undegraded":"2024-04-25T08:58:47.548585+0000","last_fullsized":"2024-04-25T08:58:47.548585+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:45.757920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.556459+0000","last_change":"2024-04-25T08:58:46.560977+0000","last_active":"2024-04-25T08:58:47.556459+0000","last_peered":"2024-04-25T08:58:47.556459+0000","last_clean":"2024-04-25T08:58:47.556459+0000","last_became_active":"2024-04-25T08:58:46.560672+0000","last_became_peered":"2024-04-25T08:58:46.560672+0000","last_unstale":"2024-04-25T08:58:47.556459+0000","last_undegraded":"2024-04-25T08:58:47.556459+0000","last_fullsized":"2024-04-25T08:58:47.556459+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:33.116719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.920983+0000","last_change":"2024-04-25T08:58:46.555160+0000","last_active":"2024-04-25T08:58:47.920983+0000","last_peered":"2024-04-25T08:58:47.920983+0000","last_clean":"2024-04-25T08:58:47.920983+0000","last_became_active":"2024-04-25T08:58:46.554563+0000","last_became_peered":"2024-04-25T08:58:46.554563+0000","last_unstale":"2024-04-25T08:58:47.920983+0000","last_undegraded":"2024-04-25T08:58:47.920983+0000","last_fullsized":"2024-04-25T08:58:47.920983+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:49.915188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.920829+0000","last_change":"2024-04-25T08:58:46.550878+0000","last_active":"2024-04-25T08:58:47.920829+0000","last_peered":"2024-04-25T08:58:47.920829+0000","last_clean":"2024-04-25T08:58:47.920829+0000","last_became_active":"2024-04-25T08:58:46.550530+0000","last_became_peered":"2024-04-25T08:58:46.550530+0000","last_unstale":"2024-04-25T08:58:47.920829+0000","last_undegraded":"2024-04-25T08:58:47.920829+0000","last_fullsized":"2024-04-25T08:58:47.920829+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:56.222512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.547762+0000","last_change":"2024-04-25T08:58:46.546951+0000","last_active":"2024-04-25T08:58:47.547762+0000","last_peered":"2024-04-25T08:58:47.547762+0000","last_clean":"2024-04-25T08:58:47.547762+0000","last_became_active":"2024-04-25T08:58:46.546687+0000","last_became_peered":"2024-04-25T08:58:46.546687+0000","last_unstale":"2024-04-25T08:58:47.547762+0000","last_undegraded":"2024-04-25T08:58:47.547762+0000","last_fullsized":"2024-04-25T08:58:47.547762+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:20.996902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.547512+0000","last_change":"2024-04-25T08:58:46.561556+0000","last_active":"2024-04-25T08:58:47.547512+0000","last_peered":"2024-04-25T08:58:47.547512+0000","last_clean":"2024-04-25T08:58:47.547512+0000","last_became_active":"2024-04-25T08:58:46.561244+0000","last_became_peered":"2024-04-25T08:58:46.561244+0000","last_unstale":"2024-04-25T08:58:47.547512+0000","last_undegraded":"2024-04-25T08:58:47.547512+0000","last_fullsized":"2024-04-25T08:58:47.547512+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:13.640828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921166+0000","last_change":"2024-04-25T08:58:46.554691+0000","last_active":"2024-04-25T08:58:47.921166+0000","last_peered":"2024-04-25T08:58:47.921166+0000","last_clean":"2024-04-25T08:58:47.921166+0000","last_became_active":"2024-04-25T08:58:46.554410+0000","last_became_peered":"2024-04-25T08:58:46.554410+0000","last_unstale":"2024-04-25T08:58:47.921166+0000","last_undegraded":"2024-04-25T08:58:47.921166+0000","last_fullsized":"2024-04-25T08:58:47.921166+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:00.061522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.554524+0000","last_change":"2024-04-25T08:58:46.559256+0000","last_active":"2024-04-25T08:58:47.554524+0000","last_peered":"2024-04-25T08:58:47.554524+0000","last_clean":"2024-04-25T08:58:47.554524+0000","last_became_active":"2024-04-25T08:58:46.558803+0000","last_became_peered":"2024-04-25T08:58:46.558803+0000","last_unstale":"2024-04-25T08:58:47.554524+0000","last_undegraded":"2024-04-25T08:58:47.554524+0000","last_fullsized":"2024-04-25T08:58:47.554524+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:33.116719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921567+0000","last_change":"2024-04-25T08:58:46.548248+0000","last_active":"2024-04-25T08:58:47.921567+0000","last_peered":"2024-04-25T08:58:47.921567+0000","last_clean":"2024-04-25T08:58:47.921567+0000","last_became_active":"2024-04-25T08:58:46.547960+0000","last_became_peered":"2024-04-25T08:58:46.547960+0000","last_unstale":"2024-04-25T08:58:47.921567+0000","last_undegraded":"2024-04-25T08:58:47.921567+0000","last_fullsized":"2024-04-25T08:58:47.921567+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:11:44.139340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921020+0000","last_change":"2024-04-25T08:58:46.550792+0000","last_active":"2024-04-25T08:58:47.921020+0000","last_peered":"2024-04-25T08:58:47.921020+0000","last_clean":"2024-04-25T08:58:47.921020+0000","last_became_active":"2024-04-25T08:58:46.550508+0000","last_became_peered":"2024-04-25T08:58:46.550508+0000","last_unstale":"2024-04-25T08:58:47.921020+0000","last_undegraded":"2024-04-25T08:58:47.921020+0000","last_fullsized":"2024-04-25T08:58:47.921020+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:20.756748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921078+0000","last_change":"2024-04-25T08:58:46.555307+0000","last_active":"2024-04-25T08:58:47.921078+0000","last_peered":"2024-04-25T08:58:47.921078+0000","last_clean":"2024-04-25T08:58:47.921078+0000","last_became_active":"2024-04-25T08:58:46.554992+0000","last_became_peered":"2024-04-25T08:58:46.554992+0000","last_unstale":"2024-04-25T08:58:47.921078+0000","last_undegraded":"2024-04-25T08:58:47.921078+0000","last_fullsized":"2024-04-25T08:58:47.921078+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:29:19.756407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.554290+0000","last_change":"2024-04-25T08:58:46.554809+0000","last_active":"2024-04-25T08:58:47.554290+0000","last_peered":"2024-04-25T08:58:47.554290+0000","last_clean":"2024-04-25T08:58:47.554290+0000","last_became_active":"2024-04-25T08:58:46.554526+0000","last_became_peered":"2024-04-25T08:58:46.554526+0000","last_unstale":"2024-04-25T08:58:47.554290+0000","last_undegraded":"2024-04-25T08:58:47.554290+0000","last_fullsized":"2024-04-25T08:58:47.554290+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:45.020761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.554464+0000","last_change":"2024-04-25T08:58:46.551042+0000","last_active":"2024-04-25T08:58:47.554464+0000","last_peered":"2024-04-25T08:58:47.554464+0000","last_clean":"2024-04-25T08:58:47.554464+0000","last_became_active":"2024-04-25T08:58:46.550738+0000","last_became_peered":"2024-04-25T08:58:46.550738+0000","last_unstale":"2024-04-25T08:58:47.554464+0000","last_undegraded":"2024-04-25T08:58:47.554464+0000","last_fullsized":"2024-04-25T08:58:47.554464+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:18.357607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.547669+0000","last_change":"2024-04-25T08:58:46.552873+0000","last_active":"2024-04-25T08:58:47.547669+0000","last_peered":"2024-04-25T08:58:47.547669+0000","last_clean":"2024-04-25T08:58:47.547669+0000","last_became_active":"2024-04-25T08:58:46.551313+0000","last_became_peered":"2024-04-25T08:58:46.551313+0000","last_unstale":"2024-04-25T08:58:47.547669+0000","last_undegraded":"2024-04-25T08:58:47.547669+0000","last_fullsized":"2024-04-25T08:58:47.547669+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:49.915188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.548456+0000","last_change":"2024-04-25T08:58:46.561672+0000","last_active":"2024-04-25T08:58:47.548456+0000","last_peered":"2024-04-25T08:58:47.548456+0000","last_clean":"2024-04-25T08:58:47.548456+0000","last_became_active":"2024-04-25T08:58:46.561384+0000","last_became_peered":"2024-04-25T08:58:46.561384+0000","last_unstale":"2024-04-25T08:58:47.548456+0000","last_undegraded":"2024-04-25T08:58:47.548456+0000","last_fullsized":"2024-04-25T08:58:47.548456+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:56.419286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"72'4","reported_seq":18,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.549389+0000","last_change":"2024-04-25T08:58:45.552209+0000","last_active":"2024-04-25T08:58:47.549389+0000","last_peered":"2024-04-25T08:58:47.549389+0000","last_clean":"2024-04-25T08:58:47.549389+0000","last_became_active":"2024-04-25T08:58:45.551711+0000","last_became_peered":"2024-04-25T08:58:45.551711+0000","last_unstale":"2024-04-25T08:58:47.549389+0000","last_undegraded":"2024-04-25T08:58:47.549389+0000","last_fullsized":"2024-04-25T08:58:47.549389+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:41:08.904655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921563+0000","last_change":"2024-04-25T08:58:46.549376+0000","last_active":"2024-04-25T08:58:47.921563+0000","last_peered":"2024-04-25T08:58:47.921563+0000","last_clean":"2024-04-25T08:58:47.921563+0000","last_became_active":"2024-04-25T08:58:46.549051+0000","last_became_peered":"2024-04-25T08:58:46.549051+0000","last_unstale":"2024-04-25T08:58:47.921563+0000","last_undegraded":"2024-04-25T08:58:47.921563+0000","last_fullsized":"2024-04-25T08:58:47.921563+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:31.951240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"72'1","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.556551+0000","last_change":"2024-04-25T08:58:45.548533+0000","last_active":"2024-04-25T08:58:47.556551+0000","last_peered":"2024-04-25T08:58:47.556551+0000","last_clean":"2024-04-25T08:58:47.556551+0000","last_became_active":"2024-04-25T08:58:45.547726+0000","last_became_peered":"2024-04-25T08:58:45.547726+0000","last_unstale":"2024-04-25T08:58:47.556551+0000","last_undegraded":"2024-04-25T08:58:47.556551+0000","last_fullsized":"2024-04-25T08:58:47.556551+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:31.469571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.548195+0000","last_change":"2024-04-25T08:58:46.557827+0000","last_active":"2024-04-25T08:58:47.548195+0000","last_peered":"2024-04-25T08:58:47.548195+0000","last_clean":"2024-04-25T08:58:47.548195+0000","last_became_active":"2024-04-25T08:58:46.557443+0000","last_became_peered":"2024-04-25T08:58:46.557443+0000","last_unstale":"2024-04-25T08:58:47.548195+0000","last_undegraded":"2024-04-25T08:58:47.548195+0000","last_fullsized":"2024-04-25T08:58:47.548195+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:26.721498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"72'2","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921076+0000","last_change":"2024-04-25T08:58:45.551223+0000","last_active":"2024-04-25T08:58:47.921076+0000","last_peered":"2024-04-25T08:58:47.921076+0000","last_clean":"2024-04-25T08:58:47.921076+0000","last_became_active":"2024-04-25T08:58:45.550909+0000","last_became_peered":"2024-04-25T08:58:45.550909+0000","last_unstale":"2024-04-25T08:58:47.921076+0000","last_undegraded":"2024-04-25T08:58:47.921076+0000","last_fullsized":"2024-04-25T08:58:47.921076+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:39.804943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.548886+0000","last_change":"2024-04-25T08:58:46.552878+0000","last_active":"2024-04-25T08:58:47.548886+0000","last_peered":"2024-04-25T08:58:47.548886+0000","last_clean":"2024-04-25T08:58:47.548886+0000","last_became_active":"2024-04-25T08:58:46.552616+0000","last_became_peered":"2024-04-25T08:58:46.552616+0000","last_unstale":"2024-04-25T08:58:47.548886+0000","last_undegraded":"2024-04-25T08:58:47.548886+0000","last_fullsized":"2024-04-25T08:58:47.548886+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:02.850873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"72'2","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.548695+0000","last_change":"2024-04-25T08:58:45.553283+0000","last_active":"2024-04-25T08:58:47.548695+0000","last_peered":"2024-04-25T08:58:47.548695+0000","last_clean":"2024-04-25T08:58:47.548695+0000","last_became_active":"2024-04-25T08:58:45.553012+0000","last_became_peered":"2024-04-25T08:58:45.553012+0000","last_unstale":"2024-04-25T08:58:47.548695+0000","last_undegraded":"2024-04-25T08:58:47.548695+0000","last_fullsized":"2024-04-25T08:58:47.548695+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:17.858568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.554350+0000","last_change":"2024-04-25T08:58:46.546837+0000","last_active":"2024-04-25T08:58:47.554350+0000","last_peered":"2024-04-25T08:58:47.554350+0000","last_clean":"2024-04-25T08:58:47.554350+0000","last_became_active":"2024-04-25T08:58:46.546582+0000","last_became_peered":"2024-04-25T08:58:46.546582+0000","last_unstale":"2024-04-25T08:58:47.554350+0000","last_undegraded":"2024-04-25T08:58:47.554350+0000","last_fullsized":"2024-04-25T08:58:47.554350+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:41.339390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"72'1","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921209+0000","last_change":"2024-04-25T08:58:45.550618+0000","last_active":"2024-04-25T08:58:47.921209+0000","last_peered":"2024-04-25T08:58:47.921209+0000","last_clean":"2024-04-25T08:58:47.921209+0000","last_became_active":"2024-04-25T08:58:45.550332+0000","last_became_peered":"2024-04-25T08:58:45.550332+0000","last_unstale":"2024-04-25T08:58:47.921209+0000","last_undegraded":"2024-04-25T08:58:47.921209+0000","last_fullsized":"2024-04-25T08:58:47.921209+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:09:01.845971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921079+0000","last_change":"2024-04-25T08:58:46.555264+0000","last_active":"2024-04-25T08:58:47.921079+0000","last_peered":"2024-04-25T08:58:47.921079+0000","last_clean":"2024-04-25T08:58:47.921079+0000","last_became_active":"2024-04-25T08:58:46.554909+0000","last_became_peered":"2024-04-25T08:58:46.554909+0000","last_unstale":"2024-04-25T08:58:47.921079+0000","last_undegraded":"2024-04-25T08:58:47.921079+0000","last_fullsized":"2024-04-25T08:58:47.921079+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:26.721498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"72'1","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.548211+0000","last_change":"2024-04-25T08:58:45.554390+0000","last_active":"2024-04-25T08:58:47.548211+0000","last_peered":"2024-04-25T08:58:47.548211+0000","last_clean":"2024-04-25T08:58:47.548211+0000","last_became_active":"2024-04-25T08:58:45.554067+0000","last_became_peered":"2024-04-25T08:58:45.554067+0000","last_unstale":"2024-04-25T08:58:47.548211+0000","last_undegraded":"2024-04-25T08:58:47.548211+0000","last_fullsized":"2024-04-25T08:58:47.548211+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:23.089008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.920981+0000","last_change":"2024-04-25T08:58:46.546861+0000","last_active":"2024-04-25T08:58:47.920981+0000","last_peered":"2024-04-25T08:58:47.920981+0000","last_clean":"2024-04-25T08:58:47.920981+0000","last_became_active":"2024-04-25T08:58:46.546555+0000","last_became_peered":"2024-04-25T08:58:46.546555+0000","last_unstale":"2024-04-25T08:58:47.920981+0000","last_undegraded":"2024-04-25T08:58:47.920981+0000","last_fullsized":"2024-04-25T08:58:47.920981+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:10.527777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.546418+0000","last_change":"2024-04-25T08:58:45.542798+0000","last_active":"2024-04-25T08:58:47.546418+0000","last_peered":"2024-04-25T08:58:47.546418+0000","last_clean":"2024-04-25T08:58:47.546418+0000","last_became_active":"2024-04-25T08:58:45.542020+0000","last_became_peered":"2024-04-25T08:58:45.542020+0000","last_unstale":"2024-04-25T08:58:47.546418+0000","last_undegraded":"2024-04-25T08:58:47.546418+0000","last_fullsized":"2024-04-25T08:58:47.546418+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:17.858568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.556386+0000","last_change":"2024-04-25T08:58:46.550466+0000","last_active":"2024-04-25T08:58:47.556386+0000","last_peered":"2024-04-25T08:58:47.556386+0000","last_clean":"2024-04-25T08:58:47.556386+0000","last_became_active":"2024-04-25T08:58:46.550099+0000","last_became_peered":"2024-04-25T08:58:46.550099+0000","last_unstale":"2024-04-25T08:58:47.556386+0000","last_undegraded":"2024-04-25T08:58:47.556386+0000","last_fullsized":"2024-04-25T08:58:47.556386+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:18.357607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.920901+0000","last_change":"2024-04-25T08:58:45.550842+0000","last_active":"2024-04-25T08:58:47.920901+0000","last_peered":"2024-04-25T08:58:47.920901+0000","last_clean":"2024-04-25T08:58:47.920901+0000","last_became_active":"2024-04-25T08:58:45.550579+0000","last_became_peered":"2024-04-25T08:58:45.550579+0000","last_unstale":"2024-04-25T08:58:47.920901+0000","last_undegraded":"2024-04-25T08:58:47.920901+0000","last_fullsized":"2024-04-25T08:58:47.920901+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:19:36.533485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.554550+0000","last_change":"2024-04-25T08:58:46.550247+0000","last_active":"2024-04-25T08:58:47.554550+0000","last_peered":"2024-04-25T08:58:47.554550+0000","last_clean":"2024-04-25T08:58:47.554550+0000","last_became_active":"2024-04-25T08:58:46.549966+0000","last_became_peered":"2024-04-25T08:58:46.549966+0000","last_unstale":"2024-04-25T08:58:47.554550+0000","last_undegraded":"2024-04-25T08:58:47.554550+0000","last_fullsized":"2024-04-25T08:58:47.554550+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:13.640828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921160+0000","last_change":"2024-04-25T08:58:45.547081+0000","last_active":"2024-04-25T08:58:47.921160+0000","last_peered":"2024-04-25T08:58:47.921160+0000","last_clean":"2024-04-25T08:58:47.921160+0000","last_became_active":"2024-04-25T08:58:45.546672+0000","last_became_peered":"2024-04-25T08:58:45.546672+0000","last_unstale":"2024-04-25T08:58:47.921160+0000","last_undegraded":"2024-04-25T08:58:47.921160+0000","last_fullsized":"2024-04-25T08:58:47.921160+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:33.945526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.549255+0000","last_change":"2024-04-25T08:58:46.560124+0000","last_active":"2024-04-25T08:58:47.549255+0000","last_peered":"2024-04-25T08:58:47.549255+0000","last_clean":"2024-04-25T08:58:47.549255+0000","last_became_active":"2024-04-25T08:58:46.559836+0000","last_became_peered":"2024-04-25T08:58:46.559836+0000","last_unstale":"2024-04-25T08:58:47.549255+0000","last_undegraded":"2024-04-25T08:58:47.549255+0000","last_fullsized":"2024-04-25T08:58:47.549255+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:20:11.068522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.549167+0000","last_change":"2024-04-25T08:58:45.553198+0000","last_active":"2024-04-25T08:58:47.549167+0000","last_peered":"2024-04-25T08:58:47.549167+0000","last_clean":"2024-04-25T08:58:47.549167+0000","last_became_active":"2024-04-25T08:58:45.552506+0000","last_became_peered":"2024-04-25T08:58:45.552506+0000","last_unstale":"2024-04-25T08:58:47.549167+0000","last_undegraded":"2024-04-25T08:58:47.549167+0000","last_fullsized":"2024-04-25T08:58:47.549167+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:56.102307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.548585+0000","last_change":"2024-04-25T08:58:46.557705+0000","last_active":"2024-04-25T08:58:47.548585+0000","last_peered":"2024-04-25T08:58:47.548585+0000","last_clean":"2024-04-25T08:58:47.548585+0000","last_became_active":"2024-04-25T08:58:46.557081+0000","last_became_peered":"2024-04-25T08:58:46.557081+0000","last_unstale":"2024-04-25T08:58:47.548585+0000","last_undegraded":"2024-04-25T08:58:47.548585+0000","last_fullsized":"2024-04-25T08:58:47.548585+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:05.982057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"72'3","reported_seq":17,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921707+0000","last_change":"2024-04-25T08:58:45.548063+0000","last_active":"2024-04-25T08:58:47.921707+0000","last_peered":"2024-04-25T08:58:47.921707+0000","last_clean":"2024-04-25T08:58:47.921707+0000","last_became_active":"2024-04-25T08:58:45.546207+0000","last_became_peered":"2024-04-25T08:58:45.546207+0000","last_unstale":"2024-04-25T08:58:47.921707+0000","last_undegraded":"2024-04-25T08:58:47.921707+0000","last_fullsized":"2024-04-25T08:58:47.921707+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:36:00.988928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921381+0000","last_change":"2024-04-25T08:58:46.551175+0000","last_active":"2024-04-25T08:58:47.921381+0000","last_peered":"2024-04-25T08:58:47.921381+0000","last_clean":"2024-04-25T08:58:47.921381+0000","last_became_active":"2024-04-25T08:58:46.550799+0000","last_became_peered":"2024-04-25T08:58:46.550799+0000","last_unstale":"2024-04-25T08:58:47.921381+0000","last_undegraded":"2024-04-25T08:58:47.921381+0000","last_fullsized":"2024-04-25T08:58:47.921381+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:58.018327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"72'6","reported_seq":20,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.546805+0000","last_change":"2024-04-25T08:58:45.547581+0000","last_active":"2024-04-25T08:58:47.546805+0000","last_peered":"2024-04-25T08:58:47.546805+0000","last_clean":"2024-04-25T08:58:47.546805+0000","last_became_active":"2024-04-25T08:58:45.547280+0000","last_became_peered":"2024-04-25T08:58:45.547280+0000","last_unstale":"2024-04-25T08:58:47.546805+0000","last_undegraded":"2024-04-25T08:58:47.546805+0000","last_fullsized":"2024-04-25T08:58:47.546805+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:33.494101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.547956+0000","last_change":"2024-04-25T08:58:46.554406+0000","last_active":"2024-04-25T08:58:47.547956+0000","last_peered":"2024-04-25T08:58:47.547956+0000","last_clean":"2024-04-25T08:58:47.547956+0000","last_became_active":"2024-04-25T08:58:46.554096+0000","last_became_peered":"2024-04-25T08:58:46.554096+0000","last_unstale":"2024-04-25T08:58:47.547956+0000","last_undegraded":"2024-04-25T08:58:47.547956+0000","last_fullsized":"2024-04-25T08:58:47.547956+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:00.935851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"72'2","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.546518+0000","last_change":"2024-04-25T08:58:45.547352+0000","last_active":"2024-04-25T08:58:47.546518+0000","last_peered":"2024-04-25T08:58:47.546518+0000","last_clean":"2024-04-25T08:58:47.546518+0000","last_became_active":"2024-04-25T08:58:45.546900+0000","last_became_peered":"2024-04-25T08:58:45.546900+0000","last_unstale":"2024-04-25T08:58:47.546518+0000","last_undegraded":"2024-04-25T08:58:47.546518+0000","last_fullsized":"2024-04-25T08:58:47.546518+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:56.102307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.554453+0000","last_change":"2024-04-25T08:58:46.542814+0000","last_active":"2024-04-25T08:58:47.554453+0000","last_peered":"2024-04-25T08:58:47.554453+0000","last_clean":"2024-04-25T08:58:47.554453+0000","last_became_active":"2024-04-25T08:58:46.542530+0000","last_became_peered":"2024-04-25T08:58:46.542530+0000","last_unstale":"2024-04-25T08:58:47.554453+0000","last_undegraded":"2024-04-25T08:58:47.554453+0000","last_fullsized":"2024-04-25T08:58:47.554453+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:44.144300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.548678+0000","last_change":"2024-04-25T08:58:45.553000+0000","last_active":"2024-04-25T08:58:47.548678+0000","last_peered":"2024-04-25T08:58:47.548678+0000","last_clean":"2024-04-25T08:58:47.548678+0000","last_became_active":"2024-04-25T08:58:45.552593+0000","last_became_peered":"2024-04-25T08:58:45.552593+0000","last_unstale":"2024-04-25T08:58:47.548678+0000","last_undegraded":"2024-04-25T08:58:47.548678+0000","last_fullsized":"2024-04-25T08:58:47.548678+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:11:30.946338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.556297+0000","last_change":"2024-04-25T08:58:46.559888+0000","last_active":"2024-04-25T08:58:47.556297+0000","last_peered":"2024-04-25T08:58:47.556297+0000","last_clean":"2024-04-25T08:58:47.556297+0000","last_became_active":"2024-04-25T08:58:46.559566+0000","last_became_peered":"2024-04-25T08:58:46.559566+0000","last_unstale":"2024-04-25T08:58:47.556297+0000","last_undegraded":"2024-04-25T08:58:47.556297+0000","last_fullsized":"2024-04-25T08:58:47.556297+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:41.339390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.554171+0000","last_change":"2024-04-25T08:58:45.542358+0000","last_active":"2024-04-25T08:58:47.554171+0000","last_peered":"2024-04-25T08:58:47.554171+0000","last_clean":"2024-04-25T08:58:47.554171+0000","last_became_active":"2024-04-25T08:58:45.542071+0000","last_became_peered":"2024-04-25T08:58:45.542071+0000","last_unstale":"2024-04-25T08:58:47.554171+0000","last_undegraded":"2024-04-25T08:58:47.554171+0000","last_fullsized":"2024-04-25T08:58:47.554171+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:48.910286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921560+0000","last_change":"2024-04-25T08:58:46.547715+0000","last_active":"2024-04-25T08:58:47.921560+0000","last_peered":"2024-04-25T08:58:47.921560+0000","last_clean":"2024-04-25T08:58:47.921560+0000","last_became_active":"2024-04-25T08:58:46.547408+0000","last_became_peered":"2024-04-25T08:58:46.547408+0000","last_unstale":"2024-04-25T08:58:47.921560+0000","last_undegraded":"2024-04-25T08:58:47.921560+0000","last_fullsized":"2024-04-25T08:58:47.921560+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:21.813962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"72'8","reported_seq":22,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.920768+0000","last_change":"2024-04-25T08:58:45.546702+0000","last_active":"2024-04-25T08:58:47.920768+0000","last_peered":"2024-04-25T08:58:47.920768+0000","last_clean":"2024-04-25T08:58:47.920768+0000","last_became_active":"2024-04-25T08:58:45.546346+0000","last_became_peered":"2024-04-25T08:58:45.546346+0000","last_unstale":"2024-04-25T08:58:47.920768+0000","last_undegraded":"2024-04-25T08:58:47.920768+0000","last_fullsized":"2024-04-25T08:58:47.920768+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:17.858568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921198+0000","last_change":"2024-04-25T08:58:46.555032+0000","last_active":"2024-04-25T08:58:47.921198+0000","last_peered":"2024-04-25T08:58:47.921198+0000","last_clean":"2024-04-25T08:58:47.921198+0000","last_became_active":"2024-04-25T08:58:46.554239+0000","last_became_peered":"2024-04-25T08:58:46.554239+0000","last_unstale":"2024-04-25T08:58:47.921198+0000","last_undegraded":"2024-04-25T08:58:47.921198+0000","last_fullsized":"2024-04-25T08:58:47.921198+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:45.757920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"72'1","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.548912+0000","last_change":"2024-04-25T08:58:45.551995+0000","last_active":"2024-04-25T08:58:47.548912+0000","last_peered":"2024-04-25T08:58:47.548912+0000","last_clean":"2024-04-25T08:58:47.548912+0000","last_became_active":"2024-04-25T08:58:45.551744+0000","last_became_peered":"2024-04-25T08:58:45.551744+0000","last_unstale":"2024-04-25T08:58:47.548912+0000","last_undegraded":"2024-04-25T08:58:47.548912+0000","last_fullsized":"2024-04-25T08:58:47.548912+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:33.494101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.549337+0000","last_change":"2024-04-25T08:58:46.565290+0000","last_active":"2024-04-25T08:58:47.549337+0000","last_peered":"2024-04-25T08:58:47.549337+0000","last_clean":"2024-04-25T08:58:47.549337+0000","last_became_active":"2024-04-25T08:58:46.564451+0000","last_became_peered":"2024-04-25T08:58:46.564451+0000","last_unstale":"2024-04-25T08:58:47.549337+0000","last_undegraded":"2024-04-25T08:58:47.549337+0000","last_fullsized":"2024-04-25T08:58:47.549337+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:31.000732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"72'2","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921262+0000","last_change":"2024-04-25T08:58:45.551495+0000","last_active":"2024-04-25T08:58:47.921262+0000","last_peered":"2024-04-25T08:58:47.921262+0000","last_clean":"2024-04-25T08:58:47.921262+0000","last_became_active":"2024-04-25T08:58:45.551222+0000","last_became_peered":"2024-04-25T08:58:45.551222+0000","last_unstale":"2024-04-25T08:58:47.921262+0000","last_undegraded":"2024-04-25T08:58:47.921262+0000","last_fullsized":"2024-04-25T08:58:47.921262+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:33.494101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.546739+0000","last_change":"2024-04-25T08:58:46.552609+0000","last_active":"2024-04-25T08:58:47.546739+0000","last_peered":"2024-04-25T08:58:47.546739+0000","last_clean":"2024-04-25T08:58:47.546739+0000","last_became_active":"2024-04-25T08:58:46.551554+0000","last_became_peered":"2024-04-25T08:58:46.551554+0000","last_unstale":"2024-04-25T08:58:47.546739+0000","last_undegraded":"2024-04-25T08:58:47.546739+0000","last_fullsized":"2024-04-25T08:58:47.546739+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:52.614541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"72'1","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.548679+0000","last_change":"2024-04-25T08:58:45.549628+0000","last_active":"2024-04-25T08:58:47.548679+0000","last_peered":"2024-04-25T08:58:47.548679+0000","last_clean":"2024-04-25T08:58:47.548679+0000","last_became_active":"2024-04-25T08:58:45.549330+0000","last_became_peered":"2024-04-25T08:58:45.549330+0000","last_unstale":"2024-04-25T08:58:47.548679+0000","last_undegraded":"2024-04-25T08:58:47.548679+0000","last_fullsized":"2024-04-25T08:58:47.548679+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:57.275171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921547+0000","last_change":"2024-04-25T08:58:46.556781+0000","last_active":"2024-04-25T08:58:47.921547+0000","last_peered":"2024-04-25T08:58:47.921547+0000","last_clean":"2024-04-25T08:58:47.921547+0000","last_became_active":"2024-04-25T08:58:46.556485+0000","last_became_peered":"2024-04-25T08:58:46.556485+0000","last_unstale":"2024-04-25T08:58:47.921547+0000","last_undegraded":"2024-04-25T08:58:47.921547+0000","last_fullsized":"2024-04-25T08:58:47.921547+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:07:24.093910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.548306+0000","last_change":"2024-04-25T08:58:45.550699+0000","last_active":"2024-04-25T08:58:47.548306+0000","last_peered":"2024-04-25T08:58:47.548306+0000","last_clean":"2024-04-25T08:58:47.548306+0000","last_became_active":"2024-04-25T08:58:45.550408+0000","last_became_peered":"2024-04-25T08:58:45.550408+0000","last_unstale":"2024-04-25T08:58:47.548306+0000","last_undegraded":"2024-04-25T08:58:47.548306+0000","last_fullsized":"2024-04-25T08:58:47.548306+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:31.469571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921673+0000","last_change":"2024-04-25T08:58:46.543314+0000","last_active":"2024-04-25T08:58:47.921673+0000","last_peered":"2024-04-25T08:58:47.921673+0000","last_clean":"2024-04-25T08:58:47.921673+0000","last_became_active":"2024-04-25T08:58:46.543011+0000","last_became_peered":"2024-04-25T08:58:46.543011+0000","last_unstale":"2024-04-25T08:58:47.921673+0000","last_undegraded":"2024-04-25T08:58:47.921673+0000","last_fullsized":"2024-04-25T08:58:47.921673+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:16.379484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.548773+0000","last_change":"2024-04-25T08:58:45.554464+0000","last_active":"2024-04-25T08:58:47.548773+0000","last_peered":"2024-04-25T08:58:47.548773+0000","last_clean":"2024-04-25T08:58:47.548773+0000","last_became_active":"2024-04-25T08:58:45.554162+0000","last_became_peered":"2024-04-25T08:58:45.554162+0000","last_unstale":"2024-04-25T08:58:47.548773+0000","last_undegraded":"2024-04-25T08:58:47.548773+0000","last_fullsized":"2024-04-25T08:58:47.548773+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:20.809060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.920947+0000","last_change":"2024-04-25T08:58:46.559421+0000","last_active":"2024-04-25T08:58:47.920947+0000","last_peered":"2024-04-25T08:58:47.920947+0000","last_clean":"2024-04-25T08:58:47.920947+0000","last_became_active":"2024-04-25T08:58:46.559130+0000","last_became_peered":"2024-04-25T08:58:46.559130+0000","last_unstale":"2024-04-25T08:58:47.920947+0000","last_undegraded":"2024-04-25T08:58:47.920947+0000","last_fullsized":"2024-04-25T08:58:47.920947+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:01:29.992024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.554395+0000","last_change":"2024-04-25T08:58:45.547534+0000","last_active":"2024-04-25T08:58:47.554395+0000","last_peered":"2024-04-25T08:58:47.554395+0000","last_clean":"2024-04-25T08:58:47.554395+0000","last_became_active":"2024-04-25T08:58:45.547251+0000","last_became_peered":"2024-04-25T08:58:45.547251+0000","last_unstale":"2024-04-25T08:58:47.554395+0000","last_undegraded":"2024-04-25T08:58:47.554395+0000","last_fullsized":"2024-04-25T08:58:47.554395+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:19.751845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921074+0000","last_change":"2024-04-25T08:58:46.558692+0000","last_active":"2024-04-25T08:58:47.921074+0000","last_peered":"2024-04-25T08:58:47.921074+0000","last_clean":"2024-04-25T08:58:47.921074+0000","last_became_active":"2024-04-25T08:58:46.558034+0000","last_became_peered":"2024-04-25T08:58:46.558034+0000","last_unstale":"2024-04-25T08:58:47.921074+0000","last_undegraded":"2024-04-25T08:58:47.921074+0000","last_fullsized":"2024-04-25T08:58:47.921074+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:06.462171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.556239+0000","last_change":"2024-04-25T08:58:45.548289+0000","last_active":"2024-04-25T08:58:47.556239+0000","last_peered":"2024-04-25T08:58:47.556239+0000","last_clean":"2024-04-25T08:58:47.556239+0000","last_became_active":"2024-04-25T08:58:45.547912+0000","last_became_peered":"2024-04-25T08:58:45.547912+0000","last_unstale":"2024-04-25T08:58:47.556239+0000","last_undegraded":"2024-04-25T08:58:47.556239+0000","last_fullsized":"2024-04-25T08:58:47.556239+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:59.882831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.556524+0000","last_change":"2024-04-25T08:58:46.560297+0000","last_active":"2024-04-25T08:58:47.556524+0000","last_peered":"2024-04-25T08:58:47.556524+0000","last_clean":"2024-04-25T08:58:47.556524+0000","last_became_active":"2024-04-25T08:58:46.559975+0000","last_became_peered":"2024-04-25T08:58:46.559975+0000","last_unstale":"2024-04-25T08:58:47.556524+0000","last_undegraded":"2024-04-25T08:58:47.556524+0000","last_fullsized":"2024-04-25T08:58:47.556524+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:13.640828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"72'2","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.549012+0000","last_change":"2024-04-25T08:58:45.551966+0000","last_active":"2024-04-25T08:58:47.549012+0000","last_peered":"2024-04-25T08:58:47.549012+0000","last_clean":"2024-04-25T08:58:47.549012+0000","last_became_active":"2024-04-25T08:58:45.551405+0000","last_became_peered":"2024-04-25T08:58:45.551405+0000","last_unstale":"2024-04-25T08:58:47.549012+0000","last_undegraded":"2024-04-25T08:58:47.549012+0000","last_fullsized":"2024-04-25T08:58:47.549012+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:39.804943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.554514+0000","last_change":"2024-04-25T08:58:46.548415+0000","last_active":"2024-04-25T08:58:47.554514+0000","last_peered":"2024-04-25T08:58:47.554514+0000","last_clean":"2024-04-25T08:58:47.554514+0000","last_became_active":"2024-04-25T08:58:46.548134+0000","last_became_peered":"2024-04-25T08:58:46.548134+0000","last_unstale":"2024-04-25T08:58:47.554514+0000","last_undegraded":"2024-04-25T08:58:47.554514+0000","last_fullsized":"2024-04-25T08:58:47.554514+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:52.919994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"73'1","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.555891+0000","last_change":"2024-04-25T08:58:45.543388+0000","last_active":"2024-04-25T08:58:47.555891+0000","last_peered":"2024-04-25T08:58:47.555891+0000","last_clean":"2024-04-25T08:58:47.555891+0000","last_became_active":"2024-04-25T08:58:45.543130+0000","last_became_peered":"2024-04-25T08:58:45.543130+0000","last_unstale":"2024-04-25T08:58:47.555891+0000","last_undegraded":"2024-04-25T08:58:47.555891+0000","last_fullsized":"2024-04-25T08:58:47.555891+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:25.716596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.547710+0000","last_change":"2024-04-25T08:58:46.561203+0000","last_active":"2024-04-25T08:58:47.547710+0000","last_peered":"2024-04-25T08:58:47.547710+0000","last_clean":"2024-04-25T08:58:47.547710+0000","last_became_active":"2024-04-25T08:58:46.560912+0000","last_became_peered":"2024-04-25T08:58:46.560912+0000","last_unstale":"2024-04-25T08:58:47.547710+0000","last_undegraded":"2024-04-25T08:58:47.547710+0000","last_fullsized":"2024-04-25T08:58:47.547710+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:42.343818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921491+0000","last_change":"2024-04-25T08:58:45.548036+0000","last_active":"2024-04-25T08:58:47.921491+0000","last_peered":"2024-04-25T08:58:47.921491+0000","last_clean":"2024-04-25T08:58:47.921491+0000","last_became_active":"2024-04-25T08:58:45.547730+0000","last_became_peered":"2024-04-25T08:58:45.547730+0000","last_unstale":"2024-04-25T08:58:47.921491+0000","last_undegraded":"2024-04-25T08:58:47.921491+0000","last_fullsized":"2024-04-25T08:58:47.921491+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:18:32.063884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.549040+0000","last_change":"2024-04-25T08:58:46.552300+0000","last_active":"2024-04-25T08:58:47.549040+0000","last_peered":"2024-04-25T08:58:47.549040+0000","last_clean":"2024-04-25T08:58:47.549040+0000","last_became_active":"2024-04-25T08:58:46.552006+0000","last_became_peered":"2024-04-25T08:58:46.552006+0000","last_unstale":"2024-04-25T08:58:47.549040+0000","last_undegraded":"2024-04-25T08:58:47.549040+0000","last_fullsized":"2024-04-25T08:58:47.549040+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:07.410153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.556434+0000","last_change":"2024-04-25T08:58:45.546323+0000","last_active":"2024-04-25T08:58:47.556434+0000","last_peered":"2024-04-25T08:58:47.556434+0000","last_clean":"2024-04-25T08:58:47.556434+0000","last_became_active":"2024-04-25T08:58:45.545900+0000","last_became_peered":"2024-04-25T08:58:45.545900+0000","last_unstale":"2024-04-25T08:58:47.556434+0000","last_undegraded":"2024-04-25T08:58:47.556434+0000","last_fullsized":"2024-04-25T08:58:47.556434+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:04.977155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921216+0000","last_change":"2024-04-25T08:58:46.552672+0000","last_active":"2024-04-25T08:58:47.921216+0000","last_peered":"2024-04-25T08:58:47.921216+0000","last_clean":"2024-04-25T08:58:47.921216+0000","last_became_active":"2024-04-25T08:58:46.552265+0000","last_became_peered":"2024-04-25T08:58:46.552265+0000","last_unstale":"2024-04-25T08:58:47.921216+0000","last_undegraded":"2024-04-25T08:58:47.921216+0000","last_fullsized":"2024-04-25T08:58:47.921216+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:17.939556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.920999+0000","last_change":"2024-04-25T08:58:45.548287+0000","last_active":"2024-04-25T08:58:47.920999+0000","last_peered":"2024-04-25T08:58:47.920999+0000","last_clean":"2024-04-25T08:58:47.920999+0000","last_became_active":"2024-04-25T08:58:45.547386+0000","last_became_peered":"2024-04-25T08:58:45.547386+0000","last_unstale":"2024-04-25T08:58:47.920999+0000","last_undegraded":"2024-04-25T08:58:47.920999+0000","last_fullsized":"2024-04-25T08:58:47.920999+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:08.516997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.548447+0000","last_change":"2024-04-25T08:58:46.556933+0000","last_active":"2024-04-25T08:58:47.548447+0000","last_peered":"2024-04-25T08:58:47.548447+0000","last_clean":"2024-04-25T08:58:47.548447+0000","last_became_active":"2024-04-25T08:58:46.556606+0000","last_became_peered":"2024-04-25T08:58:46.556606+0000","last_unstale":"2024-04-25T08:58:47.548447+0000","last_undegraded":"2024-04-25T08:58:47.548447+0000","last_fullsized":"2024-04-25T08:58:47.548447+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:00.061522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.546411+0000","last_change":"2024-04-25T08:58:45.541082+0000","last_active":"2024-04-25T08:58:47.546411+0000","last_peered":"2024-04-25T08:58:47.546411+0000","last_clean":"2024-04-25T08:58:47.546411+0000","last_became_active":"2024-04-25T08:58:45.540543+0000","last_became_peered":"2024-04-25T08:58:45.540543+0000","last_unstale":"2024-04-25T08:58:47.546411+0000","last_undegraded":"2024-04-25T08:58:47.546411+0000","last_fullsized":"2024-04-25T08:58:47.546411+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:57.275171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.546506+0000","last_change":"2024-04-25T08:58:46.543187+0000","last_active":"2024-04-25T08:58:47.546506+0000","last_peered":"2024-04-25T08:58:47.546506+0000","last_clean":"2024-04-25T08:58:47.546506+0000","last_became_active":"2024-04-25T08:58:46.542866+0000","last_became_peered":"2024-04-25T08:58:46.542866+0000","last_unstale":"2024-04-25T08:58:47.546506+0000","last_undegraded":"2024-04-25T08:58:47.546506+0000","last_fullsized":"2024-04-25T08:58:47.546506+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:21:20.948989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.920717+0000","last_change":"2024-04-25T08:58:45.547880+0000","last_active":"2024-04-25T08:58:47.920717+0000","last_peered":"2024-04-25T08:58:47.920717+0000","last_clean":"2024-04-25T08:58:47.920717+0000","last_became_active":"2024-04-25T08:58:45.547619+0000","last_became_peered":"2024-04-25T08:58:45.547619+0000","last_unstale":"2024-04-25T08:58:47.920717+0000","last_undegraded":"2024-04-25T08:58:47.920717+0000","last_fullsized":"2024-04-25T08:58:47.920717+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:56.102307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.547751+0000","last_change":"2024-04-25T08:58:46.556812+0000","last_active":"2024-04-25T08:58:47.547751+0000","last_peered":"2024-04-25T08:58:47.547751+0000","last_clean":"2024-04-25T08:58:47.547751+0000","last_became_active":"2024-04-25T08:58:46.556518+0000","last_became_peered":"2024-04-25T08:58:46.556518+0000","last_unstale":"2024-04-25T08:58:47.547751+0000","last_undegraded":"2024-04-25T08:58:47.547751+0000","last_fullsized":"2024-04-25T08:58:47.547751+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:20.756748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":14,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.547756+0000","last_change":"2024-04-25T08:58:45.553821+0000","last_active":"2024-04-25T08:58:47.547756+0000","last_peered":"2024-04-25T08:58:47.547756+0000","last_clean":"2024-04-25T08:58:47.547756+0000","last_became_active":"2024-04-25T08:58:45.553473+0000","last_became_peered":"2024-04-25T08:58:45.553473+0000","last_unstale":"2024-04-25T08:58:47.547756+0000","last_undegraded":"2024-04-25T08:58:47.547756+0000","last_fullsized":"2024-04-25T08:58:47.547756+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:37.384269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":12,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.921249+0000","last_change":"2024-04-25T08:58:46.557743+0000","last_active":"2024-04-25T08:58:47.921249+0000","last_peered":"2024-04-25T08:58:47.921249+0000","last_clean":"2024-04-25T08:58:47.921249+0000","last_became_active":"2024-04-25T08:58:46.557454+0000","last_became_peered":"2024-04-25T08:58:46.557454+0000","last_unstale":"2024-04-25T08:58:47.921249+0000","last_undegraded":"2024-04-25T08:58:47.921249+0000","last_fullsized":"2024-04-25T08:58:47.921249+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:45.526604+0000","last_clean_scrub_stamp":"2024-04-25T08:58:45.526604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:16.939892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"72'2","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.548552+0000","last_change":"2024-04-25T08:58:45.549705+0000","last_active":"2024-04-25T08:58:47.548552+0000","last_peered":"2024-04-25T08:58:47.548552+0000","last_clean":"2024-04-25T08:58:47.548552+0000","last_became_active":"2024-04-25T08:58:45.547577+0000","last_became_peered":"2024-04-25T08:58:45.547577+0000","last_unstale":"2024-04-25T08:58:47.548552+0000","last_undegraded":"2024-04-25T08:58:47.548552+0000","last_fullsized":"2024-04-25T08:58:47.548552+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:58:44.521702+0000","last_clean_scrub_stamp":"2024-04-25T08:58:44.521702+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:49:33.945526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":267,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-04-25T08:58:47.547576+0000","last_change":"2024-04-25T08:54:00.818044+0000","last_active":"2024-04-25T08:58:47.547576+0000","last_peered":"2024-04-25T08:58:47.547576+0000","last_clean":"2024-04-25T08:58:47.547576+0000","last_became_active":"2024-04-25T08:54:00.817098+0000","last_became_peered":"2024-04-25T08:54:00.817098+0000","last_unstale":"2024-04-25T08:58:47.547576+0000","last_undegraded":"2024-04-25T08:58:47.547576+0000","last_fullsized":"2024-04-25T08:58:47.547576+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_clean_scrub_stamp":"2024-04-25T08:53:59.739677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:48.129513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574908,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40956,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330884,"statfs":{"total":96636764160,"available":96594825216,"internally_reserved":0,"allocated":1490944,"data_stored":969990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574909,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40384,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331456,"statfs":{"total":96636764160,"available":96595410944,"internally_reserved":0,"allocated":897024,"data_stored":379098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574911,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40372,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331468,"statfs":{"total":96636764160,"available":96595423232,"internally_reserved":0,"allocated":892928,"data_stored":379096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574909,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45940,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325900,"statfs":{"total":96636764160,"available":96589721600,"internally_reserved":0,"allocated":892928,"data_stored":379076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574909,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58412,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313428,"statfs":{"total":96636764160,"available":96576950272,"internally_reserved":0,"allocated":884736,"data_stored":372818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574909,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40380,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331460,"statfs":{"total":96636764160,"available":96595415040,"internally_reserved":0,"allocated":901120,"data_stored":380618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574909,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50116,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321724,"statfs":{"total":96636764160,"available":96585445376,"internally_reserved":0,"allocated":909312,"data_stored":381234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574910,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28476,"kb_used_data":1448,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343364,"statfs":{"total":96636764160,"available":96607604736,"internally_reserved":0,"allocated":1482752,"data_stored":969374,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T08:58:50.894 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T08:58:50.894 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T08:58:51.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.049+0000 7fdf5021f640 1 Processor -- start 2024-04-25T08:58:51.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.050+0000 7fdf5021f640 1 -- start start 2024-04-25T08:58:51.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.050+0000 7fdf5021f640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fdf480cf690 0x7fdf480d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.050+0000 7fdf5021f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf480ceb80 con 0x7fdf4815bbe0 2024-04-25T08:58:51.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.050+0000 7fdf5021f640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf480cee60 con 0x7fdf480d04c0 2024-04-25T08:58:51.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.050+0000 7fdf5021f640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf480cf140 con 0x7fdf480cf690 2024-04-25T08:58:51.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.050+0000 7fdf4dfbb640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fdf4815bbe0 0x7fdf480cfdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45306/0 (socket says 172.21.15.43:45306) 2024-04-25T08:58:51.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.050+0000 7fdf4dfbb640 1 -- 172.21.15.43:0/3303187611 learned_addr learned my addr 172.21.15.43:0/3303187611 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:51.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.051+0000 7fdf4e7bc640 1 --2- 172.21.15.43:0/3303187611 >> v2:172.21.15.193:3300/0 conn(0x7fdf480cf690 0x7fdf480d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.051+0000 7fdf4cfb9640 1 -- 172.21.15.43:0/3303187611 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2378179943 0 0) 0x7fdf480ceb80 con 0x7fdf4815bbe0 2024-04-25T08:58:51.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.051+0000 7fdf4cfb9640 1 -- 172.21.15.43:0/3303187611 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf30003520 con 0x7fdf4815bbe0 2024-04-25T08:58:51.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.051+0000 7fdf4cfb9640 1 -- 172.21.15.43:0/3303187611 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 334277315 0 0) 0x7fdf30003520 con 0x7fdf4815bbe0 2024-04-25T08:58:51.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.051+0000 7fdf4cfb9640 1 -- 172.21.15.43:0/3303187611 >> v2:172.21.15.193:3300/0 conn(0x7fdf480cf690 msgr2=0x7fdf480d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:51.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.051+0000 7fdf4cfb9640 1 --2- 172.21.15.43:0/3303187611 >> v2:172.21.15.193:3300/0 conn(0x7fdf480cf690 0x7fdf480d22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.051+0000 7fdf4cfb9640 1 -- 172.21.15.43:0/3303187611 >> v1:172.21.15.193:6789/0 conn(0x7fdf480d04c0 legacy=0x7fdf480b16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:51.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.051+0000 7fdf4cfb9640 1 -- 172.21.15.43:0/3303187611 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf480b0ef0 con 0x7fdf4815bbe0 2024-04-25T08:58:51.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.051+0000 7fdf4cfb9640 1 -- 172.21.15.43:0/3303187611 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7fdf40001670 con 0x7fdf4815bbe0 2024-04-25T08:58:51.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.051+0000 7fdf4cfb9640 1 -- 172.21.15.43:0/3303187611 >> v1:172.21.15.43:6789/0 conn(0x7fdf4815bbe0 legacy=0x7fdf480cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:51.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.051+0000 7fdf4cfb9640 1 --2- 172.21.15.43:0/3303187611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf300039b0 0x7fdf30003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.051+0000 7fdf4cfb9640 1 --2- 172.21.15.43:0/3303187611 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdf30004800 0x7fdf30004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.051+0000 7fdf4cfb9640 1 --2- 172.21.15.43:0/3303187611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf300052e0 0x7fdf3000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.051+0000 7fdf4cfb9640 1 -- 172.21.15.43:0/3303187611 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf480cf140 con 0x7fdf300052e0 2024-04-25T08:58:51.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.051+0000 7fdf4cfb9640 1 -- 172.21.15.43:0/3303187611 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3000c1b0 con 0x7fdf300039b0 2024-04-25T08:58:51.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.051+0000 7fdf4cfb9640 1 -- 172.21.15.43:0/3303187611 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3000c3b0 con 0x7fdf30004800 2024-04-25T08:58:51.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.051+0000 7fdf4e7bc640 1 --2- 172.21.15.43:0/3303187611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf300052e0 0x7fdf3000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.051+0000 7fdf4dfbb640 1 --2- 172.21.15.43:0/3303187611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf300039b0 0x7fdf30003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.051+0000 7fdf4d7ba640 1 --2- 172.21.15.43:0/3303187611 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdf30004800 0x7fdf30004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.051+0000 7fdf4e7bc640 1 -- 172.21.15.43:0/3303187611 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdf30004800 msgr2=0x7fdf30004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:51.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.051+0000 7fdf4e7bc640 1 --2- 172.21.15.43:0/3303187611 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdf30004800 0x7fdf30004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.051+0000 7fdf4e7bc640 1 -- 172.21.15.43:0/3303187611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf300039b0 msgr2=0x7fdf30003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:51.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.051+0000 7fdf4e7bc640 1 --2- 172.21.15.43:0/3303187611 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf300039b0 0x7fdf30003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.051+0000 7fdf4e7bc640 1 -- 172.21.15.43:0/3303187611 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf30003520 con 0x7fdf300052e0 2024-04-25T08:58:51.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.052+0000 7fdf4e7bc640 1 -- 172.21.15.43:0/3303187611 reap_dead start 2024-04-25T08:58:51.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.052+0000 7fdf4e7bc640 1 --2- 172.21.15.43:0/3303187611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf300052e0 0x7fdf3000b9c0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fdf38009f90 tx=0x7fdf30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c94eacb777175d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:51.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.052+0000 7fdf4cfb9640 1 -- 172.21.15.43:0/3303187611 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf38035020 con 0x7fdf300052e0 2024-04-25T08:58:51.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.052+0000 7fdf4cfb9640 1 -- 172.21.15.43:0/3303187611 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf38003b90 con 0x7fdf300052e0 2024-04-25T08:58:51.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.052+0000 7fdf4cfb9640 1 -- 172.21.15.43:0/3303187611 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf38009c10 con 0x7fdf300052e0 2024-04-25T08:58:51.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.053+0000 7fdf5021f640 1 -- 172.21.15.43:0/3303187611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf300052e0 msgr2=0x7fdf3000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:51.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.053+0000 7fdf5021f640 1 --2- 172.21.15.43:0/3303187611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf300052e0 0x7fdf3000b9c0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fdf38009f90 tx=0x7fdf30005830 comp rx=0 tx=0).stop 2024-04-25T08:58:51.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.053+0000 7fdf5021f640 1 -- 172.21.15.43:0/3303187611 shutdown_connections 2024-04-25T08:58:51.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.053+0000 7fdf5021f640 1 --2- 172.21.15.43:0/3303187611 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf300052e0 0x7fdf3000b9c0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.053+0000 7fdf5021f640 1 -- 172.21.15.43:0/3303187611 >> 172.21.15.43:0/3303187611 conn(0x7fdf480d17d0 msgr2=0x7fdf480d1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:51.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.053+0000 7fdf5021f640 1 -- 172.21.15.43:0/3303187611 shutdown_connections 2024-04-25T08:58:51.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.053+0000 7fdf5021f640 1 -- 172.21.15.43:0/3303187611 wait complete. 2024-04-25T08:58:51.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.054+0000 7fdf5021f640 1 Processor -- start 2024-04-25T08:58:51.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.054+0000 7fdf5021f640 1 -- start start 2024-04-25T08:58:51.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.055+0000 7fdf5021f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdf300052e0 0x7fdf480c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.055+0000 7fdf5021f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf48148060 0x7fdf480c89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.055+0000 7fdf5021f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf481484e0 0x7fdf480c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.055+0000 7fdf5021f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4815c140 con 0x7fdf481484e0 2024-04-25T08:58:51.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.055+0000 7fdf5021f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf4815bfc0 con 0x7fdf48148060 2024-04-25T08:58:51.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.055+0000 7fdf5021f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf480d22f0 con 0x7fdf300052e0 2024-04-25T08:58:51.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.055+0000 7fdf4e7bc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf481484e0 0x7fdf480c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.055+0000 7fdf4e7bc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf481484e0 0x7fdf480c8f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49098/0 (socket says 172.21.15.43:49098) 2024-04-25T08:58:51.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.055+0000 7fdf4e7bc640 1 -- 172.21.15.43:0/1746937405 learned_addr learned my addr 172.21.15.43:0/1746937405 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:51.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.055+0000 7fdf4dfbb640 1 --2- 172.21.15.43:0/1746937405 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdf300052e0 0x7fdf480c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.055+0000 7fdf4d7ba640 1 --2- 172.21.15.43:0/1746937405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf48148060 0x7fdf480c89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.055+0000 7fdf4e7bc640 1 -- 172.21.15.43:0/1746937405 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdf300052e0 msgr2=0x7fdf480c7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:51.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.055+0000 7fdf4e7bc640 1 --2- 172.21.15.43:0/1746937405 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdf300052e0 0x7fdf480c7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.055+0000 7fdf4e7bc640 1 -- 172.21.15.43:0/1746937405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf48148060 msgr2=0x7fdf480c89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:51.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.055+0000 7fdf4e7bc640 1 --2- 172.21.15.43:0/1746937405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf48148060 0x7fdf480c89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.055+0000 7fdf4e7bc640 1 -- 172.21.15.43:0/1746937405 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf480cadc0 con 0x7fdf481484e0 2024-04-25T08:58:51.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.056+0000 7fdf4e7bc640 1 --2- 172.21.15.43:0/1746937405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf481484e0 0x7fdf480c8f30 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fdf3800e230 tx=0x7fdf38009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:51.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.056+0000 7fdf467fc640 1 -- 172.21.15.43:0/1746937405 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf38035070 con 0x7fdf481484e0 2024-04-25T08:58:51.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.056+0000 7fdf467fc640 1 -- 172.21.15.43:0/1746937405 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf38041070 con 0x7fdf481484e0 2024-04-25T08:58:51.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.056+0000 7fdf5021f640 1 -- 172.21.15.43:0/1746937405 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf480cb0a0 con 0x7fdf481484e0 2024-04-25T08:58:51.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.056+0000 7fdf467fc640 1 -- 172.21.15.43:0/1746937405 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf38009c10 con 0x7fdf481484e0 2024-04-25T08:58:51.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.056+0000 7fdf5021f640 1 -- 172.21.15.43:0/1746937405 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf480cb660 con 0x7fdf481484e0 2024-04-25T08:58:51.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.057+0000 7fdf5021f640 1 -- 172.21.15.43:0/1746937405 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf48074f60 con 0x7fdf481484e0 2024-04-25T08:58:51.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.057+0000 7fdf467fc640 1 -- 172.21.15.43:0/1746937405 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdf38005d10 con 0x7fdf481484e0 2024-04-25T08:58:51.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.058+0000 7fdf467fc640 1 --2- 172.21.15.43:0/1746937405 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdf3406b4d0 0x7fdf3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.058+0000 7fdf467fc640 1 -- 172.21.15.43:0/1746937405 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdf380aae20 con 0x7fdf481484e0 2024-04-25T08:58:51.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.058+0000 7fdf4dfbb640 1 --2- 172.21.15.43:0/1746937405 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdf3406b4d0 0x7fdf3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.058+0000 7fdf4dfbb640 1 --2- 172.21.15.43:0/1746937405 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdf3406b4d0 0x7fdf3406d990 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fdf40001eb0 tx=0x7fdf40007040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:51.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.062+0000 7fdf467fc640 1 -- 172.21.15.43:0/1746937405 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdf38079650 con 0x7fdf481484e0 2024-04-25T08:58:51.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.316+0000 7fdf5021f640 1 -- 172.21.15.43:0/1746937405 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fdf480b0340 con 0x7fdf481484e0 2024-04-25T08:58:51.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.319+0000 7fdf467fc640 1 -- 172.21.15.43:0/1746937405 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+147 (secure 0 0 0) 0x7fdf3807d6f0 con 0x7fdf481484e0 2024-04-25T08:58:51.308 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2024-04-25T08:58:51.308 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQD8GSpmVOjWGxAAFNmxeGi0h2D//pa6Ne3eQw== 2024-04-25T08:58:51.308 INFO:teuthology.orchestra.run.smithi043.stdout: caps mds = "allow" 2024-04-25T08:58:51.308 INFO:teuthology.orchestra.run.smithi043.stdout: caps mon = "allow r" 2024-04-25T08:58:51.308 INFO:teuthology.orchestra.run.smithi043.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T08:58:51.309 INFO:teuthology.orchestra.run.smithi043.stderr:updated caps for client.0 2024-04-25T08:58:51.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.321+0000 7fdf5021f640 1 -- 172.21.15.43:0/1746937405 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdf3406b4d0 msgr2=0x7fdf3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:51.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.321+0000 7fdf5021f640 1 --2- 172.21.15.43:0/1746937405 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdf3406b4d0 0x7fdf3406d990 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fdf40001eb0 tx=0x7fdf40007040 comp rx=0 tx=0).stop 2024-04-25T08:58:51.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.321+0000 7fdf5021f640 1 -- 172.21.15.43:0/1746937405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf481484e0 msgr2=0x7fdf480c8f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:51.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.321+0000 7fdf5021f640 1 --2- 172.21.15.43:0/1746937405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf481484e0 0x7fdf480c8f30 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fdf3800e230 tx=0x7fdf38009750 comp rx=0 tx=0).stop 2024-04-25T08:58:51.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.321+0000 7fdf5021f640 1 -- 172.21.15.43:0/1746937405 shutdown_connections 2024-04-25T08:58:51.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.321+0000 7fdf5021f640 1 --2- 172.21.15.43:0/1746937405 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdf481484e0 0x7fdf480c8f30 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.321+0000 7fdf5021f640 1 --2- 172.21.15.43:0/1746937405 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdf3406b4d0 0x7fdf3406d990 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.321+0000 7fdf5021f640 1 --2- 172.21.15.43:0/1746937405 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdf48148060 0x7fdf480c89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.321+0000 7fdf5021f640 1 --2- 172.21.15.43:0/1746937405 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdf300052e0 0x7fdf480c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.321+0000 7fdf5021f640 1 -- 172.21.15.43:0/1746937405 >> 172.21.15.43:0/1746937405 conn(0x7fdf480d17d0 msgr2=0x7fdf480b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:51.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.321+0000 7fdf5021f640 1 -- 172.21.15.43:0/1746937405 shutdown_connections 2024-04-25T08:58:51.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.321+0000 7fdf5021f640 1 -- 172.21.15.43:0/1746937405 wait complete. 2024-04-25T08:58:51.321 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T08:58:51.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.432+0000 7f226f9e2640 1 Processor -- start 2024-04-25T08:58:51.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.432+0000 7f226f9e2640 1 -- start start 2024-04-25T08:58:51.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.433+0000 7f226f9e2640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f226815df70 0x7f22680cc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.433+0000 7f226f9e2640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22680cbee0 con 0x7f22680b0870 2024-04-25T08:58:51.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.433+0000 7f226f9e2640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22680d0650 con 0x7f226806c510 2024-04-25T08:58:51.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.433+0000 7f226f9e2640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22680d0400 con 0x7f226815df70 2024-04-25T08:58:51.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.433+0000 7f226df7f640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f22680b0870 0x7f22680ccbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45318/0 (socket says 172.21.15.43:45318) 2024-04-25T08:58:51.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.433+0000 7f226df7f640 1 -- 172.21.15.43:0/3393457540 learned_addr learned my addr 172.21.15.43:0/3393457540 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:51.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.434+0000 7f226cf7d640 1 --2- 172.21.15.43:0/3393457540 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f226815df70 0x7f22680cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.434+0000 7f2267fff640 1 -- 172.21.15.43:0/3393457540 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 547325413 0 0) 0x7f22680cbee0 con 0x7f22680b0870 2024-04-25T08:58:51.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.434+0000 7f2267fff640 1 -- 172.21.15.43:0/3393457540 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2250003520 con 0x7f22680b0870 2024-04-25T08:58:51.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.434+0000 7f2267fff640 1 -- 172.21.15.43:0/3393457540 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 725167219 0 0) 0x7f22680d0650 con 0x7f226806c510 2024-04-25T08:58:51.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.434+0000 7f226cf7d640 1 -- 172.21.15.43:0/3393457540 >> v1:172.21.15.193:6789/0 conn(0x7f226806c510 legacy=0x7f226806c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:51.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.434+0000 7f226cf7d640 1 -- 172.21.15.43:0/3393457540 >> v1:172.21.15.43:6789/0 conn(0x7f22680b0870 legacy=0x7f22680ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:51.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.434+0000 7f226cf7d640 1 -- 172.21.15.43:0/3393457540 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22680d0930 con 0x7f226815df70 2024-04-25T08:58:51.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.435+0000 7f226cf7d640 1 --2- 172.21.15.43:0/3393457540 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f226815df70 0x7f22680cc670 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f225c0097d0 tx=0x7f225c02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d80563033352207 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:51.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.435+0000 7f2267fff640 1 -- 172.21.15.43:0/3393457540 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c037070 con 0x7f226815df70 2024-04-25T08:58:51.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.435+0000 7f2267fff640 1 -- 172.21.15.43:0/3393457540 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f225c032440 con 0x7f226815df70 2024-04-25T08:58:51.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.435+0000 7f2267fff640 1 -- 172.21.15.43:0/3393457540 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c0327f0 con 0x7f226815df70 2024-04-25T08:58:51.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.435+0000 7f226f9e2640 1 -- 172.21.15.43:0/3393457540 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f226815df70 msgr2=0x7f22680cc670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:51.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.435+0000 7f226f9e2640 1 --2- 172.21.15.43:0/3393457540 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f226815df70 0x7f22680cc670 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f225c0097d0 tx=0x7f225c02a210 comp rx=0 tx=0).stop 2024-04-25T08:58:51.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.435+0000 7f226f9e2640 1 -- 172.21.15.43:0/3393457540 shutdown_connections 2024-04-25T08:58:51.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.435+0000 7f226f9e2640 1 --2- 172.21.15.43:0/3393457540 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f226815df70 0x7f22680cc670 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.435+0000 7f226f9e2640 1 -- 172.21.15.43:0/3393457540 >> 172.21.15.43:0/3393457540 conn(0x7f22680d17f0 msgr2=0x7f22680d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:51.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.435+0000 7f226f9e2640 1 -- 172.21.15.43:0/3393457540 shutdown_connections 2024-04-25T08:58:51.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.436+0000 7f226f9e2640 1 -- 172.21.15.43:0/3393457540 wait complete. 2024-04-25T08:58:51.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.437+0000 7f226f9e2640 1 Processor -- start 2024-04-25T08:58:51.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.437+0000 7f226f9e2640 1 -- start start 2024-04-25T08:58:51.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.437+0000 7f226f9e2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f226806c510 0x7f22680af900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.437+0000 7f226f9e2640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f22680b0870 0x7f22680c7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.437+0000 7f226f9e2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f226815df70 0x7f22680c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.437+0000 7f226f9e2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226806bd10 con 0x7f226815df70 2024-04-25T08:58:51.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.437+0000 7f226f9e2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22680d0c10 con 0x7f226806c510 2024-04-25T08:58:51.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.437+0000 7f226f9e2640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f226806be90 con 0x7f22680b0870 2024-04-25T08:58:51.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.438+0000 7f226df7f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f226815df70 0x7f22680c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.438+0000 7f226df7f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f226815df70 0x7f22680c7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49114/0 (socket says 172.21.15.43:49114) 2024-04-25T08:58:51.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.438+0000 7f226df7f640 1 -- 172.21.15.43:0/1240445839 learned_addr learned my addr 172.21.15.43:0/1240445839 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:51.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.438+0000 7f226cf7d640 1 --2- 172.21.15.43:0/1240445839 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f22680b0870 0x7f22680c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.438+0000 7f226d77e640 1 --2- 172.21.15.43:0/1240445839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f226806c510 0x7f22680af900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.438+0000 7f226df7f640 1 -- 172.21.15.43:0/1240445839 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f22680b0870 msgr2=0x7f22680c7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:51.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.438+0000 7f226df7f640 1 --2- 172.21.15.43:0/1240445839 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f22680b0870 0x7f22680c7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.438+0000 7f226df7f640 1 -- 172.21.15.43:0/1240445839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f226806c510 msgr2=0x7f22680af900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:51.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.438+0000 7f226df7f640 1 --2- 172.21.15.43:0/1240445839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f226806c510 0x7f22680af900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.438+0000 7f226df7f640 1 -- 172.21.15.43:0/1240445839 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22681f3ab0 con 0x7f226815df70 2024-04-25T08:58:51.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.438+0000 7f226df7f640 1 --2- 172.21.15.43:0/1240445839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f226815df70 0x7f22680c7d90 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f225800c3c0 tx=0x7f2258001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:51.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.438+0000 7f2265ffb640 1 -- 172.21.15.43:0/1240445839 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2258017070 con 0x7f226815df70 2024-04-25T08:58:51.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.438+0000 7f2265ffb640 1 -- 172.21.15.43:0/1240445839 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2258005ce0 con 0x7f226815df70 2024-04-25T08:58:51.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.438+0000 7f226f9e2640 1 -- 172.21.15.43:0/1240445839 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22681f3d30 con 0x7f226815df70 2024-04-25T08:58:51.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.438+0000 7f2265ffb640 1 -- 172.21.15.43:0/1240445839 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2258007310 con 0x7f226815df70 2024-04-25T08:58:51.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.438+0000 7f226f9e2640 1 -- 172.21.15.43:0/1240445839 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22681f42f0 con 0x7f226815df70 2024-04-25T08:58:51.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.439+0000 7f226f9e2640 1 -- 172.21.15.43:0/1240445839 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2268074ef0 con 0x7f226815df70 2024-04-25T08:58:51.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.440+0000 7f2265ffb640 1 -- 172.21.15.43:0/1240445839 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f225801a070 con 0x7f226815df70 2024-04-25T08:58:51.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.440+0000 7f2265ffb640 1 --2- 172.21.15.43:0/1240445839 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f225406b4d0 0x7f225406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:51.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.440+0000 7f2265ffb640 1 -- 172.21.15.43:0/1240445839 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2258089460 con 0x7f226815df70 2024-04-25T08:58:51.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.440+0000 7f226d77e640 1 --2- 172.21.15.43:0/1240445839 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f225406b4d0 0x7f225406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:51.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.441+0000 7f226d77e640 1 --2- 172.21.15.43:0/1240445839 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f225406b4d0 0x7f225406d990 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f22600012e0 tx=0x7f2260009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:51.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.444+0000 7f2265ffb640 1 -- 172.21.15.43:0/1240445839 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f22681f3ab0 con 0x7f226815df70 2024-04-25T08:58:51.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.661+0000 7f226f9e2640 1 -- 172.21.15.43:0/1240445839 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22680cffd0 con 0x7f226815df70 2024-04-25T08:58:51.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.661+0000 7f2265ffb640 1 -- 172.21.15.43:0/1240445839 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f2258057d20 con 0x7f226815df70 2024-04-25T08:58:51.651 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T08:58:51.651 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":47,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6373,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3648859409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":3648859409},{"type":"v1","addr":"172.21.15.193:6837","nonce":3648859409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":7149,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/356516962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":356516962},{"type":"v1","addr":"172.21.15.43:6834","nonce":356516962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":7152,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3032637719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":3032637719},{"type":"v1","addr":"172.21.15.43:6835","nonce":3032637719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T08:58:45.906679+0000","modified":"2024-04-25T08:58:48.767705+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7164},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7164":{"gid":7164,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.193:6836/3458098162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":3458098162},{"type":"v1","addr":"172.21.15.193:6836","nonce":3458098162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T08:58:51.651 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 47 2024-04-25T08:58:51.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.663+0000 7f226f9e2640 1 -- 172.21.15.43:0/1240445839 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f225406b4d0 msgr2=0x7f225406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:51.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.663+0000 7f226f9e2640 1 --2- 172.21.15.43:0/1240445839 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f225406b4d0 0x7f225406d990 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f22600012e0 tx=0x7f2260009040 comp rx=0 tx=0).stop 2024-04-25T08:58:51.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.663+0000 7f226f9e2640 1 -- 172.21.15.43:0/1240445839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f226815df70 msgr2=0x7f22680c7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:51.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.663+0000 7f226f9e2640 1 --2- 172.21.15.43:0/1240445839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f226815df70 0x7f22680c7d90 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f225800c3c0 tx=0x7f2258001e70 comp rx=0 tx=0).stop 2024-04-25T08:58:51.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.663+0000 7f226f9e2640 1 -- 172.21.15.43:0/1240445839 shutdown_connections 2024-04-25T08:58:51.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.663+0000 7f226f9e2640 1 --2- 172.21.15.43:0/1240445839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f226815df70 0x7f22680c7d90 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.663+0000 7f226f9e2640 1 --2- 172.21.15.43:0/1240445839 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f22680b0870 0x7f22680c7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.663+0000 7f226f9e2640 1 --2- 172.21.15.43:0/1240445839 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f225406b4d0 0x7f225406d990 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.663+0000 7f226f9e2640 1 --2- 172.21.15.43:0/1240445839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f226806c510 0x7f22680af900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:51.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.663+0000 7f226f9e2640 1 -- 172.21.15.43:0/1240445839 >> 172.21.15.43:0/1240445839 conn(0x7f22680d17f0 msgr2=0x7f22680d1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:51.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.663+0000 7f226f9e2640 1 -- 172.21.15.43:0/1240445839 shutdown_connections 2024-04-25T08:58:51.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:51.663+0000 7f226f9e2640 1 -- 172.21.15.43:0/1240445839 wait complete. 2024-04-25T08:58:52.067 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 47, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T08:58:45.906679+0000', 'modified': '2024-04-25T08:58:48.767705+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7164}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7164': {'gid': 7164, 'name': 'd', 'rank': 0, 'incarnation': 44, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.193:6836/3458098162', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6835', 'nonce': 3458098162}, {'type': 'v1', 'addr': '172.21.15.193:6836', 'nonce': 3458098162}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T08:58:52.067 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T08:58:52.067 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T08:58:52.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.184+0000 7f045b2ee640 1 Processor -- start 2024-04-25T08:58:52.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.185+0000 7f045b2ee640 1 -- start start 2024-04-25T08:58:52.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.185+0000 7f045b2ee640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f04540cd010 0x7f04540b4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.185+0000 7f045b2ee640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04540b2780 con 0x7f04540cbfc0 2024-04-25T08:58:52.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.185+0000 7f045b2ee640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04540b2a60 con 0x7f04540ccbb0 2024-04-25T08:58:52.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.185+0000 7f045b2ee640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04540b2d40 con 0x7f04540cd010 2024-04-25T08:58:52.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.186+0000 7f045908a640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f04540cbfc0 0x7f045406dc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45332/0 (socket says 172.21.15.43:45332) 2024-04-25T08:58:52.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.186+0000 7f045908a640 1 -- 172.21.15.43:0/333931932 learned_addr learned my addr 172.21.15.43:0/333931932 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:52.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.186+0000 7f045988b640 1 --2- 172.21.15.43:0/333931932 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f04540cd010 0x7f04540b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.186+0000 7f044bfff640 1 -- 172.21.15.43:0/333931932 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4215893088 0 0) 0x7f04540b2780 con 0x7f04540cbfc0 2024-04-25T08:58:52.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.186+0000 7f044bfff640 1 -- 172.21.15.43:0/333931932 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0440003520 con 0x7f04540cbfc0 2024-04-25T08:58:52.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.186+0000 7f044bfff640 1 -- 172.21.15.43:0/333931932 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3891035050 0 0) 0x7f04540b2a60 con 0x7f04540ccbb0 2024-04-25T08:58:52.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.186+0000 7f045988b640 1 -- 172.21.15.43:0/333931932 >> v1:172.21.15.193:6789/0 conn(0x7f04540ccbb0 legacy=0x7f04540b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:52.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.186+0000 7f045988b640 1 -- 172.21.15.43:0/333931932 >> v1:172.21.15.43:6789/0 conn(0x7f04540cbfc0 legacy=0x7f045406dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:52.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.186+0000 7f045988b640 1 -- 172.21.15.43:0/333931932 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04540b3020 con 0x7f04540cd010 2024-04-25T08:58:52.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.187+0000 7f045988b640 1 --2- 172.21.15.43:0/333931932 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f04540cd010 0x7f04540b4210 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f044c009dc0 tx=0x7f044c02a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1f714ecc123c530 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:52.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.187+0000 7f044bfff640 1 -- 172.21.15.43:0/333931932 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044c035020 con 0x7f04540cd010 2024-04-25T08:58:52.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.187+0000 7f044bfff640 1 -- 172.21.15.43:0/333931932 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f044c007540 con 0x7f04540cd010 2024-04-25T08:58:52.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.187+0000 7f044bfff640 1 -- 172.21.15.43:0/333931932 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0450001670 con 0x7f04540cd010 2024-04-25T08:58:52.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.188+0000 7f045b2ee640 1 -- 172.21.15.43:0/333931932 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f04540cd010 msgr2=0x7f04540b4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:52.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.188+0000 7f045b2ee640 1 --2- 172.21.15.43:0/333931932 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f04540cd010 0x7f04540b4210 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f044c009dc0 tx=0x7f044c02a640 comp rx=0 tx=0).stop 2024-04-25T08:58:52.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.188+0000 7f045b2ee640 1 -- 172.21.15.43:0/333931932 shutdown_connections 2024-04-25T08:58:52.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.188+0000 7f045b2ee640 1 --2- 172.21.15.43:0/333931932 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f04540cd010 0x7f04540b4210 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.188+0000 7f045b2ee640 1 -- 172.21.15.43:0/333931932 >> 172.21.15.43:0/333931932 conn(0x7f04540b1480 msgr2=0x7f04540b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:52.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.188+0000 7f045b2ee640 1 -- 172.21.15.43:0/333931932 shutdown_connections 2024-04-25T08:58:52.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.188+0000 7f045b2ee640 1 -- 172.21.15.43:0/333931932 wait complete. 2024-04-25T08:58:52.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.189+0000 7f045b2ee640 1 Processor -- start 2024-04-25T08:58:52.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.189+0000 7f045b2ee640 1 -- start start 2024-04-25T08:58:52.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.190+0000 7f045b2ee640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04540cbfc0 0x7f045415cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.190+0000 7f045b2ee640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f04540ccbb0 0x7f04540ae290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.190+0000 7f045b2ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04540cd010 0x7f04540ac220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.190+0000 7f045b2ee640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04540cd4a0 con 0x7f04540cd010 2024-04-25T08:58:52.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.190+0000 7f045b2ee640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04540b35e0 con 0x7f04540cbfc0 2024-04-25T08:58:52.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.190+0000 7f045b2ee640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04540cc3a0 con 0x7f04540ccbb0 2024-04-25T08:58:52.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.190+0000 7f045988b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04540cd010 0x7f04540ac220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.190+0000 7f045988b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04540cd010 0x7f04540ac220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49116/0 (socket says 172.21.15.43:49116) 2024-04-25T08:58:52.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.190+0000 7f045988b640 1 -- 172.21.15.43:0/2822791246 learned_addr learned my addr 172.21.15.43:0/2822791246 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:52.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.190+0000 7f0458889640 1 --2- 172.21.15.43:0/2822791246 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f04540ccbb0 0x7f04540ae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.190+0000 7f045908a640 1 --2- 172.21.15.43:0/2822791246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04540cbfc0 0x7f045415cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.190+0000 7f045988b640 1 -- 172.21.15.43:0/2822791246 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f04540ccbb0 msgr2=0x7f04540ae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:52.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.190+0000 7f045988b640 1 --2- 172.21.15.43:0/2822791246 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f04540ccbb0 0x7f04540ae290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.190+0000 7f045988b640 1 -- 172.21.15.43:0/2822791246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04540cbfc0 msgr2=0x7f045415cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:52.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.190+0000 7f045988b640 1 --2- 172.21.15.43:0/2822791246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04540cbfc0 0x7f045415cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.190+0000 7f045988b640 1 -- 172.21.15.43:0/2822791246 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f045415d5e0 con 0x7f04540cd010 2024-04-25T08:58:52.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.190+0000 7f045908a640 1 --2- 172.21.15.43:0/2822791246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04540cbfc0 0x7f045415cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:52.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.190+0000 7f0458889640 1 --2- 172.21.15.43:0/2822791246 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f04540ccbb0 0x7f04540ae290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:52.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.190+0000 7f045988b640 1 --2- 172.21.15.43:0/2822791246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04540cd010 0x7f04540ac220 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f044c009ee0 tx=0x7f044c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:52.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.191+0000 7f0449ffb640 1 -- 172.21.15.43:0/2822791246 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044c035030 con 0x7f04540cd010 2024-04-25T08:58:52.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.191+0000 7f0449ffb640 1 -- 172.21.15.43:0/2822791246 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f044c031c40 con 0x7f04540cd010 2024-04-25T08:58:52.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.191+0000 7f045b2ee640 1 -- 172.21.15.43:0/2822791246 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f045415d8c0 con 0x7f04540cd010 2024-04-25T08:58:52.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.191+0000 7f0449ffb640 1 -- 172.21.15.43:0/2822791246 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044c009450 con 0x7f04540cd010 2024-04-25T08:58:52.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.191+0000 7f045b2ee640 1 -- 172.21.15.43:0/2822791246 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f045415de80 con 0x7f04540cd010 2024-04-25T08:58:52.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.192+0000 7f0449ffb640 1 -- 172.21.15.43:0/2822791246 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f044c03b460 con 0x7f04540cd010 2024-04-25T08:58:52.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.193+0000 7f0449ffb640 1 --2- 172.21.15.43:0/2822791246 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f043806b4d0 0x7f043806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.193+0000 7f0449ffb640 1 -- 172.21.15.43:0/2822791246 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f044c0ab3a0 con 0x7f04540cd010 2024-04-25T08:58:52.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.193+0000 7f045908a640 1 --2- 172.21.15.43:0/2822791246 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f043806b4d0 0x7f043806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.194+0000 7f045dfeb740 1 Processor -- start 2024-04-25T08:58:52.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.194+0000 7f045908a640 1 --2- 172.21.15.43:0/2822791246 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f043806b4d0 0x7f043806d990 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f04500017e0 tx=0x7f0450009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:52.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.194+0000 7f045dfeb740 1 -- start start 2024-04-25T08:58:52.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.194+0000 7f045dfeb740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561aea833230 0x561aea833610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.194+0000 7f045dfeb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aea850ad0 0x561aea851020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.194+0000 7f045dfeb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561aea808f10 0x561aea84f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.194+0000 7f045dfeb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aea76ba10 con 0x561aea850ad0 2024-04-25T08:58:52.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.194+0000 7f045dfeb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aea714b30 con 0x561aea808f10 2024-04-25T08:58:52.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.194+0000 7f045dfeb740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x561aea767c50 con 0x561aea833230 2024-04-25T08:58:52.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.194+0000 7f0458889640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aea850ad0 0x561aea851020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.194+0000 7f0458889640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aea850ad0 0x561aea851020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49124/0 (socket says 172.21.15.43:49124) 2024-04-25T08:58:52.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.195+0000 7f0458889640 1 -- 172.21.15.43:0/1541121497 learned_addr learned my addr 172.21.15.43:0/1541121497 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:52.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.195+0000 7f045988b640 1 --2- 172.21.15.43:0/1541121497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561aea808f10 0x561aea84f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.195+0000 7f045908a640 1 --2- 172.21.15.43:0/1541121497 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561aea833230 0x561aea833610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.195+0000 7f0458889640 1 -- 172.21.15.43:0/1541121497 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561aea833230 msgr2=0x561aea833610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:52.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.195+0000 7f0458889640 1 --2- 172.21.15.43:0/1541121497 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561aea833230 0x561aea833610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.195+0000 7f0458889640 1 -- 172.21.15.43:0/1541121497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561aea808f10 msgr2=0x561aea84f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:52.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.195+0000 7f0458889640 1 --2- 172.21.15.43:0/1541121497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561aea808f10 0x561aea84f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.195+0000 7f0458889640 1 -- 172.21.15.43:0/1541121497 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561aea729e10 con 0x561aea850ad0 2024-04-25T08:58:52.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.195+0000 7f0458889640 1 --2- 172.21.15.43:0/1541121497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aea850ad0 0x561aea851020 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f04440017b0 tx=0x7f0444001c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=886a03fc2eb17250 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:52.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.195+0000 7f043effd640 1 -- 172.21.15.43:0/1541121497 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0444018070 con 0x561aea850ad0 2024-04-25T08:58:52.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.195+0000 7f043effd640 1 -- 172.21.15.43:0/1541121497 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f044400d730 con 0x561aea850ad0 2024-04-25T08:58:52.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.195+0000 7f043effd640 1 -- 172.21.15.43:0/1541121497 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044400db20 con 0x561aea850ad0 2024-04-25T08:58:52.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.196+0000 7f045dfeb740 1 -- 172.21.15.43:0/1541121497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aea850ad0 msgr2=0x561aea851020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:52.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.196+0000 7f045dfeb740 1 --2- 172.21.15.43:0/1541121497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aea850ad0 0x561aea851020 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f04440017b0 tx=0x7f0444001c70 comp rx=0 tx=0).stop 2024-04-25T08:58:52.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.196+0000 7f045dfeb740 1 -- 172.21.15.43:0/1541121497 shutdown_connections 2024-04-25T08:58:52.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.196+0000 7f045dfeb740 1 --2- 172.21.15.43:0/1541121497 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561aea808f10 0x561aea84f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.196+0000 7f045dfeb740 1 --2- 172.21.15.43:0/1541121497 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aea850ad0 0x561aea851020 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.196+0000 7f045dfeb740 1 --2- 172.21.15.43:0/1541121497 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561aea833230 0x561aea833610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.196+0000 7f045dfeb740 1 -- 172.21.15.43:0/1541121497 >> 172.21.15.43:0/1541121497 conn(0x561aea805840 msgr2=0x561aea84e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:52.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.196+0000 7f045dfeb740 1 -- 172.21.15.43:0/1541121497 shutdown_connections 2024-04-25T08:58:52.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.196+0000 7f045dfeb740 1 -- 172.21.15.43:0/1541121497 wait complete. 2024-04-25T08:58:52.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.196+0000 7f045dfeb740 1 -- start start 2024-04-25T08:58:52.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.196+0000 7f045dfeb740 1 Processor -- start 2024-04-25T08:58:52.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.197+0000 7f045dfeb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561aea833230 0x561aea836ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.197+0000 7f045dfeb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aea850ad0 0x561aea833f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.197+0000 7f045dfeb740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561aea8344c0 0x561aea82c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.197+0000 7f045dfeb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aea809cb0 con 0x561aea850ad0 2024-04-25T08:58:52.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.197+0000 7f045dfeb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aea7280c0 con 0x561aea833230 2024-04-25T08:58:52.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.197+0000 7f045dfeb740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x561aea803770 con 0x561aea8344c0 2024-04-25T08:58:52.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.197+0000 7f045dfeb740 10 client.0 _ll_register_callbacks cb 0x561aea71bec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:58:52.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.197+0000 7f0458889640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aea850ad0 0x561aea833f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.197+0000 7f0458889640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aea850ad0 0x561aea833f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49134/0 (socket says 172.21.15.43:49134) 2024-04-25T08:58:52.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.197+0000 7f0458889640 1 -- 172.21.15.43:0/4280234875 learned_addr learned my addr 172.21.15.43:0/4280234875 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:52.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.198+0000 7f045908a640 1 --2- 172.21.15.43:0/4280234875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561aea833230 0x561aea836ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.198+0000 7f045988b640 1 --2- 172.21.15.43:0/4280234875 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561aea8344c0 0x561aea82c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.198+0000 7f0458889640 1 -- 172.21.15.43:0/4280234875 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561aea8344c0 msgr2=0x561aea82c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:52.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.198+0000 7f0458889640 1 --2- 172.21.15.43:0/4280234875 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561aea8344c0 0x561aea82c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.198+0000 7f0458889640 1 -- 172.21.15.43:0/4280234875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561aea833230 msgr2=0x561aea836ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:52.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.198+0000 7f0458889640 1 --2- 172.21.15.43:0/4280234875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561aea833230 0x561aea836ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.198+0000 7f0458889640 1 -- 172.21.15.43:0/4280234875 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561aea4802d0 con 0x561aea850ad0 2024-04-25T08:58:52.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.198+0000 7f0458889640 1 --2- 172.21.15.43:0/4280234875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aea850ad0 0x561aea833f80 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f0444012000 tx=0x7f0444000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a28df63ba54445 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:52.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.198+0000 7f0433fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:58:52.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.198+0000 7f0433fff640 1 -- 172.21.15.43:0/4280234875 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0444023020 con 0x561aea850ad0 2024-04-25T08:58:52.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.198+0000 7f0433fff640 1 -- 172.21.15.43:0/4280234875 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f044400dd70 con 0x561aea850ad0 2024-04-25T08:58:52.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.198+0000 7f045dfeb740 1 -- 172.21.15.43:0/4280234875 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f044400d730 con 0x561aea850ad0 2024-04-25T08:58:52.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.198+0000 7f0433fff640 1 -- 172.21.15.43:0/4280234875 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f044402d050 con 0x561aea850ad0 2024-04-25T08:58:52.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.198+0000 7f045dfeb740 10 client.7293 fetch_fsmap learned FSMap version 47 2024-04-25T08:58:52.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.198+0000 7f045dfeb740 1 -- 172.21.15.43:0/4280234875 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x561aea54a6b0 con 0x561aea850ad0 2024-04-25T08:58:52.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.198+0000 7f0433fff640 1 -- 172.21.15.43:0/4280234875 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f044400d730 con 0x561aea850ad0 2024-04-25T08:58:52.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.199+0000 7f0433fff640 1 -- 172.21.15.43:0/4280234875 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 47) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0444030020 con 0x561aea850ad0 2024-04-25T08:58:52.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.199+0000 7f045dfeb740 10 client.7293 fetch_fsmap finished waiting for FSMap version 47 2024-04-25T08:58:52.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.199+0000 7f045dfeb740 10 client.7293 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T08:58:52.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.199+0000 7f045dfeb740 20 client.7293 populate_metadata read hostname 'smithi043' 2024-04-25T08:58:52.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.199+0000 7f045dfeb740 1 --2- 172.21.15.43:0/4280234875 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x561aea862280 0x561aea845910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.199+0000 7f045dfeb740 4 client.7293 mds_command: new command op to 7164 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:58:52.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.199+0000 7f045dfeb740 1 -- 172.21.15.43:0/4280234875 --> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561aea7ca920 con 0x561aea862280 2024-04-25T08:58:52.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.199+0000 7f045908a640 1 --2- 172.21.15.43:0/4280234875 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x561aea862280 0x561aea845910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.199+0000 7f045908a640 1 --2- 172.21.15.43:0/4280234875 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x561aea862280 0x561aea845910 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4ed65bd27b152c server_cookie=d5078bf7f0d1c8b1 in_seq=0 out_seq=0 2024-04-25T08:58:52.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.199+0000 7f0433fff640 10 client.7293 ms_handle_connect on v2:172.21.15.193:6835/3458098162 2024-04-25T08:58:52.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.200+0000 7f0433fff640 1 -- 172.21.15.43:0/4280234875 <== mds.0 v2:172.21.15.193:6835/3458098162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561aea7ca920 con 0x561aea862280 2024-04-25T08:58:52.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.200+0000 7f0433fff640 10 client.7293 handle_command_reply: tid=0 2024-04-25T08:58:52.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.200+0000 7f045dfeb740 1 client.7293 shutdown 2024-04-25T08:58:52.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.200+0000 7f045dfeb740 1 -- 172.21.15.43:0/4280234875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aea850ad0 msgr2=0x561aea833f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:52.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.200+0000 7f045dfeb740 1 --2- 172.21.15.43:0/4280234875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aea850ad0 0x561aea833f80 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f0444012000 tx=0x7f0444000ed0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.200+0000 7f045dfeb740 1 -- 172.21.15.43:0/4280234875 shutdown_connections 2024-04-25T08:58:52.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.200+0000 7f045dfeb740 1 --2- 172.21.15.43:0/4280234875 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x561aea862280 0x561aea845910 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.200+0000 7f045dfeb740 1 --2- 172.21.15.43:0/4280234875 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561aea8344c0 0x561aea82c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.200+0000 7f045dfeb740 1 --2- 172.21.15.43:0/4280234875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aea850ad0 0x561aea833f80 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.200+0000 7f0433fff640 0 client.7293 ms_handle_reset on v2:172.21.15.193:6835/3458098162 2024-04-25T08:58:52.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.200+0000 7f045dfeb740 1 --2- 172.21.15.43:0/4280234875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561aea833230 0x561aea836ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.200+0000 7f045dfeb740 1 -- 172.21.15.43:0/4280234875 >> 172.21.15.43:0/4280234875 conn(0x561aea808f10 msgr2=0x561aea851e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:52.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.200+0000 7f045dfeb740 1 -- 172.21.15.43:0/4280234875 shutdown_connections 2024-04-25T08:58:52.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.200+0000 7f045dfeb740 1 -- 172.21.15.43:0/4280234875 wait complete. 2024-04-25T08:58:52.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.201+0000 7f045dfeb740 20 client.7293 trim_cache size 0 max 16384 2024-04-25T08:58:52.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.216+0000 7f045dfeb740 1 Processor -- start 2024-04-25T08:58:52.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.216+0000 7f045dfeb740 1 -- start start 2024-04-25T08:58:52.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.217+0000 7f045dfeb740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561aea833230 0x561aea832a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.217+0000 7f045dfeb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561aea8344c0 0x561aea854be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.217+0000 7f045dfeb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aea850ad0 0x561aea8469e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.217+0000 7f045dfeb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04541305c0 con 0x561aea850ad0 2024-04-25T08:58:52.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.217+0000 7f045dfeb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0454131d30 con 0x561aea8344c0 2024-04-25T08:58:52.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.217+0000 7f045dfeb740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f045412ffc0 con 0x561aea833230 2024-04-25T08:58:52.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.217+0000 7f045988b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aea850ad0 0x561aea8469e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.217+0000 7f045988b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aea850ad0 0x561aea8469e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49146/0 (socket says 172.21.15.43:49146) 2024-04-25T08:58:52.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.217+0000 7f045988b640 1 -- 172.21.15.43:0/3185006505 learned_addr learned my addr 172.21.15.43:0/3185006505 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:52.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.217+0000 7f045908a640 1 --2- 172.21.15.43:0/3185006505 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561aea833230 0x561aea832a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.217+0000 7f0458889640 1 --2- 172.21.15.43:0/3185006505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561aea8344c0 0x561aea854be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.217+0000 7f045988b640 1 -- 172.21.15.43:0/3185006505 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561aea833230 msgr2=0x561aea832a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:52.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.217+0000 7f045988b640 1 --2- 172.21.15.43:0/3185006505 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561aea833230 0x561aea832a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.217+0000 7f045988b640 1 -- 172.21.15.43:0/3185006505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561aea8344c0 msgr2=0x561aea854be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:52.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.217+0000 7f045988b640 1 --2- 172.21.15.43:0/3185006505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561aea8344c0 0x561aea854be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.217+0000 7f045988b640 1 -- 172.21.15.43:0/3185006505 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561aea82bd10 con 0x561aea850ad0 2024-04-25T08:58:52.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.218+0000 7f045988b640 1 --2- 172.21.15.43:0/3185006505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aea850ad0 0x561aea8469e0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f044c009eb0 tx=0x7f044c0ab040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52045a146f5be227 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:52.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.218+0000 7f043e7fc640 1 -- 172.21.15.43:0/3185006505 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044c03f020 con 0x561aea850ad0 2024-04-25T08:58:52.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.218+0000 7f043e7fc640 1 -- 172.21.15.43:0/3185006505 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f044c082210 con 0x561aea850ad0 2024-04-25T08:58:52.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.218+0000 7f043e7fc640 1 -- 172.21.15.43:0/3185006505 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044c04a040 con 0x561aea850ad0 2024-04-25T08:58:52.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.218+0000 7f045dfeb740 1 -- 172.21.15.43:0/3185006505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aea850ad0 msgr2=0x561aea8469e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:52.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.218+0000 7f045dfeb740 1 --2- 172.21.15.43:0/3185006505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aea850ad0 0x561aea8469e0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f044c009eb0 tx=0x7f044c0ab040 comp rx=0 tx=0).stop 2024-04-25T08:58:52.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.218+0000 7f045dfeb740 1 -- 172.21.15.43:0/3185006505 shutdown_connections 2024-04-25T08:58:52.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.218+0000 7f045dfeb740 1 --2- 172.21.15.43:0/3185006505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aea850ad0 0x561aea8469e0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.218+0000 7f045dfeb740 1 --2- 172.21.15.43:0/3185006505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561aea8344c0 0x561aea854be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.218+0000 7f045dfeb740 1 --2- 172.21.15.43:0/3185006505 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561aea833230 0x561aea832a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.218+0000 7f045dfeb740 1 -- 172.21.15.43:0/3185006505 >> 172.21.15.43:0/3185006505 conn(0x561aea808f10 msgr2=0x561aea8092f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:52.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.218+0000 7f045dfeb740 1 -- 172.21.15.43:0/3185006505 shutdown_connections 2024-04-25T08:58:52.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.218+0000 7f045dfeb740 1 -- 172.21.15.43:0/3185006505 wait complete. 2024-04-25T08:58:52.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.219+0000 7f045dfeb740 1 -- start start 2024-04-25T08:58:52.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.219+0000 7f045dfeb740 1 Processor -- start 2024-04-25T08:58:52.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.220+0000 7f045dfeb740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561aea833230 0x561aea8bd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.220+0000 7f045dfeb740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561aea8344c0 0x561aea835990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.220+0000 7f045dfeb740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aea850ad0 0x561aea835ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.220+0000 7f045dfeb740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aea862100 con 0x561aea850ad0 2024-04-25T08:58:52.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.220+0000 7f045dfeb740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x561aea835810 con 0x561aea833230 2024-04-25T08:58:52.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.220+0000 7f045dfeb740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x561aea8626d0 con 0x561aea8344c0 2024-04-25T08:58:52.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.220+0000 7f045dfeb740 10 client.0 _ll_register_callbacks cb 0x561aea803c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:58:52.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.220+0000 7f045988b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aea850ad0 0x561aea835ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.220+0000 7f045988b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aea850ad0 0x561aea835ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49156/0 (socket says 172.21.15.43:49156) 2024-04-25T08:58:52.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.220+0000 7f045988b640 1 -- 172.21.15.43:0/246025297 learned_addr learned my addr 172.21.15.43:0/246025297 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:52.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.220+0000 7f045908a640 1 --2- 172.21.15.43:0/246025297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561aea833230 0x561aea8bd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.220+0000 7f0458889640 1 --2- 172.21.15.43:0/246025297 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561aea8344c0 0x561aea835990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.220+0000 7f045988b640 1 -- 172.21.15.43:0/246025297 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561aea8344c0 msgr2=0x561aea835990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:52.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.220+0000 7f045988b640 1 --2- 172.21.15.43:0/246025297 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561aea8344c0 0x561aea835990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.220+0000 7f045988b640 1 -- 172.21.15.43:0/246025297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561aea833230 msgr2=0x561aea8bd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:52.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.220+0000 7f045988b640 1 --2- 172.21.15.43:0/246025297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561aea833230 0x561aea8bd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.220+0000 7f045988b640 1 -- 172.21.15.43:0/246025297 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561aea833ef0 con 0x561aea850ad0 2024-04-25T08:58:52.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.220+0000 7f045988b640 1 --2- 172.21.15.43:0/246025297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aea850ad0 0x561aea835ed0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f044c077db0 tx=0x7f044c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e44a5b838a567ad1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:52.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.220+0000 7f0433fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:58:52.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.220+0000 7f0433fff640 1 -- 172.21.15.43:0/246025297 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044c096060 con 0x561aea850ad0 2024-04-25T08:58:52.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.220+0000 7f0433fff640 1 -- 172.21.15.43:0/246025297 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f044c081d80 con 0x561aea850ad0 2024-04-25T08:58:52.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.221+0000 7f045dfeb740 1 -- 172.21.15.43:0/246025297 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f044c082210 con 0x561aea850ad0 2024-04-25T08:58:52.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.221+0000 7f0433fff640 1 -- 172.21.15.43:0/246025297 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f044c04a070 con 0x561aea850ad0 2024-04-25T08:58:52.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.221+0000 7f045dfeb740 10 client.7299 fetch_fsmap learned FSMap version 47 2024-04-25T08:58:52.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.221+0000 7f045dfeb740 1 -- 172.21.15.43:0/246025297 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x561aea833ef0 con 0x561aea850ad0 2024-04-25T08:58:52.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.221+0000 7f0433fff640 1 -- 172.21.15.43:0/246025297 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f044c082210 con 0x561aea850ad0 2024-04-25T08:58:52.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.221+0000 7f0433fff640 1 -- 172.21.15.43:0/246025297 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 47) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f044c07eb70 con 0x561aea850ad0 2024-04-25T08:58:52.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.221+0000 7f045dfeb740 10 client.7299 fetch_fsmap finished waiting for FSMap version 47 2024-04-25T08:58:52.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.221+0000 7f045dfeb740 10 client.7299 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T08:58:52.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.221+0000 7f045dfeb740 20 client.7299 populate_metadata read hostname 'smithi043' 2024-04-25T08:58:52.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.221+0000 7f045dfeb740 1 --2- 172.21.15.43:0/246025297 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x561aea806b90 0x561aea806f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.221+0000 7f045dfeb740 4 client.7299 mds_command: new command op to 7164 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T08:58:52.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.221+0000 7f045dfeb740 1 -- 172.21.15.43:0/246025297 --> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561aea848be0 con 0x561aea806b90 2024-04-25T08:58:52.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.221+0000 7f045908a640 1 --2- 172.21.15.43:0/246025297 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x561aea806b90 0x561aea806f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.221+0000 7f045908a640 1 --2- 172.21.15.43:0/246025297 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x561aea806b90 0x561aea806f70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=488b9c11e939a437 server_cookie=d389e33b35268b2d in_seq=0 out_seq=0 2024-04-25T08:58:52.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.221+0000 7f0433fff640 10 client.7299 ms_handle_connect on v2:172.21.15.193:6835/3458098162 2024-04-25T08:58:52.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.222+0000 7f0433fff640 1 -- 172.21.15.43:0/246025297 <== mds.0 v2:172.21.15.193:6835/3458098162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x561aea848be0 con 0x561aea806b90 2024-04-25T08:58:52.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.222+0000 7f0433fff640 10 client.7299 handle_command_reply: tid=0 2024-04-25T08:58:52.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.222+0000 7f045dfeb740 1 client.7299 shutdown 2024-04-25T08:58:52.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.222+0000 7f045dfeb740 1 -- 172.21.15.43:0/246025297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aea850ad0 msgr2=0x561aea835ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:52.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.222+0000 7f045dfeb740 1 --2- 172.21.15.43:0/246025297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aea850ad0 0x561aea835ed0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f044c077db0 tx=0x7f044c009750 comp rx=0 tx=0).stop 2024-04-25T08:58:52.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.222+0000 7f045dfeb740 1 -- 172.21.15.43:0/246025297 shutdown_connections 2024-04-25T08:58:52.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.222+0000 7f045dfeb740 1 --2- 172.21.15.43:0/246025297 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x561aea806b90 0x561aea806f70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.222+0000 7f045dfeb740 1 --2- 172.21.15.43:0/246025297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561aea850ad0 0x561aea835ed0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.222+0000 7f045dfeb740 1 --2- 172.21.15.43:0/246025297 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561aea8344c0 0x561aea835990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.222+0000 7f045dfeb740 1 --2- 172.21.15.43:0/246025297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561aea833230 0x561aea8bd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.222+0000 7f0433fff640 0 client.7299 ms_handle_reset on v2:172.21.15.193:6835/3458098162 2024-04-25T08:58:52.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.222+0000 7f045dfeb740 1 -- 172.21.15.43:0/246025297 >> 172.21.15.43:0/246025297 conn(0x561aea808f10 msgr2=0x561aea854a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:52.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.222+0000 7f045dfeb740 1 -- 172.21.15.43:0/246025297 shutdown_connections 2024-04-25T08:58:52.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.222+0000 7f045dfeb740 1 -- 172.21.15.43:0/246025297 wait complete. 2024-04-25T08:58:52.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.222+0000 7f045dfeb740 20 client.7299 trim_cache size 0 max 16384 2024-04-25T08:58:52.217 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T08:58:52.217 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "6c4bd50e-aa00-4a7f-8d09-352344ef5a4b", 2024-04-25T08:58:52.218 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-04-25T08:58:52.218 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 7164, 2024-04-25T08:58:52.218 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-04-25T08:58:52.218 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-04-25T08:58:52.218 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-04-25T08:58:52.218 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 5.6788739340000003, 2024-04-25T08:58:52.218 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 47, 2024-04-25T08:58:52.218 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 73, 2024-04-25T08:58:52.218 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 73, 2024-04-25T08:58:52.218 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 10.68172049 2024-04-25T08:58:52.218 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T08:58:52.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.223+0000 7f045b2ee640 1 -- 172.21.15.43:0/2822791246 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f043806b4d0 msgr2=0x7f043806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:52.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.223+0000 7f045b2ee640 1 --2- 172.21.15.43:0/2822791246 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f043806b4d0 0x7f043806d990 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f04500017e0 tx=0x7f0450009040 comp rx=0 tx=0).stop 2024-04-25T08:58:52.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.223+0000 7f045b2ee640 1 -- 172.21.15.43:0/2822791246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04540cd010 msgr2=0x7f04540ac220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:52.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.223+0000 7f045b2ee640 1 --2- 172.21.15.43:0/2822791246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04540cd010 0x7f04540ac220 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f044c009ee0 tx=0x7f044c004a40 comp rx=0 tx=0).stop 2024-04-25T08:58:52.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.223+0000 7f045b2ee640 1 -- 172.21.15.43:0/2822791246 shutdown_connections 2024-04-25T08:58:52.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.223+0000 7f045b2ee640 1 --2- 172.21.15.43:0/2822791246 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04540cd010 0x7f04540ac220 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.223+0000 7f045b2ee640 1 --2- 172.21.15.43:0/2822791246 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f04540ccbb0 0x7f04540ae290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.223+0000 7f045b2ee640 1 --2- 172.21.15.43:0/2822791246 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f043806b4d0 0x7f043806d990 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.223+0000 7f045b2ee640 1 --2- 172.21.15.43:0/2822791246 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04540cbfc0 0x7f045415cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.223+0000 7f045b2ee640 1 -- 172.21.15.43:0/2822791246 >> 172.21.15.43:0/2822791246 conn(0x7f04540b1480 msgr2=0x7f045406d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:52.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.223+0000 7f045b2ee640 1 -- 172.21.15.43:0/2822791246 shutdown_connections 2024-04-25T08:58:52.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.223+0000 7f045b2ee640 1 -- 172.21.15.43:0/2822791246 wait complete. 2024-04-25T08:58:52.223 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T08:58:52.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.334+0000 7fb57a676640 1 Processor -- start 2024-04-25T08:58:52.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.334+0000 7fb57a676640 1 -- start start 2024-04-25T08:58:52.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.335+0000 7fb57a676640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5740cfd70 0x7fb5740d01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.335+0000 7fb57a676640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5740cc760 con 0x7fb5740b0e50 2024-04-25T08:58:52.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.335+0000 7fb57a676640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5740cca40 con 0x7fb57415be60 2024-04-25T08:58:52.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.335+0000 7fb57a676640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb57415c240 con 0x7fb5740cfd70 2024-04-25T08:58:52.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.335+0000 7fb578c13640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb5740b0e50 0x7fb5740b12e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45344/0 (socket says 172.21.15.43:45344) 2024-04-25T08:58:52.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.335+0000 7fb578c13640 1 -- 172.21.15.43:0/4257483094 learned_addr learned my addr 172.21.15.43:0/4257483094 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:58:52.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.335+0000 7fb5737fe640 1 --2- 172.21.15.43:0/4257483094 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5740cfd70 0x7fb5740d01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.335+0000 7fb572ffd640 1 -- 172.21.15.43:0/4257483094 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 565769531 0 0) 0x7fb5740cc760 con 0x7fb5740b0e50 2024-04-25T08:58:52.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.335+0000 7fb572ffd640 1 -- 172.21.15.43:0/4257483094 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb560003520 con 0x7fb5740b0e50 2024-04-25T08:58:52.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.335+0000 7fb572ffd640 1 -- 172.21.15.43:0/4257483094 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1107458094 0 0) 0x7fb5740cca40 con 0x7fb57415be60 2024-04-25T08:58:52.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.335+0000 7fb572ffd640 1 -- 172.21.15.43:0/4257483094 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5740cc760 con 0x7fb57415be60 2024-04-25T08:58:52.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.336+0000 7fb5737fe640 1 -- 172.21.15.43:0/4257483094 >> v1:172.21.15.193:6789/0 conn(0x7fb57415be60 legacy=0x7fb5740ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:52.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.336+0000 7fb5737fe640 1 -- 172.21.15.43:0/4257483094 >> v1:172.21.15.43:6789/0 conn(0x7fb5740b0e50 legacy=0x7fb5740b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:52.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.336+0000 7fb572ffd640 1 -- 172.21.15.43:0/4257483094 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 473293573 0 0) 0x7fb560003520 con 0x7fb5740b0e50 2024-04-25T08:58:52.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.336+0000 7fb5737fe640 1 -- 172.21.15.43:0/4257483094 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5740ccd20 con 0x7fb5740cfd70 2024-04-25T08:58:52.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.336+0000 7fb5737fe640 1 --2- 172.21.15.43:0/4257483094 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5740cfd70 0x7fb5740d01d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb55c0097c0 tx=0x7fb55c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cf8e775de64fb8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:52.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.336+0000 7fb572ffd640 1 -- 172.21.15.43:0/4257483094 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c037070 con 0x7fb5740cfd70 2024-04-25T08:58:52.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.337+0000 7fb572ffd640 1 -- 172.21.15.43:0/4257483094 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb55c02ac10 con 0x7fb5740cfd70 2024-04-25T08:58:52.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.337+0000 7fb572ffd640 1 -- 172.21.15.43:0/4257483094 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb56c0019f0 con 0x7fb5740cfd70 2024-04-25T08:58:52.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.337+0000 7fb57a676640 1 -- 172.21.15.43:0/4257483094 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5740cfd70 msgr2=0x7fb5740d01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:52.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.337+0000 7fb57a676640 1 --2- 172.21.15.43:0/4257483094 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5740cfd70 0x7fb5740d01d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb55c0097c0 tx=0x7fb55c02a180 comp rx=0 tx=0).stop 2024-04-25T08:58:52.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.337+0000 7fb57a676640 1 -- 172.21.15.43:0/4257483094 shutdown_connections 2024-04-25T08:58:52.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.337+0000 7fb57a676640 1 --2- 172.21.15.43:0/4257483094 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5740cfd70 0x7fb5740d01d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.337+0000 7fb57a676640 1 -- 172.21.15.43:0/4257483094 >> 172.21.15.43:0/4257483094 conn(0x7fb5740b32e0 msgr2=0x7fb5740b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:52.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.337+0000 7fb57a676640 1 -- 172.21.15.43:0/4257483094 shutdown_connections 2024-04-25T08:58:52.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.337+0000 7fb57a676640 1 -- 172.21.15.43:0/4257483094 wait complete. 2024-04-25T08:58:52.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.338+0000 7fb57a676640 1 Processor -- start 2024-04-25T08:58:52.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.338+0000 7fb57a676640 1 -- start start 2024-04-25T08:58:52.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.339+0000 7fb57a676640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5740b0e50 0x7fb57406bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.339+0000 7fb57a676640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5740cfd70 0x7fb5740ca850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.339+0000 7fb57a676640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb57415be60 0x7fb574152530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.339+0000 7fb57a676640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5740cc330 con 0x7fb5740cfd70 2024-04-25T08:58:52.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.339+0000 7fb57a676640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5740cc4b0 con 0x7fb57415be60 2024-04-25T08:58:52.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.339+0000 7fb57a676640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5740cef90 con 0x7fb5740b0e50 2024-04-25T08:58:52.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.339+0000 7fb5737fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5740cfd70 0x7fb5740ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.339+0000 7fb5737fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5740cfd70 0x7fb5740ca850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49168/0 (socket says 172.21.15.43:49168) 2024-04-25T08:58:52.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.339+0000 7fb5737fe640 1 -- 172.21.15.43:0/2012051227 learned_addr learned my addr 172.21.15.43:0/2012051227 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:52.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.340+0000 7fb573fff640 1 --2- 172.21.15.43:0/2012051227 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5740b0e50 0x7fb57406bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.340+0000 7fb5737fe640 1 -- 172.21.15.43:0/2012051227 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5740b0e50 msgr2=0x7fb57406bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:52.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.340+0000 7fb578c13640 1 --2- 172.21.15.43:0/2012051227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb57415be60 0x7fb574152530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.340+0000 7fb5737fe640 1 --2- 172.21.15.43:0/2012051227 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5740b0e50 0x7fb57406bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.340+0000 7fb5737fe640 1 -- 172.21.15.43:0/2012051227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb57415be60 msgr2=0x7fb574152530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:52.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.340+0000 7fb5737fe640 1 --2- 172.21.15.43:0/2012051227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb57415be60 0x7fb574152530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.340+0000 7fb5737fe640 1 -- 172.21.15.43:0/2012051227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5740cad90 con 0x7fb5740cfd70 2024-04-25T08:58:52.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.340+0000 7fb573fff640 1 --2- 172.21.15.43:0/2012051227 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5740b0e50 0x7fb57406bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:52.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.340+0000 7fb578c13640 1 --2- 172.21.15.43:0/2012051227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb57415be60 0x7fb574152530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:58:52.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.340+0000 7fb5737fe640 1 --2- 172.21.15.43:0/2012051227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5740cfd70 0x7fb5740ca850 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fb55c035040 tx=0x7fb55c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:52.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.340+0000 7fb570ff9640 1 -- 172.21.15.43:0/2012051227 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c037040 con 0x7fb5740cfd70 2024-04-25T08:58:52.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.340+0000 7fb570ff9640 1 -- 172.21.15.43:0/2012051227 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb55c005990 con 0x7fb5740cfd70 2024-04-25T08:58:52.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.340+0000 7fb57a676640 1 -- 172.21.15.43:0/2012051227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb57406c2d0 con 0x7fb5740cfd70 2024-04-25T08:58:52.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.340+0000 7fb570ff9640 1 -- 172.21.15.43:0/2012051227 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55c005d80 con 0x7fb5740cfd70 2024-04-25T08:58:52.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.340+0000 7fb57a676640 1 -- 172.21.15.43:0/2012051227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb57406c890 con 0x7fb5740cfd70 2024-04-25T08:58:52.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.341+0000 7fb57a676640 1 -- 172.21.15.43:0/2012051227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb574074ef0 con 0x7fb5740cfd70 2024-04-25T08:58:52.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.342+0000 7fb570ff9640 1 -- 172.21.15.43:0/2012051227 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb55c03a070 con 0x7fb5740cfd70 2024-04-25T08:58:52.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.342+0000 7fb570ff9640 1 --2- 172.21.15.43:0/2012051227 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb55406b4d0 0x7fb55406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:52.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.342+0000 7fb570ff9640 1 -- 172.21.15.43:0/2012051227 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb55c0aa4a0 con 0x7fb5740cfd70 2024-04-25T08:58:52.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.342+0000 7fb573fff640 1 --2- 172.21.15.43:0/2012051227 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb55406b4d0 0x7fb55406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:52.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.343+0000 7fb573fff640 1 --2- 172.21.15.43:0/2012051227 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb55406b4d0 0x7fb55406d990 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb5640011e0 tx=0x7fb564009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:52.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.346+0000 7fb570ff9640 1 -- 172.21.15.43:0/2012051227 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb55c041020 con 0x7fb5740cfd70 2024-04-25T08:58:52.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.565+0000 7fb57a676640 1 -- 172.21.15.43:0/2012051227 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb5740cfb50 con 0x7fb5740cfd70 2024-04-25T08:58:52.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.567+0000 7fb570ff9640 1 -- 172.21.15.43:0/2012051227 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v47) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb55c0793b0 con 0x7fb5740cfd70 2024-04-25T08:58:52.556 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T08:58:52.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.569+0000 7fb57a676640 1 -- 172.21.15.43:0/2012051227 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb55406b4d0 msgr2=0x7fb55406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:52.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.569+0000 7fb57a676640 1 --2- 172.21.15.43:0/2012051227 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb55406b4d0 0x7fb55406d990 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb5640011e0 tx=0x7fb564009040 comp rx=0 tx=0).stop 2024-04-25T08:58:52.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.569+0000 7fb57a676640 1 -- 172.21.15.43:0/2012051227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5740cfd70 msgr2=0x7fb5740ca850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:52.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.569+0000 7fb57a676640 1 --2- 172.21.15.43:0/2012051227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5740cfd70 0x7fb5740ca850 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fb55c035040 tx=0x7fb55c0046c0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.569+0000 7fb57a676640 1 -- 172.21.15.43:0/2012051227 shutdown_connections 2024-04-25T08:58:52.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.569+0000 7fb57a676640 1 --2- 172.21.15.43:0/2012051227 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb55406b4d0 0x7fb55406d990 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.569+0000 7fb57a676640 1 --2- 172.21.15.43:0/2012051227 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb57415be60 0x7fb574152530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.569+0000 7fb57a676640 1 --2- 172.21.15.43:0/2012051227 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5740cfd70 0x7fb5740ca850 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.569+0000 7fb57a676640 1 --2- 172.21.15.43:0/2012051227 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5740b0e50 0x7fb57406bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:52.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.569+0000 7fb57a676640 1 -- 172.21.15.43:0/2012051227 >> 172.21.15.43:0/2012051227 conn(0x7fb5740b32e0 msgr2=0x7fb5740cd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:52.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.569+0000 7fb57a676640 1 -- 172.21.15.43:0/2012051227 shutdown_connections 2024-04-25T08:58:52.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:52.569+0000 7fb57a676640 1 -- 172.21.15.43:0/2012051227 wait complete. 2024-04-25T08:58:52.570 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T08:58:52.570 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T08:58:52.570 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-04-25T08:58:52.570 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T08:58:52.570 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T08:58:52.570 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T08:58:52.570 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T08:58:52.570 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T08:58:52.570 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:58:52.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:52.571 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-04-25T08:58:52.587 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T08:58:52.588 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T08:58:52.588 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T08:58:52.588 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:58:52.588 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-04-25T08:58:52.588 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:58:52.588 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:58:52.588 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-04-25T08:58:52.588 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-04-25T08:58:52.588 INFO:teuthology.orchestra.run.smithi043.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T08:58:52.588 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:58:52.588 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-04-25T08:58:52.589 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T08:58:52.589 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 42485sec preferred_lft 42485sec 2024-04-25T08:58:52.589 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link noprefixroute 2024-04-25T08:58:52.589 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:58:52.589 INFO:teuthology.orchestra.run.smithi043.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:58:52.589 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-04-25T08:58:52.589 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-04-25T08:58:52.589 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T08:58:52.589 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:58:52.590 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-04-25T08:58:52.590 INFO:teuthology.orchestra.run.smithi043.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T08:58:52.590 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 12:2c:7a:72:36:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T08:58:52.590 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T08:58:52.590 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:58:52.590 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ccff:4eff:fe9d:c8bf/64 scope link 2024-04-25T08:58:52.590 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:58:52.590 INFO:teuthology.orchestra.run.smithi043.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:58:52.591 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 16:9a:63:37:46:24 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:58:52.591 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::149a:63ff:fe37:4624/64 scope link 2024-04-25T08:58:52.591 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:58:52.591 INFO:teuthology.orchestra.run.smithi043.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T08:58:52.591 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 12:2c:7a:72:36:14 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:58:52.591 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::102c:7aff:fe72:3614/64 scope link 2024-04-25T08:58:52.591 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T08:58:52.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:52.592 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T08:58:52.651 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-04-25T08:58:52.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:52.653 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-04-25T08:58:52.709 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-04-25T08:58:52.709 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T08:58:52.709 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T08:58:52.709 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T08:58:52.709 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T08:58:52.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:52.709 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2024-04-25T08:58:52.710 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T08:58:52.710 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T08:58:52.710 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T08:58:52.710 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T08:58:52.710 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T08:58:52.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:52.796 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-04-25T08:58:52.811 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T08:58:52.812 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T08:58:52.812 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:58:52.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:52.812 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-04-25T08:58:52.877 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T08:58:52.877 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:58:52.877 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T08:58:52.934 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T08:58:52.935 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T08:58:52.935 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:58:52.990 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-04-25T08:58:53.056 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:58:53.112 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-04-25T08:58:53.113 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-04-25T08:58:53.113 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-04-25T08:58:53.113 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-04-25T08:58:53.113 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-04-25T08:58:53.113 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-04-25T08:58:53.113 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-04-25T08:58:53.113 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-04-25T08:58:53.113 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-04-25T08:58:53.113 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-04-25T08:58:53.113 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-04-25T08:58:53.113 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/selinux 2024-04-25T08:58:53.113 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:58:53.113 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-04-25T08:58:53.113 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-04-25T08:58:53.113 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-04-25T08:58:53.114 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-04-25T08:58:53.114 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:58:53.114 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-04-25T08:58:53.114 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-04-25T08:58:53.114 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:58:53.114 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:58:53.114 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:58:53.114 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:58:53.114 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-04-25T08:58:53.114 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-04-25T08:58:53.114 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:58:53.114 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:58:53.114 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:58:53.115 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:58:53.115 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-04-25T08:58:53.115 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:58:53.115 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:58:53.115 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:58:53.115 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:58:53.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:53.115 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-04-25T08:58:53.168 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T08:58:53.169 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T08:58:53.212 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-04-25T08:58:53.236 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T08:58:53.292 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-04-25T08:58:53.292 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-04-25T08:58:53.292 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-04-25T08:58:53.292 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-04-25T08:58:53.292 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-04-25T08:58:53.293 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-04-25T08:58:53.293 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-04-25T08:58:53.293 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-04-25T08:58:53.293 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-04-25T08:58:53.293 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-04-25T08:58:53.293 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-04-25T08:58:53.293 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/selinux 2024-04-25T08:58:53.293 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T08:58:53.293 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-04-25T08:58:53.293 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-04-25T08:58:53.293 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-04-25T08:58:53.293 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-04-25T08:58:53.293 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T08:58:53.294 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-04-25T08:58:53.294 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-04-25T08:58:53.294 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T08:58:53.294 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T08:58:53.294 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T08:58:53.294 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T08:58:53.294 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-04-25T08:58:53.294 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-04-25T08:58:53.294 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T08:58:53.294 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T08:58:53.294 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T08:58:53.294 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T08:58:53.294 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-04-25T08:58:53.295 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:58:53.295 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T08:58:53.295 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:58:53.295 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T08:58:53.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:53.295 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-04-25T08:58:53.296 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:2024-04-25T08:58:53.303+0000 7f2037f1a480 -1 init, newargv = 0x55cb8abd4970 newargc=15 2024-04-25T08:58:53.296 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[67107]: starting ceph client 2024-04-25T08:58:53.298 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[67107]: starting fuse 2024-04-25T08:58:53.310 INFO:teuthology.orchestra.run.smithi043.stdout:43 2024-04-25T08:58:53.310 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-25T08:58:53.310 DEBUG:teuthology.orchestra.run.smithi043:> sudo stdin-killer -- python3 -c ' 2024-04-25T08:58:53.310 DEBUG:teuthology.orchestra.run.smithi043:> import glob 2024-04-25T08:58:53.311 DEBUG:teuthology.orchestra.run.smithi043:> import re 2024-04-25T08:58:53.311 DEBUG:teuthology.orchestra.run.smithi043:> import os 2024-04-25T08:58:53.311 DEBUG:teuthology.orchestra.run.smithi043:> import subprocess 2024-04-25T08:58:53.311 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T08:58:53.311 DEBUG:teuthology.orchestra.run.smithi043:> def _find_admin_socket(client_name): 2024-04-25T08:58:53.311 DEBUG:teuthology.orchestra.run.smithi043:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T08:58:53.311 DEBUG:teuthology.orchestra.run.smithi043:> files = glob.glob(asok_path) 2024-04-25T08:58:53.311 DEBUG:teuthology.orchestra.run.smithi043:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T08:58:53.311 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T08:58:53.311 DEBUG:teuthology.orchestra.run.smithi043:> # Given a non-glob path, it better be there 2024-04-25T08:58:53.311 DEBUG:teuthology.orchestra.run.smithi043:> if "*" not in asok_path: 2024-04-25T08:58:53.311 DEBUG:teuthology.orchestra.run.smithi043:> assert(len(files) == 1) 2024-04-25T08:58:53.311 DEBUG:teuthology.orchestra.run.smithi043:> return files[0] 2024-04-25T08:58:53.311 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T08:58:53.311 DEBUG:teuthology.orchestra.run.smithi043:> for f in files: 2024-04-25T08:58:53.312 DEBUG:teuthology.orchestra.run.smithi043:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T08:58:53.312 DEBUG:teuthology.orchestra.run.smithi043:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T08:58:53.312 DEBUG:teuthology.orchestra.run.smithi043:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T08:58:53.312 DEBUG:teuthology.orchestra.run.smithi043:> contents = proc_f.read() 2024-04-25T08:58:53.312 DEBUG:teuthology.orchestra.run.smithi043:> if mountpoint in contents: 2024-04-25T08:58:53.312 DEBUG:teuthology.orchestra.run.smithi043:> return f 2024-04-25T08:58:53.312 DEBUG:teuthology.orchestra.run.smithi043:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T08:58:53.312 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T08:58:53.312 DEBUG:teuthology.orchestra.run.smithi043:> print(_find_admin_socket("client.0")) 2024-04-25T08:58:53.312 DEBUG:teuthology.orchestra.run.smithi043:> ' 2024-04-25T08:58:53.429 INFO:teuthology.orchestra.run.smithi043.stdout:/var/run/ceph/ceph-client.0.67107.asok 2024-04-25T08:58:53.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:58:53.439 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.67107.asok 2024-04-25T08:58:53.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:53.440 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.67107.asok status 2024-04-25T08:58:53.579 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T08:58:53.580 INFO:teuthology.orchestra.run.smithi043.stdout: "metadata": { 2024-04-25T08:58:53.580 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T08:58:53.580 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T08:58:53.580 INFO:teuthology.orchestra.run.smithi043.stdout: "entity_id": "0", 2024-04-25T08:58:53.580 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": "smithi043", 2024-04-25T08:58:53.580 INFO:teuthology.orchestra.run.smithi043.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T08:58:53.580 INFO:teuthology.orchestra.run.smithi043.stdout: "pid": "67107", 2024-04-25T08:58:53.580 INFO:teuthology.orchestra.run.smithi043.stdout: "root": "/" 2024-04-25T08:58:53.580 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:58:53.580 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_count": 0, 2024-04-25T08:58:53.580 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_pinned_count": 0, 2024-04-25T08:58:53.580 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 7311, 2024-04-25T08:58:53.580 INFO:teuthology.orchestra.run.smithi043.stdout: "inst": { 2024-04-25T08:58:53.581 INFO:teuthology.orchestra.run.smithi043.stdout: "name": { 2024-04-25T08:58:53.581 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "client", 2024-04-25T08:58:53.581 INFO:teuthology.orchestra.run.smithi043.stdout: "num": 7311 2024-04-25T08:58:53.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:58:53.581 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-04-25T08:58:53.581 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-04-25T08:58:53.581 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:58:53.581 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 2136887169 2024-04-25T08:58:53.581 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:58:53.581 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:58:53.581 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-04-25T08:58:53.581 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-04-25T08:58:53.581 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2024-04-25T08:58:53.581 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 2136887169 2024-04-25T08:58:53.582 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:58:53.582 INFO:teuthology.orchestra.run.smithi043.stdout: "inst_str": "client.7311 192.168.0.1:0/2136887169", 2024-04-25T08:58:53.582 INFO:teuthology.orchestra.run.smithi043.stdout: "addr_str": "192.168.0.1:0/2136887169", 2024-04-25T08:58:53.582 INFO:teuthology.orchestra.run.smithi043.stdout: "inode_count": 1, 2024-04-25T08:58:53.582 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_epoch": 47, 2024-04-25T08:58:53.582 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch": 74, 2024-04-25T08:58:53.582 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch_barrier": 0, 2024-04-25T08:58:53.582 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklisted": false, 2024-04-25T08:58:53.582 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs" 2024-04-25T08:58:53.582 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T08:58:53.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:53.588 DEBUG:teuthology.orchestra.run.smithi043:> stat --file-system '--printf=%T 2024-04-25T08:58:53.588 DEBUG:teuthology.orchestra.run.smithi043:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T08:58:53.604 INFO:teuthology.orchestra.run.smithi043.stdout:fuseblk 2024-04-25T08:58:53.604 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T08:58:53.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:53.604 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T08:58:53.669 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2024-04-25T08:58:53.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.825+0000 7f49fe930640 1 Processor -- start 2024-04-25T08:58:53.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.825+0000 7f49fe930640 1 -- start start 2024-04-25T08:58:53.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.826+0000 7f49fe930640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f49f815b6c0 0x7f49f815baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:53.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.826+0000 7f49fe930640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f49f80b4ea0 0x7f49f80d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:53.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.826+0000 7f49fe930640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49f80b37a0 con 0x7f49f815e410 2024-04-25T08:58:53.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.826+0000 7f49fe930640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49f815fce0 con 0x7f49f815b6c0 2024-04-25T08:58:53.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.826+0000 7f49fe930640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49f815eab0 con 0x7f49f80b4ea0 2024-04-25T08:58:53.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.826+0000 7f49f7fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f49f815b6c0 0x7f49f815baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:53.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.826+0000 7f49f7fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f49f815b6c0 0x7f49f815baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49180/0 (socket says 172.21.15.43:49180) 2024-04-25T08:58:53.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.826+0000 7f49f7fff640 1 -- 172.21.15.43:0/229363657 learned_addr learned my addr 172.21.15.43:0/229363657 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:53.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.826+0000 7f49fcecd640 1 --2- 172.21.15.43:0/229363657 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f49f80b4ea0 0x7f49f80d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:53.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.826+0000 7f49f7fff640 1 -- 172.21.15.43:0/229363657 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f49f80b4ea0 msgr2=0x7f49f80d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:53.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.826+0000 7f49f7fff640 1 --2- 172.21.15.43:0/229363657 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f49f80b4ea0 0x7f49f80d3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:53.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.826+0000 7f49f7fff640 1 -- 172.21.15.43:0/229363657 >> v1:172.21.15.193:6789/0 conn(0x7f49f815e410 legacy=0x7f49f815f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:58:53.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.826+0000 7f49f7fff640 1 -- 172.21.15.43:0/229363657 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f815ed90 con 0x7f49f815b6c0 2024-04-25T08:58:53.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.827+0000 7f49f7fff640 1 --2- 172.21.15.43:0/229363657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49f815b6c0 0x7f49f815baa0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f49e8009870 tx=0x7f49e802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c261f92052e2ae9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:53.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.827+0000 7f49f6ffd640 1 -- 172.21.15.43:0/229363657 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e8037070 con 0x7f49f815b6c0 2024-04-25T08:58:53.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.827+0000 7f49f6ffd640 1 -- 172.21.15.43:0/229363657 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49e8032440 con 0x7f49f815b6c0 2024-04-25T08:58:53.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.827+0000 7f49f6ffd640 1 -- 172.21.15.43:0/229363657 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e8032810 con 0x7f49f815b6c0 2024-04-25T08:58:53.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.828+0000 7f49fe930640 1 -- 172.21.15.43:0/229363657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49f815b6c0 msgr2=0x7f49f815baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:53.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.828+0000 7f49fe930640 1 --2- 172.21.15.43:0/229363657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49f815b6c0 0x7f49f815baa0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f49e8009870 tx=0x7f49e802a270 comp rx=0 tx=0).stop 2024-04-25T08:58:53.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.828+0000 7f49fe930640 1 -- 172.21.15.43:0/229363657 shutdown_connections 2024-04-25T08:58:53.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.828+0000 7f49fe930640 1 --2- 172.21.15.43:0/229363657 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f49f80b4ea0 0x7f49f80d3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:53.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.828+0000 7f49fe930640 1 --2- 172.21.15.43:0/229363657 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49f815b6c0 0x7f49f815baa0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:53.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.828+0000 7f49fe930640 1 -- 172.21.15.43:0/229363657 >> 172.21.15.43:0/229363657 conn(0x7f49f808d460 msgr2=0x7f49f809cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:53.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.828+0000 7f49fe930640 1 -- 172.21.15.43:0/229363657 shutdown_connections 2024-04-25T08:58:53.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.828+0000 7f49fe930640 1 -- 172.21.15.43:0/229363657 wait complete. 2024-04-25T08:58:53.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.829+0000 7f49fe930640 1 Processor -- start 2024-04-25T08:58:53.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.829+0000 7f49fe930640 1 -- start start 2024-04-25T08:58:53.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.830+0000 7f49fe930640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49f80b4ea0 0x7f49f81f7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:53.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.830+0000 7f49fe930640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f49f815b6c0 0x7f49f80c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:53.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.830+0000 7f49fe930640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49f815e410 0x7f49f80b56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:53.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.830+0000 7f49fe930640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f80b52f0 con 0x7f49f815e410 2024-04-25T08:58:53.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.830+0000 7f49fe930640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f815f350 con 0x7f49f80b4ea0 2024-04-25T08:58:53.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.830+0000 7f49fe930640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49f80d3ca0 con 0x7f49f815b6c0 2024-04-25T08:58:53.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.830+0000 7f49fcecd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49f815e410 0x7f49f80b56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:53.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.830+0000 7f49fcecd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49f815e410 0x7f49f80b56e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49184/0 (socket says 172.21.15.43:49184) 2024-04-25T08:58:53.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.830+0000 7f49fcecd640 1 -- 172.21.15.43:0/3077686559 learned_addr learned my addr 172.21.15.43:0/3077686559 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:58:53.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.830+0000 7f49f7fff640 1 --2- 172.21.15.43:0/3077686559 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49f80b4ea0 0x7f49f81f7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:53.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.830+0000 7f49f77fe640 1 --2- 172.21.15.43:0/3077686559 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f49f815b6c0 0x7f49f80c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:53.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.830+0000 7f49fcecd640 1 -- 172.21.15.43:0/3077686559 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f49f815b6c0 msgr2=0x7f49f80c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:53.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.830+0000 7f49fcecd640 1 --2- 172.21.15.43:0/3077686559 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f49f815b6c0 0x7f49f80c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:53.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.830+0000 7f49fcecd640 1 -- 172.21.15.43:0/3077686559 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49f80b4ea0 msgr2=0x7f49f81f7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:53.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.830+0000 7f49fcecd640 1 --2- 172.21.15.43:0/3077686559 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49f80b4ea0 0x7f49f81f7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:53.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.830+0000 7f49fcecd640 1 -- 172.21.15.43:0/3077686559 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f81f7f90 con 0x7f49f815e410 2024-04-25T08:58:53.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.830+0000 7f49fcecd640 1 --2- 172.21.15.43:0/3077686559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49f815e410 0x7f49f80b56e0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f49f0007e70 tx=0x7f49f000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:53.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.831+0000 7f49f4ff9640 1 -- 172.21.15.43:0/3077686559 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f0014620 con 0x7f49f815e410 2024-04-25T08:58:53.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.831+0000 7f49f4ff9640 1 -- 172.21.15.43:0/3077686559 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49f0019070 con 0x7f49f815e410 2024-04-25T08:58:53.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.831+0000 7f49fe930640 1 -- 172.21.15.43:0/3077686559 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49f81f76f0 con 0x7f49f815e410 2024-04-25T08:58:53.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.831+0000 7f49f4ff9640 1 -- 172.21.15.43:0/3077686559 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f0014a20 con 0x7f49f815e410 2024-04-25T08:58:53.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.831+0000 7f49fe930640 1 -- 172.21.15.43:0/3077686559 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49f81f7cb0 con 0x7f49f815e410 2024-04-25T08:58:53.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.832+0000 7f49fe930640 1 -- 172.21.15.43:0/3077686559 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49f8068a40 con 0x7f49f815e410 2024-04-25T08:58:53.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.832+0000 7f49f4ff9640 1 -- 172.21.15.43:0/3077686559 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f49f0014bc0 con 0x7f49f815e410 2024-04-25T08:58:53.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.833+0000 7f49f4ff9640 1 --2- 172.21.15.43:0/3077686559 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f49d806b4d0 0x7f49d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:58:53.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.833+0000 7f49f4ff9640 1 -- 172.21.15.43:0/3077686559 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f49f008abe0 con 0x7f49f815e410 2024-04-25T08:58:53.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.833+0000 7f49f7fff640 1 --2- 172.21.15.43:0/3077686559 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f49d806b4d0 0x7f49d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:58:53.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.833+0000 7f49f7fff640 1 --2- 172.21.15.43:0/3077686559 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f49d806b4d0 0x7f49d806d990 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f49e8034000 tx=0x7f49e802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:58:53.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:53.837+0000 7f49f4ff9640 1 -- 172.21.15.43:0/3077686559 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f49f0059980 con 0x7f49f815e410 2024-04-25T08:58:54.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:54.055+0000 7f49fe930640 1 -- 172.21.15.43:0/3077686559 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f49f815d2e0 con 0x7f49f815e410 2024-04-25T08:58:54.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:54.577+0000 7f49f4ff9640 1 -- 172.21.15.43:0/3077686559 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v48) v1 ==== 167+0+0 (secure 0 0 0) 0x7f49f0059280 con 0x7f49f815e410 2024-04-25T08:58:54.566 INFO:teuthology.orchestra.run.smithi043.stderr:enabled new snapshots 2024-04-25T08:58:54.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:54.579+0000 7f49fe930640 1 -- 172.21.15.43:0/3077686559 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f49d806b4d0 msgr2=0x7f49d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:54.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:54.579+0000 7f49fe930640 1 --2- 172.21.15.43:0/3077686559 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f49d806b4d0 0x7f49d806d990 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f49e8034000 tx=0x7f49e802e000 comp rx=0 tx=0).stop 2024-04-25T08:58:54.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:54.579+0000 7f49fe930640 1 -- 172.21.15.43:0/3077686559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49f815e410 msgr2=0x7f49f80b56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:58:54.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:54.579+0000 7f49fe930640 1 --2- 172.21.15.43:0/3077686559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49f815e410 0x7f49f80b56e0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f49f0007e70 tx=0x7f49f000e4b0 comp rx=0 tx=0).stop 2024-04-25T08:58:54.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:54.579+0000 7f49fe930640 1 -- 172.21.15.43:0/3077686559 shutdown_connections 2024-04-25T08:58:54.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:54.579+0000 7f49fe930640 1 --2- 172.21.15.43:0/3077686559 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f49f815e410 0x7f49f80b56e0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:54.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:54.579+0000 7f49fe930640 1 --2- 172.21.15.43:0/3077686559 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f49f815b6c0 0x7f49f80c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:54.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:54.579+0000 7f49fe930640 1 --2- 172.21.15.43:0/3077686559 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f49d806b4d0 0x7f49d806d990 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:54.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:54.579+0000 7f49fe930640 1 --2- 172.21.15.43:0/3077686559 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f49f80b4ea0 0x7f49f81f7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:58:54.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:54.579+0000 7f49fe930640 1 -- 172.21.15.43:0/3077686559 >> 172.21.15.43:0/3077686559 conn(0x7f49f808d460 msgr2=0x7f49f815dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:58:54.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:54.579+0000 7f49fe930640 1 -- 172.21.15.43:0/3077686559 shutdown_connections 2024-04-25T08:58:54.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:58:54.579+0000 7f49fe930640 1 -- 172.21.15.43:0/3077686559 wait complete. 2024-04-25T08:58:54.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:58:54.580 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir stray_eval_dir) 2024-04-25T08:58:54.603 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T08:58:54.603 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T08:58:54.660 DEBUG:teuthology.orchestra.run.smithi043:> stdin-killer -- python3 -c ' 2024-04-25T08:58:54.660 DEBUG:teuthology.orchestra.run.smithi043:> import os 2024-04-25T08:58:54.660 DEBUG:teuthology.orchestra.run.smithi043:> import uuid 2024-04-25T08:58:54.660 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T08:58:54.660 DEBUG:teuthology.orchestra.run.smithi043:> n = 2000 2024-04-25T08:58:54.660 DEBUG:teuthology.orchestra.run.smithi043:> create_hard_links = False 2024-04-25T08:58:54.660 DEBUG:teuthology.orchestra.run.smithi043:> if 3 > 0: 2024-04-25T08:58:54.660 DEBUG:teuthology.orchestra.run.smithi043:> create_hard_links = True 2024-04-25T08:58:54.660 DEBUG:teuthology.orchestra.run.smithi043:> path = "/home/ubuntu/cephtest/mnt.0/stray_eval_dir/file" 2024-04-25T08:58:54.660 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T08:58:54.660 DEBUG:teuthology.orchestra.run.smithi043:> dpath = os.path.dirname(path) 2024-04-25T08:58:54.660 DEBUG:teuthology.orchestra.run.smithi043:> fnameprefix = os.path.basename(path) 2024-04-25T08:58:54.660 DEBUG:teuthology.orchestra.run.smithi043:> os.makedirs(dpath, exist_ok=True) 2024-04-25T08:58:54.661 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T08:58:54.661 DEBUG:teuthology.orchestra.run.smithi043:> try: 2024-04-25T08:58:54.661 DEBUG:teuthology.orchestra.run.smithi043:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-25T08:58:54.661 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T08:58:54.661 DEBUG:teuthology.orchestra.run.smithi043:> for i in range(n): 2024-04-25T08:58:54.661 DEBUG:teuthology.orchestra.run.smithi043:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-25T08:58:54.661 DEBUG:teuthology.orchestra.run.smithi043:> with open(fpath, '"'"'w'"'"') as f: 2024-04-25T08:58:54.661 DEBUG:teuthology.orchestra.run.smithi043:> f.write(f"{i}") 2024-04-25T08:58:54.661 DEBUG:teuthology.orchestra.run.smithi043:> if False: 2024-04-25T08:58:54.661 DEBUG:teuthology.orchestra.run.smithi043:> f.flush() 2024-04-25T08:58:54.661 DEBUG:teuthology.orchestra.run.smithi043:> os.fsync(f.fileno()) 2024-04-25T08:58:54.661 DEBUG:teuthology.orchestra.run.smithi043:> if False: 2024-04-25T08:58:54.661 DEBUG:teuthology.orchestra.run.smithi043:> os.unlink(fpath) 2024-04-25T08:58:54.661 DEBUG:teuthology.orchestra.run.smithi043:> if False: 2024-04-25T08:58:54.661 DEBUG:teuthology.orchestra.run.smithi043:> os.fsync(dirfd) 2024-04-25T08:58:54.661 DEBUG:teuthology.orchestra.run.smithi043:> if create_hard_links: 2024-04-25T08:58:54.662 DEBUG:teuthology.orchestra.run.smithi043:> for j in range(3): 2024-04-25T08:58:54.662 DEBUG:teuthology.orchestra.run.smithi043:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-25T08:58:54.662 DEBUG:teuthology.orchestra.run.smithi043:> if False: 2024-04-25T08:58:54.662 DEBUG:teuthology.orchestra.run.smithi043:> os.fsync(dirfd) 2024-04-25T08:58:54.662 DEBUG:teuthology.orchestra.run.smithi043:> finally: 2024-04-25T08:58:54.662 DEBUG:teuthology.orchestra.run.smithi043:> os.close(dirfd) 2024-04-25T08:58:54.662 DEBUG:teuthology.orchestra.run.smithi043:> ' 2024-04-25T08:59:14.776 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:59:14.779 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:59:22.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T08:59:22.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:22.792 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T08:59:22.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:22.842 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec find /home/ubuntu/cephtest/mnt.0/stray_eval_dir/ -type f -delete) 2024-04-25T08:59:45.085 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:59:45.088 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T08:59:54.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:54.105 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T08:59:54.168 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T08:59:54.168 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 perf dump 2024-04-25T08:59:54.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.281+0000 7f2b4c893640 1 Processor -- start 2024-04-25T08:59:54.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.281+0000 7f2b4c893640 1 -- start start 2024-04-25T08:59:54.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.282+0000 7f2b4c893640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2b4815e090 0x7f2b48161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.282+0000 7f2b4c893640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f2b480b5c70 0x7f2b480d2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.282+0000 7f2b4c893640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b48162610 con 0x7f2b48161fe0 2024-04-25T08:59:54.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.282+0000 7f2b4c893640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b480d1da0 con 0x7f2b4815e090 2024-04-25T08:59:54.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.282+0000 7f2b4c893640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b480d1750 con 0x7f2b480b5c70 2024-04-25T08:59:54.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.283+0000 7f2b4659c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2b4815e090 0x7f2b48161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.283+0000 7f2b4659c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2b4815e090 0x7f2b48161aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55210/0 (socket says 172.21.15.43:55210) 2024-04-25T08:59:54.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.283+0000 7f2b4659c640 1 -- 172.21.15.43:0/3145040454 learned_addr learned my addr 172.21.15.43:0/3145040454 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:59:54.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.283+0000 7f2b46d9d640 1 --2- 172.21.15.43:0/3145040454 >> v2:172.21.15.193:3300/0 conn(0x7f2b480b5c70 0x7f2b480d2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.283+0000 7f2b4659c640 1 -- 172.21.15.43:0/3145040454 >> v2:172.21.15.193:3300/0 conn(0x7f2b480b5c70 msgr2=0x7f2b480d2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:54.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.283+0000 7f2b4659c640 1 --2- 172.21.15.43:0/3145040454 >> v2:172.21.15.193:3300/0 conn(0x7f2b480b5c70 0x7f2b480d2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.283+0000 7f2b4659c640 1 -- 172.21.15.43:0/3145040454 >> v1:172.21.15.193:6789/0 conn(0x7f2b48161fe0 legacy=0x7f2b480b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:54.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.283+0000 7f2b4659c640 1 -- 172.21.15.43:0/3145040454 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b480d1a30 con 0x7f2b4815e090 2024-04-25T08:59:54.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.284+0000 7f2b4659c640 1 --2- 172.21.15.43:0/3145040454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b4815e090 0x7f2b48161aa0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f2b481619e0 tx=0x7f2b3802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a56e84c76162f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:54.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.284+0000 7f2b4559a640 1 -- 172.21.15.43:0/3145040454 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b38037070 con 0x7f2b4815e090 2024-04-25T08:59:54.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.284+0000 7f2b4559a640 1 -- 172.21.15.43:0/3145040454 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b3802ae50 con 0x7f2b4815e090 2024-04-25T08:59:54.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.284+0000 7f2b4559a640 1 -- 172.21.15.43:0/3145040454 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b38032860 con 0x7f2b4815e090 2024-04-25T08:59:54.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.284+0000 7f2b4c893640 1 -- 172.21.15.43:0/3145040454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b4815e090 msgr2=0x7f2b48161aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:54.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.284+0000 7f2b4c893640 1 --2- 172.21.15.43:0/3145040454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b4815e090 0x7f2b48161aa0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f2b481619e0 tx=0x7f2b3802a300 comp rx=0 tx=0).stop 2024-04-25T08:59:54.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.284+0000 7f2b4c893640 1 -- 172.21.15.43:0/3145040454 shutdown_connections 2024-04-25T08:59:54.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.284+0000 7f2b4c893640 1 --2- 172.21.15.43:0/3145040454 >> v2:172.21.15.193:3300/0 conn(0x7f2b480b5c70 0x7f2b480d2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.284+0000 7f2b4c893640 1 --2- 172.21.15.43:0/3145040454 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b4815e090 0x7f2b48161aa0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.284+0000 7f2b4c893640 1 -- 172.21.15.43:0/3145040454 >> 172.21.15.43:0/3145040454 conn(0x7f2b480d0560 msgr2=0x7f2b480d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:54.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.284+0000 7f2b4c893640 1 -- 172.21.15.43:0/3145040454 shutdown_connections 2024-04-25T08:59:54.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.284+0000 7f2b4c893640 1 -- 172.21.15.43:0/3145040454 wait complete. 2024-04-25T08:59:54.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.286+0000 7f2b4c893640 1 Processor -- start 2024-04-25T08:59:54.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.286+0000 7f2b4c893640 1 -- start start 2024-04-25T08:59:54.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.287+0000 7f2b4c893640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2b480b5c70 0x7f2b48153080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.287+0000 7f2b4c893640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b4815e090 0x7f2b48160500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.287+0000 7f2b4c893640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b48161fe0 0x7f2b48160a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.287+0000 7f2b4c893640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b480d2770 con 0x7f2b48161fe0 2024-04-25T08:59:54.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.287+0000 7f2b4c893640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b480b3280 con 0x7f2b4815e090 2024-04-25T08:59:54.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.287+0000 7f2b4c893640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b480d28f0 con 0x7f2b480b5c70 2024-04-25T08:59:54.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.287+0000 7f2b46d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b48161fe0 0x7f2b48160a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.287+0000 7f2b46d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b48161fe0 0x7f2b48160a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55214/0 (socket says 172.21.15.43:55214) 2024-04-25T08:59:54.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.287+0000 7f2b46d9d640 1 -- 172.21.15.43:0/2206435619 learned_addr learned my addr 172.21.15.43:0/2206435619 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:59:54.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.287+0000 7f2b4659c640 1 --2- 172.21.15.43:0/2206435619 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2b480b5c70 0x7f2b48153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.287+0000 7f2b45d9b640 1 --2- 172.21.15.43:0/2206435619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b4815e090 0x7f2b48160500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.287+0000 7f2b46d9d640 1 -- 172.21.15.43:0/2206435619 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2b480b5c70 msgr2=0x7f2b48153080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:54.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.287+0000 7f2b46d9d640 1 --2- 172.21.15.43:0/2206435619 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2b480b5c70 0x7f2b48153080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.287+0000 7f2b46d9d640 1 -- 172.21.15.43:0/2206435619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b4815e090 msgr2=0x7f2b48160500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:54.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.287+0000 7f2b46d9d640 1 --2- 172.21.15.43:0/2206435619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b4815e090 0x7f2b48160500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.287+0000 7f2b46d9d640 1 -- 172.21.15.43:0/2206435619 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b48156330 con 0x7f2b48161fe0 2024-04-25T08:59:54.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.287+0000 7f2b45d9b640 1 --2- 172.21.15.43:0/2206435619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b4815e090 0x7f2b48160500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:59:54.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.287+0000 7f2b46d9d640 1 --2- 172.21.15.43:0/2206435619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b48161fe0 0x7f2b48160a40 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f2b4000efc0 tx=0x7f2b4000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:54.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.287+0000 7f2b36ffd640 1 -- 172.21.15.43:0/2206435619 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b40018020 con 0x7f2b48161fe0 2024-04-25T08:59:54.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.287+0000 7f2b36ffd640 1 -- 172.21.15.43:0/2206435619 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b40014480 con 0x7f2b48161fe0 2024-04-25T08:59:54.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.287+0000 7f2b4c893640 1 -- 172.21.15.43:0/2206435619 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b48156610 con 0x7f2b48161fe0 2024-04-25T08:59:54.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.287+0000 7f2b36ffd640 1 -- 172.21.15.43:0/2206435619 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b40014870 con 0x7f2b48161fe0 2024-04-25T08:59:54.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.287+0000 7f2b4c893640 1 -- 172.21.15.43:0/2206435619 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b481535c0 con 0x7f2b48161fe0 2024-04-25T08:59:54.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.289+0000 7f2b36ffd640 1 -- 172.21.15.43:0/2206435619 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2b40005d10 con 0x7f2b48161fe0 2024-04-25T08:59:54.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.289+0000 7f2b36ffd640 1 --2- 172.21.15.43:0/2206435619 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2b2806b500 0x7f2b2806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.289+0000 7f2b36ffd640 1 -- 172.21.15.43:0/2206435619 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f2b4008ab90 con 0x7f2b48161fe0 2024-04-25T08:59:54.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.289+0000 7f2b4659c640 1 --2- 172.21.15.43:0/2206435619 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2b2806b500 0x7f2b2806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.290+0000 7f2b4659c640 1 --2- 172.21.15.43:0/2206435619 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2b2806b500 0x7f2b2806d9c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2b38034000 tx=0x7f2b3802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:54.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.290+0000 7f2b4f590740 1 Processor -- start 2024-04-25T08:59:54.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.291+0000 7f2b4f590740 1 -- start start 2024-04-25T08:59:54.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.291+0000 7f2b4f590740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b5bec1ca0 0x558b5bec3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.291+0000 7f2b4f590740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b5befe370 0x558b5bec2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.291+0000 7f2b4f590740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558b5beec760 0x558b5beecbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.291+0000 7f2b4f590740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b5be24b90 con 0x558b5befe370 2024-04-25T08:59:54.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.291+0000 7f2b4f590740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b5bdcde40 con 0x558b5bec1ca0 2024-04-25T08:59:54.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.291+0000 7f2b4f590740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x558b5be20c30 con 0x558b5beec760 2024-04-25T08:59:54.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.291+0000 7f2b45d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b5befe370 0x558b5bec2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.291+0000 7f2b45d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b5befe370 0x558b5bec2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55216/0 (socket says 172.21.15.43:55216) 2024-04-25T08:59:54.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.291+0000 7f2b45d9b640 1 -- 172.21.15.43:0/3990539822 learned_addr learned my addr 172.21.15.43:0/3990539822 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:59:54.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.291+0000 7f2b46d9d640 1 --2- 172.21.15.43:0/3990539822 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558b5beec760 0x558b5beecbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.291+0000 7f2b4659c640 1 --2- 172.21.15.43:0/3990539822 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b5bec1ca0 0x558b5bec3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.292+0000 7f2b45d9b640 1 -- 172.21.15.43:0/3990539822 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558b5beec760 msgr2=0x558b5beecbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:54.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.292+0000 7f2b45d9b640 1 --2- 172.21.15.43:0/3990539822 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558b5beec760 0x558b5beecbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.292+0000 7f2b45d9b640 1 -- 172.21.15.43:0/3990539822 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b5bec1ca0 msgr2=0x558b5bec3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:54.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.292+0000 7f2b45d9b640 1 --2- 172.21.15.43:0/3990539822 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b5bec1ca0 0x558b5bec3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.292+0000 7f2b45d9b640 1 -- 172.21.15.43:0/3990539822 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b5bc037d0 con 0x558b5befe370 2024-04-25T08:59:54.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.292+0000 7f2b46d9d640 1 --2- 172.21.15.43:0/3990539822 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558b5beec760 0x558b5beecbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:59:54.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.292+0000 7f2b45d9b640 1 --2- 172.21.15.43:0/3990539822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b5befe370 0x558b5bec2080 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c001580 tx=0x7f2b3c001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a3897a1aa28782 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:54.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.292+0000 7f2b27fff640 1 -- 172.21.15.43:0/3990539822 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c019070 con 0x558b5befe370 2024-04-25T08:59:54.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.292+0000 7f2b27fff640 1 -- 172.21.15.43:0/3990539822 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b3c0050d0 con 0x558b5befe370 2024-04-25T08:59:54.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.292+0000 7f2b27fff640 1 -- 172.21.15.43:0/3990539822 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c00e6e0 con 0x558b5befe370 2024-04-25T08:59:54.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.293+0000 7f2b4f590740 1 -- 172.21.15.43:0/3990539822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b5befe370 msgr2=0x558b5bec2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:54.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.293+0000 7f2b4f590740 1 --2- 172.21.15.43:0/3990539822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b5befe370 0x558b5bec2080 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c001580 tx=0x7f2b3c001a40 comp rx=0 tx=0).stop 2024-04-25T08:59:54.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.293+0000 7f2b4f590740 1 -- 172.21.15.43:0/3990539822 shutdown_connections 2024-04-25T08:59:54.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.293+0000 7f2b4f590740 1 --2- 172.21.15.43:0/3990539822 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558b5beec760 0x558b5beecbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.293+0000 7f2b4f590740 1 --2- 172.21.15.43:0/3990539822 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b5befe370 0x558b5bec2080 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.293+0000 7f2b4f590740 1 --2- 172.21.15.43:0/3990539822 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b5bec1ca0 0x558b5bec3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.293+0000 7f2b4f590740 1 -- 172.21.15.43:0/3990539822 >> 172.21.15.43:0/3990539822 conn(0x558b5be89870 msgr2=0x558b5bf07760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:54.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.293+0000 7f2b4f590740 1 -- 172.21.15.43:0/3990539822 shutdown_connections 2024-04-25T08:59:54.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.293+0000 7f2b4f590740 1 -- 172.21.15.43:0/3990539822 wait complete. 2024-04-25T08:59:54.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.293+0000 7f2b4f590740 1 -- start start 2024-04-25T08:59:54.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.293+0000 7f2b4f590740 1 Processor -- start 2024-04-25T08:59:54.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.294+0000 7f2b4f590740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b5bec1ca0 0x558b5bed3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.294+0000 7f2b4f590740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558b5beec760 0x558b5bee35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.294+0000 7f2b4f590740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b5befe370 0x558b5bee8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.294+0000 7f2b4f590740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b5bde13e0 con 0x558b5bec1ca0 2024-04-25T08:59:54.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.294+0000 7f2b4f590740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b5beea2b0 con 0x558b5befe370 2024-04-25T08:59:54.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.294+0000 7f2b4f590740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x558b5bf18740 con 0x558b5beec760 2024-04-25T08:59:54.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.294+0000 7f2b4f590740 10 client.0 _ll_register_callbacks cb 0x558b5bde0db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:59:54.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.294+0000 7f2b4659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b5bec1ca0 0x558b5bed3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.294+0000 7f2b4659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b5bec1ca0 0x558b5bed3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55220/0 (socket says 172.21.15.43:55220) 2024-04-25T08:59:54.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.294+0000 7f2b4659c640 1 -- 172.21.15.43:0/3436705513 learned_addr learned my addr 172.21.15.43:0/3436705513 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:59:54.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.294+0000 7f2b45d9b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558b5beec760 0x558b5bee35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.294+0000 7f2b46d9d640 1 --2- 172.21.15.43:0/3436705513 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b5befe370 0x558b5bee8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.295+0000 7f2b4659c640 1 -- 172.21.15.43:0/3436705513 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558b5beec760 msgr2=0x558b5bee35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:54.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.295+0000 7f2b4659c640 1 --2- 172.21.15.43:0/3436705513 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558b5beec760 0x558b5bee35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.295+0000 7f2b4659c640 1 -- 172.21.15.43:0/3436705513 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b5befe370 msgr2=0x558b5bee8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:54.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.295+0000 7f2b4659c640 1 --2- 172.21.15.43:0/3436705513 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b5befe370 0x558b5bee8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.295+0000 7f2b4659c640 1 -- 172.21.15.43:0/3436705513 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b5baf90f0 con 0x558b5bec1ca0 2024-04-25T08:59:54.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.295+0000 7f2b46d9d640 1 --2- 172.21.15.43:0/3436705513 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b5befe370 0x558b5bee8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:59:54.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.295+0000 7f2b45d9b640 1 --2- 172.21.15.43:0/3436705513 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558b5beec760 0x558b5bee35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:59:54.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.295+0000 7f2b4659c640 1 --2- 172.21.15.43:0/3436705513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b5bec1ca0 0x558b5bed3090 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f2b380090b0 tx=0x7f2b38004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c79d9f78dc2e1f93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:54.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.295+0000 7f2b257fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:59:54.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.295+0000 7f2b257fa640 1 -- 172.21.15.43:0/3436705513 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3804d070 con 0x558b5bec1ca0 2024-04-25T08:59:54.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.295+0000 7f2b4f590740 1 -- 172.21.15.43:0/3436705513 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2b3c0050d0 con 0x558b5bec1ca0 2024-04-25T08:59:54.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.295+0000 7f2b257fa640 1 -- 172.21.15.43:0/3436705513 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(75..75 src has 1..75) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f2b3803c850 con 0x558b5bec1ca0 2024-04-25T08:59:54.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.295+0000 7f2b257fa640 1 -- 172.21.15.43:0/3436705513 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b38058050 con 0x558b5bec1ca0 2024-04-25T08:59:54.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.295+0000 7f2b4f590740 10 client.7329 fetch_fsmap learned FSMap version 48 2024-04-25T08:59:54.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.295+0000 7f2b4f590740 1 -- 172.21.15.43:0/3436705513 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x558b5bc63f60 con 0x558b5bec1ca0 2024-04-25T08:59:54.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.295+0000 7f2b257fa640 1 -- 172.21.15.43:0/3436705513 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b3c0050d0 con 0x558b5bec1ca0 2024-04-25T08:59:54.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.296+0000 7f2b257fa640 1 -- 172.21.15.43:0/3436705513 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 48) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2b3805b020 con 0x558b5bec1ca0 2024-04-25T08:59:54.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.296+0000 7f2b4f590740 10 client.7329 fetch_fsmap finished waiting for FSMap version 48 2024-04-25T08:59:54.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.296+0000 7f2b4f590740 10 client.7329 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T08:59:54.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.296+0000 7f2b4f590740 20 client.7329 populate_metadata read hostname 'smithi043' 2024-04-25T08:59:54.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.296+0000 7f2b4f590740 1 --2- 172.21.15.43:0/3436705513 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x558b5bedf290 0x558b5bf17630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.296+0000 7f2b4f590740 4 client.7329 mds_command: new command op to 7164 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:59:54.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.296+0000 7f2b4f590740 1 -- 172.21.15.43:0/3436705513 --> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558b5be320d0 con 0x558b5bedf290 2024-04-25T08:59:54.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.296+0000 7f2b45d9b640 1 --2- 172.21.15.43:0/3436705513 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x558b5bedf290 0x558b5bf17630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.296+0000 7f2b45d9b640 1 --2- 172.21.15.43:0/3436705513 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x558b5bedf290 0x558b5bf17630 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58c3379d8de54f6 server_cookie=d29048d9b2ffc3d8 in_seq=0 out_seq=0 2024-04-25T08:59:54.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.296+0000 7f2b257fa640 10 client.7329 ms_handle_connect on v2:172.21.15.193:6835/3458098162 2024-04-25T08:59:54.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.297+0000 7f2b257fa640 1 -- 172.21.15.43:0/3436705513 <== mds.0 v2:172.21.15.193:6835/3458098162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558b5be320d0 con 0x558b5bedf290 2024-04-25T08:59:54.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.297+0000 7f2b257fa640 10 client.7329 handle_command_reply: tid=0 2024-04-25T08:59:54.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.297+0000 7f2b4f590740 1 client.7329 shutdown 2024-04-25T08:59:54.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.297+0000 7f2b4f590740 1 -- 172.21.15.43:0/3436705513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b5bec1ca0 msgr2=0x558b5bed3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:54.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.297+0000 7f2b4f590740 1 --2- 172.21.15.43:0/3436705513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b5bec1ca0 0x558b5bed3090 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f2b380090b0 tx=0x7f2b38004730 comp rx=0 tx=0).stop 2024-04-25T08:59:54.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.297+0000 7f2b4f590740 1 -- 172.21.15.43:0/3436705513 shutdown_connections 2024-04-25T08:59:54.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.297+0000 7f2b4f590740 1 --2- 172.21.15.43:0/3436705513 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x558b5bedf290 0x558b5bf17630 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.297+0000 7f2b4f590740 1 --2- 172.21.15.43:0/3436705513 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b5befe370 0x558b5bee8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.297+0000 7f2b4f590740 1 --2- 172.21.15.43:0/3436705513 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558b5beec760 0x558b5bee35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.297+0000 7f2b257fa640 0 client.7329 ms_handle_reset on v2:172.21.15.193:6835/3458098162 2024-04-25T08:59:54.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.297+0000 7f2b4f590740 1 --2- 172.21.15.43:0/3436705513 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b5bec1ca0 0x558b5bed3090 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.297+0000 7f2b4f590740 1 -- 172.21.15.43:0/3436705513 >> 172.21.15.43:0/3436705513 conn(0x558b5be89870 msgr2=0x558b5beea490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:54.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.297+0000 7f2b4f590740 1 -- 172.21.15.43:0/3436705513 shutdown_connections 2024-04-25T08:59:54.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.297+0000 7f2b4f590740 1 -- 172.21.15.43:0/3436705513 wait complete. 2024-04-25T08:59:54.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.297+0000 7f2b4f590740 20 client.7329 trim_cache size 0 max 16384 2024-04-25T08:59:54.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.311+0000 7f2b4f590740 1 Processor -- start 2024-04-25T08:59:54.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.311+0000 7f2b4f590740 1 -- start start 2024-04-25T08:59:54.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.312+0000 7f2b4f590740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b5bec1ca0 0x558b5bf685f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.312+0000 7f2b4f590740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b5bedf290 0x558b5bee93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.312+0000 7f2b4f590740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558b5beec760 0x558b5bee9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.312+0000 7f2b4f590740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b481ffff0 con 0x558b5bedf290 2024-04-25T08:59:54.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.312+0000 7f2b4f590740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b481ff1d0 con 0x558b5bec1ca0 2024-04-25T08:59:54.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.312+0000 7f2b4f590740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b481fe6d0 con 0x558b5beec760 2024-04-25T08:59:54.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.312+0000 7f2b45d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b5bedf290 0x558b5bee93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.312+0000 7f2b45d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b5bedf290 0x558b5bee93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55234/0 (socket says 172.21.15.43:55234) 2024-04-25T08:59:54.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.312+0000 7f2b45d9b640 1 -- 172.21.15.43:0/2470840362 learned_addr learned my addr 172.21.15.43:0/2470840362 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:59:54.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.312+0000 7f2b4659c640 1 --2- 172.21.15.43:0/2470840362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b5bec1ca0 0x558b5bf685f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.312+0000 7f2b46d9d640 1 --2- 172.21.15.43:0/2470840362 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558b5beec760 0x558b5bee9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.312+0000 7f2b45d9b640 1 -- 172.21.15.43:0/2470840362 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558b5beec760 msgr2=0x558b5bee9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:54.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.313+0000 7f2b45d9b640 1 --2- 172.21.15.43:0/2470840362 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558b5beec760 0x558b5bee9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.313+0000 7f2b45d9b640 1 -- 172.21.15.43:0/2470840362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b5bec1ca0 msgr2=0x558b5bf685f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:54.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.313+0000 7f2b45d9b640 1 --2- 172.21.15.43:0/2470840362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b5bec1ca0 0x558b5bf685f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.313+0000 7f2b45d9b640 1 -- 172.21.15.43:0/2470840362 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b5bad21b0 con 0x558b5bedf290 2024-04-25T08:59:54.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.313+0000 7f2b4659c640 1 --2- 172.21.15.43:0/2470840362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b5bec1ca0 0x558b5bf685f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:59:54.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.313+0000 7f2b46d9d640 1 --2- 172.21.15.43:0/2470840362 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558b5beec760 0x558b5bee9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:59:54.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.313+0000 7f2b45d9b640 1 --2- 172.21.15.43:0/2470840362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b5bedf290 0x558b5bee93c0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c001a80 tx=0x7f2b3c005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4771129e03378b8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:54.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.313+0000 7f2b277fe640 1 -- 172.21.15.43:0/2470840362 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c028070 con 0x558b5bedf290 2024-04-25T08:59:54.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.313+0000 7f2b277fe640 1 -- 172.21.15.43:0/2470840362 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b3c01a040 con 0x558b5bedf290 2024-04-25T08:59:54.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.313+0000 7f2b277fe640 1 -- 172.21.15.43:0/2470840362 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c019040 con 0x558b5bedf290 2024-04-25T08:59:54.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.313+0000 7f2b4f590740 1 -- 172.21.15.43:0/2470840362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b5bedf290 msgr2=0x558b5bee93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:54.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.313+0000 7f2b4f590740 1 --2- 172.21.15.43:0/2470840362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b5bedf290 0x558b5bee93c0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c001a80 tx=0x7f2b3c005000 comp rx=0 tx=0).stop 2024-04-25T08:59:54.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.313+0000 7f2b4f590740 1 -- 172.21.15.43:0/2470840362 shutdown_connections 2024-04-25T08:59:54.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.313+0000 7f2b4f590740 1 --2- 172.21.15.43:0/2470840362 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558b5beec760 0x558b5bee9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.313+0000 7f2b4f590740 1 --2- 172.21.15.43:0/2470840362 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b5bedf290 0x558b5bee93c0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.313+0000 7f2b4f590740 1 --2- 172.21.15.43:0/2470840362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b5bec1ca0 0x558b5bf685f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.313+0000 7f2b4f590740 1 -- 172.21.15.43:0/2470840362 >> 172.21.15.43:0/2470840362 conn(0x558b5be89870 msgr2=0x558b5bede5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:54.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.313+0000 7f2b4f590740 1 -- 172.21.15.43:0/2470840362 shutdown_connections 2024-04-25T08:59:54.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.313+0000 7f2b4f590740 1 -- 172.21.15.43:0/2470840362 wait complete. 2024-04-25T08:59:54.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.314+0000 7f2b4f590740 1 -- start start 2024-04-25T08:59:54.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.314+0000 7f2b4f590740 1 Processor -- start 2024-04-25T08:59:54.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.315+0000 7f2b4f590740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b5bec1ca0 0x558b5bede510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.315+0000 7f2b4f590740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558b5bedf290 0x558b5bedea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.315+0000 7f2b4f590740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b5beec760 0x558b5bf67120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.315+0000 7f2b4f590740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b481febd0 con 0x558b5bec1ca0 2024-04-25T08:59:54.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.315+0000 7f2b4f590740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b5bee8c30 con 0x558b5beec760 2024-04-25T08:59:54.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.315+0000 7f2b4f590740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x558b5bee9c40 con 0x558b5bedf290 2024-04-25T08:59:54.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.315+0000 7f2b4f590740 10 client.0 _ll_register_callbacks cb 0x7f2b380096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:59:54.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.315+0000 7f2b4659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b5bec1ca0 0x558b5bede510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.315+0000 7f2b4659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b5bec1ca0 0x558b5bede510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55240/0 (socket says 172.21.15.43:55240) 2024-04-25T08:59:54.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.315+0000 7f2b4659c640 1 -- 172.21.15.43:0/521898707 learned_addr learned my addr 172.21.15.43:0/521898707 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:59:54.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.315+0000 7f2b4659c640 1 -- 172.21.15.43:0/521898707 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558b5bedf290 msgr2=0x558b5bedea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:54.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.315+0000 7f2b4659c640 1 --2- 172.21.15.43:0/521898707 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558b5bedf290 0x558b5bedea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.315+0000 7f2b46d9d640 1 --2- 172.21.15.43:0/521898707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b5beec760 0x558b5bf67120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.315+0000 7f2b4659c640 1 -- 172.21.15.43:0/521898707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b5beec760 msgr2=0x558b5bf67120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:54.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.315+0000 7f2b4659c640 1 --2- 172.21.15.43:0/521898707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b5beec760 0x558b5bf67120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.315+0000 7f2b4659c640 1 -- 172.21.15.43:0/521898707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b5beeaf40 con 0x558b5bec1ca0 2024-04-25T08:59:54.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.315+0000 7f2b4659c640 1 --2- 172.21.15.43:0/521898707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b5bec1ca0 0x558b5bede510 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f2b380090b0 tx=0x7f2b38009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f9ba481282ce88b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:54.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.315+0000 7f2b257fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:59:54.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.316+0000 7f2b257fa640 1 -- 172.21.15.43:0/521898707 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3804d040 con 0x558b5bec1ca0 2024-04-25T08:59:54.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.316+0000 7f2b257fa640 1 -- 172.21.15.43:0/521898707 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(75..75 src has 1..75) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f2b38052850 con 0x558b5bec1ca0 2024-04-25T08:59:54.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.316+0000 7f2b4f590740 1 -- 172.21.15.43:0/521898707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2b3c01a040 con 0x558b5bec1ca0 2024-04-25T08:59:54.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.316+0000 7f2b257fa640 1 -- 172.21.15.43:0/521898707 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b38069050 con 0x558b5bec1ca0 2024-04-25T08:59:54.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.316+0000 7f2b4f590740 10 client.7335 fetch_fsmap learned FSMap version 48 2024-04-25T08:59:54.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.316+0000 7f2b4f590740 1 -- 172.21.15.43:0/521898707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x558b5bec38e0 con 0x558b5bec1ca0 2024-04-25T08:59:54.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.316+0000 7f2b257fa640 1 -- 172.21.15.43:0/521898707 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b3c01a040 con 0x558b5bec1ca0 2024-04-25T08:59:54.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.316+0000 7f2b257fa640 1 -- 172.21.15.43:0/521898707 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 48) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2b3806c020 con 0x558b5bec1ca0 2024-04-25T08:59:54.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.316+0000 7f2b4f590740 10 client.7335 fetch_fsmap finished waiting for FSMap version 48 2024-04-25T08:59:54.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.316+0000 7f2b4f590740 10 client.7335 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T08:59:54.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.316+0000 7f2b4f590740 20 client.7335 populate_metadata read hostname 'smithi043' 2024-04-25T08:59:54.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.316+0000 7f2b4f590740 1 --2- 172.21.15.43:0/521898707 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x558b5befe370 0x558b5bf67660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.316+0000 7f2b4f590740 4 client.7335 mds_command: new command op to 7164 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T08:59:54.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.316+0000 7f2b4f590740 1 -- 172.21.15.43:0/521898707 --> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x558b5be2c420 con 0x558b5befe370 2024-04-25T08:59:54.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.316+0000 7f2b45d9b640 1 --2- 172.21.15.43:0/521898707 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x558b5befe370 0x558b5bf67660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.317+0000 7f2b45d9b640 1 --2- 172.21.15.43:0/521898707 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x558b5befe370 0x558b5bf67660 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=526ba6d7e9801d94 server_cookie=536b1fdeee47afa0 in_seq=0 out_seq=0 2024-04-25T08:59:54.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.317+0000 7f2b257fa640 10 client.7335 ms_handle_connect on v2:172.21.15.193:6835/3458098162 2024-04-25T08:59:54.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.317+0000 7f2b257fa640 1 -- 172.21.15.43:0/521898707 <== mds.0 v2:172.21.15.193:6835/3458098162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17923 (crc 0 0 0) 0x558b5be2c420 con 0x558b5befe370 2024-04-25T08:59:54.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.317+0000 7f2b257fa640 10 client.7335 handle_command_reply: tid=0 2024-04-25T08:59:54.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.317+0000 7f2b4f590740 1 client.7335 shutdown 2024-04-25T08:59:54.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.317+0000 7f2b4f590740 1 -- 172.21.15.43:0/521898707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b5bec1ca0 msgr2=0x558b5bede510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:54.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.317+0000 7f2b4f590740 1 --2- 172.21.15.43:0/521898707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b5bec1ca0 0x558b5bede510 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f2b380090b0 tx=0x7f2b38009240 comp rx=0 tx=0).stop 2024-04-25T08:59:54.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.317+0000 7f2b4f590740 1 -- 172.21.15.43:0/521898707 shutdown_connections 2024-04-25T08:59:54.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.317+0000 7f2b4f590740 1 --2- 172.21.15.43:0/521898707 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x558b5befe370 0x558b5bf67660 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.317+0000 7f2b4f590740 1 --2- 172.21.15.43:0/521898707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x558b5beec760 0x558b5bf67120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.317+0000 7f2b4f590740 1 --2- 172.21.15.43:0/521898707 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x558b5bedf290 0x558b5bedea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.317+0000 7f2b257fa640 0 client.7335 ms_handle_reset on v2:172.21.15.193:6835/3458098162 2024-04-25T08:59:54.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.317+0000 7f2b4f590740 1 --2- 172.21.15.43:0/521898707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x558b5bec1ca0 0x558b5bede510 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.317+0000 7f2b4f590740 1 -- 172.21.15.43:0/521898707 >> 172.21.15.43:0/521898707 conn(0x558b5be89870 msgr2=0x558b5bef00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:54.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.317+0000 7f2b4f590740 1 -- 172.21.15.43:0/521898707 shutdown_connections 2024-04-25T08:59:54.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.317+0000 7f2b4f590740 1 -- 172.21.15.43:0/521898707 wait complete. 2024-04-25T08:59:54.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.318+0000 7f2b4f590740 20 client.7335 trim_cache size 0 max 16384 2024-04-25T08:59:54.313 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T08:59:54.313 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T08:59:54.313 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 2649, 2024-04-25T08:59:54.313 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 2650, 2024-04-25T08:59:54.313 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 824406, 2024-04-25T08:59:54.313 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 30859852, 2024-04-25T08:59:54.314 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 9, 2024-04-25T08:59:54.314 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 3, 2024-04-25T08:59:54.314 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.226563408, 2024-04-25T08:59:54.314 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.108257164, 2024-04-25T08:59:54.314 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.042669028, 2024-04-25T08:59:54.314 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.062897288, 2024-04-25T08:59:54.314 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T08:59:54.314 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 2648, 2024-04-25T08:59:54.314 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.271078504, 2024-04-25T08:59:54.314 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000102371 2024-04-25T08:59:54.314 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.314 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-25T08:59:54.314 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 2, 2024-04-25T08:59:54.315 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000115, 2024-04-25T08:59:54.315 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000057 2024-04-25T08:59:54.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.315 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T08:59:54.315 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T08:59:54.315 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.315 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T08:59:54.315 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 35667, 2024-04-25T08:59:54.315 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 61588, 2024-04-25T08:59:54.315 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 12369931, 2024-04-25T08:59:54.315 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 58779274, 2024-04-25T08:59:54.315 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 7, 2024-04-25T08:59:54.315 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 4, 2024-04-25T08:59:54.316 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 2.442264142, 2024-04-25T08:59:54.316 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 1.143740372, 2024-04-25T08:59:54.316 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 1.457314134, 2024-04-25T08:59:54.316 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.076531154, 2024-04-25T08:59:54.316 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T08:59:54.316 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 61588, 2024-04-25T08:59:54.316 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 1.272525337, 2024-04-25T08:59:54.316 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000020661 2024-04-25T08:59:54.316 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.316 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-25T08:59:54.316 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 88551, 2024-04-25T08:59:54.316 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.009449205, 2024-04-25T08:59:54.316 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000106 2024-04-25T08:59:54.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.317 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 119840, 2024-04-25T08:59:54.317 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 20528 2024-04-25T08:59:54.317 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.317 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T08:59:54.317 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 2450, 2024-04-25T08:59:54.317 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 2466, 2024-04-25T08:59:54.317 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 607937, 2024-04-25T08:59:54.317 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 20926932, 2024-04-25T08:59:54.318 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 6, 2024-04-25T08:59:54.318 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 4, 2024-04-25T08:59:54.318 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.268111208, 2024-04-25T08:59:54.318 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.094465976, 2024-04-25T08:59:54.318 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.040871990, 2024-04-25T08:59:54.318 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.118786160, 2024-04-25T08:59:54.318 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T08:59:54.318 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 2465, 2024-04-25T08:59:54.318 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.277064359, 2024-04-25T08:59:54.318 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000112399 2024-04-25T08:59:54.318 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.318 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-25T08:59:54.318 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:59:54.319 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:59:54.319 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:59:54.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.319 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T08:59:54.319 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 111920 2024-04-25T08:59:54.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.319 INFO:teuthology.orchestra.run.smithi043.stdout: "cct": { 2024-04-25T08:59:54.319 INFO:teuthology.orchestra.run.smithi043.stdout: "total_workers": 1, 2024-04-25T08:59:54.319 INFO:teuthology.orchestra.run.smithi043.stdout: "unhealthy_workers": 0 2024-04-25T08:59:54.319 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.319 INFO:teuthology.orchestra.run.smithi043.stdout: "finisher-MDSRank": { 2024-04-25T08:59:54.319 INFO:teuthology.orchestra.run.smithi043.stdout: "queue_len": 0, 2024-04-25T08:59:54.319 INFO:teuthology.orchestra.run.smithi043.stdout: "complete_latency": { 2024-04-25T08:59:54.319 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 9374, 2024-04-25T08:59:54.319 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 11.806593559, 2024-04-25T08:59:54.320 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.001259504 2024-04-25T08:59:54.320 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:59:54.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.320 INFO:teuthology.orchestra.run.smithi043.stdout: "finisher-PurgeQueue": { 2024-04-25T08:59:54.320 INFO:teuthology.orchestra.run.smithi043.stdout: "queue_len": 0, 2024-04-25T08:59:54.320 INFO:teuthology.orchestra.run.smithi043.stdout: "complete_latency": { 2024-04-25T08:59:54.320 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 1412, 2024-04-25T08:59:54.320 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.405337470, 2024-04-25T08:59:54.320 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000287066 2024-04-25T08:59:54.320 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:59:54.320 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.320 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": { 2024-04-25T08:59:54.320 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 28285, 2024-04-25T08:59:54.320 INFO:teuthology.orchestra.run.smithi043.stdout: "reply": 28285, 2024-04-25T08:59:54.320 INFO:teuthology.orchestra.run.smithi043.stdout: "reply_latency": { 2024-04-25T08:59:54.320 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 28285, 2024-04-25T08:59:54.321 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 222.806273216, 2024-04-25T08:59:54.321 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.007877188 2024-04-25T08:59:54.321 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.321 INFO:teuthology.orchestra.run.smithi043.stdout: "slow_reply": 0, 2024-04-25T08:59:54.321 INFO:teuthology.orchestra.run.smithi043.stdout: "forward": 0, 2024-04-25T08:59:54.321 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_fetch_complete": 0, 2024-04-25T08:59:54.321 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_fetch_keys": 0, 2024-04-25T08:59:54.321 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_commit": 12, 2024-04-25T08:59:54.321 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_split": 0, 2024-04-25T08:59:54.321 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_merge": 0, 2024-04-25T08:59:54.321 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes": 19838, 2024-04-25T08:59:54.321 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_top": 0, 2024-04-25T08:59:54.321 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_bottom": 19838, 2024-04-25T08:59:54.321 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_pin_tail": 0, 2024-04-25T08:59:54.321 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_pinned": 19838, 2024-04-25T08:59:54.322 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_expired": 0, 2024-04-25T08:59:54.322 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_with_caps": 474, 2024-04-25T08:59:54.322 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 474, 2024-04-25T08:59:54.322 INFO:teuthology.orchestra.run.smithi043.stdout: "subtrees": 2, 2024-04-25T08:59:54.322 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse": 46903, 2024-04-25T08:59:54.322 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_hit": 46419, 2024-04-25T08:59:54.322 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_forward": 0, 2024-04-25T08:59:54.322 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_discover": 0, 2024-04-25T08:59:54.322 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_dir_fetch": 0, 2024-04-25T08:59:54.322 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_remote_ino": 0, 2024-04-25T08:59:54.322 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_lock": 157, 2024-04-25T08:59:54.322 INFO:teuthology.orchestra.run.smithi043.stdout: "load_cent": 0, 2024-04-25T08:59:54.322 INFO:teuthology.orchestra.run.smithi043.stdout: "q": 0, 2024-04-25T08:59:54.322 INFO:teuthology.orchestra.run.smithi043.stdout: "exported": 0, 2024-04-25T08:59:54.323 INFO:teuthology.orchestra.run.smithi043.stdout: "exported_inodes": 0, 2024-04-25T08:59:54.323 INFO:teuthology.orchestra.run.smithi043.stdout: "imported": 0, 2024-04-25T08:59:54.323 INFO:teuthology.orchestra.run.smithi043.stdout: "imported_inodes": 0, 2024-04-25T08:59:54.323 INFO:teuthology.orchestra.run.smithi043.stdout: "openino_dir_fetch": 0, 2024-04-25T08:59:54.323 INFO:teuthology.orchestra.run.smithi043.stdout: "openino_backtrace_fetch": 0, 2024-04-25T08:59:54.323 INFO:teuthology.orchestra.run.smithi043.stdout: "openino_peer_discover": 0, 2024-04-25T08:59:54.323 INFO:teuthology.orchestra.run.smithi043.stdout: "root_rfiles": 0, 2024-04-25T08:59:54.323 INFO:teuthology.orchestra.run.smithi043.stdout: "root_rbytes": 0, 2024-04-25T08:59:54.323 INFO:teuthology.orchestra.run.smithi043.stdout: "root_rsnaps": 0, 2024-04-25T08:59:54.323 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T08:59:54.323 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_set_tag": 0, 2024-04-25T08:59:54.323 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T08:59:54.323 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_inotable_repaired": 0, 2024-04-25T08:59:54.323 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_dir_inodes": 0, 2024-04-25T08:59:54.324 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T08:59:54.324 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T08:59:54.324 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_file_inodes": 0, 2024-04-25T08:59:54.324 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_inode_file_caps": 0, 2024-04-25T08:59:54.324 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_revoke": 6190, 2024-04-25T08:59:54.324 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_grant": 6010, 2024-04-25T08:59:54.324 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T08:59:54.324 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T08:59:54.324 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T08:59:54.324 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_caps": 6190, 2024-04-25T08:59:54.324 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_caps_dirty": 2000, 2024-04-25T08:59:54.324 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_cap_release": 26, 2024-04-25T08:59:54.324 INFO:teuthology.orchestra.run.smithi043.stdout: "process_request_cap_release": 17838 2024-04-25T08:59:54.324 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.324 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_cache": { 2024-04-25T08:59:54.325 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_update": 0, 2024-04-25T08:59:54.325 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_update_receipt": 0, 2024-04-25T08:59:54.325 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_try_discover": 0, 2024-04-25T08:59:54.325 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_send_discover": 0, 2024-04-25T08:59:54.325 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_handle_discover": 0, 2024-04-25T08:59:54.325 INFO:teuthology.orchestra.run.smithi043.stdout: "num_strays": 2000, 2024-04-25T08:59:54.325 INFO:teuthology.orchestra.run.smithi043.stdout: "num_strays_delayed": 1, 2024-04-25T08:59:54.325 INFO:teuthology.orchestra.run.smithi043.stdout: "num_strays_enqueuing": 0, 2024-04-25T08:59:54.325 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_created": 4116, 2024-04-25T08:59:54.325 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_enqueued": 1528, 2024-04-25T08:59:54.325 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_reintegrated": 2216, 2024-04-25T08:59:54.325 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_migrated": 0, 2024-04-25T08:59:54.325 INFO:teuthology.orchestra.run.smithi043.stdout: "num_recovering_processing": 0, 2024-04-25T08:59:54.325 INFO:teuthology.orchestra.run.smithi043.stdout: "num_recovering_enqueued": 0, 2024-04-25T08:59:54.326 INFO:teuthology.orchestra.run.smithi043.stdout: "num_recovering_prioritized": 0, 2024-04-25T08:59:54.326 INFO:teuthology.orchestra.run.smithi043.stdout: "recovery_started": 0, 2024-04-25T08:59:54.326 INFO:teuthology.orchestra.run.smithi043.stdout: "recovery_completed": 0, 2024-04-25T08:59:54.326 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T08:59:54.326 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_exportdir": 0, 2024-04-25T08:59:54.326 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_flush": 0, 2024-04-25T08:59:54.326 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_fragmentdir": 0, 2024-04-25T08:59:54.326 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_fragstats": 0, 2024-04-25T08:59:54.326 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_inodestats": 0 2024-04-25T08:59:54.326 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.326 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_log": { 2024-04-25T08:59:54.326 INFO:teuthology.orchestra.run.smithi043.stdout: "evlrg": 0, 2024-04-25T08:59:54.326 INFO:teuthology.orchestra.run.smithi043.stdout: "evadd": 21705, 2024-04-25T08:59:54.326 INFO:teuthology.orchestra.run.smithi043.stdout: "evex": 0, 2024-04-25T08:59:54.327 INFO:teuthology.orchestra.run.smithi043.stdout: "evtrm": 0, 2024-04-25T08:59:54.327 INFO:teuthology.orchestra.run.smithi043.stdout: "ev": 21705, 2024-04-25T08:59:54.327 INFO:teuthology.orchestra.run.smithi043.stdout: "evexg": 0, 2024-04-25T08:59:54.327 INFO:teuthology.orchestra.run.smithi043.stdout: "evexd": 0, 2024-04-25T08:59:54.327 INFO:teuthology.orchestra.run.smithi043.stdout: "segadd": 29, 2024-04-25T08:59:54.327 INFO:teuthology.orchestra.run.smithi043.stdout: "segex": 0, 2024-04-25T08:59:54.327 INFO:teuthology.orchestra.run.smithi043.stdout: "segtrm": 0, 2024-04-25T08:59:54.327 INFO:teuthology.orchestra.run.smithi043.stdout: "seg": 29, 2024-04-25T08:59:54.327 INFO:teuthology.orchestra.run.smithi043.stdout: "segmjr": 3, 2024-04-25T08:59:54.327 INFO:teuthology.orchestra.run.smithi043.stdout: "segexg": 0, 2024-04-25T08:59:54.327 INFO:teuthology.orchestra.run.smithi043.stdout: "segexd": 0, 2024-04-25T08:59:54.327 INFO:teuthology.orchestra.run.smithi043.stdout: "expos": 4194304, 2024-04-25T08:59:54.327 INFO:teuthology.orchestra.run.smithi043.stdout: "wrpos": 81111496, 2024-04-25T08:59:54.327 INFO:teuthology.orchestra.run.smithi043.stdout: "rdpos": 0, 2024-04-25T08:59:54.327 INFO:teuthology.orchestra.run.smithi043.stdout: "jlat": { 2024-04-25T08:59:54.328 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 6503, 2024-04-25T08:59:54.328 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 64.951789188, 2024-04-25T08:59:54.328 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.009987973 2024-04-25T08:59:54.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.328 INFO:teuthology.orchestra.run.smithi043.stdout: "replayed": 0 2024-04-25T08:59:54.328 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.328 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_mem": { 2024-04-25T08:59:54.328 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 2014, 2024-04-25T08:59:54.328 INFO:teuthology.orchestra.run.smithi043.stdout: "ino+": 2014, 2024-04-25T08:59:54.328 INFO:teuthology.orchestra.run.smithi043.stdout: "ino-": 0, 2024-04-25T08:59:54.328 INFO:teuthology.orchestra.run.smithi043.stdout: "dir": 13, 2024-04-25T08:59:54.328 INFO:teuthology.orchestra.run.smithi043.stdout: "dir+": 13, 2024-04-25T08:59:54.328 INFO:teuthology.orchestra.run.smithi043.stdout: "dir-": 0, 2024-04-25T08:59:54.328 INFO:teuthology.orchestra.run.smithi043.stdout: "dn": 19838, 2024-04-25T08:59:54.328 INFO:teuthology.orchestra.run.smithi043.stdout: "dn+": 19838, 2024-04-25T08:59:54.329 INFO:teuthology.orchestra.run.smithi043.stdout: "dn-": 0, 2024-04-25T08:59:54.329 INFO:teuthology.orchestra.run.smithi043.stdout: "cap": 474, 2024-04-25T08:59:54.329 INFO:teuthology.orchestra.run.smithi043.stdout: "cap+": 2002, 2024-04-25T08:59:54.329 INFO:teuthology.orchestra.run.smithi043.stdout: "cap-": 1528, 2024-04-25T08:59:54.329 INFO:teuthology.orchestra.run.smithi043.stdout: "rss": 105512, 2024-04-25T08:59:54.329 INFO:teuthology.orchestra.run.smithi043.stdout: "heap": 215396 2024-04-25T08:59:54.329 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.329 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_server": { 2024-04-25T08:59:54.329 INFO:teuthology.orchestra.run.smithi043.stdout: "dispatch_client_request": 37689, 2024-04-25T08:59:54.329 INFO:teuthology.orchestra.run.smithi043.stdout: "dispatch_server_request": 0, 2024-04-25T08:59:54.329 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_request": 28285, 2024-04-25T08:59:54.329 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_session": 5, 2024-04-25T08:59:54.329 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_peer_request": 0, 2024-04-25T08:59:54.329 INFO:teuthology.orchestra.run.smithi043.stdout: "req_create_latency": { 2024-04-25T08:59:54.330 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 2000, 2024-04-25T08:59:54.330 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 1.320921704, 2024-04-25T08:59:54.330 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000660460 2024-04-25T08:59:54.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.330 INFO:teuthology.orchestra.run.smithi043.stdout: "req_getattr_latency": { 2024-04-25T08:59:54.330 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 2008, 2024-04-25T08:59:54.330 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.553011189, 2024-04-25T08:59:54.330 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000275403 2024-04-25T08:59:54.330 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.330 INFO:teuthology.orchestra.run.smithi043.stdout: "req_getfilelock_latency": { 2024-04-25T08:59:54.330 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:59:54.330 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:59:54.330 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:59:54.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.331 INFO:teuthology.orchestra.run.smithi043.stdout: "req_link_latency": { 2024-04-25T08:59:54.331 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 6000, 2024-04-25T08:59:54.331 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 9.405248756, 2024-04-25T08:59:54.331 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.001567541 2024-04-25T08:59:54.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.331 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookup_latency": { 2024-04-25T08:59:54.331 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 8048, 2024-04-25T08:59:54.331 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 7.188344113, 2024-04-25T08:59:54.331 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000893183 2024-04-25T08:59:54.331 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.331 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookuphash_latency": { 2024-04-25T08:59:54.331 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:59:54.331 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:59:54.331 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:59:54.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.332 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupino_latency": { 2024-04-25T08:59:54.332 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:59:54.332 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:59:54.332 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:59:54.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.332 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupname_latency": { 2024-04-25T08:59:54.332 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:59:54.332 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:59:54.332 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:59:54.332 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.332 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupparent_latency": { 2024-04-25T08:59:54.332 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:59:54.332 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:59:54.333 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:59:54.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.333 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupsnap_latency": { 2024-04-25T08:59:54.333 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 3, 2024-04-25T08:59:54.333 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000479137, 2024-04-25T08:59:54.333 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000159712 2024-04-25T08:59:54.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.333 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lssnap_latency": { 2024-04-25T08:59:54.333 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:59:54.333 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:59:54.333 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:59:54.333 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.333 INFO:teuthology.orchestra.run.smithi043.stdout: "req_mkdir_latency": { 2024-04-25T08:59:54.333 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 1, 2024-04-25T08:59:54.333 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.002887482, 2024-04-25T08:59:54.334 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.002887482 2024-04-25T08:59:54.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.334 INFO:teuthology.orchestra.run.smithi043.stdout: "req_mknod_latency": { 2024-04-25T08:59:54.334 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:59:54.334 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:59:54.334 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:59:54.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.334 INFO:teuthology.orchestra.run.smithi043.stdout: "req_mksnap_latency": { 2024-04-25T08:59:54.334 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 1, 2024-04-25T08:59:54.334 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.005256154, 2024-04-25T08:59:54.334 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.005256154 2024-04-25T08:59:54.334 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.334 INFO:teuthology.orchestra.run.smithi043.stdout: "req_open_latency": { 2024-04-25T08:59:54.334 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:59:54.334 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:59:54.335 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:59:54.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.335 INFO:teuthology.orchestra.run.smithi043.stdout: "req_readdir_latency": { 2024-04-25T08:59:54.335 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 6, 2024-04-25T08:59:54.335 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.092175118, 2024-04-25T08:59:54.335 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.015362519 2024-04-25T08:59:54.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.335 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rename_latency": { 2024-04-25T08:59:54.335 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 2216, 2024-04-25T08:59:54.335 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 189.223382261, 2024-04-25T08:59:54.335 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.085389612 2024-04-25T08:59:54.335 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.335 INFO:teuthology.orchestra.run.smithi043.stdout: "req_renamesnap_latency": { 2024-04-25T08:59:54.335 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:59:54.336 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:59:54.336 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:59:54.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.336 INFO:teuthology.orchestra.run.smithi043.stdout: "req_snapdiff_latency": { 2024-04-25T08:59:54.336 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:59:54.336 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:59:54.336 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:59:54.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.336 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rmdir_latency": { 2024-04-25T08:59:54.336 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:59:54.336 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:59:54.336 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:59:54.336 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.336 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rmsnap_latency": { 2024-04-25T08:59:54.336 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 1, 2024-04-25T08:59:54.337 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.002953385, 2024-04-25T08:59:54.337 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.002953385 2024-04-25T08:59:54.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.337 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rmxattr_latency": { 2024-04-25T08:59:54.337 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:59:54.337 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:59:54.337 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:59:54.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.337 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setattr_latency": { 2024-04-25T08:59:54.337 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 1, 2024-04-25T08:59:54.337 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000294992, 2024-04-25T08:59:54.337 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000294992 2024-04-25T08:59:54.337 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.337 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setdirlayout_latency": { 2024-04-25T08:59:54.337 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:59:54.338 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:59:54.338 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:59:54.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.338 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setfilelock_latency": { 2024-04-25T08:59:54.338 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:59:54.338 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:59:54.338 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:59:54.338 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.338 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setlayout_latency": { 2024-04-25T08:59:54.338 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:59:54.338 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:59:54.341 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:59:54.341 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.341 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setxattr_latency": { 2024-04-25T08:59:54.341 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:59:54.342 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:59:54.342 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:59:54.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.342 INFO:teuthology.orchestra.run.smithi043.stdout: "req_symlink_latency": { 2024-04-25T08:59:54.342 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:59:54.342 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:59:54.342 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:59:54.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.342 INFO:teuthology.orchestra.run.smithi043.stdout: "req_unlink_latency": { 2024-04-25T08:59:54.342 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 8000, 2024-04-25T08:59:54.342 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 15.011318925, 2024-04-25T08:59:54.342 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.001876414 2024-04-25T08:59:54.342 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.342 INFO:teuthology.orchestra.run.smithi043.stdout: "cap_revoke_eviction": 0, 2024-04-25T08:59:54.343 INFO:teuthology.orchestra.run.smithi043.stdout: "cap_acquisition_throttle": 0, 2024-04-25T08:59:54.343 INFO:teuthology.orchestra.run.smithi043.stdout: "req_getvxattr_latency": { 2024-04-25T08:59:54.343 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:59:54.343 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:59:54.343 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:59:54.343 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:59:54.343 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.343 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_sessions": { 2024-04-25T08:59:54.343 INFO:teuthology.orchestra.run.smithi043.stdout: "session_count": 1, 2024-04-25T08:59:54.343 INFO:teuthology.orchestra.run.smithi043.stdout: "session_add": 1, 2024-04-25T08:59:54.343 INFO:teuthology.orchestra.run.smithi043.stdout: "session_remove": 0, 2024-04-25T08:59:54.343 INFO:teuthology.orchestra.run.smithi043.stdout: "sessions_open": 1, 2024-04-25T08:59:54.343 INFO:teuthology.orchestra.run.smithi043.stdout: "sessions_stale": 0, 2024-04-25T08:59:54.343 INFO:teuthology.orchestra.run.smithi043.stdout: "total_load": 19391, 2024-04-25T08:59:54.343 INFO:teuthology.orchestra.run.smithi043.stdout: "average_load": 19391, 2024-04-25T08:59:54.344 INFO:teuthology.orchestra.run.smithi043.stdout: "avg_session_uptime": 58, 2024-04-25T08:59:54.344 INFO:teuthology.orchestra.run.smithi043.stdout: "mdthresh_evicted": 0 2024-04-25T08:59:54.344 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.344 INFO:teuthology.orchestra.run.smithi043.stdout: "mempool": { 2024-04-25T08:59:54.344 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_bytes": 0, 2024-04-25T08:59:54.344 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_items": 0, 2024-04-25T08:59:54.344 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T08:59:54.344 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_items": 0, 2024-04-25T08:59:54.344 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T08:59:54.344 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_items": 0, 2024-04-25T08:59:54.344 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T08:59:54.344 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T08:59:54.344 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T08:59:54.344 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T08:59:54.344 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T08:59:54.345 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_items": 0, 2024-04-25T08:59:54.345 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T08:59:54.345 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T08:59:54.345 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_bytes": 0, 2024-04-25T08:59:54.345 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_items": 0, 2024-04-25T08:59:54.345 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_bytes": 0, 2024-04-25T08:59:54.345 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_items": 0, 2024-04-25T08:59:54.345 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T08:59:54.345 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T08:59:54.345 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T08:59:54.345 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T08:59:54.345 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T08:59:54.345 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_items": 0, 2024-04-25T08:59:54.345 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_bytes": 0, 2024-04-25T08:59:54.346 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_items": 0, 2024-04-25T08:59:54.346 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T08:59:54.346 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T08:59:54.346 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_bytes": 0, 2024-04-25T08:59:54.346 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_items": 0, 2024-04-25T08:59:54.346 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_bytes": 0, 2024-04-25T08:59:54.346 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_items": 0, 2024-04-25T08:59:54.346 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T08:59:54.346 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_items": 0, 2024-04-25T08:59:54.346 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T08:59:54.346 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_items": 0, 2024-04-25T08:59:54.346 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_bytes": 17993078, 2024-04-25T08:59:54.346 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_items": 4577, 2024-04-25T08:59:54.346 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_bytes": 0, 2024-04-25T08:59:54.347 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_items": 0, 2024-04-25T08:59:54.347 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_bytes": 0, 2024-04-25T08:59:54.347 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_items": 0, 2024-04-25T08:59:54.347 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_bytes": 0, 2024-04-25T08:59:54.347 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_items": 0, 2024-04-25T08:59:54.347 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_bytes": 0, 2024-04-25T08:59:54.347 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_items": 0, 2024-04-25T08:59:54.347 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_bytes": 5504, 2024-04-25T08:59:54.347 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_items": 79, 2024-04-25T08:59:54.347 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T08:59:54.347 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_items": 0, 2024-04-25T08:59:54.347 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_bytes": 0, 2024-04-25T08:59:54.347 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_items": 0, 2024-04-25T08:59:54.347 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_bytes": 20684349, 2024-04-25T08:59:54.347 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_items": 807594, 2024-04-25T08:59:54.347 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_bytes": 0, 2024-04-25T08:59:54.348 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_items": 0, 2024-04-25T08:59:54.348 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_bytes": 0, 2024-04-25T08:59:54.348 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_items": 0 2024-04-25T08:59:54.348 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.348 INFO:teuthology.orchestra.run.smithi043.stdout: "objecter": { 2024-04-25T08:59:54.348 INFO:teuthology.orchestra.run.smithi043.stdout: "op_active": 0, 2024-04-25T08:59:54.348 INFO:teuthology.orchestra.run.smithi043.stdout: "op_laggy": 0, 2024-04-25T08:59:54.348 INFO:teuthology.orchestra.run.smithi043.stdout: "op_send": 8289, 2024-04-25T08:59:54.348 INFO:teuthology.orchestra.run.smithi043.stdout: "op_send_bytes": 77100983, 2024-04-25T08:59:54.348 INFO:teuthology.orchestra.run.smithi043.stdout: "op_resend": 37, 2024-04-25T08:59:54.348 INFO:teuthology.orchestra.run.smithi043.stdout: "op_reply": 8289, 2024-04-25T08:59:54.348 INFO:teuthology.orchestra.run.smithi043.stdout: "op_latency": { 2024-04-25T08:59:54.348 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 8289, 2024-04-25T08:59:54.348 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 18.207720024, 2024-04-25T08:59:54.348 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.002196612 2024-04-25T08:59:54.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.349 INFO:teuthology.orchestra.run.smithi043.stdout: "op_inflight": 0, 2024-04-25T08:59:54.349 INFO:teuthology.orchestra.run.smithi043.stdout: "oplen_avg": { 2024-04-25T08:59:54.349 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 8289, 2024-04-25T08:59:54.349 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 8348 2024-04-25T08:59:54.349 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.349 INFO:teuthology.orchestra.run.smithi043.stdout: "op": 8289, 2024-04-25T08:59:54.349 INFO:teuthology.orchestra.run.smithi043.stdout: "op_r": 62, 2024-04-25T08:59:54.349 INFO:teuthology.orchestra.run.smithi043.stdout: "op_w": 8227, 2024-04-25T08:59:54.349 INFO:teuthology.orchestra.run.smithi043.stdout: "op_rmw": 0, 2024-04-25T08:59:54.349 INFO:teuthology.orchestra.run.smithi043.stdout: "op_pg": 0, 2024-04-25T08:59:54.349 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_stat": 0, 2024-04-25T08:59:54.349 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_create": 11, 2024-04-25T08:59:54.349 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_read": 62, 2024-04-25T08:59:54.349 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_write": 6565, 2024-04-25T08:59:54.350 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_writefull": 38, 2024-04-25T08:59:54.350 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_writesame": 0, 2024-04-25T08:59:54.350 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_append": 0, 2024-04-25T08:59:54.350 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_zero": 1528, 2024-04-25T08:59:54.350 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_truncate": 0, 2024-04-25T08:59:54.350 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_delete": 30, 2024-04-25T08:59:54.350 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_mapext": 0, 2024-04-25T08:59:54.350 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_sparse_read": 0, 2024-04-25T08:59:54.350 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_clonerange": 0, 2024-04-25T08:59:54.350 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_getxattr": 0, 2024-04-25T08:59:54.350 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_setxattr": 22, 2024-04-25T08:59:54.350 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_cmpxattr": 0, 2024-04-25T08:59:54.350 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_rmxattr": 0, 2024-04-25T08:59:54.350 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_resetxattrs": 0, 2024-04-25T08:59:54.351 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_call": 0, 2024-04-25T08:59:54.351 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_watch": 0, 2024-04-25T08:59:54.351 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_notify": 0, 2024-04-25T08:59:54.351 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T08:59:54.351 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_pgls": 0, 2024-04-25T08:59:54.351 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_pgls_filter": 0, 2024-04-25T08:59:54.351 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_other": 0, 2024-04-25T08:59:54.351 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_active": 0, 2024-04-25T08:59:54.351 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_send": 0, 2024-04-25T08:59:54.351 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_resend": 0, 2024-04-25T08:59:54.351 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_ping": 0, 2024-04-25T08:59:54.351 INFO:teuthology.orchestra.run.smithi043.stdout: "poolop_active": 0, 2024-04-25T08:59:54.351 INFO:teuthology.orchestra.run.smithi043.stdout: "poolop_send": 0, 2024-04-25T08:59:54.351 INFO:teuthology.orchestra.run.smithi043.stdout: "poolop_resend": 0, 2024-04-25T08:59:54.352 INFO:teuthology.orchestra.run.smithi043.stdout: "poolstat_active": 0, 2024-04-25T08:59:54.352 INFO:teuthology.orchestra.run.smithi043.stdout: "poolstat_send": 0, 2024-04-25T08:59:54.352 INFO:teuthology.orchestra.run.smithi043.stdout: "poolstat_resend": 0, 2024-04-25T08:59:54.352 INFO:teuthology.orchestra.run.smithi043.stdout: "statfs_active": 0, 2024-04-25T08:59:54.352 INFO:teuthology.orchestra.run.smithi043.stdout: "statfs_send": 0, 2024-04-25T08:59:54.352 INFO:teuthology.orchestra.run.smithi043.stdout: "statfs_resend": 0, 2024-04-25T08:59:54.352 INFO:teuthology.orchestra.run.smithi043.stdout: "command_active": 0, 2024-04-25T08:59:54.352 INFO:teuthology.orchestra.run.smithi043.stdout: "command_send": 0, 2024-04-25T08:59:54.352 INFO:teuthology.orchestra.run.smithi043.stdout: "command_resend": 0, 2024-04-25T08:59:54.352 INFO:teuthology.orchestra.run.smithi043.stdout: "map_epoch": 75, 2024-04-25T08:59:54.352 INFO:teuthology.orchestra.run.smithi043.stdout: "map_full": 0, 2024-04-25T08:59:54.352 INFO:teuthology.orchestra.run.smithi043.stdout: "map_inc": 3, 2024-04-25T08:59:54.352 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_sessions": 8, 2024-04-25T08:59:54.352 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_session_open": 8, 2024-04-25T08:59:54.352 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_session_close": 0, 2024-04-25T08:59:54.353 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_laggy": 0, 2024-04-25T08:59:54.353 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_wr": 78, 2024-04-25T08:59:54.353 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_rd": 0, 2024-04-25T08:59:54.353 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_del": 14 2024-04-25T08:59:54.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.353 INFO:teuthology.orchestra.run.smithi043.stdout: "oft": { 2024-04-25T08:59:54.353 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_objs": 1, 2024-04-25T08:59:54.353 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_kv_pairs": 0, 2024-04-25T08:59:54.353 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_updates": 0, 2024-04-25T08:59:54.353 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_removes": 0 2024-04-25T08:59:54.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.353 INFO:teuthology.orchestra.run.smithi043.stdout: "osd-slow-ops": { 2024-04-25T08:59:54.353 INFO:teuthology.orchestra.run.smithi043.stdout: "slow_ops_count": 0 2024-04-25T08:59:54.353 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.354 INFO:teuthology.orchestra.run.smithi043.stdout: "purge_queue": { 2024-04-25T08:59:54.354 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing_ops": 0, 2024-04-25T08:59:54.354 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T08:59:54.354 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing": 0, 2024-04-25T08:59:54.354 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing_high_water": 32, 2024-04-25T08:59:54.354 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executed_ops": 1528, 2024-04-25T08:59:54.354 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executed": 1528, 2024-04-25T08:59:54.354 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_item_in_journal": 0 2024-04-25T08:59:54.354 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.354 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T08:59:54.354 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-25T08:59:54.354 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 104857600, 2024-04-25T08:59:54.354 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-25T08:59:54.355 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 40761, 2024-04-25T08:59:54.355 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 10128594, 2024-04-25T08:59:54.355 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-25T08:59:54.355 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 40761, 2024-04-25T08:59:54.355 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-04-25T08:59:54.355 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-04-25T08:59:54.355 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 40761, 2024-04-25T08:59:54.355 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 10128594, 2024-04-25T08:59:54.355 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-25T08:59:54.355 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:59:54.355 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:59:54.355 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:59:54.355 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:59:54.355 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.355 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-objecter_bytes": { 2024-04-25T08:59:54.356 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-25T08:59:54.356 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 104857600, 2024-04-25T08:59:54.356 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-25T08:59:54.356 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 0, 2024-04-25T08:59:54.356 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 0, 2024-04-25T08:59:54.356 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-25T08:59:54.356 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 0, 2024-04-25T08:59:54.356 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 8289, 2024-04-25T08:59:54.356 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 77267535, 2024-04-25T08:59:54.356 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 8289, 2024-04-25T08:59:54.356 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 77267535, 2024-04-25T08:59:54.356 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-25T08:59:54.356 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:59:54.356 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:59:54.356 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:59:54.357 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:59:54.357 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.357 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-objecter_ops": { 2024-04-25T08:59:54.357 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-25T08:59:54.357 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 1024, 2024-04-25T08:59:54.357 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-25T08:59:54.357 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 0, 2024-04-25T08:59:54.357 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 0, 2024-04-25T08:59:54.357 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-25T08:59:54.357 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 0, 2024-04-25T08:59:54.357 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 8289, 2024-04-25T08:59:54.357 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 8289, 2024-04-25T08:59:54.357 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 8289, 2024-04-25T08:59:54.357 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 8289, 2024-04-25T08:59:54.358 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-25T08:59:54.358 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:59:54.358 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:59:54.358 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:59:54.358 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:59:54.358 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.358 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-write_buf_throttle": { 2024-04-25T08:59:54.358 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-25T08:59:54.358 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 3758096384, 2024-04-25T08:59:54.358 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-25T08:59:54.358 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 1528, 2024-04-25T08:59:54.358 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 166552, 2024-04-25T08:59:54.358 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-25T08:59:54.358 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 1528, 2024-04-25T08:59:54.359 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-04-25T08:59:54.359 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-04-25T08:59:54.359 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 62, 2024-04-25T08:59:54.359 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 166552, 2024-04-25T08:59:54.359 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-25T08:59:54.359 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:59:54.359 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:59:54.359 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:59:54.359 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:59:54.359 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T08:59:54.359 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-write_buf_throttle-0x5623a11346e0": { 2024-04-25T08:59:54.359 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 84, 2024-04-25T08:59:54.359 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 3758096384, 2024-04-25T08:59:54.359 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-25T08:59:54.360 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 21705, 2024-04-25T08:59:54.360 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 76917192, 2024-04-25T08:59:54.360 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-25T08:59:54.360 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 21705, 2024-04-25T08:59:54.360 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-04-25T08:59:54.360 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-04-25T08:59:54.360 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 6503, 2024-04-25T08:59:54.360 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 76917108, 2024-04-25T08:59:54.360 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-25T08:59:54.360 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T08:59:54.360 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T08:59:54.360 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T08:59:54.360 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:59:54.360 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:59:54.360 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T08:59:54.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.318+0000 7f2b4c893640 1 -- 172.21.15.43:0/2206435619 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2b2806b500 msgr2=0x7f2b2806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:54.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.318+0000 7f2b4c893640 1 --2- 172.21.15.43:0/2206435619 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2b2806b500 0x7f2b2806d9c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f2b38034000 tx=0x7f2b3802e000 comp rx=0 tx=0).stop 2024-04-25T08:59:54.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.318+0000 7f2b4c893640 1 -- 172.21.15.43:0/2206435619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b48161fe0 msgr2=0x7f2b48160a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:54.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.318+0000 7f2b4c893640 1 --2- 172.21.15.43:0/2206435619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b48161fe0 0x7f2b48160a40 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f2b4000efc0 tx=0x7f2b4000c490 comp rx=0 tx=0).stop 2024-04-25T08:59:54.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.318+0000 7f2b4c893640 1 -- 172.21.15.43:0/2206435619 shutdown_connections 2024-04-25T08:59:54.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.318+0000 7f2b4c893640 1 --2- 172.21.15.43:0/2206435619 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2b48161fe0 0x7f2b48160a40 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.318+0000 7f2b4c893640 1 --2- 172.21.15.43:0/2206435619 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2b2806b500 0x7f2b2806d9c0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.318+0000 7f2b4c893640 1 --2- 172.21.15.43:0/2206435619 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2b4815e090 0x7f2b48160500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.318+0000 7f2b4c893640 1 --2- 172.21.15.43:0/2206435619 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2b480b5c70 0x7f2b48153080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.318+0000 7f2b4c893640 1 -- 172.21.15.43:0/2206435619 >> 172.21.15.43:0/2206435619 conn(0x7f2b480d0560 msgr2=0x7f2b480b3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:54.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.318+0000 7f2b4c893640 1 -- 172.21.15.43:0/2206435619 shutdown_connections 2024-04-25T08:59:54.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.318+0000 7f2b4c893640 1 -- 172.21.15.43:0/2206435619 wait complete. 2024-04-25T08:59:54.363 INFO:tasks.cephfs.test_scrub_checks:num of strays: 2000 2024-04-25T08:59:54.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:54.364 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start '~mdsdir' recursive 2024-04-25T08:59:54.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.481+0000 7f69cae3f640 1 Processor -- start 2024-04-25T08:59:54.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.481+0000 7f69cae3f640 1 -- start start 2024-04-25T08:59:54.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.482+0000 7f69cae3f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f69c415baa0 0x7f69c415e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.482+0000 7f69cae3f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69c40b37a0 con 0x7f69c40b4ea0 2024-04-25T08:59:54.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.482+0000 7f69cae3f640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69c415eab0 con 0x7f69c415b6c0 2024-04-25T08:59:54.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.482+0000 7f69cae3f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69c415fce0 con 0x7f69c415baa0 2024-04-25T08:59:54.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.482+0000 7f69c3fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f69c415baa0 0x7f69c415e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.482+0000 7f69c93dc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f69c40b4ea0 0x7f69c40d3760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:46420/0 (socket says 172.21.15.43:46420) 2024-04-25T08:59:54.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.482+0000 7f69c93dc640 1 -- 172.21.15.43:0/1179815891 learned_addr learned my addr 172.21.15.43:0/1179815891 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T08:59:54.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.482+0000 7f69c3fff640 1 -- 172.21.15.43:0/1179815891 >> v1:172.21.15.193:6789/0 conn(0x7f69c415b6c0 legacy=0x7f69c415f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:54.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.482+0000 7f69c3fff640 1 -- 172.21.15.43:0/1179815891 >> v1:172.21.15.43:6789/0 conn(0x7f69c40b4ea0 legacy=0x7f69c40d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:54.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.482+0000 7f69c3fff640 1 -- 172.21.15.43:0/1179815891 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c415ed90 con 0x7f69c415baa0 2024-04-25T08:59:54.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.483+0000 7f69c3fff640 1 --2- 172.21.15.43:0/1179815891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69c415baa0 0x7f69c415e410 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f69b00097a0 tx=0x7f69b002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acc85672f62894aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:54.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.483+0000 7f69c37fe640 1 -- 172.21.15.43:0/1179815891 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b0037070 con 0x7f69c415baa0 2024-04-25T08:59:54.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.483+0000 7f69c37fe640 1 -- 172.21.15.43:0/1179815891 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b0032440 con 0x7f69c415baa0 2024-04-25T08:59:54.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.483+0000 7f69c37fe640 1 -- 172.21.15.43:0/1179815891 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b0032810 con 0x7f69c415baa0 2024-04-25T08:59:54.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.483+0000 7f69cae3f640 1 -- 172.21.15.43:0/1179815891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69c415baa0 msgr2=0x7f69c415e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:54.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.483+0000 7f69cae3f640 1 --2- 172.21.15.43:0/1179815891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69c415baa0 0x7f69c415e410 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f69b00097a0 tx=0x7f69b002a1e0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.484+0000 7f69cae3f640 1 -- 172.21.15.43:0/1179815891 shutdown_connections 2024-04-25T08:59:54.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.484+0000 7f69cae3f640 1 --2- 172.21.15.43:0/1179815891 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69c415baa0 0x7f69c415e410 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.484+0000 7f69cae3f640 1 -- 172.21.15.43:0/1179815891 >> 172.21.15.43:0/1179815891 conn(0x7f69c408d460 msgr2=0x7f69c409cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:54.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.484+0000 7f69cae3f640 1 -- 172.21.15.43:0/1179815891 shutdown_connections 2024-04-25T08:59:54.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.484+0000 7f69cae3f640 1 -- 172.21.15.43:0/1179815891 wait complete. 2024-04-25T08:59:54.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.485+0000 7f69cae3f640 1 Processor -- start 2024-04-25T08:59:54.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.485+0000 7f69cae3f640 1 -- start start 2024-04-25T08:59:54.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.486+0000 7f69cae3f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69c40b4ea0 0x7f69c40b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.486+0000 7f69cae3f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69c415b6c0 0x7f69c40c9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.486+0000 7f69cae3f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69c415baa0 0x7f69c40ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.486+0000 7f69cae3f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c40b5310 con 0x7f69c40b4ea0 2024-04-25T08:59:54.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.486+0000 7f69cae3f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c415f350 con 0x7f69c415baa0 2024-04-25T08:59:54.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.486+0000 7f69cae3f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69c40d3e70 con 0x7f69c415b6c0 2024-04-25T08:59:54.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.486+0000 7f69c8bdb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69c40b4ea0 0x7f69c40b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.486+0000 7f69c8bdb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69c40b4ea0 0x7f69c40b3ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55260/0 (socket says 172.21.15.43:55260) 2024-04-25T08:59:54.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.486+0000 7f69c8bdb640 1 -- 172.21.15.43:0/1919468102 learned_addr learned my addr 172.21.15.43:0/1919468102 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:59:54.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.486+0000 7f69c3fff640 1 --2- 172.21.15.43:0/1919468102 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69c415b6c0 0x7f69c40c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.486+0000 7f69c93dc640 1 --2- 172.21.15.43:0/1919468102 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69c415baa0 0x7f69c40ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.486+0000 7f69c8bdb640 1 -- 172.21.15.43:0/1919468102 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69c415b6c0 msgr2=0x7f69c40c9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:54.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.486+0000 7f69c8bdb640 1 --2- 172.21.15.43:0/1919468102 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69c415b6c0 0x7f69c40c9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.486+0000 7f69c8bdb640 1 -- 172.21.15.43:0/1919468102 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69c415baa0 msgr2=0x7f69c40ccfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:54.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.486+0000 7f69c8bdb640 1 --2- 172.21.15.43:0/1919468102 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69c415baa0 0x7f69c40ccfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.486+0000 7f69c8bdb640 1 -- 172.21.15.43:0/1919468102 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c40c9d10 con 0x7f69c40b4ea0 2024-04-25T08:59:54.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.486+0000 7f69c8bdb640 1 --2- 172.21.15.43:0/1919468102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69c40b4ea0 0x7f69c40b3ef0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f69b800c430 tx=0x7f69b800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:54.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.487+0000 7f69c17fa640 1 -- 172.21.15.43:0/1919468102 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b8011790 con 0x7f69c40b4ea0 2024-04-25T08:59:54.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.487+0000 7f69c17fa640 1 -- 172.21.15.43:0/1919468102 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b8017070 con 0x7f69c40b4ea0 2024-04-25T08:59:54.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.487+0000 7f69c17fa640 1 -- 172.21.15.43:0/1919468102 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b8011bb0 con 0x7f69c40b4ea0 2024-04-25T08:59:54.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.487+0000 7f69cae3f640 1 -- 172.21.15.43:0/1919468102 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69c40c9ff0 con 0x7f69c40b4ea0 2024-04-25T08:59:54.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.487+0000 7f69cae3f640 1 -- 172.21.15.43:0/1919468102 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69c414ae80 con 0x7f69c40b4ea0 2024-04-25T08:59:54.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.488+0000 7f69c17fa640 1 -- 172.21.15.43:0/1919468102 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f69b801c460 con 0x7f69c40b4ea0 2024-04-25T08:59:54.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.489+0000 7f69c17fa640 1 --2- 172.21.15.43:0/1919468102 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f69a806b4d0 0x7f69a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.489+0000 7f69c17fa640 1 -- 172.21.15.43:0/1919468102 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f69b808a040 con 0x7f69c40b4ea0 2024-04-25T08:59:54.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.489+0000 7f69c3fff640 1 --2- 172.21.15.43:0/1919468102 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f69a806b4d0 0x7f69a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.489+0000 7f69c3fff640 1 --2- 172.21.15.43:0/1919468102 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f69a806b4d0 0x7f69a806d990 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f69b0034000 tx=0x7f69b002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:54.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.489+0000 7f69cdc18740 1 Processor -- start 2024-04-25T08:59:54.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.490+0000 7f69cdc18740 1 -- start start 2024-04-25T08:59:54.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.490+0000 7f69cdc18740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561466e97080 0x561466e97460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.490+0000 7f69cdc18740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561466e97aa0 0x561466e67200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.490+0000 7f69cdc18740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561466e67740 0x561466e8f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.490+0000 7f69cdc18740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x561466dd00a0 con 0x561466e97aa0 2024-04-25T08:59:54.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.490+0000 7f69cdc18740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x561466d78d40 con 0x561466e67740 2024-04-25T08:59:54.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.490+0000 7f69cdc18740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x561466dcc140 con 0x561466e97080 2024-04-25T08:59:54.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.490+0000 7f69c3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561466e97aa0 0x561466e67200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.490+0000 7f69c3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561466e97aa0 0x561466e67200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55270/0 (socket says 172.21.15.43:55270) 2024-04-25T08:59:54.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.490+0000 7f69c3fff640 1 -- 172.21.15.43:0/2864949504 learned_addr learned my addr 172.21.15.43:0/2864949504 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:59:54.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.490+0000 7f69c8bdb640 1 --2- 172.21.15.43:0/2864949504 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561466e97080 0x561466e97460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.491+0000 7f69c3fff640 1 -- 172.21.15.43:0/2864949504 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561466e97080 msgr2=0x561466e97460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:54.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.491+0000 7f69c3fff640 1 --2- 172.21.15.43:0/2864949504 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561466e97080 0x561466e97460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.491+0000 7f69c3fff640 1 -- 172.21.15.43:0/2864949504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561466e67740 msgr2=0x561466e8f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:54.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.491+0000 7f69c93dc640 1 --2- 172.21.15.43:0/2864949504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561466e67740 0x561466e8f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.491+0000 7f69c3fff640 1 --2- 172.21.15.43:0/2864949504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561466e67740 0x561466e8f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.491+0000 7f69c3fff640 1 -- 172.21.15.43:0/2864949504 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561466bb0a20 con 0x561466e97aa0 2024-04-25T08:59:54.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.491+0000 7f69c93dc640 1 --2- 172.21.15.43:0/2864949504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561466e67740 0x561466e8f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T08:59:54.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.491+0000 7f69c3fff640 1 --2- 172.21.15.43:0/2864949504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561466e97aa0 0x561466e67200 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f69b0009890 tx=0x7f69b002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1c73025f13a7298 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:54.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.491+0000 7f69ae7fc640 1 -- 172.21.15.43:0/2864949504 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b004e070 con 0x561466e97aa0 2024-04-25T08:59:54.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.491+0000 7f69ae7fc640 1 -- 172.21.15.43:0/2864949504 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b0032de0 con 0x561466e97aa0 2024-04-25T08:59:54.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.491+0000 7f69ae7fc640 1 -- 172.21.15.43:0/2864949504 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b0037040 con 0x561466e97aa0 2024-04-25T08:59:54.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.491+0000 7f69cdc18740 1 -- 172.21.15.43:0/2864949504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561466e97aa0 msgr2=0x561466e67200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:54.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.491+0000 7f69cdc18740 1 --2- 172.21.15.43:0/2864949504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561466e97aa0 0x561466e67200 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f69b0009890 tx=0x7f69b002c000 comp rx=0 tx=0).stop 2024-04-25T08:59:54.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.492+0000 7f69cdc18740 1 -- 172.21.15.43:0/2864949504 shutdown_connections 2024-04-25T08:59:54.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.492+0000 7f69cdc18740 1 --2- 172.21.15.43:0/2864949504 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561466e67740 0x561466e8f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.492+0000 7f69cdc18740 1 --2- 172.21.15.43:0/2864949504 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561466e97aa0 0x561466e67200 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.492+0000 7f69cdc18740 1 --2- 172.21.15.43:0/2864949504 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561466e97080 0x561466e97460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.492+0000 7f69cdc18740 1 -- 172.21.15.43:0/2864949504 >> 172.21.15.43:0/2864949504 conn(0x561466e69470 msgr2=0x561466eb1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:54.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.492+0000 7f69cdc18740 1 -- 172.21.15.43:0/2864949504 shutdown_connections 2024-04-25T08:59:54.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.492+0000 7f69cdc18740 1 -- 172.21.15.43:0/2864949504 wait complete. 2024-04-25T08:59:54.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.492+0000 7f69cdc18740 1 -- start start 2024-04-25T08:59:54.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.492+0000 7f69cdc18740 1 Processor -- start 2024-04-25T08:59:54.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.493+0000 7f69cdc18740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561466e97080 0x561466e98820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.493+0000 7f69cdc18740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561466e97aa0 0x561466e85d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.493+0000 7f69cdc18740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561466e969a0 0x561466e86260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.493+0000 7f69cdc18740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x561466e937d0 con 0x561466e97080 2024-04-25T08:59:54.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.493+0000 7f69cdc18740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x561466d6e3d0 con 0x561466e97aa0 2024-04-25T08:59:54.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.493+0000 7f69cdc18740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x561466e93a70 con 0x561466e969a0 2024-04-25T08:59:54.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.493+0000 7f69cdc18740 10 client.0 _ll_register_callbacks cb 0x561466ae3e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:59:54.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.493+0000 7f69c8bdb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561466e97080 0x561466e98820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.493+0000 7f69c8bdb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561466e97080 0x561466e98820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55274/0 (socket says 172.21.15.43:55274) 2024-04-25T08:59:54.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.493+0000 7f69c8bdb640 1 -- 172.21.15.43:0/3668633952 learned_addr learned my addr 172.21.15.43:0/3668633952 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:59:54.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.493+0000 7f69c3fff640 1 --2- 172.21.15.43:0/3668633952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561466e97aa0 0x561466e85d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.493+0000 7f69c93dc640 1 --2- 172.21.15.43:0/3668633952 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561466e969a0 0x561466e86260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.493+0000 7f69c8bdb640 1 -- 172.21.15.43:0/3668633952 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561466e969a0 msgr2=0x561466e86260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:54.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.494+0000 7f69c8bdb640 1 --2- 172.21.15.43:0/3668633952 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561466e969a0 0x561466e86260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.494+0000 7f69c8bdb640 1 -- 172.21.15.43:0/3668633952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561466e97aa0 msgr2=0x561466e85d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:54.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.494+0000 7f69c8bdb640 1 --2- 172.21.15.43:0/3668633952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561466e97aa0 0x561466e85d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.494+0000 7f69c8bdb640 1 -- 172.21.15.43:0/3668633952 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561466aa4310 con 0x561466e97080 2024-04-25T08:59:54.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.494+0000 7f69c8bdb640 1 --2- 172.21.15.43:0/3668633952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561466e97080 0x561466e98820 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f69b8058520 tx=0x7f69b805c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=680633fbbe666c25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:54.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.494+0000 7f69a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:59:54.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.494+0000 7f69a37fe640 1 -- 172.21.15.43:0/3668633952 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b8086060 con 0x561466e97080 2024-04-25T08:59:54.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.494+0000 7f69a37fe640 1 -- 172.21.15.43:0/3668633952 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(75..75 src has 1..75) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f69b8054e90 con 0x561466e97080 2024-04-25T08:59:54.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.494+0000 7f69cdc18740 1 -- 172.21.15.43:0/3668633952 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f69b0032de0 con 0x561466e97080 2024-04-25T08:59:54.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.494+0000 7f69a37fe640 1 -- 172.21.15.43:0/3668633952 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69b8070070 con 0x561466e97080 2024-04-25T08:59:54.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.494+0000 7f69cdc18740 10 client.7350 fetch_fsmap learned FSMap version 48 2024-04-25T08:59:54.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.494+0000 7f69cdc18740 1 -- 172.21.15.43:0/3668633952 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x561466c111b0 con 0x561466e97080 2024-04-25T08:59:54.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.494+0000 7f69a37fe640 1 -- 172.21.15.43:0/3668633952 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b0032de0 con 0x561466e97080 2024-04-25T08:59:54.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.494+0000 7f69a37fe640 1 -- 172.21.15.43:0/3668633952 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 48) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f69b8057c70 con 0x561466e97080 2024-04-25T08:59:54.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.494+0000 7f69cdc18740 10 client.7350 fetch_fsmap finished waiting for FSMap version 48 2024-04-25T08:59:54.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.494+0000 7f69cdc18740 10 client.7350 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T08:59:54.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.494+0000 7f69cdc18740 20 client.7350 populate_metadata read hostname 'smithi043' 2024-04-25T08:59:54.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.494+0000 7f69cdc18740 1 --2- 172.21.15.43:0/3668633952 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x561466e951a0 0x561466e955a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.494+0000 7f69cdc18740 4 client.7350 mds_command: new command op to 7164 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:59:54.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.494+0000 7f69cdc18740 1 -- 172.21.15.43:0/3668633952 --> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561466ddd5e0 con 0x561466e951a0 2024-04-25T08:59:54.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.495+0000 7f69c3fff640 1 --2- 172.21.15.43:0/3668633952 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x561466e951a0 0x561466e955a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.495+0000 7f69c3fff640 1 --2- 172.21.15.43:0/3668633952 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x561466e951a0 0x561466e955a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3783a348689c08b3 server_cookie=58183d6c5e71a7d3 in_seq=0 out_seq=0 2024-04-25T08:59:54.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.495+0000 7f69a37fe640 10 client.7350 ms_handle_connect on v2:172.21.15.193:6835/3458098162 2024-04-25T08:59:54.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.495+0000 7f69a37fe640 1 -- 172.21.15.43:0/3668633952 <== mds.0 v2:172.21.15.193:6835/3458098162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561466ddd5e0 con 0x561466e951a0 2024-04-25T08:59:54.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.495+0000 7f69a37fe640 10 client.7350 handle_command_reply: tid=0 2024-04-25T08:59:54.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.495+0000 7f69cdc18740 1 client.7350 shutdown 2024-04-25T08:59:54.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.496+0000 7f69cdc18740 1 -- 172.21.15.43:0/3668633952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561466e97080 msgr2=0x561466e98820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:54.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.496+0000 7f69cdc18740 1 --2- 172.21.15.43:0/3668633952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561466e97080 0x561466e98820 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f69b8058520 tx=0x7f69b805c980 comp rx=0 tx=0).stop 2024-04-25T08:59:54.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.496+0000 7f69cdc18740 1 -- 172.21.15.43:0/3668633952 shutdown_connections 2024-04-25T08:59:54.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.496+0000 7f69cdc18740 1 --2- 172.21.15.43:0/3668633952 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x561466e951a0 0x561466e955a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.496+0000 7f69cdc18740 1 --2- 172.21.15.43:0/3668633952 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561466e969a0 0x561466e86260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.496+0000 7f69a37fe640 0 client.7350 ms_handle_reset on v2:172.21.15.193:6835/3458098162 2024-04-25T08:59:54.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.496+0000 7f69cdc18740 1 --2- 172.21.15.43:0/3668633952 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561466e97aa0 0x561466e85d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.496+0000 7f69cdc18740 1 --2- 172.21.15.43:0/3668633952 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561466e97080 0x561466e98820 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.496+0000 7f69cdc18740 1 -- 172.21.15.43:0/3668633952 >> 172.21.15.43:0/3668633952 conn(0x561466e67740 msgr2=0x561466e8ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:54.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.496+0000 7f69cdc18740 1 -- 172.21.15.43:0/3668633952 shutdown_connections 2024-04-25T08:59:54.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.496+0000 7f69cdc18740 1 -- 172.21.15.43:0/3668633952 wait complete. 2024-04-25T08:59:54.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.496+0000 7f69cdc18740 20 client.7350 trim_cache size 0 max 16384 2024-04-25T08:59:54.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.511+0000 7f69cdc18740 1 Processor -- start 2024-04-25T08:59:54.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.511+0000 7f69cdc18740 1 -- start start 2024-04-25T08:59:54.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.512+0000 7f69cdc18740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561466e969a0 0x561466e97ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.512+0000 7f69cdc18740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561466e97080 0x561466e983e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.512+0000 7f69cdc18740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561466e97aa0 0x561466ec4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.512+0000 7f69cdc18740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c41ffe70 con 0x561466e97aa0 2024-04-25T08:59:54.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.512+0000 7f69cdc18740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c414c4d0 con 0x561466e969a0 2024-04-25T08:59:54.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.512+0000 7f69cdc18740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69c406bd60 con 0x561466e97080 2024-04-25T08:59:54.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.512+0000 7f69c93dc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561466e97aa0 0x561466ec4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.512+0000 7f69c93dc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561466e97aa0 0x561466ec4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55280/0 (socket says 172.21.15.43:55280) 2024-04-25T08:59:54.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.512+0000 7f69c93dc640 1 -- 172.21.15.43:0/4257425486 learned_addr learned my addr 172.21.15.43:0/4257425486 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:59:54.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.512+0000 7f69c3fff640 1 --2- 172.21.15.43:0/4257425486 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561466e97080 0x561466e983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.512+0000 7f69c8bdb640 1 --2- 172.21.15.43:0/4257425486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561466e969a0 0x561466e97ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.512+0000 7f69c93dc640 1 -- 172.21.15.43:0/4257425486 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561466e97080 msgr2=0x561466e983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:54.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.512+0000 7f69c93dc640 1 --2- 172.21.15.43:0/4257425486 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561466e97080 0x561466e983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.512+0000 7f69c93dc640 1 -- 172.21.15.43:0/4257425486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561466e969a0 msgr2=0x561466e97ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:54.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.512+0000 7f69c93dc640 1 --2- 172.21.15.43:0/4257425486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561466e969a0 0x561466e97ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.512+0000 7f69c93dc640 1 -- 172.21.15.43:0/4257425486 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561466a7d490 con 0x561466e97aa0 2024-04-25T08:59:54.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.512+0000 7f69c8bdb640 1 --2- 172.21.15.43:0/4257425486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561466e969a0 0x561466e97ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:59:54.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.512+0000 7f69c93dc640 1 --2- 172.21.15.43:0/4257425486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561466e97aa0 0x561466ec4270 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f69bc002150 tx=0x7f69bc0019f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a4c8d5b867874a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:54.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.512+0000 7f69adffb640 1 -- 172.21.15.43:0/4257425486 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69bc01a020 con 0x561466e97aa0 2024-04-25T08:59:54.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.512+0000 7f69adffb640 1 -- 172.21.15.43:0/4257425486 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69bc007790 con 0x561466e97aa0 2024-04-25T08:59:54.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.512+0000 7f69adffb640 1 -- 172.21.15.43:0/4257425486 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69bc016450 con 0x561466e97aa0 2024-04-25T08:59:54.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.513+0000 7f69cdc18740 1 -- 172.21.15.43:0/4257425486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561466e97aa0 msgr2=0x561466ec4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:54.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.513+0000 7f69cdc18740 1 --2- 172.21.15.43:0/4257425486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561466e97aa0 0x561466ec4270 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f69bc002150 tx=0x7f69bc0019f0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.513+0000 7f69cdc18740 1 -- 172.21.15.43:0/4257425486 shutdown_connections 2024-04-25T08:59:54.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.513+0000 7f69cdc18740 1 --2- 172.21.15.43:0/4257425486 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561466e97aa0 0x561466ec4270 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.513+0000 7f69cdc18740 1 --2- 172.21.15.43:0/4257425486 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561466e97080 0x561466e983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.513+0000 7f69cdc18740 1 --2- 172.21.15.43:0/4257425486 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561466e969a0 0x561466e97ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.513+0000 7f69cdc18740 1 -- 172.21.15.43:0/4257425486 >> 172.21.15.43:0/4257425486 conn(0x561466e67740 msgr2=0x561466eb5280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:54.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.513+0000 7f69cdc18740 1 -- 172.21.15.43:0/4257425486 shutdown_connections 2024-04-25T08:59:54.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.513+0000 7f69cdc18740 1 -- 172.21.15.43:0/4257425486 wait complete. 2024-04-25T08:59:54.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.514+0000 7f69cdc18740 1 -- start start 2024-04-25T08:59:54.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.514+0000 7f69cdc18740 1 Processor -- start 2024-04-25T08:59:54.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.514+0000 7f69cdc18740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561466e969a0 0x561466ecb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.514+0000 7f69cdc18740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561466e97080 0x561466ecbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.514+0000 7f69cdc18740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561466e97aa0 0x561466f1c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.515+0000 7f69cdc18740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c414bed0 con 0x561466e97080 2024-04-25T08:59:54.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.515+0000 7f69cdc18740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x561466dc75e0 con 0x561466e97aa0 2024-04-25T08:59:54.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.515+0000 7f69cdc18740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x561466eb5280 con 0x561466e969a0 2024-04-25T08:59:54.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.515+0000 7f69cdc18740 10 client.0 _ll_register_callbacks cb 0x7f69b0009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:59:54.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.515+0000 7f69c3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561466e97080 0x561466ecbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.515+0000 7f69c3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561466e97080 0x561466ecbc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55292/0 (socket says 172.21.15.43:55292) 2024-04-25T08:59:54.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.515+0000 7f69c3fff640 1 -- 172.21.15.43:0/3824966176 learned_addr learned my addr 172.21.15.43:0/3824966176 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:59:54.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.515+0000 7f69c8bdb640 1 --2- 172.21.15.43:0/3824966176 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561466e969a0 0x561466ecb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.515+0000 7f69c93dc640 1 --2- 172.21.15.43:0/3824966176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561466e97aa0 0x561466f1c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.515+0000 7f69c3fff640 1 -- 172.21.15.43:0/3824966176 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561466e969a0 msgr2=0x561466ecb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.515+0000 7f69c3fff640 1 --2- 172.21.15.43:0/3824966176 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561466e969a0 0x561466ecb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.515+0000 7f69c3fff640 1 -- 172.21.15.43:0/3824966176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561466e97aa0 msgr2=0x561466f1c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.515+0000 7f69c3fff640 1 --2- 172.21.15.43:0/3824966176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561466e97aa0 0x561466f1c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.515+0000 7f69c3fff640 1 -- 172.21.15.43:0/3824966176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561466e667f0 con 0x561466e97080 2024-04-25T08:59:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.515+0000 7f69c3fff640 1 --2- 172.21.15.43:0/3824966176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561466e97080 0x561466ecbc40 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f69b0009560 tx=0x7f69b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc4d5814a271557a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.515+0000 7f69a37fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:59:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.515+0000 7f69a37fe640 1 -- 172.21.15.43:0/3824966176 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b004e040 con 0x561466e97080 2024-04-25T08:59:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.515+0000 7f69a37fe640 1 -- 172.21.15.43:0/3824966176 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(75..75 src has 1..75) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f69b00562f0 con 0x561466e97080 2024-04-25T08:59:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.515+0000 7f69cdc18740 1 -- 172.21.15.43:0/3824966176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f69bc007790 con 0x561466e97080 2024-04-25T08:59:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.515+0000 7f69a37fe640 1 -- 172.21.15.43:0/3824966176 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69b0065050 con 0x561466e97080 2024-04-25T08:59:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.515+0000 7f69cdc18740 10 client.7356 fetch_fsmap learned FSMap version 48 2024-04-25T08:59:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.515+0000 7f69cdc18740 1 -- 172.21.15.43:0/3824966176 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x561466f1cce0 con 0x561466e97080 2024-04-25T08:59:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.516+0000 7f69a37fe640 1 -- 172.21.15.43:0/3824966176 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69bc007790 con 0x561466e97080 2024-04-25T08:59:54.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.516+0000 7f69a37fe640 1 -- 172.21.15.43:0/3824966176 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 48) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f69b0068020 con 0x561466e97080 2024-04-25T08:59:54.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.516+0000 7f69cdc18740 10 client.7356 fetch_fsmap finished waiting for FSMap version 48 2024-04-25T08:59:54.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.516+0000 7f69cdc18740 10 client.7356 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T08:59:54.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.516+0000 7f69cdc18740 20 client.7356 populate_metadata read hostname 'smithi043' 2024-04-25T08:59:54.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.516+0000 7f69cdc18740 1 --2- 172.21.15.43:0/3824966176 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x561466e6c350 0x561466e6c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.516+0000 7f69cdc18740 4 client.7356 mds_command: new command op to 7164 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}] 2024-04-25T08:59:54.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.516+0000 7f69cdc18740 1 -- 172.21.15.43:0/3824966176 --> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] -- command(tid 0: {"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}) v1 -- 0x561466dd7930 con 0x561466e6c350 2024-04-25T08:59:54.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.516+0000 7f69c8bdb640 1 --2- 172.21.15.43:0/3824966176 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x561466e6c350 0x561466e6c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.516+0000 7f69c8bdb640 1 --2- 172.21.15.43:0/3824966176 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x561466e6c350 0x561466e6c760 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a55aac18b248ff62 server_cookie=54d4965c4e6629da in_seq=0 out_seq=0 2024-04-25T08:59:54.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.516+0000 7f69a37fe640 10 client.7356 ms_handle_connect on v2:172.21.15.193:6835/3458098162 2024-04-25T08:59:54.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.525+0000 7f69a37fe640 1 -- 172.21.15.43:0/3824966176 <== mds.0 v2:172.21.15.193:6835/3458098162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x561466dd7930 con 0x561466e6c350 2024-04-25T08:59:54.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.525+0000 7f69a37fe640 10 client.7356 handle_command_reply: tid=0 2024-04-25T08:59:54.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.525+0000 7f69cdc18740 1 client.7356 shutdown 2024-04-25T08:59:54.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.525+0000 7f69cdc18740 1 -- 172.21.15.43:0/3824966176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561466e97080 msgr2=0x561466ecbc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:54.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.525+0000 7f69cdc18740 1 --2- 172.21.15.43:0/3824966176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561466e97080 0x561466ecbc40 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f69b0009560 tx=0x7f69b00046c0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.525+0000 7f69cdc18740 1 -- 172.21.15.43:0/3824966176 shutdown_connections 2024-04-25T08:59:54.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.525+0000 7f69cdc18740 1 --2- 172.21.15.43:0/3824966176 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x561466e6c350 0x561466e6c760 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.525+0000 7f69cdc18740 1 --2- 172.21.15.43:0/3824966176 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x561466e97aa0 0x561466f1c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.525+0000 7f69cdc18740 1 --2- 172.21.15.43:0/3824966176 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x561466e97080 0x561466ecbc40 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.525+0000 7f69a37fe640 0 client.7356 ms_handle_reset on v2:172.21.15.193:6835/3458098162 2024-04-25T08:59:54.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.525+0000 7f69cdc18740 1 --2- 172.21.15.43:0/3824966176 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x561466e969a0 0x561466ecb700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.525+0000 7f69cdc18740 1 -- 172.21.15.43:0/3824966176 >> 172.21.15.43:0/3824966176 conn(0x561466e67740 msgr2=0x561466e66040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:54.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.525+0000 7f69cdc18740 1 -- 172.21.15.43:0/3824966176 shutdown_connections 2024-04-25T08:59:54.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.525+0000 7f69cdc18740 1 -- 172.21.15.43:0/3824966176 wait complete. 2024-04-25T08:59:54.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.525+0000 7f69cdc18740 20 client.7356 trim_cache size 0 max 16384 2024-04-25T08:59:54.516 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T08:59:54.516 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-25T08:59:54.516 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "7e31711a-53ad-4f74-8918-a0c2a6362edf", 2024-04-25T08:59:54.516 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-25T08:59:54.516 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T08:59:54.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.526+0000 7f69cae3f640 1 -- 172.21.15.43:0/1919468102 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f69a806b4d0 msgr2=0x7f69a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:54.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.526+0000 7f69cae3f640 1 --2- 172.21.15.43:0/1919468102 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f69a806b4d0 0x7f69a806d990 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f69b0034000 tx=0x7f69b002e000 comp rx=0 tx=0).stop 2024-04-25T08:59:54.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.526+0000 7f69cae3f640 1 -- 172.21.15.43:0/1919468102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69c40b4ea0 msgr2=0x7f69c40b3ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:54.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.526+0000 7f69cae3f640 1 --2- 172.21.15.43:0/1919468102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69c40b4ea0 0x7f69c40b3ef0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f69b800c430 tx=0x7f69b800c8f0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.526+0000 7f69cae3f640 1 -- 172.21.15.43:0/1919468102 shutdown_connections 2024-04-25T08:59:54.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.526+0000 7f69cae3f640 1 --2- 172.21.15.43:0/1919468102 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f69a806b4d0 0x7f69a806d990 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.526+0000 7f69cae3f640 1 --2- 172.21.15.43:0/1919468102 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69c415baa0 0x7f69c40ccfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.526+0000 7f69cae3f640 1 --2- 172.21.15.43:0/1919468102 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69c415b6c0 0x7f69c40c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.526+0000 7f69cae3f640 1 --2- 172.21.15.43:0/1919468102 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69c40b4ea0 0x7f69c40b3ef0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.526+0000 7f69cae3f640 1 -- 172.21.15.43:0/1919468102 >> 172.21.15.43:0/1919468102 conn(0x7f69c408d460 msgr2=0x7f69c415ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:54.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.526+0000 7f69cae3f640 1 -- 172.21.15.43:0/1919468102 shutdown_connections 2024-04-25T08:59:54.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.526+0000 7f69cae3f640 1 -- 172.21.15.43:0/1919468102 wait complete. 2024-04-25T08:59:54.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T08:59:54.526 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T08:59:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.659+0000 7f601276a640 1 Processor -- start 2024-04-25T08:59:54.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.659+0000 7f601276a640 1 -- start start 2024-04-25T08:59:54.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.660+0000 7f601276a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f600c0b5560 0x7f600c0b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.660+0000 7f601276a640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f600c0d2230 0x7f600c0d26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.660+0000 7f601276a640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f600c162610 con 0x7f600c15e090 2024-04-25T08:59:54.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.660+0000 7f601276a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f600c0d1da0 con 0x7f600c0b5560 2024-04-25T08:59:54.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.660+0000 7f601276a640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f600c0d1750 con 0x7f600c0d2230 2024-04-25T08:59:54.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.660+0000 7f600b7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f600c0b5560 0x7f600c0b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.660+0000 7f600b7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f600c0b5560 0x7f600c0b59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55298/0 (socket says 172.21.15.43:55298) 2024-04-25T08:59:54.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.660+0000 7f600b7fe640 1 -- 172.21.15.43:0/976059796 learned_addr learned my addr 172.21.15.43:0/976059796 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:59:54.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.660+0000 7f6010d07640 1 --2- 172.21.15.43:0/976059796 >> v2:172.21.15.193:3300/0 conn(0x7f600c0d2230 0x7f600c0d26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.660+0000 7f600b7fe640 1 -- 172.21.15.43:0/976059796 >> v2:172.21.15.193:3300/0 conn(0x7f600c0d2230 msgr2=0x7f600c0d26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:54.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.660+0000 7f600b7fe640 1 --2- 172.21.15.43:0/976059796 >> v2:172.21.15.193:3300/0 conn(0x7f600c0d2230 0x7f600c0d26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.660+0000 7f600b7fe640 1 -- 172.21.15.43:0/976059796 >> v1:172.21.15.193:6789/0 conn(0x7f600c15e090 legacy=0x7f600c161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:54.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.660+0000 7f600b7fe640 1 -- 172.21.15.43:0/976059796 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f600c0d1a30 con 0x7f600c0b5560 2024-04-25T08:59:54.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.660+0000 7f6010d07640 1 --2- 172.21.15.43:0/976059796 >> v2:172.21.15.193:3300/0 conn(0x7f600c0d2230 0x7f600c0d26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:59:54.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.661+0000 7f600b7fe640 1 --2- 172.21.15.43:0/976059796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f600c0b5560 0x7f600c0b59e0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f600c0b24d0 tx=0x7f600002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=401e24e6f599e67e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:54.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.661+0000 7f600affd640 1 -- 172.21.15.43:0/976059796 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6000037070 con 0x7f600c0b5560 2024-04-25T08:59:54.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.662+0000 7f600affd640 1 -- 172.21.15.43:0/976059796 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600002ae50 con 0x7f600c0b5560 2024-04-25T08:59:54.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.662+0000 7f600affd640 1 -- 172.21.15.43:0/976059796 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6000032860 con 0x7f600c0b5560 2024-04-25T08:59:54.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.662+0000 7f601276a640 1 -- 172.21.15.43:0/976059796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f600c0b5560 msgr2=0x7f600c0b59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:54.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.662+0000 7f601276a640 1 --2- 172.21.15.43:0/976059796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f600c0b5560 0x7f600c0b59e0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f600c0b24d0 tx=0x7f600002a300 comp rx=0 tx=0).stop 2024-04-25T08:59:54.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.662+0000 7f601276a640 1 -- 172.21.15.43:0/976059796 shutdown_connections 2024-04-25T08:59:54.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.662+0000 7f601276a640 1 --2- 172.21.15.43:0/976059796 >> v2:172.21.15.193:3300/0 conn(0x7f600c0d2230 0x7f600c0d26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.662+0000 7f601276a640 1 --2- 172.21.15.43:0/976059796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f600c0b5560 0x7f600c0b59e0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.662+0000 7f601276a640 1 -- 172.21.15.43:0/976059796 >> 172.21.15.43:0/976059796 conn(0x7f600c0d0560 msgr2=0x7f600c0d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:54.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.662+0000 7f601276a640 1 -- 172.21.15.43:0/976059796 shutdown_connections 2024-04-25T08:59:54.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.662+0000 7f601276a640 1 -- 172.21.15.43:0/976059796 wait complete. 2024-04-25T08:59:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.664+0000 7f601276a640 1 Processor -- start 2024-04-25T08:59:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.664+0000 7f601276a640 1 -- start start 2024-04-25T08:59:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.665+0000 7f601276a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f600c0b5560 0x7f600c1f4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.665+0000 7f601276a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f600c0d2230 0x7f600c160200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.665+0000 7f601276a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f600c15e090 0x7f600c1f75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.665+0000 7f601276a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600c1621b0 con 0x7f600c0b5560 2024-04-25T08:59:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.665+0000 7f601276a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600c0b3280 con 0x7f600c15e090 2024-04-25T08:59:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.665+0000 7f601276a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f600c162330 con 0x7f600c0d2230 2024-04-25T08:59:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.665+0000 7f600bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f600c0b5560 0x7f600c1f4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.665+0000 7f600bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f600c0b5560 0x7f600c1f4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55304/0 (socket says 172.21.15.43:55304) 2024-04-25T08:59:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.665+0000 7f600bfff640 1 -- 172.21.15.43:0/2800573311 learned_addr learned my addr 172.21.15.43:0/2800573311 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:59:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.665+0000 7f6010d07640 1 --2- 172.21.15.43:0/2800573311 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f600c15e090 0x7f600c1f75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.665+0000 7f600b7fe640 1 --2- 172.21.15.43:0/2800573311 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f600c0d2230 0x7f600c160200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.665+0000 7f600bfff640 1 -- 172.21.15.43:0/2800573311 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f600c0d2230 msgr2=0x7f600c160200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.665+0000 7f600bfff640 1 --2- 172.21.15.43:0/2800573311 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f600c0d2230 0x7f600c160200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.665+0000 7f600bfff640 1 -- 172.21.15.43:0/2800573311 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f600c15e090 msgr2=0x7f600c1f75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.665+0000 7f600bfff640 1 --2- 172.21.15.43:0/2800573311 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f600c15e090 0x7f600c1f75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.665+0000 7f600bfff640 1 -- 172.21.15.43:0/2800573311 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f600c160740 con 0x7f600c0b5560 2024-04-25T08:59:54.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.666+0000 7f600bfff640 1 --2- 172.21.15.43:0/2800573311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f600c0b5560 0x7f600c1f4ad0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc00c3f0 tx=0x7f5ffc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:54.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.666+0000 7f6008ff9640 1 -- 172.21.15.43:0/2800573311 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc0052d0 con 0x7f600c0b5560 2024-04-25T08:59:54.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.666+0000 7f6008ff9640 1 -- 172.21.15.43:0/2800573311 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ffc005490 con 0x7f600c0b5560 2024-04-25T08:59:54.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.666+0000 7f601276a640 1 -- 172.21.15.43:0/2800573311 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f600c160a10 con 0x7f600c0b5560 2024-04-25T08:59:54.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.666+0000 7f6008ff9640 1 -- 172.21.15.43:0/2800573311 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc007260 con 0x7f600c0b5560 2024-04-25T08:59:54.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.666+0000 7f601276a640 1 -- 172.21.15.43:0/2800573311 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f600c160fd0 con 0x7f600c0b5560 2024-04-25T08:59:54.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.668+0000 7f6008ff9640 1 -- 172.21.15.43:0/2800573311 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5ffc007400 con 0x7f600c0b5560 2024-04-25T08:59:54.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.668+0000 7f6008ff9640 1 --2- 172.21.15.43:0/2800573311 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5ff806b4d0 0x7f5ff806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.668+0000 7f6008ff9640 1 -- 172.21.15.43:0/2800573311 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f5ffc0895e0 con 0x7f600c0b5560 2024-04-25T08:59:54.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.669+0000 7f600b7fe640 1 --2- 172.21.15.43:0/2800573311 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5ff806b4d0 0x7f5ff806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.669+0000 7f600b7fe640 1 --2- 172.21.15.43:0/2800573311 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5ff806b4d0 0x7f5ff806d990 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f6000034000 tx=0x7f600002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:54.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.669+0000 7f601545f740 1 Processor -- start 2024-04-25T08:59:54.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.669+0000 7f601545f740 1 -- start start 2024-04-25T08:59:54.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.670+0000 7f601545f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b36b973b10 0x55b36b976320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.670+0000 7f601545f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b36b973ef0 0x55b36b976860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.670+0000 7f601545f740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55b36b946770 0x55b36b949f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.670+0000 7f601545f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b36b8afda0 con 0x55b36b973b10 2024-04-25T08:59:54.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.670+0000 7f601545f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b36b858b80 con 0x55b36b973ef0 2024-04-25T08:59:54.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.670+0000 7f601545f740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b36b8abe40 con 0x55b36b946770 2024-04-25T08:59:54.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.671+0000 7f600b7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b36b973ef0 0x55b36b976860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.671+0000 7f6010d07640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55b36b946770 0x55b36b949f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.671+0000 7f600bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b36b973b10 0x55b36b976320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.671+0000 7f6010d07640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55b36b946770 0x55b36b949f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:48932/0 (socket says 172.21.15.43:48932) 2024-04-25T08:59:54.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.671+0000 7f600b7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b36b973ef0 0x55b36b976860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:54498/0 (socket says 172.21.15.43:54498) 2024-04-25T08:59:54.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.671+0000 7f6010d07640 1 -- 172.21.15.43:0/3088865184 learned_addr learned my addr 172.21.15.43:0/3088865184 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:59:54.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.671+0000 7f600bfff640 1 -- 172.21.15.43:0/3088865184 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55b36b946770 msgr2=0x55b36b949f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:54.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.671+0000 7f600bfff640 1 --2- 172.21.15.43:0/3088865184 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55b36b946770 0x55b36b949f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.671+0000 7f600bfff640 1 -- 172.21.15.43:0/3088865184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b36b973ef0 msgr2=0x55b36b976860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:54.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.671+0000 7f600bfff640 1 --2- 172.21.15.43:0/3088865184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b36b973ef0 0x55b36b976860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.671+0000 7f600bfff640 1 -- 172.21.15.43:0/3088865184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b36b862f10 con 0x55b36b973b10 2024-04-25T08:59:54.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.671+0000 7f600b7fe640 1 --2- 172.21.15.43:0/3088865184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b36b973ef0 0x55b36b976860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:59:54.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.671+0000 7f600bfff640 1 --2- 172.21.15.43:0/3088865184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b36b973b10 0x55b36b976320 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc087bb0 tx=0x7f5ffc00a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff9eb7751bc600ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:54.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.671+0000 7f5ff1ffb640 1 -- 172.21.15.43:0/3088865184 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc056160 con 0x55b36b973b10 2024-04-25T08:59:54.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.671+0000 7f5ff1ffb640 1 -- 172.21.15.43:0/3088865184 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ffc05b070 con 0x55b36b973b10 2024-04-25T08:59:54.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.671+0000 7f5ff1ffb640 1 -- 172.21.15.43:0/3088865184 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc0544d0 con 0x55b36b973b10 2024-04-25T08:59:54.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.672+0000 7f601545f740 1 -- 172.21.15.43:0/3088865184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b36b973b10 msgr2=0x55b36b976320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:54.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.672+0000 7f601545f740 1 --2- 172.21.15.43:0/3088865184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b36b973b10 0x55b36b976320 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc087bb0 tx=0x7f5ffc00a000 comp rx=0 tx=0).stop 2024-04-25T08:59:54.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.672+0000 7f601545f740 1 -- 172.21.15.43:0/3088865184 shutdown_connections 2024-04-25T08:59:54.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.672+0000 7f601545f740 1 --2- 172.21.15.43:0/3088865184 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55b36b946770 0x55b36b949f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.672+0000 7f601545f740 1 --2- 172.21.15.43:0/3088865184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b36b973ef0 0x55b36b976860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.672+0000 7f601545f740 1 --2- 172.21.15.43:0/3088865184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b36b973b10 0x55b36b976320 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.672+0000 7f601545f740 1 -- 172.21.15.43:0/3088865184 >> 172.21.15.43:0/3088865184 conn(0x55b36b9489e0 msgr2=0x55b36b991320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:54.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.672+0000 7f601545f740 1 -- 172.21.15.43:0/3088865184 shutdown_connections 2024-04-25T08:59:54.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.672+0000 7f601545f740 1 -- 172.21.15.43:0/3088865184 wait complete. 2024-04-25T08:59:54.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.673+0000 7f601545f740 1 -- start start 2024-04-25T08:59:54.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.673+0000 7f601545f740 1 Processor -- start 2024-04-25T08:59:54.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.675+0000 7f601545f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b36b973b10 0x55b36b98d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.675+0000 7f601545f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b36b973ef0 0x55b36b975e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.675+0000 7f601545f740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55b36b98e1a0 0x55b36b976360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.675+0000 7f601545f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b36b94bb90 con 0x55b36b973ef0 2024-04-25T08:59:54.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.675+0000 7f601545f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b36b84e4b0 con 0x55b36b973b10 2024-04-25T08:59:54.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.675+0000 7f601545f740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b36b94be70 con 0x55b36b98e1a0 2024-04-25T08:59:54.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.675+0000 7f600b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b36b973ef0 0x55b36b975e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.675+0000 7f600b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b36b973ef0 0x55b36b975e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55318/0 (socket says 172.21.15.43:55318) 2024-04-25T08:59:54.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.675+0000 7f600b7fe640 1 -- 172.21.15.43:0/2805740277 learned_addr learned my addr 172.21.15.43:0/2805740277 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:59:54.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.675+0000 7f600bfff640 1 --2- 172.21.15.43:0/2805740277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b36b973b10 0x55b36b98d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.675+0000 7f601545f740 10 client.0 _ll_register_callbacks cb 0x55b36b88e180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:59:54.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.675+0000 7f6010d07640 1 --2- 172.21.15.43:0/2805740277 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55b36b98e1a0 0x55b36b976360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.675+0000 7f600b7fe640 1 -- 172.21.15.43:0/2805740277 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55b36b98e1a0 msgr2=0x55b36b976360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:54.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.675+0000 7f600b7fe640 1 --2- 172.21.15.43:0/2805740277 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55b36b98e1a0 0x55b36b976360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.675+0000 7f600b7fe640 1 -- 172.21.15.43:0/2805740277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b36b973b10 msgr2=0x55b36b98d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:54.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.675+0000 7f600b7fe640 1 --2- 172.21.15.43:0/2805740277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b36b973b10 0x55b36b98d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.675+0000 7f600b7fe640 1 -- 172.21.15.43:0/2805740277 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b36b68e7d0 con 0x55b36b973ef0 2024-04-25T08:59:54.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.675+0000 7f600bfff640 1 --2- 172.21.15.43:0/2805740277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b36b973b10 0x55b36b98d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T08:59:54.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.676+0000 7f600b7fe640 1 --2- 172.21.15.43:0/2805740277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b36b973ef0 0x55b36b975e20 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f60000090b0 tx=0x7f6000004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bef43446295cd32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:54.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.676+0000 7f5feaffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:59:54.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.676+0000 7f5feaffd640 1 -- 172.21.15.43:0/2805740277 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6000037040 con 0x55b36b973ef0 2024-04-25T08:59:54.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.676+0000 7f5feaffd640 1 -- 172.21.15.43:0/2805740277 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(75..75 src has 1..75) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f60000478e0 con 0x55b36b973ef0 2024-04-25T08:59:54.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.676+0000 7f601545f740 1 -- 172.21.15.43:0/2805740277 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ffc05b070 con 0x55b36b973ef0 2024-04-25T08:59:54.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.676+0000 7f5feaffd640 1 -- 172.21.15.43:0/2805740277 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f600005b050 con 0x55b36b973ef0 2024-04-25T08:59:54.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.676+0000 7f601545f740 10 client.7368 fetch_fsmap learned FSMap version 48 2024-04-25T08:59:54.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.676+0000 7f601545f740 1 -- 172.21.15.43:0/2805740277 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x55b36b5c43f0 con 0x55b36b973ef0 2024-04-25T08:59:54.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.677+0000 7f5feaffd640 1 -- 172.21.15.43:0/2805740277 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600002ae50 con 0x55b36b973ef0 2024-04-25T08:59:54.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.677+0000 7f5feaffd640 1 -- 172.21.15.43:0/2805740277 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 48) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f600005e020 con 0x55b36b973ef0 2024-04-25T08:59:54.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.677+0000 7f601545f740 10 client.7368 fetch_fsmap finished waiting for FSMap version 48 2024-04-25T08:59:54.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.677+0000 7f601545f740 10 client.7368 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T08:59:54.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.677+0000 7f601545f740 20 client.7368 populate_metadata read hostname 'smithi043' 2024-04-25T08:59:54.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.677+0000 7f601545f740 1 --2- 172.21.15.43:0/2805740277 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x55b36b9930e0 0x55b36b993530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.677+0000 7f601545f740 4 client.7368 mds_command: new command op to 7164 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T08:59:54.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.677+0000 7f601545f740 1 -- 172.21.15.43:0/2805740277 --> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5ffc05b070 con 0x55b36b9930e0 2024-04-25T08:59:54.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.677+0000 7f600bfff640 1 --2- 172.21.15.43:0/2805740277 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x55b36b9930e0 0x55b36b993530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.677+0000 7f5feaffd640 10 client.7368 ms_handle_connect on v2:172.21.15.193:6835/3458098162 2024-04-25T08:59:54.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.677+0000 7f600bfff640 1 --2- 172.21.15.43:0/2805740277 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x55b36b9930e0 0x55b36b993530 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ae302f8f024a435 server_cookie=eae19f56d9c55737 in_seq=0 out_seq=0 2024-04-25T08:59:54.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.678+0000 7f5feaffd640 1 -- 172.21.15.43:0/2805740277 <== mds.0 v2:172.21.15.193:6835/3458098162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7f5ffc05b070 con 0x55b36b9930e0 2024-04-25T08:59:54.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.678+0000 7f5feaffd640 10 client.7368 handle_command_reply: tid=0 2024-04-25T08:59:54.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.678+0000 7f601545f740 1 client.7368 shutdown 2024-04-25T08:59:54.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.679+0000 7f601545f740 1 -- 172.21.15.43:0/2805740277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b36b973ef0 msgr2=0x55b36b975e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:54.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.679+0000 7f601545f740 1 --2- 172.21.15.43:0/2805740277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b36b973ef0 0x55b36b975e20 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f60000090b0 tx=0x7f6000004730 comp rx=0 tx=0).stop 2024-04-25T08:59:54.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.679+0000 7f601545f740 1 -- 172.21.15.43:0/2805740277 shutdown_connections 2024-04-25T08:59:54.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.679+0000 7f601545f740 1 --2- 172.21.15.43:0/2805740277 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x55b36b9930e0 0x55b36b993530 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.679+0000 7f601545f740 1 --2- 172.21.15.43:0/2805740277 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55b36b98e1a0 0x55b36b976360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.679+0000 7f5feaffd640 0 client.7368 ms_handle_reset on v2:172.21.15.193:6835/3458098162 2024-04-25T08:59:54.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.679+0000 7f601545f740 1 --2- 172.21.15.43:0/2805740277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b36b973ef0 0x55b36b975e20 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.679+0000 7f601545f740 1 --2- 172.21.15.43:0/2805740277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b36b973b10 0x55b36b98d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.679+0000 7f601545f740 1 -- 172.21.15.43:0/2805740277 >> 172.21.15.43:0/2805740277 conn(0x55b36b946770 msgr2=0x55b36b94b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:54.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.679+0000 7f601545f740 1 -- 172.21.15.43:0/2805740277 shutdown_connections 2024-04-25T08:59:54.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.679+0000 7f601545f740 1 -- 172.21.15.43:0/2805740277 wait complete. 2024-04-25T08:59:54.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.679+0000 7f601545f740 20 client.7368 trim_cache size 0 max 16384 2024-04-25T08:59:54.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.698+0000 7f601545f740 1 Processor -- start 2024-04-25T08:59:54.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.698+0000 7f601545f740 1 -- start start 2024-04-25T08:59:54.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.699+0000 7f601545f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b36b973b10 0x55b36b94b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.699+0000 7f601545f740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55b36b973ef0 0x55b36b94d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.699+0000 7f601545f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b36b9930e0 0x55b36b969940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.699+0000 7f601545f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600c1ffe30 con 0x55b36b973b10 2024-04-25T08:59:54.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.699+0000 7f601545f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600c1ff110 con 0x55b36b9930e0 2024-04-25T08:59:54.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.699+0000 7f601545f740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f600c148c30 con 0x55b36b973ef0 2024-04-25T08:59:54.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.699+0000 7f600bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b36b973b10 0x55b36b94b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.699+0000 7f600bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b36b973b10 0x55b36b94b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55332/0 (socket says 172.21.15.43:55332) 2024-04-25T08:59:54.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.699+0000 7f600bfff640 1 -- 172.21.15.43:0/1522414557 learned_addr learned my addr 172.21.15.43:0/1522414557 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:59:54.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.699+0000 7f600b7fe640 1 --2- 172.21.15.43:0/1522414557 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55b36b973ef0 0x55b36b94d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.699+0000 7f6010d07640 1 --2- 172.21.15.43:0/1522414557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b36b9930e0 0x55b36b969940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.699+0000 7f600bfff640 1 -- 172.21.15.43:0/1522414557 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55b36b973ef0 msgr2=0x55b36b94d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:54.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.699+0000 7f600bfff640 1 --2- 172.21.15.43:0/1522414557 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55b36b973ef0 0x55b36b94d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.699+0000 7f600bfff640 1 -- 172.21.15.43:0/1522414557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b36b9930e0 msgr2=0x55b36b969940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:54.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.699+0000 7f600bfff640 1 --2- 172.21.15.43:0/1522414557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b36b9930e0 0x55b36b969940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.699+0000 7f600bfff640 1 -- 172.21.15.43:0/1522414557 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b36b5840f0 con 0x55b36b973b10 2024-04-25T08:59:54.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.699+0000 7f600b7fe640 1 --2- 172.21.15.43:0/1522414557 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55b36b973ef0 0x55b36b94d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T08:59:54.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.699+0000 7f600bfff640 1 --2- 172.21.15.43:0/1522414557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b36b973b10 0x55b36b94b6f0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc087bb0 tx=0x7f5ffc05e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=739ee327ba2ce068 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:54.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.700+0000 7f5ff17fa640 1 -- 172.21.15.43:0/1522414557 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc0544d0 con 0x55b36b973b10 2024-04-25T08:59:54.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.700+0000 7f5ff17fa640 1 -- 172.21.15.43:0/1522414557 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ffc05b070 con 0x55b36b973b10 2024-04-25T08:59:54.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.700+0000 7f5ff17fa640 1 -- 172.21.15.43:0/1522414557 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc054670 con 0x55b36b973b10 2024-04-25T08:59:54.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.700+0000 7f601545f740 1 -- 172.21.15.43:0/1522414557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b36b973b10 msgr2=0x55b36b94b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:54.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.700+0000 7f601545f740 1 --2- 172.21.15.43:0/1522414557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b36b973b10 0x55b36b94b6f0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc087bb0 tx=0x7f5ffc05e000 comp rx=0 tx=0).stop 2024-04-25T08:59:54.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.700+0000 7f601545f740 1 -- 172.21.15.43:0/1522414557 shutdown_connections 2024-04-25T08:59:54.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.700+0000 7f601545f740 1 --2- 172.21.15.43:0/1522414557 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b36b9930e0 0x55b36b969940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.700+0000 7f601545f740 1 --2- 172.21.15.43:0/1522414557 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55b36b973ef0 0x55b36b94d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.700+0000 7f601545f740 1 --2- 172.21.15.43:0/1522414557 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b36b973b10 0x55b36b94b6f0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.700+0000 7f601545f740 1 -- 172.21.15.43:0/1522414557 >> 172.21.15.43:0/1522414557 conn(0x55b36b946770 msgr2=0x55b36b994000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:54.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.701+0000 7f601545f740 1 -- 172.21.15.43:0/1522414557 shutdown_connections 2024-04-25T08:59:54.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.701+0000 7f601545f740 1 -- 172.21.15.43:0/1522414557 wait complete. 2024-04-25T08:59:54.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.701+0000 7f601545f740 1 -- start start 2024-04-25T08:59:54.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.701+0000 7f601545f740 1 Processor -- start 2024-04-25T08:59:54.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.702+0000 7f601545f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b36b973b10 0x55b36b972060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.702+0000 7f601545f740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55b36b973ef0 0x55b36b945a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.702+0000 7f601545f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b36b9930e0 0x55b36b9e1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.702+0000 7f601545f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b36b941f00 con 0x55b36b973b10 2024-04-25T08:59:54.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.702+0000 7f601545f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600c0ca430 con 0x55b36b9930e0 2024-04-25T08:59:54.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.702+0000 7f601545f740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b36b94db40 con 0x55b36b973ef0 2024-04-25T08:59:54.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.702+0000 7f601545f740 10 client.0 _ll_register_callbacks cb 0x7f6000035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T08:59:54.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.702+0000 7f600bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b36b973b10 0x55b36b972060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.702+0000 7f600bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b36b973b10 0x55b36b972060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55334/0 (socket says 172.21.15.43:55334) 2024-04-25T08:59:54.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.702+0000 7f600bfff640 1 -- 172.21.15.43:0/1217959373 learned_addr learned my addr 172.21.15.43:0/1217959373 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T08:59:54.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.702+0000 7f600b7fe640 1 --2- 172.21.15.43:0/1217959373 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55b36b973ef0 0x55b36b945a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.703+0000 7f6010d07640 1 --2- 172.21.15.43:0/1217959373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b36b9930e0 0x55b36b9e1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.703+0000 7f600bfff640 1 -- 172.21.15.43:0/1217959373 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55b36b973ef0 msgr2=0x55b36b945a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:54.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.703+0000 7f600bfff640 1 --2- 172.21.15.43:0/1217959373 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55b36b973ef0 0x55b36b945a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.703+0000 7f600bfff640 1 -- 172.21.15.43:0/1217959373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b36b9930e0 msgr2=0x55b36b9e1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T08:59:54.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.703+0000 7f600bfff640 1 --2- 172.21.15.43:0/1217959373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b36b9930e0 0x55b36b9e1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.703+0000 7f600bfff640 1 -- 172.21.15.43:0/1217959373 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b36b945f40 con 0x55b36b973b10 2024-04-25T08:59:54.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.703+0000 7f600bfff640 1 --2- 172.21.15.43:0/1217959373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b36b973b10 0x55b36b972060 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc00cdb0 tx=0x7f5ffc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9824069069c83edf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T08:59:54.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.703+0000 7f5feaffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T08:59:54.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.703+0000 7f5feaffd640 1 -- 172.21.15.43:0/1217959373 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc0544d0 con 0x55b36b973b10 2024-04-25T08:59:54.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.703+0000 7f5feaffd640 1 -- 172.21.15.43:0/1217959373 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(75..75 src has 1..75) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f5ffc06fb40 con 0x55b36b973b10 2024-04-25T08:59:54.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.703+0000 7f601545f740 1 -- 172.21.15.43:0/1217959373 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ffc05b070 con 0x55b36b973b10 2024-04-25T08:59:54.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.703+0000 7f5feaffd640 1 -- 172.21.15.43:0/1217959373 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ffc054670 con 0x55b36b973b10 2024-04-25T08:59:54.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.703+0000 7f601545f740 10 client.7374 fetch_fsmap learned FSMap version 48 2024-04-25T08:59:54.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.703+0000 7f601545f740 1 -- 172.21.15.43:0/1217959373 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x55b36b969fa0 con 0x55b36b973b10 2024-04-25T08:59:54.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.703+0000 7f5feaffd640 1 -- 172.21.15.43:0/1217959373 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ffc05b070 con 0x55b36b973b10 2024-04-25T08:59:54.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.704+0000 7f5feaffd640 1 -- 172.21.15.43:0/1217959373 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 48) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5ffc0572b0 con 0x55b36b973b10 2024-04-25T08:59:54.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.704+0000 7f601545f740 10 client.7374 fetch_fsmap finished waiting for FSMap version 48 2024-04-25T08:59:54.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.704+0000 7f601545f740 10 client.7374 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T08:59:54.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.704+0000 7f601545f740 20 client.7374 populate_metadata read hostname 'smithi043' 2024-04-25T08:59:54.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.704+0000 7f601545f740 1 --2- 172.21.15.43:0/1217959373 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x55b36b9f5480 0x55b36b9e19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T08:59:54.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.704+0000 7f601545f740 4 client.7374 mds_command: new command op to 7164 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T08:59:54.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.704+0000 7f601545f740 1 -- 172.21.15.43:0/1217959373 --> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b36b8bd2e0 con 0x55b36b9f5480 2024-04-25T08:59:54.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.704+0000 7f600b7fe640 1 --2- 172.21.15.43:0/1217959373 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x55b36b9f5480 0x55b36b9e19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T08:59:54.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.705+0000 7f600b7fe640 1 --2- 172.21.15.43:0/1217959373 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x55b36b9f5480 0x55b36b9e19c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e58f74fbc45840ce server_cookie=bcebfea8df8ae6d4 in_seq=0 out_seq=0 2024-04-25T08:59:54.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.705+0000 7f5feaffd640 10 client.7374 ms_handle_connect on v2:172.21.15.193:6835/3458098162 2024-04-25T08:59:54.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.705+0000 7f5feaffd640 1 -- 172.21.15.43:0/1217959373 <== mds.0 v2:172.21.15.193:6835/3458098162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55b36b8bd2e0 con 0x55b36b9f5480 2024-04-25T08:59:54.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.705+0000 7f5feaffd640 10 client.7374 handle_command_reply: tid=0 2024-04-25T08:59:54.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.705+0000 7f601545f740 1 client.7374 shutdown 2024-04-25T08:59:54.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.705+0000 7f601545f740 1 -- 172.21.15.43:0/1217959373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b36b973b10 msgr2=0x55b36b972060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:54.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.705+0000 7f601545f740 1 --2- 172.21.15.43:0/1217959373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b36b973b10 0x55b36b972060 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc00cdb0 tx=0x7f5ffc000bc0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.705+0000 7f601545f740 1 -- 172.21.15.43:0/1217959373 shutdown_connections 2024-04-25T08:59:54.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.705+0000 7f601545f740 1 --2- 172.21.15.43:0/1217959373 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x55b36b9f5480 0x55b36b9e19c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.705+0000 7f601545f740 1 --2- 172.21.15.43:0/1217959373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55b36b9930e0 0x55b36b9e1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.705+0000 7f601545f740 1 --2- 172.21.15.43:0/1217959373 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55b36b973ef0 0x55b36b945a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.705+0000 7f5feaffd640 0 client.7374 ms_handle_reset on v2:172.21.15.193:6835/3458098162 2024-04-25T08:59:54.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.705+0000 7f601545f740 1 --2- 172.21.15.43:0/1217959373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55b36b973b10 0x55b36b972060 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.705+0000 7f601545f740 1 -- 172.21.15.43:0/1217959373 >> 172.21.15.43:0/1217959373 conn(0x55b36b946770 msgr2=0x55b36b94b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:54.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.705+0000 7f601545f740 1 -- 172.21.15.43:0/1217959373 shutdown_connections 2024-04-25T08:59:54.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.705+0000 7f601545f740 1 -- 172.21.15.43:0/1217959373 wait complete. 2024-04-25T08:59:54.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.705+0000 7f601545f740 20 client.7374 trim_cache size 0 max 16384 2024-04-25T08:59:54.700 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T08:59:54.700 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "scrub active (200 inodes in the stack)", 2024-04-25T08:59:54.700 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-25T08:59:54.700 INFO:teuthology.orchestra.run.smithi043.stdout: "7e31711a-53ad-4f74-8918-a0c2a6362edf": { 2024-04-25T08:59:54.700 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "~mds0", 2024-04-25T08:59:54.700 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "7e31711a-53ad-4f74-8918-a0c2a6362edf", 2024-04-25T08:59:54.700 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive" 2024-04-25T08:59:54.700 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:59:54.700 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T08:59:54.700 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T08:59:54.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.706+0000 7f601276a640 1 -- 172.21.15.43:0/2800573311 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5ff806b4d0 msgr2=0x7f5ff806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:54.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.706+0000 7f601276a640 1 --2- 172.21.15.43:0/2800573311 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5ff806b4d0 0x7f5ff806d990 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f6000034000 tx=0x7f600002e000 comp rx=0 tx=0).stop 2024-04-25T08:59:54.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.706+0000 7f601276a640 1 -- 172.21.15.43:0/2800573311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f600c0b5560 msgr2=0x7f600c1f4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T08:59:54.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.706+0000 7f601276a640 1 --2- 172.21.15.43:0/2800573311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f600c0b5560 0x7f600c1f4ad0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc00c3f0 tx=0x7f5ffc00c8b0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.706+0000 7f601276a640 1 -- 172.21.15.43:0/2800573311 shutdown_connections 2024-04-25T08:59:54.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.706+0000 7f601276a640 1 --2- 172.21.15.43:0/2800573311 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5ff806b4d0 0x7f5ff806d990 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.706+0000 7f601276a640 1 --2- 172.21.15.43:0/2800573311 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f600c15e090 0x7f600c1f75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.706+0000 7f601276a640 1 --2- 172.21.15.43:0/2800573311 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f600c0d2230 0x7f600c160200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.706+0000 7f601276a640 1 --2- 172.21.15.43:0/2800573311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f600c0b5560 0x7f600c1f4ad0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T08:59:54.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.706+0000 7f601276a640 1 -- 172.21.15.43:0/2800573311 >> 172.21.15.43:0/2800573311 conn(0x7f600c0d0560 msgr2=0x7f600c0d2610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T08:59:54.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.706+0000 7f601276a640 1 -- 172.21.15.43:0/2800573311 shutdown_connections 2024-04-25T08:59:54.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T08:59:54.706+0000 7f601276a640 1 -- 172.21.15.43:0/2800573311 wait complete. 2024-04-25T08:59:54.708 INFO:tasks.cephfs.filesystem:scrub status for tag:7e31711a-53ad-4f74-8918-a0c2a6362edf - {'path': '~mds0', 'tag': '7e31711a-53ad-4f74-8918-a0c2a6362edf', 'options': 'recursive'} 2024-04-25T09:00:15.412 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:00:15.415 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:00:24.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:24.709 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T09:00:24.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.836+0000 7f21ddfa5640 1 Processor -- start 2024-04-25T09:00:24.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.836+0000 7f21ddfa5640 1 -- start start 2024-04-25T09:00:24.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.837+0000 7f21ddfa5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f21d80b40a0 0x7f21d80ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:24.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.837+0000 7f21ddfa5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d80af0a0 con 0x7f21d80aec70 2024-04-25T09:00:24.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.837+0000 7f21ddfa5640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d815b110 con 0x7f21d815a960 2024-04-25T09:00:24.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.837+0000 7f21ddfa5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d815b3f0 con 0x7f21d80b40a0 2024-04-25T09:00:24.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.837+0000 7f21d6ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f21d80b40a0 0x7f21d80ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:24.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.837+0000 7f21d6ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f21d80b40a0 0x7f21d80ae730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39948/0 (socket says 172.21.15.43:39948) 2024-04-25T09:00:24.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.837+0000 7f21d6ffd640 1 -- 172.21.15.43:0/414273843 learned_addr learned my addr 172.21.15.43:0/414273843 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:24.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.837+0000 7f21d7fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f21d80aec70 0x7f21d8070330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32948/0 (socket says 172.21.15.43:32948) 2024-04-25T09:00:24.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.837+0000 7f21d6ffd640 1 -- 172.21.15.43:0/414273843 >> v1:172.21.15.193:6789/0 conn(0x7f21d815a960 legacy=0x7f21d80b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:24.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.837+0000 7f21d6ffd640 1 -- 172.21.15.43:0/414273843 >> v1:172.21.15.43:6789/0 conn(0x7f21d80aec70 legacy=0x7f21d8070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:24.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.837+0000 7f21d6ffd640 1 -- 172.21.15.43:0/414273843 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d815b6d0 con 0x7f21d80b40a0 2024-04-25T09:00:24.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.838+0000 7f21d6ffd640 1 --2- 172.21.15.43:0/414273843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21d80b40a0 0x7f21d80ae730 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f21cc009870 tx=0x7f21cc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c93bb9adbc3057 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:24.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.838+0000 7f21d67fc640 1 -- 172.21.15.43:0/414273843 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc037070 con 0x7f21d80b40a0 2024-04-25T09:00:24.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.838+0000 7f21d67fc640 1 -- 172.21.15.43:0/414273843 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21cc032440 con 0x7f21d80b40a0 2024-04-25T09:00:24.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.838+0000 7f21d67fc640 1 -- 172.21.15.43:0/414273843 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc032810 con 0x7f21d80b40a0 2024-04-25T09:00:24.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.839+0000 7f21ddfa5640 1 -- 172.21.15.43:0/414273843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21d80b40a0 msgr2=0x7f21d80ae730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:24.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.839+0000 7f21ddfa5640 1 --2- 172.21.15.43:0/414273843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21d80b40a0 0x7f21d80ae730 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f21cc009870 tx=0x7f21cc02a210 comp rx=0 tx=0).stop 2024-04-25T09:00:24.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.839+0000 7f21ddfa5640 1 -- 172.21.15.43:0/414273843 shutdown_connections 2024-04-25T09:00:24.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.839+0000 7f21ddfa5640 1 --2- 172.21.15.43:0/414273843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21d80b40a0 0x7f21d80ae730 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.839+0000 7f21ddfa5640 1 -- 172.21.15.43:0/414273843 >> 172.21.15.43:0/414273843 conn(0x7f21d806ab30 msgr2=0x7f21d806af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:24.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.839+0000 7f21ddfa5640 1 -- 172.21.15.43:0/414273843 shutdown_connections 2024-04-25T09:00:24.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.839+0000 7f21ddfa5640 1 -- 172.21.15.43:0/414273843 wait complete. 2024-04-25T09:00:24.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.840+0000 7f21ddfa5640 1 Processor -- start 2024-04-25T09:00:24.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.841+0000 7f21ddfa5640 1 -- start start 2024-04-25T09:00:24.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.841+0000 7f21ddfa5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21d80aec70 0x7f21d8151120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:24.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.841+0000 7f21ddfa5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f21d80b40a0 0x7f21d806cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:24.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.841+0000 7f21ddfa5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21d815a960 0x7f21d80ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:24.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.841+0000 7f21ddfa5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d80cd940 con 0x7f21d815a960 2024-04-25T09:00:24.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.841+0000 7f21ddfa5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d80cc440 con 0x7f21d80aec70 2024-04-25T09:00:24.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.841+0000 7f21ddfa5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d815ad40 con 0x7f21d80b40a0 2024-04-25T09:00:24.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.841+0000 7f21d7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21d815a960 0x7f21d80ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:24.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.841+0000 7f21d7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21d815a960 0x7f21d80ad8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39956/0 (socket says 172.21.15.43:39956) 2024-04-25T09:00:24.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.841+0000 7f21d7fff640 1 -- 172.21.15.43:0/227158610 learned_addr learned my addr 172.21.15.43:0/227158610 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:24.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.842+0000 7f21d77fe640 1 --2- 172.21.15.43:0/227158610 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21d80aec70 0x7f21d8151120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:24.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.842+0000 7f21d6ffd640 1 --2- 172.21.15.43:0/227158610 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f21d80b40a0 0x7f21d806cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:24.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.842+0000 7f21d7fff640 1 -- 172.21.15.43:0/227158610 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f21d80b40a0 msgr2=0x7f21d806cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:24.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.842+0000 7f21d7fff640 1 --2- 172.21.15.43:0/227158610 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f21d80b40a0 0x7f21d806cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.842+0000 7f21d7fff640 1 -- 172.21.15.43:0/227158610 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21d80aec70 msgr2=0x7f21d8151120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:24.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.842+0000 7f21d7fff640 1 --2- 172.21.15.43:0/227158610 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21d80aec70 0x7f21d8151120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.842+0000 7f21d7fff640 1 -- 172.21.15.43:0/227158610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d81520b0 con 0x7f21d815a960 2024-04-25T09:00:24.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.842+0000 7f21d7fff640 1 --2- 172.21.15.43:0/227158610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21d815a960 0x7f21d80ad8e0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f21d000c780 tx=0x7f21d000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:24.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.842+0000 7f21bffff640 1 -- 172.21.15.43:0/227158610 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d0017070 con 0x7f21d815a960 2024-04-25T09:00:24.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.842+0000 7f21bffff640 1 -- 172.21.15.43:0/227158610 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21d000a5e0 con 0x7f21d815a960 2024-04-25T09:00:24.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.842+0000 7f21bffff640 1 -- 172.21.15.43:0/227158610 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d000a9d0 con 0x7f21d815a960 2024-04-25T09:00:24.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.842+0000 7f21ddfa5640 1 -- 172.21.15.43:0/227158610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21d8152390 con 0x7f21d815a960 2024-04-25T09:00:24.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.842+0000 7f21ddfa5640 1 -- 172.21.15.43:0/227158610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21d8152950 con 0x7f21d815a960 2024-04-25T09:00:24.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.844+0000 7f21bffff640 1 -- 172.21.15.43:0/227158610 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f21d001a070 con 0x7f21d815a960 2024-04-25T09:00:24.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.845+0000 7f21bffff640 1 --2- 172.21.15.43:0/227158610 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f21c406b4d0 0x7f21c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:24.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.845+0000 7f21bffff640 1 -- 172.21.15.43:0/227158610 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f21d0089680 con 0x7f21d815a960 2024-04-25T09:00:24.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.845+0000 7f21d77fe640 1 --2- 172.21.15.43:0/227158610 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f21c406b4d0 0x7f21c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:24.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.845+0000 7f21e0d87740 1 Processor -- start 2024-04-25T09:00:24.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.846+0000 7f21e0d87740 1 -- start start 2024-04-25T09:00:24.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.846+0000 7f21d77fe640 1 --2- 172.21.15.43:0/227158610 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f21c406b4d0 0x7f21c406d990 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f21c8006fd0 tx=0x7f21c8009060 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:24.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.846+0000 7f21e0d87740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a4cdfe1b10 0x55a4cdfe4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:24.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.846+0000 7f21e0d87740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4cdfe1ef0 0x55a4cdfe4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:24.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.846+0000 7f21e0d87740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4cdfb4770 0x55a4cdfb7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:24.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.846+0000 7f21e0d87740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4cdf1dda0 con 0x55a4cdfe1ef0 2024-04-25T09:00:24.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.846+0000 7f21e0d87740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4cdec6b80 con 0x55a4cdfb4770 2024-04-25T09:00:24.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.846+0000 7f21e0d87740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a4cdf19e40 con 0x55a4cdfe1b10 2024-04-25T09:00:24.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.846+0000 7f21d6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4cdfe1ef0 0x55a4cdfe4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:24.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.847+0000 7f21d6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4cdfe1ef0 0x55a4cdfe4860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39964/0 (socket says 172.21.15.43:39964) 2024-04-25T09:00:24.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.847+0000 7f21d6ffd640 1 -- 172.21.15.43:0/529370877 learned_addr learned my addr 172.21.15.43:0/529370877 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:24.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.847+0000 7f21d7fff640 1 --2- 172.21.15.43:0/529370877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4cdfb4770 0x55a4cdfb7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:24.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.847+0000 7f21d77fe640 1 --2- 172.21.15.43:0/529370877 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a4cdfe1b10 0x55a4cdfe4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:24.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.847+0000 7f21d6ffd640 1 -- 172.21.15.43:0/529370877 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a4cdfe1b10 msgr2=0x55a4cdfe4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:24.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.847+0000 7f21d6ffd640 1 --2- 172.21.15.43:0/529370877 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a4cdfe1b10 0x55a4cdfe4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.847+0000 7f21d6ffd640 1 -- 172.21.15.43:0/529370877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4cdfb4770 msgr2=0x55a4cdfb7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:24.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.847+0000 7f21d6ffd640 1 --2- 172.21.15.43:0/529370877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4cdfb4770 0x55a4cdfb7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.847+0000 7f21d6ffd640 1 -- 172.21.15.43:0/529370877 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4cded0f10 con 0x55a4cdfe1ef0 2024-04-25T09:00:24.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.847+0000 7f21d77fe640 1 --2- 172.21.15.43:0/529370877 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a4cdfe1b10 0x55a4cdfe4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:00:24.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.847+0000 7f21d6ffd640 1 --2- 172.21.15.43:0/529370877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4cdfe1ef0 0x55a4cdfe4860 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0090b0 tx=0x7f21cc02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d63f46ce4fa375f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:24.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.847+0000 7f21bd7fa640 1 -- 172.21.15.43:0/529370877 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc037040 con 0x55a4cdfe1ef0 2024-04-25T09:00:24.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.847+0000 7f21bd7fa640 1 -- 172.21.15.43:0/529370877 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21cc032800 con 0x55a4cdfe1ef0 2024-04-25T09:00:24.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.847+0000 7f21bd7fa640 1 -- 172.21.15.43:0/529370877 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc032bb0 con 0x55a4cdfe1ef0 2024-04-25T09:00:24.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.848+0000 7f21e0d87740 1 -- 172.21.15.43:0/529370877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4cdfe1ef0 msgr2=0x55a4cdfe4860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:24.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.848+0000 7f21e0d87740 1 --2- 172.21.15.43:0/529370877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4cdfe1ef0 0x55a4cdfe4860 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0090b0 tx=0x7f21cc02c000 comp rx=0 tx=0).stop 2024-04-25T09:00:24.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.848+0000 7f21e0d87740 1 -- 172.21.15.43:0/529370877 shutdown_connections 2024-04-25T09:00:24.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.848+0000 7f21e0d87740 1 --2- 172.21.15.43:0/529370877 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4cdfb4770 0x55a4cdfb7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.848+0000 7f21e0d87740 1 --2- 172.21.15.43:0/529370877 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4cdfe1ef0 0x55a4cdfe4860 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.848+0000 7f21e0d87740 1 --2- 172.21.15.43:0/529370877 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a4cdfe1b10 0x55a4cdfe4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.848+0000 7f21e0d87740 1 -- 172.21.15.43:0/529370877 >> 172.21.15.43:0/529370877 conn(0x55a4cdfb69e0 msgr2=0x55a4cdfff320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:24.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.848+0000 7f21e0d87740 1 -- 172.21.15.43:0/529370877 shutdown_connections 2024-04-25T09:00:24.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.848+0000 7f21e0d87740 1 -- 172.21.15.43:0/529370877 wait complete. 2024-04-25T09:00:24.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.848+0000 7f21e0d87740 1 -- start start 2024-04-25T09:00:24.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.849+0000 7f21e0d87740 1 Processor -- start 2024-04-25T09:00:24.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.849+0000 7f21e0d87740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4cdfe1b10 0x55a4ce002ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:24.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.849+0000 7f21e0d87740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a4cdfe1ef0 0x55a4cdfd9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:24.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.849+0000 7f21e0d87740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4cdfdad40 0x55a4cdfda0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:24.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.849+0000 7f21e0d87740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4cdfb9b90 con 0x55a4cdfdad40 2024-04-25T09:00:24.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.849+0000 7f21e0d87740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4cdebc4b0 con 0x55a4cdfe1b10 2024-04-25T09:00:24.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.849+0000 7f21e0d87740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a4cdfb9e70 con 0x55a4cdfe1ef0 2024-04-25T09:00:24.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.849+0000 7f21e0d87740 10 client.0 _ll_register_callbacks cb 0x55a4cdefc180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:00:24.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.849+0000 7f21d7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4cdfdad40 0x55a4cdfda0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:24.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.849+0000 7f21d7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4cdfdad40 0x55a4cdfda0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39968/0 (socket says 172.21.15.43:39968) 2024-04-25T09:00:24.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.850+0000 7f21d7fff640 1 -- 172.21.15.43:0/3312752025 learned_addr learned my addr 172.21.15.43:0/3312752025 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:24.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.850+0000 7f21d6ffd640 1 --2- 172.21.15.43:0/3312752025 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a4cdfe1ef0 0x55a4cdfd9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:24.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.850+0000 7f21d77fe640 1 --2- 172.21.15.43:0/3312752025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4cdfe1b10 0x55a4ce002ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:24.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.850+0000 7f21d7fff640 1 -- 172.21.15.43:0/3312752025 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a4cdfe1ef0 msgr2=0x55a4cdfd9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:24.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.850+0000 7f21d7fff640 1 --2- 172.21.15.43:0/3312752025 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a4cdfe1ef0 0x55a4cdfd9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.850+0000 7f21d7fff640 1 -- 172.21.15.43:0/3312752025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4cdfe1b10 msgr2=0x55a4ce002ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:24.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.850+0000 7f21d7fff640 1 --2- 172.21.15.43:0/3312752025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4cdfe1b10 0x55a4ce002ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.850+0000 7f21d7fff640 1 -- 172.21.15.43:0/3312752025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a4cdcfc7d0 con 0x55a4cdfdad40 2024-04-25T09:00:24.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.850+0000 7f21d77fe640 1 --2- 172.21.15.43:0/3312752025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4cdfe1b10 0x55a4ce002ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:00:24.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.850+0000 7f21d7fff640 1 --2- 172.21.15.43:0/3312752025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4cdfdad40 0x55a4cdfda0a0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f21d0087be0 tx=0x7f21d00578b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9719c22f5bd2d6c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:24.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.850+0000 7f21b67fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:00:24.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.850+0000 7f21b67fc640 1 -- 172.21.15.43:0/3312752025 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d0060540 con 0x55a4cdfdad40 2024-04-25T09:00:24.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.850+0000 7f21b67fc640 1 -- 172.21.15.43:0/3312752025 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f21d00561e0 con 0x55a4cdfdad40 2024-04-25T09:00:24.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.850+0000 7f21e0d87740 1 -- 172.21.15.43:0/3312752025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f21cc032800 con 0x55a4cdfdad40 2024-04-25T09:00:24.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.850+0000 7f21b67fc640 1 -- 172.21.15.43:0/3312752025 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f21d005e0b0 con 0x55a4cdfdad40 2024-04-25T09:00:24.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.850+0000 7f21e0d87740 10 client.7389 fetch_fsmap learned FSMap version 48 2024-04-25T09:00:24.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.850+0000 7f21e0d87740 1 -- 172.21.15.43:0/3312752025 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x55a4cdc323f0 con 0x55a4cdfdad40 2024-04-25T09:00:24.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.851+0000 7f21b67fc640 1 -- 172.21.15.43:0/3312752025 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21cc032800 con 0x55a4cdfdad40 2024-04-25T09:00:24.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.851+0000 7f21b67fc640 1 -- 172.21.15.43:0/3312752025 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 48) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f21d00600b0 con 0x55a4cdfdad40 2024-04-25T09:00:24.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.851+0000 7f21e0d87740 10 client.7389 fetch_fsmap finished waiting for FSMap version 48 2024-04-25T09:00:24.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.851+0000 7f21e0d87740 10 client.7389 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T09:00:24.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.851+0000 7f21e0d87740 20 client.7389 populate_metadata read hostname 'smithi043' 2024-04-25T09:00:24.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.851+0000 7f21e0d87740 1 --2- 172.21.15.43:0/3312752025 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x55a4ce0014b0 0x55a4ce0018d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:24.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.851+0000 7f21e0d87740 4 client.7389 mds_command: new command op to 7164 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:00:24.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.851+0000 7f21e0d87740 1 -- 172.21.15.43:0/3312752025 --> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a4cdf2b2e0 con 0x55a4ce0014b0 2024-04-25T09:00:24.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.851+0000 7f21d77fe640 1 --2- 172.21.15.43:0/3312752025 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x55a4ce0014b0 0x55a4ce0018d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:24.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.851+0000 7f21d77fe640 1 --2- 172.21.15.43:0/3312752025 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x55a4ce0014b0 0x55a4ce0018d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=621ce336249ffc7e server_cookie=af92342610f54644 in_seq=0 out_seq=0 2024-04-25T09:00:24.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.851+0000 7f21b67fc640 10 client.7389 ms_handle_connect on v2:172.21.15.193:6835/3458098162 2024-04-25T09:00:24.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.852+0000 7f21b67fc640 1 -- 172.21.15.43:0/3312752025 <== mds.0 v2:172.21.15.193:6835/3458098162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a4cdf2b2e0 con 0x55a4ce0014b0 2024-04-25T09:00:24.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.852+0000 7f21b67fc640 10 client.7389 handle_command_reply: tid=0 2024-04-25T09:00:24.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.852+0000 7f21e0d87740 1 client.7389 shutdown 2024-04-25T09:00:24.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.852+0000 7f21e0d87740 1 -- 172.21.15.43:0/3312752025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4cdfdad40 msgr2=0x55a4cdfda0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:24.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.852+0000 7f21e0d87740 1 --2- 172.21.15.43:0/3312752025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4cdfdad40 0x55a4cdfda0a0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f21d0087be0 tx=0x7f21d00578b0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.852+0000 7f21e0d87740 1 -- 172.21.15.43:0/3312752025 shutdown_connections 2024-04-25T09:00:24.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.852+0000 7f21e0d87740 1 --2- 172.21.15.43:0/3312752025 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x55a4ce0014b0 0x55a4ce0018d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.852+0000 7f21e0d87740 1 --2- 172.21.15.43:0/3312752025 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4cdfdad40 0x55a4cdfda0a0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.852+0000 7f21e0d87740 1 --2- 172.21.15.43:0/3312752025 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a4cdfe1ef0 0x55a4cdfd9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.852+0000 7f21e0d87740 1 --2- 172.21.15.43:0/3312752025 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4cdfe1b10 0x55a4ce002ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.852+0000 7f21b67fc640 0 client.7389 ms_handle_reset on v2:172.21.15.193:6835/3458098162 2024-04-25T09:00:24.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.852+0000 7f21e0d87740 1 -- 172.21.15.43:0/3312752025 >> 172.21.15.43:0/3312752025 conn(0x55a4cdfb4770 msgr2=0x55a4cdfb5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:24.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.852+0000 7f21e0d87740 1 -- 172.21.15.43:0/3312752025 shutdown_connections 2024-04-25T09:00:24.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.852+0000 7f21e0d87740 1 -- 172.21.15.43:0/3312752025 wait complete. 2024-04-25T09:00:24.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.853+0000 7f21e0d87740 20 client.7389 trim_cache size 0 max 16384 2024-04-25T09:00:24.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.868+0000 7f21e0d87740 1 Processor -- start 2024-04-25T09:00:24.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.868+0000 7f21e0d87740 1 -- start start 2024-04-25T09:00:24.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.869+0000 7f21e0d87740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4cdfdad40 0x55a4ce002c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:24.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.869+0000 7f21e0d87740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4cdfe1b10 0x55a4cdfb4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:24.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.869+0000 7f21e0d87740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a4cdfe1ef0 0x55a4cdfb5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:24.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.869+0000 7f21e0d87740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d81fffb0 con 0x55a4cdfe1b10 2024-04-25T09:00:24.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.869+0000 7f21e0d87740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d81ff150 con 0x55a4cdfdad40 2024-04-25T09:00:24.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.869+0000 7f21e0d87740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d81f79d0 con 0x55a4cdfe1ef0 2024-04-25T09:00:24.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.869+0000 7f21d6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4cdfe1b10 0x55a4cdfb4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:24.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.869+0000 7f21d6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4cdfe1b10 0x55a4cdfb4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39978/0 (socket says 172.21.15.43:39978) 2024-04-25T09:00:24.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.869+0000 7f21d6ffd640 1 -- 172.21.15.43:0/3134790106 learned_addr learned my addr 172.21.15.43:0/3134790106 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:24.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.869+0000 7f21d7fff640 1 --2- 172.21.15.43:0/3134790106 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a4cdfe1ef0 0x55a4cdfb5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:24.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.869+0000 7f21d77fe640 1 --2- 172.21.15.43:0/3134790106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4cdfdad40 0x55a4ce002c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:24.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.869+0000 7f21d6ffd640 1 -- 172.21.15.43:0/3134790106 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a4cdfe1ef0 msgr2=0x55a4cdfb5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:24.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.869+0000 7f21d6ffd640 1 --2- 172.21.15.43:0/3134790106 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a4cdfe1ef0 0x55a4cdfb5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.869+0000 7f21d6ffd640 1 -- 172.21.15.43:0/3134790106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4cdfdad40 msgr2=0x55a4ce002c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:24.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.869+0000 7f21d6ffd640 1 --2- 172.21.15.43:0/3134790106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4cdfdad40 0x55a4ce002c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.869+0000 7f21d6ffd640 1 -- 172.21.15.43:0/3134790106 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a4cdfd8df0 con 0x55a4cdfe1b10 2024-04-25T09:00:24.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.869+0000 7f21d6ffd640 1 --2- 172.21.15.43:0/3134790106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4cdfe1b10 0x55a4cdfb4cc0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f21cc009990 tx=0x7f21cc008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb0d53f2e3d26a4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:24.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.869+0000 7f21bcff9640 1 -- 172.21.15.43:0/3134790106 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc037040 con 0x55a4cdfe1b10 2024-04-25T09:00:24.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.870+0000 7f21bcff9640 1 -- 172.21.15.43:0/3134790106 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21cc004400 con 0x55a4cdfe1b10 2024-04-25T09:00:24.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.870+0000 7f21bcff9640 1 -- 172.21.15.43:0/3134790106 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc003850 con 0x55a4cdfe1b10 2024-04-25T09:00:24.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.870+0000 7f21e0d87740 1 -- 172.21.15.43:0/3134790106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4cdfe1b10 msgr2=0x55a4cdfb4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:24.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.870+0000 7f21e0d87740 1 --2- 172.21.15.43:0/3134790106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4cdfe1b10 0x55a4cdfb4cc0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f21cc009990 tx=0x7f21cc008030 comp rx=0 tx=0).stop 2024-04-25T09:00:24.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.870+0000 7f21e0d87740 1 -- 172.21.15.43:0/3134790106 shutdown_connections 2024-04-25T09:00:24.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.870+0000 7f21e0d87740 1 --2- 172.21.15.43:0/3134790106 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a4cdfe1ef0 0x55a4cdfb5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.870+0000 7f21e0d87740 1 --2- 172.21.15.43:0/3134790106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4cdfe1b10 0x55a4cdfb4cc0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.870+0000 7f21e0d87740 1 --2- 172.21.15.43:0/3134790106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4cdfdad40 0x55a4ce002c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.870+0000 7f21e0d87740 1 -- 172.21.15.43:0/3134790106 >> 172.21.15.43:0/3134790106 conn(0x55a4cdfb4770 msgr2=0x55a4cdfdcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:24.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.870+0000 7f21e0d87740 1 -- 172.21.15.43:0/3134790106 shutdown_connections 2024-04-25T09:00:24.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.870+0000 7f21e0d87740 1 -- 172.21.15.43:0/3134790106 wait complete. 2024-04-25T09:00:24.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.871+0000 7f21e0d87740 1 -- start start 2024-04-25T09:00:24.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.871+0000 7f21e0d87740 1 Processor -- start 2024-04-25T09:00:24.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.871+0000 7f21e0d87740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4cdfdad40 0x55a4cdfb4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:24.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.872+0000 7f21e0d87740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4cdfe1b10 0x55a4cdfb5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:24.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.872+0000 7f21e0d87740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a4cdfe1ef0 0x55a4cdfb82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:24.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.872+0000 7f21e0d87740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4cdfb7580 con 0x55a4cdfe1b10 2024-04-25T09:00:24.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.872+0000 7f21e0d87740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d8153d90 con 0x55a4cdfdad40 2024-04-25T09:00:24.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.872+0000 7f21e0d87740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a4cdf2b2e0 con 0x55a4cdfe1ef0 2024-04-25T09:00:24.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.872+0000 7f21e0d87740 10 client.0 _ll_register_callbacks cb 0x7f21cc0096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:00:24.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.872+0000 7f21d6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4cdfe1b10 0x55a4cdfb5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:24.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.872+0000 7f21d6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4cdfe1b10 0x55a4cdfb5190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39982/0 (socket says 172.21.15.43:39982) 2024-04-25T09:00:24.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.872+0000 7f21d6ffd640 1 -- 172.21.15.43:0/1698909530 learned_addr learned my addr 172.21.15.43:0/1698909530 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:24.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.872+0000 7f21d77fe640 1 --2- 172.21.15.43:0/1698909530 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4cdfdad40 0x55a4cdfb4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:24.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.872+0000 7f21d7fff640 1 --2- 172.21.15.43:0/1698909530 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a4cdfe1ef0 0x55a4cdfb82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:24.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.872+0000 7f21d6ffd640 1 -- 172.21.15.43:0/1698909530 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a4cdfe1ef0 msgr2=0x55a4cdfb82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:24.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.872+0000 7f21d6ffd640 1 --2- 172.21.15.43:0/1698909530 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a4cdfe1ef0 0x55a4cdfb82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.872+0000 7f21d6ffd640 1 -- 172.21.15.43:0/1698909530 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4cdfdad40 msgr2=0x55a4cdfb4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:24.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.872+0000 7f21d6ffd640 1 --2- 172.21.15.43:0/1698909530 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4cdfdad40 0x55a4cdfb4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.872+0000 7f21d6ffd640 1 -- 172.21.15.43:0/1698909530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a4cdbf20f0 con 0x55a4cdfe1b10 2024-04-25T09:00:24.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.872+0000 7f21d7fff640 1 --2- 172.21.15.43:0/1698909530 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a4cdfe1ef0 0x55a4cdfb82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:24.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.872+0000 7f21d6ffd640 1 --2- 172.21.15.43:0/1698909530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4cdfe1b10 0x55a4cdfb5190 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f21cc009870 tx=0x7f21cc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a8ddaa5a3fbc88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:24.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.872+0000 7f21b67fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:00:24.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.872+0000 7f21b67fc640 1 -- 172.21.15.43:0/1698909530 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc037070 con 0x55a4cdfe1b10 2024-04-25T09:00:24.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.872+0000 7f21b67fc640 1 -- 172.21.15.43:0/1698909530 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f21cc04c040 con 0x55a4cdfe1b10 2024-04-25T09:00:24.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.872+0000 7f21e0d87740 1 -- 172.21.15.43:0/1698909530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f21cc004400 con 0x55a4cdfe1b10 2024-04-25T09:00:24.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.872+0000 7f21b67fc640 1 -- 172.21.15.43:0/1698909530 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f21cc032810 con 0x55a4cdfe1b10 2024-04-25T09:00:24.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.872+0000 7f21e0d87740 10 client.7395 fetch_fsmap learned FSMap version 48 2024-04-25T09:00:24.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.872+0000 7f21e0d87740 1 -- 172.21.15.43:0/1698909530 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x55a4ce06a960 con 0x55a4cdfe1b10 2024-04-25T09:00:24.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.873+0000 7f21b67fc640 1 -- 172.21.15.43:0/1698909530 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21cc004400 con 0x55a4cdfe1b10 2024-04-25T09:00:24.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.873+0000 7f21b67fc640 1 -- 172.21.15.43:0/1698909530 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 48) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f21cc0329b0 con 0x55a4cdfe1b10 2024-04-25T09:00:24.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.873+0000 7f21e0d87740 10 client.7395 fetch_fsmap finished waiting for FSMap version 48 2024-04-25T09:00:24.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.873+0000 7f21e0d87740 10 client.7395 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T09:00:24.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.873+0000 7f21e0d87740 20 client.7395 populate_metadata read hostname 'smithi043' 2024-04-25T09:00:24.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.873+0000 7f21e0d87740 1 --2- 172.21.15.43:0/1698909530 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x55a4cdfe13f0 0x55a4cdfb8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:24.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.873+0000 7f21e0d87740 4 client.7395 mds_command: new command op to 7164 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:00:24.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.873+0000 7f21e0d87740 1 -- 172.21.15.43:0/1698909530 --> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a4cdf25630 con 0x55a4cdfe13f0 2024-04-25T09:00:24.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.873+0000 7f21d77fe640 1 --2- 172.21.15.43:0/1698909530 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x55a4cdfe13f0 0x55a4cdfb8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:24.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.873+0000 7f21b67fc640 10 client.7395 ms_handle_connect on v2:172.21.15.193:6835/3458098162 2024-04-25T09:00:24.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.873+0000 7f21d77fe640 1 --2- 172.21.15.43:0/1698909530 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x55a4cdfe13f0 0x55a4cdfb8810 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faac8511bdfa8078 server_cookie=ad1c60b30fb24b80 in_seq=0 out_seq=0 2024-04-25T09:00:24.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.874+0000 7f21b67fc640 1 -- 172.21.15.43:0/1698909530 <== mds.0 v2:172.21.15.193:6835/3458098162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a4cdf25630 con 0x55a4cdfe13f0 2024-04-25T09:00:24.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.874+0000 7f21b67fc640 10 client.7395 handle_command_reply: tid=0 2024-04-25T09:00:24.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.874+0000 7f21e0d87740 1 client.7395 shutdown 2024-04-25T09:00:24.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.874+0000 7f21e0d87740 1 -- 172.21.15.43:0/1698909530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4cdfe1b10 msgr2=0x55a4cdfb5190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:24.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.874+0000 7f21e0d87740 1 --2- 172.21.15.43:0/1698909530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4cdfe1b10 0x55a4cdfb5190 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f21cc009870 tx=0x7f21cc004730 comp rx=0 tx=0).stop 2024-04-25T09:00:24.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.874+0000 7f21e0d87740 1 -- 172.21.15.43:0/1698909530 shutdown_connections 2024-04-25T09:00:24.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.874+0000 7f21e0d87740 1 --2- 172.21.15.43:0/1698909530 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x55a4cdfe13f0 0x55a4cdfb8810 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.874+0000 7f21e0d87740 1 --2- 172.21.15.43:0/1698909530 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a4cdfe1ef0 0x55a4cdfb82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.874+0000 7f21e0d87740 1 --2- 172.21.15.43:0/1698909530 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a4cdfe1b10 0x55a4cdfb5190 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.874+0000 7f21b67fc640 0 client.7395 ms_handle_reset on v2:172.21.15.193:6835/3458098162 2024-04-25T09:00:24.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.874+0000 7f21e0d87740 1 --2- 172.21.15.43:0/1698909530 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a4cdfdad40 0x55a4cdfb4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.874+0000 7f21e0d87740 1 -- 172.21.15.43:0/1698909530 >> 172.21.15.43:0/1698909530 conn(0x55a4cdfb4770 msgr2=0x55a4ce002ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:24.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.874+0000 7f21e0d87740 1 -- 172.21.15.43:0/1698909530 shutdown_connections 2024-04-25T09:00:24.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.874+0000 7f21e0d87740 1 -- 172.21.15.43:0/1698909530 wait complete. 2024-04-25T09:00:24.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.874+0000 7f21e0d87740 20 client.7395 trim_cache size 0 max 16384 2024-04-25T09:00:24.884 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:00:24.884 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-25T09:00:24.885 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-25T09:00:24.885 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:00:24.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.874+0000 7f21ddfa5640 1 -- 172.21.15.43:0/227158610 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f21c406b4d0 msgr2=0x7f21c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:24.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.874+0000 7f21ddfa5640 1 --2- 172.21.15.43:0/227158610 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f21c406b4d0 0x7f21c406d990 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f21c8006fd0 tx=0x7f21c8009060 comp rx=0 tx=0).stop 2024-04-25T09:00:24.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.874+0000 7f21ddfa5640 1 -- 172.21.15.43:0/227158610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21d815a960 msgr2=0x7f21d80ad8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:24.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.874+0000 7f21ddfa5640 1 --2- 172.21.15.43:0/227158610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21d815a960 0x7f21d80ad8e0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f21d000c780 tx=0x7f21d000cc40 comp rx=0 tx=0).stop 2024-04-25T09:00:24.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.874+0000 7f21ddfa5640 1 -- 172.21.15.43:0/227158610 shutdown_connections 2024-04-25T09:00:24.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.874+0000 7f21ddfa5640 1 --2- 172.21.15.43:0/227158610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21d815a960 0x7f21d80ad8e0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.874+0000 7f21ddfa5640 1 --2- 172.21.15.43:0/227158610 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f21d80b40a0 0x7f21d806cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.875+0000 7f21ddfa5640 1 --2- 172.21.15.43:0/227158610 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f21c406b4d0 0x7f21c406d990 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.875+0000 7f21ddfa5640 1 --2- 172.21.15.43:0/227158610 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f21d80aec70 0x7f21d8151120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.875+0000 7f21ddfa5640 1 -- 172.21.15.43:0/227158610 >> 172.21.15.43:0/227158610 conn(0x7f21d806ab30 msgr2=0x7f21d8070330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:24.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.875+0000 7f21ddfa5640 1 -- 172.21.15.43:0/227158610 shutdown_connections 2024-04-25T09:00:24.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:24.875+0000 7f21ddfa5640 1 -- 172.21.15.43:0/227158610 wait complete. 2024-04-25T09:00:24.891 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T09:00:24.891 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T09:00:24.891 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 perf dump 2024-04-25T09:00:24.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.005+0000 7fcae2e76640 1 Processor -- start 2024-04-25T09:00:24.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.005+0000 7fcae2e76640 1 -- start start 2024-04-25T09:00:24.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.006+0000 7fcae2e76640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fcadc06d7d0 0x7fcadc0ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:24.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.006+0000 7fcae2e76640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcadc0cd120 0x7fcadc15d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:24.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.006+0000 7fcae2e76640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcadc0cd570 con 0x7fcadc0aee80 2024-04-25T09:00:24.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.006+0000 7fcae2e76640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcadc0af260 con 0x7fcadc0cd120 2024-04-25T09:00:24.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.006+0000 7fcae2e76640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcadc15c280 con 0x7fcadc06d7d0 2024-04-25T09:00:24.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.006+0000 7fcae1413640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcadc0cd120 0x7fcadc15d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:24.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.006+0000 7fcae0c12640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fcadc0aee80 0x7fcadc06d0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32960/0 (socket says 172.21.15.43:32960) 2024-04-25T09:00:24.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.006+0000 7fcae0c12640 1 -- 172.21.15.43:0/3604927510 learned_addr learned my addr 172.21.15.43:0/3604927510 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:00:24.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.006+0000 7fcadbfff640 1 --2- 172.21.15.43:0/3604927510 >> v2:172.21.15.193:3300/0 conn(0x7fcadc06d7d0 0x7fcadc0ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:24.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.006+0000 7fcadb7fe640 1 -- 172.21.15.43:0/3604927510 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1495569746 0 0) 0x7fcadc0cd570 con 0x7fcadc0aee80 2024-04-25T09:00:24.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.006+0000 7fcadb7fe640 1 -- 172.21.15.43:0/3604927510 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcacc003520 con 0x7fcadc0aee80 2024-04-25T09:00:24.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.006+0000 7fcae1413640 1 -- 172.21.15.43:0/3604927510 >> v2:172.21.15.193:3300/0 conn(0x7fcadc06d7d0 msgr2=0x7fcadc0ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:24.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.006+0000 7fcae1413640 1 --2- 172.21.15.43:0/3604927510 >> v2:172.21.15.193:3300/0 conn(0x7fcadc06d7d0 0x7fcadc0ccbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:24.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.006+0000 7fcae1413640 1 -- 172.21.15.43:0/3604927510 >> v1:172.21.15.43:6789/0 conn(0x7fcadc0aee80 legacy=0x7fcadc06d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:24.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.006+0000 7fcae1413640 1 -- 172.21.15.43:0/3604927510 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcadc15c560 con 0x7fcadc0cd120 2024-04-25T09:00:24.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.007+0000 7fcae1413640 1 --2- 172.21.15.43:0/3604927510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcadc0cd120 0x7fcadc15d590 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fcadc072c20 tx=0x7fcad402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20b8db5a0daf1ae1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:24.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.007+0000 7fcadb7fe640 1 -- 172.21.15.43:0/3604927510 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad4007740 con 0x7fcadc0cd120 2024-04-25T09:00:24.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.007+0000 7fcadb7fe640 1 -- 172.21.15.43:0/3604927510 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad40078e0 con 0x7fcadc0cd120 2024-04-25T09:00:24.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.007+0000 7fcadb7fe640 1 -- 172.21.15.43:0/3604927510 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad4007c90 con 0x7fcadc0cd120 2024-04-25T09:00:25.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.008+0000 7fcae2e76640 1 -- 172.21.15.43:0/3604927510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcadc0cd120 msgr2=0x7fcadc15d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:25.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.008+0000 7fcae2e76640 1 --2- 172.21.15.43:0/3604927510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcadc0cd120 0x7fcadc15d590 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fcadc072c20 tx=0x7fcad402a680 comp rx=0 tx=0).stop 2024-04-25T09:00:25.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.008+0000 7fcae2e76640 1 -- 172.21.15.43:0/3604927510 shutdown_connections 2024-04-25T09:00:25.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.008+0000 7fcae2e76640 1 --2- 172.21.15.43:0/3604927510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcadc0cd120 0x7fcadc15d590 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.008+0000 7fcae2e76640 1 --2- 172.21.15.43:0/3604927510 >> v2:172.21.15.193:3300/0 conn(0x7fcadc06d7d0 0x7fcadc0ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.008+0000 7fcae2e76640 1 -- 172.21.15.43:0/3604927510 >> 172.21.15.43:0/3604927510 conn(0x7fcadc06ab30 msgr2=0x7fcadc06af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:25.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.008+0000 7fcae2e76640 1 -- 172.21.15.43:0/3604927510 shutdown_connections 2024-04-25T09:00:25.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.008+0000 7fcae2e76640 1 -- 172.21.15.43:0/3604927510 wait complete. 2024-04-25T09:00:25.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.009+0000 7fcae2e76640 1 Processor -- start 2024-04-25T09:00:25.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.009+0000 7fcae2e76640 1 -- start start 2024-04-25T09:00:25.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.010+0000 7fcae2e76640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcadc06d7d0 0x7fcadc144100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.010+0000 7fcae2e76640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcadc0aee80 0x7fcadc0a94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.010+0000 7fcae2e76640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcadc0cd120 0x7fcadc156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.010+0000 7fcae2e76640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc15dad0 con 0x7fcadc0cd120 2024-04-25T09:00:25.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.010+0000 7fcae2e76640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc15cb20 con 0x7fcadc06d7d0 2024-04-25T09:00:25.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.010+0000 7fcae2e76640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc15dc50 con 0x7fcadc0aee80 2024-04-25T09:00:25.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.010+0000 7fcae1413640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcadc0cd120 0x7fcadc156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.010+0000 7fcae1413640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcadc0cd120 0x7fcadc156c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39998/0 (socket says 172.21.15.43:39998) 2024-04-25T09:00:25.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.010+0000 7fcae1413640 1 -- 172.21.15.43:0/3679477912 learned_addr learned my addr 172.21.15.43:0/3679477912 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:25.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.010+0000 7fcae1413640 1 -- 172.21.15.43:0/3679477912 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcadc0aee80 msgr2=0x7fcadc0a94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:25.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.010+0000 7fcae1413640 1 --2- 172.21.15.43:0/3679477912 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcadc0aee80 0x7fcadc0a94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.010+0000 7fcae1413640 1 -- 172.21.15.43:0/3679477912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcadc06d7d0 msgr2=0x7fcadc144100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:25.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.010+0000 7fcae0c12640 1 --2- 172.21.15.43:0/3679477912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcadc06d7d0 0x7fcadc144100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.010+0000 7fcae1413640 1 --2- 172.21.15.43:0/3679477912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcadc06d7d0 0x7fcadc144100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.010+0000 7fcae1413640 1 -- 172.21.15.43:0/3679477912 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcadc151af0 con 0x7fcadc0cd120 2024-04-25T09:00:25.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.010+0000 7fcae1413640 1 --2- 172.21.15.43:0/3679477912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcadc0cd120 0x7fcadc156c10 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fcad402ac70 tx=0x7fcad4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:25.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.011+0000 7fcad97fa640 1 -- 172.21.15.43:0/3679477912 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad403f070 con 0x7fcadc0cd120 2024-04-25T09:00:25.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.011+0000 7fcad97fa640 1 -- 172.21.15.43:0/3679477912 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad4031c60 con 0x7fcadc0cd120 2024-04-25T09:00:25.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.011+0000 7fcae2e76640 1 -- 172.21.15.43:0/3679477912 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcadc1449e0 con 0x7fcadc0cd120 2024-04-25T09:00:25.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.011+0000 7fcad97fa640 1 -- 172.21.15.43:0/3679477912 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad4009450 con 0x7fcadc0cd120 2024-04-25T09:00:25.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.011+0000 7fcae2e76640 1 -- 172.21.15.43:0/3679477912 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcadc144fa0 con 0x7fcadc0cd120 2024-04-25T09:00:25.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.012+0000 7fcad97fa640 1 -- 172.21.15.43:0/3679477912 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcad403b410 con 0x7fcadc0cd120 2024-04-25T09:00:25.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.012+0000 7fcad97fa640 1 --2- 172.21.15.43:0/3679477912 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcac006b4d0 0x7fcac006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.013+0000 7fcad97fa640 1 -- 172.21.15.43:0/3679477912 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fcad40ab600 con 0x7fcadc0cd120 2024-04-25T09:00:25.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.013+0000 7fcae0c12640 1 --2- 172.21.15.43:0/3679477912 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcac006b4d0 0x7fcac006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.013+0000 7fcae5b7c740 1 Processor -- start 2024-04-25T09:00:25.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.013+0000 7fcae0c12640 1 --2- 172.21.15.43:0/3679477912 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcac006b4d0 0x7fcac006d990 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fcad0000f60 tx=0x7fcad0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:25.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.013+0000 7fcae5b7c740 1 -- start start 2024-04-25T09:00:25.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.014+0000 7fcae5b7c740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563b10422ca0 0x563b10424e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.014+0000 7fcae5b7c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b1045f370 0x563b10423080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.014+0000 7fcae5b7c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b1044d760 0x563b1044dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.014+0000 7fcae5b7c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b10385b90 con 0x563b1044d760 2024-04-25T09:00:25.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.014+0000 7fcae5b7c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b1032ee40 con 0x563b1045f370 2024-04-25T09:00:25.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.014+0000 7fcae5b7c740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b10381c30 con 0x563b10422ca0 2024-04-25T09:00:25.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.014+0000 7fcae1413640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b1044d760 0x563b1044dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.014+0000 7fcae1413640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b1044d760 0x563b1044dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40008/0 (socket says 172.21.15.43:40008) 2024-04-25T09:00:25.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.014+0000 7fcae1413640 1 -- 172.21.15.43:0/4288277808 learned_addr learned my addr 172.21.15.43:0/4288277808 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:25.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.014+0000 7fcae0c12640 1 --2- 172.21.15.43:0/4288277808 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563b10422ca0 0x563b10424e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.014+0000 7fcadbfff640 1 --2- 172.21.15.43:0/4288277808 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b1045f370 0x563b10423080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.015+0000 7fcae1413640 1 -- 172.21.15.43:0/4288277808 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563b10422ca0 msgr2=0x563b10424e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:25.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.015+0000 7fcae1413640 1 --2- 172.21.15.43:0/4288277808 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563b10422ca0 0x563b10424e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.015+0000 7fcae1413640 1 -- 172.21.15.43:0/4288277808 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b1045f370 msgr2=0x563b10423080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:25.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.015+0000 7fcae1413640 1 --2- 172.21.15.43:0/4288277808 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b1045f370 0x563b10423080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.015+0000 7fcae1413640 1 -- 172.21.15.43:0/4288277808 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b101647d0 con 0x563b1044d760 2024-04-25T09:00:25.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.015+0000 7fcae1413640 1 --2- 172.21.15.43:0/4288277808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b1044d760 0x563b1044dbe0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fcad40a9a40 tx=0x7fcad4079c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c15ed0c2ea3f368f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:25.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.015+0000 7fcac67fc640 1 -- 172.21.15.43:0/4288277808 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad4079d20 con 0x563b1044d760 2024-04-25T09:00:25.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.015+0000 7fcac67fc640 1 -- 172.21.15.43:0/4288277808 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad407de70 con 0x563b1044d760 2024-04-25T09:00:25.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.015+0000 7fcac67fc640 1 -- 172.21.15.43:0/4288277808 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad4082430 con 0x563b1044d760 2024-04-25T09:00:25.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.015+0000 7fcae5b7c740 1 -- 172.21.15.43:0/4288277808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b1044d760 msgr2=0x563b1044dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:25.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.015+0000 7fcae5b7c740 1 --2- 172.21.15.43:0/4288277808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b1044d760 0x563b1044dbe0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fcad40a9a40 tx=0x7fcad4079c20 comp rx=0 tx=0).stop 2024-04-25T09:00:25.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.015+0000 7fcae5b7c740 1 -- 172.21.15.43:0/4288277808 shutdown_connections 2024-04-25T09:00:25.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.015+0000 7fcae5b7c740 1 --2- 172.21.15.43:0/4288277808 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b1044d760 0x563b1044dbe0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.015+0000 7fcae5b7c740 1 --2- 172.21.15.43:0/4288277808 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b1045f370 0x563b10423080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.015+0000 7fcae5b7c740 1 --2- 172.21.15.43:0/4288277808 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563b10422ca0 0x563b10424e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.015+0000 7fcae5b7c740 1 -- 172.21.15.43:0/4288277808 >> 172.21.15.43:0/4288277808 conn(0x563b103ea870 msgr2=0x563b10468760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:25.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.016+0000 7fcae5b7c740 1 -- 172.21.15.43:0/4288277808 shutdown_connections 2024-04-25T09:00:25.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.016+0000 7fcae5b7c740 1 -- 172.21.15.43:0/4288277808 wait complete. 2024-04-25T09:00:25.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.016+0000 7fcae5b7c740 1 -- start start 2024-04-25T09:00:25.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.016+0000 7fcae5b7c740 1 Processor -- start 2024-04-25T09:00:25.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.017+0000 7fcae5b7c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b10422ca0 0x563b104421c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.017+0000 7fcae5b7c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b1044d760 0x563b10433230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.017+0000 7fcae5b7c740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563b1045f370 0x563b10433770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.017+0000 7fcae5b7c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b103423e0 con 0x563b10422ca0 2024-04-25T09:00:25.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.017+0000 7fcae5b7c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b1044b2b0 con 0x563b1044d760 2024-04-25T09:00:25.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.017+0000 7fcae5b7c740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b1044f050 con 0x563b1045f370 2024-04-25T09:00:25.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.017+0000 7fcae5b7c740 10 client.0 _ll_register_callbacks cb 0x563b10341db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:00:25.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.017+0000 7fcae0c12640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b10422ca0 0x563b104421c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.017+0000 7fcae0c12640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b10422ca0 0x563b104421c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40014/0 (socket says 172.21.15.43:40014) 2024-04-25T09:00:25.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.017+0000 7fcae0c12640 1 -- 172.21.15.43:0/996415873 learned_addr learned my addr 172.21.15.43:0/996415873 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:25.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.017+0000 7fcadbfff640 1 --2- 172.21.15.43:0/996415873 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b1044d760 0x563b10433230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.017+0000 7fcae1413640 1 --2- 172.21.15.43:0/996415873 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563b1045f370 0x563b10433770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.017+0000 7fcae0c12640 1 -- 172.21.15.43:0/996415873 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563b1045f370 msgr2=0x563b10433770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.017+0000 7fcae0c12640 1 --2- 172.21.15.43:0/996415873 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563b1045f370 0x563b10433770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.017+0000 7fcae0c12640 1 -- 172.21.15.43:0/996415873 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b1044d760 msgr2=0x563b10433230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.017+0000 7fcae0c12640 1 --2- 172.21.15.43:0/996415873 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b1044d760 0x563b10433230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.017+0000 7fcae0c12640 1 -- 172.21.15.43:0/996415873 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b1005a0f0 con 0x563b10422ca0 2024-04-25T09:00:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.018+0000 7fcae0c12640 1 --2- 172.21.15.43:0/996415873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b10422ca0 0x563b104421c0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fcad0007680 tx=0x7fcad0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee8e7973dd593e5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.018+0000 7fcabb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:00:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.018+0000 7fcabb7fe640 1 -- 172.21.15.43:0/996415873 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad0026060 con 0x563b10422ca0 2024-04-25T09:00:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.018+0000 7fcabb7fe640 1 -- 172.21.15.43:0/996415873 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fcad0021b00 con 0x563b10422ca0 2024-04-25T09:00:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.018+0000 7fcae5b7c740 1 -- 172.21.15.43:0/996415873 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcad407de70 con 0x563b10422ca0 2024-04-25T09:00:25.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.018+0000 7fcabb7fe640 1 -- 172.21.15.43:0/996415873 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcad0021d00 con 0x563b10422ca0 2024-04-25T09:00:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.018+0000 7fcae5b7c740 10 client.7410 fetch_fsmap learned FSMap version 48 2024-04-25T09:00:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.018+0000 7fcae5b7c740 1 -- 172.21.15.43:0/996415873 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x563b101c4f60 con 0x563b10422ca0 2024-04-25T09:00:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.018+0000 7fcabb7fe640 1 -- 172.21.15.43:0/996415873 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcad407de70 con 0x563b10422ca0 2024-04-25T09:00:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.018+0000 7fcabb7fe640 1 -- 172.21.15.43:0/996415873 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 48) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcad0012410 con 0x563b10422ca0 2024-04-25T09:00:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.018+0000 7fcae5b7c740 10 client.7410 fetch_fsmap finished waiting for FSMap version 48 2024-04-25T09:00:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.018+0000 7fcae5b7c740 10 client.7410 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T09:00:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.018+0000 7fcae5b7c740 20 client.7410 populate_metadata read hostname 'smithi043' 2024-04-25T09:00:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.018+0000 7fcae5b7c740 1 --2- 172.21.15.43:0/996415873 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x563b10444a50 0x563b1044e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.018+0000 7fcae5b7c740 4 client.7410 mds_command: new command op to 7164 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:00:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.018+0000 7fcae5b7c740 1 -- 172.21.15.43:0/996415873 --> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563b103930d0 con 0x563b10444a50 2024-04-25T09:00:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.019+0000 7fcadbfff640 1 --2- 172.21.15.43:0/996415873 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x563b10444a50 0x563b1044e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.019+0000 7fcabb7fe640 10 client.7410 ms_handle_connect on v2:172.21.15.193:6835/3458098162 2024-04-25T09:00:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.019+0000 7fcadbfff640 1 --2- 172.21.15.43:0/996415873 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x563b10444a50 0x563b1044e1e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53601715c5984a7b server_cookie=13a79ffe7553d0e1 in_seq=0 out_seq=0 2024-04-25T09:00:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.019+0000 7fcabb7fe640 1 -- 172.21.15.43:0/996415873 <== mds.0 v2:172.21.15.193:6835/3458098162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563b103930d0 con 0x563b10444a50 2024-04-25T09:00:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.019+0000 7fcabb7fe640 10 client.7410 handle_command_reply: tid=0 2024-04-25T09:00:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.020+0000 7fcae5b7c740 1 client.7410 shutdown 2024-04-25T09:00:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.020+0000 7fcae5b7c740 1 -- 172.21.15.43:0/996415873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b10422ca0 msgr2=0x563b104421c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.020+0000 7fcae5b7c740 1 --2- 172.21.15.43:0/996415873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b10422ca0 0x563b104421c0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fcad0007680 tx=0x7fcad0000ed0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.020+0000 7fcae5b7c740 1 -- 172.21.15.43:0/996415873 shutdown_connections 2024-04-25T09:00:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.020+0000 7fcae5b7c740 1 --2- 172.21.15.43:0/996415873 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x563b10444a50 0x563b1044e1e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.020+0000 7fcae5b7c740 1 --2- 172.21.15.43:0/996415873 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563b1045f370 0x563b10433770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.020+0000 7fcae5b7c740 1 --2- 172.21.15.43:0/996415873 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b1044d760 0x563b10433230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.020+0000 7fcabb7fe640 0 client.7410 ms_handle_reset on v2:172.21.15.193:6835/3458098162 2024-04-25T09:00:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.020+0000 7fcae5b7c740 1 --2- 172.21.15.43:0/996415873 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b10422ca0 0x563b104421c0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.020+0000 7fcae5b7c740 1 -- 172.21.15.43:0/996415873 >> 172.21.15.43:0/996415873 conn(0x563b103ea870 msgr2=0x563b1046bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.020+0000 7fcae5b7c740 1 -- 172.21.15.43:0/996415873 shutdown_connections 2024-04-25T09:00:25.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.020+0000 7fcae5b7c740 1 -- 172.21.15.43:0/996415873 wait complete. 2024-04-25T09:00:25.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.020+0000 7fcae5b7c740 20 client.7410 trim_cache size 0 max 16384 2024-04-25T09:00:25.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.034+0000 7fcae5b7c740 1 Processor -- start 2024-04-25T09:00:25.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.034+0000 7fcae5b7c740 1 -- start start 2024-04-25T09:00:25.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.035+0000 7fcae5b7c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b10422ca0 0x563b10451400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.035+0000 7fcae5b7c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b1044d760 0x563b10477810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.035+0000 7fcae5b7c740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563b1045f370 0x563b10477d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.035+0000 7fcae5b7c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc1fd9c0 con 0x563b1044d760 2024-04-25T09:00:25.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.035+0000 7fcae5b7c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc0b0960 con 0x563b10422ca0 2024-04-25T09:00:25.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.035+0000 7fcae5b7c740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc0afd60 con 0x563b1045f370 2024-04-25T09:00:25.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.035+0000 7fcadbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b1044d760 0x563b10477810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.035+0000 7fcadbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b1044d760 0x563b10477810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40022/0 (socket says 172.21.15.43:40022) 2024-04-25T09:00:25.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.035+0000 7fcadbfff640 1 -- 172.21.15.43:0/2533892839 learned_addr learned my addr 172.21.15.43:0/2533892839 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:25.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.035+0000 7fcadbfff640 1 -- 172.21.15.43:0/2533892839 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563b1045f370 msgr2=0x563b10477d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:25.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.035+0000 7fcadbfff640 1 --2- 172.21.15.43:0/2533892839 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563b1045f370 0x563b10477d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.035+0000 7fcadbfff640 1 -- 172.21.15.43:0/2533892839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b10422ca0 msgr2=0x563b10451400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:25.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.035+0000 7fcadbfff640 1 --2- 172.21.15.43:0/2533892839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b10422ca0 0x563b10451400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.035+0000 7fcadbfff640 1 -- 172.21.15.43:0/2533892839 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b100331b0 con 0x563b1044d760 2024-04-25T09:00:25.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.035+0000 7fcadbfff640 1 --2- 172.21.15.43:0/2533892839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b1044d760 0x563b10477810 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x563b10420280 tx=0x7fcac8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb3881b9a8bacd10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:25.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.035+0000 7fcac5ffb640 1 -- 172.21.15.43:0/2533892839 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac801f020 con 0x563b1044d760 2024-04-25T09:00:25.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.036+0000 7fcac5ffb640 1 -- 172.21.15.43:0/2533892839 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcac801bb00 con 0x563b1044d760 2024-04-25T09:00:25.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.036+0000 7fcac5ffb640 1 -- 172.21.15.43:0/2533892839 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac801b130 con 0x563b1044d760 2024-04-25T09:00:25.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.036+0000 7fcae5b7c740 1 -- 172.21.15.43:0/2533892839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b1044d760 msgr2=0x563b10477810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:25.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.036+0000 7fcae5b7c740 1 --2- 172.21.15.43:0/2533892839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b1044d760 0x563b10477810 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x563b10420280 tx=0x7fcac8008040 comp rx=0 tx=0).stop 2024-04-25T09:00:25.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.036+0000 7fcae5b7c740 1 -- 172.21.15.43:0/2533892839 shutdown_connections 2024-04-25T09:00:25.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.036+0000 7fcae5b7c740 1 --2- 172.21.15.43:0/2533892839 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563b1045f370 0x563b10477d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.036+0000 7fcae5b7c740 1 --2- 172.21.15.43:0/2533892839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b1044d760 0x563b10477810 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.036+0000 7fcae5b7c740 1 --2- 172.21.15.43:0/2533892839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b10422ca0 0x563b10451400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.036+0000 7fcae5b7c740 1 -- 172.21.15.43:0/2533892839 >> 172.21.15.43:0/2533892839 conn(0x563b103ea870 msgr2=0x563b104335a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:25.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.036+0000 7fcae5b7c740 1 -- 172.21.15.43:0/2533892839 shutdown_connections 2024-04-25T09:00:25.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.036+0000 7fcae5b7c740 1 -- 172.21.15.43:0/2533892839 wait complete. 2024-04-25T09:00:25.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.037+0000 7fcae5b7c740 1 -- start start 2024-04-25T09:00:25.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.037+0000 7fcae5b7c740 1 Processor -- start 2024-04-25T09:00:25.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.038+0000 7fcae5b7c740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563b10422ca0 0x563b104d1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.038+0000 7fcae5b7c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b1044d760 0x563b10477810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.038+0000 7fcae5b7c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b1045f370 0x563b10477d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.038+0000 7fcae5b7c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcadc0b0360 con 0x563b1044d760 2024-04-25T09:00:25.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.038+0000 7fcae5b7c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b1044aaa0 con 0x563b1045f370 2024-04-25T09:00:25.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.038+0000 7fcae5b7c740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b104d2090 con 0x563b10422ca0 2024-04-25T09:00:25.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.038+0000 7fcae5b7c740 10 client.0 _ll_register_callbacks cb 0x7fcad00059b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:00:25.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.038+0000 7fcadbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b1044d760 0x563b10477810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.038+0000 7fcadbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b1044d760 0x563b10477810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40034/0 (socket says 172.21.15.43:40034) 2024-04-25T09:00:25.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.038+0000 7fcadbfff640 1 -- 172.21.15.43:0/6738422 learned_addr learned my addr 172.21.15.43:0/6738422 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:25.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.038+0000 7fcae0c12640 1 --2- 172.21.15.43:0/6738422 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563b10422ca0 0x563b104d1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.038+0000 7fcae1413640 1 --2- 172.21.15.43:0/6738422 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b1045f370 0x563b10477d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.038+0000 7fcadbfff640 1 -- 172.21.15.43:0/6738422 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563b10422ca0 msgr2=0x563b104d1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:25.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.038+0000 7fcadbfff640 1 --2- 172.21.15.43:0/6738422 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563b10422ca0 0x563b104d1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.038+0000 7fcadbfff640 1 -- 172.21.15.43:0/6738422 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b1045f370 msgr2=0x563b10477d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:25.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.038+0000 7fcadbfff640 1 --2- 172.21.15.43:0/6738422 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b1045f370 0x563b10477d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.038+0000 7fcadbfff640 1 -- 172.21.15.43:0/6738422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b10482420 con 0x563b1044d760 2024-04-25T09:00:25.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.038+0000 7fcadbfff640 1 --2- 172.21.15.43:0/6738422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b1044d760 0x563b10477810 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fcac8004700 tx=0x7fcac801ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef97d42e737ae8d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:25.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.038+0000 7fcabb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:00:25.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.038+0000 7fcabb7fe640 1 -- 172.21.15.43:0/6738422 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac801b6f0 con 0x563b1044d760 2024-04-25T09:00:25.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.038+0000 7fcabb7fe640 1 -- 172.21.15.43:0/6738422 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(77..77 src has 1..77) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fcac801b890 con 0x563b1044d760 2024-04-25T09:00:25.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.038+0000 7fcae5b7c740 1 -- 172.21.15.43:0/6738422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcac801bb00 con 0x563b1044d760 2024-04-25T09:00:25.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.039+0000 7fcabb7fe640 1 -- 172.21.15.43:0/6738422 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcac8035050 con 0x563b1044d760 2024-04-25T09:00:25.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.039+0000 7fcae5b7c740 10 client.7416 fetch_fsmap learned FSMap version 48 2024-04-25T09:00:25.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.039+0000 7fcae5b7c740 1 -- 172.21.15.43:0/6738422 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=2+}) v3 -- 0x563b10441310 con 0x563b1044d760 2024-04-25T09:00:25.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.039+0000 7fcabb7fe640 1 -- 172.21.15.43:0/6738422 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcac801bb00 con 0x563b1044d760 2024-04-25T09:00:25.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.039+0000 7fcabb7fe640 1 -- 172.21.15.43:0/6738422 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 48) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcac8038020 con 0x563b1044d760 2024-04-25T09:00:25.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.039+0000 7fcae5b7c740 10 client.7416 fetch_fsmap finished waiting for FSMap version 48 2024-04-25T09:00:25.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.039+0000 7fcae5b7c740 10 client.7416 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T09:00:25.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.039+0000 7fcae5b7c740 20 client.7416 populate_metadata read hostname 'smithi043' 2024-04-25T09:00:25.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.039+0000 7fcae5b7c740 1 --2- 172.21.15.43:0/6738422 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x563b10422190 0x563b1047a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.039+0000 7fcae5b7c740 4 client.7416 mds_command: new command op to 7164 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T09:00:25.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.039+0000 7fcae5b7c740 1 -- 172.21.15.43:0/6738422 --> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x563b1038d420 con 0x563b10422190 2024-04-25T09:00:25.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.039+0000 7fcae0c12640 1 --2- 172.21.15.43:0/6738422 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x563b10422190 0x563b1047a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.039+0000 7fcae0c12640 1 --2- 172.21.15.43:0/6738422 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x563b10422190 0x563b1047a3c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d54eeb7f0b9e50 server_cookie=d6b8df50abba6d28 in_seq=0 out_seq=0 2024-04-25T09:00:25.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.039+0000 7fcabb7fe640 10 client.7416 ms_handle_connect on v2:172.21.15.193:6835/3458098162 2024-04-25T09:00:25.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.040+0000 7fcabb7fe640 1 -- 172.21.15.43:0/6738422 <== mds.0 v2:172.21.15.193:6835/3458098162 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17948 (crc 0 0 0) 0x563b1038d420 con 0x563b10422190 2024-04-25T09:00:25.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.040+0000 7fcabb7fe640 10 client.7416 handle_command_reply: tid=0 2024-04-25T09:00:25.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.040+0000 7fcae5b7c740 1 client.7416 shutdown 2024-04-25T09:00:25.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.040+0000 7fcae5b7c740 1 -- 172.21.15.43:0/6738422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b1044d760 msgr2=0x563b10477810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:25.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.040+0000 7fcae5b7c740 1 --2- 172.21.15.43:0/6738422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b1044d760 0x563b10477810 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fcac8004700 tx=0x7fcac801ba90 comp rx=0 tx=0).stop 2024-04-25T09:00:25.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.040+0000 7fcae5b7c740 1 -- 172.21.15.43:0/6738422 shutdown_connections 2024-04-25T09:00:25.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.040+0000 7fcae5b7c740 1 --2- 172.21.15.43:0/6738422 >> [v2:172.21.15.193:6835/3458098162,v1:172.21.15.193:6836/3458098162] conn(0x563b10422190 0x563b1047a3c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.040+0000 7fcae5b7c740 1 --2- 172.21.15.43:0/6738422 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b1045f370 0x563b10477d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.040+0000 7fcae5b7c740 1 --2- 172.21.15.43:0/6738422 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b1044d760 0x563b10477810 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.040+0000 7fcabb7fe640 0 client.7416 ms_handle_reset on v2:172.21.15.193:6835/3458098162 2024-04-25T09:00:25.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.040+0000 7fcae5b7c740 1 --2- 172.21.15.43:0/6738422 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563b10422ca0 0x563b104d1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.040+0000 7fcae5b7c740 1 -- 172.21.15.43:0/6738422 >> 172.21.15.43:0/6738422 conn(0x563b103ea870 msgr2=0x563b104513c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:25.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.040+0000 7fcae5b7c740 1 -- 172.21.15.43:0/6738422 shutdown_connections 2024-04-25T09:00:25.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.040+0000 7fcae5b7c740 1 -- 172.21.15.43:0/6738422 wait complete. 2024-04-25T09:00:25.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.040+0000 7fcae5b7c740 20 client.7416 trim_cache size 0 max 16384 2024-04-25T09:00:25.036 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:00:25.036 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T09:00:25.036 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 4018, 2024-04-25T09:00:25.036 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 4019, 2024-04-25T09:00:25.036 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 1373620, 2024-04-25T09:00:25.036 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 32807687, 2024-04-25T09:00:25.036 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 17, 2024-04-25T09:00:25.036 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 6, 2024-04-25T09:00:25.036 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.314973409, 2024-04-25T09:00:25.036 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.145187774, 2024-04-25T09:00:25.037 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.061916618, 2024-04-25T09:00:25.037 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.089900690, 2024-04-25T09:00:25.037 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:00:25.037 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 4017, 2024-04-25T09:00:25.037 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.380034370, 2024-04-25T09:00:25.037 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000094606 2024-04-25T09:00:25.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.037 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:00:25.037 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 10, 2024-04-25T09:00:25.037 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000873, 2024-04-25T09:00:25.037 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000087 2024-04-25T09:00:25.037 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.037 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T09:00:25.037 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T09:00:25.038 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.038 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T09:00:25.038 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 36645, 2024-04-25T09:00:25.038 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 62533, 2024-04-25T09:00:25.038 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 12634803, 2024-04-25T09:00:25.038 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 59092084, 2024-04-25T09:00:25.038 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 7, 2024-04-25T09:00:25.038 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 4, 2024-04-25T09:00:25.038 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 2.501270295, 2024-04-25T09:00:25.038 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 1.163078019, 2024-04-25T09:00:25.038 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 1.470965488, 2024-04-25T09:00:25.038 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.096001971, 2024-04-25T09:00:25.038 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:00:25.038 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 62533, 2024-04-25T09:00:25.039 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 1.337934056, 2024-04-25T09:00:25.039 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000021395 2024-04-25T09:00:25.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.039 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:00:25.039 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 88588, 2024-04-25T09:00:25.039 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.009455094, 2024-04-25T09:00:25.039 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000106 2024-04-25T09:00:25.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.039 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 124144, 2024-04-25T09:00:25.039 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 27568 2024-04-25T09:00:25.039 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.039 INFO:teuthology.orchestra.run.smithi043.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T09:00:25.039 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_messages": 4195, 2024-04-25T09:00:25.039 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages": 4217, 2024-04-25T09:00:25.039 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_bytes": 1051747, 2024-04-25T09:00:25.040 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_bytes": 21511784, 2024-04-25T09:00:25.040 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_created_connections": 6, 2024-04-25T09:00:25.040 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_active_connections": 4, 2024-04-25T09:00:25.040 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_total_time": 0.359336382, 2024-04-25T09:00:25.040 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_send_time": 0.127362186, 2024-04-25T09:00:25.040 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_recv_time": 0.062410951, 2024-04-25T09:00:25.040 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_running_fast_dispatch_time": 0.149485405, 2024-04-25T09:00:25.040 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T09:00:25.040 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 4216, 2024-04-25T09:00:25.040 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.425261133, 2024-04-25T09:00:25.040 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000100868 2024-04-25T09:00:25.040 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.040 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_handle_ack_lat": { 2024-04-25T09:00:25.041 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T09:00:25.041 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T09:00:25.041 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T09:00:25.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.041 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_recv_encrypted_bytes": 160, 2024-04-25T09:00:25.041 INFO:teuthology.orchestra.run.smithi043.stdout: "msgr_send_encrypted_bytes": 122672 2024-04-25T09:00:25.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.041 INFO:teuthology.orchestra.run.smithi043.stdout: "cct": { 2024-04-25T09:00:25.041 INFO:teuthology.orchestra.run.smithi043.stdout: "total_workers": 1, 2024-04-25T09:00:25.041 INFO:teuthology.orchestra.run.smithi043.stdout: "unhealthy_workers": 0 2024-04-25T09:00:25.041 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.041 INFO:teuthology.orchestra.run.smithi043.stdout: "finisher-MDSRank": { 2024-04-25T09:00:25.041 INFO:teuthology.orchestra.run.smithi043.stdout: "queue_len": 0, 2024-04-25T09:00:25.041 INFO:teuthology.orchestra.run.smithi043.stdout: "complete_latency": { 2024-04-25T09:00:25.042 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 10864, 2024-04-25T09:00:25.042 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 12.126782222, 2024-04-25T09:00:25.042 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.001116235 2024-04-25T09:00:25.042 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:00:25.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.042 INFO:teuthology.orchestra.run.smithi043.stdout: "finisher-PurgeQueue": { 2024-04-25T09:00:25.042 INFO:teuthology.orchestra.run.smithi043.stdout: "queue_len": 0, 2024-04-25T09:00:25.042 INFO:teuthology.orchestra.run.smithi043.stdout: "complete_latency": { 2024-04-25T09:00:25.042 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 3442, 2024-04-25T09:00:25.042 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.609301295, 2024-04-25T09:00:25.042 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000177019 2024-04-25T09:00:25.042 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:00:25.042 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.043 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": { 2024-04-25T09:00:25.043 INFO:teuthology.orchestra.run.smithi043.stdout: "request": 28285, 2024-04-25T09:00:25.043 INFO:teuthology.orchestra.run.smithi043.stdout: "reply": 28285, 2024-04-25T09:00:25.043 INFO:teuthology.orchestra.run.smithi043.stdout: "reply_latency": { 2024-04-25T09:00:25.043 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 28285, 2024-04-25T09:00:25.043 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 222.806273216, 2024-04-25T09:00:25.043 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.007877188 2024-04-25T09:00:25.043 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.043 INFO:teuthology.orchestra.run.smithi043.stdout: "slow_reply": 0, 2024-04-25T09:00:25.043 INFO:teuthology.orchestra.run.smithi043.stdout: "forward": 0, 2024-04-25T09:00:25.043 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_fetch_complete": 0, 2024-04-25T09:00:25.043 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_fetch_keys": 0, 2024-04-25T09:00:25.043 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_commit": 12, 2024-04-25T09:00:25.043 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_split": 0, 2024-04-25T09:00:25.044 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_merge": 0, 2024-04-25T09:00:25.044 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes": 17838, 2024-04-25T09:00:25.044 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_top": 0, 2024-04-25T09:00:25.044 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_bottom": 17838, 2024-04-25T09:00:25.044 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_pin_tail": 0, 2024-04-25T09:00:25.044 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_pinned": 17838, 2024-04-25T09:00:25.044 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_expired": 0, 2024-04-25T09:00:25.044 INFO:teuthology.orchestra.run.smithi043.stdout: "inodes_with_caps": 2, 2024-04-25T09:00:25.044 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": 2, 2024-04-25T09:00:25.044 INFO:teuthology.orchestra.run.smithi043.stdout: "subtrees": 2, 2024-04-25T09:00:25.044 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse": 46904, 2024-04-25T09:00:25.044 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_hit": 46420, 2024-04-25T09:00:25.044 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_forward": 0, 2024-04-25T09:00:25.045 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_discover": 0, 2024-04-25T09:00:25.045 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_dir_fetch": 0, 2024-04-25T09:00:25.045 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_remote_ino": 0, 2024-04-25T09:00:25.045 INFO:teuthology.orchestra.run.smithi043.stdout: "traverse_lock": 157, 2024-04-25T09:00:25.045 INFO:teuthology.orchestra.run.smithi043.stdout: "load_cent": 0, 2024-04-25T09:00:25.045 INFO:teuthology.orchestra.run.smithi043.stdout: "q": 0, 2024-04-25T09:00:25.045 INFO:teuthology.orchestra.run.smithi043.stdout: "exported": 0, 2024-04-25T09:00:25.045 INFO:teuthology.orchestra.run.smithi043.stdout: "exported_inodes": 0, 2024-04-25T09:00:25.045 INFO:teuthology.orchestra.run.smithi043.stdout: "imported": 0, 2024-04-25T09:00:25.045 INFO:teuthology.orchestra.run.smithi043.stdout: "imported_inodes": 0, 2024-04-25T09:00:25.045 INFO:teuthology.orchestra.run.smithi043.stdout: "openino_dir_fetch": 0, 2024-04-25T09:00:25.045 INFO:teuthology.orchestra.run.smithi043.stdout: "openino_backtrace_fetch": 2011, 2024-04-25T09:00:25.045 INFO:teuthology.orchestra.run.smithi043.stdout: "openino_peer_discover": 0, 2024-04-25T09:00:25.045 INFO:teuthology.orchestra.run.smithi043.stdout: "root_rfiles": 0, 2024-04-25T09:00:25.046 INFO:teuthology.orchestra.run.smithi043.stdout: "root_rbytes": 0, 2024-04-25T09:00:25.046 INFO:teuthology.orchestra.run.smithi043.stdout: "root_rsnaps": 0, 2024-04-25T09:00:25.046 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_backtrace_fetch": 2011, 2024-04-25T09:00:25.046 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_set_tag": 0, 2024-04-25T09:00:25.046 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T09:00:25.046 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_inotable_repaired": 0, 2024-04-25T09:00:25.046 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_dir_inodes": 11, 2024-04-25T09:00:25.046 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_dir_base_inodes": 1, 2024-04-25T09:00:25.046 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_dirfrag_rstats": 10, 2024-04-25T09:00:25.046 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_file_inodes": 2000, 2024-04-25T09:00:25.046 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_inode_file_caps": 0, 2024-04-25T09:00:25.046 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_revoke": 6190, 2024-04-25T09:00:25.047 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_grant": 6010, 2024-04-25T09:00:25.047 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T09:00:25.047 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T09:00:25.047 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T09:00:25.047 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_caps": 6190, 2024-04-25T09:00:25.047 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_caps_dirty": 2000, 2024-04-25T09:00:25.047 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_cap_release": 28, 2024-04-25T09:00:25.047 INFO:teuthology.orchestra.run.smithi043.stdout: "process_request_cap_release": 17838 2024-04-25T09:00:25.047 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.047 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_cache": { 2024-04-25T09:00:25.047 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_update": 0, 2024-04-25T09:00:25.047 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_update_receipt": 0, 2024-04-25T09:00:25.047 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_try_discover": 0, 2024-04-25T09:00:25.047 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_send_discover": 0, 2024-04-25T09:00:25.048 INFO:teuthology.orchestra.run.smithi043.stdout: "dir_handle_discover": 0, 2024-04-25T09:00:25.048 INFO:teuthology.orchestra.run.smithi043.stdout: "num_strays": 0, 2024-04-25T09:00:25.048 INFO:teuthology.orchestra.run.smithi043.stdout: "num_strays_delayed": 0, 2024-04-25T09:00:25.048 INFO:teuthology.orchestra.run.smithi043.stdout: "num_strays_enqueuing": 0, 2024-04-25T09:00:25.048 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_created": 4116, 2024-04-25T09:00:25.048 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_enqueued": 3528, 2024-04-25T09:00:25.048 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_reintegrated": 2216, 2024-04-25T09:00:25.048 INFO:teuthology.orchestra.run.smithi043.stdout: "strays_migrated": 0, 2024-04-25T09:00:25.048 INFO:teuthology.orchestra.run.smithi043.stdout: "num_recovering_processing": 0, 2024-04-25T09:00:25.048 INFO:teuthology.orchestra.run.smithi043.stdout: "num_recovering_enqueued": 0, 2024-04-25T09:00:25.048 INFO:teuthology.orchestra.run.smithi043.stdout: "num_recovering_prioritized": 0, 2024-04-25T09:00:25.048 INFO:teuthology.orchestra.run.smithi043.stdout: "recovery_started": 0, 2024-04-25T09:00:25.048 INFO:teuthology.orchestra.run.smithi043.stdout: "recovery_completed": 0, 2024-04-25T09:00:25.049 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_enqueue_scrub": 1, 2024-04-25T09:00:25.049 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_exportdir": 0, 2024-04-25T09:00:25.049 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_flush": 0, 2024-04-25T09:00:25.049 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_fragmentdir": 0, 2024-04-25T09:00:25.049 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_fragstats": 0, 2024-04-25T09:00:25.049 INFO:teuthology.orchestra.run.smithi043.stdout: "ireq_inodestats": 0 2024-04-25T09:00:25.049 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.049 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_log": { 2024-04-25T09:00:25.049 INFO:teuthology.orchestra.run.smithi043.stdout: "evlrg": 0, 2024-04-25T09:00:25.049 INFO:teuthology.orchestra.run.smithi043.stdout: "evadd": 23712, 2024-04-25T09:00:25.049 INFO:teuthology.orchestra.run.smithi043.stdout: "evex": 0, 2024-04-25T09:00:25.049 INFO:teuthology.orchestra.run.smithi043.stdout: "evtrm": 0, 2024-04-25T09:00:25.049 INFO:teuthology.orchestra.run.smithi043.stdout: "ev": 23712, 2024-04-25T09:00:25.050 INFO:teuthology.orchestra.run.smithi043.stdout: "evexg": 0, 2024-04-25T09:00:25.050 INFO:teuthology.orchestra.run.smithi043.stdout: "evexd": 0, 2024-04-25T09:00:25.050 INFO:teuthology.orchestra.run.smithi043.stdout: "segadd": 31, 2024-04-25T09:00:25.050 INFO:teuthology.orchestra.run.smithi043.stdout: "segex": 0, 2024-04-25T09:00:25.050 INFO:teuthology.orchestra.run.smithi043.stdout: "segtrm": 0, 2024-04-25T09:00:25.050 INFO:teuthology.orchestra.run.smithi043.stdout: "seg": 31, 2024-04-25T09:00:25.050 INFO:teuthology.orchestra.run.smithi043.stdout: "segmjr": 3, 2024-04-25T09:00:25.050 INFO:teuthology.orchestra.run.smithi043.stdout: "segexg": 0, 2024-04-25T09:00:25.050 INFO:teuthology.orchestra.run.smithi043.stdout: "segexd": 0, 2024-04-25T09:00:25.050 INFO:teuthology.orchestra.run.smithi043.stdout: "expos": 4194304, 2024-04-25T09:00:25.050 INFO:teuthology.orchestra.run.smithi043.stdout: "wrpos": 82314052, 2024-04-25T09:00:25.050 INFO:teuthology.orchestra.run.smithi043.stdout: "rdpos": 0, 2024-04-25T09:00:25.050 INFO:teuthology.orchestra.run.smithi043.stdout: "jlat": { 2024-04-25T09:00:25.051 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 6507, 2024-04-25T09:00:25.051 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 64.974305998, 2024-04-25T09:00:25.051 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.009985293 2024-04-25T09:00:25.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.051 INFO:teuthology.orchestra.run.smithi043.stdout: "replayed": 0 2024-04-25T09:00:25.051 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.051 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_mem": { 2024-04-25T09:00:25.051 INFO:teuthology.orchestra.run.smithi043.stdout: "ino": 14, 2024-04-25T09:00:25.051 INFO:teuthology.orchestra.run.smithi043.stdout: "ino+": 2015, 2024-04-25T09:00:25.051 INFO:teuthology.orchestra.run.smithi043.stdout: "ino-": 2001, 2024-04-25T09:00:25.051 INFO:teuthology.orchestra.run.smithi043.stdout: "dir": 13, 2024-04-25T09:00:25.051 INFO:teuthology.orchestra.run.smithi043.stdout: "dir+": 13, 2024-04-25T09:00:25.051 INFO:teuthology.orchestra.run.smithi043.stdout: "dir-": 0, 2024-04-25T09:00:25.051 INFO:teuthology.orchestra.run.smithi043.stdout: "dn": 17838, 2024-04-25T09:00:25.052 INFO:teuthology.orchestra.run.smithi043.stdout: "dn+": 19838, 2024-04-25T09:00:25.052 INFO:teuthology.orchestra.run.smithi043.stdout: "dn-": 2000, 2024-04-25T09:00:25.052 INFO:teuthology.orchestra.run.smithi043.stdout: "cap": 2, 2024-04-25T09:00:25.052 INFO:teuthology.orchestra.run.smithi043.stdout: "cap+": 2002, 2024-04-25T09:00:25.052 INFO:teuthology.orchestra.run.smithi043.stdout: "cap-": 2000, 2024-04-25T09:00:25.052 INFO:teuthology.orchestra.run.smithi043.stdout: "rss": 96688, 2024-04-25T09:00:25.052 INFO:teuthology.orchestra.run.smithi043.stdout: "heap": 215396 2024-04-25T09:00:25.052 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.052 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_server": { 2024-04-25T09:00:25.052 INFO:teuthology.orchestra.run.smithi043.stdout: "dispatch_client_request": 37689, 2024-04-25T09:00:25.052 INFO:teuthology.orchestra.run.smithi043.stdout: "dispatch_server_request": 0, 2024-04-25T09:00:25.052 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_request": 28285, 2024-04-25T09:00:25.052 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_client_session": 7, 2024-04-25T09:00:25.052 INFO:teuthology.orchestra.run.smithi043.stdout: "handle_peer_request": 0, 2024-04-25T09:00:25.053 INFO:teuthology.orchestra.run.smithi043.stdout: "req_create_latency": { 2024-04-25T09:00:25.053 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 2000, 2024-04-25T09:00:25.053 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 1.320921704, 2024-04-25T09:00:25.053 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000660460 2024-04-25T09:00:25.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.053 INFO:teuthology.orchestra.run.smithi043.stdout: "req_getattr_latency": { 2024-04-25T09:00:25.053 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 2008, 2024-04-25T09:00:25.053 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.553011189, 2024-04-25T09:00:25.053 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000275403 2024-04-25T09:00:25.053 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.053 INFO:teuthology.orchestra.run.smithi043.stdout: "req_getfilelock_latency": { 2024-04-25T09:00:25.053 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T09:00:25.053 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T09:00:25.053 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T09:00:25.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.054 INFO:teuthology.orchestra.run.smithi043.stdout: "req_link_latency": { 2024-04-25T09:00:25.054 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 6000, 2024-04-25T09:00:25.054 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 9.405248756, 2024-04-25T09:00:25.054 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.001567541 2024-04-25T09:00:25.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.054 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookup_latency": { 2024-04-25T09:00:25.054 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 8048, 2024-04-25T09:00:25.054 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 7.188344113, 2024-04-25T09:00:25.054 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000893183 2024-04-25T09:00:25.054 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.054 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookuphash_latency": { 2024-04-25T09:00:25.054 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T09:00:25.054 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T09:00:25.055 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T09:00:25.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.055 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupino_latency": { 2024-04-25T09:00:25.055 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T09:00:25.055 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T09:00:25.055 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T09:00:25.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.055 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupname_latency": { 2024-04-25T09:00:25.055 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T09:00:25.055 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T09:00:25.055 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T09:00:25.055 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.055 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupparent_latency": { 2024-04-25T09:00:25.056 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T09:00:25.056 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T09:00:25.056 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T09:00:25.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.056 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lookupsnap_latency": { 2024-04-25T09:00:25.056 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 3, 2024-04-25T09:00:25.056 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000479137, 2024-04-25T09:00:25.056 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000159712 2024-04-25T09:00:25.056 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.056 INFO:teuthology.orchestra.run.smithi043.stdout: "req_lssnap_latency": { 2024-04-25T09:00:25.056 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T09:00:25.056 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T09:00:25.056 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T09:00:25.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.057 INFO:teuthology.orchestra.run.smithi043.stdout: "req_mkdir_latency": { 2024-04-25T09:00:25.057 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 1, 2024-04-25T09:00:25.057 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.002887482, 2024-04-25T09:00:25.057 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.002887482 2024-04-25T09:00:25.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.057 INFO:teuthology.orchestra.run.smithi043.stdout: "req_mknod_latency": { 2024-04-25T09:00:25.057 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T09:00:25.057 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T09:00:25.057 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T09:00:25.057 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.057 INFO:teuthology.orchestra.run.smithi043.stdout: "req_mksnap_latency": { 2024-04-25T09:00:25.057 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 1, 2024-04-25T09:00:25.057 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.005256154, 2024-04-25T09:00:25.057 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.005256154 2024-04-25T09:00:25.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.058 INFO:teuthology.orchestra.run.smithi043.stdout: "req_open_latency": { 2024-04-25T09:00:25.058 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T09:00:25.058 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T09:00:25.058 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T09:00:25.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.058 INFO:teuthology.orchestra.run.smithi043.stdout: "req_readdir_latency": { 2024-04-25T09:00:25.058 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 6, 2024-04-25T09:00:25.058 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.092175118, 2024-04-25T09:00:25.058 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.015362519 2024-04-25T09:00:25.058 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.058 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rename_latency": { 2024-04-25T09:00:25.058 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 2216, 2024-04-25T09:00:25.058 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 189.223382261, 2024-04-25T09:00:25.059 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.085389612 2024-04-25T09:00:25.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.059 INFO:teuthology.orchestra.run.smithi043.stdout: "req_renamesnap_latency": { 2024-04-25T09:00:25.059 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T09:00:25.059 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T09:00:25.059 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T09:00:25.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.059 INFO:teuthology.orchestra.run.smithi043.stdout: "req_snapdiff_latency": { 2024-04-25T09:00:25.059 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T09:00:25.059 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T09:00:25.059 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T09:00:25.059 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.059 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rmdir_latency": { 2024-04-25T09:00:25.059 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T09:00:25.059 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T09:00:25.060 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T09:00:25.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.060 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rmsnap_latency": { 2024-04-25T09:00:25.060 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 1, 2024-04-25T09:00:25.060 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.002953385, 2024-04-25T09:00:25.060 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.002953385 2024-04-25T09:00:25.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.060 INFO:teuthology.orchestra.run.smithi043.stdout: "req_rmxattr_latency": { 2024-04-25T09:00:25.060 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T09:00:25.060 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T09:00:25.060 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T09:00:25.060 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.060 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setattr_latency": { 2024-04-25T09:00:25.060 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 1, 2024-04-25T09:00:25.061 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000294992, 2024-04-25T09:00:25.061 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000294992 2024-04-25T09:00:25.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.061 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setdirlayout_latency": { 2024-04-25T09:00:25.061 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T09:00:25.061 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T09:00:25.061 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T09:00:25.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.061 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setfilelock_latency": { 2024-04-25T09:00:25.061 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T09:00:25.061 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T09:00:25.061 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T09:00:25.061 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.061 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setlayout_latency": { 2024-04-25T09:00:25.061 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T09:00:25.062 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T09:00:25.062 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T09:00:25.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.062 INFO:teuthology.orchestra.run.smithi043.stdout: "req_setxattr_latency": { 2024-04-25T09:00:25.062 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T09:00:25.062 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T09:00:25.062 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T09:00:25.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.062 INFO:teuthology.orchestra.run.smithi043.stdout: "req_symlink_latency": { 2024-04-25T09:00:25.062 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T09:00:25.062 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T09:00:25.062 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T09:00:25.062 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.062 INFO:teuthology.orchestra.run.smithi043.stdout: "req_unlink_latency": { 2024-04-25T09:00:25.063 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 8000, 2024-04-25T09:00:25.063 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 15.011318925, 2024-04-25T09:00:25.063 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.001876414 2024-04-25T09:00:25.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.063 INFO:teuthology.orchestra.run.smithi043.stdout: "cap_revoke_eviction": 0, 2024-04-25T09:00:25.063 INFO:teuthology.orchestra.run.smithi043.stdout: "cap_acquisition_throttle": 0, 2024-04-25T09:00:25.063 INFO:teuthology.orchestra.run.smithi043.stdout: "req_getvxattr_latency": { 2024-04-25T09:00:25.063 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T09:00:25.063 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T09:00:25.063 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T09:00:25.063 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:00:25.063 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.063 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_sessions": { 2024-04-25T09:00:25.063 INFO:teuthology.orchestra.run.smithi043.stdout: "session_count": 1, 2024-04-25T09:00:25.064 INFO:teuthology.orchestra.run.smithi043.stdout: "session_add": 1, 2024-04-25T09:00:25.064 INFO:teuthology.orchestra.run.smithi043.stdout: "session_remove": 0, 2024-04-25T09:00:25.064 INFO:teuthology.orchestra.run.smithi043.stdout: "sessions_open": 1, 2024-04-25T09:00:25.064 INFO:teuthology.orchestra.run.smithi043.stdout: "sessions_stale": 0, 2024-04-25T09:00:25.064 INFO:teuthology.orchestra.run.smithi043.stdout: "total_load": 19391, 2024-04-25T09:00:25.064 INFO:teuthology.orchestra.run.smithi043.stdout: "average_load": 19391, 2024-04-25T09:00:25.064 INFO:teuthology.orchestra.run.smithi043.stdout: "avg_session_uptime": 88, 2024-04-25T09:00:25.064 INFO:teuthology.orchestra.run.smithi043.stdout: "mdthresh_evicted": 0 2024-04-25T09:00:25.064 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.064 INFO:teuthology.orchestra.run.smithi043.stdout: "mempool": { 2024-04-25T09:00:25.064 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_bytes": 0, 2024-04-25T09:00:25.064 INFO:teuthology.orchestra.run.smithi043.stdout: "bloom_filter_items": 0, 2024-04-25T09:00:25.064 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T09:00:25.064 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_alloc_items": 0, 2024-04-25T09:00:25.064 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T09:00:25.065 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_data_items": 0, 2024-04-25T09:00:25.065 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T09:00:25.065 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T09:00:25.065 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T09:00:25.065 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T09:00:25.065 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T09:00:25.065 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_other_items": 0, 2024-04-25T09:00:25.065 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T09:00:25.065 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T09:00:25.065 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_bytes": 0, 2024-04-25T09:00:25.065 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_extent_items": 0, 2024-04-25T09:00:25.065 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_bytes": 0, 2024-04-25T09:00:25.065 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_blob_items": 0, 2024-04-25T09:00:25.065 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T09:00:25.065 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T09:00:25.066 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T09:00:25.066 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T09:00:25.066 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T09:00:25.066 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_fsck_items": 0, 2024-04-25T09:00:25.066 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_bytes": 0, 2024-04-25T09:00:25.066 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_txc_items": 0, 2024-04-25T09:00:25.066 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T09:00:25.066 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T09:00:25.066 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_bytes": 0, 2024-04-25T09:00:25.066 INFO:teuthology.orchestra.run.smithi043.stdout: "bluestore_writing_items": 0, 2024-04-25T09:00:25.066 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_bytes": 0, 2024-04-25T09:00:25.066 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_items": 0, 2024-04-25T09:00:25.066 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T09:00:25.066 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_reader_items": 0, 2024-04-25T09:00:25.067 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T09:00:25.067 INFO:teuthology.orchestra.run.smithi043.stdout: "bluefs_file_writer_items": 0, 2024-04-25T09:00:25.067 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_bytes": 251182, 2024-04-25T09:00:25.067 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_anon_items": 86, 2024-04-25T09:00:25.067 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_bytes": 0, 2024-04-25T09:00:25.067 INFO:teuthology.orchestra.run.smithi043.stdout: "buffer_meta_items": 0, 2024-04-25T09:00:25.067 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_bytes": 0, 2024-04-25T09:00:25.067 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_items": 0, 2024-04-25T09:00:25.067 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_bytes": 0, 2024-04-25T09:00:25.067 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_mapbl_items": 0, 2024-04-25T09:00:25.067 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_bytes": 0, 2024-04-25T09:00:25.067 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_pglog_items": 0, 2024-04-25T09:00:25.067 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_bytes": 5608, 2024-04-25T09:00:25.067 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_items": 81, 2024-04-25T09:00:25.068 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T09:00:25.068 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_mapping_items": 0, 2024-04-25T09:00:25.068 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_bytes": 0, 2024-04-25T09:00:25.068 INFO:teuthology.orchestra.run.smithi043.stdout: "pgmap_items": 0, 2024-04-25T09:00:25.068 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_bytes": 13432524, 2024-04-25T09:00:25.068 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_co_items": 660959, 2024-04-25T09:00:25.068 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_bytes": 0, 2024-04-25T09:00:25.068 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_1_items": 0, 2024-04-25T09:00:25.068 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_bytes": 0, 2024-04-25T09:00:25.068 INFO:teuthology.orchestra.run.smithi043.stdout: "unittest_2_items": 0 2024-04-25T09:00:25.068 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.068 INFO:teuthology.orchestra.run.smithi043.stdout: "objecter": { 2024-04-25T09:00:25.068 INFO:teuthology.orchestra.run.smithi043.stdout: "op_active": 0, 2024-04-25T09:00:25.068 INFO:teuthology.orchestra.run.smithi043.stdout: "op_laggy": 0, 2024-04-25T09:00:25.068 INFO:teuthology.orchestra.run.smithi043.stdout: "op_send": 12323, 2024-04-25T09:00:25.069 INFO:teuthology.orchestra.run.smithi043.stdout: "op_send_bytes": 78518113, 2024-04-25T09:00:25.069 INFO:teuthology.orchestra.run.smithi043.stdout: "op_resend": 37, 2024-04-25T09:00:25.069 INFO:teuthology.orchestra.run.smithi043.stdout: "op_reply": 12323, 2024-04-25T09:00:25.069 INFO:teuthology.orchestra.run.smithi043.stdout: "op_latency": { 2024-04-25T09:00:25.069 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 12323, 2024-04-25T09:00:25.069 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 27.504577122, 2024-04-25T09:00:25.069 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.002231970 2024-04-25T09:00:25.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.069 INFO:teuthology.orchestra.run.smithi043.stdout: "op_inflight": 0, 2024-04-25T09:00:25.069 INFO:teuthology.orchestra.run.smithi043.stdout: "oplen_avg": { 2024-04-25T09:00:25.069 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 12323, 2024-04-25T09:00:25.069 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 12384 2024-04-25T09:00:25.069 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.070 INFO:teuthology.orchestra.run.smithi043.stdout: "op": 12323, 2024-04-25T09:00:25.070 INFO:teuthology.orchestra.run.smithi043.stdout: "op_r": 2081, 2024-04-25T09:00:25.070 INFO:teuthology.orchestra.run.smithi043.stdout: "op_w": 10242, 2024-04-25T09:00:25.070 INFO:teuthology.orchestra.run.smithi043.stdout: "op_rmw": 0, 2024-04-25T09:00:25.070 INFO:teuthology.orchestra.run.smithi043.stdout: "op_pg": 0, 2024-04-25T09:00:25.070 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_stat": 0, 2024-04-25T09:00:25.070 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_create": 11, 2024-04-25T09:00:25.070 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_read": 69, 2024-04-25T09:00:25.070 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_write": 6575, 2024-04-25T09:00:25.070 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_writefull": 41, 2024-04-25T09:00:25.070 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_writesame": 0, 2024-04-25T09:00:25.070 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_append": 0, 2024-04-25T09:00:25.070 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_zero": 1528, 2024-04-25T09:00:25.070 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_truncate": 0, 2024-04-25T09:00:25.071 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_delete": 2030, 2024-04-25T09:00:25.071 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_mapext": 0, 2024-04-25T09:00:25.071 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_sparse_read": 0, 2024-04-25T09:00:25.071 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_clonerange": 0, 2024-04-25T09:00:25.071 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_getxattr": 2012, 2024-04-25T09:00:25.071 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_setxattr": 22, 2024-04-25T09:00:25.071 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_cmpxattr": 0, 2024-04-25T09:00:25.071 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_rmxattr": 0, 2024-04-25T09:00:25.071 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_resetxattrs": 0, 2024-04-25T09:00:25.071 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_call": 0, 2024-04-25T09:00:25.071 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_watch": 0, 2024-04-25T09:00:25.071 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_notify": 0, 2024-04-25T09:00:25.071 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T09:00:25.071 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_pgls": 0, 2024-04-25T09:00:25.072 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_pgls_filter": 0, 2024-04-25T09:00:25.072 INFO:teuthology.orchestra.run.smithi043.stdout: "osdop_other": 0, 2024-04-25T09:00:25.072 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_active": 0, 2024-04-25T09:00:25.072 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_send": 0, 2024-04-25T09:00:25.072 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_resend": 0, 2024-04-25T09:00:25.072 INFO:teuthology.orchestra.run.smithi043.stdout: "linger_ping": 0, 2024-04-25T09:00:25.072 INFO:teuthology.orchestra.run.smithi043.stdout: "poolop_active": 0, 2024-04-25T09:00:25.072 INFO:teuthology.orchestra.run.smithi043.stdout: "poolop_send": 0, 2024-04-25T09:00:25.072 INFO:teuthology.orchestra.run.smithi043.stdout: "poolop_resend": 0, 2024-04-25T09:00:25.072 INFO:teuthology.orchestra.run.smithi043.stdout: "poolstat_active": 0, 2024-04-25T09:00:25.072 INFO:teuthology.orchestra.run.smithi043.stdout: "poolstat_send": 0, 2024-04-25T09:00:25.072 INFO:teuthology.orchestra.run.smithi043.stdout: "poolstat_resend": 0, 2024-04-25T09:00:25.072 INFO:teuthology.orchestra.run.smithi043.stdout: "statfs_active": 0, 2024-04-25T09:00:25.072 INFO:teuthology.orchestra.run.smithi043.stdout: "statfs_send": 0, 2024-04-25T09:00:25.073 INFO:teuthology.orchestra.run.smithi043.stdout: "statfs_resend": 0, 2024-04-25T09:00:25.073 INFO:teuthology.orchestra.run.smithi043.stdout: "command_active": 0, 2024-04-25T09:00:25.073 INFO:teuthology.orchestra.run.smithi043.stdout: "command_send": 0, 2024-04-25T09:00:25.073 INFO:teuthology.orchestra.run.smithi043.stdout: "command_resend": 0, 2024-04-25T09:00:25.073 INFO:teuthology.orchestra.run.smithi043.stdout: "map_epoch": 77, 2024-04-25T09:00:25.073 INFO:teuthology.orchestra.run.smithi043.stdout: "map_full": 0, 2024-04-25T09:00:25.073 INFO:teuthology.orchestra.run.smithi043.stdout: "map_inc": 5, 2024-04-25T09:00:25.073 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_sessions": 8, 2024-04-25T09:00:25.073 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_session_open": 8, 2024-04-25T09:00:25.073 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_session_close": 0, 2024-04-25T09:00:25.073 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_laggy": 0, 2024-04-25T09:00:25.073 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_wr": 80, 2024-04-25T09:00:25.073 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_rd": 0, 2024-04-25T09:00:25.074 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_del": 16 2024-04-25T09:00:25.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.074 INFO:teuthology.orchestra.run.smithi043.stdout: "oft": { 2024-04-25T09:00:25.074 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_objs": 1, 2024-04-25T09:00:25.074 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_kv_pairs": 0, 2024-04-25T09:00:25.074 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_updates": 0, 2024-04-25T09:00:25.074 INFO:teuthology.orchestra.run.smithi043.stdout: "omap_total_removes": 0 2024-04-25T09:00:25.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.074 INFO:teuthology.orchestra.run.smithi043.stdout: "osd-slow-ops": { 2024-04-25T09:00:25.074 INFO:teuthology.orchestra.run.smithi043.stdout: "slow_ops_count": 0 2024-04-25T09:00:25.074 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.074 INFO:teuthology.orchestra.run.smithi043.stdout: "purge_queue": { 2024-04-25T09:00:25.074 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing_ops": 0, 2024-04-25T09:00:25.075 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T09:00:25.075 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing": 0, 2024-04-25T09:00:25.075 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executing_high_water": 32, 2024-04-25T09:00:25.075 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executed_ops": 3528, 2024-04-25T09:00:25.075 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_executed": 3528, 2024-04-25T09:00:25.075 INFO:teuthology.orchestra.run.smithi043.stdout: "pq_item_in_journal": 0 2024-04-25T09:00:25.075 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.075 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T09:00:25.075 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-25T09:00:25.075 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 104857600, 2024-04-25T09:00:25.075 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-25T09:00:25.075 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 44853, 2024-04-25T09:00:25.075 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 11017495, 2024-04-25T09:00:25.075 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-25T09:00:25.076 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 44853, 2024-04-25T09:00:25.076 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-04-25T09:00:25.076 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-04-25T09:00:25.076 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 44853, 2024-04-25T09:00:25.076 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 11017495, 2024-04-25T09:00:25.076 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-25T09:00:25.076 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T09:00:25.076 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T09:00:25.076 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T09:00:25.076 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:00:25.076 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.076 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-objecter_bytes": { 2024-04-25T09:00:25.076 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-25T09:00:25.076 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 104857600, 2024-04-25T09:00:25.077 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-25T09:00:25.077 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 0, 2024-04-25T09:00:25.077 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 0, 2024-04-25T09:00:25.077 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-25T09:00:25.077 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 0, 2024-04-25T09:00:25.077 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 12323, 2024-04-25T09:00:25.077 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 78886665, 2024-04-25T09:00:25.077 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 12323, 2024-04-25T09:00:25.077 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 78886665, 2024-04-25T09:00:25.077 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-25T09:00:25.077 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T09:00:25.078 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T09:00:25.078 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T09:00:25.078 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:00:25.078 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.078 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-objecter_ops": { 2024-04-25T09:00:25.078 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-25T09:00:25.078 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 1024, 2024-04-25T09:00:25.078 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-25T09:00:25.078 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 0, 2024-04-25T09:00:25.078 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 0, 2024-04-25T09:00:25.078 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-25T09:00:25.078 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 0, 2024-04-25T09:00:25.078 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 12323, 2024-04-25T09:00:25.078 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 12323, 2024-04-25T09:00:25.079 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 12323, 2024-04-25T09:00:25.079 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 12323, 2024-04-25T09:00:25.079 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-25T09:00:25.079 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T09:00:25.079 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T09:00:25.079 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T09:00:25.079 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:00:25.079 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.079 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-write_buf_throttle": { 2024-04-25T09:00:25.079 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-25T09:00:25.079 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 3758096384, 2024-04-25T09:00:25.079 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-25T09:00:25.079 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 3528, 2024-04-25T09:00:25.079 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 368552, 2024-04-25T09:00:25.080 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-25T09:00:25.080 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 3528, 2024-04-25T09:00:25.080 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-04-25T09:00:25.080 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-04-25T09:00:25.080 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 68, 2024-04-25T09:00:25.080 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 368552, 2024-04-25T09:00:25.080 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-25T09:00:25.080 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T09:00:25.080 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T09:00:25.080 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T09:00:25.080 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:00:25.080 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:25.080 INFO:teuthology.orchestra.run.smithi043.stdout: "throttle-write_buf_throttle-0x5623a11346e0": { 2024-04-25T09:00:25.081 INFO:teuthology.orchestra.run.smithi043.stdout: "val": 0, 2024-04-25T09:00:25.081 INFO:teuthology.orchestra.run.smithi043.stdout: "max": 3758096384, 2024-04-25T09:00:25.081 INFO:teuthology.orchestra.run.smithi043.stdout: "get_started": 0, 2024-04-25T09:00:25.081 INFO:teuthology.orchestra.run.smithi043.stdout: "get": 23712, 2024-04-25T09:00:25.081 INFO:teuthology.orchestra.run.smithi043.stdout: "get_sum": 78119748, 2024-04-25T09:00:25.081 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_fail": 0, 2024-04-25T09:00:25.081 INFO:teuthology.orchestra.run.smithi043.stdout: "get_or_fail_success": 23712, 2024-04-25T09:00:25.081 INFO:teuthology.orchestra.run.smithi043.stdout: "take": 0, 2024-04-25T09:00:25.081 INFO:teuthology.orchestra.run.smithi043.stdout: "take_sum": 0, 2024-04-25T09:00:25.081 INFO:teuthology.orchestra.run.smithi043.stdout: "put": 6507, 2024-04-25T09:00:25.081 INFO:teuthology.orchestra.run.smithi043.stdout: "put_sum": 78119748, 2024-04-25T09:00:25.081 INFO:teuthology.orchestra.run.smithi043.stdout: "wait": { 2024-04-25T09:00:25.081 INFO:teuthology.orchestra.run.smithi043.stdout: "avgcount": 0, 2024-04-25T09:00:25.082 INFO:teuthology.orchestra.run.smithi043.stdout: "sum": 0.000000000, 2024-04-25T09:00:25.082 INFO:teuthology.orchestra.run.smithi043.stdout: "avgtime": 0.000000000 2024-04-25T09:00:25.082 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:00:25.082 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:00:25.082 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:00:25.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.041+0000 7fcae2e76640 1 -- 172.21.15.43:0/3679477912 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcac006b4d0 msgr2=0x7fcac006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:25.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.041+0000 7fcae2e76640 1 --2- 172.21.15.43:0/3679477912 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcac006b4d0 0x7fcac006d990 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fcad0000f60 tx=0x7fcad0009040 comp rx=0 tx=0).stop 2024-04-25T09:00:25.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.041+0000 7fcae2e76640 1 -- 172.21.15.43:0/3679477912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcadc0cd120 msgr2=0x7fcadc156c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:25.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.041+0000 7fcae2e76640 1 --2- 172.21.15.43:0/3679477912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcadc0cd120 0x7fcadc156c10 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fcad402ac70 tx=0x7fcad4004a40 comp rx=0 tx=0).stop 2024-04-25T09:00:25.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.041+0000 7fcae2e76640 1 -- 172.21.15.43:0/3679477912 shutdown_connections 2024-04-25T09:00:25.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.041+0000 7fcae2e76640 1 --2- 172.21.15.43:0/3679477912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcadc0cd120 0x7fcadc156c10 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.041+0000 7fcae2e76640 1 --2- 172.21.15.43:0/3679477912 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcadc0aee80 0x7fcadc0a94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.041+0000 7fcae2e76640 1 --2- 172.21.15.43:0/3679477912 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcac006b4d0 0x7fcac006d990 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.041+0000 7fcae2e76640 1 --2- 172.21.15.43:0/3679477912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcadc06d7d0 0x7fcadc144100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.041+0000 7fcae2e76640 1 -- 172.21.15.43:0/3679477912 >> 172.21.15.43:0/3679477912 conn(0x7fcadc06ab30 msgr2=0x7fcadc06c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:25.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.041+0000 7fcae2e76640 1 -- 172.21.15.43:0/3679477912 shutdown_connections 2024-04-25T09:00:25.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.041+0000 7fcae2e76640 1 -- 172.21.15.43:0/3679477912 wait complete. 2024-04-25T09:00:25.085 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:00:25.085 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:00:25.129 DEBUG:teuthology.orchestra.run.smithi193:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:00:25.180 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:00:25.181 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T09:00:25.181 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:00:25.197 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:00:25.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.348+0000 7f978c29e640 1 Processor -- start 2024-04-25T09:00:25.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.348+0000 7f978c29e640 1 -- start start 2024-04-25T09:00:25.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.349+0000 7f978c29e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f97840cbfc0 0x7f978406dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.349+0000 7f978c29e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97840ccbb0 0x7f97840cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.349+0000 7f978c29e640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f978415f5e0 0x7f978415fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.349+0000 7f978c29e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97840cd550 con 0x7f97840cbfc0 2024-04-25T09:00:25.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.349+0000 7f978c29e640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97840b2780 con 0x7f978415f5e0 2024-04-25T09:00:25.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.349+0000 7f978c29e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97840b2a60 con 0x7f97840ccbb0 2024-04-25T09:00:25.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.349+0000 7f978a03a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f97840cbfc0 0x7f978406dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.349+0000 7f978a03a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f97840cbfc0 0x7f978406dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40040/0 (socket says 172.21.15.43:40040) 2024-04-25T09:00:25.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.349+0000 7f978a03a640 1 -- 172.21.15.43:0/2618634373 learned_addr learned my addr 172.21.15.43:0/2618634373 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:25.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.350+0000 7f978a03a640 1 -- 172.21.15.43:0/2618634373 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97840ccbb0 msgr2=0x7f97840cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:25.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.350+0000 7f9789839640 1 --2- 172.21.15.43:0/2618634373 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97840ccbb0 0x7f97840cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.350+0000 7f978a83b640 1 --2- 172.21.15.43:0/2618634373 >> v2:172.21.15.193:3300/0 conn(0x7f978415f5e0 0x7f978415fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.350+0000 7f978a03a640 1 --2- 172.21.15.43:0/2618634373 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97840ccbb0 0x7f97840cd010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.350+0000 7f978a03a640 1 -- 172.21.15.43:0/2618634373 >> v2:172.21.15.193:3300/0 conn(0x7f978415f5e0 msgr2=0x7f978415fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:25.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.350+0000 7f978a03a640 1 --2- 172.21.15.43:0/2618634373 >> v2:172.21.15.193:3300/0 conn(0x7f978415f5e0 0x7f978415fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.350+0000 7f978a03a640 1 -- 172.21.15.43:0/2618634373 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97840b2d40 con 0x7f97840cbfc0 2024-04-25T09:00:25.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.350+0000 7f978a03a640 1 --2- 172.21.15.43:0/2618634373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97840cbfc0 0x7f978406dc60 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f97840cd7b0 tx=0x7f977c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=689af3610756b095 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:25.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.350+0000 7f9789038640 1 -- 172.21.15.43:0/2618634373 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977c037070 con 0x7f97840cbfc0 2024-04-25T09:00:25.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.350+0000 7f9789038640 1 -- 172.21.15.43:0/2618634373 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f977c032440 con 0x7f97840cbfc0 2024-04-25T09:00:25.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.350+0000 7f9789038640 1 -- 172.21.15.43:0/2618634373 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977c032810 con 0x7f97840cbfc0 2024-04-25T09:00:25.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.351+0000 7f978c29e640 1 -- 172.21.15.43:0/2618634373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97840cbfc0 msgr2=0x7f978406dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:25.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.351+0000 7f978c29e640 1 --2- 172.21.15.43:0/2618634373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97840cbfc0 0x7f978406dc60 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f97840cd7b0 tx=0x7f977c02a1e0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.351+0000 7f978c29e640 1 -- 172.21.15.43:0/2618634373 shutdown_connections 2024-04-25T09:00:25.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.351+0000 7f978c29e640 1 --2- 172.21.15.43:0/2618634373 >> v2:172.21.15.193:3300/0 conn(0x7f978415f5e0 0x7f978415fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.351+0000 7f978c29e640 1 --2- 172.21.15.43:0/2618634373 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97840ccbb0 0x7f97840cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.351+0000 7f978c29e640 1 --2- 172.21.15.43:0/2618634373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97840cbfc0 0x7f978406dc60 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.351+0000 7f978c29e640 1 -- 172.21.15.43:0/2618634373 >> 172.21.15.43:0/2618634373 conn(0x7f97840b1480 msgr2=0x7f97840b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:25.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.351+0000 7f978c29e640 1 -- 172.21.15.43:0/2618634373 shutdown_connections 2024-04-25T09:00:25.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.351+0000 7f978c29e640 1 -- 172.21.15.43:0/2618634373 wait complete. 2024-04-25T09:00:25.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.352+0000 7f978c29e640 1 Processor -- start 2024-04-25T09:00:25.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.352+0000 7f978c29e640 1 -- start start 2024-04-25T09:00:25.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.353+0000 7f978c29e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97840cbfc0 0x7f978415b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.353+0000 7f978c29e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97840ccbb0 0x7f97840c46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.353+0000 7f978c29e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f978415f5e0 0x7f97840c4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.353+0000 7f978c29e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97840b35e0 con 0x7f97840ccbb0 2024-04-25T09:00:25.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.353+0000 7f978c29e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97840b3300 con 0x7f97840cbfc0 2024-04-25T09:00:25.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.353+0000 7f978c29e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f978406e1a0 con 0x7f978415f5e0 2024-04-25T09:00:25.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.353+0000 7f9789839640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97840ccbb0 0x7f97840c46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.353+0000 7f9789839640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97840ccbb0 0x7f97840c46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40050/0 (socket says 172.21.15.43:40050) 2024-04-25T09:00:25.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.353+0000 7f9789839640 1 -- 172.21.15.43:0/1835750315 learned_addr learned my addr 172.21.15.43:0/1835750315 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:25.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.353+0000 7f9789839640 1 -- 172.21.15.43:0/1835750315 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f978415f5e0 msgr2=0x7f97840c4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:25.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.353+0000 7f9789839640 1 --2- 172.21.15.43:0/1835750315 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f978415f5e0 0x7f97840c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.353+0000 7f9789839640 1 -- 172.21.15.43:0/1835750315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97840cbfc0 msgr2=0x7f978415b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:25.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.353+0000 7f9789839640 1 --2- 172.21.15.43:0/1835750315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97840cbfc0 0x7f978415b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.353+0000 7f9789839640 1 -- 172.21.15.43:0/1835750315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97840aa120 con 0x7f97840ccbb0 2024-04-25T09:00:25.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.354+0000 7f9789839640 1 --2- 172.21.15.43:0/1835750315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97840ccbb0 0x7f97840c46f0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f977400c820 tx=0x7f977400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:25.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.354+0000 7f977affd640 1 -- 172.21.15.43:0/1835750315 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9774015070 con 0x7f97840ccbb0 2024-04-25T09:00:25.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.354+0000 7f977affd640 1 -- 172.21.15.43:0/1835750315 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9774004220 con 0x7f97840ccbb0 2024-04-25T09:00:25.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.354+0000 7f978c29e640 1 -- 172.21.15.43:0/1835750315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97840aa400 con 0x7f97840ccbb0 2024-04-25T09:00:25.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.354+0000 7f977affd640 1 -- 172.21.15.43:0/1835750315 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977400ce10 con 0x7f97840ccbb0 2024-04-25T09:00:25.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.354+0000 7f978c29e640 1 -- 172.21.15.43:0/1835750315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97840aa9c0 con 0x7f97840ccbb0 2024-04-25T09:00:25.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.355+0000 7f978c29e640 1 -- 172.21.15.43:0/1835750315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9784074ef0 con 0x7f97840ccbb0 2024-04-25T09:00:25.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.355+0000 7f977affd640 1 -- 172.21.15.43:0/1835750315 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9774007830 con 0x7f97840ccbb0 2024-04-25T09:00:25.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.356+0000 7f977affd640 1 --2- 172.21.15.43:0/1835750315 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f976806b4d0 0x7f976806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.356+0000 7f977affd640 1 -- 172.21.15.43:0/1835750315 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f977408a120 con 0x7f97840ccbb0 2024-04-25T09:00:25.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.356+0000 7f978a03a640 1 --2- 172.21.15.43:0/1835750315 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f976806b4d0 0x7f976806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.357+0000 7f978a03a640 1 --2- 172.21.15.43:0/1835750315 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f976806b4d0 0x7f976806d990 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f977c034000 tx=0x7f977c02c000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:25.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.360+0000 7f977affd640 1 -- 172.21.15.43:0/1835750315 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9774058eb0 con 0x7f97840ccbb0 2024-04-25T09:00:25.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.542+0000 7f978c29e640 1 -- 172.21.15.43:0/1835750315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f97840af0d0 con 0x7f97840ccbb0 2024-04-25T09:00:25.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.543+0000 7f977affd640 1 -- 172.21.15.43:0/1835750315 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+16795 (secure 0 0 0) 0x7f97740587b0 con 0x7f97840ccbb0 2024-04-25T09:00:25.532 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:00:25.532 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":77,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:00:02.408776+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:58:44.188924+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T08:58:44.877113+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":76,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.1","mappings":[{"from":6,"to":0}]},{"pgid":"11.2e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:00:25.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.545+0000 7f978c29e640 1 -- 172.21.15.43:0/1835750315 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f976806b4d0 msgr2=0x7f976806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:25.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.545+0000 7f978c29e640 1 --2- 172.21.15.43:0/1835750315 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f976806b4d0 0x7f976806d990 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f977c034000 tx=0x7f977c02c000 comp rx=0 tx=0).stop 2024-04-25T09:00:25.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.545+0000 7f978c29e640 1 -- 172.21.15.43:0/1835750315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97840ccbb0 msgr2=0x7f97840c46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:25.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.545+0000 7f978c29e640 1 --2- 172.21.15.43:0/1835750315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97840ccbb0 0x7f97840c46f0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f977400c820 tx=0x7f977400cce0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.545+0000 7f978c29e640 1 -- 172.21.15.43:0/1835750315 shutdown_connections 2024-04-25T09:00:25.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.545+0000 7f978c29e640 1 --2- 172.21.15.43:0/1835750315 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f978415f5e0 0x7f97840c4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.545+0000 7f978c29e640 1 --2- 172.21.15.43:0/1835750315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97840ccbb0 0x7f97840c46f0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.545+0000 7f978c29e640 1 --2- 172.21.15.43:0/1835750315 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f976806b4d0 0x7f976806d990 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.545+0000 7f978c29e640 1 --2- 172.21.15.43:0/1835750315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97840cbfc0 0x7f978415b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.545+0000 7f978c29e640 1 -- 172.21.15.43:0/1835750315 >> 172.21.15.43:0/1835750315 conn(0x7f97840b1480 msgr2=0x7f978415ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:25.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.545+0000 7f978c29e640 1 -- 172.21.15.43:0/1835750315 shutdown_connections 2024-04-25T09:00:25.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.545+0000 7f978c29e640 1 -- 172.21.15.43:0/1835750315 wait complete. 2024-04-25T09:00:25.545 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:58:44.188924+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '72', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:58:44.877113+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 76, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:00:25.545 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:00:25.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.660+0000 7f88608fc640 1 Processor -- start 2024-04-25T09:00:25.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.660+0000 7f88608fc640 1 -- start start 2024-04-25T09:00:25.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.661+0000 7f88608fc640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f885c0d04a0 0x7f885c15ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.661+0000 7f88608fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f885c15dec0 0x7f885c0d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.661+0000 7f88608fc640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f885c0ce8c0 0x7f885c1617a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.661+0000 7f88608fc640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f885c0d0dc0 con 0x7f885c15dec0 2024-04-25T09:00:25.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.661+0000 7f88608fc640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f885c0cf110 con 0x7f885c0d04a0 2024-04-25T09:00:25.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.661+0000 7f88608fc640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f885c0cf3f0 con 0x7f885c0ce8c0 2024-04-25T09:00:25.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.661+0000 7f8853fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f885c15dec0 0x7f885c0d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.661+0000 7f8853fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f885c15dec0 0x7f885c0d0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40054/0 (socket says 172.21.15.43:40054) 2024-04-25T09:00:25.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.661+0000 7f8853fff640 1 -- 172.21.15.43:0/3096732542 learned_addr learned my addr 172.21.15.43:0/3096732542 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:25.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.661+0000 7f8853fff640 1 -- 172.21.15.43:0/3096732542 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f885c0ce8c0 msgr2=0x7f885c1617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:25.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.661+0000 7f885ad9d640 1 --2- 172.21.15.43:0/3096732542 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f885c0ce8c0 0x7f885c1617a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.661+0000 7f885a59c640 1 --2- 172.21.15.43:0/3096732542 >> v2:172.21.15.193:3300/0 conn(0x7f885c0d04a0 0x7f885c15ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.661+0000 7f8853fff640 1 --2- 172.21.15.43:0/3096732542 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f885c0ce8c0 0x7f885c1617a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.661+0000 7f8853fff640 1 -- 172.21.15.43:0/3096732542 >> v2:172.21.15.193:3300/0 conn(0x7f885c0d04a0 msgr2=0x7f885c15ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:25.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.661+0000 7f8853fff640 1 --2- 172.21.15.43:0/3096732542 >> v2:172.21.15.193:3300/0 conn(0x7f885c0d04a0 0x7f885c15ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.661+0000 7f8853fff640 1 -- 172.21.15.43:0/3096732542 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f885c0cf6d0 con 0x7f885c15dec0 2024-04-25T09:00:25.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.661+0000 7f885ad9d640 1 --2- 172.21.15.43:0/3096732542 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f885c0ce8c0 0x7f885c1617a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:25.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.661+0000 7f885a59c640 1 --2- 172.21.15.43:0/3096732542 >> v2:172.21.15.193:3300/0 conn(0x7f885c0d04a0 0x7f885c15ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:25.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.662+0000 7f8853fff640 1 --2- 172.21.15.43:0/3096732542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f885c15dec0 0x7f885c0d0880 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f885c0b6690 tx=0x7f884402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f987248a4b236e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:25.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.662+0000 7f8859d9b640 1 -- 172.21.15.43:0/3096732542 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8844037070 con 0x7f885c15dec0 2024-04-25T09:00:25.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.662+0000 7f8859d9b640 1 -- 172.21.15.43:0/3096732542 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8844032440 con 0x7f885c15dec0 2024-04-25T09:00:25.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.662+0000 7f8859d9b640 1 -- 172.21.15.43:0/3096732542 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8844032810 con 0x7f885c15dec0 2024-04-25T09:00:25.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.663+0000 7f88608fc640 1 -- 172.21.15.43:0/3096732542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f885c15dec0 msgr2=0x7f885c0d0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:25.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.663+0000 7f88608fc640 1 --2- 172.21.15.43:0/3096732542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f885c15dec0 0x7f885c0d0880 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f885c0b6690 tx=0x7f884402a1e0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.663+0000 7f88608fc640 1 -- 172.21.15.43:0/3096732542 shutdown_connections 2024-04-25T09:00:25.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.663+0000 7f88608fc640 1 --2- 172.21.15.43:0/3096732542 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f885c0ce8c0 0x7f885c1617a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.663+0000 7f88608fc640 1 --2- 172.21.15.43:0/3096732542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f885c15dec0 0x7f885c0d0880 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.663+0000 7f88608fc640 1 --2- 172.21.15.43:0/3096732542 >> v2:172.21.15.193:3300/0 conn(0x7f885c0d04a0 0x7f885c15ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.663+0000 7f88608fc640 1 -- 172.21.15.43:0/3096732542 >> 172.21.15.43:0/3096732542 conn(0x7f885c09cd40 msgr2=0x7f885c0d3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:25.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.663+0000 7f88608fc640 1 -- 172.21.15.43:0/3096732542 shutdown_connections 2024-04-25T09:00:25.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.663+0000 7f88608fc640 1 -- 172.21.15.43:0/3096732542 wait complete. 2024-04-25T09:00:25.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.664+0000 7f88608fc640 1 Processor -- start 2024-04-25T09:00:25.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.664+0000 7f88608fc640 1 -- start start 2024-04-25T09:00:25.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.665+0000 7f88608fc640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f885c0ce8c0 0x7f885c157930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.665+0000 7f88608fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f885c0d04a0 0x7f885c15b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.665+0000 7f88608fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f885c15dec0 0x7f885c15c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.665+0000 7f88608fc640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885c0ced10 con 0x7f885c15dec0 2024-04-25T09:00:25.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.665+0000 7f88608fc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f885c0cfc90 con 0x7f885c0d04a0 2024-04-25T09:00:25.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.665+0000 7f88608fc640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f885c0cee90 con 0x7f885c0ce8c0 2024-04-25T09:00:25.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.665+0000 7f885ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f885c15dec0 0x7f885c15c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.665+0000 7f885ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f885c15dec0 0x7f885c15c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40066/0 (socket says 172.21.15.43:40066) 2024-04-25T09:00:25.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.665+0000 7f885ad9d640 1 -- 172.21.15.43:0/3899131164 learned_addr learned my addr 172.21.15.43:0/3899131164 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:25.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.665+0000 7f885ad9d640 1 -- 172.21.15.43:0/3899131164 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f885c0ce8c0 msgr2=0x7f885c157930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:25.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.665+0000 7f885ad9d640 1 --2- 172.21.15.43:0/3899131164 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f885c0ce8c0 0x7f885c157930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.665+0000 7f8853fff640 1 --2- 172.21.15.43:0/3899131164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f885c0d04a0 0x7f885c15b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.665+0000 7f885ad9d640 1 -- 172.21.15.43:0/3899131164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f885c0d04a0 msgr2=0x7f885c15b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:25.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.665+0000 7f885ad9d640 1 --2- 172.21.15.43:0/3899131164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f885c0d04a0 0x7f885c15b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.665+0000 7f885ad9d640 1 -- 172.21.15.43:0/3899131164 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f885c0b6830 con 0x7f885c15dec0 2024-04-25T09:00:25.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.665+0000 7f885ad9d640 1 --2- 172.21.15.43:0/3899131164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f885c15dec0 0x7f885c15c1d0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f885400cd10 tx=0x7f885400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:25.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.665+0000 7f8852ffd640 1 -- 172.21.15.43:0/3899131164 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885400f680 con 0x7f885c15dec0 2024-04-25T09:00:25.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.665+0000 7f8852ffd640 1 -- 172.21.15.43:0/3899131164 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f885400f840 con 0x7f885c15dec0 2024-04-25T09:00:25.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.665+0000 7f88608fc640 1 -- 172.21.15.43:0/3899131164 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f885c0b6b10 con 0x7f885c15dec0 2024-04-25T09:00:25.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.665+0000 7f8852ffd640 1 -- 172.21.15.43:0/3899131164 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f885400fc30 con 0x7f885c15dec0 2024-04-25T09:00:25.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.665+0000 7f88608fc640 1 -- 172.21.15.43:0/3899131164 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f885c0b70d0 con 0x7f885c15dec0 2024-04-25T09:00:25.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.666+0000 7f88608fc640 1 -- 172.21.15.43:0/3899131164 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f885c074f60 con 0x7f885c15dec0 2024-04-25T09:00:25.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.667+0000 7f8852ffd640 1 -- 172.21.15.43:0/3899131164 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f88540213f0 con 0x7f885c15dec0 2024-04-25T09:00:25.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.667+0000 7f8852ffd640 1 --2- 172.21.15.43:0/3899131164 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f883c06b4d0 0x7f883c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.667+0000 7f8852ffd640 1 -- 172.21.15.43:0/3899131164 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f885408b050 con 0x7f885c15dec0 2024-04-25T09:00:25.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.668+0000 7f885a59c640 1 --2- 172.21.15.43:0/3899131164 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f883c06b4d0 0x7f883c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.668+0000 7f885a59c640 1 --2- 172.21.15.43:0/3899131164 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f883c06b4d0 0x7f883c06d990 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f884c006fd0 tx=0x7f884c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:25.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.671+0000 7f8852ffd640 1 -- 172.21.15.43:0/3899131164 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8854014030 con 0x7f885c15dec0 2024-04-25T09:00:25.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.851+0000 7f88608fc640 1 -- 172.21.15.43:0/3899131164 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f885c0c6d40 con 0x7f885c15dec0 2024-04-25T09:00:25.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.851+0000 7f8852ffd640 1 -- 172.21.15.43:0/3899131164 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8854059780 con 0x7f885c15dec0 2024-04-25T09:00:25.840 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T09:00:25.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.853+0000 7f88608fc640 1 -- 172.21.15.43:0/3899131164 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f883c06b4d0 msgr2=0x7f883c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:25.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.853+0000 7f88608fc640 1 --2- 172.21.15.43:0/3899131164 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f883c06b4d0 0x7f883c06d990 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f884c006fd0 tx=0x7f884c008040 comp rx=0 tx=0).stop 2024-04-25T09:00:25.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.853+0000 7f88608fc640 1 -- 172.21.15.43:0/3899131164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f885c15dec0 msgr2=0x7f885c15c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:25.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.853+0000 7f88608fc640 1 --2- 172.21.15.43:0/3899131164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f885c15dec0 0x7f885c15c1d0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f885400cd10 tx=0x7f885400a650 comp rx=0 tx=0).stop 2024-04-25T09:00:25.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.853+0000 7f88608fc640 1 -- 172.21.15.43:0/3899131164 shutdown_connections 2024-04-25T09:00:25.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.853+0000 7f88608fc640 1 --2- 172.21.15.43:0/3899131164 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f885c15dec0 0x7f885c15c1d0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.853+0000 7f88608fc640 1 --2- 172.21.15.43:0/3899131164 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f883c06b4d0 0x7f883c06d990 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.853+0000 7f88608fc640 1 --2- 172.21.15.43:0/3899131164 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f885c0d04a0 0x7f885c15b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.853+0000 7f88608fc640 1 --2- 172.21.15.43:0/3899131164 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f885c0ce8c0 0x7f885c157930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.853+0000 7f88608fc640 1 -- 172.21.15.43:0/3899131164 >> 172.21.15.43:0/3899131164 conn(0x7f885c09cd40 msgr2=0x7f885c06b130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:25.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.853+0000 7f88608fc640 1 -- 172.21.15.43:0/3899131164 shutdown_connections 2024-04-25T09:00:25.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.853+0000 7f88608fc640 1 -- 172.21.15.43:0/3899131164 wait complete. 2024-04-25T09:00:25.853 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:00:25.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.967+0000 7f44a54e1640 1 Processor -- start 2024-04-25T09:00:25.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.967+0000 7f44a54e1640 1 -- start start 2024-04-25T09:00:25.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.968+0000 7f44a54e1640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44a00d04a0 0x7f44a015ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.968+0000 7f44a54e1640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f44a015dec0 0x7f44a00d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.968+0000 7f44a54e1640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44a00d0dc0 con 0x7f44a00ce8c0 2024-04-25T09:00:25.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.968+0000 7f44a54e1640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44a00cf110 con 0x7f44a015dec0 2024-04-25T09:00:25.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.968+0000 7f44a54e1640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a00cf3f0 con 0x7f44a00d04a0 2024-04-25T09:00:25.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.968+0000 7f449e7fc640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f44a015dec0 0x7f44a00d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.968+0000 7f449effd640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44a00d04a0 0x7f44a015ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.968+0000 7f449effd640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44a00d04a0 0x7f44a015ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:37696/0 (socket says 172.21.15.43:37696) 2024-04-25T09:00:25.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.968+0000 7f449e7fc640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f44a015dec0 0x7f44a00d0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:42526/0 (socket says 172.21.15.43:42526) 2024-04-25T09:00:25.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.968+0000 7f449effd640 1 -- 172.21.15.43:0/2965303116 learned_addr learned my addr 172.21.15.43:0/2965303116 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:25.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.969+0000 7f449e7fc640 1 -- 172.21.15.43:0/2965303116 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44a00d04a0 msgr2=0x7f44a015ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:25.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.969+0000 7f449e7fc640 1 --2- 172.21.15.43:0/2965303116 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44a00d04a0 0x7f44a015ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.969+0000 7f449e7fc640 1 -- 172.21.15.43:0/2965303116 >> v1:172.21.15.193:6789/0 conn(0x7f44a00ce8c0 legacy=0x7f44a01617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:25.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.969+0000 7f449e7fc640 1 -- 172.21.15.43:0/2965303116 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a00cf6d0 con 0x7f44a015dec0 2024-04-25T09:00:25.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.969+0000 7f449effd640 1 --2- 172.21.15.43:0/2965303116 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44a00d04a0 0x7f44a015ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:00:25.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.970+0000 7f449e7fc640 1 --2- 172.21.15.43:0/2965303116 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44a015dec0 0x7f44a00d0880 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f44880097a0 tx=0x7f448802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94aaf6d9f5bf4d0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:25.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.970+0000 7f449dffb640 1 -- 172.21.15.43:0/2965303116 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4488037070 con 0x7f44a015dec0 2024-04-25T09:00:25.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.970+0000 7f449dffb640 1 -- 172.21.15.43:0/2965303116 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4488032440 con 0x7f44a015dec0 2024-04-25T09:00:25.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.970+0000 7f449dffb640 1 -- 172.21.15.43:0/2965303116 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4488032810 con 0x7f44a015dec0 2024-04-25T09:00:25.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.970+0000 7f44a54e1640 1 -- 172.21.15.43:0/2965303116 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44a015dec0 msgr2=0x7f44a00d0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:25.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.970+0000 7f44a54e1640 1 --2- 172.21.15.43:0/2965303116 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44a015dec0 0x7f44a00d0880 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f44880097a0 tx=0x7f448802a180 comp rx=0 tx=0).stop 2024-04-25T09:00:25.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.970+0000 7f44a54e1640 1 -- 172.21.15.43:0/2965303116 shutdown_connections 2024-04-25T09:00:25.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.970+0000 7f44a54e1640 1 --2- 172.21.15.43:0/2965303116 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44a015dec0 0x7f44a00d0880 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.970+0000 7f44a54e1640 1 --2- 172.21.15.43:0/2965303116 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44a00d04a0 0x7f44a015ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.970+0000 7f44a54e1640 1 -- 172.21.15.43:0/2965303116 >> 172.21.15.43:0/2965303116 conn(0x7f44a009cd40 msgr2=0x7f44a00d3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:25.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.970+0000 7f44a54e1640 1 -- 172.21.15.43:0/2965303116 shutdown_connections 2024-04-25T09:00:25.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.970+0000 7f44a54e1640 1 -- 172.21.15.43:0/2965303116 wait complete. 2024-04-25T09:00:25.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.972+0000 7f44a54e1640 1 Processor -- start 2024-04-25T09:00:25.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.972+0000 7f44a54e1640 1 -- start start 2024-04-25T09:00:25.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.972+0000 7f44a54e1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44a00ce8c0 0x7f44a01592e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.972+0000 7f44a54e1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44a00d04a0 0x7f44a01e75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.972+0000 7f44a54e1640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44a015dec0 0x7f44a00cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.972+0000 7f44a54e1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a00cecf0 con 0x7f44a00d04a0 2024-04-25T09:00:25.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.972+0000 7f44a54e1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a00cfc90 con 0x7f44a00ce8c0 2024-04-25T09:00:25.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.972+0000 7f44a54e1640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a00cee70 con 0x7f44a015dec0 2024-04-25T09:00:25.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.973+0000 7f449e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44a00d04a0 0x7f44a01e75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.973+0000 7f449e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44a00d04a0 0x7f44a01e75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40070/0 (socket says 172.21.15.43:40070) 2024-04-25T09:00:25.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.973+0000 7f449e7fc640 1 -- 172.21.15.43:0/175833241 learned_addr learned my addr 172.21.15.43:0/175833241 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:25.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.973+0000 7f449effd640 1 --2- 172.21.15.43:0/175833241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44a00ce8c0 0x7f44a01592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.973+0000 7f449f7fe640 1 --2- 172.21.15.43:0/175833241 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44a015dec0 0x7f44a00cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.973+0000 7f449e7fc640 1 -- 172.21.15.43:0/175833241 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44a015dec0 msgr2=0x7f44a00cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:25.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.973+0000 7f449e7fc640 1 --2- 172.21.15.43:0/175833241 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44a015dec0 0x7f44a00cc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.973+0000 7f449e7fc640 1 -- 172.21.15.43:0/175833241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44a00ce8c0 msgr2=0x7f44a01592e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:25.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.973+0000 7f449e7fc640 1 --2- 172.21.15.43:0/175833241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44a00ce8c0 0x7f44a01592e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:25.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.973+0000 7f449e7fc640 1 -- 172.21.15.43:0/175833241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a01e8830 con 0x7f44a00d04a0 2024-04-25T09:00:25.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.973+0000 7f449e7fc640 1 --2- 172.21.15.43:0/175833241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44a00d04a0 0x7f44a01e75a0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f4488035040 tx=0x7f44880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:25.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.973+0000 7f44977fe640 1 -- 172.21.15.43:0/175833241 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4488037040 con 0x7f44a00d04a0 2024-04-25T09:00:25.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.973+0000 7f44977fe640 1 -- 172.21.15.43:0/175833241 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4488030500 con 0x7f44a00d04a0 2024-04-25T09:00:25.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.973+0000 7f44a54e1640 1 -- 172.21.15.43:0/175833241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44a01e7cb0 con 0x7f44a00d04a0 2024-04-25T09:00:25.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.973+0000 7f44977fe640 1 -- 172.21.15.43:0/175833241 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4488030910 con 0x7f44a00d04a0 2024-04-25T09:00:25.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.973+0000 7f44a54e1640 1 -- 172.21.15.43:0/175833241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44a01e8270 con 0x7f44a00d04a0 2024-04-25T09:00:25.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.974+0000 7f44a54e1640 1 -- 172.21.15.43:0/175833241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44a0074f60 con 0x7f44a00d04a0 2024-04-25T09:00:25.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.975+0000 7f44977fe640 1 -- 172.21.15.43:0/175833241 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4488044020 con 0x7f44a00d04a0 2024-04-25T09:00:25.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.975+0000 7f44977fe640 1 --2- 172.21.15.43:0/175833241 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f448006b4d0 0x7f448006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:25.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.975+0000 7f44977fe640 1 -- 172.21.15.43:0/175833241 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f44880aa460 con 0x7f44a00d04a0 2024-04-25T09:00:25.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.975+0000 7f449effd640 1 --2- 172.21.15.43:0/175833241 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f448006b4d0 0x7f448006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:25.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.976+0000 7f449effd640 1 --2- 172.21.15.43:0/175833241 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f448006b4d0 0x7f448006d990 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4498006ca0 tx=0x7f4498009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:25.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:25.979+0000 7f44977fe640 1 -- 172.21.15.43:0/175833241 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4488078bb0 con 0x7f44a00d04a0 2024-04-25T09:00:26.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.162+0000 7f44a54e1640 1 -- 172.21.15.43:0/175833241 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f44a008daf0 con 0x7f44a00d04a0 2024-04-25T09:00:26.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.163+0000 7f44977fe640 1 -- 172.21.15.43:0/175833241 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) v1 ==== 104+0+11 (secure 0 0 0) 0x7f448807cc50 con 0x7f44a00d04a0 2024-04-25T09:00:26.152 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T09:00:26.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.164+0000 7f44a54e1640 1 -- 172.21.15.43:0/175833241 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f448006b4d0 msgr2=0x7f448006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:26.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.164+0000 7f44a54e1640 1 --2- 172.21.15.43:0/175833241 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f448006b4d0 0x7f448006d990 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4498006ca0 tx=0x7f4498009240 comp rx=0 tx=0).stop 2024-04-25T09:00:26.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.164+0000 7f44a54e1640 1 -- 172.21.15.43:0/175833241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44a00d04a0 msgr2=0x7f44a01e75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:26.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.164+0000 7f44a54e1640 1 --2- 172.21.15.43:0/175833241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44a00d04a0 0x7f44a01e75a0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f4488035040 tx=0x7f44880046c0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.164+0000 7f44a54e1640 1 -- 172.21.15.43:0/175833241 shutdown_connections 2024-04-25T09:00:26.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.164+0000 7f44a54e1640 1 --2- 172.21.15.43:0/175833241 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44a015dec0 0x7f44a00cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.164+0000 7f44a54e1640 1 --2- 172.21.15.43:0/175833241 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44a00d04a0 0x7f44a01e75a0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.164+0000 7f44a54e1640 1 --2- 172.21.15.43:0/175833241 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f448006b4d0 0x7f448006d990 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.164+0000 7f44a54e1640 1 --2- 172.21.15.43:0/175833241 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44a00ce8c0 0x7f44a01592e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.164+0000 7f44a54e1640 1 -- 172.21.15.43:0/175833241 >> 172.21.15.43:0/175833241 conn(0x7f44a009cd40 msgr2=0x7f44a00d16a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:26.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.164+0000 7f44a54e1640 1 -- 172.21.15.43:0/175833241 shutdown_connections 2024-04-25T09:00:26.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.164+0000 7f44a54e1640 1 -- 172.21.15.43:0/175833241 wait complete. 2024-04-25T09:00:26.164 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:00:26.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.278+0000 7f02c8be2640 1 Processor -- start 2024-04-25T09:00:26.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.278+0000 7f02c8be2640 1 -- start start 2024-04-25T09:00:26.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.279+0000 7f02c8be2640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f02c406fa90 0x7f02c40d0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:26.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.279+0000 7f02c8be2640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f02c40cf5d0 0x7f02c40d1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:26.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.279+0000 7f02c8be2640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02c406fe70 con 0x7f02c415cce0 2024-04-25T09:00:26.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.279+0000 7f02c8be2640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02c40cfaa0 con 0x7f02c40cf5d0 2024-04-25T09:00:26.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.279+0000 7f02c8be2640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02c415c200 con 0x7f02c406fa90 2024-04-25T09:00:26.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.280+0000 7f02c2d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f02c40cf5d0 0x7f02c40d1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:26.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.280+0000 7f02c2d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f02c40cf5d0 0x7f02c40d1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40082/0 (socket says 172.21.15.43:40082) 2024-04-25T09:00:26.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.280+0000 7f02c2d9d640 1 -- 172.21.15.43:0/287101031 learned_addr learned my addr 172.21.15.43:0/287101031 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:26.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.280+0000 7f02c259c640 1 --2- 172.21.15.43:0/287101031 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f02c406fa90 0x7f02c40d0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:26.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.280+0000 7f02c2d9d640 1 -- 172.21.15.43:0/287101031 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f02c406fa90 msgr2=0x7f02c40d0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:26.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.280+0000 7f02c2d9d640 1 --2- 172.21.15.43:0/287101031 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f02c406fa90 0x7f02c40d0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.280+0000 7f02c2d9d640 1 -- 172.21.15.43:0/287101031 >> v1:172.21.15.193:6789/0 conn(0x7f02c415cce0 legacy=0x7f02c40ceec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:26.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.280+0000 7f02c2d9d640 1 -- 172.21.15.43:0/287101031 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c415c4e0 con 0x7f02c40cf5d0 2024-04-25T09:00:26.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.280+0000 7f02c259c640 1 --2- 172.21.15.43:0/287101031 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f02c406fa90 0x7f02c40d0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:00:26.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.281+0000 7f02c2d9d640 1 --2- 172.21.15.43:0/287101031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02c40cf5d0 0x7f02c40d1030 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f02b4009e50 tx=0x7f02b402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d353c5e0f47cbf4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:26.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.281+0000 7f02c159a640 1 -- 172.21.15.43:0/287101031 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b4036070 con 0x7f02c40cf5d0 2024-04-25T09:00:26.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.281+0000 7f02c159a640 1 -- 172.21.15.43:0/287101031 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02b4031600 con 0x7f02c40cf5d0 2024-04-25T09:00:26.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.281+0000 7f02c159a640 1 -- 172.21.15.43:0/287101031 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b40319d0 con 0x7f02c40cf5d0 2024-04-25T09:00:26.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.281+0000 7f02c8be2640 1 -- 172.21.15.43:0/287101031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02c40cf5d0 msgr2=0x7f02c40d1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:26.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.281+0000 7f02c8be2640 1 --2- 172.21.15.43:0/287101031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02c40cf5d0 0x7f02c40d1030 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f02b4009e50 tx=0x7f02b402a710 comp rx=0 tx=0).stop 2024-04-25T09:00:26.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.281+0000 7f02c8be2640 1 -- 172.21.15.43:0/287101031 shutdown_connections 2024-04-25T09:00:26.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.281+0000 7f02c8be2640 1 --2- 172.21.15.43:0/287101031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02c40cf5d0 0x7f02c40d1030 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.281+0000 7f02c8be2640 1 --2- 172.21.15.43:0/287101031 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f02c406fa90 0x7f02c40d0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.281+0000 7f02c8be2640 1 -- 172.21.15.43:0/287101031 >> 172.21.15.43:0/287101031 conn(0x7f02c406c060 msgr2=0x7f02c406c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:26.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.281+0000 7f02c8be2640 1 -- 172.21.15.43:0/287101031 shutdown_connections 2024-04-25T09:00:26.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.281+0000 7f02c8be2640 1 -- 172.21.15.43:0/287101031 wait complete. 2024-04-25T09:00:26.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.283+0000 7f02c8be2640 1 Processor -- start 2024-04-25T09:00:26.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.283+0000 7f02c8be2640 1 -- start start 2024-04-25T09:00:26.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.283+0000 7f02c8be2640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f02c406fa90 0x7f02c40b4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:26.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.283+0000 7f02c8be2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02c40cf5d0 0x7f02c414d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:26.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.283+0000 7f02c8be2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02c415cce0 0x7f02c40aed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:26.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.283+0000 7f02c8be2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c406e240 con 0x7f02c40cf5d0 2024-04-25T09:00:26.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.283+0000 7f02c8be2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02c406e0c0 con 0x7f02c415cce0 2024-04-25T09:00:26.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.283+0000 7f02c8be2640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02c406e3c0 con 0x7f02c406fa90 2024-04-25T09:00:26.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.283+0000 7f02c1d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02c40cf5d0 0x7f02c414d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:26.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.284+0000 7f02c1d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02c40cf5d0 0x7f02c414d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40086/0 (socket says 172.21.15.43:40086) 2024-04-25T09:00:26.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.284+0000 7f02c1d9b640 1 -- 172.21.15.43:0/495710800 learned_addr learned my addr 172.21.15.43:0/495710800 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:26.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.284+0000 7f02c259c640 1 --2- 172.21.15.43:0/495710800 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f02c406fa90 0x7f02c40b4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:26.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.284+0000 7f02c2d9d640 1 --2- 172.21.15.43:0/495710800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02c415cce0 0x7f02c40aed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:26.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.284+0000 7f02c1d9b640 1 -- 172.21.15.43:0/495710800 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f02c406fa90 msgr2=0x7f02c40b4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:26.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.284+0000 7f02c1d9b640 1 --2- 172.21.15.43:0/495710800 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f02c406fa90 0x7f02c40b4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.284+0000 7f02c1d9b640 1 -- 172.21.15.43:0/495710800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02c415cce0 msgr2=0x7f02c40aed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:26.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.284+0000 7f02c1d9b640 1 --2- 172.21.15.43:0/495710800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02c415cce0 0x7f02c40aed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.284+0000 7f02c1d9b640 1 -- 172.21.15.43:0/495710800 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c40b3780 con 0x7f02c40cf5d0 2024-04-25T09:00:26.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.284+0000 7f02c1d9b640 1 --2- 172.21.15.43:0/495710800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02c40cf5d0 0x7f02c414d340 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f02ac00c3c0 tx=0x7f02ac00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:26.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.284+0000 7f02baffd640 1 -- 172.21.15.43:0/495710800 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ac0052d0 con 0x7f02c40cf5d0 2024-04-25T09:00:26.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.284+0000 7f02baffd640 1 -- 172.21.15.43:0/495710800 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02ac005490 con 0x7f02c40cf5d0 2024-04-25T09:00:26.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.284+0000 7f02baffd640 1 -- 172.21.15.43:0/495710800 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ac007260 con 0x7f02c40cf5d0 2024-04-25T09:00:26.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.284+0000 7f02c8be2640 1 -- 172.21.15.43:0/495710800 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02c40b3a60 con 0x7f02c40cf5d0 2024-04-25T09:00:26.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.284+0000 7f02c8be2640 1 -- 172.21.15.43:0/495710800 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02c40b4020 con 0x7f02c40cf5d0 2024-04-25T09:00:26.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.285+0000 7f02c8be2640 1 -- 172.21.15.43:0/495710800 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02c4074f60 con 0x7f02c40cf5d0 2024-04-25T09:00:26.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.286+0000 7f02baffd640 1 -- 172.21.15.43:0/495710800 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f02ac007400 con 0x7f02c40cf5d0 2024-04-25T09:00:26.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.286+0000 7f02baffd640 1 --2- 172.21.15.43:0/495710800 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f02a406b4d0 0x7f02a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:26.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.286+0000 7f02baffd640 1 -- 172.21.15.43:0/495710800 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f02ac089600 con 0x7f02c40cf5d0 2024-04-25T09:00:26.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.286+0000 7f02c259c640 1 --2- 172.21.15.43:0/495710800 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f02a406b4d0 0x7f02a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:26.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.287+0000 7f02c259c640 1 --2- 172.21.15.43:0/495710800 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f02a406b4d0 0x7f02a406d990 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f02c40ccf40 tx=0x7f02bc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:26.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.290+0000 7f02baffd640 1 -- 172.21.15.43:0/495710800 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f02ac057e60 con 0x7f02c40cf5d0 2024-04-25T09:00:26.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.473+0000 7f02c8be2640 1 -- 172.21.15.43:0/495710800 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f02c4154cf0 con 0x7f02c40cf5d0 2024-04-25T09:00:26.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.474+0000 7f02baffd640 1 -- 172.21.15.43:0/495710800 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) v1 ==== 100+0+11 (secure 0 0 0) 0x7f02ac05ce20 con 0x7f02c40cf5d0 2024-04-25T09:00:26.463 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T09:00:26.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.475+0000 7f02c8be2640 1 -- 172.21.15.43:0/495710800 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f02a406b4d0 msgr2=0x7f02a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:26.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.475+0000 7f02c8be2640 1 --2- 172.21.15.43:0/495710800 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f02a406b4d0 0x7f02a406d990 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f02c40ccf40 tx=0x7f02bc008040 comp rx=0 tx=0).stop 2024-04-25T09:00:26.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.475+0000 7f02c8be2640 1 -- 172.21.15.43:0/495710800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02c40cf5d0 msgr2=0x7f02c414d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:26.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.475+0000 7f02c8be2640 1 --2- 172.21.15.43:0/495710800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02c40cf5d0 0x7f02c414d340 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f02ac00c3c0 tx=0x7f02ac00c880 comp rx=0 tx=0).stop 2024-04-25T09:00:26.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.476+0000 7f02c8be2640 1 -- 172.21.15.43:0/495710800 shutdown_connections 2024-04-25T09:00:26.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.476+0000 7f02c8be2640 1 --2- 172.21.15.43:0/495710800 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f02a406b4d0 0x7f02a406d990 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.476+0000 7f02c8be2640 1 --2- 172.21.15.43:0/495710800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02c415cce0 0x7f02c40aed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.476+0000 7f02c8be2640 1 --2- 172.21.15.43:0/495710800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02c40cf5d0 0x7f02c414d340 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.476+0000 7f02c8be2640 1 --2- 172.21.15.43:0/495710800 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f02c406fa90 0x7f02c40b4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.476+0000 7f02c8be2640 1 -- 172.21.15.43:0/495710800 >> 172.21.15.43:0/495710800 conn(0x7f02c406c060 msgr2=0x7f02c40ce5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:26.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.476+0000 7f02c8be2640 1 -- 172.21.15.43:0/495710800 shutdown_connections 2024-04-25T09:00:26.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.476+0000 7f02c8be2640 1 -- 172.21.15.43:0/495710800 wait complete. 2024-04-25T09:00:26.475 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:00:26.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.588+0000 7fcfaa612640 1 Processor -- start 2024-04-25T09:00:26.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.588+0000 7fcfaa612640 1 -- start start 2024-04-25T09:00:26.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.589+0000 7fcfaa612640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcfa4161390 0x7fcfa40b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:26.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.589+0000 7fcfaa612640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fcfa40cecf0 0x7fcfa40d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:26.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.589+0000 7fcfaa612640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfa40b4c30 con 0x7fcfa40d02a0 2024-04-25T09:00:26.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.589+0000 7fcfaa612640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfa40d1910 con 0x7fcfa40cecf0 2024-04-25T09:00:26.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.589+0000 7fcfaa612640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa40cf2c0 con 0x7fcfa4161390 2024-04-25T09:00:26.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.589+0000 7fcfa8baf640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fcfa40cecf0 0x7fcfa40d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:26.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.589+0000 7fcfa3fff640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcfa4161390 0x7fcfa40b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:26.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.589+0000 7fcfa37fe640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7fcfa40d02a0 0x7fcfa40ce5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:38022/0 (socket says 172.21.15.43:38022) 2024-04-25T09:00:26.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.589+0000 7fcfa37fe640 1 -- 172.21.15.43:0/991483438 learned_addr learned my addr 172.21.15.43:0/991483438 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:00:26.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.590+0000 7fcfa2ffd640 1 -- 172.21.15.43:0/991483438 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 996033951 0 0) 0x7fcf8c001110 con 0x7fcfa40d02a0 2024-04-25T09:00:26.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.590+0000 7fcfa2ffd640 1 -- 172.21.15.43:0/991483438 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf90003520 con 0x7fcfa40d02a0 2024-04-25T09:00:26.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.590+0000 7fcfa3fff640 1 -- 172.21.15.43:0/991483438 >> v2:172.21.15.193:3300/0 conn(0x7fcfa40cecf0 msgr2=0x7fcfa40d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:26.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.590+0000 7fcfa3fff640 1 --2- 172.21.15.43:0/991483438 >> v2:172.21.15.193:3300/0 conn(0x7fcfa40cecf0 0x7fcfa40d0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.590+0000 7fcfa3fff640 1 -- 172.21.15.43:0/991483438 >> v1:172.21.15.193:6789/0 conn(0x7fcfa40d02a0 legacy=0x7fcfa40ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:26.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.590+0000 7fcfa3fff640 1 -- 172.21.15.43:0/991483438 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa40cf5a0 con 0x7fcfa4161390 2024-04-25T09:00:26.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.590+0000 7fcfa8baf640 1 --2- 172.21.15.43:0/991483438 >> v2:172.21.15.193:3300/0 conn(0x7fcfa40cecf0 0x7fcfa40d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:00:26.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.590+0000 7fcfa3fff640 1 --2- 172.21.15.43:0/991483438 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcfa4161390 0x7fcfa40b6820 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcf940097a0 tx=0x7fcf9402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca7e6347b559a95d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:26.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.591+0000 7fcfa2ffd640 1 -- 172.21.15.43:0/991483438 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf94037070 con 0x7fcfa4161390 2024-04-25T09:00:26.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.591+0000 7fcfa2ffd640 1 -- 172.21.15.43:0/991483438 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf94032440 con 0x7fcfa4161390 2024-04-25T09:00:26.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.591+0000 7fcfa2ffd640 1 -- 172.21.15.43:0/991483438 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf940327f0 con 0x7fcfa4161390 2024-04-25T09:00:26.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.591+0000 7fcfaa612640 1 -- 172.21.15.43:0/991483438 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcfa4161390 msgr2=0x7fcfa40b6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:26.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.591+0000 7fcfaa612640 1 --2- 172.21.15.43:0/991483438 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcfa4161390 0x7fcfa40b6820 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcf940097a0 tx=0x7fcf9402a180 comp rx=0 tx=0).stop 2024-04-25T09:00:26.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.591+0000 7fcfaa612640 1 -- 172.21.15.43:0/991483438 shutdown_connections 2024-04-25T09:00:26.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.591+0000 7fcfaa612640 1 --2- 172.21.15.43:0/991483438 >> v2:172.21.15.193:3300/0 conn(0x7fcfa40cecf0 0x7fcfa40d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.591+0000 7fcfaa612640 1 --2- 172.21.15.43:0/991483438 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcfa4161390 0x7fcfa40b6820 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.591+0000 7fcfaa612640 1 -- 172.21.15.43:0/991483438 >> 172.21.15.43:0/991483438 conn(0x7fcfa409cd20 msgr2=0x7fcfa40d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:26.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.591+0000 7fcfaa612640 1 -- 172.21.15.43:0/991483438 shutdown_connections 2024-04-25T09:00:26.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.591+0000 7fcfaa612640 1 -- 172.21.15.43:0/991483438 wait complete. 2024-04-25T09:00:26.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.592+0000 7fcfaa612640 1 Processor -- start 2024-04-25T09:00:26.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.593+0000 7fcfaa612640 1 -- start start 2024-04-25T09:00:26.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.593+0000 7fcfaa612640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcfa40cecf0 0x7fcfa4158160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:26.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.593+0000 7fcfaa612640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfa40d02a0 0x7fcfa415d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:26.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.593+0000 7fcfaa612640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcfa4161390 0x7fcfa41f4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:26.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.593+0000 7fcfaa612640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa4161770 con 0x7fcfa40d02a0 2024-04-25T09:00:26.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.593+0000 7fcfaa612640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa40d15a0 con 0x7fcfa4161390 2024-04-25T09:00:26.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.593+0000 7fcfaa612640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfa40d1da0 con 0x7fcfa40cecf0 2024-04-25T09:00:26.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.593+0000 7fcfa37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfa40d02a0 0x7fcfa415d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:26.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.593+0000 7fcfa37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfa40d02a0 0x7fcfa415d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40090/0 (socket says 172.21.15.43:40090) 2024-04-25T09:00:26.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.593+0000 7fcfa37fe640 1 -- 172.21.15.43:0/187684323 learned_addr learned my addr 172.21.15.43:0/187684323 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:26.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.593+0000 7fcfa8baf640 1 --2- 172.21.15.43:0/187684323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcfa4161390 0x7fcfa41f4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:26.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.593+0000 7fcfa3fff640 1 --2- 172.21.15.43:0/187684323 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcfa40cecf0 0x7fcfa4158160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:26.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.594+0000 7fcfa37fe640 1 -- 172.21.15.43:0/187684323 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcfa40cecf0 msgr2=0x7fcfa4158160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:26.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.594+0000 7fcfa37fe640 1 --2- 172.21.15.43:0/187684323 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcfa40cecf0 0x7fcfa4158160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.594+0000 7fcfa37fe640 1 -- 172.21.15.43:0/187684323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcfa4161390 msgr2=0x7fcfa41f4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:26.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.594+0000 7fcfa37fe640 1 --2- 172.21.15.43:0/187684323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcfa4161390 0x7fcfa41f4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.594+0000 7fcfa37fe640 1 -- 172.21.15.43:0/187684323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa41570c0 con 0x7fcfa40d02a0 2024-04-25T09:00:26.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.594+0000 7fcfa37fe640 1 --2- 172.21.15.43:0/187684323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfa40d02a0 0x7fcfa415d340 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c00c760 tx=0x7fcf8c00cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:26.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.594+0000 7fcfa0ff9640 1 -- 172.21.15.43:0/187684323 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c017070 con 0x7fcfa40d02a0 2024-04-25T09:00:26.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.594+0000 7fcfa0ff9640 1 -- 172.21.15.43:0/187684323 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf8c005ab0 con 0x7fcfa40d02a0 2024-04-25T09:00:26.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.594+0000 7fcfaa612640 1 -- 172.21.15.43:0/187684323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfa41573a0 con 0x7fcfa40d02a0 2024-04-25T09:00:26.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.594+0000 7fcfa0ff9640 1 -- 172.21.15.43:0/187684323 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c007070 con 0x7fcfa40d02a0 2024-04-25T09:00:26.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.594+0000 7fcfaa612640 1 -- 172.21.15.43:0/187684323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfa4157960 con 0x7fcfa40d02a0 2024-04-25T09:00:26.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.595+0000 7fcfaa612640 1 -- 172.21.15.43:0/187684323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfa40680f0 con 0x7fcfa40d02a0 2024-04-25T09:00:26.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.596+0000 7fcfa0ff9640 1 -- 172.21.15.43:0/187684323 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcf8c01a070 con 0x7fcfa40d02a0 2024-04-25T09:00:26.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.596+0000 7fcfa0ff9640 1 --2- 172.21.15.43:0/187684323 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcf8406b4d0 0x7fcf8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:26.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.596+0000 7fcfa0ff9640 1 -- 172.21.15.43:0/187684323 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fcf8c0895d0 con 0x7fcfa40d02a0 2024-04-25T09:00:26.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.596+0000 7fcfa3fff640 1 --2- 172.21.15.43:0/187684323 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcf8406b4d0 0x7fcf8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:26.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.597+0000 7fcfa3fff640 1 --2- 172.21.15.43:0/187684323 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcf8406b4d0 0x7fcf8406d990 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fcf94034000 tx=0x7fcf9402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:26.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.600+0000 7fcfa0ff9640 1 -- 172.21.15.43:0/187684323 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcfa40b4c30 con 0x7fcfa40d02a0 2024-04-25T09:00:26.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.793+0000 7fcfaa612640 1 -- 172.21.15.43:0/187684323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcfa4074ef0 con 0x7fcfa40d02a0 2024-04-25T09:00:26.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.793+0000 7fcfa0ff9640 1 -- 172.21.15.43:0/187684323 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v77) v1 ==== 80+0+0 (secure 0 0 0) 0x7fcf8c087e20 con 0x7fcfa40d02a0 2024-04-25T09:00:26.782 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-04-25T09:00:26.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.795+0000 7fcfaa612640 1 -- 172.21.15.43:0/187684323 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcf8406b4d0 msgr2=0x7fcf8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:26.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.795+0000 7fcfaa612640 1 --2- 172.21.15.43:0/187684323 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcf8406b4d0 0x7fcf8406d990 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fcf94034000 tx=0x7fcf9402e000 comp rx=0 tx=0).stop 2024-04-25T09:00:26.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.795+0000 7fcfaa612640 1 -- 172.21.15.43:0/187684323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfa40d02a0 msgr2=0x7fcfa415d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:26.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.795+0000 7fcfaa612640 1 --2- 172.21.15.43:0/187684323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfa40d02a0 0x7fcfa415d340 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c00c760 tx=0x7fcf8c00cb60 comp rx=0 tx=0).stop 2024-04-25T09:00:26.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.795+0000 7fcfaa612640 1 -- 172.21.15.43:0/187684323 shutdown_connections 2024-04-25T09:00:26.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.795+0000 7fcfaa612640 1 --2- 172.21.15.43:0/187684323 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcf8406b4d0 0x7fcf8406d990 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.795+0000 7fcfaa612640 1 --2- 172.21.15.43:0/187684323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcfa4161390 0x7fcfa41f4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.795+0000 7fcfaa612640 1 --2- 172.21.15.43:0/187684323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcfa40d02a0 0x7fcfa415d340 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.795+0000 7fcfaa612640 1 --2- 172.21.15.43:0/187684323 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcfa40cecf0 0x7fcfa4158160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:26.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.795+0000 7fcfaa612640 1 -- 172.21.15.43:0/187684323 >> 172.21.15.43:0/187684323 conn(0x7fcfa409cd20 msgr2=0x7fcfa406c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:26.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.795+0000 7fcfaa612640 1 -- 172.21.15.43:0/187684323 shutdown_connections 2024-04-25T09:00:26.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:26.795+0000 7fcfaa612640 1 -- 172.21.15.43:0/187684323 wait complete. 2024-04-25T09:00:26.795 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi043.front.sepia.ceph.com... 2024-04-25T09:00:26.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:26.795 DEBUG:teuthology.orchestra.run.smithi043:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T09:00:26.817 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[67107]: fuse finished with error 0 and tester_r 0 2024-04-25T09:00:26.823 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-04-25T09:00:26.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:26.823 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:00:26.992 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T09:00:26.992 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:00:27.009 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19508 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:00:27.009 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi193.front.sepia.ceph.com 2024-04-25T09:00:27.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:27.010 DEBUG:teuthology.orchestra.run.smithi193:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T09:00:27.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:00:27.066 INFO:teuthology.orchestra.run.smithi193.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T09:00:27.067 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T09:00:27.067 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:00:27.083 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2374 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:00:27.083 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-04-25T09:00:27.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:27.084 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T09:00:27.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:00:27.140 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T09:00:27.140 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T09:00:27.140 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:00:27.157 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19508 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:00:27.157 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi193.front.sepia.ceph.com 2024-04-25T09:00:27.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:27.157 DEBUG:teuthology.orchestra.run.smithi193:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T09:00:27.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:00:27.214 INFO:teuthology.orchestra.run.smithi193.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T09:00:27.215 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:00:27.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.328+0000 7fb6d599e640 1 Processor -- start 2024-04-25T09:00:27.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.328+0000 7fb6d599e640 1 -- start start 2024-04-25T09:00:27.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.329+0000 7fb6d599e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb6d00d39d0 0x7fb6d00d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.329+0000 7fb6d599e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb6d00ce2a0 0x7fb6d00d3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.329+0000 7fb6d599e640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb6d0161710 0x7fb6d0161ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.329+0000 7fb6d599e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6d00b6c50 con 0x7fb6d00d39d0 2024-04-25T09:00:27.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.329+0000 7fb6d599e640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6d015e010 con 0x7fb6d0161710 2024-04-25T09:00:27.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.329+0000 7fb6d599e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d00d2020 con 0x7fb6d00ce2a0 2024-04-25T09:00:27.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.329+0000 7fb6ceffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb6d00d39d0 0x7fb6d00d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.329+0000 7fb6ceffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb6d00d39d0 0x7fb6d00d0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40098/0 (socket says 172.21.15.43:40098) 2024-04-25T09:00:27.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.329+0000 7fb6ceffd640 1 -- 172.21.15.43:0/2488220579 learned_addr learned my addr 172.21.15.43:0/2488220579 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:27.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.329+0000 7fb6ce7fc640 1 --2- 172.21.15.43:0/2488220579 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb6d00ce2a0 0x7fb6d00d3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.329+0000 7fb6cf7fe640 1 --2- 172.21.15.43:0/2488220579 >> v2:172.21.15.193:3300/0 conn(0x7fb6d0161710 0x7fb6d0161ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.329+0000 7fb6ceffd640 1 -- 172.21.15.43:0/2488220579 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb6d00ce2a0 msgr2=0x7fb6d00d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:27.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.329+0000 7fb6ceffd640 1 --2- 172.21.15.43:0/2488220579 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb6d00ce2a0 0x7fb6d00d3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.329+0000 7fb6ceffd640 1 -- 172.21.15.43:0/2488220579 >> v2:172.21.15.193:3300/0 conn(0x7fb6d0161710 msgr2=0x7fb6d0161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:27.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.329+0000 7fb6ceffd640 1 --2- 172.21.15.43:0/2488220579 >> v2:172.21.15.193:3300/0 conn(0x7fb6d0161710 0x7fb6d0161ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.329+0000 7fb6ceffd640 1 -- 172.21.15.43:0/2488220579 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d00d42f0 con 0x7fb6d00d39d0 2024-04-25T09:00:27.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.330+0000 7fb6cf7fe640 1 --2- 172.21.15.43:0/2488220579 >> v2:172.21.15.193:3300/0 conn(0x7fb6d0161710 0x7fb6d0161ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:27.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.330+0000 7fb6ceffd640 1 --2- 172.21.15.43:0/2488220579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6d00d39d0 0x7fb6d00d0d90 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fb6d006ccf0 tx=0x7fb6c802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4546036259f96dfb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:27.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.330+0000 7fb6cdffb640 1 -- 172.21.15.43:0/2488220579 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c8037070 con 0x7fb6d00d39d0 2024-04-25T09:00:27.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.330+0000 7fb6cdffb640 1 -- 172.21.15.43:0/2488220579 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c8032440 con 0x7fb6d00d39d0 2024-04-25T09:00:27.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.330+0000 7fb6cdffb640 1 -- 172.21.15.43:0/2488220579 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c8032810 con 0x7fb6d00d39d0 2024-04-25T09:00:27.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.331+0000 7fb6d599e640 1 -- 172.21.15.43:0/2488220579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6d00d39d0 msgr2=0x7fb6d00d0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.331+0000 7fb6d599e640 1 --2- 172.21.15.43:0/2488220579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6d00d39d0 0x7fb6d00d0d90 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fb6d006ccf0 tx=0x7fb6c802a140 comp rx=0 tx=0).stop 2024-04-25T09:00:27.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.331+0000 7fb6d599e640 1 -- 172.21.15.43:0/2488220579 shutdown_connections 2024-04-25T09:00:27.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.331+0000 7fb6d599e640 1 --2- 172.21.15.43:0/2488220579 >> v2:172.21.15.193:3300/0 conn(0x7fb6d0161710 0x7fb6d0161ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.331+0000 7fb6d599e640 1 --2- 172.21.15.43:0/2488220579 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb6d00ce2a0 0x7fb6d00d3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.331+0000 7fb6d599e640 1 --2- 172.21.15.43:0/2488220579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6d00d39d0 0x7fb6d00d0d90 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.331+0000 7fb6d599e640 1 -- 172.21.15.43:0/2488220579 >> 172.21.15.43:0/2488220579 conn(0x7fb6d006b000 msgr2=0x7fb6d006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:27.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.331+0000 7fb6d599e640 1 -- 172.21.15.43:0/2488220579 shutdown_connections 2024-04-25T09:00:27.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.331+0000 7fb6d599e640 1 -- 172.21.15.43:0/2488220579 wait complete. 2024-04-25T09:00:27.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.332+0000 7fb6d599e640 1 Processor -- start 2024-04-25T09:00:27.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.332+0000 7fb6d599e640 1 -- start start 2024-04-25T09:00:27.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.333+0000 7fb6d599e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb6d00ce2a0 0x7fb6d0153f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.333+0000 7fb6d599e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6d00d39d0 0x7fb6d00b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.333+0000 7fb6d599e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6d0161710 0x7fb6d00c8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.333+0000 7fb6d599e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d00b2f00 con 0x7fb6d00d39d0 2024-04-25T09:00:27.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.333+0000 7fb6d599e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d00cef20 con 0x7fb6d0161710 2024-04-25T09:00:27.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.333+0000 7fb6d599e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d00cf200 con 0x7fb6d00ce2a0 2024-04-25T09:00:27.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.333+0000 7fb6ce7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6d00d39d0 0x7fb6d00b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.333+0000 7fb6ce7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6d00d39d0 0x7fb6d00b4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40106/0 (socket says 172.21.15.43:40106) 2024-04-25T09:00:27.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.333+0000 7fb6ce7fc640 1 -- 172.21.15.43:0/2095833626 learned_addr learned my addr 172.21.15.43:0/2095833626 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:27.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.333+0000 7fb6ceffd640 1 --2- 172.21.15.43:0/2095833626 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb6d00ce2a0 0x7fb6d0153f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.333+0000 7fb6cf7fe640 1 --2- 172.21.15.43:0/2095833626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6d0161710 0x7fb6d00c8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.334+0000 7fb6ce7fc640 1 -- 172.21.15.43:0/2095833626 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb6d00ce2a0 msgr2=0x7fb6d0153f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.334+0000 7fb6ce7fc640 1 --2- 172.21.15.43:0/2095833626 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb6d00ce2a0 0x7fb6d0153f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.334+0000 7fb6ce7fc640 1 -- 172.21.15.43:0/2095833626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6d0161710 msgr2=0x7fb6d00c8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.334+0000 7fb6ce7fc640 1 --2- 172.21.15.43:0/2095833626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6d0161710 0x7fb6d00c8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.334+0000 7fb6ce7fc640 1 -- 172.21.15.43:0/2095833626 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d00b59a0 con 0x7fb6d00d39d0 2024-04-25T09:00:27.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.334+0000 7fb6cf7fe640 1 --2- 172.21.15.43:0/2095833626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6d0161710 0x7fb6d00c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:00:27.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.334+0000 7fb6ce7fc640 1 --2- 172.21.15.43:0/2095833626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6d00d39d0 0x7fb6d00b4ee0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fb6b800d800 tx=0x7fb6b800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:27.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.334+0000 7fb6c7fff640 1 -- 172.21.15.43:0/2095833626 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b8018020 con 0x7fb6d00d39d0 2024-04-25T09:00:27.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.334+0000 7fb6c7fff640 1 -- 172.21.15.43:0/2095833626 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6b80096c0 con 0x7fb6d00d39d0 2024-04-25T09:00:27.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.334+0000 7fb6d599e640 1 -- 172.21.15.43:0/2095833626 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6d00b5c80 con 0x7fb6d00d39d0 2024-04-25T09:00:27.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.334+0000 7fb6c7fff640 1 -- 172.21.15.43:0/2095833626 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b8009ad0 con 0x7fb6d00d39d0 2024-04-25T09:00:27.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.334+0000 7fb6d599e640 1 -- 172.21.15.43:0/2095833626 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6d00b6240 con 0x7fb6d00d39d0 2024-04-25T09:00:27.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.335+0000 7fb6d599e640 1 -- 172.21.15.43:0/2095833626 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6d0074ef0 con 0x7fb6d00d39d0 2024-04-25T09:00:27.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.335+0000 7fb6c7fff640 1 -- 172.21.15.43:0/2095833626 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb6b801b020 con 0x7fb6d00d39d0 2024-04-25T09:00:27.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.336+0000 7fb6c7fff640 1 --2- 172.21.15.43:0/2095833626 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb6b006b4d0 0x7fb6b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.336+0000 7fb6c7fff640 1 -- 172.21.15.43:0/2095833626 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb6b808a3b0 con 0x7fb6d00d39d0 2024-04-25T09:00:27.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.336+0000 7fb6ceffd640 1 --2- 172.21.15.43:0/2095833626 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb6b006b4d0 0x7fb6b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.337+0000 7fb6ceffd640 1 --2- 172.21.15.43:0/2095833626 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb6b006b4d0 0x7fb6b006d990 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8034000 tx=0x7fb6c802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:27.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.340+0000 7fb6c7fff640 1 -- 172.21.15.43:0/2095833626 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb6b8022030 con 0x7fb6d00d39d0 2024-04-25T09:00:27.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.557+0000 7fb6d599e640 1 -- 172.21.15.43:0/2095833626 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6d00ce880 con 0x7fb6d00d39d0 2024-04-25T09:00:27.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.557+0000 7fb6c7fff640 1 -- 172.21.15.43:0/2095833626 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fb6b8059260 con 0x7fb6d00d39d0 2024-04-25T09:00:27.547 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:00:27.547 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":48,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6373,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3648859409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":3648859409},{"type":"v1","addr":"172.21.15.193:6837","nonce":3648859409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":7149,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/356516962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":356516962},{"type":"v1","addr":"172.21.15.43:6834","nonce":356516962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":7152,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3032637719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":3032637719},{"type":"v1","addr":"172.21.15.43:6835","nonce":3032637719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T08:58:45.906679+0000","modified":"2024-04-25T08:58:54.057379+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7164},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7164":{"gid":7164,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.193:6836/3458098162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":3458098162},{"type":"v1","addr":"172.21.15.193:6836","nonce":3458098162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T09:00:27.547 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 48 2024-04-25T09:00:27.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.559+0000 7fb6d599e640 1 -- 172.21.15.43:0/2095833626 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb6b006b4d0 msgr2=0x7fb6b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.559+0000 7fb6d599e640 1 --2- 172.21.15.43:0/2095833626 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb6b006b4d0 0x7fb6b006d990 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8034000 tx=0x7fb6c802e000 comp rx=0 tx=0).stop 2024-04-25T09:00:27.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.559+0000 7fb6d599e640 1 -- 172.21.15.43:0/2095833626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6d00d39d0 msgr2=0x7fb6d00b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.559+0000 7fb6d599e640 1 --2- 172.21.15.43:0/2095833626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6d00d39d0 0x7fb6d00b4ee0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fb6b800d800 tx=0x7fb6b800dcc0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.559+0000 7fb6d599e640 1 -- 172.21.15.43:0/2095833626 shutdown_connections 2024-04-25T09:00:27.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.559+0000 7fb6d599e640 1 --2- 172.21.15.43:0/2095833626 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb6b006b4d0 0x7fb6b006d990 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.559+0000 7fb6d599e640 1 --2- 172.21.15.43:0/2095833626 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6d0161710 0x7fb6d00c8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.559+0000 7fb6d599e640 1 --2- 172.21.15.43:0/2095833626 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6d00d39d0 0x7fb6d00b4ee0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.559+0000 7fb6d599e640 1 --2- 172.21.15.43:0/2095833626 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb6d00ce2a0 0x7fb6d0153f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.559+0000 7fb6d599e640 1 -- 172.21.15.43:0/2095833626 >> 172.21.15.43:0/2095833626 conn(0x7fb6d006b000 msgr2=0x7fb6d015e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:27.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.560+0000 7fb6d599e640 1 -- 172.21.15.43:0/2095833626 shutdown_connections 2024-04-25T09:00:27.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.560+0000 7fb6d599e640 1 -- 172.21.15.43:0/2095833626 wait complete. 2024-04-25T09:00:27.560 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:00:27.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.672+0000 7fcc2be93640 1 Processor -- start 2024-04-25T09:00:27.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.672+0000 7fcc2be93640 1 -- start start 2024-04-25T09:00:27.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.673+0000 7fcc2be93640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcc2415dce0 0x7fcc2415ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.673+0000 7fcc2be93640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc240d0af0 0x7fcc24160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.673+0000 7fcc2be93640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc240cee80 con 0x7fcc240ce550 2024-04-25T09:00:27.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.673+0000 7fcc2be93640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc240d2920 con 0x7fcc2415dce0 2024-04-25T09:00:27.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.673+0000 7fcc2be93640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc240cf160 con 0x7fcc240d0af0 2024-04-25T09:00:27.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.673+0000 7fcc29c2f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcc2415dce0 0x7fcc2415ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.673+0000 7fcc2942e640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fcc240ce550 0x7fcc240d03e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32974/0 (socket says 172.21.15.43:32974) 2024-04-25T09:00:27.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.673+0000 7fcc29c2f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcc2415dce0 0x7fcc2415ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40108/0 (socket says 172.21.15.43:40108) 2024-04-25T09:00:27.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.673+0000 7fcc2942e640 1 -- 172.21.15.43:0/2172361354 learned_addr learned my addr 172.21.15.43:0/2172361354 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:00:27.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.673+0000 7fcc2a430640 1 --2- 172.21.15.43:0/2172361354 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc240d0af0 0x7fcc24160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.674+0000 7fcc29c2f640 1 -- 172.21.15.43:0/2172361354 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc240d0af0 msgr2=0x7fcc24160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:27.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.674+0000 7fcc28c2d640 1 -- 172.21.15.43:0/2172361354 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1427290154 0 0) 0x7fcc240cee80 con 0x7fcc240ce550 2024-04-25T09:00:27.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.674+0000 7fcc29c2f640 1 --2- 172.21.15.43:0/2172361354 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc240d0af0 0x7fcc24160980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.674+0000 7fcc29c2f640 1 -- 172.21.15.43:0/2172361354 >> v1:172.21.15.43:6789/0 conn(0x7fcc240ce550 legacy=0x7fcc240d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.674+0000 7fcc29c2f640 1 -- 172.21.15.43:0/2172361354 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc240cf440 con 0x7fcc2415dce0 2024-04-25T09:00:27.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.674+0000 7fcc29c2f640 1 --2- 172.21.15.43:0/2172361354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc2415dce0 0x7fcc2415ef50 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fcc200097a0 tx=0x7fcc2002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d16f9a78aa131c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:27.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.674+0000 7fcc28c2d640 1 -- 172.21.15.43:0/2172361354 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc20037070 con 0x7fcc2415dce0 2024-04-25T09:00:27.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.674+0000 7fcc28c2d640 1 -- 172.21.15.43:0/2172361354 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc2002ae50 con 0x7fcc2415dce0 2024-04-25T09:00:27.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.674+0000 7fcc28c2d640 1 -- 172.21.15.43:0/2172361354 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc20032660 con 0x7fcc2415dce0 2024-04-25T09:00:27.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.675+0000 7fcc2be93640 1 -- 172.21.15.43:0/2172361354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc2415dce0 msgr2=0x7fcc2415ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.675+0000 7fcc2be93640 1 --2- 172.21.15.43:0/2172361354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc2415dce0 0x7fcc2415ef50 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7fcc200097a0 tx=0x7fcc2002a100 comp rx=0 tx=0).stop 2024-04-25T09:00:27.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.675+0000 7fcc2be93640 1 -- 172.21.15.43:0/2172361354 shutdown_connections 2024-04-25T09:00:27.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.675+0000 7fcc2be93640 1 --2- 172.21.15.43:0/2172361354 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc240d0af0 0x7fcc24160980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.675+0000 7fcc2be93640 1 --2- 172.21.15.43:0/2172361354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc2415dce0 0x7fcc2415ef50 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.675+0000 7fcc2be93640 1 -- 172.21.15.43:0/2172361354 >> 172.21.15.43:0/2172361354 conn(0x7fcc2409cd00 msgr2=0x7fcc240d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:27.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.675+0000 7fcc2be93640 1 -- 172.21.15.43:0/2172361354 shutdown_connections 2024-04-25T09:00:27.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.675+0000 7fcc2be93640 1 -- 172.21.15.43:0/2172361354 wait complete. 2024-04-25T09:00:27.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.676+0000 7fcc2be93640 1 Processor -- start 2024-04-25T09:00:27.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.676+0000 7fcc2be93640 1 -- start start 2024-04-25T09:00:27.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.677+0000 7fcc2be93640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc240ce550 0x7fcc2414fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.677+0000 7fcc2be93640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc240d0af0 0x7fcc240b8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.677+0000 7fcc2be93640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc2415dce0 0x7fcc241f3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.677+0000 7fcc2be93640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2415e0c0 con 0x7fcc240d0af0 2024-04-25T09:00:27.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.677+0000 7fcc2be93640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2415f6b0 con 0x7fcc2415dce0 2024-04-25T09:00:27.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.677+0000 7fcc2be93640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc240ce9c0 con 0x7fcc240ce550 2024-04-25T09:00:27.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.677+0000 7fcc2942e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc240d0af0 0x7fcc240b8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.677+0000 7fcc2942e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc240d0af0 0x7fcc240b8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40114/0 (socket says 172.21.15.43:40114) 2024-04-25T09:00:27.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.677+0000 7fcc2942e640 1 -- 172.21.15.43:0/1028921266 learned_addr learned my addr 172.21.15.43:0/1028921266 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:27.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.677+0000 7fcc2a430640 1 --2- 172.21.15.43:0/1028921266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc2415dce0 0x7fcc241f3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.677+0000 7fcc29c2f640 1 --2- 172.21.15.43:0/1028921266 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc240ce550 0x7fcc2414fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.678+0000 7fcc2942e640 1 -- 172.21.15.43:0/1028921266 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc240ce550 msgr2=0x7fcc2414fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.678+0000 7fcc2942e640 1 --2- 172.21.15.43:0/1028921266 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc240ce550 0x7fcc2414fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.678+0000 7fcc2942e640 1 -- 172.21.15.43:0/1028921266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc2415dce0 msgr2=0x7fcc241f3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.678+0000 7fcc2942e640 1 --2- 172.21.15.43:0/1028921266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc2415dce0 0x7fcc241f3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.678+0000 7fcc2942e640 1 -- 172.21.15.43:0/1028921266 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc241e49d0 con 0x7fcc240d0af0 2024-04-25T09:00:27.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.678+0000 7fcc2942e640 1 --2- 172.21.15.43:0/1028921266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc240d0af0 0x7fcc240b8dc0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fcc1400c400 tx=0x7fcc1400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:27.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.678+0000 7fcc1a7fc640 1 -- 172.21.15.43:0/1028921266 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc14017070 con 0x7fcc240d0af0 2024-04-25T09:00:27.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.678+0000 7fcc1a7fc640 1 -- 172.21.15.43:0/1028921266 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc14005ce0 con 0x7fcc240d0af0 2024-04-25T09:00:27.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.678+0000 7fcc2be93640 1 -- 172.21.15.43:0/1028921266 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc241e4f90 con 0x7fcc240d0af0 2024-04-25T09:00:27.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.678+0000 7fcc1a7fc640 1 -- 172.21.15.43:0/1028921266 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc14007310 con 0x7fcc240d0af0 2024-04-25T09:00:27.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.678+0000 7fcc2be93640 1 -- 172.21.15.43:0/1028921266 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc241e5550 con 0x7fcc240d0af0 2024-04-25T09:00:27.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.679+0000 7fcc2be93640 1 -- 172.21.15.43:0/1028921266 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc240680f0 con 0x7fcc240d0af0 2024-04-25T09:00:27.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.679+0000 7fcc1a7fc640 1 -- 172.21.15.43:0/1028921266 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcc1401a070 con 0x7fcc240d0af0 2024-04-25T09:00:27.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.680+0000 7fcc1a7fc640 1 --2- 172.21.15.43:0/1028921266 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcc0806b4d0 0x7fcc0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.680+0000 7fcc1a7fc640 1 -- 172.21.15.43:0/1028921266 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fcc140896c0 con 0x7fcc240d0af0 2024-04-25T09:00:27.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.680+0000 7fcc29c2f640 1 --2- 172.21.15.43:0/1028921266 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcc0806b4d0 0x7fcc0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.681+0000 7fcc29c2f640 1 --2- 172.21.15.43:0/1028921266 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcc0806b4d0 0x7fcc0806d990 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fcc20034000 tx=0x7fcc2002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:27.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.684+0000 7fcc1a7fc640 1 -- 172.21.15.43:0/1028921266 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcc140584c0 con 0x7fcc240d0af0 2024-04-25T09:00:27.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.866+0000 7fcc2be93640 1 -- 172.21.15.43:0/1028921266 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc24074ef0 con 0x7fcc240d0af0 2024-04-25T09:00:27.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.867+0000 7fcc1a7fc640 1 -- 172.21.15.43:0/1028921266 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+16795 (secure 0 0 0) 0x7fcc14057dc0 con 0x7fcc240d0af0 2024-04-25T09:00:27.856 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:00:27.857 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":77,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:00:02.408776+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:58:44.188924+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T08:58:44.877113+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":76,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.1","mappings":[{"from":6,"to":0}]},{"pgid":"11.2e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:00:27.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.869+0000 7fcc2be93640 1 -- 172.21.15.43:0/1028921266 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcc0806b4d0 msgr2=0x7fcc0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.869+0000 7fcc2be93640 1 --2- 172.21.15.43:0/1028921266 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcc0806b4d0 0x7fcc0806d990 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fcc20034000 tx=0x7fcc2002e000 comp rx=0 tx=0).stop 2024-04-25T09:00:27.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.869+0000 7fcc2be93640 1 -- 172.21.15.43:0/1028921266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc240d0af0 msgr2=0x7fcc240b8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.869+0000 7fcc2be93640 1 --2- 172.21.15.43:0/1028921266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc240d0af0 0x7fcc240b8dc0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fcc1400c400 tx=0x7fcc1400c8c0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.869+0000 7fcc2be93640 1 -- 172.21.15.43:0/1028921266 shutdown_connections 2024-04-25T09:00:27.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.869+0000 7fcc2be93640 1 --2- 172.21.15.43:0/1028921266 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcc0806b4d0 0x7fcc0806d990 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.869+0000 7fcc2be93640 1 --2- 172.21.15.43:0/1028921266 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcc2415dce0 0x7fcc241f3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.869+0000 7fcc2be93640 1 --2- 172.21.15.43:0/1028921266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcc240d0af0 0x7fcc240b8dc0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.869+0000 7fcc2be93640 1 --2- 172.21.15.43:0/1028921266 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcc240ce550 0x7fcc2414fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.869+0000 7fcc2be93640 1 -- 172.21.15.43:0/1028921266 >> 172.21.15.43:0/1028921266 conn(0x7fcc2409cd00 msgr2=0x7fcc2415ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:27.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.869+0000 7fcc2be93640 1 -- 172.21.15.43:0/1028921266 shutdown_connections 2024-04-25T09:00:27.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.869+0000 7fcc2be93640 1 -- 172.21.15.43:0/1028921266 wait complete. 2024-04-25T09:00:27.870 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T08:58:44.188924+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '72', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T08:58:44.877113+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 76, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:00:27.871 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:00:27.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.986+0000 7fb592742640 1 Processor -- start 2024-04-25T09:00:27.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.986+0000 7fb592742640 1 -- start start 2024-04-25T09:00:27.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.986+0000 7fb592742640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb58c15e090 0x7fb58c161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.986+0000 7fb592742640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb58c161fe0 0x7fb58c0b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.986+0000 7fb592742640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb58c162610 con 0x7fb58c0b5aa0 2024-04-25T09:00:27.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.986+0000 7fb592742640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb58c0b5f00 con 0x7fb58c15e090 2024-04-25T09:00:27.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.986+0000 7fb592742640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb58c0d1da0 con 0x7fb58c161fe0 2024-04-25T09:00:27.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.987+0000 7fb58b7fe640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb58c161fe0 0x7fb58c0b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.987+0000 7fb58bfff640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb58c15e090 0x7fb58c161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.987+0000 7fb58b7fe640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb58c161fe0 0x7fb58c0b5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:37770/0 (socket says 172.21.15.43:37770) 2024-04-25T09:00:27.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.987+0000 7fb58b7fe640 1 -- 172.21.15.43:0/201404732 learned_addr learned my addr 172.21.15.43:0/201404732 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:27.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.987+0000 7fb58b7fe640 1 -- 172.21.15.43:0/201404732 >> v2:172.21.15.193:3300/0 conn(0x7fb58c15e090 msgr2=0x7fb58c161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:27.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.987+0000 7fb58b7fe640 1 --2- 172.21.15.43:0/201404732 >> v2:172.21.15.193:3300/0 conn(0x7fb58c15e090 0x7fb58c161aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.987+0000 7fb58affd640 1 -- 172.21.15.43:0/201404732 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 174533570 0 0) 0x7fb58c162610 con 0x7fb58c0b5aa0 2024-04-25T09:00:27.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.987+0000 7fb58b7fe640 1 -- 172.21.15.43:0/201404732 >> v1:172.21.15.193:6789/0 conn(0x7fb58c0b5aa0 legacy=0x7fb58c0ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.987+0000 7fb58b7fe640 1 -- 172.21.15.43:0/201404732 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb58c0d1750 con 0x7fb58c161fe0 2024-04-25T09:00:27.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.988+0000 7fb58b7fe640 1 --2- 172.21.15.43:0/201404732 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb58c161fe0 0x7fb58c0b5560 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb580009870 tx=0x7fb58002a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9be7ed0f3f5be072 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:27.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.988+0000 7fb58affd640 1 -- 172.21.15.43:0/201404732 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb580037070 con 0x7fb58c161fe0 2024-04-25T09:00:27.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.988+0000 7fb58affd640 1 -- 172.21.15.43:0/201404732 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb58002ae50 con 0x7fb58c161fe0 2024-04-25T09:00:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.988+0000 7fb58affd640 1 -- 172.21.15.43:0/201404732 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb580032860 con 0x7fb58c161fe0 2024-04-25T09:00:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.989+0000 7fb592742640 1 -- 172.21.15.43:0/201404732 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb58c161fe0 msgr2=0x7fb58c0b5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.989+0000 7fb592742640 1 --2- 172.21.15.43:0/201404732 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb58c161fe0 0x7fb58c0b5560 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb580009870 tx=0x7fb58002a300 comp rx=0 tx=0).stop 2024-04-25T09:00:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.989+0000 7fb592742640 1 -- 172.21.15.43:0/201404732 shutdown_connections 2024-04-25T09:00:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.989+0000 7fb592742640 1 --2- 172.21.15.43:0/201404732 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb58c161fe0 0x7fb58c0b5560 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.989+0000 7fb592742640 1 --2- 172.21.15.43:0/201404732 >> v2:172.21.15.193:3300/0 conn(0x7fb58c15e090 0x7fb58c161aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.989+0000 7fb592742640 1 -- 172.21.15.43:0/201404732 >> 172.21.15.43:0/201404732 conn(0x7fb58c0d0560 msgr2=0x7fb58c0d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:27.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.989+0000 7fb592742640 1 -- 172.21.15.43:0/201404732 shutdown_connections 2024-04-25T09:00:27.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.989+0000 7fb592742640 1 -- 172.21.15.43:0/201404732 wait complete. 2024-04-25T09:00:27.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.990+0000 7fb592742640 1 Processor -- start 2024-04-25T09:00:27.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.990+0000 7fb592742640 1 -- start start 2024-04-25T09:00:27.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.991+0000 7fb592742640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb58c0b5aa0 0x7fb58c158420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.991+0000 7fb592742640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb58c15e090 0x7fb58c15f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.991+0000 7fb592742640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb58c161fe0 0x7fb58c0ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.991+0000 7fb592742640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58c0b3280 con 0x7fb58c161fe0 2024-04-25T09:00:27.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.991+0000 7fb592742640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58c1628f0 con 0x7fb58c0b5aa0 2024-04-25T09:00:27.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.991+0000 7fb592742640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb58c0d3240 con 0x7fb58c15e090 2024-04-25T09:00:27.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.991+0000 7fb590cdf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb58c161fe0 0x7fb58c0ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.991+0000 7fb590cdf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb58c161fe0 0x7fb58c0ca1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40124/0 (socket says 172.21.15.43:40124) 2024-04-25T09:00:27.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.991+0000 7fb590cdf640 1 -- 172.21.15.43:0/1366739584 learned_addr learned my addr 172.21.15.43:0/1366739584 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:27.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.991+0000 7fb58b7fe640 1 --2- 172.21.15.43:0/1366739584 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb58c15e090 0x7fb58c15f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.991+0000 7fb58bfff640 1 --2- 172.21.15.43:0/1366739584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb58c0b5aa0 0x7fb58c158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.991+0000 7fb590cdf640 1 -- 172.21.15.43:0/1366739584 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb58c15e090 msgr2=0x7fb58c15f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.991+0000 7fb590cdf640 1 --2- 172.21.15.43:0/1366739584 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb58c15e090 0x7fb58c15f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.991+0000 7fb590cdf640 1 -- 172.21.15.43:0/1366739584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb58c0b5aa0 msgr2=0x7fb58c158420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:27.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.991+0000 7fb590cdf640 1 --2- 172.21.15.43:0/1366739584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb58c0b5aa0 0x7fb58c158420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:27.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.991+0000 7fb590cdf640 1 -- 172.21.15.43:0/1366739584 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb58c15fa00 con 0x7fb58c161fe0 2024-04-25T09:00:27.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.992+0000 7fb590cdf640 1 --2- 172.21.15.43:0/1366739584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb58c161fe0 0x7fb58c0ca1d0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fb58400c760 tx=0x7fb58400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:27.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.992+0000 7fb588ff9640 1 -- 172.21.15.43:0/1366739584 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb584003da0 con 0x7fb58c161fe0 2024-04-25T09:00:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.992+0000 7fb588ff9640 1 -- 172.21.15.43:0/1366739584 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb584005070 con 0x7fb58c161fe0 2024-04-25T09:00:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.992+0000 7fb592742640 1 -- 172.21.15.43:0/1366739584 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb58c15fc80 con 0x7fb58c161fe0 2024-04-25T09:00:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.992+0000 7fb588ff9640 1 -- 172.21.15.43:0/1366739584 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb584005460 con 0x7fb58c161fe0 2024-04-25T09:00:27.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.992+0000 7fb592742640 1 -- 172.21.15.43:0/1366739584 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb58c160240 con 0x7fb58c161fe0 2024-04-25T09:00:27.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.993+0000 7fb592742640 1 -- 172.21.15.43:0/1366739584 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb58c06bf00 con 0x7fb58c161fe0 2024-04-25T09:00:27.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.993+0000 7fb588ff9640 1 -- 172.21.15.43:0/1366739584 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb584005600 con 0x7fb58c161fe0 2024-04-25T09:00:27.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.994+0000 7fb588ff9640 1 --2- 172.21.15.43:0/1366739584 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb57806b4d0 0x7fb57806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:27.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.994+0000 7fb588ff9640 1 -- 172.21.15.43:0/1366739584 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb58408a210 con 0x7fb58c161fe0 2024-04-25T09:00:27.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.994+0000 7fb58bfff640 1 --2- 172.21.15.43:0/1366739584 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb57806b4d0 0x7fb57806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:27.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.994+0000 7fb58bfff640 1 --2- 172.21.15.43:0/1366739584 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb57806b4d0 0x7fb57806d990 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb57c009ad0 tx=0x7fb57c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:27.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:27.998+0000 7fb588ff9640 1 -- 172.21.15.43:0/1366739584 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb58c06bf00 con 0x7fb58c161fe0 2024-04-25T09:00:28.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.178+0000 7fb592742640 1 -- 172.21.15.43:0/1366739584 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb58c074f60 con 0x7fb58c161fe0 2024-04-25T09:00:28.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.178+0000 7fb588ff9640 1 -- 172.21.15.43:0/1366739584 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb58c15fa00 con 0x7fb58c161fe0 2024-04-25T09:00:28.168 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T09:00:28.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.180+0000 7fb592742640 1 -- 172.21.15.43:0/1366739584 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb57806b4d0 msgr2=0x7fb57806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:28.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.180+0000 7fb592742640 1 --2- 172.21.15.43:0/1366739584 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb57806b4d0 0x7fb57806d990 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb57c009ad0 tx=0x7fb57c008040 comp rx=0 tx=0).stop 2024-04-25T09:00:28.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.180+0000 7fb592742640 1 -- 172.21.15.43:0/1366739584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb58c161fe0 msgr2=0x7fb58c0ca1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:28.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.180+0000 7fb592742640 1 --2- 172.21.15.43:0/1366739584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb58c161fe0 0x7fb58c0ca1d0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7fb58400c760 tx=0x7fb58400cc20 comp rx=0 tx=0).stop 2024-04-25T09:00:28.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.180+0000 7fb592742640 1 -- 172.21.15.43:0/1366739584 shutdown_connections 2024-04-25T09:00:28.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.180+0000 7fb592742640 1 --2- 172.21.15.43:0/1366739584 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb58c161fe0 0x7fb58c0ca1d0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.180+0000 7fb592742640 1 --2- 172.21.15.43:0/1366739584 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb58c15e090 0x7fb58c15f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.180+0000 7fb592742640 1 --2- 172.21.15.43:0/1366739584 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb57806b4d0 0x7fb57806d990 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.180+0000 7fb592742640 1 --2- 172.21.15.43:0/1366739584 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb58c0b5aa0 0x7fb58c158420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.180+0000 7fb592742640 1 -- 172.21.15.43:0/1366739584 >> 172.21.15.43:0/1366739584 conn(0x7fb58c0d0560 msgr2=0x7fb58c15bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:28.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.180+0000 7fb592742640 1 -- 172.21.15.43:0/1366739584 shutdown_connections 2024-04-25T09:00:28.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.180+0000 7fb592742640 1 -- 172.21.15.43:0/1366739584 wait complete. 2024-04-25T09:00:28.181 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:00:28.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.295+0000 7f5c1c64c640 1 Processor -- start 2024-04-25T09:00:28.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.296+0000 7f5c1c64c640 1 -- start start 2024-04-25T09:00:28.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.296+0000 7f5c1c64c640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f5c1406f6a0 0x7f5c140b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:28.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.297+0000 7f5c1c64c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5c140cd470 0x7f5c1406c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:28.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.297+0000 7f5c1c64c640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c140d2620 con 0x7f5c1415ddd0 2024-04-25T09:00:28.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.297+0000 7f5c1c64c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c140b0b80 con 0x7f5c140cd470 2024-04-25T09:00:28.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.297+0000 7f5c1c64c640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c140b0e60 con 0x7f5c1406f6a0 2024-04-25T09:00:28.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.297+0000 7f5c1abe9640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5c140cd470 0x7f5c1406c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:28.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.297+0000 7f5c1abe9640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5c140cd470 0x7f5c1406c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40136/0 (socket says 172.21.15.43:40136) 2024-04-25T09:00:28.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.297+0000 7f5c1abe9640 1 -- 172.21.15.43:0/3258231772 learned_addr learned my addr 172.21.15.43:0/3258231772 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:28.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.297+0000 7f5c19be7640 1 --2- 172.21.15.43:0/3258231772 >> v2:172.21.15.193:3300/0 conn(0x7f5c1406f6a0 0x7f5c140b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:28.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.297+0000 7f5c1abe9640 1 -- 172.21.15.43:0/3258231772 >> v2:172.21.15.193:3300/0 conn(0x7f5c1406f6a0 msgr2=0x7f5c140b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:28.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.297+0000 7f5c1abe9640 1 --2- 172.21.15.43:0/3258231772 >> v2:172.21.15.193:3300/0 conn(0x7f5c1406f6a0 0x7f5c140b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.297+0000 7f5c1abe9640 1 -- 172.21.15.43:0/3258231772 >> v1:172.21.15.193:6789/0 conn(0x7f5c1415ddd0 legacy=0x7f5c1406ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:28.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.297+0000 7f5c1abe9640 1 -- 172.21.15.43:0/3258231772 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c140cbf40 con 0x7f5c140cd470 2024-04-25T09:00:28.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.298+0000 7f5c1abe9640 1 --2- 172.21.15.43:0/3258231772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c140cd470 0x7f5c1406c4e0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f5c140d1790 tx=0x7f5c0402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=632525f5ab2b4768 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:28.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.298+0000 7f5c193e6640 1 -- 172.21.15.43:0/3258231772 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c04036070 con 0x7f5c140cd470 2024-04-25T09:00:28.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.298+0000 7f5c193e6640 1 -- 172.21.15.43:0/3258231772 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c04031600 con 0x7f5c140cd470 2024-04-25T09:00:28.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.298+0000 7f5c193e6640 1 -- 172.21.15.43:0/3258231772 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c040319d0 con 0x7f5c140cd470 2024-04-25T09:00:28.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.298+0000 7f5c1c64c640 1 -- 172.21.15.43:0/3258231772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c140cd470 msgr2=0x7f5c1406c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:28.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.298+0000 7f5c1c64c640 1 --2- 172.21.15.43:0/3258231772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c140cd470 0x7f5c1406c4e0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f5c140d1790 tx=0x7f5c0402a6f0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.299+0000 7f5c1c64c640 1 -- 172.21.15.43:0/3258231772 shutdown_connections 2024-04-25T09:00:28.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.299+0000 7f5c1c64c640 1 --2- 172.21.15.43:0/3258231772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c140cd470 0x7f5c1406c4e0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.299+0000 7f5c1c64c640 1 --2- 172.21.15.43:0/3258231772 >> v2:172.21.15.193:3300/0 conn(0x7f5c1406f6a0 0x7f5c140b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.299+0000 7f5c1c64c640 1 -- 172.21.15.43:0/3258231772 >> 172.21.15.43:0/3258231772 conn(0x7f5c140d17e0 msgr2=0x7f5c140d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:28.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.299+0000 7f5c1c64c640 1 -- 172.21.15.43:0/3258231772 shutdown_connections 2024-04-25T09:00:28.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.299+0000 7f5c1c64c640 1 -- 172.21.15.43:0/3258231772 wait complete. 2024-04-25T09:00:28.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.300+0000 7f5c1c64c640 1 Processor -- start 2024-04-25T09:00:28.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.300+0000 7f5c1c64c640 1 -- start start 2024-04-25T09:00:28.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.301+0000 7f5c1c64c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c1406f6a0 0x7f5c140b2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:28.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.301+0000 7f5c1c64c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c140cd470 0x7f5c140b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:28.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.301+0000 7f5c1c64c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c1415ddd0 0x7f5c14152280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:28.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.301+0000 7f5c1c64c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1406cdb0 con 0x7f5c1406f6a0 2024-04-25T09:00:28.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.301+0000 7f5c1c64c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1406cc30 con 0x7f5c140cd470 2024-04-25T09:00:28.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.301+0000 7f5c1c64c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1406cf30 con 0x7f5c1415ddd0 2024-04-25T09:00:28.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.301+0000 7f5c1a3e8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c1406f6a0 0x7f5c140b2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:28.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.301+0000 7f5c1a3e8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c1406f6a0 0x7f5c140b2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40142/0 (socket says 172.21.15.43:40142) 2024-04-25T09:00:28.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.301+0000 7f5c1a3e8640 1 -- 172.21.15.43:0/1671749218 learned_addr learned my addr 172.21.15.43:0/1671749218 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:28.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.301+0000 7f5c19be7640 1 --2- 172.21.15.43:0/1671749218 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c140cd470 0x7f5c140b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:28.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.301+0000 7f5c1abe9640 1 --2- 172.21.15.43:0/1671749218 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c1415ddd0 0x7f5c14152280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:28.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.301+0000 7f5c1a3e8640 1 -- 172.21.15.43:0/1671749218 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c1415ddd0 msgr2=0x7f5c14152280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:28.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.301+0000 7f5c1a3e8640 1 --2- 172.21.15.43:0/1671749218 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c1415ddd0 0x7f5c14152280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.301+0000 7f5c1a3e8640 1 -- 172.21.15.43:0/1671749218 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c140cd470 msgr2=0x7f5c140b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:28.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.301+0000 7f5c1a3e8640 1 --2- 172.21.15.43:0/1671749218 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c140cd470 0x7f5c140b10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.301+0000 7f5c1a3e8640 1 -- 172.21.15.43:0/1671749218 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c140b1840 con 0x7f5c1406f6a0 2024-04-25T09:00:28.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.301+0000 7f5c1abe9640 1 --2- 172.21.15.43:0/1671749218 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c1415ddd0 0x7f5c14152280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:00:28.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.301+0000 7f5c1a3e8640 1 --2- 172.21.15.43:0/1671749218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c1406f6a0 0x7f5c140b2b10 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c00b420 tx=0x7f5c0c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:28.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.301+0000 7f5c12ffd640 1 -- 172.21.15.43:0/1671749218 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c011830 con 0x7f5c1406f6a0 2024-04-25T09:00:28.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.301+0000 7f5c12ffd640 1 -- 172.21.15.43:0/1671749218 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0c016070 con 0x7f5c1406f6a0 2024-04-25T09:00:28.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.301+0000 7f5c12ffd640 1 -- 172.21.15.43:0/1671749218 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c011c10 con 0x7f5c1406f6a0 2024-04-25T09:00:28.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.301+0000 7f5c1c64c640 1 -- 172.21.15.43:0/1671749218 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c140b1b20 con 0x7f5c1406f6a0 2024-04-25T09:00:28.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.301+0000 7f5c1c64c640 1 -- 172.21.15.43:0/1671749218 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c141527c0 con 0x7f5c1406f6a0 2024-04-25T09:00:28.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.302+0000 7f5c1c64c640 1 -- 172.21.15.43:0/1671749218 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c140b2040 con 0x7f5c1406f6a0 2024-04-25T09:00:28.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.303+0000 7f5c12ffd640 1 -- 172.21.15.43:0/1671749218 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5c0c01b460 con 0x7f5c1406f6a0 2024-04-25T09:00:28.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.303+0000 7f5c12ffd640 1 --2- 172.21.15.43:0/1671749218 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5c0006b4d0 0x7f5c0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:28.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.303+0000 7f5c12ffd640 1 -- 172.21.15.43:0/1671749218 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f5c0c089c00 con 0x7f5c1406f6a0 2024-04-25T09:00:28.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.303+0000 7f5c19be7640 1 --2- 172.21.15.43:0/1671749218 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5c0006b4d0 0x7f5c0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:28.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.304+0000 7f5c19be7640 1 --2- 172.21.15.43:0/1671749218 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5c0006b4d0 0x7f5c0006d990 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5c140d24e0 tx=0x7f5c08008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:28.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.308+0000 7f5c12ffd640 1 -- 172.21.15.43:0/1671749218 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5c140b2040 con 0x7f5c1406f6a0 2024-04-25T09:00:28.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.490+0000 7f5c1c64c640 1 -- 172.21.15.43:0/1671749218 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5c140d2030 con 0x7f5c1406f6a0 2024-04-25T09:00:28.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.490+0000 7f5c12ffd640 1 -- 172.21.15.43:0/1671749218 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5c140d2030 con 0x7f5c1406f6a0 2024-04-25T09:00:28.480 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T09:00:28.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.492+0000 7f5c1c64c640 1 -- 172.21.15.43:0/1671749218 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5c0006b4d0 msgr2=0x7f5c0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:28.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.492+0000 7f5c1c64c640 1 --2- 172.21.15.43:0/1671749218 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5c0006b4d0 0x7f5c0006d990 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f5c140d24e0 tx=0x7f5c08008040 comp rx=0 tx=0).stop 2024-04-25T09:00:28.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.492+0000 7f5c1c64c640 1 -- 172.21.15.43:0/1671749218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c1406f6a0 msgr2=0x7f5c140b2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:28.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.492+0000 7f5c1c64c640 1 --2- 172.21.15.43:0/1671749218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c1406f6a0 0x7f5c140b2b10 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c00b420 tx=0x7f5c0c00b8e0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.492+0000 7f5c1c64c640 1 -- 172.21.15.43:0/1671749218 shutdown_connections 2024-04-25T09:00:28.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.492+0000 7f5c1c64c640 1 --2- 172.21.15.43:0/1671749218 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c1415ddd0 0x7f5c14152280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.492+0000 7f5c1c64c640 1 --2- 172.21.15.43:0/1671749218 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5c0006b4d0 0x7f5c0006d990 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.492+0000 7f5c1c64c640 1 --2- 172.21.15.43:0/1671749218 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c140cd470 0x7f5c140b10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.492+0000 7f5c1c64c640 1 --2- 172.21.15.43:0/1671749218 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c1406f6a0 0x7f5c140b2b10 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.492+0000 7f5c1c64c640 1 -- 172.21.15.43:0/1671749218 >> 172.21.15.43:0/1671749218 conn(0x7f5c140d17e0 msgr2=0x7f5c140ce860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:28.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.493+0000 7f5c1c64c640 1 -- 172.21.15.43:0/1671749218 shutdown_connections 2024-04-25T09:00:28.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.493+0000 7f5c1c64c640 1 -- 172.21.15.43:0/1671749218 wait complete. 2024-04-25T09:00:28.493 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:00:28.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.607+0000 7ffabb63f640 1 Processor -- start 2024-04-25T09:00:28.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.608+0000 7ffabb63f640 1 -- start start 2024-04-25T09:00:28.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.608+0000 7ffabb63f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffab415dec0 0x7ffab415ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:28.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.608+0000 7ffabb63f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffab40cf110 con 0x7ffab40d04a0 2024-04-25T09:00:28.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.608+0000 7ffabb63f640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffab40cf3f0 con 0x7ffab40ce8c0 2024-04-25T09:00:28.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.608+0000 7ffabb63f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffab40cf6d0 con 0x7ffab415dec0 2024-04-25T09:00:28.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.608+0000 7ffab93db640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ffab40d04a0 0x7ffab40d0880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32990/0 (socket says 172.21.15.43:32990) 2024-04-25T09:00:28.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.608+0000 7ffab93db640 1 -- 172.21.15.43:0/1754041970 learned_addr learned my addr 172.21.15.43:0/1754041970 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:00:28.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.609+0000 7ffab8bda640 1 --2- 172.21.15.43:0/1754041970 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffab415dec0 0x7ffab415ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:28.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.609+0000 7ffaabfff640 1 -- 172.21.15.43:0/1754041970 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2766841072 0 0) 0x7ffab40cf110 con 0x7ffab40d04a0 2024-04-25T09:00:28.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.609+0000 7ffaabfff640 1 -- 172.21.15.43:0/1754041970 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffaa0003520 con 0x7ffab40d04a0 2024-04-25T09:00:28.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.609+0000 7ffaabfff640 1 -- 172.21.15.43:0/1754041970 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1779347075 0 0) 0x7ffab40cf3f0 con 0x7ffab40ce8c0 2024-04-25T09:00:28.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.609+0000 7ffab8bda640 1 -- 172.21.15.43:0/1754041970 >> v1:172.21.15.193:6789/0 conn(0x7ffab40ce8c0 legacy=0x7ffab40d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:28.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.609+0000 7ffab8bda640 1 -- 172.21.15.43:0/1754041970 >> v1:172.21.15.43:6789/0 conn(0x7ffab40d04a0 legacy=0x7ffab40d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:28.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.609+0000 7ffab8bda640 1 -- 172.21.15.43:0/1754041970 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab40cf9b0 con 0x7ffab415dec0 2024-04-25T09:00:28.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.610+0000 7ffab8bda640 1 --2- 172.21.15.43:0/1754041970 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffab415dec0 0x7ffab415ed90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ffaa40097d0 tx=0x7ffaa402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=660ef18325e989cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:28.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.610+0000 7ffaabfff640 1 -- 172.21.15.43:0/1754041970 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4037070 con 0x7ffab415dec0 2024-04-25T09:00:28.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.610+0000 7ffaabfff640 1 -- 172.21.15.43:0/1754041970 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaa402ac10 con 0x7ffab415dec0 2024-04-25T09:00:28.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.610+0000 7ffaabfff640 1 -- 172.21.15.43:0/1754041970 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab00015f0 con 0x7ffab415dec0 2024-04-25T09:00:28.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.611+0000 7ffabb63f640 1 -- 172.21.15.43:0/1754041970 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffab415dec0 msgr2=0x7ffab415ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:28.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.611+0000 7ffabb63f640 1 --2- 172.21.15.43:0/1754041970 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffab415dec0 0x7ffab415ed90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ffaa40097d0 tx=0x7ffaa402a180 comp rx=0 tx=0).stop 2024-04-25T09:00:28.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.611+0000 7ffabb63f640 1 -- 172.21.15.43:0/1754041970 shutdown_connections 2024-04-25T09:00:28.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.611+0000 7ffabb63f640 1 --2- 172.21.15.43:0/1754041970 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffab415dec0 0x7ffab415ed90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.611+0000 7ffabb63f640 1 -- 172.21.15.43:0/1754041970 >> 172.21.15.43:0/1754041970 conn(0x7ffab409cd40 msgr2=0x7ffab40d3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:28.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.611+0000 7ffabb63f640 1 -- 172.21.15.43:0/1754041970 shutdown_connections 2024-04-25T09:00:28.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.611+0000 7ffabb63f640 1 -- 172.21.15.43:0/1754041970 wait complete. 2024-04-25T09:00:28.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.612+0000 7ffabb63f640 1 Processor -- start 2024-04-25T09:00:28.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.612+0000 7ffabb63f640 1 -- start start 2024-04-25T09:00:28.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.613+0000 7ffabb63f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffab40ce8c0 0x7ffab40cc750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:28.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.613+0000 7ffabb63f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffab40d04a0 0x7ffab4151c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:28.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.613+0000 7ffabb63f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffab415dec0 0x7ffab4150630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:28.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.613+0000 7ffabb63f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab40cee70 con 0x7ffab415dec0 2024-04-25T09:00:28.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.613+0000 7ffabb63f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab40cecf0 con 0x7ffab40ce8c0 2024-04-25T09:00:28.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.613+0000 7ffabb63f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffab40d1eb0 con 0x7ffab40d04a0 2024-04-25T09:00:28.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.613+0000 7ffab9bdc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffab415dec0 0x7ffab4150630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:28.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.613+0000 7ffab9bdc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffab415dec0 0x7ffab4150630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40156/0 (socket says 172.21.15.43:40156) 2024-04-25T09:00:28.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.613+0000 7ffab9bdc640 1 -- 172.21.15.43:0/2090444076 learned_addr learned my addr 172.21.15.43:0/2090444076 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:28.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.613+0000 7ffab93db640 1 --2- 172.21.15.43:0/2090444076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffab40ce8c0 0x7ffab40cc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:28.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.613+0000 7ffab8bda640 1 --2- 172.21.15.43:0/2090444076 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffab40d04a0 0x7ffab4151c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:28.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.613+0000 7ffab9bdc640 1 -- 172.21.15.43:0/2090444076 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffab40d04a0 msgr2=0x7ffab4151c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:28.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.613+0000 7ffab9bdc640 1 --2- 172.21.15.43:0/2090444076 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffab40d04a0 0x7ffab4151c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.613+0000 7ffab9bdc640 1 -- 172.21.15.43:0/2090444076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffab40ce8c0 msgr2=0x7ffab40cc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:28.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.613+0000 7ffab9bdc640 1 --2- 172.21.15.43:0/2090444076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffab40ce8c0 0x7ffab40cc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.613+0000 7ffab9bdc640 1 -- 172.21.15.43:0/2090444076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab40b31d0 con 0x7ffab415dec0 2024-04-25T09:00:28.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.613+0000 7ffab9bdc640 1 --2- 172.21.15.43:0/2090444076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffab415dec0 0x7ffab4150630 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00c810 tx=0x7ffaac00ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:28.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.613+0000 7ffaa9ffb640 1 -- 172.21.15.43:0/2090444076 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac017070 con 0x7ffab415dec0 2024-04-25T09:00:28.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.613+0000 7ffaa9ffb640 1 -- 172.21.15.43:0/2090444076 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaac003da0 con 0x7ffab415dec0 2024-04-25T09:00:28.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.613+0000 7ffabb63f640 1 -- 172.21.15.43:0/2090444076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffab40b34b0 con 0x7ffab415dec0 2024-04-25T09:00:28.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.614+0000 7ffaa9ffb640 1 -- 172.21.15.43:0/2090444076 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac005290 con 0x7ffab415dec0 2024-04-25T09:00:28.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.614+0000 7ffabb63f640 1 -- 172.21.15.43:0/2090444076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffab40b3a70 con 0x7ffab415dec0 2024-04-25T09:00:28.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.615+0000 7ffabb63f640 1 -- 172.21.15.43:0/2090444076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffab40b3c70 con 0x7ffab415dec0 2024-04-25T09:00:28.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.615+0000 7ffaa9ffb640 1 -- 172.21.15.43:0/2090444076 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffaac01a070 con 0x7ffab415dec0 2024-04-25T09:00:28.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.616+0000 7ffaa9ffb640 1 --2- 172.21.15.43:0/2090444076 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ffa9806b4d0 0x7ffa9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:28.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.616+0000 7ffaa9ffb640 1 -- 172.21.15.43:0/2090444076 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ffaac0898d0 con 0x7ffab415dec0 2024-04-25T09:00:28.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.616+0000 7ffab93db640 1 --2- 172.21.15.43:0/2090444076 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ffa9806b4d0 0x7ffa9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:28.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.616+0000 7ffab93db640 1 --2- 172.21.15.43:0/2090444076 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ffa9806b4d0 0x7ffa9806d990 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ffab0001540 tx=0x7ffab00014b0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:28.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.620+0000 7ffaa9ffb640 1 -- 172.21.15.43:0/2090444076 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffaac0580b0 con 0x7ffab415dec0 2024-04-25T09:00:28.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.800+0000 7ffabb63f640 1 -- 172.21.15.43:0/2090444076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ffab408daf0 con 0x7ffab415dec0 2024-04-25T09:00:28.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.801+0000 7ffaa9ffb640 1 -- 172.21.15.43:0/2090444076 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) v1 ==== 100+0+11 (secure 0 0 0) 0x7ffaac05c150 con 0x7ffab415dec0 2024-04-25T09:00:28.790 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T09:00:28.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.803+0000 7ffabb63f640 1 -- 172.21.15.43:0/2090444076 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ffa9806b4d0 msgr2=0x7ffa9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:28.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.803+0000 7ffabb63f640 1 --2- 172.21.15.43:0/2090444076 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ffa9806b4d0 0x7ffa9806d990 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7ffab0001540 tx=0x7ffab00014b0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.803+0000 7ffabb63f640 1 -- 172.21.15.43:0/2090444076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffab415dec0 msgr2=0x7ffab4150630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:28.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.803+0000 7ffabb63f640 1 --2- 172.21.15.43:0/2090444076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffab415dec0 0x7ffab4150630 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00c810 tx=0x7ffaac00ccd0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.803+0000 7ffabb63f640 1 -- 172.21.15.43:0/2090444076 shutdown_connections 2024-04-25T09:00:28.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.803+0000 7ffabb63f640 1 --2- 172.21.15.43:0/2090444076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffab415dec0 0x7ffab4150630 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.803+0000 7ffabb63f640 1 --2- 172.21.15.43:0/2090444076 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffab40d04a0 0x7ffab4151c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.803+0000 7ffabb63f640 1 --2- 172.21.15.43:0/2090444076 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ffa9806b4d0 0x7ffa9806d990 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.803+0000 7ffabb63f640 1 --2- 172.21.15.43:0/2090444076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffab40ce8c0 0x7ffab40cc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.803+0000 7ffabb63f640 1 -- 172.21.15.43:0/2090444076 >> 172.21.15.43:0/2090444076 conn(0x7ffab409cd40 msgr2=0x7ffab40b0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:28.794 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.803+0000 7ffabb63f640 1 -- 172.21.15.43:0/2090444076 shutdown_connections 2024-04-25T09:00:28.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.803+0000 7ffabb63f640 1 -- 172.21.15.43:0/2090444076 wait complete. 2024-04-25T09:00:28.804 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:00:28.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.916+0000 7fd4f2fed640 1 Processor -- start 2024-04-25T09:00:28.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.916+0000 7fd4f2fed640 1 -- start start 2024-04-25T09:00:28.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.917+0000 7fd4f2fed640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd4ec15dce0 0x7fd4ec15ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:28.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.917+0000 7fd4f2fed640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd4ec0d0af0 0x7fd4ec160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:28.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.917+0000 7fd4f2fed640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4ec0cee80 con 0x7fd4ec0ce550 2024-04-25T09:00:28.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.917+0000 7fd4f2fed640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4ec0d2920 con 0x7fd4ec15dce0 2024-04-25T09:00:28.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.917+0000 7fd4f2fed640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec0cf160 con 0x7fd4ec0d0af0 2024-04-25T09:00:28.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.917+0000 7fd4f0d89640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd4ec15dce0 0x7fd4ec15ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:28.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.917+0000 7fd4f0d89640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd4ec15dce0 0x7fd4ec15ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40160/0 (socket says 172.21.15.43:40160) 2024-04-25T09:00:28.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.917+0000 7fd4f0d89640 1 -- 172.21.15.43:0/85776987 learned_addr learned my addr 172.21.15.43:0/85776987 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:28.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.917+0000 7fd4f158a640 1 --2- 172.21.15.43:0/85776987 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd4ec0d0af0 0x7fd4ec160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:28.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.917+0000 7fd4f0d89640 1 -- 172.21.15.43:0/85776987 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd4ec0d0af0 msgr2=0x7fd4ec160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:28.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.917+0000 7fd4f0d89640 1 --2- 172.21.15.43:0/85776987 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd4ec0d0af0 0x7fd4ec160980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.917+0000 7fd4f0d89640 1 -- 172.21.15.43:0/85776987 >> v1:172.21.15.193:6789/0 conn(0x7fd4ec0ce550 legacy=0x7fd4ec0d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:28.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.917+0000 7fd4f0d89640 1 -- 172.21.15.43:0/85776987 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ec0cf440 con 0x7fd4ec15dce0 2024-04-25T09:00:28.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.918+0000 7fd4f0d89640 1 --2- 172.21.15.43:0/85776987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ec15dce0 0x7fd4ec15ef50 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8009870 tx=0x7fd4e802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5a6c5c1796a89aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:28.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.918+0000 7fd4e37fe640 1 -- 172.21.15.43:0/85776987 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8037070 con 0x7fd4ec15dce0 2024-04-25T09:00:28.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.918+0000 7fd4e37fe640 1 -- 172.21.15.43:0/85776987 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e8032440 con 0x7fd4ec15dce0 2024-04-25T09:00:28.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.918+0000 7fd4e37fe640 1 -- 172.21.15.43:0/85776987 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8032810 con 0x7fd4ec15dce0 2024-04-25T09:00:28.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.919+0000 7fd4f2fed640 1 -- 172.21.15.43:0/85776987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ec15dce0 msgr2=0x7fd4ec15ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:28.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.919+0000 7fd4f2fed640 1 --2- 172.21.15.43:0/85776987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ec15dce0 0x7fd4ec15ef50 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8009870 tx=0x7fd4e802a1c0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.919+0000 7fd4f2fed640 1 -- 172.21.15.43:0/85776987 shutdown_connections 2024-04-25T09:00:28.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.919+0000 7fd4f2fed640 1 --2- 172.21.15.43:0/85776987 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd4ec0d0af0 0x7fd4ec160980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.919+0000 7fd4f2fed640 1 --2- 172.21.15.43:0/85776987 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ec15dce0 0x7fd4ec15ef50 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.919+0000 7fd4f2fed640 1 -- 172.21.15.43:0/85776987 >> 172.21.15.43:0/85776987 conn(0x7fd4ec09cd00 msgr2=0x7fd4ec0d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:28.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.919+0000 7fd4f2fed640 1 -- 172.21.15.43:0/85776987 shutdown_connections 2024-04-25T09:00:28.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.919+0000 7fd4f2fed640 1 -- 172.21.15.43:0/85776987 wait complete. 2024-04-25T09:00:28.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.920+0000 7fd4f2fed640 1 Processor -- start 2024-04-25T09:00:28.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.920+0000 7fd4f2fed640 1 -- start start 2024-04-25T09:00:28.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.921+0000 7fd4f2fed640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd4ec0ce550 0x7fd4ec15a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:28.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.921+0000 7fd4f2fed640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4ec0d0af0 0x7fd4ec0b7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:28.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.921+0000 7fd4f2fed640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ec15dce0 0x7fd4ec156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:28.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.921+0000 7fd4f2fed640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec15e0c0 con 0x7fd4ec15dce0 2024-04-25T09:00:28.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.921+0000 7fd4f2fed640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec15f6b0 con 0x7fd4ec0d0af0 2024-04-25T09:00:28.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.921+0000 7fd4f2fed640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec0ce9c0 con 0x7fd4ec0ce550 2024-04-25T09:00:28.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.921+0000 7fd4f158a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ec15dce0 0x7fd4ec156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:28.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.921+0000 7fd4f158a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ec15dce0 0x7fd4ec156b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40162/0 (socket says 172.21.15.43:40162) 2024-04-25T09:00:28.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.921+0000 7fd4f158a640 1 -- 172.21.15.43:0/832887653 learned_addr learned my addr 172.21.15.43:0/832887653 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:28.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.921+0000 7fd4f0d89640 1 --2- 172.21.15.43:0/832887653 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd4ec0ce550 0x7fd4ec15a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:28.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.921+0000 7fd4e3fff640 1 --2- 172.21.15.43:0/832887653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4ec0d0af0 0x7fd4ec0b7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:28.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.921+0000 7fd4f158a640 1 -- 172.21.15.43:0/832887653 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd4ec0ce550 msgr2=0x7fd4ec15a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:28.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.921+0000 7fd4f158a640 1 --2- 172.21.15.43:0/832887653 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd4ec0ce550 0x7fd4ec15a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.921+0000 7fd4f158a640 1 -- 172.21.15.43:0/832887653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4ec0d0af0 msgr2=0x7fd4ec0b7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:28.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.921+0000 7fd4f158a640 1 --2- 172.21.15.43:0/832887653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4ec0d0af0 0x7fd4ec0b7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:28.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.921+0000 7fd4f158a640 1 -- 172.21.15.43:0/832887653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ec158170 con 0x7fd4ec15dce0 2024-04-25T09:00:28.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.922+0000 7fd4f158a640 1 --2- 172.21.15.43:0/832887653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ec15dce0 0x7fd4ec156b00 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4009fd0 tx=0x7fd4e400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:28.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.922+0000 7fd4e17fa640 1 -- 172.21.15.43:0/832887653 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4014420 con 0x7fd4ec15dce0 2024-04-25T09:00:28.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.922+0000 7fd4e17fa640 1 -- 172.21.15.43:0/832887653 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e4019070 con 0x7fd4ec15dce0 2024-04-25T09:00:28.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.922+0000 7fd4f2fed640 1 -- 172.21.15.43:0/832887653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4ec157a80 con 0x7fd4ec15dce0 2024-04-25T09:00:28.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.922+0000 7fd4e17fa640 1 -- 172.21.15.43:0/832887653 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4014800 con 0x7fd4ec15dce0 2024-04-25T09:00:28.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.922+0000 7fd4f2fed640 1 -- 172.21.15.43:0/832887653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4ec1576e0 con 0x7fd4ec15dce0 2024-04-25T09:00:28.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.923+0000 7fd4f2fed640 1 -- 172.21.15.43:0/832887653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4ec0680f0 con 0x7fd4ec15dce0 2024-04-25T09:00:28.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.923+0000 7fd4e17fa640 1 -- 172.21.15.43:0/832887653 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd4e40149a0 con 0x7fd4ec15dce0 2024-04-25T09:00:28.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.924+0000 7fd4e17fa640 1 --2- 172.21.15.43:0/832887653 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd4d806b500 0x7fd4d806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:28.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.924+0000 7fd4e17fa640 1 -- 172.21.15.43:0/832887653 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fd4e408aa00 con 0x7fd4ec15dce0 2024-04-25T09:00:28.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.924+0000 7fd4f0d89640 1 --2- 172.21.15.43:0/832887653 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd4d806b500 0x7fd4d806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:28.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.925+0000 7fd4f0d89640 1 --2- 172.21.15.43:0/832887653 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd4d806b500 0x7fd4d806d9c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8034000 tx=0x7fd4e802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:28.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:28.928+0000 7fd4e17fa640 1 -- 172.21.15.43:0/832887653 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd4e4059130 con 0x7fd4ec15dce0 2024-04-25T09:00:29.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.144+0000 7fd4f2fed640 1 -- 172.21.15.43:0/832887653 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4ec158670 con 0x7fd4ec15dce0 2024-04-25T09:00:29.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.145+0000 7fd4e17fa640 1 -- 172.21.15.43:0/832887653 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fd4ec158670 con 0x7fd4ec15dce0 2024-04-25T09:00:29.134 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:00:29.134 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":48,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6373,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3648859409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":3648859409},{"type":"v1","addr":"172.21.15.193:6837","nonce":3648859409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":7149,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/356516962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":356516962},{"type":"v1","addr":"172.21.15.43:6834","nonce":356516962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":7152,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3032637719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":3032637719},{"type":"v1","addr":"172.21.15.43:6835","nonce":3032637719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T08:58:45.906679+0000","modified":"2024-04-25T08:58:54.057379+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7164},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7164":{"gid":7164,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.193:6836/3458098162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":3458098162},{"type":"v1","addr":"172.21.15.193:6836","nonce":3458098162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T09:00:29.134 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 48 2024-04-25T09:00:29.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.147+0000 7fd4f2fed640 1 -- 172.21.15.43:0/832887653 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd4d806b500 msgr2=0x7fd4d806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:29.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.147+0000 7fd4f2fed640 1 --2- 172.21.15.43:0/832887653 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd4d806b500 0x7fd4d806d9c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8034000 tx=0x7fd4e802e000 comp rx=0 tx=0).stop 2024-04-25T09:00:29.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.147+0000 7fd4f2fed640 1 -- 172.21.15.43:0/832887653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ec15dce0 msgr2=0x7fd4ec156b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:29.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.147+0000 7fd4f2fed640 1 --2- 172.21.15.43:0/832887653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ec15dce0 0x7fd4ec156b00 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4009fd0 tx=0x7fd4e400e4b0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.147+0000 7fd4f2fed640 1 -- 172.21.15.43:0/832887653 shutdown_connections 2024-04-25T09:00:29.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.147+0000 7fd4f2fed640 1 --2- 172.21.15.43:0/832887653 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd4ec15dce0 0x7fd4ec156b00 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.147+0000 7fd4f2fed640 1 --2- 172.21.15.43:0/832887653 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd4d806b500 0x7fd4d806d9c0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.147+0000 7fd4f2fed640 1 --2- 172.21.15.43:0/832887653 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd4ec0d0af0 0x7fd4ec0b7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.147+0000 7fd4f2fed640 1 --2- 172.21.15.43:0/832887653 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd4ec0ce550 0x7fd4ec15a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.147+0000 7fd4f2fed640 1 -- 172.21.15.43:0/832887653 >> 172.21.15.43:0/832887653 conn(0x7fd4ec09cd00 msgr2=0x7fd4ec15e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:29.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.147+0000 7fd4f2fed640 1 -- 172.21.15.43:0/832887653 shutdown_connections 2024-04-25T09:00:29.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.147+0000 7fd4f2fed640 1 -- 172.21.15.43:0/832887653 wait complete. 2024-04-25T09:00:29.147 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:00:29.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.258+0000 7fbd2b510640 1 Processor -- start 2024-04-25T09:00:29.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.259+0000 7fbd2b510640 1 -- start start 2024-04-25T09:00:29.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.259+0000 7fbd2b510640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fbd2415dce0 0x7fbd2415ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.259+0000 7fbd2b510640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd240d0af0 0x7fbd24160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.259+0000 7fbd2b510640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd240cee80 con 0x7fbd240ce550 2024-04-25T09:00:29.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.259+0000 7fbd2b510640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd240d2920 con 0x7fbd2415dce0 2024-04-25T09:00:29.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.259+0000 7fbd2b510640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd240cf160 con 0x7fbd240d0af0 2024-04-25T09:00:29.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.260+0000 7fbd29aad640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd240d0af0 0x7fbd24160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.260+0000 7fbd29aad640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd240d0af0 0x7fbd24160980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:37814/0 (socket says 172.21.15.43:37814) 2024-04-25T09:00:29.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.260+0000 7fbd292ac640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fbd2415dce0 0x7fbd2415ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.260+0000 7fbd29aad640 1 -- 172.21.15.43:0/2415360861 learned_addr learned my addr 172.21.15.43:0/2415360861 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:29.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.260+0000 7fbd29aad640 1 -- 172.21.15.43:0/2415360861 >> v2:172.21.15.193:3300/0 conn(0x7fbd2415dce0 msgr2=0x7fbd2415ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:29.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.260+0000 7fbd29aad640 1 --2- 172.21.15.43:0/2415360861 >> v2:172.21.15.193:3300/0 conn(0x7fbd2415dce0 0x7fbd2415ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.260+0000 7fbd29aad640 1 -- 172.21.15.43:0/2415360861 >> v1:172.21.15.193:6789/0 conn(0x7fbd240ce550 legacy=0x7fbd240d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:29.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.260+0000 7fbd1bfff640 1 -- 172.21.15.43:0/2415360861 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2977077546 0 0) 0x7fbd240cee80 con 0x7fbd240ce550 2024-04-25T09:00:29.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.260+0000 7fbd29aad640 1 -- 172.21.15.43:0/2415360861 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd240cf440 con 0x7fbd240d0af0 2024-04-25T09:00:29.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.260+0000 7fbd292ac640 1 --2- 172.21.15.43:0/2415360861 >> v2:172.21.15.193:3300/0 conn(0x7fbd2415dce0 0x7fbd2415ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:00:29.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.261+0000 7fbd29aad640 1 --2- 172.21.15.43:0/2415360861 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd240d0af0 0x7fbd24160980 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c009e50 tx=0x7fbd1c02a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ff2467d361c01cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:29.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.261+0000 7fbd1bfff640 1 -- 172.21.15.43:0/2415360861 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c036070 con 0x7fbd240d0af0 2024-04-25T09:00:29.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.261+0000 7fbd1bfff640 1 -- 172.21.15.43:0/2415360861 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd1c031600 con 0x7fbd240d0af0 2024-04-25T09:00:29.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.261+0000 7fbd1bfff640 1 -- 172.21.15.43:0/2415360861 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1c0319d0 con 0x7fbd240d0af0 2024-04-25T09:00:29.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.262+0000 7fbd2b510640 1 -- 172.21.15.43:0/2415360861 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd240d0af0 msgr2=0x7fbd24160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:29.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.262+0000 7fbd2b510640 1 --2- 172.21.15.43:0/2415360861 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd240d0af0 0x7fbd24160980 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fbd1c009e50 tx=0x7fbd1c02a680 comp rx=0 tx=0).stop 2024-04-25T09:00:29.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.262+0000 7fbd2b510640 1 -- 172.21.15.43:0/2415360861 shutdown_connections 2024-04-25T09:00:29.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.262+0000 7fbd2b510640 1 --2- 172.21.15.43:0/2415360861 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd240d0af0 0x7fbd24160980 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.262+0000 7fbd2b510640 1 --2- 172.21.15.43:0/2415360861 >> v2:172.21.15.193:3300/0 conn(0x7fbd2415dce0 0x7fbd2415ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.262+0000 7fbd2b510640 1 -- 172.21.15.43:0/2415360861 >> 172.21.15.43:0/2415360861 conn(0x7fbd2409cd00 msgr2=0x7fbd240d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:29.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.262+0000 7fbd2b510640 1 -- 172.21.15.43:0/2415360861 shutdown_connections 2024-04-25T09:00:29.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.262+0000 7fbd2b510640 1 -- 172.21.15.43:0/2415360861 wait complete. 2024-04-25T09:00:29.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.263+0000 7fbd2b510640 1 Processor -- start 2024-04-25T09:00:29.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.263+0000 7fbd2b510640 1 -- start start 2024-04-25T09:00:29.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.264+0000 7fbd2b510640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd240ce550 0x7fbd241f78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.264+0000 7fbd2b510640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd240d0af0 0x7fbd240caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.264+0000 7fbd2b510640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd2415dce0 0x7fbd240b7d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.264+0000 7fbd2b510640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2415e0c0 con 0x7fbd240d0af0 2024-04-25T09:00:29.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.264+0000 7fbd2b510640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2415f6b0 con 0x7fbd2415dce0 2024-04-25T09:00:29.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.264+0000 7fbd2b510640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd240ce9c0 con 0x7fbd240ce550 2024-04-25T09:00:29.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.264+0000 7fbd28aab640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd240d0af0 0x7fbd240caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.264+0000 7fbd28aab640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd240d0af0 0x7fbd240caaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40176/0 (socket says 172.21.15.43:40176) 2024-04-25T09:00:29.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.264+0000 7fbd28aab640 1 -- 172.21.15.43:0/2583238124 learned_addr learned my addr 172.21.15.43:0/2583238124 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:29.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.264+0000 7fbd292ac640 1 --2- 172.21.15.43:0/2583238124 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd240ce550 0x7fbd241f78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.264+0000 7fbd29aad640 1 --2- 172.21.15.43:0/2583238124 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd2415dce0 0x7fbd240b7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.264+0000 7fbd28aab640 1 -- 172.21.15.43:0/2583238124 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd240ce550 msgr2=0x7fbd241f78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:29.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.264+0000 7fbd28aab640 1 --2- 172.21.15.43:0/2583238124 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd240ce550 0x7fbd241f78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.264+0000 7fbd28aab640 1 -- 172.21.15.43:0/2583238124 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd2415dce0 msgr2=0x7fbd240b7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:29.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.264+0000 7fbd28aab640 1 --2- 172.21.15.43:0/2583238124 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd2415dce0 0x7fbd240b7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.264+0000 7fbd28aab640 1 -- 172.21.15.43:0/2583238124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd240cb240 con 0x7fbd240d0af0 2024-04-25T09:00:29.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.264+0000 7fbd29aad640 1 --2- 172.21.15.43:0/2583238124 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd2415dce0 0x7fbd240b7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:29.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.264+0000 7fbd28aab640 1 --2- 172.21.15.43:0/2583238124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd240d0af0 0x7fbd240caaa0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fbd1400c520 tx=0x7fbd1400c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:29.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.264+0000 7fbd19ffb640 1 -- 172.21.15.43:0/2583238124 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd14017070 con 0x7fbd240d0af0 2024-04-25T09:00:29.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.264+0000 7fbd19ffb640 1 -- 172.21.15.43:0/2583238124 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd140052d0 con 0x7fbd240d0af0 2024-04-25T09:00:29.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.265+0000 7fbd2b510640 1 -- 172.21.15.43:0/2583238124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd240cb520 con 0x7fbd240d0af0 2024-04-25T09:00:29.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.265+0000 7fbd19ffb640 1 -- 172.21.15.43:0/2583238124 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd14012870 con 0x7fbd240d0af0 2024-04-25T09:00:29.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.265+0000 7fbd2b510640 1 -- 172.21.15.43:0/2583238124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd240b7800 con 0x7fbd240d0af0 2024-04-25T09:00:29.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.266+0000 7fbd2b510640 1 -- 172.21.15.43:0/2583238124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd240680f0 con 0x7fbd240d0af0 2024-04-25T09:00:29.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.266+0000 7fbd19ffb640 1 -- 172.21.15.43:0/2583238124 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbd1401a070 con 0x7fbd240d0af0 2024-04-25T09:00:29.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.266+0000 7fbd19ffb640 1 --2- 172.21.15.43:0/2583238124 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbd0806b4d0 0x7fbd0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.267+0000 7fbd19ffb640 1 -- 172.21.15.43:0/2583238124 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fbd140895d0 con 0x7fbd240d0af0 2024-04-25T09:00:29.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.267+0000 7fbd292ac640 1 --2- 172.21.15.43:0/2583238124 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbd0806b4d0 0x7fbd0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.267+0000 7fbd292ac640 1 --2- 172.21.15.43:0/2583238124 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbd0806b4d0 0x7fbd0806d990 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fbd20009a30 tx=0x7fbd20008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:29.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.271+0000 7fbd19ffb640 1 -- 172.21.15.43:0/2583238124 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbd14058400 con 0x7fbd240d0af0 2024-04-25T09:00:29.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.450+0000 7fbd2b510640 1 -- 172.21.15.43:0/2583238124 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbd24074ef0 con 0x7fbd240d0af0 2024-04-25T09:00:29.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.451+0000 7fbd19ffb640 1 -- 172.21.15.43:0/2583238124 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+16795 (secure 0 0 0) 0x7fbd14057d00 con 0x7fbd240d0af0 2024-04-25T09:00:29.441 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:00:29.441 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":77,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:00:02.408776+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:58:44.188924+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T08:58:44.877113+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":76,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.1","mappings":[{"from":6,"to":0}]},{"pgid":"11.2e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:00:29.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.453+0000 7fbd2b510640 1 -- 172.21.15.43:0/2583238124 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbd0806b4d0 msgr2=0x7fbd0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:29.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.453+0000 7fbd2b510640 1 --2- 172.21.15.43:0/2583238124 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbd0806b4d0 0x7fbd0806d990 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fbd20009a30 tx=0x7fbd20008040 comp rx=0 tx=0).stop 2024-04-25T09:00:29.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.453+0000 7fbd2b510640 1 -- 172.21.15.43:0/2583238124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd240d0af0 msgr2=0x7fbd240caaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:29.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.454+0000 7fbd2b510640 1 --2- 172.21.15.43:0/2583238124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd240d0af0 0x7fbd240caaa0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fbd1400c520 tx=0x7fbd1400c9e0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.454+0000 7fbd2b510640 1 -- 172.21.15.43:0/2583238124 shutdown_connections 2024-04-25T09:00:29.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.454+0000 7fbd2b510640 1 --2- 172.21.15.43:0/2583238124 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbd0806b4d0 0x7fbd0806d990 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.454+0000 7fbd2b510640 1 --2- 172.21.15.43:0/2583238124 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd2415dce0 0x7fbd240b7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.454+0000 7fbd2b510640 1 --2- 172.21.15.43:0/2583238124 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd240d0af0 0x7fbd240caaa0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.454+0000 7fbd2b510640 1 --2- 172.21.15.43:0/2583238124 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd240ce550 0x7fbd241f78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.454+0000 7fbd2b510640 1 -- 172.21.15.43:0/2583238124 >> 172.21.15.43:0/2583238124 conn(0x7fbd2409cd00 msgr2=0x7fbd240d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:29.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.454+0000 7fbd2b510640 1 -- 172.21.15.43:0/2583238124 shutdown_connections 2024-04-25T09:00:29.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.454+0000 7fbd2b510640 1 -- 172.21.15.43:0/2583238124 wait complete. 2024-04-25T09:00:29.454 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:00:29.454 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:00:29.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.567+0000 7fb12a61f640 1 Processor -- start 2024-04-25T09:00:29.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.568+0000 7fb12a61f640 1 -- start start 2024-04-25T09:00:29.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.568+0000 7fb12a61f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb12415df70 0x7fb1240cc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.568+0000 7fb12a61f640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb1240b0870 0x7fb1240ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.568+0000 7fb12a61f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1240cbee0 con 0x7fb12406c510 2024-04-25T09:00:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.568+0000 7fb12a61f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1240d0400 con 0x7fb12415df70 2024-04-25T09:00:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.568+0000 7fb12a61f640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1240d0650 con 0x7fb1240b0870 2024-04-25T09:00:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.568+0000 7fb1237fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb12415df70 0x7fb1240cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.568+0000 7fb123fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb12406c510 0x7fb12406c8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32998/0 (socket says 172.21.15.43:32998) 2024-04-25T09:00:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.568+0000 7fb123fff640 1 -- 172.21.15.43:0/4285557634 learned_addr learned my addr 172.21.15.43:0/4285557634 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:00:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.569+0000 7fb128bbc640 1 --2- 172.21.15.43:0/4285557634 >> v2:172.21.15.193:3300/0 conn(0x7fb1240b0870 0x7fb1240ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.569+0000 7fb1237fe640 1 -- 172.21.15.43:0/4285557634 >> v2:172.21.15.193:3300/0 conn(0x7fb1240b0870 msgr2=0x7fb1240ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.569+0000 7fb1237fe640 1 --2- 172.21.15.43:0/4285557634 >> v2:172.21.15.193:3300/0 conn(0x7fb1240b0870 0x7fb1240ccbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.569+0000 7fb1237fe640 1 -- 172.21.15.43:0/4285557634 >> v1:172.21.15.43:6789/0 conn(0x7fb12406c510 legacy=0x7fb12406c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.569+0000 7fb122ffd640 1 -- 172.21.15.43:0/4285557634 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1725281779 0 0) 0x7fb1240cbee0 con 0x7fb12406c510 2024-04-25T09:00:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.569+0000 7fb1237fe640 1 -- 172.21.15.43:0/4285557634 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1240d0930 con 0x7fb12415df70 2024-04-25T09:00:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.569+0000 7fb1237fe640 1 --2- 172.21.15.43:0/4285557634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb12415df70 0x7fb1240cc670 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fb1140097a0 tx=0x7fb11402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffefa83a1e606913 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:29.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.570+0000 7fb122ffd640 1 -- 172.21.15.43:0/4285557634 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb114037070 con 0x7fb12415df70 2024-04-25T09:00:29.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.570+0000 7fb122ffd640 1 -- 172.21.15.43:0/4285557634 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb11402ae50 con 0x7fb12415df70 2024-04-25T09:00:29.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.570+0000 7fb122ffd640 1 -- 172.21.15.43:0/4285557634 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb114032660 con 0x7fb12415df70 2024-04-25T09:00:29.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.570+0000 7fb12a61f640 1 -- 172.21.15.43:0/4285557634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb12415df70 msgr2=0x7fb1240cc670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:29.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.570+0000 7fb12a61f640 1 --2- 172.21.15.43:0/4285557634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb12415df70 0x7fb1240cc670 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fb1140097a0 tx=0x7fb11402a140 comp rx=0 tx=0).stop 2024-04-25T09:00:29.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.570+0000 7fb12a61f640 1 -- 172.21.15.43:0/4285557634 shutdown_connections 2024-04-25T09:00:29.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.570+0000 7fb12a61f640 1 --2- 172.21.15.43:0/4285557634 >> v2:172.21.15.193:3300/0 conn(0x7fb1240b0870 0x7fb1240ccbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.570+0000 7fb12a61f640 1 --2- 172.21.15.43:0/4285557634 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb12415df70 0x7fb1240cc670 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.570+0000 7fb12a61f640 1 -- 172.21.15.43:0/4285557634 >> 172.21.15.43:0/4285557634 conn(0x7fb1240d17f0 msgr2=0x7fb1240d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:29.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.570+0000 7fb12a61f640 1 -- 172.21.15.43:0/4285557634 shutdown_connections 2024-04-25T09:00:29.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.570+0000 7fb12a61f640 1 -- 172.21.15.43:0/4285557634 wait complete. 2024-04-25T09:00:29.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.571+0000 7fb12a61f640 1 Processor -- start 2024-04-25T09:00:29.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.572+0000 7fb12a61f640 1 -- start start 2024-04-25T09:00:29.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.572+0000 7fb12a61f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb12406c510 0x7fb1241f2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.572+0000 7fb12a61f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1240b0870 0x7fb12415e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.572+0000 7fb12a61f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb12415df70 0x7fb12415e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.572+0000 7fb12a61f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12406bd10 con 0x7fb1240b0870 2024-04-25T09:00:29.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.572+0000 7fb12a61f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1240d0c10 con 0x7fb12406c510 2024-04-25T09:00:29.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.572+0000 7fb12a61f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb12406be90 con 0x7fb12415df70 2024-04-25T09:00:29.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.573+0000 7fb1237fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1240b0870 0x7fb12415e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.573+0000 7fb1237fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1240b0870 0x7fb12415e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:40204/0 (socket says 172.21.15.43:40204) 2024-04-25T09:00:29.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.573+0000 7fb1237fe640 1 -- 172.21.15.43:0/1100146736 learned_addr learned my addr 172.21.15.43:0/1100146736 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:29.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.573+0000 7fb123fff640 1 --2- 172.21.15.43:0/1100146736 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb12406c510 0x7fb1241f2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.573+0000 7fb128bbc640 1 --2- 172.21.15.43:0/1100146736 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb12415df70 0x7fb12415e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.573+0000 7fb1237fe640 1 -- 172.21.15.43:0/1100146736 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb12415df70 msgr2=0x7fb12415e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:29.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.573+0000 7fb1237fe640 1 --2- 172.21.15.43:0/1100146736 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb12415df70 0x7fb12415e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.573+0000 7fb1237fe640 1 -- 172.21.15.43:0/1100146736 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb12406c510 msgr2=0x7fb1241f2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:29.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.573+0000 7fb1237fe640 1 --2- 172.21.15.43:0/1100146736 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb12406c510 0x7fb1241f2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.573+0000 7fb1237fe640 1 -- 172.21.15.43:0/1100146736 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb124149a40 con 0x7fb1240b0870 2024-04-25T09:00:29.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.573+0000 7fb1237fe640 1 --2- 172.21.15.43:0/1100146736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1240b0870 0x7fb12415e380 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fb114000c00 tx=0x7fb1140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:29.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.573+0000 7fb120ff9640 1 -- 172.21.15.43:0/1100146736 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb114041070 con 0x7fb1240b0870 2024-04-25T09:00:29.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.573+0000 7fb120ff9640 1 -- 172.21.15.43:0/1100146736 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb11403c670 con 0x7fb1240b0870 2024-04-25T09:00:29.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.573+0000 7fb12a61f640 1 -- 172.21.15.43:0/1100146736 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb124149d20 con 0x7fb1240b0870 2024-04-25T09:00:29.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.573+0000 7fb120ff9640 1 -- 172.21.15.43:0/1100146736 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb114037040 con 0x7fb1240b0870 2024-04-25T09:00:29.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.573+0000 7fb12a61f640 1 -- 172.21.15.43:0/1100146736 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb12414a5c0 con 0x7fb1240b0870 2024-04-25T09:00:29.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.574+0000 7fb12a61f640 1 -- 172.21.15.43:0/1100146736 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb124074ef0 con 0x7fb1240b0870 2024-04-25T09:00:29.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.575+0000 7fb120ff9640 1 -- 172.21.15.43:0/1100146736 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb114044070 con 0x7fb1240b0870 2024-04-25T09:00:29.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.575+0000 7fb120ff9640 1 --2- 172.21.15.43:0/1100146736 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb10406b4d0 0x7fb10406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.575+0000 7fb120ff9640 1 -- 172.21.15.43:0/1100146736 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb1140aa830 con 0x7fb1240b0870 2024-04-25T09:00:29.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.576+0000 7fb123fff640 1 --2- 172.21.15.43:0/1100146736 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb10406b4d0 0x7fb10406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.576+0000 7fb123fff640 1 --2- 172.21.15.43:0/1100146736 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb10406b4d0 0x7fb10406d990 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb10c007fd0 tx=0x7fb10c007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:29.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.580+0000 7fb120ff9640 1 -- 172.21.15.43:0/1100146736 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb114079090 con 0x7fb1240b0870 2024-04-25T09:00:29.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.798+0000 7fb12a61f640 1 -- 172.21.15.43:0/1100146736 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb12414a790 con 0x7fb1240b0870 2024-04-25T09:00:29.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.798+0000 7fb120ff9640 1 -- 172.21.15.43:0/1100146736 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fb11407d130 con 0x7fb1240b0870 2024-04-25T09:00:29.787 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:00:29.787 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":48,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6373,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3648859409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":3648859409},{"type":"v1","addr":"172.21.15.193:6837","nonce":3648859409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":7149,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/356516962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":356516962},{"type":"v1","addr":"172.21.15.43:6834","nonce":356516962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":7152,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3032637719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":3032637719},{"type":"v1","addr":"172.21.15.43:6835","nonce":3032637719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T08:58:45.906679+0000","modified":"2024-04-25T08:58:54.057379+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7164},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7164":{"gid":7164,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.193:6836/3458098162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":3458098162},{"type":"v1","addr":"172.21.15.193:6836","nonce":3458098162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T09:00:29.788 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 48 2024-04-25T09:00:29.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.800+0000 7fb12a61f640 1 -- 172.21.15.43:0/1100146736 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb10406b4d0 msgr2=0x7fb10406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:29.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.800+0000 7fb12a61f640 1 --2- 172.21.15.43:0/1100146736 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb10406b4d0 0x7fb10406d990 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb10c007fd0 tx=0x7fb10c007de0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.800+0000 7fb12a61f640 1 -- 172.21.15.43:0/1100146736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1240b0870 msgr2=0x7fb12415e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:29.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.800+0000 7fb12a61f640 1 --2- 172.21.15.43:0/1100146736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1240b0870 0x7fb12415e380 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fb114000c00 tx=0x7fb1140046c0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.800+0000 7fb12a61f640 1 -- 172.21.15.43:0/1100146736 shutdown_connections 2024-04-25T09:00:29.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.800+0000 7fb12a61f640 1 --2- 172.21.15.43:0/1100146736 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb12415df70 0x7fb12415e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.800+0000 7fb12a61f640 1 --2- 172.21.15.43:0/1100146736 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb1240b0870 0x7fb12415e380 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.800+0000 7fb12a61f640 1 --2- 172.21.15.43:0/1100146736 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb10406b4d0 0x7fb10406d990 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.800+0000 7fb12a61f640 1 --2- 172.21.15.43:0/1100146736 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb12406c510 0x7fb1241f2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.800+0000 7fb12a61f640 1 -- 172.21.15.43:0/1100146736 >> 172.21.15.43:0/1100146736 conn(0x7fb1240d17f0 msgr2=0x7fb1240d1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:29.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.800+0000 7fb12a61f640 1 -- 172.21.15.43:0/1100146736 shutdown_connections 2024-04-25T09:00:29.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.800+0000 7fb12a61f640 1 -- 172.21.15.43:0/1100146736 wait complete. 2024-04-25T09:00:29.801 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:00:29.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.913+0000 7f8bc18fc640 1 Processor -- start 2024-04-25T09:00:29.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.913+0000 7f8bc18fc640 1 -- start start 2024-04-25T09:00:29.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.914+0000 7f8bc18fc640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8bbc15df50 0x7f8bbc1619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.914+0000 7f8bc18fc640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f8bbc0cf670 0x7f8bbc162370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.914+0000 7f8bc18fc640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bbc0d24a0 con 0x7f8bbc161f30 2024-04-25T09:00:29.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.914+0000 7f8bc18fc640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bbc0cfa80 con 0x7f8bbc0cf670 2024-04-25T09:00:29.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.914+0000 7f8bc18fc640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bbc15e330 con 0x7f8bbc15df50 2024-04-25T09:00:29.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.914+0000 7f8bbaffd640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8bbc15df50 0x7f8bbc1619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.914+0000 7f8bbb7fe640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f8bbc0cf670 0x7f8bbc162370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.914+0000 7f8bb3fff640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f8bbc161f30 0x7f8bbc0c7cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:46038/0 (socket says 172.21.15.43:46038) 2024-04-25T09:00:29.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.914+0000 7f8bb3fff640 1 -- 172.21.15.43:0/683283128 learned_addr learned my addr 172.21.15.43:0/683283128 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:00:29.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.915+0000 7f8bba7fc640 1 -- 172.21.15.43:0/683283128 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2541964171 0 0) 0x7f8bbc0d24a0 con 0x7f8bbc161f30 2024-04-25T09:00:29.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.915+0000 7f8bba7fc640 1 -- 172.21.15.43:0/683283128 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ba8003520 con 0x7f8bbc161f30 2024-04-25T09:00:29.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.915+0000 7f8bbaffd640 1 -- 172.21.15.43:0/683283128 >> v2:172.21.15.193:3300/0 conn(0x7f8bbc0cf670 msgr2=0x7f8bbc162370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:29.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.915+0000 7f8bbaffd640 1 --2- 172.21.15.43:0/683283128 >> v2:172.21.15.193:3300/0 conn(0x7f8bbc0cf670 0x7f8bbc162370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.915+0000 7f8bbaffd640 1 -- 172.21.15.43:0/683283128 >> v1:172.21.15.193:6789/0 conn(0x7f8bbc161f30 legacy=0x7f8bbc0c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:29.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.915+0000 7f8bbaffd640 1 -- 172.21.15.43:0/683283128 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bbc0d2740 con 0x7f8bbc15df50 2024-04-25T09:00:29.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.915+0000 7f8bbb7fe640 1 --2- 172.21.15.43:0/683283128 >> v2:172.21.15.193:3300/0 conn(0x7f8bbc0cf670 0x7f8bbc162370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:00:29.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.916+0000 7f8bbaffd640 1 --2- 172.21.15.43:0/683283128 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8bbc15df50 0x7f8bbc1619f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8bac0097a0 tx=0x7f8bac02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a18eed31d6b397b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:29.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.916+0000 7f8bba7fc640 1 -- 172.21.15.43:0/683283128 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac037070 con 0x7f8bbc15df50 2024-04-25T09:00:29.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.916+0000 7f8bba7fc640 1 -- 172.21.15.43:0/683283128 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bac032440 con 0x7f8bbc15df50 2024-04-25T09:00:29.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.916+0000 7f8bba7fc640 1 -- 172.21.15.43:0/683283128 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac0327f0 con 0x7f8bbc15df50 2024-04-25T09:00:29.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.916+0000 7f8bc18fc640 1 -- 172.21.15.43:0/683283128 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8bbc15df50 msgr2=0x7f8bbc1619f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:29.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.916+0000 7f8bc18fc640 1 --2- 172.21.15.43:0/683283128 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8bbc15df50 0x7f8bbc1619f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8bac0097a0 tx=0x7f8bac02a180 comp rx=0 tx=0).stop 2024-04-25T09:00:29.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.916+0000 7f8bc18fc640 1 -- 172.21.15.43:0/683283128 shutdown_connections 2024-04-25T09:00:29.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.916+0000 7f8bc18fc640 1 --2- 172.21.15.43:0/683283128 >> v2:172.21.15.193:3300/0 conn(0x7f8bbc0cf670 0x7f8bbc162370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.916+0000 7f8bc18fc640 1 --2- 172.21.15.43:0/683283128 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8bbc15df50 0x7f8bbc1619f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.916+0000 7f8bc18fc640 1 -- 172.21.15.43:0/683283128 >> 172.21.15.43:0/683283128 conn(0x7f8bbc0d3a30 msgr2=0x7f8bbc0d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:29.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.916+0000 7f8bc18fc640 1 -- 172.21.15.43:0/683283128 shutdown_connections 2024-04-25T09:00:29.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.916+0000 7f8bc18fc640 1 -- 172.21.15.43:0/683283128 wait complete. 2024-04-25T09:00:29.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.918+0000 7f8bc18fc640 1 Processor -- start 2024-04-25T09:00:29.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.918+0000 7f8bc18fc640 1 -- start start 2024-04-25T09:00:29.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.918+0000 7f8bc18fc640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8bbc0cf670 0x7f8bbc09f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.918+0000 7f8bc18fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bbc15df50 0x7f8bbc160e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.918+0000 7f8bc18fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bbc161f30 0x7f8bbc0c6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.918+0000 7f8bc18fc640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bbc0c77d0 con 0x7f8bbc15df50 2024-04-25T09:00:29.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.918+0000 7f8bc18fc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bbc0c74f0 con 0x7f8bbc161f30 2024-04-25T09:00:29.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.918+0000 7f8bc18fc640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bbc0d2190 con 0x7f8bbc0cf670 2024-04-25T09:00:29.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.918+0000 7f8bb3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bbc15df50 0x7f8bbc160e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.918+0000 7f8bb3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bbc15df50 0x7f8bbc160e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52310/0 (socket says 172.21.15.43:52310) 2024-04-25T09:00:29.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.918+0000 7f8bb3fff640 1 -- 172.21.15.43:0/3851797282 learned_addr learned my addr 172.21.15.43:0/3851797282 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:29.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.918+0000 7f8bbb7fe640 1 --2- 172.21.15.43:0/3851797282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bbc161f30 0x7f8bbc0c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.918+0000 7f8bbaffd640 1 --2- 172.21.15.43:0/3851797282 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8bbc0cf670 0x7f8bbc09f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.919+0000 7f8bb3fff640 1 -- 172.21.15.43:0/3851797282 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8bbc0cf670 msgr2=0x7f8bbc09f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:29.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.919+0000 7f8bb3fff640 1 --2- 172.21.15.43:0/3851797282 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8bbc0cf670 0x7f8bbc09f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.919+0000 7f8bb3fff640 1 -- 172.21.15.43:0/3851797282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bbc161f30 msgr2=0x7f8bbc0c6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:29.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.919+0000 7f8bb3fff640 1 --2- 172.21.15.43:0/3851797282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bbc161f30 0x7f8bbc0c6150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:29.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.919+0000 7f8bb3fff640 1 -- 172.21.15.43:0/3851797282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bbc09e560 con 0x7f8bbc15df50 2024-04-25T09:00:29.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.919+0000 7f8bb3fff640 1 --2- 172.21.15.43:0/3851797282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bbc15df50 0x7f8bbc160e60 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f8ba400c390 tx=0x7f8ba400c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:29.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.919+0000 7f8bb8ff9640 1 -- 172.21.15.43:0/3851797282 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba4017070 con 0x7f8bbc15df50 2024-04-25T09:00:29.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.919+0000 7f8bb8ff9640 1 -- 172.21.15.43:0/3851797282 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba4005ce0 con 0x7f8bbc15df50 2024-04-25T09:00:29.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.919+0000 7f8bc18fc640 1 -- 172.21.15.43:0/3851797282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bbc09e840 con 0x7f8bbc15df50 2024-04-25T09:00:29.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.919+0000 7f8bb8ff9640 1 -- 172.21.15.43:0/3851797282 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba40072f0 con 0x7f8bbc15df50 2024-04-25T09:00:29.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.919+0000 7f8bc18fc640 1 -- 172.21.15.43:0/3851797282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bbc09ee00 con 0x7f8bbc15df50 2024-04-25T09:00:29.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.920+0000 7f8bc18fc640 1 -- 172.21.15.43:0/3851797282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bbc0680f0 con 0x7f8bbc15df50 2024-04-25T09:00:29.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.921+0000 7f8bb8ff9640 1 -- 172.21.15.43:0/3851797282 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8ba401a070 con 0x7f8bbc15df50 2024-04-25T09:00:29.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.921+0000 7f8bb8ff9640 1 --2- 172.21.15.43:0/3851797282 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8b9c06b4d0 0x7f8b9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:29.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.921+0000 7f8bb8ff9640 1 -- 172.21.15.43:0/3851797282 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f8ba4089480 con 0x7f8bbc15df50 2024-04-25T09:00:29.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.921+0000 7f8bbaffd640 1 --2- 172.21.15.43:0/3851797282 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8b9c06b4d0 0x7f8b9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:29.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.922+0000 7f8bbaffd640 1 --2- 172.21.15.43:0/3851797282 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8b9c06b4d0 0x7f8b9c06d990 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8bac034000 tx=0x7f8bac02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:29.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:29.926+0000 7f8bb8ff9640 1 -- 172.21.15.43:0/3851797282 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8ba4057ce0 con 0x7f8bbc15df50 2024-04-25T09:00:30.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.141+0000 7f8bc18fc640 1 -- 172.21.15.43:0/3851797282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8bbc0ca400 con 0x7f8bbc15df50 2024-04-25T09:00:30.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.142+0000 7f8bb8ff9640 1 -- 172.21.15.43:0/3851797282 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f8ba405bd80 con 0x7f8bbc15df50 2024-04-25T09:00:30.131 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:00:30.131 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":48,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6373,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3648859409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":3648859409},{"type":"v1","addr":"172.21.15.193:6837","nonce":3648859409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":7149,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/356516962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":356516962},{"type":"v1","addr":"172.21.15.43:6834","nonce":356516962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":7152,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3032637719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":3032637719},{"type":"v1","addr":"172.21.15.43:6835","nonce":3032637719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":48,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T08:58:45.906679+0000","modified":"2024-04-25T08:58:54.057379+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7164},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7164":{"gid":7164,"name":"d","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.193:6836/3458098162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":3458098162},{"type":"v1","addr":"172.21.15.193:6836","nonce":3458098162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T09:00:30.131 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 48 2024-04-25T09:00:30.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.144+0000 7f8bc18fc640 1 -- 172.21.15.43:0/3851797282 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8b9c06b4d0 msgr2=0x7f8b9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.144+0000 7f8bc18fc640 1 --2- 172.21.15.43:0/3851797282 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8b9c06b4d0 0x7f8b9c06d990 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8bac034000 tx=0x7f8bac02e000 comp rx=0 tx=0).stop 2024-04-25T09:00:30.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.144+0000 7f8bc18fc640 1 -- 172.21.15.43:0/3851797282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bbc15df50 msgr2=0x7f8bbc160e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.144+0000 7f8bc18fc640 1 --2- 172.21.15.43:0/3851797282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bbc15df50 0x7f8bbc160e60 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f8ba400c390 tx=0x7f8ba400c790 comp rx=0 tx=0).stop 2024-04-25T09:00:30.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.144+0000 7f8bc18fc640 1 -- 172.21.15.43:0/3851797282 shutdown_connections 2024-04-25T09:00:30.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.144+0000 7f8bc18fc640 1 --2- 172.21.15.43:0/3851797282 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8b9c06b4d0 0x7f8b9c06d990 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.144+0000 7f8bc18fc640 1 --2- 172.21.15.43:0/3851797282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bbc161f30 0x7f8bbc0c6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.144+0000 7f8bc18fc640 1 --2- 172.21.15.43:0/3851797282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bbc15df50 0x7f8bbc160e60 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.144+0000 7f8bc18fc640 1 --2- 172.21.15.43:0/3851797282 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8bbc0cf670 0x7f8bbc09f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.144+0000 7f8bc18fc640 1 -- 172.21.15.43:0/3851797282 >> 172.21.15.43:0/3851797282 conn(0x7f8bbc0d3a30 msgr2=0x7f8bbc0c9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:30.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.144+0000 7f8bc18fc640 1 -- 172.21.15.43:0/3851797282 shutdown_connections 2024-04-25T09:00:30.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.144+0000 7f8bc18fc640 1 -- 172.21.15.43:0/3851797282 wait complete. 2024-04-25T09:00:30.144 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:00:30.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.256+0000 7fc613e4a640 1 Processor -- start 2024-04-25T09:00:30.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.256+0000 7fc613e4a640 1 -- start start 2024-04-25T09:00:30.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.257+0000 7fc613e4a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc60c06f300 0x7fc60c06e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.257+0000 7fc613e4a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc60c0d0d80 con 0x7fc60c15dcf0 2024-04-25T09:00:30.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.257+0000 7fc611be6640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc60c15dcf0 0x7fc60c06ebf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58104/0 (socket says 172.21.15.43:58104) 2024-04-25T09:00:30.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.257+0000 7fc613e4a640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc60c0d04e0 con 0x7fc60c06c4e0 2024-04-25T09:00:30.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.257+0000 7fc611be6640 1 -- 172.21.15.43:0/1623160505 learned_addr learned my addr 172.21.15.43:0/1623160505 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:00:30.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.257+0000 7fc613e4a640 1 -- 172.21.15.43:0/1623160505 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc60c06c050 con 0x7fc60c06f300 2024-04-25T09:00:30.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.257+0000 7fc6113e5640 1 --2- 172.21.15.43:0/1623160505 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc60c06f300 0x7fc60c06e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:30.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.257+0000 7fc610be4640 1 -- 172.21.15.43:0/1623160505 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4096710646 0 0) 0x7fc60c0d0d80 con 0x7fc60c15dcf0 2024-04-25T09:00:30.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.257+0000 7fc610be4640 1 -- 172.21.15.43:0/1623160505 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5f8003520 con 0x7fc60c15dcf0 2024-04-25T09:00:30.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.257+0000 7fc610be4640 1 -- 172.21.15.43:0/1623160505 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3587045547 0 0) 0x7fc60c0d04e0 con 0x7fc60c06c4e0 2024-04-25T09:00:30.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.257+0000 7fc610be4640 1 -- 172.21.15.43:0/1623160505 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc60c0d0d80 con 0x7fc60c06c4e0 2024-04-25T09:00:30.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.257+0000 7fc610be4640 1 -- 172.21.15.43:0/1623160505 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3647650550 0 0) 0x7fc5f8003520 con 0x7fc60c15dcf0 2024-04-25T09:00:30.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.257+0000 7fc6113e5640 1 -- 172.21.15.43:0/1623160505 >> v1:172.21.15.193:6789/0 conn(0x7fc60c06c4e0 legacy=0x7fc60c06c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.257+0000 7fc6113e5640 1 -- 172.21.15.43:0/1623160505 >> v1:172.21.15.43:6789/0 conn(0x7fc60c15dcf0 legacy=0x7fc60c06ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.257+0000 7fc6113e5640 1 -- 172.21.15.43:0/1623160505 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc60c0d07c0 con 0x7fc60c06f300 2024-04-25T09:00:30.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.258+0000 7fc6113e5640 1 --2- 172.21.15.43:0/1623160505 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc60c06f300 0x7fc60c06e530 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc0097d0 tx=0x7fc5fc02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d1b5710e7764404 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:30.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.258+0000 7fc610be4640 1 -- 172.21.15.43:0/1623160505 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc037070 con 0x7fc60c06f300 2024-04-25T09:00:30.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.258+0000 7fc610be4640 1 -- 172.21.15.43:0/1623160505 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5fc02ab90 con 0x7fc60c06f300 2024-04-25T09:00:30.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.258+0000 7fc610be4640 1 -- 172.21.15.43:0/1623160505 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6080019f0 con 0x7fc60c06f300 2024-04-25T09:00:30.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.259+0000 7fc613e4a640 1 -- 172.21.15.43:0/1623160505 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc60c06f300 msgr2=0x7fc60c06e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.259+0000 7fc613e4a640 1 --2- 172.21.15.43:0/1623160505 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc60c06f300 0x7fc60c06e530 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc0097d0 tx=0x7fc5fc02a100 comp rx=0 tx=0).stop 2024-04-25T09:00:30.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.259+0000 7fc613e4a640 1 -- 172.21.15.43:0/1623160505 shutdown_connections 2024-04-25T09:00:30.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.259+0000 7fc613e4a640 1 --2- 172.21.15.43:0/1623160505 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc60c06f300 0x7fc60c06e530 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.259+0000 7fc613e4a640 1 -- 172.21.15.43:0/1623160505 >> 172.21.15.43:0/1623160505 conn(0x7fc60c0d1700 msgr2=0x7fc60c0d1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:30.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.259+0000 7fc613e4a640 1 -- 172.21.15.43:0/1623160505 shutdown_connections 2024-04-25T09:00:30.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.259+0000 7fc613e4a640 1 -- 172.21.15.43:0/1623160505 wait complete. 2024-04-25T09:00:30.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.260+0000 7fc613e4a640 1 Processor -- start 2024-04-25T09:00:30.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.260+0000 7fc613e4a640 1 -- start start 2024-04-25T09:00:30.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.261+0000 7fc613e4a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc60c06c4e0 0x7fc60c154d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.261+0000 7fc613e4a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc60c06f300 0x7fc60c0b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.261+0000 7fc613e4a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc60c15dcf0 0x7fc60c0b2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.261+0000 7fc613e4a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc60c0b0d50 con 0x7fc60c06f300 2024-04-25T09:00:30.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.261+0000 7fc613e4a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc60c06bce0 con 0x7fc60c15dcf0 2024-04-25T09:00:30.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.261+0000 7fc613e4a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc60c0b0ed0 con 0x7fc60c06c4e0 2024-04-25T09:00:30.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.261+0000 7fc6113e5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc60c06f300 0x7fc60c0b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:30.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.261+0000 7fc6113e5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc60c06f300 0x7fc60c0b22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52326/0 (socket says 172.21.15.43:52326) 2024-04-25T09:00:30.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.261+0000 7fc6113e5640 1 -- 172.21.15.43:0/3434196838 learned_addr learned my addr 172.21.15.43:0/3434196838 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:30.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.261+0000 7fc611be6640 1 --2- 172.21.15.43:0/3434196838 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc60c06c4e0 0x7fc60c154d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:30.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.261+0000 7fc6123e7640 1 --2- 172.21.15.43:0/3434196838 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc60c15dcf0 0x7fc60c0b2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:30.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.261+0000 7fc6113e5640 1 -- 172.21.15.43:0/3434196838 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc60c06c4e0 msgr2=0x7fc60c154d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.261+0000 7fc6113e5640 1 --2- 172.21.15.43:0/3434196838 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc60c06c4e0 0x7fc60c154d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.261+0000 7fc6113e5640 1 -- 172.21.15.43:0/3434196838 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc60c15dcf0 msgr2=0x7fc60c0b2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.261+0000 7fc6113e5640 1 --2- 172.21.15.43:0/3434196838 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc60c15dcf0 0x7fc60c0b2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.261+0000 7fc6113e5640 1 -- 172.21.15.43:0/3434196838 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc60c15aae0 con 0x7fc60c06f300 2024-04-25T09:00:30.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.262+0000 7fc6113e5640 1 --2- 172.21.15.43:0/3434196838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc60c06f300 0x7fc60c0b22e0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc035040 tx=0x7fc5fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:30.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.262+0000 7fc6027fc640 1 -- 172.21.15.43:0/3434196838 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc041020 con 0x7fc60c06f300 2024-04-25T09:00:30.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.262+0000 7fc6027fc640 1 -- 172.21.15.43:0/3434196838 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5fc03d640 con 0x7fc60c06f300 2024-04-25T09:00:30.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.262+0000 7fc613e4a640 1 -- 172.21.15.43:0/3434196838 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc60c15ad80 con 0x7fc60c06f300 2024-04-25T09:00:30.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.262+0000 7fc6027fc640 1 -- 172.21.15.43:0/3434196838 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5fc037040 con 0x7fc60c06f300 2024-04-25T09:00:30.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.262+0000 7fc613e4a640 1 -- 172.21.15.43:0/3434196838 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc60c15b340 con 0x7fc60c06f300 2024-04-25T09:00:30.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.263+0000 7fc613e4a640 1 -- 172.21.15.43:0/3434196838 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc60c074ef0 con 0x7fc60c06f300 2024-04-25T09:00:30.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.263+0000 7fc6027fc640 1 -- 172.21.15.43:0/3434196838 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc5fc044020 con 0x7fc60c06f300 2024-04-25T09:00:30.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.264+0000 7fc6027fc640 1 --2- 172.21.15.43:0/3434196838 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc5f006b4d0 0x7fc5f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.264+0000 7fc6027fc640 1 -- 172.21.15.43:0/3434196838 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc5fc0aa5b0 con 0x7fc60c06f300 2024-04-25T09:00:30.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.264+0000 7fc611be6640 1 --2- 172.21.15.43:0/3434196838 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc5f006b4d0 0x7fc5f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:30.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.264+0000 7fc611be6640 1 --2- 172.21.15.43:0/3434196838 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc5f006b4d0 0x7fc5f006d990 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc608001560 tx=0x7fc608009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:30.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.268+0000 7fc6027fc640 1 -- 172.21.15.43:0/3434196838 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc5fc04b030 con 0x7fc60c06f300 2024-04-25T09:00:30.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.450+0000 7fc613e4a640 1 -- 172.21.15.43:0/3434196838 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc60c0d2060 con 0x7fc60c06f300 2024-04-25T09:00:30.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.451+0000 7fc6027fc640 1 -- 172.21.15.43:0/3434196838 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+16795 (secure 0 0 0) 0x7fc5fc078e10 con 0x7fc60c06f300 2024-04-25T09:00:30.440 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:00:30.440 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":77,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:00:02.408776+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-04-25T08:58:44.188924+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-04-25T08:58:44.877113+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":76,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.1","mappings":[{"from":6,"to":0}]},{"pgid":"11.2e","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:00:30.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.453+0000 7fc613e4a640 1 -- 172.21.15.43:0/3434196838 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc5f006b4d0 msgr2=0x7fc5f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.453+0000 7fc613e4a640 1 --2- 172.21.15.43:0/3434196838 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc5f006b4d0 0x7fc5f006d990 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc608001560 tx=0x7fc608009040 comp rx=0 tx=0).stop 2024-04-25T09:00:30.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.453+0000 7fc613e4a640 1 -- 172.21.15.43:0/3434196838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc60c06f300 msgr2=0x7fc60c0b22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.453+0000 7fc613e4a640 1 --2- 172.21.15.43:0/3434196838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc60c06f300 0x7fc60c0b22e0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fc5fc035040 tx=0x7fc5fc0046c0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.453+0000 7fc613e4a640 1 -- 172.21.15.43:0/3434196838 shutdown_connections 2024-04-25T09:00:30.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.453+0000 7fc613e4a640 1 --2- 172.21.15.43:0/3434196838 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc5f006b4d0 0x7fc5f006d990 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.453+0000 7fc613e4a640 1 --2- 172.21.15.43:0/3434196838 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc60c15dcf0 0x7fc60c0b2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.453+0000 7fc613e4a640 1 --2- 172.21.15.43:0/3434196838 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc60c06f300 0x7fc60c0b22e0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.453+0000 7fc613e4a640 1 --2- 172.21.15.43:0/3434196838 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc60c06c4e0 0x7fc60c154d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.453+0000 7fc613e4a640 1 -- 172.21.15.43:0/3434196838 >> 172.21.15.43:0/3434196838 conn(0x7fc60c0d1700 msgr2=0x7fc60c0af250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:30.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.454+0000 7fc613e4a640 1 -- 172.21.15.43:0/3434196838 shutdown_connections 2024-04-25T09:00:30.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.454+0000 7fc613e4a640 1 -- 172.21.15.43:0/3434196838 wait complete. 2024-04-25T09:00:30.455 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:00:30.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.569+0000 7fc4bde2a640 1 Processor -- start 2024-04-25T09:00:30.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.569+0000 7fc4bde2a640 1 -- start start 2024-04-25T09:00:30.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.570+0000 7fc4bde2a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc4b80d35c0 0x7fc4b80ce260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.570+0000 7fc4bde2a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc4b80ce7a0 0x7fc4b80b6a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.570+0000 7fc4bde2a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4b80cec30 con 0x7fc4b815dd00 2024-04-25T09:00:30.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.570+0000 7fc4bde2a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4b80d24e0 con 0x7fc4b80d35c0 2024-04-25T09:00:30.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.570+0000 7fc4bde2a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b815e0e0 con 0x7fc4b80ce7a0 2024-04-25T09:00:30.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.570+0000 7fc4b77fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc4b815dd00 0x7fc4b80d2eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58118/0 (socket says 172.21.15.43:58118) 2024-04-25T09:00:30.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.570+0000 7fc4b77fe640 1 -- 172.21.15.43:0/993317139 learned_addr learned my addr 172.21.15.43:0/993317139 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:00:30.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.570+0000 7fc4b6ffd640 1 --2- 172.21.15.43:0/993317139 >> v2:172.21.15.43:3300/0 conn(0x7fc4b80d35c0 0x7fc4b80ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:30.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.571+0000 7fc4b6ffd640 1 -- 172.21.15.43:0/993317139 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc4b80ce7a0 msgr2=0x7fc4b80b6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:30.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.571+0000 7fc4b6ffd640 1 --2- 172.21.15.43:0/993317139 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc4b80ce7a0 0x7fc4b80b6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.571+0000 7fc4b6ffd640 1 -- 172.21.15.43:0/993317139 >> v1:172.21.15.43:6789/0 conn(0x7fc4b815dd00 legacy=0x7fc4b80d2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.571+0000 7fc4b6ffd640 1 -- 172.21.15.43:0/993317139 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b80ceeb0 con 0x7fc4b80d35c0 2024-04-25T09:00:30.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.571+0000 7fc4b67fc640 1 -- 172.21.15.43:0/993317139 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2307395595 0 0) 0x7fc4a8001110 con 0x7fc4b815dd00 2024-04-25T09:00:30.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.571+0000 7fc4b6ffd640 1 --2- 172.21.15.43:0/993317139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4b80d35c0 0x7fc4b80ce260 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac0097a0 tx=0x7fc4ac02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fea631faa7b703c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:30.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.571+0000 7fc4b67fc640 1 -- 172.21.15.43:0/993317139 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac037070 con 0x7fc4b80d35c0 2024-04-25T09:00:30.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.571+0000 7fc4b67fc640 1 -- 172.21.15.43:0/993317139 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4ac032440 con 0x7fc4b80d35c0 2024-04-25T09:00:30.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.571+0000 7fc4b67fc640 1 -- 172.21.15.43:0/993317139 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac032810 con 0x7fc4b80d35c0 2024-04-25T09:00:30.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.572+0000 7fc4bde2a640 1 -- 172.21.15.43:0/993317139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4b80d35c0 msgr2=0x7fc4b80ce260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.572+0000 7fc4bde2a640 1 --2- 172.21.15.43:0/993317139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4b80d35c0 0x7fc4b80ce260 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac0097a0 tx=0x7fc4ac02a180 comp rx=0 tx=0).stop 2024-04-25T09:00:30.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.572+0000 7fc4bde2a640 1 -- 172.21.15.43:0/993317139 shutdown_connections 2024-04-25T09:00:30.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.572+0000 7fc4bde2a640 1 --2- 172.21.15.43:0/993317139 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc4b80ce7a0 0x7fc4b80b6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.572+0000 7fc4bde2a640 1 --2- 172.21.15.43:0/993317139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4b80d35c0 0x7fc4b80ce260 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.572+0000 7fc4bde2a640 1 -- 172.21.15.43:0/993317139 >> 172.21.15.43:0/993317139 conn(0x7fc4b806cbd0 msgr2=0x7fc4b806cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:30.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.572+0000 7fc4bde2a640 1 -- 172.21.15.43:0/993317139 shutdown_connections 2024-04-25T09:00:30.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.572+0000 7fc4bde2a640 1 -- 172.21.15.43:0/993317139 wait complete. 2024-04-25T09:00:30.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.573+0000 7fc4bde2a640 1 Processor -- start 2024-04-25T09:00:30.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.573+0000 7fc4bde2a640 1 -- start start 2024-04-25T09:00:30.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.574+0000 7fc4bde2a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4b80ce7a0 0x7fc4b815a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.574+0000 7fc4bde2a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc4b80d35c0 0x7fc4b8160490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.574+0000 7fc4bde2a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4b815dd00 0x7fc4b81609d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.574+0000 7fc4bde2a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b80d2170 con 0x7fc4b80ce7a0 2024-04-25T09:00:30.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.574+0000 7fc4bde2a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b80d1e90 con 0x7fc4b815dd00 2024-04-25T09:00:30.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.574+0000 7fc4bde2a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b80d2970 con 0x7fc4b80d35c0 2024-04-25T09:00:30.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.574+0000 7fc4b77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4b80ce7a0 0x7fc4b815a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:30.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.574+0000 7fc4b77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4b80ce7a0 0x7fc4b815a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52344/0 (socket says 172.21.15.43:52344) 2024-04-25T09:00:30.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.574+0000 7fc4b77fe640 1 -- 172.21.15.43:0/2206928442 learned_addr learned my addr 172.21.15.43:0/2206928442 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:30.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.574+0000 7fc4b6ffd640 1 --2- 172.21.15.43:0/2206928442 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc4b80d35c0 0x7fc4b8160490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:30.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.574+0000 7fc4b7fff640 1 --2- 172.21.15.43:0/2206928442 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4b815dd00 0x7fc4b81609d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:30.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.575+0000 7fc4b77fe640 1 -- 172.21.15.43:0/2206928442 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc4b80d35c0 msgr2=0x7fc4b8160490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.575+0000 7fc4b77fe640 1 --2- 172.21.15.43:0/2206928442 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc4b80d35c0 0x7fc4b8160490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.575+0000 7fc4b77fe640 1 -- 172.21.15.43:0/2206928442 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4b815dd00 msgr2=0x7fc4b81609d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.575+0000 7fc4b77fe640 1 --2- 172.21.15.43:0/2206928442 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4b815dd00 0x7fc4b81609d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.575+0000 7fc4b77fe640 1 -- 172.21.15.43:0/2206928442 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b80b32e0 con 0x7fc4b80ce7a0 2024-04-25T09:00:30.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.575+0000 7fc4b77fe640 1 --2- 172.21.15.43:0/2206928442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4b80ce7a0 0x7fc4b815a110 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fc4a800c780 tx=0x7fc4a800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:30.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.575+0000 7fc49ffff640 1 -- 172.21.15.43:0/2206928442 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a8018070 con 0x7fc4b80ce7a0 2024-04-25T09:00:30.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.575+0000 7fc49ffff640 1 -- 172.21.15.43:0/2206928442 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4a80072d0 con 0x7fc4b80ce7a0 2024-04-25T09:00:30.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.575+0000 7fc4bde2a640 1 -- 172.21.15.43:0/2206928442 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4b80b35c0 con 0x7fc4b80ce7a0 2024-04-25T09:00:30.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.575+0000 7fc49ffff640 1 -- 172.21.15.43:0/2206928442 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a80047e0 con 0x7fc4b80ce7a0 2024-04-25T09:00:30.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.575+0000 7fc4bde2a640 1 -- 172.21.15.43:0/2206928442 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4b80b3b80 con 0x7fc4b80ce7a0 2024-04-25T09:00:30.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.576+0000 7fc4bde2a640 1 -- 172.21.15.43:0/2206928442 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4b8074ef0 con 0x7fc4b80ce7a0 2024-04-25T09:00:30.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.576+0000 7fc49ffff640 1 -- 172.21.15.43:0/2206928442 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc4a801b070 con 0x7fc4b80ce7a0 2024-04-25T09:00:30.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.577+0000 7fc49ffff640 1 --2- 172.21.15.43:0/2206928442 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc4a406b4d0 0x7fc4a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.577+0000 7fc49ffff640 1 -- 172.21.15.43:0/2206928442 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fc4a8089510 con 0x7fc4b80ce7a0 2024-04-25T09:00:30.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.577+0000 7fc4b6ffd640 1 --2- 172.21.15.43:0/2206928442 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc4a406b4d0 0x7fc4a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:30.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.578+0000 7fc4b6ffd640 1 --2- 172.21.15.43:0/2206928442 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc4a406b4d0 0x7fc4a406d990 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac034000 tx=0x7fc4ac02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:30.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.581+0000 7fc49ffff640 1 -- 172.21.15.43:0/2206928442 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc4b80cec30 con 0x7fc4b80ce7a0 2024-04-25T09:00:30.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.801+0000 7fc4bde2a640 1 -- 172.21.15.43:0/2206928442 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc4b8068080 con 0x7fc4b80ce7a0 2024-04-25T09:00:30.790 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-04-25T09:00:30.801+0000 7fe4fb198640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:00:30.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.803+0000 7fc49ffff640 1 -- 172.21.15.43:0/2206928442 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v49) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc4a8057ce0 con 0x7fc4b80ce7a0 2024-04-25T09:00:30.792 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:00:30.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.806+0000 7fc4bde2a640 1 -- 172.21.15.43:0/2206928442 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc4a406b4d0 msgr2=0x7fc4a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.806+0000 7fc4bde2a640 1 --2- 172.21.15.43:0/2206928442 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc4a406b4d0 0x7fc4a406d990 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac034000 tx=0x7fc4ac02e000 comp rx=0 tx=0).stop 2024-04-25T09:00:30.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.806+0000 7fc4bde2a640 1 -- 172.21.15.43:0/2206928442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4b80ce7a0 msgr2=0x7fc4b815a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.806+0000 7fc4bde2a640 1 --2- 172.21.15.43:0/2206928442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4b80ce7a0 0x7fc4b815a110 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fc4a800c780 tx=0x7fc4a800cc40 comp rx=0 tx=0).stop 2024-04-25T09:00:30.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.806+0000 7fc4bde2a640 1 -- 172.21.15.43:0/2206928442 shutdown_connections 2024-04-25T09:00:30.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.806+0000 7fc4bde2a640 1 --2- 172.21.15.43:0/2206928442 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc4a406b4d0 0x7fc4a406d990 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.806+0000 7fc4bde2a640 1 --2- 172.21.15.43:0/2206928442 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc4b815dd00 0x7fc4b81609d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.806+0000 7fc4bde2a640 1 --2- 172.21.15.43:0/2206928442 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc4b80d35c0 0x7fc4b8160490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.806+0000 7fc4bde2a640 1 --2- 172.21.15.43:0/2206928442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc4b80ce7a0 0x7fc4b815a110 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.806+0000 7fc4bde2a640 1 -- 172.21.15.43:0/2206928442 >> 172.21.15.43:0/2206928442 conn(0x7fc4b806cbd0 msgr2=0x7fc4b80b8250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:30.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.806+0000 7fc4bde2a640 1 -- 172.21.15.43:0/2206928442 shutdown_connections 2024-04-25T09:00:30.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.806+0000 7fc4bde2a640 1 -- 172.21.15.43:0/2206928442 wait complete. 2024-04-25T09:00:30.806 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:00:30.835 INFO:tasks.ceph.mds.d.smithi193.stdout:starting mds.d at 2024-04-25T09:00:30.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.920+0000 7fd19bbc1640 1 Processor -- start 2024-04-25T09:00:30.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.920+0000 7fd19bbc1640 1 -- start start 2024-04-25T09:00:30.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.921+0000 7fd19bbc1640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd19415dec0 0x7fd19415ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.921+0000 7fd19bbc1640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd1940ce8c0 0x7fd1940d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.921+0000 7fd19bbc1640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1940cf110 con 0x7fd1940d04a0 2024-04-25T09:00:30.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.921+0000 7fd19bbc1640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1940cf3f0 con 0x7fd19415dec0 2024-04-25T09:00:30.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.921+0000 7fd19bbc1640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1940cf6d0 con 0x7fd1940ce8c0 2024-04-25T09:00:30.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.921+0000 7fd19915c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd19415dec0 0x7fd19415ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:30.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.921+0000 7fd19995d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fd1940d04a0 0x7fd1940d0880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58124/0 (socket says 172.21.15.43:58124) 2024-04-25T09:00:30.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.921+0000 7fd19995d640 1 -- 172.21.15.43:0/3081075001 learned_addr learned my addr 172.21.15.43:0/3081075001 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:00:30.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.921+0000 7fd19a15e640 1 --2- 172.21.15.43:0/3081075001 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd1940ce8c0 0x7fd1940d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:30.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.921+0000 7fd19915c640 1 -- 172.21.15.43:0/3081075001 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd1940ce8c0 msgr2=0x7fd1940d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:30.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.921+0000 7fd19915c640 1 --2- 172.21.15.43:0/3081075001 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd1940ce8c0 0x7fd1940d17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.921+0000 7fd19915c640 1 -- 172.21.15.43:0/3081075001 >> v1:172.21.15.43:6789/0 conn(0x7fd1940d04a0 legacy=0x7fd1940d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.921+0000 7fd19895b640 1 -- 172.21.15.43:0/3081075001 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2085206417 0 0) 0x7fd1940cf110 con 0x7fd1940d04a0 2024-04-25T09:00:30.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.921+0000 7fd19915c640 1 -- 172.21.15.43:0/3081075001 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1940cf9b0 con 0x7fd19415dec0 2024-04-25T09:00:30.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.921+0000 7fd19a15e640 1 --2- 172.21.15.43:0/3081075001 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd1940ce8c0 0x7fd1940d17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:30.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.922+0000 7fd19915c640 1 --2- 172.21.15.43:0/3081075001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd19415dec0 0x7fd19415ed90 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fd1840097a0 tx=0x7fd18402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecae59dca2659426 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:30.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.922+0000 7fd19895b640 1 -- 172.21.15.43:0/3081075001 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd184037070 con 0x7fd19415dec0 2024-04-25T09:00:30.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.922+0000 7fd19895b640 1 -- 172.21.15.43:0/3081075001 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd184032440 con 0x7fd19415dec0 2024-04-25T09:00:30.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.922+0000 7fd19895b640 1 -- 172.21.15.43:0/3081075001 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd184032810 con 0x7fd19415dec0 2024-04-25T09:00:30.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.923+0000 7fd19bbc1640 1 -- 172.21.15.43:0/3081075001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd19415dec0 msgr2=0x7fd19415ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.923+0000 7fd19bbc1640 1 --2- 172.21.15.43:0/3081075001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd19415dec0 0x7fd19415ed90 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fd1840097a0 tx=0x7fd18402a180 comp rx=0 tx=0).stop 2024-04-25T09:00:30.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.923+0000 7fd19bbc1640 1 -- 172.21.15.43:0/3081075001 shutdown_connections 2024-04-25T09:00:30.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.923+0000 7fd19bbc1640 1 --2- 172.21.15.43:0/3081075001 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd1940ce8c0 0x7fd1940d17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.923+0000 7fd19bbc1640 1 --2- 172.21.15.43:0/3081075001 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd19415dec0 0x7fd19415ed90 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.923+0000 7fd19bbc1640 1 -- 172.21.15.43:0/3081075001 >> 172.21.15.43:0/3081075001 conn(0x7fd19409cd40 msgr2=0x7fd1940d3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:30.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.923+0000 7fd19bbc1640 1 -- 172.21.15.43:0/3081075001 shutdown_connections 2024-04-25T09:00:30.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.923+0000 7fd19bbc1640 1 -- 172.21.15.43:0/3081075001 wait complete. 2024-04-25T09:00:30.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.924+0000 7fd19bbc1640 1 Processor -- start 2024-04-25T09:00:30.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.924+0000 7fd19bbc1640 1 -- start start 2024-04-25T09:00:30.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.925+0000 7fd19bbc1640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd1940ce8c0 0x7fd19415a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.925+0000 7fd19bbc1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1940d04a0 0x7fd1940b84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.925+0000 7fd19bbc1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd19415dec0 0x7fd1941f72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.925+0000 7fd19bbc1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1940cee70 con 0x7fd19415dec0 2024-04-25T09:00:30.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.925+0000 7fd19bbc1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1940cecf0 con 0x7fd1940d04a0 2024-04-25T09:00:30.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.925+0000 7fd19bbc1640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1940d1ce0 con 0x7fd1940ce8c0 2024-04-25T09:00:30.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.925+0000 7fd19a15e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd19415dec0 0x7fd1941f72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:30.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.925+0000 7fd19a15e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd19415dec0 0x7fd1941f72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52358/0 (socket says 172.21.15.43:52358) 2024-04-25T09:00:30.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.925+0000 7fd19a15e640 1 -- 172.21.15.43:0/3373377610 learned_addr learned my addr 172.21.15.43:0/3373377610 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:30.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.925+0000 7fd19915c640 1 --2- 172.21.15.43:0/3373377610 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1940d04a0 0x7fd1940b84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:30.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.925+0000 7fd19995d640 1 --2- 172.21.15.43:0/3373377610 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd1940ce8c0 0x7fd19415a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:30.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.925+0000 7fd19a15e640 1 -- 172.21.15.43:0/3373377610 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd1940ce8c0 msgr2=0x7fd19415a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.925+0000 7fd19a15e640 1 --2- 172.21.15.43:0/3373377610 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd1940ce8c0 0x7fd19415a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.925+0000 7fd19a15e640 1 -- 172.21.15.43:0/3373377610 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1940d04a0 msgr2=0x7fd1940b84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:30.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.925+0000 7fd19a15e640 1 --2- 172.21.15.43:0/3373377610 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1940d04a0 0x7fd1940b84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:30.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.925+0000 7fd19a15e640 1 -- 172.21.15.43:0/3373377610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1940b8a00 con 0x7fd19415dec0 2024-04-25T09:00:30.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.926+0000 7fd19a15e640 1 --2- 172.21.15.43:0/3373377610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd19415dec0 0x7fd1941f72d0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fd19000c7f0 tx=0x7fd19000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:30.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.926+0000 7fd18a7fc640 1 -- 172.21.15.43:0/3373377610 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd190015070 con 0x7fd19415dec0 2024-04-25T09:00:30.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.926+0000 7fd18a7fc640 1 -- 172.21.15.43:0/3373377610 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd190007820 con 0x7fd19415dec0 2024-04-25T09:00:30.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.926+0000 7fd19bbc1640 1 -- 172.21.15.43:0/3373377610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1940b8cc0 con 0x7fd19415dec0 2024-04-25T09:00:30.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.926+0000 7fd18a7fc640 1 -- 172.21.15.43:0/3373377610 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd190007c50 con 0x7fd19415dec0 2024-04-25T09:00:30.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.926+0000 7fd19bbc1640 1 -- 172.21.15.43:0/3373377610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1940b9280 con 0x7fd19415dec0 2024-04-25T09:00:30.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.927+0000 7fd19bbc1640 1 -- 172.21.15.43:0/3373377610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd194074f60 con 0x7fd19415dec0 2024-04-25T09:00:30.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.927+0000 7fd18a7fc640 1 -- 172.21.15.43:0/3373377610 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd190018070 con 0x7fd19415dec0 2024-04-25T09:00:30.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.928+0000 7fd18a7fc640 1 --2- 172.21.15.43:0/3373377610 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd17806b4d0 0x7fd17806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:30.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.928+0000 7fd18a7fc640 1 -- 172.21.15.43:0/3373377610 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fd19008a330 con 0x7fd19415dec0 2024-04-25T09:00:30.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.928+0000 7fd19995d640 1 --2- 172.21.15.43:0/3373377610 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd17806b4d0 0x7fd17806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:30.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.928+0000 7fd19995d640 1 --2- 172.21.15.43:0/3373377610 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd17806b4d0 0x7fd17806d990 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd18c0012e0 tx=0x7fd18c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:30.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:30.932+0000 7fd18a7fc640 1 -- 172.21.15.43:0/3373377610 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd1900591b0 con 0x7fd19415dec0 2024-04-25T09:00:31.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.148+0000 7fd19bbc1640 1 -- 172.21.15.43:0/3373377610 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd19415d670 con 0x7fd19415dec0 2024-04-25T09:00:31.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.804+0000 7fd18a7fc640 1 -- 172.21.15.43:0/3373377610 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v50) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd190058ab0 con 0x7fd19415dec0 2024-04-25T09:00:31.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.807+0000 7fd19bbc1640 1 -- 172.21.15.43:0/3373377610 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd17806b4d0 msgr2=0x7fd17806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:31.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.807+0000 7fd19bbc1640 1 --2- 172.21.15.43:0/3373377610 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd17806b4d0 0x7fd17806d990 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd18c0012e0 tx=0x7fd18c009040 comp rx=0 tx=0).stop 2024-04-25T09:00:31.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.807+0000 7fd19bbc1640 1 -- 172.21.15.43:0/3373377610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd19415dec0 msgr2=0x7fd1941f72d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:31.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.807+0000 7fd19bbc1640 1 --2- 172.21.15.43:0/3373377610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd19415dec0 0x7fd1941f72d0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7fd19000c7f0 tx=0x7fd19000ccb0 comp rx=0 tx=0).stop 2024-04-25T09:00:31.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.807+0000 7fd19bbc1640 1 -- 172.21.15.43:0/3373377610 shutdown_connections 2024-04-25T09:00:31.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.807+0000 7fd19bbc1640 1 --2- 172.21.15.43:0/3373377610 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd19415dec0 0x7fd1941f72d0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:31.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.807+0000 7fd19bbc1640 1 --2- 172.21.15.43:0/3373377610 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd17806b4d0 0x7fd17806d990 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:31.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.807+0000 7fd19bbc1640 1 --2- 172.21.15.43:0/3373377610 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1940d04a0 0x7fd1940b84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:31.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.807+0000 7fd19bbc1640 1 --2- 172.21.15.43:0/3373377610 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd1940ce8c0 0x7fd19415a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:31.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.807+0000 7fd19bbc1640 1 -- 172.21.15.43:0/3373377610 >> 172.21.15.43:0/3373377610 conn(0x7fd19409cd40 msgr2=0x7fd1940b0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:31.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.807+0000 7fd19bbc1640 1 -- 172.21.15.43:0/3373377610 shutdown_connections 2024-04-25T09:00:31.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.807+0000 7fd19bbc1640 1 -- 172.21.15.43:0/3373377610 wait complete. 2024-04-25T09:00:31.809 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:00:31.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.928+0000 7fd052288640 1 Processor -- start 2024-04-25T09:00:31.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.928+0000 7fd052288640 1 -- start start 2024-04-25T09:00:31.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.928+0000 7fd052288640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd04c15b6c0 0x7fd04c15baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:31.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.929+0000 7fd052288640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd04c0b37a0 con 0x7fd04c0b4ea0 2024-04-25T09:00:31.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.929+0000 7fd052288640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd04c15eab0 con 0x7fd04c15e410 2024-04-25T09:00:31.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.929+0000 7fd052288640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd04c15fce0 con 0x7fd04c15b6c0 2024-04-25T09:00:31.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.929+0000 7fd04bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd04c15b6c0 0x7fd04c15baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:31.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.929+0000 7fd050825640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fd04c0b4ea0 0x7fd04c0d3760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58136/0 (socket says 172.21.15.43:58136) 2024-04-25T09:00:31.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.929+0000 7fd050825640 1 -- 172.21.15.43:0/3068204248 learned_addr learned my addr 172.21.15.43:0/3068204248 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:00:31.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.929+0000 7fd04bfff640 1 -- 172.21.15.43:0/3068204248 >> v1:172.21.15.193:6789/0 conn(0x7fd04c15e410 legacy=0x7fd04c15f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:31.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.929+0000 7fd04bfff640 1 -- 172.21.15.43:0/3068204248 >> v1:172.21.15.43:6789/0 conn(0x7fd04c0b4ea0 legacy=0x7fd04c0d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:31.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.929+0000 7fd04bfff640 1 -- 172.21.15.43:0/3068204248 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04c15ed90 con 0x7fd04c15b6c0 2024-04-25T09:00:31.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.929+0000 7fd04affd640 1 -- 172.21.15.43:0/3068204248 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2840243942 0 0) 0x7fd04c0b37a0 con 0x7fd04c0b4ea0 2024-04-25T09:00:31.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.930+0000 7fd04bfff640 1 --2- 172.21.15.43:0/3068204248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd04c15b6c0 0x7fd04c15baa0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fd03c0097a0 tx=0x7fd03c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c490afd6e0337 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:31.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.930+0000 7fd04affd640 1 -- 172.21.15.43:0/3068204248 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03c037070 con 0x7fd04c15b6c0 2024-04-25T09:00:31.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.930+0000 7fd04affd640 1 -- 172.21.15.43:0/3068204248 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd03c032440 con 0x7fd04c15b6c0 2024-04-25T09:00:31.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.930+0000 7fd04affd640 1 -- 172.21.15.43:0/3068204248 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03c032810 con 0x7fd04c15b6c0 2024-04-25T09:00:31.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.930+0000 7fd052288640 1 -- 172.21.15.43:0/3068204248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd04c15b6c0 msgr2=0x7fd04c15baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:31.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.930+0000 7fd052288640 1 --2- 172.21.15.43:0/3068204248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd04c15b6c0 0x7fd04c15baa0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fd03c0097a0 tx=0x7fd03c02a230 comp rx=0 tx=0).stop 2024-04-25T09:00:31.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.930+0000 7fd052288640 1 -- 172.21.15.43:0/3068204248 shutdown_connections 2024-04-25T09:00:31.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.930+0000 7fd052288640 1 --2- 172.21.15.43:0/3068204248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd04c15b6c0 0x7fd04c15baa0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:31.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.930+0000 7fd052288640 1 -- 172.21.15.43:0/3068204248 >> 172.21.15.43:0/3068204248 conn(0x7fd04c08d460 msgr2=0x7fd04c09cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:31.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.931+0000 7fd052288640 1 -- 172.21.15.43:0/3068204248 shutdown_connections 2024-04-25T09:00:31.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.931+0000 7fd052288640 1 -- 172.21.15.43:0/3068204248 wait complete. 2024-04-25T09:00:31.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.932+0000 7fd052288640 1 Processor -- start 2024-04-25T09:00:31.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.932+0000 7fd052288640 1 -- start start 2024-04-25T09:00:31.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.933+0000 7fd052288640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd04c0b4ea0 0x7fd04c158f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:31.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.933+0000 7fd052288640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd04c15b6c0 0x7fd04c0cd250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:31.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.933+0000 7fd052288640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd04c15e410 0x7fd04c0cd790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:31.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.933+0000 7fd052288640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c0b52f0 con 0x7fd04c0b4ea0 2024-04-25T09:00:31.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.933+0000 7fd052288640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c15f350 con 0x7fd04c15b6c0 2024-04-25T09:00:31.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.933+0000 7fd052288640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c0d3e70 con 0x7fd04c15e410 2024-04-25T09:00:31.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.933+0000 7fd04bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd04c0b4ea0 0x7fd04c158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:31.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.933+0000 7fd04bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd04c0b4ea0 0x7fd04c158f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52384/0 (socket says 172.21.15.43:52384) 2024-04-25T09:00:31.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.933+0000 7fd04bfff640 1 -- 172.21.15.43:0/77588163 learned_addr learned my addr 172.21.15.43:0/77588163 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:31.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.933+0000 7fd050825640 1 --2- 172.21.15.43:0/77588163 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd04c15e410 0x7fd04c0cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:31.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.933+0000 7fd04bfff640 1 -- 172.21.15.43:0/77588163 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd04c15e410 msgr2=0x7fd04c0cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:31.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.933+0000 7fd04bfff640 1 --2- 172.21.15.43:0/77588163 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd04c15e410 0x7fd04c0cd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:31.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.933+0000 7fd04bfff640 1 -- 172.21.15.43:0/77588163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd04c15b6c0 msgr2=0x7fd04c0cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:31.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.933+0000 7fd04bfff640 1 --2- 172.21.15.43:0/77588163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd04c15b6c0 0x7fd04c0cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:31.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.933+0000 7fd04bfff640 1 -- 172.21.15.43:0/77588163 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04c06c260 con 0x7fd04c0b4ea0 2024-04-25T09:00:31.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.933+0000 7fd04bfff640 1 --2- 172.21.15.43:0/77588163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd04c0b4ea0 0x7fd04c158f80 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fd03c0097a0 tx=0x7fd03c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:31.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.933+0000 7fd048ff9640 1 -- 172.21.15.43:0/77588163 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03c037050 con 0x7fd04c0b4ea0 2024-04-25T09:00:31.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.933+0000 7fd048ff9640 1 -- 172.21.15.43:0/77588163 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd03c038040 con 0x7fd04c0b4ea0 2024-04-25T09:00:31.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.933+0000 7fd048ff9640 1 -- 172.21.15.43:0/77588163 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03c004470 con 0x7fd04c0b4ea0 2024-04-25T09:00:31.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.933+0000 7fd052288640 1 -- 172.21.15.43:0/77588163 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd04c06c540 con 0x7fd04c0b4ea0 2024-04-25T09:00:31.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.933+0000 7fd052288640 1 -- 172.21.15.43:0/77588163 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd04c06cb00 con 0x7fd04c0b4ea0 2024-04-25T09:00:31.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.934+0000 7fd052288640 1 -- 172.21.15.43:0/77588163 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd04c068a40 con 0x7fd04c0b4ea0 2024-04-25T09:00:31.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.935+0000 7fd048ff9640 1 -- 172.21.15.43:0/77588163 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd03c030900 con 0x7fd04c0b4ea0 2024-04-25T09:00:31.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.935+0000 7fd048ff9640 1 --2- 172.21.15.43:0/77588163 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd03806b4d0 0x7fd03806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:31.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.935+0000 7fd048ff9640 1 -- 172.21.15.43:0/77588163 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fd03c0aaee0 con 0x7fd04c0b4ea0 2024-04-25T09:00:31.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.936+0000 7fd04b7fe640 1 --2- 172.21.15.43:0/77588163 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd03806b4d0 0x7fd03806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:31.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.937+0000 7fd04b7fe640 1 --2- 172.21.15.43:0/77588163 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd03806b4d0 0x7fd03806d990 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd040000f60 tx=0x7fd0400079b0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:31.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:31.939+0000 7fd048ff9640 1 -- 172.21.15.43:0/77588163 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd03c079df0 con 0x7fd04c0b4ea0 2024-04-25T09:00:32.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.123+0000 7fd052288640 1 -- 172.21.15.43:0/77588163 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd04c0b06c0 con 0x7fd04c0b4ea0 2024-04-25T09:00:32.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.806+0000 7fd048ff9640 1 -- 172.21.15.43:0/77588163 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v79) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd03c0796f0 con 0x7fd04c0b4ea0 2024-04-25T09:00:32.795 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:00:32.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.809+0000 7fd052288640 1 -- 172.21.15.43:0/77588163 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd03806b4d0 msgr2=0x7fd03806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:32.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.809+0000 7fd052288640 1 --2- 172.21.15.43:0/77588163 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd03806b4d0 0x7fd03806d990 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd040000f60 tx=0x7fd0400079b0 comp rx=0 tx=0).stop 2024-04-25T09:00:32.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.809+0000 7fd052288640 1 -- 172.21.15.43:0/77588163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd04c0b4ea0 msgr2=0x7fd04c158f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:32.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.809+0000 7fd052288640 1 --2- 172.21.15.43:0/77588163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd04c0b4ea0 0x7fd04c158f80 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fd03c0097a0 tx=0x7fd03c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:00:32.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.809+0000 7fd052288640 1 -- 172.21.15.43:0/77588163 shutdown_connections 2024-04-25T09:00:32.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.809+0000 7fd052288640 1 --2- 172.21.15.43:0/77588163 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd04c15e410 0x7fd04c0cd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:32.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.809+0000 7fd052288640 1 --2- 172.21.15.43:0/77588163 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd03806b4d0 0x7fd03806d990 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:32.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.810+0000 7fd052288640 1 --2- 172.21.15.43:0/77588163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd04c15b6c0 0x7fd04c0cd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:32.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.810+0000 7fd052288640 1 --2- 172.21.15.43:0/77588163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd04c0b4ea0 0x7fd04c158f80 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:32.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.810+0000 7fd052288640 1 -- 172.21.15.43:0/77588163 >> 172.21.15.43:0/77588163 conn(0x7fd04c08d460 msgr2=0x7fd04c0d0990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:32.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.810+0000 7fd052288640 1 -- 172.21.15.43:0/77588163 shutdown_connections 2024-04-25T09:00:32.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.810+0000 7fd052288640 1 -- 172.21.15.43:0/77588163 wait complete. 2024-04-25T09:00:32.817 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:00:32.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.940+0000 7f2296c6b640 1 Processor -- start 2024-04-25T09:00:32.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.940+0000 7f2296c6b640 1 -- start start 2024-04-25T09:00:32.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.941+0000 7f2296c6b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f22900af220 0x7f22900719f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:32.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.941+0000 7f2296c6b640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f229015e370 0x7f229015e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:32.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.941+0000 7f2296c6b640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2290070c30 con 0x7f2290158160 2024-04-25T09:00:32.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.941+0000 7f2296c6b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f229015f0e0 con 0x7f22900af220 2024-04-25T09:00:32.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.941+0000 7f2296c6b640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2290070f10 con 0x7f229015e370 2024-04-25T09:00:32.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.941+0000 7f228ffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f22900af220 0x7f22900719f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:32.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.941+0000 7f228ffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f22900af220 0x7f22900719f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52390/0 (socket says 172.21.15.43:52390) 2024-04-25T09:00:32.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.941+0000 7f228ffff640 1 -- 172.21.15.43:0/1796347980 learned_addr learned my addr 172.21.15.43:0/1796347980 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:32.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.941+0000 7f2295208640 1 --2- 172.21.15.43:0/1796347980 >> v2:172.21.15.193:3300/0 conn(0x7f229015e370 0x7f229015e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:32.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.942+0000 7f228ffff640 1 -- 172.21.15.43:0/1796347980 >> v2:172.21.15.193:3300/0 conn(0x7f229015e370 msgr2=0x7f229015e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:32.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.942+0000 7f228ffff640 1 --2- 172.21.15.43:0/1796347980 >> v2:172.21.15.193:3300/0 conn(0x7f229015e370 0x7f229015e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:32.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.942+0000 7f228ffff640 1 -- 172.21.15.43:0/1796347980 >> v1:172.21.15.193:6789/0 conn(0x7f2290158160 legacy=0x7f22900aeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:32.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.942+0000 7f228ffff640 1 -- 172.21.15.43:0/1796347980 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22900711f0 con 0x7f22900af220 2024-04-25T09:00:32.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.942+0000 7f228ffff640 1 --2- 172.21.15.43:0/1796347980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22900af220 0x7f22900719f0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f22900af960 tx=0x7f228402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53bbc8c64c06f95e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:32.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.942+0000 7f228f7fe640 1 -- 172.21.15.43:0/1796347980 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2284037070 con 0x7f22900af220 2024-04-25T09:00:32.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.943+0000 7f228f7fe640 1 -- 172.21.15.43:0/1796347980 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2284032440 con 0x7f22900af220 2024-04-25T09:00:32.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.943+0000 7f228f7fe640 1 -- 172.21.15.43:0/1796347980 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2284032810 con 0x7f22900af220 2024-04-25T09:00:32.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.943+0000 7f2296c6b640 1 -- 172.21.15.43:0/1796347980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22900af220 msgr2=0x7f22900719f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:32.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.943+0000 7f2296c6b640 1 --2- 172.21.15.43:0/1796347980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22900af220 0x7f22900719f0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f22900af960 tx=0x7f228402a1e0 comp rx=0 tx=0).stop 2024-04-25T09:00:32.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.943+0000 7f2296c6b640 1 -- 172.21.15.43:0/1796347980 shutdown_connections 2024-04-25T09:00:32.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.943+0000 7f2296c6b640 1 --2- 172.21.15.43:0/1796347980 >> v2:172.21.15.193:3300/0 conn(0x7f229015e370 0x7f229015e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:32.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.943+0000 7f2296c6b640 1 --2- 172.21.15.43:0/1796347980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22900af220 0x7f22900719f0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:32.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.943+0000 7f2296c6b640 1 -- 172.21.15.43:0/1796347980 >> 172.21.15.43:0/1796347980 conn(0x7f229006ab30 msgr2=0x7f229006af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:32.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.943+0000 7f2296c6b640 1 -- 172.21.15.43:0/1796347980 shutdown_connections 2024-04-25T09:00:32.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.943+0000 7f2296c6b640 1 -- 172.21.15.43:0/1796347980 wait complete. 2024-04-25T09:00:32.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.944+0000 7f2296c6b640 1 Processor -- start 2024-04-25T09:00:32.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.944+0000 7f2296c6b640 1 -- start start 2024-04-25T09:00:32.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.945+0000 7f2296c6b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22900af220 0x7f2290142c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:32.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.945+0000 7f2296c6b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2290158160 0x7f22901f69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:32.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.945+0000 7f2296c6b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f229015e370 0x7f229009aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:32.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.945+0000 7f2296c6b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f229015ecc0 con 0x7f22900af220 2024-04-25T09:00:32.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.945+0000 7f2296c6b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2290158540 con 0x7f2290158160 2024-04-25T09:00:32.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.945+0000 7f2296c6b640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f229015ee40 con 0x7f229015e370 2024-04-25T09:00:32.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.945+0000 7f2294a07640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22900af220 0x7f2290142c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:32.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.945+0000 7f2294a07640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22900af220 0x7f2290142c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52404/0 (socket says 172.21.15.43:52404) 2024-04-25T09:00:32.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.945+0000 7f2294a07640 1 -- 172.21.15.43:0/1387798179 learned_addr learned my addr 172.21.15.43:0/1387798179 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:32.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.946+0000 7f228ffff640 1 --2- 172.21.15.43:0/1387798179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2290158160 0x7f22901f69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:32.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.946+0000 7f2294a07640 1 -- 172.21.15.43:0/1387798179 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f229015e370 msgr2=0x7f229009aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:32.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.946+0000 7f2295208640 1 --2- 172.21.15.43:0/1387798179 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f229015e370 0x7f229009aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:32.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.946+0000 7f2294a07640 1 --2- 172.21.15.43:0/1387798179 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f229015e370 0x7f229009aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:32.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.946+0000 7f2294a07640 1 -- 172.21.15.43:0/1387798179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2290158160 msgr2=0x7f22901f69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:32.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.946+0000 7f2294a07640 1 --2- 172.21.15.43:0/1387798179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2290158160 0x7f22901f69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:32.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.946+0000 7f2294a07640 1 -- 172.21.15.43:0/1387798179 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22901f7440 con 0x7f22900af220 2024-04-25T09:00:32.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.946+0000 7f228ffff640 1 --2- 172.21.15.43:0/1387798179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2290158160 0x7f22901f69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:32.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.946+0000 7f2294a07640 1 --2- 172.21.15.43:0/1387798179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22900af220 0x7f2290142c20 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f228800c790 tx=0x7f228800cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:32.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.946+0000 7f228d7fa640 1 -- 172.21.15.43:0/1387798179 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2288016020 con 0x7f22900af220 2024-04-25T09:00:32.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.946+0000 7f228d7fa640 1 -- 172.21.15.43:0/1387798179 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2288006d50 con 0x7f22900af220 2024-04-25T09:00:32.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.946+0000 7f2296c6b640 1 -- 172.21.15.43:0/1387798179 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22901f7720 con 0x7f22900af220 2024-04-25T09:00:32.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.946+0000 7f228d7fa640 1 -- 172.21.15.43:0/1387798179 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2288003bb0 con 0x7f22900af220 2024-04-25T09:00:32.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.946+0000 7f2296c6b640 1 -- 172.21.15.43:0/1387798179 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2290155c70 con 0x7f22900af220 2024-04-25T09:00:32.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.947+0000 7f2296c6b640 1 -- 172.21.15.43:0/1387798179 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2290068a40 con 0x7f22900af220 2024-04-25T09:00:32.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.948+0000 7f228d7fa640 1 -- 172.21.15.43:0/1387798179 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f228800a850 con 0x7f22900af220 2024-04-25T09:00:32.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.948+0000 7f228d7fa640 1 --2- 172.21.15.43:0/1387798179 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f227c06b4d0 0x7f227c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:32.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.948+0000 7f228d7fa640 1 -- 172.21.15.43:0/1387798179 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5415+0+0 (secure 0 0 0) 0x7f2288089db0 con 0x7f22900af220 2024-04-25T09:00:32.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.948+0000 7f228ffff640 1 --2- 172.21.15.43:0/1387798179 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f227c06b4d0 0x7f227c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:32.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.949+0000 7f228ffff640 1 --2- 172.21.15.43:0/1387798179 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f227c06b4d0 0x7f227c06d990 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2284034000 tx=0x7f228402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:32.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:32.952+0000 7f228d7fa640 1 -- 172.21.15.43:0/1387798179 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2288020020 con 0x7f22900af220 2024-04-25T09:00:33.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.132+0000 7f2296c6b640 1 -- 172.21.15.43:0/1387798179 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f22901f78f0 con 0x7f22900af220 2024-04-25T09:00:33.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.808+0000 7f228d7fa640 1 -- 172.21.15.43:0/1387798179 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v80) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2288019310 con 0x7f22900af220 2024-04-25T09:00:33.797 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' removed 2024-04-25T09:00:33.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.811+0000 7f2296c6b640 1 -- 172.21.15.43:0/1387798179 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f227c06b4d0 msgr2=0x7f227c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:33.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.811+0000 7f2296c6b640 1 --2- 172.21.15.43:0/1387798179 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f227c06b4d0 0x7f227c06d990 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2284034000 tx=0x7f228402e000 comp rx=0 tx=0).stop 2024-04-25T09:00:33.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.811+0000 7f2296c6b640 1 -- 172.21.15.43:0/1387798179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22900af220 msgr2=0x7f2290142c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:33.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.811+0000 7f2296c6b640 1 --2- 172.21.15.43:0/1387798179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22900af220 0x7f2290142c20 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f228800c790 tx=0x7f228800cc50 comp rx=0 tx=0).stop 2024-04-25T09:00:33.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.812+0000 7f2296c6b640 1 -- 172.21.15.43:0/1387798179 shutdown_connections 2024-04-25T09:00:33.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.812+0000 7f2296c6b640 1 --2- 172.21.15.43:0/1387798179 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f229015e370 0x7f229009aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:33.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.812+0000 7f2296c6b640 1 --2- 172.21.15.43:0/1387798179 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f227c06b4d0 0x7f227c06d990 secure :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f2284034000 tx=0x7f228402e000 comp rx=0 tx=0).stop 2024-04-25T09:00:33.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.812+0000 7f2296c6b640 1 --2- 172.21.15.43:0/1387798179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2290158160 0x7f22901f69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:33.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.812+0000 7f2296c6b640 1 --2- 172.21.15.43:0/1387798179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22900af220 0x7f2290142c20 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:33.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.812+0000 7f2296c6b640 1 -- 172.21.15.43:0/1387798179 >> 172.21.15.43:0/1387798179 conn(0x7f229006ab30 msgr2=0x7f22900aead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:33.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.812+0000 7f2296c6b640 1 -- 172.21.15.43:0/1387798179 shutdown_connections 2024-04-25T09:00:33.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.812+0000 7f2296c6b640 1 -- 172.21.15.43:0/1387798179 wait complete. 2024-04-25T09:00:33.817 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T09:00:33.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.937+0000 7f358c929640 1 Processor -- start 2024-04-25T09:00:33.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.937+0000 7f358c929640 1 -- start start 2024-04-25T09:00:33.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.938+0000 7f358c929640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f35880b40a0 0x7f35880ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:33.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.938+0000 7f358c929640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35880aec70 0x7f3588070330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:33.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.938+0000 7f358c929640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35880af070 con 0x7f358815a960 2024-04-25T09:00:33.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.938+0000 7f358c929640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f358815b110 con 0x7f35880b40a0 2024-04-25T09:00:33.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.938+0000 7f358c929640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f358815b3f0 con 0x7f35880aec70 2024-04-25T09:00:33.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.938+0000 7f358659c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f358815a960 0x7f35880b3990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58152/0 (socket says 172.21.15.43:58152) 2024-04-25T09:00:33.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.938+0000 7f358659c640 1 -- 172.21.15.43:0/1355969858 learned_addr learned my addr 172.21.15.43:0/1355969858 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:00:33.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.938+0000 7f3586d9d640 1 --2- 172.21.15.43:0/1355969858 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35880aec70 0x7f3588070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:33.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.939+0000 7f3585d9b640 1 --2- 172.21.15.43:0/1355969858 >> v2:172.21.15.193:3300/0 conn(0x7f35880b40a0 0x7f35880ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:33.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.939+0000 7f358559a640 1 -- 172.21.15.43:0/1355969858 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1874797161 0 0) 0x7f35880af070 con 0x7f358815a960 2024-04-25T09:00:33.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.939+0000 7f358559a640 1 -- 172.21.15.43:0/1355969858 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3570003520 con 0x7f358815a960 2024-04-25T09:00:33.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.939+0000 7f358559a640 1 -- 172.21.15.43:0/1355969858 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 69095360 0 0) 0x7f3570003520 con 0x7f358815a960 2024-04-25T09:00:33.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.939+0000 7f358559a640 1 -- 172.21.15.43:0/1355969858 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35880aec70 msgr2=0x7f3588070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:33.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.939+0000 7f358559a640 1 --2- 172.21.15.43:0/1355969858 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35880aec70 0x7f3588070330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:33.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.939+0000 7f358559a640 1 -- 172.21.15.43:0/1355969858 >> v2:172.21.15.193:3300/0 conn(0x7f35880b40a0 msgr2=0x7f35880ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:33.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.939+0000 7f358559a640 1 --2- 172.21.15.43:0/1355969858 >> v2:172.21.15.193:3300/0 conn(0x7f35880b40a0 0x7f35880ae730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:33.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.939+0000 7f358559a640 1 -- 172.21.15.43:0/1355969858 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f358815b6d0 con 0x7f358815a960 2024-04-25T09:00:33.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.939+0000 7f358559a640 1 -- 172.21.15.43:0/1355969858 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f3580001670 con 0x7f358815a960 2024-04-25T09:00:33.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.939+0000 7f358559a640 1 -- 172.21.15.43:0/1355969858 >> v1:172.21.15.43:6789/0 conn(0x7f358815a960 legacy=0x7f35880b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:33.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.939+0000 7f358559a640 1 --2- 172.21.15.43:0/1355969858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35700039b0 0x7f3570003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:33.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.939+0000 7f358559a640 1 --2- 172.21.15.43:0/1355969858 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3570004800 0x7f3570004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:33.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.939+0000 7f358559a640 1 --2- 172.21.15.43:0/1355969858 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35880aec70 0x7f35700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:33.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.939+0000 7f358559a640 1 -- 172.21.15.43:0/1355969858 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358815b110 con 0x7f35700039b0 2024-04-25T09:00:33.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.939+0000 7f358559a640 1 -- 172.21.15.43:0/1355969858 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358815b3f0 con 0x7f3570004800 2024-04-25T09:00:33.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.939+0000 7f358559a640 1 -- 172.21.15.43:0/1355969858 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f357000b9c0 con 0x7f35880aec70 2024-04-25T09:00:33.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.939+0000 7f358659c640 1 --2- 172.21.15.43:0/1355969858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35700039b0 0x7f3570003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:33.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.939+0000 7f3585d9b640 1 --2- 172.21.15.43:0/1355969858 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3570004800 0x7f3570004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:33.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.939+0000 7f3586d9d640 1 --2- 172.21.15.43:0/1355969858 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35880aec70 0x7f35700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:33.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.939+0000 7f358659c640 1 -- 172.21.15.43:0/1355969858 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35880aec70 msgr2=0x7f35700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:33.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.939+0000 7f358659c640 1 --2- 172.21.15.43:0/1355969858 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35880aec70 0x7f35700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:33.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.939+0000 7f358659c640 1 -- 172.21.15.43:0/1355969858 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3570004800 msgr2=0x7f3570004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:33.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.939+0000 7f358659c640 1 --2- 172.21.15.43:0/1355969858 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3570004800 0x7f3570004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:33.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.939+0000 7f358659c640 1 -- 172.21.15.43:0/1355969858 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3570003520 con 0x7f35700039b0 2024-04-25T09:00:33.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.940+0000 7f358659c640 1 --2- 172.21.15.43:0/1355969858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35700039b0 0x7f3570003dd0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f3580007fd0 tx=0x7f3570005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6038c5c7dd9fed1c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:33.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.940+0000 7f358559a640 1 -- 172.21.15.43:0/1355969858 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3580007d50 con 0x7f35700039b0 2024-04-25T09:00:33.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.940+0000 7f358559a640 1 -- 172.21.15.43:0/1355969858 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3580005ad0 con 0x7f35700039b0 2024-04-25T09:00:33.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.940+0000 7f358559a640 1 -- 172.21.15.43:0/1355969858 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3580007070 con 0x7f35700039b0 2024-04-25T09:00:33.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.941+0000 7f358c929640 1 -- 172.21.15.43:0/1355969858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35700039b0 msgr2=0x7f3570003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:33.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.941+0000 7f358c929640 1 --2- 172.21.15.43:0/1355969858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35700039b0 0x7f3570003dd0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f3580007fd0 tx=0x7f3570005830 comp rx=0 tx=0).stop 2024-04-25T09:00:33.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.941+0000 7f3586d9d640 1 -- 172.21.15.43:0/1355969858 reap_dead start 2024-04-25T09:00:33.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.941+0000 7f358c929640 1 -- 172.21.15.43:0/1355969858 shutdown_connections 2024-04-25T09:00:33.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.941+0000 7f358c929640 1 -- 172.21.15.43:0/1355969858 >> 172.21.15.43:0/1355969858 conn(0x7f358806ab30 msgr2=0x7f358806af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:33.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.941+0000 7f358c929640 1 -- 172.21.15.43:0/1355969858 shutdown_connections 2024-04-25T09:00:33.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.941+0000 7f358c929640 1 -- 172.21.15.43:0/1355969858 wait complete. 2024-04-25T09:00:33.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.942+0000 7f358c929640 1 Processor -- start 2024-04-25T09:00:33.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.942+0000 7f358c929640 1 -- start start 2024-04-25T09:00:33.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.943+0000 7f358c929640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35700039b0 0x7f3588154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:33.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.943+0000 7f358c929640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35880c5860 0x7f35880ad910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:33.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.943+0000 7f358c929640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35880c5ce0 0x7f35880ade50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:33.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.943+0000 7f358c929640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35880cd940 con 0x7f35880c5860 2024-04-25T09:00:33.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.943+0000 7f358c929640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35880cc440 con 0x7f35700039b0 2024-04-25T09:00:33.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.943+0000 7f358c929640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f358815ad40 con 0x7f35880c5ce0 2024-04-25T09:00:33.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.943+0000 7f3585d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35880c5860 0x7f35880ad910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:33.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.943+0000 7f3585d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35880c5860 0x7f35880ad910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52428/0 (socket says 172.21.15.43:52428) 2024-04-25T09:00:33.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.943+0000 7f3585d9b640 1 -- 172.21.15.43:0/1948221911 learned_addr learned my addr 172.21.15.43:0/1948221911 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:33.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.943+0000 7f358659c640 1 --2- 172.21.15.43:0/1948221911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35700039b0 0x7f3588154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:33.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.943+0000 7f3586d9d640 1 --2- 172.21.15.43:0/1948221911 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35880c5ce0 0x7f35880ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:33.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.944+0000 7f3585d9b640 1 -- 172.21.15.43:0/1948221911 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35880c5ce0 msgr2=0x7f35880ade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:33.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.944+0000 7f3585d9b640 1 --2- 172.21.15.43:0/1948221911 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35880c5ce0 0x7f35880ade50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:33.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.944+0000 7f3585d9b640 1 -- 172.21.15.43:0/1948221911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35700039b0 msgr2=0x7f3588154a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:33.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.944+0000 7f3585d9b640 1 --2- 172.21.15.43:0/1948221911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35700039b0 0x7f3588154a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:33.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.944+0000 7f3585d9b640 1 -- 172.21.15.43:0/1948221911 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3588154fb0 con 0x7f35880c5860 2024-04-25T09:00:33.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.944+0000 7f3585d9b640 1 --2- 172.21.15.43:0/1948221911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35880c5860 0x7f35880ad910 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f357c011960 tx=0x7f357c011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:33.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.944+0000 7f3576ffd640 1 -- 172.21.15.43:0/1948221911 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357c00fa90 con 0x7f35880c5860 2024-04-25T09:00:33.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.944+0000 7f3576ffd640 1 -- 172.21.15.43:0/1948221911 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f357c00fc30 con 0x7f35880c5860 2024-04-25T09:00:33.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.944+0000 7f358c929640 1 -- 172.21.15.43:0/1948221911 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35881563e0 con 0x7f35880c5860 2024-04-25T09:00:33.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.944+0000 7f3576ffd640 1 -- 172.21.15.43:0/1948221911 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357c014440 con 0x7f35880c5860 2024-04-25T09:00:33.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.944+0000 7f358c929640 1 -- 172.21.15.43:0/1948221911 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35881569a0 con 0x7f35880c5860 2024-04-25T09:00:33.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.945+0000 7f358c929640 1 -- 172.21.15.43:0/1948221911 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3588069000 con 0x7f35880c5860 2024-04-25T09:00:33.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.945+0000 7f3576ffd640 1 -- 172.21.15.43:0/1948221911 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f357c012040 con 0x7f35880c5860 2024-04-25T09:00:33.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.946+0000 7f3576ffd640 1 --2- 172.21.15.43:0/1948221911 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f356806b470 0x7f356806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:33.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.946+0000 7f3576ffd640 1 -- 172.21.15.43:0/1948221911 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f357c08a550 con 0x7f35880c5860 2024-04-25T09:00:33.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.946+0000 7f358659c640 1 --2- 172.21.15.43:0/1948221911 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f356806b470 0x7f356806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:33.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.947+0000 7f358659c640 1 --2- 172.21.15.43:0/1948221911 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f356806b470 0x7f356806d930 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3580033000 tx=0x7f3580009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:33.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:33.950+0000 7f3576ffd640 1 -- 172.21.15.43:0/1948221911 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3588069000 con 0x7f35880c5860 2024-04-25T09:00:34.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.146+0000 7f358c929640 1 -- 172.21.15.43:0/1948221911 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f35880b0a30 con 0x7f35880c5860 2024-04-25T09:00:34.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.149+0000 7f3576ffd640 1 -- 172.21.15.43:0/1948221911 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f357c019030 con 0x7f35880c5860 2024-04-25T09:00:34.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.151+0000 7f358c929640 1 -- 172.21.15.43:0/1948221911 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f356806b470 msgr2=0x7f356806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:34.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.151+0000 7f358c929640 1 --2- 172.21.15.43:0/1948221911 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f356806b470 0x7f356806d930 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f3580033000 tx=0x7f3580009030 comp rx=0 tx=0).stop 2024-04-25T09:00:34.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.151+0000 7f358c929640 1 -- 172.21.15.43:0/1948221911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35880c5860 msgr2=0x7f35880ad910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:34.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.151+0000 7f358c929640 1 --2- 172.21.15.43:0/1948221911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35880c5860 0x7f35880ad910 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f357c011960 tx=0x7f357c011e20 comp rx=0 tx=0).stop 2024-04-25T09:00:34.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.151+0000 7f358c929640 1 -- 172.21.15.43:0/1948221911 shutdown_connections 2024-04-25T09:00:34.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.151+0000 7f358c929640 1 --2- 172.21.15.43:0/1948221911 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35880c5ce0 0x7f35880ade50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.151+0000 7f358c929640 1 --2- 172.21.15.43:0/1948221911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35880c5860 0x7f35880ad910 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.151+0000 7f358c929640 1 --2- 172.21.15.43:0/1948221911 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f356806b470 0x7f356806d930 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.151+0000 7f358c929640 1 --2- 172.21.15.43:0/1948221911 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35700039b0 0x7f3588154a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.151+0000 7f358c929640 1 -- 172.21.15.43:0/1948221911 >> 172.21.15.43:0/1948221911 conn(0x7f358806ab30 msgr2=0x7f35880af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:34.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.152+0000 7f358c929640 1 -- 172.21.15.43:0/1948221911 shutdown_connections 2024-04-25T09:00:34.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.152+0000 7f358c929640 1 -- 172.21.15.43:0/1948221911 wait complete. 2024-04-25T09:00:34.152 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub' 2024-04-25T09:00:34.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.264+0000 7fdd4469e640 1 Processor -- start 2024-04-25T09:00:34.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.264+0000 7fdd4469e640 1 -- start start 2024-04-25T09:00:34.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.265+0000 7fdd4469e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdd3c15df50 0x7fdd3c1619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:34.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.265+0000 7fdd4469e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdd3c161f30 0x7fdd3c162370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:34.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.265+0000 7fdd4469e640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd3c0d24a0 con 0x7fdd3c0c7cf0 2024-04-25T09:00:34.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.265+0000 7fdd4469e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd3c15e330 con 0x7fdd3c161f30 2024-04-25T09:00:34.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.265+0000 7fdd4469e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3c0d2740 con 0x7fdd3c15df50 2024-04-25T09:00:34.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.265+0000 7fdd41c39640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdd3c161f30 0x7fdd3c162370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:34.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.265+0000 7fdd41c39640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdd3c161f30 0x7fdd3c162370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52430/0 (socket says 172.21.15.43:52430) 2024-04-25T09:00:34.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.265+0000 7fdd41c39640 1 -- 172.21.15.43:0/418471283 learned_addr learned my addr 172.21.15.43:0/418471283 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:34.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.265+0000 7fdd4243a640 1 --2- 172.21.15.43:0/418471283 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdd3c15df50 0x7fdd3c1619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:34.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.265+0000 7fdd41c39640 1 -- 172.21.15.43:0/418471283 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdd3c15df50 msgr2=0x7fdd3c1619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:34.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.266+0000 7fdd41c39640 1 --2- 172.21.15.43:0/418471283 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdd3c15df50 0x7fdd3c1619f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.266+0000 7fdd41c39640 1 -- 172.21.15.43:0/418471283 >> v1:172.21.15.193:6789/0 conn(0x7fdd3c0c7cf0 legacy=0x7fdd3c0c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:34.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.266+0000 7fdd41c39640 1 -- 172.21.15.43:0/418471283 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd3c0d2a20 con 0x7fdd3c161f30 2024-04-25T09:00:34.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.266+0000 7fdd41c39640 1 --2- 172.21.15.43:0/418471283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd3c161f30 0x7fdd3c162370 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c009870 tx=0x7fdd2c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4adeb9f51d8b7337 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:34.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.266+0000 7fdd41438640 1 -- 172.21.15.43:0/418471283 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c037070 con 0x7fdd3c161f30 2024-04-25T09:00:34.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.266+0000 7fdd41438640 1 -- 172.21.15.43:0/418471283 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd2c032440 con 0x7fdd3c161f30 2024-04-25T09:00:34.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.266+0000 7fdd41438640 1 -- 172.21.15.43:0/418471283 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd2c032810 con 0x7fdd3c161f30 2024-04-25T09:00:34.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.267+0000 7fdd4469e640 1 -- 172.21.15.43:0/418471283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd3c161f30 msgr2=0x7fdd3c162370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:34.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.267+0000 7fdd4469e640 1 --2- 172.21.15.43:0/418471283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd3c161f30 0x7fdd3c162370 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c009870 tx=0x7fdd2c02a270 comp rx=0 tx=0).stop 2024-04-25T09:00:34.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.267+0000 7fdd4469e640 1 -- 172.21.15.43:0/418471283 shutdown_connections 2024-04-25T09:00:34.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.267+0000 7fdd4469e640 1 --2- 172.21.15.43:0/418471283 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd3c161f30 0x7fdd3c162370 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.267+0000 7fdd4469e640 1 --2- 172.21.15.43:0/418471283 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdd3c15df50 0x7fdd3c1619f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.267+0000 7fdd4469e640 1 -- 172.21.15.43:0/418471283 >> 172.21.15.43:0/418471283 conn(0x7fdd3c0d3a30 msgr2=0x7fdd3c0d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:34.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.267+0000 7fdd4469e640 1 -- 172.21.15.43:0/418471283 shutdown_connections 2024-04-25T09:00:34.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.267+0000 7fdd4469e640 1 -- 172.21.15.43:0/418471283 wait complete. 2024-04-25T09:00:34.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.268+0000 7fdd4469e640 1 Processor -- start 2024-04-25T09:00:34.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.268+0000 7fdd4469e640 1 -- start start 2024-04-25T09:00:34.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.269+0000 7fdd4469e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd3c0c7cf0 0x7fdd3c1e7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:34.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.269+0000 7fdd4469e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd3c15df50 0x7fdd3c06b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:34.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.269+0000 7fdd4469e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdd3c161f30 0x7fdd3c0ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:34.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.269+0000 7fdd4469e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3c0d2190 con 0x7fdd3c0c7cf0 2024-04-25T09:00:34.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.269+0000 7fdd4469e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3c0c77d0 con 0x7fdd3c15df50 2024-04-25T09:00:34.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.269+0000 7fdd4469e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd3c0cf670 con 0x7fdd3c161f30 2024-04-25T09:00:34.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.269+0000 7fdd4243a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd3c0c7cf0 0x7fdd3c1e7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:34.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.269+0000 7fdd4243a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd3c0c7cf0 0x7fdd3c1e7a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52434/0 (socket says 172.21.15.43:52434) 2024-04-25T09:00:34.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.269+0000 7fdd4243a640 1 -- 172.21.15.43:0/138578647 learned_addr learned my addr 172.21.15.43:0/138578647 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:34.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.269+0000 7fdd4243a640 1 -- 172.21.15.43:0/138578647 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdd3c161f30 msgr2=0x7fdd3c0ccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:34.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.269+0000 7fdd4243a640 1 --2- 172.21.15.43:0/138578647 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdd3c161f30 0x7fdd3c0ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.269+0000 7fdd4243a640 1 -- 172.21.15.43:0/138578647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd3c15df50 msgr2=0x7fdd3c06b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:34.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.269+0000 7fdd4243a640 1 --2- 172.21.15.43:0/138578647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd3c15df50 0x7fdd3c06b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.269+0000 7fdd4243a640 1 -- 172.21.15.43:0/138578647 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd3c0caf70 con 0x7fdd3c0c7cf0 2024-04-25T09:00:34.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.270+0000 7fdd4243a640 1 --2- 172.21.15.43:0/138578647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd3c0c7cf0 0x7fdd3c1e7a30 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fdd3800d830 tx=0x7fdd3800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:34.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.270+0000 7fdd32ffd640 1 -- 172.21.15.43:0/138578647 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3800b6c0 con 0x7fdd3c0c7cf0 2024-04-25T09:00:34.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.270+0000 7fdd32ffd640 1 -- 172.21.15.43:0/138578647 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd38019070 con 0x7fdd3c0c7cf0 2024-04-25T09:00:34.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.270+0000 7fdd4469e640 1 -- 172.21.15.43:0/138578647 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd3c1e8880 con 0x7fdd3c0c7cf0 2024-04-25T09:00:34.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.270+0000 7fdd32ffd640 1 -- 172.21.15.43:0/138578647 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3800ba80 con 0x7fdd3c0c7cf0 2024-04-25T09:00:34.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.270+0000 7fdd4469e640 1 -- 172.21.15.43:0/138578647 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd3c1e7f70 con 0x7fdd3c0c7cf0 2024-04-25T09:00:34.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.271+0000 7fdd4469e640 1 -- 172.21.15.43:0/138578647 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd3c0680f0 con 0x7fdd3c0c7cf0 2024-04-25T09:00:34.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.271+0000 7fdd32ffd640 1 -- 172.21.15.43:0/138578647 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdd3801e460 con 0x7fdd3c0c7cf0 2024-04-25T09:00:34.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.272+0000 7fdd32ffd640 1 --2- 172.21.15.43:0/138578647 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdd2006b4d0 0x7fdd2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:34.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.272+0000 7fdd32ffd640 1 -- 172.21.15.43:0/138578647 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fdd3808a6c0 con 0x7fdd3c0c7cf0 2024-04-25T09:00:34.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.272+0000 7fdd41c39640 1 --2- 172.21.15.43:0/138578647 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdd2006b4d0 0x7fdd2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:34.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.273+0000 7fdd41c39640 1 --2- 172.21.15.43:0/138578647 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdd2006b4d0 0x7fdd2006d990 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c034000 tx=0x7fdd2c02c000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:34.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.276+0000 7fdd32ffd640 1 -- 172.21.15.43:0/138578647 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdd380597b0 con 0x7fdd3c0c7cf0 2024-04-25T09:00:34.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.526+0000 7fdd4469e640 1 -- 172.21.15.43:0/138578647 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]} v 0) v1 -- 0x7fdd3c074ef0 con 0x7fdd3c0c7cf0 2024-04-25T09:00:34.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.809+0000 7fdd32ffd640 1 -- 172.21.15.43:0/138578647 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]}]=0 v303) v1 ==== 158+0+0 (secure 0 0 0) 0x7fdd380590b0 con 0x7fdd3c0c7cf0 2024-04-25T09:00:34.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.811+0000 7fdd4469e640 1 -- 172.21.15.43:0/138578647 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdd2006b4d0 msgr2=0x7fdd2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:34.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.811+0000 7fdd4469e640 1 --2- 172.21.15.43:0/138578647 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdd2006b4d0 0x7fdd2006d990 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fdd2c034000 tx=0x7fdd2c02c000 comp rx=0 tx=0).stop 2024-04-25T09:00:34.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.811+0000 7fdd4469e640 1 -- 172.21.15.43:0/138578647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd3c0c7cf0 msgr2=0x7fdd3c1e7a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:34.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.811+0000 7fdd4469e640 1 --2- 172.21.15.43:0/138578647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd3c0c7cf0 0x7fdd3c1e7a30 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fdd3800d830 tx=0x7fdd3800dcf0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.811+0000 7fdd4469e640 1 -- 172.21.15.43:0/138578647 shutdown_connections 2024-04-25T09:00:34.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.811+0000 7fdd4469e640 1 --2- 172.21.15.43:0/138578647 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdd3c161f30 0x7fdd3c0ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.811+0000 7fdd4469e640 1 --2- 172.21.15.43:0/138578647 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdd2006b4d0 0x7fdd2006d990 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.811+0000 7fdd4469e640 1 --2- 172.21.15.43:0/138578647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd3c15df50 0x7fdd3c06b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.811+0000 7fdd4469e640 1 --2- 172.21.15.43:0/138578647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd3c0c7cf0 0x7fdd3c1e7a30 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.811+0000 7fdd4469e640 1 -- 172.21.15.43:0/138578647 >> 172.21.15.43:0/138578647 conn(0x7fdd3c0d3a30 msgr2=0x7fdd3c073b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:34.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.811+0000 7fdd4469e640 1 -- 172.21.15.43:0/138578647 shutdown_connections 2024-04-25T09:00:34.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.811+0000 7fdd4469e640 1 -- 172.21.15.43:0/138578647 wait complete. 2024-04-25T09:00:34.812 INFO:tasks.cephfs_test_runner:test that scrub can iterate over ~mdsdir and evaluate strays ... ok 2024-04-25T09:00:34.813 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T09:00:34.813 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:00:34.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.928+0000 7f426e917640 1 Processor -- start 2024-04-25T09:00:34.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.928+0000 7f426e917640 1 -- start start 2024-04-25T09:00:34.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.929+0000 7f426e917640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f426806be50 0x7f4268162100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:34.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.929+0000 7f426e917640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f426815e2d0 con 0x7f42680b20c0 2024-04-25T09:00:34.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.929+0000 7f426e917640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42680b1020 con 0x7f426815def0 2024-04-25T09:00:34.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.929+0000 7f426e917640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42680d0e90 con 0x7f426806be50 2024-04-25T09:00:34.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.929+0000 7f426ceb4640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f42680b20c0 0x7f42680b2550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58166/0 (socket says 172.21.15.43:58166) 2024-04-25T09:00:34.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.929+0000 7f426ceb4640 1 -- 172.21.15.43:0/3667531082 learned_addr learned my addr 172.21.15.43:0/3667531082 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:00:34.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.929+0000 7f42677fe640 1 --2- 172.21.15.43:0/3667531082 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f426806be50 0x7f4268162100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:34.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.929+0000 7f4266ffd640 1 -- 172.21.15.43:0/3667531082 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 728355216 0 0) 0x7f426815e2d0 con 0x7f42680b20c0 2024-04-25T09:00:34.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.929+0000 7f4266ffd640 1 -- 172.21.15.43:0/3667531082 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4250003520 con 0x7f42680b20c0 2024-04-25T09:00:34.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.929+0000 7f4266ffd640 1 -- 172.21.15.43:0/3667531082 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2355494208 0 0) 0x7f42680b1020 con 0x7f426815def0 2024-04-25T09:00:34.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.929+0000 7f4266ffd640 1 -- 172.21.15.43:0/3667531082 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f426815e2d0 con 0x7f426815def0 2024-04-25T09:00:34.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.930+0000 7f4266ffd640 1 -- 172.21.15.43:0/3667531082 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 311686462 0 0) 0x7f4250003520 con 0x7f42680b20c0 2024-04-25T09:00:34.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.930+0000 7f4266ffd640 1 -- 172.21.15.43:0/3667531082 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f426806be50 msgr2=0x7f4268162100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:34.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.930+0000 7f4266ffd640 1 --2- 172.21.15.43:0/3667531082 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f426806be50 0x7f4268162100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.930+0000 7f4266ffd640 1 -- 172.21.15.43:0/3667531082 >> v1:172.21.15.193:6789/0 conn(0x7f426815def0 legacy=0x7f42681619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:34.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.930+0000 7f4266ffd640 1 -- 172.21.15.43:0/3667531082 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42680b1300 con 0x7f42680b20c0 2024-04-25T09:00:34.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.930+0000 7f4266ffd640 1 -- 172.21.15.43:0/3667531082 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f42600019f0 con 0x7f42680b20c0 2024-04-25T09:00:34.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.930+0000 7f4266ffd640 1 -- 172.21.15.43:0/3667531082 >> v1:172.21.15.43:6789/0 conn(0x7f42680b20c0 legacy=0x7f42680b2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:34.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.930+0000 7f4266ffd640 1 --2- 172.21.15.43:0/3667531082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4250003b70 0x7f4250003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:34.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.930+0000 7f4266ffd640 1 --2- 172.21.15.43:0/3667531082 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f426806be50 0x7f4250004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:34.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.930+0000 7f4266ffd640 1 --2- 172.21.15.43:0/3667531082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4250004d40 0x7f42500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:34.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.930+0000 7f4266ffd640 1 -- 172.21.15.43:0/3667531082 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42680d0e90 con 0x7f4250003b70 2024-04-25T09:00:34.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.930+0000 7f4266ffd640 1 -- 172.21.15.43:0/3667531082 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425000ba70 con 0x7f4250004d40 2024-04-25T09:00:34.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.930+0000 7f4266ffd640 1 -- 172.21.15.43:0/3667531082 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f425000bc20 con 0x7f426806be50 2024-04-25T09:00:34.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.930+0000 7f4267fff640 1 --2- 172.21.15.43:0/3667531082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4250003b70 0x7f4250003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:34.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.930+0000 7f42677fe640 1 --2- 172.21.15.43:0/3667531082 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f426806be50 0x7f4250004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:34.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.930+0000 7f426ceb4640 1 --2- 172.21.15.43:0/3667531082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4250004d40 0x7f42500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:34.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.930+0000 7f4267fff640 1 -- 172.21.15.43:0/3667531082 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f426806be50 msgr2=0x7f4250004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:34.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.930+0000 7f4267fff640 1 --2- 172.21.15.43:0/3667531082 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f426806be50 0x7f4250004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.930+0000 7f4267fff640 1 -- 172.21.15.43:0/3667531082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4250004d40 msgr2=0x7f42500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:34.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.930+0000 7f4267fff640 1 --2- 172.21.15.43:0/3667531082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4250004d40 0x7f42500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.930+0000 7f4267fff640 1 -- 172.21.15.43:0/3667531082 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4250003520 con 0x7f4250003b70 2024-04-25T09:00:34.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.931+0000 7f4267fff640 1 --2- 172.21.15.43:0/3667531082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4250003b70 0x7f4250003f70 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f425c0018d0 tx=0x7f4250005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4369b23992c23cde server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:34.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.931+0000 7f4266ffd640 1 -- 172.21.15.43:0/3667531082 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425c0014f0 con 0x7f4250003b70 2024-04-25T09:00:34.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.931+0000 7f4266ffd640 1 -- 172.21.15.43:0/3667531082 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f425c0052d0 con 0x7f4250003b70 2024-04-25T09:00:34.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.931+0000 7f4266ffd640 1 -- 172.21.15.43:0/3667531082 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f425c02aca0 con 0x7f4250003b70 2024-04-25T09:00:34.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.931+0000 7f426e917640 1 -- 172.21.15.43:0/3667531082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4250003b70 msgr2=0x7f4250003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:34.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.931+0000 7f426e917640 1 --2- 172.21.15.43:0/3667531082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4250003b70 0x7f4250003f70 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f425c0018d0 tx=0x7f4250005830 comp rx=0 tx=0).stop 2024-04-25T09:00:34.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.931+0000 7f426ceb4640 1 -- 172.21.15.43:0/3667531082 reap_dead start 2024-04-25T09:00:34.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.931+0000 7f426e917640 1 -- 172.21.15.43:0/3667531082 shutdown_connections 2024-04-25T09:00:34.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.931+0000 7f426e917640 1 -- 172.21.15.43:0/3667531082 >> 172.21.15.43:0/3667531082 conn(0x7f42680b32e0 msgr2=0x7f42680b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:34.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.932+0000 7f426e917640 1 -- 172.21.15.43:0/3667531082 shutdown_connections 2024-04-25T09:00:34.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.932+0000 7f426e917640 1 -- 172.21.15.43:0/3667531082 wait complete. 2024-04-25T09:00:34.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.933+0000 7f426e917640 1 Processor -- start 2024-04-25T09:00:34.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.933+0000 7f426e917640 1 -- start start 2024-04-25T09:00:34.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.934+0000 7f426e917640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4250003b70 0x7f4268158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:34.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.934+0000 7f426e917640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f426809e6f0 0x7f426809b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:34.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.934+0000 7f426e917640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f426809bc80 0x7f42680b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:34.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.934+0000 7f426e917640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42680b1ba0 con 0x7f426809bc80 2024-04-25T09:00:34.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.934+0000 7f426e917640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42680b18c0 con 0x7f426809e6f0 2024-04-25T09:00:34.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.934+0000 7f426e917640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f426806c2b0 con 0x7f4250003b70 2024-04-25T09:00:34.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.934+0000 7f426ceb4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f426809bc80 0x7f42680b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:34.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.934+0000 7f426ceb4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f426809bc80 0x7f42680b53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52448/0 (socket says 172.21.15.43:52448) 2024-04-25T09:00:34.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.934+0000 7f426ceb4640 1 -- 172.21.15.43:0/1257720127 learned_addr learned my addr 172.21.15.43:0/1257720127 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:34.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.934+0000 7f4267fff640 1 --2- 172.21.15.43:0/1257720127 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4250003b70 0x7f4268158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:34.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.934+0000 7f42677fe640 1 --2- 172.21.15.43:0/1257720127 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f426809e6f0 0x7f426809b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:34.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.934+0000 7f426ceb4640 1 -- 172.21.15.43:0/1257720127 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4250003b70 msgr2=0x7f4268158ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:34.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.934+0000 7f426ceb4640 1 --2- 172.21.15.43:0/1257720127 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4250003b70 0x7f4268158ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.934+0000 7f426ceb4640 1 -- 172.21.15.43:0/1257720127 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f426809e6f0 msgr2=0x7f426809b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:34.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.934+0000 7f426ceb4640 1 --2- 172.21.15.43:0/1257720127 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f426809e6f0 0x7f426809b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:34.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.934+0000 7f426ceb4640 1 -- 172.21.15.43:0/1257720127 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f426809d3b0 con 0x7f426809bc80 2024-04-25T09:00:34.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.934+0000 7f426ceb4640 1 --2- 172.21.15.43:0/1257720127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f426809bc80 0x7f42680b53a0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f4268071e30 tx=0x7f426000a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:34.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.934+0000 7f4264ff9640 1 -- 172.21.15.43:0/1257720127 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426000aca0 con 0x7f426809bc80 2024-04-25T09:00:34.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.934+0000 7f4264ff9640 1 -- 172.21.15.43:0/1257720127 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4260018070 con 0x7f426809bc80 2024-04-25T09:00:34.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.934+0000 7f426e917640 1 -- 172.21.15.43:0/1257720127 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f426809d650 con 0x7f426809bc80 2024-04-25T09:00:34.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.934+0000 7f4264ff9640 1 -- 172.21.15.43:0/1257720127 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426000ae60 con 0x7f426809bc80 2024-04-25T09:00:34.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.935+0000 7f426e917640 1 -- 172.21.15.43:0/1257720127 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f426809dc10 con 0x7f426809bc80 2024-04-25T09:00:34.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.936+0000 7f426e917640 1 -- 172.21.15.43:0/1257720127 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4268074ef0 con 0x7f426809bc80 2024-04-25T09:00:34.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.936+0000 7f4264ff9640 1 -- 172.21.15.43:0/1257720127 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4260013630 con 0x7f426809bc80 2024-04-25T09:00:34.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.936+0000 7f4264ff9640 1 --2- 172.21.15.43:0/1257720127 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f425406b4d0 0x7f425406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:34.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.936+0000 7f4264ff9640 1 -- 172.21.15.43:0/1257720127 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f426008a480 con 0x7f426809bc80 2024-04-25T09:00:34.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.936+0000 7f4267fff640 1 --2- 172.21.15.43:0/1257720127 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f425406b4d0 0x7f425406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:34.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.937+0000 7f4267fff640 1 --2- 172.21.15.43:0/1257720127 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f425406b4d0 0x7f425406d990 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f425c032000 tx=0x7f425c009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:34.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:34.941+0000 7f4264ff9640 1 -- 172.21.15.43:0/1257720127 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4260059690 con 0x7f426809bc80 2024-04-25T09:00:35.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.120+0000 7f426e917640 1 -- 172.21.15.43:0/1257720127 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42680b7710 con 0x7f426809bc80 2024-04-25T09:00:35.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.121+0000 7f4264ff9640 1 -- 172.21.15.43:0/1257720127 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+12987 (secure 0 0 0) 0x7f4260058f90 con 0x7f426809bc80 2024-04-25T09:00:35.110 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:00:35.110 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":80,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:00:33.807207+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.193:6836/3458098162":"2024-04-26T09:00:30.802647+0000","172.21.15.193:6835/3458098162":"2024-04-26T09:00:30.802647+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:00:35.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.123+0000 7f426e917640 1 -- 172.21.15.43:0/1257720127 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f425406b4d0 msgr2=0x7f425406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:35.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.123+0000 7f426e917640 1 --2- 172.21.15.43:0/1257720127 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f425406b4d0 0x7f425406d990 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f425c032000 tx=0x7f425c009030 comp rx=0 tx=0).stop 2024-04-25T09:00:35.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.123+0000 7f426e917640 1 -- 172.21.15.43:0/1257720127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f426809bc80 msgr2=0x7f42680b53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:35.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.123+0000 7f426e917640 1 --2- 172.21.15.43:0/1257720127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f426809bc80 0x7f42680b53a0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f4268071e30 tx=0x7f426000a6a0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.123+0000 7f426e917640 1 -- 172.21.15.43:0/1257720127 shutdown_connections 2024-04-25T09:00:35.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.123+0000 7f426e917640 1 --2- 172.21.15.43:0/1257720127 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f426809bc80 0x7f42680b53a0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.123+0000 7f426e917640 1 --2- 172.21.15.43:0/1257720127 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f425406b4d0 0x7f425406d990 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.123+0000 7f426e917640 1 --2- 172.21.15.43:0/1257720127 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f426809e6f0 0x7f426809b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.123+0000 7f426e917640 1 --2- 172.21.15.43:0/1257720127 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4250003b70 0x7f4268158ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.123+0000 7f426e917640 1 -- 172.21.15.43:0/1257720127 >> 172.21.15.43:0/1257720127 conn(0x7f42680b32e0 msgr2=0x7f426815d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:35.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.123+0000 7f426e917640 1 -- 172.21.15.43:0/1257720127 shutdown_connections 2024-04-25T09:00:35.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.123+0000 7f426e917640 1 -- 172.21.15.43:0/1257720127 wait complete. 2024-04-25T09:00:35.123 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:00:35.124 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:00:35.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.239+0000 7f8ded4f5640 1 Processor -- start 2024-04-25T09:00:35.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.239+0000 7f8ded4f5640 1 -- start start 2024-04-25T09:00:35.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.240+0000 7f8ded4f5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8de80d1a20 0x7f8de80b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:35.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.240+0000 7f8ded4f5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8de80d2260 0x7f8de80b5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:35.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.240+0000 7f8ded4f5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8de815bf90 con 0x7f8de80d1e00 2024-04-25T09:00:35.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.240+0000 7f8ded4f5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8de80cf430 con 0x7f8de80d2260 2024-04-25T09:00:35.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.240+0000 7f8ded4f5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8de80cee70 con 0x7f8de80d1a20 2024-04-25T09:00:35.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.240+0000 7f8de77fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8de80d2260 0x7f8de80b5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:35.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.240+0000 7f8de67fc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8de80d1e00 0x7f8de80b5100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58178/0 (socket says 172.21.15.43:58178) 2024-04-25T09:00:35.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.240+0000 7f8de67fc640 1 -- 172.21.15.43:0/1267075183 learned_addr learned my addr 172.21.15.43:0/1267075183 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:00:35.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.240+0000 7f8de6ffd640 1 --2- 172.21.15.43:0/1267075183 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8de80d1a20 0x7f8de80b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:35.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.240+0000 7f8de77fe640 1 -- 172.21.15.43:0/1267075183 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8de80d1a20 msgr2=0x7f8de80b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:35.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.240+0000 7f8de77fe640 1 --2- 172.21.15.43:0/1267075183 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8de80d1a20 0x7f8de80b2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.240+0000 7f8de77fe640 1 -- 172.21.15.43:0/1267075183 >> v1:172.21.15.43:6789/0 conn(0x7f8de80d1e00 legacy=0x7f8de80b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:35.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.240+0000 7f8de5ffb640 1 -- 172.21.15.43:0/1267075183 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2559206155 0 0) 0x7f8de815bf90 con 0x7f8de80d1e00 2024-04-25T09:00:35.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.240+0000 7f8de77fe640 1 -- 172.21.15.43:0/1267075183 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de80cf150 con 0x7f8de80d2260 2024-04-25T09:00:35.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.241+0000 7f8de77fe640 1 --2- 172.21.15.43:0/1267075183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8de80d2260 0x7f8de80b5810 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f8dd80097a0 tx=0x7f8dd802a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d0c06dc6cfe26de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:35.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.241+0000 7f8de5ffb640 1 -- 172.21.15.43:0/1267075183 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd8037070 con 0x7f8de80d2260 2024-04-25T09:00:35.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.241+0000 7f8de5ffb640 1 -- 172.21.15.43:0/1267075183 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dd802adb0 con 0x7f8de80d2260 2024-04-25T09:00:35.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.241+0000 7f8de5ffb640 1 -- 172.21.15.43:0/1267075183 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd8032660 con 0x7f8de80d2260 2024-04-25T09:00:35.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.242+0000 7f8ded4f5640 1 -- 172.21.15.43:0/1267075183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8de80d2260 msgr2=0x7f8de80b5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:35.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.242+0000 7f8ded4f5640 1 --2- 172.21.15.43:0/1267075183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8de80d2260 0x7f8de80b5810 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f8dd80097a0 tx=0x7f8dd802a060 comp rx=0 tx=0).stop 2024-04-25T09:00:35.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.242+0000 7f8ded4f5640 1 -- 172.21.15.43:0/1267075183 shutdown_connections 2024-04-25T09:00:35.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.242+0000 7f8ded4f5640 1 --2- 172.21.15.43:0/1267075183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8de80d2260 0x7f8de80b5810 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.242+0000 7f8ded4f5640 1 --2- 172.21.15.43:0/1267075183 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8de80d1a20 0x7f8de80b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.242+0000 7f8ded4f5640 1 -- 172.21.15.43:0/1267075183 >> 172.21.15.43:0/1267075183 conn(0x7f8de8072aa0 msgr2=0x7f8de809aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:35.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.242+0000 7f8ded4f5640 1 -- 172.21.15.43:0/1267075183 shutdown_connections 2024-04-25T09:00:35.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.242+0000 7f8ded4f5640 1 -- 172.21.15.43:0/1267075183 wait complete. 2024-04-25T09:00:35.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.243+0000 7f8ded4f5640 1 Processor -- start 2024-04-25T09:00:35.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.243+0000 7f8ded4f5640 1 -- start start 2024-04-25T09:00:35.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.244+0000 7f8ded4f5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8de80d1a20 0x7f8de815c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:35.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.244+0000 7f8ded4f5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8de80d1e00 0x7f8de815aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:35.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.244+0000 7f8ded4f5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8de80d2260 0x7f8de815afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:35.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.244+0000 7f8ded4f5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de80cea60 con 0x7f8de80d2260 2024-04-25T09:00:35.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.244+0000 7f8ded4f5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8de80b4be0 con 0x7f8de80d1e00 2024-04-25T09:00:35.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.244+0000 7f8ded4f5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8de80cebe0 con 0x7f8de80d1a20 2024-04-25T09:00:35.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.244+0000 7f8de77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8de80d2260 0x7f8de815afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:35.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.244+0000 7f8de77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8de80d2260 0x7f8de815afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52478/0 (socket says 172.21.15.43:52478) 2024-04-25T09:00:35.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.244+0000 7f8de77fe640 1 -- 172.21.15.43:0/826426966 learned_addr learned my addr 172.21.15.43:0/826426966 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:35.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.244+0000 7f8de6ffd640 1 --2- 172.21.15.43:0/826426966 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8de80d1a20 0x7f8de815c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:35.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.244+0000 7f8de67fc640 1 --2- 172.21.15.43:0/826426966 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8de80d1e00 0x7f8de815aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:35.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.244+0000 7f8de77fe640 1 -- 172.21.15.43:0/826426966 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8de80d1a20 msgr2=0x7f8de815c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:35.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.244+0000 7f8de77fe640 1 --2- 172.21.15.43:0/826426966 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8de80d1a20 0x7f8de815c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.244+0000 7f8de77fe640 1 -- 172.21.15.43:0/826426966 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8de80d1e00 msgr2=0x7f8de815aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:35.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.244+0000 7f8de77fe640 1 --2- 172.21.15.43:0/826426966 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8de80d1e00 0x7f8de815aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.244+0000 7f8de77fe640 1 -- 172.21.15.43:0/826426966 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de806bce0 con 0x7f8de80d2260 2024-04-25T09:00:35.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.244+0000 7f8de67fc640 1 --2- 172.21.15.43:0/826426966 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8de80d1e00 0x7f8de815aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:35.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.245+0000 7f8de77fe640 1 --2- 172.21.15.43:0/826426966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8de80d2260 0x7f8de815afc0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f8dd80098c0 tx=0x7f8dd80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:35.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.245+0000 7f8dcf7fe640 1 -- 172.21.15.43:0/826426966 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd8041020 con 0x7f8de80d2260 2024-04-25T09:00:35.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.245+0000 7f8dcf7fe640 1 -- 172.21.15.43:0/826426966 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dd803c670 con 0x7f8de80d2260 2024-04-25T09:00:35.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.245+0000 7f8ded4f5640 1 -- 172.21.15.43:0/826426966 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8de815e7b0 con 0x7f8de80d2260 2024-04-25T09:00:35.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.245+0000 7f8dcf7fe640 1 -- 172.21.15.43:0/826426966 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dd8037040 con 0x7f8de80d2260 2024-04-25T09:00:35.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.245+0000 7f8ded4f5640 1 -- 172.21.15.43:0/826426966 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8de806c260 con 0x7f8de80d2260 2024-04-25T09:00:35.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.246+0000 7f8ded4f5640 1 -- 172.21.15.43:0/826426966 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8de8074f60 con 0x7f8de80d2260 2024-04-25T09:00:35.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.246+0000 7f8dcf7fe640 1 -- 172.21.15.43:0/826426966 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8dd8044020 con 0x7f8de80d2260 2024-04-25T09:00:35.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.247+0000 7f8dcf7fe640 1 --2- 172.21.15.43:0/826426966 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8dd406b4d0 0x7f8dd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:35.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.247+0000 7f8dcf7fe640 1 -- 172.21.15.43:0/826426966 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8dd80aa5d0 con 0x7f8de80d2260 2024-04-25T09:00:35.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.247+0000 7f8de6ffd640 1 --2- 172.21.15.43:0/826426966 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8dd406b4d0 0x7f8dd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:35.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.247+0000 7f8de6ffd640 1 --2- 172.21.15.43:0/826426966 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8dd406b4d0 0x7f8dd406d990 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8de8151cb0 tx=0x7f8de000b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:35.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.251+0000 7f8dcf7fe640 1 -- 172.21.15.43:0/826426966 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8dd804b030 con 0x7f8de80d2260 2024-04-25T09:00:35.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.429+0000 7f8ded4f5640 1 -- 172.21.15.43:0/826426966 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8de815b840 con 0x7f8de80d2260 2024-04-25T09:00:35.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.430+0000 7f8dcf7fe640 1 -- 172.21.15.43:0/826426966 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8dd80797e0 con 0x7f8de80d2260 2024-04-25T09:00:35.419 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T09:00:35.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.432+0000 7f8ded4f5640 1 -- 172.21.15.43:0/826426966 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8dd406b4d0 msgr2=0x7f8dd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:35.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.432+0000 7f8ded4f5640 1 --2- 172.21.15.43:0/826426966 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8dd406b4d0 0x7f8dd406d990 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f8de8151cb0 tx=0x7f8de000b040 comp rx=0 tx=0).stop 2024-04-25T09:00:35.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.432+0000 7f8ded4f5640 1 -- 172.21.15.43:0/826426966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8de80d2260 msgr2=0x7f8de815afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:35.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.432+0000 7f8ded4f5640 1 --2- 172.21.15.43:0/826426966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8de80d2260 0x7f8de815afc0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f8dd80098c0 tx=0x7f8dd80046c0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.432+0000 7f8ded4f5640 1 -- 172.21.15.43:0/826426966 shutdown_connections 2024-04-25T09:00:35.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.432+0000 7f8ded4f5640 1 --2- 172.21.15.43:0/826426966 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8de80d2260 0x7f8de815afc0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.432+0000 7f8ded4f5640 1 --2- 172.21.15.43:0/826426966 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8dd406b4d0 0x7f8dd406d990 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.432+0000 7f8ded4f5640 1 --2- 172.21.15.43:0/826426966 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8de80d1e00 0x7f8de815aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.432+0000 7f8ded4f5640 1 --2- 172.21.15.43:0/826426966 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8de80d1a20 0x7f8de815c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.432+0000 7f8ded4f5640 1 -- 172.21.15.43:0/826426966 >> 172.21.15.43:0/826426966 conn(0x7f8de8072aa0 msgr2=0x7f8de806ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:35.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.432+0000 7f8ded4f5640 1 -- 172.21.15.43:0/826426966 shutdown_connections 2024-04-25T09:00:35.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.432+0000 7f8ded4f5640 1 -- 172.21.15.43:0/826426966 wait complete. 2024-04-25T09:00:35.432 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort' 2024-04-25T09:00:35.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.544+0000 7efdb6bc4640 1 Processor -- start 2024-04-25T09:00:35.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.544+0000 7efdb6bc4640 1 -- start start 2024-04-25T09:00:35.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.545+0000 7efdb6bc4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7efdb00cbfc0 0x7efdb006dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:35.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.545+0000 7efdb6bc4640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efdb00ccbb0 0x7efdb00cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:35.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.545+0000 7efdb6bc4640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7efdb015f5e0 0x7efdb015fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:35.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.545+0000 7efdb6bc4640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdb00cd550 con 0x7efdb00cbfc0 2024-04-25T09:00:35.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.545+0000 7efdb6bc4640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdb00b2780 con 0x7efdb015f5e0 2024-04-25T09:00:35.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.545+0000 7efdb6bc4640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdb00b2a60 con 0x7efdb00ccbb0 2024-04-25T09:00:35.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.545+0000 7efdb4960640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7efdb00cbfc0 0x7efdb006dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:35.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.545+0000 7efdb4960640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7efdb00cbfc0 0x7efdb006dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52492/0 (socket says 172.21.15.43:52492) 2024-04-25T09:00:35.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.545+0000 7efdb4960640 1 -- 172.21.15.43:0/1188303138 learned_addr learned my addr 172.21.15.43:0/1188303138 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:35.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.545+0000 7efdb5161640 1 --2- 172.21.15.43:0/1188303138 >> v2:172.21.15.193:3300/0 conn(0x7efdb015f5e0 0x7efdb015fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:35.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.545+0000 7efda7fff640 1 --2- 172.21.15.43:0/1188303138 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efdb00ccbb0 0x7efdb00cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:35.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.545+0000 7efdb4960640 1 -- 172.21.15.43:0/1188303138 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efdb00ccbb0 msgr2=0x7efdb00cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:35.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.545+0000 7efdb4960640 1 --2- 172.21.15.43:0/1188303138 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efdb00ccbb0 0x7efdb00cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.545+0000 7efdb4960640 1 -- 172.21.15.43:0/1188303138 >> v2:172.21.15.193:3300/0 conn(0x7efdb015f5e0 msgr2=0x7efdb015fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:35.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.545+0000 7efdb4960640 1 --2- 172.21.15.43:0/1188303138 >> v2:172.21.15.193:3300/0 conn(0x7efdb015f5e0 0x7efdb015fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.545+0000 7efdb4960640 1 -- 172.21.15.43:0/1188303138 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb00b2d40 con 0x7efdb00cbfc0 2024-04-25T09:00:35.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.545+0000 7efda7fff640 1 --2- 172.21.15.43:0/1188303138 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efdb00ccbb0 0x7efdb00cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:35.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.546+0000 7efdb4960640 1 --2- 172.21.15.43:0/1188303138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdb00cbfc0 0x7efdb006dc60 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7efdb00cd7b0 tx=0x7efdac02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abef94bb4c5f6c9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:35.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.546+0000 7efda77fe640 1 -- 172.21.15.43:0/1188303138 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdac037070 con 0x7efdb00cbfc0 2024-04-25T09:00:35.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.546+0000 7efda77fe640 1 -- 172.21.15.43:0/1188303138 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdac032440 con 0x7efdb00cbfc0 2024-04-25T09:00:35.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.546+0000 7efda77fe640 1 -- 172.21.15.43:0/1188303138 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdac032810 con 0x7efdb00cbfc0 2024-04-25T09:00:35.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.547+0000 7efdb6bc4640 1 -- 172.21.15.43:0/1188303138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdb00cbfc0 msgr2=0x7efdb006dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:35.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.547+0000 7efdb6bc4640 1 --2- 172.21.15.43:0/1188303138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdb00cbfc0 0x7efdb006dc60 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7efdb00cd7b0 tx=0x7efdac02a1e0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.547+0000 7efdb6bc4640 1 -- 172.21.15.43:0/1188303138 shutdown_connections 2024-04-25T09:00:35.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.547+0000 7efdb6bc4640 1 --2- 172.21.15.43:0/1188303138 >> v2:172.21.15.193:3300/0 conn(0x7efdb015f5e0 0x7efdb015fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.547+0000 7efdb6bc4640 1 --2- 172.21.15.43:0/1188303138 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efdb00ccbb0 0x7efdb00cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.547+0000 7efdb6bc4640 1 --2- 172.21.15.43:0/1188303138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdb00cbfc0 0x7efdb006dc60 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.547+0000 7efdb6bc4640 1 -- 172.21.15.43:0/1188303138 >> 172.21.15.43:0/1188303138 conn(0x7efdb00b1480 msgr2=0x7efdb00b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:35.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.547+0000 7efdb6bc4640 1 -- 172.21.15.43:0/1188303138 shutdown_connections 2024-04-25T09:00:35.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.547+0000 7efdb6bc4640 1 -- 172.21.15.43:0/1188303138 wait complete. 2024-04-25T09:00:35.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.548+0000 7efdb6bc4640 1 Processor -- start 2024-04-25T09:00:35.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.548+0000 7efdb6bc4640 1 -- start start 2024-04-25T09:00:35.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.549+0000 7efdb6bc4640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efdb00cbfc0 0x7efdb0133a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:35.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.549+0000 7efdb6bc4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdb00ccbb0 0x7efdb01f2360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:35.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.549+0000 7efdb6bc4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdb015f5e0 0x7efdb01eed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:35.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.549+0000 7efdb6bc4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb00b35e0 con 0x7efdb00ccbb0 2024-04-25T09:00:35.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.549+0000 7efdb6bc4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb00b3300 con 0x7efdb015f5e0 2024-04-25T09:00:35.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.549+0000 7efdb6bc4640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdb006e1a0 con 0x7efdb00cbfc0 2024-04-25T09:00:35.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.549+0000 7efda7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdb00ccbb0 0x7efdb01f2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:35.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.549+0000 7efda7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdb00ccbb0 0x7efdb01f2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52496/0 (socket says 172.21.15.43:52496) 2024-04-25T09:00:35.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.549+0000 7efda7fff640 1 -- 172.21.15.43:0/3550673376 learned_addr learned my addr 172.21.15.43:0/3550673376 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:35.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.549+0000 7efda7fff640 1 -- 172.21.15.43:0/3550673376 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efdb00cbfc0 msgr2=0x7efdb0133a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:35.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.549+0000 7efdb5161640 1 --2- 172.21.15.43:0/3550673376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdb015f5e0 0x7efdb01eed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:35.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.549+0000 7efda7fff640 1 --2- 172.21.15.43:0/3550673376 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efdb00cbfc0 0x7efdb0133a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.549+0000 7efda7fff640 1 -- 172.21.15.43:0/3550673376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdb015f5e0 msgr2=0x7efdb01eed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:35.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.549+0000 7efda7fff640 1 --2- 172.21.15.43:0/3550673376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdb015f5e0 0x7efdb01eed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:35.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.549+0000 7efda7fff640 1 -- 172.21.15.43:0/3550673376 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb01f28a0 con 0x7efdb00ccbb0 2024-04-25T09:00:35.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.549+0000 7efda7fff640 1 --2- 172.21.15.43:0/3550673376 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdb00ccbb0 0x7efdb01f2360 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7efda000c7f0 tx=0x7efda000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:35.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.549+0000 7efda57fa640 1 -- 172.21.15.43:0/3550673376 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda0015070 con 0x7efdb00ccbb0 2024-04-25T09:00:35.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.549+0000 7efda57fa640 1 -- 172.21.15.43:0/3550673376 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efda0007a30 con 0x7efdb00ccbb0 2024-04-25T09:00:35.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.549+0000 7efdb6bc4640 1 -- 172.21.15.43:0/3550673376 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdb0134070 con 0x7efdb00ccbb0 2024-04-25T09:00:35.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.549+0000 7efda57fa640 1 -- 172.21.15.43:0/3550673376 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda0007e00 con 0x7efdb00ccbb0 2024-04-25T09:00:35.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.549+0000 7efdb6bc4640 1 -- 172.21.15.43:0/3550673376 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdb0134630 con 0x7efdb00ccbb0 2024-04-25T09:00:35.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.550+0000 7efdb6bc4640 1 -- 172.21.15.43:0/3550673376 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdb0074ef0 con 0x7efdb00ccbb0 2024-04-25T09:00:35.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.551+0000 7efda57fa640 1 -- 172.21.15.43:0/3550673376 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efda0018070 con 0x7efdb00ccbb0 2024-04-25T09:00:35.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.551+0000 7efda57fa640 1 --2- 172.21.15.43:0/3550673376 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7efd9c06b4d0 0x7efd9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:35.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.551+0000 7efda57fa640 1 -- 172.21.15.43:0/3550673376 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4943+0+0 (secure 0 0 0) 0x7efda0089e40 con 0x7efdb00ccbb0 2024-04-25T09:00:35.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.552+0000 7efdb4960640 1 --2- 172.21.15.43:0/3550673376 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7efd9c06b4d0 0x7efd9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:35.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.552+0000 7efdb4960640 1 --2- 172.21.15.43:0/3550673376 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7efd9c06b4d0 0x7efd9c06d990 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7efdac034000 tx=0x7efdac02c000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:35.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.555+0000 7efda57fa640 1 -- 172.21.15.43:0/3550673376 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efdb01f28a0 con 0x7efdb00ccbb0 2024-04-25T09:00:35.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:35.811+0000 7efdb6bc4640 1 -- 172.21.15.43:0/3550673376 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]} v 0) v1 -- 0x7efdb00af0d0 con 0x7efdb00ccbb0 2024-04-25T09:00:36.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.815+0000 7efda57fa640 1 -- 172.21.15.43:0/3550673376 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]}]=0 v305) v1 ==== 147+0+0 (secure 0 0 0) 0x7efda0058fc0 con 0x7efdb00ccbb0 2024-04-25T09:00:36.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.818+0000 7efdb6bc4640 1 -- 172.21.15.43:0/3550673376 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7efd9c06b4d0 msgr2=0x7efd9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:36.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.818+0000 7efdb6bc4640 1 --2- 172.21.15.43:0/3550673376 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7efd9c06b4d0 0x7efd9c06d990 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7efdac034000 tx=0x7efdac02c000 comp rx=0 tx=0).stop 2024-04-25T09:00:36.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.818+0000 7efdb6bc4640 1 -- 172.21.15.43:0/3550673376 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdb00ccbb0 msgr2=0x7efdb01f2360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:36.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.818+0000 7efdb6bc4640 1 --2- 172.21.15.43:0/3550673376 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdb00ccbb0 0x7efdb01f2360 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7efda000c7f0 tx=0x7efda000ccb0 comp rx=0 tx=0).stop 2024-04-25T09:00:36.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.818+0000 7efdb6bc4640 1 -- 172.21.15.43:0/3550673376 shutdown_connections 2024-04-25T09:00:36.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.818+0000 7efdb6bc4640 1 --2- 172.21.15.43:0/3550673376 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7efd9c06b4d0 0x7efd9c06d990 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:36.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.818+0000 7efdb6bc4640 1 --2- 172.21.15.43:0/3550673376 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efdb015f5e0 0x7efdb01eed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:36.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.818+0000 7efdb6bc4640 1 --2- 172.21.15.43:0/3550673376 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efdb00ccbb0 0x7efdb01f2360 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:36.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.818+0000 7efdb6bc4640 1 --2- 172.21.15.43:0/3550673376 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efdb00cbfc0 0x7efdb0133a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:36.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.818+0000 7efdb6bc4640 1 -- 172.21.15.43:0/3550673376 >> 172.21.15.43:0/3550673376 conn(0x7efdb00b1480 msgr2=0x7efdb015c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:36.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.818+0000 7efdb6bc4640 1 -- 172.21.15.43:0/3550673376 shutdown_connections 2024-04-25T09:00:36.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.818+0000 7efdb6bc4640 1 -- 172.21.15.43:0/3550673376 wait complete. 2024-04-25T09:00:36.821 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T09:00:36.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.943+0000 7f19ce334640 1 Processor -- start 2024-04-25T09:00:36.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.943+0000 7f19ce334640 1 -- start start 2024-04-25T09:00:36.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.944+0000 7f19ce334640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f19c815e090 0x7f19c8161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:36.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.944+0000 7f19ce334640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f19c8161fe0 0x7f19c80b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:36.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.944+0000 7f19ce334640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19c8162610 con 0x7f19c80b5aa0 2024-04-25T09:00:36.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.944+0000 7f19ce334640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19c80b5f00 con 0x7f19c8161fe0 2024-04-25T09:00:36.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.944+0000 7f19ce334640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19c80d1da0 con 0x7f19c815e090 2024-04-25T09:00:36.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.944+0000 7f19cc8d1640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f19c80b5aa0 0x7f19c80ce560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58190/0 (socket says 172.21.15.43:58190) 2024-04-25T09:00:36.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.944+0000 7f19c77fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f19c8161fe0 0x7f19c80b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:36.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.944+0000 7f19cc8d1640 1 -- 172.21.15.43:0/3997880995 learned_addr learned my addr 172.21.15.43:0/3997880995 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:00:36.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.945+0000 7f19c7fff640 1 --2- 172.21.15.43:0/3997880995 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f19c815e090 0x7f19c8161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:36.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.945+0000 7f19c77fe640 1 -- 172.21.15.43:0/3997880995 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f19c815e090 msgr2=0x7f19c8161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:36.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.945+0000 7f19c77fe640 1 --2- 172.21.15.43:0/3997880995 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f19c815e090 0x7f19c8161aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:36.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.945+0000 7f19c77fe640 1 -- 172.21.15.43:0/3997880995 >> v1:172.21.15.43:6789/0 conn(0x7f19c80b5aa0 legacy=0x7f19c80ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:36.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.945+0000 7f19c6ffd640 1 -- 172.21.15.43:0/3997880995 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1718474059 0 0) 0x7f19c8162610 con 0x7f19c80b5aa0 2024-04-25T09:00:36.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.945+0000 7f19c77fe640 1 -- 172.21.15.43:0/3997880995 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c80d1750 con 0x7f19c8161fe0 2024-04-25T09:00:36.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.945+0000 7f19c77fe640 1 --2- 172.21.15.43:0/3997880995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19c8161fe0 0x7f19c80b5560 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f19bc0097a0 tx=0x7f19bc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60cdfb7b633c4e18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:36.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.946+0000 7f19c6ffd640 1 -- 172.21.15.43:0/3997880995 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19bc037070 con 0x7f19c8161fe0 2024-04-25T09:00:36.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.946+0000 7f19c6ffd640 1 -- 172.21.15.43:0/3997880995 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19bc032440 con 0x7f19c8161fe0 2024-04-25T09:00:36.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.946+0000 7f19c6ffd640 1 -- 172.21.15.43:0/3997880995 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19bc032810 con 0x7f19c8161fe0 2024-04-25T09:00:36.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.946+0000 7f19ce334640 1 -- 172.21.15.43:0/3997880995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19c8161fe0 msgr2=0x7f19c80b5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:36.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.946+0000 7f19ce334640 1 --2- 172.21.15.43:0/3997880995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19c8161fe0 0x7f19c80b5560 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f19bc0097a0 tx=0x7f19bc02a210 comp rx=0 tx=0).stop 2024-04-25T09:00:36.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.946+0000 7f19ce334640 1 -- 172.21.15.43:0/3997880995 shutdown_connections 2024-04-25T09:00:36.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.946+0000 7f19ce334640 1 --2- 172.21.15.43:0/3997880995 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19c8161fe0 0x7f19c80b5560 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:36.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.946+0000 7f19ce334640 1 --2- 172.21.15.43:0/3997880995 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f19c815e090 0x7f19c8161aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:36.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.946+0000 7f19ce334640 1 -- 172.21.15.43:0/3997880995 >> 172.21.15.43:0/3997880995 conn(0x7f19c80d0560 msgr2=0x7f19c80d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:36.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.946+0000 7f19ce334640 1 -- 172.21.15.43:0/3997880995 shutdown_connections 2024-04-25T09:00:36.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.946+0000 7f19ce334640 1 -- 172.21.15.43:0/3997880995 wait complete. 2024-04-25T09:00:36.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.947+0000 7f19ce334640 1 Processor -- start 2024-04-25T09:00:36.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.948+0000 7f19ce334640 1 -- start start 2024-04-25T09:00:36.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.948+0000 7f19ce334640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19c80b5aa0 0x7f19c81603e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:36.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.948+0000 7f19ce334640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f19c815e090 0x7f19c81565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:36.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.948+0000 7f19ce334640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19c8161fe0 0x7f19c80c9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:36.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.948+0000 7f19ce334640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c80b3280 con 0x7f19c80b5aa0 2024-04-25T09:00:36.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.948+0000 7f19ce334640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c81628f0 con 0x7f19c8161fe0 2024-04-25T09:00:36.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.948+0000 7f19ce334640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19c80d3240 con 0x7f19c815e090 2024-04-25T09:00:36.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.948+0000 7f19c7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19c80b5aa0 0x7f19c81603e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:36.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.949+0000 7f19c7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19c80b5aa0 0x7f19c81603e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52524/0 (socket says 172.21.15.43:52524) 2024-04-25T09:00:36.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.949+0000 7f19c7fff640 1 -- 172.21.15.43:0/552142014 learned_addr learned my addr 172.21.15.43:0/552142014 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:36.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.949+0000 7f19cc8d1640 1 --2- 172.21.15.43:0/552142014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19c8161fe0 0x7f19c80c9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:36.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.949+0000 7f19c7fff640 1 -- 172.21.15.43:0/552142014 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f19c815e090 msgr2=0x7f19c81565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:36.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.949+0000 7f19c77fe640 1 --2- 172.21.15.43:0/552142014 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f19c815e090 0x7f19c81565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:36.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.949+0000 7f19c7fff640 1 --2- 172.21.15.43:0/552142014 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f19c815e090 0x7f19c81565c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:36.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.949+0000 7f19c7fff640 1 -- 172.21.15.43:0/552142014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19c8161fe0 msgr2=0x7f19c80c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:36.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.949+0000 7f19c7fff640 1 --2- 172.21.15.43:0/552142014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19c8161fe0 0x7f19c80c9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:36.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.949+0000 7f19c7fff640 1 -- 172.21.15.43:0/552142014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c8160cc0 con 0x7f19c80b5aa0 2024-04-25T09:00:36.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.949+0000 7f19cc8d1640 1 --2- 172.21.15.43:0/552142014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19c8161fe0 0x7f19c80c9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:36.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.949+0000 7f19c7fff640 1 --2- 172.21.15.43:0/552142014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19c80b5aa0 0x7f19c81603e0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f19b800e9b0 tx=0x7f19b800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:36.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.949+0000 7f19c4ff9640 1 -- 172.21.15.43:0/552142014 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b8017070 con 0x7f19c80b5aa0 2024-04-25T09:00:36.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.949+0000 7f19c4ff9640 1 -- 172.21.15.43:0/552142014 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19b800c9a0 con 0x7f19c80b5aa0 2024-04-25T09:00:36.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.949+0000 7f19ce334640 1 -- 172.21.15.43:0/552142014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19c8153db0 con 0x7f19c80b5aa0 2024-04-25T09:00:36.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.949+0000 7f19c4ff9640 1 -- 172.21.15.43:0/552142014 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b800cd90 con 0x7f19c80b5aa0 2024-04-25T09:00:36.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.949+0000 7f19ce334640 1 -- 172.21.15.43:0/552142014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19c8160920 con 0x7f19c80b5aa0 2024-04-25T09:00:36.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.950+0000 7f19ce334640 1 -- 172.21.15.43:0/552142014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19c806bf00 con 0x7f19c80b5aa0 2024-04-25T09:00:36.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.951+0000 7f19c4ff9640 1 -- 172.21.15.43:0/552142014 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f19b801a070 con 0x7f19c80b5aa0 2024-04-25T09:00:36.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.951+0000 7f19c4ff9640 1 --2- 172.21.15.43:0/552142014 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f19b406b500 0x7f19b406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:36.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.951+0000 7f19c4ff9640 1 -- 172.21.15.43:0/552142014 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f19b808a290 con 0x7f19c80b5aa0 2024-04-25T09:00:36.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.951+0000 7f19c77fe640 1 --2- 172.21.15.43:0/552142014 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f19b406b500 0x7f19b406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:36.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.952+0000 7f19c77fe640 1 --2- 172.21.15.43:0/552142014 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f19b406b500 0x7f19b406d9c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f19bc034000 tx=0x7f19bc02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:36.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:36.956+0000 7f19c4ff9640 1 -- 172.21.15.43:0/552142014 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f19c806bf00 con 0x7f19c80b5aa0 2024-04-25T09:00:37.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.134+0000 7f19ce334640 1 -- 172.21.15.43:0/552142014 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f19c8153ba0 con 0x7f19c80b5aa0 2024-04-25T09:00:37.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.137+0000 7f19c4ff9640 1 -- 172.21.15.43:0/552142014 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f19b8059410 con 0x7f19c80b5aa0 2024-04-25T09:00:37.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.140+0000 7f19ce334640 1 -- 172.21.15.43:0/552142014 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f19b406b500 msgr2=0x7f19b406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:37.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.140+0000 7f19ce334640 1 --2- 172.21.15.43:0/552142014 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f19b406b500 0x7f19b406d9c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f19bc034000 tx=0x7f19bc02e000 comp rx=0 tx=0).stop 2024-04-25T09:00:37.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.140+0000 7f19ce334640 1 -- 172.21.15.43:0/552142014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19c80b5aa0 msgr2=0x7f19c81603e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:37.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.140+0000 7f19ce334640 1 --2- 172.21.15.43:0/552142014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19c80b5aa0 0x7f19c81603e0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f19b800e9b0 tx=0x7f19b800ee70 comp rx=0 tx=0).stop 2024-04-25T09:00:37.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.140+0000 7f19ce334640 1 -- 172.21.15.43:0/552142014 shutdown_connections 2024-04-25T09:00:37.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.140+0000 7f19ce334640 1 --2- 172.21.15.43:0/552142014 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f19b406b500 0x7f19b406d9c0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.140+0000 7f19ce334640 1 --2- 172.21.15.43:0/552142014 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19c8161fe0 0x7f19c80c9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.140+0000 7f19ce334640 1 --2- 172.21.15.43:0/552142014 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f19c815e090 0x7f19c81565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.140+0000 7f19ce334640 1 --2- 172.21.15.43:0/552142014 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19c80b5aa0 0x7f19c81603e0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.140+0000 7f19ce334640 1 -- 172.21.15.43:0/552142014 >> 172.21.15.43:0/552142014 conn(0x7f19c80d0560 msgr2=0x7f19c80b7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:37.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.140+0000 7f19ce334640 1 -- 172.21.15.43:0/552142014 shutdown_connections 2024-04-25T09:00:37.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.140+0000 7f19ce334640 1 -- 172.21.15.43:0/552142014 wait complete. 2024-04-25T09:00:37.140 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:00:37.140 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:00:37.176 DEBUG:teuthology.orchestra.run.smithi193:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:00:37.218 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:00:37.219 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T09:00:37.219 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:00:37.234 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2374 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:00:37.235 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T09:00:37.235 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:00:37.276 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19508 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:00:37.276 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T09:00:37.277 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:00:37.291 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2374 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:00:37.292 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T09:00:37.292 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:00:37.334 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19508 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:00:37.334 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T09:00:37.334 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:00:37.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.446+0000 7f94067ce640 1 Processor -- start 2024-04-25T09:00:37.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.446+0000 7f94067ce640 1 -- start start 2024-04-25T09:00:37.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.447+0000 7f94067ce640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f940015dce0 0x7f940015ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:37.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.447+0000 7f94067ce640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f94000ce550 0x7f94000d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:37.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.447+0000 7f94067ce640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94000cee80 con 0x7f94000ce990 2024-04-25T09:00:37.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.447+0000 7f94067ce640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94000d2920 con 0x7f94000ce550 2024-04-25T09:00:37.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.447+0000 7f94067ce640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94000cf160 con 0x7f940015dce0 2024-04-25T09:00:37.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.447+0000 7f93ff7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f94000ce550 0x7f94000d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:37.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.447+0000 7f93ff7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f94000ce550 0x7f94000d03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52534/0 (socket says 172.21.15.43:52534) 2024-04-25T09:00:37.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.447+0000 7f93ff7fe640 1 -- 172.21.15.43:0/3189717479 learned_addr learned my addr 172.21.15.43:0/3189717479 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:37.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.447+0000 7f93fffff640 1 --2- 172.21.15.43:0/3189717479 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f940015dce0 0x7f940015ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:37.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.447+0000 7f93ff7fe640 1 -- 172.21.15.43:0/3189717479 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f940015dce0 msgr2=0x7f940015ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:37.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.447+0000 7f93ff7fe640 1 --2- 172.21.15.43:0/3189717479 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f940015dce0 0x7f940015ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.447+0000 7f93ff7fe640 1 -- 172.21.15.43:0/3189717479 >> v1:172.21.15.193:6789/0 conn(0x7f94000ce990 legacy=0x7f9400160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:37.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.447+0000 7f93ff7fe640 1 -- 172.21.15.43:0/3189717479 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94000cf440 con 0x7f94000ce550 2024-04-25T09:00:37.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.447+0000 7f93fffff640 1 --2- 172.21.15.43:0/3189717479 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f940015dce0 0x7f940015ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:00:37.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.448+0000 7f93ff7fe640 1 --2- 172.21.15.43:0/3189717479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94000ce550 0x7f94000d03e0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f93f4009870 tx=0x7f93f402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a1a3a601774ae3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:37.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.448+0000 7f93feffd640 1 -- 172.21.15.43:0/3189717479 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f4037070 con 0x7f94000ce550 2024-04-25T09:00:37.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.448+0000 7f93feffd640 1 -- 172.21.15.43:0/3189717479 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f4032440 con 0x7f94000ce550 2024-04-25T09:00:37.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.448+0000 7f93feffd640 1 -- 172.21.15.43:0/3189717479 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f4032810 con 0x7f94000ce550 2024-04-25T09:00:37.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.449+0000 7f94067ce640 1 -- 172.21.15.43:0/3189717479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94000ce550 msgr2=0x7f94000d03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:37.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.449+0000 7f94067ce640 1 --2- 172.21.15.43:0/3189717479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94000ce550 0x7f94000d03e0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f93f4009870 tx=0x7f93f402a1c0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.449+0000 7f94067ce640 1 -- 172.21.15.43:0/3189717479 shutdown_connections 2024-04-25T09:00:37.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.449+0000 7f94067ce640 1 --2- 172.21.15.43:0/3189717479 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94000ce550 0x7f94000d03e0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.449+0000 7f94067ce640 1 --2- 172.21.15.43:0/3189717479 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f940015dce0 0x7f940015ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.449+0000 7f94067ce640 1 -- 172.21.15.43:0/3189717479 >> 172.21.15.43:0/3189717479 conn(0x7f940009cd00 msgr2=0x7f94000d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:37.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.449+0000 7f94067ce640 1 -- 172.21.15.43:0/3189717479 shutdown_connections 2024-04-25T09:00:37.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.449+0000 7f94067ce640 1 -- 172.21.15.43:0/3189717479 wait complete. 2024-04-25T09:00:37.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.450+0000 7f94067ce640 1 Processor -- start 2024-04-25T09:00:37.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.450+0000 7f94067ce640 1 -- start start 2024-04-25T09:00:37.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.451+0000 7f94067ce640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94000ce550 0x7f94000b6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:37.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.451+0000 7f94067ce640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94000ce990 0x7f94001591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:37.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.451+0000 7f94067ce640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f940015dce0 0x7f9400159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:37.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.451+0000 7f94067ce640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f940015e0c0 con 0x7f94000ce550 2024-04-25T09:00:37.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.451+0000 7f94067ce640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f940015f6b0 con 0x7f94000ce990 2024-04-25T09:00:37.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.451+0000 7f94067ce640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94000d0920 con 0x7f940015dce0 2024-04-25T09:00:37.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.451+0000 7f93fffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94000ce550 0x7f94000b6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:37.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.451+0000 7f93fffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94000ce550 0x7f94000b6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52544/0 (socket says 172.21.15.43:52544) 2024-04-25T09:00:37.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.451+0000 7f93fffff640 1 -- 172.21.15.43:0/4194889341 learned_addr learned my addr 172.21.15.43:0/4194889341 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:37.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.451+0000 7f93fffff640 1 -- 172.21.15.43:0/4194889341 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f940015dce0 msgr2=0x7f9400159700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:37.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.451+0000 7f93fffff640 1 --2- 172.21.15.43:0/4194889341 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f940015dce0 0x7f9400159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.451+0000 7f93fffff640 1 -- 172.21.15.43:0/4194889341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94000ce990 msgr2=0x7f94001591c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:37.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.451+0000 7f93ff7fe640 1 --2- 172.21.15.43:0/4194889341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94000ce990 0x7f94001591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:37.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.451+0000 7f93fffff640 1 --2- 172.21.15.43:0/4194889341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94000ce990 0x7f94001591c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.451+0000 7f93fffff640 1 -- 172.21.15.43:0/4194889341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94000cbc20 con 0x7f94000ce550 2024-04-25T09:00:37.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.451+0000 7f93ff7fe640 1 --2- 172.21.15.43:0/4194889341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94000ce990 0x7f94001591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:37.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.451+0000 7f93fffff640 1 --2- 172.21.15.43:0/4194889341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94000ce550 0x7f94000b6f70 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f93f000d800 tx=0x7f93f000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:37.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.451+0000 7f93fcff9640 1 -- 172.21.15.43:0/4194889341 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f0004030 con 0x7f94000ce550 2024-04-25T09:00:37.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.452+0000 7f93fcff9640 1 -- 172.21.15.43:0/4194889341 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93f00041d0 con 0x7f94000ce550 2024-04-25T09:00:37.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.452+0000 7f93fcff9640 1 -- 172.21.15.43:0/4194889341 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93f000b6c0 con 0x7f94000ce550 2024-04-25T09:00:37.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.452+0000 7f94067ce640 1 -- 172.21.15.43:0/4194889341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94000b7670 con 0x7f94000ce550 2024-04-25T09:00:37.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.452+0000 7f94067ce640 1 -- 172.21.15.43:0/4194889341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94000b7c30 con 0x7f94000ce550 2024-04-25T09:00:37.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.452+0000 7f94067ce640 1 -- 172.21.15.43:0/4194889341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94000680f0 con 0x7f94000ce550 2024-04-25T09:00:37.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.453+0000 7f93fcff9640 1 -- 172.21.15.43:0/4194889341 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f93f000b860 con 0x7f94000ce550 2024-04-25T09:00:37.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.453+0000 7f93fcff9640 1 --2- 172.21.15.43:0/4194889341 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f93ec06b4d0 0x7f93ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:37.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.454+0000 7f93fcff9640 1 -- 172.21.15.43:0/4194889341 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f93f008a130 con 0x7f94000ce550 2024-04-25T09:00:37.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.454+0000 7f93ff7fe640 1 --2- 172.21.15.43:0/4194889341 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f93ec06b4d0 0x7f93ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:37.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.454+0000 7f93ff7fe640 1 --2- 172.21.15.43:0/4194889341 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f93ec06b4d0 0x7f93ec06d990 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f93f4034000 tx=0x7f93f402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:37.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.458+0000 7f93fcff9640 1 -- 172.21.15.43:0/4194889341 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f93f0059340 con 0x7f94000ce550 2024-04-25T09:00:37.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.674+0000 7f94067ce640 1 -- 172.21.15.43:0/4194889341 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94000b7e70 con 0x7f94000ce550 2024-04-25T09:00:37.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.675+0000 7f93fcff9640 1 -- 172.21.15.43:0/4194889341 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 50 v50) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f94000b7e70 con 0x7f94000ce550 2024-04-25T09:00:37.664 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:00:37.664 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":50,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6373,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3648859409","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":3648859409},{"type":"v1","addr":"172.21.15.193:6837","nonce":3648859409}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":6661,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/3867640889","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":3867640889},{"type":"v1","addr":"172.21.15.193:6836","nonce":3867640889}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7149,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/356516962","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":356516962},{"type":"v1","addr":"172.21.15.43:6834","nonce":356516962}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":42},{"gid":7152,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/3032637719","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":3032637719},{"type":"v1","addr":"172.21.15.43:6835","nonce":3032637719}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[]} 2024-04-25T09:00:37.664 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 50 2024-04-25T09:00:37.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.677+0000 7f94067ce640 1 -- 172.21.15.43:0/4194889341 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f93ec06b4d0 msgr2=0x7f93ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:37.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.677+0000 7f94067ce640 1 --2- 172.21.15.43:0/4194889341 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f93ec06b4d0 0x7f93ec06d990 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f93f4034000 tx=0x7f93f402e000 comp rx=0 tx=0).stop 2024-04-25T09:00:37.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.677+0000 7f94067ce640 1 -- 172.21.15.43:0/4194889341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94000ce550 msgr2=0x7f94000b6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:37.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.677+0000 7f94067ce640 1 --2- 172.21.15.43:0/4194889341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94000ce550 0x7f94000b6f70 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f93f000d800 tx=0x7f93f000dcc0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.677+0000 7f94067ce640 1 -- 172.21.15.43:0/4194889341 shutdown_connections 2024-04-25T09:00:37.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.677+0000 7f94067ce640 1 --2- 172.21.15.43:0/4194889341 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f940015dce0 0x7f9400159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.677+0000 7f94067ce640 1 --2- 172.21.15.43:0/4194889341 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f93ec06b4d0 0x7f93ec06d990 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.677+0000 7f94067ce640 1 --2- 172.21.15.43:0/4194889341 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f94000ce990 0x7f94001591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.677+0000 7f94067ce640 1 --2- 172.21.15.43:0/4194889341 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f94000ce550 0x7f94000b6f70 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.677+0000 7f94067ce640 1 -- 172.21.15.43:0/4194889341 >> 172.21.15.43:0/4194889341 conn(0x7f940009cd00 msgr2=0x7f94000c74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:37.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.677+0000 7f94067ce640 1 -- 172.21.15.43:0/4194889341 shutdown_connections 2024-04-25T09:00:37.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.677+0000 7f94067ce640 1 -- 172.21.15.43:0/4194889341 wait complete. 2024-04-25T09:00:37.678 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:00:37.678 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T09:00:37.678 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:00:37.678 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:00:37.678 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:00:37.678 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T09:00:37.679 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:00:37.679 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:00:37.679 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:00:37.679 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T09:00:37.679 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:00:37.679 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:00:37.679 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:00:37.679 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T09:00:37.680 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:00:37.680 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:00:37.690 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:00:37.691 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:00:37.692 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:00:37.692 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:00:37.694 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:00:37.695 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:00:37.696 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:00:37.696 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:00:37.733 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:00:37.733 DEBUG:teuthology.parallel:result is None 2024-04-25T09:00:37.734 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:00:37.734 DEBUG:teuthology.parallel:result is None 2024-04-25T09:00:37.737 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:00:37.737 DEBUG:teuthology.parallel:result is None 2024-04-25T09:00:37.738 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:00:37.738 DEBUG:teuthology.parallel:result is None 2024-04-25T09:00:37.738 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T09:00:37.809 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-04-25T09:00:37.810 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2024-04-25T09:00:37.817 INFO:tasks.ceph.mds.b.smithi193.stdout:starting mds.b at 2024-04-25T09:00:37.817 INFO:tasks.ceph.mds.d.smithi193.stdout:starting mds.d at 2024-04-25T09:00:37.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.857+0000 7fbe961f5640 1 Processor -- start 2024-04-25T09:00:37.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.857+0000 7fbe961f5640 1 -- start start 2024-04-25T09:00:37.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.858+0000 7fbe961f5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbe9015df50 0x7fbe901619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:37.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.858+0000 7fbe961f5640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fbe900cf670 0x7fbe90162380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:37.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.858+0000 7fbe961f5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe900d24a0 con 0x7fbe90161f30 2024-04-25T09:00:37.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.858+0000 7fbe961f5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe900cfa80 con 0x7fbe9015df50 2024-04-25T09:00:37.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.858+0000 7fbe961f5640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe9015e330 con 0x7fbe900cf670 2024-04-25T09:00:37.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.858+0000 7fbe8effd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fbe90161f30 0x7fbe900c7cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58208/0 (socket says 172.21.15.43:58208) 2024-04-25T09:00:37.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.858+0000 7fbe8effd640 1 -- 172.21.15.43:0/2644013493 learned_addr learned my addr 172.21.15.43:0/2644013493 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:00:37.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.858+0000 7fbe8f7fe640 1 --2- 172.21.15.43:0/2644013493 >> v2:172.21.15.43:3300/0 conn(0x7fbe9015df50 0x7fbe901619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:37.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.858+0000 7fbe8ffff640 1 --2- 172.21.15.43:0/2644013493 >> v2:172.21.15.193:3300/0 conn(0x7fbe900cf670 0x7fbe90162380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:37.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.858+0000 7fbe8e7fc640 1 -- 172.21.15.43:0/2644013493 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3155074814 0 0) 0x7fbe900d24a0 con 0x7fbe90161f30 2024-04-25T09:00:37.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.858+0000 7fbe8e7fc640 1 -- 172.21.15.43:0/2644013493 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe78003520 con 0x7fbe90161f30 2024-04-25T09:00:37.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.859+0000 7fbe8f7fe640 1 -- 172.21.15.43:0/2644013493 >> v2:172.21.15.193:3300/0 conn(0x7fbe900cf670 msgr2=0x7fbe90162380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:37.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.859+0000 7fbe8f7fe640 1 --2- 172.21.15.43:0/2644013493 >> v2:172.21.15.193:3300/0 conn(0x7fbe900cf670 0x7fbe90162380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.859+0000 7fbe8f7fe640 1 -- 172.21.15.43:0/2644013493 >> v1:172.21.15.43:6789/0 conn(0x7fbe90161f30 legacy=0x7fbe900c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:37.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.859+0000 7fbe8e7fc640 1 -- 172.21.15.43:0/2644013493 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 237546983 0 0) 0x7fbe78003520 con 0x7fbe90161f30 2024-04-25T09:00:37.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.859+0000 7fbe8f7fe640 1 -- 172.21.15.43:0/2644013493 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe900d2740 con 0x7fbe9015df50 2024-04-25T09:00:37.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.859+0000 7fbe8f7fe640 1 --2- 172.21.15.43:0/2644013493 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe9015df50 0x7fbe901619f0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fbe900caa00 tx=0x7fbe8002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddfbdbe78c58bd41 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:37.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.859+0000 7fbe8e7fc640 1 -- 172.21.15.43:0/2644013493 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe80037070 con 0x7fbe9015df50 2024-04-25T09:00:37.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.859+0000 7fbe8e7fc640 1 -- 172.21.15.43:0/2644013493 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe8002abc0 con 0x7fbe9015df50 2024-04-25T09:00:37.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.860+0000 7fbe8e7fc640 1 -- 172.21.15.43:0/2644013493 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe84001690 con 0x7fbe9015df50 2024-04-25T09:00:37.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.860+0000 7fbe961f5640 1 -- 172.21.15.43:0/2644013493 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe9015df50 msgr2=0x7fbe901619f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:37.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.860+0000 7fbe961f5640 1 --2- 172.21.15.43:0/2644013493 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe9015df50 0x7fbe901619f0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fbe900caa00 tx=0x7fbe8002a130 comp rx=0 tx=0).stop 2024-04-25T09:00:37.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.860+0000 7fbe961f5640 1 -- 172.21.15.43:0/2644013493 shutdown_connections 2024-04-25T09:00:37.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.860+0000 7fbe961f5640 1 --2- 172.21.15.43:0/2644013493 >> v2:172.21.15.193:3300/0 conn(0x7fbe900cf670 0x7fbe90162380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.860+0000 7fbe961f5640 1 --2- 172.21.15.43:0/2644013493 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe9015df50 0x7fbe901619f0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.860+0000 7fbe961f5640 1 -- 172.21.15.43:0/2644013493 >> 172.21.15.43:0/2644013493 conn(0x7fbe900d3a30 msgr2=0x7fbe900d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:37.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.860+0000 7fbe961f5640 1 -- 172.21.15.43:0/2644013493 shutdown_connections 2024-04-25T09:00:37.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.860+0000 7fbe961f5640 1 -- 172.21.15.43:0/2644013493 wait complete. 2024-04-25T09:00:37.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.862+0000 7fbe961f5640 1 Processor -- start 2024-04-25T09:00:37.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.862+0000 7fbe961f5640 1 -- start start 2024-04-25T09:00:37.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.863+0000 7fbe961f5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe900cf670 0x7fbe90161510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:37.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.863+0000 7fbe961f5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbe9015df50 0x7fbe9009fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:37.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.863+0000 7fbe961f5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe90161f30 0x7fbe9015bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:37.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.863+0000 7fbe961f5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe900c77d0 con 0x7fbe90161f30 2024-04-25T09:00:37.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.863+0000 7fbe961f5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe900c74f0 con 0x7fbe900cf670 2024-04-25T09:00:37.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.863+0000 7fbe961f5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe900d2190 con 0x7fbe9015df50 2024-04-25T09:00:37.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.863+0000 7fbe8ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe90161f30 0x7fbe9015bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:37.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.863+0000 7fbe8ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe90161f30 0x7fbe9015bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52618/0 (socket says 172.21.15.43:52618) 2024-04-25T09:00:37.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.863+0000 7fbe8ffff640 1 -- 172.21.15.43:0/2001885120 learned_addr learned my addr 172.21.15.43:0/2001885120 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:37.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.863+0000 7fbe8f7fe640 1 --2- 172.21.15.43:0/2001885120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe900cf670 0x7fbe90161510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:37.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.863+0000 7fbe8effd640 1 --2- 172.21.15.43:0/2001885120 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbe9015df50 0x7fbe9009fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:37.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.863+0000 7fbe8ffff640 1 -- 172.21.15.43:0/2001885120 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbe9015df50 msgr2=0x7fbe9009fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:37.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.863+0000 7fbe8ffff640 1 --2- 172.21.15.43:0/2001885120 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbe9015df50 0x7fbe9009fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.863+0000 7fbe8ffff640 1 -- 172.21.15.43:0/2001885120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe900cf670 msgr2=0x7fbe90161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:37.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.863+0000 7fbe8ffff640 1 --2- 172.21.15.43:0/2001885120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe900cf670 0x7fbe90161510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:37.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.863+0000 7fbe8ffff640 1 -- 172.21.15.43:0/2001885120 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe900a0870 con 0x7fbe90161f30 2024-04-25T09:00:37.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.863+0000 7fbe8ffff640 1 --2- 172.21.15.43:0/2001885120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe90161f30 0x7fbe9015bd40 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fbe8800efc0 tx=0x7fbe8800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:37.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.863+0000 7fbe77fff640 1 -- 172.21.15.43:0/2001885120 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe88018020 con 0x7fbe90161f30 2024-04-25T09:00:37.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.863+0000 7fbe77fff640 1 -- 172.21.15.43:0/2001885120 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe88014480 con 0x7fbe90161f30 2024-04-25T09:00:37.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.863+0000 7fbe77fff640 1 -- 172.21.15.43:0/2001885120 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe88014870 con 0x7fbe90161f30 2024-04-25T09:00:37.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.864+0000 7fbe961f5640 1 -- 172.21.15.43:0/2001885120 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe900a0b50 con 0x7fbe90161f30 2024-04-25T09:00:37.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.864+0000 7fbe961f5640 1 -- 172.21.15.43:0/2001885120 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe900a1110 con 0x7fbe90161f30 2024-04-25T09:00:37.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.865+0000 7fbe961f5640 1 -- 172.21.15.43:0/2001885120 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe900680f0 con 0x7fbe90161f30 2024-04-25T09:00:37.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.865+0000 7fbe77fff640 1 -- 172.21.15.43:0/2001885120 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbe88005d10 con 0x7fbe90161f30 2024-04-25T09:00:37.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.865+0000 7fbe77fff640 1 --2- 172.21.15.43:0/2001885120 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbe7c06b500 0x7fbe7c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:37.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.866+0000 7fbe77fff640 1 -- 172.21.15.43:0/2001885120 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbe8808acf0 con 0x7fbe90161f30 2024-04-25T09:00:37.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.866+0000 7fbe8f7fe640 1 --2- 172.21.15.43:0/2001885120 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbe7c06b500 0x7fbe7c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:37.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.866+0000 7fbe8f7fe640 1 --2- 172.21.15.43:0/2001885120 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbe7c06b500 0x7fbe7c06d9c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fbe80034000 tx=0x7fbe8002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:37.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:37.870+0000 7fbe77fff640 1 -- 172.21.15.43:0/2001885120 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbe8808f070 con 0x7fbe90161f30 2024-04-25T09:00:38.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:38.065+0000 7fbe961f5640 1 -- 172.21.15.43:0/2001885120 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fbe90074ef0 con 0x7fbe90161f30 2024-04-25T09:00:39.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.151+0000 7fbe77fff640 1 -- 172.21.15.43:0/2001885120 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v85) v1 ==== 97+0+0 (secure 0 0 0) 0x7fbe88022030 con 0x7fbe90161f30 2024-04-25T09:00:39.140 INFO:teuthology.orchestra.run.smithi043.stderr: removed all blocklist entries 2024-04-25T09:00:39.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.154+0000 7fbe961f5640 1 -- 172.21.15.43:0/2001885120 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbe7c06b500 msgr2=0x7fbe7c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.154+0000 7fbe961f5640 1 --2- 172.21.15.43:0/2001885120 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbe7c06b500 0x7fbe7c06d9c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fbe80034000 tx=0x7fbe8002e000 comp rx=0 tx=0).stop 2024-04-25T09:00:39.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.154+0000 7fbe961f5640 1 -- 172.21.15.43:0/2001885120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe90161f30 msgr2=0x7fbe9015bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.154+0000 7fbe961f5640 1 --2- 172.21.15.43:0/2001885120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe90161f30 0x7fbe9015bd40 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fbe8800efc0 tx=0x7fbe8800c490 comp rx=0 tx=0).stop 2024-04-25T09:00:39.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.154+0000 7fbe961f5640 1 -- 172.21.15.43:0/2001885120 shutdown_connections 2024-04-25T09:00:39.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.154+0000 7fbe961f5640 1 --2- 172.21.15.43:0/2001885120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe90161f30 0x7fbe9015bd40 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.154+0000 7fbe961f5640 1 --2- 172.21.15.43:0/2001885120 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbe9015df50 0x7fbe9009fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.154+0000 7fbe961f5640 1 --2- 172.21.15.43:0/2001885120 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbe7c06b500 0x7fbe7c06d9c0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.154+0000 7fbe961f5640 1 --2- 172.21.15.43:0/2001885120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe900cf670 0x7fbe90161510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.154+0000 7fbe961f5640 1 -- 172.21.15.43:0/2001885120 >> 172.21.15.43:0/2001885120 conn(0x7fbe900d3a30 msgr2=0x7fbe900c87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:39.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.154+0000 7fbe961f5640 1 -- 172.21.15.43:0/2001885120 shutdown_connections 2024-04-25T09:00:39.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.154+0000 7fbe961f5640 1 -- 172.21.15.43:0/2001885120 wait complete. 2024-04-25T09:00:39.157 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T09:00:39.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.275+0000 7f6f829f5640 1 Processor -- start 2024-04-25T09:00:39.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.275+0000 7f6f829f5640 1 -- start start 2024-04-25T09:00:39.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.276+0000 7f6f829f5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6f7c15ef50 0x7f6f7c0d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.276+0000 7f6f829f5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f7c0d2920 con 0x7f6f7c0d0920 2024-04-25T09:00:39.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.276+0000 7f6f829f5640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f7c0cee80 con 0x7f6f7c15dce0 2024-04-25T09:00:39.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.276+0000 7f6f829f5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7c0cec60 con 0x7f6f7c15ef50 2024-04-25T09:00:39.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.276+0000 7f6f80f92640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f6f7c0d0920 0x7f6f7c160980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58220/0 (socket says 172.21.15.43:58220) 2024-04-25T09:00:39.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.276+0000 7f6f80f92640 1 -- 172.21.15.43:0/2726710893 learned_addr learned my addr 172.21.15.43:0/2726710893 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:00:39.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.277+0000 7f6f7b7fe640 1 --2- 172.21.15.43:0/2726710893 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6f7c15ef50 0x7f6f7c0d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.277+0000 7f6f7affd640 1 -- 172.21.15.43:0/2726710893 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1582372081 0 0) 0x7f6f7c0d2920 con 0x7f6f7c0d0920 2024-04-25T09:00:39.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.277+0000 7f6f7affd640 1 -- 172.21.15.43:0/2726710893 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f64003520 con 0x7f6f7c0d0920 2024-04-25T09:00:39.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.277+0000 7f6f7affd640 1 -- 172.21.15.43:0/2726710893 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2005037641 0 0) 0x7f6f64003520 con 0x7f6f7c0d0920 2024-04-25T09:00:39.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.277+0000 7f6f7affd640 1 -- 172.21.15.43:0/2726710893 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6f7c15ef50 msgr2=0x7f6f7c0d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:39.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.277+0000 7f6f7affd640 1 --2- 172.21.15.43:0/2726710893 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6f7c15ef50 0x7f6f7c0d03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.277+0000 7f6f7affd640 1 -- 172.21.15.43:0/2726710893 >> v1:172.21.15.193:6789/0 conn(0x7f6f7c15dce0 legacy=0x7f6f7c0ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.277+0000 7f6f7affd640 1 -- 172.21.15.43:0/2726710893 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f7c0cf160 con 0x7f6f7c0d0920 2024-04-25T09:00:39.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.277+0000 7f6f7affd640 1 -- 172.21.15.43:0/2726710893 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f6f740019f0 con 0x7f6f7c0d0920 2024-04-25T09:00:39.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.277+0000 7f6f7affd640 1 -- 172.21.15.43:0/2726710893 >> v1:172.21.15.43:6789/0 conn(0x7f6f7c0d0920 legacy=0x7f6f7c160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.277+0000 7f6f7affd640 1 --2- 172.21.15.43:0/2726710893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f640039e0 0x7f6f64003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.277+0000 7f6f7affd640 1 --2- 172.21.15.43:0/2726710893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f64004390 0x7f6f64004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.277+0000 7f6f7affd640 1 --2- 172.21.15.43:0/2726710893 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6f7c15ef50 0x7f6f64005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.277+0000 7f6f7affd640 1 -- 172.21.15.43:0/2726710893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7c0cec60 con 0x7f6f640039e0 2024-04-25T09:00:39.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.277+0000 7f6f7affd640 1 -- 172.21.15.43:0/2726710893 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6400ba70 con 0x7f6f64004390 2024-04-25T09:00:39.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.277+0000 7f6f7affd640 1 -- 172.21.15.43:0/2726710893 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6400bc70 con 0x7f6f7c15ef50 2024-04-25T09:00:39.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.277+0000 7f6f7bfff640 1 --2- 172.21.15.43:0/2726710893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f640039e0 0x7f6f64003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.277+0000 7f6f7b7fe640 1 --2- 172.21.15.43:0/2726710893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f64004390 0x7f6f64004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.277+0000 7f6f80f92640 1 --2- 172.21.15.43:0/2726710893 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6f7c15ef50 0x7f6f64005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.277+0000 7f6f7bfff640 1 -- 172.21.15.43:0/2726710893 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6f7c15ef50 msgr2=0x7f6f64005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:39.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.277+0000 7f6f7bfff640 1 --2- 172.21.15.43:0/2726710893 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6f7c15ef50 0x7f6f64005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.277+0000 7f6f7bfff640 1 -- 172.21.15.43:0/2726710893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f64004390 msgr2=0x7f6f64004860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:39.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.277+0000 7f6f7bfff640 1 --2- 172.21.15.43:0/2726710893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f64004390 0x7f6f64004860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.277+0000 7f6f7bfff640 1 -- 172.21.15.43:0/2726710893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f64003520 con 0x7f6f640039e0 2024-04-25T09:00:39.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.278+0000 7f6f7bfff640 1 --2- 172.21.15.43:0/2726710893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f640039e0 0x7f6f64003dc0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f6f70001830 tx=0x7f6f64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44dc6a7cb235cab8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:39.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.278+0000 7f6f7affd640 1 -- 172.21.15.43:0/2726710893 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f70001430 con 0x7f6f640039e0 2024-04-25T09:00:39.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.278+0000 7f6f7affd640 1 -- 172.21.15.43:0/2726710893 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f700052f0 con 0x7f6f640039e0 2024-04-25T09:00:39.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.278+0000 7f6f7affd640 1 -- 172.21.15.43:0/2726710893 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f7002ac10 con 0x7f6f640039e0 2024-04-25T09:00:39.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.279+0000 7f6f829f5640 1 -- 172.21.15.43:0/2726710893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f640039e0 msgr2=0x7f6f64003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.279+0000 7f6f829f5640 1 --2- 172.21.15.43:0/2726710893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f640039e0 0x7f6f64003dc0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f6f70001830 tx=0x7f6f64005830 comp rx=0 tx=0).stop 2024-04-25T09:00:39.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.279+0000 7f6f80f92640 1 -- 172.21.15.43:0/2726710893 reap_dead start 2024-04-25T09:00:39.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.279+0000 7f6f829f5640 1 -- 172.21.15.43:0/2726710893 shutdown_connections 2024-04-25T09:00:39.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.279+0000 7f6f829f5640 1 -- 172.21.15.43:0/2726710893 >> 172.21.15.43:0/2726710893 conn(0x7f6f7c09cd00 msgr2=0x7f6f7c0d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:39.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.279+0000 7f6f829f5640 1 -- 172.21.15.43:0/2726710893 shutdown_connections 2024-04-25T09:00:39.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.279+0000 7f6f829f5640 1 -- 172.21.15.43:0/2726710893 wait complete. 2024-04-25T09:00:39.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.280+0000 7f6f829f5640 1 Processor -- start 2024-04-25T09:00:39.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.280+0000 7f6f829f5640 1 -- start start 2024-04-25T09:00:39.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.281+0000 7f6f829f5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6f640039e0 0x7f6f7c0a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.281+0000 7f6f829f5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f7c15bd80 0x7f6f7c154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.281+0000 7f6f829f5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f7c1f51b0 0x7f6f7c09f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.281+0000 7f6f829f5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7c0cf720 con 0x7f6f7c15bd80 2024-04-25T09:00:39.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.281+0000 7f6f829f5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7c0d0d70 con 0x7f6f7c1f51b0 2024-04-25T09:00:39.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.281+0000 7f6f829f5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7c15f6b0 con 0x7f6f640039e0 2024-04-25T09:00:39.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.281+0000 7f6f7b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f7c15bd80 0x7f6f7c154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.281+0000 7f6f7b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f7c15bd80 0x7f6f7c154e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:52638/0 (socket says 172.21.15.43:52638) 2024-04-25T09:00:39.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.281+0000 7f6f7b7fe640 1 -- 172.21.15.43:0/1082099710 learned_addr learned my addr 172.21.15.43:0/1082099710 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:39.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.281+0000 7f6f80f92640 1 --2- 172.21.15.43:0/1082099710 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f7c1f51b0 0x7f6f7c09f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.281+0000 7f6f7bfff640 1 --2- 172.21.15.43:0/1082099710 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6f640039e0 0x7f6f7c0a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.281+0000 7f6f7b7fe640 1 -- 172.21.15.43:0/1082099710 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6f640039e0 msgr2=0x7f6f7c0a12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.281+0000 7f6f7b7fe640 1 --2- 172.21.15.43:0/1082099710 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6f640039e0 0x7f6f7c0a12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.281+0000 7f6f7b7fe640 1 -- 172.21.15.43:0/1082099710 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f7c1f51b0 msgr2=0x7f6f7c09f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.281+0000 7f6f7b7fe640 1 --2- 172.21.15.43:0/1082099710 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f7c1f51b0 0x7f6f7c09f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.281+0000 7f6f7b7fe640 1 -- 172.21.15.43:0/1082099710 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f7c09eb60 con 0x7f6f7c15bd80 2024-04-25T09:00:39.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.282+0000 7f6f7b7fe640 1 --2- 172.21.15.43:0/1082099710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f7c15bd80 0x7f6f7c154e80 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c0118e0 tx=0x7f6f6c011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:39.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.282+0000 7f6f78ff9640 1 -- 172.21.15.43:0/1082099710 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c00faf0 con 0x7f6f7c15bd80 2024-04-25T09:00:39.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.282+0000 7f6f78ff9640 1 -- 172.21.15.43:0/1082099710 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6c00fc90 con 0x7f6f7c15bd80 2024-04-25T09:00:39.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.282+0000 7f6f829f5640 1 -- 172.21.15.43:0/1082099710 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f7c09ee40 con 0x7f6f7c15bd80 2024-04-25T09:00:39.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.282+0000 7f6f78ff9640 1 -- 172.21.15.43:0/1082099710 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c014440 con 0x7f6f7c15bd80 2024-04-25T09:00:39.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.282+0000 7f6f829f5640 1 -- 172.21.15.43:0/1082099710 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f7c0cae20 con 0x7f6f7c15bd80 2024-04-25T09:00:39.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.283+0000 7f6f829f5640 1 -- 172.21.15.43:0/1082099710 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f7c0680f0 con 0x7f6f7c15bd80 2024-04-25T09:00:39.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.283+0000 7f6f78ff9640 1 -- 172.21.15.43:0/1082099710 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6f6c012040 con 0x7f6f7c15bd80 2024-04-25T09:00:39.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.284+0000 7f6f78ff9640 1 --2- 172.21.15.43:0/1082099710 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6f6806b500 0x7f6f6806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.284+0000 7f6f78ff9640 1 -- 172.21.15.43:0/1082099710 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6f6c08a490 con 0x7f6f7c15bd80 2024-04-25T09:00:39.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.284+0000 7f6f7bfff640 1 --2- 172.21.15.43:0/1082099710 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6f6806b500 0x7f6f6806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.285+0000 7f6f7bfff640 1 --2- 172.21.15.43:0/1082099710 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6f6806b500 0x7f6f6806d9c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f6f70032000 tx=0x7f6f70009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:39.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.288+0000 7f6f78ff9640 1 -- 172.21.15.43:0/1082099710 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6f6c019030 con 0x7f6f7c15bd80 2024-04-25T09:00:39.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.541+0000 7f6f829f5640 1 -- 172.21.15.43:0/1082099710 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f6f7c0ca800 con 0x7f6f7c15bd80 2024-04-25T09:00:39.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.542+0000 7f6f78ff9640 1 -- 172.21.15.43:0/1082099710 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f6f6c059000 con 0x7f6f7c15bd80 2024-04-25T09:00:39.531 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:00:39.531 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:00:39.531 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_dump": [ 2024-04-25T09:00:39.531 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:00:39.531 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.a", 2024-04-25T09:00:39.531 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpmIjzEBRAAs2L10XrFO3mThjZevna5wA==", 2024-04-25T09:00:39.531 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:00:39.531 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T09:00:39.531 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T09:00:39.532 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T09:00:39.532 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:00:39.532 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:00:39.532 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:39.532 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:00:39.532 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.b", 2024-04-25T09:00:39.532 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm51cmERAAij4vy8xyihh8u/4dkmcllw==", 2024-04-25T09:00:39.532 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:00:39.532 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T09:00:39.532 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T09:00:39.532 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T09:00:39.532 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:00:39.532 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:00:39.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:39.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:00:39.533 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.c", 2024-04-25T09:00:39.533 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm1OOtDBAAlUYbCh1mBEEka9CWRnUz9A==", 2024-04-25T09:00:39.533 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:00:39.533 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T09:00:39.533 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T09:00:39.533 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T09:00:39.533 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:00:39.533 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:00:39.533 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:39.533 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:00:39.533 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.d", 2024-04-25T09:00:39.533 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpmpEvsFxAAXX1h37UWdFkyJ3Iw3xd8kw==", 2024-04-25T09:00:39.534 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:00:39.534 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T09:00:39.534 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T09:00:39.534 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T09:00:39.534 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:00:39.534 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:00:39.534 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:39.534 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:00:39.534 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.0", 2024-04-25T09:00:39.534 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD9GSpmi7qkORAAwSNgwzcVMitrPPf3nDyqvQ==", 2024-04-25T09:00:39.534 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:00:39.534 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T09:00:39.534 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T09:00:39.534 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:00:39.535 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:00:39.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:39.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:00:39.535 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.1", 2024-04-25T09:00:39.535 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAAGipmJDT3NxAAcWBCd9nt7Au+Xftjqcuc5w==", 2024-04-25T09:00:39.535 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:00:39.535 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T09:00:39.535 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T09:00:39.535 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:00:39.535 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:00:39.535 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:39.535 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:00:39.535 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.2", 2024-04-25T09:00:39.535 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQADGipm68kzNhAA0ZOnlr4edWDjx6RfoEHYIw==", 2024-04-25T09:00:39.535 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:00:39.536 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T09:00:39.536 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T09:00:39.536 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:00:39.536 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:00:39.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:39.536 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:00:39.536 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.3", 2024-04-25T09:00:39.536 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAGGipmT1AeNBAAoNPiEtkznSxxhVJ1EmZ1ZA==", 2024-04-25T09:00:39.536 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:00:39.536 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T09:00:39.536 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T09:00:39.536 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:00:39.536 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:00:39.536 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:39.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:00:39.537 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.4", 2024-04-25T09:00:39.537 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAMGipmBjHHNxAAH/SCzKHus8fKxxI3UN4wWQ==", 2024-04-25T09:00:39.537 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:00:39.537 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T09:00:39.537 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T09:00:39.537 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:00:39.537 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:00:39.537 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:39.537 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:00:39.537 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.5", 2024-04-25T09:00:39.537 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPGipm3FXKNRAAL97YYWONg4QHr4ajC5D7fA==", 2024-04-25T09:00:39.537 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:00:39.537 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T09:00:39.538 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T09:00:39.538 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:00:39.538 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:00:39.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:39.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:00:39.538 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.6", 2024-04-25T09:00:39.538 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQASGipmCwKaMxAAcM0p9Q2sbNQviGkwSLg8AA==", 2024-04-25T09:00:39.538 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:00:39.538 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T09:00:39.538 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T09:00:39.538 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:00:39.538 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:00:39.538 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:39.538 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:00:39.539 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.7", 2024-04-25T09:00:39.539 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAVGipmP7A2MRAAxnFJFosMcbZ6VDbgH8r6wA==", 2024-04-25T09:00:39.539 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:00:39.539 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T09:00:39.539 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T09:00:39.539 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:00:39.539 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:00:39.539 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:39.539 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:00:39.539 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.0", 2024-04-25T09:00:39.539 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpmVOjWGxAAFNmxeGi0h2D//pa6Ne3eQw==", 2024-04-25T09:00:39.539 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:00:39.539 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T09:00:39.539 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow r", 2024-04-25T09:00:39.540 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T09:00:39.540 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:00:39.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:39.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:00:39.540 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.1", 2024-04-25T09:00:39.540 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm24laJBAA9ijbDlFDTmF/ZZ4IhLfBVQ==", 2024-04-25T09:00:39.540 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:00:39.540 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T09:00:39.540 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-04-25T09:00:39.540 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-04-25T09:00:39.540 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-04-25T09:00:39.540 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:00:39.540 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:39.540 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:00:39.541 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.2", 2024-04-25T09:00:39.541 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm8b5UIRAACcOwDL630oJrXfp7s+fNTA==", 2024-04-25T09:00:39.541 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:00:39.541 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T09:00:39.541 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-04-25T09:00:39.541 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-04-25T09:00:39.541 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-04-25T09:00:39.541 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:00:39.541 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:39.541 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:00:39.541 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.3", 2024-04-25T09:00:39.541 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpmLsbBKRAAIKWtRdpuyCaW1LnKMWTMiw==", 2024-04-25T09:00:39.541 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:00:39.541 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T09:00:39.541 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-04-25T09:00:39.542 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-04-25T09:00:39.542 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-04-25T09:00:39.542 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:00:39.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:39.542 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:00:39.542 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.admin", 2024-04-25T09:00:39.542 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD7GSpmE9C8JRAAS263n6a4L2DXzMyS9o42NQ==", 2024-04-25T09:00:39.542 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:00:39.542 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-04-25T09:00:39.542 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T09:00:39.542 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T09:00:39.542 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:00:39.542 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:00:39.542 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:39.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:00:39.543 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.x", 2024-04-25T09:00:39.543 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm/uGhAhAAr0FaJ6SwZ8Su1MEqyZwNRA==", 2024-04-25T09:00:39.543 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:00:39.543 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-04-25T09:00:39.543 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2024-04-25T09:00:39.543 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:00:39.543 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:00:39.543 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:39.543 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:00:39.543 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.y", 2024-04-25T09:00:39.543 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD7GSpmPwwpOhAALeiQ/LKM2zgjE7yu16RQOg==", 2024-04-25T09:00:39.543 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:00:39.543 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-04-25T09:00:39.543 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2024-04-25T09:00:39.544 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:00:39.544 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:00:39.544 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:00:39.544 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-04-25T09:00:39.544 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:00:39.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.544+0000 7f6f829f5640 1 -- 172.21.15.43:0/1082099710 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6f6806b500 msgr2=0x7f6f6806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.544+0000 7f6f829f5640 1 --2- 172.21.15.43:0/1082099710 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6f6806b500 0x7f6f6806d9c0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f6f70032000 tx=0x7f6f70009030 comp rx=0 tx=0).stop 2024-04-25T09:00:39.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.544+0000 7f6f829f5640 1 -- 172.21.15.43:0/1082099710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f7c15bd80 msgr2=0x7f6f7c154e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.544+0000 7f6f829f5640 1 --2- 172.21.15.43:0/1082099710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f7c15bd80 0x7f6f7c154e80 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c0118e0 tx=0x7f6f6c011da0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.544+0000 7f6f829f5640 1 -- 172.21.15.43:0/1082099710 shutdown_connections 2024-04-25T09:00:39.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.544+0000 7f6f829f5640 1 --2- 172.21.15.43:0/1082099710 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6f6806b500 0x7f6f6806d9c0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.544+0000 7f6f829f5640 1 --2- 172.21.15.43:0/1082099710 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f7c1f51b0 0x7f6f7c09f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.544+0000 7f6f829f5640 1 --2- 172.21.15.43:0/1082099710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f7c15bd80 0x7f6f7c154e80 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.544+0000 7f6f829f5640 1 --2- 172.21.15.43:0/1082099710 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6f640039e0 0x7f6f7c0a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.544+0000 7f6f829f5640 1 -- 172.21.15.43:0/1082099710 >> 172.21.15.43:0/1082099710 conn(0x7f6f7c09cd00 msgr2=0x7f6f7c0cfa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:39.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.544+0000 7f6f829f5640 1 -- 172.21.15.43:0/1082099710 shutdown_connections 2024-04-25T09:00:39.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.544+0000 7f6f829f5640 1 -- 172.21.15.43:0/1082099710 wait complete. 2024-04-25T09:00:39.547 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:00:39.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.658+0000 7f5c6389e640 1 Processor -- start 2024-04-25T09:00:39.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.658+0000 7f5c6389e640 1 -- start start 2024-04-25T09:00:39.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.659+0000 7f5c6389e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c5c06c510 0x7f5c5c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.659+0000 7f5c6389e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5c5c0cd680 0x7f5c5c0ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.659+0000 7f5c6389e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c5c0cbee0 con 0x7f5c5c15df70 2024-04-25T09:00:39.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.659+0000 7f5c6389e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c5c0d0400 con 0x7f5c5c0cd680 2024-04-25T09:00:39.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.659+0000 7f5c6389e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5c06ce30 con 0x7f5c5c06c510 2024-04-25T09:00:39.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.659+0000 7f5c61e3b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5c5c0cd680 0x7f5c5c0ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.659+0000 7f5c60e39640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f5c5c15df70 0x7f5c5c0ce070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44234/0 (socket says 172.21.15.43:44234) 2024-04-25T09:00:39.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.659+0000 7f5c60e39640 1 -- 172.21.15.43:0/3229021968 learned_addr learned my addr 172.21.15.43:0/3229021968 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:00:39.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.660+0000 7f5c6163a640 1 --2- 172.21.15.43:0/3229021968 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c5c06c510 0x7f5c5c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.660+0000 7f5c5bfff640 1 -- 172.21.15.43:0/3229021968 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1578044707 0 0) 0x7f5c5c0cbee0 con 0x7f5c5c15df70 2024-04-25T09:00:39.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.660+0000 7f5c5bfff640 1 -- 172.21.15.43:0/3229021968 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c44003520 con 0x7f5c5c15df70 2024-04-25T09:00:39.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.660+0000 7f5c61e3b640 1 -- 172.21.15.43:0/3229021968 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c5c06c510 msgr2=0x7f5c5c06c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:39.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.660+0000 7f5c61e3b640 1 --2- 172.21.15.43:0/3229021968 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c5c06c510 0x7f5c5c06c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.660+0000 7f5c61e3b640 1 -- 172.21.15.43:0/3229021968 >> v1:172.21.15.43:6789/0 conn(0x7f5c5c15df70 legacy=0x7f5c5c0ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.660+0000 7f5c61e3b640 1 -- 172.21.15.43:0/3229021968 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c5c0d0650 con 0x7f5c5c0cd680 2024-04-25T09:00:39.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.661+0000 7f5c61e3b640 1 --2- 172.21.15.43:0/3229021968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c5c0cd680 0x7f5c5c0ce780 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f5c540097a0 tx=0x7f5c5402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=543d05b51390a253 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:39.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.661+0000 7f5c5bfff640 1 -- 172.21.15.43:0/3229021968 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c54037070 con 0x7f5c5c0cd680 2024-04-25T09:00:39.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.661+0000 7f5c5bfff640 1 -- 172.21.15.43:0/3229021968 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c54032440 con 0x7f5c5c0cd680 2024-04-25T09:00:39.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.661+0000 7f5c5bfff640 1 -- 172.21.15.43:0/3229021968 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c54032810 con 0x7f5c5c0cd680 2024-04-25T09:00:39.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.661+0000 7f5c6389e640 1 -- 172.21.15.43:0/3229021968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c5c0cd680 msgr2=0x7f5c5c0ce780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.661+0000 7f5c6389e640 1 --2- 172.21.15.43:0/3229021968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c5c0cd680 0x7f5c5c0ce780 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f5c540097a0 tx=0x7f5c5402a180 comp rx=0 tx=0).stop 2024-04-25T09:00:39.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.661+0000 7f5c6389e640 1 -- 172.21.15.43:0/3229021968 shutdown_connections 2024-04-25T09:00:39.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.661+0000 7f5c6389e640 1 --2- 172.21.15.43:0/3229021968 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c5c0cd680 0x7f5c5c0ce780 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.661+0000 7f5c6389e640 1 --2- 172.21.15.43:0/3229021968 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c5c06c510 0x7f5c5c06c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.661+0000 7f5c6389e640 1 -- 172.21.15.43:0/3229021968 >> 172.21.15.43:0/3229021968 conn(0x7f5c5c0d17f0 msgr2=0x7f5c5c0d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:39.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.661+0000 7f5c6389e640 1 -- 172.21.15.43:0/3229021968 shutdown_connections 2024-04-25T09:00:39.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.661+0000 7f5c6389e640 1 -- 172.21.15.43:0/3229021968 wait complete. 2024-04-25T09:00:39.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.663+0000 7f5c6389e640 1 Processor -- start 2024-04-25T09:00:39.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.663+0000 7f5c6389e640 1 -- start start 2024-04-25T09:00:39.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.663+0000 7f5c6389e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c5c06c510 0x7f5c5c14f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.663+0000 7f5c6389e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c5c0cd680 0x7f5c5c14cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.663+0000 7f5c6389e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c5c15df70 0x7f5c5c14c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.663+0000 7f5c6389e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5c0d0c10 con 0x7f5c5c06c510 2024-04-25T09:00:39.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.663+0000 7f5c6389e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5c0cc150 con 0x7f5c5c0cd680 2024-04-25T09:00:39.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.663+0000 7f5c6389e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5c0cecc0 con 0x7f5c5c15df70 2024-04-25T09:00:39.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.663+0000 7f5c6163a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c5c06c510 0x7f5c5c14f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.663+0000 7f5c6163a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c5c06c510 0x7f5c5c14f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43522/0 (socket says 172.21.15.43:43522) 2024-04-25T09:00:39.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.663+0000 7f5c6163a640 1 -- 172.21.15.43:0/3136695569 learned_addr learned my addr 172.21.15.43:0/3136695569 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:39.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.664+0000 7f5c60e39640 1 --2- 172.21.15.43:0/3136695569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c5c0cd680 0x7f5c5c14cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.664+0000 7f5c61e3b640 1 --2- 172.21.15.43:0/3136695569 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c5c15df70 0x7f5c5c14c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.664+0000 7f5c6163a640 1 -- 172.21.15.43:0/3136695569 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c5c15df70 msgr2=0x7f5c5c14c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.664+0000 7f5c6163a640 1 --2- 172.21.15.43:0/3136695569 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c5c15df70 0x7f5c5c14c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.664+0000 7f5c6163a640 1 -- 172.21.15.43:0/3136695569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c5c0cd680 msgr2=0x7f5c5c14cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.664+0000 7f5c6163a640 1 --2- 172.21.15.43:0/3136695569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c5c0cd680 0x7f5c5c14cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.664+0000 7f5c6163a640 1 -- 172.21.15.43:0/3136695569 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c5c14ff10 con 0x7f5c5c06c510 2024-04-25T09:00:39.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.664+0000 7f5c61e3b640 1 --2- 172.21.15.43:0/3136695569 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c5c15df70 0x7f5c5c14c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:39.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.664+0000 7f5c6163a640 1 --2- 172.21.15.43:0/3136695569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c5c06c510 0x7f5c5c14f9d0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c00efc0 tx=0x7f5c4c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:39.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.664+0000 7f5c59ffb640 1 -- 172.21.15.43:0/3136695569 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c018020 con 0x7f5c5c06c510 2024-04-25T09:00:39.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.664+0000 7f5c59ffb640 1 -- 172.21.15.43:0/3136695569 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c4c014480 con 0x7f5c5c06c510 2024-04-25T09:00:39.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.664+0000 7f5c59ffb640 1 -- 172.21.15.43:0/3136695569 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c014850 con 0x7f5c5c06c510 2024-04-25T09:00:39.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.664+0000 7f5c6389e640 1 -- 172.21.15.43:0/3136695569 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c5c0c7cf0 con 0x7f5c5c06c510 2024-04-25T09:00:39.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.664+0000 7f5c6389e640 1 -- 172.21.15.43:0/3136695569 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c5c0b5ed0 con 0x7f5c5c06c510 2024-04-25T09:00:39.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.665+0000 7f5c6389e640 1 -- 172.21.15.43:0/3136695569 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c5c067f10 con 0x7f5c5c06c510 2024-04-25T09:00:39.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.666+0000 7f5c59ffb640 1 -- 172.21.15.43:0/3136695569 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5c4c005d10 con 0x7f5c5c06c510 2024-04-25T09:00:39.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.666+0000 7f5c59ffb640 1 --2- 172.21.15.43:0/3136695569 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5c4806b4d0 0x7f5c4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.666+0000 7f5c59ffb640 1 -- 172.21.15.43:0/3136695569 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5c4c08a920 con 0x7f5c5c06c510 2024-04-25T09:00:39.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.666+0000 7f5c60e39640 1 --2- 172.21.15.43:0/3136695569 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5c4806b4d0 0x7f5c4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.667+0000 7f5c60e39640 1 --2- 172.21.15.43:0/3136695569 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5c4806b4d0 0x7f5c4806d990 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c0cb8c0 tx=0x7f5c50009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:39.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.670+0000 7f5c59ffb640 1 -- 172.21.15.43:0/3136695569 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5c5c067f10 con 0x7f5c5c06c510 2024-04-25T09:00:39.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.852+0000 7f5c6389e640 1 -- 172.21.15.43:0/3136695569 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c5c06d100 con 0x7f5c5c06c510 2024-04-25T09:00:39.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.853+0000 7f5c59ffb640 1 -- 172.21.15.43:0/3136695569 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+12856 (secure 0 0 0) 0x7f5c5c06d100 con 0x7f5c5c06c510 2024-04-25T09:00:39.842 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:00:39.842 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":85,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:00:39.150354+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":74,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:00:39.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.855+0000 7f5c6389e640 1 -- 172.21.15.43:0/3136695569 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5c4806b4d0 msgr2=0x7f5c4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.855+0000 7f5c6389e640 1 --2- 172.21.15.43:0/3136695569 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5c4806b4d0 0x7f5c4806d990 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c0cb8c0 tx=0x7f5c50009040 comp rx=0 tx=0).stop 2024-04-25T09:00:39.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.855+0000 7f5c6389e640 1 -- 172.21.15.43:0/3136695569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c5c06c510 msgr2=0x7f5c5c14f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.855+0000 7f5c6389e640 1 --2- 172.21.15.43:0/3136695569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c5c06c510 0x7f5c5c14f9d0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c00efc0 tx=0x7f5c4c00c490 comp rx=0 tx=0).stop 2024-04-25T09:00:39.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.855+0000 7f5c6389e640 1 -- 172.21.15.43:0/3136695569 shutdown_connections 2024-04-25T09:00:39.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.855+0000 7f5c6389e640 1 --2- 172.21.15.43:0/3136695569 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c5c15df70 0x7f5c5c14c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.855+0000 7f5c6389e640 1 --2- 172.21.15.43:0/3136695569 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5c4806b4d0 0x7f5c4806d990 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.855+0000 7f5c6389e640 1 --2- 172.21.15.43:0/3136695569 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c5c0cd680 0x7f5c5c14cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.855+0000 7f5c6389e640 1 --2- 172.21.15.43:0/3136695569 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c5c06c510 0x7f5c5c14f9d0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.855+0000 7f5c6389e640 1 -- 172.21.15.43:0/3136695569 >> 172.21.15.43:0/3136695569 conn(0x7f5c5c0d17f0 msgr2=0x7f5c5c0b2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:39.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.855+0000 7f5c6389e640 1 -- 172.21.15.43:0/3136695569 shutdown_connections 2024-04-25T09:00:39.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.855+0000 7f5c6389e640 1 -- 172.21.15.43:0/3136695569 wait complete. 2024-04-25T09:00:39.855 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:00:39.855 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:00:39.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.970+0000 7fcd18b5c640 1 Processor -- start 2024-04-25T09:00:39.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.970+0000 7fcd18b5c640 1 -- start start 2024-04-25T09:00:39.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.971+0000 7fcd18b5c640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fcd140ad010 0x7fcd141618d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.971+0000 7fcd18b5c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcd14146f70 0x7fcd1406eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.971+0000 7fcd18b5c640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd141621d0 con 0x7fcd140ad3f0 2024-04-25T09:00:39.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.971+0000 7fcd18b5c640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd14070ef0 con 0x7fcd140ad010 2024-04-25T09:00:39.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.971+0000 7fcd18b5c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd141624b0 con 0x7fcd14146f70 2024-04-25T09:00:39.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.971+0000 7fcd11d9b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcd14146f70 0x7fcd1406eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.971+0000 7fcd1259c640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fcd140ad010 0x7fcd141618d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.971+0000 7fcd12d9d640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7fcd140ad3f0 0x7fcd14071380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:33532/0 (socket says 172.21.15.43:33532) 2024-04-25T09:00:39.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.971+0000 7fcd12d9d640 1 -- 172.21.15.43:0/4017390948 learned_addr learned my addr 172.21.15.43:0/4017390948 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:00:39.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.971+0000 7fcd1259c640 1 -- 172.21.15.43:0/4017390948 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcd14146f70 msgr2=0x7fcd1406eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:39.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.971+0000 7fcd1259c640 1 --2- 172.21.15.43:0/4017390948 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcd14146f70 0x7fcd1406eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.972+0000 7fcd1259c640 1 -- 172.21.15.43:0/4017390948 >> v1:172.21.15.193:6789/0 conn(0x7fcd140ad3f0 legacy=0x7fcd14071380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.972+0000 7fcd1159a640 1 -- 172.21.15.43:0/4017390948 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3285178365 0 0) 0x7fcd141621d0 con 0x7fcd140ad3f0 2024-04-25T09:00:39.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.972+0000 7fcd1259c640 1 -- 172.21.15.43:0/4017390948 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd14162790 con 0x7fcd140ad010 2024-04-25T09:00:39.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.972+0000 7fcd11d9b640 1 --2- 172.21.15.43:0/4017390948 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcd14146f70 0x7fcd1406eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:00:39.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.972+0000 7fcd1259c640 1 --2- 172.21.15.43:0/4017390948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd140ad010 0x7fcd141618d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fcd040097a0 tx=0x7fcd0402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9e74a6bb7f3e52a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:39.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.973+0000 7fcd1159a640 1 -- 172.21.15.43:0/4017390948 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd04037070 con 0x7fcd140ad010 2024-04-25T09:00:39.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.973+0000 7fcd1159a640 1 -- 172.21.15.43:0/4017390948 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd04032440 con 0x7fcd140ad010 2024-04-25T09:00:39.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.973+0000 7fcd1159a640 1 -- 172.21.15.43:0/4017390948 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd04032810 con 0x7fcd140ad010 2024-04-25T09:00:39.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.973+0000 7fcd18b5c640 1 -- 172.21.15.43:0/4017390948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd140ad010 msgr2=0x7fcd141618d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.973+0000 7fcd18b5c640 1 --2- 172.21.15.43:0/4017390948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd140ad010 0x7fcd141618d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fcd040097a0 tx=0x7fcd0402a180 comp rx=0 tx=0).stop 2024-04-25T09:00:39.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.973+0000 7fcd18b5c640 1 -- 172.21.15.43:0/4017390948 shutdown_connections 2024-04-25T09:00:39.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.973+0000 7fcd18b5c640 1 --2- 172.21.15.43:0/4017390948 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcd14146f70 0x7fcd1406eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.973+0000 7fcd18b5c640 1 --2- 172.21.15.43:0/4017390948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd140ad010 0x7fcd141618d0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.973+0000 7fcd18b5c640 1 -- 172.21.15.43:0/4017390948 >> 172.21.15.43:0/4017390948 conn(0x7fcd140d0260 msgr2=0x7fcd140d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:39.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.973+0000 7fcd18b5c640 1 -- 172.21.15.43:0/4017390948 shutdown_connections 2024-04-25T09:00:39.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.973+0000 7fcd18b5c640 1 -- 172.21.15.43:0/4017390948 wait complete. 2024-04-25T09:00:39.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.974+0000 7fcd18b5c640 1 Processor -- start 2024-04-25T09:00:39.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.975+0000 7fcd18b5c640 1 -- start start 2024-04-25T09:00:39.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.975+0000 7fcd18b5c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd140ad010 0x7fcd141409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.975+0000 7fcd18b5c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd140ad3f0 0x7fcd141f4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.975+0000 7fcd18b5c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcd14146f70 0x7fcd1409b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.975+0000 7fcd18b5c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd14161e10 con 0x7fcd140ad010 2024-04-25T09:00:39.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.975+0000 7fcd18b5c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd14141450 con 0x7fcd140ad3f0 2024-04-25T09:00:39.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.975+0000 7fcd18b5c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd14161f90 con 0x7fcd14146f70 2024-04-25T09:00:39.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.975+0000 7fcd1259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd140ad010 0x7fcd141409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.975+0000 7fcd1259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd140ad010 0x7fcd141409d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43524/0 (socket says 172.21.15.43:43524) 2024-04-25T09:00:39.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.975+0000 7fcd1259c640 1 -- 172.21.15.43:0/3488648800 learned_addr learned my addr 172.21.15.43:0/3488648800 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:39.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.976+0000 7fcd12d9d640 1 --2- 172.21.15.43:0/3488648800 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcd14146f70 0x7fcd1409b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.976+0000 7fcd11d9b640 1 --2- 172.21.15.43:0/3488648800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd140ad3f0 0x7fcd141f4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.976+0000 7fcd1259c640 1 -- 172.21.15.43:0/3488648800 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcd14146f70 msgr2=0x7fcd1409b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.976+0000 7fcd1259c640 1 --2- 172.21.15.43:0/3488648800 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcd14146f70 0x7fcd1409b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.976+0000 7fcd1259c640 1 -- 172.21.15.43:0/3488648800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd140ad3f0 msgr2=0x7fcd141f4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:39.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.976+0000 7fcd1259c640 1 --2- 172.21.15.43:0/3488648800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd140ad3f0 0x7fcd141f4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:39.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.976+0000 7fcd1259c640 1 -- 172.21.15.43:0/3488648800 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd1409c730 con 0x7fcd140ad010 2024-04-25T09:00:39.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.976+0000 7fcd11d9b640 1 --2- 172.21.15.43:0/3488648800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd140ad3f0 0x7fcd141f4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:39.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.976+0000 7fcd12d9d640 1 --2- 172.21.15.43:0/3488648800 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcd14146f70 0x7fcd1409b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:39.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.976+0000 7fcd1259c640 1 --2- 172.21.15.43:0/3488648800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd140ad010 0x7fcd141409d0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fcd040097a0 tx=0x7fcd040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:39.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.976+0000 7fcd02ffd640 1 -- 172.21.15.43:0/3488648800 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd04037040 con 0x7fcd140ad010 2024-04-25T09:00:39.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.976+0000 7fcd02ffd640 1 -- 172.21.15.43:0/3488648800 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd04004230 con 0x7fcd140ad010 2024-04-25T09:00:39.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.976+0000 7fcd18b5c640 1 -- 172.21.15.43:0/3488648800 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd1409ca10 con 0x7fcd140ad010 2024-04-25T09:00:39.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.976+0000 7fcd02ffd640 1 -- 172.21.15.43:0/3488648800 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd04003610 con 0x7fcd140ad010 2024-04-25T09:00:39.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.976+0000 7fcd18b5c640 1 -- 172.21.15.43:0/3488648800 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd1409cfd0 con 0x7fcd140ad010 2024-04-25T09:00:39.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.977+0000 7fcd18b5c640 1 -- 172.21.15.43:0/3488648800 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd14074f60 con 0x7fcd140ad010 2024-04-25T09:00:39.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.978+0000 7fcd02ffd640 1 -- 172.21.15.43:0/3488648800 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcd040037b0 con 0x7fcd140ad010 2024-04-25T09:00:39.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.978+0000 7fcd02ffd640 1 --2- 172.21.15.43:0/3488648800 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fccf406b4d0 0x7fccf406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:39.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.978+0000 7fcd02ffd640 1 -- 172.21.15.43:0/3488648800 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcd040aaf40 con 0x7fcd140ad010 2024-04-25T09:00:39.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.978+0000 7fcd11d9b640 1 --2- 172.21.15.43:0/3488648800 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fccf406b4d0 0x7fccf406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:39.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.979+0000 7fcd11d9b640 1 --2- 172.21.15.43:0/3488648800 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fccf406b4d0 0x7fccf406d990 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fcd14144180 tx=0x7fcd08008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:39.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:39.982+0000 7fcd02ffd640 1 -- 172.21.15.43:0/3488648800 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcd0407a2c0 con 0x7fcd140ad010 2024-04-25T09:00:40.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.162+0000 7fcd18b5c640 1 -- 172.21.15.43:0/3488648800 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcd140b3690 con 0x7fcd140ad010 2024-04-25T09:00:40.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.162+0000 7fcd02ffd640 1 -- 172.21.15.43:0/3488648800 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcd04079bc0 con 0x7fcd140ad010 2024-04-25T09:00:40.151 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T09:00:40.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.164+0000 7fcd18b5c640 1 -- 172.21.15.43:0/3488648800 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fccf406b4d0 msgr2=0x7fccf406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:40.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.164+0000 7fcd18b5c640 1 --2- 172.21.15.43:0/3488648800 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fccf406b4d0 0x7fccf406d990 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fcd14144180 tx=0x7fcd08008040 comp rx=0 tx=0).stop 2024-04-25T09:00:40.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.164+0000 7fcd18b5c640 1 -- 172.21.15.43:0/3488648800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd140ad010 msgr2=0x7fcd141409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:40.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.164+0000 7fcd18b5c640 1 --2- 172.21.15.43:0/3488648800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd140ad010 0x7fcd141409d0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fcd040097a0 tx=0x7fcd040046c0 comp rx=0 tx=0).stop 2024-04-25T09:00:40.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.164+0000 7fcd18b5c640 1 -- 172.21.15.43:0/3488648800 shutdown_connections 2024-04-25T09:00:40.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.164+0000 7fcd18b5c640 1 --2- 172.21.15.43:0/3488648800 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcd14146f70 0x7fcd1409b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:40.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.164+0000 7fcd18b5c640 1 --2- 172.21.15.43:0/3488648800 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fccf406b4d0 0x7fccf406d990 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:40.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.164+0000 7fcd18b5c640 1 --2- 172.21.15.43:0/3488648800 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd140ad3f0 0x7fcd141f4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:40.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.164+0000 7fcd18b5c640 1 --2- 172.21.15.43:0/3488648800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd140ad010 0x7fcd141409d0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:40.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.164+0000 7fcd18b5c640 1 -- 172.21.15.43:0/3488648800 >> 172.21.15.43:0/3488648800 conn(0x7fcd140d0260 msgr2=0x7fcd140b3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:40.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.164+0000 7fcd18b5c640 1 -- 172.21.15.43:0/3488648800 shutdown_connections 2024-04-25T09:00:40.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.164+0000 7fcd18b5c640 1 -- 172.21.15.43:0/3488648800 wait complete. 2024-04-25T09:00:40.164 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:00:40.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.276+0000 7f9a1bc38640 1 Processor -- start 2024-04-25T09:00:40.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.276+0000 7f9a1bc38640 1 -- start start 2024-04-25T09:00:40.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.277+0000 7f9a1bc38640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9a140d0af0 0x7f9a14160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:40.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.277+0000 7f9a1bc38640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a140d2920 con 0x7f9a1415dce0 2024-04-25T09:00:40.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.277+0000 7f9a1bc38640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a140cee80 con 0x7f9a1415ef50 2024-04-25T09:00:40.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.277+0000 7f9a1bc38640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a140cec60 con 0x7f9a140d0af0 2024-04-25T09:00:40.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.277+0000 7f9a199d4640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f9a1415dce0 0x7f9a140ce550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44244/0 (socket says 172.21.15.43:44244) 2024-04-25T09:00:40.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.277+0000 7f9a199d4640 1 -- 172.21.15.43:0/2103641893 learned_addr learned my addr 172.21.15.43:0/2103641893 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:00:40.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.277+0000 7f9a1a1d5640 1 --2- 172.21.15.43:0/2103641893 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9a140d0af0 0x7f9a14160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:40.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.277+0000 7f9a189d2640 1 -- 172.21.15.43:0/2103641893 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3619915051 0 0) 0x7f9a140d2920 con 0x7f9a1415dce0 2024-04-25T09:00:40.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.277+0000 7f9a189d2640 1 -- 172.21.15.43:0/2103641893 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a00003520 con 0x7f9a1415dce0 2024-04-25T09:00:40.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.278+0000 7f9a189d2640 1 -- 172.21.15.43:0/2103641893 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4013473201 0 0) 0x7f9a00003520 con 0x7f9a1415dce0 2024-04-25T09:00:40.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.278+0000 7f9a189d2640 1 -- 172.21.15.43:0/2103641893 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9a140d0af0 msgr2=0x7f9a14160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:40.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.278+0000 7f9a189d2640 1 --2- 172.21.15.43:0/2103641893 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9a140d0af0 0x7f9a14160980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:40.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.278+0000 7f9a189d2640 1 -- 172.21.15.43:0/2103641893 >> v1:172.21.15.193:6789/0 conn(0x7f9a1415ef50 legacy=0x7f9a140d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:40.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.278+0000 7f9a189d2640 1 -- 172.21.15.43:0/2103641893 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a140cf160 con 0x7f9a1415dce0 2024-04-25T09:00:40.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.278+0000 7f9a189d2640 1 -- 172.21.15.43:0/2103641893 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f9a10001690 con 0x7f9a1415dce0 2024-04-25T09:00:40.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.278+0000 7f9a189d2640 1 -- 172.21.15.43:0/2103641893 >> v1:172.21.15.43:6789/0 conn(0x7f9a1415dce0 legacy=0x7f9a140ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:40.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.278+0000 7f9a189d2640 1 --2- 172.21.15.43:0/2103641893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a000039e0 0x7f9a00003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:40.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.278+0000 7f9a189d2640 1 --2- 172.21.15.43:0/2103641893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a00004800 0x7f9a00004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:40.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.278+0000 7f9a189d2640 1 --2- 172.21.15.43:0/2103641893 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9a140d0af0 0x7f9a00005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:40.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.278+0000 7f9a189d2640 1 -- 172.21.15.43:0/2103641893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a140cec60 con 0x7f9a000039e0 2024-04-25T09:00:40.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.278+0000 7f9a189d2640 1 -- 172.21.15.43:0/2103641893 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0000bb20 con 0x7f9a00004800 2024-04-25T09:00:40.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.278+0000 7f9a189d2640 1 -- 172.21.15.43:0/2103641893 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0000bd20 con 0x7f9a140d0af0 2024-04-25T09:00:40.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.278+0000 7f9a199d4640 1 --2- 172.21.15.43:0/2103641893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a000039e0 0x7f9a00003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:40.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.278+0000 7f9a191d3640 1 --2- 172.21.15.43:0/2103641893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a00004800 0x7f9a00004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:40.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.278+0000 7f9a1a1d5640 1 --2- 172.21.15.43:0/2103641893 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9a140d0af0 0x7f9a00005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:40.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.278+0000 7f9a199d4640 1 -- 172.21.15.43:0/2103641893 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9a140d0af0 msgr2=0x7f9a00005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:40.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.278+0000 7f9a199d4640 1 --2- 172.21.15.43:0/2103641893 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9a140d0af0 0x7f9a00005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:40.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.278+0000 7f9a199d4640 1 -- 172.21.15.43:0/2103641893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a00004800 msgr2=0x7f9a00004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:40.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.278+0000 7f9a199d4640 1 --2- 172.21.15.43:0/2103641893 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a00004800 0x7f9a00004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:40.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.278+0000 7f9a199d4640 1 -- 172.21.15.43:0/2103641893 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a00003520 con 0x7f9a000039e0 2024-04-25T09:00:40.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.279+0000 7f9a199d4640 1 --2- 172.21.15.43:0/2103641893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a000039e0 0x7f9a00003e00 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f9a1000c210 tx=0x7f9a00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=322c62bb377dd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:40.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.279+0000 7f9a189d2640 1 -- 172.21.15.43:0/2103641893 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a10007d30 con 0x7f9a000039e0 2024-04-25T09:00:40.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.279+0000 7f9a189d2640 1 -- 172.21.15.43:0/2103641893 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a100058d0 con 0x7f9a000039e0 2024-04-25T09:00:40.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.279+0000 7f9a189d2640 1 -- 172.21.15.43:0/2103641893 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a10005ca0 con 0x7f9a000039e0 2024-04-25T09:00:40.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.280+0000 7f9a1bc38640 1 -- 172.21.15.43:0/2103641893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a000039e0 msgr2=0x7f9a00003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:40.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.280+0000 7f9a1bc38640 1 --2- 172.21.15.43:0/2103641893 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a000039e0 0x7f9a00003e00 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f9a1000c210 tx=0x7f9a00005830 comp rx=0 tx=0).stop 2024-04-25T09:00:40.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.280+0000 7f9a1a1d5640 1 -- 172.21.15.43:0/2103641893 reap_dead start 2024-04-25T09:00:40.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.280+0000 7f9a1bc38640 1 -- 172.21.15.43:0/2103641893 shutdown_connections 2024-04-25T09:00:40.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.280+0000 7f9a1bc38640 1 -- 172.21.15.43:0/2103641893 >> 172.21.15.43:0/2103641893 conn(0x7f9a1409cd00 msgr2=0x7f9a140d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:40.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.280+0000 7f9a1bc38640 1 -- 172.21.15.43:0/2103641893 shutdown_connections 2024-04-25T09:00:40.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.280+0000 7f9a1bc38640 1 -- 172.21.15.43:0/2103641893 wait complete. 2024-04-25T09:00:40.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.281+0000 7f9a1bc38640 1 Processor -- start 2024-04-25T09:00:40.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.281+0000 7f9a1bc38640 1 -- start start 2024-04-25T09:00:40.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.282+0000 7f9a1bc38640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a000039e0 0x7f9a140b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:40.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.282+0000 7f9a1bc38640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9a140cd680 0x7f9a140cac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:40.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.282+0000 7f9a1bc38640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a141e64d0 0x7f9a141e6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:40.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.282+0000 7f9a1bc38640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1415f6b0 con 0x7f9a000039e0 2024-04-25T09:00:40.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.282+0000 7f9a1bc38640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a140cf720 con 0x7f9a141e64d0 2024-04-25T09:00:40.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.282+0000 7f9a1bc38640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1415e0c0 con 0x7f9a140cd680 2024-04-25T09:00:40.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.282+0000 7f9a199d4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a000039e0 0x7f9a140b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:40.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.282+0000 7f9a199d4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a000039e0 0x7f9a140b8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43540/0 (socket says 172.21.15.43:43540) 2024-04-25T09:00:40.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.282+0000 7f9a199d4640 1 -- 172.21.15.43:0/19659755 learned_addr learned my addr 172.21.15.43:0/19659755 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.282+0000 7f9a1a1d5640 1 --2- 172.21.15.43:0/19659755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a141e64d0 0x7f9a141e6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.282+0000 7f9a191d3640 1 --2- 172.21.15.43:0/19659755 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9a140cd680 0x7f9a140cac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.282+0000 7f9a199d4640 1 -- 172.21.15.43:0/19659755 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9a140cd680 msgr2=0x7f9a140cac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.282+0000 7f9a199d4640 1 --2- 172.21.15.43:0/19659755 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9a140cd680 0x7f9a140cac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.282+0000 7f9a199d4640 1 -- 172.21.15.43:0/19659755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a141e64d0 msgr2=0x7f9a141e6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.282+0000 7f9a199d4640 1 --2- 172.21.15.43:0/19659755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a141e64d0 0x7f9a141e6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.282+0000 7f9a199d4640 1 -- 172.21.15.43:0/19659755 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a141e6ea0 con 0x7f9a000039e0 2024-04-25T09:00:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.282+0000 7f9a1a1d5640 1 --2- 172.21.15.43:0/19659755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a141e64d0 0x7f9a141e6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:00:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.282+0000 7f9a199d4640 1 --2- 172.21.15.43:0/19659755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a000039e0 0x7f9a140b8b30 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f9a1002adf0 tx=0x7f9a140cfa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.282+0000 7f9a0a7fc640 1 -- 172.21.15.43:0/19659755 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a10042020 con 0x7f9a000039e0 2024-04-25T09:00:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.282+0000 7f9a0a7fc640 1 -- 172.21.15.43:0/19659755 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a10000ed0 con 0x7f9a000039e0 2024-04-25T09:00:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.282+0000 7f9a1bc38640 1 -- 172.21.15.43:0/19659755 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a141e5710 con 0x7f9a000039e0 2024-04-25T09:00:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.282+0000 7f9a0a7fc640 1 -- 172.21.15.43:0/19659755 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a10007d30 con 0x7f9a000039e0 2024-04-25T09:00:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.283+0000 7f9a1bc38640 1 -- 172.21.15.43:0/19659755 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a141e5cd0 con 0x7f9a000039e0 2024-04-25T09:00:40.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.283+0000 7f9a1bc38640 1 -- 172.21.15.43:0/19659755 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a140680f0 con 0x7f9a000039e0 2024-04-25T09:00:40.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.284+0000 7f9a0a7fc640 1 -- 172.21.15.43:0/19659755 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9a10045020 con 0x7f9a000039e0 2024-04-25T09:00:40.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.284+0000 7f9a0a7fc640 1 --2- 172.21.15.43:0/19659755 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f99f806b4d0 0x7f99f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:40.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.284+0000 7f9a0a7fc640 1 -- 172.21.15.43:0/19659755 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9a100aa2d0 con 0x7f9a000039e0 2024-04-25T09:00:40.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.284+0000 7f9a191d3640 1 --2- 172.21.15.43:0/19659755 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f99f806b4d0 0x7f99f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:40.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.285+0000 7f9a191d3640 1 --2- 172.21.15.43:0/19659755 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f99f806b4d0 0x7f99f806d990 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9a140cdaa0 tx=0x7f9a04009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:40.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.288+0000 7f9a0a7fc640 1 -- 172.21.15.43:0/19659755 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9a1004c030 con 0x7f9a000039e0 2024-04-25T09:00:40.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.471+0000 7f9a1bc38640 1 -- 172.21.15.43:0/19659755 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9a14074ef0 con 0x7f9a000039e0 2024-04-25T09:00:40.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.471+0000 7f9a0a7fc640 1 -- 172.21.15.43:0/19659755 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v85) v1 ==== 84+0+65 (secure 0 0 0) 0x7f9a10079490 con 0x7f9a000039e0 2024-04-25T09:00:40.460 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:00:40.460 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2024-04-25T09:00:40.461 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:00:40.461 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 1, 2024-04-25T09:00:40.461 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": ".mgr" 2024-04-25T09:00:40.461 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:00:40.461 INFO:teuthology.orchestra.run.smithi043.stdout:] 2024-04-25T09:00:40.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.473+0000 7f9a1bc38640 1 -- 172.21.15.43:0/19659755 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f99f806b4d0 msgr2=0x7f99f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:40.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.473+0000 7f9a1bc38640 1 --2- 172.21.15.43:0/19659755 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f99f806b4d0 0x7f99f806d990 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9a140cdaa0 tx=0x7f9a04009040 comp rx=0 tx=0).stop 2024-04-25T09:00:40.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.473+0000 7f9a1bc38640 1 -- 172.21.15.43:0/19659755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a000039e0 msgr2=0x7f9a140b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:40.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.473+0000 7f9a1bc38640 1 --2- 172.21.15.43:0/19659755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a000039e0 0x7f9a140b8b30 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f9a1002adf0 tx=0x7f9a140cfa00 comp rx=0 tx=0).stop 2024-04-25T09:00:40.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.473+0000 7f9a1bc38640 1 -- 172.21.15.43:0/19659755 shutdown_connections 2024-04-25T09:00:40.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.473+0000 7f9a1bc38640 1 --2- 172.21.15.43:0/19659755 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f99f806b4d0 0x7f99f806d990 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:40.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.473+0000 7f9a1bc38640 1 --2- 172.21.15.43:0/19659755 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9a141e64d0 0x7f9a141e6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:40.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.473+0000 7f9a1bc38640 1 --2- 172.21.15.43:0/19659755 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9a140cd680 0x7f9a140cac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:40.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.473+0000 7f9a1bc38640 1 --2- 172.21.15.43:0/19659755 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9a000039e0 0x7f9a140b8b30 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:40.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.473+0000 7f9a1bc38640 1 -- 172.21.15.43:0/19659755 >> 172.21.15.43:0/19659755 conn(0x7f9a1409cd00 msgr2=0x7f9a1415e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:40.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.473+0000 7f9a1bc38640 1 -- 172.21.15.43:0/19659755 shutdown_connections 2024-04-25T09:00:40.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.473+0000 7f9a1bc38640 1 -- 172.21.15.43:0/19659755 wait complete. 2024-04-25T09:00:40.473 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:00:40.473 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:00:40.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.589+0000 7f968e1b7640 1 Processor -- start 2024-04-25T09:00:40.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.589+0000 7f968e1b7640 1 -- start start 2024-04-25T09:00:40.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.590+0000 7f968e1b7640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f968815b6c0 0x7f968815baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:40.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.590+0000 7f968e1b7640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f968815e410 0x7f96880b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:40.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.590+0000 7f968e1b7640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96880b37a0 con 0x7f968815f5d0 2024-04-25T09:00:40.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.590+0000 7f968e1b7640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f968815e870 con 0x7f968815b6c0 2024-04-25T09:00:40.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.590+0000 7f968e1b7640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96880b53e0 con 0x7f968815e410 2024-04-25T09:00:40.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.590+0000 7f96877fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f968815b6c0 0x7f968815baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:40.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.590+0000 7f9687fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f968815f5d0 0x7f96880d3760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44258/0 (socket says 172.21.15.43:44258) 2024-04-25T09:00:40.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.590+0000 7f9687fff640 1 -- 172.21.15.43:0/2860108782 learned_addr learned my addr 172.21.15.43:0/2860108782 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:00:40.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.590+0000 7f96877fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f968815b6c0 0x7f968815baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43554/0 (socket says 172.21.15.43:43554) 2024-04-25T09:00:40.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.590+0000 7f9686ffd640 1 --2- 172.21.15.43:0/2860108782 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f968815e410 0x7f96880b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:40.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.591+0000 7f96877fe640 1 -- 172.21.15.43:0/2860108782 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f968815e410 msgr2=0x7f96880b4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:40.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.591+0000 7f96877fe640 1 --2- 172.21.15.43:0/2860108782 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f968815e410 0x7f96880b4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:40.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.591+0000 7f96877fe640 1 -- 172.21.15.43:0/2860108782 >> v1:172.21.15.43:6789/0 conn(0x7f968815f5d0 legacy=0x7f96880d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:40.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.591+0000 7f96877fe640 1 -- 172.21.15.43:0/2860108782 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f968815eab0 con 0x7f968815b6c0 2024-04-25T09:00:40.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.591+0000 7f96867fc640 1 -- 172.21.15.43:0/2860108782 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1983534302 0 0) 0x7f96880b37a0 con 0x7f968815f5d0 2024-04-25T09:00:40.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.591+0000 7f96877fe640 1 --2- 172.21.15.43:0/2860108782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f968815b6c0 0x7f968815baa0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f96780097a0 tx=0x7f967802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e36c68aa36f9a5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:40.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.591+0000 7f96867fc640 1 -- 172.21.15.43:0/2860108782 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9678037070 con 0x7f968815b6c0 2024-04-25T09:00:40.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.591+0000 7f96867fc640 1 -- 172.21.15.43:0/2860108782 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9678032440 con 0x7f968815b6c0 2024-04-25T09:00:40.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.592+0000 7f96867fc640 1 -- 172.21.15.43:0/2860108782 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9678032810 con 0x7f968815b6c0 2024-04-25T09:00:40.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.592+0000 7f968e1b7640 1 -- 172.21.15.43:0/2860108782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f968815b6c0 msgr2=0x7f968815baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:40.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.592+0000 7f968e1b7640 1 --2- 172.21.15.43:0/2860108782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f968815b6c0 0x7f968815baa0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f96780097a0 tx=0x7f967802a180 comp rx=0 tx=0).stop 2024-04-25T09:00:40.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.592+0000 7f968e1b7640 1 -- 172.21.15.43:0/2860108782 shutdown_connections 2024-04-25T09:00:40.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.592+0000 7f968e1b7640 1 --2- 172.21.15.43:0/2860108782 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f968815e410 0x7f96880b4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:40.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.592+0000 7f968e1b7640 1 --2- 172.21.15.43:0/2860108782 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f968815b6c0 0x7f968815baa0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:40.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.592+0000 7f968e1b7640 1 -- 172.21.15.43:0/2860108782 >> 172.21.15.43:0/2860108782 conn(0x7f968808d460 msgr2=0x7f968809cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:40.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.592+0000 7f968e1b7640 1 -- 172.21.15.43:0/2860108782 shutdown_connections 2024-04-25T09:00:40.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.592+0000 7f968e1b7640 1 -- 172.21.15.43:0/2860108782 wait complete. 2024-04-25T09:00:40.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.593+0000 7f968e1b7640 1 Processor -- start 2024-04-25T09:00:40.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.593+0000 7f968e1b7640 1 -- start start 2024-04-25T09:00:40.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.594+0000 7f968e1b7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f968815b6c0 0x7f968815d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:40.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.594+0000 7f968e1b7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f968815e410 0x7f968814aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:40.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.594+0000 7f968e1b7640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f968815f5d0 0x7f968814afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:40.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.594+0000 7f968e1b7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968815fa40 con 0x7f968815e410 2024-04-25T09:00:40.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.594+0000 7f968e1b7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968815f350 con 0x7f968815b6c0 2024-04-25T09:00:40.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.594+0000 7f968e1b7640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f968815fbc0 con 0x7f968815f5d0 2024-04-25T09:00:40.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.594+0000 7f9686ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f968815e410 0x7f968814aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:40.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.594+0000 7f9686ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f968815e410 0x7f968814aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43560/0 (socket says 172.21.15.43:43560) 2024-04-25T09:00:40.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.594+0000 7f9686ffd640 1 -- 172.21.15.43:0/3631634244 learned_addr learned my addr 172.21.15.43:0/3631634244 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:40.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.594+0000 7f9687fff640 1 --2- 172.21.15.43:0/3631634244 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f968815f5d0 0x7f968814afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:40.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.595+0000 7f96877fe640 1 --2- 172.21.15.43:0/3631634244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f968815b6c0 0x7f968815d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:40.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.595+0000 7f9686ffd640 1 -- 172.21.15.43:0/3631634244 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f968815f5d0 msgr2=0x7f968814afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:40.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.595+0000 7f9686ffd640 1 --2- 172.21.15.43:0/3631634244 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f968815f5d0 0x7f968814afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:40.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.595+0000 7f9686ffd640 1 -- 172.21.15.43:0/3631634244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f968815b6c0 msgr2=0x7f968815d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:40.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.595+0000 7f9686ffd640 1 --2- 172.21.15.43:0/3631634244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f968815b6c0 0x7f968815d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:40.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.595+0000 7f9686ffd640 1 -- 172.21.15.43:0/3631634244 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96881f6ed0 con 0x7f968815e410 2024-04-25T09:00:40.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.595+0000 7f9686ffd640 1 --2- 172.21.15.43:0/3631634244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f968815e410 0x7f968814aa70 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f967c00d800 tx=0x7f967c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:40.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.595+0000 7f966ffff640 1 -- 172.21.15.43:0/3631634244 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c00b6c0 con 0x7f968815e410 2024-04-25T09:00:40.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.595+0000 7f966ffff640 1 -- 172.21.15.43:0/3631634244 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f967c019070 con 0x7f968815e410 2024-04-25T09:00:40.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.595+0000 7f966ffff640 1 -- 172.21.15.43:0/3631634244 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967c00ba80 con 0x7f968815e410 2024-04-25T09:00:40.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.595+0000 7f968e1b7640 1 -- 172.21.15.43:0/3631634244 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96881f71b0 con 0x7f968815e410 2024-04-25T09:00:40.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.595+0000 7f968e1b7640 1 -- 172.21.15.43:0/3631634244 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96881f7770 con 0x7f968815e410 2024-04-25T09:00:40.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.596+0000 7f968e1b7640 1 -- 172.21.15.43:0/3631634244 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9688068a40 con 0x7f968815e410 2024-04-25T09:00:40.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.597+0000 7f966ffff640 1 -- 172.21.15.43:0/3631634244 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f967c00bc20 con 0x7f968815e410 2024-04-25T09:00:40.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.597+0000 7f966ffff640 1 --2- 172.21.15.43:0/3631634244 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f967406b500 0x7f967406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:40.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.597+0000 7f966ffff640 1 -- 172.21.15.43:0/3631634244 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f967c08a1b0 con 0x7f968815e410 2024-04-25T09:00:40.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.597+0000 7f96877fe640 1 --2- 172.21.15.43:0/3631634244 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f967406b500 0x7f967406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:40.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.598+0000 7f96877fe640 1 --2- 172.21.15.43:0/3631634244 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f967406b500 0x7f967406d9c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9678034000 tx=0x7f967802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:40.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.602+0000 7f966ffff640 1 -- 172.21.15.43:0/3631634244 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f967c058d30 con 0x7f968815e410 2024-04-25T09:00:40.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:40.784+0000 7f968e1b7640 1 -- 172.21.15.43:0/3631634244 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f9688073f80 con 0x7f968815e410 2024-04-25T09:00:41.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.154+0000 7f966ffff640 1 -- 172.21.15.43:0/3631634244 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v86) v1 ==== 138+0+0 (secure 0 0 0) 0x7f967c05cdd0 con 0x7f968815e410 2024-04-25T09:00:41.143 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' created 2024-04-25T09:00:41.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.157+0000 7f968e1b7640 1 -- 172.21.15.43:0/3631634244 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f967406b500 msgr2=0x7f967406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:41.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.157+0000 7f968e1b7640 1 --2- 172.21.15.43:0/3631634244 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f967406b500 0x7f967406d9c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9678034000 tx=0x7f967802e000 comp rx=0 tx=0).stop 2024-04-25T09:00:41.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.157+0000 7f968e1b7640 1 -- 172.21.15.43:0/3631634244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f968815e410 msgr2=0x7f968814aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:41.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.157+0000 7f968e1b7640 1 --2- 172.21.15.43:0/3631634244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f968815e410 0x7f968814aa70 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f967c00d800 tx=0x7f967c00dcc0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.157+0000 7f968e1b7640 1 -- 172.21.15.43:0/3631634244 shutdown_connections 2024-04-25T09:00:41.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.157+0000 7f968e1b7640 1 --2- 172.21.15.43:0/3631634244 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f968815f5d0 0x7f968814afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.157+0000 7f968e1b7640 1 --2- 172.21.15.43:0/3631634244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f968815e410 0x7f968814aa70 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.157+0000 7f968e1b7640 1 --2- 172.21.15.43:0/3631634244 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f967406b500 0x7f967406d9c0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.157+0000 7f968e1b7640 1 --2- 172.21.15.43:0/3631634244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f968815b6c0 0x7f968815d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.157+0000 7f968e1b7640 1 -- 172.21.15.43:0/3631634244 >> 172.21.15.43:0/3631634244 conn(0x7f968808d460 msgr2=0x7f96880d0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:41.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.157+0000 7f968e1b7640 1 -- 172.21.15.43:0/3631634244 shutdown_connections 2024-04-25T09:00:41.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.157+0000 7f968e1b7640 1 -- 172.21.15.43:0/3631634244 wait complete. 2024-04-25T09:00:41.162 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:00:41.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.281+0000 7f73eb26f640 1 Processor -- start 2024-04-25T09:00:41.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.282+0000 7f73eb26f640 1 -- start start 2024-04-25T09:00:41.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.282+0000 7f73eb26f640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f73e40afd20 0x7f73e40af740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:41.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.282+0000 7f73eb26f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f73e40cb890 0x7f73e40cbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:41.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.282+0000 7f73eb26f640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73e4071a40 con 0x7f73e40cf270 2024-04-25T09:00:41.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.282+0000 7f73eb26f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73e40b0200 con 0x7f73e40cb890 2024-04-25T09:00:41.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.282+0000 7f73eb26f640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73e4072370 con 0x7f73e40afd20 2024-04-25T09:00:41.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.283+0000 7f73e980c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f73e40cb890 0x7f73e40cbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:41.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.283+0000 7f73e980c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f73e40cb890 0x7f73e40cbd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43566/0 (socket says 172.21.15.43:43566) 2024-04-25T09:00:41.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.283+0000 7f73e980c640 1 -- 172.21.15.43:0/2585635375 learned_addr learned my addr 172.21.15.43:0/2585635375 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:41.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.283+0000 7f73e980c640 1 -- 172.21.15.43:0/2585635375 >> v2:172.21.15.193:3300/0 conn(0x7f73e40afd20 msgr2=0x7f73e40af740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:41.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.283+0000 7f73e880a640 1 --2- 172.21.15.43:0/2585635375 >> v2:172.21.15.193:3300/0 conn(0x7f73e40afd20 0x7f73e40af740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:41.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.283+0000 7f73e980c640 1 --2- 172.21.15.43:0/2585635375 >> v2:172.21.15.193:3300/0 conn(0x7f73e40afd20 0x7f73e40af740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.283+0000 7f73e980c640 1 -- 172.21.15.43:0/2585635375 >> v1:172.21.15.193:6789/0 conn(0x7f73e40cf270 legacy=0x7f73e40cf650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:41.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.283+0000 7f73e980c640 1 -- 172.21.15.43:0/2585635375 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e4071d20 con 0x7f73e40cb890 2024-04-25T09:00:41.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.283+0000 7f73e880a640 1 --2- 172.21.15.43:0/2585635375 >> v2:172.21.15.193:3300/0 conn(0x7f73e40afd20 0x7f73e40af740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:41.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.284+0000 7f73e980c640 1 --2- 172.21.15.43:0/2585635375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73e40cb890 0x7f73e40cbd40 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f73e406cc20 tx=0x7f73dc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2824fb010722bdc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:41.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.284+0000 7f73e3fff640 1 -- 172.21.15.43:0/2585635375 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73dc037070 con 0x7f73e40cb890 2024-04-25T09:00:41.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.284+0000 7f73e3fff640 1 -- 172.21.15.43:0/2585635375 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73dc032440 con 0x7f73e40cb890 2024-04-25T09:00:41.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.284+0000 7f73e3fff640 1 -- 172.21.15.43:0/2585635375 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73dc032810 con 0x7f73e40cb890 2024-04-25T09:00:41.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.284+0000 7f73eb26f640 1 -- 172.21.15.43:0/2585635375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73e40cb890 msgr2=0x7f73e40cbd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:41.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.284+0000 7f73eb26f640 1 --2- 172.21.15.43:0/2585635375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73e40cb890 0x7f73e40cbd40 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f73e406cc20 tx=0x7f73dc02a1a0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.284+0000 7f73eb26f640 1 -- 172.21.15.43:0/2585635375 shutdown_connections 2024-04-25T09:00:41.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.284+0000 7f73eb26f640 1 --2- 172.21.15.43:0/2585635375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73e40cb890 0x7f73e40cbd40 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.284+0000 7f73eb26f640 1 --2- 172.21.15.43:0/2585635375 >> v2:172.21.15.193:3300/0 conn(0x7f73e40afd20 0x7f73e40af740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.284+0000 7f73eb26f640 1 -- 172.21.15.43:0/2585635375 >> 172.21.15.43:0/2585635375 conn(0x7f73e40d0730 msgr2=0x7f73e40d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:41.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.284+0000 7f73eb26f640 1 -- 172.21.15.43:0/2585635375 shutdown_connections 2024-04-25T09:00:41.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.284+0000 7f73eb26f640 1 -- 172.21.15.43:0/2585635375 wait complete. 2024-04-25T09:00:41.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.286+0000 7f73eb26f640 1 Processor -- start 2024-04-25T09:00:41.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.286+0000 7f73eb26f640 1 -- start start 2024-04-25T09:00:41.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.286+0000 7f73eb26f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f73e40afd20 0x7f73e4154c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:41.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.286+0000 7f73eb26f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73e40cb890 0x7f73e40b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:41.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.286+0000 7f73eb26f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73e40cf270 0x7f73e40b57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:41.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.286+0000 7f73eb26f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e40d1490 con 0x7f73e40cb890 2024-04-25T09:00:41.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.286+0000 7f73eb26f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e415e0e0 con 0x7f73e40cf270 2024-04-25T09:00:41.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.286+0000 7f73eb26f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e40cc280 con 0x7f73e40afd20 2024-04-25T09:00:41.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.286+0000 7f73e880a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73e40cb890 0x7f73e40b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:41.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.286+0000 7f73e880a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73e40cb890 0x7f73e40b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43582/0 (socket says 172.21.15.43:43582) 2024-04-25T09:00:41.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.286+0000 7f73e880a640 1 -- 172.21.15.43:0/2302357237 learned_addr learned my addr 172.21.15.43:0/2302357237 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:41.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.287+0000 7f73e980c640 1 --2- 172.21.15.43:0/2302357237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73e40cf270 0x7f73e40b57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:41.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.287+0000 7f73e900b640 1 --2- 172.21.15.43:0/2302357237 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f73e40afd20 0x7f73e4154c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:41.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.287+0000 7f73e880a640 1 -- 172.21.15.43:0/2302357237 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f73e40afd20 msgr2=0x7f73e4154c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:41.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.287+0000 7f73e880a640 1 --2- 172.21.15.43:0/2302357237 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f73e40afd20 0x7f73e4154c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.287+0000 7f73e880a640 1 -- 172.21.15.43:0/2302357237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73e40cf270 msgr2=0x7f73e40b57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:41.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.287+0000 7f73e880a640 1 --2- 172.21.15.43:0/2302357237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73e40cf270 0x7f73e40b57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:41.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.287+0000 7f73e880a640 1 -- 172.21.15.43:0/2302357237 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e4155160 con 0x7f73e40cb890 2024-04-25T09:00:41.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.287+0000 7f73e880a640 1 --2- 172.21.15.43:0/2302357237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73e40cb890 0x7f73e40b52b0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f73d800c7f0 tx=0x7f73d800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:41.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.287+0000 7f73e1ffb640 1 -- 172.21.15.43:0/2302357237 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d8015070 con 0x7f73e40cb890 2024-04-25T09:00:41.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.287+0000 7f73e1ffb640 1 -- 172.21.15.43:0/2302357237 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73d8007820 con 0x7f73e40cb890 2024-04-25T09:00:41.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.287+0000 7f73eb26f640 1 -- 172.21.15.43:0/2302357237 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73e40b6430 con 0x7f73e40cb890 2024-04-25T09:00:41.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.287+0000 7f73e1ffb640 1 -- 172.21.15.43:0/2302357237 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d8007bf0 con 0x7f73e40cb890 2024-04-25T09:00:41.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.287+0000 7f73eb26f640 1 -- 172.21.15.43:0/2302357237 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73e415f1a0 con 0x7f73e40cb890 2024-04-25T09:00:41.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.289+0000 7f73eb26f640 1 -- 172.21.15.43:0/2302357237 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73e4074f60 con 0x7f73e40cb890 2024-04-25T09:00:41.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.289+0000 7f73e1ffb640 1 -- 172.21.15.43:0/2302357237 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f73d8018070 con 0x7f73e40cb890 2024-04-25T09:00:41.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.289+0000 7f73e1ffb640 1 --2- 172.21.15.43:0/2302357237 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f73d006b4d0 0x7f73d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:41.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.289+0000 7f73e1ffb640 1 -- 172.21.15.43:0/2302357237 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f73d808a040 con 0x7f73e40cb890 2024-04-25T09:00:41.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.290+0000 7f73e900b640 1 --2- 172.21.15.43:0/2302357237 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f73d006b4d0 0x7f73d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:41.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.290+0000 7f73e900b640 1 --2- 172.21.15.43:0/2302357237 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f73d006b4d0 0x7f73d006d990 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f73d400b450 tx=0x7f73d400d040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:41.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.295+0000 7f73e1ffb640 1 -- 172.21.15.43:0/2302357237 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f73d8058ae0 con 0x7f73e40cb890 2024-04-25T09:00:41.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:41.476+0000 7f73eb26f640 1 -- 172.21.15.43:0/2302357237 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f73e40d0170 con 0x7f73e40cb890 2024-04-25T09:00:42.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.161+0000 7f73e1ffb640 1 -- 172.21.15.43:0/2302357237 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v87) v1 ==== 170+0+0 (secure 0 0 0) 0x7f73d805cb80 con 0x7f73e40cb890 2024-04-25T09:00:42.149 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' created 2024-04-25T09:00:42.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.164+0000 7f73eb26f640 1 -- 172.21.15.43:0/2302357237 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f73d006b4d0 msgr2=0x7f73d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:42.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.164+0000 7f73eb26f640 1 --2- 172.21.15.43:0/2302357237 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f73d006b4d0 0x7f73d006d990 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f73d400b450 tx=0x7f73d400d040 comp rx=0 tx=0).stop 2024-04-25T09:00:42.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.164+0000 7f73eb26f640 1 -- 172.21.15.43:0/2302357237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73e40cb890 msgr2=0x7f73e40b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:42.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.164+0000 7f73eb26f640 1 --2- 172.21.15.43:0/2302357237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73e40cb890 0x7f73e40b52b0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f73d800c7f0 tx=0x7f73d800ccb0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.164+0000 7f73eb26f640 1 -- 172.21.15.43:0/2302357237 shutdown_connections 2024-04-25T09:00:42.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.164+0000 7f73eb26f640 1 --2- 172.21.15.43:0/2302357237 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f73d006b4d0 0x7f73d006d990 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.164+0000 7f73eb26f640 1 --2- 172.21.15.43:0/2302357237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f73e40cf270 0x7f73e40b57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.164+0000 7f73eb26f640 1 --2- 172.21.15.43:0/2302357237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73e40cb890 0x7f73e40b52b0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.164+0000 7f73eb26f640 1 --2- 172.21.15.43:0/2302357237 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f73e40afd20 0x7f73e4154c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.164+0000 7f73eb26f640 1 -- 172.21.15.43:0/2302357237 >> 172.21.15.43:0/2302357237 conn(0x7f73e40d0730 msgr2=0x7f73e415b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:42.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.164+0000 7f73eb26f640 1 -- 172.21.15.43:0/2302357237 shutdown_connections 2024-04-25T09:00:42.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.164+0000 7f73eb26f640 1 -- 172.21.15.43:0/2302357237 wait complete. 2024-04-25T09:00:42.180 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:00:42.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.300+0000 7f3b31807640 1 Processor -- start 2024-04-25T09:00:42.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.301+0000 7f3b31807640 1 -- start start 2024-04-25T09:00:42.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.301+0000 7f3b31807640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3b2c15e090 0x7f3b2c161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:42.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.301+0000 7f3b31807640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b2c162610 con 0x7f3b2c161fe0 2024-04-25T09:00:42.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.301+0000 7f3b31807640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b2c0d1750 con 0x7f3b2c0b5c70 2024-04-25T09:00:42.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.301+0000 7f3b31807640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b2c0d1da0 con 0x7f3b2c15e090 2024-04-25T09:00:42.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.301+0000 7f3b2affd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3b2c15e090 0x7f3b2c161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:42.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.301+0000 7f3b2a7fc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f3b2c161fe0 0x7f3b2c0b5560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44272/0 (socket says 172.21.15.43:44272) 2024-04-25T09:00:42.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.301+0000 7f3b2a7fc640 1 -- 172.21.15.43:0/646394147 learned_addr learned my addr 172.21.15.43:0/646394147 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:00:42.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.302+0000 7f3b2affd640 1 -- 172.21.15.43:0/646394147 >> v1:172.21.15.193:6789/0 conn(0x7f3b2c0b5c70 legacy=0x7f3b2c0d2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:42.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.302+0000 7f3b2affd640 1 -- 172.21.15.43:0/646394147 >> v1:172.21.15.43:6789/0 conn(0x7f3b2c161fe0 legacy=0x7f3b2c0b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:42.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.302+0000 7f3b2affd640 1 -- 172.21.15.43:0/646394147 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b2c0d1a30 con 0x7f3b2c15e090 2024-04-25T09:00:42.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.302+0000 7f3b29ffb640 1 -- 172.21.15.43:0/646394147 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 200913520 0 0) 0x7f3b2c162610 con 0x7f3b2c161fe0 2024-04-25T09:00:42.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.303+0000 7f3b2affd640 1 --2- 172.21.15.43:0/646394147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b2c15e090 0x7f3b2c161aa0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c0097a0 tx=0x7f3b1c02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ba7004577663 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:42.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.303+0000 7f3b29ffb640 1 -- 172.21.15.43:0/646394147 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c037070 con 0x7f3b2c15e090 2024-04-25T09:00:42.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.303+0000 7f3b29ffb640 1 -- 172.21.15.43:0/646394147 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b1c032640 con 0x7f3b2c15e090 2024-04-25T09:00:42.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.303+0000 7f3b29ffb640 1 -- 172.21.15.43:0/646394147 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c032a10 con 0x7f3b2c15e090 2024-04-25T09:00:42.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.303+0000 7f3b31807640 1 -- 172.21.15.43:0/646394147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b2c15e090 msgr2=0x7f3b2c161aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:42.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.303+0000 7f3b31807640 1 --2- 172.21.15.43:0/646394147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b2c15e090 0x7f3b2c161aa0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c0097a0 tx=0x7f3b1c02a2c0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.303+0000 7f3b31807640 1 -- 172.21.15.43:0/646394147 shutdown_connections 2024-04-25T09:00:42.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.303+0000 7f3b31807640 1 --2- 172.21.15.43:0/646394147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b2c15e090 0x7f3b2c161aa0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.303+0000 7f3b31807640 1 -- 172.21.15.43:0/646394147 >> 172.21.15.43:0/646394147 conn(0x7f3b2c0d0560 msgr2=0x7f3b2c0d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:42.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.303+0000 7f3b31807640 1 -- 172.21.15.43:0/646394147 shutdown_connections 2024-04-25T09:00:42.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.303+0000 7f3b31807640 1 -- 172.21.15.43:0/646394147 wait complete. 2024-04-25T09:00:42.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.304+0000 7f3b31807640 1 Processor -- start 2024-04-25T09:00:42.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.305+0000 7f3b31807640 1 -- start start 2024-04-25T09:00:42.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.305+0000 7f3b31807640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b2c0b5c70 0x7f3b2c15b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:42.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.305+0000 7f3b31807640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b2c15e090 0x7f3b2c0cec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:42.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.305+0000 7f3b31807640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b2c161fe0 0x7f3b2c0b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:42.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.305+0000 7f3b31807640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2c0d2940 con 0x7f3b2c0b5c70 2024-04-25T09:00:42.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.305+0000 7f3b31807640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2c0b3280 con 0x7f3b2c15e090 2024-04-25T09:00:42.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.305+0000 7f3b31807640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2c0d2ac0 con 0x7f3b2c161fe0 2024-04-25T09:00:42.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.306+0000 7f3b2affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b2c0b5c70 0x7f3b2c15b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:42.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.306+0000 7f3b2affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b2c0b5c70 0x7f3b2c15b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43606/0 (socket says 172.21.15.43:43606) 2024-04-25T09:00:42.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.306+0000 7f3b2affd640 1 -- 172.21.15.43:0/424574754 learned_addr learned my addr 172.21.15.43:0/424574754 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:42.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.306+0000 7f3b2a7fc640 1 --2- 172.21.15.43:0/424574754 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b2c15e090 0x7f3b2c0cec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:42.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.306+0000 7f3b2b7fe640 1 --2- 172.21.15.43:0/424574754 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b2c161fe0 0x7f3b2c0b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:42.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.306+0000 7f3b2affd640 1 -- 172.21.15.43:0/424574754 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b2c161fe0 msgr2=0x7f3b2c0b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:42.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.306+0000 7f3b2affd640 1 --2- 172.21.15.43:0/424574754 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b2c161fe0 0x7f3b2c0b3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.306+0000 7f3b2affd640 1 -- 172.21.15.43:0/424574754 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b2c15e090 msgr2=0x7f3b2c0cec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:42.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.306+0000 7f3b2affd640 1 --2- 172.21.15.43:0/424574754 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b2c15e090 0x7f3b2c0cec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:42.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.306+0000 7f3b2affd640 1 -- 172.21.15.43:0/424574754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b2c0aa4f0 con 0x7f3b2c0b5c70 2024-04-25T09:00:42.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.306+0000 7f3b2b7fe640 1 --2- 172.21.15.43:0/424574754 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b2c161fe0 0x7f3b2c0b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:42.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.306+0000 7f3b2affd640 1 --2- 172.21.15.43:0/424574754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b2c0b5c70 0x7f3b2c15b540 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c035040 tx=0x7f3b1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:42.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.306+0000 7f3b13fff640 1 -- 172.21.15.43:0/424574754 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c041030 con 0x7f3b2c0b5c70 2024-04-25T09:00:42.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.306+0000 7f3b13fff640 1 -- 172.21.15.43:0/424574754 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b1c030750 con 0x7f3b2c0b5c70 2024-04-25T09:00:42.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.306+0000 7f3b31807640 1 -- 172.21.15.43:0/424574754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b2c0aa7d0 con 0x7f3b2c0b5c70 2024-04-25T09:00:42.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.306+0000 7f3b13fff640 1 -- 172.21.15.43:0/424574754 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c037040 con 0x7f3b2c0b5c70 2024-04-25T09:00:42.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.306+0000 7f3b31807640 1 -- 172.21.15.43:0/424574754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b2c0aad90 con 0x7f3b2c0b5c70 2024-04-25T09:00:42.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.307+0000 7f3b31807640 1 -- 172.21.15.43:0/424574754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b2c06bf00 con 0x7f3b2c0b5c70 2024-04-25T09:00:42.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.308+0000 7f3b13fff640 1 -- 172.21.15.43:0/424574754 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3b1c044070 con 0x7f3b2c0b5c70 2024-04-25T09:00:42.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.308+0000 7f3b13fff640 1 --2- 172.21.15.43:0/424574754 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3b1806b4d0 0x7f3b1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:42.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.308+0000 7f3b13fff640 1 -- 172.21.15.43:0/424574754 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f3b1c0aaa60 con 0x7f3b2c0b5c70 2024-04-25T09:00:42.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.309+0000 7f3b2a7fc640 1 --2- 172.21.15.43:0/424574754 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3b1806b4d0 0x7f3b1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:42.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.309+0000 7f3b2a7fc640 1 --2- 172.21.15.43:0/424574754 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3b1806b4d0 0x7f3b1806d990 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c0ac160 tx=0x7f3b20007d00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:42.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.313+0000 7f3b13fff640 1 -- 172.21.15.43:0/424574754 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3b2c06bf00 con 0x7f3b2c0b5c70 2024-04-25T09:00:42.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:42.535+0000 7f3b31807640 1 -- 172.21.15.43:0/424574754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3b2c074f60 con 0x7f3b2c0b5c70 2024-04-25T09:00:43.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.162+0000 7f3b13fff640 1 -- 172.21.15.43:0/424574754 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v52) v1 ==== 174+0+0 (secure 0 0 0) 0x7f3b1c079af0 con 0x7f3b2c0b5c70 2024-04-25T09:00:43.151 INFO:teuthology.orchestra.run.smithi043.stderr:new fs with metadata pool 12 and data pool 13 2024-04-25T09:00:43.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.166+0000 7f3b31807640 1 -- 172.21.15.43:0/424574754 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3b1806b4d0 msgr2=0x7f3b1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.166+0000 7f3b31807640 1 --2- 172.21.15.43:0/424574754 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3b1806b4d0 0x7f3b1806d990 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c0ac160 tx=0x7f3b20007d00 comp rx=0 tx=0).stop 2024-04-25T09:00:43.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.166+0000 7f3b31807640 1 -- 172.21.15.43:0/424574754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b2c0b5c70 msgr2=0x7f3b2c15b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.166+0000 7f3b31807640 1 --2- 172.21.15.43:0/424574754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b2c0b5c70 0x7f3b2c15b540 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c035040 tx=0x7f3b1c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.166+0000 7f3b31807640 1 -- 172.21.15.43:0/424574754 shutdown_connections 2024-04-25T09:00:43.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.166+0000 7f3b31807640 1 --2- 172.21.15.43:0/424574754 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3b2c161fe0 0x7f3b2c0b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.166+0000 7f3b31807640 1 --2- 172.21.15.43:0/424574754 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3b1806b4d0 0x7f3b1806d990 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.166+0000 7f3b31807640 1 --2- 172.21.15.43:0/424574754 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3b2c15e090 0x7f3b2c0cec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.166+0000 7f3b31807640 1 --2- 172.21.15.43:0/424574754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3b2c0b5c70 0x7f3b2c15b540 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.166+0000 7f3b31807640 1 -- 172.21.15.43:0/424574754 >> 172.21.15.43:0/424574754 conn(0x7f3b2c0d0560 msgr2=0x7f3b2c160580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:43.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.167+0000 7f3b31807640 1 -- 172.21.15.43:0/424574754 shutdown_connections 2024-04-25T09:00:43.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.167+0000 7f3b31807640 1 -- 172.21.15.43:0/424574754 wait complete. 2024-04-25T09:00:43.184 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:00:43.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.300+0000 7f2a90ce1640 1 Processor -- start 2024-04-25T09:00:43.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.300+0000 7f2a90ce1640 1 -- start start 2024-04-25T09:00:43.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.301+0000 7f2a90ce1640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a8c161390 0x7f2a8c0b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.301+0000 7f2a90ce1640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f2a8c0cecf0 0x7f2a8c0d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.301+0000 7f2a90ce1640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a8c0b4c30 con 0x7f2a8c0d02a0 2024-04-25T09:00:43.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.301+0000 7f2a90ce1640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a8c0d1910 con 0x7f2a8c0cecf0 2024-04-25T09:00:43.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.301+0000 7f2a90ce1640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8c0cf2c0 con 0x7f2a8c161390 2024-04-25T09:00:43.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.301+0000 7f2a89d9b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f2a8c0d02a0 0x7f2a8c0ce5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44284/0 (socket says 172.21.15.43:44284) 2024-04-25T09:00:43.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.301+0000 7f2a89d9b640 1 -- 172.21.15.43:0/1199059194 learned_addr learned my addr 172.21.15.43:0/1199059194 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:00:43.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.301+0000 7f2a8959a640 1 -- 172.21.15.43:0/1199059194 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2202485872 0 0) 0x7f2a80001110 con 0x7f2a8c0d02a0 2024-04-25T09:00:43.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.301+0000 7f2a8a59c640 1 --2- 172.21.15.43:0/1199059194 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a8c161390 0x7f2a8c0b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.301+0000 7f2a8ad9d640 1 --2- 172.21.15.43:0/1199059194 >> v2:172.21.15.193:3300/0 conn(0x7f2a8c0cecf0 0x7f2a8c0d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.301+0000 7f2a8959a640 1 -- 172.21.15.43:0/1199059194 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a74003520 con 0x7f2a8c0d02a0 2024-04-25T09:00:43.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.301+0000 7f2a8959a640 1 -- 172.21.15.43:0/1199059194 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2299974402 0 0) 0x7f2a74003520 con 0x7f2a8c0d02a0 2024-04-25T09:00:43.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.301+0000 7f2a8959a640 1 -- 172.21.15.43:0/1199059194 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a8c161390 msgr2=0x7f2a8c0b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:43.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.301+0000 7f2a8959a640 1 --2- 172.21.15.43:0/1199059194 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a8c161390 0x7f2a8c0b6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.301+0000 7f2a8959a640 1 -- 172.21.15.43:0/1199059194 >> v2:172.21.15.193:3300/0 conn(0x7f2a8c0cecf0 msgr2=0x7f2a8c0d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:43.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.301+0000 7f2a8959a640 1 --2- 172.21.15.43:0/1199059194 >> v2:172.21.15.193:3300/0 conn(0x7f2a8c0cecf0 0x7f2a8c0d0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.301+0000 7f2a8959a640 1 -- 172.21.15.43:0/1199059194 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a8c0cf5a0 con 0x7f2a8c0d02a0 2024-04-25T09:00:43.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.301+0000 7f2a8959a640 1 -- 172.21.15.43:0/1199059194 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f2a80001840 con 0x7f2a8c0d02a0 2024-04-25T09:00:43.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.301+0000 7f2a8959a640 1 -- 172.21.15.43:0/1199059194 >> v1:172.21.15.43:6789/0 conn(0x7f2a8c0d02a0 legacy=0x7f2a8c0ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.301+0000 7f2a8959a640 1 --2- 172.21.15.43:0/1199059194 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a8c161390 0x7f2a740039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.301+0000 7f2a8959a640 1 --2- 172.21.15.43:0/1199059194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a74003f30 0x7f2a74004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.302+0000 7f2a8959a640 1 --2- 172.21.15.43:0/1199059194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a74004d40 0x7f2a740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.302+0000 7f2a8959a640 1 -- 172.21.15.43:0/1199059194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8c0d1910 con 0x7f2a74004d40 2024-04-25T09:00:43.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.302+0000 7f2a8959a640 1 -- 172.21.15.43:0/1199059194 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8c0cf2c0 con 0x7f2a74003f30 2024-04-25T09:00:43.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.302+0000 7f2a8959a640 1 -- 172.21.15.43:0/1199059194 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7400b9c0 con 0x7f2a8c161390 2024-04-25T09:00:43.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.302+0000 7f2a8ad9d640 1 --2- 172.21.15.43:0/1199059194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a74004d40 0x7f2a740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.302+0000 7f2a89d9b640 1 --2- 172.21.15.43:0/1199059194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a74003f30 0x7f2a74004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.302+0000 7f2a8a59c640 1 --2- 172.21.15.43:0/1199059194 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a8c161390 0x7f2a740039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.302+0000 7f2a8ad9d640 1 -- 172.21.15.43:0/1199059194 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a8c161390 msgr2=0x7f2a740039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:43.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.302+0000 7f2a8ad9d640 1 --2- 172.21.15.43:0/1199059194 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a8c161390 0x7f2a740039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.302+0000 7f2a8ad9d640 1 -- 172.21.15.43:0/1199059194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a74003f30 msgr2=0x7f2a74004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:43.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.302+0000 7f2a8ad9d640 1 --2- 172.21.15.43:0/1199059194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a74003f30 0x7f2a74004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.302+0000 7f2a8ad9d640 1 -- 172.21.15.43:0/1199059194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a74003520 con 0x7f2a74004d40 2024-04-25T09:00:43.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.303+0000 7f2a8ad9d640 1 --2- 172.21.15.43:0/1199059194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a74004d40 0x7f2a740051c0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f2a84009fc0 tx=0x7f2a74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76afe6e05234340 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:43.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.303+0000 7f2a8959a640 1 -- 172.21.15.43:0/1199059194 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a84009c10 con 0x7f2a74004d40 2024-04-25T09:00:43.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.303+0000 7f2a8959a640 1 -- 172.21.15.43:0/1199059194 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a84036070 con 0x7f2a74004d40 2024-04-25T09:00:43.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.303+0000 7f2a8959a640 1 -- 172.21.15.43:0/1199059194 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8402d070 con 0x7f2a74004d40 2024-04-25T09:00:43.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.303+0000 7f2a90ce1640 1 -- 172.21.15.43:0/1199059194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a74004d40 msgr2=0x7f2a740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.303+0000 7f2a90ce1640 1 --2- 172.21.15.43:0/1199059194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a74004d40 0x7f2a740051c0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f2a84009fc0 tx=0x7f2a74005830 comp rx=0 tx=0).stop 2024-04-25T09:00:43.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.303+0000 7f2a8ad9d640 1 -- 172.21.15.43:0/1199059194 reap_dead start 2024-04-25T09:00:43.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.303+0000 7f2a90ce1640 1 -- 172.21.15.43:0/1199059194 shutdown_connections 2024-04-25T09:00:43.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.303+0000 7f2a90ce1640 1 -- 172.21.15.43:0/1199059194 >> 172.21.15.43:0/1199059194 conn(0x7f2a8c09cd20 msgr2=0x7f2a8c0d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:43.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.303+0000 7f2a90ce1640 1 -- 172.21.15.43:0/1199059194 shutdown_connections 2024-04-25T09:00:43.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.304+0000 7f2a90ce1640 1 -- 172.21.15.43:0/1199059194 wait complete. 2024-04-25T09:00:43.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.305+0000 7f2a90ce1640 1 Processor -- start 2024-04-25T09:00:43.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.305+0000 7f2a90ce1640 1 -- start start 2024-04-25T09:00:43.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.306+0000 7f2a90ce1640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a74004d40 0x7f2a8c15ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.306+0000 7f2a90ce1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a8c14dcc0 0x7f2a8c0cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.306+0000 7f2a90ce1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a8c14e140 0x7f2a8c0ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.306+0000 7f2a90ce1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8c161770 con 0x7f2a8c14dcc0 2024-04-25T09:00:43.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.306+0000 7f2a90ce1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8c0d15a0 con 0x7f2a8c14e140 2024-04-25T09:00:43.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.306+0000 7f2a90ce1640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8c0b6820 con 0x7f2a74004d40 2024-04-25T09:00:43.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.306+0000 7f2a89d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a8c14dcc0 0x7f2a8c0cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.306+0000 7f2a89d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a8c14dcc0 0x7f2a8c0cc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43634/0 (socket says 172.21.15.43:43634) 2024-04-25T09:00:43.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.306+0000 7f2a89d9b640 1 -- 172.21.15.43:0/74758936 learned_addr learned my addr 172.21.15.43:0/74758936 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:43.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.306+0000 7f2a8a59c640 1 --2- 172.21.15.43:0/74758936 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a74004d40 0x7f2a8c15ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.306+0000 7f2a8ad9d640 1 --2- 172.21.15.43:0/74758936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a8c14e140 0x7f2a8c0ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.306+0000 7f2a89d9b640 1 -- 172.21.15.43:0/74758936 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a74004d40 msgr2=0x7f2a8c15ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.306+0000 7f2a89d9b640 1 --2- 172.21.15.43:0/74758936 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a74004d40 0x7f2a8c15ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.306+0000 7f2a89d9b640 1 -- 172.21.15.43:0/74758936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a8c14e140 msgr2=0x7f2a8c0ccb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.306+0000 7f2a89d9b640 1 --2- 172.21.15.43:0/74758936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a8c14e140 0x7f2a8c0ccb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.306+0000 7f2a89d9b640 1 -- 172.21.15.43:0/74758936 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a8c1f7060 con 0x7f2a8c14dcc0 2024-04-25T09:00:43.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.306+0000 7f2a8ad9d640 1 --2- 172.21.15.43:0/74758936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a8c14e140 0x7f2a8c0ccb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:00:43.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.306+0000 7f2a89d9b640 1 --2- 172.21.15.43:0/74758936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a8c14dcc0 0x7f2a8c0cc640 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f2a80001c40 tx=0x7f2a800076e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:43.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.306+0000 7f2a7affd640 1 -- 172.21.15.43:0/74758936 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a80004e50 con 0x7f2a8c14dcc0 2024-04-25T09:00:43.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.306+0000 7f2a7affd640 1 -- 172.21.15.43:0/74758936 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a80018070 con 0x7f2a8c14dcc0 2024-04-25T09:00:43.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.306+0000 7f2a7affd640 1 -- 172.21.15.43:0/74758936 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8000fbd0 con 0x7f2a8c14dcc0 2024-04-25T09:00:43.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.306+0000 7f2a90ce1640 1 -- 172.21.15.43:0/74758936 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a8c1f7340 con 0x7f2a8c14dcc0 2024-04-25T09:00:43.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.306+0000 7f2a90ce1640 1 -- 172.21.15.43:0/74758936 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a8c1f7900 con 0x7f2a8c14dcc0 2024-04-25T09:00:43.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.308+0000 7f2a90ce1640 1 -- 172.21.15.43:0/74758936 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a8c0680f0 con 0x7f2a8c14dcc0 2024-04-25T09:00:43.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.308+0000 7f2a7affd640 1 -- 172.21.15.43:0/74758936 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2a8001d460 con 0x7f2a8c14dcc0 2024-04-25T09:00:43.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.308+0000 7f2a7affd640 1 --2- 172.21.15.43:0/74758936 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2a6c06b670 0x7f2a6c06db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.309+0000 7f2a7affd640 1 -- 172.21.15.43:0/74758936 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2a8008a200 con 0x7f2a8c14dcc0 2024-04-25T09:00:43.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.309+0000 7f2a8a59c640 1 --2- 172.21.15.43:0/74758936 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2a6c06b670 0x7f2a6c06db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.309+0000 7f2a8a59c640 1 --2- 172.21.15.43:0/74758936 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2a6c06b670 0x7f2a6c06db30 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c009bc0 tx=0x7f2a7c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:43.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.313+0000 7f2a7affd640 1 -- 172.21.15.43:0/74758936 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2a8c0b4c30 con 0x7f2a8c14dcc0 2024-04-25T09:00:43.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.494+0000 7f2a90ce1640 1 -- 172.21.15.43:0/74758936 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2a8c074ef0 con 0x7f2a8c14dcc0 2024-04-25T09:00:43.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.495+0000 7f2a7affd640 1 -- 172.21.15.43:0/74758936 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+16637 (secure 0 0 0) 0x7f2a80059110 con 0x7f2a8c14dcc0 2024-04-25T09:00:43.483 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:00:43.483 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":88,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:00:42.536263+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:00:40.816230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T09:00:41.508021+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:00:43.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.497+0000 7f2a90ce1640 1 -- 172.21.15.43:0/74758936 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2a6c06b670 msgr2=0x7f2a6c06db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.497+0000 7f2a90ce1640 1 --2- 172.21.15.43:0/74758936 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2a6c06b670 0x7f2a6c06db30 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c009bc0 tx=0x7f2a7c009190 comp rx=0 tx=0).stop 2024-04-25T09:00:43.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.497+0000 7f2a90ce1640 1 -- 172.21.15.43:0/74758936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a8c14dcc0 msgr2=0x7f2a8c0cc640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.497+0000 7f2a90ce1640 1 --2- 172.21.15.43:0/74758936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a8c14dcc0 0x7f2a8c0cc640 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f2a80001c40 tx=0x7f2a800076e0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.497+0000 7f2a90ce1640 1 -- 172.21.15.43:0/74758936 shutdown_connections 2024-04-25T09:00:43.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.497+0000 7f2a90ce1640 1 --2- 172.21.15.43:0/74758936 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2a6c06b670 0x7f2a6c06db30 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.497+0000 7f2a90ce1640 1 --2- 172.21.15.43:0/74758936 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a8c14e140 0x7f2a8c0ccb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.497+0000 7f2a90ce1640 1 --2- 172.21.15.43:0/74758936 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a8c14dcc0 0x7f2a8c0cc640 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.497+0000 7f2a90ce1640 1 --2- 172.21.15.43:0/74758936 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a74004d40 0x7f2a8c15ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.497+0000 7f2a90ce1640 1 -- 172.21.15.43:0/74758936 >> 172.21.15.43:0/74758936 conn(0x7f2a8c09cd20 msgr2=0x7f2a8c0b76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:43.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.497+0000 7f2a90ce1640 1 -- 172.21.15.43:0/74758936 shutdown_connections 2024-04-25T09:00:43.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.497+0000 7f2a90ce1640 1 -- 172.21.15.43:0/74758936 wait complete. 2024-04-25T09:00:43.496 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:00:43.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.609+0000 7f8a5d532640 1 Processor -- start 2024-04-25T09:00:43.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.609+0000 7f8a5d532640 1 -- start start 2024-04-25T09:00:43.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.610+0000 7f8a5d532640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8a5806be50 0x7f8a58162100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.610+0000 7f8a5d532640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8a580b20c0 0x7f8a580b2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.610+0000 7f8a5d532640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a5815e2d0 con 0x7f8a5815def0 2024-04-25T09:00:43.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.610+0000 7f8a5d532640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a580d0e90 con 0x7f8a5806be50 2024-04-25T09:00:43.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.610+0000 7f8a5d532640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a580b1020 con 0x7f8a580b20c0 2024-04-25T09:00:43.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.610+0000 7f8a567fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8a5806be50 0x7f8a58162100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.610+0000 7f8a56ffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8a5815def0 0x7f8a581619f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44286/0 (socket says 172.21.15.43:44286) 2024-04-25T09:00:43.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.610+0000 7f8a56ffd640 1 -- 172.21.15.43:0/3502178549 learned_addr learned my addr 172.21.15.43:0/3502178549 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:00:43.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.610+0000 7f8a567fc640 1 -- 172.21.15.43:0/3502178549 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8a580b20c0 msgr2=0x7f8a580b2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:43.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.610+0000 7f8a577fe640 1 --2- 172.21.15.43:0/3502178549 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8a580b20c0 0x7f8a580b2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.610+0000 7f8a55ffb640 1 -- 172.21.15.43:0/3502178549 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3053726403 0 0) 0x7f8a5815e2d0 con 0x7f8a5815def0 2024-04-25T09:00:43.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.610+0000 7f8a567fc640 1 --2- 172.21.15.43:0/3502178549 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8a580b20c0 0x7f8a580b2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.610+0000 7f8a567fc640 1 -- 172.21.15.43:0/3502178549 >> v1:172.21.15.43:6789/0 conn(0x7f8a5815def0 legacy=0x7f8a581619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.611+0000 7f8a567fc640 1 -- 172.21.15.43:0/3502178549 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a580b1300 con 0x7f8a5806be50 2024-04-25T09:00:43.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.611+0000 7f8a567fc640 1 --2- 172.21.15.43:0/3502178549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a5806be50 0x7f8a58162100 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f8a40009790 tx=0x7f8a4002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21e53e9b1b4e0fdf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:43.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.611+0000 7f8a55ffb640 1 -- 172.21.15.43:0/3502178549 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a40037070 con 0x7f8a5806be50 2024-04-25T09:00:43.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.611+0000 7f8a55ffb640 1 -- 172.21.15.43:0/3502178549 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a40032440 con 0x7f8a5806be50 2024-04-25T09:00:43.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.611+0000 7f8a55ffb640 1 -- 172.21.15.43:0/3502178549 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a40032810 con 0x7f8a5806be50 2024-04-25T09:00:43.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.612+0000 7f8a5d532640 1 -- 172.21.15.43:0/3502178549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a5806be50 msgr2=0x7f8a58162100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.612+0000 7f8a5d532640 1 --2- 172.21.15.43:0/3502178549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a5806be50 0x7f8a58162100 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f8a40009790 tx=0x7f8a4002a180 comp rx=0 tx=0).stop 2024-04-25T09:00:43.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.612+0000 7f8a5d532640 1 -- 172.21.15.43:0/3502178549 shutdown_connections 2024-04-25T09:00:43.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.612+0000 7f8a5d532640 1 --2- 172.21.15.43:0/3502178549 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8a580b20c0 0x7f8a580b2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.612+0000 7f8a5d532640 1 --2- 172.21.15.43:0/3502178549 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a5806be50 0x7f8a58162100 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.612+0000 7f8a5d532640 1 -- 172.21.15.43:0/3502178549 >> 172.21.15.43:0/3502178549 conn(0x7f8a580b32e0 msgr2=0x7f8a580b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:43.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.612+0000 7f8a5d532640 1 -- 172.21.15.43:0/3502178549 shutdown_connections 2024-04-25T09:00:43.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.612+0000 7f8a5d532640 1 -- 172.21.15.43:0/3502178549 wait complete. 2024-04-25T09:00:43.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.613+0000 7f8a5d532640 1 Processor -- start 2024-04-25T09:00:43.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.613+0000 7f8a5d532640 1 -- start start 2024-04-25T09:00:43.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.614+0000 7f8a5d532640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a5806be50 0x7f8a5814b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.614+0000 7f8a5d532640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8a580b20c0 0x7f8a5814d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.614+0000 7f8a5d532640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a5815def0 0x7f8a5814baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.614+0000 7f8a5d532640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a580b1ba0 con 0x7f8a5815def0 2024-04-25T09:00:43.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.614+0000 7f8a5d532640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a580b18c0 con 0x7f8a5806be50 2024-04-25T09:00:43.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.614+0000 7f8a5d532640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5806c2b0 con 0x7f8a580b20c0 2024-04-25T09:00:43.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.614+0000 7f8a577fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a5815def0 0x7f8a5814baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.614+0000 7f8a577fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a5815def0 0x7f8a5814baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43664/0 (socket says 172.21.15.43:43664) 2024-04-25T09:00:43.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.614+0000 7f8a577fe640 1 -- 172.21.15.43:0/2937858639 learned_addr learned my addr 172.21.15.43:0/2937858639 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:43.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.614+0000 7f8a56ffd640 1 --2- 172.21.15.43:0/2937858639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a5806be50 0x7f8a5814b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.614+0000 7f8a567fc640 1 --2- 172.21.15.43:0/2937858639 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8a580b20c0 0x7f8a5814d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.615+0000 7f8a577fe640 1 -- 172.21.15.43:0/2937858639 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8a580b20c0 msgr2=0x7f8a5814d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.615+0000 7f8a577fe640 1 --2- 172.21.15.43:0/2937858639 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8a580b20c0 0x7f8a5814d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.615+0000 7f8a577fe640 1 -- 172.21.15.43:0/2937858639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a5806be50 msgr2=0x7f8a5814b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.615+0000 7f8a577fe640 1 --2- 172.21.15.43:0/2937858639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a5806be50 0x7f8a5814b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.615+0000 7f8a577fe640 1 -- 172.21.15.43:0/2937858639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a5814b570 con 0x7f8a5815def0 2024-04-25T09:00:43.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.615+0000 7f8a56ffd640 1 --2- 172.21.15.43:0/2937858639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a5806be50 0x7f8a5814b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:00:43.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.615+0000 7f8a567fc640 1 --2- 172.21.15.43:0/2937858639 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8a580b20c0 0x7f8a5814d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:00:43.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.615+0000 7f8a577fe640 1 --2- 172.21.15.43:0/2937858639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a5815def0 0x7f8a5814baf0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f8a5000cd70 tx=0x7f8a5000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:43.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.615+0000 7f8a4f7fe640 1 -- 172.21.15.43:0/2937858639 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a50003b90 con 0x7f8a5815def0 2024-04-25T09:00:43.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.615+0000 7f8a4f7fe640 1 -- 172.21.15.43:0/2937858639 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a50003d30 con 0x7f8a5815def0 2024-04-25T09:00:43.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.615+0000 7f8a5d532640 1 -- 172.21.15.43:0/2937858639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a58148950 con 0x7f8a5815def0 2024-04-25T09:00:43.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.615+0000 7f8a4f7fe640 1 -- 172.21.15.43:0/2937858639 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a50004240 con 0x7f8a5815def0 2024-04-25T09:00:43.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.615+0000 7f8a5d532640 1 -- 172.21.15.43:0/2937858639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a58148f10 con 0x7f8a5815def0 2024-04-25T09:00:43.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.616+0000 7f8a5d532640 1 -- 172.21.15.43:0/2937858639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a580b7710 con 0x7f8a5815def0 2024-04-25T09:00:43.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.616+0000 7f8a4f7fe640 1 -- 172.21.15.43:0/2937858639 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8a500043e0 con 0x7f8a5815def0 2024-04-25T09:00:43.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.617+0000 7f8a4f7fe640 1 --2- 172.21.15.43:0/2937858639 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8a3806b4d0 0x7f8a3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.617+0000 7f8a4f7fe640 1 -- 172.21.15.43:0/2937858639 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a5008ad80 con 0x7f8a5815def0 2024-04-25T09:00:43.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.617+0000 7f8a56ffd640 1 --2- 172.21.15.43:0/2937858639 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8a3806b4d0 0x7f8a3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.618+0000 7f8a56ffd640 1 --2- 172.21.15.43:0/2937858639 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8a3806b4d0 0x7f8a3806d990 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8a48000f60 tx=0x7f8a48007c10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:43.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.621+0000 7f8a4f7fe640 1 -- 172.21.15.43:0/2937858639 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8a5001d340 con 0x7f8a5815def0 2024-04-25T09:00:43.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.804+0000 7f8a5d532640 1 -- 172.21.15.43:0/2937858639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a58074ed0 con 0x7f8a5815def0 2024-04-25T09:00:43.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.805+0000 7f8a4f7fe640 1 -- 172.21.15.43:0/2937858639 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+16637 (secure 0 0 0) 0x7f8a50059c00 con 0x7f8a5815def0 2024-04-25T09:00:43.793 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:00:43.793 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":88,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:00:42.536263+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:00:40.816230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T09:00:41.508021+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:00:43.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.807+0000 7f8a5d532640 1 -- 172.21.15.43:0/2937858639 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8a3806b4d0 msgr2=0x7f8a3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.807+0000 7f8a5d532640 1 --2- 172.21.15.43:0/2937858639 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8a3806b4d0 0x7f8a3806d990 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8a48000f60 tx=0x7f8a48007c10 comp rx=0 tx=0).stop 2024-04-25T09:00:43.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.807+0000 7f8a5d532640 1 -- 172.21.15.43:0/2937858639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a5815def0 msgr2=0x7f8a5814baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.807+0000 7f8a5d532640 1 --2- 172.21.15.43:0/2937858639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a5815def0 0x7f8a5814baf0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f8a5000cd70 tx=0x7f8a5000a660 comp rx=0 tx=0).stop 2024-04-25T09:00:43.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.807+0000 7f8a5d532640 1 -- 172.21.15.43:0/2937858639 shutdown_connections 2024-04-25T09:00:43.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.807+0000 7f8a5d532640 1 --2- 172.21.15.43:0/2937858639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a5815def0 0x7f8a5814baf0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.807+0000 7f8a5d532640 1 --2- 172.21.15.43:0/2937858639 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8a580b20c0 0x7f8a5814d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.807+0000 7f8a5d532640 1 --2- 172.21.15.43:0/2937858639 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8a3806b4d0 0x7f8a3806d990 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.807+0000 7f8a5d532640 1 --2- 172.21.15.43:0/2937858639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a5806be50 0x7f8a5814b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.807+0000 7f8a5d532640 1 -- 172.21.15.43:0/2937858639 >> 172.21.15.43:0/2937858639 conn(0x7f8a580b32e0 msgr2=0x7f8a5815ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:43.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.807+0000 7f8a5d532640 1 -- 172.21.15.43:0/2937858639 shutdown_connections 2024-04-25T09:00:43.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.807+0000 7f8a5d532640 1 -- 172.21.15.43:0/2937858639 wait complete. 2024-04-25T09:00:43.807 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:00:43.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.923+0000 7f0566563640 1 Processor -- start 2024-04-25T09:00:43.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.923+0000 7f0566563640 1 -- start start 2024-04-25T09:00:43.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.924+0000 7f0566563640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f05600d19f0 0x7f05600d1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.924+0000 7f0566563640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f05600d2370 0x7f05600c7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.924+0000 7f0566563640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05600cf030 con 0x7f0560161310 2024-04-25T09:00:43.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.924+0000 7f0566563640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05600d4870 con 0x7f05600d19f0 2024-04-25T09:00:43.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.924+0000 7f0566563640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05600cf310 con 0x7f05600d2370 2024-04-25T09:00:43.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.924+0000 7f0564b00640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f05600d2370 0x7f05600c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.924+0000 7f055f7fe640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f05600d19f0 0x7f05600d1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.924+0000 7f055ffff640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f0560161310 0x7f05600d4080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:33576/0 (socket says 172.21.15.43:33576) 2024-04-25T09:00:43.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.924+0000 7f055ffff640 1 -- 172.21.15.43:0/128188565 learned_addr learned my addr 172.21.15.43:0/128188565 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:00:43.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.924+0000 7f055effd640 1 -- 172.21.15.43:0/128188565 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 127156690 0 0) 0x7f05600cf030 con 0x7f0560161310 2024-04-25T09:00:43.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.924+0000 7f055effd640 1 -- 172.21.15.43:0/128188565 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f054c003520 con 0x7f0560161310 2024-04-25T09:00:43.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.924+0000 7f0564b00640 1 -- 172.21.15.43:0/128188565 >> v2:172.21.15.193:3300/0 conn(0x7f05600d19f0 msgr2=0x7f05600d1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:43.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.924+0000 7f0564b00640 1 --2- 172.21.15.43:0/128188565 >> v2:172.21.15.193:3300/0 conn(0x7f05600d19f0 0x7f05600d1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.925+0000 7f0564b00640 1 -- 172.21.15.43:0/128188565 >> v1:172.21.15.193:6789/0 conn(0x7f0560161310 legacy=0x7f05600d4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.925+0000 7f0564b00640 1 -- 172.21.15.43:0/128188565 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05600cf5f0 con 0x7f05600d2370 2024-04-25T09:00:43.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.925+0000 7f055f7fe640 1 --2- 172.21.15.43:0/128188565 >> v2:172.21.15.193:3300/0 conn(0x7f05600d19f0 0x7f05600d1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:00:43.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.925+0000 7f0564b00640 1 --2- 172.21.15.43:0/128188565 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f05600d2370 0x7f05600c7860 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0558009d90 tx=0x7f055802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=98b3423b5eb7ffe0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:43.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.925+0000 7f055effd640 1 -- 172.21.15.43:0/128188565 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0558007740 con 0x7f05600d2370 2024-04-25T09:00:43.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.925+0000 7f055effd640 1 -- 172.21.15.43:0/128188565 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05580078e0 con 0x7f05600d2370 2024-04-25T09:00:43.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.925+0000 7f055effd640 1 -- 172.21.15.43:0/128188565 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0558007c90 con 0x7f05600d2370 2024-04-25T09:00:43.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.926+0000 7f0566563640 1 -- 172.21.15.43:0/128188565 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f05600d2370 msgr2=0x7f05600c7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.926+0000 7f0566563640 1 --2- 172.21.15.43:0/128188565 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f05600d2370 0x7f05600c7860 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0558009d90 tx=0x7f055802a660 comp rx=0 tx=0).stop 2024-04-25T09:00:43.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.926+0000 7f0566563640 1 -- 172.21.15.43:0/128188565 shutdown_connections 2024-04-25T09:00:43.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.926+0000 7f0566563640 1 --2- 172.21.15.43:0/128188565 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f05600d2370 0x7f05600c7860 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.926+0000 7f0566563640 1 --2- 172.21.15.43:0/128188565 >> v2:172.21.15.193:3300/0 conn(0x7f05600d19f0 0x7f05600d1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.926+0000 7f0566563640 1 -- 172.21.15.43:0/128188565 >> 172.21.15.43:0/128188565 conn(0x7f056009cd20 msgr2=0x7f05600d3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:43.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.926+0000 7f0566563640 1 -- 172.21.15.43:0/128188565 shutdown_connections 2024-04-25T09:00:43.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.926+0000 7f0566563640 1 -- 172.21.15.43:0/128188565 wait complete. 2024-04-25T09:00:43.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.927+0000 7f0566563640 1 Processor -- start 2024-04-25T09:00:43.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.927+0000 7f0566563640 1 -- start start 2024-04-25T09:00:43.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.928+0000 7f0566563640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f05600d19f0 0x7f05601f4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.928+0000 7f0566563640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05600d2370 0x7f05600cbff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.928+0000 7f0566563640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0560161310 0x7f05600d0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.928+0000 7f0566563640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05600c7da0 con 0x7f05600d2370 2024-04-25T09:00:43.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.928+0000 7f0566563640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05600d2800 con 0x7f0560161310 2024-04-25T09:00:43.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.928+0000 7f0566563640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05600c7f20 con 0x7f05600d19f0 2024-04-25T09:00:43.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.928+0000 7f055f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05600d2370 0x7f05600cbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.928+0000 7f055f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05600d2370 0x7f05600cbff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43668/0 (socket says 172.21.15.43:43668) 2024-04-25T09:00:43.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.928+0000 7f055f7fe640 1 -- 172.21.15.43:0/2535141707 learned_addr learned my addr 172.21.15.43:0/2535141707 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:43.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.928+0000 7f0564b00640 1 --2- 172.21.15.43:0/2535141707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0560161310 0x7f05600d0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.929+0000 7f055ffff640 1 --2- 172.21.15.43:0/2535141707 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f05600d19f0 0x7f05601f4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.929+0000 7f055f7fe640 1 -- 172.21.15.43:0/2535141707 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f05600d19f0 msgr2=0x7f05601f4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.929+0000 7f055f7fe640 1 --2- 172.21.15.43:0/2535141707 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f05600d19f0 0x7f05601f4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.929+0000 7f055f7fe640 1 -- 172.21.15.43:0/2535141707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0560161310 msgr2=0x7f05600d0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:43.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.929+0000 7f055f7fe640 1 --2- 172.21.15.43:0/2535141707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0560161310 0x7f05600d0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:43.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.929+0000 7f055f7fe640 1 -- 172.21.15.43:0/2535141707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05600cc530 con 0x7f05600d2370 2024-04-25T09:00:43.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.929+0000 7f0564b00640 1 --2- 172.21.15.43:0/2535141707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0560161310 0x7f05600d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:43.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.929+0000 7f055f7fe640 1 --2- 172.21.15.43:0/2535141707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05600d2370 0x7f05600cbff0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f054800ebe0 tx=0x7f054800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:43.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.929+0000 7f055cff9640 1 -- 172.21.15.43:0/2535141707 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0548017070 con 0x7f05600d2370 2024-04-25T09:00:43.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.929+0000 7f055cff9640 1 -- 172.21.15.43:0/2535141707 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f054800cc30 con 0x7f05600d2370 2024-04-25T09:00:43.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.929+0000 7f0566563640 1 -- 172.21.15.43:0/2535141707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05600b86e0 con 0x7f05600d2370 2024-04-25T09:00:43.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.929+0000 7f055cff9640 1 -- 172.21.15.43:0/2535141707 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0548013470 con 0x7f05600d2370 2024-04-25T09:00:43.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.929+0000 7f0566563640 1 -- 172.21.15.43:0/2535141707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05600b8ca0 con 0x7f05600d2370 2024-04-25T09:00:43.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.930+0000 7f0566563640 1 -- 172.21.15.43:0/2535141707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0560074f60 con 0x7f05600d2370 2024-04-25T09:00:43.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.930+0000 7f055cff9640 1 -- 172.21.15.43:0/2535141707 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f054801a070 con 0x7f05600d2370 2024-04-25T09:00:43.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.931+0000 7f055cff9640 1 --2- 172.21.15.43:0/2535141707 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f054006b4d0 0x7f054006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:43.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.931+0000 7f055cff9640 1 -- 172.21.15.43:0/2535141707 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f054808a510 con 0x7f05600d2370 2024-04-25T09:00:43.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.931+0000 7f055ffff640 1 --2- 172.21.15.43:0/2535141707 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f054006b4d0 0x7f054006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:43.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.932+0000 7f055ffff640 1 --2- 172.21.15.43:0/2535141707 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f054006b4d0 0x7f054006d990 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0550000f60 tx=0x7f0550009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:43.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:43.935+0000 7f055cff9640 1 -- 172.21.15.43:0/2535141707 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0548059390 con 0x7f05600d2370 2024-04-25T09:00:44.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.155+0000 7f0566563640 1 -- 172.21.15.43:0/2535141707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f05600b8380 con 0x7f05600d2370 2024-04-25T09:00:44.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.178+0000 7f055cff9640 1 -- 172.21.15.43:0/2535141707 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v53) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0548058c90 con 0x7f05600d2370 2024-04-25T09:00:44.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.180+0000 7f0566563640 1 -- 172.21.15.43:0/2535141707 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f054006b4d0 msgr2=0x7f054006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:44.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.180+0000 7f0566563640 1 --2- 172.21.15.43:0/2535141707 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f054006b4d0 0x7f054006d990 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f0550000f60 tx=0x7f0550009040 comp rx=0 tx=0).stop 2024-04-25T09:00:44.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.180+0000 7f0566563640 1 -- 172.21.15.43:0/2535141707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05600d2370 msgr2=0x7f05600cbff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:44.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.180+0000 7f0566563640 1 --2- 172.21.15.43:0/2535141707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05600d2370 0x7f05600cbff0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f054800ebe0 tx=0x7f054800c520 comp rx=0 tx=0).stop 2024-04-25T09:00:44.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.180+0000 7f0566563640 1 -- 172.21.15.43:0/2535141707 shutdown_connections 2024-04-25T09:00:44.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.180+0000 7f0566563640 1 --2- 172.21.15.43:0/2535141707 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f054006b4d0 0x7f054006d990 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:44.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.180+0000 7f0566563640 1 --2- 172.21.15.43:0/2535141707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0560161310 0x7f05600d0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:44.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.180+0000 7f0566563640 1 --2- 172.21.15.43:0/2535141707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05600d2370 0x7f05600cbff0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:44.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.180+0000 7f0566563640 1 --2- 172.21.15.43:0/2535141707 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f05600d19f0 0x7f05601f4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:44.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.180+0000 7f0566563640 1 -- 172.21.15.43:0/2535141707 >> 172.21.15.43:0/2535141707 conn(0x7f056009cd20 msgr2=0x7f056015f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:44.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.180+0000 7f0566563640 1 -- 172.21.15.43:0/2535141707 shutdown_connections 2024-04-25T09:00:44.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.180+0000 7f0566563640 1 -- 172.21.15.43:0/2535141707 wait complete. 2024-04-25T09:00:44.180 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T09:00:44.181 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:00:44.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.295+0000 7f87cbd6a640 1 Processor -- start 2024-04-25T09:00:44.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.295+0000 7f87cbd6a640 1 -- start start 2024-04-25T09:00:44.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.296+0000 7f87cbd6a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87c406f6a0 0x7f87c40b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:44.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.296+0000 7f87cbd6a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f87c40cd470 0x7f87c406c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:44.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.296+0000 7f87cbd6a640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87c40d2620 con 0x7f87c415ddd0 2024-04-25T09:00:44.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.296+0000 7f87cbd6a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87c40b0b80 con 0x7f87c40cd470 2024-04-25T09:00:44.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.296+0000 7f87cbd6a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87c40b0e60 con 0x7f87c406f6a0 2024-04-25T09:00:44.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.296+0000 7f87ca307640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f87c40cd470 0x7f87c406c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:44.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.296+0000 7f87ca307640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f87c40cd470 0x7f87c406c4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43678/0 (socket says 172.21.15.43:43678) 2024-04-25T09:00:44.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.296+0000 7f87ca307640 1 -- 172.21.15.43:0/3077385831 learned_addr learned my addr 172.21.15.43:0/3077385831 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:44.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.297+0000 7f87ca307640 1 -- 172.21.15.43:0/3077385831 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87c406f6a0 msgr2=0x7f87c40b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:44.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.297+0000 7f87c9305640 1 --2- 172.21.15.43:0/3077385831 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87c406f6a0 0x7f87c40b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:44.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.297+0000 7f87ca307640 1 --2- 172.21.15.43:0/3077385831 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87c406f6a0 0x7f87c40b3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:44.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.297+0000 7f87ca307640 1 -- 172.21.15.43:0/3077385831 >> v1:172.21.15.193:6789/0 conn(0x7f87c415ddd0 legacy=0x7f87c406ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:44.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.297+0000 7f87ca307640 1 -- 172.21.15.43:0/3077385831 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c40cbf40 con 0x7f87c40cd470 2024-04-25T09:00:44.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.297+0000 7f87ca307640 1 --2- 172.21.15.43:0/3077385831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87c40cd470 0x7f87c406c4e0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f87b4009e50 tx=0x7f87b402a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d28442d208702f21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:44.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.298+0000 7f87c8b04640 1 -- 172.21.15.43:0/3077385831 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b4036070 con 0x7f87c40cd470 2024-04-25T09:00:44.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.298+0000 7f87c8b04640 1 -- 172.21.15.43:0/3077385831 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b4031600 con 0x7f87c40cd470 2024-04-25T09:00:44.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.298+0000 7f87c8b04640 1 -- 172.21.15.43:0/3077385831 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b40319d0 con 0x7f87c40cd470 2024-04-25T09:00:44.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.298+0000 7f87cbd6a640 1 -- 172.21.15.43:0/3077385831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87c40cd470 msgr2=0x7f87c406c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:44.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.298+0000 7f87cbd6a640 1 --2- 172.21.15.43:0/3077385831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87c40cd470 0x7f87c406c4e0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f87b4009e50 tx=0x7f87b402a630 comp rx=0 tx=0).stop 2024-04-25T09:00:44.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.298+0000 7f87cbd6a640 1 -- 172.21.15.43:0/3077385831 shutdown_connections 2024-04-25T09:00:44.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.298+0000 7f87cbd6a640 1 --2- 172.21.15.43:0/3077385831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87c40cd470 0x7f87c406c4e0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:44.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.298+0000 7f87cbd6a640 1 --2- 172.21.15.43:0/3077385831 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87c406f6a0 0x7f87c40b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:44.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.298+0000 7f87cbd6a640 1 -- 172.21.15.43:0/3077385831 >> 172.21.15.43:0/3077385831 conn(0x7f87c40d17e0 msgr2=0x7f87c40d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:44.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.298+0000 7f87cbd6a640 1 -- 172.21.15.43:0/3077385831 shutdown_connections 2024-04-25T09:00:44.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.298+0000 7f87cbd6a640 1 -- 172.21.15.43:0/3077385831 wait complete. 2024-04-25T09:00:44.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.299+0000 7f87cbd6a640 1 Processor -- start 2024-04-25T09:00:44.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.300+0000 7f87cbd6a640 1 -- start start 2024-04-25T09:00:44.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.300+0000 7f87cbd6a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87c406f6a0 0x7f87c41f7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:44.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.300+0000 7f87cbd6a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87c40cd470 0x7f87c41f41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:44.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.300+0000 7f87cbd6a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87c415ddd0 0x7f87c4160a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:44.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.300+0000 7f87cbd6a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c406ce30 con 0x7f87c415ddd0 2024-04-25T09:00:44.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.300+0000 7f87cbd6a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87c406ccb0 con 0x7f87c40cd470 2024-04-25T09:00:44.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.300+0000 7f87cbd6a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87c406cfb0 con 0x7f87c406f6a0 2024-04-25T09:00:44.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.300+0000 7f87ca307640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87c415ddd0 0x7f87c4160a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:44.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.300+0000 7f87ca307640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87c415ddd0 0x7f87c4160a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43694/0 (socket says 172.21.15.43:43694) 2024-04-25T09:00:44.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.300+0000 7f87ca307640 1 -- 172.21.15.43:0/2280515581 learned_addr learned my addr 172.21.15.43:0/2280515581 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:44.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.301+0000 7f87c9305640 1 --2- 172.21.15.43:0/2280515581 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87c40cd470 0x7f87c41f41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:44.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.301+0000 7f87c9b06640 1 --2- 172.21.15.43:0/2280515581 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87c406f6a0 0x7f87c41f7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:44.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.301+0000 7f87ca307640 1 -- 172.21.15.43:0/2280515581 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87c406f6a0 msgr2=0x7f87c41f7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:44.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.301+0000 7f87ca307640 1 --2- 172.21.15.43:0/2280515581 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87c406f6a0 0x7f87c41f7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:44.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.301+0000 7f87ca307640 1 -- 172.21.15.43:0/2280515581 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87c40cd470 msgr2=0x7f87c41f41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:44.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.301+0000 7f87ca307640 1 --2- 172.21.15.43:0/2280515581 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87c40cd470 0x7f87c41f41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:44.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.301+0000 7f87ca307640 1 -- 172.21.15.43:0/2280515581 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87c41f83b0 con 0x7f87c415ddd0 2024-04-25T09:00:44.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.301+0000 7f87c9b06640 1 --2- 172.21.15.43:0/2280515581 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87c406f6a0 0x7f87c41f7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:44.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.301+0000 7f87c9305640 1 --2- 172.21.15.43:0/2280515581 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87c40cd470 0x7f87c41f41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:44.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.301+0000 7f87ca307640 1 --2- 172.21.15.43:0/2280515581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87c415ddd0 0x7f87c4160a20 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f87b4009e50 tx=0x7f87b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:44.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.301+0000 7f87c27fc640 1 -- 172.21.15.43:0/2280515581 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b4036050 con 0x7f87c415ddd0 2024-04-25T09:00:44.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.301+0000 7f87c27fc640 1 -- 172.21.15.43:0/2280515581 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87b4042070 con 0x7f87c415ddd0 2024-04-25T09:00:44.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.301+0000 7f87cbd6a640 1 -- 172.21.15.43:0/2280515581 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87c415e530 con 0x7f87c415ddd0 2024-04-25T09:00:44.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.301+0000 7f87c27fc640 1 -- 172.21.15.43:0/2280515581 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87b4009510 con 0x7f87c415ddd0 2024-04-25T09:00:44.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.301+0000 7f87cbd6a640 1 -- 172.21.15.43:0/2280515581 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87c4160220 con 0x7f87c415ddd0 2024-04-25T09:00:44.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.302+0000 7f87cbd6a640 1 -- 172.21.15.43:0/2280515581 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87c4074f60 con 0x7f87c415ddd0 2024-04-25T09:00:44.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.303+0000 7f87c27fc640 1 -- 172.21.15.43:0/2280515581 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f87b4007bf0 con 0x7f87c415ddd0 2024-04-25T09:00:44.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.303+0000 7f87c27fc640 1 --2- 172.21.15.43:0/2280515581 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f87b006b4d0 0x7f87b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:44.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.303+0000 7f87c27fc640 1 -- 172.21.15.43:0/2280515581 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f87b40ab0b0 con 0x7f87c415ddd0 2024-04-25T09:00:44.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.303+0000 7f87c9b06640 1 --2- 172.21.15.43:0/2280515581 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f87b006b4d0 0x7f87b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:44.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.304+0000 7f87c9b06640 1 --2- 172.21.15.43:0/2280515581 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f87b006b4d0 0x7f87b006d990 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f87c406e6a0 tx=0x7f87bc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:44.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.307+0000 7f87c27fc640 1 -- 172.21.15.43:0/2280515581 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f87b407a0e0 con 0x7f87c415ddd0 2024-04-25T09:00:44.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:44.529+0000 7f87cbd6a640 1 -- 172.21.15.43:0/2280515581 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f87c40cec80 con 0x7f87c415ddd0 2024-04-25T09:00:45.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.187+0000 7f87c27fc640 1 -- 172.21.15.43:0/2280515581 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v54) v1 ==== 122+0+0 (secure 0 0 0) 0x7f87b40799e0 con 0x7f87c415ddd0 2024-04-25T09:00:45.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.189+0000 7f87cbd6a640 1 -- 172.21.15.43:0/2280515581 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f87b006b4d0 msgr2=0x7f87b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:45.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.189+0000 7f87cbd6a640 1 --2- 172.21.15.43:0/2280515581 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f87b006b4d0 0x7f87b006d990 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f87c406e6a0 tx=0x7f87bc008040 comp rx=0 tx=0).stop 2024-04-25T09:00:45.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.189+0000 7f87cbd6a640 1 -- 172.21.15.43:0/2280515581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87c415ddd0 msgr2=0x7f87c4160a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:45.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.189+0000 7f87cbd6a640 1 --2- 172.21.15.43:0/2280515581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87c415ddd0 0x7f87c4160a20 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f87b4009e50 tx=0x7f87b4004a40 comp rx=0 tx=0).stop 2024-04-25T09:00:45.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.189+0000 7f87cbd6a640 1 -- 172.21.15.43:0/2280515581 shutdown_connections 2024-04-25T09:00:45.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.189+0000 7f87cbd6a640 1 --2- 172.21.15.43:0/2280515581 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87c415ddd0 0x7f87c4160a20 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:45.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.190+0000 7f87cbd6a640 1 --2- 172.21.15.43:0/2280515581 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f87b006b4d0 0x7f87b006d990 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:45.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.190+0000 7f87cbd6a640 1 --2- 172.21.15.43:0/2280515581 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87c40cd470 0x7f87c41f41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:45.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.190+0000 7f87cbd6a640 1 --2- 172.21.15.43:0/2280515581 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87c406f6a0 0x7f87c41f7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:45.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.190+0000 7f87cbd6a640 1 -- 172.21.15.43:0/2280515581 >> 172.21.15.43:0/2280515581 conn(0x7f87c40d17e0 msgr2=0x7f87c40aedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:45.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.190+0000 7f87cbd6a640 1 -- 172.21.15.43:0/2280515581 shutdown_connections 2024-04-25T09:00:45.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.190+0000 7f87cbd6a640 1 -- 172.21.15.43:0/2280515581 wait complete. 2024-04-25T09:00:45.194 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:00:45.194 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:00:45.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.310+0000 7fb5dd7e2640 1 Processor -- start 2024-04-25T09:00:45.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.310+0000 7fb5dd7e2640 1 -- start start 2024-04-25T09:00:45.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.310+0000 7fb5dd7e2640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5d80d0af0 0x7fb5d8160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:45.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.310+0000 7fb5dd7e2640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5d80d2920 con 0x7fb5d815ef50 2024-04-25T09:00:45.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.310+0000 7fb5dd7e2640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5d80cee80 con 0x7fb5d815dce0 2024-04-25T09:00:45.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.310+0000 7fb5dd7e2640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d80cec60 con 0x7fb5d80d0af0 2024-04-25T09:00:45.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.310+0000 7fb5d67fc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb5d815ef50 0x7fb5d80d03e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44292/0 (socket says 172.21.15.43:44292) 2024-04-25T09:00:45.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.311+0000 7fb5d67fc640 1 -- 172.21.15.43:0/80301695 learned_addr learned my addr 172.21.15.43:0/80301695 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:00:45.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.311+0000 7fb5d77fe640 1 --2- 172.21.15.43:0/80301695 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5d80d0af0 0x7fb5d8160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:45.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.311+0000 7fb5d5ffb640 1 -- 172.21.15.43:0/80301695 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2801634769 0 0) 0x7fb5d80d2920 con 0x7fb5d815ef50 2024-04-25T09:00:45.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.311+0000 7fb5d5ffb640 1 -- 172.21.15.43:0/80301695 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5c0003520 con 0x7fb5d815ef50 2024-04-25T09:00:45.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.311+0000 7fb5d5ffb640 1 -- 172.21.15.43:0/80301695 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3745342124 0 0) 0x7fb5d80cee80 con 0x7fb5d815dce0 2024-04-25T09:00:45.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.311+0000 7fb5d5ffb640 1 -- 172.21.15.43:0/80301695 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5d80d2920 con 0x7fb5d815dce0 2024-04-25T09:00:45.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.311+0000 7fb5d5ffb640 1 -- 172.21.15.43:0/80301695 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 856524330 0 0) 0x7fb5c0003520 con 0x7fb5d815ef50 2024-04-25T09:00:45.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.311+0000 7fb5d5ffb640 1 -- 172.21.15.43:0/80301695 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5d80d0af0 msgr2=0x7fb5d8160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:45.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.311+0000 7fb5d5ffb640 1 --2- 172.21.15.43:0/80301695 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5d80d0af0 0x7fb5d8160980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:45.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.311+0000 7fb5d5ffb640 1 -- 172.21.15.43:0/80301695 >> v1:172.21.15.193:6789/0 conn(0x7fb5d815dce0 legacy=0x7fb5d80ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:45.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.311+0000 7fb5d5ffb640 1 -- 172.21.15.43:0/80301695 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d80cf160 con 0x7fb5d815ef50 2024-04-25T09:00:45.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.311+0000 7fb5d5ffb640 1 -- 172.21.15.43:0/80301695 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7fb5cc001690 con 0x7fb5d815ef50 2024-04-25T09:00:45.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.311+0000 7fb5d5ffb640 1 -- 172.21.15.43:0/80301695 >> v1:172.21.15.43:6789/0 conn(0x7fb5d815ef50 legacy=0x7fb5d80d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:45.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.311+0000 7fb5d5ffb640 1 --2- 172.21.15.43:0/80301695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5c0003b70 0x7fb5c0003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:45.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.311+0000 7fb5d5ffb640 1 --2- 172.21.15.43:0/80301695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5c0004800 0x7fb5c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:45.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.311+0000 7fb5d5ffb640 1 --2- 172.21.15.43:0/80301695 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5d80d0af0 0x7fb5c00051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:45.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.311+0000 7fb5d5ffb640 1 -- 172.21.15.43:0/80301695 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d80cec60 con 0x7fb5c0004800 2024-04-25T09:00:45.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.311+0000 7fb5d5ffb640 1 -- 172.21.15.43:0/80301695 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c000bb00 con 0x7fb5c0003b70 2024-04-25T09:00:45.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.311+0000 7fb5d5ffb640 1 -- 172.21.15.43:0/80301695 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5c000bd00 con 0x7fb5d80d0af0 2024-04-25T09:00:45.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.311+0000 7fb5d67fc640 1 --2- 172.21.15.43:0/80301695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5c0004800 0x7fb5c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:45.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.311+0000 7fb5d6ffd640 1 --2- 172.21.15.43:0/80301695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5c0003b70 0x7fb5c0003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:45.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.311+0000 7fb5d77fe640 1 --2- 172.21.15.43:0/80301695 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5d80d0af0 0x7fb5c00051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:45.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.312+0000 7fb5d67fc640 1 -- 172.21.15.43:0/80301695 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5d80d0af0 msgr2=0x7fb5c00051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:45.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.312+0000 7fb5d67fc640 1 --2- 172.21.15.43:0/80301695 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5d80d0af0 0x7fb5c00051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:45.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.312+0000 7fb5d67fc640 1 -- 172.21.15.43:0/80301695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5c0003b70 msgr2=0x7fb5c0003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:45.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.312+0000 7fb5d67fc640 1 --2- 172.21.15.43:0/80301695 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5c0003b70 0x7fb5c0003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:45.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.312+0000 7fb5d67fc640 1 -- 172.21.15.43:0/80301695 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c0003520 con 0x7fb5c0004800 2024-04-25T09:00:45.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.312+0000 7fb5d77fe640 1 --2- 172.21.15.43:0/80301695 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5d80d0af0 0x7fb5c00051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:00:45.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.312+0000 7fb5d67fc640 1 --2- 172.21.15.43:0/80301695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5c0004800 0x7fb5c0004c80 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc00c210 tx=0x7fb5c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df84e21b1c20e437 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:45.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.312+0000 7fb5d5ffb640 1 -- 172.21.15.43:0/80301695 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc007ce0 con 0x7fb5c0004800 2024-04-25T09:00:45.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.312+0000 7fb5d5ffb640 1 -- 172.21.15.43:0/80301695 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5cc005af0 con 0x7fb5c0004800 2024-04-25T09:00:45.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.312+0000 7fb5d5ffb640 1 -- 172.21.15.43:0/80301695 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc007070 con 0x7fb5c0004800 2024-04-25T09:00:45.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.313+0000 7fb5dd7e2640 1 -- 172.21.15.43:0/80301695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5c0004800 msgr2=0x7fb5c0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:45.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.313+0000 7fb5dd7e2640 1 --2- 172.21.15.43:0/80301695 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5c0004800 0x7fb5c0004c80 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc00c210 tx=0x7fb5c0005830 comp rx=0 tx=0).stop 2024-04-25T09:00:45.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.313+0000 7fb5d77fe640 1 -- 172.21.15.43:0/80301695 reap_dead start 2024-04-25T09:00:45.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.313+0000 7fb5dd7e2640 1 -- 172.21.15.43:0/80301695 shutdown_connections 2024-04-25T09:00:45.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.313+0000 7fb5dd7e2640 1 -- 172.21.15.43:0/80301695 >> 172.21.15.43:0/80301695 conn(0x7fb5d809cd00 msgr2=0x7fb5d80d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:45.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.313+0000 7fb5dd7e2640 1 -- 172.21.15.43:0/80301695 shutdown_connections 2024-04-25T09:00:45.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.313+0000 7fb5dd7e2640 1 -- 172.21.15.43:0/80301695 wait complete. 2024-04-25T09:00:45.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.314+0000 7fb5dd7e2640 1 Processor -- start 2024-04-25T09:00:45.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.314+0000 7fb5dd7e2640 1 -- start start 2024-04-25T09:00:45.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.315+0000 7fb5dd7e2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5c0004800 0x7fb5d815c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:45.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.315+0000 7fb5dd7e2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5d80b12f0 0x7fb5d81f40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:45.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.315+0000 7fb5dd7e2640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5d81f4610 0x7fb5d80b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:45.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.315+0000 7fb5dd7e2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d815f6b0 con 0x7fb5d80b12f0 2024-04-25T09:00:45.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.315+0000 7fb5dd7e2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d80cf720 con 0x7fb5c0004800 2024-04-25T09:00:45.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.315+0000 7fb5dd7e2640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d815e0c0 con 0x7fb5d81f4610 2024-04-25T09:00:45.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.315+0000 7fb5d67fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5d80b12f0 0x7fb5d81f40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:45.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.315+0000 7fb5d67fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5d80b12f0 0x7fb5d81f40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43716/0 (socket says 172.21.15.43:43716) 2024-04-25T09:00:45.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.315+0000 7fb5d67fc640 1 -- 172.21.15.43:0/2164794896 learned_addr learned my addr 172.21.15.43:0/2164794896 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:45.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.315+0000 7fb5d6ffd640 1 --2- 172.21.15.43:0/2164794896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5c0004800 0x7fb5d815c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:45.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.315+0000 7fb5d77fe640 1 --2- 172.21.15.43:0/2164794896 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5d81f4610 0x7fb5d80b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:45.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.315+0000 7fb5d67fc640 1 -- 172.21.15.43:0/2164794896 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5d81f4610 msgr2=0x7fb5d80b27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:45.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.315+0000 7fb5d67fc640 1 --2- 172.21.15.43:0/2164794896 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5d81f4610 0x7fb5d80b27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:45.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.316+0000 7fb5d67fc640 1 -- 172.21.15.43:0/2164794896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5c0004800 msgr2=0x7fb5d815c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:45.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.316+0000 7fb5d67fc640 1 --2- 172.21.15.43:0/2164794896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5c0004800 0x7fb5d815c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:45.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.316+0000 7fb5d67fc640 1 -- 172.21.15.43:0/2164794896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d80cacf0 con 0x7fb5d80b12f0 2024-04-25T09:00:45.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.316+0000 7fb5d6ffd640 1 --2- 172.21.15.43:0/2164794896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5c0004800 0x7fb5d815c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:00:45.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.316+0000 7fb5d67fc640 1 --2- 172.21.15.43:0/2164794896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5d80b12f0 0x7fb5d81f40d0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc02ad80 tx=0x7fb5d80cfa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:45.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.316+0000 7fb5bf7fe640 1 -- 172.21.15.43:0/2164794896 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc041020 con 0x7fb5d80b12f0 2024-04-25T09:00:45.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.316+0000 7fb5bf7fe640 1 -- 172.21.15.43:0/2164794896 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5cc005930 con 0x7fb5d80b12f0 2024-04-25T09:00:45.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.316+0000 7fb5dd7e2640 1 -- 172.21.15.43:0/2164794896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5d80cafd0 con 0x7fb5d80b12f0 2024-04-25T09:00:45.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.316+0000 7fb5bf7fe640 1 -- 172.21.15.43:0/2164794896 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc007070 con 0x7fb5d80b12f0 2024-04-25T09:00:45.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.316+0000 7fb5dd7e2640 1 -- 172.21.15.43:0/2164794896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5d80cd940 con 0x7fb5d80b12f0 2024-04-25T09:00:45.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.317+0000 7fb5dd7e2640 1 -- 172.21.15.43:0/2164794896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5d80caa80 con 0x7fb5d80b12f0 2024-04-25T09:00:45.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.317+0000 7fb5bf7fe640 1 -- 172.21.15.43:0/2164794896 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb5cc044020 con 0x7fb5d80b12f0 2024-04-25T09:00:45.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.318+0000 7fb5bf7fe640 1 --2- 172.21.15.43:0/2164794896 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb5c406b500 0x7fb5c406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:45.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.318+0000 7fb5bf7fe640 1 -- 172.21.15.43:0/2164794896 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb5cc0aa4e0 con 0x7fb5d80b12f0 2024-04-25T09:00:45.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.318+0000 7fb5d6ffd640 1 --2- 172.21.15.43:0/2164794896 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb5c406b500 0x7fb5c406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:45.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.319+0000 7fb5d6ffd640 1 --2- 172.21.15.43:0/2164794896 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb5c406b500 0x7fb5c406d9c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0003f10 tx=0x7fb5d0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:45.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.322+0000 7fb5bf7fe640 1 -- 172.21.15.43:0/2164794896 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb5cc079360 con 0x7fb5d80b12f0 2024-04-25T09:00:45.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.539+0000 7fb5dd7e2640 1 -- 172.21.15.43:0/2164794896 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb5d80b9770 con 0x7fb5d80b12f0 2024-04-25T09:00:45.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.540+0000 7fb5bf7fe640 1 -- 172.21.15.43:0/2164794896 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fb5d80b9770 con 0x7fb5d80b12f0 2024-04-25T09:00:45.529 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:00:45.529 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":54,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6200,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2258031526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2258031526},{"type":"v1","addr":"172.21.15.43:6834","nonce":2258031526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6206,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/1773440177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1773440177},{"type":"v1","addr":"172.21.15.193:6836","nonce":1773440177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6209,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3239725307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":3239725307},{"type":"v1","addr":"172.21.15.193:6837","nonce":3239725307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:00:42.536578+0000","modified":"2024-04-25T09:00:44.531104+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7596},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7596":{"gid":7596,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/1341028696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1341028696},{"type":"v1","addr":"172.21.15.43:6835","nonce":1341028696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:00:45.529 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 54 2024-04-25T09:00:45.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.542+0000 7fb5dd7e2640 1 -- 172.21.15.43:0/2164794896 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb5c406b500 msgr2=0x7fb5c406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:45.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.542+0000 7fb5dd7e2640 1 --2- 172.21.15.43:0/2164794896 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb5c406b500 0x7fb5c406d9c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0003f10 tx=0x7fb5d0009040 comp rx=0 tx=0).stop 2024-04-25T09:00:45.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.542+0000 7fb5dd7e2640 1 -- 172.21.15.43:0/2164794896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5d80b12f0 msgr2=0x7fb5d81f40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:45.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.542+0000 7fb5dd7e2640 1 --2- 172.21.15.43:0/2164794896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5d80b12f0 0x7fb5d81f40d0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc02ad80 tx=0x7fb5d80cfa00 comp rx=0 tx=0).stop 2024-04-25T09:00:45.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.542+0000 7fb5dd7e2640 1 -- 172.21.15.43:0/2164794896 shutdown_connections 2024-04-25T09:00:45.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.542+0000 7fb5dd7e2640 1 --2- 172.21.15.43:0/2164794896 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5d81f4610 0x7fb5d80b27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:45.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.542+0000 7fb5dd7e2640 1 --2- 172.21.15.43:0/2164794896 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5d80b12f0 0x7fb5d81f40d0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:45.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.542+0000 7fb5dd7e2640 1 --2- 172.21.15.43:0/2164794896 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb5c406b500 0x7fb5c406d9c0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:45.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.542+0000 7fb5dd7e2640 1 --2- 172.21.15.43:0/2164794896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5c0004800 0x7fb5d815c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:45.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.542+0000 7fb5dd7e2640 1 -- 172.21.15.43:0/2164794896 >> 172.21.15.43:0/2164794896 conn(0x7fb5d809cd00 msgr2=0x7fb5d80d1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:45.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.542+0000 7fb5dd7e2640 1 -- 172.21.15.43:0/2164794896 shutdown_connections 2024-04-25T09:00:45.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.542+0000 7fb5dd7e2640 1 -- 172.21.15.43:0/2164794896 wait complete. 2024-04-25T09:00:45.542 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:00:45.543 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:00:45.585 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:00:45.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.697+0000 7fcf6b009640 1 Processor -- start 2024-04-25T09:00:45.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.697+0000 7fcf6b009640 1 -- start start 2024-04-25T09:00:45.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.698+0000 7fcf6b009640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcf6415dcf0 0x7fcf6406e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:45.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.698+0000 7fcf6b009640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcf6406d4f0 0x7fcf6406f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:45.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.698+0000 7fcf6b009640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf640d0d80 con 0x7fcf6406ebf0 2024-04-25T09:00:45.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.698+0000 7fcf6b009640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf6406f590 con 0x7fcf6415dcf0 2024-04-25T09:00:45.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.698+0000 7fcf6b009640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6406c050 con 0x7fcf6406d4f0 2024-04-25T09:00:45.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.698+0000 7fcf68da5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcf6415dcf0 0x7fcf6406e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:45.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.698+0000 7fcf63fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fcf6406ebf0 0x7fcf641616e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44294/0 (socket says 172.21.15.43:44294) 2024-04-25T09:00:45.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.698+0000 7fcf63fff640 1 -- 172.21.15.43:0/1013390256 learned_addr learned my addr 172.21.15.43:0/1013390256 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:00:45.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.698+0000 7fcf68da5640 1 -- 172.21.15.43:0/1013390256 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcf6406d4f0 msgr2=0x7fcf6406f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:45.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.698+0000 7fcf68da5640 1 --2- 172.21.15.43:0/1013390256 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcf6406d4f0 0x7fcf6406f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:45.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.698+0000 7fcf68da5640 1 -- 172.21.15.43:0/1013390256 >> v1:172.21.15.43:6789/0 conn(0x7fcf6406ebf0 legacy=0x7fcf641616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:45.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.698+0000 7fcf68da5640 1 -- 172.21.15.43:0/1013390256 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf640d04e0 con 0x7fcf6415dcf0 2024-04-25T09:00:45.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.698+0000 7fcf637fe640 1 -- 172.21.15.43:0/1013390256 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2609031344 0 0) 0x7fcf640d0d80 con 0x7fcf6406ebf0 2024-04-25T09:00:45.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.699+0000 7fcf68da5640 1 --2- 172.21.15.43:0/1013390256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf6415dcf0 0x7fcf6406e530 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fcf580097a0 tx=0x7fcf5802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8a7378551c1d5a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:45.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.699+0000 7fcf637fe640 1 -- 172.21.15.43:0/1013390256 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf58037070 con 0x7fcf6415dcf0 2024-04-25T09:00:45.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.699+0000 7fcf637fe640 1 -- 172.21.15.43:0/1013390256 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf5802ae50 con 0x7fcf6415dcf0 2024-04-25T09:00:45.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.699+0000 7fcf637fe640 1 -- 172.21.15.43:0/1013390256 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf58032660 con 0x7fcf6415dcf0 2024-04-25T09:00:45.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.700+0000 7fcf6b009640 1 -- 172.21.15.43:0/1013390256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf6415dcf0 msgr2=0x7fcf6406e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:45.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.700+0000 7fcf6b009640 1 --2- 172.21.15.43:0/1013390256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf6415dcf0 0x7fcf6406e530 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fcf580097a0 tx=0x7fcf5802a100 comp rx=0 tx=0).stop 2024-04-25T09:00:45.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.700+0000 7fcf6b009640 1 -- 172.21.15.43:0/1013390256 shutdown_connections 2024-04-25T09:00:45.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.700+0000 7fcf6b009640 1 --2- 172.21.15.43:0/1013390256 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcf6406d4f0 0x7fcf6406f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:45.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.700+0000 7fcf6b009640 1 --2- 172.21.15.43:0/1013390256 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf6415dcf0 0x7fcf6406e530 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:45.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.700+0000 7fcf6b009640 1 -- 172.21.15.43:0/1013390256 >> 172.21.15.43:0/1013390256 conn(0x7fcf640d1700 msgr2=0x7fcf640d1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:45.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.700+0000 7fcf6b009640 1 -- 172.21.15.43:0/1013390256 shutdown_connections 2024-04-25T09:00:45.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.700+0000 7fcf6b009640 1 -- 172.21.15.43:0/1013390256 wait complete. 2024-04-25T09:00:45.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.701+0000 7fcf6b009640 1 Processor -- start 2024-04-25T09:00:45.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.701+0000 7fcf6b009640 1 -- start start 2024-04-25T09:00:45.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.702+0000 7fcf6b009640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf6406d4f0 0x7fcf6415ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:45.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.702+0000 7fcf6b009640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf6406ebf0 0x7fcf64156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:45.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.702+0000 7fcf6b009640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcf6415dcf0 0x7fcf640c7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:45.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.702+0000 7fcf6b009640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6406bce0 con 0x7fcf6406ebf0 2024-04-25T09:00:45.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.702+0000 7fcf6b009640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf640d0aa0 con 0x7fcf6406d4f0 2024-04-25T09:00:45.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.702+0000 7fcf6b009640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf640b0d50 con 0x7fcf6415dcf0 2024-04-25T09:00:45.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.702+0000 7fcf63fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf6406ebf0 0x7fcf64156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:45.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.702+0000 7fcf63fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf6406ebf0 0x7fcf64156350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43748/0 (socket says 172.21.15.43:43748) 2024-04-25T09:00:45.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.702+0000 7fcf63fff640 1 -- 172.21.15.43:0/253451150 learned_addr learned my addr 172.21.15.43:0/253451150 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:45.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.702+0000 7fcf68da5640 1 --2- 172.21.15.43:0/253451150 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf6406d4f0 0x7fcf6415ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:45.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.702+0000 7fcf695a6640 1 --2- 172.21.15.43:0/253451150 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcf6415dcf0 0x7fcf640c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:45.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.702+0000 7fcf63fff640 1 -- 172.21.15.43:0/253451150 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcf6415dcf0 msgr2=0x7fcf640c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:45.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.702+0000 7fcf63fff640 1 --2- 172.21.15.43:0/253451150 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcf6415dcf0 0x7fcf640c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:45.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.702+0000 7fcf63fff640 1 -- 172.21.15.43:0/253451150 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf6406d4f0 msgr2=0x7fcf6415ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:45.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.702+0000 7fcf63fff640 1 --2- 172.21.15.43:0/253451150 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf6406d4f0 0x7fcf6415ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:45.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.702+0000 7fcf63fff640 1 -- 172.21.15.43:0/253451150 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf6415a090 con 0x7fcf6406ebf0 2024-04-25T09:00:45.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.702+0000 7fcf68da5640 1 --2- 172.21.15.43:0/253451150 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf6406d4f0 0x7fcf6415ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:00:45.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.703+0000 7fcf63fff640 1 --2- 172.21.15.43:0/253451150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf6406ebf0 0x7fcf64156350 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000c4d0 tx=0x7fcf5000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:45.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.703+0000 7fcf617fa640 1 -- 172.21.15.43:0/253451150 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf50018070 con 0x7fcf6406ebf0 2024-04-25T09:00:45.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.703+0000 7fcf617fa640 1 -- 172.21.15.43:0/253451150 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf50007500 con 0x7fcf6406ebf0 2024-04-25T09:00:45.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.703+0000 7fcf6b009640 1 -- 172.21.15.43:0/253451150 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf6415a370 con 0x7fcf6406ebf0 2024-04-25T09:00:45.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.703+0000 7fcf617fa640 1 -- 172.21.15.43:0/253451150 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf50004a20 con 0x7fcf6406ebf0 2024-04-25T09:00:45.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.703+0000 7fcf6b009640 1 -- 172.21.15.43:0/253451150 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf6415a930 con 0x7fcf6406ebf0 2024-04-25T09:00:45.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.704+0000 7fcf6b009640 1 -- 172.21.15.43:0/253451150 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf64074ef0 con 0x7fcf6406ebf0 2024-04-25T09:00:45.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.704+0000 7fcf617fa640 1 -- 172.21.15.43:0/253451150 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcf5001b070 con 0x7fcf6406ebf0 2024-04-25T09:00:45.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.705+0000 7fcf617fa640 1 --2- 172.21.15.43:0/253451150 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcf4806b500 0x7fcf4806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:45.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.705+0000 7fcf617fa640 1 -- 172.21.15.43:0/253451150 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcf500894e0 con 0x7fcf6406ebf0 2024-04-25T09:00:45.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.705+0000 7fcf68da5640 1 --2- 172.21.15.43:0/253451150 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcf4806b500 0x7fcf4806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:45.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.706+0000 7fcf68da5640 1 --2- 172.21.15.43:0/253451150 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcf4806b500 0x7fcf4806d9c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fcf58034000 tx=0x7fcf5802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:45.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.709+0000 7fcf617fa640 1 -- 172.21.15.43:0/253451150 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcf50058360 con 0x7fcf6406ebf0 2024-04-25T09:00:45.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.890+0000 7fcf6b009640 1 -- 172.21.15.43:0/253451150 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcf640ae940 con 0x7fcf6406ebf0 2024-04-25T09:00:45.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.892+0000 7fcf617fa640 1 -- 172.21.15.43:0/253451150 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+16624 (secure 0 0 0) 0x7fcf50057c60 con 0x7fcf6406ebf0 2024-04-25T09:00:45.881 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:00:45.881 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":89,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:00:44.170942+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:00:40.816230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T09:00:41.508021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:00:45.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.894+0000 7fcf6b009640 1 -- 172.21.15.43:0/253451150 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcf4806b500 msgr2=0x7fcf4806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:45.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.894+0000 7fcf6b009640 1 --2- 172.21.15.43:0/253451150 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcf4806b500 0x7fcf4806d9c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fcf58034000 tx=0x7fcf5802e000 comp rx=0 tx=0).stop 2024-04-25T09:00:45.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.894+0000 7fcf6b009640 1 -- 172.21.15.43:0/253451150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf6406ebf0 msgr2=0x7fcf64156350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:45.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.894+0000 7fcf6b009640 1 --2- 172.21.15.43:0/253451150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf6406ebf0 0x7fcf64156350 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fcf5000c4d0 tx=0x7fcf5000c990 comp rx=0 tx=0).stop 2024-04-25T09:00:45.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.894+0000 7fcf6b009640 1 -- 172.21.15.43:0/253451150 shutdown_connections 2024-04-25T09:00:45.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.894+0000 7fcf6b009640 1 --2- 172.21.15.43:0/253451150 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcf6415dcf0 0x7fcf640c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:45.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.894+0000 7fcf6b009640 1 --2- 172.21.15.43:0/253451150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcf6406ebf0 0x7fcf64156350 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:45.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.894+0000 7fcf6b009640 1 --2- 172.21.15.43:0/253451150 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcf4806b500 0x7fcf4806d9c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:45.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.894+0000 7fcf6b009640 1 --2- 172.21.15.43:0/253451150 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcf6406d4f0 0x7fcf6415ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:45.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.894+0000 7fcf6b009640 1 -- 172.21.15.43:0/253451150 >> 172.21.15.43:0/253451150 conn(0x7fcf640d1700 msgr2=0x7fcf6415cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:45.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.894+0000 7fcf6b009640 1 -- 172.21.15.43:0/253451150 shutdown_connections 2024-04-25T09:00:45.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:45.894+0000 7fcf6b009640 1 -- 172.21.15.43:0/253451150 wait complete. 2024-04-25T09:00:45.895 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:00:45.895 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:00:45.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.007+0000 7f8c0e748640 1 Processor -- start 2024-04-25T09:00:45.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.007+0000 7f8c0e748640 1 -- start start 2024-04-25T09:00:45.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.008+0000 7f8c0e748640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f8c0815dce0 0x7f8c0815ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:45.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.008+0000 7f8c0e748640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8c080d0af0 0x7f8c08160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:45.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.008+0000 7f8c0e748640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c080cee80 con 0x7f8c080ce550 2024-04-25T09:00:45.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.008+0000 7f8c0e748640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c080d2920 con 0x7f8c080d0af0 2024-04-25T09:00:45.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.008+0000 7f8c0e748640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c080cf160 con 0x7f8c0815dce0 2024-04-25T09:00:45.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.008+0000 7f8c0cce5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8c080d0af0 0x7f8c08160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:45.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.008+0000 7f8c077fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8c080ce550 0x7f8c080d03e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44304/0 (socket says 172.21.15.43:44304) 2024-04-25T09:00:45.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.008+0000 7f8c077fe640 1 -- 172.21.15.43:0/3926927631 learned_addr learned my addr 172.21.15.43:0/3926927631 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:00:45.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.008+0000 7f8c07fff640 1 --2- 172.21.15.43:0/3926927631 >> v2:172.21.15.193:3300/0 conn(0x7f8c0815dce0 0x7f8c0815ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:45.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.008+0000 7f8c0cce5640 1 -- 172.21.15.43:0/3926927631 >> v2:172.21.15.193:3300/0 conn(0x7f8c0815dce0 msgr2=0x7f8c0815ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:45.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.008+0000 7f8c0cce5640 1 --2- 172.21.15.43:0/3926927631 >> v2:172.21.15.193:3300/0 conn(0x7f8c0815dce0 0x7f8c0815ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:45.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.008+0000 7f8c0cce5640 1 -- 172.21.15.43:0/3926927631 >> v1:172.21.15.43:6789/0 conn(0x7f8c080ce550 legacy=0x7f8c080d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:45.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.008+0000 7f8c06ffd640 1 -- 172.21.15.43:0/3926927631 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1871704648 0 0) 0x7f8c080cee80 con 0x7f8c080ce550 2024-04-25T09:00:45.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.008+0000 7f8c0cce5640 1 -- 172.21.15.43:0/3926927631 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c080cf440 con 0x7f8c080d0af0 2024-04-25T09:00:45.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.009+0000 7f8c0cce5640 1 --2- 172.21.15.43:0/3926927631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c080d0af0 0x7f8c08160980 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f8c0809d3c0 tx=0x7f8c0002a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f96ab3f1b5badb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:46.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.009+0000 7f8c06ffd640 1 -- 172.21.15.43:0/3926927631 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00007740 con 0x7f8c080d0af0 2024-04-25T09:00:46.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.009+0000 7f8c06ffd640 1 -- 172.21.15.43:0/3926927631 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c000078e0 con 0x7f8c080d0af0 2024-04-25T09:00:46.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.009+0000 7f8c06ffd640 1 -- 172.21.15.43:0/3926927631 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00007cb0 con 0x7f8c080d0af0 2024-04-25T09:00:46.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.010+0000 7f8c0e748640 1 -- 172.21.15.43:0/3926927631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c080d0af0 msgr2=0x7f8c08160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.010+0000 7f8c0e748640 1 --2- 172.21.15.43:0/3926927631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c080d0af0 0x7f8c08160980 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f8c0809d3c0 tx=0x7f8c0002a5f0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.010+0000 7f8c0e748640 1 -- 172.21.15.43:0/3926927631 shutdown_connections 2024-04-25T09:00:46.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.010+0000 7f8c0e748640 1 --2- 172.21.15.43:0/3926927631 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c080d0af0 0x7f8c08160980 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.010+0000 7f8c0e748640 1 --2- 172.21.15.43:0/3926927631 >> v2:172.21.15.193:3300/0 conn(0x7f8c0815dce0 0x7f8c0815ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.010+0000 7f8c0e748640 1 -- 172.21.15.43:0/3926927631 >> 172.21.15.43:0/3926927631 conn(0x7f8c0809cd00 msgr2=0x7f8c080d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:46.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.010+0000 7f8c0e748640 1 -- 172.21.15.43:0/3926927631 shutdown_connections 2024-04-25T09:00:46.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.010+0000 7f8c0e748640 1 -- 172.21.15.43:0/3926927631 wait complete. 2024-04-25T09:00:46.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.011+0000 7f8c0e748640 1 Processor -- start 2024-04-25T09:00:46.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.011+0000 7f8c0e748640 1 -- start start 2024-04-25T09:00:46.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.012+0000 7f8c0e748640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c080ce550 0x7f8c080caf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.012+0000 7f8c0e748640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c080d0af0 0x7f8c080cd910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.012+0000 7f8c0e748640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c0815dce0 0x7f8c08154290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.012+0000 7f8c0e748640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0815e0c0 con 0x7f8c0815dce0 2024-04-25T09:00:46.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.012+0000 7f8c0e748640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0815f6b0 con 0x7f8c080d0af0 2024-04-25T09:00:46.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.012+0000 7f8c0e748640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c080ce9c0 con 0x7f8c080ce550 2024-04-25T09:00:46.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.012+0000 7f8c0cce5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c0815dce0 0x7f8c08154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.012+0000 7f8c0cce5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c0815dce0 0x7f8c08154290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43766/0 (socket says 172.21.15.43:43766) 2024-04-25T09:00:46.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.012+0000 7f8c0cce5640 1 -- 172.21.15.43:0/760886311 learned_addr learned my addr 172.21.15.43:0/760886311 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:46.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.012+0000 7f8c077fe640 1 --2- 172.21.15.43:0/760886311 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c080d0af0 0x7f8c080cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.012+0000 7f8c07fff640 1 --2- 172.21.15.43:0/760886311 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c080ce550 0x7f8c080caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.012+0000 7f8c0cce5640 1 -- 172.21.15.43:0/760886311 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c080ce550 msgr2=0x7f8c080caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.012+0000 7f8c0cce5640 1 --2- 172.21.15.43:0/760886311 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c080ce550 0x7f8c080caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.012+0000 7f8c0cce5640 1 -- 172.21.15.43:0/760886311 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c080d0af0 msgr2=0x7f8c080cd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.012+0000 7f8c0cce5640 1 --2- 172.21.15.43:0/760886311 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c080d0af0 0x7f8c080cd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.012+0000 7f8c0cce5640 1 -- 172.21.15.43:0/760886311 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c080cde50 con 0x7f8c0815dce0 2024-04-25T09:00:46.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.012+0000 7f8c07fff640 1 --2- 172.21.15.43:0/760886311 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c080ce550 0x7f8c080caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:46.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.013+0000 7f8c0cce5640 1 --2- 172.21.15.43:0/760886311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c0815dce0 0x7f8c08154290 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f8c0002abe0 tx=0x7f8c00004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:46.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.013+0000 7f8c04ff9640 1 -- 172.21.15.43:0/760886311 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00040070 con 0x7f8c0815dce0 2024-04-25T09:00:46.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.013+0000 7f8c04ff9640 1 -- 172.21.15.43:0/760886311 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c00031bc0 con 0x7f8c0815dce0 2024-04-25T09:00:46.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.013+0000 7f8c0e748640 1 -- 172.21.15.43:0/760886311 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c0815a2b0 con 0x7f8c0815dce0 2024-04-25T09:00:46.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.013+0000 7f8c04ff9640 1 -- 172.21.15.43:0/760886311 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00045460 con 0x7f8c0815dce0 2024-04-25T09:00:46.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.013+0000 7f8c0e748640 1 -- 172.21.15.43:0/760886311 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c08156220 con 0x7f8c0815dce0 2024-04-25T09:00:46.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.014+0000 7f8c0e748640 1 -- 172.21.15.43:0/760886311 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c080680f0 con 0x7f8c0815dce0 2024-04-25T09:00:46.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.014+0000 7f8c04ff9640 1 -- 172.21.15.43:0/760886311 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8c00039040 con 0x7f8c0815dce0 2024-04-25T09:00:46.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.015+0000 7f8c04ff9640 1 --2- 172.21.15.43:0/760886311 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8be806b500 0x7f8be806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.015+0000 7f8c04ff9640 1 -- 172.21.15.43:0/760886311 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8c000abb00 con 0x7f8c0815dce0 2024-04-25T09:00:46.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.015+0000 7f8c07fff640 1 --2- 172.21.15.43:0/760886311 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8be806b500 0x7f8be806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.016+0000 7f8c07fff640 1 --2- 172.21.15.43:0/760886311 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8be806b500 0x7f8be806d9c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0006fd0 tx=0x7f8bf0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:46.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.019+0000 7f8c04ff9640 1 -- 172.21.15.43:0/760886311 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8c0007aaa0 con 0x7f8c0815dce0 2024-04-25T09:00:46.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.195+0000 7f8c0e748640 1 -- 172.21.15.43:0/760886311 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8c08074ef0 con 0x7f8be806b500 2024-04-25T09:00:46.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.202+0000 7f8c04ff9640 1 -- 172.21.15.43:0/760886311 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252016 (secure 0 0 0) 0x7f8c08074ef0 con 0x7f8be806b500 2024-04-25T09:00:46.193 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:00:46.195 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-25T09:00:46.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.204+0000 7f8c0e748640 1 -- 172.21.15.43:0/760886311 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8be806b500 msgr2=0x7f8be806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.204+0000 7f8c0e748640 1 --2- 172.21.15.43:0/760886311 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8be806b500 0x7f8be806d9c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0006fd0 tx=0x7f8bf0008040 comp rx=0 tx=0).stop 2024-04-25T09:00:46.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.204+0000 7f8c0e748640 1 -- 172.21.15.43:0/760886311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c0815dce0 msgr2=0x7f8c08154290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.204+0000 7f8c0e748640 1 --2- 172.21.15.43:0/760886311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c0815dce0 0x7f8c08154290 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f8c0002abe0 tx=0x7f8c00004a40 comp rx=0 tx=0).stop 2024-04-25T09:00:46.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.204+0000 7f8c0e748640 1 -- 172.21.15.43:0/760886311 shutdown_connections 2024-04-25T09:00:46.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.204+0000 7f8c0e748640 1 --2- 172.21.15.43:0/760886311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c0815dce0 0x7f8c08154290 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.204+0000 7f8c0e748640 1 --2- 172.21.15.43:0/760886311 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8be806b500 0x7f8be806d9c0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.204+0000 7f8c0e748640 1 --2- 172.21.15.43:0/760886311 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c080d0af0 0x7f8c080cd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.204+0000 7f8c0e748640 1 --2- 172.21.15.43:0/760886311 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c080ce550 0x7f8c080caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.204+0000 7f8c0e748640 1 -- 172.21.15.43:0/760886311 >> 172.21.15.43:0/760886311 conn(0x7f8c0809cd00 msgr2=0x7f8c080cf960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:46.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.205+0000 7f8c0e748640 1 -- 172.21.15.43:0/760886311 shutdown_connections 2024-04-25T09:00:46.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.205+0000 7f8c0e748640 1 -- 172.21.15.43:0/760886311 wait complete. 2024-04-25T09:00:46.205 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":286,"stamp":"2024-04-25T09:00:45.784568+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":76,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":517280,"kb_used_data":9620,"kb_used_omap":12,"kb_used_meta":507507,"kb_avail":754457440,"statfs":{"total":773094113280,"available":772564418560,"internally_reserved":0,"allocated":9850880,"data_stored":4891219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":519687729},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.625079"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.178273+0000","last_change":"2024-04-25T09:00:43.175387+0000","last_active":"2024-04-25T09:00:43.178273+0000","last_peered":"2024-04-25T09:00:43.178273+0000","last_clean":"2024-04-25T09:00:43.178273+0000","last_became_active":"2024-04-25T09:00:43.175117+0000","last_became_peered":"2024-04-25T09:00:43.175117+0000","last_unstale":"2024-04-25T09:00:43.178273+0000","last_undegraded":"2024-04-25T09:00:43.178273+0000","last_fullsized":"2024-04-25T09:00:43.178273+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:38.416326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.192799+0000","last_change":"2024-04-25T09:00:43.190723+0000","last_active":"2024-04-25T09:00:43.192799+0000","last_peered":"2024-04-25T09:00:43.192799+0000","last_clean":"2024-04-25T09:00:43.192799+0000","last_became_active":"2024-04-25T09:00:43.189137+0000","last_became_peered":"2024-04-25T09:00:43.189137+0000","last_unstale":"2024-04-25T09:00:43.192799+0000","last_undegraded":"2024-04-25T09:00:43.192799+0000","last_fullsized":"2024-04-25T09:00:43.192799+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:20.550630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.183283+0000","last_change":"2024-04-25T09:00:43.182278+0000","last_active":"2024-04-25T09:00:43.183283+0000","last_peered":"2024-04-25T09:00:43.183283+0000","last_clean":"2024-04-25T09:00:43.183283+0000","last_became_active":"2024-04-25T09:00:43.182005+0000","last_became_peered":"2024-04-25T09:00:43.182005+0000","last_unstale":"2024-04-25T09:00:43.183283+0000","last_undegraded":"2024-04-25T09:00:43.183283+0000","last_fullsized":"2024-04-25T09:00:43.183283+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:09.550708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.189288+0000","last_change":"2024-04-25T09:00:43.187228+0000","last_active":"2024-04-25T09:00:43.189288+0000","last_peered":"2024-04-25T09:00:43.189288+0000","last_clean":"2024-04-25T09:00:43.189288+0000","last_became_active":"2024-04-25T09:00:43.186942+0000","last_became_peered":"2024-04-25T09:00:43.186942+0000","last_unstale":"2024-04-25T09:00:43.189288+0000","last_undegraded":"2024-04-25T09:00:43.189288+0000","last_fullsized":"2024-04-25T09:00:43.189288+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:43.718412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.194341+0000","last_change":"2024-04-25T09:00:43.193885+0000","last_active":"2024-04-25T09:00:43.194341+0000","last_peered":"2024-04-25T09:00:43.194341+0000","last_clean":"2024-04-25T09:00:43.194341+0000","last_became_active":"2024-04-25T09:00:43.193578+0000","last_became_peered":"2024-04-25T09:00:43.193578+0000","last_unstale":"2024-04-25T09:00:43.194341+0000","last_undegraded":"2024-04-25T09:00:43.194341+0000","last_fullsized":"2024-04-25T09:00:43.194341+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:14.712980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.181417+0000","last_change":"2024-04-25T09:00:43.180105+0000","last_active":"2024-04-25T09:00:43.181417+0000","last_peered":"2024-04-25T09:00:43.181417+0000","last_clean":"2024-04-25T09:00:43.181417+0000","last_became_active":"2024-04-25T09:00:43.179082+0000","last_became_peered":"2024-04-25T09:00:43.179082+0000","last_unstale":"2024-04-25T09:00:43.181417+0000","last_undegraded":"2024-04-25T09:00:43.181417+0000","last_fullsized":"2024-04-25T09:00:43.181417+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:25.325832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.180054+0000","last_change":"2024-04-25T09:00:43.178271+0000","last_active":"2024-04-25T09:00:43.180054+0000","last_peered":"2024-04-25T09:00:43.180054+0000","last_clean":"2024-04-25T09:00:43.180054+0000","last_became_active":"2024-04-25T09:00:43.177965+0000","last_became_peered":"2024-04-25T09:00:43.177965+0000","last_unstale":"2024-04-25T09:00:43.180054+0000","last_undegraded":"2024-04-25T09:00:43.180054+0000","last_fullsized":"2024-04-25T09:00:43.180054+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:18.452112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.187838+0000","last_change":"2024-04-25T09:00:43.186910+0000","last_active":"2024-04-25T09:00:43.187838+0000","last_peered":"2024-04-25T09:00:43.187838+0000","last_clean":"2024-04-25T09:00:43.187838+0000","last_became_active":"2024-04-25T09:00:43.186430+0000","last_became_peered":"2024-04-25T09:00:43.186430+0000","last_unstale":"2024-04-25T09:00:43.187838+0000","last_undegraded":"2024-04-25T09:00:43.187838+0000","last_fullsized":"2024-04-25T09:00:43.187838+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:12.523164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.182690+0000","last_change":"2024-04-25T09:00:43.181731+0000","last_active":"2024-04-25T09:00:43.182690+0000","last_peered":"2024-04-25T09:00:43.182690+0000","last_clean":"2024-04-25T09:00:43.182690+0000","last_became_active":"2024-04-25T09:00:43.181437+0000","last_became_peered":"2024-04-25T09:00:43.181437+0000","last_unstale":"2024-04-25T09:00:43.182690+0000","last_undegraded":"2024-04-25T09:00:43.182690+0000","last_fullsized":"2024-04-25T09:00:43.182690+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:14.712980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.181660+0000","last_change":"2024-04-25T09:00:43.178468+0000","last_active":"2024-04-25T09:00:43.181660+0000","last_peered":"2024-04-25T09:00:43.181660+0000","last_clean":"2024-04-25T09:00:43.181660+0000","last_became_active":"2024-04-25T09:00:43.178113+0000","last_became_peered":"2024-04-25T09:00:43.178113+0000","last_unstale":"2024-04-25T09:00:43.181660+0000","last_undegraded":"2024-04-25T09:00:43.181660+0000","last_fullsized":"2024-04-25T09:00:43.181660+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:25.325832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.193105+0000","last_change":"2024-04-25T09:00:43.192022+0000","last_active":"2024-04-25T09:00:43.193105+0000","last_peered":"2024-04-25T09:00:43.193105+0000","last_clean":"2024-04-25T09:00:43.193105+0000","last_became_active":"2024-04-25T09:00:43.191700+0000","last_became_peered":"2024-04-25T09:00:43.191700+0000","last_unstale":"2024-04-25T09:00:43.193105+0000","last_undegraded":"2024-04-25T09:00:43.193105+0000","last_fullsized":"2024-04-25T09:00:43.193105+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:53.978092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.181988+0000","last_change":"2024-04-25T09:00:43.178794+0000","last_active":"2024-04-25T09:00:43.181988+0000","last_peered":"2024-04-25T09:00:43.181988+0000","last_clean":"2024-04-25T09:00:43.181988+0000","last_became_active":"2024-04-25T09:00:43.178541+0000","last_became_peered":"2024-04-25T09:00:43.178541+0000","last_unstale":"2024-04-25T09:00:43.181988+0000","last_undegraded":"2024-04-25T09:00:43.181988+0000","last_fullsized":"2024-04-25T09:00:43.181988+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:14.712980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.178781+0000","last_change":"2024-04-25T09:00:43.178202+0000","last_active":"2024-04-25T09:00:43.178781+0000","last_peered":"2024-04-25T09:00:43.178781+0000","last_clean":"2024-04-25T09:00:43.178781+0000","last_became_active":"2024-04-25T09:00:43.177917+0000","last_became_peered":"2024-04-25T09:00:43.177917+0000","last_unstale":"2024-04-25T09:00:43.178781+0000","last_undegraded":"2024-04-25T09:00:43.178781+0000","last_fullsized":"2024-04-25T09:00:43.178781+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:16.230988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.189231+0000","last_change":"2024-04-25T09:00:43.187981+0000","last_active":"2024-04-25T09:00:43.189231+0000","last_peered":"2024-04-25T09:00:43.189231+0000","last_clean":"2024-04-25T09:00:43.189231+0000","last_became_active":"2024-04-25T09:00:43.187699+0000","last_became_peered":"2024-04-25T09:00:43.187699+0000","last_unstale":"2024-04-25T09:00:43.189231+0000","last_undegraded":"2024-04-25T09:00:43.189231+0000","last_fullsized":"2024-04-25T09:00:43.189231+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:41.153048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.180591+0000","last_change":"2024-04-25T09:00:43.179345+0000","last_active":"2024-04-25T09:00:43.180591+0000","last_peered":"2024-04-25T09:00:43.180591+0000","last_clean":"2024-04-25T09:00:43.180591+0000","last_became_active":"2024-04-25T09:00:43.179044+0000","last_became_peered":"2024-04-25T09:00:43.179044+0000","last_unstale":"2024-04-25T09:00:43.180591+0000","last_undegraded":"2024-04-25T09:00:43.180591+0000","last_fullsized":"2024-04-25T09:00:43.180591+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:51.285995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.173802+0000","last_change":"2024-04-25T09:00:43.172627+0000","last_active":"2024-04-25T09:00:43.173802+0000","last_peered":"2024-04-25T09:00:43.173802+0000","last_clean":"2024-04-25T09:00:43.173802+0000","last_became_active":"2024-04-25T09:00:43.172325+0000","last_became_peered":"2024-04-25T09:00:43.172325+0000","last_unstale":"2024-04-25T09:00:43.173802+0000","last_undegraded":"2024-04-25T09:00:43.173802+0000","last_fullsized":"2024-04-25T09:00:43.173802+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:39.516623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.176370+0000","last_change":"2024-04-25T09:00:43.174285+0000","last_active":"2024-04-25T09:00:43.176370+0000","last_peered":"2024-04-25T09:00:43.176370+0000","last_clean":"2024-04-25T09:00:43.176370+0000","last_became_active":"2024-04-25T09:00:43.173939+0000","last_became_peered":"2024-04-25T09:00:43.173939+0000","last_unstale":"2024-04-25T09:00:43.176370+0000","last_undegraded":"2024-04-25T09:00:43.176370+0000","last_fullsized":"2024-04-25T09:00:43.176370+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:56.776489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.176669+0000","last_change":"2024-04-25T09:00:43.173547+0000","last_active":"2024-04-25T09:00:43.176669+0000","last_peered":"2024-04-25T09:00:43.176669+0000","last_clean":"2024-04-25T09:00:43.176669+0000","last_became_active":"2024-04-25T09:00:43.173267+0000","last_became_peered":"2024-04-25T09:00:43.173267+0000","last_unstale":"2024-04-25T09:00:43.176669+0000","last_undegraded":"2024-04-25T09:00:43.176669+0000","last_fullsized":"2024-04-25T09:00:43.176669+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:16.385240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.184130+0000","last_change":"2024-04-25T09:00:43.183175+0000","last_active":"2024-04-25T09:00:43.184130+0000","last_peered":"2024-04-25T09:00:43.184130+0000","last_clean":"2024-04-25T09:00:43.184130+0000","last_became_active":"2024-04-25T09:00:43.180734+0000","last_became_peered":"2024-04-25T09:00:43.180734+0000","last_unstale":"2024-04-25T09:00:43.184130+0000","last_undegraded":"2024-04-25T09:00:43.184130+0000","last_fullsized":"2024-04-25T09:00:43.184130+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:13.214085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.189266+0000","last_change":"2024-04-25T09:00:43.184441+0000","last_active":"2024-04-25T09:00:43.189266+0000","last_peered":"2024-04-25T09:00:43.189266+0000","last_clean":"2024-04-25T09:00:43.189266+0000","last_became_active":"2024-04-25T09:00:43.184129+0000","last_became_peered":"2024-04-25T09:00:43.184129+0000","last_unstale":"2024-04-25T09:00:43.189266+0000","last_undegraded":"2024-04-25T09:00:43.189266+0000","last_fullsized":"2024-04-25T09:00:43.189266+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:41.658911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.188994+0000","last_change":"2024-04-25T09:00:43.182774+0000","last_active":"2024-04-25T09:00:43.188994+0000","last_peered":"2024-04-25T09:00:43.188994+0000","last_clean":"2024-04-25T09:00:43.188994+0000","last_became_active":"2024-04-25T09:00:43.182496+0000","last_became_peered":"2024-04-25T09:00:43.182496+0000","last_unstale":"2024-04-25T09:00:43.188994+0000","last_undegraded":"2024-04-25T09:00:43.188994+0000","last_fullsized":"2024-04-25T09:00:43.188994+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:13.008317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.192835+0000","last_change":"2024-04-25T09:00:43.185890+0000","last_active":"2024-04-25T09:00:43.192835+0000","last_peered":"2024-04-25T09:00:43.192835+0000","last_clean":"2024-04-25T09:00:43.192835+0000","last_became_active":"2024-04-25T09:00:43.185384+0000","last_became_peered":"2024-04-25T09:00:43.185384+0000","last_unstale":"2024-04-25T09:00:43.192835+0000","last_undegraded":"2024-04-25T09:00:43.192835+0000","last_fullsized":"2024-04-25T09:00:43.192835+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:45.080353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.182042+0000","last_change":"2024-04-25T09:00:43.180540+0000","last_active":"2024-04-25T09:00:43.182042+0000","last_peered":"2024-04-25T09:00:43.182042+0000","last_clean":"2024-04-25T09:00:43.182042+0000","last_became_active":"2024-04-25T09:00:43.180116+0000","last_became_peered":"2024-04-25T09:00:43.180116+0000","last_unstale":"2024-04-25T09:00:43.182042+0000","last_undegraded":"2024-04-25T09:00:43.182042+0000","last_fullsized":"2024-04-25T09:00:43.182042+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:56.178712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.189034+0000","last_change":"2024-04-25T09:00:43.180346+0000","last_active":"2024-04-25T09:00:43.189034+0000","last_peered":"2024-04-25T09:00:43.189034+0000","last_clean":"2024-04-25T09:00:43.189034+0000","last_became_active":"2024-04-25T09:00:43.179869+0000","last_became_peered":"2024-04-25T09:00:43.179869+0000","last_unstale":"2024-04-25T09:00:43.189034+0000","last_undegraded":"2024-04-25T09:00:43.189034+0000","last_fullsized":"2024-04-25T09:00:43.189034+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:40.768173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.192254+0000","last_change":"2024-04-25T09:00:43.185325+0000","last_active":"2024-04-25T09:00:43.192254+0000","last_peered":"2024-04-25T09:00:43.192254+0000","last_clean":"2024-04-25T09:00:43.192254+0000","last_became_active":"2024-04-25T09:00:43.185034+0000","last_became_peered":"2024-04-25T09:00:43.185034+0000","last_unstale":"2024-04-25T09:00:43.192254+0000","last_undegraded":"2024-04-25T09:00:43.192254+0000","last_fullsized":"2024-04-25T09:00:43.192254+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:13.984050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.192764+0000","last_change":"2024-04-25T09:00:43.192342+0000","last_active":"2024-04-25T09:00:43.192764+0000","last_peered":"2024-04-25T09:00:43.192764+0000","last_clean":"2024-04-25T09:00:43.192764+0000","last_became_active":"2024-04-25T09:00:43.192036+0000","last_became_peered":"2024-04-25T09:00:43.192036+0000","last_unstale":"2024-04-25T09:00:43.192764+0000","last_undegraded":"2024-04-25T09:00:43.192764+0000","last_fullsized":"2024-04-25T09:00:43.192764+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:58.838562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.187621+0000","last_change":"2024-04-25T09:00:43.182867+0000","last_active":"2024-04-25T09:00:43.187621+0000","last_peered":"2024-04-25T09:00:43.187621+0000","last_clean":"2024-04-25T09:00:43.187621+0000","last_became_active":"2024-04-25T09:00:43.182512+0000","last_became_peered":"2024-04-25T09:00:43.182512+0000","last_unstale":"2024-04-25T09:00:43.187621+0000","last_undegraded":"2024-04-25T09:00:43.187621+0000","last_fullsized":"2024-04-25T09:00:43.187621+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:39.680627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.177825+0000","last_change":"2024-04-25T09:00:43.174572+0000","last_active":"2024-04-25T09:00:43.177825+0000","last_peered":"2024-04-25T09:00:43.177825+0000","last_clean":"2024-04-25T09:00:43.177825+0000","last_became_active":"2024-04-25T09:00:43.174282+0000","last_became_peered":"2024-04-25T09:00:43.174282+0000","last_unstale":"2024-04-25T09:00:43.177825+0000","last_undegraded":"2024-04-25T09:00:43.177825+0000","last_fullsized":"2024-04-25T09:00:43.177825+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:53.912544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.193265+0000","last_change":"2024-04-25T09:00:43.192377+0000","last_active":"2024-04-25T09:00:43.193265+0000","last_peered":"2024-04-25T09:00:43.193265+0000","last_clean":"2024-04-25T09:00:43.193265+0000","last_became_active":"2024-04-25T09:00:43.191972+0000","last_became_peered":"2024-04-25T09:00:43.191972+0000","last_unstale":"2024-04-25T09:00:43.193265+0000","last_undegraded":"2024-04-25T09:00:43.193265+0000","last_fullsized":"2024-04-25T09:00:43.193265+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:31.443292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.192860+0000","last_change":"2024-04-25T09:00:43.190472+0000","last_active":"2024-04-25T09:00:43.192860+0000","last_peered":"2024-04-25T09:00:43.192860+0000","last_clean":"2024-04-25T09:00:43.192860+0000","last_became_active":"2024-04-25T09:00:43.190194+0000","last_became_peered":"2024-04-25T09:00:43.190194+0000","last_unstale":"2024-04-25T09:00:43.192860+0000","last_undegraded":"2024-04-25T09:00:43.192860+0000","last_fullsized":"2024-04-25T09:00:43.192860+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:43.741781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.187760+0000","last_change":"2024-04-25T09:00:43.184746+0000","last_active":"2024-04-25T09:00:43.187760+0000","last_peered":"2024-04-25T09:00:43.187760+0000","last_clean":"2024-04-25T09:00:43.187760+0000","last_became_active":"2024-04-25T09:00:43.183653+0000","last_became_peered":"2024-04-25T09:00:43.183653+0000","last_unstale":"2024-04-25T09:00:43.187760+0000","last_undegraded":"2024-04-25T09:00:43.187760+0000","last_fullsized":"2024-04-25T09:00:43.187760+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:25.325832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.192564+0000","last_change":"2024-04-25T09:00:43.191974+0000","last_active":"2024-04-25T09:00:43.192564+0000","last_peered":"2024-04-25T09:00:43.192564+0000","last_clean":"2024-04-25T09:00:43.192564+0000","last_became_active":"2024-04-25T09:00:43.191589+0000","last_became_peered":"2024-04-25T09:00:43.191589+0000","last_unstale":"2024-04-25T09:00:43.192564+0000","last_undegraded":"2024-04-25T09:00:43.192564+0000","last_fullsized":"2024-04-25T09:00:43.192564+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:13.214085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.177396+0000","last_change":"2024-04-25T09:00:43.175200+0000","last_active":"2024-04-25T09:00:43.177396+0000","last_peered":"2024-04-25T09:00:43.177396+0000","last_clean":"2024-04-25T09:00:43.177396+0000","last_became_active":"2024-04-25T09:00:43.174712+0000","last_became_peered":"2024-04-25T09:00:43.174712+0000","last_unstale":"2024-04-25T09:00:43.177396+0000","last_undegraded":"2024-04-25T09:00:43.177396+0000","last_fullsized":"2024-04-25T09:00:43.177396+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:09.447125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"88'4","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.190053+0000","last_change":"2024-04-25T09:00:42.180799+0000","last_active":"2024-04-25T09:00:43.190053+0000","last_peered":"2024-04-25T09:00:43.190053+0000","last_clean":"2024-04-25T09:00:43.190053+0000","last_became_active":"2024-04-25T09:00:42.180501+0000","last_became_peered":"2024-04-25T09:00:42.180501+0000","last_unstale":"2024-04-25T09:00:43.190053+0000","last_undegraded":"2024-04-25T09:00:43.190053+0000","last_fullsized":"2024-04-25T09:00:43.190053+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:08.445099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.182698+0000","last_change":"2024-04-25T09:00:43.181086+0000","last_active":"2024-04-25T09:00:43.182698+0000","last_peered":"2024-04-25T09:00:43.182698+0000","last_clean":"2024-04-25T09:00:43.182698+0000","last_became_active":"2024-04-25T09:00:43.180161+0000","last_became_peered":"2024-04-25T09:00:43.180161+0000","last_unstale":"2024-04-25T09:00:43.182698+0000","last_undegraded":"2024-04-25T09:00:43.182698+0000","last_fullsized":"2024-04-25T09:00:43.182698+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:14.086119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"88'1","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.188311+0000","last_change":"2024-04-25T09:00:42.177529+0000","last_active":"2024-04-25T09:00:43.188311+0000","last_peered":"2024-04-25T09:00:43.188311+0000","last_clean":"2024-04-25T09:00:43.188311+0000","last_became_active":"2024-04-25T09:00:42.177224+0000","last_became_peered":"2024-04-25T09:00:42.177224+0000","last_unstale":"2024-04-25T09:00:43.188311+0000","last_undegraded":"2024-04-25T09:00:43.188311+0000","last_fullsized":"2024-04-25T09:00:43.188311+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:52.324815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.187236+0000","last_change":"2024-04-25T09:00:43.186525+0000","last_active":"2024-04-25T09:00:43.187236+0000","last_peered":"2024-04-25T09:00:43.187236+0000","last_clean":"2024-04-25T09:00:43.187236+0000","last_became_active":"2024-04-25T09:00:43.186206+0000","last_became_peered":"2024-04-25T09:00:43.186206+0000","last_unstale":"2024-04-25T09:00:43.187236+0000","last_undegraded":"2024-04-25T09:00:43.187236+0000","last_fullsized":"2024-04-25T09:00:43.187236+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:12.895798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"88'2","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.197931+0000","last_change":"2024-04-25T09:00:42.182572+0000","last_active":"2024-04-25T09:00:43.197931+0000","last_peered":"2024-04-25T09:00:43.197931+0000","last_clean":"2024-04-25T09:00:43.197931+0000","last_became_active":"2024-04-25T09:00:42.182284+0000","last_became_peered":"2024-04-25T09:00:42.182284+0000","last_unstale":"2024-04-25T09:00:43.197931+0000","last_undegraded":"2024-04-25T09:00:43.197931+0000","last_fullsized":"2024-04-25T09:00:43.197931+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:11.521139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.188375+0000","last_change":"2024-04-25T09:00:43.187984+0000","last_active":"2024-04-25T09:00:43.188375+0000","last_peered":"2024-04-25T09:00:43.188375+0000","last_clean":"2024-04-25T09:00:43.188375+0000","last_became_active":"2024-04-25T09:00:43.187670+0000","last_became_peered":"2024-04-25T09:00:43.187670+0000","last_unstale":"2024-04-25T09:00:43.188375+0000","last_undegraded":"2024-04-25T09:00:43.188375+0000","last_fullsized":"2024-04-25T09:00:43.188375+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:29.639967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"88'2","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.197964+0000","last_change":"2024-04-25T09:00:42.183307+0000","last_active":"2024-04-25T09:00:43.197964+0000","last_peered":"2024-04-25T09:00:43.197964+0000","last_clean":"2024-04-25T09:00:43.197964+0000","last_became_active":"2024-04-25T09:00:42.183004+0000","last_became_peered":"2024-04-25T09:00:42.183004+0000","last_unstale":"2024-04-25T09:00:43.197964+0000","last_undegraded":"2024-04-25T09:00:43.197964+0000","last_fullsized":"2024-04-25T09:00:43.197964+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:24.323807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.192273+0000","last_change":"2024-04-25T09:00:43.185640+0000","last_active":"2024-04-25T09:00:43.192273+0000","last_peered":"2024-04-25T09:00:43.192273+0000","last_clean":"2024-04-25T09:00:43.192273+0000","last_became_active":"2024-04-25T09:00:43.185012+0000","last_became_peered":"2024-04-25T09:00:43.185012+0000","last_unstale":"2024-04-25T09:00:43.192273+0000","last_undegraded":"2024-04-25T09:00:43.192273+0000","last_fullsized":"2024-04-25T09:00:43.192273+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:46.924567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"88'1","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.193467+0000","last_change":"2024-04-25T09:00:42.181945+0000","last_active":"2024-04-25T09:00:43.193467+0000","last_peered":"2024-04-25T09:00:43.193467+0000","last_clean":"2024-04-25T09:00:43.193467+0000","last_became_active":"2024-04-25T09:00:42.181439+0000","last_became_peered":"2024-04-25T09:00:42.181439+0000","last_unstale":"2024-04-25T09:00:43.193467+0000","last_undegraded":"2024-04-25T09:00:43.193467+0000","last_fullsized":"2024-04-25T09:00:43.193467+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:10.300335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.181311+0000","last_change":"2024-04-25T09:00:43.177894+0000","last_active":"2024-04-25T09:00:43.181311+0000","last_peered":"2024-04-25T09:00:43.181311+0000","last_clean":"2024-04-25T09:00:43.181311+0000","last_became_active":"2024-04-25T09:00:43.177607+0000","last_became_peered":"2024-04-25T09:00:43.177607+0000","last_unstale":"2024-04-25T09:00:43.181311+0000","last_undegraded":"2024-04-25T09:00:43.181311+0000","last_fullsized":"2024-04-25T09:00:43.181311+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:17.480739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"88'1","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.191608+0000","last_change":"2024-04-25T09:00:42.183055+0000","last_active":"2024-04-25T09:00:43.191608+0000","last_peered":"2024-04-25T09:00:43.191608+0000","last_clean":"2024-04-25T09:00:43.191608+0000","last_became_active":"2024-04-25T09:00:42.182742+0000","last_became_peered":"2024-04-25T09:00:42.182742+0000","last_unstale":"2024-04-25T09:00:43.191608+0000","last_undegraded":"2024-04-25T09:00:43.191608+0000","last_fullsized":"2024-04-25T09:00:43.191608+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:52.976067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.182591+0000","last_change":"2024-04-25T09:00:43.181180+0000","last_active":"2024-04-25T09:00:43.182591+0000","last_peered":"2024-04-25T09:00:43.182591+0000","last_clean":"2024-04-25T09:00:43.182591+0000","last_became_active":"2024-04-25T09:00:43.180923+0000","last_became_peered":"2024-04-25T09:00:43.180923+0000","last_unstale":"2024-04-25T09:00:43.182591+0000","last_undegraded":"2024-04-25T09:00:43.182591+0000","last_fullsized":"2024-04-25T09:00:43.182591+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:44.985514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.174839+0000","last_change":"2024-04-25T09:00:42.172911+0000","last_active":"2024-04-25T09:00:43.174839+0000","last_peered":"2024-04-25T09:00:43.174839+0000","last_clean":"2024-04-25T09:00:43.174839+0000","last_became_active":"2024-04-25T09:00:42.172632+0000","last_became_peered":"2024-04-25T09:00:42.172632+0000","last_unstale":"2024-04-25T09:00:43.174839+0000","last_undegraded":"2024-04-25T09:00:43.174839+0000","last_fullsized":"2024-04-25T09:00:43.174839+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:55.176686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.180075+0000","last_change":"2024-04-25T09:00:43.178947+0000","last_active":"2024-04-25T09:00:43.180075+0000","last_peered":"2024-04-25T09:00:43.180075+0000","last_clean":"2024-04-25T09:00:43.180075+0000","last_became_active":"2024-04-25T09:00:43.178489+0000","last_became_peered":"2024-04-25T09:00:43.178489+0000","last_unstale":"2024-04-25T09:00:43.180075+0000","last_undegraded":"2024-04-25T09:00:43.180075+0000","last_fullsized":"2024-04-25T09:00:43.180075+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:48.733245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.181652+0000","last_change":"2024-04-25T09:00:42.180245+0000","last_active":"2024-04-25T09:00:43.181652+0000","last_peered":"2024-04-25T09:00:43.181652+0000","last_clean":"2024-04-25T09:00:43.181652+0000","last_became_active":"2024-04-25T09:00:42.179814+0000","last_became_peered":"2024-04-25T09:00:42.179814+0000","last_unstale":"2024-04-25T09:00:43.181652+0000","last_undegraded":"2024-04-25T09:00:43.181652+0000","last_fullsized":"2024-04-25T09:00:43.181652+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:57.870517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.193483+0000","last_change":"2024-04-25T09:00:43.192917+0000","last_active":"2024-04-25T09:00:43.193483+0000","last_peered":"2024-04-25T09:00:43.193483+0000","last_clean":"2024-04-25T09:00:43.193483+0000","last_became_active":"2024-04-25T09:00:43.192562+0000","last_became_peered":"2024-04-25T09:00:43.192562+0000","last_unstale":"2024-04-25T09:00:43.193483+0000","last_undegraded":"2024-04-25T09:00:43.193483+0000","last_fullsized":"2024-04-25T09:00:43.193483+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:08.566826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.171633+0000","last_change":"2024-04-25T09:00:42.182448+0000","last_active":"2024-04-25T09:00:43.171633+0000","last_peered":"2024-04-25T09:00:43.171633+0000","last_clean":"2024-04-25T09:00:43.171633+0000","last_became_active":"2024-04-25T09:00:42.182160+0000","last_became_peered":"2024-04-25T09:00:42.182160+0000","last_unstale":"2024-04-25T09:00:43.171633+0000","last_undegraded":"2024-04-25T09:00:43.171633+0000","last_fullsized":"2024-04-25T09:00:43.171633+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:38.678601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.181187+0000","last_change":"2024-04-25T09:00:43.175117+0000","last_active":"2024-04-25T09:00:43.181187+0000","last_peered":"2024-04-25T09:00:43.181187+0000","last_clean":"2024-04-25T09:00:43.181187+0000","last_became_active":"2024-04-25T09:00:43.174841+0000","last_became_peered":"2024-04-25T09:00:43.174841+0000","last_unstale":"2024-04-25T09:00:43.181187+0000","last_undegraded":"2024-04-25T09:00:43.181187+0000","last_fullsized":"2024-04-25T09:00:43.181187+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:39.680627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.170971+0000","last_change":"2024-04-25T09:00:42.178333+0000","last_active":"2024-04-25T09:00:43.170971+0000","last_peered":"2024-04-25T09:00:43.170971+0000","last_clean":"2024-04-25T09:00:43.170971+0000","last_became_active":"2024-04-25T09:00:42.177995+0000","last_became_peered":"2024-04-25T09:00:42.177995+0000","last_unstale":"2024-04-25T09:00:43.170971+0000","last_undegraded":"2024-04-25T09:00:43.170971+0000","last_fullsized":"2024-04-25T09:00:43.170971+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:24.904271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.179439+0000","last_change":"2024-04-25T09:00:43.177655+0000","last_active":"2024-04-25T09:00:43.179439+0000","last_peered":"2024-04-25T09:00:43.179439+0000","last_clean":"2024-04-25T09:00:43.179439+0000","last_became_active":"2024-04-25T09:00:43.177308+0000","last_became_peered":"2024-04-25T09:00:43.177308+0000","last_unstale":"2024-04-25T09:00:43.179439+0000","last_undegraded":"2024-04-25T09:00:43.179439+0000","last_fullsized":"2024-04-25T09:00:43.179439+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:12.895798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"88'3","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.193492+0000","last_change":"2024-04-25T09:00:42.180005+0000","last_active":"2024-04-25T09:00:43.193492+0000","last_peered":"2024-04-25T09:00:43.193492+0000","last_clean":"2024-04-25T09:00:43.193492+0000","last_became_active":"2024-04-25T09:00:42.179805+0000","last_became_peered":"2024-04-25T09:00:42.179805+0000","last_unstale":"2024-04-25T09:00:43.193492+0000","last_undegraded":"2024-04-25T09:00:43.193492+0000","last_fullsized":"2024-04-25T09:00:43.193492+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:38.514597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.187881+0000","last_change":"2024-04-25T09:00:43.183685+0000","last_active":"2024-04-25T09:00:43.187881+0000","last_peered":"2024-04-25T09:00:43.187881+0000","last_clean":"2024-04-25T09:00:43.187881+0000","last_became_active":"2024-04-25T09:00:43.183395+0000","last_became_peered":"2024-04-25T09:00:43.183395+0000","last_unstale":"2024-04-25T09:00:43.187881+0000","last_undegraded":"2024-04-25T09:00:43.187881+0000","last_fullsized":"2024-04-25T09:00:43.187881+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:38.416326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"88'6","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.196377+0000","last_change":"2024-04-25T09:00:42.174256+0000","last_active":"2024-04-25T09:00:43.196377+0000","last_peered":"2024-04-25T09:00:43.196377+0000","last_clean":"2024-04-25T09:00:43.196377+0000","last_became_active":"2024-04-25T09:00:42.173019+0000","last_became_peered":"2024-04-25T09:00:42.173019+0000","last_unstale":"2024-04-25T09:00:43.196377+0000","last_undegraded":"2024-04-25T09:00:43.196377+0000","last_fullsized":"2024-04-25T09:00:43.196377+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:16.478714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.194006+0000","last_change":"2024-04-25T09:00:43.190145+0000","last_active":"2024-04-25T09:00:43.194006+0000","last_peered":"2024-04-25T09:00:43.194006+0000","last_clean":"2024-04-25T09:00:43.194006+0000","last_became_active":"2024-04-25T09:00:43.189866+0000","last_became_peered":"2024-04-25T09:00:43.189866+0000","last_unstale":"2024-04-25T09:00:43.194006+0000","last_undegraded":"2024-04-25T09:00:43.194006+0000","last_fullsized":"2024-04-25T09:00:43.194006+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:28.230599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"88'2","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.197594+0000","last_change":"2024-04-25T09:00:42.178220+0000","last_active":"2024-04-25T09:00:43.197594+0000","last_peered":"2024-04-25T09:00:43.197594+0000","last_clean":"2024-04-25T09:00:43.197594+0000","last_became_active":"2024-04-25T09:00:42.175334+0000","last_became_peered":"2024-04-25T09:00:42.175334+0000","last_unstale":"2024-04-25T09:00:43.197594+0000","last_undegraded":"2024-04-25T09:00:43.197594+0000","last_fullsized":"2024-04-25T09:00:43.197594+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:41.096248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.186928+0000","last_change":"2024-04-25T09:00:43.182067+0000","last_active":"2024-04-25T09:00:43.186928+0000","last_peered":"2024-04-25T09:00:43.186928+0000","last_clean":"2024-04-25T09:00:43.186928+0000","last_became_active":"2024-04-25T09:00:43.181786+0000","last_became_peered":"2024-04-25T09:00:43.181786+0000","last_unstale":"2024-04-25T09:00:43.186928+0000","last_undegraded":"2024-04-25T09:00:43.186928+0000","last_fullsized":"2024-04-25T09:00:43.186928+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:17.817332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.173961+0000","last_change":"2024-04-25T09:00:42.171759+0000","last_active":"2024-04-25T09:00:43.173961+0000","last_peered":"2024-04-25T09:00:43.173961+0000","last_clean":"2024-04-25T09:00:43.173961+0000","last_became_active":"2024-04-25T09:00:42.171450+0000","last_became_peered":"2024-04-25T09:00:42.171450+0000","last_unstale":"2024-04-25T09:00:43.173961+0000","last_undegraded":"2024-04-25T09:00:43.173961+0000","last_fullsized":"2024-04-25T09:00:43.173961+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:55.807693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.193281+0000","last_change":"2024-04-25T09:00:43.190978+0000","last_active":"2024-04-25T09:00:43.193281+0000","last_peered":"2024-04-25T09:00:43.193281+0000","last_clean":"2024-04-25T09:00:43.193281+0000","last_became_active":"2024-04-25T09:00:43.189641+0000","last_became_peered":"2024-04-25T09:00:43.189641+0000","last_unstale":"2024-04-25T09:00:43.193281+0000","last_undegraded":"2024-04-25T09:00:43.193281+0000","last_fullsized":"2024-04-25T09:00:43.193281+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:57.507520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.168137+0000","last_change":"2024-04-25T09:00:42.171903+0000","last_active":"2024-04-25T09:00:43.168137+0000","last_peered":"2024-04-25T09:00:43.168137+0000","last_clean":"2024-04-25T09:00:43.168137+0000","last_became_active":"2024-04-25T09:00:42.171606+0000","last_became_peered":"2024-04-25T09:00:42.171606+0000","last_unstale":"2024-04-25T09:00:43.168137+0000","last_undegraded":"2024-04-25T09:00:43.168137+0000","last_fullsized":"2024-04-25T09:00:43.168137+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:08.445099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.184841+0000","last_change":"2024-04-25T09:00:43.183856+0000","last_active":"2024-04-25T09:00:43.184841+0000","last_peered":"2024-04-25T09:00:43.184841+0000","last_clean":"2024-04-25T09:00:43.184841+0000","last_became_active":"2024-04-25T09:00:43.183525+0000","last_became_peered":"2024-04-25T09:00:43.183525+0000","last_unstale":"2024-04-25T09:00:43.184841+0000","last_undegraded":"2024-04-25T09:00:43.184841+0000","last_fullsized":"2024-04-25T09:00:43.184841+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:57.490239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"88'8","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.196111+0000","last_change":"2024-04-25T09:00:42.177843+0000","last_active":"2024-04-25T09:00:43.196111+0000","last_peered":"2024-04-25T09:00:43.196111+0000","last_clean":"2024-04-25T09:00:43.196111+0000","last_became_active":"2024-04-25T09:00:42.177532+0000","last_became_peered":"2024-04-25T09:00:42.177532+0000","last_unstale":"2024-04-25T09:00:43.196111+0000","last_undegraded":"2024-04-25T09:00:43.196111+0000","last_fullsized":"2024-04-25T09:00:43.196111+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:12.908403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.176115+0000","last_change":"2024-04-25T09:00:43.175301+0000","last_active":"2024-04-25T09:00:43.176115+0000","last_peered":"2024-04-25T09:00:43.176115+0000","last_clean":"2024-04-25T09:00:43.176115+0000","last_became_active":"2024-04-25T09:00:43.174976+0000","last_became_peered":"2024-04-25T09:00:43.174976+0000","last_unstale":"2024-04-25T09:00:43.176115+0000","last_undegraded":"2024-04-25T09:00:43.176115+0000","last_fullsized":"2024-04-25T09:00:43.176115+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:45.042698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"88'1","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.187252+0000","last_change":"2024-04-25T09:00:42.176002+0000","last_active":"2024-04-25T09:00:43.187252+0000","last_peered":"2024-04-25T09:00:43.187252+0000","last_clean":"2024-04-25T09:00:43.187252+0000","last_became_active":"2024-04-25T09:00:42.175640+0000","last_became_peered":"2024-04-25T09:00:42.175640+0000","last_unstale":"2024-04-25T09:00:43.187252+0000","last_undegraded":"2024-04-25T09:00:43.187252+0000","last_fullsized":"2024-04-25T09:00:43.187252+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:16.478714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.181832+0000","last_change":"2024-04-25T09:00:43.176591+0000","last_active":"2024-04-25T09:00:43.181832+0000","last_peered":"2024-04-25T09:00:43.181832+0000","last_clean":"2024-04-25T09:00:43.181832+0000","last_became_active":"2024-04-25T09:00:43.176286+0000","last_became_peered":"2024-04-25T09:00:43.176286+0000","last_unstale":"2024-04-25T09:00:43.181832+0000","last_undegraded":"2024-04-25T09:00:43.181832+0000","last_fullsized":"2024-04-25T09:00:43.181832+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:13.910429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"88'2","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.197722+0000","last_change":"2024-04-25T09:00:42.181512+0000","last_active":"2024-04-25T09:00:43.197722+0000","last_peered":"2024-04-25T09:00:43.197722+0000","last_clean":"2024-04-25T09:00:43.197722+0000","last_became_active":"2024-04-25T09:00:42.181225+0000","last_became_peered":"2024-04-25T09:00:42.181225+0000","last_unstale":"2024-04-25T09:00:43.197722+0000","last_undegraded":"2024-04-25T09:00:43.197722+0000","last_fullsized":"2024-04-25T09:00:43.197722+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:13.084093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.187897+0000","last_change":"2024-04-25T09:00:43.184712+0000","last_active":"2024-04-25T09:00:43.187897+0000","last_peered":"2024-04-25T09:00:43.187897+0000","last_clean":"2024-04-25T09:00:43.187897+0000","last_became_active":"2024-04-25T09:00:43.184414+0000","last_became_peered":"2024-04-25T09:00:43.184414+0000","last_unstale":"2024-04-25T09:00:43.187897+0000","last_undegraded":"2024-04-25T09:00:43.187897+0000","last_fullsized":"2024-04-25T09:00:43.187897+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:13.910429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.169890+0000","last_change":"2024-04-25T09:00:42.171213+0000","last_active":"2024-04-25T09:00:43.169890+0000","last_peered":"2024-04-25T09:00:43.169890+0000","last_clean":"2024-04-25T09:00:43.169890+0000","last_became_active":"2024-04-25T09:00:42.170835+0000","last_became_peered":"2024-04-25T09:00:42.170835+0000","last_unstale":"2024-04-25T09:00:43.169890+0000","last_undegraded":"2024-04-25T09:00:43.169890+0000","last_fullsized":"2024-04-25T09:00:43.169890+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:13.710954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.189209+0000","last_change":"2024-04-25T09:00:43.187928+0000","last_active":"2024-04-25T09:00:43.189209+0000","last_peered":"2024-04-25T09:00:43.189209+0000","last_clean":"2024-04-25T09:00:43.189209+0000","last_became_active":"2024-04-25T09:00:43.187654+0000","last_became_peered":"2024-04-25T09:00:43.187654+0000","last_unstale":"2024-04-25T09:00:43.189209+0000","last_undegraded":"2024-04-25T09:00:43.189209+0000","last_fullsized":"2024-04-25T09:00:43.189209+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:58.872543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"88'2","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.190305+0000","last_change":"2024-04-25T09:00:42.176911+0000","last_active":"2024-04-25T09:00:43.190305+0000","last_peered":"2024-04-25T09:00:43.190305+0000","last_clean":"2024-04-25T09:00:43.190305+0000","last_became_active":"2024-04-25T09:00:42.176547+0000","last_became_peered":"2024-04-25T09:00:42.176547+0000","last_unstale":"2024-04-25T09:00:43.190305+0000","last_undegraded":"2024-04-25T09:00:43.190305+0000","last_fullsized":"2024-04-25T09:00:43.190305+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:52.758780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.180801+0000","last_change":"2024-04-25T09:00:43.179529+0000","last_active":"2024-04-25T09:00:43.180801+0000","last_peered":"2024-04-25T09:00:43.180801+0000","last_clean":"2024-04-25T09:00:43.180801+0000","last_became_active":"2024-04-25T09:00:43.179227+0000","last_became_peered":"2024-04-25T09:00:43.179227+0000","last_unstale":"2024-04-25T09:00:43.180801+0000","last_undegraded":"2024-04-25T09:00:43.180801+0000","last_fullsized":"2024-04-25T09:00:43.180801+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:38.416326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.165919+0000","last_change":"2024-04-25T09:00:42.183308+0000","last_active":"2024-04-25T09:00:43.165919+0000","last_peered":"2024-04-25T09:00:43.165919+0000","last_clean":"2024-04-25T09:00:43.165919+0000","last_became_active":"2024-04-25T09:00:42.183019+0000","last_became_peered":"2024-04-25T09:00:42.183019+0000","last_unstale":"2024-04-25T09:00:43.165919+0000","last_undegraded":"2024-04-25T09:00:43.165919+0000","last_fullsized":"2024-04-25T09:00:43.165919+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:02.088978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.181311+0000","last_change":"2024-04-25T09:00:43.176767+0000","last_active":"2024-04-25T09:00:43.181311+0000","last_peered":"2024-04-25T09:00:43.181311+0000","last_clean":"2024-04-25T09:00:43.181311+0000","last_became_active":"2024-04-25T09:00:43.175823+0000","last_became_peered":"2024-04-25T09:00:43.175823+0000","last_unstale":"2024-04-25T09:00:43.181311+0000","last_undegraded":"2024-04-25T09:00:43.181311+0000","last_fullsized":"2024-04-25T09:00:43.181311+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:13.568725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.165519+0000","last_change":"2024-04-25T09:00:42.178120+0000","last_active":"2024-04-25T09:00:43.165519+0000","last_peered":"2024-04-25T09:00:43.165519+0000","last_clean":"2024-04-25T09:00:43.165519+0000","last_became_active":"2024-04-25T09:00:42.177786+0000","last_became_peered":"2024-04-25T09:00:42.177786+0000","last_unstale":"2024-04-25T09:00:43.165519+0000","last_undegraded":"2024-04-25T09:00:43.165519+0000","last_fullsized":"2024-04-25T09:00:43.165519+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:25.618831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.179458+0000","last_change":"2024-04-25T09:00:43.177085+0000","last_active":"2024-04-25T09:00:43.179458+0000","last_peered":"2024-04-25T09:00:43.179458+0000","last_clean":"2024-04-25T09:00:43.179458+0000","last_became_active":"2024-04-25T09:00:43.176725+0000","last_became_peered":"2024-04-25T09:00:43.176725+0000","last_unstale":"2024-04-25T09:00:43.179458+0000","last_undegraded":"2024-04-25T09:00:43.179458+0000","last_fullsized":"2024-04-25T09:00:43.179458+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:53.912544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.172280+0000","last_change":"2024-04-25T09:00:42.179763+0000","last_active":"2024-04-25T09:00:43.172280+0000","last_peered":"2024-04-25T09:00:43.172280+0000","last_clean":"2024-04-25T09:00:43.172280+0000","last_became_active":"2024-04-25T09:00:42.179482+0000","last_became_peered":"2024-04-25T09:00:42.179482+0000","last_unstale":"2024-04-25T09:00:43.172280+0000","last_undegraded":"2024-04-25T09:00:43.172280+0000","last_fullsized":"2024-04-25T09:00:43.172280+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:12.405917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.177374+0000","last_change":"2024-04-25T09:00:43.174976+0000","last_active":"2024-04-25T09:00:43.177374+0000","last_peered":"2024-04-25T09:00:43.177374+0000","last_clean":"2024-04-25T09:00:43.177374+0000","last_became_active":"2024-04-25T09:00:43.174398+0000","last_became_peered":"2024-04-25T09:00:43.174398+0000","last_unstale":"2024-04-25T09:00:43.177374+0000","last_undegraded":"2024-04-25T09:00:43.177374+0000","last_fullsized":"2024-04-25T09:00:43.177374+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:25.906297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"88'2","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.191394+0000","last_change":"2024-04-25T09:00:42.174952+0000","last_active":"2024-04-25T09:00:43.191394+0000","last_peered":"2024-04-25T09:00:43.191394+0000","last_clean":"2024-04-25T09:00:43.191394+0000","last_became_active":"2024-04-25T09:00:42.174696+0000","last_became_peered":"2024-04-25T09:00:42.174696+0000","last_unstale":"2024-04-25T09:00:43.191394+0000","last_undegraded":"2024-04-25T09:00:43.191394+0000","last_fullsized":"2024-04-25T09:00:43.191394+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:38.529151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.182624+0000","last_change":"2024-04-25T09:00:43.179282+0000","last_active":"2024-04-25T09:00:43.182624+0000","last_peered":"2024-04-25T09:00:43.182624+0000","last_clean":"2024-04-25T09:00:43.182624+0000","last_became_active":"2024-04-25T09:00:43.179012+0000","last_became_peered":"2024-04-25T09:00:43.179012+0000","last_unstale":"2024-04-25T09:00:43.182624+0000","last_undegraded":"2024-04-25T09:00:43.182624+0000","last_fullsized":"2024-04-25T09:00:43.182624+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:31.155495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.181968+0000","last_change":"2024-04-25T09:00:42.172666+0000","last_active":"2024-04-25T09:00:43.181968+0000","last_peered":"2024-04-25T09:00:43.181968+0000","last_clean":"2024-04-25T09:00:43.181968+0000","last_became_active":"2024-04-25T09:00:42.172393+0000","last_became_peered":"2024-04-25T09:00:42.172393+0000","last_unstale":"2024-04-25T09:00:43.181968+0000","last_undegraded":"2024-04-25T09:00:43.181968+0000","last_fullsized":"2024-04-25T09:00:43.181968+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:40.151022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.182134+0000","last_change":"2024-04-25T09:00:43.180722+0000","last_active":"2024-04-25T09:00:43.182134+0000","last_peered":"2024-04-25T09:00:43.182134+0000","last_clean":"2024-04-25T09:00:43.182134+0000","last_became_active":"2024-04-25T09:00:43.180278+0000","last_became_peered":"2024-04-25T09:00:43.180278+0000","last_unstale":"2024-04-25T09:00:43.182134+0000","last_undegraded":"2024-04-25T09:00:43.182134+0000","last_fullsized":"2024-04-25T09:00:43.182134+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:14.216474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.170783+0000","last_change":"2024-04-25T09:00:42.177604+0000","last_active":"2024-04-25T09:00:43.170783+0000","last_peered":"2024-04-25T09:00:43.170783+0000","last_clean":"2024-04-25T09:00:43.170783+0000","last_became_active":"2024-04-25T09:00:42.177163+0000","last_became_peered":"2024-04-25T09:00:42.177163+0000","last_unstale":"2024-04-25T09:00:43.170783+0000","last_undegraded":"2024-04-25T09:00:43.170783+0000","last_fullsized":"2024-04-25T09:00:43.170783+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:55.774463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.187644+0000","last_change":"2024-04-25T09:00:43.183055+0000","last_active":"2024-04-25T09:00:43.187644+0000","last_peered":"2024-04-25T09:00:43.187644+0000","last_clean":"2024-04-25T09:00:43.187644+0000","last_became_active":"2024-04-25T09:00:43.182733+0000","last_became_peered":"2024-04-25T09:00:43.182733+0000","last_unstale":"2024-04-25T09:00:43.187644+0000","last_undegraded":"2024-04-25T09:00:43.187644+0000","last_fullsized":"2024-04-25T09:00:43.187644+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:53.912544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.170260+0000","last_change":"2024-04-25T09:00:42.176448+0000","last_active":"2024-04-25T09:00:43.170260+0000","last_peered":"2024-04-25T09:00:43.170260+0000","last_clean":"2024-04-25T09:00:43.170260+0000","last_became_active":"2024-04-25T09:00:42.175647+0000","last_became_peered":"2024-04-25T09:00:42.175647+0000","last_unstale":"2024-04-25T09:00:43.170260+0000","last_undegraded":"2024-04-25T09:00:43.170260+0000","last_fullsized":"2024-04-25T09:00:43.170260+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:52.324815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.181095+0000","last_change":"2024-04-25T09:00:43.180112+0000","last_active":"2024-04-25T09:00:43.181095+0000","last_peered":"2024-04-25T09:00:43.181095+0000","last_clean":"2024-04-25T09:00:43.181095+0000","last_became_active":"2024-04-25T09:00:43.179827+0000","last_became_peered":"2024-04-25T09:00:43.179827+0000","last_unstale":"2024-04-25T09:00:43.181095+0000","last_undegraded":"2024-04-25T09:00:43.181095+0000","last_fullsized":"2024-04-25T09:00:43.181095+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:13.910429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.174558+0000","last_change":"2024-04-25T09:00:42.176017+0000","last_active":"2024-04-25T09:00:43.174558+0000","last_peered":"2024-04-25T09:00:43.174558+0000","last_clean":"2024-04-25T09:00:43.174558+0000","last_became_active":"2024-04-25T09:00:42.175747+0000","last_became_peered":"2024-04-25T09:00:42.175747+0000","last_unstale":"2024-04-25T09:00:43.174558+0000","last_undegraded":"2024-04-25T09:00:43.174558+0000","last_fullsized":"2024-04-25T09:00:43.174558+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:13.214448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.193967+0000","last_change":"2024-04-25T09:00:43.189763+0000","last_active":"2024-04-25T09:00:43.193967+0000","last_peered":"2024-04-25T09:00:43.193967+0000","last_clean":"2024-04-25T09:00:43.193967+0000","last_became_active":"2024-04-25T09:00:43.189494+0000","last_became_peered":"2024-04-25T09:00:43.189494+0000","last_unstale":"2024-04-25T09:00:43.193967+0000","last_undegraded":"2024-04-25T09:00:43.193967+0000","last_fullsized":"2024-04-25T09:00:43.193967+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:11.461879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.166304+0000","last_change":"2024-04-25T09:00:42.182849+0000","last_active":"2024-04-25T09:00:43.166304+0000","last_peered":"2024-04-25T09:00:43.166304+0000","last_clean":"2024-04-25T09:00:43.166304+0000","last_became_active":"2024-04-25T09:00:42.182527+0000","last_became_peered":"2024-04-25T09:00:42.182527+0000","last_unstale":"2024-04-25T09:00:43.166304+0000","last_undegraded":"2024-04-25T09:00:43.166304+0000","last_fullsized":"2024-04-25T09:00:43.166304+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:30.441266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.186717+0000","last_change":"2024-04-25T09:00:43.185757+0000","last_active":"2024-04-25T09:00:43.186717+0000","last_peered":"2024-04-25T09:00:43.186717+0000","last_clean":"2024-04-25T09:00:43.186717+0000","last_became_active":"2024-04-25T09:00:43.185410+0000","last_became_peered":"2024-04-25T09:00:43.185410+0000","last_unstale":"2024-04-25T09:00:43.186717+0000","last_undegraded":"2024-04-25T09:00:43.186717+0000","last_fullsized":"2024-04-25T09:00:43.186717+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:00.337597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.170234+0000","last_change":"2024-04-25T09:00:42.176603+0000","last_active":"2024-04-25T09:00:43.170234+0000","last_peered":"2024-04-25T09:00:43.170234+0000","last_clean":"2024-04-25T09:00:43.170234+0000","last_became_active":"2024-04-25T09:00:42.176334+0000","last_became_peered":"2024-04-25T09:00:42.176334+0000","last_unstale":"2024-04-25T09:00:43.170234+0000","last_undegraded":"2024-04-25T09:00:43.170234+0000","last_fullsized":"2024-04-25T09:00:43.170234+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:10.300335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.182012+0000","last_change":"2024-04-25T09:00:43.179590+0000","last_active":"2024-04-25T09:00:43.182012+0000","last_peered":"2024-04-25T09:00:43.182012+0000","last_clean":"2024-04-25T09:00:43.182012+0000","last_became_active":"2024-04-25T09:00:43.179270+0000","last_became_peered":"2024-04-25T09:00:43.179270+0000","last_unstale":"2024-04-25T09:00:43.182012+0000","last_undegraded":"2024-04-25T09:00:43.182012+0000","last_fullsized":"2024-04-25T09:00:43.182012+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:12.523164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.166775+0000","last_change":"2024-04-25T09:00:42.176361+0000","last_active":"2024-04-25T09:00:43.166775+0000","last_peered":"2024-04-25T09:00:43.166775+0000","last_clean":"2024-04-25T09:00:43.166775+0000","last_became_active":"2024-04-25T09:00:42.176020+0000","last_became_peered":"2024-04-25T09:00:42.176020+0000","last_unstale":"2024-04-25T09:00:43.166775+0000","last_undegraded":"2024-04-25T09:00:43.166775+0000","last_fullsized":"2024-04-25T09:00:43.166775+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:12.972700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.184308+0000","last_change":"2024-04-25T09:00:43.181469+0000","last_active":"2024-04-25T09:00:43.184308+0000","last_peered":"2024-04-25T09:00:43.184308+0000","last_clean":"2024-04-25T09:00:43.184308+0000","last_became_active":"2024-04-25T09:00:43.181061+0000","last_became_peered":"2024-04-25T09:00:43.181061+0000","last_unstale":"2024-04-25T09:00:43.184308+0000","last_undegraded":"2024-04-25T09:00:43.184308+0000","last_fullsized":"2024-04-25T09:00:43.184308+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:43.718412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"88'1","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.193829+0000","last_change":"2024-04-25T09:00:42.173305+0000","last_active":"2024-04-25T09:00:43.193829+0000","last_peered":"2024-04-25T09:00:43.193829+0000","last_clean":"2024-04-25T09:00:43.193829+0000","last_became_active":"2024-04-25T09:00:42.173016+0000","last_became_peered":"2024-04-25T09:00:42.173016+0000","last_unstale":"2024-04-25T09:00:43.193829+0000","last_undegraded":"2024-04-25T09:00:43.193829+0000","last_fullsized":"2024-04-25T09:00:43.193829+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:27.228573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":296,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.171227+0000","last_change":"2024-04-25T08:54:00.818044+0000","last_active":"2024-04-25T09:00:43.171227+0000","last_peered":"2024-04-25T09:00:43.171227+0000","last_clean":"2024-04-25T09:00:43.171227+0000","last_became_active":"2024-04-25T08:54:00.817098+0000","last_became_peered":"2024-04-25T08:54:00.817098+0000","last_unstale":"2024-04-25T09:00:43.171227+0000","last_undegraded":"2024-04-25T09:00:43.171227+0000","last_fullsized":"2024-04-25T09:00:43.171227+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_clean_scrub_stamp":"2024-04-25T08:53:59.739677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:48.129513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574931,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55016,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316824,"statfs":{"total":96636764160,"available":96580427776,"internally_reserved":0,"allocated":1662976,"data_stored":1049437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574932,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76648,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94295192,"statfs":{"total":96636764160,"available":96558276608,"internally_reserved":0,"allocated":1069056,"data_stored":459069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77396422},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574934,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50304,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321536,"statfs":{"total":96636764160,"available":96585252864,"internally_reserved":0,"allocated":1097728,"data_stored":467478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574932,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76644,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94295196,"statfs":{"total":96636764160,"available":96558280704,"internally_reserved":0,"allocated":1069056,"data_stored":459069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77396422},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574932,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68352,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303488,"statfs":{"total":96636764160,"available":96566771712,"internally_reserved":0,"allocated":1093632,"data_stored":467456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58612,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313228,"statfs":{"total":96636764160,"available":96576745472,"internally_reserved":0,"allocated":1085440,"data_stored":466426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72508,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94299332,"statfs":{"total":96636764160,"available":96562515968,"internally_reserved":0,"allocated":1093632,"data_stored":466038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574933,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59196,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312644,"statfs":{"total":96636764160,"available":96576147456,"internally_reserved":0,"allocated":1679360,"data_stored":1056246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:00:46.209 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:00:46.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.357+0000 7f2250142640 1 Processor -- start 2024-04-25T09:00:46.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.357+0000 7f2250142640 1 -- start start 2024-04-25T09:00:46.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.358+0000 7f2250142640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f22480ce970 0x7f2248168820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.358+0000 7f2250142640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f224815e0e0 con 0x7f22480d35c0 2024-04-25T09:00:46.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.358+0000 7f2250142640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22480ceeb0 con 0x7f224815dd00 2024-04-25T09:00:46.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.358+0000 7f2250142640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22480d24e0 con 0x7f22480ce970 2024-04-25T09:00:46.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.358+0000 7f224d6dd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f22480d35c0 0x7f22480ce260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44318/0 (socket says 172.21.15.43:44318) 2024-04-25T09:00:46.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.358+0000 7f224d6dd640 1 -- 172.21.15.43:0/744495830 learned_addr learned my addr 172.21.15.43:0/744495830 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:00:46.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.358+0000 7f224e6df640 1 --2- 172.21.15.43:0/744495830 >> v2:172.21.15.193:3300/0 conn(0x7f22480ce970 0x7f2248168820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.358+0000 7f224cedc640 1 -- 172.21.15.43:0/744495830 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2307192214 0 0) 0x7f224815e0e0 con 0x7f22480d35c0 2024-04-25T09:00:46.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.358+0000 7f224cedc640 1 -- 172.21.15.43:0/744495830 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2234003520 con 0x7f22480d35c0 2024-04-25T09:00:46.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.358+0000 7f224cedc640 1 -- 172.21.15.43:0/744495830 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3417361034 0 0) 0x7f2234003520 con 0x7f22480d35c0 2024-04-25T09:00:46.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.358+0000 7f224cedc640 1 -- 172.21.15.43:0/744495830 >> v2:172.21.15.193:3300/0 conn(0x7f22480ce970 msgr2=0x7f2248168820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:46.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.358+0000 7f224cedc640 1 --2- 172.21.15.43:0/744495830 >> v2:172.21.15.193:3300/0 conn(0x7f22480ce970 0x7f2248168820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.358+0000 7f224cedc640 1 -- 172.21.15.43:0/744495830 >> v1:172.21.15.193:6789/0 conn(0x7f224815dd00 legacy=0x7f22480d2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.358+0000 7f224cedc640 1 -- 172.21.15.43:0/744495830 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22480d1bb0 con 0x7f22480d35c0 2024-04-25T09:00:46.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.358+0000 7f224cedc640 1 -- 172.21.15.43:0/744495830 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f2238001670 con 0x7f22480d35c0 2024-04-25T09:00:46.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.358+0000 7f224cedc640 1 -- 172.21.15.43:0/744495830 >> v1:172.21.15.43:6789/0 conn(0x7f22480d35c0 legacy=0x7f22480ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.358+0000 7f224cedc640 1 --2- 172.21.15.43:0/744495830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22340039b0 0x7f2234003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.358+0000 7f224cedc640 1 --2- 172.21.15.43:0/744495830 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2234004800 0x7f2234004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.358+0000 7f224cedc640 1 --2- 172.21.15.43:0/744495830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22340051c0 0x7f223400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.358+0000 7f224cedc640 1 -- 172.21.15.43:0/744495830 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22480d24e0 con 0x7f22340051c0 2024-04-25T09:00:46.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.358+0000 7f224cedc640 1 -- 172.21.15.43:0/744495830 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223400c060 con 0x7f22340039b0 2024-04-25T09:00:46.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.358+0000 7f224cedc640 1 -- 172.21.15.43:0/744495830 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f223400c260 con 0x7f2234004800 2024-04-25T09:00:46.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.359+0000 7f224e6df640 1 --2- 172.21.15.43:0/744495830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22340051c0 0x7f223400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.359+0000 7f224dede640 1 --2- 172.21.15.43:0/744495830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22340039b0 0x7f2234003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.359+0000 7f224d6dd640 1 --2- 172.21.15.43:0/744495830 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2234004800 0x7f2234004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.359+0000 7f224e6df640 1 -- 172.21.15.43:0/744495830 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2234004800 msgr2=0x7f2234004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:46.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.359+0000 7f224e6df640 1 --2- 172.21.15.43:0/744495830 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2234004800 0x7f2234004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.359+0000 7f224e6df640 1 -- 172.21.15.43:0/744495830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22340039b0 msgr2=0x7f2234003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:46.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.359+0000 7f224e6df640 1 --2- 172.21.15.43:0/744495830 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22340039b0 0x7f2234003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.359+0000 7f224e6df640 1 -- 172.21.15.43:0/744495830 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2234003520 con 0x7f22340051c0 2024-04-25T09:00:46.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.359+0000 7f224e6df640 1 -- 172.21.15.43:0/744495830 reap_dead start 2024-04-25T09:00:46.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.360+0000 7f224e6df640 1 --2- 172.21.15.43:0/744495830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22340051c0 0x7f223400b9c0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f2240009f90 tx=0x7f2234005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1e4db5288656103 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:46.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.360+0000 7f224cedc640 1 -- 172.21.15.43:0/744495830 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2240035020 con 0x7f22340051c0 2024-04-25T09:00:46.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.360+0000 7f224cedc640 1 -- 172.21.15.43:0/744495830 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2240003b90 con 0x7f22340051c0 2024-04-25T09:00:46.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.360+0000 7f224cedc640 1 -- 172.21.15.43:0/744495830 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2240009c10 con 0x7f22340051c0 2024-04-25T09:00:46.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.360+0000 7f2250142640 1 -- 172.21.15.43:0/744495830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22340051c0 msgr2=0x7f223400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.360+0000 7f2250142640 1 --2- 172.21.15.43:0/744495830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22340051c0 0x7f223400b9c0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f2240009f90 tx=0x7f2234005830 comp rx=0 tx=0).stop 2024-04-25T09:00:46.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.360+0000 7f2250142640 1 -- 172.21.15.43:0/744495830 shutdown_connections 2024-04-25T09:00:46.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.360+0000 7f2250142640 1 --2- 172.21.15.43:0/744495830 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22340051c0 0x7f223400b9c0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.360+0000 7f2250142640 1 -- 172.21.15.43:0/744495830 >> 172.21.15.43:0/744495830 conn(0x7f224806cbd0 msgr2=0x7f224806cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:46.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.360+0000 7f2250142640 1 -- 172.21.15.43:0/744495830 shutdown_connections 2024-04-25T09:00:46.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.360+0000 7f2250142640 1 -- 172.21.15.43:0/744495830 wait complete. 2024-04-25T09:00:46.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.361+0000 7f2250142640 1 Processor -- start 2024-04-25T09:00:46.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.362+0000 7f2250142640 1 -- start start 2024-04-25T09:00:46.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.362+0000 7f2250142640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22340051c0 0x7f22480cbb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.362+0000 7f2250142640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22480cd1a0 0x7f224814a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.362+0000 7f2250142640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2248160b90 0x7f224815e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.362+0000 7f2250142640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22480d2970 con 0x7f22340051c0 2024-04-25T09:00:46.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.362+0000 7f2250142640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22480d2170 con 0x7f22480cd1a0 2024-04-25T09:00:46.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.362+0000 7f2250142640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22480ce260 con 0x7f2248160b90 2024-04-25T09:00:46.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.362+0000 7f224dede640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22340051c0 0x7f22480cbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.362+0000 7f224dede640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22340051c0 0x7f22480cbb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43782/0 (socket says 172.21.15.43:43782) 2024-04-25T09:00:46.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.362+0000 7f224dede640 1 -- 172.21.15.43:0/4222761929 learned_addr learned my addr 172.21.15.43:0/4222761929 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:46.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.363+0000 7f224e6df640 1 --2- 172.21.15.43:0/4222761929 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2248160b90 0x7f224815e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.363+0000 7f224d6dd640 1 --2- 172.21.15.43:0/4222761929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22480cd1a0 0x7f224814a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.363+0000 7f224dede640 1 -- 172.21.15.43:0/4222761929 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2248160b90 msgr2=0x7f224815e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.363+0000 7f224dede640 1 --2- 172.21.15.43:0/4222761929 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2248160b90 0x7f224815e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.363+0000 7f224dede640 1 -- 172.21.15.43:0/4222761929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22480cd1a0 msgr2=0x7f224814a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.363+0000 7f224dede640 1 --2- 172.21.15.43:0/4222761929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22480cd1a0 0x7f224814a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.363+0000 7f224dede640 1 -- 172.21.15.43:0/4222761929 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f224814a550 con 0x7f22340051c0 2024-04-25T09:00:46.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.363+0000 7f224dede640 1 --2- 172.21.15.43:0/4222761929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22340051c0 0x7f22480cbb20 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f224400f5f0 tx=0x7f224400fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:46.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.363+0000 7f223e7fc640 1 -- 172.21.15.43:0/4222761929 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22440052e0 con 0x7f22340051c0 2024-04-25T09:00:46.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.363+0000 7f223e7fc640 1 -- 172.21.15.43:0/4222761929 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2244018070 con 0x7f22340051c0 2024-04-25T09:00:46.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.363+0000 7f223e7fc640 1 -- 172.21.15.43:0/4222761929 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2244003810 con 0x7f22340051c0 2024-04-25T09:00:46.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.363+0000 7f2250142640 1 -- 172.21.15.43:0/4222761929 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22480cc100 con 0x7f22340051c0 2024-04-25T09:00:46.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.363+0000 7f2250142640 1 -- 172.21.15.43:0/4222761929 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22480cc6c0 con 0x7f22340051c0 2024-04-25T09:00:46.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.364+0000 7f2250142640 1 -- 172.21.15.43:0/4222761929 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2248074ef0 con 0x7f22340051c0 2024-04-25T09:00:46.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.364+0000 7f223e7fc640 1 -- 172.21.15.43:0/4222761929 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f22440039b0 con 0x7f22340051c0 2024-04-25T09:00:46.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.365+0000 7f223e7fc640 1 --2- 172.21.15.43:0/4222761929 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f222c06b500 0x7f222c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.365+0000 7f223e7fc640 1 -- 172.21.15.43:0/4222761929 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2244089fd0 con 0x7f22340051c0 2024-04-25T09:00:46.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.365+0000 7f224d6dd640 1 --2- 172.21.15.43:0/4222761929 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f222c06b500 0x7f222c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.366+0000 7f224d6dd640 1 --2- 172.21.15.43:0/4222761929 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f222c06b500 0x7f222c06d9c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f2238001050 tx=0x7f2238009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:46.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.369+0000 7f223e7fc640 1 -- 172.21.15.43:0/4222761929 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2244058db0 con 0x7f22340051c0 2024-04-25T09:00:46.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.544+0000 7f2250142640 1 -- 172.21.15.43:0/4222761929 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f22480d33e0 con 0x7f222c06b500 2024-04-25T09:00:46.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.550+0000 7f223e7fc640 1 -- 172.21.15.43:0/4222761929 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252016 (secure 0 0 0) 0x7f22480d33e0 con 0x7f222c06b500 2024-04-25T09:00:46.541 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:00:46.543 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-25T09:00:46.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.553+0000 7f2250142640 1 -- 172.21.15.43:0/4222761929 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f222c06b500 msgr2=0x7f222c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.553+0000 7f2250142640 1 --2- 172.21.15.43:0/4222761929 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f222c06b500 0x7f222c06d9c0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f2238001050 tx=0x7f2238009040 comp rx=0 tx=0).stop 2024-04-25T09:00:46.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.553+0000 7f2250142640 1 -- 172.21.15.43:0/4222761929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22340051c0 msgr2=0x7f22480cbb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.553+0000 7f2250142640 1 --2- 172.21.15.43:0/4222761929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22340051c0 0x7f22480cbb20 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f224400f5f0 tx=0x7f224400fab0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.553+0000 7f2250142640 1 -- 172.21.15.43:0/4222761929 shutdown_connections 2024-04-25T09:00:46.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.553+0000 7f2250142640 1 --2- 172.21.15.43:0/4222761929 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2248160b90 0x7f224815e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.553+0000 7f2250142640 1 --2- 172.21.15.43:0/4222761929 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f222c06b500 0x7f222c06d9c0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.553+0000 7f2250142640 1 --2- 172.21.15.43:0/4222761929 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f22480cd1a0 0x7f224814a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.553+0000 7f2250142640 1 --2- 172.21.15.43:0/4222761929 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f22340051c0 0x7f22480cbb20 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.553+0000 7f2250142640 1 -- 172.21.15.43:0/4222761929 >> 172.21.15.43:0/4222761929 conn(0x7f224806cbd0 msgr2=0x7f224815a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:46.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.553+0000 7f2250142640 1 -- 172.21.15.43:0/4222761929 shutdown_connections 2024-04-25T09:00:46.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.553+0000 7f2250142640 1 -- 172.21.15.43:0/4222761929 wait complete. 2024-04-25T09:00:46.553 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":286,"stamp":"2024-04-25T09:00:45.784568+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":76,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":517280,"kb_used_data":9620,"kb_used_omap":12,"kb_used_meta":507507,"kb_avail":754457440,"statfs":{"total":773094113280,"available":772564418560,"internally_reserved":0,"allocated":9850880,"data_stored":4891219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":519687729},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.625079"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.178273+0000","last_change":"2024-04-25T09:00:43.175387+0000","last_active":"2024-04-25T09:00:43.178273+0000","last_peered":"2024-04-25T09:00:43.178273+0000","last_clean":"2024-04-25T09:00:43.178273+0000","last_became_active":"2024-04-25T09:00:43.175117+0000","last_became_peered":"2024-04-25T09:00:43.175117+0000","last_unstale":"2024-04-25T09:00:43.178273+0000","last_undegraded":"2024-04-25T09:00:43.178273+0000","last_fullsized":"2024-04-25T09:00:43.178273+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:38.416326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.192799+0000","last_change":"2024-04-25T09:00:43.190723+0000","last_active":"2024-04-25T09:00:43.192799+0000","last_peered":"2024-04-25T09:00:43.192799+0000","last_clean":"2024-04-25T09:00:43.192799+0000","last_became_active":"2024-04-25T09:00:43.189137+0000","last_became_peered":"2024-04-25T09:00:43.189137+0000","last_unstale":"2024-04-25T09:00:43.192799+0000","last_undegraded":"2024-04-25T09:00:43.192799+0000","last_fullsized":"2024-04-25T09:00:43.192799+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:20.550630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.183283+0000","last_change":"2024-04-25T09:00:43.182278+0000","last_active":"2024-04-25T09:00:43.183283+0000","last_peered":"2024-04-25T09:00:43.183283+0000","last_clean":"2024-04-25T09:00:43.183283+0000","last_became_active":"2024-04-25T09:00:43.182005+0000","last_became_peered":"2024-04-25T09:00:43.182005+0000","last_unstale":"2024-04-25T09:00:43.183283+0000","last_undegraded":"2024-04-25T09:00:43.183283+0000","last_fullsized":"2024-04-25T09:00:43.183283+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:09.550708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.189288+0000","last_change":"2024-04-25T09:00:43.187228+0000","last_active":"2024-04-25T09:00:43.189288+0000","last_peered":"2024-04-25T09:00:43.189288+0000","last_clean":"2024-04-25T09:00:43.189288+0000","last_became_active":"2024-04-25T09:00:43.186942+0000","last_became_peered":"2024-04-25T09:00:43.186942+0000","last_unstale":"2024-04-25T09:00:43.189288+0000","last_undegraded":"2024-04-25T09:00:43.189288+0000","last_fullsized":"2024-04-25T09:00:43.189288+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:43.718412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.194341+0000","last_change":"2024-04-25T09:00:43.193885+0000","last_active":"2024-04-25T09:00:43.194341+0000","last_peered":"2024-04-25T09:00:43.194341+0000","last_clean":"2024-04-25T09:00:43.194341+0000","last_became_active":"2024-04-25T09:00:43.193578+0000","last_became_peered":"2024-04-25T09:00:43.193578+0000","last_unstale":"2024-04-25T09:00:43.194341+0000","last_undegraded":"2024-04-25T09:00:43.194341+0000","last_fullsized":"2024-04-25T09:00:43.194341+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:14.712980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.181417+0000","last_change":"2024-04-25T09:00:43.180105+0000","last_active":"2024-04-25T09:00:43.181417+0000","last_peered":"2024-04-25T09:00:43.181417+0000","last_clean":"2024-04-25T09:00:43.181417+0000","last_became_active":"2024-04-25T09:00:43.179082+0000","last_became_peered":"2024-04-25T09:00:43.179082+0000","last_unstale":"2024-04-25T09:00:43.181417+0000","last_undegraded":"2024-04-25T09:00:43.181417+0000","last_fullsized":"2024-04-25T09:00:43.181417+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:25.325832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.180054+0000","last_change":"2024-04-25T09:00:43.178271+0000","last_active":"2024-04-25T09:00:43.180054+0000","last_peered":"2024-04-25T09:00:43.180054+0000","last_clean":"2024-04-25T09:00:43.180054+0000","last_became_active":"2024-04-25T09:00:43.177965+0000","last_became_peered":"2024-04-25T09:00:43.177965+0000","last_unstale":"2024-04-25T09:00:43.180054+0000","last_undegraded":"2024-04-25T09:00:43.180054+0000","last_fullsized":"2024-04-25T09:00:43.180054+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:18.452112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.187838+0000","last_change":"2024-04-25T09:00:43.186910+0000","last_active":"2024-04-25T09:00:43.187838+0000","last_peered":"2024-04-25T09:00:43.187838+0000","last_clean":"2024-04-25T09:00:43.187838+0000","last_became_active":"2024-04-25T09:00:43.186430+0000","last_became_peered":"2024-04-25T09:00:43.186430+0000","last_unstale":"2024-04-25T09:00:43.187838+0000","last_undegraded":"2024-04-25T09:00:43.187838+0000","last_fullsized":"2024-04-25T09:00:43.187838+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:12.523164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.182690+0000","last_change":"2024-04-25T09:00:43.181731+0000","last_active":"2024-04-25T09:00:43.182690+0000","last_peered":"2024-04-25T09:00:43.182690+0000","last_clean":"2024-04-25T09:00:43.182690+0000","last_became_active":"2024-04-25T09:00:43.181437+0000","last_became_peered":"2024-04-25T09:00:43.181437+0000","last_unstale":"2024-04-25T09:00:43.182690+0000","last_undegraded":"2024-04-25T09:00:43.182690+0000","last_fullsized":"2024-04-25T09:00:43.182690+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:14.712980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.181660+0000","last_change":"2024-04-25T09:00:43.178468+0000","last_active":"2024-04-25T09:00:43.181660+0000","last_peered":"2024-04-25T09:00:43.181660+0000","last_clean":"2024-04-25T09:00:43.181660+0000","last_became_active":"2024-04-25T09:00:43.178113+0000","last_became_peered":"2024-04-25T09:00:43.178113+0000","last_unstale":"2024-04-25T09:00:43.181660+0000","last_undegraded":"2024-04-25T09:00:43.181660+0000","last_fullsized":"2024-04-25T09:00:43.181660+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:25.325832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.193105+0000","last_change":"2024-04-25T09:00:43.192022+0000","last_active":"2024-04-25T09:00:43.193105+0000","last_peered":"2024-04-25T09:00:43.193105+0000","last_clean":"2024-04-25T09:00:43.193105+0000","last_became_active":"2024-04-25T09:00:43.191700+0000","last_became_peered":"2024-04-25T09:00:43.191700+0000","last_unstale":"2024-04-25T09:00:43.193105+0000","last_undegraded":"2024-04-25T09:00:43.193105+0000","last_fullsized":"2024-04-25T09:00:43.193105+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:53.978092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.181988+0000","last_change":"2024-04-25T09:00:43.178794+0000","last_active":"2024-04-25T09:00:43.181988+0000","last_peered":"2024-04-25T09:00:43.181988+0000","last_clean":"2024-04-25T09:00:43.181988+0000","last_became_active":"2024-04-25T09:00:43.178541+0000","last_became_peered":"2024-04-25T09:00:43.178541+0000","last_unstale":"2024-04-25T09:00:43.181988+0000","last_undegraded":"2024-04-25T09:00:43.181988+0000","last_fullsized":"2024-04-25T09:00:43.181988+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:14.712980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.178781+0000","last_change":"2024-04-25T09:00:43.178202+0000","last_active":"2024-04-25T09:00:43.178781+0000","last_peered":"2024-04-25T09:00:43.178781+0000","last_clean":"2024-04-25T09:00:43.178781+0000","last_became_active":"2024-04-25T09:00:43.177917+0000","last_became_peered":"2024-04-25T09:00:43.177917+0000","last_unstale":"2024-04-25T09:00:43.178781+0000","last_undegraded":"2024-04-25T09:00:43.178781+0000","last_fullsized":"2024-04-25T09:00:43.178781+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:16.230988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.189231+0000","last_change":"2024-04-25T09:00:43.187981+0000","last_active":"2024-04-25T09:00:43.189231+0000","last_peered":"2024-04-25T09:00:43.189231+0000","last_clean":"2024-04-25T09:00:43.189231+0000","last_became_active":"2024-04-25T09:00:43.187699+0000","last_became_peered":"2024-04-25T09:00:43.187699+0000","last_unstale":"2024-04-25T09:00:43.189231+0000","last_undegraded":"2024-04-25T09:00:43.189231+0000","last_fullsized":"2024-04-25T09:00:43.189231+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:41.153048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.180591+0000","last_change":"2024-04-25T09:00:43.179345+0000","last_active":"2024-04-25T09:00:43.180591+0000","last_peered":"2024-04-25T09:00:43.180591+0000","last_clean":"2024-04-25T09:00:43.180591+0000","last_became_active":"2024-04-25T09:00:43.179044+0000","last_became_peered":"2024-04-25T09:00:43.179044+0000","last_unstale":"2024-04-25T09:00:43.180591+0000","last_undegraded":"2024-04-25T09:00:43.180591+0000","last_fullsized":"2024-04-25T09:00:43.180591+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:51.285995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.173802+0000","last_change":"2024-04-25T09:00:43.172627+0000","last_active":"2024-04-25T09:00:43.173802+0000","last_peered":"2024-04-25T09:00:43.173802+0000","last_clean":"2024-04-25T09:00:43.173802+0000","last_became_active":"2024-04-25T09:00:43.172325+0000","last_became_peered":"2024-04-25T09:00:43.172325+0000","last_unstale":"2024-04-25T09:00:43.173802+0000","last_undegraded":"2024-04-25T09:00:43.173802+0000","last_fullsized":"2024-04-25T09:00:43.173802+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:39.516623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.176370+0000","last_change":"2024-04-25T09:00:43.174285+0000","last_active":"2024-04-25T09:00:43.176370+0000","last_peered":"2024-04-25T09:00:43.176370+0000","last_clean":"2024-04-25T09:00:43.176370+0000","last_became_active":"2024-04-25T09:00:43.173939+0000","last_became_peered":"2024-04-25T09:00:43.173939+0000","last_unstale":"2024-04-25T09:00:43.176370+0000","last_undegraded":"2024-04-25T09:00:43.176370+0000","last_fullsized":"2024-04-25T09:00:43.176370+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:56.776489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.176669+0000","last_change":"2024-04-25T09:00:43.173547+0000","last_active":"2024-04-25T09:00:43.176669+0000","last_peered":"2024-04-25T09:00:43.176669+0000","last_clean":"2024-04-25T09:00:43.176669+0000","last_became_active":"2024-04-25T09:00:43.173267+0000","last_became_peered":"2024-04-25T09:00:43.173267+0000","last_unstale":"2024-04-25T09:00:43.176669+0000","last_undegraded":"2024-04-25T09:00:43.176669+0000","last_fullsized":"2024-04-25T09:00:43.176669+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:31:16.385240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.184130+0000","last_change":"2024-04-25T09:00:43.183175+0000","last_active":"2024-04-25T09:00:43.184130+0000","last_peered":"2024-04-25T09:00:43.184130+0000","last_clean":"2024-04-25T09:00:43.184130+0000","last_became_active":"2024-04-25T09:00:43.180734+0000","last_became_peered":"2024-04-25T09:00:43.180734+0000","last_unstale":"2024-04-25T09:00:43.184130+0000","last_undegraded":"2024-04-25T09:00:43.184130+0000","last_fullsized":"2024-04-25T09:00:43.184130+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:13.214085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.189266+0000","last_change":"2024-04-25T09:00:43.184441+0000","last_active":"2024-04-25T09:00:43.189266+0000","last_peered":"2024-04-25T09:00:43.189266+0000","last_clean":"2024-04-25T09:00:43.189266+0000","last_became_active":"2024-04-25T09:00:43.184129+0000","last_became_peered":"2024-04-25T09:00:43.184129+0000","last_unstale":"2024-04-25T09:00:43.189266+0000","last_undegraded":"2024-04-25T09:00:43.189266+0000","last_fullsized":"2024-04-25T09:00:43.189266+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:41.658911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.188994+0000","last_change":"2024-04-25T09:00:43.182774+0000","last_active":"2024-04-25T09:00:43.188994+0000","last_peered":"2024-04-25T09:00:43.188994+0000","last_clean":"2024-04-25T09:00:43.188994+0000","last_became_active":"2024-04-25T09:00:43.182496+0000","last_became_peered":"2024-04-25T09:00:43.182496+0000","last_unstale":"2024-04-25T09:00:43.188994+0000","last_undegraded":"2024-04-25T09:00:43.188994+0000","last_fullsized":"2024-04-25T09:00:43.188994+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:13.008317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.192835+0000","last_change":"2024-04-25T09:00:43.185890+0000","last_active":"2024-04-25T09:00:43.192835+0000","last_peered":"2024-04-25T09:00:43.192835+0000","last_clean":"2024-04-25T09:00:43.192835+0000","last_became_active":"2024-04-25T09:00:43.185384+0000","last_became_peered":"2024-04-25T09:00:43.185384+0000","last_unstale":"2024-04-25T09:00:43.192835+0000","last_undegraded":"2024-04-25T09:00:43.192835+0000","last_fullsized":"2024-04-25T09:00:43.192835+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:45.080353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.182042+0000","last_change":"2024-04-25T09:00:43.180540+0000","last_active":"2024-04-25T09:00:43.182042+0000","last_peered":"2024-04-25T09:00:43.182042+0000","last_clean":"2024-04-25T09:00:43.182042+0000","last_became_active":"2024-04-25T09:00:43.180116+0000","last_became_peered":"2024-04-25T09:00:43.180116+0000","last_unstale":"2024-04-25T09:00:43.182042+0000","last_undegraded":"2024-04-25T09:00:43.182042+0000","last_fullsized":"2024-04-25T09:00:43.182042+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:56.178712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.189034+0000","last_change":"2024-04-25T09:00:43.180346+0000","last_active":"2024-04-25T09:00:43.189034+0000","last_peered":"2024-04-25T09:00:43.189034+0000","last_clean":"2024-04-25T09:00:43.189034+0000","last_became_active":"2024-04-25T09:00:43.179869+0000","last_became_peered":"2024-04-25T09:00:43.179869+0000","last_unstale":"2024-04-25T09:00:43.189034+0000","last_undegraded":"2024-04-25T09:00:43.189034+0000","last_fullsized":"2024-04-25T09:00:43.189034+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:13:40.768173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.192254+0000","last_change":"2024-04-25T09:00:43.185325+0000","last_active":"2024-04-25T09:00:43.192254+0000","last_peered":"2024-04-25T09:00:43.192254+0000","last_clean":"2024-04-25T09:00:43.192254+0000","last_became_active":"2024-04-25T09:00:43.185034+0000","last_became_peered":"2024-04-25T09:00:43.185034+0000","last_unstale":"2024-04-25T09:00:43.192254+0000","last_undegraded":"2024-04-25T09:00:43.192254+0000","last_fullsized":"2024-04-25T09:00:43.192254+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:13.984050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.192764+0000","last_change":"2024-04-25T09:00:43.192342+0000","last_active":"2024-04-25T09:00:43.192764+0000","last_peered":"2024-04-25T09:00:43.192764+0000","last_clean":"2024-04-25T09:00:43.192764+0000","last_became_active":"2024-04-25T09:00:43.192036+0000","last_became_peered":"2024-04-25T09:00:43.192036+0000","last_unstale":"2024-04-25T09:00:43.192764+0000","last_undegraded":"2024-04-25T09:00:43.192764+0000","last_fullsized":"2024-04-25T09:00:43.192764+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:58.838562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.187621+0000","last_change":"2024-04-25T09:00:43.182867+0000","last_active":"2024-04-25T09:00:43.187621+0000","last_peered":"2024-04-25T09:00:43.187621+0000","last_clean":"2024-04-25T09:00:43.187621+0000","last_became_active":"2024-04-25T09:00:43.182512+0000","last_became_peered":"2024-04-25T09:00:43.182512+0000","last_unstale":"2024-04-25T09:00:43.187621+0000","last_undegraded":"2024-04-25T09:00:43.187621+0000","last_fullsized":"2024-04-25T09:00:43.187621+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:39.680627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.177825+0000","last_change":"2024-04-25T09:00:43.174572+0000","last_active":"2024-04-25T09:00:43.177825+0000","last_peered":"2024-04-25T09:00:43.177825+0000","last_clean":"2024-04-25T09:00:43.177825+0000","last_became_active":"2024-04-25T09:00:43.174282+0000","last_became_peered":"2024-04-25T09:00:43.174282+0000","last_unstale":"2024-04-25T09:00:43.177825+0000","last_undegraded":"2024-04-25T09:00:43.177825+0000","last_fullsized":"2024-04-25T09:00:43.177825+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:53.912544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.193265+0000","last_change":"2024-04-25T09:00:43.192377+0000","last_active":"2024-04-25T09:00:43.193265+0000","last_peered":"2024-04-25T09:00:43.193265+0000","last_clean":"2024-04-25T09:00:43.193265+0000","last_became_active":"2024-04-25T09:00:43.191972+0000","last_became_peered":"2024-04-25T09:00:43.191972+0000","last_unstale":"2024-04-25T09:00:43.193265+0000","last_undegraded":"2024-04-25T09:00:43.193265+0000","last_fullsized":"2024-04-25T09:00:43.193265+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:31.443292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.192860+0000","last_change":"2024-04-25T09:00:43.190472+0000","last_active":"2024-04-25T09:00:43.192860+0000","last_peered":"2024-04-25T09:00:43.192860+0000","last_clean":"2024-04-25T09:00:43.192860+0000","last_became_active":"2024-04-25T09:00:43.190194+0000","last_became_peered":"2024-04-25T09:00:43.190194+0000","last_unstale":"2024-04-25T09:00:43.192860+0000","last_undegraded":"2024-04-25T09:00:43.192860+0000","last_fullsized":"2024-04-25T09:00:43.192860+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:43.741781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.187760+0000","last_change":"2024-04-25T09:00:43.184746+0000","last_active":"2024-04-25T09:00:43.187760+0000","last_peered":"2024-04-25T09:00:43.187760+0000","last_clean":"2024-04-25T09:00:43.187760+0000","last_became_active":"2024-04-25T09:00:43.183653+0000","last_became_peered":"2024-04-25T09:00:43.183653+0000","last_unstale":"2024-04-25T09:00:43.187760+0000","last_undegraded":"2024-04-25T09:00:43.187760+0000","last_fullsized":"2024-04-25T09:00:43.187760+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:25.325832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.192564+0000","last_change":"2024-04-25T09:00:43.191974+0000","last_active":"2024-04-25T09:00:43.192564+0000","last_peered":"2024-04-25T09:00:43.192564+0000","last_clean":"2024-04-25T09:00:43.192564+0000","last_became_active":"2024-04-25T09:00:43.191589+0000","last_became_peered":"2024-04-25T09:00:43.191589+0000","last_unstale":"2024-04-25T09:00:43.192564+0000","last_undegraded":"2024-04-25T09:00:43.192564+0000","last_fullsized":"2024-04-25T09:00:43.192564+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:13.214085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.177396+0000","last_change":"2024-04-25T09:00:43.175200+0000","last_active":"2024-04-25T09:00:43.177396+0000","last_peered":"2024-04-25T09:00:43.177396+0000","last_clean":"2024-04-25T09:00:43.177396+0000","last_became_active":"2024-04-25T09:00:43.174712+0000","last_became_peered":"2024-04-25T09:00:43.174712+0000","last_unstale":"2024-04-25T09:00:43.177396+0000","last_undegraded":"2024-04-25T09:00:43.177396+0000","last_fullsized":"2024-04-25T09:00:43.177396+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:09.447125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"88'4","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.190053+0000","last_change":"2024-04-25T09:00:42.180799+0000","last_active":"2024-04-25T09:00:43.190053+0000","last_peered":"2024-04-25T09:00:43.190053+0000","last_clean":"2024-04-25T09:00:43.190053+0000","last_became_active":"2024-04-25T09:00:42.180501+0000","last_became_peered":"2024-04-25T09:00:42.180501+0000","last_unstale":"2024-04-25T09:00:43.190053+0000","last_undegraded":"2024-04-25T09:00:43.190053+0000","last_fullsized":"2024-04-25T09:00:43.190053+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:08.445099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.182698+0000","last_change":"2024-04-25T09:00:43.181086+0000","last_active":"2024-04-25T09:00:43.182698+0000","last_peered":"2024-04-25T09:00:43.182698+0000","last_clean":"2024-04-25T09:00:43.182698+0000","last_became_active":"2024-04-25T09:00:43.180161+0000","last_became_peered":"2024-04-25T09:00:43.180161+0000","last_unstale":"2024-04-25T09:00:43.182698+0000","last_undegraded":"2024-04-25T09:00:43.182698+0000","last_fullsized":"2024-04-25T09:00:43.182698+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:14.086119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"88'1","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.188311+0000","last_change":"2024-04-25T09:00:42.177529+0000","last_active":"2024-04-25T09:00:43.188311+0000","last_peered":"2024-04-25T09:00:43.188311+0000","last_clean":"2024-04-25T09:00:43.188311+0000","last_became_active":"2024-04-25T09:00:42.177224+0000","last_became_peered":"2024-04-25T09:00:42.177224+0000","last_unstale":"2024-04-25T09:00:43.188311+0000","last_undegraded":"2024-04-25T09:00:43.188311+0000","last_fullsized":"2024-04-25T09:00:43.188311+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:52.324815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.187236+0000","last_change":"2024-04-25T09:00:43.186525+0000","last_active":"2024-04-25T09:00:43.187236+0000","last_peered":"2024-04-25T09:00:43.187236+0000","last_clean":"2024-04-25T09:00:43.187236+0000","last_became_active":"2024-04-25T09:00:43.186206+0000","last_became_peered":"2024-04-25T09:00:43.186206+0000","last_unstale":"2024-04-25T09:00:43.187236+0000","last_undegraded":"2024-04-25T09:00:43.187236+0000","last_fullsized":"2024-04-25T09:00:43.187236+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:12.895798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"88'2","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.197931+0000","last_change":"2024-04-25T09:00:42.182572+0000","last_active":"2024-04-25T09:00:43.197931+0000","last_peered":"2024-04-25T09:00:43.197931+0000","last_clean":"2024-04-25T09:00:43.197931+0000","last_became_active":"2024-04-25T09:00:42.182284+0000","last_became_peered":"2024-04-25T09:00:42.182284+0000","last_unstale":"2024-04-25T09:00:43.197931+0000","last_undegraded":"2024-04-25T09:00:43.197931+0000","last_fullsized":"2024-04-25T09:00:43.197931+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:11.521139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.188375+0000","last_change":"2024-04-25T09:00:43.187984+0000","last_active":"2024-04-25T09:00:43.188375+0000","last_peered":"2024-04-25T09:00:43.188375+0000","last_clean":"2024-04-25T09:00:43.188375+0000","last_became_active":"2024-04-25T09:00:43.187670+0000","last_became_peered":"2024-04-25T09:00:43.187670+0000","last_unstale":"2024-04-25T09:00:43.188375+0000","last_undegraded":"2024-04-25T09:00:43.188375+0000","last_fullsized":"2024-04-25T09:00:43.188375+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:29.639967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"88'2","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.197964+0000","last_change":"2024-04-25T09:00:42.183307+0000","last_active":"2024-04-25T09:00:43.197964+0000","last_peered":"2024-04-25T09:00:43.197964+0000","last_clean":"2024-04-25T09:00:43.197964+0000","last_became_active":"2024-04-25T09:00:42.183004+0000","last_became_peered":"2024-04-25T09:00:42.183004+0000","last_unstale":"2024-04-25T09:00:43.197964+0000","last_undegraded":"2024-04-25T09:00:43.197964+0000","last_fullsized":"2024-04-25T09:00:43.197964+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:24.323807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.192273+0000","last_change":"2024-04-25T09:00:43.185640+0000","last_active":"2024-04-25T09:00:43.192273+0000","last_peered":"2024-04-25T09:00:43.192273+0000","last_clean":"2024-04-25T09:00:43.192273+0000","last_became_active":"2024-04-25T09:00:43.185012+0000","last_became_peered":"2024-04-25T09:00:43.185012+0000","last_unstale":"2024-04-25T09:00:43.192273+0000","last_undegraded":"2024-04-25T09:00:43.192273+0000","last_fullsized":"2024-04-25T09:00:43.192273+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:46.924567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"88'1","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.193467+0000","last_change":"2024-04-25T09:00:42.181945+0000","last_active":"2024-04-25T09:00:43.193467+0000","last_peered":"2024-04-25T09:00:43.193467+0000","last_clean":"2024-04-25T09:00:43.193467+0000","last_became_active":"2024-04-25T09:00:42.181439+0000","last_became_peered":"2024-04-25T09:00:42.181439+0000","last_unstale":"2024-04-25T09:00:43.193467+0000","last_undegraded":"2024-04-25T09:00:43.193467+0000","last_fullsized":"2024-04-25T09:00:43.193467+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:10.300335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.181311+0000","last_change":"2024-04-25T09:00:43.177894+0000","last_active":"2024-04-25T09:00:43.181311+0000","last_peered":"2024-04-25T09:00:43.181311+0000","last_clean":"2024-04-25T09:00:43.181311+0000","last_became_active":"2024-04-25T09:00:43.177607+0000","last_became_peered":"2024-04-25T09:00:43.177607+0000","last_unstale":"2024-04-25T09:00:43.181311+0000","last_undegraded":"2024-04-25T09:00:43.181311+0000","last_fullsized":"2024-04-25T09:00:43.181311+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:17.480739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"88'1","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.191608+0000","last_change":"2024-04-25T09:00:42.183055+0000","last_active":"2024-04-25T09:00:43.191608+0000","last_peered":"2024-04-25T09:00:43.191608+0000","last_clean":"2024-04-25T09:00:43.191608+0000","last_became_active":"2024-04-25T09:00:42.182742+0000","last_became_peered":"2024-04-25T09:00:42.182742+0000","last_unstale":"2024-04-25T09:00:43.191608+0000","last_undegraded":"2024-04-25T09:00:43.191608+0000","last_fullsized":"2024-04-25T09:00:43.191608+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:52.976067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.182591+0000","last_change":"2024-04-25T09:00:43.181180+0000","last_active":"2024-04-25T09:00:43.182591+0000","last_peered":"2024-04-25T09:00:43.182591+0000","last_clean":"2024-04-25T09:00:43.182591+0000","last_became_active":"2024-04-25T09:00:43.180923+0000","last_became_peered":"2024-04-25T09:00:43.180923+0000","last_unstale":"2024-04-25T09:00:43.182591+0000","last_undegraded":"2024-04-25T09:00:43.182591+0000","last_fullsized":"2024-04-25T09:00:43.182591+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:44.985514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.174839+0000","last_change":"2024-04-25T09:00:42.172911+0000","last_active":"2024-04-25T09:00:43.174839+0000","last_peered":"2024-04-25T09:00:43.174839+0000","last_clean":"2024-04-25T09:00:43.174839+0000","last_became_active":"2024-04-25T09:00:42.172632+0000","last_became_peered":"2024-04-25T09:00:42.172632+0000","last_unstale":"2024-04-25T09:00:43.174839+0000","last_undegraded":"2024-04-25T09:00:43.174839+0000","last_fullsized":"2024-04-25T09:00:43.174839+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:55.176686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.180075+0000","last_change":"2024-04-25T09:00:43.178947+0000","last_active":"2024-04-25T09:00:43.180075+0000","last_peered":"2024-04-25T09:00:43.180075+0000","last_clean":"2024-04-25T09:00:43.180075+0000","last_became_active":"2024-04-25T09:00:43.178489+0000","last_became_peered":"2024-04-25T09:00:43.178489+0000","last_unstale":"2024-04-25T09:00:43.180075+0000","last_undegraded":"2024-04-25T09:00:43.180075+0000","last_fullsized":"2024-04-25T09:00:43.180075+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:48.733245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.181652+0000","last_change":"2024-04-25T09:00:42.180245+0000","last_active":"2024-04-25T09:00:43.181652+0000","last_peered":"2024-04-25T09:00:43.181652+0000","last_clean":"2024-04-25T09:00:43.181652+0000","last_became_active":"2024-04-25T09:00:42.179814+0000","last_became_peered":"2024-04-25T09:00:42.179814+0000","last_unstale":"2024-04-25T09:00:43.181652+0000","last_undegraded":"2024-04-25T09:00:43.181652+0000","last_fullsized":"2024-04-25T09:00:43.181652+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:57.870517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.193483+0000","last_change":"2024-04-25T09:00:43.192917+0000","last_active":"2024-04-25T09:00:43.193483+0000","last_peered":"2024-04-25T09:00:43.193483+0000","last_clean":"2024-04-25T09:00:43.193483+0000","last_became_active":"2024-04-25T09:00:43.192562+0000","last_became_peered":"2024-04-25T09:00:43.192562+0000","last_unstale":"2024-04-25T09:00:43.193483+0000","last_undegraded":"2024-04-25T09:00:43.193483+0000","last_fullsized":"2024-04-25T09:00:43.193483+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:08.566826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.171633+0000","last_change":"2024-04-25T09:00:42.182448+0000","last_active":"2024-04-25T09:00:43.171633+0000","last_peered":"2024-04-25T09:00:43.171633+0000","last_clean":"2024-04-25T09:00:43.171633+0000","last_became_active":"2024-04-25T09:00:42.182160+0000","last_became_peered":"2024-04-25T09:00:42.182160+0000","last_unstale":"2024-04-25T09:00:43.171633+0000","last_undegraded":"2024-04-25T09:00:43.171633+0000","last_fullsized":"2024-04-25T09:00:43.171633+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:38.678601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.181187+0000","last_change":"2024-04-25T09:00:43.175117+0000","last_active":"2024-04-25T09:00:43.181187+0000","last_peered":"2024-04-25T09:00:43.181187+0000","last_clean":"2024-04-25T09:00:43.181187+0000","last_became_active":"2024-04-25T09:00:43.174841+0000","last_became_peered":"2024-04-25T09:00:43.174841+0000","last_unstale":"2024-04-25T09:00:43.181187+0000","last_undegraded":"2024-04-25T09:00:43.181187+0000","last_fullsized":"2024-04-25T09:00:43.181187+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:39.680627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.170971+0000","last_change":"2024-04-25T09:00:42.178333+0000","last_active":"2024-04-25T09:00:43.170971+0000","last_peered":"2024-04-25T09:00:43.170971+0000","last_clean":"2024-04-25T09:00:43.170971+0000","last_became_active":"2024-04-25T09:00:42.177995+0000","last_became_peered":"2024-04-25T09:00:42.177995+0000","last_unstale":"2024-04-25T09:00:43.170971+0000","last_undegraded":"2024-04-25T09:00:43.170971+0000","last_fullsized":"2024-04-25T09:00:43.170971+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:24.904271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.179439+0000","last_change":"2024-04-25T09:00:43.177655+0000","last_active":"2024-04-25T09:00:43.179439+0000","last_peered":"2024-04-25T09:00:43.179439+0000","last_clean":"2024-04-25T09:00:43.179439+0000","last_became_active":"2024-04-25T09:00:43.177308+0000","last_became_peered":"2024-04-25T09:00:43.177308+0000","last_unstale":"2024-04-25T09:00:43.179439+0000","last_undegraded":"2024-04-25T09:00:43.179439+0000","last_fullsized":"2024-04-25T09:00:43.179439+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:12.895798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"88'3","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.193492+0000","last_change":"2024-04-25T09:00:42.180005+0000","last_active":"2024-04-25T09:00:43.193492+0000","last_peered":"2024-04-25T09:00:43.193492+0000","last_clean":"2024-04-25T09:00:43.193492+0000","last_became_active":"2024-04-25T09:00:42.179805+0000","last_became_peered":"2024-04-25T09:00:42.179805+0000","last_unstale":"2024-04-25T09:00:43.193492+0000","last_undegraded":"2024-04-25T09:00:43.193492+0000","last_fullsized":"2024-04-25T09:00:43.193492+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:38.514597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.187881+0000","last_change":"2024-04-25T09:00:43.183685+0000","last_active":"2024-04-25T09:00:43.187881+0000","last_peered":"2024-04-25T09:00:43.187881+0000","last_clean":"2024-04-25T09:00:43.187881+0000","last_became_active":"2024-04-25T09:00:43.183395+0000","last_became_peered":"2024-04-25T09:00:43.183395+0000","last_unstale":"2024-04-25T09:00:43.187881+0000","last_undegraded":"2024-04-25T09:00:43.187881+0000","last_fullsized":"2024-04-25T09:00:43.187881+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:38.416326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"88'6","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.196377+0000","last_change":"2024-04-25T09:00:42.174256+0000","last_active":"2024-04-25T09:00:43.196377+0000","last_peered":"2024-04-25T09:00:43.196377+0000","last_clean":"2024-04-25T09:00:43.196377+0000","last_became_active":"2024-04-25T09:00:42.173019+0000","last_became_peered":"2024-04-25T09:00:42.173019+0000","last_unstale":"2024-04-25T09:00:43.196377+0000","last_undegraded":"2024-04-25T09:00:43.196377+0000","last_fullsized":"2024-04-25T09:00:43.196377+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:16.478714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.194006+0000","last_change":"2024-04-25T09:00:43.190145+0000","last_active":"2024-04-25T09:00:43.194006+0000","last_peered":"2024-04-25T09:00:43.194006+0000","last_clean":"2024-04-25T09:00:43.194006+0000","last_became_active":"2024-04-25T09:00:43.189866+0000","last_became_peered":"2024-04-25T09:00:43.189866+0000","last_unstale":"2024-04-25T09:00:43.194006+0000","last_undegraded":"2024-04-25T09:00:43.194006+0000","last_fullsized":"2024-04-25T09:00:43.194006+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:28.230599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"88'2","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.197594+0000","last_change":"2024-04-25T09:00:42.178220+0000","last_active":"2024-04-25T09:00:43.197594+0000","last_peered":"2024-04-25T09:00:43.197594+0000","last_clean":"2024-04-25T09:00:43.197594+0000","last_became_active":"2024-04-25T09:00:42.175334+0000","last_became_peered":"2024-04-25T09:00:42.175334+0000","last_unstale":"2024-04-25T09:00:43.197594+0000","last_undegraded":"2024-04-25T09:00:43.197594+0000","last_fullsized":"2024-04-25T09:00:43.197594+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:41.096248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.186928+0000","last_change":"2024-04-25T09:00:43.182067+0000","last_active":"2024-04-25T09:00:43.186928+0000","last_peered":"2024-04-25T09:00:43.186928+0000","last_clean":"2024-04-25T09:00:43.186928+0000","last_became_active":"2024-04-25T09:00:43.181786+0000","last_became_peered":"2024-04-25T09:00:43.181786+0000","last_unstale":"2024-04-25T09:00:43.186928+0000","last_undegraded":"2024-04-25T09:00:43.186928+0000","last_fullsized":"2024-04-25T09:00:43.186928+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:17.817332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.173961+0000","last_change":"2024-04-25T09:00:42.171759+0000","last_active":"2024-04-25T09:00:43.173961+0000","last_peered":"2024-04-25T09:00:43.173961+0000","last_clean":"2024-04-25T09:00:43.173961+0000","last_became_active":"2024-04-25T09:00:42.171450+0000","last_became_peered":"2024-04-25T09:00:42.171450+0000","last_unstale":"2024-04-25T09:00:43.173961+0000","last_undegraded":"2024-04-25T09:00:43.173961+0000","last_fullsized":"2024-04-25T09:00:43.173961+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:21:55.807693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.193281+0000","last_change":"2024-04-25T09:00:43.190978+0000","last_active":"2024-04-25T09:00:43.193281+0000","last_peered":"2024-04-25T09:00:43.193281+0000","last_clean":"2024-04-25T09:00:43.193281+0000","last_became_active":"2024-04-25T09:00:43.189641+0000","last_became_peered":"2024-04-25T09:00:43.189641+0000","last_unstale":"2024-04-25T09:00:43.193281+0000","last_undegraded":"2024-04-25T09:00:43.193281+0000","last_fullsized":"2024-04-25T09:00:43.193281+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:57.507520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.168137+0000","last_change":"2024-04-25T09:00:42.171903+0000","last_active":"2024-04-25T09:00:43.168137+0000","last_peered":"2024-04-25T09:00:43.168137+0000","last_clean":"2024-04-25T09:00:43.168137+0000","last_became_active":"2024-04-25T09:00:42.171606+0000","last_became_peered":"2024-04-25T09:00:42.171606+0000","last_unstale":"2024-04-25T09:00:43.168137+0000","last_undegraded":"2024-04-25T09:00:43.168137+0000","last_fullsized":"2024-04-25T09:00:43.168137+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:08.445099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.184841+0000","last_change":"2024-04-25T09:00:43.183856+0000","last_active":"2024-04-25T09:00:43.184841+0000","last_peered":"2024-04-25T09:00:43.184841+0000","last_clean":"2024-04-25T09:00:43.184841+0000","last_became_active":"2024-04-25T09:00:43.183525+0000","last_became_peered":"2024-04-25T09:00:43.183525+0000","last_unstale":"2024-04-25T09:00:43.184841+0000","last_undegraded":"2024-04-25T09:00:43.184841+0000","last_fullsized":"2024-04-25T09:00:43.184841+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:57.490239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"88'8","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.196111+0000","last_change":"2024-04-25T09:00:42.177843+0000","last_active":"2024-04-25T09:00:43.196111+0000","last_peered":"2024-04-25T09:00:43.196111+0000","last_clean":"2024-04-25T09:00:43.196111+0000","last_became_active":"2024-04-25T09:00:42.177532+0000","last_became_peered":"2024-04-25T09:00:42.177532+0000","last_unstale":"2024-04-25T09:00:43.196111+0000","last_undegraded":"2024-04-25T09:00:43.196111+0000","last_fullsized":"2024-04-25T09:00:43.196111+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:12.908403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.176115+0000","last_change":"2024-04-25T09:00:43.175301+0000","last_active":"2024-04-25T09:00:43.176115+0000","last_peered":"2024-04-25T09:00:43.176115+0000","last_clean":"2024-04-25T09:00:43.176115+0000","last_became_active":"2024-04-25T09:00:43.174976+0000","last_became_peered":"2024-04-25T09:00:43.174976+0000","last_unstale":"2024-04-25T09:00:43.176115+0000","last_undegraded":"2024-04-25T09:00:43.176115+0000","last_fullsized":"2024-04-25T09:00:43.176115+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:45.042698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"88'1","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.187252+0000","last_change":"2024-04-25T09:00:42.176002+0000","last_active":"2024-04-25T09:00:43.187252+0000","last_peered":"2024-04-25T09:00:43.187252+0000","last_clean":"2024-04-25T09:00:43.187252+0000","last_became_active":"2024-04-25T09:00:42.175640+0000","last_became_peered":"2024-04-25T09:00:42.175640+0000","last_unstale":"2024-04-25T09:00:43.187252+0000","last_undegraded":"2024-04-25T09:00:43.187252+0000","last_fullsized":"2024-04-25T09:00:43.187252+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:39:16.478714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.181832+0000","last_change":"2024-04-25T09:00:43.176591+0000","last_active":"2024-04-25T09:00:43.181832+0000","last_peered":"2024-04-25T09:00:43.181832+0000","last_clean":"2024-04-25T09:00:43.181832+0000","last_became_active":"2024-04-25T09:00:43.176286+0000","last_became_peered":"2024-04-25T09:00:43.176286+0000","last_unstale":"2024-04-25T09:00:43.181832+0000","last_undegraded":"2024-04-25T09:00:43.181832+0000","last_fullsized":"2024-04-25T09:00:43.181832+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:13.910429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"88'2","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.197722+0000","last_change":"2024-04-25T09:00:42.181512+0000","last_active":"2024-04-25T09:00:43.197722+0000","last_peered":"2024-04-25T09:00:43.197722+0000","last_clean":"2024-04-25T09:00:43.197722+0000","last_became_active":"2024-04-25T09:00:42.181225+0000","last_became_peered":"2024-04-25T09:00:42.181225+0000","last_unstale":"2024-04-25T09:00:43.197722+0000","last_undegraded":"2024-04-25T09:00:43.197722+0000","last_fullsized":"2024-04-25T09:00:43.197722+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:31:13.084093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.187897+0000","last_change":"2024-04-25T09:00:43.184712+0000","last_active":"2024-04-25T09:00:43.187897+0000","last_peered":"2024-04-25T09:00:43.187897+0000","last_clean":"2024-04-25T09:00:43.187897+0000","last_became_active":"2024-04-25T09:00:43.184414+0000","last_became_peered":"2024-04-25T09:00:43.184414+0000","last_unstale":"2024-04-25T09:00:43.187897+0000","last_undegraded":"2024-04-25T09:00:43.187897+0000","last_fullsized":"2024-04-25T09:00:43.187897+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:13.910429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.169890+0000","last_change":"2024-04-25T09:00:42.171213+0000","last_active":"2024-04-25T09:00:43.169890+0000","last_peered":"2024-04-25T09:00:43.169890+0000","last_clean":"2024-04-25T09:00:43.169890+0000","last_became_active":"2024-04-25T09:00:42.170835+0000","last_became_peered":"2024-04-25T09:00:42.170835+0000","last_unstale":"2024-04-25T09:00:43.169890+0000","last_undegraded":"2024-04-25T09:00:43.169890+0000","last_fullsized":"2024-04-25T09:00:43.169890+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:45:13.710954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.189209+0000","last_change":"2024-04-25T09:00:43.187928+0000","last_active":"2024-04-25T09:00:43.189209+0000","last_peered":"2024-04-25T09:00:43.189209+0000","last_clean":"2024-04-25T09:00:43.189209+0000","last_became_active":"2024-04-25T09:00:43.187654+0000","last_became_peered":"2024-04-25T09:00:43.187654+0000","last_unstale":"2024-04-25T09:00:43.189209+0000","last_undegraded":"2024-04-25T09:00:43.189209+0000","last_fullsized":"2024-04-25T09:00:43.189209+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:58.872543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"88'2","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.190305+0000","last_change":"2024-04-25T09:00:42.176911+0000","last_active":"2024-04-25T09:00:43.190305+0000","last_peered":"2024-04-25T09:00:43.190305+0000","last_clean":"2024-04-25T09:00:43.190305+0000","last_became_active":"2024-04-25T09:00:42.176547+0000","last_became_peered":"2024-04-25T09:00:42.176547+0000","last_unstale":"2024-04-25T09:00:43.190305+0000","last_undegraded":"2024-04-25T09:00:43.190305+0000","last_fullsized":"2024-04-25T09:00:43.190305+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:36:52.758780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.180801+0000","last_change":"2024-04-25T09:00:43.179529+0000","last_active":"2024-04-25T09:00:43.180801+0000","last_peered":"2024-04-25T09:00:43.180801+0000","last_clean":"2024-04-25T09:00:43.180801+0000","last_became_active":"2024-04-25T09:00:43.179227+0000","last_became_peered":"2024-04-25T09:00:43.179227+0000","last_unstale":"2024-04-25T09:00:43.180801+0000","last_undegraded":"2024-04-25T09:00:43.180801+0000","last_fullsized":"2024-04-25T09:00:43.180801+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:38.416326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.165919+0000","last_change":"2024-04-25T09:00:42.183308+0000","last_active":"2024-04-25T09:00:43.165919+0000","last_peered":"2024-04-25T09:00:43.165919+0000","last_clean":"2024-04-25T09:00:43.165919+0000","last_became_active":"2024-04-25T09:00:42.183019+0000","last_became_peered":"2024-04-25T09:00:42.183019+0000","last_unstale":"2024-04-25T09:00:43.165919+0000","last_undegraded":"2024-04-25T09:00:43.165919+0000","last_fullsized":"2024-04-25T09:00:43.165919+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:02.088978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.181311+0000","last_change":"2024-04-25T09:00:43.176767+0000","last_active":"2024-04-25T09:00:43.181311+0000","last_peered":"2024-04-25T09:00:43.181311+0000","last_clean":"2024-04-25T09:00:43.181311+0000","last_became_active":"2024-04-25T09:00:43.175823+0000","last_became_peered":"2024-04-25T09:00:43.175823+0000","last_unstale":"2024-04-25T09:00:43.181311+0000","last_undegraded":"2024-04-25T09:00:43.181311+0000","last_fullsized":"2024-04-25T09:00:43.181311+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:13.568725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.165519+0000","last_change":"2024-04-25T09:00:42.178120+0000","last_active":"2024-04-25T09:00:43.165519+0000","last_peered":"2024-04-25T09:00:43.165519+0000","last_clean":"2024-04-25T09:00:43.165519+0000","last_became_active":"2024-04-25T09:00:42.177786+0000","last_became_peered":"2024-04-25T09:00:42.177786+0000","last_unstale":"2024-04-25T09:00:43.165519+0000","last_undegraded":"2024-04-25T09:00:43.165519+0000","last_fullsized":"2024-04-25T09:00:43.165519+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:25.618831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.179458+0000","last_change":"2024-04-25T09:00:43.177085+0000","last_active":"2024-04-25T09:00:43.179458+0000","last_peered":"2024-04-25T09:00:43.179458+0000","last_clean":"2024-04-25T09:00:43.179458+0000","last_became_active":"2024-04-25T09:00:43.176725+0000","last_became_peered":"2024-04-25T09:00:43.176725+0000","last_unstale":"2024-04-25T09:00:43.179458+0000","last_undegraded":"2024-04-25T09:00:43.179458+0000","last_fullsized":"2024-04-25T09:00:43.179458+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:53.912544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.172280+0000","last_change":"2024-04-25T09:00:42.179763+0000","last_active":"2024-04-25T09:00:43.172280+0000","last_peered":"2024-04-25T09:00:43.172280+0000","last_clean":"2024-04-25T09:00:43.172280+0000","last_became_active":"2024-04-25T09:00:42.179482+0000","last_became_peered":"2024-04-25T09:00:42.179482+0000","last_unstale":"2024-04-25T09:00:43.172280+0000","last_undegraded":"2024-04-25T09:00:43.172280+0000","last_fullsized":"2024-04-25T09:00:43.172280+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:12.405917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.177374+0000","last_change":"2024-04-25T09:00:43.174976+0000","last_active":"2024-04-25T09:00:43.177374+0000","last_peered":"2024-04-25T09:00:43.177374+0000","last_clean":"2024-04-25T09:00:43.177374+0000","last_became_active":"2024-04-25T09:00:43.174398+0000","last_became_peered":"2024-04-25T09:00:43.174398+0000","last_unstale":"2024-04-25T09:00:43.177374+0000","last_undegraded":"2024-04-25T09:00:43.177374+0000","last_fullsized":"2024-04-25T09:00:43.177374+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:25.906297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"88'2","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.191394+0000","last_change":"2024-04-25T09:00:42.174952+0000","last_active":"2024-04-25T09:00:43.191394+0000","last_peered":"2024-04-25T09:00:43.191394+0000","last_clean":"2024-04-25T09:00:43.191394+0000","last_became_active":"2024-04-25T09:00:42.174696+0000","last_became_peered":"2024-04-25T09:00:42.174696+0000","last_unstale":"2024-04-25T09:00:43.191394+0000","last_undegraded":"2024-04-25T09:00:43.191394+0000","last_fullsized":"2024-04-25T09:00:43.191394+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:38.529151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.182624+0000","last_change":"2024-04-25T09:00:43.179282+0000","last_active":"2024-04-25T09:00:43.182624+0000","last_peered":"2024-04-25T09:00:43.182624+0000","last_clean":"2024-04-25T09:00:43.182624+0000","last_became_active":"2024-04-25T09:00:43.179012+0000","last_became_peered":"2024-04-25T09:00:43.179012+0000","last_unstale":"2024-04-25T09:00:43.182624+0000","last_undegraded":"2024-04-25T09:00:43.182624+0000","last_fullsized":"2024-04-25T09:00:43.182624+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:31.155495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.181968+0000","last_change":"2024-04-25T09:00:42.172666+0000","last_active":"2024-04-25T09:00:43.181968+0000","last_peered":"2024-04-25T09:00:43.181968+0000","last_clean":"2024-04-25T09:00:43.181968+0000","last_became_active":"2024-04-25T09:00:42.172393+0000","last_became_peered":"2024-04-25T09:00:42.172393+0000","last_unstale":"2024-04-25T09:00:43.181968+0000","last_undegraded":"2024-04-25T09:00:43.181968+0000","last_fullsized":"2024-04-25T09:00:43.181968+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:47:40.151022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.182134+0000","last_change":"2024-04-25T09:00:43.180722+0000","last_active":"2024-04-25T09:00:43.182134+0000","last_peered":"2024-04-25T09:00:43.182134+0000","last_clean":"2024-04-25T09:00:43.182134+0000","last_became_active":"2024-04-25T09:00:43.180278+0000","last_became_peered":"2024-04-25T09:00:43.180278+0000","last_unstale":"2024-04-25T09:00:43.182134+0000","last_undegraded":"2024-04-25T09:00:43.182134+0000","last_fullsized":"2024-04-25T09:00:43.182134+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:14.216474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.170783+0000","last_change":"2024-04-25T09:00:42.177604+0000","last_active":"2024-04-25T09:00:43.170783+0000","last_peered":"2024-04-25T09:00:43.170783+0000","last_clean":"2024-04-25T09:00:43.170783+0000","last_became_active":"2024-04-25T09:00:42.177163+0000","last_became_peered":"2024-04-25T09:00:42.177163+0000","last_unstale":"2024-04-25T09:00:43.170783+0000","last_undegraded":"2024-04-25T09:00:43.170783+0000","last_fullsized":"2024-04-25T09:00:43.170783+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:55.774463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.187644+0000","last_change":"2024-04-25T09:00:43.183055+0000","last_active":"2024-04-25T09:00:43.187644+0000","last_peered":"2024-04-25T09:00:43.187644+0000","last_clean":"2024-04-25T09:00:43.187644+0000","last_became_active":"2024-04-25T09:00:43.182733+0000","last_became_peered":"2024-04-25T09:00:43.182733+0000","last_unstale":"2024-04-25T09:00:43.187644+0000","last_undegraded":"2024-04-25T09:00:43.187644+0000","last_fullsized":"2024-04-25T09:00:43.187644+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:53.912544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.170260+0000","last_change":"2024-04-25T09:00:42.176448+0000","last_active":"2024-04-25T09:00:43.170260+0000","last_peered":"2024-04-25T09:00:43.170260+0000","last_clean":"2024-04-25T09:00:43.170260+0000","last_became_active":"2024-04-25T09:00:42.175647+0000","last_became_peered":"2024-04-25T09:00:42.175647+0000","last_unstale":"2024-04-25T09:00:43.170260+0000","last_undegraded":"2024-04-25T09:00:43.170260+0000","last_fullsized":"2024-04-25T09:00:43.170260+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:52.324815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.181095+0000","last_change":"2024-04-25T09:00:43.180112+0000","last_active":"2024-04-25T09:00:43.181095+0000","last_peered":"2024-04-25T09:00:43.181095+0000","last_clean":"2024-04-25T09:00:43.181095+0000","last_became_active":"2024-04-25T09:00:43.179827+0000","last_became_peered":"2024-04-25T09:00:43.179827+0000","last_unstale":"2024-04-25T09:00:43.181095+0000","last_undegraded":"2024-04-25T09:00:43.181095+0000","last_fullsized":"2024-04-25T09:00:43.181095+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:13.910429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.174558+0000","last_change":"2024-04-25T09:00:42.176017+0000","last_active":"2024-04-25T09:00:43.174558+0000","last_peered":"2024-04-25T09:00:43.174558+0000","last_clean":"2024-04-25T09:00:43.174558+0000","last_became_active":"2024-04-25T09:00:42.175747+0000","last_became_peered":"2024-04-25T09:00:42.175747+0000","last_unstale":"2024-04-25T09:00:43.174558+0000","last_undegraded":"2024-04-25T09:00:43.174558+0000","last_fullsized":"2024-04-25T09:00:43.174558+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:13.214448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.193967+0000","last_change":"2024-04-25T09:00:43.189763+0000","last_active":"2024-04-25T09:00:43.193967+0000","last_peered":"2024-04-25T09:00:43.193967+0000","last_clean":"2024-04-25T09:00:43.193967+0000","last_became_active":"2024-04-25T09:00:43.189494+0000","last_became_peered":"2024-04-25T09:00:43.189494+0000","last_unstale":"2024-04-25T09:00:43.193967+0000","last_undegraded":"2024-04-25T09:00:43.193967+0000","last_fullsized":"2024-04-25T09:00:43.193967+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:11.461879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.166304+0000","last_change":"2024-04-25T09:00:42.182849+0000","last_active":"2024-04-25T09:00:43.166304+0000","last_peered":"2024-04-25T09:00:43.166304+0000","last_clean":"2024-04-25T09:00:43.166304+0000","last_became_active":"2024-04-25T09:00:42.182527+0000","last_became_peered":"2024-04-25T09:00:42.182527+0000","last_unstale":"2024-04-25T09:00:43.166304+0000","last_undegraded":"2024-04-25T09:00:43.166304+0000","last_fullsized":"2024-04-25T09:00:43.166304+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:30.441266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.186717+0000","last_change":"2024-04-25T09:00:43.185757+0000","last_active":"2024-04-25T09:00:43.186717+0000","last_peered":"2024-04-25T09:00:43.186717+0000","last_clean":"2024-04-25T09:00:43.186717+0000","last_became_active":"2024-04-25T09:00:43.185410+0000","last_became_peered":"2024-04-25T09:00:43.185410+0000","last_unstale":"2024-04-25T09:00:43.186717+0000","last_undegraded":"2024-04-25T09:00:43.186717+0000","last_fullsized":"2024-04-25T09:00:43.186717+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:00.337597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.170234+0000","last_change":"2024-04-25T09:00:42.176603+0000","last_active":"2024-04-25T09:00:43.170234+0000","last_peered":"2024-04-25T09:00:43.170234+0000","last_clean":"2024-04-25T09:00:43.170234+0000","last_became_active":"2024-04-25T09:00:42.176334+0000","last_became_peered":"2024-04-25T09:00:42.176334+0000","last_unstale":"2024-04-25T09:00:43.170234+0000","last_undegraded":"2024-04-25T09:00:43.170234+0000","last_fullsized":"2024-04-25T09:00:43.170234+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:10.300335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.182012+0000","last_change":"2024-04-25T09:00:43.179590+0000","last_active":"2024-04-25T09:00:43.182012+0000","last_peered":"2024-04-25T09:00:43.182012+0000","last_clean":"2024-04-25T09:00:43.182012+0000","last_became_active":"2024-04-25T09:00:43.179270+0000","last_became_peered":"2024-04-25T09:00:43.179270+0000","last_unstale":"2024-04-25T09:00:43.182012+0000","last_undegraded":"2024-04-25T09:00:43.182012+0000","last_fullsized":"2024-04-25T09:00:43.182012+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:12.523164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.166775+0000","last_change":"2024-04-25T09:00:42.176361+0000","last_active":"2024-04-25T09:00:43.166775+0000","last_peered":"2024-04-25T09:00:43.166775+0000","last_clean":"2024-04-25T09:00:43.166775+0000","last_became_active":"2024-04-25T09:00:42.176020+0000","last_became_peered":"2024-04-25T09:00:42.176020+0000","last_unstale":"2024-04-25T09:00:43.166775+0000","last_undegraded":"2024-04-25T09:00:43.166775+0000","last_fullsized":"2024-04-25T09:00:43.166775+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:12.972700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.184308+0000","last_change":"2024-04-25T09:00:43.181469+0000","last_active":"2024-04-25T09:00:43.184308+0000","last_peered":"2024-04-25T09:00:43.184308+0000","last_clean":"2024-04-25T09:00:43.184308+0000","last_became_active":"2024-04-25T09:00:43.181061+0000","last_became_peered":"2024-04-25T09:00:43.181061+0000","last_unstale":"2024-04-25T09:00:43.184308+0000","last_undegraded":"2024-04-25T09:00:43.184308+0000","last_fullsized":"2024-04-25T09:00:43.184308+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:42.155437+0000","last_clean_scrub_stamp":"2024-04-25T09:00:42.155437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:04:43.718412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"88'1","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.193829+0000","last_change":"2024-04-25T09:00:42.173305+0000","last_active":"2024-04-25T09:00:43.193829+0000","last_peered":"2024-04-25T09:00:43.193829+0000","last_clean":"2024-04-25T09:00:43.193829+0000","last_became_active":"2024-04-25T09:00:42.173016+0000","last_became_peered":"2024-04-25T09:00:42.173016+0000","last_unstale":"2024-04-25T09:00:43.193829+0000","last_undegraded":"2024-04-25T09:00:43.193829+0000","last_fullsized":"2024-04-25T09:00:43.193829+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:00:41.153411+0000","last_clean_scrub_stamp":"2024-04-25T09:00:41.153411+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:27.228573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":296,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-25T09:00:43.171227+0000","last_change":"2024-04-25T08:54:00.818044+0000","last_active":"2024-04-25T09:00:43.171227+0000","last_peered":"2024-04-25T09:00:43.171227+0000","last_clean":"2024-04-25T09:00:43.171227+0000","last_became_active":"2024-04-25T08:54:00.817098+0000","last_became_peered":"2024-04-25T08:54:00.817098+0000","last_unstale":"2024-04-25T09:00:43.171227+0000","last_undegraded":"2024-04-25T09:00:43.171227+0000","last_fullsized":"2024-04-25T09:00:43.171227+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_clean_scrub_stamp":"2024-04-25T08:53:59.739677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:48.129513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574931,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55016,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316824,"statfs":{"total":96636764160,"available":96580427776,"internally_reserved":0,"allocated":1662976,"data_stored":1049437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574932,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76648,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94295192,"statfs":{"total":96636764160,"available":96558276608,"internally_reserved":0,"allocated":1069056,"data_stored":459069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77396422},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574934,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50304,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321536,"statfs":{"total":96636764160,"available":96585252864,"internally_reserved":0,"allocated":1097728,"data_stored":467478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574932,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76644,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94295196,"statfs":{"total":96636764160,"available":96558280704,"internally_reserved":0,"allocated":1069056,"data_stored":459069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77396422},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574932,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68352,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303488,"statfs":{"total":96636764160,"available":96566771712,"internally_reserved":0,"allocated":1093632,"data_stored":467456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58612,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313228,"statfs":{"total":96636764160,"available":96576745472,"internally_reserved":0,"allocated":1085440,"data_stored":466426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72508,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94299332,"statfs":{"total":96636764160,"available":96562515968,"internally_reserved":0,"allocated":1093632,"data_stored":466038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574933,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59196,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312644,"statfs":{"total":96636764160,"available":96576147456,"internally_reserved":0,"allocated":1679360,"data_stored":1056246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:00:46.557 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:00:46.558 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T09:00:46.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.713+0000 7fe8dfb5c640 1 Processor -- start 2024-04-25T09:00:46.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.714+0000 7fe8dfb5c640 1 -- start start 2024-04-25T09:00:46.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.714+0000 7fe8dfb5c640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fe8d815de10 0x7fe8d80cf100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.714+0000 7fe8dfb5c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe8d80d26d0 0x7fe8d80b7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.714+0000 7fe8dfb5c640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8d80b68c0 con 0x7fe8d80cf640 2024-04-25T09:00:46.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.714+0000 7fe8dfb5c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8d80b7490 con 0x7fe8d80d26d0 2024-04-25T09:00:46.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.714+0000 7fe8dfb5c640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8d80b6b60 con 0x7fe8d815de10 2024-04-25T09:00:46.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.715+0000 7fe8de0f9640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe8d80d26d0 0x7fe8d80b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.715+0000 7fe8de0f9640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe8d80d26d0 0x7fe8d80b7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43796/0 (socket says 172.21.15.43:43796) 2024-04-25T09:00:46.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.715+0000 7fe8de0f9640 1 -- 172.21.15.43:0/669688000 learned_addr learned my addr 172.21.15.43:0/669688000 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:46.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.715+0000 7fe8dd8f8640 1 --2- 172.21.15.43:0/669688000 >> v2:172.21.15.193:3300/0 conn(0x7fe8d815de10 0x7fe8d80cf100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.715+0000 7fe8de0f9640 1 -- 172.21.15.43:0/669688000 >> v2:172.21.15.193:3300/0 conn(0x7fe8d815de10 msgr2=0x7fe8d80cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:46.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.715+0000 7fe8de0f9640 1 --2- 172.21.15.43:0/669688000 >> v2:172.21.15.193:3300/0 conn(0x7fe8d815de10 0x7fe8d80cf100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.715+0000 7fe8de0f9640 1 -- 172.21.15.43:0/669688000 >> v1:172.21.15.193:6789/0 conn(0x7fe8d80cf640 legacy=0x7fe8d80d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.715+0000 7fe8de0f9640 1 -- 172.21.15.43:0/669688000 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d80b6e40 con 0x7fe8d80d26d0 2024-04-25T09:00:46.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.715+0000 7fe8dd8f8640 1 --2- 172.21.15.43:0/669688000 >> v2:172.21.15.193:3300/0 conn(0x7fe8d815de10 0x7fe8d80cf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:00:46.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.716+0000 7fe8de0f9640 1 --2- 172.21.15.43:0/669688000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8d80d26d0 0x7fe8d80b7920 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fe8d808c600 tx=0x7fe8d002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e2d9e50313365ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:46.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.716+0000 7fe8dc8f6640 1 -- 172.21.15.43:0/669688000 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0036070 con 0x7fe8d80d26d0 2024-04-25T09:00:46.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.716+0000 7fe8dc8f6640 1 -- 172.21.15.43:0/669688000 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d0031600 con 0x7fe8d80d26d0 2024-04-25T09:00:46.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.716+0000 7fe8dc8f6640 1 -- 172.21.15.43:0/669688000 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d00319d0 con 0x7fe8d80d26d0 2024-04-25T09:00:46.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.716+0000 7fe8dfb5c640 1 -- 172.21.15.43:0/669688000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8d80d26d0 msgr2=0x7fe8d80b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.716+0000 7fe8dfb5c640 1 --2- 172.21.15.43:0/669688000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8d80d26d0 0x7fe8d80b7920 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fe8d808c600 tx=0x7fe8d002a730 comp rx=0 tx=0).stop 2024-04-25T09:00:46.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.716+0000 7fe8dfb5c640 1 -- 172.21.15.43:0/669688000 shutdown_connections 2024-04-25T09:00:46.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.716+0000 7fe8dfb5c640 1 --2- 172.21.15.43:0/669688000 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8d80d26d0 0x7fe8d80b7920 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.716+0000 7fe8dfb5c640 1 --2- 172.21.15.43:0/669688000 >> v2:172.21.15.193:3300/0 conn(0x7fe8d815de10 0x7fe8d80cf100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.716+0000 7fe8dfb5c640 1 -- 172.21.15.43:0/669688000 >> 172.21.15.43:0/669688000 conn(0x7fe8d80d3a70 msgr2=0x7fe8d80d3e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:46.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.716+0000 7fe8dfb5c640 1 -- 172.21.15.43:0/669688000 shutdown_connections 2024-04-25T09:00:46.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.716+0000 7fe8dfb5c640 1 -- 172.21.15.43:0/669688000 wait complete. 2024-04-25T09:00:46.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.717+0000 7fe8dfb5c640 1 Processor -- start 2024-04-25T09:00:46.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.718+0000 7fe8dfb5c640 1 -- start start 2024-04-25T09:00:46.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.718+0000 7fe8dfb5c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8d80cf640 0x7fe8d809eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.718+0000 7fe8dfb5c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8d80d26d0 0x7fe8d815bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.718+0000 7fe8dfb5c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe8d815de10 0x7fe8d81598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.718+0000 7fe8dfb5c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d80d19e0 con 0x7fe8d80cf640 2024-04-25T09:00:46.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.718+0000 7fe8dfb5c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d815e1f0 con 0x7fe8d80d26d0 2024-04-25T09:00:46.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.718+0000 7fe8dfb5c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8d80b7e60 con 0x7fe8d815de10 2024-04-25T09:00:46.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.718+0000 7fe8dd8f8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8d80cf640 0x7fe8d809eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.718+0000 7fe8dd8f8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8d80cf640 0x7fe8d809eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43802/0 (socket says 172.21.15.43:43802) 2024-04-25T09:00:46.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.718+0000 7fe8dd8f8640 1 -- 172.21.15.43:0/927010498 learned_addr learned my addr 172.21.15.43:0/927010498 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:46.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.719+0000 7fe8dd0f7640 1 --2- 172.21.15.43:0/927010498 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8d80d26d0 0x7fe8d815bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.719+0000 7fe8de0f9640 1 --2- 172.21.15.43:0/927010498 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe8d815de10 0x7fe8d81598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.719+0000 7fe8dd8f8640 1 -- 172.21.15.43:0/927010498 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe8d815de10 msgr2=0x7fe8d81598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.719+0000 7fe8dd8f8640 1 --2- 172.21.15.43:0/927010498 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe8d815de10 0x7fe8d81598b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.719+0000 7fe8dd8f8640 1 -- 172.21.15.43:0/927010498 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8d80d26d0 msgr2=0x7fe8d815bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.719+0000 7fe8dd8f8640 1 --2- 172.21.15.43:0/927010498 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8d80d26d0 0x7fe8d815bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.719+0000 7fe8dd8f8640 1 -- 172.21.15.43:0/927010498 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d80c9130 con 0x7fe8d80cf640 2024-04-25T09:00:46.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.719+0000 7fe8dd8f8640 1 --2- 172.21.15.43:0/927010498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8d80cf640 0x7fe8d809eb90 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fe8d400d7d0 tx=0x7fe8d400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:46.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.719+0000 7fe8ce7fc640 1 -- 172.21.15.43:0/927010498 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d4004030 con 0x7fe8d80cf640 2024-04-25T09:00:46.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.719+0000 7fe8ce7fc640 1 -- 172.21.15.43:0/927010498 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d40041d0 con 0x7fe8d80cf640 2024-04-25T09:00:46.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.719+0000 7fe8ce7fc640 1 -- 172.21.15.43:0/927010498 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d40096a0 con 0x7fe8d80cf640 2024-04-25T09:00:46.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.719+0000 7fe8dfb5c640 1 -- 172.21.15.43:0/927010498 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8d80c9410 con 0x7fe8d80cf640 2024-04-25T09:00:46.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.719+0000 7fe8dfb5c640 1 -- 172.21.15.43:0/927010498 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8d80cb110 con 0x7fe8d80cf640 2024-04-25T09:00:46.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.720+0000 7fe8dfb5c640 1 -- 172.21.15.43:0/927010498 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8d80680f0 con 0x7fe8d80cf640 2024-04-25T09:00:46.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.721+0000 7fe8ce7fc640 1 -- 172.21.15.43:0/927010498 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe8d4009840 con 0x7fe8d80cf640 2024-04-25T09:00:46.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.721+0000 7fe8ce7fc640 1 --2- 172.21.15.43:0/927010498 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe8bc06b4d0 0x7fe8bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:46.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.721+0000 7fe8ce7fc640 1 -- 172.21.15.43:0/927010498 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe8d408a530 con 0x7fe8d80cf640 2024-04-25T09:00:46.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.721+0000 7fe8dd0f7640 1 --2- 172.21.15.43:0/927010498 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe8bc06b4d0 0x7fe8bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:46.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.722+0000 7fe8dd0f7640 1 --2- 172.21.15.43:0/927010498 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe8bc06b4d0 0x7fe8bc06d990 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe8d815a6d0 tx=0x7fe8c8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:46.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.725+0000 7fe8ce7fc640 1 -- 172.21.15.43:0/927010498 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe8d40593b0 con 0x7fe8d80cf640 2024-04-25T09:00:46.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.975+0000 7fe8dfb5c640 1 -- 172.21.15.43:0/927010498 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fe8d8074f60 con 0x7fe8d80cf640 2024-04-25T09:00:46.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.977+0000 7fe8ce7fc640 1 -- 172.21.15.43:0/927010498 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+147 (secure 0 0 0) 0x7fe8d4058cb0 con 0x7fe8d80cf640 2024-04-25T09:00:46.966 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2024-04-25T09:00:46.966 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQD8GSpmVOjWGxAAFNmxeGi0h2D//pa6Ne3eQw== 2024-04-25T09:00:46.966 INFO:teuthology.orchestra.run.smithi043.stdout: caps mds = "allow" 2024-04-25T09:00:46.966 INFO:teuthology.orchestra.run.smithi043.stdout: caps mon = "allow r" 2024-04-25T09:00:46.966 INFO:teuthology.orchestra.run.smithi043.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T09:00:46.966 INFO:teuthology.orchestra.run.smithi043.stderr:updated caps for client.0 2024-04-25T09:00:46.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.979+0000 7fe8dfb5c640 1 -- 172.21.15.43:0/927010498 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe8bc06b4d0 msgr2=0x7fe8bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.979+0000 7fe8dfb5c640 1 --2- 172.21.15.43:0/927010498 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe8bc06b4d0 0x7fe8bc06d990 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fe8d815a6d0 tx=0x7fe8c8009040 comp rx=0 tx=0).stop 2024-04-25T09:00:46.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.979+0000 7fe8dfb5c640 1 -- 172.21.15.43:0/927010498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8d80cf640 msgr2=0x7fe8d809eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:46.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.979+0000 7fe8dfb5c640 1 --2- 172.21.15.43:0/927010498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8d80cf640 0x7fe8d809eb90 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fe8d400d7d0 tx=0x7fe8d400dc90 comp rx=0 tx=0).stop 2024-04-25T09:00:46.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.979+0000 7fe8dfb5c640 1 -- 172.21.15.43:0/927010498 shutdown_connections 2024-04-25T09:00:46.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.979+0000 7fe8dfb5c640 1 --2- 172.21.15.43:0/927010498 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe8d815de10 0x7fe8d81598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.979+0000 7fe8dfb5c640 1 --2- 172.21.15.43:0/927010498 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe8bc06b4d0 0x7fe8bc06d990 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.979+0000 7fe8dfb5c640 1 --2- 172.21.15.43:0/927010498 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe8d80d26d0 0x7fe8d815bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.979+0000 7fe8dfb5c640 1 --2- 172.21.15.43:0/927010498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe8d80cf640 0x7fe8d809eb90 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:46.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.979+0000 7fe8dfb5c640 1 -- 172.21.15.43:0/927010498 >> 172.21.15.43:0/927010498 conn(0x7fe8d80d3a70 msgr2=0x7fe8d8160d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:46.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.979+0000 7fe8dfb5c640 1 -- 172.21.15.43:0/927010498 shutdown_connections 2024-04-25T09:00:46.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:46.979+0000 7fe8dfb5c640 1 -- 172.21.15.43:0/927010498 wait complete. 2024-04-25T09:00:46.979 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:00:47.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.091+0000 7fb6a7577640 1 Processor -- start 2024-04-25T09:00:47.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.091+0000 7fb6a7577640 1 -- start start 2024-04-25T09:00:47.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.092+0000 7fb6a7577640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb6a00d09b0 0x7fb6a00b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.092+0000 7fb6a7577640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb6a00ce5e0 0x7fb6a00cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.092+0000 7fb6a7577640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6a00b4c30 con 0x7fb6a0161390 2024-04-25T09:00:47.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.092+0000 7fb6a7577640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6a00d1910 con 0x7fb6a00d09b0 2024-04-25T09:00:47.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.092+0000 7fb6a7577640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a00cf2c0 con 0x7fb6a00ce5e0 2024-04-25T09:00:47.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.092+0000 7fb6a4b12640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb6a00d09b0 0x7fb6a00b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.092+0000 7fb6a4b12640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb6a00d09b0 0x7fb6a00b6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43816/0 (socket says 172.21.15.43:43816) 2024-04-25T09:00:47.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.092+0000 7fb6a4b12640 1 -- 172.21.15.43:0/3218271834 learned_addr learned my addr 172.21.15.43:0/3218271834 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:47.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.092+0000 7fb6a5b14640 1 --2- 172.21.15.43:0/3218271834 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb6a00ce5e0 0x7fb6a00cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.092+0000 7fb6a4b12640 1 -- 172.21.15.43:0/3218271834 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb6a00ce5e0 msgr2=0x7fb6a00cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:47.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.092+0000 7fb6a4b12640 1 --2- 172.21.15.43:0/3218271834 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb6a00ce5e0 0x7fb6a00cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.092+0000 7fb6a4b12640 1 -- 172.21.15.43:0/3218271834 >> v1:172.21.15.193:6789/0 conn(0x7fb6a0161390 legacy=0x7fb6a00d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:47.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.092+0000 7fb6a4b12640 1 -- 172.21.15.43:0/3218271834 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a00cf5a0 con 0x7fb6a00d09b0 2024-04-25T09:00:47.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.093+0000 7fb6a4b12640 1 --2- 172.21.15.43:0/3218271834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6a00d09b0 0x7fb6a00b6820 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fb690009870 tx=0x7fb69002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cc9744f28c10f22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:47.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.093+0000 7fb697fff640 1 -- 172.21.15.43:0/3218271834 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb690037070 con 0x7fb6a00d09b0 2024-04-25T09:00:47.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.093+0000 7fb697fff640 1 -- 172.21.15.43:0/3218271834 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb690032440 con 0x7fb6a00d09b0 2024-04-25T09:00:47.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.093+0000 7fb697fff640 1 -- 172.21.15.43:0/3218271834 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb690032810 con 0x7fb6a00d09b0 2024-04-25T09:00:47.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.094+0000 7fb6a7577640 1 -- 172.21.15.43:0/3218271834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6a00d09b0 msgr2=0x7fb6a00b6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:47.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.094+0000 7fb6a7577640 1 --2- 172.21.15.43:0/3218271834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6a00d09b0 0x7fb6a00b6820 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fb690009870 tx=0x7fb69002a270 comp rx=0 tx=0).stop 2024-04-25T09:00:47.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.094+0000 7fb6a7577640 1 -- 172.21.15.43:0/3218271834 shutdown_connections 2024-04-25T09:00:47.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.094+0000 7fb6a7577640 1 --2- 172.21.15.43:0/3218271834 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb6a00ce5e0 0x7fb6a00cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.094+0000 7fb6a7577640 1 --2- 172.21.15.43:0/3218271834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6a00d09b0 0x7fb6a00b6820 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.094+0000 7fb6a7577640 1 -- 172.21.15.43:0/3218271834 >> 172.21.15.43:0/3218271834 conn(0x7fb6a009cd20 msgr2=0x7fb6a00d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:47.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.094+0000 7fb6a7577640 1 -- 172.21.15.43:0/3218271834 shutdown_connections 2024-04-25T09:00:47.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.094+0000 7fb6a7577640 1 -- 172.21.15.43:0/3218271834 wait complete. 2024-04-25T09:00:47.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.095+0000 7fb6a7577640 1 Processor -- start 2024-04-25T09:00:47.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.095+0000 7fb6a7577640 1 -- start start 2024-04-25T09:00:47.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.096+0000 7fb6a7577640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb6a00ce5e0 0x7fb6a00ca4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.096+0000 7fb6a7577640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6a00d09b0 0x7fb6a00c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.096+0000 7fb6a7577640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6a0161390 0x7fb6a015ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.096+0000 7fb6a7577640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a0161770 con 0x7fb6a00d09b0 2024-04-25T09:00:47.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.096+0000 7fb6a7577640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a00d15a0 con 0x7fb6a0161390 2024-04-25T09:00:47.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.096+0000 7fb6a7577640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a00cefb0 con 0x7fb6a00ce5e0 2024-04-25T09:00:47.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.096+0000 7fb6a4b12640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6a00d09b0 0x7fb6a00c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.096+0000 7fb6a4b12640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6a00d09b0 0x7fb6a00c82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43822/0 (socket says 172.21.15.43:43822) 2024-04-25T09:00:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.096+0000 7fb6a4b12640 1 -- 172.21.15.43:0/378044691 learned_addr learned my addr 172.21.15.43:0/378044691 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.096+0000 7fb6a4b12640 1 -- 172.21.15.43:0/378044691 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb6a00ce5e0 msgr2=0x7fb6a00ca4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.096+0000 7fb6a5313640 1 --2- 172.21.15.43:0/378044691 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb6a00ce5e0 0x7fb6a00ca4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.096+0000 7fb6a5b14640 1 --2- 172.21.15.43:0/378044691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6a0161390 0x7fb6a015ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.096+0000 7fb6a4b12640 1 --2- 172.21.15.43:0/378044691 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb6a00ce5e0 0x7fb6a00ca4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.096+0000 7fb6a4b12640 1 -- 172.21.15.43:0/378044691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6a0161390 msgr2=0x7fb6a015ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.096+0000 7fb6a4b12640 1 --2- 172.21.15.43:0/378044691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6a0161390 0x7fb6a015ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.096+0000 7fb6a4b12640 1 -- 172.21.15.43:0/378044691 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a00c8810 con 0x7fb6a00d09b0 2024-04-25T09:00:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.096+0000 7fb6a5b14640 1 --2- 172.21.15.43:0/378044691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6a0161390 0x7fb6a015ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.096+0000 7fb6a5313640 1 --2- 172.21.15.43:0/378044691 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb6a00ce5e0 0x7fb6a00ca4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.096+0000 7fb6a4b12640 1 --2- 172.21.15.43:0/378044691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6a00d09b0 0x7fb6a00c82d0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fb690000c00 tx=0x7fb6900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.096+0000 7fb695ffb640 1 -- 172.21.15.43:0/378044691 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb690041020 con 0x7fb6a00d09b0 2024-04-25T09:00:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.096+0000 7fb695ffb640 1 -- 172.21.15.43:0/378044691 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6900306c0 con 0x7fb6a00d09b0 2024-04-25T09:00:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.096+0000 7fb6a7577640 1 -- 172.21.15.43:0/378044691 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6a00c8ab0 con 0x7fb6a00d09b0 2024-04-25T09:00:47.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.096+0000 7fb695ffb640 1 -- 172.21.15.43:0/378044691 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb690037040 con 0x7fb6a00d09b0 2024-04-25T09:00:47.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.097+0000 7fb6a7577640 1 -- 172.21.15.43:0/378044691 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6a00cc0b0 con 0x7fb6a00d09b0 2024-04-25T09:00:47.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.098+0000 7fb6a7577640 1 -- 172.21.15.43:0/378044691 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6a00680f0 con 0x7fb6a00d09b0 2024-04-25T09:00:47.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.098+0000 7fb695ffb640 1 -- 172.21.15.43:0/378044691 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb690044020 con 0x7fb6a00d09b0 2024-04-25T09:00:47.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.098+0000 7fb695ffb640 1 --2- 172.21.15.43:0/378044691 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb68406b4d0 0x7fb68406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.099+0000 7fb695ffb640 1 -- 172.21.15.43:0/378044691 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb6900aa9d0 con 0x7fb6a00d09b0 2024-04-25T09:00:47.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.099+0000 7fb6a5313640 1 --2- 172.21.15.43:0/378044691 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb68406b4d0 0x7fb68406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.099+0000 7fb6a5313640 1 --2- 172.21.15.43:0/378044691 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb68406b4d0 0x7fb68406d990 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb69c0015e0 tx=0x7fb69c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:47.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.103+0000 7fb695ffb640 1 -- 172.21.15.43:0/378044691 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb69004b030 con 0x7fb6a00d09b0 2024-04-25T09:00:47.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.319+0000 7fb6a7577640 1 -- 172.21.15.43:0/378044691 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6a0074ef0 con 0x7fb6a00d09b0 2024-04-25T09:00:47.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.319+0000 7fb695ffb640 1 -- 172.21.15.43:0/378044691 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fb690079850 con 0x7fb6a00d09b0 2024-04-25T09:00:47.308 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:00:47.308 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":54,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6200,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2258031526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2258031526},{"type":"v1","addr":"172.21.15.43:6834","nonce":2258031526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6206,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/1773440177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1773440177},{"type":"v1","addr":"172.21.15.193:6836","nonce":1773440177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6209,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3239725307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":3239725307},{"type":"v1","addr":"172.21.15.193:6837","nonce":3239725307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:00:42.536578+0000","modified":"2024-04-25T09:00:44.531104+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7596},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7596":{"gid":7596,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/1341028696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1341028696},{"type":"v1","addr":"172.21.15.43:6835","nonce":1341028696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:00:47.309 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 54 2024-04-25T09:00:47.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.321+0000 7fb6a7577640 1 -- 172.21.15.43:0/378044691 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb68406b4d0 msgr2=0x7fb68406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:47.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.321+0000 7fb6a7577640 1 --2- 172.21.15.43:0/378044691 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb68406b4d0 0x7fb68406d990 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb69c0015e0 tx=0x7fb69c009040 comp rx=0 tx=0).stop 2024-04-25T09:00:47.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.321+0000 7fb6a7577640 1 -- 172.21.15.43:0/378044691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6a00d09b0 msgr2=0x7fb6a00c82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:47.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.321+0000 7fb6a7577640 1 --2- 172.21.15.43:0/378044691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6a00d09b0 0x7fb6a00c82d0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fb690000c00 tx=0x7fb6900046c0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.321+0000 7fb6a7577640 1 -- 172.21.15.43:0/378044691 shutdown_connections 2024-04-25T09:00:47.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.321+0000 7fb6a7577640 1 --2- 172.21.15.43:0/378044691 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb68406b4d0 0x7fb68406d990 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.321+0000 7fb6a7577640 1 --2- 172.21.15.43:0/378044691 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6a0161390 0x7fb6a015ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.321+0000 7fb6a7577640 1 --2- 172.21.15.43:0/378044691 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6a00d09b0 0x7fb6a00c82d0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.321+0000 7fb6a7577640 1 --2- 172.21.15.43:0/378044691 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb6a00ce5e0 0x7fb6a00ca4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.321+0000 7fb6a7577640 1 -- 172.21.15.43:0/378044691 >> 172.21.15.43:0/378044691 conn(0x7fb6a009cd20 msgr2=0x7fb6a00b6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:47.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.321+0000 7fb6a7577640 1 -- 172.21.15.43:0/378044691 shutdown_connections 2024-04-25T09:00:47.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.321+0000 7fb6a7577640 1 -- 172.21.15.43:0/378044691 wait complete. 2024-04-25T09:00:47.723 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 54, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:00:42.536578+0000', 'modified': '2024-04-25T09:00:44.531104+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7596}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7596': {'gid': 7596, 'name': 'a', 'rank': 0, 'incarnation': 52, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.43:6835/1341028696', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6833', 'nonce': 1341028696}, {'type': 'v1', 'addr': '172.21.15.43:6835', 'nonce': 1341028696}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:00:47.724 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:00:47.724 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T09:00:47.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.839+0000 7fe6ae86f640 1 Processor -- start 2024-04-25T09:00:47.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.840+0000 7fe6ae86f640 1 -- start start 2024-04-25T09:00:47.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.840+0000 7fe6ae86f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe6a80d39d0 0x7fe6a80d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.840+0000 7fe6ae86f640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fe6a80d2500 0x7fe6a80d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.840+0000 7fe6ae86f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6a80b6c50 con 0x7fe6a80ce2a0 2024-04-25T09:00:47.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.840+0000 7fe6ae86f640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6a815e010 con 0x7fe6a80d2500 2024-04-25T09:00:47.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.840+0000 7fe6ae86f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a80d2020 con 0x7fe6a80d39d0 2024-04-25T09:00:47.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.840+0000 7fe6a77fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe6a80ce2a0 0x7fe6a80d3db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44330/0 (socket says 172.21.15.43:44330) 2024-04-25T09:00:47.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.841+0000 7fe6a77fe640 1 -- 172.21.15.43:0/1849583909 learned_addr learned my addr 172.21.15.43:0/1849583909 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:00:47.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.841+0000 7fe6ace0c640 1 --2- 172.21.15.43:0/1849583909 >> v2:172.21.15.193:3300/0 conn(0x7fe6a80d2500 0x7fe6a80d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.841+0000 7fe6a7fff640 1 --2- 172.21.15.43:0/1849583909 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe6a80d39d0 0x7fe6a80d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.841+0000 7fe6a6ffd640 1 -- 172.21.15.43:0/1849583909 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 869525167 0 0) 0x7fe698001110 con 0x7fe6a80ce2a0 2024-04-25T09:00:47.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.841+0000 7fe6a6ffd640 1 -- 172.21.15.43:0/1849583909 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe690003520 con 0x7fe6a80ce2a0 2024-04-25T09:00:47.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.841+0000 7fe6ace0c640 1 -- 172.21.15.43:0/1849583909 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe6a80d39d0 msgr2=0x7fe6a80d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:47.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.841+0000 7fe6ace0c640 1 --2- 172.21.15.43:0/1849583909 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe6a80d39d0 0x7fe6a80d0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.841+0000 7fe6ace0c640 1 -- 172.21.15.43:0/1849583909 >> v1:172.21.15.43:6789/0 conn(0x7fe6a80ce2a0 legacy=0x7fe6a80d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:47.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.841+0000 7fe6a6ffd640 1 -- 172.21.15.43:0/1849583909 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1802316458 0 0) 0x7fe690003520 con 0x7fe6a80ce2a0 2024-04-25T09:00:47.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.841+0000 7fe6ace0c640 1 -- 172.21.15.43:0/1849583909 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a80cec40 con 0x7fe6a80d2500 2024-04-25T09:00:47.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.841+0000 7fe6a7fff640 1 --2- 172.21.15.43:0/1849583909 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe6a80d39d0 0x7fe6a80d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:00:47.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.842+0000 7fe6ace0c640 1 --2- 172.21.15.43:0/1849583909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6a80d2500 0x7fe6a80d2990 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0009d90 tx=0x7fe6a002a560 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9508735467a413e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:47.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.842+0000 7fe6a6ffd640 1 -- 172.21.15.43:0/1849583909 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe698001840 con 0x7fe6a80d2500 2024-04-25T09:00:47.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.842+0000 7fe6a6ffd640 1 -- 172.21.15.43:0/1849583909 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a0007540 con 0x7fe6a80d2500 2024-04-25T09:00:47.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.842+0000 7fe6a6ffd640 1 -- 172.21.15.43:0/1849583909 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a0007910 con 0x7fe6a80d2500 2024-04-25T09:00:47.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.842+0000 7fe6ae86f640 1 -- 172.21.15.43:0/1849583909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6a80d2500 msgr2=0x7fe6a80d2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:47.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.842+0000 7fe6ae86f640 1 --2- 172.21.15.43:0/1849583909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6a80d2500 0x7fe6a80d2990 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0009d90 tx=0x7fe6a002a560 comp rx=0 tx=0).stop 2024-04-25T09:00:47.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.843+0000 7fe6ae86f640 1 -- 172.21.15.43:0/1849583909 shutdown_connections 2024-04-25T09:00:47.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.843+0000 7fe6ae86f640 1 --2- 172.21.15.43:0/1849583909 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6a80d2500 0x7fe6a80d2990 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.843+0000 7fe6ae86f640 1 --2- 172.21.15.43:0/1849583909 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe6a80d39d0 0x7fe6a80d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.843+0000 7fe6ae86f640 1 -- 172.21.15.43:0/1849583909 >> 172.21.15.43:0/1849583909 conn(0x7fe6a806b000 msgr2=0x7fe6a806b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:47.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.843+0000 7fe6ae86f640 1 -- 172.21.15.43:0/1849583909 shutdown_connections 2024-04-25T09:00:47.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.843+0000 7fe6ae86f640 1 -- 172.21.15.43:0/1849583909 wait complete. 2024-04-25T09:00:47.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.844+0000 7fe6ae86f640 1 Processor -- start 2024-04-25T09:00:47.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.844+0000 7fe6ae86f640 1 -- start start 2024-04-25T09:00:47.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.845+0000 7fe6ae86f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6a80ce2a0 0x7fe6a80ca030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.845+0000 7fe6ae86f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe6a80d2500 0x7fe6a80c7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.845+0000 7fe6ae86f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6a80d39d0 0x7fe6a80ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.845+0000 7fe6ae86f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a80cf200 con 0x7fe6a80ce2a0 2024-04-25T09:00:47.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.845+0000 7fe6ae86f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a80b2f00 con 0x7fe6a80d39d0 2024-04-25T09:00:47.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.845+0000 7fe6ae86f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a80d2ed0 con 0x7fe6a80d2500 2024-04-25T09:00:47.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.845+0000 7fe6a7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6a80ce2a0 0x7fe6a80ca030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.845+0000 7fe6a7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6a80ce2a0 0x7fe6a80ca030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43832/0 (socket says 172.21.15.43:43832) 2024-04-25T09:00:47.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.845+0000 7fe6a7fff640 1 -- 172.21.15.43:0/2571518823 learned_addr learned my addr 172.21.15.43:0/2571518823 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:47.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.845+0000 7fe6ace0c640 1 --2- 172.21.15.43:0/2571518823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6a80d39d0 0x7fe6a80ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.845+0000 7fe6a77fe640 1 --2- 172.21.15.43:0/2571518823 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe6a80d2500 0x7fe6a80c7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.845+0000 7fe6a7fff640 1 -- 172.21.15.43:0/2571518823 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe6a80d2500 msgr2=0x7fe6a80c7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:47.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.845+0000 7fe6a7fff640 1 --2- 172.21.15.43:0/2571518823 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe6a80d2500 0x7fe6a80c7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.845+0000 7fe6a7fff640 1 -- 172.21.15.43:0/2571518823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6a80d39d0 msgr2=0x7fe6a80ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:47.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.845+0000 7fe6a7fff640 1 --2- 172.21.15.43:0/2571518823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6a80d39d0 0x7fe6a80ccca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.845+0000 7fe6a7fff640 1 -- 172.21.15.43:0/2571518823 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a80cab10 con 0x7fe6a80ce2a0 2024-04-25T09:00:47.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.845+0000 7fe6a77fe640 1 --2- 172.21.15.43:0/2571518823 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe6a80d2500 0x7fe6a80c7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:47.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.845+0000 7fe6a7fff640 1 --2- 172.21.15.43:0/2571518823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6a80ce2a0 0x7fe6a80ca030 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fe69c00e990 tx=0x7fe69c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:47.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.845+0000 7fe6a4ff9640 1 -- 172.21.15.43:0/2571518823 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c017070 con 0x7fe6a80ce2a0 2024-04-25T09:00:47.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.845+0000 7fe6a4ff9640 1 -- 172.21.15.43:0/2571518823 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe69c00c980 con 0x7fe6a80ce2a0 2024-04-25T09:00:47.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.845+0000 7fe6ae86f640 1 -- 172.21.15.43:0/2571518823 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6a80c8620 con 0x7fe6a80ce2a0 2024-04-25T09:00:47.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.845+0000 7fe6a4ff9640 1 -- 172.21.15.43:0/2571518823 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c00cd50 con 0x7fe6a80ce2a0 2024-04-25T09:00:47.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.845+0000 7fe6ae86f640 1 -- 172.21.15.43:0/2571518823 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6a80c8be0 con 0x7fe6a80ce2a0 2024-04-25T09:00:47.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.847+0000 7fe6a4ff9640 1 -- 172.21.15.43:0/2571518823 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe69c01a070 con 0x7fe6a80ce2a0 2024-04-25T09:00:47.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.847+0000 7fe6a4ff9640 1 --2- 172.21.15.43:0/2571518823 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe69406b4d0 0x7fe69406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.847+0000 7fe6a4ff9640 1 -- 172.21.15.43:0/2571518823 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe69c08a080 con 0x7fe6a80ce2a0 2024-04-25T09:00:47.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.848+0000 7fe6a77fe640 1 --2- 172.21.15.43:0/2571518823 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe69406b4d0 0x7fe69406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.848+0000 7fe6a77fe640 1 --2- 172.21.15.43:0/2571518823 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe69406b4d0 0x7fe69406d990 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe698001eb0 tx=0x7fe698009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:47.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.848+0000 7fe6b157a740 1 Processor -- start 2024-04-25T09:00:47.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.848+0000 7fe6b157a740 1 -- start start 2024-04-25T09:00:47.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.849+0000 7fe6b157a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d930d9230 0x562d930d9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.849+0000 7fe6b157a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d930f6ad0 0x562d930f7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.849+0000 7fe6b157a740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x562d930aef10 0x562d930f5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.849+0000 7fe6b157a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d93011a10 con 0x562d930f6ad0 2024-04-25T09:00:47.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.849+0000 7fe6b157a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d92fbab30 con 0x562d930d9230 2024-04-25T09:00:47.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.849+0000 7fe6b157a740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d9300dc50 con 0x562d930aef10 2024-04-25T09:00:47.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.849+0000 7fe6a77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d930f6ad0 0x562d930f7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.849+0000 7fe6a77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d930f6ad0 0x562d930f7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43836/0 (socket says 172.21.15.43:43836) 2024-04-25T09:00:47.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.849+0000 7fe6a77fe640 1 -- 172.21.15.43:0/532394315 learned_addr learned my addr 172.21.15.43:0/532394315 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:47.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.849+0000 7fe6ace0c640 1 --2- 172.21.15.43:0/532394315 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x562d930aef10 0x562d930f5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.849+0000 7fe6a7fff640 1 --2- 172.21.15.43:0/532394315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d930d9230 0x562d930d9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.849+0000 7fe6a77fe640 1 -- 172.21.15.43:0/532394315 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x562d930aef10 msgr2=0x562d930f5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:47.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.849+0000 7fe6a77fe640 1 --2- 172.21.15.43:0/532394315 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x562d930aef10 0x562d930f5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.849+0000 7fe6a77fe640 1 -- 172.21.15.43:0/532394315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d930d9230 msgr2=0x562d930d9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:47.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.849+0000 7fe6a77fe640 1 --2- 172.21.15.43:0/532394315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d930d9230 0x562d930d9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.849+0000 7fe6a77fe640 1 -- 172.21.15.43:0/532394315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d92fcfe10 con 0x562d930f6ad0 2024-04-25T09:00:47.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.849+0000 7fe6a77fe640 1 --2- 172.21.15.43:0/532394315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d930f6ad0 0x562d930f7020 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fe698000f60 tx=0x7fe698010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0e38dbf7352686e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:47.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.850+0000 7fe68dffb640 1 -- 172.21.15.43:0/532394315 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69800db50 con 0x562d930f6ad0 2024-04-25T09:00:47.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.850+0000 7fe68dffb640 1 -- 172.21.15.43:0/532394315 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe69800dd10 con 0x562d930f6ad0 2024-04-25T09:00:47.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.850+0000 7fe68dffb640 1 -- 172.21.15.43:0/532394315 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe698020690 con 0x562d930f6ad0 2024-04-25T09:00:47.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.850+0000 7fe6b157a740 1 -- 172.21.15.43:0/532394315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d930f6ad0 msgr2=0x562d930f7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:47.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.850+0000 7fe6b157a740 1 --2- 172.21.15.43:0/532394315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d930f6ad0 0x562d930f7020 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fe698000f60 tx=0x7fe698010000 comp rx=0 tx=0).stop 2024-04-25T09:00:47.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.850+0000 7fe6b157a740 1 -- 172.21.15.43:0/532394315 shutdown_connections 2024-04-25T09:00:47.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.850+0000 7fe6b157a740 1 --2- 172.21.15.43:0/532394315 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x562d930aef10 0x562d930f5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.850+0000 7fe6b157a740 1 --2- 172.21.15.43:0/532394315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d930f6ad0 0x562d930f7020 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.850+0000 7fe6b157a740 1 --2- 172.21.15.43:0/532394315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d930d9230 0x562d930d9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.850+0000 7fe6b157a740 1 -- 172.21.15.43:0/532394315 >> 172.21.15.43:0/532394315 conn(0x562d930ab840 msgr2=0x562d930f4040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:47.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.850+0000 7fe6b157a740 1 -- 172.21.15.43:0/532394315 shutdown_connections 2024-04-25T09:00:47.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.850+0000 7fe6b157a740 1 -- 172.21.15.43:0/532394315 wait complete. 2024-04-25T09:00:47.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.851+0000 7fe6b157a740 1 -- start start 2024-04-25T09:00:47.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.851+0000 7fe6b157a740 1 Processor -- start 2024-04-25T09:00:47.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.852+0000 7fe6b157a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d930d9230 0x562d930d5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.852+0000 7fe6b157a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d930f6ad0 0x562d930eb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.852+0000 7fe6b157a740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x562d931086f0 0x562d930d7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.852+0000 7fe6b157a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d930afcb0 con 0x562d930d9230 2024-04-25T09:00:47.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.852+0000 7fe6b157a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d92fce0c0 con 0x562d930f6ad0 2024-04-25T09:00:47.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.852+0000 7fe6b157a740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d930a9770 con 0x562d931086f0 2024-04-25T09:00:47.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.852+0000 7fe6b157a740 10 client.0 _ll_register_callbacks cb 0x562d92fc1ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:00:47.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.852+0000 7fe6a7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d930d9230 0x562d930d5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.852+0000 7fe6a7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d930d9230 0x562d930d5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43838/0 (socket says 172.21.15.43:43838) 2024-04-25T09:00:47.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.852+0000 7fe6a7fff640 1 -- 172.21.15.43:0/1355366603 learned_addr learned my addr 172.21.15.43:0/1355366603 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:47.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.852+0000 7fe6a77fe640 1 --2- 172.21.15.43:0/1355366603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d930f6ad0 0x562d930eb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.852+0000 7fe6ace0c640 1 --2- 172.21.15.43:0/1355366603 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x562d931086f0 0x562d930d7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.852+0000 7fe6a7fff640 1 -- 172.21.15.43:0/1355366603 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x562d931086f0 msgr2=0x562d930d7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:47.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.852+0000 7fe6a7fff640 1 --2- 172.21.15.43:0/1355366603 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x562d931086f0 0x562d930d7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.852+0000 7fe6a7fff640 1 -- 172.21.15.43:0/1355366603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d930f6ad0 msgr2=0x562d930eb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:47.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.852+0000 7fe6a7fff640 1 --2- 172.21.15.43:0/1355366603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d930f6ad0 0x562d930eb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.852+0000 7fe6a7fff640 1 -- 172.21.15.43:0/1355366603 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d92d262d0 con 0x562d930d9230 2024-04-25T09:00:47.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.852+0000 7fe6a7fff640 1 --2- 172.21.15.43:0/1355366603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d930d9230 0x562d930d5a80 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fe69c058790 tx=0x7fe69c05c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78201035a1921513 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:47.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.852+0000 7fe686ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:00:47.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.853+0000 7fe686ffd640 1 -- 172.21.15.43:0/1355366603 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c070030 con 0x562d930d9230 2024-04-25T09:00:47.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.853+0000 7fe686ffd640 1 -- 172.21.15.43:0/1355366603 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe69c0701d0 con 0x562d930d9230 2024-04-25T09:00:47.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.853+0000 7fe6b157a740 1 -- 172.21.15.43:0/1355366603 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe69800dd10 con 0x562d930d9230 2024-04-25T09:00:47.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.853+0000 7fe686ffd640 1 -- 172.21.15.43:0/1355366603 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe69c021040 con 0x562d930d9230 2024-04-25T09:00:47.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.853+0000 7fe6b157a740 10 client.7740 fetch_fsmap learned FSMap version 54 2024-04-25T09:00:47.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.853+0000 7fe6b157a740 1 -- 172.21.15.43:0/1355366603 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x562d92df06b0 con 0x562d930d9230 2024-04-25T09:00:47.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.853+0000 7fe686ffd640 1 -- 172.21.15.43:0/1355366603 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe69800dd10 con 0x562d930d9230 2024-04-25T09:00:47.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.853+0000 7fe686ffd640 1 -- 172.21.15.43:0/1355366603 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 54) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe69c024060 con 0x562d930d9230 2024-04-25T09:00:47.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.853+0000 7fe6b157a740 10 client.7740 fetch_fsmap finished waiting for FSMap version 54 2024-04-25T09:00:47.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.853+0000 7fe6b157a740 10 client.7740 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:00:47.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.853+0000 7fe6b157a740 20 client.7740 populate_metadata read hostname 'smithi043' 2024-04-25T09:00:47.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.853+0000 7fe6b157a740 1 --2- 172.21.15.43:0/1355366603 >> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] conn(0x562d930da4c0 0x562d930da8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.853+0000 7fe6b157a740 4 client.7740 mds_command: new command op to 7596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:00:47.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.853+0000 7fe6b157a740 1 -- 172.21.15.43:0/1355366603 --> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562d93070920 con 0x562d930da4c0 2024-04-25T09:00:47.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.853+0000 7fe6a77fe640 1 --2- 172.21.15.43:0/1355366603 >> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] conn(0x562d930da4c0 0x562d930da8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.854+0000 7fe6a77fe640 1 --2- 172.21.15.43:0/1355366603 >> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] conn(0x562d930da4c0 0x562d930da8a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3418b733b4e549c5 server_cookie=fa477d046da743a9 in_seq=0 out_seq=0 2024-04-25T09:00:47.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.854+0000 7fe686ffd640 10 client.7740 ms_handle_connect on v2:172.21.15.43:6833/1341028696 2024-04-25T09:00:47.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.854+0000 7fe686ffd640 1 -- 172.21.15.43:0/1355366603 <== mds.0 v2:172.21.15.43:6833/1341028696 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562d93070920 con 0x562d930da4c0 2024-04-25T09:00:47.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.854+0000 7fe686ffd640 10 client.7740 handle_command_reply: tid=0 2024-04-25T09:00:47.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.854+0000 7fe6b157a740 1 client.7740 shutdown 2024-04-25T09:00:47.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.854+0000 7fe6b157a740 1 -- 172.21.15.43:0/1355366603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d930d9230 msgr2=0x562d930d5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:47.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.854+0000 7fe6b157a740 1 --2- 172.21.15.43:0/1355366603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d930d9230 0x562d930d5a80 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fe69c058790 tx=0x7fe69c05c950 comp rx=0 tx=0).stop 2024-04-25T09:00:47.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.854+0000 7fe6b157a740 1 -- 172.21.15.43:0/1355366603 shutdown_connections 2024-04-25T09:00:47.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.854+0000 7fe6b157a740 1 --2- 172.21.15.43:0/1355366603 >> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] conn(0x562d930da4c0 0x562d930da8a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.854+0000 7fe6b157a740 1 --2- 172.21.15.43:0/1355366603 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x562d931086f0 0x562d930d7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.854+0000 7fe6b157a740 1 --2- 172.21.15.43:0/1355366603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d930f6ad0 0x562d930eb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.854+0000 7fe6b157a740 1 --2- 172.21.15.43:0/1355366603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d930d9230 0x562d930d5a80 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.854+0000 7fe686ffd640 0 client.7740 ms_handle_reset on v2:172.21.15.43:6833/1341028696 2024-04-25T09:00:47.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.854+0000 7fe6b157a740 1 -- 172.21.15.43:0/1355366603 >> 172.21.15.43:0/1355366603 conn(0x562d930aef10 msgr2=0x562d930a8980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:47.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.855+0000 7fe6b157a740 1 -- 172.21.15.43:0/1355366603 shutdown_connections 2024-04-25T09:00:47.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.855+0000 7fe6b157a740 1 -- 172.21.15.43:0/1355366603 wait complete. 2024-04-25T09:00:47.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.855+0000 7fe6b157a740 20 client.7740 trim_cache size 0 max 16384 2024-04-25T09:00:47.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.870+0000 7fe6b157a740 1 Processor -- start 2024-04-25T09:00:47.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.870+0000 7fe6b157a740 1 -- start start 2024-04-25T09:00:47.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.871+0000 7fe6b157a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d930d9230 0x562d930d89c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.871+0000 7fe6b157a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d930f6ad0 0x562d930ae140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.871+0000 7fe6b157a740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x562d931086f0 0x562d930af4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.871+0000 7fe6b157a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a814f2b0 con 0x562d930d9230 2024-04-25T09:00:47.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.871+0000 7fe6b157a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a814e6b0 con 0x562d930f6ad0 2024-04-25T09:00:47.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.871+0000 7fe6b157a740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a814ecb0 con 0x562d931086f0 2024-04-25T09:00:47.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.871+0000 7fe6a7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d930d9230 0x562d930d89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.871+0000 7fe6a7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d930d9230 0x562d930d89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43844/0 (socket says 172.21.15.43:43844) 2024-04-25T09:00:47.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.871+0000 7fe6a7fff640 1 -- 172.21.15.43:0/1042360050 learned_addr learned my addr 172.21.15.43:0/1042360050 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:47.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.872+0000 7fe6a7fff640 1 -- 172.21.15.43:0/1042360050 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x562d931086f0 msgr2=0x562d930af4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:47.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.872+0000 7fe6a7fff640 1 --2- 172.21.15.43:0/1042360050 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x562d931086f0 0x562d930af4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.872+0000 7fe6a7fff640 1 -- 172.21.15.43:0/1042360050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d930f6ad0 msgr2=0x562d930ae140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:47.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.872+0000 7fe6a7fff640 1 --2- 172.21.15.43:0/1042360050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d930f6ad0 0x562d930ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.872+0000 7fe6a7fff640 1 -- 172.21.15.43:0/1042360050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562d930dbd00 con 0x562d930d9230 2024-04-25T09:00:47.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.872+0000 7fe6a7fff640 1 --2- 172.21.15.43:0/1042360050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d930d9230 0x562d930d89c0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fe69c056ab0 tx=0x7fe69c05a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=107d3b82d9a78f6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:47.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.872+0000 7fe68d7fa640 1 -- 172.21.15.43:0/1042360050 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c070040 con 0x562d930d9230 2024-04-25T09:00:47.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.872+0000 7fe68d7fa640 1 -- 172.21.15.43:0/1042360050 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe69c0884f0 con 0x562d930d9230 2024-04-25T09:00:47.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.872+0000 7fe68d7fa640 1 -- 172.21.15.43:0/1042360050 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c057c50 con 0x562d930d9230 2024-04-25T09:00:47.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.872+0000 7fe6b157a740 1 -- 172.21.15.43:0/1042360050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d930d9230 msgr2=0x562d930d89c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:47.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.872+0000 7fe6b157a740 1 --2- 172.21.15.43:0/1042360050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d930d9230 0x562d930d89c0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7fe69c056ab0 tx=0x7fe69c05a000 comp rx=0 tx=0).stop 2024-04-25T09:00:47.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.872+0000 7fe6b157a740 1 -- 172.21.15.43:0/1042360050 shutdown_connections 2024-04-25T09:00:47.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.872+0000 7fe6b157a740 1 --2- 172.21.15.43:0/1042360050 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x562d931086f0 0x562d930af4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.872+0000 7fe6b157a740 1 --2- 172.21.15.43:0/1042360050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d930f6ad0 0x562d930ae140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.872+0000 7fe6b157a740 1 --2- 172.21.15.43:0/1042360050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d930d9230 0x562d930d89c0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.872+0000 7fe6b157a740 1 -- 172.21.15.43:0/1042360050 >> 172.21.15.43:0/1042360050 conn(0x562d930aef10 msgr2=0x562d930dcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:47.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.872+0000 7fe6b157a740 1 -- 172.21.15.43:0/1042360050 shutdown_connections 2024-04-25T09:00:47.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.872+0000 7fe6b157a740 1 -- 172.21.15.43:0/1042360050 wait complete. 2024-04-25T09:00:47.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.873+0000 7fe6b157a740 1 -- start start 2024-04-25T09:00:47.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.873+0000 7fe6b157a740 1 Processor -- start 2024-04-25T09:00:47.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.874+0000 7fe6b157a740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d930d9230 0x562d930f4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.874+0000 7fe6b157a740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d930f6ad0 0x562d930f3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.874+0000 7fe6b157a740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x562d931086f0 0x562d930f42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.874+0000 7fe6b157a740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d930eb680 con 0x562d930d9230 2024-04-25T09:00:47.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.874+0000 7fe6b157a740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562d93107f50 con 0x562d930f6ad0 2024-04-25T09:00:47.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.874+0000 7fe6b157a740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x562d930dc750 con 0x562d931086f0 2024-04-25T09:00:47.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.874+0000 7fe6b157a740 10 client.0 _ll_register_callbacks cb 0x562d930a9c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:00:47.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.874+0000 7fe6a7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d930d9230 0x562d930f4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.874+0000 7fe6a7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d930d9230 0x562d930f4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43856/0 (socket says 172.21.15.43:43856) 2024-04-25T09:00:47.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.874+0000 7fe6a7fff640 1 -- 172.21.15.43:0/4106569214 learned_addr learned my addr 172.21.15.43:0/4106569214 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:47.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.874+0000 7fe6a77fe640 1 --2- 172.21.15.43:0/4106569214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d930f6ad0 0x562d930f3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.874+0000 7fe6ace0c640 1 --2- 172.21.15.43:0/4106569214 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x562d931086f0 0x562d930f42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.874+0000 7fe6a7fff640 1 -- 172.21.15.43:0/4106569214 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x562d931086f0 msgr2=0x562d930f42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:47.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.874+0000 7fe6a7fff640 1 --2- 172.21.15.43:0/4106569214 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x562d931086f0 0x562d930f42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.874+0000 7fe6a7fff640 1 -- 172.21.15.43:0/4106569214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d930f6ad0 msgr2=0x562d930f3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:47.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.874+0000 7fe6a7fff640 1 --2- 172.21.15.43:0/4106569214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d930f6ad0 0x562d930f3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.874+0000 7fe6a7fff640 1 -- 172.21.15.43:0/4106569214 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562d930f5610 con 0x562d930d9230 2024-04-25T09:00:47.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.874+0000 7fe6a77fe640 1 --2- 172.21.15.43:0/4106569214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d930f6ad0 0x562d930f3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:00:47.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.874+0000 7fe6a7fff640 1 --2- 172.21.15.43:0/4106569214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d930d9230 0x562d930f4f00 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fe69c056d50 tx=0x7fe69c088480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1135c0cdfb3378da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:47.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.874+0000 7fe686ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:00:47.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.875+0000 7fe686ffd640 1 -- 172.21.15.43:0/4106569214 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69c070040 con 0x562d930d9230 2024-04-25T09:00:47.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.875+0000 7fe686ffd640 1 -- 172.21.15.43:0/4106569214 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe69c057c50 con 0x562d930d9230 2024-04-25T09:00:47.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.875+0000 7fe6b157a740 1 -- 172.21.15.43:0/4106569214 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe69c0884f0 con 0x562d930d9230 2024-04-25T09:00:47.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.875+0000 7fe686ffd640 1 -- 172.21.15.43:0/4106569214 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe69c0585d0 con 0x562d930d9230 2024-04-25T09:00:47.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.875+0000 7fe6b157a740 10 client.7746 fetch_fsmap learned FSMap version 54 2024-04-25T09:00:47.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.875+0000 7fe6b157a740 1 -- 172.21.15.43:0/4106569214 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x562d930d6630 con 0x562d930d9230 2024-04-25T09:00:47.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.875+0000 7fe686ffd640 1 -- 172.21.15.43:0/4106569214 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe69c0884f0 con 0x562d930d9230 2024-04-25T09:00:47.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.875+0000 7fe686ffd640 1 -- 172.21.15.43:0/4106569214 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 54) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe69c054550 con 0x562d930d9230 2024-04-25T09:00:47.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.875+0000 7fe6b157a740 10 client.7746 fetch_fsmap finished waiting for FSMap version 54 2024-04-25T09:00:47.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.875+0000 7fe6b157a740 10 client.7746 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:00:47.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.875+0000 7fe6b157a740 20 client.7746 populate_metadata read hostname 'smithi043' 2024-04-25T09:00:47.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.875+0000 7fe6b157a740 1 --2- 172.21.15.43:0/4106569214 >> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] conn(0x562d93106c20 0x562d93107000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.875+0000 7fe6b157a740 4 client.7746 mds_command: new command op to 7596 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:00:47.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.875+0000 7fe6b157a740 1 -- 172.21.15.43:0/4106569214 --> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562d930ed4a0 con 0x562d93106c20 2024-04-25T09:00:47.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.875+0000 7fe6a77fe640 1 --2- 172.21.15.43:0/4106569214 >> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] conn(0x562d93106c20 0x562d93107000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.875+0000 7fe6a77fe640 1 --2- 172.21.15.43:0/4106569214 >> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] conn(0x562d93106c20 0x562d93107000 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=824b330024d80f31 server_cookie=4dde776d596ce4a0 in_seq=0 out_seq=0 2024-04-25T09:00:47.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.875+0000 7fe686ffd640 10 client.7746 ms_handle_connect on v2:172.21.15.43:6833/1341028696 2024-04-25T09:00:47.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.876+0000 7fe686ffd640 1 -- 172.21.15.43:0/4106569214 <== mds.0 v2:172.21.15.43:6833/1341028696 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x562d930ed4a0 con 0x562d93106c20 2024-04-25T09:00:47.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.876+0000 7fe686ffd640 10 client.7746 handle_command_reply: tid=0 2024-04-25T09:00:47.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.876+0000 7fe6b157a740 1 client.7746 shutdown 2024-04-25T09:00:47.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.876+0000 7fe6b157a740 1 -- 172.21.15.43:0/4106569214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d930d9230 msgr2=0x562d930f4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:47.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.876+0000 7fe6b157a740 1 --2- 172.21.15.43:0/4106569214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d930d9230 0x562d930f4f00 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fe69c056d50 tx=0x7fe69c088480 comp rx=0 tx=0).stop 2024-04-25T09:00:47.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.876+0000 7fe6b157a740 1 -- 172.21.15.43:0/4106569214 shutdown_connections 2024-04-25T09:00:47.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.876+0000 7fe6b157a740 1 --2- 172.21.15.43:0/4106569214 >> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] conn(0x562d93106c20 0x562d93107000 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.876+0000 7fe6b157a740 1 --2- 172.21.15.43:0/4106569214 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x562d931086f0 0x562d930f42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.876+0000 7fe6b157a740 1 --2- 172.21.15.43:0/4106569214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x562d930f6ad0 0x562d930f3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.876+0000 7fe686ffd640 0 client.7746 ms_handle_reset on v2:172.21.15.43:6833/1341028696 2024-04-25T09:00:47.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.876+0000 7fe6b157a740 1 --2- 172.21.15.43:0/4106569214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562d930d9230 0x562d930f4f00 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.876+0000 7fe6b157a740 1 -- 172.21.15.43:0/4106569214 >> 172.21.15.43:0/4106569214 conn(0x562d930aef10 msgr2=0x562d93103980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:47.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.876+0000 7fe6b157a740 1 -- 172.21.15.43:0/4106569214 shutdown_connections 2024-04-25T09:00:47.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.876+0000 7fe6b157a740 1 -- 172.21.15.43:0/4106569214 wait complete. 2024-04-25T09:00:47.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.876+0000 7fe6b157a740 20 client.7746 trim_cache size 0 max 16384 2024-04-25T09:00:47.871 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:00:47.871 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "6c4bd50e-aa00-4a7f-8d09-352344ef5a4b", 2024-04-25T09:00:47.872 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-04-25T09:00:47.872 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 7596, 2024-04-25T09:00:47.872 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-04-25T09:00:47.872 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-04-25T09:00:47.872 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-04-25T09:00:47.872 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 4.7138731189999996, 2024-04-25T09:00:47.872 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 54, 2024-04-25T09:00:47.872 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 89, 2024-04-25T09:00:47.872 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 89, 2024-04-25T09:00:47.872 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 10.051564486 2024-04-25T09:00:47.872 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:00:47.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.876+0000 7fe6ae86f640 1 -- 172.21.15.43:0/2571518823 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe69406b4d0 msgr2=0x7fe69406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:47.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.876+0000 7fe6ae86f640 1 --2- 172.21.15.43:0/2571518823 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe69406b4d0 0x7fe69406d990 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fe698001eb0 tx=0x7fe698009040 comp rx=0 tx=0).stop 2024-04-25T09:00:47.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.876+0000 7fe6ae86f640 1 -- 172.21.15.43:0/2571518823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6a80ce2a0 msgr2=0x7fe6a80ca030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:47.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.876+0000 7fe6ae86f640 1 --2- 172.21.15.43:0/2571518823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6a80ce2a0 0x7fe6a80ca030 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fe69c00e990 tx=0x7fe69c00ee50 comp rx=0 tx=0).stop 2024-04-25T09:00:47.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.877+0000 7fe6ae86f640 1 -- 172.21.15.43:0/2571518823 shutdown_connections 2024-04-25T09:00:47.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.877+0000 7fe6ae86f640 1 --2- 172.21.15.43:0/2571518823 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe69406b4d0 0x7fe69406d990 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.877+0000 7fe6ae86f640 1 --2- 172.21.15.43:0/2571518823 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe6a80d39d0 0x7fe6a80ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.877+0000 7fe6ae86f640 1 --2- 172.21.15.43:0/2571518823 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe6a80d2500 0x7fe6a80c7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.877+0000 7fe6ae86f640 1 --2- 172.21.15.43:0/2571518823 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe6a80ce2a0 0x7fe6a80ca030 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.877+0000 7fe6ae86f640 1 -- 172.21.15.43:0/2571518823 >> 172.21.15.43:0/2571518823 conn(0x7fe6a806b000 msgr2=0x7fe6a815d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:47.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.877+0000 7fe6ae86f640 1 -- 172.21.15.43:0/2571518823 shutdown_connections 2024-04-25T09:00:47.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.877+0000 7fe6ae86f640 1 -- 172.21.15.43:0/2571518823 wait complete. 2024-04-25T09:00:47.877 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:00:47.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.988+0000 7f38e35e0640 1 Processor -- start 2024-04-25T09:00:47.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.989+0000 7f38e35e0640 1 -- start start 2024-04-25T09:00:47.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.989+0000 7f38e35e0640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f38dc15df50 0x7f38dc1619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.989+0000 7f38e35e0640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f38dc0cf670 0x7f38dc162380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.989+0000 7f38e35e0640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38dc0d24a0 con 0x7f38dc161f30 2024-04-25T09:00:47.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.989+0000 7f38e35e0640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38dc0cfa80 con 0x7f38dc0cf670 2024-04-25T09:00:47.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.989+0000 7f38e35e0640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38dc15e330 con 0x7f38dc15df50 2024-04-25T09:00:47.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.989+0000 7f38e1b7d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f38dc0cf670 0x7f38dc162380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.990+0000 7f38e1b7d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f38dc0cf670 0x7f38dc162380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43862/0 (socket says 172.21.15.43:43862) 2024-04-25T09:00:47.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.990+0000 7f38e1b7d640 1 -- 172.21.15.43:0/2390208107 learned_addr learned my addr 172.21.15.43:0/2390208107 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:47.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.990+0000 7f38e137c640 1 --2- 172.21.15.43:0/2390208107 >> v2:172.21.15.193:3300/0 conn(0x7f38dc15df50 0x7f38dc1619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.990+0000 7f38e1b7d640 1 -- 172.21.15.43:0/2390208107 >> v2:172.21.15.193:3300/0 conn(0x7f38dc15df50 msgr2=0x7f38dc1619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:47.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.990+0000 7f38e1b7d640 1 --2- 172.21.15.43:0/2390208107 >> v2:172.21.15.193:3300/0 conn(0x7f38dc15df50 0x7f38dc1619f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.990+0000 7f38e1b7d640 1 -- 172.21.15.43:0/2390208107 >> v1:172.21.15.193:6789/0 conn(0x7f38dc161f30 legacy=0x7f38dc0c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:00:47.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.990+0000 7f38e1b7d640 1 -- 172.21.15.43:0/2390208107 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38dc0d2740 con 0x7f38dc0cf670 2024-04-25T09:00:47.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.990+0000 7f38e137c640 1 --2- 172.21.15.43:0/2390208107 >> v2:172.21.15.193:3300/0 conn(0x7f38dc15df50 0x7f38dc1619f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:00:47.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.991+0000 7f38e1b7d640 1 --2- 172.21.15.43:0/2390208107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38dc0cf670 0x7f38dc162380 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f38dc0caa00 tx=0x7f38d802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef18b52941dc203d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:47.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.991+0000 7f38d3fff640 1 -- 172.21.15.43:0/2390208107 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d8036070 con 0x7f38dc0cf670 2024-04-25T09:00:47.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.991+0000 7f38d3fff640 1 -- 172.21.15.43:0/2390208107 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d8031600 con 0x7f38dc0cf670 2024-04-25T09:00:47.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.991+0000 7f38d3fff640 1 -- 172.21.15.43:0/2390208107 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d80319d0 con 0x7f38dc0cf670 2024-04-25T09:00:47.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.991+0000 7f38e35e0640 1 -- 172.21.15.43:0/2390208107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38dc0cf670 msgr2=0x7f38dc162380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:47.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.991+0000 7f38e35e0640 1 --2- 172.21.15.43:0/2390208107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38dc0cf670 0x7f38dc162380 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f38dc0caa00 tx=0x7f38d802a6c0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.991+0000 7f38e35e0640 1 -- 172.21.15.43:0/2390208107 shutdown_connections 2024-04-25T09:00:47.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.991+0000 7f38e35e0640 1 --2- 172.21.15.43:0/2390208107 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38dc0cf670 0x7f38dc162380 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.991+0000 7f38e35e0640 1 --2- 172.21.15.43:0/2390208107 >> v2:172.21.15.193:3300/0 conn(0x7f38dc15df50 0x7f38dc1619f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.991+0000 7f38e35e0640 1 -- 172.21.15.43:0/2390208107 >> 172.21.15.43:0/2390208107 conn(0x7f38dc0d3a30 msgr2=0x7f38dc0d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:47.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.991+0000 7f38e35e0640 1 -- 172.21.15.43:0/2390208107 shutdown_connections 2024-04-25T09:00:47.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.991+0000 7f38e35e0640 1 -- 172.21.15.43:0/2390208107 wait complete. 2024-04-25T09:00:47.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.993+0000 7f38e35e0640 1 Processor -- start 2024-04-25T09:00:47.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.993+0000 7f38e35e0640 1 -- start start 2024-04-25T09:00:47.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.993+0000 7f38e35e0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38dc0cf670 0x7f38dc14ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.993+0000 7f38e35e0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38dc15df50 0x7f38dc15d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.993+0000 7f38e35e0640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f38dc161f30 0x7f38dc0ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.993+0000 7f38e35e0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38dc0c77d0 con 0x7f38dc0cf670 2024-04-25T09:00:47.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.993+0000 7f38e35e0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38dc0c74f0 con 0x7f38dc15df50 2024-04-25T09:00:47.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.993+0000 7f38e35e0640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38dc0d2190 con 0x7f38dc161f30 2024-04-25T09:00:47.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.993+0000 7f38e137c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38dc0cf670 0x7f38dc14ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.993+0000 7f38e137c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38dc0cf670 0x7f38dc14ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43876/0 (socket says 172.21.15.43:43876) 2024-04-25T09:00:47.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.993+0000 7f38e137c640 1 -- 172.21.15.43:0/2725765608 learned_addr learned my addr 172.21.15.43:0/2725765608 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:00:47.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.994+0000 7f38e0b7b640 1 --2- 172.21.15.43:0/2725765608 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38dc15df50 0x7f38dc15d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.994+0000 7f38e1b7d640 1 --2- 172.21.15.43:0/2725765608 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f38dc161f30 0x7f38dc0ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.994+0000 7f38e137c640 1 -- 172.21.15.43:0/2725765608 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f38dc161f30 msgr2=0x7f38dc0ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:47.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.994+0000 7f38e137c640 1 --2- 172.21.15.43:0/2725765608 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f38dc161f30 0x7f38dc0ccc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.994+0000 7f38e137c640 1 -- 172.21.15.43:0/2725765608 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38dc15df50 msgr2=0x7f38dc15d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:47.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.994+0000 7f38e137c640 1 --2- 172.21.15.43:0/2725765608 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38dc15df50 0x7f38dc15d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:47.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.994+0000 7f38e137c640 1 -- 172.21.15.43:0/2725765608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38dc15ff30 con 0x7f38dc0cf670 2024-04-25T09:00:47.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.994+0000 7f38e0b7b640 1 --2- 172.21.15.43:0/2725765608 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38dc15df50 0x7f38dc15d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:00:47.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.994+0000 7f38e137c640 1 --2- 172.21.15.43:0/2725765608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38dc0cf670 0x7f38dc14ca80 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f38d400c820 tx=0x7f38d400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:47.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.994+0000 7f38d1ffb640 1 -- 172.21.15.43:0/2725765608 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d4015070 con 0x7f38dc0cf670 2024-04-25T09:00:47.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.994+0000 7f38d1ffb640 1 -- 172.21.15.43:0/2725765608 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d4007a30 con 0x7f38dc0cf670 2024-04-25T09:00:47.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.994+0000 7f38e35e0640 1 -- 172.21.15.43:0/2725765608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38dc160210 con 0x7f38dc0cf670 2024-04-25T09:00:47.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.994+0000 7f38d1ffb640 1 -- 172.21.15.43:0/2725765608 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d4007e00 con 0x7f38dc0cf670 2024-04-25T09:00:47.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.994+0000 7f38e35e0640 1 -- 172.21.15.43:0/2725765608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38dc1607d0 con 0x7f38dc0cf670 2024-04-25T09:00:47.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.995+0000 7f38e35e0640 1 -- 172.21.15.43:0/2725765608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38dc0680f0 con 0x7f38dc0cf670 2024-04-25T09:00:47.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.996+0000 7f38d1ffb640 1 -- 172.21.15.43:0/2725765608 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f38d4018070 con 0x7f38dc0cf670 2024-04-25T09:00:47.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.996+0000 7f38d1ffb640 1 --2- 172.21.15.43:0/2725765608 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f38c006b4d0 0x7f38c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:00:47.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.996+0000 7f38d1ffb640 1 -- 172.21.15.43:0/2725765608 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f38d408a1b0 con 0x7f38dc0cf670 2024-04-25T09:00:47.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.996+0000 7f38e0b7b640 1 --2- 172.21.15.43:0/2725765608 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f38c006b4d0 0x7f38c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:00:47.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:47.997+0000 7f38e0b7b640 1 --2- 172.21.15.43:0/2725765608 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f38c006b4d0 0x7f38c006d990 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f38cc006fd0 tx=0x7f38cc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:00:47.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:48.000+0000 7f38d1ffb640 1 -- 172.21.15.43:0/2725765608 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f38d4059030 con 0x7f38dc0cf670 2024-04-25T09:00:48.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:48.218+0000 7f38e35e0640 1 -- 172.21.15.43:0/2725765608 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f38dc0ca400 con 0x7f38dc0cf670 2024-04-25T09:00:48.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:48.219+0000 7f38d1ffb640 1 -- 172.21.15.43:0/2725765608 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v54) v1 ==== 53+0+73 (secure 0 0 0) 0x7f38d4058930 con 0x7f38dc0cf670 2024-04-25T09:00:48.208 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:00:48.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:48.221+0000 7f38e35e0640 1 -- 172.21.15.43:0/2725765608 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f38c006b4d0 msgr2=0x7f38c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:48.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:48.221+0000 7f38e35e0640 1 --2- 172.21.15.43:0/2725765608 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f38c006b4d0 0x7f38c006d990 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f38cc006fd0 tx=0x7f38cc008040 comp rx=0 tx=0).stop 2024-04-25T09:00:48.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:48.221+0000 7f38e35e0640 1 -- 172.21.15.43:0/2725765608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38dc0cf670 msgr2=0x7f38dc14ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:00:48.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:48.221+0000 7f38e35e0640 1 --2- 172.21.15.43:0/2725765608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38dc0cf670 0x7f38dc14ca80 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f38d400c820 tx=0x7f38d400cc20 comp rx=0 tx=0).stop 2024-04-25T09:00:48.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:48.221+0000 7f38e35e0640 1 -- 172.21.15.43:0/2725765608 shutdown_connections 2024-04-25T09:00:48.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:48.221+0000 7f38e35e0640 1 --2- 172.21.15.43:0/2725765608 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f38dc161f30 0x7f38dc0ccc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:48.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:48.221+0000 7f38e35e0640 1 --2- 172.21.15.43:0/2725765608 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f38c006b4d0 0x7f38c006d990 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:48.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:48.221+0000 7f38e35e0640 1 --2- 172.21.15.43:0/2725765608 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38dc15df50 0x7f38dc15d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:48.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:48.221+0000 7f38e35e0640 1 --2- 172.21.15.43:0/2725765608 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f38dc0cf670 0x7f38dc14ca80 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:00:48.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:48.221+0000 7f38e35e0640 1 -- 172.21.15.43:0/2725765608 >> 172.21.15.43:0/2725765608 conn(0x7f38dc0d3a30 msgr2=0x7f38dc157670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:00:48.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:48.221+0000 7f38e35e0640 1 -- 172.21.15.43:0/2725765608 shutdown_connections 2024-04-25T09:00:48.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:48.221+0000 7f38e35e0640 1 -- 172.21.15.43:0/2725765608 wait complete. 2024-04-25T09:00:48.221 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T09:00:48.221 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:00:48.221 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-04-25T09:00:48.221 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:00:48.221 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:00:48.221 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T09:00:48.221 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T09:00:48.221 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:00:48.221 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:00:48.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:48.222 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-04-25T09:00:48.237 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:00:48.237 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:00:48.237 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:00:48.237 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:00:48.238 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-04-25T09:00:48.238 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:00:48.238 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:00:48.238 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-04-25T09:00:48.238 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-04-25T09:00:48.238 INFO:teuthology.orchestra.run.smithi043.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:00:48.238 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:00:48.238 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-04-25T09:00:48.238 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T09:00:48.238 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 42370sec preferred_lft 42370sec 2024-04-25T09:00:48.238 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link noprefixroute 2024-04-25T09:00:48.238 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:00:48.238 INFO:teuthology.orchestra.run.smithi043.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:00:48.238 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-04-25T09:00:48.239 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-04-25T09:00:48.239 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:00:48.239 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:00:48.239 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-04-25T09:00:48.239 INFO:teuthology.orchestra.run.smithi043.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:00:48.239 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 12:2c:7a:72:36:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:00:48.239 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:00:48.239 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:00:48.239 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ccff:4eff:fe9d:c8bf/64 scope link 2024-04-25T09:00:48.239 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:00:48.239 INFO:teuthology.orchestra.run.smithi043.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:00:48.239 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 16:9a:63:37:46:24 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:00:48.239 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::149a:63ff:fe37:4624/64 scope link 2024-04-25T09:00:48.239 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:00:48.240 INFO:teuthology.orchestra.run.smithi043.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:00:48.240 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 12:2c:7a:72:36:14 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:00:48.240 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::102c:7aff:fe72:3614/64 scope link 2024-04-25T09:00:48.240 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:00:48.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:48.240 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:00:48.301 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-04-25T09:00:48.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:48.302 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-04-25T09:00:48.358 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-04-25T09:00:48.358 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:00:48.358 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T09:00:48.359 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T09:00:48.359 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:00:48.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:48.359 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2024-04-25T09:00:48.359 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:00:48.359 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T09:00:48.359 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T09:00:48.359 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T09:00:48.359 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:00:48.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:48.444 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-04-25T09:00:48.459 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:00:48.459 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:00:48.459 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:00:48.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:48.459 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-04-25T09:00:48.524 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T09:00:48.524 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:00:48.525 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:00:48.580 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:00:48.580 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:00:48.581 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:00:48.635 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-04-25T09:00:48.702 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:00:48.759 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-04-25T09:00:48.759 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-04-25T09:00:48.759 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-04-25T09:00:48.759 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-04-25T09:00:48.759 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-04-25T09:00:48.759 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-04-25T09:00:48.759 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-04-25T09:00:48.759 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-04-25T09:00:48.759 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-04-25T09:00:48.759 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-04-25T09:00:48.759 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-04-25T09:00:48.760 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/selinux 2024-04-25T09:00:48.760 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:00:48.760 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-04-25T09:00:48.760 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-04-25T09:00:48.760 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-04-25T09:00:48.760 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-04-25T09:00:48.760 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:00:48.760 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-04-25T09:00:48.760 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-04-25T09:00:48.760 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T09:00:48.760 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T09:00:48.760 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T09:00:48.760 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T09:00:48.760 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-04-25T09:00:48.761 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-04-25T09:00:48.761 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:00:48.761 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:00:48.761 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:00:48.761 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:00:48.761 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-04-25T09:00:48.761 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:00:48.761 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:00:48.761 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:00:48.761 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:00:48.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:48.761 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-04-25T09:00:48.813 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T09:00:48.813 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T09:00:48.817 INFO:tasks.ceph.mds.a.smithi043.stderr:2024-04-25T09:00:48.828+0000 7fe436e43640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:00:48.819 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-04-25T09:00:48.842 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:00:48.901 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:2024-04-25T09:00:48.912+0000 7efda0f1f480 -1 init, newargv = 0x55671dd09780 newargc=15 2024-04-25T09:00:48.901 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[76245]: starting ceph client 2024-04-25T09:00:48.903 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-04-25T09:00:48.903 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-04-25T09:00:48.903 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-04-25T09:00:48.903 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-04-25T09:00:48.903 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-04-25T09:00:48.904 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-04-25T09:00:48.904 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-04-25T09:00:48.904 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-04-25T09:00:48.904 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-04-25T09:00:48.905 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-04-25T09:00:48.905 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-04-25T09:00:48.905 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/selinux 2024-04-25T09:00:48.905 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:00:48.906 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-04-25T09:00:48.906 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-04-25T09:00:48.906 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-04-25T09:00:48.906 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-04-25T09:00:48.906 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:00:48.906 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-04-25T09:00:48.907 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-04-25T09:00:48.907 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T09:00:48.907 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T09:00:48.907 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T09:00:48.907 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T09:00:48.907 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-04-25T09:00:48.908 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-04-25T09:00:48.908 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:00:48.908 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:00:48.908 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:00:48.908 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:00:48.908 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-04-25T09:00:48.909 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:00:48.909 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:00:48.909 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:00:48.909 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:00:48.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:48.910 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-04-25T09:00:48.911 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[76245]: starting fuse 2024-04-25T09:00:48.925 INFO:teuthology.orchestra.run.smithi043.stdout:43 2024-04-25T09:00:48.926 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-25T09:00:48.926 DEBUG:teuthology.orchestra.run.smithi043:> sudo stdin-killer -- python3 -c ' 2024-04-25T09:00:48.926 DEBUG:teuthology.orchestra.run.smithi043:> import glob 2024-04-25T09:00:48.926 DEBUG:teuthology.orchestra.run.smithi043:> import re 2024-04-25T09:00:48.926 DEBUG:teuthology.orchestra.run.smithi043:> import os 2024-04-25T09:00:48.927 DEBUG:teuthology.orchestra.run.smithi043:> import subprocess 2024-04-25T09:00:48.927 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T09:00:48.927 DEBUG:teuthology.orchestra.run.smithi043:> def _find_admin_socket(client_name): 2024-04-25T09:00:48.927 DEBUG:teuthology.orchestra.run.smithi043:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T09:00:48.927 DEBUG:teuthology.orchestra.run.smithi043:> files = glob.glob(asok_path) 2024-04-25T09:00:48.927 DEBUG:teuthology.orchestra.run.smithi043:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T09:00:48.928 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T09:00:48.928 DEBUG:teuthology.orchestra.run.smithi043:> # Given a non-glob path, it better be there 2024-04-25T09:00:48.928 DEBUG:teuthology.orchestra.run.smithi043:> if "*" not in asok_path: 2024-04-25T09:00:48.928 DEBUG:teuthology.orchestra.run.smithi043:> assert(len(files) == 1) 2024-04-25T09:00:48.928 DEBUG:teuthology.orchestra.run.smithi043:> return files[0] 2024-04-25T09:00:48.928 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T09:00:48.928 DEBUG:teuthology.orchestra.run.smithi043:> for f in files: 2024-04-25T09:00:48.929 DEBUG:teuthology.orchestra.run.smithi043:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T09:00:48.929 DEBUG:teuthology.orchestra.run.smithi043:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T09:00:48.929 DEBUG:teuthology.orchestra.run.smithi043:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T09:00:48.929 DEBUG:teuthology.orchestra.run.smithi043:> contents = proc_f.read() 2024-04-25T09:00:48.929 DEBUG:teuthology.orchestra.run.smithi043:> if mountpoint in contents: 2024-04-25T09:00:48.929 DEBUG:teuthology.orchestra.run.smithi043:> return f 2024-04-25T09:00:48.930 DEBUG:teuthology.orchestra.run.smithi043:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T09:00:48.930 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T09:00:48.930 DEBUG:teuthology.orchestra.run.smithi043:> print(_find_admin_socket("client.0")) 2024-04-25T09:00:48.930 DEBUG:teuthology.orchestra.run.smithi043:> ' 2024-04-25T09:00:49.045 INFO:teuthology.orchestra.run.smithi043.stdout:/var/run/ceph/ceph-client.0.76245.asok 2024-04-25T09:00:49.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:00:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:00:49.056 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.76245.asok 2024-04-25T09:00:49.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:49.056 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.76245.asok status 2024-04-25T09:00:49.191 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:00:49.191 INFO:teuthology.orchestra.run.smithi043.stdout: "metadata": { 2024-04-25T09:00:49.191 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T09:00:49.192 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T09:00:49.192 INFO:teuthology.orchestra.run.smithi043.stdout: "entity_id": "0", 2024-04-25T09:00:49.192 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": "smithi043", 2024-04-25T09:00:49.192 INFO:teuthology.orchestra.run.smithi043.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T09:00:49.192 INFO:teuthology.orchestra.run.smithi043.stdout: "pid": "76245", 2024-04-25T09:00:49.192 INFO:teuthology.orchestra.run.smithi043.stdout: "root": "/" 2024-04-25T09:00:49.192 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:49.192 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_count": 0, 2024-04-25T09:00:49.192 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_pinned_count": 0, 2024-04-25T09:00:49.192 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 7755, 2024-04-25T09:00:49.192 INFO:teuthology.orchestra.run.smithi043.stdout: "inst": { 2024-04-25T09:00:49.192 INFO:teuthology.orchestra.run.smithi043.stdout: "name": { 2024-04-25T09:00:49.193 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "client", 2024-04-25T09:00:49.193 INFO:teuthology.orchestra.run.smithi043.stdout: "num": 7755 2024-04-25T09:00:49.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:49.193 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-04-25T09:00:49.193 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-04-25T09:00:49.193 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:00:49.193 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 430348997 2024-04-25T09:00:49.193 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:00:49.193 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:49.193 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-04-25T09:00:49.193 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-04-25T09:00:49.193 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:00:49.194 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 430348997 2024-04-25T09:00:49.194 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:00:49.194 INFO:teuthology.orchestra.run.smithi043.stdout: "inst_str": "client.7755 192.168.0.1:0/430348997", 2024-04-25T09:00:49.194 INFO:teuthology.orchestra.run.smithi043.stdout: "addr_str": "192.168.0.1:0/430348997", 2024-04-25T09:00:49.194 INFO:teuthology.orchestra.run.smithi043.stdout: "inode_count": 1, 2024-04-25T09:00:49.194 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_epoch": 54, 2024-04-25T09:00:49.194 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch": 89, 2024-04-25T09:00:49.194 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch_barrier": 0, 2024-04-25T09:00:49.194 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklisted": false, 2024-04-25T09:00:49.194 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs" 2024-04-25T09:00:49.194 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:00:49.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:49.200 DEBUG:teuthology.orchestra.run.smithi043:> stat --file-system '--printf=%T 2024-04-25T09:00:49.200 DEBUG:teuthology.orchestra.run.smithi043:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T09:00:49.216 INFO:teuthology.orchestra.run.smithi043.stdout:fuseblk 2024-04-25T09:00:49.216 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T09:00:49.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:49.216 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:00:49.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:49.281 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:00:49.281 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:00:49.281 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T09:00:49.281 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:00:49.281 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:00:49.282 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:00:49.282 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:00:49.361 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.361 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.361 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106444 s, 98.5 MB/s 2024-04-25T09:00:49.374 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.374 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.374 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0105499 s, 99.4 MB/s 2024-04-25T09:00:49.386 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.386 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.386 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00941487 s, 111 MB/s 2024-04-25T09:00:49.398 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.398 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.398 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00920631 s, 114 MB/s 2024-04-25T09:00:49.409 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.409 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.409 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00874313 s, 120 MB/s 2024-04-25T09:00:49.422 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.422 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.423 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108362 s, 96.8 MB/s 2024-04-25T09:00:49.434 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.434 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.435 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00941906 s, 111 MB/s 2024-04-25T09:00:49.445 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.445 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.445 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767136 s, 137 MB/s 2024-04-25T09:00:49.457 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.457 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.458 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103104 s, 102 MB/s 2024-04-25T09:00:49.469 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.469 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.469 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00923857 s, 113 MB/s 2024-04-25T09:00:49.479 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.479 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.479 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793276 s, 132 MB/s 2024-04-25T09:00:49.489 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.489 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.489 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00795899 s, 132 MB/s 2024-04-25T09:00:49.498 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.499 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.499 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727918 s, 144 MB/s 2024-04-25T09:00:49.508 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.509 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.509 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736078 s, 142 MB/s 2024-04-25T09:00:49.520 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.520 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.520 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00959581 s, 109 MB/s 2024-04-25T09:00:49.531 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.531 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.532 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00917797 s, 114 MB/s 2024-04-25T09:00:49.543 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.543 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.543 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00938099 s, 112 MB/s 2024-04-25T09:00:49.554 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.554 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.554 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861439 s, 122 MB/s 2024-04-25T09:00:49.564 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.564 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.564 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826334 s, 127 MB/s 2024-04-25T09:00:49.573 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.574 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.574 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726946 s, 144 MB/s 2024-04-25T09:00:49.583 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.583 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.583 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725117 s, 145 MB/s 2024-04-25T09:00:49.594 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.595 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.595 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00940929 s, 111 MB/s 2024-04-25T09:00:49.605 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.605 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.605 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830544 s, 126 MB/s 2024-04-25T09:00:49.616 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.616 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.616 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00911349 s, 115 MB/s 2024-04-25T09:00:49.628 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.628 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.628 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00932691 s, 112 MB/s 2024-04-25T09:00:49.640 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.641 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.641 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107285 s, 97.7 MB/s 2024-04-25T09:00:49.651 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.651 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.651 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00801695 s, 131 MB/s 2024-04-25T09:00:49.661 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.661 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.661 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825448 s, 127 MB/s 2024-04-25T09:00:49.673 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.673 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.673 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00975654 s, 107 MB/s 2024-04-25T09:00:49.683 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.683 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.683 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730448 s, 144 MB/s 2024-04-25T09:00:49.694 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.694 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.694 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00907551 s, 116 MB/s 2024-04-25T09:00:49.706 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.706 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.706 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101689 s, 103 MB/s 2024-04-25T09:00:49.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:49.708 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:00:49.708 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:00:49.708 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T09:00:49.708 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:00:49.708 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:00:49.708 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:00:49.708 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:00:49.777 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.777 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.777 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00951636 s, 110 MB/s 2024-04-25T09:00:49.787 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.787 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.787 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756085 s, 139 MB/s 2024-04-25T09:00:49.798 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.798 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.798 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00885542 s, 118 MB/s 2024-04-25T09:00:49.809 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.809 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.809 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082361 s, 127 MB/s 2024-04-25T09:00:49.819 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.819 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.819 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0080253 s, 131 MB/s 2024-04-25T09:00:49.829 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.829 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.830 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827825 s, 127 MB/s 2024-04-25T09:00:49.840 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.841 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.841 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00880079 s, 119 MB/s 2024-04-25T09:00:49.851 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.851 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.851 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00786358 s, 133 MB/s 2024-04-25T09:00:49.862 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.862 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.862 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00879334 s, 119 MB/s 2024-04-25T09:00:49.873 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.873 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.873 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00889936 s, 118 MB/s 2024-04-25T09:00:49.885 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.886 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.886 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00789429 s, 133 MB/s 2024-04-25T09:00:49.894 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.894 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.895 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838258 s, 125 MB/s 2024-04-25T09:00:49.903 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.904 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.904 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752284 s, 139 MB/s 2024-04-25T09:00:49.914 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.915 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.915 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00887895 s, 118 MB/s 2024-04-25T09:00:49.927 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.927 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.928 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00952826 s, 110 MB/s 2024-04-25T09:00:49.936 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.936 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.936 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725133 s, 145 MB/s 2024-04-25T09:00:49.946 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.947 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.947 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00882146 s, 119 MB/s 2024-04-25T09:00:49.957 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.957 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.958 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00885923 s, 118 MB/s 2024-04-25T09:00:49.968 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.969 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.969 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00813843 s, 129 MB/s 2024-04-25T09:00:49.977 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.977 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.977 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722159 s, 145 MB/s 2024-04-25T09:00:49.987 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.988 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.988 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00859252 s, 122 MB/s 2024-04-25T09:00:49.998 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:49.999 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:49.999 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00889632 s, 118 MB/s 2024-04-25T09:00:50.008 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.008 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.008 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731253 s, 143 MB/s 2024-04-25T09:00:50.019 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.019 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.019 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861848 s, 122 MB/s 2024-04-25T09:00:50.030 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.030 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.030 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00862039 s, 122 MB/s 2024-04-25T09:00:50.041 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.041 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.041 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00910684 s, 115 MB/s 2024-04-25T09:00:50.052 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.052 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.052 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00865292 s, 121 MB/s 2024-04-25T09:00:50.063 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.063 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.064 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00901466 s, 116 MB/s 2024-04-25T09:00:50.072 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.073 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.073 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072942 s, 144 MB/s 2024-04-25T09:00:50.083 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.083 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.083 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0080522 s, 130 MB/s 2024-04-25T09:00:50.092 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.092 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.092 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727372 s, 144 MB/s 2024-04-25T09:00:50.101 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.101 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.102 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722332 s, 145 MB/s 2024-04-25T09:00:50.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:50.103 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:00:50.103 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:00:50.103 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T09:00:50.103 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:00:50.103 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:00:50.103 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:00:50.103 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:00:50.173 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.173 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.173 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00897359 s, 117 MB/s 2024-04-25T09:00:50.185 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.185 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.185 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00948732 s, 111 MB/s 2024-04-25T09:00:50.195 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.195 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.195 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075379 s, 139 MB/s 2024-04-25T09:00:50.204 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.204 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.204 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073114 s, 143 MB/s 2024-04-25T09:00:50.216 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.216 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.216 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00927814 s, 113 MB/s 2024-04-25T09:00:50.226 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.226 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.226 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752268 s, 139 MB/s 2024-04-25T09:00:50.236 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.236 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.236 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752774 s, 139 MB/s 2024-04-25T09:00:50.245 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.245 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.245 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717003 s, 146 MB/s 2024-04-25T09:00:50.256 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.256 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.256 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00919651 s, 114 MB/s 2024-04-25T09:00:50.268 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.269 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.269 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00976998 s, 107 MB/s 2024-04-25T09:00:50.279 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.279 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.279 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00801083 s, 131 MB/s 2024-04-25T09:00:50.290 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.290 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.291 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00957487 s, 110 MB/s 2024-04-25T09:00:50.301 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.301 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.301 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818015 s, 128 MB/s 2024-04-25T09:00:50.310 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.311 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.311 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007476 s, 140 MB/s 2024-04-25T09:00:50.322 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.322 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.322 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.009436 s, 111 MB/s 2024-04-25T09:00:50.336 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.336 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.336 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.009168 s, 114 MB/s 2024-04-25T09:00:50.347 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.347 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.347 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082596 s, 127 MB/s 2024-04-25T09:00:50.356 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.356 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.356 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00914602 s, 115 MB/s 2024-04-25T09:00:50.367 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.368 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.368 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00894306 s, 117 MB/s 2024-04-25T09:00:50.379 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.380 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.380 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00903806 s, 116 MB/s 2024-04-25T09:00:50.389 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.389 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.389 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711732 s, 147 MB/s 2024-04-25T09:00:50.400 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.400 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.400 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00878757 s, 119 MB/s 2024-04-25T09:00:50.411 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.412 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.412 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00864089 s, 121 MB/s 2024-04-25T09:00:50.422 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.422 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.422 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00790798 s, 133 MB/s 2024-04-25T09:00:50.433 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.433 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.433 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834282 s, 126 MB/s 2024-04-25T09:00:50.444 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.444 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.444 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841345 s, 125 MB/s 2024-04-25T09:00:50.454 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.454 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.454 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730897 s, 143 MB/s 2024-04-25T09:00:50.466 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.466 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.466 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00975547 s, 107 MB/s 2024-04-25T09:00:50.478 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.478 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.478 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00892268 s, 118 MB/s 2024-04-25T09:00:50.489 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.489 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.490 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00891464 s, 118 MB/s 2024-04-25T09:00:50.501 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.501 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.501 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00886145 s, 118 MB/s 2024-04-25T09:00:50.511 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.511 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.511 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756751 s, 139 MB/s 2024-04-25T09:00:50.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:50.512 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:00:50.512 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:00:50.512 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T09:00:50.512 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:00:50.512 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:00:50.513 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:00:50.513 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:00:50.585 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.585 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.585 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0099641 s, 105 MB/s 2024-04-25T09:00:50.596 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.597 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.597 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00951638 s, 110 MB/s 2024-04-25T09:00:50.608 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.608 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.608 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00903097 s, 116 MB/s 2024-04-25T09:00:50.619 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.619 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.619 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00808401 s, 130 MB/s 2024-04-25T09:00:50.630 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.630 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.630 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824203 s, 127 MB/s 2024-04-25T09:00:50.641 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.641 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.642 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00985784 s, 106 MB/s 2024-04-25T09:00:50.652 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.652 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.652 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082073 s, 128 MB/s 2024-04-25T09:00:50.663 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.663 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.663 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0086553 s, 121 MB/s 2024-04-25T09:00:50.673 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.674 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.674 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804453 s, 130 MB/s 2024-04-25T09:00:50.684 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.685 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.685 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00878573 s, 119 MB/s 2024-04-25T09:00:50.694 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.694 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.694 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007773 s, 135 MB/s 2024-04-25T09:00:50.705 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.705 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.705 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827334 s, 127 MB/s 2024-04-25T09:00:50.714 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.714 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.714 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736342 s, 142 MB/s 2024-04-25T09:00:50.725 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.725 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.725 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00897335 s, 117 MB/s 2024-04-25T09:00:50.735 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.735 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.735 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00790086 s, 133 MB/s 2024-04-25T09:00:50.746 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.747 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.747 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00799438 s, 131 MB/s 2024-04-25T09:00:50.756 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.756 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.756 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00789105 s, 133 MB/s 2024-04-25T09:00:50.767 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.767 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.767 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00912908 s, 115 MB/s 2024-04-25T09:00:50.778 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.779 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.779 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00897852 s, 117 MB/s 2024-04-25T09:00:50.790 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.790 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.790 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00924197 s, 113 MB/s 2024-04-25T09:00:50.802 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.802 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.802 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00961256 s, 109 MB/s 2024-04-25T09:00:50.813 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.814 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.814 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00939859 s, 112 MB/s 2024-04-25T09:00:50.823 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.823 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.823 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074268 s, 141 MB/s 2024-04-25T09:00:50.834 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.834 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.834 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00875566 s, 120 MB/s 2024-04-25T09:00:50.844 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.844 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.844 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791554 s, 132 MB/s 2024-04-25T09:00:50.854 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.854 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.854 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723955 s, 145 MB/s 2024-04-25T09:00:50.865 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.865 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.865 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00902091 s, 116 MB/s 2024-04-25T09:00:50.875 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.875 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.875 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00784843 s, 134 MB/s 2024-04-25T09:00:50.885 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.885 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.885 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748582 s, 140 MB/s 2024-04-25T09:00:50.896 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.896 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.896 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829133 s, 126 MB/s 2024-04-25T09:00:50.907 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.908 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.908 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00903347 s, 116 MB/s 2024-04-25T09:00:50.918 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.918 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.918 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00796175 s, 132 MB/s 2024-04-25T09:00:50.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:50.919 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:00:50.919 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:00:50.919 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T09:00:50.920 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:00:50.920 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:00:50.920 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:00:50.920 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:00:50.989 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:50.990 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:50.990 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0085625 s, 122 MB/s 2024-04-25T09:00:51.001 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.001 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.001 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00939689 s, 112 MB/s 2024-04-25T09:00:51.010 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.011 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.011 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731579 s, 143 MB/s 2024-04-25T09:00:51.022 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.022 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.022 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00970217 s, 108 MB/s 2024-04-25T09:00:51.033 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.033 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.033 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00853666 s, 123 MB/s 2024-04-25T09:00:51.044 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.045 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.045 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0091288 s, 115 MB/s 2024-04-25T09:00:51.055 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.055 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.055 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00843565 s, 124 MB/s 2024-04-25T09:00:51.065 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.066 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.066 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00809013 s, 130 MB/s 2024-04-25T09:00:51.078 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.078 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.078 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101612 s, 103 MB/s 2024-04-25T09:00:51.090 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.090 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.090 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00949964 s, 110 MB/s 2024-04-25T09:00:51.101 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.102 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.102 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00937152 s, 112 MB/s 2024-04-25T09:00:51.111 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.111 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.111 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709052 s, 148 MB/s 2024-04-25T09:00:51.121 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.121 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.121 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814288 s, 129 MB/s 2024-04-25T09:00:51.134 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.134 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.135 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0080217 s, 131 MB/s 2024-04-25T09:00:51.142 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.142 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.142 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822052 s, 128 MB/s 2024-04-25T09:00:51.153 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.153 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.153 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00915973 s, 114 MB/s 2024-04-25T09:00:51.162 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.163 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.163 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709656 s, 148 MB/s 2024-04-25T09:00:51.172 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.173 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.173 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00790801 s, 133 MB/s 2024-04-25T09:00:51.185 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.185 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.185 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00969631 s, 108 MB/s 2024-04-25T09:00:51.194 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.194 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.194 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073428 s, 143 MB/s 2024-04-25T09:00:51.204 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.204 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.204 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742443 s, 141 MB/s 2024-04-25T09:00:51.214 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.214 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.214 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770639 s, 136 MB/s 2024-04-25T09:00:51.224 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.224 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.225 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829674 s, 126 MB/s 2024-04-25T09:00:51.237 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.237 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.237 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00999602 s, 105 MB/s 2024-04-25T09:00:51.247 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.247 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.247 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00785803 s, 133 MB/s 2024-04-25T09:00:51.257 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.258 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.258 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00837367 s, 125 MB/s 2024-04-25T09:00:51.267 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.267 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.267 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007478 s, 140 MB/s 2024-04-25T09:00:51.279 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.279 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.279 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00956755 s, 110 MB/s 2024-04-25T09:00:51.290 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.290 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.290 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00918878 s, 114 MB/s 2024-04-25T09:00:51.302 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.302 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.302 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00978667 s, 107 MB/s 2024-04-25T09:00:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.313 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780508 s, 134 MB/s 2024-04-25T09:00:51.323 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.323 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.323 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824645 s, 127 MB/s 2024-04-25T09:00:51.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:51.324 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:00:51.324 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:00:51.324 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T09:00:51.324 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:00:51.324 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:00:51.324 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:00:51.324 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:00:51.394 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.394 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.394 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00924351 s, 113 MB/s 2024-04-25T09:00:51.405 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.405 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.405 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00853597 s, 123 MB/s 2024-04-25T09:00:51.414 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.415 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.415 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752277 s, 139 MB/s 2024-04-25T09:00:51.424 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.424 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.424 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741372 s, 141 MB/s 2024-04-25T09:00:51.434 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.435 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.435 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0083418 s, 126 MB/s 2024-04-25T09:00:51.444 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.444 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.445 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743002 s, 141 MB/s 2024-04-25T09:00:51.456 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.456 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.456 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00944898 s, 111 MB/s 2024-04-25T09:00:51.468 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.468 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.468 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00973362 s, 108 MB/s 2024-04-25T09:00:51.479 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.479 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.479 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829516 s, 126 MB/s 2024-04-25T09:00:51.489 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.489 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.489 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762662 s, 137 MB/s 2024-04-25T09:00:51.501 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.501 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.501 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00976115 s, 107 MB/s 2024-04-25T09:00:51.512 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.513 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.513 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00951793 s, 110 MB/s 2024-04-25T09:00:51.524 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.524 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.524 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00918921 s, 114 MB/s 2024-04-25T09:00:51.534 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.534 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.535 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816964 s, 128 MB/s 2024-04-25T09:00:51.545 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.545 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.545 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00860695 s, 122 MB/s 2024-04-25T09:00:51.555 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.555 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.555 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749109 s, 140 MB/s 2024-04-25T09:00:51.565 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.566 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.566 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835785 s, 125 MB/s 2024-04-25T09:00:51.576 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.576 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.576 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810468 s, 129 MB/s 2024-04-25T09:00:51.588 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.588 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.588 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00966072 s, 109 MB/s 2024-04-25T09:00:51.597 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.598 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.598 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746566 s, 140 MB/s 2024-04-25T09:00:51.608 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.608 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.608 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817113 s, 128 MB/s 2024-04-25T09:00:51.620 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.620 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.620 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00968286 s, 108 MB/s 2024-04-25T09:00:51.631 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.631 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.631 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00920393 s, 114 MB/s 2024-04-25T09:00:51.641 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.641 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.641 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007221 s, 145 MB/s 2024-04-25T09:00:51.650 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.650 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.650 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724106 s, 145 MB/s 2024-04-25T09:00:51.662 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.662 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.662 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00952663 s, 110 MB/s 2024-04-25T09:00:51.673 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.673 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.673 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00894743 s, 117 MB/s 2024-04-25T09:00:51.684 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.684 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.684 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0088531 s, 118 MB/s 2024-04-25T09:00:51.695 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.695 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.695 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823391 s, 127 MB/s 2024-04-25T09:00:51.706 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.706 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.706 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00891162 s, 118 MB/s 2024-04-25T09:00:51.716 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.716 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.716 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839566 s, 125 MB/s 2024-04-25T09:00:51.728 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.728 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.728 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00901479 s, 116 MB/s 2024-04-25T09:00:51.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:51.729 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:00:51.729 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:00:51.729 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T09:00:51.729 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:00:51.729 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:00:51.729 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:00:51.729 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:00:51.798 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.799 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.799 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0095441 s, 110 MB/s 2024-04-25T09:00:51.809 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.809 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.809 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0087287 s, 120 MB/s 2024-04-25T09:00:51.820 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.820 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.820 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00864251 s, 121 MB/s 2024-04-25T09:00:51.831 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.831 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.831 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842731 s, 124 MB/s 2024-04-25T09:00:51.842 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.842 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.842 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00903917 s, 116 MB/s 2024-04-25T09:00:51.852 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.852 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.853 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00805809 s, 130 MB/s 2024-04-25T09:00:51.862 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.862 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.863 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00785523 s, 133 MB/s 2024-04-25T09:00:51.872 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.872 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.873 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765466 s, 137 MB/s 2024-04-25T09:00:51.883 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.883 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.883 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802587 s, 131 MB/s 2024-04-25T09:00:51.893 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.894 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.894 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838505 s, 125 MB/s 2024-04-25T09:00:51.904 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.904 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.904 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00877197 s, 120 MB/s 2024-04-25T09:00:51.914 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.914 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.914 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757467 s, 138 MB/s 2024-04-25T09:00:51.925 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.925 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.925 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00866587 s, 121 MB/s 2024-04-25T09:00:51.937 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.937 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.937 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00943342 s, 111 MB/s 2024-04-25T09:00:51.948 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.948 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.948 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00863162 s, 121 MB/s 2024-04-25T09:00:51.958 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.958 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.958 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00779781 s, 134 MB/s 2024-04-25T09:00:51.967 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.967 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.967 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669706 s, 157 MB/s 2024-04-25T09:00:51.977 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.977 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.978 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00848742 s, 124 MB/s 2024-04-25T09:00:51.988 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.988 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.988 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00870598 s, 120 MB/s 2024-04-25T09:00:51.999 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:51.999 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:51.999 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0087377 s, 120 MB/s 2024-04-25T09:00:52.010 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.010 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.010 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00844878 s, 124 MB/s 2024-04-25T09:00:52.021 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.021 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.021 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00909856 s, 115 MB/s 2024-04-25T09:00:52.031 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.031 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.031 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753999 s, 139 MB/s 2024-04-25T09:00:52.042 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.042 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.042 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00902596 s, 116 MB/s 2024-04-25T09:00:52.053 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.053 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.053 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861153 s, 122 MB/s 2024-04-25T09:00:52.064 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.064 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.065 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00885798 s, 118 MB/s 2024-04-25T09:00:52.074 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.074 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.075 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744918 s, 141 MB/s 2024-04-25T09:00:52.085 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.085 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.086 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00848944 s, 124 MB/s 2024-04-25T09:00:52.095 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.096 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.096 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753172 s, 139 MB/s 2024-04-25T09:00:52.106 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.106 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.107 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828188 s, 127 MB/s 2024-04-25T09:00:52.116 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.116 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.116 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693497 s, 151 MB/s 2024-04-25T09:00:52.126 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.126 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.126 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725068 s, 145 MB/s 2024-04-25T09:00:52.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:52.127 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:00:52.127 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:00:52.127 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T09:00:52.127 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:00:52.127 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:00:52.127 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:00:52.127 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:00:52.195 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.195 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.195 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00798885 s, 131 MB/s 2024-04-25T09:00:52.206 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.206 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.206 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00909212 s, 115 MB/s 2024-04-25T09:00:52.216 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.216 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.217 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00805161 s, 130 MB/s 2024-04-25T09:00:52.225 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.225 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.226 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679677 s, 154 MB/s 2024-04-25T09:00:52.234 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.234 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.234 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672806 s, 156 MB/s 2024-04-25T09:00:52.245 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.245 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.246 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00907662 s, 116 MB/s 2024-04-25T09:00:52.255 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.255 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.255 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702336 s, 149 MB/s 2024-04-25T09:00:52.265 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.265 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.266 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861078 s, 122 MB/s 2024-04-25T09:00:52.274 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.275 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.275 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670082 s, 156 MB/s 2024-04-25T09:00:52.284 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.284 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.284 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073599 s, 142 MB/s 2024-04-25T09:00:52.294 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.294 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.294 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731364 s, 143 MB/s 2024-04-25T09:00:52.304 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.304 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.304 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00849097 s, 123 MB/s 2024-04-25T09:00:52.314 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.314 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.314 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753422 s, 139 MB/s 2024-04-25T09:00:52.325 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.325 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.325 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00902881 s, 116 MB/s 2024-04-25T09:00:52.335 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.335 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.335 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076381 s, 137 MB/s 2024-04-25T09:00:52.344 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.345 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.345 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071535 s, 147 MB/s 2024-04-25T09:00:52.354 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.354 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.354 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074082 s, 142 MB/s 2024-04-25T09:00:52.363 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.363 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.363 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710632 s, 148 MB/s 2024-04-25T09:00:52.374 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.374 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.374 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829552 s, 126 MB/s 2024-04-25T09:00:52.384 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.384 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.384 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783995 s, 134 MB/s 2024-04-25T09:00:52.394 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.394 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.394 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00849727 s, 123 MB/s 2024-04-25T09:00:52.404 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.404 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.404 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766686 s, 137 MB/s 2024-04-25T09:00:52.413 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.413 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.414 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706084 s, 149 MB/s 2024-04-25T09:00:52.424 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.425 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.425 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0086949 s, 121 MB/s 2024-04-25T09:00:52.433 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.433 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.434 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067604 s, 155 MB/s 2024-04-25T09:00:52.445 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.445 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.445 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00893533 s, 117 MB/s 2024-04-25T09:00:52.454 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.454 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.454 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674552 s, 155 MB/s 2024-04-25T09:00:52.464 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.464 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.464 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767941 s, 137 MB/s 2024-04-25T09:00:52.473 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.473 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.473 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691698 s, 152 MB/s 2024-04-25T09:00:52.484 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.484 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.484 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00904463 s, 116 MB/s 2024-04-25T09:00:52.493 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.493 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.493 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698 s, 150 MB/s 2024-04-25T09:00:52.504 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.504 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.504 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00849365 s, 123 MB/s 2024-04-25T09:00:52.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:52.505 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:00:52.505 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:00:52.505 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T09:00:52.505 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:00:52.506 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:00:52.506 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:00:52.506 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:00:52.574 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.574 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.574 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00847232 s, 124 MB/s 2024-04-25T09:00:52.584 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.584 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.585 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817581 s, 128 MB/s 2024-04-25T09:00:52.593 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.593 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.593 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684706 s, 153 MB/s 2024-04-25T09:00:52.603 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.603 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.603 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712471 s, 147 MB/s 2024-04-25T09:00:52.612 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.613 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.613 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778913 s, 135 MB/s 2024-04-25T09:00:52.623 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.623 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.623 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850254 s, 123 MB/s 2024-04-25T09:00:52.634 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.634 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.634 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841425 s, 125 MB/s 2024-04-25T09:00:52.644 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.644 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.644 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767776 s, 137 MB/s 2024-04-25T09:00:52.654 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.655 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.655 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851539 s, 123 MB/s 2024-04-25T09:00:52.664 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.664 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.664 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731013 s, 143 MB/s 2024-04-25T09:00:52.675 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.675 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.675 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00875359 s, 120 MB/s 2024-04-25T09:00:52.685 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.685 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.685 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742237 s, 141 MB/s 2024-04-25T09:00:52.695 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.695 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.695 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840533 s, 125 MB/s 2024-04-25T09:00:52.704 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.704 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.704 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067223 s, 156 MB/s 2024-04-25T09:00:52.715 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.715 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.715 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832712 s, 126 MB/s 2024-04-25T09:00:52.724 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.725 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.725 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762624 s, 137 MB/s 2024-04-25T09:00:52.733 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.733 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.733 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685664 s, 153 MB/s 2024-04-25T09:00:52.744 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.744 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.744 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00860034 s, 122 MB/s 2024-04-25T09:00:52.754 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.754 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.754 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730823 s, 143 MB/s 2024-04-25T09:00:52.763 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.764 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.764 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761707 s, 138 MB/s 2024-04-25T09:00:52.774 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.774 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.774 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839278 s, 125 MB/s 2024-04-25T09:00:52.784 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.784 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.784 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751861 s, 139 MB/s 2024-04-25T09:00:52.794 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.794 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.794 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832315 s, 126 MB/s 2024-04-25T09:00:52.804 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.804 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.804 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00785023 s, 134 MB/s 2024-04-25T09:00:52.814 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.815 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.815 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815956 s, 129 MB/s 2024-04-25T09:00:52.826 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.826 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.826 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00880891 s, 119 MB/s 2024-04-25T09:00:52.836 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.836 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.836 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783904 s, 134 MB/s 2024-04-25T09:00:52.847 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.847 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.847 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797817 s, 131 MB/s 2024-04-25T09:00:52.858 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.858 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.858 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0091785 s, 114 MB/s 2024-04-25T09:00:52.870 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.871 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.871 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00966157 s, 109 MB/s 2024-04-25T09:00:52.882 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.882 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.882 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815087 s, 129 MB/s 2024-04-25T09:00:52.892 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.892 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.892 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.008106 s, 129 MB/s 2024-04-25T09:00:52.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:52.893 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:00:52.893 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:00:52.893 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T09:00:52.893 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:00:52.893 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:00:52.894 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:00:52.894 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:00:52.963 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.963 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.963 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765274 s, 137 MB/s 2024-04-25T09:00:52.973 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.973 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.974 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829855 s, 126 MB/s 2024-04-25T09:00:52.984 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.984 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.984 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834046 s, 126 MB/s 2024-04-25T09:00:52.995 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:52.996 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:52.996 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839375 s, 125 MB/s 2024-04-25T09:00:53.005 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:53.006 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:53.006 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747197 s, 140 MB/s 2024-04-25T09:00:53.017 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:53.017 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:53.017 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841486 s, 125 MB/s 2024-04-25T09:00:53.026 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:53.026 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:53.026 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689478 s, 152 MB/s 2024-04-25T09:00:53.037 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:53.037 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:53.037 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827314 s, 127 MB/s 2024-04-25T09:00:53.048 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:53.048 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:53.048 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00849432 s, 123 MB/s 2024-04-25T09:00:53.058 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:53.058 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:53.058 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708887 s, 148 MB/s 2024-04-25T09:00:53.069 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:53.069 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:53.069 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0081159 s, 129 MB/s 2024-04-25T09:00:53.079 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:53.080 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:53.080 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831573 s, 126 MB/s 2024-04-25T09:00:53.090 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:53.090 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:53.090 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821476 s, 128 MB/s 2024-04-25T09:00:53.101 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:53.101 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:53.101 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831246 s, 126 MB/s 2024-04-25T09:00:53.112 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:53.112 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:53.112 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829271 s, 126 MB/s 2024-04-25T09:00:53.123 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:53.123 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:53.123 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836203 s, 125 MB/s 2024-04-25T09:00:53.133 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:53.133 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:53.133 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748089 s, 140 MB/s 2024-04-25T09:00:53.144 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:53.144 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:53.144 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833793 s, 126 MB/s 2024-04-25T09:00:53.153 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:53.153 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:53.153 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658782 s, 159 MB/s 2024-04-25T09:00:53.164 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:53.164 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:53.164 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00859018 s, 122 MB/s 2024-04-25T09:00:53.174 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:53.174 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:53.174 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760438 s, 138 MB/s 2024-04-25T09:00:53.183 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:53.183 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:53.183 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733685 s, 143 MB/s 2024-04-25T09:00:53.194 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:53.194 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:53.194 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00906085 s, 116 MB/s 2024-04-25T09:00:53.205 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:53.205 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:53.205 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797272 s, 132 MB/s 2024-04-25T09:00:53.216 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:53.216 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:53.216 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00926406 s, 113 MB/s 2024-04-25T09:00:53.226 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:53.226 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:53.226 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746733 s, 140 MB/s 2024-04-25T09:00:53.238 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:53.238 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:53.238 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00982655 s, 107 MB/s 2024-04-25T09:00:53.247 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:53.247 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:53.247 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737644 s, 142 MB/s 2024-04-25T09:00:54.237 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.237 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.237 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.98704 s, 1.1 MB/s 2024-04-25T09:00:54.251 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.251 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.251 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112207 s, 93.5 MB/s 2024-04-25T09:00:54.263 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.263 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.263 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00921724 s, 114 MB/s 2024-04-25T09:00:54.274 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.274 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.274 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840076 s, 125 MB/s 2024-04-25T09:00:54.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:54.275 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:00:54.275 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:00:54.275 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T09:00:54.275 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:00:54.275 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:00:54.276 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:00:54.276 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:00:54.349 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.349 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.349 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00961203 s, 109 MB/s 2024-04-25T09:00:54.359 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.359 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.360 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815817 s, 129 MB/s 2024-04-25T09:00:54.369 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.369 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.369 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729918 s, 144 MB/s 2024-04-25T09:00:54.380 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.380 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.380 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846539 s, 124 MB/s 2024-04-25T09:00:54.390 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.391 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.391 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818851 s, 128 MB/s 2024-04-25T09:00:54.402 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.402 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.402 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0091682 s, 114 MB/s 2024-04-25T09:00:54.414 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.414 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.414 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00923427 s, 114 MB/s 2024-04-25T09:00:54.423 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.423 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.423 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689529 s, 152 MB/s 2024-04-25T09:00:54.433 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.433 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.433 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772434 s, 136 MB/s 2024-04-25T09:00:54.442 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.442 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.442 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719446 s, 146 MB/s 2024-04-25T09:00:54.451 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.452 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.452 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690362 s, 152 MB/s 2024-04-25T09:00:54.460 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.460 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.460 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671014 s, 156 MB/s 2024-04-25T09:00:54.470 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.470 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.470 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00781894 s, 134 MB/s 2024-04-25T09:00:54.479 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.479 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.479 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676391 s, 155 MB/s 2024-04-25T09:00:54.490 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.490 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.490 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00877505 s, 119 MB/s 2024-04-25T09:00:54.501 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.501 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.501 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00849357 s, 123 MB/s 2024-04-25T09:00:54.510 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.510 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.510 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726852 s, 144 MB/s 2024-04-25T09:00:54.521 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.521 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.521 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0081979 s, 128 MB/s 2024-04-25T09:00:54.531 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.532 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.532 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842906 s, 124 MB/s 2024-04-25T09:00:54.543 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.543 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.543 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00903463 s, 116 MB/s 2024-04-25T09:00:54.553 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.553 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.553 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00781365 s, 134 MB/s 2024-04-25T09:00:54.563 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.563 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.564 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832647 s, 126 MB/s 2024-04-25T09:00:54.572 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.572 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.572 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685913 s, 153 MB/s 2024-04-25T09:00:54.583 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.583 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.583 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835446 s, 126 MB/s 2024-04-25T09:00:54.594 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.594 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.594 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00848251 s, 124 MB/s 2024-04-25T09:00:54.603 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.603 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.603 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699801 s, 150 MB/s 2024-04-25T09:00:54.614 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.615 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.615 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741672 s, 141 MB/s 2024-04-25T09:00:54.623 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.623 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.623 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00812249 s, 129 MB/s 2024-04-25T09:00:54.632 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.633 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.633 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751667 s, 139 MB/s 2024-04-25T09:00:54.644 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.644 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.644 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00886618 s, 118 MB/s 2024-04-25T09:00:54.653 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.654 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.654 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773653 s, 136 MB/s 2024-04-25T09:00:54.663 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.663 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.663 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750394 s, 140 MB/s 2024-04-25T09:00:54.664 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:54.664 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:00:54.665 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:00:54.665 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T09:00:54.665 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:00:54.665 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:00:54.665 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:00:54.665 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:00:54.733 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.733 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.733 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776097 s, 135 MB/s 2024-04-25T09:00:54.744 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.744 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.744 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00895265 s, 117 MB/s 2024-04-25T09:00:54.753 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.753 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.753 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724359 s, 145 MB/s 2024-04-25T09:00:54.764 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.764 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.764 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00808076 s, 130 MB/s 2024-04-25T09:00:54.773 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.773 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.773 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682322 s, 154 MB/s 2024-04-25T09:00:54.783 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.783 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.783 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783553 s, 134 MB/s 2024-04-25T09:00:54.793 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.793 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.793 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758716 s, 138 MB/s 2024-04-25T09:00:54.803 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.803 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.803 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833259 s, 126 MB/s 2024-04-25T09:00:54.814 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.814 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.814 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084898 s, 124 MB/s 2024-04-25T09:00:54.824 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.824 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.824 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744227 s, 141 MB/s 2024-04-25T09:00:54.834 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.834 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.834 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762275 s, 138 MB/s 2024-04-25T09:00:54.844 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.844 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.844 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827515 s, 127 MB/s 2024-04-25T09:00:54.855 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.856 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.856 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00907122 s, 116 MB/s 2024-04-25T09:00:54.866 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.867 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.867 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00870842 s, 120 MB/s 2024-04-25T09:00:54.877 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.878 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.878 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00863281 s, 121 MB/s 2024-04-25T09:00:54.888 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.888 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.889 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084551 s, 124 MB/s 2024-04-25T09:00:54.899 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.899 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.899 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834445 s, 126 MB/s 2024-04-25T09:00:54.909 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.909 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.909 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821213 s, 128 MB/s 2024-04-25T09:00:54.931 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.931 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.931 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792676 s, 132 MB/s 2024-04-25T09:00:54.940 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.941 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.941 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720787 s, 145 MB/s 2024-04-25T09:00:54.950 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.950 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.951 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00801609 s, 131 MB/s 2024-04-25T09:00:54.962 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.962 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.962 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00939554 s, 112 MB/s 2024-04-25T09:00:54.972 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.972 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.972 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748773 s, 140 MB/s 2024-04-25T09:00:54.981 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.981 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.981 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713525 s, 147 MB/s 2024-04-25T09:00:54.991 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:54.991 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:54.991 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735902 s, 142 MB/s 2024-04-25T09:00:55.004 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.004 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.005 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0113716 s, 92.2 MB/s 2024-04-25T09:00:55.017 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.017 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.017 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100847 s, 104 MB/s 2024-04-25T09:00:55.027 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.027 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.027 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828493 s, 127 MB/s 2024-04-25T09:00:55.038 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.038 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.038 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827676 s, 127 MB/s 2024-04-25T09:00:55.048 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.049 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.049 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846346 s, 124 MB/s 2024-04-25T09:00:55.057 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.057 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.057 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670237 s, 156 MB/s 2024-04-25T09:00:55.067 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.067 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.067 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729159 s, 144 MB/s 2024-04-25T09:00:55.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:55.068 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:00:55.068 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:00:55.068 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T09:00:55.068 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:00:55.068 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:00:55.068 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:00:55.069 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:00:55.137 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.137 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.138 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746652 s, 140 MB/s 2024-04-25T09:00:55.146 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.146 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.146 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074868 s, 140 MB/s 2024-04-25T09:00:55.155 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.155 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.155 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668929 s, 157 MB/s 2024-04-25T09:00:55.165 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.166 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.166 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00789045 s, 133 MB/s 2024-04-25T09:00:55.175 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.175 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.175 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754506 s, 139 MB/s 2024-04-25T09:00:55.185 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.185 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.185 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729819 s, 144 MB/s 2024-04-25T09:00:55.195 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.195 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.196 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838818 s, 125 MB/s 2024-04-25T09:00:55.206 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.250 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.251 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00869192 s, 121 MB/s 2024-04-25T09:00:55.252 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.252 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.252 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00843909 s, 124 MB/s 2024-04-25T09:00:55.252 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.252 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.252 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084487 s, 124 MB/s 2024-04-25T09:00:55.252 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.252 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.252 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00852466 s, 123 MB/s 2024-04-25T09:00:55.252 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.252 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.252 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00809271 s, 130 MB/s 2024-04-25T09:00:55.260 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.260 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.260 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00859277 s, 122 MB/s 2024-04-25T09:00:55.270 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.270 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.270 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759339 s, 138 MB/s 2024-04-25T09:00:55.279 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.279 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.279 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734432 s, 143 MB/s 2024-04-25T09:00:55.288 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.288 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.289 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665856 s, 157 MB/s 2024-04-25T09:00:55.299 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.299 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.299 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834385 s, 126 MB/s 2024-04-25T09:00:55.309 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.309 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.309 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750523 s, 140 MB/s 2024-04-25T09:00:55.318 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.319 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.319 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074753 s, 140 MB/s 2024-04-25T09:00:55.327 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.328 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.328 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693474 s, 151 MB/s 2024-04-25T09:00:55.337 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.337 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.338 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753979 s, 139 MB/s 2024-04-25T09:00:55.348 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.348 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.348 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00843499 s, 124 MB/s 2024-04-25T09:00:55.358 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.358 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.358 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769761 s, 136 MB/s 2024-04-25T09:00:55.367 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.367 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.367 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679973 s, 154 MB/s 2024-04-25T09:00:55.377 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.377 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.377 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073966 s, 142 MB/s 2024-04-25T09:00:55.387 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.387 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.387 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827979 s, 127 MB/s 2024-04-25T09:00:55.396 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.396 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.396 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698693 s, 150 MB/s 2024-04-25T09:00:55.405 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.405 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.405 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068009 s, 154 MB/s 2024-04-25T09:00:55.416 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.416 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.416 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00859905 s, 122 MB/s 2024-04-25T09:00:55.426 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.426 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.426 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741983 s, 141 MB/s 2024-04-25T09:00:55.435 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.436 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.436 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736766 s, 142 MB/s 2024-04-25T09:00:55.445 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.445 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.446 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751191 s, 140 MB/s 2024-04-25T09:00:55.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:55.447 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:00:55.447 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:00:55.447 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T09:00:55.447 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:00:55.447 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:00:55.447 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:00:55.447 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:00:55.515 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.516 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.516 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836096 s, 125 MB/s 2024-04-25T09:00:55.524 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.524 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.524 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662281 s, 158 MB/s 2024-04-25T09:00:55.533 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.533 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.533 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072395 s, 145 MB/s 2024-04-25T09:00:55.543 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.543 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.543 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0078355 s, 134 MB/s 2024-04-25T09:00:55.554 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.554 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.554 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841626 s, 125 MB/s 2024-04-25T09:00:55.563 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.564 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.564 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750423 s, 140 MB/s 2024-04-25T09:00:55.574 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.574 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.574 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828982 s, 126 MB/s 2024-04-25T09:00:55.584 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.584 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.584 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831854 s, 126 MB/s 2024-04-25T09:00:55.593 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.594 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.594 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708543 s, 148 MB/s 2024-04-25T09:00:55.604 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.604 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.604 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.008202 s, 128 MB/s 2024-04-25T09:00:55.614 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.615 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.615 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826012 s, 127 MB/s 2024-04-25T09:00:55.624 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.624 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.624 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689184 s, 152 MB/s 2024-04-25T09:00:55.634 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.634 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.634 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0083021 s, 126 MB/s 2024-04-25T09:00:55.643 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.643 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.643 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00662956 s, 158 MB/s 2024-04-25T09:00:55.653 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.653 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.653 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00806712 s, 130 MB/s 2024-04-25T09:00:55.663 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.663 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.663 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742079 s, 141 MB/s 2024-04-25T09:00:55.672 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.672 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.673 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728199 s, 144 MB/s 2024-04-25T09:00:55.682 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.682 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.682 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725911 s, 144 MB/s 2024-04-25T09:00:55.691 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.691 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.691 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680395 s, 154 MB/s 2024-04-25T09:00:55.700 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.700 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.700 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676333 s, 155 MB/s 2024-04-25T09:00:55.709 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.709 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.709 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728798 s, 144 MB/s 2024-04-25T09:00:55.720 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.720 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.720 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00857404 s, 122 MB/s 2024-04-25T09:00:55.729 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.730 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.730 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759864 s, 138 MB/s 2024-04-25T09:00:55.740 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.740 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.741 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745226 s, 141 MB/s 2024-04-25T09:00:55.749 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.749 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.749 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718789 s, 146 MB/s 2024-04-25T09:00:55.759 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.759 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.759 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820908 s, 128 MB/s 2024-04-25T09:00:55.768 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.768 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.768 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675714 s, 155 MB/s 2024-04-25T09:00:55.779 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.779 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.779 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0085341 s, 123 MB/s 2024-04-25T09:00:55.789 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.790 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.790 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840034 s, 125 MB/s 2024-04-25T09:00:55.799 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.799 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.799 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730541 s, 144 MB/s 2024-04-25T09:00:55.810 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.810 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.810 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082053 s, 128 MB/s 2024-04-25T09:00:55.818 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.819 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.819 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673483 s, 156 MB/s 2024-04-25T09:00:55.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:55.820 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:00:55.820 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:00:55.820 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T09:00:55.820 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:00:55.820 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:00:55.820 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:00:55.820 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:00:55.889 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.889 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.889 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00785592 s, 133 MB/s 2024-04-25T09:00:55.898 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.898 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.898 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697256 s, 150 MB/s 2024-04-25T09:00:55.908 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.908 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.908 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762197 s, 138 MB/s 2024-04-25T09:00:55.945 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.945 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.945 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071197 s, 147 MB/s 2024-04-25T09:00:55.956 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.956 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.956 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825591 s, 127 MB/s 2024-04-25T09:00:55.965 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.965 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.965 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728499 s, 144 MB/s 2024-04-25T09:00:55.976 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.976 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.976 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00844129 s, 124 MB/s 2024-04-25T09:00:55.985 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.985 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.985 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727301 s, 144 MB/s 2024-04-25T09:00:55.995 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:55.995 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:55.995 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760867 s, 138 MB/s 2024-04-25T09:00:56.005 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.005 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.006 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819558 s, 128 MB/s 2024-04-25T09:00:56.016 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.016 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.016 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828671 s, 127 MB/s 2024-04-25T09:00:56.026 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.026 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.026 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749565 s, 140 MB/s 2024-04-25T09:00:56.036 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.036 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.037 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841687 s, 125 MB/s 2024-04-25T09:00:56.046 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.046 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.046 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759223 s, 138 MB/s 2024-04-25T09:00:56.056 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.056 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.056 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723336 s, 145 MB/s 2024-04-25T09:00:56.065 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.065 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.066 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740055 s, 142 MB/s 2024-04-25T09:00:56.075 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.075 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.075 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703909 s, 149 MB/s 2024-04-25T09:00:56.085 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.085 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.085 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771855 s, 136 MB/s 2024-04-25T09:00:56.094 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.094 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.094 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735413 s, 143 MB/s 2024-04-25T09:00:56.104 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.104 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.104 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725389 s, 145 MB/s 2024-04-25T09:00:56.113 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.113 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.113 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695509 s, 151 MB/s 2024-04-25T09:00:56.122 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.122 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.123 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073167 s, 143 MB/s 2024-04-25T09:00:56.134 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.135 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.135 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00942646 s, 111 MB/s 2024-04-25T09:00:56.146 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.146 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.146 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00995962 s, 105 MB/s 2024-04-25T09:00:56.155 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.155 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.155 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692255 s, 151 MB/s 2024-04-25T09:00:56.165 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.165 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.165 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733872 s, 143 MB/s 2024-04-25T09:00:56.175 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.175 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.175 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745867 s, 141 MB/s 2024-04-25T09:00:56.184 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.184 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.184 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699631 s, 150 MB/s 2024-04-25T09:00:56.194 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.194 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.194 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829859 s, 126 MB/s 2024-04-25T09:00:56.203 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.204 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.204 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694905 s, 151 MB/s 2024-04-25T09:00:56.214 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.214 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.214 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828985 s, 126 MB/s 2024-04-25T09:00:56.225 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.225 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.225 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829672 s, 126 MB/s 2024-04-25T09:00:56.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:56.226 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:00:56.226 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:00:56.226 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T09:00:56.226 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:00:56.226 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:00:56.226 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:00:56.226 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:00:56.293 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.293 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.293 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726111 s, 144 MB/s 2024-04-25T09:00:56.303 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.303 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.303 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731364 s, 143 MB/s 2024-04-25T09:00:56.313 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.313 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.313 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850842 s, 123 MB/s 2024-04-25T09:00:56.324 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.324 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.324 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.008214 s, 128 MB/s 2024-04-25T09:00:56.334 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.334 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.334 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832617 s, 126 MB/s 2024-04-25T09:00:56.345 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.345 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.345 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819471 s, 128 MB/s 2024-04-25T09:00:56.356 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.356 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.356 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00886011 s, 118 MB/s 2024-04-25T09:00:56.367 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.367 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.367 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0083952 s, 125 MB/s 2024-04-25T09:00:56.377 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.377 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.377 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822034 s, 128 MB/s 2024-04-25T09:00:56.387 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.387 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.387 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750387 s, 140 MB/s 2024-04-25T09:00:56.396 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.397 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.397 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075171 s, 139 MB/s 2024-04-25T09:00:56.406 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.406 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.406 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750044 s, 140 MB/s 2024-04-25T09:00:56.416 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.416 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.416 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753519 s, 139 MB/s 2024-04-25T09:00:56.425 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.425 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.425 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700368 s, 150 MB/s 2024-04-25T09:00:56.434 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.435 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.435 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709995 s, 148 MB/s 2024-04-25T09:00:56.445 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.445 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.445 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831782 s, 126 MB/s 2024-04-25T09:00:56.455 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.455 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.455 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00794266 s, 132 MB/s 2024-04-25T09:00:56.466 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.466 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.466 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846134 s, 124 MB/s 2024-04-25T09:00:56.476 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.477 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.477 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00843407 s, 124 MB/s 2024-04-25T09:00:56.487 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.487 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.487 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00805699 s, 130 MB/s 2024-04-25T09:00:56.497 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.497 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.497 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817018 s, 128 MB/s 2024-04-25T09:00:56.508 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.508 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.508 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834426 s, 126 MB/s 2024-04-25T09:00:56.517 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.518 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.518 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070698 s, 148 MB/s 2024-04-25T09:00:56.528 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.528 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.528 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00837158 s, 125 MB/s 2024-04-25T09:00:56.537 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.537 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.537 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708744 s, 148 MB/s 2024-04-25T09:00:56.546 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.546 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.546 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694052 s, 151 MB/s 2024-04-25T09:00:56.557 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.557 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.557 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827762 s, 127 MB/s 2024-04-25T09:00:56.566 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.566 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.566 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719328 s, 146 MB/s 2024-04-25T09:00:56.576 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.576 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.576 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726695 s, 144 MB/s 2024-04-25T09:00:56.586 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.587 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.587 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00809453 s, 130 MB/s 2024-04-25T09:00:56.597 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.597 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.598 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00877434 s, 120 MB/s 2024-04-25T09:00:56.608 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.608 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.608 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810315 s, 129 MB/s 2024-04-25T09:00:56.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:56.609 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:00:56.609 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:00:56.609 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T09:00:56.609 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:00:56.609 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:00:56.609 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:00:56.609 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:00:56.676 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.677 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.677 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739603 s, 142 MB/s 2024-04-25T09:00:56.685 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.685 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.685 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690592 s, 152 MB/s 2024-04-25T09:00:56.696 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.696 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.696 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842447 s, 124 MB/s 2024-04-25T09:00:56.705 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.705 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.705 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726277 s, 144 MB/s 2024-04-25T09:00:56.715 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.715 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.715 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724761 s, 145 MB/s 2024-04-25T09:00:56.726 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.726 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.726 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00848837 s, 124 MB/s 2024-04-25T09:00:56.735 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.735 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.736 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749722 s, 140 MB/s 2024-04-25T09:00:56.746 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.746 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.746 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836869 s, 125 MB/s 2024-04-25T09:00:56.755 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.755 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.755 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701776 s, 149 MB/s 2024-04-25T09:00:56.766 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.766 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.766 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00864424 s, 121 MB/s 2024-04-25T09:00:56.776 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.776 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.776 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759141 s, 138 MB/s 2024-04-25T09:00:56.787 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.787 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.787 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00855434 s, 123 MB/s 2024-04-25T09:00:56.798 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.798 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.798 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0087063 s, 120 MB/s 2024-04-25T09:00:56.809 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.809 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.809 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832424 s, 126 MB/s 2024-04-25T09:00:56.820 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.820 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.820 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00875165 s, 120 MB/s 2024-04-25T09:00:56.829 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.829 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.830 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739832 s, 142 MB/s 2024-04-25T09:00:56.840 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.840 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.840 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00848399 s, 124 MB/s 2024-04-25T09:00:56.849 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.849 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.849 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694092 s, 151 MB/s 2024-04-25T09:00:56.860 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.861 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.861 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00853719 s, 123 MB/s 2024-04-25T09:00:56.870 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.870 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.870 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736484 s, 142 MB/s 2024-04-25T09:00:56.878 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.879 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.879 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681235 s, 154 MB/s 2024-04-25T09:00:56.889 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.889 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.889 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00855914 s, 123 MB/s 2024-04-25T09:00:56.900 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.900 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.900 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00863709 s, 121 MB/s 2024-04-25T09:00:56.910 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.910 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.910 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073651 s, 142 MB/s 2024-04-25T09:00:56.926 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.926 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.926 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136735 s, 76.7 MB/s 2024-04-25T09:00:56.951 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.951 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.951 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842332 s, 124 MB/s 2024-04-25T09:00:56.960 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.961 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.961 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713993 s, 147 MB/s 2024-04-25T09:00:56.971 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.971 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.972 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00855143 s, 123 MB/s 2024-04-25T09:00:56.982 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.982 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.982 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00864672 s, 121 MB/s 2024-04-25T09:00:56.991 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:56.991 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:56.991 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069166 s, 152 MB/s 2024-04-25T09:00:57.002 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.002 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.002 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00866526 s, 121 MB/s 2024-04-25T09:00:57.012 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.012 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.012 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762283 s, 138 MB/s 2024-04-25T09:00:57.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:57.013 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:00:57.013 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:00:57.013 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T09:00:57.013 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:00:57.013 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:00:57.013 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:00:57.014 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:00:57.082 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.083 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.083 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00852804 s, 123 MB/s 2024-04-25T09:00:57.093 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.093 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.093 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825356 s, 127 MB/s 2024-04-25T09:00:57.103 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.104 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.104 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0083311 s, 126 MB/s 2024-04-25T09:00:57.114 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.114 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.114 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816072 s, 128 MB/s 2024-04-25T09:00:57.125 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.125 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.125 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0089314 s, 117 MB/s 2024-04-25T09:00:57.134 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.135 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.135 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703228 s, 149 MB/s 2024-04-25T09:00:57.145 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.145 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.145 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831878 s, 126 MB/s 2024-04-25T09:00:57.156 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.156 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.156 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845843 s, 124 MB/s 2024-04-25T09:00:57.167 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.167 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.167 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00882124 s, 119 MB/s 2024-04-25T09:00:57.176 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.177 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.177 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743942 s, 141 MB/s 2024-04-25T09:00:57.186 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.187 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.187 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00775891 s, 135 MB/s 2024-04-25T09:00:57.197 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.197 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.197 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00848769 s, 124 MB/s 2024-04-25T09:00:57.206 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.207 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.207 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727913 s, 144 MB/s 2024-04-25T09:00:57.218 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.218 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.218 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0086028 s, 122 MB/s 2024-04-25T09:00:57.227 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.227 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.228 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753517 s, 139 MB/s 2024-04-25T09:00:57.237 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.238 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.238 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787289 s, 133 MB/s 2024-04-25T09:00:57.248 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.248 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.248 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00855498 s, 123 MB/s 2024-04-25T09:00:57.258 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.259 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.259 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776821 s, 135 MB/s 2024-04-25T09:00:57.269 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.269 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.269 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00859316 s, 122 MB/s 2024-04-25T09:00:57.279 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.279 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.279 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737269 s, 142 MB/s 2024-04-25T09:00:57.288 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.289 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.289 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753941 s, 139 MB/s 2024-04-25T09:00:57.298 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.298 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.298 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700028 s, 150 MB/s 2024-04-25T09:00:57.308 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.308 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.308 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829275 s, 126 MB/s 2024-04-25T09:00:57.319 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.319 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.319 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826224 s, 127 MB/s 2024-04-25T09:00:57.328 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.328 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.328 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731021 s, 143 MB/s 2024-04-25T09:00:57.339 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.339 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.339 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836575 s, 125 MB/s 2024-04-25T09:00:57.350 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.350 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.350 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00866519 s, 121 MB/s 2024-04-25T09:00:57.359 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.359 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.359 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729454 s, 144 MB/s 2024-04-25T09:00:57.368 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.369 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.369 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702913 s, 149 MB/s 2024-04-25T09:00:57.379 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.379 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.379 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832563 s, 126 MB/s 2024-04-25T09:00:57.389 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.389 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.389 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735403 s, 143 MB/s 2024-04-25T09:00:57.399 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.399 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.399 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765404 s, 137 MB/s 2024-04-25T09:00:57.400 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:57.400 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:00:57.400 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:00:57.400 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T09:00:57.400 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:00:57.400 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:00:57.400 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:00:57.400 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:00:57.467 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.467 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.467 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673147 s, 156 MB/s 2024-04-25T09:00:57.476 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.476 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.476 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700628 s, 150 MB/s 2024-04-25T09:00:57.486 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.486 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.486 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741711 s, 141 MB/s 2024-04-25T09:00:57.495 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.496 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.496 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740952 s, 142 MB/s 2024-04-25T09:00:57.506 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.506 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.506 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00848797 s, 124 MB/s 2024-04-25T09:00:57.516 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.516 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.516 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741126 s, 141 MB/s 2024-04-25T09:00:57.525 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.525 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.525 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707335 s, 148 MB/s 2024-04-25T09:00:57.534 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.535 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.535 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069275 s, 151 MB/s 2024-04-25T09:00:57.545 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.546 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.546 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084747 s, 124 MB/s 2024-04-25T09:00:57.556 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.556 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.556 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082204 s, 128 MB/s 2024-04-25T09:00:57.566 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.566 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.566 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00806455 s, 130 MB/s 2024-04-25T09:00:57.577 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.577 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.577 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826992 s, 127 MB/s 2024-04-25T09:00:57.587 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.587 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.587 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00798479 s, 131 MB/s 2024-04-25T09:00:57.597 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.597 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.597 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727789 s, 144 MB/s 2024-04-25T09:00:57.607 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.607 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.607 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846539 s, 124 MB/s 2024-04-25T09:00:57.617 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.617 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.617 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721503 s, 145 MB/s 2024-04-25T09:00:57.625 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.626 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.626 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671915 s, 156 MB/s 2024-04-25T09:00:57.634 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.635 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.635 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00657355 s, 160 MB/s 2024-04-25T09:00:57.645 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.645 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.645 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804861 s, 130 MB/s 2024-04-25T09:00:57.655 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.655 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.655 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776508 s, 135 MB/s 2024-04-25T09:00:57.664 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.664 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.664 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742661 s, 141 MB/s 2024-04-25T09:00:57.674 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.674 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.674 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725116 s, 145 MB/s 2024-04-25T09:00:57.684 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.684 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.684 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764186 s, 137 MB/s 2024-04-25T09:00:57.693 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.693 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.693 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687963 s, 152 MB/s 2024-04-25T09:00:57.704 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.704 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.704 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00862163 s, 122 MB/s 2024-04-25T09:00:57.715 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.715 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.715 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00881251 s, 119 MB/s 2024-04-25T09:00:57.725 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.725 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.726 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831843 s, 126 MB/s 2024-04-25T09:00:57.735 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.735 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.735 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761411 s, 138 MB/s 2024-04-25T09:00:57.745 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.745 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.745 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753621 s, 139 MB/s 2024-04-25T09:00:57.755 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.755 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.755 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746778 s, 140 MB/s 2024-04-25T09:00:57.764 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.764 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.764 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747337 s, 140 MB/s 2024-04-25T09:00:57.775 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:57.775 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:57.775 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836433 s, 125 MB/s 2024-04-25T09:00:57.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:57.776 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:00:57.776 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:00:57.776 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T09:00:57.776 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:00:57.776 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:00:57.776 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:00:57.776 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:00:58.003 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.003 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.003 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753873 s, 139 MB/s 2024-04-25T09:00:58.013 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.013 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.013 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832015 s, 126 MB/s 2024-04-25T09:00:58.023 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.023 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.023 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747497 s, 140 MB/s 2024-04-25T09:00:58.034 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.034 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.034 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00848649 s, 124 MB/s 2024-04-25T09:00:58.043 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.043 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.043 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714764 s, 147 MB/s 2024-04-25T09:00:58.054 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.054 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.054 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00871479 s, 120 MB/s 2024-04-25T09:00:58.065 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.065 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.065 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831399 s, 126 MB/s 2024-04-25T09:00:58.075 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.075 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.075 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00808577 s, 130 MB/s 2024-04-25T09:00:58.084 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.084 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.084 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691244 s, 152 MB/s 2024-04-25T09:00:58.094 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.094 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.094 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727974 s, 144 MB/s 2024-04-25T09:00:58.104 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.104 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.104 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084106 s, 125 MB/s 2024-04-25T09:00:58.115 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.115 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.115 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840281 s, 125 MB/s 2024-04-25T09:00:58.127 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.127 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.127 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830499 s, 126 MB/s 2024-04-25T09:00:58.136 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.136 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.136 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835811 s, 125 MB/s 2024-04-25T09:00:58.146 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.146 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.147 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082854 s, 127 MB/s 2024-04-25T09:00:58.157 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.157 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.157 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835899 s, 125 MB/s 2024-04-25T09:00:58.168 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.168 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.168 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00790973 s, 133 MB/s 2024-04-25T09:00:58.179 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.179 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.179 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00856489 s, 122 MB/s 2024-04-25T09:00:58.189 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.189 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.190 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00847404 s, 124 MB/s 2024-04-25T09:00:58.199 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.199 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.199 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073923 s, 142 MB/s 2024-04-25T09:00:58.210 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.210 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.211 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00915908 s, 114 MB/s 2024-04-25T09:00:58.220 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.220 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.220 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725414 s, 145 MB/s 2024-04-25T09:00:58.229 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.230 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.230 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742344 s, 141 MB/s 2024-04-25T09:00:58.240 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.240 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.240 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763199 s, 137 MB/s 2024-04-25T09:00:58.250 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.250 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.250 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823209 s, 127 MB/s 2024-04-25T09:00:58.260 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.261 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.261 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822908 s, 127 MB/s 2024-04-25T09:00:58.271 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.271 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.271 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797482 s, 131 MB/s 2024-04-25T09:00:58.280 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.280 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.280 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709725 s, 148 MB/s 2024-04-25T09:00:58.289 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732903 s, 143 MB/s 2024-04-25T09:00:58.299 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.299 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.299 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740771 s, 142 MB/s 2024-04-25T09:00:58.309 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.309 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.309 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00782974 s, 134 MB/s 2024-04-25T09:00:58.319 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.319 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.319 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722052 s, 145 MB/s 2024-04-25T09:00:58.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:58.320 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:00:58.320 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:00:58.320 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T09:00:58.320 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:00:58.320 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:00:58.320 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:00:58.320 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:00:58.347 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.348 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.348 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725827 s, 144 MB/s 2024-04-25T09:00:58.358 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.358 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.358 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00847995 s, 124 MB/s 2024-04-25T09:00:58.368 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.368 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.368 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735733 s, 143 MB/s 2024-04-25T09:00:58.377 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.377 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.377 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720878 s, 145 MB/s 2024-04-25T09:00:58.386 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.387 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.387 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751647 s, 140 MB/s 2024-04-25T09:00:58.396 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.396 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.396 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686556 s, 153 MB/s 2024-04-25T09:00:58.405 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.405 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.405 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730155 s, 144 MB/s 2024-04-25T09:00:58.414 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.414 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.414 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670533 s, 156 MB/s 2024-04-25T09:00:58.423 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.423 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.423 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700828 s, 150 MB/s 2024-04-25T09:00:58.432 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.432 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.432 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689232 s, 152 MB/s 2024-04-25T09:00:58.441 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.441 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.442 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699113 s, 150 MB/s 2024-04-25T09:00:58.450 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.450 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.451 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679528 s, 154 MB/s 2024-04-25T09:00:58.460 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.460 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.460 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749897 s, 140 MB/s 2024-04-25T09:00:58.470 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.470 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.470 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739018 s, 142 MB/s 2024-04-25T09:00:58.479 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.479 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.479 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711385 s, 147 MB/s 2024-04-25T09:00:58.490 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.490 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.490 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833112 s, 126 MB/s 2024-04-25T09:00:58.499 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.499 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.499 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704783 s, 149 MB/s 2024-04-25T09:00:58.510 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.510 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.510 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00837019 s, 125 MB/s 2024-04-25T09:00:58.520 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.520 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.520 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828184 s, 127 MB/s 2024-04-25T09:00:58.531 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.531 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.531 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846963 s, 124 MB/s 2024-04-25T09:00:58.541 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.541 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.541 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824469 s, 127 MB/s 2024-04-25T09:00:58.551 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.551 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.551 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712423 s, 147 MB/s 2024-04-25T09:00:58.561 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.561 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.561 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825958 s, 127 MB/s 2024-04-25T09:00:58.571 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.571 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.571 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726564 s, 144 MB/s 2024-04-25T09:00:58.580 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.580 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.580 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728558 s, 144 MB/s 2024-04-25T09:00:58.590 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.591 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.591 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835575 s, 125 MB/s 2024-04-25T09:00:58.601 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.601 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.601 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00867554 s, 121 MB/s 2024-04-25T09:00:58.611 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.611 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.611 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777798 s, 135 MB/s 2024-04-25T09:00:58.620 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.620 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.620 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686444 s, 153 MB/s 2024-04-25T09:00:58.630 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.630 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.630 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735678 s, 143 MB/s 2024-04-25T09:00:58.640 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.640 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.640 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753569 s, 139 MB/s 2024-04-25T09:00:58.649 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.649 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.650 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075006 s, 140 MB/s 2024-04-25T09:00:58.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:58.651 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:00:58.651 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:00:58.651 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T09:00:58.651 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:00:58.651 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:00:58.651 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:00:58.651 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:00:58.719 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.719 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.719 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825241 s, 127 MB/s 2024-04-25T09:00:58.728 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.728 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.729 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747872 s, 140 MB/s 2024-04-25T09:00:58.739 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.739 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.739 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850886 s, 123 MB/s 2024-04-25T09:00:58.749 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.749 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.749 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730697 s, 144 MB/s 2024-04-25T09:00:58.759 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.759 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.759 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084009 s, 125 MB/s 2024-04-25T09:00:58.769 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.769 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.769 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724944 s, 145 MB/s 2024-04-25T09:00:58.779 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.779 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.779 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825621 s, 127 MB/s 2024-04-25T09:00:58.789 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.789 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.789 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750657 s, 140 MB/s 2024-04-25T09:00:58.798 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.798 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.798 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710011 s, 148 MB/s 2024-04-25T09:00:58.807 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.807 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.807 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680333 s, 154 MB/s 2024-04-25T09:00:58.818 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.818 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.818 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00880403 s, 119 MB/s 2024-04-25T09:00:58.828 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.828 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.828 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765295 s, 137 MB/s 2024-04-25T09:00:58.838 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.838 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.838 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754675 s, 139 MB/s 2024-04-25T09:00:58.847 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.847 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.848 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754404 s, 139 MB/s 2024-04-25T09:00:58.857 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.858 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.858 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0080584 s, 130 MB/s 2024-04-25T09:00:58.867 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.867 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.867 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737263 s, 142 MB/s 2024-04-25T09:00:58.877 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.877 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.877 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739229 s, 142 MB/s 2024-04-25T09:00:58.886 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.886 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.886 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722732 s, 145 MB/s 2024-04-25T09:00:58.895 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.896 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.896 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725369 s, 145 MB/s 2024-04-25T09:00:58.904 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.905 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.905 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694731 s, 151 MB/s 2024-04-25T09:00:58.915 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.915 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.915 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836815 s, 125 MB/s 2024-04-25T09:00:58.926 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.926 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.926 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828289 s, 127 MB/s 2024-04-25T09:00:58.940 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.940 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.940 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118981 s, 88.1 MB/s 2024-04-25T09:00:58.949 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.949 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.949 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718422 s, 146 MB/s 2024-04-25T09:00:58.959 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.959 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.959 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754008 s, 139 MB/s 2024-04-25T09:00:58.968 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.968 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.968 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741949 s, 141 MB/s 2024-04-25T09:00:58.979 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.979 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.979 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00879148 s, 119 MB/s 2024-04-25T09:00:58.989 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:58.990 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:58.990 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814583 s, 129 MB/s 2024-04-25T09:00:59.000 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.000 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.000 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082414 s, 127 MB/s 2024-04-25T09:00:59.009 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.009 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.009 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715622 s, 147 MB/s 2024-04-25T09:00:59.020 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.020 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.020 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0081568 s, 129 MB/s 2024-04-25T09:00:59.030 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.030 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.030 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845078 s, 124 MB/s 2024-04-25T09:00:59.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:59.031 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:00:59.032 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:00:59.032 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T09:00:59.032 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:00:59.032 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:00:59.032 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:00:59.032 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:00:59.100 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.101 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.101 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839951 s, 125 MB/s 2024-04-25T09:00:59.110 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.110 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.110 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707436 s, 148 MB/s 2024-04-25T09:00:59.120 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.121 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.121 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850123 s, 123 MB/s 2024-04-25T09:00:59.131 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.131 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.131 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0085335 s, 123 MB/s 2024-04-25T09:00:59.142 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.142 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.142 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829249 s, 126 MB/s 2024-04-25T09:00:59.152 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.152 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.153 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829455 s, 126 MB/s 2024-04-25T09:00:59.162 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.162 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.162 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737257 s, 142 MB/s 2024-04-25T09:00:59.172 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.172 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.172 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073587 s, 142 MB/s 2024-04-25T09:00:59.181 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.181 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.181 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712041 s, 147 MB/s 2024-04-25T09:00:59.191 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.191 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.191 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783116 s, 134 MB/s 2024-04-25T09:00:59.201 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.201 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.201 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744669 s, 141 MB/s 2024-04-25T09:00:59.211 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.211 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.212 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830902 s, 126 MB/s 2024-04-25T09:00:59.222 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.222 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.222 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00805992 s, 130 MB/s 2024-04-25T09:00:59.232 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.233 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.233 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840608 s, 125 MB/s 2024-04-25T09:00:59.243 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.243 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.243 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00862502 s, 122 MB/s 2024-04-25T09:00:59.253 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.253 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.253 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709638 s, 148 MB/s 2024-04-25T09:00:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771213 s, 136 MB/s 2024-04-25T09:00:59.272 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.272 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.272 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072005 s, 146 MB/s 2024-04-25T09:00:59.282 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.282 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.282 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832032 s, 126 MB/s 2024-04-25T09:00:59.292 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.292 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.292 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760319 s, 138 MB/s 2024-04-25T09:00:59.303 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.303 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.303 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00871319 s, 120 MB/s 2024-04-25T09:00:59.312 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.312 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.312 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679484 s, 154 MB/s 2024-04-25T09:00:59.322 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.322 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.322 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737787 s, 142 MB/s 2024-04-25T09:00:59.331 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.331 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.331 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701785 s, 149 MB/s 2024-04-25T09:00:59.341 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.341 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.341 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733963 s, 143 MB/s 2024-04-25T09:00:59.350 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.350 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.351 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759127 s, 138 MB/s 2024-04-25T09:00:59.361 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.361 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.361 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827421 s, 127 MB/s 2024-04-25T09:00:59.370 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.370 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.370 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689024 s, 152 MB/s 2024-04-25T09:00:59.380 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.380 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.380 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727788 s, 144 MB/s 2024-04-25T09:00:59.390 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.391 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.391 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738983 s, 142 MB/s 2024-04-25T09:00:59.399 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.399 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.399 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743073 s, 141 MB/s 2024-04-25T09:00:59.410 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.410 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.410 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839996 s, 125 MB/s 2024-04-25T09:00:59.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:59.411 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:00:59.411 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:00:59.411 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T09:00:59.412 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:00:59.412 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:00:59.412 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:00:59.412 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:00:59.480 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.480 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.480 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818031 s, 128 MB/s 2024-04-25T09:00:59.490 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.490 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.491 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823573 s, 127 MB/s 2024-04-25T09:00:59.501 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.501 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.501 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084757 s, 124 MB/s 2024-04-25T09:00:59.511 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.511 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.511 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810368 s, 129 MB/s 2024-04-25T09:00:59.521 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.521 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.521 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721628 s, 145 MB/s 2024-04-25T09:00:59.530 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.530 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.530 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691356 s, 152 MB/s 2024-04-25T09:00:59.539 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.539 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.539 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693734 s, 151 MB/s 2024-04-25T09:00:59.549 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.549 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.549 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760305 s, 138 MB/s 2024-04-25T09:00:59.558 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.558 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.558 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677459 s, 155 MB/s 2024-04-25T09:00:59.568 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.568 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.568 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074782 s, 140 MB/s 2024-04-25T09:00:59.577 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.577 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.577 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753234 s, 139 MB/s 2024-04-25T09:00:59.588 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.588 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.588 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00882339 s, 119 MB/s 2024-04-25T09:00:59.598 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.598 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.598 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753324 s, 139 MB/s 2024-04-25T09:00:59.609 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.609 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.609 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845324 s, 124 MB/s 2024-04-25T09:00:59.618 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.618 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.618 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707622 s, 148 MB/s 2024-04-25T09:00:59.629 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.629 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.629 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00853757 s, 123 MB/s 2024-04-25T09:00:59.640 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.640 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.640 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00877073 s, 120 MB/s 2024-04-25T09:00:59.649 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.650 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.650 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737774 s, 142 MB/s 2024-04-25T09:00:59.659 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.659 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.659 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692954 s, 151 MB/s 2024-04-25T09:00:59.668 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.668 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.668 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705351 s, 149 MB/s 2024-04-25T09:00:59.678 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.678 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.678 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752947 s, 139 MB/s 2024-04-25T09:00:59.688 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.688 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.688 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830869 s, 126 MB/s 2024-04-25T09:00:59.699 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.699 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.699 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845913 s, 124 MB/s 2024-04-25T09:00:59.708 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.708 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.708 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697321 s, 150 MB/s 2024-04-25T09:00:59.718 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.718 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.718 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745366 s, 141 MB/s 2024-04-25T09:00:59.727 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.728 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.728 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732781 s, 143 MB/s 2024-04-25T09:00:59.738 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.739 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.739 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00863029 s, 121 MB/s 2024-04-25T09:00:59.748 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.748 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.748 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748098 s, 140 MB/s 2024-04-25T09:00:59.759 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.759 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.759 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00843822 s, 124 MB/s 2024-04-25T09:00:59.769 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.769 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.770 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835772 s, 125 MB/s 2024-04-25T09:00:59.781 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.781 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.781 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00880979 s, 119 MB/s 2024-04-25T09:00:59.791 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.792 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.792 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00855583 s, 123 MB/s 2024-04-25T09:00:59.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:00:59.793 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:00:59.793 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:00:59.793 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T09:00:59.793 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:00:59.793 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:00:59.793 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:00:59.793 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:00:59.860 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.861 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.861 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738506 s, 142 MB/s 2024-04-25T09:00:59.870 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.870 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.870 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728733 s, 144 MB/s 2024-04-25T09:00:59.880 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.880 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.880 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082953 s, 126 MB/s 2024-04-25T09:00:59.891 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.891 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.891 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830643 s, 126 MB/s 2024-04-25T09:00:59.902 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.902 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.902 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00874834 s, 120 MB/s 2024-04-25T09:00:59.911 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.912 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.912 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751214 s, 140 MB/s 2024-04-25T09:00:59.921 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.921 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.921 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700839 s, 150 MB/s 2024-04-25T09:00:59.931 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.932 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.932 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846571 s, 124 MB/s 2024-04-25T09:00:59.942 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.943 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.943 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00865959 s, 121 MB/s 2024-04-25T09:00:59.963 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.964 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.964 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070145 s, 149 MB/s 2024-04-25T09:00:59.974 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.974 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.974 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835931 s, 125 MB/s 2024-04-25T09:00:59.984 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.984 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.984 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00795118 s, 132 MB/s 2024-04-25T09:00:59.994 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:00:59.994 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:00:59.994 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715838 s, 146 MB/s 2024-04-25T09:01:00.005 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.005 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.005 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00863206 s, 121 MB/s 2024-04-25T09:01:00.015 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.015 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.016 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840547 s, 125 MB/s 2024-04-25T09:01:00.026 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.026 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.026 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829044 s, 126 MB/s 2024-04-25T09:01:00.037 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.037 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.037 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00853701 s, 123 MB/s 2024-04-25T09:01:00.047 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.047 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.047 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00779896 s, 134 MB/s 2024-04-25T09:01:00.057 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.057 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.057 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00781312 s, 134 MB/s 2024-04-25T09:01:00.067 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.067 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.067 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831293 s, 126 MB/s 2024-04-25T09:01:00.077 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.077 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.077 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700019 s, 150 MB/s 2024-04-25T09:01:00.087 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.087 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.088 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00837662 s, 125 MB/s 2024-04-25T09:01:00.097 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.097 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.097 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706683 s, 148 MB/s 2024-04-25T09:01:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.107 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.108 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836511 s, 125 MB/s 2024-04-25T09:01:00.117 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.117 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.117 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076538 s, 137 MB/s 2024-04-25T09:01:00.128 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.128 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.128 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840924 s, 125 MB/s 2024-04-25T09:01:00.137 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.137 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.137 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710353 s, 148 MB/s 2024-04-25T09:01:00.147 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.147 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.147 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740556 s, 142 MB/s 2024-04-25T09:01:00.157 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.157 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.157 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819047 s, 128 MB/s 2024-04-25T09:01:00.167 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.167 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.167 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761602 s, 138 MB/s 2024-04-25T09:01:00.178 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.178 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.178 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850432 s, 123 MB/s 2024-04-25T09:01:00.188 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.189 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.189 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0083583 s, 125 MB/s 2024-04-25T09:01:00.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:00.190 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:01:00.190 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:01:00.190 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T09:01:00.190 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:01:00.190 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:01:00.190 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:01:00.190 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:01:00.259 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.259 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.259 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00880613 s, 119 MB/s 2024-04-25T09:01:00.270 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.270 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.270 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861865 s, 122 MB/s 2024-04-25T09:01:00.279 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.279 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.279 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701431 s, 149 MB/s 2024-04-25T09:01:00.289 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.289 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.289 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731694 s, 143 MB/s 2024-04-25T09:01:00.299 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.299 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.300 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00844438 s, 124 MB/s 2024-04-25T09:01:00.310 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.311 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.311 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00844356 s, 124 MB/s 2024-04-25T09:01:00.321 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.321 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.321 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00859237 s, 122 MB/s 2024-04-25T09:01:00.330 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.330 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.330 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695199 s, 151 MB/s 2024-04-25T09:01:00.341 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.341 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.342 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00871066 s, 120 MB/s 2024-04-25T09:01:00.352 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.352 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.352 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821563 s, 128 MB/s 2024-04-25T09:01:00.361 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.361 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.361 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721443 s, 145 MB/s 2024-04-25T09:01:00.372 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.372 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.372 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822372 s, 128 MB/s 2024-04-25T09:01:00.382 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.382 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.382 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834958 s, 126 MB/s 2024-04-25T09:01:00.391 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.392 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.392 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699231 s, 150 MB/s 2024-04-25T09:01:00.401 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.401 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.401 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716884 s, 146 MB/s 2024-04-25T09:01:00.412 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.412 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.412 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00847279 s, 124 MB/s 2024-04-25T09:01:00.421 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.421 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.421 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703907 s, 149 MB/s 2024-04-25T09:01:00.431 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.431 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.431 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727106 s, 144 MB/s 2024-04-25T09:01:00.440 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.440 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.441 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075047 s, 140 MB/s 2024-04-25T09:01:00.450 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.450 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.450 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742213 s, 141 MB/s 2024-04-25T09:01:00.460 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.460 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.460 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769278 s, 136 MB/s 2024-04-25T09:01:00.471 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.471 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.471 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0085888 s, 122 MB/s 2024-04-25T09:01:00.481 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.481 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.481 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00813127 s, 129 MB/s 2024-04-25T09:01:00.492 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.492 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.492 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00848907 s, 124 MB/s 2024-04-25T09:01:00.502 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.502 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.502 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816737 s, 128 MB/s 2024-04-25T09:01:00.511 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.512 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.512 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698271 s, 150 MB/s 2024-04-25T09:01:00.521 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.521 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.521 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733213 s, 143 MB/s 2024-04-25T09:01:00.532 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.532 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.532 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836607 s, 125 MB/s 2024-04-25T09:01:00.542 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.543 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.543 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00853397 s, 123 MB/s 2024-04-25T09:01:00.554 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.554 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.554 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839769 s, 125 MB/s 2024-04-25T09:01:00.564 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.564 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.564 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00867871 s, 121 MB/s 2024-04-25T09:01:00.574 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.574 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.574 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706146 s, 148 MB/s 2024-04-25T09:01:00.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:00.575 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:01:00.575 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:01:00.575 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T09:01:00.575 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:01:00.575 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:01:00.575 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:01:00.575 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:01:00.642 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.642 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.643 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700046 s, 150 MB/s 2024-04-25T09:01:00.651 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.651 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.651 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702783 s, 149 MB/s 2024-04-25T09:01:00.662 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.662 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.662 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00860685 s, 122 MB/s 2024-04-25T09:01:00.673 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.673 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.673 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829063 s, 126 MB/s 2024-04-25T09:01:00.682 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.682 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.682 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719492 s, 146 MB/s 2024-04-25T09:01:00.693 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.693 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.693 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00858722 s, 122 MB/s 2024-04-25T09:01:00.703 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.703 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.703 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756745 s, 139 MB/s 2024-04-25T09:01:00.714 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.714 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.714 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841488 s, 125 MB/s 2024-04-25T09:01:00.723 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.723 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.723 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739738 s, 142 MB/s 2024-04-25T09:01:00.732 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.732 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.732 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684529 s, 153 MB/s 2024-04-25T09:01:00.744 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.744 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.744 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00858306 s, 122 MB/s 2024-04-25T09:01:00.754 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.754 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.754 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084951 s, 123 MB/s 2024-04-25T09:01:00.765 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.765 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.765 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845279 s, 124 MB/s 2024-04-25T09:01:00.774 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.775 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.775 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075335 s, 139 MB/s 2024-04-25T09:01:00.784 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.784 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.784 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074246 s, 141 MB/s 2024-04-25T09:01:00.794 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.794 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.795 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825016 s, 127 MB/s 2024-04-25T09:01:00.805 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.805 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.805 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00849 s, 124 MB/s 2024-04-25T09:01:00.814 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.814 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.814 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696079 s, 151 MB/s 2024-04-25T09:01:00.824 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.824 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.824 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00779025 s, 135 MB/s 2024-04-25T09:01:00.835 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.835 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.835 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846422 s, 124 MB/s 2024-04-25T09:01:00.845 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.845 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.845 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759868 s, 138 MB/s 2024-04-25T09:01:00.855 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.855 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.855 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00781423 s, 134 MB/s 2024-04-25T09:01:00.865 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.865 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.865 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738578 s, 142 MB/s 2024-04-25T09:01:00.874 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.874 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.874 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715323 s, 147 MB/s 2024-04-25T09:01:00.884 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.884 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.884 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741004 s, 142 MB/s 2024-04-25T09:01:00.894 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.894 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.894 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842706 s, 124 MB/s 2024-04-25T09:01:00.905 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.905 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.905 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834264 s, 126 MB/s 2024-04-25T09:01:00.916 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.916 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.916 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00875375 s, 120 MB/s 2024-04-25T09:01:00.926 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.926 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.927 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746128 s, 141 MB/s 2024-04-25T09:01:00.936 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.936 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.937 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846285 s, 124 MB/s 2024-04-25T09:01:00.948 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.948 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.948 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770144 s, 136 MB/s 2024-04-25T09:01:00.991 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:00.991 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:00.991 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846413 s, 124 MB/s 2024-04-25T09:01:00.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:00.993 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:01:00.993 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:01:00.993 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T09:01:00.994 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:01:00.994 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:01:00.994 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:01:00.994 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:01:01.062 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.063 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.063 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840287 s, 125 MB/s 2024-04-25T09:01:01.073 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.073 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.073 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0083368 s, 126 MB/s 2024-04-25T09:01:01.084 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.084 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.084 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00837403 s, 125 MB/s 2024-04-25T09:01:01.093 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.094 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.094 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752132 s, 139 MB/s 2024-04-25T09:01:01.104 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.104 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.104 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821742 s, 128 MB/s 2024-04-25T09:01:01.113 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.113 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.114 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728731 s, 144 MB/s 2024-04-25T09:01:01.122 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.122 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.122 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006897 s, 152 MB/s 2024-04-25T09:01:01.132 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.132 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.132 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721517 s, 145 MB/s 2024-04-25T09:01:01.143 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.143 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.143 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00867535 s, 121 MB/s 2024-04-25T09:01:01.154 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.154 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.155 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00869755 s, 121 MB/s 2024-04-25T09:01:01.163 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.163 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.164 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709013 s, 148 MB/s 2024-04-25T09:01:01.174 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.174 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.174 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836941 s, 125 MB/s 2024-04-25T09:01:01.183 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.183 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.183 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688159 s, 152 MB/s 2024-04-25T09:01:01.193 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.194 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.194 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00854538 s, 123 MB/s 2024-04-25T09:01:01.203 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.203 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.203 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742725 s, 141 MB/s 2024-04-25T09:01:01.213 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.213 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.213 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758578 s, 138 MB/s 2024-04-25T09:01:01.223 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.223 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.223 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733269 s, 143 MB/s 2024-04-25T09:01:01.233 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.233 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.233 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706591 s, 148 MB/s 2024-04-25T09:01:01.243 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.243 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.243 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739971 s, 142 MB/s 2024-04-25T09:01:01.254 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.254 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.254 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084205 s, 125 MB/s 2024-04-25T09:01:01.263 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.263 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.263 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067615 s, 155 MB/s 2024-04-25T09:01:01.274 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.274 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.274 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774217 s, 135 MB/s 2024-04-25T09:01:01.284 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.284 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.284 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825951 s, 127 MB/s 2024-04-25T09:01:01.296 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.296 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.296 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00866454 s, 121 MB/s 2024-04-25T09:01:01.306 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.306 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.306 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747762 s, 140 MB/s 2024-04-25T09:01:01.316 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.316 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.316 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742576 s, 141 MB/s 2024-04-25T09:01:01.327 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.327 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.327 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00849012 s, 124 MB/s 2024-04-25T09:01:01.337 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.337 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.337 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736608 s, 142 MB/s 2024-04-25T09:01:01.347 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.347 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.347 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735079 s, 143 MB/s 2024-04-25T09:01:01.356 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.356 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.356 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709841 s, 148 MB/s 2024-04-25T09:01:01.366 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.366 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.370 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745061 s, 141 MB/s 2024-04-25T09:01:01.377 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.378 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.378 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00855344 s, 123 MB/s 2024-04-25T09:01:01.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:01.379 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:01:01.379 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:01:01.379 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T09:01:01.379 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:01:01.379 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:01:01.379 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:01:01.379 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:01:01.447 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.447 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.447 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686573 s, 153 MB/s 2024-04-25T09:01:01.457 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.457 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.457 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747137 s, 140 MB/s 2024-04-25T09:01:01.468 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.468 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.468 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00872352 s, 120 MB/s 2024-04-25T09:01:01.478 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.478 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.478 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742425 s, 141 MB/s 2024-04-25T09:01:01.490 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.490 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.490 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00877027 s, 120 MB/s 2024-04-25T09:01:01.499 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.500 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.500 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738636 s, 142 MB/s 2024-04-25T09:01:01.509 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.509 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.509 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720237 s, 146 MB/s 2024-04-25T09:01:01.518 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.519 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.519 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686122 s, 153 MB/s 2024-04-25T09:01:01.528 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.529 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.529 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727591 s, 144 MB/s 2024-04-25T09:01:01.539 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.539 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.539 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741736 s, 141 MB/s 2024-04-25T09:01:01.549 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.549 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.549 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766643 s, 137 MB/s 2024-04-25T09:01:01.558 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.558 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.558 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719594 s, 146 MB/s 2024-04-25T09:01:01.569 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.569 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.570 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00837966 s, 125 MB/s 2024-04-25T09:01:01.581 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.581 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.581 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841051 s, 125 MB/s 2024-04-25T09:01:01.591 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.591 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.591 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00812802 s, 129 MB/s 2024-04-25T09:01:01.601 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.602 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.602 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820218 s, 128 MB/s 2024-04-25T09:01:01.611 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.611 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.611 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732504 s, 143 MB/s 2024-04-25T09:01:01.621 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.621 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.621 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751234 s, 140 MB/s 2024-04-25T09:01:01.630 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.630 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.630 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068133 s, 154 MB/s 2024-04-25T09:01:01.640 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.640 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.640 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840671 s, 125 MB/s 2024-04-25T09:01:01.651 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.651 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.652 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00880506 s, 119 MB/s 2024-04-25T09:01:01.662 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.663 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.663 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00866271 s, 121 MB/s 2024-04-25T09:01:01.673 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.673 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.673 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861099 s, 122 MB/s 2024-04-25T09:01:01.683 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.683 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.683 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756841 s, 139 MB/s 2024-04-25T09:01:01.694 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.694 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.694 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861389 s, 122 MB/s 2024-04-25T09:01:01.705 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.705 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.705 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00864959 s, 121 MB/s 2024-04-25T09:01:01.714 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.714 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.715 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737758 s, 142 MB/s 2024-04-25T09:01:01.724 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.724 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.724 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075492 s, 139 MB/s 2024-04-25T09:01:01.734 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.734 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.734 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00781125 s, 134 MB/s 2024-04-25T09:01:01.745 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.745 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.746 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00900204 s, 116 MB/s 2024-04-25T09:01:01.756 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.756 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.756 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850516 s, 123 MB/s 2024-04-25T09:01:01.767 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.767 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.767 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828675 s, 127 MB/s 2024-04-25T09:01:01.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:01.768 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:01:01.768 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:01:01.768 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T09:01:01.769 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:01:01.769 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:01:01.769 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:01:01.769 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:01:01.837 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.837 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.837 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842158 s, 125 MB/s 2024-04-25T09:01:01.847 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.847 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.847 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072119 s, 145 MB/s 2024-04-25T09:01:01.857 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.857 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.857 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826123 s, 127 MB/s 2024-04-25T09:01:01.868 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.868 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.868 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851948 s, 123 MB/s 2024-04-25T09:01:01.879 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.879 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.879 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00860665 s, 122 MB/s 2024-04-25T09:01:01.888 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.889 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.889 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713834 s, 147 MB/s 2024-04-25T09:01:01.897 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.898 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.898 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691363 s, 152 MB/s 2024-04-25T09:01:01.907 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.907 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.907 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725882 s, 144 MB/s 2024-04-25T09:01:01.916 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.917 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.917 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719751 s, 146 MB/s 2024-04-25T09:01:01.926 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.926 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.927 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761839 s, 138 MB/s 2024-04-25T09:01:01.936 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.936 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.936 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727793 s, 144 MB/s 2024-04-25T09:01:01.951 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.952 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.952 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01347 s, 77.8 MB/s 2024-04-25T09:01:01.982 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.983 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.983 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00844051 s, 124 MB/s 2024-04-25T09:01:01.991 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:01.991 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:01.992 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825594 s, 127 MB/s 2024-04-25T09:01:02.001 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.002 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.002 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773034 s, 136 MB/s 2024-04-25T09:01:02.012 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.012 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.012 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00809249 s, 130 MB/s 2024-04-25T09:01:02.022 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.022 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.022 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816417 s, 128 MB/s 2024-04-25T09:01:02.033 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.033 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.033 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00837309 s, 125 MB/s 2024-04-25T09:01:02.042 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.042 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.043 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742064 s, 141 MB/s 2024-04-25T09:01:02.053 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.053 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.053 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818856 s, 128 MB/s 2024-04-25T09:01:02.063 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.064 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.064 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833776 s, 126 MB/s 2024-04-25T09:01:02.074 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.074 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.074 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00852273 s, 123 MB/s 2024-04-25T09:01:02.084 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.084 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.084 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716137 s, 146 MB/s 2024-04-25T09:01:02.094 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.094 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.094 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832089 s, 126 MB/s 2024-04-25T09:01:02.103 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.103 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.103 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686801 s, 153 MB/s 2024-04-25T09:01:02.112 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.113 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.113 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732339 s, 143 MB/s 2024-04-25T09:01:02.123 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.123 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.123 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838439 s, 125 MB/s 2024-04-25T09:01:02.133 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.133 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.133 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723505 s, 145 MB/s 2024-04-25T09:01:02.143 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.143 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.143 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820652 s, 128 MB/s 2024-04-25T09:01:02.153 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.153 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.153 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735772 s, 143 MB/s 2024-04-25T09:01:02.162 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.162 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.162 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723728 s, 145 MB/s 2024-04-25T09:01:02.173 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.173 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.173 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00859736 s, 122 MB/s 2024-04-25T09:01:02.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:02.174 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:01:02.174 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:01:02.174 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T09:01:02.175 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:01:02.175 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:01:02.175 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:01:02.175 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:01:02.242 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.243 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.243 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760145 s, 138 MB/s 2024-04-25T09:01:02.252 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.252 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.252 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729251 s, 144 MB/s 2024-04-25T09:01:02.263 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.263 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.263 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829097 s, 126 MB/s 2024-04-25T09:01:02.273 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.273 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.273 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830468 s, 126 MB/s 2024-04-25T09:01:02.282 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.282 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.282 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676363 s, 155 MB/s 2024-04-25T09:01:02.291 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.291 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.292 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743526 s, 141 MB/s 2024-04-25T09:01:02.300 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.300 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.300 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006569 s, 160 MB/s 2024-04-25T09:01:02.310 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.310 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.310 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719229 s, 146 MB/s 2024-04-25T09:01:02.320 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.320 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.320 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811002 s, 129 MB/s 2024-04-25T09:01:02.330 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.330 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.330 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750078 s, 140 MB/s 2024-04-25T09:01:02.339 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.340 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.340 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756108 s, 139 MB/s 2024-04-25T09:01:02.350 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.350 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.350 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00858526 s, 122 MB/s 2024-04-25T09:01:02.360 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.360 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.360 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741439 s, 141 MB/s 2024-04-25T09:01:02.370 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.370 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.370 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0080316 s, 131 MB/s 2024-04-25T09:01:02.381 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.381 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.381 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829927 s, 126 MB/s 2024-04-25T09:01:02.390 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.390 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.390 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741226 s, 141 MB/s 2024-04-25T09:01:02.401 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.401 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.401 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851397 s, 123 MB/s 2024-04-25T09:01:02.410 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.410 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.410 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710899 s, 148 MB/s 2024-04-25T09:01:02.420 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.420 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.420 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071837 s, 146 MB/s 2024-04-25T09:01:02.430 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.430 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.430 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836731 s, 125 MB/s 2024-04-25T09:01:02.439 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.440 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.440 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711619 s, 147 MB/s 2024-04-25T09:01:02.449 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.450 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.451 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727538 s, 144 MB/s 2024-04-25T09:01:02.459 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.459 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.459 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739585 s, 142 MB/s 2024-04-25T09:01:02.468 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.468 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.468 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712678 s, 147 MB/s 2024-04-25T09:01:02.478 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.478 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.479 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00849623 s, 123 MB/s 2024-04-25T09:01:02.489 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.489 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.489 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00847853 s, 124 MB/s 2024-04-25T09:01:02.500 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.500 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.500 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00852482 s, 123 MB/s 2024-04-25T09:01:02.511 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.511 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.511 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823708 s, 127 MB/s 2024-04-25T09:01:02.522 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.522 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.522 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00885735 s, 118 MB/s 2024-04-25T09:01:02.532 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.532 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.533 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821572 s, 128 MB/s 2024-04-25T09:01:02.542 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.542 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.542 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762155 s, 138 MB/s 2024-04-25T09:01:02.553 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.553 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.553 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00844957 s, 124 MB/s 2024-04-25T09:01:02.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:02.554 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:01:02.554 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:01:02.554 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T09:01:02.554 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:01:02.554 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:01:02.554 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:01:02.554 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:01:02.622 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.622 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.622 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073397 s, 143 MB/s 2024-04-25T09:01:02.631 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.631 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.631 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706768 s, 148 MB/s 2024-04-25T09:01:02.640 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.641 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.641 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748185 s, 140 MB/s 2024-04-25T09:01:02.650 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.650 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.650 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711351 s, 147 MB/s 2024-04-25T09:01:02.660 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.660 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.660 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829646 s, 126 MB/s 2024-04-25T09:01:02.671 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.671 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.671 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836043 s, 125 MB/s 2024-04-25T09:01:02.680 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.680 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.680 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713784 s, 147 MB/s 2024-04-25T09:01:02.690 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.690 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.690 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724024 s, 145 MB/s 2024-04-25T09:01:02.699 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.699 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.699 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706493 s, 148 MB/s 2024-04-25T09:01:02.709 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.709 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.709 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0077387 s, 135 MB/s 2024-04-25T09:01:02.718 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.718 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.718 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680862 s, 154 MB/s 2024-04-25T09:01:02.729 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.730 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.730 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084416 s, 124 MB/s 2024-04-25T09:01:02.739 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.739 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.739 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727492 s, 144 MB/s 2024-04-25T09:01:02.748 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.748 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.748 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700114 s, 150 MB/s 2024-04-25T09:01:02.757 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.758 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.758 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757762 s, 138 MB/s 2024-04-25T09:01:02.767 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.767 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.767 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747605 s, 140 MB/s 2024-04-25T09:01:02.778 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.778 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.778 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845781 s, 124 MB/s 2024-04-25T09:01:02.789 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.789 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.789 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824855 s, 127 MB/s 2024-04-25T09:01:02.798 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.799 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.799 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733209 s, 143 MB/s 2024-04-25T09:01:02.808 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.809 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.809 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00813326 s, 129 MB/s 2024-04-25T09:01:02.818 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.818 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.818 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754195 s, 139 MB/s 2024-04-25T09:01:02.829 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.829 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.829 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00859274 s, 122 MB/s 2024-04-25T09:01:02.840 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.840 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.840 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772652 s, 136 MB/s 2024-04-25T09:01:02.854 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.854 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.854 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0098754 s, 106 MB/s 2024-04-25T09:01:02.862 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.863 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.863 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780623 s, 134 MB/s 2024-04-25T09:01:02.874 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.875 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.875 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00922741 s, 114 MB/s 2024-04-25T09:01:02.887 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.887 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.887 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102489 s, 102 MB/s 2024-04-25T09:01:02.898 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.898 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.898 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00875315 s, 120 MB/s 2024-04-25T09:01:02.910 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.910 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.910 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00922594 s, 114 MB/s 2024-04-25T09:01:02.921 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.922 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.922 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00870345 s, 120 MB/s 2024-04-25T09:01:02.932 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.932 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.932 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00812954 s, 129 MB/s 2024-04-25T09:01:02.942 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:02.942 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:02.942 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752964 s, 139 MB/s 2024-04-25T09:01:02.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:02.943 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /scrub_control_test_path recursive 2024-04-25T09:01:03.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.103+0000 7f8da6710640 1 Processor -- start 2024-04-25T09:01:03.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.103+0000 7f8da6710640 1 -- start start 2024-04-25T09:01:03.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.104+0000 7f8da6710640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8da00b45a0 0x7f8da006d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.104+0000 7f8da6710640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f8da006d950 0x7f8da00cf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.104+0000 7f8da6710640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8da006dd30 con 0x7f8da0157b70 2024-04-25T09:01:03.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.104+0000 7f8da6710640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8da00cea20 con 0x7f8da00b45a0 2024-04-25T09:01:03.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.104+0000 7f8da6710640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8da00b4a50 con 0x7f8da006d950 2024-04-25T09:01:03.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.104+0000 7f8d9f7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8da00b45a0 0x7f8da006d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.104+0000 7f8d9f7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8da00b45a0 0x7f8da006d410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39218/0 (socket says 172.21.15.43:39218) 2024-04-25T09:01:03.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.104+0000 7f8d9f7fe640 1 -- 172.21.15.43:0/4171994527 learned_addr learned my addr 172.21.15.43:0/4171994527 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:03.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.104+0000 7f8da4cad640 1 --2- 172.21.15.43:0/4171994527 >> v2:172.21.15.193:3300/0 conn(0x7f8da006d950 0x7f8da00cf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.104+0000 7f8d9f7fe640 1 -- 172.21.15.43:0/4171994527 >> v2:172.21.15.193:3300/0 conn(0x7f8da006d950 msgr2=0x7f8da00cf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:03.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.104+0000 7f8d9f7fe640 1 --2- 172.21.15.43:0/4171994527 >> v2:172.21.15.193:3300/0 conn(0x7f8da006d950 0x7f8da00cf660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.104+0000 7f8d9f7fe640 1 -- 172.21.15.43:0/4171994527 >> v1:172.21.15.193:6789/0 conn(0x7f8da0157b70 legacy=0x7f8da00d0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:03.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.104+0000 7f8d9f7fe640 1 -- 172.21.15.43:0/4171994527 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da00cec50 con 0x7f8da00b45a0 2024-04-25T09:01:03.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.105+0000 7f8d9f7fe640 1 --2- 172.21.15.43:0/4171994527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8da00b45a0 0x7f8da006d410 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f8da00af130 tx=0x7f8d9002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef63f200d7be3efb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:03.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.105+0000 7f8d9effd640 1 -- 172.21.15.43:0/4171994527 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d90037070 con 0x7f8da00b45a0 2024-04-25T09:01:03.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.105+0000 7f8d9effd640 1 -- 172.21.15.43:0/4171994527 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d90032440 con 0x7f8da00b45a0 2024-04-25T09:01:03.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.105+0000 7f8d9effd640 1 -- 172.21.15.43:0/4171994527 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d90032810 con 0x7f8da00b45a0 2024-04-25T09:01:03.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.106+0000 7f8da6710640 1 -- 172.21.15.43:0/4171994527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8da00b45a0 msgr2=0x7f8da006d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:03.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.106+0000 7f8da6710640 1 --2- 172.21.15.43:0/4171994527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8da00b45a0 0x7f8da006d410 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f8da00af130 tx=0x7f8d9002a270 comp rx=0 tx=0).stop 2024-04-25T09:01:03.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.106+0000 7f8da6710640 1 -- 172.21.15.43:0/4171994527 shutdown_connections 2024-04-25T09:01:03.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.106+0000 7f8da6710640 1 --2- 172.21.15.43:0/4171994527 >> v2:172.21.15.193:3300/0 conn(0x7f8da006d950 0x7f8da00cf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.106+0000 7f8da6710640 1 --2- 172.21.15.43:0/4171994527 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8da00b45a0 0x7f8da006d410 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.106+0000 7f8da6710640 1 -- 172.21.15.43:0/4171994527 >> 172.21.15.43:0/4171994527 conn(0x7f8da006ab30 msgr2=0x7f8da006af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:03.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.106+0000 7f8da6710640 1 -- 172.21.15.43:0/4171994527 shutdown_connections 2024-04-25T09:01:03.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.106+0000 7f8da6710640 1 -- 172.21.15.43:0/4171994527 wait complete. 2024-04-25T09:01:03.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.107+0000 7f8da6710640 1 Processor -- start 2024-04-25T09:01:03.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.107+0000 7f8da6710640 1 -- start start 2024-04-25T09:01:03.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.108+0000 7f8da6710640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8da006d950 0x7f8da015af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.108+0000 7f8da6710640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8da00b45a0 0x7f8da015d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.108+0000 7f8da6710640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8da0157b70 0x7f8da00afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.108+0000 7f8da6710640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da00b4080 con 0x7f8da006d950 2024-04-25T09:01:03.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.108+0000 7f8da6710640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da00b3da0 con 0x7f8da0157b70 2024-04-25T09:01:03.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.108+0000 7f8da6710640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8da00b3a90 con 0x7f8da00b45a0 2024-04-25T09:01:03.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.108+0000 7f8d9ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8da006d950 0x7f8da015af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.108+0000 7f8d9ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8da006d950 0x7f8da015af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39222/0 (socket says 172.21.15.43:39222) 2024-04-25T09:01:03.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.108+0000 7f8d9ffff640 1 -- 172.21.15.43:0/1111130980 learned_addr learned my addr 172.21.15.43:0/1111130980 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:03.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.108+0000 7f8da4cad640 1 --2- 172.21.15.43:0/1111130980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8da0157b70 0x7f8da00afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.108+0000 7f8d9f7fe640 1 --2- 172.21.15.43:0/1111130980 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8da00b45a0 0x7f8da015d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.108+0000 7f8d9ffff640 1 -- 172.21.15.43:0/1111130980 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8da00b45a0 msgr2=0x7f8da015d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:03.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.108+0000 7f8d9ffff640 1 --2- 172.21.15.43:0/1111130980 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8da00b45a0 0x7f8da015d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.108+0000 7f8d9ffff640 1 -- 172.21.15.43:0/1111130980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8da0157b70 msgr2=0x7f8da00afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:03.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.108+0000 7f8d9ffff640 1 --2- 172.21.15.43:0/1111130980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8da0157b70 0x7f8da00afbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.108+0000 7f8d9ffff640 1 -- 172.21.15.43:0/1111130980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da015b460 con 0x7f8da006d950 2024-04-25T09:01:03.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.108+0000 7f8d9f7fe640 1 --2- 172.21.15.43:0/1111130980 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8da00b45a0 0x7f8da015d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:01:03.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.109+0000 7f8d9ffff640 1 --2- 172.21.15.43:0/1111130980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8da006d950 0x7f8da015af20 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f8d8800c460 tx=0x7f8d8800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:03.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.109+0000 7f8d9cff9640 1 -- 172.21.15.43:0/1111130980 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d88016020 con 0x7f8da006d950 2024-04-25T09:01:03.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.109+0000 7f8d9cff9640 1 -- 172.21.15.43:0/1111130980 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d88012870 con 0x7f8da006d950 2024-04-25T09:01:03.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.109+0000 7f8d9cff9640 1 -- 172.21.15.43:0/1111130980 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d88012c60 con 0x7f8da006d950 2024-04-25T09:01:03.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.109+0000 7f8da6710640 1 -- 172.21.15.43:0/1111130980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8da015d5d0 con 0x7f8da006d950 2024-04-25T09:01:03.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.109+0000 7f8da6710640 1 -- 172.21.15.43:0/1111130980 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8da00ae840 con 0x7f8da006d950 2024-04-25T09:01:03.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.110+0000 7f8d9cff9640 1 -- 172.21.15.43:0/1111130980 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8d88019020 con 0x7f8da006d950 2024-04-25T09:01:03.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.111+0000 7f8d9cff9640 1 --2- 172.21.15.43:0/1111130980 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8d8006b500 0x7f8d8006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.111+0000 7f8d9cff9640 1 -- 172.21.15.43:0/1111130980 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f8d88089b70 con 0x7f8da006d950 2024-04-25T09:01:03.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.111+0000 7f8d9f7fe640 1 --2- 172.21.15.43:0/1111130980 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8d8006b500 0x7f8d8006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.112+0000 7f8d9f7fe640 1 --2- 172.21.15.43:0/1111130980 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8d8006b500 0x7f8d8006d9c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8d90034000 tx=0x7f8d9002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:03.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.112+0000 7f8da940e740 1 Processor -- start 2024-04-25T09:01:03.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.112+0000 7f8da940e740 1 -- start start 2024-04-25T09:01:03.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.113+0000 7f8da940e740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563b862d8200 0x563b862da360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.113+0000 7f8da940e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b863148d0 0x563b862d85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.113+0000 7f8da940e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b86302cc0 0x563b86303140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.113+0000 7f8da940e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b8623c000 con 0x563b86302cc0 2024-04-25T09:01:03.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.113+0000 7f8da940e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b861e4f20 con 0x563b863148d0 2024-04-25T09:01:03.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.113+0000 7f8da940e740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b862380a0 con 0x563b862d8200 2024-04-25T09:01:03.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.113+0000 7f8da4cad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b86302cc0 0x563b86303140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.113+0000 7f8da4cad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b86302cc0 0x563b86303140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39230/0 (socket says 172.21.15.43:39230) 2024-04-25T09:01:03.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.113+0000 7f8da4cad640 1 -- 172.21.15.43:0/245934217 learned_addr learned my addr 172.21.15.43:0/245934217 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:03.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.113+0000 7f8d9f7fe640 1 --2- 172.21.15.43:0/245934217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b863148d0 0x563b862d85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.113+0000 7f8d9ffff640 1 --2- 172.21.15.43:0/245934217 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563b862d8200 0x563b862da360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.113+0000 7f8da4cad640 1 -- 172.21.15.43:0/245934217 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563b862d8200 msgr2=0x563b862da360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:03.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.113+0000 7f8da4cad640 1 --2- 172.21.15.43:0/245934217 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563b862d8200 0x563b862da360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.113+0000 7f8da4cad640 1 -- 172.21.15.43:0/245934217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b863148d0 msgr2=0x563b862d85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:03.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.113+0000 7f8da4cad640 1 --2- 172.21.15.43:0/245934217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b863148d0 0x563b862d85e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.113+0000 7f8da4cad640 1 -- 172.21.15.43:0/245934217 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b8601cba0 con 0x563b86302cc0 2024-04-25T09:01:03.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.113+0000 7f8d9f7fe640 1 --2- 172.21.15.43:0/245934217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b863148d0 0x563b862d85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:01:03.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.113+0000 7f8da4cad640 1 --2- 172.21.15.43:0/245934217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b86302cc0 0x563b86303140 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f8d98009fa0 tx=0x7f8d98003e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c4c833cf547ddf7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:03.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.113+0000 7f8d95ffb640 1 -- 172.21.15.43:0/245934217 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d98004860 con 0x563b86302cc0 2024-04-25T09:01:03.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.113+0000 7f8d95ffb640 1 -- 172.21.15.43:0/245934217 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d9801a070 con 0x563b86302cc0 2024-04-25T09:01:03.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.113+0000 7f8d95ffb640 1 -- 172.21.15.43:0/245934217 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d98015620 con 0x563b86302cc0 2024-04-25T09:01:03.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.114+0000 7f8da940e740 1 -- 172.21.15.43:0/245934217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b86302cc0 msgr2=0x563b86303140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:03.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.114+0000 7f8da940e740 1 --2- 172.21.15.43:0/245934217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b86302cc0 0x563b86303140 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f8d98009fa0 tx=0x7f8d98003e20 comp rx=0 tx=0).stop 2024-04-25T09:01:03.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.114+0000 7f8da940e740 1 -- 172.21.15.43:0/245934217 shutdown_connections 2024-04-25T09:01:03.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.114+0000 7f8da940e740 1 --2- 172.21.15.43:0/245934217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b86302cc0 0x563b86303140 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.114+0000 7f8da940e740 1 --2- 172.21.15.43:0/245934217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b863148d0 0x563b862d85e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.114+0000 7f8da940e740 1 --2- 172.21.15.43:0/245934217 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563b862d8200 0x563b862da360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.114+0000 7f8da940e740 1 -- 172.21.15.43:0/245934217 >> 172.21.15.43:0/245934217 conn(0x563b862a0050 msgr2=0x563b8631dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:03.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.114+0000 7f8da940e740 1 -- 172.21.15.43:0/245934217 shutdown_connections 2024-04-25T09:01:03.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.114+0000 7f8da940e740 1 -- 172.21.15.43:0/245934217 wait complete. 2024-04-25T09:01:03.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.115+0000 7f8da940e740 1 -- start start 2024-04-25T09:01:03.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.115+0000 7f8da940e740 1 Processor -- start 2024-04-25T09:01:03.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.115+0000 7f8da940e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b862d8200 0x563b862f47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.115+0000 7f8da940e740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563b86302cc0 0x563b86304670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.115+0000 7f8da940e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b863148d0 0x563b86304bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.116+0000 7f8da940e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b861da610 con 0x563b863148d0 2024-04-25T09:01:03.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.116+0000 7f8da940e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b86300810 con 0x563b862d8200 2024-04-25T09:01:03.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.116+0000 7f8da940e740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b86305130 con 0x563b86302cc0 2024-04-25T09:01:03.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.116+0000 7f8da940e740 10 client.0 _ll_register_callbacks cb 0x563b861f2f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:01:03.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.116+0000 7f8da4cad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b863148d0 0x563b86304bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.116+0000 7f8da4cad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b863148d0 0x563b86304bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39232/0 (socket says 172.21.15.43:39232) 2024-04-25T09:01:03.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.116+0000 7f8da4cad640 1 -- 172.21.15.43:0/3263021725 learned_addr learned my addr 172.21.15.43:0/3263021725 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:03.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.116+0000 7f8d9ffff640 1 --2- 172.21.15.43:0/3263021725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b862d8200 0x563b862f47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.116+0000 7f8d9f7fe640 1 --2- 172.21.15.43:0/3263021725 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563b86302cc0 0x563b86304670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.116+0000 7f8da4cad640 1 -- 172.21.15.43:0/3263021725 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563b86302cc0 msgr2=0x563b86304670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:03.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.116+0000 7f8da4cad640 1 --2- 172.21.15.43:0/3263021725 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563b86302cc0 0x563b86304670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.116+0000 7f8da4cad640 1 -- 172.21.15.43:0/3263021725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b862d8200 msgr2=0x563b862f47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:03.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.116+0000 7f8da4cad640 1 --2- 172.21.15.43:0/3263021725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b862d8200 0x563b862f47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.116+0000 7f8da4cad640 1 -- 172.21.15.43:0/3263021725 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b85f10490 con 0x563b863148d0 2024-04-25T09:01:03.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.116+0000 7f8d9ffff640 1 --2- 172.21.15.43:0/3263021725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b862d8200 0x563b862f47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:01:03.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.116+0000 7f8da4cad640 1 --2- 172.21.15.43:0/3263021725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b863148d0 0x563b86304bb0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f8d98009fd0 tx=0x7f8d98004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0be25fcc46ae927 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:03.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.116+0000 7f8d7effd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:01:03.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.116+0000 7f8d7effd640 1 -- 172.21.15.43:0/3263021725 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d98015a90 con 0x563b863148d0 2024-04-25T09:01:03.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.116+0000 7f8d7effd640 1 -- 172.21.15.43:0/3263021725 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f8d98015c30 con 0x563b863148d0 2024-04-25T09:01:03.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.116+0000 7f8da940e740 1 -- 172.21.15.43:0/3263021725 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d9801a070 con 0x563b863148d0 2024-04-25T09:01:03.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.116+0000 7f8d7effd640 1 -- 172.21.15.43:0/3263021725 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d9802e050 con 0x563b863148d0 2024-04-25T09:01:03.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.117+0000 7f8da940e740 10 client.7767 fetch_fsmap learned FSMap version 54 2024-04-25T09:01:03.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.117+0000 7f8da940e740 1 -- 172.21.15.43:0/3263021725 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x563b8607d330 con 0x563b863148d0 2024-04-25T09:01:03.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.117+0000 7f8d7effd640 1 -- 172.21.15.43:0/3263021725 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d9801a070 con 0x563b863148d0 2024-04-25T09:01:03.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.117+0000 7f8d7effd640 1 -- 172.21.15.43:0/3263021725 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 54) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8d98031020 con 0x563b863148d0 2024-04-25T09:01:03.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.117+0000 7f8da940e740 10 client.7767 fetch_fsmap finished waiting for FSMap version 54 2024-04-25T09:01:03.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.117+0000 7f8da940e740 10 client.7767 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T09:01:03.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.117+0000 7f8da940e740 20 client.7767 populate_metadata read hostname 'smithi043' 2024-04-25T09:01:03.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.117+0000 7f8da940e740 1 --2- 172.21.15.43:0/3263021725 >> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] conn(0x563b86317b10 0x563b86331190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.117+0000 7f8da940e740 4 client.7767 mds_command: new command op to 7596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:01:03.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.117+0000 7f8da940e740 1 -- 172.21.15.43:0/3263021725 --> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563b86249540 con 0x563b86317b10 2024-04-25T09:01:03.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.117+0000 7f8d9ffff640 1 --2- 172.21.15.43:0/3263021725 >> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] conn(0x563b86317b10 0x563b86331190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.117+0000 7f8d7effd640 10 client.7767 ms_handle_connect on v2:172.21.15.43:6833/1341028696 2024-04-25T09:01:03.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.117+0000 7f8d9ffff640 1 --2- 172.21.15.43:0/3263021725 >> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] conn(0x563b86317b10 0x563b86331190 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66c512c08d919ece server_cookie=b201fafa78037fb1 in_seq=0 out_seq=0 2024-04-25T09:01:03.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.118+0000 7f8d7effd640 1 -- 172.21.15.43:0/3263021725 <== mds.0 v2:172.21.15.43:6833/1341028696 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563b86249540 con 0x563b86317b10 2024-04-25T09:01:03.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.118+0000 7f8d7effd640 10 client.7767 handle_command_reply: tid=0 2024-04-25T09:01:03.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.118+0000 7f8da940e740 1 client.7767 shutdown 2024-04-25T09:01:03.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.118+0000 7f8da940e740 1 -- 172.21.15.43:0/3263021725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b863148d0 msgr2=0x563b86304bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:03.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.118+0000 7f8da940e740 1 --2- 172.21.15.43:0/3263021725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b863148d0 0x563b86304bb0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f8d98009fd0 tx=0x7f8d98004ab0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.118+0000 7f8da940e740 1 -- 172.21.15.43:0/3263021725 shutdown_connections 2024-04-25T09:01:03.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.118+0000 7f8da940e740 1 --2- 172.21.15.43:0/3263021725 >> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] conn(0x563b86317b10 0x563b86331190 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.118+0000 7f8da940e740 1 --2- 172.21.15.43:0/3263021725 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b863148d0 0x563b86304bb0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.118+0000 7f8da940e740 1 --2- 172.21.15.43:0/3263021725 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563b86302cc0 0x563b86304670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.118+0000 7f8da940e740 1 --2- 172.21.15.43:0/3263021725 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b862d8200 0x563b862f47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.118+0000 7f8da940e740 1 -- 172.21.15.43:0/3263021725 >> 172.21.15.43:0/3263021725 conn(0x563b862a0050 msgr2=0x563b863009f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:03.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.118+0000 7f8d7effd640 0 client.7767 ms_handle_reset on v2:172.21.15.43:6833/1341028696 2024-04-25T09:01:03.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.118+0000 7f8da940e740 1 -- 172.21.15.43:0/3263021725 shutdown_connections 2024-04-25T09:01:03.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.118+0000 7f8da940e740 1 -- 172.21.15.43:0/3263021725 wait complete. 2024-04-25T09:01:03.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.118+0000 7f8da940e740 20 client.7767 trim_cache size 0 max 16384 2024-04-25T09:01:03.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.133+0000 7f8da940e740 1 Processor -- start 2024-04-25T09:01:03.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.133+0000 7f8da940e740 1 -- start start 2024-04-25T09:01:03.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.134+0000 7f8da940e740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563b862d8200 0x563b86340890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.134+0000 7f8da940e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b86302cc0 0x563b86313f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.134+0000 7f8da940e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b863148d0 0x563b86305920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.134+0000 7f8da940e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b86318970 con 0x563b86302cc0 2024-04-25T09:01:03.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.134+0000 7f8da940e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da01ffea0 con 0x563b863148d0 2024-04-25T09:01:03.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.134+0000 7f8da940e740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8da01ff030 con 0x563b862d8200 2024-04-25T09:01:03.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.134+0000 7f8d9f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b86302cc0 0x563b86313f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.134+0000 7f8d9f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b86302cc0 0x563b86313f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39246/0 (socket says 172.21.15.43:39246) 2024-04-25T09:01:03.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.134+0000 7f8d9f7fe640 1 -- 172.21.15.43:0/4108290831 learned_addr learned my addr 172.21.15.43:0/4108290831 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:03.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.134+0000 7f8da4cad640 1 --2- 172.21.15.43:0/4108290831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b863148d0 0x563b86305920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.134+0000 7f8d9ffff640 1 --2- 172.21.15.43:0/4108290831 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563b862d8200 0x563b86340890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.134+0000 7f8d9f7fe640 1 -- 172.21.15.43:0/4108290831 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563b862d8200 msgr2=0x563b86340890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:03.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.134+0000 7f8d9f7fe640 1 --2- 172.21.15.43:0/4108290831 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563b862d8200 0x563b86340890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.134+0000 7f8d9f7fe640 1 -- 172.21.15.43:0/4108290831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b863148d0 msgr2=0x563b86305920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:03.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.134+0000 7f8d9f7fe640 1 --2- 172.21.15.43:0/4108290831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b863148d0 0x563b86305920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.134+0000 7f8d9f7fe640 1 -- 172.21.15.43:0/4108290831 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b85ee9610 con 0x563b86302cc0 2024-04-25T09:01:03.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.134+0000 7f8da4cad640 1 --2- 172.21.15.43:0/4108290831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b863148d0 0x563b86305920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:01:03.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.135+0000 7f8d9f7fe640 1 --2- 172.21.15.43:0/4108290831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b86302cc0 0x563b86313f30 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f8d90009630 tx=0x7f8d9002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c04cddceed9647 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:03.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.135+0000 7f8d957fa640 1 -- 172.21.15.43:0/4108290831 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d90037040 con 0x563b86302cc0 2024-04-25T09:01:03.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.135+0000 7f8d957fa640 1 -- 172.21.15.43:0/4108290831 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d9004f070 con 0x563b86302cc0 2024-04-25T09:01:03.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.135+0000 7f8d957fa640 1 -- 172.21.15.43:0/4108290831 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9004a460 con 0x563b86302cc0 2024-04-25T09:01:03.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.135+0000 7f8da940e740 1 -- 172.21.15.43:0/4108290831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b86302cc0 msgr2=0x563b86313f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:03.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.135+0000 7f8da940e740 1 --2- 172.21.15.43:0/4108290831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b86302cc0 0x563b86313f30 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f8d90009630 tx=0x7f8d9002c000 comp rx=0 tx=0).stop 2024-04-25T09:01:03.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.135+0000 7f8da940e740 1 -- 172.21.15.43:0/4108290831 shutdown_connections 2024-04-25T09:01:03.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.135+0000 7f8da940e740 1 --2- 172.21.15.43:0/4108290831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b863148d0 0x563b86305920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.135+0000 7f8da940e740 1 --2- 172.21.15.43:0/4108290831 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b86302cc0 0x563b86313f30 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.135+0000 7f8da940e740 1 --2- 172.21.15.43:0/4108290831 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563b862d8200 0x563b86340890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.135+0000 7f8da940e740 1 -- 172.21.15.43:0/4108290831 >> 172.21.15.43:0/4108290831 conn(0x563b862a0050 msgr2=0x563b862d5fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:03.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.135+0000 7f8da940e740 1 -- 172.21.15.43:0/4108290831 shutdown_connections 2024-04-25T09:01:03.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.135+0000 7f8da940e740 1 -- 172.21.15.43:0/4108290831 wait complete. 2024-04-25T09:01:03.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.136+0000 7f8da940e740 1 -- start start 2024-04-25T09:01:03.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.136+0000 7f8da940e740 1 Processor -- start 2024-04-25T09:01:03.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.137+0000 7f8da940e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b862d8200 0x563b86371240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.137+0000 7f8da940e740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563b86302cc0 0x563b86305920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.137+0000 7f8da940e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b863148d0 0x563b86305e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.137+0000 7f8da940e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da0156c30 con 0x563b863148d0 2024-04-25T09:01:03.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.137+0000 7f8da940e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da0150f20 con 0x563b862d8200 2024-04-25T09:01:03.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.137+0000 7f8da940e740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x563b86340680 con 0x563b86302cc0 2024-04-25T09:01:03.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.137+0000 7f8da940e740 10 client.0 _ll_register_callbacks cb 0x7f8d98009bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:01:03.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.137+0000 7f8da4cad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b863148d0 0x563b86305e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.137+0000 7f8da4cad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b863148d0 0x563b86305e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39248/0 (socket says 172.21.15.43:39248) 2024-04-25T09:01:03.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.137+0000 7f8da4cad640 1 -- 172.21.15.43:0/3783522244 learned_addr learned my addr 172.21.15.43:0/3783522244 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:03.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.137+0000 7f8d9ffff640 1 --2- 172.21.15.43:0/3783522244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b862d8200 0x563b86371240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.137+0000 7f8d9f7fe640 1 --2- 172.21.15.43:0/3783522244 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563b86302cc0 0x563b86305920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.137+0000 7f8da4cad640 1 -- 172.21.15.43:0/3783522244 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563b86302cc0 msgr2=0x563b86305920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:03.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.137+0000 7f8da4cad640 1 --2- 172.21.15.43:0/3783522244 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563b86302cc0 0x563b86305920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.137+0000 7f8da4cad640 1 -- 172.21.15.43:0/3783522244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b862d8200 msgr2=0x563b86371240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:03.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.137+0000 7f8da4cad640 1 --2- 172.21.15.43:0/3783522244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b862d8200 0x563b86371240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.137+0000 7f8da4cad640 1 -- 172.21.15.43:0/3783522244 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b862f3ec0 con 0x563b863148d0 2024-04-25T09:01:03.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.137+0000 7f8da4cad640 1 --2- 172.21.15.43:0/3783522244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b863148d0 0x563b86305e60 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f8d98009db0 tx=0x7f8d98014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d34f0e0189083a2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:03.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.137+0000 7f8d7effd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:01:03.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.138+0000 7f8d7effd640 1 -- 172.21.15.43:0/3783522244 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9801e960 con 0x563b863148d0 2024-04-25T09:01:03.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.138+0000 7f8d7effd640 1 -- 172.21.15.43:0/3783522244 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f8d9801eb00 con 0x563b863148d0 2024-04-25T09:01:03.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.138+0000 7f8da940e740 1 -- 172.21.15.43:0/3783522244 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d9004f070 con 0x563b863148d0 2024-04-25T09:01:03.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.138+0000 7f8d7effd640 1 -- 172.21.15.43:0/3783522244 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d98043050 con 0x563b863148d0 2024-04-25T09:01:03.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.138+0000 7f8da940e740 10 client.7773 fetch_fsmap learned FSMap version 54 2024-04-25T09:01:03.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.138+0000 7f8da940e740 1 -- 172.21.15.43:0/3783522244 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x563b863889c0 con 0x563b863148d0 2024-04-25T09:01:03.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.138+0000 7f8d7effd640 1 -- 172.21.15.43:0/3783522244 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d9004f070 con 0x563b863148d0 2024-04-25T09:01:03.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.138+0000 7f8d7effd640 1 -- 172.21.15.43:0/3783522244 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 54) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8d98046020 con 0x563b863148d0 2024-04-25T09:01:03.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.138+0000 7f8da940e740 10 client.7773 fetch_fsmap finished waiting for FSMap version 54 2024-04-25T09:01:03.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.138+0000 7f8da940e740 10 client.7773 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T09:01:03.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.138+0000 7f8da940e740 20 client.7773 populate_metadata read hostname 'smithi043' 2024-04-25T09:01:03.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.138+0000 7f8da940e740 1 --2- 172.21.15.43:0/3783522244 >> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] conn(0x563b86370d50 0x563b863212d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.138+0000 7f8da940e740 4 client.7773 mds_command: new command op to 7596 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T09:01:03.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.138+0000 7f8da940e740 1 -- 172.21.15.43:0/3783522244 --> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x563b86319180 con 0x563b86370d50 2024-04-25T09:01:03.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.138+0000 7f8d9ffff640 1 --2- 172.21.15.43:0/3783522244 >> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] conn(0x563b86370d50 0x563b863212d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.138+0000 7f8d7effd640 10 client.7773 ms_handle_connect on v2:172.21.15.43:6833/1341028696 2024-04-25T09:01:03.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.138+0000 7f8d9ffff640 1 --2- 172.21.15.43:0/3783522244 >> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] conn(0x563b86370d50 0x563b863212d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f528159460a9227 server_cookie=5abf2b2d0e5be5b0 in_seq=0 out_seq=0 2024-04-25T09:01:03.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.141+0000 7f8d7effd640 1 -- 172.21.15.43:0/3783522244 <== mds.0 v2:172.21.15.43:6833/1341028696 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x563b86319180 con 0x563b86370d50 2024-04-25T09:01:03.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.141+0000 7f8d7effd640 10 client.7773 handle_command_reply: tid=0 2024-04-25T09:01:03.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.141+0000 7f8da940e740 1 client.7773 shutdown 2024-04-25T09:01:03.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.141+0000 7f8da940e740 1 -- 172.21.15.43:0/3783522244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b863148d0 msgr2=0x563b86305e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:03.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.141+0000 7f8da940e740 1 --2- 172.21.15.43:0/3783522244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b863148d0 0x563b86305e60 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f8d98009db0 tx=0x7f8d98014000 comp rx=0 tx=0).stop 2024-04-25T09:01:03.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.141+0000 7f8da940e740 1 -- 172.21.15.43:0/3783522244 shutdown_connections 2024-04-25T09:01:03.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.141+0000 7f8da940e740 1 --2- 172.21.15.43:0/3783522244 >> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] conn(0x563b86370d50 0x563b863212d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.141+0000 7f8d7effd640 0 client.7773 ms_handle_reset on v2:172.21.15.43:6833/1341028696 2024-04-25T09:01:03.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.141+0000 7f8da940e740 1 --2- 172.21.15.43:0/3783522244 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563b863148d0 0x563b86305e60 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.141+0000 7f8da940e740 1 --2- 172.21.15.43:0/3783522244 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563b86302cc0 0x563b86305920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.141+0000 7f8da940e740 1 --2- 172.21.15.43:0/3783522244 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563b862d8200 0x563b86371240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.141+0000 7f8da940e740 1 -- 172.21.15.43:0/3783522244 >> 172.21.15.43:0/3783522244 conn(0x563b862a0050 msgr2=0x563b86313f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:03.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.141+0000 7f8da940e740 1 -- 172.21.15.43:0/3783522244 shutdown_connections 2024-04-25T09:01:03.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.141+0000 7f8da940e740 1 -- 172.21.15.43:0/3783522244 wait complete. 2024-04-25T09:01:03.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.142+0000 7f8da940e740 20 client.7773 trim_cache size 0 max 16384 2024-04-25T09:01:03.134 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:01:03.134 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-25T09:01:03.135 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "d1983208-ea9f-4dd7-aa29-f33a93518599", 2024-04-25T09:01:03.135 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-25T09:01:03.135 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:01:03.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.142+0000 7f8da6710640 1 -- 172.21.15.43:0/1111130980 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8d8006b500 msgr2=0x7f8d8006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:03.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.142+0000 7f8da6710640 1 --2- 172.21.15.43:0/1111130980 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8d8006b500 0x7f8d8006d9c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8d90034000 tx=0x7f8d9002e000 comp rx=0 tx=0).stop 2024-04-25T09:01:03.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.142+0000 7f8da6710640 1 -- 172.21.15.43:0/1111130980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8da006d950 msgr2=0x7f8da015af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:03.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.142+0000 7f8da6710640 1 --2- 172.21.15.43:0/1111130980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8da006d950 0x7f8da015af20 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f8d8800c460 tx=0x7f8d8800c920 comp rx=0 tx=0).stop 2024-04-25T09:01:03.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.142+0000 7f8da6710640 1 -- 172.21.15.43:0/1111130980 shutdown_connections 2024-04-25T09:01:03.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.142+0000 7f8da6710640 1 --2- 172.21.15.43:0/1111130980 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8d8006b500 0x7f8d8006d9c0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.142+0000 7f8da6710640 1 --2- 172.21.15.43:0/1111130980 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8da0157b70 0x7f8da00afbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.142+0000 7f8da6710640 1 --2- 172.21.15.43:0/1111130980 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8da00b45a0 0x7f8da015d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.142+0000 7f8da6710640 1 --2- 172.21.15.43:0/1111130980 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8da006d950 0x7f8da015af20 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.142+0000 7f8da6710640 1 -- 172.21.15.43:0/1111130980 >> 172.21.15.43:0/1111130980 conn(0x7f8da006ab30 msgr2=0x7f8da006c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:03.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.142+0000 7f8da6710640 1 -- 172.21.15.43:0/1111130980 shutdown_connections 2024-04-25T09:01:03.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.142+0000 7f8da6710640 1 -- 172.21.15.43:0/1111130980 wait complete. 2024-04-25T09:01:03.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:03.143 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub abort 2024-04-25T09:01:03.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.290+0000 7fc003b3b640 1 Processor -- start 2024-04-25T09:01:03.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.290+0000 7fc003b3b640 1 -- start start 2024-04-25T09:01:03.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.291+0000 7fc003b3b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbffc15a960 0x7fbffc0b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.291+0000 7fc003b3b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbffc15b110 con 0x7fbffc0aee40 2024-04-25T09:01:03.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.291+0000 7fc003b3b640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbffc15b3f0 con 0x7fbffc0b3ed0 2024-04-25T09:01:03.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.291+0000 7fc003b3b640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc15b6d0 con 0x7fbffc15a960 2024-04-25T09:01:03.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.291+0000 7fc0020d8640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fbffc0aee40 0x7fbffc070330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59290/0 (socket says 172.21.15.43:59290) 2024-04-25T09:01:03.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.291+0000 7fc0020d8640 1 -- 172.21.15.43:0/3928821158 learned_addr learned my addr 172.21.15.43:0/3928821158 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:03.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.291+0000 7fc0018d7640 1 --2- 172.21.15.43:0/3928821158 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbffc15a960 0x7fbffc0b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.291+0000 7fc0008d5640 1 -- 172.21.15.43:0/3928821158 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 418945191 0 0) 0x7fbffc15b110 con 0x7fbffc0aee40 2024-04-25T09:01:03.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.291+0000 7fc0008d5640 1 -- 172.21.15.43:0/3928821158 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbfe8003520 con 0x7fbffc0aee40 2024-04-25T09:01:03.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.292+0000 7fc0008d5640 1 -- 172.21.15.43:0/3928821158 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3527225329 0 0) 0x7fbffc15b3f0 con 0x7fbffc0b3ed0 2024-04-25T09:01:03.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.292+0000 7fc0008d5640 1 -- 172.21.15.43:0/3928821158 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbffc15b110 con 0x7fbffc0b3ed0 2024-04-25T09:01:03.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.292+0000 7fc0018d7640 1 -- 172.21.15.43:0/3928821158 >> v1:172.21.15.193:6789/0 conn(0x7fbffc0b3ed0 legacy=0x7fbffc0ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:03.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.292+0000 7fc0018d7640 1 -- 172.21.15.43:0/3928821158 >> v1:172.21.15.43:6789/0 conn(0x7fbffc0aee40 legacy=0x7fbffc070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:03.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.292+0000 7fc0018d7640 1 -- 172.21.15.43:0/3928821158 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbffc15b9b0 con 0x7fbffc15a960 2024-04-25T09:01:03.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.292+0000 7fc0018d7640 1 --2- 172.21.15.43:0/3928821158 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbffc15a960 0x7fbffc0b3990 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbff00097d0 tx=0x7fbff002a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aaec1dbfbb7c545e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:03.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.293+0000 7fc0008d5640 1 -- 172.21.15.43:0/3928821158 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff0037070 con 0x7fbffc15a960 2024-04-25T09:01:03.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.293+0000 7fc0008d5640 1 -- 172.21.15.43:0/3928821158 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff0032440 con 0x7fbffc15a960 2024-04-25T09:01:03.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.293+0000 7fc0008d5640 1 -- 172.21.15.43:0/3928821158 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff0032810 con 0x7fbffc15a960 2024-04-25T09:01:03.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.293+0000 7fc003b3b640 1 -- 172.21.15.43:0/3928821158 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbffc15a960 msgr2=0x7fbffc0b3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:03.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.293+0000 7fc003b3b640 1 --2- 172.21.15.43:0/3928821158 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbffc15a960 0x7fbffc0b3990 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fbff00097d0 tx=0x7fbff002a210 comp rx=0 tx=0).stop 2024-04-25T09:01:03.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.293+0000 7fc003b3b640 1 -- 172.21.15.43:0/3928821158 shutdown_connections 2024-04-25T09:01:03.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.293+0000 7fc003b3b640 1 --2- 172.21.15.43:0/3928821158 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbffc15a960 0x7fbffc0b3990 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.293+0000 7fc003b3b640 1 -- 172.21.15.43:0/3928821158 >> 172.21.15.43:0/3928821158 conn(0x7fbffc06ab30 msgr2=0x7fbffc06af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:03.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.293+0000 7fc003b3b640 1 -- 172.21.15.43:0/3928821158 shutdown_connections 2024-04-25T09:01:03.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.293+0000 7fc003b3b640 1 -- 172.21.15.43:0/3928821158 wait complete. 2024-04-25T09:01:03.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.294+0000 7fc003b3b640 1 Processor -- start 2024-04-25T09:01:03.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.294+0000 7fc003b3b640 1 -- start start 2024-04-25T09:01:03.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.295+0000 7fc003b3b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbffc0aee40 0x7fbffc159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.295+0000 7fc003b3b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbffc0b3ed0 0x7fbffc151880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.295+0000 7fc003b3b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbffc15a960 0x7fbffc151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.295+0000 7fc003b3b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc15ad40 con 0x7fbffc0b3ed0 2024-04-25T09:01:03.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.295+0000 7fc003b3b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc0cd940 con 0x7fbffc15a960 2024-04-25T09:01:03.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.295+0000 7fc003b3b640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc15aec0 con 0x7fbffc0aee40 2024-04-25T09:01:03.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.295+0000 7fc0010d6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbffc0b3ed0 0x7fbffc151880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.295+0000 7fc0010d6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbffc0b3ed0 0x7fbffc151880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39262/0 (socket says 172.21.15.43:39262) 2024-04-25T09:01:03.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.295+0000 7fc0010d6640 1 -- 172.21.15.43:0/866770818 learned_addr learned my addr 172.21.15.43:0/866770818 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:03.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.295+0000 7fc0020d8640 1 --2- 172.21.15.43:0/866770818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbffc15a960 0x7fbffc151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.295+0000 7fc0018d7640 1 --2- 172.21.15.43:0/866770818 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbffc0aee40 0x7fbffc159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.295+0000 7fc0010d6640 1 -- 172.21.15.43:0/866770818 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbffc0aee40 msgr2=0x7fbffc159000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:03.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.295+0000 7fc0010d6640 1 --2- 172.21.15.43:0/866770818 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbffc0aee40 0x7fbffc159000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.295+0000 7fc0010d6640 1 -- 172.21.15.43:0/866770818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbffc15a960 msgr2=0x7fbffc151130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:03.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.295+0000 7fc0010d6640 1 --2- 172.21.15.43:0/866770818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbffc15a960 0x7fbffc151130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.295+0000 7fc0010d6640 1 -- 172.21.15.43:0/866770818 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbffc0a9900 con 0x7fbffc0b3ed0 2024-04-25T09:01:03.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.296+0000 7fc0018d7640 1 --2- 172.21.15.43:0/866770818 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbffc0aee40 0x7fbffc159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:03.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.296+0000 7fc0020d8640 1 --2- 172.21.15.43:0/866770818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbffc15a960 0x7fbffc151130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:03.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.296+0000 7fc0010d6640 1 --2- 172.21.15.43:0/866770818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbffc0b3ed0 0x7fbffc151880 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fbff800c8a0 tx=0x7fbff800cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:03.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.296+0000 7fbfee7fc640 1 -- 172.21.15.43:0/866770818 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff8003b70 con 0x7fbffc0b3ed0 2024-04-25T09:01:03.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.296+0000 7fbfee7fc640 1 -- 172.21.15.43:0/866770818 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff8003d30 con 0x7fbffc0b3ed0 2024-04-25T09:01:03.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.296+0000 7fc003b3b640 1 -- 172.21.15.43:0/866770818 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbffc151670 con 0x7fbffc0b3ed0 2024-04-25T09:01:03.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.296+0000 7fbfee7fc640 1 -- 172.21.15.43:0/866770818 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff800aa90 con 0x7fbffc0b3ed0 2024-04-25T09:01:03.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.296+0000 7fc003b3b640 1 -- 172.21.15.43:0/866770818 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbffc152290 con 0x7fbffc0b3ed0 2024-04-25T09:01:03.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.298+0000 7fbfee7fc640 1 -- 172.21.15.43:0/866770818 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbff800ac30 con 0x7fbffc0b3ed0 2024-04-25T09:01:03.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.298+0000 7fbfee7fc640 1 --2- 172.21.15.43:0/866770818 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbfe006b500 0x7fbfe006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.298+0000 7fbfee7fc640 1 -- 172.21.15.43:0/866770818 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fbff808a160 con 0x7fbffc0b3ed0 2024-04-25T09:01:03.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.298+0000 7fc0018d7640 1 --2- 172.21.15.43:0/866770818 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbfe006b500 0x7fbfe006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.299+0000 7fc0018d7640 1 --2- 172.21.15.43:0/866770818 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbfe006b500 0x7fbfe006d9c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fbff0034000 tx=0x7fbff002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:03.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.299+0000 7fc006842740 1 Processor -- start 2024-04-25T09:01:03.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.299+0000 7fc006842740 1 -- start start 2024-04-25T09:01:03.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.300+0000 7fc006842740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d532f6ca0 0x564d532f8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.300+0000 7fc006842740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d53333370 0x564d532f7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.300+0000 7fc006842740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564d53321760 0x564d53321be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.300+0000 7fc006842740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d53259b90 con 0x564d532f6ca0 2024-04-25T09:01:03.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.300+0000 7fc006842740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d53202e40 con 0x564d53333370 2024-04-25T09:01:03.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.300+0000 7fc006842740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x564d53255c30 con 0x564d53321760 2024-04-25T09:01:03.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.300+0000 7fc0018d7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d532f6ca0 0x564d532f8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.300+0000 7fc0018d7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d532f6ca0 0x564d532f8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39278/0 (socket says 172.21.15.43:39278) 2024-04-25T09:01:03.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.300+0000 7fc0018d7640 1 -- 172.21.15.43:0/3041323406 learned_addr learned my addr 172.21.15.43:0/3041323406 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:03.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.300+0000 7fc0020d8640 1 --2- 172.21.15.43:0/3041323406 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564d53321760 0x564d53321be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.300+0000 7fc0010d6640 1 --2- 172.21.15.43:0/3041323406 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d53333370 0x564d532f7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.300+0000 7fc0018d7640 1 -- 172.21.15.43:0/3041323406 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564d53321760 msgr2=0x564d53321be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:03.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.300+0000 7fc0018d7640 1 --2- 172.21.15.43:0/3041323406 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564d53321760 0x564d53321be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.300+0000 7fc0018d7640 1 -- 172.21.15.43:0/3041323406 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d53333370 msgr2=0x564d532f7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:03.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.300+0000 7fc0018d7640 1 --2- 172.21.15.43:0/3041323406 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d53333370 0x564d532f7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.300+0000 7fc0018d7640 1 -- 172.21.15.43:0/3041323406 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d530387d0 con 0x564d532f6ca0 2024-04-25T09:01:03.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.300+0000 7fc0018d7640 1 --2- 172.21.15.43:0/3041323406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d532f6ca0 0x564d532f8e00 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fbff002a800 tx=0x7fbff002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53d18972f3e8f0d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:03.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.300+0000 7fbfdf7fe640 1 -- 172.21.15.43:0/3041323406 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff004d070 con 0x564d532f6ca0 2024-04-25T09:01:03.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.300+0000 7fbfdf7fe640 1 -- 172.21.15.43:0/3041323406 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff0032d80 con 0x564d532f6ca0 2024-04-25T09:01:03.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.300+0000 7fbfdf7fe640 1 -- 172.21.15.43:0/3041323406 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff0037040 con 0x564d532f6ca0 2024-04-25T09:01:03.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.301+0000 7fc006842740 1 -- 172.21.15.43:0/3041323406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d532f6ca0 msgr2=0x564d532f8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:03.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.301+0000 7fc006842740 1 --2- 172.21.15.43:0/3041323406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d532f6ca0 0x564d532f8e00 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fbff002a800 tx=0x7fbff002c000 comp rx=0 tx=0).stop 2024-04-25T09:01:03.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.301+0000 7fc006842740 1 -- 172.21.15.43:0/3041323406 shutdown_connections 2024-04-25T09:01:03.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.301+0000 7fc006842740 1 --2- 172.21.15.43:0/3041323406 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564d53321760 0x564d53321be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.301+0000 7fc006842740 1 --2- 172.21.15.43:0/3041323406 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d53333370 0x564d532f7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.301+0000 7fc006842740 1 --2- 172.21.15.43:0/3041323406 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d532f6ca0 0x564d532f8e00 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.301+0000 7fc006842740 1 -- 172.21.15.43:0/3041323406 >> 172.21.15.43:0/3041323406 conn(0x564d532be870 msgr2=0x564d5333c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:03.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.301+0000 7fc006842740 1 -- 172.21.15.43:0/3041323406 shutdown_connections 2024-04-25T09:01:03.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.301+0000 7fc006842740 1 -- 172.21.15.43:0/3041323406 wait complete. 2024-04-25T09:01:03.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.302+0000 7fc006842740 1 -- start start 2024-04-25T09:01:03.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.302+0000 7fc006842740 1 Processor -- start 2024-04-25T09:01:03.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.303+0000 7fc006842740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d532f6ca0 0x564d53310ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.303+0000 7fc006842740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d53321760 0x564d53321be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.303+0000 7fc006842740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564d53333370 0x564d5331dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.303+0000 7fc006842740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d532163e0 con 0x564d53321760 2024-04-25T09:01:03.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.303+0000 7fc006842740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d5331f2b0 con 0x564d532f6ca0 2024-04-25T09:01:03.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.303+0000 7fc006842740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x564d533253a0 con 0x564d53333370 2024-04-25T09:01:03.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.303+0000 7fc006842740 10 client.0 _ll_register_callbacks cb 0x564d53215db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:01:03.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.303+0000 7fc0010d6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d53321760 0x564d53321be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.303+0000 7fc0010d6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d53321760 0x564d53321be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39280/0 (socket says 172.21.15.43:39280) 2024-04-25T09:01:03.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.303+0000 7fc0010d6640 1 -- 172.21.15.43:0/3818799977 learned_addr learned my addr 172.21.15.43:0/3818799977 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:03.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.303+0000 7fc0018d7640 1 --2- 172.21.15.43:0/3818799977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d532f6ca0 0x564d53310ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.303+0000 7fc0020d8640 1 --2- 172.21.15.43:0/3818799977 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564d53333370 0x564d5331dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.303+0000 7fc0010d6640 1 -- 172.21.15.43:0/3818799977 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564d53333370 msgr2=0x564d5331dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:03.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.303+0000 7fc0010d6640 1 --2- 172.21.15.43:0/3818799977 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564d53333370 0x564d5331dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.303+0000 7fc0010d6640 1 -- 172.21.15.43:0/3818799977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d532f6ca0 msgr2=0x564d53310ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:03.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.303+0000 7fc0010d6640 1 --2- 172.21.15.43:0/3818799977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d532f6ca0 0x564d53310ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.303+0000 7fc0010d6640 1 -- 172.21.15.43:0/3818799977 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564d52f2e0f0 con 0x564d53321760 2024-04-25T09:01:03.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.303+0000 7fc0018d7640 1 --2- 172.21.15.43:0/3818799977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d532f6ca0 0x564d53310ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:01:03.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.303+0000 7fc0010d6640 1 --2- 172.21.15.43:0/3818799977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d53321760 0x564d53321be0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fbff800a430 tx=0x7fbff8058950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7c59af29d6b9ae1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:03.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.303+0000 7fbfdcff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:01:03.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.303+0000 7fbfdcff9640 1 -- 172.21.15.43:0/3818799977 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff8054fc0 con 0x564d53321760 2024-04-25T09:01:03.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.303+0000 7fbfdcff9640 1 -- 172.21.15.43:0/3818799977 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fbff8057da0 con 0x564d53321760 2024-04-25T09:01:03.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.303+0000 7fc006842740 1 -- 172.21.15.43:0/3818799977 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbff0032d80 con 0x564d53321760 2024-04-25T09:01:03.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.304+0000 7fbfdcff9640 1 -- 172.21.15.43:0/3818799977 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbff8055160 con 0x564d53321760 2024-04-25T09:01:03.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.304+0000 7fc006842740 10 client.7785 fetch_fsmap learned FSMap version 54 2024-04-25T09:01:03.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.304+0000 7fc006842740 1 -- 172.21.15.43:0/3818799977 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x564d53098f60 con 0x564d53321760 2024-04-25T09:01:03.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.304+0000 7fbfdcff9640 1 -- 172.21.15.43:0/3818799977 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff0032d80 con 0x564d53321760 2024-04-25T09:01:03.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.304+0000 7fbfdcff9640 1 -- 172.21.15.43:0/3818799977 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 54) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbff80772f0 con 0x564d53321760 2024-04-25T09:01:03.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.304+0000 7fc006842740 10 client.7785 fetch_fsmap finished waiting for FSMap version 54 2024-04-25T09:01:03.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.304+0000 7fc006842740 10 client.7785 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T09:01:03.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.304+0000 7fc006842740 20 client.7785 populate_metadata read hostname 'smithi043' 2024-04-25T09:01:03.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.304+0000 7fc006842740 1 --2- 172.21.15.43:0/3818799977 >> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] conn(0x564d5334fa90 0x564d5334feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.304+0000 7fc006842740 4 client.7785 mds_command: new command op to 7596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:01:03.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.304+0000 7fc006842740 1 -- 172.21.15.43:0/3818799977 --> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564d532670d0 con 0x564d5334fa90 2024-04-25T09:01:03.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.304+0000 7fc0018d7640 1 --2- 172.21.15.43:0/3818799977 >> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] conn(0x564d5334fa90 0x564d5334feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.304+0000 7fbfdcff9640 10 client.7785 ms_handle_connect on v2:172.21.15.43:6833/1341028696 2024-04-25T09:01:03.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.304+0000 7fc0018d7640 1 --2- 172.21.15.43:0/3818799977 >> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] conn(0x564d5334fa90 0x564d5334feb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6f08e01fa36db6b server_cookie=c63b87f743cc0e39 in_seq=0 out_seq=0 2024-04-25T09:01:03.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.305+0000 7fbfdcff9640 1 -- 172.21.15.43:0/3818799977 <== mds.0 v2:172.21.15.43:6833/1341028696 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564d532670d0 con 0x564d5334fa90 2024-04-25T09:01:03.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.305+0000 7fbfdcff9640 10 client.7785 handle_command_reply: tid=0 2024-04-25T09:01:03.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.305+0000 7fc006842740 1 client.7785 shutdown 2024-04-25T09:01:03.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.305+0000 7fc006842740 1 -- 172.21.15.43:0/3818799977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d53321760 msgr2=0x564d53321be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:03.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.305+0000 7fc006842740 1 --2- 172.21.15.43:0/3818799977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d53321760 0x564d53321be0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fbff800a430 tx=0x7fbff8058950 comp rx=0 tx=0).stop 2024-04-25T09:01:03.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.305+0000 7fc006842740 1 -- 172.21.15.43:0/3818799977 shutdown_connections 2024-04-25T09:01:03.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.305+0000 7fc006842740 1 --2- 172.21.15.43:0/3818799977 >> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] conn(0x564d5334fa90 0x564d5334feb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.305+0000 7fc006842740 1 --2- 172.21.15.43:0/3818799977 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564d53333370 0x564d5331dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.305+0000 7fc006842740 1 --2- 172.21.15.43:0/3818799977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d53321760 0x564d53321be0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.305+0000 7fbfdcff9640 0 client.7785 ms_handle_reset on v2:172.21.15.43:6833/1341028696 2024-04-25T09:01:03.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.305+0000 7fc006842740 1 --2- 172.21.15.43:0/3818799977 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d532f6ca0 0x564d53310ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.305+0000 7fc006842740 1 -- 172.21.15.43:0/3818799977 >> 172.21.15.43:0/3818799977 conn(0x564d532be870 msgr2=0x564d5331f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:03.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.305+0000 7fc006842740 1 -- 172.21.15.43:0/3818799977 shutdown_connections 2024-04-25T09:01:03.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.305+0000 7fc006842740 1 -- 172.21.15.43:0/3818799977 wait complete. 2024-04-25T09:01:03.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.305+0000 7fc006842740 20 client.7785 trim_cache size 0 max 16384 2024-04-25T09:01:03.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.320+0000 7fc006842740 1 Processor -- start 2024-04-25T09:01:03.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.320+0000 7fc006842740 1 -- start start 2024-04-25T09:01:03.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.321+0000 7fc006842740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d532f6ca0 0x564d53355ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.321+0000 7fc006842740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564d53321760 0x564d5331b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.321+0000 7fc006842740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d53333370 0x564d5331bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.321+0000 7fc006842740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc204230 con 0x564d532f6ca0 2024-04-25T09:01:03.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.321+0000 7fc006842740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc2033b0 con 0x564d53333370 2024-04-25T09:01:03.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.321+0000 7fc006842740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc0aaaa0 con 0x564d53321760 2024-04-25T09:01:03.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.321+0000 7fc0018d7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d532f6ca0 0x564d53355ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.321+0000 7fc0018d7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d532f6ca0 0x564d53355ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39294/0 (socket says 172.21.15.43:39294) 2024-04-25T09:01:03.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.321+0000 7fc0018d7640 1 -- 172.21.15.43:0/4144182425 learned_addr learned my addr 172.21.15.43:0/4144182425 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:03.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.321+0000 7fc0018d7640 1 -- 172.21.15.43:0/4144182425 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564d53321760 msgr2=0x564d5331b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:03.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.321+0000 7fc0010d6640 1 --2- 172.21.15.43:0/4144182425 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564d53321760 0x564d5331b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.321+0000 7fc0020d8640 1 --2- 172.21.15.43:0/4144182425 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d53333370 0x564d5331bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.321+0000 7fc0018d7640 1 --2- 172.21.15.43:0/4144182425 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564d53321760 0x564d5331b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.321+0000 7fc0018d7640 1 -- 172.21.15.43:0/4144182425 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d53333370 msgr2=0x564d5331bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:03.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.321+0000 7fc0018d7640 1 --2- 172.21.15.43:0/4144182425 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d53333370 0x564d5331bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.321+0000 7fc0018d7640 1 -- 172.21.15.43:0/4144182425 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564d52f071b0 con 0x564d532f6ca0 2024-04-25T09:01:03.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.321+0000 7fc0010d6640 1 --2- 172.21.15.43:0/4144182425 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564d53321760 0x564d5331b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:03.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.321+0000 7fc0020d8640 1 --2- 172.21.15.43:0/4144182425 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d53333370 0x564d5331bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:03.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.321+0000 7fc0018d7640 1 --2- 172.21.15.43:0/4144182425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d532f6ca0 0x564d53355ea0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fbff004b040 tx=0x7fbff003a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29127bb6d4a6d3ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:03.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.321+0000 7fbfdeffd640 1 -- 172.21.15.43:0/4144182425 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff0057070 con 0x564d532f6ca0 2024-04-25T09:01:03.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.322+0000 7fbfdeffd640 1 -- 172.21.15.43:0/4144182425 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff0051450 con 0x564d532f6ca0 2024-04-25T09:01:03.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.322+0000 7fbfdeffd640 1 -- 172.21.15.43:0/4144182425 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff004d070 con 0x564d532f6ca0 2024-04-25T09:01:03.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.322+0000 7fc006842740 1 -- 172.21.15.43:0/4144182425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d532f6ca0 msgr2=0x564d53355ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:03.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.322+0000 7fc006842740 1 --2- 172.21.15.43:0/4144182425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d532f6ca0 0x564d53355ea0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fbff004b040 tx=0x7fbff003a000 comp rx=0 tx=0).stop 2024-04-25T09:01:03.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.322+0000 7fc006842740 1 -- 172.21.15.43:0/4144182425 shutdown_connections 2024-04-25T09:01:03.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.322+0000 7fc006842740 1 --2- 172.21.15.43:0/4144182425 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d53333370 0x564d5331bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.322+0000 7fc006842740 1 --2- 172.21.15.43:0/4144182425 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564d53321760 0x564d5331b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.322+0000 7fc006842740 1 --2- 172.21.15.43:0/4144182425 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d532f6ca0 0x564d53355ea0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.322+0000 7fc006842740 1 -- 172.21.15.43:0/4144182425 >> 172.21.15.43:0/4144182425 conn(0x564d532be870 msgr2=0x564d532ecf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:03.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.322+0000 7fc006842740 1 -- 172.21.15.43:0/4144182425 shutdown_connections 2024-04-25T09:01:03.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.322+0000 7fc006842740 1 -- 172.21.15.43:0/4144182425 wait complete. 2024-04-25T09:01:03.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.323+0000 7fc006842740 1 -- start start 2024-04-25T09:01:03.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.323+0000 7fc006842740 1 Processor -- start 2024-04-25T09:01:03.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.324+0000 7fc006842740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d532f6ca0 0x564d533a9670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.324+0000 7fc006842740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d53321760 0x564d532f6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.324+0000 7fc006842740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564d53333370 0x564d533328a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.324+0000 7fc006842740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x564d53264a40 con 0x564d532f6ca0 2024-04-25T09:01:03.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.324+0000 7fc006842740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc0ab0a0 con 0x564d53321760 2024-04-25T09:01:03.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.324+0000 7fc006842740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x564d533567f0 con 0x564d53333370 2024-04-25T09:01:03.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.324+0000 7fc006842740 10 client.0 _ll_register_callbacks cb 0x7fbff80583a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:01:03.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.324+0000 7fc0018d7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d532f6ca0 0x564d533a9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.324+0000 7fc0018d7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d532f6ca0 0x564d533a9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39304/0 (socket says 172.21.15.43:39304) 2024-04-25T09:01:03.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.324+0000 7fc0018d7640 1 -- 172.21.15.43:0/2599106251 learned_addr learned my addr 172.21.15.43:0/2599106251 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:03.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.324+0000 7fc0010d6640 1 --2- 172.21.15.43:0/2599106251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d53321760 0x564d532f6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.324+0000 7fc0020d8640 1 --2- 172.21.15.43:0/2599106251 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564d53333370 0x564d533328a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.324+0000 7fc0018d7640 1 -- 172.21.15.43:0/2599106251 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564d53333370 msgr2=0x564d533328a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:03.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.324+0000 7fc0018d7640 1 --2- 172.21.15.43:0/2599106251 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564d53333370 0x564d533328a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.324+0000 7fc0018d7640 1 -- 172.21.15.43:0/2599106251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d53321760 msgr2=0x564d532f6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:03.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.324+0000 7fc0018d7640 1 --2- 172.21.15.43:0/2599106251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d53321760 0x564d532f6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.324+0000 7fc0018d7640 1 -- 172.21.15.43:0/2599106251 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564d53332190 con 0x564d532f6ca0 2024-04-25T09:01:03.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.324+0000 7fc0018d7640 1 --2- 172.21.15.43:0/2599106251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d532f6ca0 0x564d533a9670 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fbff0047000 tx=0x7fbff0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5516287aef20026e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:03.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.324+0000 7fbfdcff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:01:03.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.324+0000 7fbfdcff9640 1 -- 172.21.15.43:0/2599106251 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff0060060 con 0x564d532f6ca0 2024-04-25T09:01:03.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.324+0000 7fbfdcff9640 1 -- 172.21.15.43:0/2599106251 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fbff0051600 con 0x564d532f6ca0 2024-04-25T09:01:03.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.324+0000 7fc006842740 1 -- 172.21.15.43:0/2599106251 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbff0051450 con 0x564d532f6ca0 2024-04-25T09:01:03.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.325+0000 7fbfdcff9640 1 -- 172.21.15.43:0/2599106251 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbff0066050 con 0x564d532f6ca0 2024-04-25T09:01:03.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.325+0000 7fc006842740 10 client.7791 fetch_fsmap learned FSMap version 54 2024-04-25T09:01:03.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.325+0000 7fc006842740 1 -- 172.21.15.43:0/2599106251 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x564d5334ade0 con 0x564d532f6ca0 2024-04-25T09:01:03.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.325+0000 7fbfdcff9640 1 -- 172.21.15.43:0/2599106251 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbff0051450 con 0x564d532f6ca0 2024-04-25T09:01:03.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.325+0000 7fbfdcff9640 1 -- 172.21.15.43:0/2599106251 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 54) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbff0069020 con 0x564d532f6ca0 2024-04-25T09:01:03.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.325+0000 7fc006842740 10 client.7791 fetch_fsmap finished waiting for FSMap version 54 2024-04-25T09:01:03.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.325+0000 7fc006842740 10 client.7791 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T09:01:03.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.325+0000 7fc006842740 20 client.7791 populate_metadata read hostname 'smithi043' 2024-04-25T09:01:03.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.325+0000 7fc006842740 1 --2- 172.21.15.43:0/2599106251 >> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] conn(0x564d532f5460 0x564d532f5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.325+0000 7fc006842740 4 client.7791 mds_command: new command op to 7596 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-04-25T09:01:03.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.325+0000 7fc006842740 1 -- 172.21.15.43:0/2599106251 --> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] -- command(tid 0: {"prefix": "scrub abort"}) v1 -- 0x564d532f81b0 con 0x564d532f5460 2024-04-25T09:01:03.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.325+0000 7fc0010d6640 1 --2- 172.21.15.43:0/2599106251 >> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] conn(0x564d532f5460 0x564d532f5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.325+0000 7fbfdcff9640 10 client.7791 ms_handle_connect on v2:172.21.15.43:6833/1341028696 2024-04-25T09:01:03.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.325+0000 7fc0010d6640 1 --2- 172.21.15.43:0/2599106251 >> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] conn(0x564d532f5460 0x564d532f5840 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51627084e60a72e server_cookie=92c0af6205743a67 in_seq=0 out_seq=0 2024-04-25T09:01:03.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.325+0000 7fbfdcff9640 1 -- 172.21.15.43:0/2599106251 <== mds.0 v2:172.21.15.43:6833/1341028696 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x564d532f81b0 con 0x564d532f5460 2024-04-25T09:01:03.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.325+0000 7fbfdcff9640 10 client.7791 handle_command_reply: tid=0 2024-04-25T09:01:03.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.325+0000 7fc006842740 1 client.7791 shutdown 2024-04-25T09:01:03.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.325+0000 7fc006842740 1 -- 172.21.15.43:0/2599106251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d532f6ca0 msgr2=0x564d533a9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:03.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.325+0000 7fc006842740 1 --2- 172.21.15.43:0/2599106251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d532f6ca0 0x564d533a9670 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fbff0047000 tx=0x7fbff0004730 comp rx=0 tx=0).stop 2024-04-25T09:01:03.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.326+0000 7fc006842740 1 -- 172.21.15.43:0/2599106251 shutdown_connections 2024-04-25T09:01:03.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.326+0000 7fc006842740 1 --2- 172.21.15.43:0/2599106251 >> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] conn(0x564d532f5460 0x564d532f5840 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.326+0000 7fc006842740 1 --2- 172.21.15.43:0/2599106251 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x564d53333370 0x564d533328a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.326+0000 7fc006842740 1 --2- 172.21.15.43:0/2599106251 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x564d53321760 0x564d532f6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.326+0000 7fbfdcff9640 0 client.7791 ms_handle_reset on v2:172.21.15.43:6833/1341028696 2024-04-25T09:01:03.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.326+0000 7fc006842740 1 --2- 172.21.15.43:0/2599106251 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x564d532f6ca0 0x564d533a9670 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.326+0000 7fc006842740 1 -- 172.21.15.43:0/2599106251 >> 172.21.15.43:0/2599106251 conn(0x564d532be870 msgr2=0x564d5331b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:03.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.326+0000 7fc006842740 1 -- 172.21.15.43:0/2599106251 shutdown_connections 2024-04-25T09:01:03.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.326+0000 7fc006842740 1 -- 172.21.15.43:0/2599106251 wait complete. 2024-04-25T09:01:03.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.326+0000 7fc006842740 20 client.7791 trim_cache size 0 max 16384 2024-04-25T09:01:03.321 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:01:03.321 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0 2024-04-25T09:01:03.322 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:01:03.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.326+0000 7fc003b3b640 1 -- 172.21.15.43:0/866770818 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbfe006b500 msgr2=0x7fbfe006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:03.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.326+0000 7fc003b3b640 1 --2- 172.21.15.43:0/866770818 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbfe006b500 0x7fbfe006d9c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fbff0034000 tx=0x7fbff002e000 comp rx=0 tx=0).stop 2024-04-25T09:01:03.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.326+0000 7fc003b3b640 1 -- 172.21.15.43:0/866770818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbffc0b3ed0 msgr2=0x7fbffc151880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:03.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.326+0000 7fc003b3b640 1 --2- 172.21.15.43:0/866770818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbffc0b3ed0 0x7fbffc151880 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fbff800c8a0 tx=0x7fbff800cd60 comp rx=0 tx=0).stop 2024-04-25T09:01:03.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.326+0000 7fc003b3b640 1 -- 172.21.15.43:0/866770818 shutdown_connections 2024-04-25T09:01:03.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.326+0000 7fc003b3b640 1 --2- 172.21.15.43:0/866770818 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbfe006b500 0x7fbfe006d9c0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.326+0000 7fc003b3b640 1 --2- 172.21.15.43:0/866770818 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbffc15a960 0x7fbffc151130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.326+0000 7fc003b3b640 1 --2- 172.21.15.43:0/866770818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbffc0b3ed0 0x7fbffc151880 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.326+0000 7fc003b3b640 1 --2- 172.21.15.43:0/866770818 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbffc0aee40 0x7fbffc159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.326+0000 7fc003b3b640 1 -- 172.21.15.43:0/866770818 >> 172.21.15.43:0/866770818 conn(0x7fbffc06ab30 msgr2=0x7fbffc071750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:03.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.326+0000 7fc003b3b640 1 -- 172.21.15.43:0/866770818 shutdown_connections 2024-04-25T09:01:03.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.326+0000 7fc003b3b640 1 -- 172.21.15.43:0/866770818 wait complete. 2024-04-25T09:01:03.326 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T09:01:03.326 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T09:01:03.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.439+0000 7ffac32cb640 1 Processor -- start 2024-04-25T09:01:03.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.439+0000 7ffac32cb640 1 -- start start 2024-04-25T09:01:03.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.440+0000 7ffac32cb640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ffabc15bbe0 0x7ffabc0cfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.440+0000 7ffac32cb640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffabc0cfff0 0x7ffabc0b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.440+0000 7ffac32cb640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffabc070e90 con 0x7ffabc15b800 2024-04-25T09:01:03.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.440+0000 7ffac32cb640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffabc160640 con 0x7ffabc15bbe0 2024-04-25T09:01:03.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.440+0000 7ffac32cb640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc0d0460 con 0x7ffabc0cfff0 2024-04-25T09:01:03.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.440+0000 7ffac0866640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ffabc15bbe0 0x7ffabc0cfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.440+0000 7ffac0866640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ffabc15bbe0 0x7ffabc0cfab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39314/0 (socket says 172.21.15.43:39314) 2024-04-25T09:01:03.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.440+0000 7ffac0866640 1 -- 172.21.15.43:0/3200133668 learned_addr learned my addr 172.21.15.43:0/3200133668 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:03.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.440+0000 7ffac1868640 1 --2- 172.21.15.43:0/3200133668 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffabc0cfff0 0x7ffabc0b2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.441+0000 7ffac0866640 1 -- 172.21.15.43:0/3200133668 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffabc0cfff0 msgr2=0x7ffabc0b2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:03.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.441+0000 7ffac0866640 1 --2- 172.21.15.43:0/3200133668 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffabc0cfff0 0x7ffabc0b2ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.441+0000 7ffac0866640 1 -- 172.21.15.43:0/3200133668 >> v1:172.21.15.193:6789/0 conn(0x7ffabc15b800 legacy=0x7ffabc071bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:03.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.441+0000 7ffac0866640 1 -- 172.21.15.43:0/3200133668 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffabc071130 con 0x7ffabc15bbe0 2024-04-25T09:01:03.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.441+0000 7ffac0866640 1 --2- 172.21.15.43:0/3200133668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffabc15bbe0 0x7ffabc0cfab0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7ffab0009870 tx=0x7ffab002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aec3f912cd43f04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:03.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.441+0000 7ffabbfff640 1 -- 172.21.15.43:0/3200133668 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab0037070 con 0x7ffabc15bbe0 2024-04-25T09:01:03.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.442+0000 7ffabbfff640 1 -- 172.21.15.43:0/3200133668 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffab0032440 con 0x7ffabc15bbe0 2024-04-25T09:01:03.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.442+0000 7ffabbfff640 1 -- 172.21.15.43:0/3200133668 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab0032810 con 0x7ffabc15bbe0 2024-04-25T09:01:03.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.442+0000 7ffac32cb640 1 -- 172.21.15.43:0/3200133668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffabc15bbe0 msgr2=0x7ffabc0cfab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:03.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.442+0000 7ffac32cb640 1 --2- 172.21.15.43:0/3200133668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffabc15bbe0 0x7ffabc0cfab0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7ffab0009870 tx=0x7ffab002a270 comp rx=0 tx=0).stop 2024-04-25T09:01:03.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.442+0000 7ffac32cb640 1 -- 172.21.15.43:0/3200133668 shutdown_connections 2024-04-25T09:01:03.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.442+0000 7ffac32cb640 1 --2- 172.21.15.43:0/3200133668 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffabc0cfff0 0x7ffabc0b2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.442+0000 7ffac32cb640 1 --2- 172.21.15.43:0/3200133668 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffabc15bbe0 0x7ffabc0cfab0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.442+0000 7ffac32cb640 1 -- 172.21.15.43:0/3200133668 >> 172.21.15.43:0/3200133668 conn(0x7ffabc06ab30 msgr2=0x7ffabc06af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:03.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.442+0000 7ffac32cb640 1 -- 172.21.15.43:0/3200133668 shutdown_connections 2024-04-25T09:01:03.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.442+0000 7ffac32cb640 1 -- 172.21.15.43:0/3200133668 wait complete. 2024-04-25T09:01:03.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.443+0000 7ffac32cb640 1 Processor -- start 2024-04-25T09:01:03.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.443+0000 7ffac32cb640 1 -- start start 2024-04-25T09:01:03.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.444+0000 7ffac32cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffabc0cfff0 0x7ffabc14b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.444+0000 7ffac32cb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffabc15b800 0x7ffabc0a9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.444+0000 7ffac32cb640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffabc15bbe0 0x7ffabc0ae890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.444+0000 7ffac32cb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc071740 con 0x7ffabc0cfff0 2024-04-25T09:01:03.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.444+0000 7ffac32cb640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc0722e0 con 0x7ffabc15b800 2024-04-25T09:01:03.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.444+0000 7ffac32cb640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc0713d0 con 0x7ffabc15bbe0 2024-04-25T09:01:03.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.444+0000 7ffac1067640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffabc0cfff0 0x7ffabc14b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.444+0000 7ffac1067640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffabc0cfff0 0x7ffabc14b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39328/0 (socket says 172.21.15.43:39328) 2024-04-25T09:01:03.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.444+0000 7ffac1067640 1 -- 172.21.15.43:0/1643045864 learned_addr learned my addr 172.21.15.43:0/1643045864 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:03.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.444+0000 7ffac0866640 1 --2- 172.21.15.43:0/1643045864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffabc15b800 0x7ffabc0a9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.444+0000 7ffac1868640 1 --2- 172.21.15.43:0/1643045864 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffabc15bbe0 0x7ffabc0ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.444+0000 7ffac1067640 1 -- 172.21.15.43:0/1643045864 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffabc15bbe0 msgr2=0x7ffabc0ae890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:03.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.444+0000 7ffac1067640 1 --2- 172.21.15.43:0/1643045864 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffabc15bbe0 0x7ffabc0ae890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.444+0000 7ffac1067640 1 -- 172.21.15.43:0/1643045864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffabc15b800 msgr2=0x7ffabc0a9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:03.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.444+0000 7ffac1067640 1 --2- 172.21.15.43:0/1643045864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffabc15b800 0x7ffabc0a9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.444+0000 7ffac1067640 1 -- 172.21.15.43:0/1643045864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffabc0aa430 con 0x7ffabc0cfff0 2024-04-25T09:01:03.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.444+0000 7ffac0866640 1 --2- 172.21.15.43:0/1643045864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffabc15b800 0x7ffabc0a9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:03.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.444+0000 7ffac1868640 1 --2- 172.21.15.43:0/1643045864 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffabc15bbe0 0x7ffabc0ae890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:03.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.445+0000 7ffac1067640 1 --2- 172.21.15.43:0/1643045864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffabc0cfff0 0x7ffabc14b790 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00c7a0 tx=0x7ffaac00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:03.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.445+0000 7ffab9ffb640 1 -- 172.21.15.43:0/1643045864 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac003b90 con 0x7ffabc0cfff0 2024-04-25T09:01:03.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.445+0000 7ffab9ffb640 1 -- 172.21.15.43:0/1643045864 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaac003d50 con 0x7ffabc0cfff0 2024-04-25T09:01:03.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.445+0000 7ffac32cb640 1 -- 172.21.15.43:0/1643045864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffabc1f1b20 con 0x7ffabc0cfff0 2024-04-25T09:01:03.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.445+0000 7ffab9ffb640 1 -- 172.21.15.43:0/1643045864 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac00aa90 con 0x7ffabc0cfff0 2024-04-25T09:01:03.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.445+0000 7ffac32cb640 1 -- 172.21.15.43:0/1643045864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffabc1f20e0 con 0x7ffabc0cfff0 2024-04-25T09:01:03.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.447+0000 7ffab9ffb640 1 -- 172.21.15.43:0/1643045864 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffaac00ac30 con 0x7ffabc0cfff0 2024-04-25T09:01:03.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.447+0000 7ffab9ffb640 1 --2- 172.21.15.43:0/1643045864 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ffaa806b4d0 0x7ffaa806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.447+0000 7ffab9ffb640 1 -- 172.21.15.43:0/1643045864 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7ffaac08a0e0 con 0x7ffabc0cfff0 2024-04-25T09:01:03.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.447+0000 7ffac0866640 1 --2- 172.21.15.43:0/1643045864 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ffaa806b4d0 0x7ffaa806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.448+0000 7ffac0866640 1 --2- 172.21.15.43:0/1643045864 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ffaa806b4d0 0x7ffaa806d990 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ffab0034000 tx=0x7ffab002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:03.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.448+0000 7ffac5fd0740 1 Processor -- start 2024-04-25T09:01:03.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.448+0000 7ffac5fd0740 1 -- start start 2024-04-25T09:01:03.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.449+0000 7ffac5fd0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcdcf34b10 0x55dcdcf37320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.449+0000 7ffac5fd0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dcdcf34ef0 0x55dcdcf37860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.449+0000 7ffac5fd0740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55dcdcf07770 0x55dcdcf0af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.449+0000 7ffac5fd0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcdce70da0 con 0x55dcdcf34b10 2024-04-25T09:01:03.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.449+0000 7ffac5fd0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcdce19b80 con 0x55dcdcf34ef0 2024-04-25T09:01:03.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.449+0000 7ffac5fd0740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dcdce6ce40 con 0x55dcdcf07770 2024-04-25T09:01:03.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.449+0000 7ffac1067640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcdcf34b10 0x55dcdcf37320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.449+0000 7ffac1067640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcdcf34b10 0x55dcdcf37320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39330/0 (socket says 172.21.15.43:39330) 2024-04-25T09:01:03.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.449+0000 7ffac1067640 1 -- 172.21.15.43:0/2221367539 learned_addr learned my addr 172.21.15.43:0/2221367539 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:03.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.449+0000 7ffac0866640 1 --2- 172.21.15.43:0/2221367539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dcdcf34ef0 0x55dcdcf37860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.449+0000 7ffac1868640 1 --2- 172.21.15.43:0/2221367539 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55dcdcf07770 0x55dcdcf0af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.449+0000 7ffac1067640 1 -- 172.21.15.43:0/2221367539 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55dcdcf07770 msgr2=0x55dcdcf0af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:03.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.449+0000 7ffac1067640 1 --2- 172.21.15.43:0/2221367539 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55dcdcf07770 0x55dcdcf0af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.449+0000 7ffac1067640 1 -- 172.21.15.43:0/2221367539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dcdcf34ef0 msgr2=0x55dcdcf37860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:03.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.449+0000 7ffac1067640 1 --2- 172.21.15.43:0/2221367539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dcdcf34ef0 0x55dcdcf37860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.449+0000 7ffac1067640 1 -- 172.21.15.43:0/2221367539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dcdce23f10 con 0x55dcdcf34b10 2024-04-25T09:01:03.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.449+0000 7ffac0866640 1 --2- 172.21.15.43:0/2221367539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dcdcf34ef0 0x55dcdcf37860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:01:03.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.449+0000 7ffac1067640 1 --2- 172.21.15.43:0/2221367539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcdcf34b10 0x55dcdcf37320 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7ffaac060ab0 tx=0x7ffaac0582f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a02a603dfba72ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:03.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.449+0000 7ffaa2ffd640 1 -- 172.21.15.43:0/2221367539 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac060fe0 con 0x55dcdcf34b10 2024-04-25T09:01:03.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.449+0000 7ffaa2ffd640 1 -- 172.21.15.43:0/2221367539 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaac056bd0 con 0x55dcdcf34b10 2024-04-25T09:01:03.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.449+0000 7ffaa2ffd640 1 -- 172.21.15.43:0/2221367539 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac054f40 con 0x55dcdcf34b10 2024-04-25T09:01:03.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.450+0000 7ffac5fd0740 1 -- 172.21.15.43:0/2221367539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcdcf34b10 msgr2=0x55dcdcf37320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:03.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.450+0000 7ffac5fd0740 1 --2- 172.21.15.43:0/2221367539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcdcf34b10 0x55dcdcf37320 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7ffaac060ab0 tx=0x7ffaac0582f0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.450+0000 7ffac5fd0740 1 -- 172.21.15.43:0/2221367539 shutdown_connections 2024-04-25T09:01:03.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.450+0000 7ffac5fd0740 1 --2- 172.21.15.43:0/2221367539 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55dcdcf07770 0x55dcdcf0af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.450+0000 7ffac5fd0740 1 --2- 172.21.15.43:0/2221367539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dcdcf34ef0 0x55dcdcf37860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.450+0000 7ffac5fd0740 1 --2- 172.21.15.43:0/2221367539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcdcf34b10 0x55dcdcf37320 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.450+0000 7ffac5fd0740 1 -- 172.21.15.43:0/2221367539 >> 172.21.15.43:0/2221367539 conn(0x55dcdcf099e0 msgr2=0x55dcdcf52320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:03.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.450+0000 7ffac5fd0740 1 -- 172.21.15.43:0/2221367539 shutdown_connections 2024-04-25T09:01:03.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.450+0000 7ffac5fd0740 1 -- 172.21.15.43:0/2221367539 wait complete. 2024-04-25T09:01:03.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.450+0000 7ffac5fd0740 1 -- start start 2024-04-25T09:01:03.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.451+0000 7ffac5fd0740 1 Processor -- start 2024-04-25T09:01:03.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.451+0000 7ffac5fd0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dcdcf34b10 0x55dcdcf2bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.451+0000 7ffac5fd0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcdcf34ef0 0x55dcdcf4baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.451+0000 7ffac5fd0740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55dcdcf4bfe0 0x55dcdcf498c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.451+0000 7ffac5fd0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcdcf0cb90 con 0x55dcdcf34ef0 2024-04-25T09:01:03.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.451+0000 7ffac5fd0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcdce0f4b0 con 0x55dcdcf34b10 2024-04-25T09:01:03.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.451+0000 7ffac5fd0740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dcdcf0ce70 con 0x55dcdcf4bfe0 2024-04-25T09:01:03.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.452+0000 7ffac5fd0740 10 client.0 _ll_register_callbacks cb 0x55dcdce4f180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:01:03.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.452+0000 7ffac0866640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcdcf34ef0 0x55dcdcf4baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.452+0000 7ffac0866640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcdcf34ef0 0x55dcdcf4baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39344/0 (socket says 172.21.15.43:39344) 2024-04-25T09:01:03.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.452+0000 7ffac0866640 1 -- 172.21.15.43:0/3764253382 learned_addr learned my addr 172.21.15.43:0/3764253382 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:03.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.452+0000 7ffac1067640 1 --2- 172.21.15.43:0/3764253382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dcdcf34b10 0x55dcdcf2bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.452+0000 7ffac1868640 1 --2- 172.21.15.43:0/3764253382 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55dcdcf4bfe0 0x55dcdcf498c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.452+0000 7ffac0866640 1 -- 172.21.15.43:0/3764253382 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55dcdcf4bfe0 msgr2=0x55dcdcf498c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:03.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.452+0000 7ffac0866640 1 --2- 172.21.15.43:0/3764253382 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55dcdcf4bfe0 0x55dcdcf498c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.452+0000 7ffac0866640 1 -- 172.21.15.43:0/3764253382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dcdcf34b10 msgr2=0x55dcdcf2bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:03.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.452+0000 7ffac0866640 1 --2- 172.21.15.43:0/3764253382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dcdcf34b10 0x55dcdcf2bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.452+0000 7ffac0866640 1 -- 172.21.15.43:0/3764253382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dcdcc4f7d0 con 0x55dcdcf34ef0 2024-04-25T09:01:03.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.452+0000 7ffac0866640 1 --2- 172.21.15.43:0/3764253382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcdcf34ef0 0x55dcdcf4baa0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7ffab0009960 tx=0x7ffab0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd2d0f66bdeaf803 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:03.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.452+0000 7ffa9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:01:03.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.452+0000 7ffa9bfff640 1 -- 172.21.15.43:0/3764253382 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab0037040 con 0x55dcdcf34ef0 2024-04-25T09:01:03.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.452+0000 7ffa9bfff640 1 -- 172.21.15.43:0/3764253382 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7ffab003cd70 con 0x55dcdcf34ef0 2024-04-25T09:01:03.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.452+0000 7ffac5fd0740 1 -- 172.21.15.43:0/3764253382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffaac056bd0 con 0x55dcdcf34ef0 2024-04-25T09:01:03.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.452+0000 7ffa9bfff640 1 -- 172.21.15.43:0/3764253382 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffab0055050 con 0x55dcdcf34ef0 2024-04-25T09:01:03.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.452+0000 7ffac5fd0740 10 client.7803 fetch_fsmap learned FSMap version 54 2024-04-25T09:01:03.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.452+0000 7ffac5fd0740 1 -- 172.21.15.43:0/3764253382 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x55dcdcb853f0 con 0x55dcdcf34ef0 2024-04-25T09:01:03.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.453+0000 7ffa9bfff640 1 -- 172.21.15.43:0/3764253382 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaac056bd0 con 0x55dcdcf34ef0 2024-04-25T09:01:03.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.453+0000 7ffa9bfff640 1 -- 172.21.15.43:0/3764253382 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 54) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ffab0058020 con 0x55dcdcf34ef0 2024-04-25T09:01:03.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.453+0000 7ffac5fd0740 10 client.7803 fetch_fsmap finished waiting for FSMap version 54 2024-04-25T09:01:03.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.453+0000 7ffac5fd0740 10 client.7803 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T09:01:03.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.453+0000 7ffac5fd0740 20 client.7803 populate_metadata read hostname 'smithi043' 2024-04-25T09:01:03.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.453+0000 7ffac5fd0740 1 --2- 172.21.15.43:0/3764253382 >> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] conn(0x55dcdcf547e0 0x55dcdcf54c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.453+0000 7ffac5fd0740 4 client.7803 mds_command: new command op to 7596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:01:03.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.453+0000 7ffac5fd0740 1 -- 172.21.15.43:0/3764253382 --> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dcdce7e2e0 con 0x55dcdcf547e0 2024-04-25T09:01:03.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.453+0000 7ffac1067640 1 --2- 172.21.15.43:0/3764253382 >> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] conn(0x55dcdcf547e0 0x55dcdcf54c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.453+0000 7ffa9bfff640 10 client.7803 ms_handle_connect on v2:172.21.15.43:6833/1341028696 2024-04-25T09:01:03.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.453+0000 7ffac1067640 1 --2- 172.21.15.43:0/3764253382 >> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] conn(0x55dcdcf547e0 0x55dcdcf54c10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c787429d6433372f server_cookie=7b9e78395355f42b in_seq=0 out_seq=0 2024-04-25T09:01:03.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.454+0000 7ffa9bfff640 1 -- 172.21.15.43:0/3764253382 <== mds.0 v2:172.21.15.43:6833/1341028696 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dcdce7e2e0 con 0x55dcdcf547e0 2024-04-25T09:01:03.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.454+0000 7ffa9bfff640 10 client.7803 handle_command_reply: tid=0 2024-04-25T09:01:03.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.454+0000 7ffac5fd0740 1 client.7803 shutdown 2024-04-25T09:01:03.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.454+0000 7ffac5fd0740 1 -- 172.21.15.43:0/3764253382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcdcf34ef0 msgr2=0x55dcdcf4baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:03.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.454+0000 7ffac5fd0740 1 --2- 172.21.15.43:0/3764253382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcdcf34ef0 0x55dcdcf4baa0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7ffab0009960 tx=0x7ffab0004730 comp rx=0 tx=0).stop 2024-04-25T09:01:03.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.454+0000 7ffac5fd0740 1 -- 172.21.15.43:0/3764253382 shutdown_connections 2024-04-25T09:01:03.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.454+0000 7ffac5fd0740 1 --2- 172.21.15.43:0/3764253382 >> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] conn(0x55dcdcf547e0 0x55dcdcf54c10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.454+0000 7ffac5fd0740 1 --2- 172.21.15.43:0/3764253382 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55dcdcf4bfe0 0x55dcdcf498c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.454+0000 7ffac5fd0740 1 --2- 172.21.15.43:0/3764253382 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcdcf34ef0 0x55dcdcf4baa0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.454+0000 7ffac5fd0740 1 --2- 172.21.15.43:0/3764253382 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dcdcf34b10 0x55dcdcf2bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.454+0000 7ffa9bfff640 0 client.7803 ms_handle_reset on v2:172.21.15.43:6833/1341028696 2024-04-25T09:01:03.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.454+0000 7ffac5fd0740 1 -- 172.21.15.43:0/3764253382 >> 172.21.15.43:0/3764253382 conn(0x55dcdcf07770 msgr2=0x55dcdcf08400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:03.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.454+0000 7ffac5fd0740 1 -- 172.21.15.43:0/3764253382 shutdown_connections 2024-04-25T09:01:03.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.454+0000 7ffac5fd0740 1 -- 172.21.15.43:0/3764253382 wait complete. 2024-04-25T09:01:03.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.454+0000 7ffac5fd0740 20 client.7803 trim_cache size 0 max 16384 2024-04-25T09:01:03.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.469+0000 7ffac5fd0740 1 Processor -- start 2024-04-25T09:01:03.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.469+0000 7ffac5fd0740 1 -- start start 2024-04-25T09:01:03.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.470+0000 7ffac5fd0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcdcf34b10 0x55dcdcf2bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.470+0000 7ffac5fd0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dcdcf4bfe0 0x55dcdcf518c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.470+0000 7ffac5fd0740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55dcdcf51e00 0x55dcdcf07c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.470+0000 7ffac5fd0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc1fdd80 con 0x55dcdcf34b10 2024-04-25T09:01:03.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.470+0000 7ffac5fd0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc0abad0 con 0x55dcdcf4bfe0 2024-04-25T09:01:03.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.470+0000 7ffac5fd0740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc1f5b80 con 0x55dcdcf51e00 2024-04-25T09:01:03.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.470+0000 7ffac1067640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcdcf34b10 0x55dcdcf2bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.470+0000 7ffac1067640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcdcf34b10 0x55dcdcf2bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39356/0 (socket says 172.21.15.43:39356) 2024-04-25T09:01:03.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.470+0000 7ffac1067640 1 -- 172.21.15.43:0/2616053726 learned_addr learned my addr 172.21.15.43:0/2616053726 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:03.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.470+0000 7ffac0866640 1 --2- 172.21.15.43:0/2616053726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dcdcf4bfe0 0x55dcdcf518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.470+0000 7ffac1868640 1 --2- 172.21.15.43:0/2616053726 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55dcdcf51e00 0x55dcdcf07c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.470+0000 7ffac1067640 1 -- 172.21.15.43:0/2616053726 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55dcdcf51e00 msgr2=0x55dcdcf07c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:03.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.470+0000 7ffac1067640 1 --2- 172.21.15.43:0/2616053726 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55dcdcf51e00 0x55dcdcf07c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.470+0000 7ffac1067640 1 -- 172.21.15.43:0/2616053726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dcdcf4bfe0 msgr2=0x55dcdcf518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:03.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.470+0000 7ffac1067640 1 --2- 172.21.15.43:0/2616053726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dcdcf4bfe0 0x55dcdcf518c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.470+0000 7ffac1067640 1 -- 172.21.15.43:0/2616053726 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dcdcb450f0 con 0x55dcdcf34b10 2024-04-25T09:01:03.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.470+0000 7ffac0866640 1 --2- 172.21.15.43:0/2616053726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dcdcf4bfe0 0x55dcdcf518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:01:03.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.470+0000 7ffac1067640 1 --2- 172.21.15.43:0/2616053726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcdcf34b10 0x55dcdcf2bb60 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7ffaac08a400 tx=0x7ffaac005070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e9b5e645a4e2c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:03.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.471+0000 7ffaa27fc640 1 -- 172.21.15.43:0/2616053726 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac087070 con 0x55dcdcf34b10 2024-04-25T09:01:03.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.471+0000 7ffaa27fc640 1 -- 172.21.15.43:0/2616053726 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaac060fe0 con 0x55dcdcf34b10 2024-04-25T09:01:03.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.471+0000 7ffaa27fc640 1 -- 172.21.15.43:0/2616053726 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffaac010dc0 con 0x55dcdcf34b10 2024-04-25T09:01:03.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.471+0000 7ffac5fd0740 1 -- 172.21.15.43:0/2616053726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcdcf34b10 msgr2=0x55dcdcf2bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:03.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.471+0000 7ffac5fd0740 1 --2- 172.21.15.43:0/2616053726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcdcf34b10 0x55dcdcf2bb60 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7ffaac08a400 tx=0x7ffaac005070 comp rx=0 tx=0).stop 2024-04-25T09:01:03.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.471+0000 7ffac5fd0740 1 -- 172.21.15.43:0/2616053726 shutdown_connections 2024-04-25T09:01:03.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.471+0000 7ffac5fd0740 1 --2- 172.21.15.43:0/2616053726 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55dcdcf51e00 0x55dcdcf07c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.471+0000 7ffac5fd0740 1 --2- 172.21.15.43:0/2616053726 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dcdcf4bfe0 0x55dcdcf518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.471+0000 7ffac5fd0740 1 --2- 172.21.15.43:0/2616053726 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcdcf34b10 0x55dcdcf2bb60 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.471+0000 7ffac5fd0740 1 -- 172.21.15.43:0/2616053726 >> 172.21.15.43:0/2616053726 conn(0x55dcdcf07770 msgr2=0x55dcdcf4a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:03.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.471+0000 7ffac5fd0740 1 -- 172.21.15.43:0/2616053726 shutdown_connections 2024-04-25T09:01:03.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.471+0000 7ffac5fd0740 1 -- 172.21.15.43:0/2616053726 wait complete. 2024-04-25T09:01:03.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.472+0000 7ffac5fd0740 1 -- start start 2024-04-25T09:01:03.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.472+0000 7ffac5fd0740 1 Processor -- start 2024-04-25T09:01:03.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.473+0000 7ffac5fd0740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dcdcf34b10 0x55dcdcf07c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.473+0000 7ffac5fd0740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55dcdcf4bfe0 0x55dcdcf08190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.473+0000 7ffac5fd0740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcdcf51e00 0x55dcdcf3a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.473+0000 7ffac5fd0740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dcdcf3b870 con 0x55dcdcf51e00 2024-04-25T09:01:03.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.473+0000 7ffac5fd0740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffabc0ab4d0 con 0x55dcdcf34b10 2024-04-25T09:01:03.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.473+0000 7ffac5fd0740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dcdcf0d7a0 con 0x55dcdcf4bfe0 2024-04-25T09:01:03.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.473+0000 7ffac5fd0740 10 client.0 _ll_register_callbacks cb 0x7ffaac001840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:01:03.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.473+0000 7ffac1868640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcdcf51e00 0x55dcdcf3a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.473+0000 7ffac1868640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcdcf51e00 0x55dcdcf3a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39366/0 (socket says 172.21.15.43:39366) 2024-04-25T09:01:03.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.473+0000 7ffac1868640 1 -- 172.21.15.43:0/1985413442 learned_addr learned my addr 172.21.15.43:0/1985413442 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:03.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.473+0000 7ffac0866640 1 --2- 172.21.15.43:0/1985413442 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55dcdcf4bfe0 0x55dcdcf08190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.473+0000 7ffac1067640 1 --2- 172.21.15.43:0/1985413442 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dcdcf34b10 0x55dcdcf07c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.473+0000 7ffac1868640 1 -- 172.21.15.43:0/1985413442 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55dcdcf4bfe0 msgr2=0x55dcdcf08190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:03.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.473+0000 7ffac1868640 1 --2- 172.21.15.43:0/1985413442 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55dcdcf4bfe0 0x55dcdcf08190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.473+0000 7ffac1868640 1 -- 172.21.15.43:0/1985413442 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dcdcf34b10 msgr2=0x55dcdcf07c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:03.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.473+0000 7ffac1868640 1 --2- 172.21.15.43:0/1985413442 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dcdcf34b10 0x55dcdcf07c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.473+0000 7ffac1868640 1 -- 172.21.15.43:0/1985413442 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dcdcf48d10 con 0x55dcdcf51e00 2024-04-25T09:01:03.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.473+0000 7ffac0866640 1 --2- 172.21.15.43:0/1985413442 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55dcdcf4bfe0 0x55dcdcf08190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:01:03.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.473+0000 7ffac1868640 1 --2- 172.21.15.43:0/1985413442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcdcf51e00 0x55dcdcf3a540 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7ffab4009ce0 tx=0x7ffab4015730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5a31f423a297c5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:03.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.473+0000 7ffa9bfff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:01:03.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.473+0000 7ffa9bfff640 1 -- 172.21.15.43:0/1985413442 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab401b070 con 0x55dcdcf51e00 2024-04-25T09:01:03.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.474+0000 7ffa9bfff640 1 -- 172.21.15.43:0/1985413442 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7ffab4004230 con 0x55dcdcf51e00 2024-04-25T09:01:03.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.474+0000 7ffac5fd0740 1 -- 172.21.15.43:0/1985413442 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffaac060fe0 con 0x55dcdcf51e00 2024-04-25T09:01:03.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.474+0000 7ffa9bfff640 1 -- 172.21.15.43:0/1985413442 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffab4027050 con 0x55dcdcf51e00 2024-04-25T09:01:03.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.474+0000 7ffac5fd0740 10 client.7809 fetch_fsmap learned FSMap version 54 2024-04-25T09:01:03.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.474+0000 7ffac5fd0740 1 -- 172.21.15.43:0/1985413442 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x55dcdcf48730 con 0x55dcdcf51e00 2024-04-25T09:01:03.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.474+0000 7ffa9bfff640 1 -- 172.21.15.43:0/1985413442 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffaac060fe0 con 0x55dcdcf51e00 2024-04-25T09:01:03.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.474+0000 7ffa9bfff640 1 -- 172.21.15.43:0/1985413442 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 54) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ffab402a020 con 0x55dcdcf51e00 2024-04-25T09:01:03.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.474+0000 7ffac5fd0740 10 client.7809 fetch_fsmap finished waiting for FSMap version 54 2024-04-25T09:01:03.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.474+0000 7ffac5fd0740 10 client.7809 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T09:01:03.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.474+0000 7ffac5fd0740 20 client.7809 populate_metadata read hostname 'smithi043' 2024-04-25T09:01:03.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.474+0000 7ffac5fd0740 1 --2- 172.21.15.43:0/1985413442 >> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] conn(0x55dcdcf48fb0 0x55dcdcf3aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:03.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.474+0000 7ffac5fd0740 4 client.7809 mds_command: new command op to 7596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:01:03.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.474+0000 7ffac5fd0740 1 -- 172.21.15.43:0/1985413442 --> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55dcdce78630 con 0x55dcdcf48fb0 2024-04-25T09:01:03.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.474+0000 7ffac1067640 1 --2- 172.21.15.43:0/1985413442 >> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] conn(0x55dcdcf48fb0 0x55dcdcf3aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:03.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.474+0000 7ffa9bfff640 10 client.7809 ms_handle_connect on v2:172.21.15.43:6833/1341028696 2024-04-25T09:01:03.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.474+0000 7ffac1067640 1 --2- 172.21.15.43:0/1985413442 >> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] conn(0x55dcdcf48fb0 0x55dcdcf3aa80 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ddc1eb8a55db52 server_cookie=9268788347e0823c in_seq=0 out_seq=0 2024-04-25T09:01:03.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.475+0000 7ffa9bfff640 1 -- 172.21.15.43:0/1985413442 <== mds.0 v2:172.21.15.43:6833/1341028696 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x55dcdce78630 con 0x55dcdcf48fb0 2024-04-25T09:01:03.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.475+0000 7ffa9bfff640 10 client.7809 handle_command_reply: tid=0 2024-04-25T09:01:03.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.475+0000 7ffac5fd0740 1 client.7809 shutdown 2024-04-25T09:01:03.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.475+0000 7ffac5fd0740 1 -- 172.21.15.43:0/1985413442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcdcf51e00 msgr2=0x55dcdcf3a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:03.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.475+0000 7ffac5fd0740 1 --2- 172.21.15.43:0/1985413442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcdcf51e00 0x55dcdcf3a540 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7ffab4009ce0 tx=0x7ffab4015730 comp rx=0 tx=0).stop 2024-04-25T09:01:03.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.475+0000 7ffac5fd0740 1 -- 172.21.15.43:0/1985413442 shutdown_connections 2024-04-25T09:01:03.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.475+0000 7ffac5fd0740 1 --2- 172.21.15.43:0/1985413442 >> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] conn(0x55dcdcf48fb0 0x55dcdcf3aa80 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.475+0000 7ffac5fd0740 1 --2- 172.21.15.43:0/1985413442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55dcdcf51e00 0x55dcdcf3a540 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.475+0000 7ffac5fd0740 1 --2- 172.21.15.43:0/1985413442 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55dcdcf4bfe0 0x55dcdcf08190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.475+0000 7ffa9bfff640 0 client.7809 ms_handle_reset on v2:172.21.15.43:6833/1341028696 2024-04-25T09:01:03.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.475+0000 7ffac5fd0740 1 --2- 172.21.15.43:0/1985413442 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55dcdcf34b10 0x55dcdcf07c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.475+0000 7ffac5fd0740 1 -- 172.21.15.43:0/1985413442 >> 172.21.15.43:0/1985413442 conn(0x55dcdcf07770 msgr2=0x55dcdcf5a280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:03.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.475+0000 7ffac5fd0740 1 -- 172.21.15.43:0/1985413442 shutdown_connections 2024-04-25T09:01:03.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.475+0000 7ffac5fd0740 1 -- 172.21.15.43:0/1985413442 wait complete. 2024-04-25T09:01:03.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.475+0000 7ffac5fd0740 20 client.7809 trim_cache size 0 max 16384 2024-04-25T09:01:03.471 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:01:03.471 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T09:01:03.471 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-25T09:01:03.471 INFO:teuthology.orchestra.run.smithi043.stdout: "d1983208-ea9f-4dd7-aa29-f33a93518599": { 2024-04-25T09:01:03.471 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/scrub_control_test_path", 2024-04-25T09:01:03.471 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "d1983208-ea9f-4dd7-aa29-f33a93518599", 2024-04-25T09:01:03.471 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive" 2024-04-25T09:01:03.471 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:01:03.471 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:01:03.471 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:01:03.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.476+0000 7ffac32cb640 1 -- 172.21.15.43:0/1643045864 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ffaa806b4d0 msgr2=0x7ffaa806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:03.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.476+0000 7ffac32cb640 1 --2- 172.21.15.43:0/1643045864 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ffaa806b4d0 0x7ffaa806d990 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ffab0034000 tx=0x7ffab002e000 comp rx=0 tx=0).stop 2024-04-25T09:01:03.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.476+0000 7ffac32cb640 1 -- 172.21.15.43:0/1643045864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffabc0cfff0 msgr2=0x7ffabc14b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:03.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.476+0000 7ffac32cb640 1 --2- 172.21.15.43:0/1643045864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffabc0cfff0 0x7ffabc14b790 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00c7a0 tx=0x7ffaac00cc60 comp rx=0 tx=0).stop 2024-04-25T09:01:03.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.476+0000 7ffac32cb640 1 -- 172.21.15.43:0/1643045864 shutdown_connections 2024-04-25T09:01:03.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.476+0000 7ffac32cb640 1 --2- 172.21.15.43:0/1643045864 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffabc15bbe0 0x7ffabc0ae890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.476+0000 7ffac32cb640 1 --2- 172.21.15.43:0/1643045864 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ffaa806b4d0 0x7ffaa806d990 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.476+0000 7ffac32cb640 1 --2- 172.21.15.43:0/1643045864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffabc15b800 0x7ffabc0a9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.476+0000 7ffac32cb640 1 --2- 172.21.15.43:0/1643045864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffabc0cfff0 0x7ffabc14b790 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:03.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.476+0000 7ffac32cb640 1 -- 172.21.15.43:0/1643045864 >> 172.21.15.43:0/1643045864 conn(0x7ffabc06ab30 msgr2=0x7ffabc06af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:03.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.476+0000 7ffac32cb640 1 -- 172.21.15.43:0/1643045864 shutdown_connections 2024-04-25T09:01:03.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:03.476+0000 7ffac32cb640 1 -- 172.21.15.43:0/1643045864 wait complete. 2024-04-25T09:01:08.475 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T09:01:08.476 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T09:01:08.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.601+0000 7f85757a5640 1 Processor -- start 2024-04-25T09:01:08.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.602+0000 7f85757a5640 1 -- start start 2024-04-25T09:01:08.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.602+0000 7f85757a5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f857015ddd0 0x7f85700b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:08.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.602+0000 7f85757a5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f857006d4f0 0x7f857006f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:08.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.602+0000 7f85757a5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f857006d940 con 0x7f857006ef90 2024-04-25T09:01:08.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.602+0000 7f85757a5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f857006f930 con 0x7f857006d4f0 2024-04-25T09:01:08.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.602+0000 7f85757a5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85700d2620 con 0x7f857015ddd0 2024-04-25T09:01:08.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.603+0000 7f856f7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f857006d4f0 0x7f857006f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:08.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.603+0000 7f856e7fc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f857006ef90 0x7f85700cc460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59300/0 (socket says 172.21.15.43:59300) 2024-04-25T09:01:08.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.603+0000 7f856e7fc640 1 -- 172.21.15.43:0/3333936202 learned_addr learned my addr 172.21.15.43:0/3333936202 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:08.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.603+0000 7f856effd640 1 --2- 172.21.15.43:0/3333936202 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f857015ddd0 0x7f85700b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:08.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.603+0000 7f856dffb640 1 -- 172.21.15.43:0/3333936202 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1666836390 0 0) 0x7f857006d940 con 0x7f857006ef90 2024-04-25T09:01:08.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.603+0000 7f856dffb640 1 -- 172.21.15.43:0/3333936202 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f855c003520 con 0x7f857006ef90 2024-04-25T09:01:08.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.603+0000 7f856f7fe640 1 -- 172.21.15.43:0/3333936202 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f857015ddd0 msgr2=0x7f85700b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:08.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.603+0000 7f856f7fe640 1 --2- 172.21.15.43:0/3333936202 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f857015ddd0 0x7f85700b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.603+0000 7f856f7fe640 1 -- 172.21.15.43:0/3333936202 >> v1:172.21.15.43:6789/0 conn(0x7f857006ef90 legacy=0x7f85700cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:08.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.603+0000 7f856f7fe640 1 -- 172.21.15.43:0/3333936202 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85700b0b80 con 0x7f857006d4f0 2024-04-25T09:01:08.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.604+0000 7f856f7fe640 1 --2- 172.21.15.43:0/3333936202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f857006d4f0 0x7f857006f3f0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f8560009d90 tx=0x7f856002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cdebac5b1f769a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:08.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.604+0000 7f856dffb640 1 -- 172.21.15.43:0/3333936202 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8560007740 con 0x7f857006d4f0 2024-04-25T09:01:08.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.604+0000 7f856dffb640 1 -- 172.21.15.43:0/3333936202 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85600078e0 con 0x7f857006d4f0 2024-04-25T09:01:08.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.604+0000 7f856dffb640 1 -- 172.21.15.43:0/3333936202 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8560007cb0 con 0x7f857006d4f0 2024-04-25T09:01:08.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.604+0000 7f85757a5640 1 -- 172.21.15.43:0/3333936202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f857006d4f0 msgr2=0x7f857006f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:08.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.604+0000 7f85757a5640 1 --2- 172.21.15.43:0/3333936202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f857006d4f0 0x7f857006f3f0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f8560009d90 tx=0x7f856002a660 comp rx=0 tx=0).stop 2024-04-25T09:01:08.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.604+0000 7f85757a5640 1 -- 172.21.15.43:0/3333936202 shutdown_connections 2024-04-25T09:01:08.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.604+0000 7f85757a5640 1 --2- 172.21.15.43:0/3333936202 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f857006d4f0 0x7f857006f3f0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.604+0000 7f85757a5640 1 --2- 172.21.15.43:0/3333936202 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f857015ddd0 0x7f85700b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.604+0000 7f85757a5640 1 -- 172.21.15.43:0/3333936202 >> 172.21.15.43:0/3333936202 conn(0x7f85700d17e0 msgr2=0x7f85700d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:08.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.604+0000 7f85757a5640 1 -- 172.21.15.43:0/3333936202 shutdown_connections 2024-04-25T09:01:08.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.605+0000 7f85757a5640 1 -- 172.21.15.43:0/3333936202 wait complete. 2024-04-25T09:01:08.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.606+0000 7f85757a5640 1 Processor -- start 2024-04-25T09:01:08.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.606+0000 7f85757a5640 1 -- start start 2024-04-25T09:01:08.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.607+0000 7f85757a5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f857006d4f0 0x7f85700c8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:08.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.607+0000 7f85757a5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f857006ef90 0x7f857014a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:08.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.607+0000 7f85757a5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f857015ddd0 0x7f85700b2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:08.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.607+0000 7f85757a5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85700ccb70 con 0x7f857015ddd0 2024-04-25T09:01:08.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.607+0000 7f85757a5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85700cbf40 con 0x7f857006ef90 2024-04-25T09:01:08.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.607+0000 7f85757a5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85700cccf0 con 0x7f857006d4f0 2024-04-25T09:01:08.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.607+0000 7f856f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f857015ddd0 0x7f85700b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:08.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.607+0000 7f856f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f857015ddd0 0x7f85700b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39384/0 (socket says 172.21.15.43:39384) 2024-04-25T09:01:08.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.607+0000 7f856f7fe640 1 -- 172.21.15.43:0/3818734543 learned_addr learned my addr 172.21.15.43:0/3818734543 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:08.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.607+0000 7f856e7fc640 1 --2- 172.21.15.43:0/3818734543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f857006ef90 0x7f857014a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:08.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.607+0000 7f856f7fe640 1 -- 172.21.15.43:0/3818734543 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f857006d4f0 msgr2=0x7f85700c8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:08.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.607+0000 7f856f7fe640 1 --2- 172.21.15.43:0/3818734543 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f857006d4f0 0x7f85700c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.607+0000 7f856f7fe640 1 -- 172.21.15.43:0/3818734543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f857006ef90 msgr2=0x7f857014a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:08.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.607+0000 7f856f7fe640 1 --2- 172.21.15.43:0/3818734543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f857006ef90 0x7f857014a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.607+0000 7f856f7fe640 1 -- 172.21.15.43:0/3818734543 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f857014ab90 con 0x7f857015ddd0 2024-04-25T09:01:08.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.607+0000 7f856f7fe640 1 --2- 172.21.15.43:0/3818734543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f857015ddd0 0x7f85700b2270 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f8560009eb0 tx=0x7f8560004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:08.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.607+0000 7f85677fe640 1 -- 172.21.15.43:0/3818734543 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8560040070 con 0x7f857015ddd0 2024-04-25T09:01:08.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.608+0000 7f85677fe640 1 -- 172.21.15.43:0/3818734543 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8560031bc0 con 0x7f857015ddd0 2024-04-25T09:01:08.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.608+0000 7f85757a5640 1 -- 172.21.15.43:0/3818734543 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85700b11d0 con 0x7f857015ddd0 2024-04-25T09:01:08.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.608+0000 7f85677fe640 1 -- 172.21.15.43:0/3818734543 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8560009450 con 0x7f857015ddd0 2024-04-25T09:01:08.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.608+0000 7f85757a5640 1 -- 172.21.15.43:0/3818734543 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85700b1790 con 0x7f857015ddd0 2024-04-25T09:01:08.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.609+0000 7f85677fe640 1 -- 172.21.15.43:0/3818734543 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8560039040 con 0x7f857015ddd0 2024-04-25T09:01:08.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.609+0000 7f85677fe640 1 --2- 172.21.15.43:0/3818734543 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f855006b4d0 0x7f855006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:08.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.610+0000 7f85677fe640 1 -- 172.21.15.43:0/3818734543 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f85600ab960 con 0x7f857015ddd0 2024-04-25T09:01:08.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.610+0000 7f856effd640 1 --2- 172.21.15.43:0/3818734543 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f855006b4d0 0x7f855006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:08.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.610+0000 7f856effd640 1 --2- 172.21.15.43:0/3818734543 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f855006b4d0 0x7f855006d990 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8568006a90 tx=0x7f8568006a20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:08.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.611+0000 7f8578586740 1 Processor -- start 2024-04-25T09:01:08.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.611+0000 7f8578586740 1 -- start start 2024-04-25T09:01:08.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.611+0000 7f8578586740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d505d7b10 0x560d505da320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:08.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.611+0000 7f8578586740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x560d505d7ef0 0x560d505da860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:08.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.611+0000 7f8578586740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d505aa770 0x560d505adf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:08.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.612+0000 7f8578586740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d50513da0 con 0x560d505d7b10 2024-04-25T09:01:08.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.612+0000 7f8578586740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d504bcb80 con 0x560d505aa770 2024-04-25T09:01:08.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.612+0000 7f8578586740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x560d5050fe40 con 0x560d505d7ef0 2024-04-25T09:01:08.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.612+0000 7f856effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d505d7b10 0x560d505da320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:08.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.612+0000 7f856effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d505d7b10 0x560d505da320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39396/0 (socket says 172.21.15.43:39396) 2024-04-25T09:01:08.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.612+0000 7f856effd640 1 -- 172.21.15.43:0/4132769775 learned_addr learned my addr 172.21.15.43:0/4132769775 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:08.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.612+0000 7f856f7fe640 1 --2- 172.21.15.43:0/4132769775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d505aa770 0x560d505adf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:08.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.612+0000 7f856e7fc640 1 --2- 172.21.15.43:0/4132769775 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x560d505d7ef0 0x560d505da860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.612+0000 7f856effd640 1 -- 172.21.15.43:0/4132769775 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x560d505d7ef0 msgr2=0x560d505da860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.612+0000 7f856effd640 1 --2- 172.21.15.43:0/4132769775 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x560d505d7ef0 0x560d505da860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.612+0000 7f856effd640 1 -- 172.21.15.43:0/4132769775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d505aa770 msgr2=0x560d505adf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.612+0000 7f856effd640 1 --2- 172.21.15.43:0/4132769775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d505aa770 0x560d505adf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.612+0000 7f856effd640 1 -- 172.21.15.43:0/4132769775 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d504c6f10 con 0x560d505d7b10 2024-04-25T09:01:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.612+0000 7f856effd640 1 --2- 172.21.15.43:0/4132769775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d505d7b10 0x560d505da320 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f856800eb30 tx=0x7f8568011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7189253dffb53aa0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.612+0000 7f8564ff9640 1 -- 172.21.15.43:0/4132769775 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856801b040 con 0x560d505d7b10 2024-04-25T09:01:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.612+0000 7f8564ff9640 1 -- 172.21.15.43:0/4132769775 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8568024070 con 0x560d505d7b10 2024-04-25T09:01:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.612+0000 7f8564ff9640 1 -- 172.21.15.43:0/4132769775 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8568015ce0 con 0x560d505d7b10 2024-04-25T09:01:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.613+0000 7f8578586740 1 -- 172.21.15.43:0/4132769775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d505d7b10 msgr2=0x560d505da320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.613+0000 7f8578586740 1 --2- 172.21.15.43:0/4132769775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d505d7b10 0x560d505da320 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f856800eb30 tx=0x7f8568011000 comp rx=0 tx=0).stop 2024-04-25T09:01:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.613+0000 7f8578586740 1 -- 172.21.15.43:0/4132769775 shutdown_connections 2024-04-25T09:01:08.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.613+0000 7f8578586740 1 --2- 172.21.15.43:0/4132769775 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d505aa770 0x560d505adf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.613+0000 7f8578586740 1 --2- 172.21.15.43:0/4132769775 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x560d505d7ef0 0x560d505da860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.613+0000 7f8578586740 1 --2- 172.21.15.43:0/4132769775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d505d7b10 0x560d505da320 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.613+0000 7f8578586740 1 -- 172.21.15.43:0/4132769775 >> 172.21.15.43:0/4132769775 conn(0x560d505ac9e0 msgr2=0x560d505f5320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:08.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.613+0000 7f8578586740 1 -- 172.21.15.43:0/4132769775 shutdown_connections 2024-04-25T09:01:08.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.613+0000 7f8578586740 1 -- 172.21.15.43:0/4132769775 wait complete. 2024-04-25T09:01:08.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.614+0000 7f8578586740 1 -- start start 2024-04-25T09:01:08.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.614+0000 7f8578586740 1 Processor -- start 2024-04-25T09:01:08.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.615+0000 7f8578586740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d505d7b10 0x560d505dc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:08.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.615+0000 7f8578586740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x560d505d7ef0 0x560d505bba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:08.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.615+0000 7f8578586740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d505bbf40 0x560d505a9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:08.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.615+0000 7f8578586740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d505afb90 con 0x560d505bbf40 2024-04-25T09:01:08.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.615+0000 7f8578586740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d504b24b0 con 0x560d505d7b10 2024-04-25T09:01:08.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.615+0000 7f8578586740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x560d505afe70 con 0x560d505d7ef0 2024-04-25T09:01:08.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.615+0000 7f8578586740 10 client.0 _ll_register_callbacks cb 0x560d504f2180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:01:08.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.615+0000 7f856f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d505bbf40 0x560d505a9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:08.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.615+0000 7f856f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d505bbf40 0x560d505a9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39404/0 (socket says 172.21.15.43:39404) 2024-04-25T09:01:08.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.615+0000 7f856f7fe640 1 -- 172.21.15.43:0/4049207843 learned_addr learned my addr 172.21.15.43:0/4049207843 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:08.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.615+0000 7f856e7fc640 1 --2- 172.21.15.43:0/4049207843 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x560d505d7ef0 0x560d505bba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:08.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.615+0000 7f856effd640 1 --2- 172.21.15.43:0/4049207843 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d505d7b10 0x560d505dc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:08.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.615+0000 7f856f7fe640 1 -- 172.21.15.43:0/4049207843 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x560d505d7ef0 msgr2=0x560d505bba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:08.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.615+0000 7f856f7fe640 1 --2- 172.21.15.43:0/4049207843 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x560d505d7ef0 0x560d505bba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.615+0000 7f856f7fe640 1 -- 172.21.15.43:0/4049207843 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d505d7b10 msgr2=0x560d505dc750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:08.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.615+0000 7f856f7fe640 1 --2- 172.21.15.43:0/4049207843 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d505d7b10 0x560d505dc750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.615+0000 7f856f7fe640 1 -- 172.21.15.43:0/4049207843 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560d502f27d0 con 0x560d505bbf40 2024-04-25T09:01:08.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.615+0000 7f856effd640 1 --2- 172.21.15.43:0/4049207843 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d505d7b10 0x560d505dc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:01:08.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.615+0000 7f856e7fc640 1 --2- 172.21.15.43:0/4049207843 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x560d505d7ef0 0x560d505bba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:01:08.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.615+0000 7f856f7fe640 1 --2- 172.21.15.43:0/4049207843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d505bbf40 0x560d505a9a60 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f8560009d90 tx=0x7f856007a330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=751211f1966984a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:08.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.615+0000 7f854dffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:01:08.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.615+0000 7f854dffb640 1 -- 172.21.15.43:0/4049207843 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8560007840 con 0x560d505bbf40 2024-04-25T09:01:08.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.615+0000 7f854dffb640 1 -- 172.21.15.43:0/4049207843 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f8560078570 con 0x560d505bbf40 2024-04-25T09:01:08.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.615+0000 7f8578586740 1 -- 172.21.15.43:0/4049207843 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8568024070 con 0x560d505bbf40 2024-04-25T09:01:08.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.615+0000 7f854dffb640 1 -- 172.21.15.43:0/4049207843 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f85600768e0 con 0x560d505bbf40 2024-04-25T09:01:08.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.616+0000 7f8578586740 10 client.7824 fetch_fsmap learned FSMap version 54 2024-04-25T09:01:08.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.616+0000 7f8578586740 1 -- 172.21.15.43:0/4049207843 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x560d502283f0 con 0x560d505bbf40 2024-04-25T09:01:08.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.616+0000 7f854dffb640 1 -- 172.21.15.43:0/4049207843 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8568024070 con 0x560d505bbf40 2024-04-25T09:01:08.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.616+0000 7f854dffb640 1 -- 172.21.15.43:0/4049207843 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 54) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f856004d030 con 0x560d505bbf40 2024-04-25T09:01:08.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.616+0000 7f8578586740 10 client.7824 fetch_fsmap finished waiting for FSMap version 54 2024-04-25T09:01:08.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.616+0000 7f8578586740 10 client.7824 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T09:01:08.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.616+0000 7f8578586740 20 client.7824 populate_metadata read hostname 'smithi043' 2024-04-25T09:01:08.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.616+0000 7f8578586740 1 --2- 172.21.15.43:0/4049207843 >> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] conn(0x560d505cd9a0 0x560d505cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:08.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.616+0000 7f8578586740 4 client.7824 mds_command: new command op to 7596 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:01:08.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.616+0000 7f8578586740 1 -- 172.21.15.43:0/4049207843 --> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560d505212e0 con 0x560d505cd9a0 2024-04-25T09:01:08.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.616+0000 7f856effd640 1 --2- 172.21.15.43:0/4049207843 >> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] conn(0x560d505cd9a0 0x560d505cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:08.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.616+0000 7f854dffb640 10 client.7824 ms_handle_connect on v2:172.21.15.43:6833/1341028696 2024-04-25T09:01:08.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.616+0000 7f856effd640 1 --2- 172.21.15.43:0/4049207843 >> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] conn(0x560d505cd9a0 0x560d505cddb0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c92d30fb4723f866 server_cookie=2aa4fe5ec1dcd39 in_seq=0 out_seq=0 2024-04-25T09:01:08.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.617+0000 7f854dffb640 1 -- 172.21.15.43:0/4049207843 <== mds.0 v2:172.21.15.43:6833/1341028696 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560d505212e0 con 0x560d505cd9a0 2024-04-25T09:01:08.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.617+0000 7f854dffb640 10 client.7824 handle_command_reply: tid=0 2024-04-25T09:01:08.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.617+0000 7f8578586740 1 client.7824 shutdown 2024-04-25T09:01:08.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.617+0000 7f8578586740 1 -- 172.21.15.43:0/4049207843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d505bbf40 msgr2=0x560d505a9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:08.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.617+0000 7f8578586740 1 --2- 172.21.15.43:0/4049207843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d505bbf40 0x560d505a9a60 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f8560009d90 tx=0x7f856007a330 comp rx=0 tx=0).stop 2024-04-25T09:01:08.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.617+0000 7f8578586740 1 -- 172.21.15.43:0/4049207843 shutdown_connections 2024-04-25T09:01:08.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.617+0000 7f8578586740 1 --2- 172.21.15.43:0/4049207843 >> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] conn(0x560d505cd9a0 0x560d505cddb0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.617+0000 7f8578586740 1 --2- 172.21.15.43:0/4049207843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d505bbf40 0x560d505a9a60 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.617+0000 7f8578586740 1 --2- 172.21.15.43:0/4049207843 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x560d505d7ef0 0x560d505bba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.617+0000 7f8578586740 1 --2- 172.21.15.43:0/4049207843 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d505d7b10 0x560d505dc750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.617+0000 7f854dffb640 0 client.7824 ms_handle_reset on v2:172.21.15.43:6833/1341028696 2024-04-25T09:01:08.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.617+0000 7f8578586740 1 -- 172.21.15.43:0/4049207843 >> 172.21.15.43:0/4049207843 conn(0x560d505aa770 msgr2=0x560d505ab810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:08.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.617+0000 7f8578586740 1 -- 172.21.15.43:0/4049207843 shutdown_connections 2024-04-25T09:01:08.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.617+0000 7f8578586740 1 -- 172.21.15.43:0/4049207843 wait complete. 2024-04-25T09:01:08.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.617+0000 7f8578586740 20 client.7824 trim_cache size 0 max 16384 2024-04-25T09:01:08.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.633+0000 7f8578586740 1 Processor -- start 2024-04-25T09:01:08.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.633+0000 7f8578586740 1 -- start start 2024-04-25T09:01:08.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.633+0000 7f8578586740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d505bbf40 0x560d505afda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:08.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.633+0000 7f8578586740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x560d505d7b10 0x560d505dd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:08.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.633+0000 7f8578586740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d505d7ef0 0x560d505bb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:08.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.633+0000 7f8578586740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85701ffd40 con 0x560d505bbf40 2024-04-25T09:01:08.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.633+0000 7f8578586740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85700c73f0 con 0x560d505d7ef0 2024-04-25T09:01:08.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.633+0000 7f8578586740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85700c67f0 con 0x560d505d7b10 2024-04-25T09:01:08.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.634+0000 7f856effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d505bbf40 0x560d505afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:08.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.634+0000 7f856effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d505bbf40 0x560d505afda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39416/0 (socket says 172.21.15.43:39416) 2024-04-25T09:01:08.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.634+0000 7f856effd640 1 -- 172.21.15.43:0/1781534789 learned_addr learned my addr 172.21.15.43:0/1781534789 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:08.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.634+0000 7f856effd640 1 -- 172.21.15.43:0/1781534789 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x560d505d7b10 msgr2=0x560d505dd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:08.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.634+0000 7f856effd640 1 --2- 172.21.15.43:0/1781534789 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x560d505d7b10 0x560d505dd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.634+0000 7f856effd640 1 -- 172.21.15.43:0/1781534789 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d505d7ef0 msgr2=0x560d505bb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:08.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.634+0000 7f856effd640 1 --2- 172.21.15.43:0/1781534789 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d505d7ef0 0x560d505bb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.634+0000 7f856effd640 1 -- 172.21.15.43:0/1781534789 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560d501e80f0 con 0x560d505bbf40 2024-04-25T09:01:08.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.634+0000 7f856effd640 1 --2- 172.21.15.43:0/1781534789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d505bbf40 0x560d505afda0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f85680097b0 tx=0x7f856801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3914f0b03738a25f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:08.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.634+0000 7f854ffff640 1 -- 172.21.15.43:0/1781534789 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8568031330 con 0x560d505bbf40 2024-04-25T09:01:08.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.634+0000 7f854ffff640 1 -- 172.21.15.43:0/1781534789 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8568037070 con 0x560d505bbf40 2024-04-25T09:01:08.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.634+0000 7f854ffff640 1 -- 172.21.15.43:0/1781534789 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8568031710 con 0x560d505bbf40 2024-04-25T09:01:08.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.635+0000 7f8578586740 1 -- 172.21.15.43:0/1781534789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d505bbf40 msgr2=0x560d505afda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:08.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.635+0000 7f8578586740 1 --2- 172.21.15.43:0/1781534789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d505bbf40 0x560d505afda0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f85680097b0 tx=0x7f856801d000 comp rx=0 tx=0).stop 2024-04-25T09:01:08.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.635+0000 7f8578586740 1 -- 172.21.15.43:0/1781534789 shutdown_connections 2024-04-25T09:01:08.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.635+0000 7f8578586740 1 --2- 172.21.15.43:0/1781534789 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d505d7ef0 0x560d505bb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.635+0000 7f8578586740 1 --2- 172.21.15.43:0/1781534789 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x560d505d7b10 0x560d505dd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.635+0000 7f8578586740 1 --2- 172.21.15.43:0/1781534789 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d505bbf40 0x560d505afda0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.635+0000 7f8578586740 1 -- 172.21.15.43:0/1781534789 >> 172.21.15.43:0/1781534789 conn(0x560d505aa770 msgr2=0x560d505d5680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:08.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.635+0000 7f8578586740 1 -- 172.21.15.43:0/1781534789 shutdown_connections 2024-04-25T09:01:08.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.635+0000 7f8578586740 1 -- 172.21.15.43:0/1781534789 wait complete. 2024-04-25T09:01:08.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.635+0000 7f8578586740 1 -- start start 2024-04-25T09:01:08.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.635+0000 7f8578586740 1 Processor -- start 2024-04-25T09:01:08.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.636+0000 7f8578586740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x560d505bbf40 0x560d5065f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:08.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.636+0000 7f8578586740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d505d7b10 0x560d505d67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:08.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.636+0000 7f8578586740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d505d7ef0 0x560d505d6d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:08.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.636+0000 7f8578586740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85700c6df0 con 0x560d505d7b10 2024-04-25T09:01:08.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.636+0000 7f8578586740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x560d505f2710 con 0x560d505d7ef0 2024-04-25T09:01:08.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.636+0000 7f8578586740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x560d506634b0 con 0x560d505bbf40 2024-04-25T09:01:08.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.636+0000 7f8578586740 10 client.0 _ll_register_callbacks cb 0x7f8568009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:01:08.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.636+0000 7f856e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d505d7b10 0x560d505d67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:08.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.636+0000 7f856e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d505d7b10 0x560d505d67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39418/0 (socket says 172.21.15.43:39418) 2024-04-25T09:01:08.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.636+0000 7f856e7fc640 1 -- 172.21.15.43:0/3721148700 learned_addr learned my addr 172.21.15.43:0/3721148700 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:08.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.636+0000 7f856e7fc640 1 -- 172.21.15.43:0/3721148700 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x560d505bbf40 msgr2=0x560d5065f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:08.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.636+0000 7f856e7fc640 1 --2- 172.21.15.43:0/3721148700 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x560d505bbf40 0x560d5065f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.636+0000 7f856e7fc640 1 -- 172.21.15.43:0/3721148700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d505d7ef0 msgr2=0x560d505d6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:08.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.636+0000 7f856e7fc640 1 --2- 172.21.15.43:0/3721148700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d505d7ef0 0x560d505d6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.636+0000 7f856e7fc640 1 -- 172.21.15.43:0/3721148700 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560d505da440 con 0x560d505d7b10 2024-04-25T09:01:08.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.637+0000 7f856e7fc640 1 --2- 172.21.15.43:0/3721148700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d505d7b10 0x560d505d67c0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f85700af030 tx=0x7f8558010610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ba1715439c1895 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:08.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.637+0000 7f854dffb640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:01:08.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.637+0000 7f854dffb640 1 -- 172.21.15.43:0/3721148700 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8558005a50 con 0x560d505d7b10 2024-04-25T09:01:08.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.637+0000 7f854dffb640 1 -- 172.21.15.43:0/3721148700 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f8558005bf0 con 0x560d505d7b10 2024-04-25T09:01:08.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.637+0000 7f8578586740 1 -- 172.21.15.43:0/3721148700 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8568037070 con 0x560d505d7b10 2024-04-25T09:01:08.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.637+0000 7f854dffb640 1 -- 172.21.15.43:0/3721148700 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8558021050 con 0x560d505d7b10 2024-04-25T09:01:08.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.637+0000 7f8578586740 10 client.7830 fetch_fsmap learned FSMap version 54 2024-04-25T09:01:08.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.637+0000 7f8578586740 1 -- 172.21.15.43:0/3721148700 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x560d506631f0 con 0x560d505d7b10 2024-04-25T09:01:08.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.637+0000 7f854dffb640 1 -- 172.21.15.43:0/3721148700 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8568037070 con 0x560d505d7b10 2024-04-25T09:01:08.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.637+0000 7f854dffb640 1 -- 172.21.15.43:0/3721148700 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 54) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8558024020 con 0x560d505d7b10 2024-04-25T09:01:08.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.637+0000 7f8578586740 10 client.7830 fetch_fsmap finished waiting for FSMap version 54 2024-04-25T09:01:08.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.637+0000 7f8578586740 10 client.7830 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T09:01:08.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.637+0000 7f8578586740 20 client.7830 populate_metadata read hostname 'smithi043' 2024-04-25T09:01:08.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.637+0000 7f8578586740 1 --2- 172.21.15.43:0/3721148700 >> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] conn(0x560d50617fd0 0x560d505eae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:08.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.637+0000 7f8578586740 4 client.7830 mds_command: new command op to 7596 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:01:08.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.637+0000 7f8578586740 1 -- 172.21.15.43:0/3721148700 --> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560d5051b630 con 0x560d50617fd0 2024-04-25T09:01:08.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.637+0000 7f856effd640 1 --2- 172.21.15.43:0/3721148700 >> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] conn(0x560d50617fd0 0x560d505eae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:08.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.638+0000 7f856effd640 1 --2- 172.21.15.43:0/3721148700 >> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] conn(0x560d50617fd0 0x560d505eae60 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e7b65fc34dd7a79 server_cookie=f79d0f94f266626 in_seq=0 out_seq=0 2024-04-25T09:01:08.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.638+0000 7f854dffb640 10 client.7830 ms_handle_connect on v2:172.21.15.43:6833/1341028696 2024-04-25T09:01:08.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.638+0000 7f854dffb640 1 -- 172.21.15.43:0/3721148700 <== mds.0 v2:172.21.15.43:6833/1341028696 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560d5051b630 con 0x560d50617fd0 2024-04-25T09:01:08.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.638+0000 7f854dffb640 10 client.7830 handle_command_reply: tid=0 2024-04-25T09:01:08.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.638+0000 7f8578586740 1 client.7830 shutdown 2024-04-25T09:01:08.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.638+0000 7f8578586740 1 -- 172.21.15.43:0/3721148700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d505d7b10 msgr2=0x560d505d67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:08.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.638+0000 7f8578586740 1 --2- 172.21.15.43:0/3721148700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d505d7b10 0x560d505d67c0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f85700af030 tx=0x7f8558010610 comp rx=0 tx=0).stop 2024-04-25T09:01:08.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.638+0000 7f8578586740 1 -- 172.21.15.43:0/3721148700 shutdown_connections 2024-04-25T09:01:08.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.638+0000 7f8578586740 1 --2- 172.21.15.43:0/3721148700 >> [v2:172.21.15.43:6833/1341028696,v1:172.21.15.43:6835/1341028696] conn(0x560d50617fd0 0x560d505eae60 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.638+0000 7f8578586740 1 --2- 172.21.15.43:0/3721148700 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560d505d7ef0 0x560d505d6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.638+0000 7f8578586740 1 --2- 172.21.15.43:0/3721148700 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x560d505d7b10 0x560d505d67c0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.638+0000 7f854dffb640 0 client.7830 ms_handle_reset on v2:172.21.15.43:6833/1341028696 2024-04-25T09:01:08.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.638+0000 7f8578586740 1 --2- 172.21.15.43:0/3721148700 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x560d505bbf40 0x560d5065f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.638+0000 7f8578586740 1 -- 172.21.15.43:0/3721148700 >> 172.21.15.43:0/3721148700 conn(0x560d505aa770 msgr2=0x560d505ae3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:08.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.638+0000 7f8578586740 1 -- 172.21.15.43:0/3721148700 shutdown_connections 2024-04-25T09:01:08.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.638+0000 7f8578586740 1 -- 172.21.15.43:0/3721148700 wait complete. 2024-04-25T09:01:08.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.638+0000 7f8578586740 20 client.7830 trim_cache size 0 max 16384 2024-04-25T09:01:08.634 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:01:08.634 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-25T09:01:08.634 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-25T09:01:08.634 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:01:08.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.639+0000 7f85757a5640 1 -- 172.21.15.43:0/3818734543 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f855006b4d0 msgr2=0x7f855006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:08.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.639+0000 7f85757a5640 1 --2- 172.21.15.43:0/3818734543 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f855006b4d0 0x7f855006d990 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8568006a90 tx=0x7f8568006a20 comp rx=0 tx=0).stop 2024-04-25T09:01:08.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.639+0000 7f85757a5640 1 -- 172.21.15.43:0/3818734543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f857015ddd0 msgr2=0x7f85700b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:08.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.639+0000 7f85757a5640 1 --2- 172.21.15.43:0/3818734543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f857015ddd0 0x7f85700b2270 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f8560009eb0 tx=0x7f8560004a40 comp rx=0 tx=0).stop 2024-04-25T09:01:08.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.639+0000 7f85757a5640 1 -- 172.21.15.43:0/3818734543 shutdown_connections 2024-04-25T09:01:08.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.639+0000 7f85757a5640 1 --2- 172.21.15.43:0/3818734543 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f857015ddd0 0x7f85700b2270 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.639+0000 7f85757a5640 1 --2- 172.21.15.43:0/3818734543 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f855006b4d0 0x7f855006d990 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.639+0000 7f85757a5640 1 --2- 172.21.15.43:0/3818734543 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f857006ef90 0x7f857014a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.639+0000 7f85757a5640 1 --2- 172.21.15.43:0/3818734543 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f857006d4f0 0x7f85700c8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.639+0000 7f85757a5640 1 -- 172.21.15.43:0/3818734543 >> 172.21.15.43:0/3818734543 conn(0x7f85700d17e0 msgr2=0x7f857015c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:08.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.639+0000 7f85757a5640 1 -- 172.21.15.43:0/3818734543 shutdown_connections 2024-04-25T09:01:08.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.639+0000 7f85757a5640 1 -- 172.21.15.43:0/3818734543 wait complete. 2024-04-25T09:01:08.638 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T09:01:08.639 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:01:08.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.753+0000 7f5af03c8640 1 Processor -- start 2024-04-25T09:01:08.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.753+0000 7f5af03c8640 1 -- start start 2024-04-25T09:01:08.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.753+0000 7f5af03c8640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f5ae815a9d0 0x7f5ae806d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:08.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.754+0000 7f5af03c8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5ae806d7d0 0x7f5ae8071a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:08.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.754+0000 7f5af03c8640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5ae8071f90 0x7f5ae80b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:08.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.754+0000 7f5af03c8640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ae806dc30 con 0x7f5ae806d7d0 2024-04-25T09:01:08.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.754+0000 7f5af03c8640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ae8072420 con 0x7f5ae815a9d0 2024-04-25T09:01:08.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.754+0000 7f5af03c8640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae80704f0 con 0x7f5ae8071f90 2024-04-25T09:01:08.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.754+0000 7f5aed963640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5ae806d7d0 0x7f5ae8071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:08.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.754+0000 7f5aed963640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5ae806d7d0 0x7f5ae8071a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39430/0 (socket says 172.21.15.43:39430) 2024-04-25T09:01:08.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.754+0000 7f5aed963640 1 -- 172.21.15.43:0/4293104971 learned_addr learned my addr 172.21.15.43:0/4293104971 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:08.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.754+0000 7f5aee164640 1 --2- 172.21.15.43:0/4293104971 >> v2:172.21.15.193:3300/0 conn(0x7f5ae815a9d0 0x7f5ae806d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:08.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.754+0000 7f5aee965640 1 --2- 172.21.15.43:0/4293104971 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5ae8071f90 0x7f5ae80b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:08.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.754+0000 7f5aed963640 1 -- 172.21.15.43:0/4293104971 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5ae8071f90 msgr2=0x7f5ae80b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:08.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.754+0000 7f5aed963640 1 --2- 172.21.15.43:0/4293104971 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5ae8071f90 0x7f5ae80b3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.754+0000 7f5aed963640 1 -- 172.21.15.43:0/4293104971 >> v2:172.21.15.193:3300/0 conn(0x7f5ae815a9d0 msgr2=0x7f5ae806d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:08.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.754+0000 7f5aed963640 1 --2- 172.21.15.43:0/4293104971 >> v2:172.21.15.193:3300/0 conn(0x7f5ae815a9d0 0x7f5ae806d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.754+0000 7f5aed963640 1 -- 172.21.15.43:0/4293104971 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae80707d0 con 0x7f5ae806d7d0 2024-04-25T09:01:08.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.754+0000 7f5aee965640 1 --2- 172.21.15.43:0/4293104971 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5ae8071f90 0x7f5ae80b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:01:08.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.755+0000 7f5aed963640 1 --2- 172.21.15.43:0/4293104971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ae806d7d0 0x7f5ae8071a50 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f5ae806c190 tx=0x7f5ad802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6e2cedaed6ca517 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:08.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.755+0000 7f5aed162640 1 -- 172.21.15.43:0/4293104971 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8037070 con 0x7f5ae806d7d0 2024-04-25T09:01:08.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.755+0000 7f5aed162640 1 -- 172.21.15.43:0/4293104971 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad8032440 con 0x7f5ae806d7d0 2024-04-25T09:01:08.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.755+0000 7f5aed162640 1 -- 172.21.15.43:0/4293104971 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8032810 con 0x7f5ae806d7d0 2024-04-25T09:01:08.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.755+0000 7f5af03c8640 1 -- 172.21.15.43:0/4293104971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ae806d7d0 msgr2=0x7f5ae8071a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:08.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.755+0000 7f5af03c8640 1 --2- 172.21.15.43:0/4293104971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ae806d7d0 0x7f5ae8071a50 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f5ae806c190 tx=0x7f5ad802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.756+0000 7f5af03c8640 1 -- 172.21.15.43:0/4293104971 shutdown_connections 2024-04-25T09:01:08.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.756+0000 7f5af03c8640 1 --2- 172.21.15.43:0/4293104971 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5ae8071f90 0x7f5ae80b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.756+0000 7f5af03c8640 1 --2- 172.21.15.43:0/4293104971 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ae806d7d0 0x7f5ae8071a50 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.756+0000 7f5af03c8640 1 --2- 172.21.15.43:0/4293104971 >> v2:172.21.15.193:3300/0 conn(0x7f5ae815a9d0 0x7f5ae806d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.756+0000 7f5af03c8640 1 -- 172.21.15.43:0/4293104971 >> 172.21.15.43:0/4293104971 conn(0x7f5ae80b1480 msgr2=0x7f5ae80b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:08.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.756+0000 7f5af03c8640 1 -- 172.21.15.43:0/4293104971 shutdown_connections 2024-04-25T09:01:08.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.756+0000 7f5af03c8640 1 -- 172.21.15.43:0/4293104971 wait complete. 2024-04-25T09:01:08.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.757+0000 7f5af03c8640 1 Processor -- start 2024-04-25T09:01:08.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.757+0000 7f5af03c8640 1 -- start start 2024-04-25T09:01:08.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.757+0000 7f5af03c8640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5ae806d7d0 0x7f5ae814c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:08.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.757+0000 7f5af03c8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ae8071f90 0x7f5ae80a9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:08.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.757+0000 7f5af03c8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ae80a9ea0 0x7f5ae8159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:08.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.758+0000 7f5af03c8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae80712b0 con 0x7f5ae8071f90 2024-04-25T09:01:08.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.758+0000 7f5af03c8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae8070d90 con 0x7f5ae80a9ea0 2024-04-25T09:01:08.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.758+0000 7f5af03c8640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae80b3ed0 con 0x7f5ae806d7d0 2024-04-25T09:01:08.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.758+0000 7f5aed963640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ae8071f90 0x7f5ae80a9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:08.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.758+0000 7f5aed963640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ae8071f90 0x7f5ae80a9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39440/0 (socket says 172.21.15.43:39440) 2024-04-25T09:01:08.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.758+0000 7f5aed963640 1 -- 172.21.15.43:0/3856547907 learned_addr learned my addr 172.21.15.43:0/3856547907 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:08.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.758+0000 7f5aee164640 1 --2- 172.21.15.43:0/3856547907 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5ae806d7d0 0x7f5ae814c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:08.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.758+0000 7f5aee965640 1 --2- 172.21.15.43:0/3856547907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ae80a9ea0 0x7f5ae8159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:08.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.758+0000 7f5aed963640 1 -- 172.21.15.43:0/3856547907 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5ae806d7d0 msgr2=0x7f5ae814c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:08.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.758+0000 7f5aed963640 1 --2- 172.21.15.43:0/3856547907 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5ae806d7d0 0x7f5ae814c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.758+0000 7f5aed963640 1 -- 172.21.15.43:0/3856547907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ae80a9ea0 msgr2=0x7f5ae8159000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:08.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.758+0000 7f5aed963640 1 --2- 172.21.15.43:0/3856547907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ae80a9ea0 0x7f5ae8159000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.758+0000 7f5aed963640 1 -- 172.21.15.43:0/3856547907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae81537b0 con 0x7f5ae8071f90 2024-04-25T09:01:08.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.758+0000 7f5aee965640 1 --2- 172.21.15.43:0/3856547907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ae80a9ea0 0x7f5ae8159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:01:08.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.758+0000 7f5aed963640 1 --2- 172.21.15.43:0/3856547907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ae8071f90 0x7f5ae80a9960 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8000c00 tx=0x7f5ad80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:08.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.758+0000 7f5adeffd640 1 -- 172.21.15.43:0/3856547907 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8041030 con 0x7f5ae8071f90 2024-04-25T09:01:08.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.758+0000 7f5adeffd640 1 -- 172.21.15.43:0/3856547907 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad8030440 con 0x7f5ae8071f90 2024-04-25T09:01:08.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.758+0000 7f5af03c8640 1 -- 172.21.15.43:0/3856547907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ae8153a90 con 0x7f5ae8071f90 2024-04-25T09:01:08.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.758+0000 7f5adeffd640 1 -- 172.21.15.43:0/3856547907 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8037040 con 0x7f5ae8071f90 2024-04-25T09:01:08.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.758+0000 7f5af03c8640 1 -- 172.21.15.43:0/3856547907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae8154050 con 0x7f5ae8071f90 2024-04-25T09:01:08.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.759+0000 7f5af03c8640 1 -- 172.21.15.43:0/3856547907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ae815dd80 con 0x7f5ae8071f90 2024-04-25T09:01:08.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.760+0000 7f5adeffd640 1 -- 172.21.15.43:0/3856547907 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5ad8044070 con 0x7f5ae8071f90 2024-04-25T09:01:08.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.760+0000 7f5adeffd640 1 --2- 172.21.15.43:0/3856547907 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5acc06b4d0 0x7f5acc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:08.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.760+0000 7f5adeffd640 1 -- 172.21.15.43:0/3856547907 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f5ad80aa9d0 con 0x7f5ae8071f90 2024-04-25T09:01:08.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.761+0000 7f5aee164640 1 --2- 172.21.15.43:0/3856547907 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5acc06b4d0 0x7f5acc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:08.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.761+0000 7f5aee164640 1 --2- 172.21.15.43:0/3856547907 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5acc06b4d0 0x7f5acc06d990 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0009a20 tx=0x7f5ae0009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:08.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.765+0000 7f5adeffd640 1 -- 172.21.15.43:0/3856547907 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5ae815dd80 con 0x7f5ae8071f90 2024-04-25T09:01:08.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.981+0000 7f5af03c8640 1 -- 172.21.15.43:0/3856547907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ae8074ef0 con 0x7f5ae8071f90 2024-04-25T09:01:08.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.981+0000 7f5adeffd640 1 -- 172.21.15.43:0/3856547907 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f5ad80798c0 con 0x7f5ae8071f90 2024-04-25T09:01:08.970 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:01:08.971 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":54,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6200,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2258031526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2258031526},{"type":"v1","addr":"172.21.15.43:6834","nonce":2258031526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6206,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/1773440177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1773440177},{"type":"v1","addr":"172.21.15.193:6836","nonce":1773440177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6209,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3239725307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":3239725307},{"type":"v1","addr":"172.21.15.193:6837","nonce":3239725307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:00:42.536578+0000","modified":"2024-04-25T09:00:44.531104+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7596},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7596":{"gid":7596,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/1341028696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1341028696},{"type":"v1","addr":"172.21.15.43:6835","nonce":1341028696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:01:08.971 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 54 2024-04-25T09:01:08.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.983+0000 7f5af03c8640 1 -- 172.21.15.43:0/3856547907 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5acc06b4d0 msgr2=0x7f5acc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:08.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.983+0000 7f5af03c8640 1 --2- 172.21.15.43:0/3856547907 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5acc06b4d0 0x7f5acc06d990 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0009a20 tx=0x7f5ae0009190 comp rx=0 tx=0).stop 2024-04-25T09:01:08.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.983+0000 7f5af03c8640 1 -- 172.21.15.43:0/3856547907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ae8071f90 msgr2=0x7f5ae80a9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:08.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.983+0000 7f5af03c8640 1 --2- 172.21.15.43:0/3856547907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ae8071f90 0x7f5ae80a9960 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8000c00 tx=0x7f5ad80046c0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.983+0000 7f5af03c8640 1 -- 172.21.15.43:0/3856547907 shutdown_connections 2024-04-25T09:01:08.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.983+0000 7f5af03c8640 1 --2- 172.21.15.43:0/3856547907 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5acc06b4d0 0x7f5acc06d990 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.983+0000 7f5af03c8640 1 --2- 172.21.15.43:0/3856547907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5ae80a9ea0 0x7f5ae8159000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.983+0000 7f5af03c8640 1 --2- 172.21.15.43:0/3856547907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5ae8071f90 0x7f5ae80a9960 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.983+0000 7f5af03c8640 1 --2- 172.21.15.43:0/3856547907 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5ae806d7d0 0x7f5ae814c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:08.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.983+0000 7f5af03c8640 1 -- 172.21.15.43:0/3856547907 >> 172.21.15.43:0/3856547907 conn(0x7f5ae80b1480 msgr2=0x7f5ae8155ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:08.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.984+0000 7f5af03c8640 1 -- 172.21.15.43:0/3856547907 shutdown_connections 2024-04-25T09:01:08.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:08.984+0000 7f5af03c8640 1 -- 172.21.15.43:0/3856547907 wait complete. 2024-04-25T09:01:08.983 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T09:01:08.983 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:01:09.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.094+0000 7f33ba25a640 1 Processor -- start 2024-04-25T09:01:09.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.094+0000 7f33ba25a640 1 -- start start 2024-04-25T09:01:09.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.095+0000 7f33ba25a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f33b40cfd70 0x7f33b40d01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:09.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.095+0000 7f33ba25a640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f33b40b0e50 0x7f33b40b12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:09.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.095+0000 7f33ba25a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33b40cc760 con 0x7f33b415be60 2024-04-25T09:01:09.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.095+0000 7f33ba25a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33b415c240 con 0x7f33b40cfd70 2024-04-25T09:01:09.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.095+0000 7f33ba25a640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33b40d0710 con 0x7f33b40b0e50 2024-04-25T09:01:09.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.095+0000 7f33b2ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f33b40cfd70 0x7f33b40d01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:09.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.095+0000 7f33b37fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f33b415be60 0x7f33b40ce880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:59312/0 (socket says 172.21.15.43:59312) 2024-04-25T09:01:09.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.095+0000 7f33b37fe640 1 -- 172.21.15.43:0/3105478921 learned_addr learned my addr 172.21.15.43:0/3105478921 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:09.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.095+0000 7f33b3fff640 1 --2- 172.21.15.43:0/3105478921 >> v2:172.21.15.193:3300/0 conn(0x7f33b40b0e50 0x7f33b40b12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:09.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.095+0000 7f33b27fc640 1 -- 172.21.15.43:0/3105478921 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 870315029 0 0) 0x7f33b40cc760 con 0x7f33b415be60 2024-04-25T09:01:09.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.096+0000 7f33b27fc640 1 -- 172.21.15.43:0/3105478921 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f339c003520 con 0x7f33b415be60 2024-04-25T09:01:09.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.096+0000 7f33b2ffd640 1 -- 172.21.15.43:0/3105478921 >> v2:172.21.15.193:3300/0 conn(0x7f33b40b0e50 msgr2=0x7f33b40b12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:09.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.096+0000 7f33b2ffd640 1 --2- 172.21.15.43:0/3105478921 >> v2:172.21.15.193:3300/0 conn(0x7f33b40b0e50 0x7f33b40b12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:09.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.096+0000 7f33b2ffd640 1 -- 172.21.15.43:0/3105478921 >> v1:172.21.15.43:6789/0 conn(0x7f33b415be60 legacy=0x7f33b40ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:09.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.096+0000 7f33b2ffd640 1 -- 172.21.15.43:0/3105478921 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b40cca40 con 0x7f33b40cfd70 2024-04-25T09:01:09.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.096+0000 7f33b3fff640 1 --2- 172.21.15.43:0/3105478921 >> v2:172.21.15.193:3300/0 conn(0x7f33b40b0e50 0x7f33b40b12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:01:09.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.096+0000 7f33b2ffd640 1 --2- 172.21.15.43:0/3105478921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33b40cfd70 0x7f33b40d01d0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f33b4071de0 tx=0x7f33a802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2748c169963bc6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:09.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.096+0000 7f33b27fc640 1 -- 172.21.15.43:0/3105478921 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a8037070 con 0x7f33b40cfd70 2024-04-25T09:01:09.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.096+0000 7f33b27fc640 1 -- 172.21.15.43:0/3105478921 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a8032440 con 0x7f33b40cfd70 2024-04-25T09:01:09.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.096+0000 7f33b27fc640 1 -- 172.21.15.43:0/3105478921 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a8032810 con 0x7f33b40cfd70 2024-04-25T09:01:09.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.097+0000 7f33ba25a640 1 -- 172.21.15.43:0/3105478921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33b40cfd70 msgr2=0x7f33b40d01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:09.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.097+0000 7f33ba25a640 1 --2- 172.21.15.43:0/3105478921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33b40cfd70 0x7f33b40d01d0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f33b4071de0 tx=0x7f33a802a1a0 comp rx=0 tx=0).stop 2024-04-25T09:01:09.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.097+0000 7f33ba25a640 1 -- 172.21.15.43:0/3105478921 shutdown_connections 2024-04-25T09:01:09.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.097+0000 7f33ba25a640 1 --2- 172.21.15.43:0/3105478921 >> v2:172.21.15.193:3300/0 conn(0x7f33b40b0e50 0x7f33b40b12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:09.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.097+0000 7f33ba25a640 1 --2- 172.21.15.43:0/3105478921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33b40cfd70 0x7f33b40d01d0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:09.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.097+0000 7f33ba25a640 1 -- 172.21.15.43:0/3105478921 >> 172.21.15.43:0/3105478921 conn(0x7f33b40b32e0 msgr2=0x7f33b40b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:09.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.097+0000 7f33ba25a640 1 -- 172.21.15.43:0/3105478921 shutdown_connections 2024-04-25T09:01:09.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.097+0000 7f33ba25a640 1 -- 172.21.15.43:0/3105478921 wait complete. 2024-04-25T09:01:09.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.098+0000 7f33ba25a640 1 Processor -- start 2024-04-25T09:01:09.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.098+0000 7f33ba25a640 1 -- start start 2024-04-25T09:01:09.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.099+0000 7f33ba25a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33b40b0e50 0x7f33b40afeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:09.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.099+0000 7f33ba25a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f33b40cfd70 0x7f33b40c4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:09.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.099+0000 7f33ba25a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33b415be60 0x7f33b415d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:09.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.099+0000 7f33ba25a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b40cc330 con 0x7f33b415be60 2024-04-25T09:01:09.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.099+0000 7f33ba25a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b40cd000 con 0x7f33b40b0e50 2024-04-25T09:01:09.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.099+0000 7f33ba25a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b40cc4b0 con 0x7f33b40cfd70 2024-04-25T09:01:09.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.099+0000 7f33b3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33b415be60 0x7f33b415d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:09.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.099+0000 7f33b3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33b415be60 0x7f33b415d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:39466/0 (socket says 172.21.15.43:39466) 2024-04-25T09:01:09.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.099+0000 7f33b3fff640 1 -- 172.21.15.43:0/3698942866 learned_addr learned my addr 172.21.15.43:0/3698942866 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:09.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.099+0000 7f33b37fe640 1 --2- 172.21.15.43:0/3698942866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33b40b0e50 0x7f33b40afeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:09.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.099+0000 7f33b2ffd640 1 --2- 172.21.15.43:0/3698942866 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f33b40cfd70 0x7f33b40c4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:09.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.100+0000 7f33b3fff640 1 -- 172.21.15.43:0/3698942866 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f33b40cfd70 msgr2=0x7f33b40c4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:09.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.100+0000 7f33b3fff640 1 --2- 172.21.15.43:0/3698942866 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f33b40cfd70 0x7f33b40c4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:09.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.100+0000 7f33b3fff640 1 -- 172.21.15.43:0/3698942866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33b40b0e50 msgr2=0x7f33b40afeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:09.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.100+0000 7f33b3fff640 1 --2- 172.21.15.43:0/3698942866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33b40b0e50 0x7f33b40afeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:09.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.100+0000 7f33b3fff640 1 -- 172.21.15.43:0/3698942866 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b406c000 con 0x7f33b415be60 2024-04-25T09:01:09.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.100+0000 7f33b3fff640 1 --2- 172.21.15.43:0/3698942866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33b415be60 0x7f33b415d600 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f33ac007df0 tx=0x7f33ac00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:09.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.100+0000 7f339bfff640 1 -- 172.21.15.43:0/3698942866 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac01a070 con 0x7f33b415be60 2024-04-25T09:01:09.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.100+0000 7f339bfff640 1 -- 172.21.15.43:0/3698942866 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33ac003990 con 0x7f33b415be60 2024-04-25T09:01:09.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.100+0000 7f33ba25a640 1 -- 172.21.15.43:0/3698942866 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33b406c2e0 con 0x7f33b415be60 2024-04-25T09:01:09.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.100+0000 7f339bfff640 1 -- 172.21.15.43:0/3698942866 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac003d80 con 0x7f33b415be60 2024-04-25T09:01:09.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.100+0000 7f33ba25a640 1 -- 172.21.15.43:0/3698942866 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33b406c8a0 con 0x7f33b415be60 2024-04-25T09:01:09.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.101+0000 7f33ba25a640 1 -- 172.21.15.43:0/3698942866 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33b4074ef0 con 0x7f33b415be60 2024-04-25T09:01:09.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.102+0000 7f339bfff640 1 -- 172.21.15.43:0/3698942866 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f33ac01d070 con 0x7f33b415be60 2024-04-25T09:01:09.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.102+0000 7f339bfff640 1 --2- 172.21.15.43:0/3698942866 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f33a006b4d0 0x7f33a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:09.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.102+0000 7f339bfff640 1 -- 172.21.15.43:0/3698942866 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f33ac08a410 con 0x7f33b415be60 2024-04-25T09:01:09.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.102+0000 7f33b37fe640 1 --2- 172.21.15.43:0/3698942866 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f33a006b4d0 0x7f33a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:09.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.103+0000 7f33b37fe640 1 --2- 172.21.15.43:0/3698942866 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f33a006b4d0 0x7f33a006d990 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f33a4005880 tx=0x7f33a4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:09.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.106+0000 7f339bfff640 1 -- 172.21.15.43:0/3698942866 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f33ac010070 con 0x7f33b415be60 2024-04-25T09:01:09.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.354+0000 7f33ba25a640 1 -- 172.21.15.43:0/3698942866 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f33b40b30c0 con 0x7f33b415be60 2024-04-25T09:01:09.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.354+0000 7f339bfff640 1 -- 172.21.15.43:0/3698942866 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1802 (secure 0 0 0) 0x7f33ac059300 con 0x7f33b415be60 2024-04-25T09:01:09.344 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:01:09.344 INFO:teuthology.orchestra.run.smithi043.stdout:{"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":441,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":91,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035238,"num_in_osds":8,"osd_in_since":1714035229,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1077452283,"bytes_used":2690105344,"bytes_avail":770404007936,"bytes_total":773094113280,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":436813,"recovering_keys_per_sec":0,"num_objects_recovered":5,"num_bytes_recovered":5242880,"num_keys_recovered":0,"write_bytes_sec":62071980,"read_op_per_sec":0,"write_op_per_sec":61},"fsmap":{"epoch":54,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7596}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":15,"modified":"2024-04-25T09:01:05.788828+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"active paths [/scrub_control_test_path]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:01:09.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.357+0000 7f33ba25a640 1 -- 172.21.15.43:0/3698942866 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f33a006b4d0 msgr2=0x7f33a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:09.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.357+0000 7f33ba25a640 1 --2- 172.21.15.43:0/3698942866 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f33a006b4d0 0x7f33a006d990 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f33a4005880 tx=0x7f33a4009040 comp rx=0 tx=0).stop 2024-04-25T09:01:09.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.357+0000 7f33ba25a640 1 -- 172.21.15.43:0/3698942866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33b415be60 msgr2=0x7f33b415d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:09.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.357+0000 7f33ba25a640 1 --2- 172.21.15.43:0/3698942866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33b415be60 0x7f33b415d600 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f33ac007df0 tx=0x7f33ac00d4a0 comp rx=0 tx=0).stop 2024-04-25T09:01:09.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.357+0000 7f33ba25a640 1 -- 172.21.15.43:0/3698942866 shutdown_connections 2024-04-25T09:01:09.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.357+0000 7f33ba25a640 1 --2- 172.21.15.43:0/3698942866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33b415be60 0x7f33b415d600 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:09.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.357+0000 7f33ba25a640 1 --2- 172.21.15.43:0/3698942866 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f33b40cfd70 0x7f33b40c4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:09.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.357+0000 7f33ba25a640 1 --2- 172.21.15.43:0/3698942866 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f33a006b4d0 0x7f33a006d990 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:09.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.357+0000 7f33ba25a640 1 --2- 172.21.15.43:0/3698942866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33b40b0e50 0x7f33b40afeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:09.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.357+0000 7f33ba25a640 1 -- 172.21.15.43:0/3698942866 >> 172.21.15.43:0/3698942866 conn(0x7f33b40b32e0 msgr2=0x7f33b40ce7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:09.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.357+0000 7f33ba25a640 1 -- 172.21.15.43:0/3698942866 shutdown_connections 2024-04-25T09:01:09.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:09.357+0000 7f33ba25a640 1 -- 172.21.15.43:0/3698942866 wait complete. 2024-04-25T09:01:09.356 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'active paths [/scrub_control_test_path]'} 2024-04-25T09:01:10.357 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:01:10.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.482+0000 7f2df054c640 1 Processor -- start 2024-04-25T09:01:10.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.482+0000 7f2df054c640 1 -- start start 2024-04-25T09:01:10.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.483+0000 7f2df054c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2de815dce0 0x7f2de815ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:10.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.483+0000 7f2df054c640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2de80d2920 con 0x7f2de80d0af0 2024-04-25T09:01:10.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.483+0000 7f2df054c640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2de80cee80 con 0x7f2de80ce550 2024-04-25T09:01:10.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.483+0000 7f2df054c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2de80cf160 con 0x7f2de815dce0 2024-04-25T09:01:10.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.483+0000 7f2dee2e8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2de815dce0 0x7f2de815ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:10.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.483+0000 7f2deeae9640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f2de80d0af0 0x7f2de8160980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47430/0 (socket says 172.21.15.43:47430) 2024-04-25T09:01:10.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.483+0000 7f2deeae9640 1 -- 172.21.15.43:0/2714315388 learned_addr learned my addr 172.21.15.43:0/2714315388 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:10.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.483+0000 7f2dee2e8640 1 -- 172.21.15.43:0/2714315388 >> v1:172.21.15.193:6789/0 conn(0x7f2de80ce550 legacy=0x7f2de80d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:10.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.483+0000 7f2dee2e8640 1 -- 172.21.15.43:0/2714315388 >> v1:172.21.15.43:6789/0 conn(0x7f2de80d0af0 legacy=0x7f2de8160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:10.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.483+0000 7f2dee2e8640 1 -- 172.21.15.43:0/2714315388 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de80cf440 con 0x7f2de815dce0 2024-04-25T09:01:10.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.483+0000 7f2ded2e6640 1 -- 172.21.15.43:0/2714315388 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2507194246 0 0) 0x7f2de80d2920 con 0x7f2de80d0af0 2024-04-25T09:01:10.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.484+0000 7f2dee2e8640 1 --2- 172.21.15.43:0/2714315388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2de815dce0 0x7f2de815ef50 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f2de40097a0 tx=0x7f2de402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=530f79ededdf6498 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:10.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.484+0000 7f2ded2e6640 1 -- 172.21.15.43:0/2714315388 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de4037070 con 0x7f2de815dce0 2024-04-25T09:01:10.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.484+0000 7f2ded2e6640 1 -- 172.21.15.43:0/2714315388 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de402ae50 con 0x7f2de815dce0 2024-04-25T09:01:10.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.484+0000 7f2ded2e6640 1 -- 172.21.15.43:0/2714315388 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de4032660 con 0x7f2de815dce0 2024-04-25T09:01:10.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.485+0000 7f2df054c640 1 -- 172.21.15.43:0/2714315388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2de815dce0 msgr2=0x7f2de815ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:10.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.485+0000 7f2df054c640 1 --2- 172.21.15.43:0/2714315388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2de815dce0 0x7f2de815ef50 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f2de40097a0 tx=0x7f2de402a140 comp rx=0 tx=0).stop 2024-04-25T09:01:10.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.485+0000 7f2df054c640 1 -- 172.21.15.43:0/2714315388 shutdown_connections 2024-04-25T09:01:10.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.485+0000 7f2df054c640 1 --2- 172.21.15.43:0/2714315388 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2de815dce0 0x7f2de815ef50 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:10.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.485+0000 7f2df054c640 1 -- 172.21.15.43:0/2714315388 >> 172.21.15.43:0/2714315388 conn(0x7f2de809cd00 msgr2=0x7f2de80d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:10.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.485+0000 7f2df054c640 1 -- 172.21.15.43:0/2714315388 shutdown_connections 2024-04-25T09:01:10.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.485+0000 7f2df054c640 1 -- 172.21.15.43:0/2714315388 wait complete. 2024-04-25T09:01:10.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.486+0000 7f2df054c640 1 Processor -- start 2024-04-25T09:01:10.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.486+0000 7f2df054c640 1 -- start start 2024-04-25T09:01:10.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.487+0000 7f2df054c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2de80ce550 0x7f2de809dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:10.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.487+0000 7f2df054c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2de80d0af0 0x7f2de809fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:10.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.487+0000 7f2df054c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2de815dce0 0x7f2de809ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:10.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.487+0000 7f2df054c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de815e0c0 con 0x7f2de80ce550 2024-04-25T09:01:10.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.487+0000 7f2df054c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de815f6b0 con 0x7f2de80d0af0 2024-04-25T09:01:10.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.487+0000 7f2df054c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2de80ce9c0 con 0x7f2de815dce0 2024-04-25T09:01:10.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.487+0000 7f2dee2e8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2de80ce550 0x7f2de809dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:10.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.487+0000 7f2dee2e8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2de80ce550 0x7f2de809dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42458/0 (socket says 172.21.15.43:42458) 2024-04-25T09:01:10.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.487+0000 7f2dee2e8640 1 -- 172.21.15.43:0/1525147443 learned_addr learned my addr 172.21.15.43:0/1525147443 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:10.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.487+0000 7f2dee2e8640 1 -- 172.21.15.43:0/1525147443 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2de815dce0 msgr2=0x7f2de809ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:10.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.487+0000 7f2dee2e8640 1 --2- 172.21.15.43:0/1525147443 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2de815dce0 0x7f2de809ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:10.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.487+0000 7f2dedae7640 1 --2- 172.21.15.43:0/1525147443 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2de80d0af0 0x7f2de809fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:10.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.487+0000 7f2dee2e8640 1 -- 172.21.15.43:0/1525147443 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2de80d0af0 msgr2=0x7f2de809fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:10.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.487+0000 7f2dee2e8640 1 --2- 172.21.15.43:0/1525147443 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2de80d0af0 0x7f2de809fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:10.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.487+0000 7f2dee2e8640 1 -- 172.21.15.43:0/1525147443 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de809f650 con 0x7f2de80ce550 2024-04-25T09:01:10.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.488+0000 7f2dee2e8640 1 --2- 172.21.15.43:0/1525147443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2de80ce550 0x7f2de809dc40 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f2de40097a0 tx=0x7f2de40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:10.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.488+0000 7f2ddeffd640 1 -- 172.21.15.43:0/1525147443 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de4042070 con 0x7f2de80ce550 2024-04-25T09:01:10.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.488+0000 7f2ddeffd640 1 -- 172.21.15.43:0/1525147443 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de403e670 con 0x7f2de80ce550 2024-04-25T09:01:10.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.488+0000 7f2df054c640 1 -- 172.21.15.43:0/1525147443 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2de809e5b0 con 0x7f2de80ce550 2024-04-25T09:01:10.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.488+0000 7f2ddeffd640 1 -- 172.21.15.43:0/1525147443 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de4037040 con 0x7f2de80ce550 2024-04-25T09:01:10.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.488+0000 7f2df054c640 1 -- 172.21.15.43:0/1525147443 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2de809eb70 con 0x7f2de80ce550 2024-04-25T09:01:10.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.489+0000 7f2df054c640 1 -- 172.21.15.43:0/1525147443 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2de80680f0 con 0x7f2de80ce550 2024-04-25T09:01:10.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.489+0000 7f2ddeffd640 1 -- 172.21.15.43:0/1525147443 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2de4045070 con 0x7f2de80ce550 2024-04-25T09:01:10.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.490+0000 7f2ddeffd640 1 --2- 172.21.15.43:0/1525147443 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2dcc06b4d0 0x7f2dcc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:10.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.490+0000 7f2ddeffd640 1 -- 172.21.15.43:0/1525147443 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f2de40aaa20 con 0x7f2de80ce550 2024-04-25T09:01:10.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.490+0000 7f2dedae7640 1 --2- 172.21.15.43:0/1525147443 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2dcc06b4d0 0x7f2dcc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:10.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.491+0000 7f2dedae7640 1 --2- 172.21.15.43:0/1525147443 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2dcc06b4d0 0x7f2dcc06d990 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f2de815d2d0 tx=0x7f2dd8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:10.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.494+0000 7f2ddeffd640 1 -- 172.21.15.43:0/1525147443 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2de40799a0 con 0x7f2de80ce550 2024-04-25T09:01:10.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.713+0000 7f2df054c640 1 -- 172.21.15.43:0/1525147443 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2de809da20 con 0x7f2de80ce550 2024-04-25T09:01:10.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.713+0000 7f2ddeffd640 1 -- 172.21.15.43:0/1525147443 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f2de40792a0 con 0x7f2de80ce550 2024-04-25T09:01:10.702 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:01:10.702 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":54,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6200,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2258031526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2258031526},{"type":"v1","addr":"172.21.15.43:6834","nonce":2258031526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6206,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/1773440177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1773440177},{"type":"v1","addr":"172.21.15.193:6836","nonce":1773440177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6209,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3239725307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":3239725307},{"type":"v1","addr":"172.21.15.193:6837","nonce":3239725307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:00:42.536578+0000","modified":"2024-04-25T09:00:44.531104+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7596},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7596":{"gid":7596,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/1341028696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1341028696},{"type":"v1","addr":"172.21.15.43:6835","nonce":1341028696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:01:10.703 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 54 2024-04-25T09:01:10.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.715+0000 7f2df054c640 1 -- 172.21.15.43:0/1525147443 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2dcc06b4d0 msgr2=0x7f2dcc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:10.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.715+0000 7f2df054c640 1 --2- 172.21.15.43:0/1525147443 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2dcc06b4d0 0x7f2dcc06d990 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f2de815d2d0 tx=0x7f2dd8009040 comp rx=0 tx=0).stop 2024-04-25T09:01:10.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.715+0000 7f2df054c640 1 -- 172.21.15.43:0/1525147443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2de80ce550 msgr2=0x7f2de809dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:10.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.715+0000 7f2df054c640 1 --2- 172.21.15.43:0/1525147443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2de80ce550 0x7f2de809dc40 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f2de40097a0 tx=0x7f2de40046c0 comp rx=0 tx=0).stop 2024-04-25T09:01:10.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.715+0000 7f2df054c640 1 -- 172.21.15.43:0/1525147443 shutdown_connections 2024-04-25T09:01:10.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.715+0000 7f2df054c640 1 --2- 172.21.15.43:0/1525147443 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2de815dce0 0x7f2de809ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:10.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.715+0000 7f2df054c640 1 --2- 172.21.15.43:0/1525147443 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2dcc06b4d0 0x7f2dcc06d990 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:10.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.715+0000 7f2df054c640 1 --2- 172.21.15.43:0/1525147443 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2de80d0af0 0x7f2de809fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:10.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.715+0000 7f2df054c640 1 --2- 172.21.15.43:0/1525147443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2de80ce550 0x7f2de809dc40 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:10.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.715+0000 7f2df054c640 1 -- 172.21.15.43:0/1525147443 >> 172.21.15.43:0/1525147443 conn(0x7f2de809cd00 msgr2=0x7f2de80d1910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:10.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.715+0000 7f2df054c640 1 -- 172.21.15.43:0/1525147443 shutdown_connections 2024-04-25T09:01:10.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.715+0000 7f2df054c640 1 -- 172.21.15.43:0/1525147443 wait complete. 2024-04-25T09:01:10.715 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T09:01:10.715 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:01:10.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.826+0000 7f13438f5640 1 Processor -- start 2024-04-25T09:01:10.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.827+0000 7f13438f5640 1 -- start start 2024-04-25T09:01:10.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.827+0000 7f13438f5640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f133c0cf670 0x7f133c168d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:10.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.827+0000 7f13438f5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f133c162810 con 0x7f133c0c8d00 2024-04-25T09:01:10.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.827+0000 7f13438f5640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f133c0d24a0 con 0x7f133c15df50 2024-04-25T09:01:10.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.827+0000 7f13438f5640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f133c0cfa80 con 0x7f133c0cf670 2024-04-25T09:01:10.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.828+0000 7f1340e90640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f133c0c8d00 0x7f133c162100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47438/0 (socket says 172.21.15.43:47438) 2024-04-25T09:01:10.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.828+0000 7f1340e90640 1 -- 172.21.15.43:0/414909425 learned_addr learned my addr 172.21.15.43:0/414909425 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:10.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.828+0000 7f1341e92640 1 --2- 172.21.15.43:0/414909425 >> v2:172.21.15.193:3300/0 conn(0x7f133c0cf670 0x7f133c168d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:10.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.828+0000 7f133bfff640 1 -- 172.21.15.43:0/414909425 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2899586850 0 0) 0x7f133c162810 con 0x7f133c0c8d00 2024-04-25T09:01:10.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.828+0000 7f133bfff640 1 -- 172.21.15.43:0/414909425 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1324003520 con 0x7f133c0c8d00 2024-04-25T09:01:10.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.828+0000 7f1341e92640 1 -- 172.21.15.43:0/414909425 >> v1:172.21.15.193:6789/0 conn(0x7f133c15df50 legacy=0x7f133c1619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:10.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.828+0000 7f1341e92640 1 -- 172.21.15.43:0/414909425 >> v1:172.21.15.43:6789/0 conn(0x7f133c0c8d00 legacy=0x7f133c162100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:10.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.828+0000 7f1341e92640 1 -- 172.21.15.43:0/414909425 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f133c0d2740 con 0x7f133c0cf670 2024-04-25T09:01:10.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.829+0000 7f1341e92640 1 --2- 172.21.15.43:0/414909425 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f133c0cf670 0x7f133c168d20 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f132c009d90 tx=0x7f132c02a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90fbadfd32cb2264 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:10.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.829+0000 7f133bfff640 1 -- 172.21.15.43:0/414909425 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132c007740 con 0x7f133c0cf670 2024-04-25T09:01:10.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.829+0000 7f133bfff640 1 -- 172.21.15.43:0/414909425 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f132c0078e0 con 0x7f133c0cf670 2024-04-25T09:01:10.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.829+0000 7f133bfff640 1 -- 172.21.15.43:0/414909425 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f132c007c90 con 0x7f133c0cf670 2024-04-25T09:01:10.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.829+0000 7f13438f5640 1 -- 172.21.15.43:0/414909425 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f133c0cf670 msgr2=0x7f133c168d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:10.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.829+0000 7f13438f5640 1 --2- 172.21.15.43:0/414909425 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f133c0cf670 0x7f133c168d20 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f132c009d90 tx=0x7f132c02a6f0 comp rx=0 tx=0).stop 2024-04-25T09:01:10.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.830+0000 7f13438f5640 1 -- 172.21.15.43:0/414909425 shutdown_connections 2024-04-25T09:01:10.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.830+0000 7f13438f5640 1 --2- 172.21.15.43:0/414909425 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f133c0cf670 0x7f133c168d20 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:10.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.830+0000 7f13438f5640 1 -- 172.21.15.43:0/414909425 >> 172.21.15.43:0/414909425 conn(0x7f133c0d3a30 msgr2=0x7f133c0d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:10.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.830+0000 7f13438f5640 1 -- 172.21.15.43:0/414909425 shutdown_connections 2024-04-25T09:01:10.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.830+0000 7f13438f5640 1 -- 172.21.15.43:0/414909425 wait complete. 2024-04-25T09:01:10.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.831+0000 7f13438f5640 1 Processor -- start 2024-04-25T09:01:10.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.831+0000 7f13438f5640 1 -- start start 2024-04-25T09:01:10.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.831+0000 7f13438f5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f133c0c8d00 0x7f133c0c65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:10.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.831+0000 7f13438f5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f133c0cf670 0x7f133c1e9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:10.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.831+0000 7f13438f5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f133c15df50 0x7f133c1e99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:10.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.831+0000 7f13438f5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133c0c74f0 con 0x7f133c0c8d00 2024-04-25T09:01:10.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.831+0000 7f13438f5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f133c15e330 con 0x7f133c0cf670 2024-04-25T09:01:10.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.831+0000 7f13438f5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f133c0c77d0 con 0x7f133c15df50 2024-04-25T09:01:10.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.832+0000 7f1341691640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f133c0c8d00 0x7f133c0c65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:10.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.832+0000 7f1341691640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f133c0c8d00 0x7f133c0c65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42464/0 (socket says 172.21.15.43:42464) 2024-04-25T09:01:10.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.832+0000 7f1341691640 1 -- 172.21.15.43:0/4205703004 learned_addr learned my addr 172.21.15.43:0/4205703004 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:10.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.832+0000 7f1340e90640 1 --2- 172.21.15.43:0/4205703004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f133c0cf670 0x7f133c1e9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:10.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.832+0000 7f1341e92640 1 --2- 172.21.15.43:0/4205703004 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f133c15df50 0x7f133c1e99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:10.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.832+0000 7f1341691640 1 -- 172.21.15.43:0/4205703004 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f133c15df50 msgr2=0x7f133c1e99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:10.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.832+0000 7f1341691640 1 --2- 172.21.15.43:0/4205703004 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f133c15df50 0x7f133c1e99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:10.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.832+0000 7f1341691640 1 -- 172.21.15.43:0/4205703004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f133c0cf670 msgr2=0x7f133c1e9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:10.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.832+0000 7f1341691640 1 --2- 172.21.15.43:0/4205703004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f133c0cf670 0x7f133c1e9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:10.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.832+0000 7f1341691640 1 -- 172.21.15.43:0/4205703004 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f133c1e8a00 con 0x7f133c0c8d00 2024-04-25T09:01:10.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.832+0000 7f1341691640 1 --2- 172.21.15.43:0/4205703004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f133c0c8d00 0x7f133c0c65f0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f133400c3c0 tx=0x7f133400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:10.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.832+0000 7f1339ffb640 1 -- 172.21.15.43:0/4205703004 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13340052d0 con 0x7f133c0c8d00 2024-04-25T09:01:10.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.832+0000 7f1339ffb640 1 -- 172.21.15.43:0/4205703004 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1334005490 con 0x7f133c0c8d00 2024-04-25T09:01:10.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.832+0000 7f13438f5640 1 -- 172.21.15.43:0/4205703004 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f133c1e7960 con 0x7f133c0c8d00 2024-04-25T09:01:10.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.832+0000 7f1339ffb640 1 -- 172.21.15.43:0/4205703004 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1334007260 con 0x7f133c0c8d00 2024-04-25T09:01:10.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.833+0000 7f13438f5640 1 -- 172.21.15.43:0/4205703004 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f133c1e7f20 con 0x7f133c0c8d00 2024-04-25T09:01:10.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.833+0000 7f13438f5640 1 -- 172.21.15.43:0/4205703004 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f133c0680f0 con 0x7f133c0c8d00 2024-04-25T09:01:10.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.834+0000 7f1339ffb640 1 -- 172.21.15.43:0/4205703004 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1334007400 con 0x7f133c0c8d00 2024-04-25T09:01:10.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.834+0000 7f1339ffb640 1 --2- 172.21.15.43:0/4205703004 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f132806b4d0 0x7f132806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:10.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.834+0000 7f1339ffb640 1 -- 172.21.15.43:0/4205703004 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f1334089700 con 0x7f133c0c8d00 2024-04-25T09:01:10.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.835+0000 7f1340e90640 1 --2- 172.21.15.43:0/4205703004 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f132806b4d0 0x7f132806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:10.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.835+0000 7f1340e90640 1 --2- 172.21.15.43:0/4205703004 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f132806b4d0 0x7f132806d990 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f133c1f7480 tx=0x7f1330009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:10.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:10.839+0000 7f1339ffb640 1 -- 172.21.15.43:0/4205703004 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1334058560 con 0x7f133c0c8d00 2024-04-25T09:01:11.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:11.087+0000 7f13438f5640 1 -- 172.21.15.43:0/4205703004 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f133c074ef0 con 0x7f133c0c8d00 2024-04-25T09:01:11.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:11.087+0000 7f1339ffb640 1 -- 172.21.15.43:0/4205703004 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1808 (secure 0 0 0) 0x7f1334057e60 con 0x7f133c0c8d00 2024-04-25T09:01:11.076 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:01:11.076 INFO:teuthology.orchestra.run.smithi043.stdout:{"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":443,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":91,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035238,"num_in_osds":8,"osd_in_since":1714035229,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1078262393,"bytes_used":2693824512,"bytes_avail":770400288768,"bytes_total":773094113280,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":436807,"recovering_keys_per_sec":0,"num_objects_recovered":5,"num_bytes_recovered":5242880,"num_keys_recovered":0,"write_bytes_sec":62138762,"read_op_per_sec":0,"write_op_per_sec":63},"fsmap":{"epoch":54,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7596}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-04-25T09:01:09.790126+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/scrub_control_test_path]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:01:11.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:11.089+0000 7f13438f5640 1 -- 172.21.15.43:0/4205703004 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f132806b4d0 msgr2=0x7f132806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:11.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:11.089+0000 7f13438f5640 1 --2- 172.21.15.43:0/4205703004 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f132806b4d0 0x7f132806d990 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f133c1f7480 tx=0x7f1330009040 comp rx=0 tx=0).stop 2024-04-25T09:01:11.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:11.089+0000 7f13438f5640 1 -- 172.21.15.43:0/4205703004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f133c0c8d00 msgr2=0x7f133c0c65f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:11.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:11.089+0000 7f13438f5640 1 --2- 172.21.15.43:0/4205703004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f133c0c8d00 0x7f133c0c65f0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f133400c3c0 tx=0x7f133400c880 comp rx=0 tx=0).stop 2024-04-25T09:01:11.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:11.089+0000 7f13438f5640 1 -- 172.21.15.43:0/4205703004 shutdown_connections 2024-04-25T09:01:11.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:11.089+0000 7f13438f5640 1 --2- 172.21.15.43:0/4205703004 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f133c15df50 0x7f133c1e99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:11.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:11.089+0000 7f13438f5640 1 --2- 172.21.15.43:0/4205703004 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f132806b4d0 0x7f132806d990 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:11.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:11.089+0000 7f13438f5640 1 --2- 172.21.15.43:0/4205703004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f133c0cf670 0x7f133c1e9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:11.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:11.089+0000 7f13438f5640 1 --2- 172.21.15.43:0/4205703004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f133c0c8d00 0x7f133c0c65f0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:11.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:11.089+0000 7f13438f5640 1 -- 172.21.15.43:0/4205703004 >> 172.21.15.43:0/4205703004 conn(0x7f133c0d3a30 msgr2=0x7f133c15adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:11.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:11.089+0000 7f13438f5640 1 -- 172.21.15.43:0/4205703004 shutdown_connections 2024-04-25T09:01:11.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:11.089+0000 7f13438f5640 1 -- 172.21.15.43:0/4205703004 wait complete. 2024-04-25T09:01:11.089 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'idle+waiting paths [/scrub_control_test_path]'} 2024-04-25T09:01:12.090 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:01:12.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.215+0000 7f6be11e2640 1 Processor -- start 2024-04-25T09:01:12.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.215+0000 7f6be11e2640 1 -- start start 2024-04-25T09:01:12.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.216+0000 7f6be11e2640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6bdc06d9a0 0x7f6bdc071a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:12.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.216+0000 7f6be11e2640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bdc071f90 0x7f6bdc0cd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:12.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.216+0000 7f6be11e2640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bdc072420 con 0x7f6bdc15a9d0 2024-04-25T09:01:12.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.216+0000 7f6be11e2640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bdc0704f0 con 0x7f6bdc06d9a0 2024-04-25T09:01:12.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.216+0000 7f6be11e2640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc0707d0 con 0x7f6bdc071f90 2024-04-25T09:01:12.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.217+0000 7f6bdb59e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bdc071f90 0x7f6bdc0cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:12.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.217+0000 7f6bda59c640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6bdc06d9a0 0x7f6bdc071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:12.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.217+0000 7f6bdad9d640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f6bdc15a9d0 0x7f6bdc06d290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:52066/0 (socket says 172.21.15.43:52066) 2024-04-25T09:01:12.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.217+0000 7f6bdad9d640 1 -- 172.21.15.43:0/2626829142 learned_addr learned my addr 172.21.15.43:0/2626829142 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:12.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.217+0000 7f6bd9d9b640 1 -- 172.21.15.43:0/2626829142 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 211864577 0 0) 0x7f6bcc001110 con 0x7f6bdc15a9d0 2024-04-25T09:01:12.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.217+0000 7f6bd9d9b640 1 -- 172.21.15.43:0/2626829142 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bc4003520 con 0x7f6bdc15a9d0 2024-04-25T09:01:12.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.217+0000 7f6bdb59e640 1 -- 172.21.15.43:0/2626829142 >> v2:172.21.15.193:3300/0 conn(0x7f6bdc06d9a0 msgr2=0x7f6bdc071a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:12.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.217+0000 7f6bdb59e640 1 --2- 172.21.15.43:0/2626829142 >> v2:172.21.15.193:3300/0 conn(0x7f6bdc06d9a0 0x7f6bdc071a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:12.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.217+0000 7f6bdb59e640 1 -- 172.21.15.43:0/2626829142 >> v1:172.21.15.193:6789/0 conn(0x7f6bdc15a9d0 legacy=0x7f6bdc06d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:12.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.217+0000 7f6bdb59e640 1 -- 172.21.15.43:0/2626829142 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bdc070ab0 con 0x7f6bdc071f90 2024-04-25T09:01:12.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.217+0000 7f6bda59c640 1 --2- 172.21.15.43:0/2626829142 >> v2:172.21.15.193:3300/0 conn(0x7f6bdc06d9a0 0x7f6bdc071a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:01:12.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.218+0000 7f6bdb59e640 1 --2- 172.21.15.43:0/2626829142 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bdc071f90 0x7f6bdc0cd940 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4009d90 tx=0x7f6bd402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b78a464ccff482b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:12.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.218+0000 7f6bd9d9b640 1 -- 172.21.15.43:0/2626829142 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4007740 con 0x7f6bdc071f90 2024-04-25T09:01:12.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.218+0000 7f6bd9d9b640 1 -- 172.21.15.43:0/2626829142 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bd40078e0 con 0x7f6bdc071f90 2024-04-25T09:01:12.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.218+0000 7f6bd9d9b640 1 -- 172.21.15.43:0/2626829142 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd4007c90 con 0x7f6bdc071f90 2024-04-25T09:01:12.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.219+0000 7f6be11e2640 1 -- 172.21.15.43:0/2626829142 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bdc071f90 msgr2=0x7f6bdc0cd940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:12.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.219+0000 7f6be11e2640 1 --2- 172.21.15.43:0/2626829142 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bdc071f90 0x7f6bdc0cd940 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6bd4009d90 tx=0x7f6bd402a660 comp rx=0 tx=0).stop 2024-04-25T09:01:12.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.219+0000 7f6be11e2640 1 -- 172.21.15.43:0/2626829142 shutdown_connections 2024-04-25T09:01:12.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.219+0000 7f6be11e2640 1 --2- 172.21.15.43:0/2626829142 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bdc071f90 0x7f6bdc0cd940 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:12.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.219+0000 7f6be11e2640 1 --2- 172.21.15.43:0/2626829142 >> v2:172.21.15.193:3300/0 conn(0x7f6bdc06d9a0 0x7f6bdc071a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:12.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.219+0000 7f6be11e2640 1 -- 172.21.15.43:0/2626829142 >> 172.21.15.43:0/2626829142 conn(0x7f6bdc0b1480 msgr2=0x7f6bdc0b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:12.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.219+0000 7f6be11e2640 1 -- 172.21.15.43:0/2626829142 shutdown_connections 2024-04-25T09:01:12.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.219+0000 7f6be11e2640 1 -- 172.21.15.43:0/2626829142 wait complete. 2024-04-25T09:01:12.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.220+0000 7f6be11e2640 1 Processor -- start 2024-04-25T09:01:12.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.220+0000 7f6be11e2640 1 -- start start 2024-04-25T09:01:12.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.221+0000 7f6be11e2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bdc06d9a0 0x7f6bdc1e2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:12.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.221+0000 7f6be11e2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6bdc071f90 0x7f6bdc13db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:12.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.221+0000 7f6be11e2640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bdc1e5770 0x7f6bdc1e5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:12.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.221+0000 7f6be11e2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc0cde80 con 0x7f6bdc071f90 2024-04-25T09:01:12.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.221+0000 7f6be11e2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc0712b0 con 0x7f6bdc06d9a0 2024-04-25T09:01:12.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.221+0000 7f6be11e2640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc0ce000 con 0x7f6bdc1e5770 2024-04-25T09:01:12.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.221+0000 7f6bda59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6bdc071f90 0x7f6bdc13db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:12.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.221+0000 7f6bda59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6bdc071f90 0x7f6bdc13db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42478/0 (socket says 172.21.15.43:42478) 2024-04-25T09:01:12.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.221+0000 7f6bda59c640 1 -- 172.21.15.43:0/3745022351 learned_addr learned my addr 172.21.15.43:0/3745022351 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:12.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.221+0000 7f6bdad9d640 1 --2- 172.21.15.43:0/3745022351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bdc06d9a0 0x7f6bdc1e2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:12.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.221+0000 7f6bdb59e640 1 --2- 172.21.15.43:0/3745022351 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bdc1e5770 0x7f6bdc1e5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:12.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.221+0000 7f6bda59c640 1 -- 172.21.15.43:0/3745022351 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bdc1e5770 msgr2=0x7f6bdc1e5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:12.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.221+0000 7f6bda59c640 1 --2- 172.21.15.43:0/3745022351 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bdc1e5770 0x7f6bdc1e5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:12.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.221+0000 7f6bda59c640 1 -- 172.21.15.43:0/3745022351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bdc06d9a0 msgr2=0x7f6bdc1e2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:12.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.221+0000 7f6bda59c640 1 --2- 172.21.15.43:0/3745022351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bdc06d9a0 0x7f6bdc1e2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:12.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.221+0000 7f6bda59c640 1 -- 172.21.15.43:0/3745022351 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bdc13e090 con 0x7f6bdc071f90 2024-04-25T09:01:12.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.222+0000 7f6bda59c640 1 --2- 172.21.15.43:0/3745022351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6bdc071f90 0x7f6bdc13db50 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f6bd000ebe0 tx=0x7f6bd000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:12.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.222+0000 7f6bc37fe640 1 -- 172.21.15.43:0/3745022351 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0017070 con 0x7f6bdc071f90 2024-04-25T09:01:12.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.222+0000 7f6bc37fe640 1 -- 172.21.15.43:0/3745022351 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bd000cc30 con 0x7f6bdc071f90 2024-04-25T09:01:12.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.222+0000 7f6be11e2640 1 -- 172.21.15.43:0/3745022351 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bdc1f16e0 con 0x7f6bdc071f90 2024-04-25T09:01:12.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.222+0000 7f6bc37fe640 1 -- 172.21.15.43:0/3745022351 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0013470 con 0x7f6bdc071f90 2024-04-25T09:01:12.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.222+0000 7f6be11e2640 1 -- 172.21.15.43:0/3745022351 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bdc13e6f0 con 0x7f6bdc071f90 2024-04-25T09:01:12.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.223+0000 7f6be11e2640 1 -- 172.21.15.43:0/3745022351 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bdc0b3770 con 0x7f6bdc071f90 2024-04-25T09:01:12.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.223+0000 7f6bc37fe640 1 -- 172.21.15.43:0/3745022351 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6bd001a070 con 0x7f6bdc071f90 2024-04-25T09:01:12.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.224+0000 7f6bc37fe640 1 --2- 172.21.15.43:0/3745022351 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6bc806b4d0 0x7f6bc806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:12.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.224+0000 7f6bc37fe640 1 -- 172.21.15.43:0/3745022351 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f6bd008a530 con 0x7f6bdc071f90 2024-04-25T09:01:12.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.224+0000 7f6bdad9d640 1 --2- 172.21.15.43:0/3745022351 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6bc806b4d0 0x7f6bc806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:12.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.225+0000 7f6bdad9d640 1 --2- 172.21.15.43:0/3745022351 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6bc806b4d0 0x7f6bc806d990 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc007fd0 tx=0x7f6bcc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:12.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.228+0000 7f6bc37fe640 1 -- 172.21.15.43:0/3745022351 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6bdc0b3770 con 0x7f6bdc071f90 2024-04-25T09:01:12.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.445+0000 7f6be11e2640 1 -- 172.21.15.43:0/3745022351 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6bdc0cb970 con 0x7f6bdc071f90 2024-04-25T09:01:12.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.446+0000 7f6bc37fe640 1 -- 172.21.15.43:0/3745022351 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f6bdc0cb970 con 0x7f6bdc071f90 2024-04-25T09:01:12.435 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:01:12.435 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":54,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6200,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2258031526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2258031526},{"type":"v1","addr":"172.21.15.43:6834","nonce":2258031526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6206,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/1773440177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1773440177},{"type":"v1","addr":"172.21.15.193:6836","nonce":1773440177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6209,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3239725307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":3239725307},{"type":"v1","addr":"172.21.15.193:6837","nonce":3239725307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:00:42.536578+0000","modified":"2024-04-25T09:00:44.531104+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7596},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7596":{"gid":7596,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/1341028696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1341028696},{"type":"v1","addr":"172.21.15.43:6835","nonce":1341028696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:01:12.435 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 54 2024-04-25T09:01:12.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.447+0000 7f6be11e2640 1 -- 172.21.15.43:0/3745022351 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6bc806b4d0 msgr2=0x7f6bc806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:12.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.447+0000 7f6be11e2640 1 --2- 172.21.15.43:0/3745022351 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6bc806b4d0 0x7f6bc806d990 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc007fd0 tx=0x7f6bcc009040 comp rx=0 tx=0).stop 2024-04-25T09:01:12.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.447+0000 7f6be11e2640 1 -- 172.21.15.43:0/3745022351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6bdc071f90 msgr2=0x7f6bdc13db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:12.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.447+0000 7f6be11e2640 1 --2- 172.21.15.43:0/3745022351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6bdc071f90 0x7f6bdc13db50 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f6bd000ebe0 tx=0x7f6bd000c520 comp rx=0 tx=0).stop 2024-04-25T09:01:12.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.448+0000 7f6be11e2640 1 -- 172.21.15.43:0/3745022351 shutdown_connections 2024-04-25T09:01:12.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.448+0000 7f6be11e2640 1 --2- 172.21.15.43:0/3745022351 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6bdc1e5770 0x7f6bdc1e5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:12.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.448+0000 7f6be11e2640 1 --2- 172.21.15.43:0/3745022351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6bdc071f90 0x7f6bdc13db50 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:12.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.448+0000 7f6be11e2640 1 --2- 172.21.15.43:0/3745022351 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6bc806b4d0 0x7f6bc806d990 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:12.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.448+0000 7f6be11e2640 1 --2- 172.21.15.43:0/3745022351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6bdc06d9a0 0x7f6bdc1e2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:12.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.448+0000 7f6be11e2640 1 -- 172.21.15.43:0/3745022351 >> 172.21.15.43:0/3745022351 conn(0x7f6bdc0b1480 msgr2=0x7f6bdc0b0bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:12.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.448+0000 7f6be11e2640 1 -- 172.21.15.43:0/3745022351 shutdown_connections 2024-04-25T09:01:12.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.448+0000 7f6be11e2640 1 -- 172.21.15.43:0/3745022351 wait complete. 2024-04-25T09:01:12.447 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T09:01:12.447 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:01:12.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.558+0000 7f2a77fab640 1 Processor -- start 2024-04-25T09:01:12.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.558+0000 7f2a77fab640 1 -- start start 2024-04-25T09:01:12.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.559+0000 7f2a77fab640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a7015dce0 0x7f2a7015ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:12.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.559+0000 7f2a77fab640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a700d2920 con 0x7f2a700d0af0 2024-04-25T09:01:12.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.559+0000 7f2a77fab640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a700cee80 con 0x7f2a700ce550 2024-04-25T09:01:12.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.559+0000 7f2a77fab640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a700cf160 con 0x7f2a7015dce0 2024-04-25T09:01:12.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.559+0000 7f2a76548640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f2a700d0af0 0x7f2a70160980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47442/0 (socket says 172.21.15.43:47442) 2024-04-25T09:01:12.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.559+0000 7f2a76548640 1 -- 172.21.15.43:0/1929406803 learned_addr learned my addr 172.21.15.43:0/1929406803 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:12.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.559+0000 7f2a75d47640 1 --2- 172.21.15.43:0/1929406803 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a7015dce0 0x7f2a7015ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:12.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.559+0000 7f2a74d45640 1 -- 172.21.15.43:0/1929406803 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2152633478 0 0) 0x7f2a700d2920 con 0x7f2a700d0af0 2024-04-25T09:01:12.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.559+0000 7f2a74d45640 1 -- 172.21.15.43:0/1929406803 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a5c003520 con 0x7f2a700d0af0 2024-04-25T09:01:12.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.560+0000 7f2a74d45640 1 -- 172.21.15.43:0/1929406803 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2392115541 0 0) 0x7f2a5c003520 con 0x7f2a700d0af0 2024-04-25T09:01:12.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.560+0000 7f2a74d45640 1 -- 172.21.15.43:0/1929406803 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a7015dce0 msgr2=0x7f2a7015ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:12.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.560+0000 7f2a74d45640 1 --2- 172.21.15.43:0/1929406803 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a7015dce0 0x7f2a7015ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:12.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.560+0000 7f2a74d45640 1 -- 172.21.15.43:0/1929406803 >> v1:172.21.15.193:6789/0 conn(0x7f2a700ce550 legacy=0x7f2a700d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:12.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.560+0000 7f2a74d45640 1 -- 172.21.15.43:0/1929406803 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a700cf440 con 0x7f2a700d0af0 2024-04-25T09:01:12.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.560+0000 7f2a74d45640 1 -- 172.21.15.43:0/1929406803 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f2a68001a10 con 0x7f2a700d0af0 2024-04-25T09:01:12.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.560+0000 7f2a74d45640 1 -- 172.21.15.43:0/1929406803 >> v1:172.21.15.43:6789/0 conn(0x7f2a700d0af0 legacy=0x7f2a70160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:12.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.560+0000 7f2a74d45640 1 --2- 172.21.15.43:0/1929406803 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a5c0039e0 0x7f2a5c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:12.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.560+0000 7f2a74d45640 1 --2- 172.21.15.43:0/1929406803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a5c004390 0x7f2a5c004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:12.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.560+0000 7f2a74d45640 1 --2- 172.21.15.43:0/1929406803 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a7015dce0 0x7f2a5c005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:12.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.560+0000 7f2a74d45640 1 -- 172.21.15.43:0/1929406803 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a700cf160 con 0x7f2a5c004390 2024-04-25T09:01:12.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.560+0000 7f2a74d45640 1 -- 172.21.15.43:0/1929406803 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c00ba70 con 0x7f2a5c0039e0 2024-04-25T09:01:12.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.560+0000 7f2a74d45640 1 -- 172.21.15.43:0/1929406803 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c00bc70 con 0x7f2a7015dce0 2024-04-25T09:01:12.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.560+0000 7f2a75546640 1 --2- 172.21.15.43:0/1929406803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a5c004390 0x7f2a5c004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:12.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.560+0000 7f2a75d47640 1 --2- 172.21.15.43:0/1929406803 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a5c0039e0 0x7f2a5c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:12.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.560+0000 7f2a76548640 1 --2- 172.21.15.43:0/1929406803 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a7015dce0 0x7f2a5c005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:12.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.560+0000 7f2a75546640 1 -- 172.21.15.43:0/1929406803 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a7015dce0 msgr2=0x7f2a5c005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:12.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.560+0000 7f2a75546640 1 --2- 172.21.15.43:0/1929406803 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a7015dce0 0x7f2a5c005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:12.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.560+0000 7f2a75546640 1 -- 172.21.15.43:0/1929406803 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a5c0039e0 msgr2=0x7f2a5c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:12.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.560+0000 7f2a75546640 1 --2- 172.21.15.43:0/1929406803 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a5c0039e0 0x7f2a5c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:12.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.560+0000 7f2a75546640 1 -- 172.21.15.43:0/1929406803 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a5c003520 con 0x7f2a5c004390 2024-04-25T09:01:12.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.561+0000 7f2a75546640 1 --2- 172.21.15.43:0/1929406803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a5c004390 0x7f2a5c004860 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f2a60001860 tx=0x7f2a5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=839c78879cea8036 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:12.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.561+0000 7f2a74d45640 1 -- 172.21.15.43:0/1929406803 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a60037070 con 0x7f2a5c004390 2024-04-25T09:01:12.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.561+0000 7f2a74d45640 1 -- 172.21.15.43:0/1929406803 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a6002abf0 con 0x7f2a5c004390 2024-04-25T09:01:12.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.561+0000 7f2a74d45640 1 -- 172.21.15.43:0/1929406803 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a60001430 con 0x7f2a5c004390 2024-04-25T09:01:12.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.561+0000 7f2a77fab640 1 -- 172.21.15.43:0/1929406803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a5c004390 msgr2=0x7f2a5c004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:12.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.561+0000 7f2a77fab640 1 --2- 172.21.15.43:0/1929406803 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a5c004390 0x7f2a5c004860 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f2a60001860 tx=0x7f2a5c005830 comp rx=0 tx=0).stop 2024-04-25T09:01:12.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.561+0000 7f2a76548640 1 -- 172.21.15.43:0/1929406803 reap_dead start 2024-04-25T09:01:12.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.562+0000 7f2a77fab640 1 -- 172.21.15.43:0/1929406803 shutdown_connections 2024-04-25T09:01:12.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.562+0000 7f2a77fab640 1 -- 172.21.15.43:0/1929406803 >> 172.21.15.43:0/1929406803 conn(0x7f2a7009cd00 msgr2=0x7f2a700d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:12.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.562+0000 7f2a77fab640 1 -- 172.21.15.43:0/1929406803 shutdown_connections 2024-04-25T09:01:12.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.562+0000 7f2a77fab640 1 -- 172.21.15.43:0/1929406803 wait complete. 2024-04-25T09:01:12.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.563+0000 7f2a77fab640 1 Processor -- start 2024-04-25T09:01:12.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.563+0000 7f2a77fab640 1 -- start start 2024-04-25T09:01:12.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.563+0000 7f2a77fab640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a5c004390 0x7f2a700b64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:12.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.564+0000 7f2a77fab640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a70151de0 0x7f2a700b8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:12.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.564+0000 7f2a77fab640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a70152260 0x7f2a700b8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:12.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.564+0000 7f2a77fab640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7015e0c0 con 0x7f2a70152260 2024-04-25T09:01:12.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.564+0000 7f2a77fab640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7015f6b0 con 0x7f2a5c004390 2024-04-25T09:01:12.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.564+0000 7f2a77fab640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a700ce990 con 0x7f2a70151de0 2024-04-25T09:01:12.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.564+0000 7f2a76548640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a70152260 0x7f2a700b8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:12.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.564+0000 7f2a76548640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a70152260 0x7f2a700b8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42488/0 (socket says 172.21.15.43:42488) 2024-04-25T09:01:12.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.564+0000 7f2a76548640 1 -- 172.21.15.43:0/3729892945 learned_addr learned my addr 172.21.15.43:0/3729892945 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:12.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.564+0000 7f2a75d47640 1 --2- 172.21.15.43:0/3729892945 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a5c004390 0x7f2a700b64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:12.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.564+0000 7f2a76548640 1 -- 172.21.15.43:0/3729892945 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a70151de0 msgr2=0x7f2a700b8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:12.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.564+0000 7f2a76548640 1 --2- 172.21.15.43:0/3729892945 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a70151de0 0x7f2a700b8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:12.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.564+0000 7f2a76548640 1 -- 172.21.15.43:0/3729892945 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a5c004390 msgr2=0x7f2a700b64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:12.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.564+0000 7f2a76548640 1 --2- 172.21.15.43:0/3729892945 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a5c004390 0x7f2a700b64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:12.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.564+0000 7f2a76548640 1 -- 172.21.15.43:0/3729892945 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a700b6a20 con 0x7f2a70152260 2024-04-25T09:01:12.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.564+0000 7f2a75d47640 1 --2- 172.21.15.43:0/3729892945 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a5c004390 0x7f2a700b64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:12.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.564+0000 7f2a76548640 1 --2- 172.21.15.43:0/3729892945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a70152260 0x7f2a700b8f80 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f2a68000f80 tx=0x7f2a68007bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:12.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.564+0000 7f2a667fc640 1 -- 172.21.15.43:0/3729892945 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6800ac40 con 0x7f2a70152260 2024-04-25T09:01:12.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.564+0000 7f2a667fc640 1 -- 172.21.15.43:0/3729892945 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a68019070 con 0x7f2a70152260 2024-04-25T09:01:12.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.564+0000 7f2a77fab640 1 -- 172.21.15.43:0/3729892945 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a701f50a0 con 0x7f2a70152260 2024-04-25T09:01:12.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.564+0000 7f2a667fc640 1 -- 172.21.15.43:0/3729892945 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a68013630 con 0x7f2a70152260 2024-04-25T09:01:12.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.564+0000 7f2a77fab640 1 -- 172.21.15.43:0/3729892945 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a7009daf0 con 0x7f2a70152260 2024-04-25T09:01:12.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.565+0000 7f2a77fab640 1 -- 172.21.15.43:0/3729892945 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a700680f0 con 0x7f2a70152260 2024-04-25T09:01:12.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.566+0000 7f2a667fc640 1 -- 172.21.15.43:0/3729892945 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2a680137d0 con 0x7f2a70152260 2024-04-25T09:01:12.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.566+0000 7f2a667fc640 1 --2- 172.21.15.43:0/3729892945 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2a5406b4d0 0x7f2a5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:12.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.566+0000 7f2a667fc640 1 -- 172.21.15.43:0/3729892945 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f2a6808a850 con 0x7f2a70152260 2024-04-25T09:01:12.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.566+0000 7f2a75d47640 1 --2- 172.21.15.43:0/3729892945 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2a5406b4d0 0x7f2a5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:12.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.567+0000 7f2a75d47640 1 --2- 172.21.15.43:0/3729892945 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2a5406b4d0 0x7f2a5406d990 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c009a40 tx=0x7f2a6c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:12.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.571+0000 7f2a667fc640 1 -- 172.21.15.43:0/3729892945 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2a680596b0 con 0x7f2a70152260 2024-04-25T09:01:12.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.821+0000 7f2a77fab640 1 -- 172.21.15.43:0/3729892945 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f2a7015d0e0 con 0x7f2a70152260 2024-04-25T09:01:12.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.821+0000 7f2a667fc640 1 -- 172.21.15.43:0/3729892945 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1636 (secure 0 0 0) 0x7f2a6800ae20 con 0x7f2a70152260 2024-04-25T09:01:12.810 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:01:12.810 INFO:teuthology.orchestra.run.smithi043.stdout:{"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":445,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":91,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035238,"num_in_osds":8,"osd_in_since":1714035229,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1078262393,"bytes_used":2693824512,"bytes_avail":770400288768,"bytes_total":773094113280,"write_bytes_sec":30669294,"read_op_per_sec":0,"write_op_per_sec":31},"fsmap":{"epoch":54,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7596}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-04-25T09:01:09.790126+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/scrub_control_test_path]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:01:12.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.823+0000 7f2a77fab640 1 -- 172.21.15.43:0/3729892945 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2a5406b4d0 msgr2=0x7f2a5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:12.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.823+0000 7f2a77fab640 1 --2- 172.21.15.43:0/3729892945 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2a5406b4d0 0x7f2a5406d990 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c009a40 tx=0x7f2a6c008040 comp rx=0 tx=0).stop 2024-04-25T09:01:12.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.823+0000 7f2a77fab640 1 -- 172.21.15.43:0/3729892945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a70152260 msgr2=0x7f2a700b8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:12.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.823+0000 7f2a77fab640 1 --2- 172.21.15.43:0/3729892945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a70152260 0x7f2a700b8f80 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f2a68000f80 tx=0x7f2a68007bd0 comp rx=0 tx=0).stop 2024-04-25T09:01:12.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.824+0000 7f2a77fab640 1 -- 172.21.15.43:0/3729892945 shutdown_connections 2024-04-25T09:01:12.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.824+0000 7f2a77fab640 1 --2- 172.21.15.43:0/3729892945 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a70152260 0x7f2a700b8f80 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:12.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.824+0000 7f2a77fab640 1 --2- 172.21.15.43:0/3729892945 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a70151de0 0x7f2a700b8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:12.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.824+0000 7f2a77fab640 1 --2- 172.21.15.43:0/3729892945 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2a5406b4d0 0x7f2a5406d990 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:12.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.824+0000 7f2a77fab640 1 --2- 172.21.15.43:0/3729892945 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a5c004390 0x7f2a700b64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:12.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.824+0000 7f2a77fab640 1 -- 172.21.15.43:0/3729892945 >> 172.21.15.43:0/3729892945 conn(0x7f2a7009cd00 msgr2=0x7f2a700c74a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:12.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.824+0000 7f2a77fab640 1 -- 172.21.15.43:0/3729892945 shutdown_connections 2024-04-25T09:01:12.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:12.824+0000 7f2a77fab640 1 -- 172.21.15.43:0/3729892945 wait complete. 2024-04-25T09:01:12.824 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'idle+waiting paths [/scrub_control_test_path]'} 2024-04-25T09:01:13.825 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:01:13.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.950+0000 7fd17522a640 1 Processor -- start 2024-04-25T09:01:13.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.950+0000 7fd17522a640 1 -- start start 2024-04-25T09:01:13.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.950+0000 7fd17522a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd1700ce2a0 0x7fd1700d2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:13.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.951+0000 7fd17522a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1700b6c50 con 0x7fd1700d0d90 2024-04-25T09:01:13.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.951+0000 7fd17522a640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd17015e010 con 0x7fd1700d39d0 2024-04-25T09:01:13.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.951+0000 7fd17522a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1700d2020 con 0x7fd1700ce2a0 2024-04-25T09:01:13.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.951+0000 7fd16f59e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd1700ce2a0 0x7fd1700d2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:13.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.951+0000 7fd167fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fd1700d0d90 0x7fd1700d2500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47444/0 (socket says 172.21.15.43:47444) 2024-04-25T09:01:13.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.951+0000 7fd167fff640 1 -- 172.21.15.43:0/461270753 learned_addr learned my addr 172.21.15.43:0/461270753 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:13.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.951+0000 7fd16e59c640 1 -- 172.21.15.43:0/461270753 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4041452478 0 0) 0x7fd158001110 con 0x7fd1700d0d90 2024-04-25T09:01:13.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.951+0000 7fd16e59c640 1 -- 172.21.15.43:0/461270753 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd15c003520 con 0x7fd1700d0d90 2024-04-25T09:01:13.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.951+0000 7fd16f59e640 1 -- 172.21.15.43:0/461270753 >> v1:172.21.15.193:6789/0 conn(0x7fd1700d39d0 legacy=0x7fd1700d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:13.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.951+0000 7fd16f59e640 1 -- 172.21.15.43:0/461270753 >> v1:172.21.15.43:6789/0 conn(0x7fd1700d0d90 legacy=0x7fd1700d2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:13.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.951+0000 7fd16f59e640 1 -- 172.21.15.43:0/461270753 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1700cec40 con 0x7fd1700ce2a0 2024-04-25T09:01:13.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.952+0000 7fd16f59e640 1 --2- 172.21.15.43:0/461270753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1700ce2a0 0x7fd1700d2c10 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fd168009d90 tx=0x7fd16802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8fa5343b0fafaa7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:13.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.952+0000 7fd16e59c640 1 -- 172.21.15.43:0/461270753 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd168007740 con 0x7fd1700ce2a0 2024-04-25T09:01:13.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.952+0000 7fd16e59c640 1 -- 172.21.15.43:0/461270753 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1680078e0 con 0x7fd1700ce2a0 2024-04-25T09:01:13.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.952+0000 7fd16e59c640 1 -- 172.21.15.43:0/461270753 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd168007cb0 con 0x7fd1700ce2a0 2024-04-25T09:01:13.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.953+0000 7fd17522a640 1 -- 172.21.15.43:0/461270753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1700ce2a0 msgr2=0x7fd1700d2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:13.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.953+0000 7fd17522a640 1 --2- 172.21.15.43:0/461270753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1700ce2a0 0x7fd1700d2c10 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fd168009d90 tx=0x7fd16802a660 comp rx=0 tx=0).stop 2024-04-25T09:01:13.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.953+0000 7fd17522a640 1 -- 172.21.15.43:0/461270753 shutdown_connections 2024-04-25T09:01:13.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.953+0000 7fd17522a640 1 --2- 172.21.15.43:0/461270753 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1700ce2a0 0x7fd1700d2c10 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:13.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.953+0000 7fd17522a640 1 -- 172.21.15.43:0/461270753 >> 172.21.15.43:0/461270753 conn(0x7fd17006b000 msgr2=0x7fd17006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:13.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.953+0000 7fd17522a640 1 -- 172.21.15.43:0/461270753 shutdown_connections 2024-04-25T09:01:13.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.953+0000 7fd17522a640 1 -- 172.21.15.43:0/461270753 wait complete. 2024-04-25T09:01:13.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.954+0000 7fd17522a640 1 Processor -- start 2024-04-25T09:01:13.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.954+0000 7fd17522a640 1 -- start start 2024-04-25T09:01:13.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.955+0000 7fd17522a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd1700ce2a0 0x7fd1701567e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:13.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.955+0000 7fd17522a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1700d0d90 0x7fd1700b55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:13.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.955+0000 7fd17522a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1700d39d0 0x7fd17015a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:13.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.955+0000 7fd17522a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1700cf200 con 0x7fd1700d39d0 2024-04-25T09:01:13.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.955+0000 7fd17522a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1700b2f00 con 0x7fd1700d0d90 2024-04-25T09:01:13.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.955+0000 7fd17522a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1700d3150 con 0x7fd1700ce2a0 2024-04-25T09:01:13.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.955+0000 7fd16f59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1700d39d0 0x7fd17015a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:13.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.955+0000 7fd16f59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1700d39d0 0x7fd17015a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42510/0 (socket says 172.21.15.43:42510) 2024-04-25T09:01:13.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.955+0000 7fd16f59e640 1 -- 172.21.15.43:0/2971381585 learned_addr learned my addr 172.21.15.43:0/2971381585 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:13.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.955+0000 7fd167fff640 1 --2- 172.21.15.43:0/2971381585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1700d0d90 0x7fd1700b55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:13.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.955+0000 7fd16ed9d640 1 --2- 172.21.15.43:0/2971381585 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd1700ce2a0 0x7fd1701567e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:13.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.955+0000 7fd16f59e640 1 -- 172.21.15.43:0/2971381585 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd1700ce2a0 msgr2=0x7fd1701567e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:13.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.955+0000 7fd16f59e640 1 --2- 172.21.15.43:0/2971381585 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd1700ce2a0 0x7fd1701567e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:13.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.955+0000 7fd16f59e640 1 -- 172.21.15.43:0/2971381585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1700d0d90 msgr2=0x7fd1700b55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:13.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.955+0000 7fd16f59e640 1 --2- 172.21.15.43:0/2971381585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1700d0d90 0x7fd1700b55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:13.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.955+0000 7fd16f59e640 1 -- 172.21.15.43:0/2971381585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1700b60d0 con 0x7fd1700d39d0 2024-04-25T09:01:13.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.955+0000 7fd167fff640 1 --2- 172.21.15.43:0/2971381585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1700d0d90 0x7fd1700b55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:01:13.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.955+0000 7fd16f59e640 1 --2- 172.21.15.43:0/2971381585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1700d39d0 0x7fd17015a410 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fd168009eb0 tx=0x7fd168004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:13.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.956+0000 7fd1677fe640 1 -- 172.21.15.43:0/2971381585 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd168040070 con 0x7fd1700d39d0 2024-04-25T09:01:13.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.956+0000 7fd1677fe640 1 -- 172.21.15.43:0/2971381585 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd168031bc0 con 0x7fd1700d39d0 2024-04-25T09:01:13.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.956+0000 7fd17522a640 1 -- 172.21.15.43:0/2971381585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1700c87a0 con 0x7fd1700d39d0 2024-04-25T09:01:13.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.956+0000 7fd1677fe640 1 -- 172.21.15.43:0/2971381585 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd168009450 con 0x7fd1700d39d0 2024-04-25T09:01:13.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.956+0000 7fd17522a640 1 -- 172.21.15.43:0/2971381585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd170156d20 con 0x7fd1700d39d0 2024-04-25T09:01:13.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.957+0000 7fd17522a640 1 -- 172.21.15.43:0/2971381585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd170074ef0 con 0x7fd1700d39d0 2024-04-25T09:01:13.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.957+0000 7fd1677fe640 1 -- 172.21.15.43:0/2971381585 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd168039040 con 0x7fd1700d39d0 2024-04-25T09:01:13.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.958+0000 7fd1677fe640 1 --2- 172.21.15.43:0/2971381585 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd15006b4d0 0x7fd15006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:13.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.958+0000 7fd1677fe640 1 -- 172.21.15.43:0/2971381585 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fd1680aba60 con 0x7fd1700d39d0 2024-04-25T09:01:13.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.958+0000 7fd16ed9d640 1 --2- 172.21.15.43:0/2971381585 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd15006b4d0 0x7fd15006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:13.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.958+0000 7fd16ed9d640 1 --2- 172.21.15.43:0/2971381585 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd15006b4d0 0x7fd15006d990 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd160007fd0 tx=0x7fd160009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:13.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:13.962+0000 7fd1677fe640 1 -- 172.21.15.43:0/2971381585 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd168043360 con 0x7fd1700d39d0 2024-04-25T09:01:14.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.185+0000 7fd17522a640 1 -- 172.21.15.43:0/2971381585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1700ce880 con 0x7fd1700d39d0 2024-04-25T09:01:14.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.185+0000 7fd1677fe640 1 -- 172.21.15.43:0/2971381585 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fd16807a8b0 con 0x7fd1700d39d0 2024-04-25T09:01:14.175 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:01:14.175 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":54,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6200,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2258031526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2258031526},{"type":"v1","addr":"172.21.15.43:6834","nonce":2258031526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6206,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/1773440177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1773440177},{"type":"v1","addr":"172.21.15.193:6836","nonce":1773440177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6209,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3239725307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":3239725307},{"type":"v1","addr":"172.21.15.193:6837","nonce":3239725307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:00:42.536578+0000","modified":"2024-04-25T09:00:44.531104+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7596},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7596":{"gid":7596,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/1341028696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1341028696},{"type":"v1","addr":"172.21.15.43:6835","nonce":1341028696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:01:14.175 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 54 2024-04-25T09:01:14.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.188+0000 7fd17522a640 1 -- 172.21.15.43:0/2971381585 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd15006b4d0 msgr2=0x7fd15006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:14.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.188+0000 7fd17522a640 1 --2- 172.21.15.43:0/2971381585 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd15006b4d0 0x7fd15006d990 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd160007fd0 tx=0x7fd160009040 comp rx=0 tx=0).stop 2024-04-25T09:01:14.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.188+0000 7fd17522a640 1 -- 172.21.15.43:0/2971381585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1700d39d0 msgr2=0x7fd17015a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:14.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.188+0000 7fd17522a640 1 --2- 172.21.15.43:0/2971381585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1700d39d0 0x7fd17015a410 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7fd168009eb0 tx=0x7fd168004a40 comp rx=0 tx=0).stop 2024-04-25T09:01:14.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.188+0000 7fd17522a640 1 -- 172.21.15.43:0/2971381585 shutdown_connections 2024-04-25T09:01:14.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.188+0000 7fd17522a640 1 --2- 172.21.15.43:0/2971381585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd1700d39d0 0x7fd17015a410 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:14.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.188+0000 7fd17522a640 1 --2- 172.21.15.43:0/2971381585 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd15006b4d0 0x7fd15006d990 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:14.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.188+0000 7fd17522a640 1 --2- 172.21.15.43:0/2971381585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd1700d0d90 0x7fd1700b55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:14.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.188+0000 7fd17522a640 1 --2- 172.21.15.43:0/2971381585 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd1700ce2a0 0x7fd1701567e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:14.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.188+0000 7fd17522a640 1 -- 172.21.15.43:0/2971381585 >> 172.21.15.43:0/2971381585 conn(0x7fd17006b000 msgr2=0x7fd17015e630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:14.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.188+0000 7fd17522a640 1 -- 172.21.15.43:0/2971381585 shutdown_connections 2024-04-25T09:01:14.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.188+0000 7fd17522a640 1 -- 172.21.15.43:0/2971381585 wait complete. 2024-04-25T09:01:14.188 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T09:01:14.188 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:01:14.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.302+0000 7f3590fb0640 1 Processor -- start 2024-04-25T09:01:14.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.302+0000 7f3590fb0640 1 -- start start 2024-04-25T09:01:14.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.303+0000 7f3590fb0640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f358c15a250 0x7f358c070d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:14.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.303+0000 7f3590fb0640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f358c0b4110 0x7f358c071a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:14.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.303+0000 7f3590fb0640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f358c06dd30 con 0x7f358c0715a0 2024-04-25T09:01:14.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.303+0000 7f3590fb0640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f358c06e010 con 0x7f358c15a250 2024-04-25T09:01:14.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.303+0000 7f3590fb0640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f358c0b2900 con 0x7f358c0b4110 2024-04-25T09:01:14.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.303+0000 7f358ad9d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f358c0b4110 0x7f358c071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:14.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.303+0000 7f358a59c640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f358c15a250 0x7f358c070d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:14.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.303+0000 7f3589d9b640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f358c0715a0 0x7f358c15dc50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:52100/0 (socket says 172.21.15.43:52100) 2024-04-25T09:01:14.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.303+0000 7f3589d9b640 1 -- 172.21.15.43:0/2773859279 learned_addr learned my addr 172.21.15.43:0/2773859279 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:14.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.303+0000 7f358ad9d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f358c0b4110 0x7f358c071a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:36892/0 (socket says 172.21.15.43:36892) 2024-04-25T09:01:14.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.303+0000 7f358959a640 1 -- 172.21.15.43:0/2773859279 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2957504147 0 0) 0x7f358c06dd30 con 0x7f358c0715a0 2024-04-25T09:01:14.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.303+0000 7f358959a640 1 -- 172.21.15.43:0/2773859279 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3578004630 con 0x7f358c0715a0 2024-04-25T09:01:14.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.303+0000 7f358a59c640 1 -- 172.21.15.43:0/2773859279 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f358c0b4110 msgr2=0x7f358c071a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:14.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.303+0000 7f358a59c640 1 --2- 172.21.15.43:0/2773859279 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f358c0b4110 0x7f358c071a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:14.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.303+0000 7f358a59c640 1 -- 172.21.15.43:0/2773859279 >> v1:172.21.15.193:6789/0 conn(0x7f358c0715a0 legacy=0x7f358c15dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:14.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.303+0000 7f358a59c640 1 -- 172.21.15.43:0/2773859279 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f358c0b2be0 con 0x7f358c15a250 2024-04-25T09:01:14.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.303+0000 7f358ad9d640 1 --2- 172.21.15.43:0/2773859279 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f358c0b4110 0x7f358c071a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:01:14.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.304+0000 7f358a59c640 1 --2- 172.21.15.43:0/2773859279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f358c15a250 0x7f358c070d60 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f35840097a0 tx=0x7f358402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ddc314caee1bc13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:14.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.304+0000 7f358959a640 1 -- 172.21.15.43:0/2773859279 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3584037070 con 0x7f358c15a250 2024-04-25T09:01:14.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.304+0000 7f358959a640 1 -- 172.21.15.43:0/2773859279 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3584032440 con 0x7f358c15a250 2024-04-25T09:01:14.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.304+0000 7f358959a640 1 -- 172.21.15.43:0/2773859279 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35840327f0 con 0x7f358c15a250 2024-04-25T09:01:14.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.305+0000 7f3590fb0640 1 -- 172.21.15.43:0/2773859279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f358c15a250 msgr2=0x7f358c070d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:14.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.305+0000 7f3590fb0640 1 --2- 172.21.15.43:0/2773859279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f358c15a250 0x7f358c070d60 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f35840097a0 tx=0x7f358402a180 comp rx=0 tx=0).stop 2024-04-25T09:01:14.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.305+0000 7f3590fb0640 1 -- 172.21.15.43:0/2773859279 shutdown_connections 2024-04-25T09:01:14.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.305+0000 7f3590fb0640 1 --2- 172.21.15.43:0/2773859279 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f358c0b4110 0x7f358c071a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:14.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.305+0000 7f3590fb0640 1 --2- 172.21.15.43:0/2773859279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f358c15a250 0x7f358c070d60 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:14.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.305+0000 7f3590fb0640 1 -- 172.21.15.43:0/2773859279 >> 172.21.15.43:0/2773859279 conn(0x7f358c0b1480 msgr2=0x7f358c0b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:14.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.305+0000 7f3590fb0640 1 -- 172.21.15.43:0/2773859279 shutdown_connections 2024-04-25T09:01:14.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.305+0000 7f3590fb0640 1 -- 172.21.15.43:0/2773859279 wait complete. 2024-04-25T09:01:14.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.306+0000 7f3590fb0640 1 Processor -- start 2024-04-25T09:01:14.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.307+0000 7f3590fb0640 1 -- start start 2024-04-25T09:01:14.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.307+0000 7f3590fb0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f358c0715a0 0x7f358c0ad3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:14.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.307+0000 7f3590fb0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f358c0b4110 0x7f358c0adf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:14.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.307+0000 7f3590fb0640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f358c15a250 0x7f358c15c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:14.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.307+0000 7f3590fb0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358c0af020 con 0x7f358c0b4110 2024-04-25T09:01:14.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.307+0000 7f3590fb0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358c0aeea0 con 0x7f358c0715a0 2024-04-25T09:01:14.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.307+0000 7f3590fb0640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f358c15e360 con 0x7f358c15a250 2024-04-25T09:01:14.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.307+0000 7f3589d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f358c0b4110 0x7f358c0adf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:14.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.307+0000 7f3589d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f358c0b4110 0x7f358c0adf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42516/0 (socket says 172.21.15.43:42516) 2024-04-25T09:01:14.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.307+0000 7f3589d9b640 1 -- 172.21.15.43:0/2025593147 learned_addr learned my addr 172.21.15.43:0/2025593147 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:14.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.307+0000 7f358a59c640 1 --2- 172.21.15.43:0/2025593147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f358c0715a0 0x7f358c0ad3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:14.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.307+0000 7f358ad9d640 1 --2- 172.21.15.43:0/2025593147 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f358c15a250 0x7f358c15c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:14.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.308+0000 7f3589d9b640 1 -- 172.21.15.43:0/2025593147 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f358c15a250 msgr2=0x7f358c15c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:14.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.308+0000 7f3589d9b640 1 --2- 172.21.15.43:0/2025593147 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f358c15a250 0x7f358c15c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:14.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.308+0000 7f3589d9b640 1 -- 172.21.15.43:0/2025593147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f358c0715a0 msgr2=0x7f358c0ad3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:14.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.308+0000 7f3589d9b640 1 --2- 172.21.15.43:0/2025593147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f358c0715a0 0x7f358c0ad3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:14.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.308+0000 7f3589d9b640 1 -- 172.21.15.43:0/2025593147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f358c0ac350 con 0x7f358c0b4110 2024-04-25T09:01:14.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.308+0000 7f3589d9b640 1 --2- 172.21.15.43:0/2025593147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f358c0b4110 0x7f358c0adf90 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f357400c430 tx=0x7f357400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:14.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.308+0000 7f3582ffd640 1 -- 172.21.15.43:0/2025593147 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3574017070 con 0x7f358c0b4110 2024-04-25T09:01:14.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.308+0000 7f3582ffd640 1 -- 172.21.15.43:0/2025593147 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3574005ce0 con 0x7f358c0b4110 2024-04-25T09:01:14.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.308+0000 7f3590fb0640 1 -- 172.21.15.43:0/2025593147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f358c0ac8e0 con 0x7f358c0b4110 2024-04-25T09:01:14.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.308+0000 7f3582ffd640 1 -- 172.21.15.43:0/2025593147 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35740072f0 con 0x7f358c0b4110 2024-04-25T09:01:14.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.308+0000 7f3590fb0640 1 -- 172.21.15.43:0/2025593147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f358c0acea0 con 0x7f358c0b4110 2024-04-25T09:01:14.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.309+0000 7f3590fb0640 1 -- 172.21.15.43:0/2025593147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f358c067d30 con 0x7f358c0b4110 2024-04-25T09:01:14.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.310+0000 7f3582ffd640 1 -- 172.21.15.43:0/2025593147 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f357401a070 con 0x7f358c0b4110 2024-04-25T09:01:14.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.310+0000 7f3582ffd640 1 --2- 172.21.15.43:0/2025593147 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f356c06b4d0 0x7f356c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:14.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.310+0000 7f3582ffd640 1 -- 172.21.15.43:0/2025593147 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f3574089620 con 0x7f358c0b4110 2024-04-25T09:01:14.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.310+0000 7f358a59c640 1 --2- 172.21.15.43:0/2025593147 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f356c06b4d0 0x7f356c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:14.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.311+0000 7f358a59c640 1 --2- 172.21.15.43:0/2025593147 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f356c06b4d0 0x7f356c06d990 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3584034000 tx=0x7f358402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:14.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.314+0000 7f3582ffd640 1 -- 172.21.15.43:0/2025593147 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f358c067d30 con 0x7f358c0b4110 2024-04-25T09:01:14.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.567+0000 7f3590fb0640 1 -- 172.21.15.43:0/2025593147 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f358c074ef0 con 0x7f358c0b4110 2024-04-25T09:01:14.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.567+0000 7f3582ffd640 1 -- 172.21.15.43:0/2025593147 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1636 (secure 0 0 0) 0x7f3574057e30 con 0x7f358c0b4110 2024-04-25T09:01:14.556 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:01:14.556 INFO:teuthology.orchestra.run.smithi043.stdout:{"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":446,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":91,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035238,"num_in_osds":8,"osd_in_since":1714035229,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1078426160,"bytes_used":2694152192,"bytes_avail":770399961088,"bytes_total":773094113280,"write_bytes_sec":30683048,"read_op_per_sec":0,"write_op_per_sec":31},"fsmap":{"epoch":54,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7596}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":16,"modified":"2024-04-25T09:01:09.790126+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/scrub_control_test_path]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:01:14.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.569+0000 7f3590fb0640 1 -- 172.21.15.43:0/2025593147 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f356c06b4d0 msgr2=0x7f356c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:14.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.569+0000 7f3590fb0640 1 --2- 172.21.15.43:0/2025593147 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f356c06b4d0 0x7f356c06d990 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3584034000 tx=0x7f358402e000 comp rx=0 tx=0).stop 2024-04-25T09:01:14.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.569+0000 7f3590fb0640 1 -- 172.21.15.43:0/2025593147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f358c0b4110 msgr2=0x7f358c0adf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:14.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.569+0000 7f3590fb0640 1 --2- 172.21.15.43:0/2025593147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f358c0b4110 0x7f358c0adf90 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f357400c430 tx=0x7f357400c8f0 comp rx=0 tx=0).stop 2024-04-25T09:01:14.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.569+0000 7f3590fb0640 1 -- 172.21.15.43:0/2025593147 shutdown_connections 2024-04-25T09:01:14.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.569+0000 7f3590fb0640 1 --2- 172.21.15.43:0/2025593147 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f358c15a250 0x7f358c15c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:14.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.569+0000 7f3590fb0640 1 --2- 172.21.15.43:0/2025593147 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f358c0b4110 0x7f358c0adf90 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:14.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.569+0000 7f3590fb0640 1 --2- 172.21.15.43:0/2025593147 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f356c06b4d0 0x7f356c06d990 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:14.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.569+0000 7f3590fb0640 1 --2- 172.21.15.43:0/2025593147 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f358c0715a0 0x7f358c0ad3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:14.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.569+0000 7f3590fb0640 1 -- 172.21.15.43:0/2025593147 >> 172.21.15.43:0/2025593147 conn(0x7f358c0b1480 msgr2=0x7f358c0ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:14.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.569+0000 7f3590fb0640 1 -- 172.21.15.43:0/2025593147 shutdown_connections 2024-04-25T09:01:14.559 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:14.569+0000 7f3590fb0640 1 -- 172.21.15.43:0/2025593147 wait complete. 2024-04-25T09:01:14.569 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'idle+waiting paths [/scrub_control_test_path]'} 2024-04-25T09:01:15.570 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:01:15.647 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:01:15.650 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:01:15.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.696+0000 7fdd6e1b0640 1 Processor -- start 2024-04-25T09:01:15.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.696+0000 7fdd6e1b0640 1 -- start start 2024-04-25T09:01:15.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.697+0000 7fdd6e1b0640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fdd6815bd70 0x7fdd680b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:15.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.697+0000 7fdd6e1b0640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdd680d13a0 0x7fdd680b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:15.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.697+0000 7fdd6e1b0640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd680b4230 con 0x7fdd6815b990 2024-04-25T09:01:15.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.697+0000 7fdd6e1b0640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd680cd120 con 0x7fdd6815bd70 2024-04-25T09:01:15.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.697+0000 7fdd6e1b0640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd680b44e0 con 0x7fdd680d13a0 2024-04-25T09:01:15.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.697+0000 7fdd677fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fdd6815b990 0x7fdd680d0c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47450/0 (socket says 172.21.15.43:47450) 2024-04-25T09:01:15.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.697+0000 7fdd677fe640 1 -- 172.21.15.43:0/1958109989 learned_addr learned my addr 172.21.15.43:0/1958109989 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:15.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.698+0000 7fdd5effd640 1 --2- 172.21.15.43:0/1958109989 >> v2:172.21.15.193:3300/0 conn(0x7fdd6815bd70 0x7fdd680b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:15.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.698+0000 7fdd67fff640 1 --2- 172.21.15.43:0/1958109989 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdd680d13a0 0x7fdd680b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:15.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.698+0000 7fdd66ffd640 1 -- 172.21.15.43:0/1958109989 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 784477127 0 0) 0x7fdd680b4230 con 0x7fdd6815b990 2024-04-25T09:01:15.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.698+0000 7fdd66ffd640 1 -- 172.21.15.43:0/1958109989 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd54003520 con 0x7fdd6815b990 2024-04-25T09:01:15.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.698+0000 7fdd5effd640 1 -- 172.21.15.43:0/1958109989 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdd680d13a0 msgr2=0x7fdd680b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:15.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.698+0000 7fdd5effd640 1 --2- 172.21.15.43:0/1958109989 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdd680d13a0 0x7fdd680b3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:15.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.698+0000 7fdd5effd640 1 -- 172.21.15.43:0/1958109989 >> v1:172.21.15.43:6789/0 conn(0x7fdd6815b990 legacy=0x7fdd680d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:15.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.698+0000 7fdd5effd640 1 -- 172.21.15.43:0/1958109989 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd680cd370 con 0x7fdd6815bd70 2024-04-25T09:01:15.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.698+0000 7fdd67fff640 1 --2- 172.21.15.43:0/1958109989 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdd680d13a0 0x7fdd680b3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:01:15.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.699+0000 7fdd5effd640 1 --2- 172.21.15.43:0/1958109989 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd6815bd70 0x7fdd680b37b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fdd500097a0 tx=0x7fdd5002a060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7342bc52f7aaa777 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:15.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.699+0000 7fdd66ffd640 1 -- 172.21.15.43:0/1958109989 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd50037070 con 0x7fdd6815bd70 2024-04-25T09:01:15.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.699+0000 7fdd66ffd640 1 -- 172.21.15.43:0/1958109989 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd5002adb0 con 0x7fdd6815bd70 2024-04-25T09:01:15.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.699+0000 7fdd66ffd640 1 -- 172.21.15.43:0/1958109989 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd50032640 con 0x7fdd6815bd70 2024-04-25T09:01:15.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.699+0000 7fdd6e1b0640 1 -- 172.21.15.43:0/1958109989 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd6815bd70 msgr2=0x7fdd680b37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:15.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.700+0000 7fdd6e1b0640 1 --2- 172.21.15.43:0/1958109989 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd6815bd70 0x7fdd680b37b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fdd500097a0 tx=0x7fdd5002a060 comp rx=0 tx=0).stop 2024-04-25T09:01:15.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.700+0000 7fdd6e1b0640 1 -- 172.21.15.43:0/1958109989 shutdown_connections 2024-04-25T09:01:15.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.700+0000 7fdd6e1b0640 1 --2- 172.21.15.43:0/1958109989 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdd680d13a0 0x7fdd680b3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:15.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.700+0000 7fdd6e1b0640 1 --2- 172.21.15.43:0/1958109989 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd6815bd70 0x7fdd680b37b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:15.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.700+0000 7fdd6e1b0640 1 -- 172.21.15.43:0/1958109989 >> 172.21.15.43:0/1958109989 conn(0x7fdd6806c090 msgr2=0x7fdd6806c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:15.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.700+0000 7fdd6e1b0640 1 -- 172.21.15.43:0/1958109989 shutdown_connections 2024-04-25T09:01:15.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.700+0000 7fdd6e1b0640 1 -- 172.21.15.43:0/1958109989 wait complete. 2024-04-25T09:01:15.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.701+0000 7fdd6e1b0640 1 Processor -- start 2024-04-25T09:01:15.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.701+0000 7fdd6e1b0640 1 -- start start 2024-04-25T09:01:15.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.702+0000 7fdd6e1b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd680d13a0 0x7fdd68144070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:15.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.702+0000 7fdd6e1b0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd6815b990 0x7fdd681566a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:15.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.702+0000 7fdd6e1b0640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdd6815bd70 0x7fdd68156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:15.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.702+0000 7fdd6e1b0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd680cde50 con 0x7fdd680d13a0 2024-04-25T09:01:15.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.702+0000 7fdd6e1b0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd680cd930 con 0x7fdd6815b990 2024-04-25T09:01:15.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.702+0000 7fdd6e1b0640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd680cdfd0 con 0x7fdd6815bd70 2024-04-25T09:01:15.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.702+0000 7fdd677fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd680d13a0 0x7fdd68144070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:15.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.702+0000 7fdd677fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd680d13a0 0x7fdd68144070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42524/0 (socket says 172.21.15.43:42524) 2024-04-25T09:01:15.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.702+0000 7fdd677fe640 1 -- 172.21.15.43:0/1809084707 learned_addr learned my addr 172.21.15.43:0/1809084707 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:15.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.702+0000 7fdd5effd640 1 --2- 172.21.15.43:0/1809084707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd6815b990 0x7fdd681566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:15.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.702+0000 7fdd67fff640 1 --2- 172.21.15.43:0/1809084707 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdd6815bd70 0x7fdd68156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:15.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.702+0000 7fdd677fe640 1 -- 172.21.15.43:0/1809084707 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdd6815bd70 msgr2=0x7fdd68156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:15.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.702+0000 7fdd677fe640 1 --2- 172.21.15.43:0/1809084707 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdd6815bd70 0x7fdd68156be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:15.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.702+0000 7fdd677fe640 1 -- 172.21.15.43:0/1809084707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd6815b990 msgr2=0x7fdd681566a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:15.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.702+0000 7fdd677fe640 1 --2- 172.21.15.43:0/1809084707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd6815b990 0x7fdd681566a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:15.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.702+0000 7fdd677fe640 1 -- 172.21.15.43:0/1809084707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd68154c30 con 0x7fdd680d13a0 2024-04-25T09:01:15.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.703+0000 7fdd677fe640 1 --2- 172.21.15.43:0/1809084707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd680d13a0 0x7fdd68144070 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fdd5800c460 tx=0x7fdd5800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:15.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.703+0000 7fdd64ff9640 1 -- 172.21.15.43:0/1809084707 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd58017070 con 0x7fdd680d13a0 2024-04-25T09:01:15.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.703+0000 7fdd64ff9640 1 -- 172.21.15.43:0/1809084707 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd58005ce0 con 0x7fdd680d13a0 2024-04-25T09:01:15.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.703+0000 7fdd6e1b0640 1 -- 172.21.15.43:0/1809084707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd68154f10 con 0x7fdd680d13a0 2024-04-25T09:01:15.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.703+0000 7fdd64ff9640 1 -- 172.21.15.43:0/1809084707 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd580072f0 con 0x7fdd680d13a0 2024-04-25T09:01:15.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.703+0000 7fdd6e1b0640 1 -- 172.21.15.43:0/1809084707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd680c9eb0 con 0x7fdd680d13a0 2024-04-25T09:01:15.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.704+0000 7fdd6e1b0640 1 -- 172.21.15.43:0/1809084707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd68074ef0 con 0x7fdd680d13a0 2024-04-25T09:01:15.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.704+0000 7fdd64ff9640 1 -- 172.21.15.43:0/1809084707 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdd5801a070 con 0x7fdd680d13a0 2024-04-25T09:01:15.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.705+0000 7fdd64ff9640 1 --2- 172.21.15.43:0/1809084707 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdd4806b4d0 0x7fdd4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:15.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.705+0000 7fdd64ff9640 1 -- 172.21.15.43:0/1809084707 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fdd58089560 con 0x7fdd680d13a0 2024-04-25T09:01:15.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.705+0000 7fdd5effd640 1 --2- 172.21.15.43:0/1809084707 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdd4806b4d0 0x7fdd4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:15.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.706+0000 7fdd5effd640 1 --2- 172.21.15.43:0/1809084707 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdd4806b4d0 0x7fdd4806d990 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fdd50034000 tx=0x7fdd5002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:15.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.709+0000 7fdd64ff9640 1 -- 172.21.15.43:0/1809084707 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdd58057d70 con 0x7fdd680d13a0 2024-04-25T09:01:15.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.927+0000 7fdd6e1b0640 1 -- 172.21.15.43:0/1809084707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdd680680a0 con 0x7fdd680d13a0 2024-04-25T09:01:15.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.927+0000 7fdd64ff9640 1 -- 172.21.15.43:0/1809084707 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fdd5805e0c0 con 0x7fdd680d13a0 2024-04-25T09:01:15.916 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:01:15.916 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":54,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6200,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2258031526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2258031526},{"type":"v1","addr":"172.21.15.43:6834","nonce":2258031526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6206,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/1773440177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1773440177},{"type":"v1","addr":"172.21.15.193:6836","nonce":1773440177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6209,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3239725307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":3239725307},{"type":"v1","addr":"172.21.15.193:6837","nonce":3239725307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:00:42.536578+0000","modified":"2024-04-25T09:00:44.531104+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7596},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7596":{"gid":7596,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/1341028696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1341028696},{"type":"v1","addr":"172.21.15.43:6835","nonce":1341028696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:01:15.916 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 54 2024-04-25T09:01:15.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.930+0000 7fdd6e1b0640 1 -- 172.21.15.43:0/1809084707 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdd4806b4d0 msgr2=0x7fdd4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:15.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.930+0000 7fdd6e1b0640 1 --2- 172.21.15.43:0/1809084707 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdd4806b4d0 0x7fdd4806d990 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fdd50034000 tx=0x7fdd5002e000 comp rx=0 tx=0).stop 2024-04-25T09:01:15.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.930+0000 7fdd6e1b0640 1 -- 172.21.15.43:0/1809084707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd680d13a0 msgr2=0x7fdd68144070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:15.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.930+0000 7fdd6e1b0640 1 --2- 172.21.15.43:0/1809084707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd680d13a0 0x7fdd68144070 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fdd5800c460 tx=0x7fdd5800c860 comp rx=0 tx=0).stop 2024-04-25T09:01:15.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.930+0000 7fdd6e1b0640 1 -- 172.21.15.43:0/1809084707 shutdown_connections 2024-04-25T09:01:15.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.930+0000 7fdd6e1b0640 1 --2- 172.21.15.43:0/1809084707 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdd6815bd70 0x7fdd68156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:15.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.930+0000 7fdd6e1b0640 1 --2- 172.21.15.43:0/1809084707 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdd4806b4d0 0x7fdd4806d990 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:15.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.930+0000 7fdd6e1b0640 1 --2- 172.21.15.43:0/1809084707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdd6815b990 0x7fdd681566a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:15.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.930+0000 7fdd6e1b0640 1 --2- 172.21.15.43:0/1809084707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdd680d13a0 0x7fdd68144070 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:15.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.931+0000 7fdd6e1b0640 1 -- 172.21.15.43:0/1809084707 >> 172.21.15.43:0/1809084707 conn(0x7fdd6806c090 msgr2=0x7fdd6806dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:15.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.931+0000 7fdd6e1b0640 1 -- 172.21.15.43:0/1809084707 shutdown_connections 2024-04-25T09:01:15.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:15.931+0000 7fdd6e1b0640 1 -- 172.21.15.43:0/1809084707 wait complete. 2024-04-25T09:01:15.935 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T09:01:15.935 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:01:16.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.047+0000 7fd6d03c1640 1 Processor -- start 2024-04-25T09:01:16.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.047+0000 7fd6d03c1640 1 -- start start 2024-04-25T09:01:16.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.048+0000 7fd6d03c1640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd6c80cf670 0x7fd6c8168d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.048+0000 7fd6d03c1640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6c8162810 con 0x7fd6c815df50 2024-04-25T09:01:16.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.048+0000 7fd6d03c1640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6c80d24a0 con 0x7fd6c80c8d00 2024-04-25T09:01:16.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.048+0000 7fd6d03c1640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c815e330 con 0x7fd6c80cf670 2024-04-25T09:01:16.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.048+0000 7fd6ce15d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fd6c815df50 0x7fd6c81619f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47454/0 (socket says 172.21.15.43:47454) 2024-04-25T09:01:16.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.048+0000 7fd6ce15d640 1 -- 172.21.15.43:0/2558868898 learned_addr learned my addr 172.21.15.43:0/2558868898 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:16.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.048+0000 7fd6cd15b640 1 -- 172.21.15.43:0/2558868898 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 449937177 0 0) 0x7fd6c8162810 con 0x7fd6c815df50 2024-04-25T09:01:16.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.048+0000 7fd6ce95e640 1 --2- 172.21.15.43:0/2558868898 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd6c80cf670 0x7fd6c8168d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.048+0000 7fd6cd15b640 1 -- 172.21.15.43:0/2558868898 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6b4003520 con 0x7fd6c815df50 2024-04-25T09:01:16.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.049+0000 7fd6cd15b640 1 -- 172.21.15.43:0/2558868898 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2585070455 0 0) 0x7fd6b4003520 con 0x7fd6c815df50 2024-04-25T09:01:16.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.049+0000 7fd6cd15b640 1 -- 172.21.15.43:0/2558868898 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd6c80cf670 msgr2=0x7fd6c8168d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:16.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.049+0000 7fd6cd15b640 1 --2- 172.21.15.43:0/2558868898 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd6c80cf670 0x7fd6c8168d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.049+0000 7fd6cd15b640 1 -- 172.21.15.43:0/2558868898 >> v1:172.21.15.193:6789/0 conn(0x7fd6c80c8d00 legacy=0x7fd6c8162100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:16.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.049+0000 7fd6cd15b640 1 -- 172.21.15.43:0/2558868898 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c80d2740 con 0x7fd6c815df50 2024-04-25T09:01:16.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.049+0000 7fd6cd15b640 1 -- 172.21.15.43:0/2558868898 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7fd6b8001670 con 0x7fd6c815df50 2024-04-25T09:01:16.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.049+0000 7fd6cd15b640 1 -- 172.21.15.43:0/2558868898 >> v1:172.21.15.43:6789/0 conn(0x7fd6c815df50 legacy=0x7fd6c81619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:16.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.049+0000 7fd6cd15b640 1 --2- 172.21.15.43:0/2558868898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6b40039e0 0x7fd6b4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.049+0000 7fd6cd15b640 1 --2- 172.21.15.43:0/2558868898 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd6c80cf670 0x7fd6b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.049+0000 7fd6cd15b640 1 --2- 172.21.15.43:0/2558868898 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6b4004d40 0x7fd6b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.049+0000 7fd6cd15b640 1 -- 172.21.15.43:0/2558868898 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c815e330 con 0x7fd6b40039e0 2024-04-25T09:01:16.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.049+0000 7fd6cd15b640 1 -- 172.21.15.43:0/2558868898 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b400b9c0 con 0x7fd6b4004d40 2024-04-25T09:01:16.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.049+0000 7fd6cd15b640 1 -- 172.21.15.43:0/2558868898 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6b400bbc0 con 0x7fd6c80cf670 2024-04-25T09:01:16.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.049+0000 7fd6ce15d640 1 --2- 172.21.15.43:0/2558868898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6b40039e0 0x7fd6b4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.049+0000 7fd6cd95c640 1 --2- 172.21.15.43:0/2558868898 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd6c80cf670 0x7fd6b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.049+0000 7fd6ce95e640 1 --2- 172.21.15.43:0/2558868898 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6b4004d40 0x7fd6b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.049+0000 7fd6ce15d640 1 -- 172.21.15.43:0/2558868898 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd6c80cf670 msgr2=0x7fd6b4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:16.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.049+0000 7fd6ce15d640 1 --2- 172.21.15.43:0/2558868898 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd6c80cf670 0x7fd6b4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.049+0000 7fd6ce15d640 1 -- 172.21.15.43:0/2558868898 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6b4004d40 msgr2=0x7fd6b40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:16.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.049+0000 7fd6ce15d640 1 --2- 172.21.15.43:0/2558868898 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6b4004d40 0x7fd6b40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.049+0000 7fd6ce15d640 1 -- 172.21.15.43:0/2558868898 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6b4003520 con 0x7fd6b40039e0 2024-04-25T09:01:16.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.049+0000 7fd6ce95e640 1 --2- 172.21.15.43:0/2558868898 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6b4004d40 0x7fd6b40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:01:16.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.050+0000 7fd6ce15d640 1 --2- 172.21.15.43:0/2558868898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6b40039e0 0x7fd6b4003dc0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fd6b800c290 tx=0x7fd6b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b38159a7145a1ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:16.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.050+0000 7fd6cd15b640 1 -- 172.21.15.43:0/2558868898 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b8007d50 con 0x7fd6b40039e0 2024-04-25T09:01:16.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.050+0000 7fd6cd15b640 1 -- 172.21.15.43:0/2558868898 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6b8005af0 con 0x7fd6b40039e0 2024-04-25T09:01:16.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.050+0000 7fd6cd15b640 1 -- 172.21.15.43:0/2558868898 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6b8007070 con 0x7fd6b40039e0 2024-04-25T09:01:16.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.051+0000 7fd6d03c1640 1 -- 172.21.15.43:0/2558868898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6b40039e0 msgr2=0x7fd6b4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:16.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.051+0000 7fd6d03c1640 1 --2- 172.21.15.43:0/2558868898 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6b40039e0 0x7fd6b4003dc0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fd6b800c290 tx=0x7fd6b4005830 comp rx=0 tx=0).stop 2024-04-25T09:01:16.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.051+0000 7fd6ce95e640 1 -- 172.21.15.43:0/2558868898 reap_dead start 2024-04-25T09:01:16.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.051+0000 7fd6d03c1640 1 -- 172.21.15.43:0/2558868898 shutdown_connections 2024-04-25T09:01:16.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.051+0000 7fd6d03c1640 1 -- 172.21.15.43:0/2558868898 >> 172.21.15.43:0/2558868898 conn(0x7fd6c80d3a30 msgr2=0x7fd6c80d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:16.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.051+0000 7fd6d03c1640 1 -- 172.21.15.43:0/2558868898 shutdown_connections 2024-04-25T09:01:16.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.051+0000 7fd6d03c1640 1 -- 172.21.15.43:0/2558868898 wait complete. 2024-04-25T09:01:16.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.052+0000 7fd6d03c1640 1 Processor -- start 2024-04-25T09:01:16.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.052+0000 7fd6d03c1640 1 -- start start 2024-04-25T09:01:16.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.053+0000 7fd6d03c1640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd6b40039e0 0x7fd6c8159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.053+0000 7fd6d03c1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6c809fe60 0x7fd6c80a02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.053+0000 7fd6d03c1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6c814e7c0 0x7fd6c814ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.053+0000 7fd6d03c1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c80c77d0 con 0x7fd6c809fe60 2024-04-25T09:01:16.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.053+0000 7fd6d03c1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c80c74f0 con 0x7fd6c814e7c0 2024-04-25T09:01:16.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.053+0000 7fd6d03c1640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6c80d2190 con 0x7fd6b40039e0 2024-04-25T09:01:16.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.053+0000 7fd6cd95c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6c809fe60 0x7fd6c80a02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.053+0000 7fd6cd95c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6c809fe60 0x7fd6c80a02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42532/0 (socket says 172.21.15.43:42532) 2024-04-25T09:01:16.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.053+0000 7fd6cd95c640 1 -- 172.21.15.43:0/913204386 learned_addr learned my addr 172.21.15.43:0/913204386 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:16.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.053+0000 7fd6ce15d640 1 --2- 172.21.15.43:0/913204386 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd6b40039e0 0x7fd6c8159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.053+0000 7fd6ce95e640 1 --2- 172.21.15.43:0/913204386 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6c814e7c0 0x7fd6c814ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.054+0000 7fd6cd95c640 1 -- 172.21.15.43:0/913204386 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd6b40039e0 msgr2=0x7fd6c8159de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:16.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.054+0000 7fd6cd95c640 1 --2- 172.21.15.43:0/913204386 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd6b40039e0 0x7fd6c8159de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.054+0000 7fd6cd95c640 1 -- 172.21.15.43:0/913204386 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6c814e7c0 msgr2=0x7fd6c814ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:16.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.054+0000 7fd6cd95c640 1 --2- 172.21.15.43:0/913204386 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6c814e7c0 0x7fd6c814ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.054+0000 7fd6cd95c640 1 -- 172.21.15.43:0/913204386 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c80a0940 con 0x7fd6c809fe60 2024-04-25T09:01:16.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.054+0000 7fd6cd95c640 1 --2- 172.21.15.43:0/913204386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6c809fe60 0x7fd6c80a02e0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fd6c400f830 tx=0x7fd6c400fcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:16.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.054+0000 7fd6beffd640 1 -- 172.21.15.43:0/913204386 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c40052e0 con 0x7fd6c809fe60 2024-04-25T09:01:16.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.054+0000 7fd6beffd640 1 -- 172.21.15.43:0/913204386 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c4018070 con 0x7fd6c809fe60 2024-04-25T09:01:16.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.054+0000 7fd6d03c1640 1 -- 172.21.15.43:0/913204386 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6c80a0c20 con 0x7fd6c809fe60 2024-04-25T09:01:16.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.054+0000 7fd6beffd640 1 -- 172.21.15.43:0/913204386 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c400d910 con 0x7fd6c809fe60 2024-04-25T09:01:16.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.054+0000 7fd6d03c1640 1 -- 172.21.15.43:0/913204386 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6c80a11e0 con 0x7fd6c809fe60 2024-04-25T09:01:16.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.055+0000 7fd6d03c1640 1 -- 172.21.15.43:0/913204386 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6c80680f0 con 0x7fd6c809fe60 2024-04-25T09:01:16.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.056+0000 7fd6beffd640 1 -- 172.21.15.43:0/913204386 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd6c400dab0 con 0x7fd6c809fe60 2024-04-25T09:01:16.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.056+0000 7fd6beffd640 1 --2- 172.21.15.43:0/913204386 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd6ac06b4d0 0x7fd6ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.056+0000 7fd6beffd640 1 -- 172.21.15.43:0/913204386 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fd6c408aa10 con 0x7fd6c809fe60 2024-04-25T09:01:16.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.056+0000 7fd6ce15d640 1 --2- 172.21.15.43:0/913204386 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd6ac06b4d0 0x7fd6ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.057+0000 7fd6ce15d640 1 --2- 172.21.15.43:0/913204386 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd6ac06b4d0 0x7fd6ac06d990 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8032000 tx=0x7fd6b8034040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:16.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.060+0000 7fd6beffd640 1 -- 172.21.15.43:0/913204386 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd6c4059390 con 0x7fd6c809fe60 2024-04-25T09:01:16.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.314+0000 7fd6d03c1640 1 -- 172.21.15.43:0/913204386 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fd6c8074ef0 con 0x7fd6c809fe60 2024-04-25T09:01:16.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.314+0000 7fd6beffd640 1 -- 172.21.15.43:0/913204386 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1570 (secure 0 0 0) 0x7fd6c405d430 con 0x7fd6c809fe60 2024-04-25T09:01:16.303 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:01:16.303 INFO:teuthology.orchestra.run.smithi043.stdout:{"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":448,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":91,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035238,"num_in_osds":8,"osd_in_since":1714035229,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1078426160,"bytes_used":2694152192,"bytes_avail":770399961088,"bytes_total":773094113280,"write_bytes_sec":81300,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":54,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7596}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":17,"modified":"2024-04-25T09:01:15.791620+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:01:16.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.316+0000 7fd6d03c1640 1 -- 172.21.15.43:0/913204386 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd6ac06b4d0 msgr2=0x7fd6ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:16.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.316+0000 7fd6d03c1640 1 --2- 172.21.15.43:0/913204386 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd6ac06b4d0 0x7fd6ac06d990 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd6b8032000 tx=0x7fd6b8034040 comp rx=0 tx=0).stop 2024-04-25T09:01:16.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.316+0000 7fd6d03c1640 1 -- 172.21.15.43:0/913204386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6c809fe60 msgr2=0x7fd6c80a02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:16.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.316+0000 7fd6d03c1640 1 --2- 172.21.15.43:0/913204386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6c809fe60 0x7fd6c80a02e0 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fd6c400f830 tx=0x7fd6c400fcf0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.316+0000 7fd6d03c1640 1 -- 172.21.15.43:0/913204386 shutdown_connections 2024-04-25T09:01:16.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.316+0000 7fd6d03c1640 1 --2- 172.21.15.43:0/913204386 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd6ac06b4d0 0x7fd6ac06d990 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.316+0000 7fd6d03c1640 1 --2- 172.21.15.43:0/913204386 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd6c814e7c0 0x7fd6c814ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.316+0000 7fd6d03c1640 1 --2- 172.21.15.43:0/913204386 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd6c809fe60 0x7fd6c80a02e0 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.316+0000 7fd6d03c1640 1 --2- 172.21.15.43:0/913204386 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd6b40039e0 0x7fd6c8159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.316+0000 7fd6d03c1640 1 -- 172.21.15.43:0/913204386 >> 172.21.15.43:0/913204386 conn(0x7fd6c80d3a30 msgr2=0x7fd6c80c97b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:16.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.316+0000 7fd6d03c1640 1 -- 172.21.15.43:0/913204386 shutdown_connections 2024-04-25T09:01:16.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.316+0000 7fd6d03c1640 1 -- 172.21.15.43:0/913204386 wait complete. 2024-04-25T09:01:16.316 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T09:01:16.317 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:01:16.317 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:01:16.355 DEBUG:teuthology.orchestra.run.smithi193:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:01:16.396 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:01:16.396 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T09:01:16.396 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:01:16.412 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:01:16.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.565+0000 7f0dfc7b0640 1 Processor -- start 2024-04-25T09:01:16.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.565+0000 7f0dfc7b0640 1 -- start start 2024-04-25T09:01:16.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.566+0000 7f0dfc7b0640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0df406d9a0 0x7f0df4071a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.566+0000 7f0dfc7b0640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0df4071f90 0x7f0df40cd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.566+0000 7f0dfc7b0640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0df4072420 con 0x7f0df415a9d0 2024-04-25T09:01:16.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.566+0000 7f0dfc7b0640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0df40704f0 con 0x7f0df4071f90 2024-04-25T09:01:16.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.566+0000 7f0dfc7b0640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0df40707d0 con 0x7f0df406d9a0 2024-04-25T09:01:16.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.566+0000 7f0dfad4d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0df4071f90 0x7f0df40cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.566+0000 7f0dfad4d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0df4071f90 0x7f0df40cd940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42536/0 (socket says 172.21.15.43:42536) 2024-04-25T09:01:16.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.566+0000 7f0dfad4d640 1 -- 172.21.15.43:0/2219993509 learned_addr learned my addr 172.21.15.43:0/2219993509 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:16.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.566+0000 7f0df9d4b640 1 --2- 172.21.15.43:0/2219993509 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0df406d9a0 0x7f0df4071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.566+0000 7f0dfad4d640 1 -- 172.21.15.43:0/2219993509 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0df406d9a0 msgr2=0x7f0df4071a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:16.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.566+0000 7f0dfad4d640 1 --2- 172.21.15.43:0/2219993509 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0df406d9a0 0x7f0df4071a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.566+0000 7f0dfad4d640 1 -- 172.21.15.43:0/2219993509 >> v1:172.21.15.193:6789/0 conn(0x7f0df415a9d0 legacy=0x7f0df406d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:16.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.566+0000 7f0dfad4d640 1 -- 172.21.15.43:0/2219993509 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df4070ab0 con 0x7f0df4071f90 2024-04-25T09:01:16.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.567+0000 7f0dfad4d640 1 --2- 172.21.15.43:0/2219993509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0df4071f90 0x7f0df40cd940 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f0dec009e50 tx=0x7f0dec02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fbcba353fd9a35c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:16.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.567+0000 7f0df954a640 1 -- 172.21.15.43:0/2219993509 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dec036070 con 0x7f0df4071f90 2024-04-25T09:01:16.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.567+0000 7f0df954a640 1 -- 172.21.15.43:0/2219993509 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dec031600 con 0x7f0df4071f90 2024-04-25T09:01:16.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.567+0000 7f0df954a640 1 -- 172.21.15.43:0/2219993509 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dec0319d0 con 0x7f0df4071f90 2024-04-25T09:01:16.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.568+0000 7f0dfc7b0640 1 -- 172.21.15.43:0/2219993509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0df4071f90 msgr2=0x7f0df40cd940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:16.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.568+0000 7f0dfc7b0640 1 --2- 172.21.15.43:0/2219993509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0df4071f90 0x7f0df40cd940 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f0dec009e50 tx=0x7f0dec02a710 comp rx=0 tx=0).stop 2024-04-25T09:01:16.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.568+0000 7f0dfc7b0640 1 -- 172.21.15.43:0/2219993509 shutdown_connections 2024-04-25T09:01:16.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.568+0000 7f0dfc7b0640 1 --2- 172.21.15.43:0/2219993509 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0df4071f90 0x7f0df40cd940 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.568+0000 7f0dfc7b0640 1 --2- 172.21.15.43:0/2219993509 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0df406d9a0 0x7f0df4071a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.568+0000 7f0dfc7b0640 1 -- 172.21.15.43:0/2219993509 >> 172.21.15.43:0/2219993509 conn(0x7f0df40b1480 msgr2=0x7f0df40b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:16.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.568+0000 7f0dfc7b0640 1 -- 172.21.15.43:0/2219993509 shutdown_connections 2024-04-25T09:01:16.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.568+0000 7f0dfc7b0640 1 -- 172.21.15.43:0/2219993509 wait complete. 2024-04-25T09:01:16.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.569+0000 7f0dfc7b0640 1 Processor -- start 2024-04-25T09:01:16.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.569+0000 7f0dfc7b0640 1 -- start start 2024-04-25T09:01:16.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.570+0000 7f0dfc7b0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0df406d9a0 0x7f0df4144ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.570+0000 7f0dfc7b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0df4071f90 0x7f0df41e7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.570+0000 7f0dfc7b0640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0df41e78d0 0x7f0df41519c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.570+0000 7f0dfc7b0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df40cde80 con 0x7f0df4071f90 2024-04-25T09:01:16.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.570+0000 7f0dfc7b0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df40712b0 con 0x7f0df406d9a0 2024-04-25T09:01:16.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.570+0000 7f0dfc7b0640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0df40ce000 con 0x7f0df41e78d0 2024-04-25T09:01:16.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.570+0000 7f0df9d4b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0df4071f90 0x7f0df41e7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.570+0000 7f0df9d4b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0df4071f90 0x7f0df41e7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42550/0 (socket says 172.21.15.43:42550) 2024-04-25T09:01:16.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.570+0000 7f0df9d4b640 1 -- 172.21.15.43:0/2277090496 learned_addr learned my addr 172.21.15.43:0/2277090496 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:16.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.570+0000 7f0dfa54c640 1 --2- 172.21.15.43:0/2277090496 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0df406d9a0 0x7f0df4144ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.570+0000 7f0dfad4d640 1 --2- 172.21.15.43:0/2277090496 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0df41e78d0 0x7f0df41519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.570+0000 7f0df9d4b640 1 -- 172.21.15.43:0/2277090496 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0df41e78d0 msgr2=0x7f0df41519c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:16.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.570+0000 7f0df9d4b640 1 --2- 172.21.15.43:0/2277090496 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0df41e78d0 0x7f0df41519c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.570+0000 7f0df9d4b640 1 -- 172.21.15.43:0/2277090496 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0df406d9a0 msgr2=0x7f0df4144ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:16.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.570+0000 7f0df9d4b640 1 --2- 172.21.15.43:0/2277090496 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0df406d9a0 0x7f0df4144ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.570+0000 7f0df9d4b640 1 -- 172.21.15.43:0/2277090496 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df41503e0 con 0x7f0df4071f90 2024-04-25T09:01:16.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.570+0000 7f0dfad4d640 1 --2- 172.21.15.43:0/2277090496 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0df41e78d0 0x7f0df41519c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:16.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.571+0000 7f0df9d4b640 1 --2- 172.21.15.43:0/2277090496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0df4071f90 0x7f0df41e7390 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f0df000d830 tx=0x7f0df000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:16.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.571+0000 7f0deaffd640 1 -- 172.21.15.43:0/2277090496 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df000b6c0 con 0x7f0df4071f90 2024-04-25T09:01:16.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.571+0000 7f0deaffd640 1 -- 172.21.15.43:0/2277090496 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0df0019070 con 0x7f0df4071f90 2024-04-25T09:01:16.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.571+0000 7f0dfc7b0640 1 -- 172.21.15.43:0/2277090496 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0df41525a0 con 0x7f0df4071f90 2024-04-25T09:01:16.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.571+0000 7f0deaffd640 1 -- 172.21.15.43:0/2277090496 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df000ba80 con 0x7f0df4071f90 2024-04-25T09:01:16.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.571+0000 7f0dfc7b0640 1 -- 172.21.15.43:0/2277090496 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0df4152b60 con 0x7f0df4071f90 2024-04-25T09:01:16.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.572+0000 7f0dfc7b0640 1 -- 172.21.15.43:0/2277090496 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0df4074ef0 con 0x7f0df4071f90 2024-04-25T09:01:16.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.572+0000 7f0deaffd640 1 -- 172.21.15.43:0/2277090496 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0df001e460 con 0x7f0df4071f90 2024-04-25T09:01:16.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.573+0000 7f0deaffd640 1 --2- 172.21.15.43:0/2277090496 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0dd806b4d0 0x7f0dd806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.573+0000 7f0deaffd640 1 -- 172.21.15.43:0/2277090496 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f0df008a970 con 0x7f0df4071f90 2024-04-25T09:01:16.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.573+0000 7f0dfa54c640 1 --2- 172.21.15.43:0/2277090496 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0dd806b4d0 0x7f0dd806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.573+0000 7f0dfa54c640 1 --2- 172.21.15.43:0/2277090496 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0dd806b4d0 0x7f0dd806d990 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0de4007fd0 tx=0x7f0de4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:16.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.577+0000 7f0deaffd640 1 -- 172.21.15.43:0/2277090496 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0df0059730 con 0x7f0df4071f90 2024-04-25T09:01:16.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.759+0000 7f0dfc7b0640 1 -- 172.21.15.43:0/2277090496 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0df40b0920 con 0x7f0df4071f90 2024-04-25T09:01:16.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.760+0000 7f0deaffd640 1 -- 172.21.15.43:0/2277090496 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+16674 (secure 0 0 0) 0x7f0df0059030 con 0x7f0df4071f90 2024-04-25T09:01:16.749 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:01:16.749 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":91,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:00:54.209500+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:00:40.816230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T09:00:41.508021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.26","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:01:16.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.762+0000 7f0dfc7b0640 1 -- 172.21.15.43:0/2277090496 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0dd806b4d0 msgr2=0x7f0dd806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:16.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.762+0000 7f0dfc7b0640 1 --2- 172.21.15.43:0/2277090496 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0dd806b4d0 0x7f0dd806d990 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0de4007fd0 tx=0x7f0de4009040 comp rx=0 tx=0).stop 2024-04-25T09:01:16.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.762+0000 7f0dfc7b0640 1 -- 172.21.15.43:0/2277090496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0df4071f90 msgr2=0x7f0df41e7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:16.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.762+0000 7f0dfc7b0640 1 --2- 172.21.15.43:0/2277090496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0df4071f90 0x7f0df41e7390 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f0df000d830 tx=0x7f0df000dcf0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.762+0000 7f0dfc7b0640 1 -- 172.21.15.43:0/2277090496 shutdown_connections 2024-04-25T09:01:16.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.762+0000 7f0dfc7b0640 1 --2- 172.21.15.43:0/2277090496 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0df41e78d0 0x7f0df41519c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.762+0000 7f0dfc7b0640 1 --2- 172.21.15.43:0/2277090496 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0df4071f90 0x7f0df41e7390 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.762+0000 7f0dfc7b0640 1 --2- 172.21.15.43:0/2277090496 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0dd806b4d0 0x7f0dd806d990 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.762+0000 7f0dfc7b0640 1 --2- 172.21.15.43:0/2277090496 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0df406d9a0 0x7f0df4144ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.762+0000 7f0dfc7b0640 1 -- 172.21.15.43:0/2277090496 >> 172.21.15.43:0/2277090496 conn(0x7f0df40b1480 msgr2=0x7f0df40ce380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:16.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.762+0000 7f0dfc7b0640 1 -- 172.21.15.43:0/2277090496 shutdown_connections 2024-04-25T09:01:16.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.762+0000 7f0dfc7b0640 1 -- 172.21.15.43:0/2277090496 wait complete. 2024-04-25T09:01:16.761 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:00:40.816230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:00:41.508021+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:01:16.762 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:01:16.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.878+0000 7f9cc3322640 1 Processor -- start 2024-04-25T09:01:16.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.879+0000 7f9cc3322640 1 -- start start 2024-04-25T09:01:16.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.879+0000 7f9cc3322640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9cbc0b5560 0x7f9cbc0b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.879+0000 7f9cc3322640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9cbc0d2230 0x7f9cbc0d26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.879+0000 7f9cc3322640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cbc162610 con 0x7f9cbc15e090 2024-04-25T09:01:16.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.879+0000 7f9cc3322640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cbc0d1da0 con 0x7f9cbc0b5560 2024-04-25T09:01:16.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.879+0000 7f9cc3322640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cbc0d1750 con 0x7f9cbc0d2230 2024-04-25T09:01:16.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.879+0000 7f9cc10be640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f9cbc15e090 0x7f9cbc161aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47466/0 (socket says 172.21.15.43:47466) 2024-04-25T09:01:16.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.879+0000 7f9cc10be640 1 -- 172.21.15.43:0/4218816541 learned_addr learned my addr 172.21.15.43:0/4218816541 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:16.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.879+0000 7f9cc08bd640 1 --2- 172.21.15.43:0/4218816541 >> v2:172.21.15.43:3300/0 conn(0x7f9cbc0b5560 0x7f9cbc0b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.880+0000 7f9cc18bf640 1 --2- 172.21.15.43:0/4218816541 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9cbc0d2230 0x7f9cbc0d26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.880+0000 7f9caffff640 1 -- 172.21.15.43:0/4218816541 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1704385924 0 0) 0x7f9cbc162610 con 0x7f9cbc15e090 2024-04-25T09:01:16.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.880+0000 7f9caffff640 1 -- 172.21.15.43:0/4218816541 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ca8003520 con 0x7f9cbc15e090 2024-04-25T09:01:16.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.880+0000 7f9cc08bd640 1 -- 172.21.15.43:0/4218816541 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9cbc0d2230 msgr2=0x7f9cbc0d26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:16.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.880+0000 7f9cc08bd640 1 --2- 172.21.15.43:0/4218816541 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9cbc0d2230 0x7f9cbc0d26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.880+0000 7f9cc08bd640 1 -- 172.21.15.43:0/4218816541 >> v1:172.21.15.43:6789/0 conn(0x7f9cbc15e090 legacy=0x7f9cbc161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:16.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.880+0000 7f9cc08bd640 1 -- 172.21.15.43:0/4218816541 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cbc0d1a30 con 0x7f9cbc0b5560 2024-04-25T09:01:16.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.881+0000 7f9cc08bd640 1 --2- 172.21.15.43:0/4218816541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cbc0b5560 0x7f9cbc0b59e0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f9cb00097a0 tx=0x7f9cb002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc1cca439bd6538 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:16.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.881+0000 7f9caffff640 1 -- 172.21.15.43:0/4218816541 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0037070 con 0x7f9cbc0b5560 2024-04-25T09:01:16.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.881+0000 7f9caffff640 1 -- 172.21.15.43:0/4218816541 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cb0032440 con 0x7f9cbc0b5560 2024-04-25T09:01:16.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.881+0000 7f9caffff640 1 -- 172.21.15.43:0/4218816541 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0032810 con 0x7f9cbc0b5560 2024-04-25T09:01:16.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.881+0000 7f9cc3322640 1 -- 172.21.15.43:0/4218816541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cbc0b5560 msgr2=0x7f9cbc0b59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:16.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.881+0000 7f9cc3322640 1 --2- 172.21.15.43:0/4218816541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cbc0b5560 0x7f9cbc0b59e0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f9cb00097a0 tx=0x7f9cb002a210 comp rx=0 tx=0).stop 2024-04-25T09:01:16.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.881+0000 7f9cc3322640 1 -- 172.21.15.43:0/4218816541 shutdown_connections 2024-04-25T09:01:16.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.881+0000 7f9cc3322640 1 --2- 172.21.15.43:0/4218816541 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9cbc0d2230 0x7f9cbc0d26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.881+0000 7f9cc3322640 1 --2- 172.21.15.43:0/4218816541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cbc0b5560 0x7f9cbc0b59e0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.881+0000 7f9cc3322640 1 -- 172.21.15.43:0/4218816541 >> 172.21.15.43:0/4218816541 conn(0x7f9cbc0d0560 msgr2=0x7f9cbc0d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:16.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.881+0000 7f9cc3322640 1 -- 172.21.15.43:0/4218816541 shutdown_connections 2024-04-25T09:01:16.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.881+0000 7f9cc3322640 1 -- 172.21.15.43:0/4218816541 wait complete. 2024-04-25T09:01:16.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.883+0000 7f9cc3322640 1 Processor -- start 2024-04-25T09:01:16.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.883+0000 7f9cc3322640 1 -- start start 2024-04-25T09:01:16.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.883+0000 7f9cc3322640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9cbc0b5560 0x7f9cbc1518e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.883+0000 7f9cc3322640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cbc0d2230 0x7f9cbc0b4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.883+0000 7f9cc3322640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cbc15e090 0x7f9cbc0b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.883+0000 7f9cc3322640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cbc1621b0 con 0x7f9cbc0d2230 2024-04-25T09:01:16.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.883+0000 7f9cc3322640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cbc0b3280 con 0x7f9cbc15e090 2024-04-25T09:01:16.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.883+0000 7f9cc3322640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cbc162330 con 0x7f9cbc0b5560 2024-04-25T09:01:16.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.883+0000 7f9cc08bd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cbc0d2230 0x7f9cbc0b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.883+0000 7f9cc08bd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cbc0d2230 0x7f9cbc0b4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42582/0 (socket says 172.21.15.43:42582) 2024-04-25T09:01:16.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.883+0000 7f9cc08bd640 1 -- 172.21.15.43:0/1024832120 learned_addr learned my addr 172.21.15.43:0/1024832120 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:16.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.884+0000 7f9cc18bf640 1 --2- 172.21.15.43:0/1024832120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cbc15e090 0x7f9cbc0b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.884+0000 7f9cc10be640 1 --2- 172.21.15.43:0/1024832120 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9cbc0b5560 0x7f9cbc1518e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.884+0000 7f9cc08bd640 1 -- 172.21.15.43:0/1024832120 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9cbc0b5560 msgr2=0x7f9cbc1518e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:16.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.884+0000 7f9cc08bd640 1 --2- 172.21.15.43:0/1024832120 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9cbc0b5560 0x7f9cbc1518e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.884+0000 7f9cc08bd640 1 -- 172.21.15.43:0/1024832120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cbc15e090 msgr2=0x7f9cbc0b4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:16.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.884+0000 7f9cc08bd640 1 --2- 172.21.15.43:0/1024832120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cbc15e090 0x7f9cbc0b4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:16.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.884+0000 7f9cc08bd640 1 -- 172.21.15.43:0/1024832120 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cbc1f3780 con 0x7f9cbc0d2230 2024-04-25T09:01:16.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.884+0000 7f9cc08bd640 1 --2- 172.21.15.43:0/1024832120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cbc0d2230 0x7f9cbc0b4580 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0035040 tx=0x7f9cb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:16.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.884+0000 7f9cadffb640 1 -- 172.21.15.43:0/1024832120 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0037040 con 0x7f9cbc0d2230 2024-04-25T09:01:16.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.884+0000 7f9cadffb640 1 -- 172.21.15.43:0/1024832120 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cb0030500 con 0x7f9cbc0d2230 2024-04-25T09:01:16.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.884+0000 7f9cadffb640 1 -- 172.21.15.43:0/1024832120 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0030910 con 0x7f9cbc0d2230 2024-04-25T09:01:16.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.884+0000 7f9cc3322640 1 -- 172.21.15.43:0/1024832120 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cbc0b79e0 con 0x7f9cbc0d2230 2024-04-25T09:01:16.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.884+0000 7f9cc3322640 1 -- 172.21.15.43:0/1024832120 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cbc0c9ed0 con 0x7f9cbc0d2230 2024-04-25T09:01:16.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.885+0000 7f9cc3322640 1 -- 172.21.15.43:0/1024832120 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cbc06bf00 con 0x7f9cbc0d2230 2024-04-25T09:01:16.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.886+0000 7f9cadffb640 1 -- 172.21.15.43:0/1024832120 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9cb0044020 con 0x7f9cbc0d2230 2024-04-25T09:01:16.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.886+0000 7f9cadffb640 1 --2- 172.21.15.43:0/1024832120 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9ca006b4d0 0x7f9ca006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:16.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.886+0000 7f9cadffb640 1 -- 172.21.15.43:0/1024832120 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f9cb00aa350 con 0x7f9cbc0d2230 2024-04-25T09:01:16.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.886+0000 7f9cc10be640 1 --2- 172.21.15.43:0/1024832120 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9ca006b4d0 0x7f9ca006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:16.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.887+0000 7f9cc10be640 1 --2- 172.21.15.43:0/1024832120 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9ca006b4d0 0x7f9ca006d990 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8005880 tx=0x7f9cb8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:16.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:16.891+0000 7f9cadffb640 1 -- 172.21.15.43:0/1024832120 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9cbc06bf00 con 0x7f9cbc0d2230 2024-04-25T09:01:17.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.069+0000 7f9cc3322640 1 -- 172.21.15.43:0/1024832120 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9cbc074f60 con 0x7f9cbc0d2230 2024-04-25T09:01:17.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.069+0000 7f9cadffb640 1 -- 172.21.15.43:0/1024832120 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9cb0078b60 con 0x7f9cbc0d2230 2024-04-25T09:01:17.058 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T09:01:17.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.071+0000 7f9cc3322640 1 -- 172.21.15.43:0/1024832120 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9ca006b4d0 msgr2=0x7f9ca006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:17.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.071+0000 7f9cc3322640 1 --2- 172.21.15.43:0/1024832120 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9ca006b4d0 0x7f9ca006d990 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8005880 tx=0x7f9cb8009040 comp rx=0 tx=0).stop 2024-04-25T09:01:17.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.071+0000 7f9cc3322640 1 -- 172.21.15.43:0/1024832120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cbc0d2230 msgr2=0x7f9cbc0b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:17.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.071+0000 7f9cc3322640 1 --2- 172.21.15.43:0/1024832120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cbc0d2230 0x7f9cbc0b4580 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0035040 tx=0x7f9cb00046c0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.071+0000 7f9cc3322640 1 -- 172.21.15.43:0/1024832120 shutdown_connections 2024-04-25T09:01:17.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.071+0000 7f9cc3322640 1 --2- 172.21.15.43:0/1024832120 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9ca006b4d0 0x7f9ca006d990 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.071+0000 7f9cc3322640 1 --2- 172.21.15.43:0/1024832120 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9cbc15e090 0x7f9cbc0b4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.071+0000 7f9cc3322640 1 --2- 172.21.15.43:0/1024832120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9cbc0d2230 0x7f9cbc0b4580 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.071+0000 7f9cc3322640 1 --2- 172.21.15.43:0/1024832120 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9cbc0b5560 0x7f9cbc1518e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.071+0000 7f9cc3322640 1 -- 172.21.15.43:0/1024832120 >> 172.21.15.43:0/1024832120 conn(0x7f9cbc0d0560 msgr2=0x7f9cbc0cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:17.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.071+0000 7f9cc3322640 1 -- 172.21.15.43:0/1024832120 shutdown_connections 2024-04-25T09:01:17.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.071+0000 7f9cc3322640 1 -- 172.21.15.43:0/1024832120 wait complete. 2024-04-25T09:01:17.071 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:01:17.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.186+0000 7f9d4e1b0640 1 Processor -- start 2024-04-25T09:01:17.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.186+0000 7f9d4e1b0640 1 -- start start 2024-04-25T09:01:17.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.186+0000 7f9d4e1b0640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9d4806f6a0 0x7f9d480b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.187+0000 7f9d4e1b0640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9d480cd470 0x7f9d4806c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.187+0000 7f9d4e1b0640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d480d2620 con 0x7f9d4815ddd0 2024-04-25T09:01:17.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.187+0000 7f9d4e1b0640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d480b0b80 con 0x7f9d4806f6a0 2024-04-25T09:01:17.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.187+0000 7f9d4e1b0640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d480b0e60 con 0x7f9d480cd470 2024-04-25T09:01:17.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.187+0000 7f9d46ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9d4806f6a0 0x7f9d480b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:17.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.187+0000 7f9d46ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9d4806f6a0 0x7f9d480b3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42596/0 (socket says 172.21.15.43:42596) 2024-04-25T09:01:17.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.187+0000 7f9d46ffd640 1 -- 172.21.15.43:0/743295321 learned_addr learned my addr 172.21.15.43:0/743295321 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:17.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.187+0000 7f9d47fff640 1 --2- 172.21.15.43:0/743295321 >> v2:172.21.15.193:3300/0 conn(0x7f9d480cd470 0x7f9d4806c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:17.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.187+0000 7f9d46ffd640 1 -- 172.21.15.43:0/743295321 >> v2:172.21.15.193:3300/0 conn(0x7f9d480cd470 msgr2=0x7f9d4806c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:17.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.187+0000 7f9d46ffd640 1 --2- 172.21.15.43:0/743295321 >> v2:172.21.15.193:3300/0 conn(0x7f9d480cd470 0x7f9d4806c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.187+0000 7f9d46ffd640 1 -- 172.21.15.43:0/743295321 >> v1:172.21.15.193:6789/0 conn(0x7f9d4815ddd0 legacy=0x7f9d4806ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:17.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.187+0000 7f9d46ffd640 1 -- 172.21.15.43:0/743295321 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d480cbf40 con 0x7f9d4806f6a0 2024-04-25T09:01:17.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.188+0000 7f9d46ffd640 1 --2- 172.21.15.43:0/743295321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d4806f6a0 0x7f9d480b3dd0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f9d4806ee30 tx=0x7f9d3002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=975b5777fe2c177c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:17.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.188+0000 7f9d467fc640 1 -- 172.21.15.43:0/743295321 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d30037070 con 0x7f9d4806f6a0 2024-04-25T09:01:17.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.188+0000 7f9d467fc640 1 -- 172.21.15.43:0/743295321 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d30032440 con 0x7f9d4806f6a0 2024-04-25T09:01:17.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.188+0000 7f9d467fc640 1 -- 172.21.15.43:0/743295321 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d30032810 con 0x7f9d4806f6a0 2024-04-25T09:01:17.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.188+0000 7f9d4e1b0640 1 -- 172.21.15.43:0/743295321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d4806f6a0 msgr2=0x7f9d480b3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:17.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.188+0000 7f9d4e1b0640 1 --2- 172.21.15.43:0/743295321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d4806f6a0 0x7f9d480b3dd0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f9d4806ee30 tx=0x7f9d3002a270 comp rx=0 tx=0).stop 2024-04-25T09:01:17.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.189+0000 7f9d4e1b0640 1 -- 172.21.15.43:0/743295321 shutdown_connections 2024-04-25T09:01:17.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.189+0000 7f9d4e1b0640 1 --2- 172.21.15.43:0/743295321 >> v2:172.21.15.193:3300/0 conn(0x7f9d480cd470 0x7f9d4806c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.189+0000 7f9d4e1b0640 1 --2- 172.21.15.43:0/743295321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d4806f6a0 0x7f9d480b3dd0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.189+0000 7f9d4e1b0640 1 -- 172.21.15.43:0/743295321 >> 172.21.15.43:0/743295321 conn(0x7f9d480d17e0 msgr2=0x7f9d480d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:17.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.189+0000 7f9d4e1b0640 1 -- 172.21.15.43:0/743295321 shutdown_connections 2024-04-25T09:01:17.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.189+0000 7f9d4e1b0640 1 -- 172.21.15.43:0/743295321 wait complete. 2024-04-25T09:01:17.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.190+0000 7f9d4e1b0640 1 Processor -- start 2024-04-25T09:01:17.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.190+0000 7f9d4e1b0640 1 -- start start 2024-04-25T09:01:17.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.191+0000 7f9d4e1b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d4806f6a0 0x7f9d481f3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.191+0000 7f9d4e1b0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d480cd470 0x7f9d4814e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.191+0000 7f9d4e1b0640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d4815ddd0 0x7f9d4814bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.191+0000 7f9d4e1b0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4806cdb0 con 0x7f9d4806f6a0 2024-04-25T09:01:17.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.191+0000 7f9d4e1b0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4806cc30 con 0x7f9d480cd470 2024-04-25T09:01:17.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.191+0000 7f9d4e1b0640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4806cf30 con 0x7f9d4815ddd0 2024-04-25T09:01:17.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.191+0000 7f9d477fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d4806f6a0 0x7f9d481f3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:17.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.191+0000 7f9d477fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d4806f6a0 0x7f9d481f3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42606/0 (socket says 172.21.15.43:42606) 2024-04-25T09:01:17.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.191+0000 7f9d477fe640 1 -- 172.21.15.43:0/522804217 learned_addr learned my addr 172.21.15.43:0/522804217 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:17.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.191+0000 7f9d46ffd640 1 --2- 172.21.15.43:0/522804217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d480cd470 0x7f9d4814e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:17.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.191+0000 7f9d47fff640 1 --2- 172.21.15.43:0/522804217 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d4815ddd0 0x7f9d4814bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:17.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.191+0000 7f9d477fe640 1 -- 172.21.15.43:0/522804217 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d4815ddd0 msgr2=0x7f9d4814bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:17.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.191+0000 7f9d477fe640 1 --2- 172.21.15.43:0/522804217 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d4815ddd0 0x7f9d4814bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.191+0000 7f9d477fe640 1 -- 172.21.15.43:0/522804217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d480cd470 msgr2=0x7f9d4814e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:17.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.191+0000 7f9d477fe640 1 --2- 172.21.15.43:0/522804217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d480cd470 0x7f9d4814e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.191+0000 7f9d477fe640 1 -- 172.21.15.43:0/522804217 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d480ca750 con 0x7f9d4806f6a0 2024-04-25T09:01:17.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.191+0000 7f9d477fe640 1 --2- 172.21.15.43:0/522804217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d4806f6a0 0x7f9d481f3f30 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f9d3800c3c0 tx=0x7f9d3800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:17.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.191+0000 7f9d3ffff640 1 -- 172.21.15.43:0/522804217 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d380052d0 con 0x7f9d4806f6a0 2024-04-25T09:01:17.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.191+0000 7f9d3ffff640 1 -- 172.21.15.43:0/522804217 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d38005490 con 0x7f9d4806f6a0 2024-04-25T09:01:17.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.191+0000 7f9d4e1b0640 1 -- 172.21.15.43:0/522804217 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d481f45c0 con 0x7f9d4806f6a0 2024-04-25T09:01:17.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.191+0000 7f9d3ffff640 1 -- 172.21.15.43:0/522804217 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d38007260 con 0x7f9d4806f6a0 2024-04-25T09:01:17.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.191+0000 7f9d4e1b0640 1 -- 172.21.15.43:0/522804217 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d481f4b80 con 0x7f9d4806f6a0 2024-04-25T09:01:17.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.192+0000 7f9d4e1b0640 1 -- 172.21.15.43:0/522804217 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d48074f60 con 0x7f9d4806f6a0 2024-04-25T09:01:17.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.193+0000 7f9d3ffff640 1 -- 172.21.15.43:0/522804217 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9d38007400 con 0x7f9d4806f6a0 2024-04-25T09:01:17.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.193+0000 7f9d3ffff640 1 --2- 172.21.15.43:0/522804217 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9d2806b4d0 0x7f9d2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.193+0000 7f9d3ffff640 1 -- 172.21.15.43:0/522804217 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f9d380897a0 con 0x7f9d4806f6a0 2024-04-25T09:01:17.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.193+0000 7f9d46ffd640 1 --2- 172.21.15.43:0/522804217 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9d2806b4d0 0x7f9d2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:17.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.194+0000 7f9d46ffd640 1 --2- 172.21.15.43:0/522804217 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9d2806b4d0 0x7f9d2806d990 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f9d30034000 tx=0x7f9d3002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:17.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.197+0000 7f9d3ffff640 1 -- 172.21.15.43:0/522804217 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9d38058560 con 0x7f9d4806f6a0 2024-04-25T09:01:17.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.377+0000 7f9d4e1b0640 1 -- 172.21.15.43:0/522804217 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9d4814ec70 con 0x7f9d4806f6a0 2024-04-25T09:01:17.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.377+0000 7f9d3ffff640 1 -- 172.21.15.43:0/522804217 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9d38057e60 con 0x7f9d4806f6a0 2024-04-25T09:01:17.367 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T09:01:17.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.379+0000 7f9d4e1b0640 1 -- 172.21.15.43:0/522804217 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9d2806b4d0 msgr2=0x7f9d2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:17.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.379+0000 7f9d4e1b0640 1 --2- 172.21.15.43:0/522804217 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9d2806b4d0 0x7f9d2806d990 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f9d30034000 tx=0x7f9d3002e000 comp rx=0 tx=0).stop 2024-04-25T09:01:17.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.379+0000 7f9d4e1b0640 1 -- 172.21.15.43:0/522804217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d4806f6a0 msgr2=0x7f9d481f3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:17.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.379+0000 7f9d4e1b0640 1 --2- 172.21.15.43:0/522804217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d4806f6a0 0x7f9d481f3f30 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f9d3800c3c0 tx=0x7f9d3800c880 comp rx=0 tx=0).stop 2024-04-25T09:01:17.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.379+0000 7f9d4e1b0640 1 -- 172.21.15.43:0/522804217 shutdown_connections 2024-04-25T09:01:17.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.379+0000 7f9d4e1b0640 1 --2- 172.21.15.43:0/522804217 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d4815ddd0 0x7f9d4814bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.379+0000 7f9d4e1b0640 1 --2- 172.21.15.43:0/522804217 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9d2806b4d0 0x7f9d2806d990 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.379+0000 7f9d4e1b0640 1 --2- 172.21.15.43:0/522804217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d480cd470 0x7f9d4814e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.379+0000 7f9d4e1b0640 1 --2- 172.21.15.43:0/522804217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d4806f6a0 0x7f9d481f3f30 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.379+0000 7f9d4e1b0640 1 -- 172.21.15.43:0/522804217 >> 172.21.15.43:0/522804217 conn(0x7f9d480d17e0 msgr2=0x7f9d4815bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:17.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.380+0000 7f9d4e1b0640 1 -- 172.21.15.43:0/522804217 shutdown_connections 2024-04-25T09:01:17.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.380+0000 7f9d4e1b0640 1 -- 172.21.15.43:0/522804217 wait complete. 2024-04-25T09:01:17.379 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:01:17.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.494+0000 7f5668f7e640 1 Processor -- start 2024-04-25T09:01:17.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.494+0000 7f5668f7e640 1 -- start start 2024-04-25T09:01:17.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.495+0000 7f5668f7e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f566415a960 0x7f56640b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.495+0000 7f5668f7e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f566415b110 con 0x7f56640aee40 2024-04-25T09:01:17.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.495+0000 7f5668f7e640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f566415b3f0 con 0x7f56640b3ed0 2024-04-25T09:01:17.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.495+0000 7f5668f7e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f566415b6d0 con 0x7f566415a960 2024-04-25T09:01:17.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.495+0000 7f566259c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f566415a960 0x7f56640b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:17.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.495+0000 7f5662d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f56640aee40 0x7f5664070330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47482/0 (socket says 172.21.15.43:47482) 2024-04-25T09:01:17.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.495+0000 7f5662d9d640 1 -- 172.21.15.43:0/2077416498 learned_addr learned my addr 172.21.15.43:0/2077416498 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:17.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.495+0000 7f566259c640 1 -- 172.21.15.43:0/2077416498 >> v1:172.21.15.193:6789/0 conn(0x7f56640b3ed0 legacy=0x7f56640ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:17.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.495+0000 7f566259c640 1 -- 172.21.15.43:0/2077416498 >> v1:172.21.15.43:6789/0 conn(0x7f56640aee40 legacy=0x7f5664070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:17.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.495+0000 7f5661d9b640 1 -- 172.21.15.43:0/2077416498 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 429634257 0 0) 0x7f566415b110 con 0x7f56640aee40 2024-04-25T09:01:17.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.495+0000 7f566259c640 1 -- 172.21.15.43:0/2077416498 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f566415b9b0 con 0x7f566415a960 2024-04-25T09:01:17.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.496+0000 7f566259c640 1 --2- 172.21.15.43:0/2077416498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f566415a960 0x7f56640b3990 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f56540097a0 tx=0x7f565402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74db77fe15d96988 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:17.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.496+0000 7f5661d9b640 1 -- 172.21.15.43:0/2077416498 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5654037070 con 0x7f566415a960 2024-04-25T09:01:17.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.496+0000 7f5661d9b640 1 -- 172.21.15.43:0/2077416498 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5654032440 con 0x7f566415a960 2024-04-25T09:01:17.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.496+0000 7f5661d9b640 1 -- 172.21.15.43:0/2077416498 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5654032810 con 0x7f566415a960 2024-04-25T09:01:17.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.496+0000 7f5668f7e640 1 -- 172.21.15.43:0/2077416498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f566415a960 msgr2=0x7f56640b3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:17.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.496+0000 7f5668f7e640 1 --2- 172.21.15.43:0/2077416498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f566415a960 0x7f56640b3990 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f56540097a0 tx=0x7f565402a230 comp rx=0 tx=0).stop 2024-04-25T09:01:17.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.496+0000 7f5668f7e640 1 -- 172.21.15.43:0/2077416498 shutdown_connections 2024-04-25T09:01:17.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.496+0000 7f5668f7e640 1 --2- 172.21.15.43:0/2077416498 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f566415a960 0x7f56640b3990 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.496+0000 7f5668f7e640 1 -- 172.21.15.43:0/2077416498 >> 172.21.15.43:0/2077416498 conn(0x7f566406ab30 msgr2=0x7f566406af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:17.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.496+0000 7f5668f7e640 1 -- 172.21.15.43:0/2077416498 shutdown_connections 2024-04-25T09:01:17.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.497+0000 7f5668f7e640 1 -- 172.21.15.43:0/2077416498 wait complete. 2024-04-25T09:01:17.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.498+0000 7f5668f7e640 1 Processor -- start 2024-04-25T09:01:17.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.498+0000 7f5668f7e640 1 -- start start 2024-04-25T09:01:17.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.499+0000 7f5668f7e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56640aee40 0x7f56640a9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.499+0000 7f5668f7e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56640b3ed0 0x7f56641f46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.499+0000 7f5668f7e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f566415a960 0x7f56641f4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.499+0000 7f5668f7e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f566415ad40 con 0x7f566415a960 2024-04-25T09:01:17.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.499+0000 7f5668f7e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56640cd940 con 0x7f56640aee40 2024-04-25T09:01:17.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.499+0000 7f5668f7e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f566415aec0 con 0x7f56640b3ed0 2024-04-25T09:01:17.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.499+0000 7f5662d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f566415a960 0x7f56641f4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:17.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.499+0000 7f5662d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f566415a960 0x7f56641f4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42616/0 (socket says 172.21.15.43:42616) 2024-04-25T09:01:17.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.499+0000 7f5662d9d640 1 -- 172.21.15.43:0/3754906048 learned_addr learned my addr 172.21.15.43:0/3754906048 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:17.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.499+0000 7f566259c640 1 --2- 172.21.15.43:0/3754906048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56640aee40 0x7f56640a9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:17.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.499+0000 7f5659d9b640 1 --2- 172.21.15.43:0/3754906048 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56640b3ed0 0x7f56641f46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:17.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.499+0000 7f5662d9d640 1 -- 172.21.15.43:0/3754906048 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56640b3ed0 msgr2=0x7f56641f46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:17.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.499+0000 7f5662d9d640 1 --2- 172.21.15.43:0/3754906048 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56640b3ed0 0x7f56641f46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.499+0000 7f5662d9d640 1 -- 172.21.15.43:0/3754906048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56640aee40 msgr2=0x7f56640a9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:17.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.499+0000 7f5662d9d640 1 --2- 172.21.15.43:0/3754906048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56640aee40 0x7f56640a9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.499+0000 7f5662d9d640 1 -- 172.21.15.43:0/3754906048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56640aa960 con 0x7f566415a960 2024-04-25T09:01:17.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.499+0000 7f5659d9b640 1 --2- 172.21.15.43:0/3754906048 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56640b3ed0 0x7f56641f46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:17.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.499+0000 7f5662d9d640 1 --2- 172.21.15.43:0/3754906048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f566415a960 0x7f56641f4c10 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f565c00c7d0 tx=0x7f565c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:17.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.500+0000 7f565b7fe640 1 -- 172.21.15.43:0/3754906048 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565c018070 con 0x7f566415a960 2024-04-25T09:01:17.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.500+0000 7f565b7fe640 1 -- 172.21.15.43:0/3754906048 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f565c00a640 con 0x7f566415a960 2024-04-25T09:01:17.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.500+0000 7f565b7fe640 1 -- 172.21.15.43:0/3754906048 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f565c00aa70 con 0x7f566415a960 2024-04-25T09:01:17.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.500+0000 7f5668f7e640 1 -- 172.21.15.43:0/3754906048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56640aac40 con 0x7f566415a960 2024-04-25T09:01:17.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.500+0000 7f5668f7e640 1 -- 172.21.15.43:0/3754906048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56641473f0 con 0x7f566415a960 2024-04-25T09:01:17.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.501+0000 7f5668f7e640 1 -- 172.21.15.43:0/3754906048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5664074f60 con 0x7f566415a960 2024-04-25T09:01:17.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.501+0000 7f565b7fe640 1 -- 172.21.15.43:0/3754906048 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f565c01b070 con 0x7f566415a960 2024-04-25T09:01:17.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.502+0000 7f565b7fe640 1 --2- 172.21.15.43:0/3754906048 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f564406b4d0 0x7f564406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.502+0000 7f565b7fe640 1 -- 172.21.15.43:0/3754906048 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f565c089840 con 0x7f566415a960 2024-04-25T09:01:17.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.502+0000 7f566259c640 1 --2- 172.21.15.43:0/3754906048 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f564406b4d0 0x7f564406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:17.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.502+0000 7f566259c640 1 --2- 172.21.15.43:0/3754906048 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f564406b4d0 0x7f564406d990 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f5654034000 tx=0x7f565402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:17.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.506+0000 7f565b7fe640 1 -- 172.21.15.43:0/3754906048 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f565c0586a0 con 0x7f566415a960 2024-04-25T09:01:17.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.685+0000 7f5668f7e640 1 -- 172.21.15.43:0/3754906048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f56640cf2b0 con 0x7f566415a960 2024-04-25T09:01:17.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.685+0000 7f565b7fe640 1 -- 172.21.15.43:0/3754906048 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f565c057fa0 con 0x7f566415a960 2024-04-25T09:01:17.674 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T09:01:17.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.687+0000 7f5668f7e640 1 -- 172.21.15.43:0/3754906048 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f564406b4d0 msgr2=0x7f564406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:17.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.687+0000 7f5668f7e640 1 --2- 172.21.15.43:0/3754906048 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f564406b4d0 0x7f564406d990 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f5654034000 tx=0x7f565402e000 comp rx=0 tx=0).stop 2024-04-25T09:01:17.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.687+0000 7f5668f7e640 1 -- 172.21.15.43:0/3754906048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f566415a960 msgr2=0x7f56641f4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:17.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.687+0000 7f5668f7e640 1 --2- 172.21.15.43:0/3754906048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f566415a960 0x7f56641f4c10 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f565c00c7d0 tx=0x7f565c00cc90 comp rx=0 tx=0).stop 2024-04-25T09:01:17.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.687+0000 7f5668f7e640 1 -- 172.21.15.43:0/3754906048 shutdown_connections 2024-04-25T09:01:17.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.687+0000 7f5668f7e640 1 --2- 172.21.15.43:0/3754906048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f566415a960 0x7f56641f4c10 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.687+0000 7f5668f7e640 1 --2- 172.21.15.43:0/3754906048 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f56640b3ed0 0x7f56641f46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.687+0000 7f5668f7e640 1 --2- 172.21.15.43:0/3754906048 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f564406b4d0 0x7f564406d990 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.687+0000 7f5668f7e640 1 --2- 172.21.15.43:0/3754906048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f56640aee40 0x7f56640a9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.687+0000 7f5668f7e640 1 -- 172.21.15.43:0/3754906048 >> 172.21.15.43:0/3754906048 conn(0x7f566406ab30 msgr2=0x7f56640b08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:17.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.687+0000 7f5668f7e640 1 -- 172.21.15.43:0/3754906048 shutdown_connections 2024-04-25T09:01:17.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.687+0000 7f5668f7e640 1 -- 172.21.15.43:0/3754906048 wait complete. 2024-04-25T09:01:17.687 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:01:17.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.800+0000 7f454b3d5640 1 Processor -- start 2024-04-25T09:01:17.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.800+0000 7f454b3d5640 1 -- start start 2024-04-25T09:01:17.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.801+0000 7f454b3d5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f45440ccbb0 0x7f45440cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.801+0000 7f454b3d5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45440b3b00 0x7f45440b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.801+0000 7f454b3d5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45440b2780 con 0x7f45440cbfc0 2024-04-25T09:01:17.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.801+0000 7f454b3d5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45440cd550 con 0x7f45440ccbb0 2024-04-25T09:01:17.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.801+0000 7f454b3d5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45440b2a60 con 0x7f45440b3b00 2024-04-25T09:01:17.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.801+0000 7f4548970640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f45440ccbb0 0x7f45440cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:17.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.801+0000 7f4549171640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f45440cbfc0 0x7f454406dc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47490/0 (socket says 172.21.15.43:47490) 2024-04-25T09:01:17.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.801+0000 7f4549171640 1 -- 172.21.15.43:0/3190395189 learned_addr learned my addr 172.21.15.43:0/3190395189 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:17.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.801+0000 7f4549972640 1 --2- 172.21.15.43:0/3190395189 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45440b3b00 0x7f45440b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:17.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.801+0000 7f4548970640 1 -- 172.21.15.43:0/3190395189 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45440b3b00 msgr2=0x7f45440b3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:17.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.801+0000 7f4548970640 1 --2- 172.21.15.43:0/3190395189 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45440b3b00 0x7f45440b3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.801+0000 7f4548970640 1 -- 172.21.15.43:0/3190395189 >> v1:172.21.15.43:6789/0 conn(0x7f45440cbfc0 legacy=0x7f454406dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:17.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.801+0000 7f4548970640 1 -- 172.21.15.43:0/3190395189 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45440b2d40 con 0x7f45440ccbb0 2024-04-25T09:01:17.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.801+0000 7f4543fff640 1 -- 172.21.15.43:0/3190395189 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1483019284 0 0) 0x7f45440b2780 con 0x7f45440cbfc0 2024-04-25T09:01:17.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.802+0000 7f4548970640 1 --2- 172.21.15.43:0/3190395189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45440ccbb0 0x7f45440cd010 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f45380097a0 tx=0x7f453802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b75c0bcb1acd8a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:17.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.802+0000 7f4543fff640 1 -- 172.21.15.43:0/3190395189 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4538037070 con 0x7f45440ccbb0 2024-04-25T09:01:17.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.802+0000 7f4543fff640 1 -- 172.21.15.43:0/3190395189 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4538032440 con 0x7f45440ccbb0 2024-04-25T09:01:17.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.802+0000 7f4543fff640 1 -- 172.21.15.43:0/3190395189 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4538032810 con 0x7f45440ccbb0 2024-04-25T09:01:17.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.803+0000 7f454b3d5640 1 -- 172.21.15.43:0/3190395189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45440ccbb0 msgr2=0x7f45440cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:17.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.803+0000 7f454b3d5640 1 --2- 172.21.15.43:0/3190395189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45440ccbb0 0x7f45440cd010 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f45380097a0 tx=0x7f453802a180 comp rx=0 tx=0).stop 2024-04-25T09:01:17.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.803+0000 7f454b3d5640 1 -- 172.21.15.43:0/3190395189 shutdown_connections 2024-04-25T09:01:17.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.803+0000 7f454b3d5640 1 --2- 172.21.15.43:0/3190395189 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45440b3b00 0x7f45440b3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.803+0000 7f454b3d5640 1 --2- 172.21.15.43:0/3190395189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45440ccbb0 0x7f45440cd010 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.803+0000 7f454b3d5640 1 -- 172.21.15.43:0/3190395189 >> 172.21.15.43:0/3190395189 conn(0x7f45440b1480 msgr2=0x7f45440b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:17.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.803+0000 7f454b3d5640 1 -- 172.21.15.43:0/3190395189 shutdown_connections 2024-04-25T09:01:17.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.803+0000 7f454b3d5640 1 -- 172.21.15.43:0/3190395189 wait complete. 2024-04-25T09:01:17.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.804+0000 7f454b3d5640 1 Processor -- start 2024-04-25T09:01:17.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.804+0000 7f454b3d5640 1 -- start start 2024-04-25T09:01:17.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.805+0000 7f454b3d5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45440b3b00 0x7f4544158e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.805+0000 7f454b3d5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45440cbfc0 0x7f4544154b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.805+0000 7f454b3d5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45440ccbb0 0x7f4544155060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.805+0000 7f454b3d5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45440b35e0 con 0x7f45440b3b00 2024-04-25T09:01:17.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.805+0000 7f454b3d5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45440b3300 con 0x7f45440ccbb0 2024-04-25T09:01:17.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.805+0000 7f454b3d5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45440cc3a0 con 0x7f45440cbfc0 2024-04-25T09:01:17.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.805+0000 7f4549171640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45440b3b00 0x7f4544158e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:17.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.805+0000 7f4549171640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45440b3b00 0x7f4544158e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42626/0 (socket says 172.21.15.43:42626) 2024-04-25T09:01:17.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.805+0000 7f4549171640 1 -- 172.21.15.43:0/3632270252 learned_addr learned my addr 172.21.15.43:0/3632270252 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:17.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.805+0000 7f4549171640 1 -- 172.21.15.43:0/3632270252 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45440cbfc0 msgr2=0x7f4544154b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:17.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.805+0000 7f4549171640 1 --2- 172.21.15.43:0/3632270252 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45440cbfc0 0x7f4544154b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.805+0000 7f4549171640 1 -- 172.21.15.43:0/3632270252 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45440ccbb0 msgr2=0x7f4544155060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:17.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.805+0000 7f4549171640 1 --2- 172.21.15.43:0/3632270252 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45440ccbb0 0x7f4544155060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.805+0000 7f4549171640 1 -- 172.21.15.43:0/3632270252 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45441e6160 con 0x7f45440b3b00 2024-04-25T09:01:17.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.805+0000 7f4549171640 1 --2- 172.21.15.43:0/3632270252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45440b3b00 0x7f4544158e30 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f453c00c7f0 tx=0x7f453c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:17.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.805+0000 7f4541ffb640 1 -- 172.21.15.43:0/3632270252 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c018070 con 0x7f45440b3b00 2024-04-25T09:01:17.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.806+0000 7f4541ffb640 1 -- 172.21.15.43:0/3632270252 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453c00a640 con 0x7f45440b3b00 2024-04-25T09:01:17.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.806+0000 7f4541ffb640 1 -- 172.21.15.43:0/3632270252 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c00aa30 con 0x7f45440b3b00 2024-04-25T09:01:17.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.806+0000 7f454b3d5640 1 -- 172.21.15.43:0/3632270252 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45441e6440 con 0x7f45440b3b00 2024-04-25T09:01:17.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.806+0000 7f454b3d5640 1 -- 172.21.15.43:0/3632270252 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45441e6a00 con 0x7f45440b3b00 2024-04-25T09:01:17.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.807+0000 7f454b3d5640 1 -- 172.21.15.43:0/3632270252 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4544074ef0 con 0x7f45440b3b00 2024-04-25T09:01:17.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.807+0000 7f4541ffb640 1 -- 172.21.15.43:0/3632270252 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f453c01b070 con 0x7f45440b3b00 2024-04-25T09:01:17.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.807+0000 7f4541ffb640 1 --2- 172.21.15.43:0/3632270252 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f453006b4d0 0x7f453006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:17.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.808+0000 7f4541ffb640 1 -- 172.21.15.43:0/3632270252 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f453c089870 con 0x7f45440b3b00 2024-04-25T09:01:17.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.808+0000 7f4548970640 1 --2- 172.21.15.43:0/3632270252 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f453006b4d0 0x7f453006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:17.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.809+0000 7f4548970640 1 --2- 172.21.15.43:0/3632270252 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f453006b4d0 0x7f453006d990 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4538034000 tx=0x7f453802c000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:17.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:17.812+0000 7f4541ffb640 1 -- 172.21.15.43:0/3632270252 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f453c0586d0 con 0x7f45440b3b00 2024-04-25T09:01:17.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.005+0000 7f454b3d5640 1 -- 172.21.15.43:0/3632270252 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f45440af0d0 con 0x7f45440b3b00 2024-04-25T09:01:17.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.005+0000 7f4541ffb640 1 -- 172.21.15.43:0/3632270252 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v91) v1 ==== 80+0+0 (secure 0 0 0) 0x7f453c057fd0 con 0x7f45440b3b00 2024-04-25T09:01:17.994 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-04-25T09:01:17.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.007+0000 7f454b3d5640 1 -- 172.21.15.43:0/3632270252 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f453006b4d0 msgr2=0x7f453006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:17.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.007+0000 7f454b3d5640 1 --2- 172.21.15.43:0/3632270252 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f453006b4d0 0x7f453006d990 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4538034000 tx=0x7f453802c000 comp rx=0 tx=0).stop 2024-04-25T09:01:17.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.007+0000 7f454b3d5640 1 -- 172.21.15.43:0/3632270252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45440b3b00 msgr2=0x7f4544158e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:17.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.007+0000 7f454b3d5640 1 --2- 172.21.15.43:0/3632270252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45440b3b00 0x7f4544158e30 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f453c00c7f0 tx=0x7f453c00ccb0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.007+0000 7f454b3d5640 1 -- 172.21.15.43:0/3632270252 shutdown_connections 2024-04-25T09:01:17.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.007+0000 7f454b3d5640 1 --2- 172.21.15.43:0/3632270252 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f453006b4d0 0x7f453006d990 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.007+0000 7f454b3d5640 1 --2- 172.21.15.43:0/3632270252 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45440ccbb0 0x7f4544155060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.007+0000 7f454b3d5640 1 --2- 172.21.15.43:0/3632270252 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45440cbfc0 0x7f4544154b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.007+0000 7f454b3d5640 1 --2- 172.21.15.43:0/3632270252 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45440b3b00 0x7f4544158e30 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:17.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.007+0000 7f454b3d5640 1 -- 172.21.15.43:0/3632270252 >> 172.21.15.43:0/3632270252 conn(0x7f45440b1480 msgr2=0x7f45440b04a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:17.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.007+0000 7f454b3d5640 1 -- 172.21.15.43:0/3632270252 shutdown_connections 2024-04-25T09:01:17.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.007+0000 7f454b3d5640 1 -- 172.21.15.43:0/3632270252 wait complete. 2024-04-25T09:01:18.007 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi043.front.sepia.ceph.com... 2024-04-25T09:01:18.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:18.007 DEBUG:teuthology.orchestra.run.smithi043:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T09:01:18.030 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[76245]: fuse finished with error 0 and tester_r 0 2024-04-25T09:01:18.092 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-04-25T09:01:18.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:18.092 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:01:18.277 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T09:01:18.277 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:01:18.295 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19508 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:01:18.295 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi193.front.sepia.ceph.com 2024-04-25T09:01:18.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:18.295 DEBUG:teuthology.orchestra.run.smithi193:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T09:01:18.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:01:18.353 INFO:teuthology.orchestra.run.smithi193.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T09:01:18.353 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T09:01:18.353 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:01:18.369 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2374 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:01:18.369 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-04-25T09:01:18.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:18.370 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T09:01:18.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:01:18.425 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T09:01:18.425 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T09:01:18.425 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:01:18.442 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19508 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:01:18.442 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi193.front.sepia.ceph.com 2024-04-25T09:01:18.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:18.442 DEBUG:teuthology.orchestra.run.smithi193:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T09:01:18.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:01:18.499 INFO:teuthology.orchestra.run.smithi193.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T09:01:18.499 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:01:18.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.612+0000 7fde7fcb3640 1 Processor -- start 2024-04-25T09:01:18.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.612+0000 7fde7fcb3640 1 -- start start 2024-04-25T09:01:18.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.613+0000 7fde7fcb3640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fde78161390 0x7fde780b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:18.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.613+0000 7fde7fcb3640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fde780d02a0 0x7fde780d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:18.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.613+0000 7fde7fcb3640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde780b4c30 con 0x7fde780ce5e0 2024-04-25T09:01:18.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.613+0000 7fde7fcb3640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde780d1910 con 0x7fde780d02a0 2024-04-25T09:01:18.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.613+0000 7fde7fcb3640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde780cf2c0 con 0x7fde78161390 2024-04-25T09:01:18.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.613+0000 7fde7d24e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fde780d02a0 0x7fde780d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:18.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.613+0000 7fde7d24e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fde780d02a0 0x7fde780d0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42630/0 (socket says 172.21.15.43:42630) 2024-04-25T09:01:18.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.613+0000 7fde7d24e640 1 -- 172.21.15.43:0/1125595439 learned_addr learned my addr 172.21.15.43:0/1125595439 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:18.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.613+0000 7fde7da4f640 1 --2- 172.21.15.43:0/1125595439 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fde78161390 0x7fde780b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:18.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.613+0000 7fde7d24e640 1 -- 172.21.15.43:0/1125595439 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fde78161390 msgr2=0x7fde780b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:18.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.614+0000 7fde7d24e640 1 --2- 172.21.15.43:0/1125595439 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fde78161390 0x7fde780b6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.614+0000 7fde7d24e640 1 -- 172.21.15.43:0/1125595439 >> v1:172.21.15.193:6789/0 conn(0x7fde780ce5e0 legacy=0x7fde780cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:18.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.614+0000 7fde7d24e640 1 -- 172.21.15.43:0/1125595439 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde780cf5a0 con 0x7fde780d02a0 2024-04-25T09:01:18.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.614+0000 7fde7d24e640 1 --2- 172.21.15.43:0/1125595439 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde780d02a0 0x7fde780d0720 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fde68009870 tx=0x7fde6802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d0216359436091 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:18.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.614+0000 7fde7ca4d640 1 -- 172.21.15.43:0/1125595439 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde68037070 con 0x7fde780d02a0 2024-04-25T09:01:18.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.614+0000 7fde7ca4d640 1 -- 172.21.15.43:0/1125595439 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde68032440 con 0x7fde780d02a0 2024-04-25T09:01:18.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.614+0000 7fde7ca4d640 1 -- 172.21.15.43:0/1125595439 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde68032810 con 0x7fde780d02a0 2024-04-25T09:01:18.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.615+0000 7fde7fcb3640 1 -- 172.21.15.43:0/1125595439 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde780d02a0 msgr2=0x7fde780d0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:18.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.615+0000 7fde7fcb3640 1 --2- 172.21.15.43:0/1125595439 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde780d02a0 0x7fde780d0720 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fde68009870 tx=0x7fde6802a270 comp rx=0 tx=0).stop 2024-04-25T09:01:18.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.615+0000 7fde7fcb3640 1 -- 172.21.15.43:0/1125595439 shutdown_connections 2024-04-25T09:01:18.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.615+0000 7fde7fcb3640 1 --2- 172.21.15.43:0/1125595439 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde780d02a0 0x7fde780d0720 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.615+0000 7fde7fcb3640 1 --2- 172.21.15.43:0/1125595439 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fde78161390 0x7fde780b6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.615+0000 7fde7fcb3640 1 -- 172.21.15.43:0/1125595439 >> 172.21.15.43:0/1125595439 conn(0x7fde7809cd20 msgr2=0x7fde780d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:18.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.615+0000 7fde7fcb3640 1 -- 172.21.15.43:0/1125595439 shutdown_connections 2024-04-25T09:01:18.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.615+0000 7fde7fcb3640 1 -- 172.21.15.43:0/1125595439 wait complete. 2024-04-25T09:01:18.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.616+0000 7fde7fcb3640 1 Processor -- start 2024-04-25T09:01:18.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.616+0000 7fde7fcb3640 1 -- start start 2024-04-25T09:01:18.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.617+0000 7fde7fcb3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde780ce5e0 0x7fde7815d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:18.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.617+0000 7fde7fcb3640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fde780d02a0 0x7fde78155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:18.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.617+0000 7fde7fcb3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde78161390 0x7fde781596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:18.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.617+0000 7fde7fcb3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde78161770 con 0x7fde780ce5e0 2024-04-25T09:01:18.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.617+0000 7fde7fcb3640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde780d15a0 con 0x7fde78161390 2024-04-25T09:01:18.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.617+0000 7fde7fcb3640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde780d1da0 con 0x7fde780d02a0 2024-04-25T09:01:18.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.617+0000 7fde7da4f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde780ce5e0 0x7fde7815d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:18.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.617+0000 7fde7da4f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde780ce5e0 0x7fde7815d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42634/0 (socket says 172.21.15.43:42634) 2024-04-25T09:01:18.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.617+0000 7fde7da4f640 1 -- 172.21.15.43:0/1125250685 learned_addr learned my addr 172.21.15.43:0/1125250685 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:18.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.617+0000 7fde7e250640 1 --2- 172.21.15.43:0/1125250685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde78161390 0x7fde781596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:18.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.617+0000 7fde7d24e640 1 --2- 172.21.15.43:0/1125250685 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fde780d02a0 0x7fde78155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:18.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.617+0000 7fde7da4f640 1 -- 172.21.15.43:0/1125250685 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fde780d02a0 msgr2=0x7fde78155e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:18.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.617+0000 7fde7da4f640 1 --2- 172.21.15.43:0/1125250685 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fde780d02a0 0x7fde78155e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.617+0000 7fde7da4f640 1 -- 172.21.15.43:0/1125250685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde78161390 msgr2=0x7fde781596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:18.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.617+0000 7fde7da4f640 1 --2- 172.21.15.43:0/1125250685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde78161390 0x7fde781596e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.617+0000 7fde7da4f640 1 -- 172.21.15.43:0/1125250685 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde7815dac0 con 0x7fde780ce5e0 2024-04-25T09:01:18.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.618+0000 7fde7da4f640 1 --2- 172.21.15.43:0/1125250685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde780ce5e0 0x7fde7815d580 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fde7400d5f0 tx=0x7fde7400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:18.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.618+0000 7fde6e7fc640 1 -- 172.21.15.43:0/1125250685 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde740125f0 con 0x7fde780ce5e0 2024-04-25T09:01:18.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.618+0000 7fde6e7fc640 1 -- 172.21.15.43:0/1125250685 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde74019070 con 0x7fde780ce5e0 2024-04-25T09:01:18.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.618+0000 7fde7fcb3640 1 -- 172.21.15.43:0/1125250685 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde78156340 con 0x7fde780ce5e0 2024-04-25T09:01:18.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.618+0000 7fde6e7fc640 1 -- 172.21.15.43:0/1125250685 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde740129d0 con 0x7fde780ce5e0 2024-04-25T09:01:18.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.618+0000 7fde7fcb3640 1 -- 172.21.15.43:0/1125250685 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde78158220 con 0x7fde780ce5e0 2024-04-25T09:01:18.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.619+0000 7fde7fcb3640 1 -- 172.21.15.43:0/1125250685 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde780680f0 con 0x7fde780ce5e0 2024-04-25T09:01:18.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.620+0000 7fde6e7fc640 1 -- 172.21.15.43:0/1125250685 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fde74012b70 con 0x7fde780ce5e0 2024-04-25T09:01:18.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.620+0000 7fde6e7fc640 1 --2- 172.21.15.43:0/1125250685 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fde5c06b500 0x7fde5c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:18.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.620+0000 7fde6e7fc640 1 -- 172.21.15.43:0/1125250685 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fde7408a4b0 con 0x7fde780ce5e0 2024-04-25T09:01:18.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.620+0000 7fde7d24e640 1 --2- 172.21.15.43:0/1125250685 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fde5c06b500 0x7fde5c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:18.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.621+0000 7fde7d24e640 1 --2- 172.21.15.43:0/1125250685 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fde5c06b500 0x7fde5c06d9c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fde68034000 tx=0x7fde6802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:18.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.624+0000 7fde6e7fc640 1 -- 172.21.15.43:0/1125250685 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fde740593a0 con 0x7fde780ce5e0 2024-04-25T09:01:18.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.847+0000 7fde7fcb3640 1 -- 172.21.15.43:0/1125250685 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde780d1050 con 0x7fde780ce5e0 2024-04-25T09:01:18.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.848+0000 7fde6e7fc640 1 -- 172.21.15.43:0/1125250685 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fde74058ca0 con 0x7fde780ce5e0 2024-04-25T09:01:18.838 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:01:18.838 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":54,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6200,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2258031526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2258031526},{"type":"v1","addr":"172.21.15.43:6834","nonce":2258031526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6206,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/1773440177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1773440177},{"type":"v1","addr":"172.21.15.193:6836","nonce":1773440177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6209,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3239725307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":3239725307},{"type":"v1","addr":"172.21.15.193:6837","nonce":3239725307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:00:42.536578+0000","modified":"2024-04-25T09:00:44.531104+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7596},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7596":{"gid":7596,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/1341028696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1341028696},{"type":"v1","addr":"172.21.15.43:6835","nonce":1341028696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:01:18.838 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 54 2024-04-25T09:01:18.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.850+0000 7fde7fcb3640 1 -- 172.21.15.43:0/1125250685 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fde5c06b500 msgr2=0x7fde5c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:18.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.850+0000 7fde7fcb3640 1 --2- 172.21.15.43:0/1125250685 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fde5c06b500 0x7fde5c06d9c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fde68034000 tx=0x7fde6802e000 comp rx=0 tx=0).stop 2024-04-25T09:01:18.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.850+0000 7fde7fcb3640 1 -- 172.21.15.43:0/1125250685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde780ce5e0 msgr2=0x7fde7815d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:18.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.850+0000 7fde7fcb3640 1 --2- 172.21.15.43:0/1125250685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde780ce5e0 0x7fde7815d580 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fde7400d5f0 tx=0x7fde7400dab0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.850+0000 7fde7fcb3640 1 -- 172.21.15.43:0/1125250685 shutdown_connections 2024-04-25T09:01:18.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.850+0000 7fde7fcb3640 1 --2- 172.21.15.43:0/1125250685 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fde5c06b500 0x7fde5c06d9c0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.850+0000 7fde7fcb3640 1 --2- 172.21.15.43:0/1125250685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fde78161390 0x7fde781596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.850+0000 7fde7fcb3640 1 --2- 172.21.15.43:0/1125250685 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fde780d02a0 0x7fde78155e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.850+0000 7fde7fcb3640 1 --2- 172.21.15.43:0/1125250685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fde780ce5e0 0x7fde7815d580 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.850+0000 7fde7fcb3640 1 -- 172.21.15.43:0/1125250685 >> 172.21.15.43:0/1125250685 conn(0x7fde7809cd20 msgr2=0x7fde780b5480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:18.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.850+0000 7fde7fcb3640 1 -- 172.21.15.43:0/1125250685 shutdown_connections 2024-04-25T09:01:18.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.850+0000 7fde7fcb3640 1 -- 172.21.15.43:0/1125250685 wait complete. 2024-04-25T09:01:18.851 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:01:18.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.963+0000 7fbde6e88640 1 Processor -- start 2024-04-25T09:01:18.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.963+0000 7fbde6e88640 1 -- start start 2024-04-25T09:01:18.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.964+0000 7fbde6e88640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbde006c510 0x7fbde006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:18.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.964+0000 7fbde6e88640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbde015df70 0x7fbde00ce070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:18.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.964+0000 7fbde6e88640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbde00cbee0 con 0x7fbde00b0870 2024-04-25T09:01:18.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.964+0000 7fbde6e88640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbde00d0400 con 0x7fbde006c510 2024-04-25T09:01:18.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.964+0000 7fbde6e88640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde006ce30 con 0x7fbde015df70 2024-04-25T09:01:18.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.964+0000 7fbde4c24640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbde006c510 0x7fbde006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:18.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.964+0000 7fbde4c24640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbde006c510 0x7fbde006c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42650/0 (socket says 172.21.15.43:42650) 2024-04-25T09:01:18.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.964+0000 7fbde4c24640 1 -- 172.21.15.43:0/2498849339 learned_addr learned my addr 172.21.15.43:0/2498849339 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:18.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.964+0000 7fbddffff640 1 --2- 172.21.15.43:0/2498849339 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbde015df70 0x7fbde00ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:18.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.964+0000 7fbde4c24640 1 -- 172.21.15.43:0/2498849339 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbde015df70 msgr2=0x7fbde00ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:18.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.964+0000 7fbde4c24640 1 --2- 172.21.15.43:0/2498849339 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbde015df70 0x7fbde00ce070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.964+0000 7fbde4c24640 1 -- 172.21.15.43:0/2498849339 >> v1:172.21.15.193:6789/0 conn(0x7fbde00b0870 legacy=0x7fbde00ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:18.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.964+0000 7fbde4c24640 1 -- 172.21.15.43:0/2498849339 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde00d0650 con 0x7fbde006c510 2024-04-25T09:01:18.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.965+0000 7fbde4c24640 1 --2- 172.21.15.43:0/2498849339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbde006c510 0x7fbde006c8f0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4009870 tx=0x7fbdd402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2baad74e8b901848 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:18.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.965+0000 7fbddf7fe640 1 -- 172.21.15.43:0/2498849339 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4037070 con 0x7fbde006c510 2024-04-25T09:01:18.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.965+0000 7fbddf7fe640 1 -- 172.21.15.43:0/2498849339 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd4032440 con 0x7fbde006c510 2024-04-25T09:01:18.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.965+0000 7fbddf7fe640 1 -- 172.21.15.43:0/2498849339 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4032810 con 0x7fbde006c510 2024-04-25T09:01:18.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.966+0000 7fbde6e88640 1 -- 172.21.15.43:0/2498849339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbde006c510 msgr2=0x7fbde006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:18.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.966+0000 7fbde6e88640 1 --2- 172.21.15.43:0/2498849339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbde006c510 0x7fbde006c8f0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4009870 tx=0x7fbdd402a270 comp rx=0 tx=0).stop 2024-04-25T09:01:18.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.966+0000 7fbde6e88640 1 -- 172.21.15.43:0/2498849339 shutdown_connections 2024-04-25T09:01:18.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.966+0000 7fbde6e88640 1 --2- 172.21.15.43:0/2498849339 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbde015df70 0x7fbde00ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.966+0000 7fbde6e88640 1 --2- 172.21.15.43:0/2498849339 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbde006c510 0x7fbde006c8f0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.966+0000 7fbde6e88640 1 -- 172.21.15.43:0/2498849339 >> 172.21.15.43:0/2498849339 conn(0x7fbde00d17f0 msgr2=0x7fbde00d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:18.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.966+0000 7fbde6e88640 1 -- 172.21.15.43:0/2498849339 shutdown_connections 2024-04-25T09:01:18.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.966+0000 7fbde6e88640 1 -- 172.21.15.43:0/2498849339 wait complete. 2024-04-25T09:01:18.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.967+0000 7fbde6e88640 1 Processor -- start 2024-04-25T09:01:18.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.967+0000 7fbde6e88640 1 -- start start 2024-04-25T09:01:18.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.968+0000 7fbde6e88640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbde006c510 0x7fbde014c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:18.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.968+0000 7fbde6e88640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbde00b0870 0x7fbde014f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:18.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.968+0000 7fbde6e88640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbde015df70 0x7fbde00cde60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:18.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.968+0000 7fbde6e88640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde00d0c10 con 0x7fbde006c510 2024-04-25T09:01:18.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.968+0000 7fbde6e88640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde00cc150 con 0x7fbde015df70 2024-04-25T09:01:18.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.968+0000 7fbde6e88640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde00cecc0 con 0x7fbde00b0870 2024-04-25T09:01:18.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.968+0000 7fbde4c24640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbde006c510 0x7fbde014c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:18.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.968+0000 7fbde4c24640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbde006c510 0x7fbde014c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42664/0 (socket says 172.21.15.43:42664) 2024-04-25T09:01:18.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.968+0000 7fbde4c24640 1 -- 172.21.15.43:0/3484729347 learned_addr learned my addr 172.21.15.43:0/3484729347 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:18.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.968+0000 7fbde4c24640 1 -- 172.21.15.43:0/3484729347 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbde00b0870 msgr2=0x7fbde014f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:18.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.968+0000 7fbde4c24640 1 --2- 172.21.15.43:0/3484729347 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbde00b0870 0x7fbde014f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.968+0000 7fbde4c24640 1 -- 172.21.15.43:0/3484729347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbde015df70 msgr2=0x7fbde00cde60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:18.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.968+0000 7fbde5425640 1 --2- 172.21.15.43:0/3484729347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbde015df70 0x7fbde00cde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:18.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.968+0000 7fbde4c24640 1 --2- 172.21.15.43:0/3484729347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbde015df70 0x7fbde00cde60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:18.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.968+0000 7fbde4c24640 1 -- 172.21.15.43:0/3484729347 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde00b24e0 con 0x7fbde006c510 2024-04-25T09:01:18.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.968+0000 7fbde4c24640 1 --2- 172.21.15.43:0/3484729347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbde006c510 0x7fbde014c930 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4009870 tx=0x7fbdd40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:18.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.968+0000 7fbddd7fa640 1 -- 172.21.15.43:0/3484729347 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4041070 con 0x7fbde006c510 2024-04-25T09:01:18.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.969+0000 7fbddd7fa640 1 -- 172.21.15.43:0/3484729347 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd403c870 con 0x7fbde006c510 2024-04-25T09:01:18.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.969+0000 7fbde6e88640 1 -- 172.21.15.43:0/3484729347 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbde00c7c20 con 0x7fbde006c510 2024-04-25T09:01:18.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.969+0000 7fbddd7fa640 1 -- 172.21.15.43:0/3484729347 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd4037040 con 0x7fbde006c510 2024-04-25T09:01:18.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.969+0000 7fbde6e88640 1 -- 172.21.15.43:0/3484729347 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbde00c7ee0 con 0x7fbde006c510 2024-04-25T09:01:18.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.970+0000 7fbde6e88640 1 -- 172.21.15.43:0/3484729347 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbde00c8120 con 0x7fbde006c510 2024-04-25T09:01:18.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.970+0000 7fbddd7fa640 1 -- 172.21.15.43:0/3484729347 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbdd4044070 con 0x7fbde006c510 2024-04-25T09:01:18.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.970+0000 7fbddd7fa640 1 --2- 172.21.15.43:0/3484729347 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbdc406b500 0x7fbdc406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:18.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.971+0000 7fbddd7fa640 1 -- 172.21.15.43:0/3484729347 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fbdd40aae40 con 0x7fbde006c510 2024-04-25T09:01:18.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.971+0000 7fbddffff640 1 --2- 172.21.15.43:0/3484729347 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbdc406b500 0x7fbdc406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:18.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.971+0000 7fbddffff640 1 --2- 172.21.15.43:0/3484729347 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbdc406b500 0x7fbdc406d9c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc006fd0 tx=0x7fbdcc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:18.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:18.975+0000 7fbddd7fa640 1 -- 172.21.15.43:0/3484729347 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbde00c8120 con 0x7fbde006c510 2024-04-25T09:01:19.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.156+0000 7fbde6e88640 1 -- 172.21.15.43:0/3484729347 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbde0074ef0 con 0x7fbde006c510 2024-04-25T09:01:19.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.157+0000 7fbddd7fa640 1 -- 172.21.15.43:0/3484729347 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+16674 (secure 0 0 0) 0x7fbdd4079dc0 con 0x7fbde006c510 2024-04-25T09:01:19.146 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:01:19.146 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":91,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:00:54.209500+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:00:40.816230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T09:00:41.508021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.26","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:01:19.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.159+0000 7fbde6e88640 1 -- 172.21.15.43:0/3484729347 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbdc406b500 msgr2=0x7fbdc406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:19.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.159+0000 7fbde6e88640 1 --2- 172.21.15.43:0/3484729347 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbdc406b500 0x7fbdc406d9c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc006fd0 tx=0x7fbdcc008040 comp rx=0 tx=0).stop 2024-04-25T09:01:19.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.159+0000 7fbde6e88640 1 -- 172.21.15.43:0/3484729347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbde006c510 msgr2=0x7fbde014c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:19.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.159+0000 7fbde6e88640 1 --2- 172.21.15.43:0/3484729347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbde006c510 0x7fbde014c930 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fbdd4009870 tx=0x7fbdd40046c0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.159+0000 7fbde6e88640 1 -- 172.21.15.43:0/3484729347 shutdown_connections 2024-04-25T09:01:19.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.159+0000 7fbde6e88640 1 --2- 172.21.15.43:0/3484729347 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbdc406b500 0x7fbdc406d9c0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.159+0000 7fbde6e88640 1 --2- 172.21.15.43:0/3484729347 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbde015df70 0x7fbde00cde60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.159+0000 7fbde6e88640 1 --2- 172.21.15.43:0/3484729347 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbde00b0870 0x7fbde014f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.159+0000 7fbde6e88640 1 --2- 172.21.15.43:0/3484729347 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbde006c510 0x7fbde014c930 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.159+0000 7fbde6e88640 1 -- 172.21.15.43:0/3484729347 >> 172.21.15.43:0/3484729347 conn(0x7fbde00d17f0 msgr2=0x7fbde00b6420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:19.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.159+0000 7fbde6e88640 1 -- 172.21.15.43:0/3484729347 shutdown_connections 2024-04-25T09:01:19.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.159+0000 7fbde6e88640 1 -- 172.21.15.43:0/3484729347 wait complete. 2024-04-25T09:01:19.160 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:00:40.816230+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:00:41.508021+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:01:19.160 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:01:19.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.275+0000 7ff729570640 1 Processor -- start 2024-04-25T09:01:19.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.275+0000 7ff729570640 1 -- start start 2024-04-25T09:01:19.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.276+0000 7ff729570640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff7240acf60 0x7ff72406cb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:19.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.276+0000 7ff729570640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff72415cbe0 0x7ff7240ae180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:19.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.276+0000 7ff729570640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff72406df00 con 0x7ff72406ab30 2024-04-25T09:01:19.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.276+0000 7ff729570640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7240ae6c0 con 0x7ff7240acf60 2024-04-25T09:01:19.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.276+0000 7ff729570640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff72406e1e0 con 0x7ff72415cbe0 2024-04-25T09:01:19.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.276+0000 7ff722ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff7240acf60 0x7ff72406cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:19.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.276+0000 7ff722ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff7240acf60 0x7ff72406cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42666/0 (socket says 172.21.15.43:42666) 2024-04-25T09:01:19.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.276+0000 7ff722ffd640 1 -- 172.21.15.43:0/387022962 learned_addr learned my addr 172.21.15.43:0/387022962 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:19.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.276+0000 7ff7237fe640 1 --2- 172.21.15.43:0/387022962 >> v2:172.21.15.193:3300/0 conn(0x7ff72415cbe0 0x7ff7240ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:19.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.277+0000 7ff722ffd640 1 -- 172.21.15.43:0/387022962 >> v2:172.21.15.193:3300/0 conn(0x7ff72415cbe0 msgr2=0x7ff7240ae180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:19.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.277+0000 7ff722ffd640 1 --2- 172.21.15.43:0/387022962 >> v2:172.21.15.193:3300/0 conn(0x7ff72415cbe0 0x7ff7240ae180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.277+0000 7ff722ffd640 1 -- 172.21.15.43:0/387022962 >> v1:172.21.15.193:6789/0 conn(0x7ff72406ab30 legacy=0x7ff7240ad340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:19.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.277+0000 7ff722ffd640 1 -- 172.21.15.43:0/387022962 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff72406e4c0 con 0x7ff7240acf60 2024-04-25T09:01:19.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.277+0000 7ff722ffd640 1 --2- 172.21.15.43:0/387022962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7240acf60 0x7ff72406cb40 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7ff724070cf0 tx=0x7ff71802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ea3791c427342b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:19.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.277+0000 7ff721ffb640 1 -- 172.21.15.43:0/387022962 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff718037070 con 0x7ff7240acf60 2024-04-25T09:01:19.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.278+0000 7ff721ffb640 1 -- 172.21.15.43:0/387022962 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff718032440 con 0x7ff7240acf60 2024-04-25T09:01:19.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.278+0000 7ff721ffb640 1 -- 172.21.15.43:0/387022962 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff718032810 con 0x7ff7240acf60 2024-04-25T09:01:19.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.278+0000 7ff729570640 1 -- 172.21.15.43:0/387022962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7240acf60 msgr2=0x7ff72406cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:19.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.278+0000 7ff729570640 1 --2- 172.21.15.43:0/387022962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7240acf60 0x7ff72406cb40 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7ff724070cf0 tx=0x7ff71802a270 comp rx=0 tx=0).stop 2024-04-25T09:01:19.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.278+0000 7ff729570640 1 -- 172.21.15.43:0/387022962 shutdown_connections 2024-04-25T09:01:19.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.278+0000 7ff729570640 1 --2- 172.21.15.43:0/387022962 >> v2:172.21.15.193:3300/0 conn(0x7ff72415cbe0 0x7ff7240ae180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.278+0000 7ff729570640 1 --2- 172.21.15.43:0/387022962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7240acf60 0x7ff72406cb40 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.278+0000 7ff729570640 1 -- 172.21.15.43:0/387022962 >> 172.21.15.43:0/387022962 conn(0x7ff7240b5ad0 msgr2=0x7ff7240b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:19.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.278+0000 7ff729570640 1 -- 172.21.15.43:0/387022962 shutdown_connections 2024-04-25T09:01:19.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.278+0000 7ff729570640 1 -- 172.21.15.43:0/387022962 wait complete. 2024-04-25T09:01:19.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.279+0000 7ff729570640 1 Processor -- start 2024-04-25T09:01:19.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.280+0000 7ff729570640 1 -- start start 2024-04-25T09:01:19.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.280+0000 7ff729570640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff72406ab30 0x7ff72414ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:19.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.280+0000 7ff729570640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7240acf60 0x7ff7240b10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:19.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.280+0000 7ff729570640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff72415cbe0 0x7ff7240b1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:19.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.280+0000 7ff729570640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff724071bd0 con 0x7ff72415cbe0 2024-04-25T09:01:19.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.280+0000 7ff729570640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff724071a50 con 0x7ff7240acf60 2024-04-25T09:01:19.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.280+0000 7ff729570640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff724072070 con 0x7ff72406ab30 2024-04-25T09:01:19.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.280+0000 7ff7237fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff72415cbe0 0x7ff7240b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:19.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.280+0000 7ff7237fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff72415cbe0 0x7ff7240b1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42672/0 (socket says 172.21.15.43:42672) 2024-04-25T09:01:19.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.280+0000 7ff7237fe640 1 -- 172.21.15.43:0/2883711659 learned_addr learned my addr 172.21.15.43:0/2883711659 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.280+0000 7ff722ffd640 1 --2- 172.21.15.43:0/2883711659 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff72406ab30 0x7ff72414ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.280+0000 7ff7227fc640 1 --2- 172.21.15.43:0/2883711659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7240acf60 0x7ff7240b10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.281+0000 7ff7237fe640 1 -- 172.21.15.43:0/2883711659 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff72406ab30 msgr2=0x7ff72414ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.281+0000 7ff7237fe640 1 --2- 172.21.15.43:0/2883711659 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff72406ab30 0x7ff72414ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.281+0000 7ff7237fe640 1 -- 172.21.15.43:0/2883711659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7240acf60 msgr2=0x7ff7240b10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.281+0000 7ff7237fe640 1 --2- 172.21.15.43:0/2883711659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7240acf60 0x7ff7240b10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.281+0000 7ff7237fe640 1 -- 172.21.15.43:0/2883711659 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7241503d0 con 0x7ff72415cbe0 2024-04-25T09:01:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.281+0000 7ff7237fe640 1 --2- 172.21.15.43:0/2883711659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff72415cbe0 0x7ff7240b1630 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7ff71c007e60 tx=0x7ff71c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.281+0000 7ff70b7fe640 1 -- 172.21.15.43:0/2883711659 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71c014420 con 0x7ff72415cbe0 2024-04-25T09:01:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.281+0000 7ff70b7fe640 1 -- 172.21.15.43:0/2883711659 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff71c019070 con 0x7ff72415cbe0 2024-04-25T09:01:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.281+0000 7ff729570640 1 -- 172.21.15.43:0/2883711659 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff724158c20 con 0x7ff72415cbe0 2024-04-25T09:01:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.281+0000 7ff70b7fe640 1 -- 172.21.15.43:0/2883711659 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71c014820 con 0x7ff72415cbe0 2024-04-25T09:01:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.281+0000 7ff729570640 1 -- 172.21.15.43:0/2883711659 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7241591e0 con 0x7ff72415cbe0 2024-04-25T09:01:19.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.282+0000 7ff729570640 1 -- 172.21.15.43:0/2883711659 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff724074f60 con 0x7ff72415cbe0 2024-04-25T09:01:19.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.282+0000 7ff70b7fe640 1 -- 172.21.15.43:0/2883711659 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff71c0149c0 con 0x7ff72415cbe0 2024-04-25T09:01:19.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.283+0000 7ff70b7fe640 1 --2- 172.21.15.43:0/2883711659 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff71006b4d0 0x7ff71006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:19.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.283+0000 7ff70b7fe640 1 -- 172.21.15.43:0/2883711659 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7ff71c08a9e0 con 0x7ff72415cbe0 2024-04-25T09:01:19.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.283+0000 7ff722ffd640 1 --2- 172.21.15.43:0/2883711659 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff71006b4d0 0x7ff71006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:19.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.284+0000 7ff722ffd640 1 --2- 172.21.15.43:0/2883711659 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff71006b4d0 0x7ff71006d990 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff718034000 tx=0x7ff71802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:19.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.287+0000 7ff70b7fe640 1 -- 172.21.15.43:0/2883711659 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff71c059150 con 0x7ff72415cbe0 2024-04-25T09:01:19.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.466+0000 7ff729570640 1 -- 172.21.15.43:0/2883711659 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff7241505a0 con 0x7ff72415cbe0 2024-04-25T09:01:19.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.467+0000 7ff70b7fe640 1 -- 172.21.15.43:0/2883711659 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff71c05d1f0 con 0x7ff72415cbe0 2024-04-25T09:01:19.456 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T09:01:19.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.469+0000 7ff729570640 1 -- 172.21.15.43:0/2883711659 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff71006b4d0 msgr2=0x7ff71006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:19.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.469+0000 7ff729570640 1 --2- 172.21.15.43:0/2883711659 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff71006b4d0 0x7ff71006d990 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7ff718034000 tx=0x7ff71802e000 comp rx=0 tx=0).stop 2024-04-25T09:01:19.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.469+0000 7ff729570640 1 -- 172.21.15.43:0/2883711659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff72415cbe0 msgr2=0x7ff7240b1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:19.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.469+0000 7ff729570640 1 --2- 172.21.15.43:0/2883711659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff72415cbe0 0x7ff7240b1630 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7ff71c007e60 tx=0x7ff71c00e4b0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.469+0000 7ff729570640 1 -- 172.21.15.43:0/2883711659 shutdown_connections 2024-04-25T09:01:19.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.469+0000 7ff729570640 1 --2- 172.21.15.43:0/2883711659 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff72415cbe0 0x7ff7240b1630 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.469+0000 7ff729570640 1 --2- 172.21.15.43:0/2883711659 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff71006b4d0 0x7ff71006d990 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.469+0000 7ff729570640 1 --2- 172.21.15.43:0/2883711659 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7240acf60 0x7ff7240b10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.469+0000 7ff729570640 1 --2- 172.21.15.43:0/2883711659 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff72406ab30 0x7ff72414ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.469+0000 7ff729570640 1 -- 172.21.15.43:0/2883711659 >> 172.21.15.43:0/2883711659 conn(0x7ff7240b5ad0 msgr2=0x7ff7240ad900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:19.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.469+0000 7ff729570640 1 -- 172.21.15.43:0/2883711659 shutdown_connections 2024-04-25T09:01:19.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.469+0000 7ff729570640 1 -- 172.21.15.43:0/2883711659 wait complete. 2024-04-25T09:01:19.469 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:01:19.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.583+0000 7f87b7ef5640 1 Processor -- start 2024-04-25T09:01:19.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.584+0000 7f87b7ef5640 1 -- start start 2024-04-25T09:01:19.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.584+0000 7f87b7ef5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87b00b6110 0x7f87b00d1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:19.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.584+0000 7f87b7ef5640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f87b00d2340 0x7f87b01656d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:19.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.584+0000 7f87b7ef5640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b00b32a0 con 0x7f87b00d1a20 2024-04-25T09:01:19.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.584+0000 7f87b7ef5640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87b00cf430 con 0x7f87b00d2340 2024-04-25T09:01:19.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.584+0000 7f87b7ef5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b015bf90 con 0x7f87b00b6110 2024-04-25T09:01:19.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.585+0000 7f87b5490640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87b00b6110 0x7f87b00d1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:19.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.585+0000 7f87b6492640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f87b00d2340 0x7f87b01656d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:19.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.585+0000 7f87b5c91640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f87b00d1a20 0x7f87b00b2b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:52194/0 (socket says 172.21.15.43:52194) 2024-04-25T09:01:19.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.585+0000 7f87b5c91640 1 -- 172.21.15.43:0/1120067362 learned_addr learned my addr 172.21.15.43:0/1120067362 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:19.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.585+0000 7f87b6492640 1 -- 172.21.15.43:0/1120067362 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87b00b6110 msgr2=0x7f87b00d1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:19.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.585+0000 7f87b6492640 1 --2- 172.21.15.43:0/1120067362 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87b00b6110 0x7f87b00d1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.585+0000 7f87b6492640 1 -- 172.21.15.43:0/1120067362 >> v1:172.21.15.193:6789/0 conn(0x7f87b00d1a20 legacy=0x7f87b00b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:19.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.585+0000 7f87b6492640 1 -- 172.21.15.43:0/1120067362 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b00cee70 con 0x7f87b00d2340 2024-04-25T09:01:19.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.586+0000 7f87b6492640 1 --2- 172.21.15.43:0/1120067362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87b00d2340 0x7f87b01656d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f87a8009d90 tx=0x7f87a802a530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c71ffd7b6df8d4f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:19.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.586+0000 7f87b4c8f640 1 -- 172.21.15.43:0/1120067362 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a8007740 con 0x7f87b00d2340 2024-04-25T09:01:19.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.586+0000 7f87b4c8f640 1 -- 172.21.15.43:0/1120067362 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a80078e0 con 0x7f87b00d2340 2024-04-25T09:01:19.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.586+0000 7f87b4c8f640 1 -- 172.21.15.43:0/1120067362 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a8007cb0 con 0x7f87b00d2340 2024-04-25T09:01:19.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.587+0000 7f87b7ef5640 1 -- 172.21.15.43:0/1120067362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87b00d2340 msgr2=0x7f87b01656d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:19.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.587+0000 7f87b7ef5640 1 --2- 172.21.15.43:0/1120067362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87b00d2340 0x7f87b01656d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f87a8009d90 tx=0x7f87a802a530 comp rx=0 tx=0).stop 2024-04-25T09:01:19.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.587+0000 7f87b7ef5640 1 -- 172.21.15.43:0/1120067362 shutdown_connections 2024-04-25T09:01:19.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.587+0000 7f87b7ef5640 1 --2- 172.21.15.43:0/1120067362 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87b00d2340 0x7f87b01656d0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.587+0000 7f87b7ef5640 1 --2- 172.21.15.43:0/1120067362 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87b00b6110 0x7f87b00d1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.587+0000 7f87b7ef5640 1 -- 172.21.15.43:0/1120067362 >> 172.21.15.43:0/1120067362 conn(0x7f87b0072aa0 msgr2=0x7f87b009aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:19.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.587+0000 7f87b7ef5640 1 -- 172.21.15.43:0/1120067362 shutdown_connections 2024-04-25T09:01:19.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.587+0000 7f87b7ef5640 1 -- 172.21.15.43:0/1120067362 wait complete. 2024-04-25T09:01:19.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.588+0000 7f87b7ef5640 1 Processor -- start 2024-04-25T09:01:19.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.588+0000 7f87b7ef5640 1 -- start start 2024-04-25T09:01:19.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.589+0000 7f87b7ef5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87b00b6110 0x7f87b014a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:19.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.589+0000 7f87b7ef5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87b00d1a20 0x7f87b0158be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:19.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.589+0000 7f87b7ef5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87b00d2340 0x7f87b0159120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:19.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.589+0000 7f87b7ef5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b00b4be0 con 0x7f87b00d2340 2024-04-25T09:01:19.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.589+0000 7f87b7ef5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b00b4900 con 0x7f87b00d1a20 2024-04-25T09:01:19.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.589+0000 7f87b7ef5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b00cea60 con 0x7f87b00b6110 2024-04-25T09:01:19.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.589+0000 7f87b6492640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87b00d2340 0x7f87b0159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:19.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.589+0000 7f87b6492640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87b00d2340 0x7f87b0159120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42680/0 (socket says 172.21.15.43:42680) 2024-04-25T09:01:19.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.589+0000 7f87b6492640 1 -- 172.21.15.43:0/1523714305 learned_addr learned my addr 172.21.15.43:0/1523714305 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:19.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.589+0000 7f87b5490640 1 --2- 172.21.15.43:0/1523714305 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87b00d1a20 0x7f87b0158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:19.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.589+0000 7f87b5c91640 1 --2- 172.21.15.43:0/1523714305 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87b00b6110 0x7f87b014a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:19.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.589+0000 7f87b6492640 1 -- 172.21.15.43:0/1523714305 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87b00b6110 msgr2=0x7f87b014a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:19.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.589+0000 7f87b6492640 1 --2- 172.21.15.43:0/1523714305 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87b00b6110 0x7f87b014a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.589+0000 7f87b6492640 1 -- 172.21.15.43:0/1523714305 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87b00d1a20 msgr2=0x7f87b0158be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:19.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.589+0000 7f87b6492640 1 --2- 172.21.15.43:0/1523714305 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87b00d1a20 0x7f87b0158be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.589+0000 7f87b6492640 1 -- 172.21.15.43:0/1523714305 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b0155300 con 0x7f87b00d2340 2024-04-25T09:01:19.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.589+0000 7f87b5c91640 1 --2- 172.21.15.43:0/1523714305 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87b00b6110 0x7f87b014a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:01:19.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.589+0000 7f87b6492640 1 --2- 172.21.15.43:0/1523714305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87b00d2340 0x7f87b0159120 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f87a8009eb0 tx=0x7f87a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:19.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.590+0000 7f87a67fc640 1 -- 172.21.15.43:0/1523714305 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a8040070 con 0x7f87b00d2340 2024-04-25T09:01:19.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.590+0000 7f87a67fc640 1 -- 172.21.15.43:0/1523714305 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a8031bc0 con 0x7f87b00d2340 2024-04-25T09:01:19.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.590+0000 7f87b7ef5640 1 -- 172.21.15.43:0/1523714305 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b0155580 con 0x7f87b00d2340 2024-04-25T09:01:19.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.590+0000 7f87a67fc640 1 -- 172.21.15.43:0/1523714305 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a8009450 con 0x7f87b00d2340 2024-04-25T09:01:19.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.590+0000 7f87b7ef5640 1 -- 172.21.15.43:0/1523714305 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b0155b40 con 0x7f87b00d2340 2024-04-25T09:01:19.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.591+0000 7f87b7ef5640 1 -- 172.21.15.43:0/1523714305 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b0074f60 con 0x7f87b00d2340 2024-04-25T09:01:19.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.591+0000 7f87a67fc640 1 -- 172.21.15.43:0/1523714305 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f87a8039040 con 0x7f87b00d2340 2024-04-25T09:01:19.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.592+0000 7f87a67fc640 1 --2- 172.21.15.43:0/1523714305 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f879406b4d0 0x7f879406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:19.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.592+0000 7f87a67fc640 1 -- 172.21.15.43:0/1523714305 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f87a80aba40 con 0x7f87b00d2340 2024-04-25T09:01:19.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.592+0000 7f87b5c91640 1 --2- 172.21.15.43:0/1523714305 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f879406b4d0 0x7f879406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:19.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.592+0000 7f87b5c91640 1 --2- 172.21.15.43:0/1523714305 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f879406b4d0 0x7f879406d990 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f87ac000f60 tx=0x7f87ac009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:19.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.596+0000 7f87a67fc640 1 -- 172.21.15.43:0/1523714305 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f87b0155300 con 0x7f87b00d2340 2024-04-25T09:01:19.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.776+0000 7f87b7ef5640 1 -- 172.21.15.43:0/1523714305 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f87b006d7e0 con 0x7f87b00d2340 2024-04-25T09:01:19.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.776+0000 7f87a67fc640 1 -- 172.21.15.43:0/1523714305 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) v1 ==== 104+0+11 (secure 0 0 0) 0x7f87a807a8a0 con 0x7f87b00d2340 2024-04-25T09:01:19.765 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T09:01:19.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.778+0000 7f87b7ef5640 1 -- 172.21.15.43:0/1523714305 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f879406b4d0 msgr2=0x7f879406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:19.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.778+0000 7f87b7ef5640 1 --2- 172.21.15.43:0/1523714305 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f879406b4d0 0x7f879406d990 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f87ac000f60 tx=0x7f87ac009040 comp rx=0 tx=0).stop 2024-04-25T09:01:19.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.778+0000 7f87b7ef5640 1 -- 172.21.15.43:0/1523714305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87b00d2340 msgr2=0x7f87b0159120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:19.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.778+0000 7f87b7ef5640 1 --2- 172.21.15.43:0/1523714305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87b00d2340 0x7f87b0159120 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f87a8009eb0 tx=0x7f87a8004a40 comp rx=0 tx=0).stop 2024-04-25T09:01:19.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.778+0000 7f87b7ef5640 1 -- 172.21.15.43:0/1523714305 shutdown_connections 2024-04-25T09:01:19.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.778+0000 7f87b7ef5640 1 --2- 172.21.15.43:0/1523714305 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87b00d2340 0x7f87b0159120 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.778+0000 7f87b7ef5640 1 --2- 172.21.15.43:0/1523714305 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f879406b4d0 0x7f879406d990 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.778+0000 7f87b7ef5640 1 --2- 172.21.15.43:0/1523714305 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87b00d1a20 0x7f87b0158be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.778+0000 7f87b7ef5640 1 --2- 172.21.15.43:0/1523714305 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87b00b6110 0x7f87b014a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.778+0000 7f87b7ef5640 1 -- 172.21.15.43:0/1523714305 >> 172.21.15.43:0/1523714305 conn(0x7f87b0072aa0 msgr2=0x7f87b006ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:19.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.778+0000 7f87b7ef5640 1 -- 172.21.15.43:0/1523714305 shutdown_connections 2024-04-25T09:01:19.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.778+0000 7f87b7ef5640 1 -- 172.21.15.43:0/1523714305 wait complete. 2024-04-25T09:01:19.778 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:01:19.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.893+0000 7f03f8189640 1 Processor -- start 2024-04-25T09:01:19.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.894+0000 7f03f8189640 1 -- start start 2024-04-25T09:01:19.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.894+0000 7f03f8189640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f03f015e090 0x7f03f0161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:19.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.894+0000 7f03f8189640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f03f0161fe0 0x7f03f00b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:19.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.894+0000 7f03f8189640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03f00b5aa0 0x7f03f00ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:19.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.894+0000 7f03f8189640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03f00d1da0 con 0x7f03f015e090 2024-04-25T09:01:19.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.894+0000 7f03f8189640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03f0162610 con 0x7f03f0161fe0 2024-04-25T09:01:19.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.894+0000 7f03f8189640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03f00d1750 con 0x7f03f00b5aa0 2024-04-25T09:01:19.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.895+0000 7f03f5f25640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f03f015e090 0x7f03f0161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:19.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.895+0000 7f03f5f25640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f03f015e090 0x7f03f0161aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57988/0 (socket says 172.21.15.43:57988) 2024-04-25T09:01:19.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.895+0000 7f03f5f25640 1 -- 172.21.15.43:0/3390069183 learned_addr learned my addr 172.21.15.43:0/3390069183 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:19.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.895+0000 7f03f5724640 1 --2- 172.21.15.43:0/3390069183 >> v2:172.21.15.193:3300/0 conn(0x7f03f0161fe0 0x7f03f00b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:19.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.895+0000 7f03f6726640 1 --2- 172.21.15.43:0/3390069183 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03f00b5aa0 0x7f03f00ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:19.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.895+0000 7f03f5f25640 1 -- 172.21.15.43:0/3390069183 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03f00b5aa0 msgr2=0x7f03f00ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:19.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.895+0000 7f03f5f25640 1 --2- 172.21.15.43:0/3390069183 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03f00b5aa0 0x7f03f00ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.895+0000 7f03f5f25640 1 -- 172.21.15.43:0/3390069183 >> v2:172.21.15.193:3300/0 conn(0x7f03f0161fe0 msgr2=0x7f03f00b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:19.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.895+0000 7f03f5f25640 1 --2- 172.21.15.43:0/3390069183 >> v2:172.21.15.193:3300/0 conn(0x7f03f0161fe0 0x7f03f00b5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.895+0000 7f03f5f25640 1 -- 172.21.15.43:0/3390069183 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f00d1a30 con 0x7f03f015e090 2024-04-25T09:01:19.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.896+0000 7f03f5f25640 1 --2- 172.21.15.43:0/3390069183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03f015e090 0x7f03f0161aa0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f03f00cfe10 tx=0x7f03e802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b256bc25f3593a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:19.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.896+0000 7f03f4f23640 1 -- 172.21.15.43:0/3390069183 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e8037070 con 0x7f03f015e090 2024-04-25T09:01:19.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.896+0000 7f03f4f23640 1 -- 172.21.15.43:0/3390069183 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e8032440 con 0x7f03f015e090 2024-04-25T09:01:19.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.896+0000 7f03f4f23640 1 -- 172.21.15.43:0/3390069183 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e8032810 con 0x7f03f015e090 2024-04-25T09:01:19.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.896+0000 7f03f8189640 1 -- 172.21.15.43:0/3390069183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03f015e090 msgr2=0x7f03f0161aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:19.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.896+0000 7f03f8189640 1 --2- 172.21.15.43:0/3390069183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03f015e090 0x7f03f0161aa0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f03f00cfe10 tx=0x7f03e802a270 comp rx=0 tx=0).stop 2024-04-25T09:01:19.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.896+0000 7f03f8189640 1 -- 172.21.15.43:0/3390069183 shutdown_connections 2024-04-25T09:01:19.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.896+0000 7f03f8189640 1 --2- 172.21.15.43:0/3390069183 >> v2:172.21.15.193:3300/0 conn(0x7f03f0161fe0 0x7f03f00b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.896+0000 7f03f8189640 1 --2- 172.21.15.43:0/3390069183 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03f00b5aa0 0x7f03f00ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.896+0000 7f03f8189640 1 --2- 172.21.15.43:0/3390069183 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03f015e090 0x7f03f0161aa0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.896+0000 7f03f8189640 1 -- 172.21.15.43:0/3390069183 >> 172.21.15.43:0/3390069183 conn(0x7f03f00d0560 msgr2=0x7f03f00d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:19.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.896+0000 7f03f8189640 1 -- 172.21.15.43:0/3390069183 shutdown_connections 2024-04-25T09:01:19.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.896+0000 7f03f8189640 1 -- 172.21.15.43:0/3390069183 wait complete. 2024-04-25T09:01:19.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.898+0000 7f03f8189640 1 Processor -- start 2024-04-25T09:01:19.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.898+0000 7f03f8189640 1 -- start start 2024-04-25T09:01:19.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.898+0000 7f03f8189640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03f00b5aa0 0x7f03f014e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:19.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.898+0000 7f03f8189640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03f015e090 0x7f03f00cc740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:19.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.898+0000 7f03f8189640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03f0161fe0 0x7f03f00ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:19.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.898+0000 7f03f8189640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f00ceaa0 con 0x7f03f00b5aa0 2024-04-25T09:01:19.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.898+0000 7f03f8189640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f00b3280 con 0x7f03f015e090 2024-04-25T09:01:19.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.898+0000 7f03f8189640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03f00cec20 con 0x7f03f0161fe0 2024-04-25T09:01:19.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.898+0000 7f03f5f25640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03f00b5aa0 0x7f03f014e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:19.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.898+0000 7f03f5f25640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03f00b5aa0 0x7f03f014e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57998/0 (socket says 172.21.15.43:57998) 2024-04-25T09:01:19.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.898+0000 7f03f5f25640 1 -- 172.21.15.43:0/1659913409 learned_addr learned my addr 172.21.15.43:0/1659913409 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:19.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.899+0000 7f03f5724640 1 --2- 172.21.15.43:0/1659913409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03f015e090 0x7f03f00cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:19.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.899+0000 7f03f6726640 1 --2- 172.21.15.43:0/1659913409 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03f0161fe0 0x7f03f00ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:19.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.899+0000 7f03f5f25640 1 -- 172.21.15.43:0/1659913409 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03f0161fe0 msgr2=0x7f03f00ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:19.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.899+0000 7f03f5f25640 1 --2- 172.21.15.43:0/1659913409 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03f0161fe0 0x7f03f00ccc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.899+0000 7f03f5f25640 1 -- 172.21.15.43:0/1659913409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03f015e090 msgr2=0x7f03f00cc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:19.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.899+0000 7f03f5f25640 1 --2- 172.21.15.43:0/1659913409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03f015e090 0x7f03f00cc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:19.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.899+0000 7f03f5f25640 1 -- 172.21.15.43:0/1659913409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f00cb8e0 con 0x7f03f00b5aa0 2024-04-25T09:01:19.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.899+0000 7f03f5f25640 1 --2- 172.21.15.43:0/1659913409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03f00b5aa0 0x7f03f014e020 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f03e80097c0 tx=0x7f03e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:19.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.899+0000 7f03ee7fc640 1 -- 172.21.15.43:0/1659913409 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e8041020 con 0x7f03f00b5aa0 2024-04-25T09:01:19.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.899+0000 7f03ee7fc640 1 -- 172.21.15.43:0/1659913409 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e803d620 con 0x7f03f00b5aa0 2024-04-25T09:01:19.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.899+0000 7f03f8189640 1 -- 172.21.15.43:0/1659913409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03f00cbbc0 con 0x7f03f00b5aa0 2024-04-25T09:01:19.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.899+0000 7f03ee7fc640 1 -- 172.21.15.43:0/1659913409 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e8037040 con 0x7f03f00b5aa0 2024-04-25T09:01:19.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.899+0000 7f03f8189640 1 -- 172.21.15.43:0/1659913409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03f015e470 con 0x7f03f00b5aa0 2024-04-25T09:01:19.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.900+0000 7f03f8189640 1 -- 172.21.15.43:0/1659913409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03f0074110 con 0x7f03f00b5aa0 2024-04-25T09:01:19.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.901+0000 7f03ee7fc640 1 -- 172.21.15.43:0/1659913409 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f03e8044020 con 0x7f03f00b5aa0 2024-04-25T09:01:19.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.901+0000 7f03ee7fc640 1 --2- 172.21.15.43:0/1659913409 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f03dc06b4d0 0x7f03dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:19.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.901+0000 7f03ee7fc640 1 -- 172.21.15.43:0/1659913409 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f03e80aa860 con 0x7f03f00b5aa0 2024-04-25T09:01:19.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.901+0000 7f03f5724640 1 --2- 172.21.15.43:0/1659913409 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f03dc06b4d0 0x7f03dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:19.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.902+0000 7f03f5724640 1 --2- 172.21.15.43:0/1659913409 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f03dc06b4d0 0x7f03dc06d990 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f03f015b850 tx=0x7f03e4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:19.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:19.905+0000 7f03ee7fc640 1 -- 172.21.15.43:0/1659913409 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f03f0074110 con 0x7f03f00b5aa0 2024-04-25T09:01:20.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.083+0000 7f03f8189640 1 -- 172.21.15.43:0/1659913409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f03f006bf00 con 0x7f03f00b5aa0 2024-04-25T09:01:20.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.083+0000 7f03ee7fc640 1 -- 172.21.15.43:0/1659913409 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) v1 ==== 100+0+11 (secure 0 0 0) 0x7f03f006bf00 con 0x7f03f00b5aa0 2024-04-25T09:01:20.072 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T09:01:20.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.085+0000 7f03f8189640 1 -- 172.21.15.43:0/1659913409 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f03dc06b4d0 msgr2=0x7f03dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:20.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.085+0000 7f03f8189640 1 --2- 172.21.15.43:0/1659913409 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f03dc06b4d0 0x7f03dc06d990 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f03f015b850 tx=0x7f03e4008040 comp rx=0 tx=0).stop 2024-04-25T09:01:20.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.085+0000 7f03f8189640 1 -- 172.21.15.43:0/1659913409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03f00b5aa0 msgr2=0x7f03f014e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:20.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.085+0000 7f03f8189640 1 --2- 172.21.15.43:0/1659913409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03f00b5aa0 0x7f03f014e020 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f03e80097c0 tx=0x7f03e80046c0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.085+0000 7f03f8189640 1 -- 172.21.15.43:0/1659913409 shutdown_connections 2024-04-25T09:01:20.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.085+0000 7f03f8189640 1 --2- 172.21.15.43:0/1659913409 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03f0161fe0 0x7f03f00ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.085+0000 7f03f8189640 1 --2- 172.21.15.43:0/1659913409 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f03dc06b4d0 0x7f03dc06d990 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.085+0000 7f03f8189640 1 --2- 172.21.15.43:0/1659913409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03f015e090 0x7f03f00cc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.085+0000 7f03f8189640 1 --2- 172.21.15.43:0/1659913409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03f00b5aa0 0x7f03f014e020 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.085+0000 7f03f8189640 1 -- 172.21.15.43:0/1659913409 >> 172.21.15.43:0/1659913409 conn(0x7f03f00d0560 msgr2=0x7f03f00b6e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:20.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.085+0000 7f03f8189640 1 -- 172.21.15.43:0/1659913409 shutdown_connections 2024-04-25T09:01:20.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.085+0000 7f03f8189640 1 -- 172.21.15.43:0/1659913409 wait complete. 2024-04-25T09:01:20.085 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:01:20.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.197+0000 7f456483f640 1 Processor -- start 2024-04-25T09:01:20.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.197+0000 7f456483f640 1 -- start start 2024-04-25T09:01:20.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.198+0000 7f456483f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f456015dce0 0x7f456015ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:20.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.198+0000 7f456483f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f45600d0af0 0x7f4560160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:20.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.198+0000 7f456483f640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45600cee80 con 0x7f45600ce550 2024-04-25T09:01:20.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.198+0000 7f456483f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45600d2920 con 0x7f45600d0af0 2024-04-25T09:01:20.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.198+0000 7f456483f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45600cf160 con 0x7f456015dce0 2024-04-25T09:01:20.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.198+0000 7f455ed9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f45600d0af0 0x7f4560160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:20.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.198+0000 7f455ed9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f45600d0af0 0x7f4560160980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58014/0 (socket says 172.21.15.43:58014) 2024-04-25T09:01:20.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.198+0000 7f455ed9d640 1 -- 172.21.15.43:0/1693548416 learned_addr learned my addr 172.21.15.43:0/1693548416 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:20.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.198+0000 7f455e59c640 1 --2- 172.21.15.43:0/1693548416 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f456015dce0 0x7f456015ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:20.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.198+0000 7f455ed9d640 1 -- 172.21.15.43:0/1693548416 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f456015dce0 msgr2=0x7f456015ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:20.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.198+0000 7f455ed9d640 1 --2- 172.21.15.43:0/1693548416 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f456015dce0 0x7f456015ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.199+0000 7f455ed9d640 1 -- 172.21.15.43:0/1693548416 >> v1:172.21.15.193:6789/0 conn(0x7f45600ce550 legacy=0x7f45600d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:20.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.199+0000 7f455ed9d640 1 -- 172.21.15.43:0/1693548416 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45600cf440 con 0x7f45600d0af0 2024-04-25T09:01:20.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.199+0000 7f455ed9d640 1 --2- 172.21.15.43:0/1693548416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45600d0af0 0x7f4560160980 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f4558009e50 tx=0x7f455802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4770ae3d4ff3c55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:20.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.199+0000 7f455d59a640 1 -- 172.21.15.43:0/1693548416 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4558036070 con 0x7f45600d0af0 2024-04-25T09:01:20.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.199+0000 7f455d59a640 1 -- 172.21.15.43:0/1693548416 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4558031600 con 0x7f45600d0af0 2024-04-25T09:01:20.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.199+0000 7f455d59a640 1 -- 172.21.15.43:0/1693548416 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45580319d0 con 0x7f45600d0af0 2024-04-25T09:01:20.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.200+0000 7f456483f640 1 -- 172.21.15.43:0/1693548416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45600d0af0 msgr2=0x7f4560160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:20.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.200+0000 7f456483f640 1 --2- 172.21.15.43:0/1693548416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45600d0af0 0x7f4560160980 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f4558009e50 tx=0x7f455802a680 comp rx=0 tx=0).stop 2024-04-25T09:01:20.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.200+0000 7f456483f640 1 -- 172.21.15.43:0/1693548416 shutdown_connections 2024-04-25T09:01:20.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.200+0000 7f456483f640 1 --2- 172.21.15.43:0/1693548416 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45600d0af0 0x7f4560160980 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.200+0000 7f456483f640 1 --2- 172.21.15.43:0/1693548416 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f456015dce0 0x7f456015ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.200+0000 7f456483f640 1 -- 172.21.15.43:0/1693548416 >> 172.21.15.43:0/1693548416 conn(0x7f456009cd00 msgr2=0x7f45600d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:20.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.200+0000 7f456483f640 1 -- 172.21.15.43:0/1693548416 shutdown_connections 2024-04-25T09:01:20.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.200+0000 7f456483f640 1 -- 172.21.15.43:0/1693548416 wait complete. 2024-04-25T09:01:20.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.201+0000 7f456483f640 1 Processor -- start 2024-04-25T09:01:20.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.201+0000 7f456483f640 1 -- start start 2024-04-25T09:01:20.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.202+0000 7f456483f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45600ce550 0x7f45601f43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:20.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.202+0000 7f456483f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45600d0af0 0x7f45601f4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:20.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.202+0000 7f456483f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f456015dce0 0x7f45600b7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:20.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.202+0000 7f456483f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456015e0c0 con 0x7f45600ce550 2024-04-25T09:01:20.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.202+0000 7f456483f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456015f6b0 con 0x7f456015dce0 2024-04-25T09:01:20.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.202+0000 7f456483f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45600ce990 con 0x7f45600d0af0 2024-04-25T09:01:20.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.202+0000 7f455e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45600ce550 0x7f45601f43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:20.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.202+0000 7f455e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45600ce550 0x7f45601f43f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58022/0 (socket says 172.21.15.43:58022) 2024-04-25T09:01:20.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.202+0000 7f455e59c640 1 -- 172.21.15.43:0/2652852941 learned_addr learned my addr 172.21.15.43:0/2652852941 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:20.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.202+0000 7f455ed9d640 1 --2- 172.21.15.43:0/2652852941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f456015dce0 0x7f45600b7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:20.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.202+0000 7f455dd9b640 1 --2- 172.21.15.43:0/2652852941 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45600d0af0 0x7f45601f4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:20.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.203+0000 7f455e59c640 1 -- 172.21.15.43:0/2652852941 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45600d0af0 msgr2=0x7f45601f4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:20.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.203+0000 7f455e59c640 1 --2- 172.21.15.43:0/2652852941 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45600d0af0 0x7f45601f4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.203+0000 7f455e59c640 1 -- 172.21.15.43:0/2652852941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f456015dce0 msgr2=0x7f45600b7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:20.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.203+0000 7f455e59c640 1 --2- 172.21.15.43:0/2652852941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f456015dce0 0x7f45600b7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.203+0000 7f455e59c640 1 -- 172.21.15.43:0/2652852941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45600b6bc0 con 0x7f45600ce550 2024-04-25T09:01:20.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.203+0000 7f455ed9d640 1 --2- 172.21.15.43:0/2652852941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f456015dce0 0x7f45600b7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:01:20.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.203+0000 7f455e59c640 1 --2- 172.21.15.43:0/2652852941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45600ce550 0x7f45601f43f0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f455000d5d0 tx=0x7f455000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:20.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.203+0000 7f454effd640 1 -- 172.21.15.43:0/2652852941 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45500125f0 con 0x7f45600ce550 2024-04-25T09:01:20.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.203+0000 7f454effd640 1 -- 172.21.15.43:0/2652852941 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4550019070 con 0x7f45600ce550 2024-04-25T09:01:20.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.203+0000 7f456483f640 1 -- 172.21.15.43:0/2652852941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45600b6e60 con 0x7f45600ce550 2024-04-25T09:01:20.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.203+0000 7f454effd640 1 -- 172.21.15.43:0/2652852941 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45500129f0 con 0x7f45600ce550 2024-04-25T09:01:20.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.203+0000 7f456483f640 1 -- 172.21.15.43:0/2652852941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45600b7420 con 0x7f45600ce550 2024-04-25T09:01:20.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.204+0000 7f456483f640 1 -- 172.21.15.43:0/2652852941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45600680d0 con 0x7f45600ce550 2024-04-25T09:01:20.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.204+0000 7f454effd640 1 -- 172.21.15.43:0/2652852941 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4550012b90 con 0x7f45600ce550 2024-04-25T09:01:20.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.205+0000 7f454effd640 1 --2- 172.21.15.43:0/2652852941 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f454006b4d0 0x7f454006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:20.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.205+0000 7f454effd640 1 -- 172.21.15.43:0/2652852941 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f455008a630 con 0x7f45600ce550 2024-04-25T09:01:20.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.205+0000 7f455dd9b640 1 --2- 172.21.15.43:0/2652852941 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f454006b4d0 0x7f454006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:20.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.205+0000 7f455dd9b640 1 --2- 172.21.15.43:0/2652852941 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f454006b4d0 0x7f454006d990 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f456015cc80 tx=0x7f4554009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:20.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.209+0000 7f454effd640 1 -- 172.21.15.43:0/2652852941 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4550059490 con 0x7f45600ce550 2024-04-25T09:01:20.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.428+0000 7f456483f640 1 -- 172.21.15.43:0/2652852941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4560074ef0 con 0x7f45600ce550 2024-04-25T09:01:20.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.428+0000 7f454effd640 1 -- 172.21.15.43:0/2652852941 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f4550058d90 con 0x7f45600ce550 2024-04-25T09:01:20.417 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:01:20.418 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":54,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6200,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2258031526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2258031526},{"type":"v1","addr":"172.21.15.43:6834","nonce":2258031526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6206,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/1773440177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1773440177},{"type":"v1","addr":"172.21.15.193:6836","nonce":1773440177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6209,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3239725307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":3239725307},{"type":"v1","addr":"172.21.15.193:6837","nonce":3239725307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:00:42.536578+0000","modified":"2024-04-25T09:00:44.531104+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7596},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7596":{"gid":7596,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/1341028696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1341028696},{"type":"v1","addr":"172.21.15.43:6835","nonce":1341028696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:01:20.418 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 54 2024-04-25T09:01:20.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.430+0000 7f456483f640 1 -- 172.21.15.43:0/2652852941 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f454006b4d0 msgr2=0x7f454006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:20.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.430+0000 7f456483f640 1 --2- 172.21.15.43:0/2652852941 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f454006b4d0 0x7f454006d990 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f456015cc80 tx=0x7f4554009040 comp rx=0 tx=0).stop 2024-04-25T09:01:20.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.430+0000 7f456483f640 1 -- 172.21.15.43:0/2652852941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45600ce550 msgr2=0x7f45601f43f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:20.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.430+0000 7f456483f640 1 --2- 172.21.15.43:0/2652852941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45600ce550 0x7f45601f43f0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f455000d5d0 tx=0x7f455000da90 comp rx=0 tx=0).stop 2024-04-25T09:01:20.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.430+0000 7f456483f640 1 -- 172.21.15.43:0/2652852941 shutdown_connections 2024-04-25T09:01:20.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.430+0000 7f456483f640 1 --2- 172.21.15.43:0/2652852941 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f454006b4d0 0x7f454006d990 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.430+0000 7f456483f640 1 --2- 172.21.15.43:0/2652852941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f456015dce0 0x7f45600b7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.430+0000 7f456483f640 1 --2- 172.21.15.43:0/2652852941 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45600d0af0 0x7f45601f4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.430+0000 7f456483f640 1 --2- 172.21.15.43:0/2652852941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45600ce550 0x7f45601f43f0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.430+0000 7f456483f640 1 -- 172.21.15.43:0/2652852941 >> 172.21.15.43:0/2652852941 conn(0x7f456009cd00 msgr2=0x7f45600d03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:20.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.430+0000 7f456483f640 1 -- 172.21.15.43:0/2652852941 shutdown_connections 2024-04-25T09:01:20.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.430+0000 7f456483f640 1 -- 172.21.15.43:0/2652852941 wait complete. 2024-04-25T09:01:20.431 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:01:20.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.543+0000 7f13f9438640 1 Processor -- start 2024-04-25T09:01:20.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.543+0000 7f13f9438640 1 -- start start 2024-04-25T09:01:20.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.543+0000 7f13f9438640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13f415ded0 0x7f13f40ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:20.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.543+0000 7f13f9438640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f13f40cf250 0x7f13f40ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:20.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.543+0000 7f13f9438640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f13f406ab30 0x7f13f406afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:20.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.543+0000 7f13f9438640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13f415e2b0 con 0x7f13f406ab30 2024-04-25T09:01:20.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.543+0000 7f13f9438640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13f40cf6e0 con 0x7f13f40cf250 2024-04-25T09:01:20.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.543+0000 7f13f9438640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f40cf9c0 con 0x7f13f415ded0 2024-04-25T09:01:20.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.544+0000 7f13f37fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f13f406ab30 0x7f13f406afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:20.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.544+0000 7f13f37fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f13f406ab30 0x7f13f406afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58030/0 (socket says 172.21.15.43:58030) 2024-04-25T09:01:20.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.544+0000 7f13f37fe640 1 -- 172.21.15.43:0/688757807 learned_addr learned my addr 172.21.15.43:0/688757807 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:20.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.544+0000 7f13f27fc640 1 --2- 172.21.15.43:0/688757807 >> v2:172.21.15.193:3300/0 conn(0x7f13f40cf250 0x7f13f40ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:20.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.544+0000 7f13f2ffd640 1 --2- 172.21.15.43:0/688757807 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13f415ded0 0x7f13f40ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:20.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.544+0000 7f13f37fe640 1 -- 172.21.15.43:0/688757807 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13f415ded0 msgr2=0x7f13f40ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:20.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.544+0000 7f13f37fe640 1 --2- 172.21.15.43:0/688757807 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13f415ded0 0x7f13f40ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.544+0000 7f13f37fe640 1 -- 172.21.15.43:0/688757807 >> v2:172.21.15.193:3300/0 conn(0x7f13f40cf250 msgr2=0x7f13f40ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:20.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.544+0000 7f13f37fe640 1 --2- 172.21.15.43:0/688757807 >> v2:172.21.15.193:3300/0 conn(0x7f13f40cf250 0x7f13f40ad8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.544+0000 7f13f37fe640 1 -- 172.21.15.43:0/688757807 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f40cfca0 con 0x7f13f406ab30 2024-04-25T09:01:20.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.545+0000 7f13f37fe640 1 --2- 172.21.15.43:0/688757807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13f406ab30 0x7f13f406afc0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f13f406bc10 tx=0x7f13ec02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e5f93e0504d5343 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:20.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.545+0000 7f13f1ffb640 1 -- 172.21.15.43:0/688757807 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ec036070 con 0x7f13f406ab30 2024-04-25T09:01:20.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.545+0000 7f13f1ffb640 1 -- 172.21.15.43:0/688757807 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13ec031600 con 0x7f13f406ab30 2024-04-25T09:01:20.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.545+0000 7f13f1ffb640 1 -- 172.21.15.43:0/688757807 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ec0319d0 con 0x7f13f406ab30 2024-04-25T09:01:20.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.545+0000 7f13f9438640 1 -- 172.21.15.43:0/688757807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13f406ab30 msgr2=0x7f13f406afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:20.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.545+0000 7f13f9438640 1 --2- 172.21.15.43:0/688757807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13f406ab30 0x7f13f406afc0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f13f406bc10 tx=0x7f13ec02a6a0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.545+0000 7f13f9438640 1 -- 172.21.15.43:0/688757807 shutdown_connections 2024-04-25T09:01:20.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.545+0000 7f13f9438640 1 --2- 172.21.15.43:0/688757807 >> v2:172.21.15.193:3300/0 conn(0x7f13f40cf250 0x7f13f40ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.545+0000 7f13f9438640 1 --2- 172.21.15.43:0/688757807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13f406ab30 0x7f13f406afc0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.545+0000 7f13f9438640 1 --2- 172.21.15.43:0/688757807 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13f415ded0 0x7f13f40ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.545+0000 7f13f9438640 1 -- 172.21.15.43:0/688757807 >> 172.21.15.43:0/688757807 conn(0x7f13f40b2490 msgr2=0x7f13f40b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:20.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.546+0000 7f13f9438640 1 -- 172.21.15.43:0/688757807 shutdown_connections 2024-04-25T09:01:20.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.546+0000 7f13f9438640 1 -- 172.21.15.43:0/688757807 wait complete. 2024-04-25T09:01:20.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.547+0000 7f13f9438640 1 Processor -- start 2024-04-25T09:01:20.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.547+0000 7f13f9438640 1 -- start start 2024-04-25T09:01:20.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.548+0000 7f13f9438640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13f406ab30 0x7f13f4141f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:20.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.548+0000 7f13f9438640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13f415ded0 0x7f13f406ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:20.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.548+0000 7f13f9438640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13f4145510 0x7f13f4152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:20.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.548+0000 7f13f9438640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f40cefd0 con 0x7f13f4145510 2024-04-25T09:01:20.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.548+0000 7f13f9438640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f40cee50 con 0x7f13f406ab30 2024-04-25T09:01:20.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.548+0000 7f13f9438640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f40b4db0 con 0x7f13f415ded0 2024-04-25T09:01:20.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.548+0000 7f13f37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13f4145510 0x7f13f4152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:20.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.548+0000 7f13f37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13f4145510 0x7f13f4152dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58034/0 (socket says 172.21.15.43:58034) 2024-04-25T09:01:20.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.548+0000 7f13f37fe640 1 -- 172.21.15.43:0/3185045345 learned_addr learned my addr 172.21.15.43:0/3185045345 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:20.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.548+0000 7f13f27fc640 1 --2- 172.21.15.43:0/3185045345 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13f415ded0 0x7f13f406ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:20.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.548+0000 7f13f2ffd640 1 --2- 172.21.15.43:0/3185045345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13f406ab30 0x7f13f4141f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:20.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.548+0000 7f13f37fe640 1 -- 172.21.15.43:0/3185045345 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13f415ded0 msgr2=0x7f13f406ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:20.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.548+0000 7f13f37fe640 1 --2- 172.21.15.43:0/3185045345 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13f415ded0 0x7f13f406ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.548+0000 7f13f37fe640 1 -- 172.21.15.43:0/3185045345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13f406ab30 msgr2=0x7f13f4141f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:20.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.548+0000 7f13f37fe640 1 --2- 172.21.15.43:0/3185045345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13f406ab30 0x7f13f4141f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.548+0000 7f13f37fe640 1 -- 172.21.15.43:0/3185045345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f4142470 con 0x7f13f4145510 2024-04-25T09:01:20.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.548+0000 7f13f37fe640 1 --2- 172.21.15.43:0/3185045345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13f4145510 0x7f13f4152dd0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f13ec009f40 tx=0x7f13ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:20.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.548+0000 7f13db7fe640 1 -- 172.21.15.43:0/3185045345 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ec041020 con 0x7f13f4145510 2024-04-25T09:01:20.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.549+0000 7f13db7fe640 1 -- 172.21.15.43:0/3185045345 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13ec0077a0 con 0x7f13f4145510 2024-04-25T09:01:20.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.549+0000 7f13f9438640 1 -- 172.21.15.43:0/3185045345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13f406ee00 con 0x7f13f4145510 2024-04-25T09:01:20.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.549+0000 7f13db7fe640 1 -- 172.21.15.43:0/3185045345 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13ec036040 con 0x7f13f4145510 2024-04-25T09:01:20.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.549+0000 7f13f9438640 1 -- 172.21.15.43:0/3185045345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13f4142820 con 0x7f13f4145510 2024-04-25T09:01:20.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.550+0000 7f13f9438640 1 -- 172.21.15.43:0/3185045345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13f4074ef0 con 0x7f13f4145510 2024-04-25T09:01:20.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.550+0000 7f13db7fe640 1 -- 172.21.15.43:0/3185045345 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f13ec044020 con 0x7f13f4145510 2024-04-25T09:01:20.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.550+0000 7f13db7fe640 1 --2- 172.21.15.43:0/3185045345 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f13e006b4d0 0x7f13e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:20.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.551+0000 7f13db7fe640 1 -- 172.21.15.43:0/3185045345 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f13ec0aafe0 con 0x7f13f4145510 2024-04-25T09:01:20.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.551+0000 7f13f2ffd640 1 --2- 172.21.15.43:0/3185045345 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f13e006b4d0 0x7f13e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:20.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.551+0000 7f13f2ffd640 1 --2- 172.21.15.43:0/3185045345 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f13e006b4d0 0x7f13e006d990 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f13f415e840 tx=0x7f13e4009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:20.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.555+0000 7f13db7fe640 1 -- 172.21.15.43:0/3185045345 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f13ec04b020 con 0x7f13f4145510 2024-04-25T09:01:20.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.739+0000 7f13f9438640 1 -- 172.21.15.43:0/3185045345 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f13f40b31e0 con 0x7f13f4145510 2024-04-25T09:01:20.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.740+0000 7f13db7fe640 1 -- 172.21.15.43:0/3185045345 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+16674 (secure 0 0 0) 0x7f13ec079990 con 0x7f13f4145510 2024-04-25T09:01:20.729 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:01:20.729 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":91,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:00:54.209500+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:00:40.816230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T09:00:41.508021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.26","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:01:20.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.742+0000 7f13f9438640 1 -- 172.21.15.43:0/3185045345 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f13e006b4d0 msgr2=0x7f13e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:20.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.742+0000 7f13f9438640 1 --2- 172.21.15.43:0/3185045345 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f13e006b4d0 0x7f13e006d990 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f13f415e840 tx=0x7f13e4009190 comp rx=0 tx=0).stop 2024-04-25T09:01:20.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.742+0000 7f13f9438640 1 -- 172.21.15.43:0/3185045345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13f4145510 msgr2=0x7f13f4152dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:20.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.742+0000 7f13f9438640 1 --2- 172.21.15.43:0/3185045345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13f4145510 0x7f13f4152dd0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f13ec009f40 tx=0x7f13ec004a40 comp rx=0 tx=0).stop 2024-04-25T09:01:20.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.742+0000 7f13f9438640 1 -- 172.21.15.43:0/3185045345 shutdown_connections 2024-04-25T09:01:20.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.742+0000 7f13f9438640 1 --2- 172.21.15.43:0/3185045345 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13f4145510 0x7f13f4152dd0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.742+0000 7f13f9438640 1 --2- 172.21.15.43:0/3185045345 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13f415ded0 0x7f13f406ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.742+0000 7f13f9438640 1 --2- 172.21.15.43:0/3185045345 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f13e006b4d0 0x7f13e006d990 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.742+0000 7f13f9438640 1 --2- 172.21.15.43:0/3185045345 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13f406ab30 0x7f13f4141f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.742+0000 7f13f9438640 1 -- 172.21.15.43:0/3185045345 >> 172.21.15.43:0/3185045345 conn(0x7f13f40b2490 msgr2=0x7f13f40b3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:20.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.742+0000 7f13f9438640 1 -- 172.21.15.43:0/3185045345 shutdown_connections 2024-04-25T09:01:20.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.742+0000 7f13f9438640 1 -- 172.21.15.43:0/3185045345 wait complete. 2024-04-25T09:01:20.742 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:01:20.743 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:01:20.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.855+0000 7fb639d30640 1 Processor -- start 2024-04-25T09:01:20.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.855+0000 7fb639d30640 1 -- start start 2024-04-25T09:01:20.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.855+0000 7fb639d30640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb63406d9a0 0x7fb634071a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:20.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.856+0000 7fb639d30640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb634071f90 0x7fb6340cd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:20.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.856+0000 7fb639d30640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6340723e0 con 0x7fb63415a9d0 2024-04-25T09:01:20.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.856+0000 7fb639d30640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6340704f0 con 0x7fb63406d9a0 2024-04-25T09:01:20.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.856+0000 7fb639d30640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6340707d0 con 0x7fb634071f90 2024-04-25T09:01:20.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.856+0000 7fb6337fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb63415a9d0 0x7fb63406d290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48916/0 (socket says 172.21.15.43:48916) 2024-04-25T09:01:20.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.856+0000 7fb632ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb63406d9a0 0x7fb634071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:20.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.856+0000 7fb6337fe640 1 -- 172.21.15.43:0/1366915262 learned_addr learned my addr 172.21.15.43:0/1366915262 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:20.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.856+0000 7fb633fff640 1 --2- 172.21.15.43:0/1366915262 >> v2:172.21.15.193:3300/0 conn(0x7fb634071f90 0x7fb6340cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:20.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.856+0000 7fb632ffd640 1 -- 172.21.15.43:0/1366915262 >> v2:172.21.15.193:3300/0 conn(0x7fb634071f90 msgr2=0x7fb6340cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:20.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.856+0000 7fb632ffd640 1 --2- 172.21.15.43:0/1366915262 >> v2:172.21.15.193:3300/0 conn(0x7fb634071f90 0x7fb6340cd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.856+0000 7fb632ffd640 1 -- 172.21.15.43:0/1366915262 >> v1:172.21.15.43:6789/0 conn(0x7fb63415a9d0 legacy=0x7fb63406d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:20.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.856+0000 7fb632ffd640 1 -- 172.21.15.43:0/1366915262 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb634070ab0 con 0x7fb63406d9a0 2024-04-25T09:01:20.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.856+0000 7fb6327fc640 1 -- 172.21.15.43:0/1366915262 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1921727397 0 0) 0x7fb6340723e0 con 0x7fb63415a9d0 2024-04-25T09:01:20.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.856+0000 7fb633fff640 1 --2- 172.21.15.43:0/1366915262 >> v2:172.21.15.193:3300/0 conn(0x7fb634071f90 0x7fb6340cd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:20.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.857+0000 7fb632ffd640 1 --2- 172.21.15.43:0/1366915262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb63406d9a0 0x7fb634071a50 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fb6340b2810 tx=0x7fb62802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55083dfd4c1c0199 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:20.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.857+0000 7fb6327fc640 1 -- 172.21.15.43:0/1366915262 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb628037070 con 0x7fb63406d9a0 2024-04-25T09:01:20.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.857+0000 7fb6327fc640 1 -- 172.21.15.43:0/1366915262 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb628032440 con 0x7fb63406d9a0 2024-04-25T09:01:20.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.857+0000 7fb6327fc640 1 -- 172.21.15.43:0/1366915262 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb628032810 con 0x7fb63406d9a0 2024-04-25T09:01:20.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.857+0000 7fb639d30640 1 -- 172.21.15.43:0/1366915262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb63406d9a0 msgr2=0x7fb634071a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:20.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.857+0000 7fb639d30640 1 --2- 172.21.15.43:0/1366915262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb63406d9a0 0x7fb634071a50 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fb6340b2810 tx=0x7fb62802a1a0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.857+0000 7fb639d30640 1 -- 172.21.15.43:0/1366915262 shutdown_connections 2024-04-25T09:01:20.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.857+0000 7fb639d30640 1 --2- 172.21.15.43:0/1366915262 >> v2:172.21.15.193:3300/0 conn(0x7fb634071f90 0x7fb6340cd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.857+0000 7fb639d30640 1 --2- 172.21.15.43:0/1366915262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb63406d9a0 0x7fb634071a50 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.857+0000 7fb639d30640 1 -- 172.21.15.43:0/1366915262 >> 172.21.15.43:0/1366915262 conn(0x7fb6340b1480 msgr2=0x7fb6340b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:20.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.857+0000 7fb639d30640 1 -- 172.21.15.43:0/1366915262 shutdown_connections 2024-04-25T09:01:20.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.858+0000 7fb639d30640 1 -- 172.21.15.43:0/1366915262 wait complete. 2024-04-25T09:01:20.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.859+0000 7fb639d30640 1 Processor -- start 2024-04-25T09:01:20.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.859+0000 7fb639d30640 1 -- start start 2024-04-25T09:01:20.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.860+0000 7fb639d30640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb63406d9a0 0x7fb634149e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:20.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.860+0000 7fb639d30640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb634071f90 0x7fb6340b2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:20.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.860+0000 7fb639d30640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6340b2dc0 0x7fb6341549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:20.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.860+0000 7fb639d30640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6340cde80 con 0x7fb63406d9a0 2024-04-25T09:01:20.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.860+0000 7fb639d30640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6340712b0 con 0x7fb6340b2dc0 2024-04-25T09:01:20.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.860+0000 7fb639d30640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6340ce000 con 0x7fb634071f90 2024-04-25T09:01:20.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.860+0000 7fb6337fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb63406d9a0 0x7fb634149e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:20.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.860+0000 7fb6337fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb63406d9a0 0x7fb634149e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58058/0 (socket says 172.21.15.43:58058) 2024-04-25T09:01:20.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.860+0000 7fb6337fe640 1 -- 172.21.15.43:0/956935071 learned_addr learned my addr 172.21.15.43:0/956935071 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:20.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.860+0000 7fb632ffd640 1 --2- 172.21.15.43:0/956935071 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb634071f90 0x7fb6340b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:20.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.860+0000 7fb633fff640 1 --2- 172.21.15.43:0/956935071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6340b2dc0 0x7fb6341549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:20.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.860+0000 7fb6337fe640 1 -- 172.21.15.43:0/956935071 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb634071f90 msgr2=0x7fb6340b2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:20.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.860+0000 7fb6337fe640 1 --2- 172.21.15.43:0/956935071 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb634071f90 0x7fb6340b2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.860+0000 7fb6337fe640 1 -- 172.21.15.43:0/956935071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6340b2dc0 msgr2=0x7fb6341549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:20.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.860+0000 7fb6337fe640 1 --2- 172.21.15.43:0/956935071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6340b2dc0 0x7fb6341549b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:20.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.860+0000 7fb6337fe640 1 -- 172.21.15.43:0/956935071 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb634155750 con 0x7fb63406d9a0 2024-04-25T09:01:20.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.860+0000 7fb6337fe640 1 --2- 172.21.15.43:0/956935071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb63406d9a0 0x7fb634149e10 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fb62400c430 tx=0x7fb62400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:20.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.860+0000 7fb61bfff640 1 -- 172.21.15.43:0/956935071 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6240074e0 con 0x7fb63406d9a0 2024-04-25T09:01:20.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.860+0000 7fb61bfff640 1 -- 172.21.15.43:0/956935071 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6240047e0 con 0x7fb63406d9a0 2024-04-25T09:01:20.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.860+0000 7fb61bfff640 1 -- 172.21.15.43:0/956935071 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb624004bd0 con 0x7fb63406d9a0 2024-04-25T09:01:20.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.860+0000 7fb639d30640 1 -- 172.21.15.43:0/956935071 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb634155a30 con 0x7fb63406d9a0 2024-04-25T09:01:20.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.860+0000 7fb639d30640 1 -- 172.21.15.43:0/956935071 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6340c6c80 con 0x7fb63406d9a0 2024-04-25T09:01:20.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.861+0000 7fb639d30640 1 -- 172.21.15.43:0/956935071 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6340b31d0 con 0x7fb63406d9a0 2024-04-25T09:01:20.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.862+0000 7fb61bfff640 1 -- 172.21.15.43:0/956935071 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb624011070 con 0x7fb63406d9a0 2024-04-25T09:01:20.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.862+0000 7fb61bfff640 1 --2- 172.21.15.43:0/956935071 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb62006b4d0 0x7fb62006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:20.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.862+0000 7fb61bfff640 1 -- 172.21.15.43:0/956935071 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fb624089f80 con 0x7fb63406d9a0 2024-04-25T09:01:20.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.862+0000 7fb632ffd640 1 --2- 172.21.15.43:0/956935071 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb62006b4d0 0x7fb62006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:20.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.863+0000 7fb632ffd640 1 --2- 172.21.15.43:0/956935071 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb62006b4d0 0x7fb62006d990 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb628034000 tx=0x7fb62802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:20.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:20.866+0000 7fb61bfff640 1 -- 172.21.15.43:0/956935071 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb6340b31d0 con 0x7fb63406d9a0 2024-04-25T09:01:21.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.082+0000 7fb639d30640 1 -- 172.21.15.43:0/956935071 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb63415d9a0 con 0x7fb63406d9a0 2024-04-25T09:01:21.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.083+0000 7fb61bfff640 1 -- 172.21.15.43:0/956935071 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fb624058d40 con 0x7fb63406d9a0 2024-04-25T09:01:21.072 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:01:21.072 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":54,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6200,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2258031526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2258031526},{"type":"v1","addr":"172.21.15.43:6834","nonce":2258031526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6206,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/1773440177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1773440177},{"type":"v1","addr":"172.21.15.193:6836","nonce":1773440177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6209,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3239725307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":3239725307},{"type":"v1","addr":"172.21.15.193:6837","nonce":3239725307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:00:42.536578+0000","modified":"2024-04-25T09:00:44.531104+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7596},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7596":{"gid":7596,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/1341028696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1341028696},{"type":"v1","addr":"172.21.15.43:6835","nonce":1341028696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:01:21.072 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 54 2024-04-25T09:01:21.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.084+0000 7fb639d30640 1 -- 172.21.15.43:0/956935071 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb62006b4d0 msgr2=0x7fb62006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:21.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.084+0000 7fb639d30640 1 --2- 172.21.15.43:0/956935071 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb62006b4d0 0x7fb62006d990 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fb628034000 tx=0x7fb62802e000 comp rx=0 tx=0).stop 2024-04-25T09:01:21.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.084+0000 7fb639d30640 1 -- 172.21.15.43:0/956935071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb63406d9a0 msgr2=0x7fb634149e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:21.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.084+0000 7fb639d30640 1 --2- 172.21.15.43:0/956935071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb63406d9a0 0x7fb634149e10 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fb62400c430 tx=0x7fb62400c8f0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.085+0000 7fb639d30640 1 -- 172.21.15.43:0/956935071 shutdown_connections 2024-04-25T09:01:21.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.085+0000 7fb639d30640 1 --2- 172.21.15.43:0/956935071 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb62006b4d0 0x7fb62006d990 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.085+0000 7fb639d30640 1 --2- 172.21.15.43:0/956935071 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb6340b2dc0 0x7fb6341549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.085+0000 7fb639d30640 1 --2- 172.21.15.43:0/956935071 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb634071f90 0x7fb6340b2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.085+0000 7fb639d30640 1 --2- 172.21.15.43:0/956935071 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb63406d9a0 0x7fb634149e10 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.085+0000 7fb639d30640 1 -- 172.21.15.43:0/956935071 >> 172.21.15.43:0/956935071 conn(0x7fb6340b1480 msgr2=0x7fb6340b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:21.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.085+0000 7fb639d30640 1 -- 172.21.15.43:0/956935071 shutdown_connections 2024-04-25T09:01:21.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.085+0000 7fb639d30640 1 -- 172.21.15.43:0/956935071 wait complete. 2024-04-25T09:01:21.085 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:01:21.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.197+0000 7f1e291a9640 1 Processor -- start 2024-04-25T09:01:21.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.197+0000 7f1e291a9640 1 -- start start 2024-04-25T09:01:21.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.198+0000 7f1e291a9640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1e2406f0e0 0x7f1e240b3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:21.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.198+0000 7f1e291a9640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e2406f4c0 con 0x7f1e2406f930 2024-04-25T09:01:21.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.198+0000 7f1e291a9640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e2415f2f0 con 0x7f1e2406e460 2024-04-25T09:01:21.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.198+0000 7f1e291a9640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e24072130 con 0x7f1e2406f0e0 2024-04-25T09:01:21.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.198+0000 7f1e22d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1e2406f0e0 0x7f1e240b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:21.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.198+0000 7f1e2259c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f1e2406f930 0x7f1e2415e4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48918/0 (socket says 172.21.15.43:48918) 2024-04-25T09:01:21.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.198+0000 7f1e2259c640 1 -- 172.21.15.43:0/579892786 learned_addr learned my addr 172.21.15.43:0/579892786 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:21.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.199+0000 7f1e22d9d640 1 -- 172.21.15.43:0/579892786 >> v1:172.21.15.193:6789/0 conn(0x7f1e2406e460 legacy=0x7f1e2415ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:21.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.199+0000 7f1e22d9d640 1 -- 172.21.15.43:0/579892786 >> v1:172.21.15.43:6789/0 conn(0x7f1e2406f930 legacy=0x7f1e2415e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:21.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.199+0000 7f1e22d9d640 1 -- 172.21.15.43:0/579892786 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e240b3360 con 0x7f1e2406f0e0 2024-04-25T09:01:21.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.199+0000 7f1e21d9b640 1 -- 172.21.15.43:0/579892786 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 292579313 0 0) 0x7f1e2406f4c0 con 0x7f1e2406f930 2024-04-25T09:01:21.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.199+0000 7f1e22d9d640 1 --2- 172.21.15.43:0/579892786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e2406f0e0 0x7f1e240b3790 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c0097a0 tx=0x7f1e1c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10dadff75fe0b6fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:21.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.199+0000 7f1e21d9b640 1 -- 172.21.15.43:0/579892786 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c037070 con 0x7f1e2406f0e0 2024-04-25T09:01:21.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.200+0000 7f1e21d9b640 1 -- 172.21.15.43:0/579892786 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e1c032440 con 0x7f1e2406f0e0 2024-04-25T09:01:21.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.200+0000 7f1e21d9b640 1 -- 172.21.15.43:0/579892786 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c032810 con 0x7f1e2406f0e0 2024-04-25T09:01:21.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.200+0000 7f1e291a9640 1 -- 172.21.15.43:0/579892786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e2406f0e0 msgr2=0x7f1e240b3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:21.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.200+0000 7f1e291a9640 1 --2- 172.21.15.43:0/579892786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e2406f0e0 0x7f1e240b3790 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c0097a0 tx=0x7f1e1c02a1e0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.200+0000 7f1e291a9640 1 -- 172.21.15.43:0/579892786 shutdown_connections 2024-04-25T09:01:21.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.200+0000 7f1e291a9640 1 --2- 172.21.15.43:0/579892786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e2406f0e0 0x7f1e240b3790 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.200+0000 7f1e291a9640 1 -- 172.21.15.43:0/579892786 >> 172.21.15.43:0/579892786 conn(0x7f1e240b2490 msgr2=0x7f1e240b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:21.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.200+0000 7f1e291a9640 1 -- 172.21.15.43:0/579892786 shutdown_connections 2024-04-25T09:01:21.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.200+0000 7f1e291a9640 1 -- 172.21.15.43:0/579892786 wait complete. 2024-04-25T09:01:21.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.202+0000 7f1e291a9640 1 Processor -- start 2024-04-25T09:01:21.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.202+0000 7f1e291a9640 1 -- start start 2024-04-25T09:01:21.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.202+0000 7f1e291a9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e2406e460 0x7f1e24145770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:21.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.202+0000 7f1e291a9640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1e2406f0e0 0x7f1e24148090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:21.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.202+0000 7f1e291a9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e2406f930 0x7f1e240b1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:21.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.202+0000 7f1e291a9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e24071ae0 con 0x7f1e2406f930 2024-04-25T09:01:21.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.202+0000 7f1e291a9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e24071800 con 0x7f1e2406e460 2024-04-25T09:01:21.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.202+0000 7f1e291a9640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e24071dc0 con 0x7f1e2406f0e0 2024-04-25T09:01:21.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.202+0000 7f1e2359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e2406f930 0x7f1e240b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:21.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.202+0000 7f1e2359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e2406f930 0x7f1e240b1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58080/0 (socket says 172.21.15.43:58080) 2024-04-25T09:01:21.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.202+0000 7f1e2359e640 1 -- 172.21.15.43:0/3491977730 learned_addr learned my addr 172.21.15.43:0/3491977730 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:21.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.203+0000 7f1e22d9d640 1 --2- 172.21.15.43:0/3491977730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e2406e460 0x7f1e24145770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:21.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.203+0000 7f1e2259c640 1 --2- 172.21.15.43:0/3491977730 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1e2406f0e0 0x7f1e24148090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:21.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.203+0000 7f1e2359e640 1 -- 172.21.15.43:0/3491977730 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1e2406f0e0 msgr2=0x7f1e24148090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:21.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.203+0000 7f1e2359e640 1 --2- 172.21.15.43:0/3491977730 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1e2406f0e0 0x7f1e24148090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.203+0000 7f1e2359e640 1 -- 172.21.15.43:0/3491977730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e2406e460 msgr2=0x7f1e24145770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:21.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.203+0000 7f1e2359e640 1 --2- 172.21.15.43:0/3491977730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e2406e460 0x7f1e24145770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.203+0000 7f1e2359e640 1 -- 172.21.15.43:0/3491977730 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e24148b30 con 0x7f1e2406f930 2024-04-25T09:01:21.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.203+0000 7f1e2259c640 1 --2- 172.21.15.43:0/3491977730 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1e2406f0e0 0x7f1e24148090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:01:21.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.203+0000 7f1e2359e640 1 --2- 172.21.15.43:0/3491977730 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e2406f930 0x7f1e240b1480 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f1e1400b9a0 tx=0x7f1e1400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:21.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.203+0000 7f1e1b7fe640 1 -- 172.21.15.43:0/3491977730 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e14019070 con 0x7f1e2406f930 2024-04-25T09:01:21.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.203+0000 7f1e1b7fe640 1 -- 172.21.15.43:0/3491977730 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e1400cba0 con 0x7f1e2406f930 2024-04-25T09:01:21.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.203+0000 7f1e1b7fe640 1 -- 172.21.15.43:0/3491977730 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e14014440 con 0x7f1e2406f930 2024-04-25T09:01:21.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.203+0000 7f1e291a9640 1 -- 172.21.15.43:0/3491977730 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e24148e10 con 0x7f1e2406f930 2024-04-25T09:01:21.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.203+0000 7f1e291a9640 1 -- 172.21.15.43:0/3491977730 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e24145cb0 con 0x7f1e2406f930 2024-04-25T09:01:21.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.204+0000 7f1e291a9640 1 -- 172.21.15.43:0/3491977730 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e24068e90 con 0x7f1e2406f930 2024-04-25T09:01:21.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.205+0000 7f1e1b7fe640 1 -- 172.21.15.43:0/3491977730 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1e1401c070 con 0x7f1e2406f930 2024-04-25T09:01:21.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.205+0000 7f1e1b7fe640 1 --2- 172.21.15.43:0/3491977730 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1e0406b4d0 0x7f1e0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:21.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.205+0000 7f1e1b7fe640 1 -- 172.21.15.43:0/3491977730 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f1e14089dc0 con 0x7f1e2406f930 2024-04-25T09:01:21.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.205+0000 7f1e22d9d640 1 --2- 172.21.15.43:0/3491977730 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1e0406b4d0 0x7f1e0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:21.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.206+0000 7f1e22d9d640 1 --2- 172.21.15.43:0/3491977730 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1e0406b4d0 0x7f1e0406d990 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c034000 tx=0x7f1e1c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:21.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.209+0000 7f1e1b7fe640 1 -- 172.21.15.43:0/3491977730 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1e24068e90 con 0x7f1e2406f930 2024-04-25T09:01:21.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.428+0000 7f1e291a9640 1 -- 172.21.15.43:0/3491977730 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1e24146000 con 0x7f1e2406f930 2024-04-25T09:01:21.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.428+0000 7f1e1b7fe640 1 -- 172.21.15.43:0/3491977730 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f1e140585c0 con 0x7f1e2406f930 2024-04-25T09:01:21.417 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:01:21.417 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":54,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6200,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2258031526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2258031526},{"type":"v1","addr":"172.21.15.43:6834","nonce":2258031526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6206,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/1773440177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1773440177},{"type":"v1","addr":"172.21.15.193:6836","nonce":1773440177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6209,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3239725307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":3239725307},{"type":"v1","addr":"172.21.15.193:6837","nonce":3239725307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:00:42.536578+0000","modified":"2024-04-25T09:00:44.531104+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7596},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7596":{"gid":7596,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/1341028696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1341028696},{"type":"v1","addr":"172.21.15.43:6835","nonce":1341028696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T09:01:21.418 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 54 2024-04-25T09:01:21.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.430+0000 7f1e291a9640 1 -- 172.21.15.43:0/3491977730 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1e0406b4d0 msgr2=0x7f1e0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:21.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.430+0000 7f1e291a9640 1 --2- 172.21.15.43:0/3491977730 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1e0406b4d0 0x7f1e0406d990 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c034000 tx=0x7f1e1c02e000 comp rx=0 tx=0).stop 2024-04-25T09:01:21.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.430+0000 7f1e291a9640 1 -- 172.21.15.43:0/3491977730 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e2406f930 msgr2=0x7f1e240b1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:21.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.430+0000 7f1e291a9640 1 --2- 172.21.15.43:0/3491977730 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e2406f930 0x7f1e240b1480 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f1e1400b9a0 tx=0x7f1e1400be60 comp rx=0 tx=0).stop 2024-04-25T09:01:21.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.430+0000 7f1e291a9640 1 -- 172.21.15.43:0/3491977730 shutdown_connections 2024-04-25T09:01:21.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.430+0000 7f1e291a9640 1 --2- 172.21.15.43:0/3491977730 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1e2406f930 0x7f1e240b1480 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.430+0000 7f1e291a9640 1 --2- 172.21.15.43:0/3491977730 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1e2406f0e0 0x7f1e24148090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.430+0000 7f1e291a9640 1 --2- 172.21.15.43:0/3491977730 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1e0406b4d0 0x7f1e0406d990 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.430+0000 7f1e291a9640 1 --2- 172.21.15.43:0/3491977730 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1e2406e460 0x7f1e24145770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.430+0000 7f1e291a9640 1 -- 172.21.15.43:0/3491977730 >> 172.21.15.43:0/3491977730 conn(0x7f1e240b2490 msgr2=0x7f1e2415c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:21.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.430+0000 7f1e291a9640 1 -- 172.21.15.43:0/3491977730 shutdown_connections 2024-04-25T09:01:21.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.430+0000 7f1e291a9640 1 -- 172.21.15.43:0/3491977730 wait complete. 2024-04-25T09:01:21.431 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:01:21.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.543+0000 7f8c92d32640 1 Processor -- start 2024-04-25T09:01:21.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.543+0000 7f8c92d32640 1 -- start start 2024-04-25T09:01:21.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.544+0000 7f8c92d32640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f8c8c15dce0 0x7f8c8c15ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:21.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.544+0000 7f8c92d32640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c8c0ce550 0x7f8c8c0d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:21.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.544+0000 7f8c92d32640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c8c0cee80 con 0x7f8c8c0ce990 2024-04-25T09:01:21.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.544+0000 7f8c92d32640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c8c0d2920 con 0x7f8c8c15dce0 2024-04-25T09:01:21.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.544+0000 7f8c92d32640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8c0cf160 con 0x7f8c8c0ce550 2024-04-25T09:01:21.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.544+0000 7f8c90ace640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f8c8c15dce0 0x7f8c8c15ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:21.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.544+0000 7f8c90ace640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f8c8c15dce0 0x7f8c8c15ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:49258/0 (socket says 172.21.15.43:49258) 2024-04-25T09:01:21.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.544+0000 7f8c90ace640 1 -- 172.21.15.43:0/3045011412 learned_addr learned my addr 172.21.15.43:0/3045011412 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:21.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.544+0000 7f8c912cf640 1 --1- 172.21.15.43:0/3045011412 >> v1:172.21.15.193:6789/0 conn(0x7f8c8c0ce990 0x7f8c8c160980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:57322/0 (socket says 172.21.15.43:57322) 2024-04-25T09:01:21.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.544+0000 7f8c8bfff640 1 --2- 172.21.15.43:0/3045011412 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c8c0ce550 0x7f8c8c0d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:21.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.544+0000 7f8c90ace640 1 -- 172.21.15.43:0/3045011412 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c8c0ce550 msgr2=0x7f8c8c0d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:21.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.544+0000 7f8c90ace640 1 --2- 172.21.15.43:0/3045011412 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c8c0ce550 0x7f8c8c0d03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.544+0000 7f8c90ace640 1 -- 172.21.15.43:0/3045011412 >> v1:172.21.15.193:6789/0 conn(0x7f8c8c0ce990 legacy=0x7f8c8c160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:21.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.544+0000 7f8c90ace640 1 -- 172.21.15.43:0/3045011412 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c8c0cf440 con 0x7f8c8c15dce0 2024-04-25T09:01:21.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.545+0000 7f8c90ace640 1 --2- 172.21.15.43:0/3045011412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c8c15dce0 0x7f8c8c15ef50 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8c80009870 tx=0x7f8c8002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9910509817f77c7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:21.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.545+0000 7f8c8b7fe640 1 -- 172.21.15.43:0/3045011412 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c80037070 con 0x7f8c8c15dce0 2024-04-25T09:01:21.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.545+0000 7f8c8b7fe640 1 -- 172.21.15.43:0/3045011412 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c80032440 con 0x7f8c8c15dce0 2024-04-25T09:01:21.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.545+0000 7f8c8b7fe640 1 -- 172.21.15.43:0/3045011412 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c80032810 con 0x7f8c8c15dce0 2024-04-25T09:01:21.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.546+0000 7f8c92d32640 1 -- 172.21.15.43:0/3045011412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c8c15dce0 msgr2=0x7f8c8c15ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:21.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.546+0000 7f8c92d32640 1 --2- 172.21.15.43:0/3045011412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c8c15dce0 0x7f8c8c15ef50 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8c80009870 tx=0x7f8c8002a1c0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.546+0000 7f8c92d32640 1 -- 172.21.15.43:0/3045011412 shutdown_connections 2024-04-25T09:01:21.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.546+0000 7f8c92d32640 1 --2- 172.21.15.43:0/3045011412 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c8c0ce550 0x7f8c8c0d03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.546+0000 7f8c92d32640 1 --2- 172.21.15.43:0/3045011412 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c8c15dce0 0x7f8c8c15ef50 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.546+0000 7f8c92d32640 1 -- 172.21.15.43:0/3045011412 >> 172.21.15.43:0/3045011412 conn(0x7f8c8c09cd00 msgr2=0x7f8c8c0d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:21.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.546+0000 7f8c92d32640 1 -- 172.21.15.43:0/3045011412 shutdown_connections 2024-04-25T09:01:21.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.546+0000 7f8c92d32640 1 -- 172.21.15.43:0/3045011412 wait complete. 2024-04-25T09:01:21.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.547+0000 7f8c92d32640 1 Processor -- start 2024-04-25T09:01:21.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.547+0000 7f8c92d32640 1 -- start start 2024-04-25T09:01:21.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.548+0000 7f8c92d32640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c8c0ce550 0x7f8c8c15a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:21.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.548+0000 7f8c92d32640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c8c0ce990 0x7f8c8c0caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:21.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.548+0000 7f8c92d32640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c8c15dce0 0x7f8c8c14fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:21.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.548+0000 7f8c92d32640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8c15e0c0 con 0x7f8c8c0ce990 2024-04-25T09:01:21.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.548+0000 7f8c92d32640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8c15f6b0 con 0x7f8c8c0ce550 2024-04-25T09:01:21.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.548+0000 7f8c92d32640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c8c0d0920 con 0x7f8c8c15dce0 2024-04-25T09:01:21.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.548+0000 7f8c8bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c8c0ce990 0x7f8c8c0caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:21.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.548+0000 7f8c8bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c8c0ce990 0x7f8c8c0caaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58086/0 (socket says 172.21.15.43:58086) 2024-04-25T09:01:21.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.548+0000 7f8c8bfff640 1 -- 172.21.15.43:0/1864574297 learned_addr learned my addr 172.21.15.43:0/1864574297 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:21.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.548+0000 7f8c90ace640 1 --2- 172.21.15.43:0/1864574297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c8c0ce550 0x7f8c8c15a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:21.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.548+0000 7f8c912cf640 1 --2- 172.21.15.43:0/1864574297 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c8c15dce0 0x7f8c8c14fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:21.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.548+0000 7f8c8bfff640 1 -- 172.21.15.43:0/1864574297 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c8c15dce0 msgr2=0x7f8c8c14fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:21.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.548+0000 7f8c8bfff640 1 --2- 172.21.15.43:0/1864574297 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c8c15dce0 0x7f8c8c14fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.548+0000 7f8c8bfff640 1 -- 172.21.15.43:0/1864574297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c8c0ce550 msgr2=0x7f8c8c15a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:21.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.548+0000 7f8c8bfff640 1 --2- 172.21.15.43:0/1864574297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c8c0ce550 0x7f8c8c15a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.548+0000 7f8c8bfff640 1 -- 172.21.15.43:0/1864574297 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c8c0cd280 con 0x7f8c8c0ce990 2024-04-25T09:01:21.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.549+0000 7f8c8bfff640 1 --2- 172.21.15.43:0/1864574297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c8c0ce990 0x7f8c8c0caaa0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f8c7800e9b0 tx=0x7f8c7800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:21.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.549+0000 7f8c897fa640 1 -- 172.21.15.43:0/1864574297 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c78017070 con 0x7f8c8c0ce990 2024-04-25T09:01:21.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.549+0000 7f8c897fa640 1 -- 172.21.15.43:0/1864574297 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c7800c9a0 con 0x7f8c8c0ce990 2024-04-25T09:01:21.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.549+0000 7f8c92d32640 1 -- 172.21.15.43:0/1864574297 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c8c0cd520 con 0x7f8c8c0ce990 2024-04-25T09:01:21.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.549+0000 7f8c897fa640 1 -- 172.21.15.43:0/1864574297 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7800cd90 con 0x7f8c8c0ce990 2024-04-25T09:01:21.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.549+0000 7f8c92d32640 1 -- 172.21.15.43:0/1864574297 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c8c0cdae0 con 0x7f8c8c0ce990 2024-04-25T09:01:21.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.550+0000 7f8c92d32640 1 -- 172.21.15.43:0/1864574297 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c8c0680f0 con 0x7f8c8c0ce990 2024-04-25T09:01:21.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.550+0000 7f8c897fa640 1 -- 172.21.15.43:0/1864574297 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8c7801a070 con 0x7f8c8c0ce990 2024-04-25T09:01:21.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.551+0000 7f8c897fa640 1 --2- 172.21.15.43:0/1864574297 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8c7006b4d0 0x7f8c7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:21.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.551+0000 7f8c897fa640 1 -- 172.21.15.43:0/1864574297 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f8c7808a690 con 0x7f8c8c0ce990 2024-04-25T09:01:21.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.551+0000 7f8c90ace640 1 --2- 172.21.15.43:0/1864574297 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8c7006b4d0 0x7f8c7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:21.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.552+0000 7f8c90ace640 1 --2- 172.21.15.43:0/1864574297 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8c7006b4d0 0x7f8c7006d990 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8c80034000 tx=0x7f8c8002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:21.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.555+0000 7f8c897fa640 1 -- 172.21.15.43:0/1864574297 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8c78059460 con 0x7f8c8c0ce990 2024-04-25T09:01:21.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.739+0000 7f8c92d32640 1 -- 172.21.15.43:0/1864574297 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c8c09d1a0 con 0x7f8c8c0ce990 2024-04-25T09:01:21.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.740+0000 7f8c897fa640 1 -- 172.21.15.43:0/1864574297 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) v1 ==== 74+0+16674 (secure 0 0 0) 0x7f8c78058d60 con 0x7f8c8c0ce990 2024-04-25T09:01:21.729 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:01:21.729 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":91,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:00:54.209500+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:00:40.816230+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-04-25T09:00:41.508021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.26","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:01:21.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.742+0000 7f8c92d32640 1 -- 172.21.15.43:0/1864574297 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8c7006b4d0 msgr2=0x7f8c7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:21.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.742+0000 7f8c92d32640 1 --2- 172.21.15.43:0/1864574297 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8c7006b4d0 0x7f8c7006d990 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8c80034000 tx=0x7f8c8002e000 comp rx=0 tx=0).stop 2024-04-25T09:01:21.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.742+0000 7f8c92d32640 1 -- 172.21.15.43:0/1864574297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c8c0ce990 msgr2=0x7f8c8c0caaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:21.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.742+0000 7f8c92d32640 1 --2- 172.21.15.43:0/1864574297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c8c0ce990 0x7f8c8c0caaa0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f8c7800e9b0 tx=0x7f8c7800ee70 comp rx=0 tx=0).stop 2024-04-25T09:01:21.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.742+0000 7f8c92d32640 1 -- 172.21.15.43:0/1864574297 shutdown_connections 2024-04-25T09:01:21.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.742+0000 7f8c92d32640 1 --2- 172.21.15.43:0/1864574297 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c8c15dce0 0x7f8c8c14fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.742+0000 7f8c92d32640 1 --2- 172.21.15.43:0/1864574297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c8c0ce990 0x7f8c8c0caaa0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.742+0000 7f8c92d32640 1 --2- 172.21.15.43:0/1864574297 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8c7006b4d0 0x7f8c7006d990 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.742+0000 7f8c92d32640 1 --2- 172.21.15.43:0/1864574297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c8c0ce550 0x7f8c8c15a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.742+0000 7f8c92d32640 1 -- 172.21.15.43:0/1864574297 >> 172.21.15.43:0/1864574297 conn(0x7f8c8c09cd00 msgr2=0x7f8c8c06b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:21.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.742+0000 7f8c92d32640 1 -- 172.21.15.43:0/1864574297 shutdown_connections 2024-04-25T09:01:21.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.742+0000 7f8c92d32640 1 -- 172.21.15.43:0/1864574297 wait complete. 2024-04-25T09:01:21.742 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:01:21.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.855+0000 7f5e9f529640 1 Processor -- start 2024-04-25T09:01:21.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.856+0000 7f5e9f529640 1 -- start start 2024-04-25T09:01:21.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.856+0000 7f5e9f529640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5e9815dce0 0x7f5e9815ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:21.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.856+0000 7f5e9f529640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e980d2920 con 0x7f5e980d0af0 2024-04-25T09:01:21.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.856+0000 7f5e9f529640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e980cee80 con 0x7f5e980ce550 2024-04-25T09:01:21.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.856+0000 7f5e9f529640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e980cf160 con 0x7f5e9815dce0 2024-04-25T09:01:21.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.857+0000 7f5e9d2c5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5e9815dce0 0x7f5e9815ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:21.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.857+0000 7f5e9dac6640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f5e980d0af0 0x7f5e98160980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48932/0 (socket says 172.21.15.43:48932) 2024-04-25T09:01:21.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.857+0000 7f5e9dac6640 1 -- 172.21.15.43:0/4079050118 learned_addr learned my addr 172.21.15.43:0/4079050118 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:21.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.857+0000 7f5e9d2c5640 1 -- 172.21.15.43:0/4079050118 >> v1:172.21.15.193:6789/0 conn(0x7f5e980ce550 legacy=0x7f5e980d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:21.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.857+0000 7f5e9d2c5640 1 -- 172.21.15.43:0/4079050118 >> v1:172.21.15.43:6789/0 conn(0x7f5e980d0af0 legacy=0x7f5e98160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:21.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.857+0000 7f5e9d2c5640 1 -- 172.21.15.43:0/4079050118 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e980cf440 con 0x7f5e9815dce0 2024-04-25T09:01:21.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.857+0000 7f5e97fff640 1 -- 172.21.15.43:0/4079050118 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1167461488 0 0) 0x7f5e980d2920 con 0x7f5e980d0af0 2024-04-25T09:01:21.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.858+0000 7f5e9d2c5640 1 --2- 172.21.15.43:0/4079050118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e9815dce0 0x7f5e9815ef50 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f5e900097a0 tx=0x7f5e9002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c476afc386587be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:21.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.858+0000 7f5e97fff640 1 -- 172.21.15.43:0/4079050118 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e90037070 con 0x7f5e9815dce0 2024-04-25T09:01:21.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.858+0000 7f5e97fff640 1 -- 172.21.15.43:0/4079050118 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e90032440 con 0x7f5e9815dce0 2024-04-25T09:01:21.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.858+0000 7f5e97fff640 1 -- 172.21.15.43:0/4079050118 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e90032810 con 0x7f5e9815dce0 2024-04-25T09:01:21.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.858+0000 7f5e9f529640 1 -- 172.21.15.43:0/4079050118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e9815dce0 msgr2=0x7f5e9815ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:21.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.858+0000 7f5e9f529640 1 --2- 172.21.15.43:0/4079050118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e9815dce0 0x7f5e9815ef50 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f5e900097a0 tx=0x7f5e9002a1a0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.858+0000 7f5e9f529640 1 -- 172.21.15.43:0/4079050118 shutdown_connections 2024-04-25T09:01:21.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.858+0000 7f5e9f529640 1 --2- 172.21.15.43:0/4079050118 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e9815dce0 0x7f5e9815ef50 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.858+0000 7f5e9f529640 1 -- 172.21.15.43:0/4079050118 >> 172.21.15.43:0/4079050118 conn(0x7f5e9809cd00 msgr2=0x7f5e980d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:21.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.858+0000 7f5e9f529640 1 -- 172.21.15.43:0/4079050118 shutdown_connections 2024-04-25T09:01:21.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.859+0000 7f5e9f529640 1 -- 172.21.15.43:0/4079050118 wait complete. 2024-04-25T09:01:21.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.860+0000 7f5e9f529640 1 Processor -- start 2024-04-25T09:01:21.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.860+0000 7f5e9f529640 1 -- start start 2024-04-25T09:01:21.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.860+0000 7f5e9f529640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e980ce550 0x7f5e980a02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:21.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.860+0000 7f5e9f529640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5e980d0af0 0x7f5e981f7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:21.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.860+0000 7f5e9f529640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e9815dce0 0x7f5e9809e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:21.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.860+0000 7f5e9f529640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9815e0c0 con 0x7f5e9815dce0 2024-04-25T09:01:21.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.860+0000 7f5e9f529640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e9815f6b0 con 0x7f5e980ce550 2024-04-25T09:01:21.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.860+0000 7f5e9f529640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e980ce9c0 con 0x7f5e980d0af0 2024-04-25T09:01:21.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.861+0000 7f5e9dac6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e9815dce0 0x7f5e9809e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:21.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.861+0000 7f5e9dac6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e9815dce0 0x7f5e9809e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58112/0 (socket says 172.21.15.43:58112) 2024-04-25T09:01:21.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.861+0000 7f5e9dac6640 1 -- 172.21.15.43:0/750373231 learned_addr learned my addr 172.21.15.43:0/750373231 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:21.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.861+0000 7f5e9d2c5640 1 --2- 172.21.15.43:0/750373231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e980ce550 0x7f5e980a02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:21.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.861+0000 7f5e9cac4640 1 --2- 172.21.15.43:0/750373231 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5e980d0af0 0x7f5e981f7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:21.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.861+0000 7f5e9dac6640 1 -- 172.21.15.43:0/750373231 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5e980d0af0 msgr2=0x7f5e981f7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:21.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.861+0000 7f5e9dac6640 1 --2- 172.21.15.43:0/750373231 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5e980d0af0 0x7f5e981f7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.861+0000 7f5e9dac6640 1 -- 172.21.15.43:0/750373231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e980ce550 msgr2=0x7f5e980a02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:21.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.861+0000 7f5e9dac6640 1 --2- 172.21.15.43:0/750373231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e980ce550 0x7f5e980a02b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:21.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.861+0000 7f5e9dac6640 1 -- 172.21.15.43:0/750373231 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e9809f4f0 con 0x7f5e9815dce0 2024-04-25T09:01:21.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.861+0000 7f5e9d2c5640 1 --2- 172.21.15.43:0/750373231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e980ce550 0x7f5e980a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:21.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.861+0000 7f5e9dac6640 1 --2- 172.21.15.43:0/750373231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e9815dce0 0x7f5e9809e150 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f5e8800caf0 tx=0x7f5e8800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:21.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.861+0000 7f5e95ffb640 1 -- 172.21.15.43:0/750373231 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e8800aca0 con 0x7f5e9815dce0 2024-04-25T09:01:21.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.861+0000 7f5e95ffb640 1 -- 172.21.15.43:0/750373231 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e8800ae40 con 0x7f5e9815dce0 2024-04-25T09:01:21.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.861+0000 7f5e9f529640 1 -- 172.21.15.43:0/750373231 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e9809f7d0 con 0x7f5e9815dce0 2024-04-25T09:01:21.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.861+0000 7f5e95ffb640 1 -- 172.21.15.43:0/750373231 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e88012620 con 0x7f5e9815dce0 2024-04-25T09:01:21.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.862+0000 7f5e9f529640 1 -- 172.21.15.43:0/750373231 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e9809fd90 con 0x7f5e9815dce0 2024-04-25T09:01:21.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.863+0000 7f5e9f529640 1 -- 172.21.15.43:0/750373231 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e980680f0 con 0x7f5e9815dce0 2024-04-25T09:01:21.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.863+0000 7f5e95ffb640 1 -- 172.21.15.43:0/750373231 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e88012880 con 0x7f5e9815dce0 2024-04-25T09:01:21.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.863+0000 7f5e95ffb640 1 --2- 172.21.15.43:0/750373231 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5e8406b4d0 0x7f5e8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:21.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.863+0000 7f5e95ffb640 1 -- 172.21.15.43:0/750373231 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f5e8808a830 con 0x7f5e9815dce0 2024-04-25T09:01:21.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.864+0000 7f5e9d2c5640 1 --2- 172.21.15.43:0/750373231 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5e8406b4d0 0x7f5e8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:21.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.864+0000 7f5e9d2c5640 1 --2- 172.21.15.43:0/750373231 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5e8406b4d0 0x7f5e8406d990 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5e90034000 tx=0x7f5e9002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:21.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:21.868+0000 7f5e95ffb640 1 -- 172.21.15.43:0/750373231 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5e88059720 con 0x7f5e9815dce0 2024-04-25T09:01:22.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.087+0000 7f5e9f529640 1 -- 172.21.15.43:0/750373231 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5e98074ef0 con 0x7f5e9815dce0 2024-04-25T09:01:22.076 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-04-25T09:01:22.088+0000 7fe4fb198640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:01:22.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.089+0000 7f5e95ffb640 1 -- 172.21.15.43:0/750373231 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v55) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5e88059020 con 0x7f5e9815dce0 2024-04-25T09:01:22.078 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:01:22.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.092+0000 7f5e9f529640 1 -- 172.21.15.43:0/750373231 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5e8406b4d0 msgr2=0x7f5e8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:22.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.092+0000 7f5e9f529640 1 --2- 172.21.15.43:0/750373231 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5e8406b4d0 0x7f5e8406d990 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f5e90034000 tx=0x7f5e9002e000 comp rx=0 tx=0).stop 2024-04-25T09:01:22.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.092+0000 7f5e9f529640 1 -- 172.21.15.43:0/750373231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e9815dce0 msgr2=0x7f5e9809e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:22.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.092+0000 7f5e9f529640 1 --2- 172.21.15.43:0/750373231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e9815dce0 0x7f5e9809e150 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f5e8800caf0 tx=0x7f5e8800a060 comp rx=0 tx=0).stop 2024-04-25T09:01:22.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.092+0000 7f5e9f529640 1 -- 172.21.15.43:0/750373231 shutdown_connections 2024-04-25T09:01:22.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.092+0000 7f5e9f529640 1 --2- 172.21.15.43:0/750373231 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5e9815dce0 0x7f5e9809e150 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:22.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.092+0000 7f5e9f529640 1 --2- 172.21.15.43:0/750373231 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5e980d0af0 0x7f5e981f7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:22.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.092+0000 7f5e9f529640 1 --2- 172.21.15.43:0/750373231 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5e8406b4d0 0x7f5e8406d990 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:22.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.092+0000 7f5e9f529640 1 --2- 172.21.15.43:0/750373231 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5e980ce550 0x7f5e980a02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:22.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.092+0000 7f5e9f529640 1 -- 172.21.15.43:0/750373231 >> 172.21.15.43:0/750373231 conn(0x7f5e9809cd00 msgr2=0x7f5e9815e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:22.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.092+0000 7f5e9f529640 1 -- 172.21.15.43:0/750373231 shutdown_connections 2024-04-25T09:01:22.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.092+0000 7f5e9f529640 1 -- 172.21.15.43:0/750373231 wait complete. 2024-04-25T09:01:22.093 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:01:22.119 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2024-04-25T09:01:22.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.211+0000 7fc4a1ab3640 1 Processor -- start 2024-04-25T09:01:22.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.211+0000 7fc4a1ab3640 1 -- start start 2024-04-25T09:01:22.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.212+0000 7fc4a1ab3640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc49c161310 0x7fc49c0d4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:22.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.212+0000 7fc4a1ab3640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc49c0d0c70 0x7fc49c0c7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:22.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.212+0000 7fc4a1ab3640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc49c0d45c0 con 0x7fc49c0d0120 2024-04-25T09:01:22.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.212+0000 7fc4a1ab3640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc49c0d4870 con 0x7fc49c161310 2024-04-25T09:01:22.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.212+0000 7fc4a1ab3640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c0cf030 con 0x7fc49c0d0c70 2024-04-25T09:01:22.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.212+0000 7fc49b7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc49c161310 0x7fc49c0d4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:22.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.212+0000 7fc49b7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc49c161310 0x7fc49c0d4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58142/0 (socket says 172.21.15.43:58142) 2024-04-25T09:01:22.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.212+0000 7fc49b7fe640 1 -- 172.21.15.43:0/3886789943 learned_addr learned my addr 172.21.15.43:0/3886789943 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:22.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.212+0000 7fc49bfff640 1 --2- 172.21.15.43:0/3886789943 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc49c0d0c70 0x7fc49c0c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:22.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.212+0000 7fc49b7fe640 1 -- 172.21.15.43:0/3886789943 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc49c0d0c70 msgr2=0x7fc49c0c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:22.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.212+0000 7fc49b7fe640 1 --2- 172.21.15.43:0/3886789943 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc49c0d0c70 0x7fc49c0c7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:22.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.212+0000 7fc49b7fe640 1 -- 172.21.15.43:0/3886789943 >> v1:172.21.15.193:6789/0 conn(0x7fc49c0d0120 legacy=0x7fc49c0d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:22.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.212+0000 7fc49b7fe640 1 -- 172.21.15.43:0/3886789943 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc49c0cf310 con 0x7fc49c161310 2024-04-25T09:01:22.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.213+0000 7fc49b7fe640 1 --2- 172.21.15.43:0/3886789943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc49c161310 0x7fc49c0d4080 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fc48c009870 tx=0x7fc48c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af2324839c00cf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:22.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.213+0000 7fc49a7fc640 1 -- 172.21.15.43:0/3886789943 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48c037070 con 0x7fc49c161310 2024-04-25T09:01:22.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.213+0000 7fc49a7fc640 1 -- 172.21.15.43:0/3886789943 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc48c032440 con 0x7fc49c161310 2024-04-25T09:01:22.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.213+0000 7fc49a7fc640 1 -- 172.21.15.43:0/3886789943 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48c032810 con 0x7fc49c161310 2024-04-25T09:01:22.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.213+0000 7fc4a1ab3640 1 -- 172.21.15.43:0/3886789943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc49c161310 msgr2=0x7fc49c0d4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:22.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.213+0000 7fc4a1ab3640 1 --2- 172.21.15.43:0/3886789943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc49c161310 0x7fc49c0d4080 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fc48c009870 tx=0x7fc48c02a210 comp rx=0 tx=0).stop 2024-04-25T09:01:22.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.214+0000 7fc4a1ab3640 1 -- 172.21.15.43:0/3886789943 shutdown_connections 2024-04-25T09:01:22.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.214+0000 7fc4a1ab3640 1 --2- 172.21.15.43:0/3886789943 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc49c0d0c70 0x7fc49c0c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:22.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.214+0000 7fc4a1ab3640 1 --2- 172.21.15.43:0/3886789943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc49c161310 0x7fc49c0d4080 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:22.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.214+0000 7fc4a1ab3640 1 -- 172.21.15.43:0/3886789943 >> 172.21.15.43:0/3886789943 conn(0x7fc49c09cd20 msgr2=0x7fc49c0d3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:22.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.214+0000 7fc4a1ab3640 1 -- 172.21.15.43:0/3886789943 shutdown_connections 2024-04-25T09:01:22.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.214+0000 7fc4a1ab3640 1 -- 172.21.15.43:0/3886789943 wait complete. 2024-04-25T09:01:22.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.215+0000 7fc4a1ab3640 1 Processor -- start 2024-04-25T09:01:22.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.215+0000 7fc4a1ab3640 1 -- start start 2024-04-25T09:01:22.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.216+0000 7fc4a1ab3640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc49c0d0120 0x7fc49c159e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:22.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.216+0000 7fc4a1ab3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc49c0d0c70 0x7fc49c0b8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:22.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.216+0000 7fc4a1ab3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc49c161310 0x7fc49c1580c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:22.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.216+0000 7fc4a1ab3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c0c7da0 con 0x7fc49c161310 2024-04-25T09:01:22.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.216+0000 7fc4a1ab3640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c1616f0 con 0x7fc49c0d0c70 2024-04-25T09:01:22.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.216+0000 7fc4a1ab3640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc49c0c7f20 con 0x7fc49c0d0120 2024-04-25T09:01:22.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.216+0000 7fc49bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc49c161310 0x7fc49c1580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:22.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.216+0000 7fc49bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc49c161310 0x7fc49c1580c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58146/0 (socket says 172.21.15.43:58146) 2024-04-25T09:01:22.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.216+0000 7fc49bfff640 1 -- 172.21.15.43:0/1971634514 learned_addr learned my addr 172.21.15.43:0/1971634514 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:22.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.216+0000 7fc49affd640 1 --2- 172.21.15.43:0/1971634514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc49c0d0c70 0x7fc49c0b8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:22.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.216+0000 7fc49b7fe640 1 --2- 172.21.15.43:0/1971634514 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc49c0d0120 0x7fc49c159e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:22.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.216+0000 7fc49bfff640 1 -- 172.21.15.43:0/1971634514 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc49c0d0120 msgr2=0x7fc49c159e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:22.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.216+0000 7fc49bfff640 1 --2- 172.21.15.43:0/1971634514 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc49c0d0120 0x7fc49c159e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:22.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.216+0000 7fc49bfff640 1 -- 172.21.15.43:0/1971634514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc49c0d0c70 msgr2=0x7fc49c0b8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:22.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.216+0000 7fc49bfff640 1 --2- 172.21.15.43:0/1971634514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc49c0d0c70 0x7fc49c0b8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:22.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.216+0000 7fc49bfff640 1 -- 172.21.15.43:0/1971634514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc49c15a340 con 0x7fc49c161310 2024-04-25T09:01:22.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.216+0000 7fc49b7fe640 1 --2- 172.21.15.43:0/1971634514 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc49c0d0120 0x7fc49c159e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:22.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.216+0000 7fc49affd640 1 --2- 172.21.15.43:0/1971634514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc49c0d0c70 0x7fc49c0b8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:22.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.216+0000 7fc49bfff640 1 --2- 172.21.15.43:0/1971634514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc49c161310 0x7fc49c1580c0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fc49400cd60 tx=0x7fc49400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:22.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.216+0000 7fc483fff640 1 -- 172.21.15.43:0/1971634514 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc494003b90 con 0x7fc49c161310 2024-04-25T09:01:22.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.216+0000 7fc483fff640 1 -- 172.21.15.43:0/1971634514 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc494003d50 con 0x7fc49c161310 2024-04-25T09:01:22.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.216+0000 7fc483fff640 1 -- 172.21.15.43:0/1971634514 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc494004260 con 0x7fc49c161310 2024-04-25T09:01:22.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.217+0000 7fc4a1ab3640 1 -- 172.21.15.43:0/1971634514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc49c15b000 con 0x7fc49c161310 2024-04-25T09:01:22.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.217+0000 7fc4a1ab3640 1 -- 172.21.15.43:0/1971634514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc49c159600 con 0x7fc49c161310 2024-04-25T09:01:22.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.218+0000 7fc4a1ab3640 1 -- 172.21.15.43:0/1971634514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc49c074f60 con 0x7fc49c161310 2024-04-25T09:01:22.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.218+0000 7fc483fff640 1 -- 172.21.15.43:0/1971634514 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc494004400 con 0x7fc49c161310 2024-04-25T09:01:22.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.218+0000 7fc483fff640 1 --2- 172.21.15.43:0/1971634514 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc48806b4d0 0x7fc48806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:22.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.219+0000 7fc483fff640 1 -- 172.21.15.43:0/1971634514 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fc49408ad30 con 0x7fc49c161310 2024-04-25T09:01:22.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.219+0000 7fc49b7fe640 1 --2- 172.21.15.43:0/1971634514 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc48806b4d0 0x7fc48806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:22.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.219+0000 7fc49b7fe640 1 --2- 172.21.15.43:0/1971634514 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc48806b4d0 0x7fc48806d990 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc48c034000 tx=0x7fc48c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:22.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.223+0000 7fc483fff640 1 -- 172.21.15.43:0/1971634514 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc494059b40 con 0x7fc49c161310 2024-04-25T09:01:22.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:22.447+0000 7fc4a1ab3640 1 -- 172.21.15.43:0/1971634514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc49c0c9bc0 con 0x7fc49c161310 2024-04-25T09:01:23.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.090+0000 7fc483fff640 1 -- 172.21.15.43:0/1971634514 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v56) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc494059440 con 0x7fc49c161310 2024-04-25T09:01:23.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.093+0000 7fc4a1ab3640 1 -- 172.21.15.43:0/1971634514 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc48806b4d0 msgr2=0x7fc48806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:23.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.093+0000 7fc4a1ab3640 1 --2- 172.21.15.43:0/1971634514 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc48806b4d0 0x7fc48806d990 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fc48c034000 tx=0x7fc48c02e000 comp rx=0 tx=0).stop 2024-04-25T09:01:23.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.093+0000 7fc4a1ab3640 1 -- 172.21.15.43:0/1971634514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc49c161310 msgr2=0x7fc49c1580c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:23.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.093+0000 7fc4a1ab3640 1 --2- 172.21.15.43:0/1971634514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc49c161310 0x7fc49c1580c0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7fc49400cd60 tx=0x7fc49400a660 comp rx=0 tx=0).stop 2024-04-25T09:01:23.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.094+0000 7fc4a1ab3640 1 -- 172.21.15.43:0/1971634514 shutdown_connections 2024-04-25T09:01:23.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.094+0000 7fc4a1ab3640 1 --2- 172.21.15.43:0/1971634514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc49c161310 0x7fc49c1580c0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:23.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.094+0000 7fc4a1ab3640 1 --2- 172.21.15.43:0/1971634514 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc48806b4d0 0x7fc48806d990 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:23.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.094+0000 7fc4a1ab3640 1 --2- 172.21.15.43:0/1971634514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc49c0d0c70 0x7fc49c0b8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:23.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.094+0000 7fc4a1ab3640 1 --2- 172.21.15.43:0/1971634514 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc49c0d0120 0x7fc49c159e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:23.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.094+0000 7fc4a1ab3640 1 -- 172.21.15.43:0/1971634514 >> 172.21.15.43:0/1971634514 conn(0x7fc49c09cd20 msgr2=0x7fc49c160590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:23.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.094+0000 7fc4a1ab3640 1 -- 172.21.15.43:0/1971634514 shutdown_connections 2024-04-25T09:01:23.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.094+0000 7fc4a1ab3640 1 -- 172.21.15.43:0/1971634514 wait complete. 2024-04-25T09:01:23.097 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:01:23.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.263+0000 7ff08345d640 1 Processor -- start 2024-04-25T09:01:23.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.264+0000 7ff08345d640 1 -- start start 2024-04-25T09:01:23.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.264+0000 7ff08345d640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff07c0aea00 0x7ff07c0b3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:23.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.264+0000 7ff08345d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff07c0b36a0 0x7ff07c06cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:23.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.264+0000 7ff08345d640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff07c071690 con 0x7ff07c06d140 2024-04-25T09:01:23.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.264+0000 7ff08345d640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff07c0713f0 con 0x7ff07c0aea00 2024-04-25T09:01:23.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.264+0000 7ff08345d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff07c0b3cb0 con 0x7ff07c0b36a0 2024-04-25T09:01:23.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.265+0000 7ff0809f8640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff07c0b36a0 0x7ff07c06cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:23.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.265+0000 7ff0811f9640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff07c0aea00 0x7ff07c0b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:23.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.265+0000 7ff0811f9640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff07c0aea00 0x7ff07c0b3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:49300/0 (socket says 172.21.15.43:49300) 2024-04-25T09:01:23.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.265+0000 7ff0811f9640 1 -- 172.21.15.43:0/3101686639 learned_addr learned my addr 172.21.15.43:0/3101686639 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:23.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.265+0000 7ff073fff640 1 -- 172.21.15.43:0/3101686639 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 492212309 0 0) 0x7ff07c071690 con 0x7ff07c06d140 2024-04-25T09:01:23.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.281+0000 7ff073fff640 1 -- 172.21.15.43:0/3101686639 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff068003500 con 0x7ff07c06d140 2024-04-25T09:01:23.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.282+0000 7ff0809f8640 1 -- 172.21.15.43:0/3101686639 >> v2:172.21.15.193:3300/0 conn(0x7ff07c0aea00 msgr2=0x7ff07c0b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:23.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.282+0000 7ff0809f8640 1 --2- 172.21.15.43:0/3101686639 >> v2:172.21.15.193:3300/0 conn(0x7ff07c0aea00 0x7ff07c0b3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:23.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.282+0000 7ff0809f8640 1 -- 172.21.15.43:0/3101686639 >> v1:172.21.15.193:6789/0 conn(0x7ff07c06d140 legacy=0x7ff07c08b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:23.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.282+0000 7ff0809f8640 1 -- 172.21.15.43:0/3101686639 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff07c0b3f90 con 0x7ff07c0b36a0 2024-04-25T09:01:23.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.282+0000 7ff0811f9640 1 --2- 172.21.15.43:0/3101686639 >> v2:172.21.15.193:3300/0 conn(0x7ff07c0aea00 0x7ff07c0b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:01:23.275 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.282+0000 7ff0809f8640 1 --2- 172.21.15.43:0/3101686639 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff07c0b36a0 0x7ff07c06cc00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff06c014d10 tx=0x7ff06c032c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53f032542d30e4fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:23.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.283+0000 7ff073fff640 1 -- 172.21.15.43:0/3101686639 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06c039070 con 0x7ff07c0b36a0 2024-04-25T09:01:23.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.283+0000 7ff073fff640 1 -- 172.21.15.43:0/3101686639 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff06c0339e0 con 0x7ff07c0b36a0 2024-04-25T09:01:23.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.283+0000 7ff073fff640 1 -- 172.21.15.43:0/3101686639 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06c033db0 con 0x7ff07c0b36a0 2024-04-25T09:01:23.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.283+0000 7ff08345d640 1 -- 172.21.15.43:0/3101686639 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff07c0b36a0 msgr2=0x7ff07c06cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:23.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.283+0000 7ff08345d640 1 --2- 172.21.15.43:0/3101686639 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff07c0b36a0 0x7ff07c06cc00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff06c014d10 tx=0x7ff06c032c90 comp rx=0 tx=0).stop 2024-04-25T09:01:23.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.283+0000 7ff08345d640 1 -- 172.21.15.43:0/3101686639 shutdown_connections 2024-04-25T09:01:23.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.283+0000 7ff08345d640 1 --2- 172.21.15.43:0/3101686639 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff07c0b36a0 0x7ff07c06cc00 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:23.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.283+0000 7ff08345d640 1 --2- 172.21.15.43:0/3101686639 >> v2:172.21.15.193:3300/0 conn(0x7ff07c0aea00 0x7ff07c0b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:23.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.283+0000 7ff08345d640 1 -- 172.21.15.43:0/3101686639 >> 172.21.15.43:0/3101686639 conn(0x7ff07c06ab30 msgr2=0x7ff07c06af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:23.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.284+0000 7ff08345d640 1 -- 172.21.15.43:0/3101686639 shutdown_connections 2024-04-25T09:01:23.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.284+0000 7ff08345d640 1 -- 172.21.15.43:0/3101686639 wait complete. 2024-04-25T09:01:23.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.285+0000 7ff08345d640 1 Processor -- start 2024-04-25T09:01:23.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.285+0000 7ff08345d640 1 -- start start 2024-04-25T09:01:23.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.285+0000 7ff08345d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff07c06d140 0x7ff07c1f56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:23.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.285+0000 7ff08345d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff07c0aea00 0x7ff07c0adc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:23.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.285+0000 7ff08345d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff07c0b36a0 0x7ff07c0ab090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:23.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.285+0000 7ff08345d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07c0b4830 con 0x7ff07c0b36a0 2024-04-25T09:01:23.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.285+0000 7ff08345d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07c0b4550 con 0x7ff07c06d140 2024-04-25T09:01:23.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.285+0000 7ff08345d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff07c16e5f0 con 0x7ff07c0aea00 2024-04-25T09:01:23.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.286+0000 7ff0819fa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff07c0b36a0 0x7ff07c0ab090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:23.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.286+0000 7ff0819fa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff07c0b36a0 0x7ff07c0ab090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58156/0 (socket says 172.21.15.43:58156) 2024-04-25T09:01:23.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.286+0000 7ff0811f9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff07c06d140 0x7ff07c1f56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:23.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.286+0000 7ff0819fa640 1 -- 172.21.15.43:0/1355384752 learned_addr learned my addr 172.21.15.43:0/1355384752 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:23.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.286+0000 7ff0809f8640 1 --2- 172.21.15.43:0/1355384752 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff07c0aea00 0x7ff07c0adc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:23.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.286+0000 7ff0819fa640 1 -- 172.21.15.43:0/1355384752 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff07c0aea00 msgr2=0x7ff07c0adc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:23.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.286+0000 7ff0819fa640 1 --2- 172.21.15.43:0/1355384752 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff07c0aea00 0x7ff07c0adc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:23.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.286+0000 7ff0819fa640 1 -- 172.21.15.43:0/1355384752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff07c06d140 msgr2=0x7ff07c1f56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:23.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.286+0000 7ff0819fa640 1 --2- 172.21.15.43:0/1355384752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff07c06d140 0x7ff07c1f56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:23.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.286+0000 7ff0819fa640 1 -- 172.21.15.43:0/1355384752 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff07c0aaa00 con 0x7ff07c0b36a0 2024-04-25T09:01:23.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.286+0000 7ff0811f9640 1 --2- 172.21.15.43:0/1355384752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff07c06d140 0x7ff07c1f56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:01:23.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.286+0000 7ff0819fa640 1 --2- 172.21.15.43:0/1355384752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff07c0b36a0 0x7ff07c0ab090 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7ff07400c6f0 tx=0x7ff07400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:23.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.286+0000 7ff071ffb640 1 -- 172.21.15.43:0/1355384752 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff074003970 con 0x7ff07c0b36a0 2024-04-25T09:01:23.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.286+0000 7ff071ffb640 1 -- 172.21.15.43:0/1355384752 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff074003b30 con 0x7ff07c0b36a0 2024-04-25T09:01:23.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.286+0000 7ff071ffb640 1 -- 172.21.15.43:0/1355384752 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07400a850 con 0x7ff07c0b36a0 2024-04-25T09:01:23.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.286+0000 7ff08345d640 1 -- 172.21.15.43:0/1355384752 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff07c0a9960 con 0x7ff07c0b36a0 2024-04-25T09:01:23.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.287+0000 7ff08345d640 1 -- 172.21.15.43:0/1355384752 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff07c0a9f20 con 0x7ff07c0b36a0 2024-04-25T09:01:23.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.288+0000 7ff08345d640 1 -- 172.21.15.43:0/1355384752 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff07c068a40 con 0x7ff07c0b36a0 2024-04-25T09:01:23.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.288+0000 7ff071ffb640 1 -- 172.21.15.43:0/1355384752 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff07400a9f0 con 0x7ff07c0b36a0 2024-04-25T09:01:23.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.289+0000 7ff071ffb640 1 --2- 172.21.15.43:0/1355384752 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff06006b4d0 0x7ff06006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:23.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.289+0000 7ff071ffb640 1 -- 172.21.15.43:0/1355384752 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5824+0+0 (secure 0 0 0) 0x7ff07408a0a0 con 0x7ff07c0b36a0 2024-04-25T09:01:23.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.289+0000 7ff0811f9640 1 --2- 172.21.15.43:0/1355384752 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff06006b4d0 0x7ff06006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:23.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.290+0000 7ff0811f9640 1 --2- 172.21.15.43:0/1355384752 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff06006b4d0 0x7ff06006d990 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff07800dfd0 tx=0x7ff07800f040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:23.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.293+0000 7ff071ffb640 1 -- 172.21.15.43:0/1355384752 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff0740587e0 con 0x7ff07c0b36a0 2024-04-25T09:01:23.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:23.474+0000 7ff08345d640 1 -- 172.21.15.43:0/1355384752 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff07c08bd90 con 0x7ff07c0b36a0 2024-04-25T09:01:24.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.301+0000 7ff071ffb640 1 -- 172.21.15.43:0/1355384752 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v93) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff07405c880 con 0x7ff07c0b36a0 2024-04-25T09:01:24.290 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:01:24.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.305+0000 7ff08345d640 1 -- 172.21.15.43:0/1355384752 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff06006b4d0 msgr2=0x7ff06006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:24.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.305+0000 7ff08345d640 1 --2- 172.21.15.43:0/1355384752 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff06006b4d0 0x7ff06006d990 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7ff07800dfd0 tx=0x7ff07800f040 comp rx=0 tx=0).stop 2024-04-25T09:01:24.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.305+0000 7ff08345d640 1 -- 172.21.15.43:0/1355384752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff07c0b36a0 msgr2=0x7ff07c0ab090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:24.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.305+0000 7ff08345d640 1 --2- 172.21.15.43:0/1355384752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff07c0b36a0 0x7ff07c0ab090 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7ff07400c6f0 tx=0x7ff07400cbb0 comp rx=0 tx=0).stop 2024-04-25T09:01:24.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.305+0000 7ff08345d640 1 -- 172.21.15.43:0/1355384752 shutdown_connections 2024-04-25T09:01:24.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.305+0000 7ff08345d640 1 --2- 172.21.15.43:0/1355384752 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff07c0b36a0 0x7ff07c0ab090 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:24.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.305+0000 7ff08345d640 1 --2- 172.21.15.43:0/1355384752 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff07c0aea00 0x7ff07c0adc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:24.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.305+0000 7ff08345d640 1 --2- 172.21.15.43:0/1355384752 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff06006b4d0 0x7ff06006d990 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:24.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.305+0000 7ff08345d640 1 --2- 172.21.15.43:0/1355384752 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff07c06d140 0x7ff07c1f56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:24.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.305+0000 7ff08345d640 1 -- 172.21.15.43:0/1355384752 >> 172.21.15.43:0/1355384752 conn(0x7ff07c06ab30 msgr2=0x7ff07c06af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:24.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.305+0000 7ff08345d640 1 -- 172.21.15.43:0/1355384752 shutdown_connections 2024-04-25T09:01:24.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.305+0000 7ff08345d640 1 -- 172.21.15.43:0/1355384752 wait complete. 2024-04-25T09:01:24.309 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:01:24.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.431+0000 7f8c8941f640 1 Processor -- start 2024-04-25T09:01:24.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.431+0000 7f8c8941f640 1 -- start start 2024-04-25T09:01:24.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.431+0000 7f8c8941f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8c8415de80 0x7f8c840b4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:24.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.432+0000 7f8c8941f640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f8c840b50d0 0x7f8c840d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:24.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.432+0000 7f8c8941f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c8415e260 con 0x7f8c840d3ca0 2024-04-25T09:01:24.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.432+0000 7f8c8941f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c84161970 con 0x7f8c8415de80 2024-04-25T09:01:24.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.432+0000 7f8c8941f640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c84161c50 con 0x7f8c840b50d0 2024-04-25T09:01:24.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.432+0000 7f8c82ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8c8415de80 0x7f8c840b4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:24.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.432+0000 7f8c837fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8c840d3ca0 0x7f8c840b3950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48960/0 (socket says 172.21.15.43:48960) 2024-04-25T09:01:24.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.432+0000 7f8c837fe640 1 -- 172.21.15.43:0/3759313040 learned_addr learned my addr 172.21.15.43:0/3759313040 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:24.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.432+0000 7f8c7a7fc640 1 --2- 172.21.15.43:0/3759313040 >> v2:172.21.15.193:3300/0 conn(0x7f8c840b50d0 0x7f8c840d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:24.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.432+0000 7f8c82ffd640 1 -- 172.21.15.43:0/3759313040 >> v2:172.21.15.193:3300/0 conn(0x7f8c840b50d0 msgr2=0x7f8c840d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:24.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.432+0000 7f8c82ffd640 1 --2- 172.21.15.43:0/3759313040 >> v2:172.21.15.193:3300/0 conn(0x7f8c840b50d0 0x7f8c840d3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:24.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.432+0000 7f8c82ffd640 1 -- 172.21.15.43:0/3759313040 >> v1:172.21.15.43:6789/0 conn(0x7f8c840d3ca0 legacy=0x7f8c840b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:24.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.432+0000 7f8c82ffd640 1 -- 172.21.15.43:0/3759313040 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c84161f30 con 0x7f8c8415de80 2024-04-25T09:01:24.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.432+0000 7f8c827fc640 1 -- 172.21.15.43:0/3759313040 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 240218772 0 0) 0x7f8c8415e260 con 0x7f8c840d3ca0 2024-04-25T09:01:24.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.433+0000 7f8c82ffd640 1 --2- 172.21.15.43:0/3759313040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c8415de80 0x7f8c840b4b90 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f8c840ce460 tx=0x7f8c7402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7516967667bdbc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:24.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.433+0000 7f8c827fc640 1 -- 172.21.15.43:0/3759313040 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c74037070 con 0x7f8c8415de80 2024-04-25T09:01:24.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.433+0000 7f8c827fc640 1 -- 172.21.15.43:0/3759313040 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c74032440 con 0x7f8c8415de80 2024-04-25T09:01:24.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.433+0000 7f8c827fc640 1 -- 172.21.15.43:0/3759313040 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c7402ae60 con 0x7f8c8415de80 2024-04-25T09:01:24.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.433+0000 7f8c8941f640 1 -- 172.21.15.43:0/3759313040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c8415de80 msgr2=0x7f8c840b4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:24.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.433+0000 7f8c8941f640 1 --2- 172.21.15.43:0/3759313040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c8415de80 0x7f8c840b4b90 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f8c840ce460 tx=0x7f8c7402a110 comp rx=0 tx=0).stop 2024-04-25T09:01:24.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.434+0000 7f8c8941f640 1 -- 172.21.15.43:0/3759313040 shutdown_connections 2024-04-25T09:01:24.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.434+0000 7f8c8941f640 1 --2- 172.21.15.43:0/3759313040 >> v2:172.21.15.193:3300/0 conn(0x7f8c840b50d0 0x7f8c840d3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:24.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.434+0000 7f8c8941f640 1 --2- 172.21.15.43:0/3759313040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c8415de80 0x7f8c840b4b90 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:24.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.434+0000 7f8c8941f640 1 -- 172.21.15.43:0/3759313040 >> 172.21.15.43:0/3759313040 conn(0x7f8c8408d460 msgr2=0x7f8c8409cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:24.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.434+0000 7f8c8941f640 1 -- 172.21.15.43:0/3759313040 shutdown_connections 2024-04-25T09:01:24.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.434+0000 7f8c8941f640 1 -- 172.21.15.43:0/3759313040 wait complete. 2024-04-25T09:01:24.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.435+0000 7f8c8941f640 1 Processor -- start 2024-04-25T09:01:24.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.435+0000 7f8c8941f640 1 -- start start 2024-04-25T09:01:24.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.436+0000 7f8c8941f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c840b50d0 0x7f8c84160000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:24.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.436+0000 7f8c8941f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c840d3ca0 0x7f8c84152010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:24.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.436+0000 7f8c8941f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c8415de80 0x7f8c840b1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:24.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.436+0000 7f8c8941f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c840d40d0 con 0x7f8c840b50d0 2024-04-25T09:01:24.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.436+0000 7f8c8941f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c841624f0 con 0x7f8c8415de80 2024-04-25T09:01:24.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.436+0000 7f8c8941f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c840b3010 con 0x7f8c840d3ca0 2024-04-25T09:01:24.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.436+0000 7f8c82ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c840b50d0 0x7f8c84160000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:24.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.436+0000 7f8c82ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c840b50d0 0x7f8c84160000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58164/0 (socket says 172.21.15.43:58164) 2024-04-25T09:01:24.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.436+0000 7f8c82ffd640 1 -- 172.21.15.43:0/1570894338 learned_addr learned my addr 172.21.15.43:0/1570894338 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:24.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.436+0000 7f8c82ffd640 1 -- 172.21.15.43:0/1570894338 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c840d3ca0 msgr2=0x7f8c84152010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:24.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.436+0000 7f8c837fe640 1 --2- 172.21.15.43:0/1570894338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c8415de80 0x7f8c840b1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:24.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.436+0000 7f8c82ffd640 1 --2- 172.21.15.43:0/1570894338 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c840d3ca0 0x7f8c84152010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:24.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.436+0000 7f8c82ffd640 1 -- 172.21.15.43:0/1570894338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c8415de80 msgr2=0x7f8c840b1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:24.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.436+0000 7f8c82ffd640 1 --2- 172.21.15.43:0/1570894338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c8415de80 0x7f8c840b1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:24.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.436+0000 7f8c82ffd640 1 -- 172.21.15.43:0/1570894338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c840b84d0 con 0x7f8c840b50d0 2024-04-25T09:01:24.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.436+0000 7f8c837fe640 1 --2- 172.21.15.43:0/1570894338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c8415de80 0x7f8c840b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:24.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.436+0000 7f8c82ffd640 1 --2- 172.21.15.43:0/1570894338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c840b50d0 0x7f8c84160000 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f8c74000c00 tx=0x7f8c740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:24.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.436+0000 7f8c7bfff640 1 -- 172.21.15.43:0/1570894338 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c74037040 con 0x7f8c840b50d0 2024-04-25T09:01:24.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.436+0000 7f8c7bfff640 1 -- 172.21.15.43:0/1570894338 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c74042070 con 0x7f8c840b50d0 2024-04-25T09:01:24.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.436+0000 7f8c8941f640 1 -- 172.21.15.43:0/1570894338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c840b87b0 con 0x7f8c840b50d0 2024-04-25T09:01:24.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.437+0000 7f8c7bfff640 1 -- 172.21.15.43:0/1570894338 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c74030bb0 con 0x7f8c840b50d0 2024-04-25T09:01:24.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.437+0000 7f8c8941f640 1 -- 172.21.15.43:0/1570894338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c840b8d70 con 0x7f8c840b50d0 2024-04-25T09:01:24.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.438+0000 7f8c8941f640 1 -- 172.21.15.43:0/1570894338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c840cb980 con 0x7f8c840b50d0 2024-04-25T09:01:24.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.438+0000 7f8c7bfff640 1 -- 172.21.15.43:0/1570894338 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8c74047460 con 0x7f8c840b50d0 2024-04-25T09:01:24.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.438+0000 7f8c7bfff640 1 --2- 172.21.15.43:0/1570894338 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8c6406b4d0 0x7f8c6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:24.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.439+0000 7f8c7bfff640 1 -- 172.21.15.43:0/1570894338 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5386+0+0 (secure 0 0 0) 0x7f8c740aaca0 con 0x7f8c840b50d0 2024-04-25T09:01:24.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.439+0000 7f8c7a7fc640 1 --2- 172.21.15.43:0/1570894338 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8c6406b4d0 0x7f8c6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:24.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.439+0000 7f8c7a7fc640 1 --2- 172.21.15.43:0/1570894338 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8c6406b4d0 0x7f8c6406d990 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8c840b78b0 tx=0x7f8c6c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:24.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.443+0000 7f8c7bfff640 1 -- 172.21.15.43:0/1570894338 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8c74032e20 con 0x7f8c840b50d0 2024-04-25T09:01:24.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:24.626+0000 7f8c8941f640 1 -- 172.21.15.43:0/1570894338 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8c840b3730 con 0x7f8c840b50d0 2024-04-25T09:01:25.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.302+0000 7f8c7bfff640 1 -- 172.21.15.43:0/1570894338 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v94) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8c74045360 con 0x7f8c840b50d0 2024-04-25T09:01:25.291 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' removed 2024-04-25T09:01:25.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.306+0000 7f8c8941f640 1 -- 172.21.15.43:0/1570894338 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8c6406b4d0 msgr2=0x7f8c6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:25.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.306+0000 7f8c8941f640 1 --2- 172.21.15.43:0/1570894338 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8c6406b4d0 0x7f8c6406d990 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8c840b78b0 tx=0x7f8c6c008040 comp rx=0 tx=0).stop 2024-04-25T09:01:25.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.306+0000 7f8c8941f640 1 -- 172.21.15.43:0/1570894338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c840b50d0 msgr2=0x7f8c84160000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:25.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.306+0000 7f8c8941f640 1 --2- 172.21.15.43:0/1570894338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c840b50d0 0x7f8c84160000 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f8c74000c00 tx=0x7f8c740046c0 comp rx=0 tx=0).stop 2024-04-25T09:01:25.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.306+0000 7f8c8941f640 1 -- 172.21.15.43:0/1570894338 shutdown_connections 2024-04-25T09:01:25.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.306+0000 7f8c8941f640 1 --2- 172.21.15.43:0/1570894338 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8c6406b4d0 0x7f8c6406d990 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:25.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.306+0000 7f8c8941f640 1 --2- 172.21.15.43:0/1570894338 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c8415de80 0x7f8c840b1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:25.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.306+0000 7f8c8941f640 1 --2- 172.21.15.43:0/1570894338 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c840d3ca0 0x7f8c84152010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:25.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.306+0000 7f8c8941f640 1 --2- 172.21.15.43:0/1570894338 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c840b50d0 0x7f8c84160000 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:25.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.306+0000 7f8c8941f640 1 -- 172.21.15.43:0/1570894338 >> 172.21.15.43:0/1570894338 conn(0x7f8c8408d460 msgr2=0x7f8c84152d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:25.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.307+0000 7f8c8941f640 1 -- 172.21.15.43:0/1570894338 shutdown_connections 2024-04-25T09:01:25.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.307+0000 7f8c8941f640 1 -- 172.21.15.43:0/1570894338 wait complete. 2024-04-25T09:01:25.318 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T09:01:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.440+0000 7f1c14c4a640 1 Processor -- start 2024-04-25T09:01:25.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.440+0000 7f1c14c4a640 1 -- start start 2024-04-25T09:01:25.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.441+0000 7f1c14c4a640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f1c1006f6a0 0x7f1c100b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:25.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.441+0000 7f1c14c4a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1c100cd470 0x7f1c1006c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:25.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.441+0000 7f1c14c4a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c100d2620 con 0x7f1c1015ddd0 2024-04-25T09:01:25.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.441+0000 7f1c14c4a640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c100b0b80 con 0x7f1c1006f6a0 2024-04-25T09:01:25.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.441+0000 7f1c14c4a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c100b0e60 con 0x7f1c100cd470 2024-04-25T09:01:25.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.441+0000 7f1c0e59c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f1c1015ddd0 0x7f1c1006ef90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48962/0 (socket says 172.21.15.43:48962) 2024-04-25T09:01:25.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.441+0000 7f1c0e59c640 1 -- 172.21.15.43:0/10404551 learned_addr learned my addr 172.21.15.43:0/10404551 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:25.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.442+0000 7f1c0d59a640 1 -- 172.21.15.43:0/10404551 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 353709196 0 0) 0x7f1c100d2620 con 0x7f1c1015ddd0 2024-04-25T09:01:25.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.442+0000 7f1c0d59a640 1 -- 172.21.15.43:0/10404551 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1bfc003520 con 0x7f1c1015ddd0 2024-04-25T09:01:25.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.442+0000 7f1c0dd9b640 1 --2- 172.21.15.43:0/10404551 >> v2:172.21.15.193:3300/0 conn(0x7f1c1006f6a0 0x7f1c100b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:25.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.442+0000 7f1c0ed9d640 1 --2- 172.21.15.43:0/10404551 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1c100cd470 0x7f1c1006c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:25.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.442+0000 7f1c0d59a640 1 -- 172.21.15.43:0/10404551 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3570352746 0 0) 0x7f1bfc003520 con 0x7f1c1015ddd0 2024-04-25T09:01:25.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.442+0000 7f1c0d59a640 1 -- 172.21.15.43:0/10404551 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1c100cd470 msgr2=0x7f1c1006c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:25.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.442+0000 7f1c0d59a640 1 --2- 172.21.15.43:0/10404551 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1c100cd470 0x7f1c1006c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:25.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.442+0000 7f1c0d59a640 1 -- 172.21.15.43:0/10404551 >> v2:172.21.15.193:3300/0 conn(0x7f1c1006f6a0 msgr2=0x7f1c100b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:25.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.442+0000 7f1c0d59a640 1 --2- 172.21.15.43:0/10404551 >> v2:172.21.15.193:3300/0 conn(0x7f1c1006f6a0 0x7f1c100b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:25.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.442+0000 7f1c0d59a640 1 -- 172.21.15.43:0/10404551 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c100cbf40 con 0x7f1c1015ddd0 2024-04-25T09:01:25.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.442+0000 7f1c0d59a640 1 -- 172.21.15.43:0/10404551 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f1c08001670 con 0x7f1c1015ddd0 2024-04-25T09:01:25.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.442+0000 7f1c0d59a640 1 -- 172.21.15.43:0/10404551 >> v1:172.21.15.43:6789/0 conn(0x7f1c1015ddd0 legacy=0x7f1c1006ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:25.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.442+0000 7f1c0d59a640 1 --2- 172.21.15.43:0/10404551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bfc0039b0 0x7f1bfc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:25.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.442+0000 7f1c0d59a640 1 --2- 172.21.15.43:0/10404551 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1c100cd470 0x7f1bfc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:25.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.442+0000 7f1c0d59a640 1 --2- 172.21.15.43:0/10404551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1bfc004d40 0x7f1bfc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:25.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.442+0000 7f1c0d59a640 1 -- 172.21.15.43:0/10404551 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c100b0b80 con 0x7f1bfc0039b0 2024-04-25T09:01:25.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.442+0000 7f1c0d59a640 1 -- 172.21.15.43:0/10404551 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c100b0e60 con 0x7f1bfc004d40 2024-04-25T09:01:25.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.442+0000 7f1c0d59a640 1 -- 172.21.15.43:0/10404551 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bfc00b9c0 con 0x7f1c100cd470 2024-04-25T09:01:25.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.442+0000 7f1c0e59c640 1 --2- 172.21.15.43:0/10404551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bfc0039b0 0x7f1bfc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:25.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.442+0000 7f1c0dd9b640 1 --2- 172.21.15.43:0/10404551 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1c100cd470 0x7f1bfc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:25.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.442+0000 7f1c0ed9d640 1 --2- 172.21.15.43:0/10404551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1bfc004d40 0x7f1bfc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:25.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.442+0000 7f1c0e59c640 1 -- 172.21.15.43:0/10404551 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1c100cd470 msgr2=0x7f1bfc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:25.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.442+0000 7f1c0e59c640 1 --2- 172.21.15.43:0/10404551 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1c100cd470 0x7f1bfc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:25.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.442+0000 7f1c0e59c640 1 -- 172.21.15.43:0/10404551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1bfc004d40 msgr2=0x7f1bfc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:25.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.442+0000 7f1c0e59c640 1 --2- 172.21.15.43:0/10404551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1bfc004d40 0x7f1bfc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:25.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.442+0000 7f1c0e59c640 1 -- 172.21.15.43:0/10404551 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bfc003520 con 0x7f1bfc0039b0 2024-04-25T09:01:25.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.443+0000 7f1c0e59c640 1 --2- 172.21.15.43:0/10404551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bfc0039b0 0x7f1bfc003dd0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f1c08001050 tx=0x7f1bfc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2e6ab4120527813 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:25.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.443+0000 7f1c0d59a640 1 -- 172.21.15.43:0/10404551 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c0802e070 con 0x7f1bfc0039b0 2024-04-25T09:01:25.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.443+0000 7f1c0d59a640 1 -- 172.21.15.43:0/10404551 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c08005af0 con 0x7f1bfc0039b0 2024-04-25T09:01:25.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.443+0000 7f1c0d59a640 1 -- 172.21.15.43:0/10404551 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c08007d70 con 0x7f1bfc0039b0 2024-04-25T09:01:25.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.444+0000 7f1c14c4a640 1 -- 172.21.15.43:0/10404551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bfc0039b0 msgr2=0x7f1bfc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:25.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.444+0000 7f1c14c4a640 1 --2- 172.21.15.43:0/10404551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bfc0039b0 0x7f1bfc003dd0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f1c08001050 tx=0x7f1bfc005830 comp rx=0 tx=0).stop 2024-04-25T09:01:25.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.444+0000 7f1c0ed9d640 1 -- 172.21.15.43:0/10404551 reap_dead start 2024-04-25T09:01:25.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.444+0000 7f1c14c4a640 1 -- 172.21.15.43:0/10404551 shutdown_connections 2024-04-25T09:01:25.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.444+0000 7f1c14c4a640 1 -- 172.21.15.43:0/10404551 >> 172.21.15.43:0/10404551 conn(0x7f1c100d17e0 msgr2=0x7f1c100d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:25.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.444+0000 7f1c14c4a640 1 -- 172.21.15.43:0/10404551 shutdown_connections 2024-04-25T09:01:25.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.444+0000 7f1c14c4a640 1 -- 172.21.15.43:0/10404551 wait complete. 2024-04-25T09:01:25.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.445+0000 7f1c14c4a640 1 Processor -- start 2024-04-25T09:01:25.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.445+0000 7f1c14c4a640 1 -- start start 2024-04-25T09:01:25.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.446+0000 7f1c14c4a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bfc0039b0 0x7f1c10151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:25.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.446+0000 7f1c14c4a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c101605c0 0x7f1c10160a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:25.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.446+0000 7f1c14c4a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1c100b1340 0x7f1c100b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:25.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.446+0000 7f1c14c4a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1006c8b0 con 0x7f1bfc0039b0 2024-04-25T09:01:25.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.446+0000 7f1c14c4a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1006c730 con 0x7f1c101605c0 2024-04-25T09:01:25.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.446+0000 7f1c14c4a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c1006ca30 con 0x7f1c100b1340 2024-04-25T09:01:25.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.446+0000 7f1c0e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bfc0039b0 0x7f1c10151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:25.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.446+0000 7f1c0e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bfc0039b0 0x7f1c10151730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58182/0 (socket says 172.21.15.43:58182) 2024-04-25T09:01:25.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.446+0000 7f1c0e59c640 1 -- 172.21.15.43:0/618003856 learned_addr learned my addr 172.21.15.43:0/618003856 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:25.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.446+0000 7f1c0dd9b640 1 --2- 172.21.15.43:0/618003856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c101605c0 0x7f1c10160a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:25.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.446+0000 7f1c0ed9d640 1 --2- 172.21.15.43:0/618003856 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1c100b1340 0x7f1c100b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:25.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.447+0000 7f1c0e59c640 1 -- 172.21.15.43:0/618003856 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1c100b1340 msgr2=0x7f1c100b17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:25.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.447+0000 7f1c0e59c640 1 --2- 172.21.15.43:0/618003856 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1c100b1340 0x7f1c100b17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:25.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.447+0000 7f1c0e59c640 1 -- 172.21.15.43:0/618003856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c101605c0 msgr2=0x7f1c10160a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:25.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.447+0000 7f1c0e59c640 1 --2- 172.21.15.43:0/618003856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c101605c0 0x7f1c10160a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:25.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.447+0000 7f1c0e59c640 1 -- 172.21.15.43:0/618003856 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c100b2070 con 0x7f1bfc0039b0 2024-04-25T09:01:25.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.447+0000 7f1c0e59c640 1 --2- 172.21.15.43:0/618003856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bfc0039b0 0x7f1c10151730 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f1c0800c300 tx=0x7f1c08000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:25.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.447+0000 7f1c06ffd640 1 -- 172.21.15.43:0/618003856 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c08042070 con 0x7f1bfc0039b0 2024-04-25T09:01:25.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.447+0000 7f1c06ffd640 1 -- 172.21.15.43:0/618003856 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c08007480 con 0x7f1bfc0039b0 2024-04-25T09:01:25.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.447+0000 7f1c06ffd640 1 -- 172.21.15.43:0/618003856 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c08001cc0 con 0x7f1bfc0039b0 2024-04-25T09:01:25.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.447+0000 7f1c14c4a640 1 -- 172.21.15.43:0/618003856 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c100b2350 con 0x7f1bfc0039b0 2024-04-25T09:01:25.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.447+0000 7f1c14c4a640 1 -- 172.21.15.43:0/618003856 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c100b2910 con 0x7f1bfc0039b0 2024-04-25T09:01:25.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.448+0000 7f1c14c4a640 1 -- 172.21.15.43:0/618003856 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c10074f60 con 0x7f1bfc0039b0 2024-04-25T09:01:25.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.449+0000 7f1c06ffd640 1 -- 172.21.15.43:0/618003856 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1c08045070 con 0x7f1bfc0039b0 2024-04-25T09:01:25.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.449+0000 7f1c06ffd640 1 --2- 172.21.15.43:0/618003856 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1bf006b500 0x7f1bf006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:25.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.449+0000 7f1c06ffd640 1 -- 172.21.15.43:0/618003856 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1c080aa7d0 con 0x7f1bfc0039b0 2024-04-25T09:01:25.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.449+0000 7f1c0dd9b640 1 --2- 172.21.15.43:0/618003856 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1bf006b500 0x7f1bf006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:25.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.450+0000 7f1c0dd9b640 1 --2- 172.21.15.43:0/618003856 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1bf006b500 0x7f1bf006d9c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f1c100b75e0 tx=0x7f1bf8009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:25.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.453+0000 7f1c06ffd640 1 -- 172.21.15.43:0/618003856 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1c08005ac0 con 0x7f1bfc0039b0 2024-04-25T09:01:25.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.644+0000 7f1c14c4a640 1 -- 172.21.15.43:0/618003856 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f1c100b3b60 con 0x7f1bfc0039b0 2024-04-25T09:01:25.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.647+0000 7f1c06ffd640 1 -- 172.21.15.43:0/618003856 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1c080794b0 con 0x7f1bfc0039b0 2024-04-25T09:01:25.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.649+0000 7f1c14c4a640 1 -- 172.21.15.43:0/618003856 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1bf006b500 msgr2=0x7f1bf006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:25.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.649+0000 7f1c14c4a640 1 --2- 172.21.15.43:0/618003856 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1bf006b500 0x7f1bf006d9c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f1c100b75e0 tx=0x7f1bf8009190 comp rx=0 tx=0).stop 2024-04-25T09:01:25.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.649+0000 7f1c14c4a640 1 -- 172.21.15.43:0/618003856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bfc0039b0 msgr2=0x7f1c10151730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:25.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.649+0000 7f1c14c4a640 1 --2- 172.21.15.43:0/618003856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bfc0039b0 0x7f1c10151730 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f1c0800c300 tx=0x7f1c08000fe0 comp rx=0 tx=0).stop 2024-04-25T09:01:25.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.649+0000 7f1c14c4a640 1 -- 172.21.15.43:0/618003856 shutdown_connections 2024-04-25T09:01:25.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.649+0000 7f1c14c4a640 1 --2- 172.21.15.43:0/618003856 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1c100b1340 0x7f1c100b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:25.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.649+0000 7f1c14c4a640 1 --2- 172.21.15.43:0/618003856 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1bf006b500 0x7f1bf006d9c0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:25.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.649+0000 7f1c14c4a640 1 --2- 172.21.15.43:0/618003856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1c101605c0 0x7f1c10160a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:25.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.649+0000 7f1c14c4a640 1 --2- 172.21.15.43:0/618003856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1bfc0039b0 0x7f1c10151730 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:25.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.649+0000 7f1c14c4a640 1 -- 172.21.15.43:0/618003856 >> 172.21.15.43:0/618003856 conn(0x7f1c100d17e0 msgr2=0x7f1c100c66a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:25.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.649+0000 7f1c14c4a640 1 -- 172.21.15.43:0/618003856 shutdown_connections 2024-04-25T09:01:25.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.649+0000 7f1c14c4a640 1 -- 172.21.15.43:0/618003856 wait complete. 2024-04-25T09:01:25.649 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort' 2024-04-25T09:01:25.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.761+0000 7f7ee10c4640 1 Processor -- start 2024-04-25T09:01:25.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.762+0000 7f7ee10c4640 1 -- start start 2024-04-25T09:01:25.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.762+0000 7f7ee10c4640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7edc0d0af0 0x7f7edc160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:25.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.762+0000 7f7ee10c4640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7edc0d2920 con 0x7f7edc15dce0 2024-04-25T09:01:25.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.762+0000 7f7ee10c4640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7edc0cee80 con 0x7f7edc15ef50 2024-04-25T09:01:25.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.762+0000 7f7ee10c4640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc0cec60 con 0x7f7edc0d0af0 2024-04-25T09:01:25.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.762+0000 7f7edad9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f7edc15dce0 0x7f7edc0ce550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48966/0 (socket says 172.21.15.43:48966) 2024-04-25T09:01:25.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.763+0000 7f7edad9d640 1 -- 172.21.15.43:0/4280440129 learned_addr learned my addr 172.21.15.43:0/4280440129 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:25.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.763+0000 7f7edb59e640 1 --2- 172.21.15.43:0/4280440129 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7edc0d0af0 0x7f7edc160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:25.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.763+0000 7f7ed9d9b640 1 -- 172.21.15.43:0/4280440129 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3018968099 0 0) 0x7f7edc0d2920 con 0x7f7edc15dce0 2024-04-25T09:01:25.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.763+0000 7f7ed9d9b640 1 -- 172.21.15.43:0/4280440129 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ec4003520 con 0x7f7edc15dce0 2024-04-25T09:01:25.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.763+0000 7f7ed9d9b640 1 -- 172.21.15.43:0/4280440129 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1777913393 0 0) 0x7f7edc0cee80 con 0x7f7edc15ef50 2024-04-25T09:01:25.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.763+0000 7f7ed9d9b640 1 -- 172.21.15.43:0/4280440129 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7edc0d2920 con 0x7f7edc15ef50 2024-04-25T09:01:25.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.763+0000 7f7edb59e640 1 -- 172.21.15.43:0/4280440129 >> v1:172.21.15.193:6789/0 conn(0x7f7edc15ef50 legacy=0x7f7edc0d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:25.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.763+0000 7f7edb59e640 1 -- 172.21.15.43:0/4280440129 >> v1:172.21.15.43:6789/0 conn(0x7f7edc15dce0 legacy=0x7f7edc0ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:25.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.763+0000 7f7edb59e640 1 -- 172.21.15.43:0/4280440129 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7edc0cf160 con 0x7f7edc0d0af0 2024-04-25T09:01:25.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.764+0000 7f7edb59e640 1 --2- 172.21.15.43:0/4280440129 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7edc0d0af0 0x7f7edc160980 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4009dc0 tx=0x7f7ed402a640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ab2dadd7db462a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:25.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.764+0000 7f7ed9d9b640 1 -- 172.21.15.43:0/4280440129 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed402ae60 con 0x7f7edc0d0af0 2024-04-25T09:01:25.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.764+0000 7f7ed9d9b640 1 -- 172.21.15.43:0/4280440129 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed4007740 con 0x7f7edc0d0af0 2024-04-25T09:01:25.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.764+0000 7f7ed9d9b640 1 -- 172.21.15.43:0/4280440129 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed4007b10 con 0x7f7edc0d0af0 2024-04-25T09:01:25.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.764+0000 7f7ee10c4640 1 -- 172.21.15.43:0/4280440129 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7edc0d0af0 msgr2=0x7f7edc160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:25.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.764+0000 7f7ee10c4640 1 --2- 172.21.15.43:0/4280440129 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7edc0d0af0 0x7f7edc160980 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4009dc0 tx=0x7f7ed402a640 comp rx=0 tx=0).stop 2024-04-25T09:01:25.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.764+0000 7f7ee10c4640 1 -- 172.21.15.43:0/4280440129 shutdown_connections 2024-04-25T09:01:25.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.764+0000 7f7ee10c4640 1 --2- 172.21.15.43:0/4280440129 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7edc0d0af0 0x7f7edc160980 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:25.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.765+0000 7f7ee10c4640 1 -- 172.21.15.43:0/4280440129 >> 172.21.15.43:0/4280440129 conn(0x7f7edc09cd00 msgr2=0x7f7edc0d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:25.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.765+0000 7f7ee10c4640 1 -- 172.21.15.43:0/4280440129 shutdown_connections 2024-04-25T09:01:25.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.765+0000 7f7ee10c4640 1 -- 172.21.15.43:0/4280440129 wait complete. 2024-04-25T09:01:25.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.766+0000 7f7ee10c4640 1 Processor -- start 2024-04-25T09:01:25.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.766+0000 7f7ee10c4640 1 -- start start 2024-04-25T09:01:25.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.767+0000 7f7ee10c4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7edc0d0af0 0x7f7edc154cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:25.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.767+0000 7f7ee10c4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7edc15dce0 0x7f7edc0b8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:25.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.767+0000 7f7ee10c4640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7edc15ef50 0x7f7edc09f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:25.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.767+0000 7f7ee10c4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc15f6b0 con 0x7f7edc0d0af0 2024-04-25T09:01:25.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.767+0000 7f7ee10c4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc0cf720 con 0x7f7edc15dce0 2024-04-25T09:01:25.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.767+0000 7f7ee10c4640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc15e0c0 con 0x7f7edc15ef50 2024-04-25T09:01:25.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.767+0000 7f7edad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7edc0d0af0 0x7f7edc154cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:25.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.767+0000 7f7edad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7edc0d0af0 0x7f7edc154cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58194/0 (socket says 172.21.15.43:58194) 2024-04-25T09:01:25.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.767+0000 7f7edad9d640 1 -- 172.21.15.43:0/3168643819 learned_addr learned my addr 172.21.15.43:0/3168643819 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:25.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.767+0000 7f7edb59e640 1 --2- 172.21.15.43:0/3168643819 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7edc15ef50 0x7f7edc09f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:25.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.767+0000 7f7eda59c640 1 --2- 172.21.15.43:0/3168643819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7edc15dce0 0x7f7edc0b8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:25.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.767+0000 7f7edad9d640 1 -- 172.21.15.43:0/3168643819 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7edc15ef50 msgr2=0x7f7edc09f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:25.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.767+0000 7f7edad9d640 1 --2- 172.21.15.43:0/3168643819 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7edc15ef50 0x7f7edc09f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:25.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.767+0000 7f7edad9d640 1 -- 172.21.15.43:0/3168643819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7edc15dce0 msgr2=0x7f7edc0b8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:25.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.767+0000 7f7edad9d640 1 --2- 172.21.15.43:0/3168643819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7edc15dce0 0x7f7edc0b8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:25.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.767+0000 7f7edad9d640 1 -- 172.21.15.43:0/3168643819 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7edc15c2a0 con 0x7f7edc0d0af0 2024-04-25T09:01:25.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.767+0000 7f7edad9d640 1 --2- 172.21.15.43:0/3168643819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7edc0d0af0 0x7f7edc154cc0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc00c4c0 tx=0x7f7ecc00c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:25.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.768+0000 7f7ec37fe640 1 -- 172.21.15.43:0/3168643819 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc017070 con 0x7f7edc0d0af0 2024-04-25T09:01:25.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.768+0000 7f7ec37fe640 1 -- 172.21.15.43:0/3168643819 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ecc005ce0 con 0x7f7edc0d0af0 2024-04-25T09:01:25.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.768+0000 7f7ee10c4640 1 -- 172.21.15.43:0/3168643819 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7edc09e3d0 con 0x7f7edc0d0af0 2024-04-25T09:01:25.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.768+0000 7f7ec37fe640 1 -- 172.21.15.43:0/3168643819 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc007270 con 0x7f7edc0d0af0 2024-04-25T09:01:25.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.768+0000 7f7ee10c4640 1 -- 172.21.15.43:0/3168643819 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7edc09e990 con 0x7f7edc0d0af0 2024-04-25T09:01:25.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.769+0000 7f7ee10c4640 1 -- 172.21.15.43:0/3168643819 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7edc0680f0 con 0x7f7edc0d0af0 2024-04-25T09:01:25.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.769+0000 7f7ec37fe640 1 -- 172.21.15.43:0/3168643819 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7ecc01a070 con 0x7f7edc0d0af0 2024-04-25T09:01:25.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.769+0000 7f7ec37fe640 1 --2- 172.21.15.43:0/3168643819 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7ec806b4d0 0x7f7ec806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:25.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.769+0000 7f7ec37fe640 1 -- 172.21.15.43:0/3168643819 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7ecc089160 con 0x7f7edc0d0af0 2024-04-25T09:01:25.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.770+0000 7f7eda59c640 1 --2- 172.21.15.43:0/3168643819 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7ec806b4d0 0x7f7ec806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:25.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.770+0000 7f7eda59c640 1 --2- 172.21.15.43:0/3168643819 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7ec806b4d0 0x7f7ec806d990 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0007fd0 tx=0x7f7ed0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:25.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:25.774+0000 7f7ec37fe640 1 -- 172.21.15.43:0/3168643819 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7ecc057c90 con 0x7f7edc0d0af0 2024-04-25T09:01:26.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.049+0000 7f7ee10c4640 1 -- 172.21.15.43:0/3168643819 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]} v 0) v1 -- 0x7f7edc074ef0 con 0x7f7edc0d0af0 2024-04-25T09:01:26.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.309+0000 7f7ec37fe640 1 -- 172.21.15.43:0/3168643819 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]}]=0 v345) v1 ==== 144+0+0 (secure 0 0 0) 0x7f7ecc05bd30 con 0x7f7edc0d0af0 2024-04-25T09:01:26.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.312+0000 7f7ee10c4640 1 -- 172.21.15.43:0/3168643819 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7ec806b4d0 msgr2=0x7f7ec806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:26.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.312+0000 7f7ee10c4640 1 --2- 172.21.15.43:0/3168643819 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7ec806b4d0 0x7f7ec806d990 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7ed0007fd0 tx=0x7f7ed0009040 comp rx=0 tx=0).stop 2024-04-25T09:01:26.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.312+0000 7f7ee10c4640 1 -- 172.21.15.43:0/3168643819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7edc0d0af0 msgr2=0x7f7edc154cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:26.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.312+0000 7f7ee10c4640 1 --2- 172.21.15.43:0/3168643819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7edc0d0af0 0x7f7edc154cc0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc00c4c0 tx=0x7f7ecc00c980 comp rx=0 tx=0).stop 2024-04-25T09:01:26.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.312+0000 7f7ee10c4640 1 -- 172.21.15.43:0/3168643819 shutdown_connections 2024-04-25T09:01:26.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.312+0000 7f7ee10c4640 1 --2- 172.21.15.43:0/3168643819 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7edc15ef50 0x7f7edc09f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:26.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.312+0000 7f7ee10c4640 1 --2- 172.21.15.43:0/3168643819 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7ec806b4d0 0x7f7ec806d990 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:26.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.312+0000 7f7ee10c4640 1 --2- 172.21.15.43:0/3168643819 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7edc15dce0 0x7f7edc0b8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:26.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.312+0000 7f7ee10c4640 1 --2- 172.21.15.43:0/3168643819 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7edc0d0af0 0x7f7edc154cc0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:26.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.312+0000 7f7ee10c4640 1 -- 172.21.15.43:0/3168643819 >> 172.21.15.43:0/3168643819 conn(0x7f7edc09cd00 msgr2=0x7f7edc0d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:26.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.312+0000 7f7ee10c4640 1 -- 172.21.15.43:0/3168643819 shutdown_connections 2024-04-25T09:01:26.302 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.312+0000 7f7ee10c4640 1 -- 172.21.15.43:0/3168643819 wait complete. 2024-04-25T09:01:26.313 INFO:tasks.cephfs_test_runner:test_scrub_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T09:01:26.313 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T09:01:26.313 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:01:26.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.426+0000 7f249589a640 1 Processor -- start 2024-04-25T09:01:26.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.426+0000 7f249589a640 1 -- start start 2024-04-25T09:01:26.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.427+0000 7f249589a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2490161390 0x7f24900b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:26.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.427+0000 7f249589a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f24900d02a0 0x7f24900d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:26.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.427+0000 7f249589a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24900b4c30 con 0x7f24900ce5e0 2024-04-25T09:01:26.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.427+0000 7f249589a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24900d1910 con 0x7f2490161390 2024-04-25T09:01:26.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.427+0000 7f249589a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24900cf2c0 con 0x7f24900d02a0 2024-04-25T09:01:26.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.427+0000 7f248effd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2490161390 0x7f24900b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:26.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.427+0000 7f248f7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f24900ce5e0 0x7f24900cea70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48972/0 (socket says 172.21.15.43:48972) 2024-04-25T09:01:26.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.427+0000 7f248f7fe640 1 -- 172.21.15.43:0/2994877392 learned_addr learned my addr 172.21.15.43:0/2994877392 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:26.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.427+0000 7f248effd640 1 -- 172.21.15.43:0/2994877392 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f24900d02a0 msgr2=0x7f24900d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:26.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.427+0000 7f248effd640 1 --2- 172.21.15.43:0/2994877392 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f24900d02a0 0x7f24900d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:26.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.427+0000 7f248effd640 1 -- 172.21.15.43:0/2994877392 >> v1:172.21.15.43:6789/0 conn(0x7f24900ce5e0 legacy=0x7f24900cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:26.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.427+0000 7f248effd640 1 -- 172.21.15.43:0/2994877392 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24900cf5a0 con 0x7f2490161390 2024-04-25T09:01:26.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.427+0000 7f248dffb640 1 -- 172.21.15.43:0/2994877392 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2441462957 0 0) 0x7f2488001490 con 0x7f24900ce5e0 2024-04-25T09:01:26.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.428+0000 7f248effd640 1 --2- 172.21.15.43:0/2994877392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2490161390 0x7f24900b6820 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f24800097a0 tx=0x7f248002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21cfd7be217b3ce7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:26.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.428+0000 7f248dffb640 1 -- 172.21.15.43:0/2994877392 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2480037070 con 0x7f2490161390 2024-04-25T09:01:26.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.428+0000 7f248dffb640 1 -- 172.21.15.43:0/2994877392 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2480032440 con 0x7f2490161390 2024-04-25T09:01:26.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.428+0000 7f248dffb640 1 -- 172.21.15.43:0/2994877392 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2480032810 con 0x7f2490161390 2024-04-25T09:01:26.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.429+0000 7f249589a640 1 -- 172.21.15.43:0/2994877392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2490161390 msgr2=0x7f24900b6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:26.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.429+0000 7f249589a640 1 --2- 172.21.15.43:0/2994877392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2490161390 0x7f24900b6820 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f24800097a0 tx=0x7f248002a180 comp rx=0 tx=0).stop 2024-04-25T09:01:26.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.429+0000 7f249589a640 1 -- 172.21.15.43:0/2994877392 shutdown_connections 2024-04-25T09:01:26.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.429+0000 7f249589a640 1 --2- 172.21.15.43:0/2994877392 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f24900d02a0 0x7f24900d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:26.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.429+0000 7f249589a640 1 --2- 172.21.15.43:0/2994877392 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2490161390 0x7f24900b6820 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:26.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.429+0000 7f249589a640 1 -- 172.21.15.43:0/2994877392 >> 172.21.15.43:0/2994877392 conn(0x7f249009cd20 msgr2=0x7f24900d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:26.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.429+0000 7f249589a640 1 -- 172.21.15.43:0/2994877392 shutdown_connections 2024-04-25T09:01:26.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.429+0000 7f249589a640 1 -- 172.21.15.43:0/2994877392 wait complete. 2024-04-25T09:01:26.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.430+0000 7f249589a640 1 Processor -- start 2024-04-25T09:01:26.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.430+0000 7f249589a640 1 -- start start 2024-04-25T09:01:26.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.431+0000 7f249589a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f24900ce5e0 0x7f24900cd780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:26.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.431+0000 7f249589a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24900d02a0 0x7f24900c5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:26.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.431+0000 7f249589a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2490161390 0x7f24900c5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:26.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.431+0000 7f249589a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2490161770 con 0x7f24900d02a0 2024-04-25T09:01:26.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.431+0000 7f249589a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24900d15a0 con 0x7f2490161390 2024-04-25T09:01:26.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.431+0000 7f249589a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24900d1da0 con 0x7f24900ce5e0 2024-04-25T09:01:26.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.431+0000 7f248e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24900d02a0 0x7f24900c5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:26.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.431+0000 7f248e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24900d02a0 0x7f24900c5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58206/0 (socket says 172.21.15.43:58206) 2024-04-25T09:01:26.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.431+0000 7f248e7fc640 1 -- 172.21.15.43:0/2969327776 learned_addr learned my addr 172.21.15.43:0/2969327776 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:26.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.431+0000 7f248e7fc640 1 -- 172.21.15.43:0/2969327776 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f24900ce5e0 msgr2=0x7f24900cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:26.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.431+0000 7f248effd640 1 --2- 172.21.15.43:0/2969327776 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f24900ce5e0 0x7f24900cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:26.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.431+0000 7f248f7fe640 1 --2- 172.21.15.43:0/2969327776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2490161390 0x7f24900c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:26.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.431+0000 7f248e7fc640 1 --2- 172.21.15.43:0/2969327776 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f24900ce5e0 0x7f24900cd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:26.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.431+0000 7f248e7fc640 1 -- 172.21.15.43:0/2969327776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2490161390 msgr2=0x7f24900c5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:26.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.431+0000 7f248e7fc640 1 --2- 172.21.15.43:0/2969327776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2490161390 0x7f24900c5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:26.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.431+0000 7f248e7fc640 1 -- 172.21.15.43:0/2969327776 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24900cb8b0 con 0x7f24900d02a0 2024-04-25T09:01:26.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.431+0000 7f248f7fe640 1 --2- 172.21.15.43:0/2969327776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2490161390 0x7f24900c5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:26.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.432+0000 7f248e7fc640 1 --2- 172.21.15.43:0/2969327776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24900d02a0 0x7f24900c5000 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f248400b4a0 tx=0x7f248400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:26.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.432+0000 7f2477fff640 1 -- 172.21.15.43:0/2969327776 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2484014070 con 0x7f24900d02a0 2024-04-25T09:01:26.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.432+0000 7f2477fff640 1 -- 172.21.15.43:0/2969327776 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2484010610 con 0x7f24900d02a0 2024-04-25T09:01:26.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.432+0000 7f249589a640 1 -- 172.21.15.43:0/2969327776 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24901f4db0 con 0x7f24900d02a0 2024-04-25T09:01:26.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.432+0000 7f2477fff640 1 -- 172.21.15.43:0/2969327776 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2484010a00 con 0x7f24900d02a0 2024-04-25T09:01:26.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.432+0000 7f249589a640 1 -- 172.21.15.43:0/2969327776 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24900cbb30 con 0x7f24900d02a0 2024-04-25T09:01:26.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.433+0000 7f249589a640 1 -- 172.21.15.43:0/2969327776 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24900680f0 con 0x7f24900d02a0 2024-04-25T09:01:26.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.433+0000 7f2477fff640 1 -- 172.21.15.43:0/2969327776 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2484017070 con 0x7f24900d02a0 2024-04-25T09:01:26.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.434+0000 7f2477fff640 1 --2- 172.21.15.43:0/2969327776 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f247c06b4d0 0x7f247c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:26.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.434+0000 7f2477fff640 1 -- 172.21.15.43:0/2969327776 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2484089920 con 0x7f24900d02a0 2024-04-25T09:01:26.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.434+0000 7f248effd640 1 --2- 172.21.15.43:0/2969327776 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f247c06b4d0 0x7f247c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:26.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.434+0000 7f248effd640 1 --2- 172.21.15.43:0/2969327776 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f247c06b4d0 0x7f247c06d990 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f2480034000 tx=0x7f248002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:26.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.438+0000 7f2477fff640 1 -- 172.21.15.43:0/2969327776 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f24900cb8b0 con 0x7f24900d02a0 2024-04-25T09:01:26.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.622+0000 7f249589a640 1 -- 172.21.15.43:0/2969327776 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f24900cbd30 con 0x7f24900d02a0 2024-04-25T09:01:26.611 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.623+0000 7f2477fff640 1 -- 172.21.15.43:0/2969327776 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+12985 (secure 0 0 0) 0x7f24900cbd30 con 0x7f24900d02a0 2024-04-25T09:01:26.612 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:01:26.612 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":94,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:01:25.302097+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6835/1341028696":"2024-04-26T09:01:22.088774+0000","172.21.15.43:6833/1341028696":"2024-04-26T09:01:22.088774+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:01:26.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.625+0000 7f249589a640 1 -- 172.21.15.43:0/2969327776 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f247c06b4d0 msgr2=0x7f247c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:26.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.625+0000 7f249589a640 1 --2- 172.21.15.43:0/2969327776 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f247c06b4d0 0x7f247c06d990 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f2480034000 tx=0x7f248002e000 comp rx=0 tx=0).stop 2024-04-25T09:01:26.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.625+0000 7f249589a640 1 -- 172.21.15.43:0/2969327776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24900d02a0 msgr2=0x7f24900c5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:26.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.625+0000 7f249589a640 1 --2- 172.21.15.43:0/2969327776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24900d02a0 0x7f24900c5000 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f248400b4a0 tx=0x7f248400b960 comp rx=0 tx=0).stop 2024-04-25T09:01:26.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.625+0000 7f249589a640 1 -- 172.21.15.43:0/2969327776 shutdown_connections 2024-04-25T09:01:26.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.625+0000 7f249589a640 1 --2- 172.21.15.43:0/2969327776 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f247c06b4d0 0x7f247c06d990 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:26.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.625+0000 7f249589a640 1 --2- 172.21.15.43:0/2969327776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2490161390 0x7f24900c5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:26.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.625+0000 7f249589a640 1 --2- 172.21.15.43:0/2969327776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24900d02a0 0x7f24900c5000 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:26.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.625+0000 7f249589a640 1 --2- 172.21.15.43:0/2969327776 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f24900ce5e0 0x7f24900cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:26.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.625+0000 7f249589a640 1 -- 172.21.15.43:0/2969327776 >> 172.21.15.43:0/2969327776 conn(0x7f249009cd20 msgr2=0x7f24900b77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:26.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.625+0000 7f249589a640 1 -- 172.21.15.43:0/2969327776 shutdown_connections 2024-04-25T09:01:26.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.625+0000 7f249589a640 1 -- 172.21.15.43:0/2969327776 wait complete. 2024-04-25T09:01:26.626 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:01:26.627 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:01:26.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.742+0000 7fa5ceadd640 1 Processor -- start 2024-04-25T09:01:26.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.742+0000 7fa5ceadd640 1 -- start start 2024-04-25T09:01:26.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.743+0000 7fa5ceadd640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fa5c815dec0 0x7fa5c815ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:26.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.743+0000 7fa5ceadd640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5c80ce8c0 0x7fa5c80d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:26.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.743+0000 7fa5ceadd640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5c80cf110 con 0x7fa5c80d04a0 2024-04-25T09:01:26.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.743+0000 7fa5ceadd640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5c80cf3f0 con 0x7fa5c815dec0 2024-04-25T09:01:26.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.743+0000 7fa5ceadd640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c80cf6d0 con 0x7fa5c80ce8c0 2024-04-25T09:01:26.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.743+0000 7fa5cc879640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa5c80d04a0 0x7fa5c80d0880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48986/0 (socket says 172.21.15.43:48986) 2024-04-25T09:01:26.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.743+0000 7fa5cc879640 1 -- 172.21.15.43:0/3235358743 learned_addr learned my addr 172.21.15.43:0/3235358743 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:26.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.743+0000 7fa5c7fff640 1 --2- 172.21.15.43:0/3235358743 >> v2:172.21.15.193:3300/0 conn(0x7fa5c815dec0 0x7fa5c815ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:26.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.743+0000 7fa5cd07a640 1 --2- 172.21.15.43:0/3235358743 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5c80ce8c0 0x7fa5c80d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:26.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.743+0000 7fa5c77fe640 1 -- 172.21.15.43:0/3235358743 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3146945576 0 0) 0x7fa5c80cf110 con 0x7fa5c80d04a0 2024-04-25T09:01:26.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.743+0000 7fa5c77fe640 1 -- 172.21.15.43:0/3235358743 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5b8003520 con 0x7fa5c80d04a0 2024-04-25T09:01:26.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.743+0000 7fa5c77fe640 1 -- 172.21.15.43:0/3235358743 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 619398711 0 0) 0x7fa5b8003520 con 0x7fa5c80d04a0 2024-04-25T09:01:26.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.743+0000 7fa5c77fe640 1 -- 172.21.15.43:0/3235358743 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5c80ce8c0 msgr2=0x7fa5c80d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:26.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.743+0000 7fa5c77fe640 1 --2- 172.21.15.43:0/3235358743 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5c80ce8c0 0x7fa5c80d17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:26.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.743+0000 7fa5c77fe640 1 -- 172.21.15.43:0/3235358743 >> v2:172.21.15.193:3300/0 conn(0x7fa5c815dec0 msgr2=0x7fa5c815ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:26.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.743+0000 7fa5c77fe640 1 --2- 172.21.15.43:0/3235358743 >> v2:172.21.15.193:3300/0 conn(0x7fa5c815dec0 0x7fa5c815ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:26.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.744+0000 7fa5c77fe640 1 -- 172.21.15.43:0/3235358743 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c80cf9b0 con 0x7fa5c80d04a0 2024-04-25T09:01:26.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.744+0000 7fa5c77fe640 1 -- 172.21.15.43:0/3235358743 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7fa5bc001690 con 0x7fa5c80d04a0 2024-04-25T09:01:26.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.744+0000 7fa5c77fe640 1 -- 172.21.15.43:0/3235358743 >> v1:172.21.15.43:6789/0 conn(0x7fa5c80d04a0 legacy=0x7fa5c80d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:26.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.744+0000 7fa5c77fe640 1 --2- 172.21.15.43:0/3235358743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5b80039b0 0x7fa5b8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:26.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.744+0000 7fa5c77fe640 1 --2- 172.21.15.43:0/3235358743 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5c80ce8c0 0x7fa5b8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:26.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.744+0000 7fa5c77fe640 1 --2- 172.21.15.43:0/3235358743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5b8004d40 0x7fa5b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:26.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.744+0000 7fa5c77fe640 1 -- 172.21.15.43:0/3235358743 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c80cf3f0 con 0x7fa5b80039b0 2024-04-25T09:01:26.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.744+0000 7fa5c77fe640 1 -- 172.21.15.43:0/3235358743 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c80cf6d0 con 0x7fa5b8004d40 2024-04-25T09:01:26.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.744+0000 7fa5c77fe640 1 -- 172.21.15.43:0/3235358743 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b800b9c0 con 0x7fa5c80ce8c0 2024-04-25T09:01:26.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.744+0000 7fa5cc879640 1 --2- 172.21.15.43:0/3235358743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5b80039b0 0x7fa5b8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:26.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.744+0000 7fa5c7fff640 1 --2- 172.21.15.43:0/3235358743 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5c80ce8c0 0x7fa5b8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:26.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.744+0000 7fa5cd07a640 1 --2- 172.21.15.43:0/3235358743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5b8004d40 0x7fa5b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:26.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.744+0000 7fa5cc879640 1 -- 172.21.15.43:0/3235358743 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5c80ce8c0 msgr2=0x7fa5b8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:26.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.744+0000 7fa5cc879640 1 --2- 172.21.15.43:0/3235358743 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5c80ce8c0 0x7fa5b8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:26.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.744+0000 7fa5cc879640 1 -- 172.21.15.43:0/3235358743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5b8004d40 msgr2=0x7fa5b80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:26.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.744+0000 7fa5cc879640 1 --2- 172.21.15.43:0/3235358743 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5b8004d40 0x7fa5b80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:26.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.744+0000 7fa5cc879640 1 -- 172.21.15.43:0/3235358743 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b8003520 con 0x7fa5b80039b0 2024-04-25T09:01:26.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.745+0000 7fa5cc879640 1 --2- 172.21.15.43:0/3235358743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5b80039b0 0x7fa5b8003dd0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc00c240 tx=0x7fa5b8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd33bce16ab09f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:26.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.745+0000 7fa5c77fe640 1 -- 172.21.15.43:0/3235358743 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc007d70 con 0x7fa5b80039b0 2024-04-25T09:01:26.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.745+0000 7fa5c77fe640 1 -- 172.21.15.43:0/3235358743 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5bc005ad0 con 0x7fa5b80039b0 2024-04-25T09:01:26.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.745+0000 7fa5c77fe640 1 -- 172.21.15.43:0/3235358743 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc007070 con 0x7fa5b80039b0 2024-04-25T09:01:26.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.745+0000 7fa5ceadd640 1 -- 172.21.15.43:0/3235358743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5b80039b0 msgr2=0x7fa5b8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:26.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.745+0000 7fa5ceadd640 1 --2- 172.21.15.43:0/3235358743 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5b80039b0 0x7fa5b8003dd0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc00c240 tx=0x7fa5b8005830 comp rx=0 tx=0).stop 2024-04-25T09:01:26.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.745+0000 7fa5cd07a640 1 -- 172.21.15.43:0/3235358743 reap_dead start 2024-04-25T09:01:26.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.746+0000 7fa5ceadd640 1 -- 172.21.15.43:0/3235358743 shutdown_connections 2024-04-25T09:01:26.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.746+0000 7fa5ceadd640 1 -- 172.21.15.43:0/3235358743 >> 172.21.15.43:0/3235358743 conn(0x7fa5c809cd40 msgr2=0x7fa5c80d3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:26.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.746+0000 7fa5ceadd640 1 -- 172.21.15.43:0/3235358743 shutdown_connections 2024-04-25T09:01:26.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.746+0000 7fa5ceadd640 1 -- 172.21.15.43:0/3235358743 wait complete. 2024-04-25T09:01:26.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.747+0000 7fa5ceadd640 1 Processor -- start 2024-04-25T09:01:26.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.747+0000 7fa5ceadd640 1 -- start start 2024-04-25T09:01:26.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.748+0000 7fa5ceadd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5b80039b0 0x7fa5c8158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:26.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.748+0000 7fa5ceadd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5c80cc2d0 0x7fa5c80cb260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:26.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.748+0000 7fa5ceadd640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5c80cb7a0 0x7fa5c815fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:26.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.748+0000 7fa5ceadd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c80cee70 con 0x7fa5c80cc2d0 2024-04-25T09:01:26.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.748+0000 7fa5ceadd640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c80cecf0 con 0x7fa5b80039b0 2024-04-25T09:01:26.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.748+0000 7fa5ceadd640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c80d17a0 con 0x7fa5c80cb7a0 2024-04-25T09:01:26.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.748+0000 7fa5c7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5c80cc2d0 0x7fa5c80cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:26.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.748+0000 7fa5c7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5c80cc2d0 0x7fa5c80cb260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58230/0 (socket says 172.21.15.43:58230) 2024-04-25T09:01:26.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.748+0000 7fa5c7fff640 1 -- 172.21.15.43:0/186137579 learned_addr learned my addr 172.21.15.43:0/186137579 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:26.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.748+0000 7fa5cc879640 1 --2- 172.21.15.43:0/186137579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5b80039b0 0x7fa5c8158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:26.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.748+0000 7fa5cd07a640 1 --2- 172.21.15.43:0/186137579 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5c80cb7a0 0x7fa5c815fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:26.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.748+0000 7fa5c7fff640 1 -- 172.21.15.43:0/186137579 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5c80cb7a0 msgr2=0x7fa5c815fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:26.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.748+0000 7fa5c7fff640 1 --2- 172.21.15.43:0/186137579 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5c80cb7a0 0x7fa5c815fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:26.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.748+0000 7fa5c7fff640 1 -- 172.21.15.43:0/186137579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5b80039b0 msgr2=0x7fa5c8158b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:26.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.748+0000 7fa5c7fff640 1 --2- 172.21.15.43:0/186137579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5b80039b0 0x7fa5c8158b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:26.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.748+0000 7fa5c7fff640 1 -- 172.21.15.43:0/186137579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5c8159380 con 0x7fa5c80cc2d0 2024-04-25T09:01:26.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.748+0000 7fa5c7fff640 1 --2- 172.21.15.43:0/186137579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5c80cc2d0 0x7fa5c80cb260 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4011930 tx=0x7fa5b4011df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:26.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.748+0000 7fa5c57fa640 1 -- 172.21.15.43:0/186137579 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b400fa90 con 0x7fa5c80cc2d0 2024-04-25T09:01:26.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.748+0000 7fa5c57fa640 1 -- 172.21.15.43:0/186137579 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5b400fc30 con 0x7fa5c80cc2d0 2024-04-25T09:01:26.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.748+0000 7fa5c57fa640 1 -- 172.21.15.43:0/186137579 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5b4014440 con 0x7fa5c80cc2d0 2024-04-25T09:01:26.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.748+0000 7fa5ceadd640 1 -- 172.21.15.43:0/186137579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c80cceb0 con 0x7fa5c80cc2d0 2024-04-25T09:01:26.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.748+0000 7fa5ceadd640 1 -- 172.21.15.43:0/186137579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5c80cd470 con 0x7fa5c80cc2d0 2024-04-25T09:01:26.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.749+0000 7fa5ceadd640 1 -- 172.21.15.43:0/186137579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5c8074f60 con 0x7fa5c80cc2d0 2024-04-25T09:01:26.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.750+0000 7fa5c57fa640 1 -- 172.21.15.43:0/186137579 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa5b4012040 con 0x7fa5c80cc2d0 2024-04-25T09:01:26.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.750+0000 7fa5c57fa640 1 --2- 172.21.15.43:0/186137579 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa5ac06b4d0 0x7fa5ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:26.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.750+0000 7fa5c57fa640 1 -- 172.21.15.43:0/186137579 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa5b408a2d0 con 0x7fa5c80cc2d0 2024-04-25T09:01:26.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.751+0000 7fa5cc879640 1 --2- 172.21.15.43:0/186137579 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa5ac06b4d0 0x7fa5ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:26.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.751+0000 7fa5cc879640 1 --2- 172.21.15.43:0/186137579 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa5ac06b4d0 0x7fa5ac06d990 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc033000 tx=0x7fa5bc02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:26.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.755+0000 7fa5c57fa640 1 -- 172.21.15.43:0/186137579 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa5b4019030 con 0x7fa5c80cc2d0 2024-04-25T09:01:26.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.935+0000 7fa5ceadd640 1 -- 172.21.15.43:0/186137579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa5c808daf0 con 0x7fa5c80cc2d0 2024-04-25T09:01:26.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.935+0000 7fa5c57fa640 1 -- 172.21.15.43:0/186137579 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa5b4058f50 con 0x7fa5c80cc2d0 2024-04-25T09:01:26.924 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T09:01:26.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.937+0000 7fa5ceadd640 1 -- 172.21.15.43:0/186137579 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa5ac06b4d0 msgr2=0x7fa5ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:26.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.937+0000 7fa5ceadd640 1 --2- 172.21.15.43:0/186137579 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa5ac06b4d0 0x7fa5ac06d990 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc033000 tx=0x7fa5bc02e000 comp rx=0 tx=0).stop 2024-04-25T09:01:26.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.937+0000 7fa5ceadd640 1 -- 172.21.15.43:0/186137579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5c80cc2d0 msgr2=0x7fa5c80cb260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:26.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.937+0000 7fa5ceadd640 1 --2- 172.21.15.43:0/186137579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5c80cc2d0 0x7fa5c80cb260 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7fa5b4011930 tx=0x7fa5b4011df0 comp rx=0 tx=0).stop 2024-04-25T09:01:26.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.937+0000 7fa5ceadd640 1 -- 172.21.15.43:0/186137579 shutdown_connections 2024-04-25T09:01:26.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.937+0000 7fa5ceadd640 1 --2- 172.21.15.43:0/186137579 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5c80cb7a0 0x7fa5c815fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:26.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.937+0000 7fa5ceadd640 1 --2- 172.21.15.43:0/186137579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5c80cc2d0 0x7fa5c80cb260 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:26.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.937+0000 7fa5ceadd640 1 --2- 172.21.15.43:0/186137579 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa5ac06b4d0 0x7fa5ac06d990 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:26.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.937+0000 7fa5ceadd640 1 --2- 172.21.15.43:0/186137579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5b80039b0 0x7fa5c8158b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:26.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.937+0000 7fa5ceadd640 1 -- 172.21.15.43:0/186137579 >> 172.21.15.43:0/186137579 conn(0x7fa5c809cd40 msgr2=0x7fa5c815be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:26.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.938+0000 7fa5ceadd640 1 -- 172.21.15.43:0/186137579 shutdown_connections 2024-04-25T09:01:26.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:26.938+0000 7fa5ceadd640 1 -- 172.21.15.43:0/186137579 wait complete. 2024-04-25T09:01:26.938 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume' 2024-04-25T09:01:27.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.050+0000 7fc1f2da9640 1 Processor -- start 2024-04-25T09:01:27.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.050+0000 7fc1f2da9640 1 -- start start 2024-04-25T09:01:27.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.051+0000 7fc1f2da9640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc1ec15ef50 0x7fc1ec0d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:27.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.051+0000 7fc1f2da9640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc1ec0d0920 0x7fc1ec160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:27.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.051+0000 7fc1f2da9640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1ec0cee80 con 0x7fc1ec15dce0 2024-04-25T09:01:27.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.051+0000 7fc1f2da9640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1ec0cec60 con 0x7fc1ec0d0920 2024-04-25T09:01:27.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.051+0000 7fc1f2da9640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ec0d0d70 con 0x7fc1ec15ef50 2024-04-25T09:01:27.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.051+0000 7fc1f1346640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc1ec0d0920 0x7fc1ec160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:27.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.051+0000 7fc1f1346640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc1ec0d0920 0x7fc1ec160980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58244/0 (socket says 172.21.15.43:58244) 2024-04-25T09:01:27.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.051+0000 7fc1f1346640 1 -- 172.21.15.43:0/2772838255 learned_addr learned my addr 172.21.15.43:0/2772838255 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:27.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.051+0000 7fc1ebfff640 1 --2- 172.21.15.43:0/2772838255 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc1ec15ef50 0x7fc1ec0d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:27.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.051+0000 7fc1f1346640 1 -- 172.21.15.43:0/2772838255 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc1ec15ef50 msgr2=0x7fc1ec0d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:27.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.051+0000 7fc1f1346640 1 --2- 172.21.15.43:0/2772838255 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc1ec15ef50 0x7fc1ec0d03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:27.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.051+0000 7fc1f1346640 1 -- 172.21.15.43:0/2772838255 >> v1:172.21.15.193:6789/0 conn(0x7fc1ec15dce0 legacy=0x7fc1ec0ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:27.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.051+0000 7fc1f1346640 1 -- 172.21.15.43:0/2772838255 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1ec0d2920 con 0x7fc1ec0d0920 2024-04-25T09:01:27.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.052+0000 7fc1f1346640 1 --2- 172.21.15.43:0/2772838255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1ec0d0920 0x7fc1ec160980 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4009e50 tx=0x7fc1e402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edc6e95104e7aa73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:27.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.052+0000 7fc1eb7fe640 1 -- 172.21.15.43:0/2772838255 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4036070 con 0x7fc1ec0d0920 2024-04-25T09:01:27.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.052+0000 7fc1eb7fe640 1 -- 172.21.15.43:0/2772838255 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e4031600 con 0x7fc1ec0d0920 2024-04-25T09:01:27.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.052+0000 7fc1eb7fe640 1 -- 172.21.15.43:0/2772838255 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e40319d0 con 0x7fc1ec0d0920 2024-04-25T09:01:27.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.053+0000 7fc1f2da9640 1 -- 172.21.15.43:0/2772838255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1ec0d0920 msgr2=0x7fc1ec160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:27.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.053+0000 7fc1f2da9640 1 --2- 172.21.15.43:0/2772838255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1ec0d0920 0x7fc1ec160980 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4009e50 tx=0x7fc1e402a680 comp rx=0 tx=0).stop 2024-04-25T09:01:27.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.053+0000 7fc1f2da9640 1 -- 172.21.15.43:0/2772838255 shutdown_connections 2024-04-25T09:01:27.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.053+0000 7fc1f2da9640 1 --2- 172.21.15.43:0/2772838255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1ec0d0920 0x7fc1ec160980 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:27.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.053+0000 7fc1f2da9640 1 --2- 172.21.15.43:0/2772838255 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc1ec15ef50 0x7fc1ec0d03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:27.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.053+0000 7fc1f2da9640 1 -- 172.21.15.43:0/2772838255 >> 172.21.15.43:0/2772838255 conn(0x7fc1ec09cd00 msgr2=0x7fc1ec0d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:27.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.053+0000 7fc1f2da9640 1 -- 172.21.15.43:0/2772838255 shutdown_connections 2024-04-25T09:01:27.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.053+0000 7fc1f2da9640 1 -- 172.21.15.43:0/2772838255 wait complete. 2024-04-25T09:01:27.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.054+0000 7fc1f2da9640 1 Processor -- start 2024-04-25T09:01:27.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.054+0000 7fc1f2da9640 1 -- start start 2024-04-25T09:01:27.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.055+0000 7fc1f2da9640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc1ec0d0920 0x7fc1ec09ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:27.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.055+0000 7fc1f2da9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1ec15dce0 0x7fc1ec1f4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:27.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.055+0000 7fc1f2da9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1ec15ef50 0x7fc1ec155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:27.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.055+0000 7fc1f2da9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ec0cf720 con 0x7fc1ec15dce0 2024-04-25T09:01:27.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.055+0000 7fc1f2da9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ec0cf440 con 0x7fc1ec15ef50 2024-04-25T09:01:27.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.055+0000 7fc1f2da9640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ec15f6b0 con 0x7fc1ec0d0920 2024-04-25T09:01:27.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.055+0000 7fc1ebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1ec15dce0 0x7fc1ec1f4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:27.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.055+0000 7fc1ebfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1ec15dce0 0x7fc1ec1f4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58254/0 (socket says 172.21.15.43:58254) 2024-04-25T09:01:27.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.055+0000 7fc1ebfff640 1 -- 172.21.15.43:0/1615616724 learned_addr learned my addr 172.21.15.43:0/1615616724 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:27.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.055+0000 7fc1f0b45640 1 --2- 172.21.15.43:0/1615616724 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc1ec0d0920 0x7fc1ec09ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:27.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.055+0000 7fc1f1346640 1 --2- 172.21.15.43:0/1615616724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1ec15ef50 0x7fc1ec155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:27.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.055+0000 7fc1ebfff640 1 -- 172.21.15.43:0/1615616724 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc1ec0d0920 msgr2=0x7fc1ec09ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:27.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.055+0000 7fc1ebfff640 1 --2- 172.21.15.43:0/1615616724 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc1ec0d0920 0x7fc1ec09ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:27.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.055+0000 7fc1ebfff640 1 -- 172.21.15.43:0/1615616724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1ec15ef50 msgr2=0x7fc1ec155e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:27.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.055+0000 7fc1ebfff640 1 --2- 172.21.15.43:0/1615616724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1ec15ef50 0x7fc1ec155e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:27.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.055+0000 7fc1ebfff640 1 -- 172.21.15.43:0/1615616724 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1ec09f2c0 con 0x7fc1ec15dce0 2024-04-25T09:01:27.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.055+0000 7fc1ebfff640 1 --2- 172.21.15.43:0/1615616724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1ec15dce0 0x7fc1ec1f4870 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fc1d800e9b0 tx=0x7fc1d800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:27.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.055+0000 7fc1e97fa640 1 -- 172.21.15.43:0/1615616724 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d8017070 con 0x7fc1ec15dce0 2024-04-25T09:01:27.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.056+0000 7fc1e97fa640 1 -- 172.21.15.43:0/1615616724 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1d800c9a0 con 0x7fc1ec15dce0 2024-04-25T09:01:27.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.056+0000 7fc1f2da9640 1 -- 172.21.15.43:0/1615616724 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1ec1f4db0 con 0x7fc1ec15dce0 2024-04-25T09:01:27.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.056+0000 7fc1e97fa640 1 -- 172.21.15.43:0/1615616724 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d800cd90 con 0x7fc1ec15dce0 2024-04-25T09:01:27.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.056+0000 7fc1f2da9640 1 -- 172.21.15.43:0/1615616724 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1ec09dfc0 con 0x7fc1ec15dce0 2024-04-25T09:01:27.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.057+0000 7fc1f2da9640 1 -- 172.21.15.43:0/1615616724 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1ec0680f0 con 0x7fc1ec15dce0 2024-04-25T09:01:27.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.057+0000 7fc1e97fa640 1 -- 172.21.15.43:0/1615616724 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc1d801a070 con 0x7fc1ec15dce0 2024-04-25T09:01:27.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.058+0000 7fc1e97fa640 1 --2- 172.21.15.43:0/1615616724 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc1d006b4d0 0x7fc1d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:27.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.058+0000 7fc1e97fa640 1 -- 172.21.15.43:0/1615616724 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc1d8089eb0 con 0x7fc1ec15dce0 2024-04-25T09:01:27.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.058+0000 7fc1f0b45640 1 --2- 172.21.15.43:0/1615616724 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc1d006b4d0 0x7fc1d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:27.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.058+0000 7fc1f0b45640 1 --2- 172.21.15.43:0/1615616724 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc1d006b4d0 0x7fc1d006d990 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0001640 tx=0x7fc1e0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:27.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.062+0000 7fc1e97fa640 1 -- 172.21.15.43:0/1615616724 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc1ec1f4db0 con 0x7fc1ec15dce0 2024-04-25T09:01:27.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:27.315+0000 7fc1f2da9640 1 -- 172.21.15.43:0/1615616724 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]} v 0) v1 -- 0x7fc1ec09da20 con 0x7fc1ec15dce0 2024-04-25T09:01:28.305 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.316+0000 7fc1e97fa640 1 -- 172.21.15.43:0/1615616724 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]}]=0 v347) v1 ==== 158+0+0 (secure 0 0 0) 0x7fc1ec09f2c0 con 0x7fc1ec15dce0 2024-04-25T09:01:28.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.320+0000 7fc1f2da9640 1 -- 172.21.15.43:0/1615616724 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc1d006b4d0 msgr2=0x7fc1d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:28.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.320+0000 7fc1f2da9640 1 --2- 172.21.15.43:0/1615616724 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc1d006b4d0 0x7fc1d006d990 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0001640 tx=0x7fc1e0009040 comp rx=0 tx=0).stop 2024-04-25T09:01:28.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.320+0000 7fc1f2da9640 1 -- 172.21.15.43:0/1615616724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1ec15dce0 msgr2=0x7fc1ec1f4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:28.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.320+0000 7fc1f2da9640 1 --2- 172.21.15.43:0/1615616724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1ec15dce0 0x7fc1ec1f4870 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fc1d800e9b0 tx=0x7fc1d800ee70 comp rx=0 tx=0).stop 2024-04-25T09:01:28.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.320+0000 7fc1f2da9640 1 -- 172.21.15.43:0/1615616724 shutdown_connections 2024-04-25T09:01:28.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.320+0000 7fc1f2da9640 1 --2- 172.21.15.43:0/1615616724 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc1d006b4d0 0x7fc1d006d990 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:28.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.320+0000 7fc1f2da9640 1 --2- 172.21.15.43:0/1615616724 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc1ec15ef50 0x7fc1ec155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:28.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.320+0000 7fc1f2da9640 1 --2- 172.21.15.43:0/1615616724 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc1ec15dce0 0x7fc1ec1f4870 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:28.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.320+0000 7fc1f2da9640 1 --2- 172.21.15.43:0/1615616724 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc1ec0d0920 0x7fc1ec09ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:28.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.320+0000 7fc1f2da9640 1 -- 172.21.15.43:0/1615616724 >> 172.21.15.43:0/1615616724 conn(0x7fc1ec09cd00 msgr2=0x7fc1ec0d1910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:28.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.320+0000 7fc1f2da9640 1 -- 172.21.15.43:0/1615616724 shutdown_connections 2024-04-25T09:01:28.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.320+0000 7fc1f2da9640 1 -- 172.21.15.43:0/1615616724 wait complete. 2024-04-25T09:01:28.323 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T09:01:28.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.445+0000 7f38552f5640 1 Processor -- start 2024-04-25T09:01:28.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.445+0000 7f38552f5640 1 -- start start 2024-04-25T09:01:28.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.445+0000 7f38552f5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f385015ddd0 0x7f38500b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:28.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.445+0000 7f38552f5640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f385006ef90 0x7f385006f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:28.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.446+0000 7f38552f5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38500d2620 con 0x7f38500cc460 2024-04-25T09:01:28.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.446+0000 7f38552f5640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f385006f930 con 0x7f385006ef90 2024-04-25T09:01:28.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.446+0000 7f38552f5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38500b0b80 con 0x7f385015ddd0 2024-04-25T09:01:28.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.446+0000 7f384f7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f38500cc460 0x7f38500cc8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48994/0 (socket says 172.21.15.43:48994) 2024-04-25T09:01:28.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.446+0000 7f384f7fe640 1 -- 172.21.15.43:0/269926114 learned_addr learned my addr 172.21.15.43:0/269926114 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:28.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.446+0000 7f384dffb640 1 -- 172.21.15.43:0/269926114 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2286998580 0 0) 0x7f38500d2620 con 0x7f38500cc460 2024-04-25T09:01:28.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.446+0000 7f384dffb640 1 -- 172.21.15.43:0/269926114 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3838003520 con 0x7f38500cc460 2024-04-25T09:01:28.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.446+0000 7f384e7fc640 1 --2- 172.21.15.43:0/269926114 >> v2:172.21.15.193:3300/0 conn(0x7f385006ef90 0x7f385006f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:28.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.446+0000 7f384effd640 1 --2- 172.21.15.43:0/269926114 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f385015ddd0 0x7f38500b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:28.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.446+0000 7f384dffb640 1 -- 172.21.15.43:0/269926114 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1135226828 0 0) 0x7f3838003520 con 0x7f38500cc460 2024-04-25T09:01:28.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.446+0000 7f384dffb640 1 -- 172.21.15.43:0/269926114 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f385015ddd0 msgr2=0x7f38500b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:28.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.446+0000 7f384dffb640 1 --2- 172.21.15.43:0/269926114 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f385015ddd0 0x7f38500b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:28.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.446+0000 7f384dffb640 1 -- 172.21.15.43:0/269926114 >> v2:172.21.15.193:3300/0 conn(0x7f385006ef90 msgr2=0x7f385006f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:28.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.446+0000 7f384dffb640 1 --2- 172.21.15.43:0/269926114 >> v2:172.21.15.193:3300/0 conn(0x7f385006ef90 0x7f385006f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:28.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.446+0000 7f384dffb640 1 -- 172.21.15.43:0/269926114 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38500b0e60 con 0x7f38500cc460 2024-04-25T09:01:28.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.446+0000 7f384dffb640 1 -- 172.21.15.43:0/269926114 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f38480019f0 con 0x7f38500cc460 2024-04-25T09:01:28.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.446+0000 7f384e7fc640 1 --2- 172.21.15.43:0/269926114 >> v2:172.21.15.193:3300/0 conn(0x7f385006ef90 0x7f385006f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:01:28.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.446+0000 7f384dffb640 1 -- 172.21.15.43:0/269926114 >> v1:172.21.15.43:6789/0 conn(0x7f38500cc460 legacy=0x7f38500cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:28.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.446+0000 7f384dffb640 1 --2- 172.21.15.43:0/269926114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38380039b0 0x7f3838003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:28.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.446+0000 7f384dffb640 1 --2- 172.21.15.43:0/269926114 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f385015ddd0 0x7f3838004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:28.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.446+0000 7f384dffb640 1 --2- 172.21.15.43:0/269926114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3838004d40 0x7f38380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:28.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.446+0000 7f384dffb640 1 -- 172.21.15.43:0/269926114 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38500b0b80 con 0x7f3838004d40 2024-04-25T09:01:28.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.446+0000 7f384dffb640 1 -- 172.21.15.43:0/269926114 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f383800b9c0 con 0x7f38380039b0 2024-04-25T09:01:28.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.446+0000 7f384dffb640 1 -- 172.21.15.43:0/269926114 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f383800bb40 con 0x7f385015ddd0 2024-04-25T09:01:28.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.446+0000 7f384f7fe640 1 --2- 172.21.15.43:0/269926114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3838004d40 0x7f38380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:28.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.447+0000 7f384effd640 1 --2- 172.21.15.43:0/269926114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38380039b0 0x7f3838003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:28.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.447+0000 7f384e7fc640 1 --2- 172.21.15.43:0/269926114 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f385015ddd0 0x7f3838004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:28.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.447+0000 7f384f7fe640 1 -- 172.21.15.43:0/269926114 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f385015ddd0 msgr2=0x7f3838004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:28.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.447+0000 7f384f7fe640 1 --2- 172.21.15.43:0/269926114 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f385015ddd0 0x7f3838004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:28.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.447+0000 7f384f7fe640 1 -- 172.21.15.43:0/269926114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38380039b0 msgr2=0x7f3838003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:28.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.447+0000 7f384f7fe640 1 --2- 172.21.15.43:0/269926114 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38380039b0 0x7f3838003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:28.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.447+0000 7f384f7fe640 1 -- 172.21.15.43:0/269926114 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3838003520 con 0x7f3838004d40 2024-04-25T09:01:28.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.447+0000 7f384e7fc640 1 --2- 172.21.15.43:0/269926114 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f385015ddd0 0x7f3838004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:01:28.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.447+0000 7f384f7fe640 1 --2- 172.21.15.43:0/269926114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3838004d40 0x7f38380051c0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f384800a890 tx=0x7f3838005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10bc57de376852fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:28.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.448+0000 7f384dffb640 1 -- 172.21.15.43:0/269926114 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384800a4e0 con 0x7f3838004d40 2024-04-25T09:01:28.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.448+0000 7f384dffb640 1 -- 172.21.15.43:0/269926114 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f384802f070 con 0x7f3838004d40 2024-04-25T09:01:28.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.448+0000 7f384dffb640 1 -- 172.21.15.43:0/269926114 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384802bcf0 con 0x7f3838004d40 2024-04-25T09:01:28.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.448+0000 7f38552f5640 1 -- 172.21.15.43:0/269926114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3838004d40 msgr2=0x7f38380051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:28.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.448+0000 7f38552f5640 1 --2- 172.21.15.43:0/269926114 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3838004d40 0x7f38380051c0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f384800a890 tx=0x7f3838005830 comp rx=0 tx=0).stop 2024-04-25T09:01:28.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.448+0000 7f384f7fe640 1 -- 172.21.15.43:0/269926114 reap_dead start 2024-04-25T09:01:28.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.448+0000 7f38552f5640 1 -- 172.21.15.43:0/269926114 shutdown_connections 2024-04-25T09:01:28.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.448+0000 7f38552f5640 1 -- 172.21.15.43:0/269926114 >> 172.21.15.43:0/269926114 conn(0x7f38500d17e0 msgr2=0x7f38500d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:28.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.448+0000 7f38552f5640 1 -- 172.21.15.43:0/269926114 shutdown_connections 2024-04-25T09:01:28.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.448+0000 7f38552f5640 1 -- 172.21.15.43:0/269926114 wait complete. 2024-04-25T09:01:28.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.450+0000 7f38552f5640 1 Processor -- start 2024-04-25T09:01:28.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.450+0000 7f38552f5640 1 -- start start 2024-04-25T09:01:28.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.450+0000 7f38552f5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3838004d40 0x7f38500cb030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:28.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.450+0000 7f38552f5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3850151590 0x7f38500c8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:28.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.450+0000 7f38552f5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38500c8d60 0x7f38500d03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:28.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.450+0000 7f38552f5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38500b3f50 con 0x7f3850151590 2024-04-25T09:01:28.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.450+0000 7f38552f5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38500b3dd0 con 0x7f38500c8d60 2024-04-25T09:01:28.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.450+0000 7f38552f5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38500b40d0 con 0x7f3838004d40 2024-04-25T09:01:28.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.451+0000 7f384e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3850151590 0x7f38500c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:28.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.451+0000 7f384e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3850151590 0x7f38500c8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58280/0 (socket says 172.21.15.43:58280) 2024-04-25T09:01:28.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.451+0000 7f384e7fc640 1 -- 172.21.15.43:0/1678838403 learned_addr learned my addr 172.21.15.43:0/1678838403 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:28.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.451+0000 7f384f7fe640 1 --2- 172.21.15.43:0/1678838403 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38500c8d60 0x7f38500d03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:28.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.451+0000 7f384effd640 1 --2- 172.21.15.43:0/1678838403 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3838004d40 0x7f38500cb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:28.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.451+0000 7f384e7fc640 1 -- 172.21.15.43:0/1678838403 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3838004d40 msgr2=0x7f38500cb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:28.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.451+0000 7f384e7fc640 1 --2- 172.21.15.43:0/1678838403 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3838004d40 0x7f38500cb030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:28.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.451+0000 7f384e7fc640 1 -- 172.21.15.43:0/1678838403 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38500c8d60 msgr2=0x7f38500d03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:28.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.451+0000 7f384e7fc640 1 --2- 172.21.15.43:0/1678838403 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38500c8d60 0x7f38500d03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:28.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.451+0000 7f384e7fc640 1 -- 172.21.15.43:0/1678838403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38500cfeb0 con 0x7f3850151590 2024-04-25T09:01:28.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.451+0000 7f384effd640 1 --2- 172.21.15.43:0/1678838403 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3838004d40 0x7f38500cb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:01:28.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.451+0000 7f384e7fc640 1 --2- 172.21.15.43:0/1678838403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3850151590 0x7f38500c8820 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f3844010890 tx=0x7f3844010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:28.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.451+0000 7f38377fe640 1 -- 172.21.15.43:0/1678838403 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384401a020 con 0x7f3850151590 2024-04-25T09:01:28.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.451+0000 7f38377fe640 1 -- 172.21.15.43:0/1678838403 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f384400eb60 con 0x7f3850151590 2024-04-25T09:01:28.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.451+0000 7f38552f5640 1 -- 172.21.15.43:0/1678838403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38500b70c0 con 0x7f3850151590 2024-04-25T09:01:28.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.451+0000 7f38377fe640 1 -- 172.21.15.43:0/1678838403 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384400f040 con 0x7f3850151590 2024-04-25T09:01:28.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.451+0000 7f38552f5640 1 -- 172.21.15.43:0/1678838403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3850151a10 con 0x7f3850151590 2024-04-25T09:01:28.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.452+0000 7f38552f5640 1 -- 172.21.15.43:0/1678838403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3850074f60 con 0x7f3850151590 2024-04-25T09:01:28.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.453+0000 7f38377fe640 1 -- 172.21.15.43:0/1678838403 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f384401d020 con 0x7f3850151590 2024-04-25T09:01:28.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.453+0000 7f38377fe640 1 --2- 172.21.15.43:0/1678838403 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f383c06b4d0 0x7f383c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:28.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.453+0000 7f38377fe640 1 -- 172.21.15.43:0/1678838403 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f384408a820 con 0x7f3850151590 2024-04-25T09:01:28.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.453+0000 7f384effd640 1 --2- 172.21.15.43:0/1678838403 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f383c06b4d0 0x7f383c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:28.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.454+0000 7f384effd640 1 --2- 172.21.15.43:0/1678838403 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f383c06b4d0 0x7f383c06d990 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f3840009b40 tx=0x7f3840008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:28.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.457+0000 7f38377fe640 1 -- 172.21.15.43:0/1678838403 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3844024020 con 0x7f3850151590 2024-04-25T09:01:28.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.638+0000 7f38552f5640 1 -- 172.21.15.43:0/1678838403 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f38500ae820 con 0x7f3850151590 2024-04-25T09:01:28.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.641+0000 7f38377fe640 1 -- 172.21.15.43:0/1678838403 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7f38440599a0 con 0x7f3850151590 2024-04-25T09:01:28.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.643+0000 7f38552f5640 1 -- 172.21.15.43:0/1678838403 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f383c06b4d0 msgr2=0x7f383c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:28.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.643+0000 7f38552f5640 1 --2- 172.21.15.43:0/1678838403 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f383c06b4d0 0x7f383c06d990 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f3840009b40 tx=0x7f3840008040 comp rx=0 tx=0).stop 2024-04-25T09:01:28.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.643+0000 7f38552f5640 1 -- 172.21.15.43:0/1678838403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3850151590 msgr2=0x7f38500c8820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:28.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.643+0000 7f38552f5640 1 --2- 172.21.15.43:0/1678838403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3850151590 0x7f38500c8820 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f3844010890 tx=0x7f3844010d50 comp rx=0 tx=0).stop 2024-04-25T09:01:28.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.643+0000 7f38552f5640 1 -- 172.21.15.43:0/1678838403 shutdown_connections 2024-04-25T09:01:28.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.643+0000 7f38552f5640 1 --2- 172.21.15.43:0/1678838403 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f383c06b4d0 0x7f383c06d990 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:28.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.643+0000 7f38552f5640 1 --2- 172.21.15.43:0/1678838403 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f38500c8d60 0x7f38500d03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:28.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.643+0000 7f38552f5640 1 --2- 172.21.15.43:0/1678838403 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3850151590 0x7f38500c8820 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:28.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.643+0000 7f38552f5640 1 --2- 172.21.15.43:0/1678838403 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3838004d40 0x7f38500cb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:28.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.643+0000 7f38552f5640 1 -- 172.21.15.43:0/1678838403 >> 172.21.15.43:0/1678838403 conn(0x7f38500d17e0 msgr2=0x7f38500afc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:28.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.643+0000 7f38552f5640 1 -- 172.21.15.43:0/1678838403 shutdown_connections 2024-04-25T09:01:28.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.643+0000 7f38552f5640 1 -- 172.21.15.43:0/1678838403 wait complete. 2024-04-25T09:01:28.644 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:01:28.644 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:01:28.690 DEBUG:teuthology.orchestra.run.smithi193:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:01:28.730 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:01:28.731 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T09:01:28.731 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:01:28.746 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2374 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:01:28.747 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T09:01:28.747 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:01:28.788 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19508 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:01:28.788 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T09:01:28.788 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:01:28.804 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2374 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:01:28.804 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T09:01:28.804 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:01:28.845 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19508 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:01:28.845 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T09:01:28.845 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:01:28.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.957+0000 7fbc73c38640 1 Processor -- start 2024-04-25T09:01:28.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.957+0000 7fbc73c38640 1 -- start start 2024-04-25T09:01:28.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.958+0000 7fbc73c38640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fbc6c0715a0 0x7fbc6c071a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:28.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.958+0000 7fbc73c38640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc6c0b3100 0x7fbc6c0b3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:28.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.958+0000 7fbc73c38640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc6c06dd30 con 0x7fbc6c15a250 2024-04-25T09:01:28.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.958+0000 7fbc73c38640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc6c06e010 con 0x7fbc6c0715a0 2024-04-25T09:01:28.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.958+0000 7fbc73c38640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6c0b2900 con 0x7fbc6c0b3100 2024-04-25T09:01:28.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.959+0000 7fbc721d5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc6c0b3100 0x7fbc6c0b3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:28.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.959+0000 7fbc711d3640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fbc6c0715a0 0x7fbc6c071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:28.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.959+0000 7fbc719d4640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7fbc6c15a250 0x7fbc6c070d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:57372/0 (socket says 172.21.15.43:57372) 2024-04-25T09:01:28.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.959+0000 7fbc719d4640 1 -- 172.21.15.43:0/3591829357 learned_addr learned my addr 172.21.15.43:0/3591829357 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:28.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.959+0000 7fbc721d5640 1 -- 172.21.15.43:0/3591829357 >> v2:172.21.15.193:3300/0 conn(0x7fbc6c0715a0 msgr2=0x7fbc6c071a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:28.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.959+0000 7fbc721d5640 1 --2- 172.21.15.43:0/3591829357 >> v2:172.21.15.193:3300/0 conn(0x7fbc6c0715a0 0x7fbc6c071a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:28.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.959+0000 7fbc721d5640 1 -- 172.21.15.43:0/3591829357 >> v1:172.21.15.193:6789/0 conn(0x7fbc6c15a250 legacy=0x7fbc6c070d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:28.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.959+0000 7fbc721d5640 1 -- 172.21.15.43:0/3591829357 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc6c0b2be0 con 0x7fbc6c0b3100 2024-04-25T09:01:28.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.959+0000 7fbc709d2640 1 -- 172.21.15.43:0/3591829357 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2811451145 0 0) 0x7fbc6c06dd30 con 0x7fbc6c15a250 2024-04-25T09:01:28.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.959+0000 7fbc711d3640 1 --2- 172.21.15.43:0/3591829357 >> v2:172.21.15.193:3300/0 conn(0x7fbc6c0715a0 0x7fbc6c071a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:01:28.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.960+0000 7fbc721d5640 1 --2- 172.21.15.43:0/3591829357 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc6c0b3100 0x7fbc6c0b3510 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c009d90 tx=0x7fbc5c02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81c531b0eb6fff2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:28.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.960+0000 7fbc709d2640 1 -- 172.21.15.43:0/3591829357 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c007740 con 0x7fbc6c0b3100 2024-04-25T09:01:28.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.960+0000 7fbc709d2640 1 -- 172.21.15.43:0/3591829357 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc5c0078e0 con 0x7fbc6c0b3100 2024-04-25T09:01:28.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.960+0000 7fbc709d2640 1 -- 172.21.15.43:0/3591829357 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c007cb0 con 0x7fbc6c0b3100 2024-04-25T09:01:28.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.960+0000 7fbc73c38640 1 -- 172.21.15.43:0/3591829357 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc6c0b3100 msgr2=0x7fbc6c0b3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:28.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.960+0000 7fbc73c38640 1 --2- 172.21.15.43:0/3591829357 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc6c0b3100 0x7fbc6c0b3510 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c009d90 tx=0x7fbc5c02a660 comp rx=0 tx=0).stop 2024-04-25T09:01:28.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.961+0000 7fbc73c38640 1 -- 172.21.15.43:0/3591829357 shutdown_connections 2024-04-25T09:01:28.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.961+0000 7fbc73c38640 1 --2- 172.21.15.43:0/3591829357 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc6c0b3100 0x7fbc6c0b3510 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:28.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.961+0000 7fbc73c38640 1 --2- 172.21.15.43:0/3591829357 >> v2:172.21.15.193:3300/0 conn(0x7fbc6c0715a0 0x7fbc6c071a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:28.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.961+0000 7fbc73c38640 1 -- 172.21.15.43:0/3591829357 >> 172.21.15.43:0/3591829357 conn(0x7fbc6c0b1480 msgr2=0x7fbc6c0b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:28.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.961+0000 7fbc73c38640 1 -- 172.21.15.43:0/3591829357 shutdown_connections 2024-04-25T09:01:28.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.961+0000 7fbc73c38640 1 -- 172.21.15.43:0/3591829357 wait complete. 2024-04-25T09:01:28.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.962+0000 7fbc73c38640 1 Processor -- start 2024-04-25T09:01:28.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.962+0000 7fbc73c38640 1 -- start start 2024-04-25T09:01:28.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.963+0000 7fbc73c38640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc6c0715a0 0x7fbc6c146a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:28.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.963+0000 7fbc73c38640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc6c0b3100 0x7fbc6c14dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:28.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.963+0000 7fbc73c38640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc6c15a250 0x7fbc6c1461b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:28.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.963+0000 7fbc73c38640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6c06d410 con 0x7fbc6c15a250 2024-04-25T09:01:28.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.963+0000 7fbc73c38640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6c06d290 con 0x7fbc6c0715a0 2024-04-25T09:01:28.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.963+0000 7fbc73c38640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6c06d590 con 0x7fbc6c0b3100 2024-04-25T09:01:28.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.963+0000 7fbc721d5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc6c15a250 0x7fbc6c1461b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:28.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.963+0000 7fbc721d5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc6c15a250 0x7fbc6c1461b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58282/0 (socket says 172.21.15.43:58282) 2024-04-25T09:01:28.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.963+0000 7fbc721d5640 1 -- 172.21.15.43:0/116723006 learned_addr learned my addr 172.21.15.43:0/116723006 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:28.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.963+0000 7fbc719d4640 1 --2- 172.21.15.43:0/116723006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc6c0715a0 0x7fbc6c146a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:28.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.963+0000 7fbc711d3640 1 --2- 172.21.15.43:0/116723006 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc6c0b3100 0x7fbc6c14dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:28.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.963+0000 7fbc721d5640 1 -- 172.21.15.43:0/116723006 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc6c0b3100 msgr2=0x7fbc6c14dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:28.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.963+0000 7fbc721d5640 1 --2- 172.21.15.43:0/116723006 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc6c0b3100 0x7fbc6c14dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:28.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.963+0000 7fbc721d5640 1 -- 172.21.15.43:0/116723006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc6c0715a0 msgr2=0x7fbc6c146a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:28.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.963+0000 7fbc721d5640 1 --2- 172.21.15.43:0/116723006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc6c0715a0 0x7fbc6c146a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:28.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.963+0000 7fbc721d5640 1 -- 172.21.15.43:0/116723006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc6c147140 con 0x7fbc6c15a250 2024-04-25T09:01:28.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.963+0000 7fbc721d5640 1 --2- 172.21.15.43:0/116723006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc6c15a250 0x7fbc6c1461b0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c02ac50 tx=0x7fbc5c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:28.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.963+0000 7fbc6a7fc640 1 -- 172.21.15.43:0/116723006 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c040070 con 0x7fbc6c15a250 2024-04-25T09:01:28.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.963+0000 7fbc6a7fc640 1 -- 172.21.15.43:0/116723006 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc5c031bc0 con 0x7fbc6c15a250 2024-04-25T09:01:28.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.963+0000 7fbc73c38640 1 -- 172.21.15.43:0/116723006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc6c147420 con 0x7fbc6c15a250 2024-04-25T09:01:28.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.964+0000 7fbc6a7fc640 1 -- 172.21.15.43:0/116723006 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc5c009450 con 0x7fbc6c15a250 2024-04-25T09:01:28.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.964+0000 7fbc73c38640 1 -- 172.21.15.43:0/116723006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc6c1479e0 con 0x7fbc6c15a250 2024-04-25T09:01:28.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.965+0000 7fbc73c38640 1 -- 172.21.15.43:0/116723006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc6c074ef0 con 0x7fbc6c15a250 2024-04-25T09:01:28.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.965+0000 7fbc6a7fc640 1 -- 172.21.15.43:0/116723006 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbc5c039040 con 0x7fbc6c15a250 2024-04-25T09:01:28.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.965+0000 7fbc6a7fc640 1 --2- 172.21.15.43:0/116723006 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbc5806b4d0 0x7fbc5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:28.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.966+0000 7fbc6a7fc640 1 -- 172.21.15.43:0/116723006 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbc5c0ab610 con 0x7fbc6c15a250 2024-04-25T09:01:28.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.966+0000 7fbc719d4640 1 --2- 172.21.15.43:0/116723006 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbc5806b4d0 0x7fbc5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:28.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.966+0000 7fbc719d4640 1 --2- 172.21.15.43:0/116723006 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbc5806b4d0 0x7fbc5806d990 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fbc64007fd0 tx=0x7fbc64007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:28.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:28.970+0000 7fbc6a7fc640 1 -- 172.21.15.43:0/116723006 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbc5c07a260 con 0x7fbc6c15a250 2024-04-25T09:01:29.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.190+0000 7fbc73c38640 1 -- 172.21.15.43:0/116723006 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbc6c0ae8e0 con 0x7fbc6c15a250 2024-04-25T09:01:29.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.191+0000 7fbc6a7fc640 1 -- 172.21.15.43:0/116723006 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+3702 (secure 0 0 0) 0x7fbc5c07e300 con 0x7fbc6c15a250 2024-04-25T09:01:29.180 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:01:29.180 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":56,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6200,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2258031526","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2258031526},{"type":"v1","addr":"172.21.15.43:6834","nonce":2258031526}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6206,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/1773440177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":1773440177},{"type":"v1","addr":"172.21.15.193:6836","nonce":1773440177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":6209,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3239725307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":3239725307},{"type":"v1","addr":"172.21.15.193:6837","nonce":3239725307}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":8007,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/845142514","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":845142514},{"type":"v1","addr":"172.21.15.43:6835","nonce":845142514}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[]} 2024-04-25T09:01:29.180 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 56 2024-04-25T09:01:29.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.193+0000 7fbc73c38640 1 -- 172.21.15.43:0/116723006 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbc5806b4d0 msgr2=0x7fbc5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:29.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.193+0000 7fbc73c38640 1 --2- 172.21.15.43:0/116723006 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbc5806b4d0 0x7fbc5806d990 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fbc64007fd0 tx=0x7fbc64007de0 comp rx=0 tx=0).stop 2024-04-25T09:01:29.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.193+0000 7fbc73c38640 1 -- 172.21.15.43:0/116723006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc6c15a250 msgr2=0x7fbc6c1461b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:29.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.193+0000 7fbc73c38640 1 --2- 172.21.15.43:0/116723006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc6c15a250 0x7fbc6c1461b0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c02ac50 tx=0x7fbc5c004a40 comp rx=0 tx=0).stop 2024-04-25T09:01:29.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.193+0000 7fbc73c38640 1 -- 172.21.15.43:0/116723006 shutdown_connections 2024-04-25T09:01:29.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.193+0000 7fbc73c38640 1 --2- 172.21.15.43:0/116723006 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbc6c15a250 0x7fbc6c1461b0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:29.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.193+0000 7fbc73c38640 1 --2- 172.21.15.43:0/116723006 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbc6c0b3100 0x7fbc6c14dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:29.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.193+0000 7fbc73c38640 1 --2- 172.21.15.43:0/116723006 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbc5806b4d0 0x7fbc5806d990 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:29.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.193+0000 7fbc73c38640 1 --2- 172.21.15.43:0/116723006 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbc6c0715a0 0x7fbc6c146a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:29.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.193+0000 7fbc73c38640 1 -- 172.21.15.43:0/116723006 >> 172.21.15.43:0/116723006 conn(0x7fbc6c0b1480 msgr2=0x7fbc6c0cd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:29.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.193+0000 7fbc73c38640 1 -- 172.21.15.43:0/116723006 shutdown_connections 2024-04-25T09:01:29.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.193+0000 7fbc73c38640 1 -- 172.21.15.43:0/116723006 wait complete. 2024-04-25T09:01:29.193 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:01:29.193 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T09:01:29.193 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:01:29.193 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:01:29.194 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:01:29.194 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T09:01:29.194 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:01:29.194 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:01:29.194 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:01:29.194 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T09:01:29.194 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:01:29.194 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:01:29.195 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:01:29.195 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T09:01:29.195 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:01:29.195 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:01:29.203 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:01:29.203 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:01:29.204 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:01:29.204 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:01:29.210 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:01:29.210 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:01:29.211 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:01:29.211 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:01:29.247 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:01:29.248 DEBUG:teuthology.parallel:result is None 2024-04-25T09:01:29.248 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:01:29.248 DEBUG:teuthology.parallel:result is None 2024-04-25T09:01:29.254 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:01:29.254 DEBUG:teuthology.parallel:result is None 2024-04-25T09:01:29.254 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:01:29.255 DEBUG:teuthology.parallel:result is None 2024-04-25T09:01:29.255 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T09:01:29.324 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-04-25T09:01:29.325 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2024-04-25T09:01:29.335 INFO:tasks.ceph.mds.b.smithi193.stdout:starting mds.b at 2024-04-25T09:01:29.336 INFO:tasks.ceph.mds.d.smithi193.stdout:starting mds.d at 2024-04-25T09:01:29.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.373+0000 7f09bc93b640 1 Processor -- start 2024-04-25T09:01:29.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.373+0000 7f09bc93b640 1 -- start start 2024-04-25T09:01:29.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.373+0000 7f09bc93b640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f09b80c8d00 0x7f09b8162100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:29.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.373+0000 7f09bc93b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f09b80ce660 0x7f09b80cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:29.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.373+0000 7f09b659c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f09b815df50 0x7f09b81619f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49020/0 (socket says 172.21.15.43:49020) 2024-04-25T09:01:29.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.373+0000 7f09b659c640 1 -- 172.21.15.43:0/2032349887 learned_addr learned my addr 172.21.15.43:0/2032349887 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:29.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.373+0000 7f09bc93b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09b80d24a0 con 0x7f09b815df50 2024-04-25T09:01:29.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.374+0000 7f09bc93b640 1 -- 172.21.15.43:0/2032349887 --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09b815e330 con 0x7f09b80ce660 2024-04-25T09:01:29.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.374+0000 7f09bc93b640 1 -- 172.21.15.43:0/2032349887 --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09b80d2740 con 0x7f09b80c8d00 2024-04-25T09:01:29.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.374+0000 7f09b6d9d640 1 --2- 172.21.15.43:0/2032349887 >> v2:172.21.15.43:3300/0 conn(0x7f09b80ce660 0x7f09b80cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:29.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.374+0000 7f09affff640 1 --2- 172.21.15.43:0/2032349887 >> v2:172.21.15.193:3300/0 conn(0x7f09b80c8d00 0x7f09b8162100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:29.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.374+0000 7f09b6d9d640 1 -- 172.21.15.43:0/2032349887 >> v2:172.21.15.193:3300/0 conn(0x7f09b80c8d00 msgr2=0x7f09b8162100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:29.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.374+0000 7f09b6d9d640 1 --2- 172.21.15.43:0/2032349887 >> v2:172.21.15.193:3300/0 conn(0x7f09b80c8d00 0x7f09b8162100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:29.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.374+0000 7f09b6d9d640 1 -- 172.21.15.43:0/2032349887 >> v1:172.21.15.43:6789/0 conn(0x7f09b815df50 legacy=0x7f09b81619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:29.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.374+0000 7f09b6d9d640 1 -- 172.21.15.43:0/2032349887 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b80d2a20 con 0x7f09b80ce660 2024-04-25T09:01:29.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.375+0000 7f09b6d9d640 1 --2- 172.21.15.43:0/2032349887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09b80ce660 0x7f09b80cea70 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f09b80d45e0 tx=0x7f09b002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dee3cec500c6b48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:29.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.375+0000 7f09b5d9b640 1 -- 172.21.15.43:0/2032349887 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b0007740 con 0x7f09b80ce660 2024-04-25T09:01:29.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.375+0000 7f09b5d9b640 1 -- 172.21.15.43:0/2032349887 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b00078e0 con 0x7f09b80ce660 2024-04-25T09:01:29.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.375+0000 7f09b5d9b640 1 -- 172.21.15.43:0/2032349887 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b0007cb0 con 0x7f09b80ce660 2024-04-25T09:01:29.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.375+0000 7f09bc93b640 1 -- 172.21.15.43:0/2032349887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09b80ce660 msgr2=0x7f09b80cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:29.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.375+0000 7f09bc93b640 1 --2- 172.21.15.43:0/2032349887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09b80ce660 0x7f09b80cea70 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f09b80d45e0 tx=0x7f09b002a680 comp rx=0 tx=0).stop 2024-04-25T09:01:29.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.375+0000 7f09bc93b640 1 -- 172.21.15.43:0/2032349887 shutdown_connections 2024-04-25T09:01:29.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.375+0000 7f09bc93b640 1 --2- 172.21.15.43:0/2032349887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09b80ce660 0x7f09b80cea70 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:29.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.375+0000 7f09bc93b640 1 --2- 172.21.15.43:0/2032349887 >> v2:172.21.15.193:3300/0 conn(0x7f09b80c8d00 0x7f09b8162100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:29.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.375+0000 7f09bc93b640 1 -- 172.21.15.43:0/2032349887 >> 172.21.15.43:0/2032349887 conn(0x7f09b80d3a30 msgr2=0x7f09b80d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:29.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.375+0000 7f09bc93b640 1 -- 172.21.15.43:0/2032349887 shutdown_connections 2024-04-25T09:01:29.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.375+0000 7f09bc93b640 1 -- 172.21.15.43:0/2032349887 wait complete. 2024-04-25T09:01:29.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.377+0000 7f09bc93b640 1 Processor -- start 2024-04-25T09:01:29.366 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.377+0000 7f09bc93b640 1 -- start start 2024-04-25T09:01:29.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.377+0000 7f09bc93b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09b80c8d00 0x7f09b8150240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:29.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.377+0000 7f09bc93b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f09b80ce660 0x7f09b806be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:29.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.377+0000 7f09bc93b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09b815df50 0x7f09b806c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:29.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.377+0000 7f09bc93b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b80d2190 con 0x7f09b80c8d00 2024-04-25T09:01:29.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.377+0000 7f09bc93b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b80c77d0 con 0x7f09b815df50 2024-04-25T09:01:29.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.377+0000 7f09bc93b640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b8162640 con 0x7f09b80ce660 2024-04-25T09:01:29.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.377+0000 7f09b659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09b80c8d00 0x7f09b8150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:29.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.378+0000 7f09b659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09b80c8d00 0x7f09b8150240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58334/0 (socket says 172.21.15.43:58334) 2024-04-25T09:01:29.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.378+0000 7f09b659c640 1 -- 172.21.15.43:0/3582998585 learned_addr learned my addr 172.21.15.43:0/3582998585 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:29.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.378+0000 7f09affff640 1 --2- 172.21.15.43:0/3582998585 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f09b80ce660 0x7f09b806be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:29.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.378+0000 7f09b6d9d640 1 --2- 172.21.15.43:0/3582998585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09b815df50 0x7f09b806c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:29.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.378+0000 7f09affff640 1 -- 172.21.15.43:0/3582998585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09b815df50 msgr2=0x7f09b806c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:29.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.378+0000 7f09affff640 1 --2- 172.21.15.43:0/3582998585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09b815df50 0x7f09b806c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:29.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.378+0000 7f09affff640 1 -- 172.21.15.43:0/3582998585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09b80c8d00 msgr2=0x7f09b8150240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:29.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.378+0000 7f09affff640 1 --2- 172.21.15.43:0/3582998585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09b80c8d00 0x7f09b8150240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:29.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.378+0000 7f09affff640 1 -- 172.21.15.43:0/3582998585 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b8158030 con 0x7f09b80ce660 2024-04-25T09:01:29.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.378+0000 7f09b659c640 1 --2- 172.21.15.43:0/3582998585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09b80c8d00 0x7f09b8150240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:01:29.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.378+0000 7f09affff640 1 --2- 172.21.15.43:0/3582998585 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f09b80ce660 0x7f09b806be30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f09a000d5f0 tx=0x7f09a000dab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:29.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.378+0000 7f09aeffd640 1 -- 172.21.15.43:0/3582998585 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a00125f0 con 0x7f09b80ce660 2024-04-25T09:01:29.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.378+0000 7f09aeffd640 1 -- 172.21.15.43:0/3582998585 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a0019070 con 0x7f09b80ce660 2024-04-25T09:01:29.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.378+0000 7f09bc93b640 1 -- 172.21.15.43:0/3582998585 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09b8158310 con 0x7f09b80ce660 2024-04-25T09:01:29.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.378+0000 7f09aeffd640 1 -- 172.21.15.43:0/3582998585 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a00129d0 con 0x7f09b80ce660 2024-04-25T09:01:29.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.378+0000 7f09bc93b640 1 -- 172.21.15.43:0/3582998585 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09b81588d0 con 0x7f09b80ce660 2024-04-25T09:01:29.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.379+0000 7f09bc93b640 1 -- 172.21.15.43:0/3582998585 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09b80680f0 con 0x7f09b80ce660 2024-04-25T09:01:29.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.380+0000 7f09aeffd640 1 -- 172.21.15.43:0/3582998585 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f09a0012b70 con 0x7f09b80ce660 2024-04-25T09:01:29.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.380+0000 7f09aeffd640 1 --2- 172.21.15.43:0/3582998585 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f099806b4d0 0x7f099806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:29.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.380+0000 7f09aeffd640 1 -- 172.21.15.43:0/3582998585 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f09a008a280 con 0x7f09b80ce660 2024-04-25T09:01:29.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.380+0000 7f09b659c640 1 --2- 172.21.15.43:0/3582998585 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f099806b4d0 0x7f099806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:29.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.381+0000 7f09b659c640 1 --2- 172.21.15.43:0/3582998585 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f099806b4d0 0x7f099806d990 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f09a8000f60 tx=0x7f09a8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:29.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.385+0000 7f09aeffd640 1 -- 172.21.15.43:0/3582998585 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f09a0058cc0 con 0x7f09b80ce660 2024-04-25T09:01:29.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:29.577+0000 7f09bc93b640 1 -- 172.21.15.43:0/3582998585 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f09b80672d0 con 0x7f09b80ce660 2024-04-25T09:01:30.326 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.336+0000 7f09aeffd640 1 -- 172.21.15.43:0/3582998585 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v96) v1 ==== 97+0+0 (secure 0 0 0) 0x7f09a005cd60 con 0x7f09b80ce660 2024-04-25T09:01:30.326 INFO:teuthology.orchestra.run.smithi043.stderr: removed all blocklist entries 2024-04-25T09:01:30.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.339+0000 7f09bc93b640 1 -- 172.21.15.43:0/3582998585 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f099806b4d0 msgr2=0x7f099806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:30.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.339+0000 7f09bc93b640 1 --2- 172.21.15.43:0/3582998585 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f099806b4d0 0x7f099806d990 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f09a8000f60 tx=0x7f09a8009040 comp rx=0 tx=0).stop 2024-04-25T09:01:30.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.339+0000 7f09bc93b640 1 -- 172.21.15.43:0/3582998585 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f09b80ce660 msgr2=0x7f09b806be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:30.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.339+0000 7f09bc93b640 1 --2- 172.21.15.43:0/3582998585 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f09b80ce660 0x7f09b806be30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f09a000d5f0 tx=0x7f09a000dab0 comp rx=0 tx=0).stop 2024-04-25T09:01:30.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.340+0000 7f09bc93b640 1 -- 172.21.15.43:0/3582998585 shutdown_connections 2024-04-25T09:01:30.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.340+0000 7f09bc93b640 1 --2- 172.21.15.43:0/3582998585 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f099806b4d0 0x7f099806d990 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:30.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.340+0000 7f09bc93b640 1 --2- 172.21.15.43:0/3582998585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f09b815df50 0x7f09b806c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:30.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.340+0000 7f09bc93b640 1 --2- 172.21.15.43:0/3582998585 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f09b80ce660 0x7f09b806be30 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:30.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.340+0000 7f09bc93b640 1 --2- 172.21.15.43:0/3582998585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f09b80c8d00 0x7f09b8150240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:30.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.340+0000 7f09bc93b640 1 -- 172.21.15.43:0/3582998585 >> 172.21.15.43:0/3582998585 conn(0x7f09b80d3a30 msgr2=0x7f09b80cf4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:30.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.340+0000 7f09bc93b640 1 -- 172.21.15.43:0/3582998585 shutdown_connections 2024-04-25T09:01:30.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.340+0000 7f09bc93b640 1 -- 172.21.15.43:0/3582998585 wait complete. 2024-04-25T09:01:30.344 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T09:01:30.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.462+0000 7f368867c640 1 Processor -- start 2024-04-25T09:01:30.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.462+0000 7f368867c640 1 -- start start 2024-04-25T09:01:30.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.463+0000 7f368867c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f368015df50 0x7f36801619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:30.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.463+0000 7f368867c640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f3680161f30 0x7f3680162370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:30.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.463+0000 7f368867c640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36800d24a0 con 0x7f36800c7cf0 2024-04-25T09:01:30.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.463+0000 7f368867c640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f368015e330 con 0x7f3680161f30 2024-04-25T09:01:30.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.463+0000 7f368867c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36800d2740 con 0x7f368015df50 2024-04-25T09:01:30.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.464+0000 7f3686418640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f368015df50 0x7f36801619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:30.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.464+0000 7f3685c17640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f3680161f30 0x7f3680162370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:30.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.464+0000 7f3686c19640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f36800c7cf0 0x7f36800c8100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:55166/0 (socket says 172.21.15.43:55166) 2024-04-25T09:01:30.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.464+0000 7f3686c19640 1 -- 172.21.15.43:0/4294868986 learned_addr learned my addr 172.21.15.43:0/4294868986 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:30.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.464+0000 7f3686418640 1 -- 172.21.15.43:0/4294868986 >> v2:172.21.15.193:3300/0 conn(0x7f3680161f30 msgr2=0x7f3680162370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:30.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.464+0000 7f3686418640 1 --2- 172.21.15.43:0/4294868986 >> v2:172.21.15.193:3300/0 conn(0x7f3680161f30 0x7f3680162370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:30.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.464+0000 7f3685416640 1 -- 172.21.15.43:0/4294868986 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2409276496 0 0) 0x7f36800d24a0 con 0x7f36800c7cf0 2024-04-25T09:01:30.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.464+0000 7f3686418640 1 -- 172.21.15.43:0/4294868986 >> v1:172.21.15.193:6789/0 conn(0x7f36800c7cf0 legacy=0x7f36800c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:30.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.464+0000 7f3686418640 1 -- 172.21.15.43:0/4294868986 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36800d2a20 con 0x7f368015df50 2024-04-25T09:01:30.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.465+0000 7f3686418640 1 --2- 172.21.15.43:0/4294868986 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f368015df50 0x7f36801619f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f367c0097a0 tx=0x7f367c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea399bf62f79b980 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:30.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.465+0000 7f3685416640 1 -- 172.21.15.43:0/4294868986 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367c037070 con 0x7f368015df50 2024-04-25T09:01:30.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.465+0000 7f3685416640 1 -- 172.21.15.43:0/4294868986 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f367c032440 con 0x7f368015df50 2024-04-25T09:01:30.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.465+0000 7f3685416640 1 -- 172.21.15.43:0/4294868986 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367c032810 con 0x7f368015df50 2024-04-25T09:01:30.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.466+0000 7f368867c640 1 -- 172.21.15.43:0/4294868986 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f368015df50 msgr2=0x7f36801619f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:30.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.466+0000 7f368867c640 1 --2- 172.21.15.43:0/4294868986 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f368015df50 0x7f36801619f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f367c0097a0 tx=0x7f367c02a180 comp rx=0 tx=0).stop 2024-04-25T09:01:30.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.466+0000 7f368867c640 1 -- 172.21.15.43:0/4294868986 shutdown_connections 2024-04-25T09:01:30.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.466+0000 7f368867c640 1 --2- 172.21.15.43:0/4294868986 >> v2:172.21.15.193:3300/0 conn(0x7f3680161f30 0x7f3680162370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:30.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.466+0000 7f368867c640 1 --2- 172.21.15.43:0/4294868986 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f368015df50 0x7f36801619f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:30.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.466+0000 7f368867c640 1 -- 172.21.15.43:0/4294868986 >> 172.21.15.43:0/4294868986 conn(0x7f36800d3a30 msgr2=0x7f36800d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:30.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.466+0000 7f368867c640 1 -- 172.21.15.43:0/4294868986 shutdown_connections 2024-04-25T09:01:30.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.466+0000 7f368867c640 1 -- 172.21.15.43:0/4294868986 wait complete. 2024-04-25T09:01:30.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.467+0000 7f368867c640 1 Processor -- start 2024-04-25T09:01:30.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.467+0000 7f368867c640 1 -- start start 2024-04-25T09:01:30.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.468+0000 7f368867c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36800c7cf0 0x7f36801f5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:30.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.468+0000 7f368867c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f368015df50 0x7f368014adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:30.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.468+0000 7f368867c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3680161f30 0x7f3680155250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:30.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.468+0000 7f368867c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36800d2190 con 0x7f36800c7cf0 2024-04-25T09:01:30.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.468+0000 7f368867c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36800c77d0 con 0x7f3680161f30 2024-04-25T09:01:30.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.468+0000 7f368867c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36800cf670 con 0x7f368015df50 2024-04-25T09:01:30.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.468+0000 7f3686418640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36800c7cf0 0x7f36801f5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:30.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.468+0000 7f3686418640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36800c7cf0 0x7f36801f5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48776/0 (socket says 172.21.15.43:48776) 2024-04-25T09:01:30.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.468+0000 7f3686418640 1 -- 172.21.15.43:0/4143707921 learned_addr learned my addr 172.21.15.43:0/4143707921 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:30.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.468+0000 7f3685c17640 1 --2- 172.21.15.43:0/4143707921 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f368015df50 0x7f368014adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:30.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.468+0000 7f3686c19640 1 --2- 172.21.15.43:0/4143707921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3680161f30 0x7f3680155250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:30.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.468+0000 7f3686418640 1 -- 172.21.15.43:0/4143707921 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f368015df50 msgr2=0x7f368014adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:30.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.468+0000 7f3686418640 1 --2- 172.21.15.43:0/4143707921 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f368015df50 0x7f368014adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:30.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.468+0000 7f3686418640 1 -- 172.21.15.43:0/4143707921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3680161f30 msgr2=0x7f3680155250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:30.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.468+0000 7f3686418640 1 --2- 172.21.15.43:0/4143707921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3680161f30 0x7f3680155250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:30.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.468+0000 7f3686418640 1 -- 172.21.15.43:0/4143707921 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f368009f9c0 con 0x7f36800c7cf0 2024-04-25T09:01:30.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.468+0000 7f3686418640 1 --2- 172.21.15.43:0/4143707921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36800c7cf0 0x7f36801f5740 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f367c0097a0 tx=0x7f367c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:30.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.468+0000 7f3676ffd640 1 -- 172.21.15.43:0/4143707921 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367c037050 con 0x7f36800c7cf0 2024-04-25T09:01:30.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.469+0000 7f3676ffd640 1 -- 172.21.15.43:0/4143707921 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f367c038040 con 0x7f36800c7cf0 2024-04-25T09:01:30.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.469+0000 7f368867c640 1 -- 172.21.15.43:0/4143707921 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f368009fca0 con 0x7f36800c7cf0 2024-04-25T09:01:30.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.469+0000 7f3676ffd640 1 -- 172.21.15.43:0/4143707921 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367c004490 con 0x7f36800c7cf0 2024-04-25T09:01:30.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.469+0000 7f368867c640 1 -- 172.21.15.43:0/4143707921 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36800a0260 con 0x7f36800c7cf0 2024-04-25T09:01:30.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.469+0000 7f368867c640 1 -- 172.21.15.43:0/4143707921 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36800680f0 con 0x7f36800c7cf0 2024-04-25T09:01:30.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.470+0000 7f3676ffd640 1 -- 172.21.15.43:0/4143707921 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f367c03c6d0 con 0x7f36800c7cf0 2024-04-25T09:01:30.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.470+0000 7f3676ffd640 1 --2- 172.21.15.43:0/4143707921 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f366406b4d0 0x7f366406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:30.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.471+0000 7f3676ffd640 1 -- 172.21.15.43:0/4143707921 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f367c0aa7d0 con 0x7f36800c7cf0 2024-04-25T09:01:30.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.471+0000 7f3685c17640 1 --2- 172.21.15.43:0/4143707921 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f366406b4d0 0x7f366406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:30.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.471+0000 7f3685c17640 1 --2- 172.21.15.43:0/4143707921 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f366406b4d0 0x7f366406d990 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f36800cdc50 tx=0x7f3670009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:30.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.475+0000 7f3676ffd640 1 -- 172.21.15.43:0/4143707921 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f367c079350 con 0x7f36800c7cf0 2024-04-25T09:01:30.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.723+0000 7f368867c640 1 -- 172.21.15.43:0/4143707921 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f3680074ef0 con 0x7f36800c7cf0 2024-04-25T09:01:30.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.723+0000 7f3676ffd640 1 -- 172.21.15.43:0/4143707921 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f367c07d3f0 con 0x7f36800c7cf0 2024-04-25T09:01:30.712 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:01:30.712 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:01:30.712 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_dump": [ 2024-04-25T09:01:30.712 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:01:30.712 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.a", 2024-04-25T09:01:30.713 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpmIjzEBRAAs2L10XrFO3mThjZevna5wA==", 2024-04-25T09:01:30.713 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:01:30.713 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T09:01:30.713 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T09:01:30.713 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T09:01:30.713 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:01:30.713 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:01:30.713 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:01:30.713 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:01:30.713 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.b", 2024-04-25T09:01:30.713 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm51cmERAAij4vy8xyihh8u/4dkmcllw==", 2024-04-25T09:01:30.713 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:01:30.713 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T09:01:30.713 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T09:01:30.714 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T09:01:30.714 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:01:30.714 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:01:30.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:01:30.714 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:01:30.714 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.c", 2024-04-25T09:01:30.714 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm1OOtDBAAlUYbCh1mBEEka9CWRnUz9A==", 2024-04-25T09:01:30.714 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:01:30.714 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T09:01:30.714 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T09:01:30.714 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T09:01:30.714 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:01:30.714 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:01:30.714 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:01:30.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:01:30.715 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.d", 2024-04-25T09:01:30.715 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpmpEvsFxAAXX1h37UWdFkyJ3Iw3xd8kw==", 2024-04-25T09:01:30.715 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:01:30.715 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T09:01:30.715 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T09:01:30.715 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T09:01:30.715 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:01:30.715 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:01:30.715 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:01:30.715 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:01:30.715 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.0", 2024-04-25T09:01:30.715 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD9GSpmi7qkORAAwSNgwzcVMitrPPf3nDyqvQ==", 2024-04-25T09:01:30.715 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:01:30.715 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T09:01:30.716 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T09:01:30.716 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:01:30.716 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:01:30.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:01:30.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:01:30.716 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.1", 2024-04-25T09:01:30.716 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAAGipmJDT3NxAAcWBCd9nt7Au+Xftjqcuc5w==", 2024-04-25T09:01:30.716 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:01:30.716 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T09:01:30.716 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T09:01:30.716 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:01:30.716 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:01:30.716 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:01:30.716 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:01:30.716 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.2", 2024-04-25T09:01:30.717 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQADGipm68kzNhAA0ZOnlr4edWDjx6RfoEHYIw==", 2024-04-25T09:01:30.717 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:01:30.717 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T09:01:30.717 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T09:01:30.717 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:01:30.717 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:01:30.717 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:01:30.717 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:01:30.717 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.3", 2024-04-25T09:01:30.717 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAGGipmT1AeNBAAoNPiEtkznSxxhVJ1EmZ1ZA==", 2024-04-25T09:01:30.717 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:01:30.717 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T09:01:30.717 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T09:01:30.717 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:01:30.717 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:01:30.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:01:30.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:01:30.718 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.4", 2024-04-25T09:01:30.718 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAMGipmBjHHNxAAH/SCzKHus8fKxxI3UN4wWQ==", 2024-04-25T09:01:30.718 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:01:30.718 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T09:01:30.718 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T09:01:30.718 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:01:30.718 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:01:30.718 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:01:30.718 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:01:30.718 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.5", 2024-04-25T09:01:30.718 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPGipm3FXKNRAAL97YYWONg4QHr4ajC5D7fA==", 2024-04-25T09:01:30.718 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:01:30.719 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T09:01:30.719 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T09:01:30.719 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:01:30.719 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:01:30.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:01:30.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:01:30.719 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.6", 2024-04-25T09:01:30.719 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQASGipmCwKaMxAAcM0p9Q2sbNQviGkwSLg8AA==", 2024-04-25T09:01:30.719 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:01:30.719 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T09:01:30.719 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T09:01:30.719 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:01:30.719 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:01:30.719 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:01:30.719 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:01:30.720 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.7", 2024-04-25T09:01:30.720 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAVGipmP7A2MRAAxnFJFosMcbZ6VDbgH8r6wA==", 2024-04-25T09:01:30.720 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:01:30.720 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T09:01:30.720 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T09:01:30.720 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:01:30.720 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:01:30.720 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:01:30.720 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:01:30.720 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.0", 2024-04-25T09:01:30.720 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpmVOjWGxAAFNmxeGi0h2D//pa6Ne3eQw==", 2024-04-25T09:01:30.721 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:01:30.721 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T09:01:30.721 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow r", 2024-04-25T09:01:30.721 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T09:01:30.721 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:01:30.721 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:01:30.721 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:01:30.721 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.1", 2024-04-25T09:01:30.721 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm24laJBAA9ijbDlFDTmF/ZZ4IhLfBVQ==", 2024-04-25T09:01:30.721 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:01:30.721 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T09:01:30.721 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-04-25T09:01:30.721 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-04-25T09:01:30.721 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-04-25T09:01:30.722 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:01:30.722 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:01:30.722 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:01:30.722 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.2", 2024-04-25T09:01:30.722 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm8b5UIRAACcOwDL630oJrXfp7s+fNTA==", 2024-04-25T09:01:30.722 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:01:30.722 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T09:01:30.722 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-04-25T09:01:30.722 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-04-25T09:01:30.722 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-04-25T09:01:30.722 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:01:30.722 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:01:30.722 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:01:30.722 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.3", 2024-04-25T09:01:30.722 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpmLsbBKRAAIKWtRdpuyCaW1LnKMWTMiw==", 2024-04-25T09:01:30.723 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:01:30.723 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T09:01:30.723 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-04-25T09:01:30.723 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-04-25T09:01:30.723 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-04-25T09:01:30.723 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:01:30.723 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:01:30.723 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:01:30.723 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.admin", 2024-04-25T09:01:30.723 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD7GSpmE9C8JRAAS263n6a4L2DXzMyS9o42NQ==", 2024-04-25T09:01:30.723 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:01:30.723 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-04-25T09:01:30.723 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T09:01:30.723 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T09:01:30.723 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:01:30.724 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:01:30.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:01:30.724 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:01:30.724 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.x", 2024-04-25T09:01:30.724 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm/uGhAhAAr0FaJ6SwZ8Su1MEqyZwNRA==", 2024-04-25T09:01:30.724 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:01:30.724 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-04-25T09:01:30.724 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2024-04-25T09:01:30.724 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:01:30.724 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:01:30.724 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:01:30.724 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:01:30.724 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.y", 2024-04-25T09:01:30.724 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD7GSpmPwwpOhAALeiQ/LKM2zgjE7yu16RQOg==", 2024-04-25T09:01:30.724 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:01:30.725 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-04-25T09:01:30.725 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2024-04-25T09:01:30.725 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:01:30.725 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:01:30.725 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:01:30.725 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-04-25T09:01:30.725 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:01:30.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.725+0000 7f368867c640 1 -- 172.21.15.43:0/4143707921 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f366406b4d0 msgr2=0x7f366406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:30.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.725+0000 7f368867c640 1 --2- 172.21.15.43:0/4143707921 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f366406b4d0 0x7f366406d990 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f36800cdc50 tx=0x7f3670009190 comp rx=0 tx=0).stop 2024-04-25T09:01:30.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.725+0000 7f368867c640 1 -- 172.21.15.43:0/4143707921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36800c7cf0 msgr2=0x7f36801f5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:30.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.725+0000 7f368867c640 1 --2- 172.21.15.43:0/4143707921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36800c7cf0 0x7f36801f5740 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f367c0097a0 tx=0x7f367c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:01:30.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.725+0000 7f368867c640 1 -- 172.21.15.43:0/4143707921 shutdown_connections 2024-04-25T09:01:30.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.725+0000 7f368867c640 1 --2- 172.21.15.43:0/4143707921 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f366406b4d0 0x7f366406d990 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:30.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.725+0000 7f368867c640 1 --2- 172.21.15.43:0/4143707921 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3680161f30 0x7f3680155250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:30.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.725+0000 7f368867c640 1 --2- 172.21.15.43:0/4143707921 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f368015df50 0x7f368014adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:30.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.725+0000 7f368867c640 1 --2- 172.21.15.43:0/4143707921 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f36800c7cf0 0x7f36801f5740 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:30.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.725+0000 7f368867c640 1 -- 172.21.15.43:0/4143707921 >> 172.21.15.43:0/4143707921 conn(0x7f36800d3a30 msgr2=0x7f368006b650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:30.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.725+0000 7f368867c640 1 -- 172.21.15.43:0/4143707921 shutdown_connections 2024-04-25T09:01:30.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.725+0000 7f368867c640 1 -- 172.21.15.43:0/4143707921 wait complete. 2024-04-25T09:01:30.728 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:01:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.840+0000 7fbe40c07640 1 Processor -- start 2024-04-25T09:01:30.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.840+0000 7fbe40c07640 1 -- start start 2024-04-25T09:01:30.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.841+0000 7fbe40c07640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbe3c15a9d0 0x7fbe3c06d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:30.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.841+0000 7fbe40c07640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fbe3c072160 0x7fbe3c0cd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:30.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.841+0000 7fbe40c07640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe3c0704f0 con 0x7fbe3c06d7d0 2024-04-25T09:01:30.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.841+0000 7fbe40c07640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe3c06dc30 con 0x7fbe3c15a9d0 2024-04-25T09:01:30.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.841+0000 7fbe40c07640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe3c0707d0 con 0x7fbe3c072160 2024-04-25T09:01:30.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.841+0000 7fbe3a59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbe3c15a9d0 0x7fbe3c06d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:30.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.841+0000 7fbe39d9b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fbe3c06d7d0 0x7fbe3c071a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:37502/0 (socket says 172.21.15.43:37502) 2024-04-25T09:01:30.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.841+0000 7fbe39d9b640 1 -- 172.21.15.43:0/680908787 learned_addr learned my addr 172.21.15.43:0/680908787 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.841+0000 7fbe3ad9d640 1 --2- 172.21.15.43:0/680908787 >> v2:172.21.15.193:3300/0 conn(0x7fbe3c072160 0x7fbe3c0cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.841+0000 7fbe3a59c640 1 -- 172.21.15.43:0/680908787 >> v2:172.21.15.193:3300/0 conn(0x7fbe3c072160 msgr2=0x7fbe3c0cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.841+0000 7fbe3a59c640 1 --2- 172.21.15.43:0/680908787 >> v2:172.21.15.193:3300/0 conn(0x7fbe3c072160 0x7fbe3c0cd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.842+0000 7fbe3a59c640 1 -- 172.21.15.43:0/680908787 >> v1:172.21.15.43:6789/0 conn(0x7fbe3c06d7d0 legacy=0x7fbe3c071a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.842+0000 7fbe3a59c640 1 -- 172.21.15.43:0/680908787 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3c070ab0 con 0x7fbe3c15a9d0 2024-04-25T09:01:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.842+0000 7fbe3959a640 1 -- 172.21.15.43:0/680908787 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3924546214 0 0) 0x7fbe3c0704f0 con 0x7fbe3c06d7d0 2024-04-25T09:01:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.842+0000 7fbe3a59c640 1 --2- 172.21.15.43:0/680908787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe3c15a9d0 0x7fbe3c06d290 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c067530 tx=0x7fbe3402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6b5f5cbed633993 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.842+0000 7fbe3959a640 1 -- 172.21.15.43:0/680908787 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe34037070 con 0x7fbe3c15a9d0 2024-04-25T09:01:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.842+0000 7fbe3959a640 1 -- 172.21.15.43:0/680908787 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe34032440 con 0x7fbe3c15a9d0 2024-04-25T09:01:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.842+0000 7fbe3959a640 1 -- 172.21.15.43:0/680908787 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe34032810 con 0x7fbe3c15a9d0 2024-04-25T09:01:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.843+0000 7fbe40c07640 1 -- 172.21.15.43:0/680908787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe3c15a9d0 msgr2=0x7fbe3c06d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.843+0000 7fbe40c07640 1 --2- 172.21.15.43:0/680908787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe3c15a9d0 0x7fbe3c06d290 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c067530 tx=0x7fbe3402a1a0 comp rx=0 tx=0).stop 2024-04-25T09:01:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.843+0000 7fbe40c07640 1 -- 172.21.15.43:0/680908787 shutdown_connections 2024-04-25T09:01:30.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.843+0000 7fbe40c07640 1 --2- 172.21.15.43:0/680908787 >> v2:172.21.15.193:3300/0 conn(0x7fbe3c072160 0x7fbe3c0cd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.843+0000 7fbe40c07640 1 --2- 172.21.15.43:0/680908787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe3c15a9d0 0x7fbe3c06d290 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.843+0000 7fbe40c07640 1 -- 172.21.15.43:0/680908787 >> 172.21.15.43:0/680908787 conn(0x7fbe3c0b1480 msgr2=0x7fbe3c0b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.843+0000 7fbe40c07640 1 -- 172.21.15.43:0/680908787 shutdown_connections 2024-04-25T09:01:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.843+0000 7fbe40c07640 1 -- 172.21.15.43:0/680908787 wait complete. 2024-04-25T09:01:30.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.844+0000 7fbe40c07640 1 Processor -- start 2024-04-25T09:01:30.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.844+0000 7fbe40c07640 1 -- start start 2024-04-25T09:01:30.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.845+0000 7fbe40c07640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe3c06d7d0 0x7fbe3c14d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:30.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.845+0000 7fbe40c07640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe3c072160 0x7fbe3c151e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:30.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.845+0000 7fbe40c07640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbe3c151610 0x7fbe3c15be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:30.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.845+0000 7fbe40c07640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c0cde80 con 0x7fbe3c072160 2024-04-25T09:01:30.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.845+0000 7fbe40c07640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c0712b0 con 0x7fbe3c06d7d0 2024-04-25T09:01:30.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.845+0000 7fbe40c07640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c0ce000 con 0x7fbe3c151610 2024-04-25T09:01:30.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.845+0000 7fbe39d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe3c072160 0x7fbe3c151e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:30.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.845+0000 7fbe39d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe3c072160 0x7fbe3c151e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48798/0 (socket says 172.21.15.43:48798) 2024-04-25T09:01:30.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.845+0000 7fbe39d9b640 1 -- 172.21.15.43:0/1831658209 learned_addr learned my addr 172.21.15.43:0/1831658209 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:30.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.845+0000 7fbe3a59c640 1 --2- 172.21.15.43:0/1831658209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe3c06d7d0 0x7fbe3c14d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:30.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.845+0000 7fbe3ad9d640 1 --2- 172.21.15.43:0/1831658209 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbe3c151610 0x7fbe3c15be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:30.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.846+0000 7fbe39d9b640 1 -- 172.21.15.43:0/1831658209 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbe3c151610 msgr2=0x7fbe3c15be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:30.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.846+0000 7fbe39d9b640 1 --2- 172.21.15.43:0/1831658209 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbe3c151610 0x7fbe3c15be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:30.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.846+0000 7fbe39d9b640 1 -- 172.21.15.43:0/1831658209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe3c06d7d0 msgr2=0x7fbe3c14d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:30.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.846+0000 7fbe39d9b640 1 --2- 172.21.15.43:0/1831658209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe3c06d7d0 0x7fbe3c14d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:30.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.846+0000 7fbe39d9b640 1 -- 172.21.15.43:0/1831658209 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3c1f3b70 con 0x7fbe3c072160 2024-04-25T09:01:30.837 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.846+0000 7fbe39d9b640 1 --2- 172.21.15.43:0/1831658209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe3c072160 0x7fbe3c151e40 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fbe2400c4d0 tx=0x7fbe2400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:30.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.846+0000 7fbe32ffd640 1 -- 172.21.15.43:0/1831658209 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe24018070 con 0x7fbe3c072160 2024-04-25T09:01:30.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.846+0000 7fbe32ffd640 1 -- 172.21.15.43:0/1831658209 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe24007500 con 0x7fbe3c072160 2024-04-25T09:01:30.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.846+0000 7fbe40c07640 1 -- 172.21.15.43:0/1831658209 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe3c152a90 con 0x7fbe3c072160 2024-04-25T09:01:30.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.846+0000 7fbe32ffd640 1 -- 172.21.15.43:0/1831658209 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe24004a20 con 0x7fbe3c072160 2024-04-25T09:01:30.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.846+0000 7fbe40c07640 1 -- 172.21.15.43:0/1831658209 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe3c1f3e10 con 0x7fbe3c072160 2024-04-25T09:01:30.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.847+0000 7fbe40c07640 1 -- 172.21.15.43:0/1831658209 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe3c06dfd0 con 0x7fbe3c072160 2024-04-25T09:01:30.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.848+0000 7fbe32ffd640 1 -- 172.21.15.43:0/1831658209 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbe2401b070 con 0x7fbe3c072160 2024-04-25T09:01:30.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.848+0000 7fbe32ffd640 1 --2- 172.21.15.43:0/1831658209 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbe1c06b4d0 0x7fbe1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:30.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.848+0000 7fbe32ffd640 1 -- 172.21.15.43:0/1831658209 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbe24089100 con 0x7fbe3c072160 2024-04-25T09:01:30.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.848+0000 7fbe3a59c640 1 --2- 172.21.15.43:0/1831658209 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbe1c06b4d0 0x7fbe1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:30.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.849+0000 7fbe3a59c640 1 --2- 172.21.15.43:0/1831658209 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbe1c06b4d0 0x7fbe1c06d990 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fbe34034000 tx=0x7fbe3402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:30.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:30.852+0000 7fbe32ffd640 1 -- 172.21.15.43:0/1831658209 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbe3c06dfd0 con 0x7fbe3c072160 2024-04-25T09:01:31.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.033+0000 7fbe40c07640 1 -- 172.21.15.43:0/1831658209 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe3c074ef0 con 0x7fbe3c072160 2024-04-25T09:01:31.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.034+0000 7fbe32ffd640 1 -- 172.21.15.43:0/1831658209 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+12856 (secure 0 0 0) 0x7fbe24057c80 con 0x7fbe3c072160 2024-04-25T09:01:31.023 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:01:31.023 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":98,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:01:30.341010+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:01:31.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.036+0000 7fbe40c07640 1 -- 172.21.15.43:0/1831658209 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbe1c06b4d0 msgr2=0x7fbe1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:31.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.036+0000 7fbe40c07640 1 --2- 172.21.15.43:0/1831658209 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbe1c06b4d0 0x7fbe1c06d990 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fbe34034000 tx=0x7fbe3402e000 comp rx=0 tx=0).stop 2024-04-25T09:01:31.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.036+0000 7fbe40c07640 1 -- 172.21.15.43:0/1831658209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe3c072160 msgr2=0x7fbe3c151e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:31.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.036+0000 7fbe40c07640 1 --2- 172.21.15.43:0/1831658209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe3c072160 0x7fbe3c151e40 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fbe2400c4d0 tx=0x7fbe2400c990 comp rx=0 tx=0).stop 2024-04-25T09:01:31.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.036+0000 7fbe40c07640 1 -- 172.21.15.43:0/1831658209 shutdown_connections 2024-04-25T09:01:31.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.036+0000 7fbe40c07640 1 --2- 172.21.15.43:0/1831658209 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbe3c151610 0x7fbe3c15be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.036+0000 7fbe40c07640 1 --2- 172.21.15.43:0/1831658209 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbe3c072160 0x7fbe3c151e40 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.036+0000 7fbe40c07640 1 --2- 172.21.15.43:0/1831658209 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbe1c06b4d0 0x7fbe1c06d990 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.036+0000 7fbe40c07640 1 --2- 172.21.15.43:0/1831658209 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbe3c06d7d0 0x7fbe3c14d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.036+0000 7fbe40c07640 1 -- 172.21.15.43:0/1831658209 >> 172.21.15.43:0/1831658209 conn(0x7fbe3c0b1480 msgr2=0x7fbe3c158a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:31.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.036+0000 7fbe40c07640 1 -- 172.21.15.43:0/1831658209 shutdown_connections 2024-04-25T09:01:31.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.036+0000 7fbe40c07640 1 -- 172.21.15.43:0/1831658209 wait complete. 2024-04-25T09:01:31.036 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:01:31.037 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:01:31.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.152+0000 7f730714f640 1 Processor -- start 2024-04-25T09:01:31.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.152+0000 7f730714f640 1 -- start start 2024-04-25T09:01:31.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.153+0000 7f730714f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f730015e090 0x7f7300161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:31.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.153+0000 7f730714f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7300161fe0 0x7f73000b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:31.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.153+0000 7f730714f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7300162610 con 0x7f73000b5aa0 2024-04-25T09:01:31.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.153+0000 7f730714f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73000b5f00 con 0x7f730015e090 2024-04-25T09:01:31.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.153+0000 7f730714f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73000d1da0 con 0x7f7300161fe0 2024-04-25T09:01:31.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.153+0000 7f7304eeb640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f730015e090 0x7f7300161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:31.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.153+0000 7f73056ec640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f73000b5aa0 0x7f73000ce560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:37518/0 (socket says 172.21.15.43:37518) 2024-04-25T09:01:31.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.153+0000 7f73056ec640 1 -- 172.21.15.43:0/764125204 learned_addr learned my addr 172.21.15.43:0/764125204 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:31.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.153+0000 7f7304eeb640 1 -- 172.21.15.43:0/764125204 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7300161fe0 msgr2=0x7f73000b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:31.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.153+0000 7f7304eeb640 1 --2- 172.21.15.43:0/764125204 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7300161fe0 0x7f73000b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.153+0000 7f7304eeb640 1 -- 172.21.15.43:0/764125204 >> v1:172.21.15.43:6789/0 conn(0x7f73000b5aa0 legacy=0x7f73000ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:31.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.153+0000 7f72f77fe640 1 -- 172.21.15.43:0/764125204 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2750367458 0 0) 0x7f7300162610 con 0x7f73000b5aa0 2024-04-25T09:01:31.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.153+0000 7f7304eeb640 1 -- 172.21.15.43:0/764125204 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73000d1750 con 0x7f730015e090 2024-04-25T09:01:31.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.154+0000 7f7304eeb640 1 --2- 172.21.15.43:0/764125204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f730015e090 0x7f7300161aa0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f72fc0097a0 tx=0x7f72fc02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9918f4e28689198e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:31.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.154+0000 7f72f77fe640 1 -- 172.21.15.43:0/764125204 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72fc037070 con 0x7f730015e090 2024-04-25T09:01:31.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.154+0000 7f72f77fe640 1 -- 172.21.15.43:0/764125204 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72fc032440 con 0x7f730015e090 2024-04-25T09:01:31.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.154+0000 7f72f77fe640 1 -- 172.21.15.43:0/764125204 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72fc032810 con 0x7f730015e090 2024-04-25T09:01:31.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.154+0000 7f730714f640 1 -- 172.21.15.43:0/764125204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f730015e090 msgr2=0x7f7300161aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:31.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.154+0000 7f730714f640 1 --2- 172.21.15.43:0/764125204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f730015e090 0x7f7300161aa0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f72fc0097a0 tx=0x7f72fc02a210 comp rx=0 tx=0).stop 2024-04-25T09:01:31.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.154+0000 7f730714f640 1 -- 172.21.15.43:0/764125204 shutdown_connections 2024-04-25T09:01:31.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.154+0000 7f730714f640 1 --2- 172.21.15.43:0/764125204 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7300161fe0 0x7f73000b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.154+0000 7f730714f640 1 --2- 172.21.15.43:0/764125204 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f730015e090 0x7f7300161aa0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.154+0000 7f730714f640 1 -- 172.21.15.43:0/764125204 >> 172.21.15.43:0/764125204 conn(0x7f73000d0560 msgr2=0x7f73000d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:31.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.155+0000 7f730714f640 1 -- 172.21.15.43:0/764125204 shutdown_connections 2024-04-25T09:01:31.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.155+0000 7f730714f640 1 -- 172.21.15.43:0/764125204 wait complete. 2024-04-25T09:01:31.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.156+0000 7f730714f640 1 Processor -- start 2024-04-25T09:01:31.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.156+0000 7f730714f640 1 -- start start 2024-04-25T09:01:31.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.156+0000 7f730714f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73000b5aa0 0x7f7300158350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:31.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.156+0000 7f730714f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f730015e090 0x7f73000c9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:31.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.157+0000 7f730714f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7300161fe0 0x7f73000ca150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:31.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.157+0000 7f730714f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73000b3280 con 0x7f73000b5aa0 2024-04-25T09:01:31.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.157+0000 7f730714f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73001628f0 con 0x7f730015e090 2024-04-25T09:01:31.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.157+0000 7f730714f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73000d3240 con 0x7f7300161fe0 2024-04-25T09:01:31.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.157+0000 7f7304eeb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73000b5aa0 0x7f7300158350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:31.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.157+0000 7f7304eeb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73000b5aa0 0x7f7300158350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48810/0 (socket says 172.21.15.43:48810) 2024-04-25T09:01:31.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.157+0000 7f7304eeb640 1 -- 172.21.15.43:0/3075483414 learned_addr learned my addr 172.21.15.43:0/3075483414 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:31.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.157+0000 7f7304eeb640 1 -- 172.21.15.43:0/3075483414 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7300161fe0 msgr2=0x7f73000ca150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:31.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.157+0000 7f73056ec640 1 --2- 172.21.15.43:0/3075483414 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7300161fe0 0x7f73000ca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:31.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.157+0000 7f72f7fff640 1 --2- 172.21.15.43:0/3075483414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f730015e090 0x7f73000c9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:31.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.157+0000 7f7304eeb640 1 --2- 172.21.15.43:0/3075483414 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7300161fe0 0x7f73000ca150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.157+0000 7f7304eeb640 1 -- 172.21.15.43:0/3075483414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f730015e090 msgr2=0x7f73000c9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:31.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.157+0000 7f7304eeb640 1 --2- 172.21.15.43:0/3075483414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f730015e090 0x7f73000c9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.157+0000 7f7304eeb640 1 -- 172.21.15.43:0/3075483414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7300160230 con 0x7f73000b5aa0 2024-04-25T09:01:31.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.157+0000 7f73056ec640 1 --2- 172.21.15.43:0/3075483414 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7300161fe0 0x7f73000ca150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:31.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.157+0000 7f72f7fff640 1 --2- 172.21.15.43:0/3075483414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f730015e090 0x7f73000c9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:31.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.157+0000 7f7304eeb640 1 --2- 172.21.15.43:0/3075483414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73000b5aa0 0x7f7300158350 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f72fc0097a0 tx=0x7f72fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:31.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.157+0000 7f72f57fa640 1 -- 172.21.15.43:0/3075483414 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72fc041070 con 0x7f73000b5aa0 2024-04-25T09:01:31.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.157+0000 7f72f57fa640 1 -- 172.21.15.43:0/3075483414 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72fc03c870 con 0x7f73000b5aa0 2024-04-25T09:01:31.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.157+0000 7f730714f640 1 -- 172.21.15.43:0/3075483414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73001532d0 con 0x7f73000b5aa0 2024-04-25T09:01:31.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.157+0000 7f72f57fa640 1 -- 172.21.15.43:0/3075483414 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72fc037040 con 0x7f73000b5aa0 2024-04-25T09:01:31.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.157+0000 7f730714f640 1 -- 172.21.15.43:0/3075483414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7300153890 con 0x7f73000b5aa0 2024-04-25T09:01:31.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.158+0000 7f730714f640 1 -- 172.21.15.43:0/3075483414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f730006bf00 con 0x7f73000b5aa0 2024-04-25T09:01:31.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.159+0000 7f72f57fa640 1 -- 172.21.15.43:0/3075483414 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f72fc044070 con 0x7f73000b5aa0 2024-04-25T09:01:31.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.159+0000 7f72f57fa640 1 --2- 172.21.15.43:0/3075483414 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f72ec06b4d0 0x7f72ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:31.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.159+0000 7f72f57fa640 1 -- 172.21.15.43:0/3075483414 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f72fc0aab10 con 0x7f73000b5aa0 2024-04-25T09:01:31.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.159+0000 7f72f7fff640 1 --2- 172.21.15.43:0/3075483414 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f72ec06b4d0 0x7f72ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:31.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.160+0000 7f72f7fff640 1 --2- 172.21.15.43:0/3075483414 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f72ec06b4d0 0x7f72ec06d990 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f7300152850 tx=0x7f72f0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:31.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.163+0000 7f72f57fa640 1 -- 172.21.15.43:0/3075483414 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f730006bf00 con 0x7f73000b5aa0 2024-04-25T09:01:31.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.345+0000 7f730714f640 1 -- 172.21.15.43:0/3075483414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7300074f60 con 0x7f73000b5aa0 2024-04-25T09:01:31.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.345+0000 7f72f57fa640 1 -- 172.21.15.43:0/3075483414 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v98) v1 ==== 93+0+10 (secure 0 0 0) 0x7f72fc079e00 con 0x7f73000b5aa0 2024-04-25T09:01:31.334 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T09:01:31.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.347+0000 7f730714f640 1 -- 172.21.15.43:0/3075483414 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f72ec06b4d0 msgr2=0x7f72ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:31.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.347+0000 7f730714f640 1 --2- 172.21.15.43:0/3075483414 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f72ec06b4d0 0x7f72ec06d990 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f7300152850 tx=0x7f72f0008040 comp rx=0 tx=0).stop 2024-04-25T09:01:31.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.347+0000 7f730714f640 1 -- 172.21.15.43:0/3075483414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73000b5aa0 msgr2=0x7f7300158350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:31.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.347+0000 7f730714f640 1 --2- 172.21.15.43:0/3075483414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73000b5aa0 0x7f7300158350 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f72fc0097a0 tx=0x7f72fc0046c0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.347+0000 7f730714f640 1 -- 172.21.15.43:0/3075483414 shutdown_connections 2024-04-25T09:01:31.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.347+0000 7f730714f640 1 --2- 172.21.15.43:0/3075483414 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7300161fe0 0x7f73000ca150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.347+0000 7f730714f640 1 --2- 172.21.15.43:0/3075483414 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f72ec06b4d0 0x7f72ec06d990 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.347+0000 7f730714f640 1 --2- 172.21.15.43:0/3075483414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f730015e090 0x7f73000c9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.347+0000 7f730714f640 1 --2- 172.21.15.43:0/3075483414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f73000b5aa0 0x7f7300158350 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.347+0000 7f730714f640 1 -- 172.21.15.43:0/3075483414 >> 172.21.15.43:0/3075483414 conn(0x7f73000d0560 msgr2=0x7f730015bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:31.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.347+0000 7f730714f640 1 -- 172.21.15.43:0/3075483414 shutdown_connections 2024-04-25T09:01:31.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.347+0000 7f730714f640 1 -- 172.21.15.43:0/3075483414 wait complete. 2024-04-25T09:01:31.348 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:01:31.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.459+0000 7f91478b3640 1 Processor -- start 2024-04-25T09:01:31.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.460+0000 7f91478b3640 1 -- start start 2024-04-25T09:01:31.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.460+0000 7f91478b3640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f914015dce0 0x7f914015ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:31.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.460+0000 7f91478b3640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91400d2920 con 0x7f91400ce550 2024-04-25T09:01:31.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.461+0000 7f91478b3640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91400cee80 con 0x7f91400d0af0 2024-04-25T09:01:31.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.461+0000 7f91478b3640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91400cf160 con 0x7f914015dce0 2024-04-25T09:01:31.453 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.461+0000 7f9144e4e640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f91400ce550 0x7f91400d03e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:37532/0 (socket says 172.21.15.43:37532) 2024-04-25T09:01:31.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.461+0000 7f9144e4e640 1 -- 172.21.15.43:0/265024373 learned_addr learned my addr 172.21.15.43:0/265024373 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:31.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.461+0000 7f914564f640 1 --2- 172.21.15.43:0/265024373 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f914015dce0 0x7f914015ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:31.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.461+0000 7f913ffff640 1 -- 172.21.15.43:0/265024373 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 251597591 0 0) 0x7f91400d2920 con 0x7f91400ce550 2024-04-25T09:01:31.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.461+0000 7f913ffff640 1 -- 172.21.15.43:0/265024373 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9128003520 con 0x7f91400ce550 2024-04-25T09:01:31.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.461+0000 7f913ffff640 1 -- 172.21.15.43:0/265024373 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2651589116 0 0) 0x7f9128003520 con 0x7f91400ce550 2024-04-25T09:01:31.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.461+0000 7f913ffff640 1 -- 172.21.15.43:0/265024373 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f914015dce0 msgr2=0x7f914015ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:31.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.461+0000 7f913ffff640 1 --2- 172.21.15.43:0/265024373 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f914015dce0 0x7f914015ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.461+0000 7f913ffff640 1 -- 172.21.15.43:0/265024373 >> v1:172.21.15.193:6789/0 conn(0x7f91400d0af0 legacy=0x7f9140160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:31.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.461+0000 7f913ffff640 1 -- 172.21.15.43:0/265024373 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91400cf440 con 0x7f91400ce550 2024-04-25T09:01:31.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.461+0000 7f913ffff640 1 -- 172.21.15.43:0/265024373 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f9134001690 con 0x7f91400ce550 2024-04-25T09:01:31.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.461+0000 7f913ffff640 1 -- 172.21.15.43:0/265024373 >> v1:172.21.15.43:6789/0 conn(0x7f91400ce550 legacy=0x7f91400d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:31.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.461+0000 7f913ffff640 1 --2- 172.21.15.43:0/265024373 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f914015dce0 0x7f91280039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:31.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.461+0000 7f913ffff640 1 --2- 172.21.15.43:0/265024373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9128003f20 0x7f9128004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:31.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.461+0000 7f913ffff640 1 --2- 172.21.15.43:0/265024373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9128004d40 0x7f91280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:31.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.461+0000 7f913ffff640 1 -- 172.21.15.43:0/265024373 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91400cf160 con 0x7f9128003f20 2024-04-25T09:01:31.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.461+0000 7f913ffff640 1 -- 172.21.15.43:0/265024373 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912800bad0 con 0x7f9128004d40 2024-04-25T09:01:31.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.461+0000 7f913ffff640 1 -- 172.21.15.43:0/265024373 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f912800bcd0 con 0x7f914015dce0 2024-04-25T09:01:31.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.461+0000 7f9144e4e640 1 --2- 172.21.15.43:0/265024373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9128003f20 0x7f9128004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:31.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.462+0000 7f914564f640 1 --2- 172.21.15.43:0/265024373 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f914015dce0 0x7f91280039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:31.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.462+0000 7f9145e50640 1 --2- 172.21.15.43:0/265024373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9128004d40 0x7f91280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:31.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.462+0000 7f9144e4e640 1 -- 172.21.15.43:0/265024373 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f914015dce0 msgr2=0x7f91280039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:31.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.462+0000 7f9144e4e640 1 --2- 172.21.15.43:0/265024373 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f914015dce0 0x7f91280039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.462+0000 7f9144e4e640 1 -- 172.21.15.43:0/265024373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9128004d40 msgr2=0x7f91280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:31.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.462+0000 7f9144e4e640 1 --2- 172.21.15.43:0/265024373 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9128004d40 0x7f91280051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.462+0000 7f9144e4e640 1 -- 172.21.15.43:0/265024373 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9128003520 con 0x7f9128003f20 2024-04-25T09:01:31.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.462+0000 7f9144e4e640 1 --2- 172.21.15.43:0/265024373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9128003f20 0x7f9128004800 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f9134007fd0 tx=0x7f9128005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aabe9e3d6615f29c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:31.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.463+0000 7f913ffff640 1 -- 172.21.15.43:0/265024373 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9134007d50 con 0x7f9128003f20 2024-04-25T09:01:31.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.463+0000 7f913ffff640 1 -- 172.21.15.43:0/265024373 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91340058d0 con 0x7f9128003f20 2024-04-25T09:01:31.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.463+0000 7f913ffff640 1 -- 172.21.15.43:0/265024373 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9134005ca0 con 0x7f9128003f20 2024-04-25T09:01:31.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.463+0000 7f91478b3640 1 -- 172.21.15.43:0/265024373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9128003f20 msgr2=0x7f9128004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:31.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.463+0000 7f91478b3640 1 --2- 172.21.15.43:0/265024373 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9128003f20 0x7f9128004800 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f9134007fd0 tx=0x7f9128005830 comp rx=0 tx=0).stop 2024-04-25T09:01:31.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.463+0000 7f9145e50640 1 -- 172.21.15.43:0/265024373 reap_dead start 2024-04-25T09:01:31.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.463+0000 7f91478b3640 1 -- 172.21.15.43:0/265024373 shutdown_connections 2024-04-25T09:01:31.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.463+0000 7f91478b3640 1 -- 172.21.15.43:0/265024373 >> 172.21.15.43:0/265024373 conn(0x7f914009cd00 msgr2=0x7f91400d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:31.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.463+0000 7f91478b3640 1 -- 172.21.15.43:0/265024373 shutdown_connections 2024-04-25T09:01:31.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.463+0000 7f91478b3640 1 -- 172.21.15.43:0/265024373 wait complete. 2024-04-25T09:01:31.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.464+0000 7f91478b3640 1 Processor -- start 2024-04-25T09:01:31.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.465+0000 7f91478b3640 1 -- start start 2024-04-25T09:01:31.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.465+0000 7f91478b3640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9128003f20 0x7f914015c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:31.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.465+0000 7f91478b3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f914009e620 0x7f91401f3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:31.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.465+0000 7f91478b3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91401f4040 0x7f914009f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:31.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.465+0000 7f91478b3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914015e0c0 con 0x7f91401f4040 2024-04-25T09:01:31.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.465+0000 7f91478b3640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914015f6b0 con 0x7f914009e620 2024-04-25T09:01:31.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.465+0000 7f91478b3640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91400ce990 con 0x7f9128003f20 2024-04-25T09:01:31.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.465+0000 7f9145e50640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91401f4040 0x7f914009f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:31.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.465+0000 7f9145e50640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91401f4040 0x7f914009f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48842/0 (socket says 172.21.15.43:48842) 2024-04-25T09:01:31.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.465+0000 7f9145e50640 1 -- 172.21.15.43:0/2087730040 learned_addr learned my addr 172.21.15.43:0/2087730040 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:31.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.466+0000 7f9144e4e640 1 --2- 172.21.15.43:0/2087730040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f914009e620 0x7f91401f3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:31.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.466+0000 7f914564f640 1 --2- 172.21.15.43:0/2087730040 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9128003f20 0x7f914015c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:31.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.466+0000 7f9145e50640 1 -- 172.21.15.43:0/2087730040 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9128003f20 msgr2=0x7f914015c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:31.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.466+0000 7f9145e50640 1 --2- 172.21.15.43:0/2087730040 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9128003f20 0x7f914015c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.466+0000 7f9145e50640 1 -- 172.21.15.43:0/2087730040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f914009e620 msgr2=0x7f91401f3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:31.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.466+0000 7f9145e50640 1 --2- 172.21.15.43:0/2087730040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f914009e620 0x7f91401f3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.466+0000 7f9145e50640 1 -- 172.21.15.43:0/2087730040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91400b6640 con 0x7f91401f4040 2024-04-25T09:01:31.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.466+0000 7f914564f640 1 --2- 172.21.15.43:0/2087730040 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9128003f20 0x7f914015c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:01:31.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.466+0000 7f9145e50640 1 --2- 172.21.15.43:0/2087730040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91401f4040 0x7f914009f510 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f91400d39a0 tx=0x7f913000ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:31.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.466+0000 7f913dffb640 1 -- 172.21.15.43:0/2087730040 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9130019070 con 0x7f91401f4040 2024-04-25T09:01:31.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.466+0000 7f913dffb640 1 -- 172.21.15.43:0/2087730040 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9130006e20 con 0x7f91401f4040 2024-04-25T09:01:31.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.466+0000 7f91478b3640 1 -- 172.21.15.43:0/2087730040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91400b6920 con 0x7f91401f4040 2024-04-25T09:01:31.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.466+0000 7f913dffb640 1 -- 172.21.15.43:0/2087730040 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9130003b90 con 0x7f91401f4040 2024-04-25T09:01:31.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.466+0000 7f91478b3640 1 -- 172.21.15.43:0/2087730040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91400b6ee0 con 0x7f91401f4040 2024-04-25T09:01:31.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.467+0000 7f91478b3640 1 -- 172.21.15.43:0/2087730040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91400680f0 con 0x7f91401f4040 2024-04-25T09:01:31.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.468+0000 7f913dffb640 1 -- 172.21.15.43:0/2087730040 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f913001c070 con 0x7f91401f4040 2024-04-25T09:01:31.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.468+0000 7f913dffb640 1 --2- 172.21.15.43:0/2087730040 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f912c06b4d0 0x7f912c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:31.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.468+0000 7f913dffb640 1 -- 172.21.15.43:0/2087730040 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9130089240 con 0x7f91401f4040 2024-04-25T09:01:31.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.468+0000 7f914564f640 1 --2- 172.21.15.43:0/2087730040 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f912c06b4d0 0x7f912c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:31.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.469+0000 7f914564f640 1 --2- 172.21.15.43:0/2087730040 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f912c06b4d0 0x7f912c06d990 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f9138009bc0 tx=0x7f9138008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:31.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.472+0000 7f913dffb640 1 -- 172.21.15.43:0/2087730040 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f91300584a0 con 0x7f91401f4040 2024-04-25T09:01:31.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.653+0000 7f91478b3640 1 -- 172.21.15.43:0/2087730040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f914009da20 con 0x7f91401f4040 2024-04-25T09:01:31.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.654+0000 7f913dffb640 1 -- 172.21.15.43:0/2087730040 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v98) v1 ==== 84+0+65 (secure 0 0 0) 0x7f9130057da0 con 0x7f91401f4040 2024-04-25T09:01:31.643 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:01:31.643 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2024-04-25T09:01:31.643 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:01:31.643 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 1, 2024-04-25T09:01:31.643 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": ".mgr" 2024-04-25T09:01:31.643 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:01:31.643 INFO:teuthology.orchestra.run.smithi043.stdout:] 2024-04-25T09:01:31.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.656+0000 7f91478b3640 1 -- 172.21.15.43:0/2087730040 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f912c06b4d0 msgr2=0x7f912c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:31.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.656+0000 7f91478b3640 1 --2- 172.21.15.43:0/2087730040 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f912c06b4d0 0x7f912c06d990 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f9138009bc0 tx=0x7f9138008040 comp rx=0 tx=0).stop 2024-04-25T09:01:31.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.656+0000 7f91478b3640 1 -- 172.21.15.43:0/2087730040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91401f4040 msgr2=0x7f914009f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:31.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.656+0000 7f91478b3640 1 --2- 172.21.15.43:0/2087730040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91401f4040 0x7f914009f510 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f91400d39a0 tx=0x7f913000ff70 comp rx=0 tx=0).stop 2024-04-25T09:01:31.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.656+0000 7f91478b3640 1 -- 172.21.15.43:0/2087730040 shutdown_connections 2024-04-25T09:01:31.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.656+0000 7f91478b3640 1 --2- 172.21.15.43:0/2087730040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91401f4040 0x7f914009f510 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.656+0000 7f91478b3640 1 --2- 172.21.15.43:0/2087730040 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f912c06b4d0 0x7f912c06d990 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.656+0000 7f91478b3640 1 --2- 172.21.15.43:0/2087730040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f914009e620 0x7f91401f3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.656+0000 7f91478b3640 1 --2- 172.21.15.43:0/2087730040 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9128003f20 0x7f914015c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.656+0000 7f91478b3640 1 -- 172.21.15.43:0/2087730040 >> 172.21.15.43:0/2087730040 conn(0x7f914009cd00 msgr2=0x7f91400c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:31.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.656+0000 7f91478b3640 1 -- 172.21.15.43:0/2087730040 shutdown_connections 2024-04-25T09:01:31.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.656+0000 7f91478b3640 1 -- 172.21.15.43:0/2087730040 wait complete. 2024-04-25T09:01:31.656 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:01:31.656 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:01:31.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.772+0000 7f8d2c85d640 1 Processor -- start 2024-04-25T09:01:31.761 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.772+0000 7f8d2c85d640 1 -- start start 2024-04-25T09:01:31.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.773+0000 7f8d2c85d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8d2815baa0 0x7f8d2815e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:31.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.773+0000 7f8d2c85d640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f8d280b4ea0 0x7f8d280d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:31.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.773+0000 7f8d2c85d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d280b37a0 con 0x7f8d2815b6c0 2024-04-25T09:01:31.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.773+0000 7f8d2c85d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d2815fce0 con 0x7f8d2815baa0 2024-04-25T09:01:31.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.773+0000 7f8d2c85d640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d2815eab0 con 0x7f8d280b4ea0 2024-04-25T09:01:31.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.773+0000 7f8d1ffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8d2815baa0 0x7f8d2815e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:31.762 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.773+0000 7f8d2659c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8d2815b6c0 0x7f8d2815f5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:37538/0 (socket says 172.21.15.43:37538) 2024-04-25T09:01:31.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.773+0000 7f8d1ffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8d2815baa0 0x7f8d2815e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48856/0 (socket says 172.21.15.43:48856) 2024-04-25T09:01:31.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.773+0000 7f8d2659c640 1 -- 172.21.15.43:0/1285277533 learned_addr learned my addr 172.21.15.43:0/1285277533 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:31.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.773+0000 7f8d26d9d640 1 --2- 172.21.15.43:0/1285277533 >> v2:172.21.15.193:3300/0 conn(0x7f8d280b4ea0 0x7f8d280d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:31.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.773+0000 7f8d1ffff640 1 -- 172.21.15.43:0/1285277533 >> v2:172.21.15.193:3300/0 conn(0x7f8d280b4ea0 msgr2=0x7f8d280d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:31.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.773+0000 7f8d1ffff640 1 --2- 172.21.15.43:0/1285277533 >> v2:172.21.15.193:3300/0 conn(0x7f8d280b4ea0 0x7f8d280d3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.773+0000 7f8d1ffff640 1 -- 172.21.15.43:0/1285277533 >> v1:172.21.15.43:6789/0 conn(0x7f8d2815b6c0 legacy=0x7f8d2815f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:31.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.773+0000 7f8d1ffff640 1 -- 172.21.15.43:0/1285277533 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2815ed90 con 0x7f8d2815baa0 2024-04-25T09:01:31.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.774+0000 7f8d1ffff640 1 --2- 172.21.15.43:0/1285277533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d2815baa0 0x7f8d2815e410 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f8d280ce0e0 tx=0x7f8d1002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c192db111a905cf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:31.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.774+0000 7f8d25d9b640 1 -- 172.21.15.43:0/1285277533 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d10037070 con 0x7f8d2815baa0 2024-04-25T09:01:31.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.774+0000 7f8d25d9b640 1 -- 172.21.15.43:0/1285277533 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d10032440 con 0x7f8d2815baa0 2024-04-25T09:01:31.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.774+0000 7f8d25d9b640 1 -- 172.21.15.43:0/1285277533 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d10032810 con 0x7f8d2815baa0 2024-04-25T09:01:31.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.774+0000 7f8d2c85d640 1 -- 172.21.15.43:0/1285277533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d2815baa0 msgr2=0x7f8d2815e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:31.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.774+0000 7f8d2c85d640 1 --2- 172.21.15.43:0/1285277533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d2815baa0 0x7f8d2815e410 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f8d280ce0e0 tx=0x7f8d1002a1a0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.775+0000 7f8d2c85d640 1 -- 172.21.15.43:0/1285277533 shutdown_connections 2024-04-25T09:01:31.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.775+0000 7f8d2c85d640 1 --2- 172.21.15.43:0/1285277533 >> v2:172.21.15.193:3300/0 conn(0x7f8d280b4ea0 0x7f8d280d3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.775+0000 7f8d2c85d640 1 --2- 172.21.15.43:0/1285277533 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d2815baa0 0x7f8d2815e410 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.775+0000 7f8d2c85d640 1 -- 172.21.15.43:0/1285277533 >> 172.21.15.43:0/1285277533 conn(0x7f8d2808d460 msgr2=0x7f8d2809cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:31.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.775+0000 7f8d2c85d640 1 -- 172.21.15.43:0/1285277533 shutdown_connections 2024-04-25T09:01:31.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.775+0000 7f8d2c85d640 1 -- 172.21.15.43:0/1285277533 wait complete. 2024-04-25T09:01:31.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.776+0000 7f8d2c85d640 1 Processor -- start 2024-04-25T09:01:31.766 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.776+0000 7f8d2c85d640 1 -- start start 2024-04-25T09:01:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.777+0000 7f8d2c85d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d280b4ea0 0x7f8d28152ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.777+0000 7f8d2c85d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8d2815b6c0 0x7f8d280ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.777+0000 7f8d2c85d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d2815baa0 0x7f8d280d0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.777+0000 7f8d2c85d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d280b5310 con 0x7f8d2815baa0 2024-04-25T09:01:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.777+0000 7f8d2c85d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2815f350 con 0x7f8d280b4ea0 2024-04-25T09:01:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.777+0000 7f8d2c85d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d280d3ca0 con 0x7f8d2815b6c0 2024-04-25T09:01:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.777+0000 7f8d26d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d2815baa0 0x7f8d280d0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.777+0000 7f8d26d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d2815baa0 0x7f8d280d0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48872/0 (socket says 172.21.15.43:48872) 2024-04-25T09:01:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.777+0000 7f8d26d9d640 1 -- 172.21.15.43:0/2374251129 learned_addr learned my addr 172.21.15.43:0/2374251129 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.777+0000 7f8d2659c640 1 --2- 172.21.15.43:0/2374251129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d280b4ea0 0x7f8d28152ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.777+0000 7f8d1ffff640 1 --2- 172.21.15.43:0/2374251129 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8d2815b6c0 0x7f8d280ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.777+0000 7f8d26d9d640 1 -- 172.21.15.43:0/2374251129 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8d2815b6c0 msgr2=0x7f8d280ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:31.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.777+0000 7f8d26d9d640 1 --2- 172.21.15.43:0/2374251129 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8d2815b6c0 0x7f8d280ccc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.777+0000 7f8d26d9d640 1 -- 172.21.15.43:0/2374251129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d280b4ea0 msgr2=0x7f8d28152ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:31.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.777+0000 7f8d26d9d640 1 --2- 172.21.15.43:0/2374251129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d280b4ea0 0x7f8d28152ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:31.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.777+0000 7f8d26d9d640 1 -- 172.21.15.43:0/2374251129 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d281f73b0 con 0x7f8d2815baa0 2024-04-25T09:01:31.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.777+0000 7f8d26d9d640 1 --2- 172.21.15.43:0/2374251129 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d2815baa0 0x7f8d280d0030 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f8d20007e70 tx=0x7f8d2000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:31.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.777+0000 7f8d1effd640 1 -- 172.21.15.43:0/2374251129 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d20014620 con 0x7f8d2815baa0 2024-04-25T09:01:31.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.777+0000 7f8d1effd640 1 -- 172.21.15.43:0/2374251129 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d20019070 con 0x7f8d2815baa0 2024-04-25T09:01:31.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.777+0000 7f8d1effd640 1 -- 172.21.15.43:0/2374251129 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d20014a20 con 0x7f8d2815baa0 2024-04-25T09:01:31.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.777+0000 7f8d2c85d640 1 -- 172.21.15.43:0/2374251129 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d281f7690 con 0x7f8d2815baa0 2024-04-25T09:01:31.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.778+0000 7f8d2c85d640 1 -- 172.21.15.43:0/2374251129 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d281f7c50 con 0x7f8d2815baa0 2024-04-25T09:01:31.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.779+0000 7f8d2c85d640 1 -- 172.21.15.43:0/2374251129 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d28068a40 con 0x7f8d2815baa0 2024-04-25T09:01:31.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.779+0000 7f8d1effd640 1 -- 172.21.15.43:0/2374251129 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8d20014bc0 con 0x7f8d2815baa0 2024-04-25T09:01:31.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.779+0000 7f8d1effd640 1 --2- 172.21.15.43:0/2374251129 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8d0806b4d0 0x7f8d0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:31.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.779+0000 7f8d1effd640 1 -- 172.21.15.43:0/2374251129 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8d2008a5e0 con 0x7f8d2815baa0 2024-04-25T09:01:31.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.780+0000 7f8d2659c640 1 --2- 172.21.15.43:0/2374251129 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8d0806b4d0 0x7f8d0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:31.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.780+0000 7f8d2659c640 1 --2- 172.21.15.43:0/2374251129 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8d0806b4d0 0x7f8d0806d990 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f8d18007fd0 tx=0x7f8d18007e80 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:31.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.784+0000 7f8d1effd640 1 -- 172.21.15.43:0/2374251129 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8d200591f0 con 0x7f8d2815baa0 2024-04-25T09:01:31.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:31.966+0000 7f8d2c85d640 1 -- 172.21.15.43:0/2374251129 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f8d28073fc0 con 0x7f8d2815baa0 2024-04-25T09:01:32.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.339+0000 7f8d1effd640 1 -- 172.21.15.43:0/2374251129 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v99) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8d2005f0b0 con 0x7f8d2815baa0 2024-04-25T09:01:32.331 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' created 2024-04-25T09:01:32.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.341+0000 7f8d2c85d640 1 -- 172.21.15.43:0/2374251129 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8d0806b4d0 msgr2=0x7f8d0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:32.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.341+0000 7f8d2c85d640 1 --2- 172.21.15.43:0/2374251129 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8d0806b4d0 0x7f8d0806d990 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f8d18007fd0 tx=0x7f8d18007e80 comp rx=0 tx=0).stop 2024-04-25T09:01:32.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.341+0000 7f8d2c85d640 1 -- 172.21.15.43:0/2374251129 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d2815baa0 msgr2=0x7f8d280d0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:32.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.341+0000 7f8d2c85d640 1 --2- 172.21.15.43:0/2374251129 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d2815baa0 0x7f8d280d0030 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f8d20007e70 tx=0x7f8d2000e4b0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.342+0000 7f8d2c85d640 1 -- 172.21.15.43:0/2374251129 shutdown_connections 2024-04-25T09:01:32.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.342+0000 7f8d2c85d640 1 --2- 172.21.15.43:0/2374251129 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8d2815baa0 0x7f8d280d0030 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.342+0000 7f8d2c85d640 1 --2- 172.21.15.43:0/2374251129 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8d2815b6c0 0x7f8d280ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.342+0000 7f8d2c85d640 1 --2- 172.21.15.43:0/2374251129 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8d0806b4d0 0x7f8d0806d990 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.342+0000 7f8d2c85d640 1 --2- 172.21.15.43:0/2374251129 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8d280b4ea0 0x7f8d28152ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.342+0000 7f8d2c85d640 1 -- 172.21.15.43:0/2374251129 >> 172.21.15.43:0/2374251129 conn(0x7f8d2808d460 msgr2=0x7f8d280b0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:32.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.342+0000 7f8d2c85d640 1 -- 172.21.15.43:0/2374251129 shutdown_connections 2024-04-25T09:01:32.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.342+0000 7f8d2c85d640 1 -- 172.21.15.43:0/2374251129 wait complete. 2024-04-25T09:01:32.347 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:01:32.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.467+0000 7f654bc10640 1 Processor -- start 2024-04-25T09:01:32.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.467+0000 7f654bc10640 1 -- start start 2024-04-25T09:01:32.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.468+0000 7f654bc10640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f65440d0190 0x7f65440d0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:32.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.468+0000 7f654bc10640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f65440cf600 0x7f654406bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:32.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.468+0000 7f654bc10640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65440d2310 con 0x7f65440d0ab0 2024-04-25T09:01:32.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.468+0000 7f654bc10640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f654406b3b0 con 0x7f65440d0190 2024-04-25T09:01:32.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.468+0000 7f654bc10640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65440d25f0 con 0x7f65440cf600 2024-04-25T09:01:32.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.468+0000 7f65499ac640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f65440d0190 0x7f65440d0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:32.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.468+0000 7f65499ac640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f65440d0190 0x7f65440d0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48882/0 (socket says 172.21.15.43:48882) 2024-04-25T09:01:32.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.468+0000 7f65499ac640 1 -- 172.21.15.43:0/1226135837 learned_addr learned my addr 172.21.15.43:0/1226135837 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:32.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.468+0000 7f654a1ad640 1 --2- 172.21.15.43:0/1226135837 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f65440cf600 0x7f654406bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:32.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.468+0000 7f65499ac640 1 -- 172.21.15.43:0/1226135837 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f65440cf600 msgr2=0x7f654406bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:32.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.468+0000 7f65499ac640 1 --2- 172.21.15.43:0/1226135837 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f65440cf600 0x7f654406bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.468+0000 7f65499ac640 1 -- 172.21.15.43:0/1226135837 >> v1:172.21.15.193:6789/0 conn(0x7f65440d0ab0 legacy=0x7f654406b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:32.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.468+0000 7f65499ac640 1 -- 172.21.15.43:0/1226135837 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65440d28d0 con 0x7f65440d0190 2024-04-25T09:01:32.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.469+0000 7f65499ac640 1 --2- 172.21.15.43:0/1226135837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65440d0190 0x7f65440d0570 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f653c009870 tx=0x7f653c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fc08502b1b864af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:32.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.469+0000 7f65489aa640 1 -- 172.21.15.43:0/1226135837 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653c037070 con 0x7f65440d0190 2024-04-25T09:01:32.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.469+0000 7f65489aa640 1 -- 172.21.15.43:0/1226135837 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f653c032440 con 0x7f65440d0190 2024-04-25T09:01:32.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.469+0000 7f65489aa640 1 -- 172.21.15.43:0/1226135837 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f653c032810 con 0x7f65440d0190 2024-04-25T09:01:32.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.470+0000 7f654bc10640 1 -- 172.21.15.43:0/1226135837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65440d0190 msgr2=0x7f65440d0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:32.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.470+0000 7f654bc10640 1 --2- 172.21.15.43:0/1226135837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65440d0190 0x7f65440d0570 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f653c009870 tx=0x7f653c02a1c0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.470+0000 7f654bc10640 1 -- 172.21.15.43:0/1226135837 shutdown_connections 2024-04-25T09:01:32.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.470+0000 7f654bc10640 1 --2- 172.21.15.43:0/1226135837 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f65440cf600 0x7f654406bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.470+0000 7f654bc10640 1 --2- 172.21.15.43:0/1226135837 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65440d0190 0x7f65440d0570 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.470+0000 7f654bc10640 1 -- 172.21.15.43:0/1226135837 >> 172.21.15.43:0/1226135837 conn(0x7f65440d39e0 msgr2=0x7f65440d3df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:32.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.470+0000 7f654bc10640 1 -- 172.21.15.43:0/1226135837 shutdown_connections 2024-04-25T09:01:32.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.470+0000 7f654bc10640 1 -- 172.21.15.43:0/1226135837 wait complete. 2024-04-25T09:01:32.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.471+0000 7f654bc10640 1 Processor -- start 2024-04-25T09:01:32.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.471+0000 7f654bc10640 1 -- start start 2024-04-25T09:01:32.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.472+0000 7f654bc10640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65440cf600 0x7f65440cc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:32.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.472+0000 7f654bc10640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65440d0190 0x7f6544153f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:32.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.472+0000 7f654bc10640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f65440d0ab0 0x7f654409e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:32.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.472+0000 7f654bc10640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65440d1f90 con 0x7f65440d0190 2024-04-25T09:01:32.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.472+0000 7f654bc10640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65440d0f30 con 0x7f65440cf600 2024-04-25T09:01:32.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.472+0000 7f654bc10640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f654415df20 con 0x7f65440d0ab0 2024-04-25T09:01:32.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.472+0000 7f65491ab640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65440d0190 0x7f6544153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:32.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.472+0000 7f65491ab640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65440d0190 0x7f6544153f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48894/0 (socket says 172.21.15.43:48894) 2024-04-25T09:01:32.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.472+0000 7f65491ab640 1 -- 172.21.15.43:0/1273000902 learned_addr learned my addr 172.21.15.43:0/1273000902 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:32.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.472+0000 7f65499ac640 1 --2- 172.21.15.43:0/1273000902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65440cf600 0x7f65440cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:32.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.472+0000 7f654a1ad640 1 --2- 172.21.15.43:0/1273000902 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f65440d0ab0 0x7f654409e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:32.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.472+0000 7f65491ab640 1 -- 172.21.15.43:0/1273000902 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f65440d0ab0 msgr2=0x7f654409e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:32.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.472+0000 7f65491ab640 1 --2- 172.21.15.43:0/1273000902 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f65440d0ab0 0x7f654409e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.473+0000 7f65491ab640 1 -- 172.21.15.43:0/1273000902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65440cf600 msgr2=0x7f65440cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:32.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.473+0000 7f65491ab640 1 --2- 172.21.15.43:0/1273000902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65440cf600 0x7f65440cc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:32.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.473+0000 7f65491ab640 1 -- 172.21.15.43:0/1273000902 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65440ccfc0 con 0x7f65440d0190 2024-04-25T09:01:32.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.473+0000 7f65491ab640 1 --2- 172.21.15.43:0/1273000902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65440d0190 0x7f6544153f10 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f653800c3c0 tx=0x7f653800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:32.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.473+0000 7f65427fc640 1 -- 172.21.15.43:0/1273000902 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65380052d0 con 0x7f65440d0190 2024-04-25T09:01:32.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.473+0000 7f65427fc640 1 -- 172.21.15.43:0/1273000902 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6538005490 con 0x7f65440d0190 2024-04-25T09:01:32.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.473+0000 7f65427fc640 1 -- 172.21.15.43:0/1273000902 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6538007260 con 0x7f65440d0190 2024-04-25T09:01:32.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.473+0000 7f654bc10640 1 -- 172.21.15.43:0/1273000902 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65440cd2a0 con 0x7f65440d0190 2024-04-25T09:01:32.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.473+0000 7f654bc10640 1 -- 172.21.15.43:0/1273000902 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65440c8670 con 0x7f65440d0190 2024-04-25T09:01:32.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.474+0000 7f654bc10640 1 -- 172.21.15.43:0/1273000902 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6544074f60 con 0x7f65440d0190 2024-04-25T09:01:32.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.474+0000 7f65427fc640 1 -- 172.21.15.43:0/1273000902 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6538007400 con 0x7f65440d0190 2024-04-25T09:01:32.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.475+0000 7f65427fc640 1 --2- 172.21.15.43:0/1273000902 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f653006b4d0 0x7f653006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:32.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.475+0000 7f65427fc640 1 -- 172.21.15.43:0/1273000902 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f65380894c0 con 0x7f65440d0190 2024-04-25T09:01:32.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.475+0000 7f65499ac640 1 --2- 172.21.15.43:0/1273000902 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f653006b4d0 0x7f653006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:32.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.476+0000 7f65499ac640 1 --2- 172.21.15.43:0/1273000902 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f653006b4d0 0x7f653006d990 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f653c034000 tx=0x7f653c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:32.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.479+0000 7f65427fc640 1 -- 172.21.15.43:0/1273000902 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6538058520 con 0x7f65440d0190 2024-04-25T09:01:32.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:32.663+0000 7f654bc10640 1 -- 172.21.15.43:0/1273000902 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f65440b42c0 con 0x7f65440d0190 2024-04-25T09:01:33.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.347+0000 7f65427fc640 1 -- 172.21.15.43:0/1273000902 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v100) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6538057e20 con 0x7f65440d0190 2024-04-25T09:01:33.336 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' created 2024-04-25T09:01:33.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.350+0000 7f654bc10640 1 -- 172.21.15.43:0/1273000902 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f653006b4d0 msgr2=0x7f653006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:33.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.350+0000 7f654bc10640 1 --2- 172.21.15.43:0/1273000902 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f653006b4d0 0x7f653006d990 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f653c034000 tx=0x7f653c02e000 comp rx=0 tx=0).stop 2024-04-25T09:01:33.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.350+0000 7f654bc10640 1 -- 172.21.15.43:0/1273000902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65440d0190 msgr2=0x7f6544153f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:33.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.350+0000 7f654bc10640 1 --2- 172.21.15.43:0/1273000902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65440d0190 0x7f6544153f10 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f653800c3c0 tx=0x7f653800c880 comp rx=0 tx=0).stop 2024-04-25T09:01:33.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.352+0000 7f654bc10640 1 -- 172.21.15.43:0/1273000902 shutdown_connections 2024-04-25T09:01:33.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.352+0000 7f654bc10640 1 --2- 172.21.15.43:0/1273000902 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f65440d0ab0 0x7f654409e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:33.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.352+0000 7f654bc10640 1 --2- 172.21.15.43:0/1273000902 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f65440d0190 0x7f6544153f10 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:33.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.352+0000 7f654bc10640 1 --2- 172.21.15.43:0/1273000902 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f653006b4d0 0x7f653006d990 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:33.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.352+0000 7f654bc10640 1 --2- 172.21.15.43:0/1273000902 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f65440cf600 0x7f65440cc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:33.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.352+0000 7f654bc10640 1 -- 172.21.15.43:0/1273000902 >> 172.21.15.43:0/1273000902 conn(0x7f65440d39e0 msgr2=0x7f65440b4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:33.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.362+0000 7f654bc10640 1 -- 172.21.15.43:0/1273000902 shutdown_connections 2024-04-25T09:01:33.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.368+0000 7f654bc10640 1 -- 172.21.15.43:0/1273000902 wait complete. 2024-04-25T09:01:33.372 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:01:33.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.491+0000 7eff02838640 1 Processor -- start 2024-04-25T09:01:33.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.491+0000 7eff02838640 1 -- start start 2024-04-25T09:01:33.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.492+0000 7eff02838640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7efefc15dec0 0x7efefc15ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:33.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.492+0000 7eff02838640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7efefc0ce8c0 0x7efefc0d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:33.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.492+0000 7eff02838640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efefc0cf110 con 0x7efefc0d04a0 2024-04-25T09:01:33.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.492+0000 7eff02838640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efefc0cf3f0 con 0x7efefc0ce8c0 2024-04-25T09:01:33.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.492+0000 7eff02838640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efefc0cf6d0 con 0x7efefc15dec0 2024-04-25T09:01:33.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.492+0000 7eff00dd5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7efefc0ce8c0 0x7efefc0d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:33.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.492+0000 7efefbfff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7efefc0d04a0 0x7efefc0d0880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:37542/0 (socket says 172.21.15.43:37542) 2024-04-25T09:01:33.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.492+0000 7efefbfff640 1 -- 172.21.15.43:0/2248467321 learned_addr learned my addr 172.21.15.43:0/2248467321 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:33.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.492+0000 7efefb7fe640 1 --2- 172.21.15.43:0/2248467321 >> v2:172.21.15.193:3300/0 conn(0x7efefc15dec0 0x7efefc15ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:33.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.492+0000 7eff00dd5640 1 -- 172.21.15.43:0/2248467321 >> v2:172.21.15.193:3300/0 conn(0x7efefc15dec0 msgr2=0x7efefc15ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:33.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.492+0000 7eff00dd5640 1 --2- 172.21.15.43:0/2248467321 >> v2:172.21.15.193:3300/0 conn(0x7efefc15dec0 0x7efefc15ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:33.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.492+0000 7eff00dd5640 1 -- 172.21.15.43:0/2248467321 >> v1:172.21.15.43:6789/0 conn(0x7efefc0d04a0 legacy=0x7efefc0d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:33.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.492+0000 7eff00dd5640 1 -- 172.21.15.43:0/2248467321 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efefc0cf9b0 con 0x7efefc0ce8c0 2024-04-25T09:01:33.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.493+0000 7eff00dd5640 1 --2- 172.21.15.43:0/2248467321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efefc0ce8c0 0x7efefc0d17a0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7efefc0d0070 tx=0x7efef402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dc83bea728cae43 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:33.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.493+0000 7efefaffd640 1 -- 172.21.15.43:0/2248467321 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef4007740 con 0x7efefc0ce8c0 2024-04-25T09:01:33.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.493+0000 7efefaffd640 1 -- 172.21.15.43:0/2248467321 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efef40078e0 con 0x7efefc0ce8c0 2024-04-25T09:01:33.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.493+0000 7efefaffd640 1 -- 172.21.15.43:0/2248467321 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef4007cb0 con 0x7efefc0ce8c0 2024-04-25T09:01:33.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.494+0000 7eff02838640 1 -- 172.21.15.43:0/2248467321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efefc0ce8c0 msgr2=0x7efefc0d17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:33.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.494+0000 7eff02838640 1 --2- 172.21.15.43:0/2248467321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efefc0ce8c0 0x7efefc0d17a0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7efefc0d0070 tx=0x7efef402a680 comp rx=0 tx=0).stop 2024-04-25T09:01:33.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.494+0000 7eff02838640 1 -- 172.21.15.43:0/2248467321 shutdown_connections 2024-04-25T09:01:33.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.494+0000 7eff02838640 1 --2- 172.21.15.43:0/2248467321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efefc0ce8c0 0x7efefc0d17a0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:33.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.494+0000 7eff02838640 1 --2- 172.21.15.43:0/2248467321 >> v2:172.21.15.193:3300/0 conn(0x7efefc15dec0 0x7efefc15ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:33.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.494+0000 7eff02838640 1 -- 172.21.15.43:0/2248467321 >> 172.21.15.43:0/2248467321 conn(0x7efefc09cd40 msgr2=0x7efefc0d3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:33.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.494+0000 7eff02838640 1 -- 172.21.15.43:0/2248467321 shutdown_connections 2024-04-25T09:01:33.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.494+0000 7eff02838640 1 -- 172.21.15.43:0/2248467321 wait complete. 2024-04-25T09:01:33.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.495+0000 7eff02838640 1 Processor -- start 2024-04-25T09:01:33.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.495+0000 7eff02838640 1 -- start start 2024-04-25T09:01:33.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.496+0000 7eff02838640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efefc0ce8c0 0x7efefc14af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:33.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.496+0000 7eff02838640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efefc0d04a0 0x7efefc0c75a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:33.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.496+0000 7eff02838640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efefc15dec0 0x7efefc0c7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:33.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.496+0000 7eff02838640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efefc0cee70 con 0x7efefc0d04a0 2024-04-25T09:01:33.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.496+0000 7eff02838640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efefc0cecf0 con 0x7efefc15dec0 2024-04-25T09:01:33.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.496+0000 7eff02838640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efefc0d1ce0 con 0x7efefc0ce8c0 2024-04-25T09:01:33.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.496+0000 7efefb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efefc0d04a0 0x7efefc0c75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:33.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.496+0000 7efefb7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efefc0d04a0 0x7efefc0c75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48920/0 (socket says 172.21.15.43:48920) 2024-04-25T09:01:33.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.496+0000 7efefb7fe640 1 -- 172.21.15.43:0/38649461 learned_addr learned my addr 172.21.15.43:0/38649461 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:33.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.496+0000 7eff00dd5640 1 --2- 172.21.15.43:0/38649461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efefc15dec0 0x7efefc0c7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:33.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.496+0000 7efefbfff640 1 --2- 172.21.15.43:0/38649461 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efefc0ce8c0 0x7efefc14af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:33.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.496+0000 7efefb7fe640 1 -- 172.21.15.43:0/38649461 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efefc0ce8c0 msgr2=0x7efefc14af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:33.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.496+0000 7efefb7fe640 1 --2- 172.21.15.43:0/38649461 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efefc0ce8c0 0x7efefc14af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:33.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.496+0000 7efefb7fe640 1 -- 172.21.15.43:0/38649461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efefc15dec0 msgr2=0x7efefc0c7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:33.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.496+0000 7efefb7fe640 1 --2- 172.21.15.43:0/38649461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efefc15dec0 0x7efefc0c7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:33.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.497+0000 7efefb7fe640 1 -- 172.21.15.43:0/38649461 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efefc0b7190 con 0x7efefc0d04a0 2024-04-25T09:01:33.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.497+0000 7eff00dd5640 1 --2- 172.21.15.43:0/38649461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efefc15dec0 0x7efefc0c7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:01:33.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.497+0000 7efefb7fe640 1 --2- 172.21.15.43:0/38649461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efefc0d04a0 0x7efefc0c75a0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7efeec00d5a0 tx=0x7efeec00da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:33.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.497+0000 7efef8ff9640 1 -- 172.21.15.43:0/38649461 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec019070 con 0x7efefc0d04a0 2024-04-25T09:01:33.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.497+0000 7efef8ff9640 1 -- 172.21.15.43:0/38649461 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeec004240 con 0x7efefc0d04a0 2024-04-25T09:01:33.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.497+0000 7eff02838640 1 -- 172.21.15.43:0/38649461 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efefc0b7470 con 0x7efefc0d04a0 2024-04-25T09:01:33.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.497+0000 7efef8ff9640 1 -- 172.21.15.43:0/38649461 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec003610 con 0x7efefc0d04a0 2024-04-25T09:01:33.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.497+0000 7eff02838640 1 -- 172.21.15.43:0/38649461 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efefc0b7a30 con 0x7efefc0d04a0 2024-04-25T09:01:33.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.498+0000 7eff02838640 1 -- 172.21.15.43:0/38649461 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efefc074f60 con 0x7efefc0d04a0 2024-04-25T09:01:33.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.498+0000 7efef8ff9640 1 -- 172.21.15.43:0/38649461 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efeec01c070 con 0x7efefc0d04a0 2024-04-25T09:01:33.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.499+0000 7efef8ff9640 1 --2- 172.21.15.43:0/38649461 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7efedc06b4d0 0x7efedc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:33.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.499+0000 7efef8ff9640 1 -- 172.21.15.43:0/38649461 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5609+0+0 (secure 0 0 0) 0x7efeec0899b0 con 0x7efefc0d04a0 2024-04-25T09:01:33.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.499+0000 7efefbfff640 1 --2- 172.21.15.43:0/38649461 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7efedc06b4d0 0x7efedc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:33.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.500+0000 7efefbfff640 1 --2- 172.21.15.43:0/38649461 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7efedc06b4d0 0x7efedc06d990 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7efee4000f60 tx=0x7efee4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:33.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.503+0000 7efef8ff9640 1 -- 172.21.15.43:0/38649461 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efeec0582d0 con 0x7efefc0d04a0 2024-04-25T09:01:33.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:33.725+0000 7eff02838640 1 -- 172.21.15.43:0/38649461 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7efefc08daf0 con 0x7efefc0d04a0 2024-04-25T09:01:34.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.358+0000 7efef8ff9640 1 -- 172.21.15.43:0/38649461 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v59) v1 ==== 174+0+0 (secure 0 0 0) 0x7efeec05c370 con 0x7efefc0d04a0 2024-04-25T09:01:34.347 INFO:teuthology.orchestra.run.smithi043.stderr:new fs with metadata pool 14 and data pool 15 2024-04-25T09:01:34.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.366+0000 7eff02838640 1 -- 172.21.15.43:0/38649461 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7efedc06b4d0 msgr2=0x7efedc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:34.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.366+0000 7eff02838640 1 --2- 172.21.15.43:0/38649461 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7efedc06b4d0 0x7efedc06d990 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7efee4000f60 tx=0x7efee4009040 comp rx=0 tx=0).stop 2024-04-25T09:01:34.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.366+0000 7eff02838640 1 -- 172.21.15.43:0/38649461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efefc0d04a0 msgr2=0x7efefc0c75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:34.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.366+0000 7eff02838640 1 --2- 172.21.15.43:0/38649461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efefc0d04a0 0x7efefc0c75a0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7efeec00d5a0 tx=0x7efeec00da60 comp rx=0 tx=0).stop 2024-04-25T09:01:34.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.369+0000 7eff02838640 1 -- 172.21.15.43:0/38649461 shutdown_connections 2024-04-25T09:01:34.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.369+0000 7eff02838640 1 --2- 172.21.15.43:0/38649461 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7efedc06b4d0 0x7efedc06d990 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:34.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.369+0000 7eff02838640 1 --2- 172.21.15.43:0/38649461 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efefc15dec0 0x7efefc0c7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:34.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.369+0000 7eff02838640 1 --2- 172.21.15.43:0/38649461 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efefc0d04a0 0x7efefc0c75a0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:34.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.369+0000 7eff02838640 1 --2- 172.21.15.43:0/38649461 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efefc0ce8c0 0x7efefc14af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:34.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.369+0000 7eff02838640 1 -- 172.21.15.43:0/38649461 >> 172.21.15.43:0/38649461 conn(0x7efefc09cd40 msgr2=0x7efefc15ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:34.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.376+0000 7eff02838640 1 -- 172.21.15.43:0/38649461 shutdown_connections 2024-04-25T09:01:34.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.376+0000 7eff02838640 1 -- 172.21.15.43:0/38649461 wait complete. 2024-04-25T09:01:34.382 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:01:34.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.498+0000 7fdb48838640 1 Processor -- start 2024-04-25T09:01:34.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.498+0000 7fdb48838640 1 -- start start 2024-04-25T09:01:34.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.499+0000 7fdb48838640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdb4415def0 0x7fdb4406be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:34.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.499+0000 7fdb48838640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fdb441619f0 0x7fdb44161e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:34.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.499+0000 7fdb48838640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdb44162390 0x7fdb440cfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:34.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.499+0000 7fdb48838640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb440d0e90 con 0x7fdb4415def0 2024-04-25T09:01:34.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.499+0000 7fdb48838640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb4406c390 con 0x7fdb441619f0 2024-04-25T09:01:34.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.499+0000 7fdb48838640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb44162820 con 0x7fdb44162390 2024-04-25T09:01:34.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.499+0000 7fdb4259c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdb4415def0 0x7fdb4406be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:34.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.499+0000 7fdb4259c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdb4415def0 0x7fdb4406be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48922/0 (socket says 172.21.15.43:48922) 2024-04-25T09:01:34.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.499+0000 7fdb4259c640 1 -- 172.21.15.43:0/3598309616 learned_addr learned my addr 172.21.15.43:0/3598309616 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:34.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.499+0000 7fdb41d9b640 1 --2- 172.21.15.43:0/3598309616 >> v2:172.21.15.193:3300/0 conn(0x7fdb441619f0 0x7fdb44161e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:34.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.499+0000 7fdb4259c640 1 -- 172.21.15.43:0/3598309616 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdb44162390 msgr2=0x7fdb440cfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:34.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.499+0000 7fdb42d9d640 1 --2- 172.21.15.43:0/3598309616 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdb44162390 0x7fdb440cfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:34.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.499+0000 7fdb4259c640 1 --2- 172.21.15.43:0/3598309616 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdb44162390 0x7fdb440cfa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:34.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.499+0000 7fdb4259c640 1 -- 172.21.15.43:0/3598309616 >> v2:172.21.15.193:3300/0 conn(0x7fdb441619f0 msgr2=0x7fdb44161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:34.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.499+0000 7fdb4259c640 1 --2- 172.21.15.43:0/3598309616 >> v2:172.21.15.193:3300/0 conn(0x7fdb441619f0 0x7fdb44161e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:34.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.499+0000 7fdb4259c640 1 -- 172.21.15.43:0/3598309616 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb4415e2d0 con 0x7fdb4415def0 2024-04-25T09:01:34.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.500+0000 7fdb41d9b640 1 --2- 172.21.15.43:0/3598309616 >> v2:172.21.15.193:3300/0 conn(0x7fdb441619f0 0x7fdb44161e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:34.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.500+0000 7fdb42d9d640 1 --2- 172.21.15.43:0/3598309616 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdb44162390 0x7fdb440cfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:34.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.500+0000 7fdb4259c640 1 --2- 172.21.15.43:0/3598309616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb4415def0 0x7fdb4406be50 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fdb440d2800 tx=0x7fdb3402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d8b9d8b5c23c77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:34.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.500+0000 7fdb4159a640 1 -- 172.21.15.43:0/3598309616 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb34037070 con 0x7fdb4415def0 2024-04-25T09:01:34.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.500+0000 7fdb4159a640 1 -- 172.21.15.43:0/3598309616 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb34032440 con 0x7fdb4415def0 2024-04-25T09:01:34.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.500+0000 7fdb4159a640 1 -- 172.21.15.43:0/3598309616 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb34032810 con 0x7fdb4415def0 2024-04-25T09:01:34.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.501+0000 7fdb48838640 1 -- 172.21.15.43:0/3598309616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb4415def0 msgr2=0x7fdb4406be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:34.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.501+0000 7fdb48838640 1 --2- 172.21.15.43:0/3598309616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb4415def0 0x7fdb4406be50 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fdb440d2800 tx=0x7fdb3402a200 comp rx=0 tx=0).stop 2024-04-25T09:01:34.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.501+0000 7fdb48838640 1 -- 172.21.15.43:0/3598309616 shutdown_connections 2024-04-25T09:01:34.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.501+0000 7fdb48838640 1 --2- 172.21.15.43:0/3598309616 >> v2:172.21.15.193:3300/0 conn(0x7fdb441619f0 0x7fdb44161e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:34.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.501+0000 7fdb48838640 1 --2- 172.21.15.43:0/3598309616 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdb44162390 0x7fdb440cfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:34.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.501+0000 7fdb48838640 1 --2- 172.21.15.43:0/3598309616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb4415def0 0x7fdb4406be50 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:34.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.501+0000 7fdb48838640 1 -- 172.21.15.43:0/3598309616 >> 172.21.15.43:0/3598309616 conn(0x7fdb440b32e0 msgr2=0x7fdb440b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:34.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.501+0000 7fdb48838640 1 -- 172.21.15.43:0/3598309616 shutdown_connections 2024-04-25T09:01:34.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.501+0000 7fdb48838640 1 -- 172.21.15.43:0/3598309616 wait complete. 2024-04-25T09:01:34.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.502+0000 7fdb48838640 1 Processor -- start 2024-04-25T09:01:34.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.502+0000 7fdb48838640 1 -- start start 2024-04-25T09:01:34.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.503+0000 7fdb48838640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdb4415def0 0x7fdb440b0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:34.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.503+0000 7fdb48838640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb441619f0 0x7fdb440b67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:34.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.503+0000 7fdb48838640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb44162390 0x7fdb441602e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:34.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.503+0000 7fdb48838640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb440b15e0 con 0x7fdb441619f0 2024-04-25T09:01:34.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.503+0000 7fdb48838640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb440b1300 con 0x7fdb44162390 2024-04-25T09:01:34.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.503+0000 7fdb48838640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb440b18c0 con 0x7fdb4415def0 2024-04-25T09:01:34.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.503+0000 7fdb41d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb441619f0 0x7fdb440b67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:34.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.503+0000 7fdb41d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb441619f0 0x7fdb440b67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48928/0 (socket says 172.21.15.43:48928) 2024-04-25T09:01:34.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.503+0000 7fdb41d9b640 1 -- 172.21.15.43:0/4019363007 learned_addr learned my addr 172.21.15.43:0/4019363007 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:34.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.503+0000 7fdb42d9d640 1 --2- 172.21.15.43:0/4019363007 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb44162390 0x7fdb441602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:34.494 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.503+0000 7fdb4259c640 1 --2- 172.21.15.43:0/4019363007 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdb4415def0 0x7fdb440b0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:34.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.504+0000 7fdb41d9b640 1 -- 172.21.15.43:0/4019363007 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdb4415def0 msgr2=0x7fdb440b0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:34.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.504+0000 7fdb41d9b640 1 --2- 172.21.15.43:0/4019363007 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdb4415def0 0x7fdb440b0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:34.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.504+0000 7fdb41d9b640 1 -- 172.21.15.43:0/4019363007 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb44162390 msgr2=0x7fdb441602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:34.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.504+0000 7fdb41d9b640 1 --2- 172.21.15.43:0/4019363007 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb44162390 0x7fdb441602e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:34.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.504+0000 7fdb41d9b640 1 -- 172.21.15.43:0/4019363007 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb440b5d20 con 0x7fdb441619f0 2024-04-25T09:01:34.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.504+0000 7fdb41d9b640 1 --2- 172.21.15.43:0/4019363007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb441619f0 0x7fdb440b67c0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fdb3800c820 tx=0x7fdb3800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:34.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.504+0000 7fdb32ffd640 1 -- 172.21.15.43:0/4019363007 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb38005b90 con 0x7fdb441619f0 2024-04-25T09:01:34.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.504+0000 7fdb32ffd640 1 -- 172.21.15.43:0/4019363007 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb38005d30 con 0x7fdb441619f0 2024-04-25T09:01:34.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.504+0000 7fdb48838640 1 -- 172.21.15.43:0/4019363007 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb440b08b0 con 0x7fdb441619f0 2024-04-25T09:01:34.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.504+0000 7fdb32ffd640 1 -- 172.21.15.43:0/4019363007 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb38007a30 con 0x7fdb441619f0 2024-04-25T09:01:34.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.504+0000 7fdb48838640 1 -- 172.21.15.43:0/4019363007 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb4415f700 con 0x7fdb441619f0 2024-04-25T09:01:34.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.505+0000 7fdb48838640 1 -- 172.21.15.43:0/4019363007 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb44074ef0 con 0x7fdb441619f0 2024-04-25T09:01:34.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.506+0000 7fdb32ffd640 1 -- 172.21.15.43:0/4019363007 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdb38007bd0 con 0x7fdb441619f0 2024-04-25T09:01:34.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.506+0000 7fdb32ffd640 1 --2- 172.21.15.43:0/4019363007 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdb2406b4d0 0x7fdb2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:34.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.506+0000 7fdb32ffd640 1 -- 172.21.15.43:0/4019363007 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb3808a240 con 0x7fdb441619f0 2024-04-25T09:01:34.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.506+0000 7fdb4259c640 1 --2- 172.21.15.43:0/4019363007 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdb2406b4d0 0x7fdb2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:34.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.507+0000 7fdb4259c640 1 --2- 172.21.15.43:0/4019363007 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdb2406b4d0 0x7fdb2406d990 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fdb34034000 tx=0x7fdb3402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:34.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.510+0000 7fdb32ffd640 1 -- 172.21.15.43:0/4019363007 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdb38015020 con 0x7fdb441619f0 2024-04-25T09:01:34.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.693+0000 7fdb48838640 1 -- 172.21.15.43:0/4019363007 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdb440b1b70 con 0x7fdb441619f0 2024-04-25T09:01:34.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.694+0000 7fdb32ffd640 1 -- 172.21.15.43:0/4019363007 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+16596 (secure 0 0 0) 0x7fdb38059150 con 0x7fdb441619f0 2024-04-25T09:01:34.684 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:01:34.684 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":101,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:01:33.726490+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:01:31.997813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T09:01:32.695222+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:01:34.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.696+0000 7fdb48838640 1 -- 172.21.15.43:0/4019363007 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdb2406b4d0 msgr2=0x7fdb2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:34.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.696+0000 7fdb48838640 1 --2- 172.21.15.43:0/4019363007 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdb2406b4d0 0x7fdb2406d990 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fdb34034000 tx=0x7fdb3402e000 comp rx=0 tx=0).stop 2024-04-25T09:01:34.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.696+0000 7fdb48838640 1 -- 172.21.15.43:0/4019363007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb441619f0 msgr2=0x7fdb440b67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:34.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.696+0000 7fdb48838640 1 --2- 172.21.15.43:0/4019363007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb441619f0 0x7fdb440b67c0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fdb3800c820 tx=0x7fdb3800cc20 comp rx=0 tx=0).stop 2024-04-25T09:01:34.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.696+0000 7fdb48838640 1 -- 172.21.15.43:0/4019363007 shutdown_connections 2024-04-25T09:01:34.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.696+0000 7fdb48838640 1 --2- 172.21.15.43:0/4019363007 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdb2406b4d0 0x7fdb2406d990 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:34.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.696+0000 7fdb48838640 1 --2- 172.21.15.43:0/4019363007 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb44162390 0x7fdb441602e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:34.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.696+0000 7fdb48838640 1 --2- 172.21.15.43:0/4019363007 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb441619f0 0x7fdb440b67c0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:34.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.696+0000 7fdb48838640 1 --2- 172.21.15.43:0/4019363007 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdb4415def0 0x7fdb440b0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:34.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.696+0000 7fdb48838640 1 -- 172.21.15.43:0/4019363007 >> 172.21.15.43:0/4019363007 conn(0x7fdb440b32e0 msgr2=0x7fdb4406f260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:34.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.696+0000 7fdb48838640 1 -- 172.21.15.43:0/4019363007 shutdown_connections 2024-04-25T09:01:34.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.696+0000 7fdb48838640 1 -- 172.21.15.43:0/4019363007 wait complete. 2024-04-25T09:01:34.696 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:01:34.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.808+0000 7f624ceb3640 1 Processor -- start 2024-04-25T09:01:34.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.808+0000 7f624ceb3640 1 -- start start 2024-04-25T09:01:34.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.809+0000 7f624ceb3640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62480ccbb0 0x7f62480cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:34.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.809+0000 7f624ceb3640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f62480b3b00 0x7f62480b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:34.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.809+0000 7f624ceb3640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62480b2780 con 0x7f62480cbfc0 2024-04-25T09:01:34.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.809+0000 7f624ceb3640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62480cd550 con 0x7f62480b3b00 2024-04-25T09:01:34.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.809+0000 7f624ceb3640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62480b2a60 con 0x7f62480ccbb0 2024-04-25T09:01:34.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.809+0000 7f6246d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f62480b3b00 0x7f62480b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:34.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.809+0000 7f6246d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f62480b3b00 0x7f62480b3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48944/0 (socket says 172.21.15.43:48944) 2024-04-25T09:01:34.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.809+0000 7f6246d9d640 1 -- 172.21.15.43:0/2815148667 learned_addr learned my addr 172.21.15.43:0/2815148667 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:34.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.809+0000 7f6245d9b640 1 --2- 172.21.15.43:0/2815148667 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62480ccbb0 0x7f62480cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:34.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.810+0000 7f6246d9d640 1 -- 172.21.15.43:0/2815148667 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62480ccbb0 msgr2=0x7f62480cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:34.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.810+0000 7f6246d9d640 1 --2- 172.21.15.43:0/2815148667 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62480ccbb0 0x7f62480cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:34.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.810+0000 7f6246d9d640 1 -- 172.21.15.43:0/2815148667 >> v1:172.21.15.193:6789/0 conn(0x7f62480cbfc0 legacy=0x7f624806dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:34.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.810+0000 7f6246d9d640 1 -- 172.21.15.43:0/2815148667 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62480b2d40 con 0x7f62480b3b00 2024-04-25T09:01:34.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.810+0000 7f6246d9d640 1 --2- 172.21.15.43:0/2815148667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62480b3b00 0x7f62480b3f90 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f6238009e50 tx=0x7f623802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f9d359615dff046 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:34.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.810+0000 7f624559a640 1 -- 172.21.15.43:0/2815148667 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6238036070 con 0x7f62480b3b00 2024-04-25T09:01:34.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.810+0000 7f624559a640 1 -- 172.21.15.43:0/2815148667 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6238031600 con 0x7f62480b3b00 2024-04-25T09:01:34.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.810+0000 7f624559a640 1 -- 172.21.15.43:0/2815148667 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62380319d0 con 0x7f62480b3b00 2024-04-25T09:01:34.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.811+0000 7f624ceb3640 1 -- 172.21.15.43:0/2815148667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62480b3b00 msgr2=0x7f62480b3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:34.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.811+0000 7f624ceb3640 1 --2- 172.21.15.43:0/2815148667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62480b3b00 0x7f62480b3f90 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f6238009e50 tx=0x7f623802a710 comp rx=0 tx=0).stop 2024-04-25T09:01:34.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.811+0000 7f624ceb3640 1 -- 172.21.15.43:0/2815148667 shutdown_connections 2024-04-25T09:01:34.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.811+0000 7f624ceb3640 1 --2- 172.21.15.43:0/2815148667 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62480b3b00 0x7f62480b3f90 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:34.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.811+0000 7f624ceb3640 1 --2- 172.21.15.43:0/2815148667 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62480ccbb0 0x7f62480cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:34.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.811+0000 7f624ceb3640 1 -- 172.21.15.43:0/2815148667 >> 172.21.15.43:0/2815148667 conn(0x7f62480b1480 msgr2=0x7f62480b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:34.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.811+0000 7f624ceb3640 1 -- 172.21.15.43:0/2815148667 shutdown_connections 2024-04-25T09:01:34.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.811+0000 7f624ceb3640 1 -- 172.21.15.43:0/2815148667 wait complete. 2024-04-25T09:01:34.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.812+0000 7f624ceb3640 1 Processor -- start 2024-04-25T09:01:34.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.812+0000 7f624ceb3640 1 -- start start 2024-04-25T09:01:34.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.813+0000 7f624ceb3640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62480b3b00 0x7f624814a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:34.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.813+0000 7f624ceb3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62480cbfc0 0x7f624813be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:34.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.813+0000 7f624ceb3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62480ccbb0 0x7f624813da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:34.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.813+0000 7f624ceb3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62480b35e0 con 0x7f62480ccbb0 2024-04-25T09:01:34.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.813+0000 7f624ceb3640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62480b3300 con 0x7f62480cbfc0 2024-04-25T09:01:34.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.813+0000 7f624ceb3640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62480cc3a0 con 0x7f62480b3b00 2024-04-25T09:01:34.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.813+0000 7f6246d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62480ccbb0 0x7f624813da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:34.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.813+0000 7f6246d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62480ccbb0 0x7f624813da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48960/0 (socket says 172.21.15.43:48960) 2024-04-25T09:01:34.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.813+0000 7f6246d9d640 1 -- 172.21.15.43:0/571694616 learned_addr learned my addr 172.21.15.43:0/571694616 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:34.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.813+0000 7f624659c640 1 --2- 172.21.15.43:0/571694616 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62480b3b00 0x7f624814a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:34.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.813+0000 7f6245d9b640 1 --2- 172.21.15.43:0/571694616 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62480cbfc0 0x7f624813be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:34.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.814+0000 7f6246d9d640 1 -- 172.21.15.43:0/571694616 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62480b3b00 msgr2=0x7f624814a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:34.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.814+0000 7f6246d9d640 1 --2- 172.21.15.43:0/571694616 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62480b3b00 0x7f624814a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:34.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.814+0000 7f6246d9d640 1 -- 172.21.15.43:0/571694616 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62480cbfc0 msgr2=0x7f624813be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:34.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.814+0000 7f6246d9d640 1 --2- 172.21.15.43:0/571694616 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62480cbfc0 0x7f624813be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:34.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.814+0000 7f6246d9d640 1 -- 172.21.15.43:0/571694616 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f624813c730 con 0x7f62480ccbb0 2024-04-25T09:01:34.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.814+0000 7f6246d9d640 1 --2- 172.21.15.43:0/571694616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62480ccbb0 0x7f624813da10 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f6238034040 tx=0x7f6238004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:34.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.814+0000 7f623effd640 1 -- 172.21.15.43:0/571694616 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623803f020 con 0x7f62480ccbb0 2024-04-25T09:01:34.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.814+0000 7f623effd640 1 -- 172.21.15.43:0/571694616 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6238037040 con 0x7f62480ccbb0 2024-04-25T09:01:34.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.814+0000 7f624ceb3640 1 -- 172.21.15.43:0/571694616 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f624814b100 con 0x7f62480ccbb0 2024-04-25T09:01:34.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.814+0000 7f623effd640 1 -- 172.21.15.43:0/571694616 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6238036070 con 0x7f62480ccbb0 2024-04-25T09:01:34.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.814+0000 7f624ceb3640 1 -- 172.21.15.43:0/571694616 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f624814b6c0 con 0x7f62480ccbb0 2024-04-25T09:01:34.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.815+0000 7f624ceb3640 1 -- 172.21.15.43:0/571694616 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6248067d30 con 0x7f62480ccbb0 2024-04-25T09:01:34.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.815+0000 7f623effd640 1 -- 172.21.15.43:0/571694616 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6238042020 con 0x7f62480ccbb0 2024-04-25T09:01:34.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.816+0000 7f623effd640 1 --2- 172.21.15.43:0/571694616 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f622806b500 0x7f622806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:34.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.816+0000 7f623effd640 1 -- 172.21.15.43:0/571694616 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62380aa3f0 con 0x7f62480ccbb0 2024-04-25T09:01:34.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.816+0000 7f624659c640 1 --2- 172.21.15.43:0/571694616 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f622806b500 0x7f622806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:34.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.816+0000 7f624659c640 1 --2- 172.21.15.43:0/571694616 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f622806b500 0x7f622806d9c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6240001780 tx=0x7f6240009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:34.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:34.820+0000 7f623effd640 1 -- 172.21.15.43:0/571694616 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6248067d30 con 0x7f62480ccbb0 2024-04-25T09:01:34.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.002+0000 7f624ceb3640 1 -- 172.21.15.43:0/571694616 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62480af0d0 con 0x7f62480ccbb0 2024-04-25T09:01:34.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.003+0000 7f623effd640 1 -- 172.21.15.43:0/571694616 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+16596 (secure 0 0 0) 0x7f6238049030 con 0x7f62480ccbb0 2024-04-25T09:01:34.992 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:01:34.992 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":101,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:01:33.726490+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:01:31.997813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T09:01:32.695222+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:01:34.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.005+0000 7f624ceb3640 1 -- 172.21.15.43:0/571694616 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f622806b500 msgr2=0x7f622806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:34.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.005+0000 7f624ceb3640 1 --2- 172.21.15.43:0/571694616 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f622806b500 0x7f622806d9c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f6240001780 tx=0x7f6240009040 comp rx=0 tx=0).stop 2024-04-25T09:01:34.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.005+0000 7f624ceb3640 1 -- 172.21.15.43:0/571694616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62480ccbb0 msgr2=0x7f624813da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:34.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.005+0000 7f624ceb3640 1 --2- 172.21.15.43:0/571694616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62480ccbb0 0x7f624813da10 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f6238034040 tx=0x7f6238004a40 comp rx=0 tx=0).stop 2024-04-25T09:01:34.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.005+0000 7f624ceb3640 1 -- 172.21.15.43:0/571694616 shutdown_connections 2024-04-25T09:01:34.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.005+0000 7f624ceb3640 1 --2- 172.21.15.43:0/571694616 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62480ccbb0 0x7f624813da10 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:34.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.005+0000 7f624ceb3640 1 --2- 172.21.15.43:0/571694616 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f622806b500 0x7f622806d9c0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:34.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.005+0000 7f624ceb3640 1 --2- 172.21.15.43:0/571694616 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62480cbfc0 0x7f624813be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:34.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.005+0000 7f624ceb3640 1 --2- 172.21.15.43:0/571694616 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62480b3b00 0x7f624814a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:34.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.005+0000 7f624ceb3640 1 -- 172.21.15.43:0/571694616 >> 172.21.15.43:0/571694616 conn(0x7f62480b1480 msgr2=0x7f62480cea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:34.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.005+0000 7f624ceb3640 1 -- 172.21.15.43:0/571694616 shutdown_connections 2024-04-25T09:01:34.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.005+0000 7f624ceb3640 1 -- 172.21.15.43:0/571694616 wait complete. 2024-04-25T09:01:35.005 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:01:35.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.120+0000 7ff6090b3640 1 Processor -- start 2024-04-25T09:01:35.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.120+0000 7ff6090b3640 1 -- start start 2024-04-25T09:01:35.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.121+0000 7ff6090b3640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff604161310 0x7ff6040d4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:35.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.121+0000 7ff6090b3640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6040d45c0 con 0x7ff6040d0120 2024-04-25T09:01:35.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.121+0000 7ff6090b3640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6040cf030 con 0x7ff6040d0c70 2024-04-25T09:01:35.110 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.121+0000 7ff6090b3640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6040d4870 con 0x7ff604161310 2024-04-25T09:01:35.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.121+0000 7ff602d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff604161310 0x7ff6040d4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:35.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.121+0000 7ff60259c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff6040d0120 0x7ff6040d0560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:37552/0 (socket says 172.21.15.43:37552) 2024-04-25T09:01:35.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.121+0000 7ff60259c640 1 -- 172.21.15.43:0/2177475835 learned_addr learned my addr 172.21.15.43:0/2177475835 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:35.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.121+0000 7ff601d9b640 1 -- 172.21.15.43:0/2177475835 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1404719592 0 0) 0x7ff6040d45c0 con 0x7ff6040d0120 2024-04-25T09:01:35.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.121+0000 7ff601d9b640 1 -- 172.21.15.43:0/2177475835 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5f0003520 con 0x7ff6040d0120 2024-04-25T09:01:35.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.121+0000 7ff602d9d640 1 -- 172.21.15.43:0/2177475835 >> v1:172.21.15.193:6789/0 conn(0x7ff6040d0c70 legacy=0x7ff6040c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:35.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.121+0000 7ff602d9d640 1 -- 172.21.15.43:0/2177475835 >> v1:172.21.15.43:6789/0 conn(0x7ff6040d0120 legacy=0x7ff6040d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:35.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.121+0000 7ff602d9d640 1 -- 172.21.15.43:0/2177475835 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6040cf310 con 0x7ff604161310 2024-04-25T09:01:35.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.122+0000 7ff602d9d640 1 --2- 172.21.15.43:0/2177475835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff604161310 0x7ff6040d4080 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4009d90 tx=0x7ff5f402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5186d6aa90d4a702 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:35.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.122+0000 7ff601d9b640 1 -- 172.21.15.43:0/2177475835 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4007740 con 0x7ff604161310 2024-04-25T09:01:35.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.122+0000 7ff601d9b640 1 -- 172.21.15.43:0/2177475835 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f40078e0 con 0x7ff604161310 2024-04-25T09:01:35.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.122+0000 7ff601d9b640 1 -- 172.21.15.43:0/2177475835 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4007cb0 con 0x7ff604161310 2024-04-25T09:01:35.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.123+0000 7ff6090b3640 1 -- 172.21.15.43:0/2177475835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff604161310 msgr2=0x7ff6040d4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:35.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.123+0000 7ff6090b3640 1 --2- 172.21.15.43:0/2177475835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff604161310 0x7ff6040d4080 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4009d90 tx=0x7ff5f402a6f0 comp rx=0 tx=0).stop 2024-04-25T09:01:35.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.123+0000 7ff6090b3640 1 -- 172.21.15.43:0/2177475835 shutdown_connections 2024-04-25T09:01:35.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.123+0000 7ff6090b3640 1 --2- 172.21.15.43:0/2177475835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff604161310 0x7ff6040d4080 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:35.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.123+0000 7ff6090b3640 1 -- 172.21.15.43:0/2177475835 >> 172.21.15.43:0/2177475835 conn(0x7ff60409cd20 msgr2=0x7ff6040d3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:35.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.123+0000 7ff6090b3640 1 -- 172.21.15.43:0/2177475835 shutdown_connections 2024-04-25T09:01:35.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.123+0000 7ff6090b3640 1 -- 172.21.15.43:0/2177475835 wait complete. 2024-04-25T09:01:35.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.124+0000 7ff6090b3640 1 Processor -- start 2024-04-25T09:01:35.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.124+0000 7ff6090b3640 1 -- start start 2024-04-25T09:01:35.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.125+0000 7ff6090b3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6040d0120 0x7ff6041f40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:35.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.125+0000 7ff6090b3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff6040d0c70 0x7ff6041f6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:35.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.125+0000 7ff6090b3640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff604161310 0x7ff6041f2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:35.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.125+0000 7ff6090b3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6040c7f70 con 0x7ff6040d0120 2024-04-25T09:01:35.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.125+0000 7ff6090b3640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6041616f0 con 0x7ff6040d0c70 2024-04-25T09:01:35.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.125+0000 7ff6090b3640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6040c80f0 con 0x7ff604161310 2024-04-25T09:01:35.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.125+0000 7ff602d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6040d0120 0x7ff6041f40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:35.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.125+0000 7ff602d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6040d0120 0x7ff6041f40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48974/0 (socket says 172.21.15.43:48974) 2024-04-25T09:01:35.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.125+0000 7ff602d9d640 1 -- 172.21.15.43:0/610458316 learned_addr learned my addr 172.21.15.43:0/610458316 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:35.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.125+0000 7ff60259c640 1 --2- 172.21.15.43:0/610458316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff6040d0c70 0x7ff6041f6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:35.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.125+0000 7ff60359e640 1 --2- 172.21.15.43:0/610458316 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff604161310 0x7ff6041f2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:35.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.125+0000 7ff602d9d640 1 -- 172.21.15.43:0/610458316 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff604161310 msgr2=0x7ff6041f2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:35.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.125+0000 7ff602d9d640 1 --2- 172.21.15.43:0/610458316 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff604161310 0x7ff6041f2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:35.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.125+0000 7ff602d9d640 1 -- 172.21.15.43:0/610458316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff6040d0c70 msgr2=0x7ff6041f6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:35.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.125+0000 7ff602d9d640 1 --2- 172.21.15.43:0/610458316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff6040d0c70 0x7ff6041f6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:35.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.125+0000 7ff602d9d640 1 -- 172.21.15.43:0/610458316 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6041f7340 con 0x7ff6040d0120 2024-04-25T09:01:35.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.126+0000 7ff602d9d640 1 --2- 172.21.15.43:0/610458316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6040d0120 0x7ff6041f40c0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4009eb0 tx=0x7ff5f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:35.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.126+0000 7ff5fb7fe640 1 -- 172.21.15.43:0/610458316 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f403f070 con 0x7ff6040d0120 2024-04-25T09:01:35.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.126+0000 7ff5fb7fe640 1 -- 172.21.15.43:0/610458316 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f4031c40 con 0x7ff6040d0120 2024-04-25T09:01:35.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.126+0000 7ff6090b3640 1 -- 172.21.15.43:0/610458316 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6041f7620 con 0x7ff6040d0120 2024-04-25T09:01:35.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.126+0000 7ff5fb7fe640 1 -- 172.21.15.43:0/610458316 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4009450 con 0x7ff6040d0120 2024-04-25T09:01:35.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.126+0000 7ff6090b3640 1 -- 172.21.15.43:0/610458316 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6040cac50 con 0x7ff6040d0120 2024-04-25T09:01:35.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.127+0000 7ff6090b3640 1 -- 172.21.15.43:0/610458316 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff604074f60 con 0x7ff6040d0120 2024-04-25T09:01:35.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.127+0000 7ff5fb7fe640 1 -- 172.21.15.43:0/610458316 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff5f4043410 con 0x7ff6040d0120 2024-04-25T09:01:35.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.128+0000 7ff5fb7fe640 1 --2- 172.21.15.43:0/610458316 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff5e406b4d0 0x7ff5e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:35.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.128+0000 7ff5fb7fe640 1 -- 172.21.15.43:0/610458316 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff5f40ab660 con 0x7ff6040d0120 2024-04-25T09:01:35.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.128+0000 7ff60259c640 1 --2- 172.21.15.43:0/610458316 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff5e406b4d0 0x7ff5e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:35.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.129+0000 7ff60259c640 1 --2- 172.21.15.43:0/610458316 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff5e406b4d0 0x7ff5e406d990 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff6040b8310 tx=0x7ff5ec009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:35.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.132+0000 7ff5fb7fe640 1 -- 172.21.15.43:0/610458316 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff5f407a4e0 con 0x7ff6040d0120 2024-04-25T09:01:35.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.349+0000 7ff6090b3640 1 -- 172.21.15.43:0/610458316 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff6040caa20 con 0x7ff6040d0120 2024-04-25T09:01:35.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.356+0000 7ff5fb7fe640 1 -- 172.21.15.43:0/610458316 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v60) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff5f4079de0 con 0x7ff6040d0120 2024-04-25T09:01:35.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.358+0000 7ff6090b3640 1 -- 172.21.15.43:0/610458316 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff5e406b4d0 msgr2=0x7ff5e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:35.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.358+0000 7ff6090b3640 1 --2- 172.21.15.43:0/610458316 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff5e406b4d0 0x7ff5e406d990 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff6040b8310 tx=0x7ff5ec009040 comp rx=0 tx=0).stop 2024-04-25T09:01:35.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.358+0000 7ff6090b3640 1 -- 172.21.15.43:0/610458316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6040d0120 msgr2=0x7ff6041f40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:35.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.358+0000 7ff6090b3640 1 --2- 172.21.15.43:0/610458316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6040d0120 0x7ff6041f40c0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4009eb0 tx=0x7ff5f4004a40 comp rx=0 tx=0).stop 2024-04-25T09:01:35.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.358+0000 7ff6090b3640 1 -- 172.21.15.43:0/610458316 shutdown_connections 2024-04-25T09:01:35.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.358+0000 7ff6090b3640 1 --2- 172.21.15.43:0/610458316 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff604161310 0x7ff6041f2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:35.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.358+0000 7ff6090b3640 1 --2- 172.21.15.43:0/610458316 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff5e406b4d0 0x7ff5e406d990 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:35.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.358+0000 7ff6090b3640 1 --2- 172.21.15.43:0/610458316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff6040d0c70 0x7ff6041f6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:35.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.358+0000 7ff6090b3640 1 --2- 172.21.15.43:0/610458316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6040d0120 0x7ff6041f40c0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:35.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.358+0000 7ff6090b3640 1 -- 172.21.15.43:0/610458316 >> 172.21.15.43:0/610458316 conn(0x7ff60409cd20 msgr2=0x7ff6040c9d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:35.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.359+0000 7ff6090b3640 1 -- 172.21.15.43:0/610458316 shutdown_connections 2024-04-25T09:01:35.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.359+0000 7ff6090b3640 1 -- 172.21.15.43:0/610458316 wait complete. 2024-04-25T09:01:35.360 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T09:01:35.360 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:01:35.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.475+0000 7f7ac9d6a640 1 Processor -- start 2024-04-25T09:01:35.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.475+0000 7f7ac9d6a640 1 -- start start 2024-04-25T09:01:35.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.476+0000 7f7ac9d6a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7ac40d09d0 0x7f7ac406b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:35.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.476+0000 7f7ac9d6a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ac415e0e0 con 0x7f7ac40d25d0 2024-04-25T09:01:35.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.476+0000 7f7ac9d6a640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ac40d2a40 con 0x7f7ac415dd00 2024-04-25T09:01:35.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.476+0000 7f7ac9d6a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ac40d15c0 con 0x7f7ac40d09d0 2024-04-25T09:01:35.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.476+0000 7f7ac3fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7ac40d09d0 0x7f7ac406b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:35.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.476+0000 7f7ac2ffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f7ac40d25d0 0x7f7ac40d02c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:37558/0 (socket says 172.21.15.43:37558) 2024-04-25T09:01:35.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.476+0000 7f7ac2ffd640 1 -- 172.21.15.43:0/3021558294 learned_addr learned my addr 172.21.15.43:0/3021558294 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:35.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.476+0000 7f7ac3fff640 1 -- 172.21.15.43:0/3021558294 >> v1:172.21.15.193:6789/0 conn(0x7f7ac415dd00 legacy=0x7f7ac40cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:35.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.476+0000 7f7ac3fff640 1 -- 172.21.15.43:0/3021558294 >> v1:172.21.15.43:6789/0 conn(0x7f7ac40d25d0 legacy=0x7f7ac40d02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:35.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.476+0000 7f7ac27fc640 1 -- 172.21.15.43:0/3021558294 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2751370097 0 0) 0x7f7ac415e0e0 con 0x7f7ac40d25d0 2024-04-25T09:01:35.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.476+0000 7f7ac3fff640 1 -- 172.21.15.43:0/3021558294 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac40cf7b0 con 0x7f7ac40d09d0 2024-04-25T09:01:35.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.477+0000 7f7ac3fff640 1 --2- 172.21.15.43:0/3021558294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ac40d09d0 0x7f7ac406b040 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f7ac40d2c10 tx=0x7f7abc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2863771712b1de60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:35.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.477+0000 7f7ac27fc640 1 -- 172.21.15.43:0/3021558294 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7abc007740 con 0x7f7ac40d09d0 2024-04-25T09:01:35.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.477+0000 7f7ac27fc640 1 -- 172.21.15.43:0/3021558294 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7abc0078e0 con 0x7f7ac40d09d0 2024-04-25T09:01:35.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.477+0000 7f7ac27fc640 1 -- 172.21.15.43:0/3021558294 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7abc007cb0 con 0x7f7ac40d09d0 2024-04-25T09:01:35.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.477+0000 7f7ac9d6a640 1 -- 172.21.15.43:0/3021558294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ac40d09d0 msgr2=0x7f7ac406b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:35.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.477+0000 7f7ac9d6a640 1 --2- 172.21.15.43:0/3021558294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ac40d09d0 0x7f7ac406b040 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f7ac40d2c10 tx=0x7f7abc02a6a0 comp rx=0 tx=0).stop 2024-04-25T09:01:35.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.478+0000 7f7ac9d6a640 1 -- 172.21.15.43:0/3021558294 shutdown_connections 2024-04-25T09:01:35.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.478+0000 7f7ac9d6a640 1 --2- 172.21.15.43:0/3021558294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ac40d09d0 0x7f7ac406b040 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:35.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.478+0000 7f7ac9d6a640 1 -- 172.21.15.43:0/3021558294 >> 172.21.15.43:0/3021558294 conn(0x7f7ac409cd20 msgr2=0x7f7ac40d39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:35.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.478+0000 7f7ac9d6a640 1 -- 172.21.15.43:0/3021558294 shutdown_connections 2024-04-25T09:01:35.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.478+0000 7f7ac9d6a640 1 -- 172.21.15.43:0/3021558294 wait complete. 2024-04-25T09:01:35.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.479+0000 7f7ac9d6a640 1 Processor -- start 2024-04-25T09:01:35.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.479+0000 7f7ac9d6a640 1 -- start start 2024-04-25T09:01:35.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.480+0000 7f7ac9d6a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7ac40d09d0 0x7f7ac40c97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:35.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.480+0000 7f7ac9d6a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ac40d25d0 0x7f7ac415bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:35.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.480+0000 7f7ac9d6a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ac415dd00 0x7f7ac4154c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:35.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.480+0000 7f7ac9d6a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac40d1dd0 con 0x7f7ac415dd00 2024-04-25T09:01:35.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.480+0000 7f7ac9d6a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac40d1af0 con 0x7f7ac40d25d0 2024-04-25T09:01:35.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.480+0000 7f7ac9d6a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ac40d20b0 con 0x7f7ac40d09d0 2024-04-25T09:01:35.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.480+0000 7f7ac3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ac415dd00 0x7f7ac4154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:35.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.480+0000 7f7ac3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ac415dd00 0x7f7ac4154c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49000/0 (socket says 172.21.15.43:49000) 2024-04-25T09:01:35.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.480+0000 7f7ac3fff640 1 -- 172.21.15.43:0/3774705645 learned_addr learned my addr 172.21.15.43:0/3774705645 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:35.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.480+0000 7f7ac2ffd640 1 --2- 172.21.15.43:0/3774705645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ac40d25d0 0x7f7ac415bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:35.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.480+0000 7f7ac37fe640 1 --2- 172.21.15.43:0/3774705645 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7ac40d09d0 0x7f7ac40c97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:35.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.480+0000 7f7ac3fff640 1 -- 172.21.15.43:0/3774705645 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7ac40d09d0 msgr2=0x7f7ac40c97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:35.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.480+0000 7f7ac3fff640 1 --2- 172.21.15.43:0/3774705645 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7ac40d09d0 0x7f7ac40c97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:35.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.480+0000 7f7ac3fff640 1 -- 172.21.15.43:0/3774705645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ac40d25d0 msgr2=0x7f7ac415bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:35.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.480+0000 7f7ac3fff640 1 --2- 172.21.15.43:0/3774705645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ac40d25d0 0x7f7ac415bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:35.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.480+0000 7f7ac3fff640 1 -- 172.21.15.43:0/3774705645 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac40b7110 con 0x7f7ac415dd00 2024-04-25T09:01:35.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.480+0000 7f7ac37fe640 1 --2- 172.21.15.43:0/3774705645 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7ac40d09d0 0x7f7ac40c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:35.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.480+0000 7f7ac3fff640 1 --2- 172.21.15.43:0/3774705645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ac415dd00 0x7f7ac4154c00 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f7abc009e80 tx=0x7f7abc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:35.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.480+0000 7f7aabfff640 1 -- 172.21.15.43:0/3774705645 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7abc040070 con 0x7f7ac415dd00 2024-04-25T09:01:35.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.480+0000 7f7aabfff640 1 -- 172.21.15.43:0/3774705645 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7abc031d00 con 0x7f7ac415dd00 2024-04-25T09:01:35.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.480+0000 7f7ac9d6a640 1 -- 172.21.15.43:0/3774705645 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ac40b73f0 con 0x7f7ac415dd00 2024-04-25T09:01:35.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.480+0000 7f7aabfff640 1 -- 172.21.15.43:0/3774705645 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7abc003bd0 con 0x7f7ac415dd00 2024-04-25T09:01:35.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.481+0000 7f7ac9d6a640 1 -- 172.21.15.43:0/3774705645 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ac40b79b0 con 0x7f7ac415dd00 2024-04-25T09:01:35.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.481+0000 7f7ac9d6a640 1 -- 172.21.15.43:0/3774705645 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ac4074f60 con 0x7f7ac415dd00 2024-04-25T09:01:35.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.482+0000 7f7aabfff640 1 -- 172.21.15.43:0/3774705645 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7abc039040 con 0x7f7ac415dd00 2024-04-25T09:01:35.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.482+0000 7f7aabfff640 1 --2- 172.21.15.43:0/3774705645 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7ab006b4d0 0x7f7ab006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:35.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.483+0000 7f7aabfff640 1 -- 172.21.15.43:0/3774705645 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7abc0ab160 con 0x7f7ac415dd00 2024-04-25T09:01:35.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.483+0000 7f7ac37fe640 1 --2- 172.21.15.43:0/3774705645 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7ab006b4d0 0x7f7ab006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:35.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.483+0000 7f7ac37fe640 1 --2- 172.21.15.43:0/3774705645 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7ab006b4d0 0x7f7ab006d990 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4001730 tx=0x7f7ab4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:35.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.487+0000 7f7aabfff640 1 -- 172.21.15.43:0/3774705645 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7abc07a070 con 0x7f7ac415dd00 2024-04-25T09:01:35.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:35.705+0000 7f7ac9d6a640 1 -- 172.21.15.43:0/3774705645 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f7ac40b7b80 con 0x7f7ac415dd00 2024-04-25T09:01:36.327 INFO:tasks.ceph.mds.b.smithi193.stderr:2024-04-25T09:01:36.338+0000 7f348c682640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:01:36.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.367+0000 7f7aabfff640 1 -- 172.21.15.43:0/3774705645 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v61) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7abc079970 con 0x7f7ac415dd00 2024-04-25T09:01:36.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.370+0000 7f7ac9d6a640 1 -- 172.21.15.43:0/3774705645 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7ab006b4d0 msgr2=0x7f7ab006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:36.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.370+0000 7f7ac9d6a640 1 --2- 172.21.15.43:0/3774705645 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7ab006b4d0 0x7f7ab006d990 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f7ab4001730 tx=0x7f7ab4009040 comp rx=0 tx=0).stop 2024-04-25T09:01:36.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.370+0000 7f7ac9d6a640 1 -- 172.21.15.43:0/3774705645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ac415dd00 msgr2=0x7f7ac4154c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:36.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.370+0000 7f7ac9d6a640 1 --2- 172.21.15.43:0/3774705645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ac415dd00 0x7f7ac4154c00 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f7abc009e80 tx=0x7f7abc004a40 comp rx=0 tx=0).stop 2024-04-25T09:01:36.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.370+0000 7f7ac9d6a640 1 -- 172.21.15.43:0/3774705645 shutdown_connections 2024-04-25T09:01:36.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.370+0000 7f7ac9d6a640 1 --2- 172.21.15.43:0/3774705645 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ac415dd00 0x7f7ac4154c00 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:36.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.370+0000 7f7ac9d6a640 1 --2- 172.21.15.43:0/3774705645 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7ab006b4d0 0x7f7ab006d990 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:36.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.370+0000 7f7ac9d6a640 1 --2- 172.21.15.43:0/3774705645 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ac40d25d0 0x7f7ac415bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:36.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.370+0000 7f7ac9d6a640 1 --2- 172.21.15.43:0/3774705645 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7ac40d09d0 0x7f7ac40c97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:36.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.370+0000 7f7ac9d6a640 1 -- 172.21.15.43:0/3774705645 >> 172.21.15.43:0/3774705645 conn(0x7f7ac409cd20 msgr2=0x7f7ac40b3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:36.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.370+0000 7f7ac9d6a640 1 -- 172.21.15.43:0/3774705645 shutdown_connections 2024-04-25T09:01:36.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.370+0000 7f7ac9d6a640 1 -- 172.21.15.43:0/3774705645 wait complete. 2024-04-25T09:01:36.373 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:01:36.373 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:01:36.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.493+0000 7f4439c76640 1 Processor -- start 2024-04-25T09:01:36.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.493+0000 7f4439c76640 1 -- start start 2024-04-25T09:01:36.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.494+0000 7f4439c76640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f443415f4e0 0x7f44340d21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:36.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.494+0000 7f4439c76640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f443406e530 0x7f443406e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:36.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.494+0000 7f4439c76640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44340cef00 con 0x7f443415f8c0 2024-04-25T09:01:36.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.494+0000 7f4439c76640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44340cc5b0 con 0x7f443406e530 2024-04-25T09:01:36.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.494+0000 7f4439c76640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44340cc890 con 0x7f443415f4e0 2024-04-25T09:01:36.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.494+0000 7f4433fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f443406e530 0x7f443406e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:36.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.494+0000 7f4433fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f443406e530 0x7f443406e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49008/0 (socket says 172.21.15.43:49008) 2024-04-25T09:01:36.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.494+0000 7f4433fff640 1 -- 172.21.15.43:0/3259807730 learned_addr learned my addr 172.21.15.43:0/3259807730 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:36.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.494+0000 7f44337fe640 1 --2- 172.21.15.43:0/3259807730 >> v2:172.21.15.193:3300/0 conn(0x7f443415f4e0 0x7f44340d21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:36.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.494+0000 7f4433fff640 1 -- 172.21.15.43:0/3259807730 >> v2:172.21.15.193:3300/0 conn(0x7f443415f4e0 msgr2=0x7f44340d21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:36.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.494+0000 7f4433fff640 1 --2- 172.21.15.43:0/3259807730 >> v2:172.21.15.193:3300/0 conn(0x7f443415f4e0 0x7f44340d21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:36.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.494+0000 7f4433fff640 1 -- 172.21.15.43:0/3259807730 >> v1:172.21.15.193:6789/0 conn(0x7f443415f8c0 legacy=0x7f44340cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:36.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.494+0000 7f4433fff640 1 -- 172.21.15.43:0/3259807730 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44340ccb70 con 0x7f443406e530 2024-04-25T09:01:36.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.495+0000 7f4433fff640 1 --2- 172.21.15.43:0/3259807730 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f443406e530 0x7f443406e9c0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f44340b4260 tx=0x7f442c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e279e54ad795217a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:36.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.495+0000 7f44327fc640 1 -- 172.21.15.43:0/3259807730 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442c036070 con 0x7f443406e530 2024-04-25T09:01:36.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.495+0000 7f44327fc640 1 -- 172.21.15.43:0/3259807730 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f442c031600 con 0x7f443406e530 2024-04-25T09:01:36.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.495+0000 7f44327fc640 1 -- 172.21.15.43:0/3259807730 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442c0319d0 con 0x7f443406e530 2024-04-25T09:01:36.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.495+0000 7f4439c76640 1 -- 172.21.15.43:0/3259807730 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f443406e530 msgr2=0x7f443406e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:36.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.495+0000 7f4439c76640 1 --2- 172.21.15.43:0/3259807730 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f443406e530 0x7f443406e9c0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f44340b4260 tx=0x7f442c02a6f0 comp rx=0 tx=0).stop 2024-04-25T09:01:36.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.496+0000 7f4439c76640 1 -- 172.21.15.43:0/3259807730 shutdown_connections 2024-04-25T09:01:36.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.496+0000 7f4439c76640 1 --2- 172.21.15.43:0/3259807730 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f443406e530 0x7f443406e9c0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:36.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.496+0000 7f4439c76640 1 --2- 172.21.15.43:0/3259807730 >> v2:172.21.15.193:3300/0 conn(0x7f443415f4e0 0x7f44340d21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:36.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.496+0000 7f4439c76640 1 -- 172.21.15.43:0/3259807730 >> 172.21.15.43:0/3259807730 conn(0x7f44340d1700 msgr2=0x7f44340d1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:36.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.496+0000 7f4439c76640 1 -- 172.21.15.43:0/3259807730 shutdown_connections 2024-04-25T09:01:36.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.496+0000 7f4439c76640 1 -- 172.21.15.43:0/3259807730 wait complete. 2024-04-25T09:01:36.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.497+0000 7f4439c76640 1 Processor -- start 2024-04-25T09:01:36.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.497+0000 7f4439c76640 1 -- start start 2024-04-25T09:01:36.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.498+0000 7f4439c76640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f443406e530 0x7f44340c5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:36.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.498+0000 7f4439c76640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f443415f4e0 0x7f443414daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:36.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.498+0000 7f4439c76640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f443415f8c0 0x7f443415a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:36.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.498+0000 7f4439c76640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443406ef00 con 0x7f443415f4e0 2024-04-25T09:01:36.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.498+0000 7f4439c76640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443415fd70 con 0x7f443415f8c0 2024-04-25T09:01:36.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.498+0000 7f4439c76640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f443406f080 con 0x7f443406e530 2024-04-25T09:01:36.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.498+0000 7f4432ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f443415f4e0 0x7f443414daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:36.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.498+0000 7f4432ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f443415f4e0 0x7f443414daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49014/0 (socket says 172.21.15.43:49014) 2024-04-25T09:01:36.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.498+0000 7f4432ffd640 1 -- 172.21.15.43:0/1736995022 learned_addr learned my addr 172.21.15.43:0/1736995022 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:36.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.498+0000 7f4433fff640 1 --2- 172.21.15.43:0/1736995022 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f443415f8c0 0x7f443415a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:36.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.498+0000 7f44337fe640 1 --2- 172.21.15.43:0/1736995022 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f443406e530 0x7f44340c5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:36.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.498+0000 7f4432ffd640 1 -- 172.21.15.43:0/1736995022 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f443406e530 msgr2=0x7f44340c5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:36.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.498+0000 7f4432ffd640 1 --2- 172.21.15.43:0/1736995022 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f443406e530 0x7f44340c5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:36.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.498+0000 7f4432ffd640 1 -- 172.21.15.43:0/1736995022 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f443415f8c0 msgr2=0x7f443415a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:36.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.498+0000 7f4432ffd640 1 --2- 172.21.15.43:0/1736995022 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f443415f8c0 0x7f443415a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:36.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.498+0000 7f4432ffd640 1 -- 172.21.15.43:0/1736995022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44340c5c20 con 0x7f443415f4e0 2024-04-25T09:01:36.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.498+0000 7f4432ffd640 1 --2- 172.21.15.43:0/1736995022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f443415f4e0 0x7f443414daf0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f442800b3f0 tx=0x7f442800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:36.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.498+0000 7f441bfff640 1 -- 172.21.15.43:0/1736995022 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4428015020 con 0x7f443415f4e0 2024-04-25T09:01:36.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.498+0000 7f441bfff640 1 -- 172.21.15.43:0/1736995022 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4428011830 con 0x7f443415f4e0 2024-04-25T09:01:36.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.498+0000 7f441bfff640 1 -- 172.21.15.43:0/1736995022 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4428011c20 con 0x7f443415f4e0 2024-04-25T09:01:36.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.499+0000 7f4439c76640 1 -- 172.21.15.43:0/1736995022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44340c7ce0 con 0x7f443415f4e0 2024-04-25T09:01:36.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.499+0000 7f4439c76640 1 -- 172.21.15.43:0/1736995022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f443414e030 con 0x7f443415f4e0 2024-04-25T09:01:36.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.500+0000 7f4439c76640 1 -- 172.21.15.43:0/1736995022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4434074ef0 con 0x7f443415f4e0 2024-04-25T09:01:36.490 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.500+0000 7f441bfff640 1 -- 172.21.15.43:0/1736995022 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4428018020 con 0x7f443415f4e0 2024-04-25T09:01:36.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.500+0000 7f441bfff640 1 --2- 172.21.15.43:0/1736995022 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f442006b4d0 0x7f442006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:36.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.501+0000 7f441bfff640 1 -- 172.21.15.43:0/1736995022 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4428089cf0 con 0x7f443415f4e0 2024-04-25T09:01:36.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.501+0000 7f44337fe640 1 --2- 172.21.15.43:0/1736995022 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f442006b4d0 0x7f442006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:36.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.501+0000 7f44337fe640 1 --2- 172.21.15.43:0/1736995022 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f442006b4d0 0x7f442006d990 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f4424006fd0 tx=0x7f4424008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:36.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.505+0000 7f441bfff640 1 -- 172.21.15.43:0/1736995022 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f442801f030 con 0x7f443415f4e0 2024-04-25T09:01:36.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.725+0000 7f4439c76640 1 -- 172.21.15.43:0/1736995022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f44340b7680 con 0x7f443415f4e0 2024-04-25T09:01:36.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.725+0000 7f441bfff640 1 -- 172.21.15.43:0/1736995022 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f4428058510 con 0x7f443415f4e0 2024-04-25T09:01:36.714 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:01:36.714 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":61,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2150177203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2150177203},{"type":"v1","addr":"172.21.15.193:6837","nonce":2150177203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":8079,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/621488642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":621488642},{"type":"v1","addr":"172.21.15.43:6834","nonce":621488642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8082,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1524395840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1524395840},{"type":"v1","addr":"172.21.15.43:6835","nonce":1524395840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:01:33.726790+0000","modified":"2024-04-25T09:01:35.706689+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8094},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8094":{"gid":8094,"name":"b","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.193:6836/853943267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":853943267},{"type":"v1","addr":"172.21.15.193:6836","nonce":853943267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:01:36.715 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 61 2024-04-25T09:01:36.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.727+0000 7f4439c76640 1 -- 172.21.15.43:0/1736995022 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f442006b4d0 msgr2=0x7f442006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:36.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.727+0000 7f4439c76640 1 --2- 172.21.15.43:0/1736995022 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f442006b4d0 0x7f442006d990 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f4424006fd0 tx=0x7f4424008040 comp rx=0 tx=0).stop 2024-04-25T09:01:36.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.727+0000 7f4439c76640 1 -- 172.21.15.43:0/1736995022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f443415f4e0 msgr2=0x7f443414daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:36.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.727+0000 7f4439c76640 1 --2- 172.21.15.43:0/1736995022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f443415f4e0 0x7f443414daf0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f442800b3f0 tx=0x7f442800b8b0 comp rx=0 tx=0).stop 2024-04-25T09:01:36.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.727+0000 7f4439c76640 1 -- 172.21.15.43:0/1736995022 shutdown_connections 2024-04-25T09:01:36.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.727+0000 7f4439c76640 1 --2- 172.21.15.43:0/1736995022 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f442006b4d0 0x7f442006d990 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:36.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.727+0000 7f4439c76640 1 --2- 172.21.15.43:0/1736995022 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f443415f8c0 0x7f443415a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:36.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.727+0000 7f4439c76640 1 --2- 172.21.15.43:0/1736995022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f443415f4e0 0x7f443414daf0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:36.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.727+0000 7f4439c76640 1 --2- 172.21.15.43:0/1736995022 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f443406e530 0x7f44340c5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:36.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.727+0000 7f4439c76640 1 -- 172.21.15.43:0/1736995022 >> 172.21.15.43:0/1736995022 conn(0x7f44340d1700 msgr2=0x7f44340b31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:36.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.728+0000 7f4439c76640 1 -- 172.21.15.43:0/1736995022 shutdown_connections 2024-04-25T09:01:36.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.728+0000 7f4439c76640 1 -- 172.21.15.43:0/1736995022 wait complete. 2024-04-25T09:01:36.728 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:01:36.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.840+0000 7f4ab61d5640 1 Processor -- start 2024-04-25T09:01:36.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.840+0000 7f4ab61d5640 1 -- start start 2024-04-25T09:01:36.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.841+0000 7f4ab61d5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4ab00cbfc0 0x7f4ab006dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:36.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.841+0000 7f4ab61d5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ab0160400 con 0x7f4ab00cd010 2024-04-25T09:01:36.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.841+0000 7f4ab61d5640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ab00b2780 con 0x7f4ab00ccbb0 2024-04-25T09:01:36.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.841+0000 7f4ab61d5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab00b2a60 con 0x7f4ab00cbfc0 2024-04-25T09:01:36.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.841+0000 7f4aaffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4ab00cd010 0x7f4ab015fcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:37568/0 (socket says 172.21.15.43:37568) 2024-04-25T09:01:36.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.841+0000 7f4aaffff640 1 -- 172.21.15.43:0/4257332867 learned_addr learned my addr 172.21.15.43:0/4257332867 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:36.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.841+0000 7f4aaf7fe640 1 --2- 172.21.15.43:0/4257332867 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4ab00cbfc0 0x7f4ab006dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:36.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.841+0000 7f4aae7fc640 1 -- 172.21.15.43:0/4257332867 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2330451113 0 0) 0x7f4ab0160400 con 0x7f4ab00cd010 2024-04-25T09:01:36.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.841+0000 7f4aae7fc640 1 -- 172.21.15.43:0/4257332867 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a98003520 con 0x7f4ab00cd010 2024-04-25T09:01:36.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.841+0000 7f4aae7fc640 1 -- 172.21.15.43:0/4257332867 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1784131644 0 0) 0x7f4a98003520 con 0x7f4ab00cd010 2024-04-25T09:01:36.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.841+0000 7f4aae7fc640 1 -- 172.21.15.43:0/4257332867 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4ab00cbfc0 msgr2=0x7f4ab006dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:36.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.841+0000 7f4aae7fc640 1 --2- 172.21.15.43:0/4257332867 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4ab00cbfc0 0x7f4ab006dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:36.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.841+0000 7f4aae7fc640 1 -- 172.21.15.43:0/4257332867 >> v1:172.21.15.193:6789/0 conn(0x7f4ab00ccbb0 legacy=0x7f4ab015f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:36.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.841+0000 7f4aae7fc640 1 -- 172.21.15.43:0/4257332867 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab00b2d40 con 0x7f4ab00cd010 2024-04-25T09:01:36.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.841+0000 7f4aae7fc640 1 -- 172.21.15.43:0/4257332867 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f4aa80019f0 con 0x7f4ab00cd010 2024-04-25T09:01:36.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.842+0000 7f4aae7fc640 1 -- 172.21.15.43:0/4257332867 >> v1:172.21.15.43:6789/0 conn(0x7f4ab00cd010 legacy=0x7f4ab015fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:36.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.842+0000 7f4aae7fc640 1 --2- 172.21.15.43:0/4257332867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a980039e0 0x7f4a98003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:36.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.842+0000 7f4aae7fc640 1 --2- 172.21.15.43:0/4257332867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a98004800 0x7f4a98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:36.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.842+0000 7f4aae7fc640 1 --2- 172.21.15.43:0/4257332867 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4ab00cbfc0 0x7f4a980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:36.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.842+0000 7f4aae7fc640 1 -- 172.21.15.43:0/4257332867 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab00b2a60 con 0x7f4a980039e0 2024-04-25T09:01:36.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.842+0000 7f4aae7fc640 1 -- 172.21.15.43:0/4257332867 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9800ba70 con 0x7f4a98004800 2024-04-25T09:01:36.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.842+0000 7f4aae7fc640 1 -- 172.21.15.43:0/4257332867 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9800bc70 con 0x7f4ab00cbfc0 2024-04-25T09:01:36.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.842+0000 7f4aaf7fe640 1 --2- 172.21.15.43:0/4257332867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a980039e0 0x7f4a98003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:36.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.842+0000 7f4aaffff640 1 --2- 172.21.15.43:0/4257332867 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4ab00cbfc0 0x7f4a980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:36.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.842+0000 7f4aaeffd640 1 --2- 172.21.15.43:0/4257332867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a98004800 0x7f4a98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:36.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.842+0000 7f4aaf7fe640 1 -- 172.21.15.43:0/4257332867 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4ab00cbfc0 msgr2=0x7f4a980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:36.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.842+0000 7f4aaf7fe640 1 --2- 172.21.15.43:0/4257332867 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4ab00cbfc0 0x7f4a980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:36.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.842+0000 7f4aaf7fe640 1 -- 172.21.15.43:0/4257332867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a98004800 msgr2=0x7f4a98004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:36.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.842+0000 7f4aaf7fe640 1 --2- 172.21.15.43:0/4257332867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a98004800 0x7f4a98004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:36.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.842+0000 7f4aaf7fe640 1 -- 172.21.15.43:0/4257332867 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a98003520 con 0x7f4a980039e0 2024-04-25T09:01:36.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.842+0000 7f4aaeffd640 1 --2- 172.21.15.43:0/4257332867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a98004800 0x7f4a98004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:01:36.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.843+0000 7f4aaf7fe640 1 --2- 172.21.15.43:0/4257332867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a980039e0 0x7f4a98003e00 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0009a90 tx=0x7f4a98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f242c150303b9bc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:36.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.843+0000 7f4aae7fc640 1 -- 172.21.15.43:0/4257332867 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0036070 con 0x7f4a980039e0 2024-04-25T09:01:36.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.843+0000 7f4aae7fc640 1 -- 172.21.15.43:0/4257332867 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa0031400 con 0x7f4a980039e0 2024-04-25T09:01:36.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.843+0000 7f4aae7fc640 1 -- 172.21.15.43:0/4257332867 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0009690 con 0x7f4a980039e0 2024-04-25T09:01:36.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.843+0000 7f4ab61d5640 1 -- 172.21.15.43:0/4257332867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a980039e0 msgr2=0x7f4a98003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:36.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.843+0000 7f4ab61d5640 1 --2- 172.21.15.43:0/4257332867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a980039e0 0x7f4a98003e00 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0009a90 tx=0x7f4a98005830 comp rx=0 tx=0).stop 2024-04-25T09:01:36.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.843+0000 7f4aaffff640 1 -- 172.21.15.43:0/4257332867 reap_dead start 2024-04-25T09:01:36.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.843+0000 7f4ab61d5640 1 -- 172.21.15.43:0/4257332867 shutdown_connections 2024-04-25T09:01:36.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.843+0000 7f4ab61d5640 1 -- 172.21.15.43:0/4257332867 >> 172.21.15.43:0/4257332867 conn(0x7f4ab00b1480 msgr2=0x7f4ab00b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:36.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.844+0000 7f4ab61d5640 1 -- 172.21.15.43:0/4257332867 shutdown_connections 2024-04-25T09:01:36.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.844+0000 7f4ab61d5640 1 -- 172.21.15.43:0/4257332867 wait complete. 2024-04-25T09:01:36.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.845+0000 7f4ab61d5640 1 Processor -- start 2024-04-25T09:01:36.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.845+0000 7f4ab61d5640 1 -- start start 2024-04-25T09:01:36.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.846+0000 7f4ab61d5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a980039e0 0x7f4ab015adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:36.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.846+0000 7f4ab61d5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ab01f7a20 0x7f4ab0142590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:36.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.846+0000 7f4ab61d5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4ab0142ad0 0x7f4ab01f37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:36.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.846+0000 7f4ab61d5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab00b35e0 con 0x7f4ab01f7a20 2024-04-25T09:01:36.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.846+0000 7f4ab61d5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab00b3300 con 0x7f4a980039e0 2024-04-25T09:01:36.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.846+0000 7f4ab61d5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ab00cd4a0 con 0x7f4ab0142ad0 2024-04-25T09:01:36.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.846+0000 7f4aaeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ab01f7a20 0x7f4ab0142590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:36.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.846+0000 7f4aaeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ab01f7a20 0x7f4ab0142590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49038/0 (socket says 172.21.15.43:49038) 2024-04-25T09:01:36.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.846+0000 7f4aaeffd640 1 -- 172.21.15.43:0/2219386022 learned_addr learned my addr 172.21.15.43:0/2219386022 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:36.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.846+0000 7f4aaf7fe640 1 --2- 172.21.15.43:0/2219386022 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a980039e0 0x7f4ab015adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:36.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.846+0000 7f4aaffff640 1 --2- 172.21.15.43:0/2219386022 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4ab0142ad0 0x7f4ab01f37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:36.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.846+0000 7f4aaeffd640 1 -- 172.21.15.43:0/2219386022 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4ab0142ad0 msgr2=0x7f4ab01f37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:36.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.846+0000 7f4aaeffd640 1 --2- 172.21.15.43:0/2219386022 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4ab0142ad0 0x7f4ab01f37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:36.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.846+0000 7f4aaeffd640 1 -- 172.21.15.43:0/2219386022 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a980039e0 msgr2=0x7f4ab015adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:36.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.846+0000 7f4aaeffd640 1 --2- 172.21.15.43:0/2219386022 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a980039e0 0x7f4ab015adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:36.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.846+0000 7f4aaeffd640 1 -- 172.21.15.43:0/2219386022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab01f4770 con 0x7f4ab01f7a20 2024-04-25T09:01:36.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.846+0000 7f4aaeffd640 1 --2- 172.21.15.43:0/2219386022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ab01f7a20 0x7f4ab0142590 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f4aa400e4d0 tx=0x7f4aa400e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:36.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.846+0000 7f4a97fff640 1 -- 172.21.15.43:0/2219386022 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa40052e0 con 0x7f4ab01f7a20 2024-04-25T09:01:36.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.846+0000 7f4a97fff640 1 -- 172.21.15.43:0/2219386022 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa4018070 con 0x7f4ab01f7a20 2024-04-25T09:01:36.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.846+0000 7f4ab61d5640 1 -- 172.21.15.43:0/2219386022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ab015b2f0 con 0x7f4ab01f7a20 2024-04-25T09:01:36.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.846+0000 7f4a97fff640 1 -- 172.21.15.43:0/2219386022 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4003810 con 0x7f4ab01f7a20 2024-04-25T09:01:36.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.846+0000 7f4ab61d5640 1 -- 172.21.15.43:0/2219386022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ab01f6f40 con 0x7f4ab01f7a20 2024-04-25T09:01:36.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.848+0000 7f4ab61d5640 1 -- 172.21.15.43:0/2219386022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ab0074ef0 con 0x7f4ab01f7a20 2024-04-25T09:01:36.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.848+0000 7f4a97fff640 1 -- 172.21.15.43:0/2219386022 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4aa40039b0 con 0x7f4ab01f7a20 2024-04-25T09:01:36.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.848+0000 7f4a97fff640 1 --2- 172.21.15.43:0/2219386022 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4a9c06b500 0x7f4a9c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:36.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.849+0000 7f4a97fff640 1 -- 172.21.15.43:0/2219386022 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4aa4089cb0 con 0x7f4ab01f7a20 2024-04-25T09:01:36.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.849+0000 7f4aaf7fe640 1 --2- 172.21.15.43:0/2219386022 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4a9c06b500 0x7f4a9c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:36.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.849+0000 7f4aaf7fe640 1 --2- 172.21.15.43:0/2219386022 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4a9c06b500 0x7f4a9c06d9c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0033000 tx=0x7f4aa0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:36.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:36.853+0000 7f4a97fff640 1 -- 172.21.15.43:0/2219386022 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4ab01f4770 con 0x7f4ab01f7a20 2024-04-25T09:01:37.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.034+0000 7f4ab61d5640 1 -- 172.21.15.43:0/2219386022 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4ab0142eb0 con 0x7f4ab01f7a20 2024-04-25T09:01:37.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.035+0000 7f4a97fff640 1 -- 172.21.15.43:0/2219386022 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+16583 (secure 0 0 0) 0x7f4aa4058570 con 0x7f4ab01f7a20 2024-04-25T09:01:37.027 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:01:37.027 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":102,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:01:35.356828+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:01:31.997813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T09:01:32.695222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:01:37.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.037+0000 7f4ab61d5640 1 -- 172.21.15.43:0/2219386022 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4a9c06b500 msgr2=0x7f4a9c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:37.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.037+0000 7f4ab61d5640 1 --2- 172.21.15.43:0/2219386022 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4a9c06b500 0x7f4a9c06d9c0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f4aa0033000 tx=0x7f4aa0008030 comp rx=0 tx=0).stop 2024-04-25T09:01:37.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.037+0000 7f4ab61d5640 1 -- 172.21.15.43:0/2219386022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ab01f7a20 msgr2=0x7f4ab0142590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:37.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.037+0000 7f4ab61d5640 1 --2- 172.21.15.43:0/2219386022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ab01f7a20 0x7f4ab0142590 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f4aa400e4d0 tx=0x7f4aa400e990 comp rx=0 tx=0).stop 2024-04-25T09:01:37.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.037+0000 7f4ab61d5640 1 -- 172.21.15.43:0/2219386022 shutdown_connections 2024-04-25T09:01:37.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.037+0000 7f4ab61d5640 1 --2- 172.21.15.43:0/2219386022 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4ab0142ad0 0x7f4ab01f37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:37.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.037+0000 7f4ab61d5640 1 --2- 172.21.15.43:0/2219386022 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4ab01f7a20 0x7f4ab0142590 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:37.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.037+0000 7f4ab61d5640 1 --2- 172.21.15.43:0/2219386022 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4a9c06b500 0x7f4a9c06d9c0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:37.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.037+0000 7f4ab61d5640 1 --2- 172.21.15.43:0/2219386022 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a980039e0 0x7f4ab015adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:37.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.037+0000 7f4ab61d5640 1 -- 172.21.15.43:0/2219386022 >> 172.21.15.43:0/2219386022 conn(0x7f4ab00b1480 msgr2=0x7f4ab00b0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:37.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.038+0000 7f4ab61d5640 1 -- 172.21.15.43:0/2219386022 shutdown_connections 2024-04-25T09:01:37.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.038+0000 7f4ab61d5640 1 -- 172.21.15.43:0/2219386022 wait complete. 2024-04-25T09:01:37.038 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:01:37.039 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:01:37.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.152+0000 7fb58d54f640 1 Processor -- start 2024-04-25T09:01:37.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.152+0000 7fb58d54f640 1 -- start start 2024-04-25T09:01:37.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.153+0000 7fb58d54f640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb58815ef50 0x7fb5880d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:37.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.153+0000 7fb58d54f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5880d0920 0x7fb588160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:37.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.153+0000 7fb58d54f640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5880cee80 con 0x7fb58815dce0 2024-04-25T09:01:37.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.153+0000 7fb58d54f640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5880cec60 con 0x7fb58815ef50 2024-04-25T09:01:37.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.153+0000 7fb58d54f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5880d0d70 con 0x7fb5880d0920 2024-04-25T09:01:37.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.153+0000 7fb5877fe640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5880d0920 0x7fb588160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:37.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.153+0000 7fb5867fc640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb58815ef50 0x7fb5880d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:37.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.153+0000 7fb586ffd640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7fb58815dce0 0x7fb5880ce550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:55244/0 (socket says 172.21.15.43:55244) 2024-04-25T09:01:37.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.153+0000 7fb586ffd640 1 -- 172.21.15.43:0/1967665177 learned_addr learned my addr 172.21.15.43:0/1967665177 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:37.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.154+0000 7fb5877fe640 1 -- 172.21.15.43:0/1967665177 >> v2:172.21.15.193:3300/0 conn(0x7fb58815ef50 msgr2=0x7fb5880d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:37.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.154+0000 7fb5877fe640 1 --2- 172.21.15.43:0/1967665177 >> v2:172.21.15.193:3300/0 conn(0x7fb58815ef50 0x7fb5880d03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:37.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.154+0000 7fb5877fe640 1 -- 172.21.15.43:0/1967665177 >> v1:172.21.15.193:6789/0 conn(0x7fb58815dce0 legacy=0x7fb5880ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:37.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.154+0000 7fb5877fe640 1 -- 172.21.15.43:0/1967665177 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5880d2920 con 0x7fb5880d0920 2024-04-25T09:01:37.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.155+0000 7fb5877fe640 1 --2- 172.21.15.43:0/1967665177 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5880d0920 0x7fb588160980 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb5780097a0 tx=0x7fb57802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c031af06faf9be0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:37.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.155+0000 7fb585ffb640 1 -- 172.21.15.43:0/1967665177 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb578037070 con 0x7fb5880d0920 2024-04-25T09:01:37.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.155+0000 7fb585ffb640 1 -- 172.21.15.43:0/1967665177 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb57802ae50 con 0x7fb5880d0920 2024-04-25T09:01:37.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.155+0000 7fb585ffb640 1 -- 172.21.15.43:0/1967665177 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb578032660 con 0x7fb5880d0920 2024-04-25T09:01:37.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.155+0000 7fb58d54f640 1 -- 172.21.15.43:0/1967665177 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5880d0920 msgr2=0x7fb588160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:37.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.155+0000 7fb58d54f640 1 --2- 172.21.15.43:0/1967665177 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5880d0920 0x7fb588160980 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb5780097a0 tx=0x7fb57802a100 comp rx=0 tx=0).stop 2024-04-25T09:01:37.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.155+0000 7fb58d54f640 1 -- 172.21.15.43:0/1967665177 shutdown_connections 2024-04-25T09:01:37.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.155+0000 7fb58d54f640 1 --2- 172.21.15.43:0/1967665177 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5880d0920 0x7fb588160980 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:37.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.155+0000 7fb58d54f640 1 --2- 172.21.15.43:0/1967665177 >> v2:172.21.15.193:3300/0 conn(0x7fb58815ef50 0x7fb5880d03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:37.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.155+0000 7fb58d54f640 1 -- 172.21.15.43:0/1967665177 >> 172.21.15.43:0/1967665177 conn(0x7fb58809cd00 msgr2=0x7fb5880d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:37.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.155+0000 7fb58d54f640 1 -- 172.21.15.43:0/1967665177 shutdown_connections 2024-04-25T09:01:37.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.155+0000 7fb58d54f640 1 -- 172.21.15.43:0/1967665177 wait complete. 2024-04-25T09:01:37.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.157+0000 7fb58d54f640 1 Processor -- start 2024-04-25T09:01:37.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.157+0000 7fb58d54f640 1 -- start start 2024-04-25T09:01:37.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.158+0000 7fb58d54f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5880d0920 0x7fb5880c9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:37.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.158+0000 7fb58d54f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb58815dce0 0x7fb5880b4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:37.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.158+0000 7fb58d54f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb58815ef50 0x7fb5880b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:37.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.158+0000 7fb58d54f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5880cf720 con 0x7fb58815dce0 2024-04-25T09:01:37.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.158+0000 7fb58d54f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5880cf440 con 0x7fb5880d0920 2024-04-25T09:01:37.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.158+0000 7fb58d54f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb58815f6b0 con 0x7fb58815ef50 2024-04-25T09:01:37.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.158+0000 7fb5867fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb58815dce0 0x7fb5880b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:37.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.158+0000 7fb5867fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb58815dce0 0x7fb5880b4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49050/0 (socket says 172.21.15.43:49050) 2024-04-25T09:01:37.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.158+0000 7fb5867fc640 1 -- 172.21.15.43:0/3526119544 learned_addr learned my addr 172.21.15.43:0/3526119544 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:37.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.158+0000 7fb586ffd640 1 --2- 172.21.15.43:0/3526119544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5880d0920 0x7fb5880c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:37.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.158+0000 7fb5877fe640 1 --2- 172.21.15.43:0/3526119544 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb58815ef50 0x7fb5880b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:37.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.158+0000 7fb5867fc640 1 -- 172.21.15.43:0/3526119544 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb58815ef50 msgr2=0x7fb5880b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:37.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.158+0000 7fb5867fc640 1 --2- 172.21.15.43:0/3526119544 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb58815ef50 0x7fb5880b48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:37.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.158+0000 7fb5867fc640 1 -- 172.21.15.43:0/3526119544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5880d0920 msgr2=0x7fb5880c9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:37.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.158+0000 7fb5867fc640 1 --2- 172.21.15.43:0/3526119544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5880d0920 0x7fb5880c9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:37.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.158+0000 7fb5867fc640 1 -- 172.21.15.43:0/3526119544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb588155340 con 0x7fb58815dce0 2024-04-25T09:01:37.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.159+0000 7fb5867fc640 1 --2- 172.21.15.43:0/3526119544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb58815dce0 0x7fb5880b4370 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fb57000e9b0 tx=0x7fb57000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:37.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.159+0000 7fb57f7fe640 1 -- 172.21.15.43:0/3526119544 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb570017070 con 0x7fb58815dce0 2024-04-25T09:01:37.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.159+0000 7fb57f7fe640 1 -- 172.21.15.43:0/3526119544 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb57000c9a0 con 0x7fb58815dce0 2024-04-25T09:01:37.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.159+0000 7fb57f7fe640 1 -- 172.21.15.43:0/3526119544 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57000cd90 con 0x7fb58815dce0 2024-04-25T09:01:37.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.159+0000 7fb58d54f640 1 -- 172.21.15.43:0/3526119544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb588155620 con 0x7fb58815dce0 2024-04-25T09:01:37.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.159+0000 7fb58d54f640 1 -- 172.21.15.43:0/3526119544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb588155be0 con 0x7fb58815dce0 2024-04-25T09:01:37.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.160+0000 7fb58d54f640 1 -- 172.21.15.43:0/3526119544 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5880680f0 con 0x7fb58815dce0 2024-04-25T09:01:37.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.160+0000 7fb57f7fe640 1 -- 172.21.15.43:0/3526119544 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb57001a070 con 0x7fb58815dce0 2024-04-25T09:01:37.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.161+0000 7fb57f7fe640 1 --2- 172.21.15.43:0/3526119544 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb56806b4d0 0x7fb56806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:37.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.161+0000 7fb57f7fe640 1 -- 172.21.15.43:0/3526119544 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb57008a1e0 con 0x7fb58815dce0 2024-04-25T09:01:37.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.161+0000 7fb586ffd640 1 --2- 172.21.15.43:0/3526119544 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb56806b4d0 0x7fb56806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:37.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.162+0000 7fb586ffd640 1 --2- 172.21.15.43:0/3526119544 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb56806b4d0 0x7fb56806d990 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb5800012e0 tx=0x7fb580001a30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:37.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.165+0000 7fb57f7fe640 1 -- 172.21.15.43:0/3526119544 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb570058a10 con 0x7fb58815dce0 2024-04-25T09:01:37.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.344+0000 7fb58d54f640 1 -- 172.21.15.43:0/3526119544 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb588074ef0 con 0x7fb56806b4d0 2024-04-25T09:01:37.339 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.350+0000 7fb57f7fe640 1 -- 172.21.15.43:0/3526119544 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247013 (secure 0 0 0) 0x7fb588074ef0 con 0x7fb56806b4d0 2024-04-25T09:01:37.342 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:01:37.343 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-25T09:01:37.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.353+0000 7fb58d54f640 1 -- 172.21.15.43:0/3526119544 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb56806b4d0 msgr2=0x7fb56806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:37.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.353+0000 7fb58d54f640 1 --2- 172.21.15.43:0/3526119544 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb56806b4d0 0x7fb56806d990 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb5800012e0 tx=0x7fb580001a30 comp rx=0 tx=0).stop 2024-04-25T09:01:37.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.353+0000 7fb58d54f640 1 -- 172.21.15.43:0/3526119544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb58815dce0 msgr2=0x7fb5880b4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:37.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.353+0000 7fb58d54f640 1 --2- 172.21.15.43:0/3526119544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb58815dce0 0x7fb5880b4370 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fb57000e9b0 tx=0x7fb57000ee70 comp rx=0 tx=0).stop 2024-04-25T09:01:37.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.353+0000 7fb58d54f640 1 -- 172.21.15.43:0/3526119544 shutdown_connections 2024-04-25T09:01:37.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.353+0000 7fb58d54f640 1 --2- 172.21.15.43:0/3526119544 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb58815ef50 0x7fb5880b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:37.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.353+0000 7fb58d54f640 1 --2- 172.21.15.43:0/3526119544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb58815dce0 0x7fb5880b4370 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:37.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.353+0000 7fb58d54f640 1 --2- 172.21.15.43:0/3526119544 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb56806b4d0 0x7fb56806d990 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:37.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.353+0000 7fb58d54f640 1 --2- 172.21.15.43:0/3526119544 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5880d0920 0x7fb5880c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:37.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.353+0000 7fb58d54f640 1 -- 172.21.15.43:0/3526119544 >> 172.21.15.43:0/3526119544 conn(0x7fb58809cd00 msgr2=0x7fb5880c74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:37.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.353+0000 7fb58d54f640 1 -- 172.21.15.43:0/3526119544 shutdown_connections 2024-04-25T09:01:37.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.353+0000 7fb58d54f640 1 -- 172.21.15.43:0/3526119544 wait complete. 2024-04-25T09:01:37.354 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2024-04-25T09:01:35.796679+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":24,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":514276,"kb_used_data":10800,"kb_used_omap":12,"kb_used_meta":503347,"kb_avail":754460444,"statfs":{"total":773094113280,"available":772567494656,"internally_reserved":0,"allocated":11059200,"data_stored":5442242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":515427889},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001055"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.355534+0000","last_change":"2024-04-25T09:01:33.355534+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.355534+0000","last_undegraded":"2024-04-25T09:01:33.355534+0000","last_fullsized":"2024-04-25T09:01:33.355534+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.359660+0000","last_change":"2024-04-25T09:01:33.359660+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.359660+0000","last_undegraded":"2024-04-25T09:01:33.359660+0000","last_fullsized":"2024-04-25T09:01:33.359660+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.355300+0000","last_change":"2024-04-25T09:01:33.355300+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.355300+0000","last_undegraded":"2024-04-25T09:01:33.355300+0000","last_fullsized":"2024-04-25T09:01:33.355300+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356488+0000","last_change":"2024-04-25T09:01:33.356488+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356488+0000","last_undegraded":"2024-04-25T09:01:33.356488+0000","last_fullsized":"2024-04-25T09:01:33.356488+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.358970+0000","last_change":"2024-04-25T09:01:33.358970+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.358970+0000","last_undegraded":"2024-04-25T09:01:33.358970+0000","last_fullsized":"2024-04-25T09:01:33.358970+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.358980+0000","last_change":"2024-04-25T09:01:33.358980+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.358980+0000","last_undegraded":"2024-04-25T09:01:33.358980+0000","last_fullsized":"2024-04-25T09:01:33.358980+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356706+0000","last_change":"2024-04-25T09:01:33.356706+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356706+0000","last_undegraded":"2024-04-25T09:01:33.356706+0000","last_fullsized":"2024-04-25T09:01:33.356706+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.357495+0000","last_change":"2024-04-25T09:01:33.357495+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.357495+0000","last_undegraded":"2024-04-25T09:01:33.357495+0000","last_fullsized":"2024-04-25T09:01:33.357495+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.359846+0000","last_change":"2024-04-25T09:01:33.359846+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.359846+0000","last_undegraded":"2024-04-25T09:01:33.359846+0000","last_fullsized":"2024-04-25T09:01:33.359846+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.358143+0000","last_change":"2024-04-25T09:01:33.358143+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.358143+0000","last_undegraded":"2024-04-25T09:01:33.358143+0000","last_fullsized":"2024-04-25T09:01:33.358143+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.358698+0000","last_change":"2024-04-25T09:01:33.358698+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.358698+0000","last_undegraded":"2024-04-25T09:01:33.358698+0000","last_fullsized":"2024-04-25T09:01:33.358698+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356492+0000","last_change":"2024-04-25T09:01:33.356492+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356492+0000","last_undegraded":"2024-04-25T09:01:33.356492+0000","last_fullsized":"2024-04-25T09:01:33.356492+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.355925+0000","last_change":"2024-04-25T09:01:33.355925+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.355925+0000","last_undegraded":"2024-04-25T09:01:33.355925+0000","last_fullsized":"2024-04-25T09:01:33.355925+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.362266+0000","last_change":"2024-04-25T09:01:33.362266+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.362266+0000","last_undegraded":"2024-04-25T09:01:33.362266+0000","last_fullsized":"2024-04-25T09:01:33.362266+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.358146+0000","last_change":"2024-04-25T09:01:33.358146+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.358146+0000","last_undegraded":"2024-04-25T09:01:33.358146+0000","last_fullsized":"2024-04-25T09:01:33.358146+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356808+0000","last_change":"2024-04-25T09:01:33.356808+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356808+0000","last_undegraded":"2024-04-25T09:01:33.356808+0000","last_fullsized":"2024-04-25T09:01:33.356808+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.358063+0000","last_change":"2024-04-25T09:01:33.358063+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.358063+0000","last_undegraded":"2024-04-25T09:01:33.358063+0000","last_fullsized":"2024-04-25T09:01:33.358063+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.358540+0000","last_change":"2024-04-25T09:01:33.358540+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.358540+0000","last_undegraded":"2024-04-25T09:01:33.358540+0000","last_fullsized":"2024-04-25T09:01:33.358540+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356094+0000","last_change":"2024-04-25T09:01:33.356094+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356094+0000","last_undegraded":"2024-04-25T09:01:33.356094+0000","last_fullsized":"2024-04-25T09:01:33.356094+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.355041+0000","last_change":"2024-04-25T09:01:33.355041+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.355041+0000","last_undegraded":"2024-04-25T09:01:33.355041+0000","last_fullsized":"2024-04-25T09:01:33.355041+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.354826+0000","last_change":"2024-04-25T09:01:33.354826+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.354826+0000","last_undegraded":"2024-04-25T09:01:33.354826+0000","last_fullsized":"2024-04-25T09:01:33.354826+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.358378+0000","last_change":"2024-04-25T09:01:33.358378+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.358378+0000","last_undegraded":"2024-04-25T09:01:33.358378+0000","last_fullsized":"2024-04-25T09:01:33.358378+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356711+0000","last_change":"2024-04-25T09:01:33.356711+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356711+0000","last_undegraded":"2024-04-25T09:01:33.356711+0000","last_fullsized":"2024-04-25T09:01:33.356711+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356347+0000","last_change":"2024-04-25T09:01:33.356347+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356347+0000","last_undegraded":"2024-04-25T09:01:33.356347+0000","last_fullsized":"2024-04-25T09:01:33.356347+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.362666+0000","last_change":"2024-04-25T09:01:33.362666+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.362666+0000","last_undegraded":"2024-04-25T09:01:33.362666+0000","last_fullsized":"2024-04-25T09:01:33.362666+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.359240+0000","last_change":"2024-04-25T09:01:33.359240+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.359240+0000","last_undegraded":"2024-04-25T09:01:33.359240+0000","last_fullsized":"2024-04-25T09:01:33.359240+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.358208+0000","last_change":"2024-04-25T09:01:33.358208+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.358208+0000","last_undegraded":"2024-04-25T09:01:33.358208+0000","last_fullsized":"2024-04-25T09:01:33.358208+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.363030+0000","last_change":"2024-04-25T09:01:33.363030+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.363030+0000","last_undegraded":"2024-04-25T09:01:33.363030+0000","last_fullsized":"2024-04-25T09:01:33.363030+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.357869+0000","last_change":"2024-04-25T09:01:33.357869+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.357869+0000","last_undegraded":"2024-04-25T09:01:33.357869+0000","last_fullsized":"2024-04-25T09:01:33.357869+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.355005+0000","last_change":"2024-04-25T09:01:33.355005+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.355005+0000","last_undegraded":"2024-04-25T09:01:33.355005+0000","last_fullsized":"2024-04-25T09:01:33.355005+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.357607+0000","last_change":"2024-04-25T09:01:33.357607+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.357607+0000","last_undegraded":"2024-04-25T09:01:33.357607+0000","last_fullsized":"2024-04-25T09:01:33.357607+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.354229+0000","last_change":"2024-04-25T09:01:33.354229+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.354229+0000","last_undegraded":"2024-04-25T09:01:33.354229+0000","last_fullsized":"2024-04-25T09:01:33.354229+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.353890+0000","last_change":"2024-04-25T09:01:33.353890+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.353890+0000","last_undegraded":"2024-04-25T09:01:33.353890+0000","last_fullsized":"2024-04-25T09:01:33.353890+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.367304+0000","last_change":"2024-04-25T09:01:33.365666+0000","last_active":"2024-04-25T09:01:33.367304+0000","last_peered":"2024-04-25T09:01:33.367304+0000","last_clean":"2024-04-25T09:01:33.367304+0000","last_became_active":"2024-04-25T09:01:33.365341+0000","last_became_peered":"2024-04-25T09:01:33.365341+0000","last_unstale":"2024-04-25T09:01:33.367304+0000","last_undegraded":"2024-04-25T09:01:33.367304+0000","last_fullsized":"2024-04-25T09:01:33.367304+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:41.915410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356285+0000","last_change":"2024-04-25T09:01:33.356285+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356285+0000","last_undegraded":"2024-04-25T09:01:33.356285+0000","last_fullsized":"2024-04-25T09:01:33.356285+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.368360+0000","last_change":"2024-04-25T09:01:33.366268+0000","last_active":"2024-04-25T09:01:33.368360+0000","last_peered":"2024-04-25T09:01:33.368360+0000","last_clean":"2024-04-25T09:01:33.368360+0000","last_became_active":"2024-04-25T09:01:33.365952+0000","last_became_peered":"2024-04-25T09:01:33.365952+0000","last_unstale":"2024-04-25T09:01:33.368360+0000","last_undegraded":"2024-04-25T09:01:33.368360+0000","last_fullsized":"2024-04-25T09:01:33.368360+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:29.149495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.354216+0000","last_change":"2024-04-25T09:01:33.354216+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.354216+0000","last_undegraded":"2024-04-25T09:01:33.354216+0000","last_fullsized":"2024-04-25T09:01:33.354216+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.371487+0000","last_change":"2024-04-25T09:01:33.370140+0000","last_active":"2024-04-25T09:01:33.371487+0000","last_peered":"2024-04-25T09:01:33.371487+0000","last_clean":"2024-04-25T09:01:33.371487+0000","last_became_active":"2024-04-25T09:01:33.369677+0000","last_became_peered":"2024-04-25T09:01:33.369677+0000","last_unstale":"2024-04-25T09:01:33.371487+0000","last_undegraded":"2024-04-25T09:01:33.371487+0000","last_fullsized":"2024-04-25T09:01:33.371487+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:15.517604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.357436+0000","last_change":"2024-04-25T09:01:33.357436+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.357436+0000","last_undegraded":"2024-04-25T09:01:33.357436+0000","last_fullsized":"2024-04-25T09:01:33.357436+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.369409+0000","last_change":"2024-04-25T09:01:33.367516+0000","last_active":"2024-04-25T09:01:33.369409+0000","last_peered":"2024-04-25T09:01:33.369409+0000","last_clean":"2024-04-25T09:01:33.369409+0000","last_became_active":"2024-04-25T09:01:33.367171+0000","last_became_peered":"2024-04-25T09:01:33.367171+0000","last_unstale":"2024-04-25T09:01:33.369409+0000","last_undegraded":"2024-04-25T09:01:33.369409+0000","last_fullsized":"2024-04-25T09:01:33.369409+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:43.586923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356196+0000","last_change":"2024-04-25T09:01:33.356196+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356196+0000","last_undegraded":"2024-04-25T09:01:33.356196+0000","last_fullsized":"2024-04-25T09:01:33.356196+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.363802+0000","last_change":"2024-04-25T09:01:33.360517+0000","last_active":"2024-04-25T09:01:33.363802+0000","last_peered":"2024-04-25T09:01:33.363802+0000","last_clean":"2024-04-25T09:01:33.363802+0000","last_became_active":"2024-04-25T09:01:33.360139+0000","last_became_peered":"2024-04-25T09:01:33.360139+0000","last_unstale":"2024-04-25T09:01:33.363802+0000","last_undegraded":"2024-04-25T09:01:33.363802+0000","last_fullsized":"2024-04-25T09:01:33.363802+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:50.515806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356694+0000","last_change":"2024-04-25T09:01:33.356694+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356694+0000","last_undegraded":"2024-04-25T09:01:33.356694+0000","last_fullsized":"2024-04-25T09:01:33.356694+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.372295+0000","last_change":"2024-04-25T09:01:33.371930+0000","last_active":"2024-04-25T09:01:33.372295+0000","last_peered":"2024-04-25T09:01:33.372295+0000","last_clean":"2024-04-25T09:01:33.372295+0000","last_became_active":"2024-04-25T09:01:33.371395+0000","last_became_peered":"2024-04-25T09:01:33.371395+0000","last_unstale":"2024-04-25T09:01:33.372295+0000","last_undegraded":"2024-04-25T09:01:33.372295+0000","last_fullsized":"2024-04-25T09:01:33.372295+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:44.824026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.357302+0000","last_change":"2024-04-25T09:01:33.357302+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.357302+0000","last_undegraded":"2024-04-25T09:01:33.357302+0000","last_fullsized":"2024-04-25T09:01:33.357302+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.367580+0000","last_change":"2024-04-25T09:01:33.366526+0000","last_active":"2024-04-25T09:01:33.367580+0000","last_peered":"2024-04-25T09:01:33.367580+0000","last_clean":"2024-04-25T09:01:33.367580+0000","last_became_active":"2024-04-25T09:01:33.366246+0000","last_became_peered":"2024-04-25T09:01:33.366246+0000","last_unstale":"2024-04-25T09:01:33.367580+0000","last_undegraded":"2024-04-25T09:01:33.367580+0000","last_fullsized":"2024-04-25T09:01:33.367580+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:26.679025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.355141+0000","last_change":"2024-04-25T09:01:33.355141+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.355141+0000","last_undegraded":"2024-04-25T09:01:33.355141+0000","last_fullsized":"2024-04-25T09:01:33.355141+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.369410+0000","last_change":"2024-04-25T09:01:33.367167+0000","last_active":"2024-04-25T09:01:33.369410+0000","last_peered":"2024-04-25T09:01:33.369410+0000","last_clean":"2024-04-25T09:01:33.369410+0000","last_became_active":"2024-04-25T09:01:33.365541+0000","last_became_peered":"2024-04-25T09:01:33.365541+0000","last_unstale":"2024-04-25T09:01:33.369410+0000","last_undegraded":"2024-04-25T09:01:33.369410+0000","last_fullsized":"2024-04-25T09:01:33.369410+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:22.335074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.357332+0000","last_change":"2024-04-25T09:01:33.357332+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.357332+0000","last_undegraded":"2024-04-25T09:01:33.357332+0000","last_fullsized":"2024-04-25T09:01:33.357332+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.363381+0000","last_change":"2024-04-25T09:01:33.361825+0000","last_active":"2024-04-25T09:01:33.363381+0000","last_peered":"2024-04-25T09:01:33.363381+0000","last_clean":"2024-04-25T09:01:33.363381+0000","last_became_active":"2024-04-25T09:01:33.361544+0000","last_became_peered":"2024-04-25T09:01:33.361544+0000","last_unstale":"2024-04-25T09:01:33.363381+0000","last_undegraded":"2024-04-25T09:01:33.363381+0000","last_fullsized":"2024-04-25T09:01:33.363381+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:19.818176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.354331+0000","last_change":"2024-04-25T09:01:33.354331+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.354331+0000","last_undegraded":"2024-04-25T09:01:33.354331+0000","last_fullsized":"2024-04-25T09:01:33.354331+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.369815+0000","last_change":"2024-04-25T09:01:33.365387+0000","last_active":"2024-04-25T09:01:33.369815+0000","last_peered":"2024-04-25T09:01:33.369815+0000","last_clean":"2024-04-25T09:01:33.369815+0000","last_became_active":"2024-04-25T09:01:33.365088+0000","last_became_peered":"2024-04-25T09:01:33.365088+0000","last_unstale":"2024-04-25T09:01:33.369815+0000","last_undegraded":"2024-04-25T09:01:33.369815+0000","last_fullsized":"2024-04-25T09:01:33.369815+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:34.749007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.362314+0000","last_change":"2024-04-25T09:01:33.362314+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.362314+0000","last_undegraded":"2024-04-25T09:01:33.362314+0000","last_fullsized":"2024-04-25T09:01:33.362314+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.368520+0000","last_change":"2024-04-25T09:01:33.367517+0000","last_active":"2024-04-25T09:01:33.368520+0000","last_peered":"2024-04-25T09:01:33.368520+0000","last_clean":"2024-04-25T09:01:33.368520+0000","last_became_active":"2024-04-25T09:01:33.367216+0000","last_became_peered":"2024-04-25T09:01:33.367216+0000","last_unstale":"2024-04-25T09:01:33.368520+0000","last_undegraded":"2024-04-25T09:01:33.368520+0000","last_fullsized":"2024-04-25T09:01:33.368520+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:42.045982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.357735+0000","last_change":"2024-04-25T09:01:33.357735+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.357735+0000","last_undegraded":"2024-04-25T09:01:33.357735+0000","last_fullsized":"2024-04-25T09:01:33.357735+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.367757+0000","last_change":"2024-04-25T09:01:33.366291+0000","last_active":"2024-04-25T09:01:33.367757+0000","last_peered":"2024-04-25T09:01:33.367757+0000","last_clean":"2024-04-25T09:01:33.367757+0000","last_became_active":"2024-04-25T09:01:33.365670+0000","last_became_peered":"2024-04-25T09:01:33.365670+0000","last_unstale":"2024-04-25T09:01:33.367757+0000","last_undegraded":"2024-04-25T09:01:33.367757+0000","last_fullsized":"2024-04-25T09:01:33.367757+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:35.172322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.355949+0000","last_change":"2024-04-25T09:01:33.355949+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.355949+0000","last_undegraded":"2024-04-25T09:01:33.355949+0000","last_fullsized":"2024-04-25T09:01:33.355949+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.368724+0000","last_change":"2024-04-25T09:01:33.363784+0000","last_active":"2024-04-25T09:01:33.368724+0000","last_peered":"2024-04-25T09:01:33.368724+0000","last_clean":"2024-04-25T09:01:33.368724+0000","last_became_active":"2024-04-25T09:01:33.363279+0000","last_became_peered":"2024-04-25T09:01:33.363279+0000","last_unstale":"2024-04-25T09:01:33.368724+0000","last_undegraded":"2024-04-25T09:01:33.368724+0000","last_fullsized":"2024-04-25T09:01:33.368724+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:15.504041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.354732+0000","last_change":"2024-04-25T09:01:33.354732+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.354732+0000","last_undegraded":"2024-04-25T09:01:33.354732+0000","last_fullsized":"2024-04-25T09:01:33.354732+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.368515+0000","last_change":"2024-04-25T09:01:33.365837+0000","last_active":"2024-04-25T09:01:33.368515+0000","last_peered":"2024-04-25T09:01:33.368515+0000","last_clean":"2024-04-25T09:01:33.368515+0000","last_became_active":"2024-04-25T09:01:33.362985+0000","last_became_peered":"2024-04-25T09:01:33.362985+0000","last_unstale":"2024-04-25T09:01:33.368515+0000","last_undegraded":"2024-04-25T09:01:33.368515+0000","last_fullsized":"2024-04-25T09:01:33.368515+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:02.701373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.359064+0000","last_change":"2024-04-25T09:01:33.359064+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.359064+0000","last_undegraded":"2024-04-25T09:01:33.359064+0000","last_fullsized":"2024-04-25T09:01:33.359064+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.367829+0000","last_change":"2024-04-25T09:01:33.367032+0000","last_active":"2024-04-25T09:01:33.367829+0000","last_peered":"2024-04-25T09:01:33.367829+0000","last_clean":"2024-04-25T09:01:33.367829+0000","last_became_active":"2024-04-25T09:01:33.366662+0000","last_became_peered":"2024-04-25T09:01:33.366662+0000","last_unstale":"2024-04-25T09:01:33.367829+0000","last_undegraded":"2024-04-25T09:01:33.367829+0000","last_fullsized":"2024-04-25T09:01:33.367829+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:13.219195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.352882+0000","last_change":"2024-04-25T09:01:33.352882+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.352882+0000","last_undegraded":"2024-04-25T09:01:33.352882+0000","last_fullsized":"2024-04-25T09:01:33.352882+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.369386+0000","last_change":"2024-04-25T09:01:33.366823+0000","last_active":"2024-04-25T09:01:33.369386+0000","last_peered":"2024-04-25T09:01:33.369386+0000","last_clean":"2024-04-25T09:01:33.369386+0000","last_became_active":"2024-04-25T09:01:33.366050+0000","last_became_peered":"2024-04-25T09:01:33.366050+0000","last_unstale":"2024-04-25T09:01:33.369386+0000","last_undegraded":"2024-04-25T09:01:33.369386+0000","last_fullsized":"2024-04-25T09:01:33.369386+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:49.016771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.362269+0000","last_change":"2024-04-25T09:01:33.362269+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.362269+0000","last_undegraded":"2024-04-25T09:01:33.362269+0000","last_fullsized":"2024-04-25T09:01:33.362269+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.369665+0000","last_change":"2024-04-25T09:01:33.369232+0000","last_active":"2024-04-25T09:01:33.369665+0000","last_peered":"2024-04-25T09:01:33.369665+0000","last_clean":"2024-04-25T09:01:33.369665+0000","last_became_active":"2024-04-25T09:01:33.368905+0000","last_became_peered":"2024-04-25T09:01:33.368905+0000","last_unstale":"2024-04-25T09:01:33.369665+0000","last_undegraded":"2024-04-25T09:01:33.369665+0000","last_fullsized":"2024-04-25T09:01:33.369665+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:41.915410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.355227+0000","last_change":"2024-04-25T09:01:33.355227+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.355227+0000","last_undegraded":"2024-04-25T09:01:33.355227+0000","last_fullsized":"2024-04-25T09:01:33.355227+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.364958+0000","last_change":"2024-04-25T09:01:33.363919+0000","last_active":"2024-04-25T09:01:33.364958+0000","last_peered":"2024-04-25T09:01:33.364958+0000","last_clean":"2024-04-25T09:01:33.364958+0000","last_became_active":"2024-04-25T09:01:33.363587+0000","last_became_peered":"2024-04-25T09:01:33.363587+0000","last_unstale":"2024-04-25T09:01:33.364958+0000","last_undegraded":"2024-04-25T09:01:33.364958+0000","last_fullsized":"2024-04-25T09:01:33.364958+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:39.790922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.355533+0000","last_change":"2024-04-25T09:01:33.355533+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.355533+0000","last_undegraded":"2024-04-25T09:01:33.355533+0000","last_fullsized":"2024-04-25T09:01:33.355533+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.367244+0000","last_change":"2024-04-25T09:01:33.364976+0000","last_active":"2024-04-25T09:01:33.367244+0000","last_peered":"2024-04-25T09:01:33.367244+0000","last_clean":"2024-04-25T09:01:33.367244+0000","last_became_active":"2024-04-25T09:01:33.363617+0000","last_became_peered":"2024-04-25T09:01:33.363617+0000","last_unstale":"2024-04-25T09:01:33.367244+0000","last_undegraded":"2024-04-25T09:01:33.367244+0000","last_fullsized":"2024-04-25T09:01:33.367244+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:13.219195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.353243+0000","last_change":"2024-04-25T09:01:33.353243+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.353243+0000","last_undegraded":"2024-04-25T09:01:33.353243+0000","last_fullsized":"2024-04-25T09:01:33.353243+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.371512+0000","last_change":"2024-04-25T09:01:33.369475+0000","last_active":"2024-04-25T09:01:33.371512+0000","last_peered":"2024-04-25T09:01:33.371512+0000","last_clean":"2024-04-25T09:01:33.371512+0000","last_became_active":"2024-04-25T09:01:33.368210+0000","last_became_peered":"2024-04-25T09:01:33.368210+0000","last_unstale":"2024-04-25T09:01:33.371512+0000","last_undegraded":"2024-04-25T09:01:33.371512+0000","last_fullsized":"2024-04-25T09:01:33.371512+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:26.679025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.354197+0000","last_change":"2024-04-25T09:01:33.354197+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.354197+0000","last_undegraded":"2024-04-25T09:01:33.354197+0000","last_fullsized":"2024-04-25T09:01:33.354197+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.364947+0000","last_change":"2024-04-25T09:01:33.363025+0000","last_active":"2024-04-25T09:01:33.364947+0000","last_peered":"2024-04-25T09:01:33.364947+0000","last_clean":"2024-04-25T09:01:33.364947+0000","last_became_active":"2024-04-25T09:01:33.362724+0000","last_became_peered":"2024-04-25T09:01:33.362724+0000","last_unstale":"2024-04-25T09:01:33.364947+0000","last_undegraded":"2024-04-25T09:01:33.364947+0000","last_fullsized":"2024-04-25T09:01:33.364947+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:41.464204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.353721+0000","last_change":"2024-04-25T09:01:33.353721+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.353721+0000","last_undegraded":"2024-04-25T09:01:33.353721+0000","last_fullsized":"2024-04-25T09:01:33.353721+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.366031+0000","last_change":"2024-04-25T09:01:33.365366+0000","last_active":"2024-04-25T09:01:33.366031+0000","last_peered":"2024-04-25T09:01:33.366031+0000","last_clean":"2024-04-25T09:01:33.366031+0000","last_became_active":"2024-04-25T09:01:33.365090+0000","last_became_peered":"2024-04-25T09:01:33.365090+0000","last_unstale":"2024-04-25T09:01:33.366031+0000","last_undegraded":"2024-04-25T09:01:33.366031+0000","last_fullsized":"2024-04-25T09:01:33.366031+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:35.172322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.361446+0000","last_change":"2024-04-25T09:01:33.361446+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.361446+0000","last_undegraded":"2024-04-25T09:01:33.361446+0000","last_fullsized":"2024-04-25T09:01:33.361446+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.371551+0000","last_change":"2024-04-25T09:01:33.370545+0000","last_active":"2024-04-25T09:01:33.371551+0000","last_peered":"2024-04-25T09:01:33.371551+0000","last_clean":"2024-04-25T09:01:33.371551+0000","last_became_active":"2024-04-25T09:01:33.370255+0000","last_became_peered":"2024-04-25T09:01:33.370255+0000","last_unstale":"2024-04-25T09:01:33.371551+0000","last_undegraded":"2024-04-25T09:01:33.371551+0000","last_fullsized":"2024-04-25T09:01:33.371551+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:10.849550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356865+0000","last_change":"2024-04-25T09:01:33.356865+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356865+0000","last_undegraded":"2024-04-25T09:01:33.356865+0000","last_fullsized":"2024-04-25T09:01:33.356865+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.365141+0000","last_change":"2024-04-25T09:01:33.364409+0000","last_active":"2024-04-25T09:01:33.365141+0000","last_peered":"2024-04-25T09:01:33.365141+0000","last_clean":"2024-04-25T09:01:33.365141+0000","last_became_active":"2024-04-25T09:01:33.364089+0000","last_became_peered":"2024-04-25T09:01:33.364089+0000","last_unstale":"2024-04-25T09:01:33.365141+0000","last_undegraded":"2024-04-25T09:01:33.365141+0000","last_fullsized":"2024-04-25T09:01:33.365141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:58.745035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.355038+0000","last_change":"2024-04-25T09:01:33.355038+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.355038+0000","last_undegraded":"2024-04-25T09:01:33.355038+0000","last_fullsized":"2024-04-25T09:01:33.355038+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.367852+0000","last_change":"2024-04-25T09:01:33.366607+0000","last_active":"2024-04-25T09:01:33.367852+0000","last_peered":"2024-04-25T09:01:33.367852+0000","last_clean":"2024-04-25T09:01:33.367852+0000","last_became_active":"2024-04-25T09:01:33.366048+0000","last_became_peered":"2024-04-25T09:01:33.366048+0000","last_unstale":"2024-04-25T09:01:33.367852+0000","last_undegraded":"2024-04-25T09:01:33.367852+0000","last_fullsized":"2024-04-25T09:01:33.367852+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:07.729885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.353719+0000","last_change":"2024-04-25T09:01:33.353719+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.353719+0000","last_undegraded":"2024-04-25T09:01:33.353719+0000","last_fullsized":"2024-04-25T09:01:33.353719+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.369413+0000","last_change":"2024-04-25T09:01:33.368065+0000","last_active":"2024-04-25T09:01:33.369413+0000","last_peered":"2024-04-25T09:01:33.369413+0000","last_clean":"2024-04-25T09:01:33.369413+0000","last_became_active":"2024-04-25T09:01:33.367711+0000","last_became_peered":"2024-04-25T09:01:33.367711+0000","last_unstale":"2024-04-25T09:01:33.369413+0000","last_undegraded":"2024-04-25T09:01:33.369413+0000","last_fullsized":"2024-04-25T09:01:33.369413+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:39.790922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.358486+0000","last_change":"2024-04-25T09:01:33.358486+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.358486+0000","last_undegraded":"2024-04-25T09:01:33.358486+0000","last_fullsized":"2024-04-25T09:01:33.358486+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.366205+0000","last_change":"2024-04-25T09:01:33.365212+0000","last_active":"2024-04-25T09:01:33.366205+0000","last_peered":"2024-04-25T09:01:33.366205+0000","last_clean":"2024-04-25T09:01:33.366205+0000","last_became_active":"2024-04-25T09:01:33.364945+0000","last_became_peered":"2024-04-25T09:01:33.364945+0000","last_unstale":"2024-04-25T09:01:33.366205+0000","last_undegraded":"2024-04-25T09:01:33.366205+0000","last_fullsized":"2024-04-25T09:01:33.366205+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:15.517604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.352679+0000","last_change":"2024-04-25T09:01:33.352679+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.352679+0000","last_undegraded":"2024-04-25T09:01:33.352679+0000","last_fullsized":"2024-04-25T09:01:33.352679+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.370501+0000","last_change":"2024-04-25T09:01:33.367854+0000","last_active":"2024-04-25T09:01:33.370501+0000","last_peered":"2024-04-25T09:01:33.370501+0000","last_clean":"2024-04-25T09:01:33.370501+0000","last_became_active":"2024-04-25T09:01:33.366038+0000","last_became_peered":"2024-04-25T09:01:33.366038+0000","last_unstale":"2024-04-25T09:01:33.370501+0000","last_undegraded":"2024-04-25T09:01:33.370501+0000","last_fullsized":"2024-04-25T09:01:33.370501+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:55.938665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356269+0000","last_change":"2024-04-25T09:01:33.356269+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356269+0000","last_undegraded":"2024-04-25T09:01:33.356269+0000","last_fullsized":"2024-04-25T09:01:33.356269+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.370382+0000","last_change":"2024-04-25T09:01:33.369961+0000","last_active":"2024-04-25T09:01:33.370382+0000","last_peered":"2024-04-25T09:01:33.370382+0000","last_clean":"2024-04-25T09:01:33.370382+0000","last_became_active":"2024-04-25T09:01:33.369606+0000","last_became_peered":"2024-04-25T09:01:33.369606+0000","last_unstale":"2024-04-25T09:01:33.370382+0000","last_undegraded":"2024-04-25T09:01:33.370382+0000","last_fullsized":"2024-04-25T09:01:33.370382+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:10.849550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.353264+0000","last_change":"2024-04-25T09:01:33.353264+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.353264+0000","last_undegraded":"2024-04-25T09:01:33.353264+0000","last_fullsized":"2024-04-25T09:01:33.353264+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.369396+0000","last_change":"2024-04-25T09:01:33.364868+0000","last_active":"2024-04-25T09:01:33.369396+0000","last_peered":"2024-04-25T09:01:33.369396+0000","last_clean":"2024-04-25T09:01:33.369396+0000","last_became_active":"2024-04-25T09:01:33.364604+0000","last_became_peered":"2024-04-25T09:01:33.364604+0000","last_unstale":"2024-04-25T09:01:33.369396+0000","last_undegraded":"2024-04-25T09:01:33.369396+0000","last_fullsized":"2024-04-25T09:01:33.369396+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:29.149495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.361011+0000","last_change":"2024-04-25T09:01:33.361011+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.361011+0000","last_undegraded":"2024-04-25T09:01:33.361011+0000","last_fullsized":"2024-04-25T09:01:33.361011+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.368490+0000","last_change":"2024-04-25T09:01:33.364144+0000","last_active":"2024-04-25T09:01:33.368490+0000","last_peered":"2024-04-25T09:01:33.368490+0000","last_clean":"2024-04-25T09:01:33.368490+0000","last_became_active":"2024-04-25T09:01:33.363517+0000","last_became_peered":"2024-04-25T09:01:33.363517+0000","last_unstale":"2024-04-25T09:01:33.368490+0000","last_undegraded":"2024-04-25T09:01:33.368490+0000","last_fullsized":"2024-04-25T09:01:33.368490+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:04.891189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.352757+0000","last_change":"2024-04-25T09:01:33.352757+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.352757+0000","last_undegraded":"2024-04-25T09:01:33.352757+0000","last_fullsized":"2024-04-25T09:01:33.352757+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.368385+0000","last_change":"2024-04-25T09:01:33.367042+0000","last_active":"2024-04-25T09:01:33.368385+0000","last_peered":"2024-04-25T09:01:33.368385+0000","last_clean":"2024-04-25T09:01:33.368385+0000","last_became_active":"2024-04-25T09:01:33.366746+0000","last_became_peered":"2024-04-25T09:01:33.366746+0000","last_unstale":"2024-04-25T09:01:33.368385+0000","last_undegraded":"2024-04-25T09:01:33.368385+0000","last_fullsized":"2024-04-25T09:01:33.368385+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:34.749007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":317,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.357167+0000","last_change":"2024-04-25T08:54:00.818044+0000","last_active":"2024-04-25T09:01:33.357167+0000","last_peered":"2024-04-25T09:01:33.357167+0000","last_clean":"2024-04-25T09:01:33.357167+0000","last_became_active":"2024-04-25T08:54:00.817098+0000","last_became_peered":"2024-04-25T08:54:00.817098+0000","last_unstale":"2024-04-25T09:01:33.357167+0000","last_undegraded":"2024-04-25T09:01:33.357167+0000","last_fullsized":"2024-04-25T09:01:33.357167+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_clean_scrub_stamp":"2024-04-25T08:53:59.739677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:48.129513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574941,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55168,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316672,"statfs":{"total":96636764160,"available":96580272128,"internally_reserved":0,"allocated":1818624,"data_stored":1118350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574942,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80956,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94290884,"statfs":{"total":96636764160,"available":96553865216,"internally_reserved":0,"allocated":1224704,"data_stored":527982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":81656262},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574944,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50436,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321404,"statfs":{"total":96636764160,"available":96585117696,"internally_reserved":0,"allocated":1236992,"data_stored":534257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574942,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64328,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94307512,"statfs":{"total":96636764160,"available":96570892288,"internally_reserved":0,"allocated":1236992,"data_stored":534257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574942,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68488,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303352,"statfs":{"total":96636764160,"available":96566632448,"internally_reserved":0,"allocated":1236992,"data_stored":534257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574942,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58756,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313084,"statfs":{"total":96636764160,"available":96576598016,"internally_reserved":0,"allocated":1236992,"data_stored":534257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76804,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94295036,"statfs":{"total":96636764160,"available":96558116864,"internally_reserved":0,"allocated":1236992,"data_stored":534257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574943,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59340,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312500,"statfs":{"total":96636764160,"available":96576000000,"internally_reserved":0,"allocated":1830912,"data_stored":1124625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:01:37.358 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:01:37.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.506+0000 7f3246676640 1 Processor -- start 2024-04-25T09:01:37.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.506+0000 7f3246676640 1 -- start start 2024-04-25T09:01:37.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.507+0000 7f3246676640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f324015dce0 0x7f324015ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:37.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.507+0000 7f3246676640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32400d2920 con 0x7f32400ce550 2024-04-25T09:01:37.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.507+0000 7f3246676640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32400cee80 con 0x7f32400d0af0 2024-04-25T09:01:37.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.507+0000 7f3246676640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32400cf160 con 0x7f324015dce0 2024-04-25T09:01:37.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.507+0000 7f323f7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f32400ce550 0x7f32400d03e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:37572/0 (socket says 172.21.15.43:37572) 2024-04-25T09:01:37.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.507+0000 7f323f7fe640 1 -- 172.21.15.43:0/2275632099 learned_addr learned my addr 172.21.15.43:0/2275632099 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:37.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.507+0000 7f323ffff640 1 --2- 172.21.15.43:0/2275632099 >> v2:172.21.15.193:3300/0 conn(0x7f324015dce0 0x7f324015ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:37.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.507+0000 7f323effd640 1 -- 172.21.15.43:0/2275632099 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1143548628 0 0) 0x7f32400d2920 con 0x7f32400ce550 2024-04-25T09:01:37.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.507+0000 7f323effd640 1 -- 172.21.15.43:0/2275632099 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3228003520 con 0x7f32400ce550 2024-04-25T09:01:37.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.507+0000 7f323effd640 1 -- 172.21.15.43:0/2275632099 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1050609272 0 0) 0x7f3228003520 con 0x7f32400ce550 2024-04-25T09:01:37.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.507+0000 7f323effd640 1 -- 172.21.15.43:0/2275632099 >> v2:172.21.15.193:3300/0 conn(0x7f324015dce0 msgr2=0x7f324015ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:37.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.507+0000 7f323effd640 1 --2- 172.21.15.43:0/2275632099 >> v2:172.21.15.193:3300/0 conn(0x7f324015dce0 0x7f324015ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:37.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.507+0000 7f323effd640 1 -- 172.21.15.43:0/2275632099 >> v1:172.21.15.193:6789/0 conn(0x7f32400d0af0 legacy=0x7f3240160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:37.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.507+0000 7f323effd640 1 -- 172.21.15.43:0/2275632099 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32400cf440 con 0x7f32400ce550 2024-04-25T09:01:37.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.507+0000 7f323effd640 1 -- 172.21.15.43:0/2275632099 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f3234001690 con 0x7f32400ce550 2024-04-25T09:01:37.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.507+0000 7f323effd640 1 -- 172.21.15.43:0/2275632099 >> v1:172.21.15.43:6789/0 conn(0x7f32400ce550 legacy=0x7f32400d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:37.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.507+0000 7f323effd640 1 --2- 172.21.15.43:0/2275632099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32280039b0 0x7f3228003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:37.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.507+0000 7f323effd640 1 --2- 172.21.15.43:0/2275632099 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3228004800 0x7f3228004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:37.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.507+0000 7f323effd640 1 --2- 172.21.15.43:0/2275632099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32280051c0 0x7f322800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:37.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.507+0000 7f323effd640 1 -- 172.21.15.43:0/2275632099 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32400cf160 con 0x7f32280051c0 2024-04-25T09:01:37.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.507+0000 7f323effd640 1 -- 172.21.15.43:0/2275632099 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322800c160 con 0x7f32280039b0 2024-04-25T09:01:37.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.508+0000 7f323effd640 1 -- 172.21.15.43:0/2275632099 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f322800c360 con 0x7f3228004800 2024-04-25T09:01:37.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.508+0000 7f323ffff640 1 --2- 172.21.15.43:0/2275632099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32280039b0 0x7f3228003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:37.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.508+0000 7f3244c13640 1 --2- 172.21.15.43:0/2275632099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32280051c0 0x7f322800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:37.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.508+0000 7f323f7fe640 1 --2- 172.21.15.43:0/2275632099 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3228004800 0x7f3228004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:37.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.508+0000 7f3244c13640 1 -- 172.21.15.43:0/2275632099 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3228004800 msgr2=0x7f3228004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:37.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.508+0000 7f3244c13640 1 --2- 172.21.15.43:0/2275632099 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3228004800 0x7f3228004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:37.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.508+0000 7f3244c13640 1 -- 172.21.15.43:0/2275632099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32280039b0 msgr2=0x7f3228003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:37.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.508+0000 7f3244c13640 1 --2- 172.21.15.43:0/2275632099 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32280039b0 0x7f3228003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:37.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.508+0000 7f3244c13640 1 -- 172.21.15.43:0/2275632099 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3228003520 con 0x7f32280051c0 2024-04-25T09:01:37.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.508+0000 7f323f7fe640 1 --2- 172.21.15.43:0/2275632099 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3228004800 0x7f3228004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:01:37.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.508+0000 7f3244c13640 1 -- 172.21.15.43:0/2275632099 reap_dead start 2024-04-25T09:01:37.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.509+0000 7f3244c13640 1 --2- 172.21.15.43:0/2275632099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32280051c0 0x7f322800b9c0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f3238001e50 tx=0x7f3228005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df2a5416ab47890d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:37.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.509+0000 7f323effd640 1 -- 172.21.15.43:0/2275632099 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3238001a70 con 0x7f32280051c0 2024-04-25T09:01:37.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.509+0000 7f323effd640 1 -- 172.21.15.43:0/2275632099 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3238003970 con 0x7f32280051c0 2024-04-25T09:01:37.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.509+0000 7f323effd640 1 -- 172.21.15.43:0/2275632099 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3238003d60 con 0x7f32280051c0 2024-04-25T09:01:37.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.509+0000 7f3246676640 1 -- 172.21.15.43:0/2275632099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32280051c0 msgr2=0x7f322800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:37.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.509+0000 7f3246676640 1 --2- 172.21.15.43:0/2275632099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32280051c0 0x7f322800b9c0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f3238001e50 tx=0x7f3228005830 comp rx=0 tx=0).stop 2024-04-25T09:01:37.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.509+0000 7f3246676640 1 -- 172.21.15.43:0/2275632099 shutdown_connections 2024-04-25T09:01:37.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.509+0000 7f3246676640 1 --2- 172.21.15.43:0/2275632099 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32280051c0 0x7f322800b9c0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:37.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.509+0000 7f3246676640 1 -- 172.21.15.43:0/2275632099 >> 172.21.15.43:0/2275632099 conn(0x7f324009cd00 msgr2=0x7f32400d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:37.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.509+0000 7f3246676640 1 -- 172.21.15.43:0/2275632099 shutdown_connections 2024-04-25T09:01:37.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.509+0000 7f3246676640 1 -- 172.21.15.43:0/2275632099 wait complete. 2024-04-25T09:01:37.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.510+0000 7f3246676640 1 Processor -- start 2024-04-25T09:01:37.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.511+0000 7f3246676640 1 -- start start 2024-04-25T09:01:37.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.511+0000 7f3246676640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32280051c0 0x7f32400b8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:37.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.511+0000 7f3246676640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f32400a0ef0 0x7f32400a0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:37.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.511+0000 7f3246676640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32401f75e0 0x7f32401f7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:37.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.511+0000 7f3246676640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324015e0c0 con 0x7f32401f75e0 2024-04-25T09:01:37.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.511+0000 7f3246676640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f324015f6b0 con 0x7f32280051c0 2024-04-25T09:01:37.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.511+0000 7f3246676640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32400ce9c0 con 0x7f32400a0ef0 2024-04-25T09:01:37.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.512+0000 7f3244c13640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32401f75e0 0x7f32401f7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:37.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.512+0000 7f3244c13640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32401f75e0 0x7f32401f7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49076/0 (socket says 172.21.15.43:49076) 2024-04-25T09:01:37.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.512+0000 7f3244c13640 1 -- 172.21.15.43:0/2774430019 learned_addr learned my addr 172.21.15.43:0/2774430019 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:37.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.512+0000 7f323f7fe640 1 --2- 172.21.15.43:0/2774430019 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f32400a0ef0 0x7f32400a0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:37.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.512+0000 7f323ffff640 1 --2- 172.21.15.43:0/2774430019 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32280051c0 0x7f32400b8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:37.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.512+0000 7f3244c13640 1 -- 172.21.15.43:0/2774430019 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f32400a0ef0 msgr2=0x7f32400a0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:37.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.512+0000 7f3244c13640 1 --2- 172.21.15.43:0/2774430019 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f32400a0ef0 0x7f32400a0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:37.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.512+0000 7f3244c13640 1 -- 172.21.15.43:0/2774430019 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32280051c0 msgr2=0x7f32400b8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:37.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.512+0000 7f3244c13640 1 --2- 172.21.15.43:0/2774430019 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32280051c0 0x7f32400b8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:37.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.512+0000 7f3244c13640 1 -- 172.21.15.43:0/2774430019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32400a13f0 con 0x7f32401f75e0 2024-04-25T09:01:37.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.512+0000 7f3244c13640 1 --2- 172.21.15.43:0/2774430019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32401f75e0 0x7f32401f7a90 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f3238001f70 tx=0x7f32380014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:37.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.512+0000 7f323cff9640 1 -- 172.21.15.43:0/2774430019 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3238007db0 con 0x7f32401f75e0 2024-04-25T09:01:37.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.512+0000 7f323cff9640 1 -- 172.21.15.43:0/2774430019 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f323802ac70 con 0x7f32401f75e0 2024-04-25T09:01:37.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.512+0000 7f3246676640 1 -- 172.21.15.43:0/2774430019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32400b8510 con 0x7f32401f75e0 2024-04-25T09:01:37.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.512+0000 7f323cff9640 1 -- 172.21.15.43:0/2774430019 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3238001a70 con 0x7f32401f75e0 2024-04-25T09:01:37.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.512+0000 7f3246676640 1 -- 172.21.15.43:0/2774430019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32400b8ad0 con 0x7f32401f75e0 2024-04-25T09:01:37.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.513+0000 7f3246676640 1 -- 172.21.15.43:0/2774430019 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32400680f0 con 0x7f32401f75e0 2024-04-25T09:01:37.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.514+0000 7f323cff9640 1 -- 172.21.15.43:0/2774430019 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3238007030 con 0x7f32401f75e0 2024-04-25T09:01:37.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.514+0000 7f323cff9640 1 --2- 172.21.15.43:0/2774430019 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f322c06b500 0x7f322c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:37.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.514+0000 7f323cff9640 1 -- 172.21.15.43:0/2774430019 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f32380a9880 con 0x7f32401f75e0 2024-04-25T09:01:37.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.514+0000 7f323ffff640 1 --2- 172.21.15.43:0/2774430019 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f322c06b500 0x7f322c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:37.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.515+0000 7f323ffff640 1 --2- 172.21.15.43:0/2774430019 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f322c06b500 0x7f322c06d9c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f324009de00 tx=0x7f3230009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:37.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.519+0000 7f323cff9640 1 -- 172.21.15.43:0/2774430019 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3238043060 con 0x7f32401f75e0 2024-04-25T09:01:37.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.694+0000 7f3246676640 1 -- 172.21.15.43:0/2774430019 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f32400a16d0 con 0x7f322c06b500 2024-04-25T09:01:37.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.701+0000 7f323cff9640 1 -- 172.21.15.43:0/2774430019 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247013 (secure 0 0 0) 0x7f32400a16d0 con 0x7f322c06b500 2024-04-25T09:01:37.692 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:01:37.693 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-25T09:01:37.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.704+0000 7f3246676640 1 -- 172.21.15.43:0/2774430019 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f322c06b500 msgr2=0x7f322c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:37.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.704+0000 7f3246676640 1 --2- 172.21.15.43:0/2774430019 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f322c06b500 0x7f322c06d9c0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f324009de00 tx=0x7f3230009190 comp rx=0 tx=0).stop 2024-04-25T09:01:37.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.704+0000 7f3246676640 1 -- 172.21.15.43:0/2774430019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32401f75e0 msgr2=0x7f32401f7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:37.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.704+0000 7f3246676640 1 --2- 172.21.15.43:0/2774430019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32401f75e0 0x7f32401f7a90 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f3238001f70 tx=0x7f32380014a0 comp rx=0 tx=0).stop 2024-04-25T09:01:37.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.704+0000 7f3246676640 1 -- 172.21.15.43:0/2774430019 shutdown_connections 2024-04-25T09:01:37.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.704+0000 7f3246676640 1 --2- 172.21.15.43:0/2774430019 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f32401f75e0 0x7f32401f7a90 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:37.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.704+0000 7f3246676640 1 --2- 172.21.15.43:0/2774430019 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f32400a0ef0 0x7f32400a0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:37.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.704+0000 7f3246676640 1 --2- 172.21.15.43:0/2774430019 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f322c06b500 0x7f322c06d9c0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:37.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.704+0000 7f3246676640 1 --2- 172.21.15.43:0/2774430019 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f32280051c0 0x7f32400b8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:37.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.704+0000 7f3246676640 1 -- 172.21.15.43:0/2774430019 >> 172.21.15.43:0/2774430019 conn(0x7f324009cd00 msgr2=0x7f324006c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:37.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.704+0000 7f3246676640 1 -- 172.21.15.43:0/2774430019 shutdown_connections 2024-04-25T09:01:37.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.704+0000 7f3246676640 1 -- 172.21.15.43:0/2774430019 wait complete. 2024-04-25T09:01:37.704 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2024-04-25T09:01:35.796679+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":24,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":514276,"kb_used_data":10800,"kb_used_omap":12,"kb_used_meta":503347,"kb_avail":754460444,"statfs":{"total":773094113280,"available":772567494656,"internally_reserved":0,"allocated":11059200,"data_stored":5442242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":515427889},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001055"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.355534+0000","last_change":"2024-04-25T09:01:33.355534+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.355534+0000","last_undegraded":"2024-04-25T09:01:33.355534+0000","last_fullsized":"2024-04-25T09:01:33.355534+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.359660+0000","last_change":"2024-04-25T09:01:33.359660+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.359660+0000","last_undegraded":"2024-04-25T09:01:33.359660+0000","last_fullsized":"2024-04-25T09:01:33.359660+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.355300+0000","last_change":"2024-04-25T09:01:33.355300+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.355300+0000","last_undegraded":"2024-04-25T09:01:33.355300+0000","last_fullsized":"2024-04-25T09:01:33.355300+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356488+0000","last_change":"2024-04-25T09:01:33.356488+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356488+0000","last_undegraded":"2024-04-25T09:01:33.356488+0000","last_fullsized":"2024-04-25T09:01:33.356488+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.358970+0000","last_change":"2024-04-25T09:01:33.358970+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.358970+0000","last_undegraded":"2024-04-25T09:01:33.358970+0000","last_fullsized":"2024-04-25T09:01:33.358970+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.358980+0000","last_change":"2024-04-25T09:01:33.358980+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.358980+0000","last_undegraded":"2024-04-25T09:01:33.358980+0000","last_fullsized":"2024-04-25T09:01:33.358980+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356706+0000","last_change":"2024-04-25T09:01:33.356706+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356706+0000","last_undegraded":"2024-04-25T09:01:33.356706+0000","last_fullsized":"2024-04-25T09:01:33.356706+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.357495+0000","last_change":"2024-04-25T09:01:33.357495+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.357495+0000","last_undegraded":"2024-04-25T09:01:33.357495+0000","last_fullsized":"2024-04-25T09:01:33.357495+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.359846+0000","last_change":"2024-04-25T09:01:33.359846+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.359846+0000","last_undegraded":"2024-04-25T09:01:33.359846+0000","last_fullsized":"2024-04-25T09:01:33.359846+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.358143+0000","last_change":"2024-04-25T09:01:33.358143+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.358143+0000","last_undegraded":"2024-04-25T09:01:33.358143+0000","last_fullsized":"2024-04-25T09:01:33.358143+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.358698+0000","last_change":"2024-04-25T09:01:33.358698+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.358698+0000","last_undegraded":"2024-04-25T09:01:33.358698+0000","last_fullsized":"2024-04-25T09:01:33.358698+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356492+0000","last_change":"2024-04-25T09:01:33.356492+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356492+0000","last_undegraded":"2024-04-25T09:01:33.356492+0000","last_fullsized":"2024-04-25T09:01:33.356492+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.355925+0000","last_change":"2024-04-25T09:01:33.355925+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.355925+0000","last_undegraded":"2024-04-25T09:01:33.355925+0000","last_fullsized":"2024-04-25T09:01:33.355925+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.362266+0000","last_change":"2024-04-25T09:01:33.362266+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.362266+0000","last_undegraded":"2024-04-25T09:01:33.362266+0000","last_fullsized":"2024-04-25T09:01:33.362266+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.358146+0000","last_change":"2024-04-25T09:01:33.358146+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.358146+0000","last_undegraded":"2024-04-25T09:01:33.358146+0000","last_fullsized":"2024-04-25T09:01:33.358146+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356808+0000","last_change":"2024-04-25T09:01:33.356808+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356808+0000","last_undegraded":"2024-04-25T09:01:33.356808+0000","last_fullsized":"2024-04-25T09:01:33.356808+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.358063+0000","last_change":"2024-04-25T09:01:33.358063+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.358063+0000","last_undegraded":"2024-04-25T09:01:33.358063+0000","last_fullsized":"2024-04-25T09:01:33.358063+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.358540+0000","last_change":"2024-04-25T09:01:33.358540+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.358540+0000","last_undegraded":"2024-04-25T09:01:33.358540+0000","last_fullsized":"2024-04-25T09:01:33.358540+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356094+0000","last_change":"2024-04-25T09:01:33.356094+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356094+0000","last_undegraded":"2024-04-25T09:01:33.356094+0000","last_fullsized":"2024-04-25T09:01:33.356094+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.355041+0000","last_change":"2024-04-25T09:01:33.355041+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.355041+0000","last_undegraded":"2024-04-25T09:01:33.355041+0000","last_fullsized":"2024-04-25T09:01:33.355041+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.354826+0000","last_change":"2024-04-25T09:01:33.354826+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.354826+0000","last_undegraded":"2024-04-25T09:01:33.354826+0000","last_fullsized":"2024-04-25T09:01:33.354826+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.358378+0000","last_change":"2024-04-25T09:01:33.358378+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.358378+0000","last_undegraded":"2024-04-25T09:01:33.358378+0000","last_fullsized":"2024-04-25T09:01:33.358378+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356711+0000","last_change":"2024-04-25T09:01:33.356711+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356711+0000","last_undegraded":"2024-04-25T09:01:33.356711+0000","last_fullsized":"2024-04-25T09:01:33.356711+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356347+0000","last_change":"2024-04-25T09:01:33.356347+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356347+0000","last_undegraded":"2024-04-25T09:01:33.356347+0000","last_fullsized":"2024-04-25T09:01:33.356347+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.362666+0000","last_change":"2024-04-25T09:01:33.362666+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.362666+0000","last_undegraded":"2024-04-25T09:01:33.362666+0000","last_fullsized":"2024-04-25T09:01:33.362666+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.359240+0000","last_change":"2024-04-25T09:01:33.359240+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.359240+0000","last_undegraded":"2024-04-25T09:01:33.359240+0000","last_fullsized":"2024-04-25T09:01:33.359240+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.358208+0000","last_change":"2024-04-25T09:01:33.358208+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.358208+0000","last_undegraded":"2024-04-25T09:01:33.358208+0000","last_fullsized":"2024-04-25T09:01:33.358208+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.363030+0000","last_change":"2024-04-25T09:01:33.363030+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.363030+0000","last_undegraded":"2024-04-25T09:01:33.363030+0000","last_fullsized":"2024-04-25T09:01:33.363030+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.357869+0000","last_change":"2024-04-25T09:01:33.357869+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.357869+0000","last_undegraded":"2024-04-25T09:01:33.357869+0000","last_fullsized":"2024-04-25T09:01:33.357869+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.355005+0000","last_change":"2024-04-25T09:01:33.355005+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.355005+0000","last_undegraded":"2024-04-25T09:01:33.355005+0000","last_fullsized":"2024-04-25T09:01:33.355005+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.357607+0000","last_change":"2024-04-25T09:01:33.357607+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.357607+0000","last_undegraded":"2024-04-25T09:01:33.357607+0000","last_fullsized":"2024-04-25T09:01:33.357607+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.354229+0000","last_change":"2024-04-25T09:01:33.354229+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.354229+0000","last_undegraded":"2024-04-25T09:01:33.354229+0000","last_fullsized":"2024-04-25T09:01:33.354229+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.353890+0000","last_change":"2024-04-25T09:01:33.353890+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.353890+0000","last_undegraded":"2024-04-25T09:01:33.353890+0000","last_fullsized":"2024-04-25T09:01:33.353890+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.367304+0000","last_change":"2024-04-25T09:01:33.365666+0000","last_active":"2024-04-25T09:01:33.367304+0000","last_peered":"2024-04-25T09:01:33.367304+0000","last_clean":"2024-04-25T09:01:33.367304+0000","last_became_active":"2024-04-25T09:01:33.365341+0000","last_became_peered":"2024-04-25T09:01:33.365341+0000","last_unstale":"2024-04-25T09:01:33.367304+0000","last_undegraded":"2024-04-25T09:01:33.367304+0000","last_fullsized":"2024-04-25T09:01:33.367304+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:41.915410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356285+0000","last_change":"2024-04-25T09:01:33.356285+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356285+0000","last_undegraded":"2024-04-25T09:01:33.356285+0000","last_fullsized":"2024-04-25T09:01:33.356285+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.368360+0000","last_change":"2024-04-25T09:01:33.366268+0000","last_active":"2024-04-25T09:01:33.368360+0000","last_peered":"2024-04-25T09:01:33.368360+0000","last_clean":"2024-04-25T09:01:33.368360+0000","last_became_active":"2024-04-25T09:01:33.365952+0000","last_became_peered":"2024-04-25T09:01:33.365952+0000","last_unstale":"2024-04-25T09:01:33.368360+0000","last_undegraded":"2024-04-25T09:01:33.368360+0000","last_fullsized":"2024-04-25T09:01:33.368360+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:29.149495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.354216+0000","last_change":"2024-04-25T09:01:33.354216+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.354216+0000","last_undegraded":"2024-04-25T09:01:33.354216+0000","last_fullsized":"2024-04-25T09:01:33.354216+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.371487+0000","last_change":"2024-04-25T09:01:33.370140+0000","last_active":"2024-04-25T09:01:33.371487+0000","last_peered":"2024-04-25T09:01:33.371487+0000","last_clean":"2024-04-25T09:01:33.371487+0000","last_became_active":"2024-04-25T09:01:33.369677+0000","last_became_peered":"2024-04-25T09:01:33.369677+0000","last_unstale":"2024-04-25T09:01:33.371487+0000","last_undegraded":"2024-04-25T09:01:33.371487+0000","last_fullsized":"2024-04-25T09:01:33.371487+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:15.517604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.357436+0000","last_change":"2024-04-25T09:01:33.357436+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.357436+0000","last_undegraded":"2024-04-25T09:01:33.357436+0000","last_fullsized":"2024-04-25T09:01:33.357436+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.369409+0000","last_change":"2024-04-25T09:01:33.367516+0000","last_active":"2024-04-25T09:01:33.369409+0000","last_peered":"2024-04-25T09:01:33.369409+0000","last_clean":"2024-04-25T09:01:33.369409+0000","last_became_active":"2024-04-25T09:01:33.367171+0000","last_became_peered":"2024-04-25T09:01:33.367171+0000","last_unstale":"2024-04-25T09:01:33.369409+0000","last_undegraded":"2024-04-25T09:01:33.369409+0000","last_fullsized":"2024-04-25T09:01:33.369409+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:43.586923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356196+0000","last_change":"2024-04-25T09:01:33.356196+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356196+0000","last_undegraded":"2024-04-25T09:01:33.356196+0000","last_fullsized":"2024-04-25T09:01:33.356196+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.363802+0000","last_change":"2024-04-25T09:01:33.360517+0000","last_active":"2024-04-25T09:01:33.363802+0000","last_peered":"2024-04-25T09:01:33.363802+0000","last_clean":"2024-04-25T09:01:33.363802+0000","last_became_active":"2024-04-25T09:01:33.360139+0000","last_became_peered":"2024-04-25T09:01:33.360139+0000","last_unstale":"2024-04-25T09:01:33.363802+0000","last_undegraded":"2024-04-25T09:01:33.363802+0000","last_fullsized":"2024-04-25T09:01:33.363802+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:50.515806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356694+0000","last_change":"2024-04-25T09:01:33.356694+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356694+0000","last_undegraded":"2024-04-25T09:01:33.356694+0000","last_fullsized":"2024-04-25T09:01:33.356694+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.372295+0000","last_change":"2024-04-25T09:01:33.371930+0000","last_active":"2024-04-25T09:01:33.372295+0000","last_peered":"2024-04-25T09:01:33.372295+0000","last_clean":"2024-04-25T09:01:33.372295+0000","last_became_active":"2024-04-25T09:01:33.371395+0000","last_became_peered":"2024-04-25T09:01:33.371395+0000","last_unstale":"2024-04-25T09:01:33.372295+0000","last_undegraded":"2024-04-25T09:01:33.372295+0000","last_fullsized":"2024-04-25T09:01:33.372295+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:44.824026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.357302+0000","last_change":"2024-04-25T09:01:33.357302+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.357302+0000","last_undegraded":"2024-04-25T09:01:33.357302+0000","last_fullsized":"2024-04-25T09:01:33.357302+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.367580+0000","last_change":"2024-04-25T09:01:33.366526+0000","last_active":"2024-04-25T09:01:33.367580+0000","last_peered":"2024-04-25T09:01:33.367580+0000","last_clean":"2024-04-25T09:01:33.367580+0000","last_became_active":"2024-04-25T09:01:33.366246+0000","last_became_peered":"2024-04-25T09:01:33.366246+0000","last_unstale":"2024-04-25T09:01:33.367580+0000","last_undegraded":"2024-04-25T09:01:33.367580+0000","last_fullsized":"2024-04-25T09:01:33.367580+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:26.679025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.355141+0000","last_change":"2024-04-25T09:01:33.355141+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.355141+0000","last_undegraded":"2024-04-25T09:01:33.355141+0000","last_fullsized":"2024-04-25T09:01:33.355141+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.369410+0000","last_change":"2024-04-25T09:01:33.367167+0000","last_active":"2024-04-25T09:01:33.369410+0000","last_peered":"2024-04-25T09:01:33.369410+0000","last_clean":"2024-04-25T09:01:33.369410+0000","last_became_active":"2024-04-25T09:01:33.365541+0000","last_became_peered":"2024-04-25T09:01:33.365541+0000","last_unstale":"2024-04-25T09:01:33.369410+0000","last_undegraded":"2024-04-25T09:01:33.369410+0000","last_fullsized":"2024-04-25T09:01:33.369410+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:22.335074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.357332+0000","last_change":"2024-04-25T09:01:33.357332+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.357332+0000","last_undegraded":"2024-04-25T09:01:33.357332+0000","last_fullsized":"2024-04-25T09:01:33.357332+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.363381+0000","last_change":"2024-04-25T09:01:33.361825+0000","last_active":"2024-04-25T09:01:33.363381+0000","last_peered":"2024-04-25T09:01:33.363381+0000","last_clean":"2024-04-25T09:01:33.363381+0000","last_became_active":"2024-04-25T09:01:33.361544+0000","last_became_peered":"2024-04-25T09:01:33.361544+0000","last_unstale":"2024-04-25T09:01:33.363381+0000","last_undegraded":"2024-04-25T09:01:33.363381+0000","last_fullsized":"2024-04-25T09:01:33.363381+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:19.818176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.354331+0000","last_change":"2024-04-25T09:01:33.354331+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.354331+0000","last_undegraded":"2024-04-25T09:01:33.354331+0000","last_fullsized":"2024-04-25T09:01:33.354331+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.369815+0000","last_change":"2024-04-25T09:01:33.365387+0000","last_active":"2024-04-25T09:01:33.369815+0000","last_peered":"2024-04-25T09:01:33.369815+0000","last_clean":"2024-04-25T09:01:33.369815+0000","last_became_active":"2024-04-25T09:01:33.365088+0000","last_became_peered":"2024-04-25T09:01:33.365088+0000","last_unstale":"2024-04-25T09:01:33.369815+0000","last_undegraded":"2024-04-25T09:01:33.369815+0000","last_fullsized":"2024-04-25T09:01:33.369815+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:34.749007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.362314+0000","last_change":"2024-04-25T09:01:33.362314+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.362314+0000","last_undegraded":"2024-04-25T09:01:33.362314+0000","last_fullsized":"2024-04-25T09:01:33.362314+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.368520+0000","last_change":"2024-04-25T09:01:33.367517+0000","last_active":"2024-04-25T09:01:33.368520+0000","last_peered":"2024-04-25T09:01:33.368520+0000","last_clean":"2024-04-25T09:01:33.368520+0000","last_became_active":"2024-04-25T09:01:33.367216+0000","last_became_peered":"2024-04-25T09:01:33.367216+0000","last_unstale":"2024-04-25T09:01:33.368520+0000","last_undegraded":"2024-04-25T09:01:33.368520+0000","last_fullsized":"2024-04-25T09:01:33.368520+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:42.045982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.357735+0000","last_change":"2024-04-25T09:01:33.357735+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.357735+0000","last_undegraded":"2024-04-25T09:01:33.357735+0000","last_fullsized":"2024-04-25T09:01:33.357735+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.367757+0000","last_change":"2024-04-25T09:01:33.366291+0000","last_active":"2024-04-25T09:01:33.367757+0000","last_peered":"2024-04-25T09:01:33.367757+0000","last_clean":"2024-04-25T09:01:33.367757+0000","last_became_active":"2024-04-25T09:01:33.365670+0000","last_became_peered":"2024-04-25T09:01:33.365670+0000","last_unstale":"2024-04-25T09:01:33.367757+0000","last_undegraded":"2024-04-25T09:01:33.367757+0000","last_fullsized":"2024-04-25T09:01:33.367757+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:35.172322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.355949+0000","last_change":"2024-04-25T09:01:33.355949+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.355949+0000","last_undegraded":"2024-04-25T09:01:33.355949+0000","last_fullsized":"2024-04-25T09:01:33.355949+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.368724+0000","last_change":"2024-04-25T09:01:33.363784+0000","last_active":"2024-04-25T09:01:33.368724+0000","last_peered":"2024-04-25T09:01:33.368724+0000","last_clean":"2024-04-25T09:01:33.368724+0000","last_became_active":"2024-04-25T09:01:33.363279+0000","last_became_peered":"2024-04-25T09:01:33.363279+0000","last_unstale":"2024-04-25T09:01:33.368724+0000","last_undegraded":"2024-04-25T09:01:33.368724+0000","last_fullsized":"2024-04-25T09:01:33.368724+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:15.504041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.354732+0000","last_change":"2024-04-25T09:01:33.354732+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.354732+0000","last_undegraded":"2024-04-25T09:01:33.354732+0000","last_fullsized":"2024-04-25T09:01:33.354732+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.368515+0000","last_change":"2024-04-25T09:01:33.365837+0000","last_active":"2024-04-25T09:01:33.368515+0000","last_peered":"2024-04-25T09:01:33.368515+0000","last_clean":"2024-04-25T09:01:33.368515+0000","last_became_active":"2024-04-25T09:01:33.362985+0000","last_became_peered":"2024-04-25T09:01:33.362985+0000","last_unstale":"2024-04-25T09:01:33.368515+0000","last_undegraded":"2024-04-25T09:01:33.368515+0000","last_fullsized":"2024-04-25T09:01:33.368515+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:02.701373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.359064+0000","last_change":"2024-04-25T09:01:33.359064+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.359064+0000","last_undegraded":"2024-04-25T09:01:33.359064+0000","last_fullsized":"2024-04-25T09:01:33.359064+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.367829+0000","last_change":"2024-04-25T09:01:33.367032+0000","last_active":"2024-04-25T09:01:33.367829+0000","last_peered":"2024-04-25T09:01:33.367829+0000","last_clean":"2024-04-25T09:01:33.367829+0000","last_became_active":"2024-04-25T09:01:33.366662+0000","last_became_peered":"2024-04-25T09:01:33.366662+0000","last_unstale":"2024-04-25T09:01:33.367829+0000","last_undegraded":"2024-04-25T09:01:33.367829+0000","last_fullsized":"2024-04-25T09:01:33.367829+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:13.219195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.352882+0000","last_change":"2024-04-25T09:01:33.352882+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.352882+0000","last_undegraded":"2024-04-25T09:01:33.352882+0000","last_fullsized":"2024-04-25T09:01:33.352882+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.369386+0000","last_change":"2024-04-25T09:01:33.366823+0000","last_active":"2024-04-25T09:01:33.369386+0000","last_peered":"2024-04-25T09:01:33.369386+0000","last_clean":"2024-04-25T09:01:33.369386+0000","last_became_active":"2024-04-25T09:01:33.366050+0000","last_became_peered":"2024-04-25T09:01:33.366050+0000","last_unstale":"2024-04-25T09:01:33.369386+0000","last_undegraded":"2024-04-25T09:01:33.369386+0000","last_fullsized":"2024-04-25T09:01:33.369386+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:49.016771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.362269+0000","last_change":"2024-04-25T09:01:33.362269+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.362269+0000","last_undegraded":"2024-04-25T09:01:33.362269+0000","last_fullsized":"2024-04-25T09:01:33.362269+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.369665+0000","last_change":"2024-04-25T09:01:33.369232+0000","last_active":"2024-04-25T09:01:33.369665+0000","last_peered":"2024-04-25T09:01:33.369665+0000","last_clean":"2024-04-25T09:01:33.369665+0000","last_became_active":"2024-04-25T09:01:33.368905+0000","last_became_peered":"2024-04-25T09:01:33.368905+0000","last_unstale":"2024-04-25T09:01:33.369665+0000","last_undegraded":"2024-04-25T09:01:33.369665+0000","last_fullsized":"2024-04-25T09:01:33.369665+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:41.915410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.355227+0000","last_change":"2024-04-25T09:01:33.355227+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.355227+0000","last_undegraded":"2024-04-25T09:01:33.355227+0000","last_fullsized":"2024-04-25T09:01:33.355227+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.364958+0000","last_change":"2024-04-25T09:01:33.363919+0000","last_active":"2024-04-25T09:01:33.364958+0000","last_peered":"2024-04-25T09:01:33.364958+0000","last_clean":"2024-04-25T09:01:33.364958+0000","last_became_active":"2024-04-25T09:01:33.363587+0000","last_became_peered":"2024-04-25T09:01:33.363587+0000","last_unstale":"2024-04-25T09:01:33.364958+0000","last_undegraded":"2024-04-25T09:01:33.364958+0000","last_fullsized":"2024-04-25T09:01:33.364958+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:39.790922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.355533+0000","last_change":"2024-04-25T09:01:33.355533+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.355533+0000","last_undegraded":"2024-04-25T09:01:33.355533+0000","last_fullsized":"2024-04-25T09:01:33.355533+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.367244+0000","last_change":"2024-04-25T09:01:33.364976+0000","last_active":"2024-04-25T09:01:33.367244+0000","last_peered":"2024-04-25T09:01:33.367244+0000","last_clean":"2024-04-25T09:01:33.367244+0000","last_became_active":"2024-04-25T09:01:33.363617+0000","last_became_peered":"2024-04-25T09:01:33.363617+0000","last_unstale":"2024-04-25T09:01:33.367244+0000","last_undegraded":"2024-04-25T09:01:33.367244+0000","last_fullsized":"2024-04-25T09:01:33.367244+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:13.219195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.353243+0000","last_change":"2024-04-25T09:01:33.353243+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.353243+0000","last_undegraded":"2024-04-25T09:01:33.353243+0000","last_fullsized":"2024-04-25T09:01:33.353243+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.371512+0000","last_change":"2024-04-25T09:01:33.369475+0000","last_active":"2024-04-25T09:01:33.371512+0000","last_peered":"2024-04-25T09:01:33.371512+0000","last_clean":"2024-04-25T09:01:33.371512+0000","last_became_active":"2024-04-25T09:01:33.368210+0000","last_became_peered":"2024-04-25T09:01:33.368210+0000","last_unstale":"2024-04-25T09:01:33.371512+0000","last_undegraded":"2024-04-25T09:01:33.371512+0000","last_fullsized":"2024-04-25T09:01:33.371512+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:26.679025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.354197+0000","last_change":"2024-04-25T09:01:33.354197+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.354197+0000","last_undegraded":"2024-04-25T09:01:33.354197+0000","last_fullsized":"2024-04-25T09:01:33.354197+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.364947+0000","last_change":"2024-04-25T09:01:33.363025+0000","last_active":"2024-04-25T09:01:33.364947+0000","last_peered":"2024-04-25T09:01:33.364947+0000","last_clean":"2024-04-25T09:01:33.364947+0000","last_became_active":"2024-04-25T09:01:33.362724+0000","last_became_peered":"2024-04-25T09:01:33.362724+0000","last_unstale":"2024-04-25T09:01:33.364947+0000","last_undegraded":"2024-04-25T09:01:33.364947+0000","last_fullsized":"2024-04-25T09:01:33.364947+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:41.464204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.353721+0000","last_change":"2024-04-25T09:01:33.353721+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.353721+0000","last_undegraded":"2024-04-25T09:01:33.353721+0000","last_fullsized":"2024-04-25T09:01:33.353721+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.366031+0000","last_change":"2024-04-25T09:01:33.365366+0000","last_active":"2024-04-25T09:01:33.366031+0000","last_peered":"2024-04-25T09:01:33.366031+0000","last_clean":"2024-04-25T09:01:33.366031+0000","last_became_active":"2024-04-25T09:01:33.365090+0000","last_became_peered":"2024-04-25T09:01:33.365090+0000","last_unstale":"2024-04-25T09:01:33.366031+0000","last_undegraded":"2024-04-25T09:01:33.366031+0000","last_fullsized":"2024-04-25T09:01:33.366031+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:35.172322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.361446+0000","last_change":"2024-04-25T09:01:33.361446+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.361446+0000","last_undegraded":"2024-04-25T09:01:33.361446+0000","last_fullsized":"2024-04-25T09:01:33.361446+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.371551+0000","last_change":"2024-04-25T09:01:33.370545+0000","last_active":"2024-04-25T09:01:33.371551+0000","last_peered":"2024-04-25T09:01:33.371551+0000","last_clean":"2024-04-25T09:01:33.371551+0000","last_became_active":"2024-04-25T09:01:33.370255+0000","last_became_peered":"2024-04-25T09:01:33.370255+0000","last_unstale":"2024-04-25T09:01:33.371551+0000","last_undegraded":"2024-04-25T09:01:33.371551+0000","last_fullsized":"2024-04-25T09:01:33.371551+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:10.849550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356865+0000","last_change":"2024-04-25T09:01:33.356865+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356865+0000","last_undegraded":"2024-04-25T09:01:33.356865+0000","last_fullsized":"2024-04-25T09:01:33.356865+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.365141+0000","last_change":"2024-04-25T09:01:33.364409+0000","last_active":"2024-04-25T09:01:33.365141+0000","last_peered":"2024-04-25T09:01:33.365141+0000","last_clean":"2024-04-25T09:01:33.365141+0000","last_became_active":"2024-04-25T09:01:33.364089+0000","last_became_peered":"2024-04-25T09:01:33.364089+0000","last_unstale":"2024-04-25T09:01:33.365141+0000","last_undegraded":"2024-04-25T09:01:33.365141+0000","last_fullsized":"2024-04-25T09:01:33.365141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:58.745035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.355038+0000","last_change":"2024-04-25T09:01:33.355038+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.355038+0000","last_undegraded":"2024-04-25T09:01:33.355038+0000","last_fullsized":"2024-04-25T09:01:33.355038+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.367852+0000","last_change":"2024-04-25T09:01:33.366607+0000","last_active":"2024-04-25T09:01:33.367852+0000","last_peered":"2024-04-25T09:01:33.367852+0000","last_clean":"2024-04-25T09:01:33.367852+0000","last_became_active":"2024-04-25T09:01:33.366048+0000","last_became_peered":"2024-04-25T09:01:33.366048+0000","last_unstale":"2024-04-25T09:01:33.367852+0000","last_undegraded":"2024-04-25T09:01:33.367852+0000","last_fullsized":"2024-04-25T09:01:33.367852+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:07.729885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.353719+0000","last_change":"2024-04-25T09:01:33.353719+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.353719+0000","last_undegraded":"2024-04-25T09:01:33.353719+0000","last_fullsized":"2024-04-25T09:01:33.353719+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.369413+0000","last_change":"2024-04-25T09:01:33.368065+0000","last_active":"2024-04-25T09:01:33.369413+0000","last_peered":"2024-04-25T09:01:33.369413+0000","last_clean":"2024-04-25T09:01:33.369413+0000","last_became_active":"2024-04-25T09:01:33.367711+0000","last_became_peered":"2024-04-25T09:01:33.367711+0000","last_unstale":"2024-04-25T09:01:33.369413+0000","last_undegraded":"2024-04-25T09:01:33.369413+0000","last_fullsized":"2024-04-25T09:01:33.369413+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:39.790922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.358486+0000","last_change":"2024-04-25T09:01:33.358486+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.358486+0000","last_undegraded":"2024-04-25T09:01:33.358486+0000","last_fullsized":"2024-04-25T09:01:33.358486+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.366205+0000","last_change":"2024-04-25T09:01:33.365212+0000","last_active":"2024-04-25T09:01:33.366205+0000","last_peered":"2024-04-25T09:01:33.366205+0000","last_clean":"2024-04-25T09:01:33.366205+0000","last_became_active":"2024-04-25T09:01:33.364945+0000","last_became_peered":"2024-04-25T09:01:33.364945+0000","last_unstale":"2024-04-25T09:01:33.366205+0000","last_undegraded":"2024-04-25T09:01:33.366205+0000","last_fullsized":"2024-04-25T09:01:33.366205+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:15.517604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.352679+0000","last_change":"2024-04-25T09:01:33.352679+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.352679+0000","last_undegraded":"2024-04-25T09:01:33.352679+0000","last_fullsized":"2024-04-25T09:01:33.352679+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.370501+0000","last_change":"2024-04-25T09:01:33.367854+0000","last_active":"2024-04-25T09:01:33.370501+0000","last_peered":"2024-04-25T09:01:33.370501+0000","last_clean":"2024-04-25T09:01:33.370501+0000","last_became_active":"2024-04-25T09:01:33.366038+0000","last_became_peered":"2024-04-25T09:01:33.366038+0000","last_unstale":"2024-04-25T09:01:33.370501+0000","last_undegraded":"2024-04-25T09:01:33.370501+0000","last_fullsized":"2024-04-25T09:01:33.370501+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:55.938665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356269+0000","last_change":"2024-04-25T09:01:33.356269+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356269+0000","last_undegraded":"2024-04-25T09:01:33.356269+0000","last_fullsized":"2024-04-25T09:01:33.356269+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.370382+0000","last_change":"2024-04-25T09:01:33.369961+0000","last_active":"2024-04-25T09:01:33.370382+0000","last_peered":"2024-04-25T09:01:33.370382+0000","last_clean":"2024-04-25T09:01:33.370382+0000","last_became_active":"2024-04-25T09:01:33.369606+0000","last_became_peered":"2024-04-25T09:01:33.369606+0000","last_unstale":"2024-04-25T09:01:33.370382+0000","last_undegraded":"2024-04-25T09:01:33.370382+0000","last_fullsized":"2024-04-25T09:01:33.370382+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:10.849550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.353264+0000","last_change":"2024-04-25T09:01:33.353264+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.353264+0000","last_undegraded":"2024-04-25T09:01:33.353264+0000","last_fullsized":"2024-04-25T09:01:33.353264+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.369396+0000","last_change":"2024-04-25T09:01:33.364868+0000","last_active":"2024-04-25T09:01:33.369396+0000","last_peered":"2024-04-25T09:01:33.369396+0000","last_clean":"2024-04-25T09:01:33.369396+0000","last_became_active":"2024-04-25T09:01:33.364604+0000","last_became_peered":"2024-04-25T09:01:33.364604+0000","last_unstale":"2024-04-25T09:01:33.369396+0000","last_undegraded":"2024-04-25T09:01:33.369396+0000","last_fullsized":"2024-04-25T09:01:33.369396+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:29.149495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.361011+0000","last_change":"2024-04-25T09:01:33.361011+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.361011+0000","last_undegraded":"2024-04-25T09:01:33.361011+0000","last_fullsized":"2024-04-25T09:01:33.361011+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.368490+0000","last_change":"2024-04-25T09:01:33.364144+0000","last_active":"2024-04-25T09:01:33.368490+0000","last_peered":"2024-04-25T09:01:33.368490+0000","last_clean":"2024-04-25T09:01:33.368490+0000","last_became_active":"2024-04-25T09:01:33.363517+0000","last_became_peered":"2024-04-25T09:01:33.363517+0000","last_unstale":"2024-04-25T09:01:33.368490+0000","last_undegraded":"2024-04-25T09:01:33.368490+0000","last_fullsized":"2024-04-25T09:01:33.368490+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:04.891189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.352757+0000","last_change":"2024-04-25T09:01:33.352757+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.352757+0000","last_undegraded":"2024-04-25T09:01:33.352757+0000","last_fullsized":"2024-04-25T09:01:33.352757+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.368385+0000","last_change":"2024-04-25T09:01:33.367042+0000","last_active":"2024-04-25T09:01:33.368385+0000","last_peered":"2024-04-25T09:01:33.368385+0000","last_clean":"2024-04-25T09:01:33.368385+0000","last_became_active":"2024-04-25T09:01:33.366746+0000","last_became_peered":"2024-04-25T09:01:33.366746+0000","last_unstale":"2024-04-25T09:01:33.368385+0000","last_undegraded":"2024-04-25T09:01:33.368385+0000","last_fullsized":"2024-04-25T09:01:33.368385+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:34.749007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":317,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.357167+0000","last_change":"2024-04-25T08:54:00.818044+0000","last_active":"2024-04-25T09:01:33.357167+0000","last_peered":"2024-04-25T09:01:33.357167+0000","last_clean":"2024-04-25T09:01:33.357167+0000","last_became_active":"2024-04-25T08:54:00.817098+0000","last_became_peered":"2024-04-25T08:54:00.817098+0000","last_unstale":"2024-04-25T09:01:33.357167+0000","last_undegraded":"2024-04-25T09:01:33.357167+0000","last_fullsized":"2024-04-25T09:01:33.357167+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_clean_scrub_stamp":"2024-04-25T08:53:59.739677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:48.129513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574941,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55168,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316672,"statfs":{"total":96636764160,"available":96580272128,"internally_reserved":0,"allocated":1818624,"data_stored":1118350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574942,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80956,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94290884,"statfs":{"total":96636764160,"available":96553865216,"internally_reserved":0,"allocated":1224704,"data_stored":527982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":81656262},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574944,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50436,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321404,"statfs":{"total":96636764160,"available":96585117696,"internally_reserved":0,"allocated":1236992,"data_stored":534257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574942,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64328,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94307512,"statfs":{"total":96636764160,"available":96570892288,"internally_reserved":0,"allocated":1236992,"data_stored":534257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574942,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68488,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303352,"statfs":{"total":96636764160,"available":96566632448,"internally_reserved":0,"allocated":1236992,"data_stored":534257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574942,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58756,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313084,"statfs":{"total":96636764160,"available":96576598016,"internally_reserved":0,"allocated":1236992,"data_stored":534257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76804,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94295036,"statfs":{"total":96636764160,"available":96558116864,"internally_reserved":0,"allocated":1236992,"data_stored":534257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574943,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59340,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312500,"statfs":{"total":96636764160,"available":96576000000,"internally_reserved":0,"allocated":1830912,"data_stored":1124625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:01:37.708 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:01:37.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.857+0000 7efcb2909640 1 Processor -- start 2024-04-25T09:01:37.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.857+0000 7efcb2909640 1 -- start start 2024-04-25T09:01:37.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.858+0000 7efcb2909640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7efcac15ef50 0x7efcac0d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:37.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.858+0000 7efcb2909640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efcac0d0920 0x7efcac160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:37.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.858+0000 7efcb2909640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcac0cee80 con 0x7efcac15dce0 2024-04-25T09:01:37.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.858+0000 7efcb2909640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcac0cec60 con 0x7efcac15ef50 2024-04-25T09:01:37.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.858+0000 7efcb2909640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcac0d0d70 con 0x7efcac0d0920 2024-04-25T09:01:37.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.858+0000 7efcb0ea6640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efcac0d0920 0x7efcac160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:37.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.858+0000 7efcab7fe640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7efcac15ef50 0x7efcac0d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:37.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.858+0000 7efcabfff640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7efcac15dce0 0x7efcac0ce550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:55258/0 (socket says 172.21.15.43:55258) 2024-04-25T09:01:37.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.858+0000 7efcabfff640 1 -- 172.21.15.43:0/1780834446 learned_addr learned my addr 172.21.15.43:0/1780834446 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:37.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.858+0000 7efcb0ea6640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efcac0d0920 0x7efcac160980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:50696/0 (socket says 172.21.15.43:50696) 2024-04-25T09:01:37.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.858+0000 7efcab7fe640 1 -- 172.21.15.43:0/1780834446 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efcac0d0920 msgr2=0x7efcac160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:37.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.858+0000 7efcab7fe640 1 --2- 172.21.15.43:0/1780834446 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efcac0d0920 0x7efcac160980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:37.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.858+0000 7efcab7fe640 1 -- 172.21.15.43:0/1780834446 >> v1:172.21.15.193:6789/0 conn(0x7efcac15dce0 legacy=0x7efcac0ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:37.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.858+0000 7efcab7fe640 1 -- 172.21.15.43:0/1780834446 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcac0d2920 con 0x7efcac15ef50 2024-04-25T09:01:37.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.858+0000 7efcb0ea6640 1 --2- 172.21.15.43:0/1780834446 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efcac0d0920 0x7efcac160980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:01:37.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.859+0000 7efcab7fe640 1 --2- 172.21.15.43:0/1780834446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efcac15ef50 0x7efcac0d03e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7efc9c0097d0 tx=0x7efc9c02a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea4a4f203ffc01b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:37.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.859+0000 7efcaaffd640 1 -- 172.21.15.43:0/1780834446 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9c037070 con 0x7efcac15ef50 2024-04-25T09:01:37.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.859+0000 7efcaaffd640 1 -- 172.21.15.43:0/1780834446 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc9c02ae40 con 0x7efcac15ef50 2024-04-25T09:01:37.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.859+0000 7efcaaffd640 1 -- 172.21.15.43:0/1780834446 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc9c032660 con 0x7efcac15ef50 2024-04-25T09:01:37.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.860+0000 7efcb2909640 1 -- 172.21.15.43:0/1780834446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efcac15ef50 msgr2=0x7efcac0d03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:37.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.860+0000 7efcb2909640 1 --2- 172.21.15.43:0/1780834446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efcac15ef50 0x7efcac0d03e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7efc9c0097d0 tx=0x7efc9c02a130 comp rx=0 tx=0).stop 2024-04-25T09:01:37.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.860+0000 7efcb2909640 1 -- 172.21.15.43:0/1780834446 shutdown_connections 2024-04-25T09:01:37.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.860+0000 7efcb2909640 1 --2- 172.21.15.43:0/1780834446 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efcac0d0920 0x7efcac160980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:37.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.860+0000 7efcb2909640 1 --2- 172.21.15.43:0/1780834446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efcac15ef50 0x7efcac0d03e0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:37.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.860+0000 7efcb2909640 1 -- 172.21.15.43:0/1780834446 >> 172.21.15.43:0/1780834446 conn(0x7efcac09cd00 msgr2=0x7efcac0d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:37.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.860+0000 7efcb2909640 1 -- 172.21.15.43:0/1780834446 shutdown_connections 2024-04-25T09:01:37.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.860+0000 7efcb2909640 1 -- 172.21.15.43:0/1780834446 wait complete. 2024-04-25T09:01:37.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.861+0000 7efcb2909640 1 Processor -- start 2024-04-25T09:01:37.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.861+0000 7efcb2909640 1 -- start start 2024-04-25T09:01:37.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.862+0000 7efcb2909640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcac0d0920 0x7efcac149f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:37.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.862+0000 7efcb2909640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efcac15dce0 0x7efcac0b6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:37.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.862+0000 7efcb2909640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efcac15ef50 0x7efcac0c9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:37.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.862+0000 7efcb2909640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcac0cf720 con 0x7efcac0d0920 2024-04-25T09:01:37.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.862+0000 7efcb2909640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcac0cf440 con 0x7efcac15dce0 2024-04-25T09:01:37.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.862+0000 7efcb2909640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcac15f6b0 con 0x7efcac15ef50 2024-04-25T09:01:37.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.862+0000 7efcabfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcac0d0920 0x7efcac149f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:37.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.862+0000 7efcabfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcac0d0920 0x7efcac149f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49082/0 (socket says 172.21.15.43:49082) 2024-04-25T09:01:37.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.862+0000 7efcabfff640 1 -- 172.21.15.43:0/2172745946 learned_addr learned my addr 172.21.15.43:0/2172745946 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:37.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.862+0000 7efcab7fe640 1 --2- 172.21.15.43:0/2172745946 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efcac15dce0 0x7efcac0b6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:37.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.862+0000 7efcb0ea6640 1 --2- 172.21.15.43:0/2172745946 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efcac15ef50 0x7efcac0c9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:37.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.863+0000 7efcabfff640 1 -- 172.21.15.43:0/2172745946 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efcac15ef50 msgr2=0x7efcac0c9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:37.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.863+0000 7efcabfff640 1 --2- 172.21.15.43:0/2172745946 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efcac15ef50 0x7efcac0c9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:37.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.863+0000 7efcabfff640 1 -- 172.21.15.43:0/2172745946 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efcac15dce0 msgr2=0x7efcac0b6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:37.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.863+0000 7efcabfff640 1 --2- 172.21.15.43:0/2172745946 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efcac15dce0 0x7efcac0b6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:37.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.863+0000 7efcabfff640 1 -- 172.21.15.43:0/2172745946 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcac0b7770 con 0x7efcac0d0920 2024-04-25T09:01:37.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.863+0000 7efcabfff640 1 --2- 172.21.15.43:0/2172745946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcac0d0920 0x7efcac149f60 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7efca000c400 tx=0x7efca000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:37.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.863+0000 7efca8ff9640 1 -- 172.21.15.43:0/2172745946 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca0017070 con 0x7efcac0d0920 2024-04-25T09:01:37.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.863+0000 7efca8ff9640 1 -- 172.21.15.43:0/2172745946 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efca0005ce0 con 0x7efcac0d0920 2024-04-25T09:01:37.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.863+0000 7efca8ff9640 1 -- 172.21.15.43:0/2172745946 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca00072f0 con 0x7efcac0d0920 2024-04-25T09:01:37.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.863+0000 7efcb2909640 1 -- 172.21.15.43:0/2172745946 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcac0b7a50 con 0x7efcac0d0920 2024-04-25T09:01:37.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.863+0000 7efcb2909640 1 -- 172.21.15.43:0/2172745946 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcac0b8010 con 0x7efcac0d0920 2024-04-25T09:01:37.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.864+0000 7efcb2909640 1 -- 172.21.15.43:0/2172745946 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcac0680f0 con 0x7efcac0d0920 2024-04-25T09:01:37.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.864+0000 7efca8ff9640 1 -- 172.21.15.43:0/2172745946 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efca001a070 con 0x7efcac0d0920 2024-04-25T09:01:37.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.865+0000 7efca8ff9640 1 --2- 172.21.15.43:0/2172745946 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7efc9806b4d0 0x7efc9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:37.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.865+0000 7efca8ff9640 1 -- 172.21.15.43:0/2172745946 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efca00894d0 con 0x7efcac0d0920 2024-04-25T09:01:37.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.865+0000 7efcab7fe640 1 --2- 172.21.15.43:0/2172745946 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7efc9806b4d0 0x7efc9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:37.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.866+0000 7efcab7fe640 1 --2- 172.21.15.43:0/2172745946 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7efc9806b4d0 0x7efc9806d990 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7efc9c034000 tx=0x7efc9c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:37.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:37.869+0000 7efca8ff9640 1 -- 172.21.15.43:0/2172745946 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efca0057d00 con 0x7efcac0d0920 2024-04-25T09:01:38.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.046+0000 7efcb2909640 1 -- 172.21.15.43:0/2172745946 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efcac074ef0 con 0x7efc9806b4d0 2024-04-25T09:01:38.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.052+0000 7efca8ff9640 1 -- 172.21.15.43:0/2172745946 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247013 (secure 0 0 0) 0x7efcac074ef0 con 0x7efc9806b4d0 2024-04-25T09:01:38.043 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:01:38.046 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-25T09:01:38.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.055+0000 7efcb2909640 1 -- 172.21.15.43:0/2172745946 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7efc9806b4d0 msgr2=0x7efc9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:38.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.055+0000 7efcb2909640 1 --2- 172.21.15.43:0/2172745946 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7efc9806b4d0 0x7efc9806d990 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7efc9c034000 tx=0x7efc9c02e000 comp rx=0 tx=0).stop 2024-04-25T09:01:38.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.055+0000 7efcb2909640 1 -- 172.21.15.43:0/2172745946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcac0d0920 msgr2=0x7efcac149f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:38.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.055+0000 7efcb2909640 1 --2- 172.21.15.43:0/2172745946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcac0d0920 0x7efcac149f60 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7efca000c400 tx=0x7efca000c800 comp rx=0 tx=0).stop 2024-04-25T09:01:38.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.055+0000 7efcb2909640 1 -- 172.21.15.43:0/2172745946 shutdown_connections 2024-04-25T09:01:38.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.055+0000 7efcb2909640 1 --2- 172.21.15.43:0/2172745946 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efcac15ef50 0x7efcac0c9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:38.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.055+0000 7efcb2909640 1 --2- 172.21.15.43:0/2172745946 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7efc9806b4d0 0x7efc9806d990 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:38.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.055+0000 7efcb2909640 1 --2- 172.21.15.43:0/2172745946 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efcac15dce0 0x7efcac0b6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:38.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.055+0000 7efcb2909640 1 --2- 172.21.15.43:0/2172745946 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efcac0d0920 0x7efcac149f60 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:38.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.055+0000 7efcb2909640 1 -- 172.21.15.43:0/2172745946 >> 172.21.15.43:0/2172745946 conn(0x7efcac09cd00 msgr2=0x7efcac15e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:38.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.055+0000 7efcb2909640 1 -- 172.21.15.43:0/2172745946 shutdown_connections 2024-04-25T09:01:38.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.055+0000 7efcb2909640 1 -- 172.21.15.43:0/2172745946 wait complete. 2024-04-25T09:01:38.055 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":325,"stamp":"2024-04-25T09:01:37.797091+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":24,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":514276,"kb_used_data":10800,"kb_used_omap":12,"kb_used_meta":503347,"kb_avail":754460444,"statfs":{"total":773094113280,"available":772567494656,"internally_reserved":0,"allocated":11059200,"data_stored":5442242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":515427889},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.452898"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.355534+0000","last_change":"2024-04-25T09:01:33.355534+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.355534+0000","last_undegraded":"2024-04-25T09:01:33.355534+0000","last_fullsized":"2024-04-25T09:01:33.355534+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.359660+0000","last_change":"2024-04-25T09:01:33.359660+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.359660+0000","last_undegraded":"2024-04-25T09:01:33.359660+0000","last_fullsized":"2024-04-25T09:01:33.359660+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.355300+0000","last_change":"2024-04-25T09:01:33.355300+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.355300+0000","last_undegraded":"2024-04-25T09:01:33.355300+0000","last_fullsized":"2024-04-25T09:01:33.355300+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356488+0000","last_change":"2024-04-25T09:01:33.356488+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356488+0000","last_undegraded":"2024-04-25T09:01:33.356488+0000","last_fullsized":"2024-04-25T09:01:33.356488+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.358970+0000","last_change":"2024-04-25T09:01:33.358970+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.358970+0000","last_undegraded":"2024-04-25T09:01:33.358970+0000","last_fullsized":"2024-04-25T09:01:33.358970+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.358980+0000","last_change":"2024-04-25T09:01:33.358980+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.358980+0000","last_undegraded":"2024-04-25T09:01:33.358980+0000","last_fullsized":"2024-04-25T09:01:33.358980+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356706+0000","last_change":"2024-04-25T09:01:33.356706+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356706+0000","last_undegraded":"2024-04-25T09:01:33.356706+0000","last_fullsized":"2024-04-25T09:01:33.356706+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.357495+0000","last_change":"2024-04-25T09:01:33.357495+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.357495+0000","last_undegraded":"2024-04-25T09:01:33.357495+0000","last_fullsized":"2024-04-25T09:01:33.357495+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.359846+0000","last_change":"2024-04-25T09:01:33.359846+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.359846+0000","last_undegraded":"2024-04-25T09:01:33.359846+0000","last_fullsized":"2024-04-25T09:01:33.359846+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.358143+0000","last_change":"2024-04-25T09:01:33.358143+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.358143+0000","last_undegraded":"2024-04-25T09:01:33.358143+0000","last_fullsized":"2024-04-25T09:01:33.358143+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.358698+0000","last_change":"2024-04-25T09:01:33.358698+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.358698+0000","last_undegraded":"2024-04-25T09:01:33.358698+0000","last_fullsized":"2024-04-25T09:01:33.358698+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356492+0000","last_change":"2024-04-25T09:01:33.356492+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356492+0000","last_undegraded":"2024-04-25T09:01:33.356492+0000","last_fullsized":"2024-04-25T09:01:33.356492+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.355925+0000","last_change":"2024-04-25T09:01:33.355925+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.355925+0000","last_undegraded":"2024-04-25T09:01:33.355925+0000","last_fullsized":"2024-04-25T09:01:33.355925+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.362266+0000","last_change":"2024-04-25T09:01:33.362266+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.362266+0000","last_undegraded":"2024-04-25T09:01:33.362266+0000","last_fullsized":"2024-04-25T09:01:33.362266+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.358146+0000","last_change":"2024-04-25T09:01:33.358146+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.358146+0000","last_undegraded":"2024-04-25T09:01:33.358146+0000","last_fullsized":"2024-04-25T09:01:33.358146+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356808+0000","last_change":"2024-04-25T09:01:33.356808+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356808+0000","last_undegraded":"2024-04-25T09:01:33.356808+0000","last_fullsized":"2024-04-25T09:01:33.356808+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.358063+0000","last_change":"2024-04-25T09:01:33.358063+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.358063+0000","last_undegraded":"2024-04-25T09:01:33.358063+0000","last_fullsized":"2024-04-25T09:01:33.358063+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.358540+0000","last_change":"2024-04-25T09:01:33.358540+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.358540+0000","last_undegraded":"2024-04-25T09:01:33.358540+0000","last_fullsized":"2024-04-25T09:01:33.358540+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356094+0000","last_change":"2024-04-25T09:01:33.356094+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356094+0000","last_undegraded":"2024-04-25T09:01:33.356094+0000","last_fullsized":"2024-04-25T09:01:33.356094+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.355041+0000","last_change":"2024-04-25T09:01:33.355041+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.355041+0000","last_undegraded":"2024-04-25T09:01:33.355041+0000","last_fullsized":"2024-04-25T09:01:33.355041+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.354826+0000","last_change":"2024-04-25T09:01:33.354826+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.354826+0000","last_undegraded":"2024-04-25T09:01:33.354826+0000","last_fullsized":"2024-04-25T09:01:33.354826+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.358378+0000","last_change":"2024-04-25T09:01:33.358378+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.358378+0000","last_undegraded":"2024-04-25T09:01:33.358378+0000","last_fullsized":"2024-04-25T09:01:33.358378+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356711+0000","last_change":"2024-04-25T09:01:33.356711+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356711+0000","last_undegraded":"2024-04-25T09:01:33.356711+0000","last_fullsized":"2024-04-25T09:01:33.356711+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356347+0000","last_change":"2024-04-25T09:01:33.356347+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356347+0000","last_undegraded":"2024-04-25T09:01:33.356347+0000","last_fullsized":"2024-04-25T09:01:33.356347+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.362666+0000","last_change":"2024-04-25T09:01:33.362666+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.362666+0000","last_undegraded":"2024-04-25T09:01:33.362666+0000","last_fullsized":"2024-04-25T09:01:33.362666+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.359240+0000","last_change":"2024-04-25T09:01:33.359240+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.359240+0000","last_undegraded":"2024-04-25T09:01:33.359240+0000","last_fullsized":"2024-04-25T09:01:33.359240+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.358208+0000","last_change":"2024-04-25T09:01:33.358208+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.358208+0000","last_undegraded":"2024-04-25T09:01:33.358208+0000","last_fullsized":"2024-04-25T09:01:33.358208+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.363030+0000","last_change":"2024-04-25T09:01:33.363030+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.363030+0000","last_undegraded":"2024-04-25T09:01:33.363030+0000","last_fullsized":"2024-04-25T09:01:33.363030+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.357869+0000","last_change":"2024-04-25T09:01:33.357869+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.357869+0000","last_undegraded":"2024-04-25T09:01:33.357869+0000","last_fullsized":"2024-04-25T09:01:33.357869+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.355005+0000","last_change":"2024-04-25T09:01:33.355005+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.355005+0000","last_undegraded":"2024-04-25T09:01:33.355005+0000","last_fullsized":"2024-04-25T09:01:33.355005+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.357607+0000","last_change":"2024-04-25T09:01:33.357607+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.357607+0000","last_undegraded":"2024-04-25T09:01:33.357607+0000","last_fullsized":"2024-04-25T09:01:33.357607+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.354229+0000","last_change":"2024-04-25T09:01:33.354229+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.354229+0000","last_undegraded":"2024-04-25T09:01:33.354229+0000","last_fullsized":"2024-04-25T09:01:33.354229+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.353890+0000","last_change":"2024-04-25T09:01:33.353890+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.353890+0000","last_undegraded":"2024-04-25T09:01:33.353890+0000","last_fullsized":"2024-04-25T09:01:33.353890+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.367304+0000","last_change":"2024-04-25T09:01:33.365666+0000","last_active":"2024-04-25T09:01:33.367304+0000","last_peered":"2024-04-25T09:01:33.367304+0000","last_clean":"2024-04-25T09:01:33.367304+0000","last_became_active":"2024-04-25T09:01:33.365341+0000","last_became_peered":"2024-04-25T09:01:33.365341+0000","last_unstale":"2024-04-25T09:01:33.367304+0000","last_undegraded":"2024-04-25T09:01:33.367304+0000","last_fullsized":"2024-04-25T09:01:33.367304+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:41.915410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356285+0000","last_change":"2024-04-25T09:01:33.356285+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356285+0000","last_undegraded":"2024-04-25T09:01:33.356285+0000","last_fullsized":"2024-04-25T09:01:33.356285+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.368360+0000","last_change":"2024-04-25T09:01:33.366268+0000","last_active":"2024-04-25T09:01:33.368360+0000","last_peered":"2024-04-25T09:01:33.368360+0000","last_clean":"2024-04-25T09:01:33.368360+0000","last_became_active":"2024-04-25T09:01:33.365952+0000","last_became_peered":"2024-04-25T09:01:33.365952+0000","last_unstale":"2024-04-25T09:01:33.368360+0000","last_undegraded":"2024-04-25T09:01:33.368360+0000","last_fullsized":"2024-04-25T09:01:33.368360+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:29.149495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.354216+0000","last_change":"2024-04-25T09:01:33.354216+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.354216+0000","last_undegraded":"2024-04-25T09:01:33.354216+0000","last_fullsized":"2024-04-25T09:01:33.354216+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.371487+0000","last_change":"2024-04-25T09:01:33.370140+0000","last_active":"2024-04-25T09:01:33.371487+0000","last_peered":"2024-04-25T09:01:33.371487+0000","last_clean":"2024-04-25T09:01:33.371487+0000","last_became_active":"2024-04-25T09:01:33.369677+0000","last_became_peered":"2024-04-25T09:01:33.369677+0000","last_unstale":"2024-04-25T09:01:33.371487+0000","last_undegraded":"2024-04-25T09:01:33.371487+0000","last_fullsized":"2024-04-25T09:01:33.371487+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:15.517604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.357436+0000","last_change":"2024-04-25T09:01:33.357436+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.357436+0000","last_undegraded":"2024-04-25T09:01:33.357436+0000","last_fullsized":"2024-04-25T09:01:33.357436+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.369409+0000","last_change":"2024-04-25T09:01:33.367516+0000","last_active":"2024-04-25T09:01:33.369409+0000","last_peered":"2024-04-25T09:01:33.369409+0000","last_clean":"2024-04-25T09:01:33.369409+0000","last_became_active":"2024-04-25T09:01:33.367171+0000","last_became_peered":"2024-04-25T09:01:33.367171+0000","last_unstale":"2024-04-25T09:01:33.369409+0000","last_undegraded":"2024-04-25T09:01:33.369409+0000","last_fullsized":"2024-04-25T09:01:33.369409+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:43.586923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356196+0000","last_change":"2024-04-25T09:01:33.356196+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356196+0000","last_undegraded":"2024-04-25T09:01:33.356196+0000","last_fullsized":"2024-04-25T09:01:33.356196+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.363802+0000","last_change":"2024-04-25T09:01:33.360517+0000","last_active":"2024-04-25T09:01:33.363802+0000","last_peered":"2024-04-25T09:01:33.363802+0000","last_clean":"2024-04-25T09:01:33.363802+0000","last_became_active":"2024-04-25T09:01:33.360139+0000","last_became_peered":"2024-04-25T09:01:33.360139+0000","last_unstale":"2024-04-25T09:01:33.363802+0000","last_undegraded":"2024-04-25T09:01:33.363802+0000","last_fullsized":"2024-04-25T09:01:33.363802+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:50.515806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356694+0000","last_change":"2024-04-25T09:01:33.356694+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356694+0000","last_undegraded":"2024-04-25T09:01:33.356694+0000","last_fullsized":"2024-04-25T09:01:33.356694+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.372295+0000","last_change":"2024-04-25T09:01:33.371930+0000","last_active":"2024-04-25T09:01:33.372295+0000","last_peered":"2024-04-25T09:01:33.372295+0000","last_clean":"2024-04-25T09:01:33.372295+0000","last_became_active":"2024-04-25T09:01:33.371395+0000","last_became_peered":"2024-04-25T09:01:33.371395+0000","last_unstale":"2024-04-25T09:01:33.372295+0000","last_undegraded":"2024-04-25T09:01:33.372295+0000","last_fullsized":"2024-04-25T09:01:33.372295+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:44.824026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.357302+0000","last_change":"2024-04-25T09:01:33.357302+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.357302+0000","last_undegraded":"2024-04-25T09:01:33.357302+0000","last_fullsized":"2024-04-25T09:01:33.357302+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.367580+0000","last_change":"2024-04-25T09:01:33.366526+0000","last_active":"2024-04-25T09:01:33.367580+0000","last_peered":"2024-04-25T09:01:33.367580+0000","last_clean":"2024-04-25T09:01:33.367580+0000","last_became_active":"2024-04-25T09:01:33.366246+0000","last_became_peered":"2024-04-25T09:01:33.366246+0000","last_unstale":"2024-04-25T09:01:33.367580+0000","last_undegraded":"2024-04-25T09:01:33.367580+0000","last_fullsized":"2024-04-25T09:01:33.367580+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:26.679025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.355141+0000","last_change":"2024-04-25T09:01:33.355141+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.355141+0000","last_undegraded":"2024-04-25T09:01:33.355141+0000","last_fullsized":"2024-04-25T09:01:33.355141+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.369410+0000","last_change":"2024-04-25T09:01:33.367167+0000","last_active":"2024-04-25T09:01:33.369410+0000","last_peered":"2024-04-25T09:01:33.369410+0000","last_clean":"2024-04-25T09:01:33.369410+0000","last_became_active":"2024-04-25T09:01:33.365541+0000","last_became_peered":"2024-04-25T09:01:33.365541+0000","last_unstale":"2024-04-25T09:01:33.369410+0000","last_undegraded":"2024-04-25T09:01:33.369410+0000","last_fullsized":"2024-04-25T09:01:33.369410+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:22.335074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.357332+0000","last_change":"2024-04-25T09:01:33.357332+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.357332+0000","last_undegraded":"2024-04-25T09:01:33.357332+0000","last_fullsized":"2024-04-25T09:01:33.357332+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.363381+0000","last_change":"2024-04-25T09:01:33.361825+0000","last_active":"2024-04-25T09:01:33.363381+0000","last_peered":"2024-04-25T09:01:33.363381+0000","last_clean":"2024-04-25T09:01:33.363381+0000","last_became_active":"2024-04-25T09:01:33.361544+0000","last_became_peered":"2024-04-25T09:01:33.361544+0000","last_unstale":"2024-04-25T09:01:33.363381+0000","last_undegraded":"2024-04-25T09:01:33.363381+0000","last_fullsized":"2024-04-25T09:01:33.363381+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:19.818176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.354331+0000","last_change":"2024-04-25T09:01:33.354331+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.354331+0000","last_undegraded":"2024-04-25T09:01:33.354331+0000","last_fullsized":"2024-04-25T09:01:33.354331+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.369815+0000","last_change":"2024-04-25T09:01:33.365387+0000","last_active":"2024-04-25T09:01:33.369815+0000","last_peered":"2024-04-25T09:01:33.369815+0000","last_clean":"2024-04-25T09:01:33.369815+0000","last_became_active":"2024-04-25T09:01:33.365088+0000","last_became_peered":"2024-04-25T09:01:33.365088+0000","last_unstale":"2024-04-25T09:01:33.369815+0000","last_undegraded":"2024-04-25T09:01:33.369815+0000","last_fullsized":"2024-04-25T09:01:33.369815+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:34.749007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.362314+0000","last_change":"2024-04-25T09:01:33.362314+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.362314+0000","last_undegraded":"2024-04-25T09:01:33.362314+0000","last_fullsized":"2024-04-25T09:01:33.362314+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.368520+0000","last_change":"2024-04-25T09:01:33.367517+0000","last_active":"2024-04-25T09:01:33.368520+0000","last_peered":"2024-04-25T09:01:33.368520+0000","last_clean":"2024-04-25T09:01:33.368520+0000","last_became_active":"2024-04-25T09:01:33.367216+0000","last_became_peered":"2024-04-25T09:01:33.367216+0000","last_unstale":"2024-04-25T09:01:33.368520+0000","last_undegraded":"2024-04-25T09:01:33.368520+0000","last_fullsized":"2024-04-25T09:01:33.368520+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:42.045982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.357735+0000","last_change":"2024-04-25T09:01:33.357735+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.357735+0000","last_undegraded":"2024-04-25T09:01:33.357735+0000","last_fullsized":"2024-04-25T09:01:33.357735+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.367757+0000","last_change":"2024-04-25T09:01:33.366291+0000","last_active":"2024-04-25T09:01:33.367757+0000","last_peered":"2024-04-25T09:01:33.367757+0000","last_clean":"2024-04-25T09:01:33.367757+0000","last_became_active":"2024-04-25T09:01:33.365670+0000","last_became_peered":"2024-04-25T09:01:33.365670+0000","last_unstale":"2024-04-25T09:01:33.367757+0000","last_undegraded":"2024-04-25T09:01:33.367757+0000","last_fullsized":"2024-04-25T09:01:33.367757+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:35.172322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.355949+0000","last_change":"2024-04-25T09:01:33.355949+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.355949+0000","last_undegraded":"2024-04-25T09:01:33.355949+0000","last_fullsized":"2024-04-25T09:01:33.355949+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.368724+0000","last_change":"2024-04-25T09:01:33.363784+0000","last_active":"2024-04-25T09:01:33.368724+0000","last_peered":"2024-04-25T09:01:33.368724+0000","last_clean":"2024-04-25T09:01:33.368724+0000","last_became_active":"2024-04-25T09:01:33.363279+0000","last_became_peered":"2024-04-25T09:01:33.363279+0000","last_unstale":"2024-04-25T09:01:33.368724+0000","last_undegraded":"2024-04-25T09:01:33.368724+0000","last_fullsized":"2024-04-25T09:01:33.368724+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:15.504041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.354732+0000","last_change":"2024-04-25T09:01:33.354732+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.354732+0000","last_undegraded":"2024-04-25T09:01:33.354732+0000","last_fullsized":"2024-04-25T09:01:33.354732+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.368515+0000","last_change":"2024-04-25T09:01:33.365837+0000","last_active":"2024-04-25T09:01:33.368515+0000","last_peered":"2024-04-25T09:01:33.368515+0000","last_clean":"2024-04-25T09:01:33.368515+0000","last_became_active":"2024-04-25T09:01:33.362985+0000","last_became_peered":"2024-04-25T09:01:33.362985+0000","last_unstale":"2024-04-25T09:01:33.368515+0000","last_undegraded":"2024-04-25T09:01:33.368515+0000","last_fullsized":"2024-04-25T09:01:33.368515+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:02.701373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.359064+0000","last_change":"2024-04-25T09:01:33.359064+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.359064+0000","last_undegraded":"2024-04-25T09:01:33.359064+0000","last_fullsized":"2024-04-25T09:01:33.359064+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.367829+0000","last_change":"2024-04-25T09:01:33.367032+0000","last_active":"2024-04-25T09:01:33.367829+0000","last_peered":"2024-04-25T09:01:33.367829+0000","last_clean":"2024-04-25T09:01:33.367829+0000","last_became_active":"2024-04-25T09:01:33.366662+0000","last_became_peered":"2024-04-25T09:01:33.366662+0000","last_unstale":"2024-04-25T09:01:33.367829+0000","last_undegraded":"2024-04-25T09:01:33.367829+0000","last_fullsized":"2024-04-25T09:01:33.367829+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:13.219195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.352882+0000","last_change":"2024-04-25T09:01:33.352882+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.352882+0000","last_undegraded":"2024-04-25T09:01:33.352882+0000","last_fullsized":"2024-04-25T09:01:33.352882+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.369386+0000","last_change":"2024-04-25T09:01:33.366823+0000","last_active":"2024-04-25T09:01:33.369386+0000","last_peered":"2024-04-25T09:01:33.369386+0000","last_clean":"2024-04-25T09:01:33.369386+0000","last_became_active":"2024-04-25T09:01:33.366050+0000","last_became_peered":"2024-04-25T09:01:33.366050+0000","last_unstale":"2024-04-25T09:01:33.369386+0000","last_undegraded":"2024-04-25T09:01:33.369386+0000","last_fullsized":"2024-04-25T09:01:33.369386+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:49.016771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.362269+0000","last_change":"2024-04-25T09:01:33.362269+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.362269+0000","last_undegraded":"2024-04-25T09:01:33.362269+0000","last_fullsized":"2024-04-25T09:01:33.362269+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.369665+0000","last_change":"2024-04-25T09:01:33.369232+0000","last_active":"2024-04-25T09:01:33.369665+0000","last_peered":"2024-04-25T09:01:33.369665+0000","last_clean":"2024-04-25T09:01:33.369665+0000","last_became_active":"2024-04-25T09:01:33.368905+0000","last_became_peered":"2024-04-25T09:01:33.368905+0000","last_unstale":"2024-04-25T09:01:33.369665+0000","last_undegraded":"2024-04-25T09:01:33.369665+0000","last_fullsized":"2024-04-25T09:01:33.369665+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:41.915410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.355227+0000","last_change":"2024-04-25T09:01:33.355227+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.355227+0000","last_undegraded":"2024-04-25T09:01:33.355227+0000","last_fullsized":"2024-04-25T09:01:33.355227+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.364958+0000","last_change":"2024-04-25T09:01:33.363919+0000","last_active":"2024-04-25T09:01:33.364958+0000","last_peered":"2024-04-25T09:01:33.364958+0000","last_clean":"2024-04-25T09:01:33.364958+0000","last_became_active":"2024-04-25T09:01:33.363587+0000","last_became_peered":"2024-04-25T09:01:33.363587+0000","last_unstale":"2024-04-25T09:01:33.364958+0000","last_undegraded":"2024-04-25T09:01:33.364958+0000","last_fullsized":"2024-04-25T09:01:33.364958+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:39.790922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.355533+0000","last_change":"2024-04-25T09:01:33.355533+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.355533+0000","last_undegraded":"2024-04-25T09:01:33.355533+0000","last_fullsized":"2024-04-25T09:01:33.355533+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.367244+0000","last_change":"2024-04-25T09:01:33.364976+0000","last_active":"2024-04-25T09:01:33.367244+0000","last_peered":"2024-04-25T09:01:33.367244+0000","last_clean":"2024-04-25T09:01:33.367244+0000","last_became_active":"2024-04-25T09:01:33.363617+0000","last_became_peered":"2024-04-25T09:01:33.363617+0000","last_unstale":"2024-04-25T09:01:33.367244+0000","last_undegraded":"2024-04-25T09:01:33.367244+0000","last_fullsized":"2024-04-25T09:01:33.367244+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:13.219195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.353243+0000","last_change":"2024-04-25T09:01:33.353243+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.353243+0000","last_undegraded":"2024-04-25T09:01:33.353243+0000","last_fullsized":"2024-04-25T09:01:33.353243+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.371512+0000","last_change":"2024-04-25T09:01:33.369475+0000","last_active":"2024-04-25T09:01:33.371512+0000","last_peered":"2024-04-25T09:01:33.371512+0000","last_clean":"2024-04-25T09:01:33.371512+0000","last_became_active":"2024-04-25T09:01:33.368210+0000","last_became_peered":"2024-04-25T09:01:33.368210+0000","last_unstale":"2024-04-25T09:01:33.371512+0000","last_undegraded":"2024-04-25T09:01:33.371512+0000","last_fullsized":"2024-04-25T09:01:33.371512+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:26.679025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.354197+0000","last_change":"2024-04-25T09:01:33.354197+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.354197+0000","last_undegraded":"2024-04-25T09:01:33.354197+0000","last_fullsized":"2024-04-25T09:01:33.354197+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.364947+0000","last_change":"2024-04-25T09:01:33.363025+0000","last_active":"2024-04-25T09:01:33.364947+0000","last_peered":"2024-04-25T09:01:33.364947+0000","last_clean":"2024-04-25T09:01:33.364947+0000","last_became_active":"2024-04-25T09:01:33.362724+0000","last_became_peered":"2024-04-25T09:01:33.362724+0000","last_unstale":"2024-04-25T09:01:33.364947+0000","last_undegraded":"2024-04-25T09:01:33.364947+0000","last_fullsized":"2024-04-25T09:01:33.364947+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:41.464204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.353721+0000","last_change":"2024-04-25T09:01:33.353721+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.353721+0000","last_undegraded":"2024-04-25T09:01:33.353721+0000","last_fullsized":"2024-04-25T09:01:33.353721+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.366031+0000","last_change":"2024-04-25T09:01:33.365366+0000","last_active":"2024-04-25T09:01:33.366031+0000","last_peered":"2024-04-25T09:01:33.366031+0000","last_clean":"2024-04-25T09:01:33.366031+0000","last_became_active":"2024-04-25T09:01:33.365090+0000","last_became_peered":"2024-04-25T09:01:33.365090+0000","last_unstale":"2024-04-25T09:01:33.366031+0000","last_undegraded":"2024-04-25T09:01:33.366031+0000","last_fullsized":"2024-04-25T09:01:33.366031+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:35.172322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.361446+0000","last_change":"2024-04-25T09:01:33.361446+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.361446+0000","last_undegraded":"2024-04-25T09:01:33.361446+0000","last_fullsized":"2024-04-25T09:01:33.361446+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.371551+0000","last_change":"2024-04-25T09:01:33.370545+0000","last_active":"2024-04-25T09:01:33.371551+0000","last_peered":"2024-04-25T09:01:33.371551+0000","last_clean":"2024-04-25T09:01:33.371551+0000","last_became_active":"2024-04-25T09:01:33.370255+0000","last_became_peered":"2024-04-25T09:01:33.370255+0000","last_unstale":"2024-04-25T09:01:33.371551+0000","last_undegraded":"2024-04-25T09:01:33.371551+0000","last_fullsized":"2024-04-25T09:01:33.371551+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:10.849550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356865+0000","last_change":"2024-04-25T09:01:33.356865+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356865+0000","last_undegraded":"2024-04-25T09:01:33.356865+0000","last_fullsized":"2024-04-25T09:01:33.356865+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.365141+0000","last_change":"2024-04-25T09:01:33.364409+0000","last_active":"2024-04-25T09:01:33.365141+0000","last_peered":"2024-04-25T09:01:33.365141+0000","last_clean":"2024-04-25T09:01:33.365141+0000","last_became_active":"2024-04-25T09:01:33.364089+0000","last_became_peered":"2024-04-25T09:01:33.364089+0000","last_unstale":"2024-04-25T09:01:33.365141+0000","last_undegraded":"2024-04-25T09:01:33.365141+0000","last_fullsized":"2024-04-25T09:01:33.365141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:58.745035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.355038+0000","last_change":"2024-04-25T09:01:33.355038+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.355038+0000","last_undegraded":"2024-04-25T09:01:33.355038+0000","last_fullsized":"2024-04-25T09:01:33.355038+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.367852+0000","last_change":"2024-04-25T09:01:33.366607+0000","last_active":"2024-04-25T09:01:33.367852+0000","last_peered":"2024-04-25T09:01:33.367852+0000","last_clean":"2024-04-25T09:01:33.367852+0000","last_became_active":"2024-04-25T09:01:33.366048+0000","last_became_peered":"2024-04-25T09:01:33.366048+0000","last_unstale":"2024-04-25T09:01:33.367852+0000","last_undegraded":"2024-04-25T09:01:33.367852+0000","last_fullsized":"2024-04-25T09:01:33.367852+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:07.729885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.353719+0000","last_change":"2024-04-25T09:01:33.353719+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.353719+0000","last_undegraded":"2024-04-25T09:01:33.353719+0000","last_fullsized":"2024-04-25T09:01:33.353719+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.369413+0000","last_change":"2024-04-25T09:01:33.368065+0000","last_active":"2024-04-25T09:01:33.369413+0000","last_peered":"2024-04-25T09:01:33.369413+0000","last_clean":"2024-04-25T09:01:33.369413+0000","last_became_active":"2024-04-25T09:01:33.367711+0000","last_became_peered":"2024-04-25T09:01:33.367711+0000","last_unstale":"2024-04-25T09:01:33.369413+0000","last_undegraded":"2024-04-25T09:01:33.369413+0000","last_fullsized":"2024-04-25T09:01:33.369413+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:39.790922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.358486+0000","last_change":"2024-04-25T09:01:33.358486+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.358486+0000","last_undegraded":"2024-04-25T09:01:33.358486+0000","last_fullsized":"2024-04-25T09:01:33.358486+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.366205+0000","last_change":"2024-04-25T09:01:33.365212+0000","last_active":"2024-04-25T09:01:33.366205+0000","last_peered":"2024-04-25T09:01:33.366205+0000","last_clean":"2024-04-25T09:01:33.366205+0000","last_became_active":"2024-04-25T09:01:33.364945+0000","last_became_peered":"2024-04-25T09:01:33.364945+0000","last_unstale":"2024-04-25T09:01:33.366205+0000","last_undegraded":"2024-04-25T09:01:33.366205+0000","last_fullsized":"2024-04-25T09:01:33.366205+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:15.517604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.352679+0000","last_change":"2024-04-25T09:01:33.352679+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.352679+0000","last_undegraded":"2024-04-25T09:01:33.352679+0000","last_fullsized":"2024-04-25T09:01:33.352679+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.370501+0000","last_change":"2024-04-25T09:01:33.367854+0000","last_active":"2024-04-25T09:01:33.370501+0000","last_peered":"2024-04-25T09:01:33.370501+0000","last_clean":"2024-04-25T09:01:33.370501+0000","last_became_active":"2024-04-25T09:01:33.366038+0000","last_became_peered":"2024-04-25T09:01:33.366038+0000","last_unstale":"2024-04-25T09:01:33.370501+0000","last_undegraded":"2024-04-25T09:01:33.370501+0000","last_fullsized":"2024-04-25T09:01:33.370501+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:55.938665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356269+0000","last_change":"2024-04-25T09:01:33.356269+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356269+0000","last_undegraded":"2024-04-25T09:01:33.356269+0000","last_fullsized":"2024-04-25T09:01:33.356269+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.370382+0000","last_change":"2024-04-25T09:01:33.369961+0000","last_active":"2024-04-25T09:01:33.370382+0000","last_peered":"2024-04-25T09:01:33.370382+0000","last_clean":"2024-04-25T09:01:33.370382+0000","last_became_active":"2024-04-25T09:01:33.369606+0000","last_became_peered":"2024-04-25T09:01:33.369606+0000","last_unstale":"2024-04-25T09:01:33.370382+0000","last_undegraded":"2024-04-25T09:01:33.370382+0000","last_fullsized":"2024-04-25T09:01:33.370382+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:10.849550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.353264+0000","last_change":"2024-04-25T09:01:33.353264+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.353264+0000","last_undegraded":"2024-04-25T09:01:33.353264+0000","last_fullsized":"2024-04-25T09:01:33.353264+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.369396+0000","last_change":"2024-04-25T09:01:33.364868+0000","last_active":"2024-04-25T09:01:33.369396+0000","last_peered":"2024-04-25T09:01:33.369396+0000","last_clean":"2024-04-25T09:01:33.369396+0000","last_became_active":"2024-04-25T09:01:33.364604+0000","last_became_peered":"2024-04-25T09:01:33.364604+0000","last_unstale":"2024-04-25T09:01:33.369396+0000","last_undegraded":"2024-04-25T09:01:33.369396+0000","last_fullsized":"2024-04-25T09:01:33.369396+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:29.149495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.361011+0000","last_change":"2024-04-25T09:01:33.361011+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.361011+0000","last_undegraded":"2024-04-25T09:01:33.361011+0000","last_fullsized":"2024-04-25T09:01:33.361011+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.368490+0000","last_change":"2024-04-25T09:01:33.364144+0000","last_active":"2024-04-25T09:01:33.368490+0000","last_peered":"2024-04-25T09:01:33.368490+0000","last_clean":"2024-04-25T09:01:33.368490+0000","last_became_active":"2024-04-25T09:01:33.363517+0000","last_became_peered":"2024-04-25T09:01:33.363517+0000","last_unstale":"2024-04-25T09:01:33.368490+0000","last_undegraded":"2024-04-25T09:01:33.368490+0000","last_fullsized":"2024-04-25T09:01:33.368490+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:04.891189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.352757+0000","last_change":"2024-04-25T09:01:33.352757+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.352757+0000","last_undegraded":"2024-04-25T09:01:33.352757+0000","last_fullsized":"2024-04-25T09:01:33.352757+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.368385+0000","last_change":"2024-04-25T09:01:33.367042+0000","last_active":"2024-04-25T09:01:33.368385+0000","last_peered":"2024-04-25T09:01:33.368385+0000","last_clean":"2024-04-25T09:01:33.368385+0000","last_became_active":"2024-04-25T09:01:33.366746+0000","last_became_peered":"2024-04-25T09:01:33.366746+0000","last_unstale":"2024-04-25T09:01:33.368385+0000","last_undegraded":"2024-04-25T09:01:33.368385+0000","last_fullsized":"2024-04-25T09:01:33.368385+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:34.749007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":317,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.357167+0000","last_change":"2024-04-25T08:54:00.818044+0000","last_active":"2024-04-25T09:01:33.357167+0000","last_peered":"2024-04-25T09:01:33.357167+0000","last_clean":"2024-04-25T09:01:33.357167+0000","last_became_active":"2024-04-25T08:54:00.817098+0000","last_became_peered":"2024-04-25T08:54:00.817098+0000","last_unstale":"2024-04-25T09:01:33.357167+0000","last_undegraded":"2024-04-25T09:01:33.357167+0000","last_fullsized":"2024-04-25T09:01:33.357167+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_clean_scrub_stamp":"2024-04-25T08:53:59.739677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:48.129513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574941,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55168,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316672,"statfs":{"total":96636764160,"available":96580272128,"internally_reserved":0,"allocated":1818624,"data_stored":1118350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574942,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80956,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94290884,"statfs":{"total":96636764160,"available":96553865216,"internally_reserved":0,"allocated":1224704,"data_stored":527982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":81656262},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574944,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50436,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321404,"statfs":{"total":96636764160,"available":96585117696,"internally_reserved":0,"allocated":1236992,"data_stored":534257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574942,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64328,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94307512,"statfs":{"total":96636764160,"available":96570892288,"internally_reserved":0,"allocated":1236992,"data_stored":534257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574942,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68488,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303352,"statfs":{"total":96636764160,"available":96566632448,"internally_reserved":0,"allocated":1236992,"data_stored":534257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574942,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58756,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313084,"statfs":{"total":96636764160,"available":96576598016,"internally_reserved":0,"allocated":1236992,"data_stored":534257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76804,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94295036,"statfs":{"total":96636764160,"available":96558116864,"internally_reserved":0,"allocated":1236992,"data_stored":534257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574943,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59340,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312500,"statfs":{"total":96636764160,"available":96576000000,"internally_reserved":0,"allocated":1830912,"data_stored":1124625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:01:38.060 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3f is not active+clean 2024-04-25T09:01:38.060 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.355534+0000', 'last_change': '2024-04-25T09:01:33.355534+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.355534+0000', 'last_undegraded': '2024-04-25T09:01:33.355534+0000', 'last_fullsized': '2024-04-25T09:01:33.355534+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:01:38.060 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3e is not active+clean 2024-04-25T09:01:38.060 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.359660+0000', 'last_change': '2024-04-25T09:01:33.359660+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.359660+0000', 'last_undegraded': '2024-04-25T09:01:33.359660+0000', 'last_fullsized': '2024-04-25T09:01:33.359660+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:01:38.060 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3d is not active+clean 2024-04-25T09:01:38.060 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.355300+0000', 'last_change': '2024-04-25T09:01:33.355300+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.355300+0000', 'last_undegraded': '2024-04-25T09:01:33.355300+0000', 'last_fullsized': '2024-04-25T09:01:33.355300+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:01:38.061 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3c is not active+clean 2024-04-25T09:01:38.061 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.356488+0000', 'last_change': '2024-04-25T09:01:33.356488+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.356488+0000', 'last_undegraded': '2024-04-25T09:01:33.356488+0000', 'last_fullsized': '2024-04-25T09:01:33.356488+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:01:38.061 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3b is not active+clean 2024-04-25T09:01:38.061 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.358970+0000', 'last_change': '2024-04-25T09:01:33.358970+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.358970+0000', 'last_undegraded': '2024-04-25T09:01:33.358970+0000', 'last_fullsized': '2024-04-25T09:01:33.358970+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:01:38.061 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3a is not active+clean 2024-04-25T09:01:38.061 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.358980+0000', 'last_change': '2024-04-25T09:01:33.358980+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.358980+0000', 'last_undegraded': '2024-04-25T09:01:33.358980+0000', 'last_fullsized': '2024-04-25T09:01:33.358980+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:01:38.061 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.39 is not active+clean 2024-04-25T09:01:38.061 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.356706+0000', 'last_change': '2024-04-25T09:01:33.356706+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.356706+0000', 'last_undegraded': '2024-04-25T09:01:33.356706+0000', 'last_fullsized': '2024-04-25T09:01:33.356706+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:01:38.062 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.38 is not active+clean 2024-04-25T09:01:38.062 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.357495+0000', 'last_change': '2024-04-25T09:01:33.357495+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.357495+0000', 'last_undegraded': '2024-04-25T09:01:33.357495+0000', 'last_fullsized': '2024-04-25T09:01:33.357495+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:01:38.062 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.37 is not active+clean 2024-04-25T09:01:38.062 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.359846+0000', 'last_change': '2024-04-25T09:01:33.359846+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.359846+0000', 'last_undegraded': '2024-04-25T09:01:33.359846+0000', 'last_fullsized': '2024-04-25T09:01:33.359846+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:01:38.062 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.36 is not active+clean 2024-04-25T09:01:38.062 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.358143+0000', 'last_change': '2024-04-25T09:01:33.358143+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.358143+0000', 'last_undegraded': '2024-04-25T09:01:33.358143+0000', 'last_fullsized': '2024-04-25T09:01:33.358143+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:01:38.062 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.35 is not active+clean 2024-04-25T09:01:38.062 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.358698+0000', 'last_change': '2024-04-25T09:01:33.358698+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.358698+0000', 'last_undegraded': '2024-04-25T09:01:33.358698+0000', 'last_fullsized': '2024-04-25T09:01:33.358698+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:01:38.062 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.34 is not active+clean 2024-04-25T09:01:38.062 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.356492+0000', 'last_change': '2024-04-25T09:01:33.356492+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.356492+0000', 'last_undegraded': '2024-04-25T09:01:33.356492+0000', 'last_fullsized': '2024-04-25T09:01:33.356492+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:01:38.062 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.33 is not active+clean 2024-04-25T09:01:38.063 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.355925+0000', 'last_change': '2024-04-25T09:01:33.355925+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.355925+0000', 'last_undegraded': '2024-04-25T09:01:33.355925+0000', 'last_fullsized': '2024-04-25T09:01:33.355925+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:01:38.063 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.32 is not active+clean 2024-04-25T09:01:38.063 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.362266+0000', 'last_change': '2024-04-25T09:01:33.362266+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.362266+0000', 'last_undegraded': '2024-04-25T09:01:33.362266+0000', 'last_fullsized': '2024-04-25T09:01:33.362266+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:01:38.063 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.31 is not active+clean 2024-04-25T09:01:38.063 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.358146+0000', 'last_change': '2024-04-25T09:01:33.358146+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.358146+0000', 'last_undegraded': '2024-04-25T09:01:33.358146+0000', 'last_fullsized': '2024-04-25T09:01:33.358146+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:01:38.063 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.30 is not active+clean 2024-04-25T09:01:38.063 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.356808+0000', 'last_change': '2024-04-25T09:01:33.356808+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.356808+0000', 'last_undegraded': '2024-04-25T09:01:33.356808+0000', 'last_fullsized': '2024-04-25T09:01:33.356808+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:01:38.063 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2f is not active+clean 2024-04-25T09:01:38.063 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.358063+0000', 'last_change': '2024-04-25T09:01:33.358063+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.358063+0000', 'last_undegraded': '2024-04-25T09:01:33.358063+0000', 'last_fullsized': '2024-04-25T09:01:33.358063+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:01:38.063 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2e is not active+clean 2024-04-25T09:01:38.063 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.358540+0000', 'last_change': '2024-04-25T09:01:33.358540+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.358540+0000', 'last_undegraded': '2024-04-25T09:01:33.358540+0000', 'last_fullsized': '2024-04-25T09:01:33.358540+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:01:38.064 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2d is not active+clean 2024-04-25T09:01:38.064 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.356094+0000', 'last_change': '2024-04-25T09:01:33.356094+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.356094+0000', 'last_undegraded': '2024-04-25T09:01:33.356094+0000', 'last_fullsized': '2024-04-25T09:01:33.356094+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:01:38.064 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2c is not active+clean 2024-04-25T09:01:38.064 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.355041+0000', 'last_change': '2024-04-25T09:01:33.355041+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.355041+0000', 'last_undegraded': '2024-04-25T09:01:33.355041+0000', 'last_fullsized': '2024-04-25T09:01:33.355041+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:01:38.064 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2b is not active+clean 2024-04-25T09:01:38.064 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.354826+0000', 'last_change': '2024-04-25T09:01:33.354826+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.354826+0000', 'last_undegraded': '2024-04-25T09:01:33.354826+0000', 'last_fullsized': '2024-04-25T09:01:33.354826+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:01:38.064 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2a is not active+clean 2024-04-25T09:01:38.064 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.358378+0000', 'last_change': '2024-04-25T09:01:33.358378+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.358378+0000', 'last_undegraded': '2024-04-25T09:01:33.358378+0000', 'last_fullsized': '2024-04-25T09:01:33.358378+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:01:38.064 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.29 is not active+clean 2024-04-25T09:01:38.064 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.356711+0000', 'last_change': '2024-04-25T09:01:33.356711+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.356711+0000', 'last_undegraded': '2024-04-25T09:01:33.356711+0000', 'last_fullsized': '2024-04-25T09:01:33.356711+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:01:38.064 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.28 is not active+clean 2024-04-25T09:01:38.064 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.356347+0000', 'last_change': '2024-04-25T09:01:33.356347+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.356347+0000', 'last_undegraded': '2024-04-25T09:01:33.356347+0000', 'last_fullsized': '2024-04-25T09:01:33.356347+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:01:38.065 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.27 is not active+clean 2024-04-25T09:01:38.065 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.362666+0000', 'last_change': '2024-04-25T09:01:33.362666+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.362666+0000', 'last_undegraded': '2024-04-25T09:01:33.362666+0000', 'last_fullsized': '2024-04-25T09:01:33.362666+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:01:38.065 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.26 is not active+clean 2024-04-25T09:01:38.065 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.359240+0000', 'last_change': '2024-04-25T09:01:33.359240+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.359240+0000', 'last_undegraded': '2024-04-25T09:01:33.359240+0000', 'last_fullsized': '2024-04-25T09:01:33.359240+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:01:38.065 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.25 is not active+clean 2024-04-25T09:01:38.065 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.358208+0000', 'last_change': '2024-04-25T09:01:33.358208+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.358208+0000', 'last_undegraded': '2024-04-25T09:01:33.358208+0000', 'last_fullsized': '2024-04-25T09:01:33.358208+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:01:38.065 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.24 is not active+clean 2024-04-25T09:01:38.065 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.363030+0000', 'last_change': '2024-04-25T09:01:33.363030+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.363030+0000', 'last_undegraded': '2024-04-25T09:01:33.363030+0000', 'last_fullsized': '2024-04-25T09:01:33.363030+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:01:38.065 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.23 is not active+clean 2024-04-25T09:01:38.065 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.357869+0000', 'last_change': '2024-04-25T09:01:33.357869+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.357869+0000', 'last_undegraded': '2024-04-25T09:01:33.357869+0000', 'last_fullsized': '2024-04-25T09:01:33.357869+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:01:38.066 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.22 is not active+clean 2024-04-25T09:01:38.066 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.355005+0000', 'last_change': '2024-04-25T09:01:33.355005+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.355005+0000', 'last_undegraded': '2024-04-25T09:01:33.355005+0000', 'last_fullsized': '2024-04-25T09:01:33.355005+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:01:38.066 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.21 is not active+clean 2024-04-25T09:01:38.066 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.357607+0000', 'last_change': '2024-04-25T09:01:33.357607+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.357607+0000', 'last_undegraded': '2024-04-25T09:01:33.357607+0000', 'last_fullsized': '2024-04-25T09:01:33.357607+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:01:38.066 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.20 is not active+clean 2024-04-25T09:01:38.066 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.354229+0000', 'last_change': '2024-04-25T09:01:33.354229+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.354229+0000', 'last_undegraded': '2024-04-25T09:01:33.354229+0000', 'last_fullsized': '2024-04-25T09:01:33.354229+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:01:38.066 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1e is not active+clean 2024-04-25T09:01:38.066 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.353890+0000', 'last_change': '2024-04-25T09:01:33.353890+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.353890+0000', 'last_undegraded': '2024-04-25T09:01:33.353890+0000', 'last_fullsized': '2024-04-25T09:01:33.353890+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:01:38.066 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1f is not active+clean 2024-04-25T09:01:38.066 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.356285+0000', 'last_change': '2024-04-25T09:01:33.356285+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.356285+0000', 'last_undegraded': '2024-04-25T09:01:33.356285+0000', 'last_fullsized': '2024-04-25T09:01:33.356285+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:01:38.066 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1c is not active+clean 2024-04-25T09:01:38.066 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.354216+0000', 'last_change': '2024-04-25T09:01:33.354216+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.354216+0000', 'last_undegraded': '2024-04-25T09:01:33.354216+0000', 'last_fullsized': '2024-04-25T09:01:33.354216+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:01:38.067 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1d is not active+clean 2024-04-25T09:01:38.067 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.357436+0000', 'last_change': '2024-04-25T09:01:33.357436+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.357436+0000', 'last_undegraded': '2024-04-25T09:01:33.357436+0000', 'last_fullsized': '2024-04-25T09:01:33.357436+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:01:38.067 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1a is not active+clean 2024-04-25T09:01:38.067 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.356196+0000', 'last_change': '2024-04-25T09:01:33.356196+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.356196+0000', 'last_undegraded': '2024-04-25T09:01:33.356196+0000', 'last_fullsized': '2024-04-25T09:01:33.356196+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:01:38.067 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1b is not active+clean 2024-04-25T09:01:38.067 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.356694+0000', 'last_change': '2024-04-25T09:01:33.356694+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.356694+0000', 'last_undegraded': '2024-04-25T09:01:33.356694+0000', 'last_fullsized': '2024-04-25T09:01:33.356694+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:01:38.067 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.18 is not active+clean 2024-04-25T09:01:38.067 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.357302+0000', 'last_change': '2024-04-25T09:01:33.357302+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.357302+0000', 'last_undegraded': '2024-04-25T09:01:33.357302+0000', 'last_fullsized': '2024-04-25T09:01:33.357302+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:01:38.068 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.19 is not active+clean 2024-04-25T09:01:38.068 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.355141+0000', 'last_change': '2024-04-25T09:01:33.355141+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.355141+0000', 'last_undegraded': '2024-04-25T09:01:33.355141+0000', 'last_fullsized': '2024-04-25T09:01:33.355141+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:01:38.068 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.16 is not active+clean 2024-04-25T09:01:38.068 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.357332+0000', 'last_change': '2024-04-25T09:01:33.357332+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.357332+0000', 'last_undegraded': '2024-04-25T09:01:33.357332+0000', 'last_fullsized': '2024-04-25T09:01:33.357332+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:01:38.068 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.17 is not active+clean 2024-04-25T09:01:38.068 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.354331+0000', 'last_change': '2024-04-25T09:01:33.354331+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.354331+0000', 'last_undegraded': '2024-04-25T09:01:33.354331+0000', 'last_fullsized': '2024-04-25T09:01:33.354331+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:01:38.068 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.14 is not active+clean 2024-04-25T09:01:38.068 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.362314+0000', 'last_change': '2024-04-25T09:01:33.362314+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.362314+0000', 'last_undegraded': '2024-04-25T09:01:33.362314+0000', 'last_fullsized': '2024-04-25T09:01:33.362314+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:01:38.068 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.15 is not active+clean 2024-04-25T09:01:38.068 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.357735+0000', 'last_change': '2024-04-25T09:01:33.357735+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.357735+0000', 'last_undegraded': '2024-04-25T09:01:33.357735+0000', 'last_fullsized': '2024-04-25T09:01:33.357735+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:01:38.068 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.12 is not active+clean 2024-04-25T09:01:38.069 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.355949+0000', 'last_change': '2024-04-25T09:01:33.355949+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.355949+0000', 'last_undegraded': '2024-04-25T09:01:33.355949+0000', 'last_fullsized': '2024-04-25T09:01:33.355949+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:01:38.069 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.13 is not active+clean 2024-04-25T09:01:38.069 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.354732+0000', 'last_change': '2024-04-25T09:01:33.354732+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.354732+0000', 'last_undegraded': '2024-04-25T09:01:33.354732+0000', 'last_fullsized': '2024-04-25T09:01:33.354732+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:01:38.069 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.10 is not active+clean 2024-04-25T09:01:38.069 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.359064+0000', 'last_change': '2024-04-25T09:01:33.359064+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.359064+0000', 'last_undegraded': '2024-04-25T09:01:33.359064+0000', 'last_fullsized': '2024-04-25T09:01:33.359064+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:01:38.069 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.11 is not active+clean 2024-04-25T09:01:38.069 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.352882+0000', 'last_change': '2024-04-25T09:01:33.352882+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.352882+0000', 'last_undegraded': '2024-04-25T09:01:33.352882+0000', 'last_fullsized': '2024-04-25T09:01:33.352882+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:01:38.069 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.f is not active+clean 2024-04-25T09:01:38.069 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.362269+0000', 'last_change': '2024-04-25T09:01:33.362269+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.362269+0000', 'last_undegraded': '2024-04-25T09:01:33.362269+0000', 'last_fullsized': '2024-04-25T09:01:33.362269+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:01:38.069 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.c is not active+clean 2024-04-25T09:01:38.069 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.355227+0000', 'last_change': '2024-04-25T09:01:33.355227+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.355227+0000', 'last_undegraded': '2024-04-25T09:01:33.355227+0000', 'last_fullsized': '2024-04-25T09:01:33.355227+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:01:38.070 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.d is not active+clean 2024-04-25T09:01:38.070 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.355533+0000', 'last_change': '2024-04-25T09:01:33.355533+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.355533+0000', 'last_undegraded': '2024-04-25T09:01:33.355533+0000', 'last_fullsized': '2024-04-25T09:01:33.355533+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:01:38.070 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.a is not active+clean 2024-04-25T09:01:38.070 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.353243+0000', 'last_change': '2024-04-25T09:01:33.353243+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.353243+0000', 'last_undegraded': '2024-04-25T09:01:33.353243+0000', 'last_fullsized': '2024-04-25T09:01:33.353243+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:01:38.070 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.b is not active+clean 2024-04-25T09:01:38.070 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.354197+0000', 'last_change': '2024-04-25T09:01:33.354197+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.354197+0000', 'last_undegraded': '2024-04-25T09:01:33.354197+0000', 'last_fullsized': '2024-04-25T09:01:33.354197+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:01:38.070 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.8 is not active+clean 2024-04-25T09:01:38.070 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.353721+0000', 'last_change': '2024-04-25T09:01:33.353721+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.353721+0000', 'last_undegraded': '2024-04-25T09:01:33.353721+0000', 'last_fullsized': '2024-04-25T09:01:33.353721+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:01:38.070 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.9 is not active+clean 2024-04-25T09:01:38.070 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.361446+0000', 'last_change': '2024-04-25T09:01:33.361446+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.361446+0000', 'last_undegraded': '2024-04-25T09:01:33.361446+0000', 'last_fullsized': '2024-04-25T09:01:33.361446+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:01:38.070 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.6 is not active+clean 2024-04-25T09:01:38.071 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.356865+0000', 'last_change': '2024-04-25T09:01:33.356865+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.356865+0000', 'last_undegraded': '2024-04-25T09:01:33.356865+0000', 'last_fullsized': '2024-04-25T09:01:33.356865+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:01:38.071 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.7 is not active+clean 2024-04-25T09:01:38.071 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.355038+0000', 'last_change': '2024-04-25T09:01:33.355038+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.355038+0000', 'last_undegraded': '2024-04-25T09:01:33.355038+0000', 'last_fullsized': '2024-04-25T09:01:33.355038+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:01:38.071 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.4 is not active+clean 2024-04-25T09:01:38.071 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.353719+0000', 'last_change': '2024-04-25T09:01:33.353719+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.353719+0000', 'last_undegraded': '2024-04-25T09:01:33.353719+0000', 'last_fullsized': '2024-04-25T09:01:33.353719+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:01:38.071 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.5 is not active+clean 2024-04-25T09:01:38.071 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.358486+0000', 'last_change': '2024-04-25T09:01:33.358486+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.358486+0000', 'last_undegraded': '2024-04-25T09:01:33.358486+0000', 'last_fullsized': '2024-04-25T09:01:33.358486+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:01:38.071 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2 is not active+clean 2024-04-25T09:01:38.071 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.352679+0000', 'last_change': '2024-04-25T09:01:33.352679+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.352679+0000', 'last_undegraded': '2024-04-25T09:01:33.352679+0000', 'last_fullsized': '2024-04-25T09:01:33.352679+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:01:38.071 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3 is not active+clean 2024-04-25T09:01:38.071 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.356269+0000', 'last_change': '2024-04-25T09:01:33.356269+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.356269+0000', 'last_undegraded': '2024-04-25T09:01:33.356269+0000', 'last_fullsized': '2024-04-25T09:01:33.356269+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:01:38.072 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.0 is not active+clean 2024-04-25T09:01:38.072 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.353264+0000', 'last_change': '2024-04-25T09:01:33.353264+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.353264+0000', 'last_undegraded': '2024-04-25T09:01:33.353264+0000', 'last_fullsized': '2024-04-25T09:01:33.353264+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:01:38.072 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1 is not active+clean 2024-04-25T09:01:38.072 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.361011+0000', 'last_change': '2024-04-25T09:01:33.361011+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.361011+0000', 'last_undegraded': '2024-04-25T09:01:33.361011+0000', 'last_fullsized': '2024-04-25T09:01:33.361011+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:01:38.072 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.e is not active+clean 2024-04-25T09:01:38.072 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 100, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:01:33.352757+0000', 'last_change': '2024-04-25T09:01:33.352757+0000', 'last_active': '2024-04-25T09:01:33.340929+0000', 'last_peered': '2024-04-25T09:01:33.340929+0000', 'last_clean': '2024-04-25T09:01:33.340929+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:01:33.352757+0000', 'last_undegraded': '2024-04-25T09:01:33.352757+0000', 'last_fullsized': '2024-04-25T09:01:33.352757+0000', 'mapping_epoch': 100, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 100, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'last_clean_scrub_stamp': '2024-04-25T09:01:33.340929+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:01:38.072 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:01:38.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.207+0000 7f7be55b7640 1 Processor -- start 2024-04-25T09:01:38.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.208+0000 7f7be55b7640 1 -- start start 2024-04-25T09:01:38.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.208+0000 7f7be55b7640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f7be006c510 0x7f7be006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:38.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.208+0000 7f7be55b7640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7be015df70 0x7f7be00ce070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:38.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.208+0000 7f7be55b7640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7be00cbee0 con 0x7f7be00b0870 2024-04-25T09:01:38.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.208+0000 7f7be55b7640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7be00d0400 con 0x7f7be015df70 2024-04-25T09:01:38.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.208+0000 7f7be55b7640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7be006ce30 con 0x7f7be006c510 2024-04-25T09:01:38.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.208+0000 7f7bd7fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7be015df70 0x7f7be00ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:38.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.209+0000 7f7bd7fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7be015df70 0x7f7be00ce070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49084/0 (socket says 172.21.15.43:49084) 2024-04-25T09:01:38.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.209+0000 7f7bd7fff640 1 -- 172.21.15.43:0/1316960431 learned_addr learned my addr 172.21.15.43:0/1316960431 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:38.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.209+0000 7f7bdeffd640 1 --2- 172.21.15.43:0/1316960431 >> v2:172.21.15.193:3300/0 conn(0x7f7be006c510 0x7f7be006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:38.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.209+0000 7f7bd7fff640 1 -- 172.21.15.43:0/1316960431 >> v2:172.21.15.193:3300/0 conn(0x7f7be006c510 msgr2=0x7f7be006c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:38.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.209+0000 7f7bd7fff640 1 --2- 172.21.15.43:0/1316960431 >> v2:172.21.15.193:3300/0 conn(0x7f7be006c510 0x7f7be006c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:38.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.209+0000 7f7bd7fff640 1 -- 172.21.15.43:0/1316960431 >> v1:172.21.15.193:6789/0 conn(0x7f7be00b0870 legacy=0x7f7be00ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:38.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.209+0000 7f7bd7fff640 1 -- 172.21.15.43:0/1316960431 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be00d0650 con 0x7f7be015df70 2024-04-25T09:01:38.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.210+0000 7f7bd7fff640 1 --2- 172.21.15.43:0/1316960431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7be015df70 0x7f7be00ce070 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f7be00727e0 tx=0x7f7bc802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ee3bf5ad8f4bda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:38.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.210+0000 7f7bde7fc640 1 -- 172.21.15.43:0/1316960431 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8037070 con 0x7f7be015df70 2024-04-25T09:01:38.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.210+0000 7f7bde7fc640 1 -- 172.21.15.43:0/1316960431 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc8032440 con 0x7f7be015df70 2024-04-25T09:01:38.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.210+0000 7f7bde7fc640 1 -- 172.21.15.43:0/1316960431 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8032810 con 0x7f7be015df70 2024-04-25T09:01:38.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.210+0000 7f7be55b7640 1 -- 172.21.15.43:0/1316960431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7be015df70 msgr2=0x7f7be00ce070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:38.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.210+0000 7f7be55b7640 1 --2- 172.21.15.43:0/1316960431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7be015df70 0x7f7be00ce070 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f7be00727e0 tx=0x7f7bc802a270 comp rx=0 tx=0).stop 2024-04-25T09:01:38.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.210+0000 7f7be55b7640 1 -- 172.21.15.43:0/1316960431 shutdown_connections 2024-04-25T09:01:38.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.210+0000 7f7be55b7640 1 --2- 172.21.15.43:0/1316960431 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7be015df70 0x7f7be00ce070 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:38.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.210+0000 7f7be55b7640 1 --2- 172.21.15.43:0/1316960431 >> v2:172.21.15.193:3300/0 conn(0x7f7be006c510 0x7f7be006c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:38.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.210+0000 7f7be55b7640 1 -- 172.21.15.43:0/1316960431 >> 172.21.15.43:0/1316960431 conn(0x7f7be00d17f0 msgr2=0x7f7be00d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:38.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.210+0000 7f7be55b7640 1 -- 172.21.15.43:0/1316960431 shutdown_connections 2024-04-25T09:01:38.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.210+0000 7f7be55b7640 1 -- 172.21.15.43:0/1316960431 wait complete. 2024-04-25T09:01:38.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.212+0000 7f7be55b7640 1 Processor -- start 2024-04-25T09:01:38.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.212+0000 7f7be55b7640 1 -- start start 2024-04-25T09:01:38.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.212+0000 7f7be55b7640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7be006c510 0x7f7be01591c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:38.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.212+0000 7f7be55b7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7be00b0870 0x7f7be00c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:38.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.212+0000 7f7be55b7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7be015df70 0x7f7be00c7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:38.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.212+0000 7f7be55b7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be00d0c10 con 0x7f7be00b0870 2024-04-25T09:01:38.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.212+0000 7f7be55b7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7be00cc150 con 0x7f7be015df70 2024-04-25T09:01:38.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.212+0000 7f7be55b7640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7be00cecc0 con 0x7f7be006c510 2024-04-25T09:01:38.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.212+0000 7f7bd7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7be00b0870 0x7f7be00c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:38.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.213+0000 7f7bd7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7be00b0870 0x7f7be00c8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49086/0 (socket says 172.21.15.43:49086) 2024-04-25T09:01:38.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.213+0000 7f7bd7fff640 1 -- 172.21.15.43:0/2733004048 learned_addr learned my addr 172.21.15.43:0/2733004048 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.213+0000 7f7bdf7fe640 1 --2- 172.21.15.43:0/2733004048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7be015df70 0x7f7be00c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.213+0000 7f7bdeffd640 1 --2- 172.21.15.43:0/2733004048 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7be006c510 0x7f7be01591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.213+0000 7f7bd7fff640 1 -- 172.21.15.43:0/2733004048 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7be006c510 msgr2=0x7f7be01591c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.213+0000 7f7bd7fff640 1 --2- 172.21.15.43:0/2733004048 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7be006c510 0x7f7be01591c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.213+0000 7f7bd7fff640 1 -- 172.21.15.43:0/2733004048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7be015df70 msgr2=0x7f7be00c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.213+0000 7f7bd7fff640 1 --2- 172.21.15.43:0/2733004048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7be015df70 0x7f7be00c7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.213+0000 7f7bd7fff640 1 -- 172.21.15.43:0/2733004048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be00c9440 con 0x7f7be00b0870 2024-04-25T09:01:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.213+0000 7f7bd7fff640 1 --2- 172.21.15.43:0/2733004048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7be00b0870 0x7f7be00c8b40 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f7bc802a2b0 tx=0x7f7bc80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.213+0000 7f7bd77fe640 1 -- 172.21.15.43:0/2733004048 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8037040 con 0x7f7be00b0870 2024-04-25T09:01:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.213+0000 7f7bd77fe640 1 -- 172.21.15.43:0/2733004048 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc8042070 con 0x7f7be00b0870 2024-04-25T09:01:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.213+0000 7f7be55b7640 1 -- 172.21.15.43:0/2733004048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7be00c9720 con 0x7f7be00b0870 2024-04-25T09:01:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.213+0000 7f7bd77fe640 1 -- 172.21.15.43:0/2733004048 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8030880 con 0x7f7be00b0870 2024-04-25T09:01:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.213+0000 7f7be55b7640 1 -- 172.21.15.43:0/2733004048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7be00c9ce0 con 0x7f7be00b0870 2024-04-25T09:01:38.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.214+0000 7f7be55b7640 1 -- 172.21.15.43:0/2733004048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7be0074ef0 con 0x7f7be00b0870 2024-04-25T09:01:38.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.215+0000 7f7bd77fe640 1 -- 172.21.15.43:0/2733004048 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7bc8030a20 con 0x7f7be00b0870 2024-04-25T09:01:38.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.215+0000 7f7bd77fe640 1 --2- 172.21.15.43:0/2733004048 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7bc006b4d0 0x7f7bc006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:38.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.215+0000 7f7bd77fe640 1 -- 172.21.15.43:0/2733004048 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7bc80aa8b0 con 0x7f7be00b0870 2024-04-25T09:01:38.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.215+0000 7f7bdeffd640 1 --2- 172.21.15.43:0/2733004048 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7bc006b4d0 0x7f7bc006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:38.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.216+0000 7f7bdeffd640 1 --2- 172.21.15.43:0/2733004048 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7bc006b4d0 0x7f7bc006d990 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0009b40 tx=0x7f7bd0009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:38.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.219+0000 7f7bd77fe640 1 -- 172.21.15.43:0/2733004048 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7bc8079170 con 0x7f7be00b0870 2024-04-25T09:01:38.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.471+0000 7f7be55b7640 1 -- 172.21.15.43:0/2733004048 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f7be00cd850 con 0x7f7be00b0870 2024-04-25T09:01:38.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.471+0000 7f7bd77fe640 1 -- 172.21.15.43:0/2733004048 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1584 (secure 0 0 0) 0x7f7bc807d210 con 0x7f7be00b0870 2024-04-25T09:01:38.461 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:01:38.461 INFO:teuthology.orchestra.run.smithi043.stdout:{"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":470,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":102,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035238,"num_in_osds":8,"osd_in_since":1714035229,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":526618624,"bytes_avail":772567494656,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":61,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"b","status":"up:active","gid":8094}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":17,"modified":"2024-04-25T09:01:15.791620+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:01:38.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.474+0000 7f7be55b7640 1 -- 172.21.15.43:0/2733004048 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7bc006b4d0 msgr2=0x7f7bc006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:38.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.474+0000 7f7be55b7640 1 --2- 172.21.15.43:0/2733004048 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7bc006b4d0 0x7f7bc006d990 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0009b40 tx=0x7f7bd0009190 comp rx=0 tx=0).stop 2024-04-25T09:01:38.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.474+0000 7f7be55b7640 1 -- 172.21.15.43:0/2733004048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7be00b0870 msgr2=0x7f7be00c8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:38.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.474+0000 7f7be55b7640 1 --2- 172.21.15.43:0/2733004048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7be00b0870 0x7f7be00c8b40 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f7bc802a2b0 tx=0x7f7bc80046c0 comp rx=0 tx=0).stop 2024-04-25T09:01:38.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.474+0000 7f7be55b7640 1 -- 172.21.15.43:0/2733004048 shutdown_connections 2024-04-25T09:01:38.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.474+0000 7f7be55b7640 1 --2- 172.21.15.43:0/2733004048 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7bc006b4d0 0x7f7bc006d990 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:38.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.474+0000 7f7be55b7640 1 --2- 172.21.15.43:0/2733004048 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7be015df70 0x7f7be00c7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:38.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.474+0000 7f7be55b7640 1 --2- 172.21.15.43:0/2733004048 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7be00b0870 0x7f7be00c8b40 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:38.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.474+0000 7f7be55b7640 1 --2- 172.21.15.43:0/2733004048 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7be006c510 0x7f7be01591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:38.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.474+0000 7f7be55b7640 1 -- 172.21.15.43:0/2733004048 >> 172.21.15.43:0/2733004048 conn(0x7f7be00d17f0 msgr2=0x7f7be00aed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:38.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.474+0000 7f7be55b7640 1 -- 172.21.15.43:0/2733004048 shutdown_connections 2024-04-25T09:01:38.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.474+0000 7f7be55b7640 1 -- 172.21.15.43:0/2733004048 wait complete. 2024-04-25T09:01:38.474 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T09:01:38.474 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:01:38.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.586+0000 7f6ba8eac640 1 Processor -- start 2024-04-25T09:01:38.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.586+0000 7f6ba8eac640 1 -- start start 2024-04-25T09:01:38.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.587+0000 7f6ba8eac640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6ba415df50 0x7f6ba41619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:38.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.587+0000 7f6ba8eac640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ba4161f30 0x7f6ba41623a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:38.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.587+0000 7f6ba8eac640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ba40d24a0 con 0x7f6ba40c7cf0 2024-04-25T09:01:38.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.587+0000 7f6ba8eac640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ba415e330 con 0x7f6ba415df50 2024-04-25T09:01:38.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.587+0000 7f6ba8eac640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba40d2740 con 0x7f6ba4161f30 2024-04-25T09:01:38.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.588+0000 7f6ba1d9b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ba4161f30 0x7f6ba41623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:38.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.588+0000 7f6ba259c640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6ba415df50 0x7f6ba41619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:38.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.588+0000 7f6ba259c640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6ba415df50 0x7f6ba41619f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:59666/0 (socket says 172.21.15.43:59666) 2024-04-25T09:01:38.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.588+0000 7f6ba259c640 1 -- 172.21.15.43:0/512522852 learned_addr learned my addr 172.21.15.43:0/512522852 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:38.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.588+0000 7f6ba2d9d640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f6ba40c7cf0 0x7f6ba40c8100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:55270/0 (socket says 172.21.15.43:55270) 2024-04-25T09:01:38.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.588+0000 7f6ba1d9b640 1 --2- 172.21.15.43:0/512522852 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ba4161f30 0x7f6ba41623a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:50730/0 (socket says 172.21.15.43:50730) 2024-04-25T09:01:38.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.588+0000 7f6ba1d9b640 1 -- 172.21.15.43:0/512522852 >> v2:172.21.15.193:3300/0 conn(0x7f6ba415df50 msgr2=0x7f6ba41619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:38.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.588+0000 7f6ba1d9b640 1 --2- 172.21.15.43:0/512522852 >> v2:172.21.15.193:3300/0 conn(0x7f6ba415df50 0x7f6ba41619f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:38.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.588+0000 7f6ba1d9b640 1 -- 172.21.15.43:0/512522852 >> v1:172.21.15.193:6789/0 conn(0x7f6ba40c7cf0 legacy=0x7f6ba40c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:38.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.588+0000 7f6ba1d9b640 1 -- 172.21.15.43:0/512522852 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba40d2a20 con 0x7f6ba4161f30 2024-04-25T09:01:38.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.588+0000 7f6ba259c640 1 --2- 172.21.15.43:0/512522852 >> v2:172.21.15.193:3300/0 conn(0x7f6ba415df50 0x7f6ba41619f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:01:38.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.589+0000 7f6ba1d9b640 1 --2- 172.21.15.43:0/512522852 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ba4161f30 0x7f6ba41623a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c0097a0 tx=0x7f6b8c02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5a57b7ab00930 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:38.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.589+0000 7f6ba159a640 1 -- 172.21.15.43:0/512522852 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c037070 con 0x7f6ba4161f30 2024-04-25T09:01:38.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.589+0000 7f6ba159a640 1 -- 172.21.15.43:0/512522852 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b8c032440 con 0x7f6ba4161f30 2024-04-25T09:01:38.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.589+0000 7f6ba159a640 1 -- 172.21.15.43:0/512522852 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8c032810 con 0x7f6ba4161f30 2024-04-25T09:01:38.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.589+0000 7f6ba8eac640 1 -- 172.21.15.43:0/512522852 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ba4161f30 msgr2=0x7f6ba41623a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:38.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.589+0000 7f6ba8eac640 1 --2- 172.21.15.43:0/512522852 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ba4161f30 0x7f6ba41623a0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c0097a0 tx=0x7f6b8c02a180 comp rx=0 tx=0).stop 2024-04-25T09:01:38.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.590+0000 7f6ba8eac640 1 -- 172.21.15.43:0/512522852 shutdown_connections 2024-04-25T09:01:38.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.590+0000 7f6ba8eac640 1 --2- 172.21.15.43:0/512522852 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ba4161f30 0x7f6ba41623a0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:38.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.590+0000 7f6ba8eac640 1 --2- 172.21.15.43:0/512522852 >> v2:172.21.15.193:3300/0 conn(0x7f6ba415df50 0x7f6ba41619f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:38.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.590+0000 7f6ba8eac640 1 -- 172.21.15.43:0/512522852 >> 172.21.15.43:0/512522852 conn(0x7f6ba40d3a30 msgr2=0x7f6ba40d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:38.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.590+0000 7f6ba8eac640 1 -- 172.21.15.43:0/512522852 shutdown_connections 2024-04-25T09:01:38.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.590+0000 7f6ba8eac640 1 -- 172.21.15.43:0/512522852 wait complete. 2024-04-25T09:01:38.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.591+0000 7f6ba8eac640 1 Processor -- start 2024-04-25T09:01:38.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.591+0000 7f6ba8eac640 1 -- start start 2024-04-25T09:01:38.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.592+0000 7f6ba8eac640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ba40c7cf0 0x7f6ba4150650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:38.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.592+0000 7f6ba8eac640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ba415df50 0x7f6ba4156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:38.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.592+0000 7f6ba8eac640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ba4161f30 0x7f6ba415cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:38.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.592+0000 7f6ba8eac640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba40d2190 con 0x7f6ba4161f30 2024-04-25T09:01:38.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.592+0000 7f6ba8eac640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba40c77d0 con 0x7f6ba40c7cf0 2024-04-25T09:01:38.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.592+0000 7f6ba8eac640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba40cf670 con 0x7f6ba415df50 2024-04-25T09:01:38.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.592+0000 7f6ba2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ba4161f30 0x7f6ba415cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:38.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.592+0000 7f6ba2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ba4161f30 0x7f6ba415cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49102/0 (socket says 172.21.15.43:49102) 2024-04-25T09:01:38.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.592+0000 7f6ba2d9d640 1 -- 172.21.15.43:0/4069284262 learned_addr learned my addr 172.21.15.43:0/4069284262 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:38.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.592+0000 7f6ba259c640 1 --2- 172.21.15.43:0/4069284262 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ba40c7cf0 0x7f6ba4150650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:38.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.592+0000 7f6ba1d9b640 1 --2- 172.21.15.43:0/4069284262 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ba415df50 0x7f6ba4156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:38.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.592+0000 7f6ba2d9d640 1 -- 172.21.15.43:0/4069284262 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ba415df50 msgr2=0x7f6ba4156590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:38.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.592+0000 7f6ba2d9d640 1 --2- 172.21.15.43:0/4069284262 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ba415df50 0x7f6ba4156590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:38.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.592+0000 7f6ba2d9d640 1 -- 172.21.15.43:0/4069284262 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ba40c7cf0 msgr2=0x7f6ba4150650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:38.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.592+0000 7f6ba2d9d640 1 --2- 172.21.15.43:0/4069284262 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ba40c7cf0 0x7f6ba4150650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:38.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.592+0000 7f6ba2d9d640 1 -- 172.21.15.43:0/4069284262 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ba41f44e0 con 0x7f6ba4161f30 2024-04-25T09:01:38.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.592+0000 7f6ba1d9b640 1 --2- 172.21.15.43:0/4069284262 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ba415df50 0x7f6ba4156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:01:38.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.592+0000 7f6ba2d9d640 1 --2- 172.21.15.43:0/4069284262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ba4161f30 0x7f6ba415cc60 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f6b9400c7b0 tx=0x7f6b9400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:38.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.593+0000 7f6b9affd640 1 -- 172.21.15.43:0/4069284262 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b94017070 con 0x7f6ba4161f30 2024-04-25T09:01:38.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.593+0000 7f6b9affd640 1 -- 172.21.15.43:0/4069284262 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b9400a640 con 0x7f6ba4161f30 2024-04-25T09:01:38.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.593+0000 7f6ba8eac640 1 -- 172.21.15.43:0/4069284262 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ba4150b90 con 0x7f6ba4161f30 2024-04-25T09:01:38.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.593+0000 7f6b9affd640 1 -- 172.21.15.43:0/4069284262 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b9400a9f0 con 0x7f6ba4161f30 2024-04-25T09:01:38.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.593+0000 7f6ba8eac640 1 -- 172.21.15.43:0/4069284262 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ba414e100 con 0x7f6ba4161f30 2024-04-25T09:01:38.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.594+0000 7f6ba8eac640 1 -- 172.21.15.43:0/4069284262 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ba40680f0 con 0x7f6ba4161f30 2024-04-25T09:01:38.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.594+0000 7f6b9affd640 1 -- 172.21.15.43:0/4069284262 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6b9401a070 con 0x7f6ba4161f30 2024-04-25T09:01:38.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.595+0000 7f6b9affd640 1 --2- 172.21.15.43:0/4069284262 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6b8406b4d0 0x7f6b8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:38.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.595+0000 7f6b9affd640 1 -- 172.21.15.43:0/4069284262 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6b940896e0 con 0x7f6ba4161f30 2024-04-25T09:01:38.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.595+0000 7f6ba259c640 1 --2- 172.21.15.43:0/4069284262 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6b8406b4d0 0x7f6b8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:38.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.595+0000 7f6ba259c640 1 --2- 172.21.15.43:0/4069284262 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6b8406b4d0 0x7f6b8406d990 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c006ca0 tx=0x7f6b9c009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:38.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.599+0000 7f6b9affd640 1 -- 172.21.15.43:0/4069284262 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6b94057f10 con 0x7f6ba4161f30 2024-04-25T09:01:38.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.774+0000 7f6ba8eac640 1 -- 172.21.15.43:0/4069284262 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6ba4074ef0 con 0x7f6b8406b4d0 2024-04-25T09:01:38.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.781+0000 7f6b9affd640 1 -- 172.21.15.43:0/4069284262 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247013 (secure 0 0 0) 0x7f6ba4074ef0 con 0x7f6b8406b4d0 2024-04-25T09:01:38.772 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:01:38.773 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-25T09:01:38.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.784+0000 7f6ba8eac640 1 -- 172.21.15.43:0/4069284262 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6b8406b4d0 msgr2=0x7f6b8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:38.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.784+0000 7f6ba8eac640 1 --2- 172.21.15.43:0/4069284262 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6b8406b4d0 0x7f6b8406d990 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c006ca0 tx=0x7f6b9c009240 comp rx=0 tx=0).stop 2024-04-25T09:01:38.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.784+0000 7f6ba8eac640 1 -- 172.21.15.43:0/4069284262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ba4161f30 msgr2=0x7f6ba415cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:38.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.784+0000 7f6ba8eac640 1 --2- 172.21.15.43:0/4069284262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ba4161f30 0x7f6ba415cc60 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f6b9400c7b0 tx=0x7f6b9400cc70 comp rx=0 tx=0).stop 2024-04-25T09:01:38.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.784+0000 7f6ba8eac640 1 -- 172.21.15.43:0/4069284262 shutdown_connections 2024-04-25T09:01:38.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.784+0000 7f6ba8eac640 1 --2- 172.21.15.43:0/4069284262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ba4161f30 0x7f6ba415cc60 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:38.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.784+0000 7f6ba8eac640 1 --2- 172.21.15.43:0/4069284262 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ba415df50 0x7f6ba4156590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:38.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.784+0000 7f6ba8eac640 1 --2- 172.21.15.43:0/4069284262 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6b8406b4d0 0x7f6b8406d990 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:38.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.784+0000 7f6ba8eac640 1 --2- 172.21.15.43:0/4069284262 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ba40c7cf0 0x7f6ba4150650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:38.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.784+0000 7f6ba8eac640 1 -- 172.21.15.43:0/4069284262 >> 172.21.15.43:0/4069284262 conn(0x7f6ba40d3a30 msgr2=0x7f6ba415a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:38.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.784+0000 7f6ba8eac640 1 -- 172.21.15.43:0/4069284262 shutdown_connections 2024-04-25T09:01:38.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:38.784+0000 7f6ba8eac640 1 -- 172.21.15.43:0/4069284262 wait complete. 2024-04-25T09:01:38.784 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":325,"stamp":"2024-04-25T09:01:37.797091+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":24,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":514276,"kb_used_data":10800,"kb_used_omap":12,"kb_used_meta":503347,"kb_avail":754460444,"statfs":{"total":773094113280,"available":772567494656,"internally_reserved":0,"allocated":11059200,"data_stored":5442242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":515427889},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.452898"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.355534+0000","last_change":"2024-04-25T09:01:33.355534+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.355534+0000","last_undegraded":"2024-04-25T09:01:33.355534+0000","last_fullsized":"2024-04-25T09:01:33.355534+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.359660+0000","last_change":"2024-04-25T09:01:33.359660+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.359660+0000","last_undegraded":"2024-04-25T09:01:33.359660+0000","last_fullsized":"2024-04-25T09:01:33.359660+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.355300+0000","last_change":"2024-04-25T09:01:33.355300+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.355300+0000","last_undegraded":"2024-04-25T09:01:33.355300+0000","last_fullsized":"2024-04-25T09:01:33.355300+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356488+0000","last_change":"2024-04-25T09:01:33.356488+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356488+0000","last_undegraded":"2024-04-25T09:01:33.356488+0000","last_fullsized":"2024-04-25T09:01:33.356488+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.358970+0000","last_change":"2024-04-25T09:01:33.358970+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.358970+0000","last_undegraded":"2024-04-25T09:01:33.358970+0000","last_fullsized":"2024-04-25T09:01:33.358970+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.358980+0000","last_change":"2024-04-25T09:01:33.358980+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.358980+0000","last_undegraded":"2024-04-25T09:01:33.358980+0000","last_fullsized":"2024-04-25T09:01:33.358980+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356706+0000","last_change":"2024-04-25T09:01:33.356706+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356706+0000","last_undegraded":"2024-04-25T09:01:33.356706+0000","last_fullsized":"2024-04-25T09:01:33.356706+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.357495+0000","last_change":"2024-04-25T09:01:33.357495+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.357495+0000","last_undegraded":"2024-04-25T09:01:33.357495+0000","last_fullsized":"2024-04-25T09:01:33.357495+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.359846+0000","last_change":"2024-04-25T09:01:33.359846+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.359846+0000","last_undegraded":"2024-04-25T09:01:33.359846+0000","last_fullsized":"2024-04-25T09:01:33.359846+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.358143+0000","last_change":"2024-04-25T09:01:33.358143+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.358143+0000","last_undegraded":"2024-04-25T09:01:33.358143+0000","last_fullsized":"2024-04-25T09:01:33.358143+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.358698+0000","last_change":"2024-04-25T09:01:33.358698+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.358698+0000","last_undegraded":"2024-04-25T09:01:33.358698+0000","last_fullsized":"2024-04-25T09:01:33.358698+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356492+0000","last_change":"2024-04-25T09:01:33.356492+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356492+0000","last_undegraded":"2024-04-25T09:01:33.356492+0000","last_fullsized":"2024-04-25T09:01:33.356492+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.355925+0000","last_change":"2024-04-25T09:01:33.355925+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.355925+0000","last_undegraded":"2024-04-25T09:01:33.355925+0000","last_fullsized":"2024-04-25T09:01:33.355925+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.362266+0000","last_change":"2024-04-25T09:01:33.362266+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.362266+0000","last_undegraded":"2024-04-25T09:01:33.362266+0000","last_fullsized":"2024-04-25T09:01:33.362266+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.358146+0000","last_change":"2024-04-25T09:01:33.358146+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.358146+0000","last_undegraded":"2024-04-25T09:01:33.358146+0000","last_fullsized":"2024-04-25T09:01:33.358146+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356808+0000","last_change":"2024-04-25T09:01:33.356808+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356808+0000","last_undegraded":"2024-04-25T09:01:33.356808+0000","last_fullsized":"2024-04-25T09:01:33.356808+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.358063+0000","last_change":"2024-04-25T09:01:33.358063+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.358063+0000","last_undegraded":"2024-04-25T09:01:33.358063+0000","last_fullsized":"2024-04-25T09:01:33.358063+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.358540+0000","last_change":"2024-04-25T09:01:33.358540+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.358540+0000","last_undegraded":"2024-04-25T09:01:33.358540+0000","last_fullsized":"2024-04-25T09:01:33.358540+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356094+0000","last_change":"2024-04-25T09:01:33.356094+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356094+0000","last_undegraded":"2024-04-25T09:01:33.356094+0000","last_fullsized":"2024-04-25T09:01:33.356094+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.355041+0000","last_change":"2024-04-25T09:01:33.355041+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.355041+0000","last_undegraded":"2024-04-25T09:01:33.355041+0000","last_fullsized":"2024-04-25T09:01:33.355041+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.354826+0000","last_change":"2024-04-25T09:01:33.354826+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.354826+0000","last_undegraded":"2024-04-25T09:01:33.354826+0000","last_fullsized":"2024-04-25T09:01:33.354826+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.358378+0000","last_change":"2024-04-25T09:01:33.358378+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.358378+0000","last_undegraded":"2024-04-25T09:01:33.358378+0000","last_fullsized":"2024-04-25T09:01:33.358378+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356711+0000","last_change":"2024-04-25T09:01:33.356711+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356711+0000","last_undegraded":"2024-04-25T09:01:33.356711+0000","last_fullsized":"2024-04-25T09:01:33.356711+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356347+0000","last_change":"2024-04-25T09:01:33.356347+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356347+0000","last_undegraded":"2024-04-25T09:01:33.356347+0000","last_fullsized":"2024-04-25T09:01:33.356347+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.362666+0000","last_change":"2024-04-25T09:01:33.362666+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.362666+0000","last_undegraded":"2024-04-25T09:01:33.362666+0000","last_fullsized":"2024-04-25T09:01:33.362666+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.359240+0000","last_change":"2024-04-25T09:01:33.359240+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.359240+0000","last_undegraded":"2024-04-25T09:01:33.359240+0000","last_fullsized":"2024-04-25T09:01:33.359240+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.358208+0000","last_change":"2024-04-25T09:01:33.358208+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.358208+0000","last_undegraded":"2024-04-25T09:01:33.358208+0000","last_fullsized":"2024-04-25T09:01:33.358208+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.363030+0000","last_change":"2024-04-25T09:01:33.363030+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.363030+0000","last_undegraded":"2024-04-25T09:01:33.363030+0000","last_fullsized":"2024-04-25T09:01:33.363030+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.357869+0000","last_change":"2024-04-25T09:01:33.357869+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.357869+0000","last_undegraded":"2024-04-25T09:01:33.357869+0000","last_fullsized":"2024-04-25T09:01:33.357869+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.355005+0000","last_change":"2024-04-25T09:01:33.355005+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.355005+0000","last_undegraded":"2024-04-25T09:01:33.355005+0000","last_fullsized":"2024-04-25T09:01:33.355005+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.357607+0000","last_change":"2024-04-25T09:01:33.357607+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.357607+0000","last_undegraded":"2024-04-25T09:01:33.357607+0000","last_fullsized":"2024-04-25T09:01:33.357607+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.354229+0000","last_change":"2024-04-25T09:01:33.354229+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.354229+0000","last_undegraded":"2024-04-25T09:01:33.354229+0000","last_fullsized":"2024-04-25T09:01:33.354229+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.353890+0000","last_change":"2024-04-25T09:01:33.353890+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.353890+0000","last_undegraded":"2024-04-25T09:01:33.353890+0000","last_fullsized":"2024-04-25T09:01:33.353890+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.367304+0000","last_change":"2024-04-25T09:01:33.365666+0000","last_active":"2024-04-25T09:01:33.367304+0000","last_peered":"2024-04-25T09:01:33.367304+0000","last_clean":"2024-04-25T09:01:33.367304+0000","last_became_active":"2024-04-25T09:01:33.365341+0000","last_became_peered":"2024-04-25T09:01:33.365341+0000","last_unstale":"2024-04-25T09:01:33.367304+0000","last_undegraded":"2024-04-25T09:01:33.367304+0000","last_fullsized":"2024-04-25T09:01:33.367304+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:41.915410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356285+0000","last_change":"2024-04-25T09:01:33.356285+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356285+0000","last_undegraded":"2024-04-25T09:01:33.356285+0000","last_fullsized":"2024-04-25T09:01:33.356285+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.368360+0000","last_change":"2024-04-25T09:01:33.366268+0000","last_active":"2024-04-25T09:01:33.368360+0000","last_peered":"2024-04-25T09:01:33.368360+0000","last_clean":"2024-04-25T09:01:33.368360+0000","last_became_active":"2024-04-25T09:01:33.365952+0000","last_became_peered":"2024-04-25T09:01:33.365952+0000","last_unstale":"2024-04-25T09:01:33.368360+0000","last_undegraded":"2024-04-25T09:01:33.368360+0000","last_fullsized":"2024-04-25T09:01:33.368360+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:29.149495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.354216+0000","last_change":"2024-04-25T09:01:33.354216+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.354216+0000","last_undegraded":"2024-04-25T09:01:33.354216+0000","last_fullsized":"2024-04-25T09:01:33.354216+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.371487+0000","last_change":"2024-04-25T09:01:33.370140+0000","last_active":"2024-04-25T09:01:33.371487+0000","last_peered":"2024-04-25T09:01:33.371487+0000","last_clean":"2024-04-25T09:01:33.371487+0000","last_became_active":"2024-04-25T09:01:33.369677+0000","last_became_peered":"2024-04-25T09:01:33.369677+0000","last_unstale":"2024-04-25T09:01:33.371487+0000","last_undegraded":"2024-04-25T09:01:33.371487+0000","last_fullsized":"2024-04-25T09:01:33.371487+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:15.517604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.357436+0000","last_change":"2024-04-25T09:01:33.357436+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.357436+0000","last_undegraded":"2024-04-25T09:01:33.357436+0000","last_fullsized":"2024-04-25T09:01:33.357436+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.369409+0000","last_change":"2024-04-25T09:01:33.367516+0000","last_active":"2024-04-25T09:01:33.369409+0000","last_peered":"2024-04-25T09:01:33.369409+0000","last_clean":"2024-04-25T09:01:33.369409+0000","last_became_active":"2024-04-25T09:01:33.367171+0000","last_became_peered":"2024-04-25T09:01:33.367171+0000","last_unstale":"2024-04-25T09:01:33.369409+0000","last_undegraded":"2024-04-25T09:01:33.369409+0000","last_fullsized":"2024-04-25T09:01:33.369409+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:43.586923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356196+0000","last_change":"2024-04-25T09:01:33.356196+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356196+0000","last_undegraded":"2024-04-25T09:01:33.356196+0000","last_fullsized":"2024-04-25T09:01:33.356196+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.363802+0000","last_change":"2024-04-25T09:01:33.360517+0000","last_active":"2024-04-25T09:01:33.363802+0000","last_peered":"2024-04-25T09:01:33.363802+0000","last_clean":"2024-04-25T09:01:33.363802+0000","last_became_active":"2024-04-25T09:01:33.360139+0000","last_became_peered":"2024-04-25T09:01:33.360139+0000","last_unstale":"2024-04-25T09:01:33.363802+0000","last_undegraded":"2024-04-25T09:01:33.363802+0000","last_fullsized":"2024-04-25T09:01:33.363802+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:50.515806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356694+0000","last_change":"2024-04-25T09:01:33.356694+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356694+0000","last_undegraded":"2024-04-25T09:01:33.356694+0000","last_fullsized":"2024-04-25T09:01:33.356694+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.372295+0000","last_change":"2024-04-25T09:01:33.371930+0000","last_active":"2024-04-25T09:01:33.372295+0000","last_peered":"2024-04-25T09:01:33.372295+0000","last_clean":"2024-04-25T09:01:33.372295+0000","last_became_active":"2024-04-25T09:01:33.371395+0000","last_became_peered":"2024-04-25T09:01:33.371395+0000","last_unstale":"2024-04-25T09:01:33.372295+0000","last_undegraded":"2024-04-25T09:01:33.372295+0000","last_fullsized":"2024-04-25T09:01:33.372295+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:44.824026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.357302+0000","last_change":"2024-04-25T09:01:33.357302+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.357302+0000","last_undegraded":"2024-04-25T09:01:33.357302+0000","last_fullsized":"2024-04-25T09:01:33.357302+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.367580+0000","last_change":"2024-04-25T09:01:33.366526+0000","last_active":"2024-04-25T09:01:33.367580+0000","last_peered":"2024-04-25T09:01:33.367580+0000","last_clean":"2024-04-25T09:01:33.367580+0000","last_became_active":"2024-04-25T09:01:33.366246+0000","last_became_peered":"2024-04-25T09:01:33.366246+0000","last_unstale":"2024-04-25T09:01:33.367580+0000","last_undegraded":"2024-04-25T09:01:33.367580+0000","last_fullsized":"2024-04-25T09:01:33.367580+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:26.679025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.355141+0000","last_change":"2024-04-25T09:01:33.355141+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.355141+0000","last_undegraded":"2024-04-25T09:01:33.355141+0000","last_fullsized":"2024-04-25T09:01:33.355141+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.369410+0000","last_change":"2024-04-25T09:01:33.367167+0000","last_active":"2024-04-25T09:01:33.369410+0000","last_peered":"2024-04-25T09:01:33.369410+0000","last_clean":"2024-04-25T09:01:33.369410+0000","last_became_active":"2024-04-25T09:01:33.365541+0000","last_became_peered":"2024-04-25T09:01:33.365541+0000","last_unstale":"2024-04-25T09:01:33.369410+0000","last_undegraded":"2024-04-25T09:01:33.369410+0000","last_fullsized":"2024-04-25T09:01:33.369410+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:22.335074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.357332+0000","last_change":"2024-04-25T09:01:33.357332+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.357332+0000","last_undegraded":"2024-04-25T09:01:33.357332+0000","last_fullsized":"2024-04-25T09:01:33.357332+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.363381+0000","last_change":"2024-04-25T09:01:33.361825+0000","last_active":"2024-04-25T09:01:33.363381+0000","last_peered":"2024-04-25T09:01:33.363381+0000","last_clean":"2024-04-25T09:01:33.363381+0000","last_became_active":"2024-04-25T09:01:33.361544+0000","last_became_peered":"2024-04-25T09:01:33.361544+0000","last_unstale":"2024-04-25T09:01:33.363381+0000","last_undegraded":"2024-04-25T09:01:33.363381+0000","last_fullsized":"2024-04-25T09:01:33.363381+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:19.818176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.354331+0000","last_change":"2024-04-25T09:01:33.354331+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.354331+0000","last_undegraded":"2024-04-25T09:01:33.354331+0000","last_fullsized":"2024-04-25T09:01:33.354331+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.369815+0000","last_change":"2024-04-25T09:01:33.365387+0000","last_active":"2024-04-25T09:01:33.369815+0000","last_peered":"2024-04-25T09:01:33.369815+0000","last_clean":"2024-04-25T09:01:33.369815+0000","last_became_active":"2024-04-25T09:01:33.365088+0000","last_became_peered":"2024-04-25T09:01:33.365088+0000","last_unstale":"2024-04-25T09:01:33.369815+0000","last_undegraded":"2024-04-25T09:01:33.369815+0000","last_fullsized":"2024-04-25T09:01:33.369815+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:34.749007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.362314+0000","last_change":"2024-04-25T09:01:33.362314+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.362314+0000","last_undegraded":"2024-04-25T09:01:33.362314+0000","last_fullsized":"2024-04-25T09:01:33.362314+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.368520+0000","last_change":"2024-04-25T09:01:33.367517+0000","last_active":"2024-04-25T09:01:33.368520+0000","last_peered":"2024-04-25T09:01:33.368520+0000","last_clean":"2024-04-25T09:01:33.368520+0000","last_became_active":"2024-04-25T09:01:33.367216+0000","last_became_peered":"2024-04-25T09:01:33.367216+0000","last_unstale":"2024-04-25T09:01:33.368520+0000","last_undegraded":"2024-04-25T09:01:33.368520+0000","last_fullsized":"2024-04-25T09:01:33.368520+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:42.045982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.357735+0000","last_change":"2024-04-25T09:01:33.357735+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.357735+0000","last_undegraded":"2024-04-25T09:01:33.357735+0000","last_fullsized":"2024-04-25T09:01:33.357735+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.367757+0000","last_change":"2024-04-25T09:01:33.366291+0000","last_active":"2024-04-25T09:01:33.367757+0000","last_peered":"2024-04-25T09:01:33.367757+0000","last_clean":"2024-04-25T09:01:33.367757+0000","last_became_active":"2024-04-25T09:01:33.365670+0000","last_became_peered":"2024-04-25T09:01:33.365670+0000","last_unstale":"2024-04-25T09:01:33.367757+0000","last_undegraded":"2024-04-25T09:01:33.367757+0000","last_fullsized":"2024-04-25T09:01:33.367757+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:35.172322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.355949+0000","last_change":"2024-04-25T09:01:33.355949+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.355949+0000","last_undegraded":"2024-04-25T09:01:33.355949+0000","last_fullsized":"2024-04-25T09:01:33.355949+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.368724+0000","last_change":"2024-04-25T09:01:33.363784+0000","last_active":"2024-04-25T09:01:33.368724+0000","last_peered":"2024-04-25T09:01:33.368724+0000","last_clean":"2024-04-25T09:01:33.368724+0000","last_became_active":"2024-04-25T09:01:33.363279+0000","last_became_peered":"2024-04-25T09:01:33.363279+0000","last_unstale":"2024-04-25T09:01:33.368724+0000","last_undegraded":"2024-04-25T09:01:33.368724+0000","last_fullsized":"2024-04-25T09:01:33.368724+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:38:15.504041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.354732+0000","last_change":"2024-04-25T09:01:33.354732+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.354732+0000","last_undegraded":"2024-04-25T09:01:33.354732+0000","last_fullsized":"2024-04-25T09:01:33.354732+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.368515+0000","last_change":"2024-04-25T09:01:33.365837+0000","last_active":"2024-04-25T09:01:33.368515+0000","last_peered":"2024-04-25T09:01:33.368515+0000","last_clean":"2024-04-25T09:01:33.368515+0000","last_became_active":"2024-04-25T09:01:33.362985+0000","last_became_peered":"2024-04-25T09:01:33.362985+0000","last_unstale":"2024-04-25T09:01:33.368515+0000","last_undegraded":"2024-04-25T09:01:33.368515+0000","last_fullsized":"2024-04-25T09:01:33.368515+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:02.701373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.359064+0000","last_change":"2024-04-25T09:01:33.359064+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.359064+0000","last_undegraded":"2024-04-25T09:01:33.359064+0000","last_fullsized":"2024-04-25T09:01:33.359064+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.367829+0000","last_change":"2024-04-25T09:01:33.367032+0000","last_active":"2024-04-25T09:01:33.367829+0000","last_peered":"2024-04-25T09:01:33.367829+0000","last_clean":"2024-04-25T09:01:33.367829+0000","last_became_active":"2024-04-25T09:01:33.366662+0000","last_became_peered":"2024-04-25T09:01:33.366662+0000","last_unstale":"2024-04-25T09:01:33.367829+0000","last_undegraded":"2024-04-25T09:01:33.367829+0000","last_fullsized":"2024-04-25T09:01:33.367829+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:13.219195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.352882+0000","last_change":"2024-04-25T09:01:33.352882+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.352882+0000","last_undegraded":"2024-04-25T09:01:33.352882+0000","last_fullsized":"2024-04-25T09:01:33.352882+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.369386+0000","last_change":"2024-04-25T09:01:33.366823+0000","last_active":"2024-04-25T09:01:33.369386+0000","last_peered":"2024-04-25T09:01:33.369386+0000","last_clean":"2024-04-25T09:01:33.369386+0000","last_became_active":"2024-04-25T09:01:33.366050+0000","last_became_peered":"2024-04-25T09:01:33.366050+0000","last_unstale":"2024-04-25T09:01:33.369386+0000","last_undegraded":"2024-04-25T09:01:33.369386+0000","last_fullsized":"2024-04-25T09:01:33.369386+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:49.016771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.362269+0000","last_change":"2024-04-25T09:01:33.362269+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.362269+0000","last_undegraded":"2024-04-25T09:01:33.362269+0000","last_fullsized":"2024-04-25T09:01:33.362269+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.369665+0000","last_change":"2024-04-25T09:01:33.369232+0000","last_active":"2024-04-25T09:01:33.369665+0000","last_peered":"2024-04-25T09:01:33.369665+0000","last_clean":"2024-04-25T09:01:33.369665+0000","last_became_active":"2024-04-25T09:01:33.368905+0000","last_became_peered":"2024-04-25T09:01:33.368905+0000","last_unstale":"2024-04-25T09:01:33.369665+0000","last_undegraded":"2024-04-25T09:01:33.369665+0000","last_fullsized":"2024-04-25T09:01:33.369665+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:41.915410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.355227+0000","last_change":"2024-04-25T09:01:33.355227+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.355227+0000","last_undegraded":"2024-04-25T09:01:33.355227+0000","last_fullsized":"2024-04-25T09:01:33.355227+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.364958+0000","last_change":"2024-04-25T09:01:33.363919+0000","last_active":"2024-04-25T09:01:33.364958+0000","last_peered":"2024-04-25T09:01:33.364958+0000","last_clean":"2024-04-25T09:01:33.364958+0000","last_became_active":"2024-04-25T09:01:33.363587+0000","last_became_peered":"2024-04-25T09:01:33.363587+0000","last_unstale":"2024-04-25T09:01:33.364958+0000","last_undegraded":"2024-04-25T09:01:33.364958+0000","last_fullsized":"2024-04-25T09:01:33.364958+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:39.790922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.355533+0000","last_change":"2024-04-25T09:01:33.355533+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.355533+0000","last_undegraded":"2024-04-25T09:01:33.355533+0000","last_fullsized":"2024-04-25T09:01:33.355533+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.367244+0000","last_change":"2024-04-25T09:01:33.364976+0000","last_active":"2024-04-25T09:01:33.367244+0000","last_peered":"2024-04-25T09:01:33.367244+0000","last_clean":"2024-04-25T09:01:33.367244+0000","last_became_active":"2024-04-25T09:01:33.363617+0000","last_became_peered":"2024-04-25T09:01:33.363617+0000","last_unstale":"2024-04-25T09:01:33.367244+0000","last_undegraded":"2024-04-25T09:01:33.367244+0000","last_fullsized":"2024-04-25T09:01:33.367244+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:13.219195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.353243+0000","last_change":"2024-04-25T09:01:33.353243+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.353243+0000","last_undegraded":"2024-04-25T09:01:33.353243+0000","last_fullsized":"2024-04-25T09:01:33.353243+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.371512+0000","last_change":"2024-04-25T09:01:33.369475+0000","last_active":"2024-04-25T09:01:33.371512+0000","last_peered":"2024-04-25T09:01:33.371512+0000","last_clean":"2024-04-25T09:01:33.371512+0000","last_became_active":"2024-04-25T09:01:33.368210+0000","last_became_peered":"2024-04-25T09:01:33.368210+0000","last_unstale":"2024-04-25T09:01:33.371512+0000","last_undegraded":"2024-04-25T09:01:33.371512+0000","last_fullsized":"2024-04-25T09:01:33.371512+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:26.679025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.354197+0000","last_change":"2024-04-25T09:01:33.354197+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.354197+0000","last_undegraded":"2024-04-25T09:01:33.354197+0000","last_fullsized":"2024-04-25T09:01:33.354197+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.364947+0000","last_change":"2024-04-25T09:01:33.363025+0000","last_active":"2024-04-25T09:01:33.364947+0000","last_peered":"2024-04-25T09:01:33.364947+0000","last_clean":"2024-04-25T09:01:33.364947+0000","last_became_active":"2024-04-25T09:01:33.362724+0000","last_became_peered":"2024-04-25T09:01:33.362724+0000","last_unstale":"2024-04-25T09:01:33.364947+0000","last_undegraded":"2024-04-25T09:01:33.364947+0000","last_fullsized":"2024-04-25T09:01:33.364947+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:41.464204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.353721+0000","last_change":"2024-04-25T09:01:33.353721+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.353721+0000","last_undegraded":"2024-04-25T09:01:33.353721+0000","last_fullsized":"2024-04-25T09:01:33.353721+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.366031+0000","last_change":"2024-04-25T09:01:33.365366+0000","last_active":"2024-04-25T09:01:33.366031+0000","last_peered":"2024-04-25T09:01:33.366031+0000","last_clean":"2024-04-25T09:01:33.366031+0000","last_became_active":"2024-04-25T09:01:33.365090+0000","last_became_peered":"2024-04-25T09:01:33.365090+0000","last_unstale":"2024-04-25T09:01:33.366031+0000","last_undegraded":"2024-04-25T09:01:33.366031+0000","last_fullsized":"2024-04-25T09:01:33.366031+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:35.172322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.361446+0000","last_change":"2024-04-25T09:01:33.361446+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.361446+0000","last_undegraded":"2024-04-25T09:01:33.361446+0000","last_fullsized":"2024-04-25T09:01:33.361446+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.371551+0000","last_change":"2024-04-25T09:01:33.370545+0000","last_active":"2024-04-25T09:01:33.371551+0000","last_peered":"2024-04-25T09:01:33.371551+0000","last_clean":"2024-04-25T09:01:33.371551+0000","last_became_active":"2024-04-25T09:01:33.370255+0000","last_became_peered":"2024-04-25T09:01:33.370255+0000","last_unstale":"2024-04-25T09:01:33.371551+0000","last_undegraded":"2024-04-25T09:01:33.371551+0000","last_fullsized":"2024-04-25T09:01:33.371551+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:10.849550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356865+0000","last_change":"2024-04-25T09:01:33.356865+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356865+0000","last_undegraded":"2024-04-25T09:01:33.356865+0000","last_fullsized":"2024-04-25T09:01:33.356865+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.365141+0000","last_change":"2024-04-25T09:01:33.364409+0000","last_active":"2024-04-25T09:01:33.365141+0000","last_peered":"2024-04-25T09:01:33.365141+0000","last_clean":"2024-04-25T09:01:33.365141+0000","last_became_active":"2024-04-25T09:01:33.364089+0000","last_became_peered":"2024-04-25T09:01:33.364089+0000","last_unstale":"2024-04-25T09:01:33.365141+0000","last_undegraded":"2024-04-25T09:01:33.365141+0000","last_fullsized":"2024-04-25T09:01:33.365141+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:58.745035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.355038+0000","last_change":"2024-04-25T09:01:33.355038+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.355038+0000","last_undegraded":"2024-04-25T09:01:33.355038+0000","last_fullsized":"2024-04-25T09:01:33.355038+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.367852+0000","last_change":"2024-04-25T09:01:33.366607+0000","last_active":"2024-04-25T09:01:33.367852+0000","last_peered":"2024-04-25T09:01:33.367852+0000","last_clean":"2024-04-25T09:01:33.367852+0000","last_became_active":"2024-04-25T09:01:33.366048+0000","last_became_peered":"2024-04-25T09:01:33.366048+0000","last_unstale":"2024-04-25T09:01:33.367852+0000","last_undegraded":"2024-04-25T09:01:33.367852+0000","last_fullsized":"2024-04-25T09:01:33.367852+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:07.729885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.353719+0000","last_change":"2024-04-25T09:01:33.353719+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.353719+0000","last_undegraded":"2024-04-25T09:01:33.353719+0000","last_fullsized":"2024-04-25T09:01:33.353719+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.369413+0000","last_change":"2024-04-25T09:01:33.368065+0000","last_active":"2024-04-25T09:01:33.369413+0000","last_peered":"2024-04-25T09:01:33.369413+0000","last_clean":"2024-04-25T09:01:33.369413+0000","last_became_active":"2024-04-25T09:01:33.367711+0000","last_became_peered":"2024-04-25T09:01:33.367711+0000","last_unstale":"2024-04-25T09:01:33.369413+0000","last_undegraded":"2024-04-25T09:01:33.369413+0000","last_fullsized":"2024-04-25T09:01:33.369413+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:39.790922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.358486+0000","last_change":"2024-04-25T09:01:33.358486+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.358486+0000","last_undegraded":"2024-04-25T09:01:33.358486+0000","last_fullsized":"2024-04-25T09:01:33.358486+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.366205+0000","last_change":"2024-04-25T09:01:33.365212+0000","last_active":"2024-04-25T09:01:33.366205+0000","last_peered":"2024-04-25T09:01:33.366205+0000","last_clean":"2024-04-25T09:01:33.366205+0000","last_became_active":"2024-04-25T09:01:33.364945+0000","last_became_peered":"2024-04-25T09:01:33.364945+0000","last_unstale":"2024-04-25T09:01:33.366205+0000","last_undegraded":"2024-04-25T09:01:33.366205+0000","last_fullsized":"2024-04-25T09:01:33.366205+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:15.517604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.352679+0000","last_change":"2024-04-25T09:01:33.352679+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.352679+0000","last_undegraded":"2024-04-25T09:01:33.352679+0000","last_fullsized":"2024-04-25T09:01:33.352679+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.370501+0000","last_change":"2024-04-25T09:01:33.367854+0000","last_active":"2024-04-25T09:01:33.370501+0000","last_peered":"2024-04-25T09:01:33.370501+0000","last_clean":"2024-04-25T09:01:33.370501+0000","last_became_active":"2024-04-25T09:01:33.366038+0000","last_became_peered":"2024-04-25T09:01:33.366038+0000","last_unstale":"2024-04-25T09:01:33.370501+0000","last_undegraded":"2024-04-25T09:01:33.370501+0000","last_fullsized":"2024-04-25T09:01:33.370501+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:55.938665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.356269+0000","last_change":"2024-04-25T09:01:33.356269+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.356269+0000","last_undegraded":"2024-04-25T09:01:33.356269+0000","last_fullsized":"2024-04-25T09:01:33.356269+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.370382+0000","last_change":"2024-04-25T09:01:33.369961+0000","last_active":"2024-04-25T09:01:33.370382+0000","last_peered":"2024-04-25T09:01:33.370382+0000","last_clean":"2024-04-25T09:01:33.370382+0000","last_became_active":"2024-04-25T09:01:33.369606+0000","last_became_peered":"2024-04-25T09:01:33.369606+0000","last_unstale":"2024-04-25T09:01:33.370382+0000","last_undegraded":"2024-04-25T09:01:33.370382+0000","last_fullsized":"2024-04-25T09:01:33.370382+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:10.849550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.353264+0000","last_change":"2024-04-25T09:01:33.353264+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.353264+0000","last_undegraded":"2024-04-25T09:01:33.353264+0000","last_fullsized":"2024-04-25T09:01:33.353264+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.369396+0000","last_change":"2024-04-25T09:01:33.364868+0000","last_active":"2024-04-25T09:01:33.369396+0000","last_peered":"2024-04-25T09:01:33.369396+0000","last_clean":"2024-04-25T09:01:33.369396+0000","last_became_active":"2024-04-25T09:01:33.364604+0000","last_became_peered":"2024-04-25T09:01:33.364604+0000","last_unstale":"2024-04-25T09:01:33.369396+0000","last_undegraded":"2024-04-25T09:01:33.369396+0000","last_fullsized":"2024-04-25T09:01:33.369396+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:29.149495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.361011+0000","last_change":"2024-04-25T09:01:33.361011+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.361011+0000","last_undegraded":"2024-04-25T09:01:33.361011+0000","last_fullsized":"2024-04-25T09:01:33.361011+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.368490+0000","last_change":"2024-04-25T09:01:33.364144+0000","last_active":"2024-04-25T09:01:33.368490+0000","last_peered":"2024-04-25T09:01:33.368490+0000","last_clean":"2024-04-25T09:01:33.368490+0000","last_became_active":"2024-04-25T09:01:33.363517+0000","last_became_peered":"2024-04-25T09:01:33.363517+0000","last_unstale":"2024-04-25T09:01:33.368490+0000","last_undegraded":"2024-04-25T09:01:33.368490+0000","last_fullsized":"2024-04-25T09:01:33.368490+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:04.891189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":100,"state":"creating+peering","last_fresh":"2024-04-25T09:01:33.352757+0000","last_change":"2024-04-25T09:01:33.352757+0000","last_active":"2024-04-25T09:01:33.340929+0000","last_peered":"2024-04-25T09:01:33.340929+0000","last_clean":"2024-04-25T09:01:33.340929+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:01:33.352757+0000","last_undegraded":"2024-04-25T09:01:33.352757+0000","last_fullsized":"2024-04-25T09:01:33.352757+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.368385+0000","last_change":"2024-04-25T09:01:33.367042+0000","last_active":"2024-04-25T09:01:33.368385+0000","last_peered":"2024-04-25T09:01:33.368385+0000","last_clean":"2024-04-25T09:01:33.368385+0000","last_became_active":"2024-04-25T09:01:33.366746+0000","last_became_peered":"2024-04-25T09:01:33.366746+0000","last_unstale":"2024-04-25T09:01:33.368385+0000","last_undegraded":"2024-04-25T09:01:33.368385+0000","last_fullsized":"2024-04-25T09:01:33.368385+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:34.749007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":317,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-25T09:01:33.357167+0000","last_change":"2024-04-25T08:54:00.818044+0000","last_active":"2024-04-25T09:01:33.357167+0000","last_peered":"2024-04-25T09:01:33.357167+0000","last_clean":"2024-04-25T09:01:33.357167+0000","last_became_active":"2024-04-25T08:54:00.817098+0000","last_became_peered":"2024-04-25T08:54:00.817098+0000","last_unstale":"2024-04-25T09:01:33.357167+0000","last_undegraded":"2024-04-25T09:01:33.357167+0000","last_fullsized":"2024-04-25T09:01:33.357167+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_clean_scrub_stamp":"2024-04-25T08:53:59.739677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:48.129513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574941,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55168,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316672,"statfs":{"total":96636764160,"available":96580272128,"internally_reserved":0,"allocated":1818624,"data_stored":1118350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574942,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":80956,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94290884,"statfs":{"total":96636764160,"available":96553865216,"internally_reserved":0,"allocated":1224704,"data_stored":527982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":81656262},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574944,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50436,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321404,"statfs":{"total":96636764160,"available":96585117696,"internally_reserved":0,"allocated":1236992,"data_stored":534257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574942,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64328,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94307512,"statfs":{"total":96636764160,"available":96570892288,"internally_reserved":0,"allocated":1236992,"data_stored":534257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574942,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68488,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303352,"statfs":{"total":96636764160,"available":96566632448,"internally_reserved":0,"allocated":1236992,"data_stored":534257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574942,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58756,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313084,"statfs":{"total":96636764160,"available":96576598016,"internally_reserved":0,"allocated":1236992,"data_stored":534257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76804,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94295036,"statfs":{"total":96636764160,"available":96558116864,"internally_reserved":0,"allocated":1236992,"data_stored":534257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574943,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59340,"kb_used_data":1788,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312500,"statfs":{"total":96636764160,"available":96576000000,"internally_reserved":0,"allocated":1830912,"data_stored":1124625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:01:41.788 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:01:41.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.913+0000 7f156c5c4640 1 Processor -- start 2024-04-25T09:01:41.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.913+0000 7f156c5c4640 1 -- start start 2024-04-25T09:01:41.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.914+0000 7f156c5c4640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1564161390 0x7f15640b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:41.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.914+0000 7f156c5c4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f15640d02a0 0x7f15640d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:41.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.914+0000 7f156c5c4640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15640b4c30 con 0x7f15640ce5e0 2024-04-25T09:01:41.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.914+0000 7f156c5c4640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15640d1910 con 0x7f15640d02a0 2024-04-25T09:01:41.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.914+0000 7f156c5c4640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15640cf2c0 con 0x7f1564161390 2024-04-25T09:01:41.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.914+0000 7f1569b5f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f15640d02a0 0x7f15640d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:41.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.914+0000 7f156ab61640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f15640ce5e0 0x7f15640cea70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58986/0 (socket says 172.21.15.43:58986) 2024-04-25T09:01:41.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.914+0000 7f156ab61640 1 -- 172.21.15.43:0/161523688 learned_addr learned my addr 172.21.15.43:0/161523688 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:41.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.914+0000 7f156a360640 1 --2- 172.21.15.43:0/161523688 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1564161390 0x7f15640b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:41.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.914+0000 7f1569b5f640 1 -- 172.21.15.43:0/161523688 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1564161390 msgr2=0x7f15640b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:41.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.914+0000 7f1569b5f640 1 --2- 172.21.15.43:0/161523688 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1564161390 0x7f15640b6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.914+0000 7f1569b5f640 1 -- 172.21.15.43:0/161523688 >> v1:172.21.15.43:6789/0 conn(0x7f15640ce5e0 legacy=0x7f15640cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:41.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.914+0000 7f156935e640 1 -- 172.21.15.43:0/161523688 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 684200873 0 0) 0x7f155c001490 con 0x7f15640ce5e0 2024-04-25T09:01:41.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.914+0000 7f1569b5f640 1 -- 172.21.15.43:0/161523688 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15640cf5a0 con 0x7f15640d02a0 2024-04-25T09:01:41.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.914+0000 7f156a360640 1 --2- 172.21.15.43:0/161523688 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1564161390 0x7f15640b6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:41.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.915+0000 7f1569b5f640 1 --2- 172.21.15.43:0/161523688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15640d02a0 0x7f15640d0720 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f15540097a0 tx=0x7f155402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2409e28179ef398 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:41.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.915+0000 7f156935e640 1 -- 172.21.15.43:0/161523688 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1554037070 con 0x7f15640d02a0 2024-04-25T09:01:41.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.915+0000 7f156935e640 1 -- 172.21.15.43:0/161523688 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1554032440 con 0x7f15640d02a0 2024-04-25T09:01:41.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.915+0000 7f156935e640 1 -- 172.21.15.43:0/161523688 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1554032810 con 0x7f15640d02a0 2024-04-25T09:01:41.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.916+0000 7f156c5c4640 1 -- 172.21.15.43:0/161523688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15640d02a0 msgr2=0x7f15640d0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:41.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.916+0000 7f156c5c4640 1 --2- 172.21.15.43:0/161523688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15640d02a0 0x7f15640d0720 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f15540097a0 tx=0x7f155402a180 comp rx=0 tx=0).stop 2024-04-25T09:01:41.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.916+0000 7f156c5c4640 1 -- 172.21.15.43:0/161523688 shutdown_connections 2024-04-25T09:01:41.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.916+0000 7f156c5c4640 1 --2- 172.21.15.43:0/161523688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15640d02a0 0x7f15640d0720 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.916+0000 7f156c5c4640 1 --2- 172.21.15.43:0/161523688 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1564161390 0x7f15640b6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.916+0000 7f156c5c4640 1 -- 172.21.15.43:0/161523688 >> 172.21.15.43:0/161523688 conn(0x7f156409cd20 msgr2=0x7f15640d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:41.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.916+0000 7f156c5c4640 1 -- 172.21.15.43:0/161523688 shutdown_connections 2024-04-25T09:01:41.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.916+0000 7f156c5c4640 1 -- 172.21.15.43:0/161523688 wait complete. 2024-04-25T09:01:41.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.917+0000 7f156c5c4640 1 Processor -- start 2024-04-25T09:01:41.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.917+0000 7f156c5c4640 1 -- start start 2024-04-25T09:01:41.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.918+0000 7f156c5c4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15640ce5e0 0x7f15641f44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:41.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.918+0000 7f156c5c4640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f15640d02a0 0x7f15641f0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:41.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.918+0000 7f156c5c4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1564161390 0x7f1564159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:41.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.918+0000 7f156c5c4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1564161770 con 0x7f15640ce5e0 2024-04-25T09:01:41.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.918+0000 7f156c5c4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15640d15a0 con 0x7f1564161390 2024-04-25T09:01:41.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.918+0000 7f156c5c4640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15640d1da0 con 0x7f15640d02a0 2024-04-25T09:01:41.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.918+0000 7f156a360640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15640ce5e0 0x7f15641f44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:41.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.918+0000 7f156a360640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15640ce5e0 0x7f15641f44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42518/0 (socket says 172.21.15.43:42518) 2024-04-25T09:01:41.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.918+0000 7f156a360640 1 -- 172.21.15.43:0/51824727 learned_addr learned my addr 172.21.15.43:0/51824727 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:41.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.918+0000 7f1569b5f640 1 --2- 172.21.15.43:0/51824727 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f15640d02a0 0x7f15641f0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:41.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.919+0000 7f156a360640 1 -- 172.21.15.43:0/51824727 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f15640d02a0 msgr2=0x7f15641f0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:41.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.919+0000 7f156a360640 1 --2- 172.21.15.43:0/51824727 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f15640d02a0 0x7f15641f0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.919+0000 7f156ab61640 1 --2- 172.21.15.43:0/51824727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1564161390 0x7f1564159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:41.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.919+0000 7f156a360640 1 -- 172.21.15.43:0/51824727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1564161390 msgr2=0x7f1564159700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:41.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.919+0000 7f156a360640 1 --2- 172.21.15.43:0/51824727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1564161390 0x7f1564159700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:41.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.919+0000 7f156a360640 1 -- 172.21.15.43:0/51824727 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1564157570 con 0x7f15640ce5e0 2024-04-25T09:01:41.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.919+0000 7f156ab61640 1 --2- 172.21.15.43:0/51824727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1564161390 0x7f1564159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:41.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.919+0000 7f156a360640 1 --2- 172.21.15.43:0/51824727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15640ce5e0 0x7f15641f44b0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f156000c820 tx=0x7f156000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:41.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.919+0000 7f155affd640 1 -- 172.21.15.43:0/51824727 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1560015070 con 0x7f15640ce5e0 2024-04-25T09:01:41.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.919+0000 7f155affd640 1 -- 172.21.15.43:0/51824727 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1560007a30 con 0x7f15640ce5e0 2024-04-25T09:01:41.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.919+0000 7f156c5c4640 1 -- 172.21.15.43:0/51824727 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15641f49f0 con 0x7f15640ce5e0 2024-04-25T09:01:41.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.919+0000 7f155affd640 1 -- 172.21.15.43:0/51824727 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1560007de0 con 0x7f15640ce5e0 2024-04-25T09:01:41.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.919+0000 7f156c5c4640 1 -- 172.21.15.43:0/51824727 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f156415d5b0 con 0x7f15640ce5e0 2024-04-25T09:01:41.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.920+0000 7f156c5c4640 1 -- 172.21.15.43:0/51824727 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15640680f0 con 0x7f15640ce5e0 2024-04-25T09:01:41.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.920+0000 7f155affd640 1 -- 172.21.15.43:0/51824727 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1560018070 con 0x7f15640ce5e0 2024-04-25T09:01:41.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.921+0000 7f155affd640 1 --2- 172.21.15.43:0/51824727 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f154806b4d0 0x7f154806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:41.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.921+0000 7f155affd640 1 -- 172.21.15.43:0/51824727 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f156008a020 con 0x7f15640ce5e0 2024-04-25T09:01:41.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.921+0000 7f1569b5f640 1 --2- 172.21.15.43:0/51824727 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f154806b4d0 0x7f154806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:41.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.922+0000 7f1569b5f640 1 --2- 172.21.15.43:0/51824727 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f154806b4d0 0x7f154806d990 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f1554034000 tx=0x7f155402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:41.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:41.925+0000 7f155affd640 1 -- 172.21.15.43:0/51824727 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1560058f30 con 0x7f15640ce5e0 2024-04-25T09:01:42.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.101+0000 7f156c5c4640 1 -- 172.21.15.43:0/51824727 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f156415d3b0 con 0x7f154806b4d0 2024-04-25T09:01:42.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.107+0000 7f155affd640 1 -- 172.21.15.43:0/51824727 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252423 (secure 0 0 0) 0x7f156415d3b0 con 0x7f154806b4d0 2024-04-25T09:01:42.099 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:01:42.100 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-25T09:01:42.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.110+0000 7f156c5c4640 1 -- 172.21.15.43:0/51824727 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f154806b4d0 msgr2=0x7f154806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:42.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.110+0000 7f156c5c4640 1 --2- 172.21.15.43:0/51824727 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f154806b4d0 0x7f154806d990 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f1554034000 tx=0x7f155402e000 comp rx=0 tx=0).stop 2024-04-25T09:01:42.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.110+0000 7f156c5c4640 1 -- 172.21.15.43:0/51824727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15640ce5e0 msgr2=0x7f15641f44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:42.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.110+0000 7f156c5c4640 1 --2- 172.21.15.43:0/51824727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15640ce5e0 0x7f15641f44b0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f156000c820 tx=0x7f156000cce0 comp rx=0 tx=0).stop 2024-04-25T09:01:42.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.110+0000 7f156c5c4640 1 -- 172.21.15.43:0/51824727 shutdown_connections 2024-04-25T09:01:42.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.110+0000 7f156c5c4640 1 --2- 172.21.15.43:0/51824727 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f154806b4d0 0x7f154806d990 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:42.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.110+0000 7f156c5c4640 1 --2- 172.21.15.43:0/51824727 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1564161390 0x7f1564159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:42.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.110+0000 7f156c5c4640 1 --2- 172.21.15.43:0/51824727 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f15640d02a0 0x7f15641f0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:42.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.110+0000 7f156c5c4640 1 --2- 172.21.15.43:0/51824727 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15640ce5e0 0x7f15641f44b0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:42.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.110+0000 7f156c5c4640 1 -- 172.21.15.43:0/51824727 >> 172.21.15.43:0/51824727 conn(0x7f156409cd20 msgr2=0x7f15640b7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:42.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.110+0000 7f156c5c4640 1 -- 172.21.15.43:0/51824727 shutdown_connections 2024-04-25T09:01:42.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.110+0000 7f156c5c4640 1 -- 172.21.15.43:0/51824727 wait complete. 2024-04-25T09:01:42.111 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-04-25T09:01:41.798365+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":518716,"kb_used_data":11080,"kb_used_omap":12,"kb_used_meta":507507,"kb_avail":754456004,"statfs":{"total":773094113280,"available":772562948096,"internally_reserved":0,"allocated":11345920,"data_stored":5564212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":519687729},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002386"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.407355+0000","last_change":"2024-04-25T09:01:34.364927+0000","last_active":"2024-04-25T09:01:36.407355+0000","last_peered":"2024-04-25T09:01:36.407355+0000","last_clean":"2024-04-25T09:01:36.407355+0000","last_became_active":"2024-04-25T09:01:34.364616+0000","last_became_peered":"2024-04-25T09:01:34.364616+0000","last_unstale":"2024-04-25T09:01:36.407355+0000","last_undegraded":"2024-04-25T09:01:36.407355+0000","last_fullsized":"2024-04-25T09:01:36.407355+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:40.798205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:35.365569+0000","last_change":"2024-04-25T09:01:34.372054+0000","last_active":"2024-04-25T09:01:35.365569+0000","last_peered":"2024-04-25T09:01:35.365569+0000","last_clean":"2024-04-25T09:01:35.365569+0000","last_became_active":"2024-04-25T09:01:34.371150+0000","last_became_peered":"2024-04-25T09:01:34.371150+0000","last_unstale":"2024-04-25T09:01:35.365569+0000","last_undegraded":"2024-04-25T09:01:35.365569+0000","last_fullsized":"2024-04-25T09:01:35.365569+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:15.749948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.407074+0000","last_change":"2024-04-25T09:01:34.364988+0000","last_active":"2024-04-25T09:01:36.407074+0000","last_peered":"2024-04-25T09:01:36.407074+0000","last_clean":"2024-04-25T09:01:36.407074+0000","last_became_active":"2024-04-25T09:01:34.364550+0000","last_became_peered":"2024-04-25T09:01:34.364550+0000","last_unstale":"2024-04-25T09:01:36.407074+0000","last_undegraded":"2024-04-25T09:01:36.407074+0000","last_fullsized":"2024-04-25T09:01:36.407074+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:44.594206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.407926+0000","last_change":"2024-04-25T09:01:34.372880+0000","last_active":"2024-04-25T09:01:36.407926+0000","last_peered":"2024-04-25T09:01:36.407926+0000","last_clean":"2024-04-25T09:01:36.407926+0000","last_became_active":"2024-04-25T09:01:34.372571+0000","last_became_peered":"2024-04-25T09:01:34.372571+0000","last_unstale":"2024-04-25T09:01:36.407926+0000","last_undegraded":"2024-04-25T09:01:36.407926+0000","last_fullsized":"2024-04-25T09:01:36.407926+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:15.749948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.406669+0000","last_change":"2024-04-25T09:01:34.364921+0000","last_active":"2024-04-25T09:01:36.406669+0000","last_peered":"2024-04-25T09:01:36.406669+0000","last_clean":"2024-04-25T09:01:36.406669+0000","last_became_active":"2024-04-25T09:01:34.364640+0000","last_became_peered":"2024-04-25T09:01:34.364640+0000","last_unstale":"2024-04-25T09:01:36.406669+0000","last_undegraded":"2024-04-25T09:01:36.406669+0000","last_fullsized":"2024-04-25T09:01:36.406669+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:13.120312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.148649+0000","last_change":"2024-04-25T09:01:34.357872+0000","last_active":"2024-04-25T09:01:36.148649+0000","last_peered":"2024-04-25T09:01:36.148649+0000","last_clean":"2024-04-25T09:01:36.148649+0000","last_became_active":"2024-04-25T09:01:34.357607+0000","last_became_peered":"2024-04-25T09:01:34.357607+0000","last_unstale":"2024-04-25T09:01:36.148649+0000","last_undegraded":"2024-04-25T09:01:36.148649+0000","last_fullsized":"2024-04-25T09:01:36.148649+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:03.767726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.148914+0000","last_change":"2024-04-25T09:01:34.356124+0000","last_active":"2024-04-25T09:01:36.148914+0000","last_peered":"2024-04-25T09:01:36.148914+0000","last_clean":"2024-04-25T09:01:36.148914+0000","last_became_active":"2024-04-25T09:01:34.355802+0000","last_became_peered":"2024-04-25T09:01:34.355802+0000","last_unstale":"2024-04-25T09:01:36.148914+0000","last_undegraded":"2024-04-25T09:01:36.148914+0000","last_fullsized":"2024-04-25T09:01:36.148914+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:21.718104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:35.365386+0000","last_change":"2024-04-25T09:01:34.369783+0000","last_active":"2024-04-25T09:01:35.365386+0000","last_peered":"2024-04-25T09:01:35.365386+0000","last_clean":"2024-04-25T09:01:35.365386+0000","last_became_active":"2024-04-25T09:01:34.369498+0000","last_became_peered":"2024-04-25T09:01:34.369498+0000","last_unstale":"2024-04-25T09:01:35.365386+0000","last_undegraded":"2024-04-25T09:01:35.365386+0000","last_fullsized":"2024-04-25T09:01:35.365386+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:16.315887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:35.366464+0000","last_change":"2024-04-25T09:01:34.377116+0000","last_active":"2024-04-25T09:01:35.366464+0000","last_peered":"2024-04-25T09:01:35.366464+0000","last_clean":"2024-04-25T09:01:35.366464+0000","last_became_active":"2024-04-25T09:01:34.376780+0000","last_became_peered":"2024-04-25T09:01:34.376780+0000","last_unstale":"2024-04-25T09:01:35.366464+0000","last_undegraded":"2024-04-25T09:01:35.366464+0000","last_fullsized":"2024-04-25T09:01:35.366464+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:55.568735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.406802+0000","last_change":"2024-04-25T09:01:34.364839+0000","last_active":"2024-04-25T09:01:36.406802+0000","last_peered":"2024-04-25T09:01:36.406802+0000","last_clean":"2024-04-25T09:01:36.406802+0000","last_became_active":"2024-04-25T09:01:34.364541+0000","last_became_peered":"2024-04-25T09:01:34.364541+0000","last_unstale":"2024-04-25T09:01:36.406802+0000","last_undegraded":"2024-04-25T09:01:36.406802+0000","last_fullsized":"2024-04-25T09:01:36.406802+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:27.774593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.148800+0000","last_change":"2024-04-25T09:01:34.371034+0000","last_active":"2024-04-25T09:01:36.148800+0000","last_peered":"2024-04-25T09:01:36.148800+0000","last_clean":"2024-04-25T09:01:36.148800+0000","last_became_active":"2024-04-25T09:01:34.370258+0000","last_became_peered":"2024-04-25T09:01:34.370258+0000","last_unstale":"2024-04-25T09:01:36.148800+0000","last_undegraded":"2024-04-25T09:01:36.148800+0000","last_fullsized":"2024-04-25T09:01:36.148800+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:27.372745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:35.366994+0000","last_change":"2024-04-25T09:01:34.364650+0000","last_active":"2024-04-25T09:01:35.366994+0000","last_peered":"2024-04-25T09:01:35.366994+0000","last_clean":"2024-04-25T09:01:35.366994+0000","last_became_active":"2024-04-25T09:01:34.364373+0000","last_became_peered":"2024-04-25T09:01:34.364373+0000","last_unstale":"2024-04-25T09:01:35.366994+0000","last_undegraded":"2024-04-25T09:01:35.366994+0000","last_fullsized":"2024-04-25T09:01:35.366994+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:43.053265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.407022+0000","last_change":"2024-04-25T09:01:34.369565+0000","last_active":"2024-04-25T09:01:36.407022+0000","last_peered":"2024-04-25T09:01:36.407022+0000","last_clean":"2024-04-25T09:01:36.407022+0000","last_became_active":"2024-04-25T09:01:34.369249+0000","last_became_peered":"2024-04-25T09:01:34.369249+0000","last_unstale":"2024-04-25T09:01:36.407022+0000","last_undegraded":"2024-04-25T09:01:36.407022+0000","last_fullsized":"2024-04-25T09:01:36.407022+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:54.185982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.407110+0000","last_change":"2024-04-25T09:01:34.376994+0000","last_active":"2024-04-25T09:01:36.407110+0000","last_peered":"2024-04-25T09:01:36.407110+0000","last_clean":"2024-04-25T09:01:36.407110+0000","last_became_active":"2024-04-25T09:01:34.376233+0000","last_became_peered":"2024-04-25T09:01:34.376233+0000","last_unstale":"2024-04-25T09:01:36.407110+0000","last_undegraded":"2024-04-25T09:01:36.407110+0000","last_fullsized":"2024-04-25T09:01:36.407110+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:37:44.946297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.408182+0000","last_change":"2024-04-25T09:01:34.361921+0000","last_active":"2024-04-25T09:01:36.408182+0000","last_peered":"2024-04-25T09:01:36.408182+0000","last_clean":"2024-04-25T09:01:36.408182+0000","last_became_active":"2024-04-25T09:01:34.361602+0000","last_became_peered":"2024-04-25T09:01:34.361602+0000","last_unstale":"2024-04-25T09:01:36.408182+0000","last_undegraded":"2024-04-25T09:01:36.408182+0000","last_fullsized":"2024-04-25T09:01:36.408182+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:27.372745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.148712+0000","last_change":"2024-04-25T09:01:34.371479+0000","last_active":"2024-04-25T09:01:36.148712+0000","last_peered":"2024-04-25T09:01:36.148712+0000","last_clean":"2024-04-25T09:01:36.148712+0000","last_became_active":"2024-04-25T09:01:34.371194+0000","last_became_peered":"2024-04-25T09:01:34.371194+0000","last_unstale":"2024-04-25T09:01:36.148712+0000","last_undegraded":"2024-04-25T09:01:36.148712+0000","last_fullsized":"2024-04-25T09:01:36.148712+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:15.749948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:35.365764+0000","last_change":"2024-04-25T09:01:34.370037+0000","last_active":"2024-04-25T09:01:35.365764+0000","last_peered":"2024-04-25T09:01:35.365764+0000","last_clean":"2024-04-25T09:01:35.365764+0000","last_became_active":"2024-04-25T09:01:34.369011+0000","last_became_peered":"2024-04-25T09:01:34.369011+0000","last_unstale":"2024-04-25T09:01:35.365764+0000","last_undegraded":"2024-04-25T09:01:35.365764+0000","last_fullsized":"2024-04-25T09:01:35.365764+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:10.753122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.407712+0000","last_change":"2024-04-25T09:01:34.373554+0000","last_active":"2024-04-25T09:01:36.407712+0000","last_peered":"2024-04-25T09:01:36.407712+0000","last_clean":"2024-04-25T09:01:36.407712+0000","last_became_active":"2024-04-25T09:01:34.373084+0000","last_became_peered":"2024-04-25T09:01:34.373084+0000","last_unstale":"2024-04-25T09:01:36.407712+0000","last_undegraded":"2024-04-25T09:01:36.407712+0000","last_fullsized":"2024-04-25T09:01:36.407712+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:03.767726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.406804+0000","last_change":"2024-04-25T09:01:34.381062+0000","last_active":"2024-04-25T09:01:36.406804+0000","last_peered":"2024-04-25T09:01:36.406804+0000","last_clean":"2024-04-25T09:01:36.406804+0000","last_became_active":"2024-04-25T09:01:34.380758+0000","last_became_peered":"2024-04-25T09:01:34.380758+0000","last_unstale":"2024-04-25T09:01:36.406804+0000","last_undegraded":"2024-04-25T09:01:36.406804+0000","last_fullsized":"2024-04-25T09:01:36.406804+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:49.972383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:35.367245+0000","last_change":"2024-04-25T09:01:34.368441+0000","last_active":"2024-04-25T09:01:35.367245+0000","last_peered":"2024-04-25T09:01:35.367245+0000","last_clean":"2024-04-25T09:01:35.367245+0000","last_became_active":"2024-04-25T09:01:34.368178+0000","last_became_peered":"2024-04-25T09:01:34.368178+0000","last_unstale":"2024-04-25T09:01:35.367245+0000","last_undegraded":"2024-04-25T09:01:35.367245+0000","last_fullsized":"2024-04-25T09:01:35.367245+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:25.757698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.406855+0000","last_change":"2024-04-25T09:01:34.369263+0000","last_active":"2024-04-25T09:01:36.406855+0000","last_peered":"2024-04-25T09:01:36.406855+0000","last_clean":"2024-04-25T09:01:36.406855+0000","last_became_active":"2024-04-25T09:01:34.368936+0000","last_became_peered":"2024-04-25T09:01:34.368936+0000","last_unstale":"2024-04-25T09:01:36.406855+0000","last_undegraded":"2024-04-25T09:01:36.406855+0000","last_fullsized":"2024-04-25T09:01:36.406855+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:01.195437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:35.365361+0000","last_change":"2024-04-25T09:01:34.370880+0000","last_active":"2024-04-25T09:01:35.365361+0000","last_peered":"2024-04-25T09:01:35.365361+0000","last_clean":"2024-04-25T09:01:35.365361+0000","last_became_active":"2024-04-25T09:01:34.370576+0000","last_became_peered":"2024-04-25T09:01:34.370576+0000","last_unstale":"2024-04-25T09:01:35.365361+0000","last_undegraded":"2024-04-25T09:01:35.365361+0000","last_fullsized":"2024-04-25T09:01:35.365361+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:40.353729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:35.365647+0000","last_change":"2024-04-25T09:01:34.370487+0000","last_active":"2024-04-25T09:01:35.365647+0000","last_peered":"2024-04-25T09:01:35.365647+0000","last_clean":"2024-04-25T09:01:35.365647+0000","last_became_active":"2024-04-25T09:01:34.370164+0000","last_became_peered":"2024-04-25T09:01:34.370164+0000","last_unstale":"2024-04-25T09:01:35.365647+0000","last_undegraded":"2024-04-25T09:01:35.365647+0000","last_fullsized":"2024-04-25T09:01:35.365647+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:27.569144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.148785+0000","last_change":"2024-04-25T09:01:34.359777+0000","last_active":"2024-04-25T09:01:36.148785+0000","last_peered":"2024-04-25T09:01:36.148785+0000","last_clean":"2024-04-25T09:01:36.148785+0000","last_became_active":"2024-04-25T09:01:34.359473+0000","last_became_peered":"2024-04-25T09:01:34.359473+0000","last_unstale":"2024-04-25T09:01:36.148785+0000","last_undegraded":"2024-04-25T09:01:36.148785+0000","last_fullsized":"2024-04-25T09:01:36.148785+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:23.207224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.406907+0000","last_change":"2024-04-25T09:01:34.374946+0000","last_active":"2024-04-25T09:01:36.406907+0000","last_peered":"2024-04-25T09:01:36.406907+0000","last_clean":"2024-04-25T09:01:36.406907+0000","last_became_active":"2024-04-25T09:01:34.374642+0000","last_became_peered":"2024-04-25T09:01:34.374642+0000","last_unstale":"2024-04-25T09:01:36.406907+0000","last_undegraded":"2024-04-25T09:01:36.406907+0000","last_fullsized":"2024-04-25T09:01:36.406907+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:54.185982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:35.365661+0000","last_change":"2024-04-25T09:01:34.371413+0000","last_active":"2024-04-25T09:01:35.365661+0000","last_peered":"2024-04-25T09:01:35.365661+0000","last_clean":"2024-04-25T09:01:35.365661+0000","last_became_active":"2024-04-25T09:01:34.368733+0000","last_became_peered":"2024-04-25T09:01:34.368733+0000","last_unstale":"2024-04-25T09:01:35.365661+0000","last_undegraded":"2024-04-25T09:01:35.365661+0000","last_fullsized":"2024-04-25T09:01:35.365661+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:30:27.372745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.148825+0000","last_change":"2024-04-25T09:01:34.375486+0000","last_active":"2024-04-25T09:01:36.148825+0000","last_peered":"2024-04-25T09:01:36.148825+0000","last_clean":"2024-04-25T09:01:36.148825+0000","last_became_active":"2024-04-25T09:01:34.374852+0000","last_became_peered":"2024-04-25T09:01:34.374852+0000","last_unstale":"2024-04-25T09:01:36.148825+0000","last_undegraded":"2024-04-25T09:01:36.148825+0000","last_fullsized":"2024-04-25T09:01:36.148825+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:10.753122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.406828+0000","last_change":"2024-04-25T09:01:34.366280+0000","last_active":"2024-04-25T09:01:36.406828+0000","last_peered":"2024-04-25T09:01:36.406828+0000","last_clean":"2024-04-25T09:01:36.406828+0000","last_became_active":"2024-04-25T09:01:34.366011+0000","last_became_peered":"2024-04-25T09:01:34.366011+0000","last_unstale":"2024-04-25T09:01:36.406828+0000","last_undegraded":"2024-04-25T09:01:36.406828+0000","last_fullsized":"2024-04-25T09:01:36.406828+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:37.515032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:35.365491+0000","last_change":"2024-04-25T09:01:34.370898+0000","last_active":"2024-04-25T09:01:35.365491+0000","last_peered":"2024-04-25T09:01:35.365491+0000","last_clean":"2024-04-25T09:01:35.365491+0000","last_became_active":"2024-04-25T09:01:34.368635+0000","last_became_peered":"2024-04-25T09:01:34.368635+0000","last_unstale":"2024-04-25T09:01:35.365491+0000","last_undegraded":"2024-04-25T09:01:35.365491+0000","last_fullsized":"2024-04-25T09:01:35.365491+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:03.767726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.407248+0000","last_change":"2024-04-25T09:01:34.381540+0000","last_active":"2024-04-25T09:01:36.407248+0000","last_peered":"2024-04-25T09:01:36.407248+0000","last_clean":"2024-04-25T09:01:36.407248+0000","last_became_active":"2024-04-25T09:01:34.381241+0000","last_became_peered":"2024-04-25T09:01:34.381241+0000","last_unstale":"2024-04-25T09:01:36.407248+0000","last_undegraded":"2024-04-25T09:01:36.407248+0000","last_fullsized":"2024-04-25T09:01:36.407248+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:14.226478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.408378+0000","last_change":"2024-04-25T09:01:34.367536+0000","last_active":"2024-04-25T09:01:36.408378+0000","last_peered":"2024-04-25T09:01:36.408378+0000","last_clean":"2024-04-25T09:01:36.408378+0000","last_became_active":"2024-04-25T09:01:34.367209+0000","last_became_peered":"2024-04-25T09:01:34.367209+0000","last_unstale":"2024-04-25T09:01:36.408378+0000","last_undegraded":"2024-04-25T09:01:36.408378+0000","last_fullsized":"2024-04-25T09:01:36.408378+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:23.207224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.406718+0000","last_change":"2024-04-25T09:01:34.376539+0000","last_active":"2024-04-25T09:01:36.406718+0000","last_peered":"2024-04-25T09:01:36.406718+0000","last_clean":"2024-04-25T09:01:36.406718+0000","last_became_active":"2024-04-25T09:01:34.376230+0000","last_became_peered":"2024-04-25T09:01:34.376230+0000","last_unstale":"2024-04-25T09:01:36.406718+0000","last_undegraded":"2024-04-25T09:01:36.406718+0000","last_fullsized":"2024-04-25T09:01:36.406718+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:08.737168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.407017+0000","last_change":"2024-04-25T09:01:34.371814+0000","last_active":"2024-04-25T09:01:36.407017+0000","last_peered":"2024-04-25T09:01:36.407017+0000","last_clean":"2024-04-25T09:01:36.407017+0000","last_became_active":"2024-04-25T09:01:34.370661+0000","last_became_peered":"2024-04-25T09:01:34.370661+0000","last_unstale":"2024-04-25T09:01:36.407017+0000","last_undegraded":"2024-04-25T09:01:36.407017+0000","last_fullsized":"2024-04-25T09:01:36.407017+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:04.593435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"101'4","reported_seq":18,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:35.365858+0000","last_change":"2024-04-25T09:01:33.365666+0000","last_active":"2024-04-25T09:01:35.365858+0000","last_peered":"2024-04-25T09:01:35.365858+0000","last_clean":"2024-04-25T09:01:35.365858+0000","last_became_active":"2024-04-25T09:01:33.365341+0000","last_became_peered":"2024-04-25T09:01:33.365341+0000","last_unstale":"2024-04-25T09:01:35.365858+0000","last_undegraded":"2024-04-25T09:01:35.365858+0000","last_fullsized":"2024-04-25T09:01:35.365858+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:16.566868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.407928+0000","last_change":"2024-04-25T09:01:34.368045+0000","last_active":"2024-04-25T09:01:36.407928+0000","last_peered":"2024-04-25T09:01:36.407928+0000","last_clean":"2024-04-25T09:01:36.407928+0000","last_became_active":"2024-04-25T09:01:34.367684+0000","last_became_peered":"2024-04-25T09:01:34.367684+0000","last_unstale":"2024-04-25T09:01:36.407928+0000","last_undegraded":"2024-04-25T09:01:36.407928+0000","last_fullsized":"2024-04-25T09:01:36.407928+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:27.569144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"101'1","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.148770+0000","last_change":"2024-04-25T09:01:33.366268+0000","last_active":"2024-04-25T09:01:36.148770+0000","last_peered":"2024-04-25T09:01:36.148770+0000","last_clean":"2024-04-25T09:01:36.148770+0000","last_became_active":"2024-04-25T09:01:33.365952+0000","last_became_peered":"2024-04-25T09:01:33.365952+0000","last_unstale":"2024-04-25T09:01:36.148770+0000","last_undegraded":"2024-04-25T09:01:36.148770+0000","last_fullsized":"2024-04-25T09:01:36.148770+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:45.832620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.407472+0000","last_change":"2024-04-25T09:01:34.372365+0000","last_active":"2024-04-25T09:01:36.407472+0000","last_peered":"2024-04-25T09:01:36.407472+0000","last_clean":"2024-04-25T09:01:36.407472+0000","last_became_active":"2024-04-25T09:01:34.372078+0000","last_became_peered":"2024-04-25T09:01:34.372078+0000","last_unstale":"2024-04-25T09:01:36.407472+0000","last_undegraded":"2024-04-25T09:01:36.407472+0000","last_fullsized":"2024-04-25T09:01:36.407472+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:16.524887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"101'2","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:35.365934+0000","last_change":"2024-04-25T09:01:33.370140+0000","last_active":"2024-04-25T09:01:35.365934+0000","last_peered":"2024-04-25T09:01:35.365934+0000","last_clean":"2024-04-25T09:01:35.365934+0000","last_became_active":"2024-04-25T09:01:33.369677+0000","last_became_peered":"2024-04-25T09:01:33.369677+0000","last_unstale":"2024-04-25T09:01:35.365934+0000","last_undegraded":"2024-04-25T09:01:35.365934+0000","last_fullsized":"2024-04-25T09:01:35.365934+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:54.561452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:35.367037+0000","last_change":"2024-04-25T09:01:34.370660+0000","last_active":"2024-04-25T09:01:35.367037+0000","last_peered":"2024-04-25T09:01:35.367037+0000","last_clean":"2024-04-25T09:01:35.367037+0000","last_became_active":"2024-04-25T09:01:34.370169+0000","last_became_peered":"2024-04-25T09:01:34.370169+0000","last_unstale":"2024-04-25T09:01:35.367037+0000","last_undegraded":"2024-04-25T09:01:35.367037+0000","last_fullsized":"2024-04-25T09:01:35.367037+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:46.839903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"101'2","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:35.365410+0000","last_change":"2024-04-25T09:01:33.367516+0000","last_active":"2024-04-25T09:01:35.365410+0000","last_peered":"2024-04-25T09:01:35.365410+0000","last_clean":"2024-04-25T09:01:35.365410+0000","last_became_active":"2024-04-25T09:01:33.367171+0000","last_became_peered":"2024-04-25T09:01:33.367171+0000","last_unstale":"2024-04-25T09:01:35.365410+0000","last_undegraded":"2024-04-25T09:01:35.365410+0000","last_fullsized":"2024-04-25T09:01:35.365410+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:13.736791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:35.367456+0000","last_change":"2024-04-25T09:01:34.364948+0000","last_active":"2024-04-25T09:01:35.367456+0000","last_peered":"2024-04-25T09:01:35.367456+0000","last_clean":"2024-04-25T09:01:35.367456+0000","last_became_active":"2024-04-25T09:01:34.364233+0000","last_became_peered":"2024-04-25T09:01:34.364233+0000","last_unstale":"2024-04-25T09:01:35.367456+0000","last_undegraded":"2024-04-25T09:01:35.367456+0000","last_fullsized":"2024-04-25T09:01:35.367456+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:27.686308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"101'1","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.407302+0000","last_change":"2024-04-25T09:01:33.360517+0000","last_active":"2024-04-25T09:01:36.407302+0000","last_peered":"2024-04-25T09:01:36.407302+0000","last_clean":"2024-04-25T09:01:36.407302+0000","last_became_active":"2024-04-25T09:01:33.360139+0000","last_became_peered":"2024-04-25T09:01:33.360139+0000","last_unstale":"2024-04-25T09:01:36.407302+0000","last_undegraded":"2024-04-25T09:01:36.407302+0000","last_fullsized":"2024-04-25T09:01:36.407302+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:41:00.188154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.148673+0000","last_change":"2024-04-25T09:01:34.366436+0000","last_active":"2024-04-25T09:01:36.148673+0000","last_peered":"2024-04-25T09:01:36.148673+0000","last_clean":"2024-04-25T09:01:36.148673+0000","last_became_active":"2024-04-25T09:01:34.366152+0000","last_became_peered":"2024-04-25T09:01:34.366152+0000","last_unstale":"2024-04-25T09:01:36.148673+0000","last_undegraded":"2024-04-25T09:01:36.148673+0000","last_fullsized":"2024-04-25T09:01:36.148673+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:26:27.569144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"101'1","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:35.365471+0000","last_change":"2024-04-25T09:01:33.371930+0000","last_active":"2024-04-25T09:01:35.365471+0000","last_peered":"2024-04-25T09:01:35.365471+0000","last_clean":"2024-04-25T09:01:35.365471+0000","last_became_active":"2024-04-25T09:01:33.371395+0000","last_became_peered":"2024-04-25T09:01:33.371395+0000","last_unstale":"2024-04-25T09:01:35.365471+0000","last_undegraded":"2024-04-25T09:01:35.365471+0000","last_fullsized":"2024-04-25T09:01:35.365471+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:45.832620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.407294+0000","last_change":"2024-04-25T09:01:34.360463+0000","last_active":"2024-04-25T09:01:36.407294+0000","last_peered":"2024-04-25T09:01:36.407294+0000","last_clean":"2024-04-25T09:01:36.407294+0000","last_became_active":"2024-04-25T09:01:34.360165+0000","last_became_peered":"2024-04-25T09:01:34.360165+0000","last_unstale":"2024-04-25T09:01:36.407294+0000","last_undegraded":"2024-04-25T09:01:36.407294+0000","last_fullsized":"2024-04-25T09:01:36.407294+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:40.353729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.406864+0000","last_change":"2024-04-25T09:01:33.366526+0000","last_active":"2024-04-25T09:01:36.406864+0000","last_peered":"2024-04-25T09:01:36.406864+0000","last_clean":"2024-04-25T09:01:36.406864+0000","last_became_active":"2024-04-25T09:01:33.366246+0000","last_became_peered":"2024-04-25T09:01:33.366246+0000","last_unstale":"2024-04-25T09:01:36.406864+0000","last_undegraded":"2024-04-25T09:01:36.406864+0000","last_fullsized":"2024-04-25T09:01:36.406864+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:13.736791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:35.366295+0000","last_change":"2024-04-25T09:01:34.366933+0000","last_active":"2024-04-25T09:01:35.366295+0000","last_peered":"2024-04-25T09:01:35.366295+0000","last_clean":"2024-04-25T09:01:35.366295+0000","last_became_active":"2024-04-25T09:01:34.366640+0000","last_became_peered":"2024-04-25T09:01:34.366640+0000","last_unstale":"2024-04-25T09:01:35.366295+0000","last_undegraded":"2024-04-25T09:01:35.366295+0000","last_fullsized":"2024-04-25T09:01:35.366295+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:17.574151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.406817+0000","last_change":"2024-04-25T09:01:33.367167+0000","last_active":"2024-04-25T09:01:36.406817+0000","last_peered":"2024-04-25T09:01:36.406817+0000","last_clean":"2024-04-25T09:01:36.406817+0000","last_became_active":"2024-04-25T09:01:33.365541+0000","last_became_peered":"2024-04-25T09:01:33.365541+0000","last_unstale":"2024-04-25T09:01:36.406817+0000","last_undegraded":"2024-04-25T09:01:36.406817+0000","last_fullsized":"2024-04-25T09:01:36.406817+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:19.696240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:35.365729+0000","last_change":"2024-04-25T09:01:34.372279+0000","last_active":"2024-04-25T09:01:35.365729+0000","last_peered":"2024-04-25T09:01:35.365729+0000","last_clean":"2024-04-25T09:01:35.365729+0000","last_became_active":"2024-04-25T09:01:34.371765+0000","last_became_peered":"2024-04-25T09:01:34.371765+0000","last_unstale":"2024-04-25T09:01:35.365729+0000","last_undegraded":"2024-04-25T09:01:35.365729+0000","last_fullsized":"2024-04-25T09:01:35.365729+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:23.207224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.407620+0000","last_change":"2024-04-25T09:01:33.361825+0000","last_active":"2024-04-25T09:01:36.407620+0000","last_peered":"2024-04-25T09:01:36.407620+0000","last_clean":"2024-04-25T09:01:36.407620+0000","last_became_active":"2024-04-25T09:01:33.361544+0000","last_became_peered":"2024-04-25T09:01:33.361544+0000","last_unstale":"2024-04-25T09:01:36.407620+0000","last_undegraded":"2024-04-25T09:01:36.407620+0000","last_fullsized":"2024-04-25T09:01:36.407620+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:46:48.965100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:35.366600+0000","last_change":"2024-04-25T09:01:34.376456+0000","last_active":"2024-04-25T09:01:35.366600+0000","last_peered":"2024-04-25T09:01:35.366600+0000","last_clean":"2024-04-25T09:01:35.366600+0000","last_became_active":"2024-04-25T09:01:34.373765+0000","last_became_peered":"2024-04-25T09:01:34.373765+0000","last_unstale":"2024-04-25T09:01:35.366600+0000","last_undegraded":"2024-04-25T09:01:35.366600+0000","last_fullsized":"2024-04-25T09:01:35.366600+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:35.756290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.407783+0000","last_change":"2024-04-25T09:01:33.365387+0000","last_active":"2024-04-25T09:01:36.407783+0000","last_peered":"2024-04-25T09:01:36.407783+0000","last_clean":"2024-04-25T09:01:36.407783+0000","last_became_active":"2024-04-25T09:01:33.365088+0000","last_became_peered":"2024-04-25T09:01:33.365088+0000","last_unstale":"2024-04-25T09:01:36.407783+0000","last_undegraded":"2024-04-25T09:01:36.407783+0000","last_fullsized":"2024-04-25T09:01:36.407783+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:45.832620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.406921+0000","last_change":"2024-04-25T09:01:34.374537+0000","last_active":"2024-04-25T09:01:36.406921+0000","last_peered":"2024-04-25T09:01:36.406921+0000","last_clean":"2024-04-25T09:01:36.406921+0000","last_became_active":"2024-04-25T09:01:34.374273+0000","last_became_peered":"2024-04-25T09:01:34.374273+0000","last_unstale":"2024-04-25T09:01:36.406921+0000","last_undegraded":"2024-04-25T09:01:36.406921+0000","last_fullsized":"2024-04-25T09:01:36.406921+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:54:35.430338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"101'3","reported_seq":17,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.149070+0000","last_change":"2024-04-25T09:01:33.367517+0000","last_active":"2024-04-25T09:01:36.149070+0000","last_peered":"2024-04-25T09:01:36.149070+0000","last_clean":"2024-04-25T09:01:36.149070+0000","last_became_active":"2024-04-25T09:01:33.367216+0000","last_became_peered":"2024-04-25T09:01:33.367216+0000","last_unstale":"2024-04-25T09:01:36.149070+0000","last_undegraded":"2024-04-25T09:01:36.149070+0000","last_fullsized":"2024-04-25T09:01:36.149070+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:58.677436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.149008+0000","last_change":"2024-04-25T09:01:34.375227+0000","last_active":"2024-04-25T09:01:36.149008+0000","last_peered":"2024-04-25T09:01:36.149008+0000","last_clean":"2024-04-25T09:01:36.149008+0000","last_became_active":"2024-04-25T09:01:34.372938+0000","last_became_peered":"2024-04-25T09:01:34.372938+0000","last_unstale":"2024-04-25T09:01:36.149008+0000","last_undegraded":"2024-04-25T09:01:36.149008+0000","last_fullsized":"2024-04-25T09:01:36.149008+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:40:23.243482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"101'6","reported_seq":20,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:35.367119+0000","last_change":"2024-04-25T09:01:33.366291+0000","last_active":"2024-04-25T09:01:35.367119+0000","last_peered":"2024-04-25T09:01:35.367119+0000","last_clean":"2024-04-25T09:01:35.367119+0000","last_became_active":"2024-04-25T09:01:33.365670+0000","last_became_peered":"2024-04-25T09:01:33.365670+0000","last_unstale":"2024-04-25T09:01:35.367119+0000","last_undegraded":"2024-04-25T09:01:35.367119+0000","last_fullsized":"2024-04-25T09:01:35.367119+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:43.586923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.407459+0000","last_change":"2024-04-25T09:01:34.356376+0000","last_active":"2024-04-25T09:01:36.407459+0000","last_peered":"2024-04-25T09:01:36.407459+0000","last_clean":"2024-04-25T09:01:36.407459+0000","last_became_active":"2024-04-25T09:01:34.356090+0000","last_became_peered":"2024-04-25T09:01:34.356090+0000","last_unstale":"2024-04-25T09:01:36.407459+0000","last_undegraded":"2024-04-25T09:01:36.407459+0000","last_fullsized":"2024-04-25T09:01:36.407459+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:16.315887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"101'2","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.407131+0000","last_change":"2024-04-25T09:01:33.363784+0000","last_active":"2024-04-25T09:01:36.407131+0000","last_peered":"2024-04-25T09:01:36.407131+0000","last_clean":"2024-04-25T09:01:36.407131+0000","last_became_active":"2024-04-25T09:01:33.363279+0000","last_became_peered":"2024-04-25T09:01:33.363279+0000","last_unstale":"2024-04-25T09:01:36.407131+0000","last_undegraded":"2024-04-25T09:01:36.407131+0000","last_fullsized":"2024-04-25T09:01:36.407131+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:04.394683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.407203+0000","last_change":"2024-04-25T09:01:34.380001+0000","last_active":"2024-04-25T09:01:36.407203+0000","last_peered":"2024-04-25T09:01:36.407203+0000","last_clean":"2024-04-25T09:01:36.407203+0000","last_became_active":"2024-04-25T09:01:34.379325+0000","last_became_peered":"2024-04-25T09:01:34.379325+0000","last_unstale":"2024-04-25T09:01:36.407203+0000","last_undegraded":"2024-04-25T09:01:36.407203+0000","last_fullsized":"2024-04-25T09:01:36.407203+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:56.945948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.407557+0000","last_change":"2024-04-25T09:01:33.365837+0000","last_active":"2024-04-25T09:01:36.407557+0000","last_peered":"2024-04-25T09:01:36.407557+0000","last_clean":"2024-04-25T09:01:36.407557+0000","last_became_active":"2024-04-25T09:01:33.362985+0000","last_became_peered":"2024-04-25T09:01:33.362985+0000","last_unstale":"2024-04-25T09:01:36.407557+0000","last_undegraded":"2024-04-25T09:01:36.407557+0000","last_fullsized":"2024-04-25T09:01:36.407557+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:04.264328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:35.366242+0000","last_change":"2024-04-25T09:01:34.366267+0000","last_active":"2024-04-25T09:01:35.366242+0000","last_peered":"2024-04-25T09:01:35.366242+0000","last_clean":"2024-04-25T09:01:35.366242+0000","last_became_active":"2024-04-25T09:01:34.365974+0000","last_became_peered":"2024-04-25T09:01:34.365974+0000","last_unstale":"2024-04-25T09:01:35.366242+0000","last_undegraded":"2024-04-25T09:01:35.366242+0000","last_fullsized":"2024-04-25T09:01:35.366242+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:02.761528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:35.366402+0000","last_change":"2024-04-25T09:01:33.367032+0000","last_active":"2024-04-25T09:01:35.366402+0000","last_peered":"2024-04-25T09:01:35.366402+0000","last_clean":"2024-04-25T09:01:35.366402+0000","last_became_active":"2024-04-25T09:01:33.366662+0000","last_became_peered":"2024-04-25T09:01:33.366662+0000","last_unstale":"2024-04-25T09:01:35.366402+0000","last_undegraded":"2024-04-25T09:01:35.366402+0000","last_fullsized":"2024-04-25T09:01:35.366402+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:07.729885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.406557+0000","last_change":"2024-04-25T09:01:34.371572+0000","last_active":"2024-04-25T09:01:36.406557+0000","last_peered":"2024-04-25T09:01:36.406557+0000","last_clean":"2024-04-25T09:01:36.406557+0000","last_became_active":"2024-04-25T09:01:34.371296+0000","last_became_peered":"2024-04-25T09:01:34.371296+0000","last_unstale":"2024-04-25T09:01:36.406557+0000","last_undegraded":"2024-04-25T09:01:36.406557+0000","last_fullsized":"2024-04-25T09:01:36.406557+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:23.342357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"101'8","reported_seq":22,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.406923+0000","last_change":"2024-04-25T09:01:33.366823+0000","last_active":"2024-04-25T09:01:36.406923+0000","last_peered":"2024-04-25T09:01:36.406923+0000","last_clean":"2024-04-25T09:01:36.406923+0000","last_became_active":"2024-04-25T09:01:33.366050+0000","last_became_peered":"2024-04-25T09:01:33.366050+0000","last_unstale":"2024-04-25T09:01:36.406923+0000","last_undegraded":"2024-04-25T09:01:36.406923+0000","last_fullsized":"2024-04-25T09:01:36.406923+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:21:03.586152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.407074+0000","last_change":"2024-04-25T09:01:34.379063+0000","last_active":"2024-04-25T09:01:36.407074+0000","last_peered":"2024-04-25T09:01:36.407074+0000","last_clean":"2024-04-25T09:01:36.407074+0000","last_became_active":"2024-04-25T09:01:34.378572+0000","last_became_peered":"2024-04-25T09:01:34.378572+0000","last_unstale":"2024-04-25T09:01:36.407074+0000","last_undegraded":"2024-04-25T09:01:36.407074+0000","last_fullsized":"2024-04-25T09:01:36.407074+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:01:40.353729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"101'2","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:35.366309+0000","last_change":"2024-04-25T09:01:33.369232+0000","last_active":"2024-04-25T09:01:35.366309+0000","last_peered":"2024-04-25T09:01:35.366309+0000","last_clean":"2024-04-25T09:01:35.366309+0000","last_became_active":"2024-04-25T09:01:33.368905+0000","last_became_peered":"2024-04-25T09:01:33.368905+0000","last_unstale":"2024-04-25T09:01:35.366309+0000","last_undegraded":"2024-04-25T09:01:35.366309+0000","last_fullsized":"2024-04-25T09:01:35.366309+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:55.938665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:35.366419+0000","last_change":"2024-04-25T09:01:34.371803+0000","last_active":"2024-04-25T09:01:35.366419+0000","last_peered":"2024-04-25T09:01:35.366419+0000","last_clean":"2024-04-25T09:01:35.366419+0000","last_became_active":"2024-04-25T09:01:34.371507+0000","last_became_peered":"2024-04-25T09:01:34.371507+0000","last_unstale":"2024-04-25T09:01:35.366419+0000","last_undegraded":"2024-04-25T09:01:35.366419+0000","last_fullsized":"2024-04-25T09:01:35.366419+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:16.524887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"101'1","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.406580+0000","last_change":"2024-04-25T09:01:33.363919+0000","last_active":"2024-04-25T09:01:36.406580+0000","last_peered":"2024-04-25T09:01:36.406580+0000","last_clean":"2024-04-25T09:01:36.406580+0000","last_became_active":"2024-04-25T09:01:33.363587+0000","last_became_peered":"2024-04-25T09:01:33.363587+0000","last_unstale":"2024-04-25T09:01:36.406580+0000","last_undegraded":"2024-04-25T09:01:36.406580+0000","last_fullsized":"2024-04-25T09:01:36.406580+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:12.754187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.148988+0000","last_change":"2024-04-25T09:01:34.371327+0000","last_active":"2024-04-25T09:01:36.148988+0000","last_peered":"2024-04-25T09:01:36.148988+0000","last_clean":"2024-04-25T09:01:36.148988+0000","last_became_active":"2024-04-25T09:01:34.370700+0000","last_became_peered":"2024-04-25T09:01:34.370700+0000","last_unstale":"2024-04-25T09:01:36.148988+0000","last_undegraded":"2024-04-25T09:01:36.148988+0000","last_fullsized":"2024-04-25T09:01:36.148988+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:01.723108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:35.365516+0000","last_change":"2024-04-25T09:01:33.364976+0000","last_active":"2024-04-25T09:01:35.365516+0000","last_peered":"2024-04-25T09:01:35.365516+0000","last_clean":"2024-04-25T09:01:35.365516+0000","last_became_active":"2024-04-25T09:01:33.363617+0000","last_became_peered":"2024-04-25T09:01:33.363617+0000","last_unstale":"2024-04-25T09:01:35.365516+0000","last_undegraded":"2024-04-25T09:01:35.365516+0000","last_fullsized":"2024-04-25T09:01:35.365516+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:18.899577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.407288+0000","last_change":"2024-04-25T09:01:34.377106+0000","last_active":"2024-04-25T09:01:36.407288+0000","last_peered":"2024-04-25T09:01:36.407288+0000","last_clean":"2024-04-25T09:01:36.407288+0000","last_became_active":"2024-04-25T09:01:34.376642+0000","last_became_peered":"2024-04-25T09:01:34.376642+0000","last_unstale":"2024-04-25T09:01:36.407288+0000","last_undegraded":"2024-04-25T09:01:36.407288+0000","last_fullsized":"2024-04-25T09:01:36.407288+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:20:42.471487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"101'2","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:35.365594+0000","last_change":"2024-04-25T09:01:33.369475+0000","last_active":"2024-04-25T09:01:35.365594+0000","last_peered":"2024-04-25T09:01:35.365594+0000","last_clean":"2024-04-25T09:01:35.365594+0000","last_became_active":"2024-04-25T09:01:33.368210+0000","last_became_peered":"2024-04-25T09:01:33.368210+0000","last_unstale":"2024-04-25T09:01:35.365594+0000","last_undegraded":"2024-04-25T09:01:35.365594+0000","last_fullsized":"2024-04-25T09:01:35.365594+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:00:34.749007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.407015+0000","last_change":"2024-04-25T09:01:34.372343+0000","last_active":"2024-04-25T09:01:36.407015+0000","last_peered":"2024-04-25T09:01:36.407015+0000","last_clean":"2024-04-25T09:01:36.407015+0000","last_became_active":"2024-04-25T09:01:34.372048+0000","last_became_peered":"2024-04-25T09:01:34.372048+0000","last_unstale":"2024-04-25T09:01:36.407015+0000","last_undegraded":"2024-04-25T09:01:36.407015+0000","last_fullsized":"2024-04-25T09:01:36.407015+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:20.703523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:35.367463+0000","last_change":"2024-04-25T09:01:33.363025+0000","last_active":"2024-04-25T09:01:35.367463+0000","last_peered":"2024-04-25T09:01:35.367463+0000","last_clean":"2024-04-25T09:01:35.367463+0000","last_became_active":"2024-04-25T09:01:33.362724+0000","last_became_peered":"2024-04-25T09:01:33.362724+0000","last_unstale":"2024-04-25T09:01:35.367463+0000","last_undegraded":"2024-04-25T09:01:35.367463+0000","last_fullsized":"2024-04-25T09:01:35.367463+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:07.729885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.407248+0000","last_change":"2024-04-25T09:01:34.373923+0000","last_active":"2024-04-25T09:01:36.407248+0000","last_peered":"2024-04-25T09:01:36.407248+0000","last_clean":"2024-04-25T09:01:36.407248+0000","last_became_active":"2024-04-25T09:01:34.373659+0000","last_became_peered":"2024-04-25T09:01:34.373659+0000","last_unstale":"2024-04-25T09:01:36.407248+0000","last_undegraded":"2024-04-25T09:01:36.407248+0000","last_fullsized":"2024-04-25T09:01:36.407248+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:39.918737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:35.367149+0000","last_change":"2024-04-25T09:01:33.365366+0000","last_active":"2024-04-25T09:01:35.367149+0000","last_peered":"2024-04-25T09:01:35.367149+0000","last_clean":"2024-04-25T09:01:35.367149+0000","last_became_active":"2024-04-25T09:01:33.365090+0000","last_became_peered":"2024-04-25T09:01:33.365090+0000","last_unstale":"2024-04-25T09:01:35.367149+0000","last_undegraded":"2024-04-25T09:01:35.367149+0000","last_fullsized":"2024-04-25T09:01:35.367149+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:41.915410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.406467+0000","last_change":"2024-04-25T09:01:34.365175+0000","last_active":"2024-04-25T09:01:36.406467+0000","last_peered":"2024-04-25T09:01:36.406467+0000","last_clean":"2024-04-25T09:01:36.406467+0000","last_became_active":"2024-04-25T09:01:34.364601+0000","last_became_peered":"2024-04-25T09:01:34.364601+0000","last_unstale":"2024-04-25T09:01:36.406467+0000","last_undegraded":"2024-04-25T09:01:36.406467+0000","last_fullsized":"2024-04-25T09:01:36.406467+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:30.937224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:35.365757+0000","last_change":"2024-04-25T09:01:33.370545+0000","last_active":"2024-04-25T09:01:35.365757+0000","last_peered":"2024-04-25T09:01:35.365757+0000","last_clean":"2024-04-25T09:01:35.365757+0000","last_became_active":"2024-04-25T09:01:33.370255+0000","last_became_peered":"2024-04-25T09:01:33.370255+0000","last_unstale":"2024-04-25T09:01:35.365757+0000","last_undegraded":"2024-04-25T09:01:35.365757+0000","last_fullsized":"2024-04-25T09:01:35.365757+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:36:12.754187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:35.365809+0000","last_change":"2024-04-25T09:01:34.372802+0000","last_active":"2024-04-25T09:01:35.365809+0000","last_peered":"2024-04-25T09:01:35.365809+0000","last_clean":"2024-04-25T09:01:35.365809+0000","last_became_active":"2024-04-25T09:01:34.372494+0000","last_became_peered":"2024-04-25T09:01:34.372494+0000","last_unstale":"2024-04-25T09:01:35.365809+0000","last_undegraded":"2024-04-25T09:01:35.365809+0000","last_fullsized":"2024-04-25T09:01:35.365809+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:02:21.718104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"101'2","reported_seq":16,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:35.367476+0000","last_change":"2024-04-25T09:01:33.364409+0000","last_active":"2024-04-25T09:01:35.367476+0000","last_peered":"2024-04-25T09:01:35.367476+0000","last_clean":"2024-04-25T09:01:35.367476+0000","last_became_active":"2024-04-25T09:01:33.364089+0000","last_became_peered":"2024-04-25T09:01:33.364089+0000","last_unstale":"2024-04-25T09:01:35.367476+0000","last_undegraded":"2024-04-25T09:01:35.367476+0000","last_fullsized":"2024-04-25T09:01:35.367476+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:34:13.219195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.407525+0000","last_change":"2024-04-25T09:01:34.380018+0000","last_active":"2024-04-25T09:01:36.407525+0000","last_peered":"2024-04-25T09:01:36.407525+0000","last_clean":"2024-04-25T09:01:36.407525+0000","last_became_active":"2024-04-25T09:01:34.379673+0000","last_became_peered":"2024-04-25T09:01:34.379673+0000","last_unstale":"2024-04-25T09:01:36.407525+0000","last_undegraded":"2024-04-25T09:01:36.407525+0000","last_fullsized":"2024-04-25T09:01:36.407525+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:27.686308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"101'1","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:35.365820+0000","last_change":"2024-04-25T09:01:33.366607+0000","last_active":"2024-04-25T09:01:35.365820+0000","last_peered":"2024-04-25T09:01:35.365820+0000","last_clean":"2024-04-25T09:01:35.365820+0000","last_became_active":"2024-04-25T09:01:33.366048+0000","last_became_peered":"2024-04-25T09:01:33.366048+0000","last_unstale":"2024-04-25T09:01:35.365820+0000","last_undegraded":"2024-04-25T09:01:35.365820+0000","last_fullsized":"2024-04-25T09:01:35.365820+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:29.149495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.407626+0000","last_change":"2024-04-25T09:01:34.373869+0000","last_active":"2024-04-25T09:01:36.407626+0000","last_peered":"2024-04-25T09:01:36.407626+0000","last_clean":"2024-04-25T09:01:36.407626+0000","last_became_active":"2024-04-25T09:01:34.373581+0000","last_became_peered":"2024-04-25T09:01:34.373581+0000","last_unstale":"2024-04-25T09:01:36.407626+0000","last_undegraded":"2024-04-25T09:01:36.407626+0000","last_fullsized":"2024-04-25T09:01:36.407626+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:13:11.856833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:35.365932+0000","last_change":"2024-04-25T09:01:33.368065+0000","last_active":"2024-04-25T09:01:35.365932+0000","last_peered":"2024-04-25T09:01:35.365932+0000","last_clean":"2024-04-25T09:01:35.365932+0000","last_became_active":"2024-04-25T09:01:33.367711+0000","last_became_peered":"2024-04-25T09:01:33.367711+0000","last_unstale":"2024-04-25T09:01:35.365932+0000","last_undegraded":"2024-04-25T09:01:35.365932+0000","last_fullsized":"2024-04-25T09:01:35.365932+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:01.754245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.406753+0000","last_change":"2024-04-25T09:01:34.363716+0000","last_active":"2024-04-25T09:01:36.406753+0000","last_peered":"2024-04-25T09:01:36.406753+0000","last_clean":"2024-04-25T09:01:36.406753+0000","last_became_active":"2024-04-25T09:01:34.363424+0000","last_became_peered":"2024-04-25T09:01:34.363424+0000","last_unstale":"2024-04-25T09:01:36.406753+0000","last_undegraded":"2024-04-25T09:01:36.406753+0000","last_fullsized":"2024-04-25T09:01:36.406753+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:26.299800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.406927+0000","last_change":"2024-04-25T09:01:33.365212+0000","last_active":"2024-04-25T09:01:36.406927+0000","last_peered":"2024-04-25T09:01:36.406927+0000","last_clean":"2024-04-25T09:01:36.406927+0000","last_became_active":"2024-04-25T09:01:33.364945+0000","last_became_peered":"2024-04-25T09:01:33.364945+0000","last_unstale":"2024-04-25T09:01:36.406927+0000","last_undegraded":"2024-04-25T09:01:36.406927+0000","last_fullsized":"2024-04-25T09:01:36.406927+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:18.899577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.407455+0000","last_change":"2024-04-25T09:01:34.376853+0000","last_active":"2024-04-25T09:01:36.407455+0000","last_peered":"2024-04-25T09:01:36.407455+0000","last_clean":"2024-04-25T09:01:36.407455+0000","last_became_active":"2024-04-25T09:01:34.375817+0000","last_became_peered":"2024-04-25T09:01:34.375817+0000","last_unstale":"2024-04-25T09:01:36.407455+0000","last_undegraded":"2024-04-25T09:01:36.407455+0000","last_fullsized":"2024-04-25T09:01:36.407455+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:32:36.179605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:35.365705+0000","last_change":"2024-04-25T09:01:33.367854+0000","last_active":"2024-04-25T09:01:35.365705+0000","last_peered":"2024-04-25T09:01:35.365705+0000","last_clean":"2024-04-25T09:01:35.365705+0000","last_became_active":"2024-04-25T09:01:33.366038+0000","last_became_peered":"2024-04-25T09:01:33.366038+0000","last_unstale":"2024-04-25T09:01:35.365705+0000","last_undegraded":"2024-04-25T09:01:35.365705+0000","last_fullsized":"2024-04-25T09:01:35.365705+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:42.045982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:35.366534+0000","last_change":"2024-04-25T09:01:34.373481+0000","last_active":"2024-04-25T09:01:35.366534+0000","last_peered":"2024-04-25T09:01:35.366534+0000","last_clean":"2024-04-25T09:01:35.366534+0000","last_became_active":"2024-04-25T09:01:34.373234+0000","last_became_peered":"2024-04-25T09:01:34.373234+0000","last_unstale":"2024-04-25T09:01:35.366534+0000","last_undegraded":"2024-04-25T09:01:35.366534+0000","last_fullsized":"2024-04-25T09:01:35.366534+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:30.156778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.406468+0000","last_change":"2024-04-25T09:01:33.369961+0000","last_active":"2024-04-25T09:01:36.406468+0000","last_peered":"2024-04-25T09:01:36.406468+0000","last_clean":"2024-04-25T09:01:36.406468+0000","last_became_active":"2024-04-25T09:01:33.369606+0000","last_became_peered":"2024-04-25T09:01:33.369606+0000","last_unstale":"2024-04-25T09:01:36.406468+0000","last_undegraded":"2024-04-25T09:01:36.406468+0000","last_fullsized":"2024-04-25T09:01:36.406468+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:45:44.824026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.407436+0000","last_change":"2024-04-25T09:01:34.371338+0000","last_active":"2024-04-25T09:01:36.407436+0000","last_peered":"2024-04-25T09:01:36.407436+0000","last_clean":"2024-04-25T09:01:36.407436+0000","last_became_active":"2024-04-25T09:01:34.370986+0000","last_became_peered":"2024-04-25T09:01:34.370986+0000","last_unstale":"2024-04-25T09:01:36.407436+0000","last_undegraded":"2024-04-25T09:01:36.407436+0000","last_fullsized":"2024-04-25T09:01:36.407436+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:33:59.752318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.408471+0000","last_change":"2024-04-25T09:01:33.364868+0000","last_active":"2024-04-25T09:01:36.408471+0000","last_peered":"2024-04-25T09:01:36.408471+0000","last_clean":"2024-04-25T09:01:36.408471+0000","last_became_active":"2024-04-25T09:01:33.364604+0000","last_became_peered":"2024-04-25T09:01:33.364604+0000","last_unstale":"2024-04-25T09:01:36.408471+0000","last_undegraded":"2024-04-25T09:01:36.408471+0000","last_fullsized":"2024-04-25T09:01:36.408471+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:24.750415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.406557+0000","last_change":"2024-04-25T09:01:34.366316+0000","last_active":"2024-04-25T09:01:36.406557+0000","last_peered":"2024-04-25T09:01:36.406557+0000","last_clean":"2024-04-25T09:01:36.406557+0000","last_became_active":"2024-04-25T09:01:34.365188+0000","last_became_peered":"2024-04-25T09:01:34.365188+0000","last_unstale":"2024-04-25T09:01:36.406557+0000","last_undegraded":"2024-04-25T09:01:36.406557+0000","last_fullsized":"2024-04-25T09:01:36.406557+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:14.694412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":14,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.407539+0000","last_change":"2024-04-25T09:01:33.364144+0000","last_active":"2024-04-25T09:01:36.407539+0000","last_peered":"2024-04-25T09:01:36.407539+0000","last_clean":"2024-04-25T09:01:36.407539+0000","last_became_active":"2024-04-25T09:01:33.363517+0000","last_became_peered":"2024-04-25T09:01:33.363517+0000","last_unstale":"2024-04-25T09:01:36.407539+0000","last_undegraded":"2024-04-25T09:01:36.407539+0000","last_fullsized":"2024-04-25T09:01:36.407539+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:46.356921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":12,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.407204+0000","last_change":"2024-04-25T09:01:34.380714+0000","last_active":"2024-04-25T09:01:36.407204+0000","last_peered":"2024-04-25T09:01:36.407204+0000","last_clean":"2024-04-25T09:01:36.407204+0000","last_became_active":"2024-04-25T09:01:34.380208+0000","last_became_peered":"2024-04-25T09:01:34.380208+0000","last_unstale":"2024-04-25T09:01:36.407204+0000","last_undegraded":"2024-04-25T09:01:36.407204+0000","last_fullsized":"2024-04-25T09:01:36.407204+0000","mapping_epoch":100,"log_start":"0'0","ondisk_log_start":"0'0","created":100,"last_epoch_clean":101,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:33.340929+0000","last_clean_scrub_stamp":"2024-04-25T09:01:33.340929+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:22:47.995211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"101'1","reported_seq":15,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.148726+0000","last_change":"2024-04-25T09:01:33.367042+0000","last_active":"2024-04-25T09:01:36.148726+0000","last_peered":"2024-04-25T09:01:36.148726+0000","last_clean":"2024-04-25T09:01:36.148726+0000","last_became_active":"2024-04-25T09:01:33.366746+0000","last_became_peered":"2024-04-25T09:01:33.366746+0000","last_unstale":"2024-04-25T09:01:36.148726+0000","last_undegraded":"2024-04-25T09:01:36.148726+0000","last_fullsized":"2024-04-25T09:01:36.148726+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:01:32.333646+0000","last_clean_scrub_stamp":"2024-04-25T09:01:32.333646+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:24.750415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":321,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-25T09:01:36.148946+0000","last_change":"2024-04-25T08:54:00.818044+0000","last_active":"2024-04-25T09:01:36.148946+0000","last_peered":"2024-04-25T09:01:36.148946+0000","last_clean":"2024-04-25T09:01:36.148946+0000","last_became_active":"2024-04-25T08:54:00.817098+0000","last_became_peered":"2024-04-25T08:54:00.817098+0000","last_unstale":"2024-04-25T09:01:36.148946+0000","last_undegraded":"2024-04-25T09:01:36.148946+0000","last_fullsized":"2024-04-25T09:01:36.148946+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_clean_scrub_stamp":"2024-04-25T08:53:59.739677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:48.129513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574942,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59372,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312468,"statfs":{"total":96636764160,"available":96575967232,"internally_reserved":0,"allocated":1863680,"data_stored":1137812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574943,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81000,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":79742,"kb_avail":94290840,"statfs":{"total":96636764160,"available":96553820160,"internally_reserved":0,"allocated":1269760,"data_stored":547984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":81656262},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574945,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50472,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321368,"statfs":{"total":96636764160,"available":96585080832,"internally_reserved":0,"allocated":1265664,"data_stored":547402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574943,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64372,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94307468,"statfs":{"total":96636764160,"available":96570847232,"internally_reserved":0,"allocated":1282048,"data_stored":549068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574943,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68524,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303316,"statfs":{"total":96636764160,"available":96566595584,"internally_reserved":0,"allocated":1277952,"data_stored":548966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":68876743},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574943,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58784,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313056,"statfs":{"total":96636764160,"available":96576569344,"internally_reserved":0,"allocated":1265664,"data_stored":547366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574943,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76828,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94295012,"statfs":{"total":96636764160,"available":96558092288,"internally_reserved":0,"allocated":1261568,"data_stored":547332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574944,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59364,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312476,"statfs":{"total":96636764160,"available":96575975424,"internally_reserved":0,"allocated":1859584,"data_stored":1138282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:01:42.115 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:01:42.115 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T09:01:42.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.270+0000 7f9d85f3b640 1 Processor -- start 2024-04-25T09:01:42.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.270+0000 7f9d85f3b640 1 -- start start 2024-04-25T09:01:42.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.271+0000 7f9d85f3b640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9d8015a6b0 0x7f9d800cc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:42.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.271+0000 7f9d85f3b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d8006dc00 0x7f9d800b35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:42.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.271+0000 7f9d85f3b640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d8015e3c0 con 0x7f9d800ae9e0 2024-04-25T09:01:42.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.271+0000 7f9d85f3b640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d8015e680 con 0x7f9d8015a6b0 2024-04-25T09:01:42.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.271+0000 7f9d85f3b640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8015e960 con 0x7f9d8006dc00 2024-04-25T09:01:42.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.271+0000 7f9d7f7fe640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9d8015a6b0 0x7f9d800cc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:42.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.271+0000 7f9d7ffff640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d8006dc00 0x7f9d800b35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:42.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.271+0000 7f9d7effd640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f9d800ae9e0 0x7f9d8006d4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:54166/0 (socket says 172.21.15.43:54166) 2024-04-25T09:01:42.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.271+0000 7f9d7effd640 1 -- 172.21.15.43:0/599594935 learned_addr learned my addr 172.21.15.43:0/599594935 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:42.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.271+0000 7f9d7f7fe640 1 -- 172.21.15.43:0/599594935 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d8006dc00 msgr2=0x7f9d800b35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:42.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.271+0000 7f9d7f7fe640 1 --2- 172.21.15.43:0/599594935 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d8006dc00 0x7f9d800b35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:42.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.271+0000 7f9d7f7fe640 1 -- 172.21.15.43:0/599594935 >> v1:172.21.15.193:6789/0 conn(0x7f9d800ae9e0 legacy=0x7f9d8006d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:42.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.271+0000 7f9d7f7fe640 1 -- 172.21.15.43:0/599594935 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d8015ec40 con 0x7f9d8015a6b0 2024-04-25T09:01:42.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.271+0000 7f9d7ffff640 1 --2- 172.21.15.43:0/599594935 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d8006dc00 0x7f9d800b35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:01:42.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.272+0000 7f9d7f7fe640 1 --2- 172.21.15.43:0/599594935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d8015a6b0 0x7f9d800cc6b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f9d680097a0 tx=0x7f9d6802a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7e31dbdbfe14aad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:42.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.272+0000 7f9d7e7fc640 1 -- 172.21.15.43:0/599594935 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d68037070 con 0x7f9d8015a6b0 2024-04-25T09:01:42.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.272+0000 7f9d7e7fc640 1 -- 172.21.15.43:0/599594935 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d68032440 con 0x7f9d8015a6b0 2024-04-25T09:01:42.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.272+0000 7f9d7e7fc640 1 -- 172.21.15.43:0/599594935 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d68032810 con 0x7f9d8015a6b0 2024-04-25T09:01:42.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.273+0000 7f9d85f3b640 1 -- 172.21.15.43:0/599594935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d8015a6b0 msgr2=0x7f9d800cc6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:42.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.273+0000 7f9d85f3b640 1 --2- 172.21.15.43:0/599594935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d8015a6b0 0x7f9d800cc6b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f9d680097a0 tx=0x7f9d6802a180 comp rx=0 tx=0).stop 2024-04-25T09:01:42.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.273+0000 7f9d85f3b640 1 -- 172.21.15.43:0/599594935 shutdown_connections 2024-04-25T09:01:42.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.273+0000 7f9d85f3b640 1 --2- 172.21.15.43:0/599594935 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d8006dc00 0x7f9d800b35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:42.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.273+0000 7f9d85f3b640 1 --2- 172.21.15.43:0/599594935 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d8015a6b0 0x7f9d800cc6b0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:42.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.273+0000 7f9d85f3b640 1 -- 172.21.15.43:0/599594935 >> 172.21.15.43:0/599594935 conn(0x7f9d8006ab30 msgr2=0x7f9d8006af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:42.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.273+0000 7f9d85f3b640 1 -- 172.21.15.43:0/599594935 shutdown_connections 2024-04-25T09:01:42.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.273+0000 7f9d85f3b640 1 -- 172.21.15.43:0/599594935 wait complete. 2024-04-25T09:01:42.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.274+0000 7f9d85f3b640 1 Processor -- start 2024-04-25T09:01:42.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.274+0000 7f9d85f3b640 1 -- start start 2024-04-25T09:01:42.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.275+0000 7f9d85f3b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d8006dc00 0x7f9d801f42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:42.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.275+0000 7f9d85f3b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d800ae9e0 0x7f9d8014afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:42.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.275+0000 7f9d85f3b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d8015a6b0 0x7f9d801e4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:42.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.275+0000 7f9d85f3b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8015aa90 con 0x7f9d8015a6b0 2024-04-25T09:01:42.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.275+0000 7f9d85f3b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8015f200 con 0x7f9d800ae9e0 2024-04-25T09:01:42.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.275+0000 7f9d85f3b640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8015ac10 con 0x7f9d8006dc00 2024-04-25T09:01:42.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.275+0000 7f9d7ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d8015a6b0 0x7f9d801e4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:42.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.275+0000 7f9d7ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d8015a6b0 0x7f9d801e4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42520/0 (socket says 172.21.15.43:42520) 2024-04-25T09:01:42.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.275+0000 7f9d7ffff640 1 -- 172.21.15.43:0/481714561 learned_addr learned my addr 172.21.15.43:0/481714561 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:42.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.275+0000 7f9d7effd640 1 --2- 172.21.15.43:0/481714561 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d800ae9e0 0x7f9d8014afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:42.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.275+0000 7f9d7f7fe640 1 --2- 172.21.15.43:0/481714561 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d8006dc00 0x7f9d801f42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:42.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.275+0000 7f9d7ffff640 1 -- 172.21.15.43:0/481714561 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d8006dc00 msgr2=0x7f9d801f42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:42.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.275+0000 7f9d7ffff640 1 --2- 172.21.15.43:0/481714561 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d8006dc00 0x7f9d801f42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:42.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.275+0000 7f9d7ffff640 1 -- 172.21.15.43:0/481714561 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d800ae9e0 msgr2=0x7f9d8014afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:42.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.275+0000 7f9d7ffff640 1 --2- 172.21.15.43:0/481714561 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d800ae9e0 0x7f9d8014afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:42.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.275+0000 7f9d7ffff640 1 -- 172.21.15.43:0/481714561 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d801f49e0 con 0x7f9d8015a6b0 2024-04-25T09:01:42.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.275+0000 7f9d7f7fe640 1 --2- 172.21.15.43:0/481714561 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d8006dc00 0x7f9d801f42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:42.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.275+0000 7f9d7effd640 1 --2- 172.21.15.43:0/481714561 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d800ae9e0 0x7f9d8014afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:42.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.276+0000 7f9d7ffff640 1 --2- 172.21.15.43:0/481714561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d8015a6b0 0x7f9d801e4950 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f9d7800efc0 tx=0x7f9d7800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:42.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.276+0000 7f9d77fff640 1 -- 172.21.15.43:0/481714561 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d78018020 con 0x7f9d8015a6b0 2024-04-25T09:01:42.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.276+0000 7f9d77fff640 1 -- 172.21.15.43:0/481714561 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d78014480 con 0x7f9d8015a6b0 2024-04-25T09:01:42.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.276+0000 7f9d85f3b640 1 -- 172.21.15.43:0/481714561 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d801f4c80 con 0x7f9d8015a6b0 2024-04-25T09:01:42.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.276+0000 7f9d77fff640 1 -- 172.21.15.43:0/481714561 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d78014870 con 0x7f9d8015a6b0 2024-04-25T09:01:42.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.276+0000 7f9d85f3b640 1 -- 172.21.15.43:0/481714561 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d801f5240 con 0x7f9d8015a6b0 2024-04-25T09:01:42.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.277+0000 7f9d85f3b640 1 -- 172.21.15.43:0/481714561 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d80074f60 con 0x7f9d8015a6b0 2024-04-25T09:01:42.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.277+0000 7f9d77fff640 1 -- 172.21.15.43:0/481714561 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9d78005d10 con 0x7f9d8015a6b0 2024-04-25T09:01:42.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.278+0000 7f9d77fff640 1 --2- 172.21.15.43:0/481714561 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9d6006b4d0 0x7f9d6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:42.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.278+0000 7f9d77fff640 1 -- 172.21.15.43:0/481714561 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9d7808af60 con 0x7f9d8015a6b0 2024-04-25T09:01:42.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.278+0000 7f9d7f7fe640 1 --2- 172.21.15.43:0/481714561 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9d6006b4d0 0x7f9d6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:42.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.278+0000 7f9d7f7fe640 1 --2- 172.21.15.43:0/481714561 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9d6006b4d0 0x7f9d6006d990 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f9d68034000 tx=0x7f9d6802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:42.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.282+0000 7f9d77fff640 1 -- 172.21.15.43:0/481714561 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9d78022030 con 0x7f9d8015a6b0 2024-04-25T09:01:42.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.536+0000 7f9d85f3b640 1 -- 172.21.15.43:0/481714561 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f9d8006d490 con 0x7f9d8015a6b0 2024-04-25T09:01:42.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.538+0000 7f9d77fff640 1 -- 172.21.15.43:0/481714561 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+147 (secure 0 0 0) 0x7f9d78010040 con 0x7f9d8015a6b0 2024-04-25T09:01:42.526 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2024-04-25T09:01:42.527 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQD8GSpmVOjWGxAAFNmxeGi0h2D//pa6Ne3eQw== 2024-04-25T09:01:42.527 INFO:teuthology.orchestra.run.smithi043.stdout: caps mds = "allow" 2024-04-25T09:01:42.527 INFO:teuthology.orchestra.run.smithi043.stdout: caps mon = "allow r" 2024-04-25T09:01:42.527 INFO:teuthology.orchestra.run.smithi043.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T09:01:42.527 INFO:teuthology.orchestra.run.smithi043.stderr:updated caps for client.0 2024-04-25T09:01:42.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.540+0000 7f9d85f3b640 1 -- 172.21.15.43:0/481714561 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9d6006b4d0 msgr2=0x7f9d6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:42.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.540+0000 7f9d85f3b640 1 --2- 172.21.15.43:0/481714561 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9d6006b4d0 0x7f9d6006d990 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f9d68034000 tx=0x7f9d6802e000 comp rx=0 tx=0).stop 2024-04-25T09:01:42.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.540+0000 7f9d85f3b640 1 -- 172.21.15.43:0/481714561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d8015a6b0 msgr2=0x7f9d801e4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:42.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.540+0000 7f9d85f3b640 1 --2- 172.21.15.43:0/481714561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d8015a6b0 0x7f9d801e4950 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f9d7800efc0 tx=0x7f9d7800c490 comp rx=0 tx=0).stop 2024-04-25T09:01:42.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.540+0000 7f9d85f3b640 1 -- 172.21.15.43:0/481714561 shutdown_connections 2024-04-25T09:01:42.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.540+0000 7f9d85f3b640 1 --2- 172.21.15.43:0/481714561 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9d8015a6b0 0x7f9d801e4950 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:42.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.540+0000 7f9d85f3b640 1 --2- 172.21.15.43:0/481714561 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9d6006b4d0 0x7f9d6006d990 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:42.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.540+0000 7f9d85f3b640 1 --2- 172.21.15.43:0/481714561 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9d800ae9e0 0x7f9d8014afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:42.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.540+0000 7f9d85f3b640 1 --2- 172.21.15.43:0/481714561 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9d8006dc00 0x7f9d801f42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:42.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.540+0000 7f9d85f3b640 1 -- 172.21.15.43:0/481714561 >> 172.21.15.43:0/481714561 conn(0x7f9d8006ab30 msgr2=0x7f9d800af960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:42.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.540+0000 7f9d85f3b640 1 -- 172.21.15.43:0/481714561 shutdown_connections 2024-04-25T09:01:42.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.541+0000 7f9d85f3b640 1 -- 172.21.15.43:0/481714561 wait complete. 2024-04-25T09:01:42.541 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:01:42.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.653+0000 7f33c125a640 1 Processor -- start 2024-04-25T09:01:42.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.653+0000 7f33c125a640 1 -- start start 2024-04-25T09:01:42.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.654+0000 7f33c125a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f33bc161390 0x7f33bc0b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:42.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.654+0000 7f33c125a640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f33bc0cecf0 0x7f33bc0d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:42.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.654+0000 7f33c125a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33bc0b4c30 con 0x7f33bc0d02a0 2024-04-25T09:01:42.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.654+0000 7f33c125a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33bc0d1910 con 0x7f33bc161390 2024-04-25T09:01:42.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.654+0000 7f33c125a640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33bc0cf2c0 con 0x7f33bc0cecf0 2024-04-25T09:01:42.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.654+0000 7f33bad9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f33bc161390 0x7f33bc0b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:42.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.654+0000 7f33b3fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f33bc0d02a0 0x7f33bc0ce5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:58994/0 (socket says 172.21.15.43:58994) 2024-04-25T09:01:42.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.654+0000 7f33b3fff640 1 -- 172.21.15.43:0/941822993 learned_addr learned my addr 172.21.15.43:0/941822993 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:42.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.654+0000 7f33bb59e640 1 --2- 172.21.15.43:0/941822993 >> v2:172.21.15.193:3300/0 conn(0x7f33bc0cecf0 0x7f33bc0d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:42.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.654+0000 7f33bad9d640 1 -- 172.21.15.43:0/941822993 >> v2:172.21.15.193:3300/0 conn(0x7f33bc0cecf0 msgr2=0x7f33bc0d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:42.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.654+0000 7f33bad9d640 1 --2- 172.21.15.43:0/941822993 >> v2:172.21.15.193:3300/0 conn(0x7f33bc0cecf0 0x7f33bc0d0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:42.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.654+0000 7f33bad9d640 1 -- 172.21.15.43:0/941822993 >> v1:172.21.15.43:6789/0 conn(0x7f33bc0d02a0 legacy=0x7f33bc0ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:42.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.654+0000 7f33bad9d640 1 -- 172.21.15.43:0/941822993 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33bc0cf5a0 con 0x7f33bc161390 2024-04-25T09:01:42.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.654+0000 7f33ba59c640 1 -- 172.21.15.43:0/941822993 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1602314580 0 0) 0x7f33a4001110 con 0x7f33bc0d02a0 2024-04-25T09:01:42.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.655+0000 7f33bad9d640 1 --2- 172.21.15.43:0/941822993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33bc161390 0x7f33bc0b6820 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f33bc06d9c0 tx=0x7f33ac02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7930cd15013b28d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:42.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.655+0000 7f33ba59c640 1 -- 172.21.15.43:0/941822993 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac037070 con 0x7f33bc161390 2024-04-25T09:01:42.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.655+0000 7f33ba59c640 1 -- 172.21.15.43:0/941822993 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33ac032440 con 0x7f33bc161390 2024-04-25T09:01:42.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.655+0000 7f33ba59c640 1 -- 172.21.15.43:0/941822993 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33ac032810 con 0x7f33bc161390 2024-04-25T09:01:42.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.656+0000 7f33c125a640 1 -- 172.21.15.43:0/941822993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33bc161390 msgr2=0x7f33bc0b6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:42.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.656+0000 7f33c125a640 1 --2- 172.21.15.43:0/941822993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33bc161390 0x7f33bc0b6820 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f33bc06d9c0 tx=0x7f33ac02a1a0 comp rx=0 tx=0).stop 2024-04-25T09:01:42.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.656+0000 7f33c125a640 1 -- 172.21.15.43:0/941822993 shutdown_connections 2024-04-25T09:01:42.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.656+0000 7f33c125a640 1 --2- 172.21.15.43:0/941822993 >> v2:172.21.15.193:3300/0 conn(0x7f33bc0cecf0 0x7f33bc0d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:42.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.656+0000 7f33c125a640 1 --2- 172.21.15.43:0/941822993 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33bc161390 0x7f33bc0b6820 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:42.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.656+0000 7f33c125a640 1 -- 172.21.15.43:0/941822993 >> 172.21.15.43:0/941822993 conn(0x7f33bc09cd20 msgr2=0x7f33bc0d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:42.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.656+0000 7f33c125a640 1 -- 172.21.15.43:0/941822993 shutdown_connections 2024-04-25T09:01:42.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.656+0000 7f33c125a640 1 -- 172.21.15.43:0/941822993 wait complete. 2024-04-25T09:01:42.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.657+0000 7f33c125a640 1 Processor -- start 2024-04-25T09:01:42.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.657+0000 7f33c125a640 1 -- start start 2024-04-25T09:01:42.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.658+0000 7f33c125a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f33bc0cecf0 0x7f33bc15d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:42.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.658+0000 7f33c125a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33bc0d02a0 0x7f33bc1f36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:42.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.658+0000 7f33c125a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33bc161390 0x7f33bc15acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:42.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.658+0000 7f33c125a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33bc161770 con 0x7f33bc0d02a0 2024-04-25T09:01:42.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.658+0000 7f33c125a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33bc0d15a0 con 0x7f33bc161390 2024-04-25T09:01:42.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.658+0000 7f33c125a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33bc0d1da0 con 0x7f33bc0cecf0 2024-04-25T09:01:42.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.658+0000 7f33b3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33bc0d02a0 0x7f33bc1f36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:42.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.658+0000 7f33b3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33bc0d02a0 0x7f33bc1f36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42534/0 (socket says 172.21.15.43:42534) 2024-04-25T09:01:42.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.658+0000 7f33b3fff640 1 -- 172.21.15.43:0/1402584861 learned_addr learned my addr 172.21.15.43:0/1402584861 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:42.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.658+0000 7f33bad9d640 1 --2- 172.21.15.43:0/1402584861 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f33bc0cecf0 0x7f33bc15d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:42.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.658+0000 7f33b3fff640 1 -- 172.21.15.43:0/1402584861 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f33bc0cecf0 msgr2=0x7f33bc15d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:42.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.658+0000 7f33b3fff640 1 --2- 172.21.15.43:0/1402584861 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f33bc0cecf0 0x7f33bc15d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:42.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.658+0000 7f33b3fff640 1 -- 172.21.15.43:0/1402584861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33bc161390 msgr2=0x7f33bc15acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:42.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.658+0000 7f33bb59e640 1 --2- 172.21.15.43:0/1402584861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33bc161390 0x7f33bc15acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:42.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.658+0000 7f33b3fff640 1 --2- 172.21.15.43:0/1402584861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33bc161390 0x7f33bc15acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:42.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.658+0000 7f33b3fff640 1 -- 172.21.15.43:0/1402584861 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33bc15da90 con 0x7f33bc0d02a0 2024-04-25T09:01:42.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.658+0000 7f33b3fff640 1 --2- 172.21.15.43:0/1402584861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33bc0d02a0 0x7f33bc1f36d0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f33a400c780 tx=0x7f33a400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:42.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.658+0000 7f33b37fe640 1 -- 172.21.15.43:0/1402584861 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a4018070 con 0x7f33bc0d02a0 2024-04-25T09:01:42.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.658+0000 7f33b37fe640 1 -- 172.21.15.43:0/1402584861 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a40072d0 con 0x7f33bc0d02a0 2024-04-25T09:01:42.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.658+0000 7f33c125a640 1 -- 172.21.15.43:0/1402584861 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33bc15bae0 con 0x7f33bc0d02a0 2024-04-25T09:01:42.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.659+0000 7f33b37fe640 1 -- 172.21.15.43:0/1402584861 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a40047e0 con 0x7f33bc0d02a0 2024-04-25T09:01:42.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.659+0000 7f33c125a640 1 -- 172.21.15.43:0/1402584861 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33bc156290 con 0x7f33bc0d02a0 2024-04-25T09:01:42.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.659+0000 7f33c125a640 1 -- 172.21.15.43:0/1402584861 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33bc0680f0 con 0x7f33bc0d02a0 2024-04-25T09:01:42.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.660+0000 7f33b37fe640 1 -- 172.21.15.43:0/1402584861 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f33a401b070 con 0x7f33bc0d02a0 2024-04-25T09:01:42.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.660+0000 7f33b37fe640 1 --2- 172.21.15.43:0/1402584861 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f339c06b4d0 0x7f339c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:42.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.661+0000 7f33b37fe640 1 -- 172.21.15.43:0/1402584861 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f33a4089570 con 0x7f33bc0d02a0 2024-04-25T09:01:42.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.661+0000 7f33bad9d640 1 --2- 172.21.15.43:0/1402584861 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f339c06b4d0 0x7f339c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:42.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.661+0000 7f33bad9d640 1 --2- 172.21.15.43:0/1402584861 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f339c06b4d0 0x7f339c06d990 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f33ac034000 tx=0x7f33ac02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:42.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.665+0000 7f33b37fe640 1 -- 172.21.15.43:0/1402584861 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f33bc0b4c30 con 0x7f33bc0d02a0 2024-04-25T09:01:42.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.884+0000 7f33c125a640 1 -- 172.21.15.43:0/1402584861 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33bc0d1050 con 0x7f33bc0d02a0 2024-04-25T09:01:42.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.884+0000 7f33b37fe640 1 -- 172.21.15.43:0/1402584861 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f33a4087e20 con 0x7f33bc0d02a0 2024-04-25T09:01:42.873 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:01:42.874 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":61,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2150177203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2150177203},{"type":"v1","addr":"172.21.15.193:6837","nonce":2150177203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":8079,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/621488642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":621488642},{"type":"v1","addr":"172.21.15.43:6834","nonce":621488642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8082,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1524395840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1524395840},{"type":"v1","addr":"172.21.15.43:6835","nonce":1524395840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:01:33.726790+0000","modified":"2024-04-25T09:01:35.706689+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8094},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8094":{"gid":8094,"name":"b","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.193:6836/853943267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":853943267},{"type":"v1","addr":"172.21.15.193:6836","nonce":853943267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:01:42.874 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 61 2024-04-25T09:01:42.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.886+0000 7f33c125a640 1 -- 172.21.15.43:0/1402584861 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f339c06b4d0 msgr2=0x7f339c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:42.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.886+0000 7f33c125a640 1 --2- 172.21.15.43:0/1402584861 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f339c06b4d0 0x7f339c06d990 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f33ac034000 tx=0x7f33ac02e000 comp rx=0 tx=0).stop 2024-04-25T09:01:42.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.886+0000 7f33c125a640 1 -- 172.21.15.43:0/1402584861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33bc0d02a0 msgr2=0x7f33bc1f36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:42.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.886+0000 7f33c125a640 1 --2- 172.21.15.43:0/1402584861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33bc0d02a0 0x7f33bc1f36d0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f33a400c780 tx=0x7f33a400cc40 comp rx=0 tx=0).stop 2024-04-25T09:01:42.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.887+0000 7f33c125a640 1 -- 172.21.15.43:0/1402584861 shutdown_connections 2024-04-25T09:01:42.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.887+0000 7f33c125a640 1 --2- 172.21.15.43:0/1402584861 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f339c06b4d0 0x7f339c06d990 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:42.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.887+0000 7f33c125a640 1 --2- 172.21.15.43:0/1402584861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f33bc161390 0x7f33bc15acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:42.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.887+0000 7f33c125a640 1 --2- 172.21.15.43:0/1402584861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f33bc0d02a0 0x7f33bc1f36d0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:42.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.887+0000 7f33c125a640 1 --2- 172.21.15.43:0/1402584861 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f33bc0cecf0 0x7f33bc15d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:42.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.887+0000 7f33c125a640 1 -- 172.21.15.43:0/1402584861 >> 172.21.15.43:0/1402584861 conn(0x7f33bc09cd20 msgr2=0x7f33bc0b87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:42.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.887+0000 7f33c125a640 1 -- 172.21.15.43:0/1402584861 shutdown_connections 2024-04-25T09:01:42.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:42.887+0000 7f33c125a640 1 -- 172.21.15.43:0/1402584861 wait complete. 2024-04-25T09:01:43.289 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 61, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:01:33.726790+0000', 'modified': '2024-04-25T09:01:35.706689+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8094}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8094': {'gid': 8094, 'name': 'b', 'rank': 0, 'incarnation': 59, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.193:6836/853943267', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6834', 'nonce': 853943267}, {'type': 'v1', 'addr': '172.21.15.193:6836', 'nonce': 853943267}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:01:43.290 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:01:43.290 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T09:01:43.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.406+0000 7f47769ab640 1 Processor -- start 2024-04-25T09:01:43.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.406+0000 7f47769ab640 1 -- start start 2024-04-25T09:01:43.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.407+0000 7f47769ab640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f47700c8d00 0x7f4770162100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:43.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.407+0000 7f47769ab640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f47700ce660 0x7f47700cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:43.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.407+0000 7f47769ab640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47700d24a0 con 0x7f477015df50 2024-04-25T09:01:43.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.407+0000 7f47769ab640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f477015e330 con 0x7f47700ce660 2024-04-25T09:01:43.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.407+0000 7f47769ab640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47700d2740 con 0x7f47700c8d00 2024-04-25T09:01:43.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.407+0000 7f4774f48640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f47700ce660 0x7f47700cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:43.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.407+0000 7f4774f48640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f47700ce660 0x7f47700cea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42538/0 (socket says 172.21.15.43:42538) 2024-04-25T09:01:43.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.407+0000 7f4774f48640 1 -- 172.21.15.43:0/436294351 learned_addr learned my addr 172.21.15.43:0/436294351 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:43.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.408+0000 7f476f7fe640 1 --2- 172.21.15.43:0/436294351 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f47700c8d00 0x7f4770162100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:43.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.408+0000 7f4774f48640 1 -- 172.21.15.43:0/436294351 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f47700c8d00 msgr2=0x7f4770162100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:43.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.408+0000 7f4774f48640 1 --2- 172.21.15.43:0/436294351 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f47700c8d00 0x7f4770162100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:43.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.408+0000 7f4774f48640 1 -- 172.21.15.43:0/436294351 >> v1:172.21.15.193:6789/0 conn(0x7f477015df50 legacy=0x7f47701619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:43.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.408+0000 7f4774f48640 1 -- 172.21.15.43:0/436294351 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47700d2a20 con 0x7f47700ce660 2024-04-25T09:01:43.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.408+0000 7f4774f48640 1 --2- 172.21.15.43:0/436294351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47700ce660 0x7f47700cea70 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f4768009e50 tx=0x7f476802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32d4e69da3c6228 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:43.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.409+0000 7f476effd640 1 -- 172.21.15.43:0/436294351 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4768036070 con 0x7f47700ce660 2024-04-25T09:01:43.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.409+0000 7f476effd640 1 -- 172.21.15.43:0/436294351 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4768031600 con 0x7f47700ce660 2024-04-25T09:01:43.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.409+0000 7f476effd640 1 -- 172.21.15.43:0/436294351 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47680319d0 con 0x7f47700ce660 2024-04-25T09:01:43.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.409+0000 7f47769ab640 1 -- 172.21.15.43:0/436294351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47700ce660 msgr2=0x7f47700cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:43.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.409+0000 7f47769ab640 1 --2- 172.21.15.43:0/436294351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47700ce660 0x7f47700cea70 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f4768009e50 tx=0x7f476802a710 comp rx=0 tx=0).stop 2024-04-25T09:01:43.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.409+0000 7f47769ab640 1 -- 172.21.15.43:0/436294351 shutdown_connections 2024-04-25T09:01:43.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.409+0000 7f47769ab640 1 --2- 172.21.15.43:0/436294351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f47700ce660 0x7f47700cea70 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:43.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.409+0000 7f47769ab640 1 --2- 172.21.15.43:0/436294351 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f47700c8d00 0x7f4770162100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:43.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.409+0000 7f47769ab640 1 -- 172.21.15.43:0/436294351 >> 172.21.15.43:0/436294351 conn(0x7f47700d3a30 msgr2=0x7f47700d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:43.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.409+0000 7f47769ab640 1 -- 172.21.15.43:0/436294351 shutdown_connections 2024-04-25T09:01:43.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.409+0000 7f47769ab640 1 -- 172.21.15.43:0/436294351 wait complete. 2024-04-25T09:01:43.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.410+0000 7f47769ab640 1 Processor -- start 2024-04-25T09:01:43.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.411+0000 7f47769ab640 1 -- start start 2024-04-25T09:01:43.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.411+0000 7f47769ab640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f47700c8d00 0x7f477014c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:43.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.411+0000 7f47769ab640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47700ce660 0x7f4770157080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:43.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.411+0000 7f47769ab640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f477015df50 0x7f4770154a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:43.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.411+0000 7f47769ab640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47700d2190 con 0x7f477015df50 2024-04-25T09:01:43.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.411+0000 7f47769ab640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47700c77d0 con 0x7f47700ce660 2024-04-25T09:01:43.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.411+0000 7f47769ab640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4770162640 con 0x7f47700c8d00 2024-04-25T09:01:43.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.411+0000 7f4774f48640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f477015df50 0x7f4770154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:43.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.411+0000 7f4774f48640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f477015df50 0x7f4770154a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42548/0 (socket says 172.21.15.43:42548) 2024-04-25T09:01:43.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.411+0000 7f4774f48640 1 -- 172.21.15.43:0/2465348334 learned_addr learned my addr 172.21.15.43:0/2465348334 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:43.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.412+0000 7f476f7fe640 1 --2- 172.21.15.43:0/2465348334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47700ce660 0x7f4770157080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:43.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.412+0000 7f476ffff640 1 --2- 172.21.15.43:0/2465348334 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f47700c8d00 0x7f477014c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:43.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.412+0000 7f4774f48640 1 -- 172.21.15.43:0/2465348334 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f47700c8d00 msgr2=0x7f477014c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:43.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.412+0000 7f4774f48640 1 --2- 172.21.15.43:0/2465348334 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f47700c8d00 0x7f477014c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:43.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.412+0000 7f4774f48640 1 -- 172.21.15.43:0/2465348334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47700ce660 msgr2=0x7f4770157080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:43.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.412+0000 7f4774f48640 1 --2- 172.21.15.43:0/2465348334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47700ce660 0x7f4770157080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:43.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.412+0000 7f4774f48640 1 -- 172.21.15.43:0/2465348334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47700cab10 con 0x7f477015df50 2024-04-25T09:01:43.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.412+0000 7f4774f48640 1 --2- 172.21.15.43:0/2465348334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f477015df50 0x7f4770154a40 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f4768034040 tx=0x7f4768004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:43.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.412+0000 7f476cff9640 1 -- 172.21.15.43:0/2465348334 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476803f020 con 0x7f477015df50 2024-04-25T09:01:43.404 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.412+0000 7f476cff9640 1 -- 172.21.15.43:0/2465348334 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4768037040 con 0x7f477015df50 2024-04-25T09:01:43.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.412+0000 7f476cff9640 1 -- 172.21.15.43:0/2465348334 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4768036070 con 0x7f477015df50 2024-04-25T09:01:43.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.412+0000 7f47769ab640 1 -- 172.21.15.43:0/2465348334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47700cadf0 con 0x7f477015df50 2024-04-25T09:01:43.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.412+0000 7f47769ab640 1 -- 172.21.15.43:0/2465348334 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47700cb3b0 con 0x7f477015df50 2024-04-25T09:01:43.405 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.414+0000 7f476cff9640 1 -- 172.21.15.43:0/2465348334 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4768042020 con 0x7f477015df50 2024-04-25T09:01:43.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.414+0000 7f476cff9640 1 --2- 172.21.15.43:0/2465348334 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f475c06b4d0 0x7f475c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:43.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.414+0000 7f476cff9640 1 -- 172.21.15.43:0/2465348334 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f47680aa4f0 con 0x7f477015df50 2024-04-25T09:01:43.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.415+0000 7f476ffff640 1 --2- 172.21.15.43:0/2465348334 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f475c06b4d0 0x7f475c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:43.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.415+0000 7f476ffff640 1 --2- 172.21.15.43:0/2465348334 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f475c06b4d0 0x7f475c06d990 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f47600016e0 tx=0x7f4760009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:43.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.415+0000 7f4779782740 1 Processor -- start 2024-04-25T09:01:43.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.416+0000 7f4779782740 1 -- start start 2024-04-25T09:01:43.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.416+0000 7f4779782740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdec96b230 0x55fdec96b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:43.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.416+0000 7f4779782740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55fdec988ad0 0x55fdec989020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:43.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.416+0000 7f4779782740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fdec940f10 0x55fdec987190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:43.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.416+0000 7f4779782740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdec8a3a10 con 0x55fdec96b230 2024-04-25T09:01:43.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.416+0000 7f4779782740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdec84cb30 con 0x55fdec940f10 2024-04-25T09:01:43.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.416+0000 7f4779782740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fdec89fc50 con 0x55fdec988ad0 2024-04-25T09:01:43.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.416+0000 7f476ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdec96b230 0x55fdec96b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:43.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.416+0000 7f476ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdec96b230 0x55fdec96b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42556/0 (socket says 172.21.15.43:42556) 2024-04-25T09:01:43.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.416+0000 7f476ffff640 1 -- 172.21.15.43:0/13175371 learned_addr learned my addr 172.21.15.43:0/13175371 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:43.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.416+0000 7f4774f48640 1 --2- 172.21.15.43:0/13175371 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fdec940f10 0x55fdec987190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:43.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.416+0000 7f476f7fe640 1 --2- 172.21.15.43:0/13175371 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55fdec988ad0 0x55fdec989020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:43.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.417+0000 7f476ffff640 1 -- 172.21.15.43:0/13175371 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55fdec988ad0 msgr2=0x55fdec989020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:43.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.417+0000 7f476ffff640 1 --2- 172.21.15.43:0/13175371 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55fdec988ad0 0x55fdec989020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:43.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.417+0000 7f476ffff640 1 -- 172.21.15.43:0/13175371 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fdec940f10 msgr2=0x55fdec987190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:43.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.417+0000 7f476ffff640 1 --2- 172.21.15.43:0/13175371 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fdec940f10 0x55fdec987190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:43.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.417+0000 7f476ffff640 1 -- 172.21.15.43:0/13175371 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fdec861e10 con 0x55fdec96b230 2024-04-25T09:01:43.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.417+0000 7f476f7fe640 1 --2- 172.21.15.43:0/13175371 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55fdec988ad0 0x55fdec989020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:01:43.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.417+0000 7f476ffff640 1 --2- 172.21.15.43:0/13175371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdec96b230 0x55fdec96b610 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f4760007bc0 tx=0x7f4760012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e967f5642aa2dc75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:43.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.417+0000 7f4755ffb640 1 -- 172.21.15.43:0/13175371 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4760026070 con 0x55fdec96b230 2024-04-25T09:01:43.414 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.417+0000 7f4755ffb640 1 -- 172.21.15.43:0/13175371 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47600067d0 con 0x55fdec96b230 2024-04-25T09:01:43.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.417+0000 7f4755ffb640 1 -- 172.21.15.43:0/13175371 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4760006b80 con 0x55fdec96b230 2024-04-25T09:01:43.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.417+0000 7f4779782740 1 -- 172.21.15.43:0/13175371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdec96b230 msgr2=0x55fdec96b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:43.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.417+0000 7f4779782740 1 --2- 172.21.15.43:0/13175371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdec96b230 0x55fdec96b610 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f4760007bc0 tx=0x7f4760012000 comp rx=0 tx=0).stop 2024-04-25T09:01:43.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.418+0000 7f4779782740 1 -- 172.21.15.43:0/13175371 shutdown_connections 2024-04-25T09:01:43.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.418+0000 7f4779782740 1 --2- 172.21.15.43:0/13175371 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fdec940f10 0x55fdec987190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:43.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.418+0000 7f4779782740 1 --2- 172.21.15.43:0/13175371 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55fdec988ad0 0x55fdec989020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:43.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.418+0000 7f4779782740 1 --2- 172.21.15.43:0/13175371 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdec96b230 0x55fdec96b610 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:43.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.418+0000 7f4779782740 1 -- 172.21.15.43:0/13175371 >> 172.21.15.43:0/13175371 conn(0x55fdec93d840 msgr2=0x55fdec986040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:43.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.418+0000 7f4779782740 1 -- 172.21.15.43:0/13175371 shutdown_connections 2024-04-25T09:01:43.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.418+0000 7f4779782740 1 -- 172.21.15.43:0/13175371 wait complete. 2024-04-25T09:01:43.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.418+0000 7f4779782740 1 -- start start 2024-04-25T09:01:43.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.418+0000 7f4779782740 1 Processor -- start 2024-04-25T09:01:43.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.419+0000 7f4779782740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55fdec96b230 0x55fdec96d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:43.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.419+0000 7f4779782740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fdec988ad0 0x55fdec96bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:43.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.419+0000 7f4779782740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdec967140 0x55fdec96e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:43.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.419+0000 7f4779782740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdec941cb0 con 0x55fdec967140 2024-04-25T09:01:43.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.419+0000 7f4779782740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdec8600c0 con 0x55fdec988ad0 2024-04-25T09:01:43.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.419+0000 7f4779782740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fdec93b770 con 0x55fdec96b230 2024-04-25T09:01:43.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.419+0000 7f4779782740 10 client.0 _ll_register_callbacks cb 0x55fdec853ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:01:43.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.419+0000 7f4774f48640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdec967140 0x55fdec96e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:43.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.419+0000 7f4774f48640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdec967140 0x55fdec96e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42558/0 (socket says 172.21.15.43:42558) 2024-04-25T09:01:43.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.419+0000 7f4774f48640 1 -- 172.21.15.43:0/3330337154 learned_addr learned my addr 172.21.15.43:0/3330337154 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:43.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.419+0000 7f476ffff640 1 --2- 172.21.15.43:0/3330337154 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55fdec96b230 0x55fdec96d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:43.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.419+0000 7f476f7fe640 1 --2- 172.21.15.43:0/3330337154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fdec988ad0 0x55fdec96bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:43.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.419+0000 7f4774f48640 1 -- 172.21.15.43:0/3330337154 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55fdec96b230 msgr2=0x55fdec96d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:43.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.419+0000 7f4774f48640 1 --2- 172.21.15.43:0/3330337154 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55fdec96b230 0x55fdec96d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:43.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.420+0000 7f4774f48640 1 -- 172.21.15.43:0/3330337154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fdec988ad0 msgr2=0x55fdec96bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:43.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.420+0000 7f4774f48640 1 --2- 172.21.15.43:0/3330337154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fdec988ad0 0x55fdec96bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:43.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.420+0000 7f4774f48640 1 -- 172.21.15.43:0/3330337154 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fdec5b82d0 con 0x55fdec967140 2024-04-25T09:01:43.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.420+0000 7f476f7fe640 1 --2- 172.21.15.43:0/3330337154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fdec988ad0 0x55fdec96bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:43.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.420+0000 7f4774f48640 1 --2- 172.21.15.43:0/3330337154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdec967140 0x55fdec96e8f0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f47680a89b0 tx=0x7f4768078ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91a3897e48a3db00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:43.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.420+0000 7f474effd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:01:43.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.420+0000 7f474effd640 1 -- 172.21.15.43:0/3330337154 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4768079170 con 0x55fdec967140 2024-04-25T09:01:43.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.420+0000 7f474effd640 1 -- 172.21.15.43:0/3330337154 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4768080ee0 con 0x55fdec967140 2024-04-25T09:01:43.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.420+0000 7f4779782740 1 -- 172.21.15.43:0/3330337154 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f47600067d0 con 0x55fdec967140 2024-04-25T09:01:43.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.420+0000 7f474effd640 1 -- 172.21.15.43:0/3330337154 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4768031600 con 0x55fdec967140 2024-04-25T09:01:43.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.420+0000 7f4779782740 10 client.8253 fetch_fsmap learned FSMap version 61 2024-04-25T09:01:43.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.420+0000 7f4779782740 1 -- 172.21.15.43:0/3330337154 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x55fdec6826b0 con 0x55fdec967140 2024-04-25T09:01:43.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.420+0000 7f474effd640 1 -- 172.21.15.43:0/3330337154 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47600067d0 con 0x55fdec967140 2024-04-25T09:01:43.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.420+0000 7f474effd640 1 -- 172.21.15.43:0/3330337154 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 61) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4768078160 con 0x55fdec967140 2024-04-25T09:01:43.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.420+0000 7f4779782740 10 client.8253 fetch_fsmap finished waiting for FSMap version 61 2024-04-25T09:01:43.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.420+0000 7f4779782740 10 client.8253 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T09:01:43.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.420+0000 7f4779782740 20 client.8253 populate_metadata read hostname 'smithi043' 2024-04-25T09:01:43.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.420+0000 7f4779782740 1 --2- 172.21.15.43:0/3330337154 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55fdec967770 0x55fdec967b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:43.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.420+0000 7f4779782740 4 client.8253 mds_command: new command op to 8094 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:01:43.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.420+0000 7f4779782740 1 -- 172.21.15.43:0/3330337154 --> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fdec902920 con 0x55fdec967770 2024-04-25T09:01:43.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.421+0000 7f476ffff640 1 --2- 172.21.15.43:0/3330337154 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55fdec967770 0x55fdec967b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:43.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.421+0000 7f474effd640 10 client.8253 ms_handle_connect on v2:172.21.15.193:6834/853943267 2024-04-25T09:01:43.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.421+0000 7f476ffff640 1 --2- 172.21.15.43:0/3330337154 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55fdec967770 0x55fdec967b50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0c94e48fd42ae38 server_cookie=c5647b2b7a16380a in_seq=0 out_seq=0 2024-04-25T09:01:43.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.422+0000 7f474effd640 1 -- 172.21.15.43:0/3330337154 <== mds.0 v2:172.21.15.193:6834/853943267 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fdec902920 con 0x55fdec967770 2024-04-25T09:01:43.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.422+0000 7f474effd640 10 client.8253 handle_command_reply: tid=0 2024-04-25T09:01:43.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.422+0000 7f4779782740 1 client.8253 shutdown 2024-04-25T09:01:43.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.422+0000 7f4779782740 1 -- 172.21.15.43:0/3330337154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdec967140 msgr2=0x55fdec96e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:43.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.422+0000 7f4779782740 1 --2- 172.21.15.43:0/3330337154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdec967140 0x55fdec96e8f0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f47680a89b0 tx=0x7f4768078ba0 comp rx=0 tx=0).stop 2024-04-25T09:01:43.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.422+0000 7f4779782740 1 -- 172.21.15.43:0/3330337154 shutdown_connections 2024-04-25T09:01:43.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.422+0000 7f4779782740 1 --2- 172.21.15.43:0/3330337154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdec967140 0x55fdec96e8f0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:43.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.422+0000 7f4779782740 1 --2- 172.21.15.43:0/3330337154 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55fdec967770 0x55fdec967b50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:43.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.422+0000 7f4779782740 1 --2- 172.21.15.43:0/3330337154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fdec988ad0 0x55fdec96bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:43.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.422+0000 7f474effd640 0 client.8253 ms_handle_reset on v2:172.21.15.193:6834/853943267 2024-04-25T09:01:43.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.422+0000 7f4779782740 1 --2- 172.21.15.43:0/3330337154 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55fdec96b230 0x55fdec96d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:43.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.422+0000 7f4779782740 1 -- 172.21.15.43:0/3330337154 >> 172.21.15.43:0/3330337154 conn(0x55fdec940f10 msgr2=0x55fdec9895c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:43.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.422+0000 7f4779782740 1 -- 172.21.15.43:0/3330337154 shutdown_connections 2024-04-25T09:01:43.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.422+0000 7f4779782740 1 -- 172.21.15.43:0/3330337154 wait complete. 2024-04-25T09:01:43.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.422+0000 7f4779782740 20 client.8253 trim_cache size 0 max 16384 2024-04-25T09:01:43.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.438+0000 7f4779782740 1 Processor -- start 2024-04-25T09:01:43.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.438+0000 7f4779782740 1 -- start start 2024-04-25T09:01:43.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.439+0000 7f4779782740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdec967140 0x55fdec96f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:43.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.439+0000 7f4779782740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55fdec967770 0x55fdec96db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:43.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.439+0000 7f4779782740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fdec96b230 0x55fdec9950b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:43.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.439+0000 7f4779782740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47701ff250 con 0x55fdec967140 2024-04-25T09:01:43.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.439+0000 7f4779782740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47700cc4c0 con 0x55fdec96b230 2024-04-25T09:01:43.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.439+0000 7f4779782740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47700ccac0 con 0x55fdec967770 2024-04-25T09:01:43.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.439+0000 7f476ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdec967140 0x55fdec96f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:43.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.439+0000 7f476ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdec967140 0x55fdec96f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42572/0 (socket says 172.21.15.43:42572) 2024-04-25T09:01:43.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.439+0000 7f476ffff640 1 -- 172.21.15.43:0/3413176698 learned_addr learned my addr 172.21.15.43:0/3413176698 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:43.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.439+0000 7f476ffff640 1 -- 172.21.15.43:0/3413176698 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55fdec967770 msgr2=0x55fdec96db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:43.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.439+0000 7f476ffff640 1 --2- 172.21.15.43:0/3413176698 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55fdec967770 0x55fdec96db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:43.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.439+0000 7f476ffff640 1 -- 172.21.15.43:0/3413176698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fdec96b230 msgr2=0x55fdec9950b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:43.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.439+0000 7f476ffff640 1 --2- 172.21.15.43:0/3413176698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fdec96b230 0x55fdec9950b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:43.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.439+0000 7f476ffff640 1 -- 172.21.15.43:0/3413176698 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fdec96ae60 con 0x55fdec967140 2024-04-25T09:01:43.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.439+0000 7f476ffff640 1 --2- 172.21.15.43:0/3413176698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdec967140 0x55fdec96f200 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f4760001510 tx=0x7f4760014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ef56387a4ded5ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:43.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.439+0000 7f47557fa640 1 -- 172.21.15.43:0/3413176698 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4760033070 con 0x55fdec967140 2024-04-25T09:01:43.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.439+0000 7f47557fa640 1 -- 172.21.15.43:0/3413176698 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4760027040 con 0x55fdec967140 2024-04-25T09:01:43.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.439+0000 7f47557fa640 1 -- 172.21.15.43:0/3413176698 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4760026040 con 0x55fdec967140 2024-04-25T09:01:43.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.440+0000 7f4779782740 1 -- 172.21.15.43:0/3413176698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdec967140 msgr2=0x55fdec96f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:43.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.440+0000 7f4779782740 1 --2- 172.21.15.43:0/3413176698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdec967140 0x55fdec96f200 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f4760001510 tx=0x7f4760014000 comp rx=0 tx=0).stop 2024-04-25T09:01:43.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.440+0000 7f4779782740 1 -- 172.21.15.43:0/3413176698 shutdown_connections 2024-04-25T09:01:43.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.440+0000 7f4779782740 1 --2- 172.21.15.43:0/3413176698 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fdec96b230 0x55fdec9950b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:43.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.440+0000 7f4779782740 1 --2- 172.21.15.43:0/3413176698 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55fdec967770 0x55fdec96db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:43.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.440+0000 7f4779782740 1 --2- 172.21.15.43:0/3413176698 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdec967140 0x55fdec96f200 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:43.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.440+0000 7f4779782740 1 -- 172.21.15.43:0/3413176698 >> 172.21.15.43:0/3413176698 conn(0x55fdec940f10 msgr2=0x55fdec96e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:43.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.440+0000 7f4779782740 1 -- 172.21.15.43:0/3413176698 shutdown_connections 2024-04-25T09:01:43.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.440+0000 7f4779782740 1 -- 172.21.15.43:0/3413176698 wait complete. 2024-04-25T09:01:43.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.440+0000 7f4779782740 1 -- start start 2024-04-25T09:01:43.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.441+0000 7f4779782740 1 Processor -- start 2024-04-25T09:01:43.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.441+0000 7f4779782740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdec967140 0x55fdec9fd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:43.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.441+0000 7f4779782740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fdec967770 0x55fdec93a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:43.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.441+0000 7f4779782740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55fdec96b230 0x55fdec987ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:43.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.441+0000 7f4779782740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdec96ce90 con 0x55fdec967140 2024-04-25T09:01:43.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.441+0000 7f4779782740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdec96ee30 con 0x55fdec967770 2024-04-25T09:01:43.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.441+0000 7f4779782740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fdec997620 con 0x55fdec96b230 2024-04-25T09:01:43.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.441+0000 7f4779782740 10 client.0 _ll_register_callbacks cb 0x55fdec93bc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:01:43.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.442+0000 7f476ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdec967140 0x55fdec9fd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:43.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.442+0000 7f476ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdec967140 0x55fdec9fd280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42584/0 (socket says 172.21.15.43:42584) 2024-04-25T09:01:43.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.442+0000 7f476ffff640 1 -- 172.21.15.43:0/192541190 learned_addr learned my addr 172.21.15.43:0/192541190 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:43.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.442+0000 7f476f7fe640 1 --2- 172.21.15.43:0/192541190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fdec967770 0x55fdec93a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:43.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.442+0000 7f4774f48640 1 --2- 172.21.15.43:0/192541190 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55fdec96b230 0x55fdec987ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:43.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.442+0000 7f476ffff640 1 -- 172.21.15.43:0/192541190 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55fdec96b230 msgr2=0x55fdec987ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:43.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.442+0000 7f476ffff640 1 --2- 172.21.15.43:0/192541190 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55fdec96b230 0x55fdec987ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:43.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.442+0000 7f476ffff640 1 -- 172.21.15.43:0/192541190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fdec967770 msgr2=0x55fdec93a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:43.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.442+0000 7f476ffff640 1 --2- 172.21.15.43:0/192541190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fdec967770 0x55fdec93a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:43.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.442+0000 7f476ffff640 1 -- 172.21.15.43:0/192541190 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fdec9a9840 con 0x55fdec967140 2024-04-25T09:01:43.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.442+0000 7f4774f48640 1 --2- 172.21.15.43:0/192541190 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55fdec96b230 0x55fdec987ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:01:43.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.442+0000 7f476ffff640 1 --2- 172.21.15.43:0/192541190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdec967140 0x55fdec9fd280 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f4760007b90 tx=0x7f4760000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9be0773cec203c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:43.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.442+0000 7f474effd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:01:43.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.442+0000 7f474effd640 1 -- 172.21.15.43:0/192541190 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4760033070 con 0x55fdec967140 2024-04-25T09:01:43.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.442+0000 7f474effd640 1 -- 172.21.15.43:0/192541190 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4760006d20 con 0x55fdec967140 2024-04-25T09:01:43.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.442+0000 7f4779782740 1 -- 172.21.15.43:0/192541190 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4760027040 con 0x55fdec967140 2024-04-25T09:01:43.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.442+0000 7f474effd640 1 -- 172.21.15.43:0/192541190 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4760047050 con 0x55fdec967140 2024-04-25T09:01:43.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.442+0000 7f4779782740 10 client.8259 fetch_fsmap learned FSMap version 61 2024-04-25T09:01:43.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.442+0000 7f4779782740 1 -- 172.21.15.43:0/192541190 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x55fdec93ad20 con 0x55fdec967140 2024-04-25T09:01:43.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.443+0000 7f474effd640 1 -- 172.21.15.43:0/192541190 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4760027040 con 0x55fdec967140 2024-04-25T09:01:43.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.443+0000 7f474effd640 1 -- 172.21.15.43:0/192541190 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 61) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f476004a020 con 0x55fdec967140 2024-04-25T09:01:43.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.443+0000 7f4779782740 10 client.8259 fetch_fsmap finished waiting for FSMap version 61 2024-04-25T09:01:43.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.443+0000 7f4779782740 10 client.8259 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T09:01:43.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.443+0000 7f4779782740 20 client.8259 populate_metadata read hostname 'smithi043' 2024-04-25T09:01:43.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.443+0000 7f4779782740 1 --2- 172.21.15.43:0/192541190 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55fdec988ad0 0x55fdec969410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:43.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.443+0000 7f4779782740 4 client.8259 mds_command: new command op to 8094 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:01:43.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.443+0000 7f4779782740 1 -- 172.21.15.43:0/192541190 --> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fdec8b0f50 con 0x55fdec988ad0 2024-04-25T09:01:43.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.443+0000 7f476f7fe640 1 --2- 172.21.15.43:0/192541190 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55fdec988ad0 0x55fdec969410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:43.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.443+0000 7f476f7fe640 1 --2- 172.21.15.43:0/192541190 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55fdec988ad0 0x55fdec969410 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=217a16dfa6ffa0be server_cookie=3088c707856313c0 in_seq=0 out_seq=0 2024-04-25T09:01:43.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.443+0000 7f474effd640 10 client.8259 ms_handle_connect on v2:172.21.15.193:6834/853943267 2024-04-25T09:01:43.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.444+0000 7f474effd640 1 -- 172.21.15.43:0/192541190 <== mds.0 v2:172.21.15.193:6834/853943267 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55fdec8b0f50 con 0x55fdec988ad0 2024-04-25T09:01:43.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.444+0000 7f474effd640 10 client.8259 handle_command_reply: tid=0 2024-04-25T09:01:43.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.444+0000 7f4779782740 1 client.8259 shutdown 2024-04-25T09:01:43.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.444+0000 7f4779782740 1 -- 172.21.15.43:0/192541190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdec967140 msgr2=0x55fdec9fd280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:43.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.444+0000 7f4779782740 1 --2- 172.21.15.43:0/192541190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdec967140 0x55fdec9fd280 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f4760007b90 tx=0x7f4760000bc0 comp rx=0 tx=0).stop 2024-04-25T09:01:43.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.444+0000 7f4779782740 1 -- 172.21.15.43:0/192541190 shutdown_connections 2024-04-25T09:01:43.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.444+0000 7f4779782740 1 --2- 172.21.15.43:0/192541190 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55fdec96b230 0x55fdec987ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:43.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.444+0000 7f4779782740 1 --2- 172.21.15.43:0/192541190 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55fdec988ad0 0x55fdec969410 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:43.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.444+0000 7f4779782740 1 --2- 172.21.15.43:0/192541190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55fdec967770 0x55fdec93a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:43.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.444+0000 7f4779782740 1 --2- 172.21.15.43:0/192541190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55fdec967140 0x55fdec9fd280 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:43.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.444+0000 7f474effd640 0 client.8259 ms_handle_reset on v2:172.21.15.193:6834/853943267 2024-04-25T09:01:43.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.444+0000 7f4779782740 1 -- 172.21.15.43:0/192541190 >> 172.21.15.43:0/192541190 conn(0x55fdec940f10 msgr2=0x55fdec96e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:43.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.444+0000 7f4779782740 1 -- 172.21.15.43:0/192541190 shutdown_connections 2024-04-25T09:01:43.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.444+0000 7f4779782740 1 -- 172.21.15.43:0/192541190 wait complete. 2024-04-25T09:01:43.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.444+0000 7f4779782740 20 client.8259 trim_cache size 0 max 16384 2024-04-25T09:01:43.439 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:01:43.440 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "6c4bd50e-aa00-4a7f-8d09-352344ef5a4b", 2024-04-25T09:01:43.440 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-04-25T09:01:43.440 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 8094, 2024-04-25T09:01:43.440 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-04-25T09:01:43.440 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-04-25T09:01:43.440 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-04-25T09:01:43.440 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 9.0876999000000005, 2024-04-25T09:01:43.440 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 61, 2024-04-25T09:01:43.440 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 102, 2024-04-25T09:01:43.440 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 102, 2024-04-25T09:01:43.440 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 14.095419294999999 2024-04-25T09:01:43.440 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:01:43.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.444+0000 7f47769ab640 1 -- 172.21.15.43:0/2465348334 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f475c06b4d0 msgr2=0x7f475c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:43.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.444+0000 7f47769ab640 1 --2- 172.21.15.43:0/2465348334 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f475c06b4d0 0x7f475c06d990 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f47600016e0 tx=0x7f4760009040 comp rx=0 tx=0).stop 2024-04-25T09:01:43.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.444+0000 7f47769ab640 1 -- 172.21.15.43:0/2465348334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f477015df50 msgr2=0x7f4770154a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:43.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.444+0000 7f47769ab640 1 --2- 172.21.15.43:0/2465348334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f477015df50 0x7f4770154a40 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f4768034040 tx=0x7f4768004a40 comp rx=0 tx=0).stop 2024-04-25T09:01:43.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.444+0000 7f47769ab640 1 -- 172.21.15.43:0/2465348334 shutdown_connections 2024-04-25T09:01:43.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.444+0000 7f47769ab640 1 --2- 172.21.15.43:0/2465348334 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f477015df50 0x7f4770154a40 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:43.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.444+0000 7f47769ab640 1 --2- 172.21.15.43:0/2465348334 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f475c06b4d0 0x7f475c06d990 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:43.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.444+0000 7f47769ab640 1 --2- 172.21.15.43:0/2465348334 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f47700ce660 0x7f4770157080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:43.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.444+0000 7f47769ab640 1 --2- 172.21.15.43:0/2465348334 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f47700c8d00 0x7f477014c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:43.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.444+0000 7f47769ab640 1 -- 172.21.15.43:0/2465348334 >> 172.21.15.43:0/2465348334 conn(0x7f47700d3a30 msgr2=0x7f4770161710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:43.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.445+0000 7f47769ab640 1 -- 172.21.15.43:0/2465348334 shutdown_connections 2024-04-25T09:01:43.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.445+0000 7f47769ab640 1 -- 172.21.15.43:0/2465348334 wait complete. 2024-04-25T09:01:43.444 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:01:43.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.556+0000 7f8bf72cb640 1 Processor -- start 2024-04-25T09:01:43.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.556+0000 7f8bf72cb640 1 -- start start 2024-04-25T09:01:43.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.557+0000 7f8bf72cb640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8bf015dce0 0x7f8bf015ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:43.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.557+0000 7f8bf72cb640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f8bf00d0af0 0x7f8bf0160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:43.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.557+0000 7f8bf72cb640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bf00cee80 con 0x7f8bf00ce550 2024-04-25T09:01:43.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.557+0000 7f8bf72cb640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bf00d2920 con 0x7f8bf015dce0 2024-04-25T09:01:43.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.557+0000 7f8bf72cb640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bf00cf160 con 0x7f8bf00d0af0 2024-04-25T09:01:43.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.557+0000 7f8bf5067640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8bf015dce0 0x7f8bf015ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:43.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.557+0000 7f8bf5067640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8bf015dce0 0x7f8bf015ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42590/0 (socket says 172.21.15.43:42590) 2024-04-25T09:01:43.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.557+0000 7f8bf5067640 1 -- 172.21.15.43:0/148891235 learned_addr learned my addr 172.21.15.43:0/148891235 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:43.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.557+0000 7f8bf5868640 1 --2- 172.21.15.43:0/148891235 >> v2:172.21.15.193:3300/0 conn(0x7f8bf00d0af0 0x7f8bf0160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:43.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.557+0000 7f8bf5067640 1 -- 172.21.15.43:0/148891235 >> v2:172.21.15.193:3300/0 conn(0x7f8bf00d0af0 msgr2=0x7f8bf0160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:43.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.557+0000 7f8bf5067640 1 --2- 172.21.15.43:0/148891235 >> v2:172.21.15.193:3300/0 conn(0x7f8bf00d0af0 0x7f8bf0160980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:43.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.557+0000 7f8bf5067640 1 -- 172.21.15.43:0/148891235 >> v1:172.21.15.193:6789/0 conn(0x7f8bf00ce550 legacy=0x7f8bf00d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:43.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.557+0000 7f8bf5067640 1 -- 172.21.15.43:0/148891235 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf00cf440 con 0x7f8bf015dce0 2024-04-25T09:01:43.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.558+0000 7f8bf5067640 1 --2- 172.21.15.43:0/148891235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bf015dce0 0x7f8bf015ef50 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f8bf00b5470 tx=0x7f8bec02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17376f8494176344 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:43.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.558+0000 7f8be7fff640 1 -- 172.21.15.43:0/148891235 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bec037070 con 0x7f8bf015dce0 2024-04-25T09:01:43.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.558+0000 7f8be7fff640 1 -- 172.21.15.43:0/148891235 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bec032440 con 0x7f8bf015dce0 2024-04-25T09:01:43.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.558+0000 7f8be7fff640 1 -- 172.21.15.43:0/148891235 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bec032810 con 0x7f8bf015dce0 2024-04-25T09:01:43.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.559+0000 7f8bf72cb640 1 -- 172.21.15.43:0/148891235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bf015dce0 msgr2=0x7f8bf015ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:43.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.559+0000 7f8bf72cb640 1 --2- 172.21.15.43:0/148891235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bf015dce0 0x7f8bf015ef50 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f8bf00b5470 tx=0x7f8bec02a1e0 comp rx=0 tx=0).stop 2024-04-25T09:01:43.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.559+0000 7f8bf72cb640 1 -- 172.21.15.43:0/148891235 shutdown_connections 2024-04-25T09:01:43.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.559+0000 7f8bf72cb640 1 --2- 172.21.15.43:0/148891235 >> v2:172.21.15.193:3300/0 conn(0x7f8bf00d0af0 0x7f8bf0160980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:43.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.559+0000 7f8bf72cb640 1 --2- 172.21.15.43:0/148891235 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bf015dce0 0x7f8bf015ef50 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:43.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.559+0000 7f8bf72cb640 1 -- 172.21.15.43:0/148891235 >> 172.21.15.43:0/148891235 conn(0x7f8bf009cd00 msgr2=0x7f8bf00d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:43.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.559+0000 7f8bf72cb640 1 -- 172.21.15.43:0/148891235 shutdown_connections 2024-04-25T09:01:43.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.559+0000 7f8bf72cb640 1 -- 172.21.15.43:0/148891235 wait complete. 2024-04-25T09:01:43.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.560+0000 7f8bf72cb640 1 Processor -- start 2024-04-25T09:01:43.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.560+0000 7f8bf72cb640 1 -- start start 2024-04-25T09:01:43.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.561+0000 7f8bf72cb640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8bf00ce550 0x7f8bf00cd980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:43.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.561+0000 7f8bf72cb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bf00d0af0 0x7f8bf01528a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:43.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.561+0000 7f8bf72cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bf015dce0 0x7f8bf0151ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:43.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.561+0000 7f8bf72cb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf015e0c0 con 0x7f8bf015dce0 2024-04-25T09:01:43.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.561+0000 7f8bf72cb640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf015f6b0 con 0x7f8bf00d0af0 2024-04-25T09:01:43.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.561+0000 7f8bf72cb640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf00ce9c0 con 0x7f8bf00ce550 2024-04-25T09:01:43.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.561+0000 7f8bf5868640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bf015dce0 0x7f8bf0151ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:43.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.561+0000 7f8bf5868640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bf015dce0 0x7f8bf0151ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:42602/0 (socket says 172.21.15.43:42602) 2024-04-25T09:01:43.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.561+0000 7f8bf5868640 1 -- 172.21.15.43:0/33751785 learned_addr learned my addr 172.21.15.43:0/33751785 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:43.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.561+0000 7f8bf4866640 1 --2- 172.21.15.43:0/33751785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bf00d0af0 0x7f8bf01528a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:43.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.561+0000 7f8bf5067640 1 --2- 172.21.15.43:0/33751785 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8bf00ce550 0x7f8bf00cd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:43.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.561+0000 7f8bf5868640 1 -- 172.21.15.43:0/33751785 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8bf00ce550 msgr2=0x7f8bf00cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:43.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.561+0000 7f8bf5868640 1 --2- 172.21.15.43:0/33751785 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8bf00ce550 0x7f8bf00cd980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:43.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.561+0000 7f8bf5868640 1 -- 172.21.15.43:0/33751785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bf00d0af0 msgr2=0x7f8bf01528a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:43.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.561+0000 7f8bf5868640 1 --2- 172.21.15.43:0/33751785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bf00d0af0 0x7f8bf01528a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:43.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.561+0000 7f8bf5868640 1 -- 172.21.15.43:0/33751785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf00ccf80 con 0x7f8bf015dce0 2024-04-25T09:01:43.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.561+0000 7f8bf4866640 1 --2- 172.21.15.43:0/33751785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bf00d0af0 0x7f8bf01528a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:43.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.561+0000 7f8bf5868640 1 --2- 172.21.15.43:0/33751785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bf015dce0 0x7f8bf0151ec0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f8be800efc0 tx=0x7f8be800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:43.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.561+0000 7f8be5ffb640 1 -- 172.21.15.43:0/33751785 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be8018020 con 0x7f8bf015dce0 2024-04-25T09:01:43.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.561+0000 7f8be5ffb640 1 -- 172.21.15.43:0/33751785 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be8014480 con 0x7f8bf015dce0 2024-04-25T09:01:43.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.561+0000 7f8bf72cb640 1 -- 172.21.15.43:0/33751785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bf00cd260 con 0x7f8bf015dce0 2024-04-25T09:01:43.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.561+0000 7f8be5ffb640 1 -- 172.21.15.43:0/33751785 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be8014870 con 0x7f8bf015dce0 2024-04-25T09:01:43.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.562+0000 7f8bf72cb640 1 -- 172.21.15.43:0/33751785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf00cd540 con 0x7f8bf015dce0 2024-04-25T09:01:43.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.562+0000 7f8bf72cb640 1 -- 172.21.15.43:0/33751785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bf00680f0 con 0x7f8bf015dce0 2024-04-25T09:01:43.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.563+0000 7f8be5ffb640 1 -- 172.21.15.43:0/33751785 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8be8005d10 con 0x7f8bf015dce0 2024-04-25T09:01:43.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.563+0000 7f8be5ffb640 1 --2- 172.21.15.43:0/33751785 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8bd406b4d0 0x7f8bd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:43.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.564+0000 7f8be5ffb640 1 -- 172.21.15.43:0/33751785 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8be808acd0 con 0x7f8bf015dce0 2024-04-25T09:01:43.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.564+0000 7f8bf5067640 1 --2- 172.21.15.43:0/33751785 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8bd406b4d0 0x7f8bd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:43.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.564+0000 7f8bf5067640 1 --2- 172.21.15.43:0/33751785 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8bd406b4d0 0x7f8bd406d990 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f8bec034000 tx=0x7f8bec02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:43.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.568+0000 7f8be5ffb640 1 -- 172.21.15.43:0/33751785 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8be8022070 con 0x7f8bf015dce0 2024-04-25T09:01:43.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.785+0000 7f8bf72cb640 1 -- 172.21.15.43:0/33751785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8bf0074ef0 con 0x7f8bf015dce0 2024-04-25T09:01:43.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.785+0000 7f8be5ffb640 1 -- 172.21.15.43:0/33751785 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v61) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8be8010040 con 0x7f8bf015dce0 2024-04-25T09:01:43.774 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:01:43.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.787+0000 7f8bf72cb640 1 -- 172.21.15.43:0/33751785 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8bd406b4d0 msgr2=0x7f8bd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:43.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.787+0000 7f8bf72cb640 1 --2- 172.21.15.43:0/33751785 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8bd406b4d0 0x7f8bd406d990 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f8bec034000 tx=0x7f8bec02e000 comp rx=0 tx=0).stop 2024-04-25T09:01:43.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.787+0000 7f8bf72cb640 1 -- 172.21.15.43:0/33751785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bf015dce0 msgr2=0x7f8bf0151ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:43.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.787+0000 7f8bf72cb640 1 --2- 172.21.15.43:0/33751785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bf015dce0 0x7f8bf0151ec0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f8be800efc0 tx=0x7f8be800c490 comp rx=0 tx=0).stop 2024-04-25T09:01:43.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.787+0000 7f8bf72cb640 1 -- 172.21.15.43:0/33751785 shutdown_connections 2024-04-25T09:01:43.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.787+0000 7f8bf72cb640 1 --2- 172.21.15.43:0/33751785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8bf015dce0 0x7f8bf0151ec0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:43.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.787+0000 7f8bf72cb640 1 --2- 172.21.15.43:0/33751785 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8bd406b4d0 0x7f8bd406d990 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:43.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.787+0000 7f8bf72cb640 1 --2- 172.21.15.43:0/33751785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8bf00d0af0 0x7f8bf01528a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:43.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.787+0000 7f8bf72cb640 1 --2- 172.21.15.43:0/33751785 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8bf00ce550 0x7f8bf00cd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:43.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.787+0000 7f8bf72cb640 1 -- 172.21.15.43:0/33751785 >> 172.21.15.43:0/33751785 conn(0x7f8bf009cd00 msgr2=0x7f8bf00c74a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:43.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.787+0000 7f8bf72cb640 1 -- 172.21.15.43:0/33751785 shutdown_connections 2024-04-25T09:01:43.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:43.787+0000 7f8bf72cb640 1 -- 172.21.15.43:0/33751785 wait complete. 2024-04-25T09:01:43.787 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T09:01:43.787 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:01:43.787 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-04-25T09:01:43.787 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:01:43.788 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:01:43.788 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T09:01:43.788 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T09:01:43.788 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:01:43.788 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:01:43.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:43.788 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-04-25T09:01:43.804 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:01:43.804 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:01:43.804 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:01:43.804 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:01:43.805 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-04-25T09:01:43.805 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:01:43.805 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:01:43.805 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-04-25T09:01:43.805 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-04-25T09:01:43.805 INFO:teuthology.orchestra.run.smithi043.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:01:43.805 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:01:43.805 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-04-25T09:01:43.805 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T09:01:43.805 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 42314sec preferred_lft 42314sec 2024-04-25T09:01:43.805 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link noprefixroute 2024-04-25T09:01:43.805 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:01:43.806 INFO:teuthology.orchestra.run.smithi043.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:01:43.806 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-04-25T09:01:43.806 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-04-25T09:01:43.806 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:01:43.806 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:01:43.806 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-04-25T09:01:43.806 INFO:teuthology.orchestra.run.smithi043.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:01:43.806 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 12:2c:7a:72:36:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:01:43.806 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:01:43.806 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:01:43.806 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ccff:4eff:fe9d:c8bf/64 scope link 2024-04-25T09:01:43.806 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:01:43.806 INFO:teuthology.orchestra.run.smithi043.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:01:43.807 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 16:9a:63:37:46:24 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:01:43.807 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::149a:63ff:fe37:4624/64 scope link 2024-04-25T09:01:43.807 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:01:43.807 INFO:teuthology.orchestra.run.smithi043.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:01:43.807 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 12:2c:7a:72:36:14 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:01:43.807 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::102c:7aff:fe72:3614/64 scope link 2024-04-25T09:01:43.807 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:01:43.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:43.807 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:01:43.868 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-04-25T09:01:43.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:43.869 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-04-25T09:01:43.927 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-04-25T09:01:43.927 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:01:43.927 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T09:01:43.927 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T09:01:43.927 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:01:43.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:43.928 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2024-04-25T09:01:43.928 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:01:43.928 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T09:01:43.929 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T09:01:43.929 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T09:01:43.929 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:01:44.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:44.014 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-04-25T09:01:44.029 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:01:44.029 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:01:44.029 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:01:44.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:44.029 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-04-25T09:01:44.093 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T09:01:44.093 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:01:44.093 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:01:44.149 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:01:44.149 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:01:44.149 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:01:44.205 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-04-25T09:01:44.270 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:01:44.326 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-04-25T09:01:44.326 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-04-25T09:01:44.326 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-04-25T09:01:44.326 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-04-25T09:01:44.326 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-04-25T09:01:44.326 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-04-25T09:01:44.326 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-04-25T09:01:44.326 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-04-25T09:01:44.327 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-04-25T09:01:44.327 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-04-25T09:01:44.327 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-04-25T09:01:44.327 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/selinux 2024-04-25T09:01:44.327 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:01:44.327 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-04-25T09:01:44.327 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-04-25T09:01:44.327 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-04-25T09:01:44.327 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-04-25T09:01:44.327 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:01:44.327 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-04-25T09:01:44.327 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-04-25T09:01:44.327 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T09:01:44.327 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T09:01:44.327 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T09:01:44.328 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T09:01:44.328 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-04-25T09:01:44.328 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-04-25T09:01:44.328 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:01:44.328 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:01:44.328 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:01:44.328 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:01:44.328 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-04-25T09:01:44.328 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:01:44.328 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:01:44.328 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:01:44.328 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:01:44.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:44.329 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-04-25T09:01:44.382 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T09:01:44.382 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T09:01:44.425 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-04-25T09:01:44.449 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:01:44.506 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-04-25T09:01:44.506 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-04-25T09:01:44.506 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-04-25T09:01:44.506 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-04-25T09:01:44.506 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-04-25T09:01:44.506 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-04-25T09:01:44.507 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-04-25T09:01:44.507 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-04-25T09:01:44.507 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-04-25T09:01:44.507 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-04-25T09:01:44.507 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-04-25T09:01:44.507 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/selinux 2024-04-25T09:01:44.507 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:01:44.507 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-04-25T09:01:44.507 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-04-25T09:01:44.507 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-04-25T09:01:44.507 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-04-25T09:01:44.508 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:01:44.508 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-04-25T09:01:44.508 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-04-25T09:01:44.508 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T09:01:44.508 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T09:01:44.508 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T09:01:44.508 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T09:01:44.508 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-04-25T09:01:44.508 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-04-25T09:01:44.508 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:01:44.508 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:01:44.508 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:01:44.509 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:01:44.509 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-04-25T09:01:44.509 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:01:44.509 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:01:44.509 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:01:44.509 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:01:44.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:44.509 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-04-25T09:01:44.510 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:2024-04-25T09:01:44.518+0000 7fbd18dee480 -1 init, newargv = 0x55cc41819a40 newargc=15 2024-04-25T09:01:44.510 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[81338]: starting ceph client 2024-04-25T09:01:44.512 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[81338]: starting fuse 2024-04-25T09:01:44.525 INFO:teuthology.orchestra.run.smithi043.stdout:43 2024-04-25T09:01:44.526 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-25T09:01:44.526 DEBUG:teuthology.orchestra.run.smithi043:> sudo stdin-killer -- python3 -c ' 2024-04-25T09:01:44.526 DEBUG:teuthology.orchestra.run.smithi043:> import glob 2024-04-25T09:01:44.526 DEBUG:teuthology.orchestra.run.smithi043:> import re 2024-04-25T09:01:44.526 DEBUG:teuthology.orchestra.run.smithi043:> import os 2024-04-25T09:01:44.526 DEBUG:teuthology.orchestra.run.smithi043:> import subprocess 2024-04-25T09:01:44.526 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T09:01:44.526 DEBUG:teuthology.orchestra.run.smithi043:> def _find_admin_socket(client_name): 2024-04-25T09:01:44.526 DEBUG:teuthology.orchestra.run.smithi043:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T09:01:44.526 DEBUG:teuthology.orchestra.run.smithi043:> files = glob.glob(asok_path) 2024-04-25T09:01:44.527 DEBUG:teuthology.orchestra.run.smithi043:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T09:01:44.527 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T09:01:44.527 DEBUG:teuthology.orchestra.run.smithi043:> # Given a non-glob path, it better be there 2024-04-25T09:01:44.527 DEBUG:teuthology.orchestra.run.smithi043:> if "*" not in asok_path: 2024-04-25T09:01:44.527 DEBUG:teuthology.orchestra.run.smithi043:> assert(len(files) == 1) 2024-04-25T09:01:44.527 DEBUG:teuthology.orchestra.run.smithi043:> return files[0] 2024-04-25T09:01:44.527 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T09:01:44.527 DEBUG:teuthology.orchestra.run.smithi043:> for f in files: 2024-04-25T09:01:44.527 DEBUG:teuthology.orchestra.run.smithi043:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T09:01:44.527 DEBUG:teuthology.orchestra.run.smithi043:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T09:01:44.527 DEBUG:teuthology.orchestra.run.smithi043:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T09:01:44.527 DEBUG:teuthology.orchestra.run.smithi043:> contents = proc_f.read() 2024-04-25T09:01:44.527 DEBUG:teuthology.orchestra.run.smithi043:> if mountpoint in contents: 2024-04-25T09:01:44.527 DEBUG:teuthology.orchestra.run.smithi043:> return f 2024-04-25T09:01:44.527 DEBUG:teuthology.orchestra.run.smithi043:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T09:01:44.527 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T09:01:44.528 DEBUG:teuthology.orchestra.run.smithi043:> print(_find_admin_socket("client.0")) 2024-04-25T09:01:44.528 DEBUG:teuthology.orchestra.run.smithi043:> ' 2024-04-25T09:01:44.645 INFO:teuthology.orchestra.run.smithi043.stdout:/var/run/ceph/ceph-client.0.81338.asok 2024-04-25T09:01:44.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:01:44.656 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.81338.asok 2024-04-25T09:01:44.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:44.656 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.81338.asok status 2024-04-25T09:01:44.793 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:01:44.793 INFO:teuthology.orchestra.run.smithi043.stdout: "metadata": { 2024-04-25T09:01:44.793 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T09:01:44.793 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T09:01:44.793 INFO:teuthology.orchestra.run.smithi043.stdout: "entity_id": "0", 2024-04-25T09:01:44.793 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": "smithi043", 2024-04-25T09:01:44.793 INFO:teuthology.orchestra.run.smithi043.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T09:01:44.793 INFO:teuthology.orchestra.run.smithi043.stdout: "pid": "81338", 2024-04-25T09:01:44.793 INFO:teuthology.orchestra.run.smithi043.stdout: "root": "/" 2024-04-25T09:01:44.793 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:01:44.793 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_count": 0, 2024-04-25T09:01:44.793 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_pinned_count": 0, 2024-04-25T09:01:44.794 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 8271, 2024-04-25T09:01:44.794 INFO:teuthology.orchestra.run.smithi043.stdout: "inst": { 2024-04-25T09:01:44.794 INFO:teuthology.orchestra.run.smithi043.stdout: "name": { 2024-04-25T09:01:44.794 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "client", 2024-04-25T09:01:44.794 INFO:teuthology.orchestra.run.smithi043.stdout: "num": 8271 2024-04-25T09:01:44.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:01:44.794 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-04-25T09:01:44.794 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-04-25T09:01:44.794 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:01:44.794 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1919346982 2024-04-25T09:01:44.794 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:01:44.794 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:01:44.794 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-04-25T09:01:44.794 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-04-25T09:01:44.795 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:01:44.795 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1919346982 2024-04-25T09:01:44.795 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:01:44.795 INFO:teuthology.orchestra.run.smithi043.stdout: "inst_str": "client.8271 192.168.0.1:0/1919346982", 2024-04-25T09:01:44.795 INFO:teuthology.orchestra.run.smithi043.stdout: "addr_str": "192.168.0.1:0/1919346982", 2024-04-25T09:01:44.795 INFO:teuthology.orchestra.run.smithi043.stdout: "inode_count": 1, 2024-04-25T09:01:44.795 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_epoch": 61, 2024-04-25T09:01:44.795 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch": 102, 2024-04-25T09:01:44.795 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch_barrier": 0, 2024-04-25T09:01:44.795 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklisted": false, 2024-04-25T09:01:44.795 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs" 2024-04-25T09:01:44.795 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:01:44.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:44.803 DEBUG:teuthology.orchestra.run.smithi043:> stat --file-system '--printf=%T 2024-04-25T09:01:44.803 DEBUG:teuthology.orchestra.run.smithi043:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T09:01:44.818 INFO:teuthology.orchestra.run.smithi043.stdout:fuseblk 2024-04-25T09:01:44.819 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T09:01:44.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:44.819 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:01:44.885 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2024-04-25T09:01:44.885 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_control_test_path 2024-04-25T09:01:44.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:44.885 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:01:44.885 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:01:44.885 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T09:01:44.885 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:01:44.885 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:01:44.885 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:01:44.885 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:01:44.968 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:44.968 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:44.968 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0104707 s, 100 MB/s 2024-04-25T09:01:44.981 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:44.982 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:44.982 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106431 s, 98.5 MB/s 2024-04-25T09:01:44.994 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:44.994 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:44.994 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101522 s, 103 MB/s 2024-04-25T09:01:45.006 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.006 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.006 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00911792 s, 115 MB/s 2024-04-25T09:01:45.018 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.018 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.018 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00867495 s, 121 MB/s 2024-04-25T09:01:45.028 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.028 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.028 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754553 s, 139 MB/s 2024-04-25T09:01:45.040 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.040 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.040 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00904814 s, 116 MB/s 2024-04-25T09:01:45.051 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.051 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.051 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0086621 s, 121 MB/s 2024-04-25T09:01:45.062 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.062 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.062 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00905636 s, 116 MB/s 2024-04-25T09:01:45.073 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.073 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.073 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0081502 s, 129 MB/s 2024-04-25T09:01:45.086 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.086 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.086 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830716 s, 126 MB/s 2024-04-25T09:01:45.097 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.097 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.097 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100229 s, 105 MB/s 2024-04-25T09:01:45.108 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.108 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.108 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861983 s, 122 MB/s 2024-04-25T09:01:45.119 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.119 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.119 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0086265 s, 122 MB/s 2024-04-25T09:01:45.130 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.131 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.131 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0087611 s, 120 MB/s 2024-04-25T09:01:45.141 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.141 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.141 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776541 s, 135 MB/s 2024-04-25T09:01:45.152 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.152 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.152 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0080458 s, 130 MB/s 2024-04-25T09:01:45.162 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.162 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.162 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075993 s, 138 MB/s 2024-04-25T09:01:45.174 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.174 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.174 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00960577 s, 109 MB/s 2024-04-25T09:01:45.186 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.186 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.187 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00926387 s, 113 MB/s 2024-04-25T09:01:45.197 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.197 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.197 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829026 s, 126 MB/s 2024-04-25T09:01:45.207 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.207 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.207 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748218 s, 140 MB/s 2024-04-25T09:01:45.218 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.218 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.218 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00799147 s, 131 MB/s 2024-04-25T09:01:45.229 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.229 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.229 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00888065 s, 118 MB/s 2024-04-25T09:01:45.241 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.241 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.241 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00905371 s, 116 MB/s 2024-04-25T09:01:45.251 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.251 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.251 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728646 s, 144 MB/s 2024-04-25T09:01:45.263 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.263 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.263 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00930994 s, 113 MB/s 2024-04-25T09:01:45.275 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.275 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.275 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00930195 s, 113 MB/s 2024-04-25T09:01:45.284 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.285 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.285 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721855 s, 145 MB/s 2024-04-25T09:01:45.295 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.295 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.295 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00795481 s, 132 MB/s 2024-04-25T09:01:45.306 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.306 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.306 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791268 s, 133 MB/s 2024-04-25T09:01:45.318 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.318 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.318 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818084 s, 128 MB/s 2024-04-25T09:01:45.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:45.318 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:01:45.318 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:01:45.318 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T09:01:45.318 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:01:45.318 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:01:45.319 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:01:45.319 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:01:45.387 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.387 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.388 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00789513 s, 133 MB/s 2024-04-25T09:01:45.397 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.397 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.397 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759004 s, 138 MB/s 2024-04-25T09:01:45.407 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.407 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.407 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00794686 s, 132 MB/s 2024-04-25T09:01:45.418 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.418 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.419 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00880387 s, 119 MB/s 2024-04-25T09:01:45.429 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.430 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.430 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00855258 s, 123 MB/s 2024-04-25T09:01:45.439 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.440 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.440 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758745 s, 138 MB/s 2024-04-25T09:01:45.451 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.451 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.451 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00898521 s, 117 MB/s 2024-04-25T09:01:45.460 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.460 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.461 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743807 s, 141 MB/s 2024-04-25T09:01:45.470 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.470 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.470 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728231 s, 144 MB/s 2024-04-25T09:01:45.481 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.481 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.481 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833946 s, 126 MB/s 2024-04-25T09:01:45.492 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.492 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.492 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00871013 s, 120 MB/s 2024-04-25T09:01:45.503 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.503 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.503 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851729 s, 123 MB/s 2024-04-25T09:01:45.515 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.515 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.515 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00984887 s, 106 MB/s 2024-04-25T09:01:45.527 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.527 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.527 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00918268 s, 114 MB/s 2024-04-25T09:01:45.537 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.537 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.537 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00785437 s, 134 MB/s 2024-04-25T09:01:45.548 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.548 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.548 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811383 s, 129 MB/s 2024-04-25T09:01:45.559 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.559 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.559 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00895543 s, 117 MB/s 2024-04-25T09:01:45.569 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.569 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.569 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760218 s, 138 MB/s 2024-04-25T09:01:45.579 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.579 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.579 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753526 s, 139 MB/s 2024-04-25T09:01:45.589 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.589 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.590 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075259 s, 139 MB/s 2024-04-25T09:01:45.599 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.599 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.599 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00785185 s, 134 MB/s 2024-04-25T09:01:45.611 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.611 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.611 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00978842 s, 107 MB/s 2024-04-25T09:01:45.622 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.622 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.622 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00803947 s, 130 MB/s 2024-04-25T09:01:45.641 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.641 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.641 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164831 s, 63.6 MB/s 2024-04-25T09:01:45.651 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.651 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.651 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075175 s, 139 MB/s 2024-04-25T09:01:45.663 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.663 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.663 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00925479 s, 113 MB/s 2024-04-25T09:01:45.674 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.674 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.674 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00891563 s, 118 MB/s 2024-04-25T09:01:45.685 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.685 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.685 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819971 s, 128 MB/s 2024-04-25T09:01:45.696 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.696 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.696 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00875054 s, 120 MB/s 2024-04-25T09:01:45.707 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.707 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.707 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845168 s, 124 MB/s 2024-04-25T09:01:45.717 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.717 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.717 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772816 s, 136 MB/s 2024-04-25T09:01:45.728 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.728 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.728 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00909109 s, 115 MB/s 2024-04-25T09:01:45.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:45.730 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:01:45.730 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:01:45.730 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T09:01:45.730 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:01:45.730 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:01:45.730 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:01:45.730 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:01:45.798 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.798 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.798 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816275 s, 128 MB/s 2024-04-25T09:01:45.808 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.808 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.808 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763702 s, 137 MB/s 2024-04-25T09:01:45.818 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.818 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.818 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00795201 s, 132 MB/s 2024-04-25T09:01:45.827 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.828 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.828 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719974 s, 146 MB/s 2024-04-25T09:01:45.837 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.837 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.837 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718497 s, 146 MB/s 2024-04-25T09:01:45.847 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.847 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.847 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731951 s, 143 MB/s 2024-04-25T09:01:45.858 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.858 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.858 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836404 s, 125 MB/s 2024-04-25T09:01:45.868 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.868 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.868 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738745 s, 142 MB/s 2024-04-25T09:01:45.877 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.878 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.878 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759883 s, 138 MB/s 2024-04-25T09:01:45.887 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.888 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.888 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770204 s, 136 MB/s 2024-04-25T09:01:45.897 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.897 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.897 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736421 s, 142 MB/s 2024-04-25T09:01:45.909 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.909 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.910 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00902044 s, 116 MB/s 2024-04-25T09:01:45.920 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.920 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.920 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00860574 s, 122 MB/s 2024-04-25T09:01:45.930 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.930 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.930 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815234 s, 129 MB/s 2024-04-25T09:01:45.940 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.940 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.940 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00779695 s, 134 MB/s 2024-04-25T09:01:45.944 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:01:45.946 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:01:45.950 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.950 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.951 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00786533 s, 133 MB/s 2024-04-25T09:01:45.961 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.961 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.961 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826289 s, 127 MB/s 2024-04-25T09:01:45.972 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.972 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.973 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00902752 s, 116 MB/s 2024-04-25T09:01:45.982 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.982 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.982 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778746 s, 135 MB/s 2024-04-25T09:01:45.994 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:45.994 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:45.994 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00926629 s, 113 MB/s 2024-04-25T09:01:46.003 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.003 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.003 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070312 s, 149 MB/s 2024-04-25T09:01:46.015 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.015 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.015 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00901372 s, 116 MB/s 2024-04-25T09:01:46.027 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.027 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.027 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00779074 s, 135 MB/s 2024-04-25T09:01:46.036 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.036 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.036 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00868439 s, 121 MB/s 2024-04-25T09:01:46.046 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.046 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.047 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00858396 s, 122 MB/s 2024-04-25T09:01:46.056 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.056 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.056 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734419 s, 143 MB/s 2024-04-25T09:01:46.066 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.067 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.067 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00805546 s, 130 MB/s 2024-04-25T09:01:46.077 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.077 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.077 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00784208 s, 134 MB/s 2024-04-25T09:01:46.087 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.088 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.088 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00871238 s, 120 MB/s 2024-04-25T09:01:46.099 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.099 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.099 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861827 s, 122 MB/s 2024-04-25T09:01:46.109 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.109 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.109 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770642 s, 136 MB/s 2024-04-25T09:01:46.119 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.119 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.119 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769912 s, 136 MB/s 2024-04-25T09:01:46.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:46.120 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:01:46.120 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:01:46.120 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T09:01:46.120 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:01:46.120 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:01:46.120 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:01:46.120 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:01:46.187 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.187 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.187 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745192 s, 141 MB/s 2024-04-25T09:01:46.197 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.197 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.197 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762978 s, 137 MB/s 2024-04-25T09:01:46.207 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.207 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.207 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773494 s, 136 MB/s 2024-04-25T09:01:46.217 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.217 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.217 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741672 s, 141 MB/s 2024-04-25T09:01:46.230 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.230 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.230 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00786566 s, 133 MB/s 2024-04-25T09:01:46.237 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.237 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.237 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729631 s, 144 MB/s 2024-04-25T09:01:46.247 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.247 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.247 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082906 s, 126 MB/s 2024-04-25T09:01:46.257 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.258 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.258 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773499 s, 136 MB/s 2024-04-25T09:01:46.267 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.267 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.267 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748792 s, 140 MB/s 2024-04-25T09:01:46.277 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.277 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.277 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756503 s, 139 MB/s 2024-04-25T09:01:46.288 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.288 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.288 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833056 s, 126 MB/s 2024-04-25T09:01:46.299 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.299 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.299 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810432 s, 129 MB/s 2024-04-25T09:01:46.310 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.310 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.310 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00896174 s, 117 MB/s 2024-04-25T09:01:46.321 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.322 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.322 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00880897 s, 119 MB/s 2024-04-25T09:01:46.332 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.332 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.332 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00790623 s, 133 MB/s 2024-04-25T09:01:46.342 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.342 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.342 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787539 s, 133 MB/s 2024-04-25T09:01:46.352 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.352 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.352 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00809116 s, 130 MB/s 2024-04-25T09:01:46.363 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.363 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.363 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774453 s, 135 MB/s 2024-04-25T09:01:46.373 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.373 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.373 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787153 s, 133 MB/s 2024-04-25T09:01:46.383 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.383 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.383 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804769 s, 130 MB/s 2024-04-25T09:01:46.397 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.397 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.397 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714337 s, 147 MB/s 2024-04-25T09:01:46.402 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.404 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.404 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737513 s, 142 MB/s 2024-04-25T09:01:46.413 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.413 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.413 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00803846 s, 130 MB/s 2024-04-25T09:01:46.424 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.424 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.424 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830341 s, 126 MB/s 2024-04-25T09:01:46.433 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.433 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.434 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772103 s, 136 MB/s 2024-04-25T09:01:46.444 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.444 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.444 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818086 s, 128 MB/s 2024-04-25T09:01:46.455 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.455 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.455 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00860612 s, 122 MB/s 2024-04-25T09:01:46.467 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.467 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.467 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00897528 s, 117 MB/s 2024-04-25T09:01:46.476 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.476 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.476 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725991 s, 144 MB/s 2024-04-25T09:01:46.486 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.486 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.486 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00790714 s, 133 MB/s 2024-04-25T09:01:46.496 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.496 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.496 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00786496 s, 133 MB/s 2024-04-25T09:01:46.506 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.506 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.506 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751731 s, 139 MB/s 2024-04-25T09:01:46.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:46.507 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:01:46.507 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:01:46.507 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T09:01:46.507 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:01:46.507 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:01:46.507 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:01:46.507 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:01:46.577 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.577 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.577 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00959097 s, 109 MB/s 2024-04-25T09:01:46.588 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.588 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.588 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00879843 s, 119 MB/s 2024-04-25T09:01:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.600 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00921506 s, 114 MB/s 2024-04-25T09:01:46.610 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.610 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.610 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073525 s, 143 MB/s 2024-04-25T09:01:46.621 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.621 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.621 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00895738 s, 117 MB/s 2024-04-25T09:01:46.634 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.634 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.634 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102802 s, 102 MB/s 2024-04-25T09:01:46.646 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.646 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.646 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00974002 s, 108 MB/s 2024-04-25T09:01:46.656 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.656 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.657 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788201 s, 133 MB/s 2024-04-25T09:01:46.666 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.666 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.666 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072679 s, 144 MB/s 2024-04-25T09:01:46.676 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.676 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.676 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740268 s, 142 MB/s 2024-04-25T09:01:46.686 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.686 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.686 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0079396 s, 132 MB/s 2024-04-25T09:01:46.696 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.696 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.696 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710248 s, 148 MB/s 2024-04-25T09:01:46.706 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.706 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.706 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774608 s, 135 MB/s 2024-04-25T09:01:46.716 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.716 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.716 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761266 s, 138 MB/s 2024-04-25T09:01:46.726 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.726 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.726 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760425 s, 138 MB/s 2024-04-25T09:01:46.736 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.736 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.736 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074564 s, 141 MB/s 2024-04-25T09:01:46.746 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.746 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.746 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00789177 s, 133 MB/s 2024-04-25T09:01:46.757 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.757 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.757 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00893297 s, 117 MB/s 2024-04-25T09:01:46.767 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.767 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.767 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774047 s, 135 MB/s 2024-04-25T09:01:46.777 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.777 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.777 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776723 s, 135 MB/s 2024-04-25T09:01:46.787 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.787 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.787 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756324 s, 139 MB/s 2024-04-25T09:01:46.797 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.797 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.797 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754674 s, 139 MB/s 2024-04-25T09:01:46.806 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.807 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.807 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723225 s, 145 MB/s 2024-04-25T09:01:46.818 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.818 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.818 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0089995 s, 117 MB/s 2024-04-25T09:01:46.828 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.828 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.828 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763056 s, 137 MB/s 2024-04-25T09:01:46.839 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.839 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.840 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00904522 s, 116 MB/s 2024-04-25T09:01:46.850 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.850 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.850 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084739 s, 124 MB/s 2024-04-25T09:01:46.860 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.860 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.860 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00798171 s, 131 MB/s 2024-04-25T09:01:46.870 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.870 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.870 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777459 s, 135 MB/s 2024-04-25T09:01:46.881 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.881 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.881 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788191 s, 133 MB/s 2024-04-25T09:01:46.891 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.891 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.891 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818706 s, 128 MB/s 2024-04-25T09:01:46.901 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.901 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.901 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767143 s, 137 MB/s 2024-04-25T09:01:46.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:46.902 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:01:46.902 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:01:46.903 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T09:01:46.903 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:01:46.903 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:01:46.903 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:01:46.903 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:01:46.972 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.972 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.972 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824504 s, 127 MB/s 2024-04-25T09:01:46.982 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.982 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.982 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00785061 s, 134 MB/s 2024-04-25T09:01:46.992 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:46.992 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:46.992 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776171 s, 135 MB/s 2024-04-25T09:01:47.001 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.001 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.001 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727006 s, 144 MB/s 2024-04-25T09:01:47.012 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.012 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.012 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777183 s, 135 MB/s 2024-04-25T09:01:47.021 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.021 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.021 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073221 s, 143 MB/s 2024-04-25T09:01:47.032 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.033 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.033 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00782803 s, 134 MB/s 2024-04-25T09:01:47.041 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.041 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.041 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744997 s, 141 MB/s 2024-04-25T09:01:47.052 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.054 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.054 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00848435 s, 124 MB/s 2024-04-25T09:01:47.061 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.061 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.061 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693644 s, 151 MB/s 2024-04-25T09:01:47.072 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.072 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.072 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826983 s, 127 MB/s 2024-04-25T09:01:47.081 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.081 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.082 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073088 s, 143 MB/s 2024-04-25T09:01:47.093 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.093 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.093 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00878386 s, 119 MB/s 2024-04-25T09:01:47.102 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.102 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.102 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718036 s, 146 MB/s 2024-04-25T09:01:47.113 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.113 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.113 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850829 s, 123 MB/s 2024-04-25T09:01:47.124 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.124 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.124 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0087608 s, 120 MB/s 2024-04-25T09:01:47.134 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.134 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.134 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00785865 s, 133 MB/s 2024-04-25T09:01:47.144 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.144 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.144 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769716 s, 136 MB/s 2024-04-25T09:01:47.153 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.154 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.154 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735132 s, 143 MB/s 2024-04-25T09:01:47.163 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.164 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.164 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756011 s, 139 MB/s 2024-04-25T09:01:47.173 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.173 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.173 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712068 s, 147 MB/s 2024-04-25T09:01:47.182 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.182 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.182 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704059 s, 149 MB/s 2024-04-25T09:01:47.192 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.192 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.192 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780361 s, 134 MB/s 2024-04-25T09:01:47.202 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.202 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.202 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734873 s, 143 MB/s 2024-04-25T09:01:47.212 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.212 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.212 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755978 s, 139 MB/s 2024-04-25T09:01:47.222 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.222 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.222 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761702 s, 138 MB/s 2024-04-25T09:01:47.236 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.236 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.236 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0116337 s, 90.1 MB/s 2024-04-25T09:01:47.246 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.246 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.246 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757461 s, 138 MB/s 2024-04-25T09:01:47.256 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.256 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.256 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00859049 s, 122 MB/s 2024-04-25T09:01:47.266 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.266 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.266 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735718 s, 143 MB/s 2024-04-25T09:01:47.276 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.276 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.276 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749552 s, 140 MB/s 2024-04-25T09:01:47.287 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.287 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.287 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00890204 s, 118 MB/s 2024-04-25T09:01:47.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:47.288 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:01:47.288 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:01:47.289 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T09:01:47.289 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:01:47.289 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:01:47.289 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:01:47.289 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:01:47.356 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.357 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.357 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00782554 s, 134 MB/s 2024-04-25T09:01:47.367 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.368 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.368 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00868148 s, 121 MB/s 2024-04-25T09:01:47.378 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.378 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.379 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00877232 s, 120 MB/s 2024-04-25T09:01:47.388 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.389 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.389 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00782622 s, 134 MB/s 2024-04-25T09:01:47.400 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.400 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.400 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00906861 s, 116 MB/s 2024-04-25T09:01:47.410 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.410 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.410 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720189 s, 146 MB/s 2024-04-25T09:01:47.420 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.420 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.420 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745554 s, 141 MB/s 2024-04-25T09:01:47.431 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.431 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.431 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783015 s, 134 MB/s 2024-04-25T09:01:47.441 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.441 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.441 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772065 s, 136 MB/s 2024-04-25T09:01:47.454 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.454 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.454 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738684 s, 142 MB/s 2024-04-25T09:01:47.462 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.462 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.462 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836706 s, 125 MB/s 2024-04-25T09:01:47.472 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.472 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.472 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754949 s, 139 MB/s 2024-04-25T09:01:47.482 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.482 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.482 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753662 s, 139 MB/s 2024-04-25T09:01:47.492 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.493 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.493 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075512 s, 139 MB/s 2024-04-25T09:01:47.502 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.502 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.502 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734577 s, 143 MB/s 2024-04-25T09:01:47.512 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.512 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.513 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751753 s, 139 MB/s 2024-04-25T09:01:47.523 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.523 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.523 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838401 s, 125 MB/s 2024-04-25T09:01:47.533 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.533 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.533 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678194 s, 155 MB/s 2024-04-25T09:01:47.543 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.543 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.543 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00806808 s, 130 MB/s 2024-04-25T09:01:47.553 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.553 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.553 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696514 s, 151 MB/s 2024-04-25T09:01:47.564 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.564 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.564 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00862765 s, 122 MB/s 2024-04-25T09:01:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.574 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719825 s, 146 MB/s 2024-04-25T09:01:47.584 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.584 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.584 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723434 s, 145 MB/s 2024-04-25T09:01:47.594 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.594 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.594 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751255 s, 140 MB/s 2024-04-25T09:01:47.605 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.605 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.605 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831845 s, 126 MB/s 2024-04-25T09:01:47.616 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.616 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.616 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00844974 s, 124 MB/s 2024-04-25T09:01:47.626 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.626 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.626 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074356 s, 141 MB/s 2024-04-25T09:01:47.636 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.636 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.637 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075522 s, 139 MB/s 2024-04-25T09:01:47.646 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.646 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.646 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770839 s, 136 MB/s 2024-04-25T09:01:47.658 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.658 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.658 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742755 s, 141 MB/s 2024-04-25T09:01:47.667 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.667 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.667 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721657 s, 145 MB/s 2024-04-25T09:01:47.678 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.678 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.678 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00867699 s, 121 MB/s 2024-04-25T09:01:47.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:47.680 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:01:47.680 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:01:47.680 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T09:01:47.680 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:01:47.680 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:01:47.680 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:01:47.680 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:01:47.749 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.749 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.749 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818452 s, 128 MB/s 2024-04-25T09:01:47.759 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.759 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.759 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713843 s, 147 MB/s 2024-04-25T09:01:47.769 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.769 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.769 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073429 s, 143 MB/s 2024-04-25T09:01:47.779 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.779 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.779 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721577 s, 145 MB/s 2024-04-25T09:01:47.789 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.789 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.789 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710023 s, 148 MB/s 2024-04-25T09:01:47.799 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.799 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.799 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075315 s, 139 MB/s 2024-04-25T09:01:47.809 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.809 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.809 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0079687 s, 132 MB/s 2024-04-25T09:01:47.819 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.819 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.819 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755555 s, 139 MB/s 2024-04-25T09:01:47.829 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.829 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.830 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761024 s, 138 MB/s 2024-04-25T09:01:47.841 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.841 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.841 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00931197 s, 113 MB/s 2024-04-25T09:01:47.852 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.852 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.852 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772309 s, 136 MB/s 2024-04-25T09:01:47.863 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.863 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.863 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069687 s, 150 MB/s 2024-04-25T09:01:47.871 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.871 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.871 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069792 s, 150 MB/s 2024-04-25T09:01:47.881 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.881 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.881 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751011 s, 140 MB/s 2024-04-25T09:01:47.892 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.892 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.892 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007744 s, 135 MB/s 2024-04-25T09:01:47.901 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.901 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.902 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710682 s, 148 MB/s 2024-04-25T09:01:47.911 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.912 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.912 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00779344 s, 135 MB/s 2024-04-25T09:01:47.922 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.922 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.922 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723737 s, 145 MB/s 2024-04-25T09:01:47.933 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.933 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.933 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00888242 s, 118 MB/s 2024-04-25T09:01:47.944 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.944 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.944 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829609 s, 126 MB/s 2024-04-25T09:01:47.954 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.954 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.954 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742045 s, 141 MB/s 2024-04-25T09:01:47.964 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.964 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.964 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727634 s, 144 MB/s 2024-04-25T09:01:47.973 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.973 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.973 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706593 s, 148 MB/s 2024-04-25T09:01:47.983 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.984 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.984 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763275 s, 137 MB/s 2024-04-25T09:01:47.993 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:47.993 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:47.993 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070589 s, 149 MB/s 2024-04-25T09:01:48.003 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.003 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.003 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703344 s, 149 MB/s 2024-04-25T09:01:48.014 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.014 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.015 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00875283 s, 120 MB/s 2024-04-25T09:01:48.025 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.025 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.025 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756601 s, 139 MB/s 2024-04-25T09:01:48.034 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.034 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.034 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743892 s, 141 MB/s 2024-04-25T09:01:48.044 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.044 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.044 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740518 s, 142 MB/s 2024-04-25T09:01:48.054 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.054 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.055 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771098 s, 136 MB/s 2024-04-25T09:01:48.067 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.067 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.067 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739186 s, 142 MB/s 2024-04-25T09:01:48.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:48.068 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:01:48.068 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:01:48.068 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T09:01:48.068 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:01:48.068 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:01:48.068 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:01:48.068 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:01:48.136 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.137 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.137 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745422 s, 141 MB/s 2024-04-25T09:01:48.146 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.147 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.147 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748087 s, 140 MB/s 2024-04-25T09:01:48.156 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.156 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.156 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741913 s, 141 MB/s 2024-04-25T09:01:48.166 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.166 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.166 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706112 s, 148 MB/s 2024-04-25T09:01:48.175 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.176 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.176 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723974 s, 145 MB/s 2024-04-25T09:01:48.185 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.185 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.185 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685332 s, 153 MB/s 2024-04-25T09:01:48.196 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.196 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.196 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828892 s, 127 MB/s 2024-04-25T09:01:48.207 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.207 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.207 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834338 s, 126 MB/s 2024-04-25T09:01:48.218 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.218 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.218 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836727 s, 125 MB/s 2024-04-25T09:01:48.229 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.229 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.229 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00813787 s, 129 MB/s 2024-04-25T09:01:48.239 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.239 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.239 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720347 s, 146 MB/s 2024-04-25T09:01:48.249 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.249 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.249 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751281 s, 140 MB/s 2024-04-25T09:01:48.258 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.259 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.259 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726799 s, 144 MB/s 2024-04-25T09:01:48.270 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.270 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.270 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750541 s, 140 MB/s 2024-04-25T09:01:48.280 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.280 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.281 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00866586 s, 121 MB/s 2024-04-25T09:01:48.290 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.290 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.290 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695369 s, 151 MB/s 2024-04-25T09:01:48.300 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.301 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.301 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818524 s, 128 MB/s 2024-04-25T09:01:48.311 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.311 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.311 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754142 s, 139 MB/s 2024-04-25T09:01:48.320 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.320 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.320 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715637 s, 147 MB/s 2024-04-25T09:01:48.330 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.330 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.330 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737336 s, 142 MB/s 2024-04-25T09:01:48.340 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.340 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.340 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723755 s, 145 MB/s 2024-04-25T09:01:48.349 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.349 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.349 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695203 s, 151 MB/s 2024-04-25T09:01:48.360 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.360 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.360 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836726 s, 125 MB/s 2024-04-25T09:01:48.369 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.370 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.370 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738019 s, 142 MB/s 2024-04-25T09:01:48.380 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.380 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.380 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824795 s, 127 MB/s 2024-04-25T09:01:48.390 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.390 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.390 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745473 s, 141 MB/s 2024-04-25T09:01:48.400 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.400 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.400 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741784 s, 141 MB/s 2024-04-25T09:01:48.410 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.410 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.410 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814039 s, 129 MB/s 2024-04-25T09:01:48.419 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.420 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.420 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705884 s, 149 MB/s 2024-04-25T09:01:48.430 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.430 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.431 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833759 s, 126 MB/s 2024-04-25T09:01:48.440 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.440 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.440 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723683 s, 145 MB/s 2024-04-25T09:01:48.450 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.450 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.450 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729802 s, 144 MB/s 2024-04-25T09:01:48.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:48.452 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:01:48.452 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:01:48.452 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T09:01:48.453 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:01:48.453 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:01:48.453 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:01:48.453 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:01:48.521 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.521 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.521 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739589 s, 142 MB/s 2024-04-25T09:01:48.530 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.530 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.530 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707578 s, 148 MB/s 2024-04-25T09:01:48.539 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.539 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.539 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697792 s, 150 MB/s 2024-04-25T09:01:48.549 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.550 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.550 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829776 s, 126 MB/s 2024-04-25T09:01:48.559 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.559 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.559 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709668 s, 148 MB/s 2024-04-25T09:01:48.568 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.568 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.568 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708785 s, 148 MB/s 2024-04-25T09:01:48.577 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.577 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.577 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679157 s, 154 MB/s 2024-04-25T09:01:48.587 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.587 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.587 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747476 s, 140 MB/s 2024-04-25T09:01:48.598 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.598 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.598 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00779257 s, 135 MB/s 2024-04-25T09:01:48.607 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.607 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.607 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689854 s, 152 MB/s 2024-04-25T09:01:48.616 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.617 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.617 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075128 s, 140 MB/s 2024-04-25T09:01:48.627 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.627 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.627 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842596 s, 124 MB/s 2024-04-25T09:01:48.637 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.637 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.637 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072138 s, 145 MB/s 2024-04-25T09:01:48.648 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.648 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.648 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00813327 s, 129 MB/s 2024-04-25T09:01:48.658 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.658 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.658 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804219 s, 130 MB/s 2024-04-25T09:01:48.668 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.668 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.668 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738849 s, 142 MB/s 2024-04-25T09:01:48.678 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.678 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.678 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764123 s, 137 MB/s 2024-04-25T09:01:48.687 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.687 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.687 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718988 s, 146 MB/s 2024-04-25T09:01:48.697 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.697 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.697 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722788 s, 145 MB/s 2024-04-25T09:01:48.707 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.707 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.707 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0081369 s, 129 MB/s 2024-04-25T09:01:48.716 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.716 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.717 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714404 s, 147 MB/s 2024-04-25T09:01:48.726 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.726 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.726 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738546 s, 142 MB/s 2024-04-25T09:01:48.735 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.735 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.736 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679988 s, 154 MB/s 2024-04-25T09:01:48.746 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.746 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.747 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845292 s, 124 MB/s 2024-04-25T09:01:48.757 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.757 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.757 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731169 s, 143 MB/s 2024-04-25T09:01:48.765 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.766 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.766 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703384 s, 149 MB/s 2024-04-25T09:01:48.775 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.775 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.775 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744134 s, 141 MB/s 2024-04-25T09:01:48.785 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.785 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.785 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731035 s, 143 MB/s 2024-04-25T09:01:48.794 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.794 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.794 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709482 s, 148 MB/s 2024-04-25T09:01:48.803 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.804 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.804 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709077 s, 148 MB/s 2024-04-25T09:01:48.813 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.813 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.813 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749599 s, 140 MB/s 2024-04-25T09:01:48.823 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.823 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.823 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709416 s, 148 MB/s 2024-04-25T09:01:48.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:48.824 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:01:48.825 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:01:48.825 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T09:01:48.825 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:01:48.825 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:01:48.825 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:01:48.825 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:01:48.893 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.893 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.893 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823452 s, 127 MB/s 2024-04-25T09:01:48.903 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.904 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.904 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753761 s, 139 MB/s 2024-04-25T09:01:48.913 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.913 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.913 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754798 s, 139 MB/s 2024-04-25T09:01:48.922 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.923 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.923 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678684 s, 155 MB/s 2024-04-25T09:01:48.932 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.932 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.932 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754784 s, 139 MB/s 2024-04-25T09:01:48.943 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.943 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.943 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773375 s, 136 MB/s 2024-04-25T09:01:48.952 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.952 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.953 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755554 s, 139 MB/s 2024-04-25T09:01:48.962 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.962 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.962 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733708 s, 143 MB/s 2024-04-25T09:01:48.972 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.972 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.972 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710552 s, 148 MB/s 2024-04-25T09:01:48.982 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.982 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.982 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834884 s, 126 MB/s 2024-04-25T09:01:48.992 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:48.992 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:48.992 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738547 s, 142 MB/s 2024-04-25T09:01:49.001 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.001 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.001 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690456 s, 152 MB/s 2024-04-25T09:01:49.012 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.012 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.012 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758037 s, 138 MB/s 2024-04-25T09:01:49.021 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.021 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.021 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740813 s, 142 MB/s 2024-04-25T09:01:49.031 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.031 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.032 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828082 s, 127 MB/s 2024-04-25T09:01:49.042 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.042 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.043 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00852667 s, 123 MB/s 2024-04-25T09:01:49.052 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.053 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.053 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758279 s, 138 MB/s 2024-04-25T09:01:49.061 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.062 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.062 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688142 s, 152 MB/s 2024-04-25T09:01:49.072 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.072 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.072 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754569 s, 139 MB/s 2024-04-25T09:01:49.081 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.081 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.081 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673973 s, 156 MB/s 2024-04-25T09:01:49.091 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.091 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.091 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0078809 s, 133 MB/s 2024-04-25T09:01:49.101 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.101 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.101 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072617 s, 144 MB/s 2024-04-25T09:01:49.110 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.111 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.111 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735225 s, 143 MB/s 2024-04-25T09:01:49.119 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.120 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.120 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676615 s, 155 MB/s 2024-04-25T09:01:49.129 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.129 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.129 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736507 s, 142 MB/s 2024-04-25T09:01:49.138 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.138 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.139 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069552 s, 151 MB/s 2024-04-25T09:01:49.148 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.148 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.148 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700088 s, 150 MB/s 2024-04-25T09:01:49.159 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.159 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.159 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845279 s, 124 MB/s 2024-04-25T09:01:49.169 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.169 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.169 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740621 s, 142 MB/s 2024-04-25T09:01:49.180 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.180 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.180 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839935 s, 125 MB/s 2024-04-25T09:01:49.189 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.189 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.189 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070627 s, 148 MB/s 2024-04-25T09:01:49.199 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.199 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.199 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763256 s, 137 MB/s 2024-04-25T09:01:49.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:49.200 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:01:49.200 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:01:49.200 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T09:01:49.200 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:01:49.200 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:01:49.201 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:01:49.201 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:01:49.268 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.269 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.269 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717422 s, 146 MB/s 2024-04-25T09:01:49.278 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.278 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.278 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727784 s, 144 MB/s 2024-04-25T09:01:49.290 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.290 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.290 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00843926 s, 124 MB/s 2024-04-25T09:01:49.299 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.299 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.299 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737909 s, 142 MB/s 2024-04-25T09:01:49.308 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.308 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.308 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730908 s, 143 MB/s 2024-04-25T09:01:49.318 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.318 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.318 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696143 s, 151 MB/s 2024-04-25T09:01:49.327 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.327 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.327 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687618 s, 152 MB/s 2024-04-25T09:01:49.338 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.338 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.338 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732271 s, 143 MB/s 2024-04-25T09:01:49.346 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.346 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.346 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733758 s, 143 MB/s 2024-04-25T09:01:49.357 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.357 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.357 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00849739 s, 123 MB/s 2024-04-25T09:01:49.369 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.369 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.369 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00895155 s, 117 MB/s 2024-04-25T09:01:49.380 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.380 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.380 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00889056 s, 118 MB/s 2024-04-25T09:01:49.392 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.392 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.392 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00917783 s, 114 MB/s 2024-04-25T09:01:49.403 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.403 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.403 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838373 s, 125 MB/s 2024-04-25T09:01:49.419 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.419 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.419 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0138026 s, 76.0 MB/s 2024-04-25T09:01:49.429 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.429 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.429 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693978 s, 151 MB/s 2024-04-25T09:01:49.438 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.439 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.439 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075294 s, 139 MB/s 2024-04-25T09:01:49.448 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.449 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.449 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755758 s, 139 MB/s 2024-04-25T09:01:49.458 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.459 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.459 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743553 s, 141 MB/s 2024-04-25T09:01:49.467 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.467 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.468 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701109 s, 150 MB/s 2024-04-25T09:01:49.477 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.477 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.477 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723336 s, 145 MB/s 2024-04-25T09:01:49.487 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.487 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.487 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734265 s, 143 MB/s 2024-04-25T09:01:49.496 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.496 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.496 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668228 s, 157 MB/s 2024-04-25T09:01:49.506 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.506 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.506 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703672 s, 149 MB/s 2024-04-25T09:01:49.518 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.518 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.518 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00962902 s, 109 MB/s 2024-04-25T09:01:49.529 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.529 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.529 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818708 s, 128 MB/s 2024-04-25T09:01:49.538 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.539 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.539 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752295 s, 139 MB/s 2024-04-25T09:01:49.548 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.548 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.548 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713903 s, 147 MB/s 2024-04-25T09:01:49.557 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.558 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.558 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727813 s, 144 MB/s 2024-04-25T09:01:49.567 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.567 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.567 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719583 s, 146 MB/s 2024-04-25T09:01:49.576 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.577 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.577 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074612 s, 141 MB/s 2024-04-25T09:01:49.587 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.587 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.587 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746971 s, 140 MB/s 2024-04-25T09:01:49.587 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:49.588 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:01:49.588 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:01:49.588 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T09:01:49.588 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:01:49.588 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:01:49.588 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:01:49.588 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:01:49.655 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.656 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.656 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709219 s, 148 MB/s 2024-04-25T09:01:49.665 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.665 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.666 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732629 s, 143 MB/s 2024-04-25T09:01:49.675 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.675 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.675 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744886 s, 141 MB/s 2024-04-25T09:01:49.686 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.686 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.686 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836518 s, 125 MB/s 2024-04-25T09:01:49.695 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.696 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.696 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736079 s, 142 MB/s 2024-04-25T09:01:49.705 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.706 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.706 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774827 s, 135 MB/s 2024-04-25T09:01:49.716 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.717 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.717 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00863335 s, 121 MB/s 2024-04-25T09:01:49.727 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.727 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.727 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753511 s, 139 MB/s 2024-04-25T09:01:49.736 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.736 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.736 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706535 s, 148 MB/s 2024-04-25T09:01:49.746 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.746 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.746 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835812 s, 125 MB/s 2024-04-25T09:01:49.756 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.756 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.756 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074243 s, 141 MB/s 2024-04-25T09:01:49.766 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.766 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.766 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722032 s, 145 MB/s 2024-04-25T09:01:49.776 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.776 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.776 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749993 s, 140 MB/s 2024-04-25T09:01:49.786 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.786 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.786 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743134 s, 141 MB/s 2024-04-25T09:01:49.795 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.795 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.795 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699525 s, 150 MB/s 2024-04-25T09:01:49.804 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.804 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.804 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068448 s, 153 MB/s 2024-04-25T09:01:49.814 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.814 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.814 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00768582 s, 136 MB/s 2024-04-25T09:01:49.825 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.825 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.825 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00863339 s, 121 MB/s 2024-04-25T09:01:49.834 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.834 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.835 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725386 s, 145 MB/s 2024-04-25T09:01:49.844 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.844 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.844 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717632 s, 146 MB/s 2024-04-25T09:01:49.853 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.853 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.853 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683178 s, 153 MB/s 2024-04-25T09:01:49.864 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.864 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.864 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00870143 s, 121 MB/s 2024-04-25T09:01:49.874 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.874 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.874 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732378 s, 143 MB/s 2024-04-25T09:01:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.884 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764854 s, 137 MB/s 2024-04-25T09:01:49.895 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.896 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.896 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769356 s, 136 MB/s 2024-04-25T09:01:49.906 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.906 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.906 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755434 s, 139 MB/s 2024-04-25T09:01:49.914 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.914 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.914 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706184 s, 148 MB/s 2024-04-25T09:01:49.923 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.923 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.923 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714991 s, 147 MB/s 2024-04-25T09:01:49.933 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.933 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.933 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758536 s, 138 MB/s 2024-04-25T09:01:49.943 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.943 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.943 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071511 s, 147 MB/s 2024-04-25T09:01:49.952 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.952 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.952 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688179 s, 152 MB/s 2024-04-25T09:01:49.961 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:49.961 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:49.961 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723285 s, 145 MB/s 2024-04-25T09:01:49.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:49.963 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:01:49.963 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:01:49.963 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T09:01:49.963 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:01:49.963 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:01:49.963 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:01:49.963 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:01:50.032 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.032 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.032 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0086329 s, 121 MB/s 2024-04-25T09:01:50.042 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.042 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.042 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0077201 s, 136 MB/s 2024-04-25T09:01:50.052 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.052 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.052 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774909 s, 135 MB/s 2024-04-25T09:01:50.062 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.062 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.062 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773648 s, 136 MB/s 2024-04-25T09:01:50.074 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.074 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.074 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00886026 s, 118 MB/s 2024-04-25T09:01:50.084 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.084 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.084 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00806053 s, 130 MB/s 2024-04-25T09:01:50.095 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.095 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.095 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828464 s, 127 MB/s 2024-04-25T09:01:50.105 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.105 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.105 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742408 s, 141 MB/s 2024-04-25T09:01:50.114 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.114 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.114 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712465 s, 147 MB/s 2024-04-25T09:01:50.125 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.125 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.125 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841125 s, 125 MB/s 2024-04-25T09:01:50.135 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.135 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.135 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761574 s, 138 MB/s 2024-04-25T09:01:50.146 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.146 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.146 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00880193 s, 119 MB/s 2024-04-25T09:01:50.157 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.157 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.157 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841119 s, 125 MB/s 2024-04-25T09:01:50.166 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.167 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.167 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075252 s, 139 MB/s 2024-04-25T09:01:50.176 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.176 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.176 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684875 s, 153 MB/s 2024-04-25T09:01:50.185 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.185 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.186 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721048 s, 145 MB/s 2024-04-25T09:01:50.195 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.195 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.195 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749912 s, 140 MB/s 2024-04-25T09:01:50.204 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.204 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.204 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683766 s, 153 MB/s 2024-04-25T09:01:50.214 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.214 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.214 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745346 s, 141 MB/s 2024-04-25T09:01:50.225 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.225 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.225 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00868469 s, 121 MB/s 2024-04-25T09:01:50.234 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.234 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.235 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695016 s, 151 MB/s 2024-04-25T09:01:50.244 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.244 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.244 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693306 s, 151 MB/s 2024-04-25T09:01:50.254 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.254 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.254 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736741 s, 142 MB/s 2024-04-25T09:01:50.264 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.264 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.264 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753633 s, 139 MB/s 2024-04-25T09:01:50.273 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.273 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.274 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759692 s, 138 MB/s 2024-04-25T09:01:50.283 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.283 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.283 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715867 s, 146 MB/s 2024-04-25T09:01:50.293 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.293 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.293 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766999 s, 137 MB/s 2024-04-25T09:01:50.303 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.303 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.303 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827412 s, 127 MB/s 2024-04-25T09:01:50.313 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.313 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.313 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734035 s, 143 MB/s 2024-04-25T09:01:50.323 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.323 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.323 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747792 s, 140 MB/s 2024-04-25T09:01:50.332 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.332 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.332 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665909 s, 157 MB/s 2024-04-25T09:01:50.342 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.342 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.342 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745526 s, 141 MB/s 2024-04-25T09:01:50.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:50.343 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:01:50.343 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:01:50.343 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T09:01:50.344 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:01:50.344 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:01:50.344 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:01:50.344 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:01:50.412 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.412 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.412 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697012 s, 150 MB/s 2024-04-25T09:01:50.421 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.421 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.421 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693241 s, 151 MB/s 2024-04-25T09:01:50.431 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.431 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.431 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736239 s, 142 MB/s 2024-04-25T09:01:50.440 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.440 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.441 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740305 s, 142 MB/s 2024-04-25T09:01:50.450 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.450 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.450 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731313 s, 143 MB/s 2024-04-25T09:01:50.460 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.460 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.460 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760444 s, 138 MB/s 2024-04-25T09:01:50.470 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.470 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.470 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829884 s, 126 MB/s 2024-04-25T09:01:50.481 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.481 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.481 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.008319 s, 126 MB/s 2024-04-25T09:01:50.491 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.491 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.491 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741247 s, 141 MB/s 2024-04-25T09:01:50.500 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.501 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.501 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731596 s, 143 MB/s 2024-04-25T09:01:50.510 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.510 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.510 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730628 s, 144 MB/s 2024-04-25T09:01:50.523 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.523 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.523 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00870999 s, 120 MB/s 2024-04-25T09:01:50.538 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.538 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.538 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745046 s, 141 MB/s 2024-04-25T09:01:50.548 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.548 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.548 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752033 s, 139 MB/s 2024-04-25T09:01:50.557 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.558 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.558 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755104 s, 139 MB/s 2024-04-25T09:01:50.566 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.567 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.567 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673821 s, 156 MB/s 2024-04-25T09:01:50.576 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.576 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.576 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713698 s, 147 MB/s 2024-04-25T09:01:50.586 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.587 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.587 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816058 s, 128 MB/s 2024-04-25T09:01:50.596 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.596 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.596 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693884 s, 151 MB/s 2024-04-25T09:01:50.605 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.605 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.605 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726106 s, 144 MB/s 2024-04-25T09:01:50.615 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.615 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.615 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710152 s, 148 MB/s 2024-04-25T09:01:50.624 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.624 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.624 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704603 s, 149 MB/s 2024-04-25T09:01:50.633 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.633 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.634 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664412 s, 158 MB/s 2024-04-25T09:01:50.643 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.643 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.643 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713342 s, 147 MB/s 2024-04-25T09:01:50.653 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.653 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.654 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00843753 s, 124 MB/s 2024-04-25T09:01:50.662 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.663 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.663 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069451 s, 151 MB/s 2024-04-25T09:01:50.672 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.672 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.673 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731923 s, 143 MB/s 2024-04-25T09:01:50.681 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.681 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.681 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692538 s, 151 MB/s 2024-04-25T09:01:50.692 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.693 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.693 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816446 s, 128 MB/s 2024-04-25T09:01:50.701 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.702 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.702 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007065 s, 148 MB/s 2024-04-25T09:01:50.711 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.711 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.711 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692306 s, 151 MB/s 2024-04-25T09:01:50.721 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.721 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.721 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728916 s, 144 MB/s 2024-04-25T09:01:50.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:50.722 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:01:50.722 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:01:50.722 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T09:01:50.722 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:01:50.722 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:01:50.722 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:01:50.723 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:01:50.790 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.791 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.791 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666007 s, 157 MB/s 2024-04-25T09:01:50.799 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.800 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.800 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708684 s, 148 MB/s 2024-04-25T09:01:50.809 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.809 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.809 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702726 s, 149 MB/s 2024-04-25T09:01:50.818 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.818 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.819 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693959 s, 151 MB/s 2024-04-25T09:01:50.828 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.828 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.828 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736721 s, 142 MB/s 2024-04-25T09:01:50.839 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.839 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.839 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00848605 s, 124 MB/s 2024-04-25T09:01:50.848 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.848 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.848 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703717 s, 149 MB/s 2024-04-25T09:01:50.858 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.858 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.858 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703476 s, 149 MB/s 2024-04-25T09:01:50.867 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.867 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.867 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071702 s, 146 MB/s 2024-04-25T09:01:50.877 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.877 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.877 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749473 s, 140 MB/s 2024-04-25T09:01:50.887 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.887 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.887 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0077127 s, 136 MB/s 2024-04-25T09:01:50.896 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.896 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.896 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704769 s, 149 MB/s 2024-04-25T09:01:50.906 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.907 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.907 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00813458 s, 129 MB/s 2024-04-25T09:01:50.916 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.916 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.916 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680747 s, 154 MB/s 2024-04-25T09:01:50.925 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.926 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.926 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747119 s, 140 MB/s 2024-04-25T09:01:50.936 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.936 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.936 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0081521 s, 129 MB/s 2024-04-25T09:01:50.947 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.947 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.947 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0085401 s, 123 MB/s 2024-04-25T09:01:50.958 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.958 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.958 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818475 s, 128 MB/s 2024-04-25T09:01:50.967 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.967 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.967 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685773 s, 153 MB/s 2024-04-25T09:01:50.978 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.978 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.978 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842381 s, 124 MB/s 2024-04-25T09:01:50.989 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.989 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.989 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00886633 s, 118 MB/s 2024-04-25T09:01:50.998 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:50.998 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:50.998 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692574 s, 151 MB/s 2024-04-25T09:01:51.008 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.008 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.008 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733282 s, 143 MB/s 2024-04-25T09:01:51.018 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.018 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.018 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756763 s, 139 MB/s 2024-04-25T09:01:51.028 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.029 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.029 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826215 s, 127 MB/s 2024-04-25T09:01:51.038 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.038 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.039 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734313 s, 143 MB/s 2024-04-25T09:01:51.047 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.048 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.048 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690476 s, 152 MB/s 2024-04-25T09:01:51.058 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.058 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.058 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824985 s, 127 MB/s 2024-04-25T09:01:51.068 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.069 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.069 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00775865 s, 135 MB/s 2024-04-25T09:01:51.079 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.079 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.079 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00812495 s, 129 MB/s 2024-04-25T09:01:51.089 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.089 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.089 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769387 s, 136 MB/s 2024-04-25T09:01:51.099 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.099 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.099 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721544 s, 145 MB/s 2024-04-25T09:01:51.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:51.100 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:01:51.100 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:01:51.100 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T09:01:51.100 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:01:51.100 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:01:51.100 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:01:51.100 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:01:51.168 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.168 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.169 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716489 s, 146 MB/s 2024-04-25T09:01:51.177 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.177 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.177 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065759 s, 159 MB/s 2024-04-25T09:01:51.188 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.188 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.188 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833357 s, 126 MB/s 2024-04-25T09:01:51.197 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.198 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.198 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723351 s, 145 MB/s 2024-04-25T09:01:51.207 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.207 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.207 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744299 s, 141 MB/s 2024-04-25T09:01:51.216 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.217 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.217 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688479 s, 152 MB/s 2024-04-25T09:01:51.226 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.226 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.226 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705105 s, 149 MB/s 2024-04-25T09:01:51.235 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.236 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.236 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724605 s, 145 MB/s 2024-04-25T09:01:51.245 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.245 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.245 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669696 s, 157 MB/s 2024-04-25T09:01:51.255 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.255 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.255 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718737 s, 146 MB/s 2024-04-25T09:01:51.264 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.264 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.264 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734076 s, 143 MB/s 2024-04-25T09:01:51.273 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.274 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.274 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714824 s, 147 MB/s 2024-04-25T09:01:51.284 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.284 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.284 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815765 s, 129 MB/s 2024-04-25T09:01:51.295 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.295 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.295 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817563 s, 128 MB/s 2024-04-25T09:01:51.304 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.304 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.304 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735705 s, 143 MB/s 2024-04-25T09:01:51.314 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.314 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.314 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731014 s, 143 MB/s 2024-04-25T09:01:51.323 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.323 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.323 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678722 s, 154 MB/s 2024-04-25T09:01:51.332 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.332 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.332 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688188 s, 152 MB/s 2024-04-25T09:01:51.342 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.342 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.342 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074245 s, 141 MB/s 2024-04-25T09:01:51.351 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.351 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.351 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721551 s, 145 MB/s 2024-04-25T09:01:51.361 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.361 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.361 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071881 s, 146 MB/s 2024-04-25T09:01:51.371 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.371 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.371 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00768322 s, 136 MB/s 2024-04-25T09:01:51.381 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.382 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.382 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819913 s, 128 MB/s 2024-04-25T09:01:51.391 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.392 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.392 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751047 s, 140 MB/s 2024-04-25T09:01:51.401 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.401 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.401 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731329 s, 143 MB/s 2024-04-25T09:01:51.412 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.412 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.412 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839602 s, 125 MB/s 2024-04-25T09:01:51.422 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.422 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.422 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736054 s, 142 MB/s 2024-04-25T09:01:51.431 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.431 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.431 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679197 s, 154 MB/s 2024-04-25T09:01:51.441 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.441 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.442 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830218 s, 126 MB/s 2024-04-25T09:01:51.451 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.452 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.452 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788287 s, 133 MB/s 2024-04-25T09:01:51.461 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.461 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.461 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695893 s, 151 MB/s 2024-04-25T09:01:51.470 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.470 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.470 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722311 s, 145 MB/s 2024-04-25T09:01:51.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:51.472 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:01:51.472 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:01:51.472 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T09:01:51.472 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:01:51.472 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:01:51.472 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:01:51.472 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:01:51.556 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.556 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.556 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683937 s, 153 MB/s 2024-04-25T09:01:51.565 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.565 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.565 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671371 s, 156 MB/s 2024-04-25T09:01:51.575 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.575 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.576 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762897 s, 137 MB/s 2024-04-25T09:01:51.584 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.584 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.584 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673202 s, 156 MB/s 2024-04-25T09:01:51.594 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.594 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.594 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749342 s, 140 MB/s 2024-04-25T09:01:51.603 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.603 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.603 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678815 s, 154 MB/s 2024-04-25T09:01:51.613 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.613 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.613 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722769 s, 145 MB/s 2024-04-25T09:01:51.622 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.622 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.622 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704484 s, 149 MB/s 2024-04-25T09:01:51.632 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.632 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.632 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749865 s, 140 MB/s 2024-04-25T09:01:51.642 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.642 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.642 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691088 s, 152 MB/s 2024-04-25T09:01:51.651 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.651 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.651 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729059 s, 144 MB/s 2024-04-25T09:01:51.661 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.661 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.661 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702934 s, 149 MB/s 2024-04-25T09:01:51.671 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.671 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.671 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757623 s, 138 MB/s 2024-04-25T09:01:51.681 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.681 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.681 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747589 s, 140 MB/s 2024-04-25T09:01:51.692 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.692 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.692 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00808931 s, 130 MB/s 2024-04-25T09:01:51.702 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.702 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.702 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733407 s, 143 MB/s 2024-04-25T09:01:51.712 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.712 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.712 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710175 s, 148 MB/s 2024-04-25T09:01:51.723 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.723 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.723 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840444 s, 125 MB/s 2024-04-25T09:01:51.733 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.733 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.733 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747651 s, 140 MB/s 2024-04-25T09:01:51.742 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.743 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.743 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694634 s, 151 MB/s 2024-04-25T09:01:51.752 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.752 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.753 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712575 s, 147 MB/s 2024-04-25T09:01:51.764 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.764 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.764 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00909491 s, 115 MB/s 2024-04-25T09:01:51.774 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.774 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.774 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728288 s, 144 MB/s 2024-04-25T09:01:51.784 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.784 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.784 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721789 s, 145 MB/s 2024-04-25T09:01:51.793 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.793 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.793 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677271 s, 155 MB/s 2024-04-25T09:01:51.804 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.804 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.804 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00856927 s, 122 MB/s 2024-04-25T09:01:51.814 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.814 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.814 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705572 s, 149 MB/s 2024-04-25T09:01:51.825 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.825 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.826 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833552 s, 126 MB/s 2024-04-25T09:01:51.835 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.835 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.836 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715388 s, 147 MB/s 2024-04-25T09:01:51.845 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.845 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.846 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826545 s, 127 MB/s 2024-04-25T09:01:51.855 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.855 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.855 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716233 s, 146 MB/s 2024-04-25T09:01:51.867 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.868 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.868 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728342 s, 144 MB/s 2024-04-25T09:01:51.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:51.869 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:01:51.869 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:01:51.869 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T09:01:51.869 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:01:51.870 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:01:51.870 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:01:51.870 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:01:51.899 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.900 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.900 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731642 s, 143 MB/s 2024-04-25T09:01:51.910 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.910 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.910 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763477 s, 137 MB/s 2024-04-25T09:01:51.920 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.920 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.920 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762726 s, 137 MB/s 2024-04-25T09:01:51.931 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.931 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.931 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084871 s, 124 MB/s 2024-04-25T09:01:51.942 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.942 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.942 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073094 s, 143 MB/s 2024-04-25T09:01:51.950 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.950 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.950 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723768 s, 145 MB/s 2024-04-25T09:01:51.960 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.960 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.960 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716061 s, 146 MB/s 2024-04-25T09:01:51.971 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.971 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.971 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829842 s, 126 MB/s 2024-04-25T09:01:51.981 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.981 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.981 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074015 s, 142 MB/s 2024-04-25T09:01:51.992 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:51.992 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:51.992 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819919 s, 128 MB/s 2024-04-25T09:01:52.003 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.003 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.003 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084009 s, 125 MB/s 2024-04-25T09:01:52.012 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.012 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.013 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716033 s, 146 MB/s 2024-04-25T09:01:52.022 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.022 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.022 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738572 s, 142 MB/s 2024-04-25T09:01:52.032 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.032 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.032 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738362 s, 142 MB/s 2024-04-25T09:01:52.043 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.043 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.043 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825372 s, 127 MB/s 2024-04-25T09:01:52.053 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.053 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.053 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724155 s, 145 MB/s 2024-04-25T09:01:52.063 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.063 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.063 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755837 s, 139 MB/s 2024-04-25T09:01:52.073 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.073 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.073 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736476 s, 142 MB/s 2024-04-25T09:01:52.082 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.083 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.083 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730322 s, 144 MB/s 2024-04-25T09:01:52.092 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.093 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.093 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747273 s, 140 MB/s 2024-04-25T09:01:52.102 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.102 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.102 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069025 s, 152 MB/s 2024-04-25T09:01:52.111 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.112 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.112 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715096 s, 147 MB/s 2024-04-25T09:01:52.121 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.121 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.121 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674785 s, 155 MB/s 2024-04-25T09:01:52.132 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.132 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.132 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842969 s, 124 MB/s 2024-04-25T09:01:52.143 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.143 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.143 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00865815 s, 121 MB/s 2024-04-25T09:01:52.153 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.153 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.153 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716206 s, 146 MB/s 2024-04-25T09:01:52.163 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.163 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.164 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811083 s, 129 MB/s 2024-04-25T09:01:52.174 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.174 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.175 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00813462 s, 129 MB/s 2024-04-25T09:01:52.184 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.184 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.184 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686586 s, 153 MB/s 2024-04-25T09:01:52.194 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.194 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.194 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750672 s, 140 MB/s 2024-04-25T09:01:52.204 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.204 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.204 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007388 s, 142 MB/s 2024-04-25T09:01:52.213 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.214 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.214 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705624 s, 149 MB/s 2024-04-25T09:01:52.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:52.215 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:01:52.215 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:01:52.215 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T09:01:52.215 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:01:52.215 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:01:52.215 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:01:52.215 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:01:52.284 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.284 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.284 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730689 s, 144 MB/s 2024-04-25T09:01:52.294 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.294 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.294 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701036 s, 150 MB/s 2024-04-25T09:01:52.303 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.303 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.303 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680043 s, 154 MB/s 2024-04-25T09:01:52.314 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.315 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.315 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00813857 s, 129 MB/s 2024-04-25T09:01:52.324 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.324 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.324 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071721 s, 146 MB/s 2024-04-25T09:01:52.333 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.334 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.334 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705427 s, 149 MB/s 2024-04-25T09:01:52.343 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.344 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.344 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755487 s, 139 MB/s 2024-04-25T09:01:52.354 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.354 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.354 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728588 s, 144 MB/s 2024-04-25T09:01:52.363 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.363 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.363 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719696 s, 146 MB/s 2024-04-25T09:01:52.373 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.373 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.373 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736316 s, 142 MB/s 2024-04-25T09:01:52.384 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.385 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.385 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861177 s, 122 MB/s 2024-04-25T09:01:52.394 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.394 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.394 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689285 s, 152 MB/s 2024-04-25T09:01:52.405 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.405 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.405 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822656 s, 127 MB/s 2024-04-25T09:01:52.414 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.414 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.415 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698856 s, 150 MB/s 2024-04-25T09:01:52.424 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.424 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.424 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724809 s, 145 MB/s 2024-04-25T09:01:52.435 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.435 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.435 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084307 s, 124 MB/s 2024-04-25T09:01:52.444 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.444 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.444 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676163 s, 155 MB/s 2024-04-25T09:01:52.454 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.454 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.454 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741159 s, 141 MB/s 2024-04-25T09:01:52.464 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.464 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.464 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721062 s, 145 MB/s 2024-04-25T09:01:52.473 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.474 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.474 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762282 s, 138 MB/s 2024-04-25T09:01:52.483 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.483 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.483 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753791 s, 139 MB/s 2024-04-25T09:01:52.492 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.492 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.492 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687597 s, 152 MB/s 2024-04-25T09:01:52.502 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.502 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.502 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723089 s, 145 MB/s 2024-04-25T09:01:52.511 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.511 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.511 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701105 s, 150 MB/s 2024-04-25T09:01:52.526 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.526 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.526 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121456 s, 86.3 MB/s 2024-04-25T09:01:52.558 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.558 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.558 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778185 s, 135 MB/s 2024-04-25T09:01:52.567 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.570 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.570 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734535 s, 143 MB/s 2024-04-25T09:01:52.577 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.577 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.577 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753178 s, 139 MB/s 2024-04-25T09:01:52.586 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.587 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.587 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732179 s, 143 MB/s 2024-04-25T09:01:52.596 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.596 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.596 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748688 s, 140 MB/s 2024-04-25T09:01:52.607 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.607 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.607 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828652 s, 127 MB/s 2024-04-25T09:01:52.617 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.617 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.617 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740802 s, 142 MB/s 2024-04-25T09:01:52.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:52.618 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:01:52.619 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:01:52.619 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T09:01:52.619 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:01:52.619 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:01:52.619 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:01:52.619 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:01:52.690 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.690 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.690 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828737 s, 127 MB/s 2024-04-25T09:01:52.700 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.701 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.701 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825612 s, 127 MB/s 2024-04-25T09:01:52.711 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.711 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.711 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00781866 s, 134 MB/s 2024-04-25T09:01:52.720 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.720 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.720 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680936 s, 154 MB/s 2024-04-25T09:01:52.730 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.730 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.731 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774566 s, 135 MB/s 2024-04-25T09:01:52.741 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.741 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.741 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732835 s, 143 MB/s 2024-04-25T09:01:52.750 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.750 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.750 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711663 s, 147 MB/s 2024-04-25T09:01:52.759 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.759 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.759 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703013 s, 149 MB/s 2024-04-25T09:01:52.768 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.768 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.768 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667347 s, 157 MB/s 2024-04-25T09:01:52.778 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.778 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.778 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720547 s, 146 MB/s 2024-04-25T09:01:52.787 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.787 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.787 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741601 s, 141 MB/s 2024-04-25T09:01:52.796 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.796 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.796 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672297 s, 156 MB/s 2024-04-25T09:01:52.806 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.806 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.806 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708878 s, 148 MB/s 2024-04-25T09:01:52.815 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.815 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.815 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676446 s, 155 MB/s 2024-04-25T09:01:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.824 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680933 s, 154 MB/s 2024-04-25T09:01:52.834 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.834 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.834 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746547 s, 140 MB/s 2024-04-25T09:01:52.845 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.845 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.845 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836894 s, 125 MB/s 2024-04-25T09:01:52.855 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.856 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.856 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832884 s, 126 MB/s 2024-04-25T09:01:52.866 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.866 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.867 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716596 s, 146 MB/s 2024-04-25T09:01:52.874 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.875 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.875 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688887 s, 152 MB/s 2024-04-25T09:01:52.884 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.885 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.885 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00806127 s, 130 MB/s 2024-04-25T09:01:52.893 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.894 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.894 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677155 s, 155 MB/s 2024-04-25T09:01:52.904 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.904 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.904 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00800227 s, 131 MB/s 2024-04-25T09:01:52.914 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.915 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.915 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815741 s, 129 MB/s 2024-04-25T09:01:52.924 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.924 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.924 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774388 s, 135 MB/s 2024-04-25T09:01:52.934 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.934 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.934 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739444 s, 142 MB/s 2024-04-25T09:01:52.943 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.943 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.944 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718668 s, 146 MB/s 2024-04-25T09:01:52.953 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.953 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.953 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071511 s, 147 MB/s 2024-04-25T09:01:52.962 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.962 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.962 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729782 s, 144 MB/s 2024-04-25T09:01:52.972 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.972 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.973 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772866 s, 136 MB/s 2024-04-25T09:01:52.982 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.982 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.982 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070701 s, 148 MB/s 2024-04-25T09:01:52.992 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:52.992 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:52.992 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677327 s, 155 MB/s 2024-04-25T09:01:52.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:52.993 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:01:52.993 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:01:52.993 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T09:01:52.993 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:01:52.993 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:01:52.994 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:01:52.994 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:01:53.062 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.062 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.062 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084088 s, 125 MB/s 2024-04-25T09:01:53.071 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.071 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.072 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732401 s, 143 MB/s 2024-04-25T09:01:53.081 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.081 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.081 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693263 s, 151 MB/s 2024-04-25T09:01:53.090 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.090 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.090 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692974 s, 151 MB/s 2024-04-25T09:01:53.100 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.100 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.100 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00784774 s, 134 MB/s 2024-04-25T09:01:53.110 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.110 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.110 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725788 s, 144 MB/s 2024-04-25T09:01:53.119 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.119 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.119 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719604 s, 146 MB/s 2024-04-25T09:01:53.130 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.131 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.131 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00876704 s, 120 MB/s 2024-04-25T09:01:53.140 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.140 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.141 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718693 s, 146 MB/s 2024-04-25T09:01:53.150 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.150 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.150 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755621 s, 139 MB/s 2024-04-25T09:01:53.159 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.159 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.160 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729088 s, 144 MB/s 2024-04-25T09:01:53.169 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.169 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.169 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072427 s, 145 MB/s 2024-04-25T09:01:53.184 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.184 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.184 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717516 s, 146 MB/s 2024-04-25T09:01:53.189 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.189 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.189 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00849429 s, 123 MB/s 2024-04-25T09:01:53.200 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.200 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.200 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711594 s, 147 MB/s 2024-04-25T09:01:53.209 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.209 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.209 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777895 s, 135 MB/s 2024-04-25T09:01:53.218 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.218 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.218 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666843 s, 157 MB/s 2024-04-25T09:01:53.228 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.228 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.228 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816895 s, 128 MB/s 2024-04-25T09:01:53.238 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.238 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.238 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735773 s, 143 MB/s 2024-04-25T09:01:53.247 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.247 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.247 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697392 s, 150 MB/s 2024-04-25T09:01:53.257 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.257 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.257 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721618 s, 145 MB/s 2024-04-25T09:01:53.266 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.267 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.267 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727606 s, 144 MB/s 2024-04-25T09:01:53.277 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.277 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.277 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861008 s, 122 MB/s 2024-04-25T09:01:53.287 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.287 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.287 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753237 s, 139 MB/s 2024-04-25T09:01:53.297 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.297 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.297 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073578 s, 143 MB/s 2024-04-25T09:01:53.306 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.306 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.306 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720111 s, 146 MB/s 2024-04-25T09:01:53.316 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.316 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.316 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732167 s, 143 MB/s 2024-04-25T09:01:53.325 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.326 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.326 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745438 s, 141 MB/s 2024-04-25T09:01:53.336 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.336 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.337 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850026 s, 123 MB/s 2024-04-25T09:01:53.346 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.346 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.346 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724583 s, 145 MB/s 2024-04-25T09:01:53.357 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.358 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.358 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00848179 s, 124 MB/s 2024-04-25T09:01:53.368 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.368 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.368 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687577 s, 153 MB/s 2024-04-25T09:01:53.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:53.368 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:01:53.368 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:01:53.368 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T09:01:53.369 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:01:53.369 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:01:53.369 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:01:53.369 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:01:53.438 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.438 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.438 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748691 s, 140 MB/s 2024-04-25T09:01:53.447 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.448 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.448 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735473 s, 143 MB/s 2024-04-25T09:01:53.458 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.458 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.458 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736698 s, 142 MB/s 2024-04-25T09:01:53.467 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.468 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.468 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741722 s, 141 MB/s 2024-04-25T09:01:53.477 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.478 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.478 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729257 s, 144 MB/s 2024-04-25T09:01:53.488 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.489 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.489 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00837982 s, 125 MB/s 2024-04-25T09:01:53.499 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.499 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.499 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743921 s, 141 MB/s 2024-04-25T09:01:53.509 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.509 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.509 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758984 s, 138 MB/s 2024-04-25T09:01:53.518 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.519 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.519 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732469 s, 143 MB/s 2024-04-25T09:01:53.530 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.530 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.530 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850301 s, 123 MB/s 2024-04-25T09:01:53.583 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.583 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.583 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00782438 s, 134 MB/s 2024-04-25T09:01:53.594 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.594 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.594 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822345 s, 128 MB/s 2024-04-25T09:01:53.604 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.604 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.604 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774147 s, 135 MB/s 2024-04-25T09:01:53.614 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.614 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.615 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682171 s, 154 MB/s 2024-04-25T09:01:53.624 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.624 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.624 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762881 s, 137 MB/s 2024-04-25T09:01:53.633 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.634 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.634 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696217 s, 151 MB/s 2024-04-25T09:01:53.643 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.643 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.643 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710232 s, 148 MB/s 2024-04-25T09:01:53.653 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.653 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.653 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748176 s, 140 MB/s 2024-04-25T09:01:53.663 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.663 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.663 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070487 s, 149 MB/s 2024-04-25T09:01:53.673 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.673 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.673 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751429 s, 140 MB/s 2024-04-25T09:01:53.683 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.683 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.683 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071038 s, 148 MB/s 2024-04-25T09:01:53.694 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.694 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.694 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833747 s, 126 MB/s 2024-04-25T09:01:53.704 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.704 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.704 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749051 s, 140 MB/s 2024-04-25T09:01:53.714 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.715 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.715 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760574 s, 138 MB/s 2024-04-25T09:01:53.726 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.727 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.727 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0086343 s, 121 MB/s 2024-04-25T09:01:53.735 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.735 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.735 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695537 s, 151 MB/s 2024-04-25T09:01:53.746 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.746 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.747 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838837 s, 125 MB/s 2024-04-25T09:01:53.758 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.758 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.758 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00877423 s, 120 MB/s 2024-04-25T09:01:53.767 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.767 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.767 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672705 s, 156 MB/s 2024-04-25T09:01:53.778 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.778 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.778 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826467 s, 127 MB/s 2024-04-25T09:01:53.787 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.788 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.788 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687642 s, 152 MB/s 2024-04-25T09:01:53.797 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.797 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.797 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691509 s, 152 MB/s 2024-04-25T09:01:53.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:53.798 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:01:53.798 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:01:53.798 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T09:01:53.798 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:01:53.799 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:01:53.799 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:01:53.799 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:01:53.867 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.867 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.867 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729862 s, 144 MB/s 2024-04-25T09:01:53.877 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.877 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.877 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738194 s, 142 MB/s 2024-04-25T09:01:53.889 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.889 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.889 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00844326 s, 124 MB/s 2024-04-25T09:01:53.898 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.898 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.898 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719523 s, 146 MB/s 2024-04-25T09:01:53.910 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.910 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.910 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0085203 s, 123 MB/s 2024-04-25T09:01:53.919 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.919 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.920 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072965 s, 144 MB/s 2024-04-25T09:01:53.929 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.930 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.930 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745016 s, 141 MB/s 2024-04-25T09:01:53.939 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.939 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.939 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687496 s, 153 MB/s 2024-04-25T09:01:53.949 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.949 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.949 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071735 s, 146 MB/s 2024-04-25T09:01:53.959 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.959 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.959 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774871 s, 135 MB/s 2024-04-25T09:01:53.969 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.969 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.969 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735775 s, 143 MB/s 2024-04-25T09:01:53.979 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.979 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.979 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745098 s, 141 MB/s 2024-04-25T09:01:53.989 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:53.989 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:53.989 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762772 s, 137 MB/s 2024-04-25T09:01:54.002 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.002 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.003 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00848708 s, 124 MB/s 2024-04-25T09:01:54.010 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.010 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.010 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748498 s, 140 MB/s 2024-04-25T09:01:54.020 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.020 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.021 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761314 s, 138 MB/s 2024-04-25T09:01:54.030 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.030 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.030 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720621 s, 146 MB/s 2024-04-25T09:01:54.040 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.040 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.040 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755421 s, 139 MB/s 2024-04-25T09:01:54.051 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.051 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.051 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751827 s, 139 MB/s 2024-04-25T09:01:54.062 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.062 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.062 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068584 s, 153 MB/s 2024-04-25T09:01:54.071 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.071 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.071 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826726 s, 127 MB/s 2024-04-25T09:01:54.080 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.080 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.080 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674292 s, 156 MB/s 2024-04-25T09:01:54.091 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.091 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.091 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00786643 s, 133 MB/s 2024-04-25T09:01:54.100 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.100 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.100 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710319 s, 148 MB/s 2024-04-25T09:01:54.111 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.111 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.111 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822988 s, 127 MB/s 2024-04-25T09:01:54.121 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.121 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.121 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713831 s, 147 MB/s 2024-04-25T09:01:54.131 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.131 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.131 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744767 s, 141 MB/s 2024-04-25T09:01:54.141 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.141 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.141 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723784 s, 145 MB/s 2024-04-25T09:01:54.151 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.152 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.152 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082527 s, 127 MB/s 2024-04-25T09:01:54.161 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.161 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.162 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699449 s, 150 MB/s 2024-04-25T09:01:54.171 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.171 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.171 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744624 s, 141 MB/s 2024-04-25T09:01:54.180 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.180 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.180 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066184 s, 158 MB/s 2024-04-25T09:01:54.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:54.181 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:01:54.182 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:01:54.182 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T09:01:54.182 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:01:54.182 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:01:54.182 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:01:54.182 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:01:54.250 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.251 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.251 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735513 s, 143 MB/s 2024-04-25T09:01:54.260 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.260 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.260 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739108 s, 142 MB/s 2024-04-25T09:01:54.272 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.272 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.272 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00862937 s, 122 MB/s 2024-04-25T09:01:54.281 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.281 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.281 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701685 s, 149 MB/s 2024-04-25T09:01:54.291 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.291 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.291 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712819 s, 147 MB/s 2024-04-25T09:01:54.301 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.301 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.301 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732676 s, 143 MB/s 2024-04-25T09:01:54.311 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.311 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.311 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708659 s, 148 MB/s 2024-04-25T09:01:54.322 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.322 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.322 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836874 s, 125 MB/s 2024-04-25T09:01:54.331 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.331 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.331 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666843 s, 157 MB/s 2024-04-25T09:01:54.341 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.341 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.341 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757728 s, 138 MB/s 2024-04-25T09:01:54.352 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.352 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.353 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0088969 s, 118 MB/s 2024-04-25T09:01:54.364 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.364 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.370 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834971 s, 126 MB/s 2024-04-25T09:01:54.373 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.373 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.373 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072614 s, 144 MB/s 2024-04-25T09:01:54.384 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.384 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.384 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0088676 s, 118 MB/s 2024-04-25T09:01:54.395 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.395 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.396 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00854835 s, 123 MB/s 2024-04-25T09:01:54.406 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.406 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.406 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826732 s, 127 MB/s 2024-04-25T09:01:54.416 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.416 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.416 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770749 s, 136 MB/s 2024-04-25T09:01:54.426 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.426 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.426 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788763 s, 133 MB/s 2024-04-25T09:01:54.437 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.437 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.437 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00786318 s, 133 MB/s 2024-04-25T09:01:54.446 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.446 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.446 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730021 s, 144 MB/s 2024-04-25T09:01:54.455 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.455 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.456 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705579 s, 149 MB/s 2024-04-25T09:01:54.466 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.466 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.467 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851251 s, 123 MB/s 2024-04-25T09:01:54.477 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.477 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.477 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861174 s, 122 MB/s 2024-04-25T09:01:54.488 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.489 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.489 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0086555 s, 121 MB/s 2024-04-25T09:01:54.499 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.499 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.499 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00837244 s, 125 MB/s 2024-04-25T09:01:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.509 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00782054 s, 134 MB/s 2024-04-25T09:01:54.518 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.519 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.519 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733289 s, 143 MB/s 2024-04-25T09:01:54.528 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.528 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.528 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727295 s, 144 MB/s 2024-04-25T09:01:54.566 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.567 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.567 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00648473 s, 162 MB/s 2024-04-25T09:01:54.576 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.576 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.576 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688589 s, 152 MB/s 2024-04-25T09:01:54.585 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.585 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.585 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071843 s, 146 MB/s 2024-04-25T09:01:54.596 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.596 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.596 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826505 s, 127 MB/s 2024-04-25T09:01:54.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:54.597 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:01:54.597 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:01:54.597 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T09:01:54.597 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:01:54.597 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:01:54.597 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:01:54.597 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:01:54.666 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.667 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.667 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826198 s, 127 MB/s 2024-04-25T09:01:54.677 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.677 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.677 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832443 s, 126 MB/s 2024-04-25T09:01:54.686 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.686 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.686 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708578 s, 148 MB/s 2024-04-25T09:01:54.696 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.696 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.696 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743756 s, 141 MB/s 2024-04-25T09:01:54.706 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.706 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.706 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740216 s, 142 MB/s 2024-04-25T09:01:54.716 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.716 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.716 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765699 s, 137 MB/s 2024-04-25T09:01:54.725 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.726 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.726 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730981 s, 143 MB/s 2024-04-25T09:01:54.735 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.735 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.735 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705336 s, 149 MB/s 2024-04-25T09:01:54.746 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.746 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.746 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819004 s, 128 MB/s 2024-04-25T09:01:54.755 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.755 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.755 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697718 s, 150 MB/s 2024-04-25T09:01:54.765 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.765 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.765 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739451 s, 142 MB/s 2024-04-25T09:01:54.776 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.776 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.776 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00847139 s, 124 MB/s 2024-04-25T09:01:54.785 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.785 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.785 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723059 s, 145 MB/s 2024-04-25T09:01:54.795 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.795 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.795 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712624 s, 147 MB/s 2024-04-25T09:01:54.804 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.804 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.805 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072742 s, 144 MB/s 2024-04-25T09:01:54.814 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.814 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.814 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728204 s, 144 MB/s 2024-04-25T09:01:54.823 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.824 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.824 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727456 s, 144 MB/s 2024-04-25T09:01:54.833 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.833 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.833 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706112 s, 148 MB/s 2024-04-25T09:01:54.843 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.843 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.843 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00779936 s, 134 MB/s 2024-04-25T09:01:54.854 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.854 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.854 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841544 s, 125 MB/s 2024-04-25T09:01:54.862 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.862 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.863 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667483 s, 157 MB/s 2024-04-25T09:01:54.872 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.872 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.872 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680212 s, 154 MB/s 2024-04-25T09:01:54.882 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.882 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.882 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757548 s, 138 MB/s 2024-04-25T09:01:54.891 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.892 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.892 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739521 s, 142 MB/s 2024-04-25T09:01:54.902 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.902 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.902 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824567 s, 127 MB/s 2024-04-25T09:01:54.912 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.912 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.912 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699042 s, 150 MB/s 2024-04-25T09:01:54.921 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.921 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.921 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729167 s, 144 MB/s 2024-04-25T09:01:54.931 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.931 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.931 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074045 s, 142 MB/s 2024-04-25T09:01:54.940 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.941 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.941 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750563 s, 140 MB/s 2024-04-25T09:01:54.950 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.951 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.951 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073814 s, 142 MB/s 2024-04-25T09:01:54.960 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.960 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.960 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719333 s, 146 MB/s 2024-04-25T09:01:54.971 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:54.971 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:54.971 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780658 s, 134 MB/s 2024-04-25T09:01:54.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:54.971 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:01:54.972 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:01:54.972 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T09:01:54.972 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:01:54.972 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:01:54.972 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:01:54.972 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:01:55.039 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.040 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.040 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706851 s, 148 MB/s 2024-04-25T09:01:55.049 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.049 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.049 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716134 s, 146 MB/s 2024-04-25T09:01:55.059 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.059 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.059 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767123 s, 137 MB/s 2024-04-25T09:01:55.070 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.070 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.070 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0087149 s, 120 MB/s 2024-04-25T09:01:55.080 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.080 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.080 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740423 s, 142 MB/s 2024-04-25T09:01:55.090 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.090 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.090 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727862 s, 144 MB/s 2024-04-25T09:01:55.100 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.100 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.100 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765974 s, 137 MB/s 2024-04-25T09:01:55.111 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.111 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.111 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00867605 s, 121 MB/s 2024-04-25T09:01:55.121 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.121 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.121 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827538 s, 127 MB/s 2024-04-25T09:01:55.132 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.132 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.132 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748243 s, 140 MB/s 2024-04-25T09:01:55.140 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.140 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.140 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680652 s, 154 MB/s 2024-04-25T09:01:55.149 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.149 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.150 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731423 s, 143 MB/s 2024-04-25T09:01:55.159 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.159 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.159 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072327 s, 145 MB/s 2024-04-25T09:01:55.169 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.169 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.169 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073113 s, 143 MB/s 2024-04-25T09:01:55.179 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.180 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.180 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00853923 s, 123 MB/s 2024-04-25T09:01:55.189 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.190 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.190 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729864 s, 144 MB/s 2024-04-25T09:01:55.199 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.199 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.199 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706749 s, 148 MB/s 2024-04-25T09:01:55.208 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.208 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.208 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675521 s, 155 MB/s 2024-04-25T09:01:55.218 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.218 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.218 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763459 s, 137 MB/s 2024-04-25T09:01:55.228 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.228 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.228 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764788 s, 137 MB/s 2024-04-25T09:01:55.238 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.238 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.238 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761389 s, 138 MB/s 2024-04-25T09:01:55.247 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.247 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.247 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725271 s, 145 MB/s 2024-04-25T09:01:55.256 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.256 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.256 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677726 s, 155 MB/s 2024-04-25T09:01:55.266 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.266 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.266 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729968 s, 144 MB/s 2024-04-25T09:01:55.276 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.277 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.277 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827684 s, 127 MB/s 2024-04-25T09:01:55.285 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.286 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.286 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681011 s, 154 MB/s 2024-04-25T09:01:55.294 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.295 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.295 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688792 s, 152 MB/s 2024-04-25T09:01:55.304 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.304 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.304 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718052 s, 146 MB/s 2024-04-25T09:01:55.314 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.314 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.314 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730634 s, 144 MB/s 2024-04-25T09:01:55.324 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.325 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.325 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00849402 s, 123 MB/s 2024-04-25T09:01:55.334 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.334 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.334 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746499 s, 140 MB/s 2024-04-25T09:01:55.345 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.345 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.345 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846351 s, 124 MB/s 2024-04-25T09:01:55.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:55.346 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:01:55.347 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:01:55.347 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T09:01:55.347 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:01:55.347 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:01:55.347 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:01:55.347 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:01:55.415 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.415 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.415 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075505 s, 139 MB/s 2024-04-25T09:01:55.426 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.426 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.426 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834444 s, 126 MB/s 2024-04-25T09:01:55.436 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.436 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.436 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726712 s, 144 MB/s 2024-04-25T09:01:55.445 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.446 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.446 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765177 s, 137 MB/s 2024-04-25T09:01:55.455 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.455 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.455 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724436 s, 145 MB/s 2024-04-25T09:01:55.465 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.465 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.465 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749193 s, 140 MB/s 2024-04-25T09:01:55.474 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.474 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.474 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701747 s, 149 MB/s 2024-04-25T09:01:55.484 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.484 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.484 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703411 s, 149 MB/s 2024-04-25T09:01:55.493 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.494 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.494 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072532 s, 145 MB/s 2024-04-25T09:01:55.503 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.503 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.503 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688343 s, 152 MB/s 2024-04-25T09:01:55.514 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.514 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.514 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00895192 s, 117 MB/s 2024-04-25T09:01:55.523 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.523 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.523 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719415 s, 146 MB/s 2024-04-25T09:01:55.533 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.533 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.533 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00768171 s, 137 MB/s 2024-04-25T09:01:55.546 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.546 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.546 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108085 s, 97.0 MB/s 2024-04-25T09:01:55.576 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.576 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.576 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679445 s, 154 MB/s 2024-04-25T09:01:55.585 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.585 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.585 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675132 s, 155 MB/s 2024-04-25T09:01:55.596 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.596 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.597 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845412 s, 124 MB/s 2024-04-25T09:01:55.606 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.607 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.607 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744207 s, 141 MB/s 2024-04-25T09:01:55.617 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.617 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.617 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822728 s, 127 MB/s 2024-04-25T09:01:55.627 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.627 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.627 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665024 s, 158 MB/s 2024-04-25T09:01:55.637 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.637 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.638 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00796288 s, 132 MB/s 2024-04-25T09:01:55.647 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.647 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.647 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735665 s, 143 MB/s 2024-04-25T09:01:55.656 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.656 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.656 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671756 s, 156 MB/s 2024-04-25T09:01:55.666 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.666 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.666 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693442 s, 151 MB/s 2024-04-25T09:01:55.675 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.676 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.676 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728003 s, 144 MB/s 2024-04-25T09:01:55.686 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.686 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.686 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822927 s, 127 MB/s 2024-04-25T09:01:55.696 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.696 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.696 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718274 s, 146 MB/s 2024-04-25T09:01:55.706 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.706 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.706 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728408 s, 144 MB/s 2024-04-25T09:01:55.716 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.716 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.717 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771177 s, 136 MB/s 2024-04-25T09:01:55.727 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.727 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.727 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750689 s, 140 MB/s 2024-04-25T09:01:55.736 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.737 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.737 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733953 s, 143 MB/s 2024-04-25T09:01:55.746 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.746 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.746 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717269 s, 146 MB/s 2024-04-25T09:01:55.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:55.747 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:01:55.747 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:01:55.747 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T09:01:55.748 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:01:55.748 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:01:55.748 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:01:55.748 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:01:55.816 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.816 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.816 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690572 s, 152 MB/s 2024-04-25T09:01:55.826 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.826 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.826 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772857 s, 136 MB/s 2024-04-25T09:01:55.839 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.839 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.839 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747778 s, 140 MB/s 2024-04-25T09:01:55.846 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.847 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.847 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00768393 s, 136 MB/s 2024-04-25T09:01:55.856 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.856 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.856 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696477 s, 151 MB/s 2024-04-25T09:01:55.866 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.866 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.866 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073984 s, 142 MB/s 2024-04-25T09:01:55.876 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.876 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.876 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726927 s, 144 MB/s 2024-04-25T09:01:55.886 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.886 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.886 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728783 s, 144 MB/s 2024-04-25T09:01:55.896 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.896 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.896 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705372 s, 149 MB/s 2024-04-25T09:01:55.906 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.906 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.906 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763946 s, 137 MB/s 2024-04-25T09:01:55.917 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.917 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.917 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00860893 s, 122 MB/s 2024-04-25T09:01:55.927 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.928 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.928 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00775697 s, 135 MB/s 2024-04-25T09:01:55.937 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.937 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.937 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712192 s, 147 MB/s 2024-04-25T09:01:55.947 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.947 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.947 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738888 s, 142 MB/s 2024-04-25T09:01:55.956 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.956 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.957 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683595 s, 153 MB/s 2024-04-25T09:01:55.967 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.967 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.967 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741595 s, 141 MB/s 2024-04-25T09:01:55.977 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.977 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.977 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822987 s, 127 MB/s 2024-04-25T09:01:55.987 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.988 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.988 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757673 s, 138 MB/s 2024-04-25T09:01:55.997 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:55.998 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:55.998 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760111 s, 138 MB/s 2024-04-25T09:01:56.007 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.007 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.008 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712912 s, 147 MB/s 2024-04-25T09:01:56.018 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.018 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.018 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772162 s, 136 MB/s 2024-04-25T09:01:56.027 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.027 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.027 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695435 s, 151 MB/s 2024-04-25T09:01:56.038 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.038 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.038 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719471 s, 146 MB/s 2024-04-25T09:01:56.047 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.048 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.048 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727092 s, 144 MB/s 2024-04-25T09:01:56.059 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.059 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.059 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00874476 s, 120 MB/s 2024-04-25T09:01:56.069 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.070 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.070 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0078186 s, 134 MB/s 2024-04-25T09:01:56.081 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.081 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.082 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767858 s, 137 MB/s 2024-04-25T09:01:56.090 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.090 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.090 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758282 s, 138 MB/s 2024-04-25T09:01:56.100 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.100 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.100 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752463 s, 139 MB/s 2024-04-25T09:01:56.110 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.111 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.111 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743287 s, 141 MB/s 2024-04-25T09:01:56.122 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.122 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.122 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00871179 s, 120 MB/s 2024-04-25T09:01:56.132 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.132 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.132 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733925 s, 143 MB/s 2024-04-25T09:01:56.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:56.133 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:01:56.133 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:01:56.133 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T09:01:56.133 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:01:56.134 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:01:56.134 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:01:56.134 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:01:56.202 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.202 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.202 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723554 s, 145 MB/s 2024-04-25T09:01:56.212 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.212 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.212 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729052 s, 144 MB/s 2024-04-25T09:01:56.222 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.222 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.222 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074324 s, 141 MB/s 2024-04-25T09:01:56.232 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.232 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.232 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746559 s, 140 MB/s 2024-04-25T09:01:56.242 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.242 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.242 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734816 s, 143 MB/s 2024-04-25T09:01:56.253 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.253 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.253 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082748 s, 127 MB/s 2024-04-25T09:01:56.263 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.264 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.264 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842121 s, 125 MB/s 2024-04-25T09:01:56.273 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.273 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.273 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710005 s, 148 MB/s 2024-04-25T09:01:56.283 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.283 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.283 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702459 s, 149 MB/s 2024-04-25T09:01:56.294 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.294 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.294 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770671 s, 136 MB/s 2024-04-25T09:01:56.303 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.303 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.303 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707907 s, 148 MB/s 2024-04-25T09:01:56.313 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.313 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.313 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695355 s, 151 MB/s 2024-04-25T09:01:56.323 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.323 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.323 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748237 s, 140 MB/s 2024-04-25T09:01:56.333 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.333 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.333 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714476 s, 147 MB/s 2024-04-25T09:01:56.343 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.343 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.343 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739555 s, 142 MB/s 2024-04-25T09:01:56.352 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.352 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.352 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698693 s, 150 MB/s 2024-04-25T09:01:56.364 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.364 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.364 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851782 s, 123 MB/s 2024-04-25T09:01:56.373 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.373 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.373 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701063 s, 150 MB/s 2024-04-25T09:01:56.385 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.386 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.386 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740256 s, 142 MB/s 2024-04-25T09:01:56.393 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.393 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.393 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742975 s, 141 MB/s 2024-04-25T09:01:56.404 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.404 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.404 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840506 s, 125 MB/s 2024-04-25T09:01:56.414 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.414 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.414 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728732 s, 144 MB/s 2024-04-25T09:01:56.423 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.424 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.424 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698451 s, 150 MB/s 2024-04-25T09:01:56.433 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.433 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.433 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701322 s, 150 MB/s 2024-04-25T09:01:56.443 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.443 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.443 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747066 s, 140 MB/s 2024-04-25T09:01:56.453 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.453 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.453 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762256 s, 138 MB/s 2024-04-25T09:01:56.464 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.464 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.464 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846387 s, 124 MB/s 2024-04-25T09:01:56.473 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.473 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.473 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701399 s, 149 MB/s 2024-04-25T09:01:56.484 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.484 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.484 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831599 s, 126 MB/s 2024-04-25T09:01:56.494 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.494 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.494 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746742 s, 140 MB/s 2024-04-25T09:01:56.503 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.503 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.504 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710406 s, 148 MB/s 2024-04-25T09:01:56.514 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.514 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.515 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835294 s, 126 MB/s 2024-04-25T09:01:56.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:56.516 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:01:56.516 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:01:56.516 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T09:01:56.516 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:01:56.516 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:01:56.516 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:01:56.516 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:01:56.583 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.584 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.584 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762882 s, 137 MB/s 2024-04-25T09:01:56.593 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.593 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.593 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753642 s, 139 MB/s 2024-04-25T09:01:56.602 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.603 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.603 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666218 s, 157 MB/s 2024-04-25T09:01:56.612 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.612 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.612 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731726 s, 143 MB/s 2024-04-25T09:01:56.621 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.622 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.622 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756268 s, 139 MB/s 2024-04-25T09:01:56.632 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.632 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.632 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00781412 s, 134 MB/s 2024-04-25T09:01:56.641 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.741 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.742 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687696 s, 152 MB/s 2024-04-25T09:01:56.743 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.743 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.743 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750703 s, 140 MB/s 2024-04-25T09:01:56.743 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.743 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.743 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709308 s, 148 MB/s 2024-04-25T09:01:56.743 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.744 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.744 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765486 s, 137 MB/s 2024-04-25T09:01:56.744 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.744 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.744 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00868237 s, 121 MB/s 2024-04-25T09:01:56.744 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.744 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.744 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084755 s, 124 MB/s 2024-04-25T09:01:56.744 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.744 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.744 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746369 s, 140 MB/s 2024-04-25T09:01:56.744 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.744 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.744 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751441 s, 140 MB/s 2024-04-25T09:01:56.744 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.745 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.745 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00887778 s, 118 MB/s 2024-04-25T09:01:56.745 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.745 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.745 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00847845 s, 124 MB/s 2024-04-25T09:01:56.745 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.745 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.745 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773219 s, 136 MB/s 2024-04-25T09:01:56.755 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.755 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.755 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821985 s, 128 MB/s 2024-04-25T09:01:56.765 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.765 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.765 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737775 s, 142 MB/s 2024-04-25T09:01:56.774 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.775 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.775 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748041 s, 140 MB/s 2024-04-25T09:01:56.784 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.784 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.784 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732581 s, 143 MB/s 2024-04-25T09:01:56.794 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.794 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.794 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738111 s, 142 MB/s 2024-04-25T09:01:56.803 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.804 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.804 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718876 s, 146 MB/s 2024-04-25T09:01:56.813 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.813 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.813 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686073 s, 153 MB/s 2024-04-25T09:01:56.823 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.823 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.823 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842734 s, 124 MB/s 2024-04-25T09:01:56.833 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.833 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.833 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718886 s, 146 MB/s 2024-04-25T09:01:56.843 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.843 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.843 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734678 s, 143 MB/s 2024-04-25T09:01:56.852 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.852 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.852 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683558 s, 153 MB/s 2024-04-25T09:01:56.863 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.863 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.863 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00868934 s, 121 MB/s 2024-04-25T09:01:56.872 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.872 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.872 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725335 s, 145 MB/s 2024-04-25T09:01:56.882 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.882 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.882 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727044 s, 144 MB/s 2024-04-25T09:01:56.892 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.892 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.892 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750375 s, 140 MB/s 2024-04-25T09:01:56.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:56.893 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:01:56.893 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:01:56.893 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T09:01:56.893 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:01:56.893 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:01:56.893 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:01:56.894 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:01:56.962 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.962 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.962 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735892 s, 142 MB/s 2024-04-25T09:01:56.971 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.972 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.972 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0079935 s, 131 MB/s 2024-04-25T09:01:56.981 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.981 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.981 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715548 s, 147 MB/s 2024-04-25T09:01:56.991 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:56.991 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:56.992 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819731 s, 128 MB/s 2024-04-25T09:01:57.000 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:57.001 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:57.001 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693814 s, 151 MB/s 2024-04-25T09:01:57.011 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:57.011 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:57.012 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850204 s, 123 MB/s 2024-04-25T09:01:57.020 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:57.021 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:57.021 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697522 s, 150 MB/s 2024-04-25T09:01:57.030 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:57.031 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:57.031 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076121 s, 138 MB/s 2024-04-25T09:01:57.040 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:57.040 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:57.040 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734793 s, 143 MB/s 2024-04-25T09:01:57.051 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:57.051 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:57.051 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00869857 s, 121 MB/s 2024-04-25T09:01:57.062 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:57.062 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:57.062 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845444 s, 124 MB/s 2024-04-25T09:01:57.072 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:57.072 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:57.072 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740276 s, 142 MB/s 2024-04-25T09:01:57.082 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:57.082 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:57.082 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746109 s, 141 MB/s 2024-04-25T09:01:57.091 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:57.091 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:57.091 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068002 s, 154 MB/s 2024-04-25T09:01:57.102 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:57.102 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:57.102 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00864408 s, 121 MB/s 2024-04-25T09:01:57.112 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:57.112 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:57.112 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777174 s, 135 MB/s 2024-04-25T09:01:57.121 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:57.121 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:57.122 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728587 s, 144 MB/s 2024-04-25T09:01:57.131 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:57.131 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:57.131 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720926 s, 145 MB/s 2024-04-25T09:01:57.141 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:57.141 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:57.141 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730206 s, 144 MB/s 2024-04-25T09:01:57.151 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:57.151 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:57.151 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818836 s, 128 MB/s 2024-04-25T09:01:57.161 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:57.161 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:57.161 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720397 s, 146 MB/s 2024-04-25T09:01:57.170 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:57.170 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:57.170 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675624 s, 155 MB/s 2024-04-25T09:01:57.180 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:57.180 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:57.180 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778649 s, 135 MB/s 2024-04-25T09:01:57.191 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:57.191 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:57.191 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840097 s, 125 MB/s 2024-04-25T09:01:57.201 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:57.201 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:57.201 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770814 s, 136 MB/s 2024-04-25T09:01:57.211 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:57.211 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:57.211 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710549 s, 148 MB/s 2024-04-25T09:01:57.219 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:57.220 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:57.220 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670414 s, 156 MB/s 2024-04-25T09:01:57.230 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:57.231 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:57.231 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836535 s, 125 MB/s 2024-04-25T09:01:57.241 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:57.241 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:57.241 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761496 s, 138 MB/s 2024-04-25T09:01:57.250 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:57.250 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:57.250 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754037 s, 139 MB/s 2024-04-25T09:01:57.260 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:57.260 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:57.260 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741488 s, 141 MB/s 2024-04-25T09:01:57.269 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:01:57.269 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:01:57.270 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070327 s, 149 MB/s 2024-04-25T09:01:57.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:57.271 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start /scrub_control_test_path recursive 2024-04-25T09:01:57.422 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.429+0000 7f58273a5640 1 Processor -- start 2024-04-25T09:01:57.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.429+0000 7f58273a5640 1 -- start start 2024-04-25T09:01:57.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.430+0000 7f58273a5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f58200cf400 0x7f58200b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.430+0000 7f58273a5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f58200b13b0 0x7f582006be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.430+0000 7f58273a5640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58200cf8b0 con 0x7f58200ce6f0 2024-04-25T09:01:57.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.430+0000 7f58273a5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58200cfb70 con 0x7f58200cf400 2024-04-25T09:01:57.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.430+0000 7f58273a5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58200cfe50 con 0x7f58200b13b0 2024-04-25T09:01:57.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.430+0000 7f5824940640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f58200cf400 0x7f58200b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.431+0000 7f5824940640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f58200cf400 0x7f58200b0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57758/0 (socket says 172.21.15.43:57758) 2024-04-25T09:01:57.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.431+0000 7f5824940640 1 -- 172.21.15.43:0/90488638 learned_addr learned my addr 172.21.15.43:0/90488638 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:57.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.431+0000 7f5825942640 1 --2- 172.21.15.43:0/90488638 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f58200b13b0 0x7f582006be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.431+0000 7f5824940640 1 -- 172.21.15.43:0/90488638 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f58200b13b0 msgr2=0x7f582006be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:57.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.431+0000 7f5824940640 1 --2- 172.21.15.43:0/90488638 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f58200b13b0 0x7f582006be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.431+0000 7f5824940640 1 -- 172.21.15.43:0/90488638 >> v1:172.21.15.193:6789/0 conn(0x7f58200ce6f0 legacy=0x7f58200d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:57.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.431+0000 7f5824940640 1 -- 172.21.15.43:0/90488638 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58200d0130 con 0x7f58200cf400 2024-04-25T09:01:57.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.432+0000 7f5824940640 1 --2- 172.21.15.43:0/90488638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58200cf400 0x7f58200b0e70 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f5810009870 tx=0x7f581002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a06bd25f8bd7b55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:57.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.432+0000 7f5817fff640 1 -- 172.21.15.43:0/90488638 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5810037070 con 0x7f58200cf400 2024-04-25T09:01:57.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.432+0000 7f5817fff640 1 -- 172.21.15.43:0/90488638 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5810032440 con 0x7f58200cf400 2024-04-25T09:01:57.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.432+0000 7f5817fff640 1 -- 172.21.15.43:0/90488638 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5810032810 con 0x7f58200cf400 2024-04-25T09:01:57.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.432+0000 7f58273a5640 1 -- 172.21.15.43:0/90488638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58200cf400 msgr2=0x7f58200b0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:57.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.432+0000 7f58273a5640 1 --2- 172.21.15.43:0/90488638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58200cf400 0x7f58200b0e70 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f5810009870 tx=0x7f581002a270 comp rx=0 tx=0).stop 2024-04-25T09:01:57.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.432+0000 7f58273a5640 1 -- 172.21.15.43:0/90488638 shutdown_connections 2024-04-25T09:01:57.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.432+0000 7f58273a5640 1 --2- 172.21.15.43:0/90488638 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f58200b13b0 0x7f582006be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.432+0000 7f58273a5640 1 --2- 172.21.15.43:0/90488638 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58200cf400 0x7f58200b0e70 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.432+0000 7f58273a5640 1 -- 172.21.15.43:0/90488638 >> 172.21.15.43:0/90488638 conn(0x7f582009aa70 msgr2=0x7f58200b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:57.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.432+0000 7f58273a5640 1 -- 172.21.15.43:0/90488638 shutdown_connections 2024-04-25T09:01:57.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.432+0000 7f58273a5640 1 -- 172.21.15.43:0/90488638 wait complete. 2024-04-25T09:01:57.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.434+0000 7f58273a5640 1 Processor -- start 2024-04-25T09:01:57.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.434+0000 7f58273a5640 1 -- start start 2024-04-25T09:01:57.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.434+0000 7f58273a5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58200b13b0 0x7f582015edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.434+0000 7f58273a5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58200ce6f0 0x7f582014f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.434+0000 7f58273a5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f58200cf400 0x7f58200c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.434+0000 7f58273a5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58200b17e0 con 0x7f58200ce6f0 2024-04-25T09:01:57.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.434+0000 7f58273a5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58200d06f0 con 0x7f58200b13b0 2024-04-25T09:01:57.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.434+0000 7f58273a5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58200b1960 con 0x7f58200cf400 2024-04-25T09:01:57.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.434+0000 7f5824940640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58200ce6f0 0x7f582014f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.435+0000 7f5824940640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58200ce6f0 0x7f582014f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57766/0 (socket says 172.21.15.43:57766) 2024-04-25T09:01:57.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.435+0000 7f5824940640 1 -- 172.21.15.43:0/4112273370 learned_addr learned my addr 172.21.15.43:0/4112273370 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:57.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.435+0000 7f5825141640 1 --2- 172.21.15.43:0/4112273370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58200b13b0 0x7f582015edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.435+0000 7f5825942640 1 --2- 172.21.15.43:0/4112273370 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f58200cf400 0x7f58200c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.435+0000 7f5824940640 1 -- 172.21.15.43:0/4112273370 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f58200cf400 msgr2=0x7f58200c8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:57.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.435+0000 7f5824940640 1 --2- 172.21.15.43:0/4112273370 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f58200cf400 0x7f58200c8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.435+0000 7f5824940640 1 -- 172.21.15.43:0/4112273370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58200b13b0 msgr2=0x7f582015edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:57.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.435+0000 7f5824940640 1 --2- 172.21.15.43:0/4112273370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58200b13b0 0x7f582015edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.435+0000 7f5824940640 1 -- 172.21.15.43:0/4112273370 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f582015f4f0 con 0x7f58200ce6f0 2024-04-25T09:01:57.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.435+0000 7f5825942640 1 --2- 172.21.15.43:0/4112273370 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f58200cf400 0x7f58200c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:01:57.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.435+0000 7f5824940640 1 --2- 172.21.15.43:0/4112273370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58200ce6f0 0x7f582014f250 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f5810000c00 tx=0x7f58100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:57.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.435+0000 7f5815ffb640 1 -- 172.21.15.43:0/4112273370 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5810037040 con 0x7f58200ce6f0 2024-04-25T09:01:57.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.435+0000 7f5815ffb640 1 -- 172.21.15.43:0/4112273370 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5810042070 con 0x7f58200ce6f0 2024-04-25T09:01:57.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.435+0000 7f58273a5640 1 -- 172.21.15.43:0/4112273370 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f582015f7d0 con 0x7f58200ce6f0 2024-04-25T09:01:57.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.435+0000 7f5815ffb640 1 -- 172.21.15.43:0/4112273370 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5810030880 con 0x7f58200ce6f0 2024-04-25T09:01:57.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.435+0000 7f58273a5640 1 -- 172.21.15.43:0/4112273370 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f582015fd90 con 0x7f58200ce6f0 2024-04-25T09:01:57.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.437+0000 7f5815ffb640 1 -- 172.21.15.43:0/4112273370 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5810030a20 con 0x7f58200ce6f0 2024-04-25T09:01:57.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.437+0000 7f5815ffb640 1 --2- 172.21.15.43:0/4112273370 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f580406b4d0 0x7f580406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.438+0000 7f5815ffb640 1 -- 172.21.15.43:0/4112273370 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f58100aa8b0 con 0x7f58200ce6f0 2024-04-25T09:01:57.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.438+0000 7f5825141640 1 --2- 172.21.15.43:0/4112273370 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f580406b4d0 0x7f580406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.438+0000 7f5825141640 1 --2- 172.21.15.43:0/4112273370 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f580406b4d0 0x7f580406d990 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f581c001780 tx=0x7f581c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:57.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.439+0000 7f582a186740 1 Processor -- start 2024-04-25T09:01:57.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.439+0000 7f582a186740 1 -- start start 2024-04-25T09:01:57.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.439+0000 7f582a186740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f4f33f3200 0x55f4f33f5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.439+0000 7f582a186740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4f342f8d0 0x55f4f33f35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.439+0000 7f582a186740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4f341dcc0 0x55f4f341e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.439+0000 7f582a186740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f3357000 con 0x55f4f342f8d0 2024-04-25T09:01:57.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.439+0000 7f582a186740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f32fff20 con 0x55f4f341dcc0 2024-04-25T09:01:57.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.439+0000 7f582a186740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f4f33530a0 con 0x55f4f33f3200 2024-04-25T09:01:57.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.440+0000 7f5824940640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4f342f8d0 0x55f4f33f35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.440+0000 7f5824940640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4f342f8d0 0x55f4f33f35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57782/0 (socket says 172.21.15.43:57782) 2024-04-25T09:01:57.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.440+0000 7f5824940640 1 -- 172.21.15.43:0/4047943868 learned_addr learned my addr 172.21.15.43:0/4047943868 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:57.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.440+0000 7f5825942640 1 --2- 172.21.15.43:0/4047943868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4f341dcc0 0x55f4f341e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.440+0000 7f5825141640 1 --2- 172.21.15.43:0/4047943868 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f4f33f3200 0x55f4f33f5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.440+0000 7f5824940640 1 -- 172.21.15.43:0/4047943868 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f4f33f3200 msgr2=0x55f4f33f5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:57.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.440+0000 7f5824940640 1 --2- 172.21.15.43:0/4047943868 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f4f33f3200 0x55f4f33f5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.440+0000 7f5824940640 1 -- 172.21.15.43:0/4047943868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4f341dcc0 msgr2=0x55f4f341e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:57.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.440+0000 7f5824940640 1 --2- 172.21.15.43:0/4047943868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4f341dcc0 0x55f4f341e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.440+0000 7f5824940640 1 -- 172.21.15.43:0/4047943868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4f3137ba0 con 0x55f4f342f8d0 2024-04-25T09:01:57.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.440+0000 7f5825942640 1 --2- 172.21.15.43:0/4047943868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4f341dcc0 0x55f4f341e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:01:57.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.440+0000 7f5824940640 1 --2- 172.21.15.43:0/4047943868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4f342f8d0 0x55f4f33f35e0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f58100812a0 tx=0x7f5810078ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18b70c0fd01607a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:57.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.440+0000 7f580affd640 1 -- 172.21.15.43:0/4047943868 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5810090030 con 0x55f4f342f8d0 2024-04-25T09:01:57.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.440+0000 7f580affd640 1 -- 172.21.15.43:0/4047943868 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f581007d210 con 0x55f4f342f8d0 2024-04-25T09:01:57.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.440+0000 7f580affd640 1 -- 172.21.15.43:0/4047943868 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58100817d0 con 0x55f4f342f8d0 2024-04-25T09:01:57.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.441+0000 7f582a186740 1 -- 172.21.15.43:0/4047943868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4f342f8d0 msgr2=0x55f4f33f35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:57.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.441+0000 7f582a186740 1 --2- 172.21.15.43:0/4047943868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4f342f8d0 0x55f4f33f35e0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f58100812a0 tx=0x7f5810078ad0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.441+0000 7f582a186740 1 -- 172.21.15.43:0/4047943868 shutdown_connections 2024-04-25T09:01:57.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.441+0000 7f582a186740 1 --2- 172.21.15.43:0/4047943868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4f341dcc0 0x55f4f341e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.441+0000 7f582a186740 1 --2- 172.21.15.43:0/4047943868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4f342f8d0 0x55f4f33f35e0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.441+0000 7f582a186740 1 --2- 172.21.15.43:0/4047943868 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f4f33f3200 0x55f4f33f5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.441+0000 7f582a186740 1 -- 172.21.15.43:0/4047943868 >> 172.21.15.43:0/4047943868 conn(0x55f4f33bb050 msgr2=0x55f4f3438cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:57.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.441+0000 7f582a186740 1 -- 172.21.15.43:0/4047943868 shutdown_connections 2024-04-25T09:01:57.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.441+0000 7f582a186740 1 -- 172.21.15.43:0/4047943868 wait complete. 2024-04-25T09:01:57.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.441+0000 7f582a186740 1 -- start start 2024-04-25T09:01:57.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.441+0000 7f582a186740 1 Processor -- start 2024-04-25T09:01:57.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.442+0000 7f582a186740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f4f33f3200 0x55f4f341cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.442+0000 7f582a186740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4f341dcc0 0x55f4f3420440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.442+0000 7f582a186740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4f342f8d0 0x55f4f3437fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.442+0000 7f582a186740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f3357000 con 0x55f4f341dcc0 2024-04-25T09:01:57.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.442+0000 7f582a186740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f32f5610 con 0x55f4f342f8d0 2024-04-25T09:01:57.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.442+0000 7f582a186740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f4f3438cd0 con 0x55f4f33f3200 2024-04-25T09:01:57.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.442+0000 7f582a186740 10 client.0 _ll_register_callbacks cb 0x55f4f330df20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:01:57.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.442+0000 7f5824940640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4f341dcc0 0x55f4f3420440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.442+0000 7f5825141640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f4f33f3200 0x55f4f341cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.442+0000 7f5824940640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4f341dcc0 0x55f4f3420440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57788/0 (socket says 172.21.15.43:57788) 2024-04-25T09:01:57.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.442+0000 7f5824940640 1 -- 172.21.15.43:0/2354368813 learned_addr learned my addr 172.21.15.43:0/2354368813 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:57.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.442+0000 7f5825942640 1 --2- 172.21.15.43:0/2354368813 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4f342f8d0 0x55f4f3437fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.443+0000 7f5824940640 1 -- 172.21.15.43:0/2354368813 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f4f33f3200 msgr2=0x55f4f341cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:57.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.443+0000 7f5824940640 1 --2- 172.21.15.43:0/2354368813 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f4f33f3200 0x55f4f341cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.443+0000 7f5824940640 1 -- 172.21.15.43:0/2354368813 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4f342f8d0 msgr2=0x55f4f3437fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:57.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.443+0000 7f5824940640 1 --2- 172.21.15.43:0/2354368813 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4f342f8d0 0x55f4f3437fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.443+0000 7f5824940640 1 -- 172.21.15.43:0/2354368813 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f4f302b490 con 0x55f4f341dcc0 2024-04-25T09:01:57.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.443+0000 7f5825141640 1 --2- 172.21.15.43:0/2354368813 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f4f33f3200 0x55f4f341cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:01:57.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.443+0000 7f5825942640 1 --2- 172.21.15.43:0/2354368813 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4f342f8d0 0x55f4f3437fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:01:57.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.443+0000 7f5824940640 1 --2- 172.21.15.43:0/2354368813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4f341dcc0 0x55f4f3420440 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f5810035040 tx=0x7f5810032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b13d755daef9b10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:57.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.443+0000 7f57fffff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:01:57.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.443+0000 7f57fffff640 1 -- 172.21.15.43:0/2354368813 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58100a8070 con 0x55f4f341dcc0 2024-04-25T09:01:57.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.443+0000 7f57fffff640 1 -- 172.21.15.43:0/2354368813 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5810077210 con 0x55f4f341dcc0 2024-04-25T09:01:57.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.443+0000 7f582a186740 1 -- 172.21.15.43:0/2354368813 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f581007d210 con 0x55f4f341dcc0 2024-04-25T09:01:57.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.443+0000 7f57fffff640 1 -- 172.21.15.43:0/2354368813 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f581009c050 con 0x55f4f341dcc0 2024-04-25T09:01:57.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.443+0000 7f582a186740 10 client.8283 fetch_fsmap learned FSMap version 61 2024-04-25T09:01:57.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.443+0000 7f582a186740 1 -- 172.21.15.43:0/2354368813 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x55f4f3198330 con 0x55f4f341dcc0 2024-04-25T09:01:57.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.443+0000 7f57fffff640 1 -- 172.21.15.43:0/2354368813 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f581007d210 con 0x55f4f341dcc0 2024-04-25T09:01:57.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.444+0000 7f57fffff640 1 -- 172.21.15.43:0/2354368813 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 61) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f581009f020 con 0x55f4f341dcc0 2024-04-25T09:01:57.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.444+0000 7f582a186740 10 client.8283 fetch_fsmap finished waiting for FSMap version 61 2024-04-25T09:01:57.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.444+0000 7f582a186740 10 client.8283 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-25T09:01:57.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.444+0000 7f582a186740 20 client.8283 populate_metadata read hostname 'smithi043' 2024-04-25T09:01:57.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.444+0000 7f582a186740 1 --2- 172.21.15.43:0/2354368813 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55f4f341d760 0x55f4f3435c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.444+0000 7f582a186740 4 client.8283 mds_command: new command op to 8094 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:01:57.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.444+0000 7f582a186740 1 -- 172.21.15.43:0/2354368813 --> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f4f3364540 con 0x55f4f341d760 2024-04-25T09:01:57.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.444+0000 7f5825141640 1 --2- 172.21.15.43:0/2354368813 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55f4f341d760 0x55f4f3435c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.444+0000 7f57fffff640 10 client.8283 ms_handle_connect on v2:172.21.15.193:6834/853943267 2024-04-25T09:01:57.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.444+0000 7f5825141640 1 --2- 172.21.15.43:0/2354368813 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55f4f341d760 0x55f4f3435c20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30b693a602555540 server_cookie=f8798364015ff7ea in_seq=0 out_seq=0 2024-04-25T09:01:57.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.445+0000 7f57fffff640 1 -- 172.21.15.43:0/2354368813 <== mds.0 v2:172.21.15.193:6834/853943267 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f4f3364540 con 0x55f4f341d760 2024-04-25T09:01:57.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.445+0000 7f57fffff640 10 client.8283 handle_command_reply: tid=0 2024-04-25T09:01:57.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.445+0000 7f582a186740 1 client.8283 shutdown 2024-04-25T09:01:57.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.445+0000 7f582a186740 1 -- 172.21.15.43:0/2354368813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4f341dcc0 msgr2=0x55f4f3420440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:57.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.445+0000 7f582a186740 1 --2- 172.21.15.43:0/2354368813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4f341dcc0 0x55f4f3420440 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f5810035040 tx=0x7f5810032f90 comp rx=0 tx=0).stop 2024-04-25T09:01:57.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.445+0000 7f582a186740 1 -- 172.21.15.43:0/2354368813 shutdown_connections 2024-04-25T09:01:57.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.445+0000 7f582a186740 1 --2- 172.21.15.43:0/2354368813 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55f4f341d760 0x55f4f3435c20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.445+0000 7f582a186740 1 --2- 172.21.15.43:0/2354368813 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4f342f8d0 0x55f4f3437fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.445+0000 7f582a186740 1 --2- 172.21.15.43:0/2354368813 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4f341dcc0 0x55f4f3420440 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.445+0000 7f57fffff640 0 client.8283 ms_handle_reset on v2:172.21.15.193:6834/853943267 2024-04-25T09:01:57.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.445+0000 7f582a186740 1 --2- 172.21.15.43:0/2354368813 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f4f33f3200 0x55f4f341cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.445+0000 7f582a186740 1 -- 172.21.15.43:0/2354368813 >> 172.21.15.43:0/2354368813 conn(0x55f4f33bb050 msgr2=0x55f4f3431d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:57.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.445+0000 7f582a186740 1 -- 172.21.15.43:0/2354368813 shutdown_connections 2024-04-25T09:01:57.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.445+0000 7f582a186740 1 -- 172.21.15.43:0/2354368813 wait complete. 2024-04-25T09:01:57.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.445+0000 7f582a186740 20 client.8283 trim_cache size 0 max 16384 2024-04-25T09:01:57.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.460+0000 7f582a186740 1 Processor -- start 2024-04-25T09:01:57.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.460+0000 7f582a186740 1 -- start start 2024-04-25T09:01:57.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.461+0000 7f582a186740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f4f33f3200 0x55f4f341cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.461+0000 7f582a186740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4f341dcc0 0x55f4f3431d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.461+0000 7f582a186740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4f342f8d0 0x55f4f3432260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.461+0000 7f582a186740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58201fffa0 con 0x55f4f342f8d0 2024-04-25T09:01:57.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.461+0000 7f582a186740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58200af0f0 con 0x55f4f341dcc0 2024-04-25T09:01:57.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.461+0000 7f582a186740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5820161260 con 0x55f4f33f3200 2024-04-25T09:01:57.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.461+0000 7f5825942640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4f342f8d0 0x55f4f3432260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.461+0000 7f5825942640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4f342f8d0 0x55f4f3432260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57790/0 (socket says 172.21.15.43:57790) 2024-04-25T09:01:57.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.461+0000 7f5825942640 1 -- 172.21.15.43:0/2579437948 learned_addr learned my addr 172.21.15.43:0/2579437948 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:57.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.461+0000 7f5825942640 1 -- 172.21.15.43:0/2579437948 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f4f33f3200 msgr2=0x55f4f341cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:57.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.461+0000 7f5825942640 1 --2- 172.21.15.43:0/2579437948 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f4f33f3200 0x55f4f341cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.461+0000 7f5825942640 1 -- 172.21.15.43:0/2579437948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4f341dcc0 msgr2=0x55f4f3431d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:57.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.461+0000 7f5825942640 1 --2- 172.21.15.43:0/2579437948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4f341dcc0 0x55f4f3431d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.461+0000 7f5825942640 1 -- 172.21.15.43:0/2579437948 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4f3438320 con 0x55f4f342f8d0 2024-04-25T09:01:57.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.462+0000 7f5825942640 1 --2- 172.21.15.43:0/2579437948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4f342f8d0 0x55f4f3432260 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f5818003e40 tx=0x7f5818009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ec75337e5351e01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:57.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.462+0000 7f580a7fc640 1 -- 172.21.15.43:0/2579437948 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5818004410 con 0x55f4f342f8d0 2024-04-25T09:01:57.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.462+0000 7f580a7fc640 1 -- 172.21.15.43:0/2579437948 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f581801b030 con 0x55f4f342f8d0 2024-04-25T09:01:57.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.462+0000 7f580a7fc640 1 -- 172.21.15.43:0/2579437948 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58180047d0 con 0x55f4f342f8d0 2024-04-25T09:01:57.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.462+0000 7f582a186740 1 -- 172.21.15.43:0/2579437948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4f342f8d0 msgr2=0x55f4f3432260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:57.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.462+0000 7f582a186740 1 --2- 172.21.15.43:0/2579437948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4f342f8d0 0x55f4f3432260 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f5818003e40 tx=0x7f5818009750 comp rx=0 tx=0).stop 2024-04-25T09:01:57.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.462+0000 7f582a186740 1 -- 172.21.15.43:0/2579437948 shutdown_connections 2024-04-25T09:01:57.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.462+0000 7f582a186740 1 --2- 172.21.15.43:0/2579437948 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4f342f8d0 0x55f4f3432260 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.462+0000 7f582a186740 1 --2- 172.21.15.43:0/2579437948 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4f341dcc0 0x55f4f3431d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.462+0000 7f582a186740 1 --2- 172.21.15.43:0/2579437948 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f4f33f3200 0x55f4f341cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.462+0000 7f582a186740 1 -- 172.21.15.43:0/2579437948 >> 172.21.15.43:0/2579437948 conn(0x55f4f33bb050 msgr2=0x55f4f33f5240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:57.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.462+0000 7f582a186740 1 -- 172.21.15.43:0/2579437948 shutdown_connections 2024-04-25T09:01:57.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.462+0000 7f582a186740 1 -- 172.21.15.43:0/2579437948 wait complete. 2024-04-25T09:01:57.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.463+0000 7f582a186740 1 -- start start 2024-04-25T09:01:57.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.463+0000 7f582a186740 1 Processor -- start 2024-04-25T09:01:57.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.464+0000 7f582a186740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f4f33f3200 0x55f4f343be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.464+0000 7f582a186740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4f341dcc0 0x55f4f343c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.464+0000 7f582a186740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4f342f8d0 0x55f4f34a27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.464+0000 7f582a186740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58200aeaf0 con 0x55f4f341dcc0 2024-04-25T09:01:57.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.464+0000 7f582a186740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4f33edd80 con 0x55f4f342f8d0 2024-04-25T09:01:57.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.464+0000 7f582a186740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f4f34a3970 con 0x55f4f33f3200 2024-04-25T09:01:57.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.464+0000 7f582a186740 10 client.0 _ll_register_callbacks cb 0x7f58100094d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:01:57.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.464+0000 7f5824940640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4f341dcc0 0x55f4f343c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.464+0000 7f5824940640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4f341dcc0 0x55f4f343c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57798/0 (socket says 172.21.15.43:57798) 2024-04-25T09:01:57.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.464+0000 7f5824940640 1 -- 172.21.15.43:0/2581620783 learned_addr learned my addr 172.21.15.43:0/2581620783 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:57.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.464+0000 7f5824940640 1 -- 172.21.15.43:0/2581620783 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f4f33f3200 msgr2=0x55f4f343be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:57.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.464+0000 7f5824940640 1 --2- 172.21.15.43:0/2581620783 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f4f33f3200 0x55f4f343be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.464+0000 7f5824940640 1 -- 172.21.15.43:0/2581620783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4f342f8d0 msgr2=0x55f4f34a27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:57.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.464+0000 7f5824940640 1 --2- 172.21.15.43:0/2581620783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4f342f8d0 0x55f4f34a27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.464+0000 7f5824940640 1 -- 172.21.15.43:0/2581620783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f4f3004610 con 0x55f4f341dcc0 2024-04-25T09:01:57.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.464+0000 7f5824940640 1 --2- 172.21.15.43:0/2581620783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4f341dcc0 0x55f4f343c340 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f5810081970 tx=0x7f5810009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=416f3eaad73aa680 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:57.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.464+0000 7f57fffff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:01:57.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.464+0000 7f57fffff640 1 -- 172.21.15.43:0/2581620783 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5810090040 con 0x55f4f341dcc0 2024-04-25T09:01:57.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.464+0000 7f57fffff640 1 -- 172.21.15.43:0/2581620783 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5810078640 con 0x55f4f341dcc0 2024-04-25T09:01:57.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.464+0000 7f582a186740 1 -- 172.21.15.43:0/2581620783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f581801b030 con 0x55f4f341dcc0 2024-04-25T09:01:57.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.465+0000 7f57fffff640 1 -- 172.21.15.43:0/2581620783 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5810032440 con 0x55f4f341dcc0 2024-04-25T09:01:57.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.465+0000 7f582a186740 10 client.8289 fetch_fsmap learned FSMap version 61 2024-04-25T09:01:57.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.465+0000 7f582a186740 1 -- 172.21.15.43:0/2581620783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x55f4f3004610 con 0x55f4f341dcc0 2024-04-25T09:01:57.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.465+0000 7f57fffff640 1 -- 172.21.15.43:0/2581620783 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f581801b030 con 0x55f4f341dcc0 2024-04-25T09:01:57.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.465+0000 7f57fffff640 1 -- 172.21.15.43:0/2581620783 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 61) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5810089070 con 0x55f4f341dcc0 2024-04-25T09:01:57.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.465+0000 7f582a186740 10 client.8289 fetch_fsmap finished waiting for FSMap version 61 2024-04-25T09:01:57.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.465+0000 7f582a186740 10 client.8289 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-25T09:01:57.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.465+0000 7f582a186740 20 client.8289 populate_metadata read hostname 'smithi043' 2024-04-25T09:01:57.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.465+0000 7f582a186740 1 --2- 172.21.15.43:0/2581620783 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55f4f3436120 0x55f4f34a2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.465+0000 7f582a186740 4 client.8289 mds_command: new command op to 8094 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T09:01:57.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.465+0000 7f582a186740 1 -- 172.21.15.43:0/2581620783 --> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x55f4f335e890 con 0x55f4f3436120 2024-04-25T09:01:57.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.465+0000 7f5825141640 1 --2- 172.21.15.43:0/2581620783 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55f4f3436120 0x55f4f34a2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.465+0000 7f57fffff640 10 client.8289 ms_handle_connect on v2:172.21.15.193:6834/853943267 2024-04-25T09:01:57.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.465+0000 7f5825141640 1 --2- 172.21.15.43:0/2581620783 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55f4f3436120 0x55f4f34a2ce0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=872732171ff895d3 server_cookie=3f75763ef3772f9a in_seq=0 out_seq=0 2024-04-25T09:01:57.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.468+0000 7f57fffff640 1 -- 172.21.15.43:0/2581620783 <== mds.0 v2:172.21.15.193:6834/853943267 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f4f335e890 con 0x55f4f3436120 2024-04-25T09:01:57.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.468+0000 7f57fffff640 10 client.8289 handle_command_reply: tid=0 2024-04-25T09:01:57.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.468+0000 7f582a186740 1 client.8289 shutdown 2024-04-25T09:01:57.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.468+0000 7f582a186740 1 -- 172.21.15.43:0/2581620783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4f341dcc0 msgr2=0x55f4f343c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:57.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.468+0000 7f582a186740 1 --2- 172.21.15.43:0/2581620783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4f341dcc0 0x55f4f343c340 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f5810081970 tx=0x7f5810009240 comp rx=0 tx=0).stop 2024-04-25T09:01:57.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.468+0000 7f582a186740 1 -- 172.21.15.43:0/2581620783 shutdown_connections 2024-04-25T09:01:57.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.468+0000 7f582a186740 1 --2- 172.21.15.43:0/2581620783 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55f4f3436120 0x55f4f34a2ce0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.468+0000 7f582a186740 1 --2- 172.21.15.43:0/2581620783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f4f342f8d0 0x55f4f34a27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.468+0000 7f582a186740 1 --2- 172.21.15.43:0/2581620783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f4f341dcc0 0x55f4f343c340 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.468+0000 7f582a186740 1 --2- 172.21.15.43:0/2581620783 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f4f33f3200 0x55f4f343be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.468+0000 7f57fffff640 0 client.8289 ms_handle_reset on v2:172.21.15.193:6834/853943267 2024-04-25T09:01:57.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.468+0000 7f582a186740 1 -- 172.21.15.43:0/2581620783 >> 172.21.15.43:0/2581620783 conn(0x55f4f33bb050 msgr2=0x55f4f341f2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:57.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.468+0000 7f582a186740 1 -- 172.21.15.43:0/2581620783 shutdown_connections 2024-04-25T09:01:57.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.468+0000 7f582a186740 1 -- 172.21.15.43:0/2581620783 wait complete. 2024-04-25T09:01:57.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.468+0000 7f582a186740 20 client.8289 trim_cache size 0 max 16384 2024-04-25T09:01:57.468 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:01:57.468 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-25T09:01:57.468 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "6c155ea2-966a-4d52-8066-f0303eb75e9e", 2024-04-25T09:01:57.468 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-25T09:01:57.468 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:01:57.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.469+0000 7f58273a5640 1 -- 172.21.15.43:0/4112273370 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f580406b4d0 msgr2=0x7f580406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:57.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.469+0000 7f58273a5640 1 --2- 172.21.15.43:0/4112273370 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f580406b4d0 0x7f580406d990 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f581c001780 tx=0x7f581c009040 comp rx=0 tx=0).stop 2024-04-25T09:01:57.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.469+0000 7f58273a5640 1 -- 172.21.15.43:0/4112273370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58200ce6f0 msgr2=0x7f582014f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:57.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.469+0000 7f58273a5640 1 --2- 172.21.15.43:0/4112273370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58200ce6f0 0x7f582014f250 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f5810000c00 tx=0x7f58100046c0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.469+0000 7f58273a5640 1 -- 172.21.15.43:0/4112273370 shutdown_connections 2024-04-25T09:01:57.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.469+0000 7f58273a5640 1 --2- 172.21.15.43:0/4112273370 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f58200cf400 0x7f58200c8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.469+0000 7f58273a5640 1 --2- 172.21.15.43:0/4112273370 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58200ce6f0 0x7f582014f250 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.469+0000 7f58273a5640 1 --2- 172.21.15.43:0/4112273370 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f580406b4d0 0x7f580406d990 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.469+0000 7f58273a5640 1 --2- 172.21.15.43:0/4112273370 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58200b13b0 0x7f582015edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.469+0000 7f58273a5640 1 -- 172.21.15.43:0/4112273370 >> 172.21.15.43:0/4112273370 conn(0x7f582009aa70 msgr2=0x7f58200cd430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:57.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.469+0000 7f58273a5640 1 -- 172.21.15.43:0/4112273370 shutdown_connections 2024-04-25T09:01:57.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.469+0000 7f58273a5640 1 -- 172.21.15.43:0/4112273370 wait complete. 2024-04-25T09:01:57.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:57.471 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub pause 2024-04-25T09:01:57.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.598+0000 7f68ea1ce640 1 Processor -- start 2024-04-25T09:01:57.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.598+0000 7f68ea1ce640 1 -- start start 2024-04-25T09:01:57.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.598+0000 7f68ea1ce640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f68e415dd80 0x7f68e40cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.598+0000 7f68ea1ce640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68e40b2fe0 con 0x7f68e40d17b0 2024-04-25T09:01:57.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.598+0000 7f68ea1ce640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68e40ced70 con 0x7f68e40b1cd0 2024-04-25T09:01:57.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.598+0000 7f68ea1ce640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e40cfc60 con 0x7f68e415dd80 2024-04-25T09:01:57.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.599+0000 7f68e2ffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f68e40d17b0 0x7f68e415e160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56772/0 (socket says 172.21.15.43:56772) 2024-04-25T09:01:57.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.599+0000 7f68e2ffd640 1 -- 172.21.15.43:0/2725684306 learned_addr learned my addr 172.21.15.43:0/2725684306 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:57.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.599+0000 7f68e37fe640 1 --2- 172.21.15.43:0/2725684306 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f68e415dd80 0x7f68e40cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.599+0000 7f68e27fc640 1 -- 172.21.15.43:0/2725684306 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4202701524 0 0) 0x7f68e40b2fe0 con 0x7f68e40d17b0 2024-04-25T09:01:57.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.599+0000 7f68e27fc640 1 -- 172.21.15.43:0/2725684306 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68cc003520 con 0x7f68e40d17b0 2024-04-25T09:01:57.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.599+0000 7f68e27fc640 1 -- 172.21.15.43:0/2725684306 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 824179358 0 0) 0x7f68e40ced70 con 0x7f68e40b1cd0 2024-04-25T09:01:57.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.599+0000 7f68e27fc640 1 -- 172.21.15.43:0/2725684306 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68e40b2fe0 con 0x7f68e40b1cd0 2024-04-25T09:01:57.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.599+0000 7f68e37fe640 1 -- 172.21.15.43:0/2725684306 >> v1:172.21.15.193:6789/0 conn(0x7f68e40b1cd0 legacy=0x7f68e415e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:57.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.599+0000 7f68e37fe640 1 -- 172.21.15.43:0/2725684306 >> v1:172.21.15.43:6789/0 conn(0x7f68e40d17b0 legacy=0x7f68e415e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:57.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.599+0000 7f68e27fc640 1 -- 172.21.15.43:0/2725684306 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3176558411 0 0) 0x7f68cc003520 con 0x7f68e40d17b0 2024-04-25T09:01:57.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.599+0000 7f68e37fe640 1 -- 172.21.15.43:0/2725684306 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e40cf050 con 0x7f68e415dd80 2024-04-25T09:01:57.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.600+0000 7f68e37fe640 1 --2- 172.21.15.43:0/2725684306 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f68e415dd80 0x7f68e40cf720 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f68d40097d0 tx=0x7f68d402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f54c422cedebf5c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:57.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.600+0000 7f68e27fc640 1 -- 172.21.15.43:0/2725684306 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d4037070 con 0x7f68e415dd80 2024-04-25T09:01:57.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.600+0000 7f68e27fc640 1 -- 172.21.15.43:0/2725684306 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d402ab90 con 0x7f68e415dd80 2024-04-25T09:01:57.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.600+0000 7f68e27fc640 1 -- 172.21.15.43:0/2725684306 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d8001690 con 0x7f68e415dd80 2024-04-25T09:01:57.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.601+0000 7f68ea1ce640 1 -- 172.21.15.43:0/2725684306 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f68e415dd80 msgr2=0x7f68e40cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:57.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.601+0000 7f68ea1ce640 1 --2- 172.21.15.43:0/2725684306 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f68e415dd80 0x7f68e40cf720 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f68d40097d0 tx=0x7f68d402a100 comp rx=0 tx=0).stop 2024-04-25T09:01:57.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.601+0000 7f68ea1ce640 1 -- 172.21.15.43:0/2725684306 shutdown_connections 2024-04-25T09:01:57.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.601+0000 7f68ea1ce640 1 --2- 172.21.15.43:0/2725684306 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f68e415dd80 0x7f68e40cf720 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.601+0000 7f68ea1ce640 1 -- 172.21.15.43:0/2725684306 >> 172.21.15.43:0/2725684306 conn(0x7f68e4072aa0 msgr2=0x7f68e409aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:57.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.601+0000 7f68ea1ce640 1 -- 172.21.15.43:0/2725684306 shutdown_connections 2024-04-25T09:01:57.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.601+0000 7f68ea1ce640 1 -- 172.21.15.43:0/2725684306 wait complete. 2024-04-25T09:01:57.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.602+0000 7f68ea1ce640 1 Processor -- start 2024-04-25T09:01:57.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.602+0000 7f68ea1ce640 1 -- start start 2024-04-25T09:01:57.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.603+0000 7f68ea1ce640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68e40b1cd0 0x7f68e41600b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.603+0000 7f68ea1ce640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f68e40d17b0 0x7f68e4157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.603+0000 7f68ea1ce640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68e415dd80 0x7f68e4159540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.603+0000 7f68ea1ce640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e40b2c40 con 0x7f68e415dd80 2024-04-25T09:01:57.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.603+0000 7f68ea1ce640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e40b32c0 con 0x7f68e40b1cd0 2024-04-25T09:01:57.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.603+0000 7f68ea1ce640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e40b2dc0 con 0x7f68e40d17b0 2024-04-25T09:01:57.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.603+0000 7f68e37fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68e40b1cd0 0x7f68e41600b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.603+0000 7f68e3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68e415dd80 0x7f68e4159540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.603+0000 7f68e37fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68e40b1cd0 0x7f68e41600b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:57010/0 (socket says 172.21.15.43:57010) 2024-04-25T09:01:57.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.603+0000 7f68e37fe640 1 -- 172.21.15.43:0/1462484298 learned_addr learned my addr 172.21.15.43:0/1462484298 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:57.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.603+0000 7f68e2ffd640 1 --2- 172.21.15.43:0/1462484298 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f68e40d17b0 0x7f68e4157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.603+0000 7f68e3fff640 1 -- 172.21.15.43:0/1462484298 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f68e40d17b0 msgr2=0x7f68e4157cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:57.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.603+0000 7f68e3fff640 1 --2- 172.21.15.43:0/1462484298 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f68e40d17b0 0x7f68e4157cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.603+0000 7f68e3fff640 1 -- 172.21.15.43:0/1462484298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68e40b1cd0 msgr2=0x7f68e41600b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:57.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.603+0000 7f68e3fff640 1 --2- 172.21.15.43:0/1462484298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68e40b1cd0 0x7f68e41600b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.603+0000 7f68e3fff640 1 -- 172.21.15.43:0/1462484298 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e4158470 con 0x7f68e415dd80 2024-04-25T09:01:57.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.603+0000 7f68e37fe640 1 --2- 172.21.15.43:0/1462484298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68e40b1cd0 0x7f68e41600b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:01:57.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.603+0000 7f68e3fff640 1 --2- 172.21.15.43:0/1462484298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68e415dd80 0x7f68e4159540 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f68dc00c6c0 tx=0x7f68dc00cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:57.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.604+0000 7f68cbfff640 1 -- 172.21.15.43:0/1462484298 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68dc003b70 con 0x7f68e415dd80 2024-04-25T09:01:57.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.604+0000 7f68cbfff640 1 -- 172.21.15.43:0/1462484298 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68dc003d10 con 0x7f68e415dd80 2024-04-25T09:01:57.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.604+0000 7f68cbfff640 1 -- 172.21.15.43:0/1462484298 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68dc005310 con 0x7f68e415dd80 2024-04-25T09:01:57.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.604+0000 7f68ea1ce640 1 -- 172.21.15.43:0/1462484298 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68e4158750 con 0x7f68e415dd80 2024-04-25T09:01:57.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.604+0000 7f68ea1ce640 1 -- 172.21.15.43:0/1462484298 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e4158d10 con 0x7f68e415dd80 2024-04-25T09:01:57.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.605+0000 7f68cbfff640 1 -- 172.21.15.43:0/1462484298 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f68dc0054b0 con 0x7f68e415dd80 2024-04-25T09:01:57.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.606+0000 7f68cbfff640 1 --2- 172.21.15.43:0/1462484298 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f68d006b4d0 0x7f68d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.606+0000 7f68cbfff640 1 -- 172.21.15.43:0/1462484298 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f68dc089ed0 con 0x7f68e415dd80 2024-04-25T09:01:57.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.606+0000 7f68e37fe640 1 --2- 172.21.15.43:0/1462484298 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f68d006b4d0 0x7f68d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.606+0000 7f68e37fe640 1 --2- 172.21.15.43:0/1462484298 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f68d006b4d0 0x7f68d006d990 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f68d402a140 tx=0x7f68d402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:57.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.607+0000 7f68ecfae740 1 Processor -- start 2024-04-25T09:01:57.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.607+0000 7f68ecfae740 1 -- start start 2024-04-25T09:01:57.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.608+0000 7f68ecfae740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c695fd6ca0 0x55c695fd8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.608+0000 7f68ecfae740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c696013370 0x55c695fd7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.608+0000 7f68ecfae740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c696001760 0x55c696001be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.608+0000 7f68ecfae740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c695f39b90 con 0x55c695fd6ca0 2024-04-25T09:01:57.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.608+0000 7f68ecfae740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c695ee2e40 con 0x55c696001760 2024-04-25T09:01:57.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.608+0000 7f68ecfae740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c695f35c30 con 0x55c696013370 2024-04-25T09:01:57.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.608+0000 7f68e37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c695fd6ca0 0x55c695fd8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.608+0000 7f68e37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c695fd6ca0 0x55c695fd8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57818/0 (socket says 172.21.15.43:57818) 2024-04-25T09:01:57.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.608+0000 7f68e37fe640 1 -- 172.21.15.43:0/2547283003 learned_addr learned my addr 172.21.15.43:0/2547283003 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:57.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.608+0000 7f68e2ffd640 1 --2- 172.21.15.43:0/2547283003 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c696013370 0x55c695fd7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.608+0000 7f68e3fff640 1 --2- 172.21.15.43:0/2547283003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c696001760 0x55c696001be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.608+0000 7f68e37fe640 1 -- 172.21.15.43:0/2547283003 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c696013370 msgr2=0x55c695fd7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:57.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.608+0000 7f68e37fe640 1 --2- 172.21.15.43:0/2547283003 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c696013370 0x55c695fd7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.608+0000 7f68e37fe640 1 -- 172.21.15.43:0/2547283003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c696001760 msgr2=0x55c696001be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:57.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.608+0000 7f68e37fe640 1 --2- 172.21.15.43:0/2547283003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c696001760 0x55c696001be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.608+0000 7f68e37fe640 1 -- 172.21.15.43:0/2547283003 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c695d187d0 con 0x55c695fd6ca0 2024-04-25T09:01:57.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.608+0000 7f68e37fe640 1 --2- 172.21.15.43:0/2547283003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c695fd6ca0 0x55c695fd8e00 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f68d402a6f0 tx=0x7f68d402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6ad5e85657e5b40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:57.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.608+0000 7f68c97fa640 1 -- 172.21.15.43:0/2547283003 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d4037040 con 0x55c695fd6ca0 2024-04-25T09:01:57.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.608+0000 7f68c97fa640 1 -- 172.21.15.43:0/2547283003 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d404f070 con 0x55c695fd6ca0 2024-04-25T09:01:57.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.609+0000 7f68c97fa640 1 -- 172.21.15.43:0/2547283003 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d403e860 con 0x55c695fd6ca0 2024-04-25T09:01:57.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.609+0000 7f68ecfae740 1 -- 172.21.15.43:0/2547283003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c695fd6ca0 msgr2=0x55c695fd8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:57.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.609+0000 7f68ecfae740 1 --2- 172.21.15.43:0/2547283003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c695fd6ca0 0x55c695fd8e00 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f68d402a6f0 tx=0x7f68d402c000 comp rx=0 tx=0).stop 2024-04-25T09:01:57.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.609+0000 7f68ecfae740 1 -- 172.21.15.43:0/2547283003 shutdown_connections 2024-04-25T09:01:57.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.609+0000 7f68ecfae740 1 --2- 172.21.15.43:0/2547283003 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c696001760 0x55c696001be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.609+0000 7f68ecfae740 1 --2- 172.21.15.43:0/2547283003 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c696013370 0x55c695fd7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.609+0000 7f68ecfae740 1 --2- 172.21.15.43:0/2547283003 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c695fd6ca0 0x55c695fd8e00 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.609+0000 7f68ecfae740 1 -- 172.21.15.43:0/2547283003 >> 172.21.15.43:0/2547283003 conn(0x55c695f9e870 msgr2=0x55c69601c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:57.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.609+0000 7f68ecfae740 1 -- 172.21.15.43:0/2547283003 shutdown_connections 2024-04-25T09:01:57.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.609+0000 7f68ecfae740 1 -- 172.21.15.43:0/2547283003 wait complete. 2024-04-25T09:01:57.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.610+0000 7f68ecfae740 1 -- start start 2024-04-25T09:01:57.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.610+0000 7f68ecfae740 1 Processor -- start 2024-04-25T09:01:57.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.611+0000 7f68ecfae740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c695fd6ca0 0x55c695ffdae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.611+0000 7f68ecfae740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c696001760 0x55c6960138f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.611+0000 7f68ecfae740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c696013370 0x55c696016220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.611+0000 7f68ecfae740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c695ef63e0 con 0x55c696013370 2024-04-25T09:01:57.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.611+0000 7f68ecfae740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c695fff2b0 con 0x55c695fd6ca0 2024-04-25T09:01:57.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.611+0000 7f68ecfae740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c696015450 con 0x55c696001760 2024-04-25T09:01:57.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.611+0000 7f68ecfae740 10 client.0 _ll_register_callbacks cb 0x55c695ef5db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:01:57.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.611+0000 7f68e3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c696013370 0x55c696016220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.611+0000 7f68e3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c696013370 0x55c696016220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57824/0 (socket says 172.21.15.43:57824) 2024-04-25T09:01:57.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.611+0000 7f68e3fff640 1 -- 172.21.15.43:0/1439060713 learned_addr learned my addr 172.21.15.43:0/1439060713 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:57.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.611+0000 7f68e37fe640 1 --2- 172.21.15.43:0/1439060713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c695fd6ca0 0x55c695ffdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.611+0000 7f68e2ffd640 1 --2- 172.21.15.43:0/1439060713 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c696001760 0x55c6960138f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.611+0000 7f68e3fff640 1 -- 172.21.15.43:0/1439060713 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c696001760 msgr2=0x55c6960138f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:57.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.611+0000 7f68e3fff640 1 --2- 172.21.15.43:0/1439060713 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c696001760 0x55c6960138f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.611+0000 7f68e3fff640 1 -- 172.21.15.43:0/1439060713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c695fd6ca0 msgr2=0x55c695ffdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:57.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.611+0000 7f68e3fff640 1 --2- 172.21.15.43:0/1439060713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c695fd6ca0 0x55c695ffdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.611+0000 7f68e3fff640 1 -- 172.21.15.43:0/1439060713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c695c0e0f0 con 0x55c696013370 2024-04-25T09:01:57.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.611+0000 7f68e3fff640 1 --2- 172.21.15.43:0/1439060713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c696013370 0x55c696016220 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f68dc0586d0 tx=0x7f68dc058860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9bab8a43c7858c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:57.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.611+0000 7f68c27fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:01:57.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.611+0000 7f68c27fc640 1 -- 172.21.15.43:0/1439060713 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68dc086060 con 0x55c696013370 2024-04-25T09:01:57.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.611+0000 7f68c27fc640 1 -- 172.21.15.43:0/1439060713 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f68dc056a90 con 0x55c696013370 2024-04-25T09:01:57.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.611+0000 7f68ecfae740 1 -- 172.21.15.43:0/1439060713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f68d404f070 con 0x55c696013370 2024-04-25T09:01:57.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.611+0000 7f68c27fc640 1 -- 172.21.15.43:0/1439060713 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68dc075050 con 0x55c696013370 2024-04-25T09:01:57.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.612+0000 7f68ecfae740 10 client.8301 fetch_fsmap learned FSMap version 61 2024-04-25T09:01:57.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.612+0000 7f68ecfae740 1 -- 172.21.15.43:0/1439060713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x55c695d78f60 con 0x55c696013370 2024-04-25T09:01:57.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.612+0000 7f68c27fc640 1 -- 172.21.15.43:0/1439060713 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d404f070 con 0x55c696013370 2024-04-25T09:01:57.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.612+0000 7f68c27fc640 1 -- 172.21.15.43:0/1439060713 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 61) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f68dc054e00 con 0x55c696013370 2024-04-25T09:01:57.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.612+0000 7f68ecfae740 10 client.8301 fetch_fsmap finished waiting for FSMap version 61 2024-04-25T09:01:57.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.612+0000 7f68ecfae740 10 client.8301 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-25T09:01:57.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.612+0000 7f68ecfae740 20 client.8301 populate_metadata read hostname 'smithi043' 2024-04-25T09:01:57.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.612+0000 7f68ecfae740 1 --2- 172.21.15.43:0/1439060713 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55c696002ef0 0x55c696003300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.612+0000 7f68ecfae740 4 client.8301 mds_command: new command op to 8094 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:01:57.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.612+0000 7f68ecfae740 1 -- 172.21.15.43:0/1439060713 --> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c695f470d0 con 0x55c696002ef0 2024-04-25T09:01:57.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.612+0000 7f68e37fe640 1 --2- 172.21.15.43:0/1439060713 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55c696002ef0 0x55c696003300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.612+0000 7f68c27fc640 10 client.8301 ms_handle_connect on v2:172.21.15.193:6834/853943267 2024-04-25T09:01:57.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.612+0000 7f68e37fe640 1 --2- 172.21.15.43:0/1439060713 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55c696002ef0 0x55c696003300 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3faacc1ccdd29bfd server_cookie=b53cd93e86c48ea9 in_seq=0 out_seq=0 2024-04-25T09:01:57.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.613+0000 7f68c27fc640 1 -- 172.21.15.43:0/1439060713 <== mds.0 v2:172.21.15.193:6834/853943267 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c695f470d0 con 0x55c696002ef0 2024-04-25T09:01:57.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.613+0000 7f68c27fc640 10 client.8301 handle_command_reply: tid=0 2024-04-25T09:01:57.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.613+0000 7f68ecfae740 1 client.8301 shutdown 2024-04-25T09:01:57.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.613+0000 7f68ecfae740 1 -- 172.21.15.43:0/1439060713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c696013370 msgr2=0x55c696016220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:57.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.613+0000 7f68ecfae740 1 --2- 172.21.15.43:0/1439060713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c696013370 0x55c696016220 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f68dc0586d0 tx=0x7f68dc058860 comp rx=0 tx=0).stop 2024-04-25T09:01:57.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.613+0000 7f68ecfae740 1 -- 172.21.15.43:0/1439060713 shutdown_connections 2024-04-25T09:01:57.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.613+0000 7f68ecfae740 1 --2- 172.21.15.43:0/1439060713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c696013370 0x55c696016220 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.613+0000 7f68ecfae740 1 --2- 172.21.15.43:0/1439060713 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c696001760 0x55c6960138f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.613+0000 7f68ecfae740 1 --2- 172.21.15.43:0/1439060713 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55c696002ef0 0x55c696003300 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.613+0000 7f68ecfae740 1 --2- 172.21.15.43:0/1439060713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c695fd6ca0 0x55c695ffdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.613+0000 7f68c27fc640 0 client.8301 ms_handle_reset on v2:172.21.15.193:6834/853943267 2024-04-25T09:01:57.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.613+0000 7f68ecfae740 1 -- 172.21.15.43:0/1439060713 >> 172.21.15.43:0/1439060713 conn(0x55c695f9e870 msgr2=0x55c695ffea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:57.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.613+0000 7f68ecfae740 1 -- 172.21.15.43:0/1439060713 shutdown_connections 2024-04-25T09:01:57.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.613+0000 7f68ecfae740 1 -- 172.21.15.43:0/1439060713 wait complete. 2024-04-25T09:01:57.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.614+0000 7f68ecfae740 20 client.8301 trim_cache size 0 max 16384 2024-04-25T09:01:57.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.628+0000 7f68ecfae740 1 Processor -- start 2024-04-25T09:01:57.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.628+0000 7f68ecfae740 1 -- start start 2024-04-25T09:01:57.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.629+0000 7f68ecfae740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c695fd6ca0 0x55c696017a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.629+0000 7f68ecfae740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c696001760 0x55c696035700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.629+0000 7f68ecfae740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c696013370 0x55c695ffa9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.629+0000 7f68ecfae740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c695fd1d80 con 0x55c695fd6ca0 2024-04-25T09:01:57.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.629+0000 7f68ecfae740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e4204280 con 0x55c696013370 2024-04-25T09:01:57.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.629+0000 7f68ecfae740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e41f97f0 con 0x55c696001760 2024-04-25T09:01:57.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.629+0000 7f68e37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c695fd6ca0 0x55c696017a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.629+0000 7f68e37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c695fd6ca0 0x55c696017a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57832/0 (socket says 172.21.15.43:57832) 2024-04-25T09:01:57.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.629+0000 7f68e37fe640 1 -- 172.21.15.43:0/795641434 learned_addr learned my addr 172.21.15.43:0/795641434 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:57.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.629+0000 7f68e37fe640 1 -- 172.21.15.43:0/795641434 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c696001760 msgr2=0x55c696035700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:57.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.629+0000 7f68e37fe640 1 --2- 172.21.15.43:0/795641434 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c696001760 0x55c696035700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.629+0000 7f68e37fe640 1 -- 172.21.15.43:0/795641434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c696013370 msgr2=0x55c695ffa9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:57.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.629+0000 7f68e37fe640 1 --2- 172.21.15.43:0/795641434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c696013370 0x55c695ffa9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.629+0000 7f68e37fe640 1 -- 172.21.15.43:0/795641434 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c695be71b0 con 0x55c695fd6ca0 2024-04-25T09:01:57.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.630+0000 7f68e37fe640 1 --2- 172.21.15.43:0/795641434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c695fd6ca0 0x55c696017a70 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f68d40098f0 tx=0x7f68d403e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4de5c8e81a3c2190 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:57.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.630+0000 7f68c8ff9640 1 -- 172.21.15.43:0/795641434 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d4037040 con 0x55c695fd6ca0 2024-04-25T09:01:57.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.630+0000 7f68c8ff9640 1 -- 172.21.15.43:0/795641434 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d404f040 con 0x55c695fd6ca0 2024-04-25T09:01:57.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.630+0000 7f68c8ff9640 1 -- 172.21.15.43:0/795641434 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d40044d0 con 0x55c695fd6ca0 2024-04-25T09:01:57.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.630+0000 7f68ecfae740 1 -- 172.21.15.43:0/795641434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c695fd6ca0 msgr2=0x55c696017a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:57.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.630+0000 7f68ecfae740 1 --2- 172.21.15.43:0/795641434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c695fd6ca0 0x55c696017a70 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f68d40098f0 tx=0x7f68d403e000 comp rx=0 tx=0).stop 2024-04-25T09:01:57.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.630+0000 7f68ecfae740 1 -- 172.21.15.43:0/795641434 shutdown_connections 2024-04-25T09:01:57.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.630+0000 7f68ecfae740 1 --2- 172.21.15.43:0/795641434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c696013370 0x55c695ffa9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.630+0000 7f68ecfae740 1 --2- 172.21.15.43:0/795641434 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c696001760 0x55c696035700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.630+0000 7f68ecfae740 1 --2- 172.21.15.43:0/795641434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c695fd6ca0 0x55c696017a70 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.630+0000 7f68ecfae740 1 -- 172.21.15.43:0/795641434 >> 172.21.15.43:0/795641434 conn(0x55c695f9e870 msgr2=0x55c695fccf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:57.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.630+0000 7f68ecfae740 1 -- 172.21.15.43:0/795641434 shutdown_connections 2024-04-25T09:01:57.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.630+0000 7f68ecfae740 1 -- 172.21.15.43:0/795641434 wait complete. 2024-04-25T09:01:57.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.631+0000 7f68ecfae740 1 -- start start 2024-04-25T09:01:57.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.631+0000 7f68ecfae740 1 Processor -- start 2024-04-25T09:01:57.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.632+0000 7f68ecfae740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c695fd6ca0 0x55c695ffc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.632+0000 7f68ecfae740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c696001760 0x55c696035700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.632+0000 7f68ecfae740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c696013370 0x55c696035c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.632+0000 7f68ecfae740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e41f91f0 con 0x55c695fd6ca0 2024-04-25T09:01:57.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.632+0000 7f68ecfae740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e415ef60 con 0x55c696013370 2024-04-25T09:01:57.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.632+0000 7f68ecfae740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c695fd1d80 con 0x55c696001760 2024-04-25T09:01:57.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.632+0000 7f68ecfae740 10 client.0 _ll_register_callbacks cb 0x7f68dc00c610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:01:57.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.632+0000 7f68e37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c695fd6ca0 0x55c695ffc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.632+0000 7f68e37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c695fd6ca0 0x55c695ffc0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57836/0 (socket says 172.21.15.43:57836) 2024-04-25T09:01:57.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.632+0000 7f68e37fe640 1 -- 172.21.15.43:0/1096964185 learned_addr learned my addr 172.21.15.43:0/1096964185 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:57.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.632+0000 7f68e2ffd640 1 --2- 172.21.15.43:0/1096964185 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c696001760 0x55c696035700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.632+0000 7f68e3fff640 1 --2- 172.21.15.43:0/1096964185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c696013370 0x55c696035c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.632+0000 7f68e37fe640 1 -- 172.21.15.43:0/1096964185 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c696001760 msgr2=0x55c696035700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:57.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.632+0000 7f68e37fe640 1 --2- 172.21.15.43:0/1096964185 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c696001760 0x55c696035700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.632+0000 7f68e37fe640 1 -- 172.21.15.43:0/1096964185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c696013370 msgr2=0x55c696035c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:57.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.632+0000 7f68e37fe640 1 --2- 172.21.15.43:0/1096964185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c696013370 0x55c696035c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.632+0000 7f68e37fe640 1 -- 172.21.15.43:0/1096964185 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c696017a90 con 0x55c695fd6ca0 2024-04-25T09:01:57.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.632+0000 7f68e2ffd640 1 --2- 172.21.15.43:0/1096964185 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c696001760 0x55c696035700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:57.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.632+0000 7f68e3fff640 1 --2- 172.21.15.43:0/1096964185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c696013370 0x55c696035c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:57.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.632+0000 7f68e37fe640 1 --2- 172.21.15.43:0/1096964185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c695fd6ca0 0x55c695ffc0a0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f68d4035040 tx=0x7f68d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6726aaaa9a8446d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:57.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.632+0000 7f68c27fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:01:57.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.632+0000 7f68c27fc640 1 -- 172.21.15.43:0/1096964185 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d4037040 con 0x55c695fd6ca0 2024-04-25T09:01:57.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.633+0000 7f68c27fc640 1 -- 172.21.15.43:0/1096964185 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f68d4059b00 con 0x55c695fd6ca0 2024-04-25T09:01:57.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.633+0000 7f68ecfae740 1 -- 172.21.15.43:0/1096964185 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f68d404f040 con 0x55c695fd6ca0 2024-04-25T09:01:57.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.633+0000 7f68c27fc640 1 -- 172.21.15.43:0/1096964185 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68d4059d00 con 0x55c695fd6ca0 2024-04-25T09:01:57.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.633+0000 7f68ecfae740 10 client.8307 fetch_fsmap learned FSMap version 61 2024-04-25T09:01:57.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.633+0000 7f68ecfae740 1 -- 172.21.15.43:0/1096964185 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x55c69602b9f0 con 0x55c695fd6ca0 2024-04-25T09:01:57.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.633+0000 7f68c27fc640 1 -- 172.21.15.43:0/1096964185 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d404f040 con 0x55c695fd6ca0 2024-04-25T09:01:57.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.633+0000 7f68c27fc640 1 -- 172.21.15.43:0/1096964185 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 61) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f68d4062070 con 0x55c695fd6ca0 2024-04-25T09:01:57.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.633+0000 7f68ecfae740 10 client.8307 fetch_fsmap finished waiting for FSMap version 61 2024-04-25T09:01:57.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.633+0000 7f68ecfae740 10 client.8307 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-25T09:01:57.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.633+0000 7f68ecfae740 20 client.8307 populate_metadata read hostname 'smithi043' 2024-04-25T09:01:57.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.633+0000 7f68ecfae740 1 --2- 172.21.15.43:0/1096964185 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55c695ffb560 0x55c69602ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.633+0000 7f68ecfae740 4 client.8307 mds_command: new command op to 8094 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-25T09:01:57.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.633+0000 7f68ecfae740 1 -- 172.21.15.43:0/1096964185 --> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x55c695f41420 con 0x55c695ffb560 2024-04-25T09:01:57.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.633+0000 7f68e2ffd640 1 --2- 172.21.15.43:0/1096964185 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55c695ffb560 0x55c69602ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.634+0000 7f68e2ffd640 1 --2- 172.21.15.43:0/1096964185 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55c695ffb560 0x55c69602ae00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c739c6daa778a781 server_cookie=a080c0c847e2d0e in_seq=0 out_seq=0 2024-04-25T09:01:57.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.634+0000 7f68c27fc640 10 client.8307 ms_handle_connect on v2:172.21.15.193:6834/853943267 2024-04-25T09:01:57.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.634+0000 7f68c27fc640 1 -- 172.21.15.43:0/1096964185 <== mds.0 v2:172.21.15.193:6834/853943267 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55c695f41420 con 0x55c695ffb560 2024-04-25T09:01:57.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.634+0000 7f68c27fc640 10 client.8307 handle_command_reply: tid=0 2024-04-25T09:01:57.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.634+0000 7f68ecfae740 1 client.8307 shutdown 2024-04-25T09:01:57.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.634+0000 7f68ecfae740 1 -- 172.21.15.43:0/1096964185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c695fd6ca0 msgr2=0x55c695ffc0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:57.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.634+0000 7f68ecfae740 1 --2- 172.21.15.43:0/1096964185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c695fd6ca0 0x55c695ffc0a0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f68d4035040 tx=0x7f68d4004730 comp rx=0 tx=0).stop 2024-04-25T09:01:57.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.634+0000 7f68ecfae740 1 -- 172.21.15.43:0/1096964185 shutdown_connections 2024-04-25T09:01:57.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.634+0000 7f68ecfae740 1 --2- 172.21.15.43:0/1096964185 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55c695ffb560 0x55c69602ae00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.634+0000 7f68ecfae740 1 --2- 172.21.15.43:0/1096964185 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c696013370 0x55c696035c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.634+0000 7f68ecfae740 1 --2- 172.21.15.43:0/1096964185 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c696001760 0x55c696035700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.634+0000 7f68c27fc640 0 client.8307 ms_handle_reset on v2:172.21.15.193:6834/853943267 2024-04-25T09:01:57.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.634+0000 7f68ecfae740 1 --2- 172.21.15.43:0/1096964185 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c695fd6ca0 0x55c695ffc0a0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.634+0000 7f68ecfae740 1 -- 172.21.15.43:0/1096964185 >> 172.21.15.43:0/1096964185 conn(0x55c695f9e870 msgr2=0x55c695ffecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:57.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.634+0000 7f68ecfae740 1 -- 172.21.15.43:0/1096964185 shutdown_connections 2024-04-25T09:01:57.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.634+0000 7f68ecfae740 1 -- 172.21.15.43:0/1096964185 wait complete. 2024-04-25T09:01:57.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.634+0000 7f68ecfae740 20 client.8307 trim_cache size 0 max 16384 2024-04-25T09:01:57.629 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:01:57.629 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0 2024-04-25T09:01:57.629 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:01:57.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.634+0000 7f68ea1ce640 1 -- 172.21.15.43:0/1462484298 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f68d006b4d0 msgr2=0x7f68d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:57.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.634+0000 7f68ea1ce640 1 --2- 172.21.15.43:0/1462484298 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f68d006b4d0 0x7f68d006d990 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f68d402a140 tx=0x7f68d402e000 comp rx=0 tx=0).stop 2024-04-25T09:01:57.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.634+0000 7f68ea1ce640 1 -- 172.21.15.43:0/1462484298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68e415dd80 msgr2=0x7f68e4159540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:57.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.634+0000 7f68ea1ce640 1 --2- 172.21.15.43:0/1462484298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68e415dd80 0x7f68e4159540 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f68dc00c6c0 tx=0x7f68dc00cb80 comp rx=0 tx=0).stop 2024-04-25T09:01:57.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.635+0000 7f68ea1ce640 1 -- 172.21.15.43:0/1462484298 shutdown_connections 2024-04-25T09:01:57.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.635+0000 7f68ea1ce640 1 --2- 172.21.15.43:0/1462484298 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68e415dd80 0x7f68e4159540 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.635+0000 7f68ea1ce640 1 --2- 172.21.15.43:0/1462484298 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f68e40d17b0 0x7f68e4157cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.635+0000 7f68ea1ce640 1 --2- 172.21.15.43:0/1462484298 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f68d006b4d0 0x7f68d006d990 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.635+0000 7f68ea1ce640 1 --2- 172.21.15.43:0/1462484298 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68e40b1cd0 0x7f68e41600b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.635+0000 7f68ea1ce640 1 -- 172.21.15.43:0/1462484298 >> 172.21.15.43:0/1462484298 conn(0x7f68e4072aa0 msgr2=0x7f68e406ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:57.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.635+0000 7f68ea1ce640 1 -- 172.21.15.43:0/1462484298 shutdown_connections 2024-04-25T09:01:57.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.635+0000 7f68ea1ce640 1 -- 172.21.15.43:0/1462484298 wait complete. 2024-04-25T09:01:57.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:01:57.635 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T09:01:57.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.748+0000 7f53eefce640 1 Processor -- start 2024-04-25T09:01:57.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.748+0000 7f53eefce640 1 -- start start 2024-04-25T09:01:57.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.749+0000 7f53eefce640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f53e80d04a0 0x7f53e815ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.749+0000 7f53eefce640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f53e80ce8c0 0x7f53e80d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.749+0000 7f53eefce640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53e80cf110 con 0x7f53e815dec0 2024-04-25T09:01:57.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.749+0000 7f53eefce640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53e80cf3f0 con 0x7f53e80ce8c0 2024-04-25T09:01:57.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.749+0000 7f53eefce640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53e80cf6d0 con 0x7f53e80d04a0 2024-04-25T09:01:57.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.749+0000 7f53ed56b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f53e80ce8c0 0x7f53e80d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.749+0000 7f53e7fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f53e815dec0 0x7f53e80d0880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56774/0 (socket says 172.21.15.43:56774) 2024-04-25T09:01:57.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.749+0000 7f53e7fff640 1 -- 172.21.15.43:0/1224638642 learned_addr learned my addr 172.21.15.43:0/1224638642 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:57.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.749+0000 7f53ecd6a640 1 --2- 172.21.15.43:0/1224638642 >> v2:172.21.15.193:3300/0 conn(0x7f53e80d04a0 0x7f53e815ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.749+0000 7f53ed56b640 1 -- 172.21.15.43:0/1224638642 >> v2:172.21.15.193:3300/0 conn(0x7f53e80d04a0 msgr2=0x7f53e815ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:57.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.749+0000 7f53ed56b640 1 --2- 172.21.15.43:0/1224638642 >> v2:172.21.15.193:3300/0 conn(0x7f53e80d04a0 0x7f53e815ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.749+0000 7f53ed56b640 1 -- 172.21.15.43:0/1224638642 >> v1:172.21.15.43:6789/0 conn(0x7f53e815dec0 legacy=0x7f53e80d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:57.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.750+0000 7f53e77fe640 1 -- 172.21.15.43:0/1224638642 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1891901292 0 0) 0x7f53e80cf110 con 0x7f53e815dec0 2024-04-25T09:01:57.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.750+0000 7f53ed56b640 1 -- 172.21.15.43:0/1224638642 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e80cf9b0 con 0x7f53e80ce8c0 2024-04-25T09:01:57.739 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.750+0000 7f53ed56b640 1 --2- 172.21.15.43:0/1224638642 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53e80ce8c0 0x7f53e80d17a0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f53e80b6690 tx=0x7f53e002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e313226eee2a3a7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:57.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.750+0000 7f53e77fe640 1 -- 172.21.15.43:0/1224638642 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e0007740 con 0x7f53e80ce8c0 2024-04-25T09:01:57.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.750+0000 7f53e77fe640 1 -- 172.21.15.43:0/1224638642 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e00078e0 con 0x7f53e80ce8c0 2024-04-25T09:01:57.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.750+0000 7f53e77fe640 1 -- 172.21.15.43:0/1224638642 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e0007cb0 con 0x7f53e80ce8c0 2024-04-25T09:01:57.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.751+0000 7f53eefce640 1 -- 172.21.15.43:0/1224638642 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53e80ce8c0 msgr2=0x7f53e80d17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:57.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.751+0000 7f53eefce640 1 --2- 172.21.15.43:0/1224638642 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53e80ce8c0 0x7f53e80d17a0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f53e80b6690 tx=0x7f53e002a680 comp rx=0 tx=0).stop 2024-04-25T09:01:57.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.751+0000 7f53eefce640 1 -- 172.21.15.43:0/1224638642 shutdown_connections 2024-04-25T09:01:57.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.751+0000 7f53eefce640 1 --2- 172.21.15.43:0/1224638642 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53e80ce8c0 0x7f53e80d17a0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.751+0000 7f53eefce640 1 --2- 172.21.15.43:0/1224638642 >> v2:172.21.15.193:3300/0 conn(0x7f53e80d04a0 0x7f53e815ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.751+0000 7f53eefce640 1 -- 172.21.15.43:0/1224638642 >> 172.21.15.43:0/1224638642 conn(0x7f53e809cd40 msgr2=0x7f53e80d3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:57.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.751+0000 7f53eefce640 1 -- 172.21.15.43:0/1224638642 shutdown_connections 2024-04-25T09:01:57.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.751+0000 7f53eefce640 1 -- 172.21.15.43:0/1224638642 wait complete. 2024-04-25T09:01:57.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.752+0000 7f53eefce640 1 Processor -- start 2024-04-25T09:01:57.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.752+0000 7f53eefce640 1 -- start start 2024-04-25T09:01:57.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.753+0000 7f53eefce640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f53e80ce8c0 0x7f53e80a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.753+0000 7f53eefce640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53e80d04a0 0x7f53e80b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.753+0000 7f53eefce640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53e815dec0 0x7f53e80b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.753+0000 7f53eefce640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e80cee50 con 0x7f53e815dec0 2024-04-25T09:01:57.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.753+0000 7f53eefce640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e80cecd0 con 0x7f53e80d04a0 2024-04-25T09:01:57.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.753+0000 7f53eefce640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e80d1ce0 con 0x7f53e80ce8c0 2024-04-25T09:01:57.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.753+0000 7f53ed56b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53e815dec0 0x7f53e80b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.753+0000 7f53ed56b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53e815dec0 0x7f53e80b34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57866/0 (socket says 172.21.15.43:57866) 2024-04-25T09:01:57.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.753+0000 7f53ed56b640 1 -- 172.21.15.43:0/1944101122 learned_addr learned my addr 172.21.15.43:0/1944101122 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:57.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.753+0000 7f53e7fff640 1 --2- 172.21.15.43:0/1944101122 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53e80d04a0 0x7f53e80b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.753+0000 7f53ecd6a640 1 --2- 172.21.15.43:0/1944101122 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f53e80ce8c0 0x7f53e80a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.753+0000 7f53ed56b640 1 -- 172.21.15.43:0/1944101122 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f53e80ce8c0 msgr2=0x7f53e80a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:57.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.753+0000 7f53ed56b640 1 --2- 172.21.15.43:0/1944101122 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f53e80ce8c0 0x7f53e80a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.753+0000 7f53ed56b640 1 -- 172.21.15.43:0/1944101122 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53e80d04a0 msgr2=0x7f53e80b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:57.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.753+0000 7f53ed56b640 1 --2- 172.21.15.43:0/1944101122 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53e80d04a0 0x7f53e80b2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.753+0000 7f53ed56b640 1 -- 172.21.15.43:0/1944101122 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e80c9f60 con 0x7f53e815dec0 2024-04-25T09:01:57.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.754+0000 7f53ed56b640 1 --2- 172.21.15.43:0/1944101122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53e815dec0 0x7f53e80b34c0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f53e002ac70 tx=0x7f53e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:57.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.754+0000 7f53e57fa640 1 -- 172.21.15.43:0/1944101122 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e0040070 con 0x7f53e815dec0 2024-04-25T09:01:57.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.754+0000 7f53e57fa640 1 -- 172.21.15.43:0/1944101122 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e0031bc0 con 0x7f53e815dec0 2024-04-25T09:01:57.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.754+0000 7f53eefce640 1 -- 172.21.15.43:0/1944101122 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e80b6c90 con 0x7f53e815dec0 2024-04-25T09:01:57.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.754+0000 7f53e57fa640 1 -- 172.21.15.43:0/1944101122 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e0009450 con 0x7f53e815dec0 2024-04-25T09:01:57.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.754+0000 7f53eefce640 1 -- 172.21.15.43:0/1944101122 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e80b7250 con 0x7f53e815dec0 2024-04-25T09:01:57.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.756+0000 7f53e57fa640 1 -- 172.21.15.43:0/1944101122 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f53e0039040 con 0x7f53e815dec0 2024-04-25T09:01:57.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.756+0000 7f53e57fa640 1 --2- 172.21.15.43:0/1944101122 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f53d406b4d0 0x7f53d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.756+0000 7f53e57fa640 1 -- 172.21.15.43:0/1944101122 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f53e00ab980 con 0x7f53e815dec0 2024-04-25T09:01:57.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.756+0000 7f53ecd6a640 1 --2- 172.21.15.43:0/1944101122 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f53d406b4d0 0x7f53d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.757+0000 7f53ecd6a640 1 --2- 172.21.15.43:0/1944101122 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f53d406b4d0 0x7f53d406d990 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f53d8009b40 tx=0x7f53d8009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:57.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.757+0000 7f53f1dab740 1 Processor -- start 2024-04-25T09:01:57.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.757+0000 7f53f1dab740 1 -- start start 2024-04-25T09:01:57.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.758+0000 7f53f1dab740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f451e38b10 0x55f451e3b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.758+0000 7f53f1dab740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f451e38ef0 0x55f451e3b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.758+0000 7f53f1dab740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f451e0b770 0x55f451e0ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.758+0000 7f53f1dab740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f451d74da0 con 0x55f451e38b10 2024-04-25T09:01:57.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.758+0000 7f53f1dab740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f451d1db80 con 0x55f451e0b770 2024-04-25T09:01:57.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.758+0000 7f53f1dab740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f451d70e40 con 0x55f451e38ef0 2024-04-25T09:01:57.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.758+0000 7f53ecd6a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f451e38b10 0x55f451e3b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.758+0000 7f53ecd6a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f451e38b10 0x55f451e3b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57876/0 (socket says 172.21.15.43:57876) 2024-04-25T09:01:57.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.758+0000 7f53ecd6a640 1 -- 172.21.15.43:0/475736541 learned_addr learned my addr 172.21.15.43:0/475736541 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:57.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.758+0000 7f53ed56b640 1 --2- 172.21.15.43:0/475736541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f451e0b770 0x55f451e0ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.758+0000 7f53ecd6a640 1 -- 172.21.15.43:0/475736541 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f451e38ef0 msgr2=0x55f451e3b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:57.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.758+0000 7f53ecd6a640 1 --2- 172.21.15.43:0/475736541 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f451e38ef0 0x55f451e3b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.758+0000 7f53ecd6a640 1 -- 172.21.15.43:0/475736541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f451e0b770 msgr2=0x55f451e0ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:57.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.758+0000 7f53ecd6a640 1 --2- 172.21.15.43:0/475736541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f451e0b770 0x55f451e0ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.758+0000 7f53ecd6a640 1 -- 172.21.15.43:0/475736541 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f451d27f10 con 0x55f451e38b10 2024-04-25T09:01:57.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.758+0000 7f53ecd6a640 1 --2- 172.21.15.43:0/475736541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f451e38b10 0x55f451e3b320 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f53d8006a90 tx=0x7f53d8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcbe8b29504de96a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:57.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.758+0000 7f53ce7fc640 1 -- 172.21.15.43:0/475736541 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d801b070 con 0x55f451e38b10 2024-04-25T09:01:57.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.758+0000 7f53ce7fc640 1 -- 172.21.15.43:0/475736541 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d8027070 con 0x55f451e38b10 2024-04-25T09:01:57.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.758+0000 7f53ce7fc640 1 -- 172.21.15.43:0/475736541 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d8003a40 con 0x55f451e38b10 2024-04-25T09:01:57.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.759+0000 7f53f1dab740 1 -- 172.21.15.43:0/475736541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f451e38b10 msgr2=0x55f451e3b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:57.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.759+0000 7f53f1dab740 1 --2- 172.21.15.43:0/475736541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f451e38b10 0x55f451e3b320 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f53d8006a90 tx=0x7f53d8014000 comp rx=0 tx=0).stop 2024-04-25T09:01:57.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.759+0000 7f53f1dab740 1 -- 172.21.15.43:0/475736541 shutdown_connections 2024-04-25T09:01:57.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.759+0000 7f53f1dab740 1 --2- 172.21.15.43:0/475736541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f451e0b770 0x55f451e0ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.759+0000 7f53f1dab740 1 --2- 172.21.15.43:0/475736541 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f451e38ef0 0x55f451e3b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.759+0000 7f53f1dab740 1 --2- 172.21.15.43:0/475736541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f451e38b10 0x55f451e3b320 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.759+0000 7f53f1dab740 1 -- 172.21.15.43:0/475736541 >> 172.21.15.43:0/475736541 conn(0x55f451e0d9e0 msgr2=0x55f451e56320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:57.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.759+0000 7f53f1dab740 1 -- 172.21.15.43:0/475736541 shutdown_connections 2024-04-25T09:01:57.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.759+0000 7f53f1dab740 1 -- 172.21.15.43:0/475736541 wait complete. 2024-04-25T09:01:57.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.760+0000 7f53f1dab740 1 -- start start 2024-04-25T09:01:57.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.760+0000 7f53f1dab740 1 Processor -- start 2024-04-25T09:01:57.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.761+0000 7f53f1dab740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f451e38b10 0x55f451e591a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.761+0000 7f53f1dab740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f451e38ef0 0x55f451e65e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.761+0000 7f53f1dab740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f451e2fea0 0x55f451e663c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.761+0000 7f53f1dab740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f451e10b90 con 0x55f451e2fea0 2024-04-25T09:01:57.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.761+0000 7f53f1dab740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f451d134b0 con 0x55f451e38ef0 2024-04-25T09:01:57.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.761+0000 7f53f1dab740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f451e10e70 con 0x55f451e38b10 2024-04-25T09:01:57.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.761+0000 7f53f1dab740 10 client.0 _ll_register_callbacks cb 0x55f451d53180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:01:57.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.761+0000 7f53ed56b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f451e2fea0 0x55f451e663c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.761+0000 7f53ed56b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f451e2fea0 0x55f451e663c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57882/0 (socket says 172.21.15.43:57882) 2024-04-25T09:01:57.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.761+0000 7f53ed56b640 1 -- 172.21.15.43:0/1115373905 learned_addr learned my addr 172.21.15.43:0/1115373905 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:57.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.761+0000 7f53e7fff640 1 --2- 172.21.15.43:0/1115373905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f451e38ef0 0x55f451e65e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.761+0000 7f53ecd6a640 1 --2- 172.21.15.43:0/1115373905 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f451e38b10 0x55f451e591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.761+0000 7f53ed56b640 1 -- 172.21.15.43:0/1115373905 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f451e38b10 msgr2=0x55f451e591a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:57.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.761+0000 7f53ed56b640 1 --2- 172.21.15.43:0/1115373905 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f451e38b10 0x55f451e591a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.761+0000 7f53ed56b640 1 -- 172.21.15.43:0/1115373905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f451e38ef0 msgr2=0x55f451e65e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:57.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.761+0000 7f53ed56b640 1 --2- 172.21.15.43:0/1115373905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f451e38ef0 0x55f451e65e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.761+0000 7f53ed56b640 1 -- 172.21.15.43:0/1115373905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f451b537d0 con 0x55f451e2fea0 2024-04-25T09:01:57.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.761+0000 7f53ed56b640 1 --2- 172.21.15.43:0/1115373905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f451e2fea0 0x55f451e663c0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f53e00a9e20 tx=0x7f53e007a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b195a8f8ed8777 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:57.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.761+0000 7f53c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:01:57.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.761+0000 7f53c77fe640 1 -- 172.21.15.43:0/1115373905 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e00a9060 con 0x55f451e2fea0 2024-04-25T09:01:57.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.761+0000 7f53c77fe640 1 -- 172.21.15.43:0/1115373905 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f53e007a380 con 0x55f451e2fea0 2024-04-25T09:01:57.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.761+0000 7f53f1dab740 1 -- 172.21.15.43:0/1115373905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53d8027070 con 0x55f451e2fea0 2024-04-25T09:01:57.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.762+0000 7f53c77fe640 1 -- 172.21.15.43:0/1115373905 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53e0098050 con 0x55f451e2fea0 2024-04-25T09:01:57.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.762+0000 7f53f1dab740 10 client.8322 fetch_fsmap learned FSMap version 61 2024-04-25T09:01:57.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.762+0000 7f53f1dab740 1 -- 172.21.15.43:0/1115373905 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x55f451a893f0 con 0x55f451e2fea0 2024-04-25T09:01:57.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.762+0000 7f53c77fe640 1 -- 172.21.15.43:0/1115373905 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d8027070 con 0x55f451e2fea0 2024-04-25T09:01:57.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.762+0000 7f53c77fe640 1 -- 172.21.15.43:0/1115373905 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 61) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f53e0082380 con 0x55f451e2fea0 2024-04-25T09:01:57.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.762+0000 7f53f1dab740 10 client.8322 fetch_fsmap finished waiting for FSMap version 61 2024-04-25T09:01:57.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.762+0000 7f53f1dab740 10 client.8322 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-25T09:01:57.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.762+0000 7f53f1dab740 20 client.8322 populate_metadata read hostname 'smithi043' 2024-04-25T09:01:57.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.762+0000 7f53f1dab740 1 --2- 172.21.15.43:0/1115373905 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55f451e58aa0 0x55f451e73600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.762+0000 7f53f1dab740 4 client.8322 mds_command: new command op to 8094 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:01:57.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.762+0000 7f53f1dab740 1 -- 172.21.15.43:0/1115373905 --> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f451d822e0 con 0x55f451e58aa0 2024-04-25T09:01:57.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.762+0000 7f53ecd6a640 1 --2- 172.21.15.43:0/1115373905 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55f451e58aa0 0x55f451e73600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.762+0000 7f53c77fe640 10 client.8322 ms_handle_connect on v2:172.21.15.193:6834/853943267 2024-04-25T09:01:57.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.762+0000 7f53ecd6a640 1 --2- 172.21.15.43:0/1115373905 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55f451e58aa0 0x55f451e73600 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ced602134d250f server_cookie=edca61144c31537a in_seq=0 out_seq=0 2024-04-25T09:01:57.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.763+0000 7f53c77fe640 1 -- 172.21.15.43:0/1115373905 <== mds.0 v2:172.21.15.193:6834/853943267 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f451d822e0 con 0x55f451e58aa0 2024-04-25T09:01:57.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.763+0000 7f53c77fe640 10 client.8322 handle_command_reply: tid=0 2024-04-25T09:01:57.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.763+0000 7f53f1dab740 1 client.8322 shutdown 2024-04-25T09:01:57.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.763+0000 7f53f1dab740 1 -- 172.21.15.43:0/1115373905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f451e2fea0 msgr2=0x55f451e663c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:57.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.763+0000 7f53f1dab740 1 --2- 172.21.15.43:0/1115373905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f451e2fea0 0x55f451e663c0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f53e00a9e20 tx=0x7f53e007a1e0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.763+0000 7f53f1dab740 1 -- 172.21.15.43:0/1115373905 shutdown_connections 2024-04-25T09:01:57.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.763+0000 7f53f1dab740 1 --2- 172.21.15.43:0/1115373905 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f451e2fea0 0x55f451e663c0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.763+0000 7f53f1dab740 1 --2- 172.21.15.43:0/1115373905 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55f451e58aa0 0x55f451e73600 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.763+0000 7f53f1dab740 1 --2- 172.21.15.43:0/1115373905 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f451e38ef0 0x55f451e65e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.763+0000 7f53f1dab740 1 --2- 172.21.15.43:0/1115373905 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f451e38b10 0x55f451e591a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.763+0000 7f53c77fe640 0 client.8322 ms_handle_reset on v2:172.21.15.193:6834/853943267 2024-04-25T09:01:57.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.763+0000 7f53f1dab740 1 -- 172.21.15.43:0/1115373905 >> 172.21.15.43:0/1115373905 conn(0x55f451e0b770 msgr2=0x55f451e36990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:57.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.763+0000 7f53f1dab740 1 -- 172.21.15.43:0/1115373905 shutdown_connections 2024-04-25T09:01:57.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.763+0000 7f53f1dab740 1 -- 172.21.15.43:0/1115373905 wait complete. 2024-04-25T09:01:57.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.763+0000 7f53f1dab740 20 client.8322 trim_cache size 0 max 16384 2024-04-25T09:01:57.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.778+0000 7f53f1dab740 1 Processor -- start 2024-04-25T09:01:57.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.778+0000 7f53f1dab740 1 -- start start 2024-04-25T09:01:57.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.779+0000 7f53f1dab740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f451e2fea0 0x55f451e59770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.779+0000 7f53f1dab740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f451e38b10 0x55f451e59cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.779+0000 7f53f1dab740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f451e38ef0 0x55f451e64c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.779+0000 7f53f1dab740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e81ffc50 con 0x55f451e38b10 2024-04-25T09:01:57.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.779+0000 7f53f1dab740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e80b8b50 con 0x55f451e38ef0 2024-04-25T09:01:57.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.779+0000 7f53f1dab740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e80b5af0 con 0x55f451e2fea0 2024-04-25T09:01:57.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.779+0000 7f53e7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f451e38b10 0x55f451e59cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.779+0000 7f53e7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f451e38b10 0x55f451e59cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57886/0 (socket says 172.21.15.43:57886) 2024-04-25T09:01:57.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.779+0000 7f53e7fff640 1 -- 172.21.15.43:0/131328571 learned_addr learned my addr 172.21.15.43:0/131328571 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:57.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.779+0000 7f53e7fff640 1 -- 172.21.15.43:0/131328571 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f451e2fea0 msgr2=0x55f451e59770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:57.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.779+0000 7f53e7fff640 1 --2- 172.21.15.43:0/131328571 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f451e2fea0 0x55f451e59770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.779+0000 7f53e7fff640 1 -- 172.21.15.43:0/131328571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f451e38ef0 msgr2=0x55f451e64c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:57.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.779+0000 7f53ed56b640 1 --2- 172.21.15.43:0/131328571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f451e38ef0 0x55f451e64c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.779+0000 7f53e7fff640 1 --2- 172.21.15.43:0/131328571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f451e38ef0 0x55f451e64c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.779+0000 7f53e7fff640 1 -- 172.21.15.43:0/131328571 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f451a490f0 con 0x55f451e38b10 2024-04-25T09:01:57.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.779+0000 7f53ed56b640 1 --2- 172.21.15.43:0/131328571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f451e38ef0 0x55f451e64c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:57.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.780+0000 7f53e7fff640 1 --2- 172.21.15.43:0/131328571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f451e38b10 0x55f451e59cb0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f53dc007fd0 tx=0x7f53dc007110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ed545ca8d836 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:57.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.780+0000 7f53cdffb640 1 -- 172.21.15.43:0/131328571 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc0016c0 con 0x55f451e38b10 2024-04-25T09:01:57.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.780+0000 7f53cdffb640 1 -- 172.21.15.43:0/131328571 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53dc01a020 con 0x55f451e38b10 2024-04-25T09:01:57.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.780+0000 7f53cdffb640 1 -- 172.21.15.43:0/131328571 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc0018c0 con 0x55f451e38b10 2024-04-25T09:01:57.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.780+0000 7f53f1dab740 1 -- 172.21.15.43:0/131328571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f451e38b10 msgr2=0x55f451e59cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:57.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.780+0000 7f53f1dab740 1 --2- 172.21.15.43:0/131328571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f451e38b10 0x55f451e59cb0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f53dc007fd0 tx=0x7f53dc007110 comp rx=0 tx=0).stop 2024-04-25T09:01:57.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.780+0000 7f53f1dab740 1 -- 172.21.15.43:0/131328571 shutdown_connections 2024-04-25T09:01:57.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.780+0000 7f53f1dab740 1 --2- 172.21.15.43:0/131328571 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f451e38ef0 0x55f451e64c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.780+0000 7f53f1dab740 1 --2- 172.21.15.43:0/131328571 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f451e38b10 0x55f451e59cb0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.780+0000 7f53f1dab740 1 --2- 172.21.15.43:0/131328571 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f451e2fea0 0x55f451e59770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.780+0000 7f53f1dab740 1 -- 172.21.15.43:0/131328571 >> 172.21.15.43:0/131328571 conn(0x55f451e0b770 msgr2=0x55f451e12600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:57.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.780+0000 7f53f1dab740 1 -- 172.21.15.43:0/131328571 shutdown_connections 2024-04-25T09:01:57.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.780+0000 7f53f1dab740 1 -- 172.21.15.43:0/131328571 wait complete. 2024-04-25T09:01:57.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.781+0000 7f53f1dab740 1 -- start start 2024-04-25T09:01:57.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.781+0000 7f53f1dab740 1 Processor -- start 2024-04-25T09:01:57.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.782+0000 7f53f1dab740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f451e2fea0 0x55f451e36090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.782+0000 7f53f1dab740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f451e38b10 0x55f451ec01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.782+0000 7f53f1dab740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f451e38ef0 0x55f451eba150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.782+0000 7f53f1dab740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f451d822e0 con 0x55f451e38b10 2024-04-25T09:01:57.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.782+0000 7f53f1dab740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e80b8550 con 0x55f451e38ef0 2024-04-25T09:01:57.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.782+0000 7f53f1dab740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f451e0bd80 con 0x55f451e2fea0 2024-04-25T09:01:57.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.782+0000 7f53f1dab740 10 client.0 _ll_register_callbacks cb 0x7f53d80095f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:01:57.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.782+0000 7f53e7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f451e38b10 0x55f451ec01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.782+0000 7f53e7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f451e38b10 0x55f451ec01b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57888/0 (socket says 172.21.15.43:57888) 2024-04-25T09:01:57.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.782+0000 7f53e7fff640 1 -- 172.21.15.43:0/3516043941 learned_addr learned my addr 172.21.15.43:0/3516043941 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:57.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.782+0000 7f53ed56b640 1 --2- 172.21.15.43:0/3516043941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f451e38ef0 0x55f451eba150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.782+0000 7f53ecd6a640 1 --2- 172.21.15.43:0/3516043941 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f451e2fea0 0x55f451e36090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.782+0000 7f53e7fff640 1 -- 172.21.15.43:0/3516043941 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f451e2fea0 msgr2=0x55f451e36090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:57.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.782+0000 7f53e7fff640 1 --2- 172.21.15.43:0/3516043941 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f451e2fea0 0x55f451e36090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.782+0000 7f53e7fff640 1 -- 172.21.15.43:0/3516043941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f451e38ef0 msgr2=0x55f451eba150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:57.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.782+0000 7f53e7fff640 1 --2- 172.21.15.43:0/3516043941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f451e38ef0 0x55f451eba150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.782+0000 7f53e7fff640 1 -- 172.21.15.43:0/3516043941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f451e539d0 con 0x55f451e38b10 2024-04-25T09:01:57.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.782+0000 7f53ecd6a640 1 --2- 172.21.15.43:0/3516043941 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f451e2fea0 0x55f451e36090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:01:57.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.782+0000 7f53e7fff640 1 --2- 172.21.15.43:0/3516043941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f451e38b10 0x55f451ec01b0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009010 tx=0x7f53dc000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa7f6ae67c8b9635 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:57.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.782+0000 7f53c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:01:57.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.783+0000 7f53c77fe640 1 -- 172.21.15.43:0/3516043941 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc028060 con 0x55f451e38b10 2024-04-25T09:01:57.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.783+0000 7f53c77fe640 1 -- 172.21.15.43:0/3516043941 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f53dc015640 con 0x55f451e38b10 2024-04-25T09:01:57.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.783+0000 7f53f1dab740 1 -- 172.21.15.43:0/3516043941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53dc01a020 con 0x55f451e38b10 2024-04-25T09:01:57.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.783+0000 7f53c77fe640 1 -- 172.21.15.43:0/3516043941 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53dc02e050 con 0x55f451e38b10 2024-04-25T09:01:57.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.783+0000 7f53f1dab740 10 client.8328 fetch_fsmap learned FSMap version 61 2024-04-25T09:01:57.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.783+0000 7f53f1dab740 1 -- 172.21.15.43:0/3516043941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x55f451e533f0 con 0x55f451e38b10 2024-04-25T09:01:57.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.783+0000 7f53c77fe640 1 -- 172.21.15.43:0/3516043941 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53dc01a020 con 0x55f451e38b10 2024-04-25T09:01:57.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.783+0000 7f53c77fe640 1 -- 172.21.15.43:0/3516043941 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 61) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f53dc031020 con 0x55f451e38b10 2024-04-25T09:01:57.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.783+0000 7f53f1dab740 10 client.8328 fetch_fsmap finished waiting for FSMap version 61 2024-04-25T09:01:57.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.783+0000 7f53f1dab740 10 client.8328 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-25T09:01:57.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.783+0000 7f53f1dab740 20 client.8328 populate_metadata read hostname 'smithi043' 2024-04-25T09:01:57.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.783+0000 7f53f1dab740 1 --2- 172.21.15.43:0/3516043941 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55f451ec3fd0 0x55f451e65360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.783+0000 7f53f1dab740 4 client.8328 mds_command: new command op to 8094 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:01:57.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.783+0000 7f53f1dab740 1 -- 172.21.15.43:0/3516043941 --> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f451d7c630 con 0x55f451ec3fd0 2024-04-25T09:01:57.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.783+0000 7f53ecd6a640 1 --2- 172.21.15.43:0/3516043941 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55f451ec3fd0 0x55f451e65360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.784+0000 7f53ecd6a640 1 --2- 172.21.15.43:0/3516043941 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55f451ec3fd0 0x55f451e65360 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d382df671c29b238 server_cookie=a70cc84bd43340dd in_seq=0 out_seq=0 2024-04-25T09:01:57.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.784+0000 7f53c77fe640 10 client.8328 ms_handle_connect on v2:172.21.15.193:6834/853943267 2024-04-25T09:01:57.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.784+0000 7f53c77fe640 1 -- 172.21.15.43:0/3516043941 <== mds.0 v2:172.21.15.193:6834/853943267 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x55f451d7c630 con 0x55f451ec3fd0 2024-04-25T09:01:57.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.784+0000 7f53c77fe640 10 client.8328 handle_command_reply: tid=0 2024-04-25T09:01:57.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.784+0000 7f53f1dab740 1 client.8328 shutdown 2024-04-25T09:01:57.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.784+0000 7f53f1dab740 1 -- 172.21.15.43:0/3516043941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f451e38b10 msgr2=0x55f451ec01b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:57.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.784+0000 7f53f1dab740 1 --2- 172.21.15.43:0/3516043941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f451e38b10 0x55f451ec01b0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f53dc009010 tx=0x7f53dc000ed0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.784+0000 7f53f1dab740 1 -- 172.21.15.43:0/3516043941 shutdown_connections 2024-04-25T09:01:57.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.784+0000 7f53f1dab740 1 --2- 172.21.15.43:0/3516043941 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55f451ec3fd0 0x55f451e65360 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.784+0000 7f53f1dab740 1 --2- 172.21.15.43:0/3516043941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f451e38ef0 0x55f451eba150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.784+0000 7f53c77fe640 0 client.8328 ms_handle_reset on v2:172.21.15.193:6834/853943267 2024-04-25T09:01:57.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.784+0000 7f53f1dab740 1 --2- 172.21.15.43:0/3516043941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f451e38b10 0x55f451ec01b0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.784+0000 7f53f1dab740 1 --2- 172.21.15.43:0/3516043941 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f451e2fea0 0x55f451e36090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.784+0000 7f53f1dab740 1 -- 172.21.15.43:0/3516043941 >> 172.21.15.43:0/3516043941 conn(0x55f451e0b770 msgr2=0x55f451e77b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:57.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.784+0000 7f53f1dab740 1 -- 172.21.15.43:0/3516043941 shutdown_connections 2024-04-25T09:01:57.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.784+0000 7f53f1dab740 1 -- 172.21.15.43:0/3516043941 wait complete. 2024-04-25T09:01:57.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.784+0000 7f53f1dab740 20 client.8328 trim_cache size 0 max 16384 2024-04-25T09:01:57.779 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:01:57.779 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T09:01:57.780 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-25T09:01:57.780 INFO:teuthology.orchestra.run.smithi043.stdout: "6c155ea2-966a-4d52-8066-f0303eb75e9e": { 2024-04-25T09:01:57.780 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/scrub_control_test_path", 2024-04-25T09:01:57.780 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "6c155ea2-966a-4d52-8066-f0303eb75e9e", 2024-04-25T09:01:57.780 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive" 2024-04-25T09:01:57.780 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:01:57.780 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:01:57.780 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:01:57.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.785+0000 7f53eefce640 1 -- 172.21.15.43:0/1944101122 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f53d406b4d0 msgr2=0x7f53d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:57.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.785+0000 7f53eefce640 1 --2- 172.21.15.43:0/1944101122 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f53d406b4d0 0x7f53d406d990 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f53d8009b40 tx=0x7f53d8009190 comp rx=0 tx=0).stop 2024-04-25T09:01:57.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.785+0000 7f53eefce640 1 -- 172.21.15.43:0/1944101122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53e815dec0 msgr2=0x7f53e80b34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:57.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.785+0000 7f53eefce640 1 --2- 172.21.15.43:0/1944101122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53e815dec0 0x7f53e80b34c0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f53e002ac70 tx=0x7f53e0004a40 comp rx=0 tx=0).stop 2024-04-25T09:01:57.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.785+0000 7f53eefce640 1 -- 172.21.15.43:0/1944101122 shutdown_connections 2024-04-25T09:01:57.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.785+0000 7f53eefce640 1 --2- 172.21.15.43:0/1944101122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f53e815dec0 0x7f53e80b34c0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.785+0000 7f53eefce640 1 --2- 172.21.15.43:0/1944101122 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f53d406b4d0 0x7f53d406d990 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.785+0000 7f53eefce640 1 --2- 172.21.15.43:0/1944101122 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f53e80d04a0 0x7f53e80b2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.785+0000 7f53eefce640 1 --2- 172.21.15.43:0/1944101122 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f53e80ce8c0 0x7f53e80a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.785+0000 7f53eefce640 1 -- 172.21.15.43:0/1944101122 >> 172.21.15.43:0/1944101122 conn(0x7f53e809cd40 msgr2=0x7f53e80caba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:57.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.785+0000 7f53eefce640 1 -- 172.21.15.43:0/1944101122 shutdown_connections 2024-04-25T09:01:57.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.785+0000 7f53eefce640 1 -- 172.21.15.43:0/1944101122 wait complete. 2024-04-25T09:01:57.784 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:01:57.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.895+0000 7fae93acb640 1 Processor -- start 2024-04-25T09:01:57.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.896+0000 7fae93acb640 1 -- start start 2024-04-25T09:01:57.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.896+0000 7fae93acb640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fae8c15f180 0x7fae8c0cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.896+0000 7fae93acb640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae8c160700 con 0x7fae8c0b2cc0 2024-04-25T09:01:57.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.896+0000 7fae93acb640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae8c0ccf10 con 0x7fae8c0b23f0 2024-04-25T09:01:57.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.896+0000 7fae93acb640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae8c0cccc0 con 0x7fae8c15f180 2024-04-25T09:01:57.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.897+0000 7fae91867640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fae8c15f180 0x7fae8c0cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.897+0000 7fae92068640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fae8c0b2cc0 0x7fae8c15fff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56788/0 (socket says 172.21.15.43:56788) 2024-04-25T09:01:57.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.897+0000 7fae92068640 1 -- 172.21.15.43:0/1768676224 learned_addr learned my addr 172.21.15.43:0/1768676224 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:57.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.897+0000 7fae91867640 1 -- 172.21.15.43:0/1768676224 >> v1:172.21.15.193:6789/0 conn(0x7fae8c0b23f0 legacy=0x7fae8c15f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:57.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.897+0000 7fae91867640 1 -- 172.21.15.43:0/1768676224 >> v1:172.21.15.43:6789/0 conn(0x7fae8c0b2cc0 legacy=0x7fae8c15fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:57.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.897+0000 7fae91867640 1 -- 172.21.15.43:0/1768676224 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae8c0cd1f0 con 0x7fae8c15f180 2024-04-25T09:01:57.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.898+0000 7fae91867640 1 --2- 172.21.15.43:0/1768676224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae8c15f180 0x7fae8c0cf860 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fae880097a0 tx=0x7fae8802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46ea675b72a7024a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:57.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.898+0000 7fae90865640 1 -- 172.21.15.43:0/1768676224 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae88037070 con 0x7fae8c15f180 2024-04-25T09:01:57.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.898+0000 7fae90865640 1 -- 172.21.15.43:0/1768676224 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae88032440 con 0x7fae8c15f180 2024-04-25T09:01:57.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.898+0000 7fae90865640 1 -- 172.21.15.43:0/1768676224 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae88032810 con 0x7fae8c15f180 2024-04-25T09:01:57.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.898+0000 7fae93acb640 1 -- 172.21.15.43:0/1768676224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae8c15f180 msgr2=0x7fae8c0cf860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:57.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.898+0000 7fae93acb640 1 --2- 172.21.15.43:0/1768676224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae8c15f180 0x7fae8c0cf860 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7fae880097a0 tx=0x7fae8802a1a0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.898+0000 7fae93acb640 1 -- 172.21.15.43:0/1768676224 shutdown_connections 2024-04-25T09:01:57.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.898+0000 7fae93acb640 1 --2- 172.21.15.43:0/1768676224 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae8c15f180 0x7fae8c0cf860 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.898+0000 7fae93acb640 1 -- 172.21.15.43:0/1768676224 >> 172.21.15.43:0/1768676224 conn(0x7fae8c0d17f0 msgr2=0x7fae8c0d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:57.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.898+0000 7fae93acb640 1 -- 172.21.15.43:0/1768676224 shutdown_connections 2024-04-25T09:01:57.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.898+0000 7fae93acb640 1 -- 172.21.15.43:0/1768676224 wait complete. 2024-04-25T09:01:57.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.900+0000 7fae93acb640 1 Processor -- start 2024-04-25T09:01:57.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.900+0000 7fae93acb640 1 -- start start 2024-04-25T09:01:57.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.900+0000 7fae93acb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae8c0b23f0 0x7fae8c14ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.900+0000 7fae93acb640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fae8c0b2cc0 0x7fae8c0c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.900+0000 7fae93acb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae8c15f180 0x7fae8c0cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.900+0000 7fae93acb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8c15f560 con 0x7fae8c0b23f0 2024-04-25T09:01:57.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.900+0000 7fae93acb640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8c0cd840 con 0x7fae8c15f180 2024-04-25T09:01:57.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.900+0000 7fae93acb640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae8c14b160 con 0x7fae8c0b2cc0 2024-04-25T09:01:57.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.900+0000 7fae91867640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae8c0b23f0 0x7fae8c14ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.900+0000 7fae91867640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae8c0b23f0 0x7fae8c14ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57906/0 (socket says 172.21.15.43:57906) 2024-04-25T09:01:57.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.900+0000 7fae91867640 1 -- 172.21.15.43:0/3399475459 learned_addr learned my addr 172.21.15.43:0/3399475459 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:57.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.901+0000 7fae92068640 1 --2- 172.21.15.43:0/3399475459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae8c15f180 0x7fae8c0cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.901+0000 7fae91066640 1 --2- 172.21.15.43:0/3399475459 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fae8c0b2cc0 0x7fae8c0c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.901+0000 7fae91867640 1 -- 172.21.15.43:0/3399475459 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fae8c0b2cc0 msgr2=0x7fae8c0c88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:57.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.901+0000 7fae91867640 1 --2- 172.21.15.43:0/3399475459 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fae8c0b2cc0 0x7fae8c0c88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.901+0000 7fae91867640 1 -- 172.21.15.43:0/3399475459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae8c15f180 msgr2=0x7fae8c0cafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:57.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.901+0000 7fae91867640 1 --2- 172.21.15.43:0/3399475459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae8c15f180 0x7fae8c0cafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:57.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.901+0000 7fae91867640 1 -- 172.21.15.43:0/3399475459 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae8c0ca3b0 con 0x7fae8c0b23f0 2024-04-25T09:01:57.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.901+0000 7fae91066640 1 --2- 172.21.15.43:0/3399475459 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fae8c0b2cc0 0x7fae8c0c88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:01:57.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.901+0000 7fae91867640 1 --2- 172.21.15.43:0/3399475459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae8c0b23f0 0x7fae8c14ac20 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fae8802a1e0 tx=0x7fae880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:57.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.901+0000 7fae827fc640 1 -- 172.21.15.43:0/3399475459 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae88037040 con 0x7fae8c0b23f0 2024-04-25T09:01:57.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.901+0000 7fae827fc640 1 -- 172.21.15.43:0/3399475459 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae88042070 con 0x7fae8c0b23f0 2024-04-25T09:01:57.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.901+0000 7fae827fc640 1 -- 172.21.15.43:0/3399475459 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae88030bb0 con 0x7fae8c0b23f0 2024-04-25T09:01:57.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.901+0000 7fae93acb640 1 -- 172.21.15.43:0/3399475459 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae8c0c9310 con 0x7fae8c0b23f0 2024-04-25T09:01:57.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.901+0000 7fae93acb640 1 -- 172.21.15.43:0/3399475459 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae8c0c98d0 con 0x7fae8c0b23f0 2024-04-25T09:01:57.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.902+0000 7fae93acb640 1 -- 172.21.15.43:0/3399475459 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae8c074ef0 con 0x7fae8c0b23f0 2024-04-25T09:01:57.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.903+0000 7fae827fc640 1 -- 172.21.15.43:0/3399475459 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae88047460 con 0x7fae8c0b23f0 2024-04-25T09:01:57.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.903+0000 7fae827fc640 1 --2- 172.21.15.43:0/3399475459 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fae7006b500 0x7fae7006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:57.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.903+0000 7fae827fc640 1 -- 172.21.15.43:0/3399475459 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae880aad90 con 0x7fae8c0b23f0 2024-04-25T09:01:57.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.903+0000 7fae91066640 1 --2- 172.21.15.43:0/3399475459 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fae7006b500 0x7fae7006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:57.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.904+0000 7fae91066640 1 --2- 172.21.15.43:0/3399475459 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fae7006b500 0x7fae7006d9c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fae8c15e850 tx=0x7fae7c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:57.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:57.908+0000 7fae827fc640 1 -- 172.21.15.43:0/3399475459 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fae88079c10 con 0x7fae8c0b23f0 2024-04-25T09:01:58.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.130+0000 7fae93acb640 1 -- 172.21.15.43:0/3399475459 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae8c0cf050 con 0x7fae8c0b23f0 2024-04-25T09:01:58.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.130+0000 7fae827fc640 1 -- 172.21.15.43:0/3399475459 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4940 (secure 0 0 0) 0x7fae88079510 con 0x7fae8c0b23f0 2024-04-25T09:01:58.119 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:01:58.119 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":61,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2150177203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2150177203},{"type":"v1","addr":"172.21.15.193:6837","nonce":2150177203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":8079,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/621488642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":621488642},{"type":"v1","addr":"172.21.15.43:6834","nonce":621488642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8082,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1524395840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1524395840},{"type":"v1","addr":"172.21.15.43:6835","nonce":1524395840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:01:33.726790+0000","modified":"2024-04-25T09:01:35.706689+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8094},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8094":{"gid":8094,"name":"b","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.193:6836/853943267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":853943267},{"type":"v1","addr":"172.21.15.193:6836","nonce":853943267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:01:58.119 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 61 2024-04-25T09:01:58.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.132+0000 7fae93acb640 1 -- 172.21.15.43:0/3399475459 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fae7006b500 msgr2=0x7fae7006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:58.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.132+0000 7fae93acb640 1 --2- 172.21.15.43:0/3399475459 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fae7006b500 0x7fae7006d9c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fae8c15e850 tx=0x7fae7c009040 comp rx=0 tx=0).stop 2024-04-25T09:01:58.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.132+0000 7fae93acb640 1 -- 172.21.15.43:0/3399475459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae8c0b23f0 msgr2=0x7fae8c14ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:58.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.132+0000 7fae93acb640 1 --2- 172.21.15.43:0/3399475459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae8c0b23f0 0x7fae8c14ac20 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fae8802a1e0 tx=0x7fae880046c0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.132+0000 7fae93acb640 1 -- 172.21.15.43:0/3399475459 shutdown_connections 2024-04-25T09:01:58.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.132+0000 7fae93acb640 1 --2- 172.21.15.43:0/3399475459 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fae7006b500 0x7fae7006d9c0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.132+0000 7fae93acb640 1 --2- 172.21.15.43:0/3399475459 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae8c15f180 0x7fae8c0cafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.132+0000 7fae93acb640 1 --2- 172.21.15.43:0/3399475459 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fae8c0b2cc0 0x7fae8c0c88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.132+0000 7fae93acb640 1 --2- 172.21.15.43:0/3399475459 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae8c0b23f0 0x7fae8c14ac20 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.132+0000 7fae93acb640 1 -- 172.21.15.43:0/3399475459 >> 172.21.15.43:0/3399475459 conn(0x7fae8c0d17f0 msgr2=0x7fae8c0cfc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:58.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.132+0000 7fae93acb640 1 -- 172.21.15.43:0/3399475459 shutdown_connections 2024-04-25T09:01:58.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.132+0000 7fae93acb640 1 -- 172.21.15.43:0/3399475459 wait complete. 2024-04-25T09:01:58.132 DEBUG:tasks.cephfs.test_scrub_checks:current active=['b'] 2024-04-25T09:01:58.132 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:01:58.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.243+0000 7f836ab09640 1 Processor -- start 2024-04-25T09:01:58.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.243+0000 7f836ab09640 1 -- start start 2024-04-25T09:01:58.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.244+0000 7f836ab09640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f836415ded0 0x7f83640ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:58.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.244+0000 7f836ab09640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f83640cf250 0x7f83640ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:58.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.244+0000 7f836ab09640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f836406ab30 0x7f836406afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:58.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.244+0000 7f836ab09640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f836415e2b0 con 0x7f836415ded0 2024-04-25T09:01:58.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.244+0000 7f836ab09640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83640cf6e0 con 0x7f836406ab30 2024-04-25T09:01:58.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.244+0000 7f836ab09640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83640cf9c0 con 0x7f83640cf250 2024-04-25T09:01:58.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.244+0000 7f83688a5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f836415ded0 0x7f83640ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:58.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.244+0000 7f83688a5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f836415ded0 0x7f83640ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57916/0 (socket says 172.21.15.43:57916) 2024-04-25T09:01:58.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.244+0000 7f83688a5640 1 -- 172.21.15.43:0/3219538918 learned_addr learned my addr 172.21.15.43:0/3219538918 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:58.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.244+0000 7f83690a6640 1 --2- 172.21.15.43:0/3219538918 >> v2:172.21.15.193:3300/0 conn(0x7f836406ab30 0x7f836406afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:58.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.244+0000 7f8363fff640 1 --2- 172.21.15.43:0/3219538918 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f83640cf250 0x7f83640ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:58.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.245+0000 7f83688a5640 1 -- 172.21.15.43:0/3219538918 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f83640cf250 msgr2=0x7f83640ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:58.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.245+0000 7f83688a5640 1 --2- 172.21.15.43:0/3219538918 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f83640cf250 0x7f83640ad8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.245+0000 7f83688a5640 1 -- 172.21.15.43:0/3219538918 >> v2:172.21.15.193:3300/0 conn(0x7f836406ab30 msgr2=0x7f836406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:58.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.245+0000 7f83688a5640 1 --2- 172.21.15.43:0/3219538918 >> v2:172.21.15.193:3300/0 conn(0x7f836406ab30 0x7f836406afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.245+0000 7f83688a5640 1 -- 172.21.15.43:0/3219538918 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83640cfca0 con 0x7f836415ded0 2024-04-25T09:01:58.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.245+0000 7f83688a5640 1 --2- 172.21.15.43:0/3219538918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f836415ded0 0x7f83640ce740 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f836406b6f0 tx=0x7f835c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cfe61586d59c2a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:58.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.245+0000 7f83637fe640 1 -- 172.21.15.43:0/3219538918 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835c036070 con 0x7f836415ded0 2024-04-25T09:01:58.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.245+0000 7f83637fe640 1 -- 172.21.15.43:0/3219538918 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f835c031600 con 0x7f836415ded0 2024-04-25T09:01:58.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.245+0000 7f83637fe640 1 -- 172.21.15.43:0/3219538918 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835c0319d0 con 0x7f836415ded0 2024-04-25T09:01:58.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.246+0000 7f836ab09640 1 -- 172.21.15.43:0/3219538918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f836415ded0 msgr2=0x7f83640ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:58.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.246+0000 7f836ab09640 1 --2- 172.21.15.43:0/3219538918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f836415ded0 0x7f83640ce740 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f836406b6f0 tx=0x7f835c02a6a0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.246+0000 7f836ab09640 1 -- 172.21.15.43:0/3219538918 shutdown_connections 2024-04-25T09:01:58.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.246+0000 7f836ab09640 1 --2- 172.21.15.43:0/3219538918 >> v2:172.21.15.193:3300/0 conn(0x7f836406ab30 0x7f836406afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.246+0000 7f836ab09640 1 --2- 172.21.15.43:0/3219538918 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f83640cf250 0x7f83640ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.246+0000 7f836ab09640 1 --2- 172.21.15.43:0/3219538918 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f836415ded0 0x7f83640ce740 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.246+0000 7f836ab09640 1 -- 172.21.15.43:0/3219538918 >> 172.21.15.43:0/3219538918 conn(0x7f83640b2490 msgr2=0x7f83640b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:58.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.246+0000 7f836ab09640 1 -- 172.21.15.43:0/3219538918 shutdown_connections 2024-04-25T09:01:58.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.246+0000 7f836ab09640 1 -- 172.21.15.43:0/3219538918 wait complete. 2024-04-25T09:01:58.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.247+0000 7f836ab09640 1 Processor -- start 2024-04-25T09:01:58.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.247+0000 7f836ab09640 1 -- start start 2024-04-25T09:01:58.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.248+0000 7f836ab09640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f836406ab30 0x7f836414c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:58.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.248+0000 7f836ab09640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f836415ded0 0x7f836415c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:58.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.248+0000 7f836ab09640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8364158d60 0x7f836415c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:58.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.248+0000 7f836ab09640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83640cefd0 con 0x7f836415ded0 2024-04-25T09:01:58.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.248+0000 7f836ab09640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83640cee50 con 0x7f8364158d60 2024-04-25T09:01:58.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.248+0000 7f836ab09640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83640b4db0 con 0x7f836406ab30 2024-04-25T09:01:58.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.248+0000 7f8363fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f836415ded0 0x7f836415c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:58.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.248+0000 7f8363fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f836415ded0 0x7f836415c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:57920/0 (socket says 172.21.15.43:57920) 2024-04-25T09:01:58.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.248+0000 7f8363fff640 1 -- 172.21.15.43:0/3771307787 learned_addr learned my addr 172.21.15.43:0/3771307787 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:58.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.248+0000 7f83690a6640 1 --2- 172.21.15.43:0/3771307787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8364158d60 0x7f836415c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:58.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.248+0000 7f83688a5640 1 --2- 172.21.15.43:0/3771307787 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f836406ab30 0x7f836414c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:58.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.249+0000 7f8363fff640 1 -- 172.21.15.43:0/3771307787 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f836406ab30 msgr2=0x7f836414c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:58.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.249+0000 7f8363fff640 1 --2- 172.21.15.43:0/3771307787 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f836406ab30 0x7f836414c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.249+0000 7f8363fff640 1 -- 172.21.15.43:0/3771307787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8364158d60 msgr2=0x7f836415c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:58.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.249+0000 7f8363fff640 1 --2- 172.21.15.43:0/3771307787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8364158d60 0x7f836415c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.249+0000 7f8363fff640 1 -- 172.21.15.43:0/3771307787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8364157cc0 con 0x7f836415ded0 2024-04-25T09:01:58.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.249+0000 7f8363fff640 1 --2- 172.21.15.43:0/3771307787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f836415ded0 0x7f836415c110 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f835800e9b0 tx=0x7f835800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:58.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.249+0000 7f83617fa640 1 -- 172.21.15.43:0/3771307787 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8358017070 con 0x7f836415ded0 2024-04-25T09:01:58.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.249+0000 7f83617fa640 1 -- 172.21.15.43:0/3771307787 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f835800c9a0 con 0x7f836415ded0 2024-04-25T09:01:58.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.249+0000 7f836ab09640 1 -- 172.21.15.43:0/3771307787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8364157fa0 con 0x7f836415ded0 2024-04-25T09:01:58.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.249+0000 7f83617fa640 1 -- 172.21.15.43:0/3771307787 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835800cd90 con 0x7f836415ded0 2024-04-25T09:01:58.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.249+0000 7f836ab09640 1 -- 172.21.15.43:0/3771307787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8364158560 con 0x7f836415ded0 2024-04-25T09:01:58.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.250+0000 7f836ab09640 1 -- 172.21.15.43:0/3771307787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8364074ef0 con 0x7f836415ded0 2024-04-25T09:01:58.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.251+0000 7f83617fa640 1 -- 172.21.15.43:0/3771307787 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f835801a070 con 0x7f836415ded0 2024-04-25T09:01:58.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.251+0000 7f83617fa640 1 --2- 172.21.15.43:0/3771307787 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f835006b4d0 0x7f835006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:58.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.251+0000 7f83617fa640 1 -- 172.21.15.43:0/3771307787 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f835808a550 con 0x7f836415ded0 2024-04-25T09:01:58.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.251+0000 7f83688a5640 1 --2- 172.21.15.43:0/3771307787 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f835006b4d0 0x7f835006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:58.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.252+0000 7f83688a5640 1 --2- 172.21.15.43:0/3771307787 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f835006b4d0 0x7f835006d990 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f835c033000 tx=0x7f835c009000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:58.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.255+0000 7f83617fa640 1 -- 172.21.15.43:0/3771307787 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8358058cf0 con 0x7f836415ded0 2024-04-25T09:01:58.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.507+0000 7f836ab09640 1 -- 172.21.15.43:0/3771307787 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f83640d1540 con 0x7f836415ded0 2024-04-25T09:01:58.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.507+0000 7f83617fa640 1 -- 172.21.15.43:0/3771307787 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1573 (secure 0 0 0) 0x7f835805cd90 con 0x7f836415ded0 2024-04-25T09:01:58.497 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:01:58.497 INFO:teuthology.orchestra.run.smithi043.stdout:{"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":490,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":102,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035238,"num_in_osds":8,"osd_in_since":1714035229,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":754,"data_bytes":766794580,"bytes_used":1964752896,"bytes_avail":771129360384,"bytes_total":773094113280,"write_bytes_sec":63842612,"read_op_per_sec":0,"write_op_per_sec":62},"fsmap":{"epoch":61,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"b","status":"up:active","gid":8094}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":17,"modified":"2024-04-25T09:01:15.791620+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:01:58.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.509+0000 7f836ab09640 1 -- 172.21.15.43:0/3771307787 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f835006b4d0 msgr2=0x7f835006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:58.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.509+0000 7f836ab09640 1 --2- 172.21.15.43:0/3771307787 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f835006b4d0 0x7f835006d990 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f835c033000 tx=0x7f835c009000 comp rx=0 tx=0).stop 2024-04-25T09:01:58.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.509+0000 7f836ab09640 1 -- 172.21.15.43:0/3771307787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f836415ded0 msgr2=0x7f836415c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:58.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.509+0000 7f836ab09640 1 --2- 172.21.15.43:0/3771307787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f836415ded0 0x7f836415c110 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f835800e9b0 tx=0x7f835800ee70 comp rx=0 tx=0).stop 2024-04-25T09:01:58.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.509+0000 7f836ab09640 1 -- 172.21.15.43:0/3771307787 shutdown_connections 2024-04-25T09:01:58.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.509+0000 7f836ab09640 1 --2- 172.21.15.43:0/3771307787 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f835006b4d0 0x7f835006d990 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.509+0000 7f836ab09640 1 --2- 172.21.15.43:0/3771307787 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8364158d60 0x7f836415c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.509+0000 7f836ab09640 1 --2- 172.21.15.43:0/3771307787 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f836415ded0 0x7f836415c110 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.509+0000 7f836ab09640 1 --2- 172.21.15.43:0/3771307787 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f836406ab30 0x7f836414c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:58.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.509+0000 7f836ab09640 1 -- 172.21.15.43:0/3771307787 >> 172.21.15.43:0/3771307787 conn(0x7f83640b2490 msgr2=0x7f83640b11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:58.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.509+0000 7f836ab09640 1 -- 172.21.15.43:0/3771307787 shutdown_connections 2024-04-25T09:01:58.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:58.509+0000 7f836ab09640 1 -- 172.21.15.43:0/3771307787 wait complete. 2024-04-25T09:01:58.509 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T09:01:59.510 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:01:59.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.626+0000 7f91d6a46640 1 Processor -- start 2024-04-25T09:01:59.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.626+0000 7f91d6a46640 1 -- start start 2024-04-25T09:01:59.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.626+0000 7f91d6a46640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f91d015def0 0x7f91d006be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:59.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.626+0000 7f91d6a46640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f91d01619f0 0x7f91d0161e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:59.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.626+0000 7f91d6a46640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91d0162820 con 0x7f91d0162390 2024-04-25T09:01:59.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.626+0000 7f91d6a46640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91d00b30d0 con 0x7f91d01619f0 2024-04-25T09:01:59.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.626+0000 7f91d6a46640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d00d0e90 con 0x7f91d015def0 2024-04-25T09:01:59.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.627+0000 7f91cf7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f91d01619f0 0x7f91d0161e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:59.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.627+0000 7f91d4fe3640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f91d0162390 0x7f91d00cfa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48714/0 (socket says 172.21.15.43:48714) 2024-04-25T09:01:59.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.627+0000 7f91d4fe3640 1 -- 172.21.15.43:0/3642121601 learned_addr learned my addr 172.21.15.43:0/3642121601 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:01:59.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.627+0000 7f91cffff640 1 --2- 172.21.15.43:0/3642121601 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f91d015def0 0x7f91d006be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:59.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.627+0000 7f91cf7fe640 1 -- 172.21.15.43:0/3642121601 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f91d015def0 msgr2=0x7f91d006be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:59.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.627+0000 7f91cf7fe640 1 --2- 172.21.15.43:0/3642121601 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f91d015def0 0x7f91d006be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:59.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.627+0000 7f91cf7fe640 1 -- 172.21.15.43:0/3642121601 >> v1:172.21.15.43:6789/0 conn(0x7f91d0162390 legacy=0x7f91d00cfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:59.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.627+0000 7f91cf7fe640 1 -- 172.21.15.43:0/3642121601 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d015e2d0 con 0x7f91d01619f0 2024-04-25T09:01:59.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.627+0000 7f91cffff640 1 --2- 172.21.15.43:0/3642121601 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f91d015def0 0x7f91d006be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:59.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.628+0000 7f91cf7fe640 1 --2- 172.21.15.43:0/3642121601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91d01619f0 0x7f91d0161e50 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f91c40097c0 tx=0x7f91c402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff8a9a3a4b005cab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:59.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.628+0000 7f91ceffd640 1 -- 172.21.15.43:0/3642121601 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c4037070 con 0x7f91d01619f0 2024-04-25T09:01:59.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.628+0000 7f91ceffd640 1 -- 172.21.15.43:0/3642121601 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c4032440 con 0x7f91d01619f0 2024-04-25T09:01:59.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.628+0000 7f91ceffd640 1 -- 172.21.15.43:0/3642121601 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c4032810 con 0x7f91d01619f0 2024-04-25T09:01:59.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.628+0000 7f91d6a46640 1 -- 172.21.15.43:0/3642121601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91d01619f0 msgr2=0x7f91d0161e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:59.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.628+0000 7f91d6a46640 1 --2- 172.21.15.43:0/3642121601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91d01619f0 0x7f91d0161e50 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f91c40097c0 tx=0x7f91c402a1a0 comp rx=0 tx=0).stop 2024-04-25T09:01:59.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.628+0000 7f91d6a46640 1 -- 172.21.15.43:0/3642121601 shutdown_connections 2024-04-25T09:01:59.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.628+0000 7f91d6a46640 1 --2- 172.21.15.43:0/3642121601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91d01619f0 0x7f91d0161e50 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:59.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.628+0000 7f91d6a46640 1 --2- 172.21.15.43:0/3642121601 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f91d015def0 0x7f91d006be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:59.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.628+0000 7f91d6a46640 1 -- 172.21.15.43:0/3642121601 >> 172.21.15.43:0/3642121601 conn(0x7f91d00b32e0 msgr2=0x7f91d00b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:59.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.628+0000 7f91d6a46640 1 -- 172.21.15.43:0/3642121601 shutdown_connections 2024-04-25T09:01:59.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.628+0000 7f91d6a46640 1 -- 172.21.15.43:0/3642121601 wait complete. 2024-04-25T09:01:59.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.629+0000 7f91d6a46640 1 Processor -- start 2024-04-25T09:01:59.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.630+0000 7f91d6a46640 1 -- start start 2024-04-25T09:01:59.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.630+0000 7f91d6a46640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f91d015def0 0x7f91d00b5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:59.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.630+0000 7f91d6a46640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91d01619f0 0x7f91d014ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:59.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.630+0000 7f91d6a46640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91d0162390 0x7f91d00cfa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:59.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.630+0000 7f91d6a46640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d00b1300 con 0x7f91d01619f0 2024-04-25T09:01:59.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.630+0000 7f91d6a46640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d006c390 con 0x7f91d0162390 2024-04-25T09:01:59.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.630+0000 7f91d6a46640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d00b15e0 con 0x7f91d015def0 2024-04-25T09:01:59.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.630+0000 7f91cf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91d01619f0 0x7f91d014ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:59.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.630+0000 7f91cf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91d01619f0 0x7f91d014ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43060/0 (socket says 172.21.15.43:43060) 2024-04-25T09:01:59.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.630+0000 7f91cf7fe640 1 -- 172.21.15.43:0/1042467393 learned_addr learned my addr 172.21.15.43:0/1042467393 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:59.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.631+0000 7f91cffff640 1 --2- 172.21.15.43:0/1042467393 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f91d015def0 0x7f91d00b5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:59.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.631+0000 7f91d4fe3640 1 --2- 172.21.15.43:0/1042467393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91d0162390 0x7f91d00cfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:59.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.631+0000 7f91cf7fe640 1 -- 172.21.15.43:0/1042467393 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f91d015def0 msgr2=0x7f91d00b5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:59.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.631+0000 7f91cf7fe640 1 --2- 172.21.15.43:0/1042467393 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f91d015def0 0x7f91d00b5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:59.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.631+0000 7f91cf7fe640 1 -- 172.21.15.43:0/1042467393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91d0162390 msgr2=0x7f91d00cfa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:59.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.631+0000 7f91cf7fe640 1 --2- 172.21.15.43:0/1042467393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91d0162390 0x7f91d00cfa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:59.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.631+0000 7f91cf7fe640 1 -- 172.21.15.43:0/1042467393 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d014d360 con 0x7f91d01619f0 2024-04-25T09:01:59.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.631+0000 7f91cffff640 1 --2- 172.21.15.43:0/1042467393 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f91d015def0 0x7f91d00b5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:01:59.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.631+0000 7f91cf7fe640 1 --2- 172.21.15.43:0/1042467393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91d01619f0 0x7f91d014ce20 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f91c40096e0 tx=0x7f91c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:59.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.631+0000 7f91ccff9640 1 -- 172.21.15.43:0/1042467393 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c4037040 con 0x7f91d01619f0 2024-04-25T09:01:59.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.631+0000 7f91ccff9640 1 -- 172.21.15.43:0/1042467393 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c40307b0 con 0x7f91d01619f0 2024-04-25T09:01:59.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.631+0000 7f91ccff9640 1 -- 172.21.15.43:0/1042467393 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c4030b80 con 0x7f91d01619f0 2024-04-25T09:01:59.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.631+0000 7f91d6a46640 1 -- 172.21.15.43:0/1042467393 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d00b6270 con 0x7f91d01619f0 2024-04-25T09:01:59.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.631+0000 7f91d6a46640 1 -- 172.21.15.43:0/1042467393 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d00b6830 con 0x7f91d01619f0 2024-04-25T09:01:59.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.632+0000 7f91d6a46640 1 -- 172.21.15.43:0/1042467393 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91d0074ef0 con 0x7f91d01619f0 2024-04-25T09:01:59.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.633+0000 7f91ccff9640 1 -- 172.21.15.43:0/1042467393 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f91c4044020 con 0x7f91d01619f0 2024-04-25T09:01:59.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.633+0000 7f91ccff9640 1 --2- 172.21.15.43:0/1042467393 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f91bc06b4d0 0x7f91bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:59.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.633+0000 7f91ccff9640 1 -- 172.21.15.43:0/1042467393 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f91c40aa4a0 con 0x7f91d01619f0 2024-04-25T09:01:59.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.633+0000 7f91cffff640 1 --2- 172.21.15.43:0/1042467393 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f91bc06b4d0 0x7f91bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:59.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.634+0000 7f91cffff640 1 --2- 172.21.15.43:0/1042467393 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f91bc06b4d0 0x7f91bc06d990 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f91c0006fd0 tx=0x7f91c0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:59.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.638+0000 7f91ccff9640 1 -- 172.21.15.43:0/1042467393 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f91c404b020 con 0x7f91d01619f0 2024-04-25T09:01:59.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.855+0000 7f91d6a46640 1 -- 172.21.15.43:0/1042467393 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f91d00b7710 con 0x7f91d01619f0 2024-04-25T09:01:59.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.856+0000 7f91ccff9640 1 -- 172.21.15.43:0/1042467393 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f91d014d360 con 0x7f91d01619f0 2024-04-25T09:01:59.845 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:01:59.845 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":61,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2150177203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2150177203},{"type":"v1","addr":"172.21.15.193:6837","nonce":2150177203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":8079,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/621488642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":621488642},{"type":"v1","addr":"172.21.15.43:6834","nonce":621488642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8082,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1524395840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1524395840},{"type":"v1","addr":"172.21.15.43:6835","nonce":1524395840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:01:33.726790+0000","modified":"2024-04-25T09:01:35.706689+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8094},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8094":{"gid":8094,"name":"b","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.193:6836/853943267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":853943267},{"type":"v1","addr":"172.21.15.193:6836","nonce":853943267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:01:59.845 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 61 2024-04-25T09:01:59.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.858+0000 7f91d6a46640 1 -- 172.21.15.43:0/1042467393 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f91bc06b4d0 msgr2=0x7f91bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:59.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.858+0000 7f91d6a46640 1 --2- 172.21.15.43:0/1042467393 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f91bc06b4d0 0x7f91bc06d990 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f91c0006fd0 tx=0x7f91c0008040 comp rx=0 tx=0).stop 2024-04-25T09:01:59.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.858+0000 7f91d6a46640 1 -- 172.21.15.43:0/1042467393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91d01619f0 msgr2=0x7f91d014ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:59.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.858+0000 7f91d6a46640 1 --2- 172.21.15.43:0/1042467393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91d01619f0 0x7f91d014ce20 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f91c40096e0 tx=0x7f91c40046c0 comp rx=0 tx=0).stop 2024-04-25T09:01:59.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.858+0000 7f91d6a46640 1 -- 172.21.15.43:0/1042467393 shutdown_connections 2024-04-25T09:01:59.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.858+0000 7f91d6a46640 1 --2- 172.21.15.43:0/1042467393 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f91bc06b4d0 0x7f91bc06d990 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:59.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.858+0000 7f91d6a46640 1 --2- 172.21.15.43:0/1042467393 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f91d0162390 0x7f91d00cfa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:59.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.858+0000 7f91d6a46640 1 --2- 172.21.15.43:0/1042467393 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91d01619f0 0x7f91d014ce20 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:59.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.858+0000 7f91d6a46640 1 --2- 172.21.15.43:0/1042467393 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f91d015def0 0x7f91d00b5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:59.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.858+0000 7f91d6a46640 1 -- 172.21.15.43:0/1042467393 >> 172.21.15.43:0/1042467393 conn(0x7f91d00b32e0 msgr2=0x7f91d015b550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:59.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.858+0000 7f91d6a46640 1 -- 172.21.15.43:0/1042467393 shutdown_connections 2024-04-25T09:01:59.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.858+0000 7f91d6a46640 1 -- 172.21.15.43:0/1042467393 wait complete. 2024-04-25T09:01:59.857 DEBUG:tasks.cephfs.test_scrub_checks:current active=['b'] 2024-04-25T09:01:59.858 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:01:59.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.969+0000 7f771b688640 1 Processor -- start 2024-04-25T09:01:59.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.969+0000 7f771b688640 1 -- start start 2024-04-25T09:01:59.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.970+0000 7f771b688640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f771415dce0 0x7f771415ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:59.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.970+0000 7f771b688640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77140ce550 0x7f77140d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:59.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.970+0000 7f771b688640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77140cee80 con 0x7f77140ce990 2024-04-25T09:01:59.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.970+0000 7f771b688640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77140d2920 con 0x7f771415dce0 2024-04-25T09:01:59.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.970+0000 7f771b688640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77140cf160 con 0x7f77140ce550 2024-04-25T09:01:59.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.970+0000 7f7719424640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f771415dce0 0x7f771415ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:59.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.970+0000 7f7718c23640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77140ce550 0x7f77140d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:59.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.970+0000 7f7719424640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f771415dce0 0x7f771415ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:40350/0 (socket says 172.21.15.43:40350) 2024-04-25T09:01:59.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.970+0000 7f7718c23640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77140ce550 0x7f77140d03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:53876/0 (socket says 172.21.15.43:53876) 2024-04-25T09:01:59.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.970+0000 7f7719424640 1 -- 172.21.15.43:0/3214041859 learned_addr learned my addr 172.21.15.43:0/3214041859 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:59.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.971+0000 7f7719424640 1 -- 172.21.15.43:0/3214041859 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77140ce550 msgr2=0x7f77140d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:01:59.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.971+0000 7f7719424640 1 --2- 172.21.15.43:0/3214041859 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77140ce550 0x7f77140d03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:59.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.971+0000 7f7719424640 1 -- 172.21.15.43:0/3214041859 >> v1:172.21.15.193:6789/0 conn(0x7f77140ce990 legacy=0x7f7714160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:59.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.971+0000 7f770bfff640 1 -- 172.21.15.43:0/3214041859 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3584602634 0 0) 0x7f77140cee80 con 0x7f77140ce990 2024-04-25T09:01:59.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.971+0000 7f7719424640 1 -- 172.21.15.43:0/3214041859 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77140cf440 con 0x7f771415dce0 2024-04-25T09:01:59.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.971+0000 7f7719424640 1 --2- 172.21.15.43:0/3214041859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f771415dce0 0x7f771415ef50 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7710009870 tx=0x7f771002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=196cd19208b2a5ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:59.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.971+0000 7f770bfff640 1 -- 172.21.15.43:0/3214041859 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7710037070 con 0x7f771415dce0 2024-04-25T09:01:59.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.972+0000 7f770bfff640 1 -- 172.21.15.43:0/3214041859 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7710032440 con 0x7f771415dce0 2024-04-25T09:01:59.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.972+0000 7f770bfff640 1 -- 172.21.15.43:0/3214041859 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7710032810 con 0x7f771415dce0 2024-04-25T09:01:59.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.972+0000 7f771b688640 1 -- 172.21.15.43:0/3214041859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f771415dce0 msgr2=0x7f771415ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:59.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.972+0000 7f771b688640 1 --2- 172.21.15.43:0/3214041859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f771415dce0 0x7f771415ef50 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f7710009870 tx=0x7f771002a1c0 comp rx=0 tx=0).stop 2024-04-25T09:01:59.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.972+0000 7f771b688640 1 -- 172.21.15.43:0/3214041859 shutdown_connections 2024-04-25T09:01:59.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.972+0000 7f771b688640 1 --2- 172.21.15.43:0/3214041859 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77140ce550 0x7f77140d03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:59.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.972+0000 7f771b688640 1 --2- 172.21.15.43:0/3214041859 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f771415dce0 0x7f771415ef50 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:59.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.972+0000 7f771b688640 1 -- 172.21.15.43:0/3214041859 >> 172.21.15.43:0/3214041859 conn(0x7f771409cd00 msgr2=0x7f77140d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:01:59.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.972+0000 7f771b688640 1 -- 172.21.15.43:0/3214041859 shutdown_connections 2024-04-25T09:01:59.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.972+0000 7f771b688640 1 -- 172.21.15.43:0/3214041859 wait complete. 2024-04-25T09:01:59.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.973+0000 7f771b688640 1 Processor -- start 2024-04-25T09:01:59.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.973+0000 7f771b688640 1 -- start start 2024-04-25T09:01:59.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.974+0000 7f771b688640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77140ce550 0x7f77140a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:59.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.974+0000 7f771b688640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77140ce990 0x7f771415ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:59.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.974+0000 7f771b688640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f771415dce0 0x7f771415ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:59.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.974+0000 7f771b688640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771415e0c0 con 0x7f771415dce0 2024-04-25T09:01:59.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.974+0000 7f771b688640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771415f6b0 con 0x7f77140ce990 2024-04-25T09:01:59.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.974+0000 7f771b688640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77140d0920 con 0x7f77140ce550 2024-04-25T09:01:59.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.974+0000 7f7719c25640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f771415dce0 0x7f771415ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:59.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.974+0000 7f7719c25640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f771415dce0 0x7f771415ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43066/0 (socket says 172.21.15.43:43066) 2024-04-25T09:01:59.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.974+0000 7f7719c25640 1 -- 172.21.15.43:0/2731703579 learned_addr learned my addr 172.21.15.43:0/2731703579 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:01:59.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.974+0000 7f7718c23640 1 --2- 172.21.15.43:0/2731703579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77140ce990 0x7f771415ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:59.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.974+0000 7f7719424640 1 --2- 172.21.15.43:0/2731703579 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77140ce550 0x7f77140a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:59.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.975+0000 7f7719c25640 1 -- 172.21.15.43:0/2731703579 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77140ce550 msgr2=0x7f77140a01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:59.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.975+0000 7f7719c25640 1 --2- 172.21.15.43:0/2731703579 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77140ce550 0x7f77140a01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:59.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.975+0000 7f7719c25640 1 -- 172.21.15.43:0/2731703579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77140ce990 msgr2=0x7f771415ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:01:59.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.975+0000 7f7719c25640 1 --2- 172.21.15.43:0/2731703579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77140ce990 0x7f771415ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:01:59.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.975+0000 7f7719c25640 1 -- 172.21.15.43:0/2731703579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77140a06f0 con 0x7f771415dce0 2024-04-25T09:01:59.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.975+0000 7f7719424640 1 --2- 172.21.15.43:0/2731703579 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77140ce550 0x7f77140a01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:01:59.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.975+0000 7f7719c25640 1 --2- 172.21.15.43:0/2731703579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f771415dce0 0x7f771415ef40 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f770c00c760 tx=0x7f770c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:59.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.975+0000 7f7709ffb640 1 -- 172.21.15.43:0/2731703579 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c003da0 con 0x7f771415dce0 2024-04-25T09:01:59.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.975+0000 7f7709ffb640 1 -- 172.21.15.43:0/2731703579 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f770c005070 con 0x7f771415dce0 2024-04-25T09:01:59.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.975+0000 7f771b688640 1 -- 172.21.15.43:0/2731703579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f771409e820 con 0x7f771415dce0 2024-04-25T09:01:59.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.975+0000 7f7709ffb640 1 -- 172.21.15.43:0/2731703579 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c005460 con 0x7f771415dce0 2024-04-25T09:01:59.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.975+0000 7f771b688640 1 -- 172.21.15.43:0/2731703579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f771409ede0 con 0x7f771415dce0 2024-04-25T09:01:59.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.976+0000 7f771b688640 1 -- 172.21.15.43:0/2731703579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77140680f0 con 0x7f771415dce0 2024-04-25T09:01:59.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.977+0000 7f7709ffb640 1 -- 172.21.15.43:0/2731703579 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f770c005600 con 0x7f771415dce0 2024-04-25T09:01:59.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.977+0000 7f7709ffb640 1 --2- 172.21.15.43:0/2731703579 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f76f806b500 0x7f76f806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:01:59.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.977+0000 7f7709ffb640 1 -- 172.21.15.43:0/2731703579 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f770c08a010 con 0x7f771415dce0 2024-04-25T09:01:59.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.977+0000 7f7719424640 1 --2- 172.21.15.43:0/2731703579 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f76f806b500 0x7f76f806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:01:59.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.978+0000 7f7719424640 1 --2- 172.21.15.43:0/2731703579 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f76f806b500 0x7f76f806d9c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f7710034000 tx=0x7f771002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:01:59.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:01:59.981+0000 7f7709ffb640 1 -- 172.21.15.43:0/2731703579 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f770c058e90 con 0x7f771415dce0 2024-04-25T09:02:00.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:00.232+0000 7f771b688640 1 -- 172.21.15.43:0/2731703579 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f7714074ef0 con 0x7f771415dce0 2024-04-25T09:02:00.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:00.233+0000 7f7709ffb640 1 -- 172.21.15.43:0/2731703579 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1573 (secure 0 0 0) 0x7f770c058790 con 0x7f771415dce0 2024-04-25T09:02:00.221 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:00.222 INFO:teuthology.orchestra.run.smithi043.stdout:{"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":492,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":102,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035238,"num_in_osds":8,"osd_in_since":1714035229,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":754,"data_bytes":766794580,"bytes_used":1964752896,"bytes_avail":771129360384,"bytes_total":773094113280,"write_bytes_sec":63842612,"read_op_per_sec":0,"write_op_per_sec":62},"fsmap":{"epoch":61,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"b","status":"up:active","gid":8094}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":17,"modified":"2024-04-25T09:01:15.791620+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:02:00.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:00.234+0000 7f771b688640 1 -- 172.21.15.43:0/2731703579 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f76f806b500 msgr2=0x7f76f806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:00.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:00.234+0000 7f771b688640 1 --2- 172.21.15.43:0/2731703579 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f76f806b500 0x7f76f806d9c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f7710034000 tx=0x7f771002e000 comp rx=0 tx=0).stop 2024-04-25T09:02:00.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:00.234+0000 7f771b688640 1 -- 172.21.15.43:0/2731703579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f771415dce0 msgr2=0x7f771415ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:00.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:00.234+0000 7f771b688640 1 --2- 172.21.15.43:0/2731703579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f771415dce0 0x7f771415ef40 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f770c00c760 tx=0x7f770c00cc20 comp rx=0 tx=0).stop 2024-04-25T09:02:00.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:00.234+0000 7f771b688640 1 -- 172.21.15.43:0/2731703579 shutdown_connections 2024-04-25T09:02:00.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:00.235+0000 7f771b688640 1 --2- 172.21.15.43:0/2731703579 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f771415dce0 0x7f771415ef40 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:00.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:00.235+0000 7f771b688640 1 --2- 172.21.15.43:0/2731703579 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f76f806b500 0x7f76f806d9c0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:00.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:00.235+0000 7f771b688640 1 --2- 172.21.15.43:0/2731703579 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f77140ce990 0x7f771415ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:00.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:00.235+0000 7f771b688640 1 --2- 172.21.15.43:0/2731703579 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f77140ce550 0x7f77140a01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:00.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:00.235+0000 7f771b688640 1 -- 172.21.15.43:0/2731703579 >> 172.21.15.43:0/2731703579 conn(0x7f771409cd00 msgr2=0x7f77140d1910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:00.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:00.235+0000 7f771b688640 1 -- 172.21.15.43:0/2731703579 shutdown_connections 2024-04-25T09:02:00.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:00.235+0000 7f771b688640 1 -- 172.21.15.43:0/2731703579 wait complete. 2024-04-25T09:02:00.234 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T09:02:01.235 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:02:01.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.360+0000 7ff3a31ce640 1 Processor -- start 2024-04-25T09:02:01.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.361+0000 7ff3a31ce640 1 -- start start 2024-04-25T09:02:01.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.361+0000 7ff3a31ce640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff39c0b5d50 0x7ff39c0d4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:01.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.361+0000 7ff3a31ce640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff39c0b6130 0x7ff39c0d19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:01.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.361+0000 7ff3a31ce640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff39c0cf310 con 0x7ff39c0d1ee0 2024-04-25T09:02:01.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.361+0000 7ff3a31ce640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff39c0b6940 con 0x7ff39c0b5d50 2024-04-25T09:02:01.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.361+0000 7ff3a31ce640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff39c0cf5f0 con 0x7ff39c0b6130 2024-04-25T09:02:01.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.362+0000 7ff3a0f6a640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff39c0b5d50 0x7ff39c0d4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:01.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.362+0000 7ff39bfff640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff39c0b6130 0x7ff39c0d19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:01.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.362+0000 7ff3a0f6a640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff39c0b5d50 0x7ff39c0d4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:40368/0 (socket says 172.21.15.43:40368) 2024-04-25T09:02:01.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.362+0000 7ff3a0f6a640 1 -- 172.21.15.43:0/2144294896 learned_addr learned my addr 172.21.15.43:0/2144294896 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:01.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.362+0000 7ff3a176b640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7ff39c0d1ee0 0x7ff39c165fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:50726/0 (socket says 172.21.15.43:50726) 2024-04-25T09:02:01.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.362+0000 7ff3a0f6a640 1 -- 172.21.15.43:0/2144294896 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff39c0b6130 msgr2=0x7ff39c0d19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:01.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.362+0000 7ff3a0f6a640 1 --2- 172.21.15.43:0/2144294896 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff39c0b6130 0x7ff39c0d19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:01.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.362+0000 7ff3a0f6a640 1 -- 172.21.15.43:0/2144294896 >> v1:172.21.15.193:6789/0 conn(0x7ff39c0d1ee0 legacy=0x7ff39c165fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:01.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.362+0000 7ff3a0f6a640 1 -- 172.21.15.43:0/2144294896 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff39c0cf8d0 con 0x7ff39c0b5d50 2024-04-25T09:02:01.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.362+0000 7ff39b7fe640 1 -- 172.21.15.43:0/2144294896 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3330749165 0 0) 0x7ff39c0cf310 con 0x7ff39c0d1ee0 2024-04-25T09:02:01.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.362+0000 7ff39bfff640 1 --2- 172.21.15.43:0/2144294896 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff39c0b6130 0x7ff39c0d19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:02:01.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.363+0000 7ff3a0f6a640 1 --2- 172.21.15.43:0/2144294896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff39c0b5d50 0x7ff39c0d4550 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff38c009870 tx=0x7ff38c02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53171fa659e2b575 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:01.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.363+0000 7ff39b7fe640 1 -- 172.21.15.43:0/2144294896 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38c037070 con 0x7ff39c0b5d50 2024-04-25T09:02:01.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.363+0000 7ff39b7fe640 1 -- 172.21.15.43:0/2144294896 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff38c032440 con 0x7ff39c0b5d50 2024-04-25T09:02:01.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.363+0000 7ff39b7fe640 1 -- 172.21.15.43:0/2144294896 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38c032810 con 0x7ff39c0b5d50 2024-04-25T09:02:01.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.364+0000 7ff3a31ce640 1 -- 172.21.15.43:0/2144294896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff39c0b5d50 msgr2=0x7ff39c0d4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:01.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.364+0000 7ff3a31ce640 1 --2- 172.21.15.43:0/2144294896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff39c0b5d50 0x7ff39c0d4550 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff38c009870 tx=0x7ff38c02a1c0 comp rx=0 tx=0).stop 2024-04-25T09:02:01.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.364+0000 7ff3a31ce640 1 -- 172.21.15.43:0/2144294896 shutdown_connections 2024-04-25T09:02:01.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.364+0000 7ff3a31ce640 1 --2- 172.21.15.43:0/2144294896 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff39c0b6130 0x7ff39c0d19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:01.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.364+0000 7ff3a31ce640 1 --2- 172.21.15.43:0/2144294896 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff39c0b5d50 0x7ff39c0d4550 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:01.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.364+0000 7ff3a31ce640 1 -- 172.21.15.43:0/2144294896 >> 172.21.15.43:0/2144294896 conn(0x7ff39c0d3a30 msgr2=0x7ff39c0d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:01.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.364+0000 7ff3a31ce640 1 -- 172.21.15.43:0/2144294896 shutdown_connections 2024-04-25T09:02:01.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.364+0000 7ff3a31ce640 1 -- 172.21.15.43:0/2144294896 wait complete. 2024-04-25T09:02:01.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.365+0000 7ff3a31ce640 1 Processor -- start 2024-04-25T09:02:01.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.365+0000 7ff3a31ce640 1 -- start start 2024-04-25T09:02:01.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.366+0000 7ff3a31ce640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff39c0b5d50 0x7ff39c163460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:01.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.366+0000 7ff3a31ce640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff39c0b6130 0x7ff39c14f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:01.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.366+0000 7ff3a31ce640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff39c0d1ee0 0x7ff39c0ce620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:01.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.366+0000 7ff3a31ce640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff39c0b7f60 con 0x7ff39c0d1ee0 2024-04-25T09:02:01.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.366+0000 7ff3a31ce640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff39c0b7de0 con 0x7ff39c0b5d50 2024-04-25T09:02:01.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.366+0000 7ff3a31ce640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff39c0b80e0 con 0x7ff39c0b6130 2024-04-25T09:02:01.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.366+0000 7ff3a176b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff39c0d1ee0 0x7ff39c0ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:01.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.366+0000 7ff3a176b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff39c0d1ee0 0x7ff39c0ce620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43070/0 (socket says 172.21.15.43:43070) 2024-04-25T09:02:01.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.366+0000 7ff3a176b640 1 -- 172.21.15.43:0/3235032293 learned_addr learned my addr 172.21.15.43:0/3235032293 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:01.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.366+0000 7ff3a0f6a640 1 --2- 172.21.15.43:0/3235032293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff39c0b5d50 0x7ff39c163460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:01.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.366+0000 7ff39bfff640 1 --2- 172.21.15.43:0/3235032293 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff39c0b6130 0x7ff39c14f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:01.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.366+0000 7ff3a176b640 1 -- 172.21.15.43:0/3235032293 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff39c0b6130 msgr2=0x7ff39c14f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:01.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.366+0000 7ff3a176b640 1 --2- 172.21.15.43:0/3235032293 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff39c0b6130 0x7ff39c14f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:01.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.367+0000 7ff3a176b640 1 -- 172.21.15.43:0/3235032293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff39c0b5d50 msgr2=0x7ff39c163460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:01.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.367+0000 7ff3a176b640 1 --2- 172.21.15.43:0/3235032293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff39c0b5d50 0x7ff39c163460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:01.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.367+0000 7ff3a176b640 1 -- 172.21.15.43:0/3235032293 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff39c164770 con 0x7ff39c0d1ee0 2024-04-25T09:02:01.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.367+0000 7ff3a176b640 1 --2- 172.21.15.43:0/3235032293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff39c0d1ee0 0x7ff39c0ce620 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7ff39400c7a0 tx=0x7ff39400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:01.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.367+0000 7ff3997fa640 1 -- 172.21.15.43:0/3235032293 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff394018070 con 0x7ff39c0d1ee0 2024-04-25T09:02:01.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.367+0000 7ff3997fa640 1 -- 172.21.15.43:0/3235032293 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff39400a640 con 0x7ff39c0d1ee0 2024-04-25T09:02:01.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.367+0000 7ff3a31ce640 1 -- 172.21.15.43:0/3235032293 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff39c164a50 con 0x7ff39c0d1ee0 2024-04-25T09:02:01.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.367+0000 7ff3997fa640 1 -- 172.21.15.43:0/3235032293 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff39400aab0 con 0x7ff39c0d1ee0 2024-04-25T09:02:01.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.367+0000 7ff3a31ce640 1 -- 172.21.15.43:0/3235032293 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff39c165010 con 0x7ff39c0d1ee0 2024-04-25T09:02:01.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.368+0000 7ff3a31ce640 1 -- 172.21.15.43:0/3235032293 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff39c0680f0 con 0x7ff39c0d1ee0 2024-04-25T09:02:01.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.368+0000 7ff3997fa640 1 -- 172.21.15.43:0/3235032293 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff39401b070 con 0x7ff39c0d1ee0 2024-04-25T09:02:01.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.369+0000 7ff3997fa640 1 --2- 172.21.15.43:0/3235032293 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff38806b4d0 0x7ff38806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:01.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.369+0000 7ff3997fa640 1 -- 172.21.15.43:0/3235032293 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff394089700 con 0x7ff39c0d1ee0 2024-04-25T09:02:01.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.369+0000 7ff3a0f6a640 1 --2- 172.21.15.43:0/3235032293 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff38806b4d0 0x7ff38806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:01.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.370+0000 7ff3a0f6a640 1 --2- 172.21.15.43:0/3235032293 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff38806b4d0 0x7ff38806d990 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ff38c034000 tx=0x7ff38c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:01.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.374+0000 7ff3997fa640 1 -- 172.21.15.43:0/3235032293 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff394057fc0 con 0x7ff39c0d1ee0 2024-04-25T09:02:01.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.595+0000 7ff3a31ce640 1 -- 172.21.15.43:0/3235032293 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff39c0b5640 con 0x7ff39c0d1ee0 2024-04-25T09:02:01.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.595+0000 7ff3997fa640 1 -- 172.21.15.43:0/3235032293 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4940 (secure 0 0 0) 0x7ff39405c060 con 0x7ff39c0d1ee0 2024-04-25T09:02:01.584 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:01.584 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":61,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2150177203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2150177203},{"type":"v1","addr":"172.21.15.193:6837","nonce":2150177203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":8079,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/621488642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":621488642},{"type":"v1","addr":"172.21.15.43:6834","nonce":621488642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8082,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1524395840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1524395840},{"type":"v1","addr":"172.21.15.43:6835","nonce":1524395840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:01:33.726790+0000","modified":"2024-04-25T09:01:35.706689+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8094},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8094":{"gid":8094,"name":"b","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.193:6836/853943267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":853943267},{"type":"v1","addr":"172.21.15.193:6836","nonce":853943267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:02:01.584 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 61 2024-04-25T09:02:01.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.597+0000 7ff3a31ce640 1 -- 172.21.15.43:0/3235032293 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff38806b4d0 msgr2=0x7ff38806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:01.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.597+0000 7ff3a31ce640 1 --2- 172.21.15.43:0/3235032293 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff38806b4d0 0x7ff38806d990 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ff38c034000 tx=0x7ff38c02e000 comp rx=0 tx=0).stop 2024-04-25T09:02:01.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.597+0000 7ff3a31ce640 1 -- 172.21.15.43:0/3235032293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff39c0d1ee0 msgr2=0x7ff39c0ce620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:01.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.597+0000 7ff3a31ce640 1 --2- 172.21.15.43:0/3235032293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff39c0d1ee0 0x7ff39c0ce620 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7ff39400c7a0 tx=0x7ff39400cc60 comp rx=0 tx=0).stop 2024-04-25T09:02:01.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.597+0000 7ff3a31ce640 1 -- 172.21.15.43:0/3235032293 shutdown_connections 2024-04-25T09:02:01.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.597+0000 7ff3a31ce640 1 --2- 172.21.15.43:0/3235032293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff39c0d1ee0 0x7ff39c0ce620 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:01.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.597+0000 7ff3a31ce640 1 --2- 172.21.15.43:0/3235032293 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff39c0b6130 0x7ff39c14f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:01.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.597+0000 7ff3a31ce640 1 --2- 172.21.15.43:0/3235032293 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff38806b4d0 0x7ff38806d990 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:01.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.597+0000 7ff3a31ce640 1 --2- 172.21.15.43:0/3235032293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff39c0b5d50 0x7ff39c163460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:01.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.597+0000 7ff3a31ce640 1 -- 172.21.15.43:0/3235032293 >> 172.21.15.43:0/3235032293 conn(0x7ff39c0d3a30 msgr2=0x7ff39c06b930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:01.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.597+0000 7ff3a31ce640 1 -- 172.21.15.43:0/3235032293 shutdown_connections 2024-04-25T09:02:01.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.597+0000 7ff3a31ce640 1 -- 172.21.15.43:0/3235032293 wait complete. 2024-04-25T09:02:01.597 DEBUG:tasks.cephfs.test_scrub_checks:current active=['b'] 2024-04-25T09:02:01.597 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:02:01.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.708+0000 7f89abe6f640 1 Processor -- start 2024-04-25T09:02:01.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.709+0000 7f89abe6f640 1 -- start start 2024-04-25T09:02:01.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.709+0000 7f89abe6f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f89a415df50 0x7f89a41619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:01.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.709+0000 7f89abe6f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f89a40cf670 0x7f89a4162370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:01.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.709+0000 7f89abe6f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89a40d24a0 con 0x7f89a4161f30 2024-04-25T09:02:01.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.709+0000 7f89abe6f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89a40cfa80 con 0x7f89a40cf670 2024-04-25T09:02:01.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.709+0000 7f89abe6f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89a415e330 con 0x7f89a415df50 2024-04-25T09:02:01.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.709+0000 7f89aa40c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f89a40cf670 0x7f89a4162370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:01.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.709+0000 7f89a940a640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f89a4161f30 0x7f89a40c7cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48716/0 (socket says 172.21.15.43:48716) 2024-04-25T09:02:01.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.709+0000 7f89a940a640 1 -- 172.21.15.43:0/1379839261 learned_addr learned my addr 172.21.15.43:0/1379839261 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:01.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.710+0000 7f89aa40c640 1 -- 172.21.15.43:0/1379839261 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f89a415df50 msgr2=0x7f89a41619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:01.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.710+0000 7f89a9c0b640 1 --2- 172.21.15.43:0/1379839261 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f89a415df50 0x7f89a41619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:01.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.710+0000 7f89aa40c640 1 --2- 172.21.15.43:0/1379839261 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f89a415df50 0x7f89a41619f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:01.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.710+0000 7f89aa40c640 1 -- 172.21.15.43:0/1379839261 >> v1:172.21.15.43:6789/0 conn(0x7f89a4161f30 legacy=0x7f89a40c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:01.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.710+0000 7f89a8c09640 1 -- 172.21.15.43:0/1379839261 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1273711893 0 0) 0x7f89a40d24a0 con 0x7f89a4161f30 2024-04-25T09:02:01.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.710+0000 7f89aa40c640 1 -- 172.21.15.43:0/1379839261 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a40d2740 con 0x7f89a40cf670 2024-04-25T09:02:01.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.710+0000 7f89aa40c640 1 --2- 172.21.15.43:0/1379839261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89a40cf670 0x7f89a4162370 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f8994009d90 tx=0x7f899402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4301989b2f8eb922 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:01.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.711+0000 7f89a8c09640 1 -- 172.21.15.43:0/1379839261 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8994007740 con 0x7f89a40cf670 2024-04-25T09:02:01.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.711+0000 7f89a8c09640 1 -- 172.21.15.43:0/1379839261 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89940078e0 con 0x7f89a40cf670 2024-04-25T09:02:01.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.711+0000 7f89a8c09640 1 -- 172.21.15.43:0/1379839261 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8994007cb0 con 0x7f89a40cf670 2024-04-25T09:02:01.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.711+0000 7f89abe6f640 1 -- 172.21.15.43:0/1379839261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89a40cf670 msgr2=0x7f89a4162370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:01.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.711+0000 7f89abe6f640 1 --2- 172.21.15.43:0/1379839261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89a40cf670 0x7f89a4162370 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f8994009d90 tx=0x7f899402a660 comp rx=0 tx=0).stop 2024-04-25T09:02:01.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.711+0000 7f89abe6f640 1 -- 172.21.15.43:0/1379839261 shutdown_connections 2024-04-25T09:02:01.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.711+0000 7f89abe6f640 1 --2- 172.21.15.43:0/1379839261 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89a40cf670 0x7f89a4162370 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:01.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.711+0000 7f89abe6f640 1 --2- 172.21.15.43:0/1379839261 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f89a415df50 0x7f89a41619f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:01.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.711+0000 7f89abe6f640 1 -- 172.21.15.43:0/1379839261 >> 172.21.15.43:0/1379839261 conn(0x7f89a40d3a30 msgr2=0x7f89a40d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:01.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.711+0000 7f89abe6f640 1 -- 172.21.15.43:0/1379839261 shutdown_connections 2024-04-25T09:02:01.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.711+0000 7f89abe6f640 1 -- 172.21.15.43:0/1379839261 wait complete. 2024-04-25T09:02:01.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.713+0000 7f89abe6f640 1 Processor -- start 2024-04-25T09:02:01.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.713+0000 7f89abe6f640 1 -- start start 2024-04-25T09:02:01.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.713+0000 7f89abe6f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89a40cf670 0x7f89a41562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:01.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.713+0000 7f89abe6f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89a415df50 0x7f89a4160540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:01.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.713+0000 7f89abe6f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f89a4161f30 0x7f89a40c8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:01.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.713+0000 7f89abe6f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a40c77d0 con 0x7f89a40cf670 2024-04-25T09:02:01.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.713+0000 7f89abe6f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a40c74f0 con 0x7f89a415df50 2024-04-25T09:02:01.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.713+0000 7f89abe6f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89a40d2190 con 0x7f89a4161f30 2024-04-25T09:02:01.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.713+0000 7f89a9c0b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89a40cf670 0x7f89a41562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:01.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.713+0000 7f89a9c0b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89a40cf670 0x7f89a41562a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43100/0 (socket says 172.21.15.43:43100) 2024-04-25T09:02:01.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.713+0000 7f89a9c0b640 1 -- 172.21.15.43:0/2856778833 learned_addr learned my addr 172.21.15.43:0/2856778833 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:01.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.714+0000 7f89a940a640 1 --2- 172.21.15.43:0/2856778833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89a415df50 0x7f89a4160540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:01.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.714+0000 7f89aa40c640 1 --2- 172.21.15.43:0/2856778833 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f89a4161f30 0x7f89a40c8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:01.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.714+0000 7f89a9c0b640 1 -- 172.21.15.43:0/2856778833 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f89a4161f30 msgr2=0x7f89a40c8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:01.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.714+0000 7f89a9c0b640 1 --2- 172.21.15.43:0/2856778833 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f89a4161f30 0x7f89a40c8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:01.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.714+0000 7f89a9c0b640 1 -- 172.21.15.43:0/2856778833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89a415df50 msgr2=0x7f89a4160540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:01.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.714+0000 7f89a9c0b640 1 --2- 172.21.15.43:0/2856778833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89a415df50 0x7f89a4160540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:01.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.714+0000 7f89a9c0b640 1 -- 172.21.15.43:0/2856778833 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a41f7260 con 0x7f89a40cf670 2024-04-25T09:02:01.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.714+0000 7f89a9c0b640 1 --2- 172.21.15.43:0/2856778833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89a40cf670 0x7f89a41562a0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f899c00c7f0 tx=0x7f899c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:01.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.714+0000 7f89a27fc640 1 -- 172.21.15.43:0/2856778833 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899c015070 con 0x7f89a40cf670 2024-04-25T09:02:01.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.714+0000 7f89a27fc640 1 -- 172.21.15.43:0/2856778833 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f899c004240 con 0x7f89a40cf670 2024-04-25T09:02:01.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.714+0000 7f89abe6f640 1 -- 172.21.15.43:0/2856778833 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89a41f7540 con 0x7f89a40cf670 2024-04-25T09:02:01.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.714+0000 7f89a27fc640 1 -- 172.21.15.43:0/2856778833 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899c007620 con 0x7f89a40cf670 2024-04-25T09:02:01.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.714+0000 7f89abe6f640 1 -- 172.21.15.43:0/2856778833 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89a41f7b00 con 0x7f89a40cf670 2024-04-25T09:02:01.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.715+0000 7f89abe6f640 1 -- 172.21.15.43:0/2856778833 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89a40680f0 con 0x7f89a40cf670 2024-04-25T09:02:01.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.716+0000 7f89a27fc640 1 -- 172.21.15.43:0/2856778833 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f899c0077c0 con 0x7f89a40cf670 2024-04-25T09:02:01.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.716+0000 7f89a27fc640 1 --2- 172.21.15.43:0/2856778833 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f899006b4d0 0x7f899006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:01.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.716+0000 7f89a27fc640 1 -- 172.21.15.43:0/2856778833 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f899c08a0b0 con 0x7f89a40cf670 2024-04-25T09:02:01.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.717+0000 7f89a940a640 1 --2- 172.21.15.43:0/2856778833 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f899006b4d0 0x7f899006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:01.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.717+0000 7f89a940a640 1 --2- 172.21.15.43:0/2856778833 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f899006b4d0 0x7f899006d990 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f8998007fd0 tx=0x7f8998007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:01.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.721+0000 7f89a27fc640 1 -- 172.21.15.43:0/2856778833 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f899c058840 con 0x7f89a40cf670 2024-04-25T09:02:01.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.970+0000 7f89abe6f640 1 -- 172.21.15.43:0/2856778833 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f89a4074ef0 con 0x7f89a40cf670 2024-04-25T09:02:01.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.971+0000 7f89a27fc640 1 -- 172.21.15.43:0/2856778833 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1575 (secure 0 0 0) 0x7f899c05c8e0 con 0x7f89a40cf670 2024-04-25T09:02:01.961 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:01.961 INFO:teuthology.orchestra.run.smithi043.stdout:{"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":494,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":102,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035238,"num_in_osds":8,"osd_in_since":1714035229,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076523892,"bytes_used":2674450432,"bytes_avail":770419662848,"bytes_total":773094113280,"write_bytes_sec":89652558,"read_op_per_sec":0,"write_op_per_sec":89},"fsmap":{"epoch":61,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"b","status":"up:active","gid":8094}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":17,"modified":"2024-04-25T09:01:15.791620+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:02:01.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.973+0000 7f89abe6f640 1 -- 172.21.15.43:0/2856778833 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f899006b4d0 msgr2=0x7f899006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:01.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.973+0000 7f89abe6f640 1 --2- 172.21.15.43:0/2856778833 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f899006b4d0 0x7f899006d990 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f8998007fd0 tx=0x7f8998007de0 comp rx=0 tx=0).stop 2024-04-25T09:02:01.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.973+0000 7f89abe6f640 1 -- 172.21.15.43:0/2856778833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89a40cf670 msgr2=0x7f89a41562a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:01.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.973+0000 7f89abe6f640 1 --2- 172.21.15.43:0/2856778833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89a40cf670 0x7f89a41562a0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f899c00c7f0 tx=0x7f899c00ccb0 comp rx=0 tx=0).stop 2024-04-25T09:02:01.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.973+0000 7f89abe6f640 1 -- 172.21.15.43:0/2856778833 shutdown_connections 2024-04-25T09:02:01.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.973+0000 7f89abe6f640 1 --2- 172.21.15.43:0/2856778833 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f89a4161f30 0x7f89a40c8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:01.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.973+0000 7f89abe6f640 1 --2- 172.21.15.43:0/2856778833 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f899006b4d0 0x7f899006d990 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:01.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.973+0000 7f89abe6f640 1 --2- 172.21.15.43:0/2856778833 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f89a415df50 0x7f89a4160540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:01.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.973+0000 7f89abe6f640 1 --2- 172.21.15.43:0/2856778833 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f89a40cf670 0x7f89a41562a0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:01.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.973+0000 7f89abe6f640 1 -- 172.21.15.43:0/2856778833 >> 172.21.15.43:0/2856778833 conn(0x7f89a40d3a30 msgr2=0x7f89a415bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:01.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.973+0000 7f89abe6f640 1 -- 172.21.15.43:0/2856778833 shutdown_connections 2024-04-25T09:02:01.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:01.973+0000 7f89abe6f640 1 -- 172.21.15.43:0/2856778833 wait complete. 2024-04-25T09:02:01.972 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T09:02:02.973 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:02:03.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.098+0000 7fa3fb390640 1 Processor -- start 2024-04-25T09:02:03.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.098+0000 7fa3fb390640 1 -- start start 2024-04-25T09:02:03.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.099+0000 7fa3fb390640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fa3f415ded0 0x7fa3f40ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.099+0000 7fa3fb390640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3f40cf6e0 con 0x7fa3f40cf250 2024-04-25T09:02:03.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.099+0000 7fa3fb390640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3f40cf9c0 con 0x7fa3f406b240 2024-04-25T09:02:03.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.099+0000 7fa3fb390640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3f415e2b0 con 0x7fa3f415ded0 2024-04-25T09:02:03.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.099+0000 7fa3f892b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa3f40cf250 0x7fa3f406ab30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48722/0 (socket says 172.21.15.43:48722) 2024-04-25T09:02:03.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.099+0000 7fa3f892b640 1 -- 172.21.15.43:0/3994099351 learned_addr learned my addr 172.21.15.43:0/3994099351 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:03.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.099+0000 7fa3f912c640 1 --2- 172.21.15.43:0/3994099351 >> v2:172.21.15.193:3300/0 conn(0x7fa3f415ded0 0x7fa3f40ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.099+0000 7fa3f3fff640 1 -- 172.21.15.43:0/3994099351 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1579351286 0 0) 0x7fa3f40cf6e0 con 0x7fa3f40cf250 2024-04-25T09:02:03.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.099+0000 7fa3f3fff640 1 -- 172.21.15.43:0/3994099351 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3dc003520 con 0x7fa3f40cf250 2024-04-25T09:02:03.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.100+0000 7fa3f3fff640 1 -- 172.21.15.43:0/3994099351 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1190598856 0 0) 0x7fa3dc003520 con 0x7fa3f40cf250 2024-04-25T09:02:03.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.100+0000 7fa3f3fff640 1 -- 172.21.15.43:0/3994099351 >> v2:172.21.15.193:3300/0 conn(0x7fa3f415ded0 msgr2=0x7fa3f40ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:03.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.100+0000 7fa3f3fff640 1 --2- 172.21.15.43:0/3994099351 >> v2:172.21.15.193:3300/0 conn(0x7fa3f415ded0 0x7fa3f40ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.100+0000 7fa3f3fff640 1 -- 172.21.15.43:0/3994099351 >> v1:172.21.15.193:6789/0 conn(0x7fa3f406b240 legacy=0x7fa3f40b4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:03.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.100+0000 7fa3f3fff640 1 -- 172.21.15.43:0/3994099351 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f40cfca0 con 0x7fa3f40cf250 2024-04-25T09:02:03.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.100+0000 7fa3f3fff640 1 -- 172.21.15.43:0/3994099351 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7fa3e8001670 con 0x7fa3f40cf250 2024-04-25T09:02:03.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.100+0000 7fa3f3fff640 1 -- 172.21.15.43:0/3994099351 >> v1:172.21.15.43:6789/0 conn(0x7fa3f40cf250 legacy=0x7fa3f406ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:03.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.100+0000 7fa3f3fff640 1 --2- 172.21.15.43:0/3994099351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3dc0039b0 0x7fa3dc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.100+0000 7fa3f3fff640 1 --2- 172.21.15.43:0/3994099351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3dc004800 0x7fa3dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.100+0000 7fa3f3fff640 1 --2- 172.21.15.43:0/3994099351 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa3dc0051c0 0x7fa3dc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.100+0000 7fa3f3fff640 1 -- 172.21.15.43:0/3994099351 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc00c030 con 0x7fa3dc0039b0 2024-04-25T09:02:03.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.100+0000 7fa3f3fff640 1 -- 172.21.15.43:0/3994099351 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc00c1e0 con 0x7fa3dc004800 2024-04-25T09:02:03.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.100+0000 7fa3f3fff640 1 -- 172.21.15.43:0/3994099351 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc00c3e0 con 0x7fa3dc0051c0 2024-04-25T09:02:03.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.100+0000 7fa3f912c640 1 --2- 172.21.15.43:0/3994099351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3dc0039b0 0x7fa3dc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.100+0000 7fa3f892b640 1 --2- 172.21.15.43:0/3994099351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3dc004800 0x7fa3dc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.100+0000 7fa3f992d640 1 --2- 172.21.15.43:0/3994099351 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa3dc0051c0 0x7fa3dc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.100+0000 7fa3f912c640 1 -- 172.21.15.43:0/3994099351 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa3dc0051c0 msgr2=0x7fa3dc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:03.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.100+0000 7fa3f912c640 1 --2- 172.21.15.43:0/3994099351 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa3dc0051c0 0x7fa3dc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.100+0000 7fa3f912c640 1 -- 172.21.15.43:0/3994099351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3dc004800 msgr2=0x7fa3dc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:03.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.100+0000 7fa3f912c640 1 --2- 172.21.15.43:0/3994099351 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3dc004800 0x7fa3dc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.100+0000 7fa3f912c640 1 -- 172.21.15.43:0/3994099351 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3dc003520 con 0x7fa3dc0039b0 2024-04-25T09:02:03.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.100+0000 7fa3f992d640 1 --2- 172.21.15.43:0/3994099351 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa3dc0051c0 0x7fa3dc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:03.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.100+0000 7fa3f992d640 1 -- 172.21.15.43:0/3994099351 reap_dead start 2024-04-25T09:02:03.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.101+0000 7fa3f912c640 1 --2- 172.21.15.43:0/3994099351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3dc0039b0 0x7fa3dc003dd0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec009ac0 tx=0x7fa3dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=129ba31c0574ced5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:03.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.101+0000 7fa3f3fff640 1 -- 172.21.15.43:0/3994099351 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec036070 con 0x7fa3dc0039b0 2024-04-25T09:02:03.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.101+0000 7fa3f3fff640 1 -- 172.21.15.43:0/3994099351 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3ec031400 con 0x7fa3dc0039b0 2024-04-25T09:02:03.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.101+0000 7fa3f3fff640 1 -- 172.21.15.43:0/3994099351 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ec009690 con 0x7fa3dc0039b0 2024-04-25T09:02:03.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.102+0000 7fa3fb390640 1 -- 172.21.15.43:0/3994099351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3dc0039b0 msgr2=0x7fa3dc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:03.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.102+0000 7fa3fb390640 1 --2- 172.21.15.43:0/3994099351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3dc0039b0 0x7fa3dc003dd0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec009ac0 tx=0x7fa3dc005830 comp rx=0 tx=0).stop 2024-04-25T09:02:03.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.102+0000 7fa3fb390640 1 -- 172.21.15.43:0/3994099351 shutdown_connections 2024-04-25T09:02:03.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.102+0000 7fa3fb390640 1 --2- 172.21.15.43:0/3994099351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3dc0039b0 0x7fa3dc003dd0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.102+0000 7fa3fb390640 1 -- 172.21.15.43:0/3994099351 >> 172.21.15.43:0/3994099351 conn(0x7fa3f40b2490 msgr2=0x7fa3f40b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:03.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.102+0000 7fa3fb390640 1 -- 172.21.15.43:0/3994099351 shutdown_connections 2024-04-25T09:02:03.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.102+0000 7fa3fb390640 1 -- 172.21.15.43:0/3994099351 wait complete. 2024-04-25T09:02:03.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.103+0000 7fa3fb390640 1 Processor -- start 2024-04-25T09:02:03.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.103+0000 7fa3fb390640 1 -- start start 2024-04-25T09:02:03.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.104+0000 7fa3fb390640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa3dc0039b0 0x7fa3f4160040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.104+0000 7fa3fb390640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3f414b0f0 0x7fa3f4160ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.104+0000 7fa3fb390640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f4160ff0 0x7fa3f414ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.104+0000 7fa3fb390640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f40cefd0 con 0x7fa3f4160ff0 2024-04-25T09:02:03.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.104+0000 7fa3fb390640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f40cee50 con 0x7fa3f414b0f0 2024-04-25T09:02:03.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.104+0000 7fa3fb390640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f40ce740 con 0x7fa3dc0039b0 2024-04-25T09:02:03.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.104+0000 7fa3f992d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f4160ff0 0x7fa3f414ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.104+0000 7fa3f992d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f4160ff0 0x7fa3f414ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43110/0 (socket says 172.21.15.43:43110) 2024-04-25T09:02:03.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.104+0000 7fa3f992d640 1 -- 172.21.15.43:0/3558913585 learned_addr learned my addr 172.21.15.43:0/3558913585 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:03.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.104+0000 7fa3f892b640 1 --2- 172.21.15.43:0/3558913585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3f414b0f0 0x7fa3f4160ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.104+0000 7fa3f912c640 1 --2- 172.21.15.43:0/3558913585 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa3dc0039b0 0x7fa3f4160040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.104+0000 7fa3f992d640 1 -- 172.21.15.43:0/3558913585 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa3dc0039b0 msgr2=0x7fa3f4160040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:03.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.104+0000 7fa3f992d640 1 --2- 172.21.15.43:0/3558913585 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa3dc0039b0 0x7fa3f4160040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.104+0000 7fa3f992d640 1 -- 172.21.15.43:0/3558913585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3f414b0f0 msgr2=0x7fa3f4160ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:03.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.104+0000 7fa3f992d640 1 --2- 172.21.15.43:0/3558913585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3f414b0f0 0x7fa3f4160ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.105+0000 7fa3f992d640 1 -- 172.21.15.43:0/3558913585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f415ecc0 con 0x7fa3f4160ff0 2024-04-25T09:02:03.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.105+0000 7fa3f892b640 1 --2- 172.21.15.43:0/3558913585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3f414b0f0 0x7fa3f4160ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:03.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.105+0000 7fa3f992d640 1 --2- 172.21.15.43:0/3558913585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f4160ff0 0x7fa3f414ba10 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fa3f406bb80 tx=0x7fa3e400eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:03.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.105+0000 7fa3f1ffb640 1 -- 172.21.15.43:0/3558913585 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e4003970 con 0x7fa3f4160ff0 2024-04-25T09:02:03.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.105+0000 7fa3f1ffb640 1 -- 172.21.15.43:0/3558913585 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e4018070 con 0x7fa3f4160ff0 2024-04-25T09:02:03.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.105+0000 7fa3f1ffb640 1 -- 172.21.15.43:0/3558913585 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e4003d50 con 0x7fa3f4160ff0 2024-04-25T09:02:03.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.105+0000 7fa3fb390640 1 -- 172.21.15.43:0/3558913585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3f415efa0 con 0x7fa3f4160ff0 2024-04-25T09:02:03.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.105+0000 7fa3fb390640 1 -- 172.21.15.43:0/3558913585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3f415f560 con 0x7fa3f4160ff0 2024-04-25T09:02:03.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.106+0000 7fa3fb390640 1 -- 172.21.15.43:0/3558913585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3f40cea40 con 0x7fa3f4160ff0 2024-04-25T09:02:03.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.107+0000 7fa3f1ffb640 1 -- 172.21.15.43:0/3558913585 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa3e40050d0 con 0x7fa3f4160ff0 2024-04-25T09:02:03.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.107+0000 7fa3f1ffb640 1 --2- 172.21.15.43:0/3558913585 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa3e006b4d0 0x7fa3e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.107+0000 7fa3f1ffb640 1 -- 172.21.15.43:0/3558913585 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa3e4089b10 con 0x7fa3f4160ff0 2024-04-25T09:02:03.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.107+0000 7fa3f912c640 1 --2- 172.21.15.43:0/3558913585 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa3e006b4d0 0x7fa3e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.108+0000 7fa3f912c640 1 --2- 172.21.15.43:0/3558913585 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa3e006b4d0 0x7fa3e006d990 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec033000 tx=0x7fa3ec008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:03.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.111+0000 7fa3f1ffb640 1 -- 172.21.15.43:0/3558913585 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa3f40cea40 con 0x7fa3f4160ff0 2024-04-25T09:02:03.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.326+0000 7fa3fb390640 1 -- 172.21.15.43:0/3558913585 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3f4074ef0 con 0x7fa3f4160ff0 2024-04-25T09:02:03.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.327+0000 7fa3f1ffb640 1 -- 172.21.15.43:0/3558913585 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4940 (secure 0 0 0) 0x7fa3e4022020 con 0x7fa3f4160ff0 2024-04-25T09:02:03.316 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:03.316 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":61,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2150177203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2150177203},{"type":"v1","addr":"172.21.15.193:6837","nonce":2150177203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":8079,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/621488642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":621488642},{"type":"v1","addr":"172.21.15.43:6834","nonce":621488642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8082,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1524395840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1524395840},{"type":"v1","addr":"172.21.15.43:6835","nonce":1524395840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:01:33.726790+0000","modified":"2024-04-25T09:01:35.706689+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8094},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8094":{"gid":8094,"name":"b","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.193:6836/853943267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":853943267},{"type":"v1","addr":"172.21.15.193:6836","nonce":853943267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:02:03.316 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 61 2024-04-25T09:02:03.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.329+0000 7fa3fb390640 1 -- 172.21.15.43:0/3558913585 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa3e006b4d0 msgr2=0x7fa3e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:03.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.329+0000 7fa3fb390640 1 --2- 172.21.15.43:0/3558913585 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa3e006b4d0 0x7fa3e006d990 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec033000 tx=0x7fa3ec008030 comp rx=0 tx=0).stop 2024-04-25T09:02:03.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.329+0000 7fa3fb390640 1 -- 172.21.15.43:0/3558913585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f4160ff0 msgr2=0x7fa3f414ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:03.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.329+0000 7fa3fb390640 1 --2- 172.21.15.43:0/3558913585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f4160ff0 0x7fa3f414ba10 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fa3f406bb80 tx=0x7fa3e400eb90 comp rx=0 tx=0).stop 2024-04-25T09:02:03.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.329+0000 7fa3fb390640 1 -- 172.21.15.43:0/3558913585 shutdown_connections 2024-04-25T09:02:03.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.329+0000 7fa3fb390640 1 --2- 172.21.15.43:0/3558913585 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3f4160ff0 0x7fa3f414ba10 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.329+0000 7fa3fb390640 1 --2- 172.21.15.43:0/3558913585 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa3e006b4d0 0x7fa3e006d990 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.329+0000 7fa3fb390640 1 --2- 172.21.15.43:0/3558913585 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3f414b0f0 0x7fa3f4160ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.329+0000 7fa3fb390640 1 --2- 172.21.15.43:0/3558913585 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa3dc0039b0 0x7fa3f4160040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.329+0000 7fa3fb390640 1 -- 172.21.15.43:0/3558913585 >> 172.21.15.43:0/3558913585 conn(0x7fa3f40b2490 msgr2=0x7fa3f415d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:03.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.329+0000 7fa3fb390640 1 -- 172.21.15.43:0/3558913585 shutdown_connections 2024-04-25T09:02:03.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.329+0000 7fa3fb390640 1 -- 172.21.15.43:0/3558913585 wait complete. 2024-04-25T09:02:03.329 DEBUG:tasks.cephfs.test_scrub_checks:current active=['b'] 2024-04-25T09:02:03.329 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:02:03.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.440+0000 7f0518f1d640 1 Processor -- start 2024-04-25T09:02:03.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.440+0000 7f0518f1d640 1 -- start start 2024-04-25T09:02:03.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.441+0000 7f0518f1d640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f051415f180 0x7f05140cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.441+0000 7f0518f1d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f05140b23f0 0x7f05140cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.441+0000 7f0518f1d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f05140b2cc0 0x7f051415f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.441+0000 7f0518f1d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05140cccc0 con 0x7f05140b2cc0 2024-04-25T09:02:03.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.441+0000 7f0518f1d640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05140cd840 con 0x7f051415f180 2024-04-25T09:02:03.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.441+0000 7f0518f1d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05140ccf10 con 0x7f05140b23f0 2024-04-25T09:02:03.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.441+0000 7f0512d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f05140b2cc0 0x7f051415f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.441+0000 7f0512d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f05140b2cc0 0x7f051415f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43124/0 (socket says 172.21.15.43:43124) 2024-04-25T09:02:03.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.441+0000 7f0512d9d640 1 -- 172.21.15.43:0/1181334287 learned_addr learned my addr 172.21.15.43:0/1181334287 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:03.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.441+0000 7f051259c640 1 --2- 172.21.15.43:0/1181334287 >> v2:172.21.15.193:3300/0 conn(0x7f051415f180 0x7f05140cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.441+0000 7f0511d9b640 1 --2- 172.21.15.43:0/1181334287 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f05140b23f0 0x7f05140cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.442+0000 7f0512d9d640 1 -- 172.21.15.43:0/1181334287 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f05140b23f0 msgr2=0x7f05140cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:03.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.442+0000 7f0512d9d640 1 --2- 172.21.15.43:0/1181334287 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f05140b23f0 0x7f05140cfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.442+0000 7f0512d9d640 1 -- 172.21.15.43:0/1181334287 >> v2:172.21.15.193:3300/0 conn(0x7f051415f180 msgr2=0x7f05140cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:03.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.442+0000 7f0512d9d640 1 --2- 172.21.15.43:0/1181334287 >> v2:172.21.15.193:3300/0 conn(0x7f051415f180 0x7f05140cf860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.442+0000 7f0512d9d640 1 -- 172.21.15.43:0/1181334287 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05140cd1f0 con 0x7f05140b2cc0 2024-04-25T09:02:03.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.442+0000 7f0511d9b640 1 --2- 172.21.15.43:0/1181334287 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f05140b23f0 0x7f05140cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:03.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.442+0000 7f0512d9d640 1 --2- 172.21.15.43:0/1181334287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05140b2cc0 0x7f051415f8e0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f0514071e60 tx=0x7f050c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2313a27f300afa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:03.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.442+0000 7f051159a640 1 -- 172.21.15.43:0/1181334287 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050c037070 con 0x7f05140b2cc0 2024-04-25T09:02:03.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.442+0000 7f051159a640 1 -- 172.21.15.43:0/1181334287 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f050c032440 con 0x7f05140b2cc0 2024-04-25T09:02:03.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.442+0000 7f051159a640 1 -- 172.21.15.43:0/1181334287 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050c032810 con 0x7f05140b2cc0 2024-04-25T09:02:03.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.443+0000 7f0518f1d640 1 -- 172.21.15.43:0/1181334287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05140b2cc0 msgr2=0x7f051415f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:03.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.443+0000 7f0518f1d640 1 --2- 172.21.15.43:0/1181334287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05140b2cc0 0x7f051415f8e0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f0514071e60 tx=0x7f050c02a140 comp rx=0 tx=0).stop 2024-04-25T09:02:03.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.443+0000 7f0518f1d640 1 -- 172.21.15.43:0/1181334287 shutdown_connections 2024-04-25T09:02:03.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.443+0000 7f0518f1d640 1 --2- 172.21.15.43:0/1181334287 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f05140b2cc0 0x7f051415f8e0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.443+0000 7f0518f1d640 1 --2- 172.21.15.43:0/1181334287 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f05140b23f0 0x7f05140cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.443+0000 7f0518f1d640 1 --2- 172.21.15.43:0/1181334287 >> v2:172.21.15.193:3300/0 conn(0x7f051415f180 0x7f05140cf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.443+0000 7f0518f1d640 1 -- 172.21.15.43:0/1181334287 >> 172.21.15.43:0/1181334287 conn(0x7f05140d17f0 msgr2=0x7f05140d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:03.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.443+0000 7f0518f1d640 1 -- 172.21.15.43:0/1181334287 shutdown_connections 2024-04-25T09:02:03.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.443+0000 7f0518f1d640 1 -- 172.21.15.43:0/1181334287 wait complete. 2024-04-25T09:02:03.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.444+0000 7f0518f1d640 1 Processor -- start 2024-04-25T09:02:03.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.444+0000 7f0518f1d640 1 -- start start 2024-04-25T09:02:03.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.445+0000 7f0518f1d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05140b23f0 0x7f05141eef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.445+0000 7f0518f1d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f05140b2cc0 0x7f05140c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.445+0000 7f0518f1d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f051415f180 0x7f05141f2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.445+0000 7f0518f1d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f051415f560 con 0x7f051415f180 2024-04-25T09:02:03.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.445+0000 7f0518f1d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05140d23f0 con 0x7f05140b23f0 2024-04-25T09:02:03.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.445+0000 7f0518f1d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05140c9b90 con 0x7f05140b2cc0 2024-04-25T09:02:03.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.445+0000 7f0512d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f051415f180 0x7f05141f2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.445+0000 7f0512d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f051415f180 0x7f05141f2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43130/0 (socket says 172.21.15.43:43130) 2024-04-25T09:02:03.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.445+0000 7f0512d9d640 1 -- 172.21.15.43:0/1703103861 learned_addr learned my addr 172.21.15.43:0/1703103861 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:03.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.445+0000 7f051259c640 1 --2- 172.21.15.43:0/1703103861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05140b23f0 0x7f05141eef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.445+0000 7f0511d9b640 1 --2- 172.21.15.43:0/1703103861 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f05140b2cc0 0x7f05140c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.446+0000 7f0512d9d640 1 -- 172.21.15.43:0/1703103861 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f05140b2cc0 msgr2=0x7f05140c88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:03.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.446+0000 7f0512d9d640 1 --2- 172.21.15.43:0/1703103861 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f05140b2cc0 0x7f05140c88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.446+0000 7f0512d9d640 1 -- 172.21.15.43:0/1703103861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05140b23f0 msgr2=0x7f05141eef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:03.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.446+0000 7f0512d9d640 1 --2- 172.21.15.43:0/1703103861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05140b23f0 0x7f05141eef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.446+0000 7f0512d9d640 1 -- 172.21.15.43:0/1703103861 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05140c8f80 con 0x7f051415f180 2024-04-25T09:02:03.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.446+0000 7f0512d9d640 1 --2- 172.21.15.43:0/1703103861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f051415f180 0x7f05141f2390 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f050c0090b0 tx=0x7f050c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:03.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.446+0000 7f050affd640 1 -- 172.21.15.43:0/1703103861 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050c037040 con 0x7f051415f180 2024-04-25T09:02:03.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.446+0000 7f050affd640 1 -- 172.21.15.43:0/1703103861 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f050c042070 con 0x7f051415f180 2024-04-25T09:02:03.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.446+0000 7f0518f1d640 1 -- 172.21.15.43:0/1703103861 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05140c9260 con 0x7f051415f180 2024-04-25T09:02:03.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.446+0000 7f050affd640 1 -- 172.21.15.43:0/1703103861 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f050c030920 con 0x7f051415f180 2024-04-25T09:02:03.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.446+0000 7f0518f1d640 1 -- 172.21.15.43:0/1703103861 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05140c9820 con 0x7f051415f180 2024-04-25T09:02:03.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.447+0000 7f0518f1d640 1 -- 172.21.15.43:0/1703103861 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0514074ef0 con 0x7f051415f180 2024-04-25T09:02:03.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.448+0000 7f050affd640 1 -- 172.21.15.43:0/1703103861 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f050c030ac0 con 0x7f051415f180 2024-04-25T09:02:03.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.448+0000 7f050affd640 1 --2- 172.21.15.43:0/1703103861 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f04f406b4d0 0x7f04f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.448+0000 7f050affd640 1 -- 172.21.15.43:0/1703103861 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f050c0aa920 con 0x7f051415f180 2024-04-25T09:02:03.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.448+0000 7f051259c640 1 --2- 172.21.15.43:0/1703103861 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f04f406b4d0 0x7f04f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.449+0000 7f051259c640 1 --2- 172.21.15.43:0/1703103861 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f04f406b4d0 0x7f04f406d990 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f05140d05a0 tx=0x7f0504009830 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:03.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.454+0000 7f050affd640 1 -- 172.21.15.43:0/1703103861 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f050c079150 con 0x7f051415f180 2024-04-25T09:02:03.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.703+0000 7f0518f1d640 1 -- 172.21.15.43:0/1703103861 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f05140cf050 con 0x7f051415f180 2024-04-25T09:02:03.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.704+0000 7f050affd640 1 -- 172.21.15.43:0/1703103861 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7f050c07d1f0 con 0x7f051415f180 2024-04-25T09:02:03.693 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:03.693 INFO:teuthology.orchestra.run.smithi043.stdout:{"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":496,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":102,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035238,"num_in_osds":8,"osd_in_since":1714035229,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076523892,"bytes_used":2674450432,"bytes_avail":770419662848,"bytes_total":773094113280,"write_bytes_sec":62479100,"read_op_per_sec":0,"write_op_per_sec":62},"fsmap":{"epoch":61,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"b","status":"up:active","gid":8094}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":18,"modified":"2024-04-25T09:02:01.803060+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:02:03.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.706+0000 7f0518f1d640 1 -- 172.21.15.43:0/1703103861 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f04f406b4d0 msgr2=0x7f04f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:03.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.706+0000 7f0518f1d640 1 --2- 172.21.15.43:0/1703103861 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f04f406b4d0 0x7f04f406d990 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f05140d05a0 tx=0x7f0504009830 comp rx=0 tx=0).stop 2024-04-25T09:02:03.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.706+0000 7f0518f1d640 1 -- 172.21.15.43:0/1703103861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f051415f180 msgr2=0x7f05141f2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:03.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.706+0000 7f0518f1d640 1 --2- 172.21.15.43:0/1703103861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f051415f180 0x7f05141f2390 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f050c0090b0 tx=0x7f050c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.706+0000 7f0518f1d640 1 -- 172.21.15.43:0/1703103861 shutdown_connections 2024-04-25T09:02:03.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.706+0000 7f0518f1d640 1 --2- 172.21.15.43:0/1703103861 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f051415f180 0x7f05141f2390 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.706+0000 7f0518f1d640 1 --2- 172.21.15.43:0/1703103861 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f05140b2cc0 0x7f05140c88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.706+0000 7f0518f1d640 1 --2- 172.21.15.43:0/1703103861 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f04f406b4d0 0x7f04f406d990 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.706+0000 7f0518f1d640 1 --2- 172.21.15.43:0/1703103861 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f05140b23f0 0x7f05141eef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.706+0000 7f0518f1d640 1 -- 172.21.15.43:0/1703103861 >> 172.21.15.43:0/1703103861 conn(0x7f05140d17f0 msgr2=0x7f05140b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:03.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.706+0000 7f0518f1d640 1 -- 172.21.15.43:0/1703103861 shutdown_connections 2024-04-25T09:02:03.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.706+0000 7f0518f1d640 1 -- 172.21.15.43:0/1703103861 wait complete. 2024-04-25T09:02:03.705 INFO:tasks.cephfs.filesystem.ceph_manager:{'b': 'paused'} 2024-04-25T09:02:03.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:03.705 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub resume 2024-04-25T09:02:03.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.820+0000 7f9e2b4fc640 1 Processor -- start 2024-04-25T09:02:03.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.820+0000 7f9e2b4fc640 1 -- start start 2024-04-25T09:02:03.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.821+0000 7f9e2b4fc640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9e2415e090 0x7f9e24161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.821+0000 7f9e2b4fc640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9e24161fe0 0x7f9e240b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.821+0000 7f9e2b4fc640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e24162610 con 0x7f9e240b5aa0 2024-04-25T09:02:03.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.821+0000 7f9e2b4fc640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e240b5f00 con 0x7f9e24161fe0 2024-04-25T09:02:03.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.821+0000 7f9e2b4fc640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e240d1da0 con 0x7f9e2415e090 2024-04-25T09:02:03.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.821+0000 7f9e29298640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9e2415e090 0x7f9e24161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.821+0000 7f9e29298640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9e2415e090 0x7f9e24161aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:53962/0 (socket says 172.21.15.43:53962) 2024-04-25T09:02:03.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.821+0000 7f9e29298640 1 -- 172.21.15.43:0/576149922 learned_addr learned my addr 172.21.15.43:0/576149922 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:03.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.821+0000 7f9e28a97640 1 --2- 172.21.15.43:0/576149922 >> v2:172.21.15.193:3300/0 conn(0x7f9e24161fe0 0x7f9e240b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.821+0000 7f9e28a97640 1 -- 172.21.15.43:0/576149922 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9e2415e090 msgr2=0x7f9e24161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:03.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.821+0000 7f9e28a97640 1 --2- 172.21.15.43:0/576149922 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9e2415e090 0x7f9e24161aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.821+0000 7f9e28a97640 1 -- 172.21.15.43:0/576149922 >> v1:172.21.15.193:6789/0 conn(0x7f9e240b5aa0 legacy=0x7f9e240ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:03.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.821+0000 7f9e1bfff640 1 -- 172.21.15.43:0/576149922 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1579360708 0 0) 0x7f9e24162610 con 0x7f9e240b5aa0 2024-04-25T09:02:03.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.821+0000 7f9e28a97640 1 -- 172.21.15.43:0/576149922 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e240d1750 con 0x7f9e24161fe0 2024-04-25T09:02:03.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.821+0000 7f9e29298640 1 --2- 172.21.15.43:0/576149922 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9e2415e090 0x7f9e24161aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:02:03.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.822+0000 7f9e28a97640 1 --2- 172.21.15.43:0/576149922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e24161fe0 0x7f9e240b5560 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9e140097a0 tx=0x7f9e1402a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5ffe3510d4df6e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:03.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.822+0000 7f9e1bfff640 1 -- 172.21.15.43:0/576149922 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e14037070 con 0x7f9e24161fe0 2024-04-25T09:02:03.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.822+0000 7f9e1bfff640 1 -- 172.21.15.43:0/576149922 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e14032440 con 0x7f9e24161fe0 2024-04-25T09:02:03.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.822+0000 7f9e1bfff640 1 -- 172.21.15.43:0/576149922 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e14032810 con 0x7f9e24161fe0 2024-04-25T09:02:03.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.823+0000 7f9e2b4fc640 1 -- 172.21.15.43:0/576149922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e24161fe0 msgr2=0x7f9e240b5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:03.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.823+0000 7f9e2b4fc640 1 --2- 172.21.15.43:0/576149922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e24161fe0 0x7f9e240b5560 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9e140097a0 tx=0x7f9e1402a210 comp rx=0 tx=0).stop 2024-04-25T09:02:03.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.823+0000 7f9e2b4fc640 1 -- 172.21.15.43:0/576149922 shutdown_connections 2024-04-25T09:02:03.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.823+0000 7f9e2b4fc640 1 --2- 172.21.15.43:0/576149922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e24161fe0 0x7f9e240b5560 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.823+0000 7f9e2b4fc640 1 --2- 172.21.15.43:0/576149922 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9e2415e090 0x7f9e24161aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.823+0000 7f9e2b4fc640 1 -- 172.21.15.43:0/576149922 >> 172.21.15.43:0/576149922 conn(0x7f9e240d0560 msgr2=0x7f9e240d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:03.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.823+0000 7f9e2b4fc640 1 -- 172.21.15.43:0/576149922 shutdown_connections 2024-04-25T09:02:03.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.823+0000 7f9e2b4fc640 1 -- 172.21.15.43:0/576149922 wait complete. 2024-04-25T09:02:03.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.824+0000 7f9e2b4fc640 1 Processor -- start 2024-04-25T09:02:03.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.824+0000 7f9e2b4fc640 1 -- start start 2024-04-25T09:02:03.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.825+0000 7f9e2b4fc640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9e240b5aa0 0x7f9e24152180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.825+0000 7f9e2b4fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e2415e090 0x7f9e2415fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.825+0000 7f9e2b4fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e24161fe0 0x7f9e24157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.825+0000 7f9e2b4fc640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e240b3280 con 0x7f9e2415e090 2024-04-25T09:02:03.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.825+0000 7f9e2b4fc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e241628f0 con 0x7f9e24161fe0 2024-04-25T09:02:03.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.825+0000 7f9e2b4fc640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e240d3240 con 0x7f9e240b5aa0 2024-04-25T09:02:03.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.825+0000 7f9e28a97640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e2415e090 0x7f9e2415fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.825+0000 7f9e28a97640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e2415e090 0x7f9e2415fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43132/0 (socket says 172.21.15.43:43132) 2024-04-25T09:02:03.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.825+0000 7f9e28a97640 1 -- 172.21.15.43:0/3580771471 learned_addr learned my addr 172.21.15.43:0/3580771471 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:03.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.825+0000 7f9e29298640 1 --2- 172.21.15.43:0/3580771471 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9e240b5aa0 0x7f9e24152180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.825+0000 7f9e29a99640 1 --2- 172.21.15.43:0/3580771471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e24161fe0 0x7f9e24157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.825+0000 7f9e28a97640 1 -- 172.21.15.43:0/3580771471 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9e240b5aa0 msgr2=0x7f9e24152180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:03.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.825+0000 7f9e28a97640 1 --2- 172.21.15.43:0/3580771471 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9e240b5aa0 0x7f9e24152180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.825+0000 7f9e28a97640 1 -- 172.21.15.43:0/3580771471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e24161fe0 msgr2=0x7f9e24157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:03.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.825+0000 7f9e28a97640 1 --2- 172.21.15.43:0/3580771471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e24161fe0 0x7f9e24157520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.825+0000 7f9e28a97640 1 -- 172.21.15.43:0/3580771471 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e241564c0 con 0x7f9e2415e090 2024-04-25T09:02:03.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.825+0000 7f9e28a97640 1 --2- 172.21.15.43:0/3580771471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e2415e090 0x7f9e2415fa90 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f9e140097a0 tx=0x7f9e140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:03.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.826+0000 7f9e19ffb640 1 -- 172.21.15.43:0/3580771471 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e14037050 con 0x7f9e2415e090 2024-04-25T09:02:03.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.826+0000 7f9e19ffb640 1 -- 172.21.15.43:0/3580771471 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e14038040 con 0x7f9e2415e090 2024-04-25T09:02:03.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.826+0000 7f9e2b4fc640 1 -- 172.21.15.43:0/3580771471 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e24156760 con 0x7f9e2415e090 2024-04-25T09:02:03.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.826+0000 7f9e19ffb640 1 -- 172.21.15.43:0/3580771471 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e14004490 con 0x7f9e2415e090 2024-04-25T09:02:03.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.826+0000 7f9e2b4fc640 1 -- 172.21.15.43:0/3580771471 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e24156d20 con 0x7f9e2415e090 2024-04-25T09:02:03.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.827+0000 7f9e19ffb640 1 -- 172.21.15.43:0/3580771471 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9e1403c6d0 con 0x7f9e2415e090 2024-04-25T09:02:03.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.828+0000 7f9e19ffb640 1 --2- 172.21.15.43:0/3580771471 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9e0806b4d0 0x7f9e0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.828+0000 7f9e19ffb640 1 -- 172.21.15.43:0/3580771471 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e140aad30 con 0x7f9e2415e090 2024-04-25T09:02:03.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.828+0000 7f9e29298640 1 --2- 172.21.15.43:0/3580771471 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9e0806b4d0 0x7f9e0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.829+0000 7f9e29298640 1 --2- 172.21.15.43:0/3580771471 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9e0806b4d0 0x7f9e0806d990 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f9e2415b7e0 tx=0x7f9e20008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:03.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.829+0000 7f9e2e1fe740 1 Processor -- start 2024-04-25T09:02:03.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.829+0000 7f9e2e1fe740 1 -- start start 2024-04-25T09:02:03.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.829+0000 7f9e2e1fe740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf50e70b10 0x55bf50e73320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.829+0000 7f9e2e1fe740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55bf50e70ef0 0x55bf50e73860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.829+0000 7f9e2e1fe740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf50e43770 0x55bf50e46f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.829+0000 7f9e2e1fe740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf50dacda0 con 0x55bf50e43770 2024-04-25T09:02:03.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.829+0000 7f9e2e1fe740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf50d55b80 con 0x55bf50e70b10 2024-04-25T09:02:03.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.829+0000 7f9e2e1fe740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf50da8e40 con 0x55bf50e70ef0 2024-04-25T09:02:03.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.829+0000 7f9e29a99640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf50e43770 0x55bf50e46f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.829+0000 7f9e29a99640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf50e43770 0x55bf50e46f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43140/0 (socket says 172.21.15.43:43140) 2024-04-25T09:02:03.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.830+0000 7f9e29a99640 1 -- 172.21.15.43:0/1525784994 learned_addr learned my addr 172.21.15.43:0/1525784994 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:03.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.830+0000 7f9e29298640 1 --2- 172.21.15.43:0/1525784994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf50e70b10 0x55bf50e73320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.830+0000 7f9e28a97640 1 --2- 172.21.15.43:0/1525784994 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55bf50e70ef0 0x55bf50e73860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.830+0000 7f9e29a99640 1 -- 172.21.15.43:0/1525784994 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55bf50e70ef0 msgr2=0x55bf50e73860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:03.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.830+0000 7f9e29a99640 1 --2- 172.21.15.43:0/1525784994 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55bf50e70ef0 0x55bf50e73860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.830+0000 7f9e29a99640 1 -- 172.21.15.43:0/1525784994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf50e70b10 msgr2=0x55bf50e73320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:03.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.830+0000 7f9e29a99640 1 --2- 172.21.15.43:0/1525784994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf50e70b10 0x55bf50e73320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.830+0000 7f9e29a99640 1 -- 172.21.15.43:0/1525784994 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf50d5ff10 con 0x55bf50e43770 2024-04-25T09:02:03.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.830+0000 7f9e29a99640 1 --2- 172.21.15.43:0/1525784994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf50e43770 0x55bf50e46f80 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c001ca0 tx=0x7f9e1c002160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa14b740e3c70e72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:03.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.830+0000 7f9e0effd640 1 -- 172.21.15.43:0/1525784994 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c019070 con 0x55bf50e43770 2024-04-25T09:02:03.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.830+0000 7f9e0effd640 1 -- 172.21.15.43:0/1525784994 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e1c003b80 con 0x55bf50e43770 2024-04-25T09:02:03.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.830+0000 7f9e0effd640 1 -- 172.21.15.43:0/1525784994 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1c00e860 con 0x55bf50e43770 2024-04-25T09:02:03.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.831+0000 7f9e2e1fe740 1 -- 172.21.15.43:0/1525784994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf50e43770 msgr2=0x55bf50e46f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:03.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.831+0000 7f9e2e1fe740 1 --2- 172.21.15.43:0/1525784994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf50e43770 0x55bf50e46f80 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c001ca0 tx=0x7f9e1c002160 comp rx=0 tx=0).stop 2024-04-25T09:02:03.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.831+0000 7f9e2e1fe740 1 -- 172.21.15.43:0/1525784994 shutdown_connections 2024-04-25T09:02:03.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.831+0000 7f9e2e1fe740 1 --2- 172.21.15.43:0/1525784994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf50e43770 0x55bf50e46f80 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.831+0000 7f9e2e1fe740 1 --2- 172.21.15.43:0/1525784994 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55bf50e70ef0 0x55bf50e73860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.831+0000 7f9e2e1fe740 1 --2- 172.21.15.43:0/1525784994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf50e70b10 0x55bf50e73320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.831+0000 7f9e2e1fe740 1 -- 172.21.15.43:0/1525784994 >> 172.21.15.43:0/1525784994 conn(0x55bf50e459e0 msgr2=0x55bf50e8e320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:03.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.831+0000 7f9e2e1fe740 1 -- 172.21.15.43:0/1525784994 shutdown_connections 2024-04-25T09:02:03.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.831+0000 7f9e2e1fe740 1 -- 172.21.15.43:0/1525784994 wait complete. 2024-04-25T09:02:03.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.831+0000 7f9e2e1fe740 1 -- start start 2024-04-25T09:02:03.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.831+0000 7f9e2e1fe740 1 Processor -- start 2024-04-25T09:02:03.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.832+0000 7f9e2e1fe740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf50e70b10 0x55bf50e6eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.832+0000 7f9e2e1fe740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf50e70ef0 0x55bf50e90ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.832+0000 7f9e2e1fe740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55bf50e6d2e0 0x55bf50e91df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.832+0000 7f9e2e1fe740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf50e48b90 con 0x55bf50e70ef0 2024-04-25T09:02:03.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.832+0000 7f9e2e1fe740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf50d4b4b0 con 0x55bf50e70b10 2024-04-25T09:02:03.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.832+0000 7f9e2e1fe740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf50e48e70 con 0x55bf50e6d2e0 2024-04-25T09:02:03.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.832+0000 7f9e2e1fe740 10 client.0 _ll_register_callbacks cb 0x55bf50d8b180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:02:03.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.832+0000 7f9e28a97640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf50e70ef0 0x55bf50e90ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.832+0000 7f9e28a97640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf50e70ef0 0x55bf50e90ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43142/0 (socket says 172.21.15.43:43142) 2024-04-25T09:02:03.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.832+0000 7f9e28a97640 1 -- 172.21.15.43:0/3179362853 learned_addr learned my addr 172.21.15.43:0/3179362853 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:03.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.832+0000 7f9e29298640 1 --2- 172.21.15.43:0/3179362853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf50e70b10 0x55bf50e6eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.833+0000 7f9e29a99640 1 --2- 172.21.15.43:0/3179362853 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55bf50e6d2e0 0x55bf50e91df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.833+0000 7f9e28a97640 1 -- 172.21.15.43:0/3179362853 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55bf50e6d2e0 msgr2=0x55bf50e91df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:03.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.833+0000 7f9e28a97640 1 --2- 172.21.15.43:0/3179362853 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55bf50e6d2e0 0x55bf50e91df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.833+0000 7f9e28a97640 1 -- 172.21.15.43:0/3179362853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf50e70b10 msgr2=0x55bf50e6eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:03.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.833+0000 7f9e28a97640 1 --2- 172.21.15.43:0/3179362853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf50e70b10 0x55bf50e6eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.833+0000 7f9e28a97640 1 -- 172.21.15.43:0/3179362853 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bf50b8b7d0 con 0x55bf50e70ef0 2024-04-25T09:02:03.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.833+0000 7f9e28a97640 1 --2- 172.21.15.43:0/3179362853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf50e70ef0 0x55bf50e90ba0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f9e140793a0 tx=0x7f9e14079bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e2798557a9ed62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:03.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.833+0000 7f9e03fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:02:03.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.833+0000 7f9e03fff640 1 -- 172.21.15.43:0/3179362853 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e140a7060 con 0x55bf50e70ef0 2024-04-25T09:02:03.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.833+0000 7f9e03fff640 1 -- 172.21.15.43:0/3179362853 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e14081890 con 0x55bf50e70ef0 2024-04-25T09:02:03.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.833+0000 7f9e2e1fe740 1 -- 172.21.15.43:0/3179362853 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9e1c003b80 con 0x55bf50e70ef0 2024-04-25T09:02:03.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.833+0000 7f9e03fff640 1 -- 172.21.15.43:0/3179362853 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e14081b20 con 0x55bf50e70ef0 2024-04-25T09:02:03.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.833+0000 7f9e2e1fe740 10 client.8388 fetch_fsmap learned FSMap version 61 2024-04-25T09:02:03.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.833+0000 7f9e2e1fe740 1 -- 172.21.15.43:0/3179362853 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x55bf50ac13f0 con 0x55bf50e70ef0 2024-04-25T09:02:03.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.833+0000 7f9e03fff640 1 -- 172.21.15.43:0/3179362853 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e1c003b80 con 0x55bf50e70ef0 2024-04-25T09:02:03.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.833+0000 7f9e03fff640 1 -- 172.21.15.43:0/3179362853 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 61) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9e14095020 con 0x55bf50e70ef0 2024-04-25T09:02:03.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.833+0000 7f9e2e1fe740 10 client.8388 fetch_fsmap finished waiting for FSMap version 61 2024-04-25T09:02:03.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.833+0000 7f9e2e1fe740 10 client.8388 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-25T09:02:03.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.833+0000 7f9e2e1fe740 20 client.8388 populate_metadata read hostname 'smithi043' 2024-04-25T09:02:03.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.834+0000 7f9e2e1fe740 1 --2- 172.21.15.43:0/3179362853 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55bf50e67f90 0x55bf50e8f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.834+0000 7f9e2e1fe740 4 client.8388 mds_command: new command op to 8094 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:02:03.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.834+0000 7f9e2e1fe740 1 -- 172.21.15.43:0/3179362853 --> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bf50dba2e0 con 0x55bf50e67f90 2024-04-25T09:02:03.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.834+0000 7f9e29298640 1 --2- 172.21.15.43:0/3179362853 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55bf50e67f90 0x55bf50e8f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.834+0000 7f9e03fff640 10 client.8388 ms_handle_connect on v2:172.21.15.193:6834/853943267 2024-04-25T09:02:03.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.834+0000 7f9e29298640 1 --2- 172.21.15.43:0/3179362853 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55bf50e67f90 0x55bf50e8f450 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50eccd0b9161816e server_cookie=ef790173a4bc4da9 in_seq=0 out_seq=0 2024-04-25T09:02:03.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.835+0000 7f9e03fff640 1 -- 172.21.15.43:0/3179362853 <== mds.0 v2:172.21.15.193:6834/853943267 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bf50dba2e0 con 0x55bf50e67f90 2024-04-25T09:02:03.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.835+0000 7f9e03fff640 10 client.8388 handle_command_reply: tid=0 2024-04-25T09:02:03.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.835+0000 7f9e2e1fe740 1 client.8388 shutdown 2024-04-25T09:02:03.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.835+0000 7f9e2e1fe740 1 -- 172.21.15.43:0/3179362853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf50e70ef0 msgr2=0x55bf50e90ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:03.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.835+0000 7f9e2e1fe740 1 --2- 172.21.15.43:0/3179362853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf50e70ef0 0x55bf50e90ba0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f9e140793a0 tx=0x7f9e14079bd0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.835+0000 7f9e2e1fe740 1 -- 172.21.15.43:0/3179362853 shutdown_connections 2024-04-25T09:02:03.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.835+0000 7f9e2e1fe740 1 --2- 172.21.15.43:0/3179362853 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55bf50e6d2e0 0x55bf50e91df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.835+0000 7f9e2e1fe740 1 --2- 172.21.15.43:0/3179362853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf50e70ef0 0x55bf50e90ba0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.835+0000 7f9e2e1fe740 1 --2- 172.21.15.43:0/3179362853 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55bf50e67f90 0x55bf50e8f450 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.835+0000 7f9e2e1fe740 1 --2- 172.21.15.43:0/3179362853 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf50e70b10 0x55bf50e6eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.835+0000 7f9e2e1fe740 1 -- 172.21.15.43:0/3179362853 >> 172.21.15.43:0/3179362853 conn(0x55bf50e43770 msgr2=0x55bf50e6f190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:03.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.835+0000 7f9e03fff640 0 client.8388 ms_handle_reset on v2:172.21.15.193:6834/853943267 2024-04-25T09:02:03.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.835+0000 7f9e2e1fe740 1 -- 172.21.15.43:0/3179362853 shutdown_connections 2024-04-25T09:02:03.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.835+0000 7f9e2e1fe740 1 -- 172.21.15.43:0/3179362853 wait complete. 2024-04-25T09:02:03.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.835+0000 7f9e2e1fe740 20 client.8388 trim_cache size 0 max 16384 2024-04-25T09:02:03.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.850+0000 7f9e2e1fe740 1 Processor -- start 2024-04-25T09:02:03.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.850+0000 7f9e2e1fe740 1 -- start start 2024-04-25T09:02:03.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.851+0000 7f9e2e1fe740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf50e6d2e0 0x55bf50e701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.851+0000 7f9e2e1fe740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf50e70b10 0x55bf50e77470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.851+0000 7f9e2e1fe740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55bf50e70ef0 0x55bf50e67f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.851+0000 7f9e2e1fe740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e241ffb60 con 0x55bf50e6d2e0 2024-04-25T09:02:03.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.851+0000 7f9e2e1fe740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e241f7ae0 con 0x55bf50e70b10 2024-04-25T09:02:03.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.851+0000 7f9e2e1fe740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e241f6ee0 con 0x55bf50e70ef0 2024-04-25T09:02:03.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.851+0000 7f9e29298640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf50e6d2e0 0x55bf50e701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.851+0000 7f9e29298640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf50e6d2e0 0x55bf50e701c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43156/0 (socket says 172.21.15.43:43156) 2024-04-25T09:02:03.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.851+0000 7f9e29298640 1 -- 172.21.15.43:0/2920431394 learned_addr learned my addr 172.21.15.43:0/2920431394 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:03.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.851+0000 7f9e28a97640 1 --2- 172.21.15.43:0/2920431394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf50e70b10 0x55bf50e77470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.851+0000 7f9e29a99640 1 --2- 172.21.15.43:0/2920431394 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55bf50e70ef0 0x55bf50e67f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.851+0000 7f9e29298640 1 -- 172.21.15.43:0/2920431394 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55bf50e70ef0 msgr2=0x55bf50e67f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:03.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.851+0000 7f9e29298640 1 --2- 172.21.15.43:0/2920431394 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55bf50e70ef0 0x55bf50e67f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.851+0000 7f9e29298640 1 -- 172.21.15.43:0/2920431394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf50e70b10 msgr2=0x55bf50e77470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:03.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.851+0000 7f9e29298640 1 --2- 172.21.15.43:0/2920431394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf50e70b10 0x55bf50e77470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.851+0000 7f9e29298640 1 -- 172.21.15.43:0/2920431394 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf50a810f0 con 0x55bf50e6d2e0 2024-04-25T09:02:03.842 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.851+0000 7f9e29a99640 1 --2- 172.21.15.43:0/2920431394 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55bf50e70ef0 0x55bf50e67f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:03.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.851+0000 7f9e29298640 1 --2- 172.21.15.43:0/2920431394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf50e6d2e0 0x55bf50e701c0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f9e20011f00 tx=0x7f9e20017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a6d3cd089debe1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:03.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.852+0000 7f9e0e7fc640 1 -- 172.21.15.43:0/2920431394 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2002abc0 con 0x55bf50e6d2e0 2024-04-25T09:02:03.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.852+0000 7f9e0e7fc640 1 -- 172.21.15.43:0/2920431394 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e2002f070 con 0x55bf50e6d2e0 2024-04-25T09:02:03.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.852+0000 7f9e0e7fc640 1 -- 172.21.15.43:0/2920431394 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e2002a2c0 con 0x55bf50e6d2e0 2024-04-25T09:02:03.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.852+0000 7f9e2e1fe740 1 -- 172.21.15.43:0/2920431394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf50e6d2e0 msgr2=0x55bf50e701c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:03.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.852+0000 7f9e2e1fe740 1 --2- 172.21.15.43:0/2920431394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf50e6d2e0 0x55bf50e701c0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f9e20011f00 tx=0x7f9e20017000 comp rx=0 tx=0).stop 2024-04-25T09:02:03.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.852+0000 7f9e2e1fe740 1 -- 172.21.15.43:0/2920431394 shutdown_connections 2024-04-25T09:02:03.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.852+0000 7f9e2e1fe740 1 --2- 172.21.15.43:0/2920431394 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55bf50e70ef0 0x55bf50e67f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.852+0000 7f9e2e1fe740 1 --2- 172.21.15.43:0/2920431394 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf50e70b10 0x55bf50e77470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.852+0000 7f9e2e1fe740 1 --2- 172.21.15.43:0/2920431394 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf50e6d2e0 0x55bf50e701c0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.852+0000 7f9e2e1fe740 1 -- 172.21.15.43:0/2920431394 >> 172.21.15.43:0/2920431394 conn(0x55bf50e43770 msgr2=0x55bf50e49810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:03.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.852+0000 7f9e2e1fe740 1 -- 172.21.15.43:0/2920431394 shutdown_connections 2024-04-25T09:02:03.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.852+0000 7f9e2e1fe740 1 -- 172.21.15.43:0/2920431394 wait complete. 2024-04-25T09:02:03.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.853+0000 7f9e2e1fe740 1 -- start start 2024-04-25T09:02:03.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.853+0000 7f9e2e1fe740 1 Processor -- start 2024-04-25T09:02:03.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.854+0000 7f9e2e1fe740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf50e6d2e0 0x55bf50ef84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.854+0000 7f9e2e1fe740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf50e70b10 0x55bf50e46e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.854+0000 7f9e2e1fe740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55bf50e70ef0 0x55bf50e473c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.854+0000 7f9e2e1fe740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e241f74e0 con 0x55bf50e70b10 2024-04-25T09:02:03.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.854+0000 7f9e2e1fe740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf50d6e4b0 con 0x55bf50e6d2e0 2024-04-25T09:02:03.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.854+0000 7f9e2e1fe740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bf50e90900 con 0x55bf50e70ef0 2024-04-25T09:02:03.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.854+0000 7f9e2e1fe740 10 client.0 _ll_register_callbacks cb 0x7f9e1c0016d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:02:03.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.854+0000 7f9e28a97640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf50e70b10 0x55bf50e46e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.854+0000 7f9e28a97640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf50e70b10 0x55bf50e46e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43164/0 (socket says 172.21.15.43:43164) 2024-04-25T09:02:03.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.854+0000 7f9e28a97640 1 -- 172.21.15.43:0/2458963407 learned_addr learned my addr 172.21.15.43:0/2458963407 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:03.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.854+0000 7f9e29a99640 1 --2- 172.21.15.43:0/2458963407 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55bf50e70ef0 0x55bf50e473c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.854+0000 7f9e29298640 1 --2- 172.21.15.43:0/2458963407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf50e6d2e0 0x55bf50ef84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.854+0000 7f9e28a97640 1 -- 172.21.15.43:0/2458963407 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55bf50e70ef0 msgr2=0x55bf50e473c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:03.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.854+0000 7f9e28a97640 1 --2- 172.21.15.43:0/2458963407 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55bf50e70ef0 0x55bf50e473c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.854+0000 7f9e28a97640 1 -- 172.21.15.43:0/2458963407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf50e6d2e0 msgr2=0x55bf50ef84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:03.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.854+0000 7f9e28a97640 1 --2- 172.21.15.43:0/2458963407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf50e6d2e0 0x55bf50ef84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.854+0000 7f9e28a97640 1 -- 172.21.15.43:0/2458963407 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bf50e8eeb0 con 0x55bf50e70b10 2024-04-25T09:02:03.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.854+0000 7f9e28a97640 1 --2- 172.21.15.43:0/2458963407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf50e70b10 0x55bf50e46e80 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f9e14075d90 tx=0x7f9e140a9090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c036a73787611232 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:03.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.854+0000 7f9e03fff640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:02:03.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.854+0000 7f9e03fff640 1 -- 172.21.15.43:0/2458963407 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e14078d70 con 0x55bf50e70b10 2024-04-25T09:02:03.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.855+0000 7f9e03fff640 1 -- 172.21.15.43:0/2458963407 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e1403bc10 con 0x55bf50e70b10 2024-04-25T09:02:03.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.855+0000 7f9e2e1fe740 1 -- 172.21.15.43:0/2458963407 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9e2002f070 con 0x55bf50e70b10 2024-04-25T09:02:03.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.855+0000 7f9e03fff640 1 -- 172.21.15.43:0/2458963407 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e140a7030 con 0x55bf50e70b10 2024-04-25T09:02:03.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.855+0000 7f9e2e1fe740 10 client.8394 fetch_fsmap learned FSMap version 61 2024-04-25T09:02:03.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.855+0000 7f9e2e1fe740 1 -- 172.21.15.43:0/2458963407 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x55bf50e8eeb0 con 0x55bf50e70b10 2024-04-25T09:02:03.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.855+0000 7f9e03fff640 1 -- 172.21.15.43:0/2458963407 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e2002f070 con 0x55bf50e70b10 2024-04-25T09:02:03.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.855+0000 7f9e03fff640 1 -- 172.21.15.43:0/2458963407 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 61) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9e14095070 con 0x55bf50e70b10 2024-04-25T09:02:03.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.855+0000 7f9e2e1fe740 10 client.8394 fetch_fsmap finished waiting for FSMap version 61 2024-04-25T09:02:03.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.855+0000 7f9e2e1fe740 10 client.8394 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-25T09:02:03.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.855+0000 7f9e2e1fe740 20 client.8394 populate_metadata read hostname 'smithi043' 2024-04-25T09:02:03.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.855+0000 7f9e2e1fe740 1 --2- 172.21.15.43:0/2458963407 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55bf50e43e40 0x55bf50e44220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.855+0000 7f9e2e1fe740 4 client.8394 mds_command: new command op to 8094 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-04-25T09:02:03.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.855+0000 7f9e2e1fe740 1 -- 172.21.15.43:0/2458963407 --> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] -- command(tid 0: {"prefix": "scrub resume"}) v1 -- 0x55bf50db4630 con 0x55bf50e43e40 2024-04-25T09:02:03.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.855+0000 7f9e29298640 1 --2- 172.21.15.43:0/2458963407 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55bf50e43e40 0x55bf50e44220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.856+0000 7f9e29298640 1 --2- 172.21.15.43:0/2458963407 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55bf50e43e40 0x55bf50e44220 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ef5cba2fa4209ff server_cookie=2d00ef0c7c993093 in_seq=0 out_seq=0 2024-04-25T09:02:03.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.856+0000 7f9e03fff640 10 client.8394 ms_handle_connect on v2:172.21.15.193:6834/853943267 2024-04-25T09:02:03.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.856+0000 7f9e03fff640 1 -- 172.21.15.43:0/2458963407 <== mds.0 v2:172.21.15.193:6834/853943267 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55bf50db4630 con 0x55bf50e43e40 2024-04-25T09:02:03.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.856+0000 7f9e03fff640 10 client.8394 handle_command_reply: tid=0 2024-04-25T09:02:03.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.856+0000 7f9e2e1fe740 1 client.8394 shutdown 2024-04-25T09:02:03.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.856+0000 7f9e2e1fe740 1 -- 172.21.15.43:0/2458963407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf50e70b10 msgr2=0x55bf50e46e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:03.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.856+0000 7f9e2e1fe740 1 --2- 172.21.15.43:0/2458963407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf50e70b10 0x55bf50e46e80 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f9e14075d90 tx=0x7f9e140a9090 comp rx=0 tx=0).stop 2024-04-25T09:02:03.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.856+0000 7f9e2e1fe740 1 -- 172.21.15.43:0/2458963407 shutdown_connections 2024-04-25T09:02:03.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.856+0000 7f9e2e1fe740 1 --2- 172.21.15.43:0/2458963407 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55bf50e70ef0 0x55bf50e473c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.856+0000 7f9e2e1fe740 1 --2- 172.21.15.43:0/2458963407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55bf50e70b10 0x55bf50e46e80 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.856+0000 7f9e2e1fe740 1 --2- 172.21.15.43:0/2458963407 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55bf50e43e40 0x55bf50e44220 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.856+0000 7f9e2e1fe740 1 --2- 172.21.15.43:0/2458963407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55bf50e6d2e0 0x55bf50ef84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.856+0000 7f9e03fff640 0 client.8394 ms_handle_reset on v2:172.21.15.193:6834/853943267 2024-04-25T09:02:03.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.856+0000 7f9e2e1fe740 1 -- 172.21.15.43:0/2458963407 >> 172.21.15.43:0/2458963407 conn(0x55bf50e43770 msgr2=0x55bf50e700a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:03.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.856+0000 7f9e2e1fe740 1 -- 172.21.15.43:0/2458963407 shutdown_connections 2024-04-25T09:02:03.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.856+0000 7f9e2e1fe740 1 -- 172.21.15.43:0/2458963407 wait complete. 2024-04-25T09:02:03.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.856+0000 7f9e2e1fe740 20 client.8394 trim_cache size 0 max 16384 2024-04-25T09:02:03.851 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:02:03.851 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0 2024-04-25T09:02:03.851 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:02:03.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.857+0000 7f9e2b4fc640 1 -- 172.21.15.43:0/3580771471 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9e0806b4d0 msgr2=0x7f9e0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:03.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.857+0000 7f9e2b4fc640 1 --2- 172.21.15.43:0/3580771471 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9e0806b4d0 0x7f9e0806d990 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f9e2415b7e0 tx=0x7f9e20008040 comp rx=0 tx=0).stop 2024-04-25T09:02:03.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.857+0000 7f9e2b4fc640 1 -- 172.21.15.43:0/3580771471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e2415e090 msgr2=0x7f9e2415fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:03.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.857+0000 7f9e2b4fc640 1 --2- 172.21.15.43:0/3580771471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e2415e090 0x7f9e2415fa90 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f9e140097a0 tx=0x7f9e140046c0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.857+0000 7f9e2b4fc640 1 -- 172.21.15.43:0/3580771471 shutdown_connections 2024-04-25T09:02:03.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.857+0000 7f9e2b4fc640 1 --2- 172.21.15.43:0/3580771471 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f9e0806b4d0 0x7f9e0806d990 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.857+0000 7f9e2b4fc640 1 --2- 172.21.15.43:0/3580771471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9e24161fe0 0x7f9e24157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.857+0000 7f9e2b4fc640 1 --2- 172.21.15.43:0/3580771471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9e2415e090 0x7f9e2415fa90 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.857+0000 7f9e2b4fc640 1 --2- 172.21.15.43:0/3580771471 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9e240b5aa0 0x7f9e24152180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.857+0000 7f9e2b4fc640 1 -- 172.21.15.43:0/3580771471 >> 172.21.15.43:0/3580771471 conn(0x7f9e240d0560 msgr2=0x7f9e2415bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:03.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.857+0000 7f9e2b4fc640 1 -- 172.21.15.43:0/3580771471 shutdown_connections 2024-04-25T09:02:03.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.857+0000 7f9e2b4fc640 1 -- 172.21.15.43:0/3580771471 wait complete. 2024-04-25T09:02:03.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:03.856 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T09:02:03.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.970+0000 7fb023c38640 1 Processor -- start 2024-04-25T09:02:03.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.970+0000 7fb023c38640 1 -- start start 2024-04-25T09:02:03.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.971+0000 7fb023c38640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb01c0d0100 0x7fb01c0ae460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.971+0000 7fb023c38640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb01c06d690 con 0x7fb01c0cdb00 2024-04-25T09:02:03.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.971+0000 7fb023c38640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb01c0cef10 con 0x7fb01c0cfd20 2024-04-25T09:02:03.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.971+0000 7fb023c38640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb01c0cecd0 con 0x7fb01c0d0100 2024-04-25T09:02:03.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.971+0000 7fb0221d5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb01c0d0100 0x7fb01c0ae460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.971+0000 7fb0211d3640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb01c0cdb00 0x7fb01c06de50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48728/0 (socket says 172.21.15.43:48728) 2024-04-25T09:02:03.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.971+0000 7fb0211d3640 1 -- 172.21.15.43:0/2383406684 learned_addr learned my addr 172.21.15.43:0/2383406684 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:03.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.971+0000 7fb0221d5640 1 -- 172.21.15.43:0/2383406684 >> v1:172.21.15.193:6789/0 conn(0x7fb01c0cfd20 legacy=0x7fb01c0ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:03.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.971+0000 7fb0221d5640 1 -- 172.21.15.43:0/2383406684 >> v1:172.21.15.43:6789/0 conn(0x7fb01c0cdb00 legacy=0x7fb01c06de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:03.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.971+0000 7fb0221d5640 1 -- 172.21.15.43:0/2383406684 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb01c0cf1f0 con 0x7fb01c0d0100 2024-04-25T09:02:03.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.971+0000 7fb0209d2640 1 -- 172.21.15.43:0/2383406684 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3453181001 0 0) 0x7fb01c06d690 con 0x7fb01c0cdb00 2024-04-25T09:02:03.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.972+0000 7fb0221d5640 1 --2- 172.21.15.43:0/2383406684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb01c0d0100 0x7fb01c0ae460 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fb0140097a0 tx=0x7fb01402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2af4621e78daa7ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:03.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.972+0000 7fb0209d2640 1 -- 172.21.15.43:0/2383406684 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb014037070 con 0x7fb01c0d0100 2024-04-25T09:02:03.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.972+0000 7fb0209d2640 1 -- 172.21.15.43:0/2383406684 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb014032440 con 0x7fb01c0d0100 2024-04-25T09:02:03.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.972+0000 7fb0209d2640 1 -- 172.21.15.43:0/2383406684 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb014032810 con 0x7fb01c0d0100 2024-04-25T09:02:03.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.973+0000 7fb023c38640 1 -- 172.21.15.43:0/2383406684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb01c0d0100 msgr2=0x7fb01c0ae460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:03.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.973+0000 7fb023c38640 1 --2- 172.21.15.43:0/2383406684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb01c0d0100 0x7fb01c0ae460 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fb0140097a0 tx=0x7fb01402a230 comp rx=0 tx=0).stop 2024-04-25T09:02:03.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.973+0000 7fb023c38640 1 -- 172.21.15.43:0/2383406684 shutdown_connections 2024-04-25T09:02:03.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.973+0000 7fb023c38640 1 --2- 172.21.15.43:0/2383406684 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb01c0d0100 0x7fb01c0ae460 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.973+0000 7fb023c38640 1 -- 172.21.15.43:0/2383406684 >> 172.21.15.43:0/2383406684 conn(0x7fb01c06ab30 msgr2=0x7fb01c06af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:03.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.973+0000 7fb023c38640 1 -- 172.21.15.43:0/2383406684 shutdown_connections 2024-04-25T09:02:03.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.973+0000 7fb023c38640 1 -- 172.21.15.43:0/2383406684 wait complete. 2024-04-25T09:02:03.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.974+0000 7fb023c38640 1 Processor -- start 2024-04-25T09:02:03.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.974+0000 7fb023c38640 1 -- start start 2024-04-25T09:02:03.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.975+0000 7fb023c38640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb01c0cdb00 0x7fb01c071780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.975+0000 7fb023c38640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb01c0cfd20 0x7fb01c071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.975+0000 7fb023c38640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb01c0d0100 0x7fb01c1f2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.975+0000 7fb023c38640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c0cf4d0 con 0x7fb01c0d0100 2024-04-25T09:02:03.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.975+0000 7fb023c38640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c06e560 con 0x7fb01c0cfd20 2024-04-25T09:02:03.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.975+0000 7fb023c38640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c0cf650 con 0x7fb01c0cdb00 2024-04-25T09:02:03.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.975+0000 7fb0221d5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb01c0d0100 0x7fb01c1f2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.975+0000 7fb0221d5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb01c0d0100 0x7fb01c1f2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43178/0 (socket says 172.21.15.43:43178) 2024-04-25T09:02:03.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.975+0000 7fb0221d5640 1 -- 172.21.15.43:0/3880806255 learned_addr learned my addr 172.21.15.43:0/3880806255 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:03.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.975+0000 7fb0211d3640 1 --2- 172.21.15.43:0/3880806255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb01c0cfd20 0x7fb01c071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.975+0000 7fb0219d4640 1 --2- 172.21.15.43:0/3880806255 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb01c0cdb00 0x7fb01c071780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.975+0000 7fb0221d5640 1 -- 172.21.15.43:0/3880806255 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb01c0cdb00 msgr2=0x7fb01c071780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:03.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.975+0000 7fb0221d5640 1 --2- 172.21.15.43:0/3880806255 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb01c0cdb00 0x7fb01c071780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.975+0000 7fb0221d5640 1 -- 172.21.15.43:0/3880806255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb01c0cfd20 msgr2=0x7fb01c071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:03.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.975+0000 7fb0221d5640 1 --2- 172.21.15.43:0/3880806255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb01c0cfd20 0x7fb01c071cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.975+0000 7fb0221d5640 1 -- 172.21.15.43:0/3880806255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb01c1f3f90 con 0x7fb01c0d0100 2024-04-25T09:02:03.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.975+0000 7fb0211d3640 1 --2- 172.21.15.43:0/3880806255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb01c0cfd20 0x7fb01c071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:03.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.975+0000 7fb0221d5640 1 --2- 172.21.15.43:0/3880806255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb01c0d0100 0x7fb01c1f2f20 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fb014035040 tx=0x7fb0140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:03.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.976+0000 7fb01a7fc640 1 -- 172.21.15.43:0/3880806255 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb014037050 con 0x7fb01c0d0100 2024-04-25T09:02:03.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.976+0000 7fb01a7fc640 1 -- 172.21.15.43:0/3880806255 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb014038040 con 0x7fb01c0d0100 2024-04-25T09:02:03.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.976+0000 7fb023c38640 1 -- 172.21.15.43:0/3880806255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb01c1f4270 con 0x7fb01c0d0100 2024-04-25T09:02:03.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.976+0000 7fb01a7fc640 1 -- 172.21.15.43:0/3880806255 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb014004490 con 0x7fb01c0d0100 2024-04-25T09:02:03.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.976+0000 7fb023c38640 1 -- 172.21.15.43:0/3880806255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb01c1f6d30 con 0x7fb01c0d0100 2024-04-25T09:02:03.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.978+0000 7fb01a7fc640 1 -- 172.21.15.43:0/3880806255 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb014030980 con 0x7fb01c0d0100 2024-04-25T09:02:03.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.978+0000 7fb01a7fc640 1 --2- 172.21.15.43:0/3880806255 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb00806b4d0 0x7fb00806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.978+0000 7fb01a7fc640 1 -- 172.21.15.43:0/3880806255 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb0140aaef0 con 0x7fb01c0d0100 2024-04-25T09:02:03.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.978+0000 7fb0219d4640 1 --2- 172.21.15.43:0/3880806255 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb00806b4d0 0x7fb00806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.979+0000 7fb0219d4640 1 --2- 172.21.15.43:0/3880806255 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb00806b4d0 0x7fb00806d990 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb00c007fd0 tx=0x7fb00c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:03.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.979+0000 7fb026a11740 1 Processor -- start 2024-04-25T09:02:03.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.979+0000 7fb026a11740 1 -- start start 2024-04-25T09:02:03.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.980+0000 7fb026a11740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7620e0b10 0x55d7620e3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.980+0000 7fb026a11740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7620e0ef0 0x55d7620e3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.980+0000 7fb026a11740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55d7620b3770 0x55d7620b6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.980+0000 7fb026a11740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d76201cda0 con 0x55d7620e0b10 2024-04-25T09:02:03.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.980+0000 7fb026a11740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d761fc5b80 con 0x55d7620e0ef0 2024-04-25T09:02:03.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.980+0000 7fb026a11740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d762018e40 con 0x55d7620b3770 2024-04-25T09:02:03.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.980+0000 7fb0219d4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7620e0b10 0x55d7620e3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.980+0000 7fb0219d4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7620e0b10 0x55d7620e3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43182/0 (socket says 172.21.15.43:43182) 2024-04-25T09:02:03.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.980+0000 7fb0219d4640 1 -- 172.21.15.43:0/2940346310 learned_addr learned my addr 172.21.15.43:0/2940346310 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:03.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.980+0000 7fb0211d3640 1 --2- 172.21.15.43:0/2940346310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7620e0ef0 0x55d7620e3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.980+0000 7fb0221d5640 1 --2- 172.21.15.43:0/2940346310 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55d7620b3770 0x55d7620b6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.980+0000 7fb0219d4640 1 -- 172.21.15.43:0/2940346310 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55d7620b3770 msgr2=0x55d7620b6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:03.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.980+0000 7fb0219d4640 1 --2- 172.21.15.43:0/2940346310 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55d7620b3770 0x55d7620b6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.980+0000 7fb0219d4640 1 -- 172.21.15.43:0/2940346310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7620e0ef0 msgr2=0x55d7620e3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:03.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.980+0000 7fb0219d4640 1 --2- 172.21.15.43:0/2940346310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7620e0ef0 0x55d7620e3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.980+0000 7fb0219d4640 1 -- 172.21.15.43:0/2940346310 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d761fcff10 con 0x55d7620e0b10 2024-04-25T09:02:03.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.980+0000 7fb0221d5640 1 --2- 172.21.15.43:0/2940346310 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55d7620b3770 0x55d7620b6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:03.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.980+0000 7fb0219d4640 1 --2- 172.21.15.43:0/2940346310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7620e0b10 0x55d7620e3320 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fb00c00dbe0 tx=0x7fb00c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=670518bf63249620 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:03.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.980+0000 7fb0037fe640 1 -- 172.21.15.43:0/2940346310 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c003970 con 0x55d7620e0b10 2024-04-25T09:02:03.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.980+0000 7fb0037fe640 1 -- 172.21.15.43:0/2940346310 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb00c003b10 con 0x55d7620e0b10 2024-04-25T09:02:03.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.980+0000 7fb0037fe640 1 -- 172.21.15.43:0/2940346310 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00c005110 con 0x55d7620e0b10 2024-04-25T09:02:03.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.981+0000 7fb026a11740 1 -- 172.21.15.43:0/2940346310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7620e0b10 msgr2=0x55d7620e3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:03.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.981+0000 7fb026a11740 1 --2- 172.21.15.43:0/2940346310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7620e0b10 0x55d7620e3320 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7fb00c00dbe0 tx=0x7fb00c010000 comp rx=0 tx=0).stop 2024-04-25T09:02:03.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.981+0000 7fb026a11740 1 -- 172.21.15.43:0/2940346310 shutdown_connections 2024-04-25T09:02:03.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.981+0000 7fb026a11740 1 --2- 172.21.15.43:0/2940346310 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55d7620b3770 0x55d7620b6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.981+0000 7fb026a11740 1 --2- 172.21.15.43:0/2940346310 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7620e0ef0 0x55d7620e3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.981+0000 7fb026a11740 1 --2- 172.21.15.43:0/2940346310 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7620e0b10 0x55d7620e3320 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.981+0000 7fb026a11740 1 -- 172.21.15.43:0/2940346310 >> 172.21.15.43:0/2940346310 conn(0x55d7620b59e0 msgr2=0x55d7620fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:03.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.981+0000 7fb026a11740 1 -- 172.21.15.43:0/2940346310 shutdown_connections 2024-04-25T09:02:03.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.981+0000 7fb026a11740 1 -- 172.21.15.43:0/2940346310 wait complete. 2024-04-25T09:02:03.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.982+0000 7fb026a11740 1 -- start start 2024-04-25T09:02:03.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.982+0000 7fb026a11740 1 Processor -- start 2024-04-25T09:02:03.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.983+0000 7fb026a11740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55d7620e0b10 0x55d7620fa7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.983+0000 7fb026a11740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7620e0ef0 0x55d7620f8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.983+0000 7fb026a11740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7620f8e90 0x55d7620f9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.983+0000 7fb026a11740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d7620b8b90 con 0x55d7620f8e90 2024-04-25T09:02:03.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.983+0000 7fb026a11740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d761fbb4b0 con 0x55d7620e0ef0 2024-04-25T09:02:03.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.983+0000 7fb026a11740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d7620b8e70 con 0x55d7620e0b10 2024-04-25T09:02:03.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.983+0000 7fb0221d5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7620f8e90 0x55d7620f9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.983+0000 7fb026a11740 10 client.0 _ll_register_callbacks cb 0x55d761ffb180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:02:03.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.983+0000 7fb0221d5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7620f8e90 0x55d7620f9310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43194/0 (socket says 172.21.15.43:43194) 2024-04-25T09:02:03.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.983+0000 7fb0221d5640 1 -- 172.21.15.43:0/2339661786 learned_addr learned my addr 172.21.15.43:0/2339661786 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:03.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.983+0000 7fb0211d3640 1 --2- 172.21.15.43:0/2339661786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7620e0ef0 0x55d7620f8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.983+0000 7fb0219d4640 1 --2- 172.21.15.43:0/2339661786 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55d7620e0b10 0x55d7620fa7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.983+0000 7fb0221d5640 1 -- 172.21.15.43:0/2339661786 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55d7620e0b10 msgr2=0x55d7620fa7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:03.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.983+0000 7fb0221d5640 1 --2- 172.21.15.43:0/2339661786 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55d7620e0b10 0x55d7620fa7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.983+0000 7fb0221d5640 1 -- 172.21.15.43:0/2339661786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7620e0ef0 msgr2=0x55d7620f8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:03.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.983+0000 7fb0221d5640 1 --2- 172.21.15.43:0/2339661786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7620e0ef0 0x55d7620f8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.983+0000 7fb0221d5640 1 -- 172.21.15.43:0/2339661786 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d761dfb7d0 con 0x55d7620f8e90 2024-04-25T09:02:03.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.983+0000 7fb0219d4640 1 --2- 172.21.15.43:0/2339661786 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55d7620e0b10 0x55d7620fa7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:03.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.983+0000 7fb0221d5640 1 --2- 172.21.15.43:0/2339661786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7620f8e90 0x55d7620f9310 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fb014079720 tx=0x7fb014032580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aefd44be2f85a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:03.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.983+0000 7fb000ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:02:03.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.983+0000 7fb000ff9640 1 -- 172.21.15.43:0/2339661786 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0140a7060 con 0x55d7620f8e90 2024-04-25T09:02:03.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.983+0000 7fb000ff9640 1 -- 172.21.15.43:0/2339661786 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb014091210 con 0x55d7620f8e90 2024-04-25T09:02:03.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.983+0000 7fb026a11740 1 -- 172.21.15.43:0/2339661786 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb00c003b10 con 0x55d7620f8e90 2024-04-25T09:02:03.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.983+0000 7fb000ff9640 1 -- 172.21.15.43:0/2339661786 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb014079790 con 0x55d7620f8e90 2024-04-25T09:02:03.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.983+0000 7fb026a11740 10 client.8409 fetch_fsmap learned FSMap version 61 2024-04-25T09:02:03.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.984+0000 7fb026a11740 1 -- 172.21.15.43:0/2339661786 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x55d761d313f0 con 0x55d7620f8e90 2024-04-25T09:02:03.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.984+0000 7fb000ff9640 1 -- 172.21.15.43:0/2339661786 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb00c003b10 con 0x55d7620f8e90 2024-04-25T09:02:03.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.984+0000 7fb000ff9640 1 -- 172.21.15.43:0/2339661786 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 61) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb014030c70 con 0x55d7620f8e90 2024-04-25T09:02:03.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.984+0000 7fb026a11740 10 client.8409 fetch_fsmap finished waiting for FSMap version 61 2024-04-25T09:02:03.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.984+0000 7fb026a11740 10 client.8409 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-25T09:02:03.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.984+0000 7fb026a11740 20 client.8409 populate_metadata read hostname 'smithi043' 2024-04-25T09:02:03.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.984+0000 7fb026a11740 1 --2- 172.21.15.43:0/2339661786 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55d7620d6da0 0x55d7620d71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.984+0000 7fb026a11740 4 client.8409 mds_command: new command op to 8094 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:02:03.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.984+0000 7fb026a11740 1 -- 172.21.15.43:0/2339661786 --> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d76202a2e0 con 0x55d7620d6da0 2024-04-25T09:02:03.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.984+0000 7fb0219d4640 1 --2- 172.21.15.43:0/2339661786 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55d7620d6da0 0x55d7620d71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.984+0000 7fb000ff9640 10 client.8409 ms_handle_connect on v2:172.21.15.193:6834/853943267 2024-04-25T09:02:03.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.984+0000 7fb0219d4640 1 --2- 172.21.15.43:0/2339661786 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55d7620d6da0 0x55d7620d71b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf351d89137b09a2 server_cookie=1b0f7bd63ed8ecea in_seq=0 out_seq=0 2024-04-25T09:02:03.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.985+0000 7fb000ff9640 1 -- 172.21.15.43:0/2339661786 <== mds.0 v2:172.21.15.193:6834/853943267 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d76202a2e0 con 0x55d7620d6da0 2024-04-25T09:02:03.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.985+0000 7fb000ff9640 10 client.8409 handle_command_reply: tid=0 2024-04-25T09:02:03.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.985+0000 7fb026a11740 1 client.8409 shutdown 2024-04-25T09:02:03.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.985+0000 7fb026a11740 1 -- 172.21.15.43:0/2339661786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7620f8e90 msgr2=0x55d7620f9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:03.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.985+0000 7fb026a11740 1 --2- 172.21.15.43:0/2339661786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7620f8e90 0x55d7620f9310 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fb014079720 tx=0x7fb014032580 comp rx=0 tx=0).stop 2024-04-25T09:02:03.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.985+0000 7fb026a11740 1 -- 172.21.15.43:0/2339661786 shutdown_connections 2024-04-25T09:02:03.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.985+0000 7fb026a11740 1 --2- 172.21.15.43:0/2339661786 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7620f8e90 0x55d7620f9310 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.985+0000 7fb026a11740 1 --2- 172.21.15.43:0/2339661786 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55d7620d6da0 0x55d7620d71b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.985+0000 7fb026a11740 1 --2- 172.21.15.43:0/2339661786 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7620e0ef0 0x55d7620f8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.985+0000 7fb026a11740 1 --2- 172.21.15.43:0/2339661786 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55d7620e0b10 0x55d7620fa7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.985+0000 7fb026a11740 1 -- 172.21.15.43:0/2339661786 >> 172.21.15.43:0/2339661786 conn(0x55d7620b3770 msgr2=0x55d7620b2600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:03.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.985+0000 7fb000ff9640 0 client.8409 ms_handle_reset on v2:172.21.15.193:6834/853943267 2024-04-25T09:02:03.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.985+0000 7fb026a11740 1 -- 172.21.15.43:0/2339661786 shutdown_connections 2024-04-25T09:02:03.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.985+0000 7fb026a11740 1 -- 172.21.15.43:0/2339661786 wait complete. 2024-04-25T09:02:03.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:03.985+0000 7fb026a11740 20 client.8409 trim_cache size 0 max 16384 2024-04-25T09:02:03.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.001+0000 7fb026a11740 1 Processor -- start 2024-04-25T09:02:03.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.001+0000 7fb026a11740 1 -- start start 2024-04-25T09:02:03.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.001+0000 7fb026a11740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55d7620e0b10 0x55d76215f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.001+0000 7fb026a11740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7620e0ef0 0x55d7620b8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.001+0000 7fb026a11740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7620f8e90 0x55d7620b8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.001+0000 7fb026a11740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c1ffd90 con 0x55d7620f8e90 2024-04-25T09:02:03.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.001+0000 7fb026a11740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c1fef10 con 0x55d7620e0ef0 2024-04-25T09:02:03.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.001+0000 7fb026a11740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c146800 con 0x55d7620e0b10 2024-04-25T09:02:03.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.001+0000 7fb0221d5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7620f8e90 0x55d7620b8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.001+0000 7fb0221d5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7620f8e90 0x55d7620b8550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43202/0 (socket says 172.21.15.43:43202) 2024-04-25T09:02:03.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.001+0000 7fb0221d5640 1 -- 172.21.15.43:0/3452595685 learned_addr learned my addr 172.21.15.43:0/3452595685 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:03.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.002+0000 7fb0221d5640 1 -- 172.21.15.43:0/3452595685 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55d7620e0b10 msgr2=0x55d76215f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:03.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.002+0000 7fb0221d5640 1 --2- 172.21.15.43:0/3452595685 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55d7620e0b10 0x55d76215f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.002+0000 7fb0221d5640 1 -- 172.21.15.43:0/3452595685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7620e0ef0 msgr2=0x55d7620b8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:03.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.002+0000 7fb0221d5640 1 --2- 172.21.15.43:0/3452595685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7620e0ef0 0x55d7620b8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.002+0000 7fb0221d5640 1 -- 172.21.15.43:0/3452595685 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d761cf10f0 con 0x55d7620f8e90 2024-04-25T09:02:03.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.002+0000 7fb0221d5640 1 --2- 172.21.15.43:0/3452595685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7620f8e90 0x55d7620b8550 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fb01402a820 tx=0x7fb01408e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=846ed16cfd35a3e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:03.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.002+0000 7fb002ffd640 1 -- 172.21.15.43:0/3452595685 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0140a7070 con 0x55d7620f8e90 2024-04-25T09:02:03.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.002+0000 7fb002ffd640 1 -- 172.21.15.43:0/3452595685 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb014081ea0 con 0x55d7620f8e90 2024-04-25T09:02:03.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.002+0000 7fb002ffd640 1 -- 172.21.15.43:0/3452595685 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01407d930 con 0x55d7620f8e90 2024-04-25T09:02:03.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.002+0000 7fb026a11740 1 -- 172.21.15.43:0/3452595685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7620f8e90 msgr2=0x55d7620b8550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:03.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.002+0000 7fb026a11740 1 --2- 172.21.15.43:0/3452595685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7620f8e90 0x55d7620b8550 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fb01402a820 tx=0x7fb01408e000 comp rx=0 tx=0).stop 2024-04-25T09:02:03.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.002+0000 7fb026a11740 1 -- 172.21.15.43:0/3452595685 shutdown_connections 2024-04-25T09:02:03.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.002+0000 7fb026a11740 1 --2- 172.21.15.43:0/3452595685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7620f8e90 0x55d7620b8550 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.002+0000 7fb026a11740 1 --2- 172.21.15.43:0/3452595685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7620e0ef0 0x55d7620b8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.002+0000 7fb026a11740 1 --2- 172.21.15.43:0/3452595685 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55d7620e0b10 0x55d76215f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.003+0000 7fb026a11740 1 -- 172.21.15.43:0/3452595685 >> 172.21.15.43:0/3452595685 conn(0x55d7620b3770 msgr2=0x55d7620f98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:03.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.003+0000 7fb026a11740 1 -- 172.21.15.43:0/3452595685 shutdown_connections 2024-04-25T09:02:03.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.003+0000 7fb026a11740 1 -- 172.21.15.43:0/3452595685 wait complete. 2024-04-25T09:02:03.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.003+0000 7fb026a11740 1 -- start start 2024-04-25T09:02:03.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.003+0000 7fb026a11740 1 Processor -- start 2024-04-25T09:02:03.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.004+0000 7fb026a11740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7620e0b10 0x55d7620b8e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.004+0000 7fb026a11740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55d7620e0ef0 0x55d7620d71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.004+0000 7fb026a11740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7620f8e90 0x55d7620d7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:03.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.004+0000 7fb026a11740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01c0b3280 con 0x55d7620f8e90 2024-04-25T09:02:03.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.004+0000 7fb026a11740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d762101870 con 0x55d7620e0b10 2024-04-25T09:02:03.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.004+0000 7fb026a11740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d76215f850 con 0x55d7620e0ef0 2024-04-25T09:02:03.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.004+0000 7fb026a11740 10 client.0 _ll_register_callbacks cb 0x7fb00c001a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:02:03.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.004+0000 7fb0221d5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7620f8e90 0x55d7620d7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.004+0000 7fb0221d5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7620f8e90 0x55d7620d7710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43214/0 (socket says 172.21.15.43:43214) 2024-04-25T09:02:03.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.004+0000 7fb0221d5640 1 -- 172.21.15.43:0/1898206534 learned_addr learned my addr 172.21.15.43:0/1898206534 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:03.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.004+0000 7fb0219d4640 1 --2- 172.21.15.43:0/1898206534 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7620e0b10 0x55d7620b8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.004+0000 7fb0211d3640 1 --2- 172.21.15.43:0/1898206534 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55d7620e0ef0 0x55d7620d71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:03.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.004+0000 7fb0221d5640 1 -- 172.21.15.43:0/1898206534 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55d7620e0ef0 msgr2=0x55d7620d71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:03.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.004+0000 7fb0221d5640 1 --2- 172.21.15.43:0/1898206534 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55d7620e0ef0 0x55d7620d71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.004+0000 7fb0221d5640 1 -- 172.21.15.43:0/1898206534 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7620e0b10 msgr2=0x55d7620b8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:03.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.004+0000 7fb0221d5640 1 --2- 172.21.15.43:0/1898206534 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7620e0b10 0x55d7620b8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:03.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.004+0000 7fb0221d5640 1 -- 172.21.15.43:0/1898206534 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d76215f380 con 0x55d7620f8e90 2024-04-25T09:02:03.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.004+0000 7fb0211d3640 1 --2- 172.21.15.43:0/1898206534 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55d7620e0ef0 0x55d7620d71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:03.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.005+0000 7fb0221d5640 1 --2- 172.21.15.43:0/1898206534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7620f8e90 0x55d7620d7710 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fb014079720 tx=0x7fb01407c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea29961877d7aee9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:03.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.005+0000 7fb000ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:02:03.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.005+0000 7fb000ff9640 1 -- 172.21.15.43:0/1898206534 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0140a7070 con 0x55d7620f8e90 2024-04-25T09:02:03.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.005+0000 7fb000ff9640 1 -- 172.21.15.43:0/1898206534 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb014081c20 con 0x55d7620f8e90 2024-04-25T09:02:03.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.005+0000 7fb026a11740 1 -- 172.21.15.43:0/1898206534 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb014081ea0 con 0x55d7620f8e90 2024-04-25T09:02:03.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.005+0000 7fb000ff9640 1 -- 172.21.15.43:0/1898206534 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb014096050 con 0x55d7620f8e90 2024-04-25T09:02:03.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.005+0000 7fb026a11740 10 client.8415 fetch_fsmap learned FSMap version 61 2024-04-25T09:02:03.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.005+0000 7fb026a11740 1 -- 172.21.15.43:0/1898206534 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x55d76215f380 con 0x55d7620f8e90 2024-04-25T09:02:03.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.005+0000 7fb000ff9640 1 -- 172.21.15.43:0/1898206534 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb014081ea0 con 0x55d7620f8e90 2024-04-25T09:02:03.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.005+0000 7fb000ff9640 1 -- 172.21.15.43:0/1898206534 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 61) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb014099080 con 0x55d7620f8e90 2024-04-25T09:02:03.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.005+0000 7fb026a11740 10 client.8415 fetch_fsmap finished waiting for FSMap version 61 2024-04-25T09:02:03.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.005+0000 7fb026a11740 10 client.8415 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.b 2024-04-25T09:02:03.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.005+0000 7fb026a11740 20 client.8415 populate_metadata read hostname 'smithi043' 2024-04-25T09:02:04.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.005+0000 7fb026a11740 1 --2- 172.21.15.43:0/1898206534 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55d76215e4a0 0x55d7620fe370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:04.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.005+0000 7fb026a11740 4 client.8415 mds_command: new command op to 8094 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:02:04.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.005+0000 7fb026a11740 1 -- 172.21.15.43:0/1898206534 --> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d7620f5cb0 con 0x55d76215e4a0 2024-04-25T09:02:04.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.005+0000 7fb0219d4640 1 --2- 172.21.15.43:0/1898206534 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55d76215e4a0 0x55d7620fe370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:04.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.006+0000 7fb0219d4640 1 --2- 172.21.15.43:0/1898206534 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55d76215e4a0 0x55d7620fe370 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a150034c92d8aba server_cookie=223aa602e7e7eaa1 in_seq=0 out_seq=0 2024-04-25T09:02:04.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.006+0000 7fb000ff9640 10 client.8415 ms_handle_connect on v2:172.21.15.193:6834/853943267 2024-04-25T09:02:04.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.006+0000 7fb000ff9640 1 -- 172.21.15.43:0/1898206534 <== mds.0 v2:172.21.15.193:6834/853943267 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d7620f5cb0 con 0x55d76215e4a0 2024-04-25T09:02:04.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.006+0000 7fb000ff9640 10 client.8415 handle_command_reply: tid=0 2024-04-25T09:02:04.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.006+0000 7fb026a11740 1 client.8415 shutdown 2024-04-25T09:02:04.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.006+0000 7fb026a11740 1 -- 172.21.15.43:0/1898206534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7620f8e90 msgr2=0x55d7620d7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:04.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.006+0000 7fb026a11740 1 --2- 172.21.15.43:0/1898206534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7620f8e90 0x55d7620d7710 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fb014079720 tx=0x7fb01407c040 comp rx=0 tx=0).stop 2024-04-25T09:02:04.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.006+0000 7fb026a11740 1 -- 172.21.15.43:0/1898206534 shutdown_connections 2024-04-25T09:02:04.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.006+0000 7fb026a11740 1 --2- 172.21.15.43:0/1898206534 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55d7620f8e90 0x55d7620d7710 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:04.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.006+0000 7fb026a11740 1 --2- 172.21.15.43:0/1898206534 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55d7620e0ef0 0x55d7620d71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:04.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.006+0000 7fb026a11740 1 --2- 172.21.15.43:0/1898206534 >> [v2:172.21.15.193:6834/853943267,v1:172.21.15.193:6836/853943267] conn(0x55d76215e4a0 0x55d7620fe370 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:04.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.006+0000 7fb026a11740 1 --2- 172.21.15.43:0/1898206534 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55d7620e0b10 0x55d7620b8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:04.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.006+0000 7fb000ff9640 0 client.8415 ms_handle_reset on v2:172.21.15.193:6834/853943267 2024-04-25T09:02:04.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.006+0000 7fb026a11740 1 -- 172.21.15.43:0/1898206534 >> 172.21.15.43:0/1898206534 conn(0x55d7620b3770 msgr2=0x55d7620e32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:04.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.006+0000 7fb026a11740 1 -- 172.21.15.43:0/1898206534 shutdown_connections 2024-04-25T09:02:04.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.006+0000 7fb026a11740 1 -- 172.21.15.43:0/1898206534 wait complete. 2024-04-25T09:02:04.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.006+0000 7fb026a11740 20 client.8415 trim_cache size 0 max 16384 2024-04-25T09:02:04.001 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:02:04.002 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-25T09:02:04.002 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-25T09:02:04.002 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:02:04.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.007+0000 7fb023c38640 1 -- 172.21.15.43:0/3880806255 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb00806b4d0 msgr2=0x7fb00806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:04.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.007+0000 7fb023c38640 1 --2- 172.21.15.43:0/3880806255 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb00806b4d0 0x7fb00806d990 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fb00c007fd0 tx=0x7fb00c009040 comp rx=0 tx=0).stop 2024-04-25T09:02:04.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.007+0000 7fb023c38640 1 -- 172.21.15.43:0/3880806255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb01c0d0100 msgr2=0x7fb01c1f2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:04.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.007+0000 7fb023c38640 1 --2- 172.21.15.43:0/3880806255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb01c0d0100 0x7fb01c1f2f20 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fb014035040 tx=0x7fb0140046c0 comp rx=0 tx=0).stop 2024-04-25T09:02:04.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.007+0000 7fb023c38640 1 -- 172.21.15.43:0/3880806255 shutdown_connections 2024-04-25T09:02:04.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.007+0000 7fb023c38640 1 --2- 172.21.15.43:0/3880806255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb01c0d0100 0x7fb01c1f2f20 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:04.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.007+0000 7fb023c38640 1 --2- 172.21.15.43:0/3880806255 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb00806b4d0 0x7fb00806d990 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:04.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.007+0000 7fb023c38640 1 --2- 172.21.15.43:0/3880806255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb01c0cfd20 0x7fb01c071cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:04.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.007+0000 7fb023c38640 1 --2- 172.21.15.43:0/3880806255 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb01c0cdb00 0x7fb01c071780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:04.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.007+0000 7fb023c38640 1 -- 172.21.15.43:0/3880806255 >> 172.21.15.43:0/3880806255 conn(0x7fb01c06ab30 msgr2=0x7fb01c06af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:04.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.007+0000 7fb023c38640 1 -- 172.21.15.43:0/3880806255 shutdown_connections 2024-04-25T09:02:04.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.007+0000 7fb023c38640 1 -- 172.21.15.43:0/3880806255 wait complete. 2024-04-25T09:02:04.006 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:02:04.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.118+0000 7f8f84be7640 1 Processor -- start 2024-04-25T09:02:04.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.118+0000 7f8f84be7640 1 -- start start 2024-04-25T09:02:04.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.119+0000 7f8f84be7640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8f8015ef50 0x7f8f800d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:04.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.119+0000 7f8f84be7640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f800d0d30 con 0x7f8f800d0920 2024-04-25T09:02:04.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.119+0000 7f8f84be7640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f800d2920 con 0x7f8f8015dce0 2024-04-25T09:02:04.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.119+0000 7f8f84be7640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f800cec60 con 0x7f8f8015ef50 2024-04-25T09:02:04.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.119+0000 7f8f77fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8f8015ef50 0x7f8f800d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:04.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.119+0000 7f8f77fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8f8015ef50 0x7f8f800d03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43216/0 (socket says 172.21.15.43:43216) 2024-04-25T09:02:04.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.119+0000 7f8f7ed9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8f800d0920 0x7f8f80160980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48736/0 (socket says 172.21.15.43:48736) 2024-04-25T09:02:04.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.119+0000 7f8f77fff640 1 -- 172.21.15.43:0/2583118511 learned_addr learned my addr 172.21.15.43:0/2583118511 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:04.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.119+0000 7f8f77fff640 1 -- 172.21.15.43:0/2583118511 >> v1:172.21.15.193:6789/0 conn(0x7f8f8015dce0 legacy=0x7f8f800ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:04.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.119+0000 7f8f77fff640 1 -- 172.21.15.43:0/2583118511 >> v1:172.21.15.43:6789/0 conn(0x7f8f800d0920 legacy=0x7f8f80160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:04.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.119+0000 7f8f77fff640 1 -- 172.21.15.43:0/2583118511 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f800cee80 con 0x7f8f8015ef50 2024-04-25T09:02:04.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.119+0000 7f8f7dd9b640 1 -- 172.21.15.43:0/2583118511 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2619026918 0 0) 0x7f8f78001490 con 0x7f8f800d0920 2024-04-25T09:02:04.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.120+0000 7f8f77fff640 1 --2- 172.21.15.43:0/2583118511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f8015ef50 0x7f8f800d03e0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f8f68009870 tx=0x7f8f6802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d245a4146b25d73 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:04.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.120+0000 7f8f7dd9b640 1 -- 172.21.15.43:0/2583118511 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f68037070 con 0x7f8f8015ef50 2024-04-25T09:02:04.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.120+0000 7f8f7dd9b640 1 -- 172.21.15.43:0/2583118511 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f68032440 con 0x7f8f8015ef50 2024-04-25T09:02:04.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.120+0000 7f8f7dd9b640 1 -- 172.21.15.43:0/2583118511 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f68032810 con 0x7f8f8015ef50 2024-04-25T09:02:04.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.121+0000 7f8f84be7640 1 -- 172.21.15.43:0/2583118511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f8015ef50 msgr2=0x7f8f800d03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:04.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.121+0000 7f8f84be7640 1 --2- 172.21.15.43:0/2583118511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f8015ef50 0x7f8f800d03e0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f8f68009870 tx=0x7f8f6802a270 comp rx=0 tx=0).stop 2024-04-25T09:02:04.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.121+0000 7f8f84be7640 1 -- 172.21.15.43:0/2583118511 shutdown_connections 2024-04-25T09:02:04.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.121+0000 7f8f84be7640 1 --2- 172.21.15.43:0/2583118511 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f8015ef50 0x7f8f800d03e0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:04.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.121+0000 7f8f84be7640 1 -- 172.21.15.43:0/2583118511 >> 172.21.15.43:0/2583118511 conn(0x7f8f8009cd00 msgr2=0x7f8f800d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:04.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.121+0000 7f8f84be7640 1 -- 172.21.15.43:0/2583118511 shutdown_connections 2024-04-25T09:02:04.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.121+0000 7f8f84be7640 1 -- 172.21.15.43:0/2583118511 wait complete. 2024-04-25T09:02:04.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.122+0000 7f8f84be7640 1 Processor -- start 2024-04-25T09:02:04.111 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.122+0000 7f8f84be7640 1 -- start start 2024-04-25T09:02:04.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.123+0000 7f8f84be7640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f800d0920 0x7f8f8014fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:04.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.123+0000 7f8f84be7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f8015dce0 0x7f8f800cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:04.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.123+0000 7f8f84be7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f8015ef50 0x7f8f8006b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:04.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.123+0000 7f8f84be7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f800cf720 con 0x7f8f8015dce0 2024-04-25T09:02:04.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.123+0000 7f8f84be7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f800cf440 con 0x7f8f8015ef50 2024-04-25T09:02:04.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.123+0000 7f8f84be7640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8015f6b0 con 0x7f8f800d0920 2024-04-25T09:02:04.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.123+0000 7f8f77fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f8015dce0 0x7f8f800cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:04.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.123+0000 7f8f77fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f8015dce0 0x7f8f800cd0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43224/0 (socket says 172.21.15.43:43224) 2024-04-25T09:02:04.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.123+0000 7f8f77fff640 1 -- 172.21.15.43:0/3184443867 learned_addr learned my addr 172.21.15.43:0/3184443867 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:04.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.123+0000 7f8f77fff640 1 -- 172.21.15.43:0/3184443867 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f800d0920 msgr2=0x7f8f8014fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:04.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.123+0000 7f8f7ed9d640 1 --2- 172.21.15.43:0/3184443867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f8015ef50 0x7f8f8006b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:04.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.123+0000 7f8f77fff640 1 --2- 172.21.15.43:0/3184443867 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f800d0920 0x7f8f8014fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:04.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.123+0000 7f8f77fff640 1 -- 172.21.15.43:0/3184443867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f8015ef50 msgr2=0x7f8f8006b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:04.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.123+0000 7f8f77fff640 1 --2- 172.21.15.43:0/3184443867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f8015ef50 0x7f8f8006b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:04.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.123+0000 7f8f77fff640 1 -- 172.21.15.43:0/3184443867 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f800b67b0 con 0x7f8f8015dce0 2024-04-25T09:02:04.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.124+0000 7f8f77fff640 1 --2- 172.21.15.43:0/3184443867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f8015dce0 0x7f8f800cd0d0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f8f68009870 tx=0x7f8f680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:04.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.124+0000 7f8f76ffd640 1 -- 172.21.15.43:0/3184443867 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f68041020 con 0x7f8f8015dce0 2024-04-25T09:02:04.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.124+0000 7f8f76ffd640 1 -- 172.21.15.43:0/3184443867 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f68030970 con 0x7f8f8015dce0 2024-04-25T09:02:04.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.124+0000 7f8f84be7640 1 -- 172.21.15.43:0/3184443867 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f800b6a90 con 0x7f8f8015dce0 2024-04-25T09:02:04.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.124+0000 7f8f76ffd640 1 -- 172.21.15.43:0/3184443867 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f68037040 con 0x7f8f8015dce0 2024-04-25T09:02:04.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.124+0000 7f8f84be7640 1 -- 172.21.15.43:0/3184443867 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f800b7050 con 0x7f8f8015dce0 2024-04-25T09:02:04.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.125+0000 7f8f84be7640 1 -- 172.21.15.43:0/3184443867 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f800680f0 con 0x7f8f8015dce0 2024-04-25T09:02:04.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.125+0000 7f8f76ffd640 1 -- 172.21.15.43:0/3184443867 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8f68044020 con 0x7f8f8015dce0 2024-04-25T09:02:04.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.126+0000 7f8f76ffd640 1 --2- 172.21.15.43:0/3184443867 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8f6006b500 0x7f8f6006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:04.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.126+0000 7f8f76ffd640 1 -- 172.21.15.43:0/3184443867 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8f680aa8c0 con 0x7f8f8015dce0 2024-04-25T09:02:04.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.126+0000 7f8f7e59c640 1 --2- 172.21.15.43:0/3184443867 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8f6006b500 0x7f8f6006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:04.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.126+0000 7f8f7e59c640 1 --2- 172.21.15.43:0/3184443867 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8f6006b500 0x7f8f6006d9c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f8f70000f60 tx=0x7f8f700079b0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:04.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.130+0000 7f8f76ffd640 1 -- 172.21.15.43:0/3184443867 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8f6804b070 con 0x7f8f8015dce0 2024-04-25T09:02:04.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.353+0000 7f8f84be7640 1 -- 172.21.15.43:0/3184443867 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f80074ef0 con 0x7f8f8015dce0 2024-04-25T09:02:04.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.353+0000 7f8f76ffd640 1 -- 172.21.15.43:0/3184443867 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f8f680797d0 con 0x7f8f8015dce0 2024-04-25T09:02:04.342 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:04.342 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":61,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2150177203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2150177203},{"type":"v1","addr":"172.21.15.193:6837","nonce":2150177203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":8079,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/621488642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":621488642},{"type":"v1","addr":"172.21.15.43:6834","nonce":621488642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8082,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1524395840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1524395840},{"type":"v1","addr":"172.21.15.43:6835","nonce":1524395840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:01:33.726790+0000","modified":"2024-04-25T09:01:35.706689+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8094},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8094":{"gid":8094,"name":"b","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.193:6836/853943267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":853943267},{"type":"v1","addr":"172.21.15.193:6836","nonce":853943267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:02:04.343 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 61 2024-04-25T09:02:04.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.356+0000 7f8f84be7640 1 -- 172.21.15.43:0/3184443867 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8f6006b500 msgr2=0x7f8f6006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:04.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.356+0000 7f8f84be7640 1 --2- 172.21.15.43:0/3184443867 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8f6006b500 0x7f8f6006d9c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f8f70000f60 tx=0x7f8f700079b0 comp rx=0 tx=0).stop 2024-04-25T09:02:04.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.356+0000 7f8f84be7640 1 -- 172.21.15.43:0/3184443867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f8015dce0 msgr2=0x7f8f800cd0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:04.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.356+0000 7f8f84be7640 1 --2- 172.21.15.43:0/3184443867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f8015dce0 0x7f8f800cd0d0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f8f68009870 tx=0x7f8f680046c0 comp rx=0 tx=0).stop 2024-04-25T09:02:04.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.356+0000 7f8f84be7640 1 -- 172.21.15.43:0/3184443867 shutdown_connections 2024-04-25T09:02:04.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.356+0000 7f8f84be7640 1 --2- 172.21.15.43:0/3184443867 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8f6006b500 0x7f8f6006d9c0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:04.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.356+0000 7f8f84be7640 1 --2- 172.21.15.43:0/3184443867 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8f8015ef50 0x7f8f8006b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:04.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.356+0000 7f8f84be7640 1 --2- 172.21.15.43:0/3184443867 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8f8015dce0 0x7f8f800cd0d0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:04.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.356+0000 7f8f84be7640 1 --2- 172.21.15.43:0/3184443867 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8f800d0920 0x7f8f8014fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:04.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.356+0000 7f8f84be7640 1 -- 172.21.15.43:0/3184443867 >> 172.21.15.43:0/3184443867 conn(0x7f8f8009cd00 msgr2=0x7f8f800ce510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:04.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.356+0000 7f8f84be7640 1 -- 172.21.15.43:0/3184443867 shutdown_connections 2024-04-25T09:02:04.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.356+0000 7f8f84be7640 1 -- 172.21.15.43:0/3184443867 wait complete. 2024-04-25T09:02:04.357 DEBUG:tasks.cephfs.test_scrub_checks:current active=['b'] 2024-04-25T09:02:04.357 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:02:04.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.470+0000 7f8442fa5640 1 Processor -- start 2024-04-25T09:02:04.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.470+0000 7f8442fa5640 1 -- start start 2024-04-25T09:02:04.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.471+0000 7f8442fa5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f843c06b540 0x7f843c0b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:04.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.471+0000 7f8442fa5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f843c0b5060 con 0x7f843c071ca0 2024-04-25T09:02:04.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.471+0000 7f8442fa5640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f843c0b5340 con 0x7f843c15de60 2024-04-25T09:02:04.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.471+0000 7f8442fa5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f843c0b4e10 con 0x7f843c06b540 2024-04-25T09:02:04.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.471+0000 7f8441542640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f843c06b540 0x7f843c0b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:04.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.471+0000 7f843bfff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f843c071ca0 0x7f843c0cf000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48748/0 (socket says 172.21.15.43:48748) 2024-04-25T09:02:04.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.471+0000 7f843bfff640 1 -- 172.21.15.43:0/4195790031 learned_addr learned my addr 172.21.15.43:0/4195790031 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:04.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.472+0000 7f8441542640 1 -- 172.21.15.43:0/4195790031 >> v1:172.21.15.193:6789/0 conn(0x7f843c15de60 legacy=0x7f843c071590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:04.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.472+0000 7f8441542640 1 -- 172.21.15.43:0/4195790031 >> v1:172.21.15.43:6789/0 conn(0x7f843c071ca0 legacy=0x7f843c0cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:04.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.472+0000 7f8441542640 1 -- 172.21.15.43:0/4195790031 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f843c0b5620 con 0x7f843c06b540 2024-04-25T09:02:04.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.472+0000 7f8441542640 1 --2- 172.21.15.43:0/4195790031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f843c06b540 0x7f843c0b48d0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f842c0097a0 tx=0x7f842c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a2c8084b6a34d13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:04.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.473+0000 7f843b7fe640 1 -- 172.21.15.43:0/4195790031 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c037070 con 0x7f843c06b540 2024-04-25T09:02:04.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.473+0000 7f843b7fe640 1 -- 172.21.15.43:0/4195790031 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f842c032440 con 0x7f843c06b540 2024-04-25T09:02:04.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.473+0000 7f843b7fe640 1 -- 172.21.15.43:0/4195790031 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c032810 con 0x7f843c06b540 2024-04-25T09:02:04.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.473+0000 7f8442fa5640 1 -- 172.21.15.43:0/4195790031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f843c06b540 msgr2=0x7f843c0b48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:04.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.473+0000 7f8442fa5640 1 --2- 172.21.15.43:0/4195790031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f843c06b540 0x7f843c0b48d0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f842c0097a0 tx=0x7f842c02a1e0 comp rx=0 tx=0).stop 2024-04-25T09:02:04.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.473+0000 7f8442fa5640 1 -- 172.21.15.43:0/4195790031 shutdown_connections 2024-04-25T09:02:04.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.473+0000 7f8442fa5640 1 --2- 172.21.15.43:0/4195790031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f843c06b540 0x7f843c0b48d0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:04.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.473+0000 7f8442fa5640 1 -- 172.21.15.43:0/4195790031 >> 172.21.15.43:0/4195790031 conn(0x7f843c0b2490 msgr2=0x7f843c0b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:04.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.473+0000 7f8442fa5640 1 -- 172.21.15.43:0/4195790031 shutdown_connections 2024-04-25T09:02:04.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.473+0000 7f8442fa5640 1 -- 172.21.15.43:0/4195790031 wait complete. 2024-04-25T09:02:04.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.474+0000 7f8442fa5640 1 Processor -- start 2024-04-25T09:02:04.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.474+0000 7f8442fa5640 1 -- start start 2024-04-25T09:02:04.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.475+0000 7f8442fa5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f843c06b540 0x7f843c153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:04.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.475+0000 7f8442fa5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f843c071ca0 0x7f843c159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:04.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.475+0000 7f8442fa5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f843c15de60 0x7f843c15f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:04.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.475+0000 7f8442fa5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843c15e240 con 0x7f843c06b540 2024-04-25T09:02:04.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.475+0000 7f8442fa5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f843c0ceae0 con 0x7f843c071ca0 2024-04-25T09:02:04.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.475+0000 7f8442fa5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f843c06b950 con 0x7f843c15de60 2024-04-25T09:02:04.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.475+0000 7f8440d41640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f843c06b540 0x7f843c153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:04.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.475+0000 7f8440d41640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f843c06b540 0x7f843c153730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43252/0 (socket says 172.21.15.43:43252) 2024-04-25T09:02:04.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.475+0000 7f8440d41640 1 -- 172.21.15.43:0/2462687654 learned_addr learned my addr 172.21.15.43:0/2462687654 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:04.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.476+0000 7f843bfff640 1 --2- 172.21.15.43:0/2462687654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f843c071ca0 0x7f843c159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:04.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.476+0000 7f8441542640 1 --2- 172.21.15.43:0/2462687654 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f843c15de60 0x7f843c15f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:04.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.476+0000 7f8440d41640 1 -- 172.21.15.43:0/2462687654 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f843c15de60 msgr2=0x7f843c15f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:04.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.476+0000 7f8440d41640 1 --2- 172.21.15.43:0/2462687654 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f843c15de60 0x7f843c15f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:04.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.476+0000 7f8440d41640 1 -- 172.21.15.43:0/2462687654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f843c071ca0 msgr2=0x7f843c159bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:04.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.476+0000 7f8440d41640 1 --2- 172.21.15.43:0/2462687654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f843c071ca0 0x7f843c159bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:04.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.476+0000 7f8440d41640 1 -- 172.21.15.43:0/2462687654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f843c0b04e0 con 0x7f843c06b540 2024-04-25T09:02:04.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.476+0000 7f843bfff640 1 --2- 172.21.15.43:0/2462687654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f843c071ca0 0x7f843c159bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:04.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.476+0000 7f8441542640 1 --2- 172.21.15.43:0/2462687654 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f843c15de60 0x7f843c15f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:04.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.476+0000 7f8440d41640 1 --2- 172.21.15.43:0/2462687654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f843c06b540 0x7f843c153730 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f843400ba50 tx=0x7f843400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:04.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.476+0000 7f84397fa640 1 -- 172.21.15.43:0/2462687654 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8434019070 con 0x7f843c06b540 2024-04-25T09:02:04.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.476+0000 7f84397fa640 1 -- 172.21.15.43:0/2462687654 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f843400cc20 con 0x7f843c06b540 2024-04-25T09:02:04.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.476+0000 7f8442fa5640 1 -- 172.21.15.43:0/2462687654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f843c15a650 con 0x7f843c06b540 2024-04-25T09:02:04.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.476+0000 7f84397fa640 1 -- 172.21.15.43:0/2462687654 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8434014440 con 0x7f843c06b540 2024-04-25T09:02:04.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.476+0000 7f8442fa5640 1 -- 172.21.15.43:0/2462687654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f843c15ac10 con 0x7f843c06b540 2024-04-25T09:02:04.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.477+0000 7f8442fa5640 1 -- 172.21.15.43:0/2462687654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f843c074ef0 con 0x7f843c06b540 2024-04-25T09:02:04.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.477+0000 7f84397fa640 1 -- 172.21.15.43:0/2462687654 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f843401c070 con 0x7f843c06b540 2024-04-25T09:02:04.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.478+0000 7f84397fa640 1 --2- 172.21.15.43:0/2462687654 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f842806b4d0 0x7f842806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:04.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.478+0000 7f84397fa640 1 -- 172.21.15.43:0/2462687654 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8434089df0 con 0x7f843c06b540 2024-04-25T09:02:04.468 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.478+0000 7f843bfff640 1 --2- 172.21.15.43:0/2462687654 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f842806b4d0 0x7f842806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:04.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.479+0000 7f843bfff640 1 --2- 172.21.15.43:0/2462687654 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f842806b4d0 0x7f842806d990 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f843c06eeb0 tx=0x7f8430009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:04.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.483+0000 7f84397fa640 1 -- 172.21.15.43:0/2462687654 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f84340586b0 con 0x7f843c06b540 2024-04-25T09:02:04.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.733+0000 7f8442fa5640 1 -- 172.21.15.43:0/2462687654 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f843c06f7c0 con 0x7f843c06b540 2024-04-25T09:02:04.722 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.733+0000 7f84397fa640 1 -- 172.21.15.43:0/2462687654 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7f843405c750 con 0x7f843c06b540 2024-04-25T09:02:04.722 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:04.722 INFO:teuthology.orchestra.run.smithi043.stdout:{"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":497,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":102,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035238,"num_in_osds":8,"osd_in_since":1714035229,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1077815292,"bytes_used":2677039104,"bytes_avail":770417074176,"bytes_total":773094113280,"write_bytes_sec":62588049,"read_op_per_sec":0,"write_op_per_sec":63},"fsmap":{"epoch":61,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"b","status":"up:active","gid":8094}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":18,"modified":"2024-04-25T09:02:01.803060+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:02:04.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.736+0000 7f8442fa5640 1 -- 172.21.15.43:0/2462687654 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f842806b4d0 msgr2=0x7f842806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:04.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.736+0000 7f8442fa5640 1 --2- 172.21.15.43:0/2462687654 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f842806b4d0 0x7f842806d990 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f843c06eeb0 tx=0x7f8430009040 comp rx=0 tx=0).stop 2024-04-25T09:02:04.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.736+0000 7f8442fa5640 1 -- 172.21.15.43:0/2462687654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f843c06b540 msgr2=0x7f843c153730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:04.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.736+0000 7f8442fa5640 1 --2- 172.21.15.43:0/2462687654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f843c06b540 0x7f843c153730 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f843400ba50 tx=0x7f843400bf10 comp rx=0 tx=0).stop 2024-04-25T09:02:04.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.736+0000 7f8442fa5640 1 -- 172.21.15.43:0/2462687654 shutdown_connections 2024-04-25T09:02:04.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.736+0000 7f8442fa5640 1 --2- 172.21.15.43:0/2462687654 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f843c15de60 0x7f843c15f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:04.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.736+0000 7f8442fa5640 1 --2- 172.21.15.43:0/2462687654 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f842806b4d0 0x7f842806d990 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:04.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.736+0000 7f8442fa5640 1 --2- 172.21.15.43:0/2462687654 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f843c071ca0 0x7f843c159bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:04.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.736+0000 7f8442fa5640 1 --2- 172.21.15.43:0/2462687654 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f843c06b540 0x7f843c153730 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:04.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.736+0000 7f8442fa5640 1 -- 172.21.15.43:0/2462687654 >> 172.21.15.43:0/2462687654 conn(0x7f843c0b2490 msgr2=0x7f843c0d0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:04.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.736+0000 7f8442fa5640 1 -- 172.21.15.43:0/2462687654 shutdown_connections 2024-04-25T09:02:04.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:04.736+0000 7f8442fa5640 1 -- 172.21.15.43:0/2462687654 wait complete. 2024-04-25T09:02:04.734 INFO:tasks.cephfs.filesystem.ceph_manager:{'b': 'paused'} 2024-04-25T09:02:05.735 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:02:05.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.862+0000 7fdeed49e640 1 Processor -- start 2024-04-25T09:02:05.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.862+0000 7fdeed49e640 1 -- start start 2024-04-25T09:02:05.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.863+0000 7fdeed49e640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fdee815dce0 0x7fdee815ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:05.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.863+0000 7fdeed49e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdee80ce550 0x7fdee80d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:05.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.863+0000 7fdeed49e640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdee80cee80 con 0x7fdee80ce9c0 2024-04-25T09:02:05.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.863+0000 7fdeed49e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdee80d2920 con 0x7fdee80ce550 2024-04-25T09:02:05.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.863+0000 7fdeed49e640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdee80cf160 con 0x7fdee815dce0 2024-04-25T09:02:05.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.863+0000 7fdedffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdee80ce550 0x7fdee80d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:05.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.863+0000 7fdedffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdee80ce550 0x7fdee80d03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43266/0 (socket says 172.21.15.43:43266) 2024-04-25T09:02:05.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.863+0000 7fdedffff640 1 -- 172.21.15.43:0/569289624 learned_addr learned my addr 172.21.15.43:0/569289624 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:05.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.863+0000 7fdee6ffd640 1 --2- 172.21.15.43:0/569289624 >> v2:172.21.15.193:3300/0 conn(0x7fdee815dce0 0x7fdee815ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:05.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.863+0000 7fdedffff640 1 -- 172.21.15.43:0/569289624 >> v2:172.21.15.193:3300/0 conn(0x7fdee815dce0 msgr2=0x7fdee815ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:05.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.863+0000 7fdedffff640 1 --2- 172.21.15.43:0/569289624 >> v2:172.21.15.193:3300/0 conn(0x7fdee815dce0 0x7fdee815ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:05.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.863+0000 7fdedffff640 1 -- 172.21.15.43:0/569289624 >> v1:172.21.15.193:6789/0 conn(0x7fdee80ce9c0 legacy=0x7fdee8160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:05.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.863+0000 7fdedffff640 1 -- 172.21.15.43:0/569289624 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee80cf440 con 0x7fdee80ce550 2024-04-25T09:02:05.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.863+0000 7fdee6ffd640 1 --2- 172.21.15.43:0/569289624 >> v2:172.21.15.193:3300/0 conn(0x7fdee815dce0 0x7fdee815ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:05.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.864+0000 7fdedffff640 1 --2- 172.21.15.43:0/569289624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdee80ce550 0x7fdee80d03e0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fdee809d3c0 tx=0x7fded002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0e2da32d73e4070 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:05.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.864+0000 7fdee67fc640 1 -- 172.21.15.43:0/569289624 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded0037070 con 0x7fdee80ce550 2024-04-25T09:02:05.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.864+0000 7fdee67fc640 1 -- 172.21.15.43:0/569289624 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded0032440 con 0x7fdee80ce550 2024-04-25T09:02:05.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.864+0000 7fdee67fc640 1 -- 172.21.15.43:0/569289624 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded0032810 con 0x7fdee80ce550 2024-04-25T09:02:05.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.865+0000 7fdeed49e640 1 -- 172.21.15.43:0/569289624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdee80ce550 msgr2=0x7fdee80d03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:05.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.865+0000 7fdeed49e640 1 --2- 172.21.15.43:0/569289624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdee80ce550 0x7fdee80d03e0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fdee809d3c0 tx=0x7fded002a1a0 comp rx=0 tx=0).stop 2024-04-25T09:02:05.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.865+0000 7fdeed49e640 1 -- 172.21.15.43:0/569289624 shutdown_connections 2024-04-25T09:02:05.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.865+0000 7fdeed49e640 1 --2- 172.21.15.43:0/569289624 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdee80ce550 0x7fdee80d03e0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:05.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.865+0000 7fdeed49e640 1 --2- 172.21.15.43:0/569289624 >> v2:172.21.15.193:3300/0 conn(0x7fdee815dce0 0x7fdee815ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:05.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.865+0000 7fdeed49e640 1 -- 172.21.15.43:0/569289624 >> 172.21.15.43:0/569289624 conn(0x7fdee809cd00 msgr2=0x7fdee80d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:05.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.865+0000 7fdeed49e640 1 -- 172.21.15.43:0/569289624 shutdown_connections 2024-04-25T09:02:05.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.865+0000 7fdeed49e640 1 -- 172.21.15.43:0/569289624 wait complete. 2024-04-25T09:02:05.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.866+0000 7fdeed49e640 1 Processor -- start 2024-04-25T09:02:05.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.866+0000 7fdeed49e640 1 -- start start 2024-04-25T09:02:05.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.867+0000 7fdeed49e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdee80ce550 0x7fdee80b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:05.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.867+0000 7fdeed49e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdee80ce9c0 0x7fdee81fb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:05.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.867+0000 7fdeed49e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdee815dce0 0x7fdee81fbc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:05.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.867+0000 7fdeed49e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee815e0c0 con 0x7fdee80ce550 2024-04-25T09:02:05.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.867+0000 7fdeed49e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee815f6b0 con 0x7fdee80ce9c0 2024-04-25T09:02:05.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.867+0000 7fdeed49e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee80d0920 con 0x7fdee815dce0 2024-04-25T09:02:05.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.867+0000 7fdee6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdee80ce550 0x7fdee80b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:05.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.867+0000 7fdee6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdee80ce550 0x7fdee80b22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43278/0 (socket says 172.21.15.43:43278) 2024-04-25T09:02:05.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.867+0000 7fdee6ffd640 1 -- 172.21.15.43:0/544599016 learned_addr learned my addr 172.21.15.43:0/544599016 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:05.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.867+0000 7fdedffff640 1 --2- 172.21.15.43:0/544599016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdee80ce9c0 0x7fdee81fb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:05.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.867+0000 7fdee6ffd640 1 -- 172.21.15.43:0/544599016 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdee815dce0 msgr2=0x7fdee81fbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:05.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.867+0000 7fdee77fe640 1 --2- 172.21.15.43:0/544599016 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdee815dce0 0x7fdee81fbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:05.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.867+0000 7fdee6ffd640 1 --2- 172.21.15.43:0/544599016 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdee815dce0 0x7fdee81fbc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:05.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.867+0000 7fdee6ffd640 1 -- 172.21.15.43:0/544599016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdee80ce9c0 msgr2=0x7fdee81fb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:05.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.867+0000 7fdee6ffd640 1 --2- 172.21.15.43:0/544599016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdee80ce9c0 0x7fdee81fb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:05.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.867+0000 7fdee6ffd640 1 -- 172.21.15.43:0/544599016 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee80b2930 con 0x7fdee80ce550 2024-04-25T09:02:05.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.867+0000 7fdedffff640 1 --2- 172.21.15.43:0/544599016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdee80ce9c0 0x7fdee81fb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:05.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.868+0000 7fdee6ffd640 1 --2- 172.21.15.43:0/544599016 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdee80ce550 0x7fdee80b22e0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fded800c7f0 tx=0x7fded800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:05.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.868+0000 7fdedf7fe640 1 -- 172.21.15.43:0/544599016 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded8015070 con 0x7fdee80ce550 2024-04-25T09:02:05.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.868+0000 7fdedf7fe640 1 -- 172.21.15.43:0/544599016 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded8007820 con 0x7fdee80ce550 2024-04-25T09:02:05.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.868+0000 7fdedf7fe640 1 -- 172.21.15.43:0/544599016 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded8007bd0 con 0x7fdee80ce550 2024-04-25T09:02:05.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.868+0000 7fdeed49e640 1 -- 172.21.15.43:0/544599016 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee80b2c10 con 0x7fdee80ce550 2024-04-25T09:02:05.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.868+0000 7fdeed49e640 1 -- 172.21.15.43:0/544599016 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee80b31d0 con 0x7fdee80ce550 2024-04-25T09:02:05.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.869+0000 7fdeed49e640 1 -- 172.21.15.43:0/544599016 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdee80680f0 con 0x7fdee80ce550 2024-04-25T09:02:05.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.870+0000 7fdedf7fe640 1 -- 172.21.15.43:0/544599016 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fded8018070 con 0x7fdee80ce550 2024-04-25T09:02:05.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.870+0000 7fdedf7fe640 1 --2- 172.21.15.43:0/544599016 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdec806b4d0 0x7fdec806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:05.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.870+0000 7fdedf7fe640 1 -- 172.21.15.43:0/544599016 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fded808a370 con 0x7fdee80ce550 2024-04-25T09:02:05.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.870+0000 7fdedffff640 1 --2- 172.21.15.43:0/544599016 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdec806b4d0 0x7fdec806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:05.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.871+0000 7fdedffff640 1 --2- 172.21.15.43:0/544599016 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdec806b4d0 0x7fdec806d990 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fded0034000 tx=0x7fded002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:05.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:05.874+0000 7fdedf7fe640 1 -- 172.21.15.43:0/544599016 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fded80591f0 con 0x7fdee80ce550 2024-04-25T09:02:06.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.091+0000 7fdeed49e640 1 -- 172.21.15.43:0/544599016 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdee8074ef0 con 0x7fdee80ce550 2024-04-25T09:02:06.080 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.092+0000 7fdedf7fe640 1 -- 172.21.15.43:0/544599016 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4940 (secure 0 0 0) 0x7fded8059e20 con 0x7fdee80ce550 2024-04-25T09:02:06.080 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:06.080 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":61,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2150177203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2150177203},{"type":"v1","addr":"172.21.15.193:6837","nonce":2150177203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":8079,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/621488642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":621488642},{"type":"v1","addr":"172.21.15.43:6834","nonce":621488642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8082,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1524395840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1524395840},{"type":"v1","addr":"172.21.15.43:6835","nonce":1524395840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:01:33.726790+0000","modified":"2024-04-25T09:01:35.706689+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8094},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8094":{"gid":8094,"name":"b","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.193:6836/853943267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":853943267},{"type":"v1","addr":"172.21.15.193:6836","nonce":853943267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:02:06.080 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 61 2024-04-25T09:02:06.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.094+0000 7fdeed49e640 1 -- 172.21.15.43:0/544599016 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdec806b4d0 msgr2=0x7fdec806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:06.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.094+0000 7fdeed49e640 1 --2- 172.21.15.43:0/544599016 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdec806b4d0 0x7fdec806d990 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fded0034000 tx=0x7fded002e000 comp rx=0 tx=0).stop 2024-04-25T09:02:06.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.094+0000 7fdeed49e640 1 -- 172.21.15.43:0/544599016 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdee80ce550 msgr2=0x7fdee80b22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:06.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.094+0000 7fdeed49e640 1 --2- 172.21.15.43:0/544599016 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdee80ce550 0x7fdee80b22e0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fded800c7f0 tx=0x7fded800ccb0 comp rx=0 tx=0).stop 2024-04-25T09:02:06.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.094+0000 7fdeed49e640 1 -- 172.21.15.43:0/544599016 shutdown_connections 2024-04-25T09:02:06.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.094+0000 7fdeed49e640 1 --2- 172.21.15.43:0/544599016 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdee815dce0 0x7fdee81fbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:06.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.094+0000 7fdeed49e640 1 --2- 172.21.15.43:0/544599016 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdec806b4d0 0x7fdec806d990 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:06.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.094+0000 7fdeed49e640 1 --2- 172.21.15.43:0/544599016 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdee80ce9c0 0x7fdee81fb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:06.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.094+0000 7fdeed49e640 1 --2- 172.21.15.43:0/544599016 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdee80ce550 0x7fdee80b22e0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:06.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.094+0000 7fdeed49e640 1 -- 172.21.15.43:0/544599016 >> 172.21.15.43:0/544599016 conn(0x7fdee809cd00 msgr2=0x7fdee80cf960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:06.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.094+0000 7fdeed49e640 1 -- 172.21.15.43:0/544599016 shutdown_connections 2024-04-25T09:02:06.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.094+0000 7fdeed49e640 1 -- 172.21.15.43:0/544599016 wait complete. 2024-04-25T09:02:06.092 DEBUG:tasks.cephfs.test_scrub_checks:current active=['b'] 2024-04-25T09:02:06.093 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:02:06.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.205+0000 7f0893953640 1 Processor -- start 2024-04-25T09:02:06.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.205+0000 7f0893953640 1 -- start start 2024-04-25T09:02:06.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.206+0000 7f0893953640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f088c06e530 0x7f088c06cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:06.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.206+0000 7f0893953640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f088c06e9c0 con 0x7f088c06f300 2024-04-25T09:02:06.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.206+0000 7f0893953640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f088c0d0d80 con 0x7f088c15dcf0 2024-04-25T09:02:06.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.206+0000 7f0893953640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f088c06c050 con 0x7f088c06e530 2024-04-25T09:02:06.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.206+0000 7f0890eee640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f088c06f300 0x7f088c06c4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48760/0 (socket says 172.21.15.43:48760) 2024-04-25T09:02:06.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.206+0000 7f0890eee640 1 -- 172.21.15.43:0/2508185421 learned_addr learned my addr 172.21.15.43:0/2508185421 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:06.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.207+0000 7f0891ef0640 1 --2- 172.21.15.43:0/2508185421 >> v2:172.21.15.193:3300/0 conn(0x7f088c06e530 0x7f088c06cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:06.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.207+0000 7f0883fff640 1 -- 172.21.15.43:0/2508185421 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1341000306 0 0) 0x7f087c001110 con 0x7f088c06f300 2024-04-25T09:02:06.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.207+0000 7f0883fff640 1 -- 172.21.15.43:0/2508185421 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0878003520 con 0x7f088c06f300 2024-04-25T09:02:06.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.207+0000 7f0883fff640 1 -- 172.21.15.43:0/2508185421 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 292664196 0 0) 0x7f088c0d0d80 con 0x7f088c15dcf0 2024-04-25T09:02:06.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.207+0000 7f0883fff640 1 -- 172.21.15.43:0/2508185421 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f087c001110 con 0x7f088c15dcf0 2024-04-25T09:02:06.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.207+0000 7f0883fff640 1 -- 172.21.15.43:0/2508185421 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1619612649 0 0) 0x7f0878003520 con 0x7f088c06f300 2024-04-25T09:02:06.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.207+0000 7f0883fff640 1 -- 172.21.15.43:0/2508185421 >> v2:172.21.15.193:3300/0 conn(0x7f088c06e530 msgr2=0x7f088c06cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:06.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.207+0000 7f0883fff640 1 --2- 172.21.15.43:0/2508185421 >> v2:172.21.15.193:3300/0 conn(0x7f088c06e530 0x7f088c06cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:06.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.207+0000 7f0883fff640 1 -- 172.21.15.43:0/2508185421 >> v1:172.21.15.193:6789/0 conn(0x7f088c15dcf0 legacy=0x7f088c06ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:06.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.207+0000 7f0883fff640 1 -- 172.21.15.43:0/2508185421 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f088c0d04e0 con 0x7f088c06f300 2024-04-25T09:02:06.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.207+0000 7f0883fff640 1 -- 172.21.15.43:0/2508185421 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f087c001840 con 0x7f088c06f300 2024-04-25T09:02:06.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.207+0000 7f0883fff640 1 -- 172.21.15.43:0/2508185421 >> v1:172.21.15.43:6789/0 conn(0x7f088c06f300 legacy=0x7f088c06c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:06.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.207+0000 7f0883fff640 1 --2- 172.21.15.43:0/2508185421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0878003b40 0x7f0878003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:06.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.207+0000 7f0883fff640 1 --2- 172.21.15.43:0/2508185421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0878004800 0x7f0878004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:06.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.207+0000 7f0883fff640 1 --2- 172.21.15.43:0/2508185421 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08780051c0 0x7f087800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:06.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.207+0000 7f0883fff640 1 -- 172.21.15.43:0/2508185421 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088c06c050 con 0x7f0878003b40 2024-04-25T09:02:06.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.207+0000 7f0883fff640 1 -- 172.21.15.43:0/2508185421 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087800c0f0 con 0x7f0878004800 2024-04-25T09:02:06.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.207+0000 7f0883fff640 1 -- 172.21.15.43:0/2508185421 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f087800c2f0 con 0x7f08780051c0 2024-04-25T09:02:06.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.207+0000 7f08916ef640 1 --2- 172.21.15.43:0/2508185421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0878003b40 0x7f0878003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:06.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.207+0000 7f0890eee640 1 --2- 172.21.15.43:0/2508185421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0878004800 0x7f0878004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:06.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.207+0000 7f0891ef0640 1 --2- 172.21.15.43:0/2508185421 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08780051c0 0x7f087800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:06.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.207+0000 7f08916ef640 1 -- 172.21.15.43:0/2508185421 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08780051c0 msgr2=0x7f087800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:06.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.207+0000 7f08916ef640 1 --2- 172.21.15.43:0/2508185421 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08780051c0 0x7f087800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:06.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.207+0000 7f08916ef640 1 -- 172.21.15.43:0/2508185421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0878004800 msgr2=0x7f0878004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:06.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.207+0000 7f08916ef640 1 --2- 172.21.15.43:0/2508185421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0878004800 0x7f0878004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:06.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.207+0000 7f08916ef640 1 -- 172.21.15.43:0/2508185421 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0878003520 con 0x7f0878003b40 2024-04-25T09:02:06.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.207+0000 7f0891ef0640 1 -- 172.21.15.43:0/2508185421 reap_dead start 2024-04-25T09:02:06.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.208+0000 7f08916ef640 1 --2- 172.21.15.43:0/2508185421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0878003b40 0x7f0878003f40 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f08880018d0 tx=0x7f0878005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc7f28a30e90384e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:06.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.208+0000 7f0883fff640 1 -- 172.21.15.43:0/2508185421 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08880014f0 con 0x7f0878003b40 2024-04-25T09:02:06.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.208+0000 7f0883fff640 1 -- 172.21.15.43:0/2508185421 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08880052d0 con 0x7f0878003b40 2024-04-25T09:02:06.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.208+0000 7f0883fff640 1 -- 172.21.15.43:0/2508185421 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f088802ac60 con 0x7f0878003b40 2024-04-25T09:02:06.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.209+0000 7f0893953640 1 -- 172.21.15.43:0/2508185421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0878003b40 msgr2=0x7f0878003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:06.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.209+0000 7f0893953640 1 --2- 172.21.15.43:0/2508185421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0878003b40 0x7f0878003f40 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f08880018d0 tx=0x7f0878005830 comp rx=0 tx=0).stop 2024-04-25T09:02:06.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.209+0000 7f0893953640 1 -- 172.21.15.43:0/2508185421 shutdown_connections 2024-04-25T09:02:06.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.209+0000 7f0893953640 1 --2- 172.21.15.43:0/2508185421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0878003b40 0x7f0878003f40 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:06.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.209+0000 7f0893953640 1 -- 172.21.15.43:0/2508185421 >> 172.21.15.43:0/2508185421 conn(0x7f088c0d1700 msgr2=0x7f088c0d1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:06.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.209+0000 7f0893953640 1 -- 172.21.15.43:0/2508185421 shutdown_connections 2024-04-25T09:02:06.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.209+0000 7f0893953640 1 -- 172.21.15.43:0/2508185421 wait complete. 2024-04-25T09:02:06.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.210+0000 7f0893953640 1 Processor -- start 2024-04-25T09:02:06.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.210+0000 7f0893953640 1 -- start start 2024-04-25T09:02:06.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.211+0000 7f0893953640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0878003b40 0x7f088c0c6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:06.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.211+0000 7f0893953640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f088c0b4590 0x7f088c155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:06.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.211+0000 7f0893953640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f088c0b4a10 0x7f088c156500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:06.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.211+0000 7f0893953640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088c06bce0 con 0x7f0878003b40 2024-04-25T09:02:06.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.211+0000 7f0893953640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f088c0d0aa0 con 0x7f088c0b4a10 2024-04-25T09:02:06.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.211+0000 7f0893953640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f088c0b0d50 con 0x7f088c0b4590 2024-04-25T09:02:06.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.211+0000 7f08916ef640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0878003b40 0x7f088c0c6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:06.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.211+0000 7f08916ef640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0878003b40 0x7f088c0c6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43298/0 (socket says 172.21.15.43:43298) 2024-04-25T09:02:06.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.211+0000 7f08916ef640 1 -- 172.21.15.43:0/4064701426 learned_addr learned my addr 172.21.15.43:0/4064701426 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:06.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.211+0000 7f08916ef640 1 -- 172.21.15.43:0/4064701426 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f088c0b4590 msgr2=0x7f088c155fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:06.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.211+0000 7f08916ef640 1 --2- 172.21.15.43:0/4064701426 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f088c0b4590 0x7f088c155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:06.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.211+0000 7f08916ef640 1 -- 172.21.15.43:0/4064701426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f088c0b4a10 msgr2=0x7f088c156500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:06.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.211+0000 7f08916ef640 1 --2- 172.21.15.43:0/4064701426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f088c0b4a10 0x7f088c156500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:06.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.211+0000 7f08916ef640 1 -- 172.21.15.43:0/4064701426 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f088c151640 con 0x7f0878003b40 2024-04-25T09:02:06.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.211+0000 7f08916ef640 1 --2- 172.21.15.43:0/4064701426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0878003b40 0x7f088c0c6690 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f0888005040 tx=0x7f08880012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:06.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.211+0000 7f0881ffb640 1 -- 172.21.15.43:0/4064701426 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0888037040 con 0x7f0878003b40 2024-04-25T09:02:06.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.211+0000 7f0881ffb640 1 -- 172.21.15.43:0/4064701426 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0888005480 con 0x7f0878003b40 2024-04-25T09:02:06.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.211+0000 7f0893953640 1 -- 172.21.15.43:0/4064701426 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f088c0c55f0 con 0x7f0878003b40 2024-04-25T09:02:06.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.211+0000 7f0881ffb640 1 -- 172.21.15.43:0/4064701426 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08880014f0 con 0x7f0878003b40 2024-04-25T09:02:06.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.212+0000 7f0893953640 1 -- 172.21.15.43:0/4064701426 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f088c0c5bb0 con 0x7f0878003b40 2024-04-25T09:02:06.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.213+0000 7f0893953640 1 -- 172.21.15.43:0/4064701426 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f088c074ef0 con 0x7f0878003b40 2024-04-25T09:02:06.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.213+0000 7f0881ffb640 1 -- 172.21.15.43:0/4064701426 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0888044020 con 0x7f0878003b40 2024-04-25T09:02:06.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.213+0000 7f0881ffb640 1 --2- 172.21.15.43:0/4064701426 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f087006b500 0x7f087006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:06.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.214+0000 7f0881ffb640 1 -- 172.21.15.43:0/4064701426 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f08880a9ff0 con 0x7f0878003b40 2024-04-25T09:02:06.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.214+0000 7f0890eee640 1 --2- 172.21.15.43:0/4064701426 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f087006b500 0x7f087006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:06.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.214+0000 7f0890eee640 1 --2- 172.21.15.43:0/4064701426 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f087006b500 0x7f087006d9c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f088c0c7bc0 tx=0x7f087c000f70 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:06.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.218+0000 7f0881ffb640 1 -- 172.21.15.43:0/4064701426 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0888078d40 con 0x7f0878003b40 2024-04-25T09:02:06.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.473+0000 7f0893953640 1 -- 172.21.15.43:0/4064701426 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f088c06dbe0 con 0x7f0878003b40 2024-04-25T09:02:06.461 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.473+0000 7f0881ffb640 1 -- 172.21.15.43:0/4064701426 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7f0888078640 con 0x7f0878003b40 2024-04-25T09:02:06.461 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:06.461 INFO:teuthology.orchestra.run.smithi043.stdout:{"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":498,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":102,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035238,"num_in_osds":8,"osd_in_since":1714035229,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1077815292,"bytes_used":2677039104,"bytes_avail":770417074176,"bytes_total":773094113280,"write_bytes_sec":25917664,"read_op_per_sec":0,"write_op_per_sec":27},"fsmap":{"epoch":61,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"b","status":"up:active","gid":8094}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":18,"modified":"2024-04-25T09:02:01.803060+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:02:06.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.475+0000 7f0893953640 1 -- 172.21.15.43:0/4064701426 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f087006b500 msgr2=0x7f087006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:06.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.475+0000 7f0893953640 1 --2- 172.21.15.43:0/4064701426 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f087006b500 0x7f087006d9c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f088c0c7bc0 tx=0x7f087c000f70 comp rx=0 tx=0).stop 2024-04-25T09:02:06.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.475+0000 7f0893953640 1 -- 172.21.15.43:0/4064701426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0878003b40 msgr2=0x7f088c0c6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:06.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.475+0000 7f0893953640 1 --2- 172.21.15.43:0/4064701426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0878003b40 0x7f088c0c6690 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f0888005040 tx=0x7f08880012d0 comp rx=0 tx=0).stop 2024-04-25T09:02:06.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.475+0000 7f0893953640 1 -- 172.21.15.43:0/4064701426 shutdown_connections 2024-04-25T09:02:06.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.475+0000 7f0893953640 1 --2- 172.21.15.43:0/4064701426 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f087006b500 0x7f087006d9c0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:06.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.475+0000 7f0893953640 1 --2- 172.21.15.43:0/4064701426 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f088c0b4a10 0x7f088c156500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:06.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.475+0000 7f0893953640 1 --2- 172.21.15.43:0/4064701426 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f088c0b4590 0x7f088c155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:06.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.475+0000 7f0893953640 1 --2- 172.21.15.43:0/4064701426 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0878003b40 0x7f088c0c6690 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:06.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.475+0000 7f0893953640 1 -- 172.21.15.43:0/4064701426 >> 172.21.15.43:0/4064701426 conn(0x7f088c0d1700 msgr2=0x7f088c0afdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:06.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.475+0000 7f0893953640 1 -- 172.21.15.43:0/4064701426 shutdown_connections 2024-04-25T09:02:06.464 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:06.475+0000 7f0893953640 1 -- 172.21.15.43:0/4064701426 wait complete. 2024-04-25T09:02:06.473 INFO:tasks.cephfs.filesystem.ceph_manager:{'b': 'paused'} 2024-04-25T09:02:07.474 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:02:07.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.602+0000 7f82b7b8b640 1 Processor -- start 2024-04-25T09:02:07.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.602+0000 7f82b7b8b640 1 -- start start 2024-04-25T09:02:07.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.603+0000 7f82b7b8b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82b015df50 0x7f82b01619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:07.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.603+0000 7f82b7b8b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82b00d24a0 con 0x7f82b0161f30 2024-04-25T09:02:07.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.603+0000 7f82b7b8b640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82b00d2740 con 0x7f82b00cf670 2024-04-25T09:02:07.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.603+0000 7f82b7b8b640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82b00cfa80 con 0x7f82b015df50 2024-04-25T09:02:07.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.603+0000 7f82b5126640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f82b0161f30 0x7f82b00c7cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48762/0 (socket says 172.21.15.43:48762) 2024-04-25T09:02:07.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.603+0000 7f82b5126640 1 -- 172.21.15.43:0/2697512497 learned_addr learned my addr 172.21.15.43:0/2697512497 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:07.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.603+0000 7f82b5927640 1 --2- 172.21.15.43:0/2697512497 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82b015df50 0x7f82b01619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:07.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.603+0000 7f82b4925640 1 -- 172.21.15.43:0/2697512497 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 756470052 0 0) 0x7f82b00d24a0 con 0x7f82b0161f30 2024-04-25T09:02:07.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.603+0000 7f82b4925640 1 -- 172.21.15.43:0/2697512497 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f829c003520 con 0x7f82b0161f30 2024-04-25T09:02:07.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.603+0000 7f82b4925640 1 -- 172.21.15.43:0/2697512497 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 267532570 0 0) 0x7f82b00d2740 con 0x7f82b00cf670 2024-04-25T09:02:07.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.603+0000 7f82b4925640 1 -- 172.21.15.43:0/2697512497 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82b00d24a0 con 0x7f82b00cf670 2024-04-25T09:02:07.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.604+0000 7f82b5927640 1 -- 172.21.15.43:0/2697512497 >> v1:172.21.15.193:6789/0 conn(0x7f82b00cf670 legacy=0x7f82b00c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:07.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.604+0000 7f82b5927640 1 -- 172.21.15.43:0/2697512497 >> v1:172.21.15.43:6789/0 conn(0x7f82b0161f30 legacy=0x7f82b00c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:07.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.604+0000 7f82b5927640 1 -- 172.21.15.43:0/2697512497 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b00d2a20 con 0x7f82b015df50 2024-04-25T09:02:07.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.604+0000 7f82b5927640 1 --2- 172.21.15.43:0/2697512497 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82b015df50 0x7f82b01619f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f82ac0097d0 tx=0x7f82ac02a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=793c009ebfab4f90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:07.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.605+0000 7f82b4925640 1 -- 172.21.15.43:0/2697512497 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ac037070 con 0x7f82b015df50 2024-04-25T09:02:07.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.605+0000 7f82b4925640 1 -- 172.21.15.43:0/2697512497 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82ac032440 con 0x7f82b015df50 2024-04-25T09:02:07.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.605+0000 7f82b4925640 1 -- 172.21.15.43:0/2697512497 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82ac032810 con 0x7f82b015df50 2024-04-25T09:02:07.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.605+0000 7f82b7b8b640 1 -- 172.21.15.43:0/2697512497 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82b015df50 msgr2=0x7f82b01619f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:07.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.605+0000 7f82b7b8b640 1 --2- 172.21.15.43:0/2697512497 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82b015df50 0x7f82b01619f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f82ac0097d0 tx=0x7f82ac02a210 comp rx=0 tx=0).stop 2024-04-25T09:02:07.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.605+0000 7f82b7b8b640 1 -- 172.21.15.43:0/2697512497 shutdown_connections 2024-04-25T09:02:07.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.605+0000 7f82b7b8b640 1 --2- 172.21.15.43:0/2697512497 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82b015df50 0x7f82b01619f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:07.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.605+0000 7f82b7b8b640 1 -- 172.21.15.43:0/2697512497 >> 172.21.15.43:0/2697512497 conn(0x7f82b00d3a30 msgr2=0x7f82b00d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:07.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.605+0000 7f82b7b8b640 1 -- 172.21.15.43:0/2697512497 shutdown_connections 2024-04-25T09:02:07.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.605+0000 7f82b7b8b640 1 -- 172.21.15.43:0/2697512497 wait complete. 2024-04-25T09:02:07.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.607+0000 7f82b7b8b640 1 Processor -- start 2024-04-25T09:02:07.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.607+0000 7f82b7b8b640 1 -- start start 2024-04-25T09:02:07.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.607+0000 7f82b7b8b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82b00cf670 0x7f82b01e7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:07.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.607+0000 7f82b7b8b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82b015df50 0x7f82b00c6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:07.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.607+0000 7f82b7b8b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82b0161f30 0x7f82b00a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:07.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.607+0000 7f82b7b8b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b00c77d0 con 0x7f82b015df50 2024-04-25T09:02:07.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.607+0000 7f82b7b8b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82b015e330 con 0x7f82b00cf670 2024-04-25T09:02:07.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.607+0000 7f82b7b8b640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82b00d2190 con 0x7f82b0161f30 2024-04-25T09:02:07.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.607+0000 7f82b5126640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82b015df50 0x7f82b00c6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:07.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.607+0000 7f82b5126640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82b015df50 0x7f82b00c6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43306/0 (socket says 172.21.15.43:43306) 2024-04-25T09:02:07.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.607+0000 7f82b5126640 1 -- 172.21.15.43:0/1675946639 learned_addr learned my addr 172.21.15.43:0/1675946639 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:07.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.608+0000 7f82b5927640 1 --2- 172.21.15.43:0/1675946639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82b00cf670 0x7f82b01e7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:07.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.608+0000 7f82b6128640 1 --2- 172.21.15.43:0/1675946639 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82b0161f30 0x7f82b00a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:07.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.608+0000 7f82b5126640 1 -- 172.21.15.43:0/1675946639 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82b0161f30 msgr2=0x7f82b00a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:07.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.608+0000 7f82b5126640 1 --2- 172.21.15.43:0/1675946639 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82b0161f30 0x7f82b00a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:07.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.608+0000 7f82b5126640 1 -- 172.21.15.43:0/1675946639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82b00cf670 msgr2=0x7f82b01e7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:07.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.608+0000 7f82b5126640 1 --2- 172.21.15.43:0/1675946639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82b00cf670 0x7f82b01e7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:07.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.608+0000 7f82b5126640 1 -- 172.21.15.43:0/1675946639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b00a0ec0 con 0x7f82b015df50 2024-04-25T09:02:07.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.608+0000 7f82b5126640 1 --2- 172.21.15.43:0/1675946639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82b015df50 0x7f82b00c6430 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f82a000c410 tx=0x7f82a000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:07.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.608+0000 7f82a67fc640 1 -- 172.21.15.43:0/1675946639 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a0017070 con 0x7f82b015df50 2024-04-25T09:02:07.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.608+0000 7f82a67fc640 1 -- 172.21.15.43:0/1675946639 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a0005ce0 con 0x7f82b015df50 2024-04-25T09:02:07.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.608+0000 7f82a67fc640 1 -- 172.21.15.43:0/1675946639 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a00072d0 con 0x7f82b015df50 2024-04-25T09:02:07.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.608+0000 7f82b7b8b640 1 -- 172.21.15.43:0/1675946639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82b00a11a0 con 0x7f82b015df50 2024-04-25T09:02:07.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.608+0000 7f82b7b8b640 1 -- 172.21.15.43:0/1675946639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82b00c6970 con 0x7f82b015df50 2024-04-25T09:02:07.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.609+0000 7f82b7b8b640 1 -- 172.21.15.43:0/1675946639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82b00680f0 con 0x7f82b015df50 2024-04-25T09:02:07.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.610+0000 7f82a67fc640 1 -- 172.21.15.43:0/1675946639 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f82a001a070 con 0x7f82b015df50 2024-04-25T09:02:07.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.610+0000 7f82a67fc640 1 --2- 172.21.15.43:0/1675946639 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f829406b4d0 0x7f829406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:07.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.610+0000 7f82a67fc640 1 -- 172.21.15.43:0/1675946639 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82a00894c0 con 0x7f82b015df50 2024-04-25T09:02:07.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.611+0000 7f82b5927640 1 --2- 172.21.15.43:0/1675946639 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f829406b4d0 0x7f829406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:07.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.611+0000 7f82b5927640 1 --2- 172.21.15.43:0/1675946639 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f829406b4d0 0x7f829406d990 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f82ac034000 tx=0x7f82ac02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:07.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.614+0000 7f82a67fc640 1 -- 172.21.15.43:0/1675946639 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f82a0057d80 con 0x7f82b015df50 2024-04-25T09:02:07.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.831+0000 7f82b7b8b640 1 -- 172.21.15.43:0/1675946639 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82b0074ef0 con 0x7f82b015df50 2024-04-25T09:02:07.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.831+0000 7f82a67fc640 1 -- 172.21.15.43:0/1675946639 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f82a005be20 con 0x7f82b015df50 2024-04-25T09:02:07.819 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:07.819 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":61,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2150177203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2150177203},{"type":"v1","addr":"172.21.15.193:6837","nonce":2150177203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":8079,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/621488642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":621488642},{"type":"v1","addr":"172.21.15.43:6834","nonce":621488642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8082,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1524395840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1524395840},{"type":"v1","addr":"172.21.15.43:6835","nonce":1524395840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:01:33.726790+0000","modified":"2024-04-25T09:01:35.706689+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8094},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8094":{"gid":8094,"name":"b","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.193:6836/853943267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":853943267},{"type":"v1","addr":"172.21.15.193:6836","nonce":853943267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:02:07.820 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 61 2024-04-25T09:02:07.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.833+0000 7f82b7b8b640 1 -- 172.21.15.43:0/1675946639 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f829406b4d0 msgr2=0x7f829406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:07.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.833+0000 7f82b7b8b640 1 --2- 172.21.15.43:0/1675946639 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f829406b4d0 0x7f829406d990 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f82ac034000 tx=0x7f82ac02e000 comp rx=0 tx=0).stop 2024-04-25T09:02:07.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.833+0000 7f82b7b8b640 1 -- 172.21.15.43:0/1675946639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82b015df50 msgr2=0x7f82b00c6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:07.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.833+0000 7f82b7b8b640 1 --2- 172.21.15.43:0/1675946639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82b015df50 0x7f82b00c6430 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f82a000c410 tx=0x7f82a000c8d0 comp rx=0 tx=0).stop 2024-04-25T09:02:07.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.833+0000 7f82b7b8b640 1 -- 172.21.15.43:0/1675946639 shutdown_connections 2024-04-25T09:02:07.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.833+0000 7f82b7b8b640 1 --2- 172.21.15.43:0/1675946639 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82b0161f30 0x7f82b00a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:07.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.833+0000 7f82b7b8b640 1 --2- 172.21.15.43:0/1675946639 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82b015df50 0x7f82b00c6430 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:07.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.833+0000 7f82b7b8b640 1 --2- 172.21.15.43:0/1675946639 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f829406b4d0 0x7f829406d990 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:07.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.833+0000 7f82b7b8b640 1 --2- 172.21.15.43:0/1675946639 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82b00cf670 0x7f82b01e7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:07.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.833+0000 7f82b7b8b640 1 -- 172.21.15.43:0/1675946639 >> 172.21.15.43:0/1675946639 conn(0x7f82b00d3a30 msgr2=0x7f82b00c8a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:07.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.833+0000 7f82b7b8b640 1 -- 172.21.15.43:0/1675946639 shutdown_connections 2024-04-25T09:02:07.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.833+0000 7f82b7b8b640 1 -- 172.21.15.43:0/1675946639 wait complete. 2024-04-25T09:02:07.833 DEBUG:tasks.cephfs.test_scrub_checks:current active=['b'] 2024-04-25T09:02:07.833 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:02:07.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.945+0000 7f8a9efe2640 1 Processor -- start 2024-04-25T09:02:07.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.946+0000 7f8a9efe2640 1 -- start start 2024-04-25T09:02:07.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.946+0000 7f8a9efe2640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8a980c8d00 0x7f8a98162100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:07.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.946+0000 7f8a9efe2640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8a980ce660 0x7f8a980ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:07.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.946+0000 7f8a9efe2640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a980d24a0 con 0x7f8a9815df50 2024-04-25T09:02:07.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.946+0000 7f8a9efe2640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a9815e330 con 0x7f8a980c8d00 2024-04-25T09:02:07.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.946+0000 7f8a9efe2640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a980d2740 con 0x7f8a980ce660 2024-04-25T09:02:07.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.946+0000 7f8a97fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8a980c8d00 0x7f8a98162100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:07.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.947+0000 7f8a9cd7e640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8a9815df50 0x7f8a981619f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48772/0 (socket says 172.21.15.43:48772) 2024-04-25T09:02:07.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.947+0000 7f8a97fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8a980c8d00 0x7f8a98162100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43322/0 (socket says 172.21.15.43:43322) 2024-04-25T09:02:07.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.947+0000 7f8a9cd7e640 1 -- 172.21.15.43:0/2101615719 learned_addr learned my addr 172.21.15.43:0/2101615719 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:07.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.947+0000 7f8a9d57f640 1 --2- 172.21.15.43:0/2101615719 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8a980ce660 0x7f8a980ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:07.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.947+0000 7f8a97fff640 1 -- 172.21.15.43:0/2101615719 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8a980ce660 msgr2=0x7f8a980ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:07.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.947+0000 7f8a97fff640 1 --2- 172.21.15.43:0/2101615719 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8a980ce660 0x7f8a980ceab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:07.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.947+0000 7f8a97fff640 1 -- 172.21.15.43:0/2101615719 >> v1:172.21.15.43:6789/0 conn(0x7f8a9815df50 legacy=0x7f8a981619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:07.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.947+0000 7f8a977fe640 1 -- 172.21.15.43:0/2101615719 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3440509644 0 0) 0x7f8a980d24a0 con 0x7f8a9815df50 2024-04-25T09:02:07.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.947+0000 7f8a97fff640 1 -- 172.21.15.43:0/2101615719 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a980d2a20 con 0x7f8a980c8d00 2024-04-25T09:02:07.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.948+0000 7f8a97fff640 1 --2- 172.21.15.43:0/2101615719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a980c8d00 0x7f8a98162100 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f8a840097a0 tx=0x7f8a8402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad99777b368339d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:07.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.948+0000 7f8a977fe640 1 -- 172.21.15.43:0/2101615719 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a84037070 con 0x7f8a980c8d00 2024-04-25T09:02:07.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.948+0000 7f8a977fe640 1 -- 172.21.15.43:0/2101615719 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a84032440 con 0x7f8a980c8d00 2024-04-25T09:02:07.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.948+0000 7f8a977fe640 1 -- 172.21.15.43:0/2101615719 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a84032810 con 0x7f8a980c8d00 2024-04-25T09:02:07.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.948+0000 7f8a9efe2640 1 -- 172.21.15.43:0/2101615719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a980c8d00 msgr2=0x7f8a98162100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:07.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.948+0000 7f8a9efe2640 1 --2- 172.21.15.43:0/2101615719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a980c8d00 0x7f8a98162100 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f8a840097a0 tx=0x7f8a8402a180 comp rx=0 tx=0).stop 2024-04-25T09:02:07.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.948+0000 7f8a9efe2640 1 -- 172.21.15.43:0/2101615719 shutdown_connections 2024-04-25T09:02:07.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.948+0000 7f8a9efe2640 1 --2- 172.21.15.43:0/2101615719 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8a980ce660 0x7f8a980ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:07.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.948+0000 7f8a9efe2640 1 --2- 172.21.15.43:0/2101615719 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a980c8d00 0x7f8a98162100 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:07.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.948+0000 7f8a9efe2640 1 -- 172.21.15.43:0/2101615719 >> 172.21.15.43:0/2101615719 conn(0x7f8a980d3a30 msgr2=0x7f8a980d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:07.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.948+0000 7f8a9efe2640 1 -- 172.21.15.43:0/2101615719 shutdown_connections 2024-04-25T09:02:07.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.948+0000 7f8a9efe2640 1 -- 172.21.15.43:0/2101615719 wait complete. 2024-04-25T09:02:07.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.949+0000 7f8a9efe2640 1 Processor -- start 2024-04-25T09:02:07.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.950+0000 7f8a9efe2640 1 -- start start 2024-04-25T09:02:07.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.950+0000 7f8a9efe2640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8a980c8d00 0x7f8a9814dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:07.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.950+0000 7f8a9efe2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a980ce660 0x7f8a9815ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:07.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.950+0000 7f8a9efe2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a9815df50 0x7f8a98160980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:07.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.950+0000 7f8a9efe2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a980d2190 con 0x7f8a9815df50 2024-04-25T09:02:07.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.950+0000 7f8a9efe2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a980c77d0 con 0x7f8a980ce660 2024-04-25T09:02:07.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.950+0000 7f8a9efe2640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a98162640 con 0x7f8a980c8d00 2024-04-25T09:02:07.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.950+0000 7f8a9d57f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a9815df50 0x7f8a98160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:07.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.951+0000 7f8a9d57f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a9815df50 0x7f8a98160980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43336/0 (socket says 172.21.15.43:43336) 2024-04-25T09:02:07.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.951+0000 7f8a9d57f640 1 -- 172.21.15.43:0/4142685353 learned_addr learned my addr 172.21.15.43:0/4142685353 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:07.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.951+0000 7f8a9cd7e640 1 --2- 172.21.15.43:0/4142685353 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8a980c8d00 0x7f8a9814dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:07.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.951+0000 7f8a97fff640 1 --2- 172.21.15.43:0/4142685353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a980ce660 0x7f8a9815ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:07.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.951+0000 7f8a9d57f640 1 -- 172.21.15.43:0/4142685353 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8a980c8d00 msgr2=0x7f8a9814dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:07.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.951+0000 7f8a9d57f640 1 --2- 172.21.15.43:0/4142685353 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8a980c8d00 0x7f8a9814dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:07.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.951+0000 7f8a9d57f640 1 -- 172.21.15.43:0/4142685353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a980ce660 msgr2=0x7f8a9815ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:07.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.951+0000 7f8a9d57f640 1 --2- 172.21.15.43:0/4142685353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a980ce660 0x7f8a9815ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:07.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.951+0000 7f8a9d57f640 1 -- 172.21.15.43:0/4142685353 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a98160500 con 0x7f8a9815df50 2024-04-25T09:02:07.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.951+0000 7f8a9d57f640 1 --2- 172.21.15.43:0/4142685353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a9815df50 0x7f8a98160980 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f8a90007e70 tx=0x7f8a9000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:07.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.951+0000 7f8a957fa640 1 -- 172.21.15.43:0/4142685353 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a90014620 con 0x7f8a9815df50 2024-04-25T09:02:07.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.951+0000 7f8a957fa640 1 -- 172.21.15.43:0/4142685353 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a90019070 con 0x7f8a9815df50 2024-04-25T09:02:07.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.951+0000 7f8a957fa640 1 -- 172.21.15.43:0/4142685353 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a90014a20 con 0x7f8a9815df50 2024-04-25T09:02:07.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.951+0000 7f8a9efe2640 1 -- 172.21.15.43:0/4142685353 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a980ccb60 con 0x7f8a9815df50 2024-04-25T09:02:07.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.951+0000 7f8a9efe2640 1 -- 172.21.15.43:0/4142685353 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a980cd120 con 0x7f8a9815df50 2024-04-25T09:02:07.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.952+0000 7f8a9efe2640 1 -- 172.21.15.43:0/4142685353 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a980680f0 con 0x7f8a9815df50 2024-04-25T09:02:07.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.953+0000 7f8a957fa640 1 -- 172.21.15.43:0/4142685353 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8a90014bc0 con 0x7f8a9815df50 2024-04-25T09:02:07.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.953+0000 7f8a957fa640 1 --2- 172.21.15.43:0/4142685353 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8a7c06b4d0 0x7f8a7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:07.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.953+0000 7f8a957fa640 1 -- 172.21.15.43:0/4142685353 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8a9008aa70 con 0x7f8a9815df50 2024-04-25T09:02:07.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.953+0000 7f8a9cd7e640 1 --2- 172.21.15.43:0/4142685353 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8a7c06b4d0 0x7f8a7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:07.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.954+0000 7f8a9cd7e640 1 --2- 172.21.15.43:0/4142685353 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8a7c06b4d0 0x7f8a7c06d990 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c000f60 tx=0x7f8a8c007c00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:07.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:07.957+0000 7f8a957fa640 1 -- 172.21.15.43:0/4142685353 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8a90059980 con 0x7f8a9815df50 2024-04-25T09:02:08.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:08.211+0000 7f8a9efe2640 1 -- 172.21.15.43:0/4142685353 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f8a98074ef0 con 0x7f8a9815df50 2024-04-25T09:02:08.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:08.212+0000 7f8a957fa640 1 -- 172.21.15.43:0/4142685353 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7f8a90059280 con 0x7f8a9815df50 2024-04-25T09:02:08.200 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:08.200 INFO:teuthology.orchestra.run.smithi043.stdout:{"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":500,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":102,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035238,"num_in_osds":8,"osd_in_since":1714035229,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1077815292,"bytes_used":2677039104,"bytes_avail":770417074176,"bytes_total":773094113280,"write_bytes_sec":25917664,"read_op_per_sec":0,"write_op_per_sec":27},"fsmap":{"epoch":61,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"b","status":"up:active","gid":8094}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":18,"modified":"2024-04-25T09:02:01.803060+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:02:08.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:08.214+0000 7f8a9efe2640 1 -- 172.21.15.43:0/4142685353 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8a7c06b4d0 msgr2=0x7f8a7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:08.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:08.214+0000 7f8a9efe2640 1 --2- 172.21.15.43:0/4142685353 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8a7c06b4d0 0x7f8a7c06d990 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c000f60 tx=0x7f8a8c007c00 comp rx=0 tx=0).stop 2024-04-25T09:02:08.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:08.214+0000 7f8a9efe2640 1 -- 172.21.15.43:0/4142685353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a9815df50 msgr2=0x7f8a98160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:08.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:08.214+0000 7f8a9efe2640 1 --2- 172.21.15.43:0/4142685353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a9815df50 0x7f8a98160980 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f8a90007e70 tx=0x7f8a9000e4b0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:08.214+0000 7f8a9efe2640 1 -- 172.21.15.43:0/4142685353 shutdown_connections 2024-04-25T09:02:08.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:08.214+0000 7f8a9efe2640 1 --2- 172.21.15.43:0/4142685353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8a9815df50 0x7f8a98160980 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:08.214+0000 7f8a9efe2640 1 --2- 172.21.15.43:0/4142685353 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8a7c06b4d0 0x7f8a7c06d990 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:08.214+0000 7f8a9efe2640 1 --2- 172.21.15.43:0/4142685353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8a980ce660 0x7f8a9815ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:08.214+0000 7f8a9efe2640 1 --2- 172.21.15.43:0/4142685353 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8a980c8d00 0x7f8a9814dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:08.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:08.214+0000 7f8a9efe2640 1 -- 172.21.15.43:0/4142685353 >> 172.21.15.43:0/4142685353 conn(0x7f8a980d3a30 msgr2=0x7f8a980c8480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:08.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:08.214+0000 7f8a9efe2640 1 -- 172.21.15.43:0/4142685353 shutdown_connections 2024-04-25T09:02:08.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:08.214+0000 7f8a9efe2640 1 -- 172.21.15.43:0/4142685353 wait complete. 2024-04-25T09:02:08.216 INFO:tasks.cephfs.filesystem.ceph_manager:{'b': 'paused'} 2024-04-25T09:02:09.217 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:02:09.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.342+0000 7f82d8c58640 1 Processor -- start 2024-04-25T09:02:09.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.342+0000 7f82d8c58640 1 -- start start 2024-04-25T09:02:09.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.343+0000 7f82d8c58640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82d40cf670 0x7f82d4168d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:09.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.343+0000 7f82d8c58640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82d4162810 con 0x7f82d40c8d00 2024-04-25T09:02:09.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.343+0000 7f82d8c58640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82d40d24a0 con 0x7f82d415df50 2024-04-25T09:02:09.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.343+0000 7f82d8c58640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82d415e330 con 0x7f82d40cf670 2024-04-25T09:02:09.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.343+0000 7f82cbfff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f82d40c8d00 0x7f82d4162100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48780/0 (socket says 172.21.15.43:48780) 2024-04-25T09:02:09.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.343+0000 7f82cbfff640 1 -- 172.21.15.43:0/2601658190 learned_addr learned my addr 172.21.15.43:0/2601658190 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:09.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.343+0000 7f82d2d9d640 1 --2- 172.21.15.43:0/2601658190 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82d40cf670 0x7f82d4168d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:09.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.343+0000 7f82d1d9b640 1 -- 172.21.15.43:0/2601658190 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 376121 0 0) 0x7f82d4162810 con 0x7f82d40c8d00 2024-04-25T09:02:09.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.343+0000 7f82d1d9b640 1 -- 172.21.15.43:0/2601658190 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82c0003520 con 0x7f82d40c8d00 2024-04-25T09:02:09.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.344+0000 7f82d1d9b640 1 -- 172.21.15.43:0/2601658190 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1545864470 0 0) 0x7f82d40d24a0 con 0x7f82d415df50 2024-04-25T09:02:09.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.344+0000 7f82d1d9b640 1 -- 172.21.15.43:0/2601658190 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82d4162810 con 0x7f82d415df50 2024-04-25T09:02:09.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.344+0000 7f82d1d9b640 1 -- 172.21.15.43:0/2601658190 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2688674746 0 0) 0x7f82c0003520 con 0x7f82d40c8d00 2024-04-25T09:02:09.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.344+0000 7f82d1d9b640 1 -- 172.21.15.43:0/2601658190 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82d40cf670 msgr2=0x7f82d4168d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:09.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.344+0000 7f82d1d9b640 1 --2- 172.21.15.43:0/2601658190 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82d40cf670 0x7f82d4168d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:09.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.344+0000 7f82d1d9b640 1 -- 172.21.15.43:0/2601658190 >> v1:172.21.15.193:6789/0 conn(0x7f82d415df50 legacy=0x7f82d41619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:09.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.344+0000 7f82d1d9b640 1 -- 172.21.15.43:0/2601658190 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d40d2740 con 0x7f82d40c8d00 2024-04-25T09:02:09.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.344+0000 7f82d1d9b640 1 -- 172.21.15.43:0/2601658190 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f82bc001670 con 0x7f82d40c8d00 2024-04-25T09:02:09.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.344+0000 7f82d1d9b640 1 -- 172.21.15.43:0/2601658190 >> v1:172.21.15.43:6789/0 conn(0x7f82d40c8d00 legacy=0x7f82d4162100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:09.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.344+0000 7f82d1d9b640 1 --2- 172.21.15.43:0/2601658190 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82c0003b70 0x7f82c0003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:09.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.344+0000 7f82d1d9b640 1 --2- 172.21.15.43:0/2601658190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82c0004800 0x7f82c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:09.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.344+0000 7f82d1d9b640 1 --2- 172.21.15.43:0/2601658190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82c00051c0 0x7f82c000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:09.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.344+0000 7f82d1d9b640 1 -- 172.21.15.43:0/2601658190 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d415e330 con 0x7f82c00051c0 2024-04-25T09:02:09.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.344+0000 7f82d1d9b640 1 -- 172.21.15.43:0/2601658190 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82c000bfb0 con 0x7f82c0004800 2024-04-25T09:02:09.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.344+0000 7f82d1d9b640 1 -- 172.21.15.43:0/2601658190 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82c000c1b0 con 0x7f82c0003b70 2024-04-25T09:02:09.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.344+0000 7f82d2d9d640 1 --2- 172.21.15.43:0/2601658190 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82d40cf670 0x7f82d4168d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:02:09.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.344+0000 7f82cbfff640 1 --2- 172.21.15.43:0/2601658190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82c0004800 0x7f82c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:09.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.344+0000 7f82d2d9d640 1 --2- 172.21.15.43:0/2601658190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82c00051c0 0x7f82c000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:09.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.344+0000 7f82d259c640 1 --2- 172.21.15.43:0/2601658190 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82c0003b70 0x7f82c0003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:09.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.344+0000 7f82d2d9d640 1 -- 172.21.15.43:0/2601658190 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82c0003b70 msgr2=0x7f82c0003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:09.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.344+0000 7f82d2d9d640 1 --2- 172.21.15.43:0/2601658190 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82c0003b70 0x7f82c0003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:09.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.344+0000 7f82d2d9d640 1 -- 172.21.15.43:0/2601658190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82c0004800 msgr2=0x7f82c0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:09.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.344+0000 7f82d2d9d640 1 --2- 172.21.15.43:0/2601658190 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82c0004800 0x7f82c0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:09.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.344+0000 7f82d2d9d640 1 -- 172.21.15.43:0/2601658190 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82c0003520 con 0x7f82c00051c0 2024-04-25T09:02:09.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.345+0000 7f82d2d9d640 1 --2- 172.21.15.43:0/2601658190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82c00051c0 0x7f82c000b9c0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f82d40cfa50 tx=0x7f82c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c6b3fc378a457f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:09.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.345+0000 7f82d1d9b640 1 -- 172.21.15.43:0/2601658190 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc036070 con 0x7f82c00051c0 2024-04-25T09:02:09.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.345+0000 7f82d1d9b640 1 -- 172.21.15.43:0/2601658190 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82cc003c50 con 0x7f82c00051c0 2024-04-25T09:02:09.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.345+0000 7f82d1d9b640 1 -- 172.21.15.43:0/2601658190 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc009d40 con 0x7f82c00051c0 2024-04-25T09:02:09.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.346+0000 7f82d8c58640 1 -- 172.21.15.43:0/2601658190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82c00051c0 msgr2=0x7f82c000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:09.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.346+0000 7f82d8c58640 1 --2- 172.21.15.43:0/2601658190 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82c00051c0 0x7f82c000b9c0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f82d40cfa50 tx=0x7f82c0005830 comp rx=0 tx=0).stop 2024-04-25T09:02:09.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.346+0000 7f82d2d9d640 1 -- 172.21.15.43:0/2601658190 reap_dead start 2024-04-25T09:02:09.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.346+0000 7f82d8c58640 1 -- 172.21.15.43:0/2601658190 shutdown_connections 2024-04-25T09:02:09.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.346+0000 7f82d8c58640 1 -- 172.21.15.43:0/2601658190 >> 172.21.15.43:0/2601658190 conn(0x7f82d40d3a30 msgr2=0x7f82d40d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:09.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.346+0000 7f82d8c58640 1 -- 172.21.15.43:0/2601658190 shutdown_connections 2024-04-25T09:02:09.338 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.346+0000 7f82d8c58640 1 -- 172.21.15.43:0/2601658190 wait complete. 2024-04-25T09:02:09.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.347+0000 7f82d8c58640 1 Processor -- start 2024-04-25T09:02:09.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.347+0000 7f82d8c58640 1 -- start start 2024-04-25T09:02:09.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.348+0000 7f82d8c58640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82c00051c0 0x7f82d41608a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:09.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.348+0000 7f82d8c58640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82d415e4f0 0x7f82d41491b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:09.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.348+0000 7f82d8c58640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82d414f790 0x7f82d414fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:09.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.348+0000 7f82d8c58640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d40c77d0 con 0x7f82d414f790 2024-04-25T09:02:09.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.348+0000 7f82d8c58640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d40c74f0 con 0x7f82c00051c0 2024-04-25T09:02:09.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.348+0000 7f82d8c58640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82d40d2190 con 0x7f82d415e4f0 2024-04-25T09:02:09.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.348+0000 7f82d2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82d414f790 0x7f82d414fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:09.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.348+0000 7f82d2d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82d414f790 0x7f82d414fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43366/0 (socket says 172.21.15.43:43366) 2024-04-25T09:02:09.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.348+0000 7f82d2d9d640 1 -- 172.21.15.43:0/1255501475 learned_addr learned my addr 172.21.15.43:0/1255501475 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:09.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.348+0000 7f82cbfff640 1 --2- 172.21.15.43:0/1255501475 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82d415e4f0 0x7f82d41491b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:09.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.348+0000 7f82d259c640 1 --2- 172.21.15.43:0/1255501475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82c00051c0 0x7f82d41608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:09.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.348+0000 7f82d2d9d640 1 -- 172.21.15.43:0/1255501475 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82d415e4f0 msgr2=0x7f82d41491b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:09.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.348+0000 7f82d2d9d640 1 --2- 172.21.15.43:0/1255501475 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82d415e4f0 0x7f82d41491b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:09.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.348+0000 7f82d2d9d640 1 -- 172.21.15.43:0/1255501475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82c00051c0 msgr2=0x7f82d41608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:09.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.348+0000 7f82d2d9d640 1 --2- 172.21.15.43:0/1255501475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82c00051c0 0x7f82d41608a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:09.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.348+0000 7f82d2d9d640 1 -- 172.21.15.43:0/1255501475 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d415e970 con 0x7f82d414f790 2024-04-25T09:02:09.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.349+0000 7f82d2d9d640 1 --2- 172.21.15.43:0/1255501475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82d414f790 0x7f82d414fc10 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f82cc02af80 tx=0x7f82cc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:09.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.349+0000 7f82caffd640 1 -- 172.21.15.43:0/1255501475 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc036070 con 0x7f82d414f790 2024-04-25T09:02:09.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.349+0000 7f82caffd640 1 -- 172.21.15.43:0/1255501475 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82cc041070 con 0x7f82d414f790 2024-04-25T09:02:09.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.349+0000 7f82d8c58640 1 -- 172.21.15.43:0/1255501475 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82d41611b0 con 0x7f82d414f790 2024-04-25T09:02:09.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.349+0000 7f82caffd640 1 -- 172.21.15.43:0/1255501475 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82cc009d40 con 0x7f82d414f790 2024-04-25T09:02:09.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.349+0000 7f82d8c58640 1 -- 172.21.15.43:0/1255501475 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82d4160ed0 con 0x7f82d414f790 2024-04-25T09:02:09.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.350+0000 7f82d8c58640 1 -- 172.21.15.43:0/1255501475 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82d40680f0 con 0x7f82d414f790 2024-04-25T09:02:09.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.350+0000 7f82caffd640 1 -- 172.21.15.43:0/1255501475 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f82cc005d10 con 0x7f82d414f790 2024-04-25T09:02:09.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.351+0000 7f82caffd640 1 --2- 172.21.15.43:0/1255501475 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f82b406b4d0 0x7f82b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:09.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.351+0000 7f82caffd640 1 -- 172.21.15.43:0/1255501475 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82cc0aa680 con 0x7f82d414f790 2024-04-25T09:02:09.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.351+0000 7f82d259c640 1 --2- 172.21.15.43:0/1255501475 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f82b406b4d0 0x7f82b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:09.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.352+0000 7f82d259c640 1 --2- 172.21.15.43:0/1255501475 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f82b406b4d0 0x7f82b406d990 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f82d40a1940 tx=0x7f82c4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:09.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.355+0000 7f82caffd640 1 -- 172.21.15.43:0/1255501475 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f82cc079590 con 0x7f82d414f790 2024-04-25T09:02:09.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.573+0000 7f82d8c58640 1 -- 172.21.15.43:0/1255501475 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82d4074ef0 con 0x7f82d414f790 2024-04-25T09:02:09.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.573+0000 7f82caffd640 1 -- 172.21.15.43:0/1255501475 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f82cc078e90 con 0x7f82d414f790 2024-04-25T09:02:09.563 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:09.563 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":61,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2150177203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2150177203},{"type":"v1","addr":"172.21.15.193:6837","nonce":2150177203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":8079,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/621488642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":621488642},{"type":"v1","addr":"172.21.15.43:6834","nonce":621488642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8082,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1524395840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1524395840},{"type":"v1","addr":"172.21.15.43:6835","nonce":1524395840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:01:33.726790+0000","modified":"2024-04-25T09:01:35.706689+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8094},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8094":{"gid":8094,"name":"b","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.193:6836/853943267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":853943267},{"type":"v1","addr":"172.21.15.193:6836","nonce":853943267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:02:09.563 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 61 2024-04-25T09:02:09.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.575+0000 7f82d8c58640 1 -- 172.21.15.43:0/1255501475 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f82b406b4d0 msgr2=0x7f82b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:09.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.575+0000 7f82d8c58640 1 --2- 172.21.15.43:0/1255501475 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f82b406b4d0 0x7f82b406d990 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f82d40a1940 tx=0x7f82c4009040 comp rx=0 tx=0).stop 2024-04-25T09:02:09.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.575+0000 7f82d8c58640 1 -- 172.21.15.43:0/1255501475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82d414f790 msgr2=0x7f82d414fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:09.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.575+0000 7f82d8c58640 1 --2- 172.21.15.43:0/1255501475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82d414f790 0x7f82d414fc10 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f82cc02af80 tx=0x7f82cc009750 comp rx=0 tx=0).stop 2024-04-25T09:02:09.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.575+0000 7f82d8c58640 1 -- 172.21.15.43:0/1255501475 shutdown_connections 2024-04-25T09:02:09.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.575+0000 7f82d8c58640 1 --2- 172.21.15.43:0/1255501475 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f82d414f790 0x7f82d414fc10 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:09.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.575+0000 7f82d8c58640 1 --2- 172.21.15.43:0/1255501475 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82d415e4f0 0x7f82d41491b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:09.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.575+0000 7f82d8c58640 1 --2- 172.21.15.43:0/1255501475 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f82b406b4d0 0x7f82b406d990 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:09.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.575+0000 7f82d8c58640 1 --2- 172.21.15.43:0/1255501475 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f82c00051c0 0x7f82d41608a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:09.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.575+0000 7f82d8c58640 1 -- 172.21.15.43:0/1255501475 >> 172.21.15.43:0/1255501475 conn(0x7f82d40d3a30 msgr2=0x7f82d4152ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:09.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.575+0000 7f82d8c58640 1 -- 172.21.15.43:0/1255501475 shutdown_connections 2024-04-25T09:02:09.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.575+0000 7f82d8c58640 1 -- 172.21.15.43:0/1255501475 wait complete. 2024-04-25T09:02:09.575 DEBUG:tasks.cephfs.test_scrub_checks:current active=['b'] 2024-04-25T09:02:09.575 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:02:09.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.686+0000 7f5c09310640 1 Processor -- start 2024-04-25T09:02:09.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.686+0000 7f5c09310640 1 -- start start 2024-04-25T09:02:09.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.687+0000 7f5c09310640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f5c0406f0e0 0x7f5c040b3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:09.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.687+0000 7f5c09310640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c0406f930 0x7f5c040b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:09.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.687+0000 7f5c09310640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c0406f4c0 con 0x7f5c0415aee0 2024-04-25T09:02:09.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.687+0000 7f5c09310640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c04072130 con 0x7f5c0406f0e0 2024-04-25T09:02:09.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.687+0000 7f5c09310640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c040b3360 con 0x7f5c0406f930 2024-04-25T09:02:09.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.687+0000 7f5c027fc640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c0406f930 0x7f5c040b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:09.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.687+0000 7f5c037fe640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f5c0415aee0 0x7f5c0415e4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:52866/0 (socket says 172.21.15.43:52866) 2024-04-25T09:02:09.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.688+0000 7f5c027fc640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c0406f930 0x7f5c040b3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:60086/0 (socket says 172.21.15.43:60086) 2024-04-25T09:02:09.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.687+0000 7f5c02ffd640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f5c0406f0e0 0x7f5c040b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:09.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.688+0000 7f5c037fe640 1 -- 172.21.15.43:0/306619339 learned_addr learned my addr 172.21.15.43:0/306619339 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:09.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.688+0000 7f5c01ffb640 1 -- 172.21.15.43:0/306619339 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4000442879 0 0) 0x7f5c0406f4c0 con 0x7f5c0415aee0 2024-04-25T09:02:09.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.688+0000 7f5c01ffb640 1 -- 172.21.15.43:0/306619339 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bec003520 con 0x7f5c0415aee0 2024-04-25T09:02:09.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.688+0000 7f5c027fc640 1 -- 172.21.15.43:0/306619339 >> v2:172.21.15.193:3300/0 conn(0x7f5c0406f0e0 msgr2=0x7f5c040b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:09.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.688+0000 7f5c027fc640 1 --2- 172.21.15.43:0/306619339 >> v2:172.21.15.193:3300/0 conn(0x7f5c0406f0e0 0x7f5c040b3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:09.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.688+0000 7f5c027fc640 1 -- 172.21.15.43:0/306619339 >> v1:172.21.15.193:6789/0 conn(0x7f5c0415aee0 legacy=0x7f5c0415e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:09.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.688+0000 7f5c027fc640 1 -- 172.21.15.43:0/306619339 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c04071520 con 0x7f5c0406f930 2024-04-25T09:02:09.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.688+0000 7f5c02ffd640 1 --2- 172.21.15.43:0/306619339 >> v2:172.21.15.193:3300/0 conn(0x7f5c0406f0e0 0x7f5c040b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:02:09.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.689+0000 7f5c027fc640 1 --2- 172.21.15.43:0/306619339 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c0406f930 0x7f5c040b3cd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5bf800ba40 tx=0x7f5bf802c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47b1cc14b4d5909 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:09.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.689+0000 7f5c01ffb640 1 -- 172.21.15.43:0/306619339 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf8032750 con 0x7f5c0406f930 2024-04-25T09:02:09.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.689+0000 7f5c01ffb640 1 -- 172.21.15.43:0/306619339 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bf8030050 con 0x7f5c0406f930 2024-04-25T09:02:09.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.689+0000 7f5c01ffb640 1 -- 172.21.15.43:0/306619339 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf8032af0 con 0x7f5c0406f930 2024-04-25T09:02:09.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.689+0000 7f5c09310640 1 -- 172.21.15.43:0/306619339 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c0406f930 msgr2=0x7f5c040b3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:09.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.689+0000 7f5c09310640 1 --2- 172.21.15.43:0/306619339 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c0406f930 0x7f5c040b3cd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5bf800ba40 tx=0x7f5bf802c850 comp rx=0 tx=0).stop 2024-04-25T09:02:09.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.689+0000 7f5c09310640 1 -- 172.21.15.43:0/306619339 shutdown_connections 2024-04-25T09:02:09.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.689+0000 7f5c09310640 1 --2- 172.21.15.43:0/306619339 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c0406f930 0x7f5c040b3cd0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:09.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.689+0000 7f5c09310640 1 --2- 172.21.15.43:0/306619339 >> v2:172.21.15.193:3300/0 conn(0x7f5c0406f0e0 0x7f5c040b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:09.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.689+0000 7f5c09310640 1 -- 172.21.15.43:0/306619339 >> 172.21.15.43:0/306619339 conn(0x7f5c040b2490 msgr2=0x7f5c040b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:09.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.689+0000 7f5c09310640 1 -- 172.21.15.43:0/306619339 shutdown_connections 2024-04-25T09:02:09.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.689+0000 7f5c09310640 1 -- 172.21.15.43:0/306619339 wait complete. 2024-04-25T09:02:09.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.691+0000 7f5c09310640 1 Processor -- start 2024-04-25T09:02:09.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.691+0000 7f5c09310640 1 -- start start 2024-04-25T09:02:09.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.691+0000 7f5c09310640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c0406f0e0 0x7f5c04153290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:09.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.691+0000 7f5c09310640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c0406f930 0x7f5c04158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:09.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.691+0000 7f5c09310640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c0415aee0 0x7f5c041591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:09.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.691+0000 7f5c09310640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c04071dc0 con 0x7f5c0415aee0 2024-04-25T09:02:09.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.691+0000 7f5c09310640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c04071ae0 con 0x7f5c0406f0e0 2024-04-25T09:02:09.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.691+0000 7f5c09310640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0406e460 con 0x7f5c0406f930 2024-04-25T09:02:09.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.692+0000 7f5c037fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c0415aee0 0x7f5c041591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:09.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.692+0000 7f5c037fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c0415aee0 0x7f5c041591f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51082/0 (socket says 172.21.15.43:51082) 2024-04-25T09:02:09.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.692+0000 7f5c037fe640 1 -- 172.21.15.43:0/1842215868 learned_addr learned my addr 172.21.15.43:0/1842215868 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:09.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.692+0000 7f5c02ffd640 1 --2- 172.21.15.43:0/1842215868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c0406f0e0 0x7f5c04153290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:09.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.692+0000 7f5c027fc640 1 --2- 172.21.15.43:0/1842215868 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c0406f930 0x7f5c04158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:09.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.692+0000 7f5c037fe640 1 -- 172.21.15.43:0/1842215868 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c0406f930 msgr2=0x7f5c04158cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:09.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.692+0000 7f5c037fe640 1 --2- 172.21.15.43:0/1842215868 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c0406f930 0x7f5c04158cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:09.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.692+0000 7f5c037fe640 1 -- 172.21.15.43:0/1842215868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c0406f0e0 msgr2=0x7f5c04153290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:09.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.692+0000 7f5c037fe640 1 --2- 172.21.15.43:0/1842215868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c0406f0e0 0x7f5c04153290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:09.683 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.692+0000 7f5c037fe640 1 -- 172.21.15.43:0/1842215868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c041f3430 con 0x7f5c0415aee0 2024-04-25T09:02:09.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.692+0000 7f5c037fe640 1 --2- 172.21.15.43:0/1842215868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c0415aee0 0x7f5c041591f0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f5bf400cad0 tx=0x7f5bf400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:09.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.692+0000 7f5beb7fe640 1 -- 172.21.15.43:0/1842215868 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf4017070 con 0x7f5c0415aee0 2024-04-25T09:02:09.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.692+0000 7f5beb7fe640 1 -- 172.21.15.43:0/1842215868 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bf400ac30 con 0x7f5c0415aee0 2024-04-25T09:02:09.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.692+0000 7f5c09310640 1 -- 172.21.15.43:0/1842215868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c04153f50 con 0x7f5c0415aee0 2024-04-25T09:02:09.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.692+0000 7f5beb7fe640 1 -- 172.21.15.43:0/1842215868 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf4012400 con 0x7f5c0415aee0 2024-04-25T09:02:09.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.692+0000 7f5c09310640 1 -- 172.21.15.43:0/1842215868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c04154510 con 0x7f5c0415aee0 2024-04-25T09:02:09.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.693+0000 7f5c09310640 1 -- 172.21.15.43:0/1842215868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c04074ef0 con 0x7f5c0415aee0 2024-04-25T09:02:09.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.694+0000 7f5beb7fe640 1 -- 172.21.15.43:0/1842215868 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5bf401a070 con 0x7f5c0415aee0 2024-04-25T09:02:09.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.694+0000 7f5beb7fe640 1 --2- 172.21.15.43:0/1842215868 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5bf006b4d0 0x7f5bf006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:09.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.694+0000 7f5beb7fe640 1 -- 172.21.15.43:0/1842215868 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5bf4089e90 con 0x7f5c0415aee0 2024-04-25T09:02:09.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.694+0000 7f5c02ffd640 1 --2- 172.21.15.43:0/1842215868 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5bf006b4d0 0x7f5bf006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:09.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.695+0000 7f5c02ffd640 1 --2- 172.21.15.43:0/1842215868 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5bf006b4d0 0x7f5bf006d990 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc009560 tx=0x7f5bfc009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:09.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.698+0000 7f5beb7fe640 1 -- 172.21.15.43:0/1842215868 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5bf4058750 con 0x7f5c0415aee0 2024-04-25T09:02:09.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.948+0000 7f5c09310640 1 -- 172.21.15.43:0/1842215868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f5c0406b8e0 con 0x7f5c0415aee0 2024-04-25T09:02:09.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.949+0000 7f5beb7fe640 1 -- 172.21.15.43:0/1842215868 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1575 (secure 0 0 0) 0x7f5bf405c7f0 con 0x7f5c0415aee0 2024-04-25T09:02:09.938 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:09.938 INFO:teuthology.orchestra.run.smithi043.stdout:{"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":502,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":102,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035238,"num_in_osds":8,"osd_in_since":1714035229,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1077815292,"bytes_used":2677039104,"bytes_avail":770417074176,"bytes_total":773094113280,"write_bytes_sec":25917874,"read_op_per_sec":0,"write_op_per_sec":27},"fsmap":{"epoch":61,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"b","status":"up:active","gid":8094}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":19,"modified":"2024-04-25T09:02:07.804567+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:02:09.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.951+0000 7f5c09310640 1 -- 172.21.15.43:0/1842215868 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5bf006b4d0 msgr2=0x7f5bf006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:09.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.951+0000 7f5c09310640 1 --2- 172.21.15.43:0/1842215868 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5bf006b4d0 0x7f5bf006d990 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f5bfc009560 tx=0x7f5bfc009190 comp rx=0 tx=0).stop 2024-04-25T09:02:09.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.951+0000 7f5c09310640 1 -- 172.21.15.43:0/1842215868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c0415aee0 msgr2=0x7f5c041591f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:09.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.951+0000 7f5c09310640 1 --2- 172.21.15.43:0/1842215868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c0415aee0 0x7f5c041591f0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f5bf400cad0 tx=0x7f5bf400cf90 comp rx=0 tx=0).stop 2024-04-25T09:02:09.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.951+0000 7f5c09310640 1 -- 172.21.15.43:0/1842215868 shutdown_connections 2024-04-25T09:02:09.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.951+0000 7f5c09310640 1 --2- 172.21.15.43:0/1842215868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5c0415aee0 0x7f5c041591f0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:09.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.951+0000 7f5c09310640 1 --2- 172.21.15.43:0/1842215868 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5c0406f930 0x7f5c04158cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:09.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.951+0000 7f5c09310640 1 --2- 172.21.15.43:0/1842215868 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5bf006b4d0 0x7f5bf006d990 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:09.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.951+0000 7f5c09310640 1 --2- 172.21.15.43:0/1842215868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5c0406f0e0 0x7f5c04153290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:09.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.951+0000 7f5c09310640 1 -- 172.21.15.43:0/1842215868 >> 172.21.15.43:0/1842215868 conn(0x7f5c040b2490 msgr2=0x7f5c0415b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:09.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.951+0000 7f5c09310640 1 -- 172.21.15.43:0/1842215868 shutdown_connections 2024-04-25T09:02:09.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:09.951+0000 7f5c09310640 1 -- 172.21.15.43:0/1842215868 wait complete. 2024-04-25T09:02:09.951 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T09:02:09.951 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:02:09.951 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:02:09.986 DEBUG:teuthology.orchestra.run.smithi193:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:02:10.039 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:02:10.039 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T09:02:10.039 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:02:10.055 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:02:10.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.207+0000 7fc5dfff9640 1 Processor -- start 2024-04-25T09:02:10.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.207+0000 7fc5dfff9640 1 -- start start 2024-04-25T09:02:10.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.208+0000 7fc5dfff9640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc5d80c8d00 0x7fc5d8162100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:10.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.208+0000 7fc5dfff9640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5d80d24a0 con 0x7fc5d815df50 2024-04-25T09:02:10.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.208+0000 7fc5dfff9640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5d80d2740 con 0x7fc5d80ce660 2024-04-25T09:02:10.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.208+0000 7fc5dfff9640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5d815e330 con 0x7fc5d80c8d00 2024-04-25T09:02:10.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.208+0000 7fc5ddd95640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc5d815df50 0x7fc5d81619f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32842/0 (socket says 172.21.15.43:32842) 2024-04-25T09:02:10.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.208+0000 7fc5dd594640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc5d80c8d00 0x7fc5d8162100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:10.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.208+0000 7fc5ddd95640 1 -- 172.21.15.43:0/1260667343 learned_addr learned my addr 172.21.15.43:0/1260667343 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:10.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.209+0000 7fc5dd594640 1 -- 172.21.15.43:0/1260667343 >> v1:172.21.15.193:6789/0 conn(0x7fc5d80ce660 legacy=0x7fc5d80cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:10.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.209+0000 7fc5dd594640 1 -- 172.21.15.43:0/1260667343 >> v1:172.21.15.43:6789/0 conn(0x7fc5d815df50 legacy=0x7fc5d81619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:10.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.209+0000 7fc5dcd93640 1 -- 172.21.15.43:0/1260667343 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 723268480 0 0) 0x7fc5d80d24a0 con 0x7fc5d815df50 2024-04-25T09:02:10.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.209+0000 7fc5dd594640 1 -- 172.21.15.43:0/1260667343 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d80d2a20 con 0x7fc5d80c8d00 2024-04-25T09:02:10.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.209+0000 7fc5dd594640 1 --2- 172.21.15.43:0/1260667343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5d80c8d00 0x7fc5d8162100 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc0097a0 tx=0x7fc5cc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e9f07087a92257d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:10.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.210+0000 7fc5dcd93640 1 -- 172.21.15.43:0/1260667343 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc037070 con 0x7fc5d80c8d00 2024-04-25T09:02:10.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.210+0000 7fc5dcd93640 1 -- 172.21.15.43:0/1260667343 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5cc032440 con 0x7fc5d80c8d00 2024-04-25T09:02:10.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.210+0000 7fc5dcd93640 1 -- 172.21.15.43:0/1260667343 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc032810 con 0x7fc5d80c8d00 2024-04-25T09:02:10.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.210+0000 7fc5dfff9640 1 -- 172.21.15.43:0/1260667343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5d80c8d00 msgr2=0x7fc5d8162100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:10.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.210+0000 7fc5dfff9640 1 --2- 172.21.15.43:0/1260667343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5d80c8d00 0x7fc5d8162100 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc0097a0 tx=0x7fc5cc02a1e0 comp rx=0 tx=0).stop 2024-04-25T09:02:10.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.210+0000 7fc5dfff9640 1 -- 172.21.15.43:0/1260667343 shutdown_connections 2024-04-25T09:02:10.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.210+0000 7fc5dfff9640 1 --2- 172.21.15.43:0/1260667343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5d80c8d00 0x7fc5d8162100 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:10.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.210+0000 7fc5dfff9640 1 -- 172.21.15.43:0/1260667343 >> 172.21.15.43:0/1260667343 conn(0x7fc5d80d3a30 msgr2=0x7fc5d80d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:10.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.210+0000 7fc5dfff9640 1 -- 172.21.15.43:0/1260667343 shutdown_connections 2024-04-25T09:02:10.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.210+0000 7fc5dfff9640 1 -- 172.21.15.43:0/1260667343 wait complete. 2024-04-25T09:02:10.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.211+0000 7fc5dfff9640 1 Processor -- start 2024-04-25T09:02:10.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.211+0000 7fc5dfff9640 1 -- start start 2024-04-25T09:02:10.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.212+0000 7fc5dfff9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5d80c8d00 0x7fc5d81e8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:10.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.212+0000 7fc5dfff9640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc5d80ce660 0x7fc5d815eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:10.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.212+0000 7fc5dfff9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5d815df50 0x7fc5d81f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:10.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.212+0000 7fc5dfff9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d80d2190 con 0x7fc5d815df50 2024-04-25T09:02:10.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.212+0000 7fc5dfff9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d80c77d0 con 0x7fc5d80c8d00 2024-04-25T09:02:10.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.212+0000 7fc5dfff9640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d8162640 con 0x7fc5d80ce660 2024-04-25T09:02:10.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.212+0000 7fc5de596640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5d815df50 0x7fc5d81f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:10.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.212+0000 7fc5de596640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5d815df50 0x7fc5d81f6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51104/0 (socket says 172.21.15.43:51104) 2024-04-25T09:02:10.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.212+0000 7fc5de596640 1 -- 172.21.15.43:0/713433260 learned_addr learned my addr 172.21.15.43:0/713433260 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:10.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.212+0000 7fc5ddd95640 1 --2- 172.21.15.43:0/713433260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5d80c8d00 0x7fc5d81e8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:10.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.212+0000 7fc5dd594640 1 --2- 172.21.15.43:0/713433260 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc5d80ce660 0x7fc5d815eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:10.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.213+0000 7fc5de596640 1 -- 172.21.15.43:0/713433260 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc5d80ce660 msgr2=0x7fc5d815eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:10.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.213+0000 7fc5de596640 1 --2- 172.21.15.43:0/713433260 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc5d80ce660 0x7fc5d815eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:10.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.213+0000 7fc5de596640 1 -- 172.21.15.43:0/713433260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5d80c8d00 msgr2=0x7fc5d81e8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:10.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.213+0000 7fc5de596640 1 --2- 172.21.15.43:0/713433260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5d80c8d00 0x7fc5d81e8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:10.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.213+0000 7fc5de596640 1 -- 172.21.15.43:0/713433260 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d815f610 con 0x7fc5d815df50 2024-04-25T09:02:10.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.213+0000 7fc5ddd95640 1 --2- 172.21.15.43:0/713433260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5d80c8d00 0x7fc5d81e8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:10.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.213+0000 7fc5de596640 1 --2- 172.21.15.43:0/713433260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5d815df50 0x7fc5d81f6c60 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800c780 tx=0x7fc5c800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:10.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.213+0000 7fc5d67fc640 1 -- 172.21.15.43:0/713433260 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8017070 con 0x7fc5d815df50 2024-04-25T09:02:10.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.213+0000 7fc5d67fc640 1 -- 172.21.15.43:0/713433260 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c8003da0 con 0x7fc5d815df50 2024-04-25T09:02:10.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.213+0000 7fc5dfff9640 1 -- 172.21.15.43:0/713433260 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5d815f8f0 con 0x7fc5d815df50 2024-04-25T09:02:10.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.213+0000 7fc5d67fc640 1 -- 172.21.15.43:0/713433260 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c80052b0 con 0x7fc5d815df50 2024-04-25T09:02:10.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.213+0000 7fc5dfff9640 1 -- 172.21.15.43:0/713433260 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5d8145cd0 con 0x7fc5d815df50 2024-04-25T09:02:10.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.214+0000 7fc5dfff9640 1 -- 172.21.15.43:0/713433260 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5d80680f0 con 0x7fc5d815df50 2024-04-25T09:02:10.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.214+0000 7fc5d67fc640 1 -- 172.21.15.43:0/713433260 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc5c801a070 con 0x7fc5d815df50 2024-04-25T09:02:10.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.215+0000 7fc5d67fc640 1 --2- 172.21.15.43:0/713433260 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc5c406b4d0 0x7fc5c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:10.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.215+0000 7fc5d67fc640 1 -- 172.21.15.43:0/713433260 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc5c80897f0 con 0x7fc5d815df50 2024-04-25T09:02:10.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.215+0000 7fc5ddd95640 1 --2- 172.21.15.43:0/713433260 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc5c406b4d0 0x7fc5c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:10.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.216+0000 7fc5ddd95640 1 --2- 172.21.15.43:0/713433260 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc5c406b4d0 0x7fc5c406d990 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0000f60 tx=0x7fc5d0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:10.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.219+0000 7fc5d67fc640 1 -- 172.21.15.43:0/713433260 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc5c80586e0 con 0x7fc5d815df50 2024-04-25T09:02:10.389 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.401+0000 7fc5dfff9640 1 -- 172.21.15.43:0/713433260 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc5d8074ef0 con 0x7fc5d815df50 2024-04-25T09:02:10.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.402+0000 7fc5d67fc640 1 -- 172.21.15.43:0/713433260 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+16583 (secure 0 0 0) 0x7fc5c8057fe0 con 0x7fc5d815df50 2024-04-25T09:02:10.391 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:10.391 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":102,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:01:35.356828+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:01:31.997813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T09:01:32.695222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:02:10.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.404+0000 7fc5dfff9640 1 -- 172.21.15.43:0/713433260 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc5c406b4d0 msgr2=0x7fc5c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:10.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.404+0000 7fc5dfff9640 1 --2- 172.21.15.43:0/713433260 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc5c406b4d0 0x7fc5c406d990 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0000f60 tx=0x7fc5d0009040 comp rx=0 tx=0).stop 2024-04-25T09:02:10.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.404+0000 7fc5dfff9640 1 -- 172.21.15.43:0/713433260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5d815df50 msgr2=0x7fc5d81f6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:10.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.404+0000 7fc5dfff9640 1 --2- 172.21.15.43:0/713433260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5d815df50 0x7fc5d81f6c60 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800c780 tx=0x7fc5c800cc40 comp rx=0 tx=0).stop 2024-04-25T09:02:10.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.404+0000 7fc5dfff9640 1 -- 172.21.15.43:0/713433260 shutdown_connections 2024-04-25T09:02:10.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.404+0000 7fc5dfff9640 1 --2- 172.21.15.43:0/713433260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5d815df50 0x7fc5d81f6c60 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:10.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.404+0000 7fc5dfff9640 1 --2- 172.21.15.43:0/713433260 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc5d80ce660 0x7fc5d815eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:10.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.404+0000 7fc5dfff9640 1 --2- 172.21.15.43:0/713433260 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc5c406b4d0 0x7fc5c406d990 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:10.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.404+0000 7fc5dfff9640 1 --2- 172.21.15.43:0/713433260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc5d80c8d00 0x7fc5d81e8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:10.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.404+0000 7fc5dfff9640 1 -- 172.21.15.43:0/713433260 >> 172.21.15.43:0/713433260 conn(0x7fc5d80d3a30 msgr2=0x7fc5d80d1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:10.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.404+0000 7fc5dfff9640 1 -- 172.21.15.43:0/713433260 shutdown_connections 2024-04-25T09:02:10.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.404+0000 7fc5dfff9640 1 -- 172.21.15.43:0/713433260 wait complete. 2024-04-25T09:02:10.405 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:01:31.997813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:01:32.695222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:02:10.405 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:02:10.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.521+0000 7f1fffd5c640 1 Processor -- start 2024-04-25T09:02:10.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.521+0000 7f1fffd5c640 1 -- start start 2024-04-25T09:02:10.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.522+0000 7f1fffd5c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1ff806f9a0 0x7f1ff815cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:10.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.522+0000 7f1fffd5c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1ff806b6d0 0x7f1ff80cf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:10.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.522+0000 7f1fffd5c640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ff815c070 con 0x7f1ff80d0af0 2024-04-25T09:02:10.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.522+0000 7f1fffd5c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ff80cfa40 con 0x7f1ff806f9a0 2024-04-25T09:02:10.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.522+0000 7f1fffd5c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff815c310 con 0x7f1ff806b6d0 2024-04-25T09:02:10.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.522+0000 7f1ffdaf8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1ff806f9a0 0x7f1ff815cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:10.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.522+0000 7f1ffdaf8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1ff806f9a0 0x7f1ff815cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51112/0 (socket says 172.21.15.43:51112) 2024-04-25T09:02:10.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.522+0000 7f1ffdaf8640 1 -- 172.21.15.43:0/3817928337 learned_addr learned my addr 172.21.15.43:0/3817928337 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:10.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.523+0000 7f1ffd2f7640 1 --2- 172.21.15.43:0/3817928337 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1ff806b6d0 0x7f1ff80cf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:10.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.523+0000 7f1ffdaf8640 1 -- 172.21.15.43:0/3817928337 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1ff806b6d0 msgr2=0x7f1ff80cf0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:10.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.523+0000 7f1ffdaf8640 1 --2- 172.21.15.43:0/3817928337 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1ff806b6d0 0x7f1ff80cf0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:10.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.523+0000 7f1ffdaf8640 1 -- 172.21.15.43:0/3817928337 >> v1:172.21.15.193:6789/0 conn(0x7f1ff80d0af0 legacy=0x7f1ff815f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:10.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.523+0000 7f1ffdaf8640 1 -- 172.21.15.43:0/3817928337 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff815c5f0 con 0x7f1ff806f9a0 2024-04-25T09:02:10.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.523+0000 7f1ffdaf8640 1 --2- 172.21.15.43:0/3817928337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ff806f9a0 0x7f1ff815cb10 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4009870 tx=0x7f1ff402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9381d681944ea8c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:10.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.524+0000 7f1ffcaf6640 1 -- 172.21.15.43:0/3817928337 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4037070 con 0x7f1ff806f9a0 2024-04-25T09:02:10.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.524+0000 7f1ffcaf6640 1 -- 172.21.15.43:0/3817928337 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff4032440 con 0x7f1ff806f9a0 2024-04-25T09:02:10.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.524+0000 7f1ffcaf6640 1 -- 172.21.15.43:0/3817928337 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4032810 con 0x7f1ff806f9a0 2024-04-25T09:02:10.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.524+0000 7f1fffd5c640 1 -- 172.21.15.43:0/3817928337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ff806f9a0 msgr2=0x7f1ff815cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:10.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.524+0000 7f1fffd5c640 1 --2- 172.21.15.43:0/3817928337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ff806f9a0 0x7f1ff815cb10 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4009870 tx=0x7f1ff402a270 comp rx=0 tx=0).stop 2024-04-25T09:02:10.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.524+0000 7f1fffd5c640 1 -- 172.21.15.43:0/3817928337 shutdown_connections 2024-04-25T09:02:10.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.524+0000 7f1fffd5c640 1 --2- 172.21.15.43:0/3817928337 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1ff806b6d0 0x7f1ff80cf0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:10.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.524+0000 7f1fffd5c640 1 --2- 172.21.15.43:0/3817928337 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ff806f9a0 0x7f1ff815cb10 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:10.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.524+0000 7f1fffd5c640 1 -- 172.21.15.43:0/3817928337 >> 172.21.15.43:0/3817928337 conn(0x7f1ff806c060 msgr2=0x7f1ff806c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:10.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.524+0000 7f1fffd5c640 1 -- 172.21.15.43:0/3817928337 shutdown_connections 2024-04-25T09:02:10.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.524+0000 7f1fffd5c640 1 -- 172.21.15.43:0/3817928337 wait complete. 2024-04-25T09:02:10.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.525+0000 7f1fffd5c640 1 Processor -- start 2024-04-25T09:02:10.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.525+0000 7f1fffd5c640 1 -- start start 2024-04-25T09:02:10.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.526+0000 7f1fffd5c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1ff806b6d0 0x7f1ff8151310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:10.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.526+0000 7f1fffd5c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ff806f9a0 0x7f1ff815a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:10.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.526+0000 7f1fffd5c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ff80d0af0 0x7f1ff815a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:10.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.526+0000 7f1fffd5c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff80cfea0 con 0x7f1ff80d0af0 2024-04-25T09:02:10.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.526+0000 7f1fffd5c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff80cfd20 con 0x7f1ff806f9a0 2024-04-25T09:02:10.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.526+0000 7f1fffd5c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ff80cf600 con 0x7f1ff806b6d0 2024-04-25T09:02:10.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.526+0000 7f1ffe2f9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ff80d0af0 0x7f1ff815a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:10.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.526+0000 7f1ffe2f9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ff80d0af0 0x7f1ff815a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51120/0 (socket says 172.21.15.43:51120) 2024-04-25T09:02:10.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.526+0000 7f1ffe2f9640 1 -- 172.21.15.43:0/2521169670 learned_addr learned my addr 172.21.15.43:0/2521169670 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:10.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.526+0000 7f1ffd2f7640 1 --2- 172.21.15.43:0/2521169670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ff806f9a0 0x7f1ff815a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:10.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.526+0000 7f1ffdaf8640 1 --2- 172.21.15.43:0/2521169670 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1ff806b6d0 0x7f1ff8151310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:10.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.527+0000 7f1ffe2f9640 1 -- 172.21.15.43:0/2521169670 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1ff806b6d0 msgr2=0x7f1ff8151310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:10.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.527+0000 7f1ffe2f9640 1 --2- 172.21.15.43:0/2521169670 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1ff806b6d0 0x7f1ff8151310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:10.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.527+0000 7f1ffe2f9640 1 -- 172.21.15.43:0/2521169670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ff806f9a0 msgr2=0x7f1ff815a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:10.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.527+0000 7f1ffe2f9640 1 --2- 172.21.15.43:0/2521169670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ff806f9a0 0x7f1ff815a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:10.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.527+0000 7f1ffe2f9640 1 -- 172.21.15.43:0/2521169670 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff81557c0 con 0x7f1ff80d0af0 2024-04-25T09:02:10.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.527+0000 7f1ffd2f7640 1 --2- 172.21.15.43:0/2521169670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ff806f9a0 0x7f1ff815a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:10.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.527+0000 7f1ffe2f9640 1 --2- 172.21.15.43:0/2521169670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ff80d0af0 0x7f1ff815a870 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f1ff000c730 tx=0x7f1ff000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:10.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.527+0000 7f1fee7fc640 1 -- 172.21.15.43:0/2521169670 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff0003b90 con 0x7f1ff80d0af0 2024-04-25T09:02:10.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.527+0000 7f1fee7fc640 1 -- 172.21.15.43:0/2521169670 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff0003d30 con 0x7f1ff80d0af0 2024-04-25T09:02:10.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.527+0000 7f1fffd5c640 1 -- 172.21.15.43:0/2521169670 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ff8155aa0 con 0x7f1ff80d0af0 2024-04-25T09:02:10.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.527+0000 7f1fee7fc640 1 -- 172.21.15.43:0/2521169670 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff0005310 con 0x7f1ff80d0af0 2024-04-25T09:02:10.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.527+0000 7f1fffd5c640 1 -- 172.21.15.43:0/2521169670 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ff8156060 con 0x7f1ff80d0af0 2024-04-25T09:02:10.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.528+0000 7f1fffd5c640 1 -- 172.21.15.43:0/2521169670 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ff8074f60 con 0x7f1ff80d0af0 2024-04-25T09:02:10.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.528+0000 7f1fee7fc640 1 -- 172.21.15.43:0/2521169670 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1ff00056c0 con 0x7f1ff80d0af0 2024-04-25T09:02:10.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.529+0000 7f1fee7fc640 1 --2- 172.21.15.43:0/2521169670 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1fdc06b4d0 0x7f1fdc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:10.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.529+0000 7f1fee7fc640 1 -- 172.21.15.43:0/2521169670 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ff008a280 con 0x7f1ff80d0af0 2024-04-25T09:02:10.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.529+0000 7f1ffdaf8640 1 --2- 172.21.15.43:0/2521169670 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1fdc06b4d0 0x7f1fdc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:10.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.530+0000 7f1ffdaf8640 1 --2- 172.21.15.43:0/2521169670 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1fdc06b4d0 0x7f1fdc06d990 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4034000 tx=0x7f1ff402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:10.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.533+0000 7f1fee7fc640 1 -- 172.21.15.43:0/2521169670 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1ff00590f0 con 0x7f1ff80d0af0 2024-04-25T09:02:10.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.713+0000 7f1fffd5c640 1 -- 172.21.15.43:0/2521169670 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1ff8151850 con 0x7f1ff80d0af0 2024-04-25T09:02:10.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.714+0000 7f1fee7fc640 1 -- 172.21.15.43:0/2521169670 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v102) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1ff00589f0 con 0x7f1ff80d0af0 2024-04-25T09:02:10.702 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T09:02:10.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.715+0000 7f1fffd5c640 1 -- 172.21.15.43:0/2521169670 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1fdc06b4d0 msgr2=0x7f1fdc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:10.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.715+0000 7f1fffd5c640 1 --2- 172.21.15.43:0/2521169670 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1fdc06b4d0 0x7f1fdc06d990 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4034000 tx=0x7f1ff402e000 comp rx=0 tx=0).stop 2024-04-25T09:02:10.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.715+0000 7f1fffd5c640 1 -- 172.21.15.43:0/2521169670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ff80d0af0 msgr2=0x7f1ff815a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:10.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.715+0000 7f1fffd5c640 1 --2- 172.21.15.43:0/2521169670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ff80d0af0 0x7f1ff815a870 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f1ff000c730 tx=0x7f1ff000cbf0 comp rx=0 tx=0).stop 2024-04-25T09:02:10.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.715+0000 7f1fffd5c640 1 -- 172.21.15.43:0/2521169670 shutdown_connections 2024-04-25T09:02:10.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.715+0000 7f1fffd5c640 1 --2- 172.21.15.43:0/2521169670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ff80d0af0 0x7f1ff815a870 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:10.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.715+0000 7f1fffd5c640 1 --2- 172.21.15.43:0/2521169670 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1fdc06b4d0 0x7f1fdc06d990 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:10.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.715+0000 7f1fffd5c640 1 --2- 172.21.15.43:0/2521169670 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ff806f9a0 0x7f1ff815a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:10.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.715+0000 7f1fffd5c640 1 --2- 172.21.15.43:0/2521169670 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1ff806b6d0 0x7f1ff8151310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:10.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.716+0000 7f1fffd5c640 1 -- 172.21.15.43:0/2521169670 >> 172.21.15.43:0/2521169670 conn(0x7f1ff806c060 msgr2=0x7f1ff806e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:10.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.716+0000 7f1fffd5c640 1 -- 172.21.15.43:0/2521169670 shutdown_connections 2024-04-25T09:02:10.705 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.716+0000 7f1fffd5c640 1 -- 172.21.15.43:0/2521169670 wait complete. 2024-04-25T09:02:10.715 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:02:10.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.830+0000 7f6fcee6f640 1 Processor -- start 2024-04-25T09:02:10.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.830+0000 7f6fcee6f640 1 -- start start 2024-04-25T09:02:10.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.831+0000 7f6fcee6f640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6fc80d04a0 0x7f6fc815ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:10.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.831+0000 7f6fcee6f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fc80cf110 con 0x7f6fc815dec0 2024-04-25T09:02:10.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.831+0000 7f6fcee6f640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fc80cf3f0 con 0x7f6fc80ce8c0 2024-04-25T09:02:10.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.831+0000 7f6fcee6f640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fc80cf6d0 con 0x7f6fc80d04a0 2024-04-25T09:02:10.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.831+0000 7f6fc7fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f6fc815dec0 0x7f6fc80d0880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32846/0 (socket says 172.21.15.43:32846) 2024-04-25T09:02:10.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.831+0000 7f6fc7fff640 1 -- 172.21.15.43:0/1651523121 learned_addr learned my addr 172.21.15.43:0/1651523121 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:10.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.831+0000 7f6fccc0b640 1 --2- 172.21.15.43:0/1651523121 >> v2:172.21.15.193:3300/0 conn(0x7f6fc80d04a0 0x7f6fc815ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:10.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.831+0000 7f6fc77fe640 1 -- 172.21.15.43:0/1651523121 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4103262960 0 0) 0x7f6fc80cf110 con 0x7f6fc815dec0 2024-04-25T09:02:10.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.832+0000 7f6fc77fe640 1 -- 172.21.15.43:0/1651523121 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fb0003520 con 0x7f6fc815dec0 2024-04-25T09:02:10.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.832+0000 7f6fc77fe640 1 -- 172.21.15.43:0/1651523121 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3589283556 0 0) 0x7f6fb0003520 con 0x7f6fc815dec0 2024-04-25T09:02:10.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.832+0000 7f6fc77fe640 1 -- 172.21.15.43:0/1651523121 >> v2:172.21.15.193:3300/0 conn(0x7f6fc80d04a0 msgr2=0x7f6fc815ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:10.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.832+0000 7f6fc77fe640 1 --2- 172.21.15.43:0/1651523121 >> v2:172.21.15.193:3300/0 conn(0x7f6fc80d04a0 0x7f6fc815ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:10.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.832+0000 7f6fc77fe640 1 -- 172.21.15.43:0/1651523121 >> v1:172.21.15.193:6789/0 conn(0x7f6fc80ce8c0 legacy=0x7f6fc80d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:10.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.832+0000 7f6fc77fe640 1 -- 172.21.15.43:0/1651523121 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc80cf9b0 con 0x7f6fc815dec0 2024-04-25T09:02:10.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.832+0000 7f6fc77fe640 1 -- 172.21.15.43:0/1651523121 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f6fbc001670 con 0x7f6fc815dec0 2024-04-25T09:02:10.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.832+0000 7f6fc77fe640 1 -- 172.21.15.43:0/1651523121 >> v1:172.21.15.43:6789/0 conn(0x7f6fc815dec0 legacy=0x7f6fc80d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:10.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.832+0000 7f6fc77fe640 1 --2- 172.21.15.43:0/1651523121 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6fb00039b0 0x7f6fb0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:10.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.832+0000 7f6fc77fe640 1 --2- 172.21.15.43:0/1651523121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6fb0004800 0x7f6fb0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:10.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.832+0000 7f6fc77fe640 1 --2- 172.21.15.43:0/1651523121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fb00051c0 0x7f6fb000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:10.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.832+0000 7f6fc77fe640 1 -- 172.21.15.43:0/1651523121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc80cf6d0 con 0x7f6fb00051c0 2024-04-25T09:02:10.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.832+0000 7f6fc77fe640 1 -- 172.21.15.43:0/1651523121 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb000c060 con 0x7f6fb0004800 2024-04-25T09:02:10.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.832+0000 7f6fc77fe640 1 -- 172.21.15.43:0/1651523121 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fb000c260 con 0x7f6fb00039b0 2024-04-25T09:02:10.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.832+0000 7f6fcd40c640 1 --2- 172.21.15.43:0/1651523121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fb00051c0 0x7f6fb000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:10.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.832+0000 7f6fc7fff640 1 --2- 172.21.15.43:0/1651523121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6fb0004800 0x7f6fb0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:10.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.832+0000 7f6fccc0b640 1 --2- 172.21.15.43:0/1651523121 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6fb00039b0 0x7f6fb0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:10.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.832+0000 7f6fcd40c640 1 -- 172.21.15.43:0/1651523121 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6fb00039b0 msgr2=0x7f6fb0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:10.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.832+0000 7f6fcd40c640 1 --2- 172.21.15.43:0/1651523121 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6fb00039b0 0x7f6fb0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:10.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.832+0000 7f6fcd40c640 1 -- 172.21.15.43:0/1651523121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6fb0004800 msgr2=0x7f6fb0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:10.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.832+0000 7f6fcd40c640 1 --2- 172.21.15.43:0/1651523121 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6fb0004800 0x7f6fb0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:10.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.832+0000 7f6fcd40c640 1 -- 172.21.15.43:0/1651523121 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fb0003520 con 0x7f6fb00051c0 2024-04-25T09:02:10.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.833+0000 7f6fcd40c640 1 -- 172.21.15.43:0/1651523121 reap_dead start 2024-04-25T09:02:10.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.833+0000 7f6fcd40c640 1 --2- 172.21.15.43:0/1651523121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fb00051c0 0x7f6fb000b9c0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0001e50 tx=0x7f6fb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69668643ee595004 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:10.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.833+0000 7f6fc77fe640 1 -- 172.21.15.43:0/1651523121 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc0001a70 con 0x7f6fb00051c0 2024-04-25T09:02:10.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.833+0000 7f6fc77fe640 1 -- 172.21.15.43:0/1651523121 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc0003970 con 0x7f6fb00051c0 2024-04-25T09:02:10.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.833+0000 7f6fc77fe640 1 -- 172.21.15.43:0/1651523121 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc0003d60 con 0x7f6fb00051c0 2024-04-25T09:02:10.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.834+0000 7f6fcee6f640 1 -- 172.21.15.43:0/1651523121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fb00051c0 msgr2=0x7f6fb000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:10.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.834+0000 7f6fcee6f640 1 --2- 172.21.15.43:0/1651523121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fb00051c0 0x7f6fb000b9c0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0001e50 tx=0x7f6fb0005830 comp rx=0 tx=0).stop 2024-04-25T09:02:10.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.834+0000 7f6fcee6f640 1 -- 172.21.15.43:0/1651523121 shutdown_connections 2024-04-25T09:02:10.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.834+0000 7f6fcee6f640 1 --2- 172.21.15.43:0/1651523121 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fb00051c0 0x7f6fb000b9c0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:10.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.834+0000 7f6fcee6f640 1 -- 172.21.15.43:0/1651523121 >> 172.21.15.43:0/1651523121 conn(0x7f6fc809cd40 msgr2=0x7f6fc80d3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:10.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.834+0000 7f6fcee6f640 1 -- 172.21.15.43:0/1651523121 shutdown_connections 2024-04-25T09:02:10.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.834+0000 7f6fcee6f640 1 -- 172.21.15.43:0/1651523121 wait complete. 2024-04-25T09:02:10.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.835+0000 7f6fcee6f640 1 Processor -- start 2024-04-25T09:02:10.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.835+0000 7f6fcee6f640 1 -- start start 2024-04-25T09:02:10.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.836+0000 7f6fcee6f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6fb00051c0 0x7f6fc81f7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:10.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.836+0000 7f6fcee6f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fc8154ff0 0x7f6fc80b2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:10.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.836+0000 7f6fcee6f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6fc80b3120 0x7f6fc8160e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:10.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.836+0000 7f6fcee6f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc80cee50 con 0x7f6fc8154ff0 2024-04-25T09:02:10.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.836+0000 7f6fcee6f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc80cecd0 con 0x7f6fc80b3120 2024-04-25T09:02:10.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.836+0000 7f6fcee6f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc815ed90 con 0x7f6fb00051c0 2024-04-25T09:02:10.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.836+0000 7f6fc7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fc8154ff0 0x7f6fc80b2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:10.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.836+0000 7f6fc7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fc8154ff0 0x7f6fc80b2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51138/0 (socket says 172.21.15.43:51138) 2024-04-25T09:02:10.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.836+0000 7f6fc7fff640 1 -- 172.21.15.43:0/794185197 learned_addr learned my addr 172.21.15.43:0/794185197 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:10.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.836+0000 7f6fccc0b640 1 --2- 172.21.15.43:0/794185197 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6fb00051c0 0x7f6fc81f7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:10.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.836+0000 7f6fcd40c640 1 --2- 172.21.15.43:0/794185197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6fc80b3120 0x7f6fc8160e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:10.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.836+0000 7f6fc7fff640 1 -- 172.21.15.43:0/794185197 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6fb00051c0 msgr2=0x7f6fc81f7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:10.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.836+0000 7f6fc7fff640 1 --2- 172.21.15.43:0/794185197 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6fb00051c0 0x7f6fc81f7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:10.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.836+0000 7f6fc7fff640 1 -- 172.21.15.43:0/794185197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6fc80b3120 msgr2=0x7f6fc8160e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:10.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.836+0000 7f6fc7fff640 1 --2- 172.21.15.43:0/794185197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6fc80b3120 0x7f6fc8160e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:10.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.836+0000 7f6fc7fff640 1 -- 172.21.15.43:0/794185197 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc8155470 con 0x7f6fc8154ff0 2024-04-25T09:02:10.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.836+0000 7f6fc7fff640 1 --2- 172.21.15.43:0/794185197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fc8154ff0 0x7f6fc80b2be0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc0072f0 tx=0x7f6fbc0077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:10.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.837+0000 7f6fc57fa640 1 -- 172.21.15.43:0/794185197 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc00f960 con 0x7f6fc8154ff0 2024-04-25T09:02:10.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.837+0000 7f6fc57fa640 1 -- 172.21.15.43:0/794185197 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fbc018070 con 0x7f6fc8154ff0 2024-04-25T09:02:10.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.837+0000 7f6fcee6f640 1 -- 172.21.15.43:0/794185197 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fc80b74b0 con 0x7f6fc8154ff0 2024-04-25T09:02:10.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.837+0000 7f6fc57fa640 1 -- 172.21.15.43:0/794185197 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc00fd40 con 0x7f6fc8154ff0 2024-04-25T09:02:10.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.837+0000 7f6fcee6f640 1 -- 172.21.15.43:0/794185197 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fc80b7a70 con 0x7f6fc8154ff0 2024-04-25T09:02:10.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.838+0000 7f6fcee6f640 1 -- 172.21.15.43:0/794185197 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fc8074f60 con 0x7f6fc8154ff0 2024-04-25T09:02:10.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.838+0000 7f6fc57fa640 1 -- 172.21.15.43:0/794185197 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6fbc01d460 con 0x7f6fc8154ff0 2024-04-25T09:02:10.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.838+0000 7f6fc57fa640 1 --2- 172.21.15.43:0/794185197 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6fb406b500 0x7f6fb406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:10.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.838+0000 7f6fc57fa640 1 -- 172.21.15.43:0/794185197 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6fbc08a210 con 0x7f6fc8154ff0 2024-04-25T09:02:10.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.839+0000 7f6fccc0b640 1 --2- 172.21.15.43:0/794185197 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6fb406b500 0x7f6fb406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:10.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.839+0000 7f6fccc0b640 1 --2- 172.21.15.43:0/794185197 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6fb406b500 0x7f6fb406d9c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8009a80 tx=0x7f6fb8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:10.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:10.843+0000 7f6fc57fa640 1 -- 172.21.15.43:0/794185197 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6fc8155470 con 0x7f6fc8154ff0 2024-04-25T09:02:11.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.026+0000 7f6fcee6f640 1 -- 172.21.15.43:0/794185197 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6fc80b3500 con 0x7f6fc8154ff0 2024-04-25T09:02:11.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.026+0000 7f6fc57fa640 1 -- 172.21.15.43:0/794185197 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v102) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6fbc059090 con 0x7f6fc8154ff0 2024-04-25T09:02:11.015 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T09:02:11.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.028+0000 7f6fcee6f640 1 -- 172.21.15.43:0/794185197 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6fb406b500 msgr2=0x7f6fb406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:11.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.028+0000 7f6fcee6f640 1 --2- 172.21.15.43:0/794185197 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6fb406b500 0x7f6fb406d9c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8009a80 tx=0x7f6fb8008040 comp rx=0 tx=0).stop 2024-04-25T09:02:11.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.028+0000 7f6fcee6f640 1 -- 172.21.15.43:0/794185197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fc8154ff0 msgr2=0x7f6fc80b2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:11.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.028+0000 7f6fcee6f640 1 --2- 172.21.15.43:0/794185197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fc8154ff0 0x7f6fc80b2be0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc0072f0 tx=0x7f6fbc0077b0 comp rx=0 tx=0).stop 2024-04-25T09:02:11.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.028+0000 7f6fcee6f640 1 -- 172.21.15.43:0/794185197 shutdown_connections 2024-04-25T09:02:11.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.028+0000 7f6fcee6f640 1 --2- 172.21.15.43:0/794185197 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6fb406b500 0x7f6fb406d9c0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:11.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.028+0000 7f6fcee6f640 1 --2- 172.21.15.43:0/794185197 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6fc80b3120 0x7f6fc8160e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:11.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.028+0000 7f6fcee6f640 1 --2- 172.21.15.43:0/794185197 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fc8154ff0 0x7f6fc80b2be0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:11.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.028+0000 7f6fcee6f640 1 --2- 172.21.15.43:0/794185197 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6fb00051c0 0x7f6fc81f7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:11.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.029+0000 7f6fcee6f640 1 -- 172.21.15.43:0/794185197 >> 172.21.15.43:0/794185197 conn(0x7f6fc809cd40 msgr2=0x7f6fc80b1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:11.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.029+0000 7f6fcee6f640 1 -- 172.21.15.43:0/794185197 shutdown_connections 2024-04-25T09:02:11.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.029+0000 7f6fcee6f640 1 -- 172.21.15.43:0/794185197 wait complete. 2024-04-25T09:02:11.028 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:02:11.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.142+0000 7ffbbf109640 1 Processor -- start 2024-04-25T09:02:11.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.143+0000 7ffbbf109640 1 -- start start 2024-04-25T09:02:11.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.143+0000 7ffbbf109640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ffbb815dd60 0x7ffbb80d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:11.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.143+0000 7ffbbf109640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ffbb80d0aa0 0x7ffbb806bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:11.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.143+0000 7ffbbf109640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbb815e140 con 0x7ffbb80d1fa0 2024-04-25T09:02:11.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.143+0000 7ffbbf109640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbb80d27d0 con 0x7ffbb80d0aa0 2024-04-25T09:02:11.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.143+0000 7ffbbf109640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbb806b1e0 con 0x7ffbb815dd60 2024-04-25T09:02:11.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.143+0000 7ffbbd6a6640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ffbb80d0aa0 0x7ffbb806bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:11.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.143+0000 7ffbb7fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ffbb80d1fa0 0x7ffbb80d0390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32850/0 (socket says 172.21.15.43:32850) 2024-04-25T09:02:11.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.143+0000 7ffbbd6a6640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ffbb80d0aa0 0x7ffbb806bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51142/0 (socket says 172.21.15.43:51142) 2024-04-25T09:02:11.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.143+0000 7ffbb7fff640 1 -- 172.21.15.43:0/2929971525 learned_addr learned my addr 172.21.15.43:0/2929971525 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:11.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.144+0000 7ffbbcea5640 1 --2- 172.21.15.43:0/2929971525 >> v2:172.21.15.193:3300/0 conn(0x7ffbb815dd60 0x7ffbb80d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:11.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.144+0000 7ffbbd6a6640 1 -- 172.21.15.43:0/2929971525 >> v2:172.21.15.193:3300/0 conn(0x7ffbb815dd60 msgr2=0x7ffbb80d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:11.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.144+0000 7ffbbd6a6640 1 --2- 172.21.15.43:0/2929971525 >> v2:172.21.15.193:3300/0 conn(0x7ffbb815dd60 0x7ffbb80d1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:11.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.144+0000 7ffbbd6a6640 1 -- 172.21.15.43:0/2929971525 >> v1:172.21.15.43:6789/0 conn(0x7ffbb80d1fa0 legacy=0x7ffbb80d0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:11.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.144+0000 7ffbbd6a6640 1 -- 172.21.15.43:0/2929971525 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb806b4c0 con 0x7ffbb80d0aa0 2024-04-25T09:02:11.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.144+0000 7ffbb77fe640 1 -- 172.21.15.43:0/2929971525 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3245828422 0 0) 0x7ffbb815e140 con 0x7ffbb80d1fa0 2024-04-25T09:02:11.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.144+0000 7ffbbcea5640 1 --2- 172.21.15.43:0/2929971525 >> v2:172.21.15.193:3300/0 conn(0x7ffbb815dd60 0x7ffbb80d1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:11.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.145+0000 7ffbbd6a6640 1 --2- 172.21.15.43:0/2929971525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbb80d0aa0 0x7ffbb806bcc0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7ffbb808c990 tx=0x7ffbb002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abab31aab9e95810 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:11.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.145+0000 7ffbb77fe640 1 -- 172.21.15.43:0/2929971525 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0007740 con 0x7ffbb80d0aa0 2024-04-25T09:02:11.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.145+0000 7ffbb77fe640 1 -- 172.21.15.43:0/2929971525 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb00078e0 con 0x7ffbb80d0aa0 2024-04-25T09:02:11.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.145+0000 7ffbb77fe640 1 -- 172.21.15.43:0/2929971525 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb0007cb0 con 0x7ffbb80d0aa0 2024-04-25T09:02:11.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.145+0000 7ffbbf109640 1 -- 172.21.15.43:0/2929971525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbb80d0aa0 msgr2=0x7ffbb806bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:11.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.145+0000 7ffbbf109640 1 --2- 172.21.15.43:0/2929971525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbb80d0aa0 0x7ffbb806bcc0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7ffbb808c990 tx=0x7ffbb002a680 comp rx=0 tx=0).stop 2024-04-25T09:02:11.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.145+0000 7ffbbf109640 1 -- 172.21.15.43:0/2929971525 shutdown_connections 2024-04-25T09:02:11.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.145+0000 7ffbbf109640 1 --2- 172.21.15.43:0/2929971525 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbb80d0aa0 0x7ffbb806bcc0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:11.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.145+0000 7ffbbf109640 1 --2- 172.21.15.43:0/2929971525 >> v2:172.21.15.193:3300/0 conn(0x7ffbb815dd60 0x7ffbb80d1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:11.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.145+0000 7ffbbf109640 1 -- 172.21.15.43:0/2929971525 >> 172.21.15.43:0/2929971525 conn(0x7ffbb80d39c0 msgr2=0x7ffbb80d3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:11.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.145+0000 7ffbbf109640 1 -- 172.21.15.43:0/2929971525 shutdown_connections 2024-04-25T09:02:11.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.145+0000 7ffbbf109640 1 -- 172.21.15.43:0/2929971525 wait complete. 2024-04-25T09:02:11.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.146+0000 7ffbbf109640 1 Processor -- start 2024-04-25T09:02:11.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.146+0000 7ffbbf109640 1 -- start start 2024-04-25T09:02:11.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.147+0000 7ffbbf109640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbb80d0aa0 0x7ffbb815a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:11.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.147+0000 7ffbbf109640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffbb80d1fa0 0x7ffbb80c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:11.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.147+0000 7ffbbf109640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffbb815dd60 0x7ffbb81f3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:11.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.147+0000 7ffbbf109640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb80d25b0 con 0x7ffbb80d0aa0 2024-04-25T09:02:11.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.147+0000 7ffbbf109640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb80d2430 con 0x7ffbb815dd60 2024-04-25T09:02:11.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.147+0000 7ffbbf109640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb806c200 con 0x7ffbb80d1fa0 2024-04-25T09:02:11.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.147+0000 7ffbbcea5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbb80d0aa0 0x7ffbb815a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:11.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.147+0000 7ffbbcea5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbb80d0aa0 0x7ffbb815a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51152/0 (socket says 172.21.15.43:51152) 2024-04-25T09:02:11.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.147+0000 7ffbbcea5640 1 -- 172.21.15.43:0/2464187157 learned_addr learned my addr 172.21.15.43:0/2464187157 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:11.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.148+0000 7ffbbd6a6640 1 --2- 172.21.15.43:0/2464187157 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffbb815dd60 0x7ffbb81f3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:11.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.148+0000 7ffbb7fff640 1 --2- 172.21.15.43:0/2464187157 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffbb80d1fa0 0x7ffbb80c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:11.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.148+0000 7ffbbcea5640 1 -- 172.21.15.43:0/2464187157 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffbb80d1fa0 msgr2=0x7ffbb80c8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:11.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.148+0000 7ffbbcea5640 1 --2- 172.21.15.43:0/2464187157 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffbb80d1fa0 0x7ffbb80c8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:11.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.148+0000 7ffbbcea5640 1 -- 172.21.15.43:0/2464187157 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffbb815dd60 msgr2=0x7ffbb81f3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:11.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.148+0000 7ffbbcea5640 1 --2- 172.21.15.43:0/2464187157 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffbb815dd60 0x7ffbb81f3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:11.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.148+0000 7ffbbcea5640 1 -- 172.21.15.43:0/2464187157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb81f41d0 con 0x7ffbb80d0aa0 2024-04-25T09:02:11.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.148+0000 7ffbbcea5640 1 --2- 172.21.15.43:0/2464187157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbb80d0aa0 0x7ffbb815a050 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00d7d0 tx=0x7ffbac00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:11.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.148+0000 7ffbb57fa640 1 -- 172.21.15.43:0/2464187157 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbac004030 con 0x7ffbb80d0aa0 2024-04-25T09:02:11.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.148+0000 7ffbb57fa640 1 -- 172.21.15.43:0/2464187157 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbac0041d0 con 0x7ffbb80d0aa0 2024-04-25T09:02:11.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.148+0000 7ffbb57fa640 1 -- 172.21.15.43:0/2464187157 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbac00b6c0 con 0x7ffbb80d0aa0 2024-04-25T09:02:11.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.148+0000 7ffbbf109640 1 -- 172.21.15.43:0/2464187157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbb81f44b0 con 0x7ffbb80d0aa0 2024-04-25T09:02:11.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.148+0000 7ffbbf109640 1 -- 172.21.15.43:0/2464187157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb80b8a50 con 0x7ffbb80d0aa0 2024-04-25T09:02:11.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.149+0000 7ffbbf109640 1 -- 172.21.15.43:0/2464187157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbb8074f60 con 0x7ffbb80d0aa0 2024-04-25T09:02:11.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.150+0000 7ffbb57fa640 1 -- 172.21.15.43:0/2464187157 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffbac00b860 con 0x7ffbb80d0aa0 2024-04-25T09:02:11.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.150+0000 7ffbb57fa640 1 --2- 172.21.15.43:0/2464187157 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ffb9c06b4d0 0x7ffb9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:11.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.150+0000 7ffbb57fa640 1 -- 172.21.15.43:0/2464187157 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffbac08a4f0 con 0x7ffbb80d0aa0 2024-04-25T09:02:11.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.150+0000 7ffbb7fff640 1 --2- 172.21.15.43:0/2464187157 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ffb9c06b4d0 0x7ffb9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:11.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.151+0000 7ffbb7fff640 1 --2- 172.21.15.43:0/2464187157 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ffb9c06b4d0 0x7ffb9c06d990 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ffbb80c8900 tx=0x7ffba4007b50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:11.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.154+0000 7ffbb57fa640 1 -- 172.21.15.43:0/2464187157 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffbac059370 con 0x7ffbb80d0aa0 2024-04-25T09:02:11.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.333+0000 7ffbbf109640 1 -- 172.21.15.43:0/2464187157 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ffbb80b2c00 con 0x7ffbb80d0aa0 2024-04-25T09:02:11.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.333+0000 7ffbb57fa640 1 -- 172.21.15.43:0/2464187157 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v102) v1 ==== 100+0+11 (secure 0 0 0) 0x7ffbac058c70 con 0x7ffbb80d0aa0 2024-04-25T09:02:11.322 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T09:02:11.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.335+0000 7ffbbf109640 1 -- 172.21.15.43:0/2464187157 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ffb9c06b4d0 msgr2=0x7ffb9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:11.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.335+0000 7ffbbf109640 1 --2- 172.21.15.43:0/2464187157 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ffb9c06b4d0 0x7ffb9c06d990 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ffbb80c8900 tx=0x7ffba4007b50 comp rx=0 tx=0).stop 2024-04-25T09:02:11.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.335+0000 7ffbbf109640 1 -- 172.21.15.43:0/2464187157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbb80d0aa0 msgr2=0x7ffbb815a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:11.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.335+0000 7ffbbf109640 1 --2- 172.21.15.43:0/2464187157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbb80d0aa0 0x7ffbb815a050 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7ffbac00d7d0 tx=0x7ffbac00dc90 comp rx=0 tx=0).stop 2024-04-25T09:02:11.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.335+0000 7ffbbf109640 1 -- 172.21.15.43:0/2464187157 shutdown_connections 2024-04-25T09:02:11.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.335+0000 7ffbbf109640 1 --2- 172.21.15.43:0/2464187157 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ffb9c06b4d0 0x7ffb9c06d990 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:11.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.335+0000 7ffbbf109640 1 --2- 172.21.15.43:0/2464187157 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ffbb815dd60 0x7ffbb81f3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:11.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.335+0000 7ffbbf109640 1 --2- 172.21.15.43:0/2464187157 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ffbb80d1fa0 0x7ffbb80c8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:11.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.335+0000 7ffbbf109640 1 --2- 172.21.15.43:0/2464187157 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ffbb80d0aa0 0x7ffbb815a050 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:11.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.335+0000 7ffbbf109640 1 -- 172.21.15.43:0/2464187157 >> 172.21.15.43:0/2464187157 conn(0x7ffbb80d39c0 msgr2=0x7ffbb80b34d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:11.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.335+0000 7ffbbf109640 1 -- 172.21.15.43:0/2464187157 shutdown_connections 2024-04-25T09:02:11.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.335+0000 7ffbbf109640 1 -- 172.21.15.43:0/2464187157 wait complete. 2024-04-25T09:02:11.335 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:02:11.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.447+0000 7f37382ac640 1 Processor -- start 2024-04-25T09:02:11.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.447+0000 7f37382ac640 1 -- start start 2024-04-25T09:02:11.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.448+0000 7f37382ac640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f37300d39d0 0x7f37300d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:11.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.448+0000 7f37382ac640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f37300d2500 0x7f37300d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:11.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.448+0000 7f37382ac640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37300b6c50 con 0x7f37300ce2a0 2024-04-25T09:02:11.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.448+0000 7f37382ac640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f373015e010 con 0x7f37300d2500 2024-04-25T09:02:11.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.448+0000 7f37382ac640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37300d2020 con 0x7f37300d39d0 2024-04-25T09:02:11.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.448+0000 7f3735847640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f37300ce2a0 0x7f37300d3db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32858/0 (socket says 172.21.15.43:32858) 2024-04-25T09:02:11.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.449+0000 7f3735847640 1 -- 172.21.15.43:0/4170969567 learned_addr learned my addr 172.21.15.43:0/4170969567 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:11.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.449+0000 7f3736849640 1 --2- 172.21.15.43:0/4170969567 >> v2:172.21.15.193:3300/0 conn(0x7f37300d2500 0x7f37300d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:11.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.449+0000 7f3736048640 1 --2- 172.21.15.43:0/4170969567 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f37300d39d0 0x7f37300d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:11.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.449+0000 7f3735046640 1 -- 172.21.15.43:0/4170969567 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4152472842 0 0) 0x7f3720001110 con 0x7f37300ce2a0 2024-04-25T09:02:11.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.449+0000 7f3735046640 1 -- 172.21.15.43:0/4170969567 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f371c003520 con 0x7f37300ce2a0 2024-04-25T09:02:11.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.449+0000 7f3735046640 1 -- 172.21.15.43:0/4170969567 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2444222477 0 0) 0x7f371c003520 con 0x7f37300ce2a0 2024-04-25T09:02:11.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.449+0000 7f3735046640 1 -- 172.21.15.43:0/4170969567 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f37300d39d0 msgr2=0x7f37300d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:11.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.449+0000 7f3735046640 1 --2- 172.21.15.43:0/4170969567 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f37300d39d0 0x7f37300d0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:11.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.449+0000 7f3735046640 1 -- 172.21.15.43:0/4170969567 >> v2:172.21.15.193:3300/0 conn(0x7f37300d2500 msgr2=0x7f37300d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:11.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.449+0000 7f3735046640 1 --2- 172.21.15.43:0/4170969567 >> v2:172.21.15.193:3300/0 conn(0x7f37300d2500 0x7f37300d2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:11.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.449+0000 7f3735046640 1 -- 172.21.15.43:0/4170969567 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37300cec40 con 0x7f37300ce2a0 2024-04-25T09:02:11.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.449+0000 7f3735046640 1 -- 172.21.15.43:0/4170969567 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f3720001840 con 0x7f37300ce2a0 2024-04-25T09:02:11.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.449+0000 7f3736849640 1 --2- 172.21.15.43:0/4170969567 >> v2:172.21.15.193:3300/0 conn(0x7f37300d2500 0x7f37300d2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:02:11.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.449+0000 7f3735046640 1 -- 172.21.15.43:0/4170969567 >> v1:172.21.15.43:6789/0 conn(0x7f37300ce2a0 legacy=0x7f37300d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:11.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.449+0000 7f3735046640 1 --2- 172.21.15.43:0/4170969567 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f371c0039b0 0x7f371c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:11.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.449+0000 7f3735046640 1 --2- 172.21.15.43:0/4170969567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f371c004800 0x7f371c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:11.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.449+0000 7f3735046640 1 --2- 172.21.15.43:0/4170969567 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f37300d39d0 0x7f371c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:11.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.449+0000 7f3735046640 1 -- 172.21.15.43:0/4170969567 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373015e010 con 0x7f371c004800 2024-04-25T09:02:11.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.449+0000 7f3735046640 1 -- 172.21.15.43:0/4170969567 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37300d2020 con 0x7f371c0039b0 2024-04-25T09:02:11.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.449+0000 7f3735046640 1 -- 172.21.15.43:0/4170969567 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f371c00b9c0 con 0x7f37300d39d0 2024-04-25T09:02:11.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.449+0000 7f3735847640 1 --2- 172.21.15.43:0/4170969567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f371c004800 0x7f371c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:11.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.449+0000 7f3736048640 1 --2- 172.21.15.43:0/4170969567 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f371c0039b0 0x7f371c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:11.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.449+0000 7f3736849640 1 --2- 172.21.15.43:0/4170969567 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f37300d39d0 0x7f371c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:11.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.450+0000 7f3735847640 1 -- 172.21.15.43:0/4170969567 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f37300d39d0 msgr2=0x7f371c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:11.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.450+0000 7f3735847640 1 --2- 172.21.15.43:0/4170969567 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f37300d39d0 0x7f371c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:11.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.450+0000 7f3735847640 1 -- 172.21.15.43:0/4170969567 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f371c0039b0 msgr2=0x7f371c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:11.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.450+0000 7f3735847640 1 --2- 172.21.15.43:0/4170969567 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f371c0039b0 0x7f371c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:11.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.450+0000 7f3735847640 1 -- 172.21.15.43:0/4170969567 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f371c003520 con 0x7f371c004800 2024-04-25T09:02:11.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.450+0000 7f3736849640 1 --2- 172.21.15.43:0/4170969567 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f37300d39d0 0x7f371c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:11.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.450+0000 7f3736048640 1 --2- 172.21.15.43:0/4170969567 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f371c0039b0 0x7f371c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:11.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.450+0000 7f3735847640 1 --2- 172.21.15.43:0/4170969567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f371c004800 0x7f371c004c80 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f372000c4c0 tx=0x7f371c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c80e866542fbe5e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:11.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.450+0000 7f3735046640 1 -- 172.21.15.43:0/4170969567 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372000c0e0 con 0x7f371c004800 2024-04-25T09:02:11.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.450+0000 7f3735046640 1 -- 172.21.15.43:0/4170969567 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3720034070 con 0x7f371c004800 2024-04-25T09:02:11.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.450+0000 7f3735046640 1 -- 172.21.15.43:0/4170969567 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3720005d10 con 0x7f371c004800 2024-04-25T09:02:11.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.451+0000 7f37382ac640 1 -- 172.21.15.43:0/4170969567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f371c004800 msgr2=0x7f371c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:11.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.451+0000 7f37382ac640 1 --2- 172.21.15.43:0/4170969567 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f371c004800 0x7f371c004c80 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f372000c4c0 tx=0x7f371c005830 comp rx=0 tx=0).stop 2024-04-25T09:02:11.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.451+0000 7f3736849640 1 -- 172.21.15.43:0/4170969567 reap_dead start 2024-04-25T09:02:11.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.451+0000 7f37382ac640 1 -- 172.21.15.43:0/4170969567 shutdown_connections 2024-04-25T09:02:11.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.451+0000 7f37382ac640 1 -- 172.21.15.43:0/4170969567 >> 172.21.15.43:0/4170969567 conn(0x7f373006b000 msgr2=0x7f373006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:11.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.451+0000 7f37382ac640 1 -- 172.21.15.43:0/4170969567 shutdown_connections 2024-04-25T09:02:11.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.451+0000 7f37382ac640 1 -- 172.21.15.43:0/4170969567 wait complete. 2024-04-25T09:02:11.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.452+0000 7f37382ac640 1 Processor -- start 2024-04-25T09:02:11.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.452+0000 7f37382ac640 1 -- start start 2024-04-25T09:02:11.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.453+0000 7f37382ac640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f371c004800 0x7f37300cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:11.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.453+0000 7f37382ac640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37300c9e70 0x7f373015b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:11.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.453+0000 7f37382ac640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37300cbb60 0x7f37300cbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:11.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.453+0000 7f37382ac640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37300cf200 con 0x7f37300c9e70 2024-04-25T09:02:11.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.453+0000 7f37382ac640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37300b2f00 con 0x7f37300cbb60 2024-04-25T09:02:11.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.453+0000 7f37382ac640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37300d0d90 con 0x7f371c004800 2024-04-25T09:02:11.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.453+0000 7f3735847640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37300c9e70 0x7f373015b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:11.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.453+0000 7f3735847640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37300c9e70 0x7f373015b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51164/0 (socket says 172.21.15.43:51164) 2024-04-25T09:02:11.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.453+0000 7f3735847640 1 -- 172.21.15.43:0/487596236 learned_addr learned my addr 172.21.15.43:0/487596236 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:11.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.453+0000 7f3735847640 1 -- 172.21.15.43:0/487596236 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f371c004800 msgr2=0x7f37300cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:11.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.453+0000 7f3735847640 1 --2- 172.21.15.43:0/487596236 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f371c004800 0x7f37300cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:11.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.453+0000 7f3736849640 1 --2- 172.21.15.43:0/487596236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37300cbb60 0x7f37300cbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:11.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.453+0000 7f3735847640 1 -- 172.21.15.43:0/487596236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37300cbb60 msgr2=0x7f37300cbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:11.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.453+0000 7f3735847640 1 --2- 172.21.15.43:0/487596236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37300cbb60 0x7f37300cbfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:11.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.453+0000 7f3735847640 1 -- 172.21.15.43:0/487596236 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37300cd8f0 con 0x7f37300c9e70 2024-04-25T09:02:11.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.453+0000 7f3736849640 1 --2- 172.21.15.43:0/487596236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37300cbb60 0x7f37300cbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:11.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.454+0000 7f3735847640 1 --2- 172.21.15.43:0/487596236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37300c9e70 0x7f373015b280 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f372000c520 tx=0x7f3720000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:11.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.454+0000 7f3726ffd640 1 -- 172.21.15.43:0/487596236 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3720007270 con 0x7f37300c9e70 2024-04-25T09:02:11.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.454+0000 7f3726ffd640 1 -- 172.21.15.43:0/487596236 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3720007410 con 0x7f37300c9e70 2024-04-25T09:02:11.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.454+0000 7f37382ac640 1 -- 172.21.15.43:0/487596236 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37300c86b0 con 0x7f37300c9e70 2024-04-25T09:02:11.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.454+0000 7f3726ffd640 1 -- 172.21.15.43:0/487596236 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37200077c0 con 0x7f37300c9e70 2024-04-25T09:02:11.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.454+0000 7f37382ac640 1 -- 172.21.15.43:0/487596236 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37300c78f0 con 0x7f37300c9e70 2024-04-25T09:02:11.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.455+0000 7f37382ac640 1 -- 172.21.15.43:0/487596236 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3730074ef0 con 0x7f37300c9e70 2024-04-25T09:02:11.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.455+0000 7f3726ffd640 1 -- 172.21.15.43:0/487596236 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f372002f480 con 0x7f37300c9e70 2024-04-25T09:02:11.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.456+0000 7f3726ffd640 1 --2- 172.21.15.43:0/487596236 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f371406b4d0 0x7f371406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:11.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.456+0000 7f3726ffd640 1 -- 172.21.15.43:0/487596236 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f37200aa980 con 0x7f37300c9e70 2024-04-25T09:02:11.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.456+0000 7f3736048640 1 --2- 172.21.15.43:0/487596236 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f371406b4d0 0x7f371406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:11.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.456+0000 7f3736048640 1 --2- 172.21.15.43:0/487596236 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f371406b4d0 0x7f371406d990 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f372c009750 tx=0x7f372c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:11.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.460+0000 7f3726ffd640 1 -- 172.21.15.43:0/487596236 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3720049030 con 0x7f37300c9e70 2024-04-25T09:02:11.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.652+0000 7f37382ac640 1 -- 172.21.15.43:0/487596236 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f37300d17c0 con 0x7f37300c9e70 2024-04-25T09:02:11.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.653+0000 7f3726ffd640 1 -- 172.21.15.43:0/487596236 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v102) v1 ==== 80+0+0 (secure 0 0 0) 0x7f37300cd8f0 con 0x7f37300c9e70 2024-04-25T09:02:11.642 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-04-25T09:02:11.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.655+0000 7f37382ac640 1 -- 172.21.15.43:0/487596236 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f371406b4d0 msgr2=0x7f371406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:11.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.655+0000 7f37382ac640 1 --2- 172.21.15.43:0/487596236 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f371406b4d0 0x7f371406d990 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f372c009750 tx=0x7f372c008040 comp rx=0 tx=0).stop 2024-04-25T09:02:11.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.655+0000 7f37382ac640 1 -- 172.21.15.43:0/487596236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37300c9e70 msgr2=0x7f373015b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:11.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.655+0000 7f37382ac640 1 --2- 172.21.15.43:0/487596236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37300c9e70 0x7f373015b280 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f372000c520 tx=0x7f3720000fe0 comp rx=0 tx=0).stop 2024-04-25T09:02:11.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.655+0000 7f37382ac640 1 -- 172.21.15.43:0/487596236 shutdown_connections 2024-04-25T09:02:11.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.655+0000 7f37382ac640 1 --2- 172.21.15.43:0/487596236 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f371406b4d0 0x7f371406d990 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:11.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.655+0000 7f37382ac640 1 --2- 172.21.15.43:0/487596236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f37300cbb60 0x7f37300cbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:11.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.655+0000 7f37382ac640 1 --2- 172.21.15.43:0/487596236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f37300c9e70 0x7f373015b280 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:11.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.655+0000 7f37382ac640 1 --2- 172.21.15.43:0/487596236 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f371c004800 0x7f37300cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:11.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.655+0000 7f37382ac640 1 -- 172.21.15.43:0/487596236 >> 172.21.15.43:0/487596236 conn(0x7f373006b000 msgr2=0x7f373015e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:11.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.655+0000 7f37382ac640 1 -- 172.21.15.43:0/487596236 shutdown_connections 2024-04-25T09:02:11.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:11.655+0000 7f37382ac640 1 -- 172.21.15.43:0/487596236 wait complete. 2024-04-25T09:02:11.654 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi043.front.sepia.ceph.com... 2024-04-25T09:02:11.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:11.654 DEBUG:teuthology.orchestra.run.smithi043:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T09:02:11.677 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[81338]: fuse finished with error 0 and tester_r 0 2024-04-25T09:02:11.751 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-04-25T09:02:11.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:11.752 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:02:11.917 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T09:02:11.917 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:02:11.935 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19508 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:02:11.935 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi193.front.sepia.ceph.com 2024-04-25T09:02:11.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:11.935 DEBUG:teuthology.orchestra.run.smithi193:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T09:02:11.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:02:11.992 INFO:teuthology.orchestra.run.smithi193.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T09:02:11.992 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T09:02:11.992 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:02:12.008 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2374 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:02:12.009 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-04-25T09:02:12.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:12.009 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T09:02:12.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:02:12.063 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T09:02:12.064 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T09:02:12.064 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:02:12.080 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19508 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:02:12.081 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi193.front.sepia.ceph.com 2024-04-25T09:02:12.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:12.082 DEBUG:teuthology.orchestra.run.smithi193:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T09:02:12.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:02:12.136 INFO:teuthology.orchestra.run.smithi193.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T09:02:12.136 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:02:12.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.248+0000 7f8226476640 1 Processor -- start 2024-04-25T09:02:12.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.248+0000 7f8226476640 1 -- start start 2024-04-25T09:02:12.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.249+0000 7f8226476640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f822015a9d0 0x7f822006d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:12.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.249+0000 7f8226476640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f822006d7d0 0x7f8220071a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:12.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.249+0000 7f8226476640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f8220071f90 0x7f82200b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:12.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.249+0000 7f8226476640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f822006dc30 con 0x7f822006d7d0 2024-04-25T09:02:12.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.249+0000 7f8226476640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8220072420 con 0x7f8220071f90 2024-04-25T09:02:12.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.249+0000 7f8226476640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82200704f0 con 0x7f822015a9d0 2024-04-25T09:02:12.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.249+0000 7f821f7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f822006d7d0 0x7f8220071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:12.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.249+0000 7f821f7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f822006d7d0 0x7f8220071a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51170/0 (socket says 172.21.15.43:51170) 2024-04-25T09:02:12.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.249+0000 7f821f7fe640 1 -- 172.21.15.43:0/1832030679 learned_addr learned my addr 172.21.15.43:0/1832030679 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:12.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.250+0000 7f821ffff640 1 --2- 172.21.15.43:0/1832030679 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f822015a9d0 0x7f822006d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:12.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.250+0000 7f8224a13640 1 --2- 172.21.15.43:0/1832030679 >> v2:172.21.15.193:3300/0 conn(0x7f8220071f90 0x7f82200b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:12.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.250+0000 7f821f7fe640 1 -- 172.21.15.43:0/1832030679 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f822015a9d0 msgr2=0x7f822006d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:12.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.250+0000 7f821f7fe640 1 --2- 172.21.15.43:0/1832030679 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f822015a9d0 0x7f822006d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.250+0000 7f821f7fe640 1 -- 172.21.15.43:0/1832030679 >> v2:172.21.15.193:3300/0 conn(0x7f8220071f90 msgr2=0x7f82200b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:12.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.250+0000 7f821f7fe640 1 --2- 172.21.15.43:0/1832030679 >> v2:172.21.15.193:3300/0 conn(0x7f8220071f90 0x7f82200b3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.250+0000 7f821f7fe640 1 -- 172.21.15.43:0/1832030679 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82200707d0 con 0x7f822006d7d0 2024-04-25T09:02:12.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.250+0000 7f821f7fe640 1 --2- 172.21.15.43:0/1832030679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f822006d7d0 0x7f8220071a50 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f82200702e0 tx=0x7f821402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d245d342034d3f41 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:12.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.251+0000 7f821effd640 1 -- 172.21.15.43:0/1832030679 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8214037070 con 0x7f822006d7d0 2024-04-25T09:02:12.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.251+0000 7f821effd640 1 -- 172.21.15.43:0/1832030679 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8214032440 con 0x7f822006d7d0 2024-04-25T09:02:12.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.251+0000 7f821effd640 1 -- 172.21.15.43:0/1832030679 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8214032810 con 0x7f822006d7d0 2024-04-25T09:02:12.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.251+0000 7f8226476640 1 -- 172.21.15.43:0/1832030679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f822006d7d0 msgr2=0x7f8220071a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:12.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.251+0000 7f8226476640 1 --2- 172.21.15.43:0/1832030679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f822006d7d0 0x7f8220071a50 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f82200702e0 tx=0x7f821402a1e0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.251+0000 7f8226476640 1 -- 172.21.15.43:0/1832030679 shutdown_connections 2024-04-25T09:02:12.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.251+0000 7f8226476640 1 --2- 172.21.15.43:0/1832030679 >> v2:172.21.15.193:3300/0 conn(0x7f8220071f90 0x7f82200b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.251+0000 7f8226476640 1 --2- 172.21.15.43:0/1832030679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f822006d7d0 0x7f8220071a50 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.251+0000 7f8226476640 1 --2- 172.21.15.43:0/1832030679 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f822015a9d0 0x7f822006d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.251+0000 7f8226476640 1 -- 172.21.15.43:0/1832030679 >> 172.21.15.43:0/1832030679 conn(0x7f82200b1480 msgr2=0x7f82200b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:12.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.251+0000 7f8226476640 1 -- 172.21.15.43:0/1832030679 shutdown_connections 2024-04-25T09:02:12.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.251+0000 7f8226476640 1 -- 172.21.15.43:0/1832030679 wait complete. 2024-04-25T09:02:12.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.252+0000 7f8226476640 1 Processor -- start 2024-04-25T09:02:12.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.253+0000 7f8226476640 1 -- start start 2024-04-25T09:02:12.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.253+0000 7f8226476640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f822006d7d0 0x7f8220140ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:12.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.253+0000 7f8226476640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8220071f90 0x7f8220157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:12.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.253+0000 7f8226476640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82200aee60 0x7f82200af2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:12.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.253+0000 7f8226476640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82200712b0 con 0x7f8220071f90 2024-04-25T09:02:12.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.253+0000 7f8226476640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8220070d90 con 0x7f822006d7d0 2024-04-25T09:02:12.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.253+0000 7f8226476640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82200b3ed0 con 0x7f82200aee60 2024-04-25T09:02:12.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.253+0000 7f821f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8220071f90 0x7f8220157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:12.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.253+0000 7f821f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8220071f90 0x7f8220157720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51176/0 (socket says 172.21.15.43:51176) 2024-04-25T09:02:12.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.253+0000 7f821f7fe640 1 -- 172.21.15.43:0/3696625034 learned_addr learned my addr 172.21.15.43:0/3696625034 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:12.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.254+0000 7f821ffff640 1 --2- 172.21.15.43:0/3696625034 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f822006d7d0 0x7f8220140ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:12.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.254+0000 7f8224a13640 1 --2- 172.21.15.43:0/3696625034 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82200aee60 0x7f82200af2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:12.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.254+0000 7f821f7fe640 1 -- 172.21.15.43:0/3696625034 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82200aee60 msgr2=0x7f82200af2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:12.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.254+0000 7f821f7fe640 1 --2- 172.21.15.43:0/3696625034 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82200aee60 0x7f82200af2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.254+0000 7f821f7fe640 1 -- 172.21.15.43:0/3696625034 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f822006d7d0 msgr2=0x7f8220140ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:12.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.254+0000 7f821f7fe640 1 --2- 172.21.15.43:0/3696625034 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f822006d7d0 0x7f8220140ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.254+0000 7f821f7fe640 1 -- 172.21.15.43:0/3696625034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82201563a0 con 0x7f8220071f90 2024-04-25T09:02:12.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.254+0000 7f821f7fe640 1 --2- 172.21.15.43:0/3696625034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8220071f90 0x7f8220157720 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f8214000c00 tx=0x7f82140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:12.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.254+0000 7f821cff9640 1 -- 172.21.15.43:0/3696625034 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8214041070 con 0x7f8220071f90 2024-04-25T09:02:12.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.254+0000 7f821cff9640 1 -- 172.21.15.43:0/3696625034 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f821403c870 con 0x7f8220071f90 2024-04-25T09:02:12.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.254+0000 7f821cff9640 1 -- 172.21.15.43:0/3696625034 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8214037040 con 0x7f8220071f90 2024-04-25T09:02:12.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.254+0000 7f8226476640 1 -- 172.21.15.43:0/3696625034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8220156680 con 0x7f8220071f90 2024-04-25T09:02:12.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.254+0000 7f8226476640 1 -- 172.21.15.43:0/3696625034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8220156c40 con 0x7f8220071f90 2024-04-25T09:02:12.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.255+0000 7f8226476640 1 -- 172.21.15.43:0/3696625034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8220074ef0 con 0x7f8220071f90 2024-04-25T09:02:12.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.256+0000 7f821cff9640 1 -- 172.21.15.43:0/3696625034 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8214044070 con 0x7f8220071f90 2024-04-25T09:02:12.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.256+0000 7f821cff9640 1 --2- 172.21.15.43:0/3696625034 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f820c06b4d0 0x7f820c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:12.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.256+0000 7f821cff9640 1 -- 172.21.15.43:0/3696625034 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82140aada0 con 0x7f8220071f90 2024-04-25T09:02:12.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.256+0000 7f821ffff640 1 --2- 172.21.15.43:0/3696625034 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f820c06b4d0 0x7f820c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:12.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.257+0000 7f821ffff640 1 --2- 172.21.15.43:0/3696625034 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f820c06b4d0 0x7f820c06d990 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f8210006fd0 tx=0x7f8210008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:12.249 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.260+0000 7f821cff9640 1 -- 172.21.15.43:0/3696625034 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8214079d40 con 0x7f8220071f90 2024-04-25T09:02:12.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.478+0000 7f8226476640 1 -- 172.21.15.43:0/3696625034 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8220071920 con 0x7f8220071f90 2024-04-25T09:02:12.467 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.478+0000 7f821cff9640 1 -- 172.21.15.43:0/3696625034 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f8214079640 con 0x7f8220071f90 2024-04-25T09:02:12.467 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:12.467 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":61,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2150177203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2150177203},{"type":"v1","addr":"172.21.15.193:6837","nonce":2150177203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":8079,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/621488642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":621488642},{"type":"v1","addr":"172.21.15.43:6834","nonce":621488642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8082,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1524395840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1524395840},{"type":"v1","addr":"172.21.15.43:6835","nonce":1524395840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:01:33.726790+0000","modified":"2024-04-25T09:01:35.706689+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8094},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8094":{"gid":8094,"name":"b","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.193:6836/853943267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":853943267},{"type":"v1","addr":"172.21.15.193:6836","nonce":853943267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:02:12.467 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 61 2024-04-25T09:02:12.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.480+0000 7f8226476640 1 -- 172.21.15.43:0/3696625034 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f820c06b4d0 msgr2=0x7f820c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:12.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.480+0000 7f8226476640 1 --2- 172.21.15.43:0/3696625034 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f820c06b4d0 0x7f820c06d990 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f8210006fd0 tx=0x7f8210008040 comp rx=0 tx=0).stop 2024-04-25T09:02:12.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.480+0000 7f8226476640 1 -- 172.21.15.43:0/3696625034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8220071f90 msgr2=0x7f8220157720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:12.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.480+0000 7f8226476640 1 --2- 172.21.15.43:0/3696625034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8220071f90 0x7f8220157720 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f8214000c00 tx=0x7f82140046c0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.480+0000 7f8226476640 1 -- 172.21.15.43:0/3696625034 shutdown_connections 2024-04-25T09:02:12.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.480+0000 7f8226476640 1 --2- 172.21.15.43:0/3696625034 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f82200aee60 0x7f82200af2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.480+0000 7f8226476640 1 --2- 172.21.15.43:0/3696625034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8220071f90 0x7f8220157720 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.480+0000 7f8226476640 1 --2- 172.21.15.43:0/3696625034 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f820c06b4d0 0x7f820c06d990 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.480+0000 7f8226476640 1 --2- 172.21.15.43:0/3696625034 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f822006d7d0 0x7f8220140ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.480+0000 7f8226476640 1 -- 172.21.15.43:0/3696625034 >> 172.21.15.43:0/3696625034 conn(0x7f82200b1480 msgr2=0x7f822015ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:12.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.481+0000 7f8226476640 1 -- 172.21.15.43:0/3696625034 shutdown_connections 2024-04-25T09:02:12.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.481+0000 7f8226476640 1 -- 172.21.15.43:0/3696625034 wait complete. 2024-04-25T09:02:12.481 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:02:12.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.594+0000 7ff7dbc5d640 1 Processor -- start 2024-04-25T09:02:12.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.594+0000 7ff7dbc5d640 1 -- start start 2024-04-25T09:02:12.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.595+0000 7ff7dbc5d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff7d40d21f0 0x7ff7d40cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:12.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.595+0000 7ff7dbc5d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7d40cef00 con 0x7ff7d415f4e0 2024-04-25T09:02:12.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.595+0000 7ff7dbc5d640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7d40cc5b0 con 0x7ff7d406e530 2024-04-25T09:02:12.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.595+0000 7ff7dbc5d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7d40d26a0 con 0x7ff7d40d21f0 2024-04-25T09:02:12.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.595+0000 7ff7d91f8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff7d40d21f0 0x7ff7d40cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:12.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.595+0000 7ff7d99f9640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff7d415f4e0 0x7ff7d415f8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32862/0 (socket says 172.21.15.43:32862) 2024-04-25T09:02:12.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.595+0000 7ff7d99f9640 1 -- 172.21.15.43:0/4266461242 learned_addr learned my addr 172.21.15.43:0/4266461242 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:12.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.595+0000 7ff7d91f8640 1 -- 172.21.15.43:0/4266461242 >> v1:172.21.15.193:6789/0 conn(0x7ff7d406e530 legacy=0x7ff7d406e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:12.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.595+0000 7ff7d91f8640 1 -- 172.21.15.43:0/4266461242 >> v1:172.21.15.43:6789/0 conn(0x7ff7d415f4e0 legacy=0x7ff7d415f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:12.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.595+0000 7ff7d91f8640 1 -- 172.21.15.43:0/4266461242 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d40cc890 con 0x7ff7d40d21f0 2024-04-25T09:02:12.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.595+0000 7ff7d89f7640 1 -- 172.21.15.43:0/4266461242 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 103189234 0 0) 0x7ff7d40cef00 con 0x7ff7d415f4e0 2024-04-25T09:02:12.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.596+0000 7ff7d91f8640 1 --2- 172.21.15.43:0/4266461242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7d40d21f0 0x7ff7d40cd090 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7ff7c80097a0 tx=0x7ff7c802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b24d8f14baa5df28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:12.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.596+0000 7ff7d89f7640 1 -- 172.21.15.43:0/4266461242 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c8037070 con 0x7ff7d40d21f0 2024-04-25T09:02:12.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.596+0000 7ff7d89f7640 1 -- 172.21.15.43:0/4266461242 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7c8032440 con 0x7ff7d40d21f0 2024-04-25T09:02:12.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.596+0000 7ff7d89f7640 1 -- 172.21.15.43:0/4266461242 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c8032810 con 0x7ff7d40d21f0 2024-04-25T09:02:12.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.597+0000 7ff7dbc5d640 1 -- 172.21.15.43:0/4266461242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7d40d21f0 msgr2=0x7ff7d40cd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:12.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.597+0000 7ff7dbc5d640 1 --2- 172.21.15.43:0/4266461242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7d40d21f0 0x7ff7d40cd090 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7ff7c80097a0 tx=0x7ff7c802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.597+0000 7ff7dbc5d640 1 -- 172.21.15.43:0/4266461242 shutdown_connections 2024-04-25T09:02:12.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.597+0000 7ff7dbc5d640 1 --2- 172.21.15.43:0/4266461242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7d40d21f0 0x7ff7d40cd090 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.597+0000 7ff7dbc5d640 1 -- 172.21.15.43:0/4266461242 >> 172.21.15.43:0/4266461242 conn(0x7ff7d40d1700 msgr2=0x7ff7d40d1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:12.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.597+0000 7ff7dbc5d640 1 -- 172.21.15.43:0/4266461242 shutdown_connections 2024-04-25T09:02:12.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.597+0000 7ff7dbc5d640 1 -- 172.21.15.43:0/4266461242 wait complete. 2024-04-25T09:02:12.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.598+0000 7ff7dbc5d640 1 Processor -- start 2024-04-25T09:02:12.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.598+0000 7ff7dbc5d640 1 -- start start 2024-04-25T09:02:12.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.599+0000 7ff7dbc5d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7d406e530 0x7ff7d414ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:12.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.599+0000 7ff7dbc5d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7d40d21f0 0x7ff7d40b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:12.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.599+0000 7ff7dbc5d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7d415f4e0 0x7ff7d40c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:12.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.599+0000 7ff7dbc5d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d40cd5d0 con 0x7ff7d406e530 2024-04-25T09:02:12.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.599+0000 7ff7dbc5d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d40cc2a0 con 0x7ff7d40d21f0 2024-04-25T09:02:12.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.599+0000 7ff7dbc5d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d40ceb20 con 0x7ff7d415f4e0 2024-04-25T09:02:12.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.599+0000 7ff7d99f9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7d406e530 0x7ff7d414ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:12.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.599+0000 7ff7d99f9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7d406e530 0x7ff7d414ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51192/0 (socket says 172.21.15.43:51192) 2024-04-25T09:02:12.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.599+0000 7ff7d99f9640 1 -- 172.21.15.43:0/2945510783 learned_addr learned my addr 172.21.15.43:0/2945510783 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:12.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.599+0000 7ff7d91f8640 1 --2- 172.21.15.43:0/2945510783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7d40d21f0 0x7ff7d40b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:12.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.599+0000 7ff7da1fa640 1 --2- 172.21.15.43:0/2945510783 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7d415f4e0 0x7ff7d40c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:12.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.599+0000 7ff7d99f9640 1 -- 172.21.15.43:0/2945510783 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7d415f4e0 msgr2=0x7ff7d40c97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:12.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.599+0000 7ff7d99f9640 1 --2- 172.21.15.43:0/2945510783 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7d415f4e0 0x7ff7d40c97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.599+0000 7ff7d99f9640 1 -- 172.21.15.43:0/2945510783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7d40d21f0 msgr2=0x7ff7d40b5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:12.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.599+0000 7ff7d99f9640 1 --2- 172.21.15.43:0/2945510783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7d40d21f0 0x7ff7d40b5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.599+0000 7ff7d99f9640 1 -- 172.21.15.43:0/2945510783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d40c8a20 con 0x7ff7d406e530 2024-04-25T09:02:12.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.599+0000 7ff7d91f8640 1 --2- 172.21.15.43:0/2945510783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7d40d21f0 0x7ff7d40b5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:12.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.599+0000 7ff7da1fa640 1 --2- 172.21.15.43:0/2945510783 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7d415f4e0 0x7ff7d40c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:12.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.599+0000 7ff7d99f9640 1 --2- 172.21.15.43:0/2945510783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7d406e530 0x7ff7d414ebe0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000c430 tx=0x7ff7d000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:12.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.599+0000 7ff7c67fc640 1 -- 172.21.15.43:0/2945510783 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d00074e0 con 0x7ff7d406e530 2024-04-25T09:02:12.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.600+0000 7ff7c67fc640 1 -- 172.21.15.43:0/2945510783 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7d00047e0 con 0x7ff7d406e530 2024-04-25T09:02:12.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.600+0000 7ff7dbc5d640 1 -- 172.21.15.43:0/2945510783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7d40c8d00 con 0x7ff7d406e530 2024-04-25T09:02:12.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.600+0000 7ff7c67fc640 1 -- 172.21.15.43:0/2945510783 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d0004bd0 con 0x7ff7d406e530 2024-04-25T09:02:12.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.600+0000 7ff7dbc5d640 1 -- 172.21.15.43:0/2945510783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7d40c92c0 con 0x7ff7d406e530 2024-04-25T09:02:12.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.601+0000 7ff7dbc5d640 1 -- 172.21.15.43:0/2945510783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7d41601c0 con 0x7ff7d406e530 2024-04-25T09:02:12.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.601+0000 7ff7c67fc640 1 -- 172.21.15.43:0/2945510783 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff7d0011070 con 0x7ff7d406e530 2024-04-25T09:02:12.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.601+0000 7ff7c67fc640 1 --2- 172.21.15.43:0/2945510783 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff7b806b4d0 0x7ff7b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:12.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.602+0000 7ff7c67fc640 1 -- 172.21.15.43:0/2945510783 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff7d0089ee0 con 0x7ff7d406e530 2024-04-25T09:02:12.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.602+0000 7ff7d91f8640 1 --2- 172.21.15.43:0/2945510783 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff7b806b4d0 0x7ff7b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:12.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.602+0000 7ff7d91f8640 1 --2- 172.21.15.43:0/2945510783 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff7b806b4d0 0x7ff7b806d990 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8034000 tx=0x7ff7c802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:12.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.606+0000 7ff7c67fc640 1 -- 172.21.15.43:0/2945510783 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff7d41601c0 con 0x7ff7d406e530 2024-04-25T09:02:12.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.787+0000 7ff7dbc5d640 1 -- 172.21.15.43:0/2945510783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7d4074ef0 con 0x7ff7d406e530 2024-04-25T09:02:12.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.788+0000 7ff7c67fc640 1 -- 172.21.15.43:0/2945510783 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+16583 (secure 0 0 0) 0x7ff7d000e280 con 0x7ff7d406e530 2024-04-25T09:02:12.777 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:12.777 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":102,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:01:35.356828+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:01:31.997813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T09:01:32.695222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:02:12.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.790+0000 7ff7dbc5d640 1 -- 172.21.15.43:0/2945510783 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff7b806b4d0 msgr2=0x7ff7b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:12.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.790+0000 7ff7dbc5d640 1 --2- 172.21.15.43:0/2945510783 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff7b806b4d0 0x7ff7b806d990 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8034000 tx=0x7ff7c802e000 comp rx=0 tx=0).stop 2024-04-25T09:02:12.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.790+0000 7ff7dbc5d640 1 -- 172.21.15.43:0/2945510783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7d406e530 msgr2=0x7ff7d414ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:12.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.790+0000 7ff7dbc5d640 1 --2- 172.21.15.43:0/2945510783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7d406e530 0x7ff7d414ebe0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7ff7d000c430 tx=0x7ff7d000c8f0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.790+0000 7ff7dbc5d640 1 -- 172.21.15.43:0/2945510783 shutdown_connections 2024-04-25T09:02:12.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.790+0000 7ff7dbc5d640 1 --2- 172.21.15.43:0/2945510783 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff7d415f4e0 0x7ff7d40c97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.790+0000 7ff7dbc5d640 1 --2- 172.21.15.43:0/2945510783 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff7b806b4d0 0x7ff7b806d990 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.790+0000 7ff7dbc5d640 1 --2- 172.21.15.43:0/2945510783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff7d40d21f0 0x7ff7d40b5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.790+0000 7ff7dbc5d640 1 --2- 172.21.15.43:0/2945510783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff7d406e530 0x7ff7d414ebe0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.790+0000 7ff7dbc5d640 1 -- 172.21.15.43:0/2945510783 >> 172.21.15.43:0/2945510783 conn(0x7ff7d40d1700 msgr2=0x7ff7d40b2480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:12.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.790+0000 7ff7dbc5d640 1 -- 172.21.15.43:0/2945510783 shutdown_connections 2024-04-25T09:02:12.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.790+0000 7ff7dbc5d640 1 -- 172.21.15.43:0/2945510783 wait complete. 2024-04-25T09:02:12.790 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:01:31.997813+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:01:32.695222+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:02:12.791 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:02:12.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.905+0000 7fa532c38640 1 Processor -- start 2024-04-25T09:02:12.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.905+0000 7fa532c38640 1 -- start start 2024-04-25T09:02:12.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.906+0000 7fa532c38640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fa52c0aee80 0x7fa52c06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:12.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.906+0000 7fa532c38640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa52c06d600 0x7fa52c0ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:12.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.906+0000 7fa532c38640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa52c15c280 con 0x7fa52c0cd120 2024-04-25T09:02:12.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.906+0000 7fa532c38640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa52c0af260 con 0x7fa52c0aee80 2024-04-25T09:02:12.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.906+0000 7fa532c38640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa52c06da60 con 0x7fa52c06d600 2024-04-25T09:02:12.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.906+0000 7fa52bfff640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa52c06d600 0x7fa52c0ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:12.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.906+0000 7fa5309d4640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fa52c0aee80 0x7fa52c06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:12.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.906+0000 7fa52bfff640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa52c06d600 0x7fa52c0ccbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:60226/0 (socket says 172.21.15.43:60226) 2024-04-25T09:02:12.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.906+0000 7fa5309d4640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fa52c0aee80 0x7fa52c06d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:38446/0 (socket says 172.21.15.43:38446) 2024-04-25T09:02:12.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.906+0000 7fa52bfff640 1 -- 172.21.15.43:0/500776326 learned_addr learned my addr 172.21.15.43:0/500776326 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:12.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.906+0000 7fa5311d5640 1 --1- 172.21.15.43:0/500776326 >> v1:172.21.15.193:6789/0 conn(0x7fa52c0cd120 0x7fa52c15f6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:52914/0 (socket says 172.21.15.43:52914) 2024-04-25T09:02:12.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.907+0000 7fa52bfff640 1 -- 172.21.15.43:0/500776326 >> v2:172.21.15.193:3300/0 conn(0x7fa52c0aee80 msgr2=0x7fa52c06d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:12.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.907+0000 7fa52bfff640 1 --2- 172.21.15.43:0/500776326 >> v2:172.21.15.193:3300/0 conn(0x7fa52c0aee80 0x7fa52c06d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.907+0000 7fa52bfff640 1 -- 172.21.15.43:0/500776326 >> v1:172.21.15.193:6789/0 conn(0x7fa52c0cd120 legacy=0x7fa52c15f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:12.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.907+0000 7fa52bfff640 1 -- 172.21.15.43:0/500776326 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa52c15c560 con 0x7fa52c06d600 2024-04-25T09:02:12.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.907+0000 7fa5309d4640 1 --2- 172.21.15.43:0/500776326 >> v2:172.21.15.193:3300/0 conn(0x7fa52c0aee80 0x7fa52c06d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:02:12.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.908+0000 7fa52bfff640 1 --2- 172.21.15.43:0/500776326 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa52c06d600 0x7fa52c0ccbe0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa514009870 tx=0x7fa51402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eab7b706dbc73fec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:12.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.908+0000 7fa52b7fe640 1 -- 172.21.15.43:0/500776326 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa514037070 con 0x7fa52c06d600 2024-04-25T09:02:12.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.908+0000 7fa52b7fe640 1 -- 172.21.15.43:0/500776326 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa514032440 con 0x7fa52c06d600 2024-04-25T09:02:12.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.908+0000 7fa52b7fe640 1 -- 172.21.15.43:0/500776326 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa514032810 con 0x7fa52c06d600 2024-04-25T09:02:12.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.908+0000 7fa532c38640 1 -- 172.21.15.43:0/500776326 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa52c06d600 msgr2=0x7fa52c0ccbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:12.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.908+0000 7fa532c38640 1 --2- 172.21.15.43:0/500776326 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa52c06d600 0x7fa52c0ccbe0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa514009870 tx=0x7fa51402a270 comp rx=0 tx=0).stop 2024-04-25T09:02:12.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.908+0000 7fa532c38640 1 -- 172.21.15.43:0/500776326 shutdown_connections 2024-04-25T09:02:12.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.908+0000 7fa532c38640 1 --2- 172.21.15.43:0/500776326 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa52c06d600 0x7fa52c0ccbe0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.908+0000 7fa532c38640 1 --2- 172.21.15.43:0/500776326 >> v2:172.21.15.193:3300/0 conn(0x7fa52c0aee80 0x7fa52c06d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.908+0000 7fa532c38640 1 -- 172.21.15.43:0/500776326 >> 172.21.15.43:0/500776326 conn(0x7fa52c06ab30 msgr2=0x7fa52c06af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:12.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.908+0000 7fa532c38640 1 -- 172.21.15.43:0/500776326 shutdown_connections 2024-04-25T09:02:12.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.908+0000 7fa532c38640 1 -- 172.21.15.43:0/500776326 wait complete. 2024-04-25T09:02:12.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.910+0000 7fa532c38640 1 Processor -- start 2024-04-25T09:02:12.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.910+0000 7fa532c38640 1 -- start start 2024-04-25T09:02:12.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.910+0000 7fa532c38640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa52c06d600 0x7fa52c1f5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:12.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.911+0000 7fa532c38640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa52c0aee80 0x7fa52c1f6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:12.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.911+0000 7fa532c38640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa52c0cd120 0x7fa52c071c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:12.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.911+0000 7fa532c38640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52c15cb20 con 0x7fa52c0cd120 2024-04-25T09:02:12.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.911+0000 7fa532c38640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52c0cd5b0 con 0x7fa52c0aee80 2024-04-25T09:02:12.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.911+0000 7fa532c38640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa52c15e5a0 con 0x7fa52c06d600 2024-04-25T09:02:12.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.911+0000 7fa5311d5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa52c0cd120 0x7fa52c071c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:12.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.911+0000 7fa5311d5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa52c0cd120 0x7fa52c071c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51206/0 (socket says 172.21.15.43:51206) 2024-04-25T09:02:12.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.911+0000 7fa5309d4640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa52c06d600 0x7fa52c1f5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:12.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.911+0000 7fa5311d5640 1 -- 172.21.15.43:0/1762644198 learned_addr learned my addr 172.21.15.43:0/1762644198 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:12.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.911+0000 7fa52bfff640 1 --2- 172.21.15.43:0/1762644198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa52c0aee80 0x7fa52c1f6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:12.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.911+0000 7fa5311d5640 1 -- 172.21.15.43:0/1762644198 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa52c06d600 msgr2=0x7fa52c1f5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:12.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.911+0000 7fa5311d5640 1 --2- 172.21.15.43:0/1762644198 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa52c06d600 0x7fa52c1f5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.911+0000 7fa5311d5640 1 -- 172.21.15.43:0/1762644198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa52c0aee80 msgr2=0x7fa52c1f6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:12.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.911+0000 7fa5311d5640 1 --2- 172.21.15.43:0/1762644198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa52c0aee80 0x7fa52c1f6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:12.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.911+0000 7fa5311d5640 1 -- 172.21.15.43:0/1762644198 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa52c141be0 con 0x7fa52c0cd120 2024-04-25T09:02:12.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.911+0000 7fa5311d5640 1 --2- 172.21.15.43:0/1762644198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa52c0cd120 0x7fa52c071c30 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fa51c001eb0 tx=0x7fa51c00c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:12.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.911+0000 7fa5297fa640 1 -- 172.21.15.43:0/1762644198 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51c0074e0 con 0x7fa52c0cd120 2024-04-25T09:02:12.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.911+0000 7fa5297fa640 1 -- 172.21.15.43:0/1762644198 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa51c007680 con 0x7fa52c0cd120 2024-04-25T09:02:12.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.911+0000 7fa5297fa640 1 -- 172.21.15.43:0/1762644198 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa51c0049e0 con 0x7fa52c0cd120 2024-04-25T09:02:12.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.911+0000 7fa532c38640 1 -- 172.21.15.43:0/1762644198 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa52c141ec0 con 0x7fa52c0cd120 2024-04-25T09:02:12.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.911+0000 7fa532c38640 1 -- 172.21.15.43:0/1762644198 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa52c1f7250 con 0x7fa52c0cd120 2024-04-25T09:02:12.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.912+0000 7fa532c38640 1 -- 172.21.15.43:0/1762644198 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa52c074f60 con 0x7fa52c0cd120 2024-04-25T09:02:12.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.913+0000 7fa5297fa640 1 -- 172.21.15.43:0/1762644198 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa51c01a020 con 0x7fa52c0cd120 2024-04-25T09:02:12.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.913+0000 7fa5297fa640 1 --2- 172.21.15.43:0/1762644198 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa50c06b4d0 0x7fa50c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:12.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.913+0000 7fa5297fa640 1 -- 172.21.15.43:0/1762644198 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa51c089740 con 0x7fa52c0cd120 2024-04-25T09:02:12.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.914+0000 7fa5309d4640 1 --2- 172.21.15.43:0/1762644198 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa50c06b4d0 0x7fa50c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:12.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.914+0000 7fa5309d4640 1 --2- 172.21.15.43:0/1762644198 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa50c06b4d0 0x7fa50c06d990 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa524009bd0 tx=0x7fa52400b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:12.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:12.917+0000 7fa5297fa640 1 -- 172.21.15.43:0/1762644198 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa51c058000 con 0x7fa52c0cd120 2024-04-25T09:02:13.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.096+0000 7fa532c38640 1 -- 172.21.15.43:0/1762644198 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa52c0725f0 con 0x7fa52c0cd120 2024-04-25T09:02:13.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.096+0000 7fa5297fa640 1 -- 172.21.15.43:0/1762644198 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v102) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa51c05c0a0 con 0x7fa52c0cd120 2024-04-25T09:02:13.085 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T09:02:13.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.098+0000 7fa532c38640 1 -- 172.21.15.43:0/1762644198 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa50c06b4d0 msgr2=0x7fa50c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:13.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.098+0000 7fa532c38640 1 --2- 172.21.15.43:0/1762644198 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa50c06b4d0 0x7fa50c06d990 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa524009bd0 tx=0x7fa52400b040 comp rx=0 tx=0).stop 2024-04-25T09:02:13.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.098+0000 7fa532c38640 1 -- 172.21.15.43:0/1762644198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa52c0cd120 msgr2=0x7fa52c071c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:13.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.098+0000 7fa532c38640 1 --2- 172.21.15.43:0/1762644198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa52c0cd120 0x7fa52c071c30 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fa51c001eb0 tx=0x7fa51c00c820 comp rx=0 tx=0).stop 2024-04-25T09:02:13.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.098+0000 7fa532c38640 1 -- 172.21.15.43:0/1762644198 shutdown_connections 2024-04-25T09:02:13.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.098+0000 7fa532c38640 1 --2- 172.21.15.43:0/1762644198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa52c0cd120 0x7fa52c071c30 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.098+0000 7fa532c38640 1 --2- 172.21.15.43:0/1762644198 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa50c06b4d0 0x7fa50c06d990 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.098+0000 7fa532c38640 1 --2- 172.21.15.43:0/1762644198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa52c0aee80 0x7fa52c1f6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.098+0000 7fa532c38640 1 --2- 172.21.15.43:0/1762644198 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa52c06d600 0x7fa52c1f5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.098+0000 7fa532c38640 1 -- 172.21.15.43:0/1762644198 >> 172.21.15.43:0/1762644198 conn(0x7fa52c06ab30 msgr2=0x7fa52c0b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:13.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.098+0000 7fa532c38640 1 -- 172.21.15.43:0/1762644198 shutdown_connections 2024-04-25T09:02:13.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.098+0000 7fa532c38640 1 -- 172.21.15.43:0/1762644198 wait complete. 2024-04-25T09:02:13.098 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:02:13.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.213+0000 7f62bdb90640 1 Processor -- start 2024-04-25T09:02:13.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.213+0000 7f62bdb90640 1 -- start start 2024-04-25T09:02:13.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.214+0000 7f62bdb90640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f62b80aee40 0x7f62b8168680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.214+0000 7f62bdb90640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62b80b4e60 con 0x7f62b80b42a0 2024-04-25T09:02:13.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.214+0000 7f62bdb90640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62b815e160 con 0x7f62b806ab30 2024-04-25T09:02:13.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.214+0000 7f62bdb90640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62b815e400 con 0x7f62b80aee40 2024-04-25T09:02:13.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.214+0000 7f62b6ffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f62b80b42a0 0x7f62b80b4750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32866/0 (socket says 172.21.15.43:32866) 2024-04-25T09:02:13.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.214+0000 7f62b7fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f62b80aee40 0x7f62b8168680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.214+0000 7f62b6ffd640 1 -- 172.21.15.43:0/768619670 learned_addr learned my addr 172.21.15.43:0/768619670 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:13.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.215+0000 7f62b67fc640 1 -- 172.21.15.43:0/768619670 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 958940470 0 0) 0x7f62b80b4e60 con 0x7f62b80b42a0 2024-04-25T09:02:13.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.215+0000 7f62b67fc640 1 -- 172.21.15.43:0/768619670 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62a0003520 con 0x7f62b80b42a0 2024-04-25T09:02:13.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.215+0000 7f62b7fff640 1 -- 172.21.15.43:0/768619670 >> v1:172.21.15.193:6789/0 conn(0x7f62b806ab30 legacy=0x7f62b806af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:13.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.215+0000 7f62b7fff640 1 -- 172.21.15.43:0/768619670 >> v1:172.21.15.43:6789/0 conn(0x7f62b80b42a0 legacy=0x7f62b80b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:13.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.215+0000 7f62b7fff640 1 -- 172.21.15.43:0/768619670 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b815e6e0 con 0x7f62b80aee40 2024-04-25T09:02:13.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.215+0000 7f62b7fff640 1 --2- 172.21.15.43:0/768619670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62b80aee40 0x7f62b8168680 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f62b0009dc0 tx=0x7f62b002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe5536609f79320f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:13.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.215+0000 7f62b67fc640 1 -- 172.21.15.43:0/768619670 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b0007740 con 0x7f62b80aee40 2024-04-25T09:02:13.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.215+0000 7f62b67fc640 1 -- 172.21.15.43:0/768619670 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62b00078e0 con 0x7f62b80aee40 2024-04-25T09:02:13.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.215+0000 7f62b67fc640 1 -- 172.21.15.43:0/768619670 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b0007cb0 con 0x7f62b80aee40 2024-04-25T09:02:13.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.216+0000 7f62bdb90640 1 -- 172.21.15.43:0/768619670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62b80aee40 msgr2=0x7f62b8168680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:13.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.216+0000 7f62bdb90640 1 --2- 172.21.15.43:0/768619670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62b80aee40 0x7f62b8168680 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f62b0009dc0 tx=0x7f62b002a730 comp rx=0 tx=0).stop 2024-04-25T09:02:13.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.216+0000 7f62bdb90640 1 -- 172.21.15.43:0/768619670 shutdown_connections 2024-04-25T09:02:13.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.216+0000 7f62bdb90640 1 --2- 172.21.15.43:0/768619670 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62b80aee40 0x7f62b8168680 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.216+0000 7f62bdb90640 1 -- 172.21.15.43:0/768619670 >> 172.21.15.43:0/768619670 conn(0x7f62b80b2590 msgr2=0x7f62b80b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:13.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.216+0000 7f62bdb90640 1 -- 172.21.15.43:0/768619670 shutdown_connections 2024-04-25T09:02:13.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.216+0000 7f62bdb90640 1 -- 172.21.15.43:0/768619670 wait complete. 2024-04-25T09:02:13.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.217+0000 7f62bdb90640 1 Processor -- start 2024-04-25T09:02:13.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.217+0000 7f62bdb90640 1 -- start start 2024-04-25T09:02:13.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.218+0000 7f62bdb90640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62b806ab30 0x7f62b81f6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.218+0000 7f62bdb90640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62b80aee40 0x7f62b80b1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.218+0000 7f62bdb90640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62b80b42a0 0x7f62b81e4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.218+0000 7f62bdb90640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b815b950 con 0x7f62b80aee40 2024-04-25T09:02:13.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.218+0000 7f62bdb90640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62b815eca0 con 0x7f62b806ab30 2024-04-25T09:02:13.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.218+0000 7f62bdb90640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62b815bad0 con 0x7f62b80b42a0 2024-04-25T09:02:13.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.218+0000 7f62b6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62b80aee40 0x7f62b80b1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.218+0000 7f62b6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62b80aee40 0x7f62b80b1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51228/0 (socket says 172.21.15.43:51228) 2024-04-25T09:02:13.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.218+0000 7f62b6ffd640 1 -- 172.21.15.43:0/3522270031 learned_addr learned my addr 172.21.15.43:0/3522270031 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:13.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.219+0000 7f62b6ffd640 1 -- 172.21.15.43:0/3522270031 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62b80b42a0 msgr2=0x7f62b81e4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:13.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.219+0000 7f62b6ffd640 1 --2- 172.21.15.43:0/3522270031 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62b80b42a0 0x7f62b81e4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.219+0000 7f62b6ffd640 1 -- 172.21.15.43:0/3522270031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62b806ab30 msgr2=0x7f62b81f6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:13.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.219+0000 7f62b6ffd640 1 --2- 172.21.15.43:0/3522270031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62b806ab30 0x7f62b81f6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.219+0000 7f62b6ffd640 1 -- 172.21.15.43:0/3522270031 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b81548d0 con 0x7f62b80aee40 2024-04-25T09:02:13.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.219+0000 7f62b6ffd640 1 --2- 172.21.15.43:0/3522270031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62b80aee40 0x7f62b80b1210 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f62ac00c440 tx=0x7f62ac00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:13.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.219+0000 7f629ffff640 1 -- 172.21.15.43:0/3522270031 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ac017070 con 0x7f62b80aee40 2024-04-25T09:02:13.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.219+0000 7f629ffff640 1 -- 172.21.15.43:0/3522270031 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62ac005ce0 con 0x7f62b80aee40 2024-04-25T09:02:13.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.219+0000 7f62bdb90640 1 -- 172.21.15.43:0/3522270031 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62b8154bb0 con 0x7f62b80aee40 2024-04-25T09:02:13.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.219+0000 7f629ffff640 1 -- 172.21.15.43:0/3522270031 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62ac0072b0 con 0x7f62b80aee40 2024-04-25T09:02:13.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.219+0000 7f62bdb90640 1 -- 172.21.15.43:0/3522270031 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62b8155170 con 0x7f62b80aee40 2024-04-25T09:02:13.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.220+0000 7f62bdb90640 1 -- 172.21.15.43:0/3522270031 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62b8074f60 con 0x7f62b80aee40 2024-04-25T09:02:13.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.221+0000 7f629ffff640 1 -- 172.21.15.43:0/3522270031 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f62ac01a070 con 0x7f62b80aee40 2024-04-25T09:02:13.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.221+0000 7f629ffff640 1 --2- 172.21.15.43:0/3522270031 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f62a406b4d0 0x7f62a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.221+0000 7f629ffff640 1 -- 172.21.15.43:0/3522270031 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62ac0894c0 con 0x7f62b80aee40 2024-04-25T09:02:13.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.221+0000 7f62b77fe640 1 --2- 172.21.15.43:0/3522270031 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f62a406b4d0 0x7f62a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.222+0000 7f62b77fe640 1 --2- 172.21.15.43:0/3522270031 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f62a406b4d0 0x7f62a406d990 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f62a8000f60 tx=0x7f62a80079d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:13.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.225+0000 7f629ffff640 1 -- 172.21.15.43:0/3522270031 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f62ac058490 con 0x7f62b80aee40 2024-04-25T09:02:13.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.407+0000 7f62bdb90640 1 -- 172.21.15.43:0/3522270031 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f62b8156790 con 0x7f62b80aee40 2024-04-25T09:02:13.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.407+0000 7f629ffff640 1 -- 172.21.15.43:0/3522270031 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v102) v1 ==== 104+0+11 (secure 0 0 0) 0x7f62ac057d90 con 0x7f62b80aee40 2024-04-25T09:02:13.396 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T09:02:13.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.409+0000 7f62bdb90640 1 -- 172.21.15.43:0/3522270031 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f62a406b4d0 msgr2=0x7f62a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:13.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.409+0000 7f62bdb90640 1 --2- 172.21.15.43:0/3522270031 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f62a406b4d0 0x7f62a406d990 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f62a8000f60 tx=0x7f62a80079d0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.409+0000 7f62bdb90640 1 -- 172.21.15.43:0/3522270031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62b80aee40 msgr2=0x7f62b80b1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:13.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.409+0000 7f62bdb90640 1 --2- 172.21.15.43:0/3522270031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62b80aee40 0x7f62b80b1210 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f62ac00c440 tx=0x7f62ac00c900 comp rx=0 tx=0).stop 2024-04-25T09:02:13.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.409+0000 7f62bdb90640 1 -- 172.21.15.43:0/3522270031 shutdown_connections 2024-04-25T09:02:13.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.409+0000 7f62bdb90640 1 --2- 172.21.15.43:0/3522270031 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62b80b42a0 0x7f62b81e4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.409+0000 7f62bdb90640 1 --2- 172.21.15.43:0/3522270031 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62b80aee40 0x7f62b80b1210 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.409+0000 7f62bdb90640 1 --2- 172.21.15.43:0/3522270031 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f62a406b4d0 0x7f62a406d990 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.409+0000 7f62bdb90640 1 --2- 172.21.15.43:0/3522270031 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f62b806ab30 0x7f62b81f6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.409+0000 7f62bdb90640 1 -- 172.21.15.43:0/3522270031 >> 172.21.15.43:0/3522270031 conn(0x7f62b80b2590 msgr2=0x7f62b806af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:13.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.409+0000 7f62bdb90640 1 -- 172.21.15.43:0/3522270031 shutdown_connections 2024-04-25T09:02:13.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.409+0000 7f62bdb90640 1 -- 172.21.15.43:0/3522270031 wait complete. 2024-04-25T09:02:13.409 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:02:13.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.525+0000 7fbd024f7640 1 Processor -- start 2024-04-25T09:02:13.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.525+0000 7fbd024f7640 1 -- start start 2024-04-25T09:02:13.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.526+0000 7fbd024f7640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fbcfc0cd2f0 0x7fbcfc166a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.526+0000 7fbd024f7640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcfc15c280 con 0x7fbcfc0aee80 2024-04-25T09:02:13.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.526+0000 7fbd024f7640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcfc15c560 con 0x7fbcfc06d7d0 2024-04-25T09:02:13.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.526+0000 7fbd024f7640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcfc0af260 con 0x7fbcfc0cd2f0 2024-04-25T09:02:13.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.526+0000 7fbcfbfff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fbcfc0aee80 0x7fbcfc06d0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32874/0 (socket says 172.21.15.43:32874) 2024-04-25T09:02:13.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.526+0000 7fbcfbfff640 1 -- 172.21.15.43:0/4284320357 learned_addr learned my addr 172.21.15.43:0/4284320357 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:13.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.526+0000 7fbcfaffd640 1 -- 172.21.15.43:0/4284320357 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2047681470 0 0) 0x7fbcfc15c280 con 0x7fbcfc0aee80 2024-04-25T09:02:13.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.526+0000 7fbcfaffd640 1 -- 172.21.15.43:0/4284320357 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbce4003520 con 0x7fbcfc0aee80 2024-04-25T09:02:13.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.526+0000 7fbd00a94640 1 --2- 172.21.15.43:0/4284320357 >> v2:172.21.15.193:3300/0 conn(0x7fbcfc0cd2f0 0x7fbcfc166a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.526+0000 7fbcfaffd640 1 -- 172.21.15.43:0/4284320357 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3107518959 0 0) 0x7fbce4003520 con 0x7fbcfc0aee80 2024-04-25T09:02:13.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.526+0000 7fbcfaffd640 1 -- 172.21.15.43:0/4284320357 >> v2:172.21.15.193:3300/0 conn(0x7fbcfc0cd2f0 msgr2=0x7fbcfc166a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:13.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.526+0000 7fbcfaffd640 1 --2- 172.21.15.43:0/4284320357 >> v2:172.21.15.193:3300/0 conn(0x7fbcfc0cd2f0 0x7fbcfc166a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.527+0000 7fbcfaffd640 1 -- 172.21.15.43:0/4284320357 >> v1:172.21.15.193:6789/0 conn(0x7fbcfc06d7d0 legacy=0x7fbcfc0ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:13.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.527+0000 7fbcfaffd640 1 -- 172.21.15.43:0/4284320357 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcfc15c840 con 0x7fbcfc0aee80 2024-04-25T09:02:13.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.527+0000 7fbcfaffd640 1 -- 172.21.15.43:0/4284320357 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7fbcf0001670 con 0x7fbcfc0aee80 2024-04-25T09:02:13.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.527+0000 7fbcfaffd640 1 -- 172.21.15.43:0/4284320357 >> v1:172.21.15.43:6789/0 conn(0x7fbcfc0aee80 legacy=0x7fbcfc06d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:13.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.527+0000 7fbcfaffd640 1 --2- 172.21.15.43:0/4284320357 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbce40039b0 0x7fbce4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.527+0000 7fbcfaffd640 1 --2- 172.21.15.43:0/4284320357 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbce4004800 0x7fbce4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.527+0000 7fbcfaffd640 1 --2- 172.21.15.43:0/4284320357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbce40051c0 0x7fbce400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.527+0000 7fbcfaffd640 1 -- 172.21.15.43:0/4284320357 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce400c030 con 0x7fbce40051c0 2024-04-25T09:02:13.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.527+0000 7fbcfaffd640 1 -- 172.21.15.43:0/4284320357 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbce400c1e0 con 0x7fbce40039b0 2024-04-25T09:02:13.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.527+0000 7fbcfaffd640 1 -- 172.21.15.43:0/4284320357 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbce400c3e0 con 0x7fbce4004800 2024-04-25T09:02:13.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.527+0000 7fbd00a94640 1 --2- 172.21.15.43:0/4284320357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbce40051c0 0x7fbce400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.527+0000 7fbcfbfff640 1 --2- 172.21.15.43:0/4284320357 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbce40039b0 0x7fbce4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.527+0000 7fbcfb7fe640 1 --2- 172.21.15.43:0/4284320357 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbce4004800 0x7fbce4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.527+0000 7fbd00a94640 1 -- 172.21.15.43:0/4284320357 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbce4004800 msgr2=0x7fbce4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:13.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.527+0000 7fbd00a94640 1 --2- 172.21.15.43:0/4284320357 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbce4004800 0x7fbce4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.527+0000 7fbd00a94640 1 -- 172.21.15.43:0/4284320357 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbce40039b0 msgr2=0x7fbce4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:13.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.527+0000 7fbd00a94640 1 --2- 172.21.15.43:0/4284320357 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbce40039b0 0x7fbce4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.527+0000 7fbd00a94640 1 -- 172.21.15.43:0/4284320357 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce4003520 con 0x7fbce40051c0 2024-04-25T09:02:13.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.528+0000 7fbd00a94640 1 -- 172.21.15.43:0/4284320357 reap_dead start 2024-04-25T09:02:13.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.528+0000 7fbd00a94640 1 --2- 172.21.15.43:0/4284320357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbce40051c0 0x7fbce400b9c0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fbcf4009f90 tx=0x7fbce4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e1480b8cfcd99a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:13.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.528+0000 7fbcfaffd640 1 -- 172.21.15.43:0/4284320357 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf4009c10 con 0x7fbce40051c0 2024-04-25T09:02:13.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.528+0000 7fbcfaffd640 1 -- 172.21.15.43:0/4284320357 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcf4031440 con 0x7fbce40051c0 2024-04-25T09:02:13.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.528+0000 7fbcfaffd640 1 -- 172.21.15.43:0/4284320357 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf40317f0 con 0x7fbce40051c0 2024-04-25T09:02:13.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.528+0000 7fbd024f7640 1 -- 172.21.15.43:0/4284320357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbce40051c0 msgr2=0x7fbce400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:13.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.528+0000 7fbd024f7640 1 --2- 172.21.15.43:0/4284320357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbce40051c0 0x7fbce400b9c0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fbcf4009f90 tx=0x7fbce4005830 comp rx=0 tx=0).stop 2024-04-25T09:02:13.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.528+0000 7fbd024f7640 1 -- 172.21.15.43:0/4284320357 shutdown_connections 2024-04-25T09:02:13.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.528+0000 7fbd024f7640 1 --2- 172.21.15.43:0/4284320357 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbce40051c0 0x7fbce400b9c0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.528+0000 7fbd024f7640 1 -- 172.21.15.43:0/4284320357 >> 172.21.15.43:0/4284320357 conn(0x7fbcfc06ab30 msgr2=0x7fbcfc06af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:13.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.529+0000 7fbd024f7640 1 -- 172.21.15.43:0/4284320357 shutdown_connections 2024-04-25T09:02:13.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.529+0000 7fbd024f7640 1 -- 172.21.15.43:0/4284320357 wait complete. 2024-04-25T09:02:13.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.530+0000 7fbd024f7640 1 Processor -- start 2024-04-25T09:02:13.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.530+0000 7fbd024f7640 1 -- start start 2024-04-25T09:02:13.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.531+0000 7fbd024f7640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbce40051c0 0x7fbcfc144070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.531+0000 7fbd024f7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcfc1546d0 0x7fbcfc0ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.531+0000 7fbd024f7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcfc0ade20 0x7fbcfc1f3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.531+0000 7fbd024f7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcfc0ccd60 con 0x7fbcfc0ade20 2024-04-25T09:02:13.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.531+0000 7fbd024f7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcfc0ccbe0 con 0x7fbcfc1546d0 2024-04-25T09:02:13.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.531+0000 7fbd024f7640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcfc0ccee0 con 0x7fbce40051c0 2024-04-25T09:02:13.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.531+0000 7fbd00a94640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcfc0ade20 0x7fbcfc1f3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.531+0000 7fbd00a94640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcfc0ade20 0x7fbcfc1f3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51252/0 (socket says 172.21.15.43:51252) 2024-04-25T09:02:13.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.531+0000 7fbd00a94640 1 -- 172.21.15.43:0/914970282 learned_addr learned my addr 172.21.15.43:0/914970282 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:13.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.531+0000 7fbcfbfff640 1 --2- 172.21.15.43:0/914970282 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbce40051c0 0x7fbcfc144070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.531+0000 7fbcfb7fe640 1 --2- 172.21.15.43:0/914970282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcfc1546d0 0x7fbcfc0ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.531+0000 7fbd00a94640 1 -- 172.21.15.43:0/914970282 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbce40051c0 msgr2=0x7fbcfc144070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:13.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.531+0000 7fbd00a94640 1 --2- 172.21.15.43:0/914970282 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbce40051c0 0x7fbcfc144070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.531+0000 7fbd00a94640 1 -- 172.21.15.43:0/914970282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcfc1546d0 msgr2=0x7fbcfc0ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:13.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.531+0000 7fbd00a94640 1 --2- 172.21.15.43:0/914970282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcfc1546d0 0x7fbcfc0ad8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.531+0000 7fbd00a94640 1 -- 172.21.15.43:0/914970282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcfc1f4b30 con 0x7fbcfc0ade20 2024-04-25T09:02:13.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.531+0000 7fbd00a94640 1 --2- 172.21.15.43:0/914970282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcfc0ade20 0x7fbcfc1f3d80 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fbcf4009f90 tx=0x7fbcf4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:13.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.531+0000 7fbcf8ff9640 1 -- 172.21.15.43:0/914970282 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf4009c10 con 0x7fbcfc0ade20 2024-04-25T09:02:13.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.531+0000 7fbcf8ff9640 1 -- 172.21.15.43:0/914970282 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcf4043070 con 0x7fbcfc0ade20 2024-04-25T09:02:13.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.531+0000 7fbd024f7640 1 -- 172.21.15.43:0/914970282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcfc153bf0 con 0x7fbcfc0ade20 2024-04-25T09:02:13.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.531+0000 7fbcf8ff9640 1 -- 172.21.15.43:0/914970282 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf403c660 con 0x7fbcfc0ade20 2024-04-25T09:02:13.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.532+0000 7fbd024f7640 1 -- 172.21.15.43:0/914970282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcfc1f42c0 con 0x7fbcfc0ade20 2024-04-25T09:02:13.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.532+0000 7fbd024f7640 1 -- 172.21.15.43:0/914970282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcfc0ce6d0 con 0x7fbcfc0ade20 2024-04-25T09:02:13.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.533+0000 7fbcf8ff9640 1 -- 172.21.15.43:0/914970282 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbcf4005d10 con 0x7fbcfc0ade20 2024-04-25T09:02:13.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.533+0000 7fbcf8ff9640 1 --2- 172.21.15.43:0/914970282 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbce806b4d0 0x7fbce806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.534+0000 7fbcf8ff9640 1 -- 172.21.15.43:0/914970282 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbcf40aaa90 con 0x7fbcfc0ade20 2024-04-25T09:02:13.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.534+0000 7fbcfbfff640 1 --2- 172.21.15.43:0/914970282 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbce806b4d0 0x7fbce806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.534+0000 7fbcfbfff640 1 --2- 172.21.15.43:0/914970282 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbce806b4d0 0x7fbce806d990 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0007c10 tx=0x7fbcf0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:13.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.538+0000 7fbcf8ff9640 1 -- 172.21.15.43:0/914970282 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbcfc0ce6d0 con 0x7fbcfc0ade20 2024-04-25T09:02:13.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.719+0000 7fbd024f7640 1 -- 172.21.15.43:0/914970282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbcfc074f60 con 0x7fbcfc0ade20 2024-04-25T09:02:13.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.719+0000 7fbcf8ff9640 1 -- 172.21.15.43:0/914970282 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v102) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbcf4079a30 con 0x7fbcfc0ade20 2024-04-25T09:02:13.708 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T09:02:13.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.721+0000 7fbd024f7640 1 -- 172.21.15.43:0/914970282 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbce806b4d0 msgr2=0x7fbce806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:13.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.721+0000 7fbd024f7640 1 --2- 172.21.15.43:0/914970282 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbce806b4d0 0x7fbce806d990 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fbcf0007c10 tx=0x7fbcf0009040 comp rx=0 tx=0).stop 2024-04-25T09:02:13.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.721+0000 7fbd024f7640 1 -- 172.21.15.43:0/914970282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcfc0ade20 msgr2=0x7fbcfc1f3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:13.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.721+0000 7fbd024f7640 1 --2- 172.21.15.43:0/914970282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcfc0ade20 0x7fbcfc1f3d80 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fbcf4009f90 tx=0x7fbcf4009750 comp rx=0 tx=0).stop 2024-04-25T09:02:13.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.721+0000 7fbd024f7640 1 -- 172.21.15.43:0/914970282 shutdown_connections 2024-04-25T09:02:13.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.721+0000 7fbd024f7640 1 --2- 172.21.15.43:0/914970282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcfc0ade20 0x7fbcfc1f3d80 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.721+0000 7fbd024f7640 1 --2- 172.21.15.43:0/914970282 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbce806b4d0 0x7fbce806d990 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.721+0000 7fbd024f7640 1 --2- 172.21.15.43:0/914970282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcfc1546d0 0x7fbcfc0ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.721+0000 7fbd024f7640 1 --2- 172.21.15.43:0/914970282 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbce40051c0 0x7fbcfc144070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.721+0000 7fbd024f7640 1 -- 172.21.15.43:0/914970282 >> 172.21.15.43:0/914970282 conn(0x7fbcfc06ab30 msgr2=0x7fbcfc0b26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:13.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.721+0000 7fbd024f7640 1 -- 172.21.15.43:0/914970282 shutdown_connections 2024-04-25T09:02:13.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.721+0000 7fbd024f7640 1 -- 172.21.15.43:0/914970282 wait complete. 2024-04-25T09:02:13.722 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:02:13.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.833+0000 7fa9f5ff5640 1 Processor -- start 2024-04-25T09:02:13.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.834+0000 7fa9f5ff5640 1 -- start start 2024-04-25T09:02:13.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.834+0000 7fa9f5ff5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa9f015df50 0x7fa9f01619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.834+0000 7fa9f5ff5640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fa9f00cf670 0x7fa9f0162380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.834+0000 7fa9f5ff5640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9f00d24a0 con 0x7fa9f0161f30 2024-04-25T09:02:13.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.834+0000 7fa9f5ff5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9f00cfa80 con 0x7fa9f015df50 2024-04-25T09:02:13.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.834+0000 7fa9f5ff5640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9f015e330 con 0x7fa9f00cf670 2024-04-25T09:02:13.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.834+0000 7fa9ef7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa9f015df50 0x7fa9f01619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.834+0000 7fa9ef7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa9f015df50 0x7fa9f01619f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51266/0 (socket says 172.21.15.43:51266) 2024-04-25T09:02:13.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.834+0000 7fa9ef7fe640 1 -- 172.21.15.43:0/3055314735 learned_addr learned my addr 172.21.15.43:0/3055314735 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:13.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.835+0000 7fa9effff640 1 --2- 172.21.15.43:0/3055314735 >> v2:172.21.15.193:3300/0 conn(0x7fa9f00cf670 0x7fa9f0162380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.835+0000 7fa9ef7fe640 1 -- 172.21.15.43:0/3055314735 >> v2:172.21.15.193:3300/0 conn(0x7fa9f00cf670 msgr2=0x7fa9f0162380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:13.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.835+0000 7fa9ef7fe640 1 --2- 172.21.15.43:0/3055314735 >> v2:172.21.15.193:3300/0 conn(0x7fa9f00cf670 0x7fa9f0162380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.835+0000 7fa9ef7fe640 1 -- 172.21.15.43:0/3055314735 >> v1:172.21.15.193:6789/0 conn(0x7fa9f0161f30 legacy=0x7fa9f00c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:13.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.835+0000 7fa9ef7fe640 1 -- 172.21.15.43:0/3055314735 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f00d2740 con 0x7fa9f015df50 2024-04-25T09:02:13.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.835+0000 7fa9ef7fe640 1 --2- 172.21.15.43:0/3055314735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9f015df50 0x7fa9f01619f0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fa9f00caa00 tx=0x7fa9e002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab5dcac907e43612 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:13.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.836+0000 7fa9ee7fc640 1 -- 172.21.15.43:0/3055314735 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0037070 con 0x7fa9f015df50 2024-04-25T09:02:13.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.836+0000 7fa9ee7fc640 1 -- 172.21.15.43:0/3055314735 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e0032440 con 0x7fa9f015df50 2024-04-25T09:02:13.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.836+0000 7fa9ee7fc640 1 -- 172.21.15.43:0/3055314735 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0032810 con 0x7fa9f015df50 2024-04-25T09:02:13.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.836+0000 7fa9f5ff5640 1 -- 172.21.15.43:0/3055314735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9f015df50 msgr2=0x7fa9f01619f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:13.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.836+0000 7fa9f5ff5640 1 --2- 172.21.15.43:0/3055314735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9f015df50 0x7fa9f01619f0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fa9f00caa00 tx=0x7fa9e002a270 comp rx=0 tx=0).stop 2024-04-25T09:02:13.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.836+0000 7fa9f5ff5640 1 -- 172.21.15.43:0/3055314735 shutdown_connections 2024-04-25T09:02:13.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.836+0000 7fa9f5ff5640 1 --2- 172.21.15.43:0/3055314735 >> v2:172.21.15.193:3300/0 conn(0x7fa9f00cf670 0x7fa9f0162380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.836+0000 7fa9f5ff5640 1 --2- 172.21.15.43:0/3055314735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9f015df50 0x7fa9f01619f0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.836+0000 7fa9f5ff5640 1 -- 172.21.15.43:0/3055314735 >> 172.21.15.43:0/3055314735 conn(0x7fa9f00d3a30 msgr2=0x7fa9f00d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:13.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.836+0000 7fa9f5ff5640 1 -- 172.21.15.43:0/3055314735 shutdown_connections 2024-04-25T09:02:13.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.836+0000 7fa9f5ff5640 1 -- 172.21.15.43:0/3055314735 wait complete. 2024-04-25T09:02:13.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.837+0000 7fa9f5ff5640 1 Processor -- start 2024-04-25T09:02:13.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.838+0000 7fa9f5ff5640 1 -- start start 2024-04-25T09:02:13.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.838+0000 7fa9f5ff5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9f00cf670 0x7fa9f014ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.838+0000 7fa9f5ff5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa9f015df50 0x7fa9f015ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.838+0000 7fa9f5ff5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9f0161f30 0x7fa9f00cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.838+0000 7fa9f5ff5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f00c77d0 con 0x7fa9f0161f30 2024-04-25T09:02:13.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.838+0000 7fa9f5ff5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f00c74f0 con 0x7fa9f00cf670 2024-04-25T09:02:13.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.838+0000 7fa9f5ff5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f00d2190 con 0x7fa9f015df50 2024-04-25T09:02:13.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.838+0000 7fa9effff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9f0161f30 0x7fa9f00cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.838+0000 7fa9effff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9f0161f30 0x7fa9f00cbb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51270/0 (socket says 172.21.15.43:51270) 2024-04-25T09:02:13.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.838+0000 7fa9effff640 1 -- 172.21.15.43:0/1275692194 learned_addr learned my addr 172.21.15.43:0/1275692194 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:13.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.839+0000 7fa9ef7fe640 1 --2- 172.21.15.43:0/1275692194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9f00cf670 0x7fa9f014ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.839+0000 7fa9eeffd640 1 --2- 172.21.15.43:0/1275692194 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa9f015df50 0x7fa9f015ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.839+0000 7fa9effff640 1 -- 172.21.15.43:0/1275692194 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa9f015df50 msgr2=0x7fa9f015ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:13.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.839+0000 7fa9effff640 1 --2- 172.21.15.43:0/1275692194 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa9f015df50 0x7fa9f015ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.839+0000 7fa9effff640 1 -- 172.21.15.43:0/1275692194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9f00cf670 msgr2=0x7fa9f014ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:13.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.839+0000 7fa9effff640 1 --2- 172.21.15.43:0/1275692194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9f00cf670 0x7fa9f014ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:13.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.839+0000 7fa9effff640 1 -- 172.21.15.43:0/1275692194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f015aad0 con 0x7fa9f0161f30 2024-04-25T09:02:13.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.839+0000 7fa9eeffd640 1 --2- 172.21.15.43:0/1275692194 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa9f015df50 0x7fa9f015ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:13.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.839+0000 7fa9effff640 1 --2- 172.21.15.43:0/1275692194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9f0161f30 0x7fa9f00cbb40 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8007e40 tx=0x7fa9e800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:13.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.839+0000 7fa9d7fff640 1 -- 172.21.15.43:0/1275692194 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8014620 con 0x7fa9f0161f30 2024-04-25T09:02:13.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.839+0000 7fa9d7fff640 1 -- 172.21.15.43:0/1275692194 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e8019070 con 0x7fa9f0161f30 2024-04-25T09:02:13.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.839+0000 7fa9f5ff5640 1 -- 172.21.15.43:0/1275692194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f015e990 con 0x7fa9f0161f30 2024-04-25T09:02:13.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.839+0000 7fa9d7fff640 1 -- 172.21.15.43:0/1275692194 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8014a20 con 0x7fa9f0161f30 2024-04-25T09:02:13.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.839+0000 7fa9f5ff5640 1 -- 172.21.15.43:0/1275692194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f015a7f0 con 0x7fa9f0161f30 2024-04-25T09:02:13.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.840+0000 7fa9f5ff5640 1 -- 172.21.15.43:0/1275692194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9f00680f0 con 0x7fa9f0161f30 2024-04-25T09:02:13.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.841+0000 7fa9d7fff640 1 -- 172.21.15.43:0/1275692194 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa9e8014bc0 con 0x7fa9f0161f30 2024-04-25T09:02:13.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.841+0000 7fa9d7fff640 1 --2- 172.21.15.43:0/1275692194 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa9dc06b4d0 0x7fa9dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:13.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.841+0000 7fa9d7fff640 1 -- 172.21.15.43:0/1275692194 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa9e808ab20 con 0x7fa9f0161f30 2024-04-25T09:02:13.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.841+0000 7fa9ef7fe640 1 --2- 172.21.15.43:0/1275692194 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa9dc06b4d0 0x7fa9dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:13.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.842+0000 7fa9ef7fe640 1 --2- 172.21.15.43:0/1275692194 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa9dc06b4d0 0x7fa9dc06d990 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0034000 tx=0x7fa9e002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:13.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:13.845+0000 7fa9d7fff640 1 -- 172.21.15.43:0/1275692194 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa9e80599a0 con 0x7fa9f0161f30 2024-04-25T09:02:14.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.068+0000 7fa9f5ff5640 1 -- 172.21.15.43:0/1275692194 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa9f0074ef0 con 0x7fa9f0161f30 2024-04-25T09:02:14.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.069+0000 7fa9d7fff640 1 -- 172.21.15.43:0/1275692194 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4940 (secure 0 0 0) 0x7fa9e80592a0 con 0x7fa9f0161f30 2024-04-25T09:02:14.057 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:14.058 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":61,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2150177203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2150177203},{"type":"v1","addr":"172.21.15.193:6837","nonce":2150177203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":8079,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/621488642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":621488642},{"type":"v1","addr":"172.21.15.43:6834","nonce":621488642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8082,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1524395840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1524395840},{"type":"v1","addr":"172.21.15.43:6835","nonce":1524395840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:01:33.726790+0000","modified":"2024-04-25T09:01:35.706689+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8094},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8094":{"gid":8094,"name":"b","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.193:6836/853943267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":853943267},{"type":"v1","addr":"172.21.15.193:6836","nonce":853943267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:02:14.058 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 61 2024-04-25T09:02:14.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.071+0000 7fa9f5ff5640 1 -- 172.21.15.43:0/1275692194 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa9dc06b4d0 msgr2=0x7fa9dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:14.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.071+0000 7fa9f5ff5640 1 --2- 172.21.15.43:0/1275692194 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa9dc06b4d0 0x7fa9dc06d990 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0034000 tx=0x7fa9e002e000 comp rx=0 tx=0).stop 2024-04-25T09:02:14.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.071+0000 7fa9f5ff5640 1 -- 172.21.15.43:0/1275692194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9f0161f30 msgr2=0x7fa9f00cbb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:14.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.071+0000 7fa9f5ff5640 1 --2- 172.21.15.43:0/1275692194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9f0161f30 0x7fa9f00cbb40 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8007e40 tx=0x7fa9e800e4b0 comp rx=0 tx=0).stop 2024-04-25T09:02:14.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.071+0000 7fa9f5ff5640 1 -- 172.21.15.43:0/1275692194 shutdown_connections 2024-04-25T09:02:14.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.071+0000 7fa9f5ff5640 1 --2- 172.21.15.43:0/1275692194 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa9f0161f30 0x7fa9f00cbb40 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:14.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.071+0000 7fa9f5ff5640 1 --2- 172.21.15.43:0/1275692194 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa9f015df50 0x7fa9f015ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:14.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.071+0000 7fa9f5ff5640 1 --2- 172.21.15.43:0/1275692194 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa9dc06b4d0 0x7fa9dc06d990 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:14.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.071+0000 7fa9f5ff5640 1 --2- 172.21.15.43:0/1275692194 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa9f00cf670 0x7fa9f014ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:14.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.071+0000 7fa9f5ff5640 1 -- 172.21.15.43:0/1275692194 >> 172.21.15.43:0/1275692194 conn(0x7fa9f00d3a30 msgr2=0x7fa9f0153e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:14.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.071+0000 7fa9f5ff5640 1 -- 172.21.15.43:0/1275692194 shutdown_connections 2024-04-25T09:02:14.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.071+0000 7fa9f5ff5640 1 -- 172.21.15.43:0/1275692194 wait complete. 2024-04-25T09:02:14.071 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:02:14.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.182+0000 7fc3e8ef7640 1 Processor -- start 2024-04-25T09:02:14.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.183+0000 7fc3e8ef7640 1 -- start start 2024-04-25T09:02:14.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.183+0000 7fc3e8ef7640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fc3e415a9d0 0x7fc3e406d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:14.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.183+0000 7fc3e8ef7640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc3e406d7d0 0x7fc3e4071a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:14.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.183+0000 7fc3e8ef7640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3e4072420 con 0x7fc3e4071f90 2024-04-25T09:02:14.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.183+0000 7fc3e8ef7640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3e406dc30 con 0x7fc3e415a9d0 2024-04-25T09:02:14.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.183+0000 7fc3e8ef7640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e40704f0 con 0x7fc3e406d7d0 2024-04-25T09:02:14.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.184+0000 7fc3e2d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc3e4071f90 0x7fc3e40b3990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32886/0 (socket says 172.21.15.43:32886) 2024-04-25T09:02:14.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.184+0000 7fc3e2d9d640 1 -- 172.21.15.43:0/1153513215 learned_addr learned my addr 172.21.15.43:0/1153513215 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:14.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.184+0000 7fc3e259c640 1 --2- 172.21.15.43:0/1153513215 >> v2:172.21.15.193:3300/0 conn(0x7fc3e415a9d0 0x7fc3e406d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:14.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.184+0000 7fc3e1d9b640 1 --2- 172.21.15.43:0/1153513215 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc3e406d7d0 0x7fc3e4071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:14.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.184+0000 7fc3e159a640 1 -- 172.21.15.43:0/1153513215 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 561494993 0 0) 0x7fc3d4001490 con 0x7fc3e4071f90 2024-04-25T09:02:14.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.184+0000 7fc3e159a640 1 -- 172.21.15.43:0/1153513215 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3cc003520 con 0x7fc3e4071f90 2024-04-25T09:02:14.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.184+0000 7fc3e259c640 1 -- 172.21.15.43:0/1153513215 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc3e406d7d0 msgr2=0x7fc3e4071a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:14.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.184+0000 7fc3e259c640 1 --2- 172.21.15.43:0/1153513215 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc3e406d7d0 0x7fc3e4071a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:14.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.184+0000 7fc3e259c640 1 -- 172.21.15.43:0/1153513215 >> v1:172.21.15.43:6789/0 conn(0x7fc3e4071f90 legacy=0x7fc3e40b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:14.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.184+0000 7fc3e259c640 1 -- 172.21.15.43:0/1153513215 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e40707d0 con 0x7fc3e415a9d0 2024-04-25T09:02:14.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.184+0000 7fc3e1d9b640 1 --2- 172.21.15.43:0/1153513215 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc3e406d7d0 0x7fc3e4071a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:02:14.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.185+0000 7fc3e259c640 1 --2- 172.21.15.43:0/1153513215 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3e415a9d0 0x7fc3e406d290 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc0097a0 tx=0x7fc3dc02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2e4f1debdf57a18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:14.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.185+0000 7fc3e159a640 1 -- 172.21.15.43:0/1153513215 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc037070 con 0x7fc3e415a9d0 2024-04-25T09:02:14.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.185+0000 7fc3e159a640 1 -- 172.21.15.43:0/1153513215 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3dc032440 con 0x7fc3e415a9d0 2024-04-25T09:02:14.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.185+0000 7fc3e159a640 1 -- 172.21.15.43:0/1153513215 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc032810 con 0x7fc3e415a9d0 2024-04-25T09:02:14.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.186+0000 7fc3e8ef7640 1 -- 172.21.15.43:0/1153513215 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3e415a9d0 msgr2=0x7fc3e406d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:14.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.186+0000 7fc3e8ef7640 1 --2- 172.21.15.43:0/1153513215 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3e415a9d0 0x7fc3e406d290 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc0097a0 tx=0x7fc3dc02a180 comp rx=0 tx=0).stop 2024-04-25T09:02:14.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.186+0000 7fc3e8ef7640 1 -- 172.21.15.43:0/1153513215 shutdown_connections 2024-04-25T09:02:14.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.186+0000 7fc3e8ef7640 1 --2- 172.21.15.43:0/1153513215 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc3e406d7d0 0x7fc3e4071a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:14.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.186+0000 7fc3e8ef7640 1 --2- 172.21.15.43:0/1153513215 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3e415a9d0 0x7fc3e406d290 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:14.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.186+0000 7fc3e8ef7640 1 -- 172.21.15.43:0/1153513215 >> 172.21.15.43:0/1153513215 conn(0x7fc3e40b1480 msgr2=0x7fc3e40b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:14.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.186+0000 7fc3e8ef7640 1 -- 172.21.15.43:0/1153513215 shutdown_connections 2024-04-25T09:02:14.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.186+0000 7fc3e8ef7640 1 -- 172.21.15.43:0/1153513215 wait complete. 2024-04-25T09:02:14.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.187+0000 7fc3e8ef7640 1 Processor -- start 2024-04-25T09:02:14.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.187+0000 7fc3e8ef7640 1 -- start start 2024-04-25T09:02:14.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.188+0000 7fc3e8ef7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3e406d7d0 0x7fc3e41529b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:14.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.188+0000 7fc3e8ef7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3e4071f90 0x7fc3e40b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:14.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.188+0000 7fc3e8ef7640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc3e4151a40 0x7fc3e4156880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:14.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.188+0000 7fc3e8ef7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e40712b0 con 0x7fc3e406d7d0 2024-04-25T09:02:14.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.188+0000 7fc3e8ef7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e4070d90 con 0x7fc3e4071f90 2024-04-25T09:02:14.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.188+0000 7fc3e8ef7640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e40ce950 con 0x7fc3e4151a40 2024-04-25T09:02:14.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.188+0000 7fc3e259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3e406d7d0 0x7fc3e41529b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:14.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.188+0000 7fc3e259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3e406d7d0 0x7fc3e41529b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51272/0 (socket says 172.21.15.43:51272) 2024-04-25T09:02:14.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.188+0000 7fc3e259c640 1 -- 172.21.15.43:0/2024372074 learned_addr learned my addr 172.21.15.43:0/2024372074 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:14.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.188+0000 7fc3e2d9d640 1 --2- 172.21.15.43:0/2024372074 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc3e4151a40 0x7fc3e4156880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:14.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.188+0000 7fc3e1d9b640 1 --2- 172.21.15.43:0/2024372074 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3e4071f90 0x7fc3e40b2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:14.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.188+0000 7fc3e259c640 1 -- 172.21.15.43:0/2024372074 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc3e4151a40 msgr2=0x7fc3e4156880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:14.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.188+0000 7fc3e259c640 1 --2- 172.21.15.43:0/2024372074 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc3e4151a40 0x7fc3e4156880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:14.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.188+0000 7fc3e259c640 1 -- 172.21.15.43:0/2024372074 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3e4071f90 msgr2=0x7fc3e40b2ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:14.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.188+0000 7fc3e259c640 1 --2- 172.21.15.43:0/2024372074 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3e4071f90 0x7fc3e40b2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:14.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.188+0000 7fc3e259c640 1 -- 172.21.15.43:0/2024372074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e4155b00 con 0x7fc3e406d7d0 2024-04-25T09:02:14.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.189+0000 7fc3e259c640 1 --2- 172.21.15.43:0/2024372074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3e406d7d0 0x7fc3e41529b0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc0097a0 tx=0x7fc3dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:14.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.189+0000 7fc3d2ffd640 1 -- 172.21.15.43:0/2024372074 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc041070 con 0x7fc3e406d7d0 2024-04-25T09:02:14.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.189+0000 7fc3d2ffd640 1 -- 172.21.15.43:0/2024372074 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3dc03c870 con 0x7fc3e406d7d0 2024-04-25T09:02:14.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.189+0000 7fc3e8ef7640 1 -- 172.21.15.43:0/2024372074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3e41524c0 con 0x7fc3e406d7d0 2024-04-25T09:02:14.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.189+0000 7fc3d2ffd640 1 -- 172.21.15.43:0/2024372074 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc037040 con 0x7fc3e406d7d0 2024-04-25T09:02:14.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.189+0000 7fc3e8ef7640 1 -- 172.21.15.43:0/2024372074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3e4156080 con 0x7fc3e406d7d0 2024-04-25T09:02:14.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.190+0000 7fc3e8ef7640 1 -- 172.21.15.43:0/2024372074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3e4074ef0 con 0x7fc3e406d7d0 2024-04-25T09:02:14.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.190+0000 7fc3d2ffd640 1 -- 172.21.15.43:0/2024372074 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc3dc044070 con 0x7fc3e406d7d0 2024-04-25T09:02:14.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.191+0000 7fc3d2ffd640 1 --2- 172.21.15.43:0/2024372074 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc3c406b4d0 0x7fc3c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:14.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.191+0000 7fc3d2ffd640 1 -- 172.21.15.43:0/2024372074 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc3dc0aab40 con 0x7fc3e406d7d0 2024-04-25T09:02:14.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.191+0000 7fc3e1d9b640 1 --2- 172.21.15.43:0/2024372074 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc3c406b4d0 0x7fc3c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:14.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.191+0000 7fc3e1d9b640 1 --2- 172.21.15.43:0/2024372074 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc3c406b4d0 0x7fc3c406d990 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4069170 tx=0x7fc3d8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:14.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.195+0000 7fc3d2ffd640 1 -- 172.21.15.43:0/2024372074 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc3dc079a50 con 0x7fc3e406d7d0 2024-04-25T09:02:14.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.381+0000 7fc3e8ef7640 1 -- 172.21.15.43:0/2024372074 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3e4159100 con 0x7fc3e406d7d0 2024-04-25T09:02:14.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.383+0000 7fc3d2ffd640 1 -- 172.21.15.43:0/2024372074 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+16583 (secure 0 0 0) 0x7fc3dc079350 con 0x7fc3e406d7d0 2024-04-25T09:02:14.371 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:14.372 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":102,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:01:35.356828+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:01:31.997813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T09:01:32.695222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:02:14.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.385+0000 7fc3e8ef7640 1 -- 172.21.15.43:0/2024372074 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc3c406b4d0 msgr2=0x7fc3c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:14.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.385+0000 7fc3e8ef7640 1 --2- 172.21.15.43:0/2024372074 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc3c406b4d0 0x7fc3c406d990 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4069170 tx=0x7fc3d8008040 comp rx=0 tx=0).stop 2024-04-25T09:02:14.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.385+0000 7fc3e8ef7640 1 -- 172.21.15.43:0/2024372074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3e406d7d0 msgr2=0x7fc3e41529b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:14.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.385+0000 7fc3e8ef7640 1 --2- 172.21.15.43:0/2024372074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3e406d7d0 0x7fc3e41529b0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc0097a0 tx=0x7fc3dc0046c0 comp rx=0 tx=0).stop 2024-04-25T09:02:14.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.385+0000 7fc3e8ef7640 1 -- 172.21.15.43:0/2024372074 shutdown_connections 2024-04-25T09:02:14.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.385+0000 7fc3e8ef7640 1 --2- 172.21.15.43:0/2024372074 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc3e4151a40 0x7fc3e4156880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:14.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.385+0000 7fc3e8ef7640 1 --2- 172.21.15.43:0/2024372074 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc3c406b4d0 0x7fc3c406d990 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:14.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.385+0000 7fc3e8ef7640 1 --2- 172.21.15.43:0/2024372074 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc3e4071f90 0x7fc3e40b2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:14.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.385+0000 7fc3e8ef7640 1 --2- 172.21.15.43:0/2024372074 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3e406d7d0 0x7fc3e41529b0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:14.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.385+0000 7fc3e8ef7640 1 -- 172.21.15.43:0/2024372074 >> 172.21.15.43:0/2024372074 conn(0x7fc3e40b1480 msgr2=0x7fc3e4157e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:14.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.385+0000 7fc3e8ef7640 1 -- 172.21.15.43:0/2024372074 shutdown_connections 2024-04-25T09:02:14.375 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.385+0000 7fc3e8ef7640 1 -- 172.21.15.43:0/2024372074 wait complete. 2024-04-25T09:02:14.396 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:02:14.396 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:02:14.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.509+0000 7fb74529e640 1 Processor -- start 2024-04-25T09:02:14.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.510+0000 7fb74529e640 1 -- start start 2024-04-25T09:02:14.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.510+0000 7fb74529e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb7400cbfc0 0x7fb74006dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:14.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.510+0000 7fb74529e640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb7400cd010 0x7fb74015fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:14.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.510+0000 7fb74529e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7400b2780 con 0x7fb7400ccbb0 2024-04-25T09:02:14.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.510+0000 7fb74529e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7400b2a60 con 0x7fb7400cbfc0 2024-04-25T09:02:14.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.510+0000 7fb74529e640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7400b2d40 con 0x7fb7400cd010 2024-04-25T09:02:14.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.510+0000 7fb73effd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb7400cbfc0 0x7fb74006dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:14.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.510+0000 7fb73e7fc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb7400ccbb0 0x7fb74015f5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32896/0 (socket says 172.21.15.43:32896) 2024-04-25T09:02:14.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.511+0000 7fb73e7fc640 1 -- 172.21.15.43:0/3834022255 learned_addr learned my addr 172.21.15.43:0/3834022255 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:14.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.511+0000 7fb73f7fe640 1 --2- 172.21.15.43:0/3834022255 >> v2:172.21.15.193:3300/0 conn(0x7fb7400cd010 0x7fb74015fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:14.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.511+0000 7fb73effd640 1 -- 172.21.15.43:0/3834022255 >> v2:172.21.15.193:3300/0 conn(0x7fb7400cd010 msgr2=0x7fb74015fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:14.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.511+0000 7fb73effd640 1 --2- 172.21.15.43:0/3834022255 >> v2:172.21.15.193:3300/0 conn(0x7fb7400cd010 0x7fb74015fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:14.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.511+0000 7fb73effd640 1 -- 172.21.15.43:0/3834022255 >> v1:172.21.15.43:6789/0 conn(0x7fb7400ccbb0 legacy=0x7fb74015f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:14.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.511+0000 7fb73dffb640 1 -- 172.21.15.43:0/3834022255 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1328010616 0 0) 0x7fb7400b2780 con 0x7fb7400ccbb0 2024-04-25T09:02:14.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.511+0000 7fb73effd640 1 -- 172.21.15.43:0/3834022255 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7400b3020 con 0x7fb7400cbfc0 2024-04-25T09:02:14.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.512+0000 7fb73effd640 1 --2- 172.21.15.43:0/3834022255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7400cbfc0 0x7fb74006dc60 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fb74006e4d0 tx=0x7fb73802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb761627fd7ca339 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:14.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.512+0000 7fb73dffb640 1 -- 172.21.15.43:0/3834022255 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb738037070 con 0x7fb7400cbfc0 2024-04-25T09:02:14.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.512+0000 7fb73dffb640 1 -- 172.21.15.43:0/3834022255 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb738032440 con 0x7fb7400cbfc0 2024-04-25T09:02:14.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.512+0000 7fb73dffb640 1 -- 172.21.15.43:0/3834022255 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb738032810 con 0x7fb7400cbfc0 2024-04-25T09:02:14.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.512+0000 7fb74529e640 1 -- 172.21.15.43:0/3834022255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7400cbfc0 msgr2=0x7fb74006dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:14.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.512+0000 7fb74529e640 1 --2- 172.21.15.43:0/3834022255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7400cbfc0 0x7fb74006dc60 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fb74006e4d0 tx=0x7fb73802a1a0 comp rx=0 tx=0).stop 2024-04-25T09:02:14.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.512+0000 7fb74529e640 1 -- 172.21.15.43:0/3834022255 shutdown_connections 2024-04-25T09:02:14.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.512+0000 7fb74529e640 1 --2- 172.21.15.43:0/3834022255 >> v2:172.21.15.193:3300/0 conn(0x7fb7400cd010 0x7fb74015fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:14.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.512+0000 7fb74529e640 1 --2- 172.21.15.43:0/3834022255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7400cbfc0 0x7fb74006dc60 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:14.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.512+0000 7fb74529e640 1 -- 172.21.15.43:0/3834022255 >> 172.21.15.43:0/3834022255 conn(0x7fb7400b1480 msgr2=0x7fb7400b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:14.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.512+0000 7fb74529e640 1 -- 172.21.15.43:0/3834022255 shutdown_connections 2024-04-25T09:02:14.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.512+0000 7fb74529e640 1 -- 172.21.15.43:0/3834022255 wait complete. 2024-04-25T09:02:14.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.514+0000 7fb74529e640 1 Processor -- start 2024-04-25T09:02:14.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.514+0000 7fb74529e640 1 -- start start 2024-04-25T09:02:14.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.514+0000 7fb74529e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7400cbfc0 0x7fb7401f5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:14.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.514+0000 7fb74529e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7400ccbb0 0x7fb7401401e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:14.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.514+0000 7fb74529e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb7400cd010 0x7fb7401391e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:14.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.514+0000 7fb74529e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7400cd480 con 0x7fb7400cbfc0 2024-04-25T09:02:14.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.514+0000 7fb74529e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7400b35e0 con 0x7fb7400ccbb0 2024-04-25T09:02:14.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.514+0000 7fb74529e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb74006e1a0 con 0x7fb7400cd010 2024-04-25T09:02:14.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.514+0000 7fb73effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7400cbfc0 0x7fb7401f5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:14.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.515+0000 7fb73effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7400cbfc0 0x7fb7401f5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51298/0 (socket says 172.21.15.43:51298) 2024-04-25T09:02:14.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.515+0000 7fb73effd640 1 -- 172.21.15.43:0/2105702851 learned_addr learned my addr 172.21.15.43:0/2105702851 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:14.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.515+0000 7fb73effd640 1 -- 172.21.15.43:0/2105702851 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb7400cd010 msgr2=0x7fb7401391e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:14.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.515+0000 7fb73e7fc640 1 --2- 172.21.15.43:0/2105702851 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7400ccbb0 0x7fb7401401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:14.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.515+0000 7fb73effd640 1 --2- 172.21.15.43:0/2105702851 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb7400cd010 0x7fb7401391e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:14.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.515+0000 7fb73effd640 1 -- 172.21.15.43:0/2105702851 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7400ccbb0 msgr2=0x7fb7401401e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:14.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.515+0000 7fb73effd640 1 --2- 172.21.15.43:0/2105702851 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7400ccbb0 0x7fb7401401e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:14.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.515+0000 7fb73effd640 1 -- 172.21.15.43:0/2105702851 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb740140910 con 0x7fb7400cbfc0 2024-04-25T09:02:14.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.515+0000 7fb73effd640 1 --2- 172.21.15.43:0/2105702851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7400cbfc0 0x7fb7401f5a70 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fb738000c00 tx=0x7fb7380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:14.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.515+0000 7fb7377fe640 1 -- 172.21.15.43:0/2105702851 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb738037050 con 0x7fb7400cbfc0 2024-04-25T09:02:14.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.515+0000 7fb7377fe640 1 -- 172.21.15.43:0/2105702851 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb738038040 con 0x7fb7400cbfc0 2024-04-25T09:02:14.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.515+0000 7fb7377fe640 1 -- 172.21.15.43:0/2105702851 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb738004470 con 0x7fb7400cbfc0 2024-04-25T09:02:14.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.515+0000 7fb74529e640 1 -- 172.21.15.43:0/2105702851 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb740140bf0 con 0x7fb7400cbfc0 2024-04-25T09:02:14.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.515+0000 7fb74529e640 1 -- 172.21.15.43:0/2105702851 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7401411b0 con 0x7fb7400cbfc0 2024-04-25T09:02:14.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.516+0000 7fb74529e640 1 -- 172.21.15.43:0/2105702851 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb740074ef0 con 0x7fb7400cbfc0 2024-04-25T09:02:14.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.517+0000 7fb7377fe640 1 -- 172.21.15.43:0/2105702851 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb738030900 con 0x7fb7400cbfc0 2024-04-25T09:02:14.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.517+0000 7fb7377fe640 1 --2- 172.21.15.43:0/2105702851 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb72006b4d0 0x7fb72006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:14.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.517+0000 7fb7377fe640 1 -- 172.21.15.43:0/2105702851 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb7380aad30 con 0x7fb7400cbfc0 2024-04-25T09:02:14.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.517+0000 7fb73e7fc640 1 --2- 172.21.15.43:0/2105702851 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb72006b4d0 0x7fb72006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:14.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.518+0000 7fb73e7fc640 1 --2- 172.21.15.43:0/2105702851 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb72006b4d0 0x7fb72006d990 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fb728007fd0 tx=0x7fb728009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:14.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.521+0000 7fb7377fe640 1 -- 172.21.15.43:0/2105702851 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb738079cd0 con 0x7fb7400cbfc0 2024-04-25T09:02:14.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.737+0000 7fb74529e640 1 -- 172.21.15.43:0/2105702851 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7400af0d0 con 0x7fb7400cbfc0 2024-04-25T09:02:14.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.737+0000 7fb7377fe640 1 -- 172.21.15.43:0/2105702851 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4940 (secure 0 0 0) 0x7fb7380795d0 con 0x7fb7400cbfc0 2024-04-25T09:02:14.726 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:14.726 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":61,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2150177203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2150177203},{"type":"v1","addr":"172.21.15.193:6837","nonce":2150177203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":8079,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/621488642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":621488642},{"type":"v1","addr":"172.21.15.43:6834","nonce":621488642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8082,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1524395840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1524395840},{"type":"v1","addr":"172.21.15.43:6835","nonce":1524395840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:01:33.726790+0000","modified":"2024-04-25T09:01:35.706689+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8094},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8094":{"gid":8094,"name":"b","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.193:6836/853943267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":853943267},{"type":"v1","addr":"172.21.15.193:6836","nonce":853943267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:02:14.727 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 61 2024-04-25T09:02:14.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.739+0000 7fb74529e640 1 -- 172.21.15.43:0/2105702851 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb72006b4d0 msgr2=0x7fb72006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:14.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.739+0000 7fb74529e640 1 --2- 172.21.15.43:0/2105702851 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb72006b4d0 0x7fb72006d990 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fb728007fd0 tx=0x7fb728009040 comp rx=0 tx=0).stop 2024-04-25T09:02:14.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.739+0000 7fb74529e640 1 -- 172.21.15.43:0/2105702851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7400cbfc0 msgr2=0x7fb7401f5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:14.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.739+0000 7fb74529e640 1 --2- 172.21.15.43:0/2105702851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7400cbfc0 0x7fb7401f5a70 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fb738000c00 tx=0x7fb7380046c0 comp rx=0 tx=0).stop 2024-04-25T09:02:14.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.739+0000 7fb74529e640 1 -- 172.21.15.43:0/2105702851 shutdown_connections 2024-04-25T09:02:14.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.739+0000 7fb74529e640 1 --2- 172.21.15.43:0/2105702851 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb7400cd010 0x7fb7401391e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:14.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.739+0000 7fb74529e640 1 --2- 172.21.15.43:0/2105702851 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb72006b4d0 0x7fb72006d990 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:14.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.739+0000 7fb74529e640 1 --2- 172.21.15.43:0/2105702851 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7400ccbb0 0x7fb7401401e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:14.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.739+0000 7fb74529e640 1 --2- 172.21.15.43:0/2105702851 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7400cbfc0 0x7fb7401f5a70 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:14.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.739+0000 7fb74529e640 1 -- 172.21.15.43:0/2105702851 >> 172.21.15.43:0/2105702851 conn(0x7fb7400b1480 msgr2=0x7fb7400af2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:14.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.739+0000 7fb74529e640 1 -- 172.21.15.43:0/2105702851 shutdown_connections 2024-04-25T09:02:14.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.739+0000 7fb74529e640 1 -- 172.21.15.43:0/2105702851 wait complete. 2024-04-25T09:02:14.740 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:02:14.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.852+0000 7f45865ee640 1 Processor -- start 2024-04-25T09:02:14.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.852+0000 7f45865ee640 1 -- start start 2024-04-25T09:02:14.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.852+0000 7f45865ee640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45800cbfc0 0x7f458006dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:14.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.852+0000 7f45865ee640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f45800cd010 0x7f458015fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:14.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.852+0000 7f45865ee640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45800b2780 con 0x7f45800ccbb0 2024-04-25T09:02:14.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.852+0000 7f45865ee640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45800b2a60 con 0x7f45800cd010 2024-04-25T09:02:14.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.852+0000 7f45865ee640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45800b2d40 con 0x7f45800cbfc0 2024-04-25T09:02:14.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.853+0000 7f4584b8b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f45800cd010 0x7f458015fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:14.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.853+0000 7f457f7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f45800ccbb0 0x7f458015f5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32906/0 (socket says 172.21.15.43:32906) 2024-04-25T09:02:14.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.853+0000 7f457f7fe640 1 -- 172.21.15.43:0/1889345637 learned_addr learned my addr 172.21.15.43:0/1889345637 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:14.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.853+0000 7f457ffff640 1 --2- 172.21.15.43:0/1889345637 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45800cbfc0 0x7f458006dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:14.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.853+0000 7f4584b8b640 1 -- 172.21.15.43:0/1889345637 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45800cbfc0 msgr2=0x7f458006dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:14.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.853+0000 7f4584b8b640 1 --2- 172.21.15.43:0/1889345637 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45800cbfc0 0x7f458006dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:14.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.853+0000 7f4584b8b640 1 -- 172.21.15.43:0/1889345637 >> v1:172.21.15.43:6789/0 conn(0x7f45800ccbb0 legacy=0x7f458015f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:14.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.853+0000 7f4584b8b640 1 -- 172.21.15.43:0/1889345637 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45800b3020 con 0x7f45800cd010 2024-04-25T09:02:14.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.853+0000 7f457effd640 1 -- 172.21.15.43:0/1889345637 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 392920144 0 0) 0x7f45800b2780 con 0x7f45800ccbb0 2024-04-25T09:02:14.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.854+0000 7f4584b8b640 1 --2- 172.21.15.43:0/1889345637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45800cd010 0x7f458015fcf0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f4578009d90 tx=0x7f457802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a5109915352f7ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:14.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.854+0000 7f457effd640 1 -- 172.21.15.43:0/1889345637 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4578007740 con 0x7f45800cd010 2024-04-25T09:02:14.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.854+0000 7f457effd640 1 -- 172.21.15.43:0/1889345637 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45780078e0 con 0x7f45800cd010 2024-04-25T09:02:14.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.854+0000 7f457effd640 1 -- 172.21.15.43:0/1889345637 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4578007cb0 con 0x7f45800cd010 2024-04-25T09:02:14.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.854+0000 7f45865ee640 1 -- 172.21.15.43:0/1889345637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45800cd010 msgr2=0x7f458015fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:14.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.854+0000 7f45865ee640 1 --2- 172.21.15.43:0/1889345637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45800cd010 0x7f458015fcf0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f4578009d90 tx=0x7f457802a660 comp rx=0 tx=0).stop 2024-04-25T09:02:14.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.854+0000 7f45865ee640 1 -- 172.21.15.43:0/1889345637 shutdown_connections 2024-04-25T09:02:14.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.854+0000 7f45865ee640 1 --2- 172.21.15.43:0/1889345637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45800cd010 0x7f458015fcf0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:14.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.854+0000 7f45865ee640 1 --2- 172.21.15.43:0/1889345637 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45800cbfc0 0x7f458006dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:14.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.854+0000 7f45865ee640 1 -- 172.21.15.43:0/1889345637 >> 172.21.15.43:0/1889345637 conn(0x7f45800b1480 msgr2=0x7f45800b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:14.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.854+0000 7f45865ee640 1 -- 172.21.15.43:0/1889345637 shutdown_connections 2024-04-25T09:02:14.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.854+0000 7f45865ee640 1 -- 172.21.15.43:0/1889345637 wait complete. 2024-04-25T09:02:14.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.855+0000 7f45865ee640 1 Processor -- start 2024-04-25T09:02:14.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.856+0000 7f45865ee640 1 -- start start 2024-04-25T09:02:14.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.856+0000 7f45865ee640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45800cbfc0 0x7f458013f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:14.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.856+0000 7f45865ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45800ccbb0 0x7f458015b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:14.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.856+0000 7f45865ee640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45800cd010 0x7f45800ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:14.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.856+0000 7f45865ee640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45800cd4a0 con 0x7f45800ccbb0 2024-04-25T09:02:14.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.856+0000 7f45865ee640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45800b35e0 con 0x7f45800cbfc0 2024-04-25T09:02:14.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.856+0000 7f45865ee640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f458006e1a0 con 0x7f45800cd010 2024-04-25T09:02:14.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.857+0000 7f457f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45800ccbb0 0x7f458015b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:14.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.857+0000 7f457f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45800ccbb0 0x7f458015b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51306/0 (socket says 172.21.15.43:51306) 2024-04-25T09:02:14.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.857+0000 7f457f7fe640 1 -- 172.21.15.43:0/1108851740 learned_addr learned my addr 172.21.15.43:0/1108851740 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:14.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.857+0000 7f457ffff640 1 --2- 172.21.15.43:0/1108851740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45800cbfc0 0x7f458013f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:14.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.857+0000 7f4584b8b640 1 --2- 172.21.15.43:0/1108851740 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45800cd010 0x7f45800ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:14.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.857+0000 7f457f7fe640 1 -- 172.21.15.43:0/1108851740 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45800cd010 msgr2=0x7f45800ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:14.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.857+0000 7f457f7fe640 1 --2- 172.21.15.43:0/1108851740 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45800cd010 0x7f45800ad8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:14.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.857+0000 7f457f7fe640 1 -- 172.21.15.43:0/1108851740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45800cbfc0 msgr2=0x7f458013f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:14.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.857+0000 7f457f7fe640 1 --2- 172.21.15.43:0/1108851740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45800cbfc0 0x7f458013f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:14.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.857+0000 7f457f7fe640 1 -- 172.21.15.43:0/1108851740 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f458013f880 con 0x7f45800ccbb0 2024-04-25T09:02:14.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.857+0000 7f457ffff640 1 --2- 172.21.15.43:0/1108851740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45800cbfc0 0x7f458013f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:14.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.857+0000 7f4584b8b640 1 --2- 172.21.15.43:0/1108851740 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45800cd010 0x7f45800ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:14.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.857+0000 7f457f7fe640 1 --2- 172.21.15.43:0/1108851740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45800ccbb0 0x7f458015b3c0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f457400c4d0 tx=0x7f457400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:14.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.857+0000 7f457cff9640 1 -- 172.21.15.43:0/1108851740 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4574018070 con 0x7f45800ccbb0 2024-04-25T09:02:14.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.857+0000 7f457cff9640 1 -- 172.21.15.43:0/1108851740 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4574007500 con 0x7f45800ccbb0 2024-04-25T09:02:14.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.857+0000 7f45865ee640 1 -- 172.21.15.43:0/1108851740 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f458013fb60 con 0x7f45800ccbb0 2024-04-25T09:02:14.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.857+0000 7f457cff9640 1 -- 172.21.15.43:0/1108851740 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4574004a20 con 0x7f45800ccbb0 2024-04-25T09:02:14.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.857+0000 7f45865ee640 1 -- 172.21.15.43:0/1108851740 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4580140120 con 0x7f45800ccbb0 2024-04-25T09:02:14.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.858+0000 7f45865ee640 1 -- 172.21.15.43:0/1108851740 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45800ce6b0 con 0x7f45800ccbb0 2024-04-25T09:02:14.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.859+0000 7f457cff9640 1 -- 172.21.15.43:0/1108851740 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f457401b070 con 0x7f45800ccbb0 2024-04-25T09:02:14.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.859+0000 7f457cff9640 1 --2- 172.21.15.43:0/1108851740 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f456c06b500 0x7f456c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:14.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.859+0000 7f457cff9640 1 -- 172.21.15.43:0/1108851740 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f45740893f0 con 0x7f45800ccbb0 2024-04-25T09:02:14.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.859+0000 7f457ffff640 1 --2- 172.21.15.43:0/1108851740 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f456c06b500 0x7f456c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:14.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.860+0000 7f457ffff640 1 --2- 172.21.15.43:0/1108851740 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f456c06b500 0x7f456c06d9c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f4570006fd0 tx=0x7f4570008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:14.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:14.863+0000 7f457cff9640 1 -- 172.21.15.43:0/1108851740 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f45800ce6b0 con 0x7f45800ccbb0 2024-04-25T09:02:15.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.087+0000 7f45865ee640 1 -- 172.21.15.43:0/1108851740 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f458006dfc0 con 0x7f45800ccbb0 2024-04-25T09:02:15.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.088+0000 7f457cff9640 1 -- 172.21.15.43:0/1108851740 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4940 (secure 0 0 0) 0x7f458006dfc0 con 0x7f45800ccbb0 2024-04-25T09:02:15.076 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:15.077 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":61,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2150177203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2150177203},{"type":"v1","addr":"172.21.15.193:6837","nonce":2150177203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":8079,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/621488642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":621488642},{"type":"v1","addr":"172.21.15.43:6834","nonce":621488642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8082,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1524395840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1524395840},{"type":"v1","addr":"172.21.15.43:6835","nonce":1524395840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:01:33.726790+0000","modified":"2024-04-25T09:01:35.706689+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8094},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8094":{"gid":8094,"name":"b","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.193:6836/853943267","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":853943267},{"type":"v1","addr":"172.21.15.193:6836","nonce":853943267}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T09:02:15.077 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 61 2024-04-25T09:02:15.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.090+0000 7f45865ee640 1 -- 172.21.15.43:0/1108851740 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f456c06b500 msgr2=0x7f456c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:15.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.090+0000 7f45865ee640 1 --2- 172.21.15.43:0/1108851740 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f456c06b500 0x7f456c06d9c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f4570006fd0 tx=0x7f4570008040 comp rx=0 tx=0).stop 2024-04-25T09:02:15.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.090+0000 7f45865ee640 1 -- 172.21.15.43:0/1108851740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45800ccbb0 msgr2=0x7f458015b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:15.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.090+0000 7f45865ee640 1 --2- 172.21.15.43:0/1108851740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45800ccbb0 0x7f458015b3c0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f457400c4d0 tx=0x7f457400c990 comp rx=0 tx=0).stop 2024-04-25T09:02:15.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.090+0000 7f45865ee640 1 -- 172.21.15.43:0/1108851740 shutdown_connections 2024-04-25T09:02:15.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.090+0000 7f45865ee640 1 --2- 172.21.15.43:0/1108851740 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f45800cd010 0x7f45800ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:15.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.090+0000 7f45865ee640 1 --2- 172.21.15.43:0/1108851740 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f45800ccbb0 0x7f458015b3c0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:15.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.090+0000 7f45865ee640 1 --2- 172.21.15.43:0/1108851740 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f456c06b500 0x7f456c06d9c0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:15.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.090+0000 7f45865ee640 1 --2- 172.21.15.43:0/1108851740 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f45800cbfc0 0x7f458013f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:15.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.090+0000 7f45865ee640 1 -- 172.21.15.43:0/1108851740 >> 172.21.15.43:0/1108851740 conn(0x7f45800b1480 msgr2=0x7f45800afa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:15.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.090+0000 7f45865ee640 1 -- 172.21.15.43:0/1108851740 shutdown_connections 2024-04-25T09:02:15.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.090+0000 7f45865ee640 1 -- 172.21.15.43:0/1108851740 wait complete. 2024-04-25T09:02:15.090 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:02:15.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.202+0000 7fa0a0e09640 1 Processor -- start 2024-04-25T09:02:15.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.202+0000 7fa0a0e09640 1 -- start start 2024-04-25T09:02:15.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.203+0000 7fa0a0e09640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fa09c15a9d0 0x7fa09c06d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:15.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.203+0000 7fa0a0e09640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa09c06d7d0 0x7fa09c071a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:15.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.203+0000 7fa0a0e09640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa09c072400 con 0x7fa09c071f90 2024-04-25T09:02:15.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.203+0000 7fa0a0e09640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa09c06dc30 con 0x7fa09c06d7d0 2024-04-25T09:02:15.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.203+0000 7fa0a0e09640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa09c0704f0 con 0x7fa09c15a9d0 2024-04-25T09:02:15.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.203+0000 7fa091d9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa09c06d7d0 0x7fa09c071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:15.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.203+0000 7fa09ad9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa09c071f90 0x7fa09c0b3990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32910/0 (socket says 172.21.15.43:32910) 2024-04-25T09:02:15.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.203+0000 7fa09ad9d640 1 -- 172.21.15.43:0/1343427363 learned_addr learned my addr 172.21.15.43:0/1343427363 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:15.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.203+0000 7fa09a59c640 1 --2- 172.21.15.43:0/1343427363 >> v2:172.21.15.193:3300/0 conn(0x7fa09c15a9d0 0x7fa09c06d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:15.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.203+0000 7fa091d9b640 1 -- 172.21.15.43:0/1343427363 >> v2:172.21.15.193:3300/0 conn(0x7fa09c15a9d0 msgr2=0x7fa09c06d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:15.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.203+0000 7fa091d9b640 1 --2- 172.21.15.43:0/1343427363 >> v2:172.21.15.193:3300/0 conn(0x7fa09c15a9d0 0x7fa09c06d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:15.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.203+0000 7fa091d9b640 1 -- 172.21.15.43:0/1343427363 >> v1:172.21.15.43:6789/0 conn(0x7fa09c071f90 legacy=0x7fa09c0b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:15.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.203+0000 7fa099d9b640 1 -- 172.21.15.43:0/1343427363 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3132598842 0 0) 0x7fa09c072400 con 0x7fa09c071f90 2024-04-25T09:02:15.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.203+0000 7fa091d9b640 1 -- 172.21.15.43:0/1343427363 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa09c0707d0 con 0x7fa09c06d7d0 2024-04-25T09:02:15.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.203+0000 7fa09a59c640 1 --2- 172.21.15.43:0/1343427363 >> v2:172.21.15.193:3300/0 conn(0x7fa09c15a9d0 0x7fa09c06d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:15.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.204+0000 7fa091d9b640 1 --2- 172.21.15.43:0/1343427363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa09c06d7d0 0x7fa09c071a50 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fa09c06c190 tx=0x7fa08402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c29271892e82b4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:15.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.204+0000 7fa099d9b640 1 -- 172.21.15.43:0/1343427363 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa084037070 con 0x7fa09c06d7d0 2024-04-25T09:02:15.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.204+0000 7fa099d9b640 1 -- 172.21.15.43:0/1343427363 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa084032440 con 0x7fa09c06d7d0 2024-04-25T09:02:15.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.204+0000 7fa099d9b640 1 -- 172.21.15.43:0/1343427363 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa084032810 con 0x7fa09c06d7d0 2024-04-25T09:02:15.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.205+0000 7fa0a0e09640 1 -- 172.21.15.43:0/1343427363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa09c06d7d0 msgr2=0x7fa09c071a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:15.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.205+0000 7fa0a0e09640 1 --2- 172.21.15.43:0/1343427363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa09c06d7d0 0x7fa09c071a50 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fa09c06c190 tx=0x7fa08402a1a0 comp rx=0 tx=0).stop 2024-04-25T09:02:15.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.205+0000 7fa0a0e09640 1 -- 172.21.15.43:0/1343427363 shutdown_connections 2024-04-25T09:02:15.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.205+0000 7fa0a0e09640 1 --2- 172.21.15.43:0/1343427363 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa09c06d7d0 0x7fa09c071a50 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:15.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.205+0000 7fa0a0e09640 1 --2- 172.21.15.43:0/1343427363 >> v2:172.21.15.193:3300/0 conn(0x7fa09c15a9d0 0x7fa09c06d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:15.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.205+0000 7fa0a0e09640 1 -- 172.21.15.43:0/1343427363 >> 172.21.15.43:0/1343427363 conn(0x7fa09c0b1480 msgr2=0x7fa09c0b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:15.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.205+0000 7fa0a0e09640 1 -- 172.21.15.43:0/1343427363 shutdown_connections 2024-04-25T09:02:15.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.205+0000 7fa0a0e09640 1 -- 172.21.15.43:0/1343427363 wait complete. 2024-04-25T09:02:15.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.206+0000 7fa0a0e09640 1 Processor -- start 2024-04-25T09:02:15.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.206+0000 7fa0a0e09640 1 -- start start 2024-04-25T09:02:15.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.207+0000 7fa0a0e09640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa09c06d7d0 0x7fa09c1f39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:15.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.207+0000 7fa0a0e09640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa09c071f90 0x7fa09c0ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:15.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.207+0000 7fa0a0e09640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa09c0ca170 0x7fa09c157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:15.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.207+0000 7fa0a0e09640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c0712b0 con 0x7fa09c071f90 2024-04-25T09:02:15.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.207+0000 7fa0a0e09640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c070d90 con 0x7fa09c0ca170 2024-04-25T09:02:15.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.207+0000 7fa0a0e09640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa09c0ce950 con 0x7fa09c06d7d0 2024-04-25T09:02:15.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.207+0000 7fa091d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa09c071f90 0x7fa09c0ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:15.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.207+0000 7fa091d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa09c071f90 0x7fa09c0ca870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51324/0 (socket says 172.21.15.43:51324) 2024-04-25T09:02:15.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.207+0000 7fa091d9b640 1 -- 172.21.15.43:0/2525444222 learned_addr learned my addr 172.21.15.43:0/2525444222 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:15.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.207+0000 7fa09ad9d640 1 --2- 172.21.15.43:0/2525444222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa09c0ca170 0x7fa09c157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:15.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.207+0000 7fa091d9b640 1 -- 172.21.15.43:0/2525444222 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa09c06d7d0 msgr2=0x7fa09c1f39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:15.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.207+0000 7fa091d9b640 1 --2- 172.21.15.43:0/2525444222 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa09c06d7d0 0x7fa09c1f39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:15.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.207+0000 7fa091d9b640 1 -- 172.21.15.43:0/2525444222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa09c0ca170 msgr2=0x7fa09c157e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:15.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.207+0000 7fa091d9b640 1 --2- 172.21.15.43:0/2525444222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa09c0ca170 0x7fa09c157e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:15.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.207+0000 7fa091d9b640 1 -- 172.21.15.43:0/2525444222 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa09c0cb1e0 con 0x7fa09c071f90 2024-04-25T09:02:15.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.207+0000 7fa09ad9d640 1 --2- 172.21.15.43:0/2525444222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa09c0ca170 0x7fa09c157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:15.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.207+0000 7fa091d9b640 1 --2- 172.21.15.43:0/2525444222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa09c071f90 0x7fa09c0ca870 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fa084000c00 tx=0x7fa0840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:15.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.208+0000 7fa0937fe640 1 -- 172.21.15.43:0/2525444222 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa084037040 con 0x7fa09c071f90 2024-04-25T09:02:15.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.208+0000 7fa0937fe640 1 -- 172.21.15.43:0/2525444222 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa084042070 con 0x7fa09c071f90 2024-04-25T09:02:15.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.208+0000 7fa0a0e09640 1 -- 172.21.15.43:0/2525444222 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa09c0c8d50 con 0x7fa09c071f90 2024-04-25T09:02:15.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.208+0000 7fa0937fe640 1 -- 172.21.15.43:0/2525444222 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa084030a50 con 0x7fa09c071f90 2024-04-25T09:02:15.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.208+0000 7fa0a0e09640 1 -- 172.21.15.43:0/2525444222 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa09c0c9310 con 0x7fa09c071f90 2024-04-25T09:02:15.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.209+0000 7fa0a0e09640 1 -- 172.21.15.43:0/2525444222 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa09c0677f0 con 0x7fa09c071f90 2024-04-25T09:02:15.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.209+0000 7fa0937fe640 1 -- 172.21.15.43:0/2525444222 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa084030bf0 con 0x7fa09c071f90 2024-04-25T09:02:15.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.209+0000 7fa0937fe640 1 --2- 172.21.15.43:0/2525444222 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa07c06b4d0 0x7fa07c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:15.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.210+0000 7fa0937fe640 1 -- 172.21.15.43:0/2525444222 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa0840aaca0 con 0x7fa09c071f90 2024-04-25T09:02:15.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.210+0000 7fa09a59c640 1 --2- 172.21.15.43:0/2525444222 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa07c06b4d0 0x7fa07c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:15.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.210+0000 7fa09a59c640 1 --2- 172.21.15.43:0/2525444222 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa07c06b4d0 0x7fa07c06d990 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa08c006fd0 tx=0x7fa08c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:15.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.214+0000 7fa0937fe640 1 -- 172.21.15.43:0/2525444222 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa09c0677f0 con 0x7fa09c071f90 2024-04-25T09:02:15.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.395+0000 7fa0a0e09640 1 -- 172.21.15.43:0/2525444222 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa09c0cc190 con 0x7fa09c071f90 2024-04-25T09:02:15.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.396+0000 7fa0937fe640 1 -- 172.21.15.43:0/2525444222 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) v1 ==== 74+0+16583 (secure 0 0 0) 0x7fa09c0cc190 con 0x7fa09c071f90 2024-04-25T09:02:15.385 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:15.385 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":102,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:01:35.356828+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:01:31.997813+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-04-25T09:01:32.695222+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:02:15.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.398+0000 7fa0a0e09640 1 -- 172.21.15.43:0/2525444222 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa07c06b4d0 msgr2=0x7fa07c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:15.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.398+0000 7fa0a0e09640 1 --2- 172.21.15.43:0/2525444222 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa07c06b4d0 0x7fa07c06d990 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa08c006fd0 tx=0x7fa08c008040 comp rx=0 tx=0).stop 2024-04-25T09:02:15.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.398+0000 7fa0a0e09640 1 -- 172.21.15.43:0/2525444222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa09c071f90 msgr2=0x7fa09c0ca870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:15.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.398+0000 7fa0a0e09640 1 --2- 172.21.15.43:0/2525444222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa09c071f90 0x7fa09c0ca870 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fa084000c00 tx=0x7fa0840046c0 comp rx=0 tx=0).stop 2024-04-25T09:02:15.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.398+0000 7fa0a0e09640 1 -- 172.21.15.43:0/2525444222 shutdown_connections 2024-04-25T09:02:15.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.398+0000 7fa0a0e09640 1 --2- 172.21.15.43:0/2525444222 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa07c06b4d0 0x7fa07c06d990 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:15.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.398+0000 7fa0a0e09640 1 --2- 172.21.15.43:0/2525444222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa09c0ca170 0x7fa09c157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:15.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.398+0000 7fa0a0e09640 1 --2- 172.21.15.43:0/2525444222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa09c071f90 0x7fa09c0ca870 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:15.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.398+0000 7fa0a0e09640 1 --2- 172.21.15.43:0/2525444222 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa09c06d7d0 0x7fa09c1f39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:15.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.398+0000 7fa0a0e09640 1 -- 172.21.15.43:0/2525444222 >> 172.21.15.43:0/2525444222 conn(0x7fa09c0b1480 msgr2=0x7fa09c0b0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:15.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.398+0000 7fa0a0e09640 1 -- 172.21.15.43:0/2525444222 shutdown_connections 2024-04-25T09:02:15.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.398+0000 7fa0a0e09640 1 -- 172.21.15.43:0/2525444222 wait complete. 2024-04-25T09:02:15.398 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:02:15.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.511+0000 7f072f7e7640 1 Processor -- start 2024-04-25T09:02:15.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.511+0000 7f072f7e7640 1 -- start start 2024-04-25T09:02:15.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.512+0000 7f072f7e7640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f072815bcb0 0x7f072806e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:15.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.512+0000 7f072f7e7640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f072806ab30 0x7f072806afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:15.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.512+0000 7f072f7e7640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07280ce6e0 con 0x7f0728071850 2024-04-25T09:02:15.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.512+0000 7f072f7e7640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0728160620 con 0x7f072806ab30 2024-04-25T09:02:15.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.512+0000 7f072f7e7640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07280cf010 con 0x7f072815bcb0 2024-04-25T09:02:15.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.513+0000 7f072cd82640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f072806ab30 0x7f072806afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:15.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.513+0000 7f072d583640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f072815bcb0 0x7f072806e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:15.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.513+0000 7f072dd84640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f0728071850 0x7f07280cf4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:52964/0 (socket says 172.21.15.43:52964) 2024-04-25T09:02:15.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.513+0000 7f072dd84640 1 -- 172.21.15.43:0/1028202326 learned_addr learned my addr 172.21.15.43:0/1028202326 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:15.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.513+0000 7f0727fff640 1 -- 172.21.15.43:0/1028202326 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2770458328 0 0) 0x7f07280ce6e0 con 0x7f0728071850 2024-04-25T09:02:15.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.513+0000 7f0727fff640 1 -- 172.21.15.43:0/1028202326 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0710003520 con 0x7f0728071850 2024-04-25T09:02:15.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.513+0000 7f072d583640 1 -- 172.21.15.43:0/1028202326 >> v2:172.21.15.193:3300/0 conn(0x7f072806ab30 msgr2=0x7f072806afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:15.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.513+0000 7f072d583640 1 --2- 172.21.15.43:0/1028202326 >> v2:172.21.15.193:3300/0 conn(0x7f072806ab30 0x7f072806afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:15.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.513+0000 7f072d583640 1 -- 172.21.15.43:0/1028202326 >> v1:172.21.15.193:6789/0 conn(0x7f0728071850 legacy=0x7f07280cf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:15.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.513+0000 7f072d583640 1 -- 172.21.15.43:0/1028202326 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07280ce9c0 con 0x7f072815bcb0 2024-04-25T09:02:15.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.513+0000 7f072cd82640 1 --2- 172.21.15.43:0/1028202326 >> v2:172.21.15.193:3300/0 conn(0x7f072806ab30 0x7f072806afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:02:15.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.514+0000 7f072d583640 1 --2- 172.21.15.43:0/1028202326 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f072815bcb0 0x7f072806e220 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0718009dc0 tx=0x7f071802a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=500cf8e1471cae65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:15.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.514+0000 7f0727fff640 1 -- 172.21.15.43:0/1028202326 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071802ae60 con 0x7f072815bcb0 2024-04-25T09:02:15.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.514+0000 7f0727fff640 1 -- 172.21.15.43:0/1028202326 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0718007740 con 0x7f072815bcb0 2024-04-25T09:02:15.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.514+0000 7f0727fff640 1 -- 172.21.15.43:0/1028202326 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0718007b10 con 0x7f072815bcb0 2024-04-25T09:02:15.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.515+0000 7f072f7e7640 1 -- 172.21.15.43:0/1028202326 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f072815bcb0 msgr2=0x7f072806e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:15.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.515+0000 7f072f7e7640 1 --2- 172.21.15.43:0/1028202326 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f072815bcb0 0x7f072806e220 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0718009dc0 tx=0x7f071802a680 comp rx=0 tx=0).stop 2024-04-25T09:02:15.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.515+0000 7f072f7e7640 1 -- 172.21.15.43:0/1028202326 shutdown_connections 2024-04-25T09:02:15.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.515+0000 7f072f7e7640 1 --2- 172.21.15.43:0/1028202326 >> v2:172.21.15.193:3300/0 conn(0x7f072806ab30 0x7f072806afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:15.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.515+0000 7f072f7e7640 1 --2- 172.21.15.43:0/1028202326 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f072815bcb0 0x7f072806e220 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:15.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.515+0000 7f072f7e7640 1 -- 172.21.15.43:0/1028202326 >> 172.21.15.43:0/1028202326 conn(0x7f07280b2490 msgr2=0x7f07280b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:15.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.515+0000 7f072f7e7640 1 -- 172.21.15.43:0/1028202326 shutdown_connections 2024-04-25T09:02:15.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.515+0000 7f072f7e7640 1 -- 172.21.15.43:0/1028202326 wait complete. 2024-04-25T09:02:15.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.516+0000 7f072f7e7640 1 Processor -- start 2024-04-25T09:02:15.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.516+0000 7f072f7e7640 1 -- start start 2024-04-25T09:02:15.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.517+0000 7f072f7e7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f072806ab30 0x7f07280aa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:15.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.517+0000 7f072f7e7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0728071850 0x7f0728156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:15.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.517+0000 7f072f7e7640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f072815bcb0 0x7f0728156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:15.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.517+0000 7f072f7e7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072806e760 con 0x7f072806ab30 2024-04-25T09:02:15.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.517+0000 7f072f7e7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0728072430 con 0x7f0728071850 2024-04-25T09:02:15.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.517+0000 7f072f7e7640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f072806e8e0 con 0x7f072815bcb0 2024-04-25T09:02:15.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.517+0000 7f072d583640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f072806ab30 0x7f07280aa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:15.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.517+0000 7f072d583640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f072806ab30 0x7f07280aa2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51328/0 (socket says 172.21.15.43:51328) 2024-04-25T09:02:15.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.517+0000 7f072d583640 1 -- 172.21.15.43:0/2138986923 learned_addr learned my addr 172.21.15.43:0/2138986923 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:15.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.517+0000 7f072cd82640 1 --2- 172.21.15.43:0/2138986923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0728071850 0x7f0728156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:15.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.517+0000 7f072dd84640 1 --2- 172.21.15.43:0/2138986923 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f072815bcb0 0x7f0728156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:15.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.517+0000 7f072d583640 1 -- 172.21.15.43:0/2138986923 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f072815bcb0 msgr2=0x7f0728156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:15.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.517+0000 7f072d583640 1 --2- 172.21.15.43:0/2138986923 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f072815bcb0 0x7f0728156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:15.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.517+0000 7f072d583640 1 -- 172.21.15.43:0/2138986923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0728071850 msgr2=0x7f0728156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:15.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.517+0000 7f072d583640 1 --2- 172.21.15.43:0/2138986923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0728071850 0x7f0728156680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:15.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.517+0000 7f072d583640 1 -- 172.21.15.43:0/2138986923 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f072814b980 con 0x7f072806ab30 2024-04-25T09:02:15.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.518+0000 7f072d583640 1 --2- 172.21.15.43:0/2138986923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f072806ab30 0x7f07280aa2f0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f0718009ee0 tx=0x7f0718004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:15.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.518+0000 7f0725ffb640 1 -- 172.21.15.43:0/2138986923 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071803f070 con 0x7f072806ab30 2024-04-25T09:02:15.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.518+0000 7f0725ffb640 1 -- 172.21.15.43:0/2138986923 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0718031a40 con 0x7f072806ab30 2024-04-25T09:02:15.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.518+0000 7f0725ffb640 1 -- 172.21.15.43:0/2138986923 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0718031c10 con 0x7f072806ab30 2024-04-25T09:02:15.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.518+0000 7f072f7e7640 1 -- 172.21.15.43:0/2138986923 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07280a9570 con 0x7f072806ab30 2024-04-25T09:02:15.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.518+0000 7f072f7e7640 1 -- 172.21.15.43:0/2138986923 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07280a9af0 con 0x7f072806ab30 2024-04-25T09:02:15.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.519+0000 7f072f7e7640 1 -- 172.21.15.43:0/2138986923 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0728074ef0 con 0x7f072806ab30 2024-04-25T09:02:15.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.520+0000 7f0725ffb640 1 -- 172.21.15.43:0/2138986923 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f071803b410 con 0x7f072806ab30 2024-04-25T09:02:15.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.520+0000 7f0725ffb640 1 --2- 172.21.15.43:0/2138986923 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f071406b500 0x7f071406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:15.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.520+0000 7f0725ffb640 1 -- 172.21.15.43:0/2138986923 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f07180ab320 con 0x7f072806ab30 2024-04-25T09:02:15.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.520+0000 7f072cd82640 1 --2- 172.21.15.43:0/2138986923 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f071406b500 0x7f071406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:15.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.521+0000 7f072cd82640 1 --2- 172.21.15.43:0/2138986923 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f071406b500 0x7f071406d9c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f071c009560 tx=0x7f071c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:15.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.524+0000 7f0725ffb640 1 -- 172.21.15.43:0/2138986923 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0718049060 con 0x7f072806ab30 2024-04-25T09:02:15.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.742+0000 7f072f7e7640 1 -- 172.21.15.43:0/2138986923 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f072806f750 con 0x7f072806ab30 2024-04-25T09:02:15.731 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-04-25T09:02:15.743+0000 7fe4fb198640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:02:15.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.744+0000 7f0725ffb640 1 -- 172.21.15.43:0/2138986923 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v62) v1 ==== 161+0+0 (secure 0 0 0) 0x7f072814b980 con 0x7f072806ab30 2024-04-25T09:02:15.733 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:02:15.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.747+0000 7f072f7e7640 1 -- 172.21.15.43:0/2138986923 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f071406b500 msgr2=0x7f071406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:15.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.747+0000 7f072f7e7640 1 --2- 172.21.15.43:0/2138986923 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f071406b500 0x7f071406d9c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f071c009560 tx=0x7f071c009190 comp rx=0 tx=0).stop 2024-04-25T09:02:15.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.747+0000 7f072f7e7640 1 -- 172.21.15.43:0/2138986923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f072806ab30 msgr2=0x7f07280aa2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:15.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.747+0000 7f072f7e7640 1 --2- 172.21.15.43:0/2138986923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f072806ab30 0x7f07280aa2f0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f0718009ee0 tx=0x7f0718004a40 comp rx=0 tx=0).stop 2024-04-25T09:02:15.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.747+0000 7f072f7e7640 1 -- 172.21.15.43:0/2138986923 shutdown_connections 2024-04-25T09:02:15.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.747+0000 7f072f7e7640 1 --2- 172.21.15.43:0/2138986923 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f072815bcb0 0x7f0728156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:15.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.747+0000 7f072f7e7640 1 --2- 172.21.15.43:0/2138986923 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f071406b500 0x7f071406d9c0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:15.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.747+0000 7f072f7e7640 1 --2- 172.21.15.43:0/2138986923 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0728071850 0x7f0728156680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:15.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.747+0000 7f072f7e7640 1 --2- 172.21.15.43:0/2138986923 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f072806ab30 0x7f07280aa2f0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:15.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.747+0000 7f072f7e7640 1 -- 172.21.15.43:0/2138986923 >> 172.21.15.43:0/2138986923 conn(0x7f07280b2490 msgr2=0x7f07280b0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:15.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.747+0000 7f072f7e7640 1 -- 172.21.15.43:0/2138986923 shutdown_connections 2024-04-25T09:02:15.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.747+0000 7f072f7e7640 1 -- 172.21.15.43:0/2138986923 wait complete. 2024-04-25T09:02:15.747 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:02:15.776 INFO:tasks.ceph.mds.b.smithi193.stdout:starting mds.b at 2024-04-25T09:02:15.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.863+0000 7fb77d570640 1 Processor -- start 2024-04-25T09:02:15.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.863+0000 7fb77d570640 1 -- start start 2024-04-25T09:02:15.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.864+0000 7fb77d570640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb7780d04a0 0x7fb77815ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:15.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.864+0000 7fb77d570640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb7780ce8c0 0x7fb7780d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:15.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.864+0000 7fb77d570640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7780cf110 con 0x7fb77815dec0 2024-04-25T09:02:15.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.864+0000 7fb77d570640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7780cf3f0 con 0x7fb7780ce8c0 2024-04-25T09:02:15.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.864+0000 7fb77d570640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7780cf6d0 con 0x7fb7780d04a0 2024-04-25T09:02:15.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.864+0000 7fb76ffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb77815dec0 0x7fb7780d0880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32924/0 (socket says 172.21.15.43:32924) 2024-04-25T09:02:15.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.864+0000 7fb76ffff640 1 -- 172.21.15.43:0/1823291301 learned_addr learned my addr 172.21.15.43:0/1823291301 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:15.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.864+0000 7fb7777fe640 1 --2- 172.21.15.43:0/1823291301 >> v2:172.21.15.193:3300/0 conn(0x7fb7780ce8c0 0x7fb7780d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:15.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.864+0000 7fb776ffd640 1 --2- 172.21.15.43:0/1823291301 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb7780d04a0 0x7fb77815ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:15.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.864+0000 7fb7767fc640 1 -- 172.21.15.43:0/1823291301 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2509122130 0 0) 0x7fb7780cf110 con 0x7fb77815dec0 2024-04-25T09:02:15.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.864+0000 7fb7767fc640 1 -- 172.21.15.43:0/1823291301 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb764003520 con 0x7fb77815dec0 2024-04-25T09:02:15.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.864+0000 7fb7767fc640 1 -- 172.21.15.43:0/1823291301 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3847328042 0 0) 0x7fb764003520 con 0x7fb77815dec0 2024-04-25T09:02:15.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.864+0000 7fb7767fc640 1 -- 172.21.15.43:0/1823291301 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb7780d04a0 msgr2=0x7fb77815ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:15.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.864+0000 7fb7767fc640 1 --2- 172.21.15.43:0/1823291301 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb7780d04a0 0x7fb77815ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:15.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.864+0000 7fb7767fc640 1 -- 172.21.15.43:0/1823291301 >> v2:172.21.15.193:3300/0 conn(0x7fb7780ce8c0 msgr2=0x7fb7780d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:15.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.864+0000 7fb7767fc640 1 --2- 172.21.15.43:0/1823291301 >> v2:172.21.15.193:3300/0 conn(0x7fb7780ce8c0 0x7fb7780d17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:15.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.864+0000 7fb7767fc640 1 -- 172.21.15.43:0/1823291301 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7780cf9b0 con 0x7fb77815dec0 2024-04-25T09:02:15.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.864+0000 7fb7767fc640 1 -- 172.21.15.43:0/1823291301 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7fb760001670 con 0x7fb77815dec0 2024-04-25T09:02:15.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.864+0000 7fb7767fc640 1 -- 172.21.15.43:0/1823291301 >> v1:172.21.15.43:6789/0 conn(0x7fb77815dec0 legacy=0x7fb7780d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:15.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.864+0000 7fb7767fc640 1 --2- 172.21.15.43:0/1823291301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7640039b0 0x7fb764003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:15.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.864+0000 7fb7767fc640 1 --2- 172.21.15.43:0/1823291301 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb764004800 0x7fb764004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:15.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.864+0000 7fb7767fc640 1 --2- 172.21.15.43:0/1823291301 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb7780d04a0 0x7fb7640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:15.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.864+0000 7fb7767fc640 1 -- 172.21.15.43:0/1823291301 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7780cf3f0 con 0x7fb764004800 2024-04-25T09:02:15.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.864+0000 7fb7767fc640 1 -- 172.21.15.43:0/1823291301 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7780cf6d0 con 0x7fb7640039b0 2024-04-25T09:02:15.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.864+0000 7fb7767fc640 1 -- 172.21.15.43:0/1823291301 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb76400b9c0 con 0x7fb7780d04a0 2024-04-25T09:02:15.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.864+0000 7fb7777fe640 1 --2- 172.21.15.43:0/1823291301 >> v2:172.21.15.193:3300/0 conn(0x7fb7780ce8c0 0x7fb7780d17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:02:15.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.865+0000 7fb76ffff640 1 --2- 172.21.15.43:0/1823291301 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb764004800 0x7fb764004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:15.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.865+0000 7fb776ffd640 1 --2- 172.21.15.43:0/1823291301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7640039b0 0x7fb764003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:15.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.865+0000 7fb7777fe640 1 --2- 172.21.15.43:0/1823291301 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb7780d04a0 0x7fb7640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:15.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.865+0000 7fb76ffff640 1 -- 172.21.15.43:0/1823291301 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb7780d04a0 msgr2=0x7fb7640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:15.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.865+0000 7fb76ffff640 1 --2- 172.21.15.43:0/1823291301 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb7780d04a0 0x7fb7640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:15.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.865+0000 7fb76ffff640 1 -- 172.21.15.43:0/1823291301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7640039b0 msgr2=0x7fb764003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:15.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.865+0000 7fb76ffff640 1 --2- 172.21.15.43:0/1823291301 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7640039b0 0x7fb764003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:15.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.865+0000 7fb76ffff640 1 -- 172.21.15.43:0/1823291301 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb764003520 con 0x7fb764004800 2024-04-25T09:02:15.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.865+0000 7fb76ffff640 1 --2- 172.21.15.43:0/1823291301 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb764004800 0x7fb764004c80 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fb760007fd0 tx=0x7fb764005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef70e29cfa9ba020 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:15.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.866+0000 7fb7767fc640 1 -- 172.21.15.43:0/1823291301 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb760007d50 con 0x7fb764004800 2024-04-25T09:02:15.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.866+0000 7fb7767fc640 1 -- 172.21.15.43:0/1823291301 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb760005af0 con 0x7fb764004800 2024-04-25T09:02:15.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.866+0000 7fb7767fc640 1 -- 172.21.15.43:0/1823291301 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb760007070 con 0x7fb764004800 2024-04-25T09:02:15.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.866+0000 7fb77d570640 1 -- 172.21.15.43:0/1823291301 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb764004800 msgr2=0x7fb764004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:15.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.866+0000 7fb77d570640 1 --2- 172.21.15.43:0/1823291301 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb764004800 0x7fb764004c80 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fb760007fd0 tx=0x7fb764005830 comp rx=0 tx=0).stop 2024-04-25T09:02:15.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.866+0000 7fb7777fe640 1 -- 172.21.15.43:0/1823291301 reap_dead start 2024-04-25T09:02:15.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.866+0000 7fb77d570640 1 -- 172.21.15.43:0/1823291301 shutdown_connections 2024-04-25T09:02:15.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.866+0000 7fb77d570640 1 -- 172.21.15.43:0/1823291301 >> 172.21.15.43:0/1823291301 conn(0x7fb77809cd40 msgr2=0x7fb7780d3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:15.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.866+0000 7fb77d570640 1 -- 172.21.15.43:0/1823291301 shutdown_connections 2024-04-25T09:02:15.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.866+0000 7fb77d570640 1 -- 172.21.15.43:0/1823291301 wait complete. 2024-04-25T09:02:15.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.867+0000 7fb77d570640 1 Processor -- start 2024-04-25T09:02:15.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.867+0000 7fb77d570640 1 -- start start 2024-04-25T09:02:15.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.868+0000 7fb77d570640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb764004800 0x7fb7780c8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:15.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.868+0000 7fb77d570640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb77815caa0 0x7fb77815bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:15.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.868+0000 7fb77d570640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb77815c410 0x7fb7780b8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:15.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.868+0000 7fb77d570640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7780cee90 con 0x7fb764004800 2024-04-25T09:02:15.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.868+0000 7fb77d570640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7780ced10 con 0x7fb77815c410 2024-04-25T09:02:15.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.868+0000 7fb77d570640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb77815ed90 con 0x7fb77815caa0 2024-04-25T09:02:15.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.868+0000 7fb776ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb764004800 0x7fb7780c8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:15.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.868+0000 7fb776ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb764004800 0x7fb7780c8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51344/0 (socket says 172.21.15.43:51344) 2024-04-25T09:02:15.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.868+0000 7fb776ffd640 1 -- 172.21.15.43:0/936937817 learned_addr learned my addr 172.21.15.43:0/936937817 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:15.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.868+0000 7fb76ffff640 1 --2- 172.21.15.43:0/936937817 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb77815caa0 0x7fb77815bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:15.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.868+0000 7fb7777fe640 1 --2- 172.21.15.43:0/936937817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb77815c410 0x7fb7780b8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:15.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.869+0000 7fb776ffd640 1 -- 172.21.15.43:0/936937817 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb77815caa0 msgr2=0x7fb77815bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:15.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.869+0000 7fb776ffd640 1 --2- 172.21.15.43:0/936937817 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb77815caa0 0x7fb77815bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:15.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.869+0000 7fb776ffd640 1 -- 172.21.15.43:0/936937817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb77815c410 msgr2=0x7fb7780b8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:15.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.869+0000 7fb776ffd640 1 --2- 172.21.15.43:0/936937817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb77815c410 0x7fb7780b8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:15.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.869+0000 7fb776ffd640 1 -- 172.21.15.43:0/936937817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7780c8110 con 0x7fb764004800 2024-04-25T09:02:15.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.869+0000 7fb776ffd640 1 --2- 172.21.15.43:0/936937817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb764004800 0x7fb7780c8b90 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fb7680118e0 tx=0x7fb768011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:15.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.869+0000 7fb76f7fe640 1 -- 172.21.15.43:0/936937817 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb76800faf0 con 0x7fb764004800 2024-04-25T09:02:15.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.869+0000 7fb76f7fe640 1 -- 172.21.15.43:0/936937817 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb76800fc90 con 0x7fb764004800 2024-04-25T09:02:15.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.869+0000 7fb76f7fe640 1 -- 172.21.15.43:0/936937817 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb768014440 con 0x7fb764004800 2024-04-25T09:02:15.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.869+0000 7fb77d570640 1 -- 172.21.15.43:0/936937817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7780ca020 con 0x7fb764004800 2024-04-25T09:02:15.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.869+0000 7fb77d570640 1 -- 172.21.15.43:0/936937817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7780c83b0 con 0x7fb764004800 2024-04-25T09:02:15.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.870+0000 7fb77d570640 1 -- 172.21.15.43:0/936937817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb778074f60 con 0x7fb764004800 2024-04-25T09:02:15.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.870+0000 7fb76f7fe640 1 -- 172.21.15.43:0/936937817 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb768012040 con 0x7fb764004800 2024-04-25T09:02:15.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.871+0000 7fb76f7fe640 1 --2- 172.21.15.43:0/936937817 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb75806b500 0x7fb75806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:15.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.871+0000 7fb76f7fe640 1 -- 172.21.15.43:0/936937817 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb76808a950 con 0x7fb764004800 2024-04-25T09:02:15.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.871+0000 7fb76ffff640 1 --2- 172.21.15.43:0/936937817 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb75806b500 0x7fb75806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:15.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.872+0000 7fb76ffff640 1 --2- 172.21.15.43:0/936937817 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb75806b500 0x7fb75806d9c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fb760033000 tx=0x7fb760009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:15.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:15.875+0000 7fb76f7fe640 1 -- 172.21.15.43:0/936937817 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb768019030 con 0x7fb764004800 2024-04-25T09:02:15.997 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:02:16.000 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:02:16.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.095+0000 7fb77d570640 1 -- 172.21.15.43:0/936937817 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb77808d7e0 con 0x7fb764004800 2024-04-25T09:02:16.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.753+0000 7fb76f7fe640 1 -- 172.21.15.43:0/936937817 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v63) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb7680596d0 con 0x7fb764004800 2024-04-25T09:02:16.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.756+0000 7fb77d570640 1 -- 172.21.15.43:0/936937817 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb75806b500 msgr2=0x7fb75806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:16.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.756+0000 7fb77d570640 1 --2- 172.21.15.43:0/936937817 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb75806b500 0x7fb75806d9c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fb760033000 tx=0x7fb760009030 comp rx=0 tx=0).stop 2024-04-25T09:02:16.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.756+0000 7fb77d570640 1 -- 172.21.15.43:0/936937817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb764004800 msgr2=0x7fb7780c8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:16.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.756+0000 7fb77d570640 1 --2- 172.21.15.43:0/936937817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb764004800 0x7fb7780c8b90 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fb7680118e0 tx=0x7fb768011da0 comp rx=0 tx=0).stop 2024-04-25T09:02:16.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.756+0000 7fb77d570640 1 -- 172.21.15.43:0/936937817 shutdown_connections 2024-04-25T09:02:16.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.756+0000 7fb77d570640 1 --2- 172.21.15.43:0/936937817 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb75806b500 0x7fb75806d9c0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:16.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.756+0000 7fb77d570640 1 --2- 172.21.15.43:0/936937817 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb77815c410 0x7fb7780b8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:16.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.756+0000 7fb77d570640 1 --2- 172.21.15.43:0/936937817 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb77815caa0 0x7fb77815bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:16.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.756+0000 7fb77d570640 1 --2- 172.21.15.43:0/936937817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb764004800 0x7fb7780c8b90 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:16.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.756+0000 7fb77d570640 1 -- 172.21.15.43:0/936937817 >> 172.21.15.43:0/936937817 conn(0x7fb77809cd40 msgr2=0x7fb7780cce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:16.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.756+0000 7fb77d570640 1 -- 172.21.15.43:0/936937817 shutdown_connections 2024-04-25T09:02:16.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.756+0000 7fb77d570640 1 -- 172.21.15.43:0/936937817 wait complete. 2024-04-25T09:02:16.757 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:02:16.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.917+0000 7fb231ce1640 1 Processor -- start 2024-04-25T09:02:16.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.917+0000 7fb231ce1640 1 -- start start 2024-04-25T09:02:16.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.918+0000 7fb231ce1640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb22c0ce1e0 0x7fb22c0b0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:16.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.918+0000 7fb231ce1640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb22c0cd790 0x7fb22c15ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:16.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.918+0000 7fb231ce1640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb22c15bc80 con 0x7fb22c0ce5c0 2024-04-25T09:02:16.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.918+0000 7fb231ce1640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb22c0cc9d0 con 0x7fb22c0ce1e0 2024-04-25T09:02:16.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.918+0000 7fb231ce1640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb22c0cccb0 con 0x7fb22c0cd790 2024-04-25T09:02:16.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.918+0000 7fb22b7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb22c0ce1e0 0x7fb22c0b0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:16.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.918+0000 7fb22b7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb22c0ce1e0 0x7fb22c0b0c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51358/0 (socket says 172.21.15.43:51358) 2024-04-25T09:02:16.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.918+0000 7fb22b7fe640 1 -- 172.21.15.43:0/1969068379 learned_addr learned my addr 172.21.15.43:0/1969068379 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:16.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.918+0000 7fb22affd640 1 --2- 172.21.15.43:0/1969068379 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb22c0cd790 0x7fb22c15ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:16.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.918+0000 7fb22b7fe640 1 -- 172.21.15.43:0/1969068379 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb22c0cd790 msgr2=0x7fb22c15ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:16.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.918+0000 7fb22b7fe640 1 --2- 172.21.15.43:0/1969068379 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb22c0cd790 0x7fb22c15ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:16.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.918+0000 7fb22b7fe640 1 -- 172.21.15.43:0/1969068379 >> v1:172.21.15.193:6789/0 conn(0x7fb22c0ce5c0 legacy=0x7fb22c1617b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:16.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.918+0000 7fb22b7fe640 1 -- 172.21.15.43:0/1969068379 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb22c0ccf90 con 0x7fb22c0ce1e0 2024-04-25T09:02:16.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.919+0000 7fb22b7fe640 1 --2- 172.21.15.43:0/1969068379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb22c0ce1e0 0x7fb22c0b0c40 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fb21c009870 tx=0x7fb21c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db68c141872fd2fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:16.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.919+0000 7fb22a7fc640 1 -- 172.21.15.43:0/1969068379 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c037070 con 0x7fb22c0ce1e0 2024-04-25T09:02:16.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.919+0000 7fb22a7fc640 1 -- 172.21.15.43:0/1969068379 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb21c032440 con 0x7fb22c0ce1e0 2024-04-25T09:02:16.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.919+0000 7fb22a7fc640 1 -- 172.21.15.43:0/1969068379 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21c032810 con 0x7fb22c0ce1e0 2024-04-25T09:02:16.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.920+0000 7fb231ce1640 1 -- 172.21.15.43:0/1969068379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb22c0ce1e0 msgr2=0x7fb22c0b0c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:16.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.920+0000 7fb231ce1640 1 --2- 172.21.15.43:0/1969068379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb22c0ce1e0 0x7fb22c0b0c40 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fb21c009870 tx=0x7fb21c02a210 comp rx=0 tx=0).stop 2024-04-25T09:02:16.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.920+0000 7fb231ce1640 1 -- 172.21.15.43:0/1969068379 shutdown_connections 2024-04-25T09:02:16.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.920+0000 7fb231ce1640 1 --2- 172.21.15.43:0/1969068379 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb22c0cd790 0x7fb22c15ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:16.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.920+0000 7fb231ce1640 1 --2- 172.21.15.43:0/1969068379 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb22c0ce1e0 0x7fb22c0b0c40 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:16.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.920+0000 7fb231ce1640 1 -- 172.21.15.43:0/1969068379 >> 172.21.15.43:0/1969068379 conn(0x7fb22c0d17d0 msgr2=0x7fb22c0d1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:16.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.920+0000 7fb231ce1640 1 -- 172.21.15.43:0/1969068379 shutdown_connections 2024-04-25T09:02:16.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.920+0000 7fb231ce1640 1 -- 172.21.15.43:0/1969068379 wait complete. 2024-04-25T09:02:16.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.921+0000 7fb231ce1640 1 Processor -- start 2024-04-25T09:02:16.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.921+0000 7fb231ce1640 1 -- start start 2024-04-25T09:02:16.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.922+0000 7fb231ce1640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb22c0cd790 0x7fb22c151e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:16.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.922+0000 7fb231ce1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb22c0ce1e0 0x7fb22c0c88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:16.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.922+0000 7fb231ce1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb22c0ce5c0 0x7fb22c06cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:16.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.922+0000 7fb231ce1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb22c0ceb90 con 0x7fb22c0ce5c0 2024-04-25T09:02:16.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.922+0000 7fb231ce1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb22c0cea10 con 0x7fb22c0ce1e0 2024-04-25T09:02:16.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.922+0000 7fb231ce1640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb22c0cc610 con 0x7fb22c0cd790 2024-04-25T09:02:16.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.922+0000 7fb22bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb22c0ce5c0 0x7fb22c06cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:16.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.922+0000 7fb22bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb22c0ce5c0 0x7fb22c06cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51360/0 (socket says 172.21.15.43:51360) 2024-04-25T09:02:16.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.922+0000 7fb22bfff640 1 -- 172.21.15.43:0/1393952131 learned_addr learned my addr 172.21.15.43:0/1393952131 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:16.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.922+0000 7fb22b7fe640 1 --2- 172.21.15.43:0/1393952131 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb22c0cd790 0x7fb22c151e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:16.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.922+0000 7fb22bfff640 1 -- 172.21.15.43:0/1393952131 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb22c0cd790 msgr2=0x7fb22c151e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:16.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.922+0000 7fb22bfff640 1 --2- 172.21.15.43:0/1393952131 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb22c0cd790 0x7fb22c151e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:16.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.922+0000 7fb22bfff640 1 -- 172.21.15.43:0/1393952131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb22c0ce1e0 msgr2=0x7fb22c0c88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:16.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.922+0000 7fb22bfff640 1 --2- 172.21.15.43:0/1393952131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb22c0ce1e0 0x7fb22c0c88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:16.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.922+0000 7fb22bfff640 1 -- 172.21.15.43:0/1393952131 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb22c1523d0 con 0x7fb22c0ce5c0 2024-04-25T09:02:16.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.923+0000 7fb22bfff640 1 --2- 172.21.15.43:0/1393952131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb22c0ce5c0 0x7fb22c06cfa0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fb22400b9e0 tx=0x7fb22400bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:16.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.923+0000 7fb213fff640 1 -- 172.21.15.43:0/1393952131 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb224003b70 con 0x7fb22c0ce5c0 2024-04-25T09:02:16.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.923+0000 7fb213fff640 1 -- 172.21.15.43:0/1393952131 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb224003d10 con 0x7fb22c0ce5c0 2024-04-25T09:02:16.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.923+0000 7fb213fff640 1 -- 172.21.15.43:0/1393952131 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22400cae0 con 0x7fb22c0ce5c0 2024-04-25T09:02:16.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.923+0000 7fb231ce1640 1 -- 172.21.15.43:0/1393952131 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb22c0c9210 con 0x7fb22c0ce5c0 2024-04-25T09:02:16.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.923+0000 7fb231ce1640 1 -- 172.21.15.43:0/1393952131 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb22c0c97d0 con 0x7fb22c0ce5c0 2024-04-25T09:02:16.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.924+0000 7fb231ce1640 1 -- 172.21.15.43:0/1393952131 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb22c0b59e0 con 0x7fb22c0ce5c0 2024-04-25T09:02:16.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.924+0000 7fb213fff640 1 -- 172.21.15.43:0/1393952131 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb22400cc80 con 0x7fb22c0ce5c0 2024-04-25T09:02:16.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.925+0000 7fb213fff640 1 --2- 172.21.15.43:0/1393952131 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb21806b4d0 0x7fb21806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:16.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.925+0000 7fb213fff640 1 -- 172.21.15.43:0/1393952131 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb224089fb0 con 0x7fb22c0ce5c0 2024-04-25T09:02:16.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.925+0000 7fb22b7fe640 1 --2- 172.21.15.43:0/1393952131 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb21806b4d0 0x7fb21806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:16.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.925+0000 7fb22b7fe640 1 --2- 172.21.15.43:0/1393952131 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb21806b4d0 0x7fb21806d990 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fb21c034000 tx=0x7fb21c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:16.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:16.929+0000 7fb213fff640 1 -- 172.21.15.43:0/1393952131 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb224058e60 con 0x7fb22c0ce5c0 2024-04-25T09:02:17.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.108+0000 7fb231ce1640 1 -- 172.21.15.43:0/1393952131 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb22c068a40 con 0x7fb22c0ce5c0 2024-04-25T09:02:17.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.755+0000 7fb213fff640 1 -- 172.21.15.43:0/1393952131 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v104) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb224058760 con 0x7fb22c0ce5c0 2024-04-25T09:02:17.744 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:02:17.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.759+0000 7fb231ce1640 1 -- 172.21.15.43:0/1393952131 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb21806b4d0 msgr2=0x7fb21806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:17.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.759+0000 7fb231ce1640 1 --2- 172.21.15.43:0/1393952131 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb21806b4d0 0x7fb21806d990 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fb21c034000 tx=0x7fb21c02e000 comp rx=0 tx=0).stop 2024-04-25T09:02:17.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.759+0000 7fb231ce1640 1 -- 172.21.15.43:0/1393952131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb22c0ce5c0 msgr2=0x7fb22c06cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:17.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.759+0000 7fb231ce1640 1 --2- 172.21.15.43:0/1393952131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb22c0ce5c0 0x7fb22c06cfa0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fb22400b9e0 tx=0x7fb22400bea0 comp rx=0 tx=0).stop 2024-04-25T09:02:17.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.759+0000 7fb231ce1640 1 -- 172.21.15.43:0/1393952131 shutdown_connections 2024-04-25T09:02:17.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.759+0000 7fb231ce1640 1 --2- 172.21.15.43:0/1393952131 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb22c0ce5c0 0x7fb22c06cfa0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:17.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.759+0000 7fb231ce1640 1 --2- 172.21.15.43:0/1393952131 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb21806b4d0 0x7fb21806d990 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:17.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.759+0000 7fb231ce1640 1 --2- 172.21.15.43:0/1393952131 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb22c0ce1e0 0x7fb22c0c88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:17.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.759+0000 7fb231ce1640 1 --2- 172.21.15.43:0/1393952131 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb22c0cd790 0x7fb22c151e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:17.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.759+0000 7fb231ce1640 1 -- 172.21.15.43:0/1393952131 >> 172.21.15.43:0/1393952131 conn(0x7fb22c0d17d0 msgr2=0x7fb22c0ae5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:17.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.759+0000 7fb231ce1640 1 -- 172.21.15.43:0/1393952131 shutdown_connections 2024-04-25T09:02:17.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.759+0000 7fb231ce1640 1 -- 172.21.15.43:0/1393952131 wait complete. 2024-04-25T09:02:17.763 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:02:17.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.885+0000 7f41f0722640 1 Processor -- start 2024-04-25T09:02:17.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.886+0000 7f41f0722640 1 -- start start 2024-04-25T09:02:17.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.886+0000 7f41f0722640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f41e815def0 0x7f41e80d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:17.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.886+0000 7f41f0722640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f41e80d4670 0x7f41e80d03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:17.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.886+0000 7f41f0722640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41e80d2370 con 0x7f41e80d0930 2024-04-25T09:02:17.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.886+0000 7f41f0722640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41e80b37b0 con 0x7f41e815def0 2024-04-25T09:02:17.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.886+0000 7f41f0722640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41e80d2650 con 0x7f41e80d4670 2024-04-25T09:02:17.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.887+0000 7f41eecbf640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f41e80d0930 0x7f41e80ce4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32932/0 (socket says 172.21.15.43:32932) 2024-04-25T09:02:17.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.887+0000 7f41ee4be640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f41e815def0 0x7f41e80d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:17.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.887+0000 7f41eecbf640 1 -- 172.21.15.43:0/1899008043 learned_addr learned my addr 172.21.15.43:0/1899008043 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:17.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.887+0000 7f41edcbd640 1 --2- 172.21.15.43:0/1899008043 >> v2:172.21.15.193:3300/0 conn(0x7f41e80d4670 0x7f41e80d03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:17.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.887+0000 7f41ee4be640 1 -- 172.21.15.43:0/1899008043 >> v2:172.21.15.193:3300/0 conn(0x7f41e80d4670 msgr2=0x7f41e80d03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:17.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.887+0000 7f41ee4be640 1 --2- 172.21.15.43:0/1899008043 >> v2:172.21.15.193:3300/0 conn(0x7f41e80d4670 0x7f41e80d03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:17.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.887+0000 7f41ee4be640 1 -- 172.21.15.43:0/1899008043 >> v1:172.21.15.43:6789/0 conn(0x7f41e80d0930 legacy=0x7f41e80ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:17.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.887+0000 7f41ed4bc640 1 -- 172.21.15.43:0/1899008043 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1162989077 0 0) 0x7f41e80d2370 con 0x7f41e80d0930 2024-04-25T09:02:17.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.887+0000 7f41ee4be640 1 -- 172.21.15.43:0/1899008043 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e80b3440 con 0x7f41e815def0 2024-04-25T09:02:17.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.888+0000 7f41ee4be640 1 --2- 172.21.15.43:0/1899008043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41e815def0 0x7f41e80d4130 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f41e80cfbe0 tx=0x7f41e402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2fd310eed4df230 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:17.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.888+0000 7f41ed4bc640 1 -- 172.21.15.43:0/1899008043 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e4037070 con 0x7f41e815def0 2024-04-25T09:02:17.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.888+0000 7f41ed4bc640 1 -- 172.21.15.43:0/1899008043 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41e4032440 con 0x7f41e815def0 2024-04-25T09:02:17.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.888+0000 7f41ed4bc640 1 -- 172.21.15.43:0/1899008043 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e402ae60 con 0x7f41e815def0 2024-04-25T09:02:17.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.888+0000 7f41f0722640 1 -- 172.21.15.43:0/1899008043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41e815def0 msgr2=0x7f41e80d4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:17.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.888+0000 7f41f0722640 1 --2- 172.21.15.43:0/1899008043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41e815def0 0x7f41e80d4130 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f41e80cfbe0 tx=0x7f41e402a110 comp rx=0 tx=0).stop 2024-04-25T09:02:17.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.889+0000 7f41f0722640 1 -- 172.21.15.43:0/1899008043 shutdown_connections 2024-04-25T09:02:17.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.889+0000 7f41f0722640 1 --2- 172.21.15.43:0/1899008043 >> v2:172.21.15.193:3300/0 conn(0x7f41e80d4670 0x7f41e80d03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:17.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.889+0000 7f41f0722640 1 --2- 172.21.15.43:0/1899008043 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41e815def0 0x7f41e80d4130 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:17.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.889+0000 7f41f0722640 1 -- 172.21.15.43:0/1899008043 >> 172.21.15.43:0/1899008043 conn(0x7f41e809cd60 msgr2=0x7f41e80d3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:17.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.889+0000 7f41f0722640 1 -- 172.21.15.43:0/1899008043 shutdown_connections 2024-04-25T09:02:17.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.889+0000 7f41f0722640 1 -- 172.21.15.43:0/1899008043 wait complete. 2024-04-25T09:02:17.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.890+0000 7f41f0722640 1 Processor -- start 2024-04-25T09:02:17.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.890+0000 7f41f0722640 1 -- start start 2024-04-25T09:02:17.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.890+0000 7f41f0722640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f41e80d0930 0x7f41e8152ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:17.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.890+0000 7f41f0722640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41e80d4670 0x7f41e8152460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:17.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.890+0000 7f41f0722640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41e815def0 0x7f41e815f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:17.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.890+0000 7f41f0722640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e80d2150 con 0x7f41e80d4670 2024-04-25T09:02:17.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.890+0000 7f41f0722640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e80d1fd0 con 0x7f41e815def0 2024-04-25T09:02:17.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.890+0000 7f41f0722640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e80b4c50 con 0x7f41e80d0930 2024-04-25T09:02:17.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.891+0000 7f41edcbd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41e80d4670 0x7f41e8152460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.891+0000 7f41edcbd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41e80d4670 0x7f41e8152460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51376/0 (socket says 172.21.15.43:51376) 2024-04-25T09:02:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.891+0000 7f41edcbd640 1 -- 172.21.15.43:0/3923593032 learned_addr learned my addr 172.21.15.43:0/3923593032 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.891+0000 7f41ee4be640 1 --2- 172.21.15.43:0/3923593032 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f41e80d0930 0x7f41e8152ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.891+0000 7f41eecbf640 1 --2- 172.21.15.43:0/3923593032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41e815def0 0x7f41e815f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.891+0000 7f41edcbd640 1 -- 172.21.15.43:0/3923593032 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f41e80d0930 msgr2=0x7f41e8152ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.891+0000 7f41edcbd640 1 --2- 172.21.15.43:0/3923593032 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f41e80d0930 0x7f41e8152ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.891+0000 7f41edcbd640 1 -- 172.21.15.43:0/3923593032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41e815def0 msgr2=0x7f41e815f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.891+0000 7f41edcbd640 1 --2- 172.21.15.43:0/3923593032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41e815def0 0x7f41e815f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.891+0000 7f41edcbd640 1 -- 172.21.15.43:0/3923593032 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e81529a0 con 0x7f41e80d4670 2024-04-25T09:02:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.891+0000 7f41edcbd640 1 --2- 172.21.15.43:0/3923593032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41e80d4670 0x7f41e8152460 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f41d800d800 tx=0x7f41d800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.891+0000 7f41deffd640 1 -- 172.21.15.43:0/3923593032 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d800b6c0 con 0x7f41e80d4670 2024-04-25T09:02:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.891+0000 7f41deffd640 1 -- 172.21.15.43:0/3923593032 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d8019070 con 0x7f41e80d4670 2024-04-25T09:02:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.891+0000 7f41f0722640 1 -- 172.21.15.43:0/3923593032 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41e8153180 con 0x7f41e80d4670 2024-04-25T09:02:17.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.891+0000 7f41deffd640 1 -- 172.21.15.43:0/3923593032 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d800ba80 con 0x7f41e80d4670 2024-04-25T09:02:17.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.891+0000 7f41f0722640 1 -- 172.21.15.43:0/3923593032 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41e80cbbe0 con 0x7f41e80d4670 2024-04-25T09:02:17.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.892+0000 7f41f0722640 1 -- 172.21.15.43:0/3923593032 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41e8068a40 con 0x7f41e80d4670 2024-04-25T09:02:17.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.893+0000 7f41deffd640 1 -- 172.21.15.43:0/3923593032 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f41d800bc20 con 0x7f41e80d4670 2024-04-25T09:02:17.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.893+0000 7f41deffd640 1 --2- 172.21.15.43:0/3923593032 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f41cc06b4d0 0x7f41cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:17.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.893+0000 7f41deffd640 1 -- 172.21.15.43:0/3923593032 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f41d808a580 con 0x7f41e80d4670 2024-04-25T09:02:17.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.893+0000 7f41ee4be640 1 --2- 172.21.15.43:0/3923593032 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f41cc06b4d0 0x7f41cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:17.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.894+0000 7f41ee4be640 1 --2- 172.21.15.43:0/3923593032 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f41cc06b4d0 0x7f41cc06d990 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f41e4034000 tx=0x7f41e402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:17.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:17.897+0000 7f41deffd640 1 -- 172.21.15.43:0/3923593032 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f41e81529a0 con 0x7f41e80d4670 2024-04-25T09:02:18.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.080+0000 7f41f0722640 1 -- 172.21.15.43:0/3923593032 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f41e81536a0 con 0x7f41e80d4670 2024-04-25T09:02:18.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.759+0000 7f41deffd640 1 -- 172.21.15.43:0/3923593032 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v105) v1 ==== 169+0+0 (secure 0 0 0) 0x7f41d801c360 con 0x7f41e80d4670 2024-04-25T09:02:18.749 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' removed 2024-04-25T09:02:18.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.763+0000 7f41f0722640 1 -- 172.21.15.43:0/3923593032 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f41cc06b4d0 msgr2=0x7f41cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:18.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.763+0000 7f41f0722640 1 --2- 172.21.15.43:0/3923593032 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f41cc06b4d0 0x7f41cc06d990 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f41e4034000 tx=0x7f41e402e000 comp rx=0 tx=0).stop 2024-04-25T09:02:18.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.763+0000 7f41f0722640 1 -- 172.21.15.43:0/3923593032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41e80d4670 msgr2=0x7f41e8152460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:18.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.763+0000 7f41f0722640 1 --2- 172.21.15.43:0/3923593032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41e80d4670 0x7f41e8152460 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f41d800d800 tx=0x7f41d800dcc0 comp rx=0 tx=0).stop 2024-04-25T09:02:18.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.764+0000 7f41f0722640 1 -- 172.21.15.43:0/3923593032 shutdown_connections 2024-04-25T09:02:18.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.764+0000 7f41f0722640 1 --2- 172.21.15.43:0/3923593032 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f41cc06b4d0 0x7f41cc06d990 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:18.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.764+0000 7f41f0722640 1 --2- 172.21.15.43:0/3923593032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f41e815def0 0x7f41e815f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:18.754 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.764+0000 7f41f0722640 1 --2- 172.21.15.43:0/3923593032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f41e80d4670 0x7f41e8152460 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:18.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.764+0000 7f41f0722640 1 --2- 172.21.15.43:0/3923593032 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f41e80d0930 0x7f41e8152ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:18.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.764+0000 7f41f0722640 1 -- 172.21.15.43:0/3923593032 >> 172.21.15.43:0/3923593032 conn(0x7f41e809cd60 msgr2=0x7f41e806b130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:18.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.764+0000 7f41f0722640 1 -- 172.21.15.43:0/3923593032 shutdown_connections 2024-04-25T09:02:18.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.764+0000 7f41f0722640 1 -- 172.21.15.43:0/3923593032 wait complete. 2024-04-25T09:02:18.774 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T09:02:18.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.893+0000 7f3052942640 1 Processor -- start 2024-04-25T09:02:18.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.893+0000 7f3052942640 1 -- start start 2024-04-25T09:02:18.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.894+0000 7f3052942640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f304c1618b0 0x7f304c15e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:18.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.894+0000 7f3052942640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f304c06f230 0x7f304c0ad230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:18.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.894+0000 7f3052942640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f304c15e8d0 con 0x7f304c0acda0 2024-04-25T09:02:18.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.894+0000 7f3052942640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f304c15ebb0 con 0x7f304c06f230 2024-04-25T09:02:18.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.894+0000 7f3052942640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f304c15ee90 con 0x7f304c1618b0 2024-04-25T09:02:18.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.894+0000 7f3050edf640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f304c06f230 0x7f304c0ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:18.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.894+0000 7f304b7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f304c0acda0 0x7f304c06eb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32934/0 (socket says 172.21.15.43:32934) 2024-04-25T09:02:18.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.894+0000 7f304b7fe640 1 -- 172.21.15.43:0/2124000617 learned_addr learned my addr 172.21.15.43:0/2124000617 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:18.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.895+0000 7f304bfff640 1 --2- 172.21.15.43:0/2124000617 >> v2:172.21.15.193:3300/0 conn(0x7f304c1618b0 0x7f304c15e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:18.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.895+0000 7f3050edf640 1 -- 172.21.15.43:0/2124000617 >> v2:172.21.15.193:3300/0 conn(0x7f304c1618b0 msgr2=0x7f304c15e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:18.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.895+0000 7f3050edf640 1 --2- 172.21.15.43:0/2124000617 >> v2:172.21.15.193:3300/0 conn(0x7f304c1618b0 0x7f304c15e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:18.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.895+0000 7f3050edf640 1 -- 172.21.15.43:0/2124000617 >> v1:172.21.15.43:6789/0 conn(0x7f304c0acda0 legacy=0x7f304c06eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:18.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.895+0000 7f3050edf640 1 -- 172.21.15.43:0/2124000617 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f304c15f170 con 0x7f304c06f230 2024-04-25T09:02:18.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.895+0000 7f304affd640 1 -- 172.21.15.43:0/2124000617 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2588169901 0 0) 0x7f304c15e8d0 con 0x7f304c0acda0 2024-04-25T09:02:18.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.895+0000 7f3050edf640 1 --2- 172.21.15.43:0/2124000617 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f304c06f230 0x7f304c0ad230 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f304c06c1d0 tx=0x7f304402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e52354bb5095ec3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:18.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.896+0000 7f304affd640 1 -- 172.21.15.43:0/2124000617 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3044007740 con 0x7f304c06f230 2024-04-25T09:02:18.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.896+0000 7f304affd640 1 -- 172.21.15.43:0/2124000617 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30440078e0 con 0x7f304c06f230 2024-04-25T09:02:18.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.896+0000 7f304affd640 1 -- 172.21.15.43:0/2124000617 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3044007cb0 con 0x7f304c06f230 2024-04-25T09:02:18.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.896+0000 7f3052942640 1 -- 172.21.15.43:0/2124000617 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f304c06f230 msgr2=0x7f304c0ad230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:18.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.896+0000 7f3052942640 1 --2- 172.21.15.43:0/2124000617 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f304c06f230 0x7f304c0ad230 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f304c06c1d0 tx=0x7f304402a680 comp rx=0 tx=0).stop 2024-04-25T09:02:18.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.896+0000 7f3052942640 1 -- 172.21.15.43:0/2124000617 shutdown_connections 2024-04-25T09:02:18.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.896+0000 7f3052942640 1 --2- 172.21.15.43:0/2124000617 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f304c06f230 0x7f304c0ad230 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:18.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.896+0000 7f3052942640 1 --2- 172.21.15.43:0/2124000617 >> v2:172.21.15.193:3300/0 conn(0x7f304c1618b0 0x7f304c15e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:18.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.896+0000 7f3052942640 1 -- 172.21.15.43:0/2124000617 >> 172.21.15.43:0/2124000617 conn(0x7f304c0d0260 msgr2=0x7f304c0d0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:18.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.896+0000 7f3052942640 1 -- 172.21.15.43:0/2124000617 shutdown_connections 2024-04-25T09:02:18.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.896+0000 7f3052942640 1 -- 172.21.15.43:0/2124000617 wait complete. 2024-04-25T09:02:18.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.897+0000 7f3052942640 1 Processor -- start 2024-04-25T09:02:18.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.898+0000 7f3052942640 1 -- start start 2024-04-25T09:02:18.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.898+0000 7f3052942640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f304c06f230 0x7f304c155a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:18.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.898+0000 7f3052942640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f304c0acda0 0x7f304c0b3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:18.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.898+0000 7f3052942640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f304c1618b0 0x7f304c0b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:18.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.898+0000 7f3052942640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c161e10 con 0x7f304c0acda0 2024-04-25T09:02:18.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.898+0000 7f3052942640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c161c90 con 0x7f304c06f230 2024-04-25T09:02:18.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.898+0000 7f3052942640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f304c161f90 con 0x7f304c1618b0 2024-04-25T09:02:18.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.898+0000 7f304b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f304c0acda0 0x7f304c0b3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:18.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.898+0000 7f304b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f304c0acda0 0x7f304c0b3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51388/0 (socket says 172.21.15.43:51388) 2024-04-25T09:02:18.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.898+0000 7f304b7fe640 1 -- 172.21.15.43:0/2687494710 learned_addr learned my addr 172.21.15.43:0/2687494710 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:18.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.898+0000 7f304bfff640 1 --2- 172.21.15.43:0/2687494710 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f304c06f230 0x7f304c155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:18.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.899+0000 7f3050edf640 1 --2- 172.21.15.43:0/2687494710 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f304c1618b0 0x7f304c0b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:18.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.899+0000 7f304b7fe640 1 -- 172.21.15.43:0/2687494710 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f304c1618b0 msgr2=0x7f304c0b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:18.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.899+0000 7f304b7fe640 1 --2- 172.21.15.43:0/2687494710 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f304c1618b0 0x7f304c0b40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:18.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.899+0000 7f304b7fe640 1 -- 172.21.15.43:0/2687494710 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f304c06f230 msgr2=0x7f304c155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:18.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.899+0000 7f304b7fe640 1 --2- 172.21.15.43:0/2687494710 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f304c06f230 0x7f304c155a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:18.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.899+0000 7f304b7fe640 1 -- 172.21.15.43:0/2687494710 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f304c09af30 con 0x7f304c0acda0 2024-04-25T09:02:18.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.899+0000 7f3050edf640 1 --2- 172.21.15.43:0/2687494710 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f304c1618b0 0x7f304c0b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:18.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.899+0000 7f304b7fe640 1 --2- 172.21.15.43:0/2687494710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f304c0acda0 0x7f304c0b3b90 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f304000c4d0 tx=0x7f304000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:18.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.899+0000 7f3048ff9640 1 -- 172.21.15.43:0/2687494710 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3040018070 con 0x7f304c0acda0 2024-04-25T09:02:18.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.899+0000 7f3048ff9640 1 -- 172.21.15.43:0/2687494710 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3040007500 con 0x7f304c0acda0 2024-04-25T09:02:18.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.899+0000 7f3052942640 1 -- 172.21.15.43:0/2687494710 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f304c09b210 con 0x7f304c0acda0 2024-04-25T09:02:18.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.899+0000 7f3048ff9640 1 -- 172.21.15.43:0/2687494710 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3040004a20 con 0x7f304c0acda0 2024-04-25T09:02:18.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.899+0000 7f3052942640 1 -- 172.21.15.43:0/2687494710 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f304c09b7d0 con 0x7f304c0acda0 2024-04-25T09:02:18.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.900+0000 7f3052942640 1 -- 172.21.15.43:0/2687494710 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f304c0aeab0 con 0x7f304c0acda0 2024-04-25T09:02:18.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.900+0000 7f3048ff9640 1 -- 172.21.15.43:0/2687494710 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f304001b070 con 0x7f304c0acda0 2024-04-25T09:02:18.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.901+0000 7f3048ff9640 1 --2- 172.21.15.43:0/2687494710 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f303806b4d0 0x7f303806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:18.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.901+0000 7f3048ff9640 1 -- 172.21.15.43:0/2687494710 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f30400891a0 con 0x7f304c0acda0 2024-04-25T09:02:18.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.901+0000 7f304bfff640 1 --2- 172.21.15.43:0/2687494710 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f303806b4d0 0x7f303806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:18.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.902+0000 7f304bfff640 1 --2- 172.21.15.43:0/2687494710 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f303806b4d0 0x7f303806d990 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f303c006fd0 tx=0x7f303c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:18.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:18.905+0000 7f3048ff9640 1 -- 172.21.15.43:0/2687494710 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f304c0aeab0 con 0x7f304c0acda0 2024-04-25T09:02:19.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.100+0000 7f3052942640 1 -- 172.21.15.43:0/2687494710 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f304c074f60 con 0x7f304c0acda0 2024-04-25T09:02:19.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.109+0000 7f3048ff9640 1 -- 172.21.15.43:0/2687494710 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3040057cd0 con 0x7f304c0acda0 2024-04-25T09:02:19.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.113+0000 7f3052942640 1 -- 172.21.15.43:0/2687494710 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f303806b4d0 msgr2=0x7f303806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:19.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.113+0000 7f3052942640 1 --2- 172.21.15.43:0/2687494710 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f303806b4d0 0x7f303806d990 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f303c006fd0 tx=0x7f303c008040 comp rx=0 tx=0).stop 2024-04-25T09:02:19.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.113+0000 7f3052942640 1 -- 172.21.15.43:0/2687494710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f304c0acda0 msgr2=0x7f304c0b3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:19.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.113+0000 7f3052942640 1 --2- 172.21.15.43:0/2687494710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f304c0acda0 0x7f304c0b3b90 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f304000c4d0 tx=0x7f304000c990 comp rx=0 tx=0).stop 2024-04-25T09:02:19.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.113+0000 7f3052942640 1 -- 172.21.15.43:0/2687494710 shutdown_connections 2024-04-25T09:02:19.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.113+0000 7f3052942640 1 --2- 172.21.15.43:0/2687494710 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f304c1618b0 0x7f304c0b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:19.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.113+0000 7f3052942640 1 --2- 172.21.15.43:0/2687494710 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f304c0acda0 0x7f304c0b3b90 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:19.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.113+0000 7f3052942640 1 --2- 172.21.15.43:0/2687494710 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f303806b4d0 0x7f303806d990 secure :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f303c006fd0 tx=0x7f303c008040 comp rx=0 tx=0).stop 2024-04-25T09:02:19.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.113+0000 7f3052942640 1 --2- 172.21.15.43:0/2687494710 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f304c06f230 0x7f304c155a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:19.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.113+0000 7f3052942640 1 -- 172.21.15.43:0/2687494710 >> 172.21.15.43:0/2687494710 conn(0x7f304c0d0260 msgr2=0x7f304c0b0420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:19.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.114+0000 7f3052942640 1 -- 172.21.15.43:0/2687494710 shutdown_connections 2024-04-25T09:02:19.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.115+0000 7f3052942640 1 -- 172.21.15.43:0/2687494710 wait complete. 2024-04-25T09:02:19.123 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume' 2024-04-25T09:02:19.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.235+0000 7f888a934640 1 Processor -- start 2024-04-25T09:02:19.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.235+0000 7f888a934640 1 -- start start 2024-04-25T09:02:19.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.236+0000 7f888a934640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f888415df50 0x7f88841619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:19.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.236+0000 7f888a934640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8884161f30 0x7f8884162380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:19.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.236+0000 7f888a934640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88840d24a0 con 0x7f88840c7cf0 2024-04-25T09:02:19.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.236+0000 7f888a934640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f888415e330 con 0x7f8884161f30 2024-04-25T09:02:19.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.236+0000 7f888a934640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88840d2740 con 0x7f888415df50 2024-04-25T09:02:19.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.236+0000 7f88837fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8884161f30 0x7f8884162380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:19.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.236+0000 7f8888ed1640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f88840c7cf0 0x7f88840c8100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:32940/0 (socket says 172.21.15.43:32940) 2024-04-25T09:02:19.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.236+0000 7f8888ed1640 1 -- 172.21.15.43:0/2971651531 learned_addr learned my addr 172.21.15.43:0/2971651531 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:19.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.236+0000 7f8883fff640 1 --2- 172.21.15.43:0/2971651531 >> v2:172.21.15.193:3300/0 conn(0x7f888415df50 0x7f88841619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:19.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.236+0000 7f8882ffd640 1 -- 172.21.15.43:0/2971651531 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 970325068 0 0) 0x7f88840d24a0 con 0x7f88840c7cf0 2024-04-25T09:02:19.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.236+0000 7f8882ffd640 1 -- 172.21.15.43:0/2971651531 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8870003520 con 0x7f88840c7cf0 2024-04-25T09:02:19.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.236+0000 7f88837fe640 1 -- 172.21.15.43:0/2971651531 >> v2:172.21.15.193:3300/0 conn(0x7f888415df50 msgr2=0x7f88841619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:19.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.236+0000 7f88837fe640 1 --2- 172.21.15.43:0/2971651531 >> v2:172.21.15.193:3300/0 conn(0x7f888415df50 0x7f88841619f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:19.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.236+0000 7f88837fe640 1 -- 172.21.15.43:0/2971651531 >> v1:172.21.15.43:6789/0 conn(0x7f88840c7cf0 legacy=0x7f88840c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:19.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.236+0000 7f88837fe640 1 -- 172.21.15.43:0/2971651531 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88840d2a20 con 0x7f8884161f30 2024-04-25T09:02:19.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.237+0000 7f88837fe640 1 --2- 172.21.15.43:0/2971651531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8884161f30 0x7f8884162380 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f888408c880 tx=0x7f886c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ddf3d30ded92a0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:19.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.237+0000 7f8882ffd640 1 -- 172.21.15.43:0/2971651531 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c037070 con 0x7f8884161f30 2024-04-25T09:02:19.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.237+0000 7f8882ffd640 1 -- 172.21.15.43:0/2971651531 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886c032440 con 0x7f8884161f30 2024-04-25T09:02:19.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.237+0000 7f8882ffd640 1 -- 172.21.15.43:0/2971651531 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c032810 con 0x7f8884161f30 2024-04-25T09:02:19.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.238+0000 7f888a934640 1 -- 172.21.15.43:0/2971651531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8884161f30 msgr2=0x7f8884162380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:19.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.238+0000 7f888a934640 1 --2- 172.21.15.43:0/2971651531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8884161f30 0x7f8884162380 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f888408c880 tx=0x7f886c02a1a0 comp rx=0 tx=0).stop 2024-04-25T09:02:19.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.238+0000 7f888a934640 1 -- 172.21.15.43:0/2971651531 shutdown_connections 2024-04-25T09:02:19.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.238+0000 7f888a934640 1 --2- 172.21.15.43:0/2971651531 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8884161f30 0x7f8884162380 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:19.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.238+0000 7f888a934640 1 --2- 172.21.15.43:0/2971651531 >> v2:172.21.15.193:3300/0 conn(0x7f888415df50 0x7f88841619f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:19.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.238+0000 7f888a934640 1 -- 172.21.15.43:0/2971651531 >> 172.21.15.43:0/2971651531 conn(0x7f88840d3a30 msgr2=0x7f88840d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.238+0000 7f888a934640 1 -- 172.21.15.43:0/2971651531 shutdown_connections 2024-04-25T09:02:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.238+0000 7f888a934640 1 -- 172.21.15.43:0/2971651531 wait complete. 2024-04-25T09:02:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.239+0000 7f888a934640 1 Processor -- start 2024-04-25T09:02:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.239+0000 7f888a934640 1 -- start start 2024-04-25T09:02:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.240+0000 7f888a934640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88840c7cf0 0x7f8884159280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.240+0000 7f888a934640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f888415df50 0x7f8884161630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.240+0000 7f888a934640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8884161f30 0x7f888414ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.240+0000 7f888a934640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88840d2190 con 0x7f88840c7cf0 2024-04-25T09:02:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.240+0000 7f888a934640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88840c77d0 con 0x7f888415df50 2024-04-25T09:02:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.240+0000 7f888a934640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88840cf670 con 0x7f8884161f30 2024-04-25T09:02:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.240+0000 7f8883fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88840c7cf0 0x7f8884159280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.240+0000 7f8883fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88840c7cf0 0x7f8884159280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51418/0 (socket says 172.21.15.43:51418) 2024-04-25T09:02:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.240+0000 7f8883fff640 1 -- 172.21.15.43:0/3717304033 learned_addr learned my addr 172.21.15.43:0/3717304033 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.240+0000 7f8888ed1640 1 --2- 172.21.15.43:0/3717304033 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8884161f30 0x7f888414ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:19.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.240+0000 7f88837fe640 1 --2- 172.21.15.43:0/3717304033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f888415df50 0x7f8884161630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:19.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.240+0000 7f8883fff640 1 -- 172.21.15.43:0/3717304033 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8884161f30 msgr2=0x7f888414ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:19.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.240+0000 7f8883fff640 1 --2- 172.21.15.43:0/3717304033 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8884161f30 0x7f888414ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:19.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.240+0000 7f8883fff640 1 -- 172.21.15.43:0/3717304033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f888415df50 msgr2=0x7f8884161630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:19.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.240+0000 7f8883fff640 1 --2- 172.21.15.43:0/3717304033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f888415df50 0x7f8884161630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:19.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.240+0000 7f8883fff640 1 -- 172.21.15.43:0/3717304033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88840caee0 con 0x7f88840c7cf0 2024-04-25T09:02:19.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.240+0000 7f88837fe640 1 --2- 172.21.15.43:0/3717304033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f888415df50 0x7f8884161630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:19.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.240+0000 7f8883fff640 1 --2- 172.21.15.43:0/3717304033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88840c7cf0 0x7f8884159280 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f887400d830 tx=0x7f887400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:19.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.241+0000 7f8880ff9640 1 -- 172.21.15.43:0/3717304033 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887400b6c0 con 0x7f88840c7cf0 2024-04-25T09:02:19.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.241+0000 7f8880ff9640 1 -- 172.21.15.43:0/3717304033 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8874019070 con 0x7f88840c7cf0 2024-04-25T09:02:19.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.241+0000 7f888a934640 1 -- 172.21.15.43:0/3717304033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88840cb1c0 con 0x7f88840c7cf0 2024-04-25T09:02:19.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.241+0000 7f8880ff9640 1 -- 172.21.15.43:0/3717304033 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887400ba80 con 0x7f88840c7cf0 2024-04-25T09:02:19.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.241+0000 7f888a934640 1 -- 172.21.15.43:0/3717304033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88840cb780 con 0x7f88840c7cf0 2024-04-25T09:02:19.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.242+0000 7f888a934640 1 -- 172.21.15.43:0/3717304033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88840680f0 con 0x7f88840c7cf0 2024-04-25T09:02:19.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.242+0000 7f8880ff9640 1 -- 172.21.15.43:0/3717304033 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f887401e460 con 0x7f88840c7cf0 2024-04-25T09:02:19.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.243+0000 7f8880ff9640 1 --2- 172.21.15.43:0/3717304033 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f886406b500 0x7f886406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:19.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.243+0000 7f8880ff9640 1 -- 172.21.15.43:0/3717304033 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f887408a530 con 0x7f88840c7cf0 2024-04-25T09:02:19.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.243+0000 7f88837fe640 1 --2- 172.21.15.43:0/3717304033 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f886406b500 0x7f886406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:19.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.243+0000 7f88837fe640 1 --2- 172.21.15.43:0/3717304033 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f886406b500 0x7f886406d9c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f886c034000 tx=0x7f886c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:19.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.247+0000 7f8880ff9640 1 -- 172.21.15.43:0/3717304033 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8874059740 con 0x7f88840c7cf0 2024-04-25T09:02:19.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.505+0000 7f888a934640 1 -- 172.21.15.43:0/3717304033 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]} v 0) v1 -- 0x7f8884074ef0 con 0x7f88840c7cf0 2024-04-25T09:02:19.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.761+0000 7f8880ff9640 1 -- 172.21.15.43:0/3717304033 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]}]=0 v391) v1 ==== 155+0+0 (secure 0 0 0) 0x7f8874059040 con 0x7f88840c7cf0 2024-04-25T09:02:19.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.763+0000 7f888a934640 1 -- 172.21.15.43:0/3717304033 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f886406b500 msgr2=0x7f886406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:19.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.763+0000 7f888a934640 1 --2- 172.21.15.43:0/3717304033 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f886406b500 0x7f886406d9c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f886c034000 tx=0x7f886c02e000 comp rx=0 tx=0).stop 2024-04-25T09:02:19.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.763+0000 7f888a934640 1 -- 172.21.15.43:0/3717304033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88840c7cf0 msgr2=0x7f8884159280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:19.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.763+0000 7f888a934640 1 --2- 172.21.15.43:0/3717304033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88840c7cf0 0x7f8884159280 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f887400d830 tx=0x7f887400dcf0 comp rx=0 tx=0).stop 2024-04-25T09:02:19.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.763+0000 7f888a934640 1 -- 172.21.15.43:0/3717304033 shutdown_connections 2024-04-25T09:02:19.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.763+0000 7f888a934640 1 --2- 172.21.15.43:0/3717304033 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8884161f30 0x7f888414ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:19.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.763+0000 7f888a934640 1 --2- 172.21.15.43:0/3717304033 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f886406b500 0x7f886406d9c0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:19.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.763+0000 7f888a934640 1 --2- 172.21.15.43:0/3717304033 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f888415df50 0x7f8884161630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:19.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.763+0000 7f888a934640 1 --2- 172.21.15.43:0/3717304033 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f88840c7cf0 0x7f8884159280 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:19.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.763+0000 7f888a934640 1 -- 172.21.15.43:0/3717304033 >> 172.21.15.43:0/3717304033 conn(0x7f88840d3a30 msgr2=0x7f8884156030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:19.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.763+0000 7f888a934640 1 -- 172.21.15.43:0/3717304033 shutdown_connections 2024-04-25T09:02:19.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.763+0000 7f888a934640 1 -- 172.21.15.43:0/3717304033 wait complete. 2024-04-25T09:02:19.763 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T09:02:19.763 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume_with_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T09:02:19.764 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:02:19.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.876+0000 7fc336093640 1 Processor -- start 2024-04-25T09:02:19.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.876+0000 7fc336093640 1 -- start start 2024-04-25T09:02:19.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.877+0000 7fc336093640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc33015dce0 0x7fc33015ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:19.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.877+0000 7fc336093640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3300d2920 con 0x7fc3300d0af0 2024-04-25T09:02:19.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.877+0000 7fc336093640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3300cee80 con 0x7fc3300ce550 2024-04-25T09:02:19.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.877+0000 7fc336093640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3300cf160 con 0x7fc33015dce0 2024-04-25T09:02:19.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.877+0000 7fc32f7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc33015dce0 0x7fc33015ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:19.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.877+0000 7fc32ffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc3300d0af0 0x7fc330160980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:33850/0 (socket says 172.21.15.43:33850) 2024-04-25T09:02:19.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.877+0000 7fc32ffff640 1 -- 172.21.15.43:0/4181235321 learned_addr learned my addr 172.21.15.43:0/4181235321 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:19.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.878+0000 7fc32f7fe640 1 -- 172.21.15.43:0/4181235321 >> v1:172.21.15.193:6789/0 conn(0x7fc3300ce550 legacy=0x7fc3300d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:19.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.878+0000 7fc32f7fe640 1 -- 172.21.15.43:0/4181235321 >> v1:172.21.15.43:6789/0 conn(0x7fc3300d0af0 legacy=0x7fc330160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:19.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.878+0000 7fc32f7fe640 1 -- 172.21.15.43:0/4181235321 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3300cf440 con 0x7fc33015dce0 2024-04-25T09:02:19.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.878+0000 7fc32e7fc640 1 -- 172.21.15.43:0/4181235321 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2403077575 0 0) 0x7fc3300d2920 con 0x7fc3300d0af0 2024-04-25T09:02:19.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.878+0000 7fc32f7fe640 1 --2- 172.21.15.43:0/4181235321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc33015dce0 0x7fc33015ef50 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fc3200097a0 tx=0x7fc32002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8401de7d9d66dbba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:19.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.878+0000 7fc32e7fc640 1 -- 172.21.15.43:0/4181235321 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc320037070 con 0x7fc33015dce0 2024-04-25T09:02:19.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.878+0000 7fc32e7fc640 1 -- 172.21.15.43:0/4181235321 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc32002ae50 con 0x7fc33015dce0 2024-04-25T09:02:19.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.879+0000 7fc32e7fc640 1 -- 172.21.15.43:0/4181235321 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc320032660 con 0x7fc33015dce0 2024-04-25T09:02:19.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.879+0000 7fc336093640 1 -- 172.21.15.43:0/4181235321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc33015dce0 msgr2=0x7fc33015ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:19.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.879+0000 7fc336093640 1 --2- 172.21.15.43:0/4181235321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc33015dce0 0x7fc33015ef50 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fc3200097a0 tx=0x7fc32002a140 comp rx=0 tx=0).stop 2024-04-25T09:02:19.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.879+0000 7fc336093640 1 -- 172.21.15.43:0/4181235321 shutdown_connections 2024-04-25T09:02:19.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.879+0000 7fc336093640 1 --2- 172.21.15.43:0/4181235321 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc33015dce0 0x7fc33015ef50 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:19.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.879+0000 7fc336093640 1 -- 172.21.15.43:0/4181235321 >> 172.21.15.43:0/4181235321 conn(0x7fc33009cd00 msgr2=0x7fc3300d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:19.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.879+0000 7fc336093640 1 -- 172.21.15.43:0/4181235321 shutdown_connections 2024-04-25T09:02:19.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.879+0000 7fc336093640 1 -- 172.21.15.43:0/4181235321 wait complete. 2024-04-25T09:02:19.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.880+0000 7fc336093640 1 Processor -- start 2024-04-25T09:02:19.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.880+0000 7fc336093640 1 -- start start 2024-04-25T09:02:19.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.881+0000 7fc336093640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3300ce550 0x7fc330154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:19.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.881+0000 7fc336093640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc3300d0af0 0x7fc3300cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:19.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.881+0000 7fc336093640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc33015dce0 0x7fc33009f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:19.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.881+0000 7fc336093640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc33015e0c0 con 0x7fc3300ce550 2024-04-25T09:02:19.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.881+0000 7fc336093640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc33015f6b0 con 0x7fc33015dce0 2024-04-25T09:02:19.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.881+0000 7fc336093640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3300ce9c0 con 0x7fc3300d0af0 2024-04-25T09:02:19.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.881+0000 7fc32f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3300ce550 0x7fc330154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:19.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.881+0000 7fc32f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3300ce550 0x7fc330154e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44758/0 (socket says 172.21.15.43:44758) 2024-04-25T09:02:19.872 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.881+0000 7fc32f7fe640 1 -- 172.21.15.43:0/3882163020 learned_addr learned my addr 172.21.15.43:0/3882163020 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:19.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.882+0000 7fc32ffff640 1 --2- 172.21.15.43:0/3882163020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc33015dce0 0x7fc33009f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:19.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.882+0000 7fc32effd640 1 --2- 172.21.15.43:0/3882163020 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc3300d0af0 0x7fc3300cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:19.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.882+0000 7fc32f7fe640 1 -- 172.21.15.43:0/3882163020 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc3300d0af0 msgr2=0x7fc3300cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:19.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.882+0000 7fc32f7fe640 1 --2- 172.21.15.43:0/3882163020 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc3300d0af0 0x7fc3300cbde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:19.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.882+0000 7fc32f7fe640 1 -- 172.21.15.43:0/3882163020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc33015dce0 msgr2=0x7fc33009f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:19.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.882+0000 7fc32f7fe640 1 --2- 172.21.15.43:0/3882163020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc33015dce0 0x7fc33009f3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:19.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.882+0000 7fc32f7fe640 1 -- 172.21.15.43:0/3882163020 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3300cdef0 con 0x7fc3300ce550 2024-04-25T09:02:19.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.882+0000 7fc32f7fe640 1 --2- 172.21.15.43:0/3882163020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3300ce550 0x7fc330154e10 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fc320000c00 tx=0x7fc3200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:19.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.882+0000 7fc317fff640 1 -- 172.21.15.43:0/3882163020 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc320037040 con 0x7fc3300ce550 2024-04-25T09:02:19.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.882+0000 7fc317fff640 1 -- 172.21.15.43:0/3882163020 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc320004030 con 0x7fc3300ce550 2024-04-25T09:02:19.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.882+0000 7fc336093640 1 -- 172.21.15.43:0/3882163020 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3300cd930 con 0x7fc3300ce550 2024-04-25T09:02:19.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.882+0000 7fc317fff640 1 -- 172.21.15.43:0/3882163020 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc320006e10 con 0x7fc3300ce550 2024-04-25T09:02:19.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.882+0000 7fc336093640 1 -- 172.21.15.43:0/3882163020 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3300cb5e0 con 0x7fc3300ce550 2024-04-25T09:02:19.873 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.883+0000 7fc336093640 1 -- 172.21.15.43:0/3882163020 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3300680f0 con 0x7fc3300ce550 2024-04-25T09:02:19.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.883+0000 7fc317fff640 1 -- 172.21.15.43:0/3882163020 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc32003d650 con 0x7fc3300ce550 2024-04-25T09:02:19.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.884+0000 7fc317fff640 1 --2- 172.21.15.43:0/3882163020 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc31c06b4d0 0x7fc31c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:19.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.884+0000 7fc317fff640 1 -- 172.21.15.43:0/3882163020 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc3200aa720 con 0x7fc3300ce550 2024-04-25T09:02:19.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.884+0000 7fc32effd640 1 --2- 172.21.15.43:0/3882163020 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc31c06b4d0 0x7fc31c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:19.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.885+0000 7fc32effd640 1 --2- 172.21.15.43:0/3882163020 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc31c06b4d0 0x7fc31c06d990 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc3300b5040 tx=0x7fc324009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:19.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:19.888+0000 7fc317fff640 1 -- 172.21.15.43:0/3882163020 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc320042020 con 0x7fc3300ce550 2024-04-25T09:02:20.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.071+0000 7fc336093640 1 -- 172.21.15.43:0/3882163020 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc330074ef0 con 0x7fc3300ce550 2024-04-25T09:02:20.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.071+0000 7fc317fff640 1 -- 172.21.15.43:0/3882163020 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+12994 (secure 0 0 0) 0x7fc320079940 con 0x7fc3300ce550 2024-04-25T09:02:20.061 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:20.061 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":105,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:02:18.756263+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.193:6836/853943267":"2024-04-26T09:02:15.743850+0000","172.21.15.193:6834/853943267":"2024-04-26T09:02:15.743850+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:02:20.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.073+0000 7fc336093640 1 -- 172.21.15.43:0/3882163020 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc31c06b4d0 msgr2=0x7fc31c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:20.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.073+0000 7fc336093640 1 --2- 172.21.15.43:0/3882163020 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc31c06b4d0 0x7fc31c06d990 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc3300b5040 tx=0x7fc324009040 comp rx=0 tx=0).stop 2024-04-25T09:02:20.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.073+0000 7fc336093640 1 -- 172.21.15.43:0/3882163020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3300ce550 msgr2=0x7fc330154e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:20.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.073+0000 7fc336093640 1 --2- 172.21.15.43:0/3882163020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3300ce550 0x7fc330154e10 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fc320000c00 tx=0x7fc3200046c0 comp rx=0 tx=0).stop 2024-04-25T09:02:20.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.073+0000 7fc336093640 1 -- 172.21.15.43:0/3882163020 shutdown_connections 2024-04-25T09:02:20.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.073+0000 7fc336093640 1 --2- 172.21.15.43:0/3882163020 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc31c06b4d0 0x7fc31c06d990 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:20.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.073+0000 7fc336093640 1 --2- 172.21.15.43:0/3882163020 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc33015dce0 0x7fc33009f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:20.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.073+0000 7fc336093640 1 --2- 172.21.15.43:0/3882163020 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc3300d0af0 0x7fc3300cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:20.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.073+0000 7fc336093640 1 --2- 172.21.15.43:0/3882163020 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc3300ce550 0x7fc330154e10 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:20.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.073+0000 7fc336093640 1 -- 172.21.15.43:0/3882163020 >> 172.21.15.43:0/3882163020 conn(0x7fc33009cd00 msgr2=0x7fc33006c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:20.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.073+0000 7fc336093640 1 -- 172.21.15.43:0/3882163020 shutdown_connections 2024-04-25T09:02:20.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.073+0000 7fc336093640 1 -- 172.21.15.43:0/3882163020 wait complete. 2024-04-25T09:02:20.074 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:02:20.074 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:02:20.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.189+0000 7fbf6e76a640 1 Processor -- start 2024-04-25T09:02:20.179 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.189+0000 7fbf6e76a640 1 -- start start 2024-04-25T09:02:20.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.190+0000 7fbf6e76a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbf6806f6a0 0x7fbf680b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:20.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.190+0000 7fbf6e76a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf680cd470 0x7fbf6806c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:20.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.190+0000 7fbf6e76a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf680d2620 con 0x7fbf6815ddd0 2024-04-25T09:02:20.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.190+0000 7fbf6e76a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf680b0b80 con 0x7fbf6806f6a0 2024-04-25T09:02:20.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.190+0000 7fbf6e76a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf680b0e60 con 0x7fbf680cd470 2024-04-25T09:02:20.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.190+0000 7fbf677fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbf6806f6a0 0x7fbf680b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:20.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.190+0000 7fbf67fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fbf6815ddd0 0x7fbf6806ef90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:33856/0 (socket says 172.21.15.43:33856) 2024-04-25T09:02:20.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.190+0000 7fbf67fff640 1 -- 172.21.15.43:0/973076427 learned_addr learned my addr 172.21.15.43:0/973076427 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:20.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.191+0000 7fbf6cd07640 1 --2- 172.21.15.43:0/973076427 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf680cd470 0x7fbf6806c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:20.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.191+0000 7fbf677fe640 1 -- 172.21.15.43:0/973076427 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf680cd470 msgr2=0x7fbf6806c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:20.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.191+0000 7fbf677fe640 1 --2- 172.21.15.43:0/973076427 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf680cd470 0x7fbf6806c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:20.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.191+0000 7fbf677fe640 1 -- 172.21.15.43:0/973076427 >> v1:172.21.15.43:6789/0 conn(0x7fbf6815ddd0 legacy=0x7fbf6806ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:20.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.191+0000 7fbf677fe640 1 -- 172.21.15.43:0/973076427 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf680cbf40 con 0x7fbf6806f6a0 2024-04-25T09:02:20.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.191+0000 7fbf66ffd640 1 -- 172.21.15.43:0/973076427 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1231481301 0 0) 0x7fbf680d2620 con 0x7fbf6815ddd0 2024-04-25T09:02:20.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.191+0000 7fbf6cd07640 1 --2- 172.21.15.43:0/973076427 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf680cd470 0x7fbf6806c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:20.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.191+0000 7fbf677fe640 1 --2- 172.21.15.43:0/973076427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf6806f6a0 0x7fbf680b3dd0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fbf580097a0 tx=0x7fbf5802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c738573b6c81de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:20.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.192+0000 7fbf66ffd640 1 -- 172.21.15.43:0/973076427 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf58037070 con 0x7fbf6806f6a0 2024-04-25T09:02:20.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.192+0000 7fbf66ffd640 1 -- 172.21.15.43:0/973076427 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf58032440 con 0x7fbf6806f6a0 2024-04-25T09:02:20.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.192+0000 7fbf66ffd640 1 -- 172.21.15.43:0/973076427 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf58032810 con 0x7fbf6806f6a0 2024-04-25T09:02:20.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.192+0000 7fbf6e76a640 1 -- 172.21.15.43:0/973076427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf6806f6a0 msgr2=0x7fbf680b3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:20.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.192+0000 7fbf6e76a640 1 --2- 172.21.15.43:0/973076427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf6806f6a0 0x7fbf680b3dd0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fbf580097a0 tx=0x7fbf5802a180 comp rx=0 tx=0).stop 2024-04-25T09:02:20.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.192+0000 7fbf6e76a640 1 -- 172.21.15.43:0/973076427 shutdown_connections 2024-04-25T09:02:20.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.192+0000 7fbf6e76a640 1 --2- 172.21.15.43:0/973076427 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf680cd470 0x7fbf6806c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:20.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.192+0000 7fbf6e76a640 1 --2- 172.21.15.43:0/973076427 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf6806f6a0 0x7fbf680b3dd0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:20.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.192+0000 7fbf6e76a640 1 -- 172.21.15.43:0/973076427 >> 172.21.15.43:0/973076427 conn(0x7fbf680d17e0 msgr2=0x7fbf680d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:20.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.192+0000 7fbf6e76a640 1 -- 172.21.15.43:0/973076427 shutdown_connections 2024-04-25T09:02:20.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.192+0000 7fbf6e76a640 1 -- 172.21.15.43:0/973076427 wait complete. 2024-04-25T09:02:20.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.193+0000 7fbf6e76a640 1 Processor -- start 2024-04-25T09:02:20.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.194+0000 7fbf6e76a640 1 -- start start 2024-04-25T09:02:20.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.194+0000 7fbf6e76a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf6806f6a0 0x7fbf681e7650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:20.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.194+0000 7fbf6e76a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf680cd470 0x7fbf68157800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:20.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.194+0000 7fbf6e76a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf6815ddd0 0x7fbf680b1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:20.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.194+0000 7fbf6e76a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6806cdb0 con 0x7fbf680cd470 2024-04-25T09:02:20.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.194+0000 7fbf6e76a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6806cc30 con 0x7fbf6806f6a0 2024-04-25T09:02:20.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.194+0000 7fbf6e76a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6806cf30 con 0x7fbf6815ddd0 2024-04-25T09:02:20.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.194+0000 7fbf677fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf680cd470 0x7fbf68157800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:20.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.194+0000 7fbf677fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf680cd470 0x7fbf68157800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44770/0 (socket says 172.21.15.43:44770) 2024-04-25T09:02:20.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.194+0000 7fbf677fe640 1 -- 172.21.15.43:0/3505801613 learned_addr learned my addr 172.21.15.43:0/3505801613 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:20.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.195+0000 7fbf6cd07640 1 --2- 172.21.15.43:0/3505801613 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf6815ddd0 0x7fbf680b1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:20.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.195+0000 7fbf67fff640 1 --2- 172.21.15.43:0/3505801613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf6806f6a0 0x7fbf681e7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:20.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.195+0000 7fbf677fe640 1 -- 172.21.15.43:0/3505801613 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf6815ddd0 msgr2=0x7fbf680b1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:20.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.195+0000 7fbf677fe640 1 --2- 172.21.15.43:0/3505801613 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf6815ddd0 0x7fbf680b1bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:20.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.195+0000 7fbf677fe640 1 -- 172.21.15.43:0/3505801613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf6806f6a0 msgr2=0x7fbf681e7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:20.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.195+0000 7fbf677fe640 1 --2- 172.21.15.43:0/3505801613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf6806f6a0 0x7fbf681e7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:20.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.195+0000 7fbf677fe640 1 -- 172.21.15.43:0/3505801613 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf681e7d80 con 0x7fbf680cd470 2024-04-25T09:02:20.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.195+0000 7fbf6cd07640 1 --2- 172.21.15.43:0/3505801613 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf6815ddd0 0x7fbf680b1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:20.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.195+0000 7fbf67fff640 1 --2- 172.21.15.43:0/3505801613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf6806f6a0 0x7fbf681e7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:20.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.195+0000 7fbf677fe640 1 --2- 172.21.15.43:0/3505801613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf680cd470 0x7fbf68157800 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fbf580097a0 tx=0x7fbf580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:20.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.195+0000 7fbf64ff9640 1 -- 172.21.15.43:0/3505801613 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf58037050 con 0x7fbf680cd470 2024-04-25T09:02:20.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.195+0000 7fbf64ff9640 1 -- 172.21.15.43:0/3505801613 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf58038040 con 0x7fbf680cd470 2024-04-25T09:02:20.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.195+0000 7fbf6e76a640 1 -- 172.21.15.43:0/3505801613 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf681e8060 con 0x7fbf680cd470 2024-04-25T09:02:20.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.195+0000 7fbf64ff9640 1 -- 172.21.15.43:0/3505801613 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf58004490 con 0x7fbf680cd470 2024-04-25T09:02:20.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.195+0000 7fbf6e76a640 1 -- 172.21.15.43:0/3505801613 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf681e8a00 con 0x7fbf680cd470 2024-04-25T09:02:20.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.196+0000 7fbf6e76a640 1 -- 172.21.15.43:0/3505801613 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf681e87f0 con 0x7fbf680cd470 2024-04-25T09:02:20.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.197+0000 7fbf64ff9640 1 -- 172.21.15.43:0/3505801613 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbf58030980 con 0x7fbf680cd470 2024-04-25T09:02:20.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.197+0000 7fbf64ff9640 1 --2- 172.21.15.43:0/3505801613 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbf4806b4d0 0x7fbf4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:20.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.197+0000 7fbf64ff9640 1 -- 172.21.15.43:0/3505801613 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbf580aaaf0 con 0x7fbf680cd470 2024-04-25T09:02:20.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.197+0000 7fbf67fff640 1 --2- 172.21.15.43:0/3505801613 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbf4806b4d0 0x7fbf4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:20.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.198+0000 7fbf67fff640 1 --2- 172.21.15.43:0/3505801613 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbf4806b4d0 0x7fbf4806d990 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fbf50000f60 tx=0x7fbf50007ca0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:20.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.201+0000 7fbf64ff9640 1 -- 172.21.15.43:0/3505801613 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbf58079d90 con 0x7fbf680cd470 2024-04-25T09:02:20.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.381+0000 7fbf6e76a640 1 -- 172.21.15.43:0/3505801613 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbf68074f60 con 0x7fbf680cd470 2024-04-25T09:02:20.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.381+0000 7fbf64ff9640 1 -- 172.21.15.43:0/3505801613 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbf58079690 con 0x7fbf680cd470 2024-04-25T09:02:20.370 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T09:02:20.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.383+0000 7fbf6e76a640 1 -- 172.21.15.43:0/3505801613 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbf4806b4d0 msgr2=0x7fbf4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:20.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.383+0000 7fbf6e76a640 1 --2- 172.21.15.43:0/3505801613 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbf4806b4d0 0x7fbf4806d990 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fbf50000f60 tx=0x7fbf50007ca0 comp rx=0 tx=0).stop 2024-04-25T09:02:20.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.383+0000 7fbf6e76a640 1 -- 172.21.15.43:0/3505801613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf680cd470 msgr2=0x7fbf68157800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:20.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.383+0000 7fbf6e76a640 1 --2- 172.21.15.43:0/3505801613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf680cd470 0x7fbf68157800 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fbf580097a0 tx=0x7fbf580046c0 comp rx=0 tx=0).stop 2024-04-25T09:02:20.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.383+0000 7fbf6e76a640 1 -- 172.21.15.43:0/3505801613 shutdown_connections 2024-04-25T09:02:20.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.383+0000 7fbf6e76a640 1 --2- 172.21.15.43:0/3505801613 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf6815ddd0 0x7fbf680b1bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:20.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.383+0000 7fbf6e76a640 1 --2- 172.21.15.43:0/3505801613 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf680cd470 0x7fbf68157800 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:20.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.383+0000 7fbf6e76a640 1 --2- 172.21.15.43:0/3505801613 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbf4806b4d0 0x7fbf4806d990 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:20.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.383+0000 7fbf6e76a640 1 --2- 172.21.15.43:0/3505801613 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf6806f6a0 0x7fbf681e7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:20.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.383+0000 7fbf6e76a640 1 -- 172.21.15.43:0/3505801613 >> 172.21.15.43:0/3505801613 conn(0x7fbf680d17e0 msgr2=0x7fbf680d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:20.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.383+0000 7fbf6e76a640 1 -- 172.21.15.43:0/3505801613 shutdown_connections 2024-04-25T09:02:20.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.383+0000 7fbf6e76a640 1 -- 172.21.15.43:0/3505801613 wait complete. 2024-04-25T09:02:20.383 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort' 2024-04-25T09:02:20.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.496+0000 7f69284fc640 1 Processor -- start 2024-04-25T09:02:20.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.496+0000 7f69284fc640 1 -- start start 2024-04-25T09:02:20.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.497+0000 7f69284fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f69200cbfc0 0x7f692006dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:20.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.497+0000 7f69284fc640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69200ccbb0 0x7f69200cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:20.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.497+0000 7f69284fc640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69200b2780 con 0x7f692015f5e0 2024-04-25T09:02:20.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.497+0000 7f69284fc640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69200cd550 con 0x7f69200cbfc0 2024-04-25T09:02:20.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.497+0000 7f69284fc640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69200b2a60 con 0x7f69200ccbb0 2024-04-25T09:02:20.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.497+0000 7f6926298640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f69200cbfc0 0x7f692006dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:20.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.497+0000 7f6926298640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f69200cbfc0 0x7f692006dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44782/0 (socket says 172.21.15.43:44782) 2024-04-25T09:02:20.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.497+0000 7f6926298640 1 -- 172.21.15.43:0/3733069002 learned_addr learned my addr 172.21.15.43:0/3733069002 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:20.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.497+0000 7f6925a97640 1 --2- 172.21.15.43:0/3733069002 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69200ccbb0 0x7f69200cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:20.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.497+0000 7f6926298640 1 -- 172.21.15.43:0/3733069002 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69200ccbb0 msgr2=0x7f69200cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:20.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.497+0000 7f6926298640 1 --2- 172.21.15.43:0/3733069002 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69200ccbb0 0x7f69200cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:20.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.497+0000 7f6926298640 1 -- 172.21.15.43:0/3733069002 >> v1:172.21.15.193:6789/0 conn(0x7f692015f5e0 legacy=0x7f692015fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:20.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.497+0000 7f6926298640 1 -- 172.21.15.43:0/3733069002 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69200b2d40 con 0x7f69200cbfc0 2024-04-25T09:02:20.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.498+0000 7f6926298640 1 --2- 172.21.15.43:0/3733069002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69200cbfc0 0x7f692006dc60 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f691c009870 tx=0x7f691c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c7bc77c93162fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:20.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.498+0000 7f6925296640 1 -- 172.21.15.43:0/3733069002 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691c037070 con 0x7f69200cbfc0 2024-04-25T09:02:20.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.498+0000 7f6925296640 1 -- 172.21.15.43:0/3733069002 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f691c032440 con 0x7f69200cbfc0 2024-04-25T09:02:20.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.498+0000 7f6925296640 1 -- 172.21.15.43:0/3733069002 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691c032810 con 0x7f69200cbfc0 2024-04-25T09:02:20.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.498+0000 7f69284fc640 1 -- 172.21.15.43:0/3733069002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69200cbfc0 msgr2=0x7f692006dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:20.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.498+0000 7f69284fc640 1 --2- 172.21.15.43:0/3733069002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69200cbfc0 0x7f692006dc60 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f691c009870 tx=0x7f691c02a270 comp rx=0 tx=0).stop 2024-04-25T09:02:20.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.499+0000 7f69284fc640 1 -- 172.21.15.43:0/3733069002 shutdown_connections 2024-04-25T09:02:20.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.499+0000 7f69284fc640 1 --2- 172.21.15.43:0/3733069002 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69200ccbb0 0x7f69200cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:20.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.499+0000 7f69284fc640 1 --2- 172.21.15.43:0/3733069002 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69200cbfc0 0x7f692006dc60 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:20.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.499+0000 7f69284fc640 1 -- 172.21.15.43:0/3733069002 >> 172.21.15.43:0/3733069002 conn(0x7f69200b1480 msgr2=0x7f69200b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:20.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.499+0000 7f69284fc640 1 -- 172.21.15.43:0/3733069002 shutdown_connections 2024-04-25T09:02:20.489 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.499+0000 7f69284fc640 1 -- 172.21.15.43:0/3733069002 wait complete. 2024-04-25T09:02:20.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.500+0000 7f69284fc640 1 Processor -- start 2024-04-25T09:02:20.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.500+0000 7f69284fc640 1 -- start start 2024-04-25T09:02:20.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.500+0000 7f69284fc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69200cbfc0 0x7f692013a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:20.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.500+0000 7f69284fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69200ccbb0 0x7f69200c50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:20.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.500+0000 7f69284fc640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f692015f5e0 0x7f69200c6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:20.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.500+0000 7f69284fc640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69200b35e0 con 0x7f69200ccbb0 2024-04-25T09:02:20.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.500+0000 7f69284fc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69200b3300 con 0x7f69200cbfc0 2024-04-25T09:02:20.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.500+0000 7f69284fc640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f692006e1a0 con 0x7f692015f5e0 2024-04-25T09:02:20.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.501+0000 7f6925a97640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69200ccbb0 0x7f69200c50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:20.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.501+0000 7f6925a97640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69200ccbb0 0x7f69200c50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44792/0 (socket says 172.21.15.43:44792) 2024-04-25T09:02:20.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.501+0000 7f6925a97640 1 -- 172.21.15.43:0/1149275179 learned_addr learned my addr 172.21.15.43:0/1149275179 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:20.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.501+0000 7f6926a99640 1 --2- 172.21.15.43:0/1149275179 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f692015f5e0 0x7f69200c6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:20.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.501+0000 7f6926298640 1 --2- 172.21.15.43:0/1149275179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69200cbfc0 0x7f692013a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:20.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.501+0000 7f6925a97640 1 -- 172.21.15.43:0/1149275179 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f692015f5e0 msgr2=0x7f69200c6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:20.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.501+0000 7f6925a97640 1 --2- 172.21.15.43:0/1149275179 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f692015f5e0 0x7f69200c6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:20.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.501+0000 7f6925a97640 1 -- 172.21.15.43:0/1149275179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69200cbfc0 msgr2=0x7f692013a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:20.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.501+0000 7f6925a97640 1 --2- 172.21.15.43:0/1149275179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69200cbfc0 0x7f692013a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:20.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.501+0000 7f6925a97640 1 -- 172.21.15.43:0/1149275179 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69200c6c60 con 0x7f69200ccbb0 2024-04-25T09:02:20.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.501+0000 7f6925a97640 1 --2- 172.21.15.43:0/1149275179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69200ccbb0 0x7f69200c50d0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f691000d800 tx=0x7f691000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:20.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.501+0000 7f6916ffd640 1 -- 172.21.15.43:0/1149275179 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691000b6c0 con 0x7f69200ccbb0 2024-04-25T09:02:20.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.501+0000 7f6916ffd640 1 -- 172.21.15.43:0/1149275179 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6910019070 con 0x7f69200ccbb0 2024-04-25T09:02:20.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.501+0000 7f6916ffd640 1 -- 172.21.15.43:0/1149275179 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f691000ba80 con 0x7f69200ccbb0 2024-04-25T09:02:20.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.501+0000 7f69284fc640 1 -- 172.21.15.43:0/1149275179 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69200c5bc0 con 0x7f69200ccbb0 2024-04-25T09:02:20.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.501+0000 7f69284fc640 1 -- 172.21.15.43:0/1149275179 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69200c6460 con 0x7f69200ccbb0 2024-04-25T09:02:20.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.502+0000 7f69284fc640 1 -- 172.21.15.43:0/1149275179 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6920074ef0 con 0x7f69200ccbb0 2024-04-25T09:02:20.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.503+0000 7f6916ffd640 1 -- 172.21.15.43:0/1149275179 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f691000bc20 con 0x7f69200ccbb0 2024-04-25T09:02:20.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.503+0000 7f6916ffd640 1 --2- 172.21.15.43:0/1149275179 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f690406b4d0 0x7f690406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:20.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.503+0000 7f6916ffd640 1 -- 172.21.15.43:0/1149275179 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f691008a1d0 con 0x7f69200ccbb0 2024-04-25T09:02:20.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.503+0000 7f6926298640 1 --2- 172.21.15.43:0/1149275179 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f690406b4d0 0x7f690406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:20.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.504+0000 7f6926298640 1 --2- 172.21.15.43:0/1149275179 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f690406b4d0 0x7f690406d990 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f691c034000 tx=0x7f691c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:20.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.507+0000 7f6916ffd640 1 -- 172.21.15.43:0/1149275179 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6910058d90 con 0x7f69200ccbb0 2024-04-25T09:02:20.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.756+0000 7f69284fc640 1 -- 172.21.15.43:0/1149275179 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]} v 0) v1 -- 0x7f6920072560 con 0x7f69200ccbb0 2024-04-25T09:02:20.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.768+0000 7f6916ffd640 1 -- 172.21.15.43:0/1149275179 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]}]=0 v392) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6910088e20 con 0x7f69200ccbb0 2024-04-25T09:02:20.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.770+0000 7f69284fc640 1 -- 172.21.15.43:0/1149275179 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f690406b4d0 msgr2=0x7f690406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:20.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.770+0000 7f69284fc640 1 --2- 172.21.15.43:0/1149275179 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f690406b4d0 0x7f690406d990 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f691c034000 tx=0x7f691c02e000 comp rx=0 tx=0).stop 2024-04-25T09:02:20.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.770+0000 7f69284fc640 1 -- 172.21.15.43:0/1149275179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69200ccbb0 msgr2=0x7f69200c50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:20.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.770+0000 7f69284fc640 1 --2- 172.21.15.43:0/1149275179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69200ccbb0 0x7f69200c50d0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f691000d800 tx=0x7f691000dcc0 comp rx=0 tx=0).stop 2024-04-25T09:02:20.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.770+0000 7f69284fc640 1 -- 172.21.15.43:0/1149275179 shutdown_connections 2024-04-25T09:02:20.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.770+0000 7f69284fc640 1 --2- 172.21.15.43:0/1149275179 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f692015f5e0 0x7f69200c6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:20.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.770+0000 7f69284fc640 1 --2- 172.21.15.43:0/1149275179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69200ccbb0 0x7f69200c50d0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:20.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.770+0000 7f69284fc640 1 --2- 172.21.15.43:0/1149275179 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f690406b4d0 0x7f690406d990 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:20.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.770+0000 7f69284fc640 1 --2- 172.21.15.43:0/1149275179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69200cbfc0 0x7f692013a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:20.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.771+0000 7f69284fc640 1 -- 172.21.15.43:0/1149275179 >> 172.21.15.43:0/1149275179 conn(0x7f69200b1480 msgr2=0x7f69200b0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:20.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.771+0000 7f69284fc640 1 -- 172.21.15.43:0/1149275179 shutdown_connections 2024-04-25T09:02:20.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.771+0000 7f69284fc640 1 -- 172.21.15.43:0/1149275179 wait complete. 2024-04-25T09:02:20.771 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T09:02:20.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.885+0000 7fc6b9132640 1 Processor -- start 2024-04-25T09:02:20.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.885+0000 7fc6b9132640 1 -- start start 2024-04-25T09:02:20.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.886+0000 7fc6b9132640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc6b40d25d0 0x7fc6b40d02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:20.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.886+0000 7fc6b9132640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fc6b40d0800 0x7fc6b406b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:20.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.886+0000 7fc6b9132640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6b40d0c10 con 0x7fc6b415dd00 2024-04-25T09:02:20.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.886+0000 7fc6b9132640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6b40d15c0 con 0x7fc6b40d25d0 2024-04-25T09:02:20.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.886+0000 7fc6b9132640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6b415e0e0 con 0x7fc6b40d0800 2024-04-25T09:02:20.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.886+0000 7fc6b259c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc6b40d25d0 0x7fc6b40d02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:20.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.886+0000 7fc6b259c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc6b40d25d0 0x7fc6b40d02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44806/0 (socket says 172.21.15.43:44806) 2024-04-25T09:02:20.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.886+0000 7fc6b259c640 1 -- 172.21.15.43:0/569537990 learned_addr learned my addr 172.21.15.43:0/569537990 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:20.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.886+0000 7fc6b359e640 1 --2- 172.21.15.43:0/569537990 >> v2:172.21.15.193:3300/0 conn(0x7fc6b40d0800 0x7fc6b406b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:20.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.887+0000 7fc6b259c640 1 -- 172.21.15.43:0/569537990 >> v2:172.21.15.193:3300/0 conn(0x7fc6b40d0800 msgr2=0x7fc6b406b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:20.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.887+0000 7fc6b259c640 1 --2- 172.21.15.43:0/569537990 >> v2:172.21.15.193:3300/0 conn(0x7fc6b40d0800 0x7fc6b406b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:20.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.887+0000 7fc6b259c640 1 -- 172.21.15.43:0/569537990 >> v1:172.21.15.193:6789/0 conn(0x7fc6b415dd00 legacy=0x7fc6b40cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:20.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.887+0000 7fc6b259c640 1 -- 172.21.15.43:0/569537990 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b40d2a40 con 0x7fc6b40d25d0 2024-04-25T09:02:20.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.887+0000 7fc6b259c640 1 --2- 172.21.15.43:0/569537990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6b40d25d0 0x7fc6b40d02c0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fc6b40d1740 tx=0x7fc69c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aa222f4cac2edcc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:20.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.887+0000 7fc6b1d9b640 1 -- 172.21.15.43:0/569537990 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69c037070 con 0x7fc6b40d25d0 2024-04-25T09:02:20.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.887+0000 7fc6b1d9b640 1 -- 172.21.15.43:0/569537990 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc69c032440 con 0x7fc6b40d25d0 2024-04-25T09:02:20.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.887+0000 7fc6b1d9b640 1 -- 172.21.15.43:0/569537990 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69c032810 con 0x7fc6b40d25d0 2024-04-25T09:02:20.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.888+0000 7fc6b9132640 1 -- 172.21.15.43:0/569537990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6b40d25d0 msgr2=0x7fc6b40d02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:20.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.888+0000 7fc6b9132640 1 --2- 172.21.15.43:0/569537990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6b40d25d0 0x7fc6b40d02c0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fc6b40d1740 tx=0x7fc69c02a210 comp rx=0 tx=0).stop 2024-04-25T09:02:20.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.888+0000 7fc6b9132640 1 -- 172.21.15.43:0/569537990 shutdown_connections 2024-04-25T09:02:20.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.888+0000 7fc6b9132640 1 --2- 172.21.15.43:0/569537990 >> v2:172.21.15.193:3300/0 conn(0x7fc6b40d0800 0x7fc6b406b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:20.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.888+0000 7fc6b9132640 1 --2- 172.21.15.43:0/569537990 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6b40d25d0 0x7fc6b40d02c0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:20.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.888+0000 7fc6b9132640 1 -- 172.21.15.43:0/569537990 >> 172.21.15.43:0/569537990 conn(0x7fc6b409cd20 msgr2=0x7fc6b40d39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:20.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.888+0000 7fc6b9132640 1 -- 172.21.15.43:0/569537990 shutdown_connections 2024-04-25T09:02:20.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.888+0000 7fc6b9132640 1 -- 172.21.15.43:0/569537990 wait complete. 2024-04-25T09:02:20.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.889+0000 7fc6b9132640 1 Processor -- start 2024-04-25T09:02:20.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.890+0000 7fc6b9132640 1 -- start start 2024-04-25T09:02:20.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.890+0000 7fc6b9132640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc6b40d0800 0x7fc6b4156e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:20.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.890+0000 7fc6b9132640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6b40d25d0 0x7fc6b40c7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:20.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.890+0000 7fc6b9132640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6b415dd00 0x7fc6b40c9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:20.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.890+0000 7fc6b9132640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b40d1af0 con 0x7fc6b415dd00 2024-04-25T09:02:20.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.890+0000 7fc6b9132640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b40d1810 con 0x7fc6b40d25d0 2024-04-25T09:02:20.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.890+0000 7fc6b9132640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b40d1dd0 con 0x7fc6b40d0800 2024-04-25T09:02:20.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.890+0000 7fc6b359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6b415dd00 0x7fc6b40c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:20.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.890+0000 7fc6b359e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6b415dd00 0x7fc6b40c9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44812/0 (socket says 172.21.15.43:44812) 2024-04-25T09:02:20.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.890+0000 7fc6b359e640 1 -- 172.21.15.43:0/3512883200 learned_addr learned my addr 172.21.15.43:0/3512883200 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:20.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.891+0000 7fc6b259c640 1 --2- 172.21.15.43:0/3512883200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6b40d25d0 0x7fc6b40c7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:20.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.891+0000 7fc6b2d9d640 1 --2- 172.21.15.43:0/3512883200 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc6b40d0800 0x7fc6b4156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:20.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.891+0000 7fc6b359e640 1 -- 172.21.15.43:0/3512883200 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc6b40d0800 msgr2=0x7fc6b4156e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:20.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.891+0000 7fc6b359e640 1 --2- 172.21.15.43:0/3512883200 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc6b40d0800 0x7fc6b4156e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:20.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.891+0000 7fc6b359e640 1 -- 172.21.15.43:0/3512883200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6b40d25d0 msgr2=0x7fc6b40c7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:20.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.891+0000 7fc6b359e640 1 --2- 172.21.15.43:0/3512883200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6b40d25d0 0x7fc6b40c7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:20.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.891+0000 7fc6b359e640 1 -- 172.21.15.43:0/3512883200 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b415f560 con 0x7fc6b415dd00 2024-04-25T09:02:20.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.891+0000 7fc6b359e640 1 --2- 172.21.15.43:0/3512883200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6b415dd00 0x7fc6b40c9700 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4007e60 tx=0x7fc6a400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:20.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.891+0000 7fc6ab7fe640 1 -- 172.21.15.43:0/3512883200 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4014420 con 0x7fc6b415dd00 2024-04-25T09:02:20.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.891+0000 7fc6ab7fe640 1 -- 172.21.15.43:0/3512883200 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a4019070 con 0x7fc6b415dd00 2024-04-25T09:02:20.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.891+0000 7fc6b9132640 1 -- 172.21.15.43:0/3512883200 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6b415f840 con 0x7fc6b415dd00 2024-04-25T09:02:20.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.891+0000 7fc6ab7fe640 1 -- 172.21.15.43:0/3512883200 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4014820 con 0x7fc6b415dd00 2024-04-25T09:02:20.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.891+0000 7fc6b9132640 1 -- 172.21.15.43:0/3512883200 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6b415fe00 con 0x7fc6b415dd00 2024-04-25T09:02:20.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.892+0000 7fc6b9132640 1 -- 172.21.15.43:0/3512883200 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6b4074f60 con 0x7fc6b415dd00 2024-04-25T09:02:20.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.893+0000 7fc6ab7fe640 1 -- 172.21.15.43:0/3512883200 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc6a40149c0 con 0x7fc6b415dd00 2024-04-25T09:02:20.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.893+0000 7fc6ab7fe640 1 --2- 172.21.15.43:0/3512883200 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc69406b4d0 0x7fc69406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:20.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.893+0000 7fc6ab7fe640 1 -- 172.21.15.43:0/3512883200 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc6a408a590 con 0x7fc6b415dd00 2024-04-25T09:02:20.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.893+0000 7fc6b2d9d640 1 --2- 172.21.15.43:0/3512883200 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc69406b4d0 0x7fc69406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:20.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.894+0000 7fc6b2d9d640 1 --2- 172.21.15.43:0/3512883200 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc69406b4d0 0x7fc69406d990 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac004f60 tx=0x7fc6ac007e30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:20.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:20.897+0000 7fc6ab7fe640 1 -- 172.21.15.43:0/3512883200 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc6a4059150 con 0x7fc6b415dd00 2024-04-25T09:02:21.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.081+0000 7fc6b9132640 1 -- 172.21.15.43:0/3512883200 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc6b40cdd00 con 0x7fc6b415dd00 2024-04-25T09:02:21.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.084+0000 7fc6ab7fe640 1 -- 172.21.15.43:0/3512883200 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc6a405d1f0 con 0x7fc6b415dd00 2024-04-25T09:02:21.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.086+0000 7fc6b9132640 1 -- 172.21.15.43:0/3512883200 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc69406b4d0 msgr2=0x7fc69406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:21.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.086+0000 7fc6b9132640 1 --2- 172.21.15.43:0/3512883200 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc69406b4d0 0x7fc69406d990 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc6ac004f60 tx=0x7fc6ac007e30 comp rx=0 tx=0).stop 2024-04-25T09:02:21.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.086+0000 7fc6b9132640 1 -- 172.21.15.43:0/3512883200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6b415dd00 msgr2=0x7fc6b40c9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:21.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.086+0000 7fc6b9132640 1 --2- 172.21.15.43:0/3512883200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6b415dd00 0x7fc6b40c9700 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fc6a4007e60 tx=0x7fc6a400e4b0 comp rx=0 tx=0).stop 2024-04-25T09:02:21.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.086+0000 7fc6b9132640 1 -- 172.21.15.43:0/3512883200 shutdown_connections 2024-04-25T09:02:21.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.086+0000 7fc6b9132640 1 --2- 172.21.15.43:0/3512883200 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6b415dd00 0x7fc6b40c9700 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:21.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.086+0000 7fc6b9132640 1 --2- 172.21.15.43:0/3512883200 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc69406b4d0 0x7fc69406d990 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:21.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.086+0000 7fc6b9132640 1 --2- 172.21.15.43:0/3512883200 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6b40d25d0 0x7fc6b40c7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:21.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.086+0000 7fc6b9132640 1 --2- 172.21.15.43:0/3512883200 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc6b40d0800 0x7fc6b4156e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:21.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.086+0000 7fc6b9132640 1 -- 172.21.15.43:0/3512883200 >> 172.21.15.43:0/3512883200 conn(0x7fc6b409cd20 msgr2=0x7fc6b40d3980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:21.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.087+0000 7fc6b9132640 1 -- 172.21.15.43:0/3512883200 shutdown_connections 2024-04-25T09:02:21.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.087+0000 7fc6b9132640 1 -- 172.21.15.43:0/3512883200 wait complete. 2024-04-25T09:02:21.087 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:02:21.087 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:02:21.130 DEBUG:teuthology.orchestra.run.smithi193:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:02:21.175 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:02:21.175 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T09:02:21.176 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:02:21.192 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2374 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:02:21.192 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T09:02:21.192 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:02:21.210 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19508 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:02:21.211 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T09:02:21.211 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:02:21.248 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2374 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:02:21.248 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T09:02:21.248 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:02:21.269 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19508 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:02:21.269 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T09:02:21.269 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:02:21.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.401+0000 7f9935b89640 1 Processor -- start 2024-04-25T09:02:21.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.401+0000 7f9935b89640 1 -- start start 2024-04-25T09:02:21.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.402+0000 7f9935b89640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f993015daa0 0x7f9930162220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:21.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.402+0000 7f9935b89640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9930162760 con 0x7f993015de80 2024-04-25T09:02:21.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.402+0000 7f9935b89640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99300b4560 con 0x7f99300cc8d0 2024-04-25T09:02:21.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.402+0000 7f9935b89640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99300cd950 con 0x7f993015daa0 2024-04-25T09:02:21.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.402+0000 7f992effd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f993015de80 0x7f99300cc1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:33862/0 (socket says 172.21.15.43:33862) 2024-04-25T09:02:21.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.402+0000 7f992effd640 1 -- 172.21.15.43:0/4223870260 learned_addr learned my addr 172.21.15.43:0/4223870260 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:21.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.402+0000 7f992f7fe640 1 --2- 172.21.15.43:0/4223870260 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f993015daa0 0x7f9930162220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:21.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.402+0000 7f992e7fc640 1 -- 172.21.15.43:0/4223870260 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 789505400 0 0) 0x7f9930162760 con 0x7f993015de80 2024-04-25T09:02:21.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.402+0000 7f992e7fc640 1 -- 172.21.15.43:0/4223870260 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9918003520 con 0x7f993015de80 2024-04-25T09:02:21.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.402+0000 7f992e7fc640 1 -- 172.21.15.43:0/4223870260 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2206886894 0 0) 0x7f9918003520 con 0x7f993015de80 2024-04-25T09:02:21.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.402+0000 7f992e7fc640 1 -- 172.21.15.43:0/4223870260 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f993015daa0 msgr2=0x7f9930162220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:21.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.402+0000 7f992e7fc640 1 --2- 172.21.15.43:0/4223870260 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f993015daa0 0x7f9930162220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:21.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.402+0000 7f992e7fc640 1 -- 172.21.15.43:0/4223870260 >> v1:172.21.15.193:6789/0 conn(0x7f99300cc8d0 legacy=0x7f99300ce0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:21.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.402+0000 7f992e7fc640 1 -- 172.21.15.43:0/4223870260 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99300b4840 con 0x7f993015de80 2024-04-25T09:02:21.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.402+0000 7f992e7fc640 1 -- 172.21.15.43:0/4223870260 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f9920001670 con 0x7f993015de80 2024-04-25T09:02:21.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.402+0000 7f992e7fc640 1 -- 172.21.15.43:0/4223870260 >> v1:172.21.15.43:6789/0 conn(0x7f993015de80 legacy=0x7f99300cc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:21.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.403+0000 7f992e7fc640 1 --2- 172.21.15.43:0/4223870260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99180039e0 0x7f9918003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:21.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.403+0000 7f992e7fc640 1 --2- 172.21.15.43:0/4223870260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9918004800 0x7f9918004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:21.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.403+0000 7f992e7fc640 1 --2- 172.21.15.43:0/4223870260 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f993015daa0 0x7f99180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:21.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.403+0000 7f992e7fc640 1 -- 172.21.15.43:0/4223870260 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99300cd950 con 0x7f99180039e0 2024-04-25T09:02:21.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.403+0000 7f992e7fc640 1 -- 172.21.15.43:0/4223870260 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991800ba70 con 0x7f9918004800 2024-04-25T09:02:21.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.403+0000 7f992e7fc640 1 -- 172.21.15.43:0/4223870260 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991800bc70 con 0x7f993015daa0 2024-04-25T09:02:21.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.403+0000 7f992f7fe640 1 --2- 172.21.15.43:0/4223870260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99180039e0 0x7f9918003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:21.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.403+0000 7f992ffff640 1 --2- 172.21.15.43:0/4223870260 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f993015daa0 0x7f99180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:21.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.403+0000 7f992effd640 1 --2- 172.21.15.43:0/4223870260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9918004800 0x7f9918004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:21.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.403+0000 7f992f7fe640 1 -- 172.21.15.43:0/4223870260 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f993015daa0 msgr2=0x7f99180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:21.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.403+0000 7f992f7fe640 1 --2- 172.21.15.43:0/4223870260 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f993015daa0 0x7f99180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:21.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.403+0000 7f992f7fe640 1 -- 172.21.15.43:0/4223870260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9918004800 msgr2=0x7f9918004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:21.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.403+0000 7f992f7fe640 1 --2- 172.21.15.43:0/4223870260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9918004800 0x7f9918004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:21.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.403+0000 7f992f7fe640 1 -- 172.21.15.43:0/4223870260 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9918003520 con 0x7f99180039e0 2024-04-25T09:02:21.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.403+0000 7f992effd640 1 --2- 172.21.15.43:0/4223870260 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9918004800 0x7f9918004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:21.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.404+0000 7f992f7fe640 1 --2- 172.21.15.43:0/4223870260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99180039e0 0x7f9918003e00 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f9924009a90 tx=0x7f9918005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ddbbf0894494112 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:21.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.404+0000 7f992e7fc640 1 -- 172.21.15.43:0/4223870260 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9924036070 con 0x7f99180039e0 2024-04-25T09:02:21.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.404+0000 7f992e7fc640 1 -- 172.21.15.43:0/4223870260 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9924031400 con 0x7f99180039e0 2024-04-25T09:02:21.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.404+0000 7f992e7fc640 1 -- 172.21.15.43:0/4223870260 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9924009690 con 0x7f99180039e0 2024-04-25T09:02:21.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.404+0000 7f9935b89640 1 -- 172.21.15.43:0/4223870260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99180039e0 msgr2=0x7f9918003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:21.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.404+0000 7f9935b89640 1 --2- 172.21.15.43:0/4223870260 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f99180039e0 0x7f9918003e00 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7f9924009a90 tx=0x7f9918005830 comp rx=0 tx=0).stop 2024-04-25T09:02:21.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.404+0000 7f992ffff640 1 -- 172.21.15.43:0/4223870260 reap_dead start 2024-04-25T09:02:21.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.404+0000 7f9935b89640 1 -- 172.21.15.43:0/4223870260 shutdown_connections 2024-04-25T09:02:21.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.404+0000 7f9935b89640 1 -- 172.21.15.43:0/4223870260 >> 172.21.15.43:0/4223870260 conn(0x7f99300b1660 msgr2=0x7f99300b1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:21.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.404+0000 7f9935b89640 1 -- 172.21.15.43:0/4223870260 shutdown_connections 2024-04-25T09:02:21.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.404+0000 7f9935b89640 1 -- 172.21.15.43:0/4223870260 wait complete. 2024-04-25T09:02:21.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.405+0000 7f9935b89640 1 Processor -- start 2024-04-25T09:02:21.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.406+0000 7f9935b89640 1 -- start start 2024-04-25T09:02:21.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.406+0000 7f9935b89640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99180039e0 0x7f9930146570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:21.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.406+0000 7f9935b89640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9930153dd0 0x7f993006ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:21.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.406+0000 7f9935b89640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f993006e2f0 0x7f9930158630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:21.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.406+0000 7f9935b89640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99300b2960 con 0x7f9930153dd0 2024-04-25T09:02:21.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.406+0000 7f9935b89640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99300cdba0 con 0x7f99180039e0 2024-04-25T09:02:21.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.406+0000 7f9935b89640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99300b2ae0 con 0x7f993006e2f0 2024-04-25T09:02:21.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.407+0000 7f992effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9930153dd0 0x7f993006ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:21.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.407+0000 7f992effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9930153dd0 0x7f993006ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44838/0 (socket says 172.21.15.43:44838) 2024-04-25T09:02:21.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.407+0000 7f992effd640 1 -- 172.21.15.43:0/1323091257 learned_addr learned my addr 172.21.15.43:0/1323091257 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:21.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.407+0000 7f992f7fe640 1 --2- 172.21.15.43:0/1323091257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99180039e0 0x7f9930146570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:21.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.407+0000 7f992ffff640 1 --2- 172.21.15.43:0/1323091257 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f993006e2f0 0x7f9930158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:21.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.407+0000 7f992effd640 1 -- 172.21.15.43:0/1323091257 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f993006e2f0 msgr2=0x7f9930158630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:21.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.407+0000 7f992effd640 1 --2- 172.21.15.43:0/1323091257 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f993006e2f0 0x7f9930158630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:21.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.407+0000 7f992effd640 1 -- 172.21.15.43:0/1323091257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99180039e0 msgr2=0x7f9930146570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:21.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.407+0000 7f992effd640 1 --2- 172.21.15.43:0/1323091257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99180039e0 0x7f9930146570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:21.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.407+0000 7f992effd640 1 -- 172.21.15.43:0/1323091257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9930159340 con 0x7f9930153dd0 2024-04-25T09:02:21.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.407+0000 7f992effd640 1 --2- 172.21.15.43:0/1323091257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9930153dd0 0x7f993006ddb0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f992000e780 tx=0x7f992000ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:21.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.407+0000 7f9917fff640 1 -- 172.21.15.43:0/1323091257 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992000c8d0 con 0x7f9930153dd0 2024-04-25T09:02:21.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.407+0000 7f9917fff640 1 -- 172.21.15.43:0/1323091257 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9920018070 con 0x7f9930153dd0 2024-04-25T09:02:21.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.407+0000 7f9917fff640 1 -- 172.21.15.43:0/1323091257 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992000ccb0 con 0x7f9930153dd0 2024-04-25T09:02:21.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.407+0000 7f9935b89640 1 -- 172.21.15.43:0/1323091257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99301532f0 con 0x7f9930153dd0 2024-04-25T09:02:21.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.407+0000 7f9935b89640 1 -- 172.21.15.43:0/1323091257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9930154230 con 0x7f9930153dd0 2024-04-25T09:02:21.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.408+0000 7f9935b89640 1 -- 172.21.15.43:0/1323091257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9930074ef0 con 0x7f9930153dd0 2024-04-25T09:02:21.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.409+0000 7f9917fff640 1 -- 172.21.15.43:0/1323091257 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f992001d460 con 0x7f9930153dd0 2024-04-25T09:02:21.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.409+0000 7f9917fff640 1 --2- 172.21.15.43:0/1323091257 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f991c06b4d0 0x7f991c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:21.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.409+0000 7f9917fff640 1 -- 172.21.15.43:0/1323091257 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9920089d60 con 0x7f9930153dd0 2024-04-25T09:02:21.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.409+0000 7f992f7fe640 1 --2- 172.21.15.43:0/1323091257 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f991c06b4d0 0x7f991c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:21.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.410+0000 7f992f7fe640 1 --2- 172.21.15.43:0/1323091257 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f991c06b4d0 0x7f991c06d990 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f9924033000 tx=0x7f9924008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:21.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.413+0000 7f9917fff640 1 -- 172.21.15.43:0/1323091257 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9920058800 con 0x7f9930153dd0 2024-04-25T09:02:21.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.628+0000 7f9935b89640 1 -- 172.21.15.43:0/1323091257 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99300729d0 con 0x7f9930153dd0 2024-04-25T09:02:21.617 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.629+0000 7f9917fff640 1 -- 172.21.15.43:0/1323091257 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+3702 (secure 0 0 0) 0x7f992005c8a0 con 0x7f9930153dd0 2024-04-25T09:02:21.618 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:21.618 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":63,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6497,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2150177203","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2150177203},{"type":"v1","addr":"172.21.15.193:6837","nonce":2150177203}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58},{"gid":7477,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/3585307677","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":3585307677},{"type":"v1","addr":"172.21.15.193:6836","nonce":3585307677}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":63},{"gid":8079,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/621488642","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":621488642},{"type":"v1","addr":"172.21.15.43:6834","nonce":621488642}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":57},{"gid":8082,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/1524395840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1524395840},{"type":"v1","addr":"172.21.15.43:6835","nonce":1524395840}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":58}],"filesystems":[]} 2024-04-25T09:02:21.618 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 63 2024-04-25T09:02:21.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.630+0000 7f9935b89640 1 -- 172.21.15.43:0/1323091257 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f991c06b4d0 msgr2=0x7f991c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:21.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.630+0000 7f9935b89640 1 --2- 172.21.15.43:0/1323091257 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f991c06b4d0 0x7f991c06d990 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f9924033000 tx=0x7f9924008030 comp rx=0 tx=0).stop 2024-04-25T09:02:21.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.631+0000 7f9935b89640 1 -- 172.21.15.43:0/1323091257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9930153dd0 msgr2=0x7f993006ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:21.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.631+0000 7f9935b89640 1 --2- 172.21.15.43:0/1323091257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9930153dd0 0x7f993006ddb0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f992000e780 tx=0x7f992000ec40 comp rx=0 tx=0).stop 2024-04-25T09:02:21.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.631+0000 7f9935b89640 1 -- 172.21.15.43:0/1323091257 shutdown_connections 2024-04-25T09:02:21.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.631+0000 7f9935b89640 1 --2- 172.21.15.43:0/1323091257 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f993006e2f0 0x7f9930158630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:21.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.631+0000 7f9935b89640 1 --2- 172.21.15.43:0/1323091257 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9930153dd0 0x7f993006ddb0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:21.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.631+0000 7f9935b89640 1 --2- 172.21.15.43:0/1323091257 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f991c06b4d0 0x7f991c06d990 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:21.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.631+0000 7f9935b89640 1 --2- 172.21.15.43:0/1323091257 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f99180039e0 0x7f9930146570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:21.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.631+0000 7f9935b89640 1 -- 172.21.15.43:0/1323091257 >> 172.21.15.43:0/1323091257 conn(0x7f99300b1660 msgr2=0x7f99300ae280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:21.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.631+0000 7f9935b89640 1 -- 172.21.15.43:0/1323091257 shutdown_connections 2024-04-25T09:02:21.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.631+0000 7f9935b89640 1 -- 172.21.15.43:0/1323091257 wait complete. 2024-04-25T09:02:21.631 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:02:21.631 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T09:02:21.631 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:02:21.631 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:02:21.632 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:02:21.632 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T09:02:21.632 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:02:21.632 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:02:21.632 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:02:21.632 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T09:02:21.633 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:02:21.633 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:02:21.633 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:02:21.633 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T09:02:21.633 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:02:21.633 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:02:21.643 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:02:21.643 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:02:21.644 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:02:21.644 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:02:21.645 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:02:21.645 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:02:21.646 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:02:21.646 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:02:21.687 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:02:21.687 DEBUG:teuthology.parallel:result is None 2024-04-25T09:02:21.687 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:02:21.688 DEBUG:teuthology.parallel:result is None 2024-04-25T09:02:21.688 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:02:21.689 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:02:21.689 DEBUG:teuthology.parallel:result is None 2024-04-25T09:02:21.689 DEBUG:teuthology.parallel:result is None 2024-04-25T09:02:21.690 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T09:02:21.763 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2024-04-25T09:02:21.764 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-04-25T09:02:21.766 INFO:tasks.ceph.mds.d.smithi193.stdout:starting mds.d at 2024-04-25T09:02:21.766 INFO:tasks.ceph.mds.b.smithi193.stdout:starting mds.b at 2024-04-25T09:02:21.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.806+0000 7f162c02a640 1 Processor -- start 2024-04-25T09:02:21.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.806+0000 7f162c02a640 1 -- start start 2024-04-25T09:02:21.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.807+0000 7f162c02a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f162415df50 0x7f16241619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:21.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.807+0000 7f162c02a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1624161f30 0x7f16241623a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:21.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.807+0000 7f162c02a640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16240d24a0 con 0x7f16240c7cf0 2024-04-25T09:02:21.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.807+0000 7f162c02a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f162415e330 con 0x7f162415df50 2024-04-25T09:02:21.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.807+0000 7f162c02a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16240d2740 con 0x7f1624161f30 2024-04-25T09:02:21.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.808+0000 7f1629dc6640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f162415df50 0x7f16241619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:21.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.808+0000 7f1629dc6640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f162415df50 0x7f16241619f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44866/0 (socket says 172.21.15.43:44866) 2024-04-25T09:02:21.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.808+0000 7f1629dc6640 1 -- 172.21.15.43:0/1724148242 learned_addr learned my addr 172.21.15.43:0/1724148242 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:21.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.808+0000 7f16295c5640 1 --2- 172.21.15.43:0/1724148242 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1624161f30 0x7f16241623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:21.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.808+0000 7f1629dc6640 1 -- 172.21.15.43:0/1724148242 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1624161f30 msgr2=0x7f16241623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:21.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.808+0000 7f1629dc6640 1 --2- 172.21.15.43:0/1724148242 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1624161f30 0x7f16241623a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:21.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.808+0000 7f1629dc6640 1 -- 172.21.15.43:0/1724148242 >> v1:172.21.15.193:6789/0 conn(0x7f16240c7cf0 legacy=0x7f16240c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:21.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.808+0000 7f1628dc4640 1 -- 172.21.15.43:0/1724148242 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 482194023 0 0) 0x7f16240d24a0 con 0x7f16240c7cf0 2024-04-25T09:02:21.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.808+0000 7f1629dc6640 1 -- 172.21.15.43:0/1724148242 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16240d2a20 con 0x7f162415df50 2024-04-25T09:02:21.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.809+0000 7f1629dc6640 1 --2- 172.21.15.43:0/1724148242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f162415df50 0x7f16241619f0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f1614009e50 tx=0x7f161402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1073a8597094c8e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:21.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.809+0000 7f1628dc4640 1 -- 172.21.15.43:0/1724148242 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1614036070 con 0x7f162415df50 2024-04-25T09:02:21.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.809+0000 7f1628dc4640 1 -- 172.21.15.43:0/1724148242 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1614031600 con 0x7f162415df50 2024-04-25T09:02:21.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.809+0000 7f1628dc4640 1 -- 172.21.15.43:0/1724148242 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16140319d0 con 0x7f162415df50 2024-04-25T09:02:21.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.809+0000 7f162c02a640 1 -- 172.21.15.43:0/1724148242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f162415df50 msgr2=0x7f16241619f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:21.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.809+0000 7f162c02a640 1 --2- 172.21.15.43:0/1724148242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f162415df50 0x7f16241619f0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f1614009e50 tx=0x7f161402a710 comp rx=0 tx=0).stop 2024-04-25T09:02:21.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.809+0000 7f162c02a640 1 -- 172.21.15.43:0/1724148242 shutdown_connections 2024-04-25T09:02:21.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.809+0000 7f162c02a640 1 --2- 172.21.15.43:0/1724148242 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1624161f30 0x7f16241623a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:21.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.809+0000 7f162c02a640 1 --2- 172.21.15.43:0/1724148242 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f162415df50 0x7f16241619f0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:21.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.809+0000 7f162c02a640 1 -- 172.21.15.43:0/1724148242 >> 172.21.15.43:0/1724148242 conn(0x7f16240d3a30 msgr2=0x7f16240d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:21.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.810+0000 7f162c02a640 1 -- 172.21.15.43:0/1724148242 shutdown_connections 2024-04-25T09:02:21.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.810+0000 7f162c02a640 1 -- 172.21.15.43:0/1724148242 wait complete. 2024-04-25T09:02:21.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.811+0000 7f162c02a640 1 Processor -- start 2024-04-25T09:02:21.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.811+0000 7f162c02a640 1 -- start start 2024-04-25T09:02:21.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.811+0000 7f162c02a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16240c7cf0 0x7f16240c6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:21.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.811+0000 7f162c02a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1624161f30 0x7f1624146840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:21.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.811+0000 7f162c02a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1624146d80 0x7f16240c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:21.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.811+0000 7f162c02a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16240d2190 con 0x7f1624161f30 2024-04-25T09:02:21.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.812+0000 7f162c02a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16240c77d0 con 0x7f16240c7cf0 2024-04-25T09:02:21.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.812+0000 7f162c02a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16240cf670 con 0x7f1624146d80 2024-04-25T09:02:21.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.812+0000 7f16295c5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1624161f30 0x7f1624146840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:21.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.812+0000 7f16295c5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1624161f30 0x7f1624146840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44868/0 (socket says 172.21.15.43:44868) 2024-04-25T09:02:21.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.812+0000 7f16295c5640 1 -- 172.21.15.43:0/2331181414 learned_addr learned my addr 172.21.15.43:0/2331181414 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:21.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.812+0000 7f1629dc6640 1 --2- 172.21.15.43:0/2331181414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16240c7cf0 0x7f16240c6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:21.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.812+0000 7f162a5c7640 1 --2- 172.21.15.43:0/2331181414 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1624146d80 0x7f16240c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:21.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.812+0000 7f16295c5640 1 -- 172.21.15.43:0/2331181414 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1624146d80 msgr2=0x7f16240c90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:21.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.812+0000 7f16295c5640 1 --2- 172.21.15.43:0/2331181414 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1624146d80 0x7f16240c90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:21.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.812+0000 7f16295c5640 1 -- 172.21.15.43:0/2331181414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16240c7cf0 msgr2=0x7f16240c6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:21.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.812+0000 7f16295c5640 1 --2- 172.21.15.43:0/2331181414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16240c7cf0 0x7f16240c6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:21.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.812+0000 7f16295c5640 1 -- 172.21.15.43:0/2331181414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1624147340 con 0x7f1624161f30 2024-04-25T09:02:21.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.812+0000 7f1629dc6640 1 --2- 172.21.15.43:0/2331181414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16240c7cf0 0x7f16240c6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:21.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.812+0000 7f16295c5640 1 --2- 172.21.15.43:0/2331181414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1624161f30 0x7f1624146840 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f161800e870 tx=0x7f161800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:21.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.812+0000 7f16227fc640 1 -- 172.21.15.43:0/2331181414 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1618017070 con 0x7f1624161f30 2024-04-25T09:02:21.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.812+0000 7f16227fc640 1 -- 172.21.15.43:0/2331181414 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161800c940 con 0x7f1624161f30 2024-04-25T09:02:21.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.812+0000 7f162c02a640 1 -- 172.21.15.43:0/2331181414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1624147620 con 0x7f1624161f30 2024-04-25T09:02:21.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.812+0000 7f16227fc640 1 -- 172.21.15.43:0/2331181414 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161800cd30 con 0x7f1624161f30 2024-04-25T09:02:21.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.812+0000 7f162c02a640 1 -- 172.21.15.43:0/2331181414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1624147be0 con 0x7f1624161f30 2024-04-25T09:02:21.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.813+0000 7f162c02a640 1 -- 172.21.15.43:0/2331181414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1624074ef0 con 0x7f1624161f30 2024-04-25T09:02:21.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.814+0000 7f16227fc640 1 -- 172.21.15.43:0/2331181414 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f161801a070 con 0x7f1624161f30 2024-04-25T09:02:21.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.814+0000 7f16227fc640 1 --2- 172.21.15.43:0/2331181414 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f161006b4d0 0x7f161006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:21.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.814+0000 7f16227fc640 1 -- 172.21.15.43:0/2331181414 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f161808a120 con 0x7f1624161f30 2024-04-25T09:02:21.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.814+0000 7f1629dc6640 1 --2- 172.21.15.43:0/2331181414 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f161006b4d0 0x7f161006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:21.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.815+0000 7f1629dc6640 1 --2- 172.21.15.43:0/2331181414 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f161006b4d0 0x7f161006d990 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f1614033000 tx=0x7f1614009000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:21.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:21.818+0000 7f16227fc640 1 -- 172.21.15.43:0/2331181414 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f16180592a0 con 0x7f1624161f30 2024-04-25T09:02:22.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.011+0000 7f162c02a640 1 -- 172.21.15.43:0/2331181414 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f16240672d0 con 0x7f1624161f30 2024-04-25T09:02:22.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.781+0000 7f16227fc640 1 -- 172.21.15.43:0/2331181414 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v107) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1618058ba0 con 0x7f1624161f30 2024-04-25T09:02:22.770 INFO:teuthology.orchestra.run.smithi043.stderr: removed all blocklist entries 2024-04-25T09:02:22.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.784+0000 7f162c02a640 1 -- 172.21.15.43:0/2331181414 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f161006b4d0 msgr2=0x7f161006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:22.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.784+0000 7f162c02a640 1 --2- 172.21.15.43:0/2331181414 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f161006b4d0 0x7f161006d990 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f1614033000 tx=0x7f1614009000 comp rx=0 tx=0).stop 2024-04-25T09:02:22.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.784+0000 7f162c02a640 1 -- 172.21.15.43:0/2331181414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1624161f30 msgr2=0x7f1624146840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:22.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.784+0000 7f162c02a640 1 --2- 172.21.15.43:0/2331181414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1624161f30 0x7f1624146840 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f161800e870 tx=0x7f161800ed30 comp rx=0 tx=0).stop 2024-04-25T09:02:22.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.784+0000 7f162c02a640 1 -- 172.21.15.43:0/2331181414 shutdown_connections 2024-04-25T09:02:22.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.784+0000 7f162c02a640 1 --2- 172.21.15.43:0/2331181414 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1624146d80 0x7f16240c90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:22.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.784+0000 7f162c02a640 1 --2- 172.21.15.43:0/2331181414 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1624161f30 0x7f1624146840 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:22.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.784+0000 7f162c02a640 1 --2- 172.21.15.43:0/2331181414 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f161006b4d0 0x7f161006d990 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:22.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.784+0000 7f162c02a640 1 --2- 172.21.15.43:0/2331181414 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f16240c7cf0 0x7f16240c6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:22.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.784+0000 7f162c02a640 1 -- 172.21.15.43:0/2331181414 >> 172.21.15.43:0/2331181414 conn(0x7f16240d3a30 msgr2=0x7f16240cad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:22.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.784+0000 7f162c02a640 1 -- 172.21.15.43:0/2331181414 shutdown_connections 2024-04-25T09:02:22.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.784+0000 7f162c02a640 1 -- 172.21.15.43:0/2331181414 wait complete. 2024-04-25T09:02:22.788 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T09:02:22.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.906+0000 7fe42b44a640 1 Processor -- start 2024-04-25T09:02:22.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.906+0000 7fe42b44a640 1 -- start start 2024-04-25T09:02:22.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.907+0000 7fe42b44a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe424161390 0x7fe4240b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:22.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.907+0000 7fe42b44a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4240b4c30 con 0x7fe4240d02a0 2024-04-25T09:02:22.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.907+0000 7fe42b44a640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4240cf2c0 con 0x7fe4240cecf0 2024-04-25T09:02:22.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.907+0000 7fe42b44a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4240d1910 con 0x7fe424161390 2024-04-25T09:02:22.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.907+0000 7fe4289e5640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fe4240d02a0 0x7fe4240ce5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:33876/0 (socket says 172.21.15.43:33876) 2024-04-25T09:02:22.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.907+0000 7fe4289e5640 1 -- 172.21.15.43:0/2019687100 learned_addr learned my addr 172.21.15.43:0/2019687100 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:22.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.907+0000 7fe4291e6640 1 --2- 172.21.15.43:0/2019687100 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe424161390 0x7fe4240b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:22.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.907+0000 7fe41bfff640 1 -- 172.21.15.43:0/2019687100 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 38252774 0 0) 0x7fe414001110 con 0x7fe4240d02a0 2024-04-25T09:02:22.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.907+0000 7fe41bfff640 1 -- 172.21.15.43:0/2019687100 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe410003520 con 0x7fe4240d02a0 2024-04-25T09:02:22.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.908+0000 7fe41bfff640 1 -- 172.21.15.43:0/2019687100 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3568098157 0 0) 0x7fe410003520 con 0x7fe4240d02a0 2024-04-25T09:02:22.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.908+0000 7fe41bfff640 1 -- 172.21.15.43:0/2019687100 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe424161390 msgr2=0x7fe4240b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:22.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.908+0000 7fe41bfff640 1 --2- 172.21.15.43:0/2019687100 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe424161390 0x7fe4240b6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:22.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.908+0000 7fe41bfff640 1 -- 172.21.15.43:0/2019687100 >> v1:172.21.15.193:6789/0 conn(0x7fe4240cecf0 legacy=0x7fe4240d1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:22.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.908+0000 7fe41bfff640 1 -- 172.21.15.43:0/2019687100 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4240cf5a0 con 0x7fe4240d02a0 2024-04-25T09:02:22.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.908+0000 7fe41bfff640 1 -- 172.21.15.43:0/2019687100 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7fe414001840 con 0x7fe4240d02a0 2024-04-25T09:02:22.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.908+0000 7fe41bfff640 1 -- 172.21.15.43:0/2019687100 >> v1:172.21.15.43:6789/0 conn(0x7fe4240d02a0 legacy=0x7fe4240ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:22.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.908+0000 7fe41bfff640 1 --2- 172.21.15.43:0/2019687100 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4100039e0 0x7fe410003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:22.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.908+0000 7fe41bfff640 1 --2- 172.21.15.43:0/2019687100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe410004390 0x7fe410004860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:22.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.908+0000 7fe41bfff640 1 --2- 172.21.15.43:0/2019687100 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe424161390 0x7fe4100050c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:22.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.908+0000 7fe41bfff640 1 -- 172.21.15.43:0/2019687100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4240d1910 con 0x7fe410004390 2024-04-25T09:02:22.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.908+0000 7fe41bfff640 1 -- 172.21.15.43:0/2019687100 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe41000b9c0 con 0x7fe4100039e0 2024-04-25T09:02:22.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.908+0000 7fe41bfff640 1 -- 172.21.15.43:0/2019687100 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe41000bbc0 con 0x7fe424161390 2024-04-25T09:02:22.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.908+0000 7fe4289e5640 1 --2- 172.21.15.43:0/2019687100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe410004390 0x7fe410004860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:22.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.908+0000 7fe4291e6640 1 --2- 172.21.15.43:0/2019687100 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4100039e0 0x7fe410003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:22.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.908+0000 7fe4299e7640 1 --2- 172.21.15.43:0/2019687100 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe424161390 0x7fe4100050c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:22.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.908+0000 7fe4289e5640 1 -- 172.21.15.43:0/2019687100 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe424161390 msgr2=0x7fe4100050c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:22.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.908+0000 7fe4289e5640 1 --2- 172.21.15.43:0/2019687100 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe424161390 0x7fe4100050c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:22.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.908+0000 7fe4289e5640 1 -- 172.21.15.43:0/2019687100 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4100039e0 msgr2=0x7fe410003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:22.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.908+0000 7fe4289e5640 1 --2- 172.21.15.43:0/2019687100 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4100039e0 0x7fe410003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:22.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.908+0000 7fe4289e5640 1 -- 172.21.15.43:0/2019687100 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe410003520 con 0x7fe410004390 2024-04-25T09:02:22.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.909+0000 7fe4289e5640 1 --2- 172.21.15.43:0/2019687100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe410004390 0x7fe410004860 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fe41400c4c0 tx=0x7fe410005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e71fe1e51fd33a68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:22.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.909+0000 7fe41bfff640 1 -- 172.21.15.43:0/2019687100 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41400c0e0 con 0x7fe410004390 2024-04-25T09:02:22.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.909+0000 7fe41bfff640 1 -- 172.21.15.43:0/2019687100 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe414034070 con 0x7fe410004390 2024-04-25T09:02:22.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.909+0000 7fe41bfff640 1 -- 172.21.15.43:0/2019687100 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe414005d10 con 0x7fe410004390 2024-04-25T09:02:22.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.910+0000 7fe42b44a640 1 -- 172.21.15.43:0/2019687100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe410004390 msgr2=0x7fe410004860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:22.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.910+0000 7fe42b44a640 1 --2- 172.21.15.43:0/2019687100 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe410004390 0x7fe410004860 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fe41400c4c0 tx=0x7fe410005830 comp rx=0 tx=0).stop 2024-04-25T09:02:22.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.910+0000 7fe4299e7640 1 -- 172.21.15.43:0/2019687100 reap_dead start 2024-04-25T09:02:22.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.910+0000 7fe42b44a640 1 -- 172.21.15.43:0/2019687100 shutdown_connections 2024-04-25T09:02:22.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.910+0000 7fe42b44a640 1 -- 172.21.15.43:0/2019687100 >> 172.21.15.43:0/2019687100 conn(0x7fe42409cd20 msgr2=0x7fe4240d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:22.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.910+0000 7fe42b44a640 1 -- 172.21.15.43:0/2019687100 shutdown_connections 2024-04-25T09:02:22.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.910+0000 7fe42b44a640 1 -- 172.21.15.43:0/2019687100 wait complete. 2024-04-25T09:02:22.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.911+0000 7fe42b44a640 1 Processor -- start 2024-04-25T09:02:22.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.911+0000 7fe42b44a640 1 -- start start 2024-04-25T09:02:22.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.912+0000 7fe42b44a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe410004390 0x7fe42415d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:22.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.912+0000 7fe42b44a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe4240c9550 0x7fe4240c99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:22.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.912+0000 7fe42b44a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4240caaa0 0x7fe4240caf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:22.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.912+0000 7fe42b44a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe424161770 con 0x7fe410004390 2024-04-25T09:02:22.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.912+0000 7fe42b44a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4240d15a0 con 0x7fe4240caaa0 2024-04-25T09:02:22.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.912+0000 7fe42b44a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4240b6820 con 0x7fe4240c9550 2024-04-25T09:02:22.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.912+0000 7fe4291e6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe410004390 0x7fe42415d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:22.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.912+0000 7fe4291e6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe410004390 0x7fe42415d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44888/0 (socket says 172.21.15.43:44888) 2024-04-25T09:02:22.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.912+0000 7fe4291e6640 1 -- 172.21.15.43:0/2094564678 learned_addr learned my addr 172.21.15.43:0/2094564678 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:22.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.912+0000 7fe4291e6640 1 -- 172.21.15.43:0/2094564678 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe4240c9550 msgr2=0x7fe4240c99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:22.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.912+0000 7fe4291e6640 1 --2- 172.21.15.43:0/2094564678 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe4240c9550 0x7fe4240c99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:22.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.912+0000 7fe4291e6640 1 -- 172.21.15.43:0/2094564678 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4240caaa0 msgr2=0x7fe4240caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:22.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.912+0000 7fe4291e6640 1 --2- 172.21.15.43:0/2094564678 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4240caaa0 0x7fe4240caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:22.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.912+0000 7fe4291e6640 1 -- 172.21.15.43:0/2094564678 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4240cb7b0 con 0x7fe410004390 2024-04-25T09:02:22.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.912+0000 7fe4291e6640 1 --2- 172.21.15.43:0/2094564678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe410004390 0x7fe42415d350 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fe420011940 tx=0x7fe420011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:22.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.913+0000 7fe419ffb640 1 -- 172.21.15.43:0/2094564678 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42000fa10 con 0x7fe410004390 2024-04-25T09:02:22.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.913+0000 7fe419ffb640 1 -- 172.21.15.43:0/2094564678 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe42000fbb0 con 0x7fe410004390 2024-04-25T09:02:22.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.913+0000 7fe42b44a640 1 -- 172.21.15.43:0/2094564678 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4240cba90 con 0x7fe410004390 2024-04-25T09:02:22.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.913+0000 7fe419ffb640 1 -- 172.21.15.43:0/2094564678 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe420014440 con 0x7fe410004390 2024-04-25T09:02:22.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.913+0000 7fe42b44a640 1 -- 172.21.15.43:0/2094564678 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4240cc050 con 0x7fe410004390 2024-04-25T09:02:22.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.914+0000 7fe42b44a640 1 -- 172.21.15.43:0/2094564678 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4240680f0 con 0x7fe410004390 2024-04-25T09:02:22.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.914+0000 7fe419ffb640 1 -- 172.21.15.43:0/2094564678 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe420012040 con 0x7fe410004390 2024-04-25T09:02:22.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.915+0000 7fe419ffb640 1 --2- 172.21.15.43:0/2094564678 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe40806b4d0 0x7fe40806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:22.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.915+0000 7fe419ffb640 1 -- 172.21.15.43:0/2094564678 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe42008a410 con 0x7fe410004390 2024-04-25T09:02:22.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.915+0000 7fe4289e5640 1 --2- 172.21.15.43:0/2094564678 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe40806b4d0 0x7fe40806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:22.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.915+0000 7fe4289e5640 1 --2- 172.21.15.43:0/2094564678 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe40806b4d0 0x7fe40806d990 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fe41400c4c0 tx=0x7fe414009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:22.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:22.919+0000 7fe419ffb640 1 -- 172.21.15.43:0/2094564678 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe420019030 con 0x7fe410004390 2024-04-25T09:02:23.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.170+0000 7fe42b44a640 1 -- 172.21.15.43:0/2094564678 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe424074ef0 con 0x7fe410004390 2024-04-25T09:02:23.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.171+0000 7fe419ffb640 1 -- 172.21.15.43:0/2094564678 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fe4200595e0 con 0x7fe410004390 2024-04-25T09:02:23.160 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:23.160 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:02:23.160 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_dump": [ 2024-04-25T09:02:23.160 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:02:23.160 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.a", 2024-04-25T09:02:23.160 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpmIjzEBRAAs2L10XrFO3mThjZevna5wA==", 2024-04-25T09:02:23.160 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:02:23.160 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T09:02:23.160 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T09:02:23.160 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T09:02:23.160 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:02:23.160 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:02:23.160 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:02:23.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:02:23.161 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.b", 2024-04-25T09:02:23.161 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm51cmERAAij4vy8xyihh8u/4dkmcllw==", 2024-04-25T09:02:23.161 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:02:23.161 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T09:02:23.161 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T09:02:23.161 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T09:02:23.161 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:02:23.161 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:02:23.161 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:02:23.161 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:02:23.161 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.c", 2024-04-25T09:02:23.161 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm1OOtDBAAlUYbCh1mBEEka9CWRnUz9A==", 2024-04-25T09:02:23.161 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:02:23.162 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T09:02:23.162 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T09:02:23.162 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T09:02:23.162 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:02:23.162 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:02:23.162 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:02:23.162 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:02:23.162 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.d", 2024-04-25T09:02:23.162 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpmpEvsFxAAXX1h37UWdFkyJ3Iw3xd8kw==", 2024-04-25T09:02:23.162 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:02:23.162 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T09:02:23.162 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T09:02:23.162 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T09:02:23.162 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:02:23.163 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:02:23.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:02:23.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:02:23.163 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.0", 2024-04-25T09:02:23.163 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD9GSpmi7qkORAAwSNgwzcVMitrPPf3nDyqvQ==", 2024-04-25T09:02:23.163 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:02:23.163 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T09:02:23.163 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T09:02:23.163 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:02:23.163 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:02:23.163 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:02:23.163 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:02:23.163 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.1", 2024-04-25T09:02:23.163 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAAGipmJDT3NxAAcWBCd9nt7Au+Xftjqcuc5w==", 2024-04-25T09:02:23.163 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:02:23.164 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T09:02:23.164 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T09:02:23.164 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:02:23.164 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:02:23.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:02:23.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:02:23.164 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.2", 2024-04-25T09:02:23.164 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQADGipm68kzNhAA0ZOnlr4edWDjx6RfoEHYIw==", 2024-04-25T09:02:23.164 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:02:23.164 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T09:02:23.164 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T09:02:23.164 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:02:23.164 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:02:23.164 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:02:23.164 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:02:23.165 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.3", 2024-04-25T09:02:23.165 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAGGipmT1AeNBAAoNPiEtkznSxxhVJ1EmZ1ZA==", 2024-04-25T09:02:23.165 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:02:23.165 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T09:02:23.165 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T09:02:23.165 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:02:23.165 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:02:23.165 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:02:23.165 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:02:23.165 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.4", 2024-04-25T09:02:23.165 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAMGipmBjHHNxAAH/SCzKHus8fKxxI3UN4wWQ==", 2024-04-25T09:02:23.165 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:02:23.165 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T09:02:23.165 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T09:02:23.165 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:02:23.166 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:02:23.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:02:23.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:02:23.166 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.5", 2024-04-25T09:02:23.166 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPGipm3FXKNRAAL97YYWONg4QHr4ajC5D7fA==", 2024-04-25T09:02:23.166 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:02:23.166 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T09:02:23.166 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T09:02:23.166 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:02:23.166 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:02:23.166 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:02:23.166 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:02:23.166 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.6", 2024-04-25T09:02:23.166 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQASGipmCwKaMxAAcM0p9Q2sbNQviGkwSLg8AA==", 2024-04-25T09:02:23.166 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:02:23.167 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T09:02:23.167 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T09:02:23.167 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:02:23.167 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:02:23.167 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:02:23.167 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:02:23.167 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.7", 2024-04-25T09:02:23.167 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAVGipmP7A2MRAAxnFJFosMcbZ6VDbgH8r6wA==", 2024-04-25T09:02:23.167 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:02:23.167 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T09:02:23.167 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T09:02:23.167 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:02:23.167 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:02:23.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:02:23.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:02:23.168 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.0", 2024-04-25T09:02:23.168 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpmVOjWGxAAFNmxeGi0h2D//pa6Ne3eQw==", 2024-04-25T09:02:23.168 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:02:23.168 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T09:02:23.168 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow r", 2024-04-25T09:02:23.168 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T09:02:23.168 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:02:23.168 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:02:23.168 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:02:23.168 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.1", 2024-04-25T09:02:23.168 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm24laJBAA9ijbDlFDTmF/ZZ4IhLfBVQ==", 2024-04-25T09:02:23.168 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:02:23.169 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T09:02:23.169 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-04-25T09:02:23.169 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-04-25T09:02:23.169 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-04-25T09:02:23.169 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:02:23.169 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:02:23.169 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:02:23.169 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.2", 2024-04-25T09:02:23.169 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm8b5UIRAACcOwDL630oJrXfp7s+fNTA==", 2024-04-25T09:02:23.169 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:02:23.169 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T09:02:23.169 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-04-25T09:02:23.169 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-04-25T09:02:23.169 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-04-25T09:02:23.170 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:02:23.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:02:23.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:02:23.170 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.3", 2024-04-25T09:02:23.170 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpmLsbBKRAAIKWtRdpuyCaW1LnKMWTMiw==", 2024-04-25T09:02:23.170 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:02:23.170 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T09:02:23.170 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-04-25T09:02:23.170 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-04-25T09:02:23.170 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-04-25T09:02:23.170 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:02:23.170 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:02:23.170 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:02:23.171 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.admin", 2024-04-25T09:02:23.171 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD7GSpmE9C8JRAAS263n6a4L2DXzMyS9o42NQ==", 2024-04-25T09:02:23.171 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:02:23.171 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-04-25T09:02:23.171 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T09:02:23.171 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T09:02:23.171 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:02:23.171 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:02:23.171 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:02:23.171 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:02:23.171 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.x", 2024-04-25T09:02:23.171 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm/uGhAhAAr0FaJ6SwZ8Su1MEqyZwNRA==", 2024-04-25T09:02:23.171 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:02:23.171 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-04-25T09:02:23.171 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2024-04-25T09:02:23.172 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:02:23.172 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:02:23.172 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:02:23.172 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:02:23.172 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.y", 2024-04-25T09:02:23.172 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD7GSpmPwwpOhAALeiQ/LKM2zgjE7yu16RQOg==", 2024-04-25T09:02:23.172 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:02:23.172 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-04-25T09:02:23.172 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2024-04-25T09:02:23.172 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:02:23.172 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:02:23.172 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:02:23.172 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-04-25T09:02:23.172 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:02:23.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.173+0000 7fe42b44a640 1 -- 172.21.15.43:0/2094564678 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe40806b4d0 msgr2=0x7fe40806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:23.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.173+0000 7fe42b44a640 1 --2- 172.21.15.43:0/2094564678 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe40806b4d0 0x7fe40806d990 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fe41400c4c0 tx=0x7fe414009030 comp rx=0 tx=0).stop 2024-04-25T09:02:23.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.173+0000 7fe42b44a640 1 -- 172.21.15.43:0/2094564678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe410004390 msgr2=0x7fe42415d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:23.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.173+0000 7fe42b44a640 1 --2- 172.21.15.43:0/2094564678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe410004390 0x7fe42415d350 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fe420011940 tx=0x7fe420011e00 comp rx=0 tx=0).stop 2024-04-25T09:02:23.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.173+0000 7fe42b44a640 1 -- 172.21.15.43:0/2094564678 shutdown_connections 2024-04-25T09:02:23.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.173+0000 7fe42b44a640 1 --2- 172.21.15.43:0/2094564678 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe40806b4d0 0x7fe40806d990 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:23.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.173+0000 7fe42b44a640 1 --2- 172.21.15.43:0/2094564678 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe4240caaa0 0x7fe4240caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:23.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.173+0000 7fe42b44a640 1 --2- 172.21.15.43:0/2094564678 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe4240c9550 0x7fe4240c99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:23.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.173+0000 7fe42b44a640 1 --2- 172.21.15.43:0/2094564678 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe410004390 0x7fe42415d350 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:23.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.173+0000 7fe42b44a640 1 -- 172.21.15.43:0/2094564678 >> 172.21.15.43:0/2094564678 conn(0x7fe42409cd20 msgr2=0x7fe4241606a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:23.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.173+0000 7fe42b44a640 1 -- 172.21.15.43:0/2094564678 shutdown_connections 2024-04-25T09:02:23.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.173+0000 7fe42b44a640 1 -- 172.21.15.43:0/2094564678 wait complete. 2024-04-25T09:02:23.175 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:02:23.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.287+0000 7f03697d5640 1 Processor -- start 2024-04-25T09:02:23.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.288+0000 7f03697d5640 1 -- start start 2024-04-25T09:02:23.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.288+0000 7f03697d5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f03640d39d0 0x7f03640d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:23.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.288+0000 7f03697d5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03640ce2a0 0x7f03640d3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:23.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.288+0000 7f03697d5640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0364161710 0x7f0364161ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:23.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.288+0000 7f03697d5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03640b6c50 con 0x7f03640d39d0 2024-04-25T09:02:23.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.288+0000 7f03697d5640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f036415e010 con 0x7f0364161710 2024-04-25T09:02:23.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.288+0000 7f03697d5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03640d2020 con 0x7f03640ce2a0 2024-04-25T09:02:23.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.288+0000 7f0362ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f03640d39d0 0x7f03640d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:23.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.288+0000 7f0362ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f03640d39d0 0x7f03640d0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44892/0 (socket says 172.21.15.43:44892) 2024-04-25T09:02:23.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.288+0000 7f0362ffd640 1 -- 172.21.15.43:0/691276404 learned_addr learned my addr 172.21.15.43:0/691276404 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:23.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.289+0000 7f03637fe640 1 --2- 172.21.15.43:0/691276404 >> v2:172.21.15.193:3300/0 conn(0x7f0364161710 0x7f0364161ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:23.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.289+0000 7f03627fc640 1 --2- 172.21.15.43:0/691276404 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03640ce2a0 0x7f03640d3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:23.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.289+0000 7f0362ffd640 1 -- 172.21.15.43:0/691276404 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03640ce2a0 msgr2=0x7f03640d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:23.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.289+0000 7f0362ffd640 1 --2- 172.21.15.43:0/691276404 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03640ce2a0 0x7f03640d3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:23.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.289+0000 7f0362ffd640 1 -- 172.21.15.43:0/691276404 >> v2:172.21.15.193:3300/0 conn(0x7f0364161710 msgr2=0x7f0364161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:23.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.289+0000 7f0362ffd640 1 --2- 172.21.15.43:0/691276404 >> v2:172.21.15.193:3300/0 conn(0x7f0364161710 0x7f0364161ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:23.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.289+0000 7f0362ffd640 1 -- 172.21.15.43:0/691276404 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03640d42f0 con 0x7f03640d39d0 2024-04-25T09:02:23.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.290+0000 7f0362ffd640 1 --2- 172.21.15.43:0/691276404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03640d39d0 0x7f03640d0d90 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f036406ccf0 tx=0x7f035c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdadf54958d15a18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:23.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.290+0000 7f0361ffb640 1 -- 172.21.15.43:0/691276404 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035c037070 con 0x7f03640d39d0 2024-04-25T09:02:23.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.290+0000 7f0361ffb640 1 -- 172.21.15.43:0/691276404 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f035c032440 con 0x7f03640d39d0 2024-04-25T09:02:23.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.290+0000 7f0361ffb640 1 -- 172.21.15.43:0/691276404 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035c032810 con 0x7f03640d39d0 2024-04-25T09:02:23.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.290+0000 7f03697d5640 1 -- 172.21.15.43:0/691276404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03640d39d0 msgr2=0x7f03640d0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:23.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.290+0000 7f03697d5640 1 --2- 172.21.15.43:0/691276404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03640d39d0 0x7f03640d0d90 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f036406ccf0 tx=0x7f035c02a140 comp rx=0 tx=0).stop 2024-04-25T09:02:23.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.290+0000 7f03697d5640 1 -- 172.21.15.43:0/691276404 shutdown_connections 2024-04-25T09:02:23.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.290+0000 7f03697d5640 1 --2- 172.21.15.43:0/691276404 >> v2:172.21.15.193:3300/0 conn(0x7f0364161710 0x7f0364161ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:23.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.290+0000 7f03697d5640 1 --2- 172.21.15.43:0/691276404 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03640ce2a0 0x7f03640d3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:23.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.290+0000 7f03697d5640 1 --2- 172.21.15.43:0/691276404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03640d39d0 0x7f03640d0d90 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:23.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.290+0000 7f03697d5640 1 -- 172.21.15.43:0/691276404 >> 172.21.15.43:0/691276404 conn(0x7f036406b000 msgr2=0x7f036406b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:23.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.290+0000 7f03697d5640 1 -- 172.21.15.43:0/691276404 shutdown_connections 2024-04-25T09:02:23.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.290+0000 7f03697d5640 1 -- 172.21.15.43:0/691276404 wait complete. 2024-04-25T09:02:23.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.292+0000 7f03697d5640 1 Processor -- start 2024-04-25T09:02:23.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.292+0000 7f03697d5640 1 -- start start 2024-04-25T09:02:23.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.292+0000 7f03697d5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03640ce2a0 0x7f03641f3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:23.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.292+0000 7f03697d5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03640d39d0 0x7f03640b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:23.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.292+0000 7f03697d5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0364161710 0x7f03641f7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:23.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.292+0000 7f03697d5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03640b2f00 con 0x7f03640ce2a0 2024-04-25T09:02:23.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.292+0000 7f03697d5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03640cef20 con 0x7f03640d39d0 2024-04-25T09:02:23.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.292+0000 7f03697d5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03640cf200 con 0x7f0364161710 2024-04-25T09:02:23.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.292+0000 7f0362ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03640ce2a0 0x7f03641f3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:23.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.292+0000 7f0362ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03640ce2a0 0x7f03641f3bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44898/0 (socket says 172.21.15.43:44898) 2024-04-25T09:02:23.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.292+0000 7f0362ffd640 1 -- 172.21.15.43:0/2560267255 learned_addr learned my addr 172.21.15.43:0/2560267255 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:23.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.293+0000 7f03627fc640 1 --2- 172.21.15.43:0/2560267255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03640d39d0 0x7f03640b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:23.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.293+0000 7f03637fe640 1 --2- 172.21.15.43:0/2560267255 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0364161710 0x7f03641f7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:23.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.293+0000 7f0362ffd640 1 -- 172.21.15.43:0/2560267255 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0364161710 msgr2=0x7f03641f7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:23.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.293+0000 7f0362ffd640 1 --2- 172.21.15.43:0/2560267255 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0364161710 0x7f03641f7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:23.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.293+0000 7f0362ffd640 1 -- 172.21.15.43:0/2560267255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03640d39d0 msgr2=0x7f03640b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:23.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.293+0000 7f0362ffd640 1 --2- 172.21.15.43:0/2560267255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03640d39d0 0x7f03640b43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:23.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.293+0000 7f0362ffd640 1 -- 172.21.15.43:0/2560267255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03640b4900 con 0x7f03640ce2a0 2024-04-25T09:02:23.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.293+0000 7f03627fc640 1 --2- 172.21.15.43:0/2560267255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03640d39d0 0x7f03640b43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:23.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.293+0000 7f0362ffd640 1 --2- 172.21.15.43:0/2560267255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03640ce2a0 0x7f03641f3bb0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f035c02a180 tx=0x7f035c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:23.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.293+0000 7f035b7fe640 1 -- 172.21.15.43:0/2560267255 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035c041070 con 0x7f03640ce2a0 2024-04-25T09:02:23.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.293+0000 7f035b7fe640 1 -- 172.21.15.43:0/2560267255 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f035c03c870 con 0x7f03640ce2a0 2024-04-25T09:02:23.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.293+0000 7f03697d5640 1 -- 172.21.15.43:0/2560267255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03640b4b80 con 0x7f03640ce2a0 2024-04-25T09:02:23.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.293+0000 7f035b7fe640 1 -- 172.21.15.43:0/2560267255 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035c037040 con 0x7f03640ce2a0 2024-04-25T09:02:23.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.293+0000 7f03697d5640 1 -- 172.21.15.43:0/2560267255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03640b5140 con 0x7f03640ce2a0 2024-04-25T09:02:23.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.294+0000 7f03697d5640 1 -- 172.21.15.43:0/2560267255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0364074ef0 con 0x7f03640ce2a0 2024-04-25T09:02:23.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.295+0000 7f035b7fe640 1 -- 172.21.15.43:0/2560267255 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f035c044070 con 0x7f03640ce2a0 2024-04-25T09:02:23.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.295+0000 7f035b7fe640 1 --2- 172.21.15.43:0/2560267255 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f034406b4d0 0x7f034406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:23.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.295+0000 7f035b7fe640 1 -- 172.21.15.43:0/2560267255 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f035c0aac60 con 0x7f03640ce2a0 2024-04-25T09:02:23.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.295+0000 7f03627fc640 1 --2- 172.21.15.43:0/2560267255 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f034406b4d0 0x7f034406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:23.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.296+0000 7f03627fc640 1 --2- 172.21.15.43:0/2560267255 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f034406b4d0 0x7f034406d990 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f03640b6060 tx=0x7f034c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:23.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.299+0000 7f035b7fe640 1 -- 172.21.15.43:0/2560267255 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f03640b4900 con 0x7f03640ce2a0 2024-04-25T09:02:23.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.480+0000 7f03697d5640 1 -- 172.21.15.43:0/2560267255 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03640d17c0 con 0x7f03640ce2a0 2024-04-25T09:02:23.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.481+0000 7f035b7fe640 1 -- 172.21.15.43:0/2560267255 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+12865 (secure 0 0 0) 0x7f035c079f50 con 0x7f03640ce2a0 2024-04-25T09:02:23.470 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:23.470 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":109,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:02:22.788793+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:02:23.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.483+0000 7f03697d5640 1 -- 172.21.15.43:0/2560267255 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f034406b4d0 msgr2=0x7f034406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:23.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.483+0000 7f03697d5640 1 --2- 172.21.15.43:0/2560267255 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f034406b4d0 0x7f034406d990 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f03640b6060 tx=0x7f034c008040 comp rx=0 tx=0).stop 2024-04-25T09:02:23.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.483+0000 7f03697d5640 1 -- 172.21.15.43:0/2560267255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03640ce2a0 msgr2=0x7f03641f3bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:23.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.483+0000 7f03697d5640 1 --2- 172.21.15.43:0/2560267255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03640ce2a0 0x7f03641f3bb0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f035c02a180 tx=0x7f035c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:02:23.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.483+0000 7f03697d5640 1 -- 172.21.15.43:0/2560267255 shutdown_connections 2024-04-25T09:02:23.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.483+0000 7f03697d5640 1 --2- 172.21.15.43:0/2560267255 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0364161710 0x7f03641f7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:23.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.483+0000 7f03697d5640 1 --2- 172.21.15.43:0/2560267255 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f034406b4d0 0x7f034406d990 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:23.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.483+0000 7f03697d5640 1 --2- 172.21.15.43:0/2560267255 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03640d39d0 0x7f03640b43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:23.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.483+0000 7f03697d5640 1 --2- 172.21.15.43:0/2560267255 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03640ce2a0 0x7f03641f3bb0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:23.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.483+0000 7f03697d5640 1 -- 172.21.15.43:0/2560267255 >> 172.21.15.43:0/2560267255 conn(0x7f036406b000 msgr2=0x7f036415f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:23.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.483+0000 7f03697d5640 1 -- 172.21.15.43:0/2560267255 shutdown_connections 2024-04-25T09:02:23.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.483+0000 7f03697d5640 1 -- 172.21.15.43:0/2560267255 wait complete. 2024-04-25T09:02:23.483 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:02:23.484 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:02:23.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.599+0000 7fedc7a9a640 1 Processor -- start 2024-04-25T09:02:23.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.599+0000 7fedc7a9a640 1 -- start start 2024-04-25T09:02:23.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.600+0000 7fedc7a9a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fedc00d04a0 0x7fedc015ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:23.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.600+0000 7fedc7a9a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fedc015dec0 0x7fedc00d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:23.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.600+0000 7fedc7a9a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc00d0dc0 con 0x7fedc00ce8c0 2024-04-25T09:02:23.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.600+0000 7fedc7a9a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedc00cf110 con 0x7fedc015dec0 2024-04-25T09:02:23.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.600+0000 7fedc7a9a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc00cf3f0 con 0x7fedc00d04a0 2024-04-25T09:02:23.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.600+0000 7fedc5035640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fedc015dec0 0x7fedc00d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:23.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.600+0000 7fedc6037640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fedc00ce8c0 0x7fedc01617a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:33884/0 (socket says 172.21.15.43:33884) 2024-04-25T09:02:23.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.600+0000 7fedc6037640 1 -- 172.21.15.43:0/1881118375 learned_addr learned my addr 172.21.15.43:0/1881118375 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:23.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.600+0000 7fedc5836640 1 --2- 172.21.15.43:0/1881118375 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fedc00d04a0 0x7fedc015ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:23.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.600+0000 7fedc5035640 1 -- 172.21.15.43:0/1881118375 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fedc00d04a0 msgr2=0x7fedc015ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:23.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.600+0000 7fedc5035640 1 --2- 172.21.15.43:0/1881118375 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fedc00d04a0 0x7fedc015ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:23.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.600+0000 7fedc5035640 1 -- 172.21.15.43:0/1881118375 >> v1:172.21.15.43:6789/0 conn(0x7fedc00ce8c0 legacy=0x7fedc01617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:23.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.600+0000 7fedc5035640 1 -- 172.21.15.43:0/1881118375 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc00cf6d0 con 0x7fedc015dec0 2024-04-25T09:02:23.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.600+0000 7fedc4834640 1 -- 172.21.15.43:0/1881118375 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 97186887 0 0) 0x7fedc00d0dc0 con 0x7fedc00ce8c0 2024-04-25T09:02:23.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.600+0000 7fedc5836640 1 --2- 172.21.15.43:0/1881118375 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fedc00d04a0 0x7fedc015ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:23.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.601+0000 7fedc5035640 1 --2- 172.21.15.43:0/1881118375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedc015dec0 0x7fedc00d0880 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fedb40097a0 tx=0x7fedb402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3505c19445b9ac1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:23.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.601+0000 7fedc4834640 1 -- 172.21.15.43:0/1881118375 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb4037070 con 0x7fedc015dec0 2024-04-25T09:02:23.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.601+0000 7fedc4834640 1 -- 172.21.15.43:0/1881118375 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb4032440 con 0x7fedc015dec0 2024-04-25T09:02:23.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.601+0000 7fedc4834640 1 -- 172.21.15.43:0/1881118375 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb4032810 con 0x7fedc015dec0 2024-04-25T09:02:23.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.602+0000 7fedc7a9a640 1 -- 172.21.15.43:0/1881118375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedc015dec0 msgr2=0x7fedc00d0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:23.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.602+0000 7fedc7a9a640 1 --2- 172.21.15.43:0/1881118375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedc015dec0 0x7fedc00d0880 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fedb40097a0 tx=0x7fedb402a180 comp rx=0 tx=0).stop 2024-04-25T09:02:23.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.602+0000 7fedc7a9a640 1 -- 172.21.15.43:0/1881118375 shutdown_connections 2024-04-25T09:02:23.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.602+0000 7fedc7a9a640 1 --2- 172.21.15.43:0/1881118375 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedc015dec0 0x7fedc00d0880 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:23.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.602+0000 7fedc7a9a640 1 --2- 172.21.15.43:0/1881118375 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fedc00d04a0 0x7fedc015ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:23.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.602+0000 7fedc7a9a640 1 -- 172.21.15.43:0/1881118375 >> 172.21.15.43:0/1881118375 conn(0x7fedc009cd40 msgr2=0x7fedc00d3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:23.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.602+0000 7fedc7a9a640 1 -- 172.21.15.43:0/1881118375 shutdown_connections 2024-04-25T09:02:23.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.602+0000 7fedc7a9a640 1 -- 172.21.15.43:0/1881118375 wait complete. 2024-04-25T09:02:23.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.603+0000 7fedc7a9a640 1 Processor -- start 2024-04-25T09:02:23.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.603+0000 7fedc7a9a640 1 -- start start 2024-04-25T09:02:23.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.604+0000 7fedc7a9a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fedc00ce8c0 0x7fedc015f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:23.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.604+0000 7fedc7a9a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedc00d04a0 0x7fedc00b87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:23.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.604+0000 7fedc7a9a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fedc015dec0 0x7fedc00b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:23.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.604+0000 7fedc7a9a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc00cecf0 con 0x7fedc00d04a0 2024-04-25T09:02:23.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.604+0000 7fedc7a9a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc00cfc90 con 0x7fedc00ce8c0 2024-04-25T09:02:23.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.604+0000 7fedc7a9a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc00cee70 con 0x7fedc015dec0 2024-04-25T09:02:23.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.604+0000 7fedc5035640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedc00d04a0 0x7fedc00b87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:23.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.604+0000 7fedc5035640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedc00d04a0 0x7fedc00b87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44914/0 (socket says 172.21.15.43:44914) 2024-04-25T09:02:23.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.604+0000 7fedc5035640 1 -- 172.21.15.43:0/72111135 learned_addr learned my addr 172.21.15.43:0/72111135 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:23.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.604+0000 7fedc5035640 1 -- 172.21.15.43:0/72111135 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fedc015dec0 msgr2=0x7fedc00b8d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:23.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.604+0000 7fedc5035640 1 --2- 172.21.15.43:0/72111135 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fedc015dec0 0x7fedc00b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:23.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.604+0000 7fedc5035640 1 -- 172.21.15.43:0/72111135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fedc00ce8c0 msgr2=0x7fedc015f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:23.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.604+0000 7fedc5836640 1 --2- 172.21.15.43:0/72111135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fedc00ce8c0 0x7fedc015f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:23.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.604+0000 7fedc5035640 1 --2- 172.21.15.43:0/72111135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fedc00ce8c0 0x7fedc015f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:23.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.604+0000 7fedc5035640 1 -- 172.21.15.43:0/72111135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc01570a0 con 0x7fedc00d04a0 2024-04-25T09:02:23.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.605+0000 7fedc5035640 1 --2- 172.21.15.43:0/72111135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedc00d04a0 0x7fedc00b87d0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fedb4035040 tx=0x7fedb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:23.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.605+0000 7fedbe7fc640 1 -- 172.21.15.43:0/72111135 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb4037040 con 0x7fedc00d04a0 2024-04-25T09:02:23.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.605+0000 7fedbe7fc640 1 -- 172.21.15.43:0/72111135 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb4030770 con 0x7fedc00d04a0 2024-04-25T09:02:23.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.605+0000 7fedc7a9a640 1 -- 172.21.15.43:0/72111135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc00b7e70 con 0x7fedc00d04a0 2024-04-25T09:02:23.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.605+0000 7fedbe7fc640 1 -- 172.21.15.43:0/72111135 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb4030b80 con 0x7fedc00d04a0 2024-04-25T09:02:23.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.605+0000 7fedc7a9a640 1 -- 172.21.15.43:0/72111135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc0161c50 con 0x7fedc00d04a0 2024-04-25T09:02:23.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.606+0000 7fedc7a9a640 1 -- 172.21.15.43:0/72111135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedc0074f60 con 0x7fedc00d04a0 2024-04-25T09:02:23.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.606+0000 7fedbe7fc640 1 -- 172.21.15.43:0/72111135 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fedb4044020 con 0x7fedc00d04a0 2024-04-25T09:02:23.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.607+0000 7fedbe7fc640 1 --2- 172.21.15.43:0/72111135 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fedac06b4d0 0x7fedac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:23.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.607+0000 7fedbe7fc640 1 -- 172.21.15.43:0/72111135 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fedb40aa470 con 0x7fedc00d04a0 2024-04-25T09:02:23.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.607+0000 7fedc5836640 1 --2- 172.21.15.43:0/72111135 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fedac06b4d0 0x7fedac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:23.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.608+0000 7fedc5836640 1 --2- 172.21.15.43:0/72111135 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fedac06b4d0 0x7fedac06d990 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fedb800d8b0 tx=0x7fedb8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:23.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.611+0000 7fedbe7fc640 1 -- 172.21.15.43:0/72111135 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fedb4032e20 con 0x7fedc00d04a0 2024-04-25T09:02:23.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.796+0000 7fedc7a9a640 1 -- 172.21.15.43:0/72111135 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fedc008daf0 con 0x7fedc00d04a0 2024-04-25T09:02:23.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.796+0000 7fedbe7fc640 1 -- 172.21.15.43:0/72111135 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v109) v1 ==== 93+0+10 (secure 0 0 0) 0x7fedb40795a0 con 0x7fedc00d04a0 2024-04-25T09:02:23.785 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T09:02:23.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.798+0000 7fedc7a9a640 1 -- 172.21.15.43:0/72111135 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fedac06b4d0 msgr2=0x7fedac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:23.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.798+0000 7fedc7a9a640 1 --2- 172.21.15.43:0/72111135 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fedac06b4d0 0x7fedac06d990 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fedb800d8b0 tx=0x7fedb8008040 comp rx=0 tx=0).stop 2024-04-25T09:02:23.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.798+0000 7fedc7a9a640 1 -- 172.21.15.43:0/72111135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedc00d04a0 msgr2=0x7fedc00b87d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:23.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.798+0000 7fedc7a9a640 1 --2- 172.21.15.43:0/72111135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedc00d04a0 0x7fedc00b87d0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7fedb4035040 tx=0x7fedb40046c0 comp rx=0 tx=0).stop 2024-04-25T09:02:23.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.798+0000 7fedc7a9a640 1 -- 172.21.15.43:0/72111135 shutdown_connections 2024-04-25T09:02:23.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.798+0000 7fedc7a9a640 1 --2- 172.21.15.43:0/72111135 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fedc015dec0 0x7fedc00b8d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:23.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.798+0000 7fedc7a9a640 1 --2- 172.21.15.43:0/72111135 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fedc00d04a0 0x7fedc00b87d0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:23.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.798+0000 7fedc7a9a640 1 --2- 172.21.15.43:0/72111135 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fedac06b4d0 0x7fedac06d990 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:23.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.798+0000 7fedc7a9a640 1 --2- 172.21.15.43:0/72111135 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fedc00ce8c0 0x7fedc015f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:23.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.798+0000 7fedc7a9a640 1 -- 172.21.15.43:0/72111135 >> 172.21.15.43:0/72111135 conn(0x7fedc009cd40 msgr2=0x7fedc00d0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:23.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.798+0000 7fedc7a9a640 1 -- 172.21.15.43:0/72111135 shutdown_connections 2024-04-25T09:02:23.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.798+0000 7fedc7a9a640 1 -- 172.21.15.43:0/72111135 wait complete. 2024-04-25T09:02:23.798 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:02:23.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.910+0000 7fb9c6110640 1 Processor -- start 2024-04-25T09:02:23.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.910+0000 7fb9c6110640 1 -- start start 2024-04-25T09:02:23.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.911+0000 7fb9c6110640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9c00b44d0 0x7fb9c00ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:23.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.911+0000 7fb9c6110640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9c00b4930 con 0x7fb9c00ade20 2024-04-25T09:02:23.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.911+0000 7fb9c6110640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9c0161e30 con 0x7fb9c00ae980 2024-04-25T09:02:23.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.911+0000 7fb9c6110640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c0162110 con 0x7fb9c00b44d0 2024-04-25T09:02:23.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.911+0000 7fb9bffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb9c00ade20 0x7fb9c0071cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:33900/0 (socket says 172.21.15.43:33900) 2024-04-25T09:02:23.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.911+0000 7fb9bffff640 1 -- 172.21.15.43:0/242152217 learned_addr learned my addr 172.21.15.43:0/242152217 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:23.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.911+0000 7fb9beffd640 1 --2- 172.21.15.43:0/242152217 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9c00b44d0 0x7fb9c00ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:23.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.911+0000 7fb9be7fc640 1 -- 172.21.15.43:0/242152217 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2467881446 0 0) 0x7fb9c00b4930 con 0x7fb9c00ade20 2024-04-25T09:02:23.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.911+0000 7fb9be7fc640 1 -- 172.21.15.43:0/242152217 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9a8003520 con 0x7fb9c00ade20 2024-04-25T09:02:23.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.911+0000 7fb9be7fc640 1 -- 172.21.15.43:0/242152217 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4270205835 0 0) 0x7fb9c0161e30 con 0x7fb9c00ae980 2024-04-25T09:02:23.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.911+0000 7fb9be7fc640 1 -- 172.21.15.43:0/242152217 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9c00b4930 con 0x7fb9c00ae980 2024-04-25T09:02:23.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.911+0000 7fb9be7fc640 1 -- 172.21.15.43:0/242152217 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2376083762 0 0) 0x7fb9a8003520 con 0x7fb9c00ade20 2024-04-25T09:02:23.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.911+0000 7fb9be7fc640 1 -- 172.21.15.43:0/242152217 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9c00b44d0 msgr2=0x7fb9c00ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:23.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.911+0000 7fb9be7fc640 1 --2- 172.21.15.43:0/242152217 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9c00b44d0 0x7fb9c00ad8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:23.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.911+0000 7fb9be7fc640 1 -- 172.21.15.43:0/242152217 >> v1:172.21.15.193:6789/0 conn(0x7fb9c00ae980 legacy=0x7fb9c0161670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:23.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.911+0000 7fb9be7fc640 1 -- 172.21.15.43:0/242152217 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c01623f0 con 0x7fb9c00ade20 2024-04-25T09:02:23.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.911+0000 7fb9be7fc640 1 -- 172.21.15.43:0/242152217 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7fb9b80019f0 con 0x7fb9c00ade20 2024-04-25T09:02:23.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.911+0000 7fb9be7fc640 1 -- 172.21.15.43:0/242152217 >> v1:172.21.15.43:6789/0 conn(0x7fb9c00ade20 legacy=0x7fb9c0071cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.911+0000 7fb9be7fc640 1 --2- 172.21.15.43:0/242152217 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9c00b44d0 0x7fb9a8003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.911+0000 7fb9be7fc640 1 --2- 172.21.15.43:0/242152217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9a80040d0 0x7fb9a8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.911+0000 7fb9be7fc640 1 --2- 172.21.15.43:0/242152217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9a8004d40 0x7fb9a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.912+0000 7fb9be7fc640 1 -- 172.21.15.43:0/242152217 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c0162110 con 0x7fb9a8004d40 2024-04-25T09:02:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.912+0000 7fb9be7fc640 1 -- 172.21.15.43:0/242152217 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a800baa0 con 0x7fb9a80040d0 2024-04-25T09:02:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.912+0000 7fb9be7fc640 1 -- 172.21.15.43:0/242152217 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a800bca0 con 0x7fb9c00b44d0 2024-04-25T09:02:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.912+0000 7fb9bffff640 1 --2- 172.21.15.43:0/242152217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9a8004d40 0x7fb9a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.912+0000 7fb9beffd640 1 --2- 172.21.15.43:0/242152217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9a80040d0 0x7fb9a8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.912+0000 7fb9bf7fe640 1 --2- 172.21.15.43:0/242152217 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9c00b44d0 0x7fb9a8003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.912+0000 7fb9bffff640 1 -- 172.21.15.43:0/242152217 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9c00b44d0 msgr2=0x7fb9a8003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.912+0000 7fb9bffff640 1 --2- 172.21.15.43:0/242152217 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9c00b44d0 0x7fb9a8003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.912+0000 7fb9bffff640 1 -- 172.21.15.43:0/242152217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9a80040d0 msgr2=0x7fb9a8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.912+0000 7fb9bffff640 1 --2- 172.21.15.43:0/242152217 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9a80040d0 0x7fb9a8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:23.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.912+0000 7fb9bffff640 1 -- 172.21.15.43:0/242152217 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a8003520 con 0x7fb9a8004d40 2024-04-25T09:02:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.912+0000 7fb9bf7fe640 1 --2- 172.21.15.43:0/242152217 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9c00b44d0 0x7fb9a8003b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.913+0000 7fb9bffff640 1 --2- 172.21.15.43:0/242152217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9a8004d40 0x7fb9a80051c0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fb9b800a810 tx=0x7fb9a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f2cfe0188fddace server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.913+0000 7fb9be7fc640 1 -- 172.21.15.43:0/242152217 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b800a430 con 0x7fb9a8004d40 2024-04-25T09:02:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.913+0000 7fb9be7fc640 1 -- 172.21.15.43:0/242152217 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b802f070 con 0x7fb9a8004d40 2024-04-25T09:02:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.913+0000 7fb9be7fc640 1 -- 172.21.15.43:0/242152217 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b802bc90 con 0x7fb9a8004d40 2024-04-25T09:02:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.913+0000 7fb9c6110640 1 -- 172.21.15.43:0/242152217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9a8004d40 msgr2=0x7fb9a80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.913+0000 7fb9c6110640 1 --2- 172.21.15.43:0/242152217 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9a8004d40 0x7fb9a80051c0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fb9b800a810 tx=0x7fb9a8005830 comp rx=0 tx=0).stop 2024-04-25T09:02:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.913+0000 7fb9bffff640 1 -- 172.21.15.43:0/242152217 reap_dead start 2024-04-25T09:02:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.913+0000 7fb9c6110640 1 -- 172.21.15.43:0/242152217 shutdown_connections 2024-04-25T09:02:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.913+0000 7fb9c6110640 1 -- 172.21.15.43:0/242152217 >> 172.21.15.43:0/242152217 conn(0x7fb9c00b1480 msgr2=0x7fb9c00b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.913+0000 7fb9c6110640 1 -- 172.21.15.43:0/242152217 shutdown_connections 2024-04-25T09:02:23.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.913+0000 7fb9c6110640 1 -- 172.21.15.43:0/242152217 wait complete. 2024-04-25T09:02:23.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.914+0000 7fb9c6110640 1 Processor -- start 2024-04-25T09:02:23.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.915+0000 7fb9c6110640 1 -- start start 2024-04-25T09:02:23.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.915+0000 7fb9c6110640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9c01e9820 0x7fb9c00c8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:23.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.915+0000 7fb9c6110640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9c01e9c00 0x7fb9c00c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:23.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.915+0000 7fb9c6110640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9c0135250 0x7fb9c01356e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:23.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.915+0000 7fb9c6110640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c00723d0 con 0x7fb9c01e9c00 2024-04-25T09:02:23.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.915+0000 7fb9c6110640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c0071490 con 0x7fb9c0135250 2024-04-25T09:02:23.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.915+0000 7fb9c6110640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c0072550 con 0x7fb9c01e9820 2024-04-25T09:02:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.915+0000 7fb9beffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9c01e9c00 0x7fb9c00c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.915+0000 7fb9beffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9c01e9c00 0x7fb9c00c92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44932/0 (socket says 172.21.15.43:44932) 2024-04-25T09:02:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.915+0000 7fb9beffd640 1 -- 172.21.15.43:0/361850978 learned_addr learned my addr 172.21.15.43:0/361850978 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.916+0000 7fb9bffff640 1 --2- 172.21.15.43:0/361850978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9c0135250 0x7fb9c01356e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.916+0000 7fb9bf7fe640 1 --2- 172.21.15.43:0/361850978 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9c01e9820 0x7fb9c00c8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.916+0000 7fb9beffd640 1 -- 172.21.15.43:0/361850978 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9c01e9820 msgr2=0x7fb9c00c8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.916+0000 7fb9beffd640 1 --2- 172.21.15.43:0/361850978 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9c01e9820 0x7fb9c00c8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.916+0000 7fb9beffd640 1 -- 172.21.15.43:0/361850978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9c0135250 msgr2=0x7fb9c01356e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.916+0000 7fb9beffd640 1 --2- 172.21.15.43:0/361850978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9c0135250 0x7fb9c01356e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.916+0000 7fb9beffd640 1 -- 172.21.15.43:0/361850978 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c0138ee0 con 0x7fb9c01e9c00 2024-04-25T09:02:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.916+0000 7fb9beffd640 1 --2- 172.21.15.43:0/361850978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9c01e9c00 0x7fb9c00c92f0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fb9b40118e0 tx=0x7fb9b4011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.916+0000 7fb9a7fff640 1 -- 172.21.15.43:0/361850978 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b400faf0 con 0x7fb9c01e9c00 2024-04-25T09:02:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.916+0000 7fb9a7fff640 1 -- 172.21.15.43:0/361850978 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b400fc90 con 0x7fb9c01e9c00 2024-04-25T09:02:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.916+0000 7fb9c6110640 1 -- 172.21.15.43:0/361850978 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9c0135fb0 con 0x7fb9c01e9c00 2024-04-25T09:02:23.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.916+0000 7fb9a7fff640 1 -- 172.21.15.43:0/361850978 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4014440 con 0x7fb9c01e9c00 2024-04-25T09:02:23.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.916+0000 7fb9c6110640 1 -- 172.21.15.43:0/361850978 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9c01333b0 con 0x7fb9c01e9c00 2024-04-25T09:02:23.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.917+0000 7fb9c6110640 1 -- 172.21.15.43:0/361850978 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9c0074ef0 con 0x7fb9c01e9c00 2024-04-25T09:02:23.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.918+0000 7fb9a7fff640 1 -- 172.21.15.43:0/361850978 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb9b4012040 con 0x7fb9c01e9c00 2024-04-25T09:02:23.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.918+0000 7fb9a7fff640 1 --2- 172.21.15.43:0/361850978 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb9ac06b4d0 0x7fb9ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:23.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.918+0000 7fb9a7fff640 1 -- 172.21.15.43:0/361850978 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb9b408a3b0 con 0x7fb9c01e9c00 2024-04-25T09:02:23.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.918+0000 7fb9bf7fe640 1 --2- 172.21.15.43:0/361850978 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb9ac06b4d0 0x7fb9ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:23.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.919+0000 7fb9bf7fe640 1 --2- 172.21.15.43:0/361850978 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb9ac06b4d0 0x7fb9ac06d990 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0007ae0 tx=0x7fb9b0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:23.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:23.922+0000 7fb9a7fff640 1 -- 172.21.15.43:0/361850978 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb9b4019030 con 0x7fb9c01e9c00 2024-04-25T09:02:24.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.104+0000 7fb9c6110640 1 -- 172.21.15.43:0/361850978 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb9c00b1de0 con 0x7fb9c01e9c00 2024-04-25T09:02:24.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.104+0000 7fb9a7fff640 1 -- 172.21.15.43:0/361850978 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v109) v1 ==== 84+0+65 (secure 0 0 0) 0x7fb9b4058fc0 con 0x7fb9c01e9c00 2024-04-25T09:02:24.093 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:24.093 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2024-04-25T09:02:24.093 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:02:24.093 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 1, 2024-04-25T09:02:24.094 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": ".mgr" 2024-04-25T09:02:24.094 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:02:24.094 INFO:teuthology.orchestra.run.smithi043.stdout:] 2024-04-25T09:02:24.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.106+0000 7fb9c6110640 1 -- 172.21.15.43:0/361850978 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb9ac06b4d0 msgr2=0x7fb9ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:24.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.106+0000 7fb9c6110640 1 --2- 172.21.15.43:0/361850978 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb9ac06b4d0 0x7fb9ac06d990 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0007ae0 tx=0x7fb9b0009040 comp rx=0 tx=0).stop 2024-04-25T09:02:24.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.106+0000 7fb9c6110640 1 -- 172.21.15.43:0/361850978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9c01e9c00 msgr2=0x7fb9c00c92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:24.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.106+0000 7fb9c6110640 1 --2- 172.21.15.43:0/361850978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9c01e9c00 0x7fb9c00c92f0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fb9b40118e0 tx=0x7fb9b4011da0 comp rx=0 tx=0).stop 2024-04-25T09:02:24.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.106+0000 7fb9c6110640 1 -- 172.21.15.43:0/361850978 shutdown_connections 2024-04-25T09:02:24.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.106+0000 7fb9c6110640 1 --2- 172.21.15.43:0/361850978 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb9ac06b4d0 0x7fb9ac06d990 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:24.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.106+0000 7fb9c6110640 1 --2- 172.21.15.43:0/361850978 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9c0135250 0x7fb9c01356e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:24.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.106+0000 7fb9c6110640 1 --2- 172.21.15.43:0/361850978 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9c01e9c00 0x7fb9c00c92f0 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:24.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.106+0000 7fb9c6110640 1 --2- 172.21.15.43:0/361850978 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9c01e9820 0x7fb9c00c8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:24.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.106+0000 7fb9c6110640 1 -- 172.21.15.43:0/361850978 >> 172.21.15.43:0/361850978 conn(0x7fb9c00b1480 msgr2=0x7fb9c00b3aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:24.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.106+0000 7fb9c6110640 1 -- 172.21.15.43:0/361850978 shutdown_connections 2024-04-25T09:02:24.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.106+0000 7fb9c6110640 1 -- 172.21.15.43:0/361850978 wait complete. 2024-04-25T09:02:24.106 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:02:24.107 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:02:24.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.223+0000 7f63e515c640 1 Processor -- start 2024-04-25T09:02:24.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.223+0000 7f63e515c640 1 -- start start 2024-04-25T09:02:24.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.223+0000 7f63e515c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f63e015baa0 0x7f63e015e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:24.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.223+0000 7f63e515c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f63e00b4ea0 0x7f63e00d3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:24.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.223+0000 7f63e515c640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63e00b37a0 con 0x7f63e015b6c0 2024-04-25T09:02:24.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.223+0000 7f63e515c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63e015fce0 con 0x7f63e015baa0 2024-04-25T09:02:24.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.223+0000 7f63e515c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63e015eab0 con 0x7f63e00b4ea0 2024-04-25T09:02:24.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.224+0000 7f63de59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f63e015baa0 0x7f63e015e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:24.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.224+0000 7f63de59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f63e015baa0 0x7f63e015e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44936/0 (socket says 172.21.15.43:44936) 2024-04-25T09:02:24.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.224+0000 7f63de59c640 1 -- 172.21.15.43:0/1969366677 learned_addr learned my addr 172.21.15.43:0/1969366677 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:24.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.224+0000 7f63df59e640 1 --2- 172.21.15.43:0/1969366677 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f63e00b4ea0 0x7f63e00d3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:24.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.224+0000 7f63de59c640 1 -- 172.21.15.43:0/1969366677 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f63e00b4ea0 msgr2=0x7f63e00d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:24.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.224+0000 7f63de59c640 1 --2- 172.21.15.43:0/1969366677 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f63e00b4ea0 0x7f63e00d3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:24.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.224+0000 7f63de59c640 1 -- 172.21.15.43:0/1969366677 >> v1:172.21.15.193:6789/0 conn(0x7f63e015b6c0 legacy=0x7f63e015f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:24.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.224+0000 7f63de59c640 1 -- 172.21.15.43:0/1969366677 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e015ed90 con 0x7f63e015baa0 2024-04-25T09:02:24.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.225+0000 7f63de59c640 1 --2- 172.21.15.43:0/1969366677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63e015baa0 0x7f63e015e410 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f63c8009870 tx=0x7f63c802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a51036c105f294 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:24.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.225+0000 7f63ddd9b640 1 -- 172.21.15.43:0/1969366677 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c8037070 con 0x7f63e015baa0 2024-04-25T09:02:24.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.225+0000 7f63ddd9b640 1 -- 172.21.15.43:0/1969366677 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63c8032440 con 0x7f63e015baa0 2024-04-25T09:02:24.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.225+0000 7f63ddd9b640 1 -- 172.21.15.43:0/1969366677 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c8032810 con 0x7f63e015baa0 2024-04-25T09:02:24.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.225+0000 7f63e515c640 1 -- 172.21.15.43:0/1969366677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63e015baa0 msgr2=0x7f63e015e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:24.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.225+0000 7f63e515c640 1 --2- 172.21.15.43:0/1969366677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63e015baa0 0x7f63e015e410 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f63c8009870 tx=0x7f63c802a270 comp rx=0 tx=0).stop 2024-04-25T09:02:24.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.225+0000 7f63e515c640 1 -- 172.21.15.43:0/1969366677 shutdown_connections 2024-04-25T09:02:24.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.225+0000 7f63e515c640 1 --2- 172.21.15.43:0/1969366677 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f63e00b4ea0 0x7f63e00d3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:24.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.225+0000 7f63e515c640 1 --2- 172.21.15.43:0/1969366677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63e015baa0 0x7f63e015e410 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:24.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.225+0000 7f63e515c640 1 -- 172.21.15.43:0/1969366677 >> 172.21.15.43:0/1969366677 conn(0x7f63e008d460 msgr2=0x7f63e009cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:24.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.225+0000 7f63e515c640 1 -- 172.21.15.43:0/1969366677 shutdown_connections 2024-04-25T09:02:24.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.225+0000 7f63e515c640 1 -- 172.21.15.43:0/1969366677 wait complete. 2024-04-25T09:02:24.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.227+0000 7f63e515c640 1 Processor -- start 2024-04-25T09:02:24.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.227+0000 7f63e515c640 1 -- start start 2024-04-25T09:02:24.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.228+0000 7f63e515c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f63e00b4ea0 0x7f63e00c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:24.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.228+0000 7f63e515c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63e015b6c0 0x7f63e00b6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:24.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.228+0000 7f63e515c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63e015baa0 0x7f63e01f6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:24.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.228+0000 7f63e515c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e00b52e0 con 0x7f63e015baa0 2024-04-25T09:02:24.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.228+0000 7f63e515c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e015f350 con 0x7f63e015b6c0 2024-04-25T09:02:24.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.228+0000 7f63e515c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63e00d3ca0 con 0x7f63e00b4ea0 2024-04-25T09:02:24.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.228+0000 7f63df59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63e015baa0 0x7f63e01f6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:24.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.228+0000 7f63df59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63e015baa0 0x7f63e01f6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44952/0 (socket says 172.21.15.43:44952) 2024-04-25T09:02:24.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.228+0000 7f63df59e640 1 -- 172.21.15.43:0/4115163701 learned_addr learned my addr 172.21.15.43:0/4115163701 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:24.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.228+0000 7f63de59c640 1 --2- 172.21.15.43:0/4115163701 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63e015b6c0 0x7f63e00b6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:24.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.228+0000 7f63ded9d640 1 --2- 172.21.15.43:0/4115163701 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f63e00b4ea0 0x7f63e00c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:24.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.228+0000 7f63df59e640 1 -- 172.21.15.43:0/4115163701 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f63e00b4ea0 msgr2=0x7f63e00c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:24.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.228+0000 7f63df59e640 1 --2- 172.21.15.43:0/4115163701 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f63e00b4ea0 0x7f63e00c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:24.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.228+0000 7f63df59e640 1 -- 172.21.15.43:0/4115163701 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63e015b6c0 msgr2=0x7f63e00b6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:24.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.228+0000 7f63df59e640 1 --2- 172.21.15.43:0/4115163701 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63e015b6c0 0x7f63e00b6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:24.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.228+0000 7f63df59e640 1 -- 172.21.15.43:0/4115163701 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e00c8390 con 0x7f63e015baa0 2024-04-25T09:02:24.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.228+0000 7f63df59e640 1 --2- 172.21.15.43:0/4115163701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63e015baa0 0x7f63e01f6ac0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f63d000efc0 tx=0x7f63d000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:24.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.228+0000 7f63d77fe640 1 -- 172.21.15.43:0/4115163701 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d0018020 con 0x7f63e015baa0 2024-04-25T09:02:24.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.228+0000 7f63d77fe640 1 -- 172.21.15.43:0/4115163701 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d0014480 con 0x7f63e015baa0 2024-04-25T09:02:24.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.228+0000 7f63e515c640 1 -- 172.21.15.43:0/4115163701 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63e00c8670 con 0x7f63e015baa0 2024-04-25T09:02:24.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.228+0000 7f63d77fe640 1 -- 172.21.15.43:0/4115163701 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d0014870 con 0x7f63e015baa0 2024-04-25T09:02:24.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.229+0000 7f63e515c640 1 -- 172.21.15.43:0/4115163701 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63e00c8c30 con 0x7f63e015baa0 2024-04-25T09:02:24.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.230+0000 7f63e515c640 1 -- 172.21.15.43:0/4115163701 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63e0068a40 con 0x7f63e015baa0 2024-04-25T09:02:24.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.230+0000 7f63d77fe640 1 -- 172.21.15.43:0/4115163701 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f63d0005d10 con 0x7f63e015baa0 2024-04-25T09:02:24.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.230+0000 7f63d77fe640 1 --2- 172.21.15.43:0/4115163701 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f63c006b500 0x7f63c006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:24.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.231+0000 7f63d77fe640 1 -- 172.21.15.43:0/4115163701 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f63d008a9e0 con 0x7f63e015baa0 2024-04-25T09:02:24.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.231+0000 7f63ded9d640 1 --2- 172.21.15.43:0/4115163701 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f63c006b500 0x7f63c006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:24.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.231+0000 7f63ded9d640 1 --2- 172.21.15.43:0/4115163701 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f63c006b500 0x7f63c006d9c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f63d8001640 tx=0x7f63d8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:24.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.235+0000 7f63d77fe640 1 -- 172.21.15.43:0/4115163701 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f63d0022070 con 0x7f63e015baa0 2024-04-25T09:02:24.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.415+0000 7f63e515c640 1 -- 172.21.15.43:0/4115163701 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f63e00d2940 con 0x7f63e015baa0 2024-04-25T09:02:24.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.784+0000 7f63d77fe640 1 -- 172.21.15.43:0/4115163701 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v110) v1 ==== 138+0+0 (secure 0 0 0) 0x7f63d0010040 con 0x7f63e015baa0 2024-04-25T09:02:24.773 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' created 2024-04-25T09:02:24.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.786+0000 7f63e515c640 1 -- 172.21.15.43:0/4115163701 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f63c006b500 msgr2=0x7f63c006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:24.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.786+0000 7f63e515c640 1 --2- 172.21.15.43:0/4115163701 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f63c006b500 0x7f63c006d9c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f63d8001640 tx=0x7f63d8009040 comp rx=0 tx=0).stop 2024-04-25T09:02:24.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.786+0000 7f63e515c640 1 -- 172.21.15.43:0/4115163701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63e015baa0 msgr2=0x7f63e01f6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:24.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.786+0000 7f63e515c640 1 --2- 172.21.15.43:0/4115163701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63e015baa0 0x7f63e01f6ac0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f63d000efc0 tx=0x7f63d000c490 comp rx=0 tx=0).stop 2024-04-25T09:02:24.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.786+0000 7f63e515c640 1 -- 172.21.15.43:0/4115163701 shutdown_connections 2024-04-25T09:02:24.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.786+0000 7f63e515c640 1 --2- 172.21.15.43:0/4115163701 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f63e015baa0 0x7f63e01f6ac0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:24.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.786+0000 7f63e515c640 1 --2- 172.21.15.43:0/4115163701 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f63c006b500 0x7f63c006d9c0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:24.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.786+0000 7f63e515c640 1 --2- 172.21.15.43:0/4115163701 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f63e015b6c0 0x7f63e00b6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:24.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.786+0000 7f63e515c640 1 --2- 172.21.15.43:0/4115163701 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f63e00b4ea0 0x7f63e00c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:24.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.786+0000 7f63e515c640 1 -- 172.21.15.43:0/4115163701 >> 172.21.15.43:0/4115163701 conn(0x7f63e008d460 msgr2=0x7f63e00b0750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:24.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.787+0000 7f63e515c640 1 -- 172.21.15.43:0/4115163701 shutdown_connections 2024-04-25T09:02:24.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.787+0000 7f63e515c640 1 -- 172.21.15.43:0/4115163701 wait complete. 2024-04-25T09:02:24.792 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:02:24.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.913+0000 7f8571fb0640 1 Processor -- start 2024-04-25T09:02:24.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.913+0000 7f8571fb0640 1 -- start start 2024-04-25T09:02:24.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.914+0000 7f8571fb0640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f856c161960 0x7f856c0b23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:24.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.914+0000 7f8571fb0640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f856c0b2990 0x7f856c161d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:24.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.914+0000 7f8571fb0640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f856c162830 con 0x7f856c162280 2024-04-25T09:02:24.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.914+0000 7f8571fb0640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f856c0d0e30 con 0x7f856c0b2990 2024-04-25T09:02:24.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.914+0000 7f8571fb0640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f856c0d0590 con 0x7f856c161960 2024-04-25T09:02:24.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.914+0000 7f856affd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f856c0b2990 0x7f856c161d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:24.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.914+0000 7f856bfff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f856c162280 0x7f856c0b1160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:33914/0 (socket says 172.21.15.43:33914) 2024-04-25T09:02:24.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.914+0000 7f856bfff640 1 -- 172.21.15.43:0/2232483326 learned_addr learned my addr 172.21.15.43:0/2232483326 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:24.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.915+0000 7f856affd640 1 -- 172.21.15.43:0/2232483326 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f856c161960 msgr2=0x7f856c0b23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:24.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.915+0000 7f856b7fe640 1 --2- 172.21.15.43:0/2232483326 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f856c161960 0x7f856c0b23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:24.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.915+0000 7f856a7fc640 1 -- 172.21.15.43:0/2232483326 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3565286599 0 0) 0x7f856c162830 con 0x7f856c162280 2024-04-25T09:02:24.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.915+0000 7f856affd640 1 --2- 172.21.15.43:0/2232483326 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f856c161960 0x7f856c0b23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:24.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.915+0000 7f856affd640 1 -- 172.21.15.43:0/2232483326 >> v1:172.21.15.43:6789/0 conn(0x7f856c162280 legacy=0x7f856c0b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:24.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.915+0000 7f856affd640 1 -- 172.21.15.43:0/2232483326 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f856c0d0870 con 0x7f856c0b2990 2024-04-25T09:02:24.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.915+0000 7f856b7fe640 1 --2- 172.21.15.43:0/2232483326 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f856c161960 0x7f856c0b23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:24.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.915+0000 7f856affd640 1 --2- 172.21.15.43:0/2232483326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f856c0b2990 0x7f856c161d40 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f85600097a0 tx=0x7f856002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c10459888516116f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:24.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.916+0000 7f856a7fc640 1 -- 172.21.15.43:0/2232483326 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8560037070 con 0x7f856c0b2990 2024-04-25T09:02:24.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.916+0000 7f856a7fc640 1 -- 172.21.15.43:0/2232483326 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8560032440 con 0x7f856c0b2990 2024-04-25T09:02:24.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.916+0000 7f856a7fc640 1 -- 172.21.15.43:0/2232483326 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8560032810 con 0x7f856c0b2990 2024-04-25T09:02:24.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.916+0000 7f8571fb0640 1 -- 172.21.15.43:0/2232483326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f856c0b2990 msgr2=0x7f856c161d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:24.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.916+0000 7f8571fb0640 1 --2- 172.21.15.43:0/2232483326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f856c0b2990 0x7f856c161d40 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f85600097a0 tx=0x7f856002a180 comp rx=0 tx=0).stop 2024-04-25T09:02:24.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.916+0000 7f8571fb0640 1 -- 172.21.15.43:0/2232483326 shutdown_connections 2024-04-25T09:02:24.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.916+0000 7f8571fb0640 1 --2- 172.21.15.43:0/2232483326 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f856c0b2990 0x7f856c161d40 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:24.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.916+0000 7f8571fb0640 1 --2- 172.21.15.43:0/2232483326 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f856c161960 0x7f856c0b23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:24.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.916+0000 7f8571fb0640 1 -- 172.21.15.43:0/2232483326 >> 172.21.15.43:0/2232483326 conn(0x7f856c0b3330 msgr2=0x7f856c0b3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:24.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.916+0000 7f8571fb0640 1 -- 172.21.15.43:0/2232483326 shutdown_connections 2024-04-25T09:02:24.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.916+0000 7f8571fb0640 1 -- 172.21.15.43:0/2232483326 wait complete. 2024-04-25T09:02:24.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.917+0000 7f8571fb0640 1 Processor -- start 2024-04-25T09:02:24.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.918+0000 7f8571fb0640 1 -- start start 2024-04-25T09:02:24.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.918+0000 7f8571fb0640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f856c0b2990 0x7f856c0c7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:24.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.918+0000 7f8571fb0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f856c161960 0x7f856c0b0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:24.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.918+0000 7f8571fb0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f856c162280 0x7f856c0b1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:24.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.918+0000 7f8571fb0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f856c15f760 con 0x7f856c162280 2024-04-25T09:02:24.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.918+0000 7f8571fb0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f856c161620 con 0x7f856c161960 2024-04-25T09:02:24.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.918+0000 7f8571fb0640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f856c15f8e0 con 0x7f856c0b2990 2024-04-25T09:02:24.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.918+0000 7f856bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f856c162280 0x7f856c0b1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:24.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.918+0000 7f856bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f856c162280 0x7f856c0b1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44956/0 (socket says 172.21.15.43:44956) 2024-04-25T09:02:24.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.918+0000 7f856bfff640 1 -- 172.21.15.43:0/1630839322 learned_addr learned my addr 172.21.15.43:0/1630839322 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:24.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.919+0000 7f856b7fe640 1 --2- 172.21.15.43:0/1630839322 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f856c0b2990 0x7f856c0c7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:24.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.919+0000 7f856affd640 1 --2- 172.21.15.43:0/1630839322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f856c161960 0x7f856c0b0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:24.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.919+0000 7f856bfff640 1 -- 172.21.15.43:0/1630839322 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f856c0b2990 msgr2=0x7f856c0c7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:24.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.919+0000 7f856bfff640 1 --2- 172.21.15.43:0/1630839322 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f856c0b2990 0x7f856c0c7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:24.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.919+0000 7f856bfff640 1 -- 172.21.15.43:0/1630839322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f856c161960 msgr2=0x7f856c0b0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:24.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.919+0000 7f856bfff640 1 --2- 172.21.15.43:0/1630839322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f856c161960 0x7f856c0b0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:24.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.919+0000 7f856bfff640 1 -- 172.21.15.43:0/1630839322 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f856c0b54a0 con 0x7f856c162280 2024-04-25T09:02:24.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.919+0000 7f856affd640 1 --2- 172.21.15.43:0/1630839322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f856c161960 0x7f856c0b0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:24.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.919+0000 7f856bfff640 1 --2- 172.21.15.43:0/1630839322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f856c162280 0x7f856c0b1370 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f856400c7d0 tx=0x7f856400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:24.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.919+0000 7f8553fff640 1 -- 172.21.15.43:0/1630839322 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8564018070 con 0x7f856c162280 2024-04-25T09:02:24.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.919+0000 7f8553fff640 1 -- 172.21.15.43:0/1630839322 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f856400a640 con 0x7f856c162280 2024-04-25T09:02:24.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.919+0000 7f8571fb0640 1 -- 172.21.15.43:0/1630839322 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f856c0b4400 con 0x7f856c162280 2024-04-25T09:02:24.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.919+0000 7f8553fff640 1 -- 172.21.15.43:0/1630839322 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f856400aa70 con 0x7f856c162280 2024-04-25T09:02:24.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.919+0000 7f8571fb0640 1 -- 172.21.15.43:0/1630839322 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f856c0b49c0 con 0x7f856c162280 2024-04-25T09:02:24.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.920+0000 7f8571fb0640 1 -- 172.21.15.43:0/1630839322 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f856c074f60 con 0x7f856c162280 2024-04-25T09:02:24.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.921+0000 7f8553fff640 1 -- 172.21.15.43:0/1630839322 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f856401b070 con 0x7f856c162280 2024-04-25T09:02:24.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.921+0000 7f8553fff640 1 --2- 172.21.15.43:0/1630839322 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f855806b4d0 0x7f855806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:24.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.921+0000 7f8553fff640 1 -- 172.21.15.43:0/1630839322 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f8564089550 con 0x7f856c162280 2024-04-25T09:02:24.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.922+0000 7f856b7fe640 1 --2- 172.21.15.43:0/1630839322 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f855806b4d0 0x7f855806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:24.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.922+0000 7f856b7fe640 1 --2- 172.21.15.43:0/1630839322 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f855806b4d0 0x7f855806d990 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f855c00ab60 tx=0x7f855c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:24.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:24.926+0000 7f8553fff640 1 -- 172.21.15.43:0/1630839322 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8564057f60 con 0x7f856c162280 2024-04-25T09:02:25.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.112+0000 7f8571fb0640 1 -- 172.21.15.43:0/1630839322 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f856c0683b0 con 0x7f856c162280 2024-04-25T09:02:25.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.786+0000 7f8553fff640 1 -- 172.21.15.43:0/1630839322 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v111) v1 ==== 170+0+0 (secure 0 0 0) 0x7f856405c000 con 0x7f856c162280 2024-04-25T09:02:25.775 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' created 2024-04-25T09:02:25.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.789+0000 7f8571fb0640 1 -- 172.21.15.43:0/1630839322 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f855806b4d0 msgr2=0x7f855806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:25.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.789+0000 7f8571fb0640 1 --2- 172.21.15.43:0/1630839322 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f855806b4d0 0x7f855806d990 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f855c00ab60 tx=0x7f855c008040 comp rx=0 tx=0).stop 2024-04-25T09:02:25.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.789+0000 7f8571fb0640 1 -- 172.21.15.43:0/1630839322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f856c162280 msgr2=0x7f856c0b1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:25.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.789+0000 7f8571fb0640 1 --2- 172.21.15.43:0/1630839322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f856c162280 0x7f856c0b1370 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f856400c7d0 tx=0x7f856400cc90 comp rx=0 tx=0).stop 2024-04-25T09:02:25.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.789+0000 7f8571fb0640 1 -- 172.21.15.43:0/1630839322 shutdown_connections 2024-04-25T09:02:25.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.789+0000 7f8571fb0640 1 --2- 172.21.15.43:0/1630839322 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f856c162280 0x7f856c0b1370 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:25.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.789+0000 7f8571fb0640 1 --2- 172.21.15.43:0/1630839322 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f855806b4d0 0x7f855806d990 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:25.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.789+0000 7f8571fb0640 1 --2- 172.21.15.43:0/1630839322 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f856c161960 0x7f856c0b0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:25.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.789+0000 7f8571fb0640 1 --2- 172.21.15.43:0/1630839322 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f856c0b2990 0x7f856c0c7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:25.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.790+0000 7f8571fb0640 1 -- 172.21.15.43:0/1630839322 >> 172.21.15.43:0/1630839322 conn(0x7f856c0b3330 msgr2=0x7f856c06be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:25.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.790+0000 7f8571fb0640 1 -- 172.21.15.43:0/1630839322 shutdown_connections 2024-04-25T09:02:25.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.790+0000 7f8571fb0640 1 -- 172.21.15.43:0/1630839322 wait complete. 2024-04-25T09:02:25.809 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:02:25.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.929+0000 7fd038f90640 1 Processor -- start 2024-04-25T09:02:25.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.929+0000 7fd038f90640 1 -- start start 2024-04-25T09:02:25.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.930+0000 7fd038f90640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fd03406f6a0 0x7fd0340b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:25.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.930+0000 7fd038f90640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0340d2620 con 0x7fd03415ddd0 2024-04-25T09:02:25.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.930+0000 7fd038f90640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0340b0b80 con 0x7fd0340cd470 2024-04-25T09:02:25.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.930+0000 7fd038f90640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0340b0e60 con 0x7fd03406f6a0 2024-04-25T09:02:25.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.930+0000 7fd03259c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fd03415ddd0 0x7fd03406ef90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:33922/0 (socket says 172.21.15.43:33922) 2024-04-25T09:02:25.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.930+0000 7fd03259c640 1 -- 172.21.15.43:0/2356670311 learned_addr learned my addr 172.21.15.43:0/2356670311 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:25.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.930+0000 7fd031d9b640 1 --2- 172.21.15.43:0/2356670311 >> v2:172.21.15.193:3300/0 conn(0x7fd03406f6a0 0x7fd0340b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:25.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.930+0000 7fd03159a640 1 -- 172.21.15.43:0/2356670311 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 36222850 0 0) 0x7fd0340d2620 con 0x7fd03415ddd0 2024-04-25T09:02:25.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.930+0000 7fd03159a640 1 -- 172.21.15.43:0/2356670311 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd01c003520 con 0x7fd03415ddd0 2024-04-25T09:02:25.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.930+0000 7fd03159a640 1 -- 172.21.15.43:0/2356670311 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 218579367 0 0) 0x7fd01c003520 con 0x7fd03415ddd0 2024-04-25T09:02:25.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.930+0000 7fd03159a640 1 -- 172.21.15.43:0/2356670311 >> v2:172.21.15.193:3300/0 conn(0x7fd03406f6a0 msgr2=0x7fd0340b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:25.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.930+0000 7fd03159a640 1 --2- 172.21.15.43:0/2356670311 >> v2:172.21.15.193:3300/0 conn(0x7fd03406f6a0 0x7fd0340b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:25.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.930+0000 7fd03159a640 1 -- 172.21.15.43:0/2356670311 >> v1:172.21.15.193:6789/0 conn(0x7fd0340cd470 legacy=0x7fd03406c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:25.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.930+0000 7fd03159a640 1 -- 172.21.15.43:0/2356670311 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0340cbf40 con 0x7fd03415ddd0 2024-04-25T09:02:25.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.930+0000 7fd03159a640 1 -- 172.21.15.43:0/2356670311 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7fd024001670 con 0x7fd03415ddd0 2024-04-25T09:02:25.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.930+0000 7fd03159a640 1 -- 172.21.15.43:0/2356670311 >> v1:172.21.15.43:6789/0 conn(0x7fd03415ddd0 legacy=0x7fd03406ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:25.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.930+0000 7fd03159a640 1 --2- 172.21.15.43:0/2356670311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd01c0039b0 0x7fd01c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:25.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.930+0000 7fd03159a640 1 --2- 172.21.15.43:0/2356670311 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd01c004800 0x7fd01c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:25.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.930+0000 7fd03159a640 1 --2- 172.21.15.43:0/2356670311 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd01c0051c0 0x7fd01c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:25.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.930+0000 7fd03159a640 1 -- 172.21.15.43:0/2356670311 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0340b0e60 con 0x7fd01c0039b0 2024-04-25T09:02:25.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.930+0000 7fd03159a640 1 -- 172.21.15.43:0/2356670311 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd01c00c060 con 0x7fd01c004800 2024-04-25T09:02:25.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.930+0000 7fd03159a640 1 -- 172.21.15.43:0/2356670311 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd01c00c260 con 0x7fd01c0051c0 2024-04-25T09:02:25.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.931+0000 7fd03259c640 1 --2- 172.21.15.43:0/2356670311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd01c0039b0 0x7fd01c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:25.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.931+0000 7fd031d9b640 1 --2- 172.21.15.43:0/2356670311 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd01c004800 0x7fd01c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:25.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.931+0000 7fd032d9d640 1 --2- 172.21.15.43:0/2356670311 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd01c0051c0 0x7fd01c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:25.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.931+0000 7fd03259c640 1 -- 172.21.15.43:0/2356670311 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd01c0051c0 msgr2=0x7fd01c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:25.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.931+0000 7fd03259c640 1 --2- 172.21.15.43:0/2356670311 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd01c0051c0 0x7fd01c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:25.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.931+0000 7fd03259c640 1 -- 172.21.15.43:0/2356670311 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd01c004800 msgr2=0x7fd01c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:25.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.931+0000 7fd03259c640 1 --2- 172.21.15.43:0/2356670311 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd01c004800 0x7fd01c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:25.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.931+0000 7fd032d9d640 1 -- 172.21.15.43:0/2356670311 reap_dead start 2024-04-25T09:02:25.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.931+0000 7fd03259c640 1 -- 172.21.15.43:0/2356670311 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd01c003520 con 0x7fd01c0039b0 2024-04-25T09:02:25.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.931+0000 7fd031d9b640 1 --2- 172.21.15.43:0/2356670311 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd01c004800 0x7fd01c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:02:25.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.932+0000 7fd03259c640 1 --2- 172.21.15.43:0/2356670311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd01c0039b0 0x7fd01c003dd0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fd02400c210 tx=0x7fd01c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=363aeca1d42007c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:25.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.932+0000 7fd03159a640 1 -- 172.21.15.43:0/2356670311 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd024007d30 con 0x7fd01c0039b0 2024-04-25T09:02:25.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.932+0000 7fd03159a640 1 -- 172.21.15.43:0/2356670311 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0240058d0 con 0x7fd01c0039b0 2024-04-25T09:02:25.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.932+0000 7fd03159a640 1 -- 172.21.15.43:0/2356670311 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd024005d20 con 0x7fd01c0039b0 2024-04-25T09:02:25.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.932+0000 7fd038f90640 1 -- 172.21.15.43:0/2356670311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd01c0039b0 msgr2=0x7fd01c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:25.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.932+0000 7fd038f90640 1 --2- 172.21.15.43:0/2356670311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd01c0039b0 0x7fd01c003dd0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7fd02400c210 tx=0x7fd01c005830 comp rx=0 tx=0).stop 2024-04-25T09:02:25.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.932+0000 7fd038f90640 1 -- 172.21.15.43:0/2356670311 shutdown_connections 2024-04-25T09:02:25.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.932+0000 7fd038f90640 1 --2- 172.21.15.43:0/2356670311 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd01c0039b0 0x7fd01c003dd0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:25.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.932+0000 7fd038f90640 1 -- 172.21.15.43:0/2356670311 >> 172.21.15.43:0/2356670311 conn(0x7fd0340d17e0 msgr2=0x7fd0340d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:25.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.932+0000 7fd038f90640 1 -- 172.21.15.43:0/2356670311 shutdown_connections 2024-04-25T09:02:25.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.932+0000 7fd038f90640 1 -- 172.21.15.43:0/2356670311 wait complete. 2024-04-25T09:02:25.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.934+0000 7fd038f90640 1 Processor -- start 2024-04-25T09:02:25.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.934+0000 7fd038f90640 1 -- start start 2024-04-25T09:02:25.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.934+0000 7fd038f90640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd01c0039b0 0x7fd0340b6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:25.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.934+0000 7fd038f90640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd03415aee0 0x7fd0341f7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:25.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.934+0000 7fd038f90640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd0340afb00 0x7fd0340affb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:25.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.934+0000 7fd038f90640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0340b4160 con 0x7fd03415aee0 2024-04-25T09:02:25.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.934+0000 7fd038f90640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0340b3fe0 con 0x7fd01c0039b0 2024-04-25T09:02:25.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.934+0000 7fd038f90640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd03406ef90 con 0x7fd0340afb00 2024-04-25T09:02:25.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.935+0000 7fd031d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd03415aee0 0x7fd0341f7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:25.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.935+0000 7fd031d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd03415aee0 0x7fd0341f7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44988/0 (socket says 172.21.15.43:44988) 2024-04-25T09:02:25.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.935+0000 7fd031d9b640 1 -- 172.21.15.43:0/2398722539 learned_addr learned my addr 172.21.15.43:0/2398722539 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:25.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.935+0000 7fd032d9d640 1 --2- 172.21.15.43:0/2398722539 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd0340afb00 0x7fd0340affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:25.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.935+0000 7fd03259c640 1 --2- 172.21.15.43:0/2398722539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd01c0039b0 0x7fd0340b6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:25.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.935+0000 7fd031d9b640 1 -- 172.21.15.43:0/2398722539 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd0340afb00 msgr2=0x7fd0340affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:25.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.935+0000 7fd031d9b640 1 --2- 172.21.15.43:0/2398722539 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd0340afb00 0x7fd0340affb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:25.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.935+0000 7fd031d9b640 1 -- 172.21.15.43:0/2398722539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd01c0039b0 msgr2=0x7fd0340b6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:25.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.935+0000 7fd031d9b640 1 --2- 172.21.15.43:0/2398722539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd01c0039b0 0x7fd0340b6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:25.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.935+0000 7fd031d9b640 1 -- 172.21.15.43:0/2398722539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0341608a0 con 0x7fd03415aee0 2024-04-25T09:02:25.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.935+0000 7fd03259c640 1 --2- 172.21.15.43:0/2398722539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd01c0039b0 0x7fd0340b6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:25.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.935+0000 7fd032d9d640 1 --2- 172.21.15.43:0/2398722539 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd0340afb00 0x7fd0340affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:25.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.935+0000 7fd031d9b640 1 --2- 172.21.15.43:0/2398722539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd03415aee0 0x7fd0341f7f40 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fd028006c00 tx=0x7fd02800f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:25.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.935+0000 7fd022ffd640 1 -- 172.21.15.43:0/2398722539 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd028009690 con 0x7fd03415aee0 2024-04-25T09:02:25.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.935+0000 7fd022ffd640 1 -- 172.21.15.43:0/2398722539 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd028011c90 con 0x7fd03415aee0 2024-04-25T09:02:25.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.935+0000 7fd022ffd640 1 -- 172.21.15.43:0/2398722539 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd028014440 con 0x7fd03415aee0 2024-04-25T09:02:25.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.935+0000 7fd038f90640 1 -- 172.21.15.43:0/2398722539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd03415ebc0 con 0x7fd03415aee0 2024-04-25T09:02:25.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.935+0000 7fd038f90640 1 -- 172.21.15.43:0/2398722539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd034160b40 con 0x7fd03415aee0 2024-04-25T09:02:25.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.936+0000 7fd038f90640 1 -- 172.21.15.43:0/2398722539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd03406f290 con 0x7fd03415aee0 2024-04-25T09:02:25.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.937+0000 7fd022ffd640 1 -- 172.21.15.43:0/2398722539 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd028012040 con 0x7fd03415aee0 2024-04-25T09:02:25.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.937+0000 7fd022ffd640 1 --2- 172.21.15.43:0/2398722539 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd01406b4d0 0x7fd01406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:25.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.937+0000 7fd022ffd640 1 -- 172.21.15.43:0/2398722539 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fd02808a6d0 con 0x7fd03415aee0 2024-04-25T09:02:25.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.938+0000 7fd03259c640 1 --2- 172.21.15.43:0/2398722539 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd01406b4d0 0x7fd01406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:25.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.938+0000 7fd03259c640 1 --2- 172.21.15.43:0/2398722539 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd01406b4d0 0x7fd01406d990 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd024033000 tx=0x7fd02402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:25.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:25.942+0000 7fd022ffd640 1 -- 172.21.15.43:0/2398722539 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd03406f290 con 0x7fd03415aee0 2024-04-25T09:02:26.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.159+0000 7fd038f90640 1 -- 172.21.15.43:0/2398722539 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd034074f60 con 0x7fd03415aee0 2024-04-25T09:02:26.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.797+0000 7fd022ffd640 1 -- 172.21.15.43:0/2398722539 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v66) v1 ==== 174+0+0 (secure 0 0 0) 0x7fd028019030 con 0x7fd03415aee0 2024-04-25T09:02:26.786 INFO:teuthology.orchestra.run.smithi043.stderr:new fs with metadata pool 16 and data pool 17 2024-04-25T09:02:26.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.802+0000 7fd038f90640 1 -- 172.21.15.43:0/2398722539 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd01406b4d0 msgr2=0x7fd01406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:26.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.802+0000 7fd038f90640 1 --2- 172.21.15.43:0/2398722539 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd01406b4d0 0x7fd01406d990 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd024033000 tx=0x7fd02402e000 comp rx=0 tx=0).stop 2024-04-25T09:02:26.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.802+0000 7fd038f90640 1 -- 172.21.15.43:0/2398722539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd03415aee0 msgr2=0x7fd0341f7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:26.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.802+0000 7fd038f90640 1 --2- 172.21.15.43:0/2398722539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd03415aee0 0x7fd0341f7f40 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7fd028006c00 tx=0x7fd02800f610 comp rx=0 tx=0).stop 2024-04-25T09:02:26.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.802+0000 7fd038f90640 1 -- 172.21.15.43:0/2398722539 shutdown_connections 2024-04-25T09:02:26.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.802+0000 7fd038f90640 1 --2- 172.21.15.43:0/2398722539 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd0340afb00 0x7fd0340affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:26.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.802+0000 7fd038f90640 1 --2- 172.21.15.43:0/2398722539 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd03415aee0 0x7fd0341f7f40 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:26.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.802+0000 7fd038f90640 1 --2- 172.21.15.43:0/2398722539 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd01406b4d0 0x7fd01406d990 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:26.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.803+0000 7fd038f90640 1 --2- 172.21.15.43:0/2398722539 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd01c0039b0 0x7fd0340b6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:26.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.803+0000 7fd038f90640 1 -- 172.21.15.43:0/2398722539 >> 172.21.15.43:0/2398722539 conn(0x7fd0340d17e0 msgr2=0x7fd03406c4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:26.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.806+0000 7fd038f90640 1 -- 172.21.15.43:0/2398722539 shutdown_connections 2024-04-25T09:02:26.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.806+0000 7fd038f90640 1 -- 172.21.15.43:0/2398722539 wait complete. 2024-04-25T09:02:26.823 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:02:26.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.936+0000 7fa4aef77640 1 Processor -- start 2024-04-25T09:02:26.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.936+0000 7fa4aef77640 1 -- start start 2024-04-25T09:02:26.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.937+0000 7fa4aef77640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa4a8161390 0x7fa4a80b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:26.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.937+0000 7fa4aef77640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa4a80cecf0 0x7fa4a80d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:26.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.937+0000 7fa4aef77640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4a80b4c30 con 0x7fa4a80d02a0 2024-04-25T09:02:26.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.937+0000 7fa4aef77640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4a80d1910 con 0x7fa4a80cecf0 2024-04-25T09:02:26.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.937+0000 7fa4aef77640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a80cf2c0 con 0x7fa4a8161390 2024-04-25T09:02:26.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.937+0000 7fa4ad514640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa4a80cecf0 0x7fa4a80d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:26.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.937+0000 7fa4ad514640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa4a80cecf0 0x7fa4a80d0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44998/0 (socket says 172.21.15.43:44998) 2024-04-25T09:02:26.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.937+0000 7fa4ad514640 1 -- 172.21.15.43:0/1921493810 learned_addr learned my addr 172.21.15.43:0/1921493810 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:26.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.938+0000 7fa4ad514640 1 -- 172.21.15.43:0/1921493810 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa4a8161390 msgr2=0x7fa4a80b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:26.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.938+0000 7fa4acd13640 1 --2- 172.21.15.43:0/1921493810 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa4a8161390 0x7fa4a80b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:26.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.938+0000 7fa4ad514640 1 --2- 172.21.15.43:0/1921493810 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa4a8161390 0x7fa4a80b6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:26.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.938+0000 7fa4ad514640 1 -- 172.21.15.43:0/1921493810 >> v1:172.21.15.193:6789/0 conn(0x7fa4a80d02a0 legacy=0x7fa4a80ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:26.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.938+0000 7fa4ad514640 1 -- 172.21.15.43:0/1921493810 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a80cf5a0 con 0x7fa4a80cecf0 2024-04-25T09:02:26.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.938+0000 7fa4acd13640 1 --2- 172.21.15.43:0/1921493810 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa4a8161390 0x7fa4a80b6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:26.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.938+0000 7fa4ad514640 1 --2- 172.21.15.43:0/1921493810 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4a80cecf0 0x7fa4a80d0720 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fa490009870 tx=0x7fa49002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f89d3fbc5748d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:26.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.938+0000 7fa4a77fe640 1 -- 172.21.15.43:0/1921493810 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa490037070 con 0x7fa4a80cecf0 2024-04-25T09:02:26.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.939+0000 7fa4a77fe640 1 -- 172.21.15.43:0/1921493810 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa490032440 con 0x7fa4a80cecf0 2024-04-25T09:02:26.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.939+0000 7fa4a77fe640 1 -- 172.21.15.43:0/1921493810 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa490032810 con 0x7fa4a80cecf0 2024-04-25T09:02:26.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.939+0000 7fa4aef77640 1 -- 172.21.15.43:0/1921493810 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4a80cecf0 msgr2=0x7fa4a80d0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:26.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.939+0000 7fa4aef77640 1 --2- 172.21.15.43:0/1921493810 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4a80cecf0 0x7fa4a80d0720 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7fa490009870 tx=0x7fa49002a210 comp rx=0 tx=0).stop 2024-04-25T09:02:26.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.939+0000 7fa4aef77640 1 -- 172.21.15.43:0/1921493810 shutdown_connections 2024-04-25T09:02:26.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.939+0000 7fa4aef77640 1 --2- 172.21.15.43:0/1921493810 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4a80cecf0 0x7fa4a80d0720 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:26.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.939+0000 7fa4aef77640 1 --2- 172.21.15.43:0/1921493810 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa4a8161390 0x7fa4a80b6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:26.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.939+0000 7fa4aef77640 1 -- 172.21.15.43:0/1921493810 >> 172.21.15.43:0/1921493810 conn(0x7fa4a809cd20 msgr2=0x7fa4a80d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:26.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.939+0000 7fa4aef77640 1 -- 172.21.15.43:0/1921493810 shutdown_connections 2024-04-25T09:02:26.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.939+0000 7fa4aef77640 1 -- 172.21.15.43:0/1921493810 wait complete. 2024-04-25T09:02:26.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.940+0000 7fa4aef77640 1 Processor -- start 2024-04-25T09:02:26.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.940+0000 7fa4aef77640 1 -- start start 2024-04-25T09:02:26.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.941+0000 7fa4aef77640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa4a80cecf0 0x7fa4a80c8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:26.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.941+0000 7fa4aef77640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4a80d02a0 0x7fa4a8159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:26.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.941+0000 7fa4aef77640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4a8161390 0x7fa4a815b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:26.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.941+0000 7fa4aef77640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a8161770 con 0x7fa4a8161390 2024-04-25T09:02:26.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.941+0000 7fa4aef77640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a80d15a0 con 0x7fa4a80d02a0 2024-04-25T09:02:26.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.941+0000 7fa4aef77640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4a80d1da0 con 0x7fa4a80cecf0 2024-04-25T09:02:26.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.941+0000 7fa4ad514640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4a8161390 0x7fa4a815b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:26.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.941+0000 7fa4ad514640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4a8161390 0x7fa4a815b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45006/0 (socket says 172.21.15.43:45006) 2024-04-25T09:02:26.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.941+0000 7fa4ad514640 1 -- 172.21.15.43:0/4057887925 learned_addr learned my addr 172.21.15.43:0/4057887925 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:26.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.941+0000 7fa4acd13640 1 --2- 172.21.15.43:0/4057887925 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa4a80cecf0 0x7fa4a80c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:26.933 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.941+0000 7fa4a7fff640 1 --2- 172.21.15.43:0/4057887925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4a80d02a0 0x7fa4a8159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:26.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.942+0000 7fa4ad514640 1 -- 172.21.15.43:0/4057887925 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa4a80cecf0 msgr2=0x7fa4a80c8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:26.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.942+0000 7fa4ad514640 1 --2- 172.21.15.43:0/4057887925 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa4a80cecf0 0x7fa4a80c8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:26.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.942+0000 7fa4ad514640 1 -- 172.21.15.43:0/4057887925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4a80d02a0 msgr2=0x7fa4a8159fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:26.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.942+0000 7fa4ad514640 1 --2- 172.21.15.43:0/4057887925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4a80d02a0 0x7fa4a8159fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:26.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.942+0000 7fa4ad514640 1 -- 172.21.15.43:0/4057887925 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4a81f6e10 con 0x7fa4a8161390 2024-04-25T09:02:26.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.942+0000 7fa4ad514640 1 --2- 172.21.15.43:0/4057887925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4a8161390 0x7fa4a815b3e0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fa49002a250 tx=0x7fa4900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:26.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.942+0000 7fa4a57fa640 1 -- 172.21.15.43:0/4057887925 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa490041070 con 0x7fa4a8161390 2024-04-25T09:02:26.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.942+0000 7fa4a57fa640 1 -- 172.21.15.43:0/4057887925 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa49003c950 con 0x7fa4a8161390 2024-04-25T09:02:26.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.942+0000 7fa4aef77640 1 -- 172.21.15.43:0/4057887925 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4a81f70f0 con 0x7fa4a8161390 2024-04-25T09:02:26.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.942+0000 7fa4a57fa640 1 -- 172.21.15.43:0/4057887925 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa490037040 con 0x7fa4a8161390 2024-04-25T09:02:26.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.942+0000 7fa4aef77640 1 -- 172.21.15.43:0/4057887925 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4a81f76b0 con 0x7fa4a8161390 2024-04-25T09:02:26.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.943+0000 7fa4aef77640 1 -- 172.21.15.43:0/4057887925 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4a80680f0 con 0x7fa4a8161390 2024-04-25T09:02:26.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.944+0000 7fa4a57fa640 1 -- 172.21.15.43:0/4057887925 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa490044070 con 0x7fa4a8161390 2024-04-25T09:02:26.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.944+0000 7fa4a57fa640 1 --2- 172.21.15.43:0/4057887925 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa48806b4d0 0x7fa48806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:26.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.944+0000 7fa4a57fa640 1 -- 172.21.15.43:0/4057887925 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa4900aabf0 con 0x7fa4a8161390 2024-04-25T09:02:26.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.944+0000 7fa4acd13640 1 --2- 172.21.15.43:0/4057887925 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa48806b4d0 0x7fa48806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:26.935 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.945+0000 7fa4acd13640 1 --2- 172.21.15.43:0/4057887925 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa48806b4d0 0x7fa48806d990 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fa4a000aac0 tx=0x7fa4a0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:26.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:26.948+0000 7fa4a57fa640 1 -- 172.21.15.43:0/4057887925 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa4a80b4c30 con 0x7fa4a8161390 2024-04-25T09:02:27.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.129+0000 7fa4aef77640 1 -- 172.21.15.43:0/4057887925 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa4a80d1050 con 0x7fa4a8161390 2024-04-25T09:02:27.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.130+0000 7fa4a57fa640 1 -- 172.21.15.43:0/4057887925 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+16604 (secure 0 0 0) 0x7fa490079420 con 0x7fa4a8161390 2024-04-25T09:02:27.119 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:27.119 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":112,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:02:26.161180+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:02:24.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T09:02:25.143723+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:02:27.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.132+0000 7fa4aef77640 1 -- 172.21.15.43:0/4057887925 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa48806b4d0 msgr2=0x7fa48806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:27.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.132+0000 7fa4aef77640 1 --2- 172.21.15.43:0/4057887925 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa48806b4d0 0x7fa48806d990 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fa4a000aac0 tx=0x7fa4a0008040 comp rx=0 tx=0).stop 2024-04-25T09:02:27.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.132+0000 7fa4aef77640 1 -- 172.21.15.43:0/4057887925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4a8161390 msgr2=0x7fa4a815b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:27.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.132+0000 7fa4aef77640 1 --2- 172.21.15.43:0/4057887925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4a8161390 0x7fa4a815b3e0 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7fa49002a250 tx=0x7fa4900046c0 comp rx=0 tx=0).stop 2024-04-25T09:02:27.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.132+0000 7fa4aef77640 1 -- 172.21.15.43:0/4057887925 shutdown_connections 2024-04-25T09:02:27.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.132+0000 7fa4aef77640 1 --2- 172.21.15.43:0/4057887925 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa4a8161390 0x7fa4a815b3e0 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:27.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.132+0000 7fa4aef77640 1 --2- 172.21.15.43:0/4057887925 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa48806b4d0 0x7fa48806d990 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:27.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.132+0000 7fa4aef77640 1 --2- 172.21.15.43:0/4057887925 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa4a80d02a0 0x7fa4a8159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:27.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.132+0000 7fa4aef77640 1 --2- 172.21.15.43:0/4057887925 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa4a80cecf0 0x7fa4a80c8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:27.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.132+0000 7fa4aef77640 1 -- 172.21.15.43:0/4057887925 >> 172.21.15.43:0/4057887925 conn(0x7fa4a809cd20 msgr2=0x7fa4a806b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:27.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.132+0000 7fa4aef77640 1 -- 172.21.15.43:0/4057887925 shutdown_connections 2024-04-25T09:02:27.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.132+0000 7fa4aef77640 1 -- 172.21.15.43:0/4057887925 wait complete. 2024-04-25T09:02:27.133 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:02:27.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.245+0000 7f0defb10640 1 Processor -- start 2024-04-25T09:02:27.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.245+0000 7f0defb10640 1 -- start start 2024-04-25T09:02:27.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.246+0000 7f0defb10640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0de806be50 0x7f0de8162100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:27.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.246+0000 7f0defb10640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0de80b20c0 0x7f0de80b2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:27.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.246+0000 7f0defb10640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0de815e2d0 con 0x7f0de815def0 2024-04-25T09:02:27.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.246+0000 7f0defb10640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0de80d0e90 con 0x7f0de806be50 2024-04-25T09:02:27.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.246+0000 7f0defb10640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0de80b1020 con 0x7f0de80b20c0 2024-04-25T09:02:27.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.246+0000 7f0dee0ad640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0de80b20c0 0x7f0de80b2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:27.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.246+0000 7f0ded0ab640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0de806be50 0x7f0de8162100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:27.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.246+0000 7f0ded8ac640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f0de815def0 0x7f0de81619f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:48932/0 (socket says 172.21.15.43:48932) 2024-04-25T09:02:27.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.246+0000 7f0ded8ac640 1 -- 172.21.15.43:0/4118386003 learned_addr learned my addr 172.21.15.43:0/4118386003 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:27.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.246+0000 7f0dee0ad640 1 -- 172.21.15.43:0/4118386003 >> v2:172.21.15.193:3300/0 conn(0x7f0de806be50 msgr2=0x7f0de8162100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:27.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.246+0000 7f0dee0ad640 1 --2- 172.21.15.43:0/4118386003 >> v2:172.21.15.193:3300/0 conn(0x7f0de806be50 0x7f0de8162100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:27.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.246+0000 7f0dee0ad640 1 -- 172.21.15.43:0/4118386003 >> v1:172.21.15.193:6789/0 conn(0x7f0de815def0 legacy=0x7f0de81619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:27.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.246+0000 7f0dee0ad640 1 -- 172.21.15.43:0/4118386003 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de80b1300 con 0x7f0de80b20c0 2024-04-25T09:02:27.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.246+0000 7f0ded0ab640 1 --2- 172.21.15.43:0/4118386003 >> v2:172.21.15.193:3300/0 conn(0x7f0de806be50 0x7f0de8162100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:02:27.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.246+0000 7f0dec8aa640 1 -- 172.21.15.43:0/4118386003 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 778066476 0 0) 0x7f0de815e2d0 con 0x7f0de815def0 2024-04-25T09:02:27.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.247+0000 7f0dee0ad640 1 --2- 172.21.15.43:0/4118386003 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0de80b20c0 0x7f0de80b2550 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0de0009d90 tx=0x7f0de002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=779bd518d1192729 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:27.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.247+0000 7f0dec8aa640 1 -- 172.21.15.43:0/4118386003 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de0007740 con 0x7f0de80b20c0 2024-04-25T09:02:27.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.247+0000 7f0dec8aa640 1 -- 172.21.15.43:0/4118386003 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0de00078e0 con 0x7f0de80b20c0 2024-04-25T09:02:27.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.247+0000 7f0dec8aa640 1 -- 172.21.15.43:0/4118386003 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de0007cb0 con 0x7f0de80b20c0 2024-04-25T09:02:27.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.248+0000 7f0defb10640 1 -- 172.21.15.43:0/4118386003 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0de80b20c0 msgr2=0x7f0de80b2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:27.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.248+0000 7f0defb10640 1 --2- 172.21.15.43:0/4118386003 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0de80b20c0 0x7f0de80b2550 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0de0009d90 tx=0x7f0de002a660 comp rx=0 tx=0).stop 2024-04-25T09:02:27.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.248+0000 7f0defb10640 1 -- 172.21.15.43:0/4118386003 shutdown_connections 2024-04-25T09:02:27.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.248+0000 7f0defb10640 1 --2- 172.21.15.43:0/4118386003 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0de80b20c0 0x7f0de80b2550 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:27.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.248+0000 7f0defb10640 1 --2- 172.21.15.43:0/4118386003 >> v2:172.21.15.193:3300/0 conn(0x7f0de806be50 0x7f0de8162100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:27.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.248+0000 7f0defb10640 1 -- 172.21.15.43:0/4118386003 >> 172.21.15.43:0/4118386003 conn(0x7f0de80b32e0 msgr2=0x7f0de80b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:27.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.248+0000 7f0defb10640 1 -- 172.21.15.43:0/4118386003 shutdown_connections 2024-04-25T09:02:27.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.248+0000 7f0defb10640 1 -- 172.21.15.43:0/4118386003 wait complete. 2024-04-25T09:02:27.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.249+0000 7f0defb10640 1 Processor -- start 2024-04-25T09:02:27.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.249+0000 7f0defb10640 1 -- start start 2024-04-25T09:02:27.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.250+0000 7f0defb10640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0de806be50 0x7f0de809b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:27.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.250+0000 7f0defb10640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0de80b20c0 0x7f0de81f3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:27.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.250+0000 7f0defb10640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0de815def0 0x7f0de81f0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:27.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.250+0000 7f0defb10640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de80b1ba0 con 0x7f0de815def0 2024-04-25T09:02:27.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.250+0000 7f0defb10640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de80b18c0 con 0x7f0de806be50 2024-04-25T09:02:27.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.250+0000 7f0defb10640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0de806c2b0 con 0x7f0de80b20c0 2024-04-25T09:02:27.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.250+0000 7f0dee0ad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0de815def0 0x7f0de81f0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:27.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.250+0000 7f0dee0ad640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0de815def0 0x7f0de81f0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45022/0 (socket says 172.21.15.43:45022) 2024-04-25T09:02:27.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.250+0000 7f0dee0ad640 1 -- 172.21.15.43:0/973914154 learned_addr learned my addr 172.21.15.43:0/973914154 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:27.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.250+0000 7f0ded0ab640 1 --2- 172.21.15.43:0/973914154 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0de80b20c0 0x7f0de81f3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:27.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.250+0000 7f0ded8ac640 1 --2- 172.21.15.43:0/973914154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0de806be50 0x7f0de809b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:27.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.250+0000 7f0dee0ad640 1 -- 172.21.15.43:0/973914154 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0de80b20c0 msgr2=0x7f0de81f3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:27.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.250+0000 7f0dee0ad640 1 --2- 172.21.15.43:0/973914154 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0de80b20c0 0x7f0de81f3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:27.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.250+0000 7f0dee0ad640 1 -- 172.21.15.43:0/973914154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0de806be50 msgr2=0x7f0de809b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:27.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.250+0000 7f0dee0ad640 1 --2- 172.21.15.43:0/973914154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0de806be50 0x7f0de809b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:27.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.250+0000 7f0dee0ad640 1 -- 172.21.15.43:0/973914154 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de81f40a0 con 0x7f0de815def0 2024-04-25T09:02:27.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.250+0000 7f0ded8ac640 1 --2- 172.21.15.43:0/973914154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0de806be50 0x7f0de809b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:27.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.251+0000 7f0dee0ad640 1 --2- 172.21.15.43:0/973914154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0de815def0 0x7f0de81f0f40 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f0de002ac50 tx=0x7f0de0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:27.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.251+0000 7f0dde7fc640 1 -- 172.21.15.43:0/973914154 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de0040070 con 0x7f0de815def0 2024-04-25T09:02:27.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.251+0000 7f0dde7fc640 1 -- 172.21.15.43:0/973914154 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0de0031bc0 con 0x7f0de815def0 2024-04-25T09:02:27.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.251+0000 7f0dde7fc640 1 -- 172.21.15.43:0/973914154 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0de0009450 con 0x7f0de815def0 2024-04-25T09:02:27.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.251+0000 7f0defb10640 1 -- 172.21.15.43:0/973914154 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0de81f4370 con 0x7f0de815def0 2024-04-25T09:02:27.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.251+0000 7f0defb10640 1 -- 172.21.15.43:0/973914154 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0de81f4930 con 0x7f0de815def0 2024-04-25T09:02:27.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.252+0000 7f0defb10640 1 -- 172.21.15.43:0/973914154 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0de8074ef0 con 0x7f0de815def0 2024-04-25T09:02:27.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.252+0000 7f0dde7fc640 1 -- 172.21.15.43:0/973914154 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0de0039040 con 0x7f0de815def0 2024-04-25T09:02:27.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.253+0000 7f0dde7fc640 1 --2- 172.21.15.43:0/973914154 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0dcc06b4d0 0x7f0dcc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:27.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.253+0000 7f0dde7fc640 1 -- 172.21.15.43:0/973914154 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0de00ab900 con 0x7f0de815def0 2024-04-25T09:02:27.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.253+0000 7f0ded8ac640 1 --2- 172.21.15.43:0/973914154 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0dcc06b4d0 0x7f0dcc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:27.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.253+0000 7f0ded8ac640 1 --2- 172.21.15.43:0/973914154 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0dcc06b4d0 0x7f0dcc06d990 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0de4000f60 tx=0x7f0de4007c10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:27.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.257+0000 7f0dde7fc640 1 -- 172.21.15.43:0/973914154 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0de007a780 con 0x7f0de815def0 2024-04-25T09:02:27.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.437+0000 7f0defb10640 1 -- 172.21.15.43:0/973914154 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0de80b7710 con 0x7f0de815def0 2024-04-25T09:02:27.496 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.438+0000 7f0dde7fc640 1 -- 172.21.15.43:0/973914154 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) v1 ==== 74+0+16604 (secure 0 0 0) 0x7f0de007a080 con 0x7f0de815def0 2024-04-25T09:02:27.496 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:27.497 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":112,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:02:26.161180+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:02:24.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T09:02:25.143723+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:02:27.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.440+0000 7f0defb10640 1 -- 172.21.15.43:0/973914154 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0dcc06b4d0 msgr2=0x7f0dcc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:27.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.440+0000 7f0defb10640 1 --2- 172.21.15.43:0/973914154 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0dcc06b4d0 0x7f0dcc06d990 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0de4000f60 tx=0x7f0de4007c10 comp rx=0 tx=0).stop 2024-04-25T09:02:27.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.440+0000 7f0defb10640 1 -- 172.21.15.43:0/973914154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0de815def0 msgr2=0x7f0de81f0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:27.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.440+0000 7f0defb10640 1 --2- 172.21.15.43:0/973914154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0de815def0 0x7f0de81f0f40 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f0de002ac50 tx=0x7f0de0004a40 comp rx=0 tx=0).stop 2024-04-25T09:02:27.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.440+0000 7f0defb10640 1 -- 172.21.15.43:0/973914154 shutdown_connections 2024-04-25T09:02:27.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.440+0000 7f0defb10640 1 --2- 172.21.15.43:0/973914154 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0de815def0 0x7f0de81f0f40 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:27.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.440+0000 7f0defb10640 1 --2- 172.21.15.43:0/973914154 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0de80b20c0 0x7f0de81f3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:27.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.440+0000 7f0defb10640 1 --2- 172.21.15.43:0/973914154 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0dcc06b4d0 0x7f0dcc06d990 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:27.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.440+0000 7f0defb10640 1 --2- 172.21.15.43:0/973914154 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0de806be50 0x7f0de809b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:27.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.440+0000 7f0defb10640 1 -- 172.21.15.43:0/973914154 >> 172.21.15.43:0/973914154 conn(0x7f0de80b32e0 msgr2=0x7f0de80b38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:27.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.440+0000 7f0defb10640 1 -- 172.21.15.43:0/973914154 shutdown_connections 2024-04-25T09:02:27.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.440+0000 7f0defb10640 1 -- 172.21.15.43:0/973914154 wait complete. 2024-04-25T09:02:27.500 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:02:27.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.615+0000 7fdb35f22640 1 Processor -- start 2024-04-25T09:02:27.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.615+0000 7fdb35f22640 1 -- start start 2024-04-25T09:02:27.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.616+0000 7fdb35f22640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fdb300b5560 0x7fdb300b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:27.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.616+0000 7fdb35f22640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb30162610 con 0x7fdb3015e090 2024-04-25T09:02:27.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.616+0000 7fdb35f22640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb300d1750 con 0x7fdb300d2230 2024-04-25T09:02:27.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.616+0000 7fdb35f22640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb300d1da0 con 0x7fdb300b5560 2024-04-25T09:02:27.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.616+0000 7fdb2f7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fdb3015e090 0x7fdb30161aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:33924/0 (socket says 172.21.15.43:33924) 2024-04-25T09:02:27.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.616+0000 7fdb2f7fe640 1 -- 172.21.15.43:0/681798220 learned_addr learned my addr 172.21.15.43:0/681798220 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:27.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.616+0000 7fdb2effd640 1 --2- 172.21.15.43:0/681798220 >> v2:172.21.15.193:3300/0 conn(0x7fdb300b5560 0x7fdb300b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:27.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.616+0000 7fdb2e7fc640 1 -- 172.21.15.43:0/681798220 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1069637865 0 0) 0x7fdb30162610 con 0x7fdb3015e090 2024-04-25T09:02:27.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.616+0000 7fdb2e7fc640 1 -- 172.21.15.43:0/681798220 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb18003520 con 0x7fdb3015e090 2024-04-25T09:02:27.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.616+0000 7fdb2e7fc640 1 -- 172.21.15.43:0/681798220 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2333826132 0 0) 0x7fdb18003520 con 0x7fdb3015e090 2024-04-25T09:02:27.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.616+0000 7fdb2e7fc640 1 -- 172.21.15.43:0/681798220 >> v2:172.21.15.193:3300/0 conn(0x7fdb300b5560 msgr2=0x7fdb300b59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:27.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.616+0000 7fdb2e7fc640 1 --2- 172.21.15.43:0/681798220 >> v2:172.21.15.193:3300/0 conn(0x7fdb300b5560 0x7fdb300b59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:27.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.616+0000 7fdb2e7fc640 1 -- 172.21.15.43:0/681798220 >> v1:172.21.15.193:6789/0 conn(0x7fdb300d2230 legacy=0x7fdb300d26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:27.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.616+0000 7fdb2e7fc640 1 -- 172.21.15.43:0/681798220 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb300d1a30 con 0x7fdb3015e090 2024-04-25T09:02:27.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.616+0000 7fdb2e7fc640 1 -- 172.21.15.43:0/681798220 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7fdb24001670 con 0x7fdb3015e090 2024-04-25T09:02:27.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.616+0000 7fdb2e7fc640 1 -- 172.21.15.43:0/681798220 >> v1:172.21.15.43:6789/0 conn(0x7fdb3015e090 legacy=0x7fdb30161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:27.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.617+0000 7fdb2e7fc640 1 --2- 172.21.15.43:0/681798220 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdb180039b0 0x7fdb18003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:27.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.617+0000 7fdb2e7fc640 1 --2- 172.21.15.43:0/681798220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb18004800 0x7fdb18004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:27.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.617+0000 7fdb2e7fc640 1 --2- 172.21.15.43:0/681798220 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb180051c0 0x7fdb1800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:27.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.617+0000 7fdb2e7fc640 1 -- 172.21.15.43:0/681798220 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb300d1da0 con 0x7fdb18004800 2024-04-25T09:02:27.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.617+0000 7fdb2e7fc640 1 -- 172.21.15.43:0/681798220 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1800c070 con 0x7fdb180051c0 2024-04-25T09:02:27.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.617+0000 7fdb2e7fc640 1 -- 172.21.15.43:0/681798220 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1800c270 con 0x7fdb180039b0 2024-04-25T09:02:27.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.617+0000 7fdb2effd640 1 --2- 172.21.15.43:0/681798220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb18004800 0x7fdb18004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:27.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.617+0000 7fdb2f7fe640 1 --2- 172.21.15.43:0/681798220 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdb180039b0 0x7fdb18003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:27.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.617+0000 7fdb2ffff640 1 --2- 172.21.15.43:0/681798220 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb180051c0 0x7fdb1800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:27.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.617+0000 7fdb2effd640 1 -- 172.21.15.43:0/681798220 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdb180039b0 msgr2=0x7fdb18003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:27.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.617+0000 7fdb2effd640 1 --2- 172.21.15.43:0/681798220 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdb180039b0 0x7fdb18003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:27.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.617+0000 7fdb2effd640 1 -- 172.21.15.43:0/681798220 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb180051c0 msgr2=0x7fdb1800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:27.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.617+0000 7fdb2effd640 1 --2- 172.21.15.43:0/681798220 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb180051c0 0x7fdb1800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:27.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.617+0000 7fdb2effd640 1 -- 172.21.15.43:0/681798220 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb18003520 con 0x7fdb18004800 2024-04-25T09:02:27.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.617+0000 7fdb2ffff640 1 -- 172.21.15.43:0/681798220 reap_dead start 2024-04-25T09:02:27.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.617+0000 7fdb2f7fe640 1 --2- 172.21.15.43:0/681798220 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdb180039b0 0x7fdb18003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:27.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.618+0000 7fdb2effd640 1 --2- 172.21.15.43:0/681798220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb18004800 0x7fdb18004c80 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fdb20009ac0 tx=0x7fdb18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe3ae06d9e376d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:27.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.618+0000 7fdb2e7fc640 1 -- 172.21.15.43:0/681798220 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb20036070 con 0x7fdb18004800 2024-04-25T09:02:27.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.618+0000 7fdb2e7fc640 1 -- 172.21.15.43:0/681798220 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb20031400 con 0x7fdb18004800 2024-04-25T09:02:27.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.618+0000 7fdb2e7fc640 1 -- 172.21.15.43:0/681798220 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb20009690 con 0x7fdb18004800 2024-04-25T09:02:27.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.618+0000 7fdb35f22640 1 -- 172.21.15.43:0/681798220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb18004800 msgr2=0x7fdb18004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:27.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.618+0000 7fdb35f22640 1 --2- 172.21.15.43:0/681798220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb18004800 0x7fdb18004c80 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fdb20009ac0 tx=0x7fdb18005830 comp rx=0 tx=0).stop 2024-04-25T09:02:27.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.618+0000 7fdb35f22640 1 -- 172.21.15.43:0/681798220 shutdown_connections 2024-04-25T09:02:27.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.618+0000 7fdb35f22640 1 --2- 172.21.15.43:0/681798220 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb18004800 0x7fdb18004c80 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:27.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.618+0000 7fdb35f22640 1 -- 172.21.15.43:0/681798220 >> 172.21.15.43:0/681798220 conn(0x7fdb300d0560 msgr2=0x7fdb300d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:27.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.618+0000 7fdb35f22640 1 -- 172.21.15.43:0/681798220 shutdown_connections 2024-04-25T09:02:27.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.618+0000 7fdb35f22640 1 -- 172.21.15.43:0/681798220 wait complete. 2024-04-25T09:02:27.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.620+0000 7fdb35f22640 1 Processor -- start 2024-04-25T09:02:27.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.620+0000 7fdb35f22640 1 -- start start 2024-04-25T09:02:27.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.621+0000 7fdb35f22640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb18004800 0x7fdb300ce600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:27.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.621+0000 7fdb35f22640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdb30161a30 0x7fdb30161eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:27.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.621+0000 7fdb35f22640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb3014ace0 0x7fdb3014b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:27.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.621+0000 7fdb35f22640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb300b59e0 con 0x7fdb3014ace0 2024-04-25T09:02:27.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.621+0000 7fdb35f22640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb300b3280 con 0x7fdb18004800 2024-04-25T09:02:27.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.621+0000 7fdb35f22640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb300b5b60 con 0x7fdb30161a30 2024-04-25T09:02:27.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.621+0000 7fdb2ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb3014ace0 0x7fdb3014b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:27.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.621+0000 7fdb2ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb3014ace0 0x7fdb3014b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45048/0 (socket says 172.21.15.43:45048) 2024-04-25T09:02:27.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.621+0000 7fdb2ffff640 1 -- 172.21.15.43:0/1178056198 learned_addr learned my addr 172.21.15.43:0/1178056198 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:27.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.621+0000 7fdb2effd640 1 --2- 172.21.15.43:0/1178056198 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdb30161a30 0x7fdb30161eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:27.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.621+0000 7fdb2f7fe640 1 --2- 172.21.15.43:0/1178056198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb18004800 0x7fdb300ce600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:27.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.621+0000 7fdb2ffff640 1 -- 172.21.15.43:0/1178056198 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdb30161a30 msgr2=0x7fdb30161eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:27.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.621+0000 7fdb2ffff640 1 --2- 172.21.15.43:0/1178056198 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdb30161a30 0x7fdb30161eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:27.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.621+0000 7fdb2ffff640 1 -- 172.21.15.43:0/1178056198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb18004800 msgr2=0x7fdb300ce600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:27.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.621+0000 7fdb2ffff640 1 --2- 172.21.15.43:0/1178056198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb18004800 0x7fdb300ce600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:27.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.621+0000 7fdb2ffff640 1 -- 172.21.15.43:0/1178056198 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb3014b9d0 con 0x7fdb3014ace0 2024-04-25T09:02:27.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.621+0000 7fdb2ffff640 1 --2- 172.21.15.43:0/1178056198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb3014ace0 0x7fdb3014b160 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fdb3008ce90 tx=0x7fdb2800e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:27.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.621+0000 7fdb17fff640 1 -- 172.21.15.43:0/1178056198 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb28019070 con 0x7fdb3014ace0 2024-04-25T09:02:27.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.621+0000 7fdb17fff640 1 -- 172.21.15.43:0/1178056198 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb28006d60 con 0x7fdb3014ace0 2024-04-25T09:02:27.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.622+0000 7fdb35f22640 1 -- 172.21.15.43:0/1178056198 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb30148dd0 con 0x7fdb3014ace0 2024-04-25T09:02:27.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.622+0000 7fdb17fff640 1 -- 172.21.15.43:0/1178056198 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb28003b90 con 0x7fdb3014ace0 2024-04-25T09:02:27.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.622+0000 7fdb35f22640 1 -- 172.21.15.43:0/1178056198 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb300ceb40 con 0x7fdb3014ace0 2024-04-25T09:02:27.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.623+0000 7fdb35f22640 1 -- 172.21.15.43:0/1178056198 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb3006bf00 con 0x7fdb3014ace0 2024-04-25T09:02:27.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.623+0000 7fdb17fff640 1 -- 172.21.15.43:0/1178056198 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdb2801c070 con 0x7fdb3014ace0 2024-04-25T09:02:27.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.623+0000 7fdb17fff640 1 --2- 172.21.15.43:0/1178056198 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdb1c06b4d0 0x7fdb1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:27.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.624+0000 7fdb17fff640 1 -- 172.21.15.43:0/1178056198 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb28089620 con 0x7fdb3014ace0 2024-04-25T09:02:27.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.624+0000 7fdb2f7fe640 1 --2- 172.21.15.43:0/1178056198 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdb1c06b4d0 0x7fdb1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:27.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.624+0000 7fdb2f7fe640 1 --2- 172.21.15.43:0/1178056198 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdb1c06b4d0 0x7fdb1c06d990 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fdb24001eb0 tx=0x7fdb24007040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:27.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.628+0000 7fdb17fff640 1 -- 172.21.15.43:0/1178056198 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdb3006bf00 con 0x7fdb3014ace0 2024-04-25T09:02:27.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:27.846+0000 7fdb35f22640 1 -- 172.21.15.43:0/1178056198 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fdb30074f60 con 0x7fdb3014ace0 2024-04-25T09:02:28.773 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-04-25T09:02:28.785+0000 7f8a22733640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:02:28.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.818+0000 7fdb17fff640 1 -- 172.21.15.43:0/1178056198 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v68) v1 ==== 118+0+0 (secure 0 0 0) 0x7fdb28057ee0 con 0x7fdb3014ace0 2024-04-25T09:02:28.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.820+0000 7fdb35f22640 1 -- 172.21.15.43:0/1178056198 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdb1c06b4d0 msgr2=0x7fdb1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:28.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.820+0000 7fdb35f22640 1 --2- 172.21.15.43:0/1178056198 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdb1c06b4d0 0x7fdb1c06d990 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fdb24001eb0 tx=0x7fdb24007040 comp rx=0 tx=0).stop 2024-04-25T09:02:28.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.820+0000 7fdb35f22640 1 -- 172.21.15.43:0/1178056198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb3014ace0 msgr2=0x7fdb3014b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:28.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.820+0000 7fdb35f22640 1 --2- 172.21.15.43:0/1178056198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb3014ace0 0x7fdb3014b160 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fdb3008ce90 tx=0x7fdb2800e940 comp rx=0 tx=0).stop 2024-04-25T09:02:28.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.820+0000 7fdb35f22640 1 -- 172.21.15.43:0/1178056198 shutdown_connections 2024-04-25T09:02:28.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.820+0000 7fdb35f22640 1 --2- 172.21.15.43:0/1178056198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdb3014ace0 0x7fdb3014b160 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:28.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.820+0000 7fdb35f22640 1 --2- 172.21.15.43:0/1178056198 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdb30161a30 0x7fdb30161eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:28.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.820+0000 7fdb35f22640 1 --2- 172.21.15.43:0/1178056198 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdb1c06b4d0 0x7fdb1c06d990 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:28.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.820+0000 7fdb35f22640 1 --2- 172.21.15.43:0/1178056198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdb18004800 0x7fdb300ce600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:28.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.820+0000 7fdb35f22640 1 -- 172.21.15.43:0/1178056198 >> 172.21.15.43:0/1178056198 conn(0x7fdb300d0560 msgr2=0x7fdb300b6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:28.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.821+0000 7fdb35f22640 1 -- 172.21.15.43:0/1178056198 shutdown_connections 2024-04-25T09:02:28.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.821+0000 7fdb35f22640 1 -- 172.21.15.43:0/1178056198 wait complete. 2024-04-25T09:02:28.823 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T09:02:28.824 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:02:28.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.945+0000 7f173a3e0640 1 Processor -- start 2024-04-25T09:02:28.934 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.945+0000 7f173a3e0640 1 -- start start 2024-04-25T09:02:28.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.946+0000 7f173a3e0640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f173415a960 0x7f17340b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:28.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.946+0000 7f173a3e0640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f17340b3ed0 0x7f17340ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:28.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.946+0000 7f173a3e0640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f17340aec70 0x7f173406d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:28.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.946+0000 7f173a3e0640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17340af0a0 con 0x7f17340aec70 2024-04-25T09:02:28.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.946+0000 7f173a3e0640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f173415b110 con 0x7f17340b3ed0 2024-04-25T09:02:28.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.946+0000 7f173a3e0640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f173415b3f0 con 0x7f173415a960 2024-04-25T09:02:28.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.946+0000 7f173897d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f17340aec70 0x7f173406d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:28.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.946+0000 7f173897d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f17340aec70 0x7f173406d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45064/0 (socket says 172.21.15.43:45064) 2024-04-25T09:02:28.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.946+0000 7f173897d640 1 -- 172.21.15.43:0/907012780 learned_addr learned my addr 172.21.15.43:0/907012780 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:28.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.946+0000 7f173897d640 1 -- 172.21.15.43:0/907012780 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f173415a960 msgr2=0x7f17340b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:28.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.946+0000 7f173897d640 1 --2- 172.21.15.43:0/907012780 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f173415a960 0x7f17340b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:28.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.946+0000 7f173897d640 1 -- 172.21.15.43:0/907012780 >> v2:172.21.15.193:3300/0 conn(0x7f17340b3ed0 msgr2=0x7f17340ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:28.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.946+0000 7f173897d640 1 --2- 172.21.15.43:0/907012780 >> v2:172.21.15.193:3300/0 conn(0x7f17340b3ed0 0x7f17340ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:28.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.946+0000 7f173897d640 1 -- 172.21.15.43:0/907012780 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f173415b6d0 con 0x7f17340aec70 2024-04-25T09:02:28.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.947+0000 7f173897d640 1 --2- 172.21.15.43:0/907012780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17340aec70 0x7f173406d0c0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f1734070020 tx=0x7f172c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9bad22989b93a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:28.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.947+0000 7f1732ffd640 1 -- 172.21.15.43:0/907012780 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172c036070 con 0x7f17340aec70 2024-04-25T09:02:28.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.947+0000 7f1732ffd640 1 -- 172.21.15.43:0/907012780 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f172c031600 con 0x7f17340aec70 2024-04-25T09:02:28.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.947+0000 7f1732ffd640 1 -- 172.21.15.43:0/907012780 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172c0319d0 con 0x7f17340aec70 2024-04-25T09:02:28.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.948+0000 7f173a3e0640 1 -- 172.21.15.43:0/907012780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17340aec70 msgr2=0x7f173406d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:28.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.948+0000 7f173a3e0640 1 --2- 172.21.15.43:0/907012780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17340aec70 0x7f173406d0c0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f1734070020 tx=0x7f172c02a6a0 comp rx=0 tx=0).stop 2024-04-25T09:02:28.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.948+0000 7f173a3e0640 1 -- 172.21.15.43:0/907012780 shutdown_connections 2024-04-25T09:02:28.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.948+0000 7f173a3e0640 1 --2- 172.21.15.43:0/907012780 >> v2:172.21.15.193:3300/0 conn(0x7f17340b3ed0 0x7f17340ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:28.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.948+0000 7f173a3e0640 1 --2- 172.21.15.43:0/907012780 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17340aec70 0x7f173406d0c0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:28.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.948+0000 7f173a3e0640 1 --2- 172.21.15.43:0/907012780 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f173415a960 0x7f17340b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:28.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.948+0000 7f173a3e0640 1 -- 172.21.15.43:0/907012780 >> 172.21.15.43:0/907012780 conn(0x7f173406ab30 msgr2=0x7f173406af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:28.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.948+0000 7f173a3e0640 1 -- 172.21.15.43:0/907012780 shutdown_connections 2024-04-25T09:02:28.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.948+0000 7f173a3e0640 1 -- 172.21.15.43:0/907012780 wait complete. 2024-04-25T09:02:28.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.949+0000 7f173a3e0640 1 Processor -- start 2024-04-25T09:02:28.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.949+0000 7f173a3e0640 1 -- start start 2024-04-25T09:02:28.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.950+0000 7f173a3e0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17340aec70 0x7f17340ae170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:28.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.950+0000 7f173a3e0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17340b3ed0 0x7f17341f2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:28.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.950+0000 7f173a3e0640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f173415a960 0x7f173415cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:28.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.950+0000 7f173a3e0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17340cd940 con 0x7f17340aec70 2024-04-25T09:02:28.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.950+0000 7f173a3e0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17340cc440 con 0x7f17340b3ed0 2024-04-25T09:02:28.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.950+0000 7f173a3e0640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f173415ad40 con 0x7f173415a960 2024-04-25T09:02:28.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.950+0000 7f1733fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17340aec70 0x7f17340ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:28.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.950+0000 7f1733fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17340aec70 0x7f17340ae170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45078/0 (socket says 172.21.15.43:45078) 2024-04-25T09:02:28.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.950+0000 7f1733fff640 1 -- 172.21.15.43:0/510439687 learned_addr learned my addr 172.21.15.43:0/510439687 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:28.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.950+0000 7f173897d640 1 --2- 172.21.15.43:0/510439687 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f173415a960 0x7f173415cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:28.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.950+0000 7f17337fe640 1 --2- 172.21.15.43:0/510439687 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17340b3ed0 0x7f17341f2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:28.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.950+0000 7f1733fff640 1 -- 172.21.15.43:0/510439687 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f173415a960 msgr2=0x7f173415cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:28.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.950+0000 7f1733fff640 1 --2- 172.21.15.43:0/510439687 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f173415a960 0x7f173415cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:28.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.950+0000 7f1733fff640 1 -- 172.21.15.43:0/510439687 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17340b3ed0 msgr2=0x7f17341f2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:28.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.950+0000 7f1733fff640 1 --2- 172.21.15.43:0/510439687 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17340b3ed0 0x7f17341f2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:28.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.950+0000 7f1733fff640 1 -- 172.21.15.43:0/510439687 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17341e6ad0 con 0x7f17340aec70 2024-04-25T09:02:28.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.950+0000 7f1733fff640 1 --2- 172.21.15.43:0/510439687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17340aec70 0x7f17340ae170 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f172400b440 tx=0x7f172400b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:28.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.951+0000 7f1730ff9640 1 -- 172.21.15.43:0/510439687 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1724015020 con 0x7f17340aec70 2024-04-25T09:02:28.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.951+0000 7f1730ff9640 1 -- 172.21.15.43:0/510439687 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1724011890 con 0x7f17340aec70 2024-04-25T09:02:28.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.951+0000 7f173a3e0640 1 -- 172.21.15.43:0/510439687 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17341e6db0 con 0x7f17340aec70 2024-04-25T09:02:28.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.951+0000 7f1730ff9640 1 -- 172.21.15.43:0/510439687 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1724011c80 con 0x7f17340aec70 2024-04-25T09:02:28.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.951+0000 7f173a3e0640 1 -- 172.21.15.43:0/510439687 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17341e7370 con 0x7f17340aec70 2024-04-25T09:02:28.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.952+0000 7f173a3e0640 1 -- 172.21.15.43:0/510439687 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1734074f60 con 0x7f17340aec70 2024-04-25T09:02:28.944 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.952+0000 7f1730ff9640 1 -- 172.21.15.43:0/510439687 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1724018020 con 0x7f17340aec70 2024-04-25T09:02:28.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.953+0000 7f1730ff9640 1 --2- 172.21.15.43:0/510439687 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f171406b4d0 0x7f171406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:28.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.953+0000 7f1730ff9640 1 -- 172.21.15.43:0/510439687 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f172408a2f0 con 0x7f17340aec70 2024-04-25T09:02:28.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.953+0000 7f17337fe640 1 --2- 172.21.15.43:0/510439687 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f171406b4d0 0x7f171406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:28.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.953+0000 7f17337fe640 1 --2- 172.21.15.43:0/510439687 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f171406b4d0 0x7f171406d990 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f17340cbd10 tx=0x7f171c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:28.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:28.957+0000 7f1730ff9640 1 -- 172.21.15.43:0/510439687 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f172401f020 con 0x7f17340aec70 2024-04-25T09:02:29.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.175+0000 7f173a3e0640 1 -- 172.21.15.43:0/510439687 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f17340af790 con 0x7f17340aec70 2024-04-25T09:02:29.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.827+0000 7f1730ff9640 1 -- 172.21.15.43:0/510439687 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v69) v1 ==== 122+0+0 (secure 0 0 0) 0x7f172400d040 con 0x7f17340aec70 2024-04-25T09:02:29.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.830+0000 7f173a3e0640 1 -- 172.21.15.43:0/510439687 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f171406b4d0 msgr2=0x7f171406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:29.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.830+0000 7f173a3e0640 1 --2- 172.21.15.43:0/510439687 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f171406b4d0 0x7f171406d990 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f17340cbd10 tx=0x7f171c008040 comp rx=0 tx=0).stop 2024-04-25T09:02:29.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.830+0000 7f173a3e0640 1 -- 172.21.15.43:0/510439687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17340aec70 msgr2=0x7f17340ae170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:29.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.830+0000 7f173a3e0640 1 --2- 172.21.15.43:0/510439687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17340aec70 0x7f17340ae170 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f172400b440 tx=0x7f172400b900 comp rx=0 tx=0).stop 2024-04-25T09:02:29.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.830+0000 7f173a3e0640 1 -- 172.21.15.43:0/510439687 shutdown_connections 2024-04-25T09:02:29.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.830+0000 7f173a3e0640 1 --2- 172.21.15.43:0/510439687 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f173415a960 0x7f173415cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:29.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.830+0000 7f173a3e0640 1 --2- 172.21.15.43:0/510439687 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f171406b4d0 0x7f171406d990 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:29.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.830+0000 7f173a3e0640 1 --2- 172.21.15.43:0/510439687 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f17340b3ed0 0x7f17341f2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:29.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.830+0000 7f173a3e0640 1 --2- 172.21.15.43:0/510439687 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f17340aec70 0x7f17340ae170 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:29.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.830+0000 7f173a3e0640 1 -- 172.21.15.43:0/510439687 >> 172.21.15.43:0/510439687 conn(0x7f173406ab30 msgr2=0x7f17340b2840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:29.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.830+0000 7f173a3e0640 1 -- 172.21.15.43:0/510439687 shutdown_connections 2024-04-25T09:02:29.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.830+0000 7f173a3e0640 1 -- 172.21.15.43:0/510439687 wait complete. 2024-04-25T09:02:29.833 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:02:29.833 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:02:29.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.952+0000 7fc9156dd640 1 Processor -- start 2024-04-25T09:02:29.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.952+0000 7fc9156dd640 1 -- start start 2024-04-25T09:02:29.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.953+0000 7fc9156dd640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc9100cfd70 0x7fc9100d01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:29.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.953+0000 7fc9156dd640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9100cc760 con 0x7fc91015be60 2024-04-25T09:02:29.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.953+0000 7fc9156dd640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9100cca40 con 0x7fc9100b0e50 2024-04-25T09:02:29.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.953+0000 7fc9156dd640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc91015c240 con 0x7fc9100cfd70 2024-04-25T09:02:29.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.953+0000 7fc90effd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc91015be60 0x7fc9100ce880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48368/0 (socket says 172.21.15.43:48368) 2024-04-25T09:02:29.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.953+0000 7fc90effd640 1 -- 172.21.15.43:0/1758235370 learned_addr learned my addr 172.21.15.43:0/1758235370 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:29.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.953+0000 7fc90e7fc640 1 --2- 172.21.15.43:0/1758235370 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc9100cfd70 0x7fc9100d01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:29.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.953+0000 7fc90dffb640 1 -- 172.21.15.43:0/1758235370 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3275785046 0 0) 0x7fc9100cc760 con 0x7fc91015be60 2024-04-25T09:02:29.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.953+0000 7fc90dffb640 1 -- 172.21.15.43:0/1758235370 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8f8003520 con 0x7fc91015be60 2024-04-25T09:02:29.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.954+0000 7fc90dffb640 1 -- 172.21.15.43:0/1758235370 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2060159154 0 0) 0x7fc9100cca40 con 0x7fc9100b0e50 2024-04-25T09:02:29.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.954+0000 7fc90dffb640 1 -- 172.21.15.43:0/1758235370 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9100cc760 con 0x7fc9100b0e50 2024-04-25T09:02:29.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.954+0000 7fc90dffb640 1 -- 172.21.15.43:0/1758235370 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3275456755 0 0) 0x7fc8f8003520 con 0x7fc91015be60 2024-04-25T09:02:29.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.954+0000 7fc90e7fc640 1 -- 172.21.15.43:0/1758235370 >> v1:172.21.15.193:6789/0 conn(0x7fc9100b0e50 legacy=0x7fc9100b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:29.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.954+0000 7fc90e7fc640 1 -- 172.21.15.43:0/1758235370 >> v1:172.21.15.43:6789/0 conn(0x7fc91015be60 legacy=0x7fc9100ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:29.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.954+0000 7fc90e7fc640 1 -- 172.21.15.43:0/1758235370 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9100ccd20 con 0x7fc9100cfd70 2024-04-25T09:02:29.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.954+0000 7fc90e7fc640 1 --2- 172.21.15.43:0/1758235370 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc9100cfd70 0x7fc9100d01d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc9040097c0 tx=0x7fc90402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=212404e302ea37e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:29.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.955+0000 7fc90dffb640 1 -- 172.21.15.43:0/1758235370 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc904037070 con 0x7fc9100cfd70 2024-04-25T09:02:29.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.955+0000 7fc90dffb640 1 -- 172.21.15.43:0/1758235370 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc90402ac10 con 0x7fc9100cfd70 2024-04-25T09:02:29.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.955+0000 7fc90dffb640 1 -- 172.21.15.43:0/1758235370 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc900001670 con 0x7fc9100cfd70 2024-04-25T09:02:29.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.955+0000 7fc9156dd640 1 -- 172.21.15.43:0/1758235370 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc9100cfd70 msgr2=0x7fc9100d01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:29.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.955+0000 7fc9156dd640 1 --2- 172.21.15.43:0/1758235370 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc9100cfd70 0x7fc9100d01d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc9040097c0 tx=0x7fc90402a180 comp rx=0 tx=0).stop 2024-04-25T09:02:29.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.955+0000 7fc9156dd640 1 -- 172.21.15.43:0/1758235370 shutdown_connections 2024-04-25T09:02:29.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.955+0000 7fc9156dd640 1 --2- 172.21.15.43:0/1758235370 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc9100cfd70 0x7fc9100d01d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:29.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.955+0000 7fc9156dd640 1 -- 172.21.15.43:0/1758235370 >> 172.21.15.43:0/1758235370 conn(0x7fc9100b32e0 msgr2=0x7fc9100b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:29.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.955+0000 7fc9156dd640 1 -- 172.21.15.43:0/1758235370 shutdown_connections 2024-04-25T09:02:29.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.955+0000 7fc9156dd640 1 -- 172.21.15.43:0/1758235370 wait complete. 2024-04-25T09:02:29.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.957+0000 7fc9156dd640 1 Processor -- start 2024-04-25T09:02:29.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.957+0000 7fc9156dd640 1 -- start start 2024-04-25T09:02:29.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.957+0000 7fc9156dd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9100b0e50 0x7fc9101f7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:29.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.957+0000 7fc9156dd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9100cfd70 0x7fc91014d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:29.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.957+0000 7fc9156dd640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc91015be60 0x7fc9100ae5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:29.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.957+0000 7fc9156dd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9100cc330 con 0x7fc9100cfd70 2024-04-25T09:02:29.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.957+0000 7fc9156dd640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9100cc4b0 con 0x7fc9100b0e50 2024-04-25T09:02:29.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.957+0000 7fc9156dd640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9100cef90 con 0x7fc91015be60 2024-04-25T09:02:29.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.957+0000 7fc90e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9100cfd70 0x7fc91014d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:29.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.957+0000 7fc90e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9100cfd70 0x7fc91014d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43692/0 (socket says 172.21.15.43:43692) 2024-04-25T09:02:29.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.957+0000 7fc90e7fc640 1 -- 172.21.15.43:0/3151627294 learned_addr learned my addr 172.21.15.43:0/3151627294 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:29.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.958+0000 7fc90effd640 1 --2- 172.21.15.43:0/3151627294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9100b0e50 0x7fc9101f7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:29.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.958+0000 7fc90f7fe640 1 --2- 172.21.15.43:0/3151627294 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc91015be60 0x7fc9100ae5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:29.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.958+0000 7fc90e7fc640 1 -- 172.21.15.43:0/3151627294 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc91015be60 msgr2=0x7fc9100ae5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:29.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.958+0000 7fc90e7fc640 1 --2- 172.21.15.43:0/3151627294 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc91015be60 0x7fc9100ae5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:29.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.958+0000 7fc90e7fc640 1 -- 172.21.15.43:0/3151627294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9100b0e50 msgr2=0x7fc9101f7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:29.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.958+0000 7fc90e7fc640 1 --2- 172.21.15.43:0/3151627294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9100b0e50 0x7fc9101f7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:29.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.958+0000 7fc90e7fc640 1 -- 172.21.15.43:0/3151627294 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9100aeec0 con 0x7fc9100cfd70 2024-04-25T09:02:29.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.958+0000 7fc90f7fe640 1 --2- 172.21.15.43:0/3151627294 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc91015be60 0x7fc9100ae5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:29.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.958+0000 7fc90effd640 1 --2- 172.21.15.43:0/3151627294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9100b0e50 0x7fc9101f7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:29.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.958+0000 7fc90e7fc640 1 --2- 172.21.15.43:0/3151627294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9100cfd70 0x7fc91014d300 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fc904035040 tx=0x7fc9040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:29.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.958+0000 7fc8f77fe640 1 -- 172.21.15.43:0/3151627294 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc904037040 con 0x7fc9100cfd70 2024-04-25T09:02:29.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.958+0000 7fc8f77fe640 1 -- 172.21.15.43:0/3151627294 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc904005990 con 0x7fc9100cfd70 2024-04-25T09:02:29.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.958+0000 7fc9156dd640 1 -- 172.21.15.43:0/3151627294 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9100af1a0 con 0x7fc9100cfd70 2024-04-25T09:02:29.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.958+0000 7fc8f77fe640 1 -- 172.21.15.43:0/3151627294 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc904005d80 con 0x7fc9100cfd70 2024-04-25T09:02:29.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.958+0000 7fc9156dd640 1 -- 172.21.15.43:0/3151627294 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9100af760 con 0x7fc9100cfd70 2024-04-25T09:02:29.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.959+0000 7fc9156dd640 1 -- 172.21.15.43:0/3151627294 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc910074ef0 con 0x7fc9100cfd70 2024-04-25T09:02:29.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.960+0000 7fc8f77fe640 1 -- 172.21.15.43:0/3151627294 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc90403a070 con 0x7fc9100cfd70 2024-04-25T09:02:29.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.960+0000 7fc8f77fe640 1 --2- 172.21.15.43:0/3151627294 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc8fc06b4d0 0x7fc8fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:29.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.960+0000 7fc8f77fe640 1 -- 172.21.15.43:0/3151627294 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc9040aa4a0 con 0x7fc9100cfd70 2024-04-25T09:02:29.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.960+0000 7fc90effd640 1 --2- 172.21.15.43:0/3151627294 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc8fc06b4d0 0x7fc8fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:29.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.961+0000 7fc90effd640 1 --2- 172.21.15.43:0/3151627294 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc8fc06b4d0 0x7fc8fc06d990 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fc9000011e0 tx=0x7fc900009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:29.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:29.964+0000 7fc8f77fe640 1 -- 172.21.15.43:0/3151627294 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc904041020 con 0x7fc9100cfd70 2024-04-25T09:02:30.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.182+0000 7fc9156dd640 1 -- 172.21.15.43:0/3151627294 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9100cfb50 con 0x7fc9100cfd70 2024-04-25T09:02:30.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.182+0000 7fc8f77fe640 1 -- 172.21.15.43:0/3151627294 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fc9040793b0 con 0x7fc9100cfd70 2024-04-25T09:02:30.175 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:30.175 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":69,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6821,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2021228703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2021228703},{"type":"v1","addr":"172.21.15.193:6837","nonce":2021228703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65},{"gid":7543,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/2398324998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2398324998},{"type":"v1","addr":"172.21.15.193:6836","nonce":2398324998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65},{"gid":8673,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2298187831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2298187831},{"type":"v1","addr":"172.21.15.43:6834","nonce":2298187831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:02:26.161487+0000","modified":"2024-04-25T09:02:29.177294+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8676":{"gid":8676,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/1586865031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1586865031},{"type":"v1","addr":"172.21.15.43:6835","nonce":1586865031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:02:30.176 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 69 2024-04-25T09:02:30.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.184+0000 7fc9156dd640 1 -- 172.21.15.43:0/3151627294 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc8fc06b4d0 msgr2=0x7fc8fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:30.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.184+0000 7fc9156dd640 1 --2- 172.21.15.43:0/3151627294 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc8fc06b4d0 0x7fc8fc06d990 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fc9000011e0 tx=0x7fc900009040 comp rx=0 tx=0).stop 2024-04-25T09:02:30.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.184+0000 7fc9156dd640 1 -- 172.21.15.43:0/3151627294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9100cfd70 msgr2=0x7fc91014d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:30.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.184+0000 7fc9156dd640 1 --2- 172.21.15.43:0/3151627294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9100cfd70 0x7fc91014d300 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7fc904035040 tx=0x7fc9040046c0 comp rx=0 tx=0).stop 2024-04-25T09:02:30.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.184+0000 7fc9156dd640 1 -- 172.21.15.43:0/3151627294 shutdown_connections 2024-04-25T09:02:30.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.184+0000 7fc9156dd640 1 --2- 172.21.15.43:0/3151627294 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc91015be60 0x7fc9100ae5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:30.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.184+0000 7fc9156dd640 1 --2- 172.21.15.43:0/3151627294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9100cfd70 0x7fc91014d300 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:30.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.184+0000 7fc9156dd640 1 --2- 172.21.15.43:0/3151627294 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc8fc06b4d0 0x7fc8fc06d990 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:30.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.184+0000 7fc9156dd640 1 --2- 172.21.15.43:0/3151627294 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9100b0e50 0x7fc9101f7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:30.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.184+0000 7fc9156dd640 1 -- 172.21.15.43:0/3151627294 >> 172.21.15.43:0/3151627294 conn(0x7fc9100b32e0 msgr2=0x7fc91015b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:30.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.185+0000 7fc9156dd640 1 -- 172.21.15.43:0/3151627294 shutdown_connections 2024-04-25T09:02:30.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.185+0000 7fc9156dd640 1 -- 172.21.15.43:0/3151627294 wait complete. 2024-04-25T09:02:30.185 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:02:30.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.298+0000 7f7ca11e7640 1 Processor -- start 2024-04-25T09:02:30.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.298+0000 7f7ca11e7640 1 -- start start 2024-04-25T09:02:30.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.298+0000 7f7ca11e7640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7c9c06c510 0x7f7c9c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:30.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.298+0000 7f7ca11e7640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c9c0cbee0 con 0x7f7c9c0cd680 2024-04-25T09:02:30.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.298+0000 7f7ca11e7640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c9c0cee90 con 0x7f7c9c15df70 2024-04-25T09:02:30.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.298+0000 7f7ca11e7640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c9c0d0400 con 0x7f7c9c06c510 2024-04-25T09:02:30.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.299+0000 7f7c9ad9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7c9c06c510 0x7f7c9c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:30.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.299+0000 7f7c9b59e640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f7c9c0cd680 0x7f7c9c0ce780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48374/0 (socket says 172.21.15.43:48374) 2024-04-25T09:02:30.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.299+0000 7f7c9b59e640 1 -- 172.21.15.43:0/3522289641 learned_addr learned my addr 172.21.15.43:0/3522289641 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:30.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.299+0000 7f7c9ad9d640 1 -- 172.21.15.43:0/3522289641 >> v1:172.21.15.193:6789/0 conn(0x7f7c9c15df70 legacy=0x7f7c9c0ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:30.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.299+0000 7f7c9a59c640 1 -- 172.21.15.43:0/3522289641 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3915621122 0 0) 0x7f7c9c0cbee0 con 0x7f7c9c0cd680 2024-04-25T09:02:30.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.299+0000 7f7c9ad9d640 1 -- 172.21.15.43:0/3522289641 >> v1:172.21.15.43:6789/0 conn(0x7f7c9c0cd680 legacy=0x7f7c9c0ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:30.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.299+0000 7f7c9ad9d640 1 -- 172.21.15.43:0/3522289641 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c9c06ce30 con 0x7f7c9c06c510 2024-04-25T09:02:30.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.300+0000 7f7c9ad9d640 1 --2- 172.21.15.43:0/3522289641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c9c06c510 0x7f7c9c06c8f0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c0097a0 tx=0x7f7c8c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d314f61cbe0c9fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:30.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.300+0000 7f7c9a59c640 1 -- 172.21.15.43:0/3522289641 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c037070 con 0x7f7c9c06c510 2024-04-25T09:02:30.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.300+0000 7f7c9a59c640 1 -- 172.21.15.43:0/3522289641 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c8c032440 con 0x7f7c9c06c510 2024-04-25T09:02:30.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.300+0000 7f7c9a59c640 1 -- 172.21.15.43:0/3522289641 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c8c032810 con 0x7f7c9c06c510 2024-04-25T09:02:30.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.300+0000 7f7ca11e7640 1 -- 172.21.15.43:0/3522289641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c9c06c510 msgr2=0x7f7c9c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:30.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.300+0000 7f7ca11e7640 1 --2- 172.21.15.43:0/3522289641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c9c06c510 0x7f7c9c06c8f0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c0097a0 tx=0x7f7c8c02a1e0 comp rx=0 tx=0).stop 2024-04-25T09:02:30.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.300+0000 7f7ca11e7640 1 -- 172.21.15.43:0/3522289641 shutdown_connections 2024-04-25T09:02:30.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.300+0000 7f7ca11e7640 1 --2- 172.21.15.43:0/3522289641 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c9c06c510 0x7f7c9c06c8f0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:30.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.300+0000 7f7ca11e7640 1 -- 172.21.15.43:0/3522289641 >> 172.21.15.43:0/3522289641 conn(0x7f7c9c0d17f0 msgr2=0x7f7c9c0d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:30.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.300+0000 7f7ca11e7640 1 -- 172.21.15.43:0/3522289641 shutdown_connections 2024-04-25T09:02:30.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.301+0000 7f7ca11e7640 1 -- 172.21.15.43:0/3522289641 wait complete. 2024-04-25T09:02:30.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.302+0000 7f7ca11e7640 1 Processor -- start 2024-04-25T09:02:30.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.302+0000 7f7ca11e7640 1 -- start start 2024-04-25T09:02:30.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.303+0000 7f7ca11e7640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7c9c06c510 0x7f7c9c14c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:30.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.303+0000 7f7ca11e7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c9c0cd680 0x7f7c9c0af720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:30.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.303+0000 7f7ca11e7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c9c15df70 0x7f7c9c0afc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:30.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.303+0000 7f7ca11e7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9c0cc150 con 0x7f7c9c0cd680 2024-04-25T09:02:30.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.303+0000 7f7ca11e7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9c0d0930 con 0x7f7c9c15df70 2024-04-25T09:02:30.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.303+0000 7f7ca11e7640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9c0d0c10 con 0x7f7c9c06c510 2024-04-25T09:02:30.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.303+0000 7f7c93fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c9c0cd680 0x7f7c9c0af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:30.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.303+0000 7f7c93fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c9c0cd680 0x7f7c9c0af720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43700/0 (socket says 172.21.15.43:43700) 2024-04-25T09:02:30.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.303+0000 7f7c93fff640 1 -- 172.21.15.43:0/3164442594 learned_addr learned my addr 172.21.15.43:0/3164442594 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:30.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.303+0000 7f7c9ad9d640 1 --2- 172.21.15.43:0/3164442594 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7c9c06c510 0x7f7c9c14c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:30.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.303+0000 7f7c9b59e640 1 --2- 172.21.15.43:0/3164442594 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c9c15df70 0x7f7c9c0afc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:30.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.303+0000 7f7c93fff640 1 -- 172.21.15.43:0/3164442594 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7c9c06c510 msgr2=0x7f7c9c14c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:30.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.303+0000 7f7c93fff640 1 --2- 172.21.15.43:0/3164442594 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7c9c06c510 0x7f7c9c14c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:30.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.303+0000 7f7c93fff640 1 -- 172.21.15.43:0/3164442594 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c9c15df70 msgr2=0x7f7c9c0afc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:30.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.303+0000 7f7c93fff640 1 --2- 172.21.15.43:0/3164442594 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c9c15df70 0x7f7c9c0afc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:30.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.303+0000 7f7c93fff640 1 -- 172.21.15.43:0/3164442594 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c9c0c9540 con 0x7f7c9c0cd680 2024-04-25T09:02:30.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.303+0000 7f7c9b59e640 1 --2- 172.21.15.43:0/3164442594 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c9c15df70 0x7f7c9c0afc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:30.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.303+0000 7f7c93fff640 1 --2- 172.21.15.43:0/3164442594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c9c0cd680 0x7f7c9c0af720 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f7c8400c400 tx=0x7f7c8400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:30.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.303+0000 7f7c937fe640 1 -- 172.21.15.43:0/3164442594 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c84011790 con 0x7f7c9c0cd680 2024-04-25T09:02:30.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.303+0000 7f7c937fe640 1 -- 172.21.15.43:0/3164442594 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c84017070 con 0x7f7c9c0cd680 2024-04-25T09:02:30.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.303+0000 7f7ca11e7640 1 -- 172.21.15.43:0/3164442594 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c9c14ca40 con 0x7f7c9c0cd680 2024-04-25T09:02:30.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.303+0000 7f7c937fe640 1 -- 172.21.15.43:0/3164442594 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c84011bb0 con 0x7f7c9c0cd680 2024-04-25T09:02:30.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.303+0000 7f7ca11e7640 1 -- 172.21.15.43:0/3164442594 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c9c14d000 con 0x7f7c9c0cd680 2024-04-25T09:02:30.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.304+0000 7f7ca11e7640 1 -- 172.21.15.43:0/3164442594 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c9c074ef0 con 0x7f7c9c0cd680 2024-04-25T09:02:30.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.305+0000 7f7c937fe640 1 -- 172.21.15.43:0/3164442594 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7c8401c460 con 0x7f7c9c0cd680 2024-04-25T09:02:30.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.305+0000 7f7c937fe640 1 --2- 172.21.15.43:0/3164442594 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7c7c06b4d0 0x7f7c7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:30.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.305+0000 7f7c937fe640 1 -- 172.21.15.43:0/3164442594 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7c8408a050 con 0x7f7c9c0cd680 2024-04-25T09:02:30.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.306+0000 7f7c9ad9d640 1 --2- 172.21.15.43:0/3164442594 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7c7c06b4d0 0x7f7c7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:30.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.306+0000 7f7c9ad9d640 1 --2- 172.21.15.43:0/3164442594 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7c7c06b4d0 0x7f7c7c06d990 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c034000 tx=0x7f7c8c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:30.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.310+0000 7f7c937fe640 1 -- 172.21.15.43:0/3164442594 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7c84058e40 con 0x7f7c9c0cd680 2024-04-25T09:02:30.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.489+0000 7f7ca11e7640 1 -- 172.21.15.43:0/3164442594 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c9c0c8560 con 0x7f7c9c0cd680 2024-04-25T09:02:30.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.491+0000 7f7c937fe640 1 -- 172.21.15.43:0/3164442594 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+16591 (secure 0 0 0) 0x7f7c84058740 con 0x7f7c9c0cd680 2024-04-25T09:02:30.479 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:30.480 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":113,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:02:27.811056+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:02:24.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T09:02:25.143723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:02:30.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.492+0000 7f7ca11e7640 1 -- 172.21.15.43:0/3164442594 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7c7c06b4d0 msgr2=0x7f7c7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:30.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.492+0000 7f7ca11e7640 1 --2- 172.21.15.43:0/3164442594 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7c7c06b4d0 0x7f7c7c06d990 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c034000 tx=0x7f7c8c02e000 comp rx=0 tx=0).stop 2024-04-25T09:02:30.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.492+0000 7f7ca11e7640 1 -- 172.21.15.43:0/3164442594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c9c0cd680 msgr2=0x7f7c9c0af720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:30.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.492+0000 7f7ca11e7640 1 --2- 172.21.15.43:0/3164442594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c9c0cd680 0x7f7c9c0af720 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f7c8400c400 tx=0x7f7c8400c8c0 comp rx=0 tx=0).stop 2024-04-25T09:02:30.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.493+0000 7f7ca11e7640 1 -- 172.21.15.43:0/3164442594 shutdown_connections 2024-04-25T09:02:30.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.493+0000 7f7ca11e7640 1 --2- 172.21.15.43:0/3164442594 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7c7c06b4d0 0x7f7c7c06d990 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:30.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.493+0000 7f7ca11e7640 1 --2- 172.21.15.43:0/3164442594 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c9c15df70 0x7f7c9c0afc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:30.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.493+0000 7f7ca11e7640 1 --2- 172.21.15.43:0/3164442594 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c9c0cd680 0x7f7c9c0af720 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:30.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.493+0000 7f7ca11e7640 1 --2- 172.21.15.43:0/3164442594 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7c9c06c510 0x7f7c9c14c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:30.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.493+0000 7f7ca11e7640 1 -- 172.21.15.43:0/3164442594 >> 172.21.15.43:0/3164442594 conn(0x7f7c9c0d17f0 msgr2=0x7f7c9c06f950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:30.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.493+0000 7f7ca11e7640 1 -- 172.21.15.43:0/3164442594 shutdown_connections 2024-04-25T09:02:30.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.493+0000 7f7ca11e7640 1 -- 172.21.15.43:0/3164442594 wait complete. 2024-04-25T09:02:30.493 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:02:30.493 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:02:30.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.606+0000 7f20a9dd5640 1 Processor -- start 2024-04-25T09:02:30.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.606+0000 7f20a9dd5640 1 -- start start 2024-04-25T09:02:30.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.606+0000 7f20a9dd5640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f20a406b240 0x7f20a40cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:30.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.606+0000 7f20a9dd5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f20a40cefa0 0x7f20a406ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:30.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.606+0000 7f20a9dd5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20a40cf790 con 0x7f20a415bbf0 2024-04-25T09:02:30.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.606+0000 7f20a9dd5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20a40cfa70 con 0x7f20a40cefa0 2024-04-25T09:02:30.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.606+0000 7f20a9dd5640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20a40cfd50 con 0x7f20a406b240 2024-04-25T09:02:30.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.607+0000 7f20a3fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f20a40cefa0 0x7f20a406ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:30.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.607+0000 7f20a37fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f20a415bbf0 0x7f20a406ab30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48378/0 (socket says 172.21.15.43:48378) 2024-04-25T09:02:30.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.607+0000 7f20a3fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f20a40cefa0 0x7f20a406ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43708/0 (socket says 172.21.15.43:43708) 2024-04-25T09:02:30.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.607+0000 7f20a37fe640 1 -- 172.21.15.43:0/3575815977 learned_addr learned my addr 172.21.15.43:0/3575815977 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:30.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.607+0000 7f20a2ffd640 1 --2- 172.21.15.43:0/3575815977 >> v2:172.21.15.193:3300/0 conn(0x7f20a406b240 0x7f20a40cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:30.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.607+0000 7f20a3fff640 1 -- 172.21.15.43:0/3575815977 >> v2:172.21.15.193:3300/0 conn(0x7f20a406b240 msgr2=0x7f20a40cea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:30.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.607+0000 7f20a3fff640 1 --2- 172.21.15.43:0/3575815977 >> v2:172.21.15.193:3300/0 conn(0x7f20a406b240 0x7f20a40cea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:30.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.607+0000 7f20a3fff640 1 -- 172.21.15.43:0/3575815977 >> v1:172.21.15.43:6789/0 conn(0x7f20a415bbf0 legacy=0x7f20a406ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:30.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.607+0000 7f20a27fc640 1 -- 172.21.15.43:0/3575815977 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3519750690 0 0) 0x7f20a40cf790 con 0x7f20a415bbf0 2024-04-25T09:02:30.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.607+0000 7f20a3fff640 1 -- 172.21.15.43:0/3575815977 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a40d0030 con 0x7f20a40cefa0 2024-04-25T09:02:30.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.608+0000 7f20a3fff640 1 --2- 172.21.15.43:0/3575815977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20a40cefa0 0x7f20a406ea50 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f20a40af9e0 tx=0x7f209c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=295a67afb73174f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:30.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.608+0000 7f20a27fc640 1 -- 172.21.15.43:0/3575815977 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209c007740 con 0x7f20a40cefa0 2024-04-25T09:02:30.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.608+0000 7f20a27fc640 1 -- 172.21.15.43:0/3575815977 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f209c0078e0 con 0x7f20a40cefa0 2024-04-25T09:02:30.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.608+0000 7f20a27fc640 1 -- 172.21.15.43:0/3575815977 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209c007cb0 con 0x7f20a40cefa0 2024-04-25T09:02:30.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.608+0000 7f20a9dd5640 1 -- 172.21.15.43:0/3575815977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20a40cefa0 msgr2=0x7f20a406ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:30.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.608+0000 7f20a9dd5640 1 --2- 172.21.15.43:0/3575815977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20a40cefa0 0x7f20a406ea50 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f20a40af9e0 tx=0x7f209c02a710 comp rx=0 tx=0).stop 2024-04-25T09:02:30.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.608+0000 7f20a9dd5640 1 -- 172.21.15.43:0/3575815977 shutdown_connections 2024-04-25T09:02:30.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.608+0000 7f20a9dd5640 1 --2- 172.21.15.43:0/3575815977 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20a40cefa0 0x7f20a406ea50 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:30.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.608+0000 7f20a9dd5640 1 --2- 172.21.15.43:0/3575815977 >> v2:172.21.15.193:3300/0 conn(0x7f20a406b240 0x7f20a40cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:30.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.608+0000 7f20a9dd5640 1 -- 172.21.15.43:0/3575815977 >> 172.21.15.43:0/3575815977 conn(0x7f20a40b2490 msgr2=0x7f20a40b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:30.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.608+0000 7f20a9dd5640 1 -- 172.21.15.43:0/3575815977 shutdown_connections 2024-04-25T09:02:30.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.609+0000 7f20a9dd5640 1 -- 172.21.15.43:0/3575815977 wait complete. 2024-04-25T09:02:30.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.610+0000 7f20a9dd5640 1 Processor -- start 2024-04-25T09:02:30.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.610+0000 7f20a9dd5640 1 -- start start 2024-04-25T09:02:30.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.610+0000 7f20a9dd5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20a406b240 0x7f20a414e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:30.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.610+0000 7f20a9dd5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20a40cefa0 0x7f20a40b4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:30.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.610+0000 7f20a9dd5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20a415bbf0 0x7f20a40adb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:30.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.611+0000 7f20a9dd5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a415c150 con 0x7f20a40cefa0 2024-04-25T09:02:30.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.611+0000 7f20a9dd5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a415bfd0 con 0x7f20a406b240 2024-04-25T09:02:30.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.611+0000 7f20a9dd5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20a406ef90 con 0x7f20a415bbf0 2024-04-25T09:02:30.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.611+0000 7f20a2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20a40cefa0 0x7f20a40b4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:30.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.611+0000 7f20a2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20a40cefa0 0x7f20a40b4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43714/0 (socket says 172.21.15.43:43714) 2024-04-25T09:02:30.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.611+0000 7f20a2ffd640 1 -- 172.21.15.43:0/2506929441 learned_addr learned my addr 172.21.15.43:0/2506929441 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:30.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.611+0000 7f20a37fe640 1 --2- 172.21.15.43:0/2506929441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20a406b240 0x7f20a414e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:30.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.611+0000 7f20a3fff640 1 --2- 172.21.15.43:0/2506929441 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20a415bbf0 0x7f20a40adb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:30.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.611+0000 7f20a2ffd640 1 -- 172.21.15.43:0/2506929441 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20a415bbf0 msgr2=0x7f20a40adb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:30.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.611+0000 7f20a2ffd640 1 --2- 172.21.15.43:0/2506929441 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20a415bbf0 0x7f20a40adb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:30.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.611+0000 7f20a2ffd640 1 -- 172.21.15.43:0/2506929441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20a406b240 msgr2=0x7f20a414e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:30.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.611+0000 7f20a2ffd640 1 --2- 172.21.15.43:0/2506929441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20a406b240 0x7f20a414e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:30.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.611+0000 7f20a2ffd640 1 -- 172.21.15.43:0/2506929441 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a40b5200 con 0x7f20a40cefa0 2024-04-25T09:02:30.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.611+0000 7f20a37fe640 1 --2- 172.21.15.43:0/2506929441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20a406b240 0x7f20a414e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:30.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.611+0000 7f20a3fff640 1 --2- 172.21.15.43:0/2506929441 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20a415bbf0 0x7f20a40adb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:30.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.611+0000 7f20a2ffd640 1 --2- 172.21.15.43:0/2506929441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20a40cefa0 0x7f20a40b4780 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f209800e960 tx=0x7f209800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:30.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.611+0000 7f208bfff640 1 -- 172.21.15.43:0/2506929441 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2098017070 con 0x7f20a40cefa0 2024-04-25T09:02:30.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.611+0000 7f208bfff640 1 -- 172.21.15.43:0/2506929441 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f209800ca00 con 0x7f20a40cefa0 2024-04-25T09:02:30.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.611+0000 7f20a9dd5640 1 -- 172.21.15.43:0/2506929441 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20a40ab7b0 con 0x7f20a40cefa0 2024-04-25T09:02:30.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.611+0000 7f208bfff640 1 -- 172.21.15.43:0/2506929441 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209800cdf0 con 0x7f20a40cefa0 2024-04-25T09:02:30.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.611+0000 7f20a9dd5640 1 -- 172.21.15.43:0/2506929441 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a40abd70 con 0x7f20a40cefa0 2024-04-25T09:02:30.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.612+0000 7f20a9dd5640 1 -- 172.21.15.43:0/2506929441 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20a4074ef0 con 0x7f20a40cefa0 2024-04-25T09:02:30.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.613+0000 7f208bfff640 1 -- 172.21.15.43:0/2506929441 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f209801a070 con 0x7f20a40cefa0 2024-04-25T09:02:30.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.613+0000 7f208bfff640 1 --2- 172.21.15.43:0/2506929441 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f209006b500 0x7f209006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:30.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.613+0000 7f208bfff640 1 -- 172.21.15.43:0/2506929441 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f209808a670 con 0x7f20a40cefa0 2024-04-25T09:02:30.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.614+0000 7f20a37fe640 1 --2- 172.21.15.43:0/2506929441 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f209006b500 0x7f209006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:30.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.614+0000 7f20a37fe640 1 --2- 172.21.15.43:0/2506929441 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f209006b500 0x7f209006d9c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f2094000f60 tx=0x7f2094009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:30.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.617+0000 7f208bfff640 1 -- 172.21.15.43:0/2506929441 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2098059450 con 0x7f20a40cefa0 2024-04-25T09:02:30.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.795+0000 7f20a9dd5640 1 -- 172.21.15.43:0/2506929441 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f20a4071710 con 0x7f209006b500 2024-04-25T09:02:30.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.802+0000 7f208bfff640 1 -- 172.21.15.43:0/2506929441 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252482 (secure 0 0 0) 0x7f20a4071710 con 0x7f209006b500 2024-04-25T09:02:30.793 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:30.795 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-25T09:02:30.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.805+0000 7f20a9dd5640 1 -- 172.21.15.43:0/2506929441 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f209006b500 msgr2=0x7f209006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:30.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.805+0000 7f20a9dd5640 1 --2- 172.21.15.43:0/2506929441 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f209006b500 0x7f209006d9c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f2094000f60 tx=0x7f2094009040 comp rx=0 tx=0).stop 2024-04-25T09:02:30.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.805+0000 7f20a9dd5640 1 -- 172.21.15.43:0/2506929441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20a40cefa0 msgr2=0x7f20a40b4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:30.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.805+0000 7f20a9dd5640 1 --2- 172.21.15.43:0/2506929441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20a40cefa0 0x7f20a40b4780 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f209800e960 tx=0x7f209800ee20 comp rx=0 tx=0).stop 2024-04-25T09:02:30.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.805+0000 7f20a9dd5640 1 -- 172.21.15.43:0/2506929441 shutdown_connections 2024-04-25T09:02:30.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.805+0000 7f20a9dd5640 1 --2- 172.21.15.43:0/2506929441 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20a415bbf0 0x7f20a40adb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:30.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.805+0000 7f20a9dd5640 1 --2- 172.21.15.43:0/2506929441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20a40cefa0 0x7f20a40b4780 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:30.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.805+0000 7f20a9dd5640 1 --2- 172.21.15.43:0/2506929441 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f209006b500 0x7f209006d9c0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:30.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.805+0000 7f20a9dd5640 1 --2- 172.21.15.43:0/2506929441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20a406b240 0x7f20a414e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:30.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.805+0000 7f20a9dd5640 1 -- 172.21.15.43:0/2506929441 >> 172.21.15.43:0/2506929441 conn(0x7f20a40b2490 msgr2=0x7f20a406f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:30.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.805+0000 7f20a9dd5640 1 -- 172.21.15.43:0/2506929441 shutdown_connections 2024-04-25T09:02:30.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.805+0000 7f20a9dd5640 1 -- 172.21.15.43:0/2506929441 wait complete. 2024-04-25T09:02:30.805 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":362,"stamp":"2024-04-25T09:02:29.809561+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":494888,"kb_used_data":12200,"kb_used_omap":12,"kb_used_meta":482547,"kb_avail":754479832,"statfs":{"total":773094113280,"available":772587347968,"internally_reserved":0,"allocated":12492800,"data_stored":6069812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":494128689},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.023517"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.863781+0000","last_change":"2024-04-25T09:02:26.822906+0000","last_active":"2024-04-25T09:02:27.863781+0000","last_peered":"2024-04-25T09:02:27.863781+0000","last_clean":"2024-04-25T09:02:27.863781+0000","last_became_active":"2024-04-25T09:02:26.820737+0000","last_became_peered":"2024-04-25T09:02:26.820737+0000","last_unstale":"2024-04-25T09:02:27.863781+0000","last_undegraded":"2024-04-25T09:02:27.863781+0000","last_fullsized":"2024-04-25T09:02:27.863781+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:49.004736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.820468+0000","last_change":"2024-04-25T09:02:26.812898+0000","last_active":"2024-04-25T09:02:27.820468+0000","last_peered":"2024-04-25T09:02:27.820468+0000","last_clean":"2024-04-25T09:02:27.820468+0000","last_became_active":"2024-04-25T09:02:26.812637+0000","last_became_peered":"2024-04-25T09:02:26.812637+0000","last_unstale":"2024-04-25T09:02:27.820468+0000","last_undegraded":"2024-04-25T09:02:27.820468+0000","last_fullsized":"2024-04-25T09:02:27.820468+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:54.415160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.863057+0000","last_change":"2024-04-25T09:02:26.823065+0000","last_active":"2024-04-25T09:02:27.863057+0000","last_peered":"2024-04-25T09:02:27.863057+0000","last_clean":"2024-04-25T09:02:27.863057+0000","last_became_active":"2024-04-25T09:02:26.820904+0000","last_became_peered":"2024-04-25T09:02:26.820904+0000","last_unstale":"2024-04-25T09:02:27.863057+0000","last_undegraded":"2024-04-25T09:02:27.863057+0000","last_fullsized":"2024-04-25T09:02:27.863057+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:13.782020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862173+0000","last_change":"2024-04-25T09:02:26.822436+0000","last_active":"2024-04-25T09:02:27.862173+0000","last_peered":"2024-04-25T09:02:27.862173+0000","last_clean":"2024-04-25T09:02:27.862173+0000","last_became_active":"2024-04-25T09:02:26.817810+0000","last_became_peered":"2024-04-25T09:02:26.817810+0000","last_unstale":"2024-04-25T09:02:27.862173+0000","last_undegraded":"2024-04-25T09:02:27.862173+0000","last_fullsized":"2024-04-25T09:02:27.862173+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:25.845136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.819307+0000","last_change":"2024-04-25T09:02:26.818665+0000","last_active":"2024-04-25T09:02:27.819307+0000","last_peered":"2024-04-25T09:02:27.819307+0000","last_clean":"2024-04-25T09:02:27.819307+0000","last_became_active":"2024-04-25T09:02:26.818410+0000","last_became_peered":"2024-04-25T09:02:26.818410+0000","last_unstale":"2024-04-25T09:02:27.819307+0000","last_undegraded":"2024-04-25T09:02:27.819307+0000","last_fullsized":"2024-04-25T09:02:27.819307+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:56.211878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862327+0000","last_change":"2024-04-25T09:02:26.817409+0000","last_active":"2024-04-25T09:02:27.862327+0000","last_peered":"2024-04-25T09:02:27.862327+0000","last_clean":"2024-04-25T09:02:27.862327+0000","last_became_active":"2024-04-25T09:02:26.817132+0000","last_became_peered":"2024-04-25T09:02:26.817132+0000","last_unstale":"2024-04-25T09:02:27.862327+0000","last_undegraded":"2024-04-25T09:02:27.862327+0000","last_fullsized":"2024-04-25T09:02:27.862327+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:54.582797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.820049+0000","last_change":"2024-04-25T09:02:26.811298+0000","last_active":"2024-04-25T09:02:27.820049+0000","last_peered":"2024-04-25T09:02:27.820049+0000","last_clean":"2024-04-25T09:02:27.820049+0000","last_became_active":"2024-04-25T09:02:26.810910+0000","last_became_peered":"2024-04-25T09:02:26.810910+0000","last_unstale":"2024-04-25T09:02:27.820049+0000","last_undegraded":"2024-04-25T09:02:27.820049+0000","last_fullsized":"2024-04-25T09:02:27.820049+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:54.582797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862330+0000","last_change":"2024-04-25T09:02:26.815668+0000","last_active":"2024-04-25T09:02:27.862330+0000","last_peered":"2024-04-25T09:02:27.862330+0000","last_clean":"2024-04-25T09:02:27.862330+0000","last_became_active":"2024-04-25T09:02:26.815398+0000","last_became_peered":"2024-04-25T09:02:26.815398+0000","last_unstale":"2024-04-25T09:02:27.862330+0000","last_undegraded":"2024-04-25T09:02:27.862330+0000","last_fullsized":"2024-04-25T09:02:27.862330+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:15.237882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.863870+0000","last_change":"2024-04-25T09:02:26.826585+0000","last_active":"2024-04-25T09:02:27.863870+0000","last_peered":"2024-04-25T09:02:27.863870+0000","last_clean":"2024-04-25T09:02:27.863870+0000","last_became_active":"2024-04-25T09:02:26.826327+0000","last_became_peered":"2024-04-25T09:02:26.826327+0000","last_unstale":"2024-04-25T09:02:27.863870+0000","last_undegraded":"2024-04-25T09:02:27.863870+0000","last_fullsized":"2024-04-25T09:02:27.863870+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:00.827972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862801+0000","last_change":"2024-04-25T09:02:26.814881+0000","last_active":"2024-04-25T09:02:27.862801+0000","last_peered":"2024-04-25T09:02:27.862801+0000","last_clean":"2024-04-25T09:02:27.862801+0000","last_became_active":"2024-04-25T09:02:26.814621+0000","last_became_peered":"2024-04-25T09:02:26.814621+0000","last_unstale":"2024-04-25T09:02:27.862801+0000","last_undegraded":"2024-04-25T09:02:27.862801+0000","last_fullsized":"2024-04-25T09:02:27.862801+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:25.970680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.820730+0000","last_change":"2024-04-25T09:02:26.810085+0000","last_active":"2024-04-25T09:02:27.820730+0000","last_peered":"2024-04-25T09:02:27.820730+0000","last_clean":"2024-04-25T09:02:27.820730+0000","last_became_active":"2024-04-25T09:02:26.809778+0000","last_became_peered":"2024-04-25T09:02:26.809778+0000","last_unstale":"2024-04-25T09:02:27.820730+0000","last_undegraded":"2024-04-25T09:02:27.820730+0000","last_fullsized":"2024-04-25T09:02:27.820730+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:02.307876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862458+0000","last_change":"2024-04-25T09:02:26.815397+0000","last_active":"2024-04-25T09:02:27.862458+0000","last_peered":"2024-04-25T09:02:27.862458+0000","last_clean":"2024-04-25T09:02:27.862458+0000","last_became_active":"2024-04-25T09:02:26.815054+0000","last_became_peered":"2024-04-25T09:02:26.815054+0000","last_unstale":"2024-04-25T09:02:27.862458+0000","last_undegraded":"2024-04-25T09:02:27.862458+0000","last_fullsized":"2024-04-25T09:02:27.862458+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:10.780038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862942+0000","last_change":"2024-04-25T09:02:26.815391+0000","last_active":"2024-04-25T09:02:27.862942+0000","last_peered":"2024-04-25T09:02:27.862942+0000","last_clean":"2024-04-25T09:02:27.862942+0000","last_became_active":"2024-04-25T09:02:26.815119+0000","last_became_peered":"2024-04-25T09:02:26.815119+0000","last_unstale":"2024-04-25T09:02:27.862942+0000","last_undegraded":"2024-04-25T09:02:27.862942+0000","last_fullsized":"2024-04-25T09:02:27.862942+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:51.706367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862300+0000","last_change":"2024-04-25T09:02:26.816722+0000","last_active":"2024-04-25T09:02:27.862300+0000","last_peered":"2024-04-25T09:02:27.862300+0000","last_clean":"2024-04-25T09:02:27.862300+0000","last_became_active":"2024-04-25T09:02:26.816455+0000","last_became_peered":"2024-04-25T09:02:26.816455+0000","last_unstale":"2024-04-25T09:02:27.862300+0000","last_undegraded":"2024-04-25T09:02:27.862300+0000","last_fullsized":"2024-04-25T09:02:27.862300+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:09.978129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862562+0000","last_change":"2024-04-25T09:02:26.818556+0000","last_active":"2024-04-25T09:02:27.862562+0000","last_peered":"2024-04-25T09:02:27.862562+0000","last_clean":"2024-04-25T09:02:27.862562+0000","last_became_active":"2024-04-25T09:02:26.818288+0000","last_became_peered":"2024-04-25T09:02:26.818288+0000","last_unstale":"2024-04-25T09:02:27.862562+0000","last_undegraded":"2024-04-25T09:02:27.862562+0000","last_fullsized":"2024-04-25T09:02:27.862562+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:02.253566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.861804+0000","last_change":"2024-04-25T09:02:26.829155+0000","last_active":"2024-04-25T09:02:27.861804+0000","last_peered":"2024-04-25T09:02:27.861804+0000","last_clean":"2024-04-25T09:02:27.861804+0000","last_became_active":"2024-04-25T09:02:26.821965+0000","last_became_peered":"2024-04-25T09:02:26.821965+0000","last_unstale":"2024-04-25T09:02:27.861804+0000","last_undegraded":"2024-04-25T09:02:27.861804+0000","last_fullsized":"2024-04-25T09:02:27.861804+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:33.563639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.820823+0000","last_change":"2024-04-25T09:02:26.811291+0000","last_active":"2024-04-25T09:02:27.820823+0000","last_peered":"2024-04-25T09:02:27.820823+0000","last_clean":"2024-04-25T09:02:27.820823+0000","last_became_active":"2024-04-25T09:02:26.810633+0000","last_became_peered":"2024-04-25T09:02:26.810633+0000","last_unstale":"2024-04-25T09:02:27.820823+0000","last_undegraded":"2024-04-25T09:02:27.820823+0000","last_fullsized":"2024-04-25T09:02:27.820823+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:09.693837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862452+0000","last_change":"2024-04-25T09:02:26.812521+0000","last_active":"2024-04-25T09:02:27.862452+0000","last_peered":"2024-04-25T09:02:27.862452+0000","last_clean":"2024-04-25T09:02:27.862452+0000","last_became_active":"2024-04-25T09:02:26.812192+0000","last_became_peered":"2024-04-25T09:02:26.812192+0000","last_unstale":"2024-04-25T09:02:27.862452+0000","last_undegraded":"2024-04-25T09:02:27.862452+0000","last_fullsized":"2024-04-25T09:02:27.862452+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:15.671723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.863192+0000","last_change":"2024-04-25T09:02:26.819158+0000","last_active":"2024-04-25T09:02:27.863192+0000","last_peered":"2024-04-25T09:02:27.863192+0000","last_clean":"2024-04-25T09:02:27.863192+0000","last_became_active":"2024-04-25T09:02:26.818576+0000","last_became_peered":"2024-04-25T09:02:26.818576+0000","last_unstale":"2024-04-25T09:02:27.863192+0000","last_undegraded":"2024-04-25T09:02:27.863192+0000","last_fullsized":"2024-04-25T09:02:27.863192+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:10.215927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.820733+0000","last_change":"2024-04-25T09:02:26.814724+0000","last_active":"2024-04-25T09:02:27.820733+0000","last_peered":"2024-04-25T09:02:27.820733+0000","last_clean":"2024-04-25T09:02:27.820733+0000","last_became_active":"2024-04-25T09:02:26.813861+0000","last_became_peered":"2024-04-25T09:02:26.813861+0000","last_unstale":"2024-04-25T09:02:27.820733+0000","last_undegraded":"2024-04-25T09:02:27.820733+0000","last_fullsized":"2024-04-25T09:02:27.820733+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:48.848804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.820889+0000","last_change":"2024-04-25T09:02:26.819381+0000","last_active":"2024-04-25T09:02:27.820889+0000","last_peered":"2024-04-25T09:02:27.820889+0000","last_clean":"2024-04-25T09:02:27.820889+0000","last_became_active":"2024-04-25T09:02:26.819122+0000","last_became_peered":"2024-04-25T09:02:26.819122+0000","last_unstale":"2024-04-25T09:02:27.820889+0000","last_undegraded":"2024-04-25T09:02:27.820889+0000","last_fullsized":"2024-04-25T09:02:27.820889+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:09.320655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862701+0000","last_change":"2024-04-25T09:02:26.815215+0000","last_active":"2024-04-25T09:02:27.862701+0000","last_peered":"2024-04-25T09:02:27.862701+0000","last_clean":"2024-04-25T09:02:27.862701+0000","last_became_active":"2024-04-25T09:02:26.814922+0000","last_became_peered":"2024-04-25T09:02:26.814922+0000","last_unstale":"2024-04-25T09:02:27.862701+0000","last_undegraded":"2024-04-25T09:02:27.862701+0000","last_fullsized":"2024-04-25T09:02:27.862701+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:51.327363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.819800+0000","last_change":"2024-04-25T09:02:26.814270+0000","last_active":"2024-04-25T09:02:27.819800+0000","last_peered":"2024-04-25T09:02:27.819800+0000","last_clean":"2024-04-25T09:02:27.819800+0000","last_became_active":"2024-04-25T09:02:26.813869+0000","last_became_peered":"2024-04-25T09:02:26.813869+0000","last_unstale":"2024-04-25T09:02:27.819800+0000","last_undegraded":"2024-04-25T09:02:27.819800+0000","last_fullsized":"2024-04-25T09:02:27.819800+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:00.537477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862651+0000","last_change":"2024-04-25T09:02:26.817901+0000","last_active":"2024-04-25T09:02:27.862651+0000","last_peered":"2024-04-25T09:02:27.862651+0000","last_clean":"2024-04-25T09:02:27.862651+0000","last_became_active":"2024-04-25T09:02:26.817575+0000","last_became_peered":"2024-04-25T09:02:26.817575+0000","last_unstale":"2024-04-25T09:02:27.862651+0000","last_undegraded":"2024-04-25T09:02:27.862651+0000","last_fullsized":"2024-04-25T09:02:27.862651+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:15.237882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.820890+0000","last_change":"2024-04-25T09:02:26.812084+0000","last_active":"2024-04-25T09:02:27.820890+0000","last_peered":"2024-04-25T09:02:27.820890+0000","last_clean":"2024-04-25T09:02:27.820890+0000","last_became_active":"2024-04-25T09:02:26.811024+0000","last_became_peered":"2024-04-25T09:02:26.811024+0000","last_unstale":"2024-04-25T09:02:27.820890+0000","last_undegraded":"2024-04-25T09:02:27.820890+0000","last_fullsized":"2024-04-25T09:02:27.820890+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:53.694921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862935+0000","last_change":"2024-04-25T09:02:26.823420+0000","last_active":"2024-04-25T09:02:27.862935+0000","last_peered":"2024-04-25T09:02:27.862935+0000","last_clean":"2024-04-25T09:02:27.862935+0000","last_became_active":"2024-04-25T09:02:26.815839+0000","last_became_peered":"2024-04-25T09:02:26.815839+0000","last_unstale":"2024-04-25T09:02:27.862935+0000","last_undegraded":"2024-04-25T09:02:27.862935+0000","last_fullsized":"2024-04-25T09:02:27.862935+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:38.689898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.863183+0000","last_change":"2024-04-25T09:02:26.817341+0000","last_active":"2024-04-25T09:02:27.863183+0000","last_peered":"2024-04-25T09:02:27.863183+0000","last_clean":"2024-04-25T09:02:27.863183+0000","last_became_active":"2024-04-25T09:02:26.816313+0000","last_became_peered":"2024-04-25T09:02:26.816313+0000","last_unstale":"2024-04-25T09:02:27.863183+0000","last_undegraded":"2024-04-25T09:02:27.863183+0000","last_fullsized":"2024-04-25T09:02:27.863183+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:19.803627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.863125+0000","last_change":"2024-04-25T09:02:26.817297+0000","last_active":"2024-04-25T09:02:27.863125+0000","last_peered":"2024-04-25T09:02:27.863125+0000","last_clean":"2024-04-25T09:02:27.863125+0000","last_became_active":"2024-04-25T09:02:26.816930+0000","last_became_peered":"2024-04-25T09:02:26.816930+0000","last_unstale":"2024-04-25T09:02:27.863125+0000","last_undegraded":"2024-04-25T09:02:27.863125+0000","last_fullsized":"2024-04-25T09:02:27.863125+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:44.579862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.820694+0000","last_change":"2024-04-25T09:02:26.816690+0000","last_active":"2024-04-25T09:02:27.820694+0000","last_peered":"2024-04-25T09:02:27.820694+0000","last_clean":"2024-04-25T09:02:27.820694+0000","last_became_active":"2024-04-25T09:02:26.816435+0000","last_became_peered":"2024-04-25T09:02:26.816435+0000","last_unstale":"2024-04-25T09:02:27.820694+0000","last_undegraded":"2024-04-25T09:02:27.820694+0000","last_fullsized":"2024-04-25T09:02:27.820694+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:08.630574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.820420+0000","last_change":"2024-04-25T09:02:26.817399+0000","last_active":"2024-04-25T09:02:27.820420+0000","last_peered":"2024-04-25T09:02:27.820420+0000","last_clean":"2024-04-25T09:02:27.820420+0000","last_became_active":"2024-04-25T09:02:26.817135+0000","last_became_peered":"2024-04-25T09:02:26.817135+0000","last_unstale":"2024-04-25T09:02:27.820420+0000","last_undegraded":"2024-04-25T09:02:27.820420+0000","last_fullsized":"2024-04-25T09:02:27.820420+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:24.003416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862054+0000","last_change":"2024-04-25T09:02:26.813767+0000","last_active":"2024-04-25T09:02:27.862054+0000","last_peered":"2024-04-25T09:02:27.862054+0000","last_clean":"2024-04-25T09:02:27.862054+0000","last_became_active":"2024-04-25T09:02:26.813482+0000","last_became_peered":"2024-04-25T09:02:26.813482+0000","last_unstale":"2024-04-25T09:02:27.862054+0000","last_undegraded":"2024-04-25T09:02:27.862054+0000","last_fullsized":"2024-04-25T09:02:27.862054+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:05.912331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862525+0000","last_change":"2024-04-25T09:02:26.819805+0000","last_active":"2024-04-25T09:02:27.862525+0000","last_peered":"2024-04-25T09:02:27.862525+0000","last_clean":"2024-04-25T09:02:27.862525+0000","last_became_active":"2024-04-25T09:02:26.819537+0000","last_became_peered":"2024-04-25T09:02:26.819537+0000","last_unstale":"2024-04-25T09:02:27.862525+0000","last_undegraded":"2024-04-25T09:02:27.862525+0000","last_fullsized":"2024-04-25T09:02:27.862525+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:35.570425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.863483+0000","last_change":"2024-04-25T09:02:26.817876+0000","last_active":"2024-04-25T09:02:27.863483+0000","last_peered":"2024-04-25T09:02:27.863483+0000","last_clean":"2024-04-25T09:02:27.863483+0000","last_became_active":"2024-04-25T09:02:26.817589+0000","last_became_peered":"2024-04-25T09:02:26.817589+0000","last_unstale":"2024-04-25T09:02:27.863483+0000","last_undegraded":"2024-04-25T09:02:27.863483+0000","last_fullsized":"2024-04-25T09:02:27.863483+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:20.819266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"112'4","reported_seq":18,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.819990+0000","last_change":"2024-04-25T09:02:25.797802+0000","last_active":"2024-04-25T09:02:27.819990+0000","last_peered":"2024-04-25T09:02:27.819990+0000","last_clean":"2024-04-25T09:02:27.819990+0000","last_became_active":"2024-04-25T09:02:25.796869+0000","last_became_peered":"2024-04-25T09:02:25.796869+0000","last_unstale":"2024-04-25T09:02:27.819990+0000","last_undegraded":"2024-04-25T09:02:27.819990+0000","last_fullsized":"2024-04-25T09:02:27.819990+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:55.207697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862827+0000","last_change":"2024-04-25T09:02:26.822648+0000","last_active":"2024-04-25T09:02:27.862827+0000","last_peered":"2024-04-25T09:02:27.862827+0000","last_clean":"2024-04-25T09:02:27.862827+0000","last_became_active":"2024-04-25T09:02:26.821791+0000","last_became_peered":"2024-04-25T09:02:26.821791+0000","last_unstale":"2024-04-25T09:02:27.862827+0000","last_undegraded":"2024-04-25T09:02:27.862827+0000","last_fullsized":"2024-04-25T09:02:27.862827+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:08.630574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862663+0000","last_change":"2024-04-25T09:02:25.804229+0000","last_active":"2024-04-25T09:02:27.862663+0000","last_peered":"2024-04-25T09:02:27.862663+0000","last_clean":"2024-04-25T09:02:27.862663+0000","last_became_active":"2024-04-25T09:02:25.803909+0000","last_became_peered":"2024-04-25T09:02:25.803909+0000","last_unstale":"2024-04-25T09:02:27.862663+0000","last_undegraded":"2024-04-25T09:02:27.862663+0000","last_fullsized":"2024-04-25T09:02:27.862663+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:01.303695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.863344+0000","last_change":"2024-04-25T09:02:26.819233+0000","last_active":"2024-04-25T09:02:27.863344+0000","last_peered":"2024-04-25T09:02:27.863344+0000","last_clean":"2024-04-25T09:02:27.863344+0000","last_became_active":"2024-04-25T09:02:26.818895+0000","last_became_peered":"2024-04-25T09:02:26.818895+0000","last_unstale":"2024-04-25T09:02:27.863344+0000","last_undegraded":"2024-04-25T09:02:27.863344+0000","last_fullsized":"2024-04-25T09:02:27.863344+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:24.481351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.819197+0000","last_change":"2024-04-25T09:02:25.803200+0000","last_active":"2024-04-25T09:02:27.819197+0000","last_peered":"2024-04-25T09:02:27.819197+0000","last_clean":"2024-04-25T09:02:27.819197+0000","last_became_active":"2024-04-25T09:02:25.802851+0000","last_became_peered":"2024-04-25T09:02:25.802851+0000","last_unstale":"2024-04-25T09:02:27.819197+0000","last_undegraded":"2024-04-25T09:02:27.819197+0000","last_fullsized":"2024-04-25T09:02:27.819197+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:17.739771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.863158+0000","last_change":"2024-04-25T09:02:26.819015+0000","last_active":"2024-04-25T09:02:27.863158+0000","last_peered":"2024-04-25T09:02:27.863158+0000","last_clean":"2024-04-25T09:02:27.863158+0000","last_became_active":"2024-04-25T09:02:26.818161+0000","last_became_peered":"2024-04-25T09:02:26.818161+0000","last_unstale":"2024-04-25T09:02:27.863158+0000","last_undegraded":"2024-04-25T09:02:27.863158+0000","last_fullsized":"2024-04-25T09:02:27.863158+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:44.579862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.819272+0000","last_change":"2024-04-25T09:02:25.801810+0000","last_active":"2024-04-25T09:02:27.819272+0000","last_peered":"2024-04-25T09:02:27.819272+0000","last_clean":"2024-04-25T09:02:27.819272+0000","last_became_active":"2024-04-25T09:02:25.801499+0000","last_became_peered":"2024-04-25T09:02:25.801499+0000","last_unstale":"2024-04-25T09:02:27.819272+0000","last_undegraded":"2024-04-25T09:02:27.819272+0000","last_fullsized":"2024-04-25T09:02:27.819272+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:04.560283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.861929+0000","last_change":"2024-04-25T09:02:26.823166+0000","last_active":"2024-04-25T09:02:27.861929+0000","last_peered":"2024-04-25T09:02:27.861929+0000","last_clean":"2024-04-25T09:02:27.861929+0000","last_became_active":"2024-04-25T09:02:26.820754+0000","last_became_peered":"2024-04-25T09:02:26.820754+0000","last_unstale":"2024-04-25T09:02:27.861929+0000","last_undegraded":"2024-04-25T09:02:27.861929+0000","last_fullsized":"2024-04-25T09:02:27.861929+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:42.999643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.819824+0000","last_change":"2024-04-25T09:02:25.802590+0000","last_active":"2024-04-25T09:02:27.819824+0000","last_peered":"2024-04-25T09:02:27.819824+0000","last_clean":"2024-04-25T09:02:27.819824+0000","last_became_active":"2024-04-25T09:02:25.802311+0000","last_became_peered":"2024-04-25T09:02:25.802311+0000","last_unstale":"2024-04-25T09:02:27.819824+0000","last_undegraded":"2024-04-25T09:02:27.819824+0000","last_fullsized":"2024-04-25T09:02:27.819824+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:07.755858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.820673+0000","last_change":"2024-04-25T09:02:26.813278+0000","last_active":"2024-04-25T09:02:27.820673+0000","last_peered":"2024-04-25T09:02:27.820673+0000","last_clean":"2024-04-25T09:02:27.820673+0000","last_became_active":"2024-04-25T09:02:26.813010+0000","last_became_peered":"2024-04-25T09:02:26.813010+0000","last_unstale":"2024-04-25T09:02:27.820673+0000","last_undegraded":"2024-04-25T09:02:27.820673+0000","last_fullsized":"2024-04-25T09:02:27.820673+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:58.133772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862952+0000","last_change":"2024-04-25T09:02:25.805830+0000","last_active":"2024-04-25T09:02:27.862952+0000","last_peered":"2024-04-25T09:02:27.862952+0000","last_clean":"2024-04-25T09:02:27.862952+0000","last_became_active":"2024-04-25T09:02:25.805475+0000","last_became_peered":"2024-04-25T09:02:25.805475+0000","last_unstale":"2024-04-25T09:02:27.862952+0000","last_undegraded":"2024-04-25T09:02:27.862952+0000","last_fullsized":"2024-04-25T09:02:27.862952+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:01.303695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.861956+0000","last_change":"2024-04-25T09:02:26.820034+0000","last_active":"2024-04-25T09:02:27.861956+0000","last_peered":"2024-04-25T09:02:27.861956+0000","last_clean":"2024-04-25T09:02:27.861956+0000","last_became_active":"2024-04-25T09:02:26.813177+0000","last_became_peered":"2024-04-25T09:02:26.813177+0000","last_unstale":"2024-04-25T09:02:27.861956+0000","last_undegraded":"2024-04-25T09:02:27.861956+0000","last_fullsized":"2024-04-25T09:02:27.861956+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:27.696682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.819916+0000","last_change":"2024-04-25T09:02:25.797181+0000","last_active":"2024-04-25T09:02:27.819916+0000","last_peered":"2024-04-25T09:02:27.819916+0000","last_clean":"2024-04-25T09:02:27.819916+0000","last_became_active":"2024-04-25T09:02:25.796880+0000","last_became_peered":"2024-04-25T09:02:25.796880+0000","last_unstale":"2024-04-25T09:02:27.819916+0000","last_undegraded":"2024-04-25T09:02:27.819916+0000","last_fullsized":"2024-04-25T09:02:27.819916+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:13.158075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.819537+0000","last_change":"2024-04-25T09:02:26.811990+0000","last_active":"2024-04-25T09:02:27.819537+0000","last_peered":"2024-04-25T09:02:27.819537+0000","last_clean":"2024-04-25T09:02:27.819537+0000","last_became_active":"2024-04-25T09:02:26.811702+0000","last_became_peered":"2024-04-25T09:02:26.811702+0000","last_unstale":"2024-04-25T09:02:27.819537+0000","last_undegraded":"2024-04-25T09:02:27.819537+0000","last_fullsized":"2024-04-25T09:02:27.819537+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:15.687634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862070+0000","last_change":"2024-04-25T09:02:25.812319+0000","last_active":"2024-04-25T09:02:27.862070+0000","last_peered":"2024-04-25T09:02:27.862070+0000","last_clean":"2024-04-25T09:02:27.862070+0000","last_became_active":"2024-04-25T09:02:25.811982+0000","last_became_peered":"2024-04-25T09:02:25.811982+0000","last_unstale":"2024-04-25T09:02:27.862070+0000","last_undegraded":"2024-04-25T09:02:27.862070+0000","last_fullsized":"2024-04-25T09:02:27.862070+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:52.690740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.820257+0000","last_change":"2024-04-25T09:02:26.811316+0000","last_active":"2024-04-25T09:02:27.820257+0000","last_peered":"2024-04-25T09:02:27.820257+0000","last_clean":"2024-04-25T09:02:27.820257+0000","last_became_active":"2024-04-25T09:02:26.810973+0000","last_became_peered":"2024-04-25T09:02:26.810973+0000","last_unstale":"2024-04-25T09:02:27.820257+0000","last_undegraded":"2024-04-25T09:02:27.820257+0000","last_fullsized":"2024-04-25T09:02:27.820257+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:35.570425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.819139+0000","last_change":"2024-04-25T09:02:25.804273+0000","last_active":"2024-04-25T09:02:27.819139+0000","last_peered":"2024-04-25T09:02:27.819139+0000","last_clean":"2024-04-25T09:02:27.819139+0000","last_became_active":"2024-04-25T09:02:25.803994+0000","last_became_peered":"2024-04-25T09:02:25.803994+0000","last_unstale":"2024-04-25T09:02:27.819139+0000","last_undegraded":"2024-04-25T09:02:27.819139+0000","last_fullsized":"2024-04-25T09:02:27.819139+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:19.214564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862999+0000","last_change":"2024-04-25T09:02:26.816903+0000","last_active":"2024-04-25T09:02:27.862999+0000","last_peered":"2024-04-25T09:02:27.862999+0000","last_clean":"2024-04-25T09:02:27.862999+0000","last_became_active":"2024-04-25T09:02:26.816364+0000","last_became_peered":"2024-04-25T09:02:26.816364+0000","last_unstale":"2024-04-25T09:02:27.862999+0000","last_undegraded":"2024-04-25T09:02:27.862999+0000","last_fullsized":"2024-04-25T09:02:27.862999+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:48.848804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862206+0000","last_change":"2024-04-25T09:02:25.804432+0000","last_active":"2024-04-25T09:02:27.862206+0000","last_peered":"2024-04-25T09:02:27.862206+0000","last_clean":"2024-04-25T09:02:27.862206+0000","last_became_active":"2024-04-25T09:02:25.804141+0000","last_became_peered":"2024-04-25T09:02:25.804141+0000","last_unstale":"2024-04-25T09:02:27.862206+0000","last_undegraded":"2024-04-25T09:02:27.862206+0000","last_fullsized":"2024-04-25T09:02:27.862206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:47.844623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.863135+0000","last_change":"2024-04-25T09:02:26.806921+0000","last_active":"2024-04-25T09:02:27.863135+0000","last_peered":"2024-04-25T09:02:27.863135+0000","last_clean":"2024-04-25T09:02:27.863135+0000","last_became_active":"2024-04-25T09:02:26.806652+0000","last_became_peered":"2024-04-25T09:02:26.806652+0000","last_unstale":"2024-04-25T09:02:27.863135+0000","last_undegraded":"2024-04-25T09:02:27.863135+0000","last_fullsized":"2024-04-25T09:02:27.863135+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:10.215927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"112'3","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862227+0000","last_change":"2024-04-25T09:02:25.804867+0000","last_active":"2024-04-25T09:02:27.862227+0000","last_peered":"2024-04-25T09:02:27.862227+0000","last_clean":"2024-04-25T09:02:27.862227+0000","last_became_active":"2024-04-25T09:02:25.804595+0000","last_became_peered":"2024-04-25T09:02:25.804595+0000","last_unstale":"2024-04-25T09:02:27.862227+0000","last_undegraded":"2024-04-25T09:02:27.862227+0000","last_fullsized":"2024-04-25T09:02:27.862227+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:46.918975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.820349+0000","last_change":"2024-04-25T09:02:26.815204+0000","last_active":"2024-04-25T09:02:27.820349+0000","last_peered":"2024-04-25T09:02:27.820349+0000","last_clean":"2024-04-25T09:02:27.820349+0000","last_became_active":"2024-04-25T09:02:26.814914+0000","last_became_peered":"2024-04-25T09:02:26.814914+0000","last_unstale":"2024-04-25T09:02:27.820349+0000","last_undegraded":"2024-04-25T09:02:27.820349+0000","last_fullsized":"2024-04-25T09:02:27.820349+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:02.253566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"112'6","reported_seq":20,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862562+0000","last_change":"2024-04-25T09:02:25.811971+0000","last_active":"2024-04-25T09:02:27.862562+0000","last_peered":"2024-04-25T09:02:27.862562+0000","last_clean":"2024-04-25T09:02:27.862562+0000","last_became_active":"2024-04-25T09:02:25.810936+0000","last_became_peered":"2024-04-25T09:02:25.810936+0000","last_unstale":"2024-04-25T09:02:27.862562+0000","last_undegraded":"2024-04-25T09:02:27.862562+0000","last_fullsized":"2024-04-25T09:02:27.862562+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:40.591769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862865+0000","last_change":"2024-04-25T09:02:26.809666+0000","last_active":"2024-04-25T09:02:27.862865+0000","last_peered":"2024-04-25T09:02:27.862865+0000","last_clean":"2024-04-25T09:02:27.862865+0000","last_became_active":"2024-04-25T09:02:26.809363+0000","last_became_peered":"2024-04-25T09:02:26.809363+0000","last_unstale":"2024-04-25T09:02:27.862865+0000","last_undegraded":"2024-04-25T09:02:27.862865+0000","last_fullsized":"2024-04-25T09:02:27.862865+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:13.782020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862958+0000","last_change":"2024-04-25T09:02:25.800185+0000","last_active":"2024-04-25T09:02:27.862958+0000","last_peered":"2024-04-25T09:02:27.862958+0000","last_clean":"2024-04-25T09:02:27.862958+0000","last_became_active":"2024-04-25T09:02:25.799855+0000","last_became_peered":"2024-04-25T09:02:25.799855+0000","last_unstale":"2024-04-25T09:02:27.862958+0000","last_undegraded":"2024-04-25T09:02:27.862958+0000","last_fullsized":"2024-04-25T09:02:27.862958+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:07.755858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862844+0000","last_change":"2024-04-25T09:02:26.817586+0000","last_active":"2024-04-25T09:02:27.862844+0000","last_peered":"2024-04-25T09:02:27.862844+0000","last_clean":"2024-04-25T09:02:27.862844+0000","last_became_active":"2024-04-25T09:02:26.816997+0000","last_became_peered":"2024-04-25T09:02:26.816997+0000","last_unstale":"2024-04-25T09:02:27.862844+0000","last_undegraded":"2024-04-25T09:02:27.862844+0000","last_fullsized":"2024-04-25T09:02:27.862844+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:49.545678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862469+0000","last_change":"2024-04-25T09:02:25.811195+0000","last_active":"2024-04-25T09:02:27.862469+0000","last_peered":"2024-04-25T09:02:27.862469+0000","last_clean":"2024-04-25T09:02:27.862469+0000","last_became_active":"2024-04-25T09:02:25.810925+0000","last_became_peered":"2024-04-25T09:02:25.810925+0000","last_unstale":"2024-04-25T09:02:27.862469+0000","last_undegraded":"2024-04-25T09:02:27.862469+0000","last_fullsized":"2024-04-25T09:02:27.862469+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:23.477170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862701+0000","last_change":"2024-04-25T09:02:26.815136+0000","last_active":"2024-04-25T09:02:27.862701+0000","last_peered":"2024-04-25T09:02:27.862701+0000","last_clean":"2024-04-25T09:02:27.862701+0000","last_became_active":"2024-04-25T09:02:26.814838+0000","last_became_peered":"2024-04-25T09:02:26.814838+0000","last_unstale":"2024-04-25T09:02:27.862701+0000","last_undegraded":"2024-04-25T09:02:27.862701+0000","last_fullsized":"2024-04-25T09:02:27.862701+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:19.763453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"112'8","reported_seq":22,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.818962+0000","last_change":"2024-04-25T09:02:25.811455+0000","last_active":"2024-04-25T09:02:27.818962+0000","last_peered":"2024-04-25T09:02:27.818962+0000","last_clean":"2024-04-25T09:02:27.818962+0000","last_became_active":"2024-04-25T09:02:25.811175+0000","last_became_peered":"2024-04-25T09:02:25.811175+0000","last_unstale":"2024-04-25T09:02:27.818962+0000","last_undegraded":"2024-04-25T09:02:27.818962+0000","last_fullsized":"2024-04-25T09:02:27.818962+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:28.955003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.819480+0000","last_change":"2024-04-25T09:02:26.819783+0000","last_active":"2024-04-25T09:02:27.819480+0000","last_peered":"2024-04-25T09:02:27.819480+0000","last_clean":"2024-04-25T09:02:27.819480+0000","last_became_active":"2024-04-25T09:02:26.819285+0000","last_became_peered":"2024-04-25T09:02:26.819285+0000","last_unstale":"2024-04-25T09:02:27.819480+0000","last_undegraded":"2024-04-25T09:02:27.819480+0000","last_fullsized":"2024-04-25T09:02:27.819480+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:08.194100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.863966+0000","last_change":"2024-04-25T09:02:25.809787+0000","last_active":"2024-04-25T09:02:27.863966+0000","last_peered":"2024-04-25T09:02:27.863966+0000","last_clean":"2024-04-25T09:02:27.863966+0000","last_became_active":"2024-04-25T09:02:25.809078+0000","last_became_peered":"2024-04-25T09:02:25.809078+0000","last_unstale":"2024-04-25T09:02:27.863966+0000","last_undegraded":"2024-04-25T09:02:27.863966+0000","last_fullsized":"2024-04-25T09:02:27.863966+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:49:24.331148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.863105+0000","last_change":"2024-04-25T09:02:26.825975+0000","last_active":"2024-04-25T09:02:27.863105+0000","last_peered":"2024-04-25T09:02:27.863105+0000","last_clean":"2024-04-25T09:02:27.863105+0000","last_became_active":"2024-04-25T09:02:26.825704+0000","last_became_peered":"2024-04-25T09:02:26.825704+0000","last_unstale":"2024-04-25T09:02:27.863105+0000","last_undegraded":"2024-04-25T09:02:27.863105+0000","last_fullsized":"2024-04-25T09:02:27.863105+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:24.003416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.819517+0000","last_change":"2024-04-25T09:02:25.798505+0000","last_active":"2024-04-25T09:02:27.819517+0000","last_peered":"2024-04-25T09:02:27.819517+0000","last_clean":"2024-04-25T09:02:27.819517+0000","last_became_active":"2024-04-25T09:02:25.798189+0000","last_became_peered":"2024-04-25T09:02:25.798189+0000","last_unstale":"2024-04-25T09:02:27.819517+0000","last_undegraded":"2024-04-25T09:02:27.819517+0000","last_fullsized":"2024-04-25T09:02:27.819517+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:26.870309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862799+0000","last_change":"2024-04-25T09:02:26.826090+0000","last_active":"2024-04-25T09:02:27.862799+0000","last_peered":"2024-04-25T09:02:27.862799+0000","last_clean":"2024-04-25T09:02:27.862799+0000","last_became_active":"2024-04-25T09:02:26.825863+0000","last_became_peered":"2024-04-25T09:02:26.825863+0000","last_unstale":"2024-04-25T09:02:27.862799+0000","last_undegraded":"2024-04-25T09:02:27.862799+0000","last_fullsized":"2024-04-25T09:02:27.862799+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:19.763453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862592+0000","last_change":"2024-04-25T09:02:25.803652+0000","last_active":"2024-04-25T09:02:27.862592+0000","last_peered":"2024-04-25T09:02:27.862592+0000","last_clean":"2024-04-25T09:02:27.862592+0000","last_became_active":"2024-04-25T09:02:25.803266+0000","last_became_peered":"2024-04-25T09:02:25.803266+0000","last_unstale":"2024-04-25T09:02:27.862592+0000","last_undegraded":"2024-04-25T09:02:27.862592+0000","last_fullsized":"2024-04-25T09:02:27.862592+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:18.799446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.863239+0000","last_change":"2024-04-25T09:02:26.822196+0000","last_active":"2024-04-25T09:02:27.863239+0000","last_peered":"2024-04-25T09:02:27.863239+0000","last_clean":"2024-04-25T09:02:27.863239+0000","last_became_active":"2024-04-25T09:02:26.820488+0000","last_became_peered":"2024-04-25T09:02:26.820488+0000","last_unstale":"2024-04-25T09:02:27.863239+0000","last_undegraded":"2024-04-25T09:02:27.863239+0000","last_fullsized":"2024-04-25T09:02:27.863239+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:58.811244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.819832+0000","last_change":"2024-04-25T09:02:25.807235+0000","last_active":"2024-04-25T09:02:27.819832+0000","last_peered":"2024-04-25T09:02:27.819832+0000","last_clean":"2024-04-25T09:02:27.819832+0000","last_became_active":"2024-04-25T09:02:25.806851+0000","last_became_peered":"2024-04-25T09:02:25.806851+0000","last_unstale":"2024-04-25T09:02:27.819832+0000","last_undegraded":"2024-04-25T09:02:27.819832+0000","last_fullsized":"2024-04-25T09:02:27.819832+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:31.793700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.863076+0000","last_change":"2024-04-25T09:02:26.818331+0000","last_active":"2024-04-25T09:02:27.863076+0000","last_peered":"2024-04-25T09:02:27.863076+0000","last_clean":"2024-04-25T09:02:27.863076+0000","last_became_active":"2024-04-25T09:02:26.818047+0000","last_became_peered":"2024-04-25T09:02:26.818047+0000","last_unstale":"2024-04-25T09:02:27.863076+0000","last_undegraded":"2024-04-25T09:02:27.863076+0000","last_fullsized":"2024-04-25T09:02:27.863076+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:50.402637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.863097+0000","last_change":"2024-04-25T09:02:25.811062+0000","last_active":"2024-04-25T09:02:27.863097+0000","last_peered":"2024-04-25T09:02:27.863097+0000","last_clean":"2024-04-25T09:02:27.863097+0000","last_became_active":"2024-04-25T09:02:25.810776+0000","last_became_peered":"2024-04-25T09:02:25.810776+0000","last_unstale":"2024-04-25T09:02:27.863097+0000","last_undegraded":"2024-04-25T09:02:27.863097+0000","last_fullsized":"2024-04-25T09:02:27.863097+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:07.189919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862262+0000","last_change":"2024-04-25T09:02:26.821200+0000","last_active":"2024-04-25T09:02:27.862262+0000","last_peered":"2024-04-25T09:02:27.862262+0000","last_clean":"2024-04-25T09:02:27.862262+0000","last_became_active":"2024-04-25T09:02:26.818797+0000","last_became_peered":"2024-04-25T09:02:26.818797+0000","last_unstale":"2024-04-25T09:02:27.862262+0000","last_undegraded":"2024-04-25T09:02:27.862262+0000","last_fullsized":"2024-04-25T09:02:27.862262+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:50.760405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.819038+0000","last_change":"2024-04-25T09:02:25.805631+0000","last_active":"2024-04-25T09:02:27.819038+0000","last_peered":"2024-04-25T09:02:27.819038+0000","last_clean":"2024-04-25T09:02:27.819038+0000","last_became_active":"2024-04-25T09:02:25.805348+0000","last_became_peered":"2024-04-25T09:02:25.805348+0000","last_unstale":"2024-04-25T09:02:27.819038+0000","last_undegraded":"2024-04-25T09:02:27.819038+0000","last_fullsized":"2024-04-25T09:02:27.819038+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:45.625953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862494+0000","last_change":"2024-04-25T09:02:26.815708+0000","last_active":"2024-04-25T09:02:27.862494+0000","last_peered":"2024-04-25T09:02:27.862494+0000","last_clean":"2024-04-25T09:02:27.862494+0000","last_became_active":"2024-04-25T09:02:26.815428+0000","last_became_peered":"2024-04-25T09:02:26.815428+0000","last_unstale":"2024-04-25T09:02:27.862494+0000","last_undegraded":"2024-04-25T09:02:27.862494+0000","last_fullsized":"2024-04-25T09:02:27.862494+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:51.327363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862712+0000","last_change":"2024-04-25T09:02:25.800620+0000","last_active":"2024-04-25T09:02:27.862712+0000","last_peered":"2024-04-25T09:02:27.862712+0000","last_clean":"2024-04-25T09:02:27.862712+0000","last_became_active":"2024-04-25T09:02:25.800346+0000","last_became_peered":"2024-04-25T09:02:25.800346+0000","last_unstale":"2024-04-25T09:02:27.862712+0000","last_undegraded":"2024-04-25T09:02:27.862712+0000","last_fullsized":"2024-04-25T09:02:27.862712+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:55.207697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862865+0000","last_change":"2024-04-25T09:02:26.820676+0000","last_active":"2024-04-25T09:02:27.862865+0000","last_peered":"2024-04-25T09:02:27.862865+0000","last_clean":"2024-04-25T09:02:27.862865+0000","last_became_active":"2024-04-25T09:02:26.816953+0000","last_became_peered":"2024-04-25T09:02:26.816953+0000","last_unstale":"2024-04-25T09:02:27.862865+0000","last_undegraded":"2024-04-25T09:02:27.862865+0000","last_fullsized":"2024-04-25T09:02:27.862865+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:50.402637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862166+0000","last_change":"2024-04-25T09:02:25.810064+0000","last_active":"2024-04-25T09:02:27.862166+0000","last_peered":"2024-04-25T09:02:27.862166+0000","last_clean":"2024-04-25T09:02:27.862166+0000","last_became_active":"2024-04-25T09:02:25.808573+0000","last_became_peered":"2024-04-25T09:02:25.808573+0000","last_unstale":"2024-04-25T09:02:27.862166+0000","last_undegraded":"2024-04-25T09:02:27.862166+0000","last_fullsized":"2024-04-25T09:02:27.862166+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:39.116454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862622+0000","last_change":"2024-04-25T09:02:26.820044+0000","last_active":"2024-04-25T09:02:27.862622+0000","last_peered":"2024-04-25T09:02:27.862622+0000","last_clean":"2024-04-25T09:02:27.862622+0000","last_became_active":"2024-04-25T09:02:26.819748+0000","last_became_peered":"2024-04-25T09:02:26.819748+0000","last_unstale":"2024-04-25T09:02:27.862622+0000","last_undegraded":"2024-04-25T09:02:27.862622+0000","last_fullsized":"2024-04-25T09:02:27.862622+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:58.133772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862514+0000","last_change":"2024-04-25T09:02:25.809249+0000","last_active":"2024-04-25T09:02:27.862514+0000","last_peered":"2024-04-25T09:02:27.862514+0000","last_clean":"2024-04-25T09:02:27.862514+0000","last_became_active":"2024-04-25T09:02:25.808906+0000","last_became_peered":"2024-04-25T09:02:25.808906+0000","last_unstale":"2024-04-25T09:02:27.862514+0000","last_undegraded":"2024-04-25T09:02:27.862514+0000","last_fullsized":"2024-04-25T09:02:27.862514+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:08.689656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.863855+0000","last_change":"2024-04-25T09:02:26.823140+0000","last_active":"2024-04-25T09:02:27.863855+0000","last_peered":"2024-04-25T09:02:27.863855+0000","last_clean":"2024-04-25T09:02:27.863855+0000","last_became_active":"2024-04-25T09:02:26.822572+0000","last_became_peered":"2024-04-25T09:02:26.822572+0000","last_unstale":"2024-04-25T09:02:27.863855+0000","last_undegraded":"2024-04-25T09:02:27.863855+0000","last_fullsized":"2024-04-25T09:02:27.863855+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:51.915253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862796+0000","last_change":"2024-04-25T09:02:25.804432+0000","last_active":"2024-04-25T09:02:27.862796+0000","last_peered":"2024-04-25T09:02:27.862796+0000","last_clean":"2024-04-25T09:02:27.862796+0000","last_became_active":"2024-04-25T09:02:25.804129+0000","last_became_peered":"2024-04-25T09:02:25.804129+0000","last_unstale":"2024-04-25T09:02:27.862796+0000","last_undegraded":"2024-04-25T09:02:27.862796+0000","last_fullsized":"2024-04-25T09:02:27.862796+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:19.815085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862575+0000","last_change":"2024-04-25T09:02:26.817576+0000","last_active":"2024-04-25T09:02:27.862575+0000","last_peered":"2024-04-25T09:02:27.862575+0000","last_clean":"2024-04-25T09:02:27.862575+0000","last_became_active":"2024-04-25T09:02:26.815719+0000","last_became_peered":"2024-04-25T09:02:26.815719+0000","last_unstale":"2024-04-25T09:02:27.862575+0000","last_undegraded":"2024-04-25T09:02:27.862575+0000","last_fullsized":"2024-04-25T09:02:27.862575+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:34.669182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862345+0000","last_change":"2024-04-25T09:02:25.807225+0000","last_active":"2024-04-25T09:02:27.862345+0000","last_peered":"2024-04-25T09:02:27.862345+0000","last_clean":"2024-04-25T09:02:27.862345+0000","last_became_active":"2024-04-25T09:02:25.806697+0000","last_became_peered":"2024-04-25T09:02:25.806697+0000","last_unstale":"2024-04-25T09:02:27.862345+0000","last_undegraded":"2024-04-25T09:02:27.862345+0000","last_fullsized":"2024-04-25T09:02:27.862345+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:33.148270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862410+0000","last_change":"2024-04-25T09:02:26.811933+0000","last_active":"2024-04-25T09:02:27.862410+0000","last_peered":"2024-04-25T09:02:27.862410+0000","last_clean":"2024-04-25T09:02:27.862410+0000","last_became_active":"2024-04-25T09:02:26.811455+0000","last_became_peered":"2024-04-25T09:02:26.811455+0000","last_unstale":"2024-04-25T09:02:27.862410+0000","last_undegraded":"2024-04-25T09:02:27.862410+0000","last_fullsized":"2024-04-25T09:02:27.862410+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:54.415160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.863227+0000","last_change":"2024-04-25T09:02:25.800897+0000","last_active":"2024-04-25T09:02:27.863227+0000","last_peered":"2024-04-25T09:02:27.863227+0000","last_clean":"2024-04-25T09:02:27.863227+0000","last_became_active":"2024-04-25T09:02:25.800622+0000","last_became_peered":"2024-04-25T09:02:25.800622+0000","last_unstale":"2024-04-25T09:02:27.863227+0000","last_undegraded":"2024-04-25T09:02:27.863227+0000","last_fullsized":"2024-04-25T09:02:27.863227+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:09.715801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.820024+0000","last_change":"2024-04-25T09:02:26.805898+0000","last_active":"2024-04-25T09:02:27.820024+0000","last_peered":"2024-04-25T09:02:27.820024+0000","last_clean":"2024-04-25T09:02:27.820024+0000","last_became_active":"2024-04-25T09:02:26.805439+0000","last_became_peered":"2024-04-25T09:02:26.805439+0000","last_unstale":"2024-04-25T09:02:27.820024+0000","last_undegraded":"2024-04-25T09:02:27.820024+0000","last_fullsized":"2024-04-25T09:02:27.820024+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:15.671723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.819947+0000","last_change":"2024-04-25T09:02:25.799689+0000","last_active":"2024-04-25T09:02:27.819947+0000","last_peered":"2024-04-25T09:02:27.819947+0000","last_clean":"2024-04-25T09:02:27.819947+0000","last_became_active":"2024-04-25T09:02:25.799414+0000","last_became_peered":"2024-04-25T09:02:25.799414+0000","last_unstale":"2024-04-25T09:02:27.819947+0000","last_undegraded":"2024-04-25T09:02:27.819947+0000","last_fullsized":"2024-04-25T09:02:27.819947+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:07.189919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.820798+0000","last_change":"2024-04-25T09:02:26.814226+0000","last_active":"2024-04-25T09:02:27.820798+0000","last_peered":"2024-04-25T09:02:27.820798+0000","last_clean":"2024-04-25T09:02:27.820798+0000","last_became_active":"2024-04-25T09:02:26.813732+0000","last_became_peered":"2024-04-25T09:02:26.813732+0000","last_unstale":"2024-04-25T09:02:27.820798+0000","last_undegraded":"2024-04-25T09:02:27.820798+0000","last_fullsized":"2024-04-25T09:02:27.820798+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:10.719983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.820586+0000","last_change":"2024-04-25T09:02:25.801797+0000","last_active":"2024-04-25T09:02:27.820586+0000","last_peered":"2024-04-25T09:02:27.820586+0000","last_clean":"2024-04-25T09:02:27.820586+0000","last_became_active":"2024-04-25T09:02:25.801516+0000","last_became_peered":"2024-04-25T09:02:25.801516+0000","last_unstale":"2024-04-25T09:02:27.820586+0000","last_undegraded":"2024-04-25T09:02:27.820586+0000","last_fullsized":"2024-04-25T09:02:27.820586+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:53.163079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862970+0000","last_change":"2024-04-25T09:02:26.817550+0000","last_active":"2024-04-25T09:02:27.862970+0000","last_peered":"2024-04-25T09:02:27.862970+0000","last_clean":"2024-04-25T09:02:27.862970+0000","last_became_active":"2024-04-25T09:02:26.817174+0000","last_became_peered":"2024-04-25T09:02:26.817174+0000","last_unstale":"2024-04-25T09:02:27.862970+0000","last_undegraded":"2024-04-25T09:02:27.862970+0000","last_fullsized":"2024-04-25T09:02:27.862970+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:49.545678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.819044+0000","last_change":"2024-04-25T09:02:25.802637+0000","last_active":"2024-04-25T09:02:27.819044+0000","last_peered":"2024-04-25T09:02:27.819044+0000","last_clean":"2024-04-25T09:02:27.819044+0000","last_became_active":"2024-04-25T09:02:25.802294+0000","last_became_peered":"2024-04-25T09:02:25.802294+0000","last_unstale":"2024-04-25T09:02:27.819044+0000","last_undegraded":"2024-04-25T09:02:27.819044+0000","last_fullsized":"2024-04-25T09:02:27.819044+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:22.377195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862420+0000","last_change":"2024-04-25T09:02:26.814711+0000","last_active":"2024-04-25T09:02:27.862420+0000","last_peered":"2024-04-25T09:02:27.862420+0000","last_clean":"2024-04-25T09:02:27.862420+0000","last_became_active":"2024-04-25T09:02:26.813816+0000","last_became_peered":"2024-04-25T09:02:26.813816+0000","last_unstale":"2024-04-25T09:02:27.862420+0000","last_undegraded":"2024-04-25T09:02:27.862420+0000","last_fullsized":"2024-04-25T09:02:27.862420+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:42.435105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862264+0000","last_change":"2024-04-25T09:02:25.808730+0000","last_active":"2024-04-25T09:02:27.862264+0000","last_peered":"2024-04-25T09:02:27.862264+0000","last_clean":"2024-04-25T09:02:27.862264+0000","last_became_active":"2024-04-25T09:02:25.808443+0000","last_became_peered":"2024-04-25T09:02:25.808443+0000","last_unstale":"2024-04-25T09:02:27.862264+0000","last_undegraded":"2024-04-25T09:02:27.862264+0000","last_fullsized":"2024-04-25T09:02:27.862264+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:50.725493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862794+0000","last_change":"2024-04-25T09:02:26.817779+0000","last_active":"2024-04-25T09:02:27.862794+0000","last_peered":"2024-04-25T09:02:27.862794+0000","last_clean":"2024-04-25T09:02:27.862794+0000","last_became_active":"2024-04-25T09:02:26.817498+0000","last_became_peered":"2024-04-25T09:02:26.817498+0000","last_unstale":"2024-04-25T09:02:27.862794+0000","last_undegraded":"2024-04-25T09:02:27.862794+0000","last_fullsized":"2024-04-25T09:02:27.862794+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:51.706367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.819742+0000","last_change":"2024-04-25T09:02:25.807278+0000","last_active":"2024-04-25T09:02:27.819742+0000","last_peered":"2024-04-25T09:02:27.819742+0000","last_clean":"2024-04-25T09:02:27.819742+0000","last_became_active":"2024-04-25T09:02:25.806915+0000","last_became_peered":"2024-04-25T09:02:25.806915+0000","last_unstale":"2024-04-25T09:02:27.819742+0000","last_undegraded":"2024-04-25T09:02:27.819742+0000","last_fullsized":"2024-04-25T09:02:27.819742+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:19.009115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":341,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862898+0000","last_change":"2024-04-25T08:54:00.818044+0000","last_active":"2024-04-25T09:02:27.862898+0000","last_peered":"2024-04-25T09:02:27.862898+0000","last_clean":"2024-04-25T09:02:27.862898+0000","last_became_active":"2024-04-25T08:54:00.817098+0000","last_became_peered":"2024-04-25T08:54:00.817098+0000","last_unstale":"2024-04-25T09:02:27.862898+0000","last_undegraded":"2024-04-25T09:02:27.862898+0000","last_fullsized":"2024-04-25T09:02:27.862898+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_clean_scrub_stamp":"2024-04-25T08:53:59.739677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:48.129513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574952,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59516,"kb_used_data":1960,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312324,"statfs":{"total":96636764160,"available":96575819776,"internally_reserved":0,"allocated":2007040,"data_stored":1201516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574953,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64500,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94307340,"statfs":{"total":96636764160,"available":96570716160,"internally_reserved":0,"allocated":1409024,"data_stored":610602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574955,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50608,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321232,"statfs":{"total":96636764160,"available":96584941568,"internally_reserved":0,"allocated":1413120,"data_stored":611136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574953,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64496,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94307344,"statfs":{"total":96636764160,"available":96570720256,"internally_reserved":0,"allocated":1404928,"data_stored":610532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574953,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72828,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94299012,"statfs":{"total":96636764160,"available":96562188288,"internally_reserved":0,"allocated":1421312,"data_stored":612214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574953,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58920,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312920,"statfs":{"total":96636764160,"available":96576430080,"internally_reserved":0,"allocated":1404928,"data_stored":610532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76976,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294864,"statfs":{"total":96636764160,"available":96557940736,"internally_reserved":0,"allocated":1413120,"data_stored":610656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574954,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47044,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324796,"statfs":{"total":96636764160,"available":96588591104,"internally_reserved":0,"allocated":2019328,"data_stored":1202624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:02:30.811 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:02:30.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.958+0000 7f1f5369e640 1 Processor -- start 2024-04-25T09:02:30.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.958+0000 7f1f5369e640 1 -- start start 2024-04-25T09:02:30.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.959+0000 7f1f5369e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1f4c15ef50 0x7f1f4c0d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:30.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.959+0000 7f1f5369e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f4c0d0d30 con 0x7f1f4c15dce0 2024-04-25T09:02:30.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.959+0000 7f1f5369e640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f4c0d2920 con 0x7f1f4c0d0920 2024-04-25T09:02:30.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.959+0000 7f1f5369e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f4c0cec60 con 0x7f1f4c15ef50 2024-04-25T09:02:30.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.959+0000 7f1f50c39640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1f4c15ef50 0x7f1f4c0d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:30.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.959+0000 7f1f5143a640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f1f4c15dce0 0x7f1f4c0ce550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48384/0 (socket says 172.21.15.43:48384) 2024-04-25T09:02:30.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.959+0000 7f1f5143a640 1 -- 172.21.15.43:0/28254766 learned_addr learned my addr 172.21.15.43:0/28254766 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:30.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.959+0000 7f1f50c39640 1 -- 172.21.15.43:0/28254766 >> v1:172.21.15.193:6789/0 conn(0x7f1f4c0d0920 legacy=0x7f1f4c160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:30.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.959+0000 7f1f50c39640 1 -- 172.21.15.43:0/28254766 >> v1:172.21.15.43:6789/0 conn(0x7f1f4c15dce0 legacy=0x7f1f4c0ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:30.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.959+0000 7f1f50c39640 1 -- 172.21.15.43:0/28254766 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f4c0cee80 con 0x7f1f4c15ef50 2024-04-25T09:02:30.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.959+0000 7f1f43fff640 1 -- 172.21.15.43:0/28254766 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2696891014 0 0) 0x7f1f48001110 con 0x7f1f4c15dce0 2024-04-25T09:02:30.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.960+0000 7f1f50c39640 1 --2- 172.21.15.43:0/28254766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f4c15ef50 0x7f1f4c0d03e0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c0097a0 tx=0x7f1f3c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48ae1974995b4a53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:30.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.960+0000 7f1f43fff640 1 -- 172.21.15.43:0/28254766 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c037070 con 0x7f1f4c15ef50 2024-04-25T09:02:30.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.960+0000 7f1f43fff640 1 -- 172.21.15.43:0/28254766 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f3c02ae50 con 0x7f1f4c15ef50 2024-04-25T09:02:30.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.960+0000 7f1f43fff640 1 -- 172.21.15.43:0/28254766 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c032660 con 0x7f1f4c15ef50 2024-04-25T09:02:30.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.961+0000 7f1f5369e640 1 -- 172.21.15.43:0/28254766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f4c15ef50 msgr2=0x7f1f4c0d03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:30.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.961+0000 7f1f5369e640 1 --2- 172.21.15.43:0/28254766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f4c15ef50 0x7f1f4c0d03e0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c0097a0 tx=0x7f1f3c02a140 comp rx=0 tx=0).stop 2024-04-25T09:02:30.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.961+0000 7f1f5369e640 1 -- 172.21.15.43:0/28254766 shutdown_connections 2024-04-25T09:02:30.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.961+0000 7f1f5369e640 1 --2- 172.21.15.43:0/28254766 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f4c15ef50 0x7f1f4c0d03e0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:30.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.961+0000 7f1f5369e640 1 -- 172.21.15.43:0/28254766 >> 172.21.15.43:0/28254766 conn(0x7f1f4c09cd00 msgr2=0x7f1f4c0d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:30.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.961+0000 7f1f5369e640 1 -- 172.21.15.43:0/28254766 shutdown_connections 2024-04-25T09:02:30.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.961+0000 7f1f5369e640 1 -- 172.21.15.43:0/28254766 wait complete. 2024-04-25T09:02:30.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.962+0000 7f1f5369e640 1 Processor -- start 2024-04-25T09:02:30.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.962+0000 7f1f5369e640 1 -- start start 2024-04-25T09:02:30.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.963+0000 7f1f5369e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1f4c0d0920 0x7f1f4c151ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:30.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.963+0000 7f1f5369e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f4c15dce0 0x7f1f4c09e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:30.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.963+0000 7f1f5369e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f4c15ef50 0x7f1f4c0b86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:30.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.963+0000 7f1f5369e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c0cf720 con 0x7f1f4c15ef50 2024-04-25T09:02:30.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.963+0000 7f1f5369e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c0cf440 con 0x7f1f4c15dce0 2024-04-25T09:02:30.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.963+0000 7f1f5369e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c15f6b0 con 0x7f1f4c0d0920 2024-04-25T09:02:30.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.963+0000 7f1f51c3b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f4c15ef50 0x7f1f4c0b86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:30.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.963+0000 7f1f51c3b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f4c15ef50 0x7f1f4c0b86f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43738/0 (socket says 172.21.15.43:43738) 2024-04-25T09:02:30.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.963+0000 7f1f51c3b640 1 -- 172.21.15.43:0/639681514 learned_addr learned my addr 172.21.15.43:0/639681514 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:30.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.963+0000 7f1f5143a640 1 --2- 172.21.15.43:0/639681514 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1f4c0d0920 0x7f1f4c151ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:30.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.963+0000 7f1f50c39640 1 --2- 172.21.15.43:0/639681514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f4c15dce0 0x7f1f4c09e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:30.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.963+0000 7f1f51c3b640 1 -- 172.21.15.43:0/639681514 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1f4c0d0920 msgr2=0x7f1f4c151ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:30.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.963+0000 7f1f51c3b640 1 --2- 172.21.15.43:0/639681514 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1f4c0d0920 0x7f1f4c151ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:30.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.963+0000 7f1f51c3b640 1 -- 172.21.15.43:0/639681514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f4c15dce0 msgr2=0x7f1f4c09e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:30.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.963+0000 7f1f51c3b640 1 --2- 172.21.15.43:0/639681514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f4c15dce0 0x7f1f4c09e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:30.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.963+0000 7f1f51c3b640 1 -- 172.21.15.43:0/639681514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f4c09d9c0 con 0x7f1f4c15ef50 2024-04-25T09:02:30.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.963+0000 7f1f51c3b640 1 --2- 172.21.15.43:0/639681514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f4c15ef50 0x7f1f4c0b86f0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f1f4400b9a0 tx=0x7f1f4400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:30.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.964+0000 7f1f41ffb640 1 -- 172.21.15.43:0/639681514 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f44019070 con 0x7f1f4c15ef50 2024-04-25T09:02:30.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.964+0000 7f1f41ffb640 1 -- 172.21.15.43:0/639681514 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f4400cba0 con 0x7f1f4c15ef50 2024-04-25T09:02:30.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.964+0000 7f1f41ffb640 1 -- 172.21.15.43:0/639681514 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f44014440 con 0x7f1f4c15ef50 2024-04-25T09:02:30.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.964+0000 7f1f5369e640 1 -- 172.21.15.43:0/639681514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f4c09dca0 con 0x7f1f4c15ef50 2024-04-25T09:02:30.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.964+0000 7f1f5369e640 1 -- 172.21.15.43:0/639681514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f4c09ecc0 con 0x7f1f4c15ef50 2024-04-25T09:02:30.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.965+0000 7f1f5369e640 1 -- 172.21.15.43:0/639681514 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f4c0680f0 con 0x7f1f4c15ef50 2024-04-25T09:02:30.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.965+0000 7f1f41ffb640 1 -- 172.21.15.43:0/639681514 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1f4401c070 con 0x7f1f4c15ef50 2024-04-25T09:02:30.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.965+0000 7f1f41ffb640 1 --2- 172.21.15.43:0/639681514 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1f3006b4d0 0x7f1f3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:30.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.966+0000 7f1f41ffb640 1 -- 172.21.15.43:0/639681514 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1f44089e50 con 0x7f1f4c15ef50 2024-04-25T09:02:30.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.966+0000 7f1f5143a640 1 --2- 172.21.15.43:0/639681514 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1f3006b4d0 0x7f1f3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:30.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.966+0000 7f1f5143a640 1 --2- 172.21.15.43:0/639681514 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1f3006b4d0 0x7f1f3006d990 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1f48000f60 tx=0x7f1f48007970 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:30.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:30.970+0000 7f1f41ffb640 1 -- 172.21.15.43:0/639681514 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1f44058cc0 con 0x7f1f4c15ef50 2024-04-25T09:02:31.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.150+0000 7f1f5369e640 1 -- 172.21.15.43:0/639681514 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1f4c09d770 con 0x7f1f3006b4d0 2024-04-25T09:02:31.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.156+0000 7f1f41ffb640 1 -- 172.21.15.43:0/639681514 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252482 (secure 0 0 0) 0x7f1f4c09d770 con 0x7f1f3006b4d0 2024-04-25T09:02:31.147 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:31.148 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-25T09:02:31.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.159+0000 7f1f5369e640 1 -- 172.21.15.43:0/639681514 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1f3006b4d0 msgr2=0x7f1f3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:31.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.159+0000 7f1f5369e640 1 --2- 172.21.15.43:0/639681514 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1f3006b4d0 0x7f1f3006d990 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f1f48000f60 tx=0x7f1f48007970 comp rx=0 tx=0).stop 2024-04-25T09:02:31.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.159+0000 7f1f5369e640 1 -- 172.21.15.43:0/639681514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f4c15ef50 msgr2=0x7f1f4c0b86f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:31.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.159+0000 7f1f5369e640 1 --2- 172.21.15.43:0/639681514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f4c15ef50 0x7f1f4c0b86f0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f1f4400b9a0 tx=0x7f1f4400be60 comp rx=0 tx=0).stop 2024-04-25T09:02:31.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.159+0000 7f1f5369e640 1 -- 172.21.15.43:0/639681514 shutdown_connections 2024-04-25T09:02:31.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.159+0000 7f1f5369e640 1 --2- 172.21.15.43:0/639681514 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f4c15ef50 0x7f1f4c0b86f0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:31.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.159+0000 7f1f5369e640 1 --2- 172.21.15.43:0/639681514 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1f3006b4d0 0x7f1f3006d990 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:31.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.159+0000 7f1f5369e640 1 --2- 172.21.15.43:0/639681514 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f4c15dce0 0x7f1f4c09e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:31.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.159+0000 7f1f5369e640 1 --2- 172.21.15.43:0/639681514 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1f4c0d0920 0x7f1f4c151ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:31.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.159+0000 7f1f5369e640 1 -- 172.21.15.43:0/639681514 >> 172.21.15.43:0/639681514 conn(0x7f1f4c09cd00 msgr2=0x7f1f4c0ce510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:31.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.159+0000 7f1f5369e640 1 -- 172.21.15.43:0/639681514 shutdown_connections 2024-04-25T09:02:31.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.159+0000 7f1f5369e640 1 -- 172.21.15.43:0/639681514 wait complete. 2024-04-25T09:02:31.159 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":362,"stamp":"2024-04-25T09:02:29.809561+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":494888,"kb_used_data":12200,"kb_used_omap":12,"kb_used_meta":482547,"kb_avail":754479832,"statfs":{"total":773094113280,"available":772587347968,"internally_reserved":0,"allocated":12492800,"data_stored":6069812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":494128689},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.023517"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.863781+0000","last_change":"2024-04-25T09:02:26.822906+0000","last_active":"2024-04-25T09:02:27.863781+0000","last_peered":"2024-04-25T09:02:27.863781+0000","last_clean":"2024-04-25T09:02:27.863781+0000","last_became_active":"2024-04-25T09:02:26.820737+0000","last_became_peered":"2024-04-25T09:02:26.820737+0000","last_unstale":"2024-04-25T09:02:27.863781+0000","last_undegraded":"2024-04-25T09:02:27.863781+0000","last_fullsized":"2024-04-25T09:02:27.863781+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:49.004736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.820468+0000","last_change":"2024-04-25T09:02:26.812898+0000","last_active":"2024-04-25T09:02:27.820468+0000","last_peered":"2024-04-25T09:02:27.820468+0000","last_clean":"2024-04-25T09:02:27.820468+0000","last_became_active":"2024-04-25T09:02:26.812637+0000","last_became_peered":"2024-04-25T09:02:26.812637+0000","last_unstale":"2024-04-25T09:02:27.820468+0000","last_undegraded":"2024-04-25T09:02:27.820468+0000","last_fullsized":"2024-04-25T09:02:27.820468+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:54.415160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.863057+0000","last_change":"2024-04-25T09:02:26.823065+0000","last_active":"2024-04-25T09:02:27.863057+0000","last_peered":"2024-04-25T09:02:27.863057+0000","last_clean":"2024-04-25T09:02:27.863057+0000","last_became_active":"2024-04-25T09:02:26.820904+0000","last_became_peered":"2024-04-25T09:02:26.820904+0000","last_unstale":"2024-04-25T09:02:27.863057+0000","last_undegraded":"2024-04-25T09:02:27.863057+0000","last_fullsized":"2024-04-25T09:02:27.863057+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:13.782020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862173+0000","last_change":"2024-04-25T09:02:26.822436+0000","last_active":"2024-04-25T09:02:27.862173+0000","last_peered":"2024-04-25T09:02:27.862173+0000","last_clean":"2024-04-25T09:02:27.862173+0000","last_became_active":"2024-04-25T09:02:26.817810+0000","last_became_peered":"2024-04-25T09:02:26.817810+0000","last_unstale":"2024-04-25T09:02:27.862173+0000","last_undegraded":"2024-04-25T09:02:27.862173+0000","last_fullsized":"2024-04-25T09:02:27.862173+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:25.845136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.819307+0000","last_change":"2024-04-25T09:02:26.818665+0000","last_active":"2024-04-25T09:02:27.819307+0000","last_peered":"2024-04-25T09:02:27.819307+0000","last_clean":"2024-04-25T09:02:27.819307+0000","last_became_active":"2024-04-25T09:02:26.818410+0000","last_became_peered":"2024-04-25T09:02:26.818410+0000","last_unstale":"2024-04-25T09:02:27.819307+0000","last_undegraded":"2024-04-25T09:02:27.819307+0000","last_fullsized":"2024-04-25T09:02:27.819307+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:56.211878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862327+0000","last_change":"2024-04-25T09:02:26.817409+0000","last_active":"2024-04-25T09:02:27.862327+0000","last_peered":"2024-04-25T09:02:27.862327+0000","last_clean":"2024-04-25T09:02:27.862327+0000","last_became_active":"2024-04-25T09:02:26.817132+0000","last_became_peered":"2024-04-25T09:02:26.817132+0000","last_unstale":"2024-04-25T09:02:27.862327+0000","last_undegraded":"2024-04-25T09:02:27.862327+0000","last_fullsized":"2024-04-25T09:02:27.862327+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:54.582797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.820049+0000","last_change":"2024-04-25T09:02:26.811298+0000","last_active":"2024-04-25T09:02:27.820049+0000","last_peered":"2024-04-25T09:02:27.820049+0000","last_clean":"2024-04-25T09:02:27.820049+0000","last_became_active":"2024-04-25T09:02:26.810910+0000","last_became_peered":"2024-04-25T09:02:26.810910+0000","last_unstale":"2024-04-25T09:02:27.820049+0000","last_undegraded":"2024-04-25T09:02:27.820049+0000","last_fullsized":"2024-04-25T09:02:27.820049+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:54.582797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862330+0000","last_change":"2024-04-25T09:02:26.815668+0000","last_active":"2024-04-25T09:02:27.862330+0000","last_peered":"2024-04-25T09:02:27.862330+0000","last_clean":"2024-04-25T09:02:27.862330+0000","last_became_active":"2024-04-25T09:02:26.815398+0000","last_became_peered":"2024-04-25T09:02:26.815398+0000","last_unstale":"2024-04-25T09:02:27.862330+0000","last_undegraded":"2024-04-25T09:02:27.862330+0000","last_fullsized":"2024-04-25T09:02:27.862330+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:15.237882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.863870+0000","last_change":"2024-04-25T09:02:26.826585+0000","last_active":"2024-04-25T09:02:27.863870+0000","last_peered":"2024-04-25T09:02:27.863870+0000","last_clean":"2024-04-25T09:02:27.863870+0000","last_became_active":"2024-04-25T09:02:26.826327+0000","last_became_peered":"2024-04-25T09:02:26.826327+0000","last_unstale":"2024-04-25T09:02:27.863870+0000","last_undegraded":"2024-04-25T09:02:27.863870+0000","last_fullsized":"2024-04-25T09:02:27.863870+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:00.827972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862801+0000","last_change":"2024-04-25T09:02:26.814881+0000","last_active":"2024-04-25T09:02:27.862801+0000","last_peered":"2024-04-25T09:02:27.862801+0000","last_clean":"2024-04-25T09:02:27.862801+0000","last_became_active":"2024-04-25T09:02:26.814621+0000","last_became_peered":"2024-04-25T09:02:26.814621+0000","last_unstale":"2024-04-25T09:02:27.862801+0000","last_undegraded":"2024-04-25T09:02:27.862801+0000","last_fullsized":"2024-04-25T09:02:27.862801+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:25.970680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.820730+0000","last_change":"2024-04-25T09:02:26.810085+0000","last_active":"2024-04-25T09:02:27.820730+0000","last_peered":"2024-04-25T09:02:27.820730+0000","last_clean":"2024-04-25T09:02:27.820730+0000","last_became_active":"2024-04-25T09:02:26.809778+0000","last_became_peered":"2024-04-25T09:02:26.809778+0000","last_unstale":"2024-04-25T09:02:27.820730+0000","last_undegraded":"2024-04-25T09:02:27.820730+0000","last_fullsized":"2024-04-25T09:02:27.820730+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:02.307876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862458+0000","last_change":"2024-04-25T09:02:26.815397+0000","last_active":"2024-04-25T09:02:27.862458+0000","last_peered":"2024-04-25T09:02:27.862458+0000","last_clean":"2024-04-25T09:02:27.862458+0000","last_became_active":"2024-04-25T09:02:26.815054+0000","last_became_peered":"2024-04-25T09:02:26.815054+0000","last_unstale":"2024-04-25T09:02:27.862458+0000","last_undegraded":"2024-04-25T09:02:27.862458+0000","last_fullsized":"2024-04-25T09:02:27.862458+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:10.780038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862942+0000","last_change":"2024-04-25T09:02:26.815391+0000","last_active":"2024-04-25T09:02:27.862942+0000","last_peered":"2024-04-25T09:02:27.862942+0000","last_clean":"2024-04-25T09:02:27.862942+0000","last_became_active":"2024-04-25T09:02:26.815119+0000","last_became_peered":"2024-04-25T09:02:26.815119+0000","last_unstale":"2024-04-25T09:02:27.862942+0000","last_undegraded":"2024-04-25T09:02:27.862942+0000","last_fullsized":"2024-04-25T09:02:27.862942+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:51.706367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862300+0000","last_change":"2024-04-25T09:02:26.816722+0000","last_active":"2024-04-25T09:02:27.862300+0000","last_peered":"2024-04-25T09:02:27.862300+0000","last_clean":"2024-04-25T09:02:27.862300+0000","last_became_active":"2024-04-25T09:02:26.816455+0000","last_became_peered":"2024-04-25T09:02:26.816455+0000","last_unstale":"2024-04-25T09:02:27.862300+0000","last_undegraded":"2024-04-25T09:02:27.862300+0000","last_fullsized":"2024-04-25T09:02:27.862300+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:09.978129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862562+0000","last_change":"2024-04-25T09:02:26.818556+0000","last_active":"2024-04-25T09:02:27.862562+0000","last_peered":"2024-04-25T09:02:27.862562+0000","last_clean":"2024-04-25T09:02:27.862562+0000","last_became_active":"2024-04-25T09:02:26.818288+0000","last_became_peered":"2024-04-25T09:02:26.818288+0000","last_unstale":"2024-04-25T09:02:27.862562+0000","last_undegraded":"2024-04-25T09:02:27.862562+0000","last_fullsized":"2024-04-25T09:02:27.862562+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:02.253566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.861804+0000","last_change":"2024-04-25T09:02:26.829155+0000","last_active":"2024-04-25T09:02:27.861804+0000","last_peered":"2024-04-25T09:02:27.861804+0000","last_clean":"2024-04-25T09:02:27.861804+0000","last_became_active":"2024-04-25T09:02:26.821965+0000","last_became_peered":"2024-04-25T09:02:26.821965+0000","last_unstale":"2024-04-25T09:02:27.861804+0000","last_undegraded":"2024-04-25T09:02:27.861804+0000","last_fullsized":"2024-04-25T09:02:27.861804+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:33.563639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.820823+0000","last_change":"2024-04-25T09:02:26.811291+0000","last_active":"2024-04-25T09:02:27.820823+0000","last_peered":"2024-04-25T09:02:27.820823+0000","last_clean":"2024-04-25T09:02:27.820823+0000","last_became_active":"2024-04-25T09:02:26.810633+0000","last_became_peered":"2024-04-25T09:02:26.810633+0000","last_unstale":"2024-04-25T09:02:27.820823+0000","last_undegraded":"2024-04-25T09:02:27.820823+0000","last_fullsized":"2024-04-25T09:02:27.820823+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:09.693837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862452+0000","last_change":"2024-04-25T09:02:26.812521+0000","last_active":"2024-04-25T09:02:27.862452+0000","last_peered":"2024-04-25T09:02:27.862452+0000","last_clean":"2024-04-25T09:02:27.862452+0000","last_became_active":"2024-04-25T09:02:26.812192+0000","last_became_peered":"2024-04-25T09:02:26.812192+0000","last_unstale":"2024-04-25T09:02:27.862452+0000","last_undegraded":"2024-04-25T09:02:27.862452+0000","last_fullsized":"2024-04-25T09:02:27.862452+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:15.671723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.863192+0000","last_change":"2024-04-25T09:02:26.819158+0000","last_active":"2024-04-25T09:02:27.863192+0000","last_peered":"2024-04-25T09:02:27.863192+0000","last_clean":"2024-04-25T09:02:27.863192+0000","last_became_active":"2024-04-25T09:02:26.818576+0000","last_became_peered":"2024-04-25T09:02:26.818576+0000","last_unstale":"2024-04-25T09:02:27.863192+0000","last_undegraded":"2024-04-25T09:02:27.863192+0000","last_fullsized":"2024-04-25T09:02:27.863192+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:10.215927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.820733+0000","last_change":"2024-04-25T09:02:26.814724+0000","last_active":"2024-04-25T09:02:27.820733+0000","last_peered":"2024-04-25T09:02:27.820733+0000","last_clean":"2024-04-25T09:02:27.820733+0000","last_became_active":"2024-04-25T09:02:26.813861+0000","last_became_peered":"2024-04-25T09:02:26.813861+0000","last_unstale":"2024-04-25T09:02:27.820733+0000","last_undegraded":"2024-04-25T09:02:27.820733+0000","last_fullsized":"2024-04-25T09:02:27.820733+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:48.848804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.820889+0000","last_change":"2024-04-25T09:02:26.819381+0000","last_active":"2024-04-25T09:02:27.820889+0000","last_peered":"2024-04-25T09:02:27.820889+0000","last_clean":"2024-04-25T09:02:27.820889+0000","last_became_active":"2024-04-25T09:02:26.819122+0000","last_became_peered":"2024-04-25T09:02:26.819122+0000","last_unstale":"2024-04-25T09:02:27.820889+0000","last_undegraded":"2024-04-25T09:02:27.820889+0000","last_fullsized":"2024-04-25T09:02:27.820889+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:23:09.320655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862701+0000","last_change":"2024-04-25T09:02:26.815215+0000","last_active":"2024-04-25T09:02:27.862701+0000","last_peered":"2024-04-25T09:02:27.862701+0000","last_clean":"2024-04-25T09:02:27.862701+0000","last_became_active":"2024-04-25T09:02:26.814922+0000","last_became_peered":"2024-04-25T09:02:26.814922+0000","last_unstale":"2024-04-25T09:02:27.862701+0000","last_undegraded":"2024-04-25T09:02:27.862701+0000","last_fullsized":"2024-04-25T09:02:27.862701+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:51.327363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.819800+0000","last_change":"2024-04-25T09:02:26.814270+0000","last_active":"2024-04-25T09:02:27.819800+0000","last_peered":"2024-04-25T09:02:27.819800+0000","last_clean":"2024-04-25T09:02:27.819800+0000","last_became_active":"2024-04-25T09:02:26.813869+0000","last_became_peered":"2024-04-25T09:02:26.813869+0000","last_unstale":"2024-04-25T09:02:27.819800+0000","last_undegraded":"2024-04-25T09:02:27.819800+0000","last_fullsized":"2024-04-25T09:02:27.819800+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:00.537477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862651+0000","last_change":"2024-04-25T09:02:26.817901+0000","last_active":"2024-04-25T09:02:27.862651+0000","last_peered":"2024-04-25T09:02:27.862651+0000","last_clean":"2024-04-25T09:02:27.862651+0000","last_became_active":"2024-04-25T09:02:26.817575+0000","last_became_peered":"2024-04-25T09:02:26.817575+0000","last_unstale":"2024-04-25T09:02:27.862651+0000","last_undegraded":"2024-04-25T09:02:27.862651+0000","last_fullsized":"2024-04-25T09:02:27.862651+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:15.237882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.820890+0000","last_change":"2024-04-25T09:02:26.812084+0000","last_active":"2024-04-25T09:02:27.820890+0000","last_peered":"2024-04-25T09:02:27.820890+0000","last_clean":"2024-04-25T09:02:27.820890+0000","last_became_active":"2024-04-25T09:02:26.811024+0000","last_became_peered":"2024-04-25T09:02:26.811024+0000","last_unstale":"2024-04-25T09:02:27.820890+0000","last_undegraded":"2024-04-25T09:02:27.820890+0000","last_fullsized":"2024-04-25T09:02:27.820890+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:53.694921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862935+0000","last_change":"2024-04-25T09:02:26.823420+0000","last_active":"2024-04-25T09:02:27.862935+0000","last_peered":"2024-04-25T09:02:27.862935+0000","last_clean":"2024-04-25T09:02:27.862935+0000","last_became_active":"2024-04-25T09:02:26.815839+0000","last_became_peered":"2024-04-25T09:02:26.815839+0000","last_unstale":"2024-04-25T09:02:27.862935+0000","last_undegraded":"2024-04-25T09:02:27.862935+0000","last_fullsized":"2024-04-25T09:02:27.862935+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:38.689898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.863183+0000","last_change":"2024-04-25T09:02:26.817341+0000","last_active":"2024-04-25T09:02:27.863183+0000","last_peered":"2024-04-25T09:02:27.863183+0000","last_clean":"2024-04-25T09:02:27.863183+0000","last_became_active":"2024-04-25T09:02:26.816313+0000","last_became_peered":"2024-04-25T09:02:26.816313+0000","last_unstale":"2024-04-25T09:02:27.863183+0000","last_undegraded":"2024-04-25T09:02:27.863183+0000","last_fullsized":"2024-04-25T09:02:27.863183+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:19.803627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.863125+0000","last_change":"2024-04-25T09:02:26.817297+0000","last_active":"2024-04-25T09:02:27.863125+0000","last_peered":"2024-04-25T09:02:27.863125+0000","last_clean":"2024-04-25T09:02:27.863125+0000","last_became_active":"2024-04-25T09:02:26.816930+0000","last_became_peered":"2024-04-25T09:02:26.816930+0000","last_unstale":"2024-04-25T09:02:27.863125+0000","last_undegraded":"2024-04-25T09:02:27.863125+0000","last_fullsized":"2024-04-25T09:02:27.863125+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:44.579862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.820694+0000","last_change":"2024-04-25T09:02:26.816690+0000","last_active":"2024-04-25T09:02:27.820694+0000","last_peered":"2024-04-25T09:02:27.820694+0000","last_clean":"2024-04-25T09:02:27.820694+0000","last_became_active":"2024-04-25T09:02:26.816435+0000","last_became_peered":"2024-04-25T09:02:26.816435+0000","last_unstale":"2024-04-25T09:02:27.820694+0000","last_undegraded":"2024-04-25T09:02:27.820694+0000","last_fullsized":"2024-04-25T09:02:27.820694+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:08.630574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.820420+0000","last_change":"2024-04-25T09:02:26.817399+0000","last_active":"2024-04-25T09:02:27.820420+0000","last_peered":"2024-04-25T09:02:27.820420+0000","last_clean":"2024-04-25T09:02:27.820420+0000","last_became_active":"2024-04-25T09:02:26.817135+0000","last_became_peered":"2024-04-25T09:02:26.817135+0000","last_unstale":"2024-04-25T09:02:27.820420+0000","last_undegraded":"2024-04-25T09:02:27.820420+0000","last_fullsized":"2024-04-25T09:02:27.820420+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:24.003416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862054+0000","last_change":"2024-04-25T09:02:26.813767+0000","last_active":"2024-04-25T09:02:27.862054+0000","last_peered":"2024-04-25T09:02:27.862054+0000","last_clean":"2024-04-25T09:02:27.862054+0000","last_became_active":"2024-04-25T09:02:26.813482+0000","last_became_peered":"2024-04-25T09:02:26.813482+0000","last_unstale":"2024-04-25T09:02:27.862054+0000","last_undegraded":"2024-04-25T09:02:27.862054+0000","last_fullsized":"2024-04-25T09:02:27.862054+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:05.912331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862525+0000","last_change":"2024-04-25T09:02:26.819805+0000","last_active":"2024-04-25T09:02:27.862525+0000","last_peered":"2024-04-25T09:02:27.862525+0000","last_clean":"2024-04-25T09:02:27.862525+0000","last_became_active":"2024-04-25T09:02:26.819537+0000","last_became_peered":"2024-04-25T09:02:26.819537+0000","last_unstale":"2024-04-25T09:02:27.862525+0000","last_undegraded":"2024-04-25T09:02:27.862525+0000","last_fullsized":"2024-04-25T09:02:27.862525+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:35.570425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.863483+0000","last_change":"2024-04-25T09:02:26.817876+0000","last_active":"2024-04-25T09:02:27.863483+0000","last_peered":"2024-04-25T09:02:27.863483+0000","last_clean":"2024-04-25T09:02:27.863483+0000","last_became_active":"2024-04-25T09:02:26.817589+0000","last_became_peered":"2024-04-25T09:02:26.817589+0000","last_unstale":"2024-04-25T09:02:27.863483+0000","last_undegraded":"2024-04-25T09:02:27.863483+0000","last_fullsized":"2024-04-25T09:02:27.863483+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:20.819266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"112'4","reported_seq":18,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.819990+0000","last_change":"2024-04-25T09:02:25.797802+0000","last_active":"2024-04-25T09:02:27.819990+0000","last_peered":"2024-04-25T09:02:27.819990+0000","last_clean":"2024-04-25T09:02:27.819990+0000","last_became_active":"2024-04-25T09:02:25.796869+0000","last_became_peered":"2024-04-25T09:02:25.796869+0000","last_unstale":"2024-04-25T09:02:27.819990+0000","last_undegraded":"2024-04-25T09:02:27.819990+0000","last_fullsized":"2024-04-25T09:02:27.819990+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:55.207697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862827+0000","last_change":"2024-04-25T09:02:26.822648+0000","last_active":"2024-04-25T09:02:27.862827+0000","last_peered":"2024-04-25T09:02:27.862827+0000","last_clean":"2024-04-25T09:02:27.862827+0000","last_became_active":"2024-04-25T09:02:26.821791+0000","last_became_peered":"2024-04-25T09:02:26.821791+0000","last_unstale":"2024-04-25T09:02:27.862827+0000","last_undegraded":"2024-04-25T09:02:27.862827+0000","last_fullsized":"2024-04-25T09:02:27.862827+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:11:08.630574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862663+0000","last_change":"2024-04-25T09:02:25.804229+0000","last_active":"2024-04-25T09:02:27.862663+0000","last_peered":"2024-04-25T09:02:27.862663+0000","last_clean":"2024-04-25T09:02:27.862663+0000","last_became_active":"2024-04-25T09:02:25.803909+0000","last_became_peered":"2024-04-25T09:02:25.803909+0000","last_unstale":"2024-04-25T09:02:27.862663+0000","last_undegraded":"2024-04-25T09:02:27.862663+0000","last_fullsized":"2024-04-25T09:02:27.862663+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:01.303695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.863344+0000","last_change":"2024-04-25T09:02:26.819233+0000","last_active":"2024-04-25T09:02:27.863344+0000","last_peered":"2024-04-25T09:02:27.863344+0000","last_clean":"2024-04-25T09:02:27.863344+0000","last_became_active":"2024-04-25T09:02:26.818895+0000","last_became_peered":"2024-04-25T09:02:26.818895+0000","last_unstale":"2024-04-25T09:02:27.863344+0000","last_undegraded":"2024-04-25T09:02:27.863344+0000","last_fullsized":"2024-04-25T09:02:27.863344+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:24.481351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.819197+0000","last_change":"2024-04-25T09:02:25.803200+0000","last_active":"2024-04-25T09:02:27.819197+0000","last_peered":"2024-04-25T09:02:27.819197+0000","last_clean":"2024-04-25T09:02:27.819197+0000","last_became_active":"2024-04-25T09:02:25.802851+0000","last_became_peered":"2024-04-25T09:02:25.802851+0000","last_unstale":"2024-04-25T09:02:27.819197+0000","last_undegraded":"2024-04-25T09:02:27.819197+0000","last_fullsized":"2024-04-25T09:02:27.819197+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:17.739771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.863158+0000","last_change":"2024-04-25T09:02:26.819015+0000","last_active":"2024-04-25T09:02:27.863158+0000","last_peered":"2024-04-25T09:02:27.863158+0000","last_clean":"2024-04-25T09:02:27.863158+0000","last_became_active":"2024-04-25T09:02:26.818161+0000","last_became_peered":"2024-04-25T09:02:26.818161+0000","last_unstale":"2024-04-25T09:02:27.863158+0000","last_undegraded":"2024-04-25T09:02:27.863158+0000","last_fullsized":"2024-04-25T09:02:27.863158+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:44.579862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.819272+0000","last_change":"2024-04-25T09:02:25.801810+0000","last_active":"2024-04-25T09:02:27.819272+0000","last_peered":"2024-04-25T09:02:27.819272+0000","last_clean":"2024-04-25T09:02:27.819272+0000","last_became_active":"2024-04-25T09:02:25.801499+0000","last_became_peered":"2024-04-25T09:02:25.801499+0000","last_unstale":"2024-04-25T09:02:27.819272+0000","last_undegraded":"2024-04-25T09:02:27.819272+0000","last_fullsized":"2024-04-25T09:02:27.819272+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:37:04.560283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.861929+0000","last_change":"2024-04-25T09:02:26.823166+0000","last_active":"2024-04-25T09:02:27.861929+0000","last_peered":"2024-04-25T09:02:27.861929+0000","last_clean":"2024-04-25T09:02:27.861929+0000","last_became_active":"2024-04-25T09:02:26.820754+0000","last_became_peered":"2024-04-25T09:02:26.820754+0000","last_unstale":"2024-04-25T09:02:27.861929+0000","last_undegraded":"2024-04-25T09:02:27.861929+0000","last_fullsized":"2024-04-25T09:02:27.861929+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:42.999643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.819824+0000","last_change":"2024-04-25T09:02:25.802590+0000","last_active":"2024-04-25T09:02:27.819824+0000","last_peered":"2024-04-25T09:02:27.819824+0000","last_clean":"2024-04-25T09:02:27.819824+0000","last_became_active":"2024-04-25T09:02:25.802311+0000","last_became_peered":"2024-04-25T09:02:25.802311+0000","last_unstale":"2024-04-25T09:02:27.819824+0000","last_undegraded":"2024-04-25T09:02:27.819824+0000","last_fullsized":"2024-04-25T09:02:27.819824+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:07.755858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.820673+0000","last_change":"2024-04-25T09:02:26.813278+0000","last_active":"2024-04-25T09:02:27.820673+0000","last_peered":"2024-04-25T09:02:27.820673+0000","last_clean":"2024-04-25T09:02:27.820673+0000","last_became_active":"2024-04-25T09:02:26.813010+0000","last_became_peered":"2024-04-25T09:02:26.813010+0000","last_unstale":"2024-04-25T09:02:27.820673+0000","last_undegraded":"2024-04-25T09:02:27.820673+0000","last_fullsized":"2024-04-25T09:02:27.820673+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:58.133772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862952+0000","last_change":"2024-04-25T09:02:25.805830+0000","last_active":"2024-04-25T09:02:27.862952+0000","last_peered":"2024-04-25T09:02:27.862952+0000","last_clean":"2024-04-25T09:02:27.862952+0000","last_became_active":"2024-04-25T09:02:25.805475+0000","last_became_peered":"2024-04-25T09:02:25.805475+0000","last_unstale":"2024-04-25T09:02:27.862952+0000","last_undegraded":"2024-04-25T09:02:27.862952+0000","last_fullsized":"2024-04-25T09:02:27.862952+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:01.303695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.861956+0000","last_change":"2024-04-25T09:02:26.820034+0000","last_active":"2024-04-25T09:02:27.861956+0000","last_peered":"2024-04-25T09:02:27.861956+0000","last_clean":"2024-04-25T09:02:27.861956+0000","last_became_active":"2024-04-25T09:02:26.813177+0000","last_became_peered":"2024-04-25T09:02:26.813177+0000","last_unstale":"2024-04-25T09:02:27.861956+0000","last_undegraded":"2024-04-25T09:02:27.861956+0000","last_fullsized":"2024-04-25T09:02:27.861956+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:27.696682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.819916+0000","last_change":"2024-04-25T09:02:25.797181+0000","last_active":"2024-04-25T09:02:27.819916+0000","last_peered":"2024-04-25T09:02:27.819916+0000","last_clean":"2024-04-25T09:02:27.819916+0000","last_became_active":"2024-04-25T09:02:25.796880+0000","last_became_peered":"2024-04-25T09:02:25.796880+0000","last_unstale":"2024-04-25T09:02:27.819916+0000","last_undegraded":"2024-04-25T09:02:27.819916+0000","last_fullsized":"2024-04-25T09:02:27.819916+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:03:13.158075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.819537+0000","last_change":"2024-04-25T09:02:26.811990+0000","last_active":"2024-04-25T09:02:27.819537+0000","last_peered":"2024-04-25T09:02:27.819537+0000","last_clean":"2024-04-25T09:02:27.819537+0000","last_became_active":"2024-04-25T09:02:26.811702+0000","last_became_peered":"2024-04-25T09:02:26.811702+0000","last_unstale":"2024-04-25T09:02:27.819537+0000","last_undegraded":"2024-04-25T09:02:27.819537+0000","last_fullsized":"2024-04-25T09:02:27.819537+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:15.687634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862070+0000","last_change":"2024-04-25T09:02:25.812319+0000","last_active":"2024-04-25T09:02:27.862070+0000","last_peered":"2024-04-25T09:02:27.862070+0000","last_clean":"2024-04-25T09:02:27.862070+0000","last_became_active":"2024-04-25T09:02:25.811982+0000","last_became_peered":"2024-04-25T09:02:25.811982+0000","last_unstale":"2024-04-25T09:02:27.862070+0000","last_undegraded":"2024-04-25T09:02:27.862070+0000","last_fullsized":"2024-04-25T09:02:27.862070+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:52.690740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.820257+0000","last_change":"2024-04-25T09:02:26.811316+0000","last_active":"2024-04-25T09:02:27.820257+0000","last_peered":"2024-04-25T09:02:27.820257+0000","last_clean":"2024-04-25T09:02:27.820257+0000","last_became_active":"2024-04-25T09:02:26.810973+0000","last_became_peered":"2024-04-25T09:02:26.810973+0000","last_unstale":"2024-04-25T09:02:27.820257+0000","last_undegraded":"2024-04-25T09:02:27.820257+0000","last_fullsized":"2024-04-25T09:02:27.820257+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:35.570425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.819139+0000","last_change":"2024-04-25T09:02:25.804273+0000","last_active":"2024-04-25T09:02:27.819139+0000","last_peered":"2024-04-25T09:02:27.819139+0000","last_clean":"2024-04-25T09:02:27.819139+0000","last_became_active":"2024-04-25T09:02:25.803994+0000","last_became_peered":"2024-04-25T09:02:25.803994+0000","last_unstale":"2024-04-25T09:02:27.819139+0000","last_undegraded":"2024-04-25T09:02:27.819139+0000","last_fullsized":"2024-04-25T09:02:27.819139+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:19.214564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862999+0000","last_change":"2024-04-25T09:02:26.816903+0000","last_active":"2024-04-25T09:02:27.862999+0000","last_peered":"2024-04-25T09:02:27.862999+0000","last_clean":"2024-04-25T09:02:27.862999+0000","last_became_active":"2024-04-25T09:02:26.816364+0000","last_became_peered":"2024-04-25T09:02:26.816364+0000","last_unstale":"2024-04-25T09:02:27.862999+0000","last_undegraded":"2024-04-25T09:02:27.862999+0000","last_fullsized":"2024-04-25T09:02:27.862999+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:48.848804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862206+0000","last_change":"2024-04-25T09:02:25.804432+0000","last_active":"2024-04-25T09:02:27.862206+0000","last_peered":"2024-04-25T09:02:27.862206+0000","last_clean":"2024-04-25T09:02:27.862206+0000","last_became_active":"2024-04-25T09:02:25.804141+0000","last_became_peered":"2024-04-25T09:02:25.804141+0000","last_unstale":"2024-04-25T09:02:27.862206+0000","last_undegraded":"2024-04-25T09:02:27.862206+0000","last_fullsized":"2024-04-25T09:02:27.862206+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:47.844623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.863135+0000","last_change":"2024-04-25T09:02:26.806921+0000","last_active":"2024-04-25T09:02:27.863135+0000","last_peered":"2024-04-25T09:02:27.863135+0000","last_clean":"2024-04-25T09:02:27.863135+0000","last_became_active":"2024-04-25T09:02:26.806652+0000","last_became_peered":"2024-04-25T09:02:26.806652+0000","last_unstale":"2024-04-25T09:02:27.863135+0000","last_undegraded":"2024-04-25T09:02:27.863135+0000","last_fullsized":"2024-04-25T09:02:27.863135+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:10.215927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"112'3","reported_seq":17,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862227+0000","last_change":"2024-04-25T09:02:25.804867+0000","last_active":"2024-04-25T09:02:27.862227+0000","last_peered":"2024-04-25T09:02:27.862227+0000","last_clean":"2024-04-25T09:02:27.862227+0000","last_became_active":"2024-04-25T09:02:25.804595+0000","last_became_peered":"2024-04-25T09:02:25.804595+0000","last_unstale":"2024-04-25T09:02:27.862227+0000","last_undegraded":"2024-04-25T09:02:27.862227+0000","last_fullsized":"2024-04-25T09:02:27.862227+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:46.918975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.820349+0000","last_change":"2024-04-25T09:02:26.815204+0000","last_active":"2024-04-25T09:02:27.820349+0000","last_peered":"2024-04-25T09:02:27.820349+0000","last_clean":"2024-04-25T09:02:27.820349+0000","last_became_active":"2024-04-25T09:02:26.814914+0000","last_became_peered":"2024-04-25T09:02:26.814914+0000","last_unstale":"2024-04-25T09:02:27.820349+0000","last_undegraded":"2024-04-25T09:02:27.820349+0000","last_fullsized":"2024-04-25T09:02:27.820349+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:25:02.253566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"112'6","reported_seq":20,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862562+0000","last_change":"2024-04-25T09:02:25.811971+0000","last_active":"2024-04-25T09:02:27.862562+0000","last_peered":"2024-04-25T09:02:27.862562+0000","last_clean":"2024-04-25T09:02:27.862562+0000","last_became_active":"2024-04-25T09:02:25.810936+0000","last_became_peered":"2024-04-25T09:02:25.810936+0000","last_unstale":"2024-04-25T09:02:27.862562+0000","last_undegraded":"2024-04-25T09:02:27.862562+0000","last_fullsized":"2024-04-25T09:02:27.862562+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:40.591769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862865+0000","last_change":"2024-04-25T09:02:26.809666+0000","last_active":"2024-04-25T09:02:27.862865+0000","last_peered":"2024-04-25T09:02:27.862865+0000","last_clean":"2024-04-25T09:02:27.862865+0000","last_became_active":"2024-04-25T09:02:26.809363+0000","last_became_peered":"2024-04-25T09:02:26.809363+0000","last_unstale":"2024-04-25T09:02:27.862865+0000","last_undegraded":"2024-04-25T09:02:27.862865+0000","last_fullsized":"2024-04-25T09:02:27.862865+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:13.782020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862958+0000","last_change":"2024-04-25T09:02:25.800185+0000","last_active":"2024-04-25T09:02:27.862958+0000","last_peered":"2024-04-25T09:02:27.862958+0000","last_clean":"2024-04-25T09:02:27.862958+0000","last_became_active":"2024-04-25T09:02:25.799855+0000","last_became_peered":"2024-04-25T09:02:25.799855+0000","last_unstale":"2024-04-25T09:02:27.862958+0000","last_undegraded":"2024-04-25T09:02:27.862958+0000","last_fullsized":"2024-04-25T09:02:27.862958+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:07.755858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862844+0000","last_change":"2024-04-25T09:02:26.817586+0000","last_active":"2024-04-25T09:02:27.862844+0000","last_peered":"2024-04-25T09:02:27.862844+0000","last_clean":"2024-04-25T09:02:27.862844+0000","last_became_active":"2024-04-25T09:02:26.816997+0000","last_became_peered":"2024-04-25T09:02:26.816997+0000","last_unstale":"2024-04-25T09:02:27.862844+0000","last_undegraded":"2024-04-25T09:02:27.862844+0000","last_fullsized":"2024-04-25T09:02:27.862844+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:49.545678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862469+0000","last_change":"2024-04-25T09:02:25.811195+0000","last_active":"2024-04-25T09:02:27.862469+0000","last_peered":"2024-04-25T09:02:27.862469+0000","last_clean":"2024-04-25T09:02:27.862469+0000","last_became_active":"2024-04-25T09:02:25.810925+0000","last_became_peered":"2024-04-25T09:02:25.810925+0000","last_unstale":"2024-04-25T09:02:27.862469+0000","last_undegraded":"2024-04-25T09:02:27.862469+0000","last_fullsized":"2024-04-25T09:02:27.862469+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:23.477170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862701+0000","last_change":"2024-04-25T09:02:26.815136+0000","last_active":"2024-04-25T09:02:27.862701+0000","last_peered":"2024-04-25T09:02:27.862701+0000","last_clean":"2024-04-25T09:02:27.862701+0000","last_became_active":"2024-04-25T09:02:26.814838+0000","last_became_peered":"2024-04-25T09:02:26.814838+0000","last_unstale":"2024-04-25T09:02:27.862701+0000","last_undegraded":"2024-04-25T09:02:27.862701+0000","last_fullsized":"2024-04-25T09:02:27.862701+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:19.763453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"112'8","reported_seq":22,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.818962+0000","last_change":"2024-04-25T09:02:25.811455+0000","last_active":"2024-04-25T09:02:27.818962+0000","last_peered":"2024-04-25T09:02:27.818962+0000","last_clean":"2024-04-25T09:02:27.818962+0000","last_became_active":"2024-04-25T09:02:25.811175+0000","last_became_peered":"2024-04-25T09:02:25.811175+0000","last_unstale":"2024-04-25T09:02:27.818962+0000","last_undegraded":"2024-04-25T09:02:27.818962+0000","last_fullsized":"2024-04-25T09:02:27.818962+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:28.955003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.819480+0000","last_change":"2024-04-25T09:02:26.819783+0000","last_active":"2024-04-25T09:02:27.819480+0000","last_peered":"2024-04-25T09:02:27.819480+0000","last_clean":"2024-04-25T09:02:27.819480+0000","last_became_active":"2024-04-25T09:02:26.819285+0000","last_became_peered":"2024-04-25T09:02:26.819285+0000","last_unstale":"2024-04-25T09:02:27.819480+0000","last_undegraded":"2024-04-25T09:02:27.819480+0000","last_fullsized":"2024-04-25T09:02:27.819480+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:08.194100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.863966+0000","last_change":"2024-04-25T09:02:25.809787+0000","last_active":"2024-04-25T09:02:27.863966+0000","last_peered":"2024-04-25T09:02:27.863966+0000","last_clean":"2024-04-25T09:02:27.863966+0000","last_became_active":"2024-04-25T09:02:25.809078+0000","last_became_peered":"2024-04-25T09:02:25.809078+0000","last_unstale":"2024-04-25T09:02:27.863966+0000","last_undegraded":"2024-04-25T09:02:27.863966+0000","last_fullsized":"2024-04-25T09:02:27.863966+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:49:24.331148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.863105+0000","last_change":"2024-04-25T09:02:26.825975+0000","last_active":"2024-04-25T09:02:27.863105+0000","last_peered":"2024-04-25T09:02:27.863105+0000","last_clean":"2024-04-25T09:02:27.863105+0000","last_became_active":"2024-04-25T09:02:26.825704+0000","last_became_peered":"2024-04-25T09:02:26.825704+0000","last_unstale":"2024-04-25T09:02:27.863105+0000","last_undegraded":"2024-04-25T09:02:27.863105+0000","last_fullsized":"2024-04-25T09:02:27.863105+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:24.003416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.819517+0000","last_change":"2024-04-25T09:02:25.798505+0000","last_active":"2024-04-25T09:02:27.819517+0000","last_peered":"2024-04-25T09:02:27.819517+0000","last_clean":"2024-04-25T09:02:27.819517+0000","last_became_active":"2024-04-25T09:02:25.798189+0000","last_became_peered":"2024-04-25T09:02:25.798189+0000","last_unstale":"2024-04-25T09:02:27.819517+0000","last_undegraded":"2024-04-25T09:02:27.819517+0000","last_fullsized":"2024-04-25T09:02:27.819517+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:26.870309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862799+0000","last_change":"2024-04-25T09:02:26.826090+0000","last_active":"2024-04-25T09:02:27.862799+0000","last_peered":"2024-04-25T09:02:27.862799+0000","last_clean":"2024-04-25T09:02:27.862799+0000","last_became_active":"2024-04-25T09:02:26.825863+0000","last_became_peered":"2024-04-25T09:02:26.825863+0000","last_unstale":"2024-04-25T09:02:27.862799+0000","last_undegraded":"2024-04-25T09:02:27.862799+0000","last_fullsized":"2024-04-25T09:02:27.862799+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:19.763453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862592+0000","last_change":"2024-04-25T09:02:25.803652+0000","last_active":"2024-04-25T09:02:27.862592+0000","last_peered":"2024-04-25T09:02:27.862592+0000","last_clean":"2024-04-25T09:02:27.862592+0000","last_became_active":"2024-04-25T09:02:25.803266+0000","last_became_peered":"2024-04-25T09:02:25.803266+0000","last_unstale":"2024-04-25T09:02:27.862592+0000","last_undegraded":"2024-04-25T09:02:27.862592+0000","last_fullsized":"2024-04-25T09:02:27.862592+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:18.799446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.863239+0000","last_change":"2024-04-25T09:02:26.822196+0000","last_active":"2024-04-25T09:02:27.863239+0000","last_peered":"2024-04-25T09:02:27.863239+0000","last_clean":"2024-04-25T09:02:27.863239+0000","last_became_active":"2024-04-25T09:02:26.820488+0000","last_became_peered":"2024-04-25T09:02:26.820488+0000","last_unstale":"2024-04-25T09:02:27.863239+0000","last_undegraded":"2024-04-25T09:02:27.863239+0000","last_fullsized":"2024-04-25T09:02:27.863239+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:42:58.811244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.819832+0000","last_change":"2024-04-25T09:02:25.807235+0000","last_active":"2024-04-25T09:02:27.819832+0000","last_peered":"2024-04-25T09:02:27.819832+0000","last_clean":"2024-04-25T09:02:27.819832+0000","last_became_active":"2024-04-25T09:02:25.806851+0000","last_became_peered":"2024-04-25T09:02:25.806851+0000","last_unstale":"2024-04-25T09:02:27.819832+0000","last_undegraded":"2024-04-25T09:02:27.819832+0000","last_fullsized":"2024-04-25T09:02:27.819832+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:31.793700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.863076+0000","last_change":"2024-04-25T09:02:26.818331+0000","last_active":"2024-04-25T09:02:27.863076+0000","last_peered":"2024-04-25T09:02:27.863076+0000","last_clean":"2024-04-25T09:02:27.863076+0000","last_became_active":"2024-04-25T09:02:26.818047+0000","last_became_peered":"2024-04-25T09:02:26.818047+0000","last_unstale":"2024-04-25T09:02:27.863076+0000","last_undegraded":"2024-04-25T09:02:27.863076+0000","last_fullsized":"2024-04-25T09:02:27.863076+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:50.402637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.863097+0000","last_change":"2024-04-25T09:02:25.811062+0000","last_active":"2024-04-25T09:02:27.863097+0000","last_peered":"2024-04-25T09:02:27.863097+0000","last_clean":"2024-04-25T09:02:27.863097+0000","last_became_active":"2024-04-25T09:02:25.810776+0000","last_became_peered":"2024-04-25T09:02:25.810776+0000","last_unstale":"2024-04-25T09:02:27.863097+0000","last_undegraded":"2024-04-25T09:02:27.863097+0000","last_fullsized":"2024-04-25T09:02:27.863097+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:07.189919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862262+0000","last_change":"2024-04-25T09:02:26.821200+0000","last_active":"2024-04-25T09:02:27.862262+0000","last_peered":"2024-04-25T09:02:27.862262+0000","last_clean":"2024-04-25T09:02:27.862262+0000","last_became_active":"2024-04-25T09:02:26.818797+0000","last_became_peered":"2024-04-25T09:02:26.818797+0000","last_unstale":"2024-04-25T09:02:27.862262+0000","last_undegraded":"2024-04-25T09:02:27.862262+0000","last_fullsized":"2024-04-25T09:02:27.862262+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:27:50.760405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.819038+0000","last_change":"2024-04-25T09:02:25.805631+0000","last_active":"2024-04-25T09:02:27.819038+0000","last_peered":"2024-04-25T09:02:27.819038+0000","last_clean":"2024-04-25T09:02:27.819038+0000","last_became_active":"2024-04-25T09:02:25.805348+0000","last_became_peered":"2024-04-25T09:02:25.805348+0000","last_unstale":"2024-04-25T09:02:27.819038+0000","last_undegraded":"2024-04-25T09:02:27.819038+0000","last_fullsized":"2024-04-25T09:02:27.819038+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:45.625953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862494+0000","last_change":"2024-04-25T09:02:26.815708+0000","last_active":"2024-04-25T09:02:27.862494+0000","last_peered":"2024-04-25T09:02:27.862494+0000","last_clean":"2024-04-25T09:02:27.862494+0000","last_became_active":"2024-04-25T09:02:26.815428+0000","last_became_peered":"2024-04-25T09:02:26.815428+0000","last_unstale":"2024-04-25T09:02:27.862494+0000","last_undegraded":"2024-04-25T09:02:27.862494+0000","last_fullsized":"2024-04-25T09:02:27.862494+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:51.327363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862712+0000","last_change":"2024-04-25T09:02:25.800620+0000","last_active":"2024-04-25T09:02:27.862712+0000","last_peered":"2024-04-25T09:02:27.862712+0000","last_clean":"2024-04-25T09:02:27.862712+0000","last_became_active":"2024-04-25T09:02:25.800346+0000","last_became_peered":"2024-04-25T09:02:25.800346+0000","last_unstale":"2024-04-25T09:02:27.862712+0000","last_undegraded":"2024-04-25T09:02:27.862712+0000","last_fullsized":"2024-04-25T09:02:27.862712+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:55.207697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862865+0000","last_change":"2024-04-25T09:02:26.820676+0000","last_active":"2024-04-25T09:02:27.862865+0000","last_peered":"2024-04-25T09:02:27.862865+0000","last_clean":"2024-04-25T09:02:27.862865+0000","last_became_active":"2024-04-25T09:02:26.816953+0000","last_became_peered":"2024-04-25T09:02:26.816953+0000","last_unstale":"2024-04-25T09:02:27.862865+0000","last_undegraded":"2024-04-25T09:02:27.862865+0000","last_fullsized":"2024-04-25T09:02:27.862865+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:33:50.402637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862166+0000","last_change":"2024-04-25T09:02:25.810064+0000","last_active":"2024-04-25T09:02:27.862166+0000","last_peered":"2024-04-25T09:02:27.862166+0000","last_clean":"2024-04-25T09:02:27.862166+0000","last_became_active":"2024-04-25T09:02:25.808573+0000","last_became_peered":"2024-04-25T09:02:25.808573+0000","last_unstale":"2024-04-25T09:02:27.862166+0000","last_undegraded":"2024-04-25T09:02:27.862166+0000","last_fullsized":"2024-04-25T09:02:27.862166+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:39.116454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862622+0000","last_change":"2024-04-25T09:02:26.820044+0000","last_active":"2024-04-25T09:02:27.862622+0000","last_peered":"2024-04-25T09:02:27.862622+0000","last_clean":"2024-04-25T09:02:27.862622+0000","last_became_active":"2024-04-25T09:02:26.819748+0000","last_became_peered":"2024-04-25T09:02:26.819748+0000","last_unstale":"2024-04-25T09:02:27.862622+0000","last_undegraded":"2024-04-25T09:02:27.862622+0000","last_fullsized":"2024-04-25T09:02:27.862622+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:58.133772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"112'2","reported_seq":16,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862514+0000","last_change":"2024-04-25T09:02:25.809249+0000","last_active":"2024-04-25T09:02:27.862514+0000","last_peered":"2024-04-25T09:02:27.862514+0000","last_clean":"2024-04-25T09:02:27.862514+0000","last_became_active":"2024-04-25T09:02:25.808906+0000","last_became_peered":"2024-04-25T09:02:25.808906+0000","last_unstale":"2024-04-25T09:02:27.862514+0000","last_undegraded":"2024-04-25T09:02:27.862514+0000","last_fullsized":"2024-04-25T09:02:27.862514+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:08.689656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.863855+0000","last_change":"2024-04-25T09:02:26.823140+0000","last_active":"2024-04-25T09:02:27.863855+0000","last_peered":"2024-04-25T09:02:27.863855+0000","last_clean":"2024-04-25T09:02:27.863855+0000","last_became_active":"2024-04-25T09:02:26.822572+0000","last_became_peered":"2024-04-25T09:02:26.822572+0000","last_unstale":"2024-04-25T09:02:27.863855+0000","last_undegraded":"2024-04-25T09:02:27.863855+0000","last_fullsized":"2024-04-25T09:02:27.863855+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:51.915253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"112'1","reported_seq":15,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862796+0000","last_change":"2024-04-25T09:02:25.804432+0000","last_active":"2024-04-25T09:02:27.862796+0000","last_peered":"2024-04-25T09:02:27.862796+0000","last_clean":"2024-04-25T09:02:27.862796+0000","last_became_active":"2024-04-25T09:02:25.804129+0000","last_became_peered":"2024-04-25T09:02:25.804129+0000","last_unstale":"2024-04-25T09:02:27.862796+0000","last_undegraded":"2024-04-25T09:02:27.862796+0000","last_fullsized":"2024-04-25T09:02:27.862796+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:21:19.815085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862575+0000","last_change":"2024-04-25T09:02:26.817576+0000","last_active":"2024-04-25T09:02:27.862575+0000","last_peered":"2024-04-25T09:02:27.862575+0000","last_clean":"2024-04-25T09:02:27.862575+0000","last_became_active":"2024-04-25T09:02:26.815719+0000","last_became_peered":"2024-04-25T09:02:26.815719+0000","last_unstale":"2024-04-25T09:02:27.862575+0000","last_undegraded":"2024-04-25T09:02:27.862575+0000","last_fullsized":"2024-04-25T09:02:27.862575+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:41:34.669182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862345+0000","last_change":"2024-04-25T09:02:25.807225+0000","last_active":"2024-04-25T09:02:27.862345+0000","last_peered":"2024-04-25T09:02:27.862345+0000","last_clean":"2024-04-25T09:02:27.862345+0000","last_became_active":"2024-04-25T09:02:25.806697+0000","last_became_peered":"2024-04-25T09:02:25.806697+0000","last_unstale":"2024-04-25T09:02:27.862345+0000","last_undegraded":"2024-04-25T09:02:27.862345+0000","last_fullsized":"2024-04-25T09:02:27.862345+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:33.148270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862410+0000","last_change":"2024-04-25T09:02:26.811933+0000","last_active":"2024-04-25T09:02:27.862410+0000","last_peered":"2024-04-25T09:02:27.862410+0000","last_clean":"2024-04-25T09:02:27.862410+0000","last_became_active":"2024-04-25T09:02:26.811455+0000","last_became_peered":"2024-04-25T09:02:26.811455+0000","last_unstale":"2024-04-25T09:02:27.862410+0000","last_undegraded":"2024-04-25T09:02:27.862410+0000","last_fullsized":"2024-04-25T09:02:27.862410+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:54.415160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.863227+0000","last_change":"2024-04-25T09:02:25.800897+0000","last_active":"2024-04-25T09:02:27.863227+0000","last_peered":"2024-04-25T09:02:27.863227+0000","last_clean":"2024-04-25T09:02:27.863227+0000","last_became_active":"2024-04-25T09:02:25.800622+0000","last_became_peered":"2024-04-25T09:02:25.800622+0000","last_unstale":"2024-04-25T09:02:27.863227+0000","last_undegraded":"2024-04-25T09:02:27.863227+0000","last_fullsized":"2024-04-25T09:02:27.863227+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:09.715801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.820024+0000","last_change":"2024-04-25T09:02:26.805898+0000","last_active":"2024-04-25T09:02:27.820024+0000","last_peered":"2024-04-25T09:02:27.820024+0000","last_clean":"2024-04-25T09:02:27.820024+0000","last_became_active":"2024-04-25T09:02:26.805439+0000","last_became_peered":"2024-04-25T09:02:26.805439+0000","last_unstale":"2024-04-25T09:02:27.820024+0000","last_undegraded":"2024-04-25T09:02:27.820024+0000","last_fullsized":"2024-04-25T09:02:27.820024+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:15.671723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.819947+0000","last_change":"2024-04-25T09:02:25.799689+0000","last_active":"2024-04-25T09:02:27.819947+0000","last_peered":"2024-04-25T09:02:27.819947+0000","last_clean":"2024-04-25T09:02:27.819947+0000","last_became_active":"2024-04-25T09:02:25.799414+0000","last_became_peered":"2024-04-25T09:02:25.799414+0000","last_unstale":"2024-04-25T09:02:27.819947+0000","last_undegraded":"2024-04-25T09:02:27.819947+0000","last_fullsized":"2024-04-25T09:02:27.819947+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:07.189919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.820798+0000","last_change":"2024-04-25T09:02:26.814226+0000","last_active":"2024-04-25T09:02:27.820798+0000","last_peered":"2024-04-25T09:02:27.820798+0000","last_clean":"2024-04-25T09:02:27.820798+0000","last_became_active":"2024-04-25T09:02:26.813732+0000","last_became_peered":"2024-04-25T09:02:26.813732+0000","last_unstale":"2024-04-25T09:02:27.820798+0000","last_undegraded":"2024-04-25T09:02:27.820798+0000","last_fullsized":"2024-04-25T09:02:27.820798+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:10.719983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.820586+0000","last_change":"2024-04-25T09:02:25.801797+0000","last_active":"2024-04-25T09:02:27.820586+0000","last_peered":"2024-04-25T09:02:27.820586+0000","last_clean":"2024-04-25T09:02:27.820586+0000","last_became_active":"2024-04-25T09:02:25.801516+0000","last_became_peered":"2024-04-25T09:02:25.801516+0000","last_unstale":"2024-04-25T09:02:27.820586+0000","last_undegraded":"2024-04-25T09:02:27.820586+0000","last_fullsized":"2024-04-25T09:02:27.820586+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:53.163079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862970+0000","last_change":"2024-04-25T09:02:26.817550+0000","last_active":"2024-04-25T09:02:27.862970+0000","last_peered":"2024-04-25T09:02:27.862970+0000","last_clean":"2024-04-25T09:02:27.862970+0000","last_became_active":"2024-04-25T09:02:26.817174+0000","last_became_peered":"2024-04-25T09:02:26.817174+0000","last_unstale":"2024-04-25T09:02:27.862970+0000","last_undegraded":"2024-04-25T09:02:27.862970+0000","last_fullsized":"2024-04-25T09:02:27.862970+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:49.545678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.819044+0000","last_change":"2024-04-25T09:02:25.802637+0000","last_active":"2024-04-25T09:02:27.819044+0000","last_peered":"2024-04-25T09:02:27.819044+0000","last_clean":"2024-04-25T09:02:27.819044+0000","last_became_active":"2024-04-25T09:02:25.802294+0000","last_became_peered":"2024-04-25T09:02:25.802294+0000","last_unstale":"2024-04-25T09:02:27.819044+0000","last_undegraded":"2024-04-25T09:02:27.819044+0000","last_fullsized":"2024-04-25T09:02:27.819044+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:22.377195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862420+0000","last_change":"2024-04-25T09:02:26.814711+0000","last_active":"2024-04-25T09:02:27.862420+0000","last_peered":"2024-04-25T09:02:27.862420+0000","last_clean":"2024-04-25T09:02:27.862420+0000","last_became_active":"2024-04-25T09:02:26.813816+0000","last_became_peered":"2024-04-25T09:02:26.813816+0000","last_unstale":"2024-04-25T09:02:27.862420+0000","last_undegraded":"2024-04-25T09:02:27.862420+0000","last_fullsized":"2024-04-25T09:02:27.862420+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:42.435105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862264+0000","last_change":"2024-04-25T09:02:25.808730+0000","last_active":"2024-04-25T09:02:27.862264+0000","last_peered":"2024-04-25T09:02:27.862264+0000","last_clean":"2024-04-25T09:02:27.862264+0000","last_became_active":"2024-04-25T09:02:25.808443+0000","last_became_peered":"2024-04-25T09:02:25.808443+0000","last_unstale":"2024-04-25T09:02:27.862264+0000","last_undegraded":"2024-04-25T09:02:27.862264+0000","last_fullsized":"2024-04-25T09:02:27.862264+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:50.725493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":12,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862794+0000","last_change":"2024-04-25T09:02:26.817779+0000","last_active":"2024-04-25T09:02:27.862794+0000","last_peered":"2024-04-25T09:02:27.862794+0000","last_clean":"2024-04-25T09:02:27.862794+0000","last_became_active":"2024-04-25T09:02:26.817498+0000","last_became_peered":"2024-04-25T09:02:26.817498+0000","last_unstale":"2024-04-25T09:02:27.862794+0000","last_undegraded":"2024-04-25T09:02:27.862794+0000","last_fullsized":"2024-04-25T09:02:27.862794+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:25.785081+0000","last_clean_scrub_stamp":"2024-04-25T09:02:25.785081+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:50:51.706367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":14,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.819742+0000","last_change":"2024-04-25T09:02:25.807278+0000","last_active":"2024-04-25T09:02:27.819742+0000","last_peered":"2024-04-25T09:02:27.819742+0000","last_clean":"2024-04-25T09:02:27.819742+0000","last_became_active":"2024-04-25T09:02:25.806915+0000","last_became_peered":"2024-04-25T09:02:25.806915+0000","last_unstale":"2024-04-25T09:02:27.819742+0000","last_undegraded":"2024-04-25T09:02:27.819742+0000","last_fullsized":"2024-04-25T09:02:27.819742+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":110,"last_epoch_clean":111,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:02:24.780900+0000","last_clean_scrub_stamp":"2024-04-25T09:02:24.780900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:19.009115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":341,"reported_epoch":113,"state":"active+clean","last_fresh":"2024-04-25T09:02:27.862898+0000","last_change":"2024-04-25T08:54:00.818044+0000","last_active":"2024-04-25T09:02:27.862898+0000","last_peered":"2024-04-25T09:02:27.862898+0000","last_clean":"2024-04-25T09:02:27.862898+0000","last_became_active":"2024-04-25T08:54:00.817098+0000","last_became_peered":"2024-04-25T08:54:00.817098+0000","last_unstale":"2024-04-25T09:02:27.862898+0000","last_undegraded":"2024-04-25T09:02:27.862898+0000","last_fullsized":"2024-04-25T09:02:27.862898+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_clean_scrub_stamp":"2024-04-25T08:53:59.739677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:48.129513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574952,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59516,"kb_used_data":1960,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312324,"statfs":{"total":96636764160,"available":96575819776,"internally_reserved":0,"allocated":2007040,"data_stored":1201516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574953,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64500,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94307340,"statfs":{"total":96636764160,"available":96570716160,"internally_reserved":0,"allocated":1409024,"data_stored":610602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574955,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50608,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321232,"statfs":{"total":96636764160,"available":96584941568,"internally_reserved":0,"allocated":1413120,"data_stored":611136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574953,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64496,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94307344,"statfs":{"total":96636764160,"available":96570720256,"internally_reserved":0,"allocated":1404928,"data_stored":610532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574953,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72828,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94299012,"statfs":{"total":96636764160,"available":96562188288,"internally_reserved":0,"allocated":1421312,"data_stored":612214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574953,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58920,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312920,"statfs":{"total":96636764160,"available":96576430080,"internally_reserved":0,"allocated":1404928,"data_stored":610532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574953,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":76976,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294864,"statfs":{"total":96636764160,"available":96557940736,"internally_reserved":0,"allocated":1413120,"data_stored":610656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574954,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47044,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324796,"statfs":{"total":96636764160,"available":96588591104,"internally_reserved":0,"allocated":2019328,"data_stored":1202624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:02:31.163 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:02:31.163 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T09:02:31.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.319+0000 7fbcb8b63640 1 Processor -- start 2024-04-25T09:02:31.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.319+0000 7fbcb8b63640 1 -- start start 2024-04-25T09:02:31.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.320+0000 7fbcb8b63640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fbcb40d1fc0 0x7fbcb40d2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:31.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.320+0000 7fbcb8b63640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbcb40b7920 0x7fbcb40b7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:31.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.320+0000 7fbcb8b63640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcb40b68c0 con 0x7fbcb415de10 2024-04-25T09:02:31.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.320+0000 7fbcb8b63640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcb40d2960 con 0x7fbcb40d1fc0 2024-04-25T09:02:31.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.320+0000 7fbcb8b63640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb40b7490 con 0x7fbcb40b7920 2024-04-25T09:02:31.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.320+0000 7fbcb1d9b640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fbcb40d1fc0 0x7fbcb40d2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:31.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.320+0000 7fbcb1d9b640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fbcb40d1fc0 0x7fbcb40d2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:42484/0 (socket says 172.21.15.43:42484) 2024-04-25T09:02:31.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.320+0000 7fbcb2d9d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbcb40b7920 0x7fbcb40b7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:31.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.320+0000 7fbcb1d9b640 1 -- 172.21.15.43:0/1449550419 learned_addr learned my addr 172.21.15.43:0/1449550419 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:31.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.320+0000 7fbcb259c640 1 --1- 172.21.15.43:0/1449550419 >> v1:172.21.15.193:6789/0 conn(0x7fbcb415de10 0x7fbcb40cf100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:55432/0 (socket says 172.21.15.43:55432) 2024-04-25T09:02:31.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.321+0000 7fbcb1d9b640 1 -- 172.21.15.43:0/1449550419 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbcb40b7920 msgr2=0x7fbcb40b7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:31.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.321+0000 7fbcb1d9b640 1 --2- 172.21.15.43:0/1449550419 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbcb40b7920 0x7fbcb40b7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:31.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.321+0000 7fbcb1d9b640 1 -- 172.21.15.43:0/1449550419 >> v1:172.21.15.193:6789/0 conn(0x7fbcb415de10 legacy=0x7fbcb40cf100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:31.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.321+0000 7fbcb159a640 1 -- 172.21.15.43:0/1449550419 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2851178572 0 0) 0x7fbcb40b68c0 con 0x7fbcb415de10 2024-04-25T09:02:31.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.321+0000 7fbcb1d9b640 1 -- 172.21.15.43:0/1449550419 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb40b6b60 con 0x7fbcb40d1fc0 2024-04-25T09:02:31.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.322+0000 7fbcb1d9b640 1 --2- 172.21.15.43:0/1449550419 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcb40d1fc0 0x7fbcb40d2420 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fbcb40b7610 tx=0x7fbc9c02a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0a9d108f710ca01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:31.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.322+0000 7fbcb159a640 1 -- 172.21.15.43:0/1449550419 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c037070 con 0x7fbcb40d1fc0 2024-04-25T09:02:31.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.322+0000 7fbcb159a640 1 -- 172.21.15.43:0/1449550419 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc9c032440 con 0x7fbcb40d1fc0 2024-04-25T09:02:31.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.322+0000 7fbcb159a640 1 -- 172.21.15.43:0/1449550419 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c032810 con 0x7fbcb40d1fc0 2024-04-25T09:02:31.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.322+0000 7fbcb8b63640 1 -- 172.21.15.43:0/1449550419 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcb40d1fc0 msgr2=0x7fbcb40d2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:31.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.322+0000 7fbcb8b63640 1 --2- 172.21.15.43:0/1449550419 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcb40d1fc0 0x7fbcb40d2420 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fbcb40b7610 tx=0x7fbc9c02a250 comp rx=0 tx=0).stop 2024-04-25T09:02:31.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.322+0000 7fbcb8b63640 1 -- 172.21.15.43:0/1449550419 shutdown_connections 2024-04-25T09:02:31.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.322+0000 7fbcb8b63640 1 --2- 172.21.15.43:0/1449550419 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbcb40b7920 0x7fbcb40b7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:31.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.322+0000 7fbcb8b63640 1 --2- 172.21.15.43:0/1449550419 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcb40d1fc0 0x7fbcb40d2420 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:31.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.322+0000 7fbcb8b63640 1 -- 172.21.15.43:0/1449550419 >> 172.21.15.43:0/1449550419 conn(0x7fbcb40d3a70 msgr2=0x7fbcb40d3e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:31.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.322+0000 7fbcb8b63640 1 -- 172.21.15.43:0/1449550419 shutdown_connections 2024-04-25T09:02:31.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.322+0000 7fbcb8b63640 1 -- 172.21.15.43:0/1449550419 wait complete. 2024-04-25T09:02:31.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.324+0000 7fbcb8b63640 1 Processor -- start 2024-04-25T09:02:31.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.324+0000 7fbcb8b63640 1 -- start start 2024-04-25T09:02:31.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.324+0000 7fbcb8b63640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcb40b7920 0x7fbcb414efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:31.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.324+0000 7fbcb8b63640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcb40d1fc0 0x7fbcb409d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:31.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.324+0000 7fbcb8b63640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbcb415de10 0x7fbcb409dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:31.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.324+0000 7fbcb8b63640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb415e1f0 con 0x7fbcb40b7920 2024-04-25T09:02:31.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.324+0000 7fbcb8b63640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb40b7120 con 0x7fbcb40d1fc0 2024-04-25T09:02:31.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.324+0000 7fbcb8b63640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb40cf810 con 0x7fbcb415de10 2024-04-25T09:02:31.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.325+0000 7fbcb259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcb40b7920 0x7fbcb414efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:31.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.325+0000 7fbcb259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcb40b7920 0x7fbcb414efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43754/0 (socket says 172.21.15.43:43754) 2024-04-25T09:02:31.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.325+0000 7fbcb259c640 1 -- 172.21.15.43:0/1879541633 learned_addr learned my addr 172.21.15.43:0/1879541633 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:31.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.325+0000 7fbcb2d9d640 1 --2- 172.21.15.43:0/1879541633 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbcb415de10 0x7fbcb409dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:31.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.325+0000 7fbcb1d9b640 1 --2- 172.21.15.43:0/1879541633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcb40d1fc0 0x7fbcb409d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:31.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.325+0000 7fbcb259c640 1 -- 172.21.15.43:0/1879541633 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbcb415de10 msgr2=0x7fbcb409dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:31.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.325+0000 7fbcb259c640 1 --2- 172.21.15.43:0/1879541633 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbcb415de10 0x7fbcb409dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:31.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.325+0000 7fbcb259c640 1 -- 172.21.15.43:0/1879541633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcb40d1fc0 msgr2=0x7fbcb409d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:31.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.325+0000 7fbcb259c640 1 --2- 172.21.15.43:0/1879541633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcb40d1fc0 0x7fbcb409d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:31.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.325+0000 7fbcb259c640 1 -- 172.21.15.43:0/1879541633 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb415a3b0 con 0x7fbcb40b7920 2024-04-25T09:02:31.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.325+0000 7fbcb259c640 1 --2- 172.21.15.43:0/1879541633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcb40b7920 0x7fbcb414efc0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fbcac00c430 tx=0x7fbcac00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:31.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.325+0000 7fbcaaffd640 1 -- 172.21.15.43:0/1879541633 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcac0074e0 con 0x7fbcb40b7920 2024-04-25T09:02:31.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.325+0000 7fbcaaffd640 1 -- 172.21.15.43:0/1879541633 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcac0047e0 con 0x7fbcb40b7920 2024-04-25T09:02:31.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.325+0000 7fbcb8b63640 1 -- 172.21.15.43:0/1879541633 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcb409e320 con 0x7fbcb40b7920 2024-04-25T09:02:31.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.325+0000 7fbcaaffd640 1 -- 172.21.15.43:0/1879541633 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcac004bd0 con 0x7fbcb40b7920 2024-04-25T09:02:31.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.325+0000 7fbcb8b63640 1 -- 172.21.15.43:0/1879541633 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcb409e8e0 con 0x7fbcb40b7920 2024-04-25T09:02:31.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.326+0000 7fbcb8b63640 1 -- 172.21.15.43:0/1879541633 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcb40680f0 con 0x7fbcb40b7920 2024-04-25T09:02:31.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.327+0000 7fbcaaffd640 1 -- 172.21.15.43:0/1879541633 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbcac011070 con 0x7fbcb40b7920 2024-04-25T09:02:31.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.327+0000 7fbcaaffd640 1 --2- 172.21.15.43:0/1879541633 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbc9406b4d0 0x7fbc9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:31.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.327+0000 7fbcaaffd640 1 -- 172.21.15.43:0/1879541633 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbcac089d30 con 0x7fbcb40b7920 2024-04-25T09:02:31.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.327+0000 7fbcb1d9b640 1 --2- 172.21.15.43:0/1879541633 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbc9406b4d0 0x7fbc9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:31.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.328+0000 7fbcb1d9b640 1 --2- 172.21.15.43:0/1879541633 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbc9406b4d0 0x7fbc9406d990 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c034000 tx=0x7fbc9c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:31.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.331+0000 7fbcaaffd640 1 -- 172.21.15.43:0/1879541633 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbcac058560 con 0x7fbcb40b7920 2024-04-25T09:02:31.566 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.578+0000 7fbcb8b63640 1 -- 172.21.15.43:0/1879541633 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fbcb4067270 con 0x7fbcb40b7920 2024-04-25T09:02:31.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.580+0000 7fbcaaffd640 1 -- 172.21.15.43:0/1879541633 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+147 (secure 0 0 0) 0x7fbcac05c600 con 0x7fbcb40b7920 2024-04-25T09:02:31.569 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2024-04-25T09:02:31.569 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQD8GSpmVOjWGxAAFNmxeGi0h2D//pa6Ne3eQw== 2024-04-25T09:02:31.569 INFO:teuthology.orchestra.run.smithi043.stdout: caps mds = "allow" 2024-04-25T09:02:31.569 INFO:teuthology.orchestra.run.smithi043.stdout: caps mon = "allow r" 2024-04-25T09:02:31.569 INFO:teuthology.orchestra.run.smithi043.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T09:02:31.569 INFO:teuthology.orchestra.run.smithi043.stderr:updated caps for client.0 2024-04-25T09:02:31.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.583+0000 7fbcb8b63640 1 -- 172.21.15.43:0/1879541633 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbc9406b4d0 msgr2=0x7fbc9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:31.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.583+0000 7fbcb8b63640 1 --2- 172.21.15.43:0/1879541633 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbc9406b4d0 0x7fbc9406d990 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c034000 tx=0x7fbc9c02e000 comp rx=0 tx=0).stop 2024-04-25T09:02:31.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.583+0000 7fbcb8b63640 1 -- 172.21.15.43:0/1879541633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcb40b7920 msgr2=0x7fbcb414efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:31.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.583+0000 7fbcb8b63640 1 --2- 172.21.15.43:0/1879541633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcb40b7920 0x7fbcb414efc0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7fbcac00c430 tx=0x7fbcac00c8f0 comp rx=0 tx=0).stop 2024-04-25T09:02:31.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.583+0000 7fbcb8b63640 1 -- 172.21.15.43:0/1879541633 shutdown_connections 2024-04-25T09:02:31.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.583+0000 7fbcb8b63640 1 --2- 172.21.15.43:0/1879541633 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbcb415de10 0x7fbcb409dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:31.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.583+0000 7fbcb8b63640 1 --2- 172.21.15.43:0/1879541633 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbc9406b4d0 0x7fbc9406d990 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:31.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.583+0000 7fbcb8b63640 1 --2- 172.21.15.43:0/1879541633 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcb40d1fc0 0x7fbcb409d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:31.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.583+0000 7fbcb8b63640 1 --2- 172.21.15.43:0/1879541633 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcb40b7920 0x7fbcb414efc0 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:31.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.583+0000 7fbcb8b63640 1 -- 172.21.15.43:0/1879541633 >> 172.21.15.43:0/1879541633 conn(0x7fbcb40d3a70 msgr2=0x7fbcb40b7db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:31.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.583+0000 7fbcb8b63640 1 -- 172.21.15.43:0/1879541633 shutdown_connections 2024-04-25T09:02:31.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.583+0000 7fbcb8b63640 1 -- 172.21.15.43:0/1879541633 wait complete. 2024-04-25T09:02:31.583 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:02:31.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.696+0000 7fcedb612640 1 Processor -- start 2024-04-25T09:02:31.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.696+0000 7fcedb612640 1 -- start start 2024-04-25T09:02:31.686 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.697+0000 7fcedb612640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fced40d0af0 0x7fced4160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:31.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.697+0000 7fcedb612640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fced40d2920 con 0x7fced415dce0 2024-04-25T09:02:31.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.697+0000 7fcedb612640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fced40cee80 con 0x7fced415ef50 2024-04-25T09:02:31.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.697+0000 7fcedb612640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fced40cec60 con 0x7fced40d0af0 2024-04-25T09:02:31.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.697+0000 7fced93ae640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fced415dce0 0x7fced40ce550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48400/0 (socket says 172.21.15.43:48400) 2024-04-25T09:02:31.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.697+0000 7fced93ae640 1 -- 172.21.15.43:0/2415312646 learned_addr learned my addr 172.21.15.43:0/2415312646 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:31.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.697+0000 7fced9baf640 1 --2- 172.21.15.43:0/2415312646 >> v2:172.21.15.193:3300/0 conn(0x7fced40d0af0 0x7fced4160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:31.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.697+0000 7fcecbfff640 1 -- 172.21.15.43:0/2415312646 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 494578667 0 0) 0x7fced40d2920 con 0x7fced415dce0 2024-04-25T09:02:31.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.697+0000 7fcecbfff640 1 -- 172.21.15.43:0/2415312646 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcec0003520 con 0x7fced415dce0 2024-04-25T09:02:31.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.697+0000 7fcecbfff640 1 -- 172.21.15.43:0/2415312646 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 799122704 0 0) 0x7fced40cee80 con 0x7fced415ef50 2024-04-25T09:02:31.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.697+0000 7fcecbfff640 1 -- 172.21.15.43:0/2415312646 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fced40d2920 con 0x7fced415ef50 2024-04-25T09:02:31.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.697+0000 7fced9baf640 1 -- 172.21.15.43:0/2415312646 >> v1:172.21.15.193:6789/0 conn(0x7fced415ef50 legacy=0x7fced40d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:31.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.697+0000 7fcecbfff640 1 -- 172.21.15.43:0/2415312646 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1699118517 0 0) 0x7fcec0003520 con 0x7fced415dce0 2024-04-25T09:02:31.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.697+0000 7fced9baf640 1 -- 172.21.15.43:0/2415312646 >> v1:172.21.15.43:6789/0 conn(0x7fced415dce0 legacy=0x7fced40ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:31.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.697+0000 7fced9baf640 1 -- 172.21.15.43:0/2415312646 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced40cf160 con 0x7fced40d0af0 2024-04-25T09:02:31.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.698+0000 7fced9baf640 1 --2- 172.21.15.43:0/2415312646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fced40d0af0 0x7fced4160980 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fcecc009d90 tx=0x7fcecc02a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cb469c41e488c96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:31.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.698+0000 7fcecbfff640 1 -- 172.21.15.43:0/2415312646 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced0001690 con 0x7fced40d0af0 2024-04-25T09:02:31.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.698+0000 7fcecbfff640 1 -- 172.21.15.43:0/2415312646 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcecc007540 con 0x7fced40d0af0 2024-04-25T09:02:31.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.698+0000 7fcecbfff640 1 -- 172.21.15.43:0/2415312646 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc007910 con 0x7fced40d0af0 2024-04-25T09:02:31.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.699+0000 7fcedb612640 1 -- 172.21.15.43:0/2415312646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fced40d0af0 msgr2=0x7fced4160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:31.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.699+0000 7fcedb612640 1 --2- 172.21.15.43:0/2415312646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fced40d0af0 0x7fced4160980 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fcecc009d90 tx=0x7fcecc02a5f0 comp rx=0 tx=0).stop 2024-04-25T09:02:31.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.699+0000 7fcedb612640 1 -- 172.21.15.43:0/2415312646 shutdown_connections 2024-04-25T09:02:31.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.699+0000 7fcedb612640 1 --2- 172.21.15.43:0/2415312646 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fced40d0af0 0x7fced4160980 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:31.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.699+0000 7fcedb612640 1 -- 172.21.15.43:0/2415312646 >> 172.21.15.43:0/2415312646 conn(0x7fced409cd00 msgr2=0x7fced40d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:31.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.699+0000 7fcedb612640 1 -- 172.21.15.43:0/2415312646 shutdown_connections 2024-04-25T09:02:31.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.699+0000 7fcedb612640 1 -- 172.21.15.43:0/2415312646 wait complete. 2024-04-25T09:02:31.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.700+0000 7fcedb612640 1 Processor -- start 2024-04-25T09:02:31.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.700+0000 7fcedb612640 1 -- start start 2024-04-25T09:02:31.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.701+0000 7fcedb612640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fced40d0af0 0x7fced4154f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:31.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.701+0000 7fcedb612640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fced415dce0 0x7fced415c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:31.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.701+0000 7fcedb612640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fced415ef50 0x7fced4157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:31.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.701+0000 7fcedb612640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced415f6b0 con 0x7fced415ef50 2024-04-25T09:02:31.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.701+0000 7fcedb612640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced40cf720 con 0x7fced40d0af0 2024-04-25T09:02:31.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.701+0000 7fcedb612640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fced415e0c0 con 0x7fced415dce0 2024-04-25T09:02:31.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.701+0000 7fced9baf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fced415ef50 0x7fced4157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:31.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.701+0000 7fced9baf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fced415ef50 0x7fced4157540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43762/0 (socket says 172.21.15.43:43762) 2024-04-25T09:02:31.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.701+0000 7fced9baf640 1 -- 172.21.15.43:0/1942588098 learned_addr learned my addr 172.21.15.43:0/1942588098 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:31.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.701+0000 7fced93ae640 1 --2- 172.21.15.43:0/1942588098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fced40d0af0 0x7fced4154f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:31.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.701+0000 7fced8bad640 1 --2- 172.21.15.43:0/1942588098 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fced415dce0 0x7fced415c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:31.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.701+0000 7fced9baf640 1 -- 172.21.15.43:0/1942588098 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fced415dce0 msgr2=0x7fced415c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:31.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.701+0000 7fced9baf640 1 --2- 172.21.15.43:0/1942588098 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fced415dce0 0x7fced415c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:31.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.701+0000 7fced9baf640 1 -- 172.21.15.43:0/1942588098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fced40d0af0 msgr2=0x7fced4154f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:31.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.701+0000 7fced9baf640 1 --2- 172.21.15.43:0/1942588098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fced40d0af0 0x7fced4154f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:31.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.701+0000 7fced9baf640 1 -- 172.21.15.43:0/1942588098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced4156290 con 0x7fced415ef50 2024-04-25T09:02:31.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.701+0000 7fced93ae640 1 --2- 172.21.15.43:0/1942588098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fced40d0af0 0x7fced4154f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:31.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.702+0000 7fced9baf640 1 --2- 172.21.15.43:0/1942588098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fced415ef50 0x7fced4157540 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fcecc009d90 tx=0x7fcecc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:31.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.702+0000 7fcec9ffb640 1 -- 172.21.15.43:0/1942588098 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc040070 con 0x7fced415ef50 2024-04-25T09:02:31.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.702+0000 7fcec9ffb640 1 -- 172.21.15.43:0/1942588098 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcecc031bc0 con 0x7fced415ef50 2024-04-25T09:02:31.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.702+0000 7fcedb612640 1 -- 172.21.15.43:0/1942588098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fced4156570 con 0x7fced415ef50 2024-04-25T09:02:31.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.702+0000 7fcec9ffb640 1 -- 172.21.15.43:0/1942588098 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc007e90 con 0x7fced415ef50 2024-04-25T09:02:31.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.702+0000 7fcedb612640 1 -- 172.21.15.43:0/1942588098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fced4156b30 con 0x7fced415ef50 2024-04-25T09:02:31.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.703+0000 7fcedb612640 1 -- 172.21.15.43:0/1942588098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fced40680f0 con 0x7fced415ef50 2024-04-25T09:02:31.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.703+0000 7fcec9ffb640 1 -- 172.21.15.43:0/1942588098 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcecc039040 con 0x7fced415ef50 2024-04-25T09:02:31.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.704+0000 7fcec9ffb640 1 --2- 172.21.15.43:0/1942588098 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fceb806b4d0 0x7fceb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:31.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.704+0000 7fcec9ffb640 1 -- 172.21.15.43:0/1942588098 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcecc0ab560 con 0x7fced415ef50 2024-04-25T09:02:31.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.704+0000 7fced93ae640 1 --2- 172.21.15.43:0/1942588098 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fceb806b4d0 0x7fceb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:31.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.705+0000 7fced93ae640 1 --2- 172.21.15.43:0/1942588098 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fceb806b4d0 0x7fceb806d990 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fced0001800 tx=0x7fced0001570 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:31.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.708+0000 7fcec9ffb640 1 -- 172.21.15.43:0/1942588098 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcecc04b340 con 0x7fced415ef50 2024-04-25T09:02:31.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.924+0000 7fcedb612640 1 -- 172.21.15.43:0/1942588098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fced409d1a0 con 0x7fced415ef50 2024-04-25T09:02:31.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.924+0000 7fcec9ffb640 1 -- 172.21.15.43:0/1942588098 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fcecc07a470 con 0x7fced415ef50 2024-04-25T09:02:31.913 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:31.913 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":69,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6821,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2021228703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2021228703},{"type":"v1","addr":"172.21.15.193:6837","nonce":2021228703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65},{"gid":7543,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/2398324998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2398324998},{"type":"v1","addr":"172.21.15.193:6836","nonce":2398324998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65},{"gid":8673,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2298187831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2298187831},{"type":"v1","addr":"172.21.15.43:6834","nonce":2298187831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:02:26.161487+0000","modified":"2024-04-25T09:02:29.177294+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8676":{"gid":8676,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/1586865031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1586865031},{"type":"v1","addr":"172.21.15.43:6835","nonce":1586865031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:02:31.914 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 69 2024-04-25T09:02:31.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.926+0000 7fcedb612640 1 -- 172.21.15.43:0/1942588098 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fceb806b4d0 msgr2=0x7fceb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:31.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.926+0000 7fcedb612640 1 --2- 172.21.15.43:0/1942588098 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fceb806b4d0 0x7fceb806d990 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fced0001800 tx=0x7fced0001570 comp rx=0 tx=0).stop 2024-04-25T09:02:31.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.926+0000 7fcedb612640 1 -- 172.21.15.43:0/1942588098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fced415ef50 msgr2=0x7fced4157540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:31.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.926+0000 7fcedb612640 1 --2- 172.21.15.43:0/1942588098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fced415ef50 0x7fced4157540 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fcecc009d90 tx=0x7fcecc004a40 comp rx=0 tx=0).stop 2024-04-25T09:02:31.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.926+0000 7fcedb612640 1 -- 172.21.15.43:0/1942588098 shutdown_connections 2024-04-25T09:02:31.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.926+0000 7fcedb612640 1 --2- 172.21.15.43:0/1942588098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fced415ef50 0x7fced4157540 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:31.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.926+0000 7fcedb612640 1 --2- 172.21.15.43:0/1942588098 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fced415dce0 0x7fced415c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:31.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.926+0000 7fcedb612640 1 --2- 172.21.15.43:0/1942588098 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fceb806b4d0 0x7fceb806d990 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:31.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.926+0000 7fcedb612640 1 --2- 172.21.15.43:0/1942588098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fced40d0af0 0x7fced4154f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:31.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.926+0000 7fcedb612640 1 -- 172.21.15.43:0/1942588098 >> 172.21.15.43:0/1942588098 conn(0x7fced409cd00 msgr2=0x7fced40c7540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:31.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.926+0000 7fcedb612640 1 -- 172.21.15.43:0/1942588098 shutdown_connections 2024-04-25T09:02:31.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:31.926+0000 7fcedb612640 1 -- 172.21.15.43:0/1942588098 wait complete. 2024-04-25T09:02:32.332 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 69, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:02:26.161487+0000', 'modified': '2024-04-25T09:02:29.177294+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8676}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8676': {'gid': 8676, 'name': 'c', 'rank': 0, 'incarnation': 66, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.43:6835/1586865031', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6833', 'nonce': 1586865031}, {'type': 'v1', 'addr': '172.21.15.43:6835', 'nonce': 1586865031}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:02:32.332 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:02:32.332 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T09:02:32.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.450+0000 7f71de24d640 1 Processor -- start 2024-04-25T09:02:32.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.450+0000 7f71de24d640 1 -- start start 2024-04-25T09:02:32.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.450+0000 7f71de24d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f71d80ccbb0 0x7f71d80cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:32.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.450+0000 7f71de24d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71d80b2780 con 0x7f71d80b3b00 2024-04-25T09:02:32.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.450+0000 7f71de24d640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71d80b2a60 con 0x7f71d80cbfc0 2024-04-25T09:02:32.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.450+0000 7f71de24d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71d80cd550 con 0x7f71d80ccbb0 2024-04-25T09:02:32.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.451+0000 7f71d6ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f71d80ccbb0 0x7f71d80cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:32.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.451+0000 7f71d7fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f71d80b3b00 0x7f71d80b3f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48404/0 (socket says 172.21.15.43:48404) 2024-04-25T09:02:32.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.451+0000 7f71d7fff640 1 -- 172.21.15.43:0/156597707 learned_addr learned my addr 172.21.15.43:0/156597707 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:32.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.451+0000 7f71d6ffd640 1 -- 172.21.15.43:0/156597707 >> v1:172.21.15.193:6789/0 conn(0x7f71d80cbfc0 legacy=0x7f71d806dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:32.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.451+0000 7f71d6ffd640 1 -- 172.21.15.43:0/156597707 >> v1:172.21.15.43:6789/0 conn(0x7f71d80b3b00 legacy=0x7f71d80b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:32.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.451+0000 7f71d67fc640 1 -- 172.21.15.43:0/156597707 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3761182661 0 0) 0x7f71d80b2780 con 0x7f71d80b3b00 2024-04-25T09:02:32.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.451+0000 7f71d6ffd640 1 -- 172.21.15.43:0/156597707 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d80b2d40 con 0x7f71d80ccbb0 2024-04-25T09:02:32.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.452+0000 7f71d6ffd640 1 --2- 172.21.15.43:0/156597707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71d80ccbb0 0x7f71d80cd010 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f71cc0097a0 tx=0x7f71cc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53167f4b87a5ca26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:32.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.452+0000 7f71d67fc640 1 -- 172.21.15.43:0/156597707 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71cc037070 con 0x7f71d80ccbb0 2024-04-25T09:02:32.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.452+0000 7f71d67fc640 1 -- 172.21.15.43:0/156597707 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71cc032440 con 0x7f71d80ccbb0 2024-04-25T09:02:32.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.452+0000 7f71d67fc640 1 -- 172.21.15.43:0/156597707 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71cc032810 con 0x7f71d80ccbb0 2024-04-25T09:02:32.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.452+0000 7f71de24d640 1 -- 172.21.15.43:0/156597707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71d80ccbb0 msgr2=0x7f71d80cd010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:32.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.452+0000 7f71de24d640 1 --2- 172.21.15.43:0/156597707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71d80ccbb0 0x7f71d80cd010 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f71cc0097a0 tx=0x7f71cc02a1e0 comp rx=0 tx=0).stop 2024-04-25T09:02:32.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.453+0000 7f71de24d640 1 -- 172.21.15.43:0/156597707 shutdown_connections 2024-04-25T09:02:32.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.453+0000 7f71de24d640 1 --2- 172.21.15.43:0/156597707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71d80ccbb0 0x7f71d80cd010 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:32.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.453+0000 7f71de24d640 1 -- 172.21.15.43:0/156597707 >> 172.21.15.43:0/156597707 conn(0x7f71d80b1480 msgr2=0x7f71d80b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:32.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.453+0000 7f71de24d640 1 -- 172.21.15.43:0/156597707 shutdown_connections 2024-04-25T09:02:32.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.453+0000 7f71de24d640 1 -- 172.21.15.43:0/156597707 wait complete. 2024-04-25T09:02:32.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.454+0000 7f71de24d640 1 Processor -- start 2024-04-25T09:02:32.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.454+0000 7f71de24d640 1 -- start start 2024-04-25T09:02:32.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.454+0000 7f71de24d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71d80b3b00 0x7f71d815c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:32.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.454+0000 7f71de24d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71d80cbfc0 0x7f71d815e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:32.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.455+0000 7f71de24d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f71d80ccbb0 0x7f71d80ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:32.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.455+0000 7f71de24d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d80b35e0 con 0x7f71d80b3b00 2024-04-25T09:02:32.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.455+0000 7f71de24d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d80b3300 con 0x7f71d80cbfc0 2024-04-25T09:02:32.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.455+0000 7f71de24d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71d80cc3a0 con 0x7f71d80ccbb0 2024-04-25T09:02:32.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.455+0000 7f71d77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71d80b3b00 0x7f71d815c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:32.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.455+0000 7f71d77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71d80b3b00 0x7f71d815c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43782/0 (socket says 172.21.15.43:43782) 2024-04-25T09:02:32.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.455+0000 7f71d77fe640 1 -- 172.21.15.43:0/3321871364 learned_addr learned my addr 172.21.15.43:0/3321871364 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:32.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.455+0000 7f71d6ffd640 1 --2- 172.21.15.43:0/3321871364 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71d80cbfc0 0x7f71d815e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:32.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.455+0000 7f71d7fff640 1 --2- 172.21.15.43:0/3321871364 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f71d80ccbb0 0x7f71d80ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:32.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.455+0000 7f71d77fe640 1 -- 172.21.15.43:0/3321871364 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f71d80ccbb0 msgr2=0x7f71d80ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:32.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.455+0000 7f71d77fe640 1 --2- 172.21.15.43:0/3321871364 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f71d80ccbb0 0x7f71d80ad8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:32.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.455+0000 7f71d77fe640 1 -- 172.21.15.43:0/3321871364 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71d80cbfc0 msgr2=0x7f71d815e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:32.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.455+0000 7f71d77fe640 1 --2- 172.21.15.43:0/3321871364 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71d80cbfc0 0x7f71d815e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:32.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.455+0000 7f71d77fe640 1 -- 172.21.15.43:0/3321871364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d8132150 con 0x7f71d80b3b00 2024-04-25T09:02:32.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.455+0000 7f71d6ffd640 1 --2- 172.21.15.43:0/3321871364 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71d80cbfc0 0x7f71d815e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:32.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.455+0000 7f71d7fff640 1 --2- 172.21.15.43:0/3321871364 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f71d80ccbb0 0x7f71d80ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:32.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.455+0000 7f71d77fe640 1 --2- 172.21.15.43:0/3321871364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71d80b3b00 0x7f71d815c7f0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f71c800b420 tx=0x7f71c800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:32.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.455+0000 7f71bffff640 1 -- 172.21.15.43:0/3321871364 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c8011830 con 0x7f71d80b3b00 2024-04-25T09:02:32.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.455+0000 7f71bffff640 1 -- 172.21.15.43:0/3321871364 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c8016070 con 0x7f71d80b3b00 2024-04-25T09:02:32.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.455+0000 7f71de24d640 1 -- 172.21.15.43:0/3321871364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71d81315d0 con 0x7f71d80b3b00 2024-04-25T09:02:32.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.455+0000 7f71bffff640 1 -- 172.21.15.43:0/3321871364 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c8011c10 con 0x7f71d80b3b00 2024-04-25T09:02:32.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.455+0000 7f71de24d640 1 -- 172.21.15.43:0/3321871364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71d8131b90 con 0x7f71d80b3b00 2024-04-25T09:02:32.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.457+0000 7f71bffff640 1 -- 172.21.15.43:0/3321871364 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f71c801b460 con 0x7f71d80b3b00 2024-04-25T09:02:32.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.457+0000 7f71bffff640 1 --2- 172.21.15.43:0/3321871364 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f71c406b4d0 0x7f71c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:32.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.457+0000 7f71bffff640 1 -- 172.21.15.43:0/3321871364 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f71c8089f60 con 0x7f71d80b3b00 2024-04-25T09:02:32.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.458+0000 7f71d6ffd640 1 --2- 172.21.15.43:0/3321871364 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f71c406b4d0 0x7f71c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:32.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.458+0000 7f71d6ffd640 1 --2- 172.21.15.43:0/3321871364 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f71c406b4d0 0x7f71c406d990 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f71cc034000 tx=0x7f71cc02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:32.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.458+0000 7f71e102c740 1 Processor -- start 2024-04-25T09:02:32.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.459+0000 7f71e102c740 1 -- start start 2024-04-25T09:02:32.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.459+0000 7f71e102c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589ff2a0230 0x5589ff2a0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:32.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.459+0000 7f71e102c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589ff2bdad0 0x5589ff2be020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:32.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.459+0000 7f71e102c740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5589ff275f10 0x5589ff2bc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:32.449 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.459+0000 7f71e102c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589ff1d8a10 con 0x5589ff2bdad0 2024-04-25T09:02:32.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.459+0000 7f71e102c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589ff181b30 con 0x5589ff2a0230 2024-04-25T09:02:32.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.459+0000 7f71e102c740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x5589ff1d4c50 con 0x5589ff275f10 2024-04-25T09:02:32.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.460+0000 7f71d6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589ff2bdad0 0x5589ff2be020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:32.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.460+0000 7f71d6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589ff2bdad0 0x5589ff2be020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43786/0 (socket says 172.21.15.43:43786) 2024-04-25T09:02:32.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.460+0000 7f71d6ffd640 1 -- 172.21.15.43:0/2875772158 learned_addr learned my addr 172.21.15.43:0/2875772158 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:32.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.460+0000 7f71d7fff640 1 --2- 172.21.15.43:0/2875772158 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5589ff275f10 0x5589ff2bc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:32.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.460+0000 7f71d77fe640 1 --2- 172.21.15.43:0/2875772158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589ff2a0230 0x5589ff2a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:32.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.460+0000 7f71d6ffd640 1 -- 172.21.15.43:0/2875772158 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5589ff275f10 msgr2=0x5589ff2bc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:32.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.460+0000 7f71d6ffd640 1 --2- 172.21.15.43:0/2875772158 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5589ff275f10 0x5589ff2bc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:32.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.460+0000 7f71d6ffd640 1 -- 172.21.15.43:0/2875772158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589ff2a0230 msgr2=0x5589ff2a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:32.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.460+0000 7f71d6ffd640 1 --2- 172.21.15.43:0/2875772158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589ff2a0230 0x5589ff2a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:32.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.460+0000 7f71d6ffd640 1 -- 172.21.15.43:0/2875772158 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589ff196e10 con 0x5589ff2bdad0 2024-04-25T09:02:32.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.460+0000 7f71d6ffd640 1 --2- 172.21.15.43:0/2875772158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589ff2bdad0 0x5589ff2be020 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f71cc02a7d0 tx=0x7f71cc02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9fdd2706611d61d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:32.450 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.460+0000 7f71bd7fa640 1 -- 172.21.15.43:0/2875772158 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71cc04e070 con 0x5589ff2bdad0 2024-04-25T09:02:32.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.460+0000 7f71bd7fa640 1 -- 172.21.15.43:0/2875772158 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71cc049470 con 0x5589ff2bdad0 2024-04-25T09:02:32.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.460+0000 7f71bd7fa640 1 -- 172.21.15.43:0/2875772158 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71cc037040 con 0x5589ff2bdad0 2024-04-25T09:02:32.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.461+0000 7f71e102c740 1 -- 172.21.15.43:0/2875772158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589ff2bdad0 msgr2=0x5589ff2be020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:32.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.461+0000 7f71e102c740 1 --2- 172.21.15.43:0/2875772158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589ff2bdad0 0x5589ff2be020 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f71cc02a7d0 tx=0x7f71cc02c000 comp rx=0 tx=0).stop 2024-04-25T09:02:32.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.461+0000 7f71e102c740 1 -- 172.21.15.43:0/2875772158 shutdown_connections 2024-04-25T09:02:32.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.461+0000 7f71e102c740 1 --2- 172.21.15.43:0/2875772158 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5589ff275f10 0x5589ff2bc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:32.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.461+0000 7f71e102c740 1 --2- 172.21.15.43:0/2875772158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589ff2bdad0 0x5589ff2be020 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:32.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.461+0000 7f71e102c740 1 --2- 172.21.15.43:0/2875772158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589ff2a0230 0x5589ff2a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:32.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.461+0000 7f71e102c740 1 -- 172.21.15.43:0/2875772158 >> 172.21.15.43:0/2875772158 conn(0x5589ff272840 msgr2=0x5589ff2bb040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:32.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.461+0000 7f71e102c740 1 -- 172.21.15.43:0/2875772158 shutdown_connections 2024-04-25T09:02:32.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.461+0000 7f71e102c740 1 -- 172.21.15.43:0/2875772158 wait complete. 2024-04-25T09:02:32.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.462+0000 7f71e102c740 1 -- start start 2024-04-25T09:02:32.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.462+0000 7f71e102c740 1 Processor -- start 2024-04-25T09:02:32.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.462+0000 7f71e102c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589ff2a0230 0x5589ff28e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:32.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.462+0000 7f71e102c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589ff2bdad0 0x5589ff29e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:32.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.462+0000 7f71e102c740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5589ff28ee70 0x5589ff2c1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:32.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.462+0000 7f71e102c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589ff276cb0 con 0x5589ff2a0230 2024-04-25T09:02:32.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.462+0000 7f71e102c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589ff1950c0 con 0x5589ff2bdad0 2024-04-25T09:02:32.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.462+0000 7f71e102c740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x5589ff270770 con 0x5589ff28ee70 2024-04-25T09:02:32.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.462+0000 7f71e102c740 10 client.0 _ll_register_callbacks cb 0x5589ff188ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:02:32.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.462+0000 7f71d77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589ff2a0230 0x5589ff28e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:32.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.463+0000 7f71d77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589ff2a0230 0x5589ff28e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43802/0 (socket says 172.21.15.43:43802) 2024-04-25T09:02:32.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.463+0000 7f71d77fe640 1 -- 172.21.15.43:0/1537294960 learned_addr learned my addr 172.21.15.43:0/1537294960 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:32.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.463+0000 7f71d6ffd640 1 --2- 172.21.15.43:0/1537294960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589ff2bdad0 0x5589ff29e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:32.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.463+0000 7f71d7fff640 1 --2- 172.21.15.43:0/1537294960 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5589ff28ee70 0x5589ff2c1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:32.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.463+0000 7f71d77fe640 1 -- 172.21.15.43:0/1537294960 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5589ff28ee70 msgr2=0x5589ff2c1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:32.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.463+0000 7f71d77fe640 1 --2- 172.21.15.43:0/1537294960 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5589ff28ee70 0x5589ff2c1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:32.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.463+0000 7f71d77fe640 1 -- 172.21.15.43:0/1537294960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589ff2bdad0 msgr2=0x5589ff29e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:32.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.463+0000 7f71d77fe640 1 --2- 172.21.15.43:0/1537294960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589ff2bdad0 0x5589ff29e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:32.457 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.463+0000 7f71d77fe640 1 -- 172.21.15.43:0/1537294960 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5589feeed2d0 con 0x5589ff2a0230 2024-04-25T09:02:32.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.463+0000 7f71d77fe640 1 --2- 172.21.15.43:0/1537294960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589ff2a0230 0x5589ff28e7f0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f71c800bde0 tx=0x7f71c805c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c8935ecd669eb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:32.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.463+0000 7f71b67fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:02:32.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.463+0000 7f71b67fc640 1 -- 172.21.15.43:0/1537294960 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c8070070 con 0x5589ff2a0230 2024-04-25T09:02:32.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.463+0000 7f71b67fc640 1 -- 172.21.15.43:0/1537294960 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f71c8057c90 con 0x5589ff2a0230 2024-04-25T09:02:32.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.463+0000 7f71e102c740 1 -- 172.21.15.43:0/1537294960 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71cc049470 con 0x5589ff2a0230 2024-04-25T09:02:32.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.463+0000 7f71b67fc640 1 -- 172.21.15.43:0/1537294960 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71c80880d0 con 0x5589ff2a0230 2024-04-25T09:02:32.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.463+0000 7f71e102c740 10 client.8814 fetch_fsmap learned FSMap version 69 2024-04-25T09:02:32.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.463+0000 7f71e102c740 1 -- 172.21.15.43:0/1537294960 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x5589fefb76b0 con 0x5589ff2a0230 2024-04-25T09:02:32.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.463+0000 7f71b67fc640 1 -- 172.21.15.43:0/1537294960 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71cc049470 con 0x5589ff2a0230 2024-04-25T09:02:32.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.463+0000 7f71b67fc640 1 -- 172.21.15.43:0/1537294960 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f71c8054390 con 0x5589ff2a0230 2024-04-25T09:02:32.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.464+0000 7f71e102c740 10 client.8814 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T09:02:32.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.464+0000 7f71e102c740 10 client.8814 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:02:32.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.464+0000 7f71e102c740 20 client.8814 populate_metadata read hostname 'smithi043' 2024-04-25T09:02:32.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.464+0000 7f71e102c740 1 --2- 172.21.15.43:0/1537294960 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x5589ff2a2020 0x5589ff2a2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:32.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.464+0000 7f71e102c740 4 client.8814 mds_command: new command op to 8676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:02:32.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.464+0000 7f71e102c740 1 -- 172.21.15.43:0/1537294960 --> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5589ff237920 con 0x5589ff2a2020 2024-04-25T09:02:32.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.464+0000 7f71d6ffd640 1 --2- 172.21.15.43:0/1537294960 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x5589ff2a2020 0x5589ff2a2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:32.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.464+0000 7f71d6ffd640 1 --2- 172.21.15.43:0/1537294960 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x5589ff2a2020 0x5589ff2a2400 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b84404af1211b1c9 server_cookie=c6b83f5bb9b17f46 in_seq=0 out_seq=0 2024-04-25T09:02:32.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.464+0000 7f71b67fc640 10 client.8814 ms_handle_connect on v2:172.21.15.43:6833/1586865031 2024-04-25T09:02:32.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.465+0000 7f71b67fc640 1 -- 172.21.15.43:0/1537294960 <== mds.0 v2:172.21.15.43:6833/1586865031 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5589ff237920 con 0x5589ff2a2020 2024-04-25T09:02:32.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.465+0000 7f71b67fc640 10 client.8814 handle_command_reply: tid=0 2024-04-25T09:02:32.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.465+0000 7f71e102c740 1 client.8814 shutdown 2024-04-25T09:02:32.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.465+0000 7f71e102c740 1 -- 172.21.15.43:0/1537294960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589ff2a0230 msgr2=0x5589ff28e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:32.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.465+0000 7f71e102c740 1 --2- 172.21.15.43:0/1537294960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589ff2a0230 0x5589ff28e7f0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f71c800bde0 tx=0x7f71c805c850 comp rx=0 tx=0).stop 2024-04-25T09:02:32.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.465+0000 7f71e102c740 1 -- 172.21.15.43:0/1537294960 shutdown_connections 2024-04-25T09:02:32.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.465+0000 7f71e102c740 1 --2- 172.21.15.43:0/1537294960 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x5589ff2a2020 0x5589ff2a2400 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:32.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.465+0000 7f71e102c740 1 --2- 172.21.15.43:0/1537294960 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5589ff28ee70 0x5589ff2c1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:32.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.465+0000 7f71e102c740 1 --2- 172.21.15.43:0/1537294960 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589ff2bdad0 0x5589ff29e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:32.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.465+0000 7f71e102c740 1 --2- 172.21.15.43:0/1537294960 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589ff2a0230 0x5589ff28e7f0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:32.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.465+0000 7f71e102c740 1 -- 172.21.15.43:0/1537294960 >> 172.21.15.43:0/1537294960 conn(0x5589ff275f10 msgr2=0x5589ff2be5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:32.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.465+0000 7f71b67fc640 0 client.8814 ms_handle_reset on v2:172.21.15.43:6833/1586865031 2024-04-25T09:02:32.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.465+0000 7f71e102c740 1 -- 172.21.15.43:0/1537294960 shutdown_connections 2024-04-25T09:02:32.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.465+0000 7f71e102c740 1 -- 172.21.15.43:0/1537294960 wait complete. 2024-04-25T09:02:32.460 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.465+0000 7f71e102c740 20 client.8814 trim_cache size 0 max 16384 2024-04-25T09:02:32.469 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.481+0000 7f71e102c740 1 Processor -- start 2024-04-25T09:02:32.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.481+0000 7f71e102c740 1 -- start start 2024-04-25T09:02:32.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.482+0000 7f71e102c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589ff2a0230 0x5589ff2a30d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:32.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.482+0000 7f71e102c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589ff2bdad0 0x5589ff2a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:32.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.482+0000 7f71e102c740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5589ff2a3610 0x5589ff2a3f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:32.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.482+0000 7f71e102c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d81ffc40 con 0x5589ff2a0230 2024-04-25T09:02:32.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.482+0000 7f71e102c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d8130410 con 0x5589ff2bdad0 2024-04-25T09:02:32.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.482+0000 7f71e102c740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71d8143c90 con 0x5589ff2a3610 2024-04-25T09:02:32.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.482+0000 7f71d77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589ff2a0230 0x5589ff2a30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:32.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.482+0000 7f71d77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589ff2a0230 0x5589ff2a30d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43812/0 (socket says 172.21.15.43:43812) 2024-04-25T09:02:32.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.482+0000 7f71d77fe640 1 -- 172.21.15.43:0/2679935424 learned_addr learned my addr 172.21.15.43:0/2679935424 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:32.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.482+0000 7f71d7fff640 1 --2- 172.21.15.43:0/2679935424 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5589ff2a3610 0x5589ff2a3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:32.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.482+0000 7f71d77fe640 1 -- 172.21.15.43:0/2679935424 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5589ff2a3610 msgr2=0x5589ff2a3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:32.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.482+0000 7f71d77fe640 1 --2- 172.21.15.43:0/2679935424 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5589ff2a3610 0x5589ff2a3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:32.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.482+0000 7f71d6ffd640 1 --2- 172.21.15.43:0/2679935424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589ff2bdad0 0x5589ff2a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:32.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.482+0000 7f71d77fe640 1 -- 172.21.15.43:0/2679935424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589ff2bdad0 msgr2=0x5589ff2a3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:32.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.482+0000 7f71d77fe640 1 --2- 172.21.15.43:0/2679935424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589ff2bdad0 0x5589ff2a3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:32.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.482+0000 7f71d77fe640 1 -- 172.21.15.43:0/2679935424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589ff2cf5a0 con 0x5589ff2a0230 2024-04-25T09:02:32.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.482+0000 7f71d6ffd640 1 --2- 172.21.15.43:0/2679935424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589ff2bdad0 0x5589ff2a3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:32.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.482+0000 7f71d77fe640 1 --2- 172.21.15.43:0/2679935424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589ff2a0230 0x5589ff2a30d0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f71c8013010 tx=0x7f71c805b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba055801819700ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:32.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.482+0000 7f71bcff9640 1 -- 172.21.15.43:0/2679935424 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c8070040 con 0x5589ff2a0230 2024-04-25T09:02:32.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.482+0000 7f71bcff9640 1 -- 172.21.15.43:0/2679935424 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c8057a90 con 0x5589ff2a0230 2024-04-25T09:02:32.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.482+0000 7f71bcff9640 1 -- 172.21.15.43:0/2679935424 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c8057500 con 0x5589ff2a0230 2024-04-25T09:02:32.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.483+0000 7f71e102c740 1 -- 172.21.15.43:0/2679935424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589ff2a0230 msgr2=0x5589ff2a30d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:32.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.483+0000 7f71e102c740 1 --2- 172.21.15.43:0/2679935424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589ff2a0230 0x5589ff2a30d0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f71c8013010 tx=0x7f71c805b000 comp rx=0 tx=0).stop 2024-04-25T09:02:32.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.483+0000 7f71e102c740 1 -- 172.21.15.43:0/2679935424 shutdown_connections 2024-04-25T09:02:32.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.483+0000 7f71e102c740 1 --2- 172.21.15.43:0/2679935424 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5589ff2a3610 0x5589ff2a3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:32.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.483+0000 7f71e102c740 1 --2- 172.21.15.43:0/2679935424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589ff2bdad0 0x5589ff2a3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:32.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.483+0000 7f71e102c740 1 --2- 172.21.15.43:0/2679935424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589ff2a0230 0x5589ff2a30d0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:32.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.483+0000 7f71e102c740 1 -- 172.21.15.43:0/2679935424 >> 172.21.15.43:0/2679935424 conn(0x5589ff275f10 msgr2=0x5589ff28e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:32.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.483+0000 7f71e102c740 1 -- 172.21.15.43:0/2679935424 shutdown_connections 2024-04-25T09:02:32.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.483+0000 7f71e102c740 1 -- 172.21.15.43:0/2679935424 wait complete. 2024-04-25T09:02:32.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.484+0000 7f71e102c740 1 -- start start 2024-04-25T09:02:32.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.484+0000 7f71e102c740 1 Processor -- start 2024-04-25T09:02:32.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.485+0000 7f71e102c740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589ff2a0230 0x5589ff26f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:32.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.485+0000 7f71e102c740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5589ff2a3610 0x5589ff2b9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:32.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.485+0000 7f71e102c740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589ff2bdad0 0x5589ff2b1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:32.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.485+0000 7f71e102c740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589ff2a1420 con 0x5589ff2a0230 2024-04-25T09:02:32.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.485+0000 7f71e102c740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d812fe10 con 0x5589ff2bdad0 2024-04-25T09:02:32.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.485+0000 7f71e102c740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x5589ff2a2ca0 con 0x5589ff2a3610 2024-04-25T09:02:32.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.485+0000 7f71e102c740 10 client.0 _ll_register_callbacks cb 0x5589ff270c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:02:32.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.485+0000 7f71d77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589ff2a0230 0x5589ff26f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:32.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.485+0000 7f71d77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589ff2a0230 0x5589ff26f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43824/0 (socket says 172.21.15.43:43824) 2024-04-25T09:02:32.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.485+0000 7f71d77fe640 1 -- 172.21.15.43:0/2783099776 learned_addr learned my addr 172.21.15.43:0/2783099776 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:32.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.485+0000 7f71d7fff640 1 --2- 172.21.15.43:0/2783099776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589ff2bdad0 0x5589ff2b1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:32.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.485+0000 7f71d6ffd640 1 --2- 172.21.15.43:0/2783099776 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5589ff2a3610 0x5589ff2b9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:32.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.485+0000 7f71d77fe640 1 -- 172.21.15.43:0/2783099776 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5589ff2a3610 msgr2=0x5589ff2b9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:32.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.485+0000 7f71d77fe640 1 --2- 172.21.15.43:0/2783099776 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5589ff2a3610 0x5589ff2b9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:32.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.485+0000 7f71d77fe640 1 -- 172.21.15.43:0/2783099776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589ff2bdad0 msgr2=0x5589ff2b1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:32.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.485+0000 7f71d77fe640 1 --2- 172.21.15.43:0/2783099776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589ff2bdad0 0x5589ff2b1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:32.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.485+0000 7f71d77fe640 1 -- 172.21.15.43:0/2783099776 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5589ff2ba210 con 0x5589ff2a0230 2024-04-25T09:02:32.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.485+0000 7f71d7fff640 1 --2- 172.21.15.43:0/2783099776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589ff2bdad0 0x5589ff2b1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:32.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.485+0000 7f71d6ffd640 1 --2- 172.21.15.43:0/2783099776 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5589ff2a3610 0x5589ff2b9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:32.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.485+0000 7f71d77fe640 1 --2- 172.21.15.43:0/2783099776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589ff2a0230 0x5589ff26f5a0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f71c8054aa0 tx=0x7f71c80882c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55997a2bc7e3331f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:32.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.485+0000 7f71b67fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:02:32.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.485+0000 7f71b67fc640 1 -- 172.21.15.43:0/2783099776 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c807b030 con 0x5589ff2a0230 2024-04-25T09:02:32.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.485+0000 7f71b67fc640 1 -- 172.21.15.43:0/2783099776 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f71c8057710 con 0x5589ff2a0230 2024-04-25T09:02:32.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.485+0000 7f71e102c740 1 -- 172.21.15.43:0/2783099776 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71c8057a90 con 0x5589ff2a0230 2024-04-25T09:02:32.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.485+0000 7f71b67fc640 1 -- 172.21.15.43:0/2783099776 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71c8070040 con 0x5589ff2a0230 2024-04-25T09:02:32.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.485+0000 7f71e102c740 10 client.8820 fetch_fsmap learned FSMap version 69 2024-04-25T09:02:32.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.485+0000 7f71e102c740 1 -- 172.21.15.43:0/2783099776 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x5589ff2b1ef0 con 0x5589ff2a0230 2024-04-25T09:02:32.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.486+0000 7f71b67fc640 1 -- 172.21.15.43:0/2783099776 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71c8057a90 con 0x5589ff2a0230 2024-04-25T09:02:32.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.486+0000 7f71b67fc640 1 -- 172.21.15.43:0/2783099776 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f71c805c160 con 0x5589ff2a0230 2024-04-25T09:02:32.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.486+0000 7f71e102c740 10 client.8820 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T09:02:32.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.486+0000 7f71e102c740 10 client.8820 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T09:02:32.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.486+0000 7f71e102c740 20 client.8820 populate_metadata read hostname 'smithi043' 2024-04-25T09:02:32.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.486+0000 7f71e102c740 1 --2- 172.21.15.43:0/2783099776 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x5589ff2a3d40 0x5589ff331160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:32.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.486+0000 7f71e102c740 4 client.8820 mds_command: new command op to 8676 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:02:32.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.486+0000 7f71e102c740 1 -- 172.21.15.43:0/2783099776 --> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5589ff1e5f50 con 0x5589ff2a3d40 2024-04-25T09:02:32.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.486+0000 7f71d6ffd640 1 --2- 172.21.15.43:0/2783099776 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x5589ff2a3d40 0x5589ff331160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:32.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.486+0000 7f71d6ffd640 1 --2- 172.21.15.43:0/2783099776 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x5589ff2a3d40 0x5589ff331160 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c00ad5190e48516 server_cookie=d0e56612458e5f9d in_seq=0 out_seq=0 2024-04-25T09:02:32.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.486+0000 7f71b67fc640 10 client.8820 ms_handle_connect on v2:172.21.15.43:6833/1586865031 2024-04-25T09:02:32.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.486+0000 7f71b67fc640 1 -- 172.21.15.43:0/2783099776 <== mds.0 v2:172.21.15.43:6833/1586865031 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5589ff1e5f50 con 0x5589ff2a3d40 2024-04-25T09:02:32.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.486+0000 7f71b67fc640 10 client.8820 handle_command_reply: tid=0 2024-04-25T09:02:32.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.486+0000 7f71e102c740 1 client.8820 shutdown 2024-04-25T09:02:32.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.486+0000 7f71e102c740 1 -- 172.21.15.43:0/2783099776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589ff2a0230 msgr2=0x5589ff26f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:32.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.486+0000 7f71e102c740 1 --2- 172.21.15.43:0/2783099776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589ff2a0230 0x5589ff26f5a0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f71c8054aa0 tx=0x7f71c80882c0 comp rx=0 tx=0).stop 2024-04-25T09:02:32.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.486+0000 7f71e102c740 1 -- 172.21.15.43:0/2783099776 shutdown_connections 2024-04-25T09:02:32.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.486+0000 7f71e102c740 1 --2- 172.21.15.43:0/2783099776 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x5589ff2a3d40 0x5589ff331160 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:32.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.486+0000 7f71e102c740 1 --2- 172.21.15.43:0/2783099776 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5589ff2bdad0 0x5589ff2b1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:32.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.486+0000 7f71e102c740 1 --2- 172.21.15.43:0/2783099776 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5589ff2a3610 0x5589ff2b9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:32.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.486+0000 7f71b67fc640 0 client.8820 ms_handle_reset on v2:172.21.15.43:6833/1586865031 2024-04-25T09:02:32.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.486+0000 7f71e102c740 1 --2- 172.21.15.43:0/2783099776 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5589ff2a0230 0x5589ff26f5a0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:32.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.486+0000 7f71e102c740 1 -- 172.21.15.43:0/2783099776 >> 172.21.15.43:0/2783099776 conn(0x5589ff275f10 msgr2=0x5589ff275400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:32.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.487+0000 7f71e102c740 1 -- 172.21.15.43:0/2783099776 shutdown_connections 2024-04-25T09:02:32.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.487+0000 7f71e102c740 1 -- 172.21.15.43:0/2783099776 wait complete. 2024-04-25T09:02:32.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.487+0000 7f71e102c740 20 client.8820 trim_cache size 0 max 16384 2024-04-25T09:02:32.484 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:02:32.484 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "6c4bd50e-aa00-4a7f-8d09-352344ef5a4b", 2024-04-25T09:02:32.484 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-04-25T09:02:32.484 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 8676, 2024-04-25T09:02:32.484 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-04-25T09:02:32.484 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-04-25T09:02:32.484 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-04-25T09:02:32.484 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 5.6885993800000003, 2024-04-25T09:02:32.485 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 69, 2024-04-25T09:02:32.485 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 113, 2024-04-25T09:02:32.485 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 113, 2024-04-25T09:02:32.485 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 10.709671282 2024-04-25T09:02:32.485 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:02:32.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.487+0000 7f71de24d640 1 -- 172.21.15.43:0/3321871364 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f71c406b4d0 msgr2=0x7f71c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:32.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.487+0000 7f71de24d640 1 --2- 172.21.15.43:0/3321871364 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f71c406b4d0 0x7f71c406d990 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f71cc034000 tx=0x7f71cc02e000 comp rx=0 tx=0).stop 2024-04-25T09:02:32.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.487+0000 7f71de24d640 1 -- 172.21.15.43:0/3321871364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71d80b3b00 msgr2=0x7f71d815c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:32.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.487+0000 7f71de24d640 1 --2- 172.21.15.43:0/3321871364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71d80b3b00 0x7f71d815c7f0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f71c800b420 tx=0x7f71c800b8e0 comp rx=0 tx=0).stop 2024-04-25T09:02:32.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.487+0000 7f71de24d640 1 -- 172.21.15.43:0/3321871364 shutdown_connections 2024-04-25T09:02:32.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.487+0000 7f71de24d640 1 --2- 172.21.15.43:0/3321871364 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f71d80ccbb0 0x7f71d80ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:32.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.487+0000 7f71de24d640 1 --2- 172.21.15.43:0/3321871364 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f71c406b4d0 0x7f71c406d990 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:32.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.487+0000 7f71de24d640 1 --2- 172.21.15.43:0/3321871364 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71d80cbfc0 0x7f71d815e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:32.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.487+0000 7f71de24d640 1 --2- 172.21.15.43:0/3321871364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71d80b3b00 0x7f71d815c7f0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:32.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.487+0000 7f71de24d640 1 -- 172.21.15.43:0/3321871364 >> 172.21.15.43:0/3321871364 conn(0x7f71d80b1480 msgr2=0x7f71d80b4180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:32.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.487+0000 7f71de24d640 1 -- 172.21.15.43:0/3321871364 shutdown_connections 2024-04-25T09:02:32.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.487+0000 7f71de24d640 1 -- 172.21.15.43:0/3321871364 wait complete. 2024-04-25T09:02:32.488 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:02:32.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.600+0000 7f01fb370640 1 Processor -- start 2024-04-25T09:02:32.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.600+0000 7f01fb370640 1 -- start start 2024-04-25T09:02:32.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.601+0000 7f01fb370640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f01f40d09b0 0x7f01f40b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:32.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.601+0000 7f01fb370640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01f40b4c30 con 0x7f01f4161390 2024-04-25T09:02:32.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.601+0000 7f01fb370640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01f40cf2c0 con 0x7f01f40ce5e0 2024-04-25T09:02:32.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.601+0000 7f01fb370640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01f40d1910 con 0x7f01f40d09b0 2024-04-25T09:02:32.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.601+0000 7f01f890b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f01f40d09b0 0x7f01f40b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:32.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.601+0000 7f01f910c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f01f4161390 0x7f01f40d02a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48418/0 (socket says 172.21.15.43:48418) 2024-04-25T09:02:32.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.601+0000 7f01f910c640 1 -- 172.21.15.43:0/2123880578 learned_addr learned my addr 172.21.15.43:0/2123880578 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:32.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.601+0000 7f01f890b640 1 -- 172.21.15.43:0/2123880578 >> v1:172.21.15.193:6789/0 conn(0x7f01f40ce5e0 legacy=0x7f01f40cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:32.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.602+0000 7f01f890b640 1 -- 172.21.15.43:0/2123880578 >> v1:172.21.15.43:6789/0 conn(0x7f01f4161390 legacy=0x7f01f40d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:32.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.602+0000 7f01f890b640 1 -- 172.21.15.43:0/2123880578 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f40cf5a0 con 0x7f01f40d09b0 2024-04-25T09:02:32.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.602+0000 7f01f3fff640 1 -- 172.21.15.43:0/2123880578 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4161839881 0 0) 0x7f01ec001110 con 0x7f01f4161390 2024-04-25T09:02:32.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.602+0000 7f01f890b640 1 --2- 172.21.15.43:0/2123880578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f40d09b0 0x7f01f40b6820 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f01e80097a0 tx=0x7f01e802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6361f887dc80dbda server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:32.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.602+0000 7f01f3fff640 1 -- 172.21.15.43:0/2123880578 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e8037070 con 0x7f01f40d09b0 2024-04-25T09:02:32.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.602+0000 7f01f3fff640 1 -- 172.21.15.43:0/2123880578 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e8032440 con 0x7f01f40d09b0 2024-04-25T09:02:32.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.602+0000 7f01f3fff640 1 -- 172.21.15.43:0/2123880578 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e8032810 con 0x7f01f40d09b0 2024-04-25T09:02:32.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.603+0000 7f01fb370640 1 -- 172.21.15.43:0/2123880578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f40d09b0 msgr2=0x7f01f40b6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:32.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.603+0000 7f01fb370640 1 --2- 172.21.15.43:0/2123880578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f40d09b0 0x7f01f40b6820 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f01e80097a0 tx=0x7f01e802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:02:32.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.603+0000 7f01fb370640 1 -- 172.21.15.43:0/2123880578 shutdown_connections 2024-04-25T09:02:32.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.603+0000 7f01fb370640 1 --2- 172.21.15.43:0/2123880578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f40d09b0 0x7f01f40b6820 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:32.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.603+0000 7f01fb370640 1 -- 172.21.15.43:0/2123880578 >> 172.21.15.43:0/2123880578 conn(0x7f01f409cd20 msgr2=0x7f01f40d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:32.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.603+0000 7f01fb370640 1 -- 172.21.15.43:0/2123880578 shutdown_connections 2024-04-25T09:02:32.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.603+0000 7f01fb370640 1 -- 172.21.15.43:0/2123880578 wait complete. 2024-04-25T09:02:32.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.604+0000 7f01fb370640 1 Processor -- start 2024-04-25T09:02:32.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.604+0000 7f01fb370640 1 -- start start 2024-04-25T09:02:32.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.605+0000 7f01fb370640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01f40ce5e0 0x7f01f414bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:32.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.605+0000 7f01fb370640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f40d09b0 0x7f01f40c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:32.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.605+0000 7f01fb370640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01f4161390 0x7f01f40cca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:32.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.605+0000 7f01fb370640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f4161770 con 0x7f01f40d09b0 2024-04-25T09:02:32.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.605+0000 7f01fb370640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f40d15a0 con 0x7f01f4161390 2024-04-25T09:02:32.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.605+0000 7f01fb370640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f40d1da0 con 0x7f01f40ce5e0 2024-04-25T09:02:32.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.605+0000 7f01f890b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f40d09b0 0x7f01f40c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:32.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.605+0000 7f01f890b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f40d09b0 0x7f01f40c9ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43846/0 (socket says 172.21.15.43:43846) 2024-04-25T09:02:32.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.605+0000 7f01f890b640 1 -- 172.21.15.43:0/283048785 learned_addr learned my addr 172.21.15.43:0/283048785 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:32.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.605+0000 7f01f990d640 1 --2- 172.21.15.43:0/283048785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01f4161390 0x7f01f40cca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:32.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.605+0000 7f01f910c640 1 --2- 172.21.15.43:0/283048785 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01f40ce5e0 0x7f01f414bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:32.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.605+0000 7f01f890b640 1 -- 172.21.15.43:0/283048785 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01f40ce5e0 msgr2=0x7f01f414bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:32.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.605+0000 7f01f890b640 1 --2- 172.21.15.43:0/283048785 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01f40ce5e0 0x7f01f414bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:32.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.605+0000 7f01f890b640 1 -- 172.21.15.43:0/283048785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01f4161390 msgr2=0x7f01f40cca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:32.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.605+0000 7f01f890b640 1 --2- 172.21.15.43:0/283048785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01f4161390 0x7f01f40cca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:32.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.606+0000 7f01f890b640 1 -- 172.21.15.43:0/283048785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f40ca420 con 0x7f01f40d09b0 2024-04-25T09:02:32.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.606+0000 7f01f890b640 1 --2- 172.21.15.43:0/283048785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f40d09b0 0x7f01f40c9ee0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f01e80097a0 tx=0x7f01e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:32.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.606+0000 7f01f1ffb640 1 -- 172.21.15.43:0/283048785 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e8037040 con 0x7f01f40d09b0 2024-04-25T09:02:32.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.606+0000 7f01f1ffb640 1 -- 172.21.15.43:0/283048785 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e8004250 con 0x7f01f40d09b0 2024-04-25T09:02:32.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.606+0000 7f01f1ffb640 1 -- 172.21.15.43:0/283048785 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e8032e20 con 0x7f01f40d09b0 2024-04-25T09:02:32.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.606+0000 7f01fb370640 1 -- 172.21.15.43:0/283048785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f40cc020 con 0x7f01f40d09b0 2024-04-25T09:02:32.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.606+0000 7f01fb370640 1 -- 172.21.15.43:0/283048785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f40c8390 con 0x7f01f40d09b0 2024-04-25T09:02:32.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.607+0000 7f01fb370640 1 -- 172.21.15.43:0/283048785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01f40680f0 con 0x7f01f40d09b0 2024-04-25T09:02:32.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.607+0000 7f01f1ffb640 1 -- 172.21.15.43:0/283048785 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f01e8030770 con 0x7f01f40d09b0 2024-04-25T09:02:32.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.608+0000 7f01f1ffb640 1 --2- 172.21.15.43:0/283048785 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f01e006b4d0 0x7f01e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:32.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.608+0000 7f01f1ffb640 1 -- 172.21.15.43:0/283048785 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f01e80ab390 con 0x7f01f40d09b0 2024-04-25T09:02:32.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.608+0000 7f01f910c640 1 --2- 172.21.15.43:0/283048785 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f01e006b4d0 0x7f01e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:32.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.609+0000 7f01f910c640 1 --2- 172.21.15.43:0/283048785 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f01e006b4d0 0x7f01e006d990 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f01ec007fd0 tx=0x7f01ec007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:32.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.612+0000 7f01f1ffb640 1 -- 172.21.15.43:0/283048785 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f01e8079bc0 con 0x7f01f40d09b0 2024-04-25T09:02:32.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.834+0000 7f01fb370640 1 -- 172.21.15.43:0/283048785 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f01f4074ef0 con 0x7f01f40d09b0 2024-04-25T09:02:32.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.834+0000 7f01f1ffb640 1 -- 172.21.15.43:0/283048785 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v69) v1 ==== 53+0+73 (secure 0 0 0) 0x7f01e807dc60 con 0x7f01f40d09b0 2024-04-25T09:02:32.823 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:02:32.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.836+0000 7f01fb370640 1 -- 172.21.15.43:0/283048785 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f01e006b4d0 msgr2=0x7f01e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:32.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.836+0000 7f01fb370640 1 --2- 172.21.15.43:0/283048785 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f01e006b4d0 0x7f01e006d990 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f01ec007fd0 tx=0x7f01ec007de0 comp rx=0 tx=0).stop 2024-04-25T09:02:32.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.836+0000 7f01fb370640 1 -- 172.21.15.43:0/283048785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f40d09b0 msgr2=0x7f01f40c9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:32.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.836+0000 7f01fb370640 1 --2- 172.21.15.43:0/283048785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f40d09b0 0x7f01f40c9ee0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f01e80097a0 tx=0x7f01e80046c0 comp rx=0 tx=0).stop 2024-04-25T09:02:32.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.836+0000 7f01fb370640 1 -- 172.21.15.43:0/283048785 shutdown_connections 2024-04-25T09:02:32.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.836+0000 7f01fb370640 1 --2- 172.21.15.43:0/283048785 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f01e006b4d0 0x7f01e006d990 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:32.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.836+0000 7f01fb370640 1 --2- 172.21.15.43:0/283048785 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f01f4161390 0x7f01f40cca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:32.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.836+0000 7f01fb370640 1 --2- 172.21.15.43:0/283048785 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f01f40d09b0 0x7f01f40c9ee0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:32.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.836+0000 7f01fb370640 1 --2- 172.21.15.43:0/283048785 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f01f40ce5e0 0x7f01f414bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:32.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.836+0000 7f01fb370640 1 -- 172.21.15.43:0/283048785 >> 172.21.15.43:0/283048785 conn(0x7f01f409cd20 msgr2=0x7f01f406c260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:32.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.836+0000 7f01fb370640 1 -- 172.21.15.43:0/283048785 shutdown_connections 2024-04-25T09:02:32.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:32.837+0000 7f01fb370640 1 -- 172.21.15.43:0/283048785 wait complete. 2024-04-25T09:02:32.836 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T09:02:32.837 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:02:32.837 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-04-25T09:02:32.837 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:02:32.837 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:02:32.837 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T09:02:32.837 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T09:02:32.837 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:02:32.837 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:02:32.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:32.838 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-04-25T09:02:32.853 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:02:32.854 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:02:32.854 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:02:32.854 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:02:32.854 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-04-25T09:02:32.854 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:02:32.854 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:02:32.854 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-04-25T09:02:32.854 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-04-25T09:02:32.854 INFO:teuthology.orchestra.run.smithi043.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:02:32.854 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:02:32.854 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-04-25T09:02:32.854 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T09:02:32.854 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 42265sec preferred_lft 42265sec 2024-04-25T09:02:32.855 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link noprefixroute 2024-04-25T09:02:32.855 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:02:32.855 INFO:teuthology.orchestra.run.smithi043.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:02:32.855 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-04-25T09:02:32.855 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-04-25T09:02:32.855 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:02:32.855 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:02:32.855 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-04-25T09:02:32.855 INFO:teuthology.orchestra.run.smithi043.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:02:32.855 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 12:2c:7a:72:36:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:02:32.855 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:02:32.855 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:02:32.855 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ccff:4eff:fe9d:c8bf/64 scope link 2024-04-25T09:02:32.855 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:02:32.856 INFO:teuthology.orchestra.run.smithi043.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:02:32.856 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 16:9a:63:37:46:24 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:02:32.856 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::149a:63ff:fe37:4624/64 scope link 2024-04-25T09:02:32.856 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:02:32.856 INFO:teuthology.orchestra.run.smithi043.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:02:32.856 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 12:2c:7a:72:36:14 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:02:32.856 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::102c:7aff:fe72:3614/64 scope link 2024-04-25T09:02:32.856 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:02:32.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:32.856 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:02:32.917 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-04-25T09:02:32.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:32.919 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-04-25T09:02:32.975 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-04-25T09:02:32.975 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:02:32.976 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T09:02:32.976 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T09:02:32.976 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:02:32.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:32.976 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2024-04-25T09:02:32.977 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:02:32.977 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T09:02:32.977 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T09:02:32.977 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T09:02:32.977 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:02:33.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:33.061 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-04-25T09:02:33.076 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:02:33.076 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:02:33.076 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:02:33.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:33.077 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-04-25T09:02:33.141 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T09:02:33.141 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:02:33.141 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:02:33.197 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:02:33.197 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:02:33.197 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:02:33.252 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-04-25T09:02:33.317 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:02:33.373 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-04-25T09:02:33.373 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-04-25T09:02:33.374 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-04-25T09:02:33.374 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-04-25T09:02:33.374 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-04-25T09:02:33.374 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-04-25T09:02:33.374 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-04-25T09:02:33.374 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-04-25T09:02:33.374 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-04-25T09:02:33.374 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-04-25T09:02:33.374 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-04-25T09:02:33.374 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/selinux 2024-04-25T09:02:33.374 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:02:33.374 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-04-25T09:02:33.374 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-04-25T09:02:33.375 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-04-25T09:02:33.375 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-04-25T09:02:33.375 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:02:33.375 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-04-25T09:02:33.375 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-04-25T09:02:33.375 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T09:02:33.375 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T09:02:33.375 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T09:02:33.375 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T09:02:33.375 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-04-25T09:02:33.375 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-04-25T09:02:33.375 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:02:33.375 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:02:33.375 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:02:33.375 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:02:33.376 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-04-25T09:02:33.376 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:02:33.376 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:02:33.376 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:02:33.376 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:02:33.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:33.376 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-04-25T09:02:33.428 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T09:02:33.428 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T09:02:33.471 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-04-25T09:02:33.495 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:02:33.551 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-04-25T09:02:33.551 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-04-25T09:02:33.551 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-04-25T09:02:33.551 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-04-25T09:02:33.551 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-04-25T09:02:33.551 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-04-25T09:02:33.551 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-04-25T09:02:33.551 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-04-25T09:02:33.551 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-04-25T09:02:33.551 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-04-25T09:02:33.551 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-04-25T09:02:33.551 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/selinux 2024-04-25T09:02:33.552 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:02:33.552 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-04-25T09:02:33.552 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-04-25T09:02:33.552 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-04-25T09:02:33.552 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-04-25T09:02:33.552 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:02:33.552 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-04-25T09:02:33.552 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-04-25T09:02:33.552 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T09:02:33.552 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T09:02:33.552 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T09:02:33.552 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T09:02:33.552 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-04-25T09:02:33.552 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-04-25T09:02:33.552 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:02:33.553 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:02:33.553 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:02:33.553 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:02:33.553 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-04-25T09:02:33.553 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:02:33.553 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:02:33.553 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:02:33.553 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:02:33.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:33.553 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-04-25T09:02:33.554 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:2024-04-25T09:02:33.563+0000 7fcfbe58d480 -1 init, newargv = 0x5644e7452490 newargc=15 2024-04-25T09:02:33.554 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[86573]: starting ceph client 2024-04-25T09:02:33.555 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[86573]: starting fuse 2024-04-25T09:02:33.570 INFO:teuthology.orchestra.run.smithi043.stdout:43 2024-04-25T09:02:33.570 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-25T09:02:33.570 DEBUG:teuthology.orchestra.run.smithi043:> sudo stdin-killer -- python3 -c ' 2024-04-25T09:02:33.570 DEBUG:teuthology.orchestra.run.smithi043:> import glob 2024-04-25T09:02:33.570 DEBUG:teuthology.orchestra.run.smithi043:> import re 2024-04-25T09:02:33.570 DEBUG:teuthology.orchestra.run.smithi043:> import os 2024-04-25T09:02:33.570 DEBUG:teuthology.orchestra.run.smithi043:> import subprocess 2024-04-25T09:02:33.570 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T09:02:33.571 DEBUG:teuthology.orchestra.run.smithi043:> def _find_admin_socket(client_name): 2024-04-25T09:02:33.571 DEBUG:teuthology.orchestra.run.smithi043:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T09:02:33.571 DEBUG:teuthology.orchestra.run.smithi043:> files = glob.glob(asok_path) 2024-04-25T09:02:33.571 DEBUG:teuthology.orchestra.run.smithi043:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T09:02:33.571 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T09:02:33.571 DEBUG:teuthology.orchestra.run.smithi043:> # Given a non-glob path, it better be there 2024-04-25T09:02:33.571 DEBUG:teuthology.orchestra.run.smithi043:> if "*" not in asok_path: 2024-04-25T09:02:33.571 DEBUG:teuthology.orchestra.run.smithi043:> assert(len(files) == 1) 2024-04-25T09:02:33.571 DEBUG:teuthology.orchestra.run.smithi043:> return files[0] 2024-04-25T09:02:33.571 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T09:02:33.571 DEBUG:teuthology.orchestra.run.smithi043:> for f in files: 2024-04-25T09:02:33.571 DEBUG:teuthology.orchestra.run.smithi043:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T09:02:33.571 DEBUG:teuthology.orchestra.run.smithi043:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T09:02:33.571 DEBUG:teuthology.orchestra.run.smithi043:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T09:02:33.571 DEBUG:teuthology.orchestra.run.smithi043:> contents = proc_f.read() 2024-04-25T09:02:33.571 DEBUG:teuthology.orchestra.run.smithi043:> if mountpoint in contents: 2024-04-25T09:02:33.572 DEBUG:teuthology.orchestra.run.smithi043:> return f 2024-04-25T09:02:33.572 DEBUG:teuthology.orchestra.run.smithi043:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T09:02:33.572 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T09:02:33.572 DEBUG:teuthology.orchestra.run.smithi043:> print(_find_admin_socket("client.0")) 2024-04-25T09:02:33.572 DEBUG:teuthology.orchestra.run.smithi043:> ' 2024-04-25T09:02:33.690 INFO:teuthology.orchestra.run.smithi043.stdout:/var/run/ceph/ceph-client.0.86573.asok 2024-04-25T09:02:33.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:02:33.701 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.86573.asok 2024-04-25T09:02:33.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:33.701 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.86573.asok status 2024-04-25T09:02:33.840 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:02:33.840 INFO:teuthology.orchestra.run.smithi043.stdout: "metadata": { 2024-04-25T09:02:33.840 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T09:02:33.840 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T09:02:33.840 INFO:teuthology.orchestra.run.smithi043.stdout: "entity_id": "0", 2024-04-25T09:02:33.840 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": "smithi043", 2024-04-25T09:02:33.840 INFO:teuthology.orchestra.run.smithi043.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T09:02:33.840 INFO:teuthology.orchestra.run.smithi043.stdout: "pid": "86573", 2024-04-25T09:02:33.841 INFO:teuthology.orchestra.run.smithi043.stdout: "root": "/" 2024-04-25T09:02:33.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:02:33.841 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_count": 0, 2024-04-25T09:02:33.841 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_pinned_count": 0, 2024-04-25T09:02:33.841 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 8835, 2024-04-25T09:02:33.841 INFO:teuthology.orchestra.run.smithi043.stdout: "inst": { 2024-04-25T09:02:33.841 INFO:teuthology.orchestra.run.smithi043.stdout: "name": { 2024-04-25T09:02:33.841 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "client", 2024-04-25T09:02:33.841 INFO:teuthology.orchestra.run.smithi043.stdout: "num": 8835 2024-04-25T09:02:33.841 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:02:33.841 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-04-25T09:02:33.841 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-04-25T09:02:33.842 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:02:33.842 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 2320481441 2024-04-25T09:02:33.842 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:02:33.842 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:02:33.842 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-04-25T09:02:33.842 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-04-25T09:02:33.842 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:02:33.842 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 2320481441 2024-04-25T09:02:33.843 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:02:33.843 INFO:teuthology.orchestra.run.smithi043.stdout: "inst_str": "client.8835 192.168.0.1:0/2320481441", 2024-04-25T09:02:33.843 INFO:teuthology.orchestra.run.smithi043.stdout: "addr_str": "192.168.0.1:0/2320481441", 2024-04-25T09:02:33.843 INFO:teuthology.orchestra.run.smithi043.stdout: "inode_count": 1, 2024-04-25T09:02:33.843 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_epoch": 69, 2024-04-25T09:02:33.843 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch": 113, 2024-04-25T09:02:33.843 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch_barrier": 0, 2024-04-25T09:02:33.843 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklisted": false, 2024-04-25T09:02:33.843 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs" 2024-04-25T09:02:33.843 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:02:33.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:33.849 DEBUG:teuthology.orchestra.run.smithi043:> stat --file-system '--printf=%T 2024-04-25T09:02:33.849 DEBUG:teuthology.orchestra.run.smithi043:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T09:02:33.865 INFO:teuthology.orchestra.run.smithi043.stdout:fuseblk 2024-04-25T09:02:33.865 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T09:02:33.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:33.866 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:02:33.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:33.931 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:02:33.931 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:02:33.931 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T09:02:33.931 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:02:33.931 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:02:33.931 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:02:33.931 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:02:34.008 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.009 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.009 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0112944 s, 92.8 MB/s 2024-04-25T09:02:34.020 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.021 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.021 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00879793 s, 119 MB/s 2024-04-25T09:02:34.032 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.032 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.032 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101484 s, 103 MB/s 2024-04-25T09:02:34.042 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.042 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.042 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743236 s, 141 MB/s 2024-04-25T09:02:34.052 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.052 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.052 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832452 s, 126 MB/s 2024-04-25T09:02:34.063 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.063 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.063 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00877342 s, 120 MB/s 2024-04-25T09:02:34.076 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.076 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.076 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102177 s, 103 MB/s 2024-04-25T09:02:34.086 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.086 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.086 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810268 s, 129 MB/s 2024-04-25T09:02:34.097 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.097 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.097 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835701 s, 125 MB/s 2024-04-25T09:02:34.108 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.108 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.108 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00858739 s, 122 MB/s 2024-04-25T09:02:34.119 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.119 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.119 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.008716 s, 120 MB/s 2024-04-25T09:02:34.129 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.129 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.129 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00803407 s, 131 MB/s 2024-04-25T09:02:34.139 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.139 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.139 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769909 s, 136 MB/s 2024-04-25T09:02:34.151 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.151 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.151 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0101096 s, 104 MB/s 2024-04-25T09:02:34.163 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.163 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.163 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00993801 s, 106 MB/s 2024-04-25T09:02:34.174 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.174 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.174 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846762 s, 124 MB/s 2024-04-25T09:02:34.184 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.184 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.184 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851283 s, 123 MB/s 2024-04-25T09:02:34.195 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.196 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.196 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842188 s, 125 MB/s 2024-04-25T09:02:34.205 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.205 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.206 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.008386 s, 125 MB/s 2024-04-25T09:02:34.216 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.216 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.216 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00853612 s, 123 MB/s 2024-04-25T09:02:34.226 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.227 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.227 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840766 s, 125 MB/s 2024-04-25T09:02:34.237 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.237 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.237 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084918 s, 123 MB/s 2024-04-25T09:02:34.246 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.246 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.246 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724652 s, 145 MB/s 2024-04-25T09:02:34.257 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.257 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.257 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834084 s, 126 MB/s 2024-04-25T09:02:34.271 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.271 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.271 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00809734 s, 129 MB/s 2024-04-25T09:02:34.278 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.278 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.278 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084319 s, 124 MB/s 2024-04-25T09:02:34.288 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.288 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.288 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821482 s, 128 MB/s 2024-04-25T09:02:34.298 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.298 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.298 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00784931 s, 134 MB/s 2024-04-25T09:02:34.308 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.308 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.308 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0079752 s, 131 MB/s 2024-04-25T09:02:34.318 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.318 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.318 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797677 s, 131 MB/s 2024-04-25T09:02:34.328 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.328 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.328 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821755 s, 128 MB/s 2024-04-25T09:02:34.339 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.339 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.339 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00893181 s, 117 MB/s 2024-04-25T09:02:34.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:34.341 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:02:34.341 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:02:34.341 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T09:02:34.341 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:02:34.341 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:02:34.341 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:02:34.341 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:02:34.409 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.409 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.409 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00789583 s, 133 MB/s 2024-04-25T09:02:34.420 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.420 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.420 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00929747 s, 113 MB/s 2024-04-25T09:02:34.430 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.430 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.431 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835474 s, 126 MB/s 2024-04-25T09:02:34.441 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.441 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.441 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839108 s, 125 MB/s 2024-04-25T09:02:34.452 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.452 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.452 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00806073 s, 130 MB/s 2024-04-25T09:02:34.462 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.462 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.462 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842769 s, 124 MB/s 2024-04-25T09:02:34.471 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.472 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.472 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792341 s, 132 MB/s 2024-04-25T09:02:34.482 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.482 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.482 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828588 s, 127 MB/s 2024-04-25T09:02:34.492 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.492 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.492 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810829 s, 129 MB/s 2024-04-25T09:02:34.502 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.502 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.502 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721984 s, 145 MB/s 2024-04-25T09:02:34.511 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.512 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.512 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772942 s, 136 MB/s 2024-04-25T09:02:34.522 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.522 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.522 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821779 s, 128 MB/s 2024-04-25T09:02:34.532 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.532 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.532 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00800383 s, 131 MB/s 2024-04-25T09:02:34.541 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.541 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.541 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709637 s, 148 MB/s 2024-04-25T09:02:34.551 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.551 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.551 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00789796 s, 133 MB/s 2024-04-25T09:02:34.561 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.561 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.561 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707207 s, 148 MB/s 2024-04-25T09:02:34.570 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.571 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.571 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769979 s, 136 MB/s 2024-04-25T09:02:34.582 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.582 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.582 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00888326 s, 118 MB/s 2024-04-25T09:02:34.593 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.593 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.593 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00899935 s, 117 MB/s 2024-04-25T09:02:34.603 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.603 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.603 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00779442 s, 135 MB/s 2024-04-25T09:02:34.613 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.613 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.613 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739567 s, 142 MB/s 2024-04-25T09:02:34.622 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.622 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.622 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715292 s, 147 MB/s 2024-04-25T09:02:34.633 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.633 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.633 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00865929 s, 121 MB/s 2024-04-25T09:02:34.642 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.642 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.642 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741446 s, 141 MB/s 2024-04-25T09:02:34.652 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.652 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.652 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727358 s, 144 MB/s 2024-04-25T09:02:34.661 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.661 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.661 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731009 s, 143 MB/s 2024-04-25T09:02:34.671 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.671 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.671 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718919 s, 146 MB/s 2024-04-25T09:02:34.681 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.681 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.681 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00801727 s, 131 MB/s 2024-04-25T09:02:34.691 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.691 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.691 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770254 s, 136 MB/s 2024-04-25T09:02:34.701 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.702 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.702 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706013 s, 149 MB/s 2024-04-25T09:02:34.711 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.711 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.711 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00901058 s, 116 MB/s 2024-04-25T09:02:34.721 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.722 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.722 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00806854 s, 130 MB/s 2024-04-25T09:02:34.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:34.723 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:02:34.723 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:02:34.723 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T09:02:34.723 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:02:34.723 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:02:34.723 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:02:34.723 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:02:34.791 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.791 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.791 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824936 s, 127 MB/s 2024-04-25T09:02:34.801 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.802 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.802 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00790986 s, 133 MB/s 2024-04-25T09:02:34.811 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.811 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.811 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787832 s, 133 MB/s 2024-04-25T09:02:34.821 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.822 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.822 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804272 s, 130 MB/s 2024-04-25T09:02:34.831 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.831 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.832 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783946 s, 134 MB/s 2024-04-25T09:02:34.843 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.843 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.843 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00915038 s, 115 MB/s 2024-04-25T09:02:34.853 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.853 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.853 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00800101 s, 131 MB/s 2024-04-25T09:02:34.863 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.863 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.864 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00807174 s, 130 MB/s 2024-04-25T09:02:34.873 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.873 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.873 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788789 s, 133 MB/s 2024-04-25T09:02:34.884 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.884 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.884 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00782438 s, 134 MB/s 2024-04-25T09:02:34.893 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.894 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.894 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00808667 s, 130 MB/s 2024-04-25T09:02:34.903 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.903 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.903 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735527 s, 143 MB/s 2024-04-25T09:02:34.913 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.913 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.914 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815582 s, 129 MB/s 2024-04-25T09:02:34.924 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.924 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.924 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825339 s, 127 MB/s 2024-04-25T09:02:34.935 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.935 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.935 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0087164 s, 120 MB/s 2024-04-25T09:02:34.945 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.946 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.946 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787965 s, 133 MB/s 2024-04-25T09:02:34.956 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.956 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.956 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00871569 s, 120 MB/s 2024-04-25T09:02:34.966 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.966 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.966 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770535 s, 136 MB/s 2024-04-25T09:02:34.976 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.976 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.977 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00799191 s, 131 MB/s 2024-04-25T09:02:34.986 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.987 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.987 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0079075 s, 133 MB/s 2024-04-25T09:02:34.998 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:34.999 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:34.999 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00953051 s, 110 MB/s 2024-04-25T09:02:35.008 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.008 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.009 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756442 s, 139 MB/s 2024-04-25T09:02:35.018 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.018 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.018 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765939 s, 137 MB/s 2024-04-25T09:02:35.029 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.029 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.029 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00800302 s, 131 MB/s 2024-04-25T09:02:35.039 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.039 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.039 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00803147 s, 131 MB/s 2024-04-25T09:02:35.048 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.049 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.049 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738967 s, 142 MB/s 2024-04-25T09:02:35.059 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.059 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.059 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811011 s, 129 MB/s 2024-04-25T09:02:35.070 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.070 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.070 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00902749 s, 116 MB/s 2024-04-25T09:02:35.081 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.081 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.081 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00879082 s, 119 MB/s 2024-04-25T09:02:35.091 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.091 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.091 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730369 s, 144 MB/s 2024-04-25T09:02:35.101 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.101 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.101 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819857 s, 128 MB/s 2024-04-25T09:02:35.112 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.112 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.112 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787068 s, 133 MB/s 2024-04-25T09:02:35.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:35.112 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:02:35.113 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:02:35.113 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T09:02:35.113 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:02:35.113 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:02:35.113 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:02:35.113 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:02:35.181 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.181 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.181 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00806459 s, 130 MB/s 2024-04-25T09:02:35.191 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.191 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.191 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774848 s, 135 MB/s 2024-04-25T09:02:35.201 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.201 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.201 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811931 s, 129 MB/s 2024-04-25T09:02:35.211 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.211 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.211 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772659 s, 136 MB/s 2024-04-25T09:02:35.220 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.221 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.221 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073037 s, 144 MB/s 2024-04-25T09:02:35.231 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.231 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.232 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824178 s, 127 MB/s 2024-04-25T09:02:35.242 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.242 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.242 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00876155 s, 120 MB/s 2024-04-25T09:02:35.251 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.252 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.252 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072494 s, 145 MB/s 2024-04-25T09:02:35.261 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.261 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.261 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739574 s, 142 MB/s 2024-04-25T09:02:35.271 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.271 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.271 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747235 s, 140 MB/s 2024-04-25T09:02:35.282 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.282 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.282 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00867798 s, 121 MB/s 2024-04-25T09:02:35.293 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.293 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.293 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00865644 s, 121 MB/s 2024-04-25T09:02:35.303 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.303 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.303 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00800844 s, 131 MB/s 2024-04-25T09:02:35.312 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.312 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.313 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732869 s, 143 MB/s 2024-04-25T09:02:35.323 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.323 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.323 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00843276 s, 124 MB/s 2024-04-25T09:02:35.333 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.333 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.333 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745236 s, 141 MB/s 2024-04-25T09:02:35.343 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.343 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.343 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00794025 s, 132 MB/s 2024-04-25T09:02:35.355 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.355 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.355 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00948181 s, 111 MB/s 2024-04-25T09:02:35.365 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.365 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.365 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822035 s, 128 MB/s 2024-04-25T09:02:35.375 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.375 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.375 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00799768 s, 131 MB/s 2024-04-25T09:02:35.385 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.385 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.385 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780277 s, 134 MB/s 2024-04-25T09:02:35.395 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.395 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.395 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0078978 s, 133 MB/s 2024-04-25T09:02:35.405 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.405 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.405 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797747 s, 131 MB/s 2024-04-25T09:02:35.415 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.415 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.415 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726099 s, 144 MB/s 2024-04-25T09:02:35.426 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.426 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.426 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0089412 s, 117 MB/s 2024-04-25T09:02:35.436 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.436 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.436 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742986 s, 141 MB/s 2024-04-25T09:02:35.446 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.446 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.446 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755543 s, 139 MB/s 2024-04-25T09:02:35.456 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.456 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.456 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818552 s, 128 MB/s 2024-04-25T09:02:35.466 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.466 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.466 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774956 s, 135 MB/s 2024-04-25T09:02:35.477 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.477 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.477 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829293 s, 126 MB/s 2024-04-25T09:02:35.487 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.488 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.488 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00885428 s, 118 MB/s 2024-04-25T09:02:35.498 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.498 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.498 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839975 s, 125 MB/s 2024-04-25T09:02:35.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:35.499 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:02:35.499 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:02:35.500 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T09:02:35.500 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:02:35.500 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:02:35.500 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:02:35.500 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:02:35.568 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.568 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.568 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793857 s, 132 MB/s 2024-04-25T09:02:35.577 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.577 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.577 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731228 s, 143 MB/s 2024-04-25T09:02:35.588 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.588 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.588 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851403 s, 123 MB/s 2024-04-25T09:02:35.598 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.598 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.598 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00798952 s, 131 MB/s 2024-04-25T09:02:35.608 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.608 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.608 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783428 s, 134 MB/s 2024-04-25T09:02:35.618 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.618 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.618 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720175 s, 146 MB/s 2024-04-25T09:02:35.627 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.627 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.627 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073273 s, 143 MB/s 2024-04-25T09:02:35.638 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.638 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.638 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831322 s, 126 MB/s 2024-04-25T09:02:35.647 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.648 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.648 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007504 s, 140 MB/s 2024-04-25T09:02:35.657 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.657 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.657 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727231 s, 144 MB/s 2024-04-25T09:02:35.668 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.669 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.669 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00922714 s, 114 MB/s 2024-04-25T09:02:35.679 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.679 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.679 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797051 s, 132 MB/s 2024-04-25T09:02:35.689 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.689 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.689 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00784076 s, 134 MB/s 2024-04-25T09:02:35.698 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.698 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.698 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753878 s, 139 MB/s 2024-04-25T09:02:35.709 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.709 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.709 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00812157 s, 129 MB/s 2024-04-25T09:02:35.718 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.718 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.719 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074242 s, 141 MB/s 2024-04-25T09:02:35.730 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.730 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.730 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00936038 s, 112 MB/s 2024-04-25T09:02:35.740 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.740 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.740 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00800251 s, 131 MB/s 2024-04-25T09:02:35.751 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.751 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.751 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00879649 s, 119 MB/s 2024-04-25T09:02:35.761 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.761 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.762 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00800424 s, 131 MB/s 2024-04-25T09:02:35.773 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.773 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.773 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00904022 s, 116 MB/s 2024-04-25T09:02:35.783 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.783 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.783 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780311 s, 134 MB/s 2024-04-25T09:02:35.794 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.794 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.794 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00781934 s, 134 MB/s 2024-04-25T09:02:35.802 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.802 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.802 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734171 s, 143 MB/s 2024-04-25T09:02:35.812 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.812 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.812 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076631 s, 137 MB/s 2024-04-25T09:02:35.822 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.822 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.822 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774241 s, 135 MB/s 2024-04-25T09:02:35.831 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.831 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.831 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742397 s, 141 MB/s 2024-04-25T09:02:35.841 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.841 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.841 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734309 s, 143 MB/s 2024-04-25T09:02:35.851 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.851 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.851 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783229 s, 134 MB/s 2024-04-25T09:02:35.861 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.861 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.861 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00805072 s, 130 MB/s 2024-04-25T09:02:35.872 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.872 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.873 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0088544 s, 118 MB/s 2024-04-25T09:02:35.882 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.883 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.883 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00794329 s, 132 MB/s 2024-04-25T09:02:35.884 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:35.884 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:02:35.884 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:02:35.884 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T09:02:35.884 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:02:35.884 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:02:35.884 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:02:35.884 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:02:35.952 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.952 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.952 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00871221 s, 120 MB/s 2024-04-25T09:02:35.964 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.964 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.964 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00852539 s, 123 MB/s 2024-04-25T09:02:35.974 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.974 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.974 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00865779 s, 121 MB/s 2024-04-25T09:02:35.984 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.984 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.984 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00794391 s, 132 MB/s 2024-04-25T09:02:35.995 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:35.995 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:35.995 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00872303 s, 120 MB/s 2024-04-25T09:02:36.006 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.006 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.006 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00902403 s, 116 MB/s 2024-04-25T09:02:36.016 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.016 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.016 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00798526 s, 131 MB/s 2024-04-25T09:02:36.026 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.026 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.026 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753506 s, 139 MB/s 2024-04-25T09:02:36.036 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.036 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.036 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841288 s, 125 MB/s 2024-04-25T09:02:36.046 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.046 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.046 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778694 s, 135 MB/s 2024-04-25T09:02:36.057 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.057 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.057 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0087513 s, 120 MB/s 2024-04-25T09:02:36.067 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.067 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.067 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741299 s, 141 MB/s 2024-04-25T09:02:36.077 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.077 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.077 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757653 s, 138 MB/s 2024-04-25T09:02:36.087 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.087 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.087 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00806519 s, 130 MB/s 2024-04-25T09:02:36.096 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.096 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.096 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737355 s, 142 MB/s 2024-04-25T09:02:36.107 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.107 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.107 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804978 s, 130 MB/s 2024-04-25T09:02:36.121 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.121 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.121 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0125574 s, 83.5 MB/s 2024-04-25T09:02:36.131 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.131 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.131 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742252 s, 141 MB/s 2024-04-25T09:02:36.140 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.140 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.140 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725367 s, 145 MB/s 2024-04-25T09:02:36.151 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.151 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.151 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00862392 s, 122 MB/s 2024-04-25T09:02:36.161 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.161 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.161 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823904 s, 127 MB/s 2024-04-25T09:02:36.170 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.171 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.171 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738672 s, 142 MB/s 2024-04-25T09:02:36.181 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.181 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.181 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00855877 s, 123 MB/s 2024-04-25T09:02:36.192 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.192 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.192 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00886279 s, 118 MB/s 2024-04-25T09:02:36.203 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.203 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.203 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00869188 s, 121 MB/s 2024-04-25T09:02:36.214 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.214 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.214 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00874082 s, 120 MB/s 2024-04-25T09:02:36.224 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.224 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.224 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757409 s, 138 MB/s 2024-04-25T09:02:36.234 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.234 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.234 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811484 s, 129 MB/s 2024-04-25T09:02:36.244 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.244 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.244 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00775641 s, 135 MB/s 2024-04-25T09:02:36.255 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.255 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.255 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00887167 s, 118 MB/s 2024-04-25T09:02:36.266 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.266 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.266 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00902966 s, 116 MB/s 2024-04-25T09:02:36.276 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.276 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.276 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739663 s, 142 MB/s 2024-04-25T09:02:36.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:36.277 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:02:36.277 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:02:36.277 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T09:02:36.277 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:02:36.277 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:02:36.277 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:02:36.277 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:02:36.345 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.345 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.346 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771865 s, 136 MB/s 2024-04-25T09:02:36.355 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.355 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.355 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737808 s, 142 MB/s 2024-04-25T09:02:36.364 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.364 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.364 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073588 s, 142 MB/s 2024-04-25T09:02:36.374 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.375 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.375 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00799698 s, 131 MB/s 2024-04-25T09:02:36.385 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.385 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.385 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00885773 s, 118 MB/s 2024-04-25T09:02:36.395 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.395 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.396 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788692 s, 133 MB/s 2024-04-25T09:02:36.406 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.406 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.406 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0085208 s, 123 MB/s 2024-04-25T09:02:36.417 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.417 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.417 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00844888 s, 124 MB/s 2024-04-25T09:02:36.428 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.428 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.428 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00884667 s, 119 MB/s 2024-04-25T09:02:36.439 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.439 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.439 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754398 s, 139 MB/s 2024-04-25T09:02:36.446 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.446 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.446 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680721 s, 154 MB/s 2024-04-25T09:02:36.456 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.456 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.456 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760755 s, 138 MB/s 2024-04-25T09:02:36.466 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.466 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.466 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749421 s, 140 MB/s 2024-04-25T09:02:36.475 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.475 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.475 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705144 s, 149 MB/s 2024-04-25T09:02:36.484 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.484 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.484 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712563 s, 147 MB/s 2024-04-25T09:02:36.495 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.495 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.495 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842557 s, 124 MB/s 2024-04-25T09:02:36.504 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.504 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.505 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744387 s, 141 MB/s 2024-04-25T09:02:36.514 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.514 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.514 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741748 s, 141 MB/s 2024-04-25T09:02:36.524 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.524 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.524 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771789 s, 136 MB/s 2024-04-25T09:02:36.533 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.533 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.533 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692879 s, 151 MB/s 2024-04-25T09:02:36.542 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.542 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.542 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687696 s, 152 MB/s 2024-04-25T09:02:36.552 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.552 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.552 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752175 s, 139 MB/s 2024-04-25T09:02:36.562 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.562 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.562 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818533 s, 128 MB/s 2024-04-25T09:02:36.571 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.571 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.571 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710411 s, 148 MB/s 2024-04-25T09:02:36.580 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.580 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.580 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682406 s, 154 MB/s 2024-04-25T09:02:36.590 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.590 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.590 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716973 s, 146 MB/s 2024-04-25T09:02:36.599 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.599 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.600 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742956 s, 141 MB/s 2024-04-25T09:02:36.610 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.610 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.610 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0081709 s, 128 MB/s 2024-04-25T09:02:36.619 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.620 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.620 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730512 s, 144 MB/s 2024-04-25T09:02:36.629 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.629 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.629 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740849 s, 142 MB/s 2024-04-25T09:02:36.639 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.639 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.639 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00812179 s, 129 MB/s 2024-04-25T09:02:36.649 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.649 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.649 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074214 s, 141 MB/s 2024-04-25T09:02:36.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:36.650 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:02:36.650 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:02:36.650 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T09:02:36.650 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:02:36.650 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:02:36.650 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:02:36.651 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:02:36.718 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.718 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.719 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745087 s, 141 MB/s 2024-04-25T09:02:36.728 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.728 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.728 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748739 s, 140 MB/s 2024-04-25T09:02:36.739 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.739 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.739 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00858554 s, 122 MB/s 2024-04-25T09:02:36.749 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.750 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.750 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00860489 s, 122 MB/s 2024-04-25T09:02:36.760 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.760 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.760 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825294 s, 127 MB/s 2024-04-25T09:02:36.770 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.771 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.771 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839879 s, 125 MB/s 2024-04-25T09:02:36.781 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.781 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.781 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00800266 s, 131 MB/s 2024-04-25T09:02:36.794 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.794 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.794 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0107452 s, 97.6 MB/s 2024-04-25T09:02:36.805 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.805 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.805 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787135 s, 133 MB/s 2024-04-25T09:02:36.815 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.815 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.815 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783115 s, 134 MB/s 2024-04-25T09:02:36.826 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.826 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.826 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0083135 s, 126 MB/s 2024-04-25T09:02:36.837 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.837 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.837 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773733 s, 136 MB/s 2024-04-25T09:02:36.846 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.846 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.846 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679569 s, 154 MB/s 2024-04-25T09:02:36.856 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.856 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.857 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830677 s, 126 MB/s 2024-04-25T09:02:36.866 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.866 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.866 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718759 s, 146 MB/s 2024-04-25T09:02:36.875 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.875 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.875 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735829 s, 143 MB/s 2024-04-25T09:02:36.885 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.885 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.885 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750248 s, 140 MB/s 2024-04-25T09:02:36.894 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.894 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.894 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698718 s, 150 MB/s 2024-04-25T09:02:36.903 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.903 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.904 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685505 s, 153 MB/s 2024-04-25T09:02:36.912 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.912 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.912 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683122 s, 153 MB/s 2024-04-25T09:02:36.923 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.923 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.923 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821739 s, 128 MB/s 2024-04-25T09:02:36.933 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.934 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.934 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00857134 s, 122 MB/s 2024-04-25T09:02:36.944 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.944 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.944 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00809636 s, 130 MB/s 2024-04-25T09:02:36.954 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.954 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.954 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777233 s, 135 MB/s 2024-04-25T09:02:36.964 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.964 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.964 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753942 s, 139 MB/s 2024-04-25T09:02:36.973 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.973 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.973 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722829 s, 145 MB/s 2024-04-25T09:02:36.983 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.983 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.983 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739895 s, 142 MB/s 2024-04-25T09:02:36.993 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:36.993 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:36.993 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783209 s, 134 MB/s 2024-04-25T09:02:37.003 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.004 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.004 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838599 s, 125 MB/s 2024-04-25T09:02:37.014 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.014 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.014 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830417 s, 126 MB/s 2024-04-25T09:02:37.023 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.023 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.023 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701606 s, 149 MB/s 2024-04-25T09:02:37.033 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.033 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.033 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074064 s, 142 MB/s 2024-04-25T09:02:37.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:37.034 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:02:37.035 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:02:37.035 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T09:02:37.035 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:02:37.035 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:02:37.035 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:02:37.035 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:02:37.102 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.102 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.103 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753104 s, 139 MB/s 2024-04-25T09:02:37.112 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.112 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.112 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732193 s, 143 MB/s 2024-04-25T09:02:37.121 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.121 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.121 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732153 s, 143 MB/s 2024-04-25T09:02:37.132 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.132 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.132 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.008533 s, 123 MB/s 2024-04-25T09:02:37.143 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.143 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.143 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861598 s, 122 MB/s 2024-04-25T09:02:37.152 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.152 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.153 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727686 s, 144 MB/s 2024-04-25T09:02:37.162 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.162 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.162 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704496 s, 149 MB/s 2024-04-25T09:02:37.171 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.171 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.171 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737286 s, 142 MB/s 2024-04-25T09:02:37.182 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.182 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.182 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836273 s, 125 MB/s 2024-04-25T09:02:37.192 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.193 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.193 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00795836 s, 132 MB/s 2024-04-25T09:02:37.202 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.202 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.202 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747788 s, 140 MB/s 2024-04-25T09:02:37.212 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.212 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.212 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834774 s, 126 MB/s 2024-04-25T09:02:37.221 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.221 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.221 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685477 s, 153 MB/s 2024-04-25T09:02:37.232 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.232 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.232 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816188 s, 128 MB/s 2024-04-25T09:02:37.241 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.241 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.241 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733375 s, 143 MB/s 2024-04-25T09:02:37.251 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.251 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.251 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072208 s, 145 MB/s 2024-04-25T09:02:37.260 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.260 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.260 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745252 s, 141 MB/s 2024-04-25T09:02:37.269 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.269 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.269 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703723 s, 149 MB/s 2024-04-25T09:02:37.279 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.279 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.279 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00807109 s, 130 MB/s 2024-04-25T09:02:37.289 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.289 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.289 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735224 s, 143 MB/s 2024-04-25T09:02:37.299 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.299 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.299 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815855 s, 129 MB/s 2024-04-25T09:02:37.309 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.309 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.309 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734158 s, 143 MB/s 2024-04-25T09:02:37.317 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.318 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.318 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674512 s, 155 MB/s 2024-04-25T09:02:37.327 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.328 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.328 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689711 s, 152 MB/s 2024-04-25T09:02:37.336 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.336 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.336 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688035 s, 152 MB/s 2024-04-25T09:02:37.345 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.346 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.346 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743968 s, 141 MB/s 2024-04-25T09:02:37.355 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.355 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.355 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699559 s, 150 MB/s 2024-04-25T09:02:37.364 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.365 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.365 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757327 s, 138 MB/s 2024-04-25T09:02:37.374 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.374 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.374 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722918 s, 145 MB/s 2024-04-25T09:02:37.383 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.383 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.383 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705182 s, 149 MB/s 2024-04-25T09:02:37.393 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.393 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.393 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739491 s, 142 MB/s 2024-04-25T09:02:37.404 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.404 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.404 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00860194 s, 122 MB/s 2024-04-25T09:02:37.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:37.405 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:02:37.405 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:02:37.405 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T09:02:37.405 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:02:37.405 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:02:37.405 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:02:37.405 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:02:37.474 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.474 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.474 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842149 s, 125 MB/s 2024-04-25T09:02:37.483 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.484 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.484 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728337 s, 144 MB/s 2024-04-25T09:02:37.493 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.493 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.493 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688111 s, 152 MB/s 2024-04-25T09:02:37.502 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.502 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.502 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700914 s, 150 MB/s 2024-04-25T09:02:37.512 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.512 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.512 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0078125 s, 134 MB/s 2024-04-25T09:02:37.522 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.522 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.522 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732778 s, 143 MB/s 2024-04-25T09:02:37.531 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.531 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.531 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727368 s, 144 MB/s 2024-04-25T09:02:37.541 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.541 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.541 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765506 s, 137 MB/s 2024-04-25T09:02:37.551 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.551 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.551 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831258 s, 126 MB/s 2024-04-25T09:02:37.562 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.562 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.562 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00882386 s, 119 MB/s 2024-04-25T09:02:37.571 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.571 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.571 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707375 s, 148 MB/s 2024-04-25T09:02:37.580 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.580 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.581 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686434 s, 153 MB/s 2024-04-25T09:02:37.590 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.590 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.590 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739545 s, 142 MB/s 2024-04-25T09:02:37.600 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.600 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.600 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815327 s, 129 MB/s 2024-04-25T09:02:37.611 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.611 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.611 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814035 s, 129 MB/s 2024-04-25T09:02:37.620 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.620 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.621 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744866 s, 141 MB/s 2024-04-25T09:02:37.630 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.630 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.630 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732413 s, 143 MB/s 2024-04-25T09:02:37.639 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.639 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.640 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073948 s, 142 MB/s 2024-04-25T09:02:37.649 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.649 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.649 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722355 s, 145 MB/s 2024-04-25T09:02:37.658 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.658 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.658 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752539 s, 139 MB/s 2024-04-25T09:02:37.669 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.669 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.669 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00844248 s, 124 MB/s 2024-04-25T09:02:37.678 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.678 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.678 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695307 s, 151 MB/s 2024-04-25T09:02:37.688 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.688 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.688 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076478 s, 137 MB/s 2024-04-25T09:02:37.697 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.698 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.698 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709363 s, 148 MB/s 2024-04-25T09:02:37.706 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.706 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.706 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00664131 s, 158 MB/s 2024-04-25T09:02:37.718 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.718 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.718 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00917078 s, 114 MB/s 2024-04-25T09:02:37.727 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.728 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.728 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732948 s, 143 MB/s 2024-04-25T09:02:37.736 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.737 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.737 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723093 s, 145 MB/s 2024-04-25T09:02:37.746 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.746 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.746 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729322 s, 144 MB/s 2024-04-25T09:02:37.755 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.756 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.756 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744339 s, 141 MB/s 2024-04-25T09:02:37.766 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.766 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.766 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818132 s, 128 MB/s 2024-04-25T09:02:37.775 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.775 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.775 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711733 s, 147 MB/s 2024-04-25T09:02:37.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:37.776 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:02:37.777 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:02:37.777 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T09:02:37.777 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:02:37.777 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:02:37.777 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:02:37.777 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:02:37.844 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.845 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.845 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728139 s, 144 MB/s 2024-04-25T09:02:37.853 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.853 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.853 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066777 s, 157 MB/s 2024-04-25T09:02:37.863 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.863 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.863 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823133 s, 127 MB/s 2024-04-25T09:02:37.873 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.873 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.873 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714112 s, 147 MB/s 2024-04-25T09:02:37.882 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.882 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.882 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728076 s, 144 MB/s 2024-04-25T09:02:37.891 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.891 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.891 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658148 s, 159 MB/s 2024-04-25T09:02:37.901 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.901 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.901 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0081728 s, 128 MB/s 2024-04-25T09:02:37.911 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.911 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.911 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701395 s, 149 MB/s 2024-04-25T09:02:37.920 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.921 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.921 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076881 s, 136 MB/s 2024-04-25T09:02:37.930 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.930 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.930 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733818 s, 143 MB/s 2024-04-25T09:02:37.939 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.939 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.939 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702514 s, 149 MB/s 2024-04-25T09:02:37.949 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.949 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.949 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007738 s, 136 MB/s 2024-04-25T09:02:37.959 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.959 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.959 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824069 s, 127 MB/s 2024-04-25T09:02:37.970 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.970 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.970 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815503 s, 129 MB/s 2024-04-25T09:02:37.978 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.979 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.979 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671672 s, 156 MB/s 2024-04-25T09:02:37.988 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.988 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.988 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703593 s, 149 MB/s 2024-04-25T09:02:37.997 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:37.997 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:37.997 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680266 s, 154 MB/s 2024-04-25T09:02:38.007 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.007 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.007 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740801 s, 142 MB/s 2024-04-25T09:02:38.017 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.017 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.017 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765878 s, 137 MB/s 2024-04-25T09:02:38.026 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.026 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.026 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711115 s, 147 MB/s 2024-04-25T09:02:38.036 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.036 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.036 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821697 s, 128 MB/s 2024-04-25T09:02:38.045 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.046 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.046 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710664 s, 148 MB/s 2024-04-25T09:02:38.055 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.055 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.055 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752888 s, 139 MB/s 2024-04-25T09:02:38.065 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.065 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.065 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711212 s, 147 MB/s 2024-04-25T09:02:38.074 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.074 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.075 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751909 s, 139 MB/s 2024-04-25T09:02:38.085 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.085 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.085 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00848811 s, 124 MB/s 2024-04-25T09:02:38.094 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.094 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.094 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699343 s, 150 MB/s 2024-04-25T09:02:38.103 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.103 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.104 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706597 s, 148 MB/s 2024-04-25T09:02:38.114 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.114 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.114 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0081821 s, 128 MB/s 2024-04-25T09:02:38.123 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.123 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.123 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068311 s, 154 MB/s 2024-04-25T09:02:38.133 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.134 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.134 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845198 s, 124 MB/s 2024-04-25T09:02:38.147 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.147 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.147 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00859696 s, 122 MB/s 2024-04-25T09:02:38.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:38.148 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:02:38.148 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:02:38.148 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T09:02:38.148 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:02:38.149 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:02:38.149 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:02:38.149 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:02:38.215 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.215 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.215 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759453 s, 138 MB/s 2024-04-25T09:02:38.225 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.225 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.225 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084604 s, 124 MB/s 2024-04-25T09:02:38.234 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.234 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.234 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716826 s, 146 MB/s 2024-04-25T09:02:38.245 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.245 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.245 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818242 s, 128 MB/s 2024-04-25T09:02:38.254 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.254 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.254 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712516 s, 147 MB/s 2024-04-25T09:02:38.264 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.264 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.264 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760365 s, 138 MB/s 2024-04-25T09:02:38.273 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.273 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.273 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696932 s, 150 MB/s 2024-04-25T09:02:38.283 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.283 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.283 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752937 s, 139 MB/s 2024-04-25T09:02:38.292 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.293 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.293 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738621 s, 142 MB/s 2024-04-25T09:02:38.303 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.303 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.303 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766136 s, 137 MB/s 2024-04-25T09:02:38.312 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.312 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.312 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728205 s, 144 MB/s 2024-04-25T09:02:38.321 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.321 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.321 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00670264 s, 156 MB/s 2024-04-25T09:02:38.330 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.330 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.330 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709474 s, 148 MB/s 2024-04-25T09:02:38.339 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.339 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.339 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00663528 s, 158 MB/s 2024-04-25T09:02:38.348 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.348 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.349 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741813 s, 141 MB/s 2024-04-25T09:02:38.358 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.358 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.358 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722372 s, 145 MB/s 2024-04-25T09:02:38.367 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.367 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.367 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733477 s, 143 MB/s 2024-04-25T09:02:38.377 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.377 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.377 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739848 s, 142 MB/s 2024-04-25T09:02:38.387 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.387 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.387 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818538 s, 128 MB/s 2024-04-25T09:02:38.397 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.397 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.397 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00803546 s, 130 MB/s 2024-04-25T09:02:38.406 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.406 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.406 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711253 s, 147 MB/s 2024-04-25T09:02:38.415 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.416 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.416 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070282 s, 149 MB/s 2024-04-25T09:02:38.424 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.424 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.424 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668585 s, 157 MB/s 2024-04-25T09:02:38.434 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.434 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.434 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073424 s, 143 MB/s 2024-04-25T09:02:38.443 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.443 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.443 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749083 s, 140 MB/s 2024-04-25T09:02:38.453 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.453 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.453 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708677 s, 148 MB/s 2024-04-25T09:02:38.462 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.462 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.463 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741952 s, 141 MB/s 2024-04-25T09:02:38.472 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.472 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.472 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750525 s, 140 MB/s 2024-04-25T09:02:38.481 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.481 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.481 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708929 s, 148 MB/s 2024-04-25T09:02:38.492 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.492 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.492 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082906 s, 126 MB/s 2024-04-25T09:02:38.501 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.501 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.501 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733747 s, 143 MB/s 2024-04-25T09:02:38.512 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.512 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.512 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839121 s, 125 MB/s 2024-04-25T09:02:38.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:38.513 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:02:38.513 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:02:38.514 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T09:02:38.514 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:02:38.514 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:02:38.514 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:02:38.514 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:02:38.583 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.584 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.584 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821994 s, 128 MB/s 2024-04-25T09:02:38.593 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.593 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.593 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731102 s, 143 MB/s 2024-04-25T09:02:38.602 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.602 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.602 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715281 s, 147 MB/s 2024-04-25T09:02:38.611 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.611 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.612 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707963 s, 148 MB/s 2024-04-25T09:02:38.622 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.622 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.622 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836674 s, 125 MB/s 2024-04-25T09:02:38.631 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.632 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.632 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754503 s, 139 MB/s 2024-04-25T09:02:38.641 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.641 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.641 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681639 s, 154 MB/s 2024-04-25T09:02:38.651 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.651 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.651 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824872 s, 127 MB/s 2024-04-25T09:02:38.661 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.661 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.661 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741746 s, 141 MB/s 2024-04-25T09:02:38.669 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.670 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.670 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682468 s, 154 MB/s 2024-04-25T09:02:38.679 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.679 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.679 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728111 s, 144 MB/s 2024-04-25T09:02:38.689 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.689 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.689 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740315 s, 142 MB/s 2024-04-25T09:02:38.699 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.699 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.699 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840695 s, 125 MB/s 2024-04-25T09:02:38.708 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.708 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.708 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694356 s, 151 MB/s 2024-04-25T09:02:38.718 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.718 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.718 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758145 s, 138 MB/s 2024-04-25T09:02:38.727 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.727 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.727 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710327 s, 148 MB/s 2024-04-25T09:02:38.736 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.736 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.737 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691736 s, 152 MB/s 2024-04-25T09:02:38.746 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.746 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.746 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743811 s, 141 MB/s 2024-04-25T09:02:38.756 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.757 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.757 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836602 s, 125 MB/s 2024-04-25T09:02:38.766 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.766 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.766 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770935 s, 136 MB/s 2024-04-25T09:02:38.776 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.776 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.776 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752595 s, 139 MB/s 2024-04-25T09:02:38.785 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.786 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.786 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704493 s, 149 MB/s 2024-04-25T09:02:38.795 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.795 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.795 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007412 s, 141 MB/s 2024-04-25T09:02:38.805 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.805 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.805 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723465 s, 145 MB/s 2024-04-25T09:02:38.815 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.815 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.815 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00862607 s, 122 MB/s 2024-04-25T09:02:38.826 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.826 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.826 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00837546 s, 125 MB/s 2024-04-25T09:02:38.836 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.836 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.836 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759254 s, 138 MB/s 2024-04-25T09:02:38.848 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.848 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.848 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716581 s, 146 MB/s 2024-04-25T09:02:38.856 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.856 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.856 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00843022 s, 124 MB/s 2024-04-25T09:02:38.865 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.866 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.866 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739396 s, 142 MB/s 2024-04-25T09:02:38.875 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.875 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.875 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741299 s, 141 MB/s 2024-04-25T09:02:38.884 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.885 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.885 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724504 s, 145 MB/s 2024-04-25T09:02:38.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:38.886 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:02:38.886 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:02:38.886 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T09:02:38.886 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:02:38.886 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:02:38.886 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:02:38.886 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:02:38.954 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.954 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.954 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069043 s, 152 MB/s 2024-04-25T09:02:38.963 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.963 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.963 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766964 s, 137 MB/s 2024-04-25T09:02:38.972 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.972 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.972 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00656972 s, 160 MB/s 2024-04-25T09:02:38.982 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.982 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.982 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830443 s, 126 MB/s 2024-04-25T09:02:38.994 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:38.994 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:38.995 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724727 s, 145 MB/s 2024-04-25T09:02:39.001 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.001 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.001 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705206 s, 149 MB/s 2024-04-25T09:02:39.010 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.010 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.010 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707431 s, 148 MB/s 2024-04-25T09:02:39.019 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.019 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.019 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676778 s, 155 MB/s 2024-04-25T09:02:39.029 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.029 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.029 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720047 s, 146 MB/s 2024-04-25T09:02:39.038 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.038 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.038 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722984 s, 145 MB/s 2024-04-25T09:02:39.047 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.047 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.047 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068071 s, 154 MB/s 2024-04-25T09:02:39.057 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.057 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.057 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743414 s, 141 MB/s 2024-04-25T09:02:39.065 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.065 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.065 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067217 s, 156 MB/s 2024-04-25T09:02:39.075 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.075 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.075 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734482 s, 143 MB/s 2024-04-25T09:02:39.084 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.084 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.084 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725652 s, 145 MB/s 2024-04-25T09:02:39.093 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.093 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.093 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697068 s, 150 MB/s 2024-04-25T09:02:39.104 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.104 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.104 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851645 s, 123 MB/s 2024-04-25T09:02:39.114 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.114 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.115 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814434 s, 129 MB/s 2024-04-25T09:02:39.124 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.124 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.124 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764696 s, 137 MB/s 2024-04-25T09:02:39.134 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.134 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.134 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743661 s, 141 MB/s 2024-04-25T09:02:39.143 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.143 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.143 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725726 s, 144 MB/s 2024-04-25T09:02:39.153 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.153 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.154 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723253 s, 145 MB/s 2024-04-25T09:02:39.162 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.162 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.162 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727959 s, 144 MB/s 2024-04-25T09:02:39.171 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.171 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.171 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071496 s, 147 MB/s 2024-04-25T09:02:39.180 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.180 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.180 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708544 s, 148 MB/s 2024-04-25T09:02:39.189 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.189 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.190 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712709 s, 147 MB/s 2024-04-25T09:02:39.198 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.198 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.198 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667286 s, 157 MB/s 2024-04-25T09:02:39.208 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.208 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.208 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0066046 s, 159 MB/s 2024-04-25T09:02:39.217 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.217 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.217 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00768484 s, 136 MB/s 2024-04-25T09:02:39.227 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.227 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.227 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746455 s, 140 MB/s 2024-04-25T09:02:39.236 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.236 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.236 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715922 s, 146 MB/s 2024-04-25T09:02:39.245 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.245 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.246 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711195 s, 147 MB/s 2024-04-25T09:02:39.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:39.247 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:02:39.247 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:02:39.247 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T09:02:39.247 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:02:39.247 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:02:39.247 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:02:39.247 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:02:39.314 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.314 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.315 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724642 s, 145 MB/s 2024-04-25T09:02:39.324 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.324 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.324 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749222 s, 140 MB/s 2024-04-25T09:02:39.334 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.334 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.334 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832947 s, 126 MB/s 2024-04-25T09:02:39.344 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.344 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.344 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738757 s, 142 MB/s 2024-04-25T09:02:39.353 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.353 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.353 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734201 s, 143 MB/s 2024-04-25T09:02:39.363 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.363 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.363 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757586 s, 138 MB/s 2024-04-25T09:02:39.372 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.372 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.372 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698433 s, 150 MB/s 2024-04-25T09:02:39.381 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.381 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.382 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068736 s, 153 MB/s 2024-04-25T09:02:39.392 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.392 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.392 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822582 s, 127 MB/s 2024-04-25T09:02:39.401 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.402 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.402 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725833 s, 144 MB/s 2024-04-25T09:02:39.411 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.411 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.411 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753435 s, 139 MB/s 2024-04-25T09:02:39.421 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.421 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.421 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718336 s, 146 MB/s 2024-04-25T09:02:39.430 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.430 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.430 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740159 s, 142 MB/s 2024-04-25T09:02:39.441 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.441 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.441 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818403 s, 128 MB/s 2024-04-25T09:02:39.452 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.452 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.452 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00912884 s, 115 MB/s 2024-04-25T09:02:39.462 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.462 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.462 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747938 s, 140 MB/s 2024-04-25T09:02:39.471 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.472 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.472 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748482 s, 140 MB/s 2024-04-25T09:02:39.481 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.481 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.481 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758402 s, 138 MB/s 2024-04-25T09:02:39.492 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.492 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.492 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826322 s, 127 MB/s 2024-04-25T09:02:39.502 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.502 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.503 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826014 s, 127 MB/s 2024-04-25T09:02:39.513 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.513 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.513 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836026 s, 125 MB/s 2024-04-25T09:02:39.523 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.523 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.523 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00782839 s, 134 MB/s 2024-04-25T09:02:39.533 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.533 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.533 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724396 s, 145 MB/s 2024-04-25T09:02:39.542 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.542 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.542 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715781 s, 146 MB/s 2024-04-25T09:02:39.552 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.552 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.552 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763385 s, 137 MB/s 2024-04-25T09:02:39.565 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.565 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.565 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0111591 s, 94.0 MB/s 2024-04-25T09:02:39.584 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.584 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.584 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00837819 s, 125 MB/s 2024-04-25T09:02:39.594 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.595 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.595 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00813601 s, 129 MB/s 2024-04-25T09:02:39.605 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.605 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.605 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818591 s, 128 MB/s 2024-04-25T09:02:39.614 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.615 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.615 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733847 s, 143 MB/s 2024-04-25T09:02:39.624 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.624 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.624 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720534 s, 146 MB/s 2024-04-25T09:02:39.634 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.634 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.635 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823837 s, 127 MB/s 2024-04-25T09:02:39.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:39.636 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:02:39.636 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:02:39.636 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T09:02:39.636 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:02:39.636 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:02:39.636 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:02:39.636 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:02:39.703 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.703 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.703 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739117 s, 142 MB/s 2024-04-25T09:02:39.712 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.712 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.712 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724623 s, 145 MB/s 2024-04-25T09:02:39.722 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.722 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.722 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761466 s, 138 MB/s 2024-04-25T09:02:39.731 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.731 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.731 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739738 s, 142 MB/s 2024-04-25T09:02:39.741 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.741 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.742 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712683 s, 147 MB/s 2024-04-25T09:02:39.750 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.750 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.751 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735093 s, 143 MB/s 2024-04-25T09:02:39.760 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.760 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.760 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722007 s, 145 MB/s 2024-04-25T09:02:39.773 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.773 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.773 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688785 s, 152 MB/s 2024-04-25T09:02:39.779 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.779 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.779 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751656 s, 140 MB/s 2024-04-25T09:02:39.789 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.790 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.790 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833646 s, 126 MB/s 2024-04-25T09:02:39.799 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.799 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.799 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709031 s, 148 MB/s 2024-04-25T09:02:39.808 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.808 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.809 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733479 s, 143 MB/s 2024-04-25T09:02:39.818 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.818 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.818 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764088 s, 137 MB/s 2024-04-25T09:02:39.828 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.829 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.829 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772117 s, 136 MB/s 2024-04-25T09:02:39.838 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.838 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.838 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778982 s, 135 MB/s 2024-04-25T09:02:39.848 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.848 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.848 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075411 s, 139 MB/s 2024-04-25T09:02:39.858 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.859 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.859 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830055 s, 126 MB/s 2024-04-25T09:02:39.869 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.869 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.869 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814352 s, 129 MB/s 2024-04-25T09:02:39.878 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.879 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.879 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737459 s, 142 MB/s 2024-04-25T09:02:39.888 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.888 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.888 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697832 s, 150 MB/s 2024-04-25T09:02:39.897 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.897 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.897 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693182 s, 151 MB/s 2024-04-25T09:02:39.906 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.906 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.906 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701572 s, 149 MB/s 2024-04-25T09:02:39.917 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.917 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.917 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845534 s, 124 MB/s 2024-04-25T09:02:39.926 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.926 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.926 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698962 s, 150 MB/s 2024-04-25T09:02:39.935 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.936 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.936 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675489 s, 155 MB/s 2024-04-25T09:02:39.945 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.945 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.945 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769693 s, 136 MB/s 2024-04-25T09:02:39.954 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.955 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.955 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719957 s, 146 MB/s 2024-04-25T09:02:39.964 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.965 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.965 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758336 s, 138 MB/s 2024-04-25T09:02:39.974 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.974 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.975 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753963 s, 139 MB/s 2024-04-25T09:02:39.983 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.983 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.983 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699942 s, 150 MB/s 2024-04-25T09:02:39.993 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:39.993 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:39.993 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746088 s, 141 MB/s 2024-04-25T09:02:40.003 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.003 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.003 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764923 s, 137 MB/s 2024-04-25T09:02:40.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:40.004 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:02:40.004 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:02:40.005 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T09:02:40.005 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:02:40.005 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:02:40.005 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:02:40.005 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:02:40.074 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.074 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.074 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00856388 s, 122 MB/s 2024-04-25T09:02:40.083 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.083 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.083 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747152 s, 140 MB/s 2024-04-25T09:02:40.092 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.092 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.093 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704724 s, 149 MB/s 2024-04-25T09:02:40.102 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.102 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.102 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727348 s, 144 MB/s 2024-04-25T09:02:40.112 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.112 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.112 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742468 s, 141 MB/s 2024-04-25T09:02:40.121 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.121 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.121 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744557 s, 141 MB/s 2024-04-25T09:02:40.130 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.131 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.131 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716344 s, 146 MB/s 2024-04-25T09:02:40.140 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.140 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.141 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778448 s, 135 MB/s 2024-04-25T09:02:40.151 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.151 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.151 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845587 s, 124 MB/s 2024-04-25T09:02:40.160 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.160 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.161 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687789 s, 152 MB/s 2024-04-25T09:02:40.171 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.171 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.171 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00849451 s, 123 MB/s 2024-04-25T09:02:40.182 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.182 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.182 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840881 s, 125 MB/s 2024-04-25T09:02:40.191 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.191 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.191 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070262 s, 149 MB/s 2024-04-25T09:02:40.201 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.201 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.201 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729863 s, 144 MB/s 2024-04-25T09:02:40.211 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.211 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.211 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772923 s, 136 MB/s 2024-04-25T09:02:40.220 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.220 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.220 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726326 s, 144 MB/s 2024-04-25T09:02:40.229 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.229 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.229 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696511 s, 151 MB/s 2024-04-25T09:02:40.240 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.240 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.240 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850891 s, 123 MB/s 2024-04-25T09:02:40.251 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.251 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.251 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00866888 s, 121 MB/s 2024-04-25T09:02:40.261 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.261 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.261 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749187 s, 140 MB/s 2024-04-25T09:02:40.271 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.271 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.271 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082391 s, 127 MB/s 2024-04-25T09:02:40.281 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.281 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.281 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742947 s, 141 MB/s 2024-04-25T09:02:40.291 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.291 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.291 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746475 s, 140 MB/s 2024-04-25T09:02:40.302 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.302 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.302 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084718 s, 124 MB/s 2024-04-25T09:02:40.312 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.312 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.312 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767316 s, 137 MB/s 2024-04-25T09:02:40.321 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.321 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.322 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730218 s, 144 MB/s 2024-04-25T09:02:40.332 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.332 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.332 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835741 s, 125 MB/s 2024-04-25T09:02:40.342 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.342 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.342 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747428 s, 140 MB/s 2024-04-25T09:02:40.351 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.351 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.351 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075088 s, 140 MB/s 2024-04-25T09:02:40.361 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.361 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.361 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716035 s, 146 MB/s 2024-04-25T09:02:40.370 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.370 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.370 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668515 s, 157 MB/s 2024-04-25T09:02:40.381 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.381 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.381 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861868 s, 122 MB/s 2024-04-25T09:02:40.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:40.382 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:02:40.382 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:02:40.382 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T09:02:40.382 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:02:40.382 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:02:40.382 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:02:40.383 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:02:40.449 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.449 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.449 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068319 s, 153 MB/s 2024-04-25T09:02:40.458 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.459 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.459 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711768 s, 147 MB/s 2024-04-25T09:02:40.470 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.470 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.470 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00880583 s, 119 MB/s 2024-04-25T09:02:40.479 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.479 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.479 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736504 s, 142 MB/s 2024-04-25T09:02:40.490 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.490 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.490 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00870695 s, 120 MB/s 2024-04-25T09:02:40.500 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.500 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.500 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750445 s, 140 MB/s 2024-04-25T09:02:40.509 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.509 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.510 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708162 s, 148 MB/s 2024-04-25T09:02:40.518 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.519 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.519 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683799 s, 153 MB/s 2024-04-25T09:02:40.528 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.528 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.528 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758843 s, 138 MB/s 2024-04-25T09:02:40.539 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.539 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.539 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829454 s, 126 MB/s 2024-04-25T09:02:40.548 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.548 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.549 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746282 s, 141 MB/s 2024-04-25T09:02:40.558 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.558 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.558 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725777 s, 144 MB/s 2024-04-25T09:02:40.571 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.571 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.571 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0109532 s, 95.7 MB/s 2024-04-25T09:02:40.599 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.599 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.599 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743039 s, 141 MB/s 2024-04-25T09:02:40.609 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.609 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.609 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838835 s, 125 MB/s 2024-04-25T09:02:40.620 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.620 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.620 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0085936 s, 122 MB/s 2024-04-25T09:02:40.629 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.629 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.629 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682493 s, 154 MB/s 2024-04-25T09:02:40.642 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.642 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.642 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724674 s, 145 MB/s 2024-04-25T09:02:40.648 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.648 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.648 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701231 s, 150 MB/s 2024-04-25T09:02:40.657 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.657 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.657 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708384 s, 148 MB/s 2024-04-25T09:02:40.667 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.667 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.667 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734972 s, 143 MB/s 2024-04-25T09:02:40.677 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.677 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.677 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00796001 s, 132 MB/s 2024-04-25T09:02:40.688 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.688 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.688 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00864885 s, 121 MB/s 2024-04-25T09:02:40.699 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.699 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.699 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00870638 s, 120 MB/s 2024-04-25T09:02:40.708 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.709 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.709 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752124 s, 139 MB/s 2024-04-25T09:02:40.718 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.718 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.718 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699616 s, 150 MB/s 2024-04-25T09:02:40.727 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.727 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.727 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685379 s, 153 MB/s 2024-04-25T09:02:40.737 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.738 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.738 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00789475 s, 133 MB/s 2024-04-25T09:02:40.747 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.748 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.748 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828272 s, 127 MB/s 2024-04-25T09:02:40.757 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.757 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.757 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708229 s, 148 MB/s 2024-04-25T09:02:40.766 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.766 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.766 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676696 s, 155 MB/s 2024-04-25T09:02:40.778 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.778 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.778 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820301 s, 128 MB/s 2024-04-25T09:02:40.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:40.779 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:02:40.779 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:02:40.779 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T09:02:40.780 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:02:40.780 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:02:40.780 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:02:40.780 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:02:40.846 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.846 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.846 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741871 s, 141 MB/s 2024-04-25T09:02:40.856 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.857 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.857 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00855778 s, 123 MB/s 2024-04-25T09:02:40.866 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.866 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.866 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740871 s, 142 MB/s 2024-04-25T09:02:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.875 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.876 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714939 s, 147 MB/s 2024-04-25T09:02:40.885 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.885 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.885 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719691 s, 146 MB/s 2024-04-25T09:02:40.895 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.895 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.895 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723506 s, 145 MB/s 2024-04-25T09:02:40.904 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.904 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.904 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072256 s, 145 MB/s 2024-04-25T09:02:40.913 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.913 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.914 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704705 s, 149 MB/s 2024-04-25T09:02:40.923 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.923 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.923 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719334 s, 146 MB/s 2024-04-25T09:02:40.933 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.934 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.934 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825084 s, 127 MB/s 2024-04-25T09:02:40.943 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.943 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.943 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762328 s, 138 MB/s 2024-04-25T09:02:40.952 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.953 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.953 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701856 s, 149 MB/s 2024-04-25T09:02:40.962 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.962 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.962 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743411 s, 141 MB/s 2024-04-25T09:02:40.973 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.973 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.973 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00865035 s, 121 MB/s 2024-04-25T09:02:40.982 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.982 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.982 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697736 s, 150 MB/s 2024-04-25T09:02:40.992 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:40.992 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:40.992 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764872 s, 137 MB/s 2024-04-25T09:02:41.002 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.002 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.002 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758328 s, 138 MB/s 2024-04-25T09:02:41.011 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.011 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.011 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723445 s, 145 MB/s 2024-04-25T09:02:41.021 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.021 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.021 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756236 s, 139 MB/s 2024-04-25T09:02:41.030 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.030 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.030 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720588 s, 146 MB/s 2024-04-25T09:02:41.040 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.040 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.040 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00768663 s, 136 MB/s 2024-04-25T09:02:41.050 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.050 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.050 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073473 s, 143 MB/s 2024-04-25T09:02:41.059 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.059 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.060 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772837 s, 136 MB/s 2024-04-25T09:02:41.070 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.070 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.070 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0077963 s, 134 MB/s 2024-04-25T09:02:41.078 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.079 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.079 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697137 s, 150 MB/s 2024-04-25T09:02:41.088 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.088 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.088 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072107 s, 145 MB/s 2024-04-25T09:02:41.097 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.097 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.097 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701587 s, 149 MB/s 2024-04-25T09:02:41.106 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.107 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.107 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658812 s, 159 MB/s 2024-04-25T09:02:41.117 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.117 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.117 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834884 s, 126 MB/s 2024-04-25T09:02:41.126 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.126 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.126 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700292 s, 150 MB/s 2024-04-25T09:02:41.137 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.137 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.137 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840913 s, 125 MB/s 2024-04-25T09:02:41.147 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.147 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.147 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769249 s, 136 MB/s 2024-04-25T09:02:41.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:41.148 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:02:41.148 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:02:41.148 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T09:02:41.148 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:02:41.148 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:02:41.148 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:02:41.148 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:02:41.218 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.218 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.218 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00864324 s, 121 MB/s 2024-04-25T09:02:41.227 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.227 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.227 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067217 s, 156 MB/s 2024-04-25T09:02:41.237 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.237 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.237 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829965 s, 126 MB/s 2024-04-25T09:02:41.246 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.246 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.247 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683369 s, 153 MB/s 2024-04-25T09:02:41.257 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.257 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.257 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.008074 s, 130 MB/s 2024-04-25T09:02:41.267 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.267 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.267 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757203 s, 138 MB/s 2024-04-25T09:02:41.276 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.276 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.276 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729722 s, 144 MB/s 2024-04-25T09:02:41.285 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.285 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.285 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072007 s, 146 MB/s 2024-04-25T09:02:41.295 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.295 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.295 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772466 s, 136 MB/s 2024-04-25T09:02:41.305 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.305 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.305 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722579 s, 145 MB/s 2024-04-25T09:02:41.314 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.314 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.314 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698676 s, 150 MB/s 2024-04-25T09:02:41.324 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.324 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.324 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750534 s, 140 MB/s 2024-04-25T09:02:41.333 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.333 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.333 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709703 s, 148 MB/s 2024-04-25T09:02:41.342 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.342 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.343 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00683358 s, 153 MB/s 2024-04-25T09:02:41.353 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.353 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.353 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838247 s, 125 MB/s 2024-04-25T09:02:41.362 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.363 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.363 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719567 s, 146 MB/s 2024-04-25T09:02:41.373 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.373 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.373 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00853604 s, 123 MB/s 2024-04-25T09:02:41.383 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.383 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.383 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714606 s, 147 MB/s 2024-04-25T09:02:41.393 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.394 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.394 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00871509 s, 120 MB/s 2024-04-25T09:02:41.403 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.404 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.404 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769877 s, 136 MB/s 2024-04-25T09:02:41.412 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.412 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.413 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687046 s, 153 MB/s 2024-04-25T09:02:41.422 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.422 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.422 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730847 s, 143 MB/s 2024-04-25T09:02:41.432 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.432 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.432 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738849 s, 142 MB/s 2024-04-25T09:02:41.441 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.441 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.442 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749013 s, 140 MB/s 2024-04-25T09:02:41.451 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.451 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.451 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076084 s, 138 MB/s 2024-04-25T09:02:41.460 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.460 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.460 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069836 s, 150 MB/s 2024-04-25T09:02:41.470 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.470 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.470 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740727 s, 142 MB/s 2024-04-25T09:02:41.479 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.479 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.480 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734215 s, 143 MB/s 2024-04-25T09:02:41.489 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.490 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.490 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754188 s, 139 MB/s 2024-04-25T09:02:41.498 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.499 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.499 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724465 s, 145 MB/s 2024-04-25T09:02:41.508 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.508 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.508 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701915 s, 149 MB/s 2024-04-25T09:02:41.518 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.518 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.518 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792492 s, 132 MB/s 2024-04-25T09:02:41.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:41.520 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:02:41.520 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:02:41.520 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T09:02:41.520 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:02:41.520 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:02:41.520 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:02:41.521 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:02:41.610 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.610 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.610 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778058 s, 135 MB/s 2024-04-25T09:02:41.620 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.620 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.620 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00768084 s, 137 MB/s 2024-04-25T09:02:41.630 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.630 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.630 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836307 s, 125 MB/s 2024-04-25T09:02:41.639 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.640 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.640 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694758 s, 151 MB/s 2024-04-25T09:02:41.650 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.650 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.650 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084626 s, 124 MB/s 2024-04-25T09:02:41.660 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.661 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.661 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836879 s, 125 MB/s 2024-04-25T09:02:41.669 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.670 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.670 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688829 s, 152 MB/s 2024-04-25T09:02:41.680 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.680 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.680 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815468 s, 129 MB/s 2024-04-25T09:02:41.691 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.691 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.691 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.008399 s, 125 MB/s 2024-04-25T09:02:41.701 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.702 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.702 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00854331 s, 123 MB/s 2024-04-25T09:02:41.711 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.711 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.711 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694336 s, 151 MB/s 2024-04-25T09:02:41.720 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.720 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.720 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741273 s, 141 MB/s 2024-04-25T09:02:41.731 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.731 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.731 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831414 s, 126 MB/s 2024-04-25T09:02:41.740 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.740 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.741 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736148 s, 142 MB/s 2024-04-25T09:02:41.751 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.751 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.751 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841933 s, 125 MB/s 2024-04-25T09:02:41.760 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.761 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.761 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719908 s, 146 MB/s 2024-04-25T09:02:41.770 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.770 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.770 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759806 s, 138 MB/s 2024-04-25T09:02:41.780 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.781 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.781 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755018 s, 139 MB/s 2024-04-25T09:02:41.791 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.792 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.792 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825422 s, 127 MB/s 2024-04-25T09:02:41.803 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.803 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.804 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00895736 s, 117 MB/s 2024-04-25T09:02:41.815 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.815 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.815 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00910248 s, 115 MB/s 2024-04-25T09:02:41.826 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.826 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.826 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00807678 s, 130 MB/s 2024-04-25T09:02:41.836 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.836 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.836 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823384 s, 127 MB/s 2024-04-25T09:02:41.847 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.847 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.847 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788394 s, 133 MB/s 2024-04-25T09:02:41.859 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.859 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.859 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00874833 s, 120 MB/s 2024-04-25T09:02:41.870 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.870 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.870 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00812669 s, 129 MB/s 2024-04-25T09:02:41.881 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.882 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.882 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00921007 s, 114 MB/s 2024-04-25T09:02:41.894 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.894 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.894 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00920744 s, 114 MB/s 2024-04-25T09:02:41.905 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.905 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.905 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00885775 s, 118 MB/s 2024-04-25T09:02:41.917 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.917 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.918 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00938948 s, 112 MB/s 2024-04-25T09:02:41.929 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.929 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.929 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00915519 s, 115 MB/s 2024-04-25T09:02:41.940 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.940 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.941 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00848757 s, 124 MB/s 2024-04-25T09:02:41.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:41.942 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:02:41.942 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:02:41.942 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T09:02:41.943 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:02:41.943 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:02:41.943 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:02:41.943 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:02:41.971 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.971 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.971 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734807 s, 143 MB/s 2024-04-25T09:02:41.981 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.981 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.981 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747913 s, 140 MB/s 2024-04-25T09:02:41.990 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:41.990 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:41.990 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730983 s, 143 MB/s 2024-04-25T09:02:42.000 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.000 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.000 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761615 s, 138 MB/s 2024-04-25T09:02:42.010 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.010 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.010 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827813 s, 127 MB/s 2024-04-25T09:02:42.020 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.020 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.020 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772255 s, 136 MB/s 2024-04-25T09:02:42.030 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.030 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.031 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740638 s, 142 MB/s 2024-04-25T09:02:42.040 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.040 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.040 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777735 s, 135 MB/s 2024-04-25T09:02:42.049 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.049 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.049 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720286 s, 146 MB/s 2024-04-25T09:02:42.059 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.059 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.060 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828544 s, 127 MB/s 2024-04-25T09:02:42.069 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.069 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.069 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741854 s, 141 MB/s 2024-04-25T09:02:42.079 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.080 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.080 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828145 s, 127 MB/s 2024-04-25T09:02:42.090 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.090 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.090 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822392 s, 128 MB/s 2024-04-25T09:02:42.099 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.099 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.099 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720064 s, 146 MB/s 2024-04-25T09:02:42.108 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.108 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.108 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701982 s, 149 MB/s 2024-04-25T09:02:42.118 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.118 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.118 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756801 s, 139 MB/s 2024-04-25T09:02:42.127 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.127 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.127 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717432 s, 146 MB/s 2024-04-25T09:02:42.137 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.137 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.137 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076909 s, 136 MB/s 2024-04-25T09:02:42.147 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.147 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.148 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826963 s, 127 MB/s 2024-04-25T09:02:42.158 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.158 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.158 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00812232 s, 129 MB/s 2024-04-25T09:02:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.168 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082185 s, 128 MB/s 2024-04-25T09:02:42.178 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.178 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.178 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075739 s, 138 MB/s 2024-04-25T09:02:42.188 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.188 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.188 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756174 s, 139 MB/s 2024-04-25T09:02:42.197 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.197 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.197 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728191 s, 144 MB/s 2024-04-25T09:02:42.207 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.207 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.208 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819253 s, 128 MB/s 2024-04-25T09:02:42.217 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.217 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.217 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729635 s, 144 MB/s 2024-04-25T09:02:42.227 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.228 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.228 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00843223 s, 124 MB/s 2024-04-25T09:02:42.238 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.238 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.238 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792168 s, 132 MB/s 2024-04-25T09:02:42.247 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.247 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.247 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688734 s, 152 MB/s 2024-04-25T09:02:42.257 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.257 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.257 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829976 s, 126 MB/s 2024-04-25T09:02:42.267 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.267 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.267 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740455 s, 142 MB/s 2024-04-25T09:02:42.277 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.277 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.277 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00781663 s, 134 MB/s 2024-04-25T09:02:42.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:42.279 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:02:42.279 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:02:42.279 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T09:02:42.279 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:02:42.279 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:02:42.279 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:02:42.279 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:02:42.346 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.346 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.346 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728118 s, 144 MB/s 2024-04-25T09:02:42.356 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.356 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.356 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828455 s, 127 MB/s 2024-04-25T09:02:42.367 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.367 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.367 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830148 s, 126 MB/s 2024-04-25T09:02:42.376 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.376 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.377 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748736 s, 140 MB/s 2024-04-25T09:02:42.387 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.387 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.387 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820369 s, 128 MB/s 2024-04-25T09:02:42.396 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.396 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.397 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734741 s, 143 MB/s 2024-04-25T09:02:42.407 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.407 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.407 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084016 s, 125 MB/s 2024-04-25T09:02:42.418 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.418 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.418 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00886961 s, 118 MB/s 2024-04-25T09:02:42.429 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.430 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.430 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075588 s, 139 MB/s 2024-04-25T09:02:42.438 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.438 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.438 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722038 s, 145 MB/s 2024-04-25T09:02:42.447 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.448 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.448 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747986 s, 140 MB/s 2024-04-25T09:02:42.458 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.458 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.458 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815588 s, 129 MB/s 2024-04-25T09:02:42.468 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.468 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.469 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00849393 s, 123 MB/s 2024-04-25T09:02:42.478 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.478 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.478 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733122 s, 143 MB/s 2024-04-25T09:02:42.490 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.491 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.491 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00862874 s, 122 MB/s 2024-04-25T09:02:42.499 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.499 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.499 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752922 s, 139 MB/s 2024-04-25T09:02:42.508 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.508 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.509 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736224 s, 142 MB/s 2024-04-25T09:02:42.519 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.519 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.519 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00837451 s, 125 MB/s 2024-04-25T09:02:42.529 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.529 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.529 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00782229 s, 134 MB/s 2024-04-25T09:02:42.540 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.540 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.540 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00858098 s, 122 MB/s 2024-04-25T09:02:42.550 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.550 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.550 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740838 s, 142 MB/s 2024-04-25T09:02:42.559 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.559 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.560 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743417 s, 141 MB/s 2024-04-25T09:02:42.570 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.570 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.570 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00863637 s, 121 MB/s 2024-04-25T09:02:42.586 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.586 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.586 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0134162 s, 78.2 MB/s 2024-04-25T09:02:42.624 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.624 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.624 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728187 s, 144 MB/s 2024-04-25T09:02:42.634 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.634 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.634 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814424 s, 129 MB/s 2024-04-25T09:02:42.645 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.645 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.645 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822509 s, 127 MB/s 2024-04-25T09:02:42.654 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.655 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.655 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742984 s, 141 MB/s 2024-04-25T09:02:42.664 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.664 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.664 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719946 s, 146 MB/s 2024-04-25T09:02:42.674 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.674 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.674 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727121 s, 144 MB/s 2024-04-25T09:02:42.683 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.683 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.684 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766693 s, 137 MB/s 2024-04-25T09:02:42.695 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.696 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.696 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733172 s, 143 MB/s 2024-04-25T09:02:42.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:42.696 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:02:42.696 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:02:42.696 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T09:02:42.696 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:02:42.696 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:02:42.696 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:02:42.697 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:02:42.765 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.766 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.766 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00874768 s, 120 MB/s 2024-04-25T09:02:42.775 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.775 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.775 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733519 s, 143 MB/s 2024-04-25T09:02:42.785 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.785 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.785 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774905 s, 135 MB/s 2024-04-25T09:02:42.794 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.794 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.795 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072185 s, 145 MB/s 2024-04-25T09:02:42.805 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.805 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.805 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00803402 s, 131 MB/s 2024-04-25T09:02:42.815 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.815 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.815 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746979 s, 140 MB/s 2024-04-25T09:02:42.826 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.826 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.826 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757791 s, 138 MB/s 2024-04-25T09:02:42.835 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.835 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.835 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736015 s, 142 MB/s 2024-04-25T09:02:42.845 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.845 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.845 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752253 s, 139 MB/s 2024-04-25T09:02:42.855 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.855 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.855 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723565 s, 145 MB/s 2024-04-25T09:02:42.864 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.864 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.865 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756057 s, 139 MB/s 2024-04-25T09:02:42.874 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.874 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.875 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819331 s, 128 MB/s 2024-04-25T09:02:42.885 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.885 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.885 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761858 s, 138 MB/s 2024-04-25T09:02:42.894 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.894 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.894 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736401 s, 142 MB/s 2024-04-25T09:02:42.904 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.904 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.904 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838854 s, 125 MB/s 2024-04-25T09:02:42.915 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.915 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.915 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074349 s, 141 MB/s 2024-04-25T09:02:42.924 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.924 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.924 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00794536 s, 132 MB/s 2024-04-25T09:02:42.934 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.935 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.935 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825605 s, 127 MB/s 2024-04-25T09:02:42.945 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.946 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.946 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834476 s, 126 MB/s 2024-04-25T09:02:42.956 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.956 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.956 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00896163 s, 117 MB/s 2024-04-25T09:02:42.966 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.966 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.966 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721495 s, 145 MB/s 2024-04-25T09:02:42.975 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.975 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.975 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717268 s, 146 MB/s 2024-04-25T09:02:42.986 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.986 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.986 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833555 s, 126 MB/s 2024-04-25T09:02:42.995 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:42.995 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:42.996 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718917 s, 146 MB/s 2024-04-25T09:02:43.004 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.004 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.004 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701786 s, 149 MB/s 2024-04-25T09:02:43.014 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.014 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.014 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752174 s, 139 MB/s 2024-04-25T09:02:43.024 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.024 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.024 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00782201 s, 134 MB/s 2024-04-25T09:02:43.035 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.035 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.035 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0087424 s, 120 MB/s 2024-04-25T09:02:43.045 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.045 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.045 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745504 s, 141 MB/s 2024-04-25T09:02:43.055 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.055 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.055 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760976 s, 138 MB/s 2024-04-25T09:02:43.065 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.066 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.066 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00837397 s, 125 MB/s 2024-04-25T09:02:43.076 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.076 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.077 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690851 s, 152 MB/s 2024-04-25T09:02:43.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:43.077 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:02:43.077 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:02:43.077 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T09:02:43.077 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:02:43.077 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:02:43.077 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:02:43.077 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:02:43.144 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.144 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.144 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751467 s, 140 MB/s 2024-04-25T09:02:43.153 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.153 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.153 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688407 s, 152 MB/s 2024-04-25T09:02:43.162 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.163 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.163 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682412 s, 154 MB/s 2024-04-25T09:02:43.171 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.172 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.172 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677793 s, 155 MB/s 2024-04-25T09:02:43.182 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.182 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.182 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819781 s, 128 MB/s 2024-04-25T09:02:43.192 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.193 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.193 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828338 s, 127 MB/s 2024-04-25T09:02:43.202 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.202 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.202 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753915 s, 139 MB/s 2024-04-25T09:02:43.211 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.212 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.212 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688823 s, 152 MB/s 2024-04-25T09:02:43.222 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.222 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.222 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829235 s, 126 MB/s 2024-04-25T09:02:43.231 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.232 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.232 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735982 s, 142 MB/s 2024-04-25T09:02:43.242 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.242 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.243 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838539 s, 125 MB/s 2024-04-25T09:02:43.252 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.252 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.252 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724193 s, 145 MB/s 2024-04-25T09:02:43.261 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.261 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.262 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737013 s, 142 MB/s 2024-04-25T09:02:43.270 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.270 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.271 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684722 s, 153 MB/s 2024-04-25T09:02:43.280 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.280 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.280 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702904 s, 149 MB/s 2024-04-25T09:02:43.289 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.289 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.289 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671586 s, 156 MB/s 2024-04-25T09:02:43.299 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.299 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.299 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822078 s, 128 MB/s 2024-04-25T09:02:43.312 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.312 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.313 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749803 s, 140 MB/s 2024-04-25T09:02:43.318 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.319 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.319 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698652 s, 150 MB/s 2024-04-25T09:02:43.328 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.328 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.329 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750796 s, 140 MB/s 2024-04-25T09:02:43.338 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.339 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.339 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827707 s, 127 MB/s 2024-04-25T09:02:43.348 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.348 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.348 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679651 s, 154 MB/s 2024-04-25T09:02:43.357 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.357 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.357 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074469 s, 141 MB/s 2024-04-25T09:02:43.366 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.366 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.366 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669484 s, 157 MB/s 2024-04-25T09:02:43.375 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.375 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.376 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725218 s, 145 MB/s 2024-04-25T09:02:43.385 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.385 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.386 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734268 s, 143 MB/s 2024-04-25T09:02:43.394 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.395 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.395 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071239 s, 147 MB/s 2024-04-25T09:02:43.405 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.405 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.405 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833594 s, 126 MB/s 2024-04-25T09:02:43.415 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.415 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.415 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0078343 s, 134 MB/s 2024-04-25T09:02:43.424 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.424 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.424 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697097 s, 150 MB/s 2024-04-25T09:02:43.434 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.434 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.435 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007444 s, 141 MB/s 2024-04-25T09:02:43.443 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.443 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.443 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682154 s, 154 MB/s 2024-04-25T09:02:43.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:43.444 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:02:43.445 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:02:43.445 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T09:02:43.445 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:02:43.445 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:02:43.445 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:02:43.445 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:02:43.515 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.516 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.517 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068908 s, 152 MB/s 2024-04-25T09:02:43.521 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.521 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.522 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746694 s, 140 MB/s 2024-04-25T09:02:43.532 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.532 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.532 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841382 s, 125 MB/s 2024-04-25T09:02:43.543 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.543 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.543 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845521 s, 124 MB/s 2024-04-25T09:02:43.552 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.552 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.552 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713476 s, 147 MB/s 2024-04-25T09:02:43.562 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.562 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.562 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074697 s, 140 MB/s 2024-04-25T09:02:43.572 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.572 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.572 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084134 s, 125 MB/s 2024-04-25T09:02:43.586 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.587 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.587 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121072 s, 86.6 MB/s 2024-04-25T09:02:43.635 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.635 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.636 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667848 s, 157 MB/s 2024-04-25T09:02:43.645 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.645 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.645 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751417 s, 140 MB/s 2024-04-25T09:02:43.654 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.654 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.655 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732227 s, 143 MB/s 2024-04-25T09:02:43.665 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.665 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.665 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0081866 s, 128 MB/s 2024-04-25T09:02:43.675 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.676 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.676 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830571 s, 126 MB/s 2024-04-25T09:02:43.685 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.685 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.685 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742597 s, 141 MB/s 2024-04-25T09:02:43.694 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.694 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.694 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665633 s, 158 MB/s 2024-04-25T09:02:43.703 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.703 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.703 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720484 s, 146 MB/s 2024-04-25T09:02:43.713 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.713 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.713 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738662 s, 142 MB/s 2024-04-25T09:02:43.723 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.723 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.723 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00807376 s, 130 MB/s 2024-04-25T09:02:43.733 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.733 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.733 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748856 s, 140 MB/s 2024-04-25T09:02:43.743 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.743 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.743 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736198 s, 142 MB/s 2024-04-25T09:02:43.752 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.753 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.753 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715122 s, 147 MB/s 2024-04-25T09:02:43.761 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.762 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.762 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681399 s, 154 MB/s 2024-04-25T09:02:43.771 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.771 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.771 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705062 s, 149 MB/s 2024-04-25T09:02:43.781 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.781 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.781 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793411 s, 132 MB/s 2024-04-25T09:02:43.790 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.791 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.791 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731491 s, 143 MB/s 2024-04-25T09:02:43.801 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.801 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.801 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829784 s, 126 MB/s 2024-04-25T09:02:43.810 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.811 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.811 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731206 s, 143 MB/s 2024-04-25T09:02:43.820 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.820 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.821 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769065 s, 136 MB/s 2024-04-25T09:02:43.830 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.830 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.830 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778319 s, 135 MB/s 2024-04-25T09:02:43.839 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.840 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.840 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710883 s, 148 MB/s 2024-04-25T09:02:43.849 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.849 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.849 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691491 s, 152 MB/s 2024-04-25T09:02:43.860 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.860 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.860 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00865967 s, 121 MB/s 2024-04-25T09:02:43.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:43.861 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:02:43.861 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:02:43.861 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T09:02:43.861 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:02:43.861 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:02:43.861 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:02:43.861 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:02:43.930 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.930 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.930 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754602 s, 139 MB/s 2024-04-25T09:02:43.940 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.940 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.940 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850252 s, 123 MB/s 2024-04-25T09:02:43.950 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.950 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.950 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709632 s, 148 MB/s 2024-04-25T09:02:43.959 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.959 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.959 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705474 s, 149 MB/s 2024-04-25T09:02:43.970 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.970 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.970 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00870736 s, 120 MB/s 2024-04-25T09:02:43.980 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.980 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.980 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00779076 s, 135 MB/s 2024-04-25T09:02:43.992 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:43.993 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:43.993 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791969 s, 132 MB/s 2024-04-25T09:02:43.999 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.000 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.000 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070796 s, 148 MB/s 2024-04-25T09:02:44.009 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.010 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.010 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783982 s, 134 MB/s 2024-04-25T09:02:44.020 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.020 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.020 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767895 s, 137 MB/s 2024-04-25T09:02:44.029 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.030 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.030 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742701 s, 141 MB/s 2024-04-25T09:02:44.039 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.039 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.040 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757348 s, 138 MB/s 2024-04-25T09:02:44.049 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.049 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.049 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754727 s, 139 MB/s 2024-04-25T09:02:44.058 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.058 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.059 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739364 s, 142 MB/s 2024-04-25T09:02:44.068 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.068 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.068 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740256 s, 142 MB/s 2024-04-25T09:02:44.078 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.078 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.078 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762558 s, 138 MB/s 2024-04-25T09:02:44.088 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.088 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.088 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747877 s, 140 MB/s 2024-04-25T09:02:44.098 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.098 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.098 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746365 s, 140 MB/s 2024-04-25T09:02:44.107 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.108 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.108 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076706 s, 137 MB/s 2024-04-25T09:02:44.117 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.117 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.117 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692607 s, 151 MB/s 2024-04-25T09:02:44.126 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.126 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.126 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701992 s, 149 MB/s 2024-04-25T09:02:44.136 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.136 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.136 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725605 s, 145 MB/s 2024-04-25T09:02:44.145 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.145 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.145 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705192 s, 149 MB/s 2024-04-25T09:02:44.155 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.155 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.155 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752147 s, 139 MB/s 2024-04-25T09:02:44.164 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.165 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.165 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007405 s, 142 MB/s 2024-04-25T09:02:44.174 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.174 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.174 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702514 s, 149 MB/s 2024-04-25T09:02:44.184 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.184 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.184 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749192 s, 140 MB/s 2024-04-25T09:02:44.193 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.194 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.194 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751984 s, 139 MB/s 2024-04-25T09:02:44.204 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.204 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.204 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00871779 s, 120 MB/s 2024-04-25T09:02:44.213 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.213 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.213 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698322 s, 150 MB/s 2024-04-25T09:02:44.223 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.223 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.223 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721249 s, 145 MB/s 2024-04-25T09:02:44.232 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.233 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.233 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739254 s, 142 MB/s 2024-04-25T09:02:44.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:44.234 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:02:44.234 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:02:44.234 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T09:02:44.234 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:02:44.234 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:02:44.234 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:02:44.234 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:02:44.302 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.302 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.302 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075586 s, 139 MB/s 2024-04-25T09:02:44.312 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.313 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.313 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713164 s, 147 MB/s 2024-04-25T09:02:44.322 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.322 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.323 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00897528 s, 117 MB/s 2024-04-25T09:02:44.332 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.333 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.333 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075959 s, 138 MB/s 2024-04-25T09:02:44.342 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.342 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.342 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737115 s, 142 MB/s 2024-04-25T09:02:44.352 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.352 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.352 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754355 s, 139 MB/s 2024-04-25T09:02:44.363 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.363 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.363 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846168 s, 124 MB/s 2024-04-25T09:02:44.373 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.373 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.373 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706474 s, 148 MB/s 2024-04-25T09:02:44.381 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.381 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.381 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689178 s, 152 MB/s 2024-04-25T09:02:44.390 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.391 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.391 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709239 s, 148 MB/s 2024-04-25T09:02:44.400 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.400 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.400 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703189 s, 149 MB/s 2024-04-25T09:02:44.409 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.410 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.410 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736879 s, 142 MB/s 2024-04-25T09:02:44.419 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.419 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.419 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734928 s, 143 MB/s 2024-04-25T09:02:44.429 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.429 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.429 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748003 s, 140 MB/s 2024-04-25T09:02:44.439 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.439 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.439 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0083525 s, 126 MB/s 2024-04-25T09:02:44.449 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.450 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.450 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758255 s, 138 MB/s 2024-04-25T09:02:44.459 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.459 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.459 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767159 s, 137 MB/s 2024-04-25T09:02:44.470 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.470 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.470 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00859579 s, 122 MB/s 2024-04-25T09:02:44.479 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.479 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.479 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730063 s, 144 MB/s 2024-04-25T09:02:44.489 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.489 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.489 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729079 s, 144 MB/s 2024-04-25T09:02:44.498 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.498 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.498 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067994 s, 154 MB/s 2024-04-25T09:02:44.508 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.508 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.509 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00847152 s, 124 MB/s 2024-04-25T09:02:44.518 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.519 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.519 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718263 s, 146 MB/s 2024-04-25T09:02:44.527 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.527 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.527 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690598 s, 152 MB/s 2024-04-25T09:02:44.537 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.537 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.537 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757494 s, 138 MB/s 2024-04-25T09:02:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713963 s, 147 MB/s 2024-04-25T09:02:44.559 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.559 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.559 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836755 s, 125 MB/s 2024-04-25T09:02:44.566 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.567 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.567 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735029 s, 143 MB/s 2024-04-25T09:02:44.577 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.577 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.577 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833254 s, 126 MB/s 2024-04-25T09:02:44.590 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.590 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.590 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0108433 s, 96.7 MB/s 2024-04-25T09:02:44.626 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.627 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.627 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756716 s, 139 MB/s 2024-04-25T09:02:44.636 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.636 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.636 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749787 s, 140 MB/s 2024-04-25T09:02:44.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:44.638 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:02:44.638 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:02:44.638 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T09:02:44.638 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:02:44.638 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:02:44.638 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:02:44.638 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:02:44.706 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.706 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.706 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763159 s, 137 MB/s 2024-04-25T09:02:44.716 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.716 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.717 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00856628 s, 122 MB/s 2024-04-25T09:02:44.726 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.726 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.726 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744341 s, 141 MB/s 2024-04-25T09:02:44.736 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.736 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.737 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835537 s, 125 MB/s 2024-04-25T09:02:44.746 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.746 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.746 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752453 s, 139 MB/s 2024-04-25T09:02:44.755 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.755 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.755 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701501 s, 149 MB/s 2024-04-25T09:02:44.766 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.766 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.766 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00890704 s, 118 MB/s 2024-04-25T09:02:44.776 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.776 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.776 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717872 s, 146 MB/s 2024-04-25T09:02:44.786 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.786 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.786 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850666 s, 123 MB/s 2024-04-25T09:02:44.796 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.796 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.796 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726703 s, 144 MB/s 2024-04-25T09:02:44.804 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.805 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.805 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674849 s, 155 MB/s 2024-04-25T09:02:44.815 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.815 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.815 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811959 s, 129 MB/s 2024-04-25T09:02:44.824 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.825 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.825 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073746 s, 142 MB/s 2024-04-25T09:02:44.835 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.835 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.835 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822113 s, 128 MB/s 2024-04-25T09:02:44.845 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.846 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.846 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717023 s, 146 MB/s 2024-04-25T09:02:44.854 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.854 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.854 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718726 s, 146 MB/s 2024-04-25T09:02:44.864 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.864 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.864 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783421 s, 134 MB/s 2024-04-25T09:02:44.873 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.873 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.873 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678839 s, 154 MB/s 2024-04-25T09:02:44.884 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.884 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.884 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845787 s, 124 MB/s 2024-04-25T09:02:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.894 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.894 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767739 s, 137 MB/s 2024-04-25T09:02:44.903 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.903 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.903 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718443 s, 146 MB/s 2024-04-25T09:02:44.913 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.913 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.913 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742552 s, 141 MB/s 2024-04-25T09:02:44.923 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.923 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.923 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745021 s, 141 MB/s 2024-04-25T09:02:44.932 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.932 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.932 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729763 s, 144 MB/s 2024-04-25T09:02:44.941 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.941 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.941 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728825 s, 144 MB/s 2024-04-25T09:02:44.952 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.952 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.952 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0083703 s, 125 MB/s 2024-04-25T09:02:44.962 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.962 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.963 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821237 s, 128 MB/s 2024-04-25T09:02:44.971 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.972 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.972 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069634 s, 151 MB/s 2024-04-25T09:02:44.982 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.982 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.982 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792698 s, 132 MB/s 2024-04-25T09:02:44.991 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:44.991 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:44.991 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719518 s, 146 MB/s 2024-04-25T09:02:45.000 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.001 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.001 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731104 s, 143 MB/s 2024-04-25T09:02:45.011 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.011 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.011 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827092 s, 127 MB/s 2024-04-25T09:02:45.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:45.012 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:02:45.012 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:02:45.013 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T09:02:45.013 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:02:45.013 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:02:45.013 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:02:45.013 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:02:45.080 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.080 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.081 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074511 s, 141 MB/s 2024-04-25T09:02:45.091 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.091 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.091 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822932 s, 127 MB/s 2024-04-25T09:02:45.100 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.100 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.101 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699741 s, 150 MB/s 2024-04-25T09:02:45.111 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.111 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.111 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0085357 s, 123 MB/s 2024-04-25T09:02:45.120 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.120 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.121 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741507 s, 141 MB/s 2024-04-25T09:02:45.130 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.131 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.131 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00785485 s, 133 MB/s 2024-04-25T09:02:45.140 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.140 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.140 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753217 s, 139 MB/s 2024-04-25T09:02:45.151 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.151 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.151 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838299 s, 125 MB/s 2024-04-25T09:02:45.161 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.162 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.162 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00781065 s, 134 MB/s 2024-04-25T09:02:45.171 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.171 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.171 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788811 s, 133 MB/s 2024-04-25T09:02:45.180 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.180 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.180 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717174 s, 146 MB/s 2024-04-25T09:02:45.190 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.190 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.190 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755708 s, 139 MB/s 2024-04-25T09:02:45.199 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.200 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.200 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682554 s, 154 MB/s 2024-04-25T09:02:45.209 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.209 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.209 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705101 s, 149 MB/s 2024-04-25T09:02:45.219 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.219 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.219 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788608 s, 133 MB/s 2024-04-25T09:02:45.228 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.228 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.228 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694024 s, 151 MB/s 2024-04-25T09:02:45.238 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.238 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.238 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816268 s, 128 MB/s 2024-04-25T09:02:45.247 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.248 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.248 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069656 s, 151 MB/s 2024-04-25T09:02:45.258 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.259 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.259 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084017 s, 125 MB/s 2024-04-25T09:02:45.268 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.269 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.269 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0081765 s, 128 MB/s 2024-04-25T09:02:45.277 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.278 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.278 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680404 s, 154 MB/s 2024-04-25T09:02:45.288 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.288 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.288 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830529 s, 126 MB/s 2024-04-25T09:02:45.297 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.297 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.297 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704911 s, 149 MB/s 2024-04-25T09:02:45.307 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.307 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.307 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705488 s, 149 MB/s 2024-04-25T09:02:45.316 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.316 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.316 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733854 s, 143 MB/s 2024-04-25T09:02:45.326 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.326 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.326 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725761 s, 144 MB/s 2024-04-25T09:02:45.334 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.335 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.335 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677327 s, 155 MB/s 2024-04-25T09:02:45.343 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.344 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.344 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672792 s, 156 MB/s 2024-04-25T09:02:45.352 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.353 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.353 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669261 s, 157 MB/s 2024-04-25T09:02:45.363 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.363 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.363 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00812874 s, 129 MB/s 2024-04-25T09:02:45.373 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.373 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.373 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0080952 s, 130 MB/s 2024-04-25T09:02:45.382 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.383 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.383 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695581 s, 151 MB/s 2024-04-25T09:02:45.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:45.384 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:02:45.384 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:02:45.384 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T09:02:45.384 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:02:45.384 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:02:45.384 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:02:45.384 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:02:45.451 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.451 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.451 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717551 s, 146 MB/s 2024-04-25T09:02:45.460 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.460 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.460 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692189 s, 151 MB/s 2024-04-25T09:02:45.470 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.470 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.470 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740058 s, 142 MB/s 2024-04-25T09:02:45.479 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.479 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.479 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678558 s, 155 MB/s 2024-04-25T09:02:45.489 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.489 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.490 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836899 s, 125 MB/s 2024-04-25T09:02:45.499 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.499 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.500 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00775563 s, 135 MB/s 2024-04-25T09:02:45.509 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.509 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.509 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761537 s, 138 MB/s 2024-04-25T09:02:45.519 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.519 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.519 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761521 s, 138 MB/s 2024-04-25T09:02:45.528 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.528 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.529 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722789 s, 145 MB/s 2024-04-25T09:02:45.538 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.538 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.538 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757788 s, 138 MB/s 2024-04-25T09:02:45.548 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.548 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.548 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724866 s, 145 MB/s 2024-04-25T09:02:45.558 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.558 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.558 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072892 s, 144 MB/s 2024-04-25T09:02:45.568 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.568 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.568 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832647 s, 126 MB/s 2024-04-25T09:02:45.578 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.578 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.578 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738146 s, 142 MB/s 2024-04-25T09:02:45.588 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.588 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.588 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748883 s, 140 MB/s 2024-04-25T09:02:45.603 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.604 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.604 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00914767 s, 115 MB/s 2024-04-25T09:02:45.641 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.641 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.641 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694352 s, 151 MB/s 2024-04-25T09:02:45.651 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.651 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.651 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751892 s, 139 MB/s 2024-04-25T09:02:45.662 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.662 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.662 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00812031 s, 129 MB/s 2024-04-25T09:02:45.672 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.672 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.672 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738172 s, 142 MB/s 2024-04-25T09:02:45.683 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.683 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.683 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00860699 s, 122 MB/s 2024-04-25T09:02:45.693 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.693 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.693 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746035 s, 141 MB/s 2024-04-25T09:02:45.704 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.704 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.704 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00837564 s, 125 MB/s 2024-04-25T09:02:45.713 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.713 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.714 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719871 s, 146 MB/s 2024-04-25T09:02:45.723 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.723 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.723 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709048 s, 148 MB/s 2024-04-25T09:02:45.733 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.733 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.733 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716202 s, 146 MB/s 2024-04-25T09:02:45.743 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.743 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.743 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747403 s, 140 MB/s 2024-04-25T09:02:45.753 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.753 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.753 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748291 s, 140 MB/s 2024-04-25T09:02:45.764 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.764 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.764 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677583 s, 155 MB/s 2024-04-25T09:02:45.772 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.772 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.772 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694642 s, 151 MB/s 2024-04-25T09:02:45.782 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.782 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.782 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747393 s, 140 MB/s 2024-04-25T09:02:45.793 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.793 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.793 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082281 s, 127 MB/s 2024-04-25T09:02:45.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:45.794 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:02:45.794 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:02:45.794 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T09:02:45.794 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:02:45.794 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:02:45.795 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:02:45.795 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:02:45.865 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.865 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.865 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00852356 s, 123 MB/s 2024-04-25T09:02:45.875 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.875 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.876 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00803733 s, 130 MB/s 2024-04-25T09:02:45.887 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.887 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.887 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0089138 s, 118 MB/s 2024-04-25T09:02:45.896 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.896 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.896 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703788 s, 149 MB/s 2024-04-25T09:02:45.906 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.906 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.906 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708297 s, 148 MB/s 2024-04-25T09:02:45.915 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.915 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.915 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717627 s, 146 MB/s 2024-04-25T09:02:45.927 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.927 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.928 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836051 s, 125 MB/s 2024-04-25T09:02:45.936 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.936 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.936 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740744 s, 142 MB/s 2024-04-25T09:02:45.947 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.947 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.947 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839342 s, 125 MB/s 2024-04-25T09:02:45.957 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.957 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.957 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727197 s, 144 MB/s 2024-04-25T09:02:45.967 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.967 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.968 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750451 s, 140 MB/s 2024-04-25T09:02:45.976 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.976 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.976 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067356 s, 156 MB/s 2024-04-25T09:02:45.987 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.987 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.987 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0086477 s, 121 MB/s 2024-04-25T09:02:45.997 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:45.997 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:45.997 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743402 s, 141 MB/s 2024-04-25T09:02:46.007 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:46.007 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:46.007 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069142 s, 152 MB/s 2024-04-25T09:02:46.017 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:46.017 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:46.018 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826637 s, 127 MB/s 2024-04-25T09:02:46.028 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:46.029 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:46.029 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851931 s, 123 MB/s 2024-04-25T09:02:46.038 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:46.038 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:46.038 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700383 s, 150 MB/s 2024-04-25T09:02:46.048 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:46.048 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:46.048 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738942 s, 142 MB/s 2024-04-25T09:02:46.054 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:02:46.056 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:02:46.058 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:46.058 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:46.058 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723135 s, 145 MB/s 2024-04-25T09:02:46.067 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:46.068 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:46.068 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755628 s, 139 MB/s 2024-04-25T09:02:46.079 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:46.079 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:46.079 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00910388 s, 115 MB/s 2024-04-25T09:02:46.089 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:46.089 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:46.089 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749259 s, 140 MB/s 2024-04-25T09:02:46.099 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:46.099 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:46.099 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707102 s, 148 MB/s 2024-04-25T09:02:46.110 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:46.110 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:46.110 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861851 s, 122 MB/s 2024-04-25T09:02:46.120 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:46.120 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:46.120 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686462 s, 153 MB/s 2024-04-25T09:02:46.129 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:46.129 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:46.129 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695297 s, 151 MB/s 2024-04-25T09:02:46.139 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:46.139 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:46.139 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735473 s, 143 MB/s 2024-04-25T09:02:46.151 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:46.151 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:46.151 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00875065 s, 120 MB/s 2024-04-25T09:02:46.160 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:46.160 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:46.160 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007066 s, 148 MB/s 2024-04-25T09:02:46.171 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:46.171 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:46.171 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826709 s, 127 MB/s 2024-04-25T09:02:46.181 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:02:46.181 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:02:46.181 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733055 s, 143 MB/s 2024-04-25T09:02:46.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:46.182 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub start /scrub_control_test_path recursive 2024-04-25T09:02:46.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.339+0000 7fa7e2153640 1 Processor -- start 2024-04-25T09:02:46.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.339+0000 7fa7e2153640 1 -- start start 2024-04-25T09:02:46.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.340+0000 7fa7e2153640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7dc0d3760 0x7fa7dc0d3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.340+0000 7fa7e2153640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7dc15e260 con 0x7fa7dc0b06c0 2024-04-25T09:02:46.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.340+0000 7fa7e2153640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7dc0b52a0 con 0x7fa7dc15de80 2024-04-25T09:02:46.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.340+0000 7fa7e2153640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7dc0d4100 con 0x7fa7dc0d3760 2024-04-25T09:02:46.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.340+0000 7fa7dbfff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa7dc0b06c0 0x7fa7dc0b0b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48420/0 (socket says 172.21.15.43:48420) 2024-04-25T09:02:46.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.340+0000 7fa7dbfff640 1 -- 172.21.15.43:0/3928111529 learned_addr learned my addr 172.21.15.43:0/3928111529 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:46.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.340+0000 7fa7daffd640 1 --2- 172.21.15.43:0/3928111529 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7dc0d3760 0x7fa7dc0d3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.340+0000 7fa7da7fc640 1 -- 172.21.15.43:0/3928111529 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2730618023 0 0) 0x7fa7dc15e260 con 0x7fa7dc0b06c0 2024-04-25T09:02:46.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.341+0000 7fa7da7fc640 1 -- 172.21.15.43:0/3928111529 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7c4003520 con 0x7fa7dc0b06c0 2024-04-25T09:02:46.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.341+0000 7fa7da7fc640 1 -- 172.21.15.43:0/3928111529 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1045200417 0 0) 0x7fa7dc0b52a0 con 0x7fa7dc15de80 2024-04-25T09:02:46.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.341+0000 7fa7da7fc640 1 -- 172.21.15.43:0/3928111529 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7dc15e260 con 0x7fa7dc15de80 2024-04-25T09:02:46.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.341+0000 7fa7daffd640 1 -- 172.21.15.43:0/3928111529 >> v1:172.21.15.193:6789/0 conn(0x7fa7dc15de80 legacy=0x7fa7dc0b4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.341+0000 7fa7daffd640 1 -- 172.21.15.43:0/3928111529 >> v1:172.21.15.43:6789/0 conn(0x7fa7dc0b06c0 legacy=0x7fa7dc0b0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.341+0000 7fa7daffd640 1 -- 172.21.15.43:0/3928111529 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7dc161970 con 0x7fa7dc0d3760 2024-04-25T09:02:46.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.341+0000 7fa7daffd640 1 --2- 172.21.15.43:0/3928111529 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7dc0d3760 0x7fa7dc0d3bc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa7d00097d0 tx=0x7fa7d002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ab08155c9362af4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:46.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.342+0000 7fa7da7fc640 1 -- 172.21.15.43:0/3928111529 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0037070 con 0x7fa7dc0d3760 2024-04-25T09:02:46.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.342+0000 7fa7da7fc640 1 -- 172.21.15.43:0/3928111529 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d0032440 con 0x7fa7dc0d3760 2024-04-25T09:02:46.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.342+0000 7fa7da7fc640 1 -- 172.21.15.43:0/3928111529 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0032810 con 0x7fa7dc0d3760 2024-04-25T09:02:46.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.342+0000 7fa7e2153640 1 -- 172.21.15.43:0/3928111529 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7dc0d3760 msgr2=0x7fa7dc0d3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.342+0000 7fa7e2153640 1 --2- 172.21.15.43:0/3928111529 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7dc0d3760 0x7fa7dc0d3bc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa7d00097d0 tx=0x7fa7d002a180 comp rx=0 tx=0).stop 2024-04-25T09:02:46.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.342+0000 7fa7e2153640 1 -- 172.21.15.43:0/3928111529 shutdown_connections 2024-04-25T09:02:46.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.342+0000 7fa7e2153640 1 --2- 172.21.15.43:0/3928111529 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7dc0d3760 0x7fa7dc0d3bc0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.342+0000 7fa7e2153640 1 -- 172.21.15.43:0/3928111529 >> 172.21.15.43:0/3928111529 conn(0x7fa7dc08d460 msgr2=0x7fa7dc09cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:46.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.342+0000 7fa7e2153640 1 -- 172.21.15.43:0/3928111529 shutdown_connections 2024-04-25T09:02:46.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.342+0000 7fa7e2153640 1 -- 172.21.15.43:0/3928111529 wait complete. 2024-04-25T09:02:46.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.343+0000 7fa7e2153640 1 Processor -- start 2024-04-25T09:02:46.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.344+0000 7fa7e2153640 1 -- start start 2024-04-25T09:02:46.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.344+0000 7fa7e2153640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7dc0b06c0 0x7fa7dc1f81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.344+0000 7fa7e2153640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7dc0d3760 0x7fa7dc160c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.344+0000 7fa7e2153640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7dc15de80 0x7fa7dc0b2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.344+0000 7fa7e2153640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7dc162210 con 0x7fa7dc15de80 2024-04-25T09:02:46.334 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.344+0000 7fa7e2153640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7dc161f30 con 0x7fa7dc0b06c0 2024-04-25T09:02:46.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.344+0000 7fa7e2153640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7dc1624f0 con 0x7fa7dc0d3760 2024-04-25T09:02:46.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.344+0000 7fa7dbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7dc15de80 0x7fa7dc0b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.344+0000 7fa7dbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7dc15de80 0x7fa7dc0b2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58694/0 (socket says 172.21.15.43:58694) 2024-04-25T09:02:46.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.344+0000 7fa7db7fe640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7dc0b06c0 0x7fa7dc1f81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.344+0000 7fa7daffd640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7dc0d3760 0x7fa7dc160c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.344+0000 7fa7dbfff640 1 -- 172.21.15.43:0/2574472872 learned_addr learned my addr 172.21.15.43:0/2574472872 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:46.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.345+0000 7fa7daffd640 1 -- 172.21.15.43:0/2574472872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7dc0b06c0 msgr2=0x7fa7dc1f81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.345+0000 7fa7daffd640 1 --2- 172.21.15.43:0/2574472872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7dc0b06c0 0x7fa7dc1f81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.345+0000 7fa7daffd640 1 -- 172.21.15.43:0/2574472872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7dc15de80 msgr2=0x7fa7dc0b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.345+0000 7fa7daffd640 1 --2- 172.21.15.43:0/2574472872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7dc15de80 0x7fa7dc0b2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.345+0000 7fa7daffd640 1 -- 172.21.15.43:0/2574472872 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7dc0cadb0 con 0x7fa7dc0d3760 2024-04-25T09:02:46.335 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.345+0000 7fa7daffd640 1 --2- 172.21.15.43:0/2574472872 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7dc0d3760 0x7fa7dc160c80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc08e170 tx=0x7fa7d00046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:46.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.345+0000 7fa7c3fff640 1 -- 172.21.15.43:0/2574472872 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0041030 con 0x7fa7dc0d3760 2024-04-25T09:02:46.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.345+0000 7fa7c3fff640 1 -- 172.21.15.43:0/2574472872 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d003d4e0 con 0x7fa7dc0d3760 2024-04-25T09:02:46.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.345+0000 7fa7e2153640 1 -- 172.21.15.43:0/2574472872 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7dc0cb090 con 0x7fa7dc0d3760 2024-04-25T09:02:46.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.345+0000 7fa7c3fff640 1 -- 172.21.15.43:0/2574472872 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0037040 con 0x7fa7dc0d3760 2024-04-25T09:02:46.336 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.345+0000 7fa7e2153640 1 -- 172.21.15.43:0/2574472872 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7dc0b2580 con 0x7fa7dc0d3760 2024-04-25T09:02:46.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.347+0000 7fa7c3fff640 1 -- 172.21.15.43:0/2574472872 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa7d003d8f0 con 0x7fa7dc0d3760 2024-04-25T09:02:46.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.348+0000 7fa7c3fff640 1 --2- 172.21.15.43:0/2574472872 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa7c806b4d0 0x7fa7c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.348+0000 7fa7c3fff640 1 -- 172.21.15.43:0/2574472872 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa7d00aa250 con 0x7fa7dc0d3760 2024-04-25T09:02:46.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.348+0000 7fa7db7fe640 1 --2- 172.21.15.43:0/2574472872 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa7c806b4d0 0x7fa7c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.348+0000 7fa7e4e55740 1 Processor -- start 2024-04-25T09:02:46.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.348+0000 7fa7db7fe640 1 --2- 172.21.15.43:0/2574472872 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa7c806b4d0 0x7fa7c806d990 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc007fd0 tx=0x7fa7cc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:46.337 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.348+0000 7fa7e4e55740 1 -- start start 2024-04-25T09:02:46.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.349+0000 7fa7e4e55740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562619012200 0x562619014360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.349+0000 7fa7e4e55740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x56261904e8d0 0x5626190125e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.349+0000 7fa7e4e55740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56261903ccc0 0x56261903d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.349+0000 7fa7e4e55740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x562618f76000 con 0x562619012200 2024-04-25T09:02:46.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.349+0000 7fa7e4e55740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x562618f1ef20 con 0x56261903ccc0 2024-04-25T09:02:46.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.349+0000 7fa7e4e55740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x562618f720a0 con 0x56261904e8d0 2024-04-25T09:02:46.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.349+0000 7fa7db7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562619012200 0x562619014360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.349+0000 7fa7db7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562619012200 0x562619014360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58700/0 (socket says 172.21.15.43:58700) 2024-04-25T09:02:46.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.349+0000 7fa7db7fe640 1 -- 172.21.15.43:0/2623086759 learned_addr learned my addr 172.21.15.43:0/2623086759 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:46.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.349+0000 7fa7daffd640 1 --2- 172.21.15.43:0/2623086759 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x56261904e8d0 0x5626190125e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.349+0000 7fa7dbfff640 1 --2- 172.21.15.43:0/2623086759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56261903ccc0 0x56261903d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.349+0000 7fa7db7fe640 1 -- 172.21.15.43:0/2623086759 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x56261904e8d0 msgr2=0x5626190125e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:46.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.350+0000 7fa7db7fe640 1 --2- 172.21.15.43:0/2623086759 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x56261904e8d0 0x5626190125e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.350+0000 7fa7db7fe640 1 -- 172.21.15.43:0/2623086759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56261903ccc0 msgr2=0x56261903d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:46.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.350+0000 7fa7db7fe640 1 --2- 172.21.15.43:0/2623086759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56261903ccc0 0x56261903d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.350+0000 7fa7db7fe640 1 -- 172.21.15.43:0/2623086759 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562618d56ba0 con 0x562619012200 2024-04-25T09:02:46.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.350+0000 7fa7db7fe640 1 --2- 172.21.15.43:0/2623086759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562619012200 0x562619014360 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc00fa10 tx=0x7fa7cc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=743bd23035db52fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:46.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.350+0000 7fa7c17fa640 1 -- 172.21.15.43:0/2623086759 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc025070 con 0x562619012200 2024-04-25T09:02:46.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.350+0000 7fa7c17fa640 1 -- 172.21.15.43:0/2623086759 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7cc01b050 con 0x562619012200 2024-04-25T09:02:46.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.350+0000 7fa7c17fa640 1 -- 172.21.15.43:0/2623086759 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc014e00 con 0x562619012200 2024-04-25T09:02:46.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.350+0000 7fa7e4e55740 1 -- 172.21.15.43:0/2623086759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562619012200 msgr2=0x562619014360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.350+0000 7fa7e4e55740 1 --2- 172.21.15.43:0/2623086759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562619012200 0x562619014360 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc00fa10 tx=0x7fa7cc012000 comp rx=0 tx=0).stop 2024-04-25T09:02:46.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.351+0000 7fa7e4e55740 1 -- 172.21.15.43:0/2623086759 shutdown_connections 2024-04-25T09:02:46.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.351+0000 7fa7e4e55740 1 --2- 172.21.15.43:0/2623086759 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56261903ccc0 0x56261903d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.351+0000 7fa7e4e55740 1 --2- 172.21.15.43:0/2623086759 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x56261904e8d0 0x5626190125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.351+0000 7fa7e4e55740 1 --2- 172.21.15.43:0/2623086759 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562619012200 0x562619014360 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.351+0000 7fa7e4e55740 1 -- 172.21.15.43:0/2623086759 >> 172.21.15.43:0/2623086759 conn(0x562618fda050 msgr2=0x562619057cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:46.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.351+0000 7fa7e4e55740 1 -- 172.21.15.43:0/2623086759 shutdown_connections 2024-04-25T09:02:46.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.351+0000 7fa7e4e55740 1 -- 172.21.15.43:0/2623086759 wait complete. 2024-04-25T09:02:46.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.351+0000 7fa7e4e55740 1 -- start start 2024-04-25T09:02:46.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.351+0000 7fa7e4e55740 1 Processor -- start 2024-04-25T09:02:46.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.352+0000 7fa7e4e55740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562619012200 0x562619050320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.352+0000 7fa7e4e55740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x56261903ccc0 0x5626190519a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.352+0000 7fa7e4e55740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56261904e8d0 0x562619027bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.352+0000 7fa7e4e55740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x562618f14610 con 0x562619012200 2024-04-25T09:02:46.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.352+0000 7fa7e4e55740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x56261903a810 con 0x56261904e8d0 2024-04-25T09:02:46.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.352+0000 7fa7e4e55740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x562619051510 con 0x56261903ccc0 2024-04-25T09:02:46.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.352+0000 7fa7e4e55740 10 client.0 _ll_register_callbacks cb 0x562618f2cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:02:46.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.352+0000 7fa7db7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562619012200 0x562619050320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.352+0000 7fa7db7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562619012200 0x562619050320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58712/0 (socket says 172.21.15.43:58712) 2024-04-25T09:02:46.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.352+0000 7fa7db7fe640 1 -- 172.21.15.43:0/2461206410 learned_addr learned my addr 172.21.15.43:0/2461206410 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:46.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.352+0000 7fa7daffd640 1 --2- 172.21.15.43:0/2461206410 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x56261903ccc0 0x5626190519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.352+0000 7fa7dbfff640 1 --2- 172.21.15.43:0/2461206410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56261904e8d0 0x562619027bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.352+0000 7fa7db7fe640 1 -- 172.21.15.43:0/2461206410 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x56261903ccc0 msgr2=0x5626190519a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:46.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.352+0000 7fa7db7fe640 1 --2- 172.21.15.43:0/2461206410 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x56261903ccc0 0x5626190519a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.352+0000 7fa7db7fe640 1 -- 172.21.15.43:0/2461206410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56261904e8d0 msgr2=0x562619027bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:46.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.352+0000 7fa7db7fe640 1 --2- 172.21.15.43:0/2461206410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56261904e8d0 0x562619027bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.352+0000 7fa7db7fe640 1 -- 172.21.15.43:0/2461206410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562618c4a490 con 0x562619012200 2024-04-25T09:02:46.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.352+0000 7fa7dbfff640 1 --2- 172.21.15.43:0/2461206410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56261904e8d0 0x562619027bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:46.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.352+0000 7fa7daffd640 1 --2- 172.21.15.43:0/2461206410 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x56261903ccc0 0x5626190519a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:46.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.353+0000 7fa7db7fe640 1 --2- 172.21.15.43:0/2461206410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562619012200 0x562619050320 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc0206c0 tx=0x7fa7cc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a60f549bd8d28b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:46.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.353+0000 7fa7ba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:02:46.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.353+0000 7fa7ba7fc640 1 -- 172.21.15.43:0/2461206410 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc025070 con 0x562619012200 2024-04-25T09:02:46.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.353+0000 7fa7ba7fc640 1 -- 172.21.15.43:0/2461206410 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa7cc020ab0 con 0x562619012200 2024-04-25T09:02:46.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.353+0000 7fa7e4e55740 1 -- 172.21.15.43:0/2461206410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa7cc01b050 con 0x562619012200 2024-04-25T09:02:46.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.353+0000 7fa7ba7fc640 1 -- 172.21.15.43:0/2461206410 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7cc038050 con 0x562619012200 2024-04-25T09:02:46.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.353+0000 7fa7e4e55740 10 client.8847 fetch_fsmap learned FSMap version 69 2024-04-25T09:02:46.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.353+0000 7fa7e4e55740 1 -- 172.21.15.43:0/2461206410 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x562618db7330 con 0x562619012200 2024-04-25T09:02:46.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.353+0000 7fa7ba7fc640 1 -- 172.21.15.43:0/2461206410 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7cc01b050 con 0x562619012200 2024-04-25T09:02:46.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.353+0000 7fa7ba7fc640 1 -- 172.21.15.43:0/2461206410 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa7cc03b020 con 0x562619012200 2024-04-25T09:02:46.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.353+0000 7fa7e4e55740 10 client.8847 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T09:02:46.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.353+0000 7fa7e4e55740 10 client.8847 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T09:02:46.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.353+0000 7fa7e4e55740 20 client.8847 populate_metadata read hostname 'smithi043' 2024-04-25T09:02:46.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.353+0000 7fa7e4e55740 1 --2- 172.21.15.43:0/2461206410 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x5626190526a0 0x5626190283f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.353+0000 7fa7e4e55740 4 client.8847 mds_command: new command op to 8676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:02:46.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.353+0000 7fa7e4e55740 1 -- 172.21.15.43:0/2461206410 --> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562618f83540 con 0x5626190526a0 2024-04-25T09:02:46.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.353+0000 7fa7daffd640 1 --2- 172.21.15.43:0/2461206410 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x5626190526a0 0x5626190283f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.354+0000 7fa7ba7fc640 10 client.8847 ms_handle_connect on v2:172.21.15.43:6833/1586865031 2024-04-25T09:02:46.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.354+0000 7fa7daffd640 1 --2- 172.21.15.43:0/2461206410 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x5626190526a0 0x5626190283f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2c192a49a014530 server_cookie=ce80883cf6e1fcf1 in_seq=0 out_seq=0 2024-04-25T09:02:46.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.354+0000 7fa7ba7fc640 1 -- 172.21.15.43:0/2461206410 <== mds.0 v2:172.21.15.43:6833/1586865031 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x562618f83540 con 0x5626190526a0 2024-04-25T09:02:46.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.354+0000 7fa7ba7fc640 10 client.8847 handle_command_reply: tid=0 2024-04-25T09:02:46.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.354+0000 7fa7e4e55740 1 client.8847 shutdown 2024-04-25T09:02:46.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.354+0000 7fa7e4e55740 1 -- 172.21.15.43:0/2461206410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562619012200 msgr2=0x562619050320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.354+0000 7fa7e4e55740 1 --2- 172.21.15.43:0/2461206410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562619012200 0x562619050320 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc0206c0 tx=0x7fa7cc000bc0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.354+0000 7fa7e4e55740 1 -- 172.21.15.43:0/2461206410 shutdown_connections 2024-04-25T09:02:46.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.354+0000 7fa7e4e55740 1 --2- 172.21.15.43:0/2461206410 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x5626190526a0 0x5626190283f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.354+0000 7fa7e4e55740 1 --2- 172.21.15.43:0/2461206410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56261904e8d0 0x562619027bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.354+0000 7fa7ba7fc640 0 client.8847 ms_handle_reset on v2:172.21.15.43:6833/1586865031 2024-04-25T09:02:46.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.354+0000 7fa7e4e55740 1 --2- 172.21.15.43:0/2461206410 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x56261903ccc0 0x5626190519a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.354+0000 7fa7e4e55740 1 --2- 172.21.15.43:0/2461206410 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562619012200 0x562619050320 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.354+0000 7fa7e4e55740 1 -- 172.21.15.43:0/2461206410 >> 172.21.15.43:0/2461206410 conn(0x562618fda050 msgr2=0x562619039e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:46.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.355+0000 7fa7e4e55740 1 -- 172.21.15.43:0/2461206410 shutdown_connections 2024-04-25T09:02:46.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.355+0000 7fa7e4e55740 1 -- 172.21.15.43:0/2461206410 wait complete. 2024-04-25T09:02:46.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.355+0000 7fa7e4e55740 20 client.8847 trim_cache size 0 max 16384 2024-04-25T09:02:46.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.370+0000 7fa7e4e55740 1 Processor -- start 2024-04-25T09:02:46.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.370+0000 7fa7e4e55740 1 -- start start 2024-04-25T09:02:46.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.371+0000 7fa7e4e55740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562619012200 0x56261904fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.371+0000 7fa7e4e55740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56261903ccc0 0x56261903a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.371+0000 7fa7e4e55740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x56261904e8d0 0x56261905ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.371+0000 7fa7e4e55740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7dc1ffd10 con 0x562619012200 2024-04-25T09:02:46.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.371+0000 7fa7e4e55740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7dc15fa80 con 0x56261903ccc0 2024-04-25T09:02:46.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.371+0000 7fa7e4e55740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7dc0ccba0 con 0x56261904e8d0 2024-04-25T09:02:46.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.371+0000 7fa7db7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562619012200 0x56261904fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.371+0000 7fa7db7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562619012200 0x56261904fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58726/0 (socket says 172.21.15.43:58726) 2024-04-25T09:02:46.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.371+0000 7fa7db7fe640 1 -- 172.21.15.43:0/2372695647 learned_addr learned my addr 172.21.15.43:0/2372695647 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:46.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.371+0000 7fa7daffd640 1 --2- 172.21.15.43:0/2372695647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56261903ccc0 0x56261903a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.371+0000 7fa7dbfff640 1 --2- 172.21.15.43:0/2372695647 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x56261904e8d0 0x56261905ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.371+0000 7fa7db7fe640 1 -- 172.21.15.43:0/2372695647 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x56261904e8d0 msgr2=0x56261905ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:46.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.371+0000 7fa7db7fe640 1 --2- 172.21.15.43:0/2372695647 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x56261904e8d0 0x56261905ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.371+0000 7fa7db7fe640 1 -- 172.21.15.43:0/2372695647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56261903ccc0 msgr2=0x56261903a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:46.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.371+0000 7fa7db7fe640 1 --2- 172.21.15.43:0/2372695647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56261903ccc0 0x56261903a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.371+0000 7fa7db7fe640 1 -- 172.21.15.43:0/2372695647 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562618c23610 con 0x562619012200 2024-04-25T09:02:46.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.371+0000 7fa7daffd640 1 --2- 172.21.15.43:0/2372695647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56261903ccc0 0x56261903a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:46.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.371+0000 7fa7db7fe640 1 --2- 172.21.15.43:0/2372695647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562619012200 0x56261904fdc0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc000c00 tx=0x7fa7cc01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1034680e00659052 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:46.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.371+0000 7fa7c0ff9640 1 -- 172.21.15.43:0/2372695647 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc020ab0 con 0x562619012200 2024-04-25T09:02:46.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.371+0000 7fa7c0ff9640 1 -- 172.21.15.43:0/2372695647 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7cc01b070 con 0x562619012200 2024-04-25T09:02:46.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.371+0000 7fa7c0ff9640 1 -- 172.21.15.43:0/2372695647 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc02b880 con 0x562619012200 2024-04-25T09:02:46.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.372+0000 7fa7e4e55740 1 -- 172.21.15.43:0/2372695647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562619012200 msgr2=0x56261904fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.372+0000 7fa7e4e55740 1 --2- 172.21.15.43:0/2372695647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562619012200 0x56261904fdc0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc000c00 tx=0x7fa7cc01e000 comp rx=0 tx=0).stop 2024-04-25T09:02:46.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.372+0000 7fa7e4e55740 1 -- 172.21.15.43:0/2372695647 shutdown_connections 2024-04-25T09:02:46.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.372+0000 7fa7e4e55740 1 --2- 172.21.15.43:0/2372695647 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x56261904e8d0 0x56261905ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.372+0000 7fa7e4e55740 1 --2- 172.21.15.43:0/2372695647 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56261903ccc0 0x56261903a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.372+0000 7fa7e4e55740 1 --2- 172.21.15.43:0/2372695647 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562619012200 0x56261904fdc0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.372+0000 7fa7e4e55740 1 -- 172.21.15.43:0/2372695647 >> 172.21.15.43:0/2372695647 conn(0x562618fda050 msgr2=0x562619052f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:46.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.372+0000 7fa7e4e55740 1 -- 172.21.15.43:0/2372695647 shutdown_connections 2024-04-25T09:02:46.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.372+0000 7fa7e4e55740 1 -- 172.21.15.43:0/2372695647 wait complete. 2024-04-25T09:02:46.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.373+0000 7fa7e4e55740 1 -- start start 2024-04-25T09:02:46.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.373+0000 7fa7e4e55740 1 Processor -- start 2024-04-25T09:02:46.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.373+0000 7fa7e4e55740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562619012200 0x562619067010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.373+0000 7fa7e4e55740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56261903ccc0 0x56261903a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.373+0000 7fa7e4e55740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x56261904e8d0 0x56261905ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.373+0000 7fa7e4e55740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x562619008630 con 0x562619012200 2024-04-25T09:02:46.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.373+0000 7fa7e4e55740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7dc15f480 con 0x56261903ccc0 2024-04-25T09:02:46.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.373+0000 7fa7e4e55740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x56261903aa20 con 0x56261904e8d0 2024-04-25T09:02:46.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.373+0000 7fa7e4e55740 10 client.0 _ll_register_callbacks cb 0x7fa7cc017040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:02:46.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.373+0000 7fa7db7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562619012200 0x562619067010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.374+0000 7fa7db7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562619012200 0x562619067010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58730/0 (socket says 172.21.15.43:58730) 2024-04-25T09:02:46.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.374+0000 7fa7db7fe640 1 -- 172.21.15.43:0/3243836937 learned_addr learned my addr 172.21.15.43:0/3243836937 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:46.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.374+0000 7fa7daffd640 1 --2- 172.21.15.43:0/3243836937 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56261903ccc0 0x56261903a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.374+0000 7fa7dbfff640 1 --2- 172.21.15.43:0/3243836937 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x56261904e8d0 0x56261905ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.374+0000 7fa7db7fe640 1 -- 172.21.15.43:0/3243836937 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x56261904e8d0 msgr2=0x56261905ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:46.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.374+0000 7fa7db7fe640 1 --2- 172.21.15.43:0/3243836937 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x56261904e8d0 0x56261905ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.374+0000 7fa7db7fe640 1 -- 172.21.15.43:0/3243836937 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56261903ccc0 msgr2=0x56261903a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:46.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.374+0000 7fa7db7fe640 1 --2- 172.21.15.43:0/3243836937 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56261903ccc0 0x56261903a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.374+0000 7fa7db7fe640 1 -- 172.21.15.43:0/3243836937 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562619026e70 con 0x562619012200 2024-04-25T09:02:46.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.374+0000 7fa7db7fe640 1 --2- 172.21.15.43:0/3243836937 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562619012200 0x562619067010 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc00f5a0 tx=0x7fa7cc000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa075cd6eacf21d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:46.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.374+0000 7fa7ba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:02:46.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.374+0000 7fa7ba7fc640 1 -- 172.21.15.43:0/3243836937 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc049020 con 0x562619012200 2024-04-25T09:02:46.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.374+0000 7fa7ba7fc640 1 -- 172.21.15.43:0/3243836937 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa7cc020ab0 con 0x562619012200 2024-04-25T09:02:46.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.374+0000 7fa7e4e55740 1 -- 172.21.15.43:0/3243836937 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa7cc01b070 con 0x562619012200 2024-04-25T09:02:46.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.374+0000 7fa7ba7fc640 1 -- 172.21.15.43:0/3243836937 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7cc053050 con 0x562619012200 2024-04-25T09:02:46.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.374+0000 7fa7e4e55740 10 client.8853 fetch_fsmap learned FSMap version 69 2024-04-25T09:02:46.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.374+0000 7fa7e4e55740 1 -- 172.21.15.43:0/3243836937 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x5626190c2170 con 0x562619012200 2024-04-25T09:02:46.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.374+0000 7fa7ba7fc640 1 -- 172.21.15.43:0/3243836937 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7cc01b070 con 0x562619012200 2024-04-25T09:02:46.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.374+0000 7fa7ba7fc640 1 -- 172.21.15.43:0/3243836937 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa7cc056020 con 0x562619012200 2024-04-25T09:02:46.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.375+0000 7fa7e4e55740 10 client.8853 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T09:02:46.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.375+0000 7fa7e4e55740 10 client.8853 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T09:02:46.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.375+0000 7fa7e4e55740 20 client.8853 populate_metadata read hostname 'smithi043' 2024-04-25T09:02:46.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.375+0000 7fa7e4e55740 1 --2- 172.21.15.43:0/3243836937 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x562619066170 0x56261905b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.375+0000 7fa7e4e55740 4 client.8853 mds_command: new command op to 8676 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T09:02:46.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.375+0000 7fa7e4e55740 1 -- 172.21.15.43:0/3243836937 --> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x562618f7d890 con 0x562619066170 2024-04-25T09:02:46.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.375+0000 7fa7daffd640 1 --2- 172.21.15.43:0/3243836937 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x562619066170 0x56261905b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.375+0000 7fa7ba7fc640 10 client.8853 ms_handle_connect on v2:172.21.15.43:6833/1586865031 2024-04-25T09:02:46.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.375+0000 7fa7daffd640 1 --2- 172.21.15.43:0/3243836937 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x562619066170 0x56261905b340 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffb41ff4166f329a server_cookie=e9b690bafad948ec in_seq=0 out_seq=0 2024-04-25T09:02:46.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.378+0000 7fa7ba7fc640 1 -- 172.21.15.43:0/3243836937 <== mds.0 v2:172.21.15.43:6833/1586865031 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x562618f7d890 con 0x562619066170 2024-04-25T09:02:46.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.378+0000 7fa7ba7fc640 10 client.8853 handle_command_reply: tid=0 2024-04-25T09:02:46.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.378+0000 7fa7e4e55740 1 client.8853 shutdown 2024-04-25T09:02:46.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.378+0000 7fa7e4e55740 1 -- 172.21.15.43:0/3243836937 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562619012200 msgr2=0x562619067010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.378+0000 7fa7e4e55740 1 --2- 172.21.15.43:0/3243836937 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562619012200 0x562619067010 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc00f5a0 tx=0x7fa7cc000ed0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.378+0000 7fa7e4e55740 1 -- 172.21.15.43:0/3243836937 shutdown_connections 2024-04-25T09:02:46.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.378+0000 7fa7e4e55740 1 --2- 172.21.15.43:0/3243836937 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x562619066170 0x56261905b340 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.378+0000 7fa7ba7fc640 0 client.8853 ms_handle_reset on v2:172.21.15.43:6833/1586865031 2024-04-25T09:02:46.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.378+0000 7fa7e4e55740 1 --2- 172.21.15.43:0/3243836937 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x56261904e8d0 0x56261905ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.371 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.378+0000 7fa7e4e55740 1 --2- 172.21.15.43:0/3243836937 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56261903ccc0 0x56261903a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.378+0000 7fa7e4e55740 1 --2- 172.21.15.43:0/3243836937 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x562619012200 0x562619067010 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.378+0000 7fa7e4e55740 1 -- 172.21.15.43:0/3243836937 >> 172.21.15.43:0/3243836937 conn(0x562618fda050 msgr2=0x56261900c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:46.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.378+0000 7fa7e4e55740 1 -- 172.21.15.43:0/3243836937 shutdown_connections 2024-04-25T09:02:46.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.378+0000 7fa7e4e55740 1 -- 172.21.15.43:0/3243836937 wait complete. 2024-04-25T09:02:46.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.378+0000 7fa7e4e55740 20 client.8853 trim_cache size 0 max 16384 2024-04-25T09:02:46.372 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:02:46.372 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-25T09:02:46.372 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "f73a089d-7e85-4d3e-88a9-e0fa4baab2b1", 2024-04-25T09:02:46.373 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-25T09:02:46.373 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:02:46.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.379+0000 7fa7e2153640 1 -- 172.21.15.43:0/2574472872 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa7c806b4d0 msgr2=0x7fa7c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.379+0000 7fa7e2153640 1 --2- 172.21.15.43:0/2574472872 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa7c806b4d0 0x7fa7c806d990 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc007fd0 tx=0x7fa7cc009040 comp rx=0 tx=0).stop 2024-04-25T09:02:46.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.379+0000 7fa7e2153640 1 -- 172.21.15.43:0/2574472872 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7dc0d3760 msgr2=0x7fa7dc160c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.379+0000 7fa7e2153640 1 --2- 172.21.15.43:0/2574472872 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7dc0d3760 0x7fa7dc160c80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc08e170 tx=0x7fa7d00046c0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.379+0000 7fa7e2153640 1 -- 172.21.15.43:0/2574472872 shutdown_connections 2024-04-25T09:02:46.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.379+0000 7fa7e2153640 1 --2- 172.21.15.43:0/2574472872 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa7dc15de80 0x7fa7dc0b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.379+0000 7fa7e2153640 1 --2- 172.21.15.43:0/2574472872 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa7dc0d3760 0x7fa7dc160c80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.379+0000 7fa7e2153640 1 --2- 172.21.15.43:0/2574472872 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa7c806b4d0 0x7fa7c806d990 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.379+0000 7fa7e2153640 1 --2- 172.21.15.43:0/2574472872 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa7dc0b06c0 0x7fa7dc1f81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.379+0000 7fa7e2153640 1 -- 172.21.15.43:0/2574472872 >> 172.21.15.43:0/2574472872 conn(0x7fa7dc08d460 msgr2=0x7fa7dc0d0920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:46.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.379+0000 7fa7e2153640 1 -- 172.21.15.43:0/2574472872 shutdown_connections 2024-04-25T09:02:46.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.379+0000 7fa7e2153640 1 -- 172.21.15.43:0/2574472872 wait complete. 2024-04-25T09:02:46.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:46.381 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub pause 2024-04-25T09:02:46.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.514+0000 7f720416a640 1 Processor -- start 2024-04-25T09:02:46.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.514+0000 7f720416a640 1 -- start start 2024-04-25T09:02:46.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.515+0000 7f720416a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f71fc0b59e0 0x7f71fc0d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.515+0000 7f720416a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71fc0b5e70 con 0x7f71fc15e090 2024-04-25T09:02:46.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.515+0000 7f720416a640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71fc162610 con 0x7f71fc0b5560 2024-04-25T09:02:46.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.515+0000 7f720416a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71fc0d1da0 con 0x7f71fc0b59e0 2024-04-25T09:02:46.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.515+0000 7f7202707640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f71fc0b59e0 0x7f71fc0d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.515+0000 7f7201f06640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f71fc15e090 0x7f71fc161aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48422/0 (socket says 172.21.15.43:48422) 2024-04-25T09:02:46.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.515+0000 7f7202707640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f71fc0b59e0 0x7f71fc0d2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58740/0 (socket says 172.21.15.43:58740) 2024-04-25T09:02:46.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.515+0000 7f7201f06640 1 -- 172.21.15.43:0/4171462818 learned_addr learned my addr 172.21.15.43:0/4171462818 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:46.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.515+0000 7f7202707640 1 -- 172.21.15.43:0/4171462818 >> v1:172.21.15.193:6789/0 conn(0x7f71fc0b5560 legacy=0x7f71fc0d2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.515+0000 7f7202707640 1 -- 172.21.15.43:0/4171462818 >> v1:172.21.15.43:6789/0 conn(0x7f71fc15e090 legacy=0x7f71fc161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.515+0000 7f7200f04640 1 -- 172.21.15.43:0/4171462818 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 30986776 0 0) 0x7f71fc0b5e70 con 0x7f71fc15e090 2024-04-25T09:02:46.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.515+0000 7f7202707640 1 -- 172.21.15.43:0/4171462818 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71fc0d1750 con 0x7f71fc0b59e0 2024-04-25T09:02:46.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.516+0000 7f7202707640 1 --2- 172.21.15.43:0/4171462818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71fc0b59e0 0x7f71fc0d2940 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f71f4009d90 tx=0x7f71f402a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f61953b6193edb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:46.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.516+0000 7f7200f04640 1 -- 172.21.15.43:0/4171462818 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f4037070 con 0x7f71fc0b59e0 2024-04-25T09:02:46.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.516+0000 7f7200f04640 1 -- 172.21.15.43:0/4171462818 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f4007740 con 0x7f71fc0b59e0 2024-04-25T09:02:46.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.516+0000 7f7200f04640 1 -- 172.21.15.43:0/4171462818 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f4007b10 con 0x7f71fc0b59e0 2024-04-25T09:02:46.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.517+0000 7f720416a640 1 -- 172.21.15.43:0/4171462818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71fc0b59e0 msgr2=0x7f71fc0d2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.517+0000 7f720416a640 1 --2- 172.21.15.43:0/4171462818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71fc0b59e0 0x7f71fc0d2940 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f71f4009d90 tx=0x7f71f402a780 comp rx=0 tx=0).stop 2024-04-25T09:02:46.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.517+0000 7f720416a640 1 -- 172.21.15.43:0/4171462818 shutdown_connections 2024-04-25T09:02:46.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.517+0000 7f720416a640 1 --2- 172.21.15.43:0/4171462818 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71fc0b59e0 0x7f71fc0d2940 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.517+0000 7f720416a640 1 -- 172.21.15.43:0/4171462818 >> 172.21.15.43:0/4171462818 conn(0x7f71fc0d0560 msgr2=0x7f71fc0d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:46.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.517+0000 7f720416a640 1 -- 172.21.15.43:0/4171462818 shutdown_connections 2024-04-25T09:02:46.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.517+0000 7f720416a640 1 -- 172.21.15.43:0/4171462818 wait complete. 2024-04-25T09:02:46.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.518+0000 7f720416a640 1 Processor -- start 2024-04-25T09:02:46.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.518+0000 7f720416a640 1 -- start start 2024-04-25T09:02:46.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.519+0000 7f720416a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71fc0b5560 0x7f71fc0cae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.519+0000 7f720416a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f71fc0b59e0 0x7f71fc160770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.519+0000 7f720416a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71fc15e090 0x7f71fc15ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.519+0000 7f720416a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc0b3280 con 0x7f71fc0b5560 2024-04-25T09:02:46.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.519+0000 7f720416a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc1628f0 con 0x7f71fc15e090 2024-04-25T09:02:46.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.519+0000 7f720416a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc1621b0 con 0x7f71fc0b59e0 2024-04-25T09:02:46.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.519+0000 7f7201f06640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71fc0b5560 0x7f71fc0cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.519+0000 7f7201f06640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71fc0b5560 0x7f71fc0cae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58744/0 (socket says 172.21.15.43:58744) 2024-04-25T09:02:46.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.519+0000 7f7201f06640 1 -- 172.21.15.43:0/1185899290 learned_addr learned my addr 172.21.15.43:0/1185899290 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:46.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.519+0000 7f7201705640 1 --2- 172.21.15.43:0/1185899290 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f71fc0b59e0 0x7f71fc160770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.519+0000 7f7202707640 1 --2- 172.21.15.43:0/1185899290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71fc15e090 0x7f71fc15ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.519+0000 7f7201f06640 1 -- 172.21.15.43:0/1185899290 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f71fc0b59e0 msgr2=0x7f71fc160770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.519+0000 7f7201f06640 1 --2- 172.21.15.43:0/1185899290 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f71fc0b59e0 0x7f71fc160770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.519+0000 7f7201f06640 1 -- 172.21.15.43:0/1185899290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71fc15e090 msgr2=0x7f71fc15ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.519+0000 7f7201f06640 1 --2- 172.21.15.43:0/1185899290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71fc15e090 0x7f71fc15ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.519+0000 7f7201f06640 1 -- 172.21.15.43:0/1185899290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71fc0b2000 con 0x7f71fc0b5560 2024-04-25T09:02:46.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.520+0000 7f7201f06640 1 --2- 172.21.15.43:0/1185899290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71fc0b5560 0x7f71fc0cae20 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f71f000c420 tx=0x7f71f000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:46.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.520+0000 7f71ee7fc640 1 -- 172.21.15.43:0/1185899290 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f00074e0 con 0x7f71fc0b5560 2024-04-25T09:02:46.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.520+0000 7f71ee7fc640 1 -- 172.21.15.43:0/1185899290 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f00047e0 con 0x7f71fc0b5560 2024-04-25T09:02:46.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.520+0000 7f720416a640 1 -- 172.21.15.43:0/1185899290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71fc15f6d0 con 0x7f71fc0b5560 2024-04-25T09:02:46.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.520+0000 7f71ee7fc640 1 -- 172.21.15.43:0/1185899290 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f0004c10 con 0x7f71fc0b5560 2024-04-25T09:02:46.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.520+0000 7f720416a640 1 -- 172.21.15.43:0/1185899290 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71fc15fc90 con 0x7f71fc0b5560 2024-04-25T09:02:46.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.521+0000 7f71ee7fc640 1 -- 172.21.15.43:0/1185899290 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f71f0011070 con 0x7f71fc0b5560 2024-04-25T09:02:46.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.522+0000 7f71ee7fc640 1 --2- 172.21.15.43:0/1185899290 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f71e006b4d0 0x7f71e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.522+0000 7f71ee7fc640 1 -- 172.21.15.43:0/1185899290 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f71f0089bf0 con 0x7f71fc0b5560 2024-04-25T09:02:46.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.522+0000 7f7201705640 1 --2- 172.21.15.43:0/1185899290 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f71e006b4d0 0x7f71e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.522+0000 7f7201705640 1 --2- 172.21.15.43:0/1185899290 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f71e006b4d0 0x7f71e006d990 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f71fc0cc320 tx=0x7f71f8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:46.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.523+0000 7f7206e5f740 1 Processor -- start 2024-04-25T09:02:46.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.523+0000 7f7206e5f740 1 -- start start 2024-04-25T09:02:46.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.523+0000 7f7206e5f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6271b1ca0 0x55c6271b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.523+0000 7f7206e5f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c6271ee370 0x55c6271b2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.523+0000 7f7206e5f740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c6271dc760 0x55c6271dcbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.523+0000 7f7206e5f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c627114b90 con 0x55c6271ee370 2024-04-25T09:02:46.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.523+0000 7f7206e5f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6270bde40 con 0x55c6271b1ca0 2024-04-25T09:02:46.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.524+0000 7f7206e5f740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c627110c30 con 0x55c6271dc760 2024-04-25T09:02:46.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.524+0000 7f7201705640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c6271ee370 0x55c6271b2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.524+0000 7f7201705640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c6271ee370 0x55c6271b2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58756/0 (socket says 172.21.15.43:58756) 2024-04-25T09:02:46.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.524+0000 7f7201705640 1 -- 172.21.15.43:0/4911236 learned_addr learned my addr 172.21.15.43:0/4911236 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:46.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.524+0000 7f7201f06640 1 --2- 172.21.15.43:0/4911236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6271b1ca0 0x55c6271b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.524+0000 7f7202707640 1 --2- 172.21.15.43:0/4911236 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c6271dc760 0x55c6271dcbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.524+0000 7f7201705640 1 -- 172.21.15.43:0/4911236 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c6271dc760 msgr2=0x55c6271dcbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:46.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.524+0000 7f7201705640 1 --2- 172.21.15.43:0/4911236 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c6271dc760 0x55c6271dcbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.524+0000 7f7201705640 1 -- 172.21.15.43:0/4911236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6271b1ca0 msgr2=0x55c6271b3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:46.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.524+0000 7f7201705640 1 --2- 172.21.15.43:0/4911236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6271b1ca0 0x55c6271b3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.524+0000 7f7201705640 1 -- 172.21.15.43:0/4911236 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c626ef37d0 con 0x55c6271ee370 2024-04-25T09:02:46.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.524+0000 7f7201705640 1 --2- 172.21.15.43:0/4911236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c6271ee370 0x55c6271b2080 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f71f800fba0 tx=0x7f71f8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a3b51ef102e97d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:46.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.524+0000 7f71df7fe640 1 -- 172.21.15.43:0/4911236 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f8025070 con 0x55c6271ee370 2024-04-25T09:02:46.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.524+0000 7f71df7fe640 1 -- 172.21.15.43:0/4911236 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f801b050 con 0x55c6271ee370 2024-04-25T09:02:46.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.524+0000 7f71df7fe640 1 -- 172.21.15.43:0/4911236 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f8014de0 con 0x55c6271ee370 2024-04-25T09:02:46.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.525+0000 7f7206e5f740 1 -- 172.21.15.43:0/4911236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c6271ee370 msgr2=0x55c6271b2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.525+0000 7f7206e5f740 1 --2- 172.21.15.43:0/4911236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c6271ee370 0x55c6271b2080 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f71f800fba0 tx=0x7f71f8012000 comp rx=0 tx=0).stop 2024-04-25T09:02:46.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.525+0000 7f7206e5f740 1 -- 172.21.15.43:0/4911236 shutdown_connections 2024-04-25T09:02:46.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.525+0000 7f7206e5f740 1 --2- 172.21.15.43:0/4911236 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c6271dc760 0x55c6271dcbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.525+0000 7f7206e5f740 1 --2- 172.21.15.43:0/4911236 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c6271ee370 0x55c6271b2080 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.525+0000 7f7206e5f740 1 --2- 172.21.15.43:0/4911236 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6271b1ca0 0x55c6271b3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.525+0000 7f7206e5f740 1 -- 172.21.15.43:0/4911236 >> 172.21.15.43:0/4911236 conn(0x55c627179870 msgr2=0x55c6271f7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:46.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.525+0000 7f7206e5f740 1 -- 172.21.15.43:0/4911236 shutdown_connections 2024-04-25T09:02:46.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.525+0000 7f7206e5f740 1 -- 172.21.15.43:0/4911236 wait complete. 2024-04-25T09:02:46.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.525+0000 7f7206e5f740 1 -- start start 2024-04-25T09:02:46.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.526+0000 7f7206e5f740 1 Processor -- start 2024-04-25T09:02:46.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.526+0000 7f7206e5f740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c6271b1ca0 0x55c6271ef010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.526+0000 7f7206e5f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6271dc760 0x55c6271c4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.526+0000 7f7206e5f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c6271ee370 0x55c6271d8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.527+0000 7f7206e5f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6270d13e0 con 0x55c6271ee370 2024-04-25T09:02:46.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.527+0000 7f7206e5f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6271da2b0 con 0x55c6271dc760 2024-04-25T09:02:46.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.527+0000 7f7206e5f740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c6271c47b0 con 0x55c6271b1ca0 2024-04-25T09:02:46.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.527+0000 7f7206e5f740 10 client.0 _ll_register_callbacks cb 0x55c6270d0db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:02:46.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.527+0000 7f7202707640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c6271ee370 0x55c6271d8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.527+0000 7f7202707640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c6271ee370 0x55c6271d8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58766/0 (socket says 172.21.15.43:58766) 2024-04-25T09:02:46.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.527+0000 7f7202707640 1 -- 172.21.15.43:0/1082552148 learned_addr learned my addr 172.21.15.43:0/1082552148 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:46.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.527+0000 7f7201f06640 1 --2- 172.21.15.43:0/1082552148 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c6271b1ca0 0x55c6271ef010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.527+0000 7f7201705640 1 --2- 172.21.15.43:0/1082552148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6271dc760 0x55c6271c4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.527+0000 7f7202707640 1 -- 172.21.15.43:0/1082552148 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c6271b1ca0 msgr2=0x55c6271ef010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:46.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.527+0000 7f7202707640 1 --2- 172.21.15.43:0/1082552148 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c6271b1ca0 0x55c6271ef010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.527+0000 7f7202707640 1 -- 172.21.15.43:0/1082552148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6271dc760 msgr2=0x55c6271c4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:46.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.527+0000 7f7202707640 1 --2- 172.21.15.43:0/1082552148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6271dc760 0x55c6271c4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.527+0000 7f7202707640 1 -- 172.21.15.43:0/1082552148 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c626de90f0 con 0x55c6271ee370 2024-04-25T09:02:46.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.527+0000 7f7202707640 1 --2- 172.21.15.43:0/1082552148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c6271ee370 0x55c6271d8cc0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f71f4035040 tx=0x7f71f4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27103ff51b6d8730 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:46.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.527+0000 7f71dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:02:46.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.527+0000 7f71dcff9640 1 -- 172.21.15.43:0/1082552148 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f4037040 con 0x55c6271ee370 2024-04-25T09:02:46.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.527+0000 7f71dcff9640 1 -- 172.21.15.43:0/1082552148 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f71f40041a0 con 0x55c6271ee370 2024-04-25T09:02:46.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.527+0000 7f7206e5f740 1 -- 172.21.15.43:0/1082552148 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71f801b050 con 0x55c6271ee370 2024-04-25T09:02:46.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.527+0000 7f71dcff9640 1 -- 172.21.15.43:0/1082552148 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71f4050050 con 0x55c6271ee370 2024-04-25T09:02:46.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.528+0000 7f7206e5f740 10 client.8868 fetch_fsmap learned FSMap version 69 2024-04-25T09:02:46.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.528+0000 7f7206e5f740 1 -- 172.21.15.43:0/1082552148 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x55c626f53f60 con 0x55c6271ee370 2024-04-25T09:02:46.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.528+0000 7f71dcff9640 1 -- 172.21.15.43:0/1082552148 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f801b050 con 0x55c6271ee370 2024-04-25T09:02:46.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.528+0000 7f71dcff9640 1 -- 172.21.15.43:0/1082552148 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f71f4053020 con 0x55c6271ee370 2024-04-25T09:02:46.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.528+0000 7f7206e5f740 10 client.8868 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T09:02:46.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.528+0000 7f7206e5f740 10 client.8868 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T09:02:46.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.528+0000 7f7206e5f740 20 client.8868 populate_metadata read hostname 'smithi043' 2024-04-25T09:02:46.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.528+0000 7f7206e5f740 1 --2- 172.21.15.43:0/1082552148 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x55c6271f9b20 0x55c6271f9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.528+0000 7f7206e5f740 4 client.8868 mds_command: new command op to 8676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:02:46.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.528+0000 7f7206e5f740 1 -- 172.21.15.43:0/1082552148 --> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6271220d0 con 0x55c6271f9b20 2024-04-25T09:02:46.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.528+0000 7f7201f06640 1 --2- 172.21.15.43:0/1082552148 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x55c6271f9b20 0x55c6271f9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.528+0000 7f71dcff9640 10 client.8868 ms_handle_connect on v2:172.21.15.43:6833/1586865031 2024-04-25T09:02:46.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.528+0000 7f7201f06640 1 --2- 172.21.15.43:0/1082552148 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x55c6271f9b20 0x55c6271f9f30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aef40b97f47ce296 server_cookie=71ba55e639d38508 in_seq=0 out_seq=0 2024-04-25T09:02:46.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.529+0000 7f71dcff9640 1 -- 172.21.15.43:0/1082552148 <== mds.0 v2:172.21.15.43:6833/1586865031 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c6271220d0 con 0x55c6271f9b20 2024-04-25T09:02:46.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.529+0000 7f71dcff9640 10 client.8868 handle_command_reply: tid=0 2024-04-25T09:02:46.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.529+0000 7f7206e5f740 1 client.8868 shutdown 2024-04-25T09:02:46.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.529+0000 7f7206e5f740 1 -- 172.21.15.43:0/1082552148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c6271ee370 msgr2=0x55c6271d8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.529+0000 7f7206e5f740 1 --2- 172.21.15.43:0/1082552148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c6271ee370 0x55c6271d8cc0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f71f4035040 tx=0x7f71f4004ab0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.529+0000 7f7206e5f740 1 -- 172.21.15.43:0/1082552148 shutdown_connections 2024-04-25T09:02:46.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.529+0000 7f7206e5f740 1 --2- 172.21.15.43:0/1082552148 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x55c6271f9b20 0x55c6271f9f30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.529+0000 7f7206e5f740 1 --2- 172.21.15.43:0/1082552148 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c6271ee370 0x55c6271d8cc0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.529+0000 7f7206e5f740 1 --2- 172.21.15.43:0/1082552148 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6271dc760 0x55c6271c4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.529+0000 7f7206e5f740 1 --2- 172.21.15.43:0/1082552148 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c6271b1ca0 0x55c6271ef010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.529+0000 7f71dcff9640 0 client.8868 ms_handle_reset on v2:172.21.15.43:6833/1586865031 2024-04-25T09:02:46.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.529+0000 7f7206e5f740 1 -- 172.21.15.43:0/1082552148 >> 172.21.15.43:0/1082552148 conn(0x55c627179870 msgr2=0x55c6271da490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:46.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.529+0000 7f7206e5f740 1 -- 172.21.15.43:0/1082552148 shutdown_connections 2024-04-25T09:02:46.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.529+0000 7f7206e5f740 1 -- 172.21.15.43:0/1082552148 wait complete. 2024-04-25T09:02:46.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.529+0000 7f7206e5f740 20 client.8868 trim_cache size 0 max 16384 2024-04-25T09:02:46.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.544+0000 7f7206e5f740 1 Processor -- start 2024-04-25T09:02:46.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.544+0000 7f7206e5f740 1 -- start start 2024-04-25T09:02:46.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.545+0000 7f7206e5f740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c6271b1ca0 0x55c6271e0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.545+0000 7f7206e5f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c6271dc760 0x55c6271b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.545+0000 7f7206e5f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6271ee370 0x55c6271b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.545+0000 7f7206e5f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc1ffdd0 con 0x55c6271dc760 2024-04-25T09:02:46.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.545+0000 7f7206e5f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc0cda70 con 0x55c6271ee370 2024-04-25T09:02:46.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.545+0000 7f7206e5f740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc0cce70 con 0x55c6271b1ca0 2024-04-25T09:02:46.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.545+0000 7f7201705640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c6271dc760 0x55c6271b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.545+0000 7f7201705640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c6271dc760 0x55c6271b0460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58774/0 (socket says 172.21.15.43:58774) 2024-04-25T09:02:46.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.545+0000 7f7201705640 1 -- 172.21.15.43:0/104868855 learned_addr learned my addr 172.21.15.43:0/104868855 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:46.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.545+0000 7f7201f06640 1 --2- 172.21.15.43:0/104868855 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c6271b1ca0 0x55c6271e0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.545+0000 7f7201705640 1 -- 172.21.15.43:0/104868855 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c6271b1ca0 msgr2=0x55c6271e0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:46.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.545+0000 7f7202707640 1 --2- 172.21.15.43:0/104868855 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6271ee370 0x55c6271b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.545+0000 7f7201705640 1 --2- 172.21.15.43:0/104868855 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c6271b1ca0 0x55c6271e0120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.545+0000 7f7201705640 1 -- 172.21.15.43:0/104868855 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6271ee370 msgr2=0x55c6271b09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:46.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.545+0000 7f7201705640 1 --2- 172.21.15.43:0/104868855 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6271ee370 0x55c6271b09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.545+0000 7f7201705640 1 -- 172.21.15.43:0/104868855 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6271b0000 con 0x55c6271dc760 2024-04-25T09:02:46.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.545+0000 7f7201705640 1 --2- 172.21.15.43:0/104868855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c6271dc760 0x55c6271b0460 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f71f8023040 tx=0x7f71f8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a09f0ca251733ec6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:46.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.546+0000 7f71deffd640 1 -- 172.21.15.43:0/104868855 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f8025070 con 0x55c6271dc760 2024-04-25T09:02:46.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.546+0000 7f71deffd640 1 -- 172.21.15.43:0/104868855 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f8026040 con 0x55c6271dc760 2024-04-25T09:02:46.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.546+0000 7f71deffd640 1 -- 172.21.15.43:0/104868855 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f8020630 con 0x55c6271dc760 2024-04-25T09:02:46.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.546+0000 7f7206e5f740 1 -- 172.21.15.43:0/104868855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c6271dc760 msgr2=0x55c6271b0460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.546+0000 7f7206e5f740 1 --2- 172.21.15.43:0/104868855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c6271dc760 0x55c6271b0460 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f71f8023040 tx=0x7f71f8015000 comp rx=0 tx=0).stop 2024-04-25T09:02:46.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.546+0000 7f7206e5f740 1 -- 172.21.15.43:0/104868855 shutdown_connections 2024-04-25T09:02:46.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.546+0000 7f7206e5f740 1 --2- 172.21.15.43:0/104868855 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6271ee370 0x55c6271b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.546+0000 7f7206e5f740 1 --2- 172.21.15.43:0/104868855 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c6271dc760 0x55c6271b0460 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.546+0000 7f7206e5f740 1 --2- 172.21.15.43:0/104868855 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c6271b1ca0 0x55c6271e0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.546+0000 7f7206e5f740 1 -- 172.21.15.43:0/104868855 >> 172.21.15.43:0/104868855 conn(0x55c627179870 msgr2=0x55c6271d9fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:46.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.546+0000 7f7206e5f740 1 -- 172.21.15.43:0/104868855 shutdown_connections 2024-04-25T09:02:46.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.546+0000 7f7206e5f740 1 -- 172.21.15.43:0/104868855 wait complete. 2024-04-25T09:02:46.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.547+0000 7f7206e5f740 1 -- start start 2024-04-25T09:02:46.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.547+0000 7f7206e5f740 1 Processor -- start 2024-04-25T09:02:46.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.548+0000 7f7206e5f740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c6271b1ca0 0x55c627258af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.548+0000 7f7206e5f740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c6271dc760 0x55c6271c8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.548+0000 7f7206e5f740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6271ee370 0x55c6271f8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.548+0000 7f7206e5f740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6271d3000 con 0x55c6271b1ca0 2024-04-25T09:02:46.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.548+0000 7f7206e5f740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71fc0cd470 con 0x55c6271ee370 2024-04-25T09:02:46.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.548+0000 7f7206e5f740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c6271af4e0 con 0x55c6271dc760 2024-04-25T09:02:46.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.548+0000 7f7206e5f740 10 client.0 _ll_register_callbacks cb 0x7f71f4003a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:02:46.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.548+0000 7f7201f06640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c6271b1ca0 0x55c627258af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.548+0000 7f7201f06640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c6271b1ca0 0x55c627258af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58790/0 (socket says 172.21.15.43:58790) 2024-04-25T09:02:46.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.548+0000 7f7201f06640 1 -- 172.21.15.43:0/365182445 learned_addr learned my addr 172.21.15.43:0/365182445 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:46.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.548+0000 7f7202707640 1 --2- 172.21.15.43:0/365182445 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6271ee370 0x55c6271f8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.548+0000 7f7201705640 1 --2- 172.21.15.43:0/365182445 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c6271dc760 0x55c6271c8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.548+0000 7f7201f06640 1 -- 172.21.15.43:0/365182445 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c6271dc760 msgr2=0x55c6271c8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:46.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.548+0000 7f7201f06640 1 --2- 172.21.15.43:0/365182445 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c6271dc760 0x55c6271c8770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.548+0000 7f7201f06640 1 -- 172.21.15.43:0/365182445 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6271ee370 msgr2=0x55c6271f8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:46.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.548+0000 7f7201f06640 1 --2- 172.21.15.43:0/365182445 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6271ee370 0x55c6271f8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.548+0000 7f7201f06640 1 -- 172.21.15.43:0/365182445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c626dc21b0 con 0x55c6271b1ca0 2024-04-25T09:02:46.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.548+0000 7f7202707640 1 --2- 172.21.15.43:0/365182445 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6271ee370 0x55c6271f8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:46.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.548+0000 7f7201f06640 1 --2- 172.21.15.43:0/365182445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c6271b1ca0 0x55c627258af0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f71f000c3f0 tx=0x7f71f0007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c82aae05b380e5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:46.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.548+0000 7f71dcff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:02:46.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.548+0000 7f71dcff9640 1 -- 172.21.15.43:0/365182445 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f0054b20 con 0x55c6271b1ca0 2024-04-25T09:02:46.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.548+0000 7f71dcff9640 1 -- 172.21.15.43:0/365182445 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f71f0060680 con 0x55c6271b1ca0 2024-04-25T09:02:46.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.548+0000 7f7206e5f740 1 -- 172.21.15.43:0/365182445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71f8026040 con 0x55c6271b1ca0 2024-04-25T09:02:46.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.548+0000 7f71dcff9640 1 -- 172.21.15.43:0/365182445 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71f0054cc0 con 0x55c6271b1ca0 2024-04-25T09:02:46.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.549+0000 7f7206e5f740 10 client.8874 fetch_fsmap learned FSMap version 69 2024-04-25T09:02:46.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.549+0000 7f7206e5f740 1 -- 172.21.15.43:0/365182445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x55c6271c8cb0 con 0x55c6271b1ca0 2024-04-25T09:02:46.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.549+0000 7f71dcff9640 1 -- 172.21.15.43:0/365182445 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71f8026040 con 0x55c6271b1ca0 2024-04-25T09:02:46.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.549+0000 7f71dcff9640 1 -- 172.21.15.43:0/365182445 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f71f0057900 con 0x55c6271b1ca0 2024-04-25T09:02:46.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.549+0000 7f7206e5f740 10 client.8874 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T09:02:46.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.549+0000 7f7206e5f740 10 client.8874 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T09:02:46.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.549+0000 7f7206e5f740 20 client.8874 populate_metadata read hostname 'smithi043' 2024-04-25T09:02:46.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.549+0000 7f7206e5f740 1 --2- 172.21.15.43:0/365182445 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x55c6271ed0f0 0x55c6271f9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.549+0000 7f7206e5f740 4 client.8874 mds_command: new command op to 8676 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-25T09:02:46.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.549+0000 7f7206e5f740 1 -- 172.21.15.43:0/365182445 --> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x55c62711c420 con 0x55c6271ed0f0 2024-04-25T09:02:46.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.549+0000 7f7201705640 1 --2- 172.21.15.43:0/365182445 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x55c6271ed0f0 0x55c6271f9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.549+0000 7f71dcff9640 10 client.8874 ms_handle_connect on v2:172.21.15.43:6833/1586865031 2024-04-25T09:02:46.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.549+0000 7f7201705640 1 --2- 172.21.15.43:0/365182445 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x55c6271ed0f0 0x55c6271f9520 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=759067d2e0e2052 server_cookie=86395952a54b5e92 in_seq=0 out_seq=0 2024-04-25T09:02:46.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.549+0000 7f71dcff9640 1 -- 172.21.15.43:0/365182445 <== mds.0 v2:172.21.15.43:6833/1586865031 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55c62711c420 con 0x55c6271ed0f0 2024-04-25T09:02:46.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.549+0000 7f71dcff9640 10 client.8874 handle_command_reply: tid=0 2024-04-25T09:02:46.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.549+0000 7f7206e5f740 1 client.8874 shutdown 2024-04-25T09:02:46.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.550+0000 7f7206e5f740 1 -- 172.21.15.43:0/365182445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c6271b1ca0 msgr2=0x55c627258af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.550+0000 7f7206e5f740 1 --2- 172.21.15.43:0/365182445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c6271b1ca0 0x55c627258af0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f71f000c3f0 tx=0x7f71f0007700 comp rx=0 tx=0).stop 2024-04-25T09:02:46.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.550+0000 7f7206e5f740 1 -- 172.21.15.43:0/365182445 shutdown_connections 2024-04-25T09:02:46.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.550+0000 7f7206e5f740 1 --2- 172.21.15.43:0/365182445 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x55c6271ed0f0 0x55c6271f9520 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.550+0000 7f7206e5f740 1 --2- 172.21.15.43:0/365182445 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c6271ee370 0x55c6271f8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.550+0000 7f7206e5f740 1 --2- 172.21.15.43:0/365182445 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c6271dc760 0x55c6271c8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.550+0000 7f71dcff9640 0 client.8874 ms_handle_reset on v2:172.21.15.43:6833/1586865031 2024-04-25T09:02:46.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.550+0000 7f7206e5f740 1 --2- 172.21.15.43:0/365182445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c6271b1ca0 0x55c627258af0 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.550+0000 7f7206e5f740 1 -- 172.21.15.43:0/365182445 >> 172.21.15.43:0/365182445 conn(0x55c627179870 msgr2=0x55c6271d9fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:46.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.550+0000 7f7206e5f740 1 -- 172.21.15.43:0/365182445 shutdown_connections 2024-04-25T09:02:46.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.550+0000 7f7206e5f740 1 -- 172.21.15.43:0/365182445 wait complete. 2024-04-25T09:02:46.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.550+0000 7f7206e5f740 20 client.8874 trim_cache size 0 max 16384 2024-04-25T09:02:46.546 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:02:46.546 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0 2024-04-25T09:02:46.546 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:02:46.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.550+0000 7f720416a640 1 -- 172.21.15.43:0/1185899290 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f71e006b4d0 msgr2=0x7f71e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.550+0000 7f720416a640 1 --2- 172.21.15.43:0/1185899290 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f71e006b4d0 0x7f71e006d990 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f71fc0cc320 tx=0x7f71f8009040 comp rx=0 tx=0).stop 2024-04-25T09:02:46.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.550+0000 7f720416a640 1 -- 172.21.15.43:0/1185899290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71fc0b5560 msgr2=0x7f71fc0cae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.550+0000 7f720416a640 1 --2- 172.21.15.43:0/1185899290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71fc0b5560 0x7f71fc0cae20 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f71f000c420 tx=0x7f71f000c8e0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.550+0000 7f720416a640 1 -- 172.21.15.43:0/1185899290 shutdown_connections 2024-04-25T09:02:46.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.550+0000 7f720416a640 1 --2- 172.21.15.43:0/1185899290 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f71e006b4d0 0x7f71e006d990 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.550+0000 7f720416a640 1 --2- 172.21.15.43:0/1185899290 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71fc15e090 0x7f71fc15ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.550+0000 7f720416a640 1 --2- 172.21.15.43:0/1185899290 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f71fc0b59e0 0x7f71fc160770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.550+0000 7f720416a640 1 --2- 172.21.15.43:0/1185899290 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71fc0b5560 0x7f71fc0cae20 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.550+0000 7f720416a640 1 -- 172.21.15.43:0/1185899290 >> 172.21.15.43:0/1185899290 conn(0x7f71fc0d0560 msgr2=0x7f71fc0b7c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:46.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.550+0000 7f720416a640 1 -- 172.21.15.43:0/1185899290 shutdown_connections 2024-04-25T09:02:46.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.550+0000 7f720416a640 1 -- 172.21.15.43:0/1185899290 wait complete. 2024-04-25T09:02:46.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:46.550 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub status 2024-04-25T09:02:46.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.666+0000 7fb9e7163640 1 Processor -- start 2024-04-25T09:02:46.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.666+0000 7fb9e7163640 1 -- start start 2024-04-25T09:02:46.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.666+0000 7fb9e7163640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb9e0161310 0x7fb9e00d4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.666+0000 7fb9e7163640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9e00d45c0 con 0x7fb9e00d0120 2024-04-25T09:02:46.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.666+0000 7fb9e7163640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9e00cf030 con 0x7fb9e00d0c70 2024-04-25T09:02:46.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.666+0000 7fb9e7163640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9e00d4870 con 0x7fb9e0161310 2024-04-25T09:02:46.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.667+0000 7fb9dffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb9e00d0120 0x7fb9e00d0560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48438/0 (socket says 172.21.15.43:48438) 2024-04-25T09:02:46.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.667+0000 7fb9dffff640 1 -- 172.21.15.43:0/4243659997 learned_addr learned my addr 172.21.15.43:0/4243659997 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:46.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.667+0000 7fb9e4eff640 1 --2- 172.21.15.43:0/4243659997 >> v2:172.21.15.193:3300/0 conn(0x7fb9e0161310 0x7fb9e00d4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.667+0000 7fb9df7fe640 1 -- 172.21.15.43:0/4243659997 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1498830231 0 0) 0x7fb9e00d45c0 con 0x7fb9e00d0120 2024-04-25T09:02:46.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.667+0000 7fb9df7fe640 1 -- 172.21.15.43:0/4243659997 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9c8003520 con 0x7fb9e00d0120 2024-04-25T09:02:46.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.667+0000 7fb9df7fe640 1 -- 172.21.15.43:0/4243659997 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1281335407 0 0) 0x7fb9c8003520 con 0x7fb9e00d0120 2024-04-25T09:02:46.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.667+0000 7fb9df7fe640 1 -- 172.21.15.43:0/4243659997 >> v2:172.21.15.193:3300/0 conn(0x7fb9e0161310 msgr2=0x7fb9e00d4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:46.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.667+0000 7fb9df7fe640 1 --2- 172.21.15.43:0/4243659997 >> v2:172.21.15.193:3300/0 conn(0x7fb9e0161310 0x7fb9e00d4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.667+0000 7fb9df7fe640 1 -- 172.21.15.43:0/4243659997 >> v1:172.21.15.193:6789/0 conn(0x7fb9e00d0c70 legacy=0x7fb9e00c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.667+0000 7fb9df7fe640 1 -- 172.21.15.43:0/4243659997 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e00cf310 con 0x7fb9e00d0120 2024-04-25T09:02:46.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.667+0000 7fb9df7fe640 1 -- 172.21.15.43:0/4243659997 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7fb9d4001670 con 0x7fb9e00d0120 2024-04-25T09:02:46.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.667+0000 7fb9e4eff640 1 --2- 172.21.15.43:0/4243659997 >> v2:172.21.15.193:3300/0 conn(0x7fb9e0161310 0x7fb9e00d4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:02:46.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.667+0000 7fb9df7fe640 1 -- 172.21.15.43:0/4243659997 >> v1:172.21.15.43:6789/0 conn(0x7fb9e00d0120 legacy=0x7fb9e00d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.667+0000 7fb9df7fe640 1 --2- 172.21.15.43:0/4243659997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9c80039b0 0x7fb9c8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.667+0000 7fb9df7fe640 1 --2- 172.21.15.43:0/4243659997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9c8004800 0x7fb9c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.667+0000 7fb9df7fe640 1 --2- 172.21.15.43:0/4243659997 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9c80051c0 0x7fb9c800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.667+0000 7fb9df7fe640 1 -- 172.21.15.43:0/4243659997 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e00d4870 con 0x7fb9c80039b0 2024-04-25T09:02:46.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.667+0000 7fb9df7fe640 1 -- 172.21.15.43:0/4243659997 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c800c060 con 0x7fb9c8004800 2024-04-25T09:02:46.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.667+0000 7fb9df7fe640 1 -- 172.21.15.43:0/4243659997 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c800c260 con 0x7fb9c80051c0 2024-04-25T09:02:46.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.667+0000 7fb9e4eff640 1 --2- 172.21.15.43:0/4243659997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9c80039b0 0x7fb9c8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.667+0000 7fb9dffff640 1 --2- 172.21.15.43:0/4243659997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9c8004800 0x7fb9c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.667+0000 7fb9e5700640 1 --2- 172.21.15.43:0/4243659997 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9c80051c0 0x7fb9c800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.668+0000 7fb9e4eff640 1 -- 172.21.15.43:0/4243659997 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9c80051c0 msgr2=0x7fb9c800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:46.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.668+0000 7fb9e4eff640 1 --2- 172.21.15.43:0/4243659997 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9c80051c0 0x7fb9c800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.668+0000 7fb9e4eff640 1 -- 172.21.15.43:0/4243659997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9c8004800 msgr2=0x7fb9c8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:46.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.668+0000 7fb9e4eff640 1 --2- 172.21.15.43:0/4243659997 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9c8004800 0x7fb9c8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.668+0000 7fb9e4eff640 1 -- 172.21.15.43:0/4243659997 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c8003520 con 0x7fb9c80039b0 2024-04-25T09:02:46.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.668+0000 7fb9e5700640 1 -- 172.21.15.43:0/4243659997 reap_dead start 2024-04-25T09:02:46.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.668+0000 7fb9e4eff640 1 --2- 172.21.15.43:0/4243659997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9c80039b0 0x7fb9c8003dd0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0009c90 tx=0x7fb9c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30663f84d0102064 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:46.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.668+0000 7fb9df7fe640 1 -- 172.21.15.43:0/4243659997 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0036070 con 0x7fb9c80039b0 2024-04-25T09:02:46.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.668+0000 7fb9df7fe640 1 -- 172.21.15.43:0/4243659997 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d0031400 con 0x7fb9c80039b0 2024-04-25T09:02:46.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.668+0000 7fb9df7fe640 1 -- 172.21.15.43:0/4243659997 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0009860 con 0x7fb9c80039b0 2024-04-25T09:02:46.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.669+0000 7fb9e7163640 1 -- 172.21.15.43:0/4243659997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9c80039b0 msgr2=0x7fb9c8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.669+0000 7fb9e7163640 1 --2- 172.21.15.43:0/4243659997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9c80039b0 0x7fb9c8003dd0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0009c90 tx=0x7fb9c8005830 comp rx=0 tx=0).stop 2024-04-25T09:02:46.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.669+0000 7fb9e7163640 1 -- 172.21.15.43:0/4243659997 shutdown_connections 2024-04-25T09:02:46.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.669+0000 7fb9e7163640 1 --2- 172.21.15.43:0/4243659997 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9c80039b0 0x7fb9c8003dd0 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.669+0000 7fb9e7163640 1 -- 172.21.15.43:0/4243659997 >> 172.21.15.43:0/4243659997 conn(0x7fb9e009cd20 msgr2=0x7fb9e00d3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:46.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.669+0000 7fb9e7163640 1 -- 172.21.15.43:0/4243659997 shutdown_connections 2024-04-25T09:02:46.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.669+0000 7fb9e7163640 1 -- 172.21.15.43:0/4243659997 wait complete. 2024-04-25T09:02:46.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.670+0000 7fb9e7163640 1 Processor -- start 2024-04-25T09:02:46.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.670+0000 7fb9e7163640 1 -- start start 2024-04-25T09:02:46.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.671+0000 7fb9e7163640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9c80039b0 0x7fb9e015ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.671+0000 7fb9e7163640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9e00cbcf0 0x7fb9e00cd680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.671+0000 7fb9e7163640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9e00cc170 0x7fb9e009d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.671+0000 7fb9e7163640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e00d4080 con 0x7fb9e00cbcf0 2024-04-25T09:02:46.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.671+0000 7fb9e7163640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e01616f0 con 0x7fb9e00cc170 2024-04-25T09:02:46.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.671+0000 7fb9e7163640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e00d4200 con 0x7fb9c80039b0 2024-04-25T09:02:46.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.671+0000 7fb9dffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9e00cbcf0 0x7fb9e00cd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.671+0000 7fb9dffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9e00cbcf0 0x7fb9e00cd680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58806/0 (socket says 172.21.15.43:58806) 2024-04-25T09:02:46.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.671+0000 7fb9dffff640 1 -- 172.21.15.43:0/2164019703 learned_addr learned my addr 172.21.15.43:0/2164019703 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:46.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.671+0000 7fb9e5700640 1 --2- 172.21.15.43:0/2164019703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9e00cc170 0x7fb9e009d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.671+0000 7fb9e4eff640 1 --2- 172.21.15.43:0/2164019703 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9c80039b0 0x7fb9e015ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.672+0000 7fb9dffff640 1 -- 172.21.15.43:0/2164019703 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9c80039b0 msgr2=0x7fb9e015ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.672+0000 7fb9dffff640 1 --2- 172.21.15.43:0/2164019703 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9c80039b0 0x7fb9e015ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.672+0000 7fb9dffff640 1 -- 172.21.15.43:0/2164019703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9e00cc170 msgr2=0x7fb9e009d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.672+0000 7fb9dffff640 1 --2- 172.21.15.43:0/2164019703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9e00cc170 0x7fb9e009d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.672+0000 7fb9dffff640 1 -- 172.21.15.43:0/2164019703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e0071a40 con 0x7fb9e00cbcf0 2024-04-25T09:02:46.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.672+0000 7fb9e5700640 1 --2- 172.21.15.43:0/2164019703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9e00cc170 0x7fb9e009d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:02:46.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.672+0000 7fb9dffff640 1 --2- 172.21.15.43:0/2164019703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9e00cbcf0 0x7fb9e00cd680 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4001050 tx=0x7fb9d4007720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:46.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.672+0000 7fb9dd7fa640 1 -- 172.21.15.43:0/2164019703 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d400f960 con 0x7fb9e00cbcf0 2024-04-25T09:02:46.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.672+0000 7fb9dd7fa640 1 -- 172.21.15.43:0/2164019703 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d4018070 con 0x7fb9e00cbcf0 2024-04-25T09:02:46.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.672+0000 7fb9e7163640 1 -- 172.21.15.43:0/2164019703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9e009e360 con 0x7fb9e00cbcf0 2024-04-25T09:02:46.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.672+0000 7fb9dd7fa640 1 -- 172.21.15.43:0/2164019703 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d400fd60 con 0x7fb9e00cbcf0 2024-04-25T09:02:46.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.672+0000 7fb9e7163640 1 -- 172.21.15.43:0/2164019703 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9e009e920 con 0x7fb9e00cbcf0 2024-04-25T09:02:46.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.674+0000 7fb9dd7fa640 1 -- 172.21.15.43:0/2164019703 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb9d401d460 con 0x7fb9e00cbcf0 2024-04-25T09:02:46.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.674+0000 7fb9dd7fa640 1 --2- 172.21.15.43:0/2164019703 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb9cc06b4d0 0x7fb9cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.674+0000 7fb9dd7fa640 1 -- 172.21.15.43:0/2164019703 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb9d408a170 con 0x7fb9e00cbcf0 2024-04-25T09:02:46.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.674+0000 7fb9e4eff640 1 --2- 172.21.15.43:0/2164019703 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb9cc06b4d0 0x7fb9cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.675+0000 7fb9e4eff640 1 --2- 172.21.15.43:0/2164019703 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb9cc06b4d0 0x7fb9cc06d990 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0033000 tx=0x7fb9d0008030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:46.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.675+0000 7fb9e9e6e740 1 Processor -- start 2024-04-25T09:02:46.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.675+0000 7fb9e9e6e740 1 -- start start 2024-04-25T09:02:46.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.676+0000 7fb9e9e6e740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5650703bbb10 0x5650703be320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.676+0000 7fb9e9e6e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650703bbef0 0x5650703be860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.676+0000 7fb9e9e6e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56507038e770 0x565070391f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.676+0000 7fb9e9e6e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650702f7da0 con 0x5650703bbef0 2024-04-25T09:02:46.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.676+0000 7fb9e9e6e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650702a0b80 con 0x56507038e770 2024-04-25T09:02:46.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.676+0000 7fb9e9e6e740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x5650702f3e40 con 0x5650703bbb10 2024-04-25T09:02:46.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.676+0000 7fb9dffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650703bbef0 0x5650703be860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.676+0000 7fb9dffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650703bbef0 0x5650703be860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58816/0 (socket says 172.21.15.43:58816) 2024-04-25T09:02:46.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.676+0000 7fb9dffff640 1 -- 172.21.15.43:0/1610158442 learned_addr learned my addr 172.21.15.43:0/1610158442 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:46.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.676+0000 7fb9e5700640 1 --2- 172.21.15.43:0/1610158442 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56507038e770 0x565070391f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.676+0000 7fb9e4eff640 1 --2- 172.21.15.43:0/1610158442 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5650703bbb10 0x5650703be320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.676+0000 7fb9dffff640 1 -- 172.21.15.43:0/1610158442 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5650703bbb10 msgr2=0x5650703be320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:46.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.676+0000 7fb9dffff640 1 --2- 172.21.15.43:0/1610158442 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5650703bbb10 0x5650703be320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.676+0000 7fb9dffff640 1 -- 172.21.15.43:0/1610158442 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56507038e770 msgr2=0x565070391f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:46.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.676+0000 7fb9dffff640 1 --2- 172.21.15.43:0/1610158442 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56507038e770 0x565070391f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.676+0000 7fb9dffff640 1 -- 172.21.15.43:0/1610158442 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5650702aaf10 con 0x5650703bbef0 2024-04-25T09:02:46.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.676+0000 7fb9dffff640 1 --2- 172.21.15.43:0/1610158442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650703bbef0 0x5650703be860 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4001970 tx=0x7fb9d4058300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=710205ee33a03cf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:46.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.676+0000 7fb9c67fc640 1 -- 172.21.15.43:0/1610158442 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d4061000 con 0x5650703bbef0 2024-04-25T09:02:46.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.676+0000 7fb9c67fc640 1 -- 172.21.15.43:0/1610158442 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d4056ca0 con 0x5650703bbef0 2024-04-25T09:02:46.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.676+0000 7fb9c67fc640 1 -- 172.21.15.43:0/1610158442 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d40701d0 con 0x5650703bbef0 2024-04-25T09:02:46.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.677+0000 7fb9e9e6e740 1 -- 172.21.15.43:0/1610158442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650703bbef0 msgr2=0x5650703be860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.677+0000 7fb9e9e6e740 1 --2- 172.21.15.43:0/1610158442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650703bbef0 0x5650703be860 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4001970 tx=0x7fb9d4058300 comp rx=0 tx=0).stop 2024-04-25T09:02:46.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.677+0000 7fb9e9e6e740 1 -- 172.21.15.43:0/1610158442 shutdown_connections 2024-04-25T09:02:46.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.677+0000 7fb9e9e6e740 1 --2- 172.21.15.43:0/1610158442 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x56507038e770 0x565070391f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.677+0000 7fb9e9e6e740 1 --2- 172.21.15.43:0/1610158442 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650703bbef0 0x5650703be860 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.677+0000 7fb9e9e6e740 1 --2- 172.21.15.43:0/1610158442 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5650703bbb10 0x5650703be320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.677+0000 7fb9e9e6e740 1 -- 172.21.15.43:0/1610158442 >> 172.21.15.43:0/1610158442 conn(0x5650703909e0 msgr2=0x5650703d9320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:46.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.677+0000 7fb9e9e6e740 1 -- 172.21.15.43:0/1610158442 shutdown_connections 2024-04-25T09:02:46.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.677+0000 7fb9e9e6e740 1 -- 172.21.15.43:0/1610158442 wait complete. 2024-04-25T09:02:46.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.677+0000 7fb9e9e6e740 1 -- start start 2024-04-25T09:02:46.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.678+0000 7fb9e9e6e740 1 Processor -- start 2024-04-25T09:02:46.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.678+0000 7fb9e9e6e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650703bbb10 0x5650703c1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.678+0000 7fb9e9e6e740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5650703bbef0 0x5650703dc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.678+0000 7fb9e9e6e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650703da450 0x5650703dcc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.678+0000 7fb9e9e6e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x565070393b90 con 0x5650703da450 2024-04-25T09:02:46.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.678+0000 7fb9e9e6e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650702964b0 con 0x5650703bbb10 2024-04-25T09:02:46.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.678+0000 7fb9e9e6e740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x565070393e70 con 0x5650703bbef0 2024-04-25T09:02:46.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.678+0000 7fb9e9e6e740 10 client.0 _ll_register_callbacks cb 0x5650702d6180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:02:46.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.679+0000 7fb9e5700640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650703da450 0x5650703dcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.679+0000 7fb9e5700640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650703da450 0x5650703dcc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58822/0 (socket says 172.21.15.43:58822) 2024-04-25T09:02:46.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.679+0000 7fb9e5700640 1 -- 172.21.15.43:0/1662127027 learned_addr learned my addr 172.21.15.43:0/1662127027 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:46.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.679+0000 7fb9dffff640 1 --2- 172.21.15.43:0/1662127027 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5650703bbef0 0x5650703dc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.679+0000 7fb9e4eff640 1 --2- 172.21.15.43:0/1662127027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650703bbb10 0x5650703c1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.679+0000 7fb9e5700640 1 -- 172.21.15.43:0/1662127027 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5650703bbef0 msgr2=0x5650703dc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:46.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.679+0000 7fb9e5700640 1 --2- 172.21.15.43:0/1662127027 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5650703bbef0 0x5650703dc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.679+0000 7fb9e5700640 1 -- 172.21.15.43:0/1662127027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650703bbb10 msgr2=0x5650703c1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:46.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.679+0000 7fb9e5700640 1 --2- 172.21.15.43:0/1662127027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650703bbb10 0x5650703c1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.679+0000 7fb9e5700640 1 -- 172.21.15.43:0/1662127027 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5650700d67d0 con 0x5650703da450 2024-04-25T09:02:46.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.679+0000 7fb9e5700640 1 --2- 172.21.15.43:0/1662127027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650703da450 0x5650703dcc10 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fb9e00d14d0 tx=0x7fb9d8013700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1579db5d907a9935 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:46.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.679+0000 7fb9bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:02:46.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.679+0000 7fb9bf7fe640 1 -- 172.21.15.43:0/1662127027 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8013d90 con 0x5650703da450 2024-04-25T09:02:46.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.679+0000 7fb9bf7fe640 1 -- 172.21.15.43:0/1662127027 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb9d8006b50 con 0x5650703da450 2024-04-25T09:02:46.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.679+0000 7fb9e9e6e740 1 -- 172.21.15.43:0/1662127027 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9d4056ca0 con 0x5650703da450 2024-04-25T09:02:46.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.679+0000 7fb9bf7fe640 1 -- 172.21.15.43:0/1662127027 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9d8026050 con 0x5650703da450 2024-04-25T09:02:46.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.679+0000 7fb9e9e6e740 10 client.8886 fetch_fsmap learned FSMap version 69 2024-04-25T09:02:46.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.679+0000 7fb9e9e6e740 1 -- 172.21.15.43:0/1662127027 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x56507000c3f0 con 0x5650703da450 2024-04-25T09:02:46.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.679+0000 7fb9bf7fe640 1 -- 172.21.15.43:0/1662127027 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d4056ca0 con 0x5650703da450 2024-04-25T09:02:46.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.680+0000 7fb9bf7fe640 1 -- 172.21.15.43:0/1662127027 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb9d8003970 con 0x5650703da450 2024-04-25T09:02:46.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.680+0000 7fb9e9e6e740 10 client.8886 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T09:02:46.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.680+0000 7fb9e9e6e740 10 client.8886 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T09:02:46.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.680+0000 7fb9e9e6e740 20 client.8886 populate_metadata read hostname 'smithi043' 2024-04-25T09:02:46.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.680+0000 7fb9e9e6e740 1 --2- 172.21.15.43:0/1662127027 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x5650703e8f70 0x5650703e9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.680+0000 7fb9e9e6e740 4 client.8886 mds_command: new command op to 8676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:02:46.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.680+0000 7fb9e9e6e740 1 -- 172.21.15.43:0/1662127027 --> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5650703052e0 con 0x5650703e8f70 2024-04-25T09:02:46.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.680+0000 7fb9e4eff640 1 --2- 172.21.15.43:0/1662127027 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x5650703e8f70 0x5650703e9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.680+0000 7fb9bf7fe640 10 client.8886 ms_handle_connect on v2:172.21.15.43:6833/1586865031 2024-04-25T09:02:46.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.680+0000 7fb9e4eff640 1 --2- 172.21.15.43:0/1662127027 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x5650703e8f70 0x5650703e9380 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6080f72463c087a3 server_cookie=60f471819cbfbfa1 in_seq=0 out_seq=0 2024-04-25T09:02:46.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.681+0000 7fb9bf7fe640 1 -- 172.21.15.43:0/1662127027 <== mds.0 v2:172.21.15.43:6833/1586865031 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5650703052e0 con 0x5650703e8f70 2024-04-25T09:02:46.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.681+0000 7fb9bf7fe640 10 client.8886 handle_command_reply: tid=0 2024-04-25T09:02:46.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.681+0000 7fb9e9e6e740 1 client.8886 shutdown 2024-04-25T09:02:46.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.681+0000 7fb9e9e6e740 1 -- 172.21.15.43:0/1662127027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650703da450 msgr2=0x5650703dcc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.681+0000 7fb9e9e6e740 1 --2- 172.21.15.43:0/1662127027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650703da450 0x5650703dcc10 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fb9e00d14d0 tx=0x7fb9d8013700 comp rx=0 tx=0).stop 2024-04-25T09:02:46.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.681+0000 7fb9e9e6e740 1 -- 172.21.15.43:0/1662127027 shutdown_connections 2024-04-25T09:02:46.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.681+0000 7fb9e9e6e740 1 --2- 172.21.15.43:0/1662127027 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x5650703e8f70 0x5650703e9380 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.681+0000 7fb9e9e6e740 1 --2- 172.21.15.43:0/1662127027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650703da450 0x5650703dcc10 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.681+0000 7fb9e9e6e740 1 --2- 172.21.15.43:0/1662127027 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5650703bbef0 0x5650703dc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.681+0000 7fb9bf7fe640 0 client.8886 ms_handle_reset on v2:172.21.15.43:6833/1586865031 2024-04-25T09:02:46.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.681+0000 7fb9e9e6e740 1 --2- 172.21.15.43:0/1662127027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650703bbb10 0x5650703c1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.681+0000 7fb9e9e6e740 1 -- 172.21.15.43:0/1662127027 >> 172.21.15.43:0/1662127027 conn(0x56507038e770 msgr2=0x56507038f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:46.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.681+0000 7fb9e9e6e740 1 -- 172.21.15.43:0/1662127027 shutdown_connections 2024-04-25T09:02:46.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.681+0000 7fb9e9e6e740 1 -- 172.21.15.43:0/1662127027 wait complete. 2024-04-25T09:02:46.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.681+0000 7fb9e9e6e740 20 client.8886 trim_cache size 0 max 16384 2024-04-25T09:02:46.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.696+0000 7fb9e9e6e740 1 Processor -- start 2024-04-25T09:02:46.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.696+0000 7fb9e9e6e740 1 -- start start 2024-04-25T09:02:46.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.697+0000 7fb9e9e6e740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5650703bbb10 0x5650703931e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.697+0000 7fb9e9e6e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650703bbef0 0x5650703bff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.697+0000 7fb9e9e6e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650703da450 0x5650703c04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.697+0000 7fb9e9e6e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e0204910 con 0x5650703da450 2024-04-25T09:02:46.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.697+0000 7fb9e9e6e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e0203ad0 con 0x5650703bbef0 2024-04-25T09:02:46.687 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.697+0000 7fb9e9e6e740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e009fbd0 con 0x5650703bbb10 2024-04-25T09:02:46.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.697+0000 7fb9e5700640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650703da450 0x5650703c04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.697+0000 7fb9e5700640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650703da450 0x5650703c04a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58834/0 (socket says 172.21.15.43:58834) 2024-04-25T09:02:46.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.697+0000 7fb9e5700640 1 -- 172.21.15.43:0/724469783 learned_addr learned my addr 172.21.15.43:0/724469783 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:46.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.697+0000 7fb9dffff640 1 --2- 172.21.15.43:0/724469783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650703bbef0 0x5650703bff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.697+0000 7fb9e4eff640 1 --2- 172.21.15.43:0/724469783 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5650703bbb10 0x5650703931e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.697+0000 7fb9e5700640 1 -- 172.21.15.43:0/724469783 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5650703bbb10 msgr2=0x5650703931e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:46.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.697+0000 7fb9e5700640 1 --2- 172.21.15.43:0/724469783 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5650703bbb10 0x5650703931e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.697+0000 7fb9e5700640 1 -- 172.21.15.43:0/724469783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650703bbef0 msgr2=0x5650703bff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:46.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.697+0000 7fb9e5700640 1 --2- 172.21.15.43:0/724469783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650703bbef0 0x5650703bff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.697+0000 7fb9e5700640 1 -- 172.21.15.43:0/724469783 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56506ffcc0f0 con 0x5650703da450 2024-04-25T09:02:46.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.698+0000 7fb9e5700640 1 --2- 172.21.15.43:0/724469783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650703da450 0x5650703c04a0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fb9d8001880 tx=0x7fb9d8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce93bd6750ad148 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:46.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.698+0000 7fb9c5ffb640 1 -- 172.21.15.43:0/724469783 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d802f070 con 0x5650703da450 2024-04-25T09:02:46.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.698+0000 7fb9c5ffb640 1 -- 172.21.15.43:0/724469783 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d8001250 con 0x5650703da450 2024-04-25T09:02:46.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.698+0000 7fb9c5ffb640 1 -- 172.21.15.43:0/724469783 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d8003970 con 0x5650703da450 2024-04-25T09:02:46.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.698+0000 7fb9e9e6e740 1 -- 172.21.15.43:0/724469783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650703da450 msgr2=0x5650703c04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.698+0000 7fb9e9e6e740 1 --2- 172.21.15.43:0/724469783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650703da450 0x5650703c04a0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fb9d8001880 tx=0x7fb9d8018000 comp rx=0 tx=0).stop 2024-04-25T09:02:46.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.698+0000 7fb9e9e6e740 1 -- 172.21.15.43:0/724469783 shutdown_connections 2024-04-25T09:02:46.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.698+0000 7fb9e9e6e740 1 --2- 172.21.15.43:0/724469783 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650703da450 0x5650703c04a0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.698+0000 7fb9e9e6e740 1 --2- 172.21.15.43:0/724469783 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650703bbef0 0x5650703bff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.698+0000 7fb9e9e6e740 1 --2- 172.21.15.43:0/724469783 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5650703bbb10 0x5650703931e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.698+0000 7fb9e9e6e740 1 -- 172.21.15.43:0/724469783 >> 172.21.15.43:0/724469783 conn(0x56507038e770 msgr2=0x5650703d6da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:46.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.698+0000 7fb9e9e6e740 1 -- 172.21.15.43:0/724469783 shutdown_connections 2024-04-25T09:02:46.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.698+0000 7fb9e9e6e740 1 -- 172.21.15.43:0/724469783 wait complete. 2024-04-25T09:02:46.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.699+0000 7fb9e9e6e740 1 -- start start 2024-04-25T09:02:46.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.699+0000 7fb9e9e6e740 1 Processor -- start 2024-04-25T09:02:46.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.700+0000 7fb9e9e6e740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5650703bbb10 0x5650703d9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.700+0000 7fb9e9e6e740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650703bbef0 0x5650703b6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.700+0000 7fb9e9e6e740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650703da450 0x5650703b6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.700+0000 7fb9e9e6e740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x5650703d74b0 con 0x5650703bbef0 2024-04-25T09:02:46.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.700+0000 7fb9e9e6e740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e02034d0 con 0x5650703da450 2024-04-25T09:02:46.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.700+0000 7fb9e9e6e740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x5650703b95d0 con 0x5650703bbb10 2024-04-25T09:02:46.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.700+0000 7fb9e9e6e740 10 client.0 _ll_register_callbacks cb 0x7fb9d4004ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:02:46.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.700+0000 7fb9dffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650703bbef0 0x5650703b6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.700+0000 7fb9dffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650703bbef0 0x5650703b6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58844/0 (socket says 172.21.15.43:58844) 2024-04-25T09:02:46.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.700+0000 7fb9dffff640 1 -- 172.21.15.43:0/2632398975 learned_addr learned my addr 172.21.15.43:0/2632398975 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:46.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.700+0000 7fb9e4eff640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5650703bbb10 0x5650703d9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.700+0000 7fb9e5700640 1 --2- 172.21.15.43:0/2632398975 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650703da450 0x5650703b6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.700+0000 7fb9dffff640 1 -- 172.21.15.43:0/2632398975 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5650703bbb10 msgr2=0x5650703d9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:46.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.700+0000 7fb9dffff640 1 --2- 172.21.15.43:0/2632398975 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5650703bbb10 0x5650703d9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.700+0000 7fb9dffff640 1 -- 172.21.15.43:0/2632398975 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650703da450 msgr2=0x5650703b6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:46.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.700+0000 7fb9dffff640 1 --2- 172.21.15.43:0/2632398975 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650703da450 0x5650703b6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.700+0000 7fb9dffff640 1 -- 172.21.15.43:0/2632398975 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5650703d9c60 con 0x5650703bbef0 2024-04-25T09:02:46.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.701+0000 7fb9dffff640 1 --2- 172.21.15.43:0/2632398975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650703bbef0 0x5650703b6750 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fb9d40886f0 tx=0x7fb9d4015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adac44fb5b1b6289 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:46.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.701+0000 7fb9bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:02:46.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.701+0000 7fb9bf7fe640 1 -- 172.21.15.43:0/2632398975 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d4061000 con 0x5650703bbef0 2024-04-25T09:02:46.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.701+0000 7fb9bf7fe640 1 -- 172.21.15.43:0/2632398975 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb9d401ee00 con 0x5650703bbef0 2024-04-25T09:02:46.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.701+0000 7fb9e9e6e740 1 -- 172.21.15.43:0/2632398975 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb9d8001250 con 0x5650703bbef0 2024-04-25T09:02:46.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.701+0000 7fb9bf7fe640 1 -- 172.21.15.43:0/2632398975 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb9d40701d0 con 0x5650703bbef0 2024-04-25T09:02:46.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.701+0000 7fb9e9e6e740 10 client.8892 fetch_fsmap learned FSMap version 69 2024-04-25T09:02:46.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.701+0000 7fb9e9e6e740 1 -- 172.21.15.43:0/2632398975 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x5650703e8ca0 con 0x5650703bbef0 2024-04-25T09:02:46.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.701+0000 7fb9bf7fe640 1 -- 172.21.15.43:0/2632398975 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d8001250 con 0x5650703bbef0 2024-04-25T09:02:46.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.701+0000 7fb9bf7fe640 1 -- 172.21.15.43:0/2632398975 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb9d4057df0 con 0x5650703bbef0 2024-04-25T09:02:46.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.701+0000 7fb9e9e6e740 10 client.8892 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T09:02:46.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.701+0000 7fb9e9e6e740 10 client.8892 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T09:02:46.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.701+0000 7fb9e9e6e740 20 client.8892 populate_metadata read hostname 'smithi043' 2024-04-25T09:02:46.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.701+0000 7fb9e9e6e740 1 --2- 172.21.15.43:0/2632398975 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x5650703947a0 0x56507038d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.701+0000 7fb9e9e6e740 4 client.8892 mds_command: new command op to 8676 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:02:46.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.701+0000 7fb9e9e6e740 1 -- 172.21.15.43:0/2632398975 --> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5650702ff630 con 0x5650703947a0 2024-04-25T09:02:46.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.701+0000 7fb9e4eff640 1 --2- 172.21.15.43:0/2632398975 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x5650703947a0 0x56507038d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.702+0000 7fb9bf7fe640 10 client.8892 ms_handle_connect on v2:172.21.15.43:6833/1586865031 2024-04-25T09:02:46.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.702+0000 7fb9e4eff640 1 --2- 172.21.15.43:0/2632398975 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x5650703947a0 0x56507038d600 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=793a3f155920014f server_cookie=5a175e6c0013f23a in_seq=0 out_seq=0 2024-04-25T09:02:46.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.702+0000 7fb9bf7fe640 1 -- 172.21.15.43:0/2632398975 <== mds.0 v2:172.21.15.43:6833/1586865031 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x5650702ff630 con 0x5650703947a0 2024-04-25T09:02:46.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.702+0000 7fb9bf7fe640 10 client.8892 handle_command_reply: tid=0 2024-04-25T09:02:46.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.702+0000 7fb9e9e6e740 1 client.8892 shutdown 2024-04-25T09:02:46.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.702+0000 7fb9e9e6e740 1 -- 172.21.15.43:0/2632398975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650703bbef0 msgr2=0x5650703b6750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.702+0000 7fb9e9e6e740 1 --2- 172.21.15.43:0/2632398975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650703bbef0 0x5650703b6750 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fb9d40886f0 tx=0x7fb9d4015040 comp rx=0 tx=0).stop 2024-04-25T09:02:46.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.702+0000 7fb9e9e6e740 1 -- 172.21.15.43:0/2632398975 shutdown_connections 2024-04-25T09:02:46.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.702+0000 7fb9e9e6e740 1 --2- 172.21.15.43:0/2632398975 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x5650703947a0 0x56507038d600 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.702+0000 7fb9e9e6e740 1 --2- 172.21.15.43:0/2632398975 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5650703da450 0x5650703b6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.702+0000 7fb9bf7fe640 0 client.8892 ms_handle_reset on v2:172.21.15.43:6833/1586865031 2024-04-25T09:02:46.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.702+0000 7fb9e9e6e740 1 --2- 172.21.15.43:0/2632398975 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5650703bbef0 0x5650703b6750 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.702+0000 7fb9e9e6e740 1 --2- 172.21.15.43:0/2632398975 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5650703bbb10 0x5650703d9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.702+0000 7fb9e9e6e740 1 -- 172.21.15.43:0/2632398975 >> 172.21.15.43:0/2632398975 conn(0x56507038e770 msgr2=0x5650703be2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:46.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.702+0000 7fb9e9e6e740 1 -- 172.21.15.43:0/2632398975 shutdown_connections 2024-04-25T09:02:46.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.702+0000 7fb9e9e6e740 1 -- 172.21.15.43:0/2632398975 wait complete. 2024-04-25T09:02:46.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.702+0000 7fb9e9e6e740 20 client.8892 trim_cache size 0 max 16384 2024-04-25T09:02:46.697 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:02:46.697 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T09:02:46.697 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-25T09:02:46.697 INFO:teuthology.orchestra.run.smithi043.stdout: "f73a089d-7e85-4d3e-88a9-e0fa4baab2b1": { 2024-04-25T09:02:46.697 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/scrub_control_test_path", 2024-04-25T09:02:46.697 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "f73a089d-7e85-4d3e-88a9-e0fa4baab2b1", 2024-04-25T09:02:46.697 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive" 2024-04-25T09:02:46.697 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:02:46.697 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:02:46.697 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:02:46.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.703+0000 7fb9e7163640 1 -- 172.21.15.43:0/2164019703 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb9cc06b4d0 msgr2=0x7fb9cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.703+0000 7fb9e7163640 1 --2- 172.21.15.43:0/2164019703 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb9cc06b4d0 0x7fb9cc06d990 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb9d0033000 tx=0x7fb9d0008030 comp rx=0 tx=0).stop 2024-04-25T09:02:46.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.703+0000 7fb9e7163640 1 -- 172.21.15.43:0/2164019703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9e00cbcf0 msgr2=0x7fb9e00cd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.703+0000 7fb9e7163640 1 --2- 172.21.15.43:0/2164019703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9e00cbcf0 0x7fb9e00cd680 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4001050 tx=0x7fb9d4007720 comp rx=0 tx=0).stop 2024-04-25T09:02:46.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.703+0000 7fb9e7163640 1 -- 172.21.15.43:0/2164019703 shutdown_connections 2024-04-25T09:02:46.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.703+0000 7fb9e7163640 1 --2- 172.21.15.43:0/2164019703 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb9cc06b4d0 0x7fb9cc06d990 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.703+0000 7fb9e7163640 1 --2- 172.21.15.43:0/2164019703 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9e00cc170 0x7fb9e009d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.703+0000 7fb9e7163640 1 --2- 172.21.15.43:0/2164019703 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9e00cbcf0 0x7fb9e00cd680 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.703+0000 7fb9e7163640 1 --2- 172.21.15.43:0/2164019703 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9c80039b0 0x7fb9e015ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.703+0000 7fb9e7163640 1 -- 172.21.15.43:0/2164019703 >> 172.21.15.43:0/2164019703 conn(0x7fb9e009cd20 msgr2=0x7fb9e006b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:46.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.703+0000 7fb9e7163640 1 -- 172.21.15.43:0/2164019703 shutdown_connections 2024-04-25T09:02:46.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.703+0000 7fb9e7163640 1 -- 172.21.15.43:0/2164019703 wait complete. 2024-04-25T09:02:46.702 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:02:46.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.820+0000 7f2316b4c640 1 Processor -- start 2024-04-25T09:02:46.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.820+0000 7f2316b4c640 1 -- start start 2024-04-25T09:02:46.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.821+0000 7f2316b4c640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f2310070b20 0x7f2310072080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.821+0000 7f2316b4c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f23101582b0 0x7f23100aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.821+0000 7f2316b4c640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f231006f0e0 con 0x7f231015e0f0 2024-04-25T09:02:46.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.821+0000 7f2316b4c640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23100725c0 con 0x7f2310070b20 2024-04-25T09:02:46.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.821+0000 7f2316b4c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23100ad0f0 con 0x7f23101582b0 2024-04-25T09:02:46.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.821+0000 7f230ffff640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f2310070b20 0x7f2310072080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.821+0000 7f23148e8640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f231015e0f0 0x7f2310157ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:56152/0 (socket says 172.21.15.43:56152) 2024-04-25T09:02:46.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.821+0000 7f23150e9640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f23101582b0 0x7f23100aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.821+0000 7f23148e8640 1 -- 172.21.15.43:0/1610104538 learned_addr learned my addr 172.21.15.43:0/1610104538 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:46.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.821+0000 7f230ffff640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f2310070b20 0x7f2310072080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:36080/0 (socket says 172.21.15.43:36080) 2024-04-25T09:02:46.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.821+0000 7f230ffff640 1 -- 172.21.15.43:0/1610104538 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f23101582b0 msgr2=0x7f23100aeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:46.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.821+0000 7f230ffff640 1 --2- 172.21.15.43:0/1610104538 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f23101582b0 0x7f23100aeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.821+0000 7f230ffff640 1 -- 172.21.15.43:0/1610104538 >> v1:172.21.15.193:6789/0 conn(0x7f231015e0f0 legacy=0x7f2310157ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.822+0000 7f230f7fe640 1 -- 172.21.15.43:0/1610104538 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2890740146 0 0) 0x7f2304001110 con 0x7f231015e0f0 2024-04-25T09:02:46.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.822+0000 7f230ffff640 1 -- 172.21.15.43:0/1610104538 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2310070fe0 con 0x7f2310070b20 2024-04-25T09:02:46.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.822+0000 7f230ffff640 1 --2- 172.21.15.43:0/1610104538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2310070b20 0x7f2310072080 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f22fc0097a0 tx=0x7f22fc02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6292245bc6db024d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:46.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.823+0000 7f230f7fe640 1 -- 172.21.15.43:0/1610104538 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22fc037070 con 0x7f2310070b20 2024-04-25T09:02:46.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.823+0000 7f230f7fe640 1 -- 172.21.15.43:0/1610104538 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22fc032440 con 0x7f2310070b20 2024-04-25T09:02:46.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.823+0000 7f230f7fe640 1 -- 172.21.15.43:0/1610104538 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22fc032810 con 0x7f2310070b20 2024-04-25T09:02:46.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.823+0000 7f2316b4c640 1 -- 172.21.15.43:0/1610104538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2310070b20 msgr2=0x7f2310072080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.823+0000 7f2316b4c640 1 --2- 172.21.15.43:0/1610104538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2310070b20 0x7f2310072080 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f22fc0097a0 tx=0x7f22fc02a180 comp rx=0 tx=0).stop 2024-04-25T09:02:46.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.823+0000 7f2316b4c640 1 -- 172.21.15.43:0/1610104538 shutdown_connections 2024-04-25T09:02:46.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.823+0000 7f2316b4c640 1 --2- 172.21.15.43:0/1610104538 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f23101582b0 0x7f23100aeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.823+0000 7f2316b4c640 1 --2- 172.21.15.43:0/1610104538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2310070b20 0x7f2310072080 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.823+0000 7f2316b4c640 1 -- 172.21.15.43:0/1610104538 >> 172.21.15.43:0/1610104538 conn(0x7f231006ab30 msgr2=0x7f231006af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:46.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.823+0000 7f2316b4c640 1 -- 172.21.15.43:0/1610104538 shutdown_connections 2024-04-25T09:02:46.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.823+0000 7f2316b4c640 1 -- 172.21.15.43:0/1610104538 wait complete. 2024-04-25T09:02:46.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.825+0000 7f2316b4c640 1 Processor -- start 2024-04-25T09:02:46.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.825+0000 7f2316b4c640 1 -- start start 2024-04-25T09:02:46.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.826+0000 7f2316b4c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2310070b20 0x7f2310156f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.826+0000 7f2316b4c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23101582b0 0x7f23100ce100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.826+0000 7f2316b4c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f231015e0f0 0x7f2310131810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.826+0000 7f2316b4c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2310071880 con 0x7f23101582b0 2024-04-25T09:02:46.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.826+0000 7f2316b4c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23100715a0 con 0x7f231015e0f0 2024-04-25T09:02:46.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.826+0000 7f2316b4c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2310071b60 con 0x7f2310070b20 2024-04-25T09:02:46.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.826+0000 7f230ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23101582b0 0x7f23100ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.826+0000 7f230ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23101582b0 0x7f23100ce100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58850/0 (socket says 172.21.15.43:58850) 2024-04-25T09:02:46.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.826+0000 7f230ffff640 1 -- 172.21.15.43:0/1914517282 learned_addr learned my addr 172.21.15.43:0/1914517282 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:46.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.826+0000 7f23150e9640 1 --2- 172.21.15.43:0/1914517282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f231015e0f0 0x7f2310131810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.826+0000 7f23148e8640 1 --2- 172.21.15.43:0/1914517282 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2310070b20 0x7f2310156f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.826+0000 7f230ffff640 1 -- 172.21.15.43:0/1914517282 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2310070b20 msgr2=0x7f2310156f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.826+0000 7f230ffff640 1 --2- 172.21.15.43:0/1914517282 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2310070b20 0x7f2310156f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.826+0000 7f230ffff640 1 -- 172.21.15.43:0/1914517282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f231015e0f0 msgr2=0x7f2310131810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:46.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.826+0000 7f230ffff640 1 --2- 172.21.15.43:0/1914517282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f231015e0f0 0x7f2310131810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:46.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.826+0000 7f230ffff640 1 -- 172.21.15.43:0/1914517282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23100cebc0 con 0x7f23101582b0 2024-04-25T09:02:46.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.827+0000 7f230ffff640 1 --2- 172.21.15.43:0/1914517282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23101582b0 0x7f23100ce100 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f22fc035040 tx=0x7f22fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:46.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.827+0000 7f230d7fa640 1 -- 172.21.15.43:0/1914517282 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22fc037040 con 0x7f23101582b0 2024-04-25T09:02:46.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.827+0000 7f230d7fa640 1 -- 172.21.15.43:0/1914517282 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22fc030500 con 0x7f23101582b0 2024-04-25T09:02:46.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.827+0000 7f2316b4c640 1 -- 172.21.15.43:0/1914517282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23100ceea0 con 0x7f23101582b0 2024-04-25T09:02:46.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.827+0000 7f230d7fa640 1 -- 172.21.15.43:0/1914517282 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22fc030910 con 0x7f23101582b0 2024-04-25T09:02:46.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.827+0000 7f2316b4c640 1 -- 172.21.15.43:0/1914517282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23100cf460 con 0x7f23101582b0 2024-04-25T09:02:46.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.828+0000 7f2316b4c640 1 -- 172.21.15.43:0/1914517282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2310074ef0 con 0x7f23101582b0 2024-04-25T09:02:46.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.829+0000 7f230d7fa640 1 -- 172.21.15.43:0/1914517282 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f22fc044020 con 0x7f23101582b0 2024-04-25T09:02:46.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.829+0000 7f230d7fa640 1 --2- 172.21.15.43:0/1914517282 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f22f406b4d0 0x7f22f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:46.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.829+0000 7f230d7fa640 1 -- 172.21.15.43:0/1914517282 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f22fc0aa380 con 0x7f23101582b0 2024-04-25T09:02:46.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.829+0000 7f23148e8640 1 --2- 172.21.15.43:0/1914517282 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f22f406b4d0 0x7f22f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:46.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.830+0000 7f23148e8640 1 --2- 172.21.15.43:0/1914517282 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f22f406b4d0 0x7f22f406d990 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f2304007fd0 tx=0x7f2304007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:46.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:46.834+0000 7f230d7fa640 1 -- 172.21.15.43:0/1914517282 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f22fc078bb0 con 0x7f23101582b0 2024-04-25T09:02:47.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.056+0000 7f2316b4c640 1 -- 172.21.15.43:0/1914517282 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23100680a0 con 0x7f23101582b0 2024-04-25T09:02:47.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.057+0000 7f230d7fa640 1 -- 172.21.15.43:0/1914517282 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f22fc07cc50 con 0x7f23101582b0 2024-04-25T09:02:47.046 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:47.046 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":69,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6821,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2021228703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2021228703},{"type":"v1","addr":"172.21.15.193:6837","nonce":2021228703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65},{"gid":7543,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/2398324998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2398324998},{"type":"v1","addr":"172.21.15.193:6836","nonce":2398324998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65},{"gid":8673,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2298187831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2298187831},{"type":"v1","addr":"172.21.15.43:6834","nonce":2298187831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:02:26.161487+0000","modified":"2024-04-25T09:02:29.177294+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8676":{"gid":8676,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/1586865031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1586865031},{"type":"v1","addr":"172.21.15.43:6835","nonce":1586865031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:02:47.046 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 69 2024-04-25T09:02:47.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.059+0000 7f2316b4c640 1 -- 172.21.15.43:0/1914517282 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f22f406b4d0 msgr2=0x7f22f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:47.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.059+0000 7f2316b4c640 1 --2- 172.21.15.43:0/1914517282 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f22f406b4d0 0x7f22f406d990 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f2304007fd0 tx=0x7f2304007de0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.059+0000 7f2316b4c640 1 -- 172.21.15.43:0/1914517282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23101582b0 msgr2=0x7f23100ce100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:47.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.059+0000 7f2316b4c640 1 --2- 172.21.15.43:0/1914517282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23101582b0 0x7f23100ce100 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f22fc035040 tx=0x7f22fc0046c0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.059+0000 7f2316b4c640 1 -- 172.21.15.43:0/1914517282 shutdown_connections 2024-04-25T09:02:47.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.059+0000 7f2316b4c640 1 --2- 172.21.15.43:0/1914517282 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f22f406b4d0 0x7f22f406d990 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.059+0000 7f2316b4c640 1 --2- 172.21.15.43:0/1914517282 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f231015e0f0 0x7f2310131810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.059+0000 7f2316b4c640 1 --2- 172.21.15.43:0/1914517282 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f23101582b0 0x7f23100ce100 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.059+0000 7f2316b4c640 1 --2- 172.21.15.43:0/1914517282 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2310070b20 0x7f2310156f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.059+0000 7f2316b4c640 1 -- 172.21.15.43:0/1914517282 >> 172.21.15.43:0/1914517282 conn(0x7f231006ab30 msgr2=0x7f23100b4730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:47.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.059+0000 7f2316b4c640 1 -- 172.21.15.43:0/1914517282 shutdown_connections 2024-04-25T09:02:47.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.059+0000 7f2316b4c640 1 -- 172.21.15.43:0/1914517282 wait complete. 2024-04-25T09:02:47.058 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T09:02:47.059 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:02:47.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.169+0000 7f8cdc332640 1 Processor -- start 2024-04-25T09:02:47.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.169+0000 7f8cdc332640 1 -- start start 2024-04-25T09:02:47.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.170+0000 7f8cdc332640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8cd415dce0 0x7f8cd415ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:47.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.170+0000 7f8cdc332640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f8cd40ce550 0x7f8cd40d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:47.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.170+0000 7f8cdc332640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cd40cee80 con 0x7f8cd40ce990 2024-04-25T09:02:47.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.170+0000 7f8cdc332640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cd40d2920 con 0x7f8cd40ce550 2024-04-25T09:02:47.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.170+0000 7f8cdc332640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd40cf160 con 0x7f8cd415dce0 2024-04-25T09:02:47.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.170+0000 7f8cda0ce640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8cd415dce0 0x7f8cd415ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:47.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.170+0000 7f8cd98cd640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f8cd40ce550 0x7f8cd40d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:47.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.170+0000 7f8cda8cf640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f8cd40ce990 0x7f8cd4160980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:56154/0 (socket says 172.21.15.43:56154) 2024-04-25T09:02:47.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.170+0000 7f8cda0ce640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8cd415dce0 0x7f8cd415ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:41930/0 (socket says 172.21.15.43:41930) 2024-04-25T09:02:47.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.170+0000 7f8cd98cd640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f8cd40ce550 0x7f8cd40d03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:36094/0 (socket says 172.21.15.43:36094) 2024-04-25T09:02:47.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.170+0000 7f8cda8cf640 1 -- 172.21.15.43:0/3999990588 learned_addr learned my addr 172.21.15.43:0/3999990588 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:47.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.171+0000 7f8cd98cd640 1 -- 172.21.15.43:0/3999990588 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8cd415dce0 msgr2=0x7f8cd415ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:47.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.171+0000 7f8cd98cd640 1 --2- 172.21.15.43:0/3999990588 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8cd415dce0 0x7f8cd415ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.171+0000 7f8cd98cd640 1 -- 172.21.15.43:0/3999990588 >> v1:172.21.15.193:6789/0 conn(0x7f8cd40ce990 legacy=0x7f8cd4160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:47.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.171+0000 7f8cd98cd640 1 -- 172.21.15.43:0/3999990588 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd40cf440 con 0x7f8cd40ce550 2024-04-25T09:02:47.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.171+0000 7f8cd90cc640 1 -- 172.21.15.43:0/3999990588 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 31826774 0 0) 0x7f8cd40cee80 con 0x7f8cd40ce990 2024-04-25T09:02:47.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.171+0000 7f8cda0ce640 1 --2- 172.21.15.43:0/3999990588 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8cd415dce0 0x7f8cd415ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:02:47.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.172+0000 7f8cd98cd640 1 --2- 172.21.15.43:0/3999990588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8cd40ce550 0x7f8cd40d03e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8cc40097a0 tx=0x7f8cc402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cf50276bf15fac1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:47.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.172+0000 7f8cd90cc640 1 -- 172.21.15.43:0/3999990588 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4037070 con 0x7f8cd40ce550 2024-04-25T09:02:47.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.172+0000 7f8cd90cc640 1 -- 172.21.15.43:0/3999990588 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc402ae50 con 0x7f8cd40ce550 2024-04-25T09:02:47.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.172+0000 7f8cd90cc640 1 -- 172.21.15.43:0/3999990588 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4032660 con 0x7f8cd40ce550 2024-04-25T09:02:47.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.172+0000 7f8cdc332640 1 -- 172.21.15.43:0/3999990588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8cd40ce550 msgr2=0x7f8cd40d03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:47.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.172+0000 7f8cdc332640 1 --2- 172.21.15.43:0/3999990588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8cd40ce550 0x7f8cd40d03e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8cc40097a0 tx=0x7f8cc402a100 comp rx=0 tx=0).stop 2024-04-25T09:02:47.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.172+0000 7f8cdc332640 1 -- 172.21.15.43:0/3999990588 shutdown_connections 2024-04-25T09:02:47.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.172+0000 7f8cdc332640 1 --2- 172.21.15.43:0/3999990588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8cd40ce550 0x7f8cd40d03e0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.172+0000 7f8cdc332640 1 --2- 172.21.15.43:0/3999990588 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8cd415dce0 0x7f8cd415ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.172+0000 7f8cdc332640 1 -- 172.21.15.43:0/3999990588 >> 172.21.15.43:0/3999990588 conn(0x7f8cd409cd00 msgr2=0x7f8cd40d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:47.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.172+0000 7f8cdc332640 1 -- 172.21.15.43:0/3999990588 shutdown_connections 2024-04-25T09:02:47.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.172+0000 7f8cdc332640 1 -- 172.21.15.43:0/3999990588 wait complete. 2024-04-25T09:02:47.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.173+0000 7f8cdc332640 1 Processor -- start 2024-04-25T09:02:47.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.174+0000 7f8cdc332640 1 -- start start 2024-04-25T09:02:47.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.174+0000 7f8cdc332640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8cd40ce550 0x7f8cd41f38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:47.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.174+0000 7f8cdc332640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cd40ce990 0x7f8cd409fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:47.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.174+0000 7f8cdc332640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8cd415dce0 0x7f8cd40a0170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:47.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.174+0000 7f8cdc332640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd415e0c0 con 0x7f8cd40ce990 2024-04-25T09:02:47.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.174+0000 7f8cdc332640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd415f6b0 con 0x7f8cd415dce0 2024-04-25T09:02:47.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.174+0000 7f8cdc332640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd40d0920 con 0x7f8cd40ce550 2024-04-25T09:02:47.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.174+0000 7f8cd98cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cd40ce990 0x7f8cd409fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:47.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.174+0000 7f8cd98cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cd40ce990 0x7f8cd409fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58854/0 (socket says 172.21.15.43:58854) 2024-04-25T09:02:47.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.174+0000 7f8cda0ce640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8cd40ce550 0x7f8cd41f38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:47.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.174+0000 7f8cd98cd640 1 -- 172.21.15.43:0/1575400781 learned_addr learned my addr 172.21.15.43:0/1575400781 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:47.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.174+0000 7f8cda8cf640 1 --2- 172.21.15.43:0/1575400781 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8cd415dce0 0x7f8cd40a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:47.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.175+0000 7f8cd98cd640 1 -- 172.21.15.43:0/1575400781 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8cd40ce550 msgr2=0x7f8cd41f38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:47.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.175+0000 7f8cd98cd640 1 --2- 172.21.15.43:0/1575400781 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8cd40ce550 0x7f8cd41f38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.175+0000 7f8cd98cd640 1 -- 172.21.15.43:0/1575400781 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8cd415dce0 msgr2=0x7f8cd40a0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:47.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.175+0000 7f8cd98cd640 1 --2- 172.21.15.43:0/1575400781 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8cd415dce0 0x7f8cd40a0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.175+0000 7f8cd98cd640 1 -- 172.21.15.43:0/1575400781 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd409ecf0 con 0x7f8cd40ce990 2024-04-25T09:02:47.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.175+0000 7f8cd98cd640 1 --2- 172.21.15.43:0/1575400781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cd40ce990 0x7f8cd409fc30 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4035040 tx=0x7f8cc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:47.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.175+0000 7f8ccaffd640 1 -- 172.21.15.43:0/1575400781 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4041070 con 0x7f8cd40ce990 2024-04-25T09:02:47.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.175+0000 7f8ccaffd640 1 -- 172.21.15.43:0/1575400781 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc403c460 con 0x7f8cd40ce990 2024-04-25T09:02:47.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.175+0000 7f8cdc332640 1 -- 172.21.15.43:0/1575400781 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cd409efd0 con 0x7f8cd40ce990 2024-04-25T09:02:47.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.175+0000 7f8ccaffd640 1 -- 172.21.15.43:0/1575400781 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4037040 con 0x7f8cd40ce990 2024-04-25T09:02:47.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.175+0000 7f8cdc332640 1 -- 172.21.15.43:0/1575400781 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cd409f590 con 0x7f8cd40ce990 2024-04-25T09:02:47.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.176+0000 7f8cdc332640 1 -- 172.21.15.43:0/1575400781 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cd40680f0 con 0x7f8cd40ce990 2024-04-25T09:02:47.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.177+0000 7f8ccaffd640 1 -- 172.21.15.43:0/1575400781 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8cc403a070 con 0x7f8cd40ce990 2024-04-25T09:02:47.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.177+0000 7f8ccaffd640 1 --2- 172.21.15.43:0/1575400781 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8cb806b4d0 0x7f8cb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:47.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.177+0000 7f8ccaffd640 1 -- 172.21.15.43:0/1575400781 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8cc40aa330 con 0x7f8cd40ce990 2024-04-25T09:02:47.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.177+0000 7f8cda0ce640 1 --2- 172.21.15.43:0/1575400781 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8cb806b4d0 0x7f8cb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:47.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.178+0000 7f8cda0ce640 1 --2- 172.21.15.43:0/1575400781 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8cb806b4d0 0x7f8cb806d990 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f8cd0009560 tx=0x7f8cd0009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:47.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.181+0000 7f8ccaffd640 1 -- 172.21.15.43:0/1575400781 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8cc4078ac0 con 0x7f8cd40ce990 2024-04-25T09:02:47.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.436+0000 7f8cdc332640 1 -- 172.21.15.43:0/1575400781 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f8cd4074ef0 con 0x7f8cd40ce990 2024-04-25T09:02:47.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.436+0000 7f8ccaffd640 1 -- 172.21.15.43:0/1575400781 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1444 (secure 0 0 0) 0x7f8cc407cb60 con 0x7f8cd40ce990 2024-04-25T09:02:47.425 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:47.425 INFO:teuthology.orchestra.run.smithi043.stdout:{"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":539,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":113,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035238,"num_in_osds":8,"osd_in_since":1714035229,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":844,"data_bytes":862220978,"bytes_used":2149277696,"bytes_avail":770944835584,"bytes_total":773094113280,"write_bytes_sec":71795494,"read_op_per_sec":0,"write_op_per_sec":70},"fsmap":{"epoch":69,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8676}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2024-04-25T09:02:15.806661+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:02:47.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.438+0000 7f8cdc332640 1 -- 172.21.15.43:0/1575400781 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8cb806b4d0 msgr2=0x7f8cb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:47.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.438+0000 7f8cdc332640 1 --2- 172.21.15.43:0/1575400781 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8cb806b4d0 0x7f8cb806d990 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f8cd0009560 tx=0x7f8cd0009190 comp rx=0 tx=0).stop 2024-04-25T09:02:47.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.438+0000 7f8cdc332640 1 -- 172.21.15.43:0/1575400781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cd40ce990 msgr2=0x7f8cd409fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:47.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.438+0000 7f8cdc332640 1 --2- 172.21.15.43:0/1575400781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cd40ce990 0x7f8cd409fc30 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4035040 tx=0x7f8cc40046c0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.438+0000 7f8cdc332640 1 -- 172.21.15.43:0/1575400781 shutdown_connections 2024-04-25T09:02:47.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.438+0000 7f8cdc332640 1 --2- 172.21.15.43:0/1575400781 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8cb806b4d0 0x7f8cb806d990 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.438+0000 7f8cdc332640 1 --2- 172.21.15.43:0/1575400781 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8cd415dce0 0x7f8cd40a0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.438+0000 7f8cdc332640 1 --2- 172.21.15.43:0/1575400781 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8cd40ce990 0x7f8cd409fc30 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.438+0000 7f8cdc332640 1 --2- 172.21.15.43:0/1575400781 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8cd40ce550 0x7f8cd41f38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:47.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.438+0000 7f8cdc332640 1 -- 172.21.15.43:0/1575400781 >> 172.21.15.43:0/1575400781 conn(0x7f8cd409cd00 msgr2=0x7f8cd40c74a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:47.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.438+0000 7f8cdc332640 1 -- 172.21.15.43:0/1575400781 shutdown_connections 2024-04-25T09:02:47.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:47.438+0000 7f8cdc332640 1 -- 172.21.15.43:0/1575400781 wait complete. 2024-04-25T09:02:47.438 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T09:02:48.439 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:02:48.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.564+0000 7f744b1b0640 1 Processor -- start 2024-04-25T09:02:48.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.564+0000 7f744b1b0640 1 -- start start 2024-04-25T09:02:48.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.564+0000 7f744b1b0640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f74440d39d0 0x7f74440d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:48.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.564+0000 7f744b1b0640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f74440ce2a0 0x7f74440d3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:48.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.564+0000 7f744b1b0640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74440b6c50 con 0x7f7444161710 2024-04-25T09:02:48.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.564+0000 7f744b1b0640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f744415e010 con 0x7f74440ce2a0 2024-04-25T09:02:48.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.564+0000 7f744b1b0640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74440d2020 con 0x7f74440d39d0 2024-04-25T09:02:48.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.565+0000 7f744974d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f7444161710 0x7f7444161ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48450/0 (socket says 172.21.15.43:48450) 2024-04-25T09:02:48.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.565+0000 7f744974d640 1 -- 172.21.15.43:0/756277434 learned_addr learned my addr 172.21.15.43:0/756277434 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:48.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.565+0000 7f7448f4c640 1 --2- 172.21.15.43:0/756277434 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f74440d39d0 0x7f74440d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:48.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.565+0000 7f7443fff640 1 --2- 172.21.15.43:0/756277434 >> v2:172.21.15.193:3300/0 conn(0x7f74440ce2a0 0x7f74440d3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:48.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.565+0000 7f74437fe640 1 -- 172.21.15.43:0/756277434 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1739889436 0 0) 0x7f743c001490 con 0x7f7444161710 2024-04-25T09:02:48.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.565+0000 7f74437fe640 1 -- 172.21.15.43:0/756277434 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7434003520 con 0x7f7444161710 2024-04-25T09:02:48.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.565+0000 7f74437fe640 1 -- 172.21.15.43:0/756277434 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 921143224 0 0) 0x7f7434003520 con 0x7f7444161710 2024-04-25T09:02:48.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.565+0000 7f74437fe640 1 -- 172.21.15.43:0/756277434 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f74440d39d0 msgr2=0x7f74440d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:48.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.565+0000 7f74437fe640 1 --2- 172.21.15.43:0/756277434 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f74440d39d0 0x7f74440d0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:48.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.565+0000 7f74437fe640 1 -- 172.21.15.43:0/756277434 >> v2:172.21.15.193:3300/0 conn(0x7f74440ce2a0 msgr2=0x7f74440d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:48.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.565+0000 7f74437fe640 1 --2- 172.21.15.43:0/756277434 >> v2:172.21.15.193:3300/0 conn(0x7f74440ce2a0 0x7f74440d3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:48.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.565+0000 7f74437fe640 1 -- 172.21.15.43:0/756277434 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74440d42f0 con 0x7f7444161710 2024-04-25T09:02:48.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.565+0000 7f74437fe640 1 -- 172.21.15.43:0/756277434 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f743c001bc0 con 0x7f7444161710 2024-04-25T09:02:48.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.565+0000 7f74437fe640 1 -- 172.21.15.43:0/756277434 >> v1:172.21.15.43:6789/0 conn(0x7f7444161710 legacy=0x7f7444161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:48.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.565+0000 7f74437fe640 1 --2- 172.21.15.43:0/756277434 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f74440d39d0 0x7f74340039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:48.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.565+0000 7f74437fe640 1 --2- 172.21.15.43:0/756277434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7434003f30 0x7f7434004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:48.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.565+0000 7f74437fe640 1 --2- 172.21.15.43:0/756277434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7434004d40 0x7f74340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:48.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.565+0000 7f74437fe640 1 -- 172.21.15.43:0/756277434 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f744415e010 con 0x7f7434004d40 2024-04-25T09:02:48.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.565+0000 7f74437fe640 1 -- 172.21.15.43:0/756277434 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74440d2020 con 0x7f7434003f30 2024-04-25T09:02:48.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.565+0000 7f74437fe640 1 -- 172.21.15.43:0/756277434 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f743400ba10 con 0x7f74440d39d0 2024-04-25T09:02:48.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.565+0000 7f744974d640 1 --2- 172.21.15.43:0/756277434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7434004d40 0x7f74340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:48.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.565+0000 7f7448f4c640 1 --2- 172.21.15.43:0/756277434 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f74440d39d0 0x7f74340039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:48.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.565+0000 7f7443fff640 1 --2- 172.21.15.43:0/756277434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7434003f30 0x7f7434004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:48.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.566+0000 7f744974d640 1 -- 172.21.15.43:0/756277434 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f74440d39d0 msgr2=0x7f74340039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:48.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.566+0000 7f744974d640 1 --2- 172.21.15.43:0/756277434 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f74440d39d0 0x7f74340039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:48.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.566+0000 7f744974d640 1 -- 172.21.15.43:0/756277434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7434003f30 msgr2=0x7f7434004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:48.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.566+0000 7f744974d640 1 --2- 172.21.15.43:0/756277434 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7434003f30 0x7f7434004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:48.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.566+0000 7f744974d640 1 -- 172.21.15.43:0/756277434 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7434003520 con 0x7f7434004d40 2024-04-25T09:02:48.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.566+0000 7f744974d640 1 --2- 172.21.15.43:0/756277434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7434004d40 0x7f74340051c0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f743c00aaa0 tx=0x7f7434005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f194b78eef12ab7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:48.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.567+0000 7f74437fe640 1 -- 172.21.15.43:0/756277434 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743c00a690 con 0x7f7434004d40 2024-04-25T09:02:48.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.567+0000 7f74437fe640 1 -- 172.21.15.43:0/756277434 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f743c02f070 con 0x7f7434004d40 2024-04-25T09:02:48.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.567+0000 7f74437fe640 1 -- 172.21.15.43:0/756277434 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743c034470 con 0x7f7434004d40 2024-04-25T09:02:48.560 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.567+0000 7f744b1b0640 1 -- 172.21.15.43:0/756277434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7434004d40 msgr2=0x7f74340051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:48.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.567+0000 7f744b1b0640 1 --2- 172.21.15.43:0/756277434 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7434004d40 0x7f74340051c0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f743c00aaa0 tx=0x7f7434005830 comp rx=0 tx=0).stop 2024-04-25T09:02:48.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.567+0000 7f744974d640 1 -- 172.21.15.43:0/756277434 reap_dead start 2024-04-25T09:02:48.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.567+0000 7f744b1b0640 1 -- 172.21.15.43:0/756277434 shutdown_connections 2024-04-25T09:02:48.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.567+0000 7f744b1b0640 1 -- 172.21.15.43:0/756277434 >> 172.21.15.43:0/756277434 conn(0x7f744406b000 msgr2=0x7f744406b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:48.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.567+0000 7f744b1b0640 1 -- 172.21.15.43:0/756277434 shutdown_connections 2024-04-25T09:02:48.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.567+0000 7f744b1b0640 1 -- 172.21.15.43:0/756277434 wait complete. 2024-04-25T09:02:48.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.568+0000 7f744b1b0640 1 Processor -- start 2024-04-25T09:02:48.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.569+0000 7f744b1b0640 1 -- start start 2024-04-25T09:02:48.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.569+0000 7f744b1b0640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7434004d40 0x7f744415f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:48.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.569+0000 7f744b1b0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74440b5720 0x7f74441587d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:48.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.569+0000 7f744b1b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7444158d10 0x7f74441f3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:48.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.569+0000 7f744b1b0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74440b2f00 con 0x7f7444158d10 2024-04-25T09:02:48.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.569+0000 7f744b1b0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74440cef20 con 0x7f74440b5720 2024-04-25T09:02:48.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.569+0000 7f744b1b0640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74440cf200 con 0x7f7434004d40 2024-04-25T09:02:48.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.570+0000 7f744974d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7444158d10 0x7f74441f3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:48.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.570+0000 7f744974d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7444158d10 0x7f74441f3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58878/0 (socket says 172.21.15.43:58878) 2024-04-25T09:02:48.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.570+0000 7f744974d640 1 -- 172.21.15.43:0/3597448150 learned_addr learned my addr 172.21.15.43:0/3597448150 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:48.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.570+0000 7f7443fff640 1 --2- 172.21.15.43:0/3597448150 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74440b5720 0x7f74441587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:48.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.570+0000 7f7448f4c640 1 --2- 172.21.15.43:0/3597448150 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7434004d40 0x7f744415f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:48.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.570+0000 7f744974d640 1 -- 172.21.15.43:0/3597448150 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7434004d40 msgr2=0x7f744415f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:48.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.570+0000 7f744974d640 1 --2- 172.21.15.43:0/3597448150 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7434004d40 0x7f744415f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:48.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.570+0000 7f744974d640 1 -- 172.21.15.43:0/3597448150 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74440b5720 msgr2=0x7f74441587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:48.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.570+0000 7f744974d640 1 --2- 172.21.15.43:0/3597448150 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74440b5720 0x7f74441587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:48.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.570+0000 7f744974d640 1 -- 172.21.15.43:0/3597448150 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74441591b0 con 0x7f7444158d10 2024-04-25T09:02:48.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.570+0000 7f7448f4c640 1 --2- 172.21.15.43:0/3597448150 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7434004d40 0x7f744415f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:48.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.570+0000 7f744974d640 1 --2- 172.21.15.43:0/3597448150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7444158d10 0x7f74441f3420 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f743c007040 tx=0x7f743c001f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:48.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.570+0000 7f74417fa640 1 -- 172.21.15.43:0/3597448150 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743c039070 con 0x7f7444158d10 2024-04-25T09:02:48.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.570+0000 7f74417fa640 1 -- 172.21.15.43:0/3597448150 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f743c001bb0 con 0x7f7444158d10 2024-04-25T09:02:48.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.570+0000 7f74417fa640 1 -- 172.21.15.43:0/3597448150 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743c00a690 con 0x7f7444158d10 2024-04-25T09:02:48.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.570+0000 7f744b1b0640 1 -- 172.21.15.43:0/3597448150 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7444159490 con 0x7f7444158d10 2024-04-25T09:02:48.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.570+0000 7f744b1b0640 1 -- 172.21.15.43:0/3597448150 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7444159a50 con 0x7f7444158d10 2024-04-25T09:02:48.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.572+0000 7f744b1b0640 1 -- 172.21.15.43:0/3597448150 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7444073270 con 0x7f7444158d10 2024-04-25T09:02:48.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.572+0000 7f74417fa640 1 -- 172.21.15.43:0/3597448150 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f743c03d030 con 0x7f7444158d10 2024-04-25T09:02:48.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.572+0000 7f74417fa640 1 --2- 172.21.15.43:0/3597448150 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f742806b4d0 0x7f742806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:48.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.572+0000 7f74417fa640 1 -- 172.21.15.43:0/3597448150 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f743c0a9b00 con 0x7f7444158d10 2024-04-25T09:02:48.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.573+0000 7f7448f4c640 1 --2- 172.21.15.43:0/3597448150 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f742806b4d0 0x7f742806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:48.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.573+0000 7f7448f4c640 1 --2- 172.21.15.43:0/3597448150 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f742806b4d0 0x7f742806d990 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f74440b5420 tx=0x7f7438008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:48.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.577+0000 7f74417fa640 1 -- 172.21.15.43:0/3597448150 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7444073270 con 0x7f7444158d10 2024-04-25T09:02:48.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.797+0000 7f744b1b0640 1 -- 172.21.15.43:0/3597448150 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74440ce880 con 0x7f7444158d10 2024-04-25T09:02:48.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.798+0000 7f74417fa640 1 -- 172.21.15.43:0/3597448150 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f74440b6c50 con 0x7f7444158d10 2024-04-25T09:02:48.787 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:48.787 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":69,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6821,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2021228703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2021228703},{"type":"v1","addr":"172.21.15.193:6837","nonce":2021228703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65},{"gid":7543,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/2398324998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2398324998},{"type":"v1","addr":"172.21.15.193:6836","nonce":2398324998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65},{"gid":8673,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2298187831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2298187831},{"type":"v1","addr":"172.21.15.43:6834","nonce":2298187831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:02:26.161487+0000","modified":"2024-04-25T09:02:29.177294+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8676":{"gid":8676,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/1586865031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1586865031},{"type":"v1","addr":"172.21.15.43:6835","nonce":1586865031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:02:48.787 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 69 2024-04-25T09:02:48.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.800+0000 7f744b1b0640 1 -- 172.21.15.43:0/3597448150 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f742806b4d0 msgr2=0x7f742806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:48.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.800+0000 7f744b1b0640 1 --2- 172.21.15.43:0/3597448150 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f742806b4d0 0x7f742806d990 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f74440b5420 tx=0x7f7438008040 comp rx=0 tx=0).stop 2024-04-25T09:02:48.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.800+0000 7f744b1b0640 1 -- 172.21.15.43:0/3597448150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7444158d10 msgr2=0x7f74441f3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:48.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.800+0000 7f744b1b0640 1 --2- 172.21.15.43:0/3597448150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7444158d10 0x7f74441f3420 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f743c007040 tx=0x7f743c001f90 comp rx=0 tx=0).stop 2024-04-25T09:02:48.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.800+0000 7f744b1b0640 1 -- 172.21.15.43:0/3597448150 shutdown_connections 2024-04-25T09:02:48.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.800+0000 7f744b1b0640 1 --2- 172.21.15.43:0/3597448150 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7444158d10 0x7f74441f3420 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:48.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.800+0000 7f744b1b0640 1 --2- 172.21.15.43:0/3597448150 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f742806b4d0 0x7f742806d990 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:48.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.800+0000 7f744b1b0640 1 --2- 172.21.15.43:0/3597448150 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74440b5720 0x7f74441587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:48.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.800+0000 7f744b1b0640 1 --2- 172.21.15.43:0/3597448150 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7434004d40 0x7f744415f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:48.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.800+0000 7f744b1b0640 1 -- 172.21.15.43:0/3597448150 >> 172.21.15.43:0/3597448150 conn(0x7f744406b000 msgr2=0x7f7444160750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:48.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.800+0000 7f744b1b0640 1 -- 172.21.15.43:0/3597448150 shutdown_connections 2024-04-25T09:02:48.789 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.800+0000 7f744b1b0640 1 -- 172.21.15.43:0/3597448150 wait complete. 2024-04-25T09:02:48.799 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T09:02:48.799 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:02:48.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.911+0000 7f8797709640 1 Processor -- start 2024-04-25T09:02:48.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.912+0000 7f8797709640 1 -- start start 2024-04-25T09:02:48.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.912+0000 7f8797709640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f879015df50 0x7f87901619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:48.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.912+0000 7f8797709640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f87900cf670 0x7f8790162370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:48.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.912+0000 7f8797709640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87900d24a0 con 0x7f8790161f30 2024-04-25T09:02:48.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.912+0000 7f8797709640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87900cfa80 con 0x7f87900cf670 2024-04-25T09:02:48.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.912+0000 7f8797709640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f879015e330 con 0x7f879015df50 2024-04-25T09:02:48.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.913+0000 7f8795ca6640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f87900cf670 0x7f8790162370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:48.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.913+0000 7f8794ca4640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8790161f30 0x7f87900c7cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48464/0 (socket says 172.21.15.43:48464) 2024-04-25T09:02:48.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.913+0000 7f8794ca4640 1 -- 172.21.15.43:0/3614862237 learned_addr learned my addr 172.21.15.43:0/3614862237 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:48.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.913+0000 7f8795ca6640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f87900cf670 0x7f8790162370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58884/0 (socket says 172.21.15.43:58884) 2024-04-25T09:02:48.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.913+0000 7f87954a5640 1 --2- 172.21.15.43:0/3614862237 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f879015df50 0x7f87901619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:48.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.913+0000 7f8795ca6640 1 -- 172.21.15.43:0/3614862237 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f879015df50 msgr2=0x7f87901619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:48.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.913+0000 7f8795ca6640 1 --2- 172.21.15.43:0/3614862237 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f879015df50 0x7f87901619f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:48.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.913+0000 7f8795ca6640 1 -- 172.21.15.43:0/3614862237 >> v1:172.21.15.43:6789/0 conn(0x7f8790161f30 legacy=0x7f87900c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:48.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.913+0000 7f8787fff640 1 -- 172.21.15.43:0/3614862237 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1026213418 0 0) 0x7f87900d24a0 con 0x7f8790161f30 2024-04-25T09:02:48.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.913+0000 7f8795ca6640 1 -- 172.21.15.43:0/3614862237 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87900d2740 con 0x7f87900cf670 2024-04-25T09:02:48.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.914+0000 7f8795ca6640 1 --2- 172.21.15.43:0/3614862237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87900cf670 0x7f8790162370 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f8788009d90 tx=0x7f878802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3cf8bec4e9b8f8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:48.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.914+0000 7f8787fff640 1 -- 172.21.15.43:0/3614862237 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8788007740 con 0x7f87900cf670 2024-04-25T09:02:48.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.914+0000 7f8787fff640 1 -- 172.21.15.43:0/3614862237 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87880078e0 con 0x7f87900cf670 2024-04-25T09:02:48.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.914+0000 7f8787fff640 1 -- 172.21.15.43:0/3614862237 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8788007cb0 con 0x7f87900cf670 2024-04-25T09:02:48.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.914+0000 7f8797709640 1 -- 172.21.15.43:0/3614862237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87900cf670 msgr2=0x7f8790162370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:48.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.914+0000 7f8797709640 1 --2- 172.21.15.43:0/3614862237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87900cf670 0x7f8790162370 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f8788009d90 tx=0x7f878802a660 comp rx=0 tx=0).stop 2024-04-25T09:02:48.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.914+0000 7f8797709640 1 -- 172.21.15.43:0/3614862237 shutdown_connections 2024-04-25T09:02:48.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.914+0000 7f8797709640 1 --2- 172.21.15.43:0/3614862237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87900cf670 0x7f8790162370 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:48.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.914+0000 7f8797709640 1 --2- 172.21.15.43:0/3614862237 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f879015df50 0x7f87901619f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:48.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.914+0000 7f8797709640 1 -- 172.21.15.43:0/3614862237 >> 172.21.15.43:0/3614862237 conn(0x7f87900d3a30 msgr2=0x7f87900d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:48.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.914+0000 7f8797709640 1 -- 172.21.15.43:0/3614862237 shutdown_connections 2024-04-25T09:02:48.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.914+0000 7f8797709640 1 -- 172.21.15.43:0/3614862237 wait complete. 2024-04-25T09:02:48.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.916+0000 7f8797709640 1 Processor -- start 2024-04-25T09:02:48.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.916+0000 7f8797709640 1 -- start start 2024-04-25T09:02:48.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.916+0000 7f8797709640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87900cf670 0x7f8790155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:48.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.916+0000 7f8797709640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f879015df50 0x7f87901606a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:48.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.916+0000 7f8797709640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8790161f30 0x7f87901e98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:48.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.916+0000 7f8797709640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87900c77d0 con 0x7f8790161f30 2024-04-25T09:02:48.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.916+0000 7f8797709640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87900c74f0 con 0x7f87900cf670 2024-04-25T09:02:48.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.916+0000 7f8797709640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87900d2190 con 0x7f879015df50 2024-04-25T09:02:48.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.917+0000 7f8795ca6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8790161f30 0x7f87901e98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:48.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.917+0000 7f8795ca6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8790161f30 0x7f87901e98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:58896/0 (socket says 172.21.15.43:58896) 2024-04-25T09:02:48.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.917+0000 7f8795ca6640 1 -- 172.21.15.43:0/454793435 learned_addr learned my addr 172.21.15.43:0/454793435 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:48.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.917+0000 7f8794ca4640 1 --2- 172.21.15.43:0/454793435 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f879015df50 0x7f87901606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:48.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.917+0000 7f87954a5640 1 --2- 172.21.15.43:0/454793435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87900cf670 0x7f8790155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:48.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.917+0000 7f8795ca6640 1 -- 172.21.15.43:0/454793435 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f879015df50 msgr2=0x7f87901606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:48.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.917+0000 7f8795ca6640 1 --2- 172.21.15.43:0/454793435 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f879015df50 0x7f87901606a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:48.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.917+0000 7f8795ca6640 1 -- 172.21.15.43:0/454793435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87900cf670 msgr2=0x7f8790155870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:48.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.917+0000 7f8795ca6640 1 --2- 172.21.15.43:0/454793435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87900cf670 0x7f8790155870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:48.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.917+0000 7f8795ca6640 1 -- 172.21.15.43:0/454793435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8790156720 con 0x7f8790161f30 2024-04-25T09:02:48.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.917+0000 7f8794ca4640 1 --2- 172.21.15.43:0/454793435 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f879015df50 0x7f87901606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:48.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.917+0000 7f87954a5640 1 --2- 172.21.15.43:0/454793435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87900cf670 0x7f8790155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:48.914 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.917+0000 7f8795ca6640 1 --2- 172.21.15.43:0/454793435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8790161f30 0x7f87901e98a0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f878802ac50 tx=0x7f8788004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:48.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.917+0000 7f8785ffb640 1 -- 172.21.15.43:0/454793435 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8788040070 con 0x7f8790161f30 2024-04-25T09:02:48.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.917+0000 7f8785ffb640 1 -- 172.21.15.43:0/454793435 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8788031c90 con 0x7f8790161f30 2024-04-25T09:02:48.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.917+0000 7f8797709640 1 -- 172.21.15.43:0/454793435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8790160be0 con 0x7f8790161f30 2024-04-25T09:02:48.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.917+0000 7f8785ffb640 1 -- 172.21.15.43:0/454793435 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8788045680 con 0x7f8790161f30 2024-04-25T09:02:48.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.917+0000 7f8797709640 1 -- 172.21.15.43:0/454793435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8790161180 con 0x7f8790161f30 2024-04-25T09:02:48.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.918+0000 7f8797709640 1 -- 172.21.15.43:0/454793435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87900680f0 con 0x7f8790161f30 2024-04-25T09:02:48.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.919+0000 7f8785ffb640 1 -- 172.21.15.43:0/454793435 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8788039040 con 0x7f8790161f30 2024-04-25T09:02:48.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.919+0000 7f8785ffb640 1 --2- 172.21.15.43:0/454793435 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f877406b4d0 0x7f877406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:48.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.919+0000 7f8785ffb640 1 -- 172.21.15.43:0/454793435 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f87880abb60 con 0x7f8790161f30 2024-04-25T09:02:48.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.919+0000 7f87954a5640 1 --2- 172.21.15.43:0/454793435 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f877406b4d0 0x7f877406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:48.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.920+0000 7f87954a5640 1 --2- 172.21.15.43:0/454793435 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f877406b4d0 0x7f877406d990 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f878c008890 tx=0x7f878c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:48.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:48.923+0000 7f8785ffb640 1 -- 172.21.15.43:0/454793435 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8788043360 con 0x7f8790161f30 2024-04-25T09:02:49.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:49.173+0000 7f8797709640 1 -- 172.21.15.43:0/454793435 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f87900ca400 con 0x7f8790161f30 2024-04-25T09:02:49.162 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:49.174+0000 7f8785ffb640 1 -- 172.21.15.43:0/454793435 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1444 (secure 0 0 0) 0x7f878807ab30 con 0x7f8790161f30 2024-04-25T09:02:49.162 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:49.163 INFO:teuthology.orchestra.run.smithi043.stdout:{"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":541,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":113,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035238,"num_in_osds":8,"osd_in_since":1714035229,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":844,"data_bytes":862220978,"bytes_used":2149277696,"bytes_avail":770944835584,"bytes_total":773094113280,"write_bytes_sec":71797024,"read_op_per_sec":0,"write_op_per_sec":70},"fsmap":{"epoch":69,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8676}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":20,"modified":"2024-04-25T09:02:15.806661+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:02:49.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:49.175+0000 7f8797709640 1 -- 172.21.15.43:0/454793435 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f877406b4d0 msgr2=0x7f877406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:49.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:49.175+0000 7f8797709640 1 --2- 172.21.15.43:0/454793435 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f877406b4d0 0x7f877406d990 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f878c008890 tx=0x7f878c008040 comp rx=0 tx=0).stop 2024-04-25T09:02:49.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:49.175+0000 7f8797709640 1 -- 172.21.15.43:0/454793435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8790161f30 msgr2=0x7f87901e98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:49.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:49.175+0000 7f8797709640 1 --2- 172.21.15.43:0/454793435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8790161f30 0x7f87901e98a0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f878802ac50 tx=0x7f8788004a40 comp rx=0 tx=0).stop 2024-04-25T09:02:49.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:49.176+0000 7f8797709640 1 -- 172.21.15.43:0/454793435 shutdown_connections 2024-04-25T09:02:49.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:49.176+0000 7f8797709640 1 --2- 172.21.15.43:0/454793435 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8790161f30 0x7f87901e98a0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:49.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:49.176+0000 7f8797709640 1 --2- 172.21.15.43:0/454793435 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f879015df50 0x7f87901606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:49.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:49.176+0000 7f8797709640 1 --2- 172.21.15.43:0/454793435 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f877406b4d0 0x7f877406d990 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:49.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:49.176+0000 7f8797709640 1 --2- 172.21.15.43:0/454793435 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87900cf670 0x7f8790155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:49.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:49.176+0000 7f8797709640 1 -- 172.21.15.43:0/454793435 >> 172.21.15.43:0/454793435 conn(0x7f87900d3a30 msgr2=0x7f879015bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:49.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:49.176+0000 7f8797709640 1 -- 172.21.15.43:0/454793435 shutdown_connections 2024-04-25T09:02:49.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:49.176+0000 7f8797709640 1 -- 172.21.15.43:0/454793435 wait complete. 2024-04-25T09:02:49.175 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T09:02:50.176 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:02:50.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.300+0000 7fb5ea6c4640 1 Processor -- start 2024-04-25T09:02:50.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.300+0000 7fb5ea6c4640 1 -- start start 2024-04-25T09:02:50.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.301+0000 7fb5ea6c4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb5e40c8d00 0x7fb5e4162100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:50.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.301+0000 7fb5ea6c4640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5e40d24a0 con 0x7fb5e40ce660 2024-04-25T09:02:50.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.301+0000 7fb5ea6c4640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5e40d2740 con 0x7fb5e415df50 2024-04-25T09:02:50.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.301+0000 7fb5ea6c4640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5e415e330 con 0x7fb5e40c8d00 2024-04-25T09:02:50.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.301+0000 7fb5e37fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb5e40c8d00 0x7fb5e4162100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:50.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.301+0000 7fb5e8c61640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb5e40ce660 0x7fb5e40cea70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:53382/0 (socket says 172.21.15.43:53382) 2024-04-25T09:02:50.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.301+0000 7fb5e8c61640 1 -- 172.21.15.43:0/4163802911 learned_addr learned my addr 172.21.15.43:0/4163802911 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:50.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.302+0000 7fb5e37fe640 1 -- 172.21.15.43:0/4163802911 >> v1:172.21.15.193:6789/0 conn(0x7fb5e415df50 legacy=0x7fb5e41619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:50.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.302+0000 7fb5e37fe640 1 -- 172.21.15.43:0/4163802911 >> v1:172.21.15.43:6789/0 conn(0x7fb5e40ce660 legacy=0x7fb5e40cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:50.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.302+0000 7fb5e2ffd640 1 -- 172.21.15.43:0/4163802911 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3676896139 0 0) 0x7fb5e40d24a0 con 0x7fb5e40ce660 2024-04-25T09:02:50.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.302+0000 7fb5e37fe640 1 -- 172.21.15.43:0/4163802911 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e40d2a20 con 0x7fb5e40c8d00 2024-04-25T09:02:50.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.303+0000 7fb5e37fe640 1 --2- 172.21.15.43:0/4163802911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e40c8d00 0x7fb5e4162100 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fb5d80097a0 tx=0x7fb5d802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a42ab7c5529117 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:50.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.303+0000 7fb5e2ffd640 1 -- 172.21.15.43:0/4163802911 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d8037070 con 0x7fb5e40c8d00 2024-04-25T09:02:50.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.303+0000 7fb5e2ffd640 1 -- 172.21.15.43:0/4163802911 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d8032440 con 0x7fb5e40c8d00 2024-04-25T09:02:50.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.303+0000 7fb5e2ffd640 1 -- 172.21.15.43:0/4163802911 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d8032810 con 0x7fb5e40c8d00 2024-04-25T09:02:50.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.303+0000 7fb5ea6c4640 1 -- 172.21.15.43:0/4163802911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e40c8d00 msgr2=0x7fb5e4162100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:50.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.303+0000 7fb5ea6c4640 1 --2- 172.21.15.43:0/4163802911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e40c8d00 0x7fb5e4162100 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fb5d80097a0 tx=0x7fb5d802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:02:50.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.303+0000 7fb5ea6c4640 1 -- 172.21.15.43:0/4163802911 shutdown_connections 2024-04-25T09:02:50.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.303+0000 7fb5ea6c4640 1 --2- 172.21.15.43:0/4163802911 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e40c8d00 0x7fb5e4162100 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:50.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.303+0000 7fb5ea6c4640 1 -- 172.21.15.43:0/4163802911 >> 172.21.15.43:0/4163802911 conn(0x7fb5e40d3a30 msgr2=0x7fb5e40d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:50.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.303+0000 7fb5ea6c4640 1 -- 172.21.15.43:0/4163802911 shutdown_connections 2024-04-25T09:02:50.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.303+0000 7fb5ea6c4640 1 -- 172.21.15.43:0/4163802911 wait complete. 2024-04-25T09:02:50.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.305+0000 7fb5ea6c4640 1 Processor -- start 2024-04-25T09:02:50.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.305+0000 7fb5ea6c4640 1 -- start start 2024-04-25T09:02:50.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.306+0000 7fb5ea6c4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e40c8d00 0x7fb5e4153440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:50.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.306+0000 7fb5ea6c4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5e40ce660 0x7fb5e4156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:50.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.306+0000 7fb5ea6c4640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5e415df50 0x7fb5e40caa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:50.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.306+0000 7fb5ea6c4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e40d2190 con 0x7fb5e40c8d00 2024-04-25T09:02:50.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.306+0000 7fb5ea6c4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e40c77d0 con 0x7fb5e40ce660 2024-04-25T09:02:50.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.306+0000 7fb5ea6c4640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e4162640 con 0x7fb5e415df50 2024-04-25T09:02:50.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.306+0000 7fb5e3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e40c8d00 0x7fb5e4153440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:50.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.306+0000 7fb5e3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e40c8d00 0x7fb5e4153440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48172/0 (socket says 172.21.15.43:48172) 2024-04-25T09:02:50.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.306+0000 7fb5e3fff640 1 -- 172.21.15.43:0/3937486452 learned_addr learned my addr 172.21.15.43:0/3937486452 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:50.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.306+0000 7fb5e37fe640 1 --2- 172.21.15.43:0/3937486452 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5e40ce660 0x7fb5e4156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:50.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.306+0000 7fb5e8c61640 1 --2- 172.21.15.43:0/3937486452 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5e415df50 0x7fb5e40caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:50.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.306+0000 7fb5e3fff640 1 -- 172.21.15.43:0/3937486452 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5e415df50 msgr2=0x7fb5e40caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:50.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.306+0000 7fb5e3fff640 1 --2- 172.21.15.43:0/3937486452 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5e415df50 0x7fb5e40caa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:50.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.306+0000 7fb5e3fff640 1 -- 172.21.15.43:0/3937486452 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5e40ce660 msgr2=0x7fb5e4156580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:50.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.306+0000 7fb5e3fff640 1 --2- 172.21.15.43:0/3937486452 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5e40ce660 0x7fb5e4156580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:50.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.306+0000 7fb5e3fff640 1 -- 172.21.15.43:0/3937486452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e41542b0 con 0x7fb5e40c8d00 2024-04-25T09:02:50.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.306+0000 7fb5e8c61640 1 --2- 172.21.15.43:0/3937486452 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5e415df50 0x7fb5e40caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:50.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.306+0000 7fb5e3fff640 1 --2- 172.21.15.43:0/3937486452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e40c8d00 0x7fb5e4153440 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fb5d400b3f0 tx=0x7fb5d400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:50.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.306+0000 7fb5e0ff9640 1 -- 172.21.15.43:0/3937486452 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d4015020 con 0x7fb5e40c8d00 2024-04-25T09:02:50.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.307+0000 7fb5e0ff9640 1 -- 172.21.15.43:0/3937486452 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d4011830 con 0x7fb5e40c8d00 2024-04-25T09:02:50.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.307+0000 7fb5ea6c4640 1 -- 172.21.15.43:0/3937486452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5e4155aa0 con 0x7fb5e40c8d00 2024-04-25T09:02:50.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.307+0000 7fb5e0ff9640 1 -- 172.21.15.43:0/3937486452 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d4011c20 con 0x7fb5e40c8d00 2024-04-25T09:02:50.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.307+0000 7fb5ea6c4640 1 -- 172.21.15.43:0/3937486452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5e4155700 con 0x7fb5e40c8d00 2024-04-25T09:02:50.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.308+0000 7fb5ea6c4640 1 -- 172.21.15.43:0/3937486452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5e40680f0 con 0x7fb5e40c8d00 2024-04-25T09:02:50.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.308+0000 7fb5e0ff9640 1 -- 172.21.15.43:0/3937486452 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb5d4018020 con 0x7fb5e40c8d00 2024-04-25T09:02:50.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.309+0000 7fb5e0ff9640 1 --2- 172.21.15.43:0/3937486452 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb5d006b4d0 0x7fb5d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:50.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.309+0000 7fb5e0ff9640 1 -- 172.21.15.43:0/3937486452 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb5d4089d10 con 0x7fb5e40c8d00 2024-04-25T09:02:50.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.309+0000 7fb5e37fe640 1 --2- 172.21.15.43:0/3937486452 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb5d006b4d0 0x7fb5d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:50.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.310+0000 7fb5e37fe640 1 --2- 172.21.15.43:0/3937486452 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb5d006b4d0 0x7fb5d006d990 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8034000 tx=0x7fb5d802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:50.301 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.313+0000 7fb5e0ff9640 1 -- 172.21.15.43:0/3937486452 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb5d401f030 con 0x7fb5e40c8d00 2024-04-25T09:02:50.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.529+0000 7fb5ea6c4640 1 -- 172.21.15.43:0/3937486452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb5e4074ef0 con 0x7fb5e40c8d00 2024-04-25T09:02:50.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.530+0000 7fb5e0ff9640 1 -- 172.21.15.43:0/3937486452 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fb5d4058c20 con 0x7fb5e40c8d00 2024-04-25T09:02:50.519 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:50.519 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":69,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6821,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2021228703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2021228703},{"type":"v1","addr":"172.21.15.193:6837","nonce":2021228703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65},{"gid":7543,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/2398324998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2398324998},{"type":"v1","addr":"172.21.15.193:6836","nonce":2398324998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65},{"gid":8673,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2298187831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2298187831},{"type":"v1","addr":"172.21.15.43:6834","nonce":2298187831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:02:26.161487+0000","modified":"2024-04-25T09:02:29.177294+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8676":{"gid":8676,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/1586865031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1586865031},{"type":"v1","addr":"172.21.15.43:6835","nonce":1586865031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:02:50.519 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 69 2024-04-25T09:02:50.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.532+0000 7fb5ea6c4640 1 -- 172.21.15.43:0/3937486452 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb5d006b4d0 msgr2=0x7fb5d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:50.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.532+0000 7fb5ea6c4640 1 --2- 172.21.15.43:0/3937486452 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb5d006b4d0 0x7fb5d006d990 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8034000 tx=0x7fb5d802e000 comp rx=0 tx=0).stop 2024-04-25T09:02:50.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.532+0000 7fb5ea6c4640 1 -- 172.21.15.43:0/3937486452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e40c8d00 msgr2=0x7fb5e4153440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:50.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.532+0000 7fb5ea6c4640 1 --2- 172.21.15.43:0/3937486452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e40c8d00 0x7fb5e4153440 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fb5d400b3f0 tx=0x7fb5d400b8b0 comp rx=0 tx=0).stop 2024-04-25T09:02:50.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.532+0000 7fb5ea6c4640 1 -- 172.21.15.43:0/3937486452 shutdown_connections 2024-04-25T09:02:50.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.532+0000 7fb5ea6c4640 1 --2- 172.21.15.43:0/3937486452 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5e415df50 0x7fb5e40caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:50.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.532+0000 7fb5ea6c4640 1 --2- 172.21.15.43:0/3937486452 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb5d006b4d0 0x7fb5d006d990 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:50.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.532+0000 7fb5ea6c4640 1 --2- 172.21.15.43:0/3937486452 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5e40ce660 0x7fb5e4156580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:50.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.532+0000 7fb5ea6c4640 1 --2- 172.21.15.43:0/3937486452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5e40c8d00 0x7fb5e4153440 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:50.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.532+0000 7fb5ea6c4640 1 -- 172.21.15.43:0/3937486452 >> 172.21.15.43:0/3937486452 conn(0x7fb5e40d3a30 msgr2=0x7fb5e40cf440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:50.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.532+0000 7fb5ea6c4640 1 -- 172.21.15.43:0/3937486452 shutdown_connections 2024-04-25T09:02:50.522 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.532+0000 7fb5ea6c4640 1 -- 172.21.15.43:0/3937486452 wait complete. 2024-04-25T09:02:50.531 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T09:02:50.533 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:02:50.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.645+0000 7fddf489e640 1 Processor -- start 2024-04-25T09:02:50.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.645+0000 7fddf489e640 1 -- start start 2024-04-25T09:02:50.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.646+0000 7fddf489e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fddf00c8d00 0x7fddf0162100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:50.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.646+0000 7fddf489e640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fddf00ce660 0x7fddf00cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:50.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.646+0000 7fddf489e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddf00d24a0 con 0x7fddf015df50 2024-04-25T09:02:50.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.646+0000 7fddf489e640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddf015e330 con 0x7fddf00ce660 2024-04-25T09:02:50.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.646+0000 7fddf489e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddf00d2740 con 0x7fddf00c8d00 2024-04-25T09:02:50.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.646+0000 7fddee59c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fddf015df50 0x7fddf01619f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:53386/0 (socket says 172.21.15.43:53386) 2024-04-25T09:02:50.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.646+0000 7fddee59c640 1 -- 172.21.15.43:0/2459557438 learned_addr learned my addr 172.21.15.43:0/2459557438 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:50.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.646+0000 7fddeed9d640 1 --2- 172.21.15.43:0/2459557438 >> v2:172.21.15.193:3300/0 conn(0x7fddf00ce660 0x7fddf00cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:50.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.646+0000 7fdde7fff640 1 --2- 172.21.15.43:0/2459557438 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fddf00c8d00 0x7fddf0162100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:50.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.646+0000 7fddedd9b640 1 -- 172.21.15.43:0/2459557438 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4259649745 0 0) 0x7fddf00d24a0 con 0x7fddf015df50 2024-04-25T09:02:50.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.646+0000 7fddedd9b640 1 -- 172.21.15.43:0/2459557438 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdddc003520 con 0x7fddf015df50 2024-04-25T09:02:50.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.647+0000 7fddeed9d640 1 -- 172.21.15.43:0/2459557438 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fddf00c8d00 msgr2=0x7fddf0162100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:50.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.647+0000 7fddeed9d640 1 --2- 172.21.15.43:0/2459557438 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fddf00c8d00 0x7fddf0162100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:50.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.647+0000 7fddeed9d640 1 -- 172.21.15.43:0/2459557438 >> v1:172.21.15.43:6789/0 conn(0x7fddf015df50 legacy=0x7fddf01619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:50.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.647+0000 7fddeed9d640 1 -- 172.21.15.43:0/2459557438 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf00d2a20 con 0x7fddf00ce660 2024-04-25T09:02:50.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.647+0000 7fdde7fff640 1 --2- 172.21.15.43:0/2459557438 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fddf00c8d00 0x7fddf0162100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:02:50.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.647+0000 7fddeed9d640 1 --2- 172.21.15.43:0/2459557438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fddf00ce660 0x7fddf00cea70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fdde8009d90 tx=0x7fdde802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a1fdf7f5c9535ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:50.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.648+0000 7fddedd9b640 1 -- 172.21.15.43:0/2459557438 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde8007740 con 0x7fddf00ce660 2024-04-25T09:02:50.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.648+0000 7fddedd9b640 1 -- 172.21.15.43:0/2459557438 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdde80078e0 con 0x7fddf00ce660 2024-04-25T09:02:50.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.648+0000 7fddedd9b640 1 -- 172.21.15.43:0/2459557438 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde8007cb0 con 0x7fddf00ce660 2024-04-25T09:02:50.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.648+0000 7fddf489e640 1 -- 172.21.15.43:0/2459557438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fddf00ce660 msgr2=0x7fddf00cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:50.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.648+0000 7fddf489e640 1 --2- 172.21.15.43:0/2459557438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fddf00ce660 0x7fddf00cea70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fdde8009d90 tx=0x7fdde802a660 comp rx=0 tx=0).stop 2024-04-25T09:02:50.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.648+0000 7fddf489e640 1 -- 172.21.15.43:0/2459557438 shutdown_connections 2024-04-25T09:02:50.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.648+0000 7fddf489e640 1 --2- 172.21.15.43:0/2459557438 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fddf00ce660 0x7fddf00cea70 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:50.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.648+0000 7fddf489e640 1 --2- 172.21.15.43:0/2459557438 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fddf00c8d00 0x7fddf0162100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:50.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.648+0000 7fddf489e640 1 -- 172.21.15.43:0/2459557438 >> 172.21.15.43:0/2459557438 conn(0x7fddf00d3a30 msgr2=0x7fddf00d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:50.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.648+0000 7fddf489e640 1 -- 172.21.15.43:0/2459557438 shutdown_connections 2024-04-25T09:02:50.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.648+0000 7fddf489e640 1 -- 172.21.15.43:0/2459557438 wait complete. 2024-04-25T09:02:50.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.650+0000 7fddf489e640 1 Processor -- start 2024-04-25T09:02:50.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.650+0000 7fddf489e640 1 -- start start 2024-04-25T09:02:50.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.650+0000 7fddf489e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddf00c8d00 0x7fddf00c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:50.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.650+0000 7fddf489e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fddf00ce660 0x7fddf0160da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:50.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.650+0000 7fddf489e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fddf015df50 0x7fddf00cc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:50.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.650+0000 7fddf489e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf00d2190 con 0x7fddf00c8d00 2024-04-25T09:02:50.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.650+0000 7fddf489e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddf00c77d0 con 0x7fddf015df50 2024-04-25T09:02:50.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.650+0000 7fddf489e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddf0162640 con 0x7fddf00ce660 2024-04-25T09:02:50.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.650+0000 7fddee59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddf00c8d00 0x7fddf00c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:50.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.650+0000 7fddee59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddf00c8d00 0x7fddf00c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48180/0 (socket says 172.21.15.43:48180) 2024-04-25T09:02:50.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.650+0000 7fddee59c640 1 -- 172.21.15.43:0/4001939992 learned_addr learned my addr 172.21.15.43:0/4001939992 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:50.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.651+0000 7fdde7fff640 1 --2- 172.21.15.43:0/4001939992 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fddf00ce660 0x7fddf0160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:50.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.651+0000 7fddeed9d640 1 --2- 172.21.15.43:0/4001939992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fddf015df50 0x7fddf00cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:50.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.651+0000 7fddee59c640 1 -- 172.21.15.43:0/4001939992 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fddf00ce660 msgr2=0x7fddf0160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:50.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.651+0000 7fddee59c640 1 --2- 172.21.15.43:0/4001939992 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fddf00ce660 0x7fddf0160da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:50.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.651+0000 7fddee59c640 1 -- 172.21.15.43:0/4001939992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fddf015df50 msgr2=0x7fddf00cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:50.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.651+0000 7fddee59c640 1 --2- 172.21.15.43:0/4001939992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fddf015df50 0x7fddf00cc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:50.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.651+0000 7fddee59c640 1 -- 172.21.15.43:0/4001939992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddf015f0d0 con 0x7fddf00c8d00 2024-04-25T09:02:50.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.651+0000 7fddee59c640 1 --2- 172.21.15.43:0/4001939992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddf00c8d00 0x7fddf00c66f0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fdde000c460 tx=0x7fdde000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:50.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.651+0000 7fdde6ffd640 1 -- 172.21.15.43:0/4001939992 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde0017070 con 0x7fddf00c8d00 2024-04-25T09:02:50.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.651+0000 7fdde6ffd640 1 -- 172.21.15.43:0/4001939992 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdde0005ce0 con 0x7fddf00c8d00 2024-04-25T09:02:50.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.651+0000 7fddf489e640 1 -- 172.21.15.43:0/4001939992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddf015f3b0 con 0x7fddf00c8d00 2024-04-25T09:02:50.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.651+0000 7fdde6ffd640 1 -- 172.21.15.43:0/4001939992 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdde0007290 con 0x7fddf00c8d00 2024-04-25T09:02:50.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.651+0000 7fddf489e640 1 -- 172.21.15.43:0/4001939992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddf01612e0 con 0x7fddf00c8d00 2024-04-25T09:02:50.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.653+0000 7fddf489e640 1 -- 172.21.15.43:0/4001939992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddf00680f0 con 0x7fddf00c8d00 2024-04-25T09:02:50.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.653+0000 7fdde6ffd640 1 -- 172.21.15.43:0/4001939992 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdde001a070 con 0x7fddf00c8d00 2024-04-25T09:02:50.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.653+0000 7fdde6ffd640 1 --2- 172.21.15.43:0/4001939992 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fddd006b500 0x7fddd006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:50.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.653+0000 7fdde6ffd640 1 -- 172.21.15.43:0/4001939992 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdde00894b0 con 0x7fddf00c8d00 2024-04-25T09:02:50.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.653+0000 7fdde7fff640 1 --2- 172.21.15.43:0/4001939992 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fddd006b500 0x7fddd006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:50.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.654+0000 7fdde7fff640 1 --2- 172.21.15.43:0/4001939992 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fddd006b500 0x7fddd006d9c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fddd8009560 tx=0x7fddd8009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:50.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.658+0000 7fdde6ffd640 1 -- 172.21.15.43:0/4001939992 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdde0057cc0 con 0x7fddf00c8d00 2024-04-25T09:02:50.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.910+0000 7fddf489e640 1 -- 172.21.15.43:0/4001939992 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fddf0074ef0 con 0x7fddf00c8d00 2024-04-25T09:02:50.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.910+0000 7fdde6ffd640 1 -- 172.21.15.43:0/4001939992 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7fdde005bd60 con 0x7fddf00c8d00 2024-04-25T09:02:50.899 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:50.899 INFO:teuthology.orchestra.run.smithi043.stdout:{"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":543,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":113,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035238,"num_in_osds":8,"osd_in_since":1714035229,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1077670390,"bytes_used":2669371392,"bytes_avail":770424741888,"bytes_total":773094113280,"write_bytes_sec":89750399,"read_op_per_sec":0,"write_op_per_sec":89},"fsmap":{"epoch":69,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8676}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":21,"modified":"2024-04-25T09:02:49.814157+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:02:50.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.912+0000 7fddf489e640 1 -- 172.21.15.43:0/4001939992 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fddd006b500 msgr2=0x7fddd006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:50.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.912+0000 7fddf489e640 1 --2- 172.21.15.43:0/4001939992 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fddd006b500 0x7fddd006d9c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fddd8009560 tx=0x7fddd8009190 comp rx=0 tx=0).stop 2024-04-25T09:02:50.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.912+0000 7fddf489e640 1 -- 172.21.15.43:0/4001939992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddf00c8d00 msgr2=0x7fddf00c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:50.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.912+0000 7fddf489e640 1 --2- 172.21.15.43:0/4001939992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddf00c8d00 0x7fddf00c66f0 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fdde000c460 tx=0x7fdde000c920 comp rx=0 tx=0).stop 2024-04-25T09:02:50.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.912+0000 7fddf489e640 1 -- 172.21.15.43:0/4001939992 shutdown_connections 2024-04-25T09:02:50.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.912+0000 7fddf489e640 1 --2- 172.21.15.43:0/4001939992 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fddd006b500 0x7fddd006d9c0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:50.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.912+0000 7fddf489e640 1 --2- 172.21.15.43:0/4001939992 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fddf015df50 0x7fddf00cc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:50.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.912+0000 7fddf489e640 1 --2- 172.21.15.43:0/4001939992 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fddf00ce660 0x7fddf0160da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:50.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.912+0000 7fddf489e640 1 --2- 172.21.15.43:0/4001939992 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fddf00c8d00 0x7fddf00c66f0 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:50.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.912+0000 7fddf489e640 1 -- 172.21.15.43:0/4001939992 >> 172.21.15.43:0/4001939992 conn(0x7fddf00d3a30 msgr2=0x7fddf006b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:50.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.912+0000 7fddf489e640 1 -- 172.21.15.43:0/4001939992 shutdown_connections 2024-04-25T09:02:50.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:50.912+0000 7fddf489e640 1 -- 172.21.15.43:0/4001939992 wait complete. 2024-04-25T09:02:50.911 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused'} 2024-04-25T09:02:50.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:50.911 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub abort 2024-04-25T09:02:51.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.025+0000 7f7de3cc6640 1 Processor -- start 2024-04-25T09:02:51.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.025+0000 7f7de3cc6640 1 -- start start 2024-04-25T09:02:51.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.026+0000 7f7de3cc6640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7ddc0d1ec0 0x7f7ddc0cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.026+0000 7f7de3cc6640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7ddc0b0cc0 0x7f7ddc0b1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.026+0000 7f7de3cc6640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ddc15f170 con 0x7f7ddc15dd80 2024-04-25T09:02:51.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.026+0000 7f7de3cc6640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ddc0cfc60 con 0x7f7ddc0d1ec0 2024-04-25T09:02:51.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.026+0000 7f7de3cc6640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc0b32c0 con 0x7f7ddc0b0cc0 2024-04-25T09:02:51.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.026+0000 7f7de1261640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7ddc0d1ec0 0x7f7ddc0cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.026+0000 7f7de1261640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7ddc0d1ec0 0x7f7ddc0cf720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48184/0 (socket says 172.21.15.43:48184) 2024-04-25T09:02:51.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.026+0000 7f7de1261640 1 -- 172.21.15.43:0/143186890 learned_addr learned my addr 172.21.15.43:0/143186890 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:51.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.027+0000 7f7de2263640 1 --2- 172.21.15.43:0/143186890 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7ddc0b0cc0 0x7f7ddc0b1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.027+0000 7f7de1261640 1 -- 172.21.15.43:0/143186890 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7ddc0b0cc0 msgr2=0x7f7ddc0b1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:51.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.027+0000 7f7de1261640 1 --2- 172.21.15.43:0/143186890 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7ddc0b0cc0 0x7f7ddc0b1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.027+0000 7f7de1261640 1 -- 172.21.15.43:0/143186890 >> v1:172.21.15.193:6789/0 conn(0x7f7ddc15dd80 legacy=0x7f7ddc0d17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:51.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.027+0000 7f7de1261640 1 -- 172.21.15.43:0/143186890 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ddc0b2fe0 con 0x7f7ddc0d1ec0 2024-04-25T09:02:51.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.027+0000 7f7de2263640 1 --2- 172.21.15.43:0/143186890 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7ddc0b0cc0 0x7f7ddc0b1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:51.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.027+0000 7f7de1261640 1 --2- 172.21.15.43:0/143186890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ddc0d1ec0 0x7f7ddc0cf720 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0009870 tx=0x7f7dd002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db25251edabf093d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:51.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.028+0000 7f7de0a60640 1 -- 172.21.15.43:0/143186890 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd0037070 con 0x7f7ddc0d1ec0 2024-04-25T09:02:51.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.028+0000 7f7de0a60640 1 -- 172.21.15.43:0/143186890 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd0032440 con 0x7f7ddc0d1ec0 2024-04-25T09:02:51.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.028+0000 7f7de0a60640 1 -- 172.21.15.43:0/143186890 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd0032810 con 0x7f7ddc0d1ec0 2024-04-25T09:02:51.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.028+0000 7f7de3cc6640 1 -- 172.21.15.43:0/143186890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ddc0d1ec0 msgr2=0x7f7ddc0cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:51.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.028+0000 7f7de3cc6640 1 --2- 172.21.15.43:0/143186890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ddc0d1ec0 0x7f7ddc0cf720 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0009870 tx=0x7f7dd002a1c0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.028+0000 7f7de3cc6640 1 -- 172.21.15.43:0/143186890 shutdown_connections 2024-04-25T09:02:51.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.028+0000 7f7de3cc6640 1 --2- 172.21.15.43:0/143186890 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7ddc0b0cc0 0x7f7ddc0b1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.028+0000 7f7de3cc6640 1 --2- 172.21.15.43:0/143186890 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ddc0d1ec0 0x7f7ddc0cf720 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.028+0000 7f7de3cc6640 1 -- 172.21.15.43:0/143186890 >> 172.21.15.43:0/143186890 conn(0x7f7ddc072aa0 msgr2=0x7f7ddc09aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:51.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.028+0000 7f7de3cc6640 1 -- 172.21.15.43:0/143186890 shutdown_connections 2024-04-25T09:02:51.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.028+0000 7f7de3cc6640 1 -- 172.21.15.43:0/143186890 wait complete. 2024-04-25T09:02:51.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.029+0000 7f7de3cc6640 1 Processor -- start 2024-04-25T09:02:51.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.029+0000 7f7de3cc6640 1 -- start start 2024-04-25T09:02:51.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.030+0000 7f7de3cc6640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7ddc0b0cc0 0x7f7ddc15ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.030+0000 7f7de3cc6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ddc0d1ec0 0x7f7ddc0b5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.030+0000 7f7de3cc6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ddc15dd80 0x7f7ddc1603e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.030+0000 7f7de3cc6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc0cf330 con 0x7f7ddc15dd80 2024-04-25T09:02:51.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.030+0000 7f7de3cc6640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc0cf050 con 0x7f7ddc0d1ec0 2024-04-25T09:02:51.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.030+0000 7f7de3cc6640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc0b2c40 con 0x7f7ddc0b0cc0 2024-04-25T09:02:51.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.030+0000 7f7de2263640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ddc15dd80 0x7f7ddc1603e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.030+0000 7f7de2263640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ddc15dd80 0x7f7ddc1603e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48188/0 (socket says 172.21.15.43:48188) 2024-04-25T09:02:51.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.030+0000 7f7de2263640 1 -- 172.21.15.43:0/707994421 learned_addr learned my addr 172.21.15.43:0/707994421 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:51.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.030+0000 7f7de1261640 1 --2- 172.21.15.43:0/707994421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ddc0d1ec0 0x7f7ddc0b5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.030+0000 7f7de1a62640 1 --2- 172.21.15.43:0/707994421 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7ddc0b0cc0 0x7f7ddc15ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.031+0000 7f7de2263640 1 -- 172.21.15.43:0/707994421 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7ddc0b0cc0 msgr2=0x7f7ddc15ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:51.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.031+0000 7f7de2263640 1 --2- 172.21.15.43:0/707994421 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7ddc0b0cc0 0x7f7ddc15ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.031+0000 7f7de2263640 1 -- 172.21.15.43:0/707994421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ddc0d1ec0 msgr2=0x7f7ddc0b5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:51.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.031+0000 7f7de2263640 1 --2- 172.21.15.43:0/707994421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ddc0d1ec0 0x7f7ddc0b5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.031+0000 7f7de2263640 1 -- 172.21.15.43:0/707994421 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ddc0b6960 con 0x7f7ddc15dd80 2024-04-25T09:02:51.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.031+0000 7f7de1261640 1 --2- 172.21.15.43:0/707994421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ddc0d1ec0 0x7f7ddc0b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:51.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.031+0000 7f7de2263640 1 --2- 172.21.15.43:0/707994421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ddc15dd80 0x7f7ddc1603e0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f7dd400d800 tx=0x7f7dd400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:51.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.031+0000 7f7dda7fc640 1 -- 172.21.15.43:0/707994421 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd4004030 con 0x7f7ddc15dd80 2024-04-25T09:02:51.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.031+0000 7f7dda7fc640 1 -- 172.21.15.43:0/707994421 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd40041d0 con 0x7f7ddc15dd80 2024-04-25T09:02:51.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.031+0000 7f7dda7fc640 1 -- 172.21.15.43:0/707994421 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd40096a0 con 0x7f7ddc15dd80 2024-04-25T09:02:51.022 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.031+0000 7f7de3cc6640 1 -- 172.21.15.43:0/707994421 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ddc15af50 con 0x7f7ddc15dd80 2024-04-25T09:02:51.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.031+0000 7f7de3cc6640 1 -- 172.21.15.43:0/707994421 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ddc15b510 con 0x7f7ddc15dd80 2024-04-25T09:02:51.023 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.032+0000 7f7dda7fc640 1 -- 172.21.15.43:0/707994421 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7dd4009840 con 0x7f7ddc15dd80 2024-04-25T09:02:51.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.033+0000 7f7dda7fc640 1 --2- 172.21.15.43:0/707994421 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7dc806b4d0 0x7f7dc806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.033+0000 7f7dda7fc640 1 -- 172.21.15.43:0/707994421 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7dd408a650 con 0x7f7ddc15dd80 2024-04-25T09:02:51.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.033+0000 7f7de1a62640 1 --2- 172.21.15.43:0/707994421 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7dc806b4d0 0x7f7dc806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.034+0000 7f7de1a62640 1 --2- 172.21.15.43:0/707994421 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7dc806b4d0 0x7f7dc806d990 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc1f6e80 tx=0x7f7dcc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:51.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.034+0000 7f7de69c5740 1 Processor -- start 2024-04-25T09:02:51.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.034+0000 7f7de69c5740 1 -- start start 2024-04-25T09:02:51.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.035+0000 7f7de69c5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eff4336ca0 0x55eff4338e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.035+0000 7f7de69c5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eff4373370 0x55eff4337080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.035+0000 7f7de69c5740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55eff4361760 0x55eff4361be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.035+0000 7f7de69c5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eff4299b90 con 0x55eff4373370 2024-04-25T09:02:51.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.035+0000 7f7de69c5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eff4242e40 con 0x55eff4336ca0 2024-04-25T09:02:51.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.035+0000 7f7de69c5740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eff4295c30 con 0x55eff4361760 2024-04-25T09:02:51.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.035+0000 7f7de1261640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eff4373370 0x55eff4337080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.035+0000 7f7de1261640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eff4373370 0x55eff4337080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48200/0 (socket says 172.21.15.43:48200) 2024-04-25T09:02:51.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.035+0000 7f7de1261640 1 -- 172.21.15.43:0/1721811285 learned_addr learned my addr 172.21.15.43:0/1721811285 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:51.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.035+0000 7f7de1a62640 1 --2- 172.21.15.43:0/1721811285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eff4336ca0 0x55eff4338e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.035+0000 7f7de2263640 1 --2- 172.21.15.43:0/1721811285 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55eff4361760 0x55eff4361be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.035+0000 7f7de1261640 1 -- 172.21.15.43:0/1721811285 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55eff4361760 msgr2=0x55eff4361be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:51.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.035+0000 7f7de1261640 1 --2- 172.21.15.43:0/1721811285 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55eff4361760 0x55eff4361be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.035+0000 7f7de1261640 1 -- 172.21.15.43:0/1721811285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eff4336ca0 msgr2=0x55eff4338e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:51.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.035+0000 7f7de1261640 1 --2- 172.21.15.43:0/1721811285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eff4336ca0 0x55eff4338e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.035+0000 7f7de1261640 1 -- 172.21.15.43:0/1721811285 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eff40787d0 con 0x55eff4373370 2024-04-25T09:02:51.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.036+0000 7f7de1261640 1 --2- 172.21.15.43:0/1721811285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eff4373370 0x55eff4337080 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0009990 tx=0x7f7dd002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3223438289e74b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:51.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.036+0000 7f7dc37fe640 1 -- 172.21.15.43:0/1721811285 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd0037040 con 0x55eff4373370 2024-04-25T09:02:51.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.036+0000 7f7dc37fe640 1 -- 172.21.15.43:0/1721811285 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd0032880 con 0x55eff4373370 2024-04-25T09:02:51.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.036+0000 7f7dc37fe640 1 -- 172.21.15.43:0/1721811285 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd0032c30 con 0x55eff4373370 2024-04-25T09:02:51.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.036+0000 7f7de69c5740 1 -- 172.21.15.43:0/1721811285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eff4373370 msgr2=0x55eff4337080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:51.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.036+0000 7f7de69c5740 1 --2- 172.21.15.43:0/1721811285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eff4373370 0x55eff4337080 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0009990 tx=0x7f7dd002c000 comp rx=0 tx=0).stop 2024-04-25T09:02:51.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.037+0000 7f7de69c5740 1 -- 172.21.15.43:0/1721811285 shutdown_connections 2024-04-25T09:02:51.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.037+0000 7f7de69c5740 1 --2- 172.21.15.43:0/1721811285 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55eff4361760 0x55eff4361be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.037+0000 7f7de69c5740 1 --2- 172.21.15.43:0/1721811285 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eff4373370 0x55eff4337080 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.037+0000 7f7de69c5740 1 --2- 172.21.15.43:0/1721811285 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eff4336ca0 0x55eff4338e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.037+0000 7f7de69c5740 1 -- 172.21.15.43:0/1721811285 >> 172.21.15.43:0/1721811285 conn(0x55eff42fe870 msgr2=0x55eff437c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:51.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.037+0000 7f7de69c5740 1 -- 172.21.15.43:0/1721811285 shutdown_connections 2024-04-25T09:02:51.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.037+0000 7f7de69c5740 1 -- 172.21.15.43:0/1721811285 wait complete. 2024-04-25T09:02:51.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.037+0000 7f7de69c5740 1 -- start start 2024-04-25T09:02:51.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.037+0000 7f7de69c5740 1 Processor -- start 2024-04-25T09:02:51.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.038+0000 7f7de69c5740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55eff4336ca0 0x55eff435bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.038+0000 7f7de69c5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eff4361760 0x55eff438c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.038+0000 7f7de69c5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eff4373370 0x55eff4376460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.038+0000 7f7de69c5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eff42563e0 con 0x55eff4373370 2024-04-25T09:02:51.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.038+0000 7f7de69c5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eff435f2b0 con 0x55eff4361760 2024-04-25T09:02:51.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.038+0000 7f7de69c5740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eff438cbd0 con 0x55eff4336ca0 2024-04-25T09:02:51.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.038+0000 7f7de69c5740 10 client.0 _ll_register_callbacks cb 0x55eff4255db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:02:51.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.038+0000 7f7de2263640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eff4373370 0x55eff4376460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.038+0000 7f7de2263640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eff4373370 0x55eff4376460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48212/0 (socket says 172.21.15.43:48212) 2024-04-25T09:02:51.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.038+0000 7f7de2263640 1 -- 172.21.15.43:0/537426117 learned_addr learned my addr 172.21.15.43:0/537426117 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:51.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.038+0000 7f7de1261640 1 --2- 172.21.15.43:0/537426117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eff4361760 0x55eff438c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.038+0000 7f7de1a62640 1 --2- 172.21.15.43:0/537426117 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55eff4336ca0 0x55eff435bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.039+0000 7f7de2263640 1 -- 172.21.15.43:0/537426117 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55eff4336ca0 msgr2=0x55eff435bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:51.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.039+0000 7f7de2263640 1 --2- 172.21.15.43:0/537426117 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55eff4336ca0 0x55eff435bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.039+0000 7f7de2263640 1 -- 172.21.15.43:0/537426117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eff4361760 msgr2=0x55eff438c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:51.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.039+0000 7f7de2263640 1 --2- 172.21.15.43:0/537426117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eff4361760 0x55eff438c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.039+0000 7f7de2263640 1 -- 172.21.15.43:0/537426117 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eff3f6e0f0 con 0x55eff4373370 2024-04-25T09:02:51.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.039+0000 7f7de2263640 1 --2- 172.21.15.43:0/537426117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eff4373370 0x55eff4376460 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4009490 tx=0x7f7dd4059590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d1c1342681c6c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:51.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.039+0000 7f7dc0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:02:51.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.039+0000 7f7dc0ff9640 1 -- 172.21.15.43:0/537426117 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd4061440 con 0x55eff4373370 2024-04-25T09:02:51.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.039+0000 7f7dc0ff9640 1 -- 172.21.15.43:0/537426117 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7dd40553a0 con 0x55eff4373370 2024-04-25T09:02:51.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.039+0000 7f7de69c5740 1 -- 172.21.15.43:0/537426117 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7dd0032880 con 0x55eff4373370 2024-04-25T09:02:51.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.039+0000 7f7dc0ff9640 1 -- 172.21.15.43:0/537426117 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7dd4077050 con 0x55eff4373370 2024-04-25T09:02:51.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.039+0000 7f7de69c5740 10 client.8940 fetch_fsmap learned FSMap version 69 2024-04-25T09:02:51.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.039+0000 7f7de69c5740 1 -- 172.21.15.43:0/537426117 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x55eff40d8f60 con 0x55eff4373370 2024-04-25T09:02:51.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.039+0000 7f7dc0ff9640 1 -- 172.21.15.43:0/537426117 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd0032880 con 0x55eff4373370 2024-04-25T09:02:51.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.039+0000 7f7dc0ff9640 1 -- 172.21.15.43:0/537426117 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7dd4013070 con 0x55eff4373370 2024-04-25T09:02:51.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.039+0000 7f7de69c5740 10 client.8940 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T09:02:51.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.039+0000 7f7de69c5740 10 client.8940 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T09:02:51.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.039+0000 7f7de69c5740 20 client.8940 populate_metadata read hostname 'smithi043' 2024-04-25T09:02:51.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.039+0000 7f7de69c5740 1 --2- 172.21.15.43:0/537426117 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x55eff4364730 0x55eff437e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.039+0000 7f7de69c5740 4 client.8940 mds_command: new command op to 8676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:02:51.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.039+0000 7f7de69c5740 1 -- 172.21.15.43:0/537426117 --> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eff42a70d0 con 0x55eff4364730 2024-04-25T09:02:51.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.040+0000 7f7de1a62640 1 --2- 172.21.15.43:0/537426117 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x55eff4364730 0x55eff437e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.040+0000 7f7dc0ff9640 10 client.8940 ms_handle_connect on v2:172.21.15.43:6833/1586865031 2024-04-25T09:02:51.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.040+0000 7f7de1a62640 1 --2- 172.21.15.43:0/537426117 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x55eff4364730 0x55eff437e390 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ea07917585b5e8b server_cookie=aa7bb2ab94322423 in_seq=0 out_seq=0 2024-04-25T09:02:51.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.040+0000 7f7dc0ff9640 1 -- 172.21.15.43:0/537426117 <== mds.0 v2:172.21.15.43:6833/1586865031 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55eff42a70d0 con 0x55eff4364730 2024-04-25T09:02:51.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.040+0000 7f7dc0ff9640 10 client.8940 handle_command_reply: tid=0 2024-04-25T09:02:51.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.040+0000 7f7de69c5740 1 client.8940 shutdown 2024-04-25T09:02:51.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.041+0000 7f7de69c5740 1 -- 172.21.15.43:0/537426117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eff4373370 msgr2=0x55eff4376460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:51.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.041+0000 7f7de69c5740 1 --2- 172.21.15.43:0/537426117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eff4373370 0x55eff4376460 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f7dd4009490 tx=0x7f7dd4059590 comp rx=0 tx=0).stop 2024-04-25T09:02:51.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.041+0000 7f7de69c5740 1 -- 172.21.15.43:0/537426117 shutdown_connections 2024-04-25T09:02:51.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.041+0000 7f7de69c5740 1 --2- 172.21.15.43:0/537426117 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x55eff4364730 0x55eff437e390 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.041+0000 7f7de69c5740 1 --2- 172.21.15.43:0/537426117 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eff4373370 0x55eff4376460 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.041+0000 7f7de69c5740 1 --2- 172.21.15.43:0/537426117 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eff4361760 0x55eff438c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.041+0000 7f7dc0ff9640 0 client.8940 ms_handle_reset on v2:172.21.15.43:6833/1586865031 2024-04-25T09:02:51.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.041+0000 7f7de69c5740 1 --2- 172.21.15.43:0/537426117 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55eff4336ca0 0x55eff435bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.041+0000 7f7de69c5740 1 -- 172.21.15.43:0/537426117 >> 172.21.15.43:0/537426117 conn(0x55eff42fe870 msgr2=0x55eff435c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:51.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.041+0000 7f7de69c5740 1 -- 172.21.15.43:0/537426117 shutdown_connections 2024-04-25T09:02:51.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.041+0000 7f7de69c5740 1 -- 172.21.15.43:0/537426117 wait complete. 2024-04-25T09:02:51.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.041+0000 7f7de69c5740 20 client.8940 trim_cache size 0 max 16384 2024-04-25T09:02:51.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.056+0000 7f7de69c5740 1 Processor -- start 2024-04-25T09:02:51.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.056+0000 7f7de69c5740 1 -- start start 2024-04-25T09:02:51.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.056+0000 7f7de69c5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eff4336ca0 0x55eff437fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.056+0000 7f7de69c5740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55eff4361760 0x55eff43739c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.056+0000 7f7de69c5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eff4373370 0x55eff4373f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.056+0000 7f7de69c5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc1ffe90 con 0x55eff4336ca0 2024-04-25T09:02:51.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.056+0000 7f7de69c5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc1ff090 con 0x55eff4373370 2024-04-25T09:02:51.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.056+0000 7f7de69c5740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc1fe490 con 0x55eff4361760 2024-04-25T09:02:51.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.056+0000 7f7de1a62640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eff4336ca0 0x55eff437fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.056+0000 7f7de1a62640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eff4336ca0 0x55eff437fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48220/0 (socket says 172.21.15.43:48220) 2024-04-25T09:02:51.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.057+0000 7f7de1a62640 1 -- 172.21.15.43:0/2436260072 learned_addr learned my addr 172.21.15.43:0/2436260072 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:51.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.057+0000 7f7de2263640 1 --2- 172.21.15.43:0/2436260072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eff4373370 0x55eff4373f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.057+0000 7f7de1261640 1 --2- 172.21.15.43:0/2436260072 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55eff4361760 0x55eff43739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.057+0000 7f7de1a62640 1 -- 172.21.15.43:0/2436260072 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55eff4361760 msgr2=0x55eff43739c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:51.047 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.057+0000 7f7de1a62640 1 --2- 172.21.15.43:0/2436260072 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55eff4361760 0x55eff43739c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.057+0000 7f7de1a62640 1 -- 172.21.15.43:0/2436260072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eff4373370 msgr2=0x55eff4373f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:51.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.057+0000 7f7de1a62640 1 --2- 172.21.15.43:0/2436260072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eff4373370 0x55eff4373f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.057+0000 7f7de1a62640 1 -- 172.21.15.43:0/2436260072 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eff435bf40 con 0x55eff4336ca0 2024-04-25T09:02:51.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.057+0000 7f7de1a62640 1 --2- 172.21.15.43:0/2436260072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eff4336ca0 0x55eff437fcb0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x55eff435c550 tx=0x7f7dcc011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=207cc3803c1ba79f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:51.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.057+0000 7f7dc2ffd640 1 -- 172.21.15.43:0/2436260072 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc02c070 con 0x55eff4336ca0 2024-04-25T09:02:51.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.057+0000 7f7dc2ffd640 1 -- 172.21.15.43:0/2436260072 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dcc017030 con 0x55eff4336ca0 2024-04-25T09:02:51.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.057+0000 7f7dc2ffd640 1 -- 172.21.15.43:0/2436260072 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc027e60 con 0x55eff4336ca0 2024-04-25T09:02:51.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.058+0000 7f7de69c5740 1 -- 172.21.15.43:0/2436260072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eff4336ca0 msgr2=0x55eff437fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:51.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.058+0000 7f7de69c5740 1 --2- 172.21.15.43:0/2436260072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eff4336ca0 0x55eff437fcb0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x55eff435c550 tx=0x7f7dcc011000 comp rx=0 tx=0).stop 2024-04-25T09:02:51.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.058+0000 7f7de69c5740 1 -- 172.21.15.43:0/2436260072 shutdown_connections 2024-04-25T09:02:51.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.058+0000 7f7de69c5740 1 --2- 172.21.15.43:0/2436260072 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eff4373370 0x55eff4373f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.058+0000 7f7de69c5740 1 --2- 172.21.15.43:0/2436260072 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55eff4361760 0x55eff43739c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.058+0000 7f7de69c5740 1 --2- 172.21.15.43:0/2436260072 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eff4336ca0 0x55eff437fcb0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.058+0000 7f7de69c5740 1 -- 172.21.15.43:0/2436260072 >> 172.21.15.43:0/2436260072 conn(0x55eff42fe870 msgr2=0x55eff432cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:51.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.058+0000 7f7de69c5740 1 -- 172.21.15.43:0/2436260072 shutdown_connections 2024-04-25T09:02:51.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.058+0000 7f7de69c5740 1 -- 172.21.15.43:0/2436260072 wait complete. 2024-04-25T09:02:51.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.058+0000 7f7de69c5740 1 -- start start 2024-04-25T09:02:51.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.058+0000 7f7de69c5740 1 Processor -- start 2024-04-25T09:02:51.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.059+0000 7f7de69c5740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eff4336ca0 0x55eff43956d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.059+0000 7f7de69c5740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eff4361760 0x55eff4395c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.059+0000 7f7de69c5740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55eff4373370 0x55eff438ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.059+0000 7f7de69c5740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ddc1fea90 con 0x55eff4361760 2024-04-25T09:02:51.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.059+0000 7f7de69c5740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eff4365470 con 0x55eff4336ca0 2024-04-25T09:02:51.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.059+0000 7f7de69c5740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eff43e7820 con 0x55eff4373370 2024-04-25T09:02:51.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.059+0000 7f7de69c5740 10 client.0 _ll_register_callbacks cb 0x7f7dd40046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:02:51.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.059+0000 7f7de1261640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eff4361760 0x55eff4395c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.059+0000 7f7de1261640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eff4361760 0x55eff4395c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48236/0 (socket says 172.21.15.43:48236) 2024-04-25T09:02:51.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.059+0000 7f7de1261640 1 -- 172.21.15.43:0/2310650189 learned_addr learned my addr 172.21.15.43:0/2310650189 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:51.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.059+0000 7f7de1a62640 1 --2- 172.21.15.43:0/2310650189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eff4336ca0 0x55eff43956d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.059+0000 7f7de2263640 1 --2- 172.21.15.43:0/2310650189 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55eff4373370 0x55eff438ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.060+0000 7f7de1261640 1 -- 172.21.15.43:0/2310650189 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55eff4373370 msgr2=0x55eff438ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:51.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.060+0000 7f7de1261640 1 --2- 172.21.15.43:0/2310650189 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55eff4373370 0x55eff438ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.060+0000 7f7de1261640 1 -- 172.21.15.43:0/2310650189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eff4336ca0 msgr2=0x55eff43956d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:51.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.060+0000 7f7de1261640 1 --2- 172.21.15.43:0/2310650189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eff4336ca0 0x55eff43956d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.060+0000 7f7de1261640 1 -- 172.21.15.43:0/2310650189 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eff3f471b0 con 0x55eff4361760 2024-04-25T09:02:51.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.060+0000 7f7de1261640 1 --2- 172.21.15.43:0/2310650189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eff4361760 0x55eff4395c10 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0009870 tx=0x7f7dd0008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a54f99f304cc9000 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:51.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.060+0000 7f7dc0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:02:51.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.060+0000 7f7dc0ff9640 1 -- 172.21.15.43:0/2310650189 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd0037040 con 0x55eff4361760 2024-04-25T09:02:51.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.060+0000 7f7dc0ff9640 1 -- 172.21.15.43:0/2310650189 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7dd0032a90 con 0x55eff4361760 2024-04-25T09:02:51.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.060+0000 7f7de69c5740 1 -- 172.21.15.43:0/2310650189 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7dcc017030 con 0x55eff4361760 2024-04-25T09:02:51.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.060+0000 7f7dc0ff9640 1 -- 172.21.15.43:0/2310650189 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7dd0058050 con 0x55eff4361760 2024-04-25T09:02:51.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.060+0000 7f7de69c5740 10 client.8946 fetch_fsmap learned FSMap version 69 2024-04-25T09:02:51.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.060+0000 7f7de69c5740 1 -- 172.21.15.43:0/2310650189 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x55eff43e7560 con 0x55eff4361760 2024-04-25T09:02:51.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.060+0000 7f7dc0ff9640 1 -- 172.21.15.43:0/2310650189 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dcc017030 con 0x55eff4361760 2024-04-25T09:02:51.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.060+0000 7f7dc0ff9640 1 -- 172.21.15.43:0/2310650189 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7dd005b020 con 0x55eff4361760 2024-04-25T09:02:51.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.060+0000 7f7de69c5740 10 client.8946 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T09:02:51.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.060+0000 7f7de69c5740 10 client.8946 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T09:02:51.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.060+0000 7f7de69c5740 20 client.8946 populate_metadata read hostname 'smithi043' 2024-04-25T09:02:51.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.060+0000 7f7de69c5740 1 --2- 172.21.15.43:0/2310650189 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x55eff4335460 0x55eff438b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.060+0000 7f7de69c5740 4 client.8946 mds_command: new command op to 8676 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-04-25T09:02:51.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.060+0000 7f7de69c5740 1 -- 172.21.15.43:0/2310650189 --> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] -- command(tid 0: {"prefix": "scrub abort"}) v1 -- 0x55eff42a1420 con 0x55eff4335460 2024-04-25T09:02:51.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.061+0000 7f7de1a62640 1 --2- 172.21.15.43:0/2310650189 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x55eff4335460 0x55eff438b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.061+0000 7f7dc0ff9640 10 client.8946 ms_handle_connect on v2:172.21.15.43:6833/1586865031 2024-04-25T09:02:51.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.061+0000 7f7de1a62640 1 --2- 172.21.15.43:0/2310650189 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x55eff4335460 0x55eff438b3a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8179ee60d270ea1 server_cookie=eb2e49d1ab2b4e6e in_seq=0 out_seq=0 2024-04-25T09:02:51.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.061+0000 7f7dc0ff9640 1 -- 172.21.15.43:0/2310650189 <== mds.0 v2:172.21.15.43:6833/1586865031 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55eff42a1420 con 0x55eff4335460 2024-04-25T09:02:51.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.061+0000 7f7dc0ff9640 10 client.8946 handle_command_reply: tid=0 2024-04-25T09:02:51.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.061+0000 7f7de69c5740 1 client.8946 shutdown 2024-04-25T09:02:51.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.061+0000 7f7de69c5740 1 -- 172.21.15.43:0/2310650189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eff4361760 msgr2=0x55eff4395c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:51.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.061+0000 7f7de69c5740 1 --2- 172.21.15.43:0/2310650189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eff4361760 0x55eff4395c10 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0009870 tx=0x7f7dd0008030 comp rx=0 tx=0).stop 2024-04-25T09:02:51.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.061+0000 7f7de69c5740 1 -- 172.21.15.43:0/2310650189 shutdown_connections 2024-04-25T09:02:51.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.061+0000 7f7de69c5740 1 --2- 172.21.15.43:0/2310650189 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x55eff4335460 0x55eff438b3a0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.061+0000 7f7de69c5740 1 --2- 172.21.15.43:0/2310650189 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55eff4373370 0x55eff438ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.061+0000 7f7de69c5740 1 --2- 172.21.15.43:0/2310650189 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55eff4361760 0x55eff4395c10 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.061+0000 7f7de69c5740 1 --2- 172.21.15.43:0/2310650189 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55eff4336ca0 0x55eff43956d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.061+0000 7f7dc0ff9640 0 client.8946 ms_handle_reset on v2:172.21.15.43:6833/1586865031 2024-04-25T09:02:51.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.061+0000 7f7de69c5740 1 -- 172.21.15.43:0/2310650189 >> 172.21.15.43:0/2310650189 conn(0x55eff42fe870 msgr2=0x55eff437fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:51.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.061+0000 7f7de69c5740 1 -- 172.21.15.43:0/2310650189 shutdown_connections 2024-04-25T09:02:51.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.061+0000 7f7de69c5740 1 -- 172.21.15.43:0/2310650189 wait complete. 2024-04-25T09:02:51.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.061+0000 7f7de69c5740 20 client.8946 trim_cache size 0 max 16384 2024-04-25T09:02:51.056 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:02:51.056 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0 2024-04-25T09:02:51.056 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:02:51.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.062+0000 7f7de3cc6640 1 -- 172.21.15.43:0/707994421 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7dc806b4d0 msgr2=0x7f7dc806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:51.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.062+0000 7f7de3cc6640 1 --2- 172.21.15.43:0/707994421 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7dc806b4d0 0x7f7dc806d990 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f7ddc1f6e80 tx=0x7f7dcc009040 comp rx=0 tx=0).stop 2024-04-25T09:02:51.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.062+0000 7f7de3cc6640 1 -- 172.21.15.43:0/707994421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ddc15dd80 msgr2=0x7f7ddc1603e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:51.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.062+0000 7f7de3cc6640 1 --2- 172.21.15.43:0/707994421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ddc15dd80 0x7f7ddc1603e0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f7dd400d800 tx=0x7f7dd400dcc0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.062+0000 7f7de3cc6640 1 -- 172.21.15.43:0/707994421 shutdown_connections 2024-04-25T09:02:51.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.062+0000 7f7de3cc6640 1 --2- 172.21.15.43:0/707994421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7ddc15dd80 0x7f7ddc1603e0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.062+0000 7f7de3cc6640 1 --2- 172.21.15.43:0/707994421 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7dc806b4d0 0x7f7dc806d990 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.062+0000 7f7de3cc6640 1 --2- 172.21.15.43:0/707994421 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7ddc0d1ec0 0x7f7ddc0b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.062+0000 7f7de3cc6640 1 --2- 172.21.15.43:0/707994421 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7ddc0b0cc0 0x7f7ddc15ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.062+0000 7f7de3cc6640 1 -- 172.21.15.43:0/707994421 >> 172.21.15.43:0/707994421 conn(0x7f7ddc072aa0 msgr2=0x7f7ddc06e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:51.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.062+0000 7f7de3cc6640 1 -- 172.21.15.43:0/707994421 shutdown_connections 2024-04-25T09:02:51.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.062+0000 7f7de3cc6640 1 -- 172.21.15.43:0/707994421 wait complete. 2024-04-25T09:02:51.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:51.061 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub status 2024-04-25T09:02:51.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.175+0000 7f8ae6238640 1 Processor -- start 2024-04-25T09:02:51.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.175+0000 7f8ae6238640 1 -- start start 2024-04-25T09:02:51.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.176+0000 7f8ae6238640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f8ae00d04a0 0x7f8ae015ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.176+0000 7f8ae6238640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ae015dec0 0x7f8ae00d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.176+0000 7f8ae6238640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8ae00ce8c0 0x7f8ae01617a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.176+0000 7f8ae6238640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ae00d0dc0 con 0x7f8ae00ce8c0 2024-04-25T09:02:51.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.176+0000 7f8ae6238640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ae00cf110 con 0x7f8ae00d04a0 2024-04-25T09:02:51.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.176+0000 7f8ae6238640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae00cf3f0 con 0x7f8ae015dec0 2024-04-25T09:02:51.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.176+0000 7f8adffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8ae00ce8c0 0x7f8ae01617a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.176+0000 7f8adffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8ae00ce8c0 0x7f8ae01617a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48248/0 (socket says 172.21.15.43:48248) 2024-04-25T09:02:51.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.176+0000 7f8adffff640 1 -- 172.21.15.43:0/1290187144 learned_addr learned my addr 172.21.15.43:0/1290187144 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:51.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.176+0000 7f8adf7fe640 1 --2- 172.21.15.43:0/1290187144 >> v2:172.21.15.193:3300/0 conn(0x7f8ae00d04a0 0x7f8ae015ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.176+0000 7f8adeffd640 1 --2- 172.21.15.43:0/1290187144 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ae015dec0 0x7f8ae00d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.176+0000 7f8adffff640 1 -- 172.21.15.43:0/1290187144 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ae015dec0 msgr2=0x7f8ae00d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:51.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.176+0000 7f8adffff640 1 --2- 172.21.15.43:0/1290187144 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ae015dec0 0x7f8ae00d0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.176+0000 7f8adffff640 1 -- 172.21.15.43:0/1290187144 >> v2:172.21.15.193:3300/0 conn(0x7f8ae00d04a0 msgr2=0x7f8ae015ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:51.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.176+0000 7f8adffff640 1 --2- 172.21.15.43:0/1290187144 >> v2:172.21.15.193:3300/0 conn(0x7f8ae00d04a0 0x7f8ae015ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.176+0000 7f8adffff640 1 -- 172.21.15.43:0/1290187144 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae00cf6d0 con 0x7f8ae00ce8c0 2024-04-25T09:02:51.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.177+0000 7f8adffff640 1 --2- 172.21.15.43:0/1290187144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ae00ce8c0 0x7f8ae01617a0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f8ae00b6690 tx=0x7f8ad802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a222af20a0a0d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:51.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.177+0000 7f8ade7fc640 1 -- 172.21.15.43:0/1290187144 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8036070 con 0x7f8ae00ce8c0 2024-04-25T09:02:51.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.177+0000 7f8ade7fc640 1 -- 172.21.15.43:0/1290187144 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad8031600 con 0x7f8ae00ce8c0 2024-04-25T09:02:51.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.177+0000 7f8ade7fc640 1 -- 172.21.15.43:0/1290187144 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad80319d0 con 0x7f8ae00ce8c0 2024-04-25T09:02:51.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.178+0000 7f8ae6238640 1 -- 172.21.15.43:0/1290187144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ae00ce8c0 msgr2=0x7f8ae01617a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:51.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.178+0000 7f8ae6238640 1 --2- 172.21.15.43:0/1290187144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ae00ce8c0 0x7f8ae01617a0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f8ae00b6690 tx=0x7f8ad802a6a0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.178+0000 7f8ae6238640 1 -- 172.21.15.43:0/1290187144 shutdown_connections 2024-04-25T09:02:51.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.178+0000 7f8ae6238640 1 --2- 172.21.15.43:0/1290187144 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ae00ce8c0 0x7f8ae01617a0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.178+0000 7f8ae6238640 1 --2- 172.21.15.43:0/1290187144 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ae015dec0 0x7f8ae00d0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.178+0000 7f8ae6238640 1 --2- 172.21.15.43:0/1290187144 >> v2:172.21.15.193:3300/0 conn(0x7f8ae00d04a0 0x7f8ae015ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.178+0000 7f8ae6238640 1 -- 172.21.15.43:0/1290187144 >> 172.21.15.43:0/1290187144 conn(0x7f8ae009cd40 msgr2=0x7f8ae00d3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:51.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.178+0000 7f8ae6238640 1 -- 172.21.15.43:0/1290187144 shutdown_connections 2024-04-25T09:02:51.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.178+0000 7f8ae6238640 1 -- 172.21.15.43:0/1290187144 wait complete. 2024-04-25T09:02:51.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.179+0000 7f8ae6238640 1 Processor -- start 2024-04-25T09:02:51.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.179+0000 7f8ae6238640 1 -- start start 2024-04-25T09:02:51.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.180+0000 7f8ae6238640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ae00ce8c0 0x7f8ae00cd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.180+0000 7f8ae6238640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ae00d04a0 0x7f8ae014dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.180+0000 7f8ae6238640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ae015dec0 0x7f8ae00b5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.180+0000 7f8ae6238640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae00cecf0 con 0x7f8ae015dec0 2024-04-25T09:02:51.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.180+0000 7f8ae6238640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae00cfc90 con 0x7f8ae00d04a0 2024-04-25T09:02:51.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.180+0000 7f8ae6238640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae00cee70 con 0x7f8ae00ce8c0 2024-04-25T09:02:51.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.180+0000 7f8adffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ae015dec0 0x7f8ae00b5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.180+0000 7f8adffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ae015dec0 0x7f8ae00b5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48260/0 (socket says 172.21.15.43:48260) 2024-04-25T09:02:51.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.180+0000 7f8adffff640 1 -- 172.21.15.43:0/1918154093 learned_addr learned my addr 172.21.15.43:0/1918154093 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:51.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.180+0000 7f8adeffd640 1 --2- 172.21.15.43:0/1918154093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ae00d04a0 0x7f8ae014dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.180+0000 7f8adf7fe640 1 --2- 172.21.15.43:0/1918154093 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ae00ce8c0 0x7f8ae00cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.180+0000 7f8adffff640 1 -- 172.21.15.43:0/1918154093 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ae00ce8c0 msgr2=0x7f8ae00cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:51.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.180+0000 7f8adffff640 1 --2- 172.21.15.43:0/1918154093 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ae00ce8c0 0x7f8ae00cd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.180+0000 7f8adffff640 1 -- 172.21.15.43:0/1918154093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ae00d04a0 msgr2=0x7f8ae014dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:51.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.180+0000 7f8adffff640 1 --2- 172.21.15.43:0/1918154093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ae00d04a0 0x7f8ae014dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.180+0000 7f8adffff640 1 -- 172.21.15.43:0/1918154093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae00cbfd0 con 0x7f8ae015dec0 2024-04-25T09:02:51.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.180+0000 7f8adf7fe640 1 --2- 172.21.15.43:0/1918154093 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ae00ce8c0 0x7f8ae00cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:51.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.181+0000 7f8adffff640 1 --2- 172.21.15.43:0/1918154093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ae015dec0 0x7f8ae00b5cf0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8034040 tx=0x7f8ad8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:51.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.181+0000 7f8ac7fff640 1 -- 172.21.15.43:0/1918154093 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8036050 con 0x7f8ae015dec0 2024-04-25T09:02:51.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.181+0000 7f8ac7fff640 1 -- 172.21.15.43:0/1918154093 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad8042070 con 0x7f8ae015dec0 2024-04-25T09:02:51.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.181+0000 7f8ae6238640 1 -- 172.21.15.43:0/1918154093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ae00cc2b0 con 0x7f8ae015dec0 2024-04-25T09:02:51.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.181+0000 7f8ac7fff640 1 -- 172.21.15.43:0/1918154093 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8009510 con 0x7f8ae015dec0 2024-04-25T09:02:51.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.181+0000 7f8ae6238640 1 -- 172.21.15.43:0/1918154093 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae00cc870 con 0x7f8ae015dec0 2024-04-25T09:02:51.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.182+0000 7f8ac7fff640 1 -- 172.21.15.43:0/1918154093 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8ad8007bf0 con 0x7f8ae015dec0 2024-04-25T09:02:51.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.183+0000 7f8ac7fff640 1 --2- 172.21.15.43:0/1918154093 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8acc06b4d0 0x7f8acc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.183+0000 7f8ac7fff640 1 -- 172.21.15.43:0/1918154093 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8ad80aaee0 con 0x7f8ae015dec0 2024-04-25T09:02:51.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.183+0000 7f8adf7fe640 1 --2- 172.21.15.43:0/1918154093 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8acc06b4d0 0x7f8acc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.183+0000 7f8adf7fe640 1 --2- 172.21.15.43:0/1918154093 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8acc06b4d0 0x7f8acc06d990 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0009a70 tx=0x7f8ad0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:51.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.184+0000 7f8ae9014740 1 Processor -- start 2024-04-25T09:02:51.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.184+0000 7f8ae9014740 1 -- start start 2024-04-25T09:02:51.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.185+0000 7f8ae9014740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617f5abeb10 0x5617f5ac1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.185+0000 7f8ae9014740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5617f5abeef0 0x5617f5ac1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.185+0000 7f8ae9014740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5617f5a91770 0x5617f5a94f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.185+0000 7f8ae9014740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f59fada0 con 0x5617f5a91770 2024-04-25T09:02:51.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.185+0000 7f8ae9014740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f59a3b80 con 0x5617f5abeb10 2024-04-25T09:02:51.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.185+0000 7f8ae9014740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x5617f59f6e40 con 0x5617f5abeef0 2024-04-25T09:02:51.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.185+0000 7f8adffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5617f5a91770 0x5617f5a94f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.185+0000 7f8adffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5617f5a91770 0x5617f5a94f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48274/0 (socket says 172.21.15.43:48274) 2024-04-25T09:02:51.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.185+0000 7f8adffff640 1 -- 172.21.15.43:0/1392382558 learned_addr learned my addr 172.21.15.43:0/1392382558 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:51.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.185+0000 7f8adeffd640 1 --2- 172.21.15.43:0/1392382558 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5617f5abeef0 0x5617f5ac1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.185+0000 7f8adf7fe640 1 --2- 172.21.15.43:0/1392382558 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617f5abeb10 0x5617f5ac1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.185+0000 7f8adffff640 1 -- 172.21.15.43:0/1392382558 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5617f5abeef0 msgr2=0x5617f5ac1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:51.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.185+0000 7f8adffff640 1 --2- 172.21.15.43:0/1392382558 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5617f5abeef0 0x5617f5ac1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.185+0000 7f8adffff640 1 -- 172.21.15.43:0/1392382558 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617f5abeb10 msgr2=0x5617f5ac1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:51.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.185+0000 7f8adffff640 1 --2- 172.21.15.43:0/1392382558 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617f5abeb10 0x5617f5ac1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.185+0000 7f8adffff640 1 -- 172.21.15.43:0/1392382558 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617f59adf10 con 0x5617f5a91770 2024-04-25T09:02:51.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.185+0000 7f8adf7fe640 1 --2- 172.21.15.43:0/1392382558 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617f5abeb10 0x5617f5ac1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:51.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.185+0000 7f8adeffd640 1 --2- 172.21.15.43:0/1392382558 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5617f5abeef0 0x5617f5ac1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:51.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.185+0000 7f8adffff640 1 --2- 172.21.15.43:0/1392382558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5617f5a91770 0x5617f5a94f80 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8075bd0 tx=0x7f8ad8079600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=739258974ba4e116 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:51.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.185+0000 7f8ac57fa640 1 -- 172.21.15.43:0/1392382558 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8091020 con 0x5617f5a91770 2024-04-25T09:02:51.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.185+0000 7f8ac57fa640 1 -- 172.21.15.43:0/1392382558 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad8037040 con 0x5617f5a91770 2024-04-25T09:02:51.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.185+0000 7f8ac57fa640 1 -- 172.21.15.43:0/1392382558 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8079e70 con 0x5617f5a91770 2024-04-25T09:02:51.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.186+0000 7f8ae9014740 1 -- 172.21.15.43:0/1392382558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5617f5a91770 msgr2=0x5617f5a94f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:51.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.186+0000 7f8ae9014740 1 --2- 172.21.15.43:0/1392382558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5617f5a91770 0x5617f5a94f80 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8075bd0 tx=0x7f8ad8079600 comp rx=0 tx=0).stop 2024-04-25T09:02:51.182 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.186+0000 7f8ae9014740 1 -- 172.21.15.43:0/1392382558 shutdown_connections 2024-04-25T09:02:51.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.186+0000 7f8ae9014740 1 --2- 172.21.15.43:0/1392382558 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5617f5a91770 0x5617f5a94f80 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.186+0000 7f8ae9014740 1 --2- 172.21.15.43:0/1392382558 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5617f5abeef0 0x5617f5ac1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.186+0000 7f8ae9014740 1 --2- 172.21.15.43:0/1392382558 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617f5abeb10 0x5617f5ac1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.186+0000 7f8ae9014740 1 -- 172.21.15.43:0/1392382558 >> 172.21.15.43:0/1392382558 conn(0x5617f5a939e0 msgr2=0x5617f5adc320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:51.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.186+0000 7f8ae9014740 1 -- 172.21.15.43:0/1392382558 shutdown_connections 2024-04-25T09:02:51.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.186+0000 7f8ae9014740 1 -- 172.21.15.43:0/1392382558 wait complete. 2024-04-25T09:02:51.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.187+0000 7f8ae9014740 1 -- start start 2024-04-25T09:02:51.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.187+0000 7f8ae9014740 1 Processor -- start 2024-04-25T09:02:51.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.187+0000 7f8ae9014740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617f5abeb10 0x5617f5ac5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.187+0000 7f8ae9014740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5617f5abeef0 0x5617f5ab8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.187+0000 7f8ae9014740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5617f5ab46a0 0x5617f5ab8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.187+0000 7f8ae9014740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f59a3b80 con 0x5617f5ab46a0 2024-04-25T09:02:51.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.187+0000 7f8ae9014740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f59994b0 con 0x5617f5abeb10 2024-04-25T09:02:51.183 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.187+0000 7f8ae9014740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x5617f59fada0 con 0x5617f5abeef0 2024-04-25T09:02:51.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.188+0000 7f8ae9014740 10 client.0 _ll_register_callbacks cb 0x5617f59d9180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:02:51.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.188+0000 7f8adffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5617f5ab46a0 0x5617f5ab8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.188+0000 7f8adffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5617f5ab46a0 0x5617f5ab8670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48278/0 (socket says 172.21.15.43:48278) 2024-04-25T09:02:51.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.188+0000 7f8adffff640 1 -- 172.21.15.43:0/1766740404 learned_addr learned my addr 172.21.15.43:0/1766740404 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:51.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.188+0000 7f8adf7fe640 1 --2- 172.21.15.43:0/1766740404 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617f5abeb10 0x5617f5ac5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.188+0000 7f8adeffd640 1 --2- 172.21.15.43:0/1766740404 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5617f5abeef0 0x5617f5ab8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.188+0000 7f8adffff640 1 -- 172.21.15.43:0/1766740404 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5617f5abeef0 msgr2=0x5617f5ab8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:51.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.188+0000 7f8adffff640 1 --2- 172.21.15.43:0/1766740404 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5617f5abeef0 0x5617f5ab8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.188+0000 7f8adffff640 1 -- 172.21.15.43:0/1766740404 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617f5abeb10 msgr2=0x5617f5ac5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:51.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.188+0000 7f8adffff640 1 --2- 172.21.15.43:0/1766740404 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617f5abeb10 0x5617f5ac5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.188+0000 7f8adffff640 1 -- 172.21.15.43:0/1766740404 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5617f57d97d0 con 0x5617f5ab46a0 2024-04-25T09:02:51.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.188+0000 7f8adeffd640 1 --2- 172.21.15.43:0/1766740404 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5617f5abeef0 0x5617f5ab8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:51.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.188+0000 7f8adffff640 1 --2- 172.21.15.43:0/1766740404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5617f5ab46a0 0x5617f5ab8670 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8031780 tx=0x7f8ad8009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c8873291260ec03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:51.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.188+0000 7f8abe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:02:51.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.188+0000 7f8abe7fc640 1 -- 172.21.15.43:0/1766740404 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8091040 con 0x5617f5ab46a0 2024-04-25T09:02:51.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.188+0000 7f8abe7fc640 1 -- 172.21.15.43:0/1766740404 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8ad80a9050 con 0x5617f5ab46a0 2024-04-25T09:02:51.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.188+0000 7f8ae9014740 1 -- 172.21.15.43:0/1766740404 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ad8037040 con 0x5617f5ab46a0 2024-04-25T09:02:51.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.188+0000 7f8abe7fc640 1 -- 172.21.15.43:0/1766740404 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ad8079770 con 0x5617f5ab46a0 2024-04-25T09:02:51.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.188+0000 7f8ae9014740 10 client.8958 fetch_fsmap learned FSMap version 69 2024-04-25T09:02:51.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.188+0000 7f8ae9014740 1 -- 172.21.15.43:0/1766740404 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x5617f570f3f0 con 0x5617f5ab46a0 2024-04-25T09:02:51.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.188+0000 7f8abe7fc640 1 -- 172.21.15.43:0/1766740404 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad8037040 con 0x5617f5ab46a0 2024-04-25T09:02:51.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.189+0000 7f8abe7fc640 1 -- 172.21.15.43:0/1766740404 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8ad8078bc0 con 0x5617f5ab46a0 2024-04-25T09:02:51.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.189+0000 7f8ae9014740 10 client.8958 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T09:02:51.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.189+0000 7f8ae9014740 10 client.8958 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T09:02:51.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.189+0000 7f8ae9014740 20 client.8958 populate_metadata read hostname 'smithi043' 2024-04-25T09:02:51.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.189+0000 7f8ae9014740 1 --2- 172.21.15.43:0/1766740404 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x5617f5add920 0x5617f5addd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.189+0000 7f8ae9014740 4 client.8958 mds_command: new command op to 8676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:02:51.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.189+0000 7f8ae9014740 1 -- 172.21.15.43:0/1766740404 --> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5617f5a082e0 con 0x5617f5add920 2024-04-25T09:02:51.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.189+0000 7f8adf7fe640 1 --2- 172.21.15.43:0/1766740404 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x5617f5add920 0x5617f5addd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.189+0000 7f8adf7fe640 1 --2- 172.21.15.43:0/1766740404 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x5617f5add920 0x5617f5addd00 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcd1764384142d01 server_cookie=ce3e3c5d76e4af92 in_seq=0 out_seq=0 2024-04-25T09:02:51.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.189+0000 7f8abe7fc640 10 client.8958 ms_handle_connect on v2:172.21.15.43:6833/1586865031 2024-04-25T09:02:51.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.190+0000 7f8abe7fc640 1 -- 172.21.15.43:0/1766740404 <== mds.0 v2:172.21.15.43:6833/1586865031 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5617f5a082e0 con 0x5617f5add920 2024-04-25T09:02:51.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.190+0000 7f8abe7fc640 10 client.8958 handle_command_reply: tid=0 2024-04-25T09:02:51.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.190+0000 7f8ae9014740 1 client.8958 shutdown 2024-04-25T09:02:51.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.190+0000 7f8ae9014740 1 -- 172.21.15.43:0/1766740404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5617f5ab46a0 msgr2=0x5617f5ab8670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:51.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.190+0000 7f8ae9014740 1 --2- 172.21.15.43:0/1766740404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5617f5ab46a0 0x5617f5ab8670 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8031780 tx=0x7f8ad8009810 comp rx=0 tx=0).stop 2024-04-25T09:02:51.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.190+0000 7f8ae9014740 1 -- 172.21.15.43:0/1766740404 shutdown_connections 2024-04-25T09:02:51.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.190+0000 7f8ae9014740 1 --2- 172.21.15.43:0/1766740404 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x5617f5add920 0x5617f5addd00 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.190+0000 7f8ae9014740 1 --2- 172.21.15.43:0/1766740404 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5617f5ab46a0 0x5617f5ab8670 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.190+0000 7f8ae9014740 1 --2- 172.21.15.43:0/1766740404 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5617f5abeef0 0x5617f5ab8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.190+0000 7f8ae9014740 1 --2- 172.21.15.43:0/1766740404 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617f5abeb10 0x5617f5ac5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.190+0000 7f8abe7fc640 0 client.8958 ms_handle_reset on v2:172.21.15.43:6833/1586865031 2024-04-25T09:02:51.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.190+0000 7f8ae9014740 1 -- 172.21.15.43:0/1766740404 >> 172.21.15.43:0/1766740404 conn(0x5617f5a91770 msgr2=0x5617f5a96e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:51.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.190+0000 7f8ae9014740 1 -- 172.21.15.43:0/1766740404 shutdown_connections 2024-04-25T09:02:51.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.190+0000 7f8ae9014740 1 -- 172.21.15.43:0/1766740404 wait complete. 2024-04-25T09:02:51.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.190+0000 7f8ae9014740 20 client.8958 trim_cache size 0 max 16384 2024-04-25T09:02:51.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.205+0000 7f8ae9014740 1 Processor -- start 2024-04-25T09:02:51.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.205+0000 7f8ae9014740 1 -- start start 2024-04-25T09:02:51.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.206+0000 7f8ae9014740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617f5ab46a0 0x5617f5adc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.206+0000 7f8ae9014740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5617f5abeb10 0x5617f5abcff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.206+0000 7f8ae9014740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5617f5abeef0 0x5617f5abd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.206+0000 7f8ae9014740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae01ffea0 con 0x5617f5abeef0 2024-04-25T09:02:51.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.206+0000 7f8ae9014740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae01f7d90 con 0x5617f5ab46a0 2024-04-25T09:02:51.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.206+0000 7f8ae9014740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae00b7ec0 con 0x5617f5abeb10 2024-04-25T09:02:51.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.206+0000 7f8adffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5617f5abeef0 0x5617f5abd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.206+0000 7f8adffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5617f5abeef0 0x5617f5abd530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48290/0 (socket says 172.21.15.43:48290) 2024-04-25T09:02:51.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.206+0000 7f8adffff640 1 -- 172.21.15.43:0/4157243575 learned_addr learned my addr 172.21.15.43:0/4157243575 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:51.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.206+0000 7f8adf7fe640 1 --2- 172.21.15.43:0/4157243575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617f5ab46a0 0x5617f5adc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.206+0000 7f8adffff640 1 -- 172.21.15.43:0/4157243575 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5617f5abeb10 msgr2=0x5617f5abcff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:51.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.206+0000 7f8adeffd640 1 --2- 172.21.15.43:0/4157243575 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5617f5abeb10 0x5617f5abcff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.206+0000 7f8adffff640 1 --2- 172.21.15.43:0/4157243575 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5617f5abeb10 0x5617f5abcff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.206+0000 7f8adffff640 1 -- 172.21.15.43:0/4157243575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617f5ab46a0 msgr2=0x5617f5adc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:51.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.206+0000 7f8adffff640 1 --2- 172.21.15.43:0/4157243575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617f5ab46a0 0x5617f5adc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.206+0000 7f8adffff640 1 -- 172.21.15.43:0/4157243575 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617f56cf0f0 con 0x5617f5abeef0 2024-04-25T09:02:51.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.206+0000 7f8adf7fe640 1 --2- 172.21.15.43:0/4157243575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617f5ab46a0 0x5617f5adc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:51.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.206+0000 7f8adeffd640 1 --2- 172.21.15.43:0/4157243575 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5617f5abeb10 0x5617f5abcff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:51.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.206+0000 7f8adffff640 1 --2- 172.21.15.43:0/4157243575 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5617f5abeef0 0x5617f5abd530 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8009c90 tx=0x7f8ad8048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f4d2b9809b247a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:51.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.206+0000 7f8ac4ff9640 1 -- 172.21.15.43:0/4157243575 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8091040 con 0x5617f5abeef0 2024-04-25T09:02:51.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.206+0000 7f8ac4ff9640 1 -- 172.21.15.43:0/4157243575 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad8079e70 con 0x5617f5abeef0 2024-04-25T09:02:51.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.206+0000 7f8ac4ff9640 1 -- 172.21.15.43:0/4157243575 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8081e80 con 0x5617f5abeef0 2024-04-25T09:02:51.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.207+0000 7f8ae9014740 1 -- 172.21.15.43:0/4157243575 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5617f5abeef0 msgr2=0x5617f5abd530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:51.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.207+0000 7f8ae9014740 1 --2- 172.21.15.43:0/4157243575 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5617f5abeef0 0x5617f5abd530 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8009c90 tx=0x7f8ad8048000 comp rx=0 tx=0).stop 2024-04-25T09:02:51.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.207+0000 7f8ae9014740 1 -- 172.21.15.43:0/4157243575 shutdown_connections 2024-04-25T09:02:51.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.207+0000 7f8ae9014740 1 --2- 172.21.15.43:0/4157243575 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5617f5abeef0 0x5617f5abd530 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.207+0000 7f8ae9014740 1 --2- 172.21.15.43:0/4157243575 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5617f5abeb10 0x5617f5abcff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.207+0000 7f8ae9014740 1 --2- 172.21.15.43:0/4157243575 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617f5ab46a0 0x5617f5adc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.207+0000 7f8ae9014740 1 -- 172.21.15.43:0/4157243575 >> 172.21.15.43:0/4157243575 conn(0x5617f5a91770 msgr2=0x5617f5a96f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:51.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.207+0000 7f8ae9014740 1 -- 172.21.15.43:0/4157243575 shutdown_connections 2024-04-25T09:02:51.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.207+0000 7f8ae9014740 1 -- 172.21.15.43:0/4157243575 wait complete. 2024-04-25T09:02:51.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.208+0000 7f8ae9014740 1 -- start start 2024-04-25T09:02:51.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.208+0000 7f8ae9014740 1 Processor -- start 2024-04-25T09:02:51.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.208+0000 7f8ae9014740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5617f5ab46a0 0x5617f5b28a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.208+0000 7f8ae9014740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617f5abeb10 0x5617f5ad1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.208+0000 7f8ae9014740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5617f5abeef0 0x5617f5ad21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.208+0000 7f8ae9014740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617f5ac5600 con 0x5617f5abeef0 2024-04-25T09:02:51.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.208+0000 7f8ae9014740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae01f7790 con 0x5617f5abeb10 2024-04-25T09:02:51.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.208+0000 7f8ae9014740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x5617f5abc990 con 0x5617f5ab46a0 2024-04-25T09:02:51.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.208+0000 7f8ae9014740 10 client.0 _ll_register_callbacks cb 0x7f8ad80092e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:02:51.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.209+0000 7f8adffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5617f5abeef0 0x5617f5ad21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.209+0000 7f8adffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5617f5abeef0 0x5617f5ad21f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48296/0 (socket says 172.21.15.43:48296) 2024-04-25T09:02:51.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.209+0000 7f8adffff640 1 -- 172.21.15.43:0/2904755683 learned_addr learned my addr 172.21.15.43:0/2904755683 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:51.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.209+0000 7f8adf7fe640 1 --2- 172.21.15.43:0/2904755683 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5617f5ab46a0 0x5617f5b28a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.209+0000 7f8adeffd640 1 --2- 172.21.15.43:0/2904755683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617f5abeb10 0x5617f5ad1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.209+0000 7f8adffff640 1 -- 172.21.15.43:0/2904755683 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5617f5ab46a0 msgr2=0x5617f5b28a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:51.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.209+0000 7f8adffff640 1 --2- 172.21.15.43:0/2904755683 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5617f5ab46a0 0x5617f5b28a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.209+0000 7f8adffff640 1 -- 172.21.15.43:0/2904755683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617f5abeb10 msgr2=0x5617f5ad1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:51.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.209+0000 7f8adffff640 1 --2- 172.21.15.43:0/2904755683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617f5abeb10 0x5617f5ad1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.209+0000 7f8adffff640 1 -- 172.21.15.43:0/2904755683 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5617f5abb9c0 con 0x5617f5abeef0 2024-04-25T09:02:51.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.209+0000 7f8adffff640 1 --2- 172.21.15.43:0/2904755683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5617f5abeef0 0x5617f5ad21f0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f8ad802a6e0 tx=0x7f8ad8031e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7288d89322639138 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:51.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.209+0000 7f8abe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:02:51.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.209+0000 7f8abe7fc640 1 -- 172.21.15.43:0/2904755683 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8091040 con 0x5617f5abeef0 2024-04-25T09:02:51.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.209+0000 7f8abe7fc640 1 -- 172.21.15.43:0/2904755683 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8ad808ede0 con 0x5617f5abeef0 2024-04-25T09:02:51.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.209+0000 7f8ae9014740 1 -- 172.21.15.43:0/2904755683 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8ad8079e70 con 0x5617f5abeef0 2024-04-25T09:02:51.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.209+0000 7f8abe7fc640 1 -- 172.21.15.43:0/2904755683 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ad80a1050 con 0x5617f5abeef0 2024-04-25T09:02:51.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.209+0000 7f8ae9014740 10 client.8964 fetch_fsmap learned FSMap version 69 2024-04-25T09:02:51.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.209+0000 7f8ae9014740 1 -- 172.21.15.43:0/2904755683 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x5617f5abb9c0 con 0x5617f5abeef0 2024-04-25T09:02:51.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.209+0000 7f8abe7fc640 1 -- 172.21.15.43:0/2904755683 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad8079e70 con 0x5617f5abeef0 2024-04-25T09:02:51.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.210+0000 7f8abe7fc640 1 -- 172.21.15.43:0/2904755683 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8ad809a070 con 0x5617f5abeef0 2024-04-25T09:02:51.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.210+0000 7f8ae9014740 10 client.8964 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T09:02:51.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.210+0000 7f8ae9014740 10 client.8964 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T09:02:51.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.210+0000 7f8ae9014740 20 client.8964 populate_metadata read hostname 'smithi043' 2024-04-25T09:02:51.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.210+0000 7f8ae9014740 1 --2- 172.21.15.43:0/2904755683 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x5617f5abb250 0x5617f5ab31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.210+0000 7f8ae9014740 4 client.8964 mds_command: new command op to 8676 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:02:51.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.210+0000 7f8ae9014740 1 -- 172.21.15.43:0/2904755683 --> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5617f5a02630 con 0x5617f5abb250 2024-04-25T09:02:51.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.210+0000 7f8adf7fe640 1 --2- 172.21.15.43:0/2904755683 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x5617f5abb250 0x5617f5ab31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.210+0000 7f8adf7fe640 1 --2- 172.21.15.43:0/2904755683 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x5617f5abb250 0x5617f5ab31c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74be43e01bbcfebe server_cookie=9f80bf16fd0270cd in_seq=0 out_seq=0 2024-04-25T09:02:51.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.210+0000 7f8abe7fc640 10 client.8964 ms_handle_connect on v2:172.21.15.43:6833/1586865031 2024-04-25T09:02:51.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.210+0000 7f8abe7fc640 1 -- 172.21.15.43:0/2904755683 <== mds.0 v2:172.21.15.43:6833/1586865031 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+69 (crc 0 0 0) 0x5617f5a02630 con 0x5617f5abb250 2024-04-25T09:02:51.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.210+0000 7f8abe7fc640 10 client.8964 handle_command_reply: tid=0 2024-04-25T09:02:51.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.210+0000 7f8ae9014740 1 client.8964 shutdown 2024-04-25T09:02:51.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.210+0000 7f8ae9014740 1 -- 172.21.15.43:0/2904755683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5617f5abeef0 msgr2=0x5617f5ad21f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:51.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.210+0000 7f8ae9014740 1 --2- 172.21.15.43:0/2904755683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5617f5abeef0 0x5617f5ad21f0 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f8ad802a6e0 tx=0x7f8ad8031e80 comp rx=0 tx=0).stop 2024-04-25T09:02:51.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.210+0000 7f8ae9014740 1 -- 172.21.15.43:0/2904755683 shutdown_connections 2024-04-25T09:02:51.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.210+0000 7f8ae9014740 1 --2- 172.21.15.43:0/2904755683 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x5617f5abb250 0x5617f5ab31c0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.210+0000 7f8ae9014740 1 --2- 172.21.15.43:0/2904755683 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5617f5abeef0 0x5617f5ad21f0 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.210+0000 7f8ae9014740 1 --2- 172.21.15.43:0/2904755683 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5617f5abeb10 0x5617f5ad1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.210+0000 7f8abe7fc640 0 client.8964 ms_handle_reset on v2:172.21.15.43:6833/1586865031 2024-04-25T09:02:51.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.210+0000 7f8ae9014740 1 --2- 172.21.15.43:0/2904755683 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5617f5ab46a0 0x5617f5b28a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.210+0000 7f8ae9014740 1 -- 172.21.15.43:0/2904755683 >> 172.21.15.43:0/2904755683 conn(0x5617f5a91770 msgr2=0x5617f5ac08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:51.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.210+0000 7f8ae9014740 1 -- 172.21.15.43:0/2904755683 shutdown_connections 2024-04-25T09:02:51.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.210+0000 7f8ae9014740 1 -- 172.21.15.43:0/2904755683 wait complete. 2024-04-25T09:02:51.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.211+0000 7f8ae9014740 20 client.8964 trim_cache size 0 max 16384 2024-04-25T09:02:51.206 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:02:51.206 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T09:02:51.206 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-25T09:02:51.206 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:02:51.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.211+0000 7f8ae6238640 1 -- 172.21.15.43:0/1918154093 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8acc06b4d0 msgr2=0x7f8acc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:51.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.211+0000 7f8ae6238640 1 --2- 172.21.15.43:0/1918154093 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8acc06b4d0 0x7f8acc06d990 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0009a70 tx=0x7f8ad0008040 comp rx=0 tx=0).stop 2024-04-25T09:02:51.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.211+0000 7f8ae6238640 1 -- 172.21.15.43:0/1918154093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ae015dec0 msgr2=0x7f8ae00b5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:51.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.211+0000 7f8ae6238640 1 --2- 172.21.15.43:0/1918154093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ae015dec0 0x7f8ae00b5cf0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8034040 tx=0x7f8ad8004a40 comp rx=0 tx=0).stop 2024-04-25T09:02:51.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.211+0000 7f8ae6238640 1 -- 172.21.15.43:0/1918154093 shutdown_connections 2024-04-25T09:02:51.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.211+0000 7f8ae6238640 1 --2- 172.21.15.43:0/1918154093 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ae015dec0 0x7f8ae00b5cf0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.211+0000 7f8ae6238640 1 --2- 172.21.15.43:0/1918154093 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8acc06b4d0 0x7f8acc06d990 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.211+0000 7f8ae6238640 1 --2- 172.21.15.43:0/1918154093 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ae00d04a0 0x7f8ae014dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.211+0000 7f8ae6238640 1 --2- 172.21.15.43:0/1918154093 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ae00ce8c0 0x7f8ae00cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.211+0000 7f8ae6238640 1 -- 172.21.15.43:0/1918154093 >> 172.21.15.43:0/1918154093 conn(0x7f8ae009cd40 msgr2=0x7f8ae0160df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:51.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.211+0000 7f8ae6238640 1 -- 172.21.15.43:0/1918154093 shutdown_connections 2024-04-25T09:02:51.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.211+0000 7f8ae6238640 1 -- 172.21.15.43:0/1918154093 wait complete. 2024-04-25T09:02:51.210 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:02:51.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.322+0000 7fc53ea9a640 1 Processor -- start 2024-04-25T09:02:51.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.322+0000 7fc53ea9a640 1 -- start start 2024-04-25T09:02:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.323+0000 7fc53ea9a640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fc53815dce0 0x7fc53815ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.323+0000 7fc53ea9a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc5380ce550 0x7fc5380d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.323+0000 7fc53ea9a640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5380cee80 con 0x7fc5380ce9c0 2024-04-25T09:02:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.323+0000 7fc53ea9a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5380d2920 con 0x7fc5380ce550 2024-04-25T09:02:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.323+0000 7fc53ea9a640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5380cf160 con 0x7fc53815dce0 2024-04-25T09:02:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.323+0000 7fc537fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc5380ce550 0x7fc5380d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.323+0000 7fc537fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc5380ce550 0x7fc5380d03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48304/0 (socket says 172.21.15.43:48304) 2024-04-25T09:02:51.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.323+0000 7fc537fff640 1 -- 172.21.15.43:0/121579806 learned_addr learned my addr 172.21.15.43:0/121579806 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:51.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.323+0000 7fc53c836640 1 --2- 172.21.15.43:0/121579806 >> v2:172.21.15.193:3300/0 conn(0x7fc53815dce0 0x7fc53815ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.323+0000 7fc537fff640 1 -- 172.21.15.43:0/121579806 >> v2:172.21.15.193:3300/0 conn(0x7fc53815dce0 msgr2=0x7fc53815ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:51.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.323+0000 7fc537fff640 1 --2- 172.21.15.43:0/121579806 >> v2:172.21.15.193:3300/0 conn(0x7fc53815dce0 0x7fc53815ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.323+0000 7fc537fff640 1 -- 172.21.15.43:0/121579806 >> v1:172.21.15.193:6789/0 conn(0x7fc5380ce9c0 legacy=0x7fc538160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:51.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.323+0000 7fc537fff640 1 -- 172.21.15.43:0/121579806 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5380cf440 con 0x7fc5380ce550 2024-04-25T09:02:51.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.324+0000 7fc537fff640 1 --2- 172.21.15.43:0/121579806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5380ce550 0x7fc5380d03e0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fc53809d3c0 tx=0x7fc52c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61bc6dfe2cc271eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:51.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.324+0000 7fc5377fe640 1 -- 172.21.15.43:0/121579806 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c037070 con 0x7fc5380ce550 2024-04-25T09:02:51.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.324+0000 7fc5377fe640 1 -- 172.21.15.43:0/121579806 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52c032440 con 0x7fc5380ce550 2024-04-25T09:02:51.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.324+0000 7fc5377fe640 1 -- 172.21.15.43:0/121579806 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c032810 con 0x7fc5380ce550 2024-04-25T09:02:51.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.325+0000 7fc53ea9a640 1 -- 172.21.15.43:0/121579806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5380ce550 msgr2=0x7fc5380d03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:51.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.325+0000 7fc53ea9a640 1 --2- 172.21.15.43:0/121579806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5380ce550 0x7fc5380d03e0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fc53809d3c0 tx=0x7fc52c02a1a0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.325+0000 7fc53ea9a640 1 -- 172.21.15.43:0/121579806 shutdown_connections 2024-04-25T09:02:51.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.325+0000 7fc53ea9a640 1 --2- 172.21.15.43:0/121579806 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5380ce550 0x7fc5380d03e0 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.325+0000 7fc53ea9a640 1 --2- 172.21.15.43:0/121579806 >> v2:172.21.15.193:3300/0 conn(0x7fc53815dce0 0x7fc53815ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.325+0000 7fc53ea9a640 1 -- 172.21.15.43:0/121579806 >> 172.21.15.43:0/121579806 conn(0x7fc53809cd00 msgr2=0x7fc5380d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:51.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.325+0000 7fc53ea9a640 1 -- 172.21.15.43:0/121579806 shutdown_connections 2024-04-25T09:02:51.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.325+0000 7fc53ea9a640 1 -- 172.21.15.43:0/121579806 wait complete. 2024-04-25T09:02:51.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.326+0000 7fc53ea9a640 1 Processor -- start 2024-04-25T09:02:51.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.326+0000 7fc53ea9a640 1 -- start start 2024-04-25T09:02:51.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.327+0000 7fc53ea9a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc5380ce550 0x7fc5381e72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.327+0000 7fc53ea9a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5380ce9c0 0x7fc5380cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.327+0000 7fc53ea9a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc53815dce0 0x7fc5380cc030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.327+0000 7fc53ea9a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53815e0c0 con 0x7fc5380ce9c0 2024-04-25T09:02:51.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.327+0000 7fc53ea9a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53815f6b0 con 0x7fc53815dce0 2024-04-25T09:02:51.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.327+0000 7fc53ea9a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5380d0920 con 0x7fc5380ce550 2024-04-25T09:02:51.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.327+0000 7fc537fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5380ce9c0 0x7fc5380cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.327+0000 7fc537fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5380ce9c0 0x7fc5380cbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48318/0 (socket says 172.21.15.43:48318) 2024-04-25T09:02:51.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.327+0000 7fc537fff640 1 -- 172.21.15.43:0/2519816713 learned_addr learned my addr 172.21.15.43:0/2519816713 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:51.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.327+0000 7fc53d037640 1 --2- 172.21.15.43:0/2519816713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc53815dce0 0x7fc5380cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.327+0000 7fc53c836640 1 --2- 172.21.15.43:0/2519816713 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc5380ce550 0x7fc5381e72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.327+0000 7fc537fff640 1 -- 172.21.15.43:0/2519816713 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc5380ce550 msgr2=0x7fc5381e72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:51.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.327+0000 7fc537fff640 1 --2- 172.21.15.43:0/2519816713 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc5380ce550 0x7fc5381e72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.327+0000 7fc537fff640 1 -- 172.21.15.43:0/2519816713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc53815dce0 msgr2=0x7fc5380cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:51.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.327+0000 7fc537fff640 1 --2- 172.21.15.43:0/2519816713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc53815dce0 0x7fc5380cc030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.327+0000 7fc537fff640 1 -- 172.21.15.43:0/2519816713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5380b6790 con 0x7fc5380ce9c0 2024-04-25T09:02:51.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.327+0000 7fc53d037640 1 --2- 172.21.15.43:0/2519816713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc53815dce0 0x7fc5380cc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:51.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.327+0000 7fc537fff640 1 --2- 172.21.15.43:0/2519816713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5380ce9c0 0x7fc5380cbaf0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fc52c000c00 tx=0x7fc52c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:51.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.328+0000 7fc5357fa640 1 -- 172.21.15.43:0/2519816713 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c041020 con 0x7fc5380ce9c0 2024-04-25T09:02:51.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.328+0000 7fc5357fa640 1 -- 172.21.15.43:0/2519816713 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc52c030970 con 0x7fc5380ce9c0 2024-04-25T09:02:51.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.328+0000 7fc53ea9a640 1 -- 172.21.15.43:0/2519816713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5380b6a70 con 0x7fc5380ce9c0 2024-04-25T09:02:51.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.328+0000 7fc5357fa640 1 -- 172.21.15.43:0/2519816713 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52c037040 con 0x7fc5380ce9c0 2024-04-25T09:02:51.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.328+0000 7fc53ea9a640 1 -- 172.21.15.43:0/2519816713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5380b7030 con 0x7fc5380ce9c0 2024-04-25T09:02:51.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.329+0000 7fc53ea9a640 1 -- 172.21.15.43:0/2519816713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5380680f0 con 0x7fc5380ce9c0 2024-04-25T09:02:51.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.329+0000 7fc5357fa640 1 -- 172.21.15.43:0/2519816713 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc52c044020 con 0x7fc5380ce9c0 2024-04-25T09:02:51.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.330+0000 7fc5357fa640 1 --2- 172.21.15.43:0/2519816713 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc52406b4d0 0x7fc52406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.330+0000 7fc5357fa640 1 -- 172.21.15.43:0/2519816713 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc52c0aaca0 con 0x7fc5380ce9c0 2024-04-25T09:02:51.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.330+0000 7fc53c836640 1 --2- 172.21.15.43:0/2519816713 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc52406b4d0 0x7fc52406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.330+0000 7fc53c836640 1 --2- 172.21.15.43:0/2519816713 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc52406b4d0 0x7fc52406d990 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fc52800b7c0 tx=0x7fc528008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:51.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.334+0000 7fc5357fa640 1 -- 172.21.15.43:0/2519816713 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc52c04b030 con 0x7fc5380ce9c0 2024-04-25T09:02:51.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.554+0000 7fc53ea9a640 1 -- 172.21.15.43:0/2519816713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc53809d1a0 con 0x7fc5380ce9c0 2024-04-25T09:02:51.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.555+0000 7fc5357fa640 1 -- 172.21.15.43:0/2519816713 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fc52c079a80 con 0x7fc5380ce9c0 2024-04-25T09:02:51.543 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:51.543 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":69,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6821,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2021228703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2021228703},{"type":"v1","addr":"172.21.15.193:6837","nonce":2021228703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65},{"gid":7543,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/2398324998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2398324998},{"type":"v1","addr":"172.21.15.193:6836","nonce":2398324998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65},{"gid":8673,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2298187831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2298187831},{"type":"v1","addr":"172.21.15.43:6834","nonce":2298187831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:02:26.161487+0000","modified":"2024-04-25T09:02:29.177294+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8676":{"gid":8676,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/1586865031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1586865031},{"type":"v1","addr":"172.21.15.43:6835","nonce":1586865031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:02:51.544 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 69 2024-04-25T09:02:51.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.557+0000 7fc53ea9a640 1 -- 172.21.15.43:0/2519816713 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc52406b4d0 msgr2=0x7fc52406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:51.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.557+0000 7fc53ea9a640 1 --2- 172.21.15.43:0/2519816713 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc52406b4d0 0x7fc52406d990 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fc52800b7c0 tx=0x7fc528008040 comp rx=0 tx=0).stop 2024-04-25T09:02:51.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.557+0000 7fc53ea9a640 1 -- 172.21.15.43:0/2519816713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5380ce9c0 msgr2=0x7fc5380cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:51.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.557+0000 7fc53ea9a640 1 --2- 172.21.15.43:0/2519816713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5380ce9c0 0x7fc5380cbaf0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fc52c000c00 tx=0x7fc52c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.557+0000 7fc53ea9a640 1 -- 172.21.15.43:0/2519816713 shutdown_connections 2024-04-25T09:02:51.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.557+0000 7fc53ea9a640 1 --2- 172.21.15.43:0/2519816713 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc52406b4d0 0x7fc52406d990 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.557+0000 7fc53ea9a640 1 --2- 172.21.15.43:0/2519816713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc53815dce0 0x7fc5380cc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.557+0000 7fc53ea9a640 1 --2- 172.21.15.43:0/2519816713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc5380ce9c0 0x7fc5380cbaf0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.557+0000 7fc53ea9a640 1 --2- 172.21.15.43:0/2519816713 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc5380ce550 0x7fc5381e72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.557+0000 7fc53ea9a640 1 -- 172.21.15.43:0/2519816713 >> 172.21.15.43:0/2519816713 conn(0x7fc53809cd00 msgr2=0x7fc53806c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:51.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.557+0000 7fc53ea9a640 1 -- 172.21.15.43:0/2519816713 shutdown_connections 2024-04-25T09:02:51.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.557+0000 7fc53ea9a640 1 -- 172.21.15.43:0/2519816713 wait complete. 2024-04-25T09:02:51.557 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T09:02:51.557 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:02:51.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.669+0000 7f12da89e640 1 Processor -- start 2024-04-25T09:02:51.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.669+0000 7f12da89e640 1 -- start start 2024-04-25T09:02:51.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.670+0000 7f12da89e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f12d40c8d00 0x7f12d4162100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.670+0000 7f12da89e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f12d40ce660 0x7f12d40cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.670+0000 7f12da89e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12d40d24a0 con 0x7f12d415df50 2024-04-25T09:02:51.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.670+0000 7f12da89e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12d415e330 con 0x7f12d40ce660 2024-04-25T09:02:51.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.670+0000 7f12da89e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d40d2740 con 0x7f12d40c8d00 2024-04-25T09:02:51.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.670+0000 7f12d3fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f12d415df50 0x7f12d41619f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:53394/0 (socket says 172.21.15.43:53394) 2024-04-25T09:02:51.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.670+0000 7f12d8e3b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f12d40ce660 0x7f12d40cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.670+0000 7f12d3fff640 1 -- 172.21.15.43:0/3866140836 learned_addr learned my addr 172.21.15.43:0/3866140836 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:51.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.671+0000 7f12d37fe640 1 --2- 172.21.15.43:0/3866140836 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f12d40c8d00 0x7f12d4162100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.671+0000 7f12d2ffd640 1 -- 172.21.15.43:0/3866140836 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1408572857 0 0) 0x7f12d40d24a0 con 0x7f12d415df50 2024-04-25T09:02:51.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.671+0000 7f12d2ffd640 1 -- 172.21.15.43:0/3866140836 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12bc003520 con 0x7f12d415df50 2024-04-25T09:02:51.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.671+0000 7f12d8e3b640 1 -- 172.21.15.43:0/3866140836 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f12d40c8d00 msgr2=0x7f12d4162100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:51.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.671+0000 7f12d8e3b640 1 --2- 172.21.15.43:0/3866140836 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f12d40c8d00 0x7f12d4162100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.671+0000 7f12d8e3b640 1 -- 172.21.15.43:0/3866140836 >> v1:172.21.15.43:6789/0 conn(0x7f12d415df50 legacy=0x7f12d41619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:51.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.671+0000 7f12d8e3b640 1 -- 172.21.15.43:0/3866140836 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d40d2a20 con 0x7f12d40ce660 2024-04-25T09:02:51.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.671+0000 7f12d37fe640 1 --2- 172.21.15.43:0/3866140836 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f12d40c8d00 0x7f12d4162100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:51.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.671+0000 7f12d8e3b640 1 --2- 172.21.15.43:0/3866140836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12d40ce660 0x7f12d40cea70 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f12cc009d90 tx=0x7f12cc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42516eb0d72f9b24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:51.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.672+0000 7f12d2ffd640 1 -- 172.21.15.43:0/3866140836 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12cc007740 con 0x7f12d40ce660 2024-04-25T09:02:51.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.672+0000 7f12d2ffd640 1 -- 172.21.15.43:0/3866140836 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12cc0078e0 con 0x7f12d40ce660 2024-04-25T09:02:51.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.672+0000 7f12d2ffd640 1 -- 172.21.15.43:0/3866140836 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12cc007cb0 con 0x7f12d40ce660 2024-04-25T09:02:51.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.672+0000 7f12da89e640 1 -- 172.21.15.43:0/3866140836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12d40ce660 msgr2=0x7f12d40cea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:51.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.672+0000 7f12da89e640 1 --2- 172.21.15.43:0/3866140836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12d40ce660 0x7f12d40cea70 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f12cc009d90 tx=0x7f12cc02a660 comp rx=0 tx=0).stop 2024-04-25T09:02:51.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.672+0000 7f12da89e640 1 -- 172.21.15.43:0/3866140836 shutdown_connections 2024-04-25T09:02:51.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.672+0000 7f12da89e640 1 --2- 172.21.15.43:0/3866140836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12d40ce660 0x7f12d40cea70 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.672+0000 7f12da89e640 1 --2- 172.21.15.43:0/3866140836 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f12d40c8d00 0x7f12d4162100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.672+0000 7f12da89e640 1 -- 172.21.15.43:0/3866140836 >> 172.21.15.43:0/3866140836 conn(0x7f12d40d3a30 msgr2=0x7f12d40d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:51.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.672+0000 7f12da89e640 1 -- 172.21.15.43:0/3866140836 shutdown_connections 2024-04-25T09:02:51.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.672+0000 7f12da89e640 1 -- 172.21.15.43:0/3866140836 wait complete. 2024-04-25T09:02:51.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.673+0000 7f12da89e640 1 Processor -- start 2024-04-25T09:02:51.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.674+0000 7f12da89e640 1 -- start start 2024-04-25T09:02:51.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.674+0000 7f12da89e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12d40c8d00 0x7f12d40c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.674+0000 7f12da89e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12d40ce660 0x7f12d4160da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.674+0000 7f12da89e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f12d415df50 0x7f12d40cc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.674+0000 7f12da89e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d40d2190 con 0x7f12d40c8d00 2024-04-25T09:02:51.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.674+0000 7f12da89e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12d40c77d0 con 0x7f12d40ce660 2024-04-25T09:02:51.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.674+0000 7f12da89e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12d4162640 con 0x7f12d415df50 2024-04-25T09:02:51.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.674+0000 7f12d3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12d40c8d00 0x7f12d40c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.674+0000 7f12d3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12d40c8d00 0x7f12d40c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48338/0 (socket says 172.21.15.43:48338) 2024-04-25T09:02:51.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.674+0000 7f12d3fff640 1 -- 172.21.15.43:0/4278458612 learned_addr learned my addr 172.21.15.43:0/4278458612 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:51.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.675+0000 7f12d8e3b640 1 --2- 172.21.15.43:0/4278458612 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f12d415df50 0x7f12d40cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.675+0000 7f12d37fe640 1 --2- 172.21.15.43:0/4278458612 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12d40ce660 0x7f12d4160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.675+0000 7f12d3fff640 1 -- 172.21.15.43:0/4278458612 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f12d415df50 msgr2=0x7f12d40cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:51.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.675+0000 7f12d3fff640 1 --2- 172.21.15.43:0/4278458612 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f12d415df50 0x7f12d40cc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.675+0000 7f12d3fff640 1 -- 172.21.15.43:0/4278458612 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12d40ce660 msgr2=0x7f12d4160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:51.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.675+0000 7f12d3fff640 1 --2- 172.21.15.43:0/4278458612 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12d40ce660 0x7f12d4160da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.675+0000 7f12d3fff640 1 -- 172.21.15.43:0/4278458612 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d415f0d0 con 0x7f12d40c8d00 2024-04-25T09:02:51.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.675+0000 7f12d3fff640 1 --2- 172.21.15.43:0/4278458612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12d40c8d00 0x7f12d40c66f0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f12c400c460 tx=0x7f12c400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:51.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.675+0000 7f12d0ff9640 1 -- 172.21.15.43:0/4278458612 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c4017070 con 0x7f12d40c8d00 2024-04-25T09:02:51.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.675+0000 7f12d0ff9640 1 -- 172.21.15.43:0/4278458612 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c4005ce0 con 0x7f12d40c8d00 2024-04-25T09:02:51.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.675+0000 7f12d0ff9640 1 -- 172.21.15.43:0/4278458612 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c4007290 con 0x7f12d40c8d00 2024-04-25T09:02:51.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.675+0000 7f12da89e640 1 -- 172.21.15.43:0/4278458612 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12d415f3b0 con 0x7f12d40c8d00 2024-04-25T09:02:51.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.675+0000 7f12da89e640 1 -- 172.21.15.43:0/4278458612 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12d41612e0 con 0x7f12d40c8d00 2024-04-25T09:02:51.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.676+0000 7f12da89e640 1 -- 172.21.15.43:0/4278458612 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12d40680f0 con 0x7f12d40c8d00 2024-04-25T09:02:51.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.677+0000 7f12d0ff9640 1 -- 172.21.15.43:0/4278458612 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f12c401a070 con 0x7f12d40c8d00 2024-04-25T09:02:51.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.677+0000 7f12d0ff9640 1 --2- 172.21.15.43:0/4278458612 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f12c006b500 0x7f12c006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:51.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.677+0000 7f12d0ff9640 1 -- 172.21.15.43:0/4278458612 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f12c4089540 con 0x7f12d40c8d00 2024-04-25T09:02:51.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.677+0000 7f12d37fe640 1 --2- 172.21.15.43:0/4278458612 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f12c006b500 0x7f12c006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:51.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.678+0000 7f12d37fe640 1 --2- 172.21.15.43:0/4278458612 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f12c006b500 0x7f12c006d9c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f12d4160660 tx=0x7f12c8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:51.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.681+0000 7f12d0ff9640 1 -- 172.21.15.43:0/4278458612 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f12c40583c0 con 0x7f12d40c8d00 2024-04-25T09:02:51.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.949+0000 7f12da89e640 1 -- 172.21.15.43:0/4278458612 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f12d4074ef0 con 0x7f12d40c8d00 2024-04-25T09:02:51.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.949+0000 7f12d0ff9640 1 -- 172.21.15.43:0/4278458612 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7f12c405e0c0 con 0x7f12d40c8d00 2024-04-25T09:02:51.938 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:51.938 INFO:teuthology.orchestra.run.smithi043.stdout:{"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":544,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":113,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035238,"num_in_osds":8,"osd_in_since":1714035229,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1077670390,"bytes_used":2669371392,"bytes_avail":770424741888,"bytes_total":773094113280,"write_bytes_sec":54673090,"read_op_per_sec":0,"write_op_per_sec":54},"fsmap":{"epoch":69,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8676}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":21,"modified":"2024-04-25T09:02:49.814157+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:02:51.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.951+0000 7f12da89e640 1 -- 172.21.15.43:0/4278458612 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f12c006b500 msgr2=0x7f12c006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:51.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.951+0000 7f12da89e640 1 --2- 172.21.15.43:0/4278458612 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f12c006b500 0x7f12c006d9c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f12d4160660 tx=0x7f12c8008040 comp rx=0 tx=0).stop 2024-04-25T09:02:51.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.951+0000 7f12da89e640 1 -- 172.21.15.43:0/4278458612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12d40c8d00 msgr2=0x7f12d40c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:51.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.951+0000 7f12da89e640 1 --2- 172.21.15.43:0/4278458612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12d40c8d00 0x7f12d40c66f0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f12c400c460 tx=0x7f12c400c920 comp rx=0 tx=0).stop 2024-04-25T09:02:51.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.951+0000 7f12da89e640 1 -- 172.21.15.43:0/4278458612 shutdown_connections 2024-04-25T09:02:51.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.951+0000 7f12da89e640 1 --2- 172.21.15.43:0/4278458612 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f12d415df50 0x7f12d40cc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.951+0000 7f12da89e640 1 --2- 172.21.15.43:0/4278458612 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f12c006b500 0x7f12c006d9c0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.951+0000 7f12da89e640 1 --2- 172.21.15.43:0/4278458612 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f12d40ce660 0x7f12d4160da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.951+0000 7f12da89e640 1 --2- 172.21.15.43:0/4278458612 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f12d40c8d00 0x7f12d40c66f0 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:51.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.951+0000 7f12da89e640 1 -- 172.21.15.43:0/4278458612 >> 172.21.15.43:0/4278458612 conn(0x7f12d40d3a30 msgr2=0x7f12d406b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:51.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.951+0000 7f12da89e640 1 -- 172.21.15.43:0/4278458612 shutdown_connections 2024-04-25T09:02:51.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:51.951+0000 7f12da89e640 1 -- 172.21.15.43:0/4278458612 wait complete. 2024-04-25T09:02:51.951 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused'} 2024-04-25T09:02:51.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:51.951 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub resume 2024-04-25T09:02:52.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.065+0000 7f4b69ec4640 1 Processor -- start 2024-04-25T09:02:52.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.065+0000 7f4b69ec4640 1 -- start start 2024-04-25T09:02:52.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.065+0000 7f4b69ec4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4b6406b240 0x7f4b640cda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.065+0000 7f4b69ec4640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f4b640cdfa0 0x7f4b640b3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.065+0000 7f4b69ec4640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b6415f4c0 con 0x7f4b641450a0 2024-04-25T09:02:52.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.066+0000 7f4b69ec4640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b641600d0 con 0x7f4b6406b240 2024-04-25T09:02:52.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.066+0000 7f4b69ec4640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b6415f7a0 con 0x7f4b640cdfa0 2024-04-25T09:02:52.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.066+0000 7f4b62ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4b6406b240 0x7f4b640cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.066+0000 7f4b637fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4b641450a0 0x7f4b6406ab30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:53400/0 (socket says 172.21.15.43:53400) 2024-04-25T09:02:52.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.066+0000 7f4b637fe640 1 -- 172.21.15.43:0/3968047686 learned_addr learned my addr 172.21.15.43:0/3968047686 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:52.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.066+0000 7f4b63fff640 1 --2- 172.21.15.43:0/3968047686 >> v2:172.21.15.193:3300/0 conn(0x7f4b640cdfa0 0x7f4b640b3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.066+0000 7f4b62ffd640 1 -- 172.21.15.43:0/3968047686 >> v2:172.21.15.193:3300/0 conn(0x7f4b640cdfa0 msgr2=0x7f4b640b3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:52.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.066+0000 7f4b62ffd640 1 --2- 172.21.15.43:0/3968047686 >> v2:172.21.15.193:3300/0 conn(0x7f4b640cdfa0 0x7f4b640b3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.066+0000 7f4b62ffd640 1 -- 172.21.15.43:0/3968047686 >> v1:172.21.15.43:6789/0 conn(0x7f4b641450a0 legacy=0x7f4b6406ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.066+0000 7f4b627fc640 1 -- 172.21.15.43:0/3968047686 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1953991003 0 0) 0x7f4b6415f4c0 con 0x7f4b641450a0 2024-04-25T09:02:52.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.066+0000 7f4b62ffd640 1 -- 172.21.15.43:0/3968047686 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b6415fa80 con 0x7f4b6406b240 2024-04-25T09:02:52.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.067+0000 7f4b62ffd640 1 --2- 172.21.15.43:0/3968047686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b6406b240 0x7f4b640cda60 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f4b640cf660 tx=0x7f4b5802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b34be387a99eaf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:52.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.067+0000 7f4b627fc640 1 -- 172.21.15.43:0/3968047686 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b58037070 con 0x7f4b6406b240 2024-04-25T09:02:52.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.067+0000 7f4b627fc640 1 -- 172.21.15.43:0/3968047686 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b58032440 con 0x7f4b6406b240 2024-04-25T09:02:52.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.067+0000 7f4b627fc640 1 -- 172.21.15.43:0/3968047686 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5802ae60 con 0x7f4b6406b240 2024-04-25T09:02:52.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.067+0000 7f4b69ec4640 1 -- 172.21.15.43:0/3968047686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b6406b240 msgr2=0x7f4b640cda60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.067+0000 7f4b69ec4640 1 --2- 172.21.15.43:0/3968047686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b6406b240 0x7f4b640cda60 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f4b640cf660 tx=0x7f4b5802a110 comp rx=0 tx=0).stop 2024-04-25T09:02:52.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.067+0000 7f4b69ec4640 1 -- 172.21.15.43:0/3968047686 shutdown_connections 2024-04-25T09:02:52.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.067+0000 7f4b69ec4640 1 --2- 172.21.15.43:0/3968047686 >> v2:172.21.15.193:3300/0 conn(0x7f4b640cdfa0 0x7f4b640b3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.067+0000 7f4b69ec4640 1 --2- 172.21.15.43:0/3968047686 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b6406b240 0x7f4b640cda60 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.067+0000 7f4b69ec4640 1 -- 172.21.15.43:0/3968047686 >> 172.21.15.43:0/3968047686 conn(0x7f4b640b2590 msgr2=0x7f4b640b2970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:52.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.067+0000 7f4b69ec4640 1 -- 172.21.15.43:0/3968047686 shutdown_connections 2024-04-25T09:02:52.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.067+0000 7f4b69ec4640 1 -- 172.21.15.43:0/3968047686 wait complete. 2024-04-25T09:02:52.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.069+0000 7f4b69ec4640 1 Processor -- start 2024-04-25T09:02:52.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.069+0000 7f4b69ec4640 1 -- start start 2024-04-25T09:02:52.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.069+0000 7f4b69ec4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b6406b240 0x7f4b64140010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.069+0000 7f4b69ec4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b640cdfa0 0x7f4b640ae2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.069+0000 7f4b69ec4640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4b641450a0 0x7f4b64142f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.069+0000 7f4b69ec4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b64145480 con 0x7f4b640cdfa0 2024-04-25T09:02:52.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.069+0000 7f4b69ec4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b640ce3f0 con 0x7f4b6406b240 2024-04-25T09:02:52.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.069+0000 7f4b69ec4640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b64145600 con 0x7f4b641450a0 2024-04-25T09:02:52.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.070+0000 7f4b62ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b640cdfa0 0x7f4b640ae2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.070+0000 7f4b62ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b640cdfa0 0x7f4b640ae2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48358/0 (socket says 172.21.15.43:48358) 2024-04-25T09:02:52.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.070+0000 7f4b62ffd640 1 -- 172.21.15.43:0/4195383098 learned_addr learned my addr 172.21.15.43:0/4195383098 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:52.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.070+0000 7f4b637fe640 1 --2- 172.21.15.43:0/4195383098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b6406b240 0x7f4b64140010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.070+0000 7f4b63fff640 1 --2- 172.21.15.43:0/4195383098 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4b641450a0 0x7f4b64142f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.070+0000 7f4b62ffd640 1 -- 172.21.15.43:0/4195383098 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4b641450a0 msgr2=0x7f4b64142f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.070+0000 7f4b62ffd640 1 --2- 172.21.15.43:0/4195383098 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4b641450a0 0x7f4b64142f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.070+0000 7f4b62ffd640 1 -- 172.21.15.43:0/4195383098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b6406b240 msgr2=0x7f4b64140010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.070+0000 7f4b62ffd640 1 --2- 172.21.15.43:0/4195383098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b6406b240 0x7f4b64140010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.070+0000 7f4b62ffd640 1 -- 172.21.15.43:0/4195383098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b640aed90 con 0x7f4b640cdfa0 2024-04-25T09:02:52.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.070+0000 7f4b63fff640 1 --2- 172.21.15.43:0/4195383098 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4b641450a0 0x7f4b64142f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:52.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.070+0000 7f4b62ffd640 1 --2- 172.21.15.43:0/4195383098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b640cdfa0 0x7f4b640ae2f0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f4b58000c00 tx=0x7f4b580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:52.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.070+0000 7f4b4bfff640 1 -- 172.21.15.43:0/4195383098 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b58037040 con 0x7f4b640cdfa0 2024-04-25T09:02:52.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.070+0000 7f4b4bfff640 1 -- 172.21.15.43:0/4195383098 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b58042070 con 0x7f4b640cdfa0 2024-04-25T09:02:52.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.070+0000 7f4b69ec4640 1 -- 172.21.15.43:0/4195383098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b640af070 con 0x7f4b640cdfa0 2024-04-25T09:02:52.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.070+0000 7f4b4bfff640 1 -- 172.21.15.43:0/4195383098 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b58030a50 con 0x7f4b640cdfa0 2024-04-25T09:02:52.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.070+0000 7f4b69ec4640 1 -- 172.21.15.43:0/4195383098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b6406e860 con 0x7f4b640cdfa0 2024-04-25T09:02:52.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.072+0000 7f4b4bfff640 1 -- 172.21.15.43:0/4195383098 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4b58030bf0 con 0x7f4b640cdfa0 2024-04-25T09:02:52.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.072+0000 7f4b4bfff640 1 --2- 172.21.15.43:0/4195383098 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4b5006b4d0 0x7f4b5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.072+0000 7f4b4bfff640 1 -- 172.21.15.43:0/4195383098 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4b580aad10 con 0x7f4b640cdfa0 2024-04-25T09:02:52.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.072+0000 7f4b637fe640 1 --2- 172.21.15.43:0/4195383098 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4b5006b4d0 0x7f4b5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.073+0000 7f4b637fe640 1 --2- 172.21.15.43:0/4195383098 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4b5006b4d0 0x7f4b5006d990 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f4b54000f60 tx=0x7f4b54007c00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:52.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.073+0000 7f4b6cbc7740 1 Processor -- start 2024-04-25T09:02:52.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.073+0000 7f4b6cbc7740 1 -- start start 2024-04-25T09:02:52.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.074+0000 7f4b6cbc7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c1789f1b10 0x55c1789f4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.074+0000 7f4b6cbc7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1789f1ef0 0x55c1789f4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.074+0000 7f4b6cbc7740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c1789c4770 0x55c1789c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.074+0000 7f4b6cbc7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c17892dda0 con 0x55c1789f1b10 2024-04-25T09:02:52.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.074+0000 7f4b6cbc7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1788d6b80 con 0x55c1789f1ef0 2024-04-25T09:02:52.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.074+0000 7f4b6cbc7740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c178929e40 con 0x55c1789c4770 2024-04-25T09:02:52.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.074+0000 7f4b637fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c1789f1b10 0x55c1789f4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.074+0000 7f4b637fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c1789f1b10 0x55c1789f4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48372/0 (socket says 172.21.15.43:48372) 2024-04-25T09:02:52.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.074+0000 7f4b637fe640 1 -- 172.21.15.43:0/2868026764 learned_addr learned my addr 172.21.15.43:0/2868026764 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:52.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.074+0000 7f4b63fff640 1 --2- 172.21.15.43:0/2868026764 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c1789c4770 0x55c1789c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.074+0000 7f4b62ffd640 1 --2- 172.21.15.43:0/2868026764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1789f1ef0 0x55c1789f4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.074+0000 7f4b637fe640 1 -- 172.21.15.43:0/2868026764 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c1789c4770 msgr2=0x55c1789c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:52.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.074+0000 7f4b637fe640 1 --2- 172.21.15.43:0/2868026764 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c1789c4770 0x55c1789c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.074+0000 7f4b637fe640 1 -- 172.21.15.43:0/2868026764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1789f1ef0 msgr2=0x55c1789f4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:52.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.074+0000 7f4b637fe640 1 --2- 172.21.15.43:0/2868026764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1789f1ef0 0x55c1789f4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.074+0000 7f4b637fe640 1 -- 172.21.15.43:0/2868026764 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1788e0f10 con 0x55c1789f1b10 2024-04-25T09:02:52.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.074+0000 7f4b637fe640 1 --2- 172.21.15.43:0/2868026764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c1789f1b10 0x55c1789f4320 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f4b54005db0 tx=0x7f4b54012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a000a0bf9273272b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:52.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.075+0000 7f4b497fa640 1 -- 172.21.15.43:0/2868026764 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b54015110 con 0x55c1789f1b10 2024-04-25T09:02:52.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.075+0000 7f4b497fa640 1 -- 172.21.15.43:0/2868026764 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b540159c0 con 0x55c1789f1b10 2024-04-25T09:02:52.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.075+0000 7f4b497fa640 1 -- 172.21.15.43:0/2868026764 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b54015d70 con 0x55c1789f1b10 2024-04-25T09:02:52.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.075+0000 7f4b6cbc7740 1 -- 172.21.15.43:0/2868026764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c1789f1b10 msgr2=0x55c1789f4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.075+0000 7f4b6cbc7740 1 --2- 172.21.15.43:0/2868026764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c1789f1b10 0x55c1789f4320 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f4b54005db0 tx=0x7f4b54012000 comp rx=0 tx=0).stop 2024-04-25T09:02:52.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.075+0000 7f4b6cbc7740 1 -- 172.21.15.43:0/2868026764 shutdown_connections 2024-04-25T09:02:52.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.075+0000 7f4b6cbc7740 1 --2- 172.21.15.43:0/2868026764 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c1789c4770 0x55c1789c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.075+0000 7f4b6cbc7740 1 --2- 172.21.15.43:0/2868026764 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1789f1ef0 0x55c1789f4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.075+0000 7f4b6cbc7740 1 --2- 172.21.15.43:0/2868026764 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c1789f1b10 0x55c1789f4320 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.075+0000 7f4b6cbc7740 1 -- 172.21.15.43:0/2868026764 >> 172.21.15.43:0/2868026764 conn(0x55c1789c69e0 msgr2=0x55c178a0f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:52.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.075+0000 7f4b6cbc7740 1 -- 172.21.15.43:0/2868026764 shutdown_connections 2024-04-25T09:02:52.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.075+0000 7f4b6cbc7740 1 -- 172.21.15.43:0/2868026764 wait complete. 2024-04-25T09:02:52.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.076+0000 7f4b6cbc7740 1 -- start start 2024-04-25T09:02:52.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.076+0000 7f4b6cbc7740 1 Processor -- start 2024-04-25T09:02:52.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.077+0000 7f4b6cbc7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1789f1b10 0x55c1789e88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.077+0000 7f4b6cbc7740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c1789f1ef0 0x55c178a073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.077+0000 7f4b6cbc7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c178a12d90 0x55c178a07910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.077+0000 7f4b6cbc7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1789c9b90 con 0x55c178a12d90 2024-04-25T09:02:52.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.077+0000 7f4b6cbc7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1788cc4b0 con 0x55c1789f1b10 2024-04-25T09:02:52.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.077+0000 7f4b6cbc7740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c1789c9e70 con 0x55c1789f1ef0 2024-04-25T09:02:52.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.077+0000 7f4b6cbc7740 10 client.0 _ll_register_callbacks cb 0x55c17890c180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:02:52.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.077+0000 7f4b63fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c178a12d90 0x55c178a07910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.077+0000 7f4b63fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c178a12d90 0x55c178a07910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48382/0 (socket says 172.21.15.43:48382) 2024-04-25T09:02:52.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.077+0000 7f4b63fff640 1 -- 172.21.15.43:0/1593687754 learned_addr learned my addr 172.21.15.43:0/1593687754 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:52.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.077+0000 7f4b637fe640 1 --2- 172.21.15.43:0/1593687754 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1789f1b10 0x55c1789e88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.077+0000 7f4b62ffd640 1 --2- 172.21.15.43:0/1593687754 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c1789f1ef0 0x55c178a073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.077+0000 7f4b63fff640 1 -- 172.21.15.43:0/1593687754 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c1789f1ef0 msgr2=0x55c178a073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:52.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.077+0000 7f4b63fff640 1 --2- 172.21.15.43:0/1593687754 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c1789f1ef0 0x55c178a073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.077+0000 7f4b63fff640 1 -- 172.21.15.43:0/1593687754 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1789f1b10 msgr2=0x55c1789e88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:52.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.077+0000 7f4b63fff640 1 --2- 172.21.15.43:0/1593687754 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1789f1b10 0x55c1789e88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.077+0000 7f4b63fff640 1 -- 172.21.15.43:0/1593687754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c17870c7d0 con 0x55c178a12d90 2024-04-25T09:02:52.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.077+0000 7f4b63fff640 1 --2- 172.21.15.43:0/1593687754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c178a12d90 0x55c178a07910 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c00ef30 tx=0x7f4b5c00d7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d2846e549ecb0df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:52.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.077+0000 7f4b427fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:02:52.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.078+0000 7f4b427fc640 1 -- 172.21.15.43:0/1593687754 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c00dea0 con 0x55c178a12d90 2024-04-25T09:02:52.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.078+0000 7f4b427fc640 1 -- 172.21.15.43:0/1593687754 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4b5c019ad0 con 0x55c178a12d90 2024-04-25T09:02:52.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.078+0000 7f4b6cbc7740 1 -- 172.21.15.43:0/1593687754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b540159c0 con 0x55c178a12d90 2024-04-25T09:02:52.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.078+0000 7f4b427fc640 1 -- 172.21.15.43:0/1593687754 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b5c019cd0 con 0x55c178a12d90 2024-04-25T09:02:52.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.078+0000 7f4b6cbc7740 10 client.8994 fetch_fsmap learned FSMap version 69 2024-04-25T09:02:52.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.078+0000 7f4b6cbc7740 1 -- 172.21.15.43:0/1593687754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x55c1786423f0 con 0x55c178a12d90 2024-04-25T09:02:52.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.078+0000 7f4b427fc640 1 -- 172.21.15.43:0/1593687754 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b540159c0 con 0x55c178a12d90 2024-04-25T09:02:52.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.078+0000 7f4b427fc640 1 -- 172.21.15.43:0/1593687754 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4b5c020050 con 0x55c178a12d90 2024-04-25T09:02:52.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.078+0000 7f4b6cbc7740 10 client.8994 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T09:02:52.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.078+0000 7f4b6cbc7740 10 client.8994 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T09:02:52.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.078+0000 7f4b6cbc7740 20 client.8994 populate_metadata read hostname 'smithi043' 2024-04-25T09:02:52.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.078+0000 7f4b6cbc7740 1 --2- 172.21.15.43:0/1593687754 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x55c178a0c0a0 0x55c178a22700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.078+0000 7f4b6cbc7740 4 client.8994 mds_command: new command op to 8676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:02:52.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.078+0000 7f4b6cbc7740 1 -- 172.21.15.43:0/1593687754 --> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c17893b2e0 con 0x55c178a0c0a0 2024-04-25T09:02:52.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.078+0000 7f4b637fe640 1 --2- 172.21.15.43:0/1593687754 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x55c178a0c0a0 0x55c178a22700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.078+0000 7f4b427fc640 10 client.8994 ms_handle_connect on v2:172.21.15.43:6833/1586865031 2024-04-25T09:02:52.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.078+0000 7f4b637fe640 1 --2- 172.21.15.43:0/1593687754 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x55c178a0c0a0 0x55c178a22700 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3be75eceb6661595 server_cookie=78febc3c53f72358 in_seq=0 out_seq=0 2024-04-25T09:02:52.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.079+0000 7f4b427fc640 1 -- 172.21.15.43:0/1593687754 <== mds.0 v2:172.21.15.43:6833/1586865031 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c17893b2e0 con 0x55c178a0c0a0 2024-04-25T09:02:52.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.079+0000 7f4b427fc640 10 client.8994 handle_command_reply: tid=0 2024-04-25T09:02:52.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.079+0000 7f4b6cbc7740 1 client.8994 shutdown 2024-04-25T09:02:52.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.079+0000 7f4b6cbc7740 1 -- 172.21.15.43:0/1593687754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c178a12d90 msgr2=0x55c178a07910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.079+0000 7f4b6cbc7740 1 --2- 172.21.15.43:0/1593687754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c178a12d90 0x55c178a07910 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c00ef30 tx=0x7f4b5c00d7b0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.079+0000 7f4b6cbc7740 1 -- 172.21.15.43:0/1593687754 shutdown_connections 2024-04-25T09:02:52.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.079+0000 7f4b6cbc7740 1 --2- 172.21.15.43:0/1593687754 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x55c178a0c0a0 0x55c178a22700 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.079+0000 7f4b6cbc7740 1 --2- 172.21.15.43:0/1593687754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c178a12d90 0x55c178a07910 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.079+0000 7f4b6cbc7740 1 --2- 172.21.15.43:0/1593687754 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c1789f1ef0 0x55c178a073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.079+0000 7f4b427fc640 0 client.8994 ms_handle_reset on v2:172.21.15.43:6833/1586865031 2024-04-25T09:02:52.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.079+0000 7f4b6cbc7740 1 --2- 172.21.15.43:0/1593687754 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1789f1b10 0x55c1789e88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.079+0000 7f4b6cbc7740 1 -- 172.21.15.43:0/1593687754 >> 172.21.15.43:0/1593687754 conn(0x55c1789c4770 msgr2=0x55c1789f0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:52.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.079+0000 7f4b6cbc7740 1 -- 172.21.15.43:0/1593687754 shutdown_connections 2024-04-25T09:02:52.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.079+0000 7f4b6cbc7740 1 -- 172.21.15.43:0/1593687754 wait complete. 2024-04-25T09:02:52.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.079+0000 7f4b6cbc7740 20 client.8994 trim_cache size 0 max 16384 2024-04-25T09:02:52.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.094+0000 7f4b6cbc7740 1 Processor -- start 2024-04-25T09:02:52.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.094+0000 7f4b6cbc7740 1 -- start start 2024-04-25T09:02:52.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.095+0000 7f4b6cbc7740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c1789f1b10 0x55c1789c50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.095+0000 7f4b6cbc7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1789f1ef0 0x55c1789f06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.095+0000 7f4b6cbc7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c178a12d90 0x55c1789f0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.095+0000 7f4b6cbc7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b641ffc50 con 0x55c178a12d90 2024-04-25T09:02:52.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.095+0000 7f4b6cbc7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b641f4f30 con 0x55c1789f1ef0 2024-04-25T09:02:52.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.095+0000 7f4b6cbc7740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b6406de20 con 0x55c1789f1b10 2024-04-25T09:02:52.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.095+0000 7f4b63fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c178a12d90 0x55c1789f0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.095+0000 7f4b63fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c178a12d90 0x55c1789f0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48394/0 (socket says 172.21.15.43:48394) 2024-04-25T09:02:52.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.095+0000 7f4b63fff640 1 -- 172.21.15.43:0/2852369836 learned_addr learned my addr 172.21.15.43:0/2852369836 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:52.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.095+0000 7f4b63fff640 1 -- 172.21.15.43:0/2852369836 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c1789f1b10 msgr2=0x55c1789c50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:52.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.095+0000 7f4b63fff640 1 --2- 172.21.15.43:0/2852369836 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c1789f1b10 0x55c1789c50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.095+0000 7f4b63fff640 1 -- 172.21.15.43:0/2852369836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1789f1ef0 msgr2=0x55c1789f06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:52.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.095+0000 7f4b63fff640 1 --2- 172.21.15.43:0/2852369836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1789f1ef0 0x55c1789f06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.095+0000 7f4b63fff640 1 -- 172.21.15.43:0/2852369836 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1786020f0 con 0x55c178a12d90 2024-04-25T09:02:52.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.095+0000 7f4b63fff640 1 --2- 172.21.15.43:0/2852369836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c178a12d90 0x55c1789f0c10 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c0074b0 tx=0x7f4b5c00b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c301f8d6653c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:52.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.096+0000 7f4b48ff9640 1 -- 172.21.15.43:0/2852369836 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c003b90 con 0x55c178a12d90 2024-04-25T09:02:52.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.096+0000 7f4b48ff9640 1 -- 172.21.15.43:0/2852369836 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b5c003d30 con 0x55c178a12d90 2024-04-25T09:02:52.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.096+0000 7f4b48ff9640 1 -- 172.21.15.43:0/2852369836 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c0195c0 con 0x55c178a12d90 2024-04-25T09:02:52.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.096+0000 7f4b6cbc7740 1 -- 172.21.15.43:0/2852369836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c178a12d90 msgr2=0x55c1789f0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.096+0000 7f4b6cbc7740 1 --2- 172.21.15.43:0/2852369836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c178a12d90 0x55c1789f0c10 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c0074b0 tx=0x7f4b5c00b000 comp rx=0 tx=0).stop 2024-04-25T09:02:52.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.096+0000 7f4b6cbc7740 1 -- 172.21.15.43:0/2852369836 shutdown_connections 2024-04-25T09:02:52.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.096+0000 7f4b6cbc7740 1 --2- 172.21.15.43:0/2852369836 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c178a12d90 0x55c1789f0c10 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.096+0000 7f4b6cbc7740 1 --2- 172.21.15.43:0/2852369836 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1789f1ef0 0x55c1789f06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.096+0000 7f4b6cbc7740 1 --2- 172.21.15.43:0/2852369836 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c1789f1b10 0x55c1789c50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.096+0000 7f4b6cbc7740 1 -- 172.21.15.43:0/2852369836 >> 172.21.15.43:0/2852369836 conn(0x55c1789c4770 msgr2=0x55c178a120b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:52.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.096+0000 7f4b6cbc7740 1 -- 172.21.15.43:0/2852369836 shutdown_connections 2024-04-25T09:02:52.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.096+0000 7f4b6cbc7740 1 -- 172.21.15.43:0/2852369836 wait complete. 2024-04-25T09:02:52.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.097+0000 7f4b6cbc7740 1 -- start start 2024-04-25T09:02:52.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.097+0000 7f4b6cbc7740 1 Processor -- start 2024-04-25T09:02:52.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.098+0000 7f4b6cbc7740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1789f1b10 0x55c1789c57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.098+0000 7f4b6cbc7740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c1789f1ef0 0x55c178a10970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.098+0000 7f4b6cbc7740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c178a12d90 0x55c178a0f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.098+0000 7f4b6cbc7740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b641f4930 con 0x55c178a12d90 2024-04-25T09:02:52.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.098+0000 7f4b6cbc7740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1789cbed0 con 0x55c1789f1b10 2024-04-25T09:02:52.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.098+0000 7f4b6cbc7740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c178a0ff30 con 0x55c1789f1ef0 2024-04-25T09:02:52.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.098+0000 7f4b6cbc7740 10 client.0 _ll_register_callbacks cb 0x7f4b54007d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:02:52.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.098+0000 7f4b63fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c178a12d90 0x55c178a0f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.098+0000 7f4b63fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c178a12d90 0x55c178a0f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48408/0 (socket says 172.21.15.43:48408) 2024-04-25T09:02:52.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.098+0000 7f4b63fff640 1 -- 172.21.15.43:0/3090150464 learned_addr learned my addr 172.21.15.43:0/3090150464 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:52.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.098+0000 7f4b637fe640 1 --2- 172.21.15.43:0/3090150464 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1789f1b10 0x55c1789c57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.098+0000 7f4b62ffd640 1 --2- 172.21.15.43:0/3090150464 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c1789f1ef0 0x55c178a10970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.098+0000 7f4b63fff640 1 -- 172.21.15.43:0/3090150464 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c1789f1ef0 msgr2=0x55c178a10970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:52.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.098+0000 7f4b63fff640 1 --2- 172.21.15.43:0/3090150464 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c1789f1ef0 0x55c178a10970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.098+0000 7f4b63fff640 1 -- 172.21.15.43:0/3090150464 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1789f1b10 msgr2=0x55c1789c57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:52.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.098+0000 7f4b63fff640 1 --2- 172.21.15.43:0/3090150464 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1789f1b10 0x55c1789c57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.098+0000 7f4b63fff640 1 -- 172.21.15.43:0/3090150464 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c178a10eb0 con 0x55c178a12d90 2024-04-25T09:02:52.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.098+0000 7f4b63fff640 1 --2- 172.21.15.43:0/3090150464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c178a12d90 0x55c178a0f320 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c00ffd0 tx=0x7f4b5c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2367b0eecb83d80e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:52.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.098+0000 7f4b427fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:02:52.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.098+0000 7f4b427fc640 1 -- 172.21.15.43:0/3090150464 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c03a060 con 0x55c178a12d90 2024-04-25T09:02:52.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.099+0000 7f4b427fc640 1 -- 172.21.15.43:0/3090150464 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4b5c019590 con 0x55c178a12d90 2024-04-25T09:02:52.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.099+0000 7f4b6cbc7740 1 -- 172.21.15.43:0/3090150464 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b5c003d30 con 0x55c178a12d90 2024-04-25T09:02:52.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.099+0000 7f4b427fc640 1 -- 172.21.15.43:0/3090150464 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b5c040050 con 0x55c178a12d90 2024-04-25T09:02:52.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.099+0000 7f4b6cbc7740 10 client.9000 fetch_fsmap learned FSMap version 69 2024-04-25T09:02:52.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.099+0000 7f4b6cbc7740 1 -- 172.21.15.43:0/3090150464 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x55c1789c39e0 con 0x55c178a12d90 2024-04-25T09:02:52.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.099+0000 7f4b427fc640 1 -- 172.21.15.43:0/3090150464 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b5c003d30 con 0x55c178a12d90 2024-04-25T09:02:52.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.099+0000 7f4b427fc640 1 -- 172.21.15.43:0/3090150464 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4b5c043020 con 0x55c178a12d90 2024-04-25T09:02:52.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.099+0000 7f4b6cbc7740 10 client.9000 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T09:02:52.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.099+0000 7f4b6cbc7740 10 client.9000 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T09:02:52.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.099+0000 7f4b6cbc7740 20 client.9000 populate_metadata read hostname 'smithi043' 2024-04-25T09:02:52.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.099+0000 7f4b6cbc7740 1 --2- 172.21.15.43:0/3090150464 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x55c178a0f860 0x55c178a1e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.099+0000 7f4b6cbc7740 4 client.9000 mds_command: new command op to 8676 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-04-25T09:02:52.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.099+0000 7f4b6cbc7740 1 -- 172.21.15.43:0/3090150464 --> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] -- command(tid 0: {"prefix": "scrub resume"}) v1 -- 0x55c178935630 con 0x55c178a0f860 2024-04-25T09:02:52.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.099+0000 7f4b637fe640 1 --2- 172.21.15.43:0/3090150464 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x55c178a0f860 0x55c178a1e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.099+0000 7f4b427fc640 10 client.9000 ms_handle_connect on v2:172.21.15.43:6833/1586865031 2024-04-25T09:02:52.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.099+0000 7f4b637fe640 1 --2- 172.21.15.43:0/3090150464 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x55c178a0f860 0x55c178a1e110 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=597d0476d76d91ca server_cookie=8e71f3f67cca0669 in_seq=0 out_seq=0 2024-04-25T09:02:52.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.100+0000 7f4b427fc640 1 -- 172.21.15.43:0/3090150464 <== mds.0 v2:172.21.15.43:6833/1586865031 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55c178935630 con 0x55c178a0f860 2024-04-25T09:02:52.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.100+0000 7f4b427fc640 10 client.9000 handle_command_reply: tid=0 2024-04-25T09:02:52.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.100+0000 7f4b6cbc7740 1 client.9000 shutdown 2024-04-25T09:02:52.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.100+0000 7f4b6cbc7740 1 -- 172.21.15.43:0/3090150464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c178a12d90 msgr2=0x55c178a0f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.100+0000 7f4b6cbc7740 1 --2- 172.21.15.43:0/3090150464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c178a12d90 0x55c178a0f320 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c00ffd0 tx=0x7f4b5c004a40 comp rx=0 tx=0).stop 2024-04-25T09:02:52.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.100+0000 7f4b6cbc7740 1 -- 172.21.15.43:0/3090150464 shutdown_connections 2024-04-25T09:02:52.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.100+0000 7f4b6cbc7740 1 --2- 172.21.15.43:0/3090150464 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x55c178a0f860 0x55c178a1e110 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.100+0000 7f4b6cbc7740 1 --2- 172.21.15.43:0/3090150464 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55c178a12d90 0x55c178a0f320 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.100+0000 7f4b6cbc7740 1 --2- 172.21.15.43:0/3090150464 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55c1789f1ef0 0x55c178a10970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.100+0000 7f4b427fc640 0 client.9000 ms_handle_reset on v2:172.21.15.43:6833/1586865031 2024-04-25T09:02:52.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.100+0000 7f4b6cbc7740 1 --2- 172.21.15.43:0/3090150464 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55c1789f1b10 0x55c1789c57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.100+0000 7f4b6cbc7740 1 -- 172.21.15.43:0/3090150464 >> 172.21.15.43:0/3090150464 conn(0x55c1789c4770 msgr2=0x55c1789c50e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:52.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.100+0000 7f4b6cbc7740 1 -- 172.21.15.43:0/3090150464 shutdown_connections 2024-04-25T09:02:52.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.100+0000 7f4b6cbc7740 1 -- 172.21.15.43:0/3090150464 wait complete. 2024-04-25T09:02:52.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.100+0000 7f4b6cbc7740 20 client.9000 trim_cache size 0 max 16384 2024-04-25T09:02:52.094 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:02:52.094 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0 2024-04-25T09:02:52.094 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:02:52.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.100+0000 7f4b69ec4640 1 -- 172.21.15.43:0/4195383098 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4b5006b4d0 msgr2=0x7f4b5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.100+0000 7f4b69ec4640 1 --2- 172.21.15.43:0/4195383098 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4b5006b4d0 0x7f4b5006d990 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f4b54000f60 tx=0x7f4b54007c00 comp rx=0 tx=0).stop 2024-04-25T09:02:52.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.100+0000 7f4b69ec4640 1 -- 172.21.15.43:0/4195383098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b640cdfa0 msgr2=0x7f4b640ae2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.100+0000 7f4b69ec4640 1 --2- 172.21.15.43:0/4195383098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b640cdfa0 0x7f4b640ae2f0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f4b58000c00 tx=0x7f4b580046c0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.101+0000 7f4b69ec4640 1 -- 172.21.15.43:0/4195383098 shutdown_connections 2024-04-25T09:02:52.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.101+0000 7f4b69ec4640 1 --2- 172.21.15.43:0/4195383098 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4b641450a0 0x7f4b64142f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.101+0000 7f4b69ec4640 1 --2- 172.21.15.43:0/4195383098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4b640cdfa0 0x7f4b640ae2f0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.101+0000 7f4b69ec4640 1 --2- 172.21.15.43:0/4195383098 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4b5006b4d0 0x7f4b5006d990 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.101+0000 7f4b69ec4640 1 --2- 172.21.15.43:0/4195383098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b6406b240 0x7f4b64140010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.101+0000 7f4b69ec4640 1 -- 172.21.15.43:0/4195383098 >> 172.21.15.43:0/4195383098 conn(0x7f4b640b2590 msgr2=0x7f4b640b1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:52.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.101+0000 7f4b69ec4640 1 -- 172.21.15.43:0/4195383098 shutdown_connections 2024-04-25T09:02:52.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.101+0000 7f4b69ec4640 1 -- 172.21.15.43:0/4195383098 wait complete. 2024-04-25T09:02:52.100 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T09:02:52.100 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.8:0 scrub status 2024-04-25T09:02:52.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.214+0000 7fcce60f5640 1 Processor -- start 2024-04-25T09:02:52.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.214+0000 7fcce60f5640 1 -- start start 2024-04-25T09:02:52.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.215+0000 7fcce60f5640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fcce015ddd0 0x7fcce00b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.215+0000 7fcce60f5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcce006d4f0 0x7fcce006f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.215+0000 7fcce60f5640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcce006d920 con 0x7fcce006ef90 2024-04-25T09:02:52.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.215+0000 7fcce60f5640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcce00d2620 con 0x7fcce015ddd0 2024-04-25T09:02:52.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.215+0000 7fcce60f5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcce00b0b80 con 0x7fcce006d4f0 2024-04-25T09:02:52.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.215+0000 7fccdf7fe640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fcce015ddd0 0x7fcce00b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.215+0000 7fccdffff640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcce006d4f0 0x7fcce006f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.215+0000 7fccdf7fe640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fcce015ddd0 0x7fcce00b3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:40672/0 (socket says 172.21.15.43:40672) 2024-04-25T09:02:52.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.215+0000 7fccdeffd640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7fcce006ef90 0x7fcce00cc460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:54854/0 (socket says 172.21.15.43:54854) 2024-04-25T09:02:52.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.215+0000 7fccdf7fe640 1 -- 172.21.15.43:0/3404869174 learned_addr learned my addr 172.21.15.43:0/3404869174 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:52.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.215+0000 7fccde7fc640 1 -- 172.21.15.43:0/3404869174 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2262917757 0 0) 0x7fcce006d920 con 0x7fcce006ef90 2024-04-25T09:02:52.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.216+0000 7fccdffff640 1 -- 172.21.15.43:0/3404869174 >> v2:172.21.15.193:3300/0 conn(0x7fcce015ddd0 msgr2=0x7fcce00b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:52.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.216+0000 7fccdffff640 1 --2- 172.21.15.43:0/3404869174 >> v2:172.21.15.193:3300/0 conn(0x7fcce015ddd0 0x7fcce00b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.216+0000 7fccdffff640 1 -- 172.21.15.43:0/3404869174 >> v1:172.21.15.193:6789/0 conn(0x7fcce006ef90 legacy=0x7fcce00cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.216+0000 7fccdffff640 1 -- 172.21.15.43:0/3404869174 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce00b0e60 con 0x7fcce006d4f0 2024-04-25T09:02:52.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.216+0000 7fccdffff640 1 --2- 172.21.15.43:0/3404869174 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcce006d4f0 0x7fcce006f420 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fccd8009d90 tx=0x7fccd802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ba0cbf1a4d8dc7b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:52.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.216+0000 7fccde7fc640 1 -- 172.21.15.43:0/3404869174 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd8007740 con 0x7fcce006d4f0 2024-04-25T09:02:52.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.217+0000 7fccde7fc640 1 -- 172.21.15.43:0/3404869174 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccd80078e0 con 0x7fcce006d4f0 2024-04-25T09:02:52.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.217+0000 7fccde7fc640 1 -- 172.21.15.43:0/3404869174 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd8007cb0 con 0x7fcce006d4f0 2024-04-25T09:02:52.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.217+0000 7fcce60f5640 1 -- 172.21.15.43:0/3404869174 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcce006d4f0 msgr2=0x7fcce006f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.217+0000 7fcce60f5640 1 --2- 172.21.15.43:0/3404869174 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcce006d4f0 0x7fcce006f420 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fccd8009d90 tx=0x7fccd802a660 comp rx=0 tx=0).stop 2024-04-25T09:02:52.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.217+0000 7fcce60f5640 1 -- 172.21.15.43:0/3404869174 shutdown_connections 2024-04-25T09:02:52.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.217+0000 7fcce60f5640 1 --2- 172.21.15.43:0/3404869174 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcce006d4f0 0x7fcce006f420 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.217+0000 7fcce60f5640 1 --2- 172.21.15.43:0/3404869174 >> v2:172.21.15.193:3300/0 conn(0x7fcce015ddd0 0x7fcce00b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.217+0000 7fcce60f5640 1 -- 172.21.15.43:0/3404869174 >> 172.21.15.43:0/3404869174 conn(0x7fcce00d17e0 msgr2=0x7fcce00d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:52.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.217+0000 7fcce60f5640 1 -- 172.21.15.43:0/3404869174 shutdown_connections 2024-04-25T09:02:52.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.217+0000 7fcce60f5640 1 -- 172.21.15.43:0/3404869174 wait complete. 2024-04-25T09:02:52.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.218+0000 7fcce60f5640 1 Processor -- start 2024-04-25T09:02:52.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.218+0000 7fcce60f5640 1 -- start start 2024-04-25T09:02:52.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.219+0000 7fcce60f5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcce006d4f0 0x7fcce0160990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.219+0000 7fcce60f5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcce006ef90 0x7fcce00d0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.219+0000 7fcce60f5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcce015ddd0 0x7fcce00c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.219+0000 7fcce60f5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce00cccf0 con 0x7fcce015ddd0 2024-04-25T09:02:52.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.219+0000 7fcce60f5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce00ccb70 con 0x7fcce006ef90 2024-04-25T09:02:52.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.219+0000 7fcce60f5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcce00cce70 con 0x7fcce006d4f0 2024-04-25T09:02:52.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.219+0000 7fccdffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcce015ddd0 0x7fcce00c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.219+0000 7fccdffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcce015ddd0 0x7fcce00c76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48414/0 (socket says 172.21.15.43:48414) 2024-04-25T09:02:52.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.219+0000 7fccdffff640 1 -- 172.21.15.43:0/445310958 learned_addr learned my addr 172.21.15.43:0/445310958 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:52.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.220+0000 7fccdf7fe640 1 --2- 172.21.15.43:0/445310958 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcce006d4f0 0x7fcce0160990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.220+0000 7fccdeffd640 1 --2- 172.21.15.43:0/445310958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcce006ef90 0x7fcce00d0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.220+0000 7fccdffff640 1 -- 172.21.15.43:0/445310958 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcce006d4f0 msgr2=0x7fcce0160990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.220+0000 7fccdffff640 1 --2- 172.21.15.43:0/445310958 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcce006d4f0 0x7fcce0160990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.220+0000 7fccdffff640 1 -- 172.21.15.43:0/445310958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcce006ef90 msgr2=0x7fcce00d0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.220+0000 7fccdffff640 1 --2- 172.21.15.43:0/445310958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcce006ef90 0x7fcce00d0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.220+0000 7fccdffff640 1 -- 172.21.15.43:0/445310958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce00b2f90 con 0x7fcce015ddd0 2024-04-25T09:02:52.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.220+0000 7fccdffff640 1 --2- 172.21.15.43:0/445310958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcce015ddd0 0x7fcce00c76f0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fccd8009eb0 tx=0x7fccd8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:52.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.220+0000 7fccd7fff640 1 -- 172.21.15.43:0/445310958 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd8040070 con 0x7fcce015ddd0 2024-04-25T09:02:52.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.220+0000 7fccd7fff640 1 -- 172.21.15.43:0/445310958 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccd8031bc0 con 0x7fcce015ddd0 2024-04-25T09:02:52.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.220+0000 7fcce60f5640 1 -- 172.21.15.43:0/445310958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcce00ca970 con 0x7fcce015ddd0 2024-04-25T09:02:52.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.220+0000 7fccd7fff640 1 -- 172.21.15.43:0/445310958 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd8009450 con 0x7fcce015ddd0 2024-04-25T09:02:52.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.220+0000 7fcce60f5640 1 -- 172.21.15.43:0/445310958 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcce00d13e0 con 0x7fcce015ddd0 2024-04-25T09:02:52.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.222+0000 7fccd7fff640 1 -- 172.21.15.43:0/445310958 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fccd8039040 con 0x7fcce015ddd0 2024-04-25T09:02:52.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.222+0000 7fccd7fff640 1 --2- 172.21.15.43:0/445310958 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fccc006b4d0 0x7fccc006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.223+0000 7fccd7fff640 1 -- 172.21.15.43:0/445310958 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fccd80abac0 con 0x7fcce015ddd0 2024-04-25T09:02:52.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.223+0000 7fccdf7fe640 1 --2- 172.21.15.43:0/445310958 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fccc006b4d0 0x7fccc006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.223+0000 7fccdf7fe640 1 --2- 172.21.15.43:0/445310958 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fccc006b4d0 0x7fccc006d990 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fccd0009ac0 tx=0x7fccd0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:52.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.223+0000 7fcce8dfa740 1 Processor -- start 2024-04-25T09:02:52.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.223+0000 7fcce8dfa740 1 -- start start 2024-04-25T09:02:52.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.224+0000 7fcce8dfa740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f9d64c0b10 0x55f9d64c3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.224+0000 7fcce8dfa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9d64c0ef0 0x55f9d64c3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.224+0000 7fcce8dfa740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9d6493770 0x55f9d6496f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.224+0000 7fcce8dfa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d63fcda0 con 0x55f9d64c0ef0 2024-04-25T09:02:52.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.224+0000 7fcce8dfa740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d63a5b80 con 0x55f9d6493770 2024-04-25T09:02:52.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.224+0000 7fcce8dfa740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f9d63f8e40 con 0x55f9d64c0b10 2024-04-25T09:02:52.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.224+0000 7fccdeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9d64c0ef0 0x55f9d64c3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.224+0000 7fccdeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9d64c0ef0 0x55f9d64c3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48428/0 (socket says 172.21.15.43:48428) 2024-04-25T09:02:52.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.224+0000 7fccdeffd640 1 -- 172.21.15.43:0/3227720588 learned_addr learned my addr 172.21.15.43:0/3227720588 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:52.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.224+0000 7fccdf7fe640 1 --2- 172.21.15.43:0/3227720588 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f9d64c0b10 0x55f9d64c3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.224+0000 7fccdffff640 1 --2- 172.21.15.43:0/3227720588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9d6493770 0x55f9d6496f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.224+0000 7fccdeffd640 1 -- 172.21.15.43:0/3227720588 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f9d64c0b10 msgr2=0x55f9d64c3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:52.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.224+0000 7fccdeffd640 1 --2- 172.21.15.43:0/3227720588 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f9d64c0b10 0x55f9d64c3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.224+0000 7fccdeffd640 1 -- 172.21.15.43:0/3227720588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9d6493770 msgr2=0x55f9d6496f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:52.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.224+0000 7fccdeffd640 1 --2- 172.21.15.43:0/3227720588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9d6493770 0x55f9d6496f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.225+0000 7fccdeffd640 1 -- 172.21.15.43:0/3227720588 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9d63aff10 con 0x55f9d64c0ef0 2024-04-25T09:02:52.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.225+0000 7fccdeffd640 1 --2- 172.21.15.43:0/3227720588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9d64c0ef0 0x55f9d64c3860 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fccc80011e0 tx=0x7fccc8005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e5f02f5e5fb1dea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:52.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.225+0000 7fccd57fa640 1 -- 172.21.15.43:0/3227720588 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc8019070 con 0x55f9d64c0ef0 2024-04-25T09:02:52.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.225+0000 7fccd57fa640 1 -- 172.21.15.43:0/3227720588 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc8005e10 con 0x55f9d64c0ef0 2024-04-25T09:02:52.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.225+0000 7fccd57fa640 1 -- 172.21.15.43:0/3227720588 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc8001a50 con 0x55f9d64c0ef0 2024-04-25T09:02:52.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.225+0000 7fcce8dfa740 1 -- 172.21.15.43:0/3227720588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9d64c0ef0 msgr2=0x55f9d64c3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.225+0000 7fcce8dfa740 1 --2- 172.21.15.43:0/3227720588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9d64c0ef0 0x55f9d64c3860 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fccc80011e0 tx=0x7fccc8005bc0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.226+0000 7fcce8dfa740 1 -- 172.21.15.43:0/3227720588 shutdown_connections 2024-04-25T09:02:52.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.226+0000 7fcce8dfa740 1 --2- 172.21.15.43:0/3227720588 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9d6493770 0x55f9d6496f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.226+0000 7fcce8dfa740 1 --2- 172.21.15.43:0/3227720588 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9d64c0ef0 0x55f9d64c3860 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.226+0000 7fcce8dfa740 1 --2- 172.21.15.43:0/3227720588 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f9d64c0b10 0x55f9d64c3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.226+0000 7fcce8dfa740 1 -- 172.21.15.43:0/3227720588 >> 172.21.15.43:0/3227720588 conn(0x55f9d64959e0 msgr2=0x55f9d64de320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:52.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.226+0000 7fcce8dfa740 1 -- 172.21.15.43:0/3227720588 shutdown_connections 2024-04-25T09:02:52.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.226+0000 7fcce8dfa740 1 -- 172.21.15.43:0/3227720588 wait complete. 2024-04-25T09:02:52.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.226+0000 7fcce8dfa740 1 -- start start 2024-04-25T09:02:52.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.226+0000 7fcce8dfa740 1 Processor -- start 2024-04-25T09:02:52.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.227+0000 7fcce8dfa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9d64c0b10 0x55f9d64beaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.227+0000 7fcce8dfa740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f9d64c0ef0 0x55f9d64bf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.227+0000 7fcce8dfa740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9d64c4e80 0x55f9d64e0d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.227+0000 7fcce8dfa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d6498b90 con 0x55f9d64c0b10 2024-04-25T09:02:52.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.227+0000 7fcce8dfa740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d639b4b0 con 0x55f9d64c4e80 2024-04-25T09:02:52.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.227+0000 7fcce8dfa740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f9d6498e70 con 0x55f9d64c0ef0 2024-04-25T09:02:52.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.227+0000 7fcce8dfa740 10 client.0 _ll_register_callbacks cb 0x55f9d63db180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:02:52.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.227+0000 7fccdf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9d64c0b10 0x55f9d64beaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.227+0000 7fccdf7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9d64c0b10 0x55f9d64beaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48440/0 (socket says 172.21.15.43:48440) 2024-04-25T09:02:52.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.227+0000 7fccdf7fe640 1 -- 172.21.15.43:0/2733333488 learned_addr learned my addr 172.21.15.43:0/2733333488 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:52.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.227+0000 7fccdeffd640 1 --2- 172.21.15.43:0/2733333488 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f9d64c0ef0 0x55f9d64bf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.227+0000 7fccdffff640 1 --2- 172.21.15.43:0/2733333488 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9d64c4e80 0x55f9d64e0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.227+0000 7fccdf7fe640 1 -- 172.21.15.43:0/2733333488 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f9d64c0ef0 msgr2=0x55f9d64bf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:52.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.227+0000 7fccdf7fe640 1 --2- 172.21.15.43:0/2733333488 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f9d64c0ef0 0x55f9d64bf030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.227+0000 7fccdf7fe640 1 -- 172.21.15.43:0/2733333488 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9d64c4e80 msgr2=0x55f9d64e0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:52.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.227+0000 7fccdf7fe640 1 --2- 172.21.15.43:0/2733333488 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9d64c4e80 0x55f9d64e0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.227+0000 7fccdf7fe640 1 -- 172.21.15.43:0/2733333488 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f9d61db7d0 con 0x55f9d64c0b10 2024-04-25T09:02:52.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.228+0000 7fccdf7fe640 1 --2- 172.21.15.43:0/2733333488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9d64c0b10 0x55f9d64beaf0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fccd00156c0 tx=0x7fccd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31d4258476865c36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:52.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.228+0000 7fccbe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:02:52.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.228+0000 7fccbe7fc640 1 -- 172.21.15.43:0/2733333488 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd0026070 con 0x55f9d64c0b10 2024-04-25T09:02:52.222 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.228+0000 7fccbe7fc640 1 -- 172.21.15.43:0/2733333488 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fccd00147b0 con 0x55f9d64c0b10 2024-04-25T09:02:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.228+0000 7fcce8dfa740 1 -- 172.21.15.43:0/2733333488 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fccc8005e10 con 0x55f9d64c0b10 2024-04-25T09:02:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.228+0000 7fccbe7fc640 1 -- 172.21.15.43:0/2733333488 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccd0031050 con 0x55f9d64c0b10 2024-04-25T09:02:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.228+0000 7fcce8dfa740 10 client.9009 fetch_fsmap learned FSMap version 69 2024-04-25T09:02:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.228+0000 7fcce8dfa740 1 -- 172.21.15.43:0/2733333488 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x55f9d61113f0 con 0x55f9d64c0b10 2024-04-25T09:02:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.228+0000 7fccbe7fc640 1 -- 172.21.15.43:0/2733333488 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc8005e10 con 0x55f9d64c0b10 2024-04-25T09:02:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.228+0000 7fccbe7fc640 1 -- 172.21.15.43:0/2733333488 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fccd0034020 con 0x55f9d64c0b10 2024-04-25T09:02:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.228+0000 7fcce8dfa740 10 client.9009 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T09:02:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.228+0000 7fcce8dfa740 10 client.9009 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T09:02:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.228+0000 7fcce8dfa740 20 client.9009 populate_metadata read hostname 'smithi043' 2024-04-25T09:02:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.228+0000 7fcce8dfa740 1 --2- 172.21.15.43:0/2733333488 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x55f9d64e5a00 0x55f9d64e1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.228+0000 7fcce8dfa740 4 client.9009 mds_command: new command op to 8676 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:02:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.228+0000 7fcce8dfa740 1 -- 172.21.15.43:0/2733333488 --> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f9d640a2e0 con 0x55f9d64e5a00 2024-04-25T09:02:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.228+0000 7fccdeffd640 1 --2- 172.21.15.43:0/2733333488 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x55f9d64e5a00 0x55f9d64e1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.229+0000 7fccdeffd640 1 --2- 172.21.15.43:0/2733333488 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x55f9d64e5a00 0x55f9d64e1760 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f12da29011647726 server_cookie=61dba528af0323d4 in_seq=0 out_seq=0 2024-04-25T09:02:52.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.229+0000 7fccbe7fc640 10 client.9009 ms_handle_connect on v2:172.21.15.43:6833/1586865031 2024-04-25T09:02:52.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.229+0000 7fccbe7fc640 1 -- 172.21.15.43:0/2733333488 <== mds.0 v2:172.21.15.43:6833/1586865031 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f9d640a2e0 con 0x55f9d64e5a00 2024-04-25T09:02:52.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.229+0000 7fccbe7fc640 10 client.9009 handle_command_reply: tid=0 2024-04-25T09:02:52.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.229+0000 7fcce8dfa740 1 client.9009 shutdown 2024-04-25T09:02:52.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.229+0000 7fcce8dfa740 1 -- 172.21.15.43:0/2733333488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9d64c0b10 msgr2=0x55f9d64beaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.229+0000 7fcce8dfa740 1 --2- 172.21.15.43:0/2733333488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9d64c0b10 0x55f9d64beaf0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7fccd00156c0 tx=0x7fccd00046c0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.229+0000 7fcce8dfa740 1 -- 172.21.15.43:0/2733333488 shutdown_connections 2024-04-25T09:02:52.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.229+0000 7fcce8dfa740 1 --2- 172.21.15.43:0/2733333488 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x55f9d64e5a00 0x55f9d64e1760 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.229+0000 7fcce8dfa740 1 --2- 172.21.15.43:0/2733333488 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9d64c4e80 0x55f9d64e0d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.229+0000 7fccbe7fc640 0 client.9009 ms_handle_reset on v2:172.21.15.43:6833/1586865031 2024-04-25T09:02:52.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.229+0000 7fcce8dfa740 1 --2- 172.21.15.43:0/2733333488 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f9d64c0ef0 0x55f9d64bf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.229+0000 7fcce8dfa740 1 --2- 172.21.15.43:0/2733333488 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9d64c0b10 0x55f9d64beaf0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.224 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.229+0000 7fcce8dfa740 1 -- 172.21.15.43:0/2733333488 >> 172.21.15.43:0/2733333488 conn(0x55f9d6493770 msgr2=0x55f9d64947a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:52.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.229+0000 7fcce8dfa740 1 -- 172.21.15.43:0/2733333488 shutdown_connections 2024-04-25T09:02:52.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.230+0000 7fcce8dfa740 1 -- 172.21.15.43:0/2733333488 wait complete. 2024-04-25T09:02:52.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.230+0000 7fcce8dfa740 20 client.9009 trim_cache size 0 max 16384 2024-04-25T09:02:52.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.245+0000 7fcce8dfa740 1 Processor -- start 2024-04-25T09:02:52.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.245+0000 7fcce8dfa740 1 -- start start 2024-04-25T09:02:52.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.246+0000 7fcce8dfa740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9d64c0b10 0x55f9d64ffb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.246+0000 7fcce8dfa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9d64c0ef0 0x55f9d64e24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.246+0000 7fcce8dfa740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f9d64c4e80 0x55f9d64de320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.246+0000 7fcce8dfa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce01ffc80 con 0x55f9d64c0ef0 2024-04-25T09:02:52.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.246+0000 7fcce8dfa740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce00cfd70 con 0x55f9d64c0b10 2024-04-25T09:02:52.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.246+0000 7fcce8dfa740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcce00b1ec0 con 0x55f9d64c4e80 2024-04-25T09:02:52.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.246+0000 7fccdeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9d64c0ef0 0x55f9d64e24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.246+0000 7fccdeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9d64c0ef0 0x55f9d64e24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48448/0 (socket says 172.21.15.43:48448) 2024-04-25T09:02:52.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.246+0000 7fccdeffd640 1 -- 172.21.15.43:0/1285393364 learned_addr learned my addr 172.21.15.43:0/1285393364 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:52.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.246+0000 7fccdf7fe640 1 --2- 172.21.15.43:0/1285393364 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9d64c0b10 0x55f9d64ffb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.246+0000 7fccdffff640 1 --2- 172.21.15.43:0/1285393364 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f9d64c4e80 0x55f9d64de320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.246+0000 7fccdeffd640 1 -- 172.21.15.43:0/1285393364 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f9d64c4e80 msgr2=0x55f9d64de320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:52.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.246+0000 7fccdeffd640 1 --2- 172.21.15.43:0/1285393364 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f9d64c4e80 0x55f9d64de320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.246+0000 7fccdeffd640 1 -- 172.21.15.43:0/1285393364 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9d64c0b10 msgr2=0x55f9d64ffb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:52.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.246+0000 7fccdeffd640 1 --2- 172.21.15.43:0/1285393364 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9d64c0b10 0x55f9d64ffb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.246+0000 7fccdeffd640 1 -- 172.21.15.43:0/1285393364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f9d60d10f0 con 0x55f9d64c0ef0 2024-04-25T09:02:52.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.246+0000 7fccdeffd640 1 --2- 172.21.15.43:0/1285393364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9d64c0ef0 0x55f9d64e24b0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fccc8001680 tx=0x7fccc800c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e66a39b3f63f8884 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:52.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.246+0000 7fccd4ff9640 1 -- 172.21.15.43:0/1285393364 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc8019040 con 0x55f9d64c0ef0 2024-04-25T09:02:52.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.246+0000 7fccd4ff9640 1 -- 172.21.15.43:0/1285393364 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc801a040 con 0x55f9d64c0ef0 2024-04-25T09:02:52.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.246+0000 7fccd4ff9640 1 -- 172.21.15.43:0/1285393364 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc80019d0 con 0x55f9d64c0ef0 2024-04-25T09:02:52.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.247+0000 7fcce8dfa740 1 -- 172.21.15.43:0/1285393364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9d64c0ef0 msgr2=0x55f9d64e24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.247+0000 7fcce8dfa740 1 --2- 172.21.15.43:0/1285393364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9d64c0ef0 0x55f9d64e24b0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7fccc8001680 tx=0x7fccc800c000 comp rx=0 tx=0).stop 2024-04-25T09:02:52.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.247+0000 7fcce8dfa740 1 -- 172.21.15.43:0/1285393364 shutdown_connections 2024-04-25T09:02:52.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.247+0000 7fcce8dfa740 1 --2- 172.21.15.43:0/1285393364 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f9d64c4e80 0x55f9d64de320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.247+0000 7fcce8dfa740 1 --2- 172.21.15.43:0/1285393364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9d64c0ef0 0x55f9d64e24b0 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.247+0000 7fcce8dfa740 1 --2- 172.21.15.43:0/1285393364 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9d64c0b10 0x55f9d64ffb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.247+0000 7fcce8dfa740 1 -- 172.21.15.43:0/1285393364 >> 172.21.15.43:0/1285393364 conn(0x55f9d6493770 msgr2=0x55f9d64bf600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:52.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.247+0000 7fcce8dfa740 1 -- 172.21.15.43:0/1285393364 shutdown_connections 2024-04-25T09:02:52.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.247+0000 7fcce8dfa740 1 -- 172.21.15.43:0/1285393364 wait complete. 2024-04-25T09:02:52.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.248+0000 7fcce8dfa740 1 -- start start 2024-04-25T09:02:52.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.248+0000 7fcce8dfa740 1 Processor -- start 2024-04-25T09:02:52.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.248+0000 7fcce8dfa740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f9d64c0b10 0x55f9d64dc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.248+0000 7fcce8dfa740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9d64c0ef0 0x55f9d64dcec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.248+0000 7fcce8dfa740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9d64c4e80 0x55f9d64928e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.248+0000 7fcce8dfa740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9d6498370 con 0x55f9d64c0ef0 2024-04-25T09:02:52.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.248+0000 7fcce8dfa740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce00cf770 con 0x55f9d64c4e80 2024-04-25T09:02:52.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.248+0000 7fcce8dfa740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f9d64dbc40 con 0x55f9d64c0b10 2024-04-25T09:02:52.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.249+0000 7fcce8dfa740 10 client.0 _ll_register_callbacks cb 0x7fccc8007d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:02:52.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.249+0000 7fccdeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9d64c0ef0 0x55f9d64dcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.249+0000 7fccdeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9d64c0ef0 0x55f9d64dcec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48456/0 (socket says 172.21.15.43:48456) 2024-04-25T09:02:52.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.249+0000 7fccdeffd640 1 -- 172.21.15.43:0/3620145603 learned_addr learned my addr 172.21.15.43:0/3620145603 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:52.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.249+0000 7fccdf7fe640 1 --2- 172.21.15.43:0/3620145603 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f9d64c0b10 0x55f9d64dc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.249+0000 7fccdffff640 1 --2- 172.21.15.43:0/3620145603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9d64c4e80 0x55f9d64928e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.249+0000 7fccdeffd640 1 -- 172.21.15.43:0/3620145603 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f9d64c0b10 msgr2=0x55f9d64dc980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:52.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.249+0000 7fccdeffd640 1 --2- 172.21.15.43:0/3620145603 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f9d64c0b10 0x55f9d64dc980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.249+0000 7fccdeffd640 1 -- 172.21.15.43:0/3620145603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9d64c4e80 msgr2=0x55f9d64928e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:52.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.249+0000 7fccdeffd640 1 --2- 172.21.15.43:0/3620145603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9d64c4e80 0x55f9d64928e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.249+0000 7fccdeffd640 1 -- 172.21.15.43:0/3620145603 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f9d64af190 con 0x55f9d64c0ef0 2024-04-25T09:02:52.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.249+0000 7fccdeffd640 1 --2- 172.21.15.43:0/3620145603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9d64c0ef0 0x55f9d64dcec0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fccc8005c00 tx=0x7fccc8000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c7f65966dfbdb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:52.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.249+0000 7fccbe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:02:52.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.249+0000 7fccbe7fc640 1 -- 172.21.15.43:0/3620145603 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc8019040 con 0x55f9d64c0ef0 2024-04-25T09:02:52.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.249+0000 7fccbe7fc640 1 -- 172.21.15.43:0/3620145603 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fccc800f040 con 0x55f9d64c0ef0 2024-04-25T09:02:52.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.249+0000 7fcce8dfa740 1 -- 172.21.15.43:0/3620145603 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fccc801a040 con 0x55f9d64c0ef0 2024-04-25T09:02:52.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.249+0000 7fccbe7fc640 1 -- 172.21.15.43:0/3620145603 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7fccc8036050 con 0x55f9d64c0ef0 2024-04-25T09:02:52.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.249+0000 7fcce8dfa740 10 client.9015 fetch_fsmap learned FSMap version 69 2024-04-25T09:02:52.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.249+0000 7fcce8dfa740 1 -- 172.21.15.43:0/3620145603 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x55f9d64af3a0 con 0x55f9d64c0ef0 2024-04-25T09:02:52.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.250+0000 7fccbe7fc640 1 -- 172.21.15.43:0/3620145603 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc801a040 con 0x55f9d64c0ef0 2024-04-25T09:02:52.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.250+0000 7fccbe7fc640 1 -- 172.21.15.43:0/3620145603 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 69) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fccc8039020 con 0x55f9d64c0ef0 2024-04-25T09:02:52.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.250+0000 7fcce8dfa740 10 client.9015 fetch_fsmap finished waiting for FSMap version 69 2024-04-25T09:02:52.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.250+0000 7fcce8dfa740 10 client.9015 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.c 2024-04-25T09:02:52.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.250+0000 7fcce8dfa740 20 client.9015 populate_metadata read hostname 'smithi043' 2024-04-25T09:02:52.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.250+0000 7fcce8dfa740 1 --2- 172.21.15.43:0/3620145603 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x55f9d64e1ed0 0x55f9d6492e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.250+0000 7fcce8dfa740 4 client.9015 mds_command: new command op to 8676 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:02:52.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.250+0000 7fcce8dfa740 1 -- 172.21.15.43:0/3620145603 --> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f9d6404630 con 0x55f9d64e1ed0 2024-04-25T09:02:52.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.250+0000 7fccdf7fe640 1 --2- 172.21.15.43:0/3620145603 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x55f9d64e1ed0 0x55f9d6492e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.250+0000 7fccdf7fe640 1 --2- 172.21.15.43:0/3620145603 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x55f9d64e1ed0 0x55f9d6492e20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53fe7383fcab4ce9 server_cookie=c1d60d7890c7e5a6 in_seq=0 out_seq=0 2024-04-25T09:02:52.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.250+0000 7fccbe7fc640 10 client.9015 ms_handle_connect on v2:172.21.15.43:6833/1586865031 2024-04-25T09:02:52.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.250+0000 7fccbe7fc640 1 -- 172.21.15.43:0/3620145603 <== mds.0 v2:172.21.15.43:6833/1586865031 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f9d6404630 con 0x55f9d64e1ed0 2024-04-25T09:02:52.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.250+0000 7fccbe7fc640 10 client.9015 handle_command_reply: tid=0 2024-04-25T09:02:52.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.250+0000 7fcce8dfa740 1 client.9015 shutdown 2024-04-25T09:02:52.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.250+0000 7fcce8dfa740 1 -- 172.21.15.43:0/3620145603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9d64c0ef0 msgr2=0x55f9d64dcec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.250+0000 7fcce8dfa740 1 --2- 172.21.15.43:0/3620145603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9d64c0ef0 0x55f9d64dcec0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7fccc8005c00 tx=0x7fccc8000ed0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.250+0000 7fcce8dfa740 1 -- 172.21.15.43:0/3620145603 shutdown_connections 2024-04-25T09:02:52.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.250+0000 7fcce8dfa740 1 --2- 172.21.15.43:0/3620145603 >> [v2:172.21.15.43:6833/1586865031,v1:172.21.15.43:6835/1586865031] conn(0x55f9d64e1ed0 0x55f9d6492e20 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.251+0000 7fcce8dfa740 1 --2- 172.21.15.43:0/3620145603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55f9d64c4e80 0x55f9d64928e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.251+0000 7fcce8dfa740 1 --2- 172.21.15.43:0/3620145603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55f9d64c0ef0 0x55f9d64dcec0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.251+0000 7fccbe7fc640 0 client.9015 ms_handle_reset on v2:172.21.15.43:6833/1586865031 2024-04-25T09:02:52.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.251+0000 7fcce8dfa740 1 --2- 172.21.15.43:0/3620145603 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55f9d64c0b10 0x55f9d64dc980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.251+0000 7fcce8dfa740 1 -- 172.21.15.43:0/3620145603 >> 172.21.15.43:0/3620145603 conn(0x55f9d6493770 msgr2=0x55f9d64ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:52.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.251+0000 7fcce8dfa740 1 -- 172.21.15.43:0/3620145603 shutdown_connections 2024-04-25T09:02:52.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.251+0000 7fcce8dfa740 1 -- 172.21.15.43:0/3620145603 wait complete. 2024-04-25T09:02:52.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.251+0000 7fcce8dfa740 20 client.9015 trim_cache size 0 max 16384 2024-04-25T09:02:52.245 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:02:52.246 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "no active scrubs running", 2024-04-25T09:02:52.246 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": {} 2024-04-25T09:02:52.246 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:02:52.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.251+0000 7fcce60f5640 1 -- 172.21.15.43:0/445310958 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fccc006b4d0 msgr2=0x7fccc006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.251+0000 7fcce60f5640 1 --2- 172.21.15.43:0/445310958 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fccc006b4d0 0x7fccc006d990 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fccd0009ac0 tx=0x7fccd0008040 comp rx=0 tx=0).stop 2024-04-25T09:02:52.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.251+0000 7fcce60f5640 1 -- 172.21.15.43:0/445310958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcce015ddd0 msgr2=0x7fcce00c76f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.251+0000 7fcce60f5640 1 --2- 172.21.15.43:0/445310958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcce015ddd0 0x7fcce00c76f0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fccd8009eb0 tx=0x7fccd8004a40 comp rx=0 tx=0).stop 2024-04-25T09:02:52.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.251+0000 7fcce60f5640 1 -- 172.21.15.43:0/445310958 shutdown_connections 2024-04-25T09:02:52.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.251+0000 7fcce60f5640 1 --2- 172.21.15.43:0/445310958 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcce015ddd0 0x7fcce00c76f0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.251+0000 7fcce60f5640 1 --2- 172.21.15.43:0/445310958 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fccc006b4d0 0x7fccc006d990 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.251+0000 7fcce60f5640 1 --2- 172.21.15.43:0/445310958 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcce006ef90 0x7fcce00d0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.251+0000 7fcce60f5640 1 --2- 172.21.15.43:0/445310958 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcce006d4f0 0x7fcce0160990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.251+0000 7fcce60f5640 1 -- 172.21.15.43:0/445310958 >> 172.21.15.43:0/445310958 conn(0x7fcce00d17e0 msgr2=0x7fcce006f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:52.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.251+0000 7fcce60f5640 1 -- 172.21.15.43:0/445310958 shutdown_connections 2024-04-25T09:02:52.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.251+0000 7fcce60f5640 1 -- 172.21.15.43:0/445310958 wait complete. 2024-04-25T09:02:52.250 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T09:02:52.251 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:02:52.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.362+0000 7f9676e8f640 1 Processor -- start 2024-04-25T09:02:52.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.363+0000 7f9676e8f640 1 -- start start 2024-04-25T09:02:52.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.363+0000 7f9676e8f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f967015def0 0x7f967006be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.363+0000 7f9676e8f640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f96701619f0 0x7f9670161e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.363+0000 7f9676e8f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9670162390 0x7f96700cfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.363+0000 7f9676e8f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96700d0e90 con 0x7f9670162390 2024-04-25T09:02:52.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.363+0000 7f9676e8f640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f967006c390 con 0x7f96701619f0 2024-04-25T09:02:52.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.363+0000 7f9676e8f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9670162820 con 0x7f967015def0 2024-04-25T09:02:52.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.364+0000 7f967542c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9670162390 0x7f96700cfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.364+0000 7f967542c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f9670162390 0x7f96700cfa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48462/0 (socket says 172.21.15.43:48462) 2024-04-25T09:02:52.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.364+0000 7f967542c640 1 -- 172.21.15.43:0/1621834395 learned_addr learned my addr 172.21.15.43:0/1621834395 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:52.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.364+0000 7f9674c2b640 1 --2- 172.21.15.43:0/1621834395 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f967015def0 0x7f967006be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.364+0000 7f966ffff640 1 --2- 172.21.15.43:0/1621834395 >> v2:172.21.15.193:3300/0 conn(0x7f96701619f0 0x7f9670161e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.364+0000 7f967542c640 1 -- 172.21.15.43:0/1621834395 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f967015def0 msgr2=0x7f967006be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:52.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.364+0000 7f967542c640 1 --2- 172.21.15.43:0/1621834395 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f967015def0 0x7f967006be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.364+0000 7f967542c640 1 -- 172.21.15.43:0/1621834395 >> v2:172.21.15.193:3300/0 conn(0x7f96701619f0 msgr2=0x7f9670161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:52.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.364+0000 7f967542c640 1 --2- 172.21.15.43:0/1621834395 >> v2:172.21.15.193:3300/0 conn(0x7f96701619f0 0x7f9670161e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.364+0000 7f967542c640 1 -- 172.21.15.43:0/1621834395 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f967015e2d0 con 0x7f9670162390 2024-04-25T09:02:52.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.365+0000 7f967542c640 1 --2- 172.21.15.43:0/1621834395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9670162390 0x7f96700cfa60 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f9668009e50 tx=0x7f966802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca5a5a4aba559916 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:52.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.365+0000 7f966f7fe640 1 -- 172.21.15.43:0/1621834395 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9668036070 con 0x7f9670162390 2024-04-25T09:02:52.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.365+0000 7f966f7fe640 1 -- 172.21.15.43:0/1621834395 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9668031600 con 0x7f9670162390 2024-04-25T09:02:52.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.365+0000 7f966f7fe640 1 -- 172.21.15.43:0/1621834395 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96680319d0 con 0x7f9670162390 2024-04-25T09:02:52.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.365+0000 7f9676e8f640 1 -- 172.21.15.43:0/1621834395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9670162390 msgr2=0x7f96700cfa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.365+0000 7f9676e8f640 1 --2- 172.21.15.43:0/1621834395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9670162390 0x7f96700cfa60 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f9668009e50 tx=0x7f966802a6c0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.365+0000 7f9676e8f640 1 -- 172.21.15.43:0/1621834395 shutdown_connections 2024-04-25T09:02:52.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.365+0000 7f9676e8f640 1 --2- 172.21.15.43:0/1621834395 >> v2:172.21.15.193:3300/0 conn(0x7f96701619f0 0x7f9670161e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.365+0000 7f9676e8f640 1 --2- 172.21.15.43:0/1621834395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9670162390 0x7f96700cfa60 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.365+0000 7f9676e8f640 1 --2- 172.21.15.43:0/1621834395 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f967015def0 0x7f967006be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.365+0000 7f9676e8f640 1 -- 172.21.15.43:0/1621834395 >> 172.21.15.43:0/1621834395 conn(0x7f96700b32e0 msgr2=0x7f96700b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:52.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.365+0000 7f9676e8f640 1 -- 172.21.15.43:0/1621834395 shutdown_connections 2024-04-25T09:02:52.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.365+0000 7f9676e8f640 1 -- 172.21.15.43:0/1621834395 wait complete. 2024-04-25T09:02:52.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.367+0000 7f9676e8f640 1 Processor -- start 2024-04-25T09:02:52.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.367+0000 7f9676e8f640 1 -- start start 2024-04-25T09:02:52.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.367+0000 7f9676e8f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f967015def0 0x7f967015c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.367+0000 7f9676e8f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f96701619f0 0x7f9670160da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.368+0000 7f9676e8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9670162390 0x7f96701f4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.368+0000 7f9676e8f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96700b15e0 con 0x7f9670162390 2024-04-25T09:02:52.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.368+0000 7f9676e8f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96700b1300 con 0x7f967015def0 2024-04-25T09:02:52.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.368+0000 7f9676e8f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96700b18c0 con 0x7f96701619f0 2024-04-25T09:02:52.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.368+0000 7f967542c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9670162390 0x7f96701f4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.368+0000 7f967542c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9670162390 0x7f96701f4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48470/0 (socket says 172.21.15.43:48470) 2024-04-25T09:02:52.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.368+0000 7f967542c640 1 -- 172.21.15.43:0/3503446908 learned_addr learned my addr 172.21.15.43:0/3503446908 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:52.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.368+0000 7f9674c2b640 1 --2- 172.21.15.43:0/3503446908 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f967015def0 0x7f967015c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.368+0000 7f966ffff640 1 --2- 172.21.15.43:0/3503446908 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f96701619f0 0x7f9670160da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.368+0000 7f967542c640 1 -- 172.21.15.43:0/3503446908 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f96701619f0 msgr2=0x7f9670160da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.368+0000 7f967542c640 1 --2- 172.21.15.43:0/3503446908 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f96701619f0 0x7f9670160da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.368+0000 7f967542c640 1 -- 172.21.15.43:0/3503446908 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f967015def0 msgr2=0x7f967015c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.368+0000 7f967542c640 1 --2- 172.21.15.43:0/3503446908 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f967015def0 0x7f967015c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.368+0000 7f967542c640 1 -- 172.21.15.43:0/3503446908 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f967015e640 con 0x7f9670162390 2024-04-25T09:02:52.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.368+0000 7f966ffff640 1 --2- 172.21.15.43:0/3503446908 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f96701619f0 0x7f9670160da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:52.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.368+0000 7f967542c640 1 --2- 172.21.15.43:0/3503446908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9670162390 0x7f96701f4640 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f9668009f70 tx=0x7f9668004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:52.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.368+0000 7f966d7fa640 1 -- 172.21.15.43:0/3503446908 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9668041070 con 0x7f9670162390 2024-04-25T09:02:52.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.368+0000 7f966d7fa640 1 -- 172.21.15.43:0/3503446908 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f966803c650 con 0x7f9670162390 2024-04-25T09:02:52.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.368+0000 7f966d7fa640 1 -- 172.21.15.43:0/3503446908 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9668036040 con 0x7f9670162390 2024-04-25T09:02:52.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.368+0000 7f9676e8f640 1 -- 172.21.15.43:0/3503446908 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f967015e920 con 0x7f9670162390 2024-04-25T09:02:52.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.368+0000 7f9676e8f640 1 -- 172.21.15.43:0/3503446908 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96700c51f0 con 0x7f9670162390 2024-04-25T09:02:52.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.369+0000 7f9676e8f640 1 -- 172.21.15.43:0/3503446908 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9670074ef0 con 0x7f9670162390 2024-04-25T09:02:52.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.370+0000 7f966d7fa640 1 -- 172.21.15.43:0/3503446908 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9668044070 con 0x7f9670162390 2024-04-25T09:02:52.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.370+0000 7f966d7fa640 1 --2- 172.21.15.43:0/3503446908 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f965406b4d0 0x7f965406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.370+0000 7f966d7fa640 1 -- 172.21.15.43:0/3503446908 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f96680aac60 con 0x7f9670162390 2024-04-25T09:02:52.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.371+0000 7f9674c2b640 1 --2- 172.21.15.43:0/3503446908 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f965406b4d0 0x7f965406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.371+0000 7f9674c2b640 1 --2- 172.21.15.43:0/3503446908 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f965406b4d0 0x7f965406d990 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f9664006fd0 tx=0x7f9664008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:52.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.374+0000 7f966d7fa640 1 -- 172.21.15.43:0/3503446908 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9668079490 con 0x7f9670162390 2024-04-25T09:02:52.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.591+0000 7f9676e8f640 1 -- 172.21.15.43:0/3503446908 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f96700b7710 con 0x7f9670162390 2024-04-25T09:02:52.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.591+0000 7f966d7fa640 1 -- 172.21.15.43:0/3503446908 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f966807d530 con 0x7f9670162390 2024-04-25T09:02:52.580 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:52.581 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":69,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6821,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2021228703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2021228703},{"type":"v1","addr":"172.21.15.193:6837","nonce":2021228703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65},{"gid":7543,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/2398324998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2398324998},{"type":"v1","addr":"172.21.15.193:6836","nonce":2398324998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65},{"gid":8673,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2298187831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2298187831},{"type":"v1","addr":"172.21.15.43:6834","nonce":2298187831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:02:26.161487+0000","modified":"2024-04-25T09:02:29.177294+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8676":{"gid":8676,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/1586865031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1586865031},{"type":"v1","addr":"172.21.15.43:6835","nonce":1586865031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:02:52.581 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 69 2024-04-25T09:02:52.582 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.593+0000 7f9676e8f640 1 -- 172.21.15.43:0/3503446908 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f965406b4d0 msgr2=0x7f965406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.593+0000 7f9676e8f640 1 --2- 172.21.15.43:0/3503446908 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f965406b4d0 0x7f965406d990 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f9664006fd0 tx=0x7f9664008040 comp rx=0 tx=0).stop 2024-04-25T09:02:52.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.593+0000 7f9676e8f640 1 -- 172.21.15.43:0/3503446908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9670162390 msgr2=0x7f96701f4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.593+0000 7f9676e8f640 1 --2- 172.21.15.43:0/3503446908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9670162390 0x7f96701f4640 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f9668009f70 tx=0x7f9668004a40 comp rx=0 tx=0).stop 2024-04-25T09:02:52.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.593+0000 7f9676e8f640 1 -- 172.21.15.43:0/3503446908 shutdown_connections 2024-04-25T09:02:52.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.593+0000 7f9676e8f640 1 --2- 172.21.15.43:0/3503446908 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f9670162390 0x7f96701f4640 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.593+0000 7f9676e8f640 1 --2- 172.21.15.43:0/3503446908 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f96701619f0 0x7f9670160da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.593+0000 7f9676e8f640 1 --2- 172.21.15.43:0/3503446908 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f965406b4d0 0x7f965406d990 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.593+0000 7f9676e8f640 1 --2- 172.21.15.43:0/3503446908 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f967015def0 0x7f967015c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.593+0000 7f9676e8f640 1 -- 172.21.15.43:0/3503446908 >> 172.21.15.43:0/3503446908 conn(0x7f96700b32e0 msgr2=0x7f967006fac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:52.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.593+0000 7f9676e8f640 1 -- 172.21.15.43:0/3503446908 shutdown_connections 2024-04-25T09:02:52.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.593+0000 7f9676e8f640 1 -- 172.21.15.43:0/3503446908 wait complete. 2024-04-25T09:02:52.593 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T09:02:52.593 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:02:52.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.706+0000 7efcaf1ce640 1 Processor -- start 2024-04-25T09:02:52.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.706+0000 7efcaf1ce640 1 -- start start 2024-04-25T09:02:52.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.707+0000 7efcaf1ce640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efca80715a0 0x7efca8071a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.707+0000 7efcaf1ce640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7efca80b3100 0x7efca80b3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.707+0000 7efcaf1ce640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efca806dd30 con 0x7efca815a250 2024-04-25T09:02:52.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.707+0000 7efcaf1ce640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efca8071f40 con 0x7efca80b3100 2024-04-25T09:02:52.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.707+0000 7efcaf1ce640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efca806e010 con 0x7efca80715a0 2024-04-25T09:02:52.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.707+0000 7efcacf6a640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7efca815a250 0x7efca8070d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:53412/0 (socket says 172.21.15.43:53412) 2024-04-25T09:02:52.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.707+0000 7efcacf6a640 1 -- 172.21.15.43:0/669119757 learned_addr learned my addr 172.21.15.43:0/669119757 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:52.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.707+0000 7efca7fff640 1 --2- 172.21.15.43:0/669119757 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efca80715a0 0x7efca8071a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.707+0000 7efcad76b640 1 --2- 172.21.15.43:0/669119757 >> v2:172.21.15.193:3300/0 conn(0x7efca80b3100 0x7efca80b3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.708+0000 7efca77fe640 1 -- 172.21.15.43:0/669119757 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1767701844 0 0) 0x7efca806dd30 con 0x7efca815a250 2024-04-25T09:02:52.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.708+0000 7efca77fe640 1 -- 172.21.15.43:0/669119757 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc94003520 con 0x7efca815a250 2024-04-25T09:02:52.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.708+0000 7efca7fff640 1 -- 172.21.15.43:0/669119757 >> v2:172.21.15.193:3300/0 conn(0x7efca80b3100 msgr2=0x7efca80b3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:52.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.708+0000 7efca7fff640 1 --2- 172.21.15.43:0/669119757 >> v2:172.21.15.193:3300/0 conn(0x7efca80b3100 0x7efca80b3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.708+0000 7efca7fff640 1 -- 172.21.15.43:0/669119757 >> v1:172.21.15.43:6789/0 conn(0x7efca815a250 legacy=0x7efca8070d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.708+0000 7efca7fff640 1 -- 172.21.15.43:0/669119757 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca80b2900 con 0x7efca80715a0 2024-04-25T09:02:52.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.709+0000 7efca7fff640 1 --2- 172.21.15.43:0/669119757 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efca80715a0 0x7efca8071a00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efc98009d90 tx=0x7efc9802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4747315708d517dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:52.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.709+0000 7efca77fe640 1 -- 172.21.15.43:0/669119757 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc98007740 con 0x7efca80715a0 2024-04-25T09:02:52.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.709+0000 7efca77fe640 1 -- 172.21.15.43:0/669119757 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc980078e0 con 0x7efca80715a0 2024-04-25T09:02:52.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.709+0000 7efca77fe640 1 -- 172.21.15.43:0/669119757 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc98007c90 con 0x7efca80715a0 2024-04-25T09:02:52.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.709+0000 7efcaf1ce640 1 -- 172.21.15.43:0/669119757 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efca80715a0 msgr2=0x7efca8071a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.709+0000 7efcaf1ce640 1 --2- 172.21.15.43:0/669119757 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efca80715a0 0x7efca8071a00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7efc98009d90 tx=0x7efc9802a660 comp rx=0 tx=0).stop 2024-04-25T09:02:52.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.709+0000 7efcaf1ce640 1 -- 172.21.15.43:0/669119757 shutdown_connections 2024-04-25T09:02:52.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.709+0000 7efcaf1ce640 1 --2- 172.21.15.43:0/669119757 >> v2:172.21.15.193:3300/0 conn(0x7efca80b3100 0x7efca80b3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.709+0000 7efcaf1ce640 1 --2- 172.21.15.43:0/669119757 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efca80715a0 0x7efca8071a00 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.709+0000 7efcaf1ce640 1 -- 172.21.15.43:0/669119757 >> 172.21.15.43:0/669119757 conn(0x7efca80b1480 msgr2=0x7efca80b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:52.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.709+0000 7efcaf1ce640 1 -- 172.21.15.43:0/669119757 shutdown_connections 2024-04-25T09:02:52.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.709+0000 7efcaf1ce640 1 -- 172.21.15.43:0/669119757 wait complete. 2024-04-25T09:02:52.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.711+0000 7efcaf1ce640 1 Processor -- start 2024-04-25T09:02:52.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.711+0000 7efcaf1ce640 1 -- start start 2024-04-25T09:02:52.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.711+0000 7efcaf1ce640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efca80715a0 0x7efca8156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.711+0000 7efcaf1ce640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efca80b3100 0x7efca815c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.711+0000 7efcaf1ce640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efca815a250 0x7efca8146be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.711+0000 7efcaf1ce640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca80af020 con 0x7efca80b3100 2024-04-25T09:02:52.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.711+0000 7efcaf1ce640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efca80aeea0 con 0x7efca80715a0 2024-04-25T09:02:52.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.711+0000 7efcaf1ce640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efca806d290 con 0x7efca815a250 2024-04-25T09:02:52.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.712+0000 7efca7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efca80b3100 0x7efca815c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.712+0000 7efca7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efca80b3100 0x7efca815c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48472/0 (socket says 172.21.15.43:48472) 2024-04-25T09:02:52.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.712+0000 7efca7fff640 1 -- 172.21.15.43:0/4287607508 learned_addr learned my addr 172.21.15.43:0/4287607508 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:52.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.712+0000 7efcad76b640 1 --2- 172.21.15.43:0/4287607508 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efca815a250 0x7efca8146be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.712+0000 7efcacf6a640 1 --2- 172.21.15.43:0/4287607508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efca80715a0 0x7efca8156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.712+0000 7efca7fff640 1 -- 172.21.15.43:0/4287607508 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efca815a250 msgr2=0x7efca8146be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.712+0000 7efca7fff640 1 --2- 172.21.15.43:0/4287607508 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efca815a250 0x7efca8146be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.712+0000 7efca7fff640 1 -- 172.21.15.43:0/4287607508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efca80715a0 msgr2=0x7efca8156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.712+0000 7efca7fff640 1 --2- 172.21.15.43:0/4287607508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efca80715a0 0x7efca8156440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.712+0000 7efca7fff640 1 -- 172.21.15.43:0/4287607508 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca815cf90 con 0x7efca80b3100 2024-04-25T09:02:52.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.712+0000 7efca7fff640 1 --2- 172.21.15.43:0/4287607508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efca80b3100 0x7efca815c4d0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7efc9802ac50 tx=0x7efc98004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:52.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.712+0000 7efca57fa640 1 -- 172.21.15.43:0/4287607508 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc98040070 con 0x7efca80b3100 2024-04-25T09:02:52.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.712+0000 7efca57fa640 1 -- 172.21.15.43:0/4287607508 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc98031c40 con 0x7efca80b3100 2024-04-25T09:02:52.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.712+0000 7efcaf1ce640 1 -- 172.21.15.43:0/4287607508 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efca815d270 con 0x7efca80b3100 2024-04-25T09:02:52.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.712+0000 7efca57fa640 1 -- 172.21.15.43:0/4287607508 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc98009450 con 0x7efca80b3100 2024-04-25T09:02:52.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.712+0000 7efcaf1ce640 1 -- 172.21.15.43:0/4287607508 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efca814f3b0 con 0x7efca80b3100 2024-04-25T09:02:52.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.713+0000 7efcaf1ce640 1 -- 172.21.15.43:0/4287607508 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efca8074ef0 con 0x7efca80b3100 2024-04-25T09:02:52.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.714+0000 7efca57fa640 1 -- 172.21.15.43:0/4287607508 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efc98039040 con 0x7efca80b3100 2024-04-25T09:02:52.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.714+0000 7efca57fa640 1 --2- 172.21.15.43:0/4287607508 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7efc8806b4d0 0x7efc8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:52.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.714+0000 7efca57fa640 1 -- 172.21.15.43:0/4287607508 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efc980ab8a0 con 0x7efca80b3100 2024-04-25T09:02:52.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.715+0000 7efcacf6a640 1 --2- 172.21.15.43:0/4287607508 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7efc8806b4d0 0x7efc8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:52.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.715+0000 7efcacf6a640 1 --2- 172.21.15.43:0/4287607508 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7efc8806b4d0 0x7efc8806d990 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7efc90005880 tx=0x7efc90009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:52.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.719+0000 7efca57fa640 1 -- 172.21.15.43:0/4287607508 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efc9807a0c0 con 0x7efca80b3100 2024-04-25T09:02:52.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.968+0000 7efcaf1ce640 1 -- 172.21.15.43:0/4287607508 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7efca80ae8e0 con 0x7efca80b3100 2024-04-25T09:02:52.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.969+0000 7efca57fa640 1 -- 172.21.15.43:0/4287607508 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7efc9807e160 con 0x7efca80b3100 2024-04-25T09:02:52.958 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:52.958 INFO:teuthology.orchestra.run.smithi043.stdout:{"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":545,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":113,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035238,"num_in_osds":8,"osd_in_since":1714035229,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1077670390,"bytes_used":2669371392,"bytes_avail":770424741888,"bytes_total":773094113280,"write_bytes_sec":54673090,"read_op_per_sec":0,"write_op_per_sec":54},"fsmap":{"epoch":69,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8676}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":21,"modified":"2024-04-25T09:02:49.814157+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:02:52.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.971+0000 7efcaf1ce640 1 -- 172.21.15.43:0/4287607508 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7efc8806b4d0 msgr2=0x7efc8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.971+0000 7efcaf1ce640 1 --2- 172.21.15.43:0/4287607508 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7efc8806b4d0 0x7efc8806d990 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7efc90005880 tx=0x7efc90009040 comp rx=0 tx=0).stop 2024-04-25T09:02:52.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.971+0000 7efcaf1ce640 1 -- 172.21.15.43:0/4287607508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efca80b3100 msgr2=0x7efca815c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:52.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.971+0000 7efcaf1ce640 1 --2- 172.21.15.43:0/4287607508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efca80b3100 0x7efca815c4d0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7efc9802ac50 tx=0x7efc98004a40 comp rx=0 tx=0).stop 2024-04-25T09:02:52.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.971+0000 7efcaf1ce640 1 -- 172.21.15.43:0/4287607508 shutdown_connections 2024-04-25T09:02:52.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.971+0000 7efcaf1ce640 1 --2- 172.21.15.43:0/4287607508 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efca815a250 0x7efca8146be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.971+0000 7efcaf1ce640 1 --2- 172.21.15.43:0/4287607508 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efca80b3100 0x7efca815c4d0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.971+0000 7efcaf1ce640 1 --2- 172.21.15.43:0/4287607508 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7efc8806b4d0 0x7efc8806d990 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.971+0000 7efcaf1ce640 1 --2- 172.21.15.43:0/4287607508 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efca80715a0 0x7efca8156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:52.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.971+0000 7efcaf1ce640 1 -- 172.21.15.43:0/4287607508 >> 172.21.15.43:0/4287607508 conn(0x7efca80b1480 msgr2=0x7efca80cd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:52.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.971+0000 7efcaf1ce640 1 -- 172.21.15.43:0/4287607508 shutdown_connections 2024-04-25T09:02:52.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:52.971+0000 7efcaf1ce640 1 -- 172.21.15.43:0/4287607508 wait complete. 2024-04-25T09:02:52.970 INFO:tasks.cephfs.filesystem.ceph_manager:{'c': 'paused'} 2024-04-25T09:02:53.972 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:02:54.086 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.098+0000 7f1a4e79e640 1 Processor -- start 2024-04-25T09:02:54.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.098+0000 7f1a4e79e640 1 -- start start 2024-04-25T09:02:54.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.099+0000 7f1a4e79e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1a48161010 0x7f1a480b2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.099+0000 7f1a4e79e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1a480d0920 0x7f1a480d1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.099+0000 7f1a4e79e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a480ce680 con 0x7f1a480ce200 2024-04-25T09:02:54.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.099+0000 7f1a4e79e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a481626a0 con 0x7f1a48161010 2024-04-25T09:02:54.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.099+0000 7f1a4e79e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a481623c0 con 0x7f1a480d0920 2024-04-25T09:02:54.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.099+0000 7f1a4d79c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1a48161010 0x7f1a480b2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.099+0000 7f1a4cf9b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f1a480ce200 0x7f1a480d0210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:53418/0 (socket says 172.21.15.43:53418) 2024-04-25T09:02:54.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.099+0000 7f1a4cf9b640 1 -- 172.21.15.43:0/1632732449 learned_addr learned my addr 172.21.15.43:0/1632732449 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:54.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.099+0000 7f1a4df9d640 1 --2- 172.21.15.43:0/1632732449 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1a480d0920 0x7f1a480d1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.099+0000 7f1a4d79c640 1 -- 172.21.15.43:0/1632732449 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1a480d0920 msgr2=0x7f1a480d1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:54.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.099+0000 7f1a4d79c640 1 --2- 172.21.15.43:0/1632732449 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1a480d0920 0x7f1a480d1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.099+0000 7f1a4d79c640 1 -- 172.21.15.43:0/1632732449 >> v1:172.21.15.43:6789/0 conn(0x7f1a480ce200 legacy=0x7f1a480d0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.099+0000 7f1a4d79c640 1 -- 172.21.15.43:0/1632732449 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a480ce980 con 0x7f1a48161010 2024-04-25T09:02:54.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.100+0000 7f1a4d79c640 1 --2- 172.21.15.43:0/1632732449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a48161010 0x7f1a480b2d90 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f1a440097a0 tx=0x7f1a4402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f9cdb9ae7e0b26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:54.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.100+0000 7f1a3ffff640 1 -- 172.21.15.43:0/1632732449 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a44037070 con 0x7f1a48161010 2024-04-25T09:02:54.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.100+0000 7f1a3ffff640 1 -- 172.21.15.43:0/1632732449 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a44032440 con 0x7f1a48161010 2024-04-25T09:02:54.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.100+0000 7f1a3ffff640 1 -- 172.21.15.43:0/1632732449 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a44032810 con 0x7f1a48161010 2024-04-25T09:02:54.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.101+0000 7f1a4e79e640 1 -- 172.21.15.43:0/1632732449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a48161010 msgr2=0x7f1a480b2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.101+0000 7f1a4e79e640 1 --2- 172.21.15.43:0/1632732449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a48161010 0x7f1a480b2d90 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f1a440097a0 tx=0x7f1a4402a180 comp rx=0 tx=0).stop 2024-04-25T09:02:54.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.101+0000 7f1a4e79e640 1 -- 172.21.15.43:0/1632732449 shutdown_connections 2024-04-25T09:02:54.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.101+0000 7f1a4e79e640 1 --2- 172.21.15.43:0/1632732449 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1a480d0920 0x7f1a480d1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.101+0000 7f1a4e79e640 1 --2- 172.21.15.43:0/1632732449 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a48161010 0x7f1a480b2d90 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.101+0000 7f1a4e79e640 1 -- 172.21.15.43:0/1632732449 >> 172.21.15.43:0/1632732449 conn(0x7f1a4806b000 msgr2=0x7f1a4806b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:54.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.101+0000 7f1a4e79e640 1 -- 172.21.15.43:0/1632732449 shutdown_connections 2024-04-25T09:02:54.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.101+0000 7f1a4e79e640 1 -- 172.21.15.43:0/1632732449 wait complete. 2024-04-25T09:02:54.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.102+0000 7f1a4e79e640 1 Processor -- start 2024-04-25T09:02:54.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.102+0000 7f1a4e79e640 1 -- start start 2024-04-25T09:02:54.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.103+0000 7f1a4e79e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a480ce200 0x7f1a480b54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.103+0000 7f1a4e79e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1a480d0920 0x7f1a4815b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.103+0000 7f1a4e79e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a48161010 0x7f1a4815bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.103+0000 7f1a4e79e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a48161570 con 0x7f1a480ce200 2024-04-25T09:02:54.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.103+0000 7f1a4e79e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a481613f0 con 0x7f1a48161010 2024-04-25T09:02:54.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.103+0000 7f1a4e79e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a480d2140 con 0x7f1a480d0920 2024-04-25T09:02:54.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.103+0000 7f1a4d79c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a480ce200 0x7f1a480b54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.103+0000 7f1a4d79c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a480ce200 0x7f1a480b54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48484/0 (socket says 172.21.15.43:48484) 2024-04-25T09:02:54.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.103+0000 7f1a4d79c640 1 -- 172.21.15.43:0/4119119941 learned_addr learned my addr 172.21.15.43:0/4119119941 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:54.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.103+0000 7f1a4cf9b640 1 --2- 172.21.15.43:0/4119119941 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1a480d0920 0x7f1a4815b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.103+0000 7f1a4df9d640 1 --2- 172.21.15.43:0/4119119941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a48161010 0x7f1a4815bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.103+0000 7f1a4d79c640 1 -- 172.21.15.43:0/4119119941 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1a480d0920 msgr2=0x7f1a4815b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.103+0000 7f1a4d79c640 1 --2- 172.21.15.43:0/4119119941 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1a480d0920 0x7f1a4815b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.103+0000 7f1a4d79c640 1 -- 172.21.15.43:0/4119119941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a48161010 msgr2=0x7f1a4815bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.103+0000 7f1a4d79c640 1 --2- 172.21.15.43:0/4119119941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a48161010 0x7f1a4815bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.103+0000 7f1a4d79c640 1 -- 172.21.15.43:0/4119119941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a480b2d90 con 0x7f1a480ce200 2024-04-25T09:02:54.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.103+0000 7f1a4d79c640 1 --2- 172.21.15.43:0/4119119941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a480ce200 0x7f1a480b54c0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f1a440098c0 tx=0x7f1a440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:54.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.104+0000 7f1a3dffb640 1 -- 172.21.15.43:0/4119119941 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a44037040 con 0x7f1a480ce200 2024-04-25T09:02:54.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.104+0000 7f1a3dffb640 1 -- 172.21.15.43:0/4119119941 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a44030770 con 0x7f1a480ce200 2024-04-25T09:02:54.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.104+0000 7f1a4e79e640 1 -- 172.21.15.43:0/4119119941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a480b3070 con 0x7f1a480ce200 2024-04-25T09:02:54.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.104+0000 7f1a3dffb640 1 -- 172.21.15.43:0/4119119941 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a44030b40 con 0x7f1a480ce200 2024-04-25T09:02:54.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.104+0000 7f1a4e79e640 1 -- 172.21.15.43:0/4119119941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a480b5a00 con 0x7f1a480ce200 2024-04-25T09:02:54.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.105+0000 7f1a4e79e640 1 -- 172.21.15.43:0/4119119941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a480680f0 con 0x7f1a480ce200 2024-04-25T09:02:54.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.105+0000 7f1a3dffb640 1 -- 172.21.15.43:0/4119119941 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1a44044020 con 0x7f1a480ce200 2024-04-25T09:02:54.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.106+0000 7f1a3dffb640 1 --2- 172.21.15.43:0/4119119941 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1a2c06b4d0 0x7f1a2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.106+0000 7f1a3dffb640 1 -- 172.21.15.43:0/4119119941 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f1a440aa7c0 con 0x7f1a480ce200 2024-04-25T09:02:54.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.106+0000 7f1a4cf9b640 1 --2- 172.21.15.43:0/4119119941 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1a2c06b4d0 0x7f1a2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.107+0000 7f1a4cf9b640 1 --2- 172.21.15.43:0/4119119941 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1a2c06b4d0 0x7f1a2c06d990 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f1a480d1ee0 tx=0x7f1a38009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:54.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.110+0000 7f1a3dffb640 1 -- 172.21.15.43:0/4119119941 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1a44032e20 con 0x7f1a480ce200 2024-04-25T09:02:54.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.330+0000 7f1a4e79e640 1 -- 172.21.15.43:0/4119119941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a480cfec0 con 0x7f1a480ce200 2024-04-25T09:02:54.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.330+0000 7f1a3dffb640 1 -- 172.21.15.43:0/4119119941 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f1a480cfec0 con 0x7f1a480ce200 2024-04-25T09:02:54.319 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:54.319 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":69,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6821,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2021228703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2021228703},{"type":"v1","addr":"172.21.15.193:6837","nonce":2021228703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65},{"gid":7543,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/2398324998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2398324998},{"type":"v1","addr":"172.21.15.193:6836","nonce":2398324998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65},{"gid":8673,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2298187831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2298187831},{"type":"v1","addr":"172.21.15.43:6834","nonce":2298187831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:02:26.161487+0000","modified":"2024-04-25T09:02:29.177294+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8676":{"gid":8676,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/1586865031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1586865031},{"type":"v1","addr":"172.21.15.43:6835","nonce":1586865031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:02:54.319 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 69 2024-04-25T09:02:54.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.332+0000 7f1a4e79e640 1 -- 172.21.15.43:0/4119119941 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1a2c06b4d0 msgr2=0x7f1a2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.332+0000 7f1a4e79e640 1 --2- 172.21.15.43:0/4119119941 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1a2c06b4d0 0x7f1a2c06d990 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f1a480d1ee0 tx=0x7f1a38009040 comp rx=0 tx=0).stop 2024-04-25T09:02:54.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.332+0000 7f1a4e79e640 1 -- 172.21.15.43:0/4119119941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a480ce200 msgr2=0x7f1a480b54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.332+0000 7f1a4e79e640 1 --2- 172.21.15.43:0/4119119941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a480ce200 0x7f1a480b54c0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f1a440098c0 tx=0x7f1a440046c0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.332+0000 7f1a4e79e640 1 -- 172.21.15.43:0/4119119941 shutdown_connections 2024-04-25T09:02:54.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.332+0000 7f1a4e79e640 1 --2- 172.21.15.43:0/4119119941 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1a2c06b4d0 0x7f1a2c06d990 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.332+0000 7f1a4e79e640 1 --2- 172.21.15.43:0/4119119941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a48161010 0x7f1a4815bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.332+0000 7f1a4e79e640 1 --2- 172.21.15.43:0/4119119941 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1a480d0920 0x7f1a4815b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.332+0000 7f1a4e79e640 1 --2- 172.21.15.43:0/4119119941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a480ce200 0x7f1a480b54c0 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.332+0000 7f1a4e79e640 1 -- 172.21.15.43:0/4119119941 >> 172.21.15.43:0/4119119941 conn(0x7f1a4806b000 msgr2=0x7f1a480b8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:54.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.332+0000 7f1a4e79e640 1 -- 172.21.15.43:0/4119119941 shutdown_connections 2024-04-25T09:02:54.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.332+0000 7f1a4e79e640 1 -- 172.21.15.43:0/4119119941 wait complete. 2024-04-25T09:02:54.332 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T09:02:54.333 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:02:54.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.444+0000 7fb83b4ee640 1 Processor -- start 2024-04-25T09:02:54.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.444+0000 7fb83b4ee640 1 -- start start 2024-04-25T09:02:54.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.445+0000 7fb83b4ee640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb83415a9d0 0x7fb83406d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.445+0000 7fb83b4ee640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb83406d7d0 0x7fb834071a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.445+0000 7fb83b4ee640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb834072420 con 0x7fb834071f90 2024-04-25T09:02:54.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.445+0000 7fb83b4ee640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb83406dc30 con 0x7fb83406d7d0 2024-04-25T09:02:54.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.445+0000 7fb83b4ee640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8340704f0 con 0x7fb83415a9d0 2024-04-25T09:02:54.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.445+0000 7fb839a8b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb834071f90 0x7fb8340b3990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:53428/0 (socket says 172.21.15.43:53428) 2024-04-25T09:02:54.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.445+0000 7fb839a8b640 1 -- 172.21.15.43:0/1429914297 learned_addr learned my addr 172.21.15.43:0/1429914297 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:54.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.445+0000 7fb83928a640 1 --2- 172.21.15.43:0/1429914297 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb83415a9d0 0x7fb83406d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.445+0000 7fb838a89640 1 --2- 172.21.15.43:0/1429914297 >> v2:172.21.15.193:3300/0 conn(0x7fb83406d7d0 0x7fb834071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.446+0000 7fb833fff640 1 -- 172.21.15.43:0/1429914297 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3856495683 0 0) 0x7fb824001110 con 0x7fb834071f90 2024-04-25T09:02:54.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.446+0000 7fb833fff640 1 -- 172.21.15.43:0/1429914297 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb81c003520 con 0x7fb834071f90 2024-04-25T09:02:54.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.446+0000 7fb833fff640 1 -- 172.21.15.43:0/1429914297 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 422032252 0 0) 0x7fb81c003520 con 0x7fb834071f90 2024-04-25T09:02:54.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.446+0000 7fb833fff640 1 -- 172.21.15.43:0/1429914297 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb83415a9d0 msgr2=0x7fb83406d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:54.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.446+0000 7fb833fff640 1 --2- 172.21.15.43:0/1429914297 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb83415a9d0 0x7fb83406d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.446+0000 7fb833fff640 1 -- 172.21.15.43:0/1429914297 >> v2:172.21.15.193:3300/0 conn(0x7fb83406d7d0 msgr2=0x7fb834071a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:54.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.446+0000 7fb833fff640 1 --2- 172.21.15.43:0/1429914297 >> v2:172.21.15.193:3300/0 conn(0x7fb83406d7d0 0x7fb834071a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.446+0000 7fb833fff640 1 -- 172.21.15.43:0/1429914297 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8340707d0 con 0x7fb834071f90 2024-04-25T09:02:54.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.446+0000 7fb833fff640 1 -- 172.21.15.43:0/1429914297 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7fb824001840 con 0x7fb834071f90 2024-04-25T09:02:54.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.446+0000 7fb833fff640 1 -- 172.21.15.43:0/1429914297 >> v1:172.21.15.43:6789/0 conn(0x7fb834071f90 legacy=0x7fb8340b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.446+0000 7fb833fff640 1 --2- 172.21.15.43:0/1429914297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb81c0039b0 0x7fb81c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.446+0000 7fb833fff640 1 --2- 172.21.15.43:0/1429914297 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb81c004800 0x7fb81c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.446+0000 7fb833fff640 1 --2- 172.21.15.43:0/1429914297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb81c0051c0 0x7fb81c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.446+0000 7fb833fff640 1 -- 172.21.15.43:0/1429914297 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8340704f0 con 0x7fb81c0039b0 2024-04-25T09:02:54.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.446+0000 7fb833fff640 1 -- 172.21.15.43:0/1429914297 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c00bf00 con 0x7fb81c0051c0 2024-04-25T09:02:54.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.446+0000 7fb833fff640 1 -- 172.21.15.43:0/1429914297 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb81c00c080 con 0x7fb81c004800 2024-04-25T09:02:54.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.446+0000 7fb83928a640 1 --2- 172.21.15.43:0/1429914297 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb83415a9d0 0x7fb83406d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:02:54.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.446+0000 7fb838a89640 1 --2- 172.21.15.43:0/1429914297 >> v2:172.21.15.193:3300/0 conn(0x7fb83406d7d0 0x7fb834071a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:02:54.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.446+0000 7fb83928a640 1 --2- 172.21.15.43:0/1429914297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb81c0039b0 0x7fb81c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.446+0000 7fb839a8b640 1 --2- 172.21.15.43:0/1429914297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb81c0051c0 0x7fb81c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.446+0000 7fb838a89640 1 --2- 172.21.15.43:0/1429914297 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb81c004800 0x7fb81c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.446+0000 7fb83928a640 1 -- 172.21.15.43:0/1429914297 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb81c004800 msgr2=0x7fb81c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:54.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.446+0000 7fb83928a640 1 --2- 172.21.15.43:0/1429914297 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb81c004800 0x7fb81c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.446+0000 7fb83928a640 1 -- 172.21.15.43:0/1429914297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb81c0051c0 msgr2=0x7fb81c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:54.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.446+0000 7fb83928a640 1 --2- 172.21.15.43:0/1429914297 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb81c0051c0 0x7fb81c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.446+0000 7fb83928a640 1 -- 172.21.15.43:0/1429914297 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb81c003520 con 0x7fb81c0039b0 2024-04-25T09:02:54.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.447+0000 7fb83928a640 1 --2- 172.21.15.43:0/1429914297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb81c0039b0 0x7fb81c003dd0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fb82c00e1f0 tx=0x7fb81c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d3f6b39c835c78c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:54.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.447+0000 7fb833fff640 1 -- 172.21.15.43:0/1429914297 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82c009d40 con 0x7fb81c0039b0 2024-04-25T09:02:54.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.447+0000 7fb833fff640 1 -- 172.21.15.43:0/1429914297 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb82c035070 con 0x7fb81c0039b0 2024-04-25T09:02:54.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.447+0000 7fb833fff640 1 -- 172.21.15.43:0/1429914297 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82c004030 con 0x7fb81c0039b0 2024-04-25T09:02:54.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.448+0000 7fb83b4ee640 1 -- 172.21.15.43:0/1429914297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb81c0039b0 msgr2=0x7fb81c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.448+0000 7fb83b4ee640 1 --2- 172.21.15.43:0/1429914297 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb81c0039b0 0x7fb81c003dd0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fb82c00e1f0 tx=0x7fb81c005830 comp rx=0 tx=0).stop 2024-04-25T09:02:54.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.448+0000 7fb839a8b640 1 -- 172.21.15.43:0/1429914297 reap_dead start 2024-04-25T09:02:54.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.448+0000 7fb83b4ee640 1 -- 172.21.15.43:0/1429914297 shutdown_connections 2024-04-25T09:02:54.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.448+0000 7fb83b4ee640 1 -- 172.21.15.43:0/1429914297 >> 172.21.15.43:0/1429914297 conn(0x7fb8340b1480 msgr2=0x7fb8340b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:54.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.448+0000 7fb83b4ee640 1 -- 172.21.15.43:0/1429914297 shutdown_connections 2024-04-25T09:02:54.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.448+0000 7fb83b4ee640 1 -- 172.21.15.43:0/1429914297 wait complete. 2024-04-25T09:02:54.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.449+0000 7fb83b4ee640 1 Processor -- start 2024-04-25T09:02:54.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.449+0000 7fb83b4ee640 1 -- start start 2024-04-25T09:02:54.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.450+0000 7fb83b4ee640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb81c0039b0 0x7fb8340aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.450+0000 7fb83b4ee640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb8340acfe0 0x7fb8340ad460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.450+0000 7fb83b4ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8340a94d0 0x7fb8340a9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.450+0000 7fb83b4ee640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8340712b0 con 0x7fb8340a94d0 2024-04-25T09:02:54.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.450+0000 7fb83b4ee640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb834070d90 con 0x7fb81c0039b0 2024-04-25T09:02:54.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.450+0000 7fb83b4ee640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8340ce950 con 0x7fb8340acfe0 2024-04-25T09:02:54.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.450+0000 7fb839a8b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8340a94d0 0x7fb8340a9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.450+0000 7fb839a8b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8340a94d0 0x7fb8340a9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48502/0 (socket says 172.21.15.43:48502) 2024-04-25T09:02:54.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.450+0000 7fb839a8b640 1 -- 172.21.15.43:0/3333432843 learned_addr learned my addr 172.21.15.43:0/3333432843 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:54.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.450+0000 7fb83928a640 1 --2- 172.21.15.43:0/3333432843 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb81c0039b0 0x7fb8340aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.450+0000 7fb838a89640 1 --2- 172.21.15.43:0/3333432843 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb8340acfe0 0x7fb8340ad460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.450+0000 7fb839a8b640 1 -- 172.21.15.43:0/3333432843 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb8340acfe0 msgr2=0x7fb8340ad460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.450+0000 7fb839a8b640 1 --2- 172.21.15.43:0/3333432843 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb8340acfe0 0x7fb8340ad460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.450+0000 7fb839a8b640 1 -- 172.21.15.43:0/3333432843 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb81c0039b0 msgr2=0x7fb8340aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.450+0000 7fb839a8b640 1 --2- 172.21.15.43:0/3333432843 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb81c0039b0 0x7fb8340aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.450+0000 7fb839a8b640 1 -- 172.21.15.43:0/3333432843 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb834155cd0 con 0x7fb8340a94d0 2024-04-25T09:02:54.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.450+0000 7fb839a8b640 1 --2- 172.21.15.43:0/3333432843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8340a94d0 0x7fb8340a9950 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fb824000c00 tx=0x7fb824004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:54.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.451+0000 7fb831ffb640 1 -- 172.21.15.43:0/3333432843 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb82401c070 con 0x7fb8340a94d0 2024-04-25T09:02:54.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.451+0000 7fb831ffb640 1 -- 172.21.15.43:0/3333432843 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb824011b30 con 0x7fb8340a94d0 2024-04-25T09:02:54.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.451+0000 7fb83b4ee640 1 -- 172.21.15.43:0/3333432843 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8340adcc0 con 0x7fb8340a94d0 2024-04-25T09:02:54.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.451+0000 7fb831ffb640 1 -- 172.21.15.43:0/3333432843 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb824010040 con 0x7fb8340a94d0 2024-04-25T09:02:54.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.451+0000 7fb83b4ee640 1 -- 172.21.15.43:0/3333432843 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8340ae280 con 0x7fb8340a94d0 2024-04-25T09:02:54.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.452+0000 7fb83b4ee640 1 -- 172.21.15.43:0/3333432843 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8340ccc80 con 0x7fb8340a94d0 2024-04-25T09:02:54.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.452+0000 7fb831ffb640 1 -- 172.21.15.43:0/3333432843 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb82401f070 con 0x7fb8340a94d0 2024-04-25T09:02:54.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.453+0000 7fb831ffb640 1 --2- 172.21.15.43:0/3333432843 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb82006b4d0 0x7fb82006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.453+0000 7fb831ffb640 1 -- 172.21.15.43:0/3333432843 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb82408bdd0 con 0x7fb8340a94d0 2024-04-25T09:02:54.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.453+0000 7fb83928a640 1 --2- 172.21.15.43:0/3333432843 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb82006b4d0 0x7fb82006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.454+0000 7fb83928a640 1 --2- 172.21.15.43:0/3333432843 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb82006b4d0 0x7fb82006d990 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fb82c034000 tx=0x7fb82c02f000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:54.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.457+0000 7fb831ffb640 1 -- 172.21.15.43:0/3333432843 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb8340ccc80 con 0x7fb8340a94d0 2024-04-25T09:02:54.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.710+0000 7fb83b4ee640 1 -- 172.21.15.43:0/3333432843 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fb834074ef0 con 0x7fb8340a94d0 2024-04-25T09:02:54.699 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.711+0000 7fb831ffb640 1 -- 172.21.15.43:0/3333432843 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1575 (secure 0 0 0) 0x7fb82408d640 con 0x7fb8340a94d0 2024-04-25T09:02:54.700 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:54.700 INFO:teuthology.orchestra.run.smithi043.stdout:{"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":547,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":114,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035238,"num_in_osds":8,"osd_in_since":1714035229,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1078423686,"bytes_used":2670878720,"bytes_avail":770423234560,"bytes_total":773094113280,"write_bytes_sec":54736617,"read_op_per_sec":0,"write_op_per_sec":55},"fsmap":{"epoch":69,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"c","status":"up:active","gid":8676}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":22,"modified":"2024-04-25T09:02:53.815110+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:02:54.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.713+0000 7fb83b4ee640 1 -- 172.21.15.43:0/3333432843 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb82006b4d0 msgr2=0x7fb82006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.713+0000 7fb83b4ee640 1 --2- 172.21.15.43:0/3333432843 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb82006b4d0 0x7fb82006d990 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fb82c034000 tx=0x7fb82c02f000 comp rx=0 tx=0).stop 2024-04-25T09:02:54.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.713+0000 7fb83b4ee640 1 -- 172.21.15.43:0/3333432843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8340a94d0 msgr2=0x7fb8340a9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.713+0000 7fb83b4ee640 1 --2- 172.21.15.43:0/3333432843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8340a94d0 0x7fb8340a9950 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fb824000c00 tx=0x7fb824004c70 comp rx=0 tx=0).stop 2024-04-25T09:02:54.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.713+0000 7fb83b4ee640 1 -- 172.21.15.43:0/3333432843 shutdown_connections 2024-04-25T09:02:54.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.713+0000 7fb83b4ee640 1 --2- 172.21.15.43:0/3333432843 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8340a94d0 0x7fb8340a9950 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.713+0000 7fb83b4ee640 1 --2- 172.21.15.43:0/3333432843 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb8340acfe0 0x7fb8340ad460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.713+0000 7fb83b4ee640 1 --2- 172.21.15.43:0/3333432843 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb82006b4d0 0x7fb82006d990 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.713+0000 7fb83b4ee640 1 --2- 172.21.15.43:0/3333432843 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb81c0039b0 0x7fb8340aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.713+0000 7fb83b4ee640 1 -- 172.21.15.43:0/3333432843 >> 172.21.15.43:0/3333432843 conn(0x7fb8340b1480 msgr2=0x7fb83415d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:54.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.713+0000 7fb83b4ee640 1 -- 172.21.15.43:0/3333432843 shutdown_connections 2024-04-25T09:02:54.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.713+0000 7fb83b4ee640 1 -- 172.21.15.43:0/3333432843 wait complete. 2024-04-25T09:02:54.712 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T09:02:54.713 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:02:54.713 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:02:54.754 DEBUG:teuthology.orchestra.run.smithi193:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:02:54.797 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:02:54.797 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T09:02:54.797 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:02:54.813 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:02:54.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.977+0000 7f5c040b5640 1 Processor -- start 2024-04-25T09:02:54.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.978+0000 7f5c040b5640 1 -- start start 2024-04-25T09:02:54.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.978+0000 7f5c040b5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5bfc15df50 0x7f5bfc1619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.978+0000 7f5c040b5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5bfc161f30 0x7f5bfc162370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.978+0000 7f5c040b5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bfc0d24a0 con 0x7f5bfc0c7cf0 2024-04-25T09:02:54.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.978+0000 7f5c040b5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bfc15e330 con 0x7f5bfc161f30 2024-04-25T09:02:54.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.978+0000 7f5c040b5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bfc0d2740 con 0x7f5bfc15df50 2024-04-25T09:02:54.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.978+0000 7f5c01650640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5bfc161f30 0x7f5bfc162370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.978+0000 7f5c02652640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f5bfc0c7cf0 0x7f5bfc0c8100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:53436/0 (socket says 172.21.15.43:53436) 2024-04-25T09:02:54.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.979+0000 7f5c02652640 1 -- 172.21.15.43:0/492260286 learned_addr learned my addr 172.21.15.43:0/492260286 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:54.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.979+0000 7f5c01650640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5bfc161f30 0x7f5bfc162370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48516/0 (socket says 172.21.15.43:48516) 2024-04-25T09:02:54.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.979+0000 7f5c01650640 1 -- 172.21.15.43:0/492260286 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5bfc15df50 msgr2=0x7f5bfc1619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:54.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.979+0000 7f5c01e51640 1 --2- 172.21.15.43:0/492260286 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5bfc15df50 0x7f5bfc1619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.979+0000 7f5c01650640 1 --2- 172.21.15.43:0/492260286 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5bfc15df50 0x7f5bfc1619f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.979+0000 7f5c01650640 1 -- 172.21.15.43:0/492260286 >> v1:172.21.15.43:6789/0 conn(0x7f5bfc0c7cf0 legacy=0x7f5bfc0c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.979+0000 7f5c00e4f640 1 -- 172.21.15.43:0/492260286 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2491926439 0 0) 0x7f5bfc0d24a0 con 0x7f5bfc0c7cf0 2024-04-25T09:02:54.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.979+0000 7f5c01650640 1 -- 172.21.15.43:0/492260286 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bfc0d2a20 con 0x7f5bfc161f30 2024-04-25T09:02:54.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.979+0000 7f5c01e51640 1 --2- 172.21.15.43:0/492260286 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5bfc15df50 0x7f5bfc1619f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:54.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.980+0000 7f5c01650640 1 --2- 172.21.15.43:0/492260286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bfc161f30 0x7f5bfc162370 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f5bf00097a0 tx=0x7f5bf002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=790888925a5dc03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:54.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.980+0000 7f5c00e4f640 1 -- 172.21.15.43:0/492260286 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf0037070 con 0x7f5bfc161f30 2024-04-25T09:02:54.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.980+0000 7f5c00e4f640 1 -- 172.21.15.43:0/492260286 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bf0032440 con 0x7f5bfc161f30 2024-04-25T09:02:54.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.980+0000 7f5c00e4f640 1 -- 172.21.15.43:0/492260286 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf0032810 con 0x7f5bfc161f30 2024-04-25T09:02:54.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.980+0000 7f5c040b5640 1 -- 172.21.15.43:0/492260286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bfc161f30 msgr2=0x7f5bfc162370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.980+0000 7f5c040b5640 1 --2- 172.21.15.43:0/492260286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bfc161f30 0x7f5bfc162370 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f5bf00097a0 tx=0x7f5bf002a180 comp rx=0 tx=0).stop 2024-04-25T09:02:54.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.980+0000 7f5c040b5640 1 -- 172.21.15.43:0/492260286 shutdown_connections 2024-04-25T09:02:54.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.980+0000 7f5c040b5640 1 --2- 172.21.15.43:0/492260286 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bfc161f30 0x7f5bfc162370 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.980+0000 7f5c040b5640 1 --2- 172.21.15.43:0/492260286 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5bfc15df50 0x7f5bfc1619f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.980+0000 7f5c040b5640 1 -- 172.21.15.43:0/492260286 >> 172.21.15.43:0/492260286 conn(0x7f5bfc0d3a30 msgr2=0x7f5bfc0d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:54.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.980+0000 7f5c040b5640 1 -- 172.21.15.43:0/492260286 shutdown_connections 2024-04-25T09:02:54.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.980+0000 7f5c040b5640 1 -- 172.21.15.43:0/492260286 wait complete. 2024-04-25T09:02:54.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.981+0000 7f5c040b5640 1 Processor -- start 2024-04-25T09:02:54.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.982+0000 7f5c040b5640 1 -- start start 2024-04-25T09:02:54.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.982+0000 7f5c040b5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5bfc0c7cf0 0x7f5bfc1f4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.982+0000 7f5c040b5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5bfc15df50 0x7f5bfc1535a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.982+0000 7f5c040b5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bfc161f30 0x7f5bfc0cc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.982+0000 7f5c040b5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bfc0d2190 con 0x7f5bfc161f30 2024-04-25T09:02:54.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.982+0000 7f5c040b5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bfc0c77d0 con 0x7f5bfc15df50 2024-04-25T09:02:54.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.982+0000 7f5c040b5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bfc0cf670 con 0x7f5bfc0c7cf0 2024-04-25T09:02:54.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.982+0000 7f5c02652640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bfc161f30 0x7f5bfc0cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.982+0000 7f5c02652640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bfc161f30 0x7f5bfc0cc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48528/0 (socket says 172.21.15.43:48528) 2024-04-25T09:02:54.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.982+0000 7f5c02652640 1 -- 172.21.15.43:0/1576189089 learned_addr learned my addr 172.21.15.43:0/1576189089 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:54.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.983+0000 7f5c01e51640 1 --2- 172.21.15.43:0/1576189089 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5bfc0c7cf0 0x7f5bfc1f4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.983+0000 7f5c01650640 1 --2- 172.21.15.43:0/1576189089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5bfc15df50 0x7f5bfc1535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.983+0000 7f5c02652640 1 -- 172.21.15.43:0/1576189089 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5bfc0c7cf0 msgr2=0x7f5bfc1f4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.983+0000 7f5c02652640 1 --2- 172.21.15.43:0/1576189089 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5bfc0c7cf0 0x7f5bfc1f4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.983+0000 7f5c02652640 1 -- 172.21.15.43:0/1576189089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5bfc15df50 msgr2=0x7f5bfc1535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:54.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.983+0000 7f5c02652640 1 --2- 172.21.15.43:0/1576189089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5bfc15df50 0x7f5bfc1535a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:54.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.983+0000 7f5c02652640 1 -- 172.21.15.43:0/1576189089 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bfc1f4a60 con 0x7f5bfc161f30 2024-04-25T09:02:54.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.983+0000 7f5c01650640 1 --2- 172.21.15.43:0/1576189089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5bfc15df50 0x7f5bfc1535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:54.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.983+0000 7f5c02652640 1 --2- 172.21.15.43:0/1576189089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bfc161f30 0x7f5bfc0cc910 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f5bec00c7a0 tx=0x7f5bec00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:54.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.983+0000 7f5bfa7fc640 1 -- 172.21.15.43:0/1576189089 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bec018070 con 0x7f5bfc161f30 2024-04-25T09:02:54.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.983+0000 7f5bfa7fc640 1 -- 172.21.15.43:0/1576189089 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bec00a640 con 0x7f5bfc161f30 2024-04-25T09:02:54.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.983+0000 7f5c040b5640 1 -- 172.21.15.43:0/1576189089 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bfc157470 con 0x7f5bfc161f30 2024-04-25T09:02:54.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.983+0000 7f5bfa7fc640 1 -- 172.21.15.43:0/1576189089 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bec00aab0 con 0x7f5bfc161f30 2024-04-25T09:02:54.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.983+0000 7f5c040b5640 1 -- 172.21.15.43:0/1576189089 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bfc0cb1f0 con 0x7f5bfc161f30 2024-04-25T09:02:54.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.984+0000 7f5c040b5640 1 -- 172.21.15.43:0/1576189089 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bfc0680f0 con 0x7f5bfc161f30 2024-04-25T09:02:54.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.985+0000 7f5bfa7fc640 1 -- 172.21.15.43:0/1576189089 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5bec01b070 con 0x7f5bfc161f30 2024-04-25T09:02:54.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.985+0000 7f5bfa7fc640 1 --2- 172.21.15.43:0/1576189089 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5be806b4d0 0x7f5be806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:54.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.985+0000 7f5bfa7fc640 1 -- 172.21.15.43:0/1576189089 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f5bec089860 con 0x7f5bfc161f30 2024-04-25T09:02:54.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.985+0000 7f5c01e51640 1 --2- 172.21.15.43:0/1576189089 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5be806b4d0 0x7f5be806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:54.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.986+0000 7f5c01e51640 1 --2- 172.21.15.43:0/1576189089 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5be806b4d0 0x7f5be806d990 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f5bf400a990 tx=0x7f5bf4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:54.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:54.989+0000 7f5bfa7fc640 1 -- 172.21.15.43:0/1576189089 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5bfc1f4a60 con 0x7f5bfc161f30 2024-04-25T09:02:55.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.177+0000 7f5c040b5640 1 -- 172.21.15.43:0/1576189089 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5bfc074ef0 con 0x7f5bfc161f30 2024-04-25T09:02:55.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.178+0000 7f5bfa7fc640 1 -- 172.21.15.43:0/1576189089 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) v1 ==== 74+0+16744 (secure 0 0 0) 0x7f5bec0586a0 con 0x7f5bfc161f30 2024-04-25T09:02:55.167 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:55.167 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":115,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:02:54.872758+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:02:24.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T09:02:25.143723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.e","mappings":[{"from":0,"to":5}]},{"pgid":"17.3f","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:02:55.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.180+0000 7f5c040b5640 1 -- 172.21.15.43:0/1576189089 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5be806b4d0 msgr2=0x7f5be806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:55.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.180+0000 7f5c040b5640 1 --2- 172.21.15.43:0/1576189089 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5be806b4d0 0x7f5be806d990 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f5bf400a990 tx=0x7f5bf4008040 comp rx=0 tx=0).stop 2024-04-25T09:02:55.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.180+0000 7f5c040b5640 1 -- 172.21.15.43:0/1576189089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bfc161f30 msgr2=0x7f5bfc0cc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:55.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.180+0000 7f5c040b5640 1 --2- 172.21.15.43:0/1576189089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bfc161f30 0x7f5bfc0cc910 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f5bec00c7a0 tx=0x7f5bec00cc60 comp rx=0 tx=0).stop 2024-04-25T09:02:55.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.180+0000 7f5c040b5640 1 -- 172.21.15.43:0/1576189089 shutdown_connections 2024-04-25T09:02:55.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.180+0000 7f5c040b5640 1 --2- 172.21.15.43:0/1576189089 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bfc161f30 0x7f5bfc0cc910 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.180+0000 7f5c040b5640 1 --2- 172.21.15.43:0/1576189089 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5be806b4d0 0x7f5be806d990 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.180+0000 7f5c040b5640 1 --2- 172.21.15.43:0/1576189089 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5bfc15df50 0x7f5bfc1535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.180+0000 7f5c040b5640 1 --2- 172.21.15.43:0/1576189089 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5bfc0c7cf0 0x7f5bfc1f4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.180+0000 7f5c040b5640 1 -- 172.21.15.43:0/1576189089 >> 172.21.15.43:0/1576189089 conn(0x7f5bfc0d3a30 msgr2=0x7f5bfc0ce620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:55.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.180+0000 7f5c040b5640 1 -- 172.21.15.43:0/1576189089 shutdown_connections 2024-04-25T09:02:55.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.180+0000 7f5c040b5640 1 -- 172.21.15.43:0/1576189089 wait complete. 2024-04-25T09:02:55.180 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:02:24.446721+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:02:25.143723+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:02:55.181 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:02:55.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.296+0000 7ff3ba87c640 1 Processor -- start 2024-04-25T09:02:55.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.296+0000 7ff3ba87c640 1 -- start start 2024-04-25T09:02:55.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.297+0000 7ff3ba87c640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff3b40d04a0 0x7ff3b415ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.297+0000 7ff3ba87c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff3b415dec0 0x7ff3b40d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.297+0000 7ff3ba87c640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b40d0dc0 con 0x7ff3b40ce8c0 2024-04-25T09:02:55.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.297+0000 7ff3ba87c640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3b40cf110 con 0x7ff3b40d04a0 2024-04-25T09:02:55.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.297+0000 7ff3ba87c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b40cf3f0 con 0x7ff3b415dec0 2024-04-25T09:02:55.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.297+0000 7ff3b8e19640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff3b40ce8c0 0x7ff3b41617a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:53450/0 (socket says 172.21.15.43:53450) 2024-04-25T09:02:55.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.297+0000 7ff3b8e19640 1 -- 172.21.15.43:0/3233885560 learned_addr learned my addr 172.21.15.43:0/3233885560 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:55.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.297+0000 7ff3b37fe640 1 --2- 172.21.15.43:0/3233885560 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff3b415dec0 0x7ff3b40d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.297+0000 7ff3b3fff640 1 --2- 172.21.15.43:0/3233885560 >> v2:172.21.15.193:3300/0 conn(0x7ff3b40d04a0 0x7ff3b415ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.297+0000 7ff3b2ffd640 1 -- 172.21.15.43:0/3233885560 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3250617632 0 0) 0x7ff3b40d0dc0 con 0x7ff3b40ce8c0 2024-04-25T09:02:55.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.297+0000 7ff3b2ffd640 1 -- 172.21.15.43:0/3233885560 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff39c003520 con 0x7ff3b40ce8c0 2024-04-25T09:02:55.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.298+0000 7ff3b2ffd640 1 -- 172.21.15.43:0/3233885560 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1709580501 0 0) 0x7ff39c003520 con 0x7ff3b40ce8c0 2024-04-25T09:02:55.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.298+0000 7ff3b2ffd640 1 -- 172.21.15.43:0/3233885560 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff3b415dec0 msgr2=0x7ff3b40d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:55.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.298+0000 7ff3b2ffd640 1 --2- 172.21.15.43:0/3233885560 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff3b415dec0 0x7ff3b40d0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.298+0000 7ff3b2ffd640 1 -- 172.21.15.43:0/3233885560 >> v2:172.21.15.193:3300/0 conn(0x7ff3b40d04a0 msgr2=0x7ff3b415ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:55.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.298+0000 7ff3b2ffd640 1 --2- 172.21.15.43:0/3233885560 >> v2:172.21.15.193:3300/0 conn(0x7ff3b40d04a0 0x7ff3b415ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.298+0000 7ff3b2ffd640 1 -- 172.21.15.43:0/3233885560 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b40cf6d0 con 0x7ff3b40ce8c0 2024-04-25T09:02:55.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.298+0000 7ff3b2ffd640 1 -- 172.21.15.43:0/3233885560 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7ff3ac0019f0 con 0x7ff3b40ce8c0 2024-04-25T09:02:55.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.298+0000 7ff3b2ffd640 1 -- 172.21.15.43:0/3233885560 >> v1:172.21.15.43:6789/0 conn(0x7ff3b40ce8c0 legacy=0x7ff3b41617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:55.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.298+0000 7ff3b2ffd640 1 --2- 172.21.15.43:0/3233885560 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff3b415dec0 0x7ff39c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.298+0000 7ff3b2ffd640 1 --2- 172.21.15.43:0/3233885560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff39c003f30 0x7ff39c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.298+0000 7ff3b2ffd640 1 --2- 172.21.15.43:0/3233885560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff39c004d40 0x7ff39c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.298+0000 7ff3b2ffd640 1 -- 172.21.15.43:0/3233885560 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b40cf110 con 0x7ff39c003f30 2024-04-25T09:02:55.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.298+0000 7ff3b2ffd640 1 -- 172.21.15.43:0/3233885560 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b40cf3f0 con 0x7ff39c004d40 2024-04-25T09:02:55.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.298+0000 7ff3b2ffd640 1 -- 172.21.15.43:0/3233885560 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff39c00b9c0 con 0x7ff3b415dec0 2024-04-25T09:02:55.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.298+0000 7ff3b37fe640 1 --2- 172.21.15.43:0/3233885560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff39c003f30 0x7ff39c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.298+0000 7ff3b3fff640 1 --2- 172.21.15.43:0/3233885560 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff3b415dec0 0x7ff39c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.298+0000 7ff3b8e19640 1 --2- 172.21.15.43:0/3233885560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff39c004d40 0x7ff39c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.298+0000 7ff3b37fe640 1 -- 172.21.15.43:0/3233885560 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff3b415dec0 msgr2=0x7ff39c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:55.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.298+0000 7ff3b37fe640 1 --2- 172.21.15.43:0/3233885560 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff3b415dec0 0x7ff39c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.298+0000 7ff3b37fe640 1 -- 172.21.15.43:0/3233885560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff39c004d40 msgr2=0x7ff39c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:55.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.298+0000 7ff3b37fe640 1 --2- 172.21.15.43:0/3233885560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff39c004d40 0x7ff39c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.298+0000 7ff3b37fe640 1 -- 172.21.15.43:0/3233885560 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff39c003520 con 0x7ff39c003f30 2024-04-25T09:02:55.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.298+0000 7ff3b8e19640 1 --2- 172.21.15.43:0/3233885560 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff39c004d40 0x7ff39c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:55.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.299+0000 7ff3b37fe640 1 --2- 172.21.15.43:0/3233885560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff39c003f30 0x7ff39c004800 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009a90 tx=0x7ff39c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a06901bd9a412b02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:55.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.299+0000 7ff3b2ffd640 1 -- 172.21.15.43:0/3233885560 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4036070 con 0x7ff39c003f30 2024-04-25T09:02:55.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.299+0000 7ff3b2ffd640 1 -- 172.21.15.43:0/3233885560 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a4031400 con 0x7ff39c003f30 2024-04-25T09:02:55.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.299+0000 7ff3b2ffd640 1 -- 172.21.15.43:0/3233885560 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4009690 con 0x7ff39c003f30 2024-04-25T09:02:55.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.300+0000 7ff3ba87c640 1 -- 172.21.15.43:0/3233885560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff39c003f30 msgr2=0x7ff39c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:55.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.300+0000 7ff3ba87c640 1 --2- 172.21.15.43:0/3233885560 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff39c003f30 0x7ff39c004800 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009a90 tx=0x7ff39c005830 comp rx=0 tx=0).stop 2024-04-25T09:02:55.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.300+0000 7ff3b8e19640 1 -- 172.21.15.43:0/3233885560 reap_dead start 2024-04-25T09:02:55.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.300+0000 7ff3ba87c640 1 -- 172.21.15.43:0/3233885560 shutdown_connections 2024-04-25T09:02:55.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.300+0000 7ff3ba87c640 1 -- 172.21.15.43:0/3233885560 >> 172.21.15.43:0/3233885560 conn(0x7ff3b409cd40 msgr2=0x7ff3b40d3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:55.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.300+0000 7ff3ba87c640 1 -- 172.21.15.43:0/3233885560 shutdown_connections 2024-04-25T09:02:55.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.300+0000 7ff3ba87c640 1 -- 172.21.15.43:0/3233885560 wait complete. 2024-04-25T09:02:55.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.301+0000 7ff3ba87c640 1 Processor -- start 2024-04-25T09:02:55.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.301+0000 7ff3ba87c640 1 -- start start 2024-04-25T09:02:55.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.302+0000 7ff3ba87c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff39c003f30 0x7ff3b41f47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.302+0000 7ff3ba87c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff3b4159250 0x7ff3b406c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.302+0000 7ff3ba87c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3b406c6c0 0x7ff3b40b7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.302+0000 7ff3ba87c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b40cecf0 con 0x7ff3b406c6c0 2024-04-25T09:02:55.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.302+0000 7ff3ba87c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b40cfc90 con 0x7ff39c003f30 2024-04-25T09:02:55.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.302+0000 7ff3ba87c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b40cee70 con 0x7ff3b4159250 2024-04-25T09:02:55.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.302+0000 7ff3b8e19640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3b406c6c0 0x7ff3b40b7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.302+0000 7ff3b8e19640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3b406c6c0 0x7ff3b40b7010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48560/0 (socket says 172.21.15.43:48560) 2024-04-25T09:02:55.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.302+0000 7ff3b8e19640 1 -- 172.21.15.43:0/1528369238 learned_addr learned my addr 172.21.15.43:0/1528369238 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:55.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.302+0000 7ff3b3fff640 1 --2- 172.21.15.43:0/1528369238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff39c003f30 0x7ff3b41f47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.302+0000 7ff3b37fe640 1 --2- 172.21.15.43:0/1528369238 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff3b4159250 0x7ff3b406c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.302+0000 7ff3b8e19640 1 -- 172.21.15.43:0/1528369238 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff3b4159250 msgr2=0x7ff3b406c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:55.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.302+0000 7ff3b8e19640 1 --2- 172.21.15.43:0/1528369238 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff3b4159250 0x7ff3b406c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.302+0000 7ff3b8e19640 1 -- 172.21.15.43:0/1528369238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff39c003f30 msgr2=0x7ff3b41f47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:55.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.302+0000 7ff3b8e19640 1 --2- 172.21.15.43:0/1528369238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff39c003f30 0x7ff3b41f47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.302+0000 7ff3b8e19640 1 -- 172.21.15.43:0/1528369238 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b40b7d80 con 0x7ff3b406c6c0 2024-04-25T09:02:55.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.302+0000 7ff3b8e19640 1 --2- 172.21.15.43:0/1528369238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3b406c6c0 0x7ff3b40b7010 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac000f80 tx=0x7ff3ac00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:55.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.302+0000 7ff3b0ff9640 1 -- 172.21.15.43:0/1528369238 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac007a90 con 0x7ff3b406c6c0 2024-04-25T09:02:55.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.302+0000 7ff3b0ff9640 1 -- 172.21.15.43:0/1528369238 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3ac018070 con 0x7ff3b406c6c0 2024-04-25T09:02:55.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.302+0000 7ff3ba87c640 1 -- 172.21.15.43:0/1528369238 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3b40cc1f0 con 0x7ff3b406c6c0 2024-04-25T09:02:55.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.303+0000 7ff3b0ff9640 1 -- 172.21.15.43:0/1528369238 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac013420 con 0x7ff3b406c6c0 2024-04-25T09:02:55.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.303+0000 7ff3ba87c640 1 -- 172.21.15.43:0/1528369238 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3b40b8060 con 0x7ff3b406c6c0 2024-04-25T09:02:55.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.304+0000 7ff3ba87c640 1 -- 172.21.15.43:0/1528369238 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3b4074f60 con 0x7ff3b406c6c0 2024-04-25T09:02:55.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.304+0000 7ff3b0ff9640 1 -- 172.21.15.43:0/1528369238 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff3ac0135c0 con 0x7ff3b406c6c0 2024-04-25T09:02:55.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.304+0000 7ff3b0ff9640 1 --2- 172.21.15.43:0/1528369238 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff3a006b500 0x7ff3a006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.305+0000 7ff3b0ff9640 1 -- 172.21.15.43:0/1528369238 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff3ac08a5f0 con 0x7ff3b406c6c0 2024-04-25T09:02:55.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.305+0000 7ff3b3fff640 1 --2- 172.21.15.43:0/1528369238 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff3a006b500 0x7ff3a006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.305+0000 7ff3b3fff640 1 --2- 172.21.15.43:0/1528369238 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff3a006b500 0x7ff3a006d9c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff3a8009b80 tx=0x7ff3a8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:55.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.309+0000 7ff3b0ff9640 1 -- 172.21.15.43:0/1528369238 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff3ac0594c0 con 0x7ff3b406c6c0 2024-04-25T09:02:55.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.487+0000 7ff3ba87c640 1 -- 172.21.15.43:0/1528369238 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff3b408daf0 con 0x7ff3b406c6c0 2024-04-25T09:02:55.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.488+0000 7ff3b0ff9640 1 -- 172.21.15.43:0/1528369238 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v115) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff3ac058dc0 con 0x7ff3b406c6c0 2024-04-25T09:02:55.477 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T09:02:55.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.490+0000 7ff3ba87c640 1 -- 172.21.15.43:0/1528369238 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff3a006b500 msgr2=0x7ff3a006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:55.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.490+0000 7ff3ba87c640 1 --2- 172.21.15.43:0/1528369238 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff3a006b500 0x7ff3a006d9c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff3a8009b80 tx=0x7ff3a8008040 comp rx=0 tx=0).stop 2024-04-25T09:02:55.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.490+0000 7ff3ba87c640 1 -- 172.21.15.43:0/1528369238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3b406c6c0 msgr2=0x7ff3b40b7010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:55.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.490+0000 7ff3ba87c640 1 --2- 172.21.15.43:0/1528369238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3b406c6c0 0x7ff3b40b7010 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac000f80 tx=0x7ff3ac00e530 comp rx=0 tx=0).stop 2024-04-25T09:02:55.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.490+0000 7ff3ba87c640 1 -- 172.21.15.43:0/1528369238 shutdown_connections 2024-04-25T09:02:55.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.490+0000 7ff3ba87c640 1 --2- 172.21.15.43:0/1528369238 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff3b406c6c0 0x7ff3b40b7010 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.490+0000 7ff3ba87c640 1 --2- 172.21.15.43:0/1528369238 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff3b4159250 0x7ff3b406c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.490+0000 7ff3ba87c640 1 --2- 172.21.15.43:0/1528369238 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff3a006b500 0x7ff3a006d9c0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.490+0000 7ff3ba87c640 1 --2- 172.21.15.43:0/1528369238 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff39c003f30 0x7ff3b41f47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.490+0000 7ff3ba87c640 1 -- 172.21.15.43:0/1528369238 >> 172.21.15.43:0/1528369238 conn(0x7ff3b409cd40 msgr2=0x7ff3b40d3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:55.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.490+0000 7ff3ba87c640 1 -- 172.21.15.43:0/1528369238 shutdown_connections 2024-04-25T09:02:55.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.490+0000 7ff3ba87c640 1 -- 172.21.15.43:0/1528369238 wait complete. 2024-04-25T09:02:55.489 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:02:55.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.604+0000 7f2732c31640 1 Processor -- start 2024-04-25T09:02:55.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.604+0000 7f2732c31640 1 -- start start 2024-04-25T09:02:55.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.605+0000 7f2732c31640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f272c15dec0 0x7f272c15ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.605+0000 7f2732c31640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f272c0ce8c0 0x7f272c0d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.605+0000 7f2732c31640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f272c0cf110 con 0x7f272c0d04a0 2024-04-25T09:02:55.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.605+0000 7f2732c31640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f272c0cf3f0 con 0x7f272c15dec0 2024-04-25T09:02:55.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.605+0000 7f2732c31640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f272c0cf6d0 con 0x7f272c0ce8c0 2024-04-25T09:02:55.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.605+0000 7f2723fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f272c15dec0 0x7f272c15ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.605+0000 7f27309cd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f272c0d04a0 0x7f272c0d0880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:53462/0 (socket says 172.21.15.43:53462) 2024-04-25T09:02:55.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.605+0000 7f27309cd640 1 -- 172.21.15.43:0/3830703440 learned_addr learned my addr 172.21.15.43:0/3830703440 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:55.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.605+0000 7f2723fff640 1 -- 172.21.15.43:0/3830703440 >> v2:172.21.15.193:3300/0 conn(0x7f272c0ce8c0 msgr2=0x7f272c0d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:55.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.605+0000 7f27311ce640 1 --2- 172.21.15.43:0/3830703440 >> v2:172.21.15.193:3300/0 conn(0x7f272c0ce8c0 0x7f272c0d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.605+0000 7f2723fff640 1 --2- 172.21.15.43:0/3830703440 >> v2:172.21.15.193:3300/0 conn(0x7f272c0ce8c0 0x7f272c0d17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.605+0000 7f27237fe640 1 -- 172.21.15.43:0/3830703440 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2239075252 0 0) 0x7f272c0cf110 con 0x7f272c0d04a0 2024-04-25T09:02:55.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.605+0000 7f2723fff640 1 -- 172.21.15.43:0/3830703440 >> v1:172.21.15.43:6789/0 conn(0x7f272c0d04a0 legacy=0x7f272c0d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:55.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.605+0000 7f2723fff640 1 -- 172.21.15.43:0/3830703440 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f272c0cf9b0 con 0x7f272c15dec0 2024-04-25T09:02:55.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.605+0000 7f27311ce640 1 --2- 172.21.15.43:0/3830703440 >> v2:172.21.15.193:3300/0 conn(0x7f272c0ce8c0 0x7f272c0d17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:55.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.606+0000 7f2723fff640 1 --2- 172.21.15.43:0/3830703440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f272c15dec0 0x7f272c15ed90 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f272c08e260 tx=0x7f271c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f801a89289d70ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:55.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.606+0000 7f27237fe640 1 -- 172.21.15.43:0/3830703440 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271c037070 con 0x7f272c15dec0 2024-04-25T09:02:55.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.606+0000 7f27237fe640 1 -- 172.21.15.43:0/3830703440 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f271c032440 con 0x7f272c15dec0 2024-04-25T09:02:55.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.606+0000 7f27237fe640 1 -- 172.21.15.43:0/3830703440 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271c032810 con 0x7f272c15dec0 2024-04-25T09:02:55.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.607+0000 7f2732c31640 1 -- 172.21.15.43:0/3830703440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f272c15dec0 msgr2=0x7f272c15ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:55.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.607+0000 7f2732c31640 1 --2- 172.21.15.43:0/3830703440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f272c15dec0 0x7f272c15ed90 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f272c08e260 tx=0x7f271c02a1a0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.607+0000 7f2732c31640 1 -- 172.21.15.43:0/3830703440 shutdown_connections 2024-04-25T09:02:55.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.607+0000 7f2732c31640 1 --2- 172.21.15.43:0/3830703440 >> v2:172.21.15.193:3300/0 conn(0x7f272c0ce8c0 0x7f272c0d17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.607+0000 7f2732c31640 1 --2- 172.21.15.43:0/3830703440 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f272c15dec0 0x7f272c15ed90 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.607+0000 7f2732c31640 1 -- 172.21.15.43:0/3830703440 >> 172.21.15.43:0/3830703440 conn(0x7f272c09cd40 msgr2=0x7f272c0d3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:55.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.607+0000 7f2732c31640 1 -- 172.21.15.43:0/3830703440 shutdown_connections 2024-04-25T09:02:55.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.607+0000 7f2732c31640 1 -- 172.21.15.43:0/3830703440 wait complete. 2024-04-25T09:02:55.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.608+0000 7f2732c31640 1 Processor -- start 2024-04-25T09:02:55.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.608+0000 7f2732c31640 1 -- start start 2024-04-25T09:02:55.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.609+0000 7f2732c31640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f272c0ce8c0 0x7f272c0b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.609+0000 7f2732c31640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f272c0d04a0 0x7f272c154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.609+0000 7f2732c31640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f272c15dec0 0x7f272c1f72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.609+0000 7f2732c31640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272c0cee70 con 0x7f272c0ce8c0 2024-04-25T09:02:55.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.609+0000 7f2732c31640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272c0cecf0 con 0x7f272c15dec0 2024-04-25T09:02:55.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.609+0000 7f2732c31640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f272c0d1ce0 con 0x7f272c0d04a0 2024-04-25T09:02:55.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.609+0000 7f27309cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f272c0ce8c0 0x7f272c0b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.609+0000 7f27309cd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f272c0ce8c0 0x7f272c0b3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48584/0 (socket says 172.21.15.43:48584) 2024-04-25T09:02:55.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.609+0000 7f27309cd640 1 -- 172.21.15.43:0/1180724174 learned_addr learned my addr 172.21.15.43:0/1180724174 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:55.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.609+0000 7f27311ce640 1 --2- 172.21.15.43:0/1180724174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f272c15dec0 0x7f272c1f72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.609+0000 7f2723fff640 1 --2- 172.21.15.43:0/1180724174 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f272c0d04a0 0x7f272c154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.609+0000 7f27309cd640 1 -- 172.21.15.43:0/1180724174 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f272c0d04a0 msgr2=0x7f272c154b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:55.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.609+0000 7f27309cd640 1 --2- 172.21.15.43:0/1180724174 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f272c0d04a0 0x7f272c154b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.609+0000 7f27309cd640 1 -- 172.21.15.43:0/1180724174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f272c15dec0 msgr2=0x7f272c1f72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:55.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.609+0000 7f27309cd640 1 --2- 172.21.15.43:0/1180724174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f272c15dec0 0x7f272c1f72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.609+0000 7f27309cd640 1 -- 172.21.15.43:0/1180724174 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f272c0b4a40 con 0x7f272c0ce8c0 2024-04-25T09:02:55.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.609+0000 7f27309cd640 1 --2- 172.21.15.43:0/1180724174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f272c0ce8c0 0x7f272c0b3ea0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f272800c500 tx=0x7f272800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:55.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.610+0000 7f27217fa640 1 -- 172.21.15.43:0/1180724174 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2728018070 con 0x7f272c0ce8c0 2024-04-25T09:02:55.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.610+0000 7f27217fa640 1 -- 172.21.15.43:0/1180724174 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2728007580 con 0x7f272c0ce8c0 2024-04-25T09:02:55.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.610+0000 7f2732c31640 1 -- 172.21.15.43:0/1180724174 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f272c0b4480 con 0x7f272c0ce8c0 2024-04-25T09:02:55.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.610+0000 7f27217fa640 1 -- 172.21.15.43:0/1180724174 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2728004a20 con 0x7f272c0ce8c0 2024-04-25T09:02:55.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.610+0000 7f2732c31640 1 -- 172.21.15.43:0/1180724174 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f272c1f4fa0 con 0x7f272c0ce8c0 2024-04-25T09:02:55.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.611+0000 7f2732c31640 1 -- 172.21.15.43:0/1180724174 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f272c0d2f70 con 0x7f272c0ce8c0 2024-04-25T09:02:55.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.611+0000 7f27217fa640 1 -- 172.21.15.43:0/1180724174 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f272801b070 con 0x7f272c0ce8c0 2024-04-25T09:02:55.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.611+0000 7f27217fa640 1 --2- 172.21.15.43:0/1180724174 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f271806b4d0 0x7f271806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.612+0000 7f27217fa640 1 -- 172.21.15.43:0/1180724174 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f27280894c0 con 0x7f272c0ce8c0 2024-04-25T09:02:55.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.612+0000 7f2723fff640 1 --2- 172.21.15.43:0/1180724174 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f271806b4d0 0x7f271806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.612+0000 7f2723fff640 1 --2- 172.21.15.43:0/1180724174 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f271806b4d0 0x7f271806d990 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f271c034000 tx=0x7f271c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:55.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.616+0000 7f27217fa640 1 -- 172.21.15.43:0/1180724174 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f272c0d2f70 con 0x7f272c0ce8c0 2024-04-25T09:02:55.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.793+0000 7f2732c31640 1 -- 172.21.15.43:0/1180724174 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f272c0ca440 con 0x7f272c0ce8c0 2024-04-25T09:02:55.782 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.794+0000 7f27217fa640 1 -- 172.21.15.43:0/1180724174 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v115) v1 ==== 104+0+11 (secure 0 0 0) 0x7f272c0b4a40 con 0x7f272c0ce8c0 2024-04-25T09:02:55.782 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T09:02:55.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.796+0000 7f2732c31640 1 -- 172.21.15.43:0/1180724174 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f271806b4d0 msgr2=0x7f271806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:55.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.796+0000 7f2732c31640 1 --2- 172.21.15.43:0/1180724174 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f271806b4d0 0x7f271806d990 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f271c034000 tx=0x7f271c02e000 comp rx=0 tx=0).stop 2024-04-25T09:02:55.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.796+0000 7f2732c31640 1 -- 172.21.15.43:0/1180724174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f272c0ce8c0 msgr2=0x7f272c0b3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:55.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.796+0000 7f2732c31640 1 --2- 172.21.15.43:0/1180724174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f272c0ce8c0 0x7f272c0b3ea0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f272800c500 tx=0x7f272800c9c0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.796+0000 7f2732c31640 1 -- 172.21.15.43:0/1180724174 shutdown_connections 2024-04-25T09:02:55.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.796+0000 7f2732c31640 1 --2- 172.21.15.43:0/1180724174 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f271806b4d0 0x7f271806d990 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.796+0000 7f2732c31640 1 --2- 172.21.15.43:0/1180724174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f272c15dec0 0x7f272c1f72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.796+0000 7f2732c31640 1 --2- 172.21.15.43:0/1180724174 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f272c0d04a0 0x7f272c154b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.796+0000 7f2732c31640 1 --2- 172.21.15.43:0/1180724174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f272c0ce8c0 0x7f272c0b3ea0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.796+0000 7f2732c31640 1 -- 172.21.15.43:0/1180724174 >> 172.21.15.43:0/1180724174 conn(0x7f272c09cd40 msgr2=0x7f272c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:55.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.796+0000 7f2732c31640 1 -- 172.21.15.43:0/1180724174 shutdown_connections 2024-04-25T09:02:55.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.796+0000 7f2732c31640 1 -- 172.21.15.43:0/1180724174 wait complete. 2024-04-25T09:02:55.795 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:02:55.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.912+0000 7f9521e15640 1 Processor -- start 2024-04-25T09:02:55.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.912+0000 7f9521e15640 1 -- start start 2024-04-25T09:02:55.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.912+0000 7f9521e15640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f951c06d7d0 0x7f951c0ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.913+0000 7f9521e15640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f951c0cd120 0x7f951c15d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.913+0000 7f9521e15640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f951c0cd540 con 0x7f951c0aee80 2024-04-25T09:02:55.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.913+0000 7f9521e15640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f951c0af260 con 0x7f951c06d7d0 2024-04-25T09:02:55.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.913+0000 7f9521e15640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f951c15c280 con 0x7f951c0cd120 2024-04-25T09:02:55.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.913+0000 7f951affd640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f951c06d7d0 0x7f951c0ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.913+0000 7f951bfff640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f951c0cd120 0x7f951c15d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.913+0000 7f951bfff640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f951c0cd120 0x7f951c15d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:59332/0 (socket says 172.21.15.43:59332) 2024-04-25T09:02:55.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.913+0000 7f951b7fe640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f951c0aee80 0x7f951c06d0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:54856/0 (socket says 172.21.15.43:54856) 2024-04-25T09:02:55.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.913+0000 7f951affd640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f951c06d7d0 0x7f951c0ccbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:40828/0 (socket says 172.21.15.43:40828) 2024-04-25T09:02:55.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.913+0000 7f951bfff640 1 -- 172.21.15.43:0/563676712 learned_addr learned my addr 172.21.15.43:0/563676712 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:55.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.913+0000 7f951bfff640 1 -- 172.21.15.43:0/563676712 >> v2:172.21.15.193:3300/0 conn(0x7f951c06d7d0 msgr2=0x7f951c0ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:55.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.913+0000 7f951bfff640 1 --2- 172.21.15.43:0/563676712 >> v2:172.21.15.193:3300/0 conn(0x7f951c06d7d0 0x7f951c0ccbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.913+0000 7f951bfff640 1 -- 172.21.15.43:0/563676712 >> v1:172.21.15.193:6789/0 conn(0x7f951c0aee80 legacy=0x7f951c06d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:55.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.913+0000 7f951bfff640 1 -- 172.21.15.43:0/563676712 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951c15c560 con 0x7f951c0cd120 2024-04-25T09:02:55.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.913+0000 7f951affd640 1 --2- 172.21.15.43:0/563676712 >> v2:172.21.15.193:3300/0 conn(0x7f951c06d7d0 0x7f951c0ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:02:55.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.914+0000 7f951bfff640 1 --2- 172.21.15.43:0/563676712 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f951c0cd120 0x7f951c15d590 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9514009e50 tx=0x7f951402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69a2fe59f147489e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:55.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.914+0000 7f951a7fc640 1 -- 172.21.15.43:0/563676712 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9514036070 con 0x7f951c0cd120 2024-04-25T09:02:55.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.914+0000 7f951a7fc640 1 -- 172.21.15.43:0/563676712 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9514031600 con 0x7f951c0cd120 2024-04-25T09:02:55.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.914+0000 7f951a7fc640 1 -- 172.21.15.43:0/563676712 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95140319d0 con 0x7f951c0cd120 2024-04-25T09:02:55.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.915+0000 7f9521e15640 1 -- 172.21.15.43:0/563676712 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f951c0cd120 msgr2=0x7f951c15d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:55.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.915+0000 7f9521e15640 1 --2- 172.21.15.43:0/563676712 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f951c0cd120 0x7f951c15d590 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9514009e50 tx=0x7f951402a710 comp rx=0 tx=0).stop 2024-04-25T09:02:55.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.915+0000 7f9521e15640 1 -- 172.21.15.43:0/563676712 shutdown_connections 2024-04-25T09:02:55.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.915+0000 7f9521e15640 1 --2- 172.21.15.43:0/563676712 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f951c0cd120 0x7f951c15d590 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.915+0000 7f9521e15640 1 --2- 172.21.15.43:0/563676712 >> v2:172.21.15.193:3300/0 conn(0x7f951c06d7d0 0x7f951c0ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.915+0000 7f9521e15640 1 -- 172.21.15.43:0/563676712 >> 172.21.15.43:0/563676712 conn(0x7f951c06ab30 msgr2=0x7f951c06af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:55.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.915+0000 7f9521e15640 1 -- 172.21.15.43:0/563676712 shutdown_connections 2024-04-25T09:02:55.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.915+0000 7f9521e15640 1 -- 172.21.15.43:0/563676712 wait complete. 2024-04-25T09:02:55.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.916+0000 7f9521e15640 1 Processor -- start 2024-04-25T09:02:55.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.916+0000 7f9521e15640 1 -- start start 2024-04-25T09:02:55.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.917+0000 7f9521e15640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f951c06d7d0 0x7f951c0b0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.917+0000 7f9521e15640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f951c0aee80 0x7f951c1587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.917+0000 7f9521e15640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f951c0cd120 0x7f951c0ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.917+0000 7f9521e15640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951c15dad0 con 0x7f951c0cd120 2024-04-25T09:02:55.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.917+0000 7f9521e15640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951c15cb20 con 0x7f951c0aee80 2024-04-25T09:02:55.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.917+0000 7f9521e15640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f951c15dc50 con 0x7f951c06d7d0 2024-04-25T09:02:55.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.917+0000 7f951bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f951c0cd120 0x7f951c0ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.917+0000 7f951bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f951c0cd120 0x7f951c0ad8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48592/0 (socket says 172.21.15.43:48592) 2024-04-25T09:02:55.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.917+0000 7f951bfff640 1 -- 172.21.15.43:0/13147849 learned_addr learned my addr 172.21.15.43:0/13147849 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:55.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.917+0000 7f951b7fe640 1 --2- 172.21.15.43:0/13147849 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f951c06d7d0 0x7f951c0b0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.917+0000 7f951affd640 1 --2- 172.21.15.43:0/13147849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f951c0aee80 0x7f951c1587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.917+0000 7f951bfff640 1 -- 172.21.15.43:0/13147849 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f951c06d7d0 msgr2=0x7f951c0b0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:55.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.917+0000 7f951bfff640 1 --2- 172.21.15.43:0/13147849 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f951c06d7d0 0x7f951c0b0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.917+0000 7f951bfff640 1 -- 172.21.15.43:0/13147849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f951c0aee80 msgr2=0x7f951c1587f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:55.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.917+0000 7f951bfff640 1 --2- 172.21.15.43:0/13147849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f951c0aee80 0x7f951c1587f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:55.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.917+0000 7f951bfff640 1 -- 172.21.15.43:0/13147849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f951c1567c0 con 0x7f951c0cd120 2024-04-25T09:02:55.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.918+0000 7f951bfff640 1 --2- 172.21.15.43:0/13147849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f951c0cd120 0x7f951c0ad8e0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f9514034040 tx=0x7f9514004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:55.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.918+0000 7f9503fff640 1 -- 172.21.15.43:0/13147849 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9514036050 con 0x7f951c0cd120 2024-04-25T09:02:55.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.918+0000 7f9503fff640 1 -- 172.21.15.43:0/13147849 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9514042070 con 0x7f951c0cd120 2024-04-25T09:02:55.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.918+0000 7f9503fff640 1 -- 172.21.15.43:0/13147849 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9514009510 con 0x7f951c0cd120 2024-04-25T09:02:55.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.918+0000 7f9521e15640 1 -- 172.21.15.43:0/13147849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f951c156aa0 con 0x7f951c0cd120 2024-04-25T09:02:55.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.918+0000 7f9521e15640 1 -- 172.21.15.43:0/13147849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f951c157060 con 0x7f951c0cd120 2024-04-25T09:02:55.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.919+0000 7f9521e15640 1 -- 172.21.15.43:0/13147849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f951c074f60 con 0x7f951c0cd120 2024-04-25T09:02:55.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.919+0000 7f9503fff640 1 -- 172.21.15.43:0/13147849 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9514007c70 con 0x7f951c0cd120 2024-04-25T09:02:55.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.920+0000 7f9503fff640 1 --2- 172.21.15.43:0/13147849 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f950806b4d0 0x7f950806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:55.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.920+0000 7f9503fff640 1 -- 172.21.15.43:0/13147849 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f95140ab030 con 0x7f951c0cd120 2024-04-25T09:02:55.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.920+0000 7f951b7fe640 1 --2- 172.21.15.43:0/13147849 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f950806b4d0 0x7f950806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:55.910 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.920+0000 7f951b7fe640 1 --2- 172.21.15.43:0/13147849 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f950806b4d0 0x7f950806d990 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f9510000f60 tx=0x7f9510007cc0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:55.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:55.924+0000 7f9503fff640 1 -- 172.21.15.43:0/13147849 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9514079f00 con 0x7f951c0cd120 2024-04-25T09:02:56.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.105+0000 7f9521e15640 1 -- 172.21.15.43:0/13147849 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f951c0b1130 con 0x7f951c0cd120 2024-04-25T09:02:56.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.106+0000 7f9503fff640 1 -- 172.21.15.43:0/13147849 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v115) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9514079800 con 0x7f951c0cd120 2024-04-25T09:02:56.095 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T09:02:56.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.108+0000 7f9521e15640 1 -- 172.21.15.43:0/13147849 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f950806b4d0 msgr2=0x7f950806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:56.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.108+0000 7f9521e15640 1 --2- 172.21.15.43:0/13147849 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f950806b4d0 0x7f950806d990 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f9510000f60 tx=0x7f9510007cc0 comp rx=0 tx=0).stop 2024-04-25T09:02:56.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.108+0000 7f9521e15640 1 -- 172.21.15.43:0/13147849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f951c0cd120 msgr2=0x7f951c0ad8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:56.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.108+0000 7f9521e15640 1 --2- 172.21.15.43:0/13147849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f951c0cd120 0x7f951c0ad8e0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f9514034040 tx=0x7f9514004a40 comp rx=0 tx=0).stop 2024-04-25T09:02:56.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.108+0000 7f9521e15640 1 -- 172.21.15.43:0/13147849 shutdown_connections 2024-04-25T09:02:56.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.108+0000 7f9521e15640 1 --2- 172.21.15.43:0/13147849 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f951c0cd120 0x7f951c0ad8e0 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:56.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.108+0000 7f9521e15640 1 --2- 172.21.15.43:0/13147849 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f950806b4d0 0x7f950806d990 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:56.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.108+0000 7f9521e15640 1 --2- 172.21.15.43:0/13147849 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f951c0aee80 0x7f951c1587f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:56.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.108+0000 7f9521e15640 1 --2- 172.21.15.43:0/13147849 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f951c06d7d0 0x7f951c0b0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:56.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.108+0000 7f9521e15640 1 -- 172.21.15.43:0/13147849 >> 172.21.15.43:0/13147849 conn(0x7f951c06ab30 msgr2=0x7f951c0af3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:56.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.108+0000 7f9521e15640 1 -- 172.21.15.43:0/13147849 shutdown_connections 2024-04-25T09:02:56.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.108+0000 7f9521e15640 1 -- 172.21.15.43:0/13147849 wait complete. 2024-04-25T09:02:56.107 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:02:56.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.220+0000 7fb7a00ee640 1 Processor -- start 2024-04-25T09:02:56.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.220+0000 7fb7a00ee640 1 -- start start 2024-04-25T09:02:56.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.221+0000 7fb7a00ee640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb7980ccbb0 0x7fb7980cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:56.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.221+0000 7fb7a00ee640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb7980b3b00 0x7fb7980b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:56.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.221+0000 7fb7a00ee640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7980b2780 con 0x7fb7980cbfc0 2024-04-25T09:02:56.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.221+0000 7fb7a00ee640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7980cd550 con 0x7fb7980b3b00 2024-04-25T09:02:56.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.221+0000 7fb7a00ee640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7980b2a60 con 0x7fb7980ccbb0 2024-04-25T09:02:56.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.221+0000 7fb79e68b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb7980b3b00 0x7fb7980b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:56.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.221+0000 7fb79e68b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb7980b3b00 0x7fb7980b3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48594/0 (socket says 172.21.15.43:48594) 2024-04-25T09:02:56.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.221+0000 7fb79e68b640 1 -- 172.21.15.43:0/4267232180 learned_addr learned my addr 172.21.15.43:0/4267232180 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:56.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.221+0000 7fb79d689640 1 --2- 172.21.15.43:0/4267232180 >> v2:172.21.15.193:3300/0 conn(0x7fb7980ccbb0 0x7fb7980cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:56.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.221+0000 7fb79e68b640 1 -- 172.21.15.43:0/4267232180 >> v2:172.21.15.193:3300/0 conn(0x7fb7980ccbb0 msgr2=0x7fb7980cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:56.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.221+0000 7fb79e68b640 1 --2- 172.21.15.43:0/4267232180 >> v2:172.21.15.193:3300/0 conn(0x7fb7980ccbb0 0x7fb7980cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:56.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.221+0000 7fb79e68b640 1 -- 172.21.15.43:0/4267232180 >> v1:172.21.15.193:6789/0 conn(0x7fb7980cbfc0 legacy=0x7fb79806dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:56.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.221+0000 7fb79e68b640 1 -- 172.21.15.43:0/4267232180 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7980b2d40 con 0x7fb7980b3b00 2024-04-25T09:02:56.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.222+0000 7fb79e68b640 1 --2- 172.21.15.43:0/4267232180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7980b3b00 0x7fb7980b3f70 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fb7980747f0 tx=0x7fb79002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c7b125e15fbc4d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:56.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.222+0000 7fb79ce88640 1 -- 172.21.15.43:0/4267232180 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb790036070 con 0x7fb7980b3b00 2024-04-25T09:02:56.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.222+0000 7fb79ce88640 1 -- 172.21.15.43:0/4267232180 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb790031600 con 0x7fb7980b3b00 2024-04-25T09:02:56.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.222+0000 7fb79ce88640 1 -- 172.21.15.43:0/4267232180 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7900319d0 con 0x7fb7980b3b00 2024-04-25T09:02:56.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.223+0000 7fb7a00ee640 1 -- 172.21.15.43:0/4267232180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7980b3b00 msgr2=0x7fb7980b3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:56.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.223+0000 7fb7a00ee640 1 --2- 172.21.15.43:0/4267232180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7980b3b00 0x7fb7980b3f70 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7fb7980747f0 tx=0x7fb79002a730 comp rx=0 tx=0).stop 2024-04-25T09:02:56.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.223+0000 7fb7a00ee640 1 -- 172.21.15.43:0/4267232180 shutdown_connections 2024-04-25T09:02:56.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.223+0000 7fb7a00ee640 1 --2- 172.21.15.43:0/4267232180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7980b3b00 0x7fb7980b3f70 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:56.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.223+0000 7fb7a00ee640 1 --2- 172.21.15.43:0/4267232180 >> v2:172.21.15.193:3300/0 conn(0x7fb7980ccbb0 0x7fb7980cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:56.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.223+0000 7fb7a00ee640 1 -- 172.21.15.43:0/4267232180 >> 172.21.15.43:0/4267232180 conn(0x7fb7980b1480 msgr2=0x7fb7980b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:56.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.223+0000 7fb7a00ee640 1 -- 172.21.15.43:0/4267232180 shutdown_connections 2024-04-25T09:02:56.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.223+0000 7fb7a00ee640 1 -- 172.21.15.43:0/4267232180 wait complete. 2024-04-25T09:02:56.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.224+0000 7fb7a00ee640 1 Processor -- start 2024-04-25T09:02:56.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.224+0000 7fb7a00ee640 1 -- start start 2024-04-25T09:02:56.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.225+0000 7fb7a00ee640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb7980b3b00 0x7fb79815cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:56.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.225+0000 7fb7a00ee640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7980cbfc0 0x7fb79813ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:56.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.225+0000 7fb7a00ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7980ccbb0 0x7fb79815ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:56.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.225+0000 7fb7a00ee640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7980b35e0 con 0x7fb7980ccbb0 2024-04-25T09:02:56.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.225+0000 7fb7a00ee640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7980b3300 con 0x7fb7980cbfc0 2024-04-25T09:02:56.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.225+0000 7fb7a00ee640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7980cc3a0 con 0x7fb7980b3b00 2024-04-25T09:02:56.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.225+0000 7fb79e68b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7980ccbb0 0x7fb79815ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:56.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.225+0000 7fb79e68b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7980ccbb0 0x7fb79815ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48608/0 (socket says 172.21.15.43:48608) 2024-04-25T09:02:56.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.225+0000 7fb79e68b640 1 -- 172.21.15.43:0/392238226 learned_addr learned my addr 172.21.15.43:0/392238226 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:56.218 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.225+0000 7fb79d689640 1 --2- 172.21.15.43:0/392238226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7980cbfc0 0x7fb79813ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:56.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.225+0000 7fb79de8a640 1 --2- 172.21.15.43:0/392238226 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb7980b3b00 0x7fb79815cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:56.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.225+0000 7fb79e68b640 1 -- 172.21.15.43:0/392238226 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb7980b3b00 msgr2=0x7fb79815cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:56.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.225+0000 7fb79e68b640 1 --2- 172.21.15.43:0/392238226 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb7980b3b00 0x7fb79815cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:56.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.225+0000 7fb79e68b640 1 -- 172.21.15.43:0/392238226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7980cbfc0 msgr2=0x7fb79813ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:56.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.225+0000 7fb79e68b640 1 --2- 172.21.15.43:0/392238226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7980cbfc0 0x7fb79813ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:56.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.225+0000 7fb79e68b640 1 -- 172.21.15.43:0/392238226 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb79813d7e0 con 0x7fb7980ccbb0 2024-04-25T09:02:56.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.225+0000 7fb79d689640 1 --2- 172.21.15.43:0/392238226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7980cbfc0 0x7fb79813ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:56.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.225+0000 7fb79de8a640 1 --2- 172.21.15.43:0/392238226 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb7980b3b00 0x7fb79815cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:56.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.225+0000 7fb79e68b640 1 --2- 172.21.15.43:0/392238226 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7980ccbb0 0x7fb79815ad90 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fb790034040 tx=0x7fb790004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:56.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.225+0000 7fb78e7fc640 1 -- 172.21.15.43:0/392238226 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb790036050 con 0x7fb7980ccbb0 2024-04-25T09:02:56.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.225+0000 7fb78e7fc640 1 -- 172.21.15.43:0/392238226 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb790042070 con 0x7fb7980ccbb0 2024-04-25T09:02:56.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.225+0000 7fb7a00ee640 1 -- 172.21.15.43:0/392238226 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb79813dd60 con 0x7fb7980ccbb0 2024-04-25T09:02:56.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.225+0000 7fb78e7fc640 1 -- 172.21.15.43:0/392238226 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb790009510 con 0x7fb7980ccbb0 2024-04-25T09:02:56.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.225+0000 7fb7a00ee640 1 -- 172.21.15.43:0/392238226 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb79813e650 con 0x7fb7980ccbb0 2024-04-25T09:02:56.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.226+0000 7fb7a00ee640 1 -- 172.21.15.43:0/392238226 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb798074ef0 con 0x7fb7980ccbb0 2024-04-25T09:02:56.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.227+0000 7fb78e7fc640 1 -- 172.21.15.43:0/392238226 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb790007bf0 con 0x7fb7980ccbb0 2024-04-25T09:02:56.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.227+0000 7fb78e7fc640 1 --2- 172.21.15.43:0/392238226 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb77c06b4d0 0x7fb77c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:56.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.228+0000 7fb78e7fc640 1 -- 172.21.15.43:0/392238226 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb7900aaed0 con 0x7fb7980ccbb0 2024-04-25T09:02:56.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.228+0000 7fb79de8a640 1 --2- 172.21.15.43:0/392238226 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb77c06b4d0 0x7fb77c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:56.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.228+0000 7fb79de8a640 1 --2- 172.21.15.43:0/392238226 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb77c06b4d0 0x7fb77c06d990 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb794001680 tx=0x7fb794009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:56.220 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.232+0000 7fb78e7fc640 1 -- 172.21.15.43:0/392238226 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb790079da0 con 0x7fb7980ccbb0 2024-04-25T09:02:56.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.424+0000 7fb7a00ee640 1 -- 172.21.15.43:0/392238226 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb7980aee30 con 0x7fb7980ccbb0 2024-04-25T09:02:56.413 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.424+0000 7fb78e7fc640 1 -- 172.21.15.43:0/392238226 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v115) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb7900796a0 con 0x7fb7980ccbb0 2024-04-25T09:02:56.413 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-04-25T09:02:56.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.426+0000 7fb7a00ee640 1 -- 172.21.15.43:0/392238226 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb77c06b4d0 msgr2=0x7fb77c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:56.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.426+0000 7fb7a00ee640 1 --2- 172.21.15.43:0/392238226 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb77c06b4d0 0x7fb77c06d990 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fb794001680 tx=0x7fb794009040 comp rx=0 tx=0).stop 2024-04-25T09:02:56.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.426+0000 7fb7a00ee640 1 -- 172.21.15.43:0/392238226 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7980ccbb0 msgr2=0x7fb79815ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:56.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.426+0000 7fb7a00ee640 1 --2- 172.21.15.43:0/392238226 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7980ccbb0 0x7fb79815ad90 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fb790034040 tx=0x7fb790004a40 comp rx=0 tx=0).stop 2024-04-25T09:02:56.415 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.426+0000 7fb7a00ee640 1 -- 172.21.15.43:0/392238226 shutdown_connections 2024-04-25T09:02:56.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.426+0000 7fb7a00ee640 1 --2- 172.21.15.43:0/392238226 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb7980ccbb0 0x7fb79815ad90 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:56.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.426+0000 7fb7a00ee640 1 --2- 172.21.15.43:0/392238226 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb77c06b4d0 0x7fb77c06d990 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:56.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.426+0000 7fb7a00ee640 1 --2- 172.21.15.43:0/392238226 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb7980cbfc0 0x7fb79813ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:56.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.426+0000 7fb7a00ee640 1 --2- 172.21.15.43:0/392238226 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb7980b3b00 0x7fb79815cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:56.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.426+0000 7fb7a00ee640 1 -- 172.21.15.43:0/392238226 >> 172.21.15.43:0/392238226 conn(0x7fb7980b1480 msgr2=0x7fb7980cea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:56.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.426+0000 7fb7a00ee640 1 -- 172.21.15.43:0/392238226 shutdown_connections 2024-04-25T09:02:56.416 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:56.426+0000 7fb7a00ee640 1 -- 172.21.15.43:0/392238226 wait complete. 2024-04-25T09:02:56.426 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi043.front.sepia.ceph.com... 2024-04-25T09:02:56.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:56.426 DEBUG:teuthology.orchestra.run.smithi043:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T09:02:56.449 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[86573]: fuse finished with error 0 and tester_r 0 2024-04-25T09:02:56.511 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-04-25T09:02:56.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:56.512 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:02:56.683 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T09:02:56.684 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:02:56.702 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19508 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:02:56.702 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi193.front.sepia.ceph.com 2024-04-25T09:02:56.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:56.702 DEBUG:teuthology.orchestra.run.smithi193:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T09:02:56.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:02:56.759 INFO:teuthology.orchestra.run.smithi193.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T09:02:56.759 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T09:02:56.759 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:02:56.775 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2374 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:02:56.775 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-04-25T09:02:56.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:56.776 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T09:02:56.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:02:56.831 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T09:02:56.831 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T09:02:56.832 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:02:56.849 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19508 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:02:56.849 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi193.front.sepia.ceph.com 2024-04-25T09:02:56.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:02:56.849 DEBUG:teuthology.orchestra.run.smithi193:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T09:02:56.905 INFO:teuthology.orchestra.run.smithi193.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T09:02:56.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:02:56.907 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:02:57.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.019+0000 7fd835f22640 1 Processor -- start 2024-04-25T09:02:57.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.019+0000 7fd835f22640 1 -- start start 2024-04-25T09:02:57.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.020+0000 7fd835f22640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd83015f180 0x7fd8300cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:57.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.020+0000 7fd835f22640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd8300b23f0 0x7fd8300cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:57.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.020+0000 7fd835f22640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8300ccf10 con 0x7fd8300b2cc0 2024-04-25T09:02:57.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.020+0000 7fd835f22640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8300cccc0 con 0x7fd8300b23f0 2024-04-25T09:02:57.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.020+0000 7fd835f22640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8300cd840 con 0x7fd83015f180 2024-04-25T09:02:57.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.020+0000 7fd82effd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd8300b23f0 0x7fd8300cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:57.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.020+0000 7fd82ffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fd8300b2cc0 0x7fd83015f8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:53466/0 (socket says 172.21.15.43:53466) 2024-04-25T09:02:57.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.020+0000 7fd82effd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd8300b23f0 0x7fd8300cfda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48612/0 (socket says 172.21.15.43:48612) 2024-04-25T09:02:57.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.020+0000 7fd82ffff640 1 -- 172.21.15.43:0/2369645294 learned_addr learned my addr 172.21.15.43:0/2369645294 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:57.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.020+0000 7fd82f7fe640 1 --2- 172.21.15.43:0/2369645294 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd83015f180 0x7fd8300cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:57.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.020+0000 7fd82effd640 1 -- 172.21.15.43:0/2369645294 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd83015f180 msgr2=0x7fd8300cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:57.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.021+0000 7fd82effd640 1 --2- 172.21.15.43:0/2369645294 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd83015f180 0x7fd8300cf860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.021+0000 7fd82effd640 1 -- 172.21.15.43:0/2369645294 >> v1:172.21.15.43:6789/0 conn(0x7fd8300b2cc0 legacy=0x7fd83015f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:57.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.021+0000 7fd82effd640 1 -- 172.21.15.43:0/2369645294 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8300cd1f0 con 0x7fd8300b23f0 2024-04-25T09:02:57.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.021+0000 7fd82e7fc640 1 -- 172.21.15.43:0/2369645294 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1303540079 0 0) 0x7fd8300ccf10 con 0x7fd8300b2cc0 2024-04-25T09:02:57.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.021+0000 7fd82f7fe640 1 --2- 172.21.15.43:0/2369645294 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd83015f180 0x7fd8300cf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:57.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.021+0000 7fd82effd640 1 --2- 172.21.15.43:0/2369645294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8300b23f0 0x7fd8300cfda0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fd8240097a0 tx=0x7fd82402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a32e383fd24a975 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:57.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.021+0000 7fd82e7fc640 1 -- 172.21.15.43:0/2369645294 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd824037070 con 0x7fd8300b23f0 2024-04-25T09:02:57.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.021+0000 7fd82e7fc640 1 -- 172.21.15.43:0/2369645294 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd82402ae50 con 0x7fd8300b23f0 2024-04-25T09:02:57.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.021+0000 7fd82e7fc640 1 -- 172.21.15.43:0/2369645294 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd824032660 con 0x7fd8300b23f0 2024-04-25T09:02:57.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.022+0000 7fd835f22640 1 -- 172.21.15.43:0/2369645294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8300b23f0 msgr2=0x7fd8300cfda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:57.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.022+0000 7fd835f22640 1 --2- 172.21.15.43:0/2369645294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8300b23f0 0x7fd8300cfda0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fd8240097a0 tx=0x7fd82402a100 comp rx=0 tx=0).stop 2024-04-25T09:02:57.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.022+0000 7fd835f22640 1 -- 172.21.15.43:0/2369645294 shutdown_connections 2024-04-25T09:02:57.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.022+0000 7fd835f22640 1 --2- 172.21.15.43:0/2369645294 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8300b23f0 0x7fd8300cfda0 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.022+0000 7fd835f22640 1 --2- 172.21.15.43:0/2369645294 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd83015f180 0x7fd8300cf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.022+0000 7fd835f22640 1 -- 172.21.15.43:0/2369645294 >> 172.21.15.43:0/2369645294 conn(0x7fd8300d17f0 msgr2=0x7fd8300d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:57.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.022+0000 7fd835f22640 1 -- 172.21.15.43:0/2369645294 shutdown_connections 2024-04-25T09:02:57.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.022+0000 7fd835f22640 1 -- 172.21.15.43:0/2369645294 wait complete. 2024-04-25T09:02:57.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.023+0000 7fd835f22640 1 Processor -- start 2024-04-25T09:02:57.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.023+0000 7fd835f22640 1 -- start start 2024-04-25T09:02:57.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.024+0000 7fd835f22640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd8300b23f0 0x7fd830158e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:57.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.024+0000 7fd835f22640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd8300b2cc0 0x7fd83006ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:57.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.024+0000 7fd835f22640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd83015f180 0x7fd83015aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:57.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.024+0000 7fd835f22640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd83015f560 con 0x7fd83015f180 2024-04-25T09:02:57.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.024+0000 7fd835f22640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8300cece0 con 0x7fd8300b23f0 2024-04-25T09:02:57.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.024+0000 7fd835f22640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8301f1db0 con 0x7fd8300b2cc0 2024-04-25T09:02:57.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.024+0000 7fd82ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd83015f180 0x7fd83015aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:57.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.024+0000 7fd82ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd83015f180 0x7fd83015aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48628/0 (socket says 172.21.15.43:48628) 2024-04-25T09:02:57.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.024+0000 7fd82ffff640 1 -- 172.21.15.43:0/2753803596 learned_addr learned my addr 172.21.15.43:0/2753803596 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:57.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.024+0000 7fd82effd640 1 --2- 172.21.15.43:0/2753803596 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd8300b2cc0 0x7fd83006ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:57.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.024+0000 7fd82f7fe640 1 --2- 172.21.15.43:0/2753803596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd8300b23f0 0x7fd830158e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:57.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.025+0000 7fd82ffff640 1 -- 172.21.15.43:0/2753803596 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd8300b2cc0 msgr2=0x7fd83006ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:57.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.025+0000 7fd82ffff640 1 --2- 172.21.15.43:0/2753803596 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd8300b2cc0 0x7fd83006ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.025+0000 7fd82ffff640 1 -- 172.21.15.43:0/2753803596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd8300b23f0 msgr2=0x7fd830158e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:57.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.025+0000 7fd82ffff640 1 --2- 172.21.15.43:0/2753803596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd8300b23f0 0x7fd830158e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.025+0000 7fd82ffff640 1 -- 172.21.15.43:0/2753803596 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8301f2090 con 0x7fd83015f180 2024-04-25T09:02:57.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.025+0000 7fd82ffff640 1 --2- 172.21.15.43:0/2753803596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd83015f180 0x7fd83015aca0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fd82800c7a0 tx=0x7fd82800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:57.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.025+0000 7fd817fff640 1 -- 172.21.15.43:0/2753803596 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd828018070 con 0x7fd83015f180 2024-04-25T09:02:57.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.025+0000 7fd817fff640 1 -- 172.21.15.43:0/2753803596 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd82800a640 con 0x7fd83015f180 2024-04-25T09:02:57.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.025+0000 7fd835f22640 1 -- 172.21.15.43:0/2753803596 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8301f2370 con 0x7fd83015f180 2024-04-25T09:02:57.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.025+0000 7fd817fff640 1 -- 172.21.15.43:0/2753803596 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82800aa70 con 0x7fd83015f180 2024-04-25T09:02:57.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.025+0000 7fd835f22640 1 -- 172.21.15.43:0/2753803596 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8301f1530 con 0x7fd83015f180 2024-04-25T09:02:57.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.026+0000 7fd835f22640 1 -- 172.21.15.43:0/2753803596 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd830074ef0 con 0x7fd83015f180 2024-04-25T09:02:57.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.027+0000 7fd817fff640 1 -- 172.21.15.43:0/2753803596 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd82800ac10 con 0x7fd83015f180 2024-04-25T09:02:57.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.027+0000 7fd817fff640 1 --2- 172.21.15.43:0/2753803596 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd81c06b4d0 0x7fd81c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:57.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.027+0000 7fd817fff640 1 -- 172.21.15.43:0/2753803596 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd828089800 con 0x7fd83015f180 2024-04-25T09:02:57.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.027+0000 7fd82f7fe640 1 --2- 172.21.15.43:0/2753803596 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd81c06b4d0 0x7fd81c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:57.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.028+0000 7fd82f7fe640 1 --2- 172.21.15.43:0/2753803596 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd81c06b4d0 0x7fd81c06d990 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fd8300b1310 tx=0x7fd820008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:57.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.031+0000 7fd817fff640 1 -- 172.21.15.43:0/2753803596 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd828058000 con 0x7fd83015f180 2024-04-25T09:02:57.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.247+0000 7fd835f22640 1 -- 172.21.15.43:0/2753803596 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd8300cf050 con 0x7fd83015f180 2024-04-25T09:02:57.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.248+0000 7fd817fff640 1 -- 172.21.15.43:0/2753803596 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4946 (secure 0 0 0) 0x7fd82805c0a0 con 0x7fd83015f180 2024-04-25T09:02:57.237 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:57.237 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":69,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6821,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2021228703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2021228703},{"type":"v1","addr":"172.21.15.193:6837","nonce":2021228703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65},{"gid":7543,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/2398324998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2398324998},{"type":"v1","addr":"172.21.15.193:6836","nonce":2398324998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65},{"gid":8673,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2298187831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2298187831},{"type":"v1","addr":"172.21.15.43:6834","nonce":2298187831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:02:26.161487+0000","modified":"2024-04-25T09:02:29.177294+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8676":{"gid":8676,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/1586865031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1586865031},{"type":"v1","addr":"172.21.15.43:6835","nonce":1586865031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:02:57.237 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 69 2024-04-25T09:02:57.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.250+0000 7fd835f22640 1 -- 172.21.15.43:0/2753803596 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd81c06b4d0 msgr2=0x7fd81c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:57.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.250+0000 7fd835f22640 1 --2- 172.21.15.43:0/2753803596 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd81c06b4d0 0x7fd81c06d990 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fd8300b1310 tx=0x7fd820008040 comp rx=0 tx=0).stop 2024-04-25T09:02:57.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.250+0000 7fd835f22640 1 -- 172.21.15.43:0/2753803596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd83015f180 msgr2=0x7fd83015aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:57.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.250+0000 7fd835f22640 1 --2- 172.21.15.43:0/2753803596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd83015f180 0x7fd83015aca0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fd82800c7a0 tx=0x7fd82800cc60 comp rx=0 tx=0).stop 2024-04-25T09:02:57.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.250+0000 7fd835f22640 1 -- 172.21.15.43:0/2753803596 shutdown_connections 2024-04-25T09:02:57.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.250+0000 7fd835f22640 1 --2- 172.21.15.43:0/2753803596 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd83015f180 0x7fd83015aca0 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.250+0000 7fd835f22640 1 --2- 172.21.15.43:0/2753803596 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd8300b2cc0 0x7fd83006ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.250+0000 7fd835f22640 1 --2- 172.21.15.43:0/2753803596 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd81c06b4d0 0x7fd81c06d990 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.250+0000 7fd835f22640 1 --2- 172.21.15.43:0/2753803596 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd8300b23f0 0x7fd830158e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.239 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.250+0000 7fd835f22640 1 -- 172.21.15.43:0/2753803596 >> 172.21.15.43:0/2753803596 conn(0x7fd8300d17f0 msgr2=0x7fd8300c8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:57.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.250+0000 7fd835f22640 1 -- 172.21.15.43:0/2753803596 shutdown_connections 2024-04-25T09:02:57.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.250+0000 7fd835f22640 1 -- 172.21.15.43:0/2753803596 wait complete. 2024-04-25T09:02:57.250 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:02:57.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.362+0000 7fbd6cf5c640 1 Processor -- start 2024-04-25T09:02:57.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.362+0000 7fbd6cf5c640 1 -- start start 2024-04-25T09:02:57.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.363+0000 7fbd6cf5c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbd6806c510 0x7fbd6806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:57.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.363+0000 7fbd6cf5c640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fbd6815df70 0x7fbd680ce070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:57.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.363+0000 7fbd6cf5c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd680b0870 0x7fbd680ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:57.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.363+0000 7fbd6cf5c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd680d0400 con 0x7fbd6806c510 2024-04-25T09:02:57.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.363+0000 7fbd6cf5c640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd680cbee0 con 0x7fbd6815df70 2024-04-25T09:02:57.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.363+0000 7fbd6cf5c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6806ce30 con 0x7fbd680b0870 2024-04-25T09:02:57.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.363+0000 7fbd6659c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbd6806c510 0x7fbd6806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:57.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.363+0000 7fbd6659c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbd6806c510 0x7fbd6806c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48644/0 (socket says 172.21.15.43:48644) 2024-04-25T09:02:57.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.363+0000 7fbd6659c640 1 -- 172.21.15.43:0/3089181059 learned_addr learned my addr 172.21.15.43:0/3089181059 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:57.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.364+0000 7fbd66d9d640 1 --2- 172.21.15.43:0/3089181059 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd680b0870 0x7fbd680ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:57.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.364+0000 7fbd65d9b640 1 --2- 172.21.15.43:0/3089181059 >> v2:172.21.15.193:3300/0 conn(0x7fbd6815df70 0x7fbd680ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:57.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.364+0000 7fbd6659c640 1 -- 172.21.15.43:0/3089181059 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd680b0870 msgr2=0x7fbd680ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:57.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.364+0000 7fbd6659c640 1 --2- 172.21.15.43:0/3089181059 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd680b0870 0x7fbd680ce5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.364+0000 7fbd6659c640 1 -- 172.21.15.43:0/3089181059 >> v2:172.21.15.193:3300/0 conn(0x7fbd6815df70 msgr2=0x7fbd680ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:57.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.364+0000 7fbd6659c640 1 --2- 172.21.15.43:0/3089181059 >> v2:172.21.15.193:3300/0 conn(0x7fbd6815df70 0x7fbd680ce070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.364+0000 7fbd6659c640 1 -- 172.21.15.43:0/3089181059 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd680d0650 con 0x7fbd6806c510 2024-04-25T09:02:57.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.364+0000 7fbd65d9b640 1 --2- 172.21.15.43:0/3089181059 >> v2:172.21.15.193:3300/0 conn(0x7fbd6815df70 0x7fbd680ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:57.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.364+0000 7fbd6659c640 1 --2- 172.21.15.43:0/3089181059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd6806c510 0x7fbd6806c8f0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fbd68068370 tx=0x7fbd6002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0ac70decf9d7fe0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:57.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.364+0000 7fbd6559a640 1 -- 172.21.15.43:0/3089181059 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd60037070 con 0x7fbd6806c510 2024-04-25T09:02:57.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.365+0000 7fbd6559a640 1 -- 172.21.15.43:0/3089181059 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd60032440 con 0x7fbd6806c510 2024-04-25T09:02:57.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.365+0000 7fbd6559a640 1 -- 172.21.15.43:0/3089181059 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd60032810 con 0x7fbd6806c510 2024-04-25T09:02:57.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.365+0000 7fbd6cf5c640 1 -- 172.21.15.43:0/3089181059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd6806c510 msgr2=0x7fbd6806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:57.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.365+0000 7fbd6cf5c640 1 --2- 172.21.15.43:0/3089181059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd6806c510 0x7fbd6806c8f0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fbd68068370 tx=0x7fbd6002a1e0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.365+0000 7fbd6cf5c640 1 -- 172.21.15.43:0/3089181059 shutdown_connections 2024-04-25T09:02:57.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.365+0000 7fbd6cf5c640 1 --2- 172.21.15.43:0/3089181059 >> v2:172.21.15.193:3300/0 conn(0x7fbd6815df70 0x7fbd680ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.365+0000 7fbd6cf5c640 1 --2- 172.21.15.43:0/3089181059 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd680b0870 0x7fbd680ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.365+0000 7fbd6cf5c640 1 --2- 172.21.15.43:0/3089181059 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd6806c510 0x7fbd6806c8f0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.365+0000 7fbd6cf5c640 1 -- 172.21.15.43:0/3089181059 >> 172.21.15.43:0/3089181059 conn(0x7fbd680d17f0 msgr2=0x7fbd680d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:57.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.365+0000 7fbd6cf5c640 1 -- 172.21.15.43:0/3089181059 shutdown_connections 2024-04-25T09:02:57.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.365+0000 7fbd6cf5c640 1 -- 172.21.15.43:0/3089181059 wait complete. 2024-04-25T09:02:57.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.367+0000 7fbd6cf5c640 1 Processor -- start 2024-04-25T09:02:57.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.367+0000 7fbd6cf5c640 1 -- start start 2024-04-25T09:02:57.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.367+0000 7fbd6cf5c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd6806c510 0x7fbd680b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:57.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.367+0000 7fbd6cf5c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd680b0870 0x7fbd681f4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:57.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.367+0000 7fbd6cf5c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd6815df70 0x7fbd681f6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:57.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.367+0000 7fbd6cf5c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd680d0c10 con 0x7fbd680b0870 2024-04-25T09:02:57.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.367+0000 7fbd6cf5c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd680cc150 con 0x7fbd6815df70 2024-04-25T09:02:57.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.367+0000 7fbd6cf5c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6806bd10 con 0x7fbd6806c510 2024-04-25T09:02:57.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.367+0000 7fbd65d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd680b0870 0x7fbd681f4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:57.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.368+0000 7fbd65d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd680b0870 0x7fbd681f4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48656/0 (socket says 172.21.15.43:48656) 2024-04-25T09:02:57.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.368+0000 7fbd65d9b640 1 -- 172.21.15.43:0/3884775846 learned_addr learned my addr 172.21.15.43:0/3884775846 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:57.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.368+0000 7fbd66d9d640 1 --2- 172.21.15.43:0/3884775846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd6815df70 0x7fbd681f6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:57.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.368+0000 7fbd6659c640 1 --2- 172.21.15.43:0/3884775846 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd6806c510 0x7fbd680b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:57.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.368+0000 7fbd65d9b640 1 -- 172.21.15.43:0/3884775846 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd6806c510 msgr2=0x7fbd680b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:57.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.368+0000 7fbd65d9b640 1 --2- 172.21.15.43:0/3884775846 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd6806c510 0x7fbd680b38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.368+0000 7fbd65d9b640 1 -- 172.21.15.43:0/3884775846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd6815df70 msgr2=0x7fbd681f6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:57.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.368+0000 7fbd65d9b640 1 --2- 172.21.15.43:0/3884775846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd6815df70 0x7fbd681f6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.368+0000 7fbd65d9b640 1 -- 172.21.15.43:0/3884775846 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd681f7d20 con 0x7fbd680b0870 2024-04-25T09:02:57.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.368+0000 7fbd6659c640 1 --2- 172.21.15.43:0/3884775846 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd6806c510 0x7fbd680b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:57.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.368+0000 7fbd66d9d640 1 --2- 172.21.15.43:0/3884775846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd6815df70 0x7fbd681f6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:57.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.368+0000 7fbd65d9b640 1 --2- 172.21.15.43:0/3884775846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd680b0870 0x7fbd681f4130 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000c7c0 tx=0x7fbd5000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:57.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.368+0000 7fbd5effd640 1 -- 172.21.15.43:0/3884775846 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd50015070 con 0x7fbd680b0870 2024-04-25T09:02:57.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.368+0000 7fbd5effd640 1 -- 172.21.15.43:0/3884775846 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd50007820 con 0x7fbd680b0870 2024-04-25T09:02:57.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.368+0000 7fbd6cf5c640 1 -- 172.21.15.43:0/3884775846 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd681f7760 con 0x7fbd680b0870 2024-04-25T09:02:57.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.368+0000 7fbd5effd640 1 -- 172.21.15.43:0/3884775846 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd50007bf0 con 0x7fbd680b0870 2024-04-25T09:02:57.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.368+0000 7fbd6cf5c640 1 -- 172.21.15.43:0/3884775846 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd681f3930 con 0x7fbd680b0870 2024-04-25T09:02:57.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.369+0000 7fbd6cf5c640 1 -- 172.21.15.43:0/3884775846 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd68074ef0 con 0x7fbd680b0870 2024-04-25T09:02:57.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.370+0000 7fbd5effd640 1 -- 172.21.15.43:0/3884775846 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbd50018070 con 0x7fbd680b0870 2024-04-25T09:02:57.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.370+0000 7fbd5effd640 1 --2- 172.21.15.43:0/3884775846 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbd4806b4d0 0x7fbd4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:57.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.370+0000 7fbd5effd640 1 -- 172.21.15.43:0/3884775846 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fbd5008a100 con 0x7fbd680b0870 2024-04-25T09:02:57.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.371+0000 7fbd6659c640 1 --2- 172.21.15.43:0/3884775846 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbd4806b4d0 0x7fbd4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:57.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.371+0000 7fbd6659c640 1 --2- 172.21.15.43:0/3884775846 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbd4806b4d0 0x7fbd4806d990 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fbd60034000 tx=0x7fbd6002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:57.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.374+0000 7fbd5effd640 1 -- 172.21.15.43:0/3884775846 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbd50058f30 con 0x7fbd680b0870 2024-04-25T09:02:57.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.553+0000 7fbd6cf5c640 1 -- 172.21.15.43:0/3884775846 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbd680b3e40 con 0x7fbd680b0870 2024-04-25T09:02:57.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.554+0000 7fbd5effd640 1 -- 172.21.15.43:0/3884775846 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) v1 ==== 74+0+16744 (secure 0 0 0) 0x7fbd50058830 con 0x7fbd680b0870 2024-04-25T09:02:57.543 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:57.543 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":115,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:02:54.872758+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:02:24.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T09:02:25.143723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.e","mappings":[{"from":0,"to":5}]},{"pgid":"17.3f","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:02:57.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.556+0000 7fbd6cf5c640 1 -- 172.21.15.43:0/3884775846 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbd4806b4d0 msgr2=0x7fbd4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:57.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.556+0000 7fbd6cf5c640 1 --2- 172.21.15.43:0/3884775846 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbd4806b4d0 0x7fbd4806d990 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fbd60034000 tx=0x7fbd6002e000 comp rx=0 tx=0).stop 2024-04-25T09:02:57.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.556+0000 7fbd6cf5c640 1 -- 172.21.15.43:0/3884775846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd680b0870 msgr2=0x7fbd681f4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:57.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.556+0000 7fbd6cf5c640 1 --2- 172.21.15.43:0/3884775846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd680b0870 0x7fbd681f4130 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fbd5000c7c0 tx=0x7fbd5000cc80 comp rx=0 tx=0).stop 2024-04-25T09:02:57.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.556+0000 7fbd6cf5c640 1 -- 172.21.15.43:0/3884775846 shutdown_connections 2024-04-25T09:02:57.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.556+0000 7fbd6cf5c640 1 --2- 172.21.15.43:0/3884775846 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbd4806b4d0 0x7fbd4806d990 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.556+0000 7fbd6cf5c640 1 --2- 172.21.15.43:0/3884775846 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd6815df70 0x7fbd681f6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.556+0000 7fbd6cf5c640 1 --2- 172.21.15.43:0/3884775846 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd680b0870 0x7fbd681f4130 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.556+0000 7fbd6cf5c640 1 --2- 172.21.15.43:0/3884775846 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd6806c510 0x7fbd680b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.556+0000 7fbd6cf5c640 1 -- 172.21.15.43:0/3884775846 >> 172.21.15.43:0/3884775846 conn(0x7fbd680d17f0 msgr2=0x7fbd6806ea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:57.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.556+0000 7fbd6cf5c640 1 -- 172.21.15.43:0/3884775846 shutdown_connections 2024-04-25T09:02:57.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.556+0000 7fbd6cf5c640 1 -- 172.21.15.43:0/3884775846 wait complete. 2024-04-25T09:02:57.557 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:02:24.446721+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '112', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:02:25.143723+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:02:57.557 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:02:57.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.673+0000 7f4f5f2f7640 1 Processor -- start 2024-04-25T09:02:57.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.674+0000 7f4f5f2f7640 1 -- start start 2024-04-25T09:02:57.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.674+0000 7f4f5f2f7640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f4f580d19f0 0x7f4f580d1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:57.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.674+0000 7f4f5f2f7640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4f580d2370 0x7f4f580c7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:57.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.674+0000 7f4f5f2f7640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f580cf030 con 0x7f4f58161310 2024-04-25T09:02:57.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.675+0000 7f4f5f2f7640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f580d4870 con 0x7f4f580d2370 2024-04-25T09:02:57.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.675+0000 7f4f5f2f7640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f580cf310 con 0x7f4f580d19f0 2024-04-25T09:02:57.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.675+0000 7f4f5d894640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4f580d2370 0x7f4f580c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:57.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.675+0000 7f4f5d093640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4f58161310 0x7f4f580d4080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:53478/0 (socket says 172.21.15.43:53478) 2024-04-25T09:02:57.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.675+0000 7f4f5d093640 1 -- 172.21.15.43:0/775655544 learned_addr learned my addr 172.21.15.43:0/775655544 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:57.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.675+0000 7f4f5c892640 1 --2- 172.21.15.43:0/775655544 >> v2:172.21.15.193:3300/0 conn(0x7f4f580d19f0 0x7f4f580d1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:57.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.675+0000 7f4f5d894640 1 -- 172.21.15.43:0/775655544 >> v2:172.21.15.193:3300/0 conn(0x7f4f580d19f0 msgr2=0x7f4f580d1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:57.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.675+0000 7f4f5d894640 1 --2- 172.21.15.43:0/775655544 >> v2:172.21.15.193:3300/0 conn(0x7f4f580d19f0 0x7f4f580d1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.675+0000 7f4f5d894640 1 -- 172.21.15.43:0/775655544 >> v1:172.21.15.43:6789/0 conn(0x7f4f58161310 legacy=0x7f4f580d4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:57.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.675+0000 7f4f57fff640 1 -- 172.21.15.43:0/775655544 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 290390264 0 0) 0x7f4f580cf030 con 0x7f4f58161310 2024-04-25T09:02:57.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.675+0000 7f4f5d894640 1 -- 172.21.15.43:0/775655544 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f580cf5f0 con 0x7f4f580d2370 2024-04-25T09:02:57.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.676+0000 7f4f5d894640 1 --2- 172.21.15.43:0/775655544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f580d2370 0x7f4f580c7860 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f4f5808d2c0 tx=0x7f4f4802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d967b6b0ea80f94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:57.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.676+0000 7f4f57fff640 1 -- 172.21.15.43:0/775655544 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f48007740 con 0x7f4f580d2370 2024-04-25T09:02:57.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.676+0000 7f4f57fff640 1 -- 172.21.15.43:0/775655544 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f480078e0 con 0x7f4f580d2370 2024-04-25T09:02:57.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.676+0000 7f4f57fff640 1 -- 172.21.15.43:0/775655544 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f48007cb0 con 0x7f4f580d2370 2024-04-25T09:02:57.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.676+0000 7f4f5f2f7640 1 -- 172.21.15.43:0/775655544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f580d2370 msgr2=0x7f4f580c7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:57.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.676+0000 7f4f5f2f7640 1 --2- 172.21.15.43:0/775655544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f580d2370 0x7f4f580c7860 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f4f5808d2c0 tx=0x7f4f4802a680 comp rx=0 tx=0).stop 2024-04-25T09:02:57.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.676+0000 7f4f5f2f7640 1 -- 172.21.15.43:0/775655544 shutdown_connections 2024-04-25T09:02:57.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.676+0000 7f4f5f2f7640 1 --2- 172.21.15.43:0/775655544 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f580d2370 0x7f4f580c7860 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.676+0000 7f4f5f2f7640 1 --2- 172.21.15.43:0/775655544 >> v2:172.21.15.193:3300/0 conn(0x7f4f580d19f0 0x7f4f580d1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.676+0000 7f4f5f2f7640 1 -- 172.21.15.43:0/775655544 >> 172.21.15.43:0/775655544 conn(0x7f4f5809cd20 msgr2=0x7f4f580d3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:57.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.677+0000 7f4f5f2f7640 1 -- 172.21.15.43:0/775655544 shutdown_connections 2024-04-25T09:02:57.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.677+0000 7f4f5f2f7640 1 -- 172.21.15.43:0/775655544 wait complete. 2024-04-25T09:02:57.667 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.678+0000 7f4f5f2f7640 1 Processor -- start 2024-04-25T09:02:57.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.678+0000 7f4f5f2f7640 1 -- start start 2024-04-25T09:02:57.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.678+0000 7f4f5f2f7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f580d19f0 0x7f4f5815e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:57.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.678+0000 7f4f5f2f7640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4f580d2370 0x7f4f58153ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:57.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.678+0000 7f4f5f2f7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f58161310 0x7f4f5809e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:57.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.678+0000 7f4f5f2f7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f580c7da0 con 0x7f4f580d19f0 2024-04-25T09:02:57.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.678+0000 7f4f5f2f7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f580d27e0 con 0x7f4f58161310 2024-04-25T09:02:57.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.678+0000 7f4f5f2f7640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f580c7f20 con 0x7f4f580d2370 2024-04-25T09:02:57.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.678+0000 7f4f5d093640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f580d19f0 0x7f4f5815e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:57.670 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.679+0000 7f4f5d093640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f580d19f0 0x7f4f5815e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48668/0 (socket says 172.21.15.43:48668) 2024-04-25T09:02:57.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.679+0000 7f4f5d093640 1 -- 172.21.15.43:0/3704995839 learned_addr learned my addr 172.21.15.43:0/3704995839 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:57.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.679+0000 7f4f5c892640 1 --2- 172.21.15.43:0/3704995839 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4f580d2370 0x7f4f58153ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:57.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.679+0000 7f4f5d894640 1 --2- 172.21.15.43:0/3704995839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f58161310 0x7f4f5809e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:57.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.679+0000 7f4f5d093640 1 -- 172.21.15.43:0/3704995839 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4f580d2370 msgr2=0x7f4f58153ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:57.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.679+0000 7f4f5d093640 1 --2- 172.21.15.43:0/3704995839 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4f580d2370 0x7f4f58153ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.679+0000 7f4f5d093640 1 -- 172.21.15.43:0/3704995839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f58161310 msgr2=0x7f4f5809e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:57.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.679+0000 7f4f5d093640 1 --2- 172.21.15.43:0/3704995839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f58161310 0x7f4f5809e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.679+0000 7f4f5d093640 1 -- 172.21.15.43:0/3704995839 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f58154430 con 0x7f4f580d19f0 2024-04-25T09:02:57.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.679+0000 7f4f5d093640 1 --2- 172.21.15.43:0/3704995839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f580d19f0 0x7f4f5815e020 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f4f5000c430 tx=0x7f4f5000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:57.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.679+0000 7f4f55ffb640 1 -- 172.21.15.43:0/3704995839 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f500074e0 con 0x7f4f580d19f0 2024-04-25T09:02:57.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.679+0000 7f4f55ffb640 1 -- 172.21.15.43:0/3704995839 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f500047e0 con 0x7f4f580d19f0 2024-04-25T09:02:57.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.679+0000 7f4f55ffb640 1 -- 172.21.15.43:0/3704995839 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f50004bd0 con 0x7f4f580d19f0 2024-04-25T09:02:57.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.679+0000 7f4f5f2f7640 1 -- 172.21.15.43:0/3704995839 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f580a08c0 con 0x7f4f580d19f0 2024-04-25T09:02:57.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.679+0000 7f4f5f2f7640 1 -- 172.21.15.43:0/3704995839 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f580a0e80 con 0x7f4f580d19f0 2024-04-25T09:02:57.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.680+0000 7f4f5f2f7640 1 -- 172.21.15.43:0/3704995839 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f58074f60 con 0x7f4f580d19f0 2024-04-25T09:02:57.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.681+0000 7f4f55ffb640 1 -- 172.21.15.43:0/3704995839 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4f50011070 con 0x7f4f580d19f0 2024-04-25T09:02:57.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.681+0000 7f4f55ffb640 1 --2- 172.21.15.43:0/3704995839 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4f4406b4d0 0x7f4f4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:57.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.681+0000 7f4f55ffb640 1 -- 172.21.15.43:0/3704995839 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4f50089f90 con 0x7f4f580d19f0 2024-04-25T09:02:57.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.681+0000 7f4f5c892640 1 --2- 172.21.15.43:0/3704995839 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4f4406b4d0 0x7f4f4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:57.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.682+0000 7f4f5c892640 1 --2- 172.21.15.43:0/3704995839 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4f4406b4d0 0x7f4f4406d990 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f4f5815e820 tx=0x7f4f4c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:57.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.686+0000 7f4f55ffb640 1 -- 172.21.15.43:0/3704995839 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4f50058d40 con 0x7f4f580d19f0 2024-04-25T09:02:57.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.868+0000 7f4f5f2f7640 1 -- 172.21.15.43:0/3704995839 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4f580ca140 con 0x7f4f580d19f0 2024-04-25T09:02:57.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.868+0000 7f4f55ffb640 1 -- 172.21.15.43:0/3704995839 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v115) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4f50058640 con 0x7f4f580d19f0 2024-04-25T09:02:57.857 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T09:02:57.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.870+0000 7f4f5f2f7640 1 -- 172.21.15.43:0/3704995839 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4f4406b4d0 msgr2=0x7f4f4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:57.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.870+0000 7f4f5f2f7640 1 --2- 172.21.15.43:0/3704995839 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4f4406b4d0 0x7f4f4406d990 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f4f5815e820 tx=0x7f4f4c008040 comp rx=0 tx=0).stop 2024-04-25T09:02:57.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.870+0000 7f4f5f2f7640 1 -- 172.21.15.43:0/3704995839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f580d19f0 msgr2=0x7f4f5815e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:57.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.870+0000 7f4f5f2f7640 1 --2- 172.21.15.43:0/3704995839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f580d19f0 0x7f4f5815e020 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f4f5000c430 tx=0x7f4f5000c8f0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.870+0000 7f4f5f2f7640 1 -- 172.21.15.43:0/3704995839 shutdown_connections 2024-04-25T09:02:57.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.870+0000 7f4f5f2f7640 1 --2- 172.21.15.43:0/3704995839 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4f4406b4d0 0x7f4f4406d990 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.870+0000 7f4f5f2f7640 1 --2- 172.21.15.43:0/3704995839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4f58161310 0x7f4f5809e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.870+0000 7f4f5f2f7640 1 --2- 172.21.15.43:0/3704995839 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4f580d2370 0x7f4f58153ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.870+0000 7f4f5f2f7640 1 --2- 172.21.15.43:0/3704995839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4f580d19f0 0x7f4f5815e020 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.870+0000 7f4f5f2f7640 1 -- 172.21.15.43:0/3704995839 >> 172.21.15.43:0/3704995839 conn(0x7f4f5809cd20 msgr2=0x7f4f5815f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:57.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.871+0000 7f4f5f2f7640 1 -- 172.21.15.43:0/3704995839 shutdown_connections 2024-04-25T09:02:57.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.871+0000 7f4f5f2f7640 1 -- 172.21.15.43:0/3704995839 wait complete. 2024-04-25T09:02:57.871 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:02:57.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.986+0000 7f3ed67d5640 1 Processor -- start 2024-04-25T09:02:57.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.986+0000 7f3ed67d5640 1 -- start start 2024-04-25T09:02:57.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.987+0000 7f3ed67d5640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f3ed00d04a0 0x7f3ed015ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:57.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.987+0000 7f3ed67d5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3ed015dec0 0x7f3ed00d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:57.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.987+0000 7f3ed67d5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ed00ce8c0 0x7f3ed01617a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:57.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.987+0000 7f3ed67d5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ed00d0dc0 con 0x7f3ed015dec0 2024-04-25T09:02:57.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.987+0000 7f3ed67d5640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ed00cf110 con 0x7f3ed00d04a0 2024-04-25T09:02:57.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.987+0000 7f3ed67d5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed00cf3f0 con 0x7f3ed00ce8c0 2024-04-25T09:02:57.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.987+0000 7f3ecf7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3ed015dec0 0x7f3ed00d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:57.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.987+0000 7f3ecf7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3ed015dec0 0x7f3ed00d0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48678/0 (socket says 172.21.15.43:48678) 2024-04-25T09:02:57.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.987+0000 7f3ecf7fe640 1 -- 172.21.15.43:0/4278212869 learned_addr learned my addr 172.21.15.43:0/4278212869 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:57.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.987+0000 7f3ed4d72640 1 --2- 172.21.15.43:0/4278212869 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ed00ce8c0 0x7f3ed01617a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:57.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.987+0000 7f3ecffff640 1 --2- 172.21.15.43:0/4278212869 >> v2:172.21.15.193:3300/0 conn(0x7f3ed00d04a0 0x7f3ed015ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:57.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.987+0000 7f3ecf7fe640 1 -- 172.21.15.43:0/4278212869 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ed00ce8c0 msgr2=0x7f3ed01617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:57.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.987+0000 7f3ecf7fe640 1 --2- 172.21.15.43:0/4278212869 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ed00ce8c0 0x7f3ed01617a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.987+0000 7f3ecf7fe640 1 -- 172.21.15.43:0/4278212869 >> v2:172.21.15.193:3300/0 conn(0x7f3ed00d04a0 msgr2=0x7f3ed015ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:57.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.987+0000 7f3ecf7fe640 1 --2- 172.21.15.43:0/4278212869 >> v2:172.21.15.193:3300/0 conn(0x7f3ed00d04a0 0x7f3ed015ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.987+0000 7f3ecf7fe640 1 -- 172.21.15.43:0/4278212869 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed00cf6d0 con 0x7f3ed015dec0 2024-04-25T09:02:57.977 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.987+0000 7f3ed4d72640 1 --2- 172.21.15.43:0/4278212869 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ed00ce8c0 0x7f3ed01617a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:57.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.987+0000 7f3ecffff640 1 --2- 172.21.15.43:0/4278212869 >> v2:172.21.15.193:3300/0 conn(0x7f3ed00d04a0 0x7f3ed015ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:57.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.988+0000 7f3ecf7fe640 1 --2- 172.21.15.43:0/4278212869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ed015dec0 0x7f3ed00d0880 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f3ed00b6690 tx=0x7f3ec402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f818990d4d71f706 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:57.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.988+0000 7f3eceffd640 1 -- 172.21.15.43:0/4278212869 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4037070 con 0x7f3ed015dec0 2024-04-25T09:02:57.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.988+0000 7f3eceffd640 1 -- 172.21.15.43:0/4278212869 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec4032440 con 0x7f3ed015dec0 2024-04-25T09:02:57.978 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.988+0000 7f3eceffd640 1 -- 172.21.15.43:0/4278212869 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4032810 con 0x7f3ed015dec0 2024-04-25T09:02:57.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.988+0000 7f3ed67d5640 1 -- 172.21.15.43:0/4278212869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ed015dec0 msgr2=0x7f3ed00d0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:57.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.988+0000 7f3ed67d5640 1 --2- 172.21.15.43:0/4278212869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ed015dec0 0x7f3ed00d0880 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f3ed00b6690 tx=0x7f3ec402a1e0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.988+0000 7f3ed67d5640 1 -- 172.21.15.43:0/4278212869 shutdown_connections 2024-04-25T09:02:57.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.988+0000 7f3ed67d5640 1 --2- 172.21.15.43:0/4278212869 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ed00ce8c0 0x7f3ed01617a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.988+0000 7f3ed67d5640 1 --2- 172.21.15.43:0/4278212869 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ed015dec0 0x7f3ed00d0880 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.988+0000 7f3ed67d5640 1 --2- 172.21.15.43:0/4278212869 >> v2:172.21.15.193:3300/0 conn(0x7f3ed00d04a0 0x7f3ed015ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.988+0000 7f3ed67d5640 1 -- 172.21.15.43:0/4278212869 >> 172.21.15.43:0/4278212869 conn(0x7f3ed009cd40 msgr2=0x7f3ed00d3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:57.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.989+0000 7f3ed67d5640 1 -- 172.21.15.43:0/4278212869 shutdown_connections 2024-04-25T09:02:57.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.989+0000 7f3ed67d5640 1 -- 172.21.15.43:0/4278212869 wait complete. 2024-04-25T09:02:57.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.990+0000 7f3ed67d5640 1 Processor -- start 2024-04-25T09:02:57.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.990+0000 7f3ed67d5640 1 -- start start 2024-04-25T09:02:57.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.991+0000 7f3ed67d5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ed00ce8c0 0x7f3ed014a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:57.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.991+0000 7f3ed67d5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ed00d04a0 0x7f3ed00b2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:57.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.991+0000 7f3ed67d5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ed015dec0 0x7f3ed00b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:57.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.991+0000 7f3ed67d5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed00ced10 con 0x7f3ed015dec0 2024-04-25T09:02:57.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.991+0000 7f3ed67d5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed00cfc90 con 0x7f3ed00ce8c0 2024-04-25T09:02:57.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.991+0000 7f3ed67d5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed00cee90 con 0x7f3ed00d04a0 2024-04-25T09:02:57.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.991+0000 7f3ed4d72640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ed015dec0 0x7f3ed00b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:57.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.991+0000 7f3ed4d72640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ed015dec0 0x7f3ed00b34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48690/0 (socket says 172.21.15.43:48690) 2024-04-25T09:02:57.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.991+0000 7f3ed4d72640 1 -- 172.21.15.43:0/682474912 learned_addr learned my addr 172.21.15.43:0/682474912 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:57.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.991+0000 7f3ecffff640 1 --2- 172.21.15.43:0/682474912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ed00ce8c0 0x7f3ed014a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:57.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.991+0000 7f3ecf7fe640 1 --2- 172.21.15.43:0/682474912 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ed00d04a0 0x7f3ed00b2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:57.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.991+0000 7f3ed4d72640 1 -- 172.21.15.43:0/682474912 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ed00d04a0 msgr2=0x7f3ed00b2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:57.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.991+0000 7f3ed4d72640 1 --2- 172.21.15.43:0/682474912 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ed00d04a0 0x7f3ed00b2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.991+0000 7f3ed4d72640 1 -- 172.21.15.43:0/682474912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ed00ce8c0 msgr2=0x7f3ed014a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:57.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.991+0000 7f3ed4d72640 1 --2- 172.21.15.43:0/682474912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ed00ce8c0 0x7f3ed014a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:57.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.991+0000 7f3ed4d72640 1 -- 172.21.15.43:0/682474912 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed00b5e50 con 0x7f3ed015dec0 2024-04-25T09:02:57.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.991+0000 7f3ed4d72640 1 --2- 172.21.15.43:0/682474912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ed015dec0 0x7f3ed00b34c0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8007db0 tx=0x7f3ec800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:57.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.991+0000 7f3eccff9640 1 -- 172.21.15.43:0/682474912 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec801a070 con 0x7f3ed015dec0 2024-04-25T09:02:57.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.991+0000 7f3eccff9640 1 -- 172.21.15.43:0/682474912 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec8003990 con 0x7f3ed015dec0 2024-04-25T09:02:57.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.991+0000 7f3ed67d5640 1 -- 172.21.15.43:0/682474912 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ed00b6130 con 0x7f3ed015dec0 2024-04-25T09:02:57.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.991+0000 7f3eccff9640 1 -- 172.21.15.43:0/682474912 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec8003d60 con 0x7f3ed015dec0 2024-04-25T09:02:57.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.991+0000 7f3ed67d5640 1 -- 172.21.15.43:0/682474912 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ed00cc120 con 0x7f3ed015dec0 2024-04-25T09:02:57.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.993+0000 7f3ed67d5640 1 -- 172.21.15.43:0/682474912 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ed0074f60 con 0x7f3ed015dec0 2024-04-25T09:02:57.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.993+0000 7f3eccff9640 1 -- 172.21.15.43:0/682474912 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ec801d070 con 0x7f3ed015dec0 2024-04-25T09:02:57.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.993+0000 7f3eccff9640 1 --2- 172.21.15.43:0/682474912 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3ebc06b4d0 0x7f3ebc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:57.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.994+0000 7f3eccff9640 1 -- 172.21.15.43:0/682474912 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3ec808a3b0 con 0x7f3ed015dec0 2024-04-25T09:02:57.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.994+0000 7f3ecffff640 1 --2- 172.21.15.43:0/682474912 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3ebc06b4d0 0x7f3ebc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:57.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.994+0000 7f3ecffff640 1 --2- 172.21.15.43:0/682474912 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3ebc06b4d0 0x7f3ebc06d990 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f3ec00096d0 tx=0x7f3ec0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:57.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:57.998+0000 7f3eccff9640 1 -- 172.21.15.43:0/682474912 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3ec8010070 con 0x7f3ed015dec0 2024-04-25T09:02:58.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.182+0000 7f3ed67d5640 1 -- 172.21.15.43:0/682474912 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3ed008daf0 con 0x7f3ed015dec0 2024-04-25T09:02:58.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.182+0000 7f3eccff9640 1 -- 172.21.15.43:0/682474912 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v115) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3ec8059280 con 0x7f3ed015dec0 2024-04-25T09:02:58.171 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T09:02:58.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.184+0000 7f3ed67d5640 1 -- 172.21.15.43:0/682474912 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3ebc06b4d0 msgr2=0x7f3ebc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:58.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.184+0000 7f3ed67d5640 1 --2- 172.21.15.43:0/682474912 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3ebc06b4d0 0x7f3ebc06d990 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f3ec00096d0 tx=0x7f3ec0008040 comp rx=0 tx=0).stop 2024-04-25T09:02:58.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.184+0000 7f3ed67d5640 1 -- 172.21.15.43:0/682474912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ed015dec0 msgr2=0x7f3ed00b34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:58.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.184+0000 7f3ed67d5640 1 --2- 172.21.15.43:0/682474912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ed015dec0 0x7f3ed00b34c0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f3ec8007db0 tx=0x7f3ec800d4a0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.184+0000 7f3ed67d5640 1 -- 172.21.15.43:0/682474912 shutdown_connections 2024-04-25T09:02:58.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.184+0000 7f3ed67d5640 1 --2- 172.21.15.43:0/682474912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ed015dec0 0x7f3ed00b34c0 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.184+0000 7f3ed67d5640 1 --2- 172.21.15.43:0/682474912 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ed00d04a0 0x7f3ed00b2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.184+0000 7f3ed67d5640 1 --2- 172.21.15.43:0/682474912 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3ebc06b4d0 0x7f3ebc06d990 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.184+0000 7f3ed67d5640 1 --2- 172.21.15.43:0/682474912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ed00ce8c0 0x7f3ed014a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.184+0000 7f3ed67d5640 1 -- 172.21.15.43:0/682474912 >> 172.21.15.43:0/682474912 conn(0x7f3ed009cd40 msgr2=0x7f3ed0160410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:58.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.184+0000 7f3ed67d5640 1 -- 172.21.15.43:0/682474912 shutdown_connections 2024-04-25T09:02:58.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.184+0000 7f3ed67d5640 1 -- 172.21.15.43:0/682474912 wait complete. 2024-04-25T09:02:58.184 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:02:58.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.299+0000 7f03b3276640 1 Processor -- start 2024-04-25T09:02:58.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.300+0000 7f03b3276640 1 -- start start 2024-04-25T09:02:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.300+0000 7f03b3276640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f03ac15e090 0x7f03ac161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.300+0000 7f03b3276640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03ac161fe0 0x7f03ac0b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.300+0000 7f03b3276640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03ac162610 con 0x7f03ac0b5aa0 2024-04-25T09:02:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.300+0000 7f03b3276640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03ac0b5f00 con 0x7f03ac15e090 2024-04-25T09:02:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.300+0000 7f03b3276640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac0d1da0 con 0x7f03ac161fe0 2024-04-25T09:02:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.300+0000 7f03b1012640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f03ac15e090 0x7f03ac161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:58.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.301+0000 7f03b1012640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f03ac15e090 0x7f03ac161aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48704/0 (socket says 172.21.15.43:48704) 2024-04-25T09:02:58.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.301+0000 7f03b1012640 1 -- 172.21.15.43:0/297955887 learned_addr learned my addr 172.21.15.43:0/297955887 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:58.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.301+0000 7f03b0811640 1 --2- 172.21.15.43:0/297955887 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03ac161fe0 0x7f03ac0b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:58.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.301+0000 7f03b1012640 1 -- 172.21.15.43:0/297955887 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03ac161fe0 msgr2=0x7f03ac0b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:58.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.301+0000 7f03b1012640 1 --2- 172.21.15.43:0/297955887 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03ac161fe0 0x7f03ac0b5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.301+0000 7f03b1012640 1 -- 172.21.15.43:0/297955887 >> v1:172.21.15.193:6789/0 conn(0x7f03ac0b5aa0 legacy=0x7f03ac0ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:58.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.301+0000 7f03b1012640 1 -- 172.21.15.43:0/297955887 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03ac0d1750 con 0x7f03ac15e090 2024-04-25T09:02:58.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.301+0000 7f03b1012640 1 --2- 172.21.15.43:0/297955887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03ac15e090 0x7f03ac161aa0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f03a8009870 tx=0x7f03a802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1792800bc7df814d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:58.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.302+0000 7f03a3fff640 1 -- 172.21.15.43:0/297955887 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a8037070 con 0x7f03ac15e090 2024-04-25T09:02:58.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.302+0000 7f03a3fff640 1 -- 172.21.15.43:0/297955887 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a8032440 con 0x7f03ac15e090 2024-04-25T09:02:58.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.302+0000 7f03a3fff640 1 -- 172.21.15.43:0/297955887 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a8032810 con 0x7f03ac15e090 2024-04-25T09:02:58.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.302+0000 7f03b3276640 1 -- 172.21.15.43:0/297955887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03ac15e090 msgr2=0x7f03ac161aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:58.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.302+0000 7f03b3276640 1 --2- 172.21.15.43:0/297955887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03ac15e090 0x7f03ac161aa0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f03a8009870 tx=0x7f03a802a2a0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.302+0000 7f03b3276640 1 -- 172.21.15.43:0/297955887 shutdown_connections 2024-04-25T09:02:58.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.302+0000 7f03b3276640 1 --2- 172.21.15.43:0/297955887 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03ac161fe0 0x7f03ac0b5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.302+0000 7f03b3276640 1 --2- 172.21.15.43:0/297955887 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03ac15e090 0x7f03ac161aa0 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.302+0000 7f03b3276640 1 -- 172.21.15.43:0/297955887 >> 172.21.15.43:0/297955887 conn(0x7f03ac0d0560 msgr2=0x7f03ac0d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:58.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.302+0000 7f03b3276640 1 -- 172.21.15.43:0/297955887 shutdown_connections 2024-04-25T09:02:58.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.302+0000 7f03b3276640 1 -- 172.21.15.43:0/297955887 wait complete. 2024-04-25T09:02:58.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.303+0000 7f03b3276640 1 Processor -- start 2024-04-25T09:02:58.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.304+0000 7f03b3276640 1 -- start start 2024-04-25T09:02:58.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.304+0000 7f03b3276640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03ac0b5aa0 0x7f03ac0cf090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:58.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.304+0000 7f03b3276640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03ac15e090 0x7f03ac1f4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:58.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.304+0000 7f03b3276640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03ac161fe0 0x7f03ac1565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:58.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.304+0000 7f03b3276640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac0b3280 con 0x7f03ac0b5aa0 2024-04-25T09:02:58.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.304+0000 7f03b3276640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac1628f0 con 0x7f03ac15e090 2024-04-25T09:02:58.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.304+0000 7f03b3276640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03ac0d3240 con 0x7f03ac161fe0 2024-04-25T09:02:58.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.304+0000 7f03b1012640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03ac0b5aa0 0x7f03ac0cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:58.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.304+0000 7f03b1012640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03ac0b5aa0 0x7f03ac0cf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48714/0 (socket says 172.21.15.43:48714) 2024-04-25T09:02:58.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.304+0000 7f03b1012640 1 -- 172.21.15.43:0/1042240962 learned_addr learned my addr 172.21.15.43:0/1042240962 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:58.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.305+0000 7f03b0811640 1 --2- 172.21.15.43:0/1042240962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03ac15e090 0x7f03ac1f4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:58.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.305+0000 7f03b1813640 1 --2- 172.21.15.43:0/1042240962 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03ac161fe0 0x7f03ac1565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:58.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.305+0000 7f03b1012640 1 -- 172.21.15.43:0/1042240962 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03ac161fe0 msgr2=0x7f03ac1565a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:58.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.305+0000 7f03b1012640 1 --2- 172.21.15.43:0/1042240962 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03ac161fe0 0x7f03ac1565a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.305+0000 7f03b1012640 1 -- 172.21.15.43:0/1042240962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03ac15e090 msgr2=0x7f03ac1f4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:58.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.305+0000 7f03b1012640 1 --2- 172.21.15.43:0/1042240962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03ac15e090 0x7f03ac1f4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.305+0000 7f03b1012640 1 -- 172.21.15.43:0/1042240962 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03ac158830 con 0x7f03ac0b5aa0 2024-04-25T09:02:58.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.305+0000 7f03b1813640 1 --2- 172.21.15.43:0/1042240962 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03ac161fe0 0x7f03ac1565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:58.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.305+0000 7f03b1012640 1 --2- 172.21.15.43:0/1042240962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03ac0b5aa0 0x7f03ac0cf090 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f03a8009870 tx=0x7f03a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:58.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.305+0000 7f03a1ffb640 1 -- 172.21.15.43:0/1042240962 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a8041020 con 0x7f03ac0b5aa0 2024-04-25T09:02:58.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.305+0000 7f03a1ffb640 1 -- 172.21.15.43:0/1042240962 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a8030970 con 0x7f03ac0b5aa0 2024-04-25T09:02:58.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.305+0000 7f03b3276640 1 -- 172.21.15.43:0/1042240962 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03ac158aa0 con 0x7f03ac0b5aa0 2024-04-25T09:02:58.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.305+0000 7f03a1ffb640 1 -- 172.21.15.43:0/1042240962 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a8037040 con 0x7f03ac0b5aa0 2024-04-25T09:02:58.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.305+0000 7f03b3276640 1 -- 172.21.15.43:0/1042240962 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03ac159060 con 0x7f03ac0b5aa0 2024-04-25T09:02:58.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.306+0000 7f03b3276640 1 -- 172.21.15.43:0/1042240962 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03ac06bf00 con 0x7f03ac0b5aa0 2024-04-25T09:02:58.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.307+0000 7f03a1ffb640 1 -- 172.21.15.43:0/1042240962 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f03a8044020 con 0x7f03ac0b5aa0 2024-04-25T09:02:58.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.307+0000 7f03a1ffb640 1 --2- 172.21.15.43:0/1042240962 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f039006b4d0 0x7f039006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:58.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.307+0000 7f03a1ffb640 1 -- 172.21.15.43:0/1042240962 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f03a80aab90 con 0x7f03ac0b5aa0 2024-04-25T09:02:58.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.307+0000 7f03b0811640 1 --2- 172.21.15.43:0/1042240962 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f039006b4d0 0x7f039006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:58.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.308+0000 7f03b0811640 1 --2- 172.21.15.43:0/1042240962 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f039006b4d0 0x7f039006d990 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f03ac0cf5d0 tx=0x7f039c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:58.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.311+0000 7f03a1ffb640 1 -- 172.21.15.43:0/1042240962 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f03ac06bf00 con 0x7f03ac0b5aa0 2024-04-25T09:02:58.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.490+0000 7f03b3276640 1 -- 172.21.15.43:0/1042240962 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f03ac160e60 con 0x7f03ac0b5aa0 2024-04-25T09:02:58.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.490+0000 7f03a1ffb640 1 -- 172.21.15.43:0/1042240962 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v115) v1 ==== 100+0+11 (secure 0 0 0) 0x7f03a804b030 con 0x7f03ac0b5aa0 2024-04-25T09:02:58.479 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T09:02:58.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.492+0000 7f03b3276640 1 -- 172.21.15.43:0/1042240962 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f039006b4d0 msgr2=0x7f039006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:58.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.492+0000 7f03b3276640 1 --2- 172.21.15.43:0/1042240962 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f039006b4d0 0x7f039006d990 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f03ac0cf5d0 tx=0x7f039c008040 comp rx=0 tx=0).stop 2024-04-25T09:02:58.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.492+0000 7f03b3276640 1 -- 172.21.15.43:0/1042240962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03ac0b5aa0 msgr2=0x7f03ac0cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:58.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.492+0000 7f03b3276640 1 --2- 172.21.15.43:0/1042240962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03ac0b5aa0 0x7f03ac0cf090 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f03a8009870 tx=0x7f03a80046c0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.492+0000 7f03b3276640 1 -- 172.21.15.43:0/1042240962 shutdown_connections 2024-04-25T09:02:58.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.492+0000 7f03b3276640 1 --2- 172.21.15.43:0/1042240962 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f03ac161fe0 0x7f03ac1565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.492+0000 7f03b3276640 1 --2- 172.21.15.43:0/1042240962 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f039006b4d0 0x7f039006d990 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.492+0000 7f03b3276640 1 --2- 172.21.15.43:0/1042240962 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f03ac15e090 0x7f03ac1f4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.492+0000 7f03b3276640 1 --2- 172.21.15.43:0/1042240962 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f03ac0b5aa0 0x7f03ac0cf090 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.492+0000 7f03b3276640 1 -- 172.21.15.43:0/1042240962 >> 172.21.15.43:0/1042240962 conn(0x7f03ac0d0560 msgr2=0x7f03ac159b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:58.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.492+0000 7f03b3276640 1 -- 172.21.15.43:0/1042240962 shutdown_connections 2024-04-25T09:02:58.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.492+0000 7f03b3276640 1 -- 172.21.15.43:0/1042240962 wait complete. 2024-04-25T09:02:58.492 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:02:58.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.604+0000 7f4c4fde7640 1 Processor -- start 2024-04-25T09:02:58.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.605+0000 7f4c4fde7640 1 -- start start 2024-04-25T09:02:58.594 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.605+0000 7f4c4fde7640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4c480c8d00 0x7f4c48162100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:58.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.605+0000 7f4c4fde7640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c480d24a0 con 0x7f4c480ce660 2024-04-25T09:02:58.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.605+0000 7f4c4fde7640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c480d2740 con 0x7f4c4815df50 2024-04-25T09:02:58.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.605+0000 7f4c4fde7640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c4815e330 con 0x7f4c480c8d00 2024-04-25T09:02:58.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.605+0000 7f4c4d382640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4c480c8d00 0x7f4c48162100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:58.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.606+0000 7f4c4e384640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4c480ce660 0x7f4c480cea70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:53486/0 (socket says 172.21.15.43:53486) 2024-04-25T09:02:58.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.606+0000 7f4c4e384640 1 -- 172.21.15.43:0/120206266 learned_addr learned my addr 172.21.15.43:0/120206266 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:58.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.606+0000 7f4c4d382640 1 -- 172.21.15.43:0/120206266 >> v1:172.21.15.193:6789/0 conn(0x7f4c4815df50 legacy=0x7f4c481619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:58.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.606+0000 7f4c4d382640 1 -- 172.21.15.43:0/120206266 >> v1:172.21.15.43:6789/0 conn(0x7f4c480ce660 legacy=0x7f4c480cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:58.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.606+0000 7f4c4cb81640 1 -- 172.21.15.43:0/120206266 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2253938015 0 0) 0x7f4c480d24a0 con 0x7f4c480ce660 2024-04-25T09:02:58.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.606+0000 7f4c4d382640 1 -- 172.21.15.43:0/120206266 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c480d2a20 con 0x7f4c480c8d00 2024-04-25T09:02:58.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.607+0000 7f4c4d382640 1 --2- 172.21.15.43:0/120206266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c480c8d00 0x7f4c48162100 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f4c380097a0 tx=0x7f4c3802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74957238aca7ada9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:58.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.607+0000 7f4c4cb81640 1 -- 172.21.15.43:0/120206266 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c38037070 con 0x7f4c480c8d00 2024-04-25T09:02:58.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.607+0000 7f4c4cb81640 1 -- 172.21.15.43:0/120206266 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c38032440 con 0x7f4c480c8d00 2024-04-25T09:02:58.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.607+0000 7f4c4cb81640 1 -- 172.21.15.43:0/120206266 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c38032810 con 0x7f4c480c8d00 2024-04-25T09:02:58.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.607+0000 7f4c4fde7640 1 -- 172.21.15.43:0/120206266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c480c8d00 msgr2=0x7f4c48162100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:58.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.607+0000 7f4c4fde7640 1 --2- 172.21.15.43:0/120206266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c480c8d00 0x7f4c48162100 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f4c380097a0 tx=0x7f4c3802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.607+0000 7f4c4fde7640 1 -- 172.21.15.43:0/120206266 shutdown_connections 2024-04-25T09:02:58.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.607+0000 7f4c4fde7640 1 --2- 172.21.15.43:0/120206266 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c480c8d00 0x7f4c48162100 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.607+0000 7f4c4fde7640 1 -- 172.21.15.43:0/120206266 >> 172.21.15.43:0/120206266 conn(0x7f4c480d3a30 msgr2=0x7f4c480d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:58.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.607+0000 7f4c4fde7640 1 -- 172.21.15.43:0/120206266 shutdown_connections 2024-04-25T09:02:58.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.607+0000 7f4c4fde7640 1 -- 172.21.15.43:0/120206266 wait complete. 2024-04-25T09:02:58.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.608+0000 7f4c4fde7640 1 Processor -- start 2024-04-25T09:02:58.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.609+0000 7f4c4fde7640 1 -- start start 2024-04-25T09:02:58.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.609+0000 7f4c4fde7640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4c480c8d00 0x7f4c480d0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:58.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.609+0000 7f4c4fde7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c480ce660 0x7f4c4814fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:58.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.609+0000 7f4c4fde7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c4815df50 0x7f4c48150160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:58.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.609+0000 7f4c4fde7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c480d2190 con 0x7f4c480ce660 2024-04-25T09:02:58.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.609+0000 7f4c4fde7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c480c77d0 con 0x7f4c4815df50 2024-04-25T09:02:58.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.609+0000 7f4c4fde7640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c48162640 con 0x7f4c480c8d00 2024-04-25T09:02:58.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.609+0000 7f4c4d382640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c480ce660 0x7f4c4814fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:58.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.609+0000 7f4c4d382640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c480ce660 0x7f4c4814fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48736/0 (socket says 172.21.15.43:48736) 2024-04-25T09:02:58.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.609+0000 7f4c4d382640 1 -- 172.21.15.43:0/1278153381 learned_addr learned my addr 172.21.15.43:0/1278153381 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:58.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.610+0000 7f4c4e384640 1 --2- 172.21.15.43:0/1278153381 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c4815df50 0x7f4c48150160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:58.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.610+0000 7f4c4db83640 1 --2- 172.21.15.43:0/1278153381 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4c480c8d00 0x7f4c480d0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:58.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.610+0000 7f4c4d382640 1 -- 172.21.15.43:0/1278153381 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4c480c8d00 msgr2=0x7f4c480d0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:58.600 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.610+0000 7f4c4d382640 1 --2- 172.21.15.43:0/1278153381 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4c480c8d00 0x7f4c480d0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.610+0000 7f4c4d382640 1 -- 172.21.15.43:0/1278153381 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c4815df50 msgr2=0x7f4c48150160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:58.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.610+0000 7f4c4d382640 1 --2- 172.21.15.43:0/1278153381 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c4815df50 0x7f4c48150160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.610+0000 7f4c4d382640 1 -- 172.21.15.43:0/1278153381 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c481f4680 con 0x7f4c480ce660 2024-04-25T09:02:58.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.610+0000 7f4c4e384640 1 --2- 172.21.15.43:0/1278153381 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c4815df50 0x7f4c48150160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:58.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.610+0000 7f4c4db83640 1 --2- 172.21.15.43:0/1278153381 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4c480c8d00 0x7f4c480d0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:02:58.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.610+0000 7f4c4d382640 1 --2- 172.21.15.43:0/1278153381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c480ce660 0x7f4c4814fc20 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f4c380097a0 tx=0x7f4c380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:58.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.610+0000 7f4c3e7fc640 1 -- 172.21.15.43:0/1278153381 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c38041070 con 0x7f4c480ce660 2024-04-25T09:02:58.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.610+0000 7f4c3e7fc640 1 -- 172.21.15.43:0/1278153381 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c3803c870 con 0x7f4c480ce660 2024-04-25T09:02:58.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.610+0000 7f4c4fde7640 1 -- 172.21.15.43:0/1278153381 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c48155960 con 0x7f4c480ce660 2024-04-25T09:02:58.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.610+0000 7f4c3e7fc640 1 -- 172.21.15.43:0/1278153381 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c38037040 con 0x7f4c480ce660 2024-04-25T09:02:58.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.610+0000 7f4c4fde7640 1 -- 172.21.15.43:0/1278153381 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c48155f20 con 0x7f4c480ce660 2024-04-25T09:02:58.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.611+0000 7f4c4fde7640 1 -- 172.21.15.43:0/1278153381 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c480680f0 con 0x7f4c480ce660 2024-04-25T09:02:58.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.612+0000 7f4c3e7fc640 1 -- 172.21.15.43:0/1278153381 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4c38044070 con 0x7f4c480ce660 2024-04-25T09:02:58.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.612+0000 7f4c3e7fc640 1 --2- 172.21.15.43:0/1278153381 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4c2c06b4d0 0x7f4c2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:58.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.612+0000 7f4c3e7fc640 1 -- 172.21.15.43:0/1278153381 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4c380aae30 con 0x7f4c480ce660 2024-04-25T09:02:58.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.612+0000 7f4c4db83640 1 --2- 172.21.15.43:0/1278153381 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4c2c06b4d0 0x7f4c2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:58.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.613+0000 7f4c4db83640 1 --2- 172.21.15.43:0/1278153381 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4c2c06b4d0 0x7f4c2c06d990 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f4c480c6ec0 tx=0x7f4c44008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:58.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.616+0000 7f4c3e7fc640 1 -- 172.21.15.43:0/1278153381 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4c38079c70 con 0x7f4c480ce660 2024-04-25T09:02:58.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.840+0000 7f4c4fde7640 1 -- 172.21.15.43:0/1278153381 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c48074ef0 con 0x7f4c480ce660 2024-04-25T09:02:58.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.840+0000 7f4c3e7fc640 1 -- 172.21.15.43:0/1278153381 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f4c38079570 con 0x7f4c480ce660 2024-04-25T09:02:58.829 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:58.829 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":69,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6821,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2021228703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2021228703},{"type":"v1","addr":"172.21.15.193:6837","nonce":2021228703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65},{"gid":7543,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/2398324998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2398324998},{"type":"v1","addr":"172.21.15.193:6836","nonce":2398324998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65},{"gid":8673,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2298187831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2298187831},{"type":"v1","addr":"172.21.15.43:6834","nonce":2298187831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:02:26.161487+0000","modified":"2024-04-25T09:02:29.177294+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8676":{"gid":8676,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/1586865031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1586865031},{"type":"v1","addr":"172.21.15.43:6835","nonce":1586865031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:02:58.829 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 69 2024-04-25T09:02:58.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.843+0000 7f4c4fde7640 1 -- 172.21.15.43:0/1278153381 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4c2c06b4d0 msgr2=0x7f4c2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:58.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.843+0000 7f4c4fde7640 1 --2- 172.21.15.43:0/1278153381 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4c2c06b4d0 0x7f4c2c06d990 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f4c480c6ec0 tx=0x7f4c44008040 comp rx=0 tx=0).stop 2024-04-25T09:02:58.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.843+0000 7f4c4fde7640 1 -- 172.21.15.43:0/1278153381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c480ce660 msgr2=0x7f4c4814fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:58.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.843+0000 7f4c4fde7640 1 --2- 172.21.15.43:0/1278153381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c480ce660 0x7f4c4814fc20 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f4c380097a0 tx=0x7f4c380046c0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.843+0000 7f4c4fde7640 1 -- 172.21.15.43:0/1278153381 shutdown_connections 2024-04-25T09:02:58.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.843+0000 7f4c4fde7640 1 --2- 172.21.15.43:0/1278153381 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4c2c06b4d0 0x7f4c2c06d990 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.843+0000 7f4c4fde7640 1 --2- 172.21.15.43:0/1278153381 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c4815df50 0x7f4c48150160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.843+0000 7f4c4fde7640 1 --2- 172.21.15.43:0/1278153381 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c480ce660 0x7f4c4814fc20 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.843+0000 7f4c4fde7640 1 --2- 172.21.15.43:0/1278153381 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4c480c8d00 0x7f4c480d0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.843+0000 7f4c4fde7640 1 -- 172.21.15.43:0/1278153381 >> 172.21.15.43:0/1278153381 conn(0x7f4c480d3a30 msgr2=0x7f4c48152dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:58.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.843+0000 7f4c4fde7640 1 -- 172.21.15.43:0/1278153381 shutdown_connections 2024-04-25T09:02:58.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.843+0000 7f4c4fde7640 1 -- 172.21.15.43:0/1278153381 wait complete. 2024-04-25T09:02:58.843 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:02:58.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.956+0000 7f7bb209e640 1 Processor -- start 2024-04-25T09:02:58.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.956+0000 7f7bb209e640 1 -- start start 2024-04-25T09:02:58.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.957+0000 7f7bb209e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7bac0d0af0 0x7f7bac160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:58.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.957+0000 7f7bb209e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bac0d2920 con 0x7f7bac15dce0 2024-04-25T09:02:58.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.957+0000 7f7bb209e640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bac0cee80 con 0x7f7bac15ef50 2024-04-25T09:02:58.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.957+0000 7f7bb209e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bac0cec60 con 0x7f7bac0d0af0 2024-04-25T09:02:58.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.957+0000 7f7babfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7bac0d0af0 0x7f7bac160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:58.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.957+0000 7f7babfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7bac0d0af0 0x7f7bac160980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48740/0 (socket says 172.21.15.43:48740) 2024-04-25T09:02:58.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.957+0000 7f7bab7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f7bac15dce0 0x7f7bac0ce550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:53496/0 (socket says 172.21.15.43:53496) 2024-04-25T09:02:58.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.957+0000 7f7babfff640 1 -- 172.21.15.43:0/3741509429 learned_addr learned my addr 172.21.15.43:0/3741509429 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:58.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.957+0000 7f7babfff640 1 -- 172.21.15.43:0/3741509429 >> v1:172.21.15.193:6789/0 conn(0x7f7bac15ef50 legacy=0x7f7bac0d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:02:58.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.957+0000 7f7babfff640 1 -- 172.21.15.43:0/3741509429 >> v1:172.21.15.43:6789/0 conn(0x7f7bac15dce0 legacy=0x7f7bac0ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:58.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.957+0000 7f7babfff640 1 -- 172.21.15.43:0/3741509429 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bac0cf160 con 0x7f7bac0d0af0 2024-04-25T09:02:58.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.958+0000 7f7babfff640 1 --2- 172.21.15.43:0/3741509429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bac0d0af0 0x7f7bac160980 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4009e50 tx=0x7f7ba402a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7065993a7af69b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:58.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.958+0000 7f7baa7fc640 1 -- 172.21.15.43:0/3741509429 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba4036070 con 0x7f7bac0d0af0 2024-04-25T09:02:58.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.958+0000 7f7baa7fc640 1 -- 172.21.15.43:0/3741509429 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba4031600 con 0x7f7bac0d0af0 2024-04-25T09:02:58.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.958+0000 7f7baa7fc640 1 -- 172.21.15.43:0/3741509429 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba40319d0 con 0x7f7bac0d0af0 2024-04-25T09:02:58.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.959+0000 7f7bb209e640 1 -- 172.21.15.43:0/3741509429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bac0d0af0 msgr2=0x7f7bac160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:58.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.959+0000 7f7bb209e640 1 --2- 172.21.15.43:0/3741509429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bac0d0af0 0x7f7bac160980 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4009e50 tx=0x7f7ba402a6d0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.959+0000 7f7bb209e640 1 -- 172.21.15.43:0/3741509429 shutdown_connections 2024-04-25T09:02:58.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.959+0000 7f7bb209e640 1 --2- 172.21.15.43:0/3741509429 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bac0d0af0 0x7f7bac160980 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.959+0000 7f7bb209e640 1 -- 172.21.15.43:0/3741509429 >> 172.21.15.43:0/3741509429 conn(0x7f7bac09cd00 msgr2=0x7f7bac0d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:58.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.959+0000 7f7bb209e640 1 -- 172.21.15.43:0/3741509429 shutdown_connections 2024-04-25T09:02:58.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.959+0000 7f7bb209e640 1 -- 172.21.15.43:0/3741509429 wait complete. 2024-04-25T09:02:58.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.960+0000 7f7bb209e640 1 Processor -- start 2024-04-25T09:02:58.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.960+0000 7f7bb209e640 1 -- start start 2024-04-25T09:02:58.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.961+0000 7f7bb209e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7bac0d0af0 0x7f7bac09e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:58.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.961+0000 7f7bb209e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7bac15dce0 0x7f7bac151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:58.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.961+0000 7f7bb209e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bac15ef50 0x7f7bac1501e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:58.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.961+0000 7f7bb209e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bac15f6b0 con 0x7f7bac15ef50 2024-04-25T09:02:58.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.961+0000 7f7bb209e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bac0cf720 con 0x7f7bac0d0af0 2024-04-25T09:02:58.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.961+0000 7f7bb209e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bac15e0c0 con 0x7f7bac15dce0 2024-04-25T09:02:58.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.961+0000 7f7babfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bac15ef50 0x7f7bac1501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:58.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.961+0000 7f7babfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bac15ef50 0x7f7bac1501e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48754/0 (socket says 172.21.15.43:48754) 2024-04-25T09:02:58.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.961+0000 7f7babfff640 1 -- 172.21.15.43:0/2852897315 learned_addr learned my addr 172.21.15.43:0/2852897315 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:58.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.961+0000 7f7bab7fe640 1 --2- 172.21.15.43:0/2852897315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7bac0d0af0 0x7f7bac09e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:58.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.961+0000 7f7baaffd640 1 --2- 172.21.15.43:0/2852897315 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7bac15dce0 0x7f7bac151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:58.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.961+0000 7f7babfff640 1 -- 172.21.15.43:0/2852897315 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7bac15dce0 msgr2=0x7f7bac151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:58.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.961+0000 7f7babfff640 1 --2- 172.21.15.43:0/2852897315 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7bac15dce0 0x7f7bac151ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.961+0000 7f7babfff640 1 -- 172.21.15.43:0/2852897315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7bac0d0af0 msgr2=0x7f7bac09e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:58.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.961+0000 7f7babfff640 1 --2- 172.21.15.43:0/2852897315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7bac0d0af0 0x7f7bac09e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:58.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.961+0000 7f7babfff640 1 -- 172.21.15.43:0/2852897315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bac151ff0 con 0x7f7bac15ef50 2024-04-25T09:02:58.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.962+0000 7f7babfff640 1 --2- 172.21.15.43:0/2852897315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bac15ef50 0x7f7bac1501e0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4009e50 tx=0x7f7ba4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:58.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.962+0000 7f7b93fff640 1 -- 172.21.15.43:0/2852897315 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba4036050 con 0x7f7bac15ef50 2024-04-25T09:02:58.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.962+0000 7f7b93fff640 1 -- 172.21.15.43:0/2852897315 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba4042070 con 0x7f7bac15ef50 2024-04-25T09:02:58.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.962+0000 7f7b93fff640 1 -- 172.21.15.43:0/2852897315 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba4009510 con 0x7f7bac15ef50 2024-04-25T09:02:58.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.962+0000 7f7bb209e640 1 -- 172.21.15.43:0/2852897315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bac150a10 con 0x7f7bac15ef50 2024-04-25T09:02:58.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.962+0000 7f7bb209e640 1 -- 172.21.15.43:0/2852897315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bac150fd0 con 0x7f7bac15ef50 2024-04-25T09:02:58.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.963+0000 7f7bb209e640 1 -- 172.21.15.43:0/2852897315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bac0680f0 con 0x7f7bac15ef50 2024-04-25T09:02:58.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.963+0000 7f7b93fff640 1 -- 172.21.15.43:0/2852897315 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7ba4007bd0 con 0x7f7bac15ef50 2024-04-25T09:02:58.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.964+0000 7f7b93fff640 1 --2- 172.21.15.43:0/2852897315 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7b9806b4d0 0x7f7b9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:58.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.964+0000 7f7b93fff640 1 -- 172.21.15.43:0/2852897315 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f7ba40ab0b0 con 0x7f7bac15ef50 2024-04-25T09:02:58.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.964+0000 7f7bab7fe640 1 --2- 172.21.15.43:0/2852897315 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7b9806b4d0 0x7f7b9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:58.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.964+0000 7f7bab7fe640 1 --2- 172.21.15.43:0/2852897315 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7b9806b4d0 0x7f7b9806d990 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c000f60 tx=0x7f7b9c007e20 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:58.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:58.968+0000 7f7b93fff640 1 -- 172.21.15.43:0/2852897315 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7ba40798a0 con 0x7f7bac15ef50 2024-04-25T09:02:59.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.147+0000 7f7bb209e640 1 -- 172.21.15.43:0/2852897315 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7bac09d1a0 con 0x7f7bac15ef50 2024-04-25T09:02:59.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.148+0000 7f7b93fff640 1 -- 172.21.15.43:0/2852897315 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) v1 ==== 74+0+16744 (secure 0 0 0) 0x7f7ba407d940 con 0x7f7bac15ef50 2024-04-25T09:02:59.136 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:59.137 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":115,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:02:54.872758+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:02:24.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T09:02:25.143723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.e","mappings":[{"from":0,"to":5}]},{"pgid":"17.3f","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:02:59.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.150+0000 7f7bb209e640 1 -- 172.21.15.43:0/2852897315 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7b9806b4d0 msgr2=0x7f7b9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:59.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.150+0000 7f7bb209e640 1 --2- 172.21.15.43:0/2852897315 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7b9806b4d0 0x7f7b9806d990 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c000f60 tx=0x7f7b9c007e20 comp rx=0 tx=0).stop 2024-04-25T09:02:59.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.150+0000 7f7bb209e640 1 -- 172.21.15.43:0/2852897315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bac15ef50 msgr2=0x7f7bac1501e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:59.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.150+0000 7f7bb209e640 1 --2- 172.21.15.43:0/2852897315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bac15ef50 0x7f7bac1501e0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4009e50 tx=0x7f7ba4004a40 comp rx=0 tx=0).stop 2024-04-25T09:02:59.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.150+0000 7f7bb209e640 1 -- 172.21.15.43:0/2852897315 shutdown_connections 2024-04-25T09:02:59.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.150+0000 7f7bb209e640 1 --2- 172.21.15.43:0/2852897315 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7bac15ef50 0x7f7bac1501e0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:59.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.150+0000 7f7bb209e640 1 --2- 172.21.15.43:0/2852897315 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7bac15dce0 0x7f7bac151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:59.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.150+0000 7f7bb209e640 1 --2- 172.21.15.43:0/2852897315 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7b9806b4d0 0x7f7b9806d990 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:59.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.150+0000 7f7bb209e640 1 --2- 172.21.15.43:0/2852897315 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7bac0d0af0 0x7f7bac09e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:59.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.150+0000 7f7bb209e640 1 -- 172.21.15.43:0/2852897315 >> 172.21.15.43:0/2852897315 conn(0x7f7bac09cd00 msgr2=0x7f7bac06b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:59.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.150+0000 7f7bb209e640 1 -- 172.21.15.43:0/2852897315 shutdown_connections 2024-04-25T09:02:59.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.150+0000 7f7bb209e640 1 -- 172.21.15.43:0/2852897315 wait complete. 2024-04-25T09:02:59.150 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:02:59.150 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:02:59.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.263+0000 7f26fcacb640 1 Processor -- start 2024-04-25T09:02:59.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.263+0000 7f26fcacb640 1 -- start start 2024-04-25T09:02:59.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.263+0000 7f26fcacb640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f26f80cf670 0x7f26f8168d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:59.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.263+0000 7f26fcacb640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26f8162810 con 0x7f26f80c8d00 2024-04-25T09:02:59.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.263+0000 7f26fcacb640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26f80d24a0 con 0x7f26f815df50 2024-04-25T09:02:59.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.263+0000 7f26fcacb640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f815e330 con 0x7f26f80cf670 2024-04-25T09:02:59.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.264+0000 7f26effff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f26f80c8d00 0x7f26f8162100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:53512/0 (socket says 172.21.15.43:53512) 2024-04-25T09:02:59.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.264+0000 7f26effff640 1 -- 172.21.15.43:0/2853836719 learned_addr learned my addr 172.21.15.43:0/2853836719 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:59.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.264+0000 7f26f6d9d640 1 --2- 172.21.15.43:0/2853836719 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f26f80cf670 0x7f26f8168d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:59.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.264+0000 7f26f5d9b640 1 -- 172.21.15.43:0/2853836719 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1879429713 0 0) 0x7f26f8162810 con 0x7f26f80c8d00 2024-04-25T09:02:59.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.264+0000 7f26f5d9b640 1 -- 172.21.15.43:0/2853836719 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26e4003520 con 0x7f26f80c8d00 2024-04-25T09:02:59.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.264+0000 7f26f5d9b640 1 -- 172.21.15.43:0/2853836719 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 275148621 0 0) 0x7f26f80d24a0 con 0x7f26f815df50 2024-04-25T09:02:59.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.264+0000 7f26f5d9b640 1 -- 172.21.15.43:0/2853836719 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26f8162810 con 0x7f26f815df50 2024-04-25T09:02:59.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.264+0000 7f26f6d9d640 1 -- 172.21.15.43:0/2853836719 >> v1:172.21.15.193:6789/0 conn(0x7f26f815df50 legacy=0x7f26f81619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:59.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.264+0000 7f26f6d9d640 1 -- 172.21.15.43:0/2853836719 >> v1:172.21.15.43:6789/0 conn(0x7f26f80c8d00 legacy=0x7f26f8162100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:59.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.264+0000 7f26f5d9b640 1 -- 172.21.15.43:0/2853836719 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3206811874 0 0) 0x7f26e4003520 con 0x7f26f80c8d00 2024-04-25T09:02:59.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.264+0000 7f26f6d9d640 1 -- 172.21.15.43:0/2853836719 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f80d2740 con 0x7f26f80cf670 2024-04-25T09:02:59.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.265+0000 7f26f6d9d640 1 --2- 172.21.15.43:0/2853836719 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f26f80cf670 0x7f26f8168d20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f26f0009dc0 tx=0x7f26f002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71b97f70cbd76690 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:59.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.265+0000 7f26f5d9b640 1 -- 172.21.15.43:0/2853836719 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e0001670 con 0x7f26f80cf670 2024-04-25T09:02:59.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.265+0000 7f26f5d9b640 1 -- 172.21.15.43:0/2853836719 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f0007540 con 0x7f26f80cf670 2024-04-25T09:02:59.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.265+0000 7f26f5d9b640 1 -- 172.21.15.43:0/2853836719 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f0007910 con 0x7f26f80cf670 2024-04-25T09:02:59.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.266+0000 7f26fcacb640 1 -- 172.21.15.43:0/2853836719 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f26f80cf670 msgr2=0x7f26f8168d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:59.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.266+0000 7f26fcacb640 1 --2- 172.21.15.43:0/2853836719 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f26f80cf670 0x7f26f8168d20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f26f0009dc0 tx=0x7f26f002a660 comp rx=0 tx=0).stop 2024-04-25T09:02:59.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.266+0000 7f26fcacb640 1 -- 172.21.15.43:0/2853836719 shutdown_connections 2024-04-25T09:02:59.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.266+0000 7f26fcacb640 1 --2- 172.21.15.43:0/2853836719 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f26f80cf670 0x7f26f8168d20 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:59.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.266+0000 7f26fcacb640 1 -- 172.21.15.43:0/2853836719 >> 172.21.15.43:0/2853836719 conn(0x7f26f80d3a30 msgr2=0x7f26f80d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:59.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.266+0000 7f26fcacb640 1 -- 172.21.15.43:0/2853836719 shutdown_connections 2024-04-25T09:02:59.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.266+0000 7f26fcacb640 1 -- 172.21.15.43:0/2853836719 wait complete. 2024-04-25T09:02:59.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.267+0000 7f26fcacb640 1 Processor -- start 2024-04-25T09:02:59.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.267+0000 7f26fcacb640 1 -- start start 2024-04-25T09:02:59.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.268+0000 7f26fcacb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26f80c8d00 0x7f26f81f4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:59.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.268+0000 7f26fcacb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26f80cf670 0x7f26f81f5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:59.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.268+0000 7f26fcacb640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f26f815df50 0x7f26f81f57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:59.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.268+0000 7f26fcacb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f80c77d0 con 0x7f26f80c8d00 2024-04-25T09:02:59.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.268+0000 7f26fcacb640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26f80c74f0 con 0x7f26f80cf670 2024-04-25T09:02:59.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.268+0000 7f26fcacb640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26f80d2190 con 0x7f26f815df50 2024-04-25T09:02:59.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.268+0000 7f26f659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26f80c8d00 0x7f26f81f4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:59.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.268+0000 7f26f659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26f80c8d00 0x7f26f81f4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:48764/0 (socket says 172.21.15.43:48764) 2024-04-25T09:02:59.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.268+0000 7f26f659c640 1 -- 172.21.15.43:0/992838277 learned_addr learned my addr 172.21.15.43:0/992838277 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:59.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.268+0000 7f26f6d9d640 1 --2- 172.21.15.43:0/992838277 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f26f815df50 0x7f26f81f57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:59.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.268+0000 7f26effff640 1 --2- 172.21.15.43:0/992838277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26f80cf670 0x7f26f81f5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:59.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.268+0000 7f26f659c640 1 -- 172.21.15.43:0/992838277 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f26f815df50 msgr2=0x7f26f81f57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:59.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.268+0000 7f26f659c640 1 --2- 172.21.15.43:0/992838277 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f26f815df50 0x7f26f81f57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:59.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.268+0000 7f26f659c640 1 -- 172.21.15.43:0/992838277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26f80cf670 msgr2=0x7f26f81f5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:59.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.268+0000 7f26f659c640 1 --2- 172.21.15.43:0/992838277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26f80cf670 0x7f26f81f5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:59.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.268+0000 7f26f659c640 1 -- 172.21.15.43:0/992838277 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f81f4f90 con 0x7f26f80c8d00 2024-04-25T09:02:59.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.268+0000 7f26f659c640 1 --2- 172.21.15.43:0/992838277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26f80c8d00 0x7f26f81f4a50 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f26e800c430 tx=0x7f26e800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:59.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.269+0000 7f26eeffd640 1 -- 172.21.15.43:0/992838277 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e8017070 con 0x7f26f80c8d00 2024-04-25T09:02:59.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.269+0000 7f26eeffd640 1 -- 172.21.15.43:0/992838277 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e80052f0 con 0x7f26f80c8d00 2024-04-25T09:02:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.269+0000 7f26fcacb640 1 -- 172.21.15.43:0/992838277 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26f815a7d0 con 0x7f26f80c8d00 2024-04-25T09:02:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.269+0000 7f26eeffd640 1 -- 172.21.15.43:0/992838277 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e8003830 con 0x7f26f80c8d00 2024-04-25T09:02:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.269+0000 7f26fcacb640 1 -- 172.21.15.43:0/992838277 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26f80cb890 con 0x7f26f80c8d00 2024-04-25T09:02:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.270+0000 7f26fcacb640 1 -- 172.21.15.43:0/992838277 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26f80680f0 con 0x7f26f80c8d00 2024-04-25T09:02:59.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.270+0000 7f26eeffd640 1 -- 172.21.15.43:0/992838277 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f26e801a070 con 0x7f26f80c8d00 2024-04-25T09:02:59.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.270+0000 7f26eeffd640 1 --2- 172.21.15.43:0/992838277 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f26d806b4d0 0x7f26d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:59.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.271+0000 7f26eeffd640 1 -- 172.21.15.43:0/992838277 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f26e8089580 con 0x7f26f80c8d00 2024-04-25T09:02:59.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.271+0000 7f26effff640 1 --2- 172.21.15.43:0/992838277 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f26d806b4d0 0x7f26d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:59.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.271+0000 7f26effff640 1 --2- 172.21.15.43:0/992838277 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f26d806b4d0 0x7f26d806d990 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f26e0001eb0 tx=0x7f26e00016d0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:59.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.275+0000 7f26eeffd640 1 -- 172.21.15.43:0/992838277 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f26f81f4f90 con 0x7f26f80c8d00 2024-04-25T09:02:59.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.491+0000 7f26fcacb640 1 -- 172.21.15.43:0/992838277 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26f8074ef0 con 0x7f26f80c8d00 2024-04-25T09:02:59.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.491+0000 7f26eeffd640 1 -- 172.21.15.43:0/992838277 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f26e805e0d0 con 0x7f26f80c8d00 2024-04-25T09:02:59.480 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:59.480 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":69,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6821,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2021228703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2021228703},{"type":"v1","addr":"172.21.15.193:6837","nonce":2021228703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65},{"gid":7543,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/2398324998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2398324998},{"type":"v1","addr":"172.21.15.193:6836","nonce":2398324998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65},{"gid":8673,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2298187831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2298187831},{"type":"v1","addr":"172.21.15.43:6834","nonce":2298187831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:02:26.161487+0000","modified":"2024-04-25T09:02:29.177294+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8676":{"gid":8676,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/1586865031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1586865031},{"type":"v1","addr":"172.21.15.43:6835","nonce":1586865031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:02:59.480 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 69 2024-04-25T09:02:59.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.493+0000 7f26fcacb640 1 -- 172.21.15.43:0/992838277 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f26d806b4d0 msgr2=0x7f26d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:59.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.493+0000 7f26fcacb640 1 --2- 172.21.15.43:0/992838277 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f26d806b4d0 0x7f26d806d990 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f26e0001eb0 tx=0x7f26e00016d0 comp rx=0 tx=0).stop 2024-04-25T09:02:59.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.493+0000 7f26fcacb640 1 -- 172.21.15.43:0/992838277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26f80c8d00 msgr2=0x7f26f81f4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:59.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.493+0000 7f26fcacb640 1 --2- 172.21.15.43:0/992838277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26f80c8d00 0x7f26f81f4a50 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f26e800c430 tx=0x7f26e800c8f0 comp rx=0 tx=0).stop 2024-04-25T09:02:59.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.493+0000 7f26fcacb640 1 -- 172.21.15.43:0/992838277 shutdown_connections 2024-04-25T09:02:59.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.493+0000 7f26fcacb640 1 --2- 172.21.15.43:0/992838277 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f26f815df50 0x7f26f81f57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:59.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.493+0000 7f26fcacb640 1 --2- 172.21.15.43:0/992838277 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f26d806b4d0 0x7f26d806d990 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:59.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.493+0000 7f26fcacb640 1 --2- 172.21.15.43:0/992838277 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f26f80cf670 0x7f26f81f5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:59.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.493+0000 7f26fcacb640 1 --2- 172.21.15.43:0/992838277 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f26f80c8d00 0x7f26f81f4a50 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:59.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.493+0000 7f26fcacb640 1 -- 172.21.15.43:0/992838277 >> 172.21.15.43:0/992838277 conn(0x7f26f80d3a30 msgr2=0x7f26f80d0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:59.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.493+0000 7f26fcacb640 1 -- 172.21.15.43:0/992838277 shutdown_connections 2024-04-25T09:02:59.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.493+0000 7f26fcacb640 1 -- 172.21.15.43:0/992838277 wait complete. 2024-04-25T09:02:59.493 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:02:59.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.606+0000 7f8c58753640 1 Processor -- start 2024-04-25T09:02:59.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.606+0000 7f8c58753640 1 -- start start 2024-04-25T09:02:59.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.607+0000 7f8c58753640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c5015ddc0 0x7f8c500b2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:59.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.607+0000 7f8c58753640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c500ce4d0 con 0x7f8c50162490 2024-04-25T09:02:59.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.607+0000 7f8c58753640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c500ce7b0 con 0x7f8c50161920 2024-04-25T09:02:59.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.607+0000 7f8c58753640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c500ceb40 con 0x7f8c5015ddc0 2024-04-25T09:02:59.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.607+0000 7f8c56cf0640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8c50162490 0x7f8c500b7910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:53526/0 (socket says 172.21.15.43:53526) 2024-04-25T09:02:59.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.607+0000 7f8c56cf0640 1 -- 172.21.15.43:0/1291280644 learned_addr learned my addr 172.21.15.43:0/1291280644 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:59.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.607+0000 7f8c564ef640 1 --2- 172.21.15.43:0/1291280644 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c5015ddc0 0x7f8c500b2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:59.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.607+0000 7f8c554ed640 1 -- 172.21.15.43:0/1291280644 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1566889443 0 0) 0x7f8c500ce4d0 con 0x7f8c50162490 2024-04-25T09:02:59.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.608+0000 7f8c554ed640 1 -- 172.21.15.43:0/1291280644 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c38003520 con 0x7f8c50162490 2024-04-25T09:02:59.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.608+0000 7f8c554ed640 1 -- 172.21.15.43:0/1291280644 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1859315149 0 0) 0x7f8c500ce7b0 con 0x7f8c50161920 2024-04-25T09:02:59.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.608+0000 7f8c554ed640 1 -- 172.21.15.43:0/1291280644 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c500ce4d0 con 0x7f8c50161920 2024-04-25T09:02:59.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.608+0000 7f8c564ef640 1 -- 172.21.15.43:0/1291280644 >> v1:172.21.15.193:6789/0 conn(0x7f8c50161920 legacy=0x7f8c50161d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:59.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.608+0000 7f8c564ef640 1 -- 172.21.15.43:0/1291280644 >> v1:172.21.15.43:6789/0 conn(0x7f8c50162490 legacy=0x7f8c500b7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:59.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.608+0000 7f8c564ef640 1 -- 172.21.15.43:0/1291280644 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c500cee20 con 0x7f8c5015ddc0 2024-04-25T09:02:59.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.608+0000 7f8c564ef640 1 --2- 172.21.15.43:0/1291280644 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c5015ddc0 0x7f8c500b2ea0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8c480097d0 tx=0x7f8c4802a210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c76cd46a5642161 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:59.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.609+0000 7f8c554ed640 1 -- 172.21.15.43:0/1291280644 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c48037070 con 0x7f8c5015ddc0 2024-04-25T09:02:59.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.609+0000 7f8c554ed640 1 -- 172.21.15.43:0/1291280644 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c48032440 con 0x7f8c5015ddc0 2024-04-25T09:02:59.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.609+0000 7f8c554ed640 1 -- 172.21.15.43:0/1291280644 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c48032810 con 0x7f8c5015ddc0 2024-04-25T09:02:59.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.609+0000 7f8c58753640 1 -- 172.21.15.43:0/1291280644 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c5015ddc0 msgr2=0x7f8c500b2ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:59.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.609+0000 7f8c58753640 1 --2- 172.21.15.43:0/1291280644 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c5015ddc0 0x7f8c500b2ea0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8c480097d0 tx=0x7f8c4802a210 comp rx=0 tx=0).stop 2024-04-25T09:02:59.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.609+0000 7f8c58753640 1 -- 172.21.15.43:0/1291280644 shutdown_connections 2024-04-25T09:02:59.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.609+0000 7f8c58753640 1 --2- 172.21.15.43:0/1291280644 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c5015ddc0 0x7f8c500b2ea0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:59.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.609+0000 7f8c58753640 1 -- 172.21.15.43:0/1291280644 >> 172.21.15.43:0/1291280644 conn(0x7f8c5006b000 msgr2=0x7f8c5006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:59.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.609+0000 7f8c58753640 1 -- 172.21.15.43:0/1291280644 shutdown_connections 2024-04-25T09:02:59.599 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.609+0000 7f8c58753640 1 -- 172.21.15.43:0/1291280644 wait complete. 2024-04-25T09:02:59.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.611+0000 7f8c58753640 1 Processor -- start 2024-04-25T09:02:59.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.611+0000 7f8c58753640 1 -- start start 2024-04-25T09:02:59.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.611+0000 7f8c58753640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c5015ddc0 0x7f8c500cd300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:59.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.611+0000 7f8c58753640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c50161920 0x7f8c501f42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:59.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.611+0000 7f8c58753640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c50162490 0x7f8c500b3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:59.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.611+0000 7f8c58753640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c500b81a0 con 0x7f8c5015ddc0 2024-04-25T09:02:59.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.611+0000 7f8c58753640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c500b8020 con 0x7f8c50162490 2024-04-25T09:02:59.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.611+0000 7f8c58753640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c500b8320 con 0x7f8c50161920 2024-04-25T09:02:59.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.611+0000 7f8c564ef640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c5015ddc0 0x7f8c500cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:59.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.611+0000 7f8c564ef640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c5015ddc0 0x7f8c500cd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44156/0 (socket says 172.21.15.43:44156) 2024-04-25T09:02:59.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.611+0000 7f8c564ef640 1 -- 172.21.15.43:0/1690099767 learned_addr learned my addr 172.21.15.43:0/1690099767 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:59.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.612+0000 7f8c56cf0640 1 --2- 172.21.15.43:0/1690099767 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c50162490 0x7f8c500b3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:59.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.612+0000 7f8c55cee640 1 --2- 172.21.15.43:0/1690099767 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c50161920 0x7f8c501f42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:59.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.612+0000 7f8c564ef640 1 -- 172.21.15.43:0/1690099767 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c50161920 msgr2=0x7f8c501f42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:59.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.612+0000 7f8c564ef640 1 --2- 172.21.15.43:0/1690099767 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c50161920 0x7f8c501f42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:59.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.612+0000 7f8c564ef640 1 -- 172.21.15.43:0/1690099767 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c50162490 msgr2=0x7f8c500b3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:59.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.612+0000 7f8c564ef640 1 --2- 172.21.15.43:0/1690099767 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c50162490 0x7f8c500b3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:59.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.612+0000 7f8c564ef640 1 -- 172.21.15.43:0/1690099767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c500b2fb0 con 0x7f8c5015ddc0 2024-04-25T09:02:59.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.612+0000 7f8c56cf0640 1 --2- 172.21.15.43:0/1690099767 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c50162490 0x7f8c500b3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:59.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.612+0000 7f8c55cee640 1 --2- 172.21.15.43:0/1690099767 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c50161920 0x7f8c501f42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:59.603 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.612+0000 7f8c564ef640 1 --2- 172.21.15.43:0/1690099767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c5015ddc0 0x7f8c500cd300 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f8c480098f0 tx=0x7f8c480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:59.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.612+0000 7f8c4effd640 1 -- 172.21.15.43:0/1690099767 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c48041020 con 0x7f8c5015ddc0 2024-04-25T09:02:59.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.612+0000 7f8c4effd640 1 -- 172.21.15.43:0/1690099767 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c4803d6e0 con 0x7f8c5015ddc0 2024-04-25T09:02:59.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.612+0000 7f8c58753640 1 -- 172.21.15.43:0/1690099767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c500b3290 con 0x7f8c5015ddc0 2024-04-25T09:02:59.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.612+0000 7f8c4effd640 1 -- 172.21.15.43:0/1690099767 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c48037040 con 0x7f8c5015ddc0 2024-04-25T09:02:59.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.612+0000 7f8c58753640 1 -- 172.21.15.43:0/1690099767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c500b3850 con 0x7f8c5015ddc0 2024-04-25T09:02:59.604 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.613+0000 7f8c58753640 1 -- 172.21.15.43:0/1690099767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c500680f0 con 0x7f8c5015ddc0 2024-04-25T09:02:59.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.613+0000 7f8c4effd640 1 -- 172.21.15.43:0/1690099767 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8c48044020 con 0x7f8c5015ddc0 2024-04-25T09:02:59.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.614+0000 7f8c4effd640 1 --2- 172.21.15.43:0/1690099767 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8c3c06b4d0 0x7f8c3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:59.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.614+0000 7f8c4effd640 1 -- 172.21.15.43:0/1690099767 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8c480aa900 con 0x7f8c5015ddc0 2024-04-25T09:02:59.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.614+0000 7f8c55cee640 1 --2- 172.21.15.43:0/1690099767 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8c3c06b4d0 0x7f8c3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:59.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.615+0000 7f8c55cee640 1 --2- 172.21.15.43:0/1690099767 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8c3c06b4d0 0x7f8c3c06d990 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f8c5015d310 tx=0x7f8c44009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:59.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.618+0000 7f8c4effd640 1 -- 172.21.15.43:0/1690099767 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8c4804b030 con 0x7f8c5015ddc0 2024-04-25T09:02:59.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.834+0000 7f8c58753640 1 -- 172.21.15.43:0/1690099767 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c50074ef0 con 0x7f8c5015ddc0 2024-04-25T09:02:59.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.835+0000 7f8c4effd640 1 -- 172.21.15.43:0/1690099767 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4946 (secure 0 0 0) 0x7f8c48079750 con 0x7f8c5015ddc0 2024-04-25T09:02:59.823 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:02:59.824 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":69,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6821,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2021228703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2021228703},{"type":"v1","addr":"172.21.15.193:6837","nonce":2021228703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65},{"gid":7543,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/2398324998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2398324998},{"type":"v1","addr":"172.21.15.193:6836","nonce":2398324998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65},{"gid":8673,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2298187831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2298187831},{"type":"v1","addr":"172.21.15.43:6834","nonce":2298187831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:02:26.161487+0000","modified":"2024-04-25T09:02:29.177294+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8676},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8676":{"gid":8676,"name":"c","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6835/1586865031","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":1586865031},{"type":"v1","addr":"172.21.15.43:6835","nonce":1586865031}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T09:02:59.824 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 69 2024-04-25T09:02:59.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.837+0000 7f8c58753640 1 -- 172.21.15.43:0/1690099767 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8c3c06b4d0 msgr2=0x7f8c3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:59.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.837+0000 7f8c58753640 1 --2- 172.21.15.43:0/1690099767 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8c3c06b4d0 0x7f8c3c06d990 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f8c5015d310 tx=0x7f8c44009040 comp rx=0 tx=0).stop 2024-04-25T09:02:59.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.837+0000 7f8c58753640 1 -- 172.21.15.43:0/1690099767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c5015ddc0 msgr2=0x7f8c500cd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:59.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.837+0000 7f8c58753640 1 --2- 172.21.15.43:0/1690099767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c5015ddc0 0x7f8c500cd300 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f8c480098f0 tx=0x7f8c480046c0 comp rx=0 tx=0).stop 2024-04-25T09:02:59.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.837+0000 7f8c58753640 1 -- 172.21.15.43:0/1690099767 shutdown_connections 2024-04-25T09:02:59.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.837+0000 7f8c58753640 1 --2- 172.21.15.43:0/1690099767 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8c3c06b4d0 0x7f8c3c06d990 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:59.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.837+0000 7f8c58753640 1 --2- 172.21.15.43:0/1690099767 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c50162490 0x7f8c500b3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:59.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.837+0000 7f8c58753640 1 --2- 172.21.15.43:0/1690099767 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c50161920 0x7f8c501f42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:59.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.837+0000 7f8c58753640 1 --2- 172.21.15.43:0/1690099767 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c5015ddc0 0x7f8c500cd300 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:59.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.837+0000 7f8c58753640 1 -- 172.21.15.43:0/1690099767 >> 172.21.15.43:0/1690099767 conn(0x7f8c5006b000 msgr2=0x7f8c500b87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:59.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.837+0000 7f8c58753640 1 -- 172.21.15.43:0/1690099767 shutdown_connections 2024-04-25T09:02:59.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.837+0000 7f8c58753640 1 -- 172.21.15.43:0/1690099767 wait complete. 2024-04-25T09:02:59.837 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:02:59.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.950+0000 7f15d356a640 1 Processor -- start 2024-04-25T09:02:59.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.950+0000 7f15d356a640 1 -- start start 2024-04-25T09:02:59.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.951+0000 7f15d356a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f15cc15dce0 0x7f15cc15ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:59.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.951+0000 7f15d356a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15cc0d2920 con 0x7f15cc0d0af0 2024-04-25T09:02:59.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.951+0000 7f15d356a640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15cc0cee80 con 0x7f15cc0ce550 2024-04-25T09:02:59.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.951+0000 7f15d356a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc0cf160 con 0x7f15cc15dce0 2024-04-25T09:02:59.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.951+0000 7f15d1b07640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f15cc0d0af0 0x7f15cc160980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:52906/0 (socket says 172.21.15.43:52906) 2024-04-25T09:02:59.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.951+0000 7f15d1b07640 1 -- 172.21.15.43:0/1292129245 learned_addr learned my addr 172.21.15.43:0/1292129245 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:02:59.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.951+0000 7f15d1306640 1 --2- 172.21.15.43:0/1292129245 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f15cc15dce0 0x7f15cc15ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:59.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.952+0000 7f15c3fff640 1 -- 172.21.15.43:0/1292129245 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1970892314 0 0) 0x7f15cc0d2920 con 0x7f15cc0d0af0 2024-04-25T09:02:59.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.952+0000 7f15c3fff640 1 -- 172.21.15.43:0/1292129245 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15b8003520 con 0x7f15cc0d0af0 2024-04-25T09:02:59.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.952+0000 7f15c3fff640 1 -- 172.21.15.43:0/1292129245 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2816998814 0 0) 0x7f15cc0cee80 con 0x7f15cc0ce550 2024-04-25T09:02:59.941 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.952+0000 7f15c3fff640 1 -- 172.21.15.43:0/1292129245 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15cc0d2920 con 0x7f15cc0ce550 2024-04-25T09:02:59.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.952+0000 7f15d1306640 1 -- 172.21.15.43:0/1292129245 >> v1:172.21.15.193:6789/0 conn(0x7f15cc0ce550 legacy=0x7f15cc0d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:59.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.952+0000 7f15d1306640 1 -- 172.21.15.43:0/1292129245 >> v1:172.21.15.43:6789/0 conn(0x7f15cc0d0af0 legacy=0x7f15cc160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:59.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.952+0000 7f15c3fff640 1 -- 172.21.15.43:0/1292129245 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1815060159 0 0) 0x7f15b8003520 con 0x7f15cc0d0af0 2024-04-25T09:02:59.942 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.952+0000 7f15d1306640 1 -- 172.21.15.43:0/1292129245 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15cc0cf440 con 0x7f15cc15dce0 2024-04-25T09:02:59.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.953+0000 7f15d1306640 1 --2- 172.21.15.43:0/1292129245 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f15cc15dce0 0x7f15cc15ef50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f15c80097d0 tx=0x7f15c802a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e79d13f554a1e388 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:59.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.953+0000 7f15c3fff640 1 -- 172.21.15.43:0/1292129245 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c8037070 con 0x7f15cc15dce0 2024-04-25T09:02:59.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.953+0000 7f15c3fff640 1 -- 172.21.15.43:0/1292129245 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c802ab90 con 0x7f15cc15dce0 2024-04-25T09:02:59.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.953+0000 7f15c3fff640 1 -- 172.21.15.43:0/1292129245 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c4001a10 con 0x7f15cc15dce0 2024-04-25T09:02:59.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.953+0000 7f15d356a640 1 -- 172.21.15.43:0/1292129245 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f15cc15dce0 msgr2=0x7f15cc15ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:59.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.953+0000 7f15d356a640 1 --2- 172.21.15.43:0/1292129245 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f15cc15dce0 0x7f15cc15ef50 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f15c80097d0 tx=0x7f15c802a100 comp rx=0 tx=0).stop 2024-04-25T09:02:59.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.953+0000 7f15d356a640 1 -- 172.21.15.43:0/1292129245 shutdown_connections 2024-04-25T09:02:59.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.953+0000 7f15d356a640 1 --2- 172.21.15.43:0/1292129245 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f15cc15dce0 0x7f15cc15ef50 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:59.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.953+0000 7f15d356a640 1 -- 172.21.15.43:0/1292129245 >> 172.21.15.43:0/1292129245 conn(0x7f15cc09cd00 msgr2=0x7f15cc0d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:02:59.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.953+0000 7f15d356a640 1 -- 172.21.15.43:0/1292129245 shutdown_connections 2024-04-25T09:02:59.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.953+0000 7f15d356a640 1 -- 172.21.15.43:0/1292129245 wait complete. 2024-04-25T09:02:59.947 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.954+0000 7f15d356a640 1 Processor -- start 2024-04-25T09:02:59.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.955+0000 7f15d356a640 1 -- start start 2024-04-25T09:02:59.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.955+0000 7f15d356a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15cc0ce550 0x7f15cc151dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:59.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.955+0000 7f15d356a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f15cc0d0af0 0x7f15cc0b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:59.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.955+0000 7f15d356a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15cc15dce0 0x7f15cc0b90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:59.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.955+0000 7f15d356a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc15e0c0 con 0x7f15cc0ce550 2024-04-25T09:02:59.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.955+0000 7f15d356a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc15f6b0 con 0x7f15cc15dce0 2024-04-25T09:02:59.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.955+0000 7f15d356a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc0ce990 con 0x7f15cc0d0af0 2024-04-25T09:02:59.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.955+0000 7f15d1306640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15cc0ce550 0x7f15cc151dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:59.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.955+0000 7f15d1306640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15cc0ce550 0x7f15cc151dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44166/0 (socket says 172.21.15.43:44166) 2024-04-25T09:02:59.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.955+0000 7f15d1306640 1 -- 172.21.15.43:0/4144207091 learned_addr learned my addr 172.21.15.43:0/4144207091 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:02:59.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.955+0000 7f15d0b05640 1 --2- 172.21.15.43:0/4144207091 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f15cc0d0af0 0x7f15cc0b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:59.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.955+0000 7f15d1b07640 1 --2- 172.21.15.43:0/4144207091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15cc15dce0 0x7f15cc0b90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:59.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.956+0000 7f15d1306640 1 -- 172.21.15.43:0/4144207091 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f15cc0d0af0 msgr2=0x7f15cc0b8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:59.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.956+0000 7f15d1306640 1 --2- 172.21.15.43:0/4144207091 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f15cc0d0af0 0x7f15cc0b8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:59.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.956+0000 7f15d1306640 1 -- 172.21.15.43:0/4144207091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15cc15dce0 msgr2=0x7f15cc0b90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:02:59.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.956+0000 7f15d1306640 1 --2- 172.21.15.43:0/4144207091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15cc15dce0 0x7f15cc0b90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:02:59.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.956+0000 7f15d1306640 1 -- 172.21.15.43:0/4144207091 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15cc09f120 con 0x7f15cc0ce550 2024-04-25T09:02:59.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.956+0000 7f15d1b07640 1 --2- 172.21.15.43:0/4144207091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15cc15dce0 0x7f15cc0b90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:02:59.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.956+0000 7f15d1306640 1 --2- 172.21.15.43:0/4144207091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15cc0ce550 0x7f15cc151dc0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f15c80097d0 tx=0x7f15c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:59.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.956+0000 7f15c1ffb640 1 -- 172.21.15.43:0/4144207091 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c8042020 con 0x7f15cc0ce550 2024-04-25T09:02:59.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.956+0000 7f15c1ffb640 1 -- 172.21.15.43:0/4144207091 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c803d670 con 0x7f15cc0ce550 2024-04-25T09:02:59.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.956+0000 7f15d356a640 1 -- 172.21.15.43:0/4144207091 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15cc09f400 con 0x7f15cc0ce550 2024-04-25T09:02:59.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.956+0000 7f15c1ffb640 1 -- 172.21.15.43:0/4144207091 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c8037040 con 0x7f15cc0ce550 2024-04-25T09:02:59.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.956+0000 7f15d356a640 1 -- 172.21.15.43:0/4144207091 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15cc09f9c0 con 0x7f15cc0ce550 2024-04-25T09:02:59.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.957+0000 7f15d356a640 1 -- 172.21.15.43:0/4144207091 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15cc0680f0 con 0x7f15cc0ce550 2024-04-25T09:02:59.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.958+0000 7f15c1ffb640 1 -- 172.21.15.43:0/4144207091 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f15c8045020 con 0x7f15cc0ce550 2024-04-25T09:02:59.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.958+0000 7f15c1ffb640 1 --2- 172.21.15.43:0/4144207091 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f15b006b500 0x7f15b006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:02:59.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.958+0000 7f15c1ffb640 1 -- 172.21.15.43:0/4144207091 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f15c80aaa50 con 0x7f15cc0ce550 2024-04-25T09:02:59.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.958+0000 7f15d0b05640 1 --2- 172.21.15.43:0/4144207091 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f15b006b500 0x7f15b006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:02:59.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.959+0000 7f15d0b05640 1 --2- 172.21.15.43:0/4144207091 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f15b006b500 0x7f15b006d9c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f15cc1f74a0 tx=0x7f15bc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:02:59.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:02:59.962+0000 7f15c1ffb640 1 -- 172.21.15.43:0/4144207091 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f15c804c030 con 0x7f15cc0ce550 2024-04-25T09:03:00.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.144+0000 7f15d356a640 1 -- 172.21.15.43:0/4144207091 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15cc074ef0 con 0x7f15cc0ce550 2024-04-25T09:03:00.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.145+0000 7f15c1ffb640 1 -- 172.21.15.43:0/4144207091 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v115) v1 ==== 74+0+16744 (secure 0 0 0) 0x7f15c8079890 con 0x7f15cc0ce550 2024-04-25T09:03:00.134 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:03:00.134 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":115,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:02:54.872758+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:02:24.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"112","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-04-25T09:02:25.143723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.e","mappings":[{"from":0,"to":5}]},{"pgid":"17.3f","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:00.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.148+0000 7f15d356a640 1 -- 172.21.15.43:0/4144207091 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f15b006b500 msgr2=0x7f15b006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:00.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.148+0000 7f15d356a640 1 --2- 172.21.15.43:0/4144207091 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f15b006b500 0x7f15b006d9c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f15cc1f74a0 tx=0x7f15bc009040 comp rx=0 tx=0).stop 2024-04-25T09:03:00.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.148+0000 7f15d356a640 1 -- 172.21.15.43:0/4144207091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15cc0ce550 msgr2=0x7f15cc151dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:00.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.148+0000 7f15d356a640 1 --2- 172.21.15.43:0/4144207091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15cc0ce550 0x7f15cc151dc0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f15c80097d0 tx=0x7f15c80046c0 comp rx=0 tx=0).stop 2024-04-25T09:03:00.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.148+0000 7f15d356a640 1 -- 172.21.15.43:0/4144207091 shutdown_connections 2024-04-25T09:03:00.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.148+0000 7f15d356a640 1 --2- 172.21.15.43:0/4144207091 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f15b006b500 0x7f15b006d9c0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:00.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.148+0000 7f15d356a640 1 --2- 172.21.15.43:0/4144207091 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f15cc15dce0 0x7f15cc0b90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:00.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.148+0000 7f15d356a640 1 --2- 172.21.15.43:0/4144207091 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f15cc0d0af0 0x7f15cc0b8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:00.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.148+0000 7f15d356a640 1 --2- 172.21.15.43:0/4144207091 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f15cc0ce550 0x7f15cc151dc0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:00.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.148+0000 7f15d356a640 1 -- 172.21.15.43:0/4144207091 >> 172.21.15.43:0/4144207091 conn(0x7f15cc09cd00 msgr2=0x7f15cc15e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:00.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.148+0000 7f15d356a640 1 -- 172.21.15.43:0/4144207091 shutdown_connections 2024-04-25T09:03:00.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.148+0000 7f15d356a640 1 -- 172.21.15.43:0/4144207091 wait complete. 2024-04-25T09:03:00.149 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:03:00.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.262+0000 7f6f5115c640 1 Processor -- start 2024-04-25T09:03:00.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.262+0000 7f6f5115c640 1 -- start start 2024-04-25T09:03:00.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.263+0000 7f6f5115c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6f4c0d09b0 0x7f6f4c0b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:00.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.263+0000 7f6f5115c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6f4c0ce5e0 0x7f6f4c0cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:00.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.263+0000 7f6f5115c640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f4c0b4c30 con 0x7f6f4c161390 2024-04-25T09:03:00.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.263+0000 7f6f5115c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f4c0d1910 con 0x7f6f4c0d09b0 2024-04-25T09:03:00.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.263+0000 7f6f5115c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4c0cf2c0 con 0x7f6f4c0ce5e0 2024-04-25T09:03:00.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.263+0000 7f6f4a59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6f4c0d09b0 0x7f6f4c0b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:00.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.263+0000 7f6f4a59c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6f4c0d09b0 0x7f6f4c0b6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44180/0 (socket says 172.21.15.43:44180) 2024-04-25T09:03:00.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.263+0000 7f6f4a59c640 1 -- 172.21.15.43:0/3382372807 learned_addr learned my addr 172.21.15.43:0/3382372807 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:00.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.263+0000 7f6f4b59e640 1 --2- 172.21.15.43:0/3382372807 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6f4c0ce5e0 0x7f6f4c0cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:00.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.263+0000 7f6f4a59c640 1 -- 172.21.15.43:0/3382372807 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6f4c0ce5e0 msgr2=0x7f6f4c0cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:00.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.263+0000 7f6f4a59c640 1 --2- 172.21.15.43:0/3382372807 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6f4c0ce5e0 0x7f6f4c0cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:00.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.263+0000 7f6f4a59c640 1 -- 172.21.15.43:0/3382372807 >> v1:172.21.15.193:6789/0 conn(0x7f6f4c161390 legacy=0x7f6f4c0d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:00.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.263+0000 7f6f4a59c640 1 -- 172.21.15.43:0/3382372807 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f4c0cf5a0 con 0x7f6f4c0d09b0 2024-04-25T09:03:00.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.263+0000 7f6f4b59e640 1 --2- 172.21.15.43:0/3382372807 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6f4c0ce5e0 0x7f6f4c0cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:00.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.264+0000 7f6f4a59c640 1 --2- 172.21.15.43:0/3382372807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f4c0d09b0 0x7f6f4c0b6820 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f6f34009870 tx=0x7f6f3402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cafa1329241ae58b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:00.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.264+0000 7f6f49d9b640 1 -- 172.21.15.43:0/3382372807 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f34037070 con 0x7f6f4c0d09b0 2024-04-25T09:03:00.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.264+0000 7f6f49d9b640 1 -- 172.21.15.43:0/3382372807 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f34032440 con 0x7f6f4c0d09b0 2024-04-25T09:03:00.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.264+0000 7f6f49d9b640 1 -- 172.21.15.43:0/3382372807 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f34032810 con 0x7f6f4c0d09b0 2024-04-25T09:03:00.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.264+0000 7f6f5115c640 1 -- 172.21.15.43:0/3382372807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f4c0d09b0 msgr2=0x7f6f4c0b6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:00.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.264+0000 7f6f5115c640 1 --2- 172.21.15.43:0/3382372807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f4c0d09b0 0x7f6f4c0b6820 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f6f34009870 tx=0x7f6f3402a210 comp rx=0 tx=0).stop 2024-04-25T09:03:00.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.265+0000 7f6f5115c640 1 -- 172.21.15.43:0/3382372807 shutdown_connections 2024-04-25T09:03:00.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.265+0000 7f6f5115c640 1 --2- 172.21.15.43:0/3382372807 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6f4c0ce5e0 0x7f6f4c0cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:00.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.265+0000 7f6f5115c640 1 --2- 172.21.15.43:0/3382372807 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f4c0d09b0 0x7f6f4c0b6820 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:00.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.265+0000 7f6f5115c640 1 -- 172.21.15.43:0/3382372807 >> 172.21.15.43:0/3382372807 conn(0x7f6f4c09cd20 msgr2=0x7f6f4c0d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:00.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.265+0000 7f6f5115c640 1 -- 172.21.15.43:0/3382372807 shutdown_connections 2024-04-25T09:03:00.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.265+0000 7f6f5115c640 1 -- 172.21.15.43:0/3382372807 wait complete. 2024-04-25T09:03:00.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.266+0000 7f6f5115c640 1 Processor -- start 2024-04-25T09:03:00.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.266+0000 7f6f5115c640 1 -- start start 2024-04-25T09:03:00.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.267+0000 7f6f5115c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f4c0ce5e0 0x7f6f4c1f3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:00.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.267+0000 7f6f5115c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6f4c0d09b0 0x7f6f4c0c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:00.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.267+0000 7f6f5115c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f4c161390 0x7f6f4c0c8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:00.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.267+0000 7f6f5115c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4c161770 con 0x7f6f4c0ce5e0 2024-04-25T09:03:00.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.267+0000 7f6f5115c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4c0d15a0 con 0x7f6f4c161390 2024-04-25T09:03:00.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.267+0000 7f6f5115c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f4c0cefb0 con 0x7f6f4c0d09b0 2024-04-25T09:03:00.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.267+0000 7f6f4ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f4c0ce5e0 0x7f6f4c1f3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:00.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.267+0000 7f6f4ad9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f4c0ce5e0 0x7f6f4c1f3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44188/0 (socket says 172.21.15.43:44188) 2024-04-25T09:03:00.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.267+0000 7f6f4ad9d640 1 -- 172.21.15.43:0/2899399222 learned_addr learned my addr 172.21.15.43:0/2899399222 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:00.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.267+0000 7f6f4a59c640 1 --2- 172.21.15.43:0/2899399222 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6f4c0d09b0 0x7f6f4c0c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:00.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.267+0000 7f6f4b59e640 1 --2- 172.21.15.43:0/2899399222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f4c161390 0x7f6f4c0c8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:00.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.267+0000 7f6f4ad9d640 1 -- 172.21.15.43:0/2899399222 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6f4c0d09b0 msgr2=0x7f6f4c0c80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:00.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.267+0000 7f6f4ad9d640 1 --2- 172.21.15.43:0/2899399222 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6f4c0d09b0 0x7f6f4c0c80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:00.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.267+0000 7f6f4ad9d640 1 -- 172.21.15.43:0/2899399222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f4c161390 msgr2=0x7f6f4c0c8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:00.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.267+0000 7f6f4ad9d640 1 --2- 172.21.15.43:0/2899399222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f4c161390 0x7f6f4c0c8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:00.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.267+0000 7f6f4ad9d640 1 -- 172.21.15.43:0/2899399222 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f4c0ca570 con 0x7f6f4c0ce5e0 2024-04-25T09:03:00.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.267+0000 7f6f4a59c640 1 --2- 172.21.15.43:0/2899399222 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6f4c0d09b0 0x7f6f4c0c80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:00.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.267+0000 7f6f4b59e640 1 --2- 172.21.15.43:0/2899399222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f4c161390 0x7f6f4c0c8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:00.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.268+0000 7f6f4ad9d640 1 --2- 172.21.15.43:0/2899399222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f4c0ce5e0 0x7f6f4c1f3d20 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f6f4400b3f0 tx=0x7f6f4400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:00.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.268+0000 7f6f437fe640 1 -- 172.21.15.43:0/2899399222 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f44015020 con 0x7f6f4c0ce5e0 2024-04-25T09:03:00.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.268+0000 7f6f437fe640 1 -- 172.21.15.43:0/2899399222 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f44011830 con 0x7f6f4c0ce5e0 2024-04-25T09:03:00.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.268+0000 7f6f5115c640 1 -- 172.21.15.43:0/2899399222 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f4c0cb5e0 con 0x7f6f4c0ce5e0 2024-04-25T09:03:00.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.268+0000 7f6f437fe640 1 -- 172.21.15.43:0/2899399222 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f44011c20 con 0x7f6f4c0ce5e0 2024-04-25T09:03:00.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.268+0000 7f6f5115c640 1 -- 172.21.15.43:0/2899399222 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f4c0cbba0 con 0x7f6f4c0ce5e0 2024-04-25T09:03:00.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.269+0000 7f6f5115c640 1 -- 172.21.15.43:0/2899399222 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f4c0680f0 con 0x7f6f4c0ce5e0 2024-04-25T09:03:00.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.269+0000 7f6f437fe640 1 -- 172.21.15.43:0/2899399222 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6f44018020 con 0x7f6f4c0ce5e0 2024-04-25T09:03:00.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.270+0000 7f6f437fe640 1 --2- 172.21.15.43:0/2899399222 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6f2c06b4d0 0x7f6f2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:00.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.270+0000 7f6f437fe640 1 -- 172.21.15.43:0/2899399222 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f6f4408a170 con 0x7f6f4c0ce5e0 2024-04-25T09:03:00.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.270+0000 7f6f4a59c640 1 --2- 172.21.15.43:0/2899399222 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6f2c06b4d0 0x7f6f2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:00.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.271+0000 7f6f4a59c640 1 --2- 172.21.15.43:0/2899399222 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6f2c06b4d0 0x7f6f2c06d990 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f6f34034000 tx=0x7f6f3402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:00.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.274+0000 7f6f437fe640 1 -- 172.21.15.43:0/2899399222 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6f4401f020 con 0x7f6f4c0ce5e0 2024-04-25T09:03:00.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.491+0000 7f6f5115c640 1 -- 172.21.15.43:0/2899399222 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6f4c074ef0 con 0x7f6f4c0ce5e0 2024-04-25T09:03:00.480 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-04-25T09:03:00.491+0000 7fe4fb198640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:03:00.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.492+0000 7f6f437fe640 1 -- 172.21.15.43:0/2899399222 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v70) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6f440172c0 con 0x7f6f4c0ce5e0 2024-04-25T09:03:00.481 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:03:00.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.496+0000 7f6f5115c640 1 -- 172.21.15.43:0/2899399222 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6f2c06b4d0 msgr2=0x7f6f2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:00.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.496+0000 7f6f5115c640 1 --2- 172.21.15.43:0/2899399222 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6f2c06b4d0 0x7f6f2c06d990 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f6f34034000 tx=0x7f6f3402e000 comp rx=0 tx=0).stop 2024-04-25T09:03:00.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.496+0000 7f6f5115c640 1 -- 172.21.15.43:0/2899399222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f4c0ce5e0 msgr2=0x7f6f4c1f3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:00.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.496+0000 7f6f5115c640 1 --2- 172.21.15.43:0/2899399222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f4c0ce5e0 0x7f6f4c1f3d20 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f6f4400b3f0 tx=0x7f6f4400b8b0 comp rx=0 tx=0).stop 2024-04-25T09:03:00.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.496+0000 7f6f5115c640 1 -- 172.21.15.43:0/2899399222 shutdown_connections 2024-04-25T09:03:00.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.496+0000 7f6f5115c640 1 --2- 172.21.15.43:0/2899399222 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6f2c06b4d0 0x7f6f2c06d990 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:00.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.496+0000 7f6f5115c640 1 --2- 172.21.15.43:0/2899399222 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6f4c161390 0x7f6f4c0c8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:00.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.496+0000 7f6f5115c640 1 --2- 172.21.15.43:0/2899399222 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6f4c0d09b0 0x7f6f4c0c80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:00.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.496+0000 7f6f5115c640 1 --2- 172.21.15.43:0/2899399222 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6f4c0ce5e0 0x7f6f4c1f3d20 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:00.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.496+0000 7f6f5115c640 1 -- 172.21.15.43:0/2899399222 >> 172.21.15.43:0/2899399222 conn(0x7f6f4c09cd20 msgr2=0x7f6f4c148cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:00.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.496+0000 7f6f5115c640 1 -- 172.21.15.43:0/2899399222 shutdown_connections 2024-04-25T09:03:00.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.496+0000 7f6f5115c640 1 -- 172.21.15.43:0/2899399222 wait complete. 2024-04-25T09:03:00.501 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:03:00.528 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-04-25T09:03:00.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.616+0000 7f4044330640 1 Processor -- start 2024-04-25T09:03:00.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.617+0000 7f4044330640 1 -- start start 2024-04-25T09:03:00.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.617+0000 7f4044330640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f403c06ccd0 0x7f403c168870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:00.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.617+0000 7f4044330640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f403c15e140 con 0x7f403c15dd60 2024-04-25T09:03:00.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.617+0000 7f4044330640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f403c0d27d0 con 0x7f403c0d1000 2024-04-25T09:03:00.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.617+0000 7f4044330640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f403c06b1e0 con 0x7f403c06ccd0 2024-04-25T09:03:00.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.618+0000 7f40428cd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f403c06ccd0 0x7f403c168870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:00.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.618+0000 7f40420cc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f403c15dd60 0x7f403c0d1fa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:52916/0 (socket says 172.21.15.43:52916) 2024-04-25T09:03:00.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.618+0000 7f40420cc640 1 -- 172.21.15.43:0/484456120 learned_addr learned my addr 172.21.15.43:0/484456120 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:03:00.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.618+0000 7f40428cd640 1 -- 172.21.15.43:0/484456120 >> v1:172.21.15.193:6789/0 conn(0x7f403c0d1000 legacy=0x7f403c0d0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:00.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.618+0000 7f40428cd640 1 -- 172.21.15.43:0/484456120 >> v1:172.21.15.43:6789/0 conn(0x7f403c15dd60 legacy=0x7f403c0d1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:00.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.618+0000 7f40428cd640 1 -- 172.21.15.43:0/484456120 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f403c06b4c0 con 0x7f403c06ccd0 2024-04-25T09:03:00.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.618+0000 7f40410ca640 1 -- 172.21.15.43:0/484456120 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2401632441 0 0) 0x7f403c15e140 con 0x7f403c15dd60 2024-04-25T09:03:00.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.619+0000 7f40428cd640 1 --2- 172.21.15.43:0/484456120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f403c06ccd0 0x7f403c168870 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f4034009d90 tx=0x7f403402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32626cf46e355328 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:00.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.619+0000 7f40410ca640 1 -- 172.21.15.43:0/484456120 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4034007740 con 0x7f403c06ccd0 2024-04-25T09:03:00.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.619+0000 7f40410ca640 1 -- 172.21.15.43:0/484456120 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40340078e0 con 0x7f403c06ccd0 2024-04-25T09:03:00.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.619+0000 7f40410ca640 1 -- 172.21.15.43:0/484456120 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4034007cb0 con 0x7f403c06ccd0 2024-04-25T09:03:00.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.619+0000 7f4044330640 1 -- 172.21.15.43:0/484456120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f403c06ccd0 msgr2=0x7f403c168870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:00.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.619+0000 7f4044330640 1 --2- 172.21.15.43:0/484456120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f403c06ccd0 0x7f403c168870 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f4034009d90 tx=0x7f403402a6f0 comp rx=0 tx=0).stop 2024-04-25T09:03:00.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.619+0000 7f4044330640 1 -- 172.21.15.43:0/484456120 shutdown_connections 2024-04-25T09:03:00.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.619+0000 7f4044330640 1 --2- 172.21.15.43:0/484456120 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f403c06ccd0 0x7f403c168870 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:00.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.619+0000 7f4044330640 1 -- 172.21.15.43:0/484456120 >> 172.21.15.43:0/484456120 conn(0x7f403c0d39c0 msgr2=0x7f403c0d3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:00.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.620+0000 7f4044330640 1 -- 172.21.15.43:0/484456120 shutdown_connections 2024-04-25T09:03:00.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.620+0000 7f4044330640 1 -- 172.21.15.43:0/484456120 wait complete. 2024-04-25T09:03:00.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.621+0000 7f4044330640 1 Processor -- start 2024-04-25T09:03:00.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.621+0000 7f4044330640 1 -- start start 2024-04-25T09:03:00.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.621+0000 7f4044330640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f403c06ccd0 0x7f403c1545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:00.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.621+0000 7f4044330640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f403c0d1000 0x7f403c15f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:00.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.621+0000 7f4044330640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f403c15dd60 0x7f403c15fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:00.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.621+0000 7f4044330640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403c0d0c20 con 0x7f403c06ccd0 2024-04-25T09:03:00.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.621+0000 7f4044330640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403c0d0aa0 con 0x7f403c15dd60 2024-04-25T09:03:00.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.621+0000 7f4044330640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f403c168db0 con 0x7f403c0d1000 2024-04-25T09:03:00.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.621+0000 7f40420cc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f403c06ccd0 0x7f403c1545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:00.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.622+0000 7f40420cc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f403c06ccd0 0x7f403c1545d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44232/0 (socket says 172.21.15.43:44232) 2024-04-25T09:03:00.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.622+0000 7f40420cc640 1 -- 172.21.15.43:0/1758974850 learned_addr learned my addr 172.21.15.43:0/1758974850 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:00.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.622+0000 7f40418cb640 1 --2- 172.21.15.43:0/1758974850 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f403c0d1000 0x7f403c15f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:00.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.622+0000 7f40428cd640 1 --2- 172.21.15.43:0/1758974850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f403c15dd60 0x7f403c15fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:00.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.622+0000 7f40420cc640 1 -- 172.21.15.43:0/1758974850 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f403c0d1000 msgr2=0x7f403c15f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:00.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.622+0000 7f40420cc640 1 --2- 172.21.15.43:0/1758974850 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f403c0d1000 0x7f403c15f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:00.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.622+0000 7f40420cc640 1 -- 172.21.15.43:0/1758974850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f403c15dd60 msgr2=0x7f403c15fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:00.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.622+0000 7f40420cc640 1 --2- 172.21.15.43:0/1758974850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f403c15dd60 0x7f403c15fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:00.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.622+0000 7f40420cc640 1 -- 172.21.15.43:0/1758974850 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f403c0cb2a0 con 0x7f403c06ccd0 2024-04-25T09:03:00.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.622+0000 7f40418cb640 1 --2- 172.21.15.43:0/1758974850 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f403c0d1000 0x7f403c15f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:00.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.622+0000 7f40420cc640 1 --2- 172.21.15.43:0/1758974850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f403c06ccd0 0x7f403c1545d0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f403800c3f0 tx=0x7f403800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:00.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.622+0000 7f4032ffd640 1 -- 172.21.15.43:0/1758974850 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40380074e0 con 0x7f403c06ccd0 2024-04-25T09:03:00.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.622+0000 7f4032ffd640 1 -- 172.21.15.43:0/1758974850 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40380047e0 con 0x7f403c06ccd0 2024-04-25T09:03:00.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.622+0000 7f4044330640 1 -- 172.21.15.43:0/1758974850 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f403c0b7d80 con 0x7f403c06ccd0 2024-04-25T09:03:00.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.622+0000 7f4032ffd640 1 -- 172.21.15.43:0/1758974850 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4038004c10 con 0x7f403c06ccd0 2024-04-25T09:03:00.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.622+0000 7f4044330640 1 -- 172.21.15.43:0/1758974850 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f403c0b8340 con 0x7f403c06ccd0 2024-04-25T09:03:00.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.623+0000 7f4044330640 1 -- 172.21.15.43:0/1758974850 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f403c074f60 con 0x7f403c06ccd0 2024-04-25T09:03:00.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.624+0000 7f4032ffd640 1 -- 172.21.15.43:0/1758974850 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4038011070 con 0x7f403c06ccd0 2024-04-25T09:03:00.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.624+0000 7f4032ffd640 1 --2- 172.21.15.43:0/1758974850 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f402006b4d0 0x7f402006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:00.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.624+0000 7f4032ffd640 1 -- 172.21.15.43:0/1758974850 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f4038089e10 con 0x7f403c06ccd0 2024-04-25T09:03:00.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.624+0000 7f40418cb640 1 --2- 172.21.15.43:0/1758974850 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f402006b4d0 0x7f402006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:00.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.625+0000 7f40418cb640 1 --2- 172.21.15.43:0/1758974850 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f402006b4d0 0x7f402006d990 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f402c007fd0 tx=0x7f402c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:00.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.628+0000 7f4032ffd640 1 -- 172.21.15.43:0/1758974850 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4038058c90 con 0x7f403c06ccd0 2024-04-25T09:03:00.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:00.851+0000 7f4044330640 1 -- 172.21.15.43:0/1758974850 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f403c0b8510 con 0x7f403c06ccd0 2024-04-25T09:03:01.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.494+0000 7f4032ffd640 1 -- 172.21.15.43:0/1758974850 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v71) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4038058590 con 0x7f403c06ccd0 2024-04-25T09:03:01.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.497+0000 7f4044330640 1 -- 172.21.15.43:0/1758974850 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f402006b4d0 msgr2=0x7f402006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:01.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.497+0000 7f4044330640 1 --2- 172.21.15.43:0/1758974850 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f402006b4d0 0x7f402006d990 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f402c007fd0 tx=0x7f402c009040 comp rx=0 tx=0).stop 2024-04-25T09:03:01.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.497+0000 7f4044330640 1 -- 172.21.15.43:0/1758974850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f403c06ccd0 msgr2=0x7f403c1545d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:01.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.497+0000 7f4044330640 1 --2- 172.21.15.43:0/1758974850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f403c06ccd0 0x7f403c1545d0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f403800c3f0 tx=0x7f403800c8b0 comp rx=0 tx=0).stop 2024-04-25T09:03:01.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.497+0000 7f4044330640 1 -- 172.21.15.43:0/1758974850 shutdown_connections 2024-04-25T09:03:01.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.497+0000 7f4044330640 1 --2- 172.21.15.43:0/1758974850 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f402006b4d0 0x7f402006d990 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:01.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.497+0000 7f4044330640 1 --2- 172.21.15.43:0/1758974850 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f403c15dd60 0x7f403c15fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:01.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.497+0000 7f4044330640 1 --2- 172.21.15.43:0/1758974850 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f403c0d1000 0x7f403c15f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:01.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.497+0000 7f4044330640 1 --2- 172.21.15.43:0/1758974850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f403c06ccd0 0x7f403c1545d0 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:01.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.497+0000 7f4044330640 1 -- 172.21.15.43:0/1758974850 >> 172.21.15.43:0/1758974850 conn(0x7f403c0d39c0 msgr2=0x7f403c0b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:01.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.497+0000 7f4044330640 1 -- 172.21.15.43:0/1758974850 shutdown_connections 2024-04-25T09:03:01.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.497+0000 7f4044330640 1 -- 172.21.15.43:0/1758974850 wait complete. 2024-04-25T09:03:01.500 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:03:01.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.667+0000 7f8ad5e5d640 1 Processor -- start 2024-04-25T09:03:01.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.667+0000 7f8ad5e5d640 1 -- start start 2024-04-25T09:03:01.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.668+0000 7f8ad5e5d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ad015b6c0 0x7f8ad015baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:01.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.668+0000 7f8ad5e5d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8ad015e410 0x7f8ad00b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:01.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.668+0000 7f8ad5e5d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ad00b37a0 con 0x7f8ad015f5d0 2024-04-25T09:03:01.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.668+0000 7f8ad5e5d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ad015e870 con 0x7f8ad015e410 2024-04-25T09:03:01.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.668+0000 7f8ad5e5d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad00b53e0 con 0x7f8ad015b6c0 2024-04-25T09:03:01.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.668+0000 7f8aceffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8ad015e410 0x7f8ad00b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:01.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.668+0000 7f8acffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8ad015f5d0 0x7f8ad00d3760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:52932/0 (socket says 172.21.15.43:52932) 2024-04-25T09:03:01.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.668+0000 7f8acffff640 1 -- 172.21.15.43:0/2783978247 learned_addr learned my addr 172.21.15.43:0/2783978247 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:03:01.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.668+0000 7f8acf7fe640 1 --2- 172.21.15.43:0/2783978247 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ad015b6c0 0x7f8ad015baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:01.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.668+0000 7f8aceffd640 1 -- 172.21.15.43:0/2783978247 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ad015b6c0 msgr2=0x7f8ad015baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:01.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.668+0000 7f8aceffd640 1 --2- 172.21.15.43:0/2783978247 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ad015b6c0 0x7f8ad015baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:01.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.668+0000 7f8aceffd640 1 -- 172.21.15.43:0/2783978247 >> v1:172.21.15.43:6789/0 conn(0x7f8ad015f5d0 legacy=0x7f8ad00d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:01.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.668+0000 7f8aceffd640 1 -- 172.21.15.43:0/2783978247 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad015eab0 con 0x7f8ad015e410 2024-04-25T09:03:01.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.669+0000 7f8aceffd640 1 --2- 172.21.15.43:0/2783978247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ad015e410 0x7f8ad00b4ea0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f8ac40097a0 tx=0x7f8ac402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32ebd3f8e5a46882 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:01.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.669+0000 7f8ace7fc640 1 -- 172.21.15.43:0/2783978247 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4037070 con 0x7f8ad015e410 2024-04-25T09:03:01.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.669+0000 7f8ace7fc640 1 -- 172.21.15.43:0/2783978247 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ac4032440 con 0x7f8ad015e410 2024-04-25T09:03:01.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.669+0000 7f8ace7fc640 1 -- 172.21.15.43:0/2783978247 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4032810 con 0x7f8ad015e410 2024-04-25T09:03:01.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.670+0000 7f8ad5e5d640 1 -- 172.21.15.43:0/2783978247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ad015e410 msgr2=0x7f8ad00b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:01.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.670+0000 7f8ad5e5d640 1 --2- 172.21.15.43:0/2783978247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ad015e410 0x7f8ad00b4ea0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f8ac40097a0 tx=0x7f8ac402a180 comp rx=0 tx=0).stop 2024-04-25T09:03:01.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.670+0000 7f8ad5e5d640 1 -- 172.21.15.43:0/2783978247 shutdown_connections 2024-04-25T09:03:01.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.670+0000 7f8ad5e5d640 1 --2- 172.21.15.43:0/2783978247 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ad015e410 0x7f8ad00b4ea0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:01.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.670+0000 7f8ad5e5d640 1 --2- 172.21.15.43:0/2783978247 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ad015b6c0 0x7f8ad015baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:01.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.670+0000 7f8ad5e5d640 1 -- 172.21.15.43:0/2783978247 >> 172.21.15.43:0/2783978247 conn(0x7f8ad008d460 msgr2=0x7f8ad009cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:01.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.670+0000 7f8ad5e5d640 1 -- 172.21.15.43:0/2783978247 shutdown_connections 2024-04-25T09:03:01.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.670+0000 7f8ad5e5d640 1 -- 172.21.15.43:0/2783978247 wait complete. 2024-04-25T09:03:01.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.671+0000 7f8ad5e5d640 1 Processor -- start 2024-04-25T09:03:01.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.671+0000 7f8ad5e5d640 1 -- start start 2024-04-25T09:03:01.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.672+0000 7f8ad5e5d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ad015b6c0 0x7f8ad00c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:01.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.672+0000 7f8ad5e5d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ad015e410 0x7f8ad00ce910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:01.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.672+0000 7f8ad5e5d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ad015f5d0 0x7f8ad00d0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:01.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.672+0000 7f8ad5e5d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad015fa40 con 0x7f8ad015f5d0 2024-04-25T09:03:01.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.672+0000 7f8ad5e5d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad015f350 con 0x7f8ad015b6c0 2024-04-25T09:03:01.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.672+0000 7f8ad5e5d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad015fbc0 con 0x7f8ad015e410 2024-04-25T09:03:01.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.672+0000 7f8acffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ad015f5d0 0x7f8ad00d0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:01.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.672+0000 7f8acffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ad015f5d0 0x7f8ad00d0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44244/0 (socket says 172.21.15.43:44244) 2024-04-25T09:03:01.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.672+0000 7f8acffff640 1 -- 172.21.15.43:0/3714984128 learned_addr learned my addr 172.21.15.43:0/3714984128 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:01.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.672+0000 7f8acf7fe640 1 --2- 172.21.15.43:0/3714984128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ad015b6c0 0x7f8ad00c9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:01.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.672+0000 7f8aceffd640 1 --2- 172.21.15.43:0/3714984128 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ad015e410 0x7f8ad00ce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:01.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.673+0000 7f8acffff640 1 -- 172.21.15.43:0/3714984128 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ad015e410 msgr2=0x7f8ad00ce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:01.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.673+0000 7f8acffff640 1 --2- 172.21.15.43:0/3714984128 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ad015e410 0x7f8ad00ce910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:01.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.673+0000 7f8acffff640 1 -- 172.21.15.43:0/3714984128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ad015b6c0 msgr2=0x7f8ad00c9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:01.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.673+0000 7f8acffff640 1 --2- 172.21.15.43:0/3714984128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ad015b6c0 0x7f8ad00c9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:01.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.673+0000 7f8acffff640 1 -- 172.21.15.43:0/3714984128 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad00cc140 con 0x7f8ad015f5d0 2024-04-25T09:03:01.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.673+0000 7f8acffff640 1 --2- 172.21.15.43:0/3714984128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ad015f5d0 0x7f8ad00d0180 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f8ac800cae0 tx=0x7f8ac800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:01.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.673+0000 7f8ab7fff640 1 -- 172.21.15.43:0/3714984128 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8017070 con 0x7f8ad015f5d0 2024-04-25T09:03:01.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.673+0000 7f8ab7fff640 1 -- 172.21.15.43:0/3714984128 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ac800ac40 con 0x7f8ad015f5d0 2024-04-25T09:03:01.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.673+0000 7f8ad5e5d640 1 -- 172.21.15.43:0/3714984128 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ad00cf080 con 0x7f8ad015f5d0 2024-04-25T09:03:01.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.673+0000 7f8ab7fff640 1 -- 172.21.15.43:0/3714984128 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8012400 con 0x7f8ad015f5d0 2024-04-25T09:03:01.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.673+0000 7f8ad5e5d640 1 -- 172.21.15.43:0/3714984128 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ad00cf640 con 0x7f8ad015f5d0 2024-04-25T09:03:01.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.674+0000 7f8ad5e5d640 1 -- 172.21.15.43:0/3714984128 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ad0068a40 con 0x7f8ad015f5d0 2024-04-25T09:03:01.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.675+0000 7f8ab7fff640 1 -- 172.21.15.43:0/3714984128 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8ac801a070 con 0x7f8ad015f5d0 2024-04-25T09:03:01.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.675+0000 7f8ab7fff640 1 --2- 172.21.15.43:0/3714984128 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8abc06b4d0 0x7f8abc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:01.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.675+0000 7f8ab7fff640 1 -- 172.21.15.43:0/3714984128 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f8ac8089f70 con 0x7f8ad015f5d0 2024-04-25T09:03:01.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.675+0000 7f8acf7fe640 1 --2- 172.21.15.43:0/3714984128 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8abc06b4d0 0x7f8abc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:01.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.676+0000 7f8acf7fe640 1 --2- 172.21.15.43:0/3714984128 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8abc06b4d0 0x7f8abc06d990 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0006fd0 tx=0x7f8ac0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:01.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.679+0000 7f8ab7fff640 1 -- 172.21.15.43:0/3714984128 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8ac8058df0 con 0x7f8ad015f5d0 2024-04-25T09:03:01.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:01.861+0000 7f8ad5e5d640 1 -- 172.21.15.43:0/3714984128 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8ad00740b0 con 0x7f8ad015f5d0 2024-04-25T09:03:02.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.498+0000 7f8ab7fff640 1 -- 172.21.15.43:0/3714984128 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v117) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8ac80586f0 con 0x7f8ad015f5d0 2024-04-25T09:03:02.487 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:03:02.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.502+0000 7f8ad5e5d640 1 -- 172.21.15.43:0/3714984128 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8abc06b4d0 msgr2=0x7f8abc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:02.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.502+0000 7f8ad5e5d640 1 --2- 172.21.15.43:0/3714984128 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8abc06b4d0 0x7f8abc06d990 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0006fd0 tx=0x7f8ac0008040 comp rx=0 tx=0).stop 2024-04-25T09:03:02.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.502+0000 7f8ad5e5d640 1 -- 172.21.15.43:0/3714984128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ad015f5d0 msgr2=0x7f8ad00d0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:02.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.502+0000 7f8ad5e5d640 1 --2- 172.21.15.43:0/3714984128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ad015f5d0 0x7f8ad00d0180 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f8ac800cae0 tx=0x7f8ac800a060 comp rx=0 tx=0).stop 2024-04-25T09:03:02.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.502+0000 7f8ad5e5d640 1 -- 172.21.15.43:0/3714984128 shutdown_connections 2024-04-25T09:03:02.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.502+0000 7f8ad5e5d640 1 --2- 172.21.15.43:0/3714984128 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8ad015f5d0 0x7f8ad00d0180 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:02.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.502+0000 7f8ad5e5d640 1 --2- 172.21.15.43:0/3714984128 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8ad015e410 0x7f8ad00ce910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:02.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.502+0000 7f8ad5e5d640 1 --2- 172.21.15.43:0/3714984128 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8abc06b4d0 0x7f8abc06d990 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:02.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.502+0000 7f8ad5e5d640 1 --2- 172.21.15.43:0/3714984128 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8ad015b6c0 0x7f8ad00c9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:02.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.503+0000 7f8ad5e5d640 1 -- 172.21.15.43:0/3714984128 >> 172.21.15.43:0/3714984128 conn(0x7f8ad008d460 msgr2=0x7f8ad015c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:02.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.503+0000 7f8ad5e5d640 1 -- 172.21.15.43:0/3714984128 shutdown_connections 2024-04-25T09:03:02.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.503+0000 7f8ad5e5d640 1 -- 172.21.15.43:0/3714984128 wait complete. 2024-04-25T09:03:02.507 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:03:02.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.630+0000 7f116c8bc640 1 Processor -- start 2024-04-25T09:03:02.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.630+0000 7f116c8bc640 1 -- start start 2024-04-25T09:03:02.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.631+0000 7f116c8bc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f116815baa0 0x7f116815e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:02.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.631+0000 7f116c8bc640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11680b37a0 con 0x7f11680b4ea0 2024-04-25T09:03:02.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.631+0000 7f116c8bc640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f116815eab0 con 0x7f116815b6c0 2024-04-25T09:03:02.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.631+0000 7f116c8bc640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f116815fce0 con 0x7f116815baa0 2024-04-25T09:03:02.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.631+0000 7f115ffff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f116815baa0 0x7f116815e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:02.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.631+0000 7f1166d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f11680b4ea0 0x7f11680d3760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:52942/0 (socket says 172.21.15.43:52942) 2024-04-25T09:03:02.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.631+0000 7f1166d9d640 1 -- 172.21.15.43:0/1563881421 learned_addr learned my addr 172.21.15.43:0/1563881421 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:03:02.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.631+0000 7f115ffff640 1 -- 172.21.15.43:0/1563881421 >> v1:172.21.15.193:6789/0 conn(0x7f116815b6c0 legacy=0x7f116815f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:02.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.631+0000 7f115ffff640 1 -- 172.21.15.43:0/1563881421 >> v1:172.21.15.43:6789/0 conn(0x7f11680b4ea0 legacy=0x7f11680d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:02.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.631+0000 7f115ffff640 1 -- 172.21.15.43:0/1563881421 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f116815ed90 con 0x7f116815baa0 2024-04-25T09:03:02.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.632+0000 7f115ffff640 1 --2- 172.21.15.43:0/1563881421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f116815baa0 0x7f116815e410 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f11500097a0 tx=0x7f115002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e64dbba35980647c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:02.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.632+0000 7f1165d9b640 1 -- 172.21.15.43:0/1563881421 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1150037070 con 0x7f116815baa0 2024-04-25T09:03:02.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.632+0000 7f1165d9b640 1 -- 172.21.15.43:0/1563881421 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1150032440 con 0x7f116815baa0 2024-04-25T09:03:02.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.632+0000 7f1165d9b640 1 -- 172.21.15.43:0/1563881421 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1150032810 con 0x7f116815baa0 2024-04-25T09:03:02.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.632+0000 7f116c8bc640 1 -- 172.21.15.43:0/1563881421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f116815baa0 msgr2=0x7f116815e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:02.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.632+0000 7f116c8bc640 1 --2- 172.21.15.43:0/1563881421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f116815baa0 0x7f116815e410 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f11500097a0 tx=0x7f115002a1e0 comp rx=0 tx=0).stop 2024-04-25T09:03:02.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.633+0000 7f116c8bc640 1 -- 172.21.15.43:0/1563881421 shutdown_connections 2024-04-25T09:03:02.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.633+0000 7f116c8bc640 1 --2- 172.21.15.43:0/1563881421 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f116815baa0 0x7f116815e410 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:02.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.633+0000 7f116c8bc640 1 -- 172.21.15.43:0/1563881421 >> 172.21.15.43:0/1563881421 conn(0x7f116808d460 msgr2=0x7f116809cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:02.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.633+0000 7f116c8bc640 1 -- 172.21.15.43:0/1563881421 shutdown_connections 2024-04-25T09:03:02.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.633+0000 7f116c8bc640 1 -- 172.21.15.43:0/1563881421 wait complete. 2024-04-25T09:03:02.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.634+0000 7f116c8bc640 1 Processor -- start 2024-04-25T09:03:02.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.634+0000 7f116c8bc640 1 -- start start 2024-04-25T09:03:02.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.635+0000 7f116c8bc640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f11680b4ea0 0x7f11680b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:02.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.635+0000 7f116c8bc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f116815b6c0 0x7f11680c9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:02.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.635+0000 7f116c8bc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f116815baa0 0x7f11680ccfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:02.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.635+0000 7f116c8bc640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11680b5310 con 0x7f116815baa0 2024-04-25T09:03:02.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.635+0000 7f116c8bc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116815f350 con 0x7f116815b6c0 2024-04-25T09:03:02.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.635+0000 7f116c8bc640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11680d3e70 con 0x7f11680b4ea0 2024-04-25T09:03:02.625 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.635+0000 7f1166d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f116815baa0 0x7f11680ccfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:02.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.635+0000 7f1166d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f116815baa0 0x7f11680ccfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44270/0 (socket says 172.21.15.43:44270) 2024-04-25T09:03:02.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.635+0000 7f1166d9d640 1 -- 172.21.15.43:0/517228076 learned_addr learned my addr 172.21.15.43:0/517228076 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:02.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.635+0000 7f115ffff640 1 --2- 172.21.15.43:0/517228076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f116815b6c0 0x7f11680c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:02.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.635+0000 7f116659c640 1 --2- 172.21.15.43:0/517228076 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f11680b4ea0 0x7f11680b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:02.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.635+0000 7f1166d9d640 1 -- 172.21.15.43:0/517228076 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f11680b4ea0 msgr2=0x7f11680b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:02.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.635+0000 7f1166d9d640 1 --2- 172.21.15.43:0/517228076 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f11680b4ea0 0x7f11680b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:02.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.635+0000 7f1166d9d640 1 -- 172.21.15.43:0/517228076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f116815b6c0 msgr2=0x7f11680c9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:02.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.635+0000 7f1166d9d640 1 --2- 172.21.15.43:0/517228076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f116815b6c0 0x7f11680c9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:02.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.635+0000 7f1166d9d640 1 -- 172.21.15.43:0/517228076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11680c9d10 con 0x7f116815baa0 2024-04-25T09:03:02.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.635+0000 7f115ffff640 1 --2- 172.21.15.43:0/517228076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f116815b6c0 0x7f11680c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:02.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.635+0000 7f116659c640 1 --2- 172.21.15.43:0/517228076 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f11680b4ea0 0x7f11680b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:02.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.635+0000 7f1166d9d640 1 --2- 172.21.15.43:0/517228076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f116815baa0 0x7f11680ccfe0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f116000cbc0 tx=0x7f116000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:02.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.636+0000 7f115effd640 1 -- 172.21.15.43:0/517228076 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1160017070 con 0x7f116815baa0 2024-04-25T09:03:02.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.636+0000 7f115effd640 1 -- 172.21.15.43:0/517228076 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f116000ad40 con 0x7f116815baa0 2024-04-25T09:03:02.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.636+0000 7f116c8bc640 1 -- 172.21.15.43:0/517228076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11680c9ff0 con 0x7f116815baa0 2024-04-25T09:03:02.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.636+0000 7f115effd640 1 -- 172.21.15.43:0/517228076 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1160012600 con 0x7f116815baa0 2024-04-25T09:03:02.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.636+0000 7f116c8bc640 1 -- 172.21.15.43:0/517228076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f116814ae80 con 0x7f116815baa0 2024-04-25T09:03:02.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.637+0000 7f116c8bc640 1 -- 172.21.15.43:0/517228076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1168068a40 con 0x7f116815baa0 2024-04-25T09:03:02.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.637+0000 7f115effd640 1 -- 172.21.15.43:0/517228076 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f116001a070 con 0x7f116815baa0 2024-04-25T09:03:02.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.638+0000 7f115effd640 1 --2- 172.21.15.43:0/517228076 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f114806b4d0 0x7f114806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:02.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.638+0000 7f115effd640 1 -- 172.21.15.43:0/517228076 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5415+0+0 (secure 0 0 0) 0x7f1160089d00 con 0x7f116815baa0 2024-04-25T09:03:02.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.638+0000 7f116659c640 1 --2- 172.21.15.43:0/517228076 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f114806b4d0 0x7f114806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:02.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.638+0000 7f116659c640 1 --2- 172.21.15.43:0/517228076 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f114806b4d0 0x7f114806d990 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f11680ca670 tx=0x7f1158009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:02.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.642+0000 7f115effd640 1 -- 172.21.15.43:0/517228076 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f11600586e0 con 0x7f116815baa0 2024-04-25T09:03:02.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:02.833+0000 7f116c8bc640 1 -- 172.21.15.43:0/517228076 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f116815dcd0 con 0x7f116815baa0 2024-04-25T09:03:03.488 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.500+0000 7f115effd640 1 -- 172.21.15.43:0/517228076 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v118) v1 ==== 169+0+0 (secure 0 0 0) 0x7f116005c780 con 0x7f116815baa0 2024-04-25T09:03:03.489 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' removed 2024-04-25T09:03:03.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.503+0000 7f116c8bc640 1 -- 172.21.15.43:0/517228076 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f114806b4d0 msgr2=0x7f114806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:03.491 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.503+0000 7f116c8bc640 1 --2- 172.21.15.43:0/517228076 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f114806b4d0 0x7f114806d990 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f11680ca670 tx=0x7f1158009040 comp rx=0 tx=0).stop 2024-04-25T09:03:03.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.503+0000 7f116c8bc640 1 -- 172.21.15.43:0/517228076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f116815baa0 msgr2=0x7f11680ccfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:03.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.503+0000 7f116c8bc640 1 --2- 172.21.15.43:0/517228076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f116815baa0 0x7f11680ccfe0 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f116000cbc0 tx=0x7f116000a430 comp rx=0 tx=0).stop 2024-04-25T09:03:03.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.503+0000 7f116c8bc640 1 -- 172.21.15.43:0/517228076 shutdown_connections 2024-04-25T09:03:03.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.503+0000 7f116c8bc640 1 --2- 172.21.15.43:0/517228076 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f116815baa0 0x7f11680ccfe0 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:03.492 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.503+0000 7f116c8bc640 1 --2- 172.21.15.43:0/517228076 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f114806b4d0 0x7f114806d990 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:03.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.503+0000 7f116c8bc640 1 --2- 172.21.15.43:0/517228076 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f116815b6c0 0x7f11680c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:03.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.503+0000 7f116c8bc640 1 --2- 172.21.15.43:0/517228076 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f11680b4ea0 0x7f11680b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:03.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.503+0000 7f116c8bc640 1 -- 172.21.15.43:0/517228076 >> 172.21.15.43:0/517228076 conn(0x7f116808d460 msgr2=0x7f116815ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:03.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.504+0000 7f116c8bc640 1 -- 172.21.15.43:0/517228076 shutdown_connections 2024-04-25T09:03:03.493 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.504+0000 7f116c8bc640 1 -- 172.21.15.43:0/517228076 wait complete. 2024-04-25T09:03:03.513 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T09:03:03.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.631+0000 7f4449abd640 1 Processor -- start 2024-04-25T09:03:03.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.631+0000 7f4449abd640 1 -- start start 2024-04-25T09:03:03.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.632+0000 7f4449abd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f44440d04a0 0x7f444415ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:03.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.632+0000 7f4449abd640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44440ce8c0 0x7f44440d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:03.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.632+0000 7f4449abd640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44440cf110 con 0x7f444415dec0 2024-04-25T09:03:03.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.632+0000 7f4449abd640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44440cf3f0 con 0x7f44440d04a0 2024-04-25T09:03:03.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.632+0000 7f4449abd640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44440cf6d0 con 0x7f44440ce8c0 2024-04-25T09:03:03.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.632+0000 7f44437fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f44440d04a0 0x7f444415ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:03.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.632+0000 7f44437fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f44440d04a0 0x7f444415ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44276/0 (socket says 172.21.15.43:44276) 2024-04-25T09:03:03.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.632+0000 7f44437fe640 1 -- 172.21.15.43:0/2168736943 learned_addr learned my addr 172.21.15.43:0/2168736943 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:03.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.632+0000 7f4443fff640 1 --2- 172.21.15.43:0/2168736943 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44440ce8c0 0x7f44440d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:03.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.632+0000 7f44437fe640 1 -- 172.21.15.43:0/2168736943 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44440ce8c0 msgr2=0x7f44440d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:03.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.632+0000 7f44437fe640 1 --2- 172.21.15.43:0/2168736943 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44440ce8c0 0x7f44440d17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:03.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.632+0000 7f44437fe640 1 -- 172.21.15.43:0/2168736943 >> v1:172.21.15.193:6789/0 conn(0x7f444415dec0 legacy=0x7f44440d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:03.622 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.632+0000 7f44437fe640 1 -- 172.21.15.43:0/2168736943 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44440cf9b0 con 0x7f44440d04a0 2024-04-25T09:03:03.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.632+0000 7f4443fff640 1 --2- 172.21.15.43:0/2168736943 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44440ce8c0 0x7f44440d17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:03.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.633+0000 7f44437fe640 1 --2- 172.21.15.43:0/2168736943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44440d04a0 0x7f444415ed90 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f44440b6690 tx=0x7f443402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e317ebe8e2cce06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:03.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.633+0000 7f44427fc640 1 -- 172.21.15.43:0/2168736943 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4434037070 con 0x7f44440d04a0 2024-04-25T09:03:03.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.633+0000 7f44427fc640 1 -- 172.21.15.43:0/2168736943 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4434032440 con 0x7f44440d04a0 2024-04-25T09:03:03.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.633+0000 7f44427fc640 1 -- 172.21.15.43:0/2168736943 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4434032810 con 0x7f44440d04a0 2024-04-25T09:03:03.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.634+0000 7f4449abd640 1 -- 172.21.15.43:0/2168736943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44440d04a0 msgr2=0x7f444415ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:03.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.634+0000 7f4449abd640 1 --2- 172.21.15.43:0/2168736943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44440d04a0 0x7f444415ed90 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f44440b6690 tx=0x7f443402a200 comp rx=0 tx=0).stop 2024-04-25T09:03:03.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.634+0000 7f4449abd640 1 -- 172.21.15.43:0/2168736943 shutdown_connections 2024-04-25T09:03:03.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.634+0000 7f4449abd640 1 --2- 172.21.15.43:0/2168736943 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44440ce8c0 0x7f44440d17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:03.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.634+0000 7f4449abd640 1 --2- 172.21.15.43:0/2168736943 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44440d04a0 0x7f444415ed90 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:03.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.634+0000 7f4449abd640 1 -- 172.21.15.43:0/2168736943 >> 172.21.15.43:0/2168736943 conn(0x7f444409cd40 msgr2=0x7f44440d3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:03.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.634+0000 7f4449abd640 1 -- 172.21.15.43:0/2168736943 shutdown_connections 2024-04-25T09:03:03.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.634+0000 7f4449abd640 1 -- 172.21.15.43:0/2168736943 wait complete. 2024-04-25T09:03:03.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.635+0000 7f4449abd640 1 Processor -- start 2024-04-25T09:03:03.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.635+0000 7f4449abd640 1 -- start start 2024-04-25T09:03:03.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.636+0000 7f4449abd640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44440ce8c0 0x7f44440b7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:03.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.636+0000 7f4449abd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44440d04a0 0x7f4444156890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:03.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.636+0000 7f4449abd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f444415dec0 0x7f44440c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:03.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.636+0000 7f4449abd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44440cee50 con 0x7f44440d04a0 2024-04-25T09:03:03.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.636+0000 7f4449abd640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44440cecd0 con 0x7f444415dec0 2024-04-25T09:03:03.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.636+0000 7f4449abd640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44440d1ce0 con 0x7f44440ce8c0 2024-04-25T09:03:03.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.636+0000 7f4442ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44440d04a0 0x7f4444156890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:03.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.636+0000 7f4442ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44440d04a0 0x7f4444156890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44292/0 (socket says 172.21.15.43:44292) 2024-04-25T09:03:03.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.636+0000 7f4442ffd640 1 -- 172.21.15.43:0/2706103994 learned_addr learned my addr 172.21.15.43:0/2706103994 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:03.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.636+0000 7f44437fe640 1 --2- 172.21.15.43:0/2706103994 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44440ce8c0 0x7f44440b7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:03.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.636+0000 7f4442ffd640 1 -- 172.21.15.43:0/2706103994 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44440ce8c0 msgr2=0x7f44440b7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:03.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.636+0000 7f4442ffd640 1 --2- 172.21.15.43:0/2706103994 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44440ce8c0 0x7f44440b7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:03.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.636+0000 7f4442ffd640 1 -- 172.21.15.43:0/2706103994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f444415dec0 msgr2=0x7f44440c9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:03.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.636+0000 7f4442ffd640 1 --2- 172.21.15.43:0/2706103994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f444415dec0 0x7f44440c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:03.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.636+0000 7f4442ffd640 1 -- 172.21.15.43:0/2706103994 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44440ca600 con 0x7f44440d04a0 2024-04-25T09:03:03.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.637+0000 7f4442ffd640 1 --2- 172.21.15.43:0/2706103994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44440d04a0 0x7f4444156890 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f443800b3f0 tx=0x7f443800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:03.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.637+0000 7f442bfff640 1 -- 172.21.15.43:0/2706103994 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4438015020 con 0x7f44440d04a0 2024-04-25T09:03:03.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.637+0000 7f442bfff640 1 -- 172.21.15.43:0/2706103994 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4438011830 con 0x7f44440d04a0 2024-04-25T09:03:03.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.637+0000 7f4449abd640 1 -- 172.21.15.43:0/2706103994 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44440cbd90 con 0x7f44440d04a0 2024-04-25T09:03:03.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.637+0000 7f442bfff640 1 -- 172.21.15.43:0/2706103994 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4438011c20 con 0x7f44440d04a0 2024-04-25T09:03:03.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.637+0000 7f4449abd640 1 -- 172.21.15.43:0/2706103994 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4444156dd0 con 0x7f44440d04a0 2024-04-25T09:03:03.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.638+0000 7f4449abd640 1 -- 172.21.15.43:0/2706103994 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4444074f60 con 0x7f44440d04a0 2024-04-25T09:03:03.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.638+0000 7f442bfff640 1 -- 172.21.15.43:0/2706103994 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4438018020 con 0x7f44440d04a0 2024-04-25T09:03:03.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.639+0000 7f442bfff640 1 --2- 172.21.15.43:0/2706103994 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f443006b4d0 0x7f443006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:03.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.639+0000 7f442bfff640 1 -- 172.21.15.43:0/2706103994 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4438089b30 con 0x7f44440d04a0 2024-04-25T09:03:03.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.639+0000 7f44437fe640 1 --2- 172.21.15.43:0/2706103994 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f443006b4d0 0x7f443006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:03.629 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.640+0000 7f44437fe640 1 --2- 172.21.15.43:0/2706103994 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f443006b4d0 0x7f443006d990 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f4434034000 tx=0x7f443402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:03.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.643+0000 7f442bfff640 1 -- 172.21.15.43:0/2706103994 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f443801f030 con 0x7f44440d04a0 2024-04-25T09:03:03.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.845+0000 7f4449abd640 1 -- 172.21.15.43:0/2706103994 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f444408daf0 con 0x7f44440d04a0 2024-04-25T09:03:03.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.848+0000 7f442bfff640 1 -- 172.21.15.43:0/2706103994 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7f44380172c0 con 0x7f44440d04a0 2024-04-25T09:03:03.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.850+0000 7f4449abd640 1 -- 172.21.15.43:0/2706103994 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f443006b4d0 msgr2=0x7f443006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:03.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.850+0000 7f4449abd640 1 --2- 172.21.15.43:0/2706103994 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f443006b4d0 0x7f443006d990 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f4434034000 tx=0x7f443402e000 comp rx=0 tx=0).stop 2024-04-25T09:03:03.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.850+0000 7f4449abd640 1 -- 172.21.15.43:0/2706103994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44440d04a0 msgr2=0x7f4444156890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:03.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.850+0000 7f4449abd640 1 --2- 172.21.15.43:0/2706103994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44440d04a0 0x7f4444156890 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f443800b3f0 tx=0x7f443800b8b0 comp rx=0 tx=0).stop 2024-04-25T09:03:03.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.851+0000 7f4449abd640 1 -- 172.21.15.43:0/2706103994 shutdown_connections 2024-04-25T09:03:03.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.851+0000 7f4449abd640 1 --2- 172.21.15.43:0/2706103994 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f443006b4d0 0x7f443006d990 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:03.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.851+0000 7f4449abd640 1 --2- 172.21.15.43:0/2706103994 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f444415dec0 0x7f44440c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:03.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.851+0000 7f4449abd640 1 --2- 172.21.15.43:0/2706103994 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44440d04a0 0x7f4444156890 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:03.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.851+0000 7f4449abd640 1 --2- 172.21.15.43:0/2706103994 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44440ce8c0 0x7f44440b7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:03.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.851+0000 7f4449abd640 1 -- 172.21.15.43:0/2706103994 >> 172.21.15.43:0/2706103994 conn(0x7f444409cd40 msgr2=0x7f44440d2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:03.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.851+0000 7f4449abd640 1 -- 172.21.15.43:0/2706103994 shutdown_connections 2024-04-25T09:03:03.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.851+0000 7f4449abd640 1 -- 172.21.15.43:0/2706103994 wait complete. 2024-04-25T09:03:03.851 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort' 2024-04-25T09:03:03.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.964+0000 7ff922ce2640 1 Processor -- start 2024-04-25T09:03:03.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.964+0000 7ff922ce2640 1 -- start start 2024-04-25T09:03:03.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.965+0000 7ff922ce2640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff91c15ef50 0x7ff91c0d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:03.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.965+0000 7ff922ce2640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff91c0d0920 0x7ff91c160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:03.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.965+0000 7ff922ce2640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff91c0cee80 con 0x7ff91c15dce0 2024-04-25T09:03:03.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.965+0000 7ff922ce2640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff91c0cec60 con 0x7ff91c15ef50 2024-04-25T09:03:03.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.965+0000 7ff922ce2640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c0d0d70 con 0x7ff91c0d0920 2024-04-25T09:03:03.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.965+0000 7ff91bfff640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff91c15ef50 0x7ff91c0d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:03.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.965+0000 7ff92127f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff91c0d0920 0x7ff91c160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:03.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.965+0000 7ff91bfff640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff91c15ef50 0x7ff91c0d03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:47862/0 (socket says 172.21.15.43:47862) 2024-04-25T09:03:03.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.965+0000 7ff920a7e640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7ff91c15dce0 0x7ff91c0ce550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:32928/0 (socket says 172.21.15.43:32928) 2024-04-25T09:03:03.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.965+0000 7ff92127f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff91c0d0920 0x7ff91c160980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:55320/0 (socket says 172.21.15.43:55320) 2024-04-25T09:03:03.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.965+0000 7ff91bfff640 1 -- 172.21.15.43:0/3323243484 learned_addr learned my addr 172.21.15.43:0/3323243484 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:03.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.966+0000 7ff91bfff640 1 -- 172.21.15.43:0/3323243484 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff91c0d0920 msgr2=0x7ff91c160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:03.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.966+0000 7ff91bfff640 1 --2- 172.21.15.43:0/3323243484 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff91c0d0920 0x7ff91c160980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:03.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.966+0000 7ff91bfff640 1 -- 172.21.15.43:0/3323243484 >> v1:172.21.15.193:6789/0 conn(0x7ff91c15dce0 legacy=0x7ff91c0ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:03.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.966+0000 7ff91bfff640 1 -- 172.21.15.43:0/3323243484 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff91c0d2920 con 0x7ff91c15ef50 2024-04-25T09:03:03.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.966+0000 7ff91b7fe640 1 -- 172.21.15.43:0/3323243484 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 242345010 0 0) 0x7ff91c0cee80 con 0x7ff91c15dce0 2024-04-25T09:03:03.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.966+0000 7ff92127f640 1 --2- 172.21.15.43:0/3323243484 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff91c0d0920 0x7ff91c160980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:03:03.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.966+0000 7ff91bfff640 1 --2- 172.21.15.43:0/3323243484 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff91c15ef50 0x7ff91c0d03e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff9100098a0 tx=0x7ff91002a200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e251b5963b8df9a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:03.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.967+0000 7ff91b7fe640 1 -- 172.21.15.43:0/3323243484 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff910037070 con 0x7ff91c15ef50 2024-04-25T09:03:03.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.967+0000 7ff91b7fe640 1 -- 172.21.15.43:0/3323243484 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff910032440 con 0x7ff91c15ef50 2024-04-25T09:03:03.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.967+0000 7ff91b7fe640 1 -- 172.21.15.43:0/3323243484 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff910032810 con 0x7ff91c15ef50 2024-04-25T09:03:03.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.967+0000 7ff922ce2640 1 -- 172.21.15.43:0/3323243484 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff91c15ef50 msgr2=0x7ff91c0d03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:03.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.967+0000 7ff922ce2640 1 --2- 172.21.15.43:0/3323243484 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff91c15ef50 0x7ff91c0d03e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff9100098a0 tx=0x7ff91002a200 comp rx=0 tx=0).stop 2024-04-25T09:03:03.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.967+0000 7ff922ce2640 1 -- 172.21.15.43:0/3323243484 shutdown_connections 2024-04-25T09:03:03.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.967+0000 7ff922ce2640 1 --2- 172.21.15.43:0/3323243484 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff91c0d0920 0x7ff91c160980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:03.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.967+0000 7ff922ce2640 1 --2- 172.21.15.43:0/3323243484 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff91c15ef50 0x7ff91c0d03e0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:03.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.967+0000 7ff922ce2640 1 -- 172.21.15.43:0/3323243484 >> 172.21.15.43:0/3323243484 conn(0x7ff91c09cd00 msgr2=0x7ff91c0d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:03.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.967+0000 7ff922ce2640 1 -- 172.21.15.43:0/3323243484 shutdown_connections 2024-04-25T09:03:03.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.967+0000 7ff922ce2640 1 -- 172.21.15.43:0/3323243484 wait complete. 2024-04-25T09:03:03.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.968+0000 7ff922ce2640 1 Processor -- start 2024-04-25T09:03:03.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.968+0000 7ff922ce2640 1 -- start start 2024-04-25T09:03:03.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.969+0000 7ff922ce2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff91c0d0920 0x7ff91c0cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:03.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.969+0000 7ff922ce2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff91c15dce0 0x7ff91c09d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:03.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.969+0000 7ff922ce2640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff91c15ef50 0x7ff91c09dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:03.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.969+0000 7ff922ce2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c0cf720 con 0x7ff91c15dce0 2024-04-25T09:03:03.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.969+0000 7ff922ce2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c0cf440 con 0x7ff91c0d0920 2024-04-25T09:03:03.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.969+0000 7ff922ce2640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff91c15f6b0 con 0x7ff91c15ef50 2024-04-25T09:03:03.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.969+0000 7ff91bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff91c15dce0 0x7ff91c09d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:03.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.970+0000 7ff91bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff91c15dce0 0x7ff91c09d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44308/0 (socket says 172.21.15.43:44308) 2024-04-25T09:03:03.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.970+0000 7ff91bfff640 1 -- 172.21.15.43:0/2466827801 learned_addr learned my addr 172.21.15.43:0/2466827801 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:03.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.970+0000 7ff920a7e640 1 --2- 172.21.15.43:0/2466827801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff91c0d0920 0x7ff91c0cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:03.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.970+0000 7ff92127f640 1 --2- 172.21.15.43:0/2466827801 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff91c15ef50 0x7ff91c09dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:03.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.970+0000 7ff91bfff640 1 -- 172.21.15.43:0/2466827801 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff91c15ef50 msgr2=0x7ff91c09dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:03.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.970+0000 7ff91bfff640 1 --2- 172.21.15.43:0/2466827801 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff91c15ef50 0x7ff91c09dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:03.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.970+0000 7ff91bfff640 1 -- 172.21.15.43:0/2466827801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff91c0d0920 msgr2=0x7ff91c0cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:03.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.970+0000 7ff91bfff640 1 --2- 172.21.15.43:0/2466827801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff91c0d0920 0x7ff91c0cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:03.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.970+0000 7ff91bfff640 1 -- 172.21.15.43:0/2466827801 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff91c0cc310 con 0x7ff91c15dce0 2024-04-25T09:03:03.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.970+0000 7ff91bfff640 1 --2- 172.21.15.43:0/2466827801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff91c15dce0 0x7ff91c09d770 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7ff9100098a0 tx=0x7ff9100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:03.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.970+0000 7ff9197fa640 1 -- 172.21.15.43:0/2466827801 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff910037040 con 0x7ff91c15dce0 2024-04-25T09:03:03.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.970+0000 7ff9197fa640 1 -- 172.21.15.43:0/2466827801 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff910004230 con 0x7ff91c15dce0 2024-04-25T09:03:03.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.970+0000 7ff922ce2640 1 -- 172.21.15.43:0/2466827801 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff91c0cc5f0 con 0x7ff91c15dce0 2024-04-25T09:03:03.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.970+0000 7ff9197fa640 1 -- 172.21.15.43:0/2466827801 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff910003610 con 0x7ff91c15dce0 2024-04-25T09:03:03.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.970+0000 7ff922ce2640 1 -- 172.21.15.43:0/2466827801 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff91c0ccbb0 con 0x7ff91c15dce0 2024-04-25T09:03:03.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.971+0000 7ff922ce2640 1 -- 172.21.15.43:0/2466827801 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff91c0680f0 con 0x7ff91c15dce0 2024-04-25T09:03:03.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.972+0000 7ff9197fa640 1 -- 172.21.15.43:0/2466827801 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff91002ad10 con 0x7ff91c15dce0 2024-04-25T09:03:03.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.972+0000 7ff9197fa640 1 --2- 172.21.15.43:0/2466827801 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff90806b4d0 0x7ff90806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:03.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.972+0000 7ff9197fa640 1 -- 172.21.15.43:0/2466827801 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff9100aaab0 con 0x7ff91c15dce0 2024-04-25T09:03:03.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.972+0000 7ff920a7e640 1 --2- 172.21.15.43:0/2466827801 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff90806b4d0 0x7ff90806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:03.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.973+0000 7ff920a7e640 1 --2- 172.21.15.43:0/2466827801 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff90806b4d0 0x7ff90806d990 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff90c000f60 tx=0x7ff90c007b50 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:03.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:03.976+0000 7ff9197fa640 1 -- 172.21.15.43:0/2466827801 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff910079d50 con 0x7ff91c15dce0 2024-04-25T09:03:04.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.234+0000 7ff922ce2640 1 -- 172.21.15.43:0/2466827801 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]} v 0) v1 -- 0x7ff91c074ef0 con 0x7ff91c15dce0 2024-04-25T09:03:04.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.513+0000 7ff9197fa640 1 -- 172.21.15.43:0/2466827801 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]}]=0 v429) v1 ==== 166+0+0 (secure 0 0 0) 0x7ff910079650 con 0x7ff91c15dce0 2024-04-25T09:03:04.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.516+0000 7ff922ce2640 1 -- 172.21.15.43:0/2466827801 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff90806b4d0 msgr2=0x7ff90806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:04.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.516+0000 7ff922ce2640 1 --2- 172.21.15.43:0/2466827801 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff90806b4d0 0x7ff90806d990 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff90c000f60 tx=0x7ff90c007b50 comp rx=0 tx=0).stop 2024-04-25T09:03:04.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.516+0000 7ff922ce2640 1 -- 172.21.15.43:0/2466827801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff91c15dce0 msgr2=0x7ff91c09d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:04.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.516+0000 7ff922ce2640 1 --2- 172.21.15.43:0/2466827801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff91c15dce0 0x7ff91c09d770 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7ff9100098a0 tx=0x7ff9100046c0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.516+0000 7ff922ce2640 1 -- 172.21.15.43:0/2466827801 shutdown_connections 2024-04-25T09:03:04.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.516+0000 7ff922ce2640 1 --2- 172.21.15.43:0/2466827801 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff91c15ef50 0x7ff91c09dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.516+0000 7ff922ce2640 1 --2- 172.21.15.43:0/2466827801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff91c15dce0 0x7ff91c09d770 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.516+0000 7ff922ce2640 1 --2- 172.21.15.43:0/2466827801 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff90806b4d0 0x7ff90806d990 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.516+0000 7ff922ce2640 1 --2- 172.21.15.43:0/2466827801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff91c0d0920 0x7ff91c0cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.516+0000 7ff922ce2640 1 -- 172.21.15.43:0/2466827801 >> 172.21.15.43:0/2466827801 conn(0x7ff91c09cd00 msgr2=0x7ff91c06c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:04.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.516+0000 7ff922ce2640 1 -- 172.21.15.43:0/2466827801 shutdown_connections 2024-04-25T09:03:04.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.517+0000 7ff922ce2640 1 -- 172.21.15.43:0/2466827801 wait complete. 2024-04-25T09:03:04.517 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume_with_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T09:03:04.517 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_task_status_on_mds_failover (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T09:03:04.518 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:04.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.630+0000 7fbcc1ae1640 1 Processor -- start 2024-04-25T09:03:04.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.630+0000 7fbcc1ae1640 1 -- start start 2024-04-25T09:03:04.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.631+0000 7fbcc1ae1640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbcbc15dce0 0x7fbcbc15ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:04.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.631+0000 7fbcc1ae1640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fbcbc0ce550 0x7fbcbc0d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:04.620 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.631+0000 7fbcc1ae1640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbcbc0ce990 0x7fbcbc0d0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:04.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.631+0000 7fbcc1ae1640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcbc0cee80 con 0x7fbcbc0ce990 2024-04-25T09:03:04.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.631+0000 7fbcc1ae1640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcbc0cf160 con 0x7fbcbc0ce550 2024-04-25T09:03:04.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.631+0000 7fbcc1ae1640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc0cf440 con 0x7fbcbc15dce0 2024-04-25T09:03:04.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.631+0000 7fbcbbfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbcbc0ce990 0x7fbcbc0d0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:04.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.631+0000 7fbcbbfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbcbc0ce990 0x7fbcbc0d0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44316/0 (socket says 172.21.15.43:44316) 2024-04-25T09:03:04.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.631+0000 7fbcbbfff640 1 -- 172.21.15.43:0/2505989491 learned_addr learned my addr 172.21.15.43:0/2505989491 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:04.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.632+0000 7fbcbaffd640 1 --2- 172.21.15.43:0/2505989491 >> v2:172.21.15.193:3300/0 conn(0x7fbcbc0ce550 0x7fbcbc0d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:04.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.632+0000 7fbcbb7fe640 1 --2- 172.21.15.43:0/2505989491 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbcbc15dce0 0x7fbcbc15ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:04.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.632+0000 7fbcbbfff640 1 -- 172.21.15.43:0/2505989491 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbcbc15dce0 msgr2=0x7fbcbc15ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:04.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.632+0000 7fbcbbfff640 1 --2- 172.21.15.43:0/2505989491 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbcbc15dce0 0x7fbcbc15ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.632+0000 7fbcbbfff640 1 -- 172.21.15.43:0/2505989491 >> v2:172.21.15.193:3300/0 conn(0x7fbcbc0ce550 msgr2=0x7fbcbc0d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:04.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.632+0000 7fbcbbfff640 1 --2- 172.21.15.43:0/2505989491 >> v2:172.21.15.193:3300/0 conn(0x7fbcbc0ce550 0x7fbcbc0d03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.621 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.632+0000 7fbcbbfff640 1 -- 172.21.15.43:0/2505989491 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcbc0cf720 con 0x7fbcbc0ce990 2024-04-25T09:03:04.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.632+0000 7fbcbbfff640 1 --2- 172.21.15.43:0/2505989491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcbc0ce990 0x7fbcbc0d0920 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc09d430 tx=0x7fbcb402a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cfaec8567916087 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:04.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.633+0000 7fbcba7fc640 1 -- 172.21.15.43:0/2505989491 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4036070 con 0x7fbcbc0ce990 2024-04-25T09:03:04.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.633+0000 7fbcba7fc640 1 -- 172.21.15.43:0/2505989491 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcb4031600 con 0x7fbcbc0ce990 2024-04-25T09:03:04.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.633+0000 7fbcba7fc640 1 -- 172.21.15.43:0/2505989491 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb40319d0 con 0x7fbcbc0ce990 2024-04-25T09:03:04.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.633+0000 7fbcc1ae1640 1 -- 172.21.15.43:0/2505989491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcbc0ce990 msgr2=0x7fbcbc0d0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:04.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.633+0000 7fbcc1ae1640 1 --2- 172.21.15.43:0/2505989491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcbc0ce990 0x7fbcbc0d0920 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc09d430 tx=0x7fbcb402a610 comp rx=0 tx=0).stop 2024-04-25T09:03:04.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.633+0000 7fbcc1ae1640 1 -- 172.21.15.43:0/2505989491 shutdown_connections 2024-04-25T09:03:04.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.633+0000 7fbcc1ae1640 1 --2- 172.21.15.43:0/2505989491 >> v2:172.21.15.193:3300/0 conn(0x7fbcbc0ce550 0x7fbcbc0d03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.633+0000 7fbcc1ae1640 1 --2- 172.21.15.43:0/2505989491 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcbc0ce990 0x7fbcbc0d0920 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.633+0000 7fbcc1ae1640 1 --2- 172.21.15.43:0/2505989491 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbcbc15dce0 0x7fbcbc15ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.633+0000 7fbcc1ae1640 1 -- 172.21.15.43:0/2505989491 >> 172.21.15.43:0/2505989491 conn(0x7fbcbc09cd00 msgr2=0x7fbcbc0d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:04.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.633+0000 7fbcc1ae1640 1 -- 172.21.15.43:0/2505989491 shutdown_connections 2024-04-25T09:03:04.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.633+0000 7fbcc1ae1640 1 -- 172.21.15.43:0/2505989491 wait complete. 2024-04-25T09:03:04.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.634+0000 7fbcc1ae1640 1 Processor -- start 2024-04-25T09:03:04.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.635+0000 7fbcc1ae1640 1 -- start start 2024-04-25T09:03:04.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.635+0000 7fbcc1ae1640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbcbc0ce550 0x7fbcbc15a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:04.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.635+0000 7fbcc1ae1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcbc0ce990 0x7fbcbc09dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:04.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.635+0000 7fbcc1ae1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcbc15dce0 0x7fbcbc0d1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:04.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.635+0000 7fbcc1ae1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc160980 con 0x7fbcbc0ce990 2024-04-25T09:03:04.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.635+0000 7fbcc1ae1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc15e0c0 con 0x7fbcbc15dce0 2024-04-25T09:03:04.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.635+0000 7fbcc1ae1640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcbc160b00 con 0x7fbcbc0ce550 2024-04-25T09:03:04.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.635+0000 7fbcbaffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcbc0ce990 0x7fbcbc09dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:04.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.635+0000 7fbcbaffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcbc0ce990 0x7fbcbc09dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44324/0 (socket says 172.21.15.43:44324) 2024-04-25T09:03:04.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.635+0000 7fbcbaffd640 1 -- 172.21.15.43:0/241569460 learned_addr learned my addr 172.21.15.43:0/241569460 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:04.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.635+0000 7fbcbbfff640 1 --2- 172.21.15.43:0/241569460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcbc15dce0 0x7fbcbc0d1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:04.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.636+0000 7fbcbb7fe640 1 --2- 172.21.15.43:0/241569460 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbcbc0ce550 0x7fbcbc15a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:04.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.636+0000 7fbcbaffd640 1 -- 172.21.15.43:0/241569460 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbcbc0ce550 msgr2=0x7fbcbc15a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:04.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.636+0000 7fbcbaffd640 1 --2- 172.21.15.43:0/241569460 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbcbc0ce550 0x7fbcbc15a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.636+0000 7fbcbaffd640 1 -- 172.21.15.43:0/241569460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcbc15dce0 msgr2=0x7fbcbc0d1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:04.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.636+0000 7fbcbaffd640 1 --2- 172.21.15.43:0/241569460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcbc15dce0 0x7fbcbc0d1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.636+0000 7fbcbaffd640 1 -- 172.21.15.43:0/241569460 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcbc15b320 con 0x7fbcbc0ce990 2024-04-25T09:03:04.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.636+0000 7fbcbb7fe640 1 --2- 172.21.15.43:0/241569460 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbcbc0ce550 0x7fbcbc15a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:04.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.636+0000 7fbcbbfff640 1 --2- 172.21.15.43:0/241569460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcbc15dce0 0x7fbcbc0d1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:04.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.636+0000 7fbcbaffd640 1 --2- 172.21.15.43:0/241569460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcbc0ce990 0x7fbcbc09dff0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000d800 tx=0x7fbcb000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:04.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.636+0000 7fbca3fff640 1 -- 172.21.15.43:0/241569460 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb000b6c0 con 0x7fbcbc0ce990 2024-04-25T09:03:04.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.636+0000 7fbca3fff640 1 -- 172.21.15.43:0/241569460 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcb0019070 con 0x7fbcbc0ce990 2024-04-25T09:03:04.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.636+0000 7fbcc1ae1640 1 -- 172.21.15.43:0/241569460 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcbc15b5e0 con 0x7fbcbc0ce990 2024-04-25T09:03:04.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.636+0000 7fbca3fff640 1 -- 172.21.15.43:0/241569460 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb000ba80 con 0x7fbcbc0ce990 2024-04-25T09:03:04.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.636+0000 7fbcc1ae1640 1 -- 172.21.15.43:0/241569460 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcbc15bba0 con 0x7fbcbc0ce990 2024-04-25T09:03:04.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.637+0000 7fbcc1ae1640 1 -- 172.21.15.43:0/241569460 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcbc0680f0 con 0x7fbcbc0ce990 2024-04-25T09:03:04.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.637+0000 7fbca3fff640 1 -- 172.21.15.43:0/241569460 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbcb000bc20 con 0x7fbcbc0ce990 2024-04-25T09:03:04.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.638+0000 7fbca3fff640 1 --2- 172.21.15.43:0/241569460 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbca806b4d0 0x7fbca806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:04.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.638+0000 7fbca3fff640 1 -- 172.21.15.43:0/241569460 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbcb008a320 con 0x7fbcbc0ce990 2024-04-25T09:03:04.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.638+0000 7fbcbb7fe640 1 --2- 172.21.15.43:0/241569460 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbca806b4d0 0x7fbca806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:04.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.639+0000 7fbcbb7fe640 1 --2- 172.21.15.43:0/241569460 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbca806b4d0 0x7fbca806d990 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fbcac00b970 tx=0x7fbcac008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:04.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.642+0000 7fbca3fff640 1 -- 172.21.15.43:0/241569460 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbcb0059530 con 0x7fbcbc0ce990 2024-04-25T09:03:04.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.841+0000 7fbcc1ae1640 1 -- 172.21.15.43:0/241569460 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbcbc0732d0 con 0x7fbcbc0ce990 2024-04-25T09:03:04.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.842+0000 7fbca3fff640 1 -- 172.21.15.43:0/241569460 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+12994 (secure 0 0 0) 0x7fbcbc0732d0 con 0x7fbcbc0ce990 2024-04-25T09:03:04.831 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:03:04.831 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":118,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:03:03.499623+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6835/1586865031":"2024-04-26T09:03:00.492434+0000","172.21.15.43:6833/1586865031":"2024-04-26T09:03:00.492434+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:04.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.844+0000 7fbcc1ae1640 1 -- 172.21.15.43:0/241569460 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbca806b4d0 msgr2=0x7fbca806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:04.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.844+0000 7fbcc1ae1640 1 --2- 172.21.15.43:0/241569460 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbca806b4d0 0x7fbca806d990 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fbcac00b970 tx=0x7fbcac008040 comp rx=0 tx=0).stop 2024-04-25T09:03:04.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.844+0000 7fbcc1ae1640 1 -- 172.21.15.43:0/241569460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcbc0ce990 msgr2=0x7fbcbc09dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:04.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.844+0000 7fbcc1ae1640 1 --2- 172.21.15.43:0/241569460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcbc0ce990 0x7fbcbc09dff0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fbcb000d800 tx=0x7fbcb000dcc0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.844+0000 7fbcc1ae1640 1 -- 172.21.15.43:0/241569460 shutdown_connections 2024-04-25T09:03:04.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.844+0000 7fbcc1ae1640 1 --2- 172.21.15.43:0/241569460 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbca806b4d0 0x7fbca806d990 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.844+0000 7fbcc1ae1640 1 --2- 172.21.15.43:0/241569460 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbcbc15dce0 0x7fbcbc0d1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.844+0000 7fbcc1ae1640 1 --2- 172.21.15.43:0/241569460 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbcbc0ce990 0x7fbcbc09dff0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.844+0000 7fbcc1ae1640 1 --2- 172.21.15.43:0/241569460 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbcbc0ce550 0x7fbcbc15a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.844+0000 7fbcc1ae1640 1 -- 172.21.15.43:0/241569460 >> 172.21.15.43:0/241569460 conn(0x7fbcbc09cd00 msgr2=0x7fbcbc06b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:04.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.844+0000 7fbcc1ae1640 1 -- 172.21.15.43:0/241569460 shutdown_connections 2024-04-25T09:03:04.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.844+0000 7fbcc1ae1640 1 -- 172.21.15.43:0/241569460 wait complete. 2024-04-25T09:03:04.844 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:03:04.845 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:03:04.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.960+0000 7f293c33b640 1 Processor -- start 2024-04-25T09:03:04.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.960+0000 7f293c33b640 1 -- start start 2024-04-25T09:03:04.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.961+0000 7f293c33b640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f29340b5560 0x7f29340b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:04.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.961+0000 7f293c33b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2934162610 con 0x7f293415e090 2024-04-25T09:03:04.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.961+0000 7f293c33b640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29340d1750 con 0x7f29340d2230 2024-04-25T09:03:04.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.961+0000 7f293c33b640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29340d1da0 con 0x7f29340b5560 2024-04-25T09:03:04.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.961+0000 7f293a0d7640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f293415e090 0x7f2934161aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:52958/0 (socket says 172.21.15.43:52958) 2024-04-25T09:03:04.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.961+0000 7f293a0d7640 1 -- 172.21.15.43:0/2966996045 learned_addr learned my addr 172.21.15.43:0/2966996045 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:03:04.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.962+0000 7f29398d6640 1 --2- 172.21.15.43:0/2966996045 >> v2:172.21.15.193:3300/0 conn(0x7f29340b5560 0x7f29340b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:04.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.962+0000 7f29390d5640 1 -- 172.21.15.43:0/2966996045 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 500361846 0 0) 0x7f2934162610 con 0x7f293415e090 2024-04-25T09:03:04.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.962+0000 7f29390d5640 1 -- 172.21.15.43:0/2966996045 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f291c003520 con 0x7f293415e090 2024-04-25T09:03:04.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.962+0000 7f29390d5640 1 -- 172.21.15.43:0/2966996045 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3304313552 0 0) 0x7f29340d1750 con 0x7f29340d2230 2024-04-25T09:03:04.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.962+0000 7f29390d5640 1 -- 172.21.15.43:0/2966996045 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2934162610 con 0x7f29340d2230 2024-04-25T09:03:04.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.962+0000 7f29390d5640 1 -- 172.21.15.43:0/2966996045 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3827192450 0 0) 0x7f291c003520 con 0x7f293415e090 2024-04-25T09:03:04.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.962+0000 7f29390d5640 1 -- 172.21.15.43:0/2966996045 >> v2:172.21.15.193:3300/0 conn(0x7f29340b5560 msgr2=0x7f29340b59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:04.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.962+0000 7f29390d5640 1 --2- 172.21.15.43:0/2966996045 >> v2:172.21.15.193:3300/0 conn(0x7f29340b5560 0x7f29340b59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.962+0000 7f29390d5640 1 -- 172.21.15.43:0/2966996045 >> v1:172.21.15.193:6789/0 conn(0x7f29340d2230 legacy=0x7f29340d26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:04.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.962+0000 7f29390d5640 1 -- 172.21.15.43:0/2966996045 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29340d1a30 con 0x7f293415e090 2024-04-25T09:03:04.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.962+0000 7f29390d5640 1 -- 172.21.15.43:0/2966996045 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f292c0015f0 con 0x7f293415e090 2024-04-25T09:03:04.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.962+0000 7f29390d5640 1 -- 172.21.15.43:0/2966996045 >> v1:172.21.15.43:6789/0 conn(0x7f293415e090 legacy=0x7f2934161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:04.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.962+0000 7f29390d5640 1 --2- 172.21.15.43:0/2966996045 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f291c003b90 0x7f291c003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:04.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.962+0000 7f29390d5640 1 --2- 172.21.15.43:0/2966996045 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f291c004800 0x7f291c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:04.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.962+0000 7f29390d5640 1 --2- 172.21.15.43:0/2966996045 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f291c0051c0 0x7f291c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:04.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.962+0000 7f29390d5640 1 -- 172.21.15.43:0/2966996045 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29340d1da0 con 0x7f291c0051c0 2024-04-25T09:03:04.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.962+0000 7f29390d5640 1 -- 172.21.15.43:0/2966996045 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f291c00bfe0 con 0x7f291c004800 2024-04-25T09:03:04.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.962+0000 7f29390d5640 1 -- 172.21.15.43:0/2966996045 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f291c00c1e0 con 0x7f291c003b90 2024-04-25T09:03:04.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.962+0000 7f293a8d8640 1 --2- 172.21.15.43:0/2966996045 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f291c0051c0 0x7f291c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:04.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.962+0000 7f29398d6640 1 --2- 172.21.15.43:0/2966996045 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f291c004800 0x7f291c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:04.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.962+0000 7f293a0d7640 1 --2- 172.21.15.43:0/2966996045 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f291c003b90 0x7f291c003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:04.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.962+0000 7f293a8d8640 1 -- 172.21.15.43:0/2966996045 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f291c003b90 msgr2=0x7f291c003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:04.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.962+0000 7f293a8d8640 1 --2- 172.21.15.43:0/2966996045 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f291c003b90 0x7f291c003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.962+0000 7f293a8d8640 1 -- 172.21.15.43:0/2966996045 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f291c004800 msgr2=0x7f291c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:04.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.962+0000 7f293a8d8640 1 --2- 172.21.15.43:0/2966996045 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f291c004800 0x7f291c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.962+0000 7f293a8d8640 1 -- 172.21.15.43:0/2966996045 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f291c003520 con 0x7f291c0051c0 2024-04-25T09:03:04.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.963+0000 7f293a8d8640 1 -- 172.21.15.43:0/2966996045 reap_dead start 2024-04-25T09:03:04.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.963+0000 7f293a8d8640 1 --2- 172.21.15.43:0/2966996045 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f291c0051c0 0x7f291c00b9c0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f2924001fd0 tx=0x7f291c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf3e42eaa4406a4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:04.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.963+0000 7f29390d5640 1 -- 172.21.15.43:0/2966996045 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2924001bf0 con 0x7f291c0051c0 2024-04-25T09:03:04.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.963+0000 7f29390d5640 1 -- 172.21.15.43:0/2966996045 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2924003b70 con 0x7f291c0051c0 2024-04-25T09:03:04.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.963+0000 7f29390d5640 1 -- 172.21.15.43:0/2966996045 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29240050d0 con 0x7f291c0051c0 2024-04-25T09:03:04.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.964+0000 7f293c33b640 1 -- 172.21.15.43:0/2966996045 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f291c0051c0 msgr2=0x7f291c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:04.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.964+0000 7f293c33b640 1 --2- 172.21.15.43:0/2966996045 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f291c0051c0 0x7f291c00b9c0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f2924001fd0 tx=0x7f291c005830 comp rx=0 tx=0).stop 2024-04-25T09:03:04.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.964+0000 7f293c33b640 1 -- 172.21.15.43:0/2966996045 shutdown_connections 2024-04-25T09:03:04.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.964+0000 7f293c33b640 1 --2- 172.21.15.43:0/2966996045 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f291c0051c0 0x7f291c00b9c0 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.964+0000 7f293c33b640 1 -- 172.21.15.43:0/2966996045 >> 172.21.15.43:0/2966996045 conn(0x7f29340d0560 msgr2=0x7f29340d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:04.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.964+0000 7f293c33b640 1 -- 172.21.15.43:0/2966996045 shutdown_connections 2024-04-25T09:03:04.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.964+0000 7f293c33b640 1 -- 172.21.15.43:0/2966996045 wait complete. 2024-04-25T09:03:04.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.965+0000 7f293c33b640 1 Processor -- start 2024-04-25T09:03:04.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.965+0000 7f293c33b640 1 -- start start 2024-04-25T09:03:04.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.966+0000 7f293c33b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f291c0051c0 0x7f2934161b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:04.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.966+0000 7f293c33b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2934153600 0x7f29341521f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:04.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.966+0000 7f293c33b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2934155260 0x7f29341556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:04.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.966+0000 7f293c33b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29340b59e0 con 0x7f2934155260 2024-04-25T09:03:04.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.966+0000 7f293c33b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29340b3280 con 0x7f291c0051c0 2024-04-25T09:03:04.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.966+0000 7f293c33b640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29340b5b60 con 0x7f2934153600 2024-04-25T09:03:04.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.966+0000 7f293a8d8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2934155260 0x7f29341556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:04.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.966+0000 7f293a8d8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2934155260 0x7f29341556e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44344/0 (socket says 172.21.15.43:44344) 2024-04-25T09:03:04.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.966+0000 7f293a8d8640 1 -- 172.21.15.43:0/2099296448 learned_addr learned my addr 172.21.15.43:0/2099296448 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:04.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.966+0000 7f293a0d7640 1 --2- 172.21.15.43:0/2099296448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f291c0051c0 0x7f2934161b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:04.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.966+0000 7f29398d6640 1 --2- 172.21.15.43:0/2099296448 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2934153600 0x7f29341521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:04.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.966+0000 7f293a8d8640 1 -- 172.21.15.43:0/2099296448 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2934153600 msgr2=0x7f29341521f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:04.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.966+0000 7f293a8d8640 1 --2- 172.21.15.43:0/2099296448 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2934153600 0x7f29341521f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.966+0000 7f293a8d8640 1 -- 172.21.15.43:0/2099296448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f291c0051c0 msgr2=0x7f2934161b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:04.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.966+0000 7f293a8d8640 1 --2- 172.21.15.43:0/2099296448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f291c0051c0 0x7f2934161b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:04.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.966+0000 7f293a8d8640 1 -- 172.21.15.43:0/2099296448 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2934162300 con 0x7f2934155260 2024-04-25T09:03:04.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.967+0000 7f293a8d8640 1 --2- 172.21.15.43:0/2099296448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2934155260 0x7f29341556e0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f29240020c0 tx=0x7f2924001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:04.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.967+0000 7f2932ffd640 1 -- 172.21.15.43:0/2099296448 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2924003970 con 0x7f2934155260 2024-04-25T09:03:04.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.967+0000 7f2932ffd640 1 -- 172.21.15.43:0/2099296448 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29240052e0 con 0x7f2934155260 2024-04-25T09:03:04.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.967+0000 7f293c33b640 1 -- 172.21.15.43:0/2099296448 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29341573d0 con 0x7f2934155260 2024-04-25T09:03:04.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.967+0000 7f2932ffd640 1 -- 172.21.15.43:0/2099296448 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29240056b0 con 0x7f2934155260 2024-04-25T09:03:04.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.967+0000 7f293c33b640 1 -- 172.21.15.43:0/2099296448 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2934157990 con 0x7f2934155260 2024-04-25T09:03:04.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.968+0000 7f293c33b640 1 -- 172.21.15.43:0/2099296448 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f293406bf00 con 0x7f2934155260 2024-04-25T09:03:04.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.968+0000 7f2932ffd640 1 -- 172.21.15.43:0/2099296448 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2924039070 con 0x7f2934155260 2024-04-25T09:03:04.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.969+0000 7f2932ffd640 1 --2- 172.21.15.43:0/2099296448 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f292006b4d0 0x7f292006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:04.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.969+0000 7f2932ffd640 1 -- 172.21.15.43:0/2099296448 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f29240a9360 con 0x7f2934155260 2024-04-25T09:03:04.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.969+0000 7f293a0d7640 1 --2- 172.21.15.43:0/2099296448 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f292006b4d0 0x7f292006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:04.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.969+0000 7f293a0d7640 1 --2- 172.21.15.43:0/2099296448 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f292006b4d0 0x7f292006d990 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f292c001050 tx=0x7f292c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:04.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:04.973+0000 7f2932ffd640 1 -- 172.21.15.43:0/2099296448 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f293406bf00 con 0x7f2934155260 2024-04-25T09:03:05.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.153+0000 7f293c33b640 1 -- 172.21.15.43:0/2099296448 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2934074f60 con 0x7f2934155260 2024-04-25T09:03:05.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.154+0000 7f2932ffd640 1 -- 172.21.15.43:0/2099296448 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v118) v1 ==== 93+0+10 (secure 0 0 0) 0x7f29240784e0 con 0x7f2934155260 2024-04-25T09:03:05.143 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T09:03:05.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.156+0000 7f293c33b640 1 -- 172.21.15.43:0/2099296448 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f292006b4d0 msgr2=0x7f292006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:05.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.156+0000 7f293c33b640 1 --2- 172.21.15.43:0/2099296448 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f292006b4d0 0x7f292006d990 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f292c001050 tx=0x7f292c009040 comp rx=0 tx=0).stop 2024-04-25T09:03:05.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.156+0000 7f293c33b640 1 -- 172.21.15.43:0/2099296448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2934155260 msgr2=0x7f29341556e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:05.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.156+0000 7f293c33b640 1 --2- 172.21.15.43:0/2099296448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2934155260 0x7f29341556e0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7f29240020c0 tx=0x7f2924001650 comp rx=0 tx=0).stop 2024-04-25T09:03:05.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.156+0000 7f293c33b640 1 -- 172.21.15.43:0/2099296448 shutdown_connections 2024-04-25T09:03:05.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.156+0000 7f293c33b640 1 --2- 172.21.15.43:0/2099296448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2934155260 0x7f29341556e0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:05.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.156+0000 7f293c33b640 1 --2- 172.21.15.43:0/2099296448 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2934153600 0x7f29341521f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:05.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.156+0000 7f293c33b640 1 --2- 172.21.15.43:0/2099296448 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f292006b4d0 0x7f292006d990 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:05.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.156+0000 7f293c33b640 1 --2- 172.21.15.43:0/2099296448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f291c0051c0 0x7f2934161b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:05.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.156+0000 7f293c33b640 1 -- 172.21.15.43:0/2099296448 >> 172.21.15.43:0/2099296448 conn(0x7f29340d0560 msgr2=0x7f293415d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:05.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.156+0000 7f293c33b640 1 -- 172.21.15.43:0/2099296448 shutdown_connections 2024-04-25T09:03:05.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.156+0000 7f293c33b640 1 -- 172.21.15.43:0/2099296448 wait complete. 2024-04-25T09:03:05.155 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover' 2024-04-25T09:03:05.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.268+0000 7fb349553640 1 Processor -- start 2024-04-25T09:03:05.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.268+0000 7fb349553640 1 -- start start 2024-04-25T09:03:05.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.269+0000 7fb349553640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb3440d09b0 0x7fb3440b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:05.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.269+0000 7fb349553640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb3440ce5e0 0x7fb3440cea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:05.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.269+0000 7fb349553640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3440b4c30 con 0x7fb344161390 2024-04-25T09:03:05.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.269+0000 7fb349553640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3440d1910 con 0x7fb3440ce5e0 2024-04-25T09:03:05.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.269+0000 7fb349553640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3440cf2c0 con 0x7fb3440d09b0 2024-04-25T09:03:05.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.269+0000 7fb3437fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb3440ce5e0 0x7fb3440cea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:05.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.269+0000 7fb3437fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb3440ce5e0 0x7fb3440cea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44360/0 (socket says 172.21.15.43:44360) 2024-04-25T09:03:05.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.269+0000 7fb3437fe640 1 -- 172.21.15.43:0/2417583895 learned_addr learned my addr 172.21.15.43:0/2417583895 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:05.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.269+0000 7fb3427fc640 1 --2- 172.21.15.43:0/2417583895 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb3440d09b0 0x7fb3440b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:05.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.269+0000 7fb3437fe640 1 -- 172.21.15.43:0/2417583895 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb3440d09b0 msgr2=0x7fb3440b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:05.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.269+0000 7fb3437fe640 1 --2- 172.21.15.43:0/2417583895 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb3440d09b0 0x7fb3440b6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:05.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.269+0000 7fb3437fe640 1 -- 172.21.15.43:0/2417583895 >> v1:172.21.15.193:6789/0 conn(0x7fb344161390 legacy=0x7fb3440d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:05.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.269+0000 7fb3437fe640 1 -- 172.21.15.43:0/2417583895 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3440cf5a0 con 0x7fb3440ce5e0 2024-04-25T09:03:05.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.270+0000 7fb3437fe640 1 --2- 172.21.15.43:0/2417583895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3440ce5e0 0x7fb3440cea90 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fb33c009e50 tx=0x7fb33c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=723a41985469bcff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:05.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.270+0000 7fb341ffb640 1 -- 172.21.15.43:0/2417583895 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c036070 con 0x7fb3440ce5e0 2024-04-25T09:03:05.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.270+0000 7fb341ffb640 1 -- 172.21.15.43:0/2417583895 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb33c031600 con 0x7fb3440ce5e0 2024-04-25T09:03:05.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.270+0000 7fb341ffb640 1 -- 172.21.15.43:0/2417583895 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33c0319d0 con 0x7fb3440ce5e0 2024-04-25T09:03:05.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.271+0000 7fb349553640 1 -- 172.21.15.43:0/2417583895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3440ce5e0 msgr2=0x7fb3440cea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:05.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.271+0000 7fb349553640 1 --2- 172.21.15.43:0/2417583895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3440ce5e0 0x7fb3440cea90 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fb33c009e50 tx=0x7fb33c02a6c0 comp rx=0 tx=0).stop 2024-04-25T09:03:05.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.271+0000 7fb349553640 1 -- 172.21.15.43:0/2417583895 shutdown_connections 2024-04-25T09:03:05.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.271+0000 7fb349553640 1 --2- 172.21.15.43:0/2417583895 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3440ce5e0 0x7fb3440cea90 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:05.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.271+0000 7fb349553640 1 --2- 172.21.15.43:0/2417583895 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb3440d09b0 0x7fb3440b6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:05.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.271+0000 7fb349553640 1 -- 172.21.15.43:0/2417583895 >> 172.21.15.43:0/2417583895 conn(0x7fb34409cd20 msgr2=0x7fb3440d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:05.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.271+0000 7fb349553640 1 -- 172.21.15.43:0/2417583895 shutdown_connections 2024-04-25T09:03:05.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.271+0000 7fb349553640 1 -- 172.21.15.43:0/2417583895 wait complete. 2024-04-25T09:03:05.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.272+0000 7fb349553640 1 Processor -- start 2024-04-25T09:03:05.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.272+0000 7fb349553640 1 -- start start 2024-04-25T09:03:05.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.273+0000 7fb349553640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb3440ce5e0 0x7fb34415f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:05.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.273+0000 7fb349553640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3440d09b0 0x7fb3441f31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:05.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.273+0000 7fb349553640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb344161390 0x7fb3441f3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:05.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.273+0000 7fb349553640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3440cefd0 con 0x7fb3440d09b0 2024-04-25T09:03:05.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.273+0000 7fb349553640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3440d15a0 con 0x7fb344161390 2024-04-25T09:03:05.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.273+0000 7fb349553640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb344161770 con 0x7fb3440ce5e0 2024-04-25T09:03:05.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.273+0000 7fb3427fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3440d09b0 0x7fb3441f31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:05.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.273+0000 7fb3427fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3440d09b0 0x7fb3441f31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44364/0 (socket says 172.21.15.43:44364) 2024-04-25T09:03:05.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.273+0000 7fb3427fc640 1 -- 172.21.15.43:0/867127568 learned_addr learned my addr 172.21.15.43:0/867127568 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:05.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.273+0000 7fb342ffd640 1 --2- 172.21.15.43:0/867127568 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb3440ce5e0 0x7fb34415f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:05.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.273+0000 7fb3427fc640 1 -- 172.21.15.43:0/867127568 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb3440ce5e0 msgr2=0x7fb34415f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:05.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.273+0000 7fb3437fe640 1 --2- 172.21.15.43:0/867127568 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb344161390 0x7fb3441f3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:05.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.273+0000 7fb3427fc640 1 --2- 172.21.15.43:0/867127568 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb3440ce5e0 0x7fb34415f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:05.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.273+0000 7fb3427fc640 1 -- 172.21.15.43:0/867127568 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb344161390 msgr2=0x7fb3441f3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:05.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.273+0000 7fb3427fc640 1 --2- 172.21.15.43:0/867127568 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb344161390 0x7fb3441f3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:05.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.273+0000 7fb3427fc640 1 -- 172.21.15.43:0/867127568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb34415dbb0 con 0x7fb3440d09b0 2024-04-25T09:03:05.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.273+0000 7fb342ffd640 1 --2- 172.21.15.43:0/867127568 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb3440ce5e0 0x7fb34415f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:05.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.273+0000 7fb3437fe640 1 --2- 172.21.15.43:0/867127568 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb344161390 0x7fb3441f3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:05.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.273+0000 7fb3427fc640 1 --2- 172.21.15.43:0/867127568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3440d09b0 0x7fb3441f31e0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fb33800d800 tx=0x7fb33800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:05.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.274+0000 7fb32b7fe640 1 -- 172.21.15.43:0/867127568 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33800b6c0 con 0x7fb3440d09b0 2024-04-25T09:03:05.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.274+0000 7fb32b7fe640 1 -- 172.21.15.43:0/867127568 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb338019070 con 0x7fb3440d09b0 2024-04-25T09:03:05.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.274+0000 7fb349553640 1 -- 172.21.15.43:0/867127568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb34415d8d0 con 0x7fb3440d09b0 2024-04-25T09:03:05.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.274+0000 7fb32b7fe640 1 -- 172.21.15.43:0/867127568 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33800ba80 con 0x7fb3440d09b0 2024-04-25T09:03:05.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.274+0000 7fb349553640 1 -- 172.21.15.43:0/867127568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3440cb6c0 con 0x7fb3440d09b0 2024-04-25T09:03:05.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.275+0000 7fb349553640 1 -- 172.21.15.43:0/867127568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3440680f0 con 0x7fb3440d09b0 2024-04-25T09:03:05.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.275+0000 7fb32b7fe640 1 -- 172.21.15.43:0/867127568 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb33800bc20 con 0x7fb3440d09b0 2024-04-25T09:03:05.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.275+0000 7fb32b7fe640 1 --2- 172.21.15.43:0/867127568 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb33006b4d0 0x7fb33006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:05.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.276+0000 7fb32b7fe640 1 -- 172.21.15.43:0/867127568 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb33808a470 con 0x7fb3440d09b0 2024-04-25T09:03:05.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.276+0000 7fb342ffd640 1 --2- 172.21.15.43:0/867127568 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb33006b4d0 0x7fb33006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:05.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.276+0000 7fb342ffd640 1 --2- 172.21.15.43:0/867127568 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb33006b4d0 0x7fb33006d990 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fb33400a830 tx=0x7fb334008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:05.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.280+0000 7fb32b7fe640 1 -- 172.21.15.43:0/867127568 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb33801c360 con 0x7fb3440d09b0 2024-04-25T09:03:05.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:05.532+0000 7fb349553640 1 -- 172.21.15.43:0/867127568 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]} v 0) v1 -- 0x7fb344074ef0 con 0x7fb3440d09b0 2024-04-25T09:03:06.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.519+0000 7fb32b7fe640 1 -- 172.21.15.43:0/867127568 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]}]=0 v431) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb338059560 con 0x7fb3440d09b0 2024-04-25T09:03:06.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.523+0000 7fb349553640 1 -- 172.21.15.43:0/867127568 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb33006b4d0 msgr2=0x7fb33006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:06.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.523+0000 7fb349553640 1 --2- 172.21.15.43:0/867127568 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb33006b4d0 0x7fb33006d990 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fb33400a830 tx=0x7fb334008040 comp rx=0 tx=0).stop 2024-04-25T09:03:06.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.523+0000 7fb349553640 1 -- 172.21.15.43:0/867127568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3440d09b0 msgr2=0x7fb3441f31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:06.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.523+0000 7fb349553640 1 --2- 172.21.15.43:0/867127568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3440d09b0 0x7fb3441f31e0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fb33800d800 tx=0x7fb33800dcc0 comp rx=0 tx=0).stop 2024-04-25T09:03:06.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.523+0000 7fb349553640 1 -- 172.21.15.43:0/867127568 shutdown_connections 2024-04-25T09:03:06.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.523+0000 7fb349553640 1 --2- 172.21.15.43:0/867127568 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb33006b4d0 0x7fb33006d990 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:06.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.523+0000 7fb349553640 1 --2- 172.21.15.43:0/867127568 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb344161390 0x7fb3441f3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:06.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.523+0000 7fb349553640 1 --2- 172.21.15.43:0/867127568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb3440d09b0 0x7fb3441f31e0 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:06.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.523+0000 7fb349553640 1 --2- 172.21.15.43:0/867127568 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb3440ce5e0 0x7fb34415f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:06.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.523+0000 7fb349553640 1 -- 172.21.15.43:0/867127568 >> 172.21.15.43:0/867127568 conn(0x7fb34409cd20 msgr2=0x7fb3440b8260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:06.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.523+0000 7fb349553640 1 -- 172.21.15.43:0/867127568 shutdown_connections 2024-04-25T09:03:06.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.523+0000 7fb349553640 1 -- 172.21.15.43:0/867127568 wait complete. 2024-04-25T09:03:06.526 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T09:03:06.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.647+0000 7f3654fd5640 1 Processor -- start 2024-04-25T09:03:06.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.647+0000 7f3654fd5640 1 -- start start 2024-04-25T09:03:06.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.648+0000 7f3654fd5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f365006d7d0 0x7f36500ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:06.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.648+0000 7f3654fd5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f36500cd120 0x7f365015d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:06.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.648+0000 7f3654fd5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36500cd540 con 0x7f36500aee80 2024-04-25T09:03:06.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.648+0000 7f3654fd5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36500af260 con 0x7f365006d7d0 2024-04-25T09:03:06.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.648+0000 7f3654fd5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f365015c280 con 0x7f36500cd120 2024-04-25T09:03:06.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.648+0000 7f364dd9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f365006d7d0 0x7f36500ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:06.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.648+0000 7f364e59c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f36500aee80 0x7f365006d0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:52970/0 (socket says 172.21.15.43:52970) 2024-04-25T09:03:06.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.648+0000 7f364e59c640 1 -- 172.21.15.43:0/647321954 learned_addr learned my addr 172.21.15.43:0/647321954 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:03:06.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.649+0000 7f364dd9b640 1 -- 172.21.15.43:0/647321954 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f36500cd120 msgr2=0x7f365015d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:06.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.649+0000 7f364dd9b640 1 --2- 172.21.15.43:0/647321954 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f36500cd120 0x7f365015d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:06.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.649+0000 7f364dd9b640 1 -- 172.21.15.43:0/647321954 >> v1:172.21.15.43:6789/0 conn(0x7f36500aee80 legacy=0x7f365006d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:06.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.649+0000 7f364dd9b640 1 -- 172.21.15.43:0/647321954 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f365015c560 con 0x7f365006d7d0 2024-04-25T09:03:06.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.649+0000 7f364d59a640 1 -- 172.21.15.43:0/647321954 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1261858059 0 0) 0x7f36500cd540 con 0x7f36500aee80 2024-04-25T09:03:06.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.649+0000 7f364dd9b640 1 --2- 172.21.15.43:0/647321954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f365006d7d0 0x7f36500ccbe0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f36380097a0 tx=0x7f363802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=251ba267de1891c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:06.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.649+0000 7f364d59a640 1 -- 172.21.15.43:0/647321954 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3638037070 con 0x7f365006d7d0 2024-04-25T09:03:06.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.649+0000 7f364d59a640 1 -- 172.21.15.43:0/647321954 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3638032440 con 0x7f365006d7d0 2024-04-25T09:03:06.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.649+0000 7f364d59a640 1 -- 172.21.15.43:0/647321954 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3638032810 con 0x7f365006d7d0 2024-04-25T09:03:06.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.650+0000 7f3654fd5640 1 -- 172.21.15.43:0/647321954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f365006d7d0 msgr2=0x7f36500ccbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:06.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.650+0000 7f3654fd5640 1 --2- 172.21.15.43:0/647321954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f365006d7d0 0x7f36500ccbe0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f36380097a0 tx=0x7f363802a180 comp rx=0 tx=0).stop 2024-04-25T09:03:06.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.650+0000 7f3654fd5640 1 -- 172.21.15.43:0/647321954 shutdown_connections 2024-04-25T09:03:06.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.650+0000 7f3654fd5640 1 --2- 172.21.15.43:0/647321954 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f36500cd120 0x7f365015d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:06.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.650+0000 7f3654fd5640 1 --2- 172.21.15.43:0/647321954 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f365006d7d0 0x7f36500ccbe0 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:06.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.650+0000 7f3654fd5640 1 -- 172.21.15.43:0/647321954 >> 172.21.15.43:0/647321954 conn(0x7f365006ab30 msgr2=0x7f365006af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:06.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.650+0000 7f3654fd5640 1 -- 172.21.15.43:0/647321954 shutdown_connections 2024-04-25T09:03:06.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.650+0000 7f3654fd5640 1 -- 172.21.15.43:0/647321954 wait complete. 2024-04-25T09:03:06.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.651+0000 7f3654fd5640 1 Processor -- start 2024-04-25T09:03:06.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.651+0000 7f3654fd5640 1 -- start start 2024-04-25T09:03:06.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.652+0000 7f3654fd5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f365006d7d0 0x7f3650147560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:06.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.652+0000 7f3654fd5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36500aee80 0x7f36500adb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:06.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.652+0000 7f3654fd5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f36500cd120 0x7f36501f6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:06.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.652+0000 7f3654fd5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365015dad0 con 0x7f365006d7d0 2024-04-25T09:03:06.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.652+0000 7f3654fd5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f365015cb20 con 0x7f36500aee80 2024-04-25T09:03:06.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.652+0000 7f3654fd5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f365015dc50 con 0x7f36500cd120 2024-04-25T09:03:06.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.652+0000 7f364e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f365006d7d0 0x7f3650147560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:06.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.652+0000 7f364e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f365006d7d0 0x7f3650147560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44382/0 (socket says 172.21.15.43:44382) 2024-04-25T09:03:06.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.652+0000 7f364e59c640 1 -- 172.21.15.43:0/225223445 learned_addr learned my addr 172.21.15.43:0/225223445 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:06.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.653+0000 7f364ed9d640 1 --2- 172.21.15.43:0/225223445 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f36500cd120 0x7f36501f6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:06.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.653+0000 7f364dd9b640 1 --2- 172.21.15.43:0/225223445 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36500aee80 0x7f36500adb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:06.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.653+0000 7f364e59c640 1 -- 172.21.15.43:0/225223445 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f36500cd120 msgr2=0x7f36501f6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:06.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.653+0000 7f364e59c640 1 --2- 172.21.15.43:0/225223445 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f36500cd120 0x7f36501f6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:06.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.653+0000 7f364e59c640 1 -- 172.21.15.43:0/225223445 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36500aee80 msgr2=0x7f36500adb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:06.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.653+0000 7f364e59c640 1 --2- 172.21.15.43:0/225223445 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36500aee80 0x7f36500adb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:06.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.653+0000 7f364e59c640 1 -- 172.21.15.43:0/225223445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36501f7b30 con 0x7f365006d7d0 2024-04-25T09:03:06.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.653+0000 7f364e59c640 1 --2- 172.21.15.43:0/225223445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f365006d7d0 0x7f3650147560 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f364800c4d0 tx=0x7f364800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:06.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.653+0000 7f3646ffd640 1 -- 172.21.15.43:0/225223445 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3648018070 con 0x7f365006d7d0 2024-04-25T09:03:06.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.653+0000 7f3646ffd640 1 -- 172.21.15.43:0/225223445 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3648007500 con 0x7f365006d7d0 2024-04-25T09:03:06.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.653+0000 7f3654fd5640 1 -- 172.21.15.43:0/225223445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36501f7e70 con 0x7f365006d7d0 2024-04-25T09:03:06.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.653+0000 7f3646ffd640 1 -- 172.21.15.43:0/225223445 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3648004a20 con 0x7f365006d7d0 2024-04-25T09:03:06.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.653+0000 7f3654fd5640 1 -- 172.21.15.43:0/225223445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3650147040 con 0x7f365006d7d0 2024-04-25T09:03:06.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.654+0000 7f3654fd5640 1 -- 172.21.15.43:0/225223445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3650151ce0 con 0x7f365006d7d0 2024-04-25T09:03:06.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.655+0000 7f3646ffd640 1 -- 172.21.15.43:0/225223445 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f364801b070 con 0x7f365006d7d0 2024-04-25T09:03:06.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.655+0000 7f3646ffd640 1 --2- 172.21.15.43:0/225223445 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f363006b4d0 0x7f363006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:06.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.655+0000 7f3646ffd640 1 -- 172.21.15.43:0/225223445 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3648089240 con 0x7f365006d7d0 2024-04-25T09:03:06.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.655+0000 7f364dd9b640 1 --2- 172.21.15.43:0/225223445 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f363006b4d0 0x7f363006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:06.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.656+0000 7f364dd9b640 1 --2- 172.21.15.43:0/225223445 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f363006b4d0 0x7f363006d990 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f3638034000 tx=0x7f363802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:06.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.659+0000 7f3646ffd640 1 -- 172.21.15.43:0/225223445 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f36501f7e70 con 0x7f365006d7d0 2024-04-25T09:03:06.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.839+0000 7f3654fd5640 1 -- 172.21.15.43:0/225223445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f36500725f0 con 0x7f365006d7d0 2024-04-25T09:03:06.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.842+0000 7f3646ffd640 1 -- 172.21.15.43:0/225223445 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f36480583c0 con 0x7f365006d7d0 2024-04-25T09:03:06.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.844+0000 7f3654fd5640 1 -- 172.21.15.43:0/225223445 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f363006b4d0 msgr2=0x7f363006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:06.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.844+0000 7f3654fd5640 1 --2- 172.21.15.43:0/225223445 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f363006b4d0 0x7f363006d990 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f3638034000 tx=0x7f363802e000 comp rx=0 tx=0).stop 2024-04-25T09:03:06.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.844+0000 7f3654fd5640 1 -- 172.21.15.43:0/225223445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f365006d7d0 msgr2=0x7f3650147560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:06.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.844+0000 7f3654fd5640 1 --2- 172.21.15.43:0/225223445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f365006d7d0 0x7f3650147560 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f364800c4d0 tx=0x7f364800c990 comp rx=0 tx=0).stop 2024-04-25T09:03:06.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.844+0000 7f3654fd5640 1 -- 172.21.15.43:0/225223445 shutdown_connections 2024-04-25T09:03:06.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.844+0000 7f3654fd5640 1 --2- 172.21.15.43:0/225223445 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f36500cd120 0x7f36501f6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:06.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.844+0000 7f3654fd5640 1 --2- 172.21.15.43:0/225223445 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f363006b4d0 0x7f363006d990 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:06.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.844+0000 7f3654fd5640 1 --2- 172.21.15.43:0/225223445 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f36500aee80 0x7f36500adb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:06.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.844+0000 7f3654fd5640 1 --2- 172.21.15.43:0/225223445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f365006d7d0 0x7f3650147560 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:06.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.844+0000 7f3654fd5640 1 -- 172.21.15.43:0/225223445 >> 172.21.15.43:0/225223445 conn(0x7f365006ab30 msgr2=0x7f36500ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:06.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.844+0000 7f3654fd5640 1 -- 172.21.15.43:0/225223445 shutdown_connections 2024-04-25T09:03:06.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:06.844+0000 7f3654fd5640 1 -- 172.21.15.43:0/225223445 wait complete. 2024-04-25T09:03:06.844 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:03:06.845 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:03:06.885 DEBUG:teuthology.orchestra.run.smithi193:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:03:06.922 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:03:06.922 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T09:03:06.922 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:03:06.941 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2374 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:03:06.941 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T09:03:06.941 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:03:06.979 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19508 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:03:06.979 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T09:03:06.979 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:03:06.996 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2374 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:03:06.996 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T09:03:06.996 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:03:07.036 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19508 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:03:07.036 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T09:03:07.036 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:07.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.148+0000 7fc6edc38640 1 Processor -- start 2024-04-25T09:03:07.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.148+0000 7fc6edc38640 1 -- start start 2024-04-25T09:03:07.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.149+0000 7fc6edc38640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc6e80c8d00 0x7fc6e8162100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:07.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.149+0000 7fc6edc38640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6e80d24a0 con 0x7fc6e80ce660 2024-04-25T09:03:07.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.149+0000 7fc6edc38640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6e80d2740 con 0x7fc6e815df50 2024-04-25T09:03:07.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.149+0000 7fc6edc38640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6e815e330 con 0x7fc6e80c8d00 2024-04-25T09:03:07.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.149+0000 7fc6e7fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc6e80ce660 0x7fc6e80cea70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:52972/0 (socket says 172.21.15.43:52972) 2024-04-25T09:03:07.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.149+0000 7fc6e7fff640 1 -- 172.21.15.43:0/3075184353 learned_addr learned my addr 172.21.15.43:0/3075184353 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:03:07.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.149+0000 7fc6e6ffd640 1 --2- 172.21.15.43:0/3075184353 >> v2:172.21.15.43:3300/0 conn(0x7fc6e80c8d00 0x7fc6e8162100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:07.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.150+0000 7fc6e67fc640 1 -- 172.21.15.43:0/3075184353 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2196543107 0 0) 0x7fc6e80d24a0 con 0x7fc6e80ce660 2024-04-25T09:03:07.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.150+0000 7fc6e67fc640 1 -- 172.21.15.43:0/3075184353 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6d0003520 con 0x7fc6e80ce660 2024-04-25T09:03:07.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.150+0000 7fc6e6ffd640 1 -- 172.21.15.43:0/3075184353 >> v1:172.21.15.193:6789/0 conn(0x7fc6e815df50 legacy=0x7fc6e81619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:07.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.150+0000 7fc6e6ffd640 1 -- 172.21.15.43:0/3075184353 >> v1:172.21.15.43:6789/0 conn(0x7fc6e80ce660 legacy=0x7fc6e80cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:07.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.150+0000 7fc6e6ffd640 1 -- 172.21.15.43:0/3075184353 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e80d2a20 con 0x7fc6e80c8d00 2024-04-25T09:03:07.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.150+0000 7fc6e6ffd640 1 --2- 172.21.15.43:0/3075184353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6e80c8d00 0x7fc6e8162100 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc0097a0 tx=0x7fc6dc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d127e4b2df931bc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:07.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.151+0000 7fc6e67fc640 1 -- 172.21.15.43:0/3075184353 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc037070 con 0x7fc6e80c8d00 2024-04-25T09:03:07.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.151+0000 7fc6e67fc640 1 -- 172.21.15.43:0/3075184353 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6dc032440 con 0x7fc6e80c8d00 2024-04-25T09:03:07.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.151+0000 7fc6e67fc640 1 -- 172.21.15.43:0/3075184353 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc0327f0 con 0x7fc6e80c8d00 2024-04-25T09:03:07.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.151+0000 7fc6edc38640 1 -- 172.21.15.43:0/3075184353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6e80c8d00 msgr2=0x7fc6e8162100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:07.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.151+0000 7fc6edc38640 1 --2- 172.21.15.43:0/3075184353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6e80c8d00 0x7fc6e8162100 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc0097a0 tx=0x7fc6dc02a230 comp rx=0 tx=0).stop 2024-04-25T09:03:07.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.151+0000 7fc6edc38640 1 -- 172.21.15.43:0/3075184353 shutdown_connections 2024-04-25T09:03:07.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.151+0000 7fc6edc38640 1 --2- 172.21.15.43:0/3075184353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6e80c8d00 0x7fc6e8162100 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:07.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.151+0000 7fc6edc38640 1 -- 172.21.15.43:0/3075184353 >> 172.21.15.43:0/3075184353 conn(0x7fc6e80d3a30 msgr2=0x7fc6e80d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:07.141 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.151+0000 7fc6edc38640 1 -- 172.21.15.43:0/3075184353 shutdown_connections 2024-04-25T09:03:07.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.151+0000 7fc6edc38640 1 -- 172.21.15.43:0/3075184353 wait complete. 2024-04-25T09:03:07.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.152+0000 7fc6edc38640 1 Processor -- start 2024-04-25T09:03:07.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.153+0000 7fc6edc38640 1 -- start start 2024-04-25T09:03:07.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.153+0000 7fc6edc38640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc6e80c8d00 0x7fc6e81538f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:07.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.153+0000 7fc6edc38640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6e80ce660 0x7fc6e8160a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:07.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.153+0000 7fc6edc38640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6e815df50 0x7fc6e80cc710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:07.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.153+0000 7fc6edc38640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e80d2190 con 0x7fc6e80ce660 2024-04-25T09:03:07.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.153+0000 7fc6edc38640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e80c77d0 con 0x7fc6e815df50 2024-04-25T09:03:07.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.153+0000 7fc6edc38640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6e8162640 con 0x7fc6e80c8d00 2024-04-25T09:03:07.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.153+0000 7fc6e6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6e80ce660 0x7fc6e8160a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:07.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.153+0000 7fc6e6ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6e80ce660 0x7fc6e8160a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44392/0 (socket says 172.21.15.43:44392) 2024-04-25T09:03:07.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.153+0000 7fc6e6ffd640 1 -- 172.21.15.43:0/4194608139 learned_addr learned my addr 172.21.15.43:0/4194608139 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:07.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.154+0000 7fc6e77fe640 1 --2- 172.21.15.43:0/4194608139 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc6e80c8d00 0x7fc6e81538f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:07.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.154+0000 7fc6e7fff640 1 --2- 172.21.15.43:0/4194608139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6e815df50 0x7fc6e80cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:07.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.154+0000 7fc6e6ffd640 1 -- 172.21.15.43:0/4194608139 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc6e80c8d00 msgr2=0x7fc6e81538f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:07.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.154+0000 7fc6e6ffd640 1 --2- 172.21.15.43:0/4194608139 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc6e80c8d00 0x7fc6e81538f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:07.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.154+0000 7fc6e6ffd640 1 -- 172.21.15.43:0/4194608139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6e815df50 msgr2=0x7fc6e80cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:07.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.154+0000 7fc6e6ffd640 1 --2- 172.21.15.43:0/4194608139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6e815df50 0x7fc6e80cc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:07.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.154+0000 7fc6e6ffd640 1 -- 172.21.15.43:0/4194608139 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e80cb980 con 0x7fc6e80ce660 2024-04-25T09:03:07.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.154+0000 7fc6e6ffd640 1 --2- 172.21.15.43:0/4194608139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6e80ce660 0x7fc6e8160a10 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc035040 tx=0x7fc6dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:07.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.154+0000 7fc6cffff640 1 -- 172.21.15.43:0/4194608139 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc037050 con 0x7fc6e80ce660 2024-04-25T09:03:07.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.154+0000 7fc6cffff640 1 -- 172.21.15.43:0/4194608139 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6dc038040 con 0x7fc6e80ce660 2024-04-25T09:03:07.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.154+0000 7fc6edc38640 1 -- 172.21.15.43:0/4194608139 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6e815e700 con 0x7fc6e80ce660 2024-04-25T09:03:07.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.154+0000 7fc6cffff640 1 -- 172.21.15.43:0/4194608139 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6dc004470 con 0x7fc6e80ce660 2024-04-25T09:03:07.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.154+0000 7fc6edc38640 1 -- 172.21.15.43:0/4194608139 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6e815ecc0 con 0x7fc6e80ce660 2024-04-25T09:03:07.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.155+0000 7fc6edc38640 1 -- 172.21.15.43:0/4194608139 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6e80680f0 con 0x7fc6e80ce660 2024-04-25T09:03:07.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.156+0000 7fc6cffff640 1 -- 172.21.15.43:0/4194608139 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc6dc03c6d0 con 0x7fc6e80ce660 2024-04-25T09:03:07.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.156+0000 7fc6cffff640 1 --2- 172.21.15.43:0/4194608139 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc6d406b4d0 0x7fc6d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:07.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.156+0000 7fc6cffff640 1 -- 172.21.15.43:0/4194608139 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc6dc0aa760 con 0x7fc6e80ce660 2024-04-25T09:03:07.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.156+0000 7fc6e77fe640 1 --2- 172.21.15.43:0/4194608139 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc6d406b4d0 0x7fc6d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:07.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.157+0000 7fc6e77fe640 1 --2- 172.21.15.43:0/4194608139 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc6d406b4d0 0x7fc6d406d990 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fc6e80cbf80 tx=0x7fc6d8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:07.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.160+0000 7fc6cffff640 1 -- 172.21.15.43:0/4194608139 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc6dc079320 con 0x7fc6e80ce660 2024-04-25T09:03:07.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.375+0000 7fc6edc38640 1 -- 172.21.15.43:0/4194608139 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6e8074ef0 con 0x7fc6e80ce660 2024-04-25T09:03:07.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.376+0000 7fc6cffff640 1 -- 172.21.15.43:0/4194608139 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+3702 (secure 0 0 0) 0x7fc6dc07d3c0 con 0x7fc6e80ce660 2024-04-25T09:03:07.365 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:03:07.365 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":71,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6821,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2021228703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2021228703},{"type":"v1","addr":"172.21.15.193:6837","nonce":2021228703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65},{"gid":7543,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/2398324998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2398324998},{"type":"v1","addr":"172.21.15.193:6836","nonce":2398324998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":65},{"gid":8673,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6834/2298187831","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2298187831},{"type":"v1","addr":"172.21.15.43:6834","nonce":2298187831}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":64},{"gid":9159,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/745813476","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":745813476},{"type":"v1","addr":"172.21.15.43:6835","nonce":745813476}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":71}],"filesystems":[]} 2024-04-25T09:03:07.365 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 71 2024-04-25T09:03:07.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.378+0000 7fc6edc38640 1 -- 172.21.15.43:0/4194608139 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc6d406b4d0 msgr2=0x7fc6d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:07.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.378+0000 7fc6edc38640 1 --2- 172.21.15.43:0/4194608139 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc6d406b4d0 0x7fc6d406d990 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fc6e80cbf80 tx=0x7fc6d8009040 comp rx=0 tx=0).stop 2024-04-25T09:03:07.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.378+0000 7fc6edc38640 1 -- 172.21.15.43:0/4194608139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6e80ce660 msgr2=0x7fc6e8160a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:07.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.378+0000 7fc6edc38640 1 --2- 172.21.15.43:0/4194608139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6e80ce660 0x7fc6e8160a10 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7fc6dc035040 tx=0x7fc6dc0046c0 comp rx=0 tx=0).stop 2024-04-25T09:03:07.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.378+0000 7fc6edc38640 1 -- 172.21.15.43:0/4194608139 shutdown_connections 2024-04-25T09:03:07.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.378+0000 7fc6edc38640 1 --2- 172.21.15.43:0/4194608139 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc6d406b4d0 0x7fc6d406d990 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:07.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.378+0000 7fc6edc38640 1 --2- 172.21.15.43:0/4194608139 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc6e815df50 0x7fc6e80cc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:07.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.378+0000 7fc6edc38640 1 --2- 172.21.15.43:0/4194608139 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc6e80ce660 0x7fc6e8160a10 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:07.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.378+0000 7fc6edc38640 1 --2- 172.21.15.43:0/4194608139 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc6e80c8d00 0x7fc6e81538f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:07.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.378+0000 7fc6edc38640 1 -- 172.21.15.43:0/4194608139 >> 172.21.15.43:0/4194608139 conn(0x7fc6e80d3a30 msgr2=0x7fc6e80cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:07.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.378+0000 7fc6edc38640 1 -- 172.21.15.43:0/4194608139 shutdown_connections 2024-04-25T09:03:07.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.378+0000 7fc6edc38640 1 -- 172.21.15.43:0/4194608139 wait complete. 2024-04-25T09:03:07.378 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:03:07.378 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T09:03:07.379 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:03:07.379 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:03:07.379 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:03:07.379 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T09:03:07.379 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:03:07.379 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:03:07.379 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:03:07.380 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T09:03:07.380 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:03:07.380 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:03:07.380 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:03:07.380 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T09:03:07.380 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:03:07.380 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:03:07.388 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:03:07.388 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:03:07.389 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:03:07.389 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:03:07.392 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:03:07.392 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:03:07.393 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:03:07.393 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:03:07.431 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:03:07.431 DEBUG:teuthology.parallel:result is None 2024-04-25T09:03:07.431 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:03:07.432 DEBUG:teuthology.parallel:result is None 2024-04-25T09:03:07.435 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:03:07.435 DEBUG:teuthology.parallel:result is None 2024-04-25T09:03:07.435 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:03:07.435 DEBUG:teuthology.parallel:result is None 2024-04-25T09:03:07.436 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T09:03:07.506 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2024-04-25T09:03:07.506 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-04-25T09:03:07.515 INFO:tasks.ceph.mds.b.smithi193.stdout:starting mds.b at 2024-04-25T09:03:07.515 INFO:tasks.ceph.mds.d.smithi193.stdout:starting mds.d at 2024-04-25T09:03:07.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.554+0000 7f8384c2a640 1 Processor -- start 2024-04-25T09:03:07.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.554+0000 7f8384c2a640 1 -- start start 2024-04-25T09:03:07.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.555+0000 7f8384c2a640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f83800d0d90 0x7f83800ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:07.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.555+0000 7f8384c2a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f83800d2500 0x7f83800d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:07.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.555+0000 7f8384c2a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83800b6c50 con 0x7f83800d39d0 2024-04-25T09:03:07.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.555+0000 7f8384c2a640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f838015e010 con 0x7f83800d0d90 2024-04-25T09:03:07.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.555+0000 7f8384c2a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83800d2020 con 0x7f83800d2500 2024-04-25T09:03:07.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.555+0000 7f837e59c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f83800d39d0 0x7f83800d3db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:52986/0 (socket says 172.21.15.43:52986) 2024-04-25T09:03:07.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.555+0000 7f837e59c640 1 -- 172.21.15.43:0/3428379454 learned_addr learned my addr 172.21.15.43:0/3428379454 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:03:07.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.555+0000 7f837ed9d640 1 --2- 172.21.15.43:0/3428379454 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f83800d2500 0x7f83800d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:07.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.555+0000 7f837dd9b640 1 --2- 172.21.15.43:0/3428379454 >> v2:172.21.15.193:3300/0 conn(0x7f83800d0d90 0x7f83800ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:07.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.555+0000 7f837d59a640 1 -- 172.21.15.43:0/3428379454 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3346053126 0 0) 0x7f8370001110 con 0x7f83800d39d0 2024-04-25T09:03:07.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.555+0000 7f837d59a640 1 -- 172.21.15.43:0/3428379454 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8368003520 con 0x7f83800d39d0 2024-04-25T09:03:07.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.555+0000 7f837ed9d640 1 -- 172.21.15.43:0/3428379454 >> v2:172.21.15.193:3300/0 conn(0x7f83800d0d90 msgr2=0x7f83800ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:07.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.555+0000 7f837ed9d640 1 --2- 172.21.15.43:0/3428379454 >> v2:172.21.15.193:3300/0 conn(0x7f83800d0d90 0x7f83800ce2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:07.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.555+0000 7f837ed9d640 1 -- 172.21.15.43:0/3428379454 >> v1:172.21.15.43:6789/0 conn(0x7f83800d39d0 legacy=0x7f83800d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:07.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.555+0000 7f837ed9d640 1 -- 172.21.15.43:0/3428379454 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83800cec40 con 0x7f83800d2500 2024-04-25T09:03:07.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.555+0000 7f837dd9b640 1 --2- 172.21.15.43:0/3428379454 >> v2:172.21.15.193:3300/0 conn(0x7f83800d0d90 0x7f83800ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:03:07.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.556+0000 7f837ed9d640 1 --2- 172.21.15.43:0/3428379454 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f83800d2500 0x7f83800d2990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8378009d90 tx=0x7f837802a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c7d9fe1a5214c9e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:07.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.556+0000 7f837d59a640 1 -- 172.21.15.43:0/3428379454 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8378031600 con 0x7f83800d2500 2024-04-25T09:03:07.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.556+0000 7f837d59a640 1 -- 172.21.15.43:0/3428379454 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83780317a0 con 0x7f83800d2500 2024-04-25T09:03:07.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.556+0000 7f837d59a640 1 -- 172.21.15.43:0/3428379454 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8378031b70 con 0x7f83800d2500 2024-04-25T09:03:07.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.557+0000 7f8384c2a640 1 -- 172.21.15.43:0/3428379454 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f83800d2500 msgr2=0x7f83800d2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:07.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.557+0000 7f8384c2a640 1 --2- 172.21.15.43:0/3428379454 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f83800d2500 0x7f83800d2990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8378009d90 tx=0x7f837802a5d0 comp rx=0 tx=0).stop 2024-04-25T09:03:07.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.557+0000 7f8384c2a640 1 -- 172.21.15.43:0/3428379454 shutdown_connections 2024-04-25T09:03:07.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.557+0000 7f8384c2a640 1 --2- 172.21.15.43:0/3428379454 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f83800d2500 0x7f83800d2990 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:07.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.557+0000 7f8384c2a640 1 --2- 172.21.15.43:0/3428379454 >> v2:172.21.15.193:3300/0 conn(0x7f83800d0d90 0x7f83800ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:07.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.557+0000 7f8384c2a640 1 -- 172.21.15.43:0/3428379454 >> 172.21.15.43:0/3428379454 conn(0x7f838006b000 msgr2=0x7f838006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:07.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.557+0000 7f8384c2a640 1 -- 172.21.15.43:0/3428379454 shutdown_connections 2024-04-25T09:03:07.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.557+0000 7f8384c2a640 1 -- 172.21.15.43:0/3428379454 wait complete. 2024-04-25T09:03:07.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.558+0000 7f8384c2a640 1 Processor -- start 2024-04-25T09:03:07.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.558+0000 7f8384c2a640 1 -- start start 2024-04-25T09:03:07.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.559+0000 7f8384c2a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f83800d0d90 0x7f83801e7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:07.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.559+0000 7f8384c2a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83800d2500 0x7f83800b3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:07.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.559+0000 7f8384c2a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83800d39d0 0x7f83800c9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:07.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.559+0000 7f8384c2a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83800cf200 con 0x7f83800d39d0 2024-04-25T09:03:07.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.559+0000 7f8384c2a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83800b2f00 con 0x7f83800d2500 2024-04-25T09:03:07.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.559+0000 7f8384c2a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83800d2ed0 con 0x7f83800d0d90 2024-04-25T09:03:07.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.559+0000 7f837ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83800d39d0 0x7f83800c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:07.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.559+0000 7f837dd9b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83800d2500 0x7f83800b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:07.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.559+0000 7f837e59c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f83800d0d90 0x7f83801e7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:07.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.559+0000 7f837ed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83800d39d0 0x7f83800c9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44420/0 (socket says 172.21.15.43:44420) 2024-04-25T09:03:07.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.559+0000 7f837dd9b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83800d2500 0x7f83800b3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:48002/0 (socket says 172.21.15.43:48002) 2024-04-25T09:03:07.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.559+0000 7f837ed9d640 1 -- 172.21.15.43:0/3462650447 learned_addr learned my addr 172.21.15.43:0/3462650447 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:07.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.560+0000 7f837e59c640 1 -- 172.21.15.43:0/3462650447 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83800d2500 msgr2=0x7f83800b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:07.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.560+0000 7f837e59c640 1 --2- 172.21.15.43:0/3462650447 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83800d2500 0x7f83800b3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:07.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.560+0000 7f837e59c640 1 -- 172.21.15.43:0/3462650447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83800d39d0 msgr2=0x7f83800c9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:07.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.560+0000 7f837e59c640 1 --2- 172.21.15.43:0/3462650447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83800d39d0 0x7f83800c9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:07.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.560+0000 7f837e59c640 1 -- 172.21.15.43:0/3462650447 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83800c8260 con 0x7f83800d0d90 2024-04-25T09:03:07.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.560+0000 7f837ed9d640 1 --2- 172.21.15.43:0/3462650447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83800d39d0 0x7f83800c9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:03:07.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.560+0000 7f837dd9b640 1 --2- 172.21.15.43:0/3462650447 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83800d2500 0x7f83800b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:03:07.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.560+0000 7f837e59c640 1 --2- 172.21.15.43:0/3462650447 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f83800d0d90 0x7f83801e7e20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f837000c3c0 tx=0x7f837000c7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:07.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.560+0000 7f836effd640 1 -- 172.21.15.43:0/3462650447 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8370017070 con 0x7f83800d0d90 2024-04-25T09:03:07.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.560+0000 7f836effd640 1 -- 172.21.15.43:0/3462650447 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8370005ce0 con 0x7f83800d0d90 2024-04-25T09:03:07.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.560+0000 7f836effd640 1 -- 172.21.15.43:0/3462650447 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83700072f0 con 0x7f83800d0d90 2024-04-25T09:03:07.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.560+0000 7f8384c2a640 1 -- 172.21.15.43:0/3462650447 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83800c8540 con 0x7f83800d0d90 2024-04-25T09:03:07.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.560+0000 7f8384c2a640 1 -- 172.21.15.43:0/3462650447 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83800c8b00 con 0x7f83800d0d90 2024-04-25T09:03:07.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.561+0000 7f8384c2a640 1 -- 172.21.15.43:0/3462650447 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8380074ef0 con 0x7f83800d0d90 2024-04-25T09:03:07.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.562+0000 7f836effd640 1 -- 172.21.15.43:0/3462650447 <== mon.2 v2:172.21.15.193:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f837001a070 con 0x7f83800d0d90 2024-04-25T09:03:07.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.562+0000 7f836effd640 1 --2- 172.21.15.43:0/3462650447 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f836006b4d0 0x7f836006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:07.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.562+0000 7f836effd640 1 -- 172.21.15.43:0/3462650447 <== mon.2 v2:172.21.15.193:3301/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8370089290 con 0x7f83800d0d90 2024-04-25T09:03:07.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.562+0000 7f837dd9b640 1 --2- 172.21.15.43:0/3462650447 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f836006b4d0 0x7f836006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:07.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.563+0000 7f837dd9b640 1 --2- 172.21.15.43:0/3462650447 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f836006b4d0 0x7f836006d990 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f83800b25b0 tx=0x7f8374009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:07.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.567+0000 7f836effd640 1 -- 172.21.15.43:0/3462650447 <== mon.2 v2:172.21.15.193:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f83800b6c50 con 0x7f83800d0d90 2024-04-25T09:03:07.595 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-04-25T09:03:07.596 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-04-25T09:03:07.596 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-04-25T09:03:07.596 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-04-25T09:03:07.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:07.760+0000 7f8384c2a640 1 -- 172.21.15.43:0/3462650447 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f83800d17c0 con 0x7f83800d0d90 2024-04-25T09:03:08.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.536+0000 7f836effd640 1 -- 172.21.15.43:0/3462650447 <== mon.2 v2:172.21.15.193:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v120) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8370057da0 con 0x7f83800d0d90 2024-04-25T09:03:08.525 INFO:teuthology.orchestra.run.smithi043.stderr: removed all blocklist entries 2024-04-25T09:03:08.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.538+0000 7f8384c2a640 1 -- 172.21.15.43:0/3462650447 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f836006b4d0 msgr2=0x7f836006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:08.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.538+0000 7f8384c2a640 1 --2- 172.21.15.43:0/3462650447 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f836006b4d0 0x7f836006d990 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f83800b25b0 tx=0x7f8374009190 comp rx=0 tx=0).stop 2024-04-25T09:03:08.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.538+0000 7f8384c2a640 1 -- 172.21.15.43:0/3462650447 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f83800d0d90 msgr2=0x7f83801e7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:08.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.538+0000 7f8384c2a640 1 --2- 172.21.15.43:0/3462650447 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f83800d0d90 0x7f83801e7e20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f837000c3c0 tx=0x7f837000c7c0 comp rx=0 tx=0).stop 2024-04-25T09:03:08.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.539+0000 7f8384c2a640 1 -- 172.21.15.43:0/3462650447 shutdown_connections 2024-04-25T09:03:08.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.539+0000 7f8384c2a640 1 --2- 172.21.15.43:0/3462650447 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f83800d39d0 0x7f83800c9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:08.528 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.539+0000 7f8384c2a640 1 --2- 172.21.15.43:0/3462650447 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f836006b4d0 0x7f836006d990 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:08.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.539+0000 7f8384c2a640 1 --2- 172.21.15.43:0/3462650447 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f83800d2500 0x7f83800b3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:08.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.539+0000 7f8384c2a640 1 --2- 172.21.15.43:0/3462650447 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f83800d0d90 0x7f83801e7e20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:08.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.539+0000 7f8384c2a640 1 -- 172.21.15.43:0/3462650447 >> 172.21.15.43:0/3462650447 conn(0x7f838006b000 msgr2=0x7f83800cdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:08.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.539+0000 7f8384c2a640 1 -- 172.21.15.43:0/3462650447 shutdown_connections 2024-04-25T09:03:08.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.539+0000 7f8384c2a640 1 -- 172.21.15.43:0/3462650447 wait complete. 2024-04-25T09:03:08.542 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T09:03:08.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.659+0000 7f6d71de7640 1 Processor -- start 2024-04-25T09:03:08.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.659+0000 7f6d71de7640 1 -- start start 2024-04-25T09:03:08.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.660+0000 7f6d71de7640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6d6c161390 0x7f6d6c0b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:08.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.660+0000 7f6d71de7640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6d6c0cecf0 0x7f6d6c0d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:08.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.660+0000 7f6d71de7640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d6c0b4c30 con 0x7f6d6c0d02a0 2024-04-25T09:03:08.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.660+0000 7f6d71de7640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d6c0d1910 con 0x7f6d6c0cecf0 2024-04-25T09:03:08.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.660+0000 7f6d71de7640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c0cf2c0 con 0x7f6d6c161390 2024-04-25T09:03:08.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.661+0000 7f6d6b7fe640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6d6c161390 0x7f6d6c0b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:08.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.661+0000 7f6d6bfff640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6d6c0cecf0 0x7f6d6c0d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:08.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.661+0000 7f6d6b7fe640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6d6c161390 0x7f6d6c0b6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:55496/0 (socket says 172.21.15.43:55496) 2024-04-25T09:03:08.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.661+0000 7f6d6b7fe640 1 -- 172.21.15.43:0/1892631831 learned_addr learned my addr 172.21.15.43:0/1892631831 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:08.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.661+0000 7f6d6bfff640 1 -- 172.21.15.43:0/1892631831 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6d6c161390 msgr2=0x7f6d6c0b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:08.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.661+0000 7f6d6bfff640 1 --2- 172.21.15.43:0/1892631831 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6d6c161390 0x7f6d6c0b6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:08.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.661+0000 7f6d6bfff640 1 -- 172.21.15.43:0/1892631831 >> v1:172.21.15.193:6789/0 conn(0x7f6d6c0d02a0 legacy=0x7f6d6c0ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:08.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.661+0000 7f6d6bfff640 1 -- 172.21.15.43:0/1892631831 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d6c0cf5a0 con 0x7f6d6c0cecf0 2024-04-25T09:03:08.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.661+0000 7f6d6a7fc640 1 -- 172.21.15.43:0/1892631831 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2653816069 0 0) 0x7f6d60001110 con 0x7f6d6c0d02a0 2024-04-25T09:03:08.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.662+0000 7f6d6bfff640 1 --2- 172.21.15.43:0/1892631831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d6c0cecf0 0x7f6d6c0d0720 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6d64009d90 tx=0x7f6d6402a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ebdf88419170b95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:08.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.662+0000 7f6d6a7fc640 1 -- 172.21.15.43:0/1892631831 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d64007740 con 0x7f6d6c0cecf0 2024-04-25T09:03:08.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.662+0000 7f6d6a7fc640 1 -- 172.21.15.43:0/1892631831 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d640078e0 con 0x7f6d6c0cecf0 2024-04-25T09:03:08.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.662+0000 7f6d6a7fc640 1 -- 172.21.15.43:0/1892631831 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d64007cb0 con 0x7f6d6c0cecf0 2024-04-25T09:03:08.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.662+0000 7f6d71de7640 1 -- 172.21.15.43:0/1892631831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d6c0cecf0 msgr2=0x7f6d6c0d0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:08.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.662+0000 7f6d71de7640 1 --2- 172.21.15.43:0/1892631831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d6c0cecf0 0x7f6d6c0d0720 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f6d64009d90 tx=0x7f6d6402a660 comp rx=0 tx=0).stop 2024-04-25T09:03:08.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.663+0000 7f6d71de7640 1 -- 172.21.15.43:0/1892631831 shutdown_connections 2024-04-25T09:03:08.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.663+0000 7f6d71de7640 1 --2- 172.21.15.43:0/1892631831 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d6c0cecf0 0x7f6d6c0d0720 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:08.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.663+0000 7f6d71de7640 1 --2- 172.21.15.43:0/1892631831 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6d6c161390 0x7f6d6c0b6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:08.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.663+0000 7f6d71de7640 1 -- 172.21.15.43:0/1892631831 >> 172.21.15.43:0/1892631831 conn(0x7f6d6c09cd20 msgr2=0x7f6d6c0d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:08.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.663+0000 7f6d71de7640 1 -- 172.21.15.43:0/1892631831 shutdown_connections 2024-04-25T09:03:08.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.663+0000 7f6d71de7640 1 -- 172.21.15.43:0/1892631831 wait complete. 2024-04-25T09:03:08.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.664+0000 7f6d71de7640 1 Processor -- start 2024-04-25T09:03:08.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.664+0000 7f6d71de7640 1 -- start start 2024-04-25T09:03:08.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.665+0000 7f6d71de7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d6c0cecf0 0x7f6d6c15cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:08.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.665+0000 7f6d71de7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d6c0d02a0 0x7f6d6c151a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:08.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.665+0000 7f6d71de7640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6d6c161390 0x7f6d6c151f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:08.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.665+0000 7f6d71de7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c161770 con 0x7f6d6c0cecf0 2024-04-25T09:03:08.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.665+0000 7f6d71de7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c0d15a0 con 0x7f6d6c0d02a0 2024-04-25T09:03:08.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.665+0000 7f6d71de7640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c0d1da0 con 0x7f6d6c161390 2024-04-25T09:03:08.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.665+0000 7f6d6b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d6c0cecf0 0x7f6d6c15cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:08.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.665+0000 7f6d6b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d6c0cecf0 0x7f6d6c15cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44424/0 (socket says 172.21.15.43:44424) 2024-04-25T09:03:08.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.665+0000 7f6d6b7fe640 1 -- 172.21.15.43:0/4039107868 learned_addr learned my addr 172.21.15.43:0/4039107868 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:08.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.665+0000 7f6d6affd640 1 --2- 172.21.15.43:0/4039107868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d6c0d02a0 0x7f6d6c151a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:08.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.665+0000 7f6d6bfff640 1 --2- 172.21.15.43:0/4039107868 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6d6c161390 0x7f6d6c151f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:08.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.665+0000 7f6d6b7fe640 1 -- 172.21.15.43:0/4039107868 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6d6c161390 msgr2=0x7f6d6c151f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:08.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.665+0000 7f6d6b7fe640 1 --2- 172.21.15.43:0/4039107868 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6d6c161390 0x7f6d6c151f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:08.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.665+0000 7f6d6b7fe640 1 -- 172.21.15.43:0/4039107868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d6c0d02a0 msgr2=0x7f6d6c151a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:08.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.665+0000 7f6d6b7fe640 1 --2- 172.21.15.43:0/4039107868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d6c0d02a0 0x7f6d6c151a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:08.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.665+0000 7f6d6b7fe640 1 -- 172.21.15.43:0/4039107868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d6c15c370 con 0x7f6d6c0cecf0 2024-04-25T09:03:08.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.665+0000 7f6d6b7fe640 1 --2- 172.21.15.43:0/4039107868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d6c0cecf0 0x7f6d6c15cf90 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c00e870 tx=0x7f6d5c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:08.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.666+0000 7f6d53fff640 1 -- 172.21.15.43:0/4039107868 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c017070 con 0x7f6d6c0cecf0 2024-04-25T09:03:08.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.666+0000 7f6d53fff640 1 -- 172.21.15.43:0/4039107868 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d5c00c940 con 0x7f6d6c0cecf0 2024-04-25T09:03:08.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.666+0000 7f6d71de7640 1 -- 172.21.15.43:0/4039107868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d6c15c650 con 0x7f6d6c0cecf0 2024-04-25T09:03:08.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.666+0000 7f6d53fff640 1 -- 172.21.15.43:0/4039107868 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c00cd30 con 0x7f6d6c0cecf0 2024-04-25T09:03:08.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.666+0000 7f6d71de7640 1 -- 172.21.15.43:0/4039107868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d6c15d4d0 con 0x7f6d6c0cecf0 2024-04-25T09:03:08.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.667+0000 7f6d71de7640 1 -- 172.21.15.43:0/4039107868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d6c0680f0 con 0x7f6d6c0cecf0 2024-04-25T09:03:08.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.667+0000 7f6d53fff640 1 -- 172.21.15.43:0/4039107868 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6d5c01a070 con 0x7f6d6c0cecf0 2024-04-25T09:03:08.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.668+0000 7f6d53fff640 1 --2- 172.21.15.43:0/4039107868 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6d5806b4d0 0x7f6d5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:08.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.668+0000 7f6d53fff640 1 -- 172.21.15.43:0/4039107868 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6d5c08a080 con 0x7f6d6c0cecf0 2024-04-25T09:03:08.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.668+0000 7f6d6affd640 1 --2- 172.21.15.43:0/4039107868 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6d5806b4d0 0x7f6d5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:08.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.669+0000 7f6d6affd640 1 --2- 172.21.15.43:0/4039107868 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6d5806b4d0 0x7f6d5806d990 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c0cecc0 tx=0x7f6d60009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:08.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.672+0000 7f6d53fff640 1 -- 172.21.15.43:0/4039107868 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6d5c058bf0 con 0x7f6d6c0cecf0 2024-04-25T09:03:08.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.922+0000 7f6d71de7640 1 -- 172.21.15.43:0/4039107868 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f6d6c074ef0 con 0x7f6d6c0cecf0 2024-04-25T09:03:08.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.923+0000 7f6d53fff640 1 -- 172.21.15.43:0/4039107868 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f6d5c05cc90 con 0x7f6d6c0cecf0 2024-04-25T09:03:08.912 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:03:08.912 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:03:08.912 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_dump": [ 2024-04-25T09:03:08.912 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:03:08.912 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.a", 2024-04-25T09:03:08.912 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpmIjzEBRAAs2L10XrFO3mThjZevna5wA==", 2024-04-25T09:03:08.912 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:03:08.912 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T09:03:08.912 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T09:03:08.912 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T09:03:08.912 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:03:08.912 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:03:08.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:03:08.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:03:08.913 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.b", 2024-04-25T09:03:08.913 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm51cmERAAij4vy8xyihh8u/4dkmcllw==", 2024-04-25T09:03:08.913 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:03:08.913 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T09:03:08.913 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T09:03:08.913 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T09:03:08.913 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:03:08.913 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:03:08.913 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:03:08.913 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:03:08.913 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.c", 2024-04-25T09:03:08.913 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm1OOtDBAAlUYbCh1mBEEka9CWRnUz9A==", 2024-04-25T09:03:08.914 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:03:08.914 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T09:03:08.914 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T09:03:08.914 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T09:03:08.914 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:03:08.914 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:03:08.914 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:03:08.914 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:03:08.914 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.d", 2024-04-25T09:03:08.914 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpmpEvsFxAAXX1h37UWdFkyJ3Iw3xd8kw==", 2024-04-25T09:03:08.914 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:03:08.914 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T09:03:08.914 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T09:03:08.915 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T09:03:08.915 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:03:08.915 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:03:08.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:03:08.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:03:08.915 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.0", 2024-04-25T09:03:08.915 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD9GSpmi7qkORAAwSNgwzcVMitrPPf3nDyqvQ==", 2024-04-25T09:03:08.915 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:03:08.915 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T09:03:08.915 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T09:03:08.915 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:03:08.915 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:03:08.915 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:03:08.915 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:03:08.915 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.1", 2024-04-25T09:03:08.916 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAAGipmJDT3NxAAcWBCd9nt7Au+Xftjqcuc5w==", 2024-04-25T09:03:08.916 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:03:08.916 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T09:03:08.916 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T09:03:08.916 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:03:08.916 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:03:08.916 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:03:08.916 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:03:08.916 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.2", 2024-04-25T09:03:08.916 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQADGipm68kzNhAA0ZOnlr4edWDjx6RfoEHYIw==", 2024-04-25T09:03:08.916 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:03:08.916 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T09:03:08.916 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T09:03:08.916 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:03:08.916 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:03:08.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:03:08.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:03:08.917 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.3", 2024-04-25T09:03:08.917 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAGGipmT1AeNBAAoNPiEtkznSxxhVJ1EmZ1ZA==", 2024-04-25T09:03:08.917 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:03:08.917 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T09:03:08.917 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T09:03:08.917 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:03:08.917 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:03:08.917 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:03:08.917 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:03:08.917 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.4", 2024-04-25T09:03:08.917 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAMGipmBjHHNxAAH/SCzKHus8fKxxI3UN4wWQ==", 2024-04-25T09:03:08.918 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:03:08.918 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T09:03:08.918 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T09:03:08.918 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:03:08.918 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:03:08.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:03:08.918 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:03:08.918 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.5", 2024-04-25T09:03:08.918 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPGipm3FXKNRAAL97YYWONg4QHr4ajC5D7fA==", 2024-04-25T09:03:08.918 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:03:08.918 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T09:03:08.918 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T09:03:08.918 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:03:08.918 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:03:08.918 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:03:08.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:03:08.919 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.6", 2024-04-25T09:03:08.919 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQASGipmCwKaMxAAcM0p9Q2sbNQviGkwSLg8AA==", 2024-04-25T09:03:08.919 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:03:08.919 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T09:03:08.919 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T09:03:08.919 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:03:08.919 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:03:08.919 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:03:08.919 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:03:08.919 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.7", 2024-04-25T09:03:08.919 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAVGipmP7A2MRAAxnFJFosMcbZ6VDbgH8r6wA==", 2024-04-25T09:03:08.919 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:03:08.919 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T09:03:08.920 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T09:03:08.920 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:03:08.920 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:03:08.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:03:08.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:03:08.920 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.0", 2024-04-25T09:03:08.920 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpmVOjWGxAAFNmxeGi0h2D//pa6Ne3eQw==", 2024-04-25T09:03:08.920 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:03:08.920 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T09:03:08.920 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow r", 2024-04-25T09:03:08.920 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T09:03:08.920 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:03:08.920 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:03:08.920 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:03:08.920 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.1", 2024-04-25T09:03:08.921 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm24laJBAA9ijbDlFDTmF/ZZ4IhLfBVQ==", 2024-04-25T09:03:08.921 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:03:08.921 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T09:03:08.921 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-04-25T09:03:08.921 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-04-25T09:03:08.921 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-04-25T09:03:08.921 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:03:08.921 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:03:08.921 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:03:08.921 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.2", 2024-04-25T09:03:08.921 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm8b5UIRAACcOwDL630oJrXfp7s+fNTA==", 2024-04-25T09:03:08.921 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:03:08.921 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T09:03:08.921 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-04-25T09:03:08.921 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-04-25T09:03:08.922 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-04-25T09:03:08.922 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:03:08.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:03:08.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:03:08.922 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.3", 2024-04-25T09:03:08.922 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpmLsbBKRAAIKWtRdpuyCaW1LnKMWTMiw==", 2024-04-25T09:03:08.922 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:03:08.922 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T09:03:08.922 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-04-25T09:03:08.922 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-04-25T09:03:08.922 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-04-25T09:03:08.922 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:03:08.922 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:03:08.922 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:03:08.923 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.admin", 2024-04-25T09:03:08.923 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD7GSpmE9C8JRAAS263n6a4L2DXzMyS9o42NQ==", 2024-04-25T09:03:08.923 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:03:08.923 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-04-25T09:03:08.923 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T09:03:08.923 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T09:03:08.923 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:03:08.923 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:03:08.923 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:03:08.923 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:03:08.923 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.x", 2024-04-25T09:03:08.923 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm/uGhAhAAr0FaJ6SwZ8Su1MEqyZwNRA==", 2024-04-25T09:03:08.923 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:03:08.923 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-04-25T09:03:08.923 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2024-04-25T09:03:08.924 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:03:08.924 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:03:08.924 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:03:08.924 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:03:08.924 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.y", 2024-04-25T09:03:08.924 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD7GSpmPwwpOhAALeiQ/LKM2zgjE7yu16RQOg==", 2024-04-25T09:03:08.924 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:03:08.924 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-04-25T09:03:08.924 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2024-04-25T09:03:08.924 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:03:08.924 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:03:08.924 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:03:08.924 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-04-25T09:03:08.924 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:03:08.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.925+0000 7f6d71de7640 1 -- 172.21.15.43:0/4039107868 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6d5806b4d0 msgr2=0x7f6d5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:08.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.925+0000 7f6d71de7640 1 --2- 172.21.15.43:0/4039107868 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6d5806b4d0 0x7f6d5806d990 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c0cecc0 tx=0x7f6d60009040 comp rx=0 tx=0).stop 2024-04-25T09:03:08.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.925+0000 7f6d71de7640 1 -- 172.21.15.43:0/4039107868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d6c0cecf0 msgr2=0x7f6d6c15cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:08.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.925+0000 7f6d71de7640 1 --2- 172.21.15.43:0/4039107868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d6c0cecf0 0x7f6d6c15cf90 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c00e870 tx=0x7f6d5c00ed30 comp rx=0 tx=0).stop 2024-04-25T09:03:08.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.925+0000 7f6d71de7640 1 -- 172.21.15.43:0/4039107868 shutdown_connections 2024-04-25T09:03:08.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.925+0000 7f6d71de7640 1 --2- 172.21.15.43:0/4039107868 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6d6c161390 0x7f6d6c151f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:08.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.925+0000 7f6d71de7640 1 --2- 172.21.15.43:0/4039107868 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6d5806b4d0 0x7f6d5806d990 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:08.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.925+0000 7f6d71de7640 1 --2- 172.21.15.43:0/4039107868 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6d6c0d02a0 0x7f6d6c151a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:08.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.925+0000 7f6d71de7640 1 --2- 172.21.15.43:0/4039107868 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6d6c0cecf0 0x7f6d6c15cf90 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:08.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.925+0000 7f6d71de7640 1 -- 172.21.15.43:0/4039107868 >> 172.21.15.43:0/4039107868 conn(0x7f6d6c09cd20 msgr2=0x7f6d6c06b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:08.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.925+0000 7f6d71de7640 1 -- 172.21.15.43:0/4039107868 shutdown_connections 2024-04-25T09:03:08.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:08.925+0000 7f6d71de7640 1 -- 172.21.15.43:0/4039107868 wait complete. 2024-04-25T09:03:08.927 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:09.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.039+0000 7fbd64de2640 1 Processor -- start 2024-04-25T09:03:09.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.039+0000 7fbd64de2640 1 -- start start 2024-04-25T09:03:09.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.040+0000 7fbd64de2640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fbd60072160 0x7fbd60165370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.040+0000 7fbd64de2640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd600704f0 con 0x7fbd6006d9a0 2024-04-25T09:03:09.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.040+0000 7fbd64de2640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd600707d0 con 0x7fbd6015a9d0 2024-04-25T09:03:09.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.040+0000 7fbd64de2640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd60070ab0 con 0x7fbd60072160 2024-04-25T09:03:09.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.041+0000 7fbd5dd9b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fbd6006d9a0 0x7fbd60071a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:53000/0 (socket says 172.21.15.43:53000) 2024-04-25T09:03:09.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.041+0000 7fbd5dd9b640 1 -- 172.21.15.43:0/3526533395 learned_addr learned my addr 172.21.15.43:0/3526533395 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:03:09.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.041+0000 7fbd5ed9d640 1 --2- 172.21.15.43:0/3526533395 >> v2:172.21.15.193:3300/0 conn(0x7fbd60072160 0x7fbd60165370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.041+0000 7fbd5d59a640 1 -- 172.21.15.43:0/3526533395 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1518646034 0 0) 0x7fbd600704f0 con 0x7fbd6006d9a0 2024-04-25T09:03:09.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.041+0000 7fbd5d59a640 1 -- 172.21.15.43:0/3526533395 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd48003520 con 0x7fbd6006d9a0 2024-04-25T09:03:09.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.041+0000 7fbd5d59a640 1 -- 172.21.15.43:0/3526533395 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2290561245 0 0) 0x7fbd48003520 con 0x7fbd6006d9a0 2024-04-25T09:03:09.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.041+0000 7fbd5d59a640 1 -- 172.21.15.43:0/3526533395 >> v2:172.21.15.193:3300/0 conn(0x7fbd60072160 msgr2=0x7fbd60165370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:09.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.041+0000 7fbd5d59a640 1 --2- 172.21.15.43:0/3526533395 >> v2:172.21.15.193:3300/0 conn(0x7fbd60072160 0x7fbd60165370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.041+0000 7fbd5d59a640 1 -- 172.21.15.43:0/3526533395 >> v1:172.21.15.193:6789/0 conn(0x7fbd6015a9d0 legacy=0x7fbd6006d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:09.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.041+0000 7fbd5d59a640 1 -- 172.21.15.43:0/3526533395 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd60070d90 con 0x7fbd6006d9a0 2024-04-25T09:03:09.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.041+0000 7fbd5d59a640 1 -- 172.21.15.43:0/3526533395 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7fbd54001670 con 0x7fbd6006d9a0 2024-04-25T09:03:09.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.041+0000 7fbd5d59a640 1 -- 172.21.15.43:0/3526533395 >> v1:172.21.15.43:6789/0 conn(0x7fbd6006d9a0 legacy=0x7fbd60071a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:09.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.041+0000 7fbd5d59a640 1 --2- 172.21.15.43:0/3526533395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd480039b0 0x7fbd48003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.041+0000 7fbd5d59a640 1 --2- 172.21.15.43:0/3526533395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd48004800 0x7fbd48004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.041+0000 7fbd5d59a640 1 --2- 172.21.15.43:0/3526533395 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd480051c0 0x7fbd4800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.041+0000 7fbd5d59a640 1 -- 172.21.15.43:0/3526533395 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd60070ab0 con 0x7fbd480039b0 2024-04-25T09:03:09.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.041+0000 7fbd5d59a640 1 -- 172.21.15.43:0/3526533395 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4800c060 con 0x7fbd48004800 2024-04-25T09:03:09.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.041+0000 7fbd5d59a640 1 -- 172.21.15.43:0/3526533395 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4800c260 con 0x7fbd480051c0 2024-04-25T09:03:09.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.041+0000 7fbd5e59c640 1 --2- 172.21.15.43:0/3526533395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd480039b0 0x7fbd48003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.041+0000 7fbd5dd9b640 1 --2- 172.21.15.43:0/3526533395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd48004800 0x7fbd48004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.041+0000 7fbd5ed9d640 1 --2- 172.21.15.43:0/3526533395 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd480051c0 0x7fbd4800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.042+0000 7fbd5e59c640 1 -- 172.21.15.43:0/3526533395 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd480051c0 msgr2=0x7fbd4800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:09.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.042+0000 7fbd5e59c640 1 --2- 172.21.15.43:0/3526533395 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd480051c0 0x7fbd4800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.042+0000 7fbd5e59c640 1 -- 172.21.15.43:0/3526533395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd48004800 msgr2=0x7fbd48004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:09.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.042+0000 7fbd5e59c640 1 --2- 172.21.15.43:0/3526533395 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd48004800 0x7fbd48004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.042+0000 7fbd5e59c640 1 -- 172.21.15.43:0/3526533395 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd48003520 con 0x7fbd480039b0 2024-04-25T09:03:09.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.042+0000 7fbd5ed9d640 1 --2- 172.21.15.43:0/3526533395 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd480051c0 0x7fbd4800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:09.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.042+0000 7fbd5ed9d640 1 -- 172.21.15.43:0/3526533395 reap_dead start 2024-04-25T09:03:09.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.042+0000 7fbd5e59c640 1 --2- 172.21.15.43:0/3526533395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd480039b0 0x7fbd48003dd0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fbd50001750 tx=0x7fbd48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7953c9f2f2917df0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:09.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.042+0000 7fbd5d59a640 1 -- 172.21.15.43:0/3526533395 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd50001370 con 0x7fbd480039b0 2024-04-25T09:03:09.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.043+0000 7fbd5d59a640 1 -- 172.21.15.43:0/3526533395 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd500052d0 con 0x7fbd480039b0 2024-04-25T09:03:09.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.043+0000 7fbd5d59a640 1 -- 172.21.15.43:0/3526533395 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5002abc0 con 0x7fbd480039b0 2024-04-25T09:03:09.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.043+0000 7fbd64de2640 1 -- 172.21.15.43:0/3526533395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd480039b0 msgr2=0x7fbd48003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:09.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.043+0000 7fbd64de2640 1 --2- 172.21.15.43:0/3526533395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd480039b0 0x7fbd48003dd0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fbd50001750 tx=0x7fbd48005830 comp rx=0 tx=0).stop 2024-04-25T09:03:09.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.043+0000 7fbd64de2640 1 -- 172.21.15.43:0/3526533395 shutdown_connections 2024-04-25T09:03:09.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.043+0000 7fbd64de2640 1 --2- 172.21.15.43:0/3526533395 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd480039b0 0x7fbd48003dd0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.043+0000 7fbd64de2640 1 -- 172.21.15.43:0/3526533395 >> 172.21.15.43:0/3526533395 conn(0x7fbd600b1480 msgr2=0x7fbd600b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:09.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.043+0000 7fbd64de2640 1 -- 172.21.15.43:0/3526533395 shutdown_connections 2024-04-25T09:03:09.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.043+0000 7fbd64de2640 1 -- 172.21.15.43:0/3526533395 wait complete. 2024-04-25T09:03:09.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.044+0000 7fbd64de2640 1 Processor -- start 2024-04-25T09:03:09.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.045+0000 7fbd64de2640 1 -- start start 2024-04-25T09:03:09.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.045+0000 7fbd64de2640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd480039b0 0x7fbd600affb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.045+0000 7fbd64de2640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd600af530 0x7fbd600adcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.045+0000 7fbd64de2640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd60145fb0 0x7fbd6006c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.045+0000 7fbd64de2640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd60071bd0 con 0x7fbd600af530 2024-04-25T09:03:09.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.045+0000 7fbd64de2640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd60071a50 con 0x7fbd480039b0 2024-04-25T09:03:09.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.045+0000 7fbd64de2640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd60071d50 con 0x7fbd60145fb0 2024-04-25T09:03:09.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.045+0000 7fbd5dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd600af530 0x7fbd600adcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.045+0000 7fbd5dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd600af530 0x7fbd600adcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44452/0 (socket says 172.21.15.43:44452) 2024-04-25T09:03:09.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.045+0000 7fbd5dd9b640 1 -- 172.21.15.43:0/296563391 learned_addr learned my addr 172.21.15.43:0/296563391 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:09.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.045+0000 7fbd5ed9d640 1 --2- 172.21.15.43:0/296563391 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd60145fb0 0x7fbd6006c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.045+0000 7fbd5e59c640 1 --2- 172.21.15.43:0/296563391 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd480039b0 0x7fbd600affb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.046+0000 7fbd5dd9b640 1 -- 172.21.15.43:0/296563391 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd60145fb0 msgr2=0x7fbd6006c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:09.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.046+0000 7fbd5dd9b640 1 --2- 172.21.15.43:0/296563391 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd60145fb0 0x7fbd6006c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.046+0000 7fbd5dd9b640 1 -- 172.21.15.43:0/296563391 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd480039b0 msgr2=0x7fbd600affb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:09.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.046+0000 7fbd5dd9b640 1 --2- 172.21.15.43:0/296563391 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd480039b0 0x7fbd600affb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.046+0000 7fbd5dd9b640 1 -- 172.21.15.43:0/296563391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd600ae1f0 con 0x7fbd600af530 2024-04-25T09:03:09.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.046+0000 7fbd5dd9b640 1 --2- 172.21.15.43:0/296563391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd600af530 0x7fbd600adcb0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fbd540072f0 tx=0x7fbd540077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:09.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.046+0000 7fbd4effd640 1 -- 172.21.15.43:0/296563391 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd54001660 con 0x7fbd600af530 2024-04-25T09:03:09.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.046+0000 7fbd4effd640 1 -- 172.21.15.43:0/296563391 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd54018070 con 0x7fbd600af530 2024-04-25T09:03:09.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.046+0000 7fbd64de2640 1 -- 172.21.15.43:0/296563391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd600ae490 con 0x7fbd600af530 2024-04-25T09:03:09.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.046+0000 7fbd4effd640 1 -- 172.21.15.43:0/296563391 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5400fb80 con 0x7fbd600af530 2024-04-25T09:03:09.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.046+0000 7fbd64de2640 1 -- 172.21.15.43:0/296563391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd600aea50 con 0x7fbd600af530 2024-04-25T09:03:09.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.047+0000 7fbd64de2640 1 -- 172.21.15.43:0/296563391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd60074ef0 con 0x7fbd600af530 2024-04-25T09:03:09.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.047+0000 7fbd4effd640 1 -- 172.21.15.43:0/296563391 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbd5400fd20 con 0x7fbd600af530 2024-04-25T09:03:09.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.048+0000 7fbd4effd640 1 --2- 172.21.15.43:0/296563391 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbd4006b6a0 0x7fbd4006db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.048+0000 7fbd4effd640 1 -- 172.21.15.43:0/296563391 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbd54089e10 con 0x7fbd600af530 2024-04-25T09:03:09.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.048+0000 7fbd5e59c640 1 --2- 172.21.15.43:0/296563391 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbd4006b6a0 0x7fbd4006db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.039 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.049+0000 7fbd5e59c640 1 --2- 172.21.15.43:0/296563391 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbd4006b6a0 0x7fbd4006db60 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fbd50032000 tx=0x7fbd5002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:09.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.052+0000 7fbd4effd640 1 -- 172.21.15.43:0/296563391 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbd54058a20 con 0x7fbd600af530 2024-04-25T09:03:09.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.234+0000 7fbd64de2640 1 -- 172.21.15.43:0/296563391 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbd600b1ce0 con 0x7fbd600af530 2024-04-25T09:03:09.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.235+0000 7fbd4effd640 1 -- 172.21.15.43:0/296563391 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+12865 (secure 0 0 0) 0x7fbd5405cac0 con 0x7fbd600af530 2024-04-25T09:03:09.224 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:03:09.224 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":122,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:03:08.540599+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":111,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:09.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.237+0000 7fbd64de2640 1 -- 172.21.15.43:0/296563391 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbd4006b6a0 msgr2=0x7fbd4006db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:09.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.237+0000 7fbd64de2640 1 --2- 172.21.15.43:0/296563391 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbd4006b6a0 0x7fbd4006db60 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fbd50032000 tx=0x7fbd5002e000 comp rx=0 tx=0).stop 2024-04-25T09:03:09.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.237+0000 7fbd64de2640 1 -- 172.21.15.43:0/296563391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd600af530 msgr2=0x7fbd600adcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:09.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.237+0000 7fbd64de2640 1 --2- 172.21.15.43:0/296563391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd600af530 0x7fbd600adcb0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fbd540072f0 tx=0x7fbd540077b0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.237+0000 7fbd64de2640 1 -- 172.21.15.43:0/296563391 shutdown_connections 2024-04-25T09:03:09.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.237+0000 7fbd64de2640 1 --2- 172.21.15.43:0/296563391 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd60145fb0 0x7fbd6006c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.237+0000 7fbd64de2640 1 --2- 172.21.15.43:0/296563391 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd600af530 0x7fbd600adcb0 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.237+0000 7fbd64de2640 1 --2- 172.21.15.43:0/296563391 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbd4006b6a0 0x7fbd4006db60 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.237+0000 7fbd64de2640 1 --2- 172.21.15.43:0/296563391 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd480039b0 0x7fbd600affb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.237+0000 7fbd64de2640 1 -- 172.21.15.43:0/296563391 >> 172.21.15.43:0/296563391 conn(0x7fbd600b1480 msgr2=0x7fbd6015cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:09.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.237+0000 7fbd64de2640 1 -- 172.21.15.43:0/296563391 shutdown_connections 2024-04-25T09:03:09.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.237+0000 7fbd64de2640 1 -- 172.21.15.43:0/296563391 wait complete. 2024-04-25T09:03:09.237 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:03:09.237 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:03:09.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.352+0000 7f5034e8f640 1 Processor -- start 2024-04-25T09:03:09.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.352+0000 7f5034e8f640 1 -- start start 2024-04-25T09:03:09.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.353+0000 7f5034e8f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f503006f6a0 0x7f50300b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.353+0000 7f5034e8f640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f50300cd470 0x7f503006c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.353+0000 7f5034e8f640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50300d2620 con 0x7f503015ddd0 2024-04-25T09:03:09.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.353+0000 7f5034e8f640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50300b0b80 con 0x7f50300cd470 2024-04-25T09:03:09.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.353+0000 7f5034e8f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50300b0e60 con 0x7f503006f6a0 2024-04-25T09:03:09.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.354+0000 7f502dd9b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f503006f6a0 0x7f50300b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.354+0000 7f502ed9d640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f50300cd470 0x7f503006c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.354+0000 7f502e59c640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f503015ddd0 0x7f503006ef90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:32992/0 (socket says 172.21.15.43:32992) 2024-04-25T09:03:09.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.354+0000 7f502e59c640 1 -- 172.21.15.43:0/4007305802 learned_addr learned my addr 172.21.15.43:0/4007305802 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:03:09.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.354+0000 7f502d59a640 1 -- 172.21.15.43:0/4007305802 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 551341937 0 0) 0x7f50300d2620 con 0x7f503015ddd0 2024-04-25T09:03:09.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.354+0000 7f502dd9b640 1 -- 172.21.15.43:0/4007305802 >> v2:172.21.15.193:3300/0 conn(0x7f50300cd470 msgr2=0x7f503006c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:09.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.354+0000 7f502dd9b640 1 --2- 172.21.15.43:0/4007305802 >> v2:172.21.15.193:3300/0 conn(0x7f50300cd470 0x7f503006c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.354+0000 7f502dd9b640 1 -- 172.21.15.43:0/4007305802 >> v1:172.21.15.193:6789/0 conn(0x7f503015ddd0 legacy=0x7f503006ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:09.343 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.354+0000 7f502dd9b640 1 -- 172.21.15.43:0/4007305802 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50300cbf40 con 0x7f503006f6a0 2024-04-25T09:03:09.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.355+0000 7f502dd9b640 1 --2- 172.21.15.43:0/4007305802 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f503006f6a0 0x7f50300b3dd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f50240097a0 tx=0x7f502402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b4817e0c70f67ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:09.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.355+0000 7f502d59a640 1 -- 172.21.15.43:0/4007305802 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5024037070 con 0x7f503006f6a0 2024-04-25T09:03:09.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.355+0000 7f502d59a640 1 -- 172.21.15.43:0/4007305802 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f502402ae50 con 0x7f503006f6a0 2024-04-25T09:03:09.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.355+0000 7f502d59a640 1 -- 172.21.15.43:0/4007305802 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5024032660 con 0x7f503006f6a0 2024-04-25T09:03:09.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.355+0000 7f5034e8f640 1 -- 172.21.15.43:0/4007305802 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f503006f6a0 msgr2=0x7f50300b3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:09.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.355+0000 7f5034e8f640 1 --2- 172.21.15.43:0/4007305802 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f503006f6a0 0x7f50300b3dd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f50240097a0 tx=0x7f502402a100 comp rx=0 tx=0).stop 2024-04-25T09:03:09.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.356+0000 7f5034e8f640 1 -- 172.21.15.43:0/4007305802 shutdown_connections 2024-04-25T09:03:09.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.356+0000 7f5034e8f640 1 --2- 172.21.15.43:0/4007305802 >> v2:172.21.15.193:3300/0 conn(0x7f50300cd470 0x7f503006c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.356+0000 7f5034e8f640 1 --2- 172.21.15.43:0/4007305802 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f503006f6a0 0x7f50300b3dd0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.356+0000 7f5034e8f640 1 -- 172.21.15.43:0/4007305802 >> 172.21.15.43:0/4007305802 conn(0x7f50300d17e0 msgr2=0x7f50300d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:09.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.356+0000 7f5034e8f640 1 -- 172.21.15.43:0/4007305802 shutdown_connections 2024-04-25T09:03:09.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.356+0000 7f5034e8f640 1 -- 172.21.15.43:0/4007305802 wait complete. 2024-04-25T09:03:09.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.357+0000 7f5034e8f640 1 Processor -- start 2024-04-25T09:03:09.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.357+0000 7f5034e8f640 1 -- start start 2024-04-25T09:03:09.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.358+0000 7f5034e8f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f503006f6a0 0x7f50300b6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.358+0000 7f5034e8f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50300cd470 0x7f50301f70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.358+0000 7f5034e8f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f503015ddd0 0x7f50301f7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.358+0000 7f5034e8f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503006ce30 con 0x7f503006f6a0 2024-04-25T09:03:09.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.358+0000 7f5034e8f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f503006ccb0 con 0x7f50300cd470 2024-04-25T09:03:09.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.358+0000 7f5034e8f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f503006cfb0 con 0x7f503015ddd0 2024-04-25T09:03:09.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.358+0000 7f502e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f503006f6a0 0x7f50300b6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.358+0000 7f502e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f503006f6a0 0x7f50300b6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:44458/0 (socket says 172.21.15.43:44458) 2024-04-25T09:03:09.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.358+0000 7f502e59c640 1 -- 172.21.15.43:0/1308662469 learned_addr learned my addr 172.21.15.43:0/1308662469 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:09.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.358+0000 7f502dd9b640 1 --2- 172.21.15.43:0/1308662469 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50300cd470 0x7f50301f70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.358+0000 7f502ed9d640 1 --2- 172.21.15.43:0/1308662469 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f503015ddd0 0x7f50301f7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.358+0000 7f502e59c640 1 -- 172.21.15.43:0/1308662469 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f503015ddd0 msgr2=0x7f50301f7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:09.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.358+0000 7f502e59c640 1 --2- 172.21.15.43:0/1308662469 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f503015ddd0 0x7f50301f7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.358+0000 7f502e59c640 1 -- 172.21.15.43:0/1308662469 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50300cd470 msgr2=0x7f50301f70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:09.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.358+0000 7f502e59c640 1 --2- 172.21.15.43:0/1308662469 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50300cd470 0x7f50301f70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.358+0000 7f502e59c640 1 -- 172.21.15.43:0/1308662469 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50300b2cf0 con 0x7f503006f6a0 2024-04-25T09:03:09.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.358+0000 7f502dd9b640 1 --2- 172.21.15.43:0/1308662469 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50300cd470 0x7f50301f70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:09.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.358+0000 7f502e59c640 1 --2- 172.21.15.43:0/1308662469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f503006f6a0 0x7f50300b6c10 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f502000c7c0 tx=0x7f502000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:09.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.358+0000 7f501effd640 1 -- 172.21.15.43:0/1308662469 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5020018070 con 0x7f503006f6a0 2024-04-25T09:03:09.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.359+0000 7f501effd640 1 -- 172.21.15.43:0/1308662469 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f502000a640 con 0x7f503006f6a0 2024-04-25T09:03:09.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.359+0000 7f5034e8f640 1 -- 172.21.15.43:0/1308662469 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50300b1c50 con 0x7f503006f6a0 2024-04-25T09:03:09.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.359+0000 7f501effd640 1 -- 172.21.15.43:0/1308662469 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502000aa30 con 0x7f503006f6a0 2024-04-25T09:03:09.349 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.359+0000 7f5034e8f640 1 -- 172.21.15.43:0/1308662469 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50300b2210 con 0x7f503006f6a0 2024-04-25T09:03:09.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.359+0000 7f5034e8f640 1 -- 172.21.15.43:0/1308662469 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5030074f60 con 0x7f503006f6a0 2024-04-25T09:03:09.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.360+0000 7f501effd640 1 -- 172.21.15.43:0/1308662469 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f502001b070 con 0x7f503006f6a0 2024-04-25T09:03:09.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.361+0000 7f501effd640 1 --2- 172.21.15.43:0/1308662469 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f501006b500 0x7f501006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.361+0000 7f501effd640 1 -- 172.21.15.43:0/1308662469 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5020089430 con 0x7f503006f6a0 2024-04-25T09:03:09.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.361+0000 7f502dd9b640 1 --2- 172.21.15.43:0/1308662469 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f501006b500 0x7f501006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.361+0000 7f502dd9b640 1 --2- 172.21.15.43:0/1308662469 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f501006b500 0x7f501006d9c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f5024034000 tx=0x7f502402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:09.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.365+0000 7f501effd640 1 -- 172.21.15.43:0/1308662469 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5020058690 con 0x7f503006f6a0 2024-04-25T09:03:09.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.545+0000 7f5034e8f640 1 -- 172.21.15.43:0/1308662469 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f503006f0c0 con 0x7f503006f6a0 2024-04-25T09:03:09.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.545+0000 7f501effd640 1 -- 172.21.15.43:0/1308662469 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v122) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5020057f90 con 0x7f503006f6a0 2024-04-25T09:03:09.534 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T09:03:09.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.547+0000 7f5034e8f640 1 -- 172.21.15.43:0/1308662469 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f501006b500 msgr2=0x7f501006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:09.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.547+0000 7f5034e8f640 1 --2- 172.21.15.43:0/1308662469 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f501006b500 0x7f501006d9c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f5024034000 tx=0x7f502402e000 comp rx=0 tx=0).stop 2024-04-25T09:03:09.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.547+0000 7f5034e8f640 1 -- 172.21.15.43:0/1308662469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f503006f6a0 msgr2=0x7f50300b6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:09.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.547+0000 7f5034e8f640 1 --2- 172.21.15.43:0/1308662469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f503006f6a0 0x7f50300b6c10 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f502000c7c0 tx=0x7f502000cc80 comp rx=0 tx=0).stop 2024-04-25T09:03:09.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.547+0000 7f5034e8f640 1 -- 172.21.15.43:0/1308662469 shutdown_connections 2024-04-25T09:03:09.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.547+0000 7f5034e8f640 1 --2- 172.21.15.43:0/1308662469 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f503015ddd0 0x7f50301f7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.547+0000 7f5034e8f640 1 --2- 172.21.15.43:0/1308662469 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f501006b500 0x7f501006d9c0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.547+0000 7f5034e8f640 1 --2- 172.21.15.43:0/1308662469 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f50300cd470 0x7f50301f70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.547+0000 7f5034e8f640 1 --2- 172.21.15.43:0/1308662469 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f503006f6a0 0x7f50300b6c10 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.547+0000 7f5034e8f640 1 -- 172.21.15.43:0/1308662469 >> 172.21.15.43:0/1308662469 conn(0x7f50300d17e0 msgr2=0x7f503015cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:09.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.547+0000 7f5034e8f640 1 -- 172.21.15.43:0/1308662469 shutdown_connections 2024-04-25T09:03:09.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.547+0000 7f5034e8f640 1 -- 172.21.15.43:0/1308662469 wait complete. 2024-04-25T09:03:09.547 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:03:09.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.660+0000 7fc88586b640 1 Processor -- start 2024-04-25T09:03:09.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.660+0000 7fc88586b640 1 -- start start 2024-04-25T09:03:09.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.661+0000 7fc88586b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc88015df50 0x7fc8801619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.661+0000 7fc88586b640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fc880161f30 0x7fc880162380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.661+0000 7fc88586b640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8800d24a0 con 0x7fc8800c7cf0 2024-04-25T09:03:09.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.661+0000 7fc88586b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc88015e330 con 0x7fc88015df50 2024-04-25T09:03:09.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.661+0000 7fc88586b640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8800d2740 con 0x7fc880161f30 2024-04-25T09:03:09.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.661+0000 7fc87effd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc88015df50 0x7fc8801619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.661+0000 7fc87effd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc88015df50 0x7fc8801619f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45066/0 (socket says 172.21.15.43:45066) 2024-04-25T09:03:09.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.661+0000 7fc87effd640 1 -- 172.21.15.43:0/2390982663 learned_addr learned my addr 172.21.15.43:0/2390982663 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:09.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.661+0000 7fc87e7fc640 1 --2- 172.21.15.43:0/2390982663 >> v2:172.21.15.193:3300/0 conn(0x7fc880161f30 0x7fc880162380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.661+0000 7fc87effd640 1 -- 172.21.15.43:0/2390982663 >> v2:172.21.15.193:3300/0 conn(0x7fc880161f30 msgr2=0x7fc880162380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:09.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.661+0000 7fc87effd640 1 --2- 172.21.15.43:0/2390982663 >> v2:172.21.15.193:3300/0 conn(0x7fc880161f30 0x7fc880162380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.661+0000 7fc87effd640 1 -- 172.21.15.43:0/2390982663 >> v1:172.21.15.193:6789/0 conn(0x7fc8800c7cf0 legacy=0x7fc8800c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:09.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.661+0000 7fc87effd640 1 -- 172.21.15.43:0/2390982663 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8800d2a20 con 0x7fc88015df50 2024-04-25T09:03:09.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.661+0000 7fc87e7fc640 1 --2- 172.21.15.43:0/2390982663 >> v2:172.21.15.193:3300/0 conn(0x7fc880161f30 0x7fc880162380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:09.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.662+0000 7fc87effd640 1 --2- 172.21.15.43:0/2390982663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc88015df50 0x7fc8801619f0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fc88008c880 tx=0x7fc87002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7409d252917a4c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:09.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.662+0000 7fc87dffb640 1 -- 172.21.15.43:0/2390982663 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc870037070 con 0x7fc88015df50 2024-04-25T09:03:09.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.662+0000 7fc87dffb640 1 -- 172.21.15.43:0/2390982663 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc870032440 con 0x7fc88015df50 2024-04-25T09:03:09.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.662+0000 7fc87dffb640 1 -- 172.21.15.43:0/2390982663 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc870032810 con 0x7fc88015df50 2024-04-25T09:03:09.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.662+0000 7fc88586b640 1 -- 172.21.15.43:0/2390982663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc88015df50 msgr2=0x7fc8801619f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:09.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.662+0000 7fc88586b640 1 --2- 172.21.15.43:0/2390982663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc88015df50 0x7fc8801619f0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fc88008c880 tx=0x7fc87002a200 comp rx=0 tx=0).stop 2024-04-25T09:03:09.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.663+0000 7fc88586b640 1 -- 172.21.15.43:0/2390982663 shutdown_connections 2024-04-25T09:03:09.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.663+0000 7fc88586b640 1 --2- 172.21.15.43:0/2390982663 >> v2:172.21.15.193:3300/0 conn(0x7fc880161f30 0x7fc880162380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.663+0000 7fc88586b640 1 --2- 172.21.15.43:0/2390982663 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc88015df50 0x7fc8801619f0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.663+0000 7fc88586b640 1 -- 172.21.15.43:0/2390982663 >> 172.21.15.43:0/2390982663 conn(0x7fc8800d3a30 msgr2=0x7fc8800d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:09.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.663+0000 7fc88586b640 1 -- 172.21.15.43:0/2390982663 shutdown_connections 2024-04-25T09:03:09.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.663+0000 7fc88586b640 1 -- 172.21.15.43:0/2390982663 wait complete. 2024-04-25T09:03:09.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.664+0000 7fc88586b640 1 Processor -- start 2024-04-25T09:03:09.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.664+0000 7fc88586b640 1 -- start start 2024-04-25T09:03:09.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.665+0000 7fc88586b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8800c7cf0 0x7fc880151f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.665+0000 7fc88586b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc88015df50 0x7fc88009d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.665+0000 7fc88586b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc880161f30 0x7fc88009dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.665+0000 7fc88586b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8800d2190 con 0x7fc8800c7cf0 2024-04-25T09:03:09.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.665+0000 7fc88586b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8800c77d0 con 0x7fc880161f30 2024-04-25T09:03:09.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.665+0000 7fc88586b640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8800cf670 con 0x7fc88015df50 2024-04-25T09:03:09.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.665+0000 7fc87effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8800c7cf0 0x7fc880151f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.665+0000 7fc87effd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8800c7cf0 0x7fc880151f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45080/0 (socket says 172.21.15.43:45080) 2024-04-25T09:03:09.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.665+0000 7fc87effd640 1 -- 172.21.15.43:0/4043282924 learned_addr learned my addr 172.21.15.43:0/4043282924 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:09.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.665+0000 7fc87e7fc640 1 --2- 172.21.15.43:0/4043282924 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc88015df50 0x7fc88009d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.665+0000 7fc87f7fe640 1 --2- 172.21.15.43:0/4043282924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc880161f30 0x7fc88009dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.665+0000 7fc87effd640 1 -- 172.21.15.43:0/4043282924 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc88015df50 msgr2=0x7fc88009d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:09.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.665+0000 7fc87effd640 1 --2- 172.21.15.43:0/4043282924 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc88015df50 0x7fc88009d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.665+0000 7fc87effd640 1 -- 172.21.15.43:0/4043282924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc880161f30 msgr2=0x7fc88009dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:09.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.665+0000 7fc87effd640 1 --2- 172.21.15.43:0/4043282924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc880161f30 0x7fc88009dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.665+0000 7fc87effd640 1 -- 172.21.15.43:0/4043282924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8801f4c10 con 0x7fc8800c7cf0 2024-04-25T09:03:09.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.665+0000 7fc87f7fe640 1 --2- 172.21.15.43:0/4043282924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc880161f30 0x7fc88009dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:09.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.665+0000 7fc87effd640 1 --2- 172.21.15.43:0/4043282924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8800c7cf0 0x7fc880151f20 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fc870000c00 tx=0x7fc8700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:09.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.665+0000 7fc867fff640 1 -- 172.21.15.43:0/4043282924 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc870041020 con 0x7fc8800c7cf0 2024-04-25T09:03:09.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.666+0000 7fc867fff640 1 -- 172.21.15.43:0/4043282924 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc870030970 con 0x7fc8800c7cf0 2024-04-25T09:03:09.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.666+0000 7fc88586b640 1 -- 172.21.15.43:0/4043282924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8801f4ef0 con 0x7fc8800c7cf0 2024-04-25T09:03:09.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.666+0000 7fc867fff640 1 -- 172.21.15.43:0/4043282924 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc870037040 con 0x7fc8800c7cf0 2024-04-25T09:03:09.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.666+0000 7fc88586b640 1 -- 172.21.15.43:0/4043282924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc880152460 con 0x7fc8800c7cf0 2024-04-25T09:03:09.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.667+0000 7fc88586b640 1 -- 172.21.15.43:0/4043282924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8800680f0 con 0x7fc8800c7cf0 2024-04-25T09:03:09.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.667+0000 7fc867fff640 1 -- 172.21.15.43:0/4043282924 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc870044020 con 0x7fc8800c7cf0 2024-04-25T09:03:09.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.668+0000 7fc867fff640 1 --2- 172.21.15.43:0/4043282924 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc86c06b4d0 0x7fc86c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.668+0000 7fc867fff640 1 -- 172.21.15.43:0/4043282924 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc8700aa760 con 0x7fc8800c7cf0 2024-04-25T09:03:09.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.668+0000 7fc87e7fc640 1 --2- 172.21.15.43:0/4043282924 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc86c06b4d0 0x7fc86c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.668+0000 7fc87e7fc640 1 --2- 172.21.15.43:0/4043282924 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc86c06b4d0 0x7fc86c06d990 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fc88009ff70 tx=0x7fc874008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:09.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.672+0000 7fc867fff640 1 -- 172.21.15.43:0/4043282924 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc87004b030 con 0x7fc8800c7cf0 2024-04-25T09:03:09.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.856+0000 7fc88586b640 1 -- 172.21.15.43:0/4043282924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc880074ef0 con 0x7fc8800c7cf0 2024-04-25T09:03:09.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.857+0000 7fc867fff640 1 -- 172.21.15.43:0/4043282924 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v122) v1 ==== 84+0+65 (secure 0 0 0) 0x7fc8700799c0 con 0x7fc8800c7cf0 2024-04-25T09:03:09.845 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:03:09.846 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2024-04-25T09:03:09.846 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:03:09.846 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 1, 2024-04-25T09:03:09.846 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": ".mgr" 2024-04-25T09:03:09.846 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:03:09.846 INFO:teuthology.orchestra.run.smithi043.stdout:] 2024-04-25T09:03:09.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.859+0000 7fc88586b640 1 -- 172.21.15.43:0/4043282924 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc86c06b4d0 msgr2=0x7fc86c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:09.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.859+0000 7fc88586b640 1 --2- 172.21.15.43:0/4043282924 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc86c06b4d0 0x7fc86c06d990 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fc88009ff70 tx=0x7fc874008040 comp rx=0 tx=0).stop 2024-04-25T09:03:09.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.859+0000 7fc88586b640 1 -- 172.21.15.43:0/4043282924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8800c7cf0 msgr2=0x7fc880151f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:09.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.859+0000 7fc88586b640 1 --2- 172.21.15.43:0/4043282924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8800c7cf0 0x7fc880151f20 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fc870000c00 tx=0x7fc8700046c0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.859+0000 7fc88586b640 1 -- 172.21.15.43:0/4043282924 shutdown_connections 2024-04-25T09:03:09.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.859+0000 7fc88586b640 1 --2- 172.21.15.43:0/4043282924 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc86c06b4d0 0x7fc86c06d990 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.859+0000 7fc88586b640 1 --2- 172.21.15.43:0/4043282924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc880161f30 0x7fc88009dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.859+0000 7fc88586b640 1 --2- 172.21.15.43:0/4043282924 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc88015df50 0x7fc88009d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.859+0000 7fc88586b640 1 --2- 172.21.15.43:0/4043282924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc8800c7cf0 0x7fc880151f20 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.859+0000 7fc88586b640 1 -- 172.21.15.43:0/4043282924 >> 172.21.15.43:0/4043282924 conn(0x7fc8800d3a30 msgr2=0x7fc880153e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:09.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.859+0000 7fc88586b640 1 -- 172.21.15.43:0/4043282924 shutdown_connections 2024-04-25T09:03:09.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.859+0000 7fc88586b640 1 -- 172.21.15.43:0/4043282924 wait complete. 2024-04-25T09:03:09.859 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:03:09.859 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:03:09.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.975+0000 7f3bb305d640 1 Processor -- start 2024-04-25T09:03:09.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.975+0000 7f3bb305d640 1 -- start start 2024-04-25T09:03:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.976+0000 7f3bb305d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3bac0cdac0 0x7f3bac0ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.976+0000 7f3bb305d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bac0cef60 con 0x7f3bac147290 2024-04-25T09:03:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.976+0000 7f3bb305d640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bac0716b0 con 0x7f3bac0ce7e0 2024-04-25T09:03:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.976+0000 7f3bb305d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bac071460 con 0x7f3bac0cdac0 2024-04-25T09:03:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.976+0000 7f3babfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3bac0cdac0 0x7f3bac0ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.976+0000 7f3bb0df9640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f3bac147290 0x7f3bac0cd3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:33502/0 (socket says 172.21.15.43:33502) 2024-04-25T09:03:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.976+0000 7f3bb0df9640 1 -- 172.21.15.43:0/611301967 learned_addr learned my addr 172.21.15.43:0/611301967 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:03:09.965 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.976+0000 7f3babfff640 1 -- 172.21.15.43:0/611301967 >> v1:172.21.15.193:6789/0 conn(0x7f3bac0ce7e0 legacy=0x7f3bac15e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.976+0000 7f3babfff640 1 -- 172.21.15.43:0/611301967 >> v1:172.21.15.43:6789/0 conn(0x7f3bac147290 legacy=0x7f3bac0cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.976+0000 7f3bab7fe640 1 -- 172.21.15.43:0/611301967 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 320784516 0 0) 0x7f3bac0cef60 con 0x7f3bac147290 2024-04-25T09:03:09.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.976+0000 7f3babfff640 1 -- 172.21.15.43:0/611301967 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bac071990 con 0x7f3bac0cdac0 2024-04-25T09:03:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.977+0000 7f3babfff640 1 --2- 172.21.15.43:0/611301967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bac0cdac0 0x7f3bac0ce2a0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f3ba00097a0 tx=0x7f3ba002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64be968d26235b57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.977+0000 7f3bab7fe640 1 -- 172.21.15.43:0/611301967 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0037070 con 0x7f3bac0cdac0 2024-04-25T09:03:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.977+0000 7f3bab7fe640 1 -- 172.21.15.43:0/611301967 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba0032440 con 0x7f3bac0cdac0 2024-04-25T09:03:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.977+0000 7f3bab7fe640 1 -- 172.21.15.43:0/611301967 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba002ae60 con 0x7f3bac0cdac0 2024-04-25T09:03:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.978+0000 7f3bb305d640 1 -- 172.21.15.43:0/611301967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bac0cdac0 msgr2=0x7f3bac0ce2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.978+0000 7f3bb305d640 1 --2- 172.21.15.43:0/611301967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bac0cdac0 0x7f3bac0ce2a0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f3ba00097a0 tx=0x7f3ba002a110 comp rx=0 tx=0).stop 2024-04-25T09:03:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.978+0000 7f3bb305d640 1 -- 172.21.15.43:0/611301967 shutdown_connections 2024-04-25T09:03:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.978+0000 7f3bb305d640 1 --2- 172.21.15.43:0/611301967 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bac0cdac0 0x7f3bac0ce2a0 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.978+0000 7f3bb305d640 1 -- 172.21.15.43:0/611301967 >> 172.21.15.43:0/611301967 conn(0x7f3bac06be20 msgr2=0x7f3bac06c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:09.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.978+0000 7f3bb305d640 1 -- 172.21.15.43:0/611301967 shutdown_connections 2024-04-25T09:03:09.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.978+0000 7f3bb305d640 1 -- 172.21.15.43:0/611301967 wait complete. 2024-04-25T09:03:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.979+0000 7f3bb305d640 1 Processor -- start 2024-04-25T09:03:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.979+0000 7f3bb305d640 1 -- start start 2024-04-25T09:03:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.980+0000 7f3bb305d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bac0cdac0 0x7f3bac15d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.980+0000 7f3bb305d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3bac0ce7e0 0x7f3bac0c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.980+0000 7f3bb305d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3bac147290 0x7f3bac0c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.980+0000 7f3bb305d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bac147670 con 0x7f3bac0cdac0 2024-04-25T09:03:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.980+0000 7f3bb305d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bac0cec70 con 0x7f3bac147290 2024-04-25T09:03:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.980+0000 7f3bb305d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bac1477f0 con 0x7f3bac0ce7e0 2024-04-25T09:03:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.980+0000 7f3bb0df9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bac0cdac0 0x7f3bac15d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.980+0000 7f3bb0df9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bac0cdac0 0x7f3bac15d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45106/0 (socket says 172.21.15.43:45106) 2024-04-25T09:03:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.980+0000 7f3bb0df9640 1 -- 172.21.15.43:0/4247480523 learned_addr learned my addr 172.21.15.43:0/4247480523 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.980+0000 7f3babfff640 1 --2- 172.21.15.43:0/4247480523 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3bac0ce7e0 0x7f3bac0c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.980+0000 7f3bb15fa640 1 --2- 172.21.15.43:0/4247480523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3bac147290 0x7f3bac0c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.980+0000 7f3bb0df9640 1 -- 172.21.15.43:0/4247480523 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3bac0ce7e0 msgr2=0x7f3bac0c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.980+0000 7f3bb0df9640 1 --2- 172.21.15.43:0/4247480523 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3bac0ce7e0 0x7f3bac0c8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.980+0000 7f3bb0df9640 1 -- 172.21.15.43:0/4247480523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3bac147290 msgr2=0x7f3bac0c89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.980+0000 7f3bb0df9640 1 --2- 172.21.15.43:0/4247480523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3bac147290 0x7f3bac0c89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:09.971 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.980+0000 7f3bb0df9640 1 -- 172.21.15.43:0/4247480523 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bac0cb430 con 0x7f3bac0cdac0 2024-04-25T09:03:09.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.980+0000 7f3bb15fa640 1 --2- 172.21.15.43:0/4247480523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3bac147290 0x7f3bac0c89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:09.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.981+0000 7f3bb0df9640 1 --2- 172.21.15.43:0/4247480523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bac0cdac0 0x7f3bac15d7a0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400c8c0 tx=0x7f3ba400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:09.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.981+0000 7f3ba97fa640 1 -- 172.21.15.43:0/4247480523 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4018070 con 0x7f3bac0cdac0 2024-04-25T09:03:09.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.981+0000 7f3ba97fa640 1 -- 172.21.15.43:0/4247480523 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba400a640 con 0x7f3bac0cdac0 2024-04-25T09:03:09.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.981+0000 7f3bb305d640 1 -- 172.21.15.43:0/4247480523 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bac0cb710 con 0x7f3bac0cdac0 2024-04-25T09:03:09.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.981+0000 7f3ba97fa640 1 -- 172.21.15.43:0/4247480523 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba400aa70 con 0x7f3bac0cdac0 2024-04-25T09:03:09.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.981+0000 7f3bb305d640 1 -- 172.21.15.43:0/4247480523 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bac0cbcd0 con 0x7f3bac0cdac0 2024-04-25T09:03:09.972 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.982+0000 7f3bb305d640 1 -- 172.21.15.43:0/4247480523 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bac068a40 con 0x7f3bac0cdac0 2024-04-25T09:03:09.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.982+0000 7f3ba97fa640 1 -- 172.21.15.43:0/4247480523 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ba401b070 con 0x7f3bac0cdac0 2024-04-25T09:03:09.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.983+0000 7f3ba97fa640 1 --2- 172.21.15.43:0/4247480523 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3b9806b500 0x7f3b9806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:09.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.983+0000 7f3ba97fa640 1 -- 172.21.15.43:0/4247480523 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3ba4089480 con 0x7f3bac0cdac0 2024-04-25T09:03:09.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.983+0000 7f3babfff640 1 --2- 172.21.15.43:0/4247480523 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3b9806b500 0x7f3b9806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:09.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.984+0000 7f3babfff640 1 --2- 172.21.15.43:0/4247480523 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3b9806b500 0x7f3b9806d9c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0034000 tx=0x7f3ba002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:09.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:09.987+0000 7f3ba97fa640 1 -- 172.21.15.43:0/4247480523 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3ba4057ff0 con 0x7f3bac0cdac0 2024-04-25T09:03:10.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.167+0000 7f3bb305d640 1 -- 172.21.15.43:0/4247480523 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3bac0b3270 con 0x7f3bac0cdac0 2024-04-25T09:03:10.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.538+0000 7f3ba97fa640 1 -- 172.21.15.43:0/4247480523 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v123) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3ba405c090 con 0x7f3bac0cdac0 2024-04-25T09:03:10.527 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' created 2024-04-25T09:03:10.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.541+0000 7f3bb305d640 1 -- 172.21.15.43:0/4247480523 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3b9806b500 msgr2=0x7f3b9806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:10.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.541+0000 7f3bb305d640 1 --2- 172.21.15.43:0/4247480523 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3b9806b500 0x7f3b9806d9c0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0034000 tx=0x7f3ba002e000 comp rx=0 tx=0).stop 2024-04-25T09:03:10.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.541+0000 7f3bb305d640 1 -- 172.21.15.43:0/4247480523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bac0cdac0 msgr2=0x7f3bac15d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:10.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.541+0000 7f3bb305d640 1 --2- 172.21.15.43:0/4247480523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bac0cdac0 0x7f3bac15d7a0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400c8c0 tx=0x7f3ba400cd80 comp rx=0 tx=0).stop 2024-04-25T09:03:10.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.541+0000 7f3bb305d640 1 -- 172.21.15.43:0/4247480523 shutdown_connections 2024-04-25T09:03:10.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.541+0000 7f3bb305d640 1 --2- 172.21.15.43:0/4247480523 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3b9806b500 0x7f3b9806d9c0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.541+0000 7f3bb305d640 1 --2- 172.21.15.43:0/4247480523 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3bac147290 0x7f3bac0c89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.541+0000 7f3bb305d640 1 --2- 172.21.15.43:0/4247480523 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3bac0ce7e0 0x7f3bac0c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.541+0000 7f3bb305d640 1 --2- 172.21.15.43:0/4247480523 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3bac0cdac0 0x7f3bac15d7a0 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.541+0000 7f3bb305d640 1 -- 172.21.15.43:0/4247480523 >> 172.21.15.43:0/4247480523 conn(0x7f3bac06be20 msgr2=0x7f3bac072170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:10.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.542+0000 7f3bb305d640 1 -- 172.21.15.43:0/4247480523 shutdown_connections 2024-04-25T09:03:10.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.542+0000 7f3bb305d640 1 -- 172.21.15.43:0/4247480523 wait complete. 2024-04-25T09:03:10.546 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:03:10.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.667+0000 7f7b0344a640 1 Processor -- start 2024-04-25T09:03:10.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.667+0000 7f7b0344a640 1 -- start start 2024-04-25T09:03:10.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.668+0000 7f7b0344a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7afc0d03a0 0x7f7afc0d1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:10.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.668+0000 7f7b0344a640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f7afc0d1560 0x7f7afc0d0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:10.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.668+0000 7f7b0344a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7afc0d0cc0 con 0x7f7afc0ce430 2024-04-25T09:03:10.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.668+0000 7f7b0344a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7afc0cf870 con 0x7f7afc0d03a0 2024-04-25T09:03:10.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.668+0000 7f7b0344a640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7afc0d2930 con 0x7f7afc0d1560 2024-04-25T09:03:10.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.668+0000 7f7b011e6640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7afc0d03a0 0x7f7afc0d1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:10.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.668+0000 7f7b011e6640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7afc0d03a0 0x7f7afc0d1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45114/0 (socket says 172.21.15.43:45114) 2024-04-25T09:03:10.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.668+0000 7f7b011e6640 1 -- 172.21.15.43:0/1066140955 learned_addr learned my addr 172.21.15.43:0/1066140955 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:10.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.669+0000 7f7b009e5640 1 --2- 172.21.15.43:0/1066140955 >> v2:172.21.15.193:3300/0 conn(0x7f7afc0d1560 0x7f7afc0d0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:10.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.669+0000 7f7b011e6640 1 -- 172.21.15.43:0/1066140955 >> v2:172.21.15.193:3300/0 conn(0x7f7afc0d1560 msgr2=0x7f7afc0d0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:10.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.669+0000 7f7b011e6640 1 --2- 172.21.15.43:0/1066140955 >> v2:172.21.15.193:3300/0 conn(0x7f7afc0d1560 0x7f7afc0d0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.669+0000 7f7b011e6640 1 -- 172.21.15.43:0/1066140955 >> v1:172.21.15.43:6789/0 conn(0x7f7afc0ce430 legacy=0x7f7afc0ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:10.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.669+0000 7f7b011e6640 1 -- 172.21.15.43:0/1066140955 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7afc0d2650 con 0x7f7afc0d03a0 2024-04-25T09:03:10.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.669+0000 7f7b011e6640 1 --2- 172.21.15.43:0/1066140955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7afc0d03a0 0x7f7afc0d1f70 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f7afc0d3960 tx=0x7f7af402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e59a9ad87bf2506 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:10.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.669+0000 7f7af3fff640 1 -- 172.21.15.43:0/1066140955 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af4037070 con 0x7f7afc0d03a0 2024-04-25T09:03:10.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.670+0000 7f7af3fff640 1 -- 172.21.15.43:0/1066140955 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af4032440 con 0x7f7afc0d03a0 2024-04-25T09:03:10.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.670+0000 7f7af3fff640 1 -- 172.21.15.43:0/1066140955 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af4032810 con 0x7f7afc0d03a0 2024-04-25T09:03:10.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.670+0000 7f7b0344a640 1 -- 172.21.15.43:0/1066140955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7afc0d03a0 msgr2=0x7f7afc0d1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:10.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.670+0000 7f7b0344a640 1 --2- 172.21.15.43:0/1066140955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7afc0d03a0 0x7f7afc0d1f70 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f7afc0d3960 tx=0x7f7af402a210 comp rx=0 tx=0).stop 2024-04-25T09:03:10.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.670+0000 7f7b0344a640 1 -- 172.21.15.43:0/1066140955 shutdown_connections 2024-04-25T09:03:10.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.670+0000 7f7b0344a640 1 --2- 172.21.15.43:0/1066140955 >> v2:172.21.15.193:3300/0 conn(0x7f7afc0d1560 0x7f7afc0d0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.670+0000 7f7b0344a640 1 --2- 172.21.15.43:0/1066140955 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7afc0d03a0 0x7f7afc0d1f70 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.670+0000 7f7b0344a640 1 -- 172.21.15.43:0/1066140955 >> 172.21.15.43:0/1066140955 conn(0x7f7afc09cd40 msgr2=0x7f7afc0d3990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:10.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.670+0000 7f7b0344a640 1 -- 172.21.15.43:0/1066140955 shutdown_connections 2024-04-25T09:03:10.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.670+0000 7f7b0344a640 1 -- 172.21.15.43:0/1066140955 wait complete. 2024-04-25T09:03:10.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.671+0000 7f7b0344a640 1 Processor -- start 2024-04-25T09:03:10.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.671+0000 7f7b0344a640 1 -- start start 2024-04-25T09:03:10.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.672+0000 7f7b0344a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7afc0ce430 0x7f7afc15c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:10.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.672+0000 7f7b0344a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7afc0d03a0 0x7f7afc1f7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:10.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.672+0000 7f7b0344a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7afc0d1560 0x7f7afc1f4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:10.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.672+0000 7f7b0344a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc15deb0 con 0x7f7afc0d1560 2024-04-25T09:03:10.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.672+0000 7f7b0344a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc0b3400 con 0x7f7afc0ce430 2024-04-25T09:03:10.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.672+0000 7f7b0344a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7afc15e030 con 0x7f7afc0d03a0 2024-04-25T09:03:10.663 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.672+0000 7f7b019e7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7afc0d1560 0x7f7afc1f4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:10.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.672+0000 7f7b019e7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7afc0d1560 0x7f7afc1f4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45122/0 (socket says 172.21.15.43:45122) 2024-04-25T09:03:10.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.672+0000 7f7b019e7640 1 -- 172.21.15.43:0/2785038852 learned_addr learned my addr 172.21.15.43:0/2785038852 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:10.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.672+0000 7f7b009e5640 1 --2- 172.21.15.43:0/2785038852 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7afc0d03a0 0x7f7afc1f7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:10.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.672+0000 7f7b011e6640 1 --2- 172.21.15.43:0/2785038852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7afc0ce430 0x7f7afc15c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:10.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.673+0000 7f7b019e7640 1 -- 172.21.15.43:0/2785038852 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7afc0d03a0 msgr2=0x7f7afc1f7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:10.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.673+0000 7f7b019e7640 1 --2- 172.21.15.43:0/2785038852 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7afc0d03a0 0x7f7afc1f7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.673+0000 7f7b019e7640 1 -- 172.21.15.43:0/2785038852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7afc0ce430 msgr2=0x7f7afc15c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:10.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.673+0000 7f7b019e7640 1 --2- 172.21.15.43:0/2785038852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7afc0ce430 0x7f7afc15c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:10.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.673+0000 7f7b019e7640 1 -- 172.21.15.43:0/2785038852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7afc1f7690 con 0x7f7afc0d1560 2024-04-25T09:03:10.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.673+0000 7f7b011e6640 1 --2- 172.21.15.43:0/2785038852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7afc0ce430 0x7f7afc15c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:10.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.673+0000 7f7b019e7640 1 --2- 172.21.15.43:0/2785038852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7afc0d1560 0x7f7afc1f4390 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f7af800c7a0 tx=0x7f7af800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:10.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.673+0000 7f7af1ffb640 1 -- 172.21.15.43:0/2785038852 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af8003b70 con 0x7f7afc0d1560 2024-04-25T09:03:10.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.673+0000 7f7af1ffb640 1 -- 172.21.15.43:0/2785038852 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af8003d30 con 0x7f7afc0d1560 2024-04-25T09:03:10.664 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.673+0000 7f7b0344a640 1 -- 172.21.15.43:0/2785038852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7afc1f7910 con 0x7f7afc0d1560 2024-04-25T09:03:10.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.673+0000 7f7af1ffb640 1 -- 172.21.15.43:0/2785038852 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af800aa90 con 0x7f7afc0d1560 2024-04-25T09:03:10.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.673+0000 7f7b0344a640 1 -- 172.21.15.43:0/2785038852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7afc1f7ed0 con 0x7f7afc0d1560 2024-04-25T09:03:10.665 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.674+0000 7f7b0344a640 1 -- 172.21.15.43:0/2785038852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7afc0680f0 con 0x7f7afc0d1560 2024-04-25T09:03:10.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.675+0000 7f7af1ffb640 1 -- 172.21.15.43:0/2785038852 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7af800ac30 con 0x7f7afc0d1560 2024-04-25T09:03:10.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.675+0000 7f7af1ffb640 1 --2- 172.21.15.43:0/2785038852 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7ae006b4d0 0x7f7ae006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:10.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.675+0000 7f7af1ffb640 1 -- 172.21.15.43:0/2785038852 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f7af8089f70 con 0x7f7afc0d1560 2024-04-25T09:03:10.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.675+0000 7f7b011e6640 1 --2- 172.21.15.43:0/2785038852 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7ae006b4d0 0x7f7ae006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:10.666 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.676+0000 7f7b011e6640 1 --2- 172.21.15.43:0/2785038852 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7ae006b4d0 0x7f7ae006d990 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f7af4034000 tx=0x7f7af402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:10.668 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.680+0000 7f7af1ffb640 1 -- 172.21.15.43:0/2785038852 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7afc1f7690 con 0x7f7afc0d1560 2024-04-25T09:03:10.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:10.860+0000 7f7b0344a640 1 -- 172.21.15.43:0/2785038852 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f7afc0b1400 con 0x7f7afc0d1560 2024-04-25T09:03:11.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.540+0000 7f7af1ffb640 1 -- 172.21.15.43:0/2785038852 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v124) v1 ==== 170+0+0 (secure 0 0 0) 0x7f7af8058fd0 con 0x7f7afc0d1560 2024-04-25T09:03:11.529 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' created 2024-04-25T09:03:11.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.544+0000 7f7b0344a640 1 -- 172.21.15.43:0/2785038852 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7ae006b4d0 msgr2=0x7f7ae006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:11.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.544+0000 7f7b0344a640 1 --2- 172.21.15.43:0/2785038852 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7ae006b4d0 0x7f7ae006d990 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f7af4034000 tx=0x7f7af402e000 comp rx=0 tx=0).stop 2024-04-25T09:03:11.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.544+0000 7f7b0344a640 1 -- 172.21.15.43:0/2785038852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7afc0d1560 msgr2=0x7f7afc1f4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:11.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.544+0000 7f7b0344a640 1 --2- 172.21.15.43:0/2785038852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7afc0d1560 0x7f7afc1f4390 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f7af800c7a0 tx=0x7f7af800cc60 comp rx=0 tx=0).stop 2024-04-25T09:03:11.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.544+0000 7f7b0344a640 1 -- 172.21.15.43:0/2785038852 shutdown_connections 2024-04-25T09:03:11.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.544+0000 7f7b0344a640 1 --2- 172.21.15.43:0/2785038852 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7afc0d1560 0x7f7afc1f4390 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:11.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.544+0000 7f7b0344a640 1 --2- 172.21.15.43:0/2785038852 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7afc0d03a0 0x7f7afc1f7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:11.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.544+0000 7f7b0344a640 1 --2- 172.21.15.43:0/2785038852 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7ae006b4d0 0x7f7ae006d990 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:11.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.544+0000 7f7b0344a640 1 --2- 172.21.15.43:0/2785038852 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7afc0ce430 0x7f7afc15c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:11.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.544+0000 7f7b0344a640 1 -- 172.21.15.43:0/2785038852 >> 172.21.15.43:0/2785038852 conn(0x7f7afc09cd40 msgr2=0x7f7afc15cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:11.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.544+0000 7f7b0344a640 1 -- 172.21.15.43:0/2785038852 shutdown_connections 2024-04-25T09:03:11.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.545+0000 7f7b0344a640 1 -- 172.21.15.43:0/2785038852 wait complete. 2024-04-25T09:03:11.563 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:03:11.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.685+0000 7ff822b77640 1 Processor -- start 2024-04-25T09:03:11.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.685+0000 7ff822b77640 1 -- start start 2024-04-25T09:03:11.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.685+0000 7ff822b77640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff81c15e090 0x7ff81c161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:11.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.685+0000 7ff822b77640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff81c161fe0 0x7ff81c0b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:11.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.685+0000 7ff822b77640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff81c162610 con 0x7ff81c0b5aa0 2024-04-25T09:03:11.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.686+0000 7ff822b77640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff81c0d1da0 con 0x7ff81c161fe0 2024-04-25T09:03:11.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.686+0000 7ff822b77640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff81c0d1750 con 0x7ff81c15e090 2024-04-25T09:03:11.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.686+0000 7ff81bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff81c161fe0 0x7ff81c0b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:11.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.686+0000 7ff81bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff81c161fe0 0x7ff81c0b5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45134/0 (socket says 172.21.15.43:45134) 2024-04-25T09:03:11.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.686+0000 7ff81bfff640 1 -- 172.21.15.43:0/3776233857 learned_addr learned my addr 172.21.15.43:0/3776233857 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:11.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.686+0000 7ff820913640 1 --2- 172.21.15.43:0/3776233857 >> v2:172.21.15.193:3300/0 conn(0x7ff81c15e090 0x7ff81c161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:11.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.686+0000 7ff81bfff640 1 -- 172.21.15.43:0/3776233857 >> v2:172.21.15.193:3300/0 conn(0x7ff81c15e090 msgr2=0x7ff81c161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:11.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.686+0000 7ff81bfff640 1 --2- 172.21.15.43:0/3776233857 >> v2:172.21.15.193:3300/0 conn(0x7ff81c15e090 0x7ff81c161aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:11.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.686+0000 7ff81bfff640 1 -- 172.21.15.43:0/3776233857 >> v1:172.21.15.193:6789/0 conn(0x7ff81c0b5aa0 legacy=0x7ff81c0ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:11.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.686+0000 7ff81bfff640 1 -- 172.21.15.43:0/3776233857 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff81c0d1a30 con 0x7ff81c161fe0 2024-04-25T09:03:11.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.687+0000 7ff81bfff640 1 --2- 172.21.15.43:0/3776233857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff81c161fe0 0x7ff81c0b5560 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7ff81c062920 tx=0x7ff80802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7991a036a91639d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:11.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.687+0000 7ff81b7fe640 1 -- 172.21.15.43:0/3776233857 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff808037070 con 0x7ff81c161fe0 2024-04-25T09:03:11.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.687+0000 7ff81b7fe640 1 -- 172.21.15.43:0/3776233857 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff80802ae50 con 0x7ff81c161fe0 2024-04-25T09:03:11.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.687+0000 7ff81b7fe640 1 -- 172.21.15.43:0/3776233857 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff808032860 con 0x7ff81c161fe0 2024-04-25T09:03:11.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.687+0000 7ff822b77640 1 -- 172.21.15.43:0/3776233857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff81c161fe0 msgr2=0x7ff81c0b5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:11.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.687+0000 7ff822b77640 1 --2- 172.21.15.43:0/3776233857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff81c161fe0 0x7ff81c0b5560 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7ff81c062920 tx=0x7ff80802a300 comp rx=0 tx=0).stop 2024-04-25T09:03:11.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.687+0000 7ff822b77640 1 -- 172.21.15.43:0/3776233857 shutdown_connections 2024-04-25T09:03:11.677 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.687+0000 7ff822b77640 1 --2- 172.21.15.43:0/3776233857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff81c161fe0 0x7ff81c0b5560 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:11.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.687+0000 7ff822b77640 1 --2- 172.21.15.43:0/3776233857 >> v2:172.21.15.193:3300/0 conn(0x7ff81c15e090 0x7ff81c161aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:11.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.687+0000 7ff822b77640 1 -- 172.21.15.43:0/3776233857 >> 172.21.15.43:0/3776233857 conn(0x7ff81c0d0560 msgr2=0x7ff81c0d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:11.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.688+0000 7ff822b77640 1 -- 172.21.15.43:0/3776233857 shutdown_connections 2024-04-25T09:03:11.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.688+0000 7ff822b77640 1 -- 172.21.15.43:0/3776233857 wait complete. 2024-04-25T09:03:11.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.689+0000 7ff822b77640 1 Processor -- start 2024-04-25T09:03:11.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.689+0000 7ff822b77640 1 -- start start 2024-04-25T09:03:11.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.690+0000 7ff822b77640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff81c0b5aa0 0x7ff81c0ce660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:11.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.690+0000 7ff822b77640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff81c15e090 0x7ff81c15e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:11.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.690+0000 7ff822b77640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff81c161fe0 0x7ff81c15e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:11.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.690+0000 7ff822b77640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c0d3240 con 0x7ff81c161fe0 2024-04-25T09:03:11.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.690+0000 7ff822b77640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c0b3280 con 0x7ff81c15e090 2024-04-25T09:03:11.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.690+0000 7ff822b77640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c0d33c0 con 0x7ff81c0b5aa0 2024-04-25T09:03:11.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.690+0000 7ff821114640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff81c161fe0 0x7ff81c15e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:11.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.690+0000 7ff821114640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff81c161fe0 0x7ff81c15e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45150/0 (socket says 172.21.15.43:45150) 2024-04-25T09:03:11.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.690+0000 7ff821114640 1 -- 172.21.15.43:0/2008889494 learned_addr learned my addr 172.21.15.43:0/2008889494 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:11.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.690+0000 7ff820913640 1 --2- 172.21.15.43:0/2008889494 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff81c0b5aa0 0x7ff81c0ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:11.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.690+0000 7ff81bfff640 1 --2- 172.21.15.43:0/2008889494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff81c15e090 0x7ff81c15e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:11.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.690+0000 7ff821114640 1 -- 172.21.15.43:0/2008889494 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff81c0b5aa0 msgr2=0x7ff81c0ce660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:11.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.690+0000 7ff821114640 1 --2- 172.21.15.43:0/2008889494 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff81c0b5aa0 0x7ff81c0ce660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:11.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.690+0000 7ff821114640 1 -- 172.21.15.43:0/2008889494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff81c15e090 msgr2=0x7ff81c15e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:11.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.690+0000 7ff821114640 1 --2- 172.21.15.43:0/2008889494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff81c15e090 0x7ff81c15e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:11.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.690+0000 7ff821114640 1 -- 172.21.15.43:0/2008889494 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff81c15f050 con 0x7ff81c161fe0 2024-04-25T09:03:11.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.690+0000 7ff821114640 1 --2- 172.21.15.43:0/2008889494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff81c161fe0 0x7ff81c15e9b0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7ff81400c740 tx=0x7ff81400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:11.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.691+0000 7ff8197fa640 1 -- 172.21.15.43:0/2008889494 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff814016020 con 0x7ff81c161fe0 2024-04-25T09:03:11.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.691+0000 7ff8197fa640 1 -- 172.21.15.43:0/2008889494 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81400ce50 con 0x7ff81c161fe0 2024-04-25T09:03:11.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.691+0000 7ff822b77640 1 -- 172.21.15.43:0/2008889494 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff81c15f330 con 0x7ff81c161fe0 2024-04-25T09:03:11.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.691+0000 7ff8197fa640 1 -- 172.21.15.43:0/2008889494 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff814003970 con 0x7ff81c161fe0 2024-04-25T09:03:11.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.691+0000 7ff822b77640 1 -- 172.21.15.43:0/2008889494 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff81c15f8f0 con 0x7ff81c161fe0 2024-04-25T09:03:11.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.692+0000 7ff822b77640 1 -- 172.21.15.43:0/2008889494 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff81c06bf00 con 0x7ff81c161fe0 2024-04-25T09:03:11.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.692+0000 7ff8197fa640 1 -- 172.21.15.43:0/2008889494 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff814003b10 con 0x7ff81c161fe0 2024-04-25T09:03:11.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.693+0000 7ff8197fa640 1 --2- 172.21.15.43:0/2008889494 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff80006b4d0 0x7ff80006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:11.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.693+0000 7ff8197fa640 1 -- 172.21.15.43:0/2008889494 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5609+0+0 (secure 0 0 0) 0x7ff814089c60 con 0x7ff81c161fe0 2024-04-25T09:03:11.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.693+0000 7ff820913640 1 --2- 172.21.15.43:0/2008889494 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff80006b4d0 0x7ff80006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:11.682 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.693+0000 7ff820913640 1 --2- 172.21.15.43:0/2008889494 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff80006b4d0 0x7ff80006d990 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff810006fd0 tx=0x7ff810008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:11.685 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.697+0000 7ff8197fa640 1 -- 172.21.15.43:0/2008889494 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff81c06bf00 con 0x7ff81c161fe0 2024-04-25T09:03:11.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:11.917+0000 7ff822b77640 1 -- 172.21.15.43:0/2008889494 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff81c074f60 con 0x7ff81c161fe0 2024-04-25T09:03:12.558 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.570+0000 7ff8197fa640 1 -- 172.21.15.43:0/2008889494 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v74) v1 ==== 174+0+0 (secure 0 0 0) 0x7ff814020020 con 0x7ff81c161fe0 2024-04-25T09:03:12.562 INFO:teuthology.orchestra.run.smithi043.stderr:new fs with metadata pool 18 and data pool 19 2024-04-25T09:03:12.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.578+0000 7ff822b77640 1 -- 172.21.15.43:0/2008889494 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff80006b4d0 msgr2=0x7ff80006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:12.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.578+0000 7ff822b77640 1 --2- 172.21.15.43:0/2008889494 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff80006b4d0 0x7ff80006d990 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff810006fd0 tx=0x7ff810008040 comp rx=0 tx=0).stop 2024-04-25T09:03:12.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.578+0000 7ff822b77640 1 -- 172.21.15.43:0/2008889494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff81c161fe0 msgr2=0x7ff81c15e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:12.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.578+0000 7ff822b77640 1 --2- 172.21.15.43:0/2008889494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff81c161fe0 0x7ff81c15e9b0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7ff81400c740 tx=0x7ff81400cc00 comp rx=0 tx=0).stop 2024-04-25T09:03:12.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.580+0000 7ff822b77640 1 -- 172.21.15.43:0/2008889494 shutdown_connections 2024-04-25T09:03:12.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.580+0000 7ff822b77640 1 --2- 172.21.15.43:0/2008889494 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff81c161fe0 0x7ff81c15e9b0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:12.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.580+0000 7ff822b77640 1 --2- 172.21.15.43:0/2008889494 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff80006b4d0 0x7ff80006d990 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:12.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.580+0000 7ff822b77640 1 --2- 172.21.15.43:0/2008889494 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff81c15e090 0x7ff81c15e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:12.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.580+0000 7ff822b77640 1 --2- 172.21.15.43:0/2008889494 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff81c0b5aa0 0x7ff81c0ce660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:12.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.580+0000 7ff822b77640 1 -- 172.21.15.43:0/2008889494 >> 172.21.15.43:0/2008889494 conn(0x7ff81c0d0560 msgr2=0x7ff81c0b7b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:12.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.581+0000 7ff822b77640 1 -- 172.21.15.43:0/2008889494 shutdown_connections 2024-04-25T09:03:12.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.581+0000 7ff822b77640 1 -- 172.21.15.43:0/2008889494 wait complete. 2024-04-25T09:03:12.585 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:12.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.699+0000 7f06c4f09640 1 Processor -- start 2024-04-25T09:03:12.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.699+0000 7f06c4f09640 1 -- start start 2024-04-25T09:03:12.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.700+0000 7f06c4f09640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f06c015a9d0 0x7f06c006d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:12.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.700+0000 7f06c4f09640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f06c006d7d0 0x7f06c0071a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:12.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.700+0000 7f06c4f09640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06c0072400 con 0x7f06c0071f90 2024-04-25T09:03:12.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.700+0000 7f06c4f09640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06c006dc30 con 0x7f06c006d7d0 2024-04-25T09:03:12.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.700+0000 7f06c4f09640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06c00704f0 con 0x7f06c015a9d0 2024-04-25T09:03:12.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.700+0000 7f06bdd9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f06c006d7d0 0x7f06c0071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:12.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.700+0000 7f06bdd9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f06c006d7d0 0x7f06c0071a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45164/0 (socket says 172.21.15.43:45164) 2024-04-25T09:03:12.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.700+0000 7f06bdd9b640 1 -- 172.21.15.43:0/2937812374 learned_addr learned my addr 172.21.15.43:0/2937812374 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:12.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.700+0000 7f06be59c640 1 --2- 172.21.15.43:0/2937812374 >> v2:172.21.15.193:3300/0 conn(0x7f06c015a9d0 0x7f06c006d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:12.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.700+0000 7f06bdd9b640 1 -- 172.21.15.43:0/2937812374 >> v2:172.21.15.193:3300/0 conn(0x7f06c015a9d0 msgr2=0x7f06c006d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:12.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.700+0000 7f06bdd9b640 1 --2- 172.21.15.43:0/2937812374 >> v2:172.21.15.193:3300/0 conn(0x7f06c015a9d0 0x7f06c006d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:12.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.700+0000 7f06bdd9b640 1 -- 172.21.15.43:0/2937812374 >> v1:172.21.15.193:6789/0 conn(0x7f06c0071f90 legacy=0x7f06c00b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:12.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.701+0000 7f06bdd9b640 1 -- 172.21.15.43:0/2937812374 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c00707d0 con 0x7f06c006d7d0 2024-04-25T09:03:12.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.701+0000 7f06bdd9b640 1 --2- 172.21.15.43:0/2937812374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06c006d7d0 0x7f06c0071a50 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f06c006c190 tx=0x7f06a802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cc6cbacabb0d697 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:12.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.701+0000 7f06bd59a640 1 -- 172.21.15.43:0/2937812374 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a8037070 con 0x7f06c006d7d0 2024-04-25T09:03:12.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.701+0000 7f06bd59a640 1 -- 172.21.15.43:0/2937812374 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a8032440 con 0x7f06c006d7d0 2024-04-25T09:03:12.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.701+0000 7f06bd59a640 1 -- 172.21.15.43:0/2937812374 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a8032810 con 0x7f06c006d7d0 2024-04-25T09:03:12.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.702+0000 7f06c4f09640 1 -- 172.21.15.43:0/2937812374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06c006d7d0 msgr2=0x7f06c0071a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:12.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.702+0000 7f06c4f09640 1 --2- 172.21.15.43:0/2937812374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06c006d7d0 0x7f06c0071a50 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f06c006c190 tx=0x7f06a802a270 comp rx=0 tx=0).stop 2024-04-25T09:03:12.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.702+0000 7f06c4f09640 1 -- 172.21.15.43:0/2937812374 shutdown_connections 2024-04-25T09:03:12.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.702+0000 7f06c4f09640 1 --2- 172.21.15.43:0/2937812374 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06c006d7d0 0x7f06c0071a50 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:12.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.702+0000 7f06c4f09640 1 --2- 172.21.15.43:0/2937812374 >> v2:172.21.15.193:3300/0 conn(0x7f06c015a9d0 0x7f06c006d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:12.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.702+0000 7f06c4f09640 1 -- 172.21.15.43:0/2937812374 >> 172.21.15.43:0/2937812374 conn(0x7f06c00b1480 msgr2=0x7f06c00b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:12.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.702+0000 7f06c4f09640 1 -- 172.21.15.43:0/2937812374 shutdown_connections 2024-04-25T09:03:12.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.702+0000 7f06c4f09640 1 -- 172.21.15.43:0/2937812374 wait complete. 2024-04-25T09:03:12.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.703+0000 7f06c4f09640 1 Processor -- start 2024-04-25T09:03:12.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.703+0000 7f06c4f09640 1 -- start start 2024-04-25T09:03:12.694 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.704+0000 7f06c4f09640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06c006d7d0 0x7f06c01e39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:12.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.704+0000 7f06c4f09640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f06c0071f90 0x7f06c0151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:12.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.704+0000 7f06c4f09640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06c01421b0 0x7f06c01456a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:12.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.704+0000 7f06c4f09640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c00712b0 con 0x7f06c01421b0 2024-04-25T09:03:12.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.704+0000 7f06c4f09640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c0070d90 con 0x7f06c006d7d0 2024-04-25T09:03:12.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.704+0000 7f06c4f09640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06c00ce950 con 0x7f06c0071f90 2024-04-25T09:03:12.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.704+0000 7f06bed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06c01421b0 0x7f06c01456a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:12.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.704+0000 7f06bed9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06c01421b0 0x7f06c01456a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45172/0 (socket says 172.21.15.43:45172) 2024-04-25T09:03:12.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.704+0000 7f06bed9d640 1 -- 172.21.15.43:0/2968690353 learned_addr learned my addr 172.21.15.43:0/2968690353 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:12.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.705+0000 7f06bdd9b640 1 --2- 172.21.15.43:0/2968690353 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f06c0071f90 0x7f06c0151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:12.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.705+0000 7f06be59c640 1 --2- 172.21.15.43:0/2968690353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06c006d7d0 0x7f06c01e39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:12.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.705+0000 7f06bed9d640 1 -- 172.21.15.43:0/2968690353 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f06c0071f90 msgr2=0x7f06c0151f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:12.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.705+0000 7f06bed9d640 1 --2- 172.21.15.43:0/2968690353 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f06c0071f90 0x7f06c0151f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:12.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.705+0000 7f06bed9d640 1 -- 172.21.15.43:0/2968690353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06c006d7d0 msgr2=0x7f06c01e39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:12.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.705+0000 7f06bed9d640 1 --2- 172.21.15.43:0/2968690353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06c006d7d0 0x7f06c01e39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:12.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.705+0000 7f06bed9d640 1 -- 172.21.15.43:0/2968690353 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c01426b0 con 0x7f06c01421b0 2024-04-25T09:03:12.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.705+0000 7f06bed9d640 1 --2- 172.21.15.43:0/2968690353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06c01421b0 0x7f06c01456a0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f06b000c730 tx=0x7f06b000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:12.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.705+0000 7f06b6ffd640 1 -- 172.21.15.43:0/2968690353 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b0003b70 con 0x7f06c01421b0 2024-04-25T09:03:12.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.705+0000 7f06b6ffd640 1 -- 172.21.15.43:0/2968690353 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b0003d30 con 0x7f06c01421b0 2024-04-25T09:03:12.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.705+0000 7f06c4f09640 1 -- 172.21.15.43:0/2968690353 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06c0142990 con 0x7f06c01421b0 2024-04-25T09:03:12.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.705+0000 7f06b6ffd640 1 -- 172.21.15.43:0/2968690353 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b0005310 con 0x7f06c01421b0 2024-04-25T09:03:12.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.705+0000 7f06c4f09640 1 -- 172.21.15.43:0/2968690353 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06c0142f50 con 0x7f06c01421b0 2024-04-25T09:03:12.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.706+0000 7f06c4f09640 1 -- 172.21.15.43:0/2968690353 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06c0074ef0 con 0x7f06c01421b0 2024-04-25T09:03:12.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.707+0000 7f06b6ffd640 1 -- 172.21.15.43:0/2968690353 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f06b00054b0 con 0x7f06c01421b0 2024-04-25T09:03:12.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.707+0000 7f06b6ffd640 1 --2- 172.21.15.43:0/2968690353 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f06a006b4d0 0x7f06a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:12.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.707+0000 7f06b6ffd640 1 -- 172.21.15.43:0/2968690353 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f06b008a080 con 0x7f06c01421b0 2024-04-25T09:03:12.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.707+0000 7f06be59c640 1 --2- 172.21.15.43:0/2968690353 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f06a006b4d0 0x7f06a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:12.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.708+0000 7f06be59c640 1 --2- 172.21.15.43:0/2968690353 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f06a006b4d0 0x7f06a006d990 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f06b8006fd0 tx=0x7f06b8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:12.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.712+0000 7f06b6ffd640 1 -- 172.21.15.43:0/2968690353 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f06b0058f00 con 0x7f06c01421b0 2024-04-25T09:03:12.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.896+0000 7f06c4f09640 1 -- 172.21.15.43:0/2968690353 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06c01e3f20 con 0x7f06c01421b0 2024-04-25T09:03:12.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.897+0000 7f06b6ffd640 1 -- 172.21.15.43:0/2968690353 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+16656 (secure 0 0 0) 0x7f06b0058800 con 0x7f06c01421b0 2024-04-25T09:03:12.886 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:03:12.886 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":125,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:03:11.918946+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:03:10.199822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T09:03:10.892664+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:12.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.899+0000 7f06c4f09640 1 -- 172.21.15.43:0/2968690353 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f06a006b4d0 msgr2=0x7f06a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:12.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.899+0000 7f06c4f09640 1 --2- 172.21.15.43:0/2968690353 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f06a006b4d0 0x7f06a006d990 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f06b8006fd0 tx=0x7f06b8008040 comp rx=0 tx=0).stop 2024-04-25T09:03:12.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.899+0000 7f06c4f09640 1 -- 172.21.15.43:0/2968690353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06c01421b0 msgr2=0x7f06c01456a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:12.888 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.899+0000 7f06c4f09640 1 --2- 172.21.15.43:0/2968690353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06c01421b0 0x7f06c01456a0 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f06b000c730 tx=0x7f06b000cbf0 comp rx=0 tx=0).stop 2024-04-25T09:03:12.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.899+0000 7f06c4f09640 1 -- 172.21.15.43:0/2968690353 shutdown_connections 2024-04-25T09:03:12.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.899+0000 7f06c4f09640 1 --2- 172.21.15.43:0/2968690353 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f06c01421b0 0x7f06c01456a0 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:12.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.899+0000 7f06c4f09640 1 --2- 172.21.15.43:0/2968690353 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f06c0071f90 0x7f06c0151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:12.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.899+0000 7f06c4f09640 1 --2- 172.21.15.43:0/2968690353 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f06a006b4d0 0x7f06a006d990 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:12.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.899+0000 7f06c4f09640 1 --2- 172.21.15.43:0/2968690353 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f06c006d7d0 0x7f06c01e39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:12.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.899+0000 7f06c4f09640 1 -- 172.21.15.43:0/2968690353 >> 172.21.15.43:0/2968690353 conn(0x7f06c00b1480 msgr2=0x7f06c01586c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:12.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.899+0000 7f06c4f09640 1 -- 172.21.15.43:0/2968690353 shutdown_connections 2024-04-25T09:03:12.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:12.899+0000 7f06c4f09640 1 -- 172.21.15.43:0/2968690353 wait complete. 2024-04-25T09:03:12.900 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:13.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.011+0000 7f71b84f5640 1 Processor -- start 2024-04-25T09:03:13.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.011+0000 7f71b84f5640 1 -- start start 2024-04-25T09:03:13.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.012+0000 7f71b84f5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f71b015df50 0x7f71b01619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:13.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.012+0000 7f71b84f5640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f71b0161f30 0x7f71b0162380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:13.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.012+0000 7f71b84f5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71b00d24a0 con 0x7f71b00c7cf0 2024-04-25T09:03:13.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.012+0000 7f71b84f5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71b015e330 con 0x7f71b015df50 2024-04-25T09:03:13.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.012+0000 7f71b84f5640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71b00d2740 con 0x7f71b0161f30 2024-04-25T09:03:13.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.012+0000 7f71b6291640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f71b015df50 0x7f71b01619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:13.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.013+0000 7f71b6291640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f71b015df50 0x7f71b01619f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45178/0 (socket says 172.21.15.43:45178) 2024-04-25T09:03:13.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.013+0000 7f71b6291640 1 -- 172.21.15.43:0/2981619471 learned_addr learned my addr 172.21.15.43:0/2981619471 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:13.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.013+0000 7f71b5a90640 1 --2- 172.21.15.43:0/2981619471 >> v2:172.21.15.193:3300/0 conn(0x7f71b0161f30 0x7f71b0162380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:13.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.013+0000 7f71b6291640 1 -- 172.21.15.43:0/2981619471 >> v2:172.21.15.193:3300/0 conn(0x7f71b0161f30 msgr2=0x7f71b0162380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:13.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.013+0000 7f71b6291640 1 --2- 172.21.15.43:0/2981619471 >> v2:172.21.15.193:3300/0 conn(0x7f71b0161f30 0x7f71b0162380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.013+0000 7f71b6291640 1 -- 172.21.15.43:0/2981619471 >> v1:172.21.15.43:6789/0 conn(0x7f71b00c7cf0 legacy=0x7f71b00c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:13.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.013+0000 7f71b6291640 1 -- 172.21.15.43:0/2981619471 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b00d2a20 con 0x7f71b015df50 2024-04-25T09:03:13.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.013+0000 7f71b6291640 1 --2- 172.21.15.43:0/2981619471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71b015df50 0x7f71b01619f0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f71b00d45e0 tx=0x7f71ac02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ef1993513654e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:13.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.014+0000 7f71b528f640 1 -- 172.21.15.43:0/2981619471 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ac037070 con 0x7f71b015df50 2024-04-25T09:03:13.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.014+0000 7f71b528f640 1 -- 172.21.15.43:0/2981619471 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71ac032440 con 0x7f71b015df50 2024-04-25T09:03:13.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.014+0000 7f71b528f640 1 -- 172.21.15.43:0/2981619471 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ac032810 con 0x7f71b015df50 2024-04-25T09:03:13.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.014+0000 7f71b84f5640 1 -- 172.21.15.43:0/2981619471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71b015df50 msgr2=0x7f71b01619f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:13.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.014+0000 7f71b84f5640 1 --2- 172.21.15.43:0/2981619471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71b015df50 0x7f71b01619f0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f71b00d45e0 tx=0x7f71ac02a270 comp rx=0 tx=0).stop 2024-04-25T09:03:13.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.014+0000 7f71b84f5640 1 -- 172.21.15.43:0/2981619471 shutdown_connections 2024-04-25T09:03:13.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.014+0000 7f71b84f5640 1 --2- 172.21.15.43:0/2981619471 >> v2:172.21.15.193:3300/0 conn(0x7f71b0161f30 0x7f71b0162380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.014+0000 7f71b84f5640 1 --2- 172.21.15.43:0/2981619471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71b015df50 0x7f71b01619f0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.014+0000 7f71b84f5640 1 -- 172.21.15.43:0/2981619471 >> 172.21.15.43:0/2981619471 conn(0x7f71b00d3a30 msgr2=0x7f71b00d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:13.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.014+0000 7f71b84f5640 1 -- 172.21.15.43:0/2981619471 shutdown_connections 2024-04-25T09:03:13.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.014+0000 7f71b84f5640 1 -- 172.21.15.43:0/2981619471 wait complete. 2024-04-25T09:03:13.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.015+0000 7f71b84f5640 1 Processor -- start 2024-04-25T09:03:13.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.016+0000 7f71b84f5640 1 -- start start 2024-04-25T09:03:13.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.016+0000 7f71b84f5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71b00c7cf0 0x7f71b009d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:13.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.016+0000 7f71b84f5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f71b015df50 0x7f71b009e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:13.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.016+0000 7f71b84f5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71b0161f30 0x7f71b009fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:13.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.016+0000 7f71b84f5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b00d2190 con 0x7f71b00c7cf0 2024-04-25T09:03:13.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.016+0000 7f71b84f5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b00c77d0 con 0x7f71b0161f30 2024-04-25T09:03:13.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.016+0000 7f71b84f5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71b00cf670 con 0x7f71b015df50 2024-04-25T09:03:13.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.016+0000 7f71b6291640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71b00c7cf0 0x7f71b009d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:13.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.016+0000 7f71b6291640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71b00c7cf0 0x7f71b009d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45180/0 (socket says 172.21.15.43:45180) 2024-04-25T09:03:13.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.016+0000 7f71b6291640 1 -- 172.21.15.43:0/1152616505 learned_addr learned my addr 172.21.15.43:0/1152616505 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:13.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.017+0000 7f71b5a90640 1 --2- 172.21.15.43:0/1152616505 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f71b015df50 0x7f71b009e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:13.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.017+0000 7f71b6a92640 1 --2- 172.21.15.43:0/1152616505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71b0161f30 0x7f71b009fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:13.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.017+0000 7f71b6291640 1 -- 172.21.15.43:0/1152616505 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f71b015df50 msgr2=0x7f71b009e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:13.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.017+0000 7f71b6291640 1 --2- 172.21.15.43:0/1152616505 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f71b015df50 0x7f71b009e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.017+0000 7f71b6291640 1 -- 172.21.15.43:0/1152616505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71b0161f30 msgr2=0x7f71b009fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:13.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.017+0000 7f71b6291640 1 --2- 172.21.15.43:0/1152616505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71b0161f30 0x7f71b009fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.017+0000 7f71b6291640 1 -- 172.21.15.43:0/1152616505 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b009ec50 con 0x7f71b00c7cf0 2024-04-25T09:03:13.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.017+0000 7f71b6291640 1 --2- 172.21.15.43:0/1152616505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71b00c7cf0 0x7f71b009d770 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f71ac02a2b0 tx=0x7f71ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:13.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.017+0000 7f71a6ffd640 1 -- 172.21.15.43:0/1152616505 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ac041020 con 0x7f71b00c7cf0 2024-04-25T09:03:13.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.017+0000 7f71a6ffd640 1 -- 172.21.15.43:0/1152616505 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71ac0306c0 con 0x7f71b00c7cf0 2024-04-25T09:03:13.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.017+0000 7f71b84f5640 1 -- 172.21.15.43:0/1152616505 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71b015e740 con 0x7f71b00c7cf0 2024-04-25T09:03:13.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.017+0000 7f71a6ffd640 1 -- 172.21.15.43:0/1152616505 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71ac037040 con 0x7f71b00c7cf0 2024-04-25T09:03:13.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.017+0000 7f71b84f5640 1 -- 172.21.15.43:0/1152616505 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b015ed00 con 0x7f71b00c7cf0 2024-04-25T09:03:13.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.018+0000 7f71b84f5640 1 -- 172.21.15.43:0/1152616505 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71b00680f0 con 0x7f71b00c7cf0 2024-04-25T09:03:13.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.019+0000 7f71a6ffd640 1 -- 172.21.15.43:0/1152616505 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f71ac044020 con 0x7f71b00c7cf0 2024-04-25T09:03:13.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.019+0000 7f71a6ffd640 1 --2- 172.21.15.43:0/1152616505 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f719406b4d0 0x7f719406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:13.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.019+0000 7f71a6ffd640 1 -- 172.21.15.43:0/1152616505 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f71ac0aaa00 con 0x7f71b00c7cf0 2024-04-25T09:03:13.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.019+0000 7f71b5a90640 1 --2- 172.21.15.43:0/1152616505 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f719406b4d0 0x7f719406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:13.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.020+0000 7f71b5a90640 1 --2- 172.21.15.43:0/1152616505 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f719406b4d0 0x7f719406d990 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f71a0006fd0 tx=0x7f71a0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:13.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.023+0000 7f71a6ffd640 1 -- 172.21.15.43:0/1152616505 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f71ac04b030 con 0x7f71b00c7cf0 2024-04-25T09:03:13.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.206+0000 7f71b84f5640 1 -- 172.21.15.43:0/1152616505 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71b0074ef0 con 0x7f71b00c7cf0 2024-04-25T09:03:13.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.207+0000 7f71a6ffd640 1 -- 172.21.15.43:0/1152616505 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+16656 (secure 0 0 0) 0x7f71ac079880 con 0x7f71b00c7cf0 2024-04-25T09:03:13.196 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:03:13.196 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":125,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:03:11.918946+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:03:10.199822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T09:03:10.892664+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:13.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.209+0000 7f71b84f5640 1 -- 172.21.15.43:0/1152616505 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f719406b4d0 msgr2=0x7f719406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:13.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.209+0000 7f71b84f5640 1 --2- 172.21.15.43:0/1152616505 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f719406b4d0 0x7f719406d990 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f71a0006fd0 tx=0x7f71a0008040 comp rx=0 tx=0).stop 2024-04-25T09:03:13.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.209+0000 7f71b84f5640 1 -- 172.21.15.43:0/1152616505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71b00c7cf0 msgr2=0x7f71b009d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:13.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.209+0000 7f71b84f5640 1 --2- 172.21.15.43:0/1152616505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71b00c7cf0 0x7f71b009d770 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f71ac02a2b0 tx=0x7f71ac0046c0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.209+0000 7f71b84f5640 1 -- 172.21.15.43:0/1152616505 shutdown_connections 2024-04-25T09:03:13.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.209+0000 7f71b84f5640 1 --2- 172.21.15.43:0/1152616505 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f719406b4d0 0x7f719406d990 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.209+0000 7f71b84f5640 1 --2- 172.21.15.43:0/1152616505 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f71b0161f30 0x7f71b009fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.209+0000 7f71b84f5640 1 --2- 172.21.15.43:0/1152616505 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f71b015df50 0x7f71b009e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.209+0000 7f71b84f5640 1 --2- 172.21.15.43:0/1152616505 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f71b00c7cf0 0x7f71b009d770 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.209+0000 7f71b84f5640 1 -- 172.21.15.43:0/1152616505 >> 172.21.15.43:0/1152616505 conn(0x7f71b00d3a30 msgr2=0x7f71b00cd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:13.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.209+0000 7f71b84f5640 1 -- 172.21.15.43:0/1152616505 shutdown_connections 2024-04-25T09:03:13.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.209+0000 7f71b84f5640 1 -- 172.21.15.43:0/1152616505 wait complete. 2024-04-25T09:03:13.209 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:03:13.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.325+0000 7f2971c1f640 1 Processor -- start 2024-04-25T09:03:13.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.325+0000 7f2971c1f640 1 -- start start 2024-04-25T09:03:13.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.326+0000 7f2971c1f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f296c0d04a0 0x7f296c15ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:13.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.326+0000 7f2971c1f640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f296c15dec0 0x7f296c0d0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:13.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.326+0000 7f2971c1f640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f296c0d0dc0 con 0x7f296c0ce8c0 2024-04-25T09:03:13.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.326+0000 7f2971c1f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f296c0cf110 con 0x7f296c0d04a0 2024-04-25T09:03:13.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.326+0000 7f2971c1f640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f296c0cf3f0 con 0x7f296c15dec0 2024-04-25T09:03:13.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.326+0000 7f296b7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f296c0d04a0 0x7f296c15ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:13.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.326+0000 7f296b7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f296c0d04a0 0x7f296c15ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45188/0 (socket says 172.21.15.43:45188) 2024-04-25T09:03:13.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.326+0000 7f296b7fe640 1 -- 172.21.15.43:0/2469282343 learned_addr learned my addr 172.21.15.43:0/2469282343 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:13.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.326+0000 7f296affd640 1 --2- 172.21.15.43:0/2469282343 >> v2:172.21.15.193:3300/0 conn(0x7f296c15dec0 0x7f296c0d0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:13.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.326+0000 7f296b7fe640 1 -- 172.21.15.43:0/2469282343 >> v2:172.21.15.193:3300/0 conn(0x7f296c15dec0 msgr2=0x7f296c0d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:13.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.326+0000 7f296b7fe640 1 --2- 172.21.15.43:0/2469282343 >> v2:172.21.15.193:3300/0 conn(0x7f296c15dec0 0x7f296c0d0880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.326+0000 7f296b7fe640 1 -- 172.21.15.43:0/2469282343 >> v1:172.21.15.193:6789/0 conn(0x7f296c0ce8c0 legacy=0x7f296c1617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:13.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.326+0000 7f296b7fe640 1 -- 172.21.15.43:0/2469282343 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f296c0cf6d0 con 0x7f296c0d04a0 2024-04-25T09:03:13.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.327+0000 7f296b7fe640 1 --2- 172.21.15.43:0/2469282343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f296c0d04a0 0x7f296c15ed90 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f296c0d0070 tx=0x7f295402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a0474358cd847e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:13.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.327+0000 7f296a7fc640 1 -- 172.21.15.43:0/2469282343 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2954037070 con 0x7f296c0d04a0 2024-04-25T09:03:13.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.327+0000 7f296a7fc640 1 -- 172.21.15.43:0/2469282343 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2954032440 con 0x7f296c0d04a0 2024-04-25T09:03:13.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.327+0000 7f296a7fc640 1 -- 172.21.15.43:0/2469282343 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2954032810 con 0x7f296c0d04a0 2024-04-25T09:03:13.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.327+0000 7f2971c1f640 1 -- 172.21.15.43:0/2469282343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f296c0d04a0 msgr2=0x7f296c15ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:13.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.327+0000 7f2971c1f640 1 --2- 172.21.15.43:0/2469282343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f296c0d04a0 0x7f296c15ed90 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f296c0d0070 tx=0x7f295402a230 comp rx=0 tx=0).stop 2024-04-25T09:03:13.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.328+0000 7f2971c1f640 1 -- 172.21.15.43:0/2469282343 shutdown_connections 2024-04-25T09:03:13.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.328+0000 7f2971c1f640 1 --2- 172.21.15.43:0/2469282343 >> v2:172.21.15.193:3300/0 conn(0x7f296c15dec0 0x7f296c0d0880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.328+0000 7f2971c1f640 1 --2- 172.21.15.43:0/2469282343 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f296c0d04a0 0x7f296c15ed90 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.328+0000 7f2971c1f640 1 -- 172.21.15.43:0/2469282343 >> 172.21.15.43:0/2469282343 conn(0x7f296c09cd40 msgr2=0x7f296c0d3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:13.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.328+0000 7f2971c1f640 1 -- 172.21.15.43:0/2469282343 shutdown_connections 2024-04-25T09:03:13.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.328+0000 7f2971c1f640 1 -- 172.21.15.43:0/2469282343 wait complete. 2024-04-25T09:03:13.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.329+0000 7f2971c1f640 1 Processor -- start 2024-04-25T09:03:13.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.329+0000 7f2971c1f640 1 -- start start 2024-04-25T09:03:13.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.330+0000 7f2971c1f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f296c0ce8c0 0x7f296c0b6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:13.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.330+0000 7f2971c1f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f296c0d04a0 0x7f296c1f7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:13.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.330+0000 7f2971c1f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f296c15dec0 0x7f296c1f6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:13.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.330+0000 7f2971c1f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296c0ced10 con 0x7f296c0ce8c0 2024-04-25T09:03:13.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.330+0000 7f2971c1f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f296c0cfc90 con 0x7f296c0d04a0 2024-04-25T09:03:13.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.330+0000 7f2971c1f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f296c0cee90 con 0x7f296c15dec0 2024-04-25T09:03:13.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.330+0000 7f296b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f296c0ce8c0 0x7f296c0b6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:13.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.330+0000 7f296b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f296c0ce8c0 0x7f296c0b6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45196/0 (socket says 172.21.15.43:45196) 2024-04-25T09:03:13.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.330+0000 7f296b7fe640 1 -- 172.21.15.43:0/1850724842 learned_addr learned my addr 172.21.15.43:0/1850724842 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:13.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.330+0000 7f296affd640 1 --2- 172.21.15.43:0/1850724842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f296c0d04a0 0x7f296c1f7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:13.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.330+0000 7f296bfff640 1 --2- 172.21.15.43:0/1850724842 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f296c15dec0 0x7f296c1f6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:13.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.330+0000 7f296b7fe640 1 -- 172.21.15.43:0/1850724842 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f296c15dec0 msgr2=0x7f296c1f6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:13.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.330+0000 7f296b7fe640 1 --2- 172.21.15.43:0/1850724842 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f296c15dec0 0x7f296c1f6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.330+0000 7f296b7fe640 1 -- 172.21.15.43:0/1850724842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f296c0d04a0 msgr2=0x7f296c1f7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:13.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.330+0000 7f296b7fe640 1 --2- 172.21.15.43:0/1850724842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f296c0d04a0 0x7f296c1f7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.330+0000 7f296b7fe640 1 -- 172.21.15.43:0/1850724842 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f296c0b6d50 con 0x7f296c0ce8c0 2024-04-25T09:03:13.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.330+0000 7f296bfff640 1 --2- 172.21.15.43:0/1850724842 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f296c15dec0 0x7f296c1f6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:13.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.330+0000 7f296b7fe640 1 --2- 172.21.15.43:0/1850724842 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f296c0ce8c0 0x7f296c0b6690 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f2954000c00 tx=0x7f29540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:13.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.330+0000 7f2963fff640 1 -- 172.21.15.43:0/1850724842 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2954041020 con 0x7f296c0ce8c0 2024-04-25T09:03:13.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.330+0000 7f2963fff640 1 -- 172.21.15.43:0/1850724842 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29540306c0 con 0x7f296c0ce8c0 2024-04-25T09:03:13.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.330+0000 7f2963fff640 1 -- 172.21.15.43:0/1850724842 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2954037040 con 0x7f296c0ce8c0 2024-04-25T09:03:13.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.331+0000 7f2971c1f640 1 -- 172.21.15.43:0/1850724842 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f296c0b7030 con 0x7f296c0ce8c0 2024-04-25T09:03:13.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.331+0000 7f2971c1f640 1 -- 172.21.15.43:0/1850724842 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f296c0b75f0 con 0x7f296c0ce8c0 2024-04-25T09:03:13.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.331+0000 7f2971c1f640 1 -- 172.21.15.43:0/1850724842 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f296c074f60 con 0x7f296c0ce8c0 2024-04-25T09:03:13.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.332+0000 7f2963fff640 1 -- 172.21.15.43:0/1850724842 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2954044020 con 0x7f296c0ce8c0 2024-04-25T09:03:13.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.332+0000 7f2963fff640 1 --2- 172.21.15.43:0/1850724842 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f294c06b4d0 0x7f294c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:13.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.332+0000 7f2963fff640 1 -- 172.21.15.43:0/1850724842 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f29540aaa30 con 0x7f296c0ce8c0 2024-04-25T09:03:13.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.332+0000 7f296affd640 1 --2- 172.21.15.43:0/1850724842 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f294c06b4d0 0x7f294c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:13.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.333+0000 7f296affd640 1 --2- 172.21.15.43:0/1850724842 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f294c06b4d0 0x7f294c06d990 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f296c151770 tx=0x7f295c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:13.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.336+0000 7f2963fff640 1 -- 172.21.15.43:0/1850724842 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f295404b030 con 0x7f296c0ce8c0 2024-04-25T09:03:13.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.553+0000 7f2971c1f640 1 -- 172.21.15.43:0/1850724842 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f296c08daf0 con 0x7f296c0ce8c0 2024-04-25T09:03:13.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.565+0000 7f2963fff640 1 -- 172.21.15.43:0/1850724842 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v75) v1 ==== 118+0+0 (secure 0 0 0) 0x7f29540798b0 con 0x7f296c0ce8c0 2024-04-25T09:03:13.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.567+0000 7f2971c1f640 1 -- 172.21.15.43:0/1850724842 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f294c06b4d0 msgr2=0x7f294c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:13.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.567+0000 7f2971c1f640 1 --2- 172.21.15.43:0/1850724842 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f294c06b4d0 0x7f294c06d990 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f296c151770 tx=0x7f295c008040 comp rx=0 tx=0).stop 2024-04-25T09:03:13.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.567+0000 7f2971c1f640 1 -- 172.21.15.43:0/1850724842 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f296c0ce8c0 msgr2=0x7f296c0b6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:13.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.567+0000 7f2971c1f640 1 --2- 172.21.15.43:0/1850724842 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f296c0ce8c0 0x7f296c0b6690 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f2954000c00 tx=0x7f29540046c0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.567+0000 7f2971c1f640 1 -- 172.21.15.43:0/1850724842 shutdown_connections 2024-04-25T09:03:13.556 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.567+0000 7f2971c1f640 1 --2- 172.21.15.43:0/1850724842 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f296c15dec0 0x7f296c1f6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.567+0000 7f2971c1f640 1 --2- 172.21.15.43:0/1850724842 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f294c06b4d0 0x7f294c06d990 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.567+0000 7f2971c1f640 1 --2- 172.21.15.43:0/1850724842 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f296c0d04a0 0x7f296c1f7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.567+0000 7f2971c1f640 1 --2- 172.21.15.43:0/1850724842 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f296c0ce8c0 0x7f296c0b6690 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.567+0000 7f2971c1f640 1 -- 172.21.15.43:0/1850724842 >> 172.21.15.43:0/1850724842 conn(0x7f296c09cd40 msgr2=0x7f296c0d23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:13.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.567+0000 7f2971c1f640 1 -- 172.21.15.43:0/1850724842 shutdown_connections 2024-04-25T09:03:13.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.567+0000 7f2971c1f640 1 -- 172.21.15.43:0/1850724842 wait complete. 2024-04-25T09:03:13.567 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T09:03:13.568 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:03:13.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.683+0000 7efc5469e640 1 Processor -- start 2024-04-25T09:03:13.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.683+0000 7efc5469e640 1 -- start start 2024-04-25T09:03:13.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.684+0000 7efc5469e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7efc4c0b6110 0x7efc4c0d1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:13.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.684+0000 7efc5469e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc4c0b32a0 con 0x7efc4c0d1a20 2024-04-25T09:03:13.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.684+0000 7efc5469e640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc4c15bf90 con 0x7efc4c0d2340 2024-04-25T09:03:13.673 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.684+0000 7efc5469e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc4c0cf430 con 0x7efc4c0b6110 2024-04-25T09:03:13.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.685+0000 7efc51c39640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7efc4c0b6110 0x7efc4c0d1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:13.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.685+0000 7efc5243a640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7efc4c0d1a20 0x7efc4c0b2b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:33530/0 (socket says 172.21.15.43:33530) 2024-04-25T09:03:13.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.685+0000 7efc5243a640 1 -- 172.21.15.43:0/983742951 learned_addr learned my addr 172.21.15.43:0/983742951 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:03:13.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.685+0000 7efc51c39640 1 -- 172.21.15.43:0/983742951 >> v1:172.21.15.193:6789/0 conn(0x7efc4c0d2340 legacy=0x7efc4c1656d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:13.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.685+0000 7efc51c39640 1 -- 172.21.15.43:0/983742951 >> v1:172.21.15.43:6789/0 conn(0x7efc4c0d1a20 legacy=0x7efc4c0b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:13.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.685+0000 7efc51c39640 1 -- 172.21.15.43:0/983742951 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc4c0cee70 con 0x7efc4c0b6110 2024-04-25T09:03:13.674 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.685+0000 7efc51438640 1 -- 172.21.15.43:0/983742951 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2842740063 0 0) 0x7efc4c0b32a0 con 0x7efc4c0d1a20 2024-04-25T09:03:13.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.686+0000 7efc51c39640 1 --2- 172.21.15.43:0/983742951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc4c0b6110 0x7efc4c0d1e00 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7efc3c0097a0 tx=0x7efc3c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbd0fc5fbb91e830 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:13.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.686+0000 7efc51438640 1 -- 172.21.15.43:0/983742951 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3c037070 con 0x7efc4c0b6110 2024-04-25T09:03:13.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.686+0000 7efc51438640 1 -- 172.21.15.43:0/983742951 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc3c032440 con 0x7efc4c0b6110 2024-04-25T09:03:13.675 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.686+0000 7efc51438640 1 -- 172.21.15.43:0/983742951 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3c02ae60 con 0x7efc4c0b6110 2024-04-25T09:03:13.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.686+0000 7efc5469e640 1 -- 172.21.15.43:0/983742951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc4c0b6110 msgr2=0x7efc4c0d1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:13.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.686+0000 7efc5469e640 1 --2- 172.21.15.43:0/983742951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc4c0b6110 0x7efc4c0d1e00 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7efc3c0097a0 tx=0x7efc3c02a110 comp rx=0 tx=0).stop 2024-04-25T09:03:13.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.686+0000 7efc5469e640 1 -- 172.21.15.43:0/983742951 shutdown_connections 2024-04-25T09:03:13.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.686+0000 7efc5469e640 1 --2- 172.21.15.43:0/983742951 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc4c0b6110 0x7efc4c0d1e00 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.686+0000 7efc5469e640 1 -- 172.21.15.43:0/983742951 >> 172.21.15.43:0/983742951 conn(0x7efc4c072aa0 msgr2=0x7efc4c09aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:13.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.686+0000 7efc5469e640 1 -- 172.21.15.43:0/983742951 shutdown_connections 2024-04-25T09:03:13.676 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.686+0000 7efc5469e640 1 -- 172.21.15.43:0/983742951 wait complete. 2024-04-25T09:03:13.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.688+0000 7efc5469e640 1 Processor -- start 2024-04-25T09:03:13.678 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.688+0000 7efc5469e640 1 -- start start 2024-04-25T09:03:13.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.688+0000 7efc5469e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc4c0b6110 0x7efc4c14c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:13.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.688+0000 7efc5469e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efc4c0d1a20 0x7efc4c156af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:13.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.688+0000 7efc5469e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc4c0d2340 0x7efc4c153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:13.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.688+0000 7efc5469e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4c0b4be0 con 0x7efc4c0d2340 2024-04-25T09:03:13.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.688+0000 7efc5469e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4c0b4900 con 0x7efc4c0b6110 2024-04-25T09:03:13.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.688+0000 7efc5469e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc4c0cea60 con 0x7efc4c0d1a20 2024-04-25T09:03:13.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.688+0000 7efc52c3b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc4c0d2340 0x7efc4c153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:13.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.688+0000 7efc52c3b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc4c0d2340 0x7efc4c153a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45216/0 (socket says 172.21.15.43:45216) 2024-04-25T09:03:13.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.688+0000 7efc52c3b640 1 -- 172.21.15.43:0/387903889 learned_addr learned my addr 172.21.15.43:0/387903889 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:13.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.689+0000 7efc5243a640 1 --2- 172.21.15.43:0/387903889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc4c0b6110 0x7efc4c14c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:13.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.689+0000 7efc51c39640 1 --2- 172.21.15.43:0/387903889 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efc4c0d1a20 0x7efc4c156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:13.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.689+0000 7efc52c3b640 1 -- 172.21.15.43:0/387903889 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efc4c0d1a20 msgr2=0x7efc4c156af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:13.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.689+0000 7efc52c3b640 1 --2- 172.21.15.43:0/387903889 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efc4c0d1a20 0x7efc4c156af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.679 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.689+0000 7efc52c3b640 1 -- 172.21.15.43:0/387903889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc4c0b6110 msgr2=0x7efc4c14c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:13.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.689+0000 7efc52c3b640 1 --2- 172.21.15.43:0/387903889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc4c0b6110 0x7efc4c14c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:13.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.689+0000 7efc52c3b640 1 -- 172.21.15.43:0/387903889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc4c14c660 con 0x7efc4c0d2340 2024-04-25T09:03:13.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.689+0000 7efc51c39640 1 --2- 172.21.15.43:0/387903889 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efc4c0d1a20 0x7efc4c156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:13.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.689+0000 7efc5243a640 1 --2- 172.21.15.43:0/387903889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc4c0b6110 0x7efc4c14c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:13.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.689+0000 7efc52c3b640 1 --2- 172.21.15.43:0/387903889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc4c0d2340 0x7efc4c153a80 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7efc4400c680 tx=0x7efc4400cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:13.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.689+0000 7efc42ffd640 1 -- 172.21.15.43:0/387903889 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc44003b70 con 0x7efc4c0d2340 2024-04-25T09:03:13.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.689+0000 7efc42ffd640 1 -- 172.21.15.43:0/387903889 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc44003d10 con 0x7efc4c0d2340 2024-04-25T09:03:13.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.689+0000 7efc5469e640 1 -- 172.21.15.43:0/387903889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc4c15d1d0 con 0x7efc4c0d2340 2024-04-25T09:03:13.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.689+0000 7efc42ffd640 1 -- 172.21.15.43:0/387903889 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc44005310 con 0x7efc4c0d2340 2024-04-25T09:03:13.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.689+0000 7efc5469e640 1 -- 172.21.15.43:0/387903889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc4c15d770 con 0x7efc4c0d2340 2024-04-25T09:03:13.680 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.690+0000 7efc5469e640 1 -- 172.21.15.43:0/387903889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc4c074f60 con 0x7efc4c0d2340 2024-04-25T09:03:13.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.691+0000 7efc42ffd640 1 -- 172.21.15.43:0/387903889 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efc440054b0 con 0x7efc4c0d2340 2024-04-25T09:03:13.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.691+0000 7efc42ffd640 1 --2- 172.21.15.43:0/387903889 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7efc3006b500 0x7efc3006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:13.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.692+0000 7efc42ffd640 1 -- 172.21.15.43:0/387903889 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efc44089f80 con 0x7efc4c0d2340 2024-04-25T09:03:13.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.692+0000 7efc5243a640 1 --2- 172.21.15.43:0/387903889 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7efc3006b500 0x7efc3006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:13.681 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.692+0000 7efc5243a640 1 --2- 172.21.15.43:0/387903889 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7efc3006b500 0x7efc3006d9c0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7efc48000f60 tx=0x7efc48009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:13.684 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.696+0000 7efc42ffd640 1 -- 172.21.15.43:0/387903889 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efc44058e90 con 0x7efc4c0d2340 2024-04-25T09:03:13.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:13.916+0000 7efc5469e640 1 -- 172.21.15.43:0/387903889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7efc4c0b5c30 con 0x7efc4c0d2340 2024-04-25T09:03:14.307 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-04-25T09:03:14.307 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-04-25T09:03:14.307 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-04-25T09:03:14.308 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-04-25T09:03:14.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.575+0000 7efc42ffd640 1 -- 172.21.15.43:0/387903889 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v76) v1 ==== 122+0+0 (secure 0 0 0) 0x7efc44058790 con 0x7efc4c0d2340 2024-04-25T09:03:14.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.578+0000 7efc5469e640 1 -- 172.21.15.43:0/387903889 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7efc3006b500 msgr2=0x7efc3006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:14.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.578+0000 7efc5469e640 1 --2- 172.21.15.43:0/387903889 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7efc3006b500 0x7efc3006d9c0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7efc48000f60 tx=0x7efc48009040 comp rx=0 tx=0).stop 2024-04-25T09:03:14.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.578+0000 7efc5469e640 1 -- 172.21.15.43:0/387903889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc4c0d2340 msgr2=0x7efc4c153a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:14.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.578+0000 7efc5469e640 1 --2- 172.21.15.43:0/387903889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc4c0d2340 0x7efc4c153a80 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7efc4400c680 tx=0x7efc4400cb40 comp rx=0 tx=0).stop 2024-04-25T09:03:14.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.578+0000 7efc5469e640 1 -- 172.21.15.43:0/387903889 shutdown_connections 2024-04-25T09:03:14.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.578+0000 7efc5469e640 1 --2- 172.21.15.43:0/387903889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efc4c0d2340 0x7efc4c153a80 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:14.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.578+0000 7efc5469e640 1 --2- 172.21.15.43:0/387903889 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efc4c0d1a20 0x7efc4c156af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:14.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.578+0000 7efc5469e640 1 --2- 172.21.15.43:0/387903889 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7efc3006b500 0x7efc3006d9c0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:14.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.578+0000 7efc5469e640 1 --2- 172.21.15.43:0/387903889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efc4c0b6110 0x7efc4c14c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:14.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.578+0000 7efc5469e640 1 -- 172.21.15.43:0/387903889 >> 172.21.15.43:0/387903889 conn(0x7efc4c072aa0 msgr2=0x7efc4c06e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:14.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.578+0000 7efc5469e640 1 -- 172.21.15.43:0/387903889 shutdown_connections 2024-04-25T09:03:14.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.578+0000 7efc5469e640 1 -- 172.21.15.43:0/387903889 wait complete. 2024-04-25T09:03:14.580 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:03:14.581 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:14.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.699+0000 7f5743968640 1 Processor -- start 2024-04-25T09:03:14.688 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.700+0000 7f5743968640 1 -- start start 2024-04-25T09:03:14.689 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.700+0000 7f5743968640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f573c0cff70 0x7f573c0cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:14.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.700+0000 7f5743968640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f573c0b23f0 0x7f573c0ce210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:14.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.700+0000 7f5743968640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f573c0ccf10 con 0x7f573c15f180 2024-04-25T09:03:14.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.700+0000 7f5743968640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f573c0cccc0 con 0x7f573c0b23f0 2024-04-25T09:03:14.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.700+0000 7f5743968640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f573c0cd840 con 0x7f573c0cff70 2024-04-25T09:03:14.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.701+0000 7f5741f05640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f573c0b23f0 0x7f573c0ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:14.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.701+0000 7f5741704640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f573c15f180 0x7f573c0cf860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:33542/0 (socket says 172.21.15.43:33542) 2024-04-25T09:03:14.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.701+0000 7f5741704640 1 -- 172.21.15.43:0/2893116177 learned_addr learned my addr 172.21.15.43:0/2893116177 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:03:14.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.701+0000 7f5740f03640 1 --2- 172.21.15.43:0/2893116177 >> v2:172.21.15.193:3300/0 conn(0x7f573c0cff70 0x7f573c0cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:14.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.701+0000 7f5741f05640 1 -- 172.21.15.43:0/2893116177 >> v2:172.21.15.193:3300/0 conn(0x7f573c0cff70 msgr2=0x7f573c0cdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:14.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.701+0000 7f5741f05640 1 --2- 172.21.15.43:0/2893116177 >> v2:172.21.15.193:3300/0 conn(0x7f573c0cff70 0x7f573c0cdcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:14.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.701+0000 7f5741f05640 1 -- 172.21.15.43:0/2893116177 >> v1:172.21.15.43:6789/0 conn(0x7f573c15f180 legacy=0x7f573c0cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:14.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.701+0000 7f5733fff640 1 -- 172.21.15.43:0/2893116177 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 406252546 0 0) 0x7f573c0ccf10 con 0x7f573c15f180 2024-04-25T09:03:14.690 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.701+0000 7f5741f05640 1 -- 172.21.15.43:0/2893116177 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f573c0cd1f0 con 0x7f573c0b23f0 2024-04-25T09:03:14.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.702+0000 7f5741f05640 1 --2- 172.21.15.43:0/2893116177 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f573c0b23f0 0x7f573c0ce210 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f573c071a40 tx=0x7f573402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=161cf64e3669b687 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:14.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.702+0000 7f5733fff640 1 -- 172.21.15.43:0/2893116177 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5734007740 con 0x7f573c0b23f0 2024-04-25T09:03:14.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.702+0000 7f5733fff640 1 -- 172.21.15.43:0/2893116177 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57340078e0 con 0x7f573c0b23f0 2024-04-25T09:03:14.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.702+0000 7f5733fff640 1 -- 172.21.15.43:0/2893116177 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5734007cb0 con 0x7f573c0b23f0 2024-04-25T09:03:14.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.702+0000 7f5743968640 1 -- 172.21.15.43:0/2893116177 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f573c0b23f0 msgr2=0x7f573c0ce210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:14.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.702+0000 7f5743968640 1 --2- 172.21.15.43:0/2893116177 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f573c0b23f0 0x7f573c0ce210 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f573c071a40 tx=0x7f573402a5f0 comp rx=0 tx=0).stop 2024-04-25T09:03:14.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.702+0000 7f5743968640 1 -- 172.21.15.43:0/2893116177 shutdown_connections 2024-04-25T09:03:14.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.702+0000 7f5743968640 1 --2- 172.21.15.43:0/2893116177 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f573c0b23f0 0x7f573c0ce210 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:14.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.702+0000 7f5743968640 1 --2- 172.21.15.43:0/2893116177 >> v2:172.21.15.193:3300/0 conn(0x7f573c0cff70 0x7f573c0cdcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:14.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.702+0000 7f5743968640 1 -- 172.21.15.43:0/2893116177 >> 172.21.15.43:0/2893116177 conn(0x7f573c0d17f0 msgr2=0x7f573c0d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:14.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.702+0000 7f5743968640 1 -- 172.21.15.43:0/2893116177 shutdown_connections 2024-04-25T09:03:14.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.703+0000 7f5743968640 1 -- 172.21.15.43:0/2893116177 wait complete. 2024-04-25T09:03:14.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.704+0000 7f5743968640 1 Processor -- start 2024-04-25T09:03:14.693 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.704+0000 7f5743968640 1 -- start start 2024-04-25T09:03:14.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.704+0000 7f5743968640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f573c0b23f0 0x7f573c1f7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:14.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.704+0000 7f5743968640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f573c0cff70 0x7f573c0cb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:14.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.704+0000 7f5743968640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f573c15f180 0x7f573c1f4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:14.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.704+0000 7f5743968640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573c15f560 con 0x7f573c0cff70 2024-04-25T09:03:14.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.704+0000 7f5743968640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573c0d23f0 con 0x7f573c15f180 2024-04-25T09:03:14.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.704+0000 7f5743968640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f573c1f76c0 con 0x7f573c0b23f0 2024-04-25T09:03:14.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.704+0000 7f5740f03640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f573c0cff70 0x7f573c0cb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:14.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.705+0000 7f5740f03640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f573c0cff70 0x7f573c0cb3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45228/0 (socket says 172.21.15.43:45228) 2024-04-25T09:03:14.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.705+0000 7f5740f03640 1 -- 172.21.15.43:0/1676133591 learned_addr learned my addr 172.21.15.43:0/1676133591 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:14.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.705+0000 7f5741704640 1 --2- 172.21.15.43:0/1676133591 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f573c0b23f0 0x7f573c1f7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:14.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.705+0000 7f5741f05640 1 --2- 172.21.15.43:0/1676133591 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f573c15f180 0x7f573c1f4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:14.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.705+0000 7f5740f03640 1 -- 172.21.15.43:0/1676133591 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f573c0b23f0 msgr2=0x7f573c1f7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:14.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.705+0000 7f5740f03640 1 --2- 172.21.15.43:0/1676133591 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f573c0b23f0 0x7f573c1f7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:14.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.705+0000 7f5740f03640 1 -- 172.21.15.43:0/1676133591 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f573c15f180 msgr2=0x7f573c1f4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:14.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.705+0000 7f5740f03640 1 --2- 172.21.15.43:0/1676133591 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f573c15f180 0x7f573c1f4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:14.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.705+0000 7f5740f03640 1 -- 172.21.15.43:0/1676133591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f573c1f79a0 con 0x7f573c0cff70 2024-04-25T09:03:14.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.705+0000 7f5741704640 1 --2- 172.21.15.43:0/1676133591 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f573c0b23f0 0x7f573c1f7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:14.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.705+0000 7f5740f03640 1 --2- 172.21.15.43:0/1676133591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f573c0cff70 0x7f573c0cb3f0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f572c00d800 tx=0x7f572c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:14.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.705+0000 7f5731ffb640 1 -- 172.21.15.43:0/1676133591 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c00b6c0 con 0x7f573c0cff70 2024-04-25T09:03:14.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.705+0000 7f5731ffb640 1 -- 172.21.15.43:0/1676133591 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f572c019070 con 0x7f573c0cff70 2024-04-25T09:03:14.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.705+0000 7f5743968640 1 -- 172.21.15.43:0/1676133591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f573c1f7c80 con 0x7f573c0cff70 2024-04-25T09:03:14.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.705+0000 7f5731ffb640 1 -- 172.21.15.43:0/1676133591 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c00ba80 con 0x7f573c0cff70 2024-04-25T09:03:14.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.705+0000 7f5743968640 1 -- 172.21.15.43:0/1676133591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f573c0c7830 con 0x7f573c0cff70 2024-04-25T09:03:14.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.706+0000 7f5743968640 1 -- 172.21.15.43:0/1676133591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f573c074ef0 con 0x7f573c0cff70 2024-04-25T09:03:14.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.707+0000 7f5731ffb640 1 -- 172.21.15.43:0/1676133591 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f572c00bc20 con 0x7f573c0cff70 2024-04-25T09:03:14.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.707+0000 7f5731ffb640 1 --2- 172.21.15.43:0/1676133591 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f572006b4d0 0x7f572006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:14.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.707+0000 7f5731ffb640 1 -- 172.21.15.43:0/1676133591 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f572c08a630 con 0x7f573c0cff70 2024-04-25T09:03:14.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.708+0000 7f5741704640 1 --2- 172.21.15.43:0/1676133591 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f572006b4d0 0x7f572006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:14.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.708+0000 7f5741704640 1 --2- 172.21.15.43:0/1676133591 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f572006b4d0 0x7f572006d990 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5738000f60 tx=0x7f5738007ca0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:14.700 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.711+0000 7f5731ffb640 1 -- 172.21.15.43:0/1676133591 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f572c059540 con 0x7f573c0cff70 2024-04-25T09:03:14.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.929+0000 7f5743968640 1 -- 172.21.15.43:0/1676133591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f573c0cbb80 con 0x7f573c0cff70 2024-04-25T09:03:14.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.930+0000 7f5731ffb640 1 -- 172.21.15.43:0/1676133591 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f572c058e40 con 0x7f573c0cff70 2024-04-25T09:03:14.919 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:03:14.919 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/891225068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":891225068},{"type":"v1","addr":"172.21.15.193:6836","nonce":891225068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":7175,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3679059390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":3679059390},{"type":"v1","addr":"172.21.15.193:6837","nonce":3679059390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":7948,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2127363183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2127363183},{"type":"v1","addr":"172.21.15.43:6835","nonce":2127363183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:11.919337+0000","modified":"2024-04-25T09:03:13.917768+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9237},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9237":{"gid":9237,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6834/2603892772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2603892772},{"type":"v1","addr":"172.21.15.43:6834","nonce":2603892772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:03:14.919 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 76 2024-04-25T09:03:14.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.932+0000 7f5743968640 1 -- 172.21.15.43:0/1676133591 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f572006b4d0 msgr2=0x7f572006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:14.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.932+0000 7f5743968640 1 --2- 172.21.15.43:0/1676133591 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f572006b4d0 0x7f572006d990 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f5738000f60 tx=0x7f5738007ca0 comp rx=0 tx=0).stop 2024-04-25T09:03:14.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.932+0000 7f5743968640 1 -- 172.21.15.43:0/1676133591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f573c0cff70 msgr2=0x7f573c0cb3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:14.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.932+0000 7f5743968640 1 --2- 172.21.15.43:0/1676133591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f573c0cff70 0x7f573c0cb3f0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f572c00d800 tx=0x7f572c00dcc0 comp rx=0 tx=0).stop 2024-04-25T09:03:14.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.932+0000 7f5743968640 1 -- 172.21.15.43:0/1676133591 shutdown_connections 2024-04-25T09:03:14.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.932+0000 7f5743968640 1 --2- 172.21.15.43:0/1676133591 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f572006b4d0 0x7f572006d990 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:14.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.932+0000 7f5743968640 1 --2- 172.21.15.43:0/1676133591 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f573c15f180 0x7f573c1f4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:14.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.932+0000 7f5743968640 1 --2- 172.21.15.43:0/1676133591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f573c0cff70 0x7f573c0cb3f0 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:14.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.932+0000 7f5743968640 1 --2- 172.21.15.43:0/1676133591 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f573c0b23f0 0x7f573c1f7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:14.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.932+0000 7f5743968640 1 -- 172.21.15.43:0/1676133591 >> 172.21.15.43:0/1676133591 conn(0x7f573c0d17f0 msgr2=0x7f573c0b4410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:14.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.932+0000 7f5743968640 1 -- 172.21.15.43:0/1676133591 shutdown_connections 2024-04-25T09:03:14.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:14.932+0000 7f5743968640 1 -- 172.21.15.43:0/1676133591 wait complete. 2024-04-25T09:03:14.932 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:15.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.044+0000 7f7ddebe7640 1 Processor -- start 2024-04-25T09:03:15.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.045+0000 7f7ddebe7640 1 -- start start 2024-04-25T09:03:15.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.045+0000 7f7ddebe7640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7dd80c8d00 0x7f7dd8162100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.045+0000 7f7ddebe7640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f7dd80ce660 0x7f7dd80cea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.045+0000 7f7ddebe7640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dd80d24a0 con 0x7f7dd815df50 2024-04-25T09:03:15.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.045+0000 7f7ddebe7640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dd815e330 con 0x7f7dd80c8d00 2024-04-25T09:03:15.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.045+0000 7f7ddebe7640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dd80d2740 con 0x7f7dd80ce660 2024-04-25T09:03:15.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.045+0000 7f7dd7fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7dd80c8d00 0x7f7dd8162100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.045+0000 7f7ddc983640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f7dd815df50 0x7f7dd81619f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:33556/0 (socket says 172.21.15.43:33556) 2024-04-25T09:03:15.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.045+0000 7f7ddc983640 1 -- 172.21.15.43:0/3891611882 learned_addr learned my addr 172.21.15.43:0/3891611882 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:03:15.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.046+0000 7f7ddd184640 1 --2- 172.21.15.43:0/3891611882 >> v2:172.21.15.193:3300/0 conn(0x7f7dd80ce660 0x7f7dd80cea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.035 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.046+0000 7f7dd7fff640 1 -- 172.21.15.43:0/3891611882 >> v2:172.21.15.193:3300/0 conn(0x7f7dd80ce660 msgr2=0x7f7dd80cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:15.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.046+0000 7f7dd7fff640 1 --2- 172.21.15.43:0/3891611882 >> v2:172.21.15.193:3300/0 conn(0x7f7dd80ce660 0x7f7dd80cea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.046+0000 7f7dd7fff640 1 -- 172.21.15.43:0/3891611882 >> v1:172.21.15.43:6789/0 conn(0x7f7dd815df50 legacy=0x7f7dd81619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.046+0000 7f7dd77fe640 1 -- 172.21.15.43:0/3891611882 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2668580587 0 0) 0x7f7dd80d24a0 con 0x7f7dd815df50 2024-04-25T09:03:15.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.046+0000 7f7dd7fff640 1 -- 172.21.15.43:0/3891611882 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd80d2a20 con 0x7f7dd80c8d00 2024-04-25T09:03:15.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.046+0000 7f7dd7fff640 1 --2- 172.21.15.43:0/3891611882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7dd80c8d00 0x7f7dd8162100 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f7dd80caa00 tx=0x7f7dcc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=749dff818f6cdfcc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:15.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.047+0000 7f7dd77fe640 1 -- 172.21.15.43:0/3891611882 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc037070 con 0x7f7dd80c8d00 2024-04-25T09:03:15.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.047+0000 7f7dd77fe640 1 -- 172.21.15.43:0/3891611882 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dcc032440 con 0x7f7dd80c8d00 2024-04-25T09:03:15.036 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.047+0000 7f7dd77fe640 1 -- 172.21.15.43:0/3891611882 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dcc032810 con 0x7f7dd80c8d00 2024-04-25T09:03:15.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.047+0000 7f7ddebe7640 1 -- 172.21.15.43:0/3891611882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7dd80c8d00 msgr2=0x7f7dd8162100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.047+0000 7f7ddebe7640 1 --2- 172.21.15.43:0/3891611882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7dd80c8d00 0x7f7dd8162100 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f7dd80caa00 tx=0x7f7dcc02a1a0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.047+0000 7f7ddebe7640 1 -- 172.21.15.43:0/3891611882 shutdown_connections 2024-04-25T09:03:15.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.047+0000 7f7ddebe7640 1 --2- 172.21.15.43:0/3891611882 >> v2:172.21.15.193:3300/0 conn(0x7f7dd80ce660 0x7f7dd80cea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.047+0000 7f7ddebe7640 1 --2- 172.21.15.43:0/3891611882 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7dd80c8d00 0x7f7dd8162100 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.047+0000 7f7ddebe7640 1 -- 172.21.15.43:0/3891611882 >> 172.21.15.43:0/3891611882 conn(0x7f7dd80d3a30 msgr2=0x7f7dd80d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:15.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.047+0000 7f7ddebe7640 1 -- 172.21.15.43:0/3891611882 shutdown_connections 2024-04-25T09:03:15.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.047+0000 7f7ddebe7640 1 -- 172.21.15.43:0/3891611882 wait complete. 2024-04-25T09:03:15.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.048+0000 7f7ddebe7640 1 Processor -- start 2024-04-25T09:03:15.038 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.049+0000 7f7ddebe7640 1 -- start start 2024-04-25T09:03:15.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.049+0000 7f7ddebe7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7dd80c8d00 0x7f7dd81f7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.049+0000 7f7ddebe7640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7dd80ce660 0x7f7dd81f7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.049+0000 7f7ddebe7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7dd815df50 0x7f7dd81f7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.049+0000 7f7ddebe7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd80d2190 con 0x7f7dd80c8d00 2024-04-25T09:03:15.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.049+0000 7f7ddebe7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd80c77d0 con 0x7f7dd815df50 2024-04-25T09:03:15.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.049+0000 7f7ddebe7640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dd8162640 con 0x7f7dd80ce660 2024-04-25T09:03:15.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.049+0000 7f7ddc983640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7dd80c8d00 0x7f7dd81f7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.050+0000 7f7ddc983640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7dd80c8d00 0x7f7dd81f7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45246/0 (socket says 172.21.15.43:45246) 2024-04-25T09:03:15.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.050+0000 7f7ddc983640 1 -- 172.21.15.43:0/4105531439 learned_addr learned my addr 172.21.15.43:0/4105531439 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:15.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.050+0000 7f7ddd184640 1 --2- 172.21.15.43:0/4105531439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7dd815df50 0x7f7dd81f7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.050+0000 7f7dd7fff640 1 --2- 172.21.15.43:0/4105531439 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7dd80ce660 0x7f7dd81f7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.050+0000 7f7ddc983640 1 -- 172.21.15.43:0/4105531439 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7dd80ce660 msgr2=0x7f7dd81f7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.050+0000 7f7ddc983640 1 --2- 172.21.15.43:0/4105531439 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7dd80ce660 0x7f7dd81f7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.050+0000 7f7ddc983640 1 -- 172.21.15.43:0/4105531439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7dd815df50 msgr2=0x7f7dd81f7d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.050+0000 7f7ddc983640 1 --2- 172.21.15.43:0/4105531439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7dd815df50 0x7f7dd81f7d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.050+0000 7f7ddc983640 1 -- 172.21.15.43:0/4105531439 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd8160c70 con 0x7f7dd80c8d00 2024-04-25T09:03:15.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.050+0000 7f7ddc983640 1 --2- 172.21.15.43:0/4105531439 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7dd80c8d00 0x7f7dd81f7300 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f7dd000cb10 tx=0x7f7dd000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:15.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.050+0000 7f7dd57fa640 1 -- 172.21.15.43:0/4105531439 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd0017070 con 0x7f7dd80c8d00 2024-04-25T09:03:15.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.050+0000 7f7dd57fa640 1 -- 172.21.15.43:0/4105531439 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dd000ac70 con 0x7f7dd80c8d00 2024-04-25T09:03:15.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.050+0000 7f7ddebe7640 1 -- 172.21.15.43:0/4105531439 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dd8160f50 con 0x7f7dd80c8d00 2024-04-25T09:03:15.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.050+0000 7f7dd57fa640 1 -- 172.21.15.43:0/4105531439 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dd000ae60 con 0x7f7dd80c8d00 2024-04-25T09:03:15.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.050+0000 7f7ddebe7640 1 -- 172.21.15.43:0/4105531439 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dd8151d30 con 0x7f7dd80c8d00 2024-04-25T09:03:15.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.051+0000 7f7ddebe7640 1 -- 172.21.15.43:0/4105531439 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dd80680f0 con 0x7f7dd80c8d00 2024-04-25T09:03:15.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.052+0000 7f7dd57fa640 1 -- 172.21.15.43:0/4105531439 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7dd001a070 con 0x7f7dd80c8d00 2024-04-25T09:03:15.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.052+0000 7f7dd57fa640 1 --2- 172.21.15.43:0/4105531439 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7dc406b4d0 0x7f7dc406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.052+0000 7f7dd57fa640 1 -- 172.21.15.43:0/4105531439 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7dd0089fd0 con 0x7f7dd80c8d00 2024-04-25T09:03:15.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.052+0000 7f7dd7fff640 1 --2- 172.21.15.43:0/4105531439 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7dc406b4d0 0x7f7dc406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.053+0000 7f7dd7fff640 1 --2- 172.21.15.43:0/4105531439 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7dc406b4d0 0x7f7dc406d990 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc034000 tx=0x7f7dcc02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:15.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.056+0000 7f7dd57fa640 1 -- 172.21.15.43:0/4105531439 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7dd0058e50 con 0x7f7dd80c8d00 2024-04-25T09:03:15.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.237+0000 7f7ddebe7640 1 -- 172.21.15.43:0/4105531439 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7dd80ca400 con 0x7f7dd80c8d00 2024-04-25T09:03:15.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.238+0000 7f7dd57fa640 1 -- 172.21.15.43:0/4105531439 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) v1 ==== 74+0+16643 (secure 0 0 0) 0x7f7dd0058750 con 0x7f7dd80c8d00 2024-04-25T09:03:15.228 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:03:15.228 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":126,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:03:13.554470+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:03:10.199822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T09:03:10.892664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:15.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.240+0000 7f7ddebe7640 1 -- 172.21.15.43:0/4105531439 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7dc406b4d0 msgr2=0x7f7dc406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.240+0000 7f7ddebe7640 1 --2- 172.21.15.43:0/4105531439 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7dc406b4d0 0x7f7dc406d990 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc034000 tx=0x7f7dcc02e000 comp rx=0 tx=0).stop 2024-04-25T09:03:15.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.240+0000 7f7ddebe7640 1 -- 172.21.15.43:0/4105531439 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7dd80c8d00 msgr2=0x7f7dd81f7300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.240+0000 7f7ddebe7640 1 --2- 172.21.15.43:0/4105531439 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7dd80c8d00 0x7f7dd81f7300 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f7dd000cb10 tx=0x7f7dd000a060 comp rx=0 tx=0).stop 2024-04-25T09:03:15.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.240+0000 7f7ddebe7640 1 -- 172.21.15.43:0/4105531439 shutdown_connections 2024-04-25T09:03:15.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.240+0000 7f7ddebe7640 1 --2- 172.21.15.43:0/4105531439 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7dc406b4d0 0x7f7dc406d990 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.240+0000 7f7ddebe7640 1 --2- 172.21.15.43:0/4105531439 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7dd815df50 0x7f7dd81f7d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.240+0000 7f7ddebe7640 1 --2- 172.21.15.43:0/4105531439 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7dd80ce660 0x7f7dd81f7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.240+0000 7f7ddebe7640 1 --2- 172.21.15.43:0/4105531439 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7dd80c8d00 0x7f7dd81f7300 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.240+0000 7f7ddebe7640 1 -- 172.21.15.43:0/4105531439 >> 172.21.15.43:0/4105531439 conn(0x7f7dd80d3a30 msgr2=0x7f7dd80c90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:15.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.240+0000 7f7ddebe7640 1 -- 172.21.15.43:0/4105531439 shutdown_connections 2024-04-25T09:03:15.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.240+0000 7f7ddebe7640 1 -- 172.21.15.43:0/4105531439 wait complete. 2024-04-25T09:03:15.241 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:03:15.241 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:03:15.341 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.353+0000 7fe51c26a640 1 Processor -- start 2024-04-25T09:03:15.342 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.353+0000 7fe51c26a640 1 -- start start 2024-04-25T09:03:15.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.354+0000 7fe51c26a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe514161390 0x7fe5140b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.354+0000 7fe51c26a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe5140cecf0 0x7fe5140d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.354+0000 7fe51c26a640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5140b4c30 con 0x7fe5140d02a0 2024-04-25T09:03:15.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.354+0000 7fe51c26a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5140d1910 con 0x7fe5140cecf0 2024-04-25T09:03:15.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.354+0000 7fe51c26a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5140cf2c0 con 0x7fe514161390 2024-04-25T09:03:15.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.354+0000 7fe51a807640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe5140cecf0 0x7fe5140d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.354+0000 7fe51a807640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe5140cecf0 0x7fe5140d0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45248/0 (socket says 172.21.15.43:45248) 2024-04-25T09:03:15.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.354+0000 7fe51a807640 1 -- 172.21.15.43:0/776336083 learned_addr learned my addr 172.21.15.43:0/776336083 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:15.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.354+0000 7fe51a006640 1 --2- 172.21.15.43:0/776336083 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe514161390 0x7fe5140b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.354+0000 7fe51a807640 1 -- 172.21.15.43:0/776336083 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe514161390 msgr2=0x7fe5140b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:15.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.354+0000 7fe51a807640 1 --2- 172.21.15.43:0/776336083 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe514161390 0x7fe5140b6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.354+0000 7fe51a807640 1 -- 172.21.15.43:0/776336083 >> v1:172.21.15.193:6789/0 conn(0x7fe5140d02a0 legacy=0x7fe5140ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:15.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.354+0000 7fe51a807640 1 -- 172.21.15.43:0/776336083 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5140cf5a0 con 0x7fe5140cecf0 2024-04-25T09:03:15.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.355+0000 7fe51a807640 1 --2- 172.21.15.43:0/776336083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5140cecf0 0x7fe5140d0720 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fe50c009e50 tx=0x7fe50c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3914299dd1209f85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:15.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.355+0000 7fe519004640 1 -- 172.21.15.43:0/776336083 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50c036070 con 0x7fe5140cecf0 2024-04-25T09:03:15.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.355+0000 7fe519004640 1 -- 172.21.15.43:0/776336083 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe50c031600 con 0x7fe5140cecf0 2024-04-25T09:03:15.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.355+0000 7fe519004640 1 -- 172.21.15.43:0/776336083 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe50c0319d0 con 0x7fe5140cecf0 2024-04-25T09:03:15.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.356+0000 7fe51c26a640 1 -- 172.21.15.43:0/776336083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5140cecf0 msgr2=0x7fe5140d0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.356+0000 7fe51c26a640 1 --2- 172.21.15.43:0/776336083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5140cecf0 0x7fe5140d0720 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fe50c009e50 tx=0x7fe50c02a6c0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.356+0000 7fe51c26a640 1 -- 172.21.15.43:0/776336083 shutdown_connections 2024-04-25T09:03:15.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.356+0000 7fe51c26a640 1 --2- 172.21.15.43:0/776336083 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5140cecf0 0x7fe5140d0720 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.356+0000 7fe51c26a640 1 --2- 172.21.15.43:0/776336083 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe514161390 0x7fe5140b6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.356+0000 7fe51c26a640 1 -- 172.21.15.43:0/776336083 >> 172.21.15.43:0/776336083 conn(0x7fe51409cd20 msgr2=0x7fe5140d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:15.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.356+0000 7fe51c26a640 1 -- 172.21.15.43:0/776336083 shutdown_connections 2024-04-25T09:03:15.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.356+0000 7fe51c26a640 1 -- 172.21.15.43:0/776336083 wait complete. 2024-04-25T09:03:15.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.357+0000 7fe51c26a640 1 Processor -- start 2024-04-25T09:03:15.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.357+0000 7fe51c26a640 1 -- start start 2024-04-25T09:03:15.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.358+0000 7fe51c26a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5140cecf0 0x7fe5140cb720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.358+0000 7fe51c26a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5140d02a0 0x7fe514156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.358+0000 7fe51c26a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe514161390 0x7fe514157270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.358+0000 7fe51c26a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe514161770 con 0x7fe5140d02a0 2024-04-25T09:03:15.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.358+0000 7fe51c26a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5140d15a0 con 0x7fe5140cecf0 2024-04-25T09:03:15.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.358+0000 7fe51c26a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5140d1da0 con 0x7fe514161390 2024-04-25T09:03:15.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.358+0000 7fe519805640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5140d02a0 0x7fe514156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.358+0000 7fe519805640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5140d02a0 0x7fe514156d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45262/0 (socket says 172.21.15.43:45262) 2024-04-25T09:03:15.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.358+0000 7fe519805640 1 -- 172.21.15.43:0/3994382060 learned_addr learned my addr 172.21.15.43:0/3994382060 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:15.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.358+0000 7fe51a807640 1 --2- 172.21.15.43:0/3994382060 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe514161390 0x7fe514157270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.358+0000 7fe51a006640 1 --2- 172.21.15.43:0/3994382060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5140cecf0 0x7fe5140cb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.358+0000 7fe519805640 1 -- 172.21.15.43:0/3994382060 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe514161390 msgr2=0x7fe514157270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.358+0000 7fe519805640 1 --2- 172.21.15.43:0/3994382060 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe514161390 0x7fe514157270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.358+0000 7fe519805640 1 -- 172.21.15.43:0/3994382060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5140cecf0 msgr2=0x7fe5140cb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.358+0000 7fe519805640 1 --2- 172.21.15.43:0/3994382060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5140cecf0 0x7fe5140cb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.358+0000 7fe519805640 1 -- 172.21.15.43:0/3994382060 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5140cbf70 con 0x7fe5140d02a0 2024-04-25T09:03:15.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.358+0000 7fe519805640 1 --2- 172.21.15.43:0/3994382060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5140d02a0 0x7fe514156d30 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fe50400c430 tx=0x7fe50400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:15.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.358+0000 7fe50affd640 1 -- 172.21.15.43:0/3994382060 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe504011790 con 0x7fe5140d02a0 2024-04-25T09:03:15.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.358+0000 7fe50affd640 1 -- 172.21.15.43:0/3994382060 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe504017070 con 0x7fe5140d02a0 2024-04-25T09:03:15.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.358+0000 7fe51c26a640 1 -- 172.21.15.43:0/3994382060 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5140cc250 con 0x7fe5140d02a0 2024-04-25T09:03:15.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.358+0000 7fe50affd640 1 -- 172.21.15.43:0/3994382060 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe504011bb0 con 0x7fe5140d02a0 2024-04-25T09:03:15.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.358+0000 7fe51c26a640 1 -- 172.21.15.43:0/3994382060 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5140cbc60 con 0x7fe5140d02a0 2024-04-25T09:03:15.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.359+0000 7fe51c26a640 1 -- 172.21.15.43:0/3994382060 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5140680f0 con 0x7fe5140d02a0 2024-04-25T09:03:15.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.360+0000 7fe50affd640 1 -- 172.21.15.43:0/3994382060 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe50401c460 con 0x7fe5140d02a0 2024-04-25T09:03:15.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.360+0000 7fe50affd640 1 --2- 172.21.15.43:0/3994382060 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe4f806b4d0 0x7fe4f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.361+0000 7fe50affd640 1 -- 172.21.15.43:0/3994382060 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe50408a0c0 con 0x7fe5140d02a0 2024-04-25T09:03:15.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.361+0000 7fe51a006640 1 --2- 172.21.15.43:0/3994382060 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe4f806b4d0 0x7fe4f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.354 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.361+0000 7fe51a006640 1 --2- 172.21.15.43:0/3994382060 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe4f806b4d0 0x7fe4f806d990 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fe510006fd0 tx=0x7fe510008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:15.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.365+0000 7fe50affd640 1 -- 172.21.15.43:0/3994382060 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe504058f40 con 0x7fe5140d02a0 2024-04-25T09:03:15.529 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.540+0000 7fe51c26a640 1 -- 172.21.15.43:0/3994382060 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe514074ef0 con 0x7fe4f806b4d0 2024-04-25T09:03:15.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.547+0000 7fe50affd640 1 -- 172.21.15.43:0/3994382060 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252483 (secure 0 0 0) 0x7fe514074ef0 con 0x7fe4f806b4d0 2024-04-25T09:03:15.538 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:03:15.539 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-25T09:03:15.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.549+0000 7fe51c26a640 1 -- 172.21.15.43:0/3994382060 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe4f806b4d0 msgr2=0x7fe4f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.549+0000 7fe51c26a640 1 --2- 172.21.15.43:0/3994382060 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe4f806b4d0 0x7fe4f806d990 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fe510006fd0 tx=0x7fe510008040 comp rx=0 tx=0).stop 2024-04-25T09:03:15.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.549+0000 7fe51c26a640 1 -- 172.21.15.43:0/3994382060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5140d02a0 msgr2=0x7fe514156d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.549+0000 7fe51c26a640 1 --2- 172.21.15.43:0/3994382060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5140d02a0 0x7fe514156d30 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fe50400c430 tx=0x7fe50400c8f0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.549+0000 7fe51c26a640 1 -- 172.21.15.43:0/3994382060 shutdown_connections 2024-04-25T09:03:15.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.549+0000 7fe51c26a640 1 --2- 172.21.15.43:0/3994382060 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe514161390 0x7fe514157270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.549+0000 7fe51c26a640 1 --2- 172.21.15.43:0/3994382060 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe5140d02a0 0x7fe514156d30 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.549+0000 7fe51c26a640 1 --2- 172.21.15.43:0/3994382060 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe4f806b4d0 0x7fe4f806d990 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.549+0000 7fe51c26a640 1 --2- 172.21.15.43:0/3994382060 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe5140cecf0 0x7fe5140cb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.549+0000 7fe51c26a640 1 -- 172.21.15.43:0/3994382060 >> 172.21.15.43:0/3994382060 conn(0x7fe51409cd20 msgr2=0x7fe51406b040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:15.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.549+0000 7fe51c26a640 1 -- 172.21.15.43:0/3994382060 shutdown_connections 2024-04-25T09:03:15.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.549+0000 7fe51c26a640 1 -- 172.21.15.43:0/3994382060 wait complete. 2024-04-25T09:03:15.549 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":397,"stamp":"2024-04-25T09:03:13.819850+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":164,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":487816,"kb_used_data":13440,"kb_used_omap":12,"kb_used_meta":474227,"kb_avail":754486904,"statfs":{"total":773094113280,"available":772594589696,"internally_reserved":0,"allocated":13762560,"data_stored":6634812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":485609009},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000786"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.557974+0000","last_change":"2024-04-25T09:03:12.573783+0000","last_active":"2024-04-25T09:03:13.557974+0000","last_peered":"2024-04-25T09:03:13.557974+0000","last_clean":"2024-04-25T09:03:13.557974+0000","last_became_active":"2024-04-25T09:03:12.573517+0000","last_became_peered":"2024-04-25T09:03:12.573517+0000","last_unstale":"2024-04-25T09:03:13.557974+0000","last_undegraded":"2024-04-25T09:03:13.557974+0000","last_fullsized":"2024-04-25T09:03:13.557974+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:37.081732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564429+0000","last_change":"2024-04-25T09:03:12.572481+0000","last_active":"2024-04-25T09:03:13.564429+0000","last_peered":"2024-04-25T09:03:13.564429+0000","last_clean":"2024-04-25T09:03:13.564429+0000","last_became_active":"2024-04-25T09:03:12.571979+0000","last_became_peered":"2024-04-25T09:03:12.571979+0000","last_unstale":"2024-04-25T09:03:13.564429+0000","last_undegraded":"2024-04-25T09:03:13.564429+0000","last_fullsized":"2024-04-25T09:03:13.564429+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:28.189475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.559540+0000","last_change":"2024-04-25T09:03:12.574900+0000","last_active":"2024-04-25T09:03:13.559540+0000","last_peered":"2024-04-25T09:03:13.559540+0000","last_clean":"2024-04-25T09:03:13.559540+0000","last_became_active":"2024-04-25T09:03:12.574542+0000","last_became_peered":"2024-04-25T09:03:12.574542+0000","last_unstale":"2024-04-25T09:03:13.559540+0000","last_undegraded":"2024-04-25T09:03:13.559540+0000","last_fullsized":"2024-04-25T09:03:13.559540+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:11.599506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.557581+0000","last_change":"2024-04-25T09:03:12.576885+0000","last_active":"2024-04-25T09:03:13.557581+0000","last_peered":"2024-04-25T09:03:13.557581+0000","last_clean":"2024-04-25T09:03:13.557581+0000","last_became_active":"2024-04-25T09:03:12.576601+0000","last_became_peered":"2024-04-25T09:03:12.576601+0000","last_unstale":"2024-04-25T09:03:13.557581+0000","last_undegraded":"2024-04-25T09:03:13.557581+0000","last_fullsized":"2024-04-25T09:03:13.557581+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:37.484044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.559774+0000","last_change":"2024-04-25T09:03:12.572728+0000","last_active":"2024-04-25T09:03:13.559774+0000","last_peered":"2024-04-25T09:03:13.559774+0000","last_clean":"2024-04-25T09:03:13.559774+0000","last_became_active":"2024-04-25T09:03:12.572176+0000","last_became_peered":"2024-04-25T09:03:12.572176+0000","last_unstale":"2024-04-25T09:03:13.559774+0000","last_undegraded":"2024-04-25T09:03:13.559774+0000","last_fullsized":"2024-04-25T09:03:13.559774+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:17.566176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565020+0000","last_change":"2024-04-25T09:03:12.584575+0000","last_active":"2024-04-25T09:03:13.565020+0000","last_peered":"2024-04-25T09:03:13.565020+0000","last_clean":"2024-04-25T09:03:13.565020+0000","last_became_active":"2024-04-25T09:03:12.583521+0000","last_became_peered":"2024-04-25T09:03:12.583521+0000","last_unstale":"2024-04-25T09:03:13.565020+0000","last_undegraded":"2024-04-25T09:03:13.565020+0000","last_fullsized":"2024-04-25T09:03:13.565020+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:11.725050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564460+0000","last_change":"2024-04-25T09:03:12.578595+0000","last_active":"2024-04-25T09:03:13.564460+0000","last_peered":"2024-04-25T09:03:13.564460+0000","last_clean":"2024-04-25T09:03:13.564460+0000","last_became_active":"2024-04-25T09:03:12.578291+0000","last_became_peered":"2024-04-25T09:03:12.578291+0000","last_unstale":"2024-04-25T09:03:13.564460+0000","last_undegraded":"2024-04-25T09:03:13.564460+0000","last_fullsized":"2024-04-25T09:03:13.564460+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:28.189475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.559998+0000","last_change":"2024-04-25T09:03:12.580399+0000","last_active":"2024-04-25T09:03:13.559998+0000","last_peered":"2024-04-25T09:03:13.559998+0000","last_clean":"2024-04-25T09:03:13.559998+0000","last_became_active":"2024-04-25T09:03:12.579505+0000","last_became_peered":"2024-04-25T09:03:12.579505+0000","last_unstale":"2024-04-25T09:03:13.559998+0000","last_undegraded":"2024-04-25T09:03:13.559998+0000","last_fullsized":"2024-04-25T09:03:13.559998+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:56.534408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.559529+0000","last_change":"2024-04-25T09:03:12.575257+0000","last_active":"2024-04-25T09:03:13.559529+0000","last_peered":"2024-04-25T09:03:13.559529+0000","last_clean":"2024-04-25T09:03:13.559529+0000","last_became_active":"2024-04-25T09:03:12.574353+0000","last_became_peered":"2024-04-25T09:03:12.574353+0000","last_unstale":"2024-04-25T09:03:13.559529+0000","last_undegraded":"2024-04-25T09:03:13.559529+0000","last_fullsized":"2024-04-25T09:03:13.559529+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:51.666701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564568+0000","last_change":"2024-04-25T09:03:12.572243+0000","last_active":"2024-04-25T09:03:13.564568+0000","last_peered":"2024-04-25T09:03:13.564568+0000","last_clean":"2024-04-25T09:03:13.564568+0000","last_became_active":"2024-04-25T09:03:12.571635+0000","last_became_peered":"2024-04-25T09:03:12.571635+0000","last_unstale":"2024-04-25T09:03:13.564568+0000","last_undegraded":"2024-04-25T09:03:13.564568+0000","last_fullsized":"2024-04-25T09:03:13.564568+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:20.423551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565896+0000","last_change":"2024-04-25T09:03:12.566485+0000","last_active":"2024-04-25T09:03:13.565896+0000","last_peered":"2024-04-25T09:03:13.565896+0000","last_clean":"2024-04-25T09:03:13.565896+0000","last_became_active":"2024-04-25T09:03:12.565930+0000","last_became_peered":"2024-04-25T09:03:12.565930+0000","last_unstale":"2024-04-25T09:03:13.565896+0000","last_undegraded":"2024-04-25T09:03:13.565896+0000","last_fullsized":"2024-04-25T09:03:13.565896+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:11.963001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.559710+0000","last_change":"2024-04-25T09:03:12.575431+0000","last_active":"2024-04-25T09:03:13.559710+0000","last_peered":"2024-04-25T09:03:13.559710+0000","last_clean":"2024-04-25T09:03:13.559710+0000","last_became_active":"2024-04-25T09:03:12.575178+0000","last_became_peered":"2024-04-25T09:03:12.575178+0000","last_unstale":"2024-04-25T09:03:13.559710+0000","last_undegraded":"2024-04-25T09:03:13.559710+0000","last_fullsized":"2024-04-25T09:03:13.559710+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:28.189475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.557779+0000","last_change":"2024-04-25T09:03:12.582827+0000","last_active":"2024-04-25T09:03:13.557779+0000","last_peered":"2024-04-25T09:03:13.557779+0000","last_clean":"2024-04-25T09:03:13.557779+0000","last_became_active":"2024-04-25T09:03:12.582533+0000","last_became_peered":"2024-04-25T09:03:12.582533+0000","last_unstale":"2024-04-25T09:03:13.557779+0000","last_undegraded":"2024-04-25T09:03:13.557779+0000","last_fullsized":"2024-04-25T09:03:13.557779+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:11.089699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564419+0000","last_change":"2024-04-25T09:03:12.564011+0000","last_active":"2024-04-25T09:03:13.564419+0000","last_peered":"2024-04-25T09:03:13.564419+0000","last_clean":"2024-04-25T09:03:13.564419+0000","last_became_active":"2024-04-25T09:03:12.563690+0000","last_became_peered":"2024-04-25T09:03:12.563690+0000","last_unstale":"2024-04-25T09:03:13.564419+0000","last_undegraded":"2024-04-25T09:03:13.564419+0000","last_fullsized":"2024-04-25T09:03:13.564419+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:36.514775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.557487+0000","last_change":"2024-04-25T09:03:12.579378+0000","last_active":"2024-04-25T09:03:13.557487+0000","last_peered":"2024-04-25T09:03:13.557487+0000","last_clean":"2024-04-25T09:03:13.557487+0000","last_became_active":"2024-04-25T09:03:12.578782+0000","last_became_peered":"2024-04-25T09:03:12.578782+0000","last_unstale":"2024-04-25T09:03:13.557487+0000","last_undegraded":"2024-04-25T09:03:13.557487+0000","last_fullsized":"2024-04-25T09:03:13.557487+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:39.449290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565429+0000","last_change":"2024-04-25T09:03:12.576695+0000","last_active":"2024-04-25T09:03:13.565429+0000","last_peered":"2024-04-25T09:03:13.565429+0000","last_clean":"2024-04-25T09:03:13.565429+0000","last_became_active":"2024-04-25T09:03:12.575847+0000","last_became_peered":"2024-04-25T09:03:12.575847+0000","last_unstale":"2024-04-25T09:03:13.565429+0000","last_undegraded":"2024-04-25T09:03:13.565429+0000","last_fullsized":"2024-04-25T09:03:13.565429+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:32.571219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564802+0000","last_change":"2024-04-25T09:03:12.568985+0000","last_active":"2024-04-25T09:03:13.564802+0000","last_peered":"2024-04-25T09:03:13.564802+0000","last_clean":"2024-04-25T09:03:13.564802+0000","last_became_active":"2024-04-25T09:03:12.568696+0000","last_became_peered":"2024-04-25T09:03:12.568696+0000","last_unstale":"2024-04-25T09:03:13.564802+0000","last_undegraded":"2024-04-25T09:03:13.564802+0000","last_fullsized":"2024-04-25T09:03:13.564802+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:20.423551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565061+0000","last_change":"2024-04-25T09:03:12.575681+0000","last_active":"2024-04-25T09:03:13.565061+0000","last_peered":"2024-04-25T09:03:13.565061+0000","last_clean":"2024-04-25T09:03:13.565061+0000","last_became_active":"2024-04-25T09:03:12.575376+0000","last_became_peered":"2024-04-25T09:03:12.575376+0000","last_unstale":"2024-04-25T09:03:13.565061+0000","last_undegraded":"2024-04-25T09:03:13.565061+0000","last_fullsized":"2024-04-25T09:03:13.565061+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:06.928915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565962+0000","last_change":"2024-04-25T09:03:12.566203+0000","last_active":"2024-04-25T09:03:13.565962+0000","last_peered":"2024-04-25T09:03:13.565962+0000","last_clean":"2024-04-25T09:03:13.565962+0000","last_became_active":"2024-04-25T09:03:12.565560+0000","last_became_peered":"2024-04-25T09:03:12.565560+0000","last_unstale":"2024-04-25T09:03:13.565962+0000","last_undegraded":"2024-04-25T09:03:13.565962+0000","last_fullsized":"2024-04-25T09:03:13.565962+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:40.106983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565712+0000","last_change":"2024-04-25T09:03:12.571546+0000","last_active":"2024-04-25T09:03:13.565712+0000","last_peered":"2024-04-25T09:03:13.565712+0000","last_clean":"2024-04-25T09:03:13.565712+0000","last_became_active":"2024-04-25T09:03:12.571156+0000","last_became_peered":"2024-04-25T09:03:12.571156+0000","last_unstale":"2024-04-25T09:03:13.565712+0000","last_undegraded":"2024-04-25T09:03:13.565712+0000","last_fullsized":"2024-04-25T09:03:13.565712+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:17.566176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.559626+0000","last_change":"2024-04-25T09:03:12.572498+0000","last_active":"2024-04-25T09:03:13.559626+0000","last_peered":"2024-04-25T09:03:13.559626+0000","last_clean":"2024-04-25T09:03:13.559626+0000","last_became_active":"2024-04-25T09:03:12.571778+0000","last_became_peered":"2024-04-25T09:03:12.571778+0000","last_unstale":"2024-04-25T09:03:13.559626+0000","last_undegraded":"2024-04-25T09:03:13.559626+0000","last_fullsized":"2024-04-25T09:03:13.559626+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:03.706189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565717+0000","last_change":"2024-04-25T09:03:12.566180+0000","last_active":"2024-04-25T09:03:13.565717+0000","last_peered":"2024-04-25T09:03:13.565717+0000","last_clean":"2024-04-25T09:03:13.565717+0000","last_became_active":"2024-04-25T09:03:12.565888+0000","last_became_peered":"2024-04-25T09:03:12.565888+0000","last_unstale":"2024-04-25T09:03:13.565717+0000","last_undegraded":"2024-04-25T09:03:13.565717+0000","last_fullsized":"2024-04-25T09:03:13.565717+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:07.938843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.560308+0000","last_change":"2024-04-25T09:03:12.576813+0000","last_active":"2024-04-25T09:03:13.560308+0000","last_peered":"2024-04-25T09:03:13.560308+0000","last_clean":"2024-04-25T09:03:13.560308+0000","last_became_active":"2024-04-25T09:03:12.576552+0000","last_became_peered":"2024-04-25T09:03:12.576552+0000","last_unstale":"2024-04-25T09:03:13.560308+0000","last_undegraded":"2024-04-25T09:03:13.560308+0000","last_fullsized":"2024-04-25T09:03:13.560308+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:24.444267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564059+0000","last_change":"2024-04-25T09:03:12.570253+0000","last_active":"2024-04-25T09:03:13.564059+0000","last_peered":"2024-04-25T09:03:13.564059+0000","last_clean":"2024-04-25T09:03:13.564059+0000","last_became_active":"2024-04-25T09:03:12.569969+0000","last_became_peered":"2024-04-25T09:03:12.569969+0000","last_unstale":"2024-04-25T09:03:13.564059+0000","last_undegraded":"2024-04-25T09:03:13.564059+0000","last_fullsized":"2024-04-25T09:03:13.564059+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:05.513864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564716+0000","last_change":"2024-04-25T09:03:12.569197+0000","last_active":"2024-04-25T09:03:13.564716+0000","last_peered":"2024-04-25T09:03:13.564716+0000","last_clean":"2024-04-25T09:03:13.564716+0000","last_became_active":"2024-04-25T09:03:12.567874+0000","last_became_peered":"2024-04-25T09:03:12.567874+0000","last_unstale":"2024-04-25T09:03:13.564716+0000","last_undegraded":"2024-04-25T09:03:13.564716+0000","last_fullsized":"2024-04-25T09:03:13.564716+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:24.444267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565202+0000","last_change":"2024-04-25T09:03:12.578579+0000","last_active":"2024-04-25T09:03:13.565202+0000","last_peered":"2024-04-25T09:03:13.565202+0000","last_clean":"2024-04-25T09:03:13.565202+0000","last_became_active":"2024-04-25T09:03:12.578313+0000","last_became_peered":"2024-04-25T09:03:12.578313+0000","last_unstale":"2024-04-25T09:03:13.565202+0000","last_undegraded":"2024-04-25T09:03:13.565202+0000","last_fullsized":"2024-04-25T09:03:13.565202+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:32.511163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.557752+0000","last_change":"2024-04-25T09:03:12.573508+0000","last_active":"2024-04-25T09:03:13.557752+0000","last_peered":"2024-04-25T09:03:13.557752+0000","last_clean":"2024-04-25T09:03:13.557752+0000","last_became_active":"2024-04-25T09:03:12.572781+0000","last_became_peered":"2024-04-25T09:03:12.572781+0000","last_unstale":"2024-04-25T09:03:13.557752+0000","last_undegraded":"2024-04-25T09:03:13.557752+0000","last_fullsized":"2024-04-25T09:03:13.557752+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:19.906821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564544+0000","last_change":"2024-04-25T09:03:12.569290+0000","last_active":"2024-04-25T09:03:13.564544+0000","last_peered":"2024-04-25T09:03:13.564544+0000","last_clean":"2024-04-25T09:03:13.564544+0000","last_became_active":"2024-04-25T09:03:12.568972+0000","last_became_peered":"2024-04-25T09:03:12.568972+0000","last_unstale":"2024-04-25T09:03:13.564544+0000","last_undegraded":"2024-04-25T09:03:13.564544+0000","last_fullsized":"2024-04-25T09:03:13.564544+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:36.157007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.557770+0000","last_change":"2024-04-25T09:03:12.579246+0000","last_active":"2024-04-25T09:03:13.557770+0000","last_peered":"2024-04-25T09:03:13.557770+0000","last_clean":"2024-04-25T09:03:13.557770+0000","last_became_active":"2024-04-25T09:03:12.578947+0000","last_became_peered":"2024-04-25T09:03:12.578947+0000","last_unstale":"2024-04-25T09:03:13.557770+0000","last_undegraded":"2024-04-25T09:03:13.557770+0000","last_fullsized":"2024-04-25T09:03:13.557770+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:27.350319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.557654+0000","last_change":"2024-04-25T09:03:12.575982+0000","last_active":"2024-04-25T09:03:13.557654+0000","last_peered":"2024-04-25T09:03:13.557654+0000","last_clean":"2024-04-25T09:03:13.557654+0000","last_became_active":"2024-04-25T09:03:12.575692+0000","last_became_peered":"2024-04-25T09:03:12.575692+0000","last_unstale":"2024-04-25T09:03:13.557654+0000","last_undegraded":"2024-04-25T09:03:13.557654+0000","last_fullsized":"2024-04-25T09:03:13.557654+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:33.677525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.559192+0000","last_change":"2024-04-25T09:03:12.581173+0000","last_active":"2024-04-25T09:03:13.559192+0000","last_peered":"2024-04-25T09:03:13.559192+0000","last_clean":"2024-04-25T09:03:13.559192+0000","last_became_active":"2024-04-25T09:03:12.580291+0000","last_became_peered":"2024-04-25T09:03:12.580291+0000","last_unstale":"2024-04-25T09:03:13.559192+0000","last_undegraded":"2024-04-25T09:03:13.559192+0000","last_fullsized":"2024-04-25T09:03:13.559192+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:47.056165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564516+0000","last_change":"2024-04-25T09:03:12.573970+0000","last_active":"2024-04-25T09:03:13.564516+0000","last_peered":"2024-04-25T09:03:13.564516+0000","last_clean":"2024-04-25T09:03:13.564516+0000","last_became_active":"2024-04-25T09:03:12.573555+0000","last_became_peered":"2024-04-25T09:03:12.573555+0000","last_unstale":"2024-04-25T09:03:13.564516+0000","last_undegraded":"2024-04-25T09:03:13.564516+0000","last_fullsized":"2024-04-25T09:03:13.564516+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:56.534408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565367+0000","last_change":"2024-04-25T09:03:12.570467+0000","last_active":"2024-04-25T09:03:13.565367+0000","last_peered":"2024-04-25T09:03:13.565367+0000","last_clean":"2024-04-25T09:03:13.565367+0000","last_became_active":"2024-04-25T09:03:12.570161+0000","last_became_peered":"2024-04-25T09:03:12.570161+0000","last_unstale":"2024-04-25T09:03:13.565367+0000","last_undegraded":"2024-04-25T09:03:13.565367+0000","last_fullsized":"2024-04-25T09:03:13.565367+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:39.272779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"125'4","reported_seq":18,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.559664+0000","last_change":"2024-04-25T09:03:11.554834+0000","last_active":"2024-04-25T09:03:13.559664+0000","last_peered":"2024-04-25T09:03:13.559664+0000","last_clean":"2024-04-25T09:03:13.559664+0000","last_became_active":"2024-04-25T09:03:11.554552+0000","last_became_peered":"2024-04-25T09:03:11.554552+0000","last_unstale":"2024-04-25T09:03:13.559664+0000","last_undegraded":"2024-04-25T09:03:13.559664+0000","last_fullsized":"2024-04-25T09:03:13.559664+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:30.646577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.559685+0000","last_change":"2024-04-25T09:03:12.585556+0000","last_active":"2024-04-25T09:03:13.559685+0000","last_peered":"2024-04-25T09:03:13.559685+0000","last_clean":"2024-04-25T09:03:13.559685+0000","last_became_active":"2024-04-25T09:03:12.584513+0000","last_became_peered":"2024-04-25T09:03:12.584513+0000","last_unstale":"2024-04-25T09:03:13.559685+0000","last_undegraded":"2024-04-25T09:03:13.559685+0000","last_fullsized":"2024-04-25T09:03:13.559685+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:36.514775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"125'1","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564963+0000","last_change":"2024-04-25T09:03:11.562517+0000","last_active":"2024-04-25T09:03:13.564963+0000","last_peered":"2024-04-25T09:03:13.564963+0000","last_clean":"2024-04-25T09:03:13.564963+0000","last_became_active":"2024-04-25T09:03:11.562150+0000","last_became_peered":"2024-04-25T09:03:11.562150+0000","last_unstale":"2024-04-25T09:03:13.564963+0000","last_undegraded":"2024-04-25T09:03:13.564963+0000","last_fullsized":"2024-04-25T09:03:13.564963+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:36.076417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.560040+0000","last_change":"2024-04-25T09:03:12.577457+0000","last_active":"2024-04-25T09:03:13.560040+0000","last_peered":"2024-04-25T09:03:13.560040+0000","last_clean":"2024-04-25T09:03:13.560040+0000","last_became_active":"2024-04-25T09:03:12.577187+0000","last_became_peered":"2024-04-25T09:03:12.577187+0000","last_unstale":"2024-04-25T09:03:13.560040+0000","last_undegraded":"2024-04-25T09:03:13.560040+0000","last_fullsized":"2024-04-25T09:03:13.560040+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:13.132184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"125'2","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565990+0000","last_change":"2024-04-25T09:03:11.565065+0000","last_active":"2024-04-25T09:03:13.565990+0000","last_peered":"2024-04-25T09:03:13.565990+0000","last_clean":"2024-04-25T09:03:13.565990+0000","last_became_active":"2024-04-25T09:03:11.564761+0000","last_became_peered":"2024-04-25T09:03:11.564761+0000","last_unstale":"2024-04-25T09:03:13.565990+0000","last_undegraded":"2024-04-25T09:03:13.565990+0000","last_fullsized":"2024-04-25T09:03:13.565990+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:12.126869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.560041+0000","last_change":"2024-04-25T09:03:12.572377+0000","last_active":"2024-04-25T09:03:13.560041+0000","last_peered":"2024-04-25T09:03:13.560041+0000","last_clean":"2024-04-25T09:03:13.560041+0000","last_became_active":"2024-04-25T09:03:12.572057+0000","last_became_peered":"2024-04-25T09:03:12.572057+0000","last_unstale":"2024-04-25T09:03:13.560041+0000","last_undegraded":"2024-04-25T09:03:13.560041+0000","last_fullsized":"2024-04-25T09:03:13.560041+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:03.891788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"125'2","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564849+0000","last_change":"2024-04-25T09:03:11.562578+0000","last_active":"2024-04-25T09:03:13.564849+0000","last_peered":"2024-04-25T09:03:13.564849+0000","last_clean":"2024-04-25T09:03:13.564849+0000","last_became_active":"2024-04-25T09:03:11.562276+0000","last_became_peered":"2024-04-25T09:03:11.562276+0000","last_unstale":"2024-04-25T09:03:13.564849+0000","last_undegraded":"2024-04-25T09:03:13.564849+0000","last_fullsized":"2024-04-25T09:03:13.564849+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:35.151692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565384+0000","last_change":"2024-04-25T09:03:12.570418+0000","last_active":"2024-04-25T09:03:13.565384+0000","last_peered":"2024-04-25T09:03:13.565384+0000","last_clean":"2024-04-25T09:03:13.565384+0000","last_became_active":"2024-04-25T09:03:12.570135+0000","last_became_peered":"2024-04-25T09:03:12.570135+0000","last_unstale":"2024-04-25T09:03:13.565384+0000","last_undegraded":"2024-04-25T09:03:13.565384+0000","last_fullsized":"2024-04-25T09:03:13.565384+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:47.056165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"125'1","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564170+0000","last_change":"2024-04-25T09:03:11.563061+0000","last_active":"2024-04-25T09:03:13.564170+0000","last_peered":"2024-04-25T09:03:13.564170+0000","last_clean":"2024-04-25T09:03:13.564170+0000","last_became_active":"2024-04-25T09:03:11.562736+0000","last_became_peered":"2024-04-25T09:03:11.562736+0000","last_unstale":"2024-04-25T09:03:13.564170+0000","last_undegraded":"2024-04-25T09:03:13.564170+0000","last_fullsized":"2024-04-25T09:03:13.564170+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:36.076417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565163+0000","last_change":"2024-04-25T09:03:12.575075+0000","last_active":"2024-04-25T09:03:13.565163+0000","last_peered":"2024-04-25T09:03:13.565163+0000","last_clean":"2024-04-25T09:03:13.565163+0000","last_became_active":"2024-04-25T09:03:12.574742+0000","last_became_peered":"2024-04-25T09:03:12.574742+0000","last_unstale":"2024-04-25T09:03:13.565163+0000","last_undegraded":"2024-04-25T09:03:13.565163+0000","last_fullsized":"2024-04-25T09:03:13.565163+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:24.444267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"125'1","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565495+0000","last_change":"2024-04-25T09:03:11.563805+0000","last_active":"2024-04-25T09:03:13.565495+0000","last_peered":"2024-04-25T09:03:13.565495+0000","last_clean":"2024-04-25T09:03:13.565495+0000","last_became_active":"2024-04-25T09:03:11.563418+0000","last_became_peered":"2024-04-25T09:03:11.563418+0000","last_unstale":"2024-04-25T09:03:13.565495+0000","last_undegraded":"2024-04-25T09:03:13.565495+0000","last_fullsized":"2024-04-25T09:03:13.565495+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:04.407967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.560187+0000","last_change":"2024-04-25T09:03:12.586165+0000","last_active":"2024-04-25T09:03:13.560187+0000","last_peered":"2024-04-25T09:03:13.560187+0000","last_clean":"2024-04-25T09:03:13.560187+0000","last_became_active":"2024-04-25T09:03:12.585393+0000","last_became_peered":"2024-04-25T09:03:12.585393+0000","last_unstale":"2024-04-25T09:03:13.560187+0000","last_undegraded":"2024-04-25T09:03:13.560187+0000","last_fullsized":"2024-04-25T09:03:13.560187+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:11.725050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564584+0000","last_change":"2024-04-25T09:03:11.557091+0000","last_active":"2024-04-25T09:03:13.564584+0000","last_peered":"2024-04-25T09:03:13.564584+0000","last_clean":"2024-04-25T09:03:13.564584+0000","last_became_active":"2024-04-25T09:03:11.556838+0000","last_became_peered":"2024-04-25T09:03:11.556838+0000","last_unstale":"2024-04-25T09:03:13.564584+0000","last_undegraded":"2024-04-25T09:03:13.564584+0000","last_fullsized":"2024-04-25T09:03:13.564584+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:54.964981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564517+0000","last_change":"2024-04-25T09:03:12.564943+0000","last_active":"2024-04-25T09:03:13.564517+0000","last_peered":"2024-04-25T09:03:13.564517+0000","last_clean":"2024-04-25T09:03:13.564517+0000","last_became_active":"2024-04-25T09:03:12.564676+0000","last_became_peered":"2024-04-25T09:03:12.564676+0000","last_unstale":"2024-04-25T09:03:13.564517+0000","last_undegraded":"2024-04-25T09:03:13.564517+0000","last_fullsized":"2024-04-25T09:03:13.564517+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:56.534408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565103+0000","last_change":"2024-04-25T09:03:11.557321+0000","last_active":"2024-04-25T09:03:13.565103+0000","last_peered":"2024-04-25T09:03:13.565103+0000","last_clean":"2024-04-25T09:03:13.565103+0000","last_became_active":"2024-04-25T09:03:11.557038+0000","last_became_peered":"2024-04-25T09:03:11.557038+0000","last_unstale":"2024-04-25T09:03:13.565103+0000","last_undegraded":"2024-04-25T09:03:13.565103+0000","last_fullsized":"2024-04-25T09:03:13.565103+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:59.986937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.557785+0000","last_change":"2024-04-25T09:03:12.566673+0000","last_active":"2024-04-25T09:03:13.557785+0000","last_peered":"2024-04-25T09:03:13.557785+0000","last_clean":"2024-04-25T09:03:13.557785+0000","last_became_active":"2024-04-25T09:03:12.566400+0000","last_became_peered":"2024-04-25T09:03:12.566400+0000","last_unstale":"2024-04-25T09:03:13.557785+0000","last_undegraded":"2024-04-25T09:03:13.557785+0000","last_fullsized":"2024-04-25T09:03:13.557785+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:08.284016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565010+0000","last_change":"2024-04-25T09:03:11.549521+0000","last_active":"2024-04-25T09:03:13.565010+0000","last_peered":"2024-04-25T09:03:13.565010+0000","last_clean":"2024-04-25T09:03:13.565010+0000","last_became_active":"2024-04-25T09:03:11.549262+0000","last_became_peered":"2024-04-25T09:03:11.549262+0000","last_unstale":"2024-04-25T09:03:13.565010+0000","last_undegraded":"2024-04-25T09:03:13.565010+0000","last_fullsized":"2024-04-25T09:03:13.565010+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:54.727184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564866+0000","last_change":"2024-04-25T09:03:12.572760+0000","last_active":"2024-04-25T09:03:13.564866+0000","last_peered":"2024-04-25T09:03:13.564866+0000","last_clean":"2024-04-25T09:03:13.564866+0000","last_became_active":"2024-04-25T09:03:12.572456+0000","last_became_peered":"2024-04-25T09:03:12.572456+0000","last_unstale":"2024-04-25T09:03:13.564866+0000","last_undegraded":"2024-04-25T09:03:13.564866+0000","last_fullsized":"2024-04-25T09:03:13.564866+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:30.334232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565418+0000","last_change":"2024-04-25T09:03:11.561683+0000","last_active":"2024-04-25T09:03:13.565418+0000","last_peered":"2024-04-25T09:03:13.565418+0000","last_clean":"2024-04-25T09:03:13.565418+0000","last_became_active":"2024-04-25T09:03:11.561135+0000","last_became_peered":"2024-04-25T09:03:11.561135+0000","last_unstale":"2024-04-25T09:03:13.565418+0000","last_undegraded":"2024-04-25T09:03:13.565418+0000","last_fullsized":"2024-04-25T09:03:13.565418+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:00.501342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.560270+0000","last_change":"2024-04-25T09:03:12.571420+0000","last_active":"2024-04-25T09:03:13.560270+0000","last_peered":"2024-04-25T09:03:13.560270+0000","last_clean":"2024-04-25T09:03:13.560270+0000","last_became_active":"2024-04-25T09:03:12.571162+0000","last_became_peered":"2024-04-25T09:03:12.571162+0000","last_unstale":"2024-04-25T09:03:13.560270+0000","last_undegraded":"2024-04-25T09:03:13.560270+0000","last_fullsized":"2024-04-25T09:03:13.560270+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:36.351839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"125'3","reported_seq":17,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564697+0000","last_change":"2024-04-25T09:03:11.560067+0000","last_active":"2024-04-25T09:03:13.564697+0000","last_peered":"2024-04-25T09:03:13.564697+0000","last_clean":"2024-04-25T09:03:13.564697+0000","last_became_active":"2024-04-25T09:03:11.559690+0000","last_became_peered":"2024-04-25T09:03:11.559690+0000","last_unstale":"2024-04-25T09:03:13.564697+0000","last_undegraded":"2024-04-25T09:03:13.564697+0000","last_fullsized":"2024-04-25T09:03:13.564697+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:10.084384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.560270+0000","last_change":"2024-04-25T09:03:12.575624+0000","last_active":"2024-04-25T09:03:13.560270+0000","last_peered":"2024-04-25T09:03:13.560270+0000","last_clean":"2024-04-25T09:03:13.560270+0000","last_became_active":"2024-04-25T09:03:12.575263+0000","last_became_peered":"2024-04-25T09:03:12.575263+0000","last_unstale":"2024-04-25T09:03:13.560270+0000","last_undegraded":"2024-04-25T09:03:13.560270+0000","last_fullsized":"2024-04-25T09:03:13.560270+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:30.470993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"125'6","reported_seq":20,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.559825+0000","last_change":"2024-04-25T09:03:11.555911+0000","last_active":"2024-04-25T09:03:13.559825+0000","last_peered":"2024-04-25T09:03:13.559825+0000","last_clean":"2024-04-25T09:03:13.559825+0000","last_became_active":"2024-04-25T09:03:11.555621+0000","last_became_peered":"2024-04-25T09:03:11.555621+0000","last_unstale":"2024-04-25T09:03:13.559825+0000","last_undegraded":"2024-04-25T09:03:13.559825+0000","last_fullsized":"2024-04-25T09:03:13.559825+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:34.294732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.557887+0000","last_change":"2024-04-25T09:03:12.577051+0000","last_active":"2024-04-25T09:03:13.557887+0000","last_peered":"2024-04-25T09:03:13.557887+0000","last_clean":"2024-04-25T09:03:13.557887+0000","last_became_active":"2024-04-25T09:03:12.573832+0000","last_became_peered":"2024-04-25T09:03:12.573832+0000","last_unstale":"2024-04-25T09:03:13.557887+0000","last_undegraded":"2024-04-25T09:03:13.557887+0000","last_fullsized":"2024-04-25T09:03:13.557887+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:55.732499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.557701+0000","last_change":"2024-04-25T09:03:11.557885+0000","last_active":"2024-04-25T09:03:13.557701+0000","last_peered":"2024-04-25T09:03:13.557701+0000","last_clean":"2024-04-25T09:03:13.557701+0000","last_became_active":"2024-04-25T09:03:11.557528+0000","last_became_peered":"2024-04-25T09:03:11.557528+0000","last_unstale":"2024-04-25T09:03:13.557701+0000","last_undegraded":"2024-04-25T09:03:13.557701+0000","last_fullsized":"2024-04-25T09:03:13.557701+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:18.942742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564045+0000","last_change":"2024-04-25T09:03:12.580756+0000","last_active":"2024-04-25T09:03:13.564045+0000","last_peered":"2024-04-25T09:03:13.564045+0000","last_clean":"2024-04-25T09:03:13.564045+0000","last_became_active":"2024-04-25T09:03:12.580404+0000","last_became_peered":"2024-04-25T09:03:12.580404+0000","last_unstale":"2024-04-25T09:03:13.564045+0000","last_undegraded":"2024-04-25T09:03:13.564045+0000","last_fullsized":"2024-04-25T09:03:13.564045+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:11.599506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564780+0000","last_change":"2024-04-25T09:03:11.556814+0000","last_active":"2024-04-25T09:03:13.564780+0000","last_peered":"2024-04-25T09:03:13.564780+0000","last_clean":"2024-04-25T09:03:13.564780+0000","last_became_active":"2024-04-25T09:03:11.556332+0000","last_became_peered":"2024-04-25T09:03:11.556332+0000","last_unstale":"2024-04-25T09:03:13.564780+0000","last_undegraded":"2024-04-25T09:03:13.564780+0000","last_fullsized":"2024-04-25T09:03:13.564780+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:36.455422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.560081+0000","last_change":"2024-04-25T09:03:12.577482+0000","last_active":"2024-04-25T09:03:13.560081+0000","last_peered":"2024-04-25T09:03:13.560081+0000","last_clean":"2024-04-25T09:03:13.560081+0000","last_became_active":"2024-04-25T09:03:12.577206+0000","last_became_peered":"2024-04-25T09:03:12.577206+0000","last_unstale":"2024-04-25T09:03:13.560081+0000","last_undegraded":"2024-04-25T09:03:13.560081+0000","last_fullsized":"2024-04-25T09:03:13.560081+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:20.423551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"125'8","reported_seq":22,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.560415+0000","last_change":"2024-04-25T09:03:11.564910+0000","last_active":"2024-04-25T09:03:13.560415+0000","last_peered":"2024-04-25T09:03:13.560415+0000","last_clean":"2024-04-25T09:03:13.560415+0000","last_became_active":"2024-04-25T09:03:11.564659+0000","last_became_peered":"2024-04-25T09:03:11.564659+0000","last_unstale":"2024-04-25T09:03:13.560415+0000","last_undegraded":"2024-04-25T09:03:13.560415+0000","last_fullsized":"2024-04-25T09:03:13.560415+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:58.531074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.560614+0000","last_change":"2024-04-25T09:03:12.581103+0000","last_active":"2024-04-25T09:03:13.560614+0000","last_peered":"2024-04-25T09:03:13.560614+0000","last_clean":"2024-04-25T09:03:13.560614+0000","last_became_active":"2024-04-25T09:03:12.580757+0000","last_became_peered":"2024-04-25T09:03:12.580757+0000","last_unstale":"2024-04-25T09:03:13.560614+0000","last_undegraded":"2024-04-25T09:03:13.560614+0000","last_fullsized":"2024-04-25T09:03:13.560614+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:34.759105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"125'1","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565070+0000","last_change":"2024-04-25T09:03:11.553990+0000","last_active":"2024-04-25T09:03:13.565070+0000","last_peered":"2024-04-25T09:03:13.565070+0000","last_clean":"2024-04-25T09:03:13.565070+0000","last_became_active":"2024-04-25T09:03:11.553630+0000","last_became_peered":"2024-04-25T09:03:11.553630+0000","last_unstale":"2024-04-25T09:03:13.565070+0000","last_undegraded":"2024-04-25T09:03:13.565070+0000","last_fullsized":"2024-04-25T09:03:13.565070+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:59.986937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565047+0000","last_change":"2024-04-25T09:03:12.570244+0000","last_active":"2024-04-25T09:03:13.565047+0000","last_peered":"2024-04-25T09:03:13.565047+0000","last_clean":"2024-04-25T09:03:13.565047+0000","last_became_active":"2024-04-25T09:03:12.569981+0000","last_became_peered":"2024-04-25T09:03:12.569981+0000","last_unstale":"2024-04-25T09:03:13.565047+0000","last_undegraded":"2024-04-25T09:03:13.565047+0000","last_fullsized":"2024-04-25T09:03:13.565047+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:55.970296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"125'2","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564342+0000","last_change":"2024-04-25T09:03:11.562060+0000","last_active":"2024-04-25T09:03:13.564342+0000","last_peered":"2024-04-25T09:03:13.564342+0000","last_clean":"2024-04-25T09:03:13.564342+0000","last_became_active":"2024-04-25T09:03:11.561765+0000","last_became_peered":"2024-04-25T09:03:11.561765+0000","last_unstale":"2024-04-25T09:03:13.564342+0000","last_undegraded":"2024-04-25T09:03:13.564342+0000","last_fullsized":"2024-04-25T09:03:13.564342+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:16.560861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565595+0000","last_change":"2024-04-25T09:03:12.565792+0000","last_active":"2024-04-25T09:03:13.565595+0000","last_peered":"2024-04-25T09:03:13.565595+0000","last_clean":"2024-04-25T09:03:13.565595+0000","last_became_active":"2024-04-25T09:03:12.564946+0000","last_became_peered":"2024-04-25T09:03:12.564946+0000","last_unstale":"2024-04-25T09:03:13.565595+0000","last_undegraded":"2024-04-25T09:03:13.565595+0000","last_fullsized":"2024-04-25T09:03:13.565595+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:04.248103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"125'1","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565268+0000","last_change":"2024-04-25T09:03:11.563479+0000","last_active":"2024-04-25T09:03:13.565268+0000","last_peered":"2024-04-25T09:03:13.565268+0000","last_clean":"2024-04-25T09:03:13.565268+0000","last_became_active":"2024-04-25T09:03:11.563181+0000","last_became_peered":"2024-04-25T09:03:11.563181+0000","last_unstale":"2024-04-25T09:03:13.565268+0000","last_undegraded":"2024-04-25T09:03:13.565268+0000","last_fullsized":"2024-04-25T09:03:13.565268+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:29.328917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564893+0000","last_change":"2024-04-25T09:03:12.565255+0000","last_active":"2024-04-25T09:03:13.564893+0000","last_peered":"2024-04-25T09:03:13.564893+0000","last_clean":"2024-04-25T09:03:13.564893+0000","last_became_active":"2024-04-25T09:03:12.564920+0000","last_became_peered":"2024-04-25T09:03:12.564920+0000","last_unstale":"2024-04-25T09:03:13.564893+0000","last_undegraded":"2024-04-25T09:03:13.564893+0000","last_fullsized":"2024-04-25T09:03:13.564893+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:37.669623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564527+0000","last_change":"2024-04-25T09:03:11.562425+0000","last_active":"2024-04-25T09:03:13.564527+0000","last_peered":"2024-04-25T09:03:13.564527+0000","last_clean":"2024-04-25T09:03:13.564527+0000","last_became_active":"2024-04-25T09:03:11.562079+0000","last_became_peered":"2024-04-25T09:03:11.562079+0000","last_unstale":"2024-04-25T09:03:13.564527+0000","last_undegraded":"2024-04-25T09:03:13.564527+0000","last_fullsized":"2024-04-25T09:03:13.564527+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:07.278701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.557581+0000","last_change":"2024-04-25T09:03:12.573116+0000","last_active":"2024-04-25T09:03:13.557581+0000","last_peered":"2024-04-25T09:03:13.557581+0000","last_clean":"2024-04-25T09:03:13.557581+0000","last_became_active":"2024-04-25T09:03:12.572811+0000","last_became_peered":"2024-04-25T09:03:12.572811+0000","last_unstale":"2024-04-25T09:03:13.557581+0000","last_undegraded":"2024-04-25T09:03:13.557581+0000","last_fullsized":"2024-04-25T09:03:13.557581+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:25.875005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"125'2","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564827+0000","last_change":"2024-04-25T09:03:11.556065+0000","last_active":"2024-04-25T09:03:13.564827+0000","last_peered":"2024-04-25T09:03:13.564827+0000","last_clean":"2024-04-25T09:03:13.564827+0000","last_became_active":"2024-04-25T09:03:11.555763+0000","last_became_peered":"2024-04-25T09:03:11.555763+0000","last_unstale":"2024-04-25T09:03:13.564827+0000","last_undegraded":"2024-04-25T09:03:13.564827+0000","last_fullsized":"2024-04-25T09:03:13.564827+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:26.345004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565191+0000","last_change":"2024-04-25T09:03:12.575785+0000","last_active":"2024-04-25T09:03:13.565191+0000","last_peered":"2024-04-25T09:03:13.565191+0000","last_clean":"2024-04-25T09:03:13.565191+0000","last_became_active":"2024-04-25T09:03:12.575524+0000","last_became_peered":"2024-04-25T09:03:12.575524+0000","last_unstale":"2024-04-25T09:03:13.565191+0000","last_undegraded":"2024-04-25T09:03:13.565191+0000","last_fullsized":"2024-04-25T09:03:13.565191+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:37.669623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565589+0000","last_change":"2024-04-25T09:03:11.561560+0000","last_active":"2024-04-25T09:03:13.565589+0000","last_peered":"2024-04-25T09:03:13.565589+0000","last_clean":"2024-04-25T09:03:13.565589+0000","last_became_active":"2024-04-25T09:03:11.561254+0000","last_became_peered":"2024-04-25T09:03:11.561254+0000","last_unstale":"2024-04-25T09:03:13.565589+0000","last_undegraded":"2024-04-25T09:03:13.565589+0000","last_fullsized":"2024-04-25T09:03:13.565589+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:30.646577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565007+0000","last_change":"2024-04-25T09:03:12.568429+0000","last_active":"2024-04-25T09:03:13.565007+0000","last_peered":"2024-04-25T09:03:13.565007+0000","last_clean":"2024-04-25T09:03:13.565007+0000","last_became_active":"2024-04-25T09:03:12.567808+0000","last_became_peered":"2024-04-25T09:03:12.567808+0000","last_unstale":"2024-04-25T09:03:13.565007+0000","last_undegraded":"2024-04-25T09:03:13.565007+0000","last_fullsized":"2024-04-25T09:03:13.565007+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:46.582342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564841+0000","last_change":"2024-04-25T09:03:11.562799+0000","last_active":"2024-04-25T09:03:13.564841+0000","last_peered":"2024-04-25T09:03:13.564841+0000","last_clean":"2024-04-25T09:03:13.564841+0000","last_became_active":"2024-04-25T09:03:11.562502+0000","last_became_peered":"2024-04-25T09:03:11.562502+0000","last_unstale":"2024-04-25T09:03:13.564841+0000","last_undegraded":"2024-04-25T09:03:13.564841+0000","last_fullsized":"2024-04-25T09:03:13.564841+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:34.580056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.558970+0000","last_change":"2024-04-25T09:03:12.580220+0000","last_active":"2024-04-25T09:03:13.558970+0000","last_peered":"2024-04-25T09:03:13.558970+0000","last_clean":"2024-04-25T09:03:13.558970+0000","last_became_active":"2024-04-25T09:03:12.579734+0000","last_became_peered":"2024-04-25T09:03:12.579734+0000","last_unstale":"2024-04-25T09:03:13.558970+0000","last_undegraded":"2024-04-25T09:03:13.558970+0000","last_fullsized":"2024-04-25T09:03:13.558970+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:39.272779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564036+0000","last_change":"2024-04-25T09:03:11.555616+0000","last_active":"2024-04-25T09:03:13.564036+0000","last_peered":"2024-04-25T09:03:13.564036+0000","last_clean":"2024-04-25T09:03:13.564036+0000","last_became_active":"2024-04-25T09:03:11.555348+0000","last_became_peered":"2024-04-25T09:03:11.555348+0000","last_unstale":"2024-04-25T09:03:13.564036+0000","last_undegraded":"2024-04-25T09:03:13.564036+0000","last_fullsized":"2024-04-25T09:03:13.564036+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:31.565904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565448+0000","last_change":"2024-04-25T09:03:12.571157+0000","last_active":"2024-04-25T09:03:13.565448+0000","last_peered":"2024-04-25T09:03:13.565448+0000","last_clean":"2024-04-25T09:03:13.565448+0000","last_became_active":"2024-04-25T09:03:12.570889+0000","last_became_peered":"2024-04-25T09:03:12.570889+0000","last_unstale":"2024-04-25T09:03:13.565448+0000","last_undegraded":"2024-04-25T09:03:13.565448+0000","last_fullsized":"2024-04-25T09:03:13.565448+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:46.582342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"125'2","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.558009+0000","last_change":"2024-04-25T09:03:11.554360+0000","last_active":"2024-04-25T09:03:13.558009+0000","last_peered":"2024-04-25T09:03:13.558009+0000","last_clean":"2024-04-25T09:03:13.558009+0000","last_became_active":"2024-04-25T09:03:11.554049+0000","last_became_peered":"2024-04-25T09:03:11.554049+0000","last_unstale":"2024-04-25T09:03:13.558009+0000","last_undegraded":"2024-04-25T09:03:13.558009+0000","last_fullsized":"2024-04-25T09:03:13.558009+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:19.067295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.557897+0000","last_change":"2024-04-25T09:03:12.578875+0000","last_active":"2024-04-25T09:03:13.557897+0000","last_peered":"2024-04-25T09:03:13.557897+0000","last_clean":"2024-04-25T09:03:13.557897+0000","last_became_active":"2024-04-25T09:03:12.578534+0000","last_became_peered":"2024-04-25T09:03:12.578534+0000","last_unstale":"2024-04-25T09:03:13.557897+0000","last_undegraded":"2024-04-25T09:03:13.557897+0000","last_fullsized":"2024-04-25T09:03:13.557897+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:00.992252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"126'1","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.572739+0000","last_change":"2024-04-25T09:03:11.555057+0000","last_active":"2024-04-25T09:03:13.572739+0000","last_peered":"2024-04-25T09:03:13.572739+0000","last_clean":"2024-04-25T09:03:13.572739+0000","last_became_active":"2024-04-25T09:03:11.554262+0000","last_became_peered":"2024-04-25T09:03:11.554262+0000","last_unstale":"2024-04-25T09:03:13.572739+0000","last_undegraded":"2024-04-25T09:03:13.572739+0000","last_fullsized":"2024-04-25T09:03:13.572739+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:04.407967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564745+0000","last_change":"2024-04-25T09:03:12.568129+0000","last_active":"2024-04-25T09:03:13.564745+0000","last_peered":"2024-04-25T09:03:13.564745+0000","last_clean":"2024-04-25T09:03:13.564745+0000","last_became_active":"2024-04-25T09:03:12.567831+0000","last_became_peered":"2024-04-25T09:03:12.567831+0000","last_unstale":"2024-04-25T09:03:13.564745+0000","last_undegraded":"2024-04-25T09:03:13.564745+0000","last_fullsized":"2024-04-25T09:03:13.564745+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:34.759105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564989+0000","last_change":"2024-04-25T09:03:11.557168+0000","last_active":"2024-04-25T09:03:13.564989+0000","last_peered":"2024-04-25T09:03:13.564989+0000","last_clean":"2024-04-25T09:03:13.564989+0000","last_became_active":"2024-04-25T09:03:11.556736+0000","last_became_peered":"2024-04-25T09:03:11.556736+0000","last_unstale":"2024-04-25T09:03:13.564989+0000","last_undegraded":"2024-04-25T09:03:13.564989+0000","last_fullsized":"2024-04-25T09:03:13.564989+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:34.294732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565178+0000","last_change":"2024-04-25T09:03:12.565793+0000","last_active":"2024-04-25T09:03:13.565178+0000","last_peered":"2024-04-25T09:03:13.565178+0000","last_clean":"2024-04-25T09:03:13.565178+0000","last_became_active":"2024-04-25T09:03:12.565472+0000","last_became_peered":"2024-04-25T09:03:12.565472+0000","last_unstale":"2024-04-25T09:03:13.565178+0000","last_undegraded":"2024-04-25T09:03:13.565178+0000","last_fullsized":"2024-04-25T09:03:13.565178+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:34.759105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565359+0000","last_change":"2024-04-25T09:03:11.562034+0000","last_active":"2024-04-25T09:03:13.565359+0000","last_peered":"2024-04-25T09:03:13.565359+0000","last_clean":"2024-04-25T09:03:13.565359+0000","last_became_active":"2024-04-25T09:03:11.561322+0000","last_became_peered":"2024-04-25T09:03:11.561322+0000","last_unstale":"2024-04-25T09:03:13.565359+0000","last_undegraded":"2024-04-25T09:03:13.565359+0000","last_fullsized":"2024-04-25T09:03:13.565359+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:02.700874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.559373+0000","last_change":"2024-04-25T09:03:12.572807+0000","last_active":"2024-04-25T09:03:13.559373+0000","last_peered":"2024-04-25T09:03:13.559373+0000","last_clean":"2024-04-25T09:03:13.559373+0000","last_became_active":"2024-04-25T09:03:12.572529+0000","last_became_peered":"2024-04-25T09:03:12.572529+0000","last_unstale":"2024-04-25T09:03:13.559373+0000","last_undegraded":"2024-04-25T09:03:13.559373+0000","last_fullsized":"2024-04-25T09:03:13.559373+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:04.248103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564264+0000","last_change":"2024-04-25T09:03:11.555690+0000","last_active":"2024-04-25T09:03:13.564264+0000","last_peered":"2024-04-25T09:03:13.564264+0000","last_clean":"2024-04-25T09:03:13.564264+0000","last_became_active":"2024-04-25T09:03:11.555405+0000","last_became_peered":"2024-04-25T09:03:11.555405+0000","last_unstale":"2024-04-25T09:03:13.564264+0000","last_undegraded":"2024-04-25T09:03:13.564264+0000","last_fullsized":"2024-04-25T09:03:13.564264+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:04.512508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565112+0000","last_change":"2024-04-25T09:03:12.580451+0000","last_active":"2024-04-25T09:03:13.565112+0000","last_peered":"2024-04-25T09:03:13.565112+0000","last_clean":"2024-04-25T09:03:13.565112+0000","last_became_active":"2024-04-25T09:03:12.579960+0000","last_became_peered":"2024-04-25T09:03:12.579960+0000","last_unstale":"2024-04-25T09:03:13.565112+0000","last_undegraded":"2024-04-25T09:03:13.565112+0000","last_fullsized":"2024-04-25T09:03:13.565112+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:44.565613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.559382+0000","last_change":"2024-04-25T09:03:11.560584+0000","last_active":"2024-04-25T09:03:13.559382+0000","last_peered":"2024-04-25T09:03:13.559382+0000","last_clean":"2024-04-25T09:03:13.559382+0000","last_became_active":"2024-04-25T09:03:11.560262+0000","last_became_peered":"2024-04-25T09:03:11.560262+0000","last_unstale":"2024-04-25T09:03:13.559382+0000","last_undegraded":"2024-04-25T09:03:13.559382+0000","last_fullsized":"2024-04-25T09:03:13.559382+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:34.580056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565599+0000","last_change":"2024-04-25T09:03:12.579612+0000","last_active":"2024-04-25T09:03:13.565599+0000","last_peered":"2024-04-25T09:03:13.565599+0000","last_clean":"2024-04-25T09:03:13.565599+0000","last_became_active":"2024-04-25T09:03:12.579329+0000","last_became_peered":"2024-04-25T09:03:12.579329+0000","last_unstale":"2024-04-25T09:03:13.565599+0000","last_undegraded":"2024-04-25T09:03:13.565599+0000","last_fullsized":"2024-04-25T09:03:13.565599+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:03.706189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564937+0000","last_change":"2024-04-25T09:03:11.559802+0000","last_active":"2024-04-25T09:03:13.564937+0000","last_peered":"2024-04-25T09:03:13.564937+0000","last_clean":"2024-04-25T09:03:13.564937+0000","last_became_active":"2024-04-25T09:03:11.559526+0000","last_became_peered":"2024-04-25T09:03:11.559526+0000","last_unstale":"2024-04-25T09:03:13.564937+0000","last_undegraded":"2024-04-25T09:03:13.564937+0000","last_fullsized":"2024-04-25T09:03:13.564937+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:36.668225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564679+0000","last_change":"2024-04-25T09:03:12.567445+0000","last_active":"2024-04-25T09:03:13.564679+0000","last_peered":"2024-04-25T09:03:13.564679+0000","last_clean":"2024-04-25T09:03:13.564679+0000","last_became_active":"2024-04-25T09:03:12.567185+0000","last_became_peered":"2024-04-25T09:03:12.567185+0000","last_unstale":"2024-04-25T09:03:13.564679+0000","last_undegraded":"2024-04-25T09:03:13.564679+0000","last_fullsized":"2024-04-25T09:03:13.564679+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:11.725050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564135+0000","last_change":"2024-04-25T09:03:11.555006+0000","last_active":"2024-04-25T09:03:13.564135+0000","last_peered":"2024-04-25T09:03:13.564135+0000","last_clean":"2024-04-25T09:03:13.564135+0000","last_became_active":"2024-04-25T09:03:11.554708+0000","last_became_peered":"2024-04-25T09:03:11.554708+0000","last_unstale":"2024-04-25T09:03:13.564135+0000","last_undegraded":"2024-04-25T09:03:13.564135+0000","last_fullsized":"2024-04-25T09:03:13.564135+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:03.242788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.559291+0000","last_change":"2024-04-25T09:03:12.580004+0000","last_active":"2024-04-25T09:03:13.559291+0000","last_peered":"2024-04-25T09:03:13.559291+0000","last_clean":"2024-04-25T09:03:13.559291+0000","last_became_active":"2024-04-25T09:03:12.579636+0000","last_became_peered":"2024-04-25T09:03:12.579636+0000","last_unstale":"2024-04-25T09:03:13.559291+0000","last_undegraded":"2024-04-25T09:03:13.559291+0000","last_fullsized":"2024-04-25T09:03:13.559291+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:32.571219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"125'2","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.559900+0000","last_change":"2024-04-25T09:03:11.558886+0000","last_active":"2024-04-25T09:03:13.559900+0000","last_peered":"2024-04-25T09:03:13.559900+0000","last_clean":"2024-04-25T09:03:13.559900+0000","last_became_active":"2024-04-25T09:03:11.558545+0000","last_became_peered":"2024-04-25T09:03:11.558545+0000","last_unstale":"2024-04-25T09:03:13.559900+0000","last_undegraded":"2024-04-25T09:03:13.559900+0000","last_fullsized":"2024-04-25T09:03:13.559900+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:00.501342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":367,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.558838+0000","last_change":"2024-04-25T08:54:00.818044+0000","last_active":"2024-04-25T09:03:13.558838+0000","last_peered":"2024-04-25T09:03:13.558838+0000","last_clean":"2024-04-25T09:03:13.558838+0000","last_became_active":"2024-04-25T08:54:00.817098+0000","last_became_peered":"2024-04-25T08:54:00.817098+0000","last_unstale":"2024-04-25T09:03:13.558838+0000","last_undegraded":"2024-04-25T09:03:13.558838+0000","last_fullsized":"2024-04-25T09:03:13.558838+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_clean_scrub_stamp":"2024-04-25T08:53:59.739677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:48.129513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574961,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59684,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312156,"statfs":{"total":96636764160,"available":96575647744,"internally_reserved":0,"allocated":2179072,"data_stored":1272200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574962,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64660,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94307180,"statfs":{"total":96636764160,"available":96570552320,"internally_reserved":0,"allocated":1572864,"data_stored":681194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574964,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50760,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321080,"statfs":{"total":96636764160,"available":96584785920,"internally_reserved":0,"allocated":1568768,"data_stored":679694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574962,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68812,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303028,"statfs":{"total":96636764160,"available":96566300672,"internally_reserved":0,"allocated":1564672,"data_stored":679652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574962,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72988,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298852,"statfs":{"total":96636764160,"available":96562024448,"internally_reserved":0,"allocated":1581056,"data_stored":686042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574962,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46600,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325240,"statfs":{"total":96636764160,"available":96589045760,"internally_reserved":0,"allocated":1568768,"data_stored":680198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574962,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77136,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294704,"statfs":{"total":96636764160,"available":96557776896,"internally_reserved":0,"allocated":1572864,"data_stored":685882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574963,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47176,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324664,"statfs":{"total":96636764160,"available":96588455936,"internally_reserved":0,"allocated":2154496,"data_stored":1269950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:03:15.554 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:03:15.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.702+0000 7fbb8f534640 1 Processor -- start 2024-04-25T09:03:15.691 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.702+0000 7fbb8f534640 1 -- start start 2024-04-25T09:03:15.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.703+0000 7fbb8f534640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbb880cd010 0x7fbb880b4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.703+0000 7fbb8f534640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb880b2780 con 0x7fbb880cbfc0 2024-04-25T09:03:15.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.703+0000 7fbb8f534640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb880b2a60 con 0x7fbb880ccbb0 2024-04-25T09:03:15.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.703+0000 7fbb8f534640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb880b2d40 con 0x7fbb880cd010 2024-04-25T09:03:15.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.703+0000 7fbb8d2d0640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fbb880cbfc0 0x7fbb8806dc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:33560/0 (socket says 172.21.15.43:33560) 2024-04-25T09:03:15.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.703+0000 7fbb8d2d0640 1 -- 172.21.15.43:0/1527046577 learned_addr learned my addr 172.21.15.43:0/1527046577 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:03:15.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.703+0000 7fbb87fff640 1 -- 172.21.15.43:0/1527046577 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 619088403 0 0) 0x7fbb880b2780 con 0x7fbb880cbfc0 2024-04-25T09:03:15.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.703+0000 7fbb8dad1640 1 --2- 172.21.15.43:0/1527046577 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbb880cd010 0x7fbb880b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.704+0000 7fbb87fff640 1 -- 172.21.15.43:0/1527046577 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb70003520 con 0x7fbb880cbfc0 2024-04-25T09:03:15.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.704+0000 7fbb87fff640 1 -- 172.21.15.43:0/1527046577 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 238182492 0 0) 0x7fbb880b2a60 con 0x7fbb880ccbb0 2024-04-25T09:03:15.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.704+0000 7fbb87fff640 1 -- 172.21.15.43:0/1527046577 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb880b2780 con 0x7fbb880ccbb0 2024-04-25T09:03:15.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.704+0000 7fbb87fff640 1 -- 172.21.15.43:0/1527046577 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3818979810 0 0) 0x7fbb70003520 con 0x7fbb880cbfc0 2024-04-25T09:03:15.695 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.704+0000 7fbb87fff640 1 -- 172.21.15.43:0/1527046577 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbb880cd010 msgr2=0x7fbb880b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:15.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.704+0000 7fbb87fff640 1 --2- 172.21.15.43:0/1527046577 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbb880cd010 0x7fbb880b4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.704+0000 7fbb87fff640 1 -- 172.21.15.43:0/1527046577 >> v1:172.21.15.193:6789/0 conn(0x7fbb880ccbb0 legacy=0x7fbb880b3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.704+0000 7fbb87fff640 1 -- 172.21.15.43:0/1527046577 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb880b3020 con 0x7fbb880cbfc0 2024-04-25T09:03:15.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.704+0000 7fbb87fff640 1 -- 172.21.15.43:0/1527046577 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7fbb80001970 con 0x7fbb880cbfc0 2024-04-25T09:03:15.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.704+0000 7fbb87fff640 1 -- 172.21.15.43:0/1527046577 >> v1:172.21.15.43:6789/0 conn(0x7fbb880cbfc0 legacy=0x7fbb8806dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.704+0000 7fbb87fff640 1 --2- 172.21.15.43:0/1527046577 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbb880cd010 0x7fbb70003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.704+0000 7fbb87fff640 1 --2- 172.21.15.43:0/1527046577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb70004120 0x7fbb70004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.704+0000 7fbb87fff640 1 --2- 172.21.15.43:0/1527046577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb70004d40 0x7fbb700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.704+0000 7fbb87fff640 1 -- 172.21.15.43:0/1527046577 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb880b2d40 con 0x7fbb70004120 2024-04-25T09:03:15.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.704+0000 7fbb87fff640 1 -- 172.21.15.43:0/1527046577 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7000bb00 con 0x7fbb70004d40 2024-04-25T09:03:15.696 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.704+0000 7fbb87fff640 1 -- 172.21.15.43:0/1527046577 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7000bd00 con 0x7fbb880cd010 2024-04-25T09:03:15.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.704+0000 7fbb8cacf640 1 --2- 172.21.15.43:0/1527046577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb70004120 0x7fbb70004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.704+0000 7fbb8d2d0640 1 --2- 172.21.15.43:0/1527046577 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbb880cd010 0x7fbb70003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.704+0000 7fbb8dad1640 1 --2- 172.21.15.43:0/1527046577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb70004d40 0x7fbb700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.704+0000 7fbb8cacf640 1 -- 172.21.15.43:0/1527046577 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbb880cd010 msgr2=0x7fbb70003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:15.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.704+0000 7fbb8cacf640 1 --2- 172.21.15.43:0/1527046577 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbb880cd010 0x7fbb70003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.704+0000 7fbb8cacf640 1 -- 172.21.15.43:0/1527046577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb70004d40 msgr2=0x7fbb700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:15.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.704+0000 7fbb8cacf640 1 --2- 172.21.15.43:0/1527046577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb70004d40 0x7fbb700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.704+0000 7fbb8cacf640 1 -- 172.21.15.43:0/1527046577 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb70003520 con 0x7fbb70004120 2024-04-25T09:03:15.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.704+0000 7fbb8dad1640 1 --2- 172.21.15.43:0/1527046577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb70004d40 0x7fbb700051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:15.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.705+0000 7fbb8cacf640 1 --2- 172.21.15.43:0/1527046577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb70004120 0x7fbb70004800 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c0018d0 tx=0x7fbb70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aca8dafc50f168b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:15.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.705+0000 7fbb87fff640 1 -- 172.21.15.43:0/1527046577 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c0014f0 con 0x7fbb70004120 2024-04-25T09:03:15.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.705+0000 7fbb87fff640 1 -- 172.21.15.43:0/1527046577 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb7c0052d0 con 0x7fbb70004120 2024-04-25T09:03:15.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.705+0000 7fbb87fff640 1 -- 172.21.15.43:0/1527046577 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c02abe0 con 0x7fbb70004120 2024-04-25T09:03:15.697 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.706+0000 7fbb8f534640 1 -- 172.21.15.43:0/1527046577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb70004120 msgr2=0x7fbb70004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.706+0000 7fbb8f534640 1 --2- 172.21.15.43:0/1527046577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb70004120 0x7fbb70004800 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c0018d0 tx=0x7fbb70005830 comp rx=0 tx=0).stop 2024-04-25T09:03:15.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.706+0000 7fbb8dad1640 1 -- 172.21.15.43:0/1527046577 reap_dead start 2024-04-25T09:03:15.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.706+0000 7fbb8f534640 1 -- 172.21.15.43:0/1527046577 shutdown_connections 2024-04-25T09:03:15.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.706+0000 7fbb8f534640 1 -- 172.21.15.43:0/1527046577 >> 172.21.15.43:0/1527046577 conn(0x7fbb880b1480 msgr2=0x7fbb880b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:15.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.706+0000 7fbb8f534640 1 -- 172.21.15.43:0/1527046577 shutdown_connections 2024-04-25T09:03:15.698 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.706+0000 7fbb8f534640 1 -- 172.21.15.43:0/1527046577 wait complete. 2024-04-25T09:03:15.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.707+0000 7fbb8f534640 1 Processor -- start 2024-04-25T09:03:15.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.707+0000 7fbb8f534640 1 -- start start 2024-04-25T09:03:15.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.708+0000 7fbb8f534640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb70004120 0x7fbb8813a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.708+0000 7fbb8f534640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb88156730 0x7fbb8815acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.708+0000 7fbb8f534640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbb8815b210 0x7fbb8806d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.708+0000 7fbb8f534640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb880cd4a0 con 0x7fbb70004120 2024-04-25T09:03:15.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.708+0000 7fbb8f534640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb880b35e0 con 0x7fbb88156730 2024-04-25T09:03:15.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.708+0000 7fbb8f534640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb880cc3a0 con 0x7fbb8815b210 2024-04-25T09:03:15.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.708+0000 7fbb8d2d0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb70004120 0x7fbb8813a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.708+0000 7fbb8d2d0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb70004120 0x7fbb8813a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45274/0 (socket says 172.21.15.43:45274) 2024-04-25T09:03:15.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.708+0000 7fbb8d2d0640 1 -- 172.21.15.43:0/568770482 learned_addr learned my addr 172.21.15.43:0/568770482 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:15.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.708+0000 7fbb8cacf640 1 --2- 172.21.15.43:0/568770482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb88156730 0x7fbb8815acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.708+0000 7fbb8dad1640 1 --2- 172.21.15.43:0/568770482 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbb8815b210 0x7fbb8806d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.708+0000 7fbb8d2d0640 1 -- 172.21.15.43:0/568770482 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbb8815b210 msgr2=0x7fbb8806d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.708+0000 7fbb8d2d0640 1 --2- 172.21.15.43:0/568770482 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbb8815b210 0x7fbb8806d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.708+0000 7fbb8d2d0640 1 -- 172.21.15.43:0/568770482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb88156730 msgr2=0x7fbb8815acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.708+0000 7fbb8d2d0640 1 --2- 172.21.15.43:0/568770482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb88156730 0x7fbb8815acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.708+0000 7fbb8d2d0640 1 -- 172.21.15.43:0/568770482 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb8813d290 con 0x7fbb70004120 2024-04-25T09:03:15.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.708+0000 7fbb8cacf640 1 --2- 172.21.15.43:0/568770482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb88156730 0x7fbb8815acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:15.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.708+0000 7fbb8dad1640 1 --2- 172.21.15.43:0/568770482 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbb8815b210 0x7fbb8806d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:15.702 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.708+0000 7fbb8d2d0640 1 --2- 172.21.15.43:0/568770482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb70004120 0x7fbb8813a8f0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fbb800079d0 tx=0x7fbb8000a600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:15.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.709+0000 7fbb85ffb640 1 -- 172.21.15.43:0/568770482 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb80017020 con 0x7fbb70004120 2024-04-25T09:03:15.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.709+0000 7fbb85ffb640 1 -- 172.21.15.43:0/568770482 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb80001950 con 0x7fbb70004120 2024-04-25T09:03:15.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.709+0000 7fbb8f534640 1 -- 172.21.15.43:0/568770482 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb8813d570 con 0x7fbb70004120 2024-04-25T09:03:15.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.709+0000 7fbb85ffb640 1 -- 172.21.15.43:0/568770482 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb80013630 con 0x7fbb70004120 2024-04-25T09:03:15.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.709+0000 7fbb8f534640 1 -- 172.21.15.43:0/568770482 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb881384b0 con 0x7fbb70004120 2024-04-25T09:03:15.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.710+0000 7fbb8f534640 1 -- 172.21.15.43:0/568770482 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb88074ef0 con 0x7fbb70004120 2024-04-25T09:03:15.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.710+0000 7fbb85ffb640 1 -- 172.21.15.43:0/568770482 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbb8001a020 con 0x7fbb70004120 2024-04-25T09:03:15.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.711+0000 7fbb85ffb640 1 --2- 172.21.15.43:0/568770482 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbb7406b4d0 0x7fbb7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:15.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.711+0000 7fbb85ffb640 1 -- 172.21.15.43:0/568770482 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb8008ab60 con 0x7fbb70004120 2024-04-25T09:03:15.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.711+0000 7fbb8cacf640 1 --2- 172.21.15.43:0/568770482 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbb7406b4d0 0x7fbb7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:15.703 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.712+0000 7fbb8cacf640 1 --2- 172.21.15.43:0/568770482 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbb7406b4d0 0x7fbb7406d990 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c032000 tx=0x7fbb7c009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:15.704 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.715+0000 7fbb85ffb640 1 -- 172.21.15.43:0/568770482 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbb80021020 con 0x7fbb70004120 2024-04-25T09:03:15.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.892+0000 7fbb8f534640 1 -- 172.21.15.43:0/568770482 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbb8813da90 con 0x7fbb7406b4d0 2024-04-25T09:03:15.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.899+0000 7fbb85ffb640 1 -- 172.21.15.43:0/568770482 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252483 (secure 0 0 0) 0x7fbb8813da90 con 0x7fbb7406b4d0 2024-04-25T09:03:15.889 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:03:15.891 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-25T09:03:15.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.901+0000 7fbb8f534640 1 -- 172.21.15.43:0/568770482 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbb7406b4d0 msgr2=0x7fbb7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.901+0000 7fbb8f534640 1 --2- 172.21.15.43:0/568770482 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbb7406b4d0 0x7fbb7406d990 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c032000 tx=0x7fbb7c009030 comp rx=0 tx=0).stop 2024-04-25T09:03:15.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.901+0000 7fbb8f534640 1 -- 172.21.15.43:0/568770482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb70004120 msgr2=0x7fbb8813a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:15.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.901+0000 7fbb8f534640 1 --2- 172.21.15.43:0/568770482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb70004120 0x7fbb8813a8f0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fbb800079d0 tx=0x7fbb8000a600 comp rx=0 tx=0).stop 2024-04-25T09:03:15.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.901+0000 7fbb8f534640 1 -- 172.21.15.43:0/568770482 shutdown_connections 2024-04-25T09:03:15.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.901+0000 7fbb8f534640 1 --2- 172.21.15.43:0/568770482 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbb8815b210 0x7fbb8806d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.901+0000 7fbb8f534640 1 --2- 172.21.15.43:0/568770482 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbb7406b4d0 0x7fbb7406d990 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.901+0000 7fbb8f534640 1 --2- 172.21.15.43:0/568770482 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb88156730 0x7fbb8815acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.901+0000 7fbb8f534640 1 --2- 172.21.15.43:0/568770482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb70004120 0x7fbb8813a8f0 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:15.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.901+0000 7fbb8f534640 1 -- 172.21.15.43:0/568770482 >> 172.21.15.43:0/568770482 conn(0x7fbb880b1480 msgr2=0x7fbb880af3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:15.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.901+0000 7fbb8f534640 1 -- 172.21.15.43:0/568770482 shutdown_connections 2024-04-25T09:03:15.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:15.901+0000 7fbb8f534640 1 -- 172.21.15.43:0/568770482 wait complete. 2024-04-25T09:03:15.901 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":398,"stamp":"2024-04-25T09:03:15.820505+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":164,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":487816,"kb_used_data":13440,"kb_used_omap":12,"kb_used_meta":474227,"kb_avail":754486904,"statfs":{"total":773094113280,"available":772594589696,"internally_reserved":0,"allocated":13762560,"data_stored":6634812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":485609009},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.279298"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.557974+0000","last_change":"2024-04-25T09:03:12.573783+0000","last_active":"2024-04-25T09:03:13.557974+0000","last_peered":"2024-04-25T09:03:13.557974+0000","last_clean":"2024-04-25T09:03:13.557974+0000","last_became_active":"2024-04-25T09:03:12.573517+0000","last_became_peered":"2024-04-25T09:03:12.573517+0000","last_unstale":"2024-04-25T09:03:13.557974+0000","last_undegraded":"2024-04-25T09:03:13.557974+0000","last_fullsized":"2024-04-25T09:03:13.557974+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:37.081732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564429+0000","last_change":"2024-04-25T09:03:12.572481+0000","last_active":"2024-04-25T09:03:13.564429+0000","last_peered":"2024-04-25T09:03:13.564429+0000","last_clean":"2024-04-25T09:03:13.564429+0000","last_became_active":"2024-04-25T09:03:12.571979+0000","last_became_peered":"2024-04-25T09:03:12.571979+0000","last_unstale":"2024-04-25T09:03:13.564429+0000","last_undegraded":"2024-04-25T09:03:13.564429+0000","last_fullsized":"2024-04-25T09:03:13.564429+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:28.189475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.559540+0000","last_change":"2024-04-25T09:03:12.574900+0000","last_active":"2024-04-25T09:03:13.559540+0000","last_peered":"2024-04-25T09:03:13.559540+0000","last_clean":"2024-04-25T09:03:13.559540+0000","last_became_active":"2024-04-25T09:03:12.574542+0000","last_became_peered":"2024-04-25T09:03:12.574542+0000","last_unstale":"2024-04-25T09:03:13.559540+0000","last_undegraded":"2024-04-25T09:03:13.559540+0000","last_fullsized":"2024-04-25T09:03:13.559540+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:11.599506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.557581+0000","last_change":"2024-04-25T09:03:12.576885+0000","last_active":"2024-04-25T09:03:13.557581+0000","last_peered":"2024-04-25T09:03:13.557581+0000","last_clean":"2024-04-25T09:03:13.557581+0000","last_became_active":"2024-04-25T09:03:12.576601+0000","last_became_peered":"2024-04-25T09:03:12.576601+0000","last_unstale":"2024-04-25T09:03:13.557581+0000","last_undegraded":"2024-04-25T09:03:13.557581+0000","last_fullsized":"2024-04-25T09:03:13.557581+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:37.484044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.559774+0000","last_change":"2024-04-25T09:03:12.572728+0000","last_active":"2024-04-25T09:03:13.559774+0000","last_peered":"2024-04-25T09:03:13.559774+0000","last_clean":"2024-04-25T09:03:13.559774+0000","last_became_active":"2024-04-25T09:03:12.572176+0000","last_became_peered":"2024-04-25T09:03:12.572176+0000","last_unstale":"2024-04-25T09:03:13.559774+0000","last_undegraded":"2024-04-25T09:03:13.559774+0000","last_fullsized":"2024-04-25T09:03:13.559774+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:17.566176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565020+0000","last_change":"2024-04-25T09:03:12.584575+0000","last_active":"2024-04-25T09:03:13.565020+0000","last_peered":"2024-04-25T09:03:13.565020+0000","last_clean":"2024-04-25T09:03:13.565020+0000","last_became_active":"2024-04-25T09:03:12.583521+0000","last_became_peered":"2024-04-25T09:03:12.583521+0000","last_unstale":"2024-04-25T09:03:13.565020+0000","last_undegraded":"2024-04-25T09:03:13.565020+0000","last_fullsized":"2024-04-25T09:03:13.565020+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:11.725050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564460+0000","last_change":"2024-04-25T09:03:12.578595+0000","last_active":"2024-04-25T09:03:13.564460+0000","last_peered":"2024-04-25T09:03:13.564460+0000","last_clean":"2024-04-25T09:03:13.564460+0000","last_became_active":"2024-04-25T09:03:12.578291+0000","last_became_peered":"2024-04-25T09:03:12.578291+0000","last_unstale":"2024-04-25T09:03:13.564460+0000","last_undegraded":"2024-04-25T09:03:13.564460+0000","last_fullsized":"2024-04-25T09:03:13.564460+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:28.189475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.559998+0000","last_change":"2024-04-25T09:03:12.580399+0000","last_active":"2024-04-25T09:03:13.559998+0000","last_peered":"2024-04-25T09:03:13.559998+0000","last_clean":"2024-04-25T09:03:13.559998+0000","last_became_active":"2024-04-25T09:03:12.579505+0000","last_became_peered":"2024-04-25T09:03:12.579505+0000","last_unstale":"2024-04-25T09:03:13.559998+0000","last_undegraded":"2024-04-25T09:03:13.559998+0000","last_fullsized":"2024-04-25T09:03:13.559998+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:56.534408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.559529+0000","last_change":"2024-04-25T09:03:12.575257+0000","last_active":"2024-04-25T09:03:13.559529+0000","last_peered":"2024-04-25T09:03:13.559529+0000","last_clean":"2024-04-25T09:03:13.559529+0000","last_became_active":"2024-04-25T09:03:12.574353+0000","last_became_peered":"2024-04-25T09:03:12.574353+0000","last_unstale":"2024-04-25T09:03:13.559529+0000","last_undegraded":"2024-04-25T09:03:13.559529+0000","last_fullsized":"2024-04-25T09:03:13.559529+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:51.666701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564568+0000","last_change":"2024-04-25T09:03:12.572243+0000","last_active":"2024-04-25T09:03:13.564568+0000","last_peered":"2024-04-25T09:03:13.564568+0000","last_clean":"2024-04-25T09:03:13.564568+0000","last_became_active":"2024-04-25T09:03:12.571635+0000","last_became_peered":"2024-04-25T09:03:12.571635+0000","last_unstale":"2024-04-25T09:03:13.564568+0000","last_undegraded":"2024-04-25T09:03:13.564568+0000","last_fullsized":"2024-04-25T09:03:13.564568+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:20.423551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565896+0000","last_change":"2024-04-25T09:03:12.566485+0000","last_active":"2024-04-25T09:03:13.565896+0000","last_peered":"2024-04-25T09:03:13.565896+0000","last_clean":"2024-04-25T09:03:13.565896+0000","last_became_active":"2024-04-25T09:03:12.565930+0000","last_became_peered":"2024-04-25T09:03:12.565930+0000","last_unstale":"2024-04-25T09:03:13.565896+0000","last_undegraded":"2024-04-25T09:03:13.565896+0000","last_fullsized":"2024-04-25T09:03:13.565896+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:11.963001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.559710+0000","last_change":"2024-04-25T09:03:12.575431+0000","last_active":"2024-04-25T09:03:13.559710+0000","last_peered":"2024-04-25T09:03:13.559710+0000","last_clean":"2024-04-25T09:03:13.559710+0000","last_became_active":"2024-04-25T09:03:12.575178+0000","last_became_peered":"2024-04-25T09:03:12.575178+0000","last_unstale":"2024-04-25T09:03:13.559710+0000","last_undegraded":"2024-04-25T09:03:13.559710+0000","last_fullsized":"2024-04-25T09:03:13.559710+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:28.189475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.557779+0000","last_change":"2024-04-25T09:03:12.582827+0000","last_active":"2024-04-25T09:03:13.557779+0000","last_peered":"2024-04-25T09:03:13.557779+0000","last_clean":"2024-04-25T09:03:13.557779+0000","last_became_active":"2024-04-25T09:03:12.582533+0000","last_became_peered":"2024-04-25T09:03:12.582533+0000","last_unstale":"2024-04-25T09:03:13.557779+0000","last_undegraded":"2024-04-25T09:03:13.557779+0000","last_fullsized":"2024-04-25T09:03:13.557779+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:11.089699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564419+0000","last_change":"2024-04-25T09:03:12.564011+0000","last_active":"2024-04-25T09:03:13.564419+0000","last_peered":"2024-04-25T09:03:13.564419+0000","last_clean":"2024-04-25T09:03:13.564419+0000","last_became_active":"2024-04-25T09:03:12.563690+0000","last_became_peered":"2024-04-25T09:03:12.563690+0000","last_unstale":"2024-04-25T09:03:13.564419+0000","last_undegraded":"2024-04-25T09:03:13.564419+0000","last_fullsized":"2024-04-25T09:03:13.564419+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:36.514775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.557487+0000","last_change":"2024-04-25T09:03:12.579378+0000","last_active":"2024-04-25T09:03:13.557487+0000","last_peered":"2024-04-25T09:03:13.557487+0000","last_clean":"2024-04-25T09:03:13.557487+0000","last_became_active":"2024-04-25T09:03:12.578782+0000","last_became_peered":"2024-04-25T09:03:12.578782+0000","last_unstale":"2024-04-25T09:03:13.557487+0000","last_undegraded":"2024-04-25T09:03:13.557487+0000","last_fullsized":"2024-04-25T09:03:13.557487+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:39.449290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565429+0000","last_change":"2024-04-25T09:03:12.576695+0000","last_active":"2024-04-25T09:03:13.565429+0000","last_peered":"2024-04-25T09:03:13.565429+0000","last_clean":"2024-04-25T09:03:13.565429+0000","last_became_active":"2024-04-25T09:03:12.575847+0000","last_became_peered":"2024-04-25T09:03:12.575847+0000","last_unstale":"2024-04-25T09:03:13.565429+0000","last_undegraded":"2024-04-25T09:03:13.565429+0000","last_fullsized":"2024-04-25T09:03:13.565429+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:32.571219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564802+0000","last_change":"2024-04-25T09:03:12.568985+0000","last_active":"2024-04-25T09:03:13.564802+0000","last_peered":"2024-04-25T09:03:13.564802+0000","last_clean":"2024-04-25T09:03:13.564802+0000","last_became_active":"2024-04-25T09:03:12.568696+0000","last_became_peered":"2024-04-25T09:03:12.568696+0000","last_unstale":"2024-04-25T09:03:13.564802+0000","last_undegraded":"2024-04-25T09:03:13.564802+0000","last_fullsized":"2024-04-25T09:03:13.564802+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:20.423551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565061+0000","last_change":"2024-04-25T09:03:12.575681+0000","last_active":"2024-04-25T09:03:13.565061+0000","last_peered":"2024-04-25T09:03:13.565061+0000","last_clean":"2024-04-25T09:03:13.565061+0000","last_became_active":"2024-04-25T09:03:12.575376+0000","last_became_peered":"2024-04-25T09:03:12.575376+0000","last_unstale":"2024-04-25T09:03:13.565061+0000","last_undegraded":"2024-04-25T09:03:13.565061+0000","last_fullsized":"2024-04-25T09:03:13.565061+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:01:06.928915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565962+0000","last_change":"2024-04-25T09:03:12.566203+0000","last_active":"2024-04-25T09:03:13.565962+0000","last_peered":"2024-04-25T09:03:13.565962+0000","last_clean":"2024-04-25T09:03:13.565962+0000","last_became_active":"2024-04-25T09:03:12.565560+0000","last_became_peered":"2024-04-25T09:03:12.565560+0000","last_unstale":"2024-04-25T09:03:13.565962+0000","last_undegraded":"2024-04-25T09:03:13.565962+0000","last_fullsized":"2024-04-25T09:03:13.565962+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:40.106983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565712+0000","last_change":"2024-04-25T09:03:12.571546+0000","last_active":"2024-04-25T09:03:13.565712+0000","last_peered":"2024-04-25T09:03:13.565712+0000","last_clean":"2024-04-25T09:03:13.565712+0000","last_became_active":"2024-04-25T09:03:12.571156+0000","last_became_peered":"2024-04-25T09:03:12.571156+0000","last_unstale":"2024-04-25T09:03:13.565712+0000","last_undegraded":"2024-04-25T09:03:13.565712+0000","last_fullsized":"2024-04-25T09:03:13.565712+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:17.566176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.559626+0000","last_change":"2024-04-25T09:03:12.572498+0000","last_active":"2024-04-25T09:03:13.559626+0000","last_peered":"2024-04-25T09:03:13.559626+0000","last_clean":"2024-04-25T09:03:13.559626+0000","last_became_active":"2024-04-25T09:03:12.571778+0000","last_became_peered":"2024-04-25T09:03:12.571778+0000","last_unstale":"2024-04-25T09:03:13.559626+0000","last_undegraded":"2024-04-25T09:03:13.559626+0000","last_fullsized":"2024-04-25T09:03:13.559626+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:03.706189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565717+0000","last_change":"2024-04-25T09:03:12.566180+0000","last_active":"2024-04-25T09:03:13.565717+0000","last_peered":"2024-04-25T09:03:13.565717+0000","last_clean":"2024-04-25T09:03:13.565717+0000","last_became_active":"2024-04-25T09:03:12.565888+0000","last_became_peered":"2024-04-25T09:03:12.565888+0000","last_unstale":"2024-04-25T09:03:13.565717+0000","last_undegraded":"2024-04-25T09:03:13.565717+0000","last_fullsized":"2024-04-25T09:03:13.565717+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:07.938843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.560308+0000","last_change":"2024-04-25T09:03:12.576813+0000","last_active":"2024-04-25T09:03:13.560308+0000","last_peered":"2024-04-25T09:03:13.560308+0000","last_clean":"2024-04-25T09:03:13.560308+0000","last_became_active":"2024-04-25T09:03:12.576552+0000","last_became_peered":"2024-04-25T09:03:12.576552+0000","last_unstale":"2024-04-25T09:03:13.560308+0000","last_undegraded":"2024-04-25T09:03:13.560308+0000","last_fullsized":"2024-04-25T09:03:13.560308+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:24.444267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564059+0000","last_change":"2024-04-25T09:03:12.570253+0000","last_active":"2024-04-25T09:03:13.564059+0000","last_peered":"2024-04-25T09:03:13.564059+0000","last_clean":"2024-04-25T09:03:13.564059+0000","last_became_active":"2024-04-25T09:03:12.569969+0000","last_became_peered":"2024-04-25T09:03:12.569969+0000","last_unstale":"2024-04-25T09:03:13.564059+0000","last_undegraded":"2024-04-25T09:03:13.564059+0000","last_fullsized":"2024-04-25T09:03:13.564059+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:05.513864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564716+0000","last_change":"2024-04-25T09:03:12.569197+0000","last_active":"2024-04-25T09:03:13.564716+0000","last_peered":"2024-04-25T09:03:13.564716+0000","last_clean":"2024-04-25T09:03:13.564716+0000","last_became_active":"2024-04-25T09:03:12.567874+0000","last_became_peered":"2024-04-25T09:03:12.567874+0000","last_unstale":"2024-04-25T09:03:13.564716+0000","last_undegraded":"2024-04-25T09:03:13.564716+0000","last_fullsized":"2024-04-25T09:03:13.564716+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:24.444267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565202+0000","last_change":"2024-04-25T09:03:12.578579+0000","last_active":"2024-04-25T09:03:13.565202+0000","last_peered":"2024-04-25T09:03:13.565202+0000","last_clean":"2024-04-25T09:03:13.565202+0000","last_became_active":"2024-04-25T09:03:12.578313+0000","last_became_peered":"2024-04-25T09:03:12.578313+0000","last_unstale":"2024-04-25T09:03:13.565202+0000","last_undegraded":"2024-04-25T09:03:13.565202+0000","last_fullsized":"2024-04-25T09:03:13.565202+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:46:32.511163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.557752+0000","last_change":"2024-04-25T09:03:12.573508+0000","last_active":"2024-04-25T09:03:13.557752+0000","last_peered":"2024-04-25T09:03:13.557752+0000","last_clean":"2024-04-25T09:03:13.557752+0000","last_became_active":"2024-04-25T09:03:12.572781+0000","last_became_peered":"2024-04-25T09:03:12.572781+0000","last_unstale":"2024-04-25T09:03:13.557752+0000","last_undegraded":"2024-04-25T09:03:13.557752+0000","last_fullsized":"2024-04-25T09:03:13.557752+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:19.906821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564544+0000","last_change":"2024-04-25T09:03:12.569290+0000","last_active":"2024-04-25T09:03:13.564544+0000","last_peered":"2024-04-25T09:03:13.564544+0000","last_clean":"2024-04-25T09:03:13.564544+0000","last_became_active":"2024-04-25T09:03:12.568972+0000","last_became_peered":"2024-04-25T09:03:12.568972+0000","last_unstale":"2024-04-25T09:03:13.564544+0000","last_undegraded":"2024-04-25T09:03:13.564544+0000","last_fullsized":"2024-04-25T09:03:13.564544+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:36.157007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.557770+0000","last_change":"2024-04-25T09:03:12.579246+0000","last_active":"2024-04-25T09:03:13.557770+0000","last_peered":"2024-04-25T09:03:13.557770+0000","last_clean":"2024-04-25T09:03:13.557770+0000","last_became_active":"2024-04-25T09:03:12.578947+0000","last_became_peered":"2024-04-25T09:03:12.578947+0000","last_unstale":"2024-04-25T09:03:13.557770+0000","last_undegraded":"2024-04-25T09:03:13.557770+0000","last_fullsized":"2024-04-25T09:03:13.557770+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:27.350319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.557654+0000","last_change":"2024-04-25T09:03:12.575982+0000","last_active":"2024-04-25T09:03:13.557654+0000","last_peered":"2024-04-25T09:03:13.557654+0000","last_clean":"2024-04-25T09:03:13.557654+0000","last_became_active":"2024-04-25T09:03:12.575692+0000","last_became_peered":"2024-04-25T09:03:12.575692+0000","last_unstale":"2024-04-25T09:03:13.557654+0000","last_undegraded":"2024-04-25T09:03:13.557654+0000","last_fullsized":"2024-04-25T09:03:13.557654+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:33.677525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.559192+0000","last_change":"2024-04-25T09:03:12.581173+0000","last_active":"2024-04-25T09:03:13.559192+0000","last_peered":"2024-04-25T09:03:13.559192+0000","last_clean":"2024-04-25T09:03:13.559192+0000","last_became_active":"2024-04-25T09:03:12.580291+0000","last_became_peered":"2024-04-25T09:03:12.580291+0000","last_unstale":"2024-04-25T09:03:13.559192+0000","last_undegraded":"2024-04-25T09:03:13.559192+0000","last_fullsized":"2024-04-25T09:03:13.559192+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:47.056165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564516+0000","last_change":"2024-04-25T09:03:12.573970+0000","last_active":"2024-04-25T09:03:13.564516+0000","last_peered":"2024-04-25T09:03:13.564516+0000","last_clean":"2024-04-25T09:03:13.564516+0000","last_became_active":"2024-04-25T09:03:12.573555+0000","last_became_peered":"2024-04-25T09:03:12.573555+0000","last_unstale":"2024-04-25T09:03:13.564516+0000","last_undegraded":"2024-04-25T09:03:13.564516+0000","last_fullsized":"2024-04-25T09:03:13.564516+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:56.534408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565367+0000","last_change":"2024-04-25T09:03:12.570467+0000","last_active":"2024-04-25T09:03:13.565367+0000","last_peered":"2024-04-25T09:03:13.565367+0000","last_clean":"2024-04-25T09:03:13.565367+0000","last_became_active":"2024-04-25T09:03:12.570161+0000","last_became_peered":"2024-04-25T09:03:12.570161+0000","last_unstale":"2024-04-25T09:03:13.565367+0000","last_undegraded":"2024-04-25T09:03:13.565367+0000","last_fullsized":"2024-04-25T09:03:13.565367+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:39.272779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"125'4","reported_seq":18,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.559664+0000","last_change":"2024-04-25T09:03:11.554834+0000","last_active":"2024-04-25T09:03:13.559664+0000","last_peered":"2024-04-25T09:03:13.559664+0000","last_clean":"2024-04-25T09:03:13.559664+0000","last_became_active":"2024-04-25T09:03:11.554552+0000","last_became_peered":"2024-04-25T09:03:11.554552+0000","last_unstale":"2024-04-25T09:03:13.559664+0000","last_undegraded":"2024-04-25T09:03:13.559664+0000","last_fullsized":"2024-04-25T09:03:13.559664+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:30.646577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.559685+0000","last_change":"2024-04-25T09:03:12.585556+0000","last_active":"2024-04-25T09:03:13.559685+0000","last_peered":"2024-04-25T09:03:13.559685+0000","last_clean":"2024-04-25T09:03:13.559685+0000","last_became_active":"2024-04-25T09:03:12.584513+0000","last_became_peered":"2024-04-25T09:03:12.584513+0000","last_unstale":"2024-04-25T09:03:13.559685+0000","last_undegraded":"2024-04-25T09:03:13.559685+0000","last_fullsized":"2024-04-25T09:03:13.559685+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:36.514775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"125'1","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564963+0000","last_change":"2024-04-25T09:03:11.562517+0000","last_active":"2024-04-25T09:03:13.564963+0000","last_peered":"2024-04-25T09:03:13.564963+0000","last_clean":"2024-04-25T09:03:13.564963+0000","last_became_active":"2024-04-25T09:03:11.562150+0000","last_became_peered":"2024-04-25T09:03:11.562150+0000","last_unstale":"2024-04-25T09:03:13.564963+0000","last_undegraded":"2024-04-25T09:03:13.564963+0000","last_fullsized":"2024-04-25T09:03:13.564963+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:36.076417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.560040+0000","last_change":"2024-04-25T09:03:12.577457+0000","last_active":"2024-04-25T09:03:13.560040+0000","last_peered":"2024-04-25T09:03:13.560040+0000","last_clean":"2024-04-25T09:03:13.560040+0000","last_became_active":"2024-04-25T09:03:12.577187+0000","last_became_peered":"2024-04-25T09:03:12.577187+0000","last_unstale":"2024-04-25T09:03:13.560040+0000","last_undegraded":"2024-04-25T09:03:13.560040+0000","last_fullsized":"2024-04-25T09:03:13.560040+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:13.132184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"125'2","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565990+0000","last_change":"2024-04-25T09:03:11.565065+0000","last_active":"2024-04-25T09:03:13.565990+0000","last_peered":"2024-04-25T09:03:13.565990+0000","last_clean":"2024-04-25T09:03:13.565990+0000","last_became_active":"2024-04-25T09:03:11.564761+0000","last_became_peered":"2024-04-25T09:03:11.564761+0000","last_unstale":"2024-04-25T09:03:13.565990+0000","last_undegraded":"2024-04-25T09:03:13.565990+0000","last_fullsized":"2024-04-25T09:03:13.565990+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:08:12.126869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.560041+0000","last_change":"2024-04-25T09:03:12.572377+0000","last_active":"2024-04-25T09:03:13.560041+0000","last_peered":"2024-04-25T09:03:13.560041+0000","last_clean":"2024-04-25T09:03:13.560041+0000","last_became_active":"2024-04-25T09:03:12.572057+0000","last_became_peered":"2024-04-25T09:03:12.572057+0000","last_unstale":"2024-04-25T09:03:13.560041+0000","last_undegraded":"2024-04-25T09:03:13.560041+0000","last_fullsized":"2024-04-25T09:03:13.560041+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:03.891788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"125'2","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564849+0000","last_change":"2024-04-25T09:03:11.562578+0000","last_active":"2024-04-25T09:03:13.564849+0000","last_peered":"2024-04-25T09:03:13.564849+0000","last_clean":"2024-04-25T09:03:13.564849+0000","last_became_active":"2024-04-25T09:03:11.562276+0000","last_became_peered":"2024-04-25T09:03:11.562276+0000","last_unstale":"2024-04-25T09:03:13.564849+0000","last_undegraded":"2024-04-25T09:03:13.564849+0000","last_fullsized":"2024-04-25T09:03:13.564849+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:34:35.151692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565384+0000","last_change":"2024-04-25T09:03:12.570418+0000","last_active":"2024-04-25T09:03:13.565384+0000","last_peered":"2024-04-25T09:03:13.565384+0000","last_clean":"2024-04-25T09:03:13.565384+0000","last_became_active":"2024-04-25T09:03:12.570135+0000","last_became_peered":"2024-04-25T09:03:12.570135+0000","last_unstale":"2024-04-25T09:03:13.565384+0000","last_undegraded":"2024-04-25T09:03:13.565384+0000","last_fullsized":"2024-04-25T09:03:13.565384+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:47.056165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"125'1","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564170+0000","last_change":"2024-04-25T09:03:11.563061+0000","last_active":"2024-04-25T09:03:13.564170+0000","last_peered":"2024-04-25T09:03:13.564170+0000","last_clean":"2024-04-25T09:03:13.564170+0000","last_became_active":"2024-04-25T09:03:11.562736+0000","last_became_peered":"2024-04-25T09:03:11.562736+0000","last_unstale":"2024-04-25T09:03:13.564170+0000","last_undegraded":"2024-04-25T09:03:13.564170+0000","last_fullsized":"2024-04-25T09:03:13.564170+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:59:36.076417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565163+0000","last_change":"2024-04-25T09:03:12.575075+0000","last_active":"2024-04-25T09:03:13.565163+0000","last_peered":"2024-04-25T09:03:13.565163+0000","last_clean":"2024-04-25T09:03:13.565163+0000","last_became_active":"2024-04-25T09:03:12.574742+0000","last_became_peered":"2024-04-25T09:03:12.574742+0000","last_unstale":"2024-04-25T09:03:13.565163+0000","last_undegraded":"2024-04-25T09:03:13.565163+0000","last_fullsized":"2024-04-25T09:03:13.565163+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:24.444267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"125'1","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565495+0000","last_change":"2024-04-25T09:03:11.563805+0000","last_active":"2024-04-25T09:03:13.565495+0000","last_peered":"2024-04-25T09:03:13.565495+0000","last_clean":"2024-04-25T09:03:13.565495+0000","last_became_active":"2024-04-25T09:03:11.563418+0000","last_became_peered":"2024-04-25T09:03:11.563418+0000","last_unstale":"2024-04-25T09:03:13.565495+0000","last_undegraded":"2024-04-25T09:03:13.565495+0000","last_fullsized":"2024-04-25T09:03:13.565495+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:04.407967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.560187+0000","last_change":"2024-04-25T09:03:12.586165+0000","last_active":"2024-04-25T09:03:13.560187+0000","last_peered":"2024-04-25T09:03:13.560187+0000","last_clean":"2024-04-25T09:03:13.560187+0000","last_became_active":"2024-04-25T09:03:12.585393+0000","last_became_peered":"2024-04-25T09:03:12.585393+0000","last_unstale":"2024-04-25T09:03:13.560187+0000","last_undegraded":"2024-04-25T09:03:13.560187+0000","last_fullsized":"2024-04-25T09:03:13.560187+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:11.725050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564584+0000","last_change":"2024-04-25T09:03:11.557091+0000","last_active":"2024-04-25T09:03:13.564584+0000","last_peered":"2024-04-25T09:03:13.564584+0000","last_clean":"2024-04-25T09:03:13.564584+0000","last_became_active":"2024-04-25T09:03:11.556838+0000","last_became_peered":"2024-04-25T09:03:11.556838+0000","last_unstale":"2024-04-25T09:03:13.564584+0000","last_undegraded":"2024-04-25T09:03:13.564584+0000","last_fullsized":"2024-04-25T09:03:13.564584+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:54.964981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564517+0000","last_change":"2024-04-25T09:03:12.564943+0000","last_active":"2024-04-25T09:03:13.564517+0000","last_peered":"2024-04-25T09:03:13.564517+0000","last_clean":"2024-04-25T09:03:13.564517+0000","last_became_active":"2024-04-25T09:03:12.564676+0000","last_became_peered":"2024-04-25T09:03:12.564676+0000","last_unstale":"2024-04-25T09:03:13.564517+0000","last_undegraded":"2024-04-25T09:03:13.564517+0000","last_fullsized":"2024-04-25T09:03:13.564517+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:56.534408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565103+0000","last_change":"2024-04-25T09:03:11.557321+0000","last_active":"2024-04-25T09:03:13.565103+0000","last_peered":"2024-04-25T09:03:13.565103+0000","last_clean":"2024-04-25T09:03:13.565103+0000","last_became_active":"2024-04-25T09:03:11.557038+0000","last_became_peered":"2024-04-25T09:03:11.557038+0000","last_unstale":"2024-04-25T09:03:13.565103+0000","last_undegraded":"2024-04-25T09:03:13.565103+0000","last_fullsized":"2024-04-25T09:03:13.565103+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:59.986937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.557785+0000","last_change":"2024-04-25T09:03:12.566673+0000","last_active":"2024-04-25T09:03:13.557785+0000","last_peered":"2024-04-25T09:03:13.557785+0000","last_clean":"2024-04-25T09:03:13.557785+0000","last_became_active":"2024-04-25T09:03:12.566400+0000","last_became_peered":"2024-04-25T09:03:12.566400+0000","last_unstale":"2024-04-25T09:03:13.557785+0000","last_undegraded":"2024-04-25T09:03:13.557785+0000","last_fullsized":"2024-04-25T09:03:13.557785+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:08.284016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565010+0000","last_change":"2024-04-25T09:03:11.549521+0000","last_active":"2024-04-25T09:03:13.565010+0000","last_peered":"2024-04-25T09:03:13.565010+0000","last_clean":"2024-04-25T09:03:13.565010+0000","last_became_active":"2024-04-25T09:03:11.549262+0000","last_became_peered":"2024-04-25T09:03:11.549262+0000","last_unstale":"2024-04-25T09:03:13.565010+0000","last_undegraded":"2024-04-25T09:03:13.565010+0000","last_fullsized":"2024-04-25T09:03:13.565010+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:54.727184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564866+0000","last_change":"2024-04-25T09:03:12.572760+0000","last_active":"2024-04-25T09:03:13.564866+0000","last_peered":"2024-04-25T09:03:13.564866+0000","last_clean":"2024-04-25T09:03:13.564866+0000","last_became_active":"2024-04-25T09:03:12.572456+0000","last_became_peered":"2024-04-25T09:03:12.572456+0000","last_unstale":"2024-04-25T09:03:13.564866+0000","last_undegraded":"2024-04-25T09:03:13.564866+0000","last_fullsized":"2024-04-25T09:03:13.564866+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:30.334232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565418+0000","last_change":"2024-04-25T09:03:11.561683+0000","last_active":"2024-04-25T09:03:13.565418+0000","last_peered":"2024-04-25T09:03:13.565418+0000","last_clean":"2024-04-25T09:03:13.565418+0000","last_became_active":"2024-04-25T09:03:11.561135+0000","last_became_peered":"2024-04-25T09:03:11.561135+0000","last_unstale":"2024-04-25T09:03:13.565418+0000","last_undegraded":"2024-04-25T09:03:13.565418+0000","last_fullsized":"2024-04-25T09:03:13.565418+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:00.501342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.560270+0000","last_change":"2024-04-25T09:03:12.571420+0000","last_active":"2024-04-25T09:03:13.560270+0000","last_peered":"2024-04-25T09:03:13.560270+0000","last_clean":"2024-04-25T09:03:13.560270+0000","last_became_active":"2024-04-25T09:03:12.571162+0000","last_became_peered":"2024-04-25T09:03:12.571162+0000","last_unstale":"2024-04-25T09:03:13.560270+0000","last_undegraded":"2024-04-25T09:03:13.560270+0000","last_fullsized":"2024-04-25T09:03:13.560270+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:36.351839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"125'3","reported_seq":17,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564697+0000","last_change":"2024-04-25T09:03:11.560067+0000","last_active":"2024-04-25T09:03:13.564697+0000","last_peered":"2024-04-25T09:03:13.564697+0000","last_clean":"2024-04-25T09:03:13.564697+0000","last_became_active":"2024-04-25T09:03:11.559690+0000","last_became_peered":"2024-04-25T09:03:11.559690+0000","last_unstale":"2024-04-25T09:03:13.564697+0000","last_undegraded":"2024-04-25T09:03:13.564697+0000","last_fullsized":"2024-04-25T09:03:13.564697+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:50:10.084384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.560270+0000","last_change":"2024-04-25T09:03:12.575624+0000","last_active":"2024-04-25T09:03:13.560270+0000","last_peered":"2024-04-25T09:03:13.560270+0000","last_clean":"2024-04-25T09:03:13.560270+0000","last_became_active":"2024-04-25T09:03:12.575263+0000","last_became_peered":"2024-04-25T09:03:12.575263+0000","last_unstale":"2024-04-25T09:03:13.560270+0000","last_undegraded":"2024-04-25T09:03:13.560270+0000","last_fullsized":"2024-04-25T09:03:13.560270+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:30.470993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"125'6","reported_seq":20,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.559825+0000","last_change":"2024-04-25T09:03:11.555911+0000","last_active":"2024-04-25T09:03:13.559825+0000","last_peered":"2024-04-25T09:03:13.559825+0000","last_clean":"2024-04-25T09:03:13.559825+0000","last_became_active":"2024-04-25T09:03:11.555621+0000","last_became_peered":"2024-04-25T09:03:11.555621+0000","last_unstale":"2024-04-25T09:03:13.559825+0000","last_undegraded":"2024-04-25T09:03:13.559825+0000","last_fullsized":"2024-04-25T09:03:13.559825+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:34.294732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.557887+0000","last_change":"2024-04-25T09:03:12.577051+0000","last_active":"2024-04-25T09:03:13.557887+0000","last_peered":"2024-04-25T09:03:13.557887+0000","last_clean":"2024-04-25T09:03:13.557887+0000","last_became_active":"2024-04-25T09:03:12.573832+0000","last_became_peered":"2024-04-25T09:03:12.573832+0000","last_unstale":"2024-04-25T09:03:13.557887+0000","last_undegraded":"2024-04-25T09:03:13.557887+0000","last_fullsized":"2024-04-25T09:03:13.557887+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:55.732499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.557701+0000","last_change":"2024-04-25T09:03:11.557885+0000","last_active":"2024-04-25T09:03:13.557701+0000","last_peered":"2024-04-25T09:03:13.557701+0000","last_clean":"2024-04-25T09:03:13.557701+0000","last_became_active":"2024-04-25T09:03:11.557528+0000","last_became_peered":"2024-04-25T09:03:11.557528+0000","last_unstale":"2024-04-25T09:03:13.557701+0000","last_undegraded":"2024-04-25T09:03:13.557701+0000","last_fullsized":"2024-04-25T09:03:13.557701+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:18.942742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564045+0000","last_change":"2024-04-25T09:03:12.580756+0000","last_active":"2024-04-25T09:03:13.564045+0000","last_peered":"2024-04-25T09:03:13.564045+0000","last_clean":"2024-04-25T09:03:13.564045+0000","last_became_active":"2024-04-25T09:03:12.580404+0000","last_became_peered":"2024-04-25T09:03:12.580404+0000","last_unstale":"2024-04-25T09:03:13.564045+0000","last_undegraded":"2024-04-25T09:03:13.564045+0000","last_fullsized":"2024-04-25T09:03:13.564045+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:25:11.599506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564780+0000","last_change":"2024-04-25T09:03:11.556814+0000","last_active":"2024-04-25T09:03:13.564780+0000","last_peered":"2024-04-25T09:03:13.564780+0000","last_clean":"2024-04-25T09:03:13.564780+0000","last_became_active":"2024-04-25T09:03:11.556332+0000","last_became_peered":"2024-04-25T09:03:11.556332+0000","last_unstale":"2024-04-25T09:03:13.564780+0000","last_undegraded":"2024-04-25T09:03:13.564780+0000","last_fullsized":"2024-04-25T09:03:13.564780+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:36.455422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.560081+0000","last_change":"2024-04-25T09:03:12.577482+0000","last_active":"2024-04-25T09:03:13.560081+0000","last_peered":"2024-04-25T09:03:13.560081+0000","last_clean":"2024-04-25T09:03:13.560081+0000","last_became_active":"2024-04-25T09:03:12.577206+0000","last_became_peered":"2024-04-25T09:03:12.577206+0000","last_unstale":"2024-04-25T09:03:13.560081+0000","last_undegraded":"2024-04-25T09:03:13.560081+0000","last_fullsized":"2024-04-25T09:03:13.560081+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:20.423551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"125'8","reported_seq":22,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.560415+0000","last_change":"2024-04-25T09:03:11.564910+0000","last_active":"2024-04-25T09:03:13.560415+0000","last_peered":"2024-04-25T09:03:13.560415+0000","last_clean":"2024-04-25T09:03:13.560415+0000","last_became_active":"2024-04-25T09:03:11.564659+0000","last_became_peered":"2024-04-25T09:03:11.564659+0000","last_unstale":"2024-04-25T09:03:13.560415+0000","last_undegraded":"2024-04-25T09:03:13.560415+0000","last_fullsized":"2024-04-25T09:03:13.560415+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:58.531074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.560614+0000","last_change":"2024-04-25T09:03:12.581103+0000","last_active":"2024-04-25T09:03:13.560614+0000","last_peered":"2024-04-25T09:03:13.560614+0000","last_clean":"2024-04-25T09:03:13.560614+0000","last_became_active":"2024-04-25T09:03:12.580757+0000","last_became_peered":"2024-04-25T09:03:12.580757+0000","last_unstale":"2024-04-25T09:03:13.560614+0000","last_undegraded":"2024-04-25T09:03:13.560614+0000","last_fullsized":"2024-04-25T09:03:13.560614+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:34.759105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"125'1","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565070+0000","last_change":"2024-04-25T09:03:11.553990+0000","last_active":"2024-04-25T09:03:13.565070+0000","last_peered":"2024-04-25T09:03:13.565070+0000","last_clean":"2024-04-25T09:03:13.565070+0000","last_became_active":"2024-04-25T09:03:11.553630+0000","last_became_peered":"2024-04-25T09:03:11.553630+0000","last_unstale":"2024-04-25T09:03:13.565070+0000","last_undegraded":"2024-04-25T09:03:13.565070+0000","last_fullsized":"2024-04-25T09:03:13.565070+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:28:59.986937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565047+0000","last_change":"2024-04-25T09:03:12.570244+0000","last_active":"2024-04-25T09:03:13.565047+0000","last_peered":"2024-04-25T09:03:13.565047+0000","last_clean":"2024-04-25T09:03:13.565047+0000","last_became_active":"2024-04-25T09:03:12.569981+0000","last_became_peered":"2024-04-25T09:03:12.569981+0000","last_unstale":"2024-04-25T09:03:13.565047+0000","last_undegraded":"2024-04-25T09:03:13.565047+0000","last_fullsized":"2024-04-25T09:03:13.565047+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:55.970296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"125'2","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564342+0000","last_change":"2024-04-25T09:03:11.562060+0000","last_active":"2024-04-25T09:03:13.564342+0000","last_peered":"2024-04-25T09:03:13.564342+0000","last_clean":"2024-04-25T09:03:13.564342+0000","last_became_active":"2024-04-25T09:03:11.561765+0000","last_became_peered":"2024-04-25T09:03:11.561765+0000","last_unstale":"2024-04-25T09:03:13.564342+0000","last_undegraded":"2024-04-25T09:03:13.564342+0000","last_fullsized":"2024-04-25T09:03:13.564342+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:16.560861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565595+0000","last_change":"2024-04-25T09:03:12.565792+0000","last_active":"2024-04-25T09:03:13.565595+0000","last_peered":"2024-04-25T09:03:13.565595+0000","last_clean":"2024-04-25T09:03:13.565595+0000","last_became_active":"2024-04-25T09:03:12.564946+0000","last_became_peered":"2024-04-25T09:03:12.564946+0000","last_unstale":"2024-04-25T09:03:13.565595+0000","last_undegraded":"2024-04-25T09:03:13.565595+0000","last_fullsized":"2024-04-25T09:03:13.565595+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:04.248103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"125'1","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565268+0000","last_change":"2024-04-25T09:03:11.563479+0000","last_active":"2024-04-25T09:03:13.565268+0000","last_peered":"2024-04-25T09:03:13.565268+0000","last_clean":"2024-04-25T09:03:13.565268+0000","last_became_active":"2024-04-25T09:03:11.563181+0000","last_became_peered":"2024-04-25T09:03:11.563181+0000","last_unstale":"2024-04-25T09:03:13.565268+0000","last_undegraded":"2024-04-25T09:03:13.565268+0000","last_fullsized":"2024-04-25T09:03:13.565268+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:29.328917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564893+0000","last_change":"2024-04-25T09:03:12.565255+0000","last_active":"2024-04-25T09:03:13.564893+0000","last_peered":"2024-04-25T09:03:13.564893+0000","last_clean":"2024-04-25T09:03:13.564893+0000","last_became_active":"2024-04-25T09:03:12.564920+0000","last_became_peered":"2024-04-25T09:03:12.564920+0000","last_unstale":"2024-04-25T09:03:13.564893+0000","last_undegraded":"2024-04-25T09:03:13.564893+0000","last_fullsized":"2024-04-25T09:03:13.564893+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:37.669623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564527+0000","last_change":"2024-04-25T09:03:11.562425+0000","last_active":"2024-04-25T09:03:13.564527+0000","last_peered":"2024-04-25T09:03:13.564527+0000","last_clean":"2024-04-25T09:03:13.564527+0000","last_became_active":"2024-04-25T09:03:11.562079+0000","last_became_peered":"2024-04-25T09:03:11.562079+0000","last_unstale":"2024-04-25T09:03:13.564527+0000","last_undegraded":"2024-04-25T09:03:13.564527+0000","last_fullsized":"2024-04-25T09:03:13.564527+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:07.278701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.557581+0000","last_change":"2024-04-25T09:03:12.573116+0000","last_active":"2024-04-25T09:03:13.557581+0000","last_peered":"2024-04-25T09:03:13.557581+0000","last_clean":"2024-04-25T09:03:13.557581+0000","last_became_active":"2024-04-25T09:03:12.572811+0000","last_became_peered":"2024-04-25T09:03:12.572811+0000","last_unstale":"2024-04-25T09:03:13.557581+0000","last_undegraded":"2024-04-25T09:03:13.557581+0000","last_fullsized":"2024-04-25T09:03:13.557581+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:25.875005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"125'2","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564827+0000","last_change":"2024-04-25T09:03:11.556065+0000","last_active":"2024-04-25T09:03:13.564827+0000","last_peered":"2024-04-25T09:03:13.564827+0000","last_clean":"2024-04-25T09:03:13.564827+0000","last_became_active":"2024-04-25T09:03:11.555763+0000","last_became_peered":"2024-04-25T09:03:11.555763+0000","last_unstale":"2024-04-25T09:03:13.564827+0000","last_undegraded":"2024-04-25T09:03:13.564827+0000","last_fullsized":"2024-04-25T09:03:13.564827+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:26.345004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565191+0000","last_change":"2024-04-25T09:03:12.575785+0000","last_active":"2024-04-25T09:03:13.565191+0000","last_peered":"2024-04-25T09:03:13.565191+0000","last_clean":"2024-04-25T09:03:13.565191+0000","last_became_active":"2024-04-25T09:03:12.575524+0000","last_became_peered":"2024-04-25T09:03:12.575524+0000","last_unstale":"2024-04-25T09:03:13.565191+0000","last_undegraded":"2024-04-25T09:03:13.565191+0000","last_fullsized":"2024-04-25T09:03:13.565191+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:13:37.669623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565589+0000","last_change":"2024-04-25T09:03:11.561560+0000","last_active":"2024-04-25T09:03:13.565589+0000","last_peered":"2024-04-25T09:03:13.565589+0000","last_clean":"2024-04-25T09:03:13.565589+0000","last_became_active":"2024-04-25T09:03:11.561254+0000","last_became_peered":"2024-04-25T09:03:11.561254+0000","last_unstale":"2024-04-25T09:03:13.565589+0000","last_undegraded":"2024-04-25T09:03:13.565589+0000","last_fullsized":"2024-04-25T09:03:13.565589+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:30.646577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565007+0000","last_change":"2024-04-25T09:03:12.568429+0000","last_active":"2024-04-25T09:03:13.565007+0000","last_peered":"2024-04-25T09:03:13.565007+0000","last_clean":"2024-04-25T09:03:13.565007+0000","last_became_active":"2024-04-25T09:03:12.567808+0000","last_became_peered":"2024-04-25T09:03:12.567808+0000","last_unstale":"2024-04-25T09:03:13.565007+0000","last_undegraded":"2024-04-25T09:03:13.565007+0000","last_fullsized":"2024-04-25T09:03:13.565007+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:46.582342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564841+0000","last_change":"2024-04-25T09:03:11.562799+0000","last_active":"2024-04-25T09:03:13.564841+0000","last_peered":"2024-04-25T09:03:13.564841+0000","last_clean":"2024-04-25T09:03:13.564841+0000","last_became_active":"2024-04-25T09:03:11.562502+0000","last_became_peered":"2024-04-25T09:03:11.562502+0000","last_unstale":"2024-04-25T09:03:13.564841+0000","last_undegraded":"2024-04-25T09:03:13.564841+0000","last_fullsized":"2024-04-25T09:03:13.564841+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:34.580056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.558970+0000","last_change":"2024-04-25T09:03:12.580220+0000","last_active":"2024-04-25T09:03:13.558970+0000","last_peered":"2024-04-25T09:03:13.558970+0000","last_clean":"2024-04-25T09:03:13.558970+0000","last_became_active":"2024-04-25T09:03:12.579734+0000","last_became_peered":"2024-04-25T09:03:12.579734+0000","last_unstale":"2024-04-25T09:03:13.558970+0000","last_undegraded":"2024-04-25T09:03:13.558970+0000","last_fullsized":"2024-04-25T09:03:13.558970+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:03:39.272779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564036+0000","last_change":"2024-04-25T09:03:11.555616+0000","last_active":"2024-04-25T09:03:13.564036+0000","last_peered":"2024-04-25T09:03:13.564036+0000","last_clean":"2024-04-25T09:03:13.564036+0000","last_became_active":"2024-04-25T09:03:11.555348+0000","last_became_peered":"2024-04-25T09:03:11.555348+0000","last_unstale":"2024-04-25T09:03:13.564036+0000","last_undegraded":"2024-04-25T09:03:13.564036+0000","last_fullsized":"2024-04-25T09:03:13.564036+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:31.565904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565448+0000","last_change":"2024-04-25T09:03:12.571157+0000","last_active":"2024-04-25T09:03:13.565448+0000","last_peered":"2024-04-25T09:03:13.565448+0000","last_clean":"2024-04-25T09:03:13.565448+0000","last_became_active":"2024-04-25T09:03:12.570889+0000","last_became_peered":"2024-04-25T09:03:12.570889+0000","last_unstale":"2024-04-25T09:03:13.565448+0000","last_undegraded":"2024-04-25T09:03:13.565448+0000","last_fullsized":"2024-04-25T09:03:13.565448+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:46.582342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"125'2","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.558009+0000","last_change":"2024-04-25T09:03:11.554360+0000","last_active":"2024-04-25T09:03:13.558009+0000","last_peered":"2024-04-25T09:03:13.558009+0000","last_clean":"2024-04-25T09:03:13.558009+0000","last_became_active":"2024-04-25T09:03:11.554049+0000","last_became_peered":"2024-04-25T09:03:11.554049+0000","last_unstale":"2024-04-25T09:03:13.558009+0000","last_undegraded":"2024-04-25T09:03:13.558009+0000","last_fullsized":"2024-04-25T09:03:13.558009+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:19.067295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.557897+0000","last_change":"2024-04-25T09:03:12.578875+0000","last_active":"2024-04-25T09:03:13.557897+0000","last_peered":"2024-04-25T09:03:13.557897+0000","last_clean":"2024-04-25T09:03:13.557897+0000","last_became_active":"2024-04-25T09:03:12.578534+0000","last_became_peered":"2024-04-25T09:03:12.578534+0000","last_unstale":"2024-04-25T09:03:13.557897+0000","last_undegraded":"2024-04-25T09:03:13.557897+0000","last_fullsized":"2024-04-25T09:03:13.557897+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:29:00.992252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"126'1","reported_seq":15,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.572739+0000","last_change":"2024-04-25T09:03:11.555057+0000","last_active":"2024-04-25T09:03:13.572739+0000","last_peered":"2024-04-25T09:03:13.572739+0000","last_clean":"2024-04-25T09:03:13.572739+0000","last_became_active":"2024-04-25T09:03:11.554262+0000","last_became_peered":"2024-04-25T09:03:11.554262+0000","last_unstale":"2024-04-25T09:03:13.572739+0000","last_undegraded":"2024-04-25T09:03:13.572739+0000","last_fullsized":"2024-04-25T09:03:13.572739+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:04.407967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564745+0000","last_change":"2024-04-25T09:03:12.568129+0000","last_active":"2024-04-25T09:03:13.564745+0000","last_peered":"2024-04-25T09:03:13.564745+0000","last_clean":"2024-04-25T09:03:13.564745+0000","last_became_active":"2024-04-25T09:03:12.567831+0000","last_became_peered":"2024-04-25T09:03:12.567831+0000","last_unstale":"2024-04-25T09:03:13.564745+0000","last_undegraded":"2024-04-25T09:03:13.564745+0000","last_fullsized":"2024-04-25T09:03:13.564745+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:34.759105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564989+0000","last_change":"2024-04-25T09:03:11.557168+0000","last_active":"2024-04-25T09:03:13.564989+0000","last_peered":"2024-04-25T09:03:13.564989+0000","last_clean":"2024-04-25T09:03:13.564989+0000","last_became_active":"2024-04-25T09:03:11.556736+0000","last_became_peered":"2024-04-25T09:03:11.556736+0000","last_unstale":"2024-04-25T09:03:13.564989+0000","last_undegraded":"2024-04-25T09:03:13.564989+0000","last_fullsized":"2024-04-25T09:03:13.564989+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:34.294732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565178+0000","last_change":"2024-04-25T09:03:12.565793+0000","last_active":"2024-04-25T09:03:13.565178+0000","last_peered":"2024-04-25T09:03:13.565178+0000","last_clean":"2024-04-25T09:03:13.565178+0000","last_became_active":"2024-04-25T09:03:12.565472+0000","last_became_peered":"2024-04-25T09:03:12.565472+0000","last_unstale":"2024-04-25T09:03:13.565178+0000","last_undegraded":"2024-04-25T09:03:13.565178+0000","last_fullsized":"2024-04-25T09:03:13.565178+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:34.759105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565359+0000","last_change":"2024-04-25T09:03:11.562034+0000","last_active":"2024-04-25T09:03:13.565359+0000","last_peered":"2024-04-25T09:03:13.565359+0000","last_clean":"2024-04-25T09:03:13.565359+0000","last_became_active":"2024-04-25T09:03:11.561322+0000","last_became_peered":"2024-04-25T09:03:11.561322+0000","last_unstale":"2024-04-25T09:03:13.565359+0000","last_undegraded":"2024-04-25T09:03:13.565359+0000","last_fullsized":"2024-04-25T09:03:13.565359+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:02.700874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.559373+0000","last_change":"2024-04-25T09:03:12.572807+0000","last_active":"2024-04-25T09:03:13.559373+0000","last_peered":"2024-04-25T09:03:13.559373+0000","last_clean":"2024-04-25T09:03:13.559373+0000","last_became_active":"2024-04-25T09:03:12.572529+0000","last_became_peered":"2024-04-25T09:03:12.572529+0000","last_unstale":"2024-04-25T09:03:13.559373+0000","last_undegraded":"2024-04-25T09:03:13.559373+0000","last_fullsized":"2024-04-25T09:03:13.559373+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:04.248103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564264+0000","last_change":"2024-04-25T09:03:11.555690+0000","last_active":"2024-04-25T09:03:13.564264+0000","last_peered":"2024-04-25T09:03:13.564264+0000","last_clean":"2024-04-25T09:03:13.564264+0000","last_became_active":"2024-04-25T09:03:11.555405+0000","last_became_peered":"2024-04-25T09:03:11.555405+0000","last_unstale":"2024-04-25T09:03:13.564264+0000","last_undegraded":"2024-04-25T09:03:13.564264+0000","last_fullsized":"2024-04-25T09:03:13.564264+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:04.512508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565112+0000","last_change":"2024-04-25T09:03:12.580451+0000","last_active":"2024-04-25T09:03:13.565112+0000","last_peered":"2024-04-25T09:03:13.565112+0000","last_clean":"2024-04-25T09:03:13.565112+0000","last_became_active":"2024-04-25T09:03:12.579960+0000","last_became_peered":"2024-04-25T09:03:12.579960+0000","last_unstale":"2024-04-25T09:03:13.565112+0000","last_undegraded":"2024-04-25T09:03:13.565112+0000","last_fullsized":"2024-04-25T09:03:13.565112+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:44.565613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.559382+0000","last_change":"2024-04-25T09:03:11.560584+0000","last_active":"2024-04-25T09:03:13.559382+0000","last_peered":"2024-04-25T09:03:13.559382+0000","last_clean":"2024-04-25T09:03:13.559382+0000","last_became_active":"2024-04-25T09:03:11.560262+0000","last_became_peered":"2024-04-25T09:03:11.560262+0000","last_unstale":"2024-04-25T09:03:13.559382+0000","last_undegraded":"2024-04-25T09:03:13.559382+0000","last_fullsized":"2024-04-25T09:03:13.559382+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:34.580056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.565599+0000","last_change":"2024-04-25T09:03:12.579612+0000","last_active":"2024-04-25T09:03:13.565599+0000","last_peered":"2024-04-25T09:03:13.565599+0000","last_clean":"2024-04-25T09:03:13.565599+0000","last_became_active":"2024-04-25T09:03:12.579329+0000","last_became_peered":"2024-04-25T09:03:12.579329+0000","last_unstale":"2024-04-25T09:03:13.565599+0000","last_undegraded":"2024-04-25T09:03:13.565599+0000","last_fullsized":"2024-04-25T09:03:13.565599+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:03.706189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564937+0000","last_change":"2024-04-25T09:03:11.559802+0000","last_active":"2024-04-25T09:03:13.564937+0000","last_peered":"2024-04-25T09:03:13.564937+0000","last_clean":"2024-04-25T09:03:13.564937+0000","last_became_active":"2024-04-25T09:03:11.559526+0000","last_became_peered":"2024-04-25T09:03:11.559526+0000","last_unstale":"2024-04-25T09:03:13.564937+0000","last_undegraded":"2024-04-25T09:03:13.564937+0000","last_fullsized":"2024-04-25T09:03:13.564937+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:36.668225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564679+0000","last_change":"2024-04-25T09:03:12.567445+0000","last_active":"2024-04-25T09:03:13.564679+0000","last_peered":"2024-04-25T09:03:13.564679+0000","last_clean":"2024-04-25T09:03:13.564679+0000","last_became_active":"2024-04-25T09:03:12.567185+0000","last_became_peered":"2024-04-25T09:03:12.567185+0000","last_unstale":"2024-04-25T09:03:13.564679+0000","last_undegraded":"2024-04-25T09:03:13.564679+0000","last_fullsized":"2024-04-25T09:03:13.564679+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:11.725050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":14,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.564135+0000","last_change":"2024-04-25T09:03:11.555006+0000","last_active":"2024-04-25T09:03:13.564135+0000","last_peered":"2024-04-25T09:03:13.564135+0000","last_clean":"2024-04-25T09:03:13.564135+0000","last_became_active":"2024-04-25T09:03:11.554708+0000","last_became_peered":"2024-04-25T09:03:11.554708+0000","last_unstale":"2024-04-25T09:03:13.564135+0000","last_undegraded":"2024-04-25T09:03:13.564135+0000","last_fullsized":"2024-04-25T09:03:13.564135+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:29:03.242788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.559291+0000","last_change":"2024-04-25T09:03:12.580004+0000","last_active":"2024-04-25T09:03:13.559291+0000","last_peered":"2024-04-25T09:03:13.559291+0000","last_clean":"2024-04-25T09:03:13.559291+0000","last_became_active":"2024-04-25T09:03:12.579636+0000","last_became_peered":"2024-04-25T09:03:12.579636+0000","last_unstale":"2024-04-25T09:03:13.559291+0000","last_undegraded":"2024-04-25T09:03:13.559291+0000","last_fullsized":"2024-04-25T09:03:13.559291+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:11.539451+0000","last_clean_scrub_stamp":"2024-04-25T09:03:11.539451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:08:32.571219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"125'2","reported_seq":16,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.559900+0000","last_change":"2024-04-25T09:03:11.558886+0000","last_active":"2024-04-25T09:03:13.559900+0000","last_peered":"2024-04-25T09:03:13.559900+0000","last_clean":"2024-04-25T09:03:13.559900+0000","last_became_active":"2024-04-25T09:03:11.558545+0000","last_became_peered":"2024-04-25T09:03:11.558545+0000","last_unstale":"2024-04-25T09:03:13.559900+0000","last_undegraded":"2024-04-25T09:03:13.559900+0000","last_fullsized":"2024-04-25T09:03:13.559900+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:03:10.534136+0000","last_clean_scrub_stamp":"2024-04-25T09:03:10.534136+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:00.501342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":367,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-04-25T09:03:13.558838+0000","last_change":"2024-04-25T08:54:00.818044+0000","last_active":"2024-04-25T09:03:13.558838+0000","last_peered":"2024-04-25T09:03:13.558838+0000","last_clean":"2024-04-25T09:03:13.558838+0000","last_became_active":"2024-04-25T08:54:00.817098+0000","last_became_peered":"2024-04-25T08:54:00.817098+0000","last_unstale":"2024-04-25T09:03:13.558838+0000","last_undegraded":"2024-04-25T09:03:13.558838+0000","last_fullsized":"2024-04-25T09:03:13.558838+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_clean_scrub_stamp":"2024-04-25T08:53:59.739677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:48.129513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574961,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59684,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312156,"statfs":{"total":96636764160,"available":96575647744,"internally_reserved":0,"allocated":2179072,"data_stored":1272200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":58915274},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574962,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64660,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":63102,"kb_avail":94307180,"statfs":{"total":96636764160,"available":96570552320,"internally_reserved":0,"allocated":1572864,"data_stored":681194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64616902},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574964,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50760,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321080,"statfs":{"total":96636764160,"available":96584785920,"internally_reserved":0,"allocated":1568768,"data_stored":679694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574962,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68812,"kb_used_data":1528,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94303028,"statfs":{"total":96636764160,"available":96566300672,"internally_reserved":0,"allocated":1564672,"data_stored":679652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574962,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":72988,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298852,"statfs":{"total":96636764160,"available":96562024448,"internally_reserved":0,"allocated":1581056,"data_stored":686042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":73136583},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574962,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46600,"kb_used_data":1532,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325240,"statfs":{"total":96636764160,"available":96589045760,"internally_reserved":0,"allocated":1568768,"data_stored":680198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574962,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77136,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294704,"statfs":{"total":96636764160,"available":96557776896,"internally_reserved":0,"allocated":1572864,"data_stored":685882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574963,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47176,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324664,"statfs":{"total":96636764160,"available":96588455936,"internally_reserved":0,"allocated":2154496,"data_stored":1269950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2250,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:03:15.905 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:03:15.905 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T09:03:16.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.061+0000 7f101c58b640 1 Processor -- start 2024-04-25T09:03:16.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.061+0000 7f101c58b640 1 -- start start 2024-04-25T09:03:16.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.062+0000 7f101c58b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f101415de80 0x7f10141657f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:16.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.062+0000 7f101c58b640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f10140aff50 0x7f1014167d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:16.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.062+0000 7f101c58b640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10140b6640 con 0x7f101406f090 2024-04-25T09:03:16.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.062+0000 7f101c58b640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10140aeab0 con 0x7f10140aff50 2024-04-25T09:03:16.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.062+0000 7f101c58b640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10140b6080 con 0x7f101415de80 2024-04-25T09:03:16.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.062+0000 7f101ab28640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f10140aff50 0x7f1014167d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:16.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.062+0000 7f1019b26640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f101415de80 0x7f10141657f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:16.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.062+0000 7f101a327640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f101406f090 0x7f101406f470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:54566/0 (socket says 172.21.15.43:54566) 2024-04-25T09:03:16.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.062+0000 7f101a327640 1 -- 172.21.15.43:0/323588410 learned_addr learned my addr 172.21.15.43:0/323588410 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:03:16.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.062+0000 7f101ab28640 1 -- 172.21.15.43:0/323588410 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f101415de80 msgr2=0x7f10141657f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:16.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.062+0000 7f101ab28640 1 --2- 172.21.15.43:0/323588410 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f101415de80 0x7f10141657f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.062+0000 7f101ab28640 1 -- 172.21.15.43:0/323588410 >> v1:172.21.15.193:6789/0 conn(0x7f101406f090 legacy=0x7f101406f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:16.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.062+0000 7f101ab28640 1 -- 172.21.15.43:0/323588410 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10140b6360 con 0x7f10140aff50 2024-04-25T09:03:16.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.062+0000 7f1019325640 1 -- 172.21.15.43:0/323588410 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1393543346 0 0) 0x7f10140b6640 con 0x7f101406f090 2024-04-25T09:03:16.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.063+0000 7f101ab28640 1 --2- 172.21.15.43:0/323588410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10140aff50 0x7f1014167d40 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f100c009d90 tx=0x7f100c02a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8114ad3ee4d189 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:16.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.063+0000 7f1019325640 1 -- 172.21.15.43:0/323588410 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100c007740 con 0x7f10140aff50 2024-04-25T09:03:16.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.063+0000 7f1019325640 1 -- 172.21.15.43:0/323588410 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f100c0078e0 con 0x7f10140aff50 2024-04-25T09:03:16.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.063+0000 7f1019325640 1 -- 172.21.15.43:0/323588410 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100c007cb0 con 0x7f10140aff50 2024-04-25T09:03:16.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.064+0000 7f101c58b640 1 -- 172.21.15.43:0/323588410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10140aff50 msgr2=0x7f1014167d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:16.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.064+0000 7f101c58b640 1 --2- 172.21.15.43:0/323588410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10140aff50 0x7f1014167d40 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f100c009d90 tx=0x7f100c02a660 comp rx=0 tx=0).stop 2024-04-25T09:03:16.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.064+0000 7f101c58b640 1 -- 172.21.15.43:0/323588410 shutdown_connections 2024-04-25T09:03:16.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.064+0000 7f101c58b640 1 --2- 172.21.15.43:0/323588410 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10140aff50 0x7f1014167d40 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.064+0000 7f101c58b640 1 --2- 172.21.15.43:0/323588410 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f101415de80 0x7f10141657f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.064+0000 7f101c58b640 1 -- 172.21.15.43:0/323588410 >> 172.21.15.43:0/323588410 conn(0x7f10140b6be0 msgr2=0x7f10140b6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:16.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.064+0000 7f101c58b640 1 -- 172.21.15.43:0/323588410 shutdown_connections 2024-04-25T09:03:16.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.064+0000 7f101c58b640 1 -- 172.21.15.43:0/323588410 wait complete. 2024-04-25T09:03:16.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.065+0000 7f101c58b640 1 Processor -- start 2024-04-25T09:03:16.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.065+0000 7f101c58b640 1 -- start start 2024-04-25T09:03:16.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.066+0000 7f101c58b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f101406f090 0x7f1014147480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:16.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.066+0000 7f101c58b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10140aff50 0x7f10140caa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:16.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.066+0000 7f101c58b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f101415de80 0x7f101406cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:16.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.066+0000 7f101c58b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10140b0860 con 0x7f101415de80 2024-04-25T09:03:16.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.066+0000 7f101c58b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10140b5d50 con 0x7f10140aff50 2024-04-25T09:03:16.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.066+0000 7f101c58b640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10140b09e0 con 0x7f101406f090 2024-04-25T09:03:16.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.066+0000 7f101ab28640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f101415de80 0x7f101406cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:16.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.066+0000 7f101ab28640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f101415de80 0x7f101406cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45280/0 (socket says 172.21.15.43:45280) 2024-04-25T09:03:16.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.066+0000 7f101ab28640 1 -- 172.21.15.43:0/2615965799 learned_addr learned my addr 172.21.15.43:0/2615965799 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:16.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.066+0000 7f1019b26640 1 --2- 172.21.15.43:0/2615965799 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10140aff50 0x7f10140caa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:16.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.066+0000 7f101ab28640 1 -- 172.21.15.43:0/2615965799 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f101406f090 msgr2=0x7f1014147480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:16.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.066+0000 7f101a327640 1 --2- 172.21.15.43:0/2615965799 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f101406f090 0x7f1014147480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:16.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.066+0000 7f101ab28640 1 --2- 172.21.15.43:0/2615965799 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f101406f090 0x7f1014147480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.066+0000 7f101ab28640 1 -- 172.21.15.43:0/2615965799 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10140aff50 msgr2=0x7f10140caa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:16.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.066+0000 7f101ab28640 1 --2- 172.21.15.43:0/2615965799 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10140aff50 0x7f10140caa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.066+0000 7f101ab28640 1 -- 172.21.15.43:0/2615965799 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10141f4c40 con 0x7f101415de80 2024-04-25T09:03:16.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.066+0000 7f1019b26640 1 --2- 172.21.15.43:0/2615965799 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10140aff50 0x7f10140caa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:16.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.067+0000 7f101ab28640 1 --2- 172.21.15.43:0/2615965799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f101415de80 0x7f101406cf10 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f100c009eb0 tx=0x7f100c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:16.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.067+0000 7f100affd640 1 -- 172.21.15.43:0/2615965799 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100c040070 con 0x7f101415de80 2024-04-25T09:03:16.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.067+0000 7f100affd640 1 -- 172.21.15.43:0/2615965799 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f100c031c40 con 0x7f101415de80 2024-04-25T09:03:16.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.067+0000 7f101c58b640 1 -- 172.21.15.43:0/2615965799 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10140b2930 con 0x7f101415de80 2024-04-25T09:03:16.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.067+0000 7f100affd640 1 -- 172.21.15.43:0/2615965799 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100c009450 con 0x7f101415de80 2024-04-25T09:03:16.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.067+0000 7f101c58b640 1 -- 172.21.15.43:0/2615965799 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10140caf80 con 0x7f101415de80 2024-04-25T09:03:16.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.068+0000 7f101c58b640 1 -- 172.21.15.43:0/2615965799 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10140b4e20 con 0x7f101415de80 2024-04-25T09:03:16.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.068+0000 7f100affd640 1 -- 172.21.15.43:0/2615965799 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f100c039040 con 0x7f101415de80 2024-04-25T09:03:16.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.069+0000 7f100affd640 1 --2- 172.21.15.43:0/2615965799 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0ff806b4d0 0x7f0ff806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:16.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.069+0000 7f100affd640 1 -- 172.21.15.43:0/2615965799 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f100c0aafd0 con 0x7f101415de80 2024-04-25T09:03:16.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.069+0000 7f101a327640 1 --2- 172.21.15.43:0/2615965799 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0ff806b4d0 0x7f0ff806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:16.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.070+0000 7f101a327640 1 --2- 172.21.15.43:0/2615965799 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0ff806b4d0 0x7f0ff806d990 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f1010000f60 tx=0x7f1010009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:16.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.073+0000 7f100affd640 1 -- 172.21.15.43:0/2615965799 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f10140b4e20 con 0x7f101415de80 2024-04-25T09:03:16.251 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:03:16.254 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:03:16.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.323+0000 7f101c58b640 1 -- 172.21.15.43:0/2615965799 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f1014074f60 con 0x7f101415de80 2024-04-25T09:03:16.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.326+0000 7f100affd640 1 -- 172.21.15.43:0/2615965799 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+147 (secure 0 0 0) 0x7f100c079ee0 con 0x7f101415de80 2024-04-25T09:03:16.315 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2024-04-25T09:03:16.315 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQD8GSpmVOjWGxAAFNmxeGi0h2D//pa6Ne3eQw== 2024-04-25T09:03:16.315 INFO:teuthology.orchestra.run.smithi043.stdout: caps mds = "allow" 2024-04-25T09:03:16.315 INFO:teuthology.orchestra.run.smithi043.stdout: caps mon = "allow r" 2024-04-25T09:03:16.315 INFO:teuthology.orchestra.run.smithi043.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T09:03:16.317 INFO:teuthology.orchestra.run.smithi043.stderr:updated caps for client.0 2024-04-25T09:03:16.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.328+0000 7f101c58b640 1 -- 172.21.15.43:0/2615965799 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0ff806b4d0 msgr2=0x7f0ff806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:16.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.328+0000 7f101c58b640 1 --2- 172.21.15.43:0/2615965799 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0ff806b4d0 0x7f0ff806d990 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f1010000f60 tx=0x7f1010009040 comp rx=0 tx=0).stop 2024-04-25T09:03:16.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.328+0000 7f101c58b640 1 -- 172.21.15.43:0/2615965799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f101415de80 msgr2=0x7f101406cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:16.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.328+0000 7f101c58b640 1 --2- 172.21.15.43:0/2615965799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f101415de80 0x7f101406cf10 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f100c009eb0 tx=0x7f100c004a40 comp rx=0 tx=0).stop 2024-04-25T09:03:16.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.328+0000 7f101c58b640 1 -- 172.21.15.43:0/2615965799 shutdown_connections 2024-04-25T09:03:16.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.328+0000 7f101c58b640 1 --2- 172.21.15.43:0/2615965799 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f101415de80 0x7f101406cf10 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.328+0000 7f101c58b640 1 --2- 172.21.15.43:0/2615965799 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0ff806b4d0 0x7f0ff806d990 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.328+0000 7f101c58b640 1 --2- 172.21.15.43:0/2615965799 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10140aff50 0x7f10140caa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.328+0000 7f101c58b640 1 --2- 172.21.15.43:0/2615965799 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f101406f090 0x7f1014147480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.328+0000 7f101c58b640 1 -- 172.21.15.43:0/2615965799 >> 172.21.15.43:0/2615965799 conn(0x7f10140b6be0 msgr2=0x7f101415bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:16.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.328+0000 7f101c58b640 1 -- 172.21.15.43:0/2615965799 shutdown_connections 2024-04-25T09:03:16.319 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.328+0000 7f101c58b640 1 -- 172.21.15.43:0/2615965799 wait complete. 2024-04-25T09:03:16.328 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:16.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.481+0000 7f4822a46640 1 Processor -- start 2024-04-25T09:03:16.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.481+0000 7f4822a46640 1 -- start start 2024-04-25T09:03:16.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.482+0000 7f4822a46640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f481c15df50 0x7f481c1619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:16.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.482+0000 7f4822a46640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f481c0cf670 0x7f481c162380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:16.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.482+0000 7f4822a46640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f481c0d24a0 con 0x7f481c161f30 2024-04-25T09:03:16.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.482+0000 7f4822a46640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f481c0cfa80 con 0x7f481c0cf670 2024-04-25T09:03:16.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.482+0000 7f4822a46640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f481c15e330 con 0x7f481c15df50 2024-04-25T09:03:16.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.482+0000 7f4820fe3640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f481c0cf670 0x7f481c162380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:16.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.482+0000 7f481b7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f481c161f30 0x7f481c0c7cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:33562/0 (socket says 172.21.15.43:33562) 2024-04-25T09:03:16.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.482+0000 7f481b7fe640 1 -- 172.21.15.43:0/3713771754 learned_addr learned my addr 172.21.15.43:0/3713771754 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:03:16.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.482+0000 7f481bfff640 1 --2- 172.21.15.43:0/3713771754 >> v2:172.21.15.193:3300/0 conn(0x7f481c15df50 0x7f481c1619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:16.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.483+0000 7f4820fe3640 1 -- 172.21.15.43:0/3713771754 >> v2:172.21.15.193:3300/0 conn(0x7f481c15df50 msgr2=0x7f481c1619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:16.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.483+0000 7f4820fe3640 1 --2- 172.21.15.43:0/3713771754 >> v2:172.21.15.193:3300/0 conn(0x7f481c15df50 0x7f481c1619f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.483+0000 7f4820fe3640 1 -- 172.21.15.43:0/3713771754 >> v1:172.21.15.43:6789/0 conn(0x7f481c161f30 legacy=0x7f481c0c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:16.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.483+0000 7f4820fe3640 1 -- 172.21.15.43:0/3713771754 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f481c0d2740 con 0x7f481c0cf670 2024-04-25T09:03:16.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.483+0000 7f481affd640 1 -- 172.21.15.43:0/3713771754 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1877114132 0 0) 0x7f481c0d24a0 con 0x7f481c161f30 2024-04-25T09:03:16.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.483+0000 7f4820fe3640 1 --2- 172.21.15.43:0/3713771754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f481c0cf670 0x7f481c162380 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f481c08c880 tx=0x7f481402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cdede7cc4475aef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:16.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.483+0000 7f481affd640 1 -- 172.21.15.43:0/3713771754 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4814007740 con 0x7f481c0cf670 2024-04-25T09:03:16.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.483+0000 7f481affd640 1 -- 172.21.15.43:0/3713771754 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48140078e0 con 0x7f481c0cf670 2024-04-25T09:03:16.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.483+0000 7f481affd640 1 -- 172.21.15.43:0/3713771754 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4814007cb0 con 0x7f481c0cf670 2024-04-25T09:03:16.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.484+0000 7f4822a46640 1 -- 172.21.15.43:0/3713771754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f481c0cf670 msgr2=0x7f481c162380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:16.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.484+0000 7f4822a46640 1 --2- 172.21.15.43:0/3713771754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f481c0cf670 0x7f481c162380 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f481c08c880 tx=0x7f481402a680 comp rx=0 tx=0).stop 2024-04-25T09:03:16.477 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.484+0000 7f4822a46640 1 -- 172.21.15.43:0/3713771754 shutdown_connections 2024-04-25T09:03:16.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.484+0000 7f4822a46640 1 --2- 172.21.15.43:0/3713771754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f481c0cf670 0x7f481c162380 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.484+0000 7f4822a46640 1 --2- 172.21.15.43:0/3713771754 >> v2:172.21.15.193:3300/0 conn(0x7f481c15df50 0x7f481c1619f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.484+0000 7f4822a46640 1 -- 172.21.15.43:0/3713771754 >> 172.21.15.43:0/3713771754 conn(0x7f481c0d3a30 msgr2=0x7f481c0d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:16.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.484+0000 7f4822a46640 1 -- 172.21.15.43:0/3713771754 shutdown_connections 2024-04-25T09:03:16.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.484+0000 7f4822a46640 1 -- 172.21.15.43:0/3713771754 wait complete. 2024-04-25T09:03:16.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.485+0000 7f4822a46640 1 Processor -- start 2024-04-25T09:03:16.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.485+0000 7f4822a46640 1 -- start start 2024-04-25T09:03:16.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.486+0000 7f4822a46640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f481c0cf670 0x7f481c15a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:16.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.486+0000 7f4822a46640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f481c15df50 0x7f481c0ccb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:16.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.486+0000 7f4822a46640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f481c161f30 0x7f481c0cd090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:16.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.486+0000 7f4822a46640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481c0c77d0 con 0x7f481c0cf670 2024-04-25T09:03:16.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.486+0000 7f4822a46640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481c0c74f0 con 0x7f481c161f30 2024-04-25T09:03:16.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.486+0000 7f4822a46640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f481c0d2190 con 0x7f481c15df50 2024-04-25T09:03:16.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.486+0000 7f481bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f481c0cf670 0x7f481c15a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:16.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.486+0000 7f481bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f481c0cf670 0x7f481c15a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45292/0 (socket says 172.21.15.43:45292) 2024-04-25T09:03:16.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.486+0000 7f481bfff640 1 -- 172.21.15.43:0/1396505446 learned_addr learned my addr 172.21.15.43:0/1396505446 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:16.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.486+0000 7f4820fe3640 1 --2- 172.21.15.43:0/1396505446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f481c161f30 0x7f481c0cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:16.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.486+0000 7f481b7fe640 1 --2- 172.21.15.43:0/1396505446 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f481c15df50 0x7f481c0ccb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:16.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.487+0000 7f481bfff640 1 -- 172.21.15.43:0/1396505446 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f481c15df50 msgr2=0x7f481c0ccb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:16.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.487+0000 7f481bfff640 1 --2- 172.21.15.43:0/1396505446 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f481c15df50 0x7f481c0ccb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.487+0000 7f481bfff640 1 -- 172.21.15.43:0/1396505446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f481c161f30 msgr2=0x7f481c0cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:16.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.487+0000 7f481bfff640 1 --2- 172.21.15.43:0/1396505446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f481c161f30 0x7f481c0cd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.487+0000 7f481bfff640 1 -- 172.21.15.43:0/1396505446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f481c158e40 con 0x7f481c0cf670 2024-04-25T09:03:16.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.487+0000 7f481b7fe640 1 --2- 172.21.15.43:0/1396505446 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f481c15df50 0x7f481c0ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:16.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.487+0000 7f4820fe3640 1 --2- 172.21.15.43:0/1396505446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f481c161f30 0x7f481c0cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:16.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.487+0000 7f481bfff640 1 --2- 172.21.15.43:0/1396505446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f481c0cf670 0x7f481c15a740 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f480400e9b0 tx=0x7f480400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:16.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.487+0000 7f4818ff9640 1 -- 172.21.15.43:0/1396505446 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4804017070 con 0x7f481c0cf670 2024-04-25T09:03:16.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.487+0000 7f4818ff9640 1 -- 172.21.15.43:0/1396505446 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480400c9a0 con 0x7f481c0cf670 2024-04-25T09:03:16.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.487+0000 7f4822a46640 1 -- 172.21.15.43:0/1396505446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f481c159120 con 0x7f481c0cf670 2024-04-25T09:03:16.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.487+0000 7f4818ff9640 1 -- 172.21.15.43:0/1396505446 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480400cd90 con 0x7f481c0cf670 2024-04-25T09:03:16.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.487+0000 7f4822a46640 1 -- 172.21.15.43:0/1396505446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f481c1596e0 con 0x7f481c0cf670 2024-04-25T09:03:16.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.488+0000 7f4822a46640 1 -- 172.21.15.43:0/1396505446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f481c0680f0 con 0x7f481c0cf670 2024-04-25T09:03:16.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.488+0000 7f4818ff9640 1 -- 172.21.15.43:0/1396505446 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f480401a070 con 0x7f481c0cf670 2024-04-25T09:03:16.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.489+0000 7f4818ff9640 1 --2- 172.21.15.43:0/1396505446 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f47fc06b4d0 0x7f47fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:16.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.489+0000 7f4818ff9640 1 -- 172.21.15.43:0/1396505446 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f480408a590 con 0x7f481c0cf670 2024-04-25T09:03:16.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.489+0000 7f481b7fe640 1 --2- 172.21.15.43:0/1396505446 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f47fc06b4d0 0x7f47fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:16.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.490+0000 7f481b7fe640 1 --2- 172.21.15.43:0/1396505446 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f47fc06b4d0 0x7f47fc06d990 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f481c161840 tx=0x7f480c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:16.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.493+0000 7f4818ff9640 1 -- 172.21.15.43:0/1396505446 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4804059410 con 0x7f481c0cf670 2024-04-25T09:03:16.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.712+0000 7f4822a46640 1 -- 172.21.15.43:0/1396505446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f481c074ef0 con 0x7f481c0cf670 2024-04-25T09:03:16.701 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.713+0000 7f4818ff9640 1 -- 172.21.15.43:0/1396505446 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f4804058d10 con 0x7f481c0cf670 2024-04-25T09:03:16.702 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:03:16.702 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/891225068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":891225068},{"type":"v1","addr":"172.21.15.193:6836","nonce":891225068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":7175,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3679059390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":3679059390},{"type":"v1","addr":"172.21.15.193:6837","nonce":3679059390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":7948,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2127363183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2127363183},{"type":"v1","addr":"172.21.15.43:6835","nonce":2127363183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:11.919337+0000","modified":"2024-04-25T09:03:13.917768+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9237},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9237":{"gid":9237,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6834/2603892772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2603892772},{"type":"v1","addr":"172.21.15.43:6834","nonce":2603892772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:03:16.702 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 76 2024-04-25T09:03:16.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.715+0000 7f4822a46640 1 -- 172.21.15.43:0/1396505446 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f47fc06b4d0 msgr2=0x7f47fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:16.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.715+0000 7f4822a46640 1 --2- 172.21.15.43:0/1396505446 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f47fc06b4d0 0x7f47fc06d990 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f481c161840 tx=0x7f480c009040 comp rx=0 tx=0).stop 2024-04-25T09:03:16.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.715+0000 7f4822a46640 1 -- 172.21.15.43:0/1396505446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f481c0cf670 msgr2=0x7f481c15a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:16.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.715+0000 7f4822a46640 1 --2- 172.21.15.43:0/1396505446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f481c0cf670 0x7f481c15a740 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f480400e9b0 tx=0x7f480400ee70 comp rx=0 tx=0).stop 2024-04-25T09:03:16.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.715+0000 7f4822a46640 1 -- 172.21.15.43:0/1396505446 shutdown_connections 2024-04-25T09:03:16.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.715+0000 7f4822a46640 1 --2- 172.21.15.43:0/1396505446 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f47fc06b4d0 0x7f47fc06d990 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.715+0000 7f4822a46640 1 --2- 172.21.15.43:0/1396505446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f481c161f30 0x7f481c0cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.715+0000 7f4822a46640 1 --2- 172.21.15.43:0/1396505446 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f481c15df50 0x7f481c0ccb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.715+0000 7f4822a46640 1 --2- 172.21.15.43:0/1396505446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f481c0cf670 0x7f481c15a740 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:16.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.715+0000 7f4822a46640 1 -- 172.21.15.43:0/1396505446 >> 172.21.15.43:0/1396505446 conn(0x7f481c0d3a30 msgr2=0x7f481c06c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:16.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.715+0000 7f4822a46640 1 -- 172.21.15.43:0/1396505446 shutdown_connections 2024-04-25T09:03:16.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:16.715+0000 7f4822a46640 1 -- 172.21.15.43:0/1396505446 wait complete. 2024-04-25T09:03:17.117 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 76, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:03:11.919337+0000', 'modified': '2024-04-25T09:03:13.917768+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9237}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9237': {'gid': 9237, 'name': 'a', 'rank': 0, 'incarnation': 74, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.43:6834/2603892772', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6832', 'nonce': 2603892772}, {'type': 'v1', 'addr': '172.21.15.43:6834', 'nonce': 2603892772}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:03:17.117 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:03:17.118 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T09:03:17.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.235+0000 7f1d8227c640 1 Processor -- start 2024-04-25T09:03:17.223 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.235+0000 7f1d8227c640 1 -- start start 2024-04-25T09:03:17.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.235+0000 7f1d8227c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1d7c15a9d0 0x7f1d7c06d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.236+0000 7f1d8227c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1d7c072160 0x7f1d7c0cd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.236+0000 7f1d8227c640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d7c0704f0 con 0x7f1d7c06d7d0 2024-04-25T09:03:17.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.236+0000 7f1d8227c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d7c06dc30 con 0x7f1d7c072160 2024-04-25T09:03:17.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.236+0000 7f1d8227c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c0707d0 con 0x7f1d7c15a9d0 2024-04-25T09:03:17.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.236+0000 7f1d80819640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1d7c072160 0x7f1d7c0cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.236+0000 7f1d80819640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1d7c072160 0x7f1d7c0cd940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45308/0 (socket says 172.21.15.43:45308) 2024-04-25T09:03:17.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.236+0000 7f1d80819640 1 -- 172.21.15.43:0/1641041732 learned_addr learned my addr 172.21.15.43:0/1641041732 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:17.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.236+0000 7f1d7bfff640 1 --2- 172.21.15.43:0/1641041732 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1d7c15a9d0 0x7f1d7c06d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.236+0000 7f1d80819640 1 -- 172.21.15.43:0/1641041732 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1d7c15a9d0 msgr2=0x7f1d7c06d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:17.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.236+0000 7f1d80819640 1 --2- 172.21.15.43:0/1641041732 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1d7c15a9d0 0x7f1d7c06d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.236+0000 7f1d80819640 1 -- 172.21.15.43:0/1641041732 >> v1:172.21.15.193:6789/0 conn(0x7f1d7c06d7d0 legacy=0x7f1d7c071a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.236+0000 7f1d80819640 1 -- 172.21.15.43:0/1641041732 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d7c070ab0 con 0x7f1d7c072160 2024-04-25T09:03:17.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.237+0000 7f1d80819640 1 --2- 172.21.15.43:0/1641041732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d7c072160 0x7f1d7c0cd940 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f1d74009e50 tx=0x7f1d7402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3bff3cb0f4c1c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:17.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.237+0000 7f1d7affd640 1 -- 172.21.15.43:0/1641041732 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d74036070 con 0x7f1d7c072160 2024-04-25T09:03:17.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.237+0000 7f1d7affd640 1 -- 172.21.15.43:0/1641041732 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d74031600 con 0x7f1d7c072160 2024-04-25T09:03:17.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.237+0000 7f1d7affd640 1 -- 172.21.15.43:0/1641041732 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d740319d0 con 0x7f1d7c072160 2024-04-25T09:03:17.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.237+0000 7f1d8227c640 1 -- 172.21.15.43:0/1641041732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d7c072160 msgr2=0x7f1d7c0cd940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.237+0000 7f1d8227c640 1 --2- 172.21.15.43:0/1641041732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d7c072160 0x7f1d7c0cd940 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f1d74009e50 tx=0x7f1d7402a710 comp rx=0 tx=0).stop 2024-04-25T09:03:17.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.237+0000 7f1d8227c640 1 -- 172.21.15.43:0/1641041732 shutdown_connections 2024-04-25T09:03:17.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.237+0000 7f1d8227c640 1 --2- 172.21.15.43:0/1641041732 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d7c072160 0x7f1d7c0cd940 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.238+0000 7f1d8227c640 1 --2- 172.21.15.43:0/1641041732 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1d7c15a9d0 0x7f1d7c06d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.238+0000 7f1d8227c640 1 -- 172.21.15.43:0/1641041732 >> 172.21.15.43:0/1641041732 conn(0x7f1d7c0b1480 msgr2=0x7f1d7c0b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:17.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.238+0000 7f1d8227c640 1 -- 172.21.15.43:0/1641041732 shutdown_connections 2024-04-25T09:03:17.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.238+0000 7f1d8227c640 1 -- 172.21.15.43:0/1641041732 wait complete. 2024-04-25T09:03:17.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.239+0000 7f1d8227c640 1 Processor -- start 2024-04-25T09:03:17.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.239+0000 7f1d8227c640 1 -- start start 2024-04-25T09:03:17.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.240+0000 7f1d8227c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d7c06d7d0 0x7f1d7c0c46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.240+0000 7f1d8227c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d7c072160 0x7f1d7c1f2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.240+0000 7f1d8227c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1d7c0afaf0 0x7f1d7c0aff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.240+0000 7f1d8227c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c0cde80 con 0x7f1d7c072160 2024-04-25T09:03:17.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.240+0000 7f1d8227c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c0712b0 con 0x7f1d7c06d7d0 2024-04-25T09:03:17.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.240+0000 7f1d8227c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c0ce000 con 0x7f1d7c0afaf0 2024-04-25T09:03:17.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.240+0000 7f1d7b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d7c072160 0x7f1d7c1f2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.240+0000 7f1d7b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d7c072160 0x7f1d7c1f2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45324/0 (socket says 172.21.15.43:45324) 2024-04-25T09:03:17.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.240+0000 7f1d7b7fe640 1 -- 172.21.15.43:0/664164158 learned_addr learned my addr 172.21.15.43:0/664164158 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:17.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.240+0000 7f1d7bfff640 1 --2- 172.21.15.43:0/664164158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d7c06d7d0 0x7f1d7c0c46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.240+0000 7f1d80819640 1 --2- 172.21.15.43:0/664164158 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1d7c0afaf0 0x7f1d7c0aff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.240+0000 7f1d7b7fe640 1 -- 172.21.15.43:0/664164158 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1d7c0afaf0 msgr2=0x7f1d7c0aff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.240+0000 7f1d7b7fe640 1 --2- 172.21.15.43:0/664164158 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1d7c0afaf0 0x7f1d7c0aff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.240+0000 7f1d7b7fe640 1 -- 172.21.15.43:0/664164158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d7c06d7d0 msgr2=0x7f1d7c0c46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.240+0000 7f1d7b7fe640 1 --2- 172.21.15.43:0/664164158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d7c06d7d0 0x7f1d7c0c46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.240+0000 7f1d7b7fe640 1 -- 172.21.15.43:0/664164158 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d7c0aa230 con 0x7f1d7c072160 2024-04-25T09:03:17.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.240+0000 7f1d7bfff640 1 --2- 172.21.15.43:0/664164158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d7c06d7d0 0x7f1d7c0c46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:17.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.240+0000 7f1d80819640 1 --2- 172.21.15.43:0/664164158 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1d7c0afaf0 0x7f1d7c0aff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:17.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.240+0000 7f1d7b7fe640 1 --2- 172.21.15.43:0/664164158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d7c072160 0x7f1d7c1f2f00 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f1d7000c400 tx=0x7f1d7000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:17.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.240+0000 7f1d78ff9640 1 -- 172.21.15.43:0/664164158 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d700052d0 con 0x7f1d7c072160 2024-04-25T09:03:17.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.240+0000 7f1d78ff9640 1 -- 172.21.15.43:0/664164158 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d70005490 con 0x7f1d7c072160 2024-04-25T09:03:17.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.240+0000 7f1d8227c640 1 -- 172.21.15.43:0/664164158 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d7c0aa510 con 0x7f1d7c072160 2024-04-25T09:03:17.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.240+0000 7f1d78ff9640 1 -- 172.21.15.43:0/664164158 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d70007260 con 0x7f1d7c072160 2024-04-25T09:03:17.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.240+0000 7f1d8227c640 1 -- 172.21.15.43:0/664164158 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d7c14aa40 con 0x7f1d7c072160 2024-04-25T09:03:17.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.242+0000 7f1d78ff9640 1 -- 172.21.15.43:0/664164158 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1d70007400 con 0x7f1d7c072160 2024-04-25T09:03:17.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.242+0000 7f1d78ff9640 1 --2- 172.21.15.43:0/664164158 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1d6806b4d0 0x7f1d6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.243+0000 7f1d78ff9640 1 -- 172.21.15.43:0/664164158 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1d700896e0 con 0x7f1d7c072160 2024-04-25T09:03:17.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.243+0000 7f1d7bfff640 1 --2- 172.21.15.43:0/664164158 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1d6806b4d0 0x7f1d6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.243+0000 7f1d84f76740 1 Processor -- start 2024-04-25T09:03:17.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.243+0000 7f1d7bfff640 1 --2- 172.21.15.43:0/664164158 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1d6806b4d0 0x7f1d6806d990 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c006fd0 tx=0x7f1d6c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:17.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.243+0000 7f1d84f76740 1 -- start start 2024-04-25T09:03:17.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.244+0000 7f1d84f76740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0f2925230 0x55e0f2925610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.244+0000 7f1d84f76740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f2942ad0 0x55e0f2943020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.244+0000 7f1d84f76740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55e0f28faf10 0x55e0f2941190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.244+0000 7f1d84f76740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f285da10 con 0x55e0f2942ad0 2024-04-25T09:03:17.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.244+0000 7f1d84f76740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f2806b30 con 0x55e0f2925230 2024-04-25T09:03:17.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.244+0000 7f1d84f76740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e0f2859c50 con 0x55e0f28faf10 2024-04-25T09:03:17.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.244+0000 7f1d7b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f2942ad0 0x55e0f2943020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.244+0000 7f1d7b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f2942ad0 0x55e0f2943020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45330/0 (socket says 172.21.15.43:45330) 2024-04-25T09:03:17.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.244+0000 7f1d7b7fe640 1 -- 172.21.15.43:0/1458558288 learned_addr learned my addr 172.21.15.43:0/1458558288 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:17.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.244+0000 7f1d7bfff640 1 --2- 172.21.15.43:0/1458558288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0f2925230 0x55e0f2925610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.244+0000 7f1d80819640 1 --2- 172.21.15.43:0/1458558288 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55e0f28faf10 0x55e0f2941190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.244+0000 7f1d7b7fe640 1 -- 172.21.15.43:0/1458558288 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55e0f28faf10 msgr2=0x55e0f2941190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:17.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.244+0000 7f1d7b7fe640 1 --2- 172.21.15.43:0/1458558288 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55e0f28faf10 0x55e0f2941190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.244+0000 7f1d7b7fe640 1 -- 172.21.15.43:0/1458558288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0f2925230 msgr2=0x55e0f2925610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:17.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.244+0000 7f1d7b7fe640 1 --2- 172.21.15.43:0/1458558288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0f2925230 0x55e0f2925610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.244+0000 7f1d7b7fe640 1 -- 172.21.15.43:0/1458558288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0f281be10 con 0x55e0f2942ad0 2024-04-25T09:03:17.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.245+0000 7f1d7b7fe640 1 --2- 172.21.15.43:0/1458558288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f2942ad0 0x55e0f2943020 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f1d70087ca0 tx=0x7f1d7000a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c3df3e41db8353d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:17.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.245+0000 7f1d61ffb640 1 -- 172.21.15.43:0/1458558288 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d700561d0 con 0x55e0f2942ad0 2024-04-25T09:03:17.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.245+0000 7f1d61ffb640 1 -- 172.21.15.43:0/1458558288 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d70054540 con 0x55e0f2942ad0 2024-04-25T09:03:17.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.245+0000 7f1d61ffb640 1 -- 172.21.15.43:0/1458558288 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7005e290 con 0x55e0f2942ad0 2024-04-25T09:03:17.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.245+0000 7f1d84f76740 1 -- 172.21.15.43:0/1458558288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f2942ad0 msgr2=0x55e0f2943020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.245+0000 7f1d84f76740 1 --2- 172.21.15.43:0/1458558288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f2942ad0 0x55e0f2943020 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f1d70087ca0 tx=0x7f1d7000a000 comp rx=0 tx=0).stop 2024-04-25T09:03:17.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.245+0000 7f1d84f76740 1 -- 172.21.15.43:0/1458558288 shutdown_connections 2024-04-25T09:03:17.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.245+0000 7f1d84f76740 1 --2- 172.21.15.43:0/1458558288 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55e0f28faf10 0x55e0f2941190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.245+0000 7f1d84f76740 1 --2- 172.21.15.43:0/1458558288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f2942ad0 0x55e0f2943020 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.245+0000 7f1d84f76740 1 --2- 172.21.15.43:0/1458558288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0f2925230 0x55e0f2925610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.245+0000 7f1d84f76740 1 -- 172.21.15.43:0/1458558288 >> 172.21.15.43:0/1458558288 conn(0x55e0f28f7840 msgr2=0x55e0f2940040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:17.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.245+0000 7f1d84f76740 1 -- 172.21.15.43:0/1458558288 shutdown_connections 2024-04-25T09:03:17.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.245+0000 7f1d84f76740 1 -- 172.21.15.43:0/1458558288 wait complete. 2024-04-25T09:03:17.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.246+0000 7f1d84f76740 1 -- start start 2024-04-25T09:03:17.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.246+0000 7f1d84f76740 1 Processor -- start 2024-04-25T09:03:17.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.247+0000 7f1d84f76740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55e0f2925230 0x55e0f2921b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.247+0000 7f1d84f76740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f2942ad0 0x55e0f2927ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.247+0000 7f1d84f76740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0f2928010 0x55e0f2923570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.247+0000 7f1d84f76740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f28fbcb0 con 0x55e0f2942ad0 2024-04-25T09:03:17.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.247+0000 7f1d84f76740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f281a0c0 con 0x55e0f2928010 2024-04-25T09:03:17.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.247+0000 7f1d84f76740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e0f28f5770 con 0x55e0f2925230 2024-04-25T09:03:17.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.247+0000 7f1d84f76740 10 client.0 _ll_register_callbacks cb 0x55e0f280dec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:17.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.247+0000 7f1d7b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f2942ad0 0x55e0f2927ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.247+0000 7f1d7b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f2942ad0 0x55e0f2927ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45332/0 (socket says 172.21.15.43:45332) 2024-04-25T09:03:17.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.247+0000 7f1d7b7fe640 1 -- 172.21.15.43:0/4237011930 learned_addr learned my addr 172.21.15.43:0/4237011930 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:17.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.247+0000 7f1d80819640 1 --2- 172.21.15.43:0/4237011930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0f2928010 0x55e0f2923570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.247+0000 7f1d7bfff640 1 --2- 172.21.15.43:0/4237011930 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55e0f2925230 0x55e0f2921b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.247+0000 7f1d7b7fe640 1 -- 172.21.15.43:0/4237011930 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55e0f2925230 msgr2=0x55e0f2921b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:17.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.247+0000 7f1d7b7fe640 1 --2- 172.21.15.43:0/4237011930 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55e0f2925230 0x55e0f2921b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.247+0000 7f1d7b7fe640 1 -- 172.21.15.43:0/4237011930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0f2928010 msgr2=0x55e0f2923570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:17.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.247+0000 7f1d7b7fe640 1 --2- 172.21.15.43:0/4237011930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0f2928010 0x55e0f2923570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.247+0000 7f1d7b7fe640 1 -- 172.21.15.43:0/4237011930 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e0f25722d0 con 0x55e0f2942ad0 2024-04-25T09:03:17.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.248+0000 7f1d7b7fe640 1 --2- 172.21.15.43:0/4237011930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f2942ad0 0x55e0f2927ad0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f1d70054330 tx=0x7f1d70000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b7770ea6f89689 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:17.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.248+0000 7f1d5affd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:03:17.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.248+0000 7f1d5affd640 1 -- 172.21.15.43:0/4237011930 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d70054170 con 0x55e0f2942ad0 2024-04-25T09:03:17.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.248+0000 7f1d5affd640 1 -- 172.21.15.43:0/4237011930 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1d7005e270 con 0x55e0f2942ad0 2024-04-25T09:03:17.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.248+0000 7f1d84f76740 1 -- 172.21.15.43:0/4237011930 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1d70054540 con 0x55e0f2942ad0 2024-04-25T09:03:17.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.248+0000 7f1d5affd640 1 -- 172.21.15.43:0/4237011930 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d70057320 con 0x55e0f2942ad0 2024-04-25T09:03:17.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.248+0000 7f1d84f76740 10 client.9384 fetch_fsmap learned FSMap version 76 2024-04-25T09:03:17.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.248+0000 7f1d84f76740 1 -- 172.21.15.43:0/4237011930 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x55e0f263c6b0 con 0x55e0f2942ad0 2024-04-25T09:03:17.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.248+0000 7f1d5affd640 1 -- 172.21.15.43:0/4237011930 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d70054540 con 0x55e0f2942ad0 2024-04-25T09:03:17.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.248+0000 7f1d5affd640 1 -- 172.21.15.43:0/4237011930 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 76) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1d700574c0 con 0x55e0f2942ad0 2024-04-25T09:03:17.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.248+0000 7f1d84f76740 10 client.9384 fetch_fsmap finished waiting for FSMap version 76 2024-04-25T09:03:17.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.248+0000 7f1d84f76740 10 client.9384 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:03:17.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.248+0000 7f1d84f76740 20 client.9384 populate_metadata read hostname 'smithi043' 2024-04-25T09:03:17.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.248+0000 7f1d84f76740 1 --2- 172.21.15.43:0/4237011930 >> [v2:172.21.15.43:6832/2603892772,v1:172.21.15.43:6834/2603892772] conn(0x55e0f2939890 0x55e0f2939c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.248+0000 7f1d84f76740 4 client.9384 mds_command: new command op to 9237 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:03:17.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.248+0000 7f1d84f76740 1 -- 172.21.15.43:0/4237011930 --> [v2:172.21.15.43:6832/2603892772,v1:172.21.15.43:6834/2603892772] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e0f28bc920 con 0x55e0f2939890 2024-04-25T09:03:17.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.248+0000 7f1d7bfff640 1 --2- 172.21.15.43:0/4237011930 >> [v2:172.21.15.43:6832/2603892772,v1:172.21.15.43:6834/2603892772] conn(0x55e0f2939890 0x55e0f2939c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.249+0000 7f1d7bfff640 1 --2- 172.21.15.43:0/4237011930 >> [v2:172.21.15.43:6832/2603892772,v1:172.21.15.43:6834/2603892772] conn(0x55e0f2939890 0x55e0f2939c70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51e59771a8de35e9 server_cookie=8279ca6489c83fee in_seq=0 out_seq=0 2024-04-25T09:03:17.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.249+0000 7f1d5affd640 10 client.9384 ms_handle_connect on v2:172.21.15.43:6832/2603892772 2024-04-25T09:03:17.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.249+0000 7f1d5affd640 1 -- 172.21.15.43:0/4237011930 <== mds.0 v2:172.21.15.43:6832/2603892772 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e0f28bc920 con 0x55e0f2939890 2024-04-25T09:03:17.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.249+0000 7f1d5affd640 10 client.9384 handle_command_reply: tid=0 2024-04-25T09:03:17.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.249+0000 7f1d84f76740 1 client.9384 shutdown 2024-04-25T09:03:17.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.250+0000 7f1d84f76740 1 -- 172.21.15.43:0/4237011930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f2942ad0 msgr2=0x55e0f2927ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.250+0000 7f1d84f76740 1 --2- 172.21.15.43:0/4237011930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f2942ad0 0x55e0f2927ad0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f1d70054330 tx=0x7f1d70000bc0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.250+0000 7f1d84f76740 1 -- 172.21.15.43:0/4237011930 shutdown_connections 2024-04-25T09:03:17.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.250+0000 7f1d84f76740 1 --2- 172.21.15.43:0/4237011930 >> [v2:172.21.15.43:6832/2603892772,v1:172.21.15.43:6834/2603892772] conn(0x55e0f2939890 0x55e0f2939c70 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.250+0000 7f1d84f76740 1 --2- 172.21.15.43:0/4237011930 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0f2928010 0x55e0f2923570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.250+0000 7f1d84f76740 1 --2- 172.21.15.43:0/4237011930 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f2942ad0 0x55e0f2927ad0 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.250+0000 7f1d84f76740 1 --2- 172.21.15.43:0/4237011930 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55e0f2925230 0x55e0f2921b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.250+0000 7f1d5affd640 0 client.9384 ms_handle_reset on v2:172.21.15.43:6832/2603892772 2024-04-25T09:03:17.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.250+0000 7f1d84f76740 1 -- 172.21.15.43:0/4237011930 >> 172.21.15.43:0/4237011930 conn(0x55e0f28faf10 msgr2=0x55e0f29435c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:17.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.250+0000 7f1d84f76740 1 -- 172.21.15.43:0/4237011930 shutdown_connections 2024-04-25T09:03:17.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.250+0000 7f1d84f76740 1 -- 172.21.15.43:0/4237011930 wait complete. 2024-04-25T09:03:17.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.250+0000 7f1d84f76740 20 client.9384 trim_cache size 0 max 16384 2024-04-25T09:03:17.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.266+0000 7f1d84f76740 1 Processor -- start 2024-04-25T09:03:17.254 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.266+0000 7f1d84f76740 1 -- start start 2024-04-25T09:03:17.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.266+0000 7f1d84f76740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55e0f2925230 0x55e0f2940200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.266+0000 7f1d84f76740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f2928010 0x55e0f29525d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.266+0000 7f1d84f76740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0f2942ad0 0x55e0f2952b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.266+0000 7f1d84f76740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c1ff1c0 con 0x55e0f2928010 2024-04-25T09:03:17.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.266+0000 7f1d84f76740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c159510 con 0x55e0f2942ad0 2024-04-25T09:03:17.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.266+0000 7f1d84f76740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c159b10 con 0x55e0f2925230 2024-04-25T09:03:17.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.266+0000 7f1d7b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f2928010 0x55e0f29525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.266+0000 7f1d7b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f2928010 0x55e0f29525d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45346/0 (socket says 172.21.15.43:45346) 2024-04-25T09:03:17.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.266+0000 7f1d7b7fe640 1 -- 172.21.15.43:0/3150373538 learned_addr learned my addr 172.21.15.43:0/3150373538 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:17.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.267+0000 7f1d7b7fe640 1 -- 172.21.15.43:0/3150373538 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55e0f2925230 msgr2=0x55e0f2940200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:17.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.267+0000 7f1d7bfff640 1 --2- 172.21.15.43:0/3150373538 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55e0f2925230 0x55e0f2940200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.267+0000 7f1d80819640 1 --2- 172.21.15.43:0/3150373538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0f2942ad0 0x55e0f2952b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.267+0000 7f1d7b7fe640 1 --2- 172.21.15.43:0/3150373538 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55e0f2925230 0x55e0f2940200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.267+0000 7f1d7b7fe640 1 -- 172.21.15.43:0/3150373538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0f2942ad0 msgr2=0x55e0f2952b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:17.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.267+0000 7f1d7b7fe640 1 --2- 172.21.15.43:0/3150373538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0f2942ad0 0x55e0f2952b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.267+0000 7f1d7b7fe640 1 -- 172.21.15.43:0/3150373538 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0f2963b20 con 0x55e0f2928010 2024-04-25T09:03:17.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.267+0000 7f1d7b7fe640 1 --2- 172.21.15.43:0/3150373538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f2928010 0x55e0f29525d0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f1d70000fb0 tx=0x7f1d70083040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f828a80309bdea8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:17.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.267+0000 7f1d617fa640 1 -- 172.21.15.43:0/3150373538 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d700585d0 con 0x55e0f2928010 2024-04-25T09:03:17.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.267+0000 7f1d617fa640 1 -- 172.21.15.43:0/3150373538 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d700574c0 con 0x55e0f2928010 2024-04-25T09:03:17.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.267+0000 7f1d617fa640 1 -- 172.21.15.43:0/3150373538 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7005e070 con 0x55e0f2928010 2024-04-25T09:03:17.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.268+0000 7f1d84f76740 1 -- 172.21.15.43:0/3150373538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f2928010 msgr2=0x55e0f29525d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.268+0000 7f1d84f76740 1 --2- 172.21.15.43:0/3150373538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f2928010 0x55e0f29525d0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f1d70000fb0 tx=0x7f1d70083040 comp rx=0 tx=0).stop 2024-04-25T09:03:17.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.268+0000 7f1d84f76740 1 -- 172.21.15.43:0/3150373538 shutdown_connections 2024-04-25T09:03:17.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.268+0000 7f1d84f76740 1 --2- 172.21.15.43:0/3150373538 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0f2942ad0 0x55e0f2952b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.268+0000 7f1d84f76740 1 --2- 172.21.15.43:0/3150373538 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f2928010 0x55e0f29525d0 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.268+0000 7f1d84f76740 1 --2- 172.21.15.43:0/3150373538 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55e0f2925230 0x55e0f2940200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.268+0000 7f1d84f76740 1 -- 172.21.15.43:0/3150373538 >> 172.21.15.43:0/3150373538 conn(0x55e0f28faf10 msgr2=0x55e0f293f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:17.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.268+0000 7f1d84f76740 1 -- 172.21.15.43:0/3150373538 shutdown_connections 2024-04-25T09:03:17.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.268+0000 7f1d84f76740 1 -- 172.21.15.43:0/3150373538 wait complete. 2024-04-25T09:03:17.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.268+0000 7f1d84f76740 1 -- start start 2024-04-25T09:03:17.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.268+0000 7f1d84f76740 1 Processor -- start 2024-04-25T09:03:17.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.269+0000 7f1d84f76740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0f2925230 0x55e0f2994430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.269+0000 7f1d84f76740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55e0f2928010 0x55e0f28fa120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.269+0000 7f1d84f76740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f2942ad0 0x55e0f28fa660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.269+0000 7f1d84f76740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f28fbcb0 con 0x55e0f2942ad0 2024-04-25T09:03:17.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.269+0000 7f1d84f76740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0f28f5770 con 0x55e0f2925230 2024-04-25T09:03:17.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.269+0000 7f1d84f76740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e0f281a0c0 con 0x55e0f2928010 2024-04-25T09:03:17.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.269+0000 7f1d84f76740 10 client.0 _ll_register_callbacks cb 0x55e0f28f5c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:17.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.269+0000 7f1d80819640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f2942ad0 0x55e0f28fa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.269+0000 7f1d80819640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f2942ad0 0x55e0f28fa660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45350/0 (socket says 172.21.15.43:45350) 2024-04-25T09:03:17.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.269+0000 7f1d80819640 1 -- 172.21.15.43:0/4087817191 learned_addr learned my addr 172.21.15.43:0/4087817191 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:17.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.269+0000 7f1d7b7fe640 1 --2- 172.21.15.43:0/4087817191 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55e0f2928010 0x55e0f28fa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.269+0000 7f1d7bfff640 1 --2- 172.21.15.43:0/4087817191 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0f2925230 0x55e0f2994430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.270+0000 7f1d80819640 1 -- 172.21.15.43:0/4087817191 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55e0f2928010 msgr2=0x55e0f28fa120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:17.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.270+0000 7f1d80819640 1 --2- 172.21.15.43:0/4087817191 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55e0f2928010 0x55e0f28fa120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.270+0000 7f1d80819640 1 -- 172.21.15.43:0/4087817191 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0f2925230 msgr2=0x55e0f2994430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:17.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.270+0000 7f1d80819640 1 --2- 172.21.15.43:0/4087817191 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0f2925230 0x55e0f2994430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.270+0000 7f1d80819640 1 -- 172.21.15.43:0/4087817191 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e0f2943020 con 0x55e0f2942ad0 2024-04-25T09:03:17.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.270+0000 7f1d80819640 1 --2- 172.21.15.43:0/4087817191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f2942ad0 0x55e0f28fa660 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f1d74009f70 tx=0x7f1d74004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f4d56de1fcf5994 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:17.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.270+0000 7f1d5affd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:03:17.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.270+0000 7f1d5affd640 1 -- 172.21.15.43:0/4087817191 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7404b060 con 0x55e0f2942ad0 2024-04-25T09:03:17.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.270+0000 7f1d5affd640 1 -- 172.21.15.43:0/4087817191 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1d7403db90 con 0x55e0f2942ad0 2024-04-25T09:03:17.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.270+0000 7f1d84f76740 1 -- 172.21.15.43:0/4087817191 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1d700574c0 con 0x55e0f2942ad0 2024-04-25T09:03:17.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.270+0000 7f1d5affd640 1 -- 172.21.15.43:0/4087817191 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d74051050 con 0x55e0f2942ad0 2024-04-25T09:03:17.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.270+0000 7f1d84f76740 10 client.9390 fetch_fsmap learned FSMap version 76 2024-04-25T09:03:17.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.270+0000 7f1d84f76740 1 -- 172.21.15.43:0/4087817191 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x55e0f29a3850 con 0x55e0f2942ad0 2024-04-25T09:03:17.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.270+0000 7f1d5affd640 1 -- 172.21.15.43:0/4087817191 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d700574c0 con 0x55e0f2942ad0 2024-04-25T09:03:17.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.270+0000 7f1d5affd640 1 -- 172.21.15.43:0/4087817191 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 76) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1d74054020 con 0x55e0f2942ad0 2024-04-25T09:03:17.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.270+0000 7f1d84f76740 10 client.9390 fetch_fsmap finished waiting for FSMap version 76 2024-04-25T09:03:17.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.270+0000 7f1d84f76740 10 client.9390 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T09:03:17.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.270+0000 7f1d84f76740 20 client.9390 populate_metadata read hostname 'smithi043' 2024-04-25T09:03:17.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.270+0000 7f1d84f76740 1 --2- 172.21.15.43:0/4087817191 >> [v2:172.21.15.43:6832/2603892772,v1:172.21.15.43:6834/2603892772] conn(0x55e0f28f9360 0x55e0f28f44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.270+0000 7f1d84f76740 4 client.9390 mds_command: new command op to 9237 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:03:17.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.270+0000 7f1d84f76740 1 -- 172.21.15.43:0/4087817191 --> [v2:172.21.15.43:6832/2603892772,v1:172.21.15.43:6834/2603892772] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e0f286af50 con 0x55e0f28f9360 2024-04-25T09:03:17.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.270+0000 7f1d7bfff640 1 --2- 172.21.15.43:0/4087817191 >> [v2:172.21.15.43:6832/2603892772,v1:172.21.15.43:6834/2603892772] conn(0x55e0f28f9360 0x55e0f28f44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.271+0000 7f1d5affd640 10 client.9390 ms_handle_connect on v2:172.21.15.43:6832/2603892772 2024-04-25T09:03:17.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.271+0000 7f1d7bfff640 1 --2- 172.21.15.43:0/4087817191 >> [v2:172.21.15.43:6832/2603892772,v1:172.21.15.43:6834/2603892772] conn(0x55e0f28f9360 0x55e0f28f44a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9309a0fbee2bdf36 server_cookie=c0290440890b91be in_seq=0 out_seq=0 2024-04-25T09:03:17.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.271+0000 7f1d5affd640 1 -- 172.21.15.43:0/4087817191 <== mds.0 v2:172.21.15.43:6832/2603892772 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55e0f286af50 con 0x55e0f28f9360 2024-04-25T09:03:17.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.271+0000 7f1d5affd640 10 client.9390 handle_command_reply: tid=0 2024-04-25T09:03:17.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.271+0000 7f1d84f76740 1 client.9390 shutdown 2024-04-25T09:03:17.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.271+0000 7f1d84f76740 1 -- 172.21.15.43:0/4087817191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f2942ad0 msgr2=0x55e0f28fa660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.271+0000 7f1d84f76740 1 --2- 172.21.15.43:0/4087817191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f2942ad0 0x55e0f28fa660 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f1d74009f70 tx=0x7f1d74004ab0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.271+0000 7f1d84f76740 1 -- 172.21.15.43:0/4087817191 shutdown_connections 2024-04-25T09:03:17.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.271+0000 7f1d84f76740 1 --2- 172.21.15.43:0/4087817191 >> [v2:172.21.15.43:6832/2603892772,v1:172.21.15.43:6834/2603892772] conn(0x55e0f28f9360 0x55e0f28f44a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.271+0000 7f1d84f76740 1 --2- 172.21.15.43:0/4087817191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55e0f2942ad0 0x55e0f28fa660 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.271+0000 7f1d84f76740 1 --2- 172.21.15.43:0/4087817191 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55e0f2928010 0x55e0f28fa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.271+0000 7f1d84f76740 1 --2- 172.21.15.43:0/4087817191 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55e0f2925230 0x55e0f2994430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.271+0000 7f1d5affd640 0 client.9390 ms_handle_reset on v2:172.21.15.43:6832/2603892772 2024-04-25T09:03:17.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.271+0000 7f1d84f76740 1 -- 172.21.15.43:0/4087817191 >> 172.21.15.43:0/4087817191 conn(0x55e0f28faf10 msgr2=0x55e0f2940400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:17.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.271+0000 7f1d84f76740 1 -- 172.21.15.43:0/4087817191 shutdown_connections 2024-04-25T09:03:17.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.271+0000 7f1d84f76740 1 -- 172.21.15.43:0/4087817191 wait complete. 2024-04-25T09:03:17.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.271+0000 7f1d84f76740 20 client.9390 trim_cache size 0 max 16384 2024-04-25T09:03:17.266 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:03:17.266 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "6c4bd50e-aa00-4a7f-8d09-352344ef5a4b", 2024-04-25T09:03:17.266 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-04-25T09:03:17.266 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 9237, 2024-04-25T09:03:17.266 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-04-25T09:03:17.266 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-04-25T09:03:17.266 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-04-25T09:03:17.266 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 4.7062849959999999, 2024-04-25T09:03:17.266 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 76, 2024-04-25T09:03:17.266 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 126, 2024-04-25T09:03:17.267 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 126, 2024-04-25T09:03:17.267 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 9.7523078240000007 2024-04-25T09:03:17.267 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:03:17.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.272+0000 7f1d8227c640 1 -- 172.21.15.43:0/664164158 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1d6806b4d0 msgr2=0x7f1d6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.272+0000 7f1d8227c640 1 --2- 172.21.15.43:0/664164158 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1d6806b4d0 0x7f1d6806d990 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c006fd0 tx=0x7f1d6c008040 comp rx=0 tx=0).stop 2024-04-25T09:03:17.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.272+0000 7f1d8227c640 1 -- 172.21.15.43:0/664164158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d7c072160 msgr2=0x7f1d7c1f2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.272+0000 7f1d8227c640 1 --2- 172.21.15.43:0/664164158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d7c072160 0x7f1d7c1f2f00 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f1d7000c400 tx=0x7f1d7000c8c0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.272+0000 7f1d8227c640 1 -- 172.21.15.43:0/664164158 shutdown_connections 2024-04-25T09:03:17.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.272+0000 7f1d8227c640 1 --2- 172.21.15.43:0/664164158 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1d7c0afaf0 0x7f1d7c0aff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.272+0000 7f1d8227c640 1 --2- 172.21.15.43:0/664164158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1d7c072160 0x7f1d7c1f2f00 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.272+0000 7f1d8227c640 1 --2- 172.21.15.43:0/664164158 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1d6806b4d0 0x7f1d6806d990 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.272+0000 7f1d8227c640 1 --2- 172.21.15.43:0/664164158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1d7c06d7d0 0x7f1d7c0c46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.272+0000 7f1d8227c640 1 -- 172.21.15.43:0/664164158 >> 172.21.15.43:0/664164158 conn(0x7f1d7c0b1480 msgr2=0x7f1d7c071430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:17.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.272+0000 7f1d8227c640 1 -- 172.21.15.43:0/664164158 shutdown_connections 2024-04-25T09:03:17.268 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.272+0000 7f1d8227c640 1 -- 172.21.15.43:0/664164158 wait complete. 2024-04-25T09:03:17.271 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:03:17.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.383+0000 7fa14e570640 1 Processor -- start 2024-04-25T09:03:17.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.383+0000 7fa14e570640 1 -- start start 2024-04-25T09:03:17.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.384+0000 7fa14e570640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fa1480d0d90 0x7fa1480ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.373 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.384+0000 7fa14e570640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa1480d2500 0x7fa1480d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.384+0000 7fa14e570640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1480b6c50 con 0x7fa1480d39d0 2024-04-25T09:03:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.384+0000 7fa14e570640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa14815e010 con 0x7fa1480d2500 2024-04-25T09:03:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.384+0000 7fa14e570640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1480d2020 con 0x7fa1480d0d90 2024-04-25T09:03:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.384+0000 7fa14cb0d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa1480d2500 0x7fa1480d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.384+0000 7fa14cb0d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa1480d2500 0x7fa1480d2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45366/0 (socket says 172.21.15.43:45366) 2024-04-25T09:03:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.384+0000 7fa14cb0d640 1 -- 172.21.15.43:0/431553377 learned_addr learned my addr 172.21.15.43:0/431553377 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.385+0000 7fa1477fe640 1 --2- 172.21.15.43:0/431553377 >> v2:172.21.15.193:3300/0 conn(0x7fa1480d0d90 0x7fa1480ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.385+0000 7fa14cb0d640 1 -- 172.21.15.43:0/431553377 >> v2:172.21.15.193:3300/0 conn(0x7fa1480d0d90 msgr2=0x7fa1480ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.385+0000 7fa14cb0d640 1 --2- 172.21.15.43:0/431553377 >> v2:172.21.15.193:3300/0 conn(0x7fa1480d0d90 0x7fa1480ce2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.385+0000 7fa14cb0d640 1 -- 172.21.15.43:0/431553377 >> v1:172.21.15.193:6789/0 conn(0x7fa1480d39d0 legacy=0x7fa1480d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.374 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.385+0000 7fa14cb0d640 1 -- 172.21.15.43:0/431553377 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1480cec40 con 0x7fa1480d2500 2024-04-25T09:03:17.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.385+0000 7fa14cb0d640 1 --2- 172.21.15.43:0/431553377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1480d2500 0x7fa1480d2990 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fa14808c490 tx=0x7fa14002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8062b0559cc50873 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:17.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.386+0000 7fa146ffd640 1 -- 172.21.15.43:0/431553377 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa140036070 con 0x7fa1480d2500 2024-04-25T09:03:17.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.386+0000 7fa146ffd640 1 -- 172.21.15.43:0/431553377 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa140031600 con 0x7fa1480d2500 2024-04-25T09:03:17.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.386+0000 7fa146ffd640 1 -- 172.21.15.43:0/431553377 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1400319d0 con 0x7fa1480d2500 2024-04-25T09:03:17.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.386+0000 7fa14e570640 1 -- 172.21.15.43:0/431553377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1480d2500 msgr2=0x7fa1480d2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.386+0000 7fa14e570640 1 --2- 172.21.15.43:0/431553377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1480d2500 0x7fa1480d2990 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7fa14808c490 tx=0x7fa14002a6a0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.386+0000 7fa14e570640 1 -- 172.21.15.43:0/431553377 shutdown_connections 2024-04-25T09:03:17.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.386+0000 7fa14e570640 1 --2- 172.21.15.43:0/431553377 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1480d2500 0x7fa1480d2990 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.386+0000 7fa14e570640 1 --2- 172.21.15.43:0/431553377 >> v2:172.21.15.193:3300/0 conn(0x7fa1480d0d90 0x7fa1480ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.376 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.386+0000 7fa14e570640 1 -- 172.21.15.43:0/431553377 >> 172.21.15.43:0/431553377 conn(0x7fa14806b000 msgr2=0x7fa14806b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:17.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.386+0000 7fa14e570640 1 -- 172.21.15.43:0/431553377 shutdown_connections 2024-04-25T09:03:17.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.386+0000 7fa14e570640 1 -- 172.21.15.43:0/431553377 wait complete. 2024-04-25T09:03:17.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.387+0000 7fa14e570640 1 Processor -- start 2024-04-25T09:03:17.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.388+0000 7fa14e570640 1 -- start start 2024-04-25T09:03:17.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.388+0000 7fa14e570640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1480d0d90 0x7fa1481f68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.388+0000 7fa14e570640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1480d2500 0x7fa1480c7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.388+0000 7fa14e570640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa1480d39d0 0x7fa1481600f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.388+0000 7fa14e570640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1480cf200 con 0x7fa1480d2500 2024-04-25T09:03:17.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.388+0000 7fa14e570640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1480b2f00 con 0x7fa1480d0d90 2024-04-25T09:03:17.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.388+0000 7fa14e570640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1480d2ed0 con 0x7fa1480d39d0 2024-04-25T09:03:17.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.388+0000 7fa1477fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1480d2500 0x7fa1480c7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.388+0000 7fa1477fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1480d2500 0x7fa1480c7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45374/0 (socket says 172.21.15.43:45374) 2024-04-25T09:03:17.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.388+0000 7fa1477fe640 1 -- 172.21.15.43:0/2016074216 learned_addr learned my addr 172.21.15.43:0/2016074216 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:17.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.389+0000 7fa14cb0d640 1 --2- 172.21.15.43:0/2016074216 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa1480d39d0 0x7fa1481600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.389+0000 7fa147fff640 1 --2- 172.21.15.43:0/2016074216 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1480d0d90 0x7fa1481f68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.389+0000 7fa1477fe640 1 -- 172.21.15.43:0/2016074216 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa1480d39d0 msgr2=0x7fa1481600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.389+0000 7fa1477fe640 1 --2- 172.21.15.43:0/2016074216 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa1480d39d0 0x7fa1481600f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.389+0000 7fa1477fe640 1 -- 172.21.15.43:0/2016074216 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1480d0d90 msgr2=0x7fa1481f68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.389+0000 7fa1477fe640 1 --2- 172.21.15.43:0/2016074216 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1480d0d90 0x7fa1481f68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.389+0000 7fa1477fe640 1 -- 172.21.15.43:0/2016074216 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1480c87b0 con 0x7fa1480d2500 2024-04-25T09:03:17.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.389+0000 7fa147fff640 1 --2- 172.21.15.43:0/2016074216 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1480d0d90 0x7fa1481f68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:17.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.389+0000 7fa1477fe640 1 --2- 172.21.15.43:0/2016074216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1480d2500 0x7fa1480c7dd0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fa13800d800 tx=0x7fa13800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:17.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.389+0000 7fa144ff9640 1 -- 172.21.15.43:0/2016074216 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13800b6c0 con 0x7fa1480d2500 2024-04-25T09:03:17.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.389+0000 7fa144ff9640 1 -- 172.21.15.43:0/2016074216 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa138019070 con 0x7fa1480d2500 2024-04-25T09:03:17.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.389+0000 7fa14e570640 1 -- 172.21.15.43:0/2016074216 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1480c8a90 con 0x7fa1480d2500 2024-04-25T09:03:17.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.389+0000 7fa144ff9640 1 -- 172.21.15.43:0/2016074216 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13800ba80 con 0x7fa1480d2500 2024-04-25T09:03:17.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.389+0000 7fa14e570640 1 -- 172.21.15.43:0/2016074216 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1480c9050 con 0x7fa1480d2500 2024-04-25T09:03:17.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.390+0000 7fa14e570640 1 -- 172.21.15.43:0/2016074216 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1480c9480 con 0x7fa1480d2500 2024-04-25T09:03:17.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.391+0000 7fa144ff9640 1 -- 172.21.15.43:0/2016074216 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa13800bc20 con 0x7fa1480d2500 2024-04-25T09:03:17.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.391+0000 7fa144ff9640 1 --2- 172.21.15.43:0/2016074216 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa13406b4d0 0x7fa13406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:17.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.391+0000 7fa144ff9640 1 -- 172.21.15.43:0/2016074216 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa13808a610 con 0x7fa1480d2500 2024-04-25T09:03:17.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.391+0000 7fa147fff640 1 --2- 172.21.15.43:0/2016074216 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa13406b4d0 0x7fa13406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:17.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.392+0000 7fa147fff640 1 --2- 172.21.15.43:0/2016074216 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa13406b4d0 0x7fa13406d990 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fa13c001660 tx=0x7fa13c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:17.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.395+0000 7fa144ff9640 1 -- 172.21.15.43:0/2016074216 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa138059490 con 0x7fa1480d2500 2024-04-25T09:03:17.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.613+0000 7fa14e570640 1 -- 172.21.15.43:0/2016074216 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa148074ef0 con 0x7fa1480d2500 2024-04-25T09:03:17.602 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.614+0000 7fa144ff9640 1 -- 172.21.15.43:0/2016074216 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v76) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa138058d90 con 0x7fa1480d2500 2024-04-25T09:03:17.603 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:03:17.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.616+0000 7fa14e570640 1 -- 172.21.15.43:0/2016074216 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa13406b4d0 msgr2=0x7fa13406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.616+0000 7fa14e570640 1 --2- 172.21.15.43:0/2016074216 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa13406b4d0 0x7fa13406d990 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fa13c001660 tx=0x7fa13c009040 comp rx=0 tx=0).stop 2024-04-25T09:03:17.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.616+0000 7fa14e570640 1 -- 172.21.15.43:0/2016074216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1480d2500 msgr2=0x7fa1480c7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:17.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.616+0000 7fa14e570640 1 --2- 172.21.15.43:0/2016074216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1480d2500 0x7fa1480c7dd0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7fa13800d800 tx=0x7fa13800dcc0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.616+0000 7fa14e570640 1 -- 172.21.15.43:0/2016074216 shutdown_connections 2024-04-25T09:03:17.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.616+0000 7fa14e570640 1 --2- 172.21.15.43:0/2016074216 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa1480d39d0 0x7fa1481600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.616+0000 7fa14e570640 1 --2- 172.21.15.43:0/2016074216 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1480d2500 0x7fa1480c7dd0 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.616+0000 7fa14e570640 1 --2- 172.21.15.43:0/2016074216 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa13406b4d0 0x7fa13406d990 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.616+0000 7fa14e570640 1 --2- 172.21.15.43:0/2016074216 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1480d0d90 0x7fa1481f68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:17.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.616+0000 7fa14e570640 1 -- 172.21.15.43:0/2016074216 >> 172.21.15.43:0/2016074216 conn(0x7fa14806b000 msgr2=0x7fa14806b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:17.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.616+0000 7fa14e570640 1 -- 172.21.15.43:0/2016074216 shutdown_connections 2024-04-25T09:03:17.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:17.616+0000 7fa14e570640 1 -- 172.21.15.43:0/2016074216 wait complete. 2024-04-25T09:03:17.616 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T09:03:17.616 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:03:17.616 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-04-25T09:03:17.616 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:03:17.616 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:03:17.616 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T09:03:17.616 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T09:03:17.616 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:03:17.616 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:03:17.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:17.616 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-04-25T09:03:17.632 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:03:17.632 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:03:17.632 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:03:17.632 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:17.633 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-04-25T09:03:17.633 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:17.633 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:03:17.633 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-04-25T09:03:17.633 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-04-25T09:03:17.633 INFO:teuthology.orchestra.run.smithi043.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:03:17.633 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:03:17.633 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-04-25T09:03:17.633 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T09:03:17.633 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 42220sec preferred_lft 42220sec 2024-04-25T09:03:17.633 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link noprefixroute 2024-04-25T09:03:17.633 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:17.633 INFO:teuthology.orchestra.run.smithi043.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:03:17.633 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-04-25T09:03:17.634 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-04-25T09:03:17.634 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:03:17.634 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:03:17.634 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-04-25T09:03:17.634 INFO:teuthology.orchestra.run.smithi043.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:03:17.634 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 12:2c:7a:72:36:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:03:17.634 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:03:17.634 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:17.634 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ccff:4eff:fe9d:c8bf/64 scope link 2024-04-25T09:03:17.634 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:17.634 INFO:teuthology.orchestra.run.smithi043.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:03:17.634 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 16:9a:63:37:46:24 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:03:17.634 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::149a:63ff:fe37:4624/64 scope link 2024-04-25T09:03:17.634 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:17.635 INFO:teuthology.orchestra.run.smithi043.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:03:17.635 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 12:2c:7a:72:36:14 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:03:17.635 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::102c:7aff:fe72:3614/64 scope link 2024-04-25T09:03:17.635 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:03:17.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:17.635 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:03:17.695 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-04-25T09:03:17.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:17.697 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-04-25T09:03:17.753 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-04-25T09:03:17.753 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:03:17.754 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T09:03:17.754 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T09:03:17.754 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:03:17.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:17.754 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2024-04-25T09:03:17.754 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:03:17.754 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T09:03:17.754 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T09:03:17.754 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T09:03:17.754 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:03:17.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:17.840 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-04-25T09:03:17.855 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:03:17.855 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:03:17.855 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:03:17.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:17.855 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-04-25T09:03:17.920 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T09:03:17.920 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:03:17.920 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:03:17.975 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:03:17.976 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:03:17.976 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:03:18.031 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-04-25T09:03:18.096 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:03:18.152 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-04-25T09:03:18.152 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-04-25T09:03:18.152 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-04-25T09:03:18.152 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-04-25T09:03:18.152 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-04-25T09:03:18.153 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-04-25T09:03:18.153 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-04-25T09:03:18.153 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-04-25T09:03:18.153 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-04-25T09:03:18.153 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-04-25T09:03:18.153 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-04-25T09:03:18.153 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/selinux 2024-04-25T09:03:18.153 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:03:18.153 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-04-25T09:03:18.153 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-04-25T09:03:18.153 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-04-25T09:03:18.153 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-04-25T09:03:18.153 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:03:18.154 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-04-25T09:03:18.154 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-04-25T09:03:18.154 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T09:03:18.154 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T09:03:18.154 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T09:03:18.154 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T09:03:18.154 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-04-25T09:03:18.154 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-04-25T09:03:18.154 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:03:18.154 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:03:18.154 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:03:18.154 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:03:18.154 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-04-25T09:03:18.154 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:03:18.155 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:03:18.155 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:03:18.155 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:03:18.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:18.155 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-04-25T09:03:18.208 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T09:03:18.208 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T09:03:18.251 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-04-25T09:03:18.274 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:03:18.331 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-04-25T09:03:18.331 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-04-25T09:03:18.331 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-04-25T09:03:18.331 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-04-25T09:03:18.331 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-04-25T09:03:18.331 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-04-25T09:03:18.331 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-04-25T09:03:18.331 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-04-25T09:03:18.331 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-04-25T09:03:18.331 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-04-25T09:03:18.331 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-04-25T09:03:18.332 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/selinux 2024-04-25T09:03:18.332 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:03:18.332 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-04-25T09:03:18.332 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-04-25T09:03:18.332 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-04-25T09:03:18.332 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-04-25T09:03:18.332 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:03:18.332 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-04-25T09:03:18.332 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-04-25T09:03:18.332 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T09:03:18.332 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T09:03:18.332 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T09:03:18.332 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T09:03:18.333 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-04-25T09:03:18.333 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-04-25T09:03:18.333 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:03:18.333 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:03:18.333 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:03:18.333 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:03:18.333 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-04-25T09:03:18.334 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:03:18.334 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:03:18.334 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:03:18.334 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:03:18.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:18.334 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-04-25T09:03:18.335 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:2024-04-25T09:03:18.343+0000 7f610e521480 -1 init, newargv = 0x562c84bd0810 newargc=15 2024-04-25T09:03:18.335 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[91777]: starting ceph client 2024-04-25T09:03:18.337 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[91777]: starting fuse 2024-04-25T09:03:18.354 INFO:teuthology.orchestra.run.smithi043.stdout:43 2024-04-25T09:03:18.354 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-25T09:03:18.355 DEBUG:teuthology.orchestra.run.smithi043:> sudo stdin-killer -- python3 -c ' 2024-04-25T09:03:18.355 DEBUG:teuthology.orchestra.run.smithi043:> import glob 2024-04-25T09:03:18.355 DEBUG:teuthology.orchestra.run.smithi043:> import re 2024-04-25T09:03:18.355 DEBUG:teuthology.orchestra.run.smithi043:> import os 2024-04-25T09:03:18.355 DEBUG:teuthology.orchestra.run.smithi043:> import subprocess 2024-04-25T09:03:18.355 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T09:03:18.355 DEBUG:teuthology.orchestra.run.smithi043:> def _find_admin_socket(client_name): 2024-04-25T09:03:18.355 DEBUG:teuthology.orchestra.run.smithi043:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T09:03:18.355 DEBUG:teuthology.orchestra.run.smithi043:> files = glob.glob(asok_path) 2024-04-25T09:03:18.355 DEBUG:teuthology.orchestra.run.smithi043:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T09:03:18.355 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T09:03:18.355 DEBUG:teuthology.orchestra.run.smithi043:> # Given a non-glob path, it better be there 2024-04-25T09:03:18.355 DEBUG:teuthology.orchestra.run.smithi043:> if "*" not in asok_path: 2024-04-25T09:03:18.355 DEBUG:teuthology.orchestra.run.smithi043:> assert(len(files) == 1) 2024-04-25T09:03:18.355 DEBUG:teuthology.orchestra.run.smithi043:> return files[0] 2024-04-25T09:03:18.356 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T09:03:18.356 DEBUG:teuthology.orchestra.run.smithi043:> for f in files: 2024-04-25T09:03:18.356 DEBUG:teuthology.orchestra.run.smithi043:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T09:03:18.356 DEBUG:teuthology.orchestra.run.smithi043:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T09:03:18.356 DEBUG:teuthology.orchestra.run.smithi043:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T09:03:18.356 DEBUG:teuthology.orchestra.run.smithi043:> contents = proc_f.read() 2024-04-25T09:03:18.356 DEBUG:teuthology.orchestra.run.smithi043:> if mountpoint in contents: 2024-04-25T09:03:18.356 DEBUG:teuthology.orchestra.run.smithi043:> return f 2024-04-25T09:03:18.356 DEBUG:teuthology.orchestra.run.smithi043:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T09:03:18.356 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T09:03:18.356 DEBUG:teuthology.orchestra.run.smithi043:> print(_find_admin_socket("client.0")) 2024-04-25T09:03:18.356 DEBUG:teuthology.orchestra.run.smithi043:> ' 2024-04-25T09:03:18.469 INFO:teuthology.orchestra.run.smithi043.stdout:/var/run/ceph/ceph-client.0.91777.asok 2024-04-25T09:03:18.472 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:03:18.479 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.91777.asok 2024-04-25T09:03:18.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:18.480 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.91777.asok status 2024-04-25T09:03:18.522 INFO:tasks.ceph.mds.a.smithi043.stderr:2024-04-25T09:03:18.534+0000 7fbe863e7640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:03:18.616 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:03:18.616 INFO:teuthology.orchestra.run.smithi043.stdout: "metadata": { 2024-04-25T09:03:18.617 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T09:03:18.617 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T09:03:18.617 INFO:teuthology.orchestra.run.smithi043.stdout: "entity_id": "0", 2024-04-25T09:03:18.617 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": "smithi043", 2024-04-25T09:03:18.617 INFO:teuthology.orchestra.run.smithi043.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T09:03:18.617 INFO:teuthology.orchestra.run.smithi043.stdout: "pid": "91777", 2024-04-25T09:03:18.617 INFO:teuthology.orchestra.run.smithi043.stdout: "root": "/" 2024-04-25T09:03:18.617 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:03:18.617 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_count": 0, 2024-04-25T09:03:18.617 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_pinned_count": 0, 2024-04-25T09:03:18.617 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 9402, 2024-04-25T09:03:18.617 INFO:teuthology.orchestra.run.smithi043.stdout: "inst": { 2024-04-25T09:03:18.617 INFO:teuthology.orchestra.run.smithi043.stdout: "name": { 2024-04-25T09:03:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "client", 2024-04-25T09:03:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: "num": 9402 2024-04-25T09:03:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:03:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-04-25T09:03:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-04-25T09:03:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:03:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 800433709 2024-04-25T09:03:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:03:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:03:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-04-25T09:03:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-04-25T09:03:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:03:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 800433709 2024-04-25T09:03:18.618 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:03:18.619 INFO:teuthology.orchestra.run.smithi043.stdout: "inst_str": "client.9402 192.168.0.1:0/800433709", 2024-04-25T09:03:18.619 INFO:teuthology.orchestra.run.smithi043.stdout: "addr_str": "192.168.0.1:0/800433709", 2024-04-25T09:03:18.619 INFO:teuthology.orchestra.run.smithi043.stdout: "inode_count": 1, 2024-04-25T09:03:18.619 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_epoch": 76, 2024-04-25T09:03:18.619 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch": 126, 2024-04-25T09:03:18.619 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch_barrier": 0, 2024-04-25T09:03:18.619 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklisted": false, 2024-04-25T09:03:18.619 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs" 2024-04-25T09:03:18.619 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:03:18.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:18.625 DEBUG:teuthology.orchestra.run.smithi043:> stat --file-system '--printf=%T 2024-04-25T09:03:18.625 DEBUG:teuthology.orchestra.run.smithi043:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T09:03:18.641 INFO:teuthology.orchestra.run.smithi043.stdout:fuseblk 2024-04-25T09:03:18.641 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T09:03:18.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:18.641 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:03:18.707 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:18.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.858+0000 7f57e7e5d640 1 Processor -- start 2024-04-25T09:03:18.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.859+0000 7f57e7e5d640 1 -- start start 2024-04-25T09:03:18.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.859+0000 7f57e7e5d640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f57e015df50 0x7f57e01619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:18.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.859+0000 7f57e7e5d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57e00d24a0 con 0x7f57e0161f30 2024-04-25T09:03:18.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.859+0000 7f57e7e5d640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57e00d2740 con 0x7f57e00cf670 2024-04-25T09:03:18.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.859+0000 7f57e7e5d640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57e00cfa80 con 0x7f57e015df50 2024-04-25T09:03:18.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.859+0000 7f57e53f8640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f57e0161f30 0x7f57e00c7cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:33576/0 (socket says 172.21.15.43:33576) 2024-04-25T09:03:18.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.860+0000 7f57e53f8640 1 -- 172.21.15.43:0/1806327555 learned_addr learned my addr 172.21.15.43:0/1806327555 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:03:18.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.860+0000 7f57e4bf7640 1 -- 172.21.15.43:0/1806327555 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4219340604 0 0) 0x7f57e00d24a0 con 0x7f57e0161f30 2024-04-25T09:03:18.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.860+0000 7f57e4bf7640 1 -- 172.21.15.43:0/1806327555 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57cc003520 con 0x7f57e0161f30 2024-04-25T09:03:18.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.860+0000 7f57e5bf9640 1 --2- 172.21.15.43:0/1806327555 >> v2:172.21.15.193:3300/0 conn(0x7f57e015df50 0x7f57e01619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:18.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.860+0000 7f57e4bf7640 1 -- 172.21.15.43:0/1806327555 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1766833501 0 0) 0x7f57cc003520 con 0x7f57e0161f30 2024-04-25T09:03:18.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.860+0000 7f57e4bf7640 1 -- 172.21.15.43:0/1806327555 >> v2:172.21.15.193:3300/0 conn(0x7f57e015df50 msgr2=0x7f57e01619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:18.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.860+0000 7f57e4bf7640 1 --2- 172.21.15.43:0/1806327555 >> v2:172.21.15.193:3300/0 conn(0x7f57e015df50 0x7f57e01619f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.860+0000 7f57e4bf7640 1 -- 172.21.15.43:0/1806327555 >> v1:172.21.15.193:6789/0 conn(0x7f57e00cf670 legacy=0x7f57e00c8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:18.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.860+0000 7f57e4bf7640 1 -- 172.21.15.43:0/1806327555 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e00d2a20 con 0x7f57e0161f30 2024-04-25T09:03:18.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.860+0000 7f57e4bf7640 1 -- 172.21.15.43:0/1806327555 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f57d0001690 con 0x7f57e0161f30 2024-04-25T09:03:18.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.860+0000 7f57e4bf7640 1 -- 172.21.15.43:0/1806327555 >> v1:172.21.15.43:6789/0 conn(0x7f57e0161f30 legacy=0x7f57e00c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:18.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.860+0000 7f57e4bf7640 1 --2- 172.21.15.43:0/1806327555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57cc0039b0 0x7f57cc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:18.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.860+0000 7f57e4bf7640 1 --2- 172.21.15.43:0/1806327555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57cc004800 0x7f57cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:18.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.860+0000 7f57e4bf7640 1 --2- 172.21.15.43:0/1806327555 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f57cc0051c0 0x7f57cc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:18.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.860+0000 7f57e4bf7640 1 -- 172.21.15.43:0/1806327555 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e00cfa80 con 0x7f57cc0039b0 2024-04-25T09:03:18.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.860+0000 7f57e4bf7640 1 -- 172.21.15.43:0/1806327555 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc00c110 con 0x7f57cc004800 2024-04-25T09:03:18.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.860+0000 7f57e4bf7640 1 -- 172.21.15.43:0/1806327555 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc00c310 con 0x7f57cc0051c0 2024-04-25T09:03:18.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.860+0000 7f57e5bf9640 1 --2- 172.21.15.43:0/1806327555 >> v2:172.21.15.193:3300/0 conn(0x7f57e015df50 0x7f57e01619f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:18.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.860+0000 7f57e5bf9640 1 --2- 172.21.15.43:0/1806327555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57cc0039b0 0x7f57cc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:18.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.860+0000 7f57e53f8640 1 --2- 172.21.15.43:0/1806327555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57cc004800 0x7f57cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:18.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.860+0000 7f57e63fa640 1 --2- 172.21.15.43:0/1806327555 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f57cc0051c0 0x7f57cc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:18.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.861+0000 7f57e5bf9640 1 -- 172.21.15.43:0/1806327555 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f57cc0051c0 msgr2=0x7f57cc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:18.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.861+0000 7f57e5bf9640 1 --2- 172.21.15.43:0/1806327555 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f57cc0051c0 0x7f57cc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.861+0000 7f57e5bf9640 1 -- 172.21.15.43:0/1806327555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57cc004800 msgr2=0x7f57cc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:18.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.861+0000 7f57e5bf9640 1 --2- 172.21.15.43:0/1806327555 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57cc004800 0x7f57cc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.861+0000 7f57e5bf9640 1 -- 172.21.15.43:0/1806327555 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57cc003520 con 0x7f57cc0039b0 2024-04-25T09:03:18.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.861+0000 7f57e63fa640 1 -- 172.21.15.43:0/1806327555 reap_dead start 2024-04-25T09:03:18.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.861+0000 7f57e5bf9640 1 --2- 172.21.15.43:0/1806327555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57cc0039b0 0x7f57cc003dd0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f57dc0098a0 tx=0x7f57cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd7b1ed7c6634895 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:18.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.861+0000 7f57e4bf7640 1 -- 172.21.15.43:0/1806327555 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc037070 con 0x7f57cc0039b0 2024-04-25T09:03:18.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.861+0000 7f57e4bf7640 1 -- 172.21.15.43:0/1806327555 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57dc02aca0 con 0x7f57cc0039b0 2024-04-25T09:03:18.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.861+0000 7f57e4bf7640 1 -- 172.21.15.43:0/1806327555 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc00f040 con 0x7f57cc0039b0 2024-04-25T09:03:18.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.862+0000 7f57e7e5d640 1 -- 172.21.15.43:0/1806327555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57cc0039b0 msgr2=0x7f57cc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:18.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.862+0000 7f57e7e5d640 1 --2- 172.21.15.43:0/1806327555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57cc0039b0 0x7f57cc003dd0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f57dc0098a0 tx=0x7f57cc005830 comp rx=0 tx=0).stop 2024-04-25T09:03:18.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.862+0000 7f57e7e5d640 1 -- 172.21.15.43:0/1806327555 shutdown_connections 2024-04-25T09:03:18.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.862+0000 7f57e7e5d640 1 --2- 172.21.15.43:0/1806327555 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57cc0039b0 0x7f57cc003dd0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.862+0000 7f57e7e5d640 1 -- 172.21.15.43:0/1806327555 >> 172.21.15.43:0/1806327555 conn(0x7f57e00d3a30 msgr2=0x7f57e00d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:18.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.862+0000 7f57e7e5d640 1 -- 172.21.15.43:0/1806327555 shutdown_connections 2024-04-25T09:03:18.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.862+0000 7f57e7e5d640 1 -- 172.21.15.43:0/1806327555 wait complete. 2024-04-25T09:03:18.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.863+0000 7f57e7e5d640 1 Processor -- start 2024-04-25T09:03:18.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.863+0000 7f57e7e5d640 1 -- start start 2024-04-25T09:03:18.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.864+0000 7f57e7e5d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57cc0039b0 0x7f57e01f76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:18.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.864+0000 7f57e7e5d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57e015a3a0 0x7f57e01f7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:18.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.864+0000 7f57e7e5d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f57e01f3110 0x7f57e01f3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:18.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.864+0000 7f57e7e5d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e00c77d0 con 0x7f57cc0039b0 2024-04-25T09:03:18.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.864+0000 7f57e7e5d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e015e330 con 0x7f57e015a3a0 2024-04-25T09:03:18.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.864+0000 7f57e7e5d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e00d2190 con 0x7f57e01f3110 2024-04-25T09:03:18.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.864+0000 7f57e5bf9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57cc0039b0 0x7f57e01f76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:18.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.864+0000 7f57e5bf9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57cc0039b0 0x7f57e01f76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45392/0 (socket says 172.21.15.43:45392) 2024-04-25T09:03:18.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.864+0000 7f57e5bf9640 1 -- 172.21.15.43:0/4037455729 learned_addr learned my addr 172.21.15.43:0/4037455729 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:18.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.864+0000 7f57e63fa640 1 --2- 172.21.15.43:0/4037455729 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f57e01f3110 0x7f57e01f3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:18.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.864+0000 7f57e53f8640 1 --2- 172.21.15.43:0/4037455729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57e015a3a0 0x7f57e01f7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:18.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.864+0000 7f57e5bf9640 1 -- 172.21.15.43:0/4037455729 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f57e01f3110 msgr2=0x7f57e01f3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:18.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.864+0000 7f57e5bf9640 1 --2- 172.21.15.43:0/4037455729 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f57e01f3110 0x7f57e01f3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.864+0000 7f57e5bf9640 1 -- 172.21.15.43:0/4037455729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57e015a3a0 msgr2=0x7f57e01f7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:18.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.864+0000 7f57e5bf9640 1 --2- 172.21.15.43:0/4037455729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57e015a3a0 0x7f57e01f7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:18.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.864+0000 7f57e5bf9640 1 -- 172.21.15.43:0/4037455729 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e014ace0 con 0x7f57cc0039b0 2024-04-25T09:03:18.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.865+0000 7f57e5bf9640 1 --2- 172.21.15.43:0/4037455729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57cc0039b0 0x7f57e01f76a0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f57dc0099c0 tx=0x7f57dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:18.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.865+0000 7f57d67fc640 1 -- 172.21.15.43:0/4037455729 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc040020 con 0x7f57cc0039b0 2024-04-25T09:03:18.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.865+0000 7f57d67fc640 1 -- 172.21.15.43:0/4037455729 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57dc038040 con 0x7f57cc0039b0 2024-04-25T09:03:18.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.865+0000 7f57d67fc640 1 -- 172.21.15.43:0/4037455729 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc037070 con 0x7f57cc0039b0 2024-04-25T09:03:18.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.865+0000 7f57e7e5d640 1 -- 172.21.15.43:0/4037455729 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57e015a910 con 0x7f57cc0039b0 2024-04-25T09:03:18.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.865+0000 7f57e7e5d640 1 -- 172.21.15.43:0/4037455729 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e00cc850 con 0x7f57cc0039b0 2024-04-25T09:03:18.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.866+0000 7f57e7e5d640 1 -- 172.21.15.43:0/4037455729 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57e00680f0 con 0x7f57cc0039b0 2024-04-25T09:03:18.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.866+0000 7f57d67fc640 1 -- 172.21.15.43:0/4037455729 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f57dc043020 con 0x7f57cc0039b0 2024-04-25T09:03:18.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.867+0000 7f57d67fc640 1 --2- 172.21.15.43:0/4037455729 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f57c406b500 0x7f57c406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:18.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.867+0000 7f57d67fc640 1 -- 172.21.15.43:0/4037455729 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f57dc0aa170 con 0x7f57cc0039b0 2024-04-25T09:03:18.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.867+0000 7f57e53f8640 1 --2- 172.21.15.43:0/4037455729 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f57c406b500 0x7f57c406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:18.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.867+0000 7f57e53f8640 1 --2- 172.21.15.43:0/4037455729 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f57c406b500 0x7f57c406d9c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f57e01588a0 tx=0x7f57d0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:18.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:18.871+0000 7f57d67fc640 1 -- 172.21.15.43:0/4037455729 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f57dc04a030 con 0x7f57cc0039b0 2024-04-25T09:03:19.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.087+0000 7f57e7e5d640 1 -- 172.21.15.43:0/4037455729 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f57e00ca400 con 0x7f57cc0039b0 2024-04-25T09:03:19.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.087+0000 7f57d67fc640 1 -- 172.21.15.43:0/4037455729 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f57dc00f000 con 0x7f57cc0039b0 2024-04-25T09:03:19.076 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:03:19.076 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/891225068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":891225068},{"type":"v1","addr":"172.21.15.193:6836","nonce":891225068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":7175,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3679059390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":3679059390},{"type":"v1","addr":"172.21.15.193:6837","nonce":3679059390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":7948,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2127363183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2127363183},{"type":"v1","addr":"172.21.15.43:6835","nonce":2127363183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:11.919337+0000","modified":"2024-04-25T09:03:13.917768+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9237},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9237":{"gid":9237,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6834/2603892772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2603892772},{"type":"v1","addr":"172.21.15.43:6834","nonce":2603892772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:03:19.076 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 76 2024-04-25T09:03:19.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.089+0000 7f57e7e5d640 1 -- 172.21.15.43:0/4037455729 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f57c406b500 msgr2=0x7f57c406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:19.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.089+0000 7f57e7e5d640 1 --2- 172.21.15.43:0/4037455729 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f57c406b500 0x7f57c406d9c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f57e01588a0 tx=0x7f57d0009040 comp rx=0 tx=0).stop 2024-04-25T09:03:19.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.089+0000 7f57e7e5d640 1 -- 172.21.15.43:0/4037455729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57cc0039b0 msgr2=0x7f57e01f76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:19.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.089+0000 7f57e7e5d640 1 --2- 172.21.15.43:0/4037455729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57cc0039b0 0x7f57e01f76a0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f57dc0099c0 tx=0x7f57dc004730 comp rx=0 tx=0).stop 2024-04-25T09:03:19.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.089+0000 7f57e7e5d640 1 -- 172.21.15.43:0/4037455729 shutdown_connections 2024-04-25T09:03:19.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.089+0000 7f57e7e5d640 1 --2- 172.21.15.43:0/4037455729 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f57e01f3110 0x7f57e01f3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.089+0000 7f57e7e5d640 1 --2- 172.21.15.43:0/4037455729 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f57c406b500 0x7f57c406d9c0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.089+0000 7f57e7e5d640 1 --2- 172.21.15.43:0/4037455729 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f57e015a3a0 0x7f57e01f7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.089+0000 7f57e7e5d640 1 --2- 172.21.15.43:0/4037455729 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f57cc0039b0 0x7f57e01f76a0 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.089+0000 7f57e7e5d640 1 -- 172.21.15.43:0/4037455729 >> 172.21.15.43:0/4037455729 conn(0x7f57e00d3a30 msgr2=0x7f57e0157730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:19.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.089+0000 7f57e7e5d640 1 -- 172.21.15.43:0/4037455729 shutdown_connections 2024-04-25T09:03:19.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.089+0000 7f57e7e5d640 1 -- 172.21.15.43:0/4037455729 wait complete. 2024-04-25T09:03:19.089 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:19.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.201+0000 7f5be3de7640 1 Processor -- start 2024-04-25T09:03:19.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.202+0000 7f5be3de7640 1 -- start start 2024-04-25T09:03:19.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.202+0000 7f5be3de7640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5bdc0d39d0 0x7f5bdc0d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:19.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.202+0000 7f5be3de7640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f5bdc0d2500 0x7f5bdc0d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:19.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.202+0000 7f5be3de7640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bdc0b6c50 con 0x7f5bdc0ce2a0 2024-04-25T09:03:19.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.202+0000 7f5be3de7640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bdc15e010 con 0x7f5bdc0d2500 2024-04-25T09:03:19.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.202+0000 7f5be3de7640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc0d2020 con 0x7f5bdc0d39d0 2024-04-25T09:03:19.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.203+0000 7f5be1b83640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5bdc0d39d0 0x7f5bdc0d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:19.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.203+0000 7f5be2384640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f5bdc0d2500 0x7f5bdc0d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:19.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.203+0000 7f5be1b83640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5bdc0d39d0 0x7f5bdc0d0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:37230/0 (socket says 172.21.15.43:37230) 2024-04-25T09:03:19.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.203+0000 7f5be1382640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f5bdc0ce2a0 0x7f5bdc0d3db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:54612/0 (socket says 172.21.15.43:54612) 2024-04-25T09:03:19.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.203+0000 7f5be1b83640 1 -- 172.21.15.43:0/1223357912 learned_addr learned my addr 172.21.15.43:0/1223357912 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:19.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.203+0000 7f5be2384640 1 -- 172.21.15.43:0/1223357912 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5bdc0d39d0 msgr2=0x7f5bdc0d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:19.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.203+0000 7f5be2384640 1 --2- 172.21.15.43:0/1223357912 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5bdc0d39d0 0x7f5bdc0d0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.203+0000 7f5be2384640 1 -- 172.21.15.43:0/1223357912 >> v1:172.21.15.193:6789/0 conn(0x7f5bdc0ce2a0 legacy=0x7f5bdc0d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:19.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.203+0000 7f5be2384640 1 -- 172.21.15.43:0/1223357912 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bdc0cec40 con 0x7f5bdc0d2500 2024-04-25T09:03:19.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.203+0000 7f5be0b81640 1 -- 172.21.15.43:0/1223357912 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2947791131 0 0) 0x7f5bcc001110 con 0x7f5bdc0ce2a0 2024-04-25T09:03:19.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.204+0000 7f5be2384640 1 --2- 172.21.15.43:0/1223357912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5bdc0d2500 0x7f5bdc0d2990 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4009d90 tx=0x7f5bd402a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33b9e2441284ec03 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:19.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.204+0000 7f5be0b81640 1 -- 172.21.15.43:0/1223357912 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4031600 con 0x7f5bdc0d2500 2024-04-25T09:03:19.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.204+0000 7f5be0b81640 1 -- 172.21.15.43:0/1223357912 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd40317a0 con 0x7f5bdc0d2500 2024-04-25T09:03:19.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.204+0000 7f5be0b81640 1 -- 172.21.15.43:0/1223357912 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4031b70 con 0x7f5bdc0d2500 2024-04-25T09:03:19.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.204+0000 7f5be3de7640 1 -- 172.21.15.43:0/1223357912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5bdc0d2500 msgr2=0x7f5bdc0d2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:19.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.204+0000 7f5be3de7640 1 --2- 172.21.15.43:0/1223357912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5bdc0d2500 0x7f5bdc0d2990 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4009d90 tx=0x7f5bd402a5d0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.205+0000 7f5be3de7640 1 -- 172.21.15.43:0/1223357912 shutdown_connections 2024-04-25T09:03:19.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.205+0000 7f5be3de7640 1 --2- 172.21.15.43:0/1223357912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5bdc0d2500 0x7f5bdc0d2990 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.205+0000 7f5be3de7640 1 --2- 172.21.15.43:0/1223357912 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5bdc0d39d0 0x7f5bdc0d0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.205+0000 7f5be3de7640 1 -- 172.21.15.43:0/1223357912 >> 172.21.15.43:0/1223357912 conn(0x7f5bdc06b000 msgr2=0x7f5bdc06b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:19.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.205+0000 7f5be3de7640 1 -- 172.21.15.43:0/1223357912 shutdown_connections 2024-04-25T09:03:19.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.205+0000 7f5be3de7640 1 -- 172.21.15.43:0/1223357912 wait complete. 2024-04-25T09:03:19.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.206+0000 7f5be3de7640 1 Processor -- start 2024-04-25T09:03:19.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.206+0000 7f5be3de7640 1 -- start start 2024-04-25T09:03:19.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.207+0000 7f5be3de7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5bdc0ce2a0 0x7f5bdc14af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:19.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.207+0000 7f5be3de7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bdc0d2500 0x7f5bdc160410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:19.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.207+0000 7f5be3de7640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5bdc0d39d0 0x7f5bdc160950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:19.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.207+0000 7f5be3de7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc0cf200 con 0x7f5bdc0d2500 2024-04-25T09:03:19.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.207+0000 7f5be3de7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc0b2f00 con 0x7f5bdc0ce2a0 2024-04-25T09:03:19.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.207+0000 7f5be3de7640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc0d2ed0 con 0x7f5bdc0d39d0 2024-04-25T09:03:19.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.207+0000 7f5be1382640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bdc0d2500 0x7f5bdc160410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:19.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.207+0000 7f5be1382640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bdc0d2500 0x7f5bdc160410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:45402/0 (socket says 172.21.15.43:45402) 2024-04-25T09:03:19.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.207+0000 7f5be1382640 1 -- 172.21.15.43:0/4128587292 learned_addr learned my addr 172.21.15.43:0/4128587292 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:19.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.207+0000 7f5be1b83640 1 --2- 172.21.15.43:0/4128587292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5bdc0ce2a0 0x7f5bdc14af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:19.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.207+0000 7f5be2384640 1 --2- 172.21.15.43:0/4128587292 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5bdc0d39d0 0x7f5bdc160950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:19.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.207+0000 7f5be1382640 1 -- 172.21.15.43:0/4128587292 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5bdc0d39d0 msgr2=0x7f5bdc160950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:19.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.207+0000 7f5be1382640 1 --2- 172.21.15.43:0/4128587292 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5bdc0d39d0 0x7f5bdc160950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.207+0000 7f5be1382640 1 -- 172.21.15.43:0/4128587292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5bdc0ce2a0 msgr2=0x7f5bdc14af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:19.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.207+0000 7f5be1382640 1 --2- 172.21.15.43:0/4128587292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5bdc0ce2a0 0x7f5bdc14af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.207+0000 7f5be1382640 1 -- 172.21.15.43:0/4128587292 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bdc0b4390 con 0x7f5bdc0d2500 2024-04-25T09:03:19.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.207+0000 7f5be1382640 1 --2- 172.21.15.43:0/4128587292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bdc0d2500 0x7f5bdc160410 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc00c780 tx=0x7f5bcc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:19.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.207+0000 7f5bd27fc640 1 -- 172.21.15.43:0/4128587292 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc018070 con 0x7f5bdc0d2500 2024-04-25T09:03:19.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.207+0000 7f5bd27fc640 1 -- 172.21.15.43:0/4128587292 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bcc0072d0 con 0x7f5bdc0d2500 2024-04-25T09:03:19.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.207+0000 7f5be3de7640 1 -- 172.21.15.43:0/4128587292 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bdc0cbb90 con 0x7f5bdc0d2500 2024-04-25T09:03:19.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.207+0000 7f5bd27fc640 1 -- 172.21.15.43:0/4128587292 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc0047e0 con 0x7f5bdc0d2500 2024-04-25T09:03:19.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.208+0000 7f5be3de7640 1 -- 172.21.15.43:0/4128587292 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bdc0b4630 con 0x7f5bdc0d2500 2024-04-25T09:03:19.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.209+0000 7f5be3de7640 1 -- 172.21.15.43:0/4128587292 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bdc073270 con 0x7f5bdc0d2500 2024-04-25T09:03:19.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.209+0000 7f5bd27fc640 1 -- 172.21.15.43:0/4128587292 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5bcc01b070 con 0x7f5bdc0d2500 2024-04-25T09:03:19.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.209+0000 7f5bd27fc640 1 --2- 172.21.15.43:0/4128587292 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5bc006b4d0 0x7f5bc006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:19.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.210+0000 7f5bd27fc640 1 -- 172.21.15.43:0/4128587292 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5bcc0893a0 con 0x7f5bdc0d2500 2024-04-25T09:03:19.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.210+0000 7f5be1b83640 1 --2- 172.21.15.43:0/4128587292 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5bc006b4d0 0x7f5bc006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:19.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.210+0000 7f5be1b83640 1 --2- 172.21.15.43:0/4128587292 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5bc006b4d0 0x7f5bc006d990 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8009b30 tx=0x7f5bd8009240 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:19.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.214+0000 7f5bd27fc640 1 -- 172.21.15.43:0/4128587292 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5bdc073270 con 0x7f5bdc0d2500 2024-04-25T09:03:19.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.429+0000 7f5be3de7640 1 -- 172.21.15.43:0/4128587292 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5bdc0b2690 con 0x7f5bdc0d2500 2024-04-25T09:03:19.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.430+0000 7f5bd27fc640 1 -- 172.21.15.43:0/4128587292 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f5bdc0b6c50 con 0x7f5bdc0d2500 2024-04-25T09:03:19.419 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:03:19.419 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/891225068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":891225068},{"type":"v1","addr":"172.21.15.193:6836","nonce":891225068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":7175,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3679059390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":3679059390},{"type":"v1","addr":"172.21.15.193:6837","nonce":3679059390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":7948,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2127363183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2127363183},{"type":"v1","addr":"172.21.15.43:6835","nonce":2127363183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:11.919337+0000","modified":"2024-04-25T09:03:13.917768+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9237},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9237":{"gid":9237,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6834/2603892772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2603892772},{"type":"v1","addr":"172.21.15.43:6834","nonce":2603892772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:03:19.419 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 76 2024-04-25T09:03:19.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.432+0000 7f5be3de7640 1 -- 172.21.15.43:0/4128587292 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5bc006b4d0 msgr2=0x7f5bc006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:19.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.432+0000 7f5be3de7640 1 --2- 172.21.15.43:0/4128587292 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5bc006b4d0 0x7f5bc006d990 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8009b30 tx=0x7f5bd8009240 comp rx=0 tx=0).stop 2024-04-25T09:03:19.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.432+0000 7f5be3de7640 1 -- 172.21.15.43:0/4128587292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bdc0d2500 msgr2=0x7f5bdc160410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:19.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.432+0000 7f5be3de7640 1 --2- 172.21.15.43:0/4128587292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bdc0d2500 0x7f5bdc160410 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc00c780 tx=0x7f5bcc00cc40 comp rx=0 tx=0).stop 2024-04-25T09:03:19.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.432+0000 7f5be3de7640 1 -- 172.21.15.43:0/4128587292 shutdown_connections 2024-04-25T09:03:19.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.432+0000 7f5be3de7640 1 --2- 172.21.15.43:0/4128587292 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5bdc0d39d0 0x7f5bdc160950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.432+0000 7f5be3de7640 1 --2- 172.21.15.43:0/4128587292 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5bdc0d2500 0x7f5bdc160410 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.432+0000 7f5be3de7640 1 --2- 172.21.15.43:0/4128587292 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5bc006b4d0 0x7f5bc006d990 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.432+0000 7f5be3de7640 1 --2- 172.21.15.43:0/4128587292 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5bdc0ce2a0 0x7f5bdc14af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:19.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.432+0000 7f5be3de7640 1 -- 172.21.15.43:0/4128587292 >> 172.21.15.43:0/4128587292 conn(0x7f5bdc06b000 msgr2=0x7f5bdc15e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:19.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.432+0000 7f5be3de7640 1 -- 172.21.15.43:0/4128587292 shutdown_connections 2024-04-25T09:03:19.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:19.432+0000 7f5be3de7640 1 -- 172.21.15.43:0/4128587292 wait complete. 2024-04-25T09:03:19.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:19.432 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:03:19.432 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:03:19.432 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T09:03:19.432 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:03:19.432 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:03:19.432 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:03:19.432 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:03:19.471 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.471 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.471 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0118966 s, 88.1 MB/s 2024-04-25T09:03:19.482 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.482 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.482 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.009277 s, 113 MB/s 2024-04-25T09:03:19.492 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.492 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.492 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758106 s, 138 MB/s 2024-04-25T09:03:19.502 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.502 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.502 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788809 s, 133 MB/s 2024-04-25T09:03:19.513 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.513 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.513 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00887524 s, 118 MB/s 2024-04-25T09:03:19.523 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.523 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.523 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073878 s, 142 MB/s 2024-04-25T09:03:19.533 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.533 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.533 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0079508 s, 132 MB/s 2024-04-25T09:03:19.543 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.544 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.544 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00855771 s, 123 MB/s 2024-04-25T09:03:19.556 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.556 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.556 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00906234 s, 116 MB/s 2024-04-25T09:03:19.565 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.565 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.565 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749189 s, 140 MB/s 2024-04-25T09:03:19.574 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.574 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.574 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718335 s, 146 MB/s 2024-04-25T09:03:19.585 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.586 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.586 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00931384 s, 113 MB/s 2024-04-25T09:03:19.595 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.596 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.596 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00786284 s, 133 MB/s 2024-04-25T09:03:19.606 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.606 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.606 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.008127 s, 129 MB/s 2024-04-25T09:03:19.618 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.619 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.619 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0102657 s, 102 MB/s 2024-04-25T09:03:19.630 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.630 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.631 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00999058 s, 105 MB/s 2024-04-25T09:03:19.640 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.640 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.640 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787502 s, 133 MB/s 2024-04-25T09:03:19.650 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.650 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.650 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804034 s, 130 MB/s 2024-04-25T09:03:19.661 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.661 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.661 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804683 s, 130 MB/s 2024-04-25T09:03:19.670 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.671 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.671 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743346 s, 141 MB/s 2024-04-25T09:03:19.680 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.680 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.680 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741735 s, 141 MB/s 2024-04-25T09:03:19.691 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.691 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.692 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00932179 s, 112 MB/s 2024-04-25T09:03:19.702 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.702 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.702 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0081816 s, 128 MB/s 2024-04-25T09:03:19.712 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.713 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.713 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00849273 s, 123 MB/s 2024-04-25T09:03:19.724 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.724 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.724 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00940099 s, 112 MB/s 2024-04-25T09:03:19.734 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.734 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.735 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00809711 s, 130 MB/s 2024-04-25T09:03:19.745 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.745 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.745 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00798984 s, 131 MB/s 2024-04-25T09:03:19.756 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.756 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.756 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00874172 s, 120 MB/s 2024-04-25T09:03:19.765 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.765 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.765 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740436 s, 142 MB/s 2024-04-25T09:03:19.774 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.774 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.775 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726965 s, 144 MB/s 2024-04-25T09:03:19.785 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.786 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.786 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00886696 s, 118 MB/s 2024-04-25T09:03:19.798 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.798 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.798 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100064 s, 105 MB/s 2024-04-25T09:03:19.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:19.799 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:03:19.800 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:03:19.800 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T09:03:19.800 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:03:19.800 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:03:19.800 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:03:19.800 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:03:19.868 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.868 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.868 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00805231 s, 130 MB/s 2024-04-25T09:03:19.878 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.878 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.878 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811565 s, 129 MB/s 2024-04-25T09:03:19.889 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.889 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.889 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00893492 s, 117 MB/s 2024-04-25T09:03:19.899 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.899 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.899 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00779554 s, 135 MB/s 2024-04-25T09:03:19.908 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.909 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.909 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757698 s, 138 MB/s 2024-04-25T09:03:19.919 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.919 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.919 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0083964 s, 125 MB/s 2024-04-25T09:03:19.929 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.929 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.929 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735814 s, 143 MB/s 2024-04-25T09:03:19.939 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.939 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.939 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777193 s, 135 MB/s 2024-04-25T09:03:19.949 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.949 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.949 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770641 s, 136 MB/s 2024-04-25T09:03:19.959 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.959 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.959 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774425 s, 135 MB/s 2024-04-25T09:03:19.970 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.970 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.970 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0090432 s, 116 MB/s 2024-04-25T09:03:19.980 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.980 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.980 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804366 s, 130 MB/s 2024-04-25T09:03:19.990 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:19.990 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:19.990 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751683 s, 139 MB/s 2024-04-25T09:03:20.000 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.000 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.001 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00799043 s, 131 MB/s 2024-04-25T09:03:20.010 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.010 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.010 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00801354 s, 131 MB/s 2024-04-25T09:03:20.020 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.020 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.020 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802804 s, 131 MB/s 2024-04-25T09:03:20.029 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.029 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.029 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696041 s, 151 MB/s 2024-04-25T09:03:20.040 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.040 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.040 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00808129 s, 130 MB/s 2024-04-25T09:03:20.049 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.049 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.050 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075726 s, 138 MB/s 2024-04-25T09:03:20.059 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.059 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.059 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773681 s, 136 MB/s 2024-04-25T09:03:20.070 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.070 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.070 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00801467 s, 131 MB/s 2024-04-25T09:03:20.079 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.079 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.080 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763683 s, 137 MB/s 2024-04-25T09:03:20.090 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.090 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.090 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082492 s, 127 MB/s 2024-04-25T09:03:20.101 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.101 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.101 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00906751 s, 116 MB/s 2024-04-25T09:03:20.112 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.112 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.112 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00877067 s, 120 MB/s 2024-04-25T09:03:20.123 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.123 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.123 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00880458 s, 119 MB/s 2024-04-25T09:03:20.134 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.134 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.134 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00870252 s, 120 MB/s 2024-04-25T09:03:20.144 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.144 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.144 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00775801 s, 135 MB/s 2024-04-25T09:03:20.154 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.154 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.154 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834623 s, 126 MB/s 2024-04-25T09:03:20.165 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.165 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.165 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00837785 s, 125 MB/s 2024-04-25T09:03:20.175 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.175 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.175 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788706 s, 133 MB/s 2024-04-25T09:03:20.185 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.185 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.185 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00785149 s, 134 MB/s 2024-04-25T09:03:20.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:20.186 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:03:20.186 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:03:20.186 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T09:03:20.186 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:03:20.187 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:03:20.187 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:03:20.187 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:03:20.255 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.255 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.255 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840023 s, 125 MB/s 2024-04-25T09:03:20.264 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.264 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.264 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725909 s, 144 MB/s 2024-04-25T09:03:20.274 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.274 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.274 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793754 s, 132 MB/s 2024-04-25T09:03:20.284 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.284 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.284 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773054 s, 136 MB/s 2024-04-25T09:03:20.293 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.293 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.293 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723702 s, 145 MB/s 2024-04-25T09:03:20.303 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.303 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.303 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769115 s, 136 MB/s 2024-04-25T09:03:20.313 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.314 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.314 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00800794 s, 131 MB/s 2024-04-25T09:03:20.323 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.324 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.324 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771377 s, 136 MB/s 2024-04-25T09:03:20.333 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.333 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.333 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697985 s, 150 MB/s 2024-04-25T09:03:20.342 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.342 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.342 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072877 s, 144 MB/s 2024-04-25T09:03:20.352 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.352 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.352 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00795324 s, 132 MB/s 2024-04-25T09:03:20.362 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.362 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.362 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757894 s, 138 MB/s 2024-04-25T09:03:20.372 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.372 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.372 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072949 s, 144 MB/s 2024-04-25T09:03:20.381 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.381 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.381 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723573 s, 145 MB/s 2024-04-25T09:03:20.394 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.394 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.394 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00908497 s, 115 MB/s 2024-04-25T09:03:20.403 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.403 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.403 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835577 s, 125 MB/s 2024-04-25T09:03:20.413 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.413 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.413 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770288 s, 136 MB/s 2024-04-25T09:03:20.423 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.423 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.423 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00796653 s, 132 MB/s 2024-04-25T09:03:20.433 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.433 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.433 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00785271 s, 134 MB/s 2024-04-25T09:03:20.443 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.443 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.443 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00805273 s, 130 MB/s 2024-04-25T09:03:20.453 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.453 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.453 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00784739 s, 134 MB/s 2024-04-25T09:03:20.464 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.464 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.464 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00883843 s, 119 MB/s 2024-04-25T09:03:20.474 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.474 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.474 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00812737 s, 129 MB/s 2024-04-25T09:03:20.484 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.484 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.484 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771981 s, 136 MB/s 2024-04-25T09:03:20.493 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.493 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.494 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715304 s, 147 MB/s 2024-04-25T09:03:20.504 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.504 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.504 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00801446 s, 131 MB/s 2024-04-25T09:03:20.513 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.514 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.514 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762865 s, 137 MB/s 2024-04-25T09:03:20.525 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.525 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.525 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00888968 s, 118 MB/s 2024-04-25T09:03:20.536 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.536 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.536 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0091997 s, 114 MB/s 2024-04-25T09:03:20.546 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.546 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.546 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755617 s, 139 MB/s 2024-04-25T09:03:20.556 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.556 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.556 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831184 s, 126 MB/s 2024-04-25T09:03:20.567 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.567 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.567 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.008338 s, 126 MB/s 2024-04-25T09:03:20.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:20.568 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:03:20.568 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:03:20.568 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T09:03:20.568 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:03:20.568 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:03:20.569 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:03:20.569 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:03:20.637 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.637 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.637 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823919 s, 127 MB/s 2024-04-25T09:03:20.646 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.646 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.647 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776708 s, 135 MB/s 2024-04-25T09:03:20.657 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.657 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.657 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815743 s, 129 MB/s 2024-04-25T09:03:20.668 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.668 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.668 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00799342 s, 131 MB/s 2024-04-25T09:03:20.678 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.678 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.678 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00867735 s, 121 MB/s 2024-04-25T09:03:20.688 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.688 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.688 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749848 s, 140 MB/s 2024-04-25T09:03:20.697 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.697 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.697 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727578 s, 144 MB/s 2024-04-25T09:03:20.707 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.707 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.707 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780666 s, 134 MB/s 2024-04-25T09:03:20.717 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.718 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.718 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00805902 s, 130 MB/s 2024-04-25T09:03:20.729 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.729 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.729 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00924452 s, 113 MB/s 2024-04-25T09:03:20.738 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.738 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.738 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703637 s, 149 MB/s 2024-04-25T09:03:20.748 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.748 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.749 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797337 s, 132 MB/s 2024-04-25T09:03:20.758 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.758 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.758 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735072 s, 143 MB/s 2024-04-25T09:03:20.768 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.768 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.768 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765927 s, 137 MB/s 2024-04-25T09:03:20.777 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.777 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.778 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726943 s, 144 MB/s 2024-04-25T09:03:20.789 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.789 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.789 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00886401 s, 118 MB/s 2024-04-25T09:03:20.798 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.799 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.799 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773801 s, 136 MB/s 2024-04-25T09:03:20.810 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.810 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.810 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00903156 s, 116 MB/s 2024-04-25T09:03:20.820 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.820 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.820 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00794693 s, 132 MB/s 2024-04-25T09:03:20.830 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.830 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.830 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760742 s, 138 MB/s 2024-04-25T09:03:20.840 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.840 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.840 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828574 s, 127 MB/s 2024-04-25T09:03:20.850 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.850 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.850 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760837 s, 138 MB/s 2024-04-25T09:03:20.860 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.860 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.860 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760459 s, 138 MB/s 2024-04-25T09:03:20.870 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.870 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.870 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0078936 s, 133 MB/s 2024-04-25T09:03:20.880 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.880 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.880 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780927 s, 134 MB/s 2024-04-25T09:03:20.890 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.890 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.890 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766206 s, 137 MB/s 2024-04-25T09:03:20.900 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.900 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.900 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780813 s, 134 MB/s 2024-04-25T09:03:20.910 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.911 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.911 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788893 s, 133 MB/s 2024-04-25T09:03:20.921 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.921 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.921 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00796602 s, 132 MB/s 2024-04-25T09:03:20.930 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.931 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.931 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761324 s, 138 MB/s 2024-04-25T09:03:20.942 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.942 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.942 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00904267 s, 116 MB/s 2024-04-25T09:03:20.952 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:20.952 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:20.952 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00782192 s, 134 MB/s 2024-04-25T09:03:20.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:20.954 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:03:20.954 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:03:20.955 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T09:03:20.955 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:03:20.955 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:03:20.955 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:03:20.955 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:03:21.024 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.024 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.024 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00907351 s, 116 MB/s 2024-04-25T09:03:21.034 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.035 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.035 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00855757 s, 123 MB/s 2024-04-25T09:03:21.045 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.045 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.045 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823446 s, 127 MB/s 2024-04-25T09:03:21.054 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.054 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.054 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071323 s, 147 MB/s 2024-04-25T09:03:21.066 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.066 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.066 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00914594 s, 115 MB/s 2024-04-25T09:03:21.076 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.076 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.076 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811471 s, 129 MB/s 2024-04-25T09:03:21.087 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.087 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.087 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818651 s, 128 MB/s 2024-04-25T09:03:21.096 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.097 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.097 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754724 s, 139 MB/s 2024-04-25T09:03:21.108 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.108 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.108 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0091264 s, 115 MB/s 2024-04-25T09:03:21.119 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.119 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.119 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00859894 s, 122 MB/s 2024-04-25T09:03:21.131 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.131 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.131 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00974447 s, 108 MB/s 2024-04-25T09:03:21.141 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.141 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.141 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00796615 s, 132 MB/s 2024-04-25T09:03:21.152 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.152 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.152 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00883889 s, 119 MB/s 2024-04-25T09:03:21.162 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.162 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.162 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772273 s, 136 MB/s 2024-04-25T09:03:21.173 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.173 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.173 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788179 s, 133 MB/s 2024-04-25T09:03:21.183 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.184 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.184 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00880387 s, 119 MB/s 2024-04-25T09:03:21.193 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.193 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.193 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732922 s, 143 MB/s 2024-04-25T09:03:21.203 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.203 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.203 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734778 s, 143 MB/s 2024-04-25T09:03:21.214 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.214 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.214 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00879974 s, 119 MB/s 2024-04-25T09:03:21.223 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.224 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.224 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751097 s, 140 MB/s 2024-04-25T09:03:21.233 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.233 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.233 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763868 s, 137 MB/s 2024-04-25T09:03:21.243 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.243 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.243 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763738 s, 137 MB/s 2024-04-25T09:03:21.253 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.253 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.253 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760622 s, 138 MB/s 2024-04-25T09:03:21.262 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.263 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.263 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741192 s, 141 MB/s 2024-04-25T09:03:21.272 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.272 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.272 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752553 s, 139 MB/s 2024-04-25T09:03:21.283 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.283 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.283 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00873884 s, 120 MB/s 2024-04-25T09:03:21.293 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.293 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.293 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762752 s, 137 MB/s 2024-04-25T09:03:21.304 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.304 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.304 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00866674 s, 121 MB/s 2024-04-25T09:03:21.315 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.315 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.315 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00891319 s, 118 MB/s 2024-04-25T09:03:21.324 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.324 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.324 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718231 s, 146 MB/s 2024-04-25T09:03:21.335 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.335 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.335 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841682 s, 125 MB/s 2024-04-25T09:03:21.344 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.344 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.344 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716014 s, 146 MB/s 2024-04-25T09:03:21.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:21.345 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:03:21.345 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:03:21.345 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T09:03:21.345 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:03:21.345 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:03:21.345 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:03:21.346 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:03:21.414 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.414 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.414 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00886704 s, 118 MB/s 2024-04-25T09:03:21.424 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.425 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.425 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0080684 s, 130 MB/s 2024-04-25T09:03:21.435 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.435 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.435 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082006 s, 128 MB/s 2024-04-25T09:03:21.445 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.445 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.445 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797623 s, 131 MB/s 2024-04-25T09:03:21.455 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.455 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.455 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744701 s, 141 MB/s 2024-04-25T09:03:21.465 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.465 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.465 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00800837 s, 131 MB/s 2024-04-25T09:03:21.476 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.476 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.476 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00888004 s, 118 MB/s 2024-04-25T09:03:21.486 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.486 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.486 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778229 s, 135 MB/s 2024-04-25T09:03:21.497 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.497 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.497 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00898597 s, 117 MB/s 2024-04-25T09:03:21.507 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.507 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.508 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797328 s, 132 MB/s 2024-04-25T09:03:21.519 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.519 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.519 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0093064 s, 113 MB/s 2024-04-25T09:03:21.530 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.530 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.531 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00903723 s, 116 MB/s 2024-04-25T09:03:21.540 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.540 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.540 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00786341 s, 133 MB/s 2024-04-25T09:03:21.550 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.550 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.550 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0078974 s, 133 MB/s 2024-04-25T09:03:21.561 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.561 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.561 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00886923 s, 118 MB/s 2024-04-25T09:03:21.570 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.571 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.571 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707735 s, 148 MB/s 2024-04-25T09:03:21.581 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.581 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.581 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826101 s, 127 MB/s 2024-04-25T09:03:21.590 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.591 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.591 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728806 s, 144 MB/s 2024-04-25T09:03:21.601 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.601 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.602 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00867561 s, 121 MB/s 2024-04-25T09:03:21.611 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.611 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.611 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708639 s, 148 MB/s 2024-04-25T09:03:21.620 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.621 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.621 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766844 s, 137 MB/s 2024-04-25T09:03:21.632 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.632 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.632 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00939767 s, 112 MB/s 2024-04-25T09:03:21.643 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.643 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.643 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00864458 s, 121 MB/s 2024-04-25T09:03:21.653 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.653 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.653 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00800676 s, 131 MB/s 2024-04-25T09:03:21.664 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.664 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.664 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835326 s, 126 MB/s 2024-04-25T09:03:21.675 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.675 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.675 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00883994 s, 119 MB/s 2024-04-25T09:03:21.685 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.685 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.685 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791956 s, 132 MB/s 2024-04-25T09:03:21.694 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.694 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.694 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699981 s, 150 MB/s 2024-04-25T09:03:21.705 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.705 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.705 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00899364 s, 117 MB/s 2024-04-25T09:03:21.716 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.716 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.716 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00863572 s, 121 MB/s 2024-04-25T09:03:21.725 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.726 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.726 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736026 s, 142 MB/s 2024-04-25T09:03:21.736 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.736 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.736 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831418 s, 126 MB/s 2024-04-25T09:03:21.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:21.737 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:03:21.737 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:03:21.737 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T09:03:21.737 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:03:21.738 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:03:21.738 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:03:21.738 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:03:21.805 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.805 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.805 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718749 s, 146 MB/s 2024-04-25T09:03:21.815 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.815 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.815 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00803133 s, 131 MB/s 2024-04-25T09:03:21.825 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.825 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.825 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820324 s, 128 MB/s 2024-04-25T09:03:21.836 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.836 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.836 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00880502 s, 119 MB/s 2024-04-25T09:03:21.847 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.847 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.847 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0085661 s, 122 MB/s 2024-04-25T09:03:21.857 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.857 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.857 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815506 s, 129 MB/s 2024-04-25T09:03:21.867 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.867 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.867 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778652 s, 135 MB/s 2024-04-25T09:03:21.877 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.877 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.877 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00785308 s, 134 MB/s 2024-04-25T09:03:21.887 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.887 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.887 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736766 s, 142 MB/s 2024-04-25T09:03:21.897 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.897 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.897 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756459 s, 139 MB/s 2024-04-25T09:03:21.907 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.907 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.907 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797319 s, 132 MB/s 2024-04-25T09:03:21.916 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.916 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.916 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684706 s, 153 MB/s 2024-04-25T09:03:21.926 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.927 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.927 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839875 s, 125 MB/s 2024-04-25T09:03:21.937 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.937 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.937 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815233 s, 129 MB/s 2024-04-25T09:03:21.947 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.947 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.947 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791498 s, 132 MB/s 2024-04-25T09:03:21.957 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.957 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.957 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769613 s, 136 MB/s 2024-04-25T09:03:21.966 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.967 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.967 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700954 s, 150 MB/s 2024-04-25T09:03:21.979 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.979 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.979 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100536 s, 104 MB/s 2024-04-25T09:03:21.988 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:21.988 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:21.988 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682144 s, 154 MB/s 2024-04-25T09:03:21.999 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.000 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.000 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00922435 s, 114 MB/s 2024-04-25T09:03:22.009 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.009 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.010 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00796767 s, 132 MB/s 2024-04-25T09:03:22.020 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.020 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.020 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00808102 s, 130 MB/s 2024-04-25T09:03:22.029 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.029 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.029 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710378 s, 148 MB/s 2024-04-25T09:03:22.040 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.040 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.040 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00854194 s, 123 MB/s 2024-04-25T09:03:22.050 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.050 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.050 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00784547 s, 134 MB/s 2024-04-25T09:03:22.060 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.060 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.060 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759723 s, 138 MB/s 2024-04-25T09:03:22.069 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.069 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.069 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754061 s, 139 MB/s 2024-04-25T09:03:22.080 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.080 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.080 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838371 s, 125 MB/s 2024-04-25T09:03:22.090 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.090 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.090 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783579 s, 134 MB/s 2024-04-25T09:03:22.101 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.101 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.101 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00870702 s, 120 MB/s 2024-04-25T09:03:22.111 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.111 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.111 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766972 s, 137 MB/s 2024-04-25T09:03:22.121 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.121 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.121 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777375 s, 135 MB/s 2024-04-25T09:03:22.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:22.122 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:03:22.122 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:03:22.122 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T09:03:22.123 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:03:22.123 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:03:22.123 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:03:22.123 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:03:22.191 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.191 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.191 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00862346 s, 122 MB/s 2024-04-25T09:03:22.200 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.200 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.201 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706809 s, 148 MB/s 2024-04-25T09:03:22.210 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.210 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.210 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762148 s, 138 MB/s 2024-04-25T09:03:22.220 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.220 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.220 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725622 s, 145 MB/s 2024-04-25T09:03:22.230 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.230 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.230 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788888 s, 133 MB/s 2024-04-25T09:03:22.241 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.241 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.241 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00870176 s, 121 MB/s 2024-04-25T09:03:22.249 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.250 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.250 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674249 s, 156 MB/s 2024-04-25T09:03:22.260 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.261 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.261 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00879985 s, 119 MB/s 2024-04-25T09:03:22.271 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.271 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.271 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00806439 s, 130 MB/s 2024-04-25T09:03:22.280 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.280 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.281 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733711 s, 143 MB/s 2024-04-25T09:03:22.290 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.291 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.291 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787249 s, 133 MB/s 2024-04-25T09:03:22.301 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.301 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.301 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00843245 s, 124 MB/s 2024-04-25T09:03:22.312 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.312 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.312 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838273 s, 125 MB/s 2024-04-25T09:03:22.322 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.322 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.322 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839186 s, 125 MB/s 2024-04-25T09:03:22.332 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.332 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.332 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737877 s, 142 MB/s 2024-04-25T09:03:22.342 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.343 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.343 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817795 s, 128 MB/s 2024-04-25T09:03:22.353 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.354 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.354 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00867712 s, 121 MB/s 2024-04-25T09:03:22.363 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.363 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.364 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762348 s, 138 MB/s 2024-04-25T09:03:22.374 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.374 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.375 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00889307 s, 118 MB/s 2024-04-25T09:03:22.385 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.385 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.385 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845286 s, 124 MB/s 2024-04-25T09:03:22.396 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.396 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.396 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00857869 s, 122 MB/s 2024-04-25T09:03:22.405 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.405 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.405 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688404 s, 152 MB/s 2024-04-25T09:03:22.415 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.415 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.415 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776249 s, 135 MB/s 2024-04-25T09:03:22.426 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.426 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.426 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00869952 s, 121 MB/s 2024-04-25T09:03:22.436 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.436 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.436 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762242 s, 138 MB/s 2024-04-25T09:03:22.446 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.446 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.446 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762734 s, 137 MB/s 2024-04-25T09:03:22.456 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.456 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.456 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00800194 s, 131 MB/s 2024-04-25T09:03:22.465 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.466 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.466 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680374 s, 154 MB/s 2024-04-25T09:03:22.474 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.474 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.474 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682578 s, 154 MB/s 2024-04-25T09:03:22.485 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.485 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.486 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00872374 s, 120 MB/s 2024-04-25T09:03:22.495 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.495 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.495 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773988 s, 135 MB/s 2024-04-25T09:03:22.506 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.506 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.506 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826191 s, 127 MB/s 2024-04-25T09:03:22.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:22.507 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:03:22.507 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:03:22.507 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T09:03:22.507 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:03:22.507 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:03:22.507 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:03:22.507 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:03:22.579 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.579 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.579 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00855493 s, 123 MB/s 2024-04-25T09:03:22.590 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.590 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.590 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00874017 s, 120 MB/s 2024-04-25T09:03:22.599 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.599 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.599 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681729 s, 154 MB/s 2024-04-25T09:03:22.609 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.609 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.609 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791565 s, 132 MB/s 2024-04-25T09:03:22.619 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.619 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.619 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767709 s, 137 MB/s 2024-04-25T09:03:22.629 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.629 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.629 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720602 s, 146 MB/s 2024-04-25T09:03:22.638 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.639 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.639 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743748 s, 141 MB/s 2024-04-25T09:03:22.649 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.650 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.650 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00865364 s, 121 MB/s 2024-04-25T09:03:22.660 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.660 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.660 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841523 s, 125 MB/s 2024-04-25T09:03:22.669 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.669 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.669 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684317 s, 153 MB/s 2024-04-25T09:03:22.679 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.679 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.680 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787854 s, 133 MB/s 2024-04-25T09:03:22.688 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.688 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.689 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070608 s, 149 MB/s 2024-04-25T09:03:22.698 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.698 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.698 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753859 s, 139 MB/s 2024-04-25T09:03:22.708 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.708 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.708 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758819 s, 138 MB/s 2024-04-25T09:03:22.717 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.718 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.718 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760935 s, 138 MB/s 2024-04-25T09:03:22.728 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.728 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.728 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840783 s, 125 MB/s 2024-04-25T09:03:22.737 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.737 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.737 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681673 s, 154 MB/s 2024-04-25T09:03:22.746 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.746 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.747 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715458 s, 147 MB/s 2024-04-25T09:03:22.756 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.756 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.756 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753224 s, 139 MB/s 2024-04-25T09:03:22.767 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.767 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.767 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832673 s, 126 MB/s 2024-04-25T09:03:22.778 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.778 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.778 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00862354 s, 122 MB/s 2024-04-25T09:03:22.788 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.788 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.788 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745019 s, 141 MB/s 2024-04-25T09:03:22.797 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.798 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.798 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764493 s, 137 MB/s 2024-04-25T09:03:22.808 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.808 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.808 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850423 s, 123 MB/s 2024-04-25T09:03:22.818 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.818 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.818 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742319 s, 141 MB/s 2024-04-25T09:03:22.828 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.828 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.828 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00858196 s, 122 MB/s 2024-04-25T09:03:22.838 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.838 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.838 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759066 s, 138 MB/s 2024-04-25T09:03:22.848 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.848 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.849 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769468 s, 136 MB/s 2024-04-25T09:03:22.859 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.859 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.859 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00855948 s, 123 MB/s 2024-04-25T09:03:22.869 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.870 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.870 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836072 s, 125 MB/s 2024-04-25T09:03:22.880 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.880 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.880 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0080621 s, 130 MB/s 2024-04-25T09:03:22.889 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.890 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.890 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759129 s, 138 MB/s 2024-04-25T09:03:22.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:22.891 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:03:22.891 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:03:22.891 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T09:03:22.891 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:03:22.891 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:03:22.891 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:03:22.891 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:03:22.958 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.958 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.958 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697708 s, 150 MB/s 2024-04-25T09:03:22.968 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.968 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.968 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744519 s, 141 MB/s 2024-04-25T09:03:22.978 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.979 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.979 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0086001 s, 122 MB/s 2024-04-25T09:03:22.988 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.988 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.988 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733402 s, 143 MB/s 2024-04-25T09:03:22.999 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:22.999 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:22.999 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838682 s, 125 MB/s 2024-04-25T09:03:23.008 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.009 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.009 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754757 s, 139 MB/s 2024-04-25T09:03:23.018 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.018 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.018 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737932 s, 142 MB/s 2024-04-25T09:03:23.029 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.029 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.029 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840861 s, 125 MB/s 2024-04-25T09:03:23.040 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.040 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.040 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00867469 s, 121 MB/s 2024-04-25T09:03:23.049 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.049 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.050 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740737 s, 142 MB/s 2024-04-25T09:03:23.060 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.060 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.060 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836992 s, 125 MB/s 2024-04-25T09:03:23.070 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.070 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.070 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731036 s, 143 MB/s 2024-04-25T09:03:23.079 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.079 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.079 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749494 s, 140 MB/s 2024-04-25T09:03:23.090 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.090 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.090 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834806 s, 126 MB/s 2024-04-25T09:03:23.100 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.100 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.100 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804492 s, 130 MB/s 2024-04-25T09:03:23.110 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.110 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.110 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710755 s, 148 MB/s 2024-04-25T09:03:23.120 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.120 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.120 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765029 s, 137 MB/s 2024-04-25T09:03:23.129 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.130 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.130 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757446 s, 138 MB/s 2024-04-25T09:03:23.139 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.139 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.139 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700999 s, 150 MB/s 2024-04-25T09:03:23.148 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.149 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.149 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747166 s, 140 MB/s 2024-04-25T09:03:23.158 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.158 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.158 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735644 s, 143 MB/s 2024-04-25T09:03:23.169 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.169 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.169 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00873171 s, 120 MB/s 2024-04-25T09:03:23.179 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.179 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.179 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744347 s, 141 MB/s 2024-04-25T09:03:23.190 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.190 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.190 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00860249 s, 122 MB/s 2024-04-25T09:03:23.201 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.201 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.201 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00889566 s, 118 MB/s 2024-04-25T09:03:23.210 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.211 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.211 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073866 s, 142 MB/s 2024-04-25T09:03:23.220 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.220 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.220 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756962 s, 139 MB/s 2024-04-25T09:03:23.230 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.230 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.230 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743172 s, 141 MB/s 2024-04-25T09:03:23.239 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.239 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.239 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700545 s, 150 MB/s 2024-04-25T09:03:23.249 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.249 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.249 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744433 s, 141 MB/s 2024-04-25T09:03:23.259 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.259 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.259 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752966 s, 139 MB/s 2024-04-25T09:03:23.269 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.269 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.270 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692252 s, 151 MB/s 2024-04-25T09:03:23.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:23.270 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:03:23.270 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:03:23.270 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T09:03:23.270 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:03:23.270 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:03:23.270 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:03:23.271 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:03:23.338 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.338 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.339 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00862424 s, 122 MB/s 2024-04-25T09:03:23.348 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.348 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.348 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783093 s, 134 MB/s 2024-04-25T09:03:23.357 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.357 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.357 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724953 s, 145 MB/s 2024-04-25T09:03:23.368 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.368 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.368 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810534 s, 129 MB/s 2024-04-25T09:03:23.378 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.378 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.379 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00869889 s, 121 MB/s 2024-04-25T09:03:23.388 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.388 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.388 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720367 s, 146 MB/s 2024-04-25T09:03:23.398 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.398 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.399 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00844916 s, 124 MB/s 2024-04-25T09:03:23.407 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.408 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.408 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682926 s, 154 MB/s 2024-04-25T09:03:23.418 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.418 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.418 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772403 s, 136 MB/s 2024-04-25T09:03:23.427 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.427 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.427 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695695 s, 151 MB/s 2024-04-25T09:03:23.436 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.436 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.437 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742921 s, 141 MB/s 2024-04-25T09:03:23.446 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.446 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.446 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721317 s, 145 MB/s 2024-04-25T09:03:23.457 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.457 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.457 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00877272 s, 120 MB/s 2024-04-25T09:03:23.467 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.467 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.467 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777421 s, 135 MB/s 2024-04-25T09:03:23.476 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.476 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.477 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736178 s, 142 MB/s 2024-04-25T09:03:23.486 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.486 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.486 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739342 s, 142 MB/s 2024-04-25T09:03:23.495 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.495 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.495 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713203 s, 147 MB/s 2024-04-25T09:03:23.505 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.505 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.505 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745558 s, 141 MB/s 2024-04-25T09:03:23.514 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.514 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.514 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680792 s, 154 MB/s 2024-04-25T09:03:23.524 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.525 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.525 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838258 s, 125 MB/s 2024-04-25T09:03:23.535 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.536 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.536 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00884835 s, 119 MB/s 2024-04-25T09:03:23.545 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.545 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.545 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734164 s, 143 MB/s 2024-04-25T09:03:23.554 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.555 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.555 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741621 s, 141 MB/s 2024-04-25T09:03:23.565 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.565 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.565 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00807666 s, 130 MB/s 2024-04-25T09:03:23.574 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.574 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.574 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708565 s, 148 MB/s 2024-04-25T09:03:23.584 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.584 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.584 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743861 s, 141 MB/s 2024-04-25T09:03:23.593 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.593 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.594 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728852 s, 144 MB/s 2024-04-25T09:03:23.603 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.603 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.603 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745044 s, 141 MB/s 2024-04-25T09:03:23.613 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.613 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.613 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722347 s, 145 MB/s 2024-04-25T09:03:23.622 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.622 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.622 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742858 s, 141 MB/s 2024-04-25T09:03:23.631 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.631 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.631 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068851 s, 152 MB/s 2024-04-25T09:03:23.640 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.640 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.640 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679585 s, 154 MB/s 2024-04-25T09:03:23.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:23.641 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:03:23.641 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:03:23.642 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T09:03:23.642 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:03:23.642 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:03:23.642 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:03:23.642 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:03:23.709 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.709 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.709 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712212 s, 147 MB/s 2024-04-25T09:03:23.720 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.720 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.720 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00879888 s, 119 MB/s 2024-04-25T09:03:23.730 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.730 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.730 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772361 s, 136 MB/s 2024-04-25T09:03:23.739 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.739 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.739 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689322 s, 152 MB/s 2024-04-25T09:03:23.748 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.748 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.748 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711938 s, 147 MB/s 2024-04-25T09:03:23.758 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.758 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.758 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730521 s, 144 MB/s 2024-04-25T09:03:23.767 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.768 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.768 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731679 s, 143 MB/s 2024-04-25T09:03:23.777 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.777 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.777 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717614 s, 146 MB/s 2024-04-25T09:03:23.787 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.788 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.788 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834678 s, 126 MB/s 2024-04-25T09:03:23.796 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.796 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.796 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669195 s, 157 MB/s 2024-04-25T09:03:23.806 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.806 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.806 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706846 s, 148 MB/s 2024-04-25T09:03:23.816 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.816 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.816 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792037 s, 132 MB/s 2024-04-25T09:03:23.825 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.825 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.825 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709874 s, 148 MB/s 2024-04-25T09:03:23.836 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.836 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.836 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00894044 s, 117 MB/s 2024-04-25T09:03:23.845 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.846 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.846 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735665 s, 143 MB/s 2024-04-25T09:03:23.857 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.857 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.857 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00910089 s, 115 MB/s 2024-04-25T09:03:23.868 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.868 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.868 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0090218 s, 116 MB/s 2024-04-25T09:03:23.880 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.880 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.880 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00900027 s, 117 MB/s 2024-04-25T09:03:23.889 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.889 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.889 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734138 s, 143 MB/s 2024-04-25T09:03:23.899 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.899 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.899 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721789 s, 145 MB/s 2024-04-25T09:03:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.910 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00907478 s, 116 MB/s 2024-04-25T09:03:23.919 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.920 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.920 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737987 s, 142 MB/s 2024-04-25T09:03:23.929 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.929 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.929 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722809 s, 145 MB/s 2024-04-25T09:03:23.938 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.938 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.939 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072023 s, 146 MB/s 2024-04-25T09:03:23.949 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.949 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.949 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00859853 s, 122 MB/s 2024-04-25T09:03:23.959 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.959 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.959 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716364 s, 146 MB/s 2024-04-25T09:03:23.969 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.969 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.969 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747548 s, 140 MB/s 2024-04-25T09:03:23.978 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.978 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.978 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732065 s, 143 MB/s 2024-04-25T09:03:23.987 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.987 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.987 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690388 s, 152 MB/s 2024-04-25T09:03:23.998 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:23.998 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:23.999 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00896007 s, 117 MB/s 2024-04-25T09:03:24.008 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.008 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.008 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724733 s, 145 MB/s 2024-04-25T09:03:24.017 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.017 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.017 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689974 s, 152 MB/s 2024-04-25T09:03:24.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:24.019 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:03:24.019 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:03:24.019 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T09:03:24.019 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:03:24.019 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:03:24.019 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:03:24.019 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:03:24.086 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.086 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.086 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738243 s, 142 MB/s 2024-04-25T09:03:24.096 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.096 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.096 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826502 s, 127 MB/s 2024-04-25T09:03:24.106 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.106 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.106 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007599 s, 138 MB/s 2024-04-25T09:03:24.115 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.115 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.115 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736567 s, 142 MB/s 2024-04-25T09:03:24.125 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.125 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.125 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739313 s, 142 MB/s 2024-04-25T09:03:24.134 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.134 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.135 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069694 s, 150 MB/s 2024-04-25T09:03:24.145 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.145 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.145 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824478 s, 127 MB/s 2024-04-25T09:03:24.155 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.155 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.155 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819765 s, 128 MB/s 2024-04-25T09:03:24.164 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.164 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.164 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686547 s, 153 MB/s 2024-04-25T09:03:24.174 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.174 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.174 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753777 s, 139 MB/s 2024-04-25T09:03:24.184 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.184 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.184 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753812 s, 139 MB/s 2024-04-25T09:03:24.193 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.193 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.193 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724853 s, 145 MB/s 2024-04-25T09:03:24.204 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.204 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.204 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00870324 s, 120 MB/s 2024-04-25T09:03:24.214 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.214 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.214 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082439 s, 127 MB/s 2024-04-25T09:03:24.223 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.224 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.224 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719081 s, 146 MB/s 2024-04-25T09:03:24.233 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.233 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.233 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717914 s, 146 MB/s 2024-04-25T09:03:24.243 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.243 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.243 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817344 s, 128 MB/s 2024-04-25T09:03:24.253 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.253 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.253 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728626 s, 144 MB/s 2024-04-25T09:03:24.262 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.262 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.262 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715879 s, 146 MB/s 2024-04-25T09:03:24.271 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.272 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.272 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703635 s, 149 MB/s 2024-04-25T09:03:24.282 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.282 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.282 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720325 s, 146 MB/s 2024-04-25T09:03:24.290 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.290 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.291 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729675 s, 144 MB/s 2024-04-25T09:03:24.300 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.300 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.300 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737624 s, 142 MB/s 2024-04-25T09:03:24.309 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.309 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.309 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070929 s, 148 MB/s 2024-04-25T09:03:24.319 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.319 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.319 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754401 s, 139 MB/s 2024-04-25T09:03:24.330 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.330 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.330 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00869403 s, 121 MB/s 2024-04-25T09:03:24.339 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.339 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.340 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714059 s, 147 MB/s 2024-04-25T09:03:24.349 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.349 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.349 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734188 s, 143 MB/s 2024-04-25T09:03:24.359 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.359 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.359 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814581 s, 129 MB/s 2024-04-25T09:03:24.369 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.369 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.369 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075848 s, 138 MB/s 2024-04-25T09:03:24.378 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.378 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.378 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713581 s, 147 MB/s 2024-04-25T09:03:24.388 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.388 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.388 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716479 s, 146 MB/s 2024-04-25T09:03:24.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:24.389 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:03:24.389 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:03:24.389 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T09:03:24.389 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:03:24.389 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:03:24.389 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:03:24.389 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:03:24.457 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.457 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.457 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724074 s, 145 MB/s 2024-04-25T09:03:24.466 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.466 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.466 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721688 s, 145 MB/s 2024-04-25T09:03:24.477 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.477 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.477 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00844042 s, 124 MB/s 2024-04-25T09:03:24.486 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.486 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.486 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734864 s, 143 MB/s 2024-04-25T09:03:24.496 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.496 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.496 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755706 s, 139 MB/s 2024-04-25T09:03:24.505 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.505 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.505 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719514 s, 146 MB/s 2024-04-25T09:03:24.514 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.514 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.515 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724891 s, 145 MB/s 2024-04-25T09:03:24.525 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.525 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.525 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841514 s, 125 MB/s 2024-04-25T09:03:24.534 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.534 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.535 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711759 s, 147 MB/s 2024-04-25T09:03:24.544 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.544 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.544 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743335 s, 141 MB/s 2024-04-25T09:03:24.553 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.554 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.554 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718964 s, 146 MB/s 2024-04-25T09:03:24.563 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.563 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.563 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740251 s, 142 MB/s 2024-04-25T09:03:24.572 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.573 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.573 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710555 s, 148 MB/s 2024-04-25T09:03:24.582 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.582 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.582 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704922 s, 149 MB/s 2024-04-25T09:03:24.594 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.594 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.594 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100567 s, 104 MB/s 2024-04-25T09:03:24.604 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.604 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.604 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709146 s, 148 MB/s 2024-04-25T09:03:24.614 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.614 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.615 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00865884 s, 121 MB/s 2024-04-25T09:03:24.624 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.624 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.624 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693099 s, 151 MB/s 2024-04-25T09:03:24.633 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.633 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.633 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073049 s, 144 MB/s 2024-04-25T09:03:24.642 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.642 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.642 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666888 s, 157 MB/s 2024-04-25T09:03:24.651 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.651 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.652 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071039 s, 148 MB/s 2024-04-25T09:03:24.660 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.661 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.661 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006771 s, 155 MB/s 2024-04-25T09:03:24.670 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.670 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.670 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070725 s, 148 MB/s 2024-04-25T09:03:24.679 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.679 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.679 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692142 s, 151 MB/s 2024-04-25T09:03:24.690 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.690 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.690 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723964 s, 145 MB/s 2024-04-25T09:03:24.698 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.698 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.698 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752638 s, 139 MB/s 2024-04-25T09:03:24.708 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.708 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.708 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727319 s, 144 MB/s 2024-04-25T09:03:24.718 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.718 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.718 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783065 s, 134 MB/s 2024-04-25T09:03:24.728 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.728 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.728 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763889 s, 137 MB/s 2024-04-25T09:03:24.737 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.737 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.737 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735893 s, 142 MB/s 2024-04-25T09:03:24.748 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.748 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.748 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0086628 s, 121 MB/s 2024-04-25T09:03:24.757 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.757 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.758 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714185 s, 147 MB/s 2024-04-25T09:03:24.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:24.759 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:03:24.759 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:03:24.759 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T09:03:24.759 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:03:24.759 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:03:24.759 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:03:24.759 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:03:24.827 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.828 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.828 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742236 s, 141 MB/s 2024-04-25T09:03:24.837 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.837 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.837 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730641 s, 144 MB/s 2024-04-25T09:03:24.847 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.847 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.847 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007566 s, 139 MB/s 2024-04-25T09:03:24.856 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.856 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.856 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071481 s, 147 MB/s 2024-04-25T09:03:24.867 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.867 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.867 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0083392 s, 126 MB/s 2024-04-25T09:03:24.876 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.876 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.876 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687333 s, 153 MB/s 2024-04-25T09:03:24.885 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.885 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.885 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073032 s, 144 MB/s 2024-04-25T09:03:24.896 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.896 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.896 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845962 s, 124 MB/s 2024-04-25T09:03:24.906 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.906 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.906 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744651 s, 141 MB/s 2024-04-25T09:03:24.917 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.917 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.917 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00887174 s, 118 MB/s 2024-04-25T09:03:24.927 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.927 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.927 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828961 s, 126 MB/s 2024-04-25T09:03:24.937 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.937 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.937 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749665 s, 140 MB/s 2024-04-25T09:03:24.948 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.948 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.948 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00860559 s, 122 MB/s 2024-04-25T09:03:24.957 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.957 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.958 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737999 s, 142 MB/s 2024-04-25T09:03:24.967 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.967 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.967 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746983 s, 140 MB/s 2024-04-25T09:03:24.977 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.977 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.978 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829533 s, 126 MB/s 2024-04-25T09:03:24.987 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.987 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.987 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705535 s, 149 MB/s 2024-04-25T09:03:24.996 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:24.996 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:24.996 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674023 s, 156 MB/s 2024-04-25T09:03:25.005 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.005 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.005 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677681 s, 155 MB/s 2024-04-25T09:03:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.015 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734972 s, 143 MB/s 2024-04-25T09:03:25.024 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.024 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.024 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736047 s, 142 MB/s 2024-04-25T09:03:25.034 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.034 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.034 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759037 s, 138 MB/s 2024-04-25T09:03:25.043 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.043 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.044 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075238 s, 139 MB/s 2024-04-25T09:03:25.053 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.053 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.053 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736122 s, 142 MB/s 2024-04-25T09:03:25.062 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.062 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.062 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714252 s, 147 MB/s 2024-04-25T09:03:25.072 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.072 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.072 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743759 s, 141 MB/s 2024-04-25T09:03:25.082 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.082 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.082 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740833 s, 142 MB/s 2024-04-25T09:03:25.092 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.092 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.092 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757771 s, 138 MB/s 2024-04-25T09:03:25.102 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.102 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.102 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825268 s, 127 MB/s 2024-04-25T09:03:25.112 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.112 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.112 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719458 s, 146 MB/s 2024-04-25T09:03:25.121 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.121 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.121 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717598 s, 146 MB/s 2024-04-25T09:03:25.132 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.132 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.132 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00843361 s, 124 MB/s 2024-04-25T09:03:25.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:25.133 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:03:25.133 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:03:25.133 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T09:03:25.133 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:03:25.133 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:03:25.133 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:03:25.134 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:03:25.201 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.201 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.201 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752051 s, 139 MB/s 2024-04-25T09:03:25.210 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.211 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.211 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716854 s, 146 MB/s 2024-04-25T09:03:25.221 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.221 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.221 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804057 s, 130 MB/s 2024-04-25T09:03:25.230 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.230 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.230 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669515 s, 157 MB/s 2024-04-25T09:03:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.241 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00868003 s, 121 MB/s 2024-04-25T09:03:25.250 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.250 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.250 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734703 s, 143 MB/s 2024-04-25T09:03:25.261 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.261 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.261 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814745 s, 129 MB/s 2024-04-25T09:03:25.271 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.271 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.271 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0079525 s, 132 MB/s 2024-04-25T09:03:25.281 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.281 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.281 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748209 s, 140 MB/s 2024-04-25T09:03:25.291 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.291 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.291 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770956 s, 136 MB/s 2024-04-25T09:03:25.302 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.302 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.302 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739154 s, 142 MB/s 2024-04-25T09:03:25.310 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.310 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.310 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731541 s, 143 MB/s 2024-04-25T09:03:25.319 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.319 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.319 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693151 s, 151 MB/s 2024-04-25T09:03:25.328 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.328 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.328 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725452 s, 145 MB/s 2024-04-25T09:03:25.337 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.337 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.338 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071384 s, 147 MB/s 2024-04-25T09:03:25.371 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.371 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.371 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778865 s, 135 MB/s 2024-04-25T09:03:25.382 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.382 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.382 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822709 s, 127 MB/s 2024-04-25T09:03:25.392 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.392 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.392 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836041 s, 125 MB/s 2024-04-25T09:03:25.402 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.402 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.402 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749434 s, 140 MB/s 2024-04-25T09:03:25.411 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.411 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.411 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072581 s, 144 MB/s 2024-04-25T09:03:25.421 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.421 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.421 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072952 s, 144 MB/s 2024-04-25T09:03:25.431 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.431 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.431 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825786 s, 127 MB/s 2024-04-25T09:03:25.440 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.440 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.440 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719025 s, 146 MB/s 2024-04-25T09:03:25.450 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.450 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.450 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734906 s, 143 MB/s 2024-04-25T09:03:25.460 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.460 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.461 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839149 s, 125 MB/s 2024-04-25T09:03:25.469 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.470 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.470 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697114 s, 150 MB/s 2024-04-25T09:03:25.478 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.478 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.479 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681513 s, 154 MB/s 2024-04-25T09:03:25.489 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.489 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.489 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00782712 s, 134 MB/s 2024-04-25T09:03:25.498 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.498 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.498 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696263 s, 151 MB/s 2024-04-25T09:03:25.507 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.507 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.507 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724808 s, 145 MB/s 2024-04-25T09:03:25.518 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.518 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.518 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00896201 s, 117 MB/s 2024-04-25T09:03:25.528 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.528 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.528 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759287 s, 138 MB/s 2024-04-25T09:03:25.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:25.529 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:03:25.529 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:03:25.530 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T09:03:25.530 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:03:25.530 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:03:25.530 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:03:25.530 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:03:25.597 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.597 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.598 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742991 s, 141 MB/s 2024-04-25T09:03:25.607 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.607 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.607 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749098 s, 140 MB/s 2024-04-25T09:03:25.616 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.616 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.616 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070508 s, 149 MB/s 2024-04-25T09:03:25.626 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.626 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.626 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733725 s, 143 MB/s 2024-04-25T09:03:25.634 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.635 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.635 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668633 s, 157 MB/s 2024-04-25T09:03:25.645 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.645 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.645 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814599 s, 129 MB/s 2024-04-25T09:03:25.655 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.655 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.655 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074991 s, 140 MB/s 2024-04-25T09:03:25.665 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.665 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.665 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757229 s, 138 MB/s 2024-04-25T09:03:25.673 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.673 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.674 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721988 s, 145 MB/s 2024-04-25T09:03:25.685 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.685 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.685 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00892894 s, 117 MB/s 2024-04-25T09:03:25.695 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.695 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.695 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00852716 s, 123 MB/s 2024-04-25T09:03:25.706 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.706 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.706 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827735 s, 127 MB/s 2024-04-25T09:03:25.716 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.716 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.716 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760819 s, 138 MB/s 2024-04-25T09:03:25.725 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.725 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.725 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726459 s, 144 MB/s 2024-04-25T09:03:25.735 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.735 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.735 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748211 s, 140 MB/s 2024-04-25T09:03:25.744 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.744 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.744 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069775 s, 150 MB/s 2024-04-25T09:03:25.755 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.755 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.755 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846946 s, 124 MB/s 2024-04-25T09:03:25.764 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.764 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.765 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735511 s, 143 MB/s 2024-04-25T09:03:25.775 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.775 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.775 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829077 s, 126 MB/s 2024-04-25T09:03:25.784 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.784 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.784 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725936 s, 144 MB/s 2024-04-25T09:03:25.794 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.794 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.794 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748217 s, 140 MB/s 2024-04-25T09:03:25.804 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.804 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.804 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703827 s, 149 MB/s 2024-04-25T09:03:25.813 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.813 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.813 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721607 s, 145 MB/s 2024-04-25T09:03:25.822 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.822 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.822 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678339 s, 155 MB/s 2024-04-25T09:03:25.831 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.831 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.831 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693063 s, 151 MB/s 2024-04-25T09:03:25.841 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.841 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.841 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751649 s, 140 MB/s 2024-04-25T09:03:25.850 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.851 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.851 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743313 s, 141 MB/s 2024-04-25T09:03:25.860 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.860 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.860 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069864 s, 150 MB/s 2024-04-25T09:03:25.870 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.870 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.870 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765508 s, 137 MB/s 2024-04-25T09:03:25.879 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.879 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.880 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753607 s, 139 MB/s 2024-04-25T09:03:25.890 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.890 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.890 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00871055 s, 120 MB/s 2024-04-25T09:03:25.900 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.900 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.900 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744353 s, 141 MB/s 2024-04-25T09:03:25.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:25.901 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:03:25.901 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:03:25.901 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T09:03:25.901 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:03:25.902 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:03:25.902 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:03:25.902 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:03:25.970 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.970 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.970 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836335 s, 125 MB/s 2024-04-25T09:03:25.980 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.980 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.981 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00813476 s, 129 MB/s 2024-04-25T09:03:25.991 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:25.991 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:25.991 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838496 s, 125 MB/s 2024-04-25T09:03:26.000 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.000 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.000 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700068 s, 150 MB/s 2024-04-25T09:03:26.011 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.011 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.011 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835676 s, 125 MB/s 2024-04-25T09:03:26.021 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.021 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.021 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739886 s, 142 MB/s 2024-04-25T09:03:26.031 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.031 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.031 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828279 s, 127 MB/s 2024-04-25T09:03:26.041 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.042 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.042 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816212 s, 128 MB/s 2024-04-25T09:03:26.052 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.052 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.052 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00859381 s, 122 MB/s 2024-04-25T09:03:26.061 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.061 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.061 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687677 s, 152 MB/s 2024-04-25T09:03:26.073 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.073 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.073 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00891948 s, 118 MB/s 2024-04-25T09:03:26.083 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.083 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.083 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839235 s, 125 MB/s 2024-04-25T09:03:26.093 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.093 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.093 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730793 s, 143 MB/s 2024-04-25T09:03:26.102 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.102 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.102 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737255 s, 142 MB/s 2024-04-25T09:03:26.113 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.114 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.114 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00888512 s, 118 MB/s 2024-04-25T09:03:26.126 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.126 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.126 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729878 s, 144 MB/s 2024-04-25T09:03:26.134 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.134 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.134 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00858573 s, 122 MB/s 2024-04-25T09:03:26.143 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.143 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.143 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697375 s, 150 MB/s 2024-04-25T09:03:26.153 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.153 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.153 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007485 s, 140 MB/s 2024-04-25T09:03:26.163 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.163 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.163 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741573 s, 141 MB/s 2024-04-25T09:03:26.172 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.172 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.172 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718642 s, 146 MB/s 2024-04-25T09:03:26.182 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.182 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.182 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735427 s, 143 MB/s 2024-04-25T09:03:26.193 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.193 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.193 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828592 s, 127 MB/s 2024-04-25T09:03:26.203 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.203 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.203 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826867 s, 127 MB/s 2024-04-25T09:03:26.213 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.213 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.213 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724848 s, 145 MB/s 2024-04-25T09:03:26.222 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.222 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.222 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733611 s, 143 MB/s 2024-04-25T09:03:26.233 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.234 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.234 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00902045 s, 116 MB/s 2024-04-25T09:03:26.243 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.243 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.243 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739145 s, 142 MB/s 2024-04-25T09:03:26.253 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.253 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.253 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765622 s, 137 MB/s 2024-04-25T09:03:26.262 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.263 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.263 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722155 s, 145 MB/s 2024-04-25T09:03:26.272 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.272 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.272 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729476 s, 144 MB/s 2024-04-25T09:03:26.282 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.282 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.282 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749656 s, 140 MB/s 2024-04-25T09:03:26.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:26.283 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:03:26.283 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:03:26.283 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T09:03:26.283 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:03:26.283 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:03:26.283 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:03:26.283 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:03:26.380 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.380 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.380 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723737 s, 145 MB/s 2024-04-25T09:03:26.389 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.389 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.389 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757051 s, 139 MB/s 2024-04-25T09:03:26.399 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.400 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.400 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811128 s, 129 MB/s 2024-04-25T09:03:26.409 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.409 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.409 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681005 s, 154 MB/s 2024-04-25T09:03:26.418 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.418 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.418 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702482 s, 149 MB/s 2024-04-25T09:03:26.429 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.429 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.429 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00849211 s, 123 MB/s 2024-04-25T09:03:26.438 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.438 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.438 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737876 s, 142 MB/s 2024-04-25T09:03:26.448 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.448 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.448 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747429 s, 140 MB/s 2024-04-25T09:03:26.458 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.459 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.459 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00855086 s, 123 MB/s 2024-04-25T09:03:26.469 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.469 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.469 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00790379 s, 133 MB/s 2024-04-25T09:03:26.478 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.478 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.478 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737036 s, 142 MB/s 2024-04-25T09:03:26.489 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.489 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.489 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833564 s, 126 MB/s 2024-04-25T09:03:26.498 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.499 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.499 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072544 s, 145 MB/s 2024-04-25T09:03:26.509 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.509 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.509 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825913 s, 127 MB/s 2024-04-25T09:03:26.518 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.519 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.519 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755105 s, 139 MB/s 2024-04-25T09:03:26.528 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.528 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.528 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690778 s, 152 MB/s 2024-04-25T09:03:26.538 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.538 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.538 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714096 s, 147 MB/s 2024-04-25T09:03:26.546 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.546 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.547 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665462 s, 158 MB/s 2024-04-25T09:03:26.557 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.557 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.558 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00885134 s, 118 MB/s 2024-04-25T09:03:26.568 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.568 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.568 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00848004 s, 124 MB/s 2024-04-25T09:03:26.577 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.577 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.577 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720469 s, 146 MB/s 2024-04-25T09:03:26.586 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.586 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.586 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711506 s, 147 MB/s 2024-04-25T09:03:26.597 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.597 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.597 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00867182 s, 121 MB/s 2024-04-25T09:03:26.607 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.607 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.607 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072745 s, 144 MB/s 2024-04-25T09:03:26.616 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.616 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.616 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072338 s, 145 MB/s 2024-04-25T09:03:26.626 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.627 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.627 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0080342 s, 131 MB/s 2024-04-25T09:03:26.636 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.636 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.636 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742038 s, 141 MB/s 2024-04-25T09:03:26.647 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.647 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.647 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832104 s, 126 MB/s 2024-04-25T09:03:26.656 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.656 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.656 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723979 s, 145 MB/s 2024-04-25T09:03:26.666 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.666 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.666 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773188 s, 136 MB/s 2024-04-25T09:03:26.676 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.676 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.676 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727399 s, 144 MB/s 2024-04-25T09:03:26.685 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.685 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.686 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731488 s, 143 MB/s 2024-04-25T09:03:26.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:26.687 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:03:26.687 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:03:26.687 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T09:03:26.687 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:03:26.687 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:03:26.687 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:03:26.687 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:03:26.714 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.715 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.715 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741171 s, 141 MB/s 2024-04-25T09:03:26.725 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.726 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.726 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833188 s, 126 MB/s 2024-04-25T09:03:26.735 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.735 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.735 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769268 s, 136 MB/s 2024-04-25T09:03:26.744 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.744 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.745 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744286 s, 141 MB/s 2024-04-25T09:03:26.754 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.754 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.754 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763021 s, 137 MB/s 2024-04-25T09:03:26.764 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.764 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.764 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720449 s, 146 MB/s 2024-04-25T09:03:26.774 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.774 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.774 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831247 s, 126 MB/s 2024-04-25T09:03:26.784 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.784 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.785 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815528 s, 129 MB/s 2024-04-25T09:03:26.794 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.794 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.794 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744279 s, 141 MB/s 2024-04-25T09:03:26.805 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.805 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.805 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828746 s, 127 MB/s 2024-04-25T09:03:26.814 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.814 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.814 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727871 s, 144 MB/s 2024-04-25T09:03:26.823 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.823 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.823 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674451 s, 155 MB/s 2024-04-25T09:03:26.834 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.834 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.834 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00847165 s, 124 MB/s 2024-04-25T09:03:26.844 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.844 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.844 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842566 s, 124 MB/s 2024-04-25T09:03:26.855 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.855 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.855 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00854101 s, 123 MB/s 2024-04-25T09:03:26.865 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.865 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.865 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702294 s, 149 MB/s 2024-04-25T09:03:26.874 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.874 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.874 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728166 s, 144 MB/s 2024-04-25T09:03:26.885 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.885 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.885 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00863717 s, 121 MB/s 2024-04-25T09:03:26.895 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.895 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.895 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831716 s, 126 MB/s 2024-04-25T09:03:26.905 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.905 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.905 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727789 s, 144 MB/s 2024-04-25T09:03:26.914 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.914 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.914 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730554 s, 144 MB/s 2024-04-25T09:03:26.924 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.924 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.924 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732725 s, 143 MB/s 2024-04-25T09:03:26.934 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.934 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.934 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743201 s, 141 MB/s 2024-04-25T09:03:26.943 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.943 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.943 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762674 s, 137 MB/s 2024-04-25T09:03:26.953 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.953 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.953 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759782 s, 138 MB/s 2024-04-25T09:03:26.962 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.962 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.962 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709867 s, 148 MB/s 2024-04-25T09:03:26.972 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.972 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.972 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727619 s, 144 MB/s 2024-04-25T09:03:26.982 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.983 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.983 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836986 s, 125 MB/s 2024-04-25T09:03:26.992 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:26.992 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:26.992 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694111 s, 151 MB/s 2024-04-25T09:03:27.002 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.002 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.002 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759074 s, 138 MB/s 2024-04-25T09:03:27.012 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.012 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.012 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830309 s, 126 MB/s 2024-04-25T09:03:27.021 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.021 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.021 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688708 s, 152 MB/s 2024-04-25T09:03:27.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:27.023 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:03:27.023 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:03:27.023 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T09:03:27.023 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:03:27.023 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:03:27.023 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:03:27.023 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:03:27.091 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.091 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.091 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724966 s, 145 MB/s 2024-04-25T09:03:27.100 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.100 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.100 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759074 s, 138 MB/s 2024-04-25T09:03:27.110 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.110 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.110 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00779888 s, 134 MB/s 2024-04-25T09:03:27.120 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.120 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.120 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073016 s, 144 MB/s 2024-04-25T09:03:27.129 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.130 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.130 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730211 s, 144 MB/s 2024-04-25T09:03:27.139 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.139 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.139 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722391 s, 145 MB/s 2024-04-25T09:03:27.150 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.150 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.150 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00874711 s, 120 MB/s 2024-04-25T09:03:27.159 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.159 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.159 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688999 s, 152 MB/s 2024-04-25T09:03:27.169 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.169 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.169 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724434 s, 145 MB/s 2024-04-25T09:03:27.178 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.179 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.179 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749461 s, 140 MB/s 2024-04-25T09:03:27.188 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.188 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.188 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749074 s, 140 MB/s 2024-04-25T09:03:27.197 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.197 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.198 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716161 s, 146 MB/s 2024-04-25T09:03:27.207 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.207 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.207 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765304 s, 137 MB/s 2024-04-25T09:03:27.217 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.217 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.217 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757473 s, 138 MB/s 2024-04-25T09:03:27.226 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.226 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.226 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721035 s, 145 MB/s 2024-04-25T09:03:27.237 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.237 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.237 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833668 s, 126 MB/s 2024-04-25T09:03:27.246 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.247 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.247 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740763 s, 142 MB/s 2024-04-25T09:03:27.257 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.257 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.257 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788484 s, 133 MB/s 2024-04-25T09:03:27.266 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.266 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.266 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723853 s, 145 MB/s 2024-04-25T09:03:27.277 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.277 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.277 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834163 s, 126 MB/s 2024-04-25T09:03:27.286 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.286 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.286 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752898 s, 139 MB/s 2024-04-25T09:03:27.296 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.296 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.296 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730176 s, 144 MB/s 2024-04-25T09:03:27.305 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.305 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.305 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070963 s, 148 MB/s 2024-04-25T09:03:27.316 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.316 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.316 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00847412 s, 124 MB/s 2024-04-25T09:03:27.325 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.326 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.326 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074419 s, 141 MB/s 2024-04-25T09:03:27.335 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.335 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.335 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743408 s, 141 MB/s 2024-04-25T09:03:27.345 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.345 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.345 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758587 s, 138 MB/s 2024-04-25T09:03:27.381 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.381 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.381 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707288 s, 148 MB/s 2024-04-25T09:03:27.390 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.390 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.390 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730014 s, 144 MB/s 2024-04-25T09:03:27.400 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.400 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.400 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833961 s, 126 MB/s 2024-04-25T09:03:27.411 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.411 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.411 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838505 s, 125 MB/s 2024-04-25T09:03:27.421 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.421 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.421 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760102 s, 138 MB/s 2024-04-25T09:03:27.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:27.422 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:03:27.422 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:03:27.422 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T09:03:27.422 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:03:27.422 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:03:27.422 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:03:27.422 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:03:27.490 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.491 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.491 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754405 s, 139 MB/s 2024-04-25T09:03:27.500 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.500 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.500 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007382 s, 142 MB/s 2024-04-25T09:03:27.510 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.510 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.510 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771627 s, 136 MB/s 2024-04-25T09:03:27.520 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.520 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.520 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073996 s, 142 MB/s 2024-04-25T09:03:27.532 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.532 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.532 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00927034 s, 113 MB/s 2024-04-25T09:03:27.543 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.543 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.543 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804561 s, 130 MB/s 2024-04-25T09:03:27.555 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.555 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.555 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00959579 s, 109 MB/s 2024-04-25T09:03:27.567 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.567 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.567 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00978008 s, 107 MB/s 2024-04-25T09:03:27.577 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.577 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.577 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746526 s, 140 MB/s 2024-04-25T09:03:27.589 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.589 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.589 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00895983 s, 117 MB/s 2024-04-25T09:03:27.599 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.600 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.600 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00784518 s, 134 MB/s 2024-04-25T09:03:27.610 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.611 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.611 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804409 s, 130 MB/s 2024-04-25T09:03:27.622 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.622 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.622 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00899652 s, 117 MB/s 2024-04-25T09:03:27.633 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.633 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.633 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832173 s, 126 MB/s 2024-04-25T09:03:27.643 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.644 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.644 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810191 s, 129 MB/s 2024-04-25T09:03:27.655 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.655 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.655 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00907691 s, 116 MB/s 2024-04-25T09:03:27.666 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.666 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.666 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833162 s, 126 MB/s 2024-04-25T09:03:27.676 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.676 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.677 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00784318 s, 134 MB/s 2024-04-25T09:03:27.686 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.686 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.686 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732908 s, 143 MB/s 2024-04-25T09:03:27.696 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.696 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.696 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772497 s, 136 MB/s 2024-04-25T09:03:27.707 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.707 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.707 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836939 s, 125 MB/s 2024-04-25T09:03:27.716 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.716 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.716 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692284 s, 151 MB/s 2024-04-25T09:03:27.726 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.726 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.726 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725436 s, 145 MB/s 2024-04-25T09:03:27.737 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.738 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.738 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00895078 s, 117 MB/s 2024-04-25T09:03:27.747 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.747 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.747 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760852 s, 138 MB/s 2024-04-25T09:03:27.758 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.758 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.758 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00860949 s, 122 MB/s 2024-04-25T09:03:27.769 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.770 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.770 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00921539 s, 114 MB/s 2024-04-25T09:03:27.779 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.779 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.779 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743071 s, 141 MB/s 2024-04-25T09:03:27.788 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.788 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.788 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686247 s, 153 MB/s 2024-04-25T09:03:27.797 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.797 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.797 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702078 s, 149 MB/s 2024-04-25T09:03:27.807 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.807 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.807 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750225 s, 140 MB/s 2024-04-25T09:03:27.819 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.819 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.819 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00924565 s, 113 MB/s 2024-04-25T09:03:27.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:27.820 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:03:27.820 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:03:27.820 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T09:03:27.820 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:03:27.820 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:03:27.820 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:03:27.820 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:03:27.887 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.888 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.888 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693926 s, 151 MB/s 2024-04-25T09:03:27.897 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.897 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.897 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710611 s, 148 MB/s 2024-04-25T09:03:27.907 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.907 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.908 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850969 s, 123 MB/s 2024-04-25T09:03:27.918 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.918 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.918 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815219 s, 129 MB/s 2024-04-25T09:03:27.928 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.929 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.929 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00853195 s, 123 MB/s 2024-04-25T09:03:27.939 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.939 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.939 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832347 s, 126 MB/s 2024-04-25T09:03:27.948 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.948 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.948 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686022 s, 153 MB/s 2024-04-25T09:03:27.958 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.958 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.958 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730981 s, 143 MB/s 2024-04-25T09:03:27.970 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.970 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.970 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693762 s, 151 MB/s 2024-04-25T09:03:27.977 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.977 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.978 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084901 s, 124 MB/s 2024-04-25T09:03:27.987 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.987 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.987 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701564 s, 149 MB/s 2024-04-25T09:03:27.997 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:27.997 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:27.997 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835123 s, 126 MB/s 2024-04-25T09:03:28.008 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.008 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.008 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00852381 s, 123 MB/s 2024-04-25T09:03:28.017 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.017 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.017 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704424 s, 149 MB/s 2024-04-25T09:03:28.028 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.028 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.028 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082218 s, 128 MB/s 2024-04-25T09:03:28.037 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.037 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.037 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722541 s, 145 MB/s 2024-04-25T09:03:28.047 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.047 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.047 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738619 s, 142 MB/s 2024-04-25T09:03:28.056 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.056 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.056 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726307 s, 144 MB/s 2024-04-25T09:03:28.067 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.067 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.067 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830007 s, 126 MB/s 2024-04-25T09:03:28.077 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.077 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.077 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828268 s, 127 MB/s 2024-04-25T09:03:28.088 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.088 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.088 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819064 s, 128 MB/s 2024-04-25T09:03:28.097 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.097 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.097 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730215 s, 144 MB/s 2024-04-25T09:03:28.107 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.107 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.107 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762775 s, 137 MB/s 2024-04-25T09:03:28.116 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.117 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.117 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737552 s, 142 MB/s 2024-04-25T09:03:28.126 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.126 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.126 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692105 s, 152 MB/s 2024-04-25T09:03:28.135 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.135 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.135 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717414 s, 146 MB/s 2024-04-25T09:03:28.146 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.146 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.146 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00852277 s, 123 MB/s 2024-04-25T09:03:28.156 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.156 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.156 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767853 s, 137 MB/s 2024-04-25T09:03:28.166 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.166 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.166 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763649 s, 137 MB/s 2024-04-25T09:03:28.178 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.178 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.178 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839108 s, 125 MB/s 2024-04-25T09:03:28.186 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.186 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.187 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766571 s, 137 MB/s 2024-04-25T09:03:28.196 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.196 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.196 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696135 s, 151 MB/s 2024-04-25T09:03:28.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:28.197 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:03:28.197 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:03:28.197 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T09:03:28.197 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:03:28.197 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:03:28.197 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:03:28.197 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:03:28.266 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.266 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.266 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839496 s, 125 MB/s 2024-04-25T09:03:28.277 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.277 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.277 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774431 s, 135 MB/s 2024-04-25T09:03:28.286 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.286 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.286 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774721 s, 135 MB/s 2024-04-25T09:03:28.296 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.296 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.297 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00844784 s, 124 MB/s 2024-04-25T09:03:28.306 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.306 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.306 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727506 s, 144 MB/s 2024-04-25T09:03:28.315 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.315 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.315 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070321 s, 149 MB/s 2024-04-25T09:03:28.324 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.324 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.324 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679315 s, 154 MB/s 2024-04-25T09:03:28.334 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.334 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.334 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00807948 s, 130 MB/s 2024-04-25T09:03:28.344 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.344 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.344 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695811 s, 151 MB/s 2024-04-25T09:03:28.366 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.366 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.366 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203278 s, 51.6 MB/s 2024-04-25T09:03:28.391 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.391 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.391 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831822 s, 126 MB/s 2024-04-25T09:03:28.402 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.402 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.402 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828855 s, 127 MB/s 2024-04-25T09:03:28.413 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.413 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.413 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00837406 s, 125 MB/s 2024-04-25T09:03:28.422 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.422 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.422 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713281 s, 147 MB/s 2024-04-25T09:03:28.432 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.432 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.432 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733477 s, 143 MB/s 2024-04-25T09:03:28.442 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.442 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.442 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684033 s, 153 MB/s 2024-04-25T09:03:28.452 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.453 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.453 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846195 s, 124 MB/s 2024-04-25T09:03:28.463 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.464 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.464 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827454 s, 127 MB/s 2024-04-25T09:03:28.473 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.473 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.473 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693561 s, 151 MB/s 2024-04-25T09:03:28.483 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.483 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.483 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073119 s, 143 MB/s 2024-04-25T09:03:28.493 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.493 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.493 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691633 s, 152 MB/s 2024-04-25T09:03:28.502 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.502 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.502 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671492 s, 156 MB/s 2024-04-25T09:03:28.513 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.513 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.513 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827235 s, 127 MB/s 2024-04-25T09:03:28.522 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.522 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.522 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697098 s, 150 MB/s 2024-04-25T09:03:28.534 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.534 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.534 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00881968 s, 119 MB/s 2024-04-25T09:03:28.545 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.545 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.545 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00812976 s, 129 MB/s 2024-04-25T09:03:28.554 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.555 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.555 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742381 s, 141 MB/s 2024-04-25T09:03:28.565 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.565 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.565 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802484 s, 131 MB/s 2024-04-25T09:03:28.575 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.575 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.575 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074482 s, 141 MB/s 2024-04-25T09:03:28.586 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.586 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.586 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00837013 s, 125 MB/s 2024-04-25T09:03:28.596 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.596 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.596 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00843667 s, 124 MB/s 2024-04-25T09:03:28.609 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.609 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.609 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00904944 s, 116 MB/s 2024-04-25T09:03:28.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:28.610 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:03:28.610 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:03:28.610 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T09:03:28.610 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:03:28.610 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:03:28.611 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:03:28.611 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:03:28.678 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.678 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.678 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827585 s, 127 MB/s 2024-04-25T09:03:28.688 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.688 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.688 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00781862 s, 134 MB/s 2024-04-25T09:03:28.697 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.697 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.698 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733886 s, 143 MB/s 2024-04-25T09:03:28.708 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.708 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.708 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835879 s, 125 MB/s 2024-04-25T09:03:28.718 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.718 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.718 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00798178 s, 131 MB/s 2024-04-25T09:03:28.727 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.727 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.727 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712936 s, 147 MB/s 2024-04-25T09:03:28.736 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.737 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.737 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690491 s, 152 MB/s 2024-04-25T09:03:28.746 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.746 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.746 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749416 s, 140 MB/s 2024-04-25T09:03:28.756 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.756 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.756 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732718 s, 143 MB/s 2024-04-25T09:03:28.766 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.766 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.766 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007471 s, 140 MB/s 2024-04-25T09:03:28.777 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.777 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.777 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00877171 s, 120 MB/s 2024-04-25T09:03:28.788 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.788 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.788 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840709 s, 125 MB/s 2024-04-25T09:03:28.797 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.797 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.797 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747117 s, 140 MB/s 2024-04-25T09:03:28.807 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.850 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.850 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736458 s, 142 MB/s 2024-04-25T09:03:28.851 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.851 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.851 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737193 s, 142 MB/s 2024-04-25T09:03:28.851 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.851 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.851 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711329 s, 147 MB/s 2024-04-25T09:03:28.852 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.852 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.852 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821562 s, 128 MB/s 2024-04-25T09:03:28.852 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.852 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.852 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836445 s, 125 MB/s 2024-04-25T09:03:28.857 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.857 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.857 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00856652 s, 122 MB/s 2024-04-25T09:03:28.868 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.868 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.868 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814653 s, 129 MB/s 2024-04-25T09:03:28.877 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.877 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.877 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722844 s, 145 MB/s 2024-04-25T09:03:28.887 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.887 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.887 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739877 s, 142 MB/s 2024-04-25T09:03:28.896 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.896 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.896 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730991 s, 143 MB/s 2024-04-25T09:03:28.907 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.907 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.907 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820742 s, 128 MB/s 2024-04-25T09:03:28.917 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.917 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.917 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815564 s, 129 MB/s 2024-04-25T09:03:28.927 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.927 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.927 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687917 s, 152 MB/s 2024-04-25T09:03:28.936 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.936 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.936 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733344 s, 143 MB/s 2024-04-25T09:03:28.946 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.946 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.946 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752625 s, 139 MB/s 2024-04-25T09:03:28.957 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.957 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.957 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00902891 s, 116 MB/s 2024-04-25T09:03:28.966 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.966 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.966 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720242 s, 146 MB/s 2024-04-25T09:03:28.977 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.977 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.977 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0083099 s, 126 MB/s 2024-04-25T09:03:28.987 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:28.987 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:28.987 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00823379 s, 127 MB/s 2024-04-25T09:03:28.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:28.989 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:03:28.989 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:03:28.989 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T09:03:28.989 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:03:28.989 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:03:28.989 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:03:28.989 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:03:29.056 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.056 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.056 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731684 s, 143 MB/s 2024-04-25T09:03:29.065 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.066 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.066 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737909 s, 142 MB/s 2024-04-25T09:03:29.075 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.075 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.075 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007216 s, 145 MB/s 2024-04-25T09:03:29.084 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.084 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.084 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727101 s, 144 MB/s 2024-04-25T09:03:29.095 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.095 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.095 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832682 s, 126 MB/s 2024-04-25T09:03:29.105 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.105 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.105 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756954 s, 139 MB/s 2024-04-25T09:03:29.116 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.116 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.116 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828247 s, 127 MB/s 2024-04-25T09:03:29.125 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.126 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.126 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764235 s, 137 MB/s 2024-04-25T09:03:29.135 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.135 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.135 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702774 s, 149 MB/s 2024-04-25T09:03:29.145 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.145 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.145 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817314 s, 128 MB/s 2024-04-25T09:03:29.155 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.155 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.155 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073185 s, 143 MB/s 2024-04-25T09:03:29.166 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.166 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.166 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00882125 s, 119 MB/s 2024-04-25T09:03:29.177 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.177 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.177 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861849 s, 122 MB/s 2024-04-25T09:03:29.186 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.186 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.186 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728503 s, 144 MB/s 2024-04-25T09:03:29.196 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.196 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.196 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734506 s, 143 MB/s 2024-04-25T09:03:29.205 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.205 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.205 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726186 s, 144 MB/s 2024-04-25T09:03:29.214 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.215 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.215 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724394 s, 145 MB/s 2024-04-25T09:03:29.224 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.224 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.224 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073565 s, 143 MB/s 2024-04-25T09:03:29.235 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.235 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.235 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840083 s, 125 MB/s 2024-04-25T09:03:29.244 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.244 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.244 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705798 s, 149 MB/s 2024-04-25T09:03:29.254 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.254 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.254 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749774 s, 140 MB/s 2024-04-25T09:03:29.264 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.264 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.264 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825512 s, 127 MB/s 2024-04-25T09:03:29.273 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.273 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.273 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716035 s, 146 MB/s 2024-04-25T09:03:29.283 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.283 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.283 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752344 s, 139 MB/s 2024-04-25T09:03:29.293 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.293 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.293 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804986 s, 130 MB/s 2024-04-25T09:03:29.303 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.303 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.303 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071568 s, 147 MB/s 2024-04-25T09:03:29.312 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.312 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.313 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748926 s, 140 MB/s 2024-04-25T09:03:29.321 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.321 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.321 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667635 s, 157 MB/s 2024-04-25T09:03:29.331 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.331 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.331 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073785 s, 142 MB/s 2024-04-25T09:03:29.341 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.341 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.341 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802959 s, 131 MB/s 2024-04-25T09:03:29.352 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.352 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.352 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00864243 s, 121 MB/s 2024-04-25T09:03:29.361 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.361 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.361 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713132 s, 147 MB/s 2024-04-25T09:03:29.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:29.362 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:03:29.363 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:03:29.363 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T09:03:29.363 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:03:29.363 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:03:29.363 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:03:29.363 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:03:29.429 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.430 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.430 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712086 s, 147 MB/s 2024-04-25T09:03:29.439 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.439 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.439 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739396 s, 142 MB/s 2024-04-25T09:03:29.449 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.450 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.450 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821003 s, 128 MB/s 2024-04-25T09:03:29.459 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.459 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.459 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730662 s, 144 MB/s 2024-04-25T09:03:29.468 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.469 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.469 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745025 s, 141 MB/s 2024-04-25T09:03:29.478 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.478 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.478 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715101 s, 147 MB/s 2024-04-25T09:03:29.487 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.487 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.487 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698465 s, 150 MB/s 2024-04-25T09:03:29.497 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.497 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.497 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740317 s, 142 MB/s 2024-04-25T09:03:29.508 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.508 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.508 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836354 s, 125 MB/s 2024-04-25T09:03:29.516 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.517 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.517 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731703 s, 143 MB/s 2024-04-25T09:03:29.526 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.526 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.526 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726316 s, 144 MB/s 2024-04-25T09:03:29.536 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.536 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.536 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819475 s, 128 MB/s 2024-04-25T09:03:29.546 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.546 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.546 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738119 s, 142 MB/s 2024-04-25T09:03:29.555 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.555 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.555 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720641 s, 146 MB/s 2024-04-25T09:03:29.564 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.564 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.564 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0065971 s, 159 MB/s 2024-04-25T09:03:29.573 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.573 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.573 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724283 s, 145 MB/s 2024-04-25T09:03:29.583 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.583 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.584 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00809817 s, 129 MB/s 2024-04-25T09:03:29.593 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.593 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.593 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741244 s, 141 MB/s 2024-04-25T09:03:29.602 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.603 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.603 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725463 s, 145 MB/s 2024-04-25T09:03:29.612 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.612 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.612 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735121 s, 143 MB/s 2024-04-25T09:03:29.622 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.623 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.623 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00859881 s, 122 MB/s 2024-04-25T09:03:29.633 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.633 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.633 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834124 s, 126 MB/s 2024-04-25T09:03:29.642 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.642 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.643 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072058 s, 146 MB/s 2024-04-25T09:03:29.651 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.652 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.652 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681046 s, 154 MB/s 2024-04-25T09:03:29.662 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.662 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.662 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834483 s, 126 MB/s 2024-04-25T09:03:29.673 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.673 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.673 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817075 s, 128 MB/s 2024-04-25T09:03:29.683 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.683 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.683 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00809938 s, 129 MB/s 2024-04-25T09:03:29.692 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.693 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.693 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729044 s, 144 MB/s 2024-04-25T09:03:29.703 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.703 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.703 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00860477 s, 122 MB/s 2024-04-25T09:03:29.713 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.713 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.713 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731997 s, 143 MB/s 2024-04-25T09:03:29.722 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.722 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.722 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725446 s, 145 MB/s 2024-04-25T09:03:29.731 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.732 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.732 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731352 s, 143 MB/s 2024-04-25T09:03:29.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:29.733 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:03:29.733 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:03:29.733 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T09:03:29.733 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:03:29.733 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:03:29.733 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:03:29.733 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:03:29.802 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.802 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.802 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814159 s, 129 MB/s 2024-04-25T09:03:29.814 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.814 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.814 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727059 s, 144 MB/s 2024-04-25T09:03:29.823 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.823 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.823 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00916217 s, 114 MB/s 2024-04-25T09:03:29.833 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.833 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.833 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819105 s, 128 MB/s 2024-04-25T09:03:29.842 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.842 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.843 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718895 s, 146 MB/s 2024-04-25T09:03:29.851 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.852 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.852 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707955 s, 148 MB/s 2024-04-25T09:03:29.861 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.861 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.861 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736156 s, 142 MB/s 2024-04-25T09:03:29.871 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.871 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.871 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071917 s, 146 MB/s 2024-04-25T09:03:29.881 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.881 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.881 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840977 s, 125 MB/s 2024-04-25T09:03:29.891 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.891 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.891 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727084 s, 144 MB/s 2024-04-25T09:03:29.900 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.900 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.900 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728126 s, 144 MB/s 2024-04-25T09:03:29.910 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.910 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.910 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756677 s, 139 MB/s 2024-04-25T09:03:29.919 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.919 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.920 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726727 s, 144 MB/s 2024-04-25T09:03:29.928 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.928 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.928 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665551 s, 158 MB/s 2024-04-25T09:03:29.937 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.938 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.938 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698902 s, 150 MB/s 2024-04-25T09:03:29.947 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.947 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.947 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748044 s, 140 MB/s 2024-04-25T09:03:29.956 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.957 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.957 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688391 s, 152 MB/s 2024-04-25T09:03:29.966 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.966 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.966 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735272 s, 143 MB/s 2024-04-25T09:03:29.976 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.976 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.976 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841784 s, 125 MB/s 2024-04-25T09:03:29.987 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.987 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.987 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845924 s, 124 MB/s 2024-04-25T09:03:29.996 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:29.996 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:29.996 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707047 s, 148 MB/s 2024-04-25T09:03:30.007 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.007 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.007 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835226 s, 126 MB/s 2024-04-25T09:03:30.016 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.016 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.016 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702186 s, 149 MB/s 2024-04-25T09:03:30.027 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.027 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.027 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816964 s, 128 MB/s 2024-04-25T09:03:30.036 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.036 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.036 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685383 s, 153 MB/s 2024-04-25T09:03:30.045 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.045 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.045 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682325 s, 154 MB/s 2024-04-25T09:03:30.056 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.056 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.056 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861363 s, 122 MB/s 2024-04-25T09:03:30.065 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.065 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.065 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731574 s, 143 MB/s 2024-04-25T09:03:30.074 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.075 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.075 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714963 s, 147 MB/s 2024-04-25T09:03:30.084 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.084 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.084 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752385 s, 139 MB/s 2024-04-25T09:03:30.094 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.095 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.095 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082626 s, 127 MB/s 2024-04-25T09:03:30.104 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.104 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.104 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744664 s, 141 MB/s 2024-04-25T09:03:30.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:30.106 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:03:30.106 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:03:30.106 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T09:03:30.106 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:03:30.106 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:03:30.106 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:03:30.106 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:03:30.173 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.173 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.173 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742387 s, 141 MB/s 2024-04-25T09:03:30.182 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.183 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.183 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752208 s, 139 MB/s 2024-04-25T09:03:30.192 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.193 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.193 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747382 s, 140 MB/s 2024-04-25T09:03:30.202 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.202 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.202 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736391 s, 142 MB/s 2024-04-25T09:03:30.212 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.212 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.213 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0081254 s, 129 MB/s 2024-04-25T09:03:30.223 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.226 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.226 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00809612 s, 130 MB/s 2024-04-25T09:03:30.232 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.233 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.233 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755245 s, 139 MB/s 2024-04-25T09:03:30.242 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.242 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.242 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756055 s, 139 MB/s 2024-04-25T09:03:30.252 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.252 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.252 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722681 s, 145 MB/s 2024-04-25T09:03:30.263 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.263 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.263 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00847051 s, 124 MB/s 2024-04-25T09:03:30.274 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.274 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.274 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0088283 s, 119 MB/s 2024-04-25T09:03:30.283 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.283 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.283 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072287 s, 145 MB/s 2024-04-25T09:03:30.292 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.292 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.292 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680648 s, 154 MB/s 2024-04-25T09:03:30.303 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.303 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.303 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839448 s, 125 MB/s 2024-04-25T09:03:30.312 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.312 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.312 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684799 s, 153 MB/s 2024-04-25T09:03:30.322 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.322 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.322 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740919 s, 142 MB/s 2024-04-25T09:03:30.331 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.331 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.331 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713888 s, 147 MB/s 2024-04-25T09:03:30.340 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.341 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.341 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743546 s, 141 MB/s 2024-04-25T09:03:30.350 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.350 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.350 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695972 s, 151 MB/s 2024-04-25T09:03:30.359 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.359 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.359 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702439 s, 149 MB/s 2024-04-25T09:03:30.370 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.370 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.370 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00872257 s, 120 MB/s 2024-04-25T09:03:30.404 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.404 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.404 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758194 s, 138 MB/s 2024-04-25T09:03:30.413 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.413 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.413 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725844 s, 144 MB/s 2024-04-25T09:03:30.422 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.422 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.422 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705452 s, 149 MB/s 2024-04-25T09:03:30.431 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.431 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.431 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694501 s, 151 MB/s 2024-04-25T09:03:30.441 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.441 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.442 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827964 s, 127 MB/s 2024-04-25T09:03:30.452 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.452 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.452 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802576 s, 131 MB/s 2024-04-25T09:03:30.462 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.462 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.462 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804232 s, 130 MB/s 2024-04-25T09:03:30.471 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.471 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.471 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703664 s, 149 MB/s 2024-04-25T09:03:30.481 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.481 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.481 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833424 s, 126 MB/s 2024-04-25T09:03:30.492 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.492 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.492 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829404 s, 126 MB/s 2024-04-25T09:03:30.503 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.503 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.503 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00860521 s, 122 MB/s 2024-04-25T09:03:30.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:30.504 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:03:30.504 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:03:30.504 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T09:03:30.504 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:03:30.504 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:03:30.504 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:03:30.504 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:03:30.572 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.572 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.572 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688836 s, 152 MB/s 2024-04-25T09:03:30.581 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.581 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.581 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735275 s, 143 MB/s 2024-04-25T09:03:30.591 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.591 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.591 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071945 s, 146 MB/s 2024-04-25T09:03:30.601 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.602 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.602 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842055 s, 125 MB/s 2024-04-25T09:03:30.612 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.612 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.612 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845351 s, 124 MB/s 2024-04-25T09:03:30.622 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.622 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.622 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689574 s, 152 MB/s 2024-04-25T09:03:30.631 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.631 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.631 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686215 s, 153 MB/s 2024-04-25T09:03:30.640 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.640 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.640 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693391 s, 151 MB/s 2024-04-25T09:03:30.650 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.650 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.650 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073461 s, 143 MB/s 2024-04-25T09:03:30.661 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.661 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.661 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00856501 s, 122 MB/s 2024-04-25T09:03:30.670 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.670 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.670 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735948 s, 142 MB/s 2024-04-25T09:03:30.679 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.680 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.680 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708838 s, 148 MB/s 2024-04-25T09:03:30.690 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.690 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.690 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0080695 s, 130 MB/s 2024-04-25T09:03:30.700 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.700 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.700 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00789565 s, 133 MB/s 2024-04-25T09:03:30.709 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.709 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.709 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0067081 s, 156 MB/s 2024-04-25T09:03:30.719 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.719 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.719 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00808519 s, 130 MB/s 2024-04-25T09:03:30.730 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.730 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.730 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00801502 s, 131 MB/s 2024-04-25T09:03:30.740 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.740 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.740 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818909 s, 128 MB/s 2024-04-25T09:03:30.749 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.750 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.750 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071507 s, 147 MB/s 2024-04-25T09:03:30.759 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.759 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.759 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707365 s, 148 MB/s 2024-04-25T09:03:30.768 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.768 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.768 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713301 s, 147 MB/s 2024-04-25T09:03:30.779 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.779 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.779 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842339 s, 124 MB/s 2024-04-25T09:03:30.789 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.789 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.789 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00795336 s, 132 MB/s 2024-04-25T09:03:30.799 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.799 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.799 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792625 s, 132 MB/s 2024-04-25T09:03:30.809 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.809 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.809 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738301 s, 142 MB/s 2024-04-25T09:03:30.819 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.819 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.819 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829785 s, 126 MB/s 2024-04-25T09:03:30.830 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.830 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.830 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00807909 s, 130 MB/s 2024-04-25T09:03:30.841 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.841 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.841 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00891331 s, 118 MB/s 2024-04-25T09:03:30.850 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.851 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.851 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744763 s, 141 MB/s 2024-04-25T09:03:30.860 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.860 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.860 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725895 s, 144 MB/s 2024-04-25T09:03:30.869 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.869 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.869 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696474 s, 151 MB/s 2024-04-25T09:03:30.880 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.880 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.880 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827231 s, 127 MB/s 2024-04-25T09:03:30.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:30.881 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:03:30.881 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:03:30.881 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T09:03:30.881 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:03:30.881 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:03:30.881 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:03:30.881 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:03:30.949 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.949 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.949 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701958 s, 149 MB/s 2024-04-25T09:03:30.959 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.959 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.959 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745411 s, 141 MB/s 2024-04-25T09:03:30.968 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.968 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.968 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706103 s, 149 MB/s 2024-04-25T09:03:30.978 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.978 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.978 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00806301 s, 130 MB/s 2024-04-25T09:03:30.987 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.987 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.987 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667233 s, 157 MB/s 2024-04-25T09:03:30.996 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:30.997 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:30.997 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699481 s, 150 MB/s 2024-04-25T09:03:31.006 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.006 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.006 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713389 s, 147 MB/s 2024-04-25T09:03:31.015 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.015 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.015 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700007 s, 150 MB/s 2024-04-25T09:03:31.026 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.026 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.026 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831393 s, 126 MB/s 2024-04-25T09:03:31.036 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.036 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.036 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819809 s, 128 MB/s 2024-04-25T09:03:31.047 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.047 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.047 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840861 s, 125 MB/s 2024-04-25T09:03:31.057 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.058 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.058 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811438 s, 129 MB/s 2024-04-25T09:03:31.067 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.067 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.067 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766195 s, 137 MB/s 2024-04-25T09:03:31.078 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.078 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.078 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0081105 s, 129 MB/s 2024-04-25T09:03:31.087 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.088 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.088 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744139 s, 141 MB/s 2024-04-25T09:03:31.097 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.097 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.097 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696508 s, 151 MB/s 2024-04-25T09:03:31.106 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.106 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.106 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752269 s, 139 MB/s 2024-04-25T09:03:31.116 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.116 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.116 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699425 s, 150 MB/s 2024-04-25T09:03:31.125 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.125 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.125 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722379 s, 145 MB/s 2024-04-25T09:03:31.136 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.136 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.136 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00809562 s, 130 MB/s 2024-04-25T09:03:31.145 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.145 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.145 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710422 s, 148 MB/s 2024-04-25T09:03:31.155 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.156 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.156 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834714 s, 126 MB/s 2024-04-25T09:03:31.165 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.165 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.165 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729514 s, 144 MB/s 2024-04-25T09:03:31.174 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.175 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.175 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073126 s, 143 MB/s 2024-04-25T09:03:31.184 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.184 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.184 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719643 s, 146 MB/s 2024-04-25T09:03:31.194 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.194 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.194 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825589 s, 127 MB/s 2024-04-25T09:03:31.204 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.204 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.204 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722725 s, 145 MB/s 2024-04-25T09:03:31.213 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.213 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.213 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668565 s, 157 MB/s 2024-04-25T09:03:31.222 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.222 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.222 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072422 s, 145 MB/s 2024-04-25T09:03:31.233 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.233 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.233 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840988 s, 125 MB/s 2024-04-25T09:03:31.242 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.242 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.242 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719147 s, 146 MB/s 2024-04-25T09:03:31.253 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.253 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.253 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0081949 s, 128 MB/s 2024-04-25T09:03:31.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:31.254 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:03:31.254 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:03:31.254 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T09:03:31.254 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:03:31.255 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:03:31.255 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:03:31.255 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:03:31.323 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.323 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.323 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00849389 s, 123 MB/s 2024-04-25T09:03:31.332 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.332 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.332 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00687436 s, 153 MB/s 2024-04-25T09:03:31.341 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.341 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.341 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678414 s, 155 MB/s 2024-04-25T09:03:31.350 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.350 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.350 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00671147 s, 156 MB/s 2024-04-25T09:03:31.360 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.360 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.361 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771759 s, 136 MB/s 2024-04-25T09:03:31.373 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.373 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.373 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0103522 s, 101 MB/s 2024-04-25T09:03:31.407 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.407 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.407 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797238 s, 132 MB/s 2024-04-25T09:03:31.416 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.416 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.416 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665775 s, 157 MB/s 2024-04-25T09:03:31.425 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.425 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.425 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072758 s, 144 MB/s 2024-04-25T09:03:31.434 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.435 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.435 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711807 s, 147 MB/s 2024-04-25T09:03:31.444 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.444 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.444 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740994 s, 142 MB/s 2024-04-25T09:03:31.454 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.454 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.454 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736042 s, 142 MB/s 2024-04-25T09:03:31.463 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.463 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.463 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732142 s, 143 MB/s 2024-04-25T09:03:31.472 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.473 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.473 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730959 s, 143 MB/s 2024-04-25T09:03:31.483 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.483 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.483 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829654 s, 126 MB/s 2024-04-25T09:03:31.492 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.493 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.493 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071373 s, 147 MB/s 2024-04-25T09:03:31.503 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.503 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.503 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818703 s, 128 MB/s 2024-04-25T09:03:31.513 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.513 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.513 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741678 s, 141 MB/s 2024-04-25T09:03:31.522 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.522 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.522 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688978 s, 152 MB/s 2024-04-25T09:03:31.531 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.531 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.531 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742166 s, 141 MB/s 2024-04-25T09:03:31.540 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.541 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.541 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714319 s, 147 MB/s 2024-04-25T09:03:31.550 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.550 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.550 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00674857 s, 155 MB/s 2024-04-25T09:03:31.559 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.559 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.559 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682157 s, 154 MB/s 2024-04-25T09:03:31.569 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.569 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.569 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824992 s, 127 MB/s 2024-04-25T09:03:31.578 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.579 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.579 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701096 s, 150 MB/s 2024-04-25T09:03:31.589 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.590 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.590 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00860747 s, 122 MB/s 2024-04-25T09:03:31.601 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.601 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.601 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851021 s, 123 MB/s 2024-04-25T09:03:31.611 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.611 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.611 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765973 s, 137 MB/s 2024-04-25T09:03:31.621 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.621 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.621 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835072 s, 126 MB/s 2024-04-25T09:03:31.631 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.631 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.631 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741889 s, 141 MB/s 2024-04-25T09:03:31.641 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.641 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.641 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742213 s, 141 MB/s 2024-04-25T09:03:31.650 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:03:31.650 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:03:31.650 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724221 s, 145 MB/s 2024-04-25T09:03:31.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:31.652 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub start /scrub_control_test_path recursive 2024-04-25T09:03:31.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.809+0000 7f46983f4640 1 Processor -- start 2024-04-25T09:03:31.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.809+0000 7f46983f4640 1 -- start start 2024-04-25T09:03:31.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.810+0000 7f46983f4640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f46900b4300 0x7f46900cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.810+0000 7f46983f4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f46900cf2e0 0x7f46900cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.810+0000 7f46983f4640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f469015ae50 con 0x7f46900aeea0 2024-04-25T09:03:31.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.810+0000 7f46983f4640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46900b2710 con 0x7f46900cf2e0 2024-04-25T09:03:31.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.810+0000 7f46983f4640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f469006dfe0 con 0x7f46900b4300 2024-04-25T09:03:31.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.810+0000 7f4696991640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f46900cf2e0 0x7f46900cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.810+0000 7f4696190640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f46900aeea0 0x7f469015ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:37418/0 (socket says 172.21.15.43:37418) 2024-04-25T09:03:31.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.810+0000 7f4696991640 1 --2- 172.21.15.43:0/129595637 >> v2:172.21.15.43:3300/0 conn(0x7f46900cf2e0 0x7f46900cf770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50924/0 (socket says 172.21.15.43:50924) 2024-04-25T09:03:31.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.810+0000 7f4696190640 1 -- 172.21.15.43:0/129595637 learned_addr learned my addr 172.21.15.43:0/129595637 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:03:31.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.810+0000 7f469598f640 1 --2- 172.21.15.43:0/129595637 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f46900b4300 0x7f46900cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.810+0000 7f4696991640 1 -- 172.21.15.43:0/129595637 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f46900b4300 msgr2=0x7f46900cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:31.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.810+0000 7f4696991640 1 --2- 172.21.15.43:0/129595637 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f46900b4300 0x7f46900cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.810+0000 7f4696991640 1 -- 172.21.15.43:0/129595637 >> v1:172.21.15.43:6789/0 conn(0x7f46900aeea0 legacy=0x7f469015ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:31.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.810+0000 7f469518e640 1 -- 172.21.15.43:0/129595637 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2323541158 0 0) 0x7f469015ae50 con 0x7f46900aeea0 2024-04-25T09:03:31.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.810+0000 7f4696991640 1 -- 172.21.15.43:0/129595637 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46900b4850 con 0x7f46900cf2e0 2024-04-25T09:03:31.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.811+0000 7f4696991640 1 --2- 172.21.15.43:0/129595637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46900cf2e0 0x7f46900cf770 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f468800bfa0 tx=0x7f468802c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d004c6c8931d7552 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:31.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.811+0000 7f469518e640 1 -- 172.21.15.43:0/129595637 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4688038070 con 0x7f46900cf2e0 2024-04-25T09:03:31.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.811+0000 7f469518e640 1 -- 172.21.15.43:0/129595637 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4688009540 con 0x7f46900cf2e0 2024-04-25T09:03:31.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.811+0000 7f469518e640 1 -- 172.21.15.43:0/129595637 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4688009910 con 0x7f46900cf2e0 2024-04-25T09:03:31.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.812+0000 7f46983f4640 1 -- 172.21.15.43:0/129595637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46900cf2e0 msgr2=0x7f46900cf770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:31.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.812+0000 7f46983f4640 1 --2- 172.21.15.43:0/129595637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46900cf2e0 0x7f46900cf770 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f468800bfa0 tx=0x7f468802c7e0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.812+0000 7f46983f4640 1 -- 172.21.15.43:0/129595637 shutdown_connections 2024-04-25T09:03:31.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.812+0000 7f46983f4640 1 --2- 172.21.15.43:0/129595637 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46900cf2e0 0x7f46900cf770 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.812+0000 7f46983f4640 1 --2- 172.21.15.43:0/129595637 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f46900b4300 0x7f46900cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.812+0000 7f46983f4640 1 -- 172.21.15.43:0/129595637 >> 172.21.15.43:0/129595637 conn(0x7f469006ab30 msgr2=0x7f469006af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:31.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.812+0000 7f46983f4640 1 -- 172.21.15.43:0/129595637 shutdown_connections 2024-04-25T09:03:31.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.812+0000 7f46983f4640 1 -- 172.21.15.43:0/129595637 wait complete. 2024-04-25T09:03:31.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.813+0000 7f46983f4640 1 Processor -- start 2024-04-25T09:03:31.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.813+0000 7f46983f4640 1 -- start start 2024-04-25T09:03:31.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.814+0000 7f46983f4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f46900aeea0 0x7f46900adb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.814+0000 7f46983f4640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f46900b4300 0x7f4690148c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.814+0000 7f46983f4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46900cf2e0 0x7f46901491c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.814+0000 7f46983f4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46900cdaa0 con 0x7f46900cf2e0 2024-04-25T09:03:31.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.814+0000 7f46983f4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46900b2c40 con 0x7f46900aeea0 2024-04-25T09:03:31.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.814+0000 7f46983f4640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46900cdc20 con 0x7f46900b4300 2024-04-25T09:03:31.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.814+0000 7f4696991640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46900cf2e0 0x7f46901491c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.814+0000 7f4696991640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46900cf2e0 0x7f46901491c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50940/0 (socket says 172.21.15.43:50940) 2024-04-25T09:03:31.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.814+0000 7f4696991640 1 -- 172.21.15.43:0/1013571607 learned_addr learned my addr 172.21.15.43:0/1013571607 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:31.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.814+0000 7f469598f640 1 --2- 172.21.15.43:0/1013571607 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f46900b4300 0x7f4690148c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.814+0000 7f4696190640 1 --2- 172.21.15.43:0/1013571607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f46900aeea0 0x7f46900adb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.814+0000 7f4696991640 1 -- 172.21.15.43:0/1013571607 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f46900b4300 msgr2=0x7f4690148c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:31.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.814+0000 7f4696991640 1 --2- 172.21.15.43:0/1013571607 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f46900b4300 0x7f4690148c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.814+0000 7f4696991640 1 -- 172.21.15.43:0/1013571607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f46900aeea0 msgr2=0x7f46900adb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:31.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.814+0000 7f4696991640 1 --2- 172.21.15.43:0/1013571607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f46900aeea0 0x7f46900adb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.814+0000 7f4696991640 1 -- 172.21.15.43:0/1013571607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46900b04a0 con 0x7f46900cf2e0 2024-04-25T09:03:31.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.814+0000 7f469598f640 1 --2- 172.21.15.43:0/1013571607 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f46900b4300 0x7f4690148c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:31.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.814+0000 7f4696190640 1 --2- 172.21.15.43:0/1013571607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f46900aeea0 0x7f46900adb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:31.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.814+0000 7f4696991640 1 --2- 172.21.15.43:0/1013571607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46900cf2e0 0x7f46901491c0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f4688036040 tx=0x7f4688004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:31.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.814+0000 7f4686ffd640 1 -- 172.21.15.43:0/1013571607 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4688038040 con 0x7f46900cf2e0 2024-04-25T09:03:31.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.814+0000 7f4686ffd640 1 -- 172.21.15.43:0/1013571607 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4688043070 con 0x7f46900cf2e0 2024-04-25T09:03:31.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.814+0000 7f46983f4640 1 -- 172.21.15.43:0/1013571607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46900b0740 con 0x7f46900cf2e0 2024-04-25T09:03:31.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.814+0000 7f4686ffd640 1 -- 172.21.15.43:0/1013571607 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468803e880 con 0x7f46900cf2e0 2024-04-25T09:03:31.804 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.814+0000 7f46983f4640 1 -- 172.21.15.43:0/1013571607 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f469015ebd0 con 0x7f46900cf2e0 2024-04-25T09:03:31.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.816+0000 7f4686ffd640 1 -- 172.21.15.43:0/1013571607 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4688009320 con 0x7f46900cf2e0 2024-04-25T09:03:31.805 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.817+0000 7f4686ffd640 1 --2- 172.21.15.43:0/1013571607 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f467406b4d0 0x7f467406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.817+0000 7f4686ffd640 1 -- 172.21.15.43:0/1013571607 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f46880ab3e0 con 0x7f46900cf2e0 2024-04-25T09:03:31.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.817+0000 7f4696190640 1 --2- 172.21.15.43:0/1013571607 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f467406b4d0 0x7f467406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.817+0000 7f4696190640 1 --2- 172.21.15.43:0/1013571607 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f467406b4d0 0x7f467406d990 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f468c000f60 tx=0x7f468c007c10 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:31.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.818+0000 7f469b1cc740 1 Processor -- start 2024-04-25T09:03:31.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.818+0000 7f469b1cc740 1 -- start start 2024-04-25T09:03:31.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.819+0000 7f469b1cc740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x555e40c50200 0x555e40c52360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.819+0000 7f469b1cc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e40c8c8d0 0x555e40c505e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.819+0000 7f469b1cc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e40c7acc0 0x555e40c7b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.819+0000 7f469b1cc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e40bb4000 con 0x555e40c8c8d0 2024-04-25T09:03:31.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.819+0000 7f469b1cc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e40b5cf20 con 0x555e40c7acc0 2024-04-25T09:03:31.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.819+0000 7f469b1cc740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e40bb00a0 con 0x555e40c50200 2024-04-25T09:03:31.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.819+0000 7f469598f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e40c8c8d0 0x555e40c505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.819+0000 7f469598f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e40c8c8d0 0x555e40c505e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50942/0 (socket says 172.21.15.43:50942) 2024-04-25T09:03:31.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.819+0000 7f469598f640 1 -- 172.21.15.43:0/2675278245 learned_addr learned my addr 172.21.15.43:0/2675278245 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:31.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.819+0000 7f4696991640 1 --2- 172.21.15.43:0/2675278245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e40c7acc0 0x555e40c7b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.819+0000 7f4696190640 1 --2- 172.21.15.43:0/2675278245 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x555e40c50200 0x555e40c52360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.819+0000 7f469598f640 1 -- 172.21.15.43:0/2675278245 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x555e40c50200 msgr2=0x555e40c52360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:31.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.819+0000 7f469598f640 1 --2- 172.21.15.43:0/2675278245 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x555e40c50200 0x555e40c52360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.819+0000 7f469598f640 1 -- 172.21.15.43:0/2675278245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e40c7acc0 msgr2=0x555e40c7b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:31.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.819+0000 7f469598f640 1 --2- 172.21.15.43:0/2675278245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e40c7acc0 0x555e40c7b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.819+0000 7f469598f640 1 -- 172.21.15.43:0/2675278245 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e40994ba0 con 0x555e40c8c8d0 2024-04-25T09:03:31.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.819+0000 7f469598f640 1 --2- 172.21.15.43:0/2675278245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e40c8c8d0 0x555e40c505e0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f4680007940 tx=0x7f4680007d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ed5e1aefc66077 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:31.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.819+0000 7f4673fff640 1 -- 172.21.15.43:0/2675278245 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4680004230 con 0x555e40c8c8d0 2024-04-25T09:03:31.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.819+0000 7f4673fff640 1 -- 172.21.15.43:0/2675278245 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46800043d0 con 0x555e40c8c8d0 2024-04-25T09:03:31.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.819+0000 7f4673fff640 1 -- 172.21.15.43:0/2675278245 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468000f9a0 con 0x555e40c8c8d0 2024-04-25T09:03:31.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.820+0000 7f469b1cc740 1 -- 172.21.15.43:0/2675278245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e40c8c8d0 msgr2=0x555e40c505e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:31.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.820+0000 7f469b1cc740 1 --2- 172.21.15.43:0/2675278245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e40c8c8d0 0x555e40c505e0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f4680007940 tx=0x7f4680007d40 comp rx=0 tx=0).stop 2024-04-25T09:03:31.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.820+0000 7f469b1cc740 1 -- 172.21.15.43:0/2675278245 shutdown_connections 2024-04-25T09:03:31.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.820+0000 7f469b1cc740 1 --2- 172.21.15.43:0/2675278245 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e40c7acc0 0x555e40c7b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.820+0000 7f469b1cc740 1 --2- 172.21.15.43:0/2675278245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e40c8c8d0 0x555e40c505e0 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.820+0000 7f469b1cc740 1 --2- 172.21.15.43:0/2675278245 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x555e40c50200 0x555e40c52360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.820+0000 7f469b1cc740 1 -- 172.21.15.43:0/2675278245 >> 172.21.15.43:0/2675278245 conn(0x555e40c18050 msgr2=0x555e40c95cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:31.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.820+0000 7f469b1cc740 1 -- 172.21.15.43:0/2675278245 shutdown_connections 2024-04-25T09:03:31.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.820+0000 7f469b1cc740 1 -- 172.21.15.43:0/2675278245 wait complete. 2024-04-25T09:03:31.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.821+0000 7f469b1cc740 1 -- start start 2024-04-25T09:03:31.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.821+0000 7f469b1cc740 1 Processor -- start 2024-04-25T09:03:31.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.822+0000 7f469b1cc740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x555e40c50200 0x555e40c90ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.822+0000 7f469b1cc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e40c7acc0 0x555e40ca5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.814 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.822+0000 7f469b1cc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e40c8c8d0 0x555e40c7d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.822+0000 7f469b1cc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e40b52610 con 0x555e40c8c8d0 2024-04-25T09:03:31.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.822+0000 7f469b1cc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e40c78810 con 0x555e40c7acc0 2024-04-25T09:03:31.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.822+0000 7f469b1cc740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e40c7ceb0 con 0x555e40c50200 2024-04-25T09:03:31.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.822+0000 7f469b1cc740 10 client.0 _ll_register_callbacks cb 0x555e40b6af20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:31.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.822+0000 7f4696991640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e40c8c8d0 0x555e40c7d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.822+0000 7f4696991640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e40c8c8d0 0x555e40c7d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50950/0 (socket says 172.21.15.43:50950) 2024-04-25T09:03:31.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.822+0000 7f4696991640 1 -- 172.21.15.43:0/2019726004 learned_addr learned my addr 172.21.15.43:0/2019726004 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:31.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.822+0000 7f4696190640 1 --2- 172.21.15.43:0/2019726004 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x555e40c50200 0x555e40c90ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.822+0000 7f469598f640 1 --2- 172.21.15.43:0/2019726004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e40c7acc0 0x555e40ca5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.822+0000 7f4696991640 1 -- 172.21.15.43:0/2019726004 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x555e40c50200 msgr2=0x555e40c90ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:31.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.822+0000 7f4696991640 1 --2- 172.21.15.43:0/2019726004 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x555e40c50200 0x555e40c90ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.815 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.822+0000 7f4696991640 1 -- 172.21.15.43:0/2019726004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e40c7acc0 msgr2=0x555e40ca5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:31.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.822+0000 7f4696991640 1 --2- 172.21.15.43:0/2019726004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e40c7acc0 0x555e40ca5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.822+0000 7f4696991640 1 -- 172.21.15.43:0/2019726004 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e40888490 con 0x555e40c8c8d0 2024-04-25T09:03:31.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.822+0000 7f4696991640 1 --2- 172.21.15.43:0/2019726004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e40c8c8d0 0x555e40c7d990 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f46880798e0 tx=0x7f4688079be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=562c5d7e1cabdf69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:31.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.822+0000 7f46717fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:03:31.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.822+0000 7f46717fa640 1 -- 172.21.15.43:0/2019726004 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4688091070 con 0x555e40c8c8d0 2024-04-25T09:03:31.816 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.823+0000 7f46717fa640 1 -- 172.21.15.43:0/2019726004 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f46880ab070 con 0x555e40c8c8d0 2024-04-25T09:03:31.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.823+0000 7f469b1cc740 1 -- 172.21.15.43:0/2019726004 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f46800043d0 con 0x555e40c8c8d0 2024-04-25T09:03:31.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.823+0000 7f46717fa640 1 -- 172.21.15.43:0/2019726004 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f46880822f0 con 0x555e40c8c8d0 2024-04-25T09:03:31.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.823+0000 7f469b1cc740 10 client.9426 fetch_fsmap learned FSMap version 76 2024-04-25T09:03:31.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.823+0000 7f469b1cc740 1 -- 172.21.15.43:0/2019726004 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x555e409f5330 con 0x555e40c8c8d0 2024-04-25T09:03:31.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.823+0000 7f46717fa640 1 -- 172.21.15.43:0/2019726004 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46800043d0 con 0x555e40c8c8d0 2024-04-25T09:03:31.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.823+0000 7f46717fa640 1 -- 172.21.15.43:0/2019726004 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 76) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f468803b030 con 0x555e40c8c8d0 2024-04-25T09:03:31.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.823+0000 7f469b1cc740 10 client.9426 fetch_fsmap finished waiting for FSMap version 76 2024-04-25T09:03:31.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.823+0000 7f469b1cc740 10 client.9426 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.a 2024-04-25T09:03:31.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.823+0000 7f469b1cc740 20 client.9426 populate_metadata read hostname 'smithi043' 2024-04-25T09:03:31.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.823+0000 7f469b1cc740 1 --2- 172.21.15.43:0/2019726004 >> [v2:172.21.15.43:6832/2603892772,v1:172.21.15.43:6834/2603892772] conn(0x555e40c7e0d0 0x555e40c8e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.823+0000 7f469b1cc740 4 client.9426 mds_command: new command op to 9237 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:03:31.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.823+0000 7f469b1cc740 1 -- 172.21.15.43:0/2019726004 --> [v2:172.21.15.43:6832/2603892772,v1:172.21.15.43:6834/2603892772] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e40bc1540 con 0x555e40c7e0d0 2024-04-25T09:03:31.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.823+0000 7f4696190640 1 --2- 172.21.15.43:0/2019726004 >> [v2:172.21.15.43:6832/2603892772,v1:172.21.15.43:6834/2603892772] conn(0x555e40c7e0d0 0x555e40c8e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.823+0000 7f46717fa640 10 client.9426 ms_handle_connect on v2:172.21.15.43:6832/2603892772 2024-04-25T09:03:31.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.823+0000 7f4696190640 1 --2- 172.21.15.43:0/2019726004 >> [v2:172.21.15.43:6832/2603892772,v1:172.21.15.43:6834/2603892772] conn(0x555e40c7e0d0 0x555e40c8e500 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=feba8d199824eef1 server_cookie=6f3183b17ab5a01c in_seq=0 out_seq=0 2024-04-25T09:03:31.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.824+0000 7f46717fa640 1 -- 172.21.15.43:0/2019726004 <== mds.0 v2:172.21.15.43:6832/2603892772 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555e40bc1540 con 0x555e40c7e0d0 2024-04-25T09:03:31.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.824+0000 7f46717fa640 10 client.9426 handle_command_reply: tid=0 2024-04-25T09:03:31.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.824+0000 7f469b1cc740 1 client.9426 shutdown 2024-04-25T09:03:31.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.824+0000 7f469b1cc740 1 -- 172.21.15.43:0/2019726004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e40c8c8d0 msgr2=0x555e40c7d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:31.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.824+0000 7f469b1cc740 1 --2- 172.21.15.43:0/2019726004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e40c8c8d0 0x555e40c7d990 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f46880798e0 tx=0x7f4688079be0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.824+0000 7f469b1cc740 1 -- 172.21.15.43:0/2019726004 shutdown_connections 2024-04-25T09:03:31.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.824+0000 7f469b1cc740 1 --2- 172.21.15.43:0/2019726004 >> [v2:172.21.15.43:6832/2603892772,v1:172.21.15.43:6834/2603892772] conn(0x555e40c7e0d0 0x555e40c8e500 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.824+0000 7f469b1cc740 1 --2- 172.21.15.43:0/2019726004 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e40c8c8d0 0x555e40c7d990 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.819 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.824+0000 7f469b1cc740 1 --2- 172.21.15.43:0/2019726004 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e40c7acc0 0x555e40ca5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.824+0000 7f46717fa640 0 client.9426 ms_handle_reset on v2:172.21.15.43:6832/2603892772 2024-04-25T09:03:31.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.824+0000 7f469b1cc740 1 --2- 172.21.15.43:0/2019726004 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x555e40c50200 0x555e40c90ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.824+0000 7f469b1cc740 1 -- 172.21.15.43:0/2019726004 >> 172.21.15.43:0/2019726004 conn(0x555e40c18050 msgr2=0x555e40c789f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:31.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.824+0000 7f469b1cc740 1 -- 172.21.15.43:0/2019726004 shutdown_connections 2024-04-25T09:03:31.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.824+0000 7f469b1cc740 1 -- 172.21.15.43:0/2019726004 wait complete. 2024-04-25T09:03:31.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.824+0000 7f469b1cc740 20 client.9426 trim_cache size 0 max 16384 2024-04-25T09:03:31.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.840+0000 7f469b1cc740 1 Processor -- start 2024-04-25T09:03:31.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.840+0000 7f469b1cc740 1 -- start start 2024-04-25T09:03:31.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.840+0000 7f469b1cc740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x555e40c50200 0x555e40c789f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.840+0000 7f469b1cc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e40c7acc0 0x555e40c91710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.840+0000 7f469b1cc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e40c8c8d0 0x555e40c8ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.840+0000 7f469b1cc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46901ffba0 con 0x555e40c8c8d0 2024-04-25T09:03:31.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.840+0000 7f469b1cc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46901fee00 con 0x555e40c7acc0 2024-04-25T09:03:31.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.840+0000 7f469b1cc740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f469015df50 con 0x555e40c50200 2024-04-25T09:03:31.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.840+0000 7f4696991640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e40c8c8d0 0x555e40c8ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.841+0000 7f4696991640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e40c8c8d0 0x555e40c8ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50964/0 (socket says 172.21.15.43:50964) 2024-04-25T09:03:31.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.841+0000 7f4696991640 1 -- 172.21.15.43:0/949303198 learned_addr learned my addr 172.21.15.43:0/949303198 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:31.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.841+0000 7f4696190640 1 --2- 172.21.15.43:0/949303198 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x555e40c50200 0x555e40c789f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.841+0000 7f469598f640 1 --2- 172.21.15.43:0/949303198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e40c7acc0 0x555e40c91710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.841+0000 7f4696991640 1 -- 172.21.15.43:0/949303198 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x555e40c50200 msgr2=0x555e40c789f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:31.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.841+0000 7f4696991640 1 --2- 172.21.15.43:0/949303198 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x555e40c50200 0x555e40c789f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.841+0000 7f4696991640 1 -- 172.21.15.43:0/949303198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e40c7acc0 msgr2=0x555e40c91710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:31.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.841+0000 7f4696991640 1 --2- 172.21.15.43:0/949303198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e40c7acc0 0x555e40c91710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.841+0000 7f4696991640 1 -- 172.21.15.43:0/949303198 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e40861610 con 0x555e40c8c8d0 2024-04-25T09:03:31.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.841+0000 7f4696991640 1 --2- 172.21.15.43:0/949303198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e40c8c8d0 0x555e40c8ede0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f468807def0 tx=0x7f468808e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e265dd1d919f548 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:31.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.841+0000 7f46737fe640 1 -- 172.21.15.43:0/949303198 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4688091040 con 0x555e40c8c8d0 2024-04-25T09:03:31.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.841+0000 7f46737fe640 1 -- 172.21.15.43:0/949303198 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46880ab070 con 0x555e40c8c8d0 2024-04-25T09:03:31.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.841+0000 7f46737fe640 1 -- 172.21.15.43:0/949303198 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4688079540 con 0x555e40c8c8d0 2024-04-25T09:03:31.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.842+0000 7f469b1cc740 1 -- 172.21.15.43:0/949303198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e40c8c8d0 msgr2=0x555e40c8ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:31.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.842+0000 7f469b1cc740 1 --2- 172.21.15.43:0/949303198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e40c8c8d0 0x555e40c8ede0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f468807def0 tx=0x7f468808e000 comp rx=0 tx=0).stop 2024-04-25T09:03:31.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.842+0000 7f469b1cc740 1 -- 172.21.15.43:0/949303198 shutdown_connections 2024-04-25T09:03:31.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.842+0000 7f469b1cc740 1 --2- 172.21.15.43:0/949303198 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e40c8c8d0 0x555e40c8ede0 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.842+0000 7f469b1cc740 1 --2- 172.21.15.43:0/949303198 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e40c7acc0 0x555e40c91710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.842+0000 7f469b1cc740 1 --2- 172.21.15.43:0/949303198 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x555e40c50200 0x555e40c789f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.842+0000 7f469b1cc740 1 -- 172.21.15.43:0/949303198 >> 172.21.15.43:0/949303198 conn(0x555e40c18050 msgr2=0x555e40cb7480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:31.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.842+0000 7f469b1cc740 1 -- 172.21.15.43:0/949303198 shutdown_connections 2024-04-25T09:03:31.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.842+0000 7f469b1cc740 1 -- 172.21.15.43:0/949303198 wait complete. 2024-04-25T09:03:31.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.842+0000 7f469b1cc740 1 -- start start 2024-04-25T09:03:31.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.842+0000 7f469b1cc740 1 Processor -- start 2024-04-25T09:03:31.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.843+0000 7f469b1cc740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e40c50200 0x555e40ca4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.843+0000 7f469b1cc740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x555e40c7acc0 0x555e40ca48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.843+0000 7f469b1cc740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e40c8c8d0 0x555e40c900d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.843+0000 7f469b1cc740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e40c98970 con 0x555e40c50200 2024-04-25T09:03:31.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.843+0000 7f469b1cc740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f469015e550 con 0x555e40c8c8d0 2024-04-25T09:03:31.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.843+0000 7f469b1cc740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e40ca5580 con 0x555e40c7acc0 2024-04-25T09:03:31.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.843+0000 7f469b1cc740 10 client.0 _ll_register_callbacks cb 0x7f4680007360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:31.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.843+0000 7f4696190640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e40c50200 0x555e40ca4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.844+0000 7f4696190640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e40c50200 0x555e40ca4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50966/0 (socket says 172.21.15.43:50966) 2024-04-25T09:03:31.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.844+0000 7f4696190640 1 -- 172.21.15.43:0/541023050 learned_addr learned my addr 172.21.15.43:0/541023050 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:31.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.844+0000 7f469598f640 1 --2- 172.21.15.43:0/541023050 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x555e40c7acc0 0x555e40ca48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.844+0000 7f4696991640 1 --2- 172.21.15.43:0/541023050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e40c8c8d0 0x555e40c900d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.844+0000 7f4696190640 1 -- 172.21.15.43:0/541023050 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x555e40c7acc0 msgr2=0x555e40ca48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:31.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.844+0000 7f4696190640 1 --2- 172.21.15.43:0/541023050 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x555e40c7acc0 0x555e40ca48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.844+0000 7f4696190640 1 -- 172.21.15.43:0/541023050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e40c8c8d0 msgr2=0x555e40c900d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:31.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.844+0000 7f4696190640 1 --2- 172.21.15.43:0/541023050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e40c8c8d0 0x555e40c900d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.844+0000 7f4696190640 1 -- 172.21.15.43:0/541023050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e40c76e00 con 0x555e40c50200 2024-04-25T09:03:31.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.844+0000 7f4696190640 1 --2- 172.21.15.43:0/541023050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e40c50200 0x555e40ca4360 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f468c001a10 tx=0x7f468c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=966e86990ad59913 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:31.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.844+0000 7f46717fa640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:03:31.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.844+0000 7f46717fa640 1 -- 172.21.15.43:0/541023050 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c024420 con 0x555e40c50200 2024-04-25T09:03:31.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.844+0000 7f46717fa640 1 -- 172.21.15.43:0/541023050 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f468c010040 con 0x555e40c50200 2024-04-25T09:03:31.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.844+0000 7f469b1cc740 1 -- 172.21.15.43:0/541023050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f46880ab070 con 0x555e40c50200 2024-04-25T09:03:31.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.844+0000 7f46717fa640 1 -- 172.21.15.43:0/541023050 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f468c0245c0 con 0x555e40c50200 2024-04-25T09:03:31.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.844+0000 7f469b1cc740 10 client.9432 fetch_fsmap learned FSMap version 76 2024-04-25T09:03:31.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.844+0000 7f469b1cc740 1 -- 172.21.15.43:0/541023050 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x555e40c91c00 con 0x555e40c50200 2024-04-25T09:03:31.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.844+0000 7f46717fa640 1 -- 172.21.15.43:0/541023050 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f468c02a030 con 0x555e40c50200 2024-04-25T09:03:31.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.844+0000 7f46717fa640 1 -- 172.21.15.43:0/541023050 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 76) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f468c024760 con 0x555e40c50200 2024-04-25T09:03:31.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.844+0000 7f469b1cc740 10 client.9432 fetch_fsmap finished waiting for FSMap version 76 2024-04-25T09:03:31.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.845+0000 7f469b1cc740 10 client.9432 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.a 2024-04-25T09:03:31.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.845+0000 7f469b1cc740 20 client.9432 populate_metadata read hostname 'smithi043' 2024-04-25T09:03:31.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.845+0000 7f469b1cc740 1 --2- 172.21.15.43:0/541023050 >> [v2:172.21.15.43:6832/2603892772,v1:172.21.15.43:6834/2603892772] conn(0x555e40cffc00 0x555e40c90610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.845+0000 7f469b1cc740 4 client.9432 mds_command: new command op to 9237 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T09:03:31.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.845+0000 7f469b1cc740 1 -- 172.21.15.43:0/541023050 --> [v2:172.21.15.43:6832/2603892772,v1:172.21.15.43:6834/2603892772] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x555e40c92450 con 0x555e40cffc00 2024-04-25T09:03:31.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.845+0000 7f469598f640 1 --2- 172.21.15.43:0/541023050 >> [v2:172.21.15.43:6832/2603892772,v1:172.21.15.43:6834/2603892772] conn(0x555e40cffc00 0x555e40c90610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.845+0000 7f46717fa640 10 client.9432 ms_handle_connect on v2:172.21.15.43:6832/2603892772 2024-04-25T09:03:31.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.845+0000 7f469598f640 1 --2- 172.21.15.43:0/541023050 >> [v2:172.21.15.43:6832/2603892772,v1:172.21.15.43:6834/2603892772] conn(0x555e40cffc00 0x555e40c90610 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de6c9615dc341900 server_cookie=3f26454b56866e66 in_seq=0 out_seq=0 2024-04-25T09:03:31.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.847+0000 7f46717fa640 1 -- 172.21.15.43:0/541023050 <== mds.0 v2:172.21.15.43:6832/2603892772 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x555e40c92450 con 0x555e40cffc00 2024-04-25T09:03:31.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.847+0000 7f46717fa640 10 client.9432 handle_command_reply: tid=0 2024-04-25T09:03:31.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.847+0000 7f469b1cc740 1 client.9432 shutdown 2024-04-25T09:03:31.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.847+0000 7f469b1cc740 1 -- 172.21.15.43:0/541023050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e40c50200 msgr2=0x555e40ca4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:31.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.847+0000 7f469b1cc740 1 --2- 172.21.15.43:0/541023050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e40c50200 0x555e40ca4360 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f468c001a10 tx=0x7f468c011000 comp rx=0 tx=0).stop 2024-04-25T09:03:31.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.848+0000 7f469b1cc740 1 -- 172.21.15.43:0/541023050 shutdown_connections 2024-04-25T09:03:31.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.848+0000 7f469b1cc740 1 --2- 172.21.15.43:0/541023050 >> [v2:172.21.15.43:6832/2603892772,v1:172.21.15.43:6834/2603892772] conn(0x555e40cffc00 0x555e40c90610 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.848+0000 7f469b1cc740 1 --2- 172.21.15.43:0/541023050 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x555e40c8c8d0 0x555e40c900d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.848+0000 7f469b1cc740 1 --2- 172.21.15.43:0/541023050 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x555e40c7acc0 0x555e40ca48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.848+0000 7f46717fa640 0 client.9432 ms_handle_reset on v2:172.21.15.43:6832/2603892772 2024-04-25T09:03:31.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.848+0000 7f469b1cc740 1 --2- 172.21.15.43:0/541023050 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x555e40c50200 0x555e40ca4360 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.848+0000 7f469b1cc740 1 -- 172.21.15.43:0/541023050 >> 172.21.15.43:0/541023050 conn(0x555e40c18050 msgr2=0x555e40c78740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:31.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.848+0000 7f469b1cc740 1 -- 172.21.15.43:0/541023050 shutdown_connections 2024-04-25T09:03:31.838 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.848+0000 7f469b1cc740 1 -- 172.21.15.43:0/541023050 wait complete. 2024-04-25T09:03:31.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.848+0000 7f469b1cc740 20 client.9432 trim_cache size 0 max 16384 2024-04-25T09:03:31.839 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:03:31.839 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0, 2024-04-25T09:03:31.839 INFO:teuthology.orchestra.run.smithi043.stdout: "scrub_tag": "c7b584d3-21e2-402b-92bd-e2f8f109206e", 2024-04-25T09:03:31.839 INFO:teuthology.orchestra.run.smithi043.stdout: "mode": "asynchronous" 2024-04-25T09:03:31.839 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:03:31.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.848+0000 7f46983f4640 1 -- 172.21.15.43:0/1013571607 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f467406b4d0 msgr2=0x7f467406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:31.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.848+0000 7f46983f4640 1 --2- 172.21.15.43:0/1013571607 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f467406b4d0 0x7f467406d990 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f468c000f60 tx=0x7f468c007c10 comp rx=0 tx=0).stop 2024-04-25T09:03:31.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.848+0000 7f46983f4640 1 -- 172.21.15.43:0/1013571607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46900cf2e0 msgr2=0x7f46901491c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:31.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.848+0000 7f46983f4640 1 --2- 172.21.15.43:0/1013571607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46900cf2e0 0x7f46901491c0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f4688036040 tx=0x7f4688004a40 comp rx=0 tx=0).stop 2024-04-25T09:03:31.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.848+0000 7f46983f4640 1 -- 172.21.15.43:0/1013571607 shutdown_connections 2024-04-25T09:03:31.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.848+0000 7f46983f4640 1 --2- 172.21.15.43:0/1013571607 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f46900cf2e0 0x7f46901491c0 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.848+0000 7f46983f4640 1 --2- 172.21.15.43:0/1013571607 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f46900b4300 0x7f4690148c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.848+0000 7f46983f4640 1 --2- 172.21.15.43:0/1013571607 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f467406b4d0 0x7f467406d990 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.848+0000 7f46983f4640 1 --2- 172.21.15.43:0/1013571607 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f46900aeea0 0x7f46900adb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.848+0000 7f46983f4640 1 -- 172.21.15.43:0/1013571607 >> 172.21.15.43:0/1013571607 conn(0x7f469006ab30 msgr2=0x7f4690071240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:31.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.848+0000 7f46983f4640 1 -- 172.21.15.43:0/1013571607 shutdown_connections 2024-04-25T09:03:31.841 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.849+0000 7f46983f4640 1 -- 172.21.15.43:0/1013571607 wait complete. 2024-04-25T09:03:31.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:31.847 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub pause 2024-04-25T09:03:31.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.987+0000 7f69ca717640 1 Processor -- start 2024-04-25T09:03:31.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.987+0000 7f69ca717640 1 -- start start 2024-04-25T09:03:31.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.988+0000 7f69ca717640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f69c40b59e0 0x7f69c40d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.988+0000 7f69ca717640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69c40b5e70 con 0x7f69c415e090 2024-04-25T09:03:31.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.988+0000 7f69ca717640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69c4162610 con 0x7f69c40b5560 2024-04-25T09:03:31.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.988+0000 7f69ca717640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69c40d1da0 con 0x7f69c40b59e0 2024-04-25T09:03:31.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.988+0000 7f69c3fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f69c415e090 0x7f69c4161aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:37426/0 (socket says 172.21.15.43:37426) 2024-04-25T09:03:31.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.988+0000 7f69c3fff640 1 -- 172.21.15.43:0/824585648 learned_addr learned my addr 172.21.15.43:0/824585648 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:03:31.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.989+0000 7f69c8cb4640 1 --2- 172.21.15.43:0/824585648 >> v2:172.21.15.193:3300/0 conn(0x7f69c40b59e0 0x7f69c40d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.989+0000 7f69c2ffd640 1 -- 172.21.15.43:0/824585648 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2600768816 0 0) 0x7f69c40b5e70 con 0x7f69c415e090 2024-04-25T09:03:31.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.989+0000 7f69c2ffd640 1 -- 172.21.15.43:0/824585648 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69ac003520 con 0x7f69c415e090 2024-04-25T09:03:31.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.989+0000 7f69c2ffd640 1 -- 172.21.15.43:0/824585648 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3140444905 0 0) 0x7f69c4162610 con 0x7f69c40b5560 2024-04-25T09:03:31.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.989+0000 7f69c2ffd640 1 -- 172.21.15.43:0/824585648 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69c40b5e70 con 0x7f69c40b5560 2024-04-25T09:03:31.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.989+0000 7f69c2ffd640 1 -- 172.21.15.43:0/824585648 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1703700029 0 0) 0x7f69ac003520 con 0x7f69c415e090 2024-04-25T09:03:31.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.989+0000 7f69c2ffd640 1 -- 172.21.15.43:0/824585648 >> v2:172.21.15.193:3300/0 conn(0x7f69c40b59e0 msgr2=0x7f69c40d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:31.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.989+0000 7f69c2ffd640 1 --2- 172.21.15.43:0/824585648 >> v2:172.21.15.193:3300/0 conn(0x7f69c40b59e0 0x7f69c40d2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.989+0000 7f69c2ffd640 1 -- 172.21.15.43:0/824585648 >> v1:172.21.15.193:6789/0 conn(0x7f69c40b5560 legacy=0x7f69c40d2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:31.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.989+0000 7f69c2ffd640 1 -- 172.21.15.43:0/824585648 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c40d1750 con 0x7f69c415e090 2024-04-25T09:03:31.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.989+0000 7f69c2ffd640 1 -- 172.21.15.43:0/824585648 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f69b4001670 con 0x7f69c415e090 2024-04-25T09:03:31.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.989+0000 7f69c8cb4640 1 --2- 172.21.15.43:0/824585648 >> v2:172.21.15.193:3300/0 conn(0x7f69c40b59e0 0x7f69c40d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:03:31.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.989+0000 7f69c2ffd640 1 -- 172.21.15.43:0/824585648 >> v1:172.21.15.43:6789/0 conn(0x7f69c415e090 legacy=0x7f69c4161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:31.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.989+0000 7f69c2ffd640 1 --2- 172.21.15.43:0/824585648 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69ac003b40 0x7f69ac003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.989+0000 7f69c2ffd640 1 --2- 172.21.15.43:0/824585648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69ac004800 0x7f69ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.989+0000 7f69c2ffd640 1 --2- 172.21.15.43:0/824585648 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69ac0051c0 0x7f69ac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.989+0000 7f69c2ffd640 1 -- 172.21.15.43:0/824585648 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c40d1da0 con 0x7f69ac004800 2024-04-25T09:03:31.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.989+0000 7f69c2ffd640 1 -- 172.21.15.43:0/824585648 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69ac00bfb0 con 0x7f69ac0051c0 2024-04-25T09:03:31.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.989+0000 7f69c2ffd640 1 -- 172.21.15.43:0/824585648 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69ac00c1b0 con 0x7f69ac003b40 2024-04-25T09:03:31.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.989+0000 7f69c37fe640 1 --2- 172.21.15.43:0/824585648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69ac004800 0x7f69ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.989+0000 7f69c3fff640 1 --2- 172.21.15.43:0/824585648 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69ac003b40 0x7f69ac003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.989+0000 7f69c8cb4640 1 --2- 172.21.15.43:0/824585648 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69ac0051c0 0x7f69ac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.989+0000 7f69c37fe640 1 -- 172.21.15.43:0/824585648 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69ac003b40 msgr2=0x7f69ac003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:31.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.989+0000 7f69c37fe640 1 --2- 172.21.15.43:0/824585648 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69ac003b40 0x7f69ac003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.989+0000 7f69c37fe640 1 -- 172.21.15.43:0/824585648 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69ac0051c0 msgr2=0x7f69ac00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:31.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.989+0000 7f69c37fe640 1 --2- 172.21.15.43:0/824585648 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69ac0051c0 0x7f69ac00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.989+0000 7f69c37fe640 1 -- 172.21.15.43:0/824585648 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69ac003520 con 0x7f69ac004800 2024-04-25T09:03:31.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.989+0000 7f69c8cb4640 1 -- 172.21.15.43:0/824585648 reap_dead start 2024-04-25T09:03:31.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.990+0000 7f69c37fe640 1 --2- 172.21.15.43:0/824585648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69ac004800 0x7f69ac004c80 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f69b80018a0 tx=0x7f69ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5acf861271999c98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:31.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.990+0000 7f69c2ffd640 1 -- 172.21.15.43:0/824585648 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b80014f0 con 0x7f69ac004800 2024-04-25T09:03:31.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.990+0000 7f69c2ffd640 1 -- 172.21.15.43:0/824585648 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b80052d0 con 0x7f69ac004800 2024-04-25T09:03:31.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.990+0000 7f69c2ffd640 1 -- 172.21.15.43:0/824585648 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b802ac00 con 0x7f69ac004800 2024-04-25T09:03:31.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.991+0000 7f69ca717640 1 -- 172.21.15.43:0/824585648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69ac004800 msgr2=0x7f69ac004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:31.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.991+0000 7f69ca717640 1 --2- 172.21.15.43:0/824585648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69ac004800 0x7f69ac004c80 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f69b80018a0 tx=0x7f69ac005830 comp rx=0 tx=0).stop 2024-04-25T09:03:31.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.991+0000 7f69ca717640 1 -- 172.21.15.43:0/824585648 shutdown_connections 2024-04-25T09:03:31.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.991+0000 7f69ca717640 1 --2- 172.21.15.43:0/824585648 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69ac004800 0x7f69ac004c80 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.991+0000 7f69ca717640 1 -- 172.21.15.43:0/824585648 >> 172.21.15.43:0/824585648 conn(0x7f69c40d0560 msgr2=0x7f69c40d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:31.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.991+0000 7f69ca717640 1 -- 172.21.15.43:0/824585648 shutdown_connections 2024-04-25T09:03:31.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.991+0000 7f69ca717640 1 -- 172.21.15.43:0/824585648 wait complete. 2024-04-25T09:03:31.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.992+0000 7f69ca717640 1 Processor -- start 2024-04-25T09:03:31.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.992+0000 7f69ca717640 1 -- start start 2024-04-25T09:03:31.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.993+0000 7f69ca717640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69ac004800 0x7f69c41500d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.993+0000 7f69ca717640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69c40c8850 0x7f69c40c8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.993+0000 7f69ca717640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69c40cabb0 0x7f69c40cb030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.993+0000 7f69ca717640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c40b3280 con 0x7f69c40c8850 2024-04-25T09:03:31.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.993+0000 7f69ca717640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c41628f0 con 0x7f69c40cabb0 2024-04-25T09:03:31.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.993+0000 7f69ca717640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69c4161aa0 con 0x7f69ac004800 2024-04-25T09:03:31.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.993+0000 7f69c37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69c40c8850 0x7f69c40c8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.993+0000 7f69c37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69c40c8850 0x7f69c40c8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50980/0 (socket says 172.21.15.43:50980) 2024-04-25T09:03:31.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.993+0000 7f69c37fe640 1 -- 172.21.15.43:0/1650790685 learned_addr learned my addr 172.21.15.43:0/1650790685 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:31.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.993+0000 7f69c8cb4640 1 --2- 172.21.15.43:0/1650790685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69c40cabb0 0x7f69c40cb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.993+0000 7f69c3fff640 1 --2- 172.21.15.43:0/1650790685 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69ac004800 0x7f69c41500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.993+0000 7f69c37fe640 1 -- 172.21.15.43:0/1650790685 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69ac004800 msgr2=0x7f69c41500d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:31.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.993+0000 7f69c37fe640 1 --2- 172.21.15.43:0/1650790685 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69ac004800 0x7f69c41500d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.993+0000 7f69c37fe640 1 -- 172.21.15.43:0/1650790685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69c40cabb0 msgr2=0x7f69c40cb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:31.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.993+0000 7f69c37fe640 1 --2- 172.21.15.43:0/1650790685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69c40cabb0 0x7f69c40cb030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.993+0000 7f69c37fe640 1 -- 172.21.15.43:0/1650790685 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69c40c77b0 con 0x7f69c40c8850 2024-04-25T09:03:31.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.994+0000 7f69c37fe640 1 --2- 172.21.15.43:0/1650790685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69c40c8850 0x7f69c40c8cd0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f69b80019c0 tx=0x7f69b80012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:31.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.994+0000 7f69c0ff9640 1 -- 172.21.15.43:0/1650790685 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b8037040 con 0x7f69c40c8850 2024-04-25T09:03:31.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.994+0000 7f69c0ff9640 1 -- 172.21.15.43:0/1650790685 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b802be50 con 0x7f69c40c8850 2024-04-25T09:03:31.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.994+0000 7f69ca717640 1 -- 172.21.15.43:0/1650790685 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69c40c7a90 con 0x7f69c40c8850 2024-04-25T09:03:31.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.994+0000 7f69c0ff9640 1 -- 172.21.15.43:0/1650790685 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b80014f0 con 0x7f69c40c8850 2024-04-25T09:03:31.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.994+0000 7f69ca717640 1 -- 172.21.15.43:0/1650790685 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69c40c8050 con 0x7f69c40c8850 2024-04-25T09:03:31.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.995+0000 7f69c0ff9640 1 -- 172.21.15.43:0/1650790685 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f69b8044020 con 0x7f69c40c8850 2024-04-25T09:03:31.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.996+0000 7f69c0ff9640 1 --2- 172.21.15.43:0/1650790685 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f69b006b4d0 0x7f69b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.996+0000 7f69c0ff9640 1 -- 172.21.15.43:0/1650790685 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f69b80a9e10 con 0x7f69c40c8850 2024-04-25T09:03:31.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.996+0000 7f69c3fff640 1 --2- 172.21.15.43:0/1650790685 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f69b006b4d0 0x7f69b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.996+0000 7f69c3fff640 1 --2- 172.21.15.43:0/1650790685 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f69b006b4d0 0x7f69b006d990 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f69b4001050 tx=0x7f69b4001e30 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:31.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.997+0000 7f69cd40d740 1 Processor -- start 2024-04-25T09:03:31.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.997+0000 7f69cd40d740 1 -- start start 2024-04-25T09:03:31.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.997+0000 7f69cd40d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0c98a5ca0 0x55a0c98a7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.997+0000 7f69cd40d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0c98e2370 0x55a0c98a6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.997+0000 7f69cd40d740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a0c98d0760 0x55a0c98d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.997+0000 7f69cd40d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0c9808b90 con 0x55a0c98e2370 2024-04-25T09:03:31.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.997+0000 7f69cd40d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0c97b1e40 con 0x55a0c98a5ca0 2024-04-25T09:03:31.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.997+0000 7f69cd40d740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a0c9804c30 con 0x55a0c98d0760 2024-04-25T09:03:31.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.997+0000 7f69c37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0c98e2370 0x55a0c98a6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.998+0000 7f69c37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0c98e2370 0x55a0c98a6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50986/0 (socket says 172.21.15.43:50986) 2024-04-25T09:03:31.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.998+0000 7f69c37fe640 1 -- 172.21.15.43:0/834000551 learned_addr learned my addr 172.21.15.43:0/834000551 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:31.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.998+0000 7f69c8cb4640 1 --2- 172.21.15.43:0/834000551 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a0c98d0760 0x55a0c98d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.998+0000 7f69c3fff640 1 --2- 172.21.15.43:0/834000551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0c98a5ca0 0x55a0c98a7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.998+0000 7f69c37fe640 1 -- 172.21.15.43:0/834000551 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a0c98d0760 msgr2=0x55a0c98d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:31.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.998+0000 7f69c37fe640 1 --2- 172.21.15.43:0/834000551 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a0c98d0760 0x55a0c98d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.998+0000 7f69c37fe640 1 -- 172.21.15.43:0/834000551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0c98a5ca0 msgr2=0x55a0c98a7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:31.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.998+0000 7f69c37fe640 1 --2- 172.21.15.43:0/834000551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0c98a5ca0 0x55a0c98a7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.998+0000 7f69c37fe640 1 -- 172.21.15.43:0/834000551 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a0c95e77d0 con 0x55a0c98e2370 2024-04-25T09:03:31.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.998+0000 7f69c37fe640 1 --2- 172.21.15.43:0/834000551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0c98e2370 0x55a0c98a6080 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f69b80a8360 tx=0x7f69b8074980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ea416a0931e9e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:31.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.998+0000 7f69a9ffb640 1 -- 172.21.15.43:0/834000551 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b80768b0 con 0x55a0c98e2370 2024-04-25T09:03:31.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.998+0000 7f69a9ffb640 1 -- 172.21.15.43:0/834000551 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b80a6070 con 0x55a0c98e2370 2024-04-25T09:03:31.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.998+0000 7f69a9ffb640 1 -- 172.21.15.43:0/834000551 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b8074c20 con 0x55a0c98e2370 2024-04-25T09:03:31.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.999+0000 7f69cd40d740 1 -- 172.21.15.43:0/834000551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0c98e2370 msgr2=0x55a0c98a6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:31.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.999+0000 7f69cd40d740 1 --2- 172.21.15.43:0/834000551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0c98e2370 0x55a0c98a6080 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f69b80a8360 tx=0x7f69b8074980 comp rx=0 tx=0).stop 2024-04-25T09:03:31.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.999+0000 7f69cd40d740 1 -- 172.21.15.43:0/834000551 shutdown_connections 2024-04-25T09:03:31.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.999+0000 7f69cd40d740 1 --2- 172.21.15.43:0/834000551 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a0c98d0760 0x55a0c98d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.999+0000 7f69cd40d740 1 --2- 172.21.15.43:0/834000551 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0c98e2370 0x55a0c98a6080 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.999+0000 7f69cd40d740 1 --2- 172.21.15.43:0/834000551 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0c98a5ca0 0x55a0c98a7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.999+0000 7f69cd40d740 1 -- 172.21.15.43:0/834000551 >> 172.21.15.43:0/834000551 conn(0x55a0c986d870 msgr2=0x55a0c98eb760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:31.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.999+0000 7f69cd40d740 1 -- 172.21.15.43:0/834000551 shutdown_connections 2024-04-25T09:03:31.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.999+0000 7f69cd40d740 1 -- 172.21.15.43:0/834000551 wait complete. 2024-04-25T09:03:31.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:31.999+0000 7f69cd40d740 1 -- start start 2024-04-25T09:03:31.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.000+0000 7f69cd40d740 1 Processor -- start 2024-04-25T09:03:31.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.000+0000 7f69cd40d740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a0c98a5ca0 0x55a0c98cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.000+0000 7f69cd40d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0c98d0760 0x55a0c98ff490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.000+0000 7f69cd40d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0c98e2370 0x55a0c98d2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.000+0000 7f69cd40d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0c97b1e40 con 0x55a0c98e2370 2024-04-25T09:03:31.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.000+0000 7f69cd40d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0c97c53e0 con 0x55a0c98d0760 2024-04-25T09:03:31.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.000+0000 7f69cd40d740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a0c9808b90 con 0x55a0c98a5ca0 2024-04-25T09:03:31.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.000+0000 7f69cd40d740 10 client.0 _ll_register_callbacks cb 0x55a0c97c4db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:31.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.000+0000 7f69c8cb4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0c98e2370 0x55a0c98d2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.000+0000 7f69c3fff640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a0c98a5ca0 0x55a0c98cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.000+0000 7f69c8cb4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0c98e2370 0x55a0c98d2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:50988/0 (socket says 172.21.15.43:50988) 2024-04-25T09:03:31.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.000+0000 7f69c8cb4640 1 -- 172.21.15.43:0/2681653761 learned_addr learned my addr 172.21.15.43:0/2681653761 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:31.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.000+0000 7f69c37fe640 1 --2- 172.21.15.43:0/2681653761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0c98d0760 0x55a0c98ff490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.001+0000 7f69c8cb4640 1 -- 172.21.15.43:0/2681653761 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a0c98a5ca0 msgr2=0x55a0c98cd5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:31.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.001+0000 7f69c8cb4640 1 --2- 172.21.15.43:0/2681653761 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a0c98a5ca0 0x55a0c98cd5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.001+0000 7f69c8cb4640 1 -- 172.21.15.43:0/2681653761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0c98d0760 msgr2=0x55a0c98ff490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:31.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.001+0000 7f69c8cb4640 1 --2- 172.21.15.43:0/2681653761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0c98d0760 0x55a0c98ff490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.001+0000 7f69c8cb4640 1 -- 172.21.15.43:0/2681653761 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a0c94dd0f0 con 0x55a0c98e2370 2024-04-25T09:03:31.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.001+0000 7f69c37fe640 1 --2- 172.21.15.43:0/2681653761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0c98d0760 0x55a0c98ff490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:03:31.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.001+0000 7f69c8cb4640 1 --2- 172.21.15.43:0/2681653761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0c98e2370 0x55a0c98d2d40 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f69c408ce90 tx=0x7f69bc015760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a18247065aa76c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:31.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.001+0000 7f69a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:03:31.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.001+0000 7f69a2ffd640 1 -- 172.21.15.43:0/2681653761 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69bc024060 con 0x55a0c98e2370 2024-04-25T09:03:31.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.001+0000 7f69a2ffd640 1 -- 172.21.15.43:0/2681653761 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f69bc003990 con 0x55a0c98e2370 2024-04-25T09:03:31.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.001+0000 7f69cd40d740 1 -- 172.21.15.43:0/2681653761 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f69b80a6070 con 0x55a0c98e2370 2024-04-25T09:03:31.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.001+0000 7f69a2ffd640 1 -- 172.21.15.43:0/2681653761 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69bc02a050 con 0x55a0c98e2370 2024-04-25T09:03:31.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.001+0000 7f69cd40d740 10 client.9444 fetch_fsmap learned FSMap version 76 2024-04-25T09:03:31.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.001+0000 7f69cd40d740 1 -- 172.21.15.43:0/2681653761 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x55a0c9647f60 con 0x55a0c98e2370 2024-04-25T09:03:31.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.001+0000 7f69a2ffd640 1 -- 172.21.15.43:0/2681653761 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b80a6070 con 0x55a0c98e2370 2024-04-25T09:03:31.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.001+0000 7f69a2ffd640 1 -- 172.21.15.43:0/2681653761 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 76) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f69bc02d020 con 0x55a0c98e2370 2024-04-25T09:03:31.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.002+0000 7f69cd40d740 10 client.9444 fetch_fsmap finished waiting for FSMap version 76 2024-04-25T09:03:31.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.002+0000 7f69cd40d740 10 client.9444 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.a 2024-04-25T09:03:31.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.002+0000 7f69cd40d740 20 client.9444 populate_metadata read hostname 'smithi043' 2024-04-25T09:03:31.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.002+0000 7f69cd40d740 1 --2- 172.21.15.43:0/2681653761 >> [v2:172.21.15.43:6832/2603892772,v1:172.21.15.43:6834/2603892772] conn(0x55a0c98e8c80 0x55a0c98e5e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:31.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.002+0000 7f69cd40d740 4 client.9444 mds_command: new command op to 9237 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:03:31.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.002+0000 7f69cd40d740 1 -- 172.21.15.43:0/2681653761 --> [v2:172.21.15.43:6832/2603892772,v1:172.21.15.43:6834/2603892772] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a0c98160d0 con 0x55a0c98e8c80 2024-04-25T09:03:31.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.002+0000 7f69c3fff640 1 --2- 172.21.15.43:0/2681653761 >> [v2:172.21.15.43:6832/2603892772,v1:172.21.15.43:6834/2603892772] conn(0x55a0c98e8c80 0x55a0c98e5e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:31.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.002+0000 7f69a2ffd640 10 client.9444 ms_handle_connect on v2:172.21.15.43:6832/2603892772 2024-04-25T09:03:31.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.002+0000 7f69c3fff640 1 --2- 172.21.15.43:0/2681653761 >> [v2:172.21.15.43:6832/2603892772,v1:172.21.15.43:6834/2603892772] conn(0x55a0c98e8c80 0x55a0c98e5e40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9aede4a384c62c39 server_cookie=a766f5ca938c64f0 in_seq=0 out_seq=0 2024-04-25T09:03:31.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.002+0000 7f69a2ffd640 1 -- 172.21.15.43:0/2681653761 <== mds.0 v2:172.21.15.43:6832/2603892772 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a0c98160d0 con 0x55a0c98e8c80 2024-04-25T09:03:31.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.002+0000 7f69a2ffd640 10 client.9444 handle_command_reply: tid=0 2024-04-25T09:03:31.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.002+0000 7f69cd40d740 1 client.9444 shutdown 2024-04-25T09:03:31.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.003+0000 7f69cd40d740 1 -- 172.21.15.43:0/2681653761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0c98e2370 msgr2=0x55a0c98d2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:31.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.003+0000 7f69cd40d740 1 --2- 172.21.15.43:0/2681653761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0c98e2370 0x55a0c98d2d40 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f69c408ce90 tx=0x7f69bc015760 comp rx=0 tx=0).stop 2024-04-25T09:03:31.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.003+0000 7f69cd40d740 1 -- 172.21.15.43:0/2681653761 shutdown_connections 2024-04-25T09:03:31.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.003+0000 7f69cd40d740 1 --2- 172.21.15.43:0/2681653761 >> [v2:172.21.15.43:6832/2603892772,v1:172.21.15.43:6834/2603892772] conn(0x55a0c98e8c80 0x55a0c98e5e40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.003+0000 7f69cd40d740 1 --2- 172.21.15.43:0/2681653761 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0c98e2370 0x55a0c98d2d40 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.003+0000 7f69cd40d740 1 --2- 172.21.15.43:0/2681653761 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0c98d0760 0x55a0c98ff490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.003+0000 7f69cd40d740 1 --2- 172.21.15.43:0/2681653761 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a0c98a5ca0 0x55a0c98cd5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:31.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.003+0000 7f69a2ffd640 0 client.9444 ms_handle_reset on v2:172.21.15.43:6832/2603892772 2024-04-25T09:03:31.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.003+0000 7f69cd40d740 1 -- 172.21.15.43:0/2681653761 >> 172.21.15.43:0/2681653761 conn(0x55a0c986d870 msgr2=0x55a0c98ee9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:31.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.003+0000 7f69cd40d740 1 -- 172.21.15.43:0/2681653761 shutdown_connections 2024-04-25T09:03:31.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.003+0000 7f69cd40d740 1 -- 172.21.15.43:0/2681653761 wait complete. 2024-04-25T09:03:31.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.003+0000 7f69cd40d740 20 client.9444 trim_cache size 0 max 16384 2024-04-25T09:03:32.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.018+0000 7f69cd40d740 1 Processor -- start 2024-04-25T09:03:32.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.018+0000 7f69cd40d740 1 -- start start 2024-04-25T09:03:32.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.018+0000 7f69cd40d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0c98a5ca0 0x55a0c98e7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.018+0000 7f69cd40d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0c98d0760 0x55a0c98cc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.018+0000 7f69cd40d740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a0c98e2370 0x55a0c98e5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.018+0000 7f69cd40d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c4200600 con 0x55a0c98d0760 2024-04-25T09:03:32.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.019+0000 7f69cd40d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c41ff7e0 con 0x55a0c98a5ca0 2024-04-25T09:03:32.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.019+0000 7f69cd40d740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69c41fec10 con 0x55a0c98e2370 2024-04-25T09:03:32.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.019+0000 7f69c37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0c98d0760 0x55a0c98cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.019+0000 7f69c37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0c98d0760 0x55a0c98cc840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51000/0 (socket says 172.21.15.43:51000) 2024-04-25T09:03:32.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.019+0000 7f69c37fe640 1 -- 172.21.15.43:0/1745591856 learned_addr learned my addr 172.21.15.43:0/1745591856 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:32.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.019+0000 7f69c37fe640 1 -- 172.21.15.43:0/1745591856 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a0c98e2370 msgr2=0x55a0c98e5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:32.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.019+0000 7f69c37fe640 1 --2- 172.21.15.43:0/1745591856 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a0c98e2370 0x55a0c98e5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.019+0000 7f69c37fe640 1 -- 172.21.15.43:0/1745591856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0c98a5ca0 msgr2=0x55a0c98e7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:32.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.019+0000 7f69c3fff640 1 --2- 172.21.15.43:0/1745591856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0c98a5ca0 0x55a0c98e7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.019+0000 7f69c37fe640 1 --2- 172.21.15.43:0/1745591856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0c98a5ca0 0x55a0c98e7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.019+0000 7f69c37fe640 1 -- 172.21.15.43:0/1745591856 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a0c94b61b0 con 0x55a0c98d0760 2024-04-25T09:03:32.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.019+0000 7f69c3fff640 1 --2- 172.21.15.43:0/1745591856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0c98a5ca0 0x55a0c98e7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:32.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.019+0000 7f69c37fe640 1 --2- 172.21.15.43:0/1745591856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0c98d0760 0x55a0c98cc840 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f69b807c810 tx=0x7f69b807b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c518131e4d619e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:32.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.019+0000 7f69a97fa640 1 -- 172.21.15.43:0/1745591856 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b8078380 con 0x55a0c98d0760 2024-04-25T09:03:32.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.019+0000 7f69a97fa640 1 -- 172.21.15.43:0/1745591856 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b80a6040 con 0x55a0c98d0760 2024-04-25T09:03:32.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.019+0000 7f69a97fa640 1 -- 172.21.15.43:0/1745591856 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b8090050 con 0x55a0c98d0760 2024-04-25T09:03:32.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.020+0000 7f69cd40d740 1 -- 172.21.15.43:0/1745591856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0c98d0760 msgr2=0x55a0c98cc840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.020+0000 7f69cd40d740 1 --2- 172.21.15.43:0/1745591856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0c98d0760 0x55a0c98cc840 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f69b807c810 tx=0x7f69b807b000 comp rx=0 tx=0).stop 2024-04-25T09:03:32.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.020+0000 7f69cd40d740 1 -- 172.21.15.43:0/1745591856 shutdown_connections 2024-04-25T09:03:32.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.020+0000 7f69cd40d740 1 --2- 172.21.15.43:0/1745591856 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a0c98e2370 0x55a0c98e5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.020+0000 7f69cd40d740 1 --2- 172.21.15.43:0/1745591856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0c98d0760 0x55a0c98cc840 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.020+0000 7f69cd40d740 1 --2- 172.21.15.43:0/1745591856 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0c98a5ca0 0x55a0c98e7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.020+0000 7f69cd40d740 1 -- 172.21.15.43:0/1745591856 >> 172.21.15.43:0/1745591856 conn(0x55a0c986d870 msgr2=0x55a0c990df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:32.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.020+0000 7f69cd40d740 1 -- 172.21.15.43:0/1745591856 shutdown_connections 2024-04-25T09:03:32.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.020+0000 7f69cd40d740 1 -- 172.21.15.43:0/1745591856 wait complete. 2024-04-25T09:03:32.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.020+0000 7f69cd40d740 1 -- start start 2024-04-25T09:03:32.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.021+0000 7f69cd40d740 1 Processor -- start 2024-04-25T09:03:32.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.021+0000 7f69cd40d740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0c98a5ca0 0x55a0c98f9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.021+0000 7f69cd40d740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a0c98d0760 0x55a0c98fa490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.021+0000 7f69cd40d740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0c98e2370 0x55a0c9904600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.021+0000 7f69cd40d740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c41ff1e0 con 0x55a0c98e2370 2024-04-25T09:03:32.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.021+0000 7f69cd40d740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a0c98a2570 con 0x55a0c98a5ca0 2024-04-25T09:03:32.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.021+0000 7f69cd40d740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a0c98edf00 con 0x55a0c98d0760 2024-04-25T09:03:32.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.021+0000 7f69cd40d740 10 client.0 _ll_register_callbacks cb 0x7f69bc009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:32.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.021+0000 7f69c8cb4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0c98e2370 0x55a0c9904600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.022+0000 7f69c37fe640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a0c98d0760 0x55a0c98fa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.022+0000 7f69c3fff640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0c98a5ca0 0x55a0c98f9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.022+0000 7f69c8cb4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0c98e2370 0x55a0c9904600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51012/0 (socket says 172.21.15.43:51012) 2024-04-25T09:03:32.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.022+0000 7f69c8cb4640 1 -- 172.21.15.43:0/87897474 learned_addr learned my addr 172.21.15.43:0/87897474 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:32.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.022+0000 7f69c8cb4640 1 -- 172.21.15.43:0/87897474 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a0c98d0760 msgr2=0x55a0c98fa490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:32.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.022+0000 7f69c8cb4640 1 --2- 172.21.15.43:0/87897474 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a0c98d0760 0x55a0c98fa490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.022+0000 7f69c8cb4640 1 -- 172.21.15.43:0/87897474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0c98a5ca0 msgr2=0x55a0c98f9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:32.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.022+0000 7f69c8cb4640 1 --2- 172.21.15.43:0/87897474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0c98a5ca0 0x55a0c98f9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.022+0000 7f69c8cb4640 1 -- 172.21.15.43:0/87897474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a0c98e8270 con 0x55a0c98e2370 2024-04-25T09:03:32.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.022+0000 7f69c3fff640 1 --2- 172.21.15.43:0/87897474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0c98a5ca0 0x55a0c98f9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:03:32.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.022+0000 7f69c8cb4640 1 --2- 172.21.15.43:0/87897474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0c98e2370 0x55a0c9904600 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f69bc009380 tx=0x7f69bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57c7a4d90912125d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:32.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.022+0000 7f69a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:03:32.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.022+0000 7f69a2ffd640 1 -- 172.21.15.43:0/87897474 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69bc032020 con 0x55a0c98e2370 2024-04-25T09:03:32.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.022+0000 7f69a2ffd640 1 -- 172.21.15.43:0/87897474 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f69bc01f450 con 0x55a0c98e2370 2024-04-25T09:03:32.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.022+0000 7f69cd40d740 1 -- 172.21.15.43:0/87897474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f69b80a6040 con 0x55a0c98e2370 2024-04-25T09:03:32.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.022+0000 7f69a2ffd640 1 -- 172.21.15.43:0/87897474 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f69bc03c050 con 0x55a0c98e2370 2024-04-25T09:03:32.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.022+0000 7f69cd40d740 10 client.9450 fetch_fsmap learned FSMap version 76 2024-04-25T09:03:32.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.022+0000 7f69cd40d740 1 -- 172.21.15.43:0/87897474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x55a0c98e7c70 con 0x55a0c98e2370 2024-04-25T09:03:32.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.022+0000 7f69a2ffd640 1 -- 172.21.15.43:0/87897474 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b80a6040 con 0x55a0c98e2370 2024-04-25T09:03:32.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.023+0000 7f69a2ffd640 1 -- 172.21.15.43:0/87897474 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 76) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f69bc03f020 con 0x55a0c98e2370 2024-04-25T09:03:32.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.023+0000 7f69cd40d740 10 client.9450 fetch_fsmap finished waiting for FSMap version 76 2024-04-25T09:03:32.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.023+0000 7f69cd40d740 10 client.9450 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.a 2024-04-25T09:03:32.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.023+0000 7f69cd40d740 20 client.9450 populate_metadata read hostname 'smithi043' 2024-04-25T09:03:32.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.023+0000 7f69cd40d740 1 --2- 172.21.15.43:0/87897474 >> [v2:172.21.15.43:6832/2603892772,v1:172.21.15.43:6834/2603892772] conn(0x55a0c9955300 0x55a0c9904b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.023+0000 7f69cd40d740 4 client.9450 mds_command: new command op to 9237 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-25T09:03:32.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.023+0000 7f69cd40d740 1 -- 172.21.15.43:0/87897474 --> [v2:172.21.15.43:6832/2603892772,v1:172.21.15.43:6834/2603892772] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x55a0c9810420 con 0x55a0c9955300 2024-04-25T09:03:32.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.023+0000 7f69c3fff640 1 --2- 172.21.15.43:0/87897474 >> [v2:172.21.15.43:6832/2603892772,v1:172.21.15.43:6834/2603892772] conn(0x55a0c9955300 0x55a0c9904b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.023+0000 7f69c3fff640 1 --2- 172.21.15.43:0/87897474 >> [v2:172.21.15.43:6832/2603892772,v1:172.21.15.43:6834/2603892772] conn(0x55a0c9955300 0x55a0c9904b40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0672e6baa9825e4 server_cookie=b88809a27443baf2 in_seq=0 out_seq=0 2024-04-25T09:03:32.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.023+0000 7f69a2ffd640 10 client.9450 ms_handle_connect on v2:172.21.15.43:6832/2603892772 2024-04-25T09:03:32.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.023+0000 7f69a2ffd640 1 -- 172.21.15.43:0/87897474 <== mds.0 v2:172.21.15.43:6832/2603892772 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55a0c9810420 con 0x55a0c9955300 2024-04-25T09:03:32.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.023+0000 7f69a2ffd640 10 client.9450 handle_command_reply: tid=0 2024-04-25T09:03:32.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.023+0000 7f69cd40d740 1 client.9450 shutdown 2024-04-25T09:03:32.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.023+0000 7f69cd40d740 1 -- 172.21.15.43:0/87897474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0c98e2370 msgr2=0x55a0c9904600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.023+0000 7f69cd40d740 1 --2- 172.21.15.43:0/87897474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0c98e2370 0x55a0c9904600 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f69bc009380 tx=0x7f69bc004a40 comp rx=0 tx=0).stop 2024-04-25T09:03:32.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.023+0000 7f69cd40d740 1 -- 172.21.15.43:0/87897474 shutdown_connections 2024-04-25T09:03:32.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.023+0000 7f69cd40d740 1 --2- 172.21.15.43:0/87897474 >> [v2:172.21.15.43:6832/2603892772,v1:172.21.15.43:6834/2603892772] conn(0x55a0c9955300 0x55a0c9904b40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.024+0000 7f69cd40d740 1 --2- 172.21.15.43:0/87897474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x55a0c98e2370 0x55a0c9904600 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.024+0000 7f69a2ffd640 0 client.9450 ms_handle_reset on v2:172.21.15.43:6832/2603892772 2024-04-25T09:03:32.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.024+0000 7f69cd40d740 1 --2- 172.21.15.43:0/87897474 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x55a0c98d0760 0x55a0c98fa490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.024+0000 7f69cd40d740 1 --2- 172.21.15.43:0/87897474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x55a0c98a5ca0 0x55a0c98f9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.024+0000 7f69cd40d740 1 -- 172.21.15.43:0/87897474 >> 172.21.15.43:0/87897474 conn(0x55a0c986d870 msgr2=0x55a0c98e5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:32.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.024+0000 7f69cd40d740 1 -- 172.21.15.43:0/87897474 shutdown_connections 2024-04-25T09:03:32.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.024+0000 7f69cd40d740 1 -- 172.21.15.43:0/87897474 wait complete. 2024-04-25T09:03:32.018 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.024+0000 7f69cd40d740 20 client.9450 trim_cache size 0 max 16384 2024-04-25T09:03:32.018 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:03:32.018 INFO:teuthology.orchestra.run.smithi043.stdout: "return_code": 0 2024-04-25T09:03:32.019 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:03:32.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.024+0000 7f69ca717640 1 -- 172.21.15.43:0/1650790685 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f69b006b4d0 msgr2=0x7f69b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.024+0000 7f69ca717640 1 --2- 172.21.15.43:0/1650790685 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f69b006b4d0 0x7f69b006d990 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f69b4001050 tx=0x7f69b4001e30 comp rx=0 tx=0).stop 2024-04-25T09:03:32.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.024+0000 7f69ca717640 1 -- 172.21.15.43:0/1650790685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69c40c8850 msgr2=0x7f69c40c8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.024+0000 7f69ca717640 1 --2- 172.21.15.43:0/1650790685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69c40c8850 0x7f69c40c8cd0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f69b80019c0 tx=0x7f69b80012d0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.024+0000 7f69ca717640 1 -- 172.21.15.43:0/1650790685 shutdown_connections 2024-04-25T09:03:32.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.024+0000 7f69ca717640 1 --2- 172.21.15.43:0/1650790685 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f69b006b4d0 0x7f69b006d990 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.024+0000 7f69ca717640 1 --2- 172.21.15.43:0/1650790685 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f69c40cabb0 0x7f69c40cb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.024+0000 7f69ca717640 1 --2- 172.21.15.43:0/1650790685 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f69c40c8850 0x7f69c40c8cd0 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.024+0000 7f69ca717640 1 --2- 172.21.15.43:0/1650790685 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f69ac004800 0x7f69c41500d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.024+0000 7f69ca717640 1 -- 172.21.15.43:0/1650790685 >> 172.21.15.43:0/1650790685 conn(0x7f69c40d0560 msgr2=0x7f69c40b3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:32.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.024+0000 7f69ca717640 1 -- 172.21.15.43:0/1650790685 shutdown_connections 2024-04-25T09:03:32.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.024+0000 7f69ca717640 1 -- 172.21.15.43:0/1650790685 wait complete. 2024-04-25T09:03:32.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:32.023 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub status 2024-04-25T09:03:32.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.138+0000 7fa652d14640 1 Processor -- start 2024-04-25T09:03:32.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.138+0000 7fa652d14640 1 -- start start 2024-04-25T09:03:32.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.138+0000 7fa652d14640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa64c15ba40 0x7fa64c0d1560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.138+0000 7fa652d14640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa64c0d1e70 con 0x7fa64c0b74e0 2024-04-25T09:03:32.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.138+0000 7fa652d14640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa64c0d2130 con 0x7fa64c0b4990 2024-04-25T09:03:32.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.138+0000 7fa652d14640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa64c0d26f0 con 0x7fa64c15ba40 2024-04-25T09:03:32.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.139+0000 7fa650ab0640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa64c15ba40 0x7fa64c0d1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.139+0000 7fa6512b1640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa64c0b74e0 0x7fa64c0b23c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:37430/0 (socket says 172.21.15.43:37430) 2024-04-25T09:03:32.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.139+0000 7fa6512b1640 1 -- 172.21.15.43:0/235541301 learned_addr learned my addr 172.21.15.43:0/235541301 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:03:32.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.139+0000 7fa650ab0640 1 -- 172.21.15.43:0/235541301 >> v1:172.21.15.193:6789/0 conn(0x7fa64c0b4990 legacy=0x7fa64c0b6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.139+0000 7fa650ab0640 1 -- 172.21.15.43:0/235541301 >> v1:172.21.15.43:6789/0 conn(0x7fa64c0b74e0 legacy=0x7fa64c0b23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.139+0000 7fa64b7fe640 1 -- 172.21.15.43:0/235541301 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 258520358 0 0) 0x7fa64c0d2130 con 0x7fa64c0b4990 2024-04-25T09:03:32.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.139+0000 7fa650ab0640 1 -- 172.21.15.43:0/235541301 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa64c0d2410 con 0x7fa64c15ba40 2024-04-25T09:03:32.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.140+0000 7fa650ab0640 1 --2- 172.21.15.43:0/235541301 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa64c15ba40 0x7fa64c0d1560 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fa644009d90 tx=0x7fa64402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7602f2ed9f2b8272 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:32.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.140+0000 7fa64b7fe640 1 -- 172.21.15.43:0/235541301 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa644007740 con 0x7fa64c15ba40 2024-04-25T09:03:32.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.140+0000 7fa64b7fe640 1 -- 172.21.15.43:0/235541301 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6440078e0 con 0x7fa64c15ba40 2024-04-25T09:03:32.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.140+0000 7fa64b7fe640 1 -- 172.21.15.43:0/235541301 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa644007cb0 con 0x7fa64c15ba40 2024-04-25T09:03:32.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.140+0000 7fa652d14640 1 -- 172.21.15.43:0/235541301 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa64c15ba40 msgr2=0x7fa64c0d1560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.140+0000 7fa652d14640 1 --2- 172.21.15.43:0/235541301 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa64c15ba40 0x7fa64c0d1560 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fa644009d90 tx=0x7fa64402a5f0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.140+0000 7fa652d14640 1 -- 172.21.15.43:0/235541301 shutdown_connections 2024-04-25T09:03:32.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.140+0000 7fa652d14640 1 --2- 172.21.15.43:0/235541301 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa64c15ba40 0x7fa64c0d1560 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.140+0000 7fa652d14640 1 -- 172.21.15.43:0/235541301 >> 172.21.15.43:0/235541301 conn(0x7fa64c09aa70 msgr2=0x7fa64c09ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:32.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.140+0000 7fa652d14640 1 -- 172.21.15.43:0/235541301 shutdown_connections 2024-04-25T09:03:32.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.140+0000 7fa652d14640 1 -- 172.21.15.43:0/235541301 wait complete. 2024-04-25T09:03:32.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.142+0000 7fa652d14640 1 Processor -- start 2024-04-25T09:03:32.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.142+0000 7fa652d14640 1 -- start start 2024-04-25T09:03:32.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.142+0000 7fa652d14640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa64c0b4990 0x7fa64c1f30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.142+0000 7fa652d14640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa64c0b74e0 0x7fa64c0d07e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.142+0000 7fa652d14640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa64c15ba40 0x7fa64c0d0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.142+0000 7fa652d14640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c0d1c20 con 0x7fa64c15ba40 2024-04-25T09:03:32.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.142+0000 7fa652d14640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c0d1aa0 con 0x7fa64c0b4990 2024-04-25T09:03:32.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.142+0000 7fa652d14640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c0aee10 con 0x7fa64c0b74e0 2024-04-25T09:03:32.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.143+0000 7fa6512b1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa64c15ba40 0x7fa64c0d0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.143+0000 7fa6512b1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa64c15ba40 0x7fa64c0d0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51026/0 (socket says 172.21.15.43:51026) 2024-04-25T09:03:32.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.143+0000 7fa6512b1640 1 -- 172.21.15.43:0/1216339090 learned_addr learned my addr 172.21.15.43:0/1216339090 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:32.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.143+0000 7fa64bfff640 1 --2- 172.21.15.43:0/1216339090 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa64c0b74e0 0x7fa64c0d07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.143+0000 7fa650ab0640 1 --2- 172.21.15.43:0/1216339090 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa64c0b4990 0x7fa64c1f30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.143+0000 7fa6512b1640 1 -- 172.21.15.43:0/1216339090 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa64c0b74e0 msgr2=0x7fa64c0d07e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.143+0000 7fa6512b1640 1 --2- 172.21.15.43:0/1216339090 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa64c0b74e0 0x7fa64c0d07e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.143+0000 7fa6512b1640 1 -- 172.21.15.43:0/1216339090 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa64c0b4990 msgr2=0x7fa64c1f30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.143+0000 7fa6512b1640 1 --2- 172.21.15.43:0/1216339090 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa64c0b4990 0x7fa64c1f30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.143+0000 7fa6512b1640 1 -- 172.21.15.43:0/1216339090 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa64c0d1260 con 0x7fa64c15ba40 2024-04-25T09:03:32.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.143+0000 7fa6512b1640 1 --2- 172.21.15.43:0/1216339090 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa64c15ba40 0x7fa64c0d0d20 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fa63400c750 tx=0x7fa63400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:32.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.143+0000 7fa6497fa640 1 -- 172.21.15.43:0/1216339090 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa634018070 con 0x7fa64c15ba40 2024-04-25T09:03:32.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.143+0000 7fa6497fa640 1 -- 172.21.15.43:0/1216339090 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6340072d0 con 0x7fa64c15ba40 2024-04-25T09:03:32.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.143+0000 7fa6497fa640 1 -- 172.21.15.43:0/1216339090 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6340047e0 con 0x7fa64c15ba40 2024-04-25T09:03:32.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.143+0000 7fa652d14640 1 -- 172.21.15.43:0/1216339090 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa64c0cfa20 con 0x7fa64c15ba40 2024-04-25T09:03:32.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.143+0000 7fa652d14640 1 -- 172.21.15.43:0/1216339090 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa64c0cffe0 con 0x7fa64c15ba40 2024-04-25T09:03:32.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.145+0000 7fa6497fa640 1 -- 172.21.15.43:0/1216339090 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa63401b070 con 0x7fa64c15ba40 2024-04-25T09:03:32.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.145+0000 7fa6497fa640 1 --2- 172.21.15.43:0/1216339090 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa62c06b4d0 0x7fa62c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.145+0000 7fa6497fa640 1 -- 172.21.15.43:0/1216339090 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa634089540 con 0x7fa64c15ba40 2024-04-25T09:03:32.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.145+0000 7fa650ab0640 1 --2- 172.21.15.43:0/1216339090 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa62c06b4d0 0x7fa62c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.146+0000 7fa650ab0640 1 --2- 172.21.15.43:0/1216339090 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa62c06b4d0 0x7fa62c06d990 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fa644034000 tx=0x7fa64402f000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:32.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.146+0000 7fa655a18740 1 Processor -- start 2024-04-25T09:03:32.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.146+0000 7fa655a18740 1 -- start start 2024-04-25T09:03:32.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.147+0000 7fa655a18740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563a403f6510 0x563a403f68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.147+0000 7fa655a18740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a403c6770 0x563a403cc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.147+0000 7fa655a18740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a40412470 0x563a404128f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.147+0000 7fa655a18740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a4032fda0 con 0x563a40412470 2024-04-25T09:03:32.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.147+0000 7fa655a18740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a402d8b80 con 0x563a403c6770 2024-04-25T09:03:32.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.147+0000 7fa655a18740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x563a4032be40 con 0x563a403f6510 2024-04-25T09:03:32.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.147+0000 7fa6512b1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a40412470 0x563a404128f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.147+0000 7fa6512b1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a40412470 0x563a404128f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51034/0 (socket says 172.21.15.43:51034) 2024-04-25T09:03:32.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.147+0000 7fa6512b1640 1 -- 172.21.15.43:0/694742763 learned_addr learned my addr 172.21.15.43:0/694742763 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:32.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.147+0000 7fa650ab0640 1 --2- 172.21.15.43:0/694742763 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563a403f6510 0x563a403f68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.147+0000 7fa64bfff640 1 --2- 172.21.15.43:0/694742763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a403c6770 0x563a403cc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.147+0000 7fa6512b1640 1 -- 172.21.15.43:0/694742763 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563a403f6510 msgr2=0x563a403f68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:32.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.147+0000 7fa6512b1640 1 --2- 172.21.15.43:0/694742763 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563a403f6510 0x563a403f68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.147+0000 7fa6512b1640 1 -- 172.21.15.43:0/694742763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a403c6770 msgr2=0x563a403cc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:32.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.147+0000 7fa6512b1640 1 --2- 172.21.15.43:0/694742763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a403c6770 0x563a403cc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.147+0000 7fa6512b1640 1 -- 172.21.15.43:0/694742763 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a402e2f10 con 0x563a40412470 2024-04-25T09:03:32.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.147+0000 7fa6512b1640 1 --2- 172.21.15.43:0/694742763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a40412470 0x563a404128f0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fa634001330 tx=0x7fa634057d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9989c079df7abd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:32.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.147+0000 7fa6427fc640 1 -- 172.21.15.43:0/694742763 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63405e030 con 0x563a40412470 2024-04-25T09:03:32.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.147+0000 7fa6427fc640 1 -- 172.21.15.43:0/694742763 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa634056070 con 0x563a40412470 2024-04-25T09:03:32.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.147+0000 7fa6427fc640 1 -- 172.21.15.43:0/694742763 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6340890c0 con 0x563a40412470 2024-04-25T09:03:32.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.148+0000 7fa655a18740 1 -- 172.21.15.43:0/694742763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a40412470 msgr2=0x563a404128f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.148+0000 7fa655a18740 1 --2- 172.21.15.43:0/694742763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a40412470 0x563a404128f0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fa634001330 tx=0x7fa634057d70 comp rx=0 tx=0).stop 2024-04-25T09:03:32.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.148+0000 7fa655a18740 1 -- 172.21.15.43:0/694742763 shutdown_connections 2024-04-25T09:03:32.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.148+0000 7fa655a18740 1 --2- 172.21.15.43:0/694742763 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a40412470 0x563a404128f0 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.148+0000 7fa655a18740 1 --2- 172.21.15.43:0/694742763 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a403c6770 0x563a403cc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.148+0000 7fa655a18740 1 --2- 172.21.15.43:0/694742763 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563a403f6510 0x563a403f68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.148+0000 7fa655a18740 1 -- 172.21.15.43:0/694742763 >> 172.21.15.43:0/694742763 conn(0x563a403c89e0 msgr2=0x563a40411320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:32.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.148+0000 7fa655a18740 1 -- 172.21.15.43:0/694742763 shutdown_connections 2024-04-25T09:03:32.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.148+0000 7fa655a18740 1 -- 172.21.15.43:0/694742763 wait complete. 2024-04-25T09:03:32.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.149+0000 7fa655a18740 1 -- start start 2024-04-25T09:03:32.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.149+0000 7fa655a18740 1 Processor -- start 2024-04-25T09:03:32.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.150+0000 7fa655a18740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a403f6510 0x563a40413a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.150+0000 7fa655a18740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a40412470 0x563a403e0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.150+0000 7fa655a18740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563a40412890 0x563a40412d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.150+0000 7fa655a18740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a403ccce0 con 0x563a40412470 2024-04-25T09:03:32.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.150+0000 7fa655a18740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a402ce4b0 con 0x563a403f6510 2024-04-25T09:03:32.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.150+0000 7fa655a18740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x563a403cc310 con 0x563a40412890 2024-04-25T09:03:32.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.150+0000 7fa655a18740 10 client.0 _ll_register_callbacks cb 0x563a4030e180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:32.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.150+0000 7fa64bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a40412470 0x563a403e0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.150+0000 7fa64bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a40412470 0x563a403e0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51046/0 (socket says 172.21.15.43:51046) 2024-04-25T09:03:32.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.150+0000 7fa64bfff640 1 -- 172.21.15.43:0/1504973941 learned_addr learned my addr 172.21.15.43:0/1504973941 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:32.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.150+0000 7fa650ab0640 1 --2- 172.21.15.43:0/1504973941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a403f6510 0x563a40413a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.150+0000 7fa6512b1640 1 --2- 172.21.15.43:0/1504973941 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563a40412890 0x563a40412d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.150+0000 7fa64bfff640 1 -- 172.21.15.43:0/1504973941 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563a40412890 msgr2=0x563a40412d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:32.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.150+0000 7fa64bfff640 1 --2- 172.21.15.43:0/1504973941 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563a40412890 0x563a40412d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.150+0000 7fa64bfff640 1 -- 172.21.15.43:0/1504973941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a403f6510 msgr2=0x563a40413a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:32.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.150+0000 7fa64bfff640 1 --2- 172.21.15.43:0/1504973941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a403f6510 0x563a40413a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.150+0000 7fa64bfff640 1 -- 172.21.15.43:0/1504973941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563a4010e7d0 con 0x563a40412470 2024-04-25T09:03:32.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.150+0000 7fa64bfff640 1 --2- 172.21.15.43:0/1504973941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a40412470 0x563a403e0d90 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fa63c003cf0 tx=0x7fa63c001680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2da71887b3db1fcb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:32.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.150+0000 7fa62b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:03:32.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.150+0000 7fa62b7fe640 1 -- 172.21.15.43:0/1504973941 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63c015470 con 0x563a40412470 2024-04-25T09:03:32.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.150+0000 7fa62b7fe640 1 -- 172.21.15.43:0/1504973941 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa63c015610 con 0x563a40412470 2024-04-25T09:03:32.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.150+0000 7fa655a18740 1 -- 172.21.15.43:0/1504973941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa634056070 con 0x563a40412470 2024-04-25T09:03:32.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.151+0000 7fa62b7fe640 1 -- 172.21.15.43:0/1504973941 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa63c022050 con 0x563a40412470 2024-04-25T09:03:32.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.151+0000 7fa655a18740 10 client.9465 fetch_fsmap learned FSMap version 76 2024-04-25T09:03:32.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.151+0000 7fa655a18740 1 -- 172.21.15.43:0/1504973941 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x563a400443f0 con 0x563a40412470 2024-04-25T09:03:32.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.151+0000 7fa62b7fe640 1 -- 172.21.15.43:0/1504973941 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa634056070 con 0x563a40412470 2024-04-25T09:03:32.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.151+0000 7fa62b7fe640 1 -- 172.21.15.43:0/1504973941 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 76) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa63c025020 con 0x563a40412470 2024-04-25T09:03:32.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.151+0000 7fa655a18740 10 client.9465 fetch_fsmap finished waiting for FSMap version 76 2024-04-25T09:03:32.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.151+0000 7fa655a18740 10 client.9465 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.a 2024-04-25T09:03:32.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.151+0000 7fa655a18740 20 client.9465 populate_metadata read hostname 'smithi043' 2024-04-25T09:03:32.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.151+0000 7fa655a18740 1 --2- 172.21.15.43:0/1504973941 >> [v2:172.21.15.43:6832/2603892772,v1:172.21.15.43:6834/2603892772] conn(0x563a403f8c90 0x563a403f9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.151+0000 7fa655a18740 4 client.9465 mds_command: new command op to 9237 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:03:32.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.151+0000 7fa655a18740 1 -- 172.21.15.43:0/1504973941 --> [v2:172.21.15.43:6832/2603892772,v1:172.21.15.43:6834/2603892772] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563a4033d2e0 con 0x563a403f8c90 2024-04-25T09:03:32.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.151+0000 7fa650ab0640 1 --2- 172.21.15.43:0/1504973941 >> [v2:172.21.15.43:6832/2603892772,v1:172.21.15.43:6834/2603892772] conn(0x563a403f8c90 0x563a403f9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.151+0000 7fa62b7fe640 10 client.9465 ms_handle_connect on v2:172.21.15.43:6832/2603892772 2024-04-25T09:03:32.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.151+0000 7fa650ab0640 1 --2- 172.21.15.43:0/1504973941 >> [v2:172.21.15.43:6832/2603892772,v1:172.21.15.43:6834/2603892772] conn(0x563a403f8c90 0x563a403f9100 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f759d52e78123acc server_cookie=b3405f08413f9704 in_seq=0 out_seq=0 2024-04-25T09:03:32.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.152+0000 7fa62b7fe640 1 -- 172.21.15.43:0/1504973941 <== mds.0 v2:172.21.15.43:6832/2603892772 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563a4033d2e0 con 0x563a403f8c90 2024-04-25T09:03:32.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.152+0000 7fa62b7fe640 10 client.9465 handle_command_reply: tid=0 2024-04-25T09:03:32.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.152+0000 7fa655a18740 1 client.9465 shutdown 2024-04-25T09:03:32.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.152+0000 7fa655a18740 1 -- 172.21.15.43:0/1504973941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a40412470 msgr2=0x563a403e0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.152+0000 7fa655a18740 1 --2- 172.21.15.43:0/1504973941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a40412470 0x563a403e0d90 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fa63c003cf0 tx=0x7fa63c001680 comp rx=0 tx=0).stop 2024-04-25T09:03:32.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.152+0000 7fa655a18740 1 -- 172.21.15.43:0/1504973941 shutdown_connections 2024-04-25T09:03:32.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.152+0000 7fa655a18740 1 --2- 172.21.15.43:0/1504973941 >> [v2:172.21.15.43:6832/2603892772,v1:172.21.15.43:6834/2603892772] conn(0x563a403f8c90 0x563a403f9100 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.152+0000 7fa655a18740 1 --2- 172.21.15.43:0/1504973941 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563a40412890 0x563a40412d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.152+0000 7fa655a18740 1 --2- 172.21.15.43:0/1504973941 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a40412470 0x563a403e0d90 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.152+0000 7fa62b7fe640 0 client.9465 ms_handle_reset on v2:172.21.15.43:6832/2603892772 2024-04-25T09:03:32.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.152+0000 7fa655a18740 1 --2- 172.21.15.43:0/1504973941 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a403f6510 0x563a40413a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.152+0000 7fa655a18740 1 -- 172.21.15.43:0/1504973941 >> 172.21.15.43:0/1504973941 conn(0x563a403c6770 msgr2=0x563a403c5e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:32.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.152+0000 7fa655a18740 1 -- 172.21.15.43:0/1504973941 shutdown_connections 2024-04-25T09:03:32.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.152+0000 7fa655a18740 1 -- 172.21.15.43:0/1504973941 wait complete. 2024-04-25T09:03:32.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.152+0000 7fa655a18740 20 client.9465 trim_cache size 0 max 16384 2024-04-25T09:03:32.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.167+0000 7fa655a18740 1 Processor -- start 2024-04-25T09:03:32.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.167+0000 7fa655a18740 1 -- start start 2024-04-25T09:03:32.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.168+0000 7fa655a18740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a403f6510 0x563a403c7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.168+0000 7fa655a18740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a40412470 0x563a40412d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.168+0000 7fa655a18740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563a40412890 0x563a403f0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.168+0000 7fa655a18740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c1fb890 con 0x563a40412470 2024-04-25T09:03:32.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.168+0000 7fa655a18740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c06d750 con 0x563a403f6510 2024-04-25T09:03:32.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.168+0000 7fa655a18740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c1f29b0 con 0x563a40412890 2024-04-25T09:03:32.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.168+0000 7fa64bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a40412470 0x563a40412d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.168+0000 7fa64bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a40412470 0x563a40412d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51056/0 (socket says 172.21.15.43:51056) 2024-04-25T09:03:32.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.168+0000 7fa64bfff640 1 -- 172.21.15.43:0/2208430760 learned_addr learned my addr 172.21.15.43:0/2208430760 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:32.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.168+0000 7fa64bfff640 1 -- 172.21.15.43:0/2208430760 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563a40412890 msgr2=0x563a403f0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:32.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.168+0000 7fa650ab0640 1 --2- 172.21.15.43:0/2208430760 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a403f6510 0x563a403c7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.168+0000 7fa6512b1640 1 --2- 172.21.15.43:0/2208430760 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563a40412890 0x563a403f0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.168+0000 7fa64bfff640 1 --2- 172.21.15.43:0/2208430760 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563a40412890 0x563a403f0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.168+0000 7fa64bfff640 1 -- 172.21.15.43:0/2208430760 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a403f6510 msgr2=0x563a403c7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:32.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.168+0000 7fa64bfff640 1 --2- 172.21.15.43:0/2208430760 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a403f6510 0x563a403c7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.158 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.168+0000 7fa64bfff640 1 -- 172.21.15.43:0/2208430760 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a400040f0 con 0x563a40412470 2024-04-25T09:03:32.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.168+0000 7fa650ab0640 1 --2- 172.21.15.43:0/2208430760 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a403f6510 0x563a403c7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:32.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.168+0000 7fa6512b1640 1 --2- 172.21.15.43:0/2208430760 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563a40412890 0x563a403f0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:32.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.169+0000 7fa64bfff640 1 --2- 172.21.15.43:0/2208430760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a40412470 0x563a40412d10 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fa63c0050c0 tx=0x7fa63c00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0a166d8c64bda29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:32.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.169+0000 7fa641ffb640 1 -- 172.21.15.43:0/2208430760 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63c02a020 con 0x563a40412470 2024-04-25T09:03:32.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.169+0000 7fa641ffb640 1 -- 172.21.15.43:0/2208430760 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa63c01a070 con 0x563a40412470 2024-04-25T09:03:32.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.169+0000 7fa641ffb640 1 -- 172.21.15.43:0/2208430760 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63c015c70 con 0x563a40412470 2024-04-25T09:03:32.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.169+0000 7fa655a18740 1 -- 172.21.15.43:0/2208430760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a40412470 msgr2=0x563a40412d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.169+0000 7fa655a18740 1 --2- 172.21.15.43:0/2208430760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a40412470 0x563a40412d10 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fa63c0050c0 tx=0x7fa63c00f000 comp rx=0 tx=0).stop 2024-04-25T09:03:32.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.169+0000 7fa655a18740 1 -- 172.21.15.43:0/2208430760 shutdown_connections 2024-04-25T09:03:32.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.169+0000 7fa655a18740 1 --2- 172.21.15.43:0/2208430760 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563a40412890 0x563a403f0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.169+0000 7fa655a18740 1 --2- 172.21.15.43:0/2208430760 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a40412470 0x563a40412d10 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.169+0000 7fa655a18740 1 --2- 172.21.15.43:0/2208430760 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a403f6510 0x563a403c7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.169+0000 7fa655a18740 1 -- 172.21.15.43:0/2208430760 >> 172.21.15.43:0/2208430760 conn(0x563a403c6770 msgr2=0x563a403e0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:32.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.169+0000 7fa655a18740 1 -- 172.21.15.43:0/2208430760 shutdown_connections 2024-04-25T09:03:32.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.169+0000 7fa655a18740 1 -- 172.21.15.43:0/2208430760 wait complete. 2024-04-25T09:03:32.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.170+0000 7fa655a18740 1 -- start start 2024-04-25T09:03:32.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.170+0000 7fa655a18740 1 Processor -- start 2024-04-25T09:03:32.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.171+0000 7fa655a18740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a403f6510 0x563a4047f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.171+0000 7fa655a18740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563a40412470 0x563a403c5950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.171+0000 7fa655a18740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a40412890 0x563a403f9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.171+0000 7fa655a18740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a403ee860 con 0x563a40412890 2024-04-25T09:03:32.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.171+0000 7fa655a18740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa64c06d150 con 0x563a403f6510 2024-04-25T09:03:32.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.171+0000 7fa655a18740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x563a402f1520 con 0x563a40412470 2024-04-25T09:03:32.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.171+0000 7fa655a18740 10 client.0 _ll_register_callbacks cb 0x7fa63c007bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:03:32.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.171+0000 7fa6512b1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a40412890 0x563a403f9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.171+0000 7fa6512b1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a40412890 0x563a403f9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51066/0 (socket says 172.21.15.43:51066) 2024-04-25T09:03:32.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.171+0000 7fa6512b1640 1 -- 172.21.15.43:0/809059293 learned_addr learned my addr 172.21.15.43:0/809059293 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:32.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.171+0000 7fa64bfff640 1 --2- 172.21.15.43:0/809059293 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563a40412470 0x563a403c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.171+0000 7fa650ab0640 1 --2- 172.21.15.43:0/809059293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a403f6510 0x563a4047f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.171+0000 7fa6512b1640 1 -- 172.21.15.43:0/809059293 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563a40412470 msgr2=0x563a403c5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:32.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.171+0000 7fa6512b1640 1 --2- 172.21.15.43:0/809059293 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563a40412470 0x563a403c5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.171+0000 7fa6512b1640 1 -- 172.21.15.43:0/809059293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a403f6510 msgr2=0x563a4047f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:32.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.171+0000 7fa6512b1640 1 --2- 172.21.15.43:0/809059293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a403f6510 0x563a4047f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.171+0000 7fa6512b1640 1 -- 172.21.15.43:0/809059293 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563a40407ee0 con 0x563a40412890 2024-04-25T09:03:32.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.171+0000 7fa64bfff640 1 --2- 172.21.15.43:0/809059293 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563a40412470 0x563a403c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:32.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.171+0000 7fa6512b1640 1 --2- 172.21.15.43:0/809059293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a40412890 0x563a403f9c40 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fa63400a430 tx=0x7fa634000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52a841807e290c07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:32.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.171+0000 7fa62b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:03:32.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.172+0000 7fa62b7fe640 1 -- 172.21.15.43:0/809059293 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6340892c0 con 0x563a40412890 2024-04-25T09:03:32.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.172+0000 7fa62b7fe640 1 -- 172.21.15.43:0/809059293 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa6340571c0 con 0x563a40412890 2024-04-25T09:03:32.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.172+0000 7fa655a18740 1 -- 172.21.15.43:0/809059293 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa63c01a070 con 0x563a40412890 2024-04-25T09:03:32.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.172+0000 7fa62b7fe640 1 -- 172.21.15.43:0/809059293 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa63405e0f0 con 0x563a40412890 2024-04-25T09:03:32.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.172+0000 7fa655a18740 10 client.9471 fetch_fsmap learned FSMap version 76 2024-04-25T09:03:32.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.172+0000 7fa655a18740 1 -- 172.21.15.43:0/809059293 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x563a403cc700 con 0x563a40412890 2024-04-25T09:03:32.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.172+0000 7fa62b7fe640 1 -- 172.21.15.43:0/809059293 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa63c01a070 con 0x563a40412890 2024-04-25T09:03:32.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.172+0000 7fa62b7fe640 1 -- 172.21.15.43:0/809059293 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 76) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa634076b90 con 0x563a40412890 2024-04-25T09:03:32.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.172+0000 7fa655a18740 10 client.9471 fetch_fsmap finished waiting for FSMap version 76 2024-04-25T09:03:32.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.172+0000 7fa655a18740 10 client.9471 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.a 2024-04-25T09:03:32.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.172+0000 7fa655a18740 20 client.9471 populate_metadata read hostname 'smithi043' 2024-04-25T09:03:32.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.172+0000 7fa655a18740 1 --2- 172.21.15.43:0/809059293 >> [v2:172.21.15.43:6832/2603892772,v1:172.21.15.43:6834/2603892772] conn(0x563a403e0bf0 0x563a403fa180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.172+0000 7fa655a18740 4 client.9471 mds_command: new command op to 9237 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T09:03:32.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.172+0000 7fa655a18740 1 -- 172.21.15.43:0/809059293 --> [v2:172.21.15.43:6832/2603892772,v1:172.21.15.43:6834/2603892772] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563a40337630 con 0x563a403e0bf0 2024-04-25T09:03:32.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.172+0000 7fa650ab0640 1 --2- 172.21.15.43:0/809059293 >> [v2:172.21.15.43:6832/2603892772,v1:172.21.15.43:6834/2603892772] conn(0x563a403e0bf0 0x563a403fa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.172+0000 7fa62b7fe640 10 client.9471 ms_handle_connect on v2:172.21.15.43:6832/2603892772 2024-04-25T09:03:32.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.172+0000 7fa650ab0640 1 --2- 172.21.15.43:0/809059293 >> [v2:172.21.15.43:6832/2603892772,v1:172.21.15.43:6834/2603892772] conn(0x563a403e0bf0 0x563a403fa180 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c765fcdcd3f80fd server_cookie=dacdf6f224eaea87 in_seq=0 out_seq=0 2024-04-25T09:03:32.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.173+0000 7fa62b7fe640 1 -- 172.21.15.43:0/809059293 <== mds.0 v2:172.21.15.43:6832/2603892772 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x563a40337630 con 0x563a403e0bf0 2024-04-25T09:03:32.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.173+0000 7fa62b7fe640 10 client.9471 handle_command_reply: tid=0 2024-04-25T09:03:32.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.173+0000 7fa655a18740 1 client.9471 shutdown 2024-04-25T09:03:32.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.173+0000 7fa655a18740 1 -- 172.21.15.43:0/809059293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a40412890 msgr2=0x563a403f9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.173+0000 7fa655a18740 1 --2- 172.21.15.43:0/809059293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a40412890 0x563a403f9c40 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fa63400a430 tx=0x7fa634000ed0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.173+0000 7fa655a18740 1 -- 172.21.15.43:0/809059293 shutdown_connections 2024-04-25T09:03:32.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.173+0000 7fa655a18740 1 --2- 172.21.15.43:0/809059293 >> [v2:172.21.15.43:6832/2603892772,v1:172.21.15.43:6834/2603892772] conn(0x563a403e0bf0 0x563a403fa180 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.173+0000 7fa655a18740 1 --2- 172.21.15.43:0/809059293 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x563a40412890 0x563a403f9c40 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.173+0000 7fa655a18740 1 --2- 172.21.15.43:0/809059293 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x563a40412470 0x563a403c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.173+0000 7fa62b7fe640 0 client.9471 ms_handle_reset on v2:172.21.15.43:6832/2603892772 2024-04-25T09:03:32.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.173+0000 7fa655a18740 1 --2- 172.21.15.43:0/809059293 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x563a403f6510 0x563a4047f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.173+0000 7fa655a18740 1 -- 172.21.15.43:0/809059293 >> 172.21.15.43:0/809059293 conn(0x563a403c6770 msgr2=0x563a403f7fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:32.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.173+0000 7fa655a18740 1 -- 172.21.15.43:0/809059293 shutdown_connections 2024-04-25T09:03:32.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.173+0000 7fa655a18740 1 -- 172.21.15.43:0/809059293 wait complete. 2024-04-25T09:03:32.167 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.173+0000 7fa655a18740 20 client.9471 trim_cache size 0 max 16384 2024-04-25T09:03:32.167 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:03:32.168 INFO:teuthology.orchestra.run.smithi043.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T09:03:32.168 INFO:teuthology.orchestra.run.smithi043.stdout: "scrubs": { 2024-04-25T09:03:32.168 INFO:teuthology.orchestra.run.smithi043.stdout: "c7b584d3-21e2-402b-92bd-e2f8f109206e": { 2024-04-25T09:03:32.168 INFO:teuthology.orchestra.run.smithi043.stdout: "path": "/scrub_control_test_path", 2024-04-25T09:03:32.168 INFO:teuthology.orchestra.run.smithi043.stdout: "tag": "c7b584d3-21e2-402b-92bd-e2f8f109206e", 2024-04-25T09:03:32.168 INFO:teuthology.orchestra.run.smithi043.stdout: "options": "recursive" 2024-04-25T09:03:32.168 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:03:32.168 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:03:32.168 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:03:32.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.173+0000 7fa652d14640 1 -- 172.21.15.43:0/1216339090 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa62c06b4d0 msgr2=0x7fa62c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.173+0000 7fa652d14640 1 --2- 172.21.15.43:0/1216339090 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa62c06b4d0 0x7fa62c06d990 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fa644034000 tx=0x7fa64402f000 comp rx=0 tx=0).stop 2024-04-25T09:03:32.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.173+0000 7fa652d14640 1 -- 172.21.15.43:0/1216339090 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa64c15ba40 msgr2=0x7fa64c0d0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.173+0000 7fa652d14640 1 --2- 172.21.15.43:0/1216339090 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa64c15ba40 0x7fa64c0d0d20 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fa63400c750 tx=0x7fa63400cc10 comp rx=0 tx=0).stop 2024-04-25T09:03:32.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.174+0000 7fa652d14640 1 -- 172.21.15.43:0/1216339090 shutdown_connections 2024-04-25T09:03:32.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.174+0000 7fa652d14640 1 --2- 172.21.15.43:0/1216339090 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa64c15ba40 0x7fa64c0d0d20 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.174+0000 7fa652d14640 1 --2- 172.21.15.43:0/1216339090 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa64c0b74e0 0x7fa64c0d07e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.174+0000 7fa652d14640 1 --2- 172.21.15.43:0/1216339090 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa62c06b4d0 0x7fa62c06d990 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.174+0000 7fa652d14640 1 --2- 172.21.15.43:0/1216339090 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa64c0b4990 0x7fa64c1f30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.174+0000 7fa652d14640 1 -- 172.21.15.43:0/1216339090 >> 172.21.15.43:0/1216339090 conn(0x7fa64c09aa70 msgr2=0x7fa64c1507f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:32.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.174+0000 7fa652d14640 1 -- 172.21.15.43:0/1216339090 shutdown_connections 2024-04-25T09:03:32.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.174+0000 7fa652d14640 1 -- 172.21.15.43:0/1216339090 wait complete. 2024-04-25T09:03:32.172 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:32.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.284+0000 7fb9e604a640 1 Processor -- start 2024-04-25T09:03:32.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.284+0000 7fb9e604a640 1 -- start start 2024-04-25T09:03:32.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.285+0000 7fb9e604a640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb9e015dce0 0x7fb9e015ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.285+0000 7fb9e604a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb9e00ce550 0x7fb9e00d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.285+0000 7fb9e604a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9e00cee80 con 0x7fb9e00ce9c0 2024-04-25T09:03:32.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.285+0000 7fb9e604a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9e00d2920 con 0x7fb9e00ce550 2024-04-25T09:03:32.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.285+0000 7fb9e604a640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9e00cf160 con 0x7fb9e015dce0 2024-04-25T09:03:32.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.285+0000 7fb9deffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb9e00ce550 0x7fb9e00d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.285+0000 7fb9deffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb9e00ce550 0x7fb9e00d03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51078/0 (socket says 172.21.15.43:51078) 2024-04-25T09:03:32.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.285+0000 7fb9deffd640 1 -- 172.21.15.43:0/1964358568 learned_addr learned my addr 172.21.15.43:0/1964358568 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:32.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.285+0000 7fb9df7fe640 1 --2- 172.21.15.43:0/1964358568 >> v2:172.21.15.193:3300/0 conn(0x7fb9e015dce0 0x7fb9e015ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.286+0000 7fb9deffd640 1 -- 172.21.15.43:0/1964358568 >> v2:172.21.15.193:3300/0 conn(0x7fb9e015dce0 msgr2=0x7fb9e015ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:32.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.286+0000 7fb9deffd640 1 --2- 172.21.15.43:0/1964358568 >> v2:172.21.15.193:3300/0 conn(0x7fb9e015dce0 0x7fb9e015ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.286+0000 7fb9deffd640 1 -- 172.21.15.43:0/1964358568 >> v1:172.21.15.43:6789/0 conn(0x7fb9e00ce9c0 legacy=0x7fb9e0160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.286+0000 7fb9deffd640 1 -- 172.21.15.43:0/1964358568 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e00cf440 con 0x7fb9e00ce550 2024-04-25T09:03:32.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.286+0000 7fb9de7fc640 1 -- 172.21.15.43:0/1964358568 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1242040142 0 0) 0x7fb9e00cee80 con 0x7fb9e00ce9c0 2024-04-25T09:03:32.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.286+0000 7fb9deffd640 1 --2- 172.21.15.43:0/1964358568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9e00ce550 0x7fb9e00d03e0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fb9e009d3c0 tx=0x7fb9d402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=666631d7650d1e6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:32.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.287+0000 7fb9de7fc640 1 -- 172.21.15.43:0/1964358568 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d4037070 con 0x7fb9e00ce550 2024-04-25T09:03:32.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.287+0000 7fb9de7fc640 1 -- 172.21.15.43:0/1964358568 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d4032440 con 0x7fb9e00ce550 2024-04-25T09:03:32.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.287+0000 7fb9de7fc640 1 -- 172.21.15.43:0/1964358568 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d4032810 con 0x7fb9e00ce550 2024-04-25T09:03:32.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.287+0000 7fb9e604a640 1 -- 172.21.15.43:0/1964358568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9e00ce550 msgr2=0x7fb9e00d03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.287+0000 7fb9e604a640 1 --2- 172.21.15.43:0/1964358568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9e00ce550 0x7fb9e00d03e0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fb9e009d3c0 tx=0x7fb9d402a1e0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.287+0000 7fb9e604a640 1 -- 172.21.15.43:0/1964358568 shutdown_connections 2024-04-25T09:03:32.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.287+0000 7fb9e604a640 1 --2- 172.21.15.43:0/1964358568 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9e00ce550 0x7fb9e00d03e0 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.287+0000 7fb9e604a640 1 --2- 172.21.15.43:0/1964358568 >> v2:172.21.15.193:3300/0 conn(0x7fb9e015dce0 0x7fb9e015ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.287+0000 7fb9e604a640 1 -- 172.21.15.43:0/1964358568 >> 172.21.15.43:0/1964358568 conn(0x7fb9e009cd00 msgr2=0x7fb9e00d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:32.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.287+0000 7fb9e604a640 1 -- 172.21.15.43:0/1964358568 shutdown_connections 2024-04-25T09:03:32.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.287+0000 7fb9e604a640 1 -- 172.21.15.43:0/1964358568 wait complete. 2024-04-25T09:03:32.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.288+0000 7fb9e604a640 1 Processor -- start 2024-04-25T09:03:32.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.289+0000 7fb9e604a640 1 -- start start 2024-04-25T09:03:32.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.289+0000 7fb9e604a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9e00ce550 0x7fb9e0153450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.289+0000 7fb9e604a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9e00ce9c0 0x7fb9e0153990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.289+0000 7fb9e604a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9e015dce0 0x7fb9e0153ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.289+0000 7fb9e604a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e015e0c0 con 0x7fb9e00ce550 2024-04-25T09:03:32.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.289+0000 7fb9e604a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e015f6b0 con 0x7fb9e00ce9c0 2024-04-25T09:03:32.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.289+0000 7fb9e604a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e00d0920 con 0x7fb9e015dce0 2024-04-25T09:03:32.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.289+0000 7fb9df7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9e00ce550 0x7fb9e0153450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.289+0000 7fb9df7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9e00ce550 0x7fb9e0153450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51082/0 (socket says 172.21.15.43:51082) 2024-04-25T09:03:32.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.289+0000 7fb9df7fe640 1 -- 172.21.15.43:0/887765713 learned_addr learned my addr 172.21.15.43:0/887765713 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:32.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.290+0000 7fb9deffd640 1 --2- 172.21.15.43:0/887765713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9e00ce9c0 0x7fb9e0153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.290+0000 7fb9df7fe640 1 -- 172.21.15.43:0/887765713 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9e015dce0 msgr2=0x7fb9e0153ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.290+0000 7fb9df7fe640 1 --2- 172.21.15.43:0/887765713 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9e015dce0 0x7fb9e0153ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.290+0000 7fb9df7fe640 1 -- 172.21.15.43:0/887765713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9e00ce9c0 msgr2=0x7fb9e0153990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.290+0000 7fb9df7fe640 1 --2- 172.21.15.43:0/887765713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9e00ce9c0 0x7fb9e0153990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.290+0000 7fb9df7fe640 1 -- 172.21.15.43:0/887765713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e0154850 con 0x7fb9e00ce550 2024-04-25T09:03:32.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.290+0000 7fb9deffd640 1 --2- 172.21.15.43:0/887765713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9e00ce9c0 0x7fb9e0153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:32.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.290+0000 7fb9df7fe640 1 --2- 172.21.15.43:0/887765713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9e00ce550 0x7fb9e0153450 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fb9d000e870 tx=0x7fb9d000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:32.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.290+0000 7fb9c7fff640 1 -- 172.21.15.43:0/887765713 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d0017070 con 0x7fb9e00ce550 2024-04-25T09:03:32.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.290+0000 7fb9c7fff640 1 -- 172.21.15.43:0/887765713 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d000c940 con 0x7fb9e00ce550 2024-04-25T09:03:32.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.290+0000 7fb9e604a640 1 -- 172.21.15.43:0/887765713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9e0154b30 con 0x7fb9e00ce550 2024-04-25T09:03:32.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.290+0000 7fb9c7fff640 1 -- 172.21.15.43:0/887765713 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d000cd30 con 0x7fb9e00ce550 2024-04-25T09:03:32.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.290+0000 7fb9e604a640 1 -- 172.21.15.43:0/887765713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9e01550f0 con 0x7fb9e00ce550 2024-04-25T09:03:32.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.291+0000 7fb9e604a640 1 -- 172.21.15.43:0/887765713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9e00680f0 con 0x7fb9e00ce550 2024-04-25T09:03:32.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.291+0000 7fb9c7fff640 1 -- 172.21.15.43:0/887765713 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb9d001a070 con 0x7fb9e00ce550 2024-04-25T09:03:32.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.292+0000 7fb9c7fff640 1 --2- 172.21.15.43:0/887765713 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb9cc06b4d0 0x7fb9cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.292+0000 7fb9c7fff640 1 -- 172.21.15.43:0/887765713 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb9d008a0c0 con 0x7fb9e00ce550 2024-04-25T09:03:32.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.292+0000 7fb9deffd640 1 --2- 172.21.15.43:0/887765713 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb9cc06b4d0 0x7fb9cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.292+0000 7fb9deffd640 1 --2- 172.21.15.43:0/887765713 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb9cc06b4d0 0x7fb9cc06d990 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4034000 tx=0x7fb9d402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:32.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.296+0000 7fb9c7fff640 1 -- 172.21.15.43:0/887765713 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb9d0058f40 con 0x7fb9e00ce550 2024-04-25T09:03:32.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.515+0000 7fb9e604a640 1 -- 172.21.15.43:0/887765713 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9e0074ef0 con 0x7fb9e00ce550 2024-04-25T09:03:32.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.516+0000 7fb9c7fff640 1 -- 172.21.15.43:0/887765713 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fb9d0058840 con 0x7fb9e00ce550 2024-04-25T09:03:32.503 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:03:32.504 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/891225068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":891225068},{"type":"v1","addr":"172.21.15.193:6836","nonce":891225068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":7175,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3679059390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":3679059390},{"type":"v1","addr":"172.21.15.193:6837","nonce":3679059390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":7948,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2127363183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2127363183},{"type":"v1","addr":"172.21.15.43:6835","nonce":2127363183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:11.919337+0000","modified":"2024-04-25T09:03:13.917768+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9237},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9237":{"gid":9237,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6834/2603892772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2603892772},{"type":"v1","addr":"172.21.15.43:6834","nonce":2603892772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:03:32.504 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 76 2024-04-25T09:03:32.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.518+0000 7fb9e604a640 1 -- 172.21.15.43:0/887765713 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb9cc06b4d0 msgr2=0x7fb9cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.518+0000 7fb9e604a640 1 --2- 172.21.15.43:0/887765713 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb9cc06b4d0 0x7fb9cc06d990 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fb9d4034000 tx=0x7fb9d402e000 comp rx=0 tx=0).stop 2024-04-25T09:03:32.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.518+0000 7fb9e604a640 1 -- 172.21.15.43:0/887765713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9e00ce550 msgr2=0x7fb9e0153450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.518+0000 7fb9e604a640 1 --2- 172.21.15.43:0/887765713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9e00ce550 0x7fb9e0153450 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fb9d000e870 tx=0x7fb9d000ed30 comp rx=0 tx=0).stop 2024-04-25T09:03:32.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.518+0000 7fb9e604a640 1 -- 172.21.15.43:0/887765713 shutdown_connections 2024-04-25T09:03:32.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.518+0000 7fb9e604a640 1 --2- 172.21.15.43:0/887765713 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb9e015dce0 0x7fb9e0153ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.518+0000 7fb9e604a640 1 --2- 172.21.15.43:0/887765713 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb9cc06b4d0 0x7fb9cc06d990 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.518+0000 7fb9e604a640 1 --2- 172.21.15.43:0/887765713 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb9e00ce9c0 0x7fb9e0153990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.518+0000 7fb9e604a640 1 --2- 172.21.15.43:0/887765713 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb9e00ce550 0x7fb9e0153450 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.518+0000 7fb9e604a640 1 -- 172.21.15.43:0/887765713 >> 172.21.15.43:0/887765713 conn(0x7fb9e009cd00 msgr2=0x7fb9e006c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:32.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.518+0000 7fb9e604a640 1 -- 172.21.15.43:0/887765713 shutdown_connections 2024-04-25T09:03:32.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.518+0000 7fb9e604a640 1 -- 172.21.15.43:0/887765713 wait complete. 2024-04-25T09:03:32.517 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T09:03:32.517 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:03:32.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.646+0000 7f44a8534640 1 Processor -- start 2024-04-25T09:03:32.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.646+0000 7f44a8534640 1 -- start start 2024-04-25T09:03:32.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.647+0000 7f44a8534640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44a015f160 0x7f44a00d23a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.647+0000 7f44a8534640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44a00b2db0 con 0x7f44a015f540 2024-04-25T09:03:32.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.647+0000 7f44a8534640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44a00b3090 con 0x7f44a00b0800 2024-04-25T09:03:32.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.647+0000 7f44a8534640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a00b2b60 con 0x7f44a015f160 2024-04-25T09:03:32.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.647+0000 7f44a5acf640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f44a015f540 0x7f44a00b1180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:37448/0 (socket says 172.21.15.43:37448) 2024-04-25T09:03:32.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.647+0000 7f44a5acf640 1 -- 172.21.15.43:0/32185749 learned_addr learned my addr 172.21.15.43:0/32185749 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:03:32.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.647+0000 7f44a62d0640 1 --2- 172.21.15.43:0/32185749 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44a015f160 0x7f44a00d23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.647+0000 7f44a52ce640 1 -- 172.21.15.43:0/32185749 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3276583650 0 0) 0x7f44a00b2db0 con 0x7f44a015f540 2024-04-25T09:03:32.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.647+0000 7f44a52ce640 1 -- 172.21.15.43:0/32185749 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f448c003520 con 0x7f44a015f540 2024-04-25T09:03:32.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.647+0000 7f44a52ce640 1 -- 172.21.15.43:0/32185749 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2565376331 0 0) 0x7f44a00b3090 con 0x7f44a00b0800 2024-04-25T09:03:32.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.647+0000 7f44a52ce640 1 -- 172.21.15.43:0/32185749 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44a00b2db0 con 0x7f44a00b0800 2024-04-25T09:03:32.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.648+0000 7f44a62d0640 1 -- 172.21.15.43:0/32185749 >> v1:172.21.15.193:6789/0 conn(0x7f44a00b0800 legacy=0x7f44a0165a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.648+0000 7f44a62d0640 1 -- 172.21.15.43:0/32185749 >> v1:172.21.15.43:6789/0 conn(0x7f44a015f540 legacy=0x7f44a00b1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.637 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.648+0000 7f44a62d0640 1 -- 172.21.15.43:0/32185749 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a00b1f90 con 0x7f44a015f160 2024-04-25T09:03:32.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.649+0000 7f44a62d0640 1 --2- 172.21.15.43:0/32185749 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44a015f160 0x7f44a00d23a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f449c0097d0 tx=0x7f449c02a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=554fc673fb9e17a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:32.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.649+0000 7f44a52ce640 1 -- 172.21.15.43:0/32185749 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449c037070 con 0x7f44a015f160 2024-04-25T09:03:32.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.649+0000 7f44a52ce640 1 -- 172.21.15.43:0/32185749 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f449c02ae50 con 0x7f44a015f160 2024-04-25T09:03:32.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.649+0000 7f44a52ce640 1 -- 172.21.15.43:0/32185749 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449c032660 con 0x7f44a015f160 2024-04-25T09:03:32.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.650+0000 7f44a8534640 1 -- 172.21.15.43:0/32185749 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44a015f160 msgr2=0x7f44a00d23a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.650+0000 7f44a8534640 1 --2- 172.21.15.43:0/32185749 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44a015f160 0x7f44a00d23a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f449c0097d0 tx=0x7f449c02a100 comp rx=0 tx=0).stop 2024-04-25T09:03:32.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.650+0000 7f44a8534640 1 -- 172.21.15.43:0/32185749 shutdown_connections 2024-04-25T09:03:32.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.650+0000 7f44a8534640 1 --2- 172.21.15.43:0/32185749 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44a015f160 0x7f44a00d23a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.650+0000 7f44a8534640 1 -- 172.21.15.43:0/32185749 >> 172.21.15.43:0/32185749 conn(0x7f44a00d1880 msgr2=0x7f44a00d1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:32.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.650+0000 7f44a8534640 1 -- 172.21.15.43:0/32185749 shutdown_connections 2024-04-25T09:03:32.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.650+0000 7f44a8534640 1 -- 172.21.15.43:0/32185749 wait complete. 2024-04-25T09:03:32.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.651+0000 7f44a8534640 1 Processor -- start 2024-04-25T09:03:32.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.651+0000 7f44a8534640 1 -- start start 2024-04-25T09:03:32.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.652+0000 7f44a8534640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44a00b0800 0x7f44a00ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.652+0000 7f44a8534640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44a015f160 0x7f44a014b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.652+0000 7f44a8534640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44a015f540 0x7f44a0150a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.652+0000 7f44a8534640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a00b2910 con 0x7f44a015f160 2024-04-25T09:03:32.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.652+0000 7f44a8534640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a00b2790 con 0x7f44a00b0800 2024-04-25T09:03:32.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.652+0000 7f44a8534640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a00b1890 con 0x7f44a015f540 2024-04-25T09:03:32.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.652+0000 7f44a5acf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44a015f160 0x7f44a014b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.652+0000 7f44a5acf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44a015f160 0x7f44a014b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51094/0 (socket says 172.21.15.43:51094) 2024-04-25T09:03:32.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.652+0000 7f44a5acf640 1 -- 172.21.15.43:0/3575042644 learned_addr learned my addr 172.21.15.43:0/3575042644 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:32.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.652+0000 7f44a62d0640 1 --2- 172.21.15.43:0/3575042644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44a00b0800 0x7f44a00ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.652+0000 7f44a6ad1640 1 --2- 172.21.15.43:0/3575042644 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44a015f540 0x7f44a0150a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.653+0000 7f44a5acf640 1 -- 172.21.15.43:0/3575042644 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44a015f540 msgr2=0x7f44a0150a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.653+0000 7f44a5acf640 1 --2- 172.21.15.43:0/3575042644 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44a015f540 0x7f44a0150a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.653+0000 7f44a5acf640 1 -- 172.21.15.43:0/3575042644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44a00b0800 msgr2=0x7f44a00ce2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.653+0000 7f44a5acf640 1 --2- 172.21.15.43:0/3575042644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44a00b0800 0x7f44a00ce2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.653+0000 7f44a5acf640 1 -- 172.21.15.43:0/3575042644 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a00cce90 con 0x7f44a015f160 2024-04-25T09:03:32.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.653+0000 7f44a62d0640 1 --2- 172.21.15.43:0/3575042644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44a00b0800 0x7f44a00ce2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:32.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.653+0000 7f44a5acf640 1 --2- 172.21.15.43:0/3575042644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44a015f160 0x7f44a014b870 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f449000c3e0 tx=0x7f449000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:32.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.653+0000 7f4496ffd640 1 -- 172.21.15.43:0/3575042644 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4490017070 con 0x7f44a015f160 2024-04-25T09:03:32.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.653+0000 7f44a8534640 1 -- 172.21.15.43:0/3575042644 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44a014bdb0 con 0x7f44a015f160 2024-04-25T09:03:32.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.653+0000 7f4496ffd640 1 -- 172.21.15.43:0/3575042644 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4490005ce0 con 0x7f44a015f160 2024-04-25T09:03:32.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.653+0000 7f4496ffd640 1 -- 172.21.15.43:0/3575042644 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44900072f0 con 0x7f44a015f160 2024-04-25T09:03:32.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.653+0000 7f44a8534640 1 -- 172.21.15.43:0/3575042644 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44a00ceca0 con 0x7f44a015f160 2024-04-25T09:03:32.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.654+0000 7f44a8534640 1 -- 172.21.15.43:0/3575042644 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44a0074ef0 con 0x7f44a015f160 2024-04-25T09:03:32.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.655+0000 7f4496ffd640 1 -- 172.21.15.43:0/3575042644 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f449001a070 con 0x7f44a015f160 2024-04-25T09:03:32.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.656+0000 7f4496ffd640 1 --2- 172.21.15.43:0/3575042644 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f448406b4d0 0x7f448406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:32.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.656+0000 7f4496ffd640 1 -- 172.21.15.43:0/3575042644 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4490089580 con 0x7f44a015f160 2024-04-25T09:03:32.644 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.656+0000 7f44a62d0640 1 --2- 172.21.15.43:0/3575042644 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f448406b4d0 0x7f448406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:32.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.657+0000 7f44a62d0640 1 --2- 172.21.15.43:0/3575042644 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f448406b4d0 0x7f448406d990 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f449c034000 tx=0x7f449c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:32.652 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.664+0000 7f4496ffd640 1 -- 172.21.15.43:0/3575042644 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4490058490 con 0x7f44a015f160 2024-04-25T09:03:32.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.927+0000 7f44a8534640 1 -- 172.21.15.43:0/3575042644 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f44a00d01e0 con 0x7f44a015f160 2024-04-25T09:03:32.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.927+0000 7f4496ffd640 1 -- 172.21.15.43:0/3575042644 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1573 (secure 0 0 0) 0x7f4490057d90 con 0x7f44a015f160 2024-04-25T09:03:32.916 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:03:32.916 INFO:teuthology.orchestra.run.smithi043.stdout:{"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":585,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":126,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035238,"num_in_osds":8,"osd_in_since":1714035229,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":800,"data_bytes":816170123,"bytes_used":2087485440,"bytes_avail":771006627840,"bytes_total":773094113280,"write_bytes_sec":67955889,"read_op_per_sec":0,"write_op_per_sec":67},"fsmap":{"epoch":76,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"a","status":"up:active","gid":9237}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":22,"modified":"2024-04-25T09:02:53.815110+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:03:32.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.929+0000 7f44a8534640 1 -- 172.21.15.43:0/3575042644 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f448406b4d0 msgr2=0x7f448406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.929+0000 7f44a8534640 1 --2- 172.21.15.43:0/3575042644 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f448406b4d0 0x7f448406d990 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f449c034000 tx=0x7f449c02e000 comp rx=0 tx=0).stop 2024-04-25T09:03:32.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.929+0000 7f44a8534640 1 -- 172.21.15.43:0/3575042644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44a015f160 msgr2=0x7f44a014b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:32.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.929+0000 7f44a8534640 1 --2- 172.21.15.43:0/3575042644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44a015f160 0x7f44a014b870 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f449000c3e0 tx=0x7f449000c8a0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.929+0000 7f44a8534640 1 -- 172.21.15.43:0/3575042644 shutdown_connections 2024-04-25T09:03:32.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.929+0000 7f44a8534640 1 --2- 172.21.15.43:0/3575042644 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f44a015f540 0x7f44a0150a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.929+0000 7f44a8534640 1 --2- 172.21.15.43:0/3575042644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f44a015f160 0x7f44a014b870 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.929+0000 7f44a8534640 1 --2- 172.21.15.43:0/3575042644 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f448406b4d0 0x7f448406d990 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.929+0000 7f44a8534640 1 --2- 172.21.15.43:0/3575042644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f44a00b0800 0x7f44a00ce2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:32.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.929+0000 7f44a8534640 1 -- 172.21.15.43:0/3575042644 >> 172.21.15.43:0/3575042644 conn(0x7f44a00d1880 msgr2=0x7f44a006fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:32.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.929+0000 7f44a8534640 1 -- 172.21.15.43:0/3575042644 shutdown_connections 2024-04-25T09:03:32.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:32.929+0000 7f44a8534640 1 -- 172.21.15.43:0/3575042644 wait complete. 2024-04-25T09:03:32.928 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T09:03:33.929 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:34.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.053+0000 7f0a88b9e640 1 Processor -- start 2024-04-25T09:03:34.042 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.053+0000 7f0a88b9e640 1 -- start start 2024-04-25T09:03:34.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.054+0000 7f0a88b9e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0a8415dcf0 0x7f0a8406e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.054+0000 7f0a88b9e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0a8406ebf0 0x7f0a8406f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.054+0000 7f0a88b9e640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a840d0d80 con 0x7f0a841616e0 2024-04-25T09:03:34.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.054+0000 7f0a88b9e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a8406f590 con 0x7f0a8415dcf0 2024-04-25T09:03:34.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.054+0000 7f0a88b9e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8406c050 con 0x7f0a8406ebf0 2024-04-25T09:03:34.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.054+0000 7f0a8259c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0a8415dcf0 0x7f0a8406e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.054+0000 7f0a8259c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f0a8415dcf0 0x7f0a8406e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51102/0 (socket says 172.21.15.43:51102) 2024-04-25T09:03:34.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.054+0000 7f0a8259c640 1 -- 172.21.15.43:0/3722417688 learned_addr learned my addr 172.21.15.43:0/3722417688 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:34.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.054+0000 7f0a7bfff640 1 --2- 172.21.15.43:0/3722417688 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0a8406ebf0 0x7f0a8406f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.054+0000 7f0a8259c640 1 -- 172.21.15.43:0/3722417688 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0a8406ebf0 msgr2=0x7f0a8406f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:34.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.054+0000 7f0a8259c640 1 --2- 172.21.15.43:0/3722417688 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0a8406ebf0 0x7f0a8406f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.055+0000 7f0a8259c640 1 -- 172.21.15.43:0/3722417688 >> v1:172.21.15.193:6789/0 conn(0x7f0a841616e0 legacy=0x7f0a84161b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.055+0000 7f0a8259c640 1 -- 172.21.15.43:0/3722417688 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a840d04e0 con 0x7f0a8415dcf0 2024-04-25T09:03:34.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.055+0000 7f0a8259c640 1 --2- 172.21.15.43:0/3722417688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a8415dcf0 0x7f0a8406e530 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f0a74009870 tx=0x7f0a7402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e899aea3e3736a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:34.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.055+0000 7f0a81d9b640 1 -- 172.21.15.43:0/3722417688 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a74037070 con 0x7f0a8415dcf0 2024-04-25T09:03:34.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.055+0000 7f0a81d9b640 1 -- 172.21.15.43:0/3722417688 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a74032440 con 0x7f0a8415dcf0 2024-04-25T09:03:34.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.056+0000 7f0a81d9b640 1 -- 172.21.15.43:0/3722417688 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a74032810 con 0x7f0a8415dcf0 2024-04-25T09:03:34.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.056+0000 7f0a88b9e640 1 -- 172.21.15.43:0/3722417688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a8415dcf0 msgr2=0x7f0a8406e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.056+0000 7f0a88b9e640 1 --2- 172.21.15.43:0/3722417688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a8415dcf0 0x7f0a8406e530 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f0a74009870 tx=0x7f0a7402a1c0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.056+0000 7f0a88b9e640 1 -- 172.21.15.43:0/3722417688 shutdown_connections 2024-04-25T09:03:34.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.056+0000 7f0a88b9e640 1 --2- 172.21.15.43:0/3722417688 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0a8406ebf0 0x7f0a8406f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.056+0000 7f0a88b9e640 1 --2- 172.21.15.43:0/3722417688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a8415dcf0 0x7f0a8406e530 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.056+0000 7f0a88b9e640 1 -- 172.21.15.43:0/3722417688 >> 172.21.15.43:0/3722417688 conn(0x7f0a840d1700 msgr2=0x7f0a840d1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:34.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.056+0000 7f0a88b9e640 1 -- 172.21.15.43:0/3722417688 shutdown_connections 2024-04-25T09:03:34.046 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.056+0000 7f0a88b9e640 1 -- 172.21.15.43:0/3722417688 wait complete. 2024-04-25T09:03:34.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.057+0000 7f0a88b9e640 1 Processor -- start 2024-04-25T09:03:34.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.058+0000 7f0a88b9e640 1 -- start start 2024-04-25T09:03:34.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.058+0000 7f0a88b9e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a8406ebf0 0x7f0a841522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.058+0000 7f0a88b9e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a8415dcf0 0x7f0a840c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.058+0000 7f0a88b9e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0a841616e0 0x7f0a840c7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.048 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.058+0000 7f0a88b9e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8406bce0 con 0x7f0a8415dcf0 2024-04-25T09:03:34.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.058+0000 7f0a88b9e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a840d0aa0 con 0x7f0a8406ebf0 2024-04-25T09:03:34.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.058+0000 7f0a88b9e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a840b0d50 con 0x7f0a841616e0 2024-04-25T09:03:34.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.058+0000 7f0a7bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a8415dcf0 0x7f0a840c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.058+0000 7f0a7bfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a8415dcf0 0x7f0a840c5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51108/0 (socket says 172.21.15.43:51108) 2024-04-25T09:03:34.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.058+0000 7f0a7bfff640 1 -- 172.21.15.43:0/2157112924 learned_addr learned my addr 172.21.15.43:0/2157112924 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:34.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.059+0000 7f0a8259c640 1 --2- 172.21.15.43:0/2157112924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a8406ebf0 0x7f0a841522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.059+0000 7f0a82d9d640 1 --2- 172.21.15.43:0/2157112924 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0a841616e0 0x7f0a840c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.059+0000 7f0a7bfff640 1 -- 172.21.15.43:0/2157112924 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0a841616e0 msgr2=0x7f0a840c7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.059+0000 7f0a7bfff640 1 --2- 172.21.15.43:0/2157112924 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0a841616e0 0x7f0a840c7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.059+0000 7f0a7bfff640 1 -- 172.21.15.43:0/2157112924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a8406ebf0 msgr2=0x7f0a841522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.059+0000 7f0a7bfff640 1 --2- 172.21.15.43:0/2157112924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a8406ebf0 0x7f0a841522c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.059+0000 7f0a7bfff640 1 -- 172.21.15.43:0/2157112924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a84160d60 con 0x7f0a8415dcf0 2024-04-25T09:03:34.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.059+0000 7f0a7bfff640 1 --2- 172.21.15.43:0/2157112924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a8415dcf0 0x7f0a840c5bd0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c00d800 tx=0x7f0a6c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:34.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.059+0000 7f0a7affd640 1 -- 172.21.15.43:0/2157112924 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c00b6c0 con 0x7f0a8415dcf0 2024-04-25T09:03:34.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.059+0000 7f0a7affd640 1 -- 172.21.15.43:0/2157112924 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a6c019070 con 0x7f0a8415dcf0 2024-04-25T09:03:34.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.059+0000 7f0a88b9e640 1 -- 172.21.15.43:0/2157112924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a84152800 con 0x7f0a8415dcf0 2024-04-25T09:03:34.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.059+0000 7f0a7affd640 1 -- 172.21.15.43:0/2157112924 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a6c00ba80 con 0x7f0a8415dcf0 2024-04-25T09:03:34.050 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.059+0000 7f0a88b9e640 1 -- 172.21.15.43:0/2157112924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a841607a0 con 0x7f0a8415dcf0 2024-04-25T09:03:34.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.060+0000 7f0a88b9e640 1 -- 172.21.15.43:0/2157112924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a84074ef0 con 0x7f0a8415dcf0 2024-04-25T09:03:34.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.061+0000 7f0a7affd640 1 -- 172.21.15.43:0/2157112924 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0a6c00bc20 con 0x7f0a8415dcf0 2024-04-25T09:03:34.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.061+0000 7f0a7affd640 1 --2- 172.21.15.43:0/2157112924 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0a6406b4d0 0x7f0a6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.061+0000 7f0a7affd640 1 -- 172.21.15.43:0/2157112924 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0a6c08a620 con 0x7f0a8415dcf0 2024-04-25T09:03:34.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.061+0000 7f0a8259c640 1 --2- 172.21.15.43:0/2157112924 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0a6406b4d0 0x7f0a6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.062+0000 7f0a8259c640 1 --2- 172.21.15.43:0/2157112924 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0a6406b4d0 0x7f0a6406d990 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f0a74034000 tx=0x7f0a7402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:34.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.065+0000 7f0a7affd640 1 -- 172.21.15.43:0/2157112924 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0a6c059530 con 0x7f0a8415dcf0 2024-04-25T09:03:34.270 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.282+0000 7f0a88b9e640 1 -- 172.21.15.43:0/2157112924 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a840af350 con 0x7f0a8415dcf0 2024-04-25T09:03:34.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.282+0000 7f0a7affd640 1 -- 172.21.15.43:0/2157112924 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f0a6c058e30 con 0x7f0a8415dcf0 2024-04-25T09:03:34.271 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:03:34.271 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/891225068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":891225068},{"type":"v1","addr":"172.21.15.193:6836","nonce":891225068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":7175,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3679059390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":3679059390},{"type":"v1","addr":"172.21.15.193:6837","nonce":3679059390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":7948,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2127363183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2127363183},{"type":"v1","addr":"172.21.15.43:6835","nonce":2127363183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:11.919337+0000","modified":"2024-04-25T09:03:13.917768+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9237},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9237":{"gid":9237,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6834/2603892772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2603892772},{"type":"v1","addr":"172.21.15.43:6834","nonce":2603892772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:03:34.271 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 76 2024-04-25T09:03:34.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.284+0000 7f0a88b9e640 1 -- 172.21.15.43:0/2157112924 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0a6406b4d0 msgr2=0x7f0a6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.284+0000 7f0a88b9e640 1 --2- 172.21.15.43:0/2157112924 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0a6406b4d0 0x7f0a6406d990 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f0a74034000 tx=0x7f0a7402e000 comp rx=0 tx=0).stop 2024-04-25T09:03:34.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.284+0000 7f0a88b9e640 1 -- 172.21.15.43:0/2157112924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a8415dcf0 msgr2=0x7f0a840c5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.284+0000 7f0a88b9e640 1 --2- 172.21.15.43:0/2157112924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a8415dcf0 0x7f0a840c5bd0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f0a6c00d800 tx=0x7f0a6c00dcc0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.284+0000 7f0a88b9e640 1 -- 172.21.15.43:0/2157112924 shutdown_connections 2024-04-25T09:03:34.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.284+0000 7f0a88b9e640 1 --2- 172.21.15.43:0/2157112924 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0a841616e0 0x7f0a840c7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.284+0000 7f0a88b9e640 1 --2- 172.21.15.43:0/2157112924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0a8415dcf0 0x7f0a840c5bd0 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.284+0000 7f0a88b9e640 1 --2- 172.21.15.43:0/2157112924 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0a6406b4d0 0x7f0a6406d990 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.284+0000 7f0a88b9e640 1 --2- 172.21.15.43:0/2157112924 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0a8406ebf0 0x7f0a841522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.284+0000 7f0a88b9e640 1 -- 172.21.15.43:0/2157112924 >> 172.21.15.43:0/2157112924 conn(0x7f0a840d1700 msgr2=0x7f0a8406c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:34.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.284+0000 7f0a88b9e640 1 -- 172.21.15.43:0/2157112924 shutdown_connections 2024-04-25T09:03:34.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.284+0000 7f0a88b9e640 1 -- 172.21.15.43:0/2157112924 wait complete. 2024-04-25T09:03:34.283 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T09:03:34.284 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:03:34.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.394+0000 7fd0a169a640 1 Processor -- start 2024-04-25T09:03:34.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.395+0000 7fd0a169a640 1 -- start start 2024-04-25T09:03:34.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.395+0000 7fd0a169a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd09c06b240 0x7fd09c0cea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.384 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.395+0000 7fd0a169a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd09c0cf790 con 0x7fd09c0cefa0 2024-04-25T09:03:34.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.395+0000 7fd0a169a640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd09c0cfa70 con 0x7fd09c15bbf0 2024-04-25T09:03:34.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.395+0000 7fd0a169a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd09c0cfd50 con 0x7fd09c06b240 2024-04-25T09:03:34.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.396+0000 7fd09a7fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd09c06b240 0x7fd09c0cea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.396+0000 7fd09b7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fd09c0cefa0 0x7fd09c06ea50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:37462/0 (socket says 172.21.15.43:37462) 2024-04-25T09:03:34.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.396+0000 7fd09b7fe640 1 -- 172.21.15.43:0/2459364627 learned_addr learned my addr 172.21.15.43:0/2459364627 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:03:34.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.396+0000 7fd09a7fc640 1 -- 172.21.15.43:0/2459364627 >> v1:172.21.15.193:6789/0 conn(0x7fd09c15bbf0 legacy=0x7fd09c06ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.396+0000 7fd09a7fc640 1 -- 172.21.15.43:0/2459364627 >> v1:172.21.15.43:6789/0 conn(0x7fd09c0cefa0 legacy=0x7fd09c06ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.396+0000 7fd099ffb640 1 -- 172.21.15.43:0/2459364627 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3912838862 0 0) 0x7fd09c0cf790 con 0x7fd09c0cefa0 2024-04-25T09:03:34.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.396+0000 7fd09a7fc640 1 -- 172.21.15.43:0/2459364627 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd09c0d0030 con 0x7fd09c06b240 2024-04-25T09:03:34.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.397+0000 7fd09a7fc640 1 --2- 172.21.15.43:0/2459364627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd09c06b240 0x7fd09c0cea60 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fd0900097a0 tx=0x7fd09002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c88f0543292cffa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:34.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.397+0000 7fd099ffb640 1 -- 172.21.15.43:0/2459364627 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd090037070 con 0x7fd09c06b240 2024-04-25T09:03:34.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.397+0000 7fd099ffb640 1 -- 172.21.15.43:0/2459364627 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd090032640 con 0x7fd09c06b240 2024-04-25T09:03:34.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.397+0000 7fd099ffb640 1 -- 172.21.15.43:0/2459364627 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd090032a10 con 0x7fd09c06b240 2024-04-25T09:03:34.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.397+0000 7fd0a169a640 1 -- 172.21.15.43:0/2459364627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd09c06b240 msgr2=0x7fd09c0cea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.397+0000 7fd0a169a640 1 --2- 172.21.15.43:0/2459364627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd09c06b240 0x7fd09c0cea60 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fd0900097a0 tx=0x7fd09002a2c0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.397+0000 7fd0a169a640 1 -- 172.21.15.43:0/2459364627 shutdown_connections 2024-04-25T09:03:34.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.397+0000 7fd0a169a640 1 --2- 172.21.15.43:0/2459364627 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd09c06b240 0x7fd09c0cea60 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.397+0000 7fd0a169a640 1 -- 172.21.15.43:0/2459364627 >> 172.21.15.43:0/2459364627 conn(0x7fd09c0b2490 msgr2=0x7fd09c0b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:34.387 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.397+0000 7fd0a169a640 1 -- 172.21.15.43:0/2459364627 shutdown_connections 2024-04-25T09:03:34.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.397+0000 7fd0a169a640 1 -- 172.21.15.43:0/2459364627 wait complete. 2024-04-25T09:03:34.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.399+0000 7fd0a169a640 1 Processor -- start 2024-04-25T09:03:34.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.399+0000 7fd0a169a640 1 -- start start 2024-04-25T09:03:34.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.400+0000 7fd0a169a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd09c06b240 0x7fd09c1577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.400+0000 7fd0a169a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd09c0cefa0 0x7fd09c1f1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.400+0000 7fd0a169a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd09c15bbf0 0x7fd09c1501d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.400+0000 7fd0a169a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09c15c150 con 0x7fd09c06b240 2024-04-25T09:03:34.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.400+0000 7fd0a169a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09c15bfd0 con 0x7fd09c0cefa0 2024-04-25T09:03:34.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.400+0000 7fd0a169a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd09c06f160 con 0x7fd09c15bbf0 2024-04-25T09:03:34.390 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.400+0000 7fd09affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd09c06b240 0x7fd09c1577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.400+0000 7fd09affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd09c06b240 0x7fd09c1577f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51136/0 (socket says 172.21.15.43:51136) 2024-04-25T09:03:34.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.400+0000 7fd09affd640 1 -- 172.21.15.43:0/670313073 learned_addr learned my addr 172.21.15.43:0/670313073 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:34.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.400+0000 7fd09b7fe640 1 --2- 172.21.15.43:0/670313073 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd09c15bbf0 0x7fd09c1501d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.400+0000 7fd09a7fc640 1 --2- 172.21.15.43:0/670313073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd09c0cefa0 0x7fd09c1f1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.401+0000 7fd09affd640 1 -- 172.21.15.43:0/670313073 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd09c15bbf0 msgr2=0x7fd09c1501d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.391 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.401+0000 7fd09affd640 1 --2- 172.21.15.43:0/670313073 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd09c15bbf0 0x7fd09c1501d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.401+0000 7fd09affd640 1 -- 172.21.15.43:0/670313073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd09c0cefa0 msgr2=0x7fd09c1f1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.401+0000 7fd09affd640 1 --2- 172.21.15.43:0/670313073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd09c0cefa0 0x7fd09c1f1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.401+0000 7fd09affd640 1 -- 172.21.15.43:0/670313073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd09c1f2170 con 0x7fd09c06b240 2024-04-25T09:03:34.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.401+0000 7fd09a7fc640 1 --2- 172.21.15.43:0/670313073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd09c0cefa0 0x7fd09c1f1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:34.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.401+0000 7fd09affd640 1 --2- 172.21.15.43:0/670313073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd09c06b240 0x7fd09c1577f0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fd08c00c430 tx=0x7fd08c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:34.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.401+0000 7fd0837fe640 1 -- 172.21.15.43:0/670313073 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08c017070 con 0x7fd09c06b240 2024-04-25T09:03:34.392 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.401+0000 7fd0837fe640 1 -- 172.21.15.43:0/670313073 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd08c0052d0 con 0x7fd09c06b240 2024-04-25T09:03:34.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.401+0000 7fd0a169a640 1 -- 172.21.15.43:0/670313073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd09c1f2410 con 0x7fd09c06b240 2024-04-25T09:03:34.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.401+0000 7fd0a169a640 1 -- 172.21.15.43:0/670313073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd09c1f29d0 con 0x7fd09c06b240 2024-04-25T09:03:34.393 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.401+0000 7fd0837fe640 1 -- 172.21.15.43:0/670313073 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08c003830 con 0x7fd09c06b240 2024-04-25T09:03:34.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.403+0000 7fd0a169a640 1 -- 172.21.15.43:0/670313073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd07c005290 con 0x7fd09c06b240 2024-04-25T09:03:34.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.403+0000 7fd0837fe640 1 -- 172.21.15.43:0/670313073 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd08c01a070 con 0x7fd09c06b240 2024-04-25T09:03:34.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.404+0000 7fd0837fe640 1 --2- 172.21.15.43:0/670313073 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd08806b4d0 0x7fd08806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.404+0000 7fd0837fe640 1 -- 172.21.15.43:0/670313073 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd08c0894b0 con 0x7fd09c06b240 2024-04-25T09:03:34.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.404+0000 7fd09a7fc640 1 --2- 172.21.15.43:0/670313073 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd08806b4d0 0x7fd08806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.404+0000 7fd09a7fc640 1 --2- 172.21.15.43:0/670313073 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd08806b4d0 0x7fd08806d990 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fd090034000 tx=0x7fd09002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:34.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.408+0000 7fd0837fe640 1 -- 172.21.15.43:0/670313073 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd08c0583c0 con 0x7fd09c06b240 2024-04-25T09:03:34.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.662+0000 7fd0a169a640 1 -- 172.21.15.43:0/670313073 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fd07c005850 con 0x7fd09c06b240 2024-04-25T09:03:34.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.662+0000 7fd0837fe640 1 -- 172.21.15.43:0/670313073 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7fd08c057cc0 con 0x7fd09c06b240 2024-04-25T09:03:34.651 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:03:34.651 INFO:teuthology.orchestra.run.smithi043.stdout:{"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":587,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":126,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035238,"num_in_osds":8,"osd_in_since":1714035229,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1077723995,"bytes_used":2685042688,"bytes_avail":770409070592,"bytes_total":773094113280,"write_bytes_sec":89752229,"read_op_per_sec":0,"write_op_per_sec":89},"fsmap":{"epoch":76,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"a","status":"up:active","gid":9237}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":23,"modified":"2024-04-25T09:03:33.825006+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:03:34.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.664+0000 7fd0a169a640 1 -- 172.21.15.43:0/670313073 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd08806b4d0 msgr2=0x7fd08806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.664+0000 7fd0a169a640 1 --2- 172.21.15.43:0/670313073 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd08806b4d0 0x7fd08806d990 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fd090034000 tx=0x7fd09002e000 comp rx=0 tx=0).stop 2024-04-25T09:03:34.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.664+0000 7fd0a169a640 1 -- 172.21.15.43:0/670313073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd09c06b240 msgr2=0x7fd09c1577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.664+0000 7fd0a169a640 1 --2- 172.21.15.43:0/670313073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd09c06b240 0x7fd09c1577f0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fd08c00c430 tx=0x7fd08c00c8f0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.664+0000 7fd0a169a640 1 -- 172.21.15.43:0/670313073 shutdown_connections 2024-04-25T09:03:34.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.664+0000 7fd0a169a640 1 --2- 172.21.15.43:0/670313073 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd09c15bbf0 0x7fd09c1501d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.664+0000 7fd0a169a640 1 --2- 172.21.15.43:0/670313073 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd08806b4d0 0x7fd08806d990 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.664+0000 7fd0a169a640 1 --2- 172.21.15.43:0/670313073 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd09c0cefa0 0x7fd09c1f1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.664+0000 7fd0a169a640 1 --2- 172.21.15.43:0/670313073 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd09c06b240 0x7fd09c1577f0 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.664+0000 7fd0a169a640 1 -- 172.21.15.43:0/670313073 >> 172.21.15.43:0/670313073 conn(0x7fd09c0b2490 msgr2=0x7fd09c071e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:34.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.664+0000 7fd0a169a640 1 -- 172.21.15.43:0/670313073 shutdown_connections 2024-04-25T09:03:34.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.664+0000 7fd0a169a640 1 -- 172.21.15.43:0/670313073 wait complete. 2024-04-25T09:03:34.663 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'paused'} 2024-04-25T09:03:34.664 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:03:34.664 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:03:34.675 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:03:34.675 INFO:tasks.cephfs.test_scrub_checks:Waiting for promotion of one of the original standbys {'c', 'd', 'b'} 2024-04-25T09:03:34.675 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-04-25T09:03:34.823 INFO:teuthology.orchestra.run.smithi043.stdout:{"mds_beacon_grace":"15.000000"} 2024-04-25T09:03:34.823 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-04-25T09:03:34.824 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:34.926 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.936+0000 7fdceb04d640 1 Processor -- start 2024-04-25T09:03:34.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.936+0000 7fdceb04d640 1 -- start start 2024-04-25T09:03:34.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.937+0000 7fdceb04d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdce415df50 0x7fdce41619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.937+0000 7fdceb04d640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fdce4161f30 0x7fdce4162380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.937+0000 7fdceb04d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdce40d24a0 con 0x7fdce40c7cf0 2024-04-25T09:03:34.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.937+0000 7fdceb04d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdce415e330 con 0x7fdce415df50 2024-04-25T09:03:34.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.937+0000 7fdceb04d640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdce40d2740 con 0x7fdce4161f30 2024-04-25T09:03:34.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.937+0000 7fdce8de9640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdce415df50 0x7fdce41619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.937+0000 7fdce95ea640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fdce40c7cf0 0x7fdce40c8100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:37476/0 (socket says 172.21.15.43:37476) 2024-04-25T09:03:34.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.937+0000 7fdce95ea640 1 -- 172.21.15.43:0/2568940775 learned_addr learned my addr 172.21.15.43:0/2568940775 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:03:34.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.938+0000 7fdce3fff640 1 --2- 172.21.15.43:0/2568940775 >> v2:172.21.15.193:3300/0 conn(0x7fdce4161f30 0x7fdce4162380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.927 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.938+0000 7fdce8de9640 1 -- 172.21.15.43:0/2568940775 >> v2:172.21.15.193:3300/0 conn(0x7fdce4161f30 msgr2=0x7fdce4162380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:34.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.938+0000 7fdce8de9640 1 --2- 172.21.15.43:0/2568940775 >> v2:172.21.15.193:3300/0 conn(0x7fdce4161f30 0x7fdce4162380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.938+0000 7fdce8de9640 1 -- 172.21.15.43:0/2568940775 >> v1:172.21.15.43:6789/0 conn(0x7fdce40c7cf0 legacy=0x7fdce40c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.938+0000 7fdce8de9640 1 -- 172.21.15.43:0/2568940775 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce40d2a20 con 0x7fdce415df50 2024-04-25T09:03:34.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.938+0000 7fdce3fff640 1 --2- 172.21.15.43:0/2568940775 >> v2:172.21.15.193:3300/0 conn(0x7fdce4161f30 0x7fdce4162380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:34.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.938+0000 7fdce8de9640 1 --2- 172.21.15.43:0/2568940775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdce415df50 0x7fdce41619f0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fdce40d45e0 tx=0x7fdcdc02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24beec5cd224513a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:34.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.939+0000 7fdce37fe640 1 -- 172.21.15.43:0/2568940775 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc007740 con 0x7fdce415df50 2024-04-25T09:03:34.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.939+0000 7fdce37fe640 1 -- 172.21.15.43:0/2568940775 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcdc0078e0 con 0x7fdce415df50 2024-04-25T09:03:34.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.939+0000 7fdce37fe640 1 -- 172.21.15.43:0/2568940775 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc007cb0 con 0x7fdce415df50 2024-04-25T09:03:34.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.939+0000 7fdceb04d640 1 -- 172.21.15.43:0/2568940775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdce415df50 msgr2=0x7fdce41619f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.939+0000 7fdceb04d640 1 --2- 172.21.15.43:0/2568940775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdce415df50 0x7fdce41619f0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fdce40d45e0 tx=0x7fdcdc02a680 comp rx=0 tx=0).stop 2024-04-25T09:03:34.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.939+0000 7fdceb04d640 1 -- 172.21.15.43:0/2568940775 shutdown_connections 2024-04-25T09:03:34.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.939+0000 7fdceb04d640 1 --2- 172.21.15.43:0/2568940775 >> v2:172.21.15.193:3300/0 conn(0x7fdce4161f30 0x7fdce4162380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.939+0000 7fdceb04d640 1 --2- 172.21.15.43:0/2568940775 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdce415df50 0x7fdce41619f0 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.939+0000 7fdceb04d640 1 -- 172.21.15.43:0/2568940775 >> 172.21.15.43:0/2568940775 conn(0x7fdce40d3a30 msgr2=0x7fdce40d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:34.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.939+0000 7fdceb04d640 1 -- 172.21.15.43:0/2568940775 shutdown_connections 2024-04-25T09:03:34.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.939+0000 7fdceb04d640 1 -- 172.21.15.43:0/2568940775 wait complete. 2024-04-25T09:03:34.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.940+0000 7fdceb04d640 1 Processor -- start 2024-04-25T09:03:34.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.940+0000 7fdceb04d640 1 -- start start 2024-04-25T09:03:34.936 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.941+0000 7fdceb04d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdce40c7cf0 0x7fdce4160ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.941+0000 7fdceb04d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdce415df50 0x7fdce409e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.941+0000 7fdceb04d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdce4161f30 0x7fdce41552c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.941+0000 7fdceb04d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce40d2190 con 0x7fdce40c7cf0 2024-04-25T09:03:34.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.941+0000 7fdceb04d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce40c77d0 con 0x7fdce4161f30 2024-04-25T09:03:34.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.941+0000 7fdceb04d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdce40cf670 con 0x7fdce415df50 2024-04-25T09:03:34.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.941+0000 7fdce8de9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdce40c7cf0 0x7fdce4160ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.941+0000 7fdce8de9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdce40c7cf0 0x7fdce4160ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:51160/0 (socket says 172.21.15.43:51160) 2024-04-25T09:03:34.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.941+0000 7fdce8de9640 1 -- 172.21.15.43:0/1012875580 learned_addr learned my addr 172.21.15.43:0/1012875580 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:34.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.942+0000 7fdce95ea640 1 --2- 172.21.15.43:0/1012875580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdce4161f30 0x7fdce41552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.942+0000 7fdce3fff640 1 --2- 172.21.15.43:0/1012875580 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdce415df50 0x7fdce409e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.942+0000 7fdce8de9640 1 -- 172.21.15.43:0/1012875580 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdce415df50 msgr2=0x7fdce409e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.942+0000 7fdce8de9640 1 --2- 172.21.15.43:0/1012875580 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdce415df50 0x7fdce409e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.937 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.942+0000 7fdce8de9640 1 -- 172.21.15.43:0/1012875580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdce4161f30 msgr2=0x7fdce41552c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:34.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.942+0000 7fdce8de9640 1 --2- 172.21.15.43:0/1012875580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdce4161f30 0x7fdce41552c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:34.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.942+0000 7fdce8de9640 1 -- 172.21.15.43:0/1012875580 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce4161430 con 0x7fdce40c7cf0 2024-04-25T09:03:34.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.942+0000 7fdce8de9640 1 --2- 172.21.15.43:0/1012875580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdce40c7cf0 0x7fdce4160ef0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc02ac70 tx=0x7fdcdc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:34.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.942+0000 7fdce17fa640 1 -- 172.21.15.43:0/1012875580 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc040070 con 0x7fdce40c7cf0 2024-04-25T09:03:34.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.942+0000 7fdce17fa640 1 -- 172.21.15.43:0/1012875580 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcdc031c40 con 0x7fdce40c7cf0 2024-04-25T09:03:34.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.942+0000 7fdceb04d640 1 -- 172.21.15.43:0/1012875580 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdce415fe50 con 0x7fdce40c7cf0 2024-04-25T09:03:34.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.942+0000 7fdce17fa640 1 -- 172.21.15.43:0/1012875580 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcdc03c420 con 0x7fdce40c7cf0 2024-04-25T09:03:34.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.942+0000 7fdceb04d640 1 -- 172.21.15.43:0/1012875580 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdce4160410 con 0x7fdce40c7cf0 2024-04-25T09:03:34.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.943+0000 7fdceb04d640 1 -- 172.21.15.43:0/1012875580 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdce40680f0 con 0x7fdce40c7cf0 2024-04-25T09:03:34.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.944+0000 7fdce17fa640 1 -- 172.21.15.43:0/1012875580 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdcdc039040 con 0x7fdce40c7cf0 2024-04-25T09:03:34.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.944+0000 7fdce17fa640 1 --2- 172.21.15.43:0/1012875580 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdcd006b500 0x7fdcd006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:34.938 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.944+0000 7fdce17fa640 1 -- 172.21.15.43:0/1012875580 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdcdc0ab710 con 0x7fdce40c7cf0 2024-04-25T09:03:34.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.944+0000 7fdce3fff640 1 --2- 172.21.15.43:0/1012875580 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdcd006b500 0x7fdcd006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:34.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.945+0000 7fdce3fff640 1 --2- 172.21.15.43:0/1012875580 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdcd006b500 0x7fdcd006d9c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fdce40a1910 tx=0x7fdcd8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:34.939 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:34.948+0000 7fdce17fa640 1 -- 172.21.15.43:0/1012875580 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdcdc07a620 con 0x7fdce40c7cf0 2024-04-25T09:03:35.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:35.167+0000 7fdceb04d640 1 -- 172.21.15.43:0/1012875580 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdce4074ef0 con 0x7fdce40c7cf0 2024-04-25T09:03:35.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:35.168+0000 7fdce17fa640 1 -- 172.21.15.43:0/1012875580 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fdcdc079f20 con 0x7fdce40c7cf0 2024-04-25T09:03:35.157 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:03:35.157 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/891225068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":891225068},{"type":"v1","addr":"172.21.15.193:6836","nonce":891225068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":7175,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3679059390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":3679059390},{"type":"v1","addr":"172.21.15.193:6837","nonce":3679059390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":7948,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2127363183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2127363183},{"type":"v1","addr":"172.21.15.43:6835","nonce":2127363183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:11.919337+0000","modified":"2024-04-25T09:03:13.917768+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9237},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9237":{"gid":9237,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6834/2603892772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2603892772},{"type":"v1","addr":"172.21.15.43:6834","nonce":2603892772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:03:35.157 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 76 2024-04-25T09:03:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:35.170+0000 7fdceb04d640 1 -- 172.21.15.43:0/1012875580 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdcd006b500 msgr2=0x7fdcd006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:35.170+0000 7fdceb04d640 1 --2- 172.21.15.43:0/1012875580 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdcd006b500 0x7fdcd006d9c0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fdce40a1910 tx=0x7fdcd8008040 comp rx=0 tx=0).stop 2024-04-25T09:03:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:35.170+0000 7fdceb04d640 1 -- 172.21.15.43:0/1012875580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdce40c7cf0 msgr2=0x7fdce4160ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:35.170+0000 7fdceb04d640 1 --2- 172.21.15.43:0/1012875580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdce40c7cf0 0x7fdce4160ef0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc02ac70 tx=0x7fdcdc004a40 comp rx=0 tx=0).stop 2024-04-25T09:03:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:35.170+0000 7fdceb04d640 1 -- 172.21.15.43:0/1012875580 shutdown_connections 2024-04-25T09:03:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:35.170+0000 7fdceb04d640 1 --2- 172.21.15.43:0/1012875580 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fdcd006b500 0x7fdcd006d9c0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:35.170+0000 7fdceb04d640 1 --2- 172.21.15.43:0/1012875580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdce4161f30 0x7fdce41552c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:35.170+0000 7fdceb04d640 1 --2- 172.21.15.43:0/1012875580 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdce415df50 0x7fdce409e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:35.170+0000 7fdceb04d640 1 --2- 172.21.15.43:0/1012875580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdce40c7cf0 0x7fdce4160ef0 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:35.170+0000 7fdceb04d640 1 -- 172.21.15.43:0/1012875580 >> 172.21.15.43:0/1012875580 conn(0x7fdce40d3a30 msgr2=0x7fdce40ce620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:35.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:35.170+0000 7fdceb04d640 1 -- 172.21.15.43:0/1012875580 shutdown_connections 2024-04-25T09:03:35.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:35.170+0000 7fdceb04d640 1 -- 172.21.15.43:0/1012875580 wait complete. 2024-04-25T09:03:35.171 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T09:03:40.172 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:40.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.295+0000 7f6a1aa1f640 1 Processor -- start 2024-04-25T09:03:40.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.295+0000 7f6a1aa1f640 1 -- start start 2024-04-25T09:03:40.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.296+0000 7f6a1aa1f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6a140d09b0 0x7f6a140b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:40.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.296+0000 7f6a1aa1f640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6a140ce5e0 0x7f6a140cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:40.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.296+0000 7f6a1aa1f640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a140b4c30 con 0x7f6a14161390 2024-04-25T09:03:40.291 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.296+0000 7f6a1aa1f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a140d1910 con 0x7f6a140d09b0 2024-04-25T09:03:40.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.296+0000 7f6a1aa1f640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a140cf2c0 con 0x7f6a140ce5e0 2024-04-25T09:03:40.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.296+0000 7f6a137fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6a140d09b0 0x7f6a140b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:40.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.296+0000 7f6a137fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6a140d09b0 0x7f6a140b6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49728/0 (socket says 172.21.15.43:49728) 2024-04-25T09:03:40.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.296+0000 7f6a137fe640 1 -- 172.21.15.43:0/2257271441 learned_addr learned my addr 172.21.15.43:0/2257271441 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:40.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.296+0000 7f6a18fbc640 1 --2- 172.21.15.43:0/2257271441 >> v2:172.21.15.193:3300/0 conn(0x7f6a140ce5e0 0x7f6a140cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:40.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.296+0000 7f6a137fe640 1 -- 172.21.15.43:0/2257271441 >> v2:172.21.15.193:3300/0 conn(0x7f6a140ce5e0 msgr2=0x7f6a140cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:40.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.296+0000 7f6a137fe640 1 --2- 172.21.15.43:0/2257271441 >> v2:172.21.15.193:3300/0 conn(0x7f6a140ce5e0 0x7f6a140cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:40.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.296+0000 7f6a137fe640 1 -- 172.21.15.43:0/2257271441 >> v1:172.21.15.193:6789/0 conn(0x7f6a14161390 legacy=0x7f6a140d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:40.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.296+0000 7f6a137fe640 1 -- 172.21.15.43:0/2257271441 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a140cf5a0 con 0x7f6a140d09b0 2024-04-25T09:03:40.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.297+0000 7f6a137fe640 1 --2- 172.21.15.43:0/2257271441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a140d09b0 0x7f6a140b6820 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f6a1408dd30 tx=0x7f6a0802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7121135fa07b3464 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:40.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.297+0000 7f6a12ffd640 1 -- 172.21.15.43:0/2257271441 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a08037070 con 0x7f6a140d09b0 2024-04-25T09:03:40.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.297+0000 7f6a12ffd640 1 -- 172.21.15.43:0/2257271441 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a08032440 con 0x7f6a140d09b0 2024-04-25T09:03:40.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.297+0000 7f6a12ffd640 1 -- 172.21.15.43:0/2257271441 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a08032810 con 0x7f6a140d09b0 2024-04-25T09:03:40.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.297+0000 7f6a1aa1f640 1 -- 172.21.15.43:0/2257271441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a140d09b0 msgr2=0x7f6a140b6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:40.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.297+0000 7f6a1aa1f640 1 --2- 172.21.15.43:0/2257271441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a140d09b0 0x7f6a140b6820 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f6a1408dd30 tx=0x7f6a0802a270 comp rx=0 tx=0).stop 2024-04-25T09:03:40.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.298+0000 7f6a1aa1f640 1 -- 172.21.15.43:0/2257271441 shutdown_connections 2024-04-25T09:03:40.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.298+0000 7f6a1aa1f640 1 --2- 172.21.15.43:0/2257271441 >> v2:172.21.15.193:3300/0 conn(0x7f6a140ce5e0 0x7f6a140cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:40.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.298+0000 7f6a1aa1f640 1 --2- 172.21.15.43:0/2257271441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a140d09b0 0x7f6a140b6820 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:40.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.298+0000 7f6a1aa1f640 1 -- 172.21.15.43:0/2257271441 >> 172.21.15.43:0/2257271441 conn(0x7f6a1409cd20 msgr2=0x7f6a140d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:40.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.298+0000 7f6a1aa1f640 1 -- 172.21.15.43:0/2257271441 shutdown_connections 2024-04-25T09:03:40.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.298+0000 7f6a1aa1f640 1 -- 172.21.15.43:0/2257271441 wait complete. 2024-04-25T09:03:40.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.299+0000 7f6a1aa1f640 1 Processor -- start 2024-04-25T09:03:40.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.299+0000 7f6a1aa1f640 1 -- start start 2024-04-25T09:03:40.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.300+0000 7f6a1aa1f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a140ce5e0 0x7f6a140c7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:40.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.300+0000 7f6a1aa1f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a140d09b0 0x7f6a140ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:40.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.300+0000 7f6a1aa1f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a14161390 0x7f6a141f3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:40.294 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.300+0000 7f6a1aa1f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a14161770 con 0x7f6a14161390 2024-04-25T09:03:40.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.300+0000 7f6a1aa1f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a140d15a0 con 0x7f6a140d09b0 2024-04-25T09:03:40.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.300+0000 7f6a1aa1f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a140cef70 con 0x7f6a140ce5e0 2024-04-25T09:03:40.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.300+0000 7f6a18fbc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a14161390 0x7f6a141f3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:40.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.300+0000 7f6a18fbc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a14161390 0x7f6a141f3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49730/0 (socket says 172.21.15.43:49730) 2024-04-25T09:03:40.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.300+0000 7f6a18fbc640 1 -- 172.21.15.43:0/2462720065 learned_addr learned my addr 172.21.15.43:0/2462720065 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:40.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.300+0000 7f6a137fe640 1 --2- 172.21.15.43:0/2462720065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a140d09b0 0x7f6a140ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:40.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.300+0000 7f6a13fff640 1 --2- 172.21.15.43:0/2462720065 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a140ce5e0 0x7f6a140c7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:40.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.300+0000 7f6a18fbc640 1 -- 172.21.15.43:0/2462720065 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a140ce5e0 msgr2=0x7f6a140c7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:40.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.300+0000 7f6a18fbc640 1 --2- 172.21.15.43:0/2462720065 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a140ce5e0 0x7f6a140c7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:40.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.300+0000 7f6a18fbc640 1 -- 172.21.15.43:0/2462720065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a140d09b0 msgr2=0x7f6a140ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:40.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.300+0000 7f6a18fbc640 1 --2- 172.21.15.43:0/2462720065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a140d09b0 0x7f6a140ccfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:40.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.300+0000 7f6a18fbc640 1 -- 172.21.15.43:0/2462720065 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a140cd640 con 0x7f6a14161390 2024-04-25T09:03:40.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.300+0000 7f6a18fbc640 1 --2- 172.21.15.43:0/2462720065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a14161390 0x7f6a141f3790 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c00efc0 tx=0x7f6a0c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:40.295 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.300+0000 7f6a10ff9640 1 -- 172.21.15.43:0/2462720065 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c018020 con 0x7f6a14161390 2024-04-25T09:03:40.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.300+0000 7f6a10ff9640 1 -- 172.21.15.43:0/2462720065 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a0c014480 con 0x7f6a14161390 2024-04-25T09:03:40.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.301+0000 7f6a1aa1f640 1 -- 172.21.15.43:0/2462720065 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a140cd920 con 0x7f6a14161390 2024-04-25T09:03:40.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.301+0000 7f6a10ff9640 1 -- 172.21.15.43:0/2462720065 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c014870 con 0x7f6a14161390 2024-04-25T09:03:40.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.301+0000 7f6a1aa1f640 1 -- 172.21.15.43:0/2462720065 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a1415bcf0 con 0x7f6a14161390 2024-04-25T09:03:40.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.302+0000 7f6a1aa1f640 1 -- 172.21.15.43:0/2462720065 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a140680f0 con 0x7f6a14161390 2024-04-25T09:03:40.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.302+0000 7f6a10ff9640 1 -- 172.21.15.43:0/2462720065 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6a0c005d10 con 0x7f6a14161390 2024-04-25T09:03:40.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.303+0000 7f6a10ff9640 1 --2- 172.21.15.43:0/2462720065 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6a0006b4d0 0x7f6a0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:40.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.303+0000 7f6a10ff9640 1 -- 172.21.15.43:0/2462720065 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a0c08ad70 con 0x7f6a14161390 2024-04-25T09:03:40.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.303+0000 7f6a13fff640 1 --2- 172.21.15.43:0/2462720065 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6a0006b4d0 0x7f6a0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:40.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.303+0000 7f6a13fff640 1 --2- 172.21.15.43:0/2462720065 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6a0006b4d0 0x7f6a0006d990 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6a04007fd0 tx=0x7f6a04009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:40.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.307+0000 7f6a10ff9640 1 -- 172.21.15.43:0/2462720065 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6a0c022070 con 0x7f6a14161390 2024-04-25T09:03:40.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.527+0000 7f6a1aa1f640 1 -- 172.21.15.43:0/2462720065 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a14074ef0 con 0x7f6a14161390 2024-04-25T09:03:40.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.528+0000 7f6a10ff9640 1 -- 172.21.15.43:0/2462720065 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4943 (secure 0 0 0) 0x7f6a0c010040 con 0x7f6a14161390 2024-04-25T09:03:40.517 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:03:40.517 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/891225068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":891225068},{"type":"v1","addr":"172.21.15.193:6836","nonce":891225068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":7175,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3679059390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":3679059390},{"type":"v1","addr":"172.21.15.193:6837","nonce":3679059390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":7948,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2127363183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2127363183},{"type":"v1","addr":"172.21.15.43:6835","nonce":2127363183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:11.919337+0000","modified":"2024-04-25T09:03:13.917768+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9237},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9237":{"gid":9237,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6834/2603892772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2603892772},{"type":"v1","addr":"172.21.15.43:6834","nonce":2603892772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:03:40.517 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 76 2024-04-25T09:03:40.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.530+0000 7f6a1aa1f640 1 -- 172.21.15.43:0/2462720065 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6a0006b4d0 msgr2=0x7f6a0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:40.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.530+0000 7f6a1aa1f640 1 --2- 172.21.15.43:0/2462720065 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6a0006b4d0 0x7f6a0006d990 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6a04007fd0 tx=0x7f6a04009040 comp rx=0 tx=0).stop 2024-04-25T09:03:40.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.530+0000 7f6a1aa1f640 1 -- 172.21.15.43:0/2462720065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a14161390 msgr2=0x7f6a141f3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:40.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.530+0000 7f6a1aa1f640 1 --2- 172.21.15.43:0/2462720065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a14161390 0x7f6a141f3790 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c00efc0 tx=0x7f6a0c00c490 comp rx=0 tx=0).stop 2024-04-25T09:03:40.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.530+0000 7f6a1aa1f640 1 -- 172.21.15.43:0/2462720065 shutdown_connections 2024-04-25T09:03:40.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.530+0000 7f6a1aa1f640 1 --2- 172.21.15.43:0/2462720065 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6a14161390 0x7f6a141f3790 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:40.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.530+0000 7f6a1aa1f640 1 --2- 172.21.15.43:0/2462720065 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6a0006b4d0 0x7f6a0006d990 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:40.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.530+0000 7f6a1aa1f640 1 --2- 172.21.15.43:0/2462720065 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6a140d09b0 0x7f6a140ccfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:40.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.530+0000 7f6a1aa1f640 1 --2- 172.21.15.43:0/2462720065 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6a140ce5e0 0x7f6a140c7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:40.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.530+0000 7f6a1aa1f640 1 -- 172.21.15.43:0/2462720065 >> 172.21.15.43:0/2462720065 conn(0x7f6a1409cd20 msgr2=0x7f6a140b7580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:40.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.530+0000 7f6a1aa1f640 1 -- 172.21.15.43:0/2462720065 shutdown_connections 2024-04-25T09:03:40.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:40.530+0000 7f6a1aa1f640 1 -- 172.21.15.43:0/2462720065 wait complete. 2024-04-25T09:03:40.530 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T09:03:45.531 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:45.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.656+0000 7fe3c3d09640 1 Processor -- start 2024-04-25T09:03:45.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.657+0000 7fe3c3d09640 1 -- start start 2024-04-25T09:03:45.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.657+0000 7fe3c3d09640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fe3bc06c510 0x7fe3bc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.657+0000 7fe3c3d09640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe3bc15df70 0x7fe3bc0ce070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.658+0000 7fe3c3d09640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3bc0cbee0 con 0x7fe3bc0b0870 2024-04-25T09:03:45.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.658+0000 7fe3c3d09640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3bc0d0400 con 0x7fe3bc06c510 2024-04-25T09:03:45.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.658+0000 7fe3c3d09640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc06ce30 con 0x7fe3bc15df70 2024-04-25T09:03:45.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.658+0000 7fe3c12a4640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe3bc15df70 0x7fe3bc0ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:45.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.658+0000 7fe3c12a4640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe3bc15df70 0x7fe3bc0ce070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:49314/0 (socket says 172.21.15.43:49314) 2024-04-25T09:03:45.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.658+0000 7fe3c12a4640 1 -- 172.21.15.43:0/3113129964 learned_addr learned my addr 172.21.15.43:0/3113129964 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:45.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.658+0000 7fe3c1aa5640 1 --2- 172.21.15.43:0/3113129964 >> v2:172.21.15.193:3300/0 conn(0x7fe3bc06c510 0x7fe3bc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:45.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.658+0000 7fe3c12a4640 1 -- 172.21.15.43:0/3113129964 >> v2:172.21.15.193:3300/0 conn(0x7fe3bc06c510 msgr2=0x7fe3bc06c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:45.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.658+0000 7fe3c12a4640 1 --2- 172.21.15.43:0/3113129964 >> v2:172.21.15.193:3300/0 conn(0x7fe3bc06c510 0x7fe3bc06c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.658+0000 7fe3c12a4640 1 -- 172.21.15.43:0/3113129964 >> v1:172.21.15.193:6789/0 conn(0x7fe3bc0b0870 legacy=0x7fe3bc0ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.658+0000 7fe3c12a4640 1 -- 172.21.15.43:0/3113129964 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3bc0d0650 con 0x7fe3bc15df70 2024-04-25T09:03:45.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.659+0000 7fe3c12a4640 1 --2- 172.21.15.43:0/3113129964 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe3bc15df70 0x7fe3bc0ce070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac009870 tx=0x7fe3ac02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ce1da46ebfcd365 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:45.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.659+0000 7fe3c0aa3640 1 -- 172.21.15.43:0/3113129964 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac037070 con 0x7fe3bc15df70 2024-04-25T09:03:45.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.659+0000 7fe3c0aa3640 1 -- 172.21.15.43:0/3113129964 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3ac032440 con 0x7fe3bc15df70 2024-04-25T09:03:45.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.659+0000 7fe3c0aa3640 1 -- 172.21.15.43:0/3113129964 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac032810 con 0x7fe3bc15df70 2024-04-25T09:03:45.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.660+0000 7fe3c3d09640 1 -- 172.21.15.43:0/3113129964 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe3bc15df70 msgr2=0x7fe3bc0ce070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.660+0000 7fe3c3d09640 1 --2- 172.21.15.43:0/3113129964 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe3bc15df70 0x7fe3bc0ce070 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac009870 tx=0x7fe3ac02a270 comp rx=0 tx=0).stop 2024-04-25T09:03:45.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.660+0000 7fe3c3d09640 1 -- 172.21.15.43:0/3113129964 shutdown_connections 2024-04-25T09:03:45.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.660+0000 7fe3c3d09640 1 --2- 172.21.15.43:0/3113129964 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe3bc15df70 0x7fe3bc0ce070 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.660+0000 7fe3c3d09640 1 --2- 172.21.15.43:0/3113129964 >> v2:172.21.15.193:3300/0 conn(0x7fe3bc06c510 0x7fe3bc06c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.660+0000 7fe3c3d09640 1 -- 172.21.15.43:0/3113129964 >> 172.21.15.43:0/3113129964 conn(0x7fe3bc0d17f0 msgr2=0x7fe3bc0d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:45.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.660+0000 7fe3c3d09640 1 -- 172.21.15.43:0/3113129964 shutdown_connections 2024-04-25T09:03:45.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.660+0000 7fe3c3d09640 1 -- 172.21.15.43:0/3113129964 wait complete. 2024-04-25T09:03:45.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.661+0000 7fe3c3d09640 1 Processor -- start 2024-04-25T09:03:45.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.661+0000 7fe3c3d09640 1 -- start start 2024-04-25T09:03:45.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.662+0000 7fe3c3d09640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe3bc06c510 0x7fe3bc1f7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.662+0000 7fe3c3d09640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3bc0b0870 0x7fe3bc154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.662+0000 7fe3c3d09640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe3bc15df70 0x7fe3bc15f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.662+0000 7fe3c3d09640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc0d0c10 con 0x7fe3bc0b0870 2024-04-25T09:03:45.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.662+0000 7fe3c3d09640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc0cc150 con 0x7fe3bc15df70 2024-04-25T09:03:45.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.662+0000 7fe3c3d09640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc0cecc0 con 0x7fe3bc06c510 2024-04-25T09:03:45.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.662+0000 7fe3c12a4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3bc0b0870 0x7fe3bc154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:45.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.662+0000 7fe3c12a4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3bc0b0870 0x7fe3bc154e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:49738/0 (socket says 172.21.15.43:49738) 2024-04-25T09:03:45.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.662+0000 7fe3c12a4640 1 -- 172.21.15.43:0/2039251040 learned_addr learned my addr 172.21.15.43:0/2039251040 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:45.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.662+0000 7fe3c1aa5640 1 --2- 172.21.15.43:0/2039251040 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe3bc06c510 0x7fe3bc1f7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:45.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.662+0000 7fe3c22a6640 1 --2- 172.21.15.43:0/2039251040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe3bc15df70 0x7fe3bc15f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:45.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.663+0000 7fe3c12a4640 1 -- 172.21.15.43:0/2039251040 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe3bc06c510 msgr2=0x7fe3bc1f7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.663+0000 7fe3c12a4640 1 --2- 172.21.15.43:0/2039251040 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe3bc06c510 0x7fe3bc1f7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.663+0000 7fe3c12a4640 1 -- 172.21.15.43:0/2039251040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe3bc15df70 msgr2=0x7fe3bc15f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.663+0000 7fe3c12a4640 1 --2- 172.21.15.43:0/2039251040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe3bc15df70 0x7fe3bc15f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.663+0000 7fe3c12a4640 1 -- 172.21.15.43:0/2039251040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3bc155390 con 0x7fe3bc0b0870 2024-04-25T09:03:45.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.663+0000 7fe3c1aa5640 1 --2- 172.21.15.43:0/2039251040 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe3bc06c510 0x7fe3bc1f7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:03:45.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.663+0000 7fe3c12a4640 1 --2- 172.21.15.43:0/2039251040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3bc0b0870 0x7fe3bc154e50 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac000c00 tx=0x7fe3ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:45.660 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.663+0000 7fe3b27fc640 1 -- 172.21.15.43:0/2039251040 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac041070 con 0x7fe3bc0b0870 2024-04-25T09:03:45.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.663+0000 7fe3b27fc640 1 -- 172.21.15.43:0/2039251040 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3ac03c890 con 0x7fe3bc0b0870 2024-04-25T09:03:45.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.663+0000 7fe3c3d09640 1 -- 172.21.15.43:0/2039251040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3bc15fe40 con 0x7fe3bc0b0870 2024-04-25T09:03:45.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.663+0000 7fe3b27fc640 1 -- 172.21.15.43:0/2039251040 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac037040 con 0x7fe3bc0b0870 2024-04-25T09:03:45.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.663+0000 7fe3c3d09640 1 -- 172.21.15.43:0/2039251040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3bc160400 con 0x7fe3bc0b0870 2024-04-25T09:03:45.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.664+0000 7fe3c3d09640 1 -- 172.21.15.43:0/2039251040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3bc074ef0 con 0x7fe3bc0b0870 2024-04-25T09:03:45.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.665+0000 7fe3b27fc640 1 -- 172.21.15.43:0/2039251040 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe3ac044070 con 0x7fe3bc0b0870 2024-04-25T09:03:45.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.665+0000 7fe3b27fc640 1 --2- 172.21.15.43:0/2039251040 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe3a006b4d0 0x7fe3a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:45.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.665+0000 7fe3b27fc640 1 -- 172.21.15.43:0/2039251040 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe3ac0aaad0 con 0x7fe3bc0b0870 2024-04-25T09:03:45.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.665+0000 7fe3c1aa5640 1 --2- 172.21.15.43:0/2039251040 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe3a006b4d0 0x7fe3a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:45.661 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.666+0000 7fe3c1aa5640 1 --2- 172.21.15.43:0/2039251040 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe3a006b4d0 0x7fe3a006d990 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc15cfa0 tx=0x7fe3b8009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:45.662 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.669+0000 7fe3b27fc640 1 -- 172.21.15.43:0/2039251040 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe3bc155390 con 0x7fe3bc0b0870 2024-04-25T09:03:45.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.886+0000 7fe3c3d09640 1 -- 172.21.15.43:0/2039251040 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3bc06c120 con 0x7fe3bc0b0870 2024-04-25T09:03:45.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.887+0000 7fe3b27fc640 1 -- 172.21.15.43:0/2039251040 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fe3ac0794b0 con 0x7fe3bc0b0870 2024-04-25T09:03:45.876 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:03:45.876 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/891225068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":891225068},{"type":"v1","addr":"172.21.15.193:6836","nonce":891225068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":7175,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3679059390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":3679059390},{"type":"v1","addr":"172.21.15.193:6837","nonce":3679059390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":7948,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2127363183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2127363183},{"type":"v1","addr":"172.21.15.43:6835","nonce":2127363183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:11.919337+0000","modified":"2024-04-25T09:03:13.917768+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9237},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9237":{"gid":9237,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6834/2603892772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2603892772},{"type":"v1","addr":"172.21.15.43:6834","nonce":2603892772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:03:45.876 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 76 2024-04-25T09:03:45.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.889+0000 7fe3c3d09640 1 -- 172.21.15.43:0/2039251040 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe3a006b4d0 msgr2=0x7fe3a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.889+0000 7fe3c3d09640 1 --2- 172.21.15.43:0/2039251040 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe3a006b4d0 0x7fe3a006d990 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc15cfa0 tx=0x7fe3b8009190 comp rx=0 tx=0).stop 2024-04-25T09:03:45.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.889+0000 7fe3c3d09640 1 -- 172.21.15.43:0/2039251040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3bc0b0870 msgr2=0x7fe3bc154e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:45.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.889+0000 7fe3c3d09640 1 --2- 172.21.15.43:0/2039251040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3bc0b0870 0x7fe3bc154e50 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac000c00 tx=0x7fe3ac0046c0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.889+0000 7fe3c3d09640 1 -- 172.21.15.43:0/2039251040 shutdown_connections 2024-04-25T09:03:45.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.889+0000 7fe3c3d09640 1 --2- 172.21.15.43:0/2039251040 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe3a006b4d0 0x7fe3a006d990 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.889+0000 7fe3c3d09640 1 --2- 172.21.15.43:0/2039251040 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe3bc15df70 0x7fe3bc15f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.889+0000 7fe3c3d09640 1 --2- 172.21.15.43:0/2039251040 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe3bc0b0870 0x7fe3bc154e50 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.889+0000 7fe3c3d09640 1 --2- 172.21.15.43:0/2039251040 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe3bc06c510 0x7fe3bc1f7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:45.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.889+0000 7fe3c3d09640 1 -- 172.21.15.43:0/2039251040 >> 172.21.15.43:0/2039251040 conn(0x7fe3bc0d17f0 msgr2=0x7fe3bc06d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:45.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.889+0000 7fe3c3d09640 1 -- 172.21.15.43:0/2039251040 shutdown_connections 2024-04-25T09:03:45.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:45.889+0000 7fe3c3d09640 1 -- 172.21.15.43:0/2039251040 wait complete. 2024-04-25T09:03:45.889 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T09:03:46.382 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:03:46.385 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:03:50.889 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:51.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.015+0000 7fbddd5b7640 1 Processor -- start 2024-04-25T09:03:51.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.015+0000 7fbddd5b7640 1 -- start start 2024-04-25T09:03:51.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.016+0000 7fbddd5b7640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fbdd815def0 0x7fbdd806be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:51.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.016+0000 7fbddd5b7640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdd815e2d0 con 0x7fbdd8162560 2024-04-25T09:03:51.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.016+0000 7fbddd5b7640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdd80b1020 con 0x7fbdd81619f0 2024-04-25T09:03:51.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.016+0000 7fbddd5b7640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdd80d0e90 con 0x7fbdd815def0 2024-04-25T09:03:51.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.016+0000 7fbdd77fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fbdd8162560 0x7fbdd80b20c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:47996/0 (socket says 172.21.15.43:47996) 2024-04-25T09:03:51.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.016+0000 7fbdd77fe640 1 -- 172.21.15.43:0/2780544368 learned_addr learned my addr 172.21.15.43:0/2780544368 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:03:51.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.016+0000 7fbdd6ffd640 1 --2- 172.21.15.43:0/2780544368 >> v2:172.21.15.193:3300/0 conn(0x7fbdd815def0 0x7fbdd806be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:51.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.016+0000 7fbdd67fc640 1 -- 172.21.15.43:0/2780544368 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3136712875 0 0) 0x7fbdd815e2d0 con 0x7fbdd8162560 2024-04-25T09:03:51.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.016+0000 7fbdd67fc640 1 -- 172.21.15.43:0/2780544368 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdc4003520 con 0x7fbdd8162560 2024-04-25T09:03:51.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.016+0000 7fbdd67fc640 1 -- 172.21.15.43:0/2780544368 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 112045063 0 0) 0x7fbdc4003520 con 0x7fbdd8162560 2024-04-25T09:03:51.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.016+0000 7fbdd67fc640 1 -- 172.21.15.43:0/2780544368 >> v2:172.21.15.193:3300/0 conn(0x7fbdd815def0 msgr2=0x7fbdd806be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:51.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.016+0000 7fbdd67fc640 1 --2- 172.21.15.43:0/2780544368 >> v2:172.21.15.193:3300/0 conn(0x7fbdd815def0 0x7fbdd806be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.016+0000 7fbdd67fc640 1 -- 172.21.15.43:0/2780544368 >> v1:172.21.15.193:6789/0 conn(0x7fbdd81619f0 legacy=0x7fbdd8161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:51.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.016+0000 7fbdd67fc640 1 -- 172.21.15.43:0/2780544368 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd80b1300 con 0x7fbdd8162560 2024-04-25T09:03:51.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.016+0000 7fbdd67fc640 1 -- 172.21.15.43:0/2780544368 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7fbdd00019f0 con 0x7fbdd8162560 2024-04-25T09:03:51.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.017+0000 7fbdd67fc640 1 -- 172.21.15.43:0/2780544368 >> v1:172.21.15.43:6789/0 conn(0x7fbdd8162560 legacy=0x7fbdd80b20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:51.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.017+0000 7fbdd67fc640 1 --2- 172.21.15.43:0/2780544368 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbdc40039b0 0x7fbdc4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:51.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.017+0000 7fbdd67fc640 1 --2- 172.21.15.43:0/2780544368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdc4004800 0x7fbdc4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:51.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.017+0000 7fbdd67fc640 1 --2- 172.21.15.43:0/2780544368 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbdc40051c0 0x7fbdc400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:51.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.017+0000 7fbdd67fc640 1 -- 172.21.15.43:0/2780544368 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd80d0e90 con 0x7fbdc4004800 2024-04-25T09:03:51.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.017+0000 7fbdd67fc640 1 -- 172.21.15.43:0/2780544368 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc400c060 con 0x7fbdc40051c0 2024-04-25T09:03:51.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.017+0000 7fbdd67fc640 1 -- 172.21.15.43:0/2780544368 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc400c260 con 0x7fbdc40039b0 2024-04-25T09:03:51.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.017+0000 7fbdcffff640 1 --2- 172.21.15.43:0/2780544368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdc4004800 0x7fbdc4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:51.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.017+0000 7fbdd6ffd640 1 --2- 172.21.15.43:0/2780544368 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbdc40039b0 0x7fbdc4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:51.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.017+0000 7fbdd77fe640 1 --2- 172.21.15.43:0/2780544368 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbdc40051c0 0x7fbdc400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:51.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.017+0000 7fbdcffff640 1 -- 172.21.15.43:0/2780544368 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbdc40039b0 msgr2=0x7fbdc4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:51.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.017+0000 7fbdcffff640 1 --2- 172.21.15.43:0/2780544368 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbdc40039b0 0x7fbdc4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.017+0000 7fbdcffff640 1 -- 172.21.15.43:0/2780544368 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbdc40051c0 msgr2=0x7fbdc400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:51.008 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.017+0000 7fbdcffff640 1 --2- 172.21.15.43:0/2780544368 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbdc40051c0 0x7fbdc400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.017+0000 7fbdcffff640 1 -- 172.21.15.43:0/2780544368 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc4003520 con 0x7fbdc4004800 2024-04-25T09:03:51.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.017+0000 7fbdd77fe640 1 -- 172.21.15.43:0/2780544368 reap_dead start 2024-04-25T09:03:51.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.018+0000 7fbdcffff640 1 --2- 172.21.15.43:0/2780544368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdc4004800 0x7fbdc4004c80 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0001770 tx=0x7fbdc4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=166faadc20b65042 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:51.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.018+0000 7fbdd67fc640 1 -- 172.21.15.43:0/2780544368 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc00013c0 con 0x7fbdc4004800 2024-04-25T09:03:51.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.018+0000 7fbdd67fc640 1 -- 172.21.15.43:0/2780544368 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdc00052d0 con 0x7fbdc4004800 2024-04-25T09:03:51.009 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.018+0000 7fbdd67fc640 1 -- 172.21.15.43:0/2780544368 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc002a8a0 con 0x7fbdc4004800 2024-04-25T09:03:51.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.019+0000 7fbddd5b7640 1 -- 172.21.15.43:0/2780544368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdc4004800 msgr2=0x7fbdc4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:51.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.019+0000 7fbddd5b7640 1 --2- 172.21.15.43:0/2780544368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdc4004800 0x7fbdc4004c80 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0001770 tx=0x7fbdc4005830 comp rx=0 tx=0).stop 2024-04-25T09:03:51.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.019+0000 7fbddd5b7640 1 -- 172.21.15.43:0/2780544368 shutdown_connections 2024-04-25T09:03:51.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.019+0000 7fbddd5b7640 1 --2- 172.21.15.43:0/2780544368 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdc4004800 0x7fbdc4004c80 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.019+0000 7fbddd5b7640 1 -- 172.21.15.43:0/2780544368 >> 172.21.15.43:0/2780544368 conn(0x7fbdd80b32e0 msgr2=0x7fbdd80b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:51.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.019+0000 7fbddd5b7640 1 -- 172.21.15.43:0/2780544368 shutdown_connections 2024-04-25T09:03:51.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.019+0000 7fbddd5b7640 1 -- 172.21.15.43:0/2780544368 wait complete. 2024-04-25T09:03:51.012 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.020+0000 7fbddd5b7640 1 Processor -- start 2024-04-25T09:03:51.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.020+0000 7fbddd5b7640 1 -- start start 2024-04-25T09:03:51.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.021+0000 7fbddd5b7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbdc4004800 0x7fbdd8155010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:51.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.021+0000 7fbddd5b7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdd80c70b0 0x7fbdd80c7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:51.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.021+0000 7fbddd5b7640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbdd806f510 0x7fbdd806f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:51.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.021+0000 7fbddd5b7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd80b1ba0 con 0x7fbdd80c70b0 2024-04-25T09:03:51.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.021+0000 7fbddd5b7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd80b18c0 con 0x7fbdc4004800 2024-04-25T09:03:51.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.021+0000 7fbddd5b7640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd80ae5c0 con 0x7fbdd806f510 2024-04-25T09:03:51.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.021+0000 7fbdcffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdd80c70b0 0x7fbdd80c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:51.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.021+0000 7fbdcffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdd80c70b0 0x7fbdd80c7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54950/0 (socket says 172.21.15.43:54950) 2024-04-25T09:03:51.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.021+0000 7fbdcffff640 1 -- 172.21.15.43:0/3803167316 learned_addr learned my addr 172.21.15.43:0/3803167316 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:51.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.021+0000 7fbdd77fe640 1 --2- 172.21.15.43:0/3803167316 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbdd806f510 0x7fbdd806f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:51.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.021+0000 7fbdcffff640 1 -- 172.21.15.43:0/3803167316 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbdd806f510 msgr2=0x7fbdd806f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:51.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.021+0000 7fbdd6ffd640 1 --2- 172.21.15.43:0/3803167316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbdc4004800 0x7fbdd8155010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:51.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.021+0000 7fbdcffff640 1 --2- 172.21.15.43:0/3803167316 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbdd806f510 0x7fbdd806f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.013 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.021+0000 7fbdcffff640 1 -- 172.21.15.43:0/3803167316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbdc4004800 msgr2=0x7fbdd8155010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:51.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.021+0000 7fbdcffff640 1 --2- 172.21.15.43:0/3803167316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbdc4004800 0x7fbdd8155010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.021+0000 7fbdcffff640 1 -- 172.21.15.43:0/3803167316 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd81e7890 con 0x7fbdd80c70b0 2024-04-25T09:03:51.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.021+0000 7fbdd6ffd640 1 --2- 172.21.15.43:0/3803167316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbdc4004800 0x7fbdd8155010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:51.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.021+0000 7fbdd77fe640 1 --2- 172.21.15.43:0/3803167316 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbdd806f510 0x7fbdd806f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:51.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.021+0000 7fbdcffff640 1 --2- 172.21.15.43:0/3803167316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdd80c70b0 0x7fbdd80c7530 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fbdc002a550 tx=0x7fbdc0001170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:51.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.022+0000 7fbdcf7fe640 1 -- 172.21.15.43:0/3803167316 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc003f030 con 0x7fbdd80c70b0 2024-04-25T09:03:51.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.022+0000 7fbdcf7fe640 1 -- 172.21.15.43:0/3803167316 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdc00050d0 con 0x7fbdd80c70b0 2024-04-25T09:03:51.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.022+0000 7fbddd5b7640 1 -- 172.21.15.43:0/3803167316 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdd81e88f0 con 0x7fbdd80c70b0 2024-04-25T09:03:51.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.022+0000 7fbdcf7fe640 1 -- 172.21.15.43:0/3803167316 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0037040 con 0x7fbdd80c70b0 2024-04-25T09:03:51.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.022+0000 7fbddd5b7640 1 -- 172.21.15.43:0/3803167316 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdd81e7e10 con 0x7fbdd80c70b0 2024-04-25T09:03:51.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.023+0000 7fbddd5b7640 1 -- 172.21.15.43:0/3803167316 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdd8074ef0 con 0x7fbdd80c70b0 2024-04-25T09:03:51.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.023+0000 7fbdcf7fe640 1 -- 172.21.15.43:0/3803167316 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbdc0042070 con 0x7fbdd80c70b0 2024-04-25T09:03:51.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.024+0000 7fbdcf7fe640 1 --2- 172.21.15.43:0/3803167316 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbdb806b4d0 0x7fbdb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:51.014 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.024+0000 7fbdcf7fe640 1 -- 172.21.15.43:0/3803167316 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbdc00a9910 con 0x7fbdd80c70b0 2024-04-25T09:03:51.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.024+0000 7fbdd6ffd640 1 --2- 172.21.15.43:0/3803167316 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbdb806b4d0 0x7fbdb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:51.015 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.025+0000 7fbdd6ffd640 1 --2- 172.21.15.43:0/3803167316 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbdb806b4d0 0x7fbdb806d990 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fbdd80b4420 tx=0x7fbdc8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:51.016 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.028+0000 7fbdcf7fe640 1 -- 172.21.15.43:0/3803167316 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbdc0049070 con 0x7fbdd80c70b0 2024-04-25T09:03:51.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.251+0000 7fbddd5b7640 1 -- 172.21.15.43:0/3803167316 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbdd815e2d0 con 0x7fbdd80c70b0 2024-04-25T09:03:51.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.252+0000 7fbdcf7fe640 1 -- 172.21.15.43:0/3803167316 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4943 (secure 0 0 0) 0x7fbdc0078790 con 0x7fbdd80c70b0 2024-04-25T09:03:51.241 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:03:51.241 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":76,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/891225068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":891225068},{"type":"v1","addr":"172.21.15.193:6836","nonce":891225068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":7175,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3679059390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":3679059390},{"type":"v1","addr":"172.21.15.193:6837","nonce":3679059390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":7948,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2127363183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2127363183},{"type":"v1","addr":"172.21.15.43:6835","nonce":2127363183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:11.919337+0000","modified":"2024-04-25T09:03:13.917768+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9237},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9237":{"gid":9237,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.43:6834/2603892772","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2603892772},{"type":"v1","addr":"172.21.15.43:6834","nonce":2603892772}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:03:51.241 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 76 2024-04-25T09:03:51.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.254+0000 7fbddd5b7640 1 -- 172.21.15.43:0/3803167316 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbdb806b4d0 msgr2=0x7fbdb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:51.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.254+0000 7fbddd5b7640 1 --2- 172.21.15.43:0/3803167316 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbdb806b4d0 0x7fbdb806d990 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fbdd80b4420 tx=0x7fbdc8008040 comp rx=0 tx=0).stop 2024-04-25T09:03:51.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.254+0000 7fbddd5b7640 1 -- 172.21.15.43:0/3803167316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdd80c70b0 msgr2=0x7fbdd80c7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:51.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.254+0000 7fbddd5b7640 1 --2- 172.21.15.43:0/3803167316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdd80c70b0 0x7fbdd80c7530 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fbdc002a550 tx=0x7fbdc0001170 comp rx=0 tx=0).stop 2024-04-25T09:03:51.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.254+0000 7fbddd5b7640 1 -- 172.21.15.43:0/3803167316 shutdown_connections 2024-04-25T09:03:51.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.254+0000 7fbddd5b7640 1 --2- 172.21.15.43:0/3803167316 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbdd806f510 0x7fbdd806f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.254+0000 7fbddd5b7640 1 --2- 172.21.15.43:0/3803167316 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbdd80c70b0 0x7fbdd80c7530 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.254+0000 7fbddd5b7640 1 --2- 172.21.15.43:0/3803167316 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbdb806b4d0 0x7fbdb806d990 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.254+0000 7fbddd5b7640 1 --2- 172.21.15.43:0/3803167316 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbdc4004800 0x7fbdd8155010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:51.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.254+0000 7fbddd5b7640 1 -- 172.21.15.43:0/3803167316 >> 172.21.15.43:0/3803167316 conn(0x7fbdd80b32e0 msgr2=0x7fbdd806c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:51.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.254+0000 7fbddd5b7640 1 -- 172.21.15.43:0/3803167316 shutdown_connections 2024-04-25T09:03:51.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:51.254+0000 7fbddd5b7640 1 -- 172.21.15.43:0/3803167316 wait complete. 2024-04-25T09:03:51.253 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T09:03:56.255 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:56.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.372+0000 7f609a4db640 1 Processor -- start 2024-04-25T09:03:56.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.372+0000 7f609a4db640 1 -- start start 2024-04-25T09:03:56.361 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.372+0000 7f609a4db640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f609415df50 0x7f60941619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.372+0000 7f609a4db640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6094161f30 0x7f6094162370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.372+0000 7f609a4db640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60940d24a0 con 0x7f60940c7cf0 2024-04-25T09:03:56.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.372+0000 7f609a4db640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f609415e330 con 0x7f6094161f30 2024-04-25T09:03:56.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.372+0000 7f609a4db640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60940d2740 con 0x7f609415df50 2024-04-25T09:03:56.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.373+0000 7f6093fff640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f609415df50 0x7f60941619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.373+0000 7f60937fe640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6094161f30 0x7f6094162370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.373+0000 7f6098a78640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f60940c7cf0 0x7f60940c8100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:54466/0 (socket says 172.21.15.43:54466) 2024-04-25T09:03:56.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.373+0000 7f60937fe640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6094161f30 0x7f6094162370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:54232/0 (socket says 172.21.15.43:54232) 2024-04-25T09:03:56.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.373+0000 7f6098a78640 1 -- 172.21.15.43:0/2802204382 learned_addr learned my addr 172.21.15.43:0/2802204382 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:03:56.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.373+0000 7f6093fff640 1 --2- 172.21.15.43:0/2802204382 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f609415df50 0x7f60941619f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:38166/0 (socket says 172.21.15.43:38166) 2024-04-25T09:03:56.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.373+0000 7f6092ffd640 1 -- 172.21.15.43:0/2802204382 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 652326385 0 0) 0x7f60940d24a0 con 0x7f60940c7cf0 2024-04-25T09:03:56.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.373+0000 7f6092ffd640 1 -- 172.21.15.43:0/2802204382 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f607c003520 con 0x7f60940c7cf0 2024-04-25T09:03:56.362 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.373+0000 7f6093fff640 1 -- 172.21.15.43:0/2802204382 >> v2:172.21.15.193:3300/0 conn(0x7f6094161f30 msgr2=0x7f6094162370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:56.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.373+0000 7f6093fff640 1 --2- 172.21.15.43:0/2802204382 >> v2:172.21.15.193:3300/0 conn(0x7f6094161f30 0x7f6094162370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.373+0000 7f6093fff640 1 -- 172.21.15.43:0/2802204382 >> v1:172.21.15.193:6789/0 conn(0x7f60940c7cf0 legacy=0x7f60940c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.373+0000 7f6093fff640 1 -- 172.21.15.43:0/2802204382 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60940d2a20 con 0x7f609415df50 2024-04-25T09:03:56.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.373+0000 7f60937fe640 1 --2- 172.21.15.43:0/2802204382 >> v2:172.21.15.193:3300/0 conn(0x7f6094161f30 0x7f6094162370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:03:56.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.374+0000 7f6093fff640 1 --2- 172.21.15.43:0/2802204382 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f609415df50 0x7f60941619f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f60840097a0 tx=0x7f608402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e51ae02e7a805615 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:56.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.374+0000 7f6092ffd640 1 -- 172.21.15.43:0/2802204382 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6084037070 con 0x7f609415df50 2024-04-25T09:03:56.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.374+0000 7f6092ffd640 1 -- 172.21.15.43:0/2802204382 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6084032440 con 0x7f609415df50 2024-04-25T09:03:56.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.374+0000 7f6092ffd640 1 -- 172.21.15.43:0/2802204382 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60840327f0 con 0x7f609415df50 2024-04-25T09:03:56.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.375+0000 7f609a4db640 1 -- 172.21.15.43:0/2802204382 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f609415df50 msgr2=0x7f60941619f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.375+0000 7f609a4db640 1 --2- 172.21.15.43:0/2802204382 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f609415df50 0x7f60941619f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f60840097a0 tx=0x7f608402a180 comp rx=0 tx=0).stop 2024-04-25T09:03:56.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.375+0000 7f609a4db640 1 -- 172.21.15.43:0/2802204382 shutdown_connections 2024-04-25T09:03:56.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.375+0000 7f609a4db640 1 --2- 172.21.15.43:0/2802204382 >> v2:172.21.15.193:3300/0 conn(0x7f6094161f30 0x7f6094162370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.375+0000 7f609a4db640 1 --2- 172.21.15.43:0/2802204382 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f609415df50 0x7f60941619f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.375+0000 7f609a4db640 1 -- 172.21.15.43:0/2802204382 >> 172.21.15.43:0/2802204382 conn(0x7f60940d3a30 msgr2=0x7f60940d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:56.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.375+0000 7f609a4db640 1 -- 172.21.15.43:0/2802204382 shutdown_connections 2024-04-25T09:03:56.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.375+0000 7f609a4db640 1 -- 172.21.15.43:0/2802204382 wait complete. 2024-04-25T09:03:56.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.376+0000 7f609a4db640 1 Processor -- start 2024-04-25T09:03:56.365 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.376+0000 7f609a4db640 1 -- start start 2024-04-25T09:03:56.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.377+0000 7f609a4db640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60940c7cf0 0x7f60941f38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.377+0000 7f609a4db640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f609415df50 0x7f609415c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.377+0000 7f609a4db640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6094161f30 0x7f609415cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.377+0000 7f609a4db640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60940d2190 con 0x7f60940c7cf0 2024-04-25T09:03:56.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.377+0000 7f609a4db640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60940c77d0 con 0x7f6094161f30 2024-04-25T09:03:56.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.377+0000 7f609a4db640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60940cf670 con 0x7f609415df50 2024-04-25T09:03:56.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.377+0000 7f6093fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60940c7cf0 0x7f60941f38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.377+0000 7f6093fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60940c7cf0 0x7f60941f38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54952/0 (socket says 172.21.15.43:54952) 2024-04-25T09:03:56.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.377+0000 7f6093fff640 1 -- 172.21.15.43:0/3973231519 learned_addr learned my addr 172.21.15.43:0/3973231519 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:56.367 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.377+0000 7f60937fe640 1 --2- 172.21.15.43:0/3973231519 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f609415df50 0x7f609415c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.377+0000 7f6098a78640 1 --2- 172.21.15.43:0/3973231519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6094161f30 0x7f609415cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.377+0000 7f6093fff640 1 -- 172.21.15.43:0/3973231519 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f609415df50 msgr2=0x7f609415c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.377+0000 7f6093fff640 1 --2- 172.21.15.43:0/3973231519 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f609415df50 0x7f609415c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.377+0000 7f6093fff640 1 -- 172.21.15.43:0/3973231519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6094161f30 msgr2=0x7f609415cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.377+0000 7f6093fff640 1 --2- 172.21.15.43:0/3973231519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6094161f30 0x7f609415cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.377+0000 7f6093fff640 1 -- 172.21.15.43:0/3973231519 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f609415ef00 con 0x7f60940c7cf0 2024-04-25T09:03:56.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.377+0000 7f6093fff640 1 --2- 172.21.15.43:0/3973231519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60940c7cf0 0x7f60941f38d0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f608402a1c0 tx=0x7f60840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:56.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.378+0000 7f6090ff9640 1 -- 172.21.15.43:0/3973231519 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6084037040 con 0x7f60940c7cf0 2024-04-25T09:03:56.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.378+0000 7f6090ff9640 1 -- 172.21.15.43:0/3973231519 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6084042070 con 0x7f60940c7cf0 2024-04-25T09:03:56.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.378+0000 7f609a4db640 1 -- 172.21.15.43:0/3973231519 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f609409e560 con 0x7f60940c7cf0 2024-04-25T09:03:56.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.378+0000 7f6090ff9640 1 -- 172.21.15.43:0/3973231519 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608403b930 con 0x7f60940c7cf0 2024-04-25T09:03:56.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.378+0000 7f609a4db640 1 -- 172.21.15.43:0/3973231519 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f609415f1c0 con 0x7f60940c7cf0 2024-04-25T09:03:56.368 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.379+0000 7f609a4db640 1 -- 172.21.15.43:0/3973231519 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60940680f0 con 0x7f60940c7cf0 2024-04-25T09:03:56.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.379+0000 7f6090ff9640 1 -- 172.21.15.43:0/3973231519 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f608403bad0 con 0x7f60940c7cf0 2024-04-25T09:03:56.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.380+0000 7f6090ff9640 1 --2- 172.21.15.43:0/3973231519 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f608006b4d0 0x7f608006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.380+0000 7f6090ff9640 1 -- 172.21.15.43:0/3973231519 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f60840aaac0 con 0x7f60940c7cf0 2024-04-25T09:03:56.369 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.380+0000 7f60937fe640 1 --2- 172.21.15.43:0/3973231519 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f608006b4d0 0x7f608006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.370 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.380+0000 7f60937fe640 1 --2- 172.21.15.43:0/3973231519 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f608006b4d0 0x7f608006d990 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6088009560 tx=0x7f6088009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:56.372 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.384+0000 7f6090ff9640 1 -- 172.21.15.43:0/3973231519 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6084079150 con 0x7f60940c7cf0 2024-04-25T09:03:56.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.604+0000 7f609a4db640 1 -- 172.21.15.43:0/3973231519 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6094074ef0 con 0x7f60940c7cf0 2024-04-25T09:03:56.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.605+0000 7f6090ff9640 1 -- 172.21.15.43:0/3973231519 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4155 (secure 0 0 0) 0x7f608407d1f0 con 0x7f60940c7cf0 2024-04-25T09:03:56.594 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:03:56.594 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":80,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/891225068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":891225068},{"type":"v1","addr":"172.21.15.193:6836","nonce":891225068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":7175,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3679059390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":3679059390},{"type":"v1","addr":"172.21.15.193:6837","nonce":3679059390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:11.919337+0000","modified":"2024-04-25T09:03:54.462418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7948},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7948":{"gid":7948,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":15,"addr":"172.21.15.43:6835/2127363183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2127363183},{"type":"v1","addr":"172.21.15.43:6835","nonce":2127363183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:03:56.594 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 80 2024-04-25T09:03:56.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.607+0000 7f609a4db640 1 -- 172.21.15.43:0/3973231519 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f608006b4d0 msgr2=0x7f608006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.607+0000 7f609a4db640 1 --2- 172.21.15.43:0/3973231519 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f608006b4d0 0x7f608006d990 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f6088009560 tx=0x7f6088009190 comp rx=0 tx=0).stop 2024-04-25T09:03:56.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.607+0000 7f609a4db640 1 -- 172.21.15.43:0/3973231519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60940c7cf0 msgr2=0x7f60941f38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.607+0000 7f609a4db640 1 --2- 172.21.15.43:0/3973231519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60940c7cf0 0x7f60941f38d0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f608402a1c0 tx=0x7f60840046c0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.607+0000 7f609a4db640 1 -- 172.21.15.43:0/3973231519 shutdown_connections 2024-04-25T09:03:56.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.607+0000 7f609a4db640 1 --2- 172.21.15.43:0/3973231519 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f608006b4d0 0x7f608006d990 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.607+0000 7f609a4db640 1 --2- 172.21.15.43:0/3973231519 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6094161f30 0x7f609415cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.607+0000 7f609a4db640 1 --2- 172.21.15.43:0/3973231519 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f609415df50 0x7f609415c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.607+0000 7f609a4db640 1 --2- 172.21.15.43:0/3973231519 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f60940c7cf0 0x7f60941f38d0 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.607+0000 7f609a4db640 1 -- 172.21.15.43:0/3973231519 >> 172.21.15.43:0/3973231519 conn(0x7f60940d3a30 msgr2=0x7f6094154010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:56.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.607+0000 7f609a4db640 1 -- 172.21.15.43:0/3973231519 shutdown_connections 2024-04-25T09:03:56.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.607+0000 7f609a4db640 1 -- 172.21.15.43:0/3973231519 wait complete. 2024-04-25T09:03:56.607 DEBUG:tasks.ceph_test_case:wait_until_true: success in 20s and 0 retries 2024-04-25T09:03:56.607 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:56.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.720+0000 7f04e1569640 1 Processor -- start 2024-04-25T09:03:56.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.721+0000 7f04e1569640 1 -- start start 2024-04-25T09:03:56.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.721+0000 7f04e1569640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f04dc15dcf0 0x7f04dc06e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.721+0000 7f04e1569640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f04dc06ebf0 0x7f04dc06f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.721+0000 7f04e1569640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04dc0d0d80 con 0x7f04dc1616e0 2024-04-25T09:03:56.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.721+0000 7f04e1569640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04dc06f590 con 0x7f04dc06ebf0 2024-04-25T09:03:56.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.721+0000 7f04e1569640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04dc06c050 con 0x7f04dc15dcf0 2024-04-25T09:03:56.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.721+0000 7f04db7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f04dc1616e0 0x7f04dc161b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48000/0 (socket says 172.21.15.43:48000) 2024-04-25T09:03:56.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.721+0000 7f04db7fe640 1 -- 172.21.15.43:0/4033980123 learned_addr learned my addr 172.21.15.43:0/4033980123 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:03:56.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.722+0000 7f04daffd640 1 --2- 172.21.15.43:0/4033980123 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f04dc15dcf0 0x7f04dc06e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.722+0000 7f04da7fc640 1 --2- 172.21.15.43:0/4033980123 >> v2:172.21.15.193:3300/0 conn(0x7f04dc06ebf0 0x7f04dc06f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.722+0000 7f04d9ffb640 1 -- 172.21.15.43:0/4033980123 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1600633224 0 0) 0x7f04dc0d0d80 con 0x7f04dc1616e0 2024-04-25T09:03:56.713 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.722+0000 7f04d9ffb640 1 -- 172.21.15.43:0/4033980123 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f04c4003520 con 0x7f04dc1616e0 2024-04-25T09:03:56.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.722+0000 7f04d9ffb640 1 -- 172.21.15.43:0/4033980123 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 359840418 0 0) 0x7f04c4003520 con 0x7f04dc1616e0 2024-04-25T09:03:56.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.722+0000 7f04d9ffb640 1 -- 172.21.15.43:0/4033980123 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f04dc15dcf0 msgr2=0x7f04dc06e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:56.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.722+0000 7f04d9ffb640 1 --2- 172.21.15.43:0/4033980123 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f04dc15dcf0 0x7f04dc06e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.722+0000 7f04d9ffb640 1 -- 172.21.15.43:0/4033980123 >> v2:172.21.15.193:3300/0 conn(0x7f04dc06ebf0 msgr2=0x7f04dc06f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:56.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.722+0000 7f04d9ffb640 1 --2- 172.21.15.43:0/4033980123 >> v2:172.21.15.193:3300/0 conn(0x7f04dc06ebf0 0x7f04dc06f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.722+0000 7f04d9ffb640 1 -- 172.21.15.43:0/4033980123 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04dc0d04e0 con 0x7f04dc1616e0 2024-04-25T09:03:56.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.722+0000 7f04d9ffb640 1 -- 172.21.15.43:0/4033980123 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f04d40019f0 con 0x7f04dc1616e0 2024-04-25T09:03:56.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.722+0000 7f04da7fc640 1 --2- 172.21.15.43:0/4033980123 >> v2:172.21.15.193:3300/0 conn(0x7f04dc06ebf0 0x7f04dc06f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:03:56.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.722+0000 7f04d9ffb640 1 -- 172.21.15.43:0/4033980123 >> v1:172.21.15.43:6789/0 conn(0x7f04dc1616e0 legacy=0x7f04dc161b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.722+0000 7f04d9ffb640 1 --2- 172.21.15.43:0/4033980123 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04c40039b0 0x7f04c4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.722+0000 7f04d9ffb640 1 --2- 172.21.15.43:0/4033980123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04c4004800 0x7f04c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.722+0000 7f04d9ffb640 1 --2- 172.21.15.43:0/4033980123 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f04dc15dcf0 0x7f04c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.714 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.722+0000 7f04d9ffb640 1 -- 172.21.15.43:0/4033980123 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04dc06c050 con 0x7f04c4004800 2024-04-25T09:03:56.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.722+0000 7f04d9ffb640 1 -- 172.21.15.43:0/4033980123 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c400b9c0 con 0x7f04c40039b0 2024-04-25T09:03:56.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.722+0000 7f04d9ffb640 1 -- 172.21.15.43:0/4033980123 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04c400bb40 con 0x7f04dc15dcf0 2024-04-25T09:03:56.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.722+0000 7f04da7fc640 1 --2- 172.21.15.43:0/4033980123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04c4004800 0x7f04c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.722+0000 7f04daffd640 1 --2- 172.21.15.43:0/4033980123 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04c40039b0 0x7f04c4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.722+0000 7f04db7fe640 1 --2- 172.21.15.43:0/4033980123 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f04dc15dcf0 0x7f04c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.723+0000 7f04da7fc640 1 -- 172.21.15.43:0/4033980123 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f04dc15dcf0 msgr2=0x7f04c40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:56.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.723+0000 7f04da7fc640 1 --2- 172.21.15.43:0/4033980123 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f04dc15dcf0 0x7f04c40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.723+0000 7f04da7fc640 1 -- 172.21.15.43:0/4033980123 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04c40039b0 msgr2=0x7f04c4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:56.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.723+0000 7f04da7fc640 1 --2- 172.21.15.43:0/4033980123 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04c40039b0 0x7f04c4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.723+0000 7f04da7fc640 1 -- 172.21.15.43:0/4033980123 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c4003520 con 0x7f04c4004800 2024-04-25T09:03:56.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.723+0000 7f04db7fe640 1 --2- 172.21.15.43:0/4033980123 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f04dc15dcf0 0x7f04c40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:56.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.723+0000 7f04da7fc640 1 --2- 172.21.15.43:0/4033980123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04c4004800 0x7f04c4004c80 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f04d0009cc0 tx=0x7f04c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ccea5a49281bd16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:56.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.723+0000 7f04d9ffb640 1 -- 172.21.15.43:0/4033980123 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d0036070 con 0x7f04c4004800 2024-04-25T09:03:56.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.723+0000 7f04d9ffb640 1 -- 172.21.15.43:0/4033980123 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04d0031400 con 0x7f04c4004800 2024-04-25T09:03:56.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.723+0000 7f04d9ffb640 1 -- 172.21.15.43:0/4033980123 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d0009860 con 0x7f04c4004800 2024-04-25T09:03:56.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.724+0000 7f04e1569640 1 -- 172.21.15.43:0/4033980123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04c4004800 msgr2=0x7f04c4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.724+0000 7f04e1569640 1 --2- 172.21.15.43:0/4033980123 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04c4004800 0x7f04c4004c80 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f04d0009cc0 tx=0x7f04c4005830 comp rx=0 tx=0).stop 2024-04-25T09:03:56.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.724+0000 7f04db7fe640 1 -- 172.21.15.43:0/4033980123 reap_dead start 2024-04-25T09:03:56.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.724+0000 7f04e1569640 1 -- 172.21.15.43:0/4033980123 shutdown_connections 2024-04-25T09:03:56.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.724+0000 7f04e1569640 1 -- 172.21.15.43:0/4033980123 >> 172.21.15.43:0/4033980123 conn(0x7f04dc0d1700 msgr2=0x7f04dc0d1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:56.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.724+0000 7f04e1569640 1 -- 172.21.15.43:0/4033980123 shutdown_connections 2024-04-25T09:03:56.716 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.724+0000 7f04e1569640 1 -- 172.21.15.43:0/4033980123 wait complete. 2024-04-25T09:03:56.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.725+0000 7f04e1569640 1 Processor -- start 2024-04-25T09:03:56.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.725+0000 7f04e1569640 1 -- start start 2024-04-25T09:03:56.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.726+0000 7f04e1569640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f04c4004800 0x7f04dc15ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.726+0000 7f04e1569640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04dc15f8a0 0x7f04dc0b5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.726+0000 7f04e1569640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04dc14d4a0 0x7f04dc14d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.726+0000 7f04e1569640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04dc06bce0 con 0x7f04dc14d4a0 2024-04-25T09:03:56.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.726+0000 7f04e1569640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04dc0d0aa0 con 0x7f04dc15f8a0 2024-04-25T09:03:56.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.726+0000 7f04e1569640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04dc0b0d50 con 0x7f04c4004800 2024-04-25T09:03:56.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.726+0000 7f04db7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04dc14d4a0 0x7f04dc14d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.726+0000 7f04db7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04dc14d4a0 0x7f04dc14d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54974/0 (socket says 172.21.15.43:54974) 2024-04-25T09:03:56.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.726+0000 7f04db7fe640 1 -- 172.21.15.43:0/646304106 learned_addr learned my addr 172.21.15.43:0/646304106 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:56.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.726+0000 7f04da7fc640 1 --2- 172.21.15.43:0/646304106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04dc15f8a0 0x7f04dc0b5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.719 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.726+0000 7f04daffd640 1 --2- 172.21.15.43:0/646304106 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f04c4004800 0x7f04dc15ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.726+0000 7f04db7fe640 1 -- 172.21.15.43:0/646304106 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f04c4004800 msgr2=0x7f04dc15ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.726+0000 7f04db7fe640 1 --2- 172.21.15.43:0/646304106 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f04c4004800 0x7f04dc15ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.726+0000 7f04db7fe640 1 -- 172.21.15.43:0/646304106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04dc15f8a0 msgr2=0x7f04dc0b5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.726+0000 7f04db7fe640 1 --2- 172.21.15.43:0/646304106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04dc15f8a0 0x7f04dc0b5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.726+0000 7f04db7fe640 1 -- 172.21.15.43:0/646304106 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04dc160600 con 0x7f04dc14d4a0 2024-04-25T09:03:56.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.727+0000 7f04db7fe640 1 --2- 172.21.15.43:0/646304106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04dc14d4a0 0x7f04dc14d950 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f04dc071ea0 tx=0x7f04d400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:56.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.727+0000 7f04c37fe640 1 -- 172.21.15.43:0/646304106 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d400eda0 con 0x7f04dc14d4a0 2024-04-25T09:03:56.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.727+0000 7f04c37fe640 1 -- 172.21.15.43:0/646304106 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04d4018070 con 0x7f04dc14d4a0 2024-04-25T09:03:56.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.727+0000 7f04c37fe640 1 -- 172.21.15.43:0/646304106 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04d4013620 con 0x7f04dc14d4a0 2024-04-25T09:03:56.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.727+0000 7f04e1569640 1 -- 172.21.15.43:0/646304106 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04dc15eae0 con 0x7f04dc14d4a0 2024-04-25T09:03:56.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.727+0000 7f04e1569640 1 -- 172.21.15.43:0/646304106 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04dc15f0a0 con 0x7f04dc14d4a0 2024-04-25T09:03:56.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.728+0000 7f04e1569640 1 -- 172.21.15.43:0/646304106 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04dc06dfc0 con 0x7f04dc14d4a0 2024-04-25T09:03:56.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.728+0000 7f04c37fe640 1 -- 172.21.15.43:0/646304106 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f04d40137c0 con 0x7f04dc14d4a0 2024-04-25T09:03:56.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.729+0000 7f04c37fe640 1 --2- 172.21.15.43:0/646304106 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f04c806b4d0 0x7f04c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:56.720 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.729+0000 7f04c37fe640 1 -- 172.21.15.43:0/646304106 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f04d408a640 con 0x7f04dc14d4a0 2024-04-25T09:03:56.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.729+0000 7f04daffd640 1 --2- 172.21.15.43:0/646304106 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f04c806b4d0 0x7f04c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:56.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.730+0000 7f04daffd640 1 --2- 172.21.15.43:0/646304106 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f04c806b4d0 0x7f04c806d990 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f04cc009900 tx=0x7f04cc008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:56.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.733+0000 7f04c37fe640 1 -- 172.21.15.43:0/646304106 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f04dc06dfc0 con 0x7f04dc14d4a0 2024-04-25T09:03:56.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.952+0000 7f04e1569640 1 -- 172.21.15.43:0/646304106 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f04dc067c10 con 0x7f04dc14d4a0 2024-04-25T09:03:56.940 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.952+0000 7f04c37fe640 1 -- 172.21.15.43:0/646304106 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4155 (secure 0 0 0) 0x7f04dc067c10 con 0x7f04dc14d4a0 2024-04-25T09:03:56.941 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:03:56.941 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":80,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/891225068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":891225068},{"type":"v1","addr":"172.21.15.193:6836","nonce":891225068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":7175,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3679059390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":3679059390},{"type":"v1","addr":"172.21.15.193:6837","nonce":3679059390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:11.919337+0000","modified":"2024-04-25T09:03:54.462418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7948},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7948":{"gid":7948,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":15,"addr":"172.21.15.43:6835/2127363183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2127363183},{"type":"v1","addr":"172.21.15.43:6835","nonce":2127363183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:03:56.941 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 80 2024-04-25T09:03:56.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.954+0000 7f04e1569640 1 -- 172.21.15.43:0/646304106 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f04c806b4d0 msgr2=0x7f04c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.954+0000 7f04e1569640 1 --2- 172.21.15.43:0/646304106 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f04c806b4d0 0x7f04c806d990 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f04cc009900 tx=0x7f04cc008040 comp rx=0 tx=0).stop 2024-04-25T09:03:56.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.954+0000 7f04e1569640 1 -- 172.21.15.43:0/646304106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04dc14d4a0 msgr2=0x7f04dc14d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:56.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.954+0000 7f04e1569640 1 --2- 172.21.15.43:0/646304106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04dc14d4a0 0x7f04dc14d950 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f04dc071ea0 tx=0x7f04d400e530 comp rx=0 tx=0).stop 2024-04-25T09:03:56.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.954+0000 7f04e1569640 1 -- 172.21.15.43:0/646304106 shutdown_connections 2024-04-25T09:03:56.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.954+0000 7f04e1569640 1 --2- 172.21.15.43:0/646304106 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f04dc14d4a0 0x7f04dc14d950 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.954+0000 7f04e1569640 1 --2- 172.21.15.43:0/646304106 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f04c806b4d0 0x7f04c806d990 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.954+0000 7f04e1569640 1 --2- 172.21.15.43:0/646304106 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f04dc15f8a0 0x7f04dc0b5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.954+0000 7f04e1569640 1 --2- 172.21.15.43:0/646304106 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f04c4004800 0x7f04dc15ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:56.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.954+0000 7f04e1569640 1 -- 172.21.15.43:0/646304106 >> 172.21.15.43:0/646304106 conn(0x7f04dc0d1700 msgr2=0x7f04dc0b1160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:56.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.954+0000 7f04e1569640 1 -- 172.21.15.43:0/646304106 shutdown_connections 2024-04-25T09:03:56.943 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:56.954+0000 7f04e1569640 1 -- 172.21.15.43:0/646304106 wait complete. 2024-04-25T09:03:56.954 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T09:03:56.954 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:03:57.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.065+0000 7fed9bc6f640 1 Processor -- start 2024-04-25T09:03:57.053 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.065+0000 7fed9bc6f640 1 -- start start 2024-04-25T09:03:57.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.066+0000 7fed9bc6f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fed9415ded0 0x7fed940ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.066+0000 7fed9bc6f640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fed940cf250 0x7fed940ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.066+0000 7fed9bc6f640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed940cf6e0 con 0x7fed9406ab30 2024-04-25T09:03:57.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.066+0000 7fed9bc6f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed9415e2b0 con 0x7fed9415ded0 2024-04-25T09:03:57.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.066+0000 7fed9bc6f640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed940cf9c0 con 0x7fed940cf250 2024-04-25T09:03:57.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.066+0000 7fed99a0b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fed9415ded0 0x7fed940ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.066+0000 7fed99a0b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fed9415ded0 0x7fed940ce740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54984/0 (socket says 172.21.15.43:54984) 2024-04-25T09:03:57.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.066+0000 7fed99a0b640 1 -- 172.21.15.43:0/465357735 learned_addr learned my addr 172.21.15.43:0/465357735 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:57.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.066+0000 7fed9920a640 1 --2- 172.21.15.43:0/465357735 >> v2:172.21.15.193:3300/0 conn(0x7fed940cf250 0x7fed940ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.066+0000 7fed99a0b640 1 -- 172.21.15.43:0/465357735 >> v2:172.21.15.193:3300/0 conn(0x7fed940cf250 msgr2=0x7fed940ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:57.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.066+0000 7fed99a0b640 1 --2- 172.21.15.43:0/465357735 >> v2:172.21.15.193:3300/0 conn(0x7fed940cf250 0x7fed940ad8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.066+0000 7fed99a0b640 1 -- 172.21.15.43:0/465357735 >> v1:172.21.15.193:6789/0 conn(0x7fed9406ab30 legacy=0x7fed940b4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.056 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.066+0000 7fed99a0b640 1 -- 172.21.15.43:0/465357735 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed940cfca0 con 0x7fed9415ded0 2024-04-25T09:03:57.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.067+0000 7fed99a0b640 1 --2- 172.21.15.43:0/465357735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed9415ded0 0x7fed940ce740 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7fed940ce6f0 tx=0x7fed9002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0ff049f69185b79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:57.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.067+0000 7fed98a09640 1 -- 172.21.15.43:0/465357735 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed90037070 con 0x7fed9415ded0 2024-04-25T09:03:57.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.067+0000 7fed98a09640 1 -- 172.21.15.43:0/465357735 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed90032440 con 0x7fed9415ded0 2024-04-25T09:03:57.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.067+0000 7fed98a09640 1 -- 172.21.15.43:0/465357735 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed90032810 con 0x7fed9415ded0 2024-04-25T09:03:57.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.068+0000 7fed9bc6f640 1 -- 172.21.15.43:0/465357735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed9415ded0 msgr2=0x7fed940ce740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.068+0000 7fed9bc6f640 1 --2- 172.21.15.43:0/465357735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed9415ded0 0x7fed940ce740 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7fed940ce6f0 tx=0x7fed9002a270 comp rx=0 tx=0).stop 2024-04-25T09:03:57.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.068+0000 7fed9bc6f640 1 -- 172.21.15.43:0/465357735 shutdown_connections 2024-04-25T09:03:57.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.068+0000 7fed9bc6f640 1 --2- 172.21.15.43:0/465357735 >> v2:172.21.15.193:3300/0 conn(0x7fed940cf250 0x7fed940ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.068+0000 7fed9bc6f640 1 --2- 172.21.15.43:0/465357735 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed9415ded0 0x7fed940ce740 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.068+0000 7fed9bc6f640 1 -- 172.21.15.43:0/465357735 >> 172.21.15.43:0/465357735 conn(0x7fed940b2490 msgr2=0x7fed940b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:57.057 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.068+0000 7fed9bc6f640 1 -- 172.21.15.43:0/465357735 shutdown_connections 2024-04-25T09:03:57.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.068+0000 7fed9bc6f640 1 -- 172.21.15.43:0/465357735 wait complete. 2024-04-25T09:03:57.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.069+0000 7fed9bc6f640 1 Processor -- start 2024-04-25T09:03:57.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.069+0000 7fed9bc6f640 1 -- start start 2024-04-25T09:03:57.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.070+0000 7fed9bc6f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed9406ab30 0x7fed94160d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.070+0000 7fed9bc6f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed9415ded0 0x7fed94154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.070+0000 7fed9bc6f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fed94154e50 0x7fed94155300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.070+0000 7fed9bc6f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed940cefd0 con 0x7fed9406ab30 2024-04-25T09:03:57.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.070+0000 7fed9bc6f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed940cee50 con 0x7fed9415ded0 2024-04-25T09:03:57.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.070+0000 7fed9bc6f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed9406afa0 con 0x7fed94154e50 2024-04-25T09:03:57.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.070+0000 7fed99a0b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed9406ab30 0x7fed94160d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.070+0000 7fed99a0b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed9406ab30 0x7fed94160d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:54986/0 (socket says 172.21.15.43:54986) 2024-04-25T09:03:57.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.070+0000 7fed99a0b640 1 -- 172.21.15.43:0/3821265158 learned_addr learned my addr 172.21.15.43:0/3821265158 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:57.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.070+0000 7fed9920a640 1 --2- 172.21.15.43:0/3821265158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed9415ded0 0x7fed94154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.070+0000 7fed9a20c640 1 --2- 172.21.15.43:0/3821265158 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fed94154e50 0x7fed94155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.070+0000 7fed99a0b640 1 -- 172.21.15.43:0/3821265158 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fed94154e50 msgr2=0x7fed94155300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.070+0000 7fed99a0b640 1 --2- 172.21.15.43:0/3821265158 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fed94154e50 0x7fed94155300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.070+0000 7fed99a0b640 1 -- 172.21.15.43:0/3821265158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed9415ded0 msgr2=0x7fed94154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.070+0000 7fed99a0b640 1 --2- 172.21.15.43:0/3821265158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed9415ded0 0x7fed94154910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.070+0000 7fed99a0b640 1 -- 172.21.15.43:0/3821265158 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed94161290 con 0x7fed9406ab30 2024-04-25T09:03:57.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.070+0000 7fed99a0b640 1 --2- 172.21.15.43:0/3821265158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed9406ab30 0x7fed94160d50 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7fed90000c00 tx=0x7fed900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:57.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.070+0000 7fed8a7fc640 1 -- 172.21.15.43:0/3821265158 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed90037040 con 0x7fed9406ab30 2024-04-25T09:03:57.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.070+0000 7fed8a7fc640 1 -- 172.21.15.43:0/3821265158 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed90042070 con 0x7fed9406ab30 2024-04-25T09:03:57.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.071+0000 7fed9bc6f640 1 -- 172.21.15.43:0/3821265158 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed94155840 con 0x7fed9406ab30 2024-04-25T09:03:57.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.071+0000 7fed8a7fc640 1 -- 172.21.15.43:0/3821265158 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed90030880 con 0x7fed9406ab30 2024-04-25T09:03:57.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.071+0000 7fed9bc6f640 1 -- 172.21.15.43:0/3821265158 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed9406e600 con 0x7fed9406ab30 2024-04-25T09:03:57.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.072+0000 7fed9bc6f640 1 -- 172.21.15.43:0/3821265158 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed9406b7b0 con 0x7fed9406ab30 2024-04-25T09:03:57.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.072+0000 7fed8a7fc640 1 -- 172.21.15.43:0/3821265158 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fed90030a20 con 0x7fed9406ab30 2024-04-25T09:03:57.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.073+0000 7fed8a7fc640 1 --2- 172.21.15.43:0/3821265158 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fed7806b4d0 0x7fed7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.073+0000 7fed8a7fc640 1 -- 172.21.15.43:0/3821265158 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fed900aaae0 con 0x7fed9406ab30 2024-04-25T09:03:57.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.073+0000 7fed9920a640 1 --2- 172.21.15.43:0/3821265158 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fed7806b4d0 0x7fed7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.073+0000 7fed9920a640 1 --2- 172.21.15.43:0/3821265158 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fed7806b4d0 0x7fed7806d990 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fed941f7390 tx=0x7fed84008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:57.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.077+0000 7fed8a7fc640 1 -- 172.21.15.43:0/3821265158 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fed9406b7b0 con 0x7fed9406ab30 2024-04-25T09:03:57.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.325+0000 7fed9bc6f640 1 -- 172.21.15.43:0/3821265158 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fed94074ef0 con 0x7fed9406ab30 2024-04-25T09:03:57.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.326+0000 7fed8a7fc640 1 -- 172.21.15.43:0/3821265158 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1442 (secure 0 0 0) 0x7fed94155840 con 0x7fed9406ab30 2024-04-25T09:03:57.315 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:03:57.315 INFO:teuthology.orchestra.run.smithi043.stdout:{"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":609,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":129,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035238,"num_in_osds":8,"osd_in_since":1714035229,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1077723995,"bytes_used":2689499136,"bytes_avail":770404614144,"bytes_total":773094113280,"read_bytes_sec":569160,"read_op_per_sec":2,"write_op_per_sec":0},"fsmap":{"epoch":80,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"c","status":"up:active","gid":7948}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2024-04-25T09:03:53.830554+0000","services":{"mds":{"daemons":{"summary":"","b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:03:57.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.328+0000 7fed9bc6f640 1 -- 172.21.15.43:0/3821265158 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fed7806b4d0 msgr2=0x7fed7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.328+0000 7fed9bc6f640 1 --2- 172.21.15.43:0/3821265158 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fed7806b4d0 0x7fed7806d990 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fed941f7390 tx=0x7fed84008040 comp rx=0 tx=0).stop 2024-04-25T09:03:57.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.328+0000 7fed9bc6f640 1 -- 172.21.15.43:0/3821265158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed9406ab30 msgr2=0x7fed94160d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.328+0000 7fed9bc6f640 1 --2- 172.21.15.43:0/3821265158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed9406ab30 0x7fed94160d50 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7fed90000c00 tx=0x7fed900046c0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.328+0000 7fed9bc6f640 1 -- 172.21.15.43:0/3821265158 shutdown_connections 2024-04-25T09:03:57.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.328+0000 7fed9bc6f640 1 --2- 172.21.15.43:0/3821265158 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fed94154e50 0x7fed94155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.328+0000 7fed9bc6f640 1 --2- 172.21.15.43:0/3821265158 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fed7806b4d0 0x7fed7806d990 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.328+0000 7fed9bc6f640 1 --2- 172.21.15.43:0/3821265158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed9415ded0 0x7fed94154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.328+0000 7fed9bc6f640 1 --2- 172.21.15.43:0/3821265158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed9406ab30 0x7fed94160d50 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.328+0000 7fed9bc6f640 1 -- 172.21.15.43:0/3821265158 >> 172.21.15.43:0/3821265158 conn(0x7fed940b2490 msgr2=0x7fed9415beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:57.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.328+0000 7fed9bc6f640 1 -- 172.21.15.43:0/3821265158 shutdown_connections 2024-04-25T09:03:57.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.328+0000 7fed9bc6f640 1 -- 172.21.15.43:0/3821265158 wait complete. 2024-04-25T09:03:57.327 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T09:03:57.328 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:03:57.328 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:03:57.378 DEBUG:teuthology.orchestra.run.smithi193:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:03:57.421 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:03:57.421 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T09:03:57.421 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:03:57.437 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:57.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.590+0000 7ff956c3f640 1 Processor -- start 2024-04-25T09:03:57.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.590+0000 7ff956c3f640 1 -- start start 2024-04-25T09:03:57.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.591+0000 7ff956c3f640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff9500715a0 0x7ff950071a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.591+0000 7ff956c3f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff95006dd30 con 0x7ff9500b3100 2024-04-25T09:03:57.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.591+0000 7ff956c3f640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff95006e010 con 0x7ff95015a250 2024-04-25T09:03:57.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.591+0000 7ff956c3f640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9500b2900 con 0x7ff9500715a0 2024-04-25T09:03:57.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.591+0000 7ff9551dc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff9500b3100 0x7ff9500b3510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48008/0 (socket says 172.21.15.43:48008) 2024-04-25T09:03:57.585 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.591+0000 7ff9551dc640 1 -- 172.21.15.43:0/747600169 learned_addr learned my addr 172.21.15.43:0/747600169 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:03:57.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.591+0000 7ff94f7fe640 1 -- 172.21.15.43:0/747600169 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1641140320 0 0) 0x7ff95006dd30 con 0x7ff9500b3100 2024-04-25T09:03:57.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.591+0000 7ff94f7fe640 1 -- 172.21.15.43:0/747600169 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff940003520 con 0x7ff9500b3100 2024-04-25T09:03:57.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.591+0000 7ff94ffff640 1 --2- 172.21.15.43:0/747600169 >> v2:172.21.15.193:3300/0 conn(0x7ff9500715a0 0x7ff950071a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.591+0000 7ff94f7fe640 1 -- 172.21.15.43:0/747600169 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 471814345 0 0) 0x7ff940003520 con 0x7ff9500b3100 2024-04-25T09:03:57.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.591+0000 7ff94f7fe640 1 -- 172.21.15.43:0/747600169 >> v2:172.21.15.193:3300/0 conn(0x7ff9500715a0 msgr2=0x7ff950071a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:57.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.591+0000 7ff94f7fe640 1 --2- 172.21.15.43:0/747600169 >> v2:172.21.15.193:3300/0 conn(0x7ff9500715a0 0x7ff950071a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.591+0000 7ff94f7fe640 1 -- 172.21.15.43:0/747600169 >> v1:172.21.15.193:6789/0 conn(0x7ff95015a250 legacy=0x7ff950070d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.591+0000 7ff94f7fe640 1 -- 172.21.15.43:0/747600169 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9500b2be0 con 0x7ff9500b3100 2024-04-25T09:03:57.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.592+0000 7ff94f7fe640 1 -- 172.21.15.43:0/747600169 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7ff9480019f0 con 0x7ff9500b3100 2024-04-25T09:03:57.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.592+0000 7ff94f7fe640 1 -- 172.21.15.43:0/747600169 >> v1:172.21.15.43:6789/0 conn(0x7ff9500b3100 legacy=0x7ff9500b3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.592+0000 7ff94f7fe640 1 --2- 172.21.15.43:0/747600169 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff9400039b0 0x7ff940003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.592+0000 7ff94f7fe640 1 --2- 172.21.15.43:0/747600169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff940004800 0x7ff940004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.586 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.592+0000 7ff94f7fe640 1 --2- 172.21.15.43:0/747600169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9400051c0 0x7ff94000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.592+0000 7ff94f7fe640 1 -- 172.21.15.43:0/747600169 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9500b2900 con 0x7ff940004800 2024-04-25T09:03:57.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.592+0000 7ff94f7fe640 1 -- 172.21.15.43:0/747600169 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94000e270 con 0x7ff9400051c0 2024-04-25T09:03:57.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.592+0000 7ff94f7fe640 1 -- 172.21.15.43:0/747600169 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff94000e470 con 0x7ff9400039b0 2024-04-25T09:03:57.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.592+0000 7ff94ffff640 1 --2- 172.21.15.43:0/747600169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff940004800 0x7ff940004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.592+0000 7ff9549db640 1 --2- 172.21.15.43:0/747600169 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff9400039b0 0x7ff940003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.592+0000 7ff9551dc640 1 --2- 172.21.15.43:0/747600169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9400051c0 0x7ff94000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.592+0000 7ff94ffff640 1 -- 172.21.15.43:0/747600169 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff9400039b0 msgr2=0x7ff940003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:57.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.592+0000 7ff94ffff640 1 --2- 172.21.15.43:0/747600169 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff9400039b0 0x7ff940003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.592+0000 7ff94ffff640 1 -- 172.21.15.43:0/747600169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9400051c0 msgr2=0x7ff94000dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:57.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.592+0000 7ff94ffff640 1 --2- 172.21.15.43:0/747600169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9400051c0 0x7ff94000dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.592+0000 7ff94ffff640 1 -- 172.21.15.43:0/747600169 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff940003520 con 0x7ff940004800 2024-04-25T09:03:57.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.592+0000 7ff9549db640 1 --2- 172.21.15.43:0/747600169 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff9400039b0 0x7ff940003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:57.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.592+0000 7ff9551dc640 1 --2- 172.21.15.43:0/747600169 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff9400051c0 0x7ff94000dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:57.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.592+0000 7ff9551dc640 1 -- 172.21.15.43:0/747600169 reap_dead start 2024-04-25T09:03:57.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.593+0000 7ff94ffff640 1 --2- 172.21.15.43:0/747600169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff940004800 0x7ff940004c80 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7ff93c009ac0 tx=0x7ff940005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35dbb93f0180c05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:57.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.593+0000 7ff94f7fe640 1 -- 172.21.15.43:0/747600169 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93c036070 con 0x7ff940004800 2024-04-25T09:03:57.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.593+0000 7ff94f7fe640 1 -- 172.21.15.43:0/747600169 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff93c031400 con 0x7ff940004800 2024-04-25T09:03:57.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.593+0000 7ff94f7fe640 1 -- 172.21.15.43:0/747600169 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93c009690 con 0x7ff940004800 2024-04-25T09:03:57.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.593+0000 7ff956c3f640 1 -- 172.21.15.43:0/747600169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff940004800 msgr2=0x7ff940004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.593+0000 7ff956c3f640 1 --2- 172.21.15.43:0/747600169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff940004800 0x7ff940004c80 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7ff93c009ac0 tx=0x7ff940005830 comp rx=0 tx=0).stop 2024-04-25T09:03:57.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.594+0000 7ff956c3f640 1 -- 172.21.15.43:0/747600169 shutdown_connections 2024-04-25T09:03:57.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.594+0000 7ff956c3f640 1 --2- 172.21.15.43:0/747600169 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff940004800 0x7ff940004c80 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.594+0000 7ff956c3f640 1 -- 172.21.15.43:0/747600169 >> 172.21.15.43:0/747600169 conn(0x7ff9500b1480 msgr2=0x7ff9500b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:57.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.594+0000 7ff956c3f640 1 -- 172.21.15.43:0/747600169 shutdown_connections 2024-04-25T09:03:57.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.594+0000 7ff956c3f640 1 -- 172.21.15.43:0/747600169 wait complete. 2024-04-25T09:03:57.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.595+0000 7ff956c3f640 1 Processor -- start 2024-04-25T09:03:57.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.595+0000 7ff956c3f640 1 -- start start 2024-04-25T09:03:57.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.595+0000 7ff956c3f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff940004800 0x7ff950145640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.596+0000 7ff956c3f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff950155470 0x7ff9500a9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.596+0000 7ff956c3f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff9500ac710 0x7ff9500acb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.596+0000 7ff956c3f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff950071b30 con 0x7ff950155470 2024-04-25T09:03:57.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.596+0000 7ff956c3f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9500719b0 con 0x7ff940004800 2024-04-25T09:03:57.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.596+0000 7ff956c3f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff950071cb0 con 0x7ff9500ac710 2024-04-25T09:03:57.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.596+0000 7ff94ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff950155470 0x7ff9500a9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.596+0000 7ff94ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff950155470 0x7ff9500a9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55006/0 (socket says 172.21.15.43:55006) 2024-04-25T09:03:57.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.596+0000 7ff94ffff640 1 -- 172.21.15.43:0/1605108970 learned_addr learned my addr 172.21.15.43:0/1605108970 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:57.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.596+0000 7ff9549db640 1 --2- 172.21.15.43:0/1605108970 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff940004800 0x7ff950145640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.596+0000 7ff9551dc640 1 --2- 172.21.15.43:0/1605108970 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff9500ac710 0x7ff9500acb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.596+0000 7ff94ffff640 1 -- 172.21.15.43:0/1605108970 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff9500ac710 msgr2=0x7ff9500acb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.596+0000 7ff94ffff640 1 --2- 172.21.15.43:0/1605108970 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff9500ac710 0x7ff9500acb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.596+0000 7ff94ffff640 1 -- 172.21.15.43:0/1605108970 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff940004800 msgr2=0x7ff950145640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.596+0000 7ff94ffff640 1 --2- 172.21.15.43:0/1605108970 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff940004800 0x7ff950145640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.596+0000 7ff94ffff640 1 -- 172.21.15.43:0/1605108970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff950145e40 con 0x7ff950155470 2024-04-25T09:03:57.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.596+0000 7ff9549db640 1 --2- 172.21.15.43:0/1605108970 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff940004800 0x7ff950145640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:57.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.596+0000 7ff94ffff640 1 --2- 172.21.15.43:0/1605108970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff950155470 0x7ff9500a9950 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7ff93c034040 tx=0x7ff93c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:57.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.596+0000 7ff94d7fa640 1 -- 172.21.15.43:0/1605108970 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93c03f020 con 0x7ff950155470 2024-04-25T09:03:57.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.596+0000 7ff94d7fa640 1 -- 172.21.15.43:0/1605108970 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff93c037040 con 0x7ff950155470 2024-04-25T09:03:57.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.596+0000 7ff956c3f640 1 -- 172.21.15.43:0/1605108970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9501460e0 con 0x7ff950155470 2024-04-25T09:03:57.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.596+0000 7ff94d7fa640 1 -- 172.21.15.43:0/1605108970 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93c036070 con 0x7ff950155470 2024-04-25T09:03:57.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.596+0000 7ff956c3f640 1 -- 172.21.15.43:0/1605108970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9501466a0 con 0x7ff950155470 2024-04-25T09:03:57.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.597+0000 7ff956c3f640 1 -- 172.21.15.43:0/1605108970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff950074ef0 con 0x7ff950155470 2024-04-25T09:03:57.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.598+0000 7ff94d7fa640 1 -- 172.21.15.43:0/1605108970 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff93c042020 con 0x7ff950155470 2024-04-25T09:03:57.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.598+0000 7ff94d7fa640 1 --2- 172.21.15.43:0/1605108970 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff93406b4d0 0x7ff93406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.598+0000 7ff94d7fa640 1 -- 172.21.15.43:0/1605108970 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5853+0+0 (secure 0 0 0) 0x7ff93c0aa300 con 0x7ff950155470 2024-04-25T09:03:57.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.598+0000 7ff9549db640 1 --2- 172.21.15.43:0/1605108970 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff93406b4d0 0x7ff93406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.599+0000 7ff9549db640 1 --2- 172.21.15.43:0/1605108970 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff93406b4d0 0x7ff93406d990 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7ff944007fd0 tx=0x7ff944009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:57.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.603+0000 7ff94d7fa640 1 -- 172.21.15.43:0/1605108970 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff93c049030 con 0x7ff950155470 2024-04-25T09:03:57.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.788+0000 7ff956c3f640 1 -- 172.21.15.43:0/1605108970 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9500ae8e0 con 0x7ff950155470 2024-04-25T09:03:57.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.789+0000 7ff94d7fa640 1 -- 172.21.15.43:0/1605108970 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) v1 ==== 74+0+16814 (secure 0 0 0) 0x7ff93c079240 con 0x7ff950155470 2024-04-25T09:03:57.777 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:03:57.778 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":129,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:03:55.660750+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:03:10.199822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T09:03:10.892664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.31","mappings":[{"from":2,"to":0}]},{"pgid":"19.3c","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6834/2603892772":"2024-04-26T09:03:52.443908+0000","172.21.15.43:6832/2603892772":"2024-04-26T09:03:52.443908+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:03:57.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.791+0000 7ff956c3f640 1 -- 172.21.15.43:0/1605108970 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff93406b4d0 msgr2=0x7ff93406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.791+0000 7ff956c3f640 1 --2- 172.21.15.43:0/1605108970 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff93406b4d0 0x7ff93406d990 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7ff944007fd0 tx=0x7ff944009040 comp rx=0 tx=0).stop 2024-04-25T09:03:57.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.791+0000 7ff956c3f640 1 -- 172.21.15.43:0/1605108970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff950155470 msgr2=0x7ff9500a9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.791+0000 7ff956c3f640 1 --2- 172.21.15.43:0/1605108970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff950155470 0x7ff9500a9950 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7ff93c034040 tx=0x7ff93c004730 comp rx=0 tx=0).stop 2024-04-25T09:03:57.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.791+0000 7ff956c3f640 1 -- 172.21.15.43:0/1605108970 shutdown_connections 2024-04-25T09:03:57.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.791+0000 7ff956c3f640 1 --2- 172.21.15.43:0/1605108970 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff9500ac710 0x7ff9500acb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.791+0000 7ff956c3f640 1 --2- 172.21.15.43:0/1605108970 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff950155470 0x7ff9500a9950 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.791+0000 7ff956c3f640 1 --2- 172.21.15.43:0/1605108970 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff93406b4d0 0x7ff93406d990 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.791+0000 7ff956c3f640 1 --2- 172.21.15.43:0/1605108970 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff940004800 0x7ff950145640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.791+0000 7ff956c3f640 1 -- 172.21.15.43:0/1605108970 >> 172.21.15.43:0/1605108970 conn(0x7ff9500b1480 msgr2=0x7ff9500cb970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:57.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.791+0000 7ff956c3f640 1 -- 172.21.15.43:0/1605108970 shutdown_connections 2024-04-25T09:03:57.780 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.791+0000 7ff956c3f640 1 -- 172.21.15.43:0/1605108970 wait complete. 2024-04-25T09:03:57.792 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:03:10.199822+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '125', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:03:10.892664+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:03:57.792 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:03:57.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.907+0000 7f08fb3c8640 1 Processor -- start 2024-04-25T09:03:57.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.908+0000 7f08fb3c8640 1 -- start start 2024-04-25T09:03:57.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.908+0000 7f08fb3c8640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f08f415e090 0x7f08f4161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.908+0000 7f08fb3c8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f08f4161fe0 0x7f08f40b5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.908+0000 7f08fb3c8640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08f4162610 con 0x7f08f40b5aa0 2024-04-25T09:03:57.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.908+0000 7f08fb3c8640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08f40d1da0 con 0x7f08f4161fe0 2024-04-25T09:03:57.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.908+0000 7f08fb3c8640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08f40d1750 con 0x7f08f415e090 2024-04-25T09:03:57.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.909+0000 7f08f8963640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f08f4161fe0 0x7f08f40b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.909+0000 7f08f9965640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f08f40b5aa0 0x7f08f40ce560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48010/0 (socket says 172.21.15.43:48010) 2024-04-25T09:03:57.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.909+0000 7f08f9965640 1 -- 172.21.15.43:0/4293333180 learned_addr learned my addr 172.21.15.43:0/4293333180 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:03:57.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.909+0000 7f08f9164640 1 --2- 172.21.15.43:0/4293333180 >> v2:172.21.15.193:3300/0 conn(0x7f08f415e090 0x7f08f4161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.909+0000 7f08f8963640 1 -- 172.21.15.43:0/4293333180 >> v2:172.21.15.193:3300/0 conn(0x7f08f415e090 msgr2=0x7f08f4161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:57.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.909+0000 7f08f8963640 1 --2- 172.21.15.43:0/4293333180 >> v2:172.21.15.193:3300/0 conn(0x7f08f415e090 0x7f08f4161aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.909+0000 7f08f8963640 1 -- 172.21.15.43:0/4293333180 >> v1:172.21.15.43:6789/0 conn(0x7f08f40b5aa0 legacy=0x7f08f40ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.909+0000 7f08f8963640 1 -- 172.21.15.43:0/4293333180 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f40d1a30 con 0x7f08f4161fe0 2024-04-25T09:03:57.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.909+0000 7f08f9164640 1 --2- 172.21.15.43:0/4293333180 >> v2:172.21.15.193:3300/0 conn(0x7f08f415e090 0x7f08f4161aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:57.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.910+0000 7f08f8963640 1 --2- 172.21.15.43:0/4293333180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08f4161fe0 0x7f08f40b5560 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f08f4062920 tx=0x7f08e802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eac5dc471cfff99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:57.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.910+0000 7f08f3fff640 1 -- 172.21.15.43:0/4293333180 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e8037070 con 0x7f08f4161fe0 2024-04-25T09:03:57.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.910+0000 7f08f3fff640 1 -- 172.21.15.43:0/4293333180 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e8032440 con 0x7f08f4161fe0 2024-04-25T09:03:57.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.910+0000 7f08f3fff640 1 -- 172.21.15.43:0/4293333180 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e8032810 con 0x7f08f4161fe0 2024-04-25T09:03:57.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.910+0000 7f08fb3c8640 1 -- 172.21.15.43:0/4293333180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08f4161fe0 msgr2=0x7f08f40b5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.910+0000 7f08fb3c8640 1 --2- 172.21.15.43:0/4293333180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08f4161fe0 0x7f08f40b5560 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f08f4062920 tx=0x7f08e802a250 comp rx=0 tx=0).stop 2024-04-25T09:03:57.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.910+0000 7f08fb3c8640 1 -- 172.21.15.43:0/4293333180 shutdown_connections 2024-04-25T09:03:57.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.910+0000 7f08fb3c8640 1 --2- 172.21.15.43:0/4293333180 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08f4161fe0 0x7f08f40b5560 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.910+0000 7f08fb3c8640 1 --2- 172.21.15.43:0/4293333180 >> v2:172.21.15.193:3300/0 conn(0x7f08f415e090 0x7f08f4161aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.910+0000 7f08fb3c8640 1 -- 172.21.15.43:0/4293333180 >> 172.21.15.43:0/4293333180 conn(0x7f08f40d0560 msgr2=0x7f08f40d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:57.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.910+0000 7f08fb3c8640 1 -- 172.21.15.43:0/4293333180 shutdown_connections 2024-04-25T09:03:57.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.910+0000 7f08fb3c8640 1 -- 172.21.15.43:0/4293333180 wait complete. 2024-04-25T09:03:57.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.912+0000 7f08fb3c8640 1 Processor -- start 2024-04-25T09:03:57.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.912+0000 7f08fb3c8640 1 -- start start 2024-04-25T09:03:57.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.912+0000 7f08fb3c8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08f40b5aa0 0x7f08f414bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.912+0000 7f08fb3c8640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08f415e090 0x7f08f41e8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.912+0000 7f08fb3c8640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08f4161fe0 0x7f08f40d2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.912+0000 7f08fb3c8640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f40d3240 con 0x7f08f40b5aa0 2024-04-25T09:03:57.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.912+0000 7f08fb3c8640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f40b3280 con 0x7f08f4161fe0 2024-04-25T09:03:57.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.912+0000 7f08fb3c8640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f40d33c0 con 0x7f08f415e090 2024-04-25T09:03:57.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.912+0000 7f08f9164640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08f40b5aa0 0x7f08f414bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.912+0000 7f08f9164640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08f40b5aa0 0x7f08f414bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55026/0 (socket says 172.21.15.43:55026) 2024-04-25T09:03:57.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.912+0000 7f08f9164640 1 -- 172.21.15.43:0/3741142651 learned_addr learned my addr 172.21.15.43:0/3741142651 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:57.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.913+0000 7f08f9965640 1 --2- 172.21.15.43:0/3741142651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08f4161fe0 0x7f08f40d2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.913+0000 7f08f8963640 1 --2- 172.21.15.43:0/3741142651 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08f415e090 0x7f08f41e8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.913+0000 7f08f9164640 1 -- 172.21.15.43:0/3741142651 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08f415e090 msgr2=0x7f08f41e8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.913+0000 7f08f9164640 1 --2- 172.21.15.43:0/3741142651 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08f415e090 0x7f08f41e8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.913+0000 7f08f9164640 1 -- 172.21.15.43:0/3741142651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08f4161fe0 msgr2=0x7f08f40d2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:57.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.913+0000 7f08f9164640 1 --2- 172.21.15.43:0/3741142651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08f4161fe0 0x7f08f40d2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:57.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.913+0000 7f08f9164640 1 -- 172.21.15.43:0/3741142651 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f40c76b0 con 0x7f08f40b5aa0 2024-04-25T09:03:57.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.913+0000 7f08f9965640 1 --2- 172.21.15.43:0/3741142651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08f4161fe0 0x7f08f40d2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:03:57.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.913+0000 7f08f9164640 1 --2- 172.21.15.43:0/3741142651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08f40b5aa0 0x7f08f414bf70 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f08e400cf60 tx=0x7f08e400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:57.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.913+0000 7f08f1ffb640 1 -- 172.21.15.43:0/3741142651 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e401a070 con 0x7f08f40b5aa0 2024-04-25T09:03:57.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.913+0000 7f08f1ffb640 1 -- 172.21.15.43:0/3741142651 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e400d840 con 0x7f08f40b5aa0 2024-04-25T09:03:57.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.913+0000 7f08fb3c8640 1 -- 172.21.15.43:0/3741142651 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08f40c7990 con 0x7f08f40b5aa0 2024-04-25T09:03:57.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.913+0000 7f08f1ffb640 1 -- 172.21.15.43:0/3741142651 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e400dc10 con 0x7f08f40b5aa0 2024-04-25T09:03:57.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.913+0000 7f08fb3c8640 1 -- 172.21.15.43:0/3741142651 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08f40c7f50 con 0x7f08f40b5aa0 2024-04-25T09:03:57.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.914+0000 7f08fb3c8640 1 -- 172.21.15.43:0/3741142651 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08f406bf00 con 0x7f08f40b5aa0 2024-04-25T09:03:57.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.915+0000 7f08f1ffb640 1 -- 172.21.15.43:0/3741142651 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f08e401d070 con 0x7f08f40b5aa0 2024-04-25T09:03:57.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.915+0000 7f08f1ffb640 1 --2- 172.21.15.43:0/3741142651 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f08e006b4d0 0x7f08e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:57.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.915+0000 7f08f1ffb640 1 -- 172.21.15.43:0/3741142651 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f08e408ab30 con 0x7f08f40b5aa0 2024-04-25T09:03:57.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.915+0000 7f08f8963640 1 --2- 172.21.15.43:0/3741142651 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f08e006b4d0 0x7f08e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:57.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.916+0000 7f08f8963640 1 --2- 172.21.15.43:0/3741142651 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f08e006b4d0 0x7f08e006d990 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f08e8034000 tx=0x7f08e802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:57.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:57.919+0000 7f08f1ffb640 1 -- 172.21.15.43:0/3741142651 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f08f406bf00 con 0x7f08f40b5aa0 2024-04-25T09:03:58.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.101+0000 7f08fb3c8640 1 -- 172.21.15.43:0/3741142651 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f08f4074f60 con 0x7f08f40b5aa0 2024-04-25T09:03:58.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.102+0000 7f08f1ffb640 1 -- 172.21.15.43:0/3741142651 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v129) v1 ==== 93+0+10 (secure 0 0 0) 0x7f08e4059240 con 0x7f08f40b5aa0 2024-04-25T09:03:58.090 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T09:03:58.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.104+0000 7f08fb3c8640 1 -- 172.21.15.43:0/3741142651 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f08e006b4d0 msgr2=0x7f08e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:58.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.104+0000 7f08fb3c8640 1 --2- 172.21.15.43:0/3741142651 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f08e006b4d0 0x7f08e006d990 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f08e8034000 tx=0x7f08e802e000 comp rx=0 tx=0).stop 2024-04-25T09:03:58.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.104+0000 7f08fb3c8640 1 -- 172.21.15.43:0/3741142651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08f40b5aa0 msgr2=0x7f08f414bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:58.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.104+0000 7f08fb3c8640 1 --2- 172.21.15.43:0/3741142651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08f40b5aa0 0x7f08f414bf70 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f08e400cf60 tx=0x7f08e400cf90 comp rx=0 tx=0).stop 2024-04-25T09:03:58.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.104+0000 7f08fb3c8640 1 -- 172.21.15.43:0/3741142651 shutdown_connections 2024-04-25T09:03:58.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.104+0000 7f08fb3c8640 1 --2- 172.21.15.43:0/3741142651 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f08e006b4d0 0x7f08e006d990 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.104+0000 7f08fb3c8640 1 --2- 172.21.15.43:0/3741142651 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f08f4161fe0 0x7f08f40d2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.104+0000 7f08fb3c8640 1 --2- 172.21.15.43:0/3741142651 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f08f415e090 0x7f08f41e8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.104+0000 7f08fb3c8640 1 --2- 172.21.15.43:0/3741142651 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f08f40b5aa0 0x7f08f414bf70 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.104+0000 7f08fb3c8640 1 -- 172.21.15.43:0/3741142651 >> 172.21.15.43:0/3741142651 conn(0x7f08f40d0560 msgr2=0x7f08f4160d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:58.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.104+0000 7f08fb3c8640 1 -- 172.21.15.43:0/3741142651 shutdown_connections 2024-04-25T09:03:58.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.104+0000 7f08fb3c8640 1 -- 172.21.15.43:0/3741142651 wait complete. 2024-04-25T09:03:58.103 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:03:58.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.219+0000 7faf2a17e640 1 Processor -- start 2024-04-25T09:03:58.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.219+0000 7faf2a17e640 1 -- start start 2024-04-25T09:03:58.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.220+0000 7faf2a17e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7faf2415dec0 0x7faf2415ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:58.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.220+0000 7faf2a17e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faf240ce8c0 0x7faf240d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:58.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.220+0000 7faf2a17e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf240cf110 con 0x7faf240d04a0 2024-04-25T09:03:58.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.220+0000 7faf2a17e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf240cf3f0 con 0x7faf2415dec0 2024-04-25T09:03:58.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.220+0000 7faf2a17e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf240cf6d0 con 0x7faf240ce8c0 2024-04-25T09:03:58.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.220+0000 7faf22ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7faf2415dec0 0x7faf2415ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:58.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.220+0000 7faf237fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7faf240d04a0 0x7faf240d0880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48026/0 (socket says 172.21.15.43:48026) 2024-04-25T09:03:58.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.220+0000 7faf237fe640 1 -- 172.21.15.43:0/802783443 learned_addr learned my addr 172.21.15.43:0/802783443 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:03:58.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.220+0000 7faf22ffd640 1 -- 172.21.15.43:0/802783443 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faf240ce8c0 msgr2=0x7faf240d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:58.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.220+0000 7faf22ffd640 1 --2- 172.21.15.43:0/802783443 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faf240ce8c0 0x7faf240d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.220+0000 7faf22ffd640 1 -- 172.21.15.43:0/802783443 >> v1:172.21.15.43:6789/0 conn(0x7faf240d04a0 legacy=0x7faf240d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:58.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.220+0000 7faf22ffd640 1 -- 172.21.15.43:0/802783443 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf240cf9b0 con 0x7faf2415dec0 2024-04-25T09:03:58.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.220+0000 7faf227fc640 1 -- 172.21.15.43:0/802783443 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1534240872 0 0) 0x7faf240cf110 con 0x7faf240d04a0 2024-04-25T09:03:58.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.221+0000 7faf22ffd640 1 --2- 172.21.15.43:0/802783443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf2415dec0 0x7faf2415ed90 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7faf180097a0 tx=0x7faf1802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37a83b112cdbc3f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:58.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.221+0000 7faf227fc640 1 -- 172.21.15.43:0/802783443 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf18037070 con 0x7faf2415dec0 2024-04-25T09:03:58.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.221+0000 7faf227fc640 1 -- 172.21.15.43:0/802783443 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf18032440 con 0x7faf2415dec0 2024-04-25T09:03:58.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.221+0000 7faf227fc640 1 -- 172.21.15.43:0/802783443 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf18032810 con 0x7faf2415dec0 2024-04-25T09:03:58.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.222+0000 7faf2a17e640 1 -- 172.21.15.43:0/802783443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf2415dec0 msgr2=0x7faf2415ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:58.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.222+0000 7faf2a17e640 1 --2- 172.21.15.43:0/802783443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf2415dec0 0x7faf2415ed90 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7faf180097a0 tx=0x7faf1802a180 comp rx=0 tx=0).stop 2024-04-25T09:03:58.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.222+0000 7faf2a17e640 1 -- 172.21.15.43:0/802783443 shutdown_connections 2024-04-25T09:03:58.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.222+0000 7faf2a17e640 1 --2- 172.21.15.43:0/802783443 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faf240ce8c0 0x7faf240d17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.222+0000 7faf2a17e640 1 --2- 172.21.15.43:0/802783443 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf2415dec0 0x7faf2415ed90 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.222+0000 7faf2a17e640 1 -- 172.21.15.43:0/802783443 >> 172.21.15.43:0/802783443 conn(0x7faf2409cd40 msgr2=0x7faf240d3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:58.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.222+0000 7faf2a17e640 1 -- 172.21.15.43:0/802783443 shutdown_connections 2024-04-25T09:03:58.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.222+0000 7faf2a17e640 1 -- 172.21.15.43:0/802783443 wait complete. 2024-04-25T09:03:58.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.223+0000 7faf2a17e640 1 Processor -- start 2024-04-25T09:03:58.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.223+0000 7faf2a17e640 1 -- start start 2024-04-25T09:03:58.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.224+0000 7faf2a17e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faf240ce8c0 0x7faf24151140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:58.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.224+0000 7faf2a17e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf240d04a0 0x7faf240b66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:58.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.224+0000 7faf2a17e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf2415dec0 0x7faf240ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:58.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.224+0000 7faf2a17e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf240cee70 con 0x7faf240d04a0 2024-04-25T09:03:58.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.224+0000 7faf2a17e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf240cecf0 con 0x7faf2415dec0 2024-04-25T09:03:58.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.224+0000 7faf2a17e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf240d1ce0 con 0x7faf240ce8c0 2024-04-25T09:03:58.214 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.224+0000 7faf22ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf240d04a0 0x7faf240b66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:58.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.224+0000 7faf22ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf240d04a0 0x7faf240b66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55038/0 (socket says 172.21.15.43:55038) 2024-04-25T09:03:58.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.224+0000 7faf22ffd640 1 -- 172.21.15.43:0/3317432473 learned_addr learned my addr 172.21.15.43:0/3317432473 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:58.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.224+0000 7faf23fff640 1 --2- 172.21.15.43:0/3317432473 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf2415dec0 0x7faf240ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:58.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.224+0000 7faf237fe640 1 --2- 172.21.15.43:0/3317432473 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faf240ce8c0 0x7faf24151140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:58.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.224+0000 7faf22ffd640 1 -- 172.21.15.43:0/3317432473 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faf240ce8c0 msgr2=0x7faf24151140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:58.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.225+0000 7faf22ffd640 1 --2- 172.21.15.43:0/3317432473 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faf240ce8c0 0x7faf24151140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.225+0000 7faf22ffd640 1 -- 172.21.15.43:0/3317432473 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf2415dec0 msgr2=0x7faf240ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:58.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.225+0000 7faf22ffd640 1 --2- 172.21.15.43:0/3317432473 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf2415dec0 0x7faf240ccd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.225+0000 7faf22ffd640 1 -- 172.21.15.43:0/3317432473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf240b7140 con 0x7faf240d04a0 2024-04-25T09:03:58.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.225+0000 7faf23fff640 1 --2- 172.21.15.43:0/3317432473 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf2415dec0 0x7faf240ccd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:58.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.225+0000 7faf22ffd640 1 --2- 172.21.15.43:0/3317432473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf240d04a0 0x7faf240b66c0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7faf180097a0 tx=0x7faf180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:58.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.225+0000 7faf0bfff640 1 -- 172.21.15.43:0/3317432473 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf18041070 con 0x7faf240d04a0 2024-04-25T09:03:58.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.225+0000 7faf0bfff640 1 -- 172.21.15.43:0/3317432473 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf1803c870 con 0x7faf240d04a0 2024-04-25T09:03:58.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.225+0000 7faf2a17e640 1 -- 172.21.15.43:0/3317432473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf240b7420 con 0x7faf240d04a0 2024-04-25T09:03:58.215 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.225+0000 7faf0bfff640 1 -- 172.21.15.43:0/3317432473 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf18037040 con 0x7faf240d04a0 2024-04-25T09:03:58.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.225+0000 7faf2a17e640 1 -- 172.21.15.43:0/3317432473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf240b7cc0 con 0x7faf240d04a0 2024-04-25T09:03:58.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.226+0000 7faf2a17e640 1 -- 172.21.15.43:0/3317432473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf240b7810 con 0x7faf240d04a0 2024-04-25T09:03:58.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.227+0000 7faf0bfff640 1 -- 172.21.15.43:0/3317432473 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faf18044070 con 0x7faf240d04a0 2024-04-25T09:03:58.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.227+0000 7faf0bfff640 1 --2- 172.21.15.43:0/3317432473 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7faf1006b4d0 0x7faf1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:58.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.227+0000 7faf0bfff640 1 -- 172.21.15.43:0/3317432473 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5853+0+0 (secure 0 0 0) 0x7faf180ab140 con 0x7faf240d04a0 2024-04-25T09:03:58.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.227+0000 7faf237fe640 1 --2- 172.21.15.43:0/3317432473 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7faf1006b4d0 0x7faf1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:58.217 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.228+0000 7faf237fe640 1 --2- 172.21.15.43:0/3317432473 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7faf1006b4d0 0x7faf1006d990 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7faf14000f60 tx=0x7faf14009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:58.219 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.231+0000 7faf0bfff640 1 -- 172.21.15.43:0/3317432473 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faf1807a050 con 0x7faf240d04a0 2024-04-25T09:03:58.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.409+0000 7faf2a17e640 1 -- 172.21.15.43:0/3317432473 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faf24074f60 con 0x7faf240d04a0 2024-04-25T09:03:58.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.409+0000 7faf0bfff640 1 -- 172.21.15.43:0/3317432473 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v129) v1 ==== 104+0+11 (secure 0 0 0) 0x7faf18079950 con 0x7faf240d04a0 2024-04-25T09:03:58.398 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T09:03:58.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.411+0000 7faf2a17e640 1 -- 172.21.15.43:0/3317432473 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7faf1006b4d0 msgr2=0x7faf1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:58.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.411+0000 7faf2a17e640 1 --2- 172.21.15.43:0/3317432473 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7faf1006b4d0 0x7faf1006d990 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7faf14000f60 tx=0x7faf14009040 comp rx=0 tx=0).stop 2024-04-25T09:03:58.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.411+0000 7faf2a17e640 1 -- 172.21.15.43:0/3317432473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf240d04a0 msgr2=0x7faf240b66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:58.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.411+0000 7faf2a17e640 1 --2- 172.21.15.43:0/3317432473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf240d04a0 0x7faf240b66c0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7faf180097a0 tx=0x7faf180046c0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.411+0000 7faf2a17e640 1 -- 172.21.15.43:0/3317432473 shutdown_connections 2024-04-25T09:03:58.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.411+0000 7faf2a17e640 1 --2- 172.21.15.43:0/3317432473 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7faf1006b4d0 0x7faf1006d990 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.411+0000 7faf2a17e640 1 --2- 172.21.15.43:0/3317432473 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faf2415dec0 0x7faf240ccd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.411+0000 7faf2a17e640 1 --2- 172.21.15.43:0/3317432473 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7faf240d04a0 0x7faf240b66c0 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.411+0000 7faf2a17e640 1 --2- 172.21.15.43:0/3317432473 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faf240ce8c0 0x7faf24151140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.411+0000 7faf2a17e640 1 -- 172.21.15.43:0/3317432473 >> 172.21.15.43:0/3317432473 conn(0x7faf2409cd40 msgr2=0x7faf24160890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:58.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.411+0000 7faf2a17e640 1 -- 172.21.15.43:0/3317432473 shutdown_connections 2024-04-25T09:03:58.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.411+0000 7faf2a17e640 1 -- 172.21.15.43:0/3317432473 wait complete. 2024-04-25T09:03:58.411 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:03:58.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.526+0000 7f1f05c9e640 1 Processor -- start 2024-04-25T09:03:58.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.526+0000 7f1f05c9e640 1 -- start start 2024-04-25T09:03:58.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.527+0000 7f1f05c9e640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f1f000d1000 0x7f1f000d0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:58.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.527+0000 7f1f05c9e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f000d0d00 con 0x7f1f000d08d0 2024-04-25T09:03:58.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.527+0000 7f1f05c9e640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f0015e140 con 0x7f1f0015dd60 2024-04-25T09:03:58.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.527+0000 7f1f05c9e640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f000d27d0 con 0x7f1f000d1000 2024-04-25T09:03:58.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.527+0000 7f1efffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f1f000d08d0 0x7f1f0006bcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48028/0 (socket says 172.21.15.43:48028) 2024-04-25T09:03:58.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.527+0000 7f1efffff640 1 -- 172.21.15.43:0/1025706364 learned_addr learned my addr 172.21.15.43:0/1025706364 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:03:58.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.528+0000 7f1efeffd640 1 --2- 172.21.15.43:0/1025706364 >> v2:172.21.15.193:3300/0 conn(0x7f1f000d1000 0x7f1f000d0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:58.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.528+0000 7f1efe7fc640 1 -- 172.21.15.43:0/1025706364 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2400866427 0 0) 0x7f1ef8001490 con 0x7f1f000d08d0 2024-04-25T09:03:58.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.528+0000 7f1efe7fc640 1 -- 172.21.15.43:0/1025706364 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ee8003520 con 0x7f1f000d08d0 2024-04-25T09:03:58.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.528+0000 7f1efe7fc640 1 -- 172.21.15.43:0/1025706364 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1337451098 0 0) 0x7f1ee8003520 con 0x7f1f000d08d0 2024-04-25T09:03:58.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.528+0000 7f1efe7fc640 1 -- 172.21.15.43:0/1025706364 >> v2:172.21.15.193:3300/0 conn(0x7f1f000d1000 msgr2=0x7f1f000d0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:58.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.528+0000 7f1efe7fc640 1 --2- 172.21.15.43:0/1025706364 >> v2:172.21.15.193:3300/0 conn(0x7f1f000d1000 0x7f1f000d0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.528+0000 7f1efe7fc640 1 -- 172.21.15.43:0/1025706364 >> v1:172.21.15.193:6789/0 conn(0x7f1f0015dd60 legacy=0x7f1f000d1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:58.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.528+0000 7f1efe7fc640 1 -- 172.21.15.43:0/1025706364 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0006b1e0 con 0x7f1f000d08d0 2024-04-25T09:03:58.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.528+0000 7f1efe7fc640 1 -- 172.21.15.43:0/1025706364 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f1ef8001bc0 con 0x7f1f000d08d0 2024-04-25T09:03:58.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.528+0000 7f1efe7fc640 1 -- 172.21.15.43:0/1025706364 >> v1:172.21.15.43:6789/0 conn(0x7f1f000d08d0 legacy=0x7f1f0006bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:58.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.528+0000 7f1efe7fc640 1 --2- 172.21.15.43:0/1025706364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ee80039b0 0x7f1ee8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:58.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.528+0000 7f1efe7fc640 1 --2- 172.21.15.43:0/1025706364 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1ee8004800 0x7f1ee8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:58.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.528+0000 7f1efe7fc640 1 --2- 172.21.15.43:0/1025706364 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ee80051c0 0x7f1ee800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:58.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.528+0000 7f1efe7fc640 1 -- 172.21.15.43:0/1025706364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f000d27d0 con 0x7f1ee80039b0 2024-04-25T09:03:58.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.528+0000 7f1efe7fc640 1 -- 172.21.15.43:0/1025706364 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee800c060 con 0x7f1ee80051c0 2024-04-25T09:03:58.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.528+0000 7f1efe7fc640 1 -- 172.21.15.43:0/1025706364 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee800c260 con 0x7f1ee8004800 2024-04-25T09:03:58.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.528+0000 7f1eff7fe640 1 --2- 172.21.15.43:0/1025706364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ee80039b0 0x7f1ee8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:58.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.528+0000 7f1efeffd640 1 --2- 172.21.15.43:0/1025706364 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1ee8004800 0x7f1ee8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:58.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.528+0000 7f1efffff640 1 --2- 172.21.15.43:0/1025706364 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ee80051c0 0x7f1ee800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:58.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.528+0000 7f1eff7fe640 1 -- 172.21.15.43:0/1025706364 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1ee8004800 msgr2=0x7f1ee8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:58.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.528+0000 7f1eff7fe640 1 --2- 172.21.15.43:0/1025706364 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1ee8004800 0x7f1ee8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.528+0000 7f1eff7fe640 1 -- 172.21.15.43:0/1025706364 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ee80051c0 msgr2=0x7f1ee800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:58.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.528+0000 7f1eff7fe640 1 --2- 172.21.15.43:0/1025706364 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ee80051c0 0x7f1ee800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.528+0000 7f1eff7fe640 1 -- 172.21.15.43:0/1025706364 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee8003520 con 0x7f1ee80039b0 2024-04-25T09:03:58.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.528+0000 7f1efffff640 1 -- 172.21.15.43:0/1025706364 reap_dead start 2024-04-25T09:03:58.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.529+0000 7f1eff7fe640 1 --2- 172.21.15.43:0/1025706364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ee80039b0 0x7f1ee8003dd0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0001750 tx=0x7f1ee8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3de29178d3efbac1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:58.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.529+0000 7f1efe7fc640 1 -- 172.21.15.43:0/1025706364 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef0001370 con 0x7f1ee80039b0 2024-04-25T09:03:58.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.529+0000 7f1efe7fc640 1 -- 172.21.15.43:0/1025706364 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef00052d0 con 0x7f1ee80039b0 2024-04-25T09:03:58.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.529+0000 7f1efe7fc640 1 -- 172.21.15.43:0/1025706364 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef002abc0 con 0x7f1ee80039b0 2024-04-25T09:03:58.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.530+0000 7f1f05c9e640 1 -- 172.21.15.43:0/1025706364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ee80039b0 msgr2=0x7f1ee8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:58.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.530+0000 7f1f05c9e640 1 --2- 172.21.15.43:0/1025706364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ee80039b0 0x7f1ee8003dd0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0001750 tx=0x7f1ee8005830 comp rx=0 tx=0).stop 2024-04-25T09:03:58.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.530+0000 7f1f05c9e640 1 -- 172.21.15.43:0/1025706364 shutdown_connections 2024-04-25T09:03:58.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.530+0000 7f1f05c9e640 1 --2- 172.21.15.43:0/1025706364 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ee80039b0 0x7f1ee8003dd0 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.530+0000 7f1f05c9e640 1 -- 172.21.15.43:0/1025706364 >> 172.21.15.43:0/1025706364 conn(0x7f1f000d39c0 msgr2=0x7f1f000d3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:58.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.530+0000 7f1f05c9e640 1 -- 172.21.15.43:0/1025706364 shutdown_connections 2024-04-25T09:03:58.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.530+0000 7f1f05c9e640 1 -- 172.21.15.43:0/1025706364 wait complete. 2024-04-25T09:03:58.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.531+0000 7f1f05c9e640 1 Processor -- start 2024-04-25T09:03:58.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.531+0000 7f1f05c9e640 1 -- start start 2024-04-25T09:03:58.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.532+0000 7f1f05c9e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1ee80039b0 0x7f1f001f3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:58.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.532+0000 7f1f05c9e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f000cc6c0 0x7f1f000b8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:58.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.532+0000 7f1f05c9e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f000b7c10 0x7f1f000b4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:58.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.532+0000 7f1f05c9e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f000d0390 con 0x7f1f000cc6c0 2024-04-25T09:03:58.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.532+0000 7f1f05c9e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0006b7a0 con 0x7f1f000b7c10 2024-04-25T09:03:58.524 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.532+0000 7f1f05c9e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f000d0510 con 0x7f1ee80039b0 2024-04-25T09:03:58.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.532+0000 7f1efeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f000cc6c0 0x7f1f000b8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:58.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.532+0000 7f1efeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f000cc6c0 0x7f1f000b8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55048/0 (socket says 172.21.15.43:55048) 2024-04-25T09:03:58.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.532+0000 7f1efeffd640 1 -- 172.21.15.43:0/293143090 learned_addr learned my addr 172.21.15.43:0/293143090 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:58.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.532+0000 7f1eff7fe640 1 --2- 172.21.15.43:0/293143090 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1ee80039b0 0x7f1f001f3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:58.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.532+0000 7f1efffff640 1 --2- 172.21.15.43:0/293143090 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f000b7c10 0x7f1f000b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:58.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.532+0000 7f1efeffd640 1 -- 172.21.15.43:0/293143090 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1ee80039b0 msgr2=0x7f1f001f3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:58.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.532+0000 7f1efeffd640 1 --2- 172.21.15.43:0/293143090 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1ee80039b0 0x7f1f001f3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.532+0000 7f1efeffd640 1 -- 172.21.15.43:0/293143090 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f000b7c10 msgr2=0x7f1f000b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:58.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.532+0000 7f1efeffd640 1 --2- 172.21.15.43:0/293143090 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f000b7c10 0x7f1f000b4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.532+0000 7f1efeffd640 1 -- 172.21.15.43:0/293143090 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0015f380 con 0x7f1f000cc6c0 2024-04-25T09:03:58.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.532+0000 7f1eff7fe640 1 --2- 172.21.15.43:0/293143090 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1ee80039b0 0x7f1f001f3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:58.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.532+0000 7f1efffff640 1 --2- 172.21.15.43:0/293143090 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f000b7c10 0x7f1f000b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:58.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.533+0000 7f1efeffd640 1 --2- 172.21.15.43:0/293143090 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f000cc6c0 0x7f1f000b8940 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4011960 tx=0x7f1ef4011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:58.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.533+0000 7f1ee7fff640 1 -- 172.21.15.43:0/293143090 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef400faf0 con 0x7f1f000cc6c0 2024-04-25T09:03:58.525 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.533+0000 7f1ee7fff640 1 -- 172.21.15.43:0/293143090 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef400fc90 con 0x7f1f000cc6c0 2024-04-25T09:03:58.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.533+0000 7f1f05c9e640 1 -- 172.21.15.43:0/293143090 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f001f3eb0 con 0x7f1f000cc6c0 2024-04-25T09:03:58.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.533+0000 7f1ee7fff640 1 -- 172.21.15.43:0/293143090 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4014440 con 0x7f1f000cc6c0 2024-04-25T09:03:58.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.533+0000 7f1f05c9e640 1 -- 172.21.15.43:0/293143090 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f001f4470 con 0x7f1f000cc6c0 2024-04-25T09:03:58.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.534+0000 7f1f05c9e640 1 -- 172.21.15.43:0/293143090 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f00074f60 con 0x7f1f000cc6c0 2024-04-25T09:03:58.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.534+0000 7f1ee7fff640 1 -- 172.21.15.43:0/293143090 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1ef4012040 con 0x7f1f000cc6c0 2024-04-25T09:03:58.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.535+0000 7f1ee7fff640 1 --2- 172.21.15.43:0/293143090 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1eec06b4d0 0x7f1eec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:58.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.535+0000 7f1ee7fff640 1 -- 172.21.15.43:0/293143090 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f1ef408a8a0 con 0x7f1f000cc6c0 2024-04-25T09:03:58.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.535+0000 7f1eff7fe640 1 --2- 172.21.15.43:0/293143090 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1eec06b4d0 0x7f1eec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:58.526 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.536+0000 7f1eff7fe640 1 --2- 172.21.15.43:0/293143090 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1eec06b4d0 0x7f1eec06d990 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0032000 tx=0x7f1ef002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:58.527 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.539+0000 7f1ee7fff640 1 -- 172.21.15.43:0/293143090 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1ef4019030 con 0x7f1f000cc6c0 2024-04-25T09:03:58.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.720+0000 7f1f05c9e640 1 -- 172.21.15.43:0/293143090 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1f000cfd20 con 0x7f1f000cc6c0 2024-04-25T09:03:58.708 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.720+0000 7f1ee7fff640 1 -- 172.21.15.43:0/293143090 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v129) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1ef4059040 con 0x7f1f000cc6c0 2024-04-25T09:03:58.709 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T09:03:58.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.722+0000 7f1f05c9e640 1 -- 172.21.15.43:0/293143090 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1eec06b4d0 msgr2=0x7f1eec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:58.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.722+0000 7f1f05c9e640 1 --2- 172.21.15.43:0/293143090 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1eec06b4d0 0x7f1eec06d990 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f1ef0032000 tx=0x7f1ef002e000 comp rx=0 tx=0).stop 2024-04-25T09:03:58.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.722+0000 7f1f05c9e640 1 -- 172.21.15.43:0/293143090 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f000cc6c0 msgr2=0x7f1f000b8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:58.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.722+0000 7f1f05c9e640 1 --2- 172.21.15.43:0/293143090 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f000cc6c0 0x7f1f000b8940 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4011960 tx=0x7f1ef4011e20 comp rx=0 tx=0).stop 2024-04-25T09:03:58.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.722+0000 7f1f05c9e640 1 -- 172.21.15.43:0/293143090 shutdown_connections 2024-04-25T09:03:58.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.722+0000 7f1f05c9e640 1 --2- 172.21.15.43:0/293143090 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1eec06b4d0 0x7f1eec06d990 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.722+0000 7f1f05c9e640 1 --2- 172.21.15.43:0/293143090 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1f000b7c10 0x7f1f000b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.722+0000 7f1f05c9e640 1 --2- 172.21.15.43:0/293143090 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1f000cc6c0 0x7f1f000b8940 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.722+0000 7f1f05c9e640 1 --2- 172.21.15.43:0/293143090 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1ee80039b0 0x7f1f001f3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.711 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.722+0000 7f1f05c9e640 1 -- 172.21.15.43:0/293143090 >> 172.21.15.43:0/293143090 conn(0x7f1f000d39c0 msgr2=0x7f1f000ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:58.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.722+0000 7f1f05c9e640 1 -- 172.21.15.43:0/293143090 shutdown_connections 2024-04-25T09:03:58.712 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.722+0000 7f1f05c9e640 1 -- 172.21.15.43:0/293143090 wait complete. 2024-04-25T09:03:58.722 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:03:58.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.834+0000 7f4a29038640 1 Processor -- start 2024-04-25T09:03:58.823 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.834+0000 7f4a29038640 1 -- start start 2024-04-25T09:03:58.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.835+0000 7f4a29038640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4a2415ef50 0x7f4a240d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:58.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.835+0000 7f4a29038640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a240d2920 con 0x7f4a2415dce0 2024-04-25T09:03:58.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.835+0000 7f4a29038640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a240cee80 con 0x7f4a240d0920 2024-04-25T09:03:58.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.835+0000 7f4a29038640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a240cec60 con 0x7f4a2415ef50 2024-04-25T09:03:58.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.835+0000 7f4a22d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4a2415dce0 0x7f4a240ce550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:48030/0 (socket says 172.21.15.43:48030) 2024-04-25T09:03:58.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.835+0000 7f4a22d9d640 1 -- 172.21.15.43:0/1288574603 learned_addr learned my addr 172.21.15.43:0/1288574603 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:03:58.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.835+0000 7f4a21d9b640 1 -- 172.21.15.43:0/1288574603 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 266266228 0 0) 0x7f4a240d2920 con 0x7f4a2415dce0 2024-04-25T09:03:58.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.835+0000 7f4a21d9b640 1 -- 172.21.15.43:0/1288574603 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a0c003520 con 0x7f4a2415dce0 2024-04-25T09:03:58.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.835+0000 7f4a2259c640 1 --2- 172.21.15.43:0/1288574603 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4a2415ef50 0x7f4a240d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:58.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.835+0000 7f4a21d9b640 1 -- 172.21.15.43:0/1288574603 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1306605151 0 0) 0x7f4a0c003520 con 0x7f4a2415dce0 2024-04-25T09:03:58.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.835+0000 7f4a21d9b640 1 -- 172.21.15.43:0/1288574603 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4a2415ef50 msgr2=0x7f4a240d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:58.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.835+0000 7f4a21d9b640 1 --2- 172.21.15.43:0/1288574603 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4a2415ef50 0x7f4a240d03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.835+0000 7f4a21d9b640 1 -- 172.21.15.43:0/1288574603 >> v1:172.21.15.193:6789/0 conn(0x7f4a240d0920 legacy=0x7f4a24160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:58.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.835+0000 7f4a21d9b640 1 -- 172.21.15.43:0/1288574603 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a240cf160 con 0x7f4a2415dce0 2024-04-25T09:03:58.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.835+0000 7f4a21d9b640 1 -- 172.21.15.43:0/1288574603 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f4a1c001670 con 0x7f4a2415dce0 2024-04-25T09:03:58.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.836+0000 7f4a21d9b640 1 -- 172.21.15.43:0/1288574603 >> v1:172.21.15.43:6789/0 conn(0x7f4a2415dce0 legacy=0x7f4a240ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:58.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.836+0000 7f4a21d9b640 1 --2- 172.21.15.43:0/1288574603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a0c0039e0 0x7f4a0c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:58.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.836+0000 7f4a21d9b640 1 --2- 172.21.15.43:0/1288574603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a0c004800 0x7f4a0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:58.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.836+0000 7f4a21d9b640 1 --2- 172.21.15.43:0/1288574603 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4a2415ef50 0x7f4a0c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:58.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.836+0000 7f4a21d9b640 1 -- 172.21.15.43:0/1288574603 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a240cec60 con 0x7f4a0c0039e0 2024-04-25T09:03:58.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.836+0000 7f4a21d9b640 1 -- 172.21.15.43:0/1288574603 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0c005520 con 0x7f4a0c004800 2024-04-25T09:03:58.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.836+0000 7f4a21d9b640 1 -- 172.21.15.43:0/1288574603 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0c00bf00 con 0x7f4a2415ef50 2024-04-25T09:03:58.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.836+0000 7f4a22d9d640 1 --2- 172.21.15.43:0/1288574603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a0c0039e0 0x7f4a0c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:58.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.836+0000 7f4a2259c640 1 --2- 172.21.15.43:0/1288574603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a0c004800 0x7f4a0c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:58.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.836+0000 7f4a2359e640 1 --2- 172.21.15.43:0/1288574603 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4a2415ef50 0x7f4a0c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:58.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.836+0000 7f4a22d9d640 1 -- 172.21.15.43:0/1288574603 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4a2415ef50 msgr2=0x7f4a0c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:58.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.836+0000 7f4a22d9d640 1 --2- 172.21.15.43:0/1288574603 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4a2415ef50 0x7f4a0c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.836+0000 7f4a22d9d640 1 -- 172.21.15.43:0/1288574603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a0c004800 msgr2=0x7f4a0c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:58.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.836+0000 7f4a22d9d640 1 --2- 172.21.15.43:0/1288574603 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a0c004800 0x7f4a0c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.836+0000 7f4a22d9d640 1 -- 172.21.15.43:0/1288574603 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a0c003520 con 0x7f4a0c0039e0 2024-04-25T09:03:58.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.837+0000 7f4a22d9d640 1 --2- 172.21.15.43:0/1288574603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a0c0039e0 0x7f4a0c003e00 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c00c240 tx=0x7f4a0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e18e2f97f57801ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:58.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.837+0000 7f4a21d9b640 1 -- 172.21.15.43:0/1288574603 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c007d10 con 0x7f4a0c0039e0 2024-04-25T09:03:58.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.837+0000 7f4a21d9b640 1 -- 172.21.15.43:0/1288574603 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1c005ad0 con 0x7f4a0c0039e0 2024-04-25T09:03:58.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.837+0000 7f4a21d9b640 1 -- 172.21.15.43:0/1288574603 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c007070 con 0x7f4a0c0039e0 2024-04-25T09:03:58.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.837+0000 7f4a29038640 1 -- 172.21.15.43:0/1288574603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a0c0039e0 msgr2=0x7f4a0c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:58.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.837+0000 7f4a29038640 1 --2- 172.21.15.43:0/1288574603 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a0c0039e0 0x7f4a0c003e00 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c00c240 tx=0x7f4a0c005830 comp rx=0 tx=0).stop 2024-04-25T09:03:58.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.837+0000 7f4a2359e640 1 -- 172.21.15.43:0/1288574603 reap_dead start 2024-04-25T09:03:58.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.837+0000 7f4a29038640 1 -- 172.21.15.43:0/1288574603 shutdown_connections 2024-04-25T09:03:58.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.837+0000 7f4a29038640 1 -- 172.21.15.43:0/1288574603 >> 172.21.15.43:0/1288574603 conn(0x7f4a2409cd00 msgr2=0x7f4a240d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:58.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.837+0000 7f4a29038640 1 -- 172.21.15.43:0/1288574603 shutdown_connections 2024-04-25T09:03:58.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.838+0000 7f4a29038640 1 -- 172.21.15.43:0/1288574603 wait complete. 2024-04-25T09:03:58.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.839+0000 7f4a29038640 1 Processor -- start 2024-04-25T09:03:58.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.839+0000 7f4a29038640 1 -- start start 2024-04-25T09:03:58.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.840+0000 7f4a29038640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a0c0039e0 0x7f4a240b9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:58.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.840+0000 7f4a29038640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4a241e5030 0x7f4a241590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:58.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.840+0000 7f4a29038640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a241e6940 0x7f4a241e6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:58.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.840+0000 7f4a29038640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a240cf720 con 0x7f4a0c0039e0 2024-04-25T09:03:58.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.840+0000 7f4a29038640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a240d0d70 con 0x7f4a241e6940 2024-04-25T09:03:58.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.840+0000 7f4a29038640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a2415f6b0 con 0x7f4a241e5030 2024-04-25T09:03:58.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.840+0000 7f4a22d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a0c0039e0 0x7f4a240b9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:58.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.840+0000 7f4a22d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a0c0039e0 0x7f4a240b9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55062/0 (socket says 172.21.15.43:55062) 2024-04-25T09:03:58.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.840+0000 7f4a22d9d640 1 -- 172.21.15.43:0/4254275452 learned_addr learned my addr 172.21.15.43:0/4254275452 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:58.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.840+0000 7f4a2359e640 1 --2- 172.21.15.43:0/4254275452 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a241e6940 0x7f4a241e6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:58.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.840+0000 7f4a2259c640 1 --2- 172.21.15.43:0/4254275452 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4a241e5030 0x7f4a241590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:58.833 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.840+0000 7f4a22d9d640 1 -- 172.21.15.43:0/4254275452 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4a241e5030 msgr2=0x7f4a241590e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:58.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.840+0000 7f4a22d9d640 1 --2- 172.21.15.43:0/4254275452 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4a241e5030 0x7f4a241590e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.840+0000 7f4a22d9d640 1 -- 172.21.15.43:0/4254275452 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a241e6940 msgr2=0x7f4a241e6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:58.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.840+0000 7f4a22d9d640 1 --2- 172.21.15.43:0/4254275452 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a241e6940 0x7f4a241e6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:58.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.840+0000 7f4a22d9d640 1 -- 172.21.15.43:0/4254275452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a241e57a0 con 0x7f4a0c0039e0 2024-04-25T09:03:58.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.840+0000 7f4a22d9d640 1 --2- 172.21.15.43:0/4254275452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a0c0039e0 0x7f4a240b9000 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c034040 tx=0x7f4a240cfa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:58.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.841+0000 7f4a0b7fe640 1 -- 172.21.15.43:0/4254275452 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c042020 con 0x7f4a0c0039e0 2024-04-25T09:03:58.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.841+0000 7f4a0b7fe640 1 -- 172.21.15.43:0/4254275452 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a1c004bf0 con 0x7f4a0c0039e0 2024-04-25T09:03:58.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.841+0000 7f4a29038640 1 -- 172.21.15.43:0/4254275452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a241e5a80 con 0x7f4a0c0039e0 2024-04-25T09:03:58.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.841+0000 7f4a0b7fe640 1 -- 172.21.15.43:0/4254275452 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a1c007070 con 0x7f4a0c0039e0 2024-04-25T09:03:58.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.841+0000 7f4a29038640 1 -- 172.21.15.43:0/4254275452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a241e54b0 con 0x7f4a0c0039e0 2024-04-25T09:03:58.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.842+0000 7f4a29038640 1 -- 172.21.15.43:0/4254275452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a240680f0 con 0x7f4a0c0039e0 2024-04-25T09:03:58.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.842+0000 7f4a0b7fe640 1 -- 172.21.15.43:0/4254275452 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4a1c045020 con 0x7f4a0c0039e0 2024-04-25T09:03:58.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.843+0000 7f4a0b7fe640 1 --2- 172.21.15.43:0/4254275452 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4a1006b500 0x7f4a1006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:58.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.843+0000 7f4a0b7fe640 1 -- 172.21.15.43:0/4254275452 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f4a1c0aa590 con 0x7f4a0c0039e0 2024-04-25T09:03:58.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.843+0000 7f4a2259c640 1 --2- 172.21.15.43:0/4254275452 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4a1006b500 0x7f4a1006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:58.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.843+0000 7f4a2259c640 1 --2- 172.21.15.43:0/4254275452 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4a1006b500 0x7f4a1006d9c0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f4a241e6160 tx=0x7f4a18008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:58.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:58.847+0000 7f4a0b7fe640 1 -- 172.21.15.43:0/4254275452 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4a1c079490 con 0x7f4a0c0039e0 2024-04-25T09:03:59.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.042+0000 7f4a29038640 1 -- 172.21.15.43:0/4254275452 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4a24074ef0 con 0x7f4a0c0039e0 2024-04-25T09:03:59.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.043+0000 7f4a0b7fe640 1 -- 172.21.15.43:0/4254275452 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v129) v1 ==== 80+0+122 (secure 0 0 0) 0x7f4a1c078d90 con 0x7f4a0c0039e0 2024-04-25T09:03:59.031 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6834/2603892772 2024-04-26T09:03:52.443908+0000 2024-04-25T09:03:59.031 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.15.43:6832/2603892772 2024-04-26T09:03:52.443908+0000 2024-04-25T09:03:59.032 INFO:teuthology.orchestra.run.smithi043.stderr:listed 2 entries 2024-04-25T09:03:59.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.044+0000 7f4a29038640 1 -- 172.21.15.43:0/4254275452 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4a1006b500 msgr2=0x7f4a1006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:59.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.044+0000 7f4a29038640 1 --2- 172.21.15.43:0/4254275452 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4a1006b500 0x7f4a1006d9c0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f4a241e6160 tx=0x7f4a18008040 comp rx=0 tx=0).stop 2024-04-25T09:03:59.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.044+0000 7f4a29038640 1 -- 172.21.15.43:0/4254275452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a0c0039e0 msgr2=0x7f4a240b9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:59.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.044+0000 7f4a29038640 1 --2- 172.21.15.43:0/4254275452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a0c0039e0 0x7f4a240b9000 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f4a1c034040 tx=0x7f4a240cfa00 comp rx=0 tx=0).stop 2024-04-25T09:03:59.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.044+0000 7f4a29038640 1 -- 172.21.15.43:0/4254275452 shutdown_connections 2024-04-25T09:03:59.033 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.044+0000 7f4a29038640 1 --2- 172.21.15.43:0/4254275452 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4a1006b500 0x7f4a1006d9c0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.044+0000 7f4a29038640 1 --2- 172.21.15.43:0/4254275452 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4a241e6940 0x7f4a241e6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.044+0000 7f4a29038640 1 --2- 172.21.15.43:0/4254275452 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4a241e5030 0x7f4a241590e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.044+0000 7f4a29038640 1 --2- 172.21.15.43:0/4254275452 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4a0c0039e0 0x7f4a240b9000 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.044+0000 7f4a29038640 1 -- 172.21.15.43:0/4254275452 >> 172.21.15.43:0/4254275452 conn(0x7f4a2409cd00 msgr2=0x7f4a2406c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:59.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.044+0000 7f4a29038640 1 -- 172.21.15.43:0/4254275452 shutdown_connections 2024-04-25T09:03:59.034 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.044+0000 7f4a29038640 1 -- 172.21.15.43:0/4254275452 wait complete. 2024-04-25T09:03:59.044 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi043.front.sepia.ceph.com... 2024-04-25T09:03:59.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:59.044 DEBUG:teuthology.orchestra.run.smithi043:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T09:03:59.067 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[91777]: fuse finished with error 0 and tester_r 0 2024-04-25T09:03:59.068 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-04-25T09:03:59.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:59.068 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:03:59.174 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T09:03:59.174 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:03:59.193 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19508 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:03:59.193 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi193.front.sepia.ceph.com 2024-04-25T09:03:59.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:59.193 DEBUG:teuthology.orchestra.run.smithi193:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T09:03:59.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:03:59.250 INFO:teuthology.orchestra.run.smithi193.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T09:03:59.251 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T09:03:59.251 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:03:59.266 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2374 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:03:59.266 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-04-25T09:03:59.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:59.267 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T09:03:59.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:03:59.322 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T09:03:59.322 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T09:03:59.322 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:03:59.340 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19508 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:03:59.340 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi193.front.sepia.ceph.com 2024-04-25T09:03:59.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:03:59.341 DEBUG:teuthology.orchestra.run.smithi193:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T09:03:59.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:03:59.398 INFO:teuthology.orchestra.run.smithi193.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T09:03:59.398 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:03:59.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.510+0000 7fd6413f4640 1 Processor -- start 2024-04-25T09:03:59.499 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.510+0000 7fd6413f4640 1 -- start start 2024-04-25T09:03:59.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.511+0000 7fd6413f4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd63c06d9a0 0x7fd63c071a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:59.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.511+0000 7fd6413f4640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd63c071f90 0x7fd63c0cd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:59.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.511+0000 7fd6413f4640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd63c072420 con 0x7fd63c15a9d0 2024-04-25T09:03:59.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.511+0000 7fd6413f4640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd63c0704f0 con 0x7fd63c06d9a0 2024-04-25T09:03:59.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.511+0000 7fd6413f4640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c0707d0 con 0x7fd63c071f90 2024-04-25T09:03:59.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.511+0000 7fd63a7fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd63c06d9a0 0x7fd63c071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:59.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.511+0000 7fd63a7fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fd63c06d9a0 0x7fd63c071a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55078/0 (socket says 172.21.15.43:55078) 2024-04-25T09:03:59.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.511+0000 7fd63a7fc640 1 -- 172.21.15.43:0/2794111228 learned_addr learned my addr 172.21.15.43:0/2794111228 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:59.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.512+0000 7fd63b7fe640 1 --2- 172.21.15.43:0/2794111228 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd63c071f90 0x7fd63c0cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:59.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.512+0000 7fd63a7fc640 1 -- 172.21.15.43:0/2794111228 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd63c071f90 msgr2=0x7fd63c0cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:59.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.512+0000 7fd63a7fc640 1 --2- 172.21.15.43:0/2794111228 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd63c071f90 0x7fd63c0cd940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.512+0000 7fd63a7fc640 1 -- 172.21.15.43:0/2794111228 >> v1:172.21.15.193:6789/0 conn(0x7fd63c15a9d0 legacy=0x7fd63c06d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:59.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.512+0000 7fd63a7fc640 1 -- 172.21.15.43:0/2794111228 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd63c070ab0 con 0x7fd63c06d9a0 2024-04-25T09:03:59.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.512+0000 7fd63a7fc640 1 --2- 172.21.15.43:0/2794111228 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd63c06d9a0 0x7fd63c071a50 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fd630009870 tx=0x7fd63002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=210a30ad43db3e32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:59.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.513+0000 7fd639ffb640 1 -- 172.21.15.43:0/2794111228 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd630037070 con 0x7fd63c06d9a0 2024-04-25T09:03:59.502 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.513+0000 7fd639ffb640 1 -- 172.21.15.43:0/2794111228 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd630032440 con 0x7fd63c06d9a0 2024-04-25T09:03:59.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.513+0000 7fd639ffb640 1 -- 172.21.15.43:0/2794111228 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd630032810 con 0x7fd63c06d9a0 2024-04-25T09:03:59.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.513+0000 7fd6413f4640 1 -- 172.21.15.43:0/2794111228 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd63c06d9a0 msgr2=0x7fd63c071a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:59.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.513+0000 7fd6413f4640 1 --2- 172.21.15.43:0/2794111228 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd63c06d9a0 0x7fd63c071a50 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fd630009870 tx=0x7fd63002a210 comp rx=0 tx=0).stop 2024-04-25T09:03:59.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.513+0000 7fd6413f4640 1 -- 172.21.15.43:0/2794111228 shutdown_connections 2024-04-25T09:03:59.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.513+0000 7fd6413f4640 1 --2- 172.21.15.43:0/2794111228 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd63c071f90 0x7fd63c0cd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.513+0000 7fd6413f4640 1 --2- 172.21.15.43:0/2794111228 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd63c06d9a0 0x7fd63c071a50 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.513+0000 7fd6413f4640 1 -- 172.21.15.43:0/2794111228 >> 172.21.15.43:0/2794111228 conn(0x7fd63c0b1480 msgr2=0x7fd63c0b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:59.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.513+0000 7fd6413f4640 1 -- 172.21.15.43:0/2794111228 shutdown_connections 2024-04-25T09:03:59.503 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.513+0000 7fd6413f4640 1 -- 172.21.15.43:0/2794111228 wait complete. 2024-04-25T09:03:59.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.514+0000 7fd6413f4640 1 Processor -- start 2024-04-25T09:03:59.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.515+0000 7fd6413f4640 1 -- start start 2024-04-25T09:03:59.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.515+0000 7fd6413f4640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd63c06d9a0 0x7fd63c1f0f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:59.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.515+0000 7fd6413f4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd63c071f90 0x7fd63c1e4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:59.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.515+0000 7fd6413f4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd63c1e5490 0x7fd63c0ca890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:59.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.515+0000 7fd6413f4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c0cde80 con 0x7fd63c1e5490 2024-04-25T09:03:59.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.515+0000 7fd6413f4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c0712b0 con 0x7fd63c071f90 2024-04-25T09:03:59.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.515+0000 7fd6413f4640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c0ce000 con 0x7fd63c06d9a0 2024-04-25T09:03:59.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.515+0000 7fd63b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd63c1e5490 0x7fd63c0ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:59.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.515+0000 7fd63b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd63c1e5490 0x7fd63c0ca890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55094/0 (socket says 172.21.15.43:55094) 2024-04-25T09:03:59.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.515+0000 7fd63b7fe640 1 -- 172.21.15.43:0/2260300044 learned_addr learned my addr 172.21.15.43:0/2260300044 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:59.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.516+0000 7fd63affd640 1 --2- 172.21.15.43:0/2260300044 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd63c06d9a0 0x7fd63c1f0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:59.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.516+0000 7fd63a7fc640 1 --2- 172.21.15.43:0/2260300044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd63c071f90 0x7fd63c1e4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:59.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.516+0000 7fd63b7fe640 1 -- 172.21.15.43:0/2260300044 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd63c06d9a0 msgr2=0x7fd63c1f0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:59.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.516+0000 7fd63b7fe640 1 --2- 172.21.15.43:0/2260300044 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd63c06d9a0 0x7fd63c1f0f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.516+0000 7fd63b7fe640 1 -- 172.21.15.43:0/2260300044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd63c071f90 msgr2=0x7fd63c1e4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:59.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.516+0000 7fd63b7fe640 1 --2- 172.21.15.43:0/2260300044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd63c071f90 0x7fd63c1e4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.516+0000 7fd63b7fe640 1 -- 172.21.15.43:0/2260300044 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd63c1e5940 con 0x7fd63c1e5490 2024-04-25T09:03:59.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.516+0000 7fd63a7fc640 1 --2- 172.21.15.43:0/2260300044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd63c071f90 0x7fd63c1e4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:59.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.516+0000 7fd63b7fe640 1 --2- 172.21.15.43:0/2260300044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd63c1e5490 0x7fd63c0ca890 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fd63400cd20 tx=0x7fd63400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:59.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.516+0000 7fd6237fe640 1 -- 172.21.15.43:0/2260300044 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd634003b90 con 0x7fd63c1e5490 2024-04-25T09:03:59.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.516+0000 7fd6237fe640 1 -- 172.21.15.43:0/2260300044 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd634003d30 con 0x7fd63c1e5490 2024-04-25T09:03:59.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.516+0000 7fd6413f4640 1 -- 172.21.15.43:0/2260300044 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd63c1e5c20 con 0x7fd63c1e5490 2024-04-25T09:03:59.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.516+0000 7fd6237fe640 1 -- 172.21.15.43:0/2260300044 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd634004240 con 0x7fd63c1e5490 2024-04-25T09:03:59.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.516+0000 7fd6413f4640 1 -- 172.21.15.43:0/2260300044 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd63c1431f0 con 0x7fd63c1e5490 2024-04-25T09:03:59.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.517+0000 7fd6413f4640 1 -- 172.21.15.43:0/2260300044 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd63c143010 con 0x7fd63c1e5490 2024-04-25T09:03:59.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.518+0000 7fd6237fe640 1 -- 172.21.15.43:0/2260300044 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd6340043e0 con 0x7fd63c1e5490 2024-04-25T09:03:59.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.518+0000 7fd6237fe640 1 --2- 172.21.15.43:0/2260300044 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd62806b4d0 0x7fd62806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:59.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.518+0000 7fd6237fe640 1 -- 172.21.15.43:0/2260300044 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fd63408ada0 con 0x7fd63c1e5490 2024-04-25T09:03:59.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.518+0000 7fd63affd640 1 --2- 172.21.15.43:0/2260300044 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd62806b4d0 0x7fd62806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:59.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.519+0000 7fd63affd640 1 --2- 172.21.15.43:0/2260300044 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd62806b4d0 0x7fd62806d990 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fd62c006fd0 tx=0x7fd62c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:59.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.522+0000 7fd6237fe640 1 -- 172.21.15.43:0/2260300044 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd63c143010 con 0x7fd63c1e5490 2024-04-25T09:03:59.525 INFO:tasks.ceph.mds.c.smithi043.stderr:2024-04-25T09:03:59.537+0000 7f60f3e03640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:03:59.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.742+0000 7fd6413f4640 1 -- 172.21.15.43:0/2260300044 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd63c074ef0 con 0x7fd63c1e5490 2024-04-25T09:03:59.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.742+0000 7fd6237fe640 1 -- 172.21.15.43:0/2260300044 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4155 (secure 0 0 0) 0x7fd634059b90 con 0x7fd63c1e5490 2024-04-25T09:03:59.731 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:03:59.731 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":80,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/891225068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":891225068},{"type":"v1","addr":"172.21.15.193:6836","nonce":891225068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":7175,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3679059390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":3679059390},{"type":"v1","addr":"172.21.15.193:6837","nonce":3679059390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:11.919337+0000","modified":"2024-04-25T09:03:54.462418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7948},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7948":{"gid":7948,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":15,"addr":"172.21.15.43:6835/2127363183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2127363183},{"type":"v1","addr":"172.21.15.43:6835","nonce":2127363183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:03:59.731 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 80 2024-04-25T09:03:59.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.744+0000 7fd6413f4640 1 -- 172.21.15.43:0/2260300044 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd62806b4d0 msgr2=0x7fd62806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:59.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.744+0000 7fd6413f4640 1 --2- 172.21.15.43:0/2260300044 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd62806b4d0 0x7fd62806d990 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fd62c006fd0 tx=0x7fd62c008040 comp rx=0 tx=0).stop 2024-04-25T09:03:59.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.744+0000 7fd6413f4640 1 -- 172.21.15.43:0/2260300044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd63c1e5490 msgr2=0x7fd63c0ca890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:59.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.744+0000 7fd6413f4640 1 --2- 172.21.15.43:0/2260300044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd63c1e5490 0x7fd63c0ca890 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fd63400cd20 tx=0x7fd63400a660 comp rx=0 tx=0).stop 2024-04-25T09:03:59.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.745+0000 7fd6413f4640 1 -- 172.21.15.43:0/2260300044 shutdown_connections 2024-04-25T09:03:59.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.745+0000 7fd6413f4640 1 --2- 172.21.15.43:0/2260300044 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd63c1e5490 0x7fd63c0ca890 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.745+0000 7fd6413f4640 1 --2- 172.21.15.43:0/2260300044 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd62806b4d0 0x7fd62806d990 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.745+0000 7fd6413f4640 1 --2- 172.21.15.43:0/2260300044 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd63c071f90 0x7fd63c1e4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.745+0000 7fd6413f4640 1 --2- 172.21.15.43:0/2260300044 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd63c06d9a0 0x7fd63c1f0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.745+0000 7fd6413f4640 1 -- 172.21.15.43:0/2260300044 >> 172.21.15.43:0/2260300044 conn(0x7fd63c0b1480 msgr2=0x7fd63c06c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:59.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.745+0000 7fd6413f4640 1 -- 172.21.15.43:0/2260300044 shutdown_connections 2024-04-25T09:03:59.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.745+0000 7fd6413f4640 1 -- 172.21.15.43:0/2260300044 wait complete. 2024-04-25T09:03:59.745 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:03:59.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.857+0000 7f6fc24cb640 1 Processor -- start 2024-04-25T09:03:59.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.857+0000 7f6fc24cb640 1 -- start start 2024-04-25T09:03:59.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.858+0000 7f6fc24cb640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6fbc15be60 0x7f6fbc0ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:59.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.858+0000 7f6fc24cb640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6fbc0d0480 0x7f6fbc0b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:59.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.858+0000 7f6fc24cb640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fbc0cc760 con 0x7f6fbc0cedc0 2024-04-25T09:03:59.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.858+0000 7f6fc24cb640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fbc15c240 con 0x7f6fbc15be60 2024-04-25T09:03:59.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.858+0000 7f6fc24cb640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fbc0cca40 con 0x7f6fbc0d0480 2024-04-25T09:03:59.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.858+0000 7f6fbbfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6fbc15be60 0x7f6fbc0ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:59.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.858+0000 7f6fbbfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6fbc15be60 0x7f6fbc0ce880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41310/0 (socket says 172.21.15.43:41310) 2024-04-25T09:03:59.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.858+0000 7f6fbbfff640 1 -- 172.21.15.43:0/524842158 learned_addr learned my addr 172.21.15.43:0/524842158 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:59.857 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.858+0000 7f6fbbfff640 1 -- 172.21.15.43:0/524842158 >> v2:172.21.15.193:3300/0 conn(0x7f6fbc0d0480 msgr2=0x7f6fbc0b0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:59.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.858+0000 7f6fc0a68640 1 --2- 172.21.15.43:0/524842158 >> v2:172.21.15.193:3300/0 conn(0x7f6fbc0d0480 0x7f6fbc0b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:59.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.858+0000 7f6fbbfff640 1 --2- 172.21.15.43:0/524842158 >> v2:172.21.15.193:3300/0 conn(0x7f6fbc0d0480 0x7f6fbc0b0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.858+0000 7f6fbbfff640 1 -- 172.21.15.43:0/524842158 >> v1:172.21.15.193:6789/0 conn(0x7f6fbc0cedc0 legacy=0x7f6fbc0cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:03:59.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.858+0000 7f6fbbfff640 1 -- 172.21.15.43:0/524842158 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fbc0ccd20 con 0x7f6fbc15be60 2024-04-25T09:03:59.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.858+0000 7f6fc0a68640 1 --2- 172.21.15.43:0/524842158 >> v2:172.21.15.193:3300/0 conn(0x7f6fbc0d0480 0x7f6fbc0b0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:03:59.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.859+0000 7f6fbbfff640 1 --2- 172.21.15.43:0/524842158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fbc15be60 0x7f6fbc0ce880 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc071de0 tx=0x7f6fa402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5426f6f243a2d684 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:59.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.859+0000 7f6fbaffd640 1 -- 172.21.15.43:0/524842158 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4037070 con 0x7f6fbc15be60 2024-04-25T09:03:59.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.859+0000 7f6fbaffd640 1 -- 172.21.15.43:0/524842158 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa4032440 con 0x7f6fbc15be60 2024-04-25T09:03:59.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.859+0000 7f6fbaffd640 1 -- 172.21.15.43:0/524842158 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4032810 con 0x7f6fbc15be60 2024-04-25T09:03:59.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.860+0000 7f6fc24cb640 1 -- 172.21.15.43:0/524842158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fbc15be60 msgr2=0x7f6fbc0ce880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:59.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.860+0000 7f6fc24cb640 1 --2- 172.21.15.43:0/524842158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fbc15be60 0x7f6fbc0ce880 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc071de0 tx=0x7f6fa402a230 comp rx=0 tx=0).stop 2024-04-25T09:03:59.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.860+0000 7f6fc24cb640 1 -- 172.21.15.43:0/524842158 shutdown_connections 2024-04-25T09:03:59.858 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.860+0000 7f6fc24cb640 1 --2- 172.21.15.43:0/524842158 >> v2:172.21.15.193:3300/0 conn(0x7f6fbc0d0480 0x7f6fbc0b0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.860+0000 7f6fc24cb640 1 --2- 172.21.15.43:0/524842158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fbc15be60 0x7f6fbc0ce880 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.860+0000 7f6fc24cb640 1 -- 172.21.15.43:0/524842158 >> 172.21.15.43:0/524842158 conn(0x7f6fbc0b32e0 msgr2=0x7f6fbc0b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:03:59.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.860+0000 7f6fc24cb640 1 -- 172.21.15.43:0/524842158 shutdown_connections 2024-04-25T09:03:59.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.860+0000 7f6fc24cb640 1 -- 172.21.15.43:0/524842158 wait complete. 2024-04-25T09:03:59.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.861+0000 7f6fc24cb640 1 Processor -- start 2024-04-25T09:03:59.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.861+0000 7f6fc24cb640 1 -- start start 2024-04-25T09:03:59.859 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.862+0000 7f6fc24cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fbc0cedc0 0x7f6fbc0b6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:59.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.862+0000 7f6fc24cb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6fbc0d0480 0x7f6fbc1551b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:59.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.862+0000 7f6fc24cb640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6fbc15be60 0x7f6fbc0c5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:59.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.862+0000 7f6fc24cb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc0cc4b0 con 0x7f6fbc0cedc0 2024-04-25T09:03:59.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.862+0000 7f6fc24cb640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc0cc330 con 0x7f6fbc0d0480 2024-04-25T09:03:59.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.862+0000 7f6fc24cb640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc0b1390 con 0x7f6fbc15be60 2024-04-25T09:03:59.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.862+0000 7f6fbbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fbc0cedc0 0x7f6fbc0b6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:59.860 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.862+0000 7f6fbbfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fbc0cedc0 0x7f6fbc0b6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41322/0 (socket says 172.21.15.43:41322) 2024-04-25T09:03:59.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.862+0000 7f6fbbfff640 1 -- 172.21.15.43:0/303914506 learned_addr learned my addr 172.21.15.43:0/303914506 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:03:59.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.862+0000 7f6fbbfff640 1 -- 172.21.15.43:0/303914506 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6fbc15be60 msgr2=0x7f6fbc0c5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:59.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.862+0000 7f6fbbfff640 1 --2- 172.21.15.43:0/303914506 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6fbc15be60 0x7f6fbc0c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.862+0000 7f6fbbfff640 1 -- 172.21.15.43:0/303914506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6fbc0d0480 msgr2=0x7f6fbc1551b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:03:59.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.862+0000 7f6fbbfff640 1 --2- 172.21.15.43:0/303914506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6fbc0d0480 0x7f6fbc1551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:03:59.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.862+0000 7f6fbbfff640 1 -- 172.21.15.43:0/303914506 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fbc15d760 con 0x7f6fbc0cedc0 2024-04-25T09:03:59.861 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.863+0000 7f6fbbfff640 1 --2- 172.21.15.43:0/303914506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fbc0cedc0 0x7f6fbc0b6ab0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4000c00 tx=0x7f6fa40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:59.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.863+0000 7f6fb8ff9640 1 -- 172.21.15.43:0/303914506 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4041020 con 0x7f6fbc0cedc0 2024-04-25T09:03:59.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.863+0000 7f6fb8ff9640 1 -- 172.21.15.43:0/303914506 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fa4030970 con 0x7f6fbc0cedc0 2024-04-25T09:03:59.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.863+0000 7f6fb8ff9640 1 -- 172.21.15.43:0/303914506 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa4037040 con 0x7f6fbc0cedc0 2024-04-25T09:03:59.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.863+0000 7f6fc24cb640 1 -- 172.21.15.43:0/303914506 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fbc15da40 con 0x7f6fbc0cedc0 2024-04-25T09:03:59.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.863+0000 7f6fc24cb640 1 -- 172.21.15.43:0/303914506 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fbc15e000 con 0x7f6fbc0cedc0 2024-04-25T09:03:59.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.864+0000 7f6fc24cb640 1 -- 172.21.15.43:0/303914506 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fbc0cfb50 con 0x7f6fbc0cedc0 2024-04-25T09:03:59.862 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.864+0000 7f6fb8ff9640 1 -- 172.21.15.43:0/303914506 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6fa4044020 con 0x7f6fbc0cedc0 2024-04-25T09:03:59.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.865+0000 7f6fb8ff9640 1 --2- 172.21.15.43:0/303914506 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6f9c06b500 0x7f6f9c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:03:59.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.865+0000 7f6fb8ff9640 1 -- 172.21.15.43:0/303914506 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f6fa40aadf0 con 0x7f6fbc0cedc0 2024-04-25T09:03:59.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.865+0000 7f6fbb7fe640 1 --2- 172.21.15.43:0/303914506 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6f9c06b500 0x7f6f9c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:03:59.863 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.865+0000 7f6fbb7fe640 1 --2- 172.21.15.43:0/303914506 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6f9c06b500 0x7f6f9c06d9c0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc0b7670 tx=0x7f6fac007a00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:03:59.864 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:03:59.869+0000 7f6fb8ff9640 1 -- 172.21.15.43:0/303914506 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6fa404b030 con 0x7f6fbc0cedc0 2024-04-25T09:04:00.040 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.052+0000 7f6fc24cb640 1 -- 172.21.15.43:0/303914506 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6fbc0b7710 con 0x7f6fbc0cedc0 2024-04-25T09:04:00.041 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.053+0000 7f6fb8ff9640 1 -- 172.21.15.43:0/303914506 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) v1 ==== 74+0+16814 (secure 0 0 0) 0x7f6fa4079ab0 con 0x7f6fbc0cedc0 2024-04-25T09:04:00.042 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:04:00.042 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":129,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:03:55.660750+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:03:10.199822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T09:03:10.892664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.31","mappings":[{"from":2,"to":0}]},{"pgid":"19.3c","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6834/2603892772":"2024-04-26T09:03:52.443908+0000","172.21.15.43:6832/2603892772":"2024-04-26T09:03:52.443908+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:04:00.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.055+0000 7f6fc24cb640 1 -- 172.21.15.43:0/303914506 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6f9c06b500 msgr2=0x7f6f9c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:00.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.055+0000 7f6fc24cb640 1 --2- 172.21.15.43:0/303914506 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6f9c06b500 0x7f6f9c06d9c0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc0b7670 tx=0x7f6fac007a00 comp rx=0 tx=0).stop 2024-04-25T09:04:00.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.055+0000 7f6fc24cb640 1 -- 172.21.15.43:0/303914506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fbc0cedc0 msgr2=0x7f6fbc0b6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:00.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.055+0000 7f6fc24cb640 1 --2- 172.21.15.43:0/303914506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fbc0cedc0 0x7f6fbc0b6ab0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f6fa4000c00 tx=0x7f6fa40046c0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.055+0000 7f6fc24cb640 1 -- 172.21.15.43:0/303914506 shutdown_connections 2024-04-25T09:04:00.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.055+0000 7f6fc24cb640 1 --2- 172.21.15.43:0/303914506 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6fbc15be60 0x7f6fbc0c5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.055+0000 7f6fc24cb640 1 --2- 172.21.15.43:0/303914506 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6f9c06b500 0x7f6f9c06d9c0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.055+0000 7f6fc24cb640 1 --2- 172.21.15.43:0/303914506 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6fbc0d0480 0x7f6fbc1551b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.055+0000 7f6fc24cb640 1 --2- 172.21.15.43:0/303914506 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6fbc0cedc0 0x7f6fbc0b6ab0 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.055+0000 7f6fc24cb640 1 -- 172.21.15.43:0/303914506 >> 172.21.15.43:0/303914506 conn(0x7f6fbc0b32e0 msgr2=0x7f6fbc06f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:00.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.055+0000 7f6fc24cb640 1 -- 172.21.15.43:0/303914506 shutdown_connections 2024-04-25T09:04:00.045 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.055+0000 7f6fc24cb640 1 -- 172.21.15.43:0/303914506 wait complete. 2024-04-25T09:04:00.055 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:03:10.199822+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '125', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:03:10.892664+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:04:00.055 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:04:00.159 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.171+0000 7f0d8670a640 1 Processor -- start 2024-04-25T09:04:00.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.171+0000 7f0d8670a640 1 -- start start 2024-04-25T09:04:00.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.172+0000 7f0d8670a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0d8015ddd0 0x7f0d800b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:00.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.172+0000 7f0d8670a640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0d8006d4f0 0x7f0d8006f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:00.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.172+0000 7f0d8670a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d8006d940 con 0x7f0d8006ef90 2024-04-25T09:04:00.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.172+0000 7f0d8670a640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d8006f930 con 0x7f0d8006d4f0 2024-04-25T09:04:00.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.172+0000 7f0d8670a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d800d2620 con 0x7f0d8015ddd0 2024-04-25T09:04:00.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.172+0000 7f0d7f7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f0d8006ef90 0x7f0d800cc460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56680/0 (socket says 172.21.15.43:56680) 2024-04-25T09:04:00.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.172+0000 7f0d7f7fe640 1 -- 172.21.15.43:0/1607412922 learned_addr learned my addr 172.21.15.43:0/1607412922 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:00.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.172+0000 7f0d84ca7640 1 --2- 172.21.15.43:0/1607412922 >> v2:172.21.15.193:3300/0 conn(0x7f0d8006d4f0 0x7f0d8006f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:00.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.172+0000 7f0d7ffff640 1 --2- 172.21.15.43:0/1607412922 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0d8015ddd0 0x7f0d800b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:00.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.172+0000 7f0d7effd640 1 -- 172.21.15.43:0/1607412922 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4265636022 0 0) 0x7f0d8006d940 con 0x7f0d8006ef90 2024-04-25T09:04:00.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.172+0000 7f0d7effd640 1 -- 172.21.15.43:0/1607412922 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d6c003520 con 0x7f0d8006ef90 2024-04-25T09:04:00.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.172+0000 7f0d7effd640 1 -- 172.21.15.43:0/1607412922 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3931265682 0 0) 0x7f0d6c003520 con 0x7f0d8006ef90 2024-04-25T09:04:00.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.172+0000 7f0d7effd640 1 -- 172.21.15.43:0/1607412922 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0d8015ddd0 msgr2=0x7f0d800b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:00.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.172+0000 7f0d7effd640 1 --2- 172.21.15.43:0/1607412922 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0d8015ddd0 0x7f0d800b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.172+0000 7f0d7effd640 1 -- 172.21.15.43:0/1607412922 >> v2:172.21.15.193:3300/0 conn(0x7f0d8006d4f0 msgr2=0x7f0d8006f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:00.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.172+0000 7f0d7effd640 1 --2- 172.21.15.43:0/1607412922 >> v2:172.21.15.193:3300/0 conn(0x7f0d8006d4f0 0x7f0d8006f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.172+0000 7f0d7effd640 1 -- 172.21.15.43:0/1607412922 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d800b0b80 con 0x7f0d8006ef90 2024-04-25T09:04:00.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.173+0000 7f0d7effd640 1 -- 172.21.15.43:0/1607412922 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f0d70001670 con 0x7f0d8006ef90 2024-04-25T09:04:00.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.173+0000 7f0d7effd640 1 -- 172.21.15.43:0/1607412922 >> v1:172.21.15.43:6789/0 conn(0x7f0d8006ef90 legacy=0x7f0d800cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:00.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.173+0000 7f0d7effd640 1 --2- 172.21.15.43:0/1607412922 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0d8015ddd0 0x7f0d6c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:00.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.173+0000 7f0d7effd640 1 --2- 172.21.15.43:0/1607412922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d6c003f30 0x7f0d6c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:00.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.173+0000 7f0d7effd640 1 --2- 172.21.15.43:0/1607412922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d6c004d40 0x7f0d6c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:00.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.173+0000 7f0d7effd640 1 -- 172.21.15.43:0/1607412922 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d800d2620 con 0x7f0d6c004d40 2024-04-25T09:04:00.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.173+0000 7f0d7effd640 1 -- 172.21.15.43:0/1607412922 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6c00b9c0 con 0x7f0d6c003f30 2024-04-25T09:04:00.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.173+0000 7f0d7effd640 1 -- 172.21.15.43:0/1607412922 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6c00bb40 con 0x7f0d8015ddd0 2024-04-25T09:04:00.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.173+0000 7f0d84ca7640 1 --2- 172.21.15.43:0/1607412922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d6c004d40 0x7f0d6c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:00.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.173+0000 7f0d7f7fe640 1 --2- 172.21.15.43:0/1607412922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d6c003f30 0x7f0d6c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:00.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.173+0000 7f0d7ffff640 1 --2- 172.21.15.43:0/1607412922 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0d8015ddd0 0x7f0d6c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:00.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.173+0000 7f0d84ca7640 1 -- 172.21.15.43:0/1607412922 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0d8015ddd0 msgr2=0x7f0d6c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:00.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.173+0000 7f0d84ca7640 1 --2- 172.21.15.43:0/1607412922 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0d8015ddd0 0x7f0d6c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.173+0000 7f0d84ca7640 1 -- 172.21.15.43:0/1607412922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d6c003f30 msgr2=0x7f0d6c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:00.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.173+0000 7f0d84ca7640 1 --2- 172.21.15.43:0/1607412922 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d6c003f30 0x7f0d6c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.173+0000 7f0d84ca7640 1 -- 172.21.15.43:0/1607412922 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d6c003520 con 0x7f0d6c004d40 2024-04-25T09:04:00.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.174+0000 7f0d84ca7640 1 --2- 172.21.15.43:0/1607412922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d6c004d40 0x7f0d6c0051c0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f0d78009fc0 tx=0x7f0d6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb18e7ba73cacc3a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:00.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.174+0000 7f0d7effd640 1 -- 172.21.15.43:0/1607412922 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d78009c10 con 0x7f0d6c004d40 2024-04-25T09:04:00.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.174+0000 7f0d7effd640 1 -- 172.21.15.43:0/1607412922 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d78036070 con 0x7f0d6c004d40 2024-04-25T09:04:00.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.174+0000 7f0d7effd640 1 -- 172.21.15.43:0/1607412922 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7802d070 con 0x7f0d6c004d40 2024-04-25T09:04:00.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.174+0000 7f0d8670a640 1 -- 172.21.15.43:0/1607412922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d6c004d40 msgr2=0x7f0d6c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:00.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.174+0000 7f0d8670a640 1 --2- 172.21.15.43:0/1607412922 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d6c004d40 0x7f0d6c0051c0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f0d78009fc0 tx=0x7f0d6c005830 comp rx=0 tx=0).stop 2024-04-25T09:04:00.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.174+0000 7f0d84ca7640 1 -- 172.21.15.43:0/1607412922 reap_dead start 2024-04-25T09:04:00.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.174+0000 7f0d8670a640 1 -- 172.21.15.43:0/1607412922 shutdown_connections 2024-04-25T09:04:00.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.174+0000 7f0d8670a640 1 -- 172.21.15.43:0/1607412922 >> 172.21.15.43:0/1607412922 conn(0x7f0d800d17e0 msgr2=0x7f0d800d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:00.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.174+0000 7f0d8670a640 1 -- 172.21.15.43:0/1607412922 shutdown_connections 2024-04-25T09:04:00.166 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.174+0000 7f0d8670a640 1 -- 172.21.15.43:0/1607412922 wait complete. 2024-04-25T09:04:00.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.176+0000 7f0d8670a640 1 Processor -- start 2024-04-25T09:04:00.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.176+0000 7f0d8670a640 1 -- start start 2024-04-25T09:04:00.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.177+0000 7f0d8670a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d6c004d40 0x7f0d8014e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:00.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.177+0000 7f0d8670a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d800b6a00 0x7f0d800b4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:00.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.177+0000 7f0d8670a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0d801f6e00 0x7f0d801f7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:00.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.177+0000 7f0d8670a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d800b3dd0 con 0x7f0d800b6a00 2024-04-25T09:04:00.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.177+0000 7f0d8670a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d800cbf40 con 0x7f0d6c004d40 2024-04-25T09:04:00.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.177+0000 7f0d8670a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d800b3f50 con 0x7f0d801f6e00 2024-04-25T09:04:00.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.177+0000 7f0d7f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d800b6a00 0x7f0d800b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:00.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.177+0000 7f0d7f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d800b6a00 0x7f0d800b4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41330/0 (socket says 172.21.15.43:41330) 2024-04-25T09:04:00.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.177+0000 7f0d7f7fe640 1 -- 172.21.15.43:0/536431330 learned_addr learned my addr 172.21.15.43:0/536431330 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:00.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.177+0000 7f0d7ffff640 1 --2- 172.21.15.43:0/536431330 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d6c004d40 0x7f0d8014e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:00.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.177+0000 7f0d84ca7640 1 --2- 172.21.15.43:0/536431330 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0d801f6e00 0x7f0d801f7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:00.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.177+0000 7f0d7f7fe640 1 -- 172.21.15.43:0/536431330 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0d801f6e00 msgr2=0x7f0d801f7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:00.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.177+0000 7f0d7f7fe640 1 --2- 172.21.15.43:0/536431330 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0d801f6e00 0x7f0d801f7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.177+0000 7f0d7f7fe640 1 -- 172.21.15.43:0/536431330 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d6c004d40 msgr2=0x7f0d8014e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:00.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.177+0000 7f0d7f7fe640 1 --2- 172.21.15.43:0/536431330 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d6c004d40 0x7f0d8014e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.177+0000 7f0d7f7fe640 1 -- 172.21.15.43:0/536431330 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d800b6e80 con 0x7f0d800b6a00 2024-04-25T09:04:00.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.177+0000 7f0d84ca7640 1 --2- 172.21.15.43:0/536431330 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0d801f6e00 0x7f0d801f7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:00.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.177+0000 7f0d7ffff640 1 --2- 172.21.15.43:0/536431330 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d6c004d40 0x7f0d8014e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:00.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.177+0000 7f0d7f7fe640 1 --2- 172.21.15.43:0/536431330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d800b6a00 0x7f0d800b4590 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f0d70007370 tx=0x7f0d70007830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:00.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.177+0000 7f0d7cff9640 1 -- 172.21.15.43:0/536431330 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7001a070 con 0x7f0d800b6a00 2024-04-25T09:04:00.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.177+0000 7f0d7cff9640 1 -- 172.21.15.43:0/536431330 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7000f9c0 con 0x7f0d800b6a00 2024-04-25T09:04:00.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.177+0000 7f0d7cff9640 1 -- 172.21.15.43:0/536431330 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7000fd90 con 0x7f0d800b6a00 2024-04-25T09:04:00.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.177+0000 7f0d8670a640 1 -- 172.21.15.43:0/536431330 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d800b7130 con 0x7f0d800b6a00 2024-04-25T09:04:00.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.177+0000 7f0d8670a640 1 -- 172.21.15.43:0/536431330 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d8015b130 con 0x7f0d800b6a00 2024-04-25T09:04:00.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.178+0000 7f0d8670a640 1 -- 172.21.15.43:0/536431330 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d800d1510 con 0x7f0d800b6a00 2024-04-25T09:04:00.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.179+0000 7f0d7cff9640 1 -- 172.21.15.43:0/536431330 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0d7001d070 con 0x7f0d800b6a00 2024-04-25T09:04:00.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.179+0000 7f0d7cff9640 1 --2- 172.21.15.43:0/536431330 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0d6006b4d0 0x7f0d6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:00.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.179+0000 7f0d7cff9640 1 -- 172.21.15.43:0/536431330 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f0d70089c60 con 0x7f0d800b6a00 2024-04-25T09:04:00.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.180+0000 7f0d7ffff640 1 --2- 172.21.15.43:0/536431330 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0d6006b4d0 0x7f0d6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:00.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.180+0000 7f0d7ffff640 1 --2- 172.21.15.43:0/536431330 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0d6006b4d0 0x7f0d6006d990 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f0d68009bc0 tx=0x7f0d68008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:00.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.183+0000 7f0d7cff9640 1 -- 172.21.15.43:0/536431330 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0d800d1510 con 0x7f0d800b6a00 2024-04-25T09:04:00.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.367+0000 7f0d8670a640 1 -- 172.21.15.43:0/536431330 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0d80074f60 con 0x7f0d800b6a00 2024-04-25T09:04:00.356 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.367+0000 7f0d7cff9640 1 -- 172.21.15.43:0/536431330 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v129) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0d70058ae0 con 0x7f0d800b6a00 2024-04-25T09:04:00.356 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T09:04:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.369+0000 7f0d8670a640 1 -- 172.21.15.43:0/536431330 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0d6006b4d0 msgr2=0x7f0d6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.369+0000 7f0d8670a640 1 --2- 172.21.15.43:0/536431330 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0d6006b4d0 0x7f0d6006d990 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f0d68009bc0 tx=0x7f0d68008040 comp rx=0 tx=0).stop 2024-04-25T09:04:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.369+0000 7f0d8670a640 1 -- 172.21.15.43:0/536431330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d800b6a00 msgr2=0x7f0d800b4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.369+0000 7f0d8670a640 1 --2- 172.21.15.43:0/536431330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d800b6a00 0x7f0d800b4590 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f0d70007370 tx=0x7f0d70007830 comp rx=0 tx=0).stop 2024-04-25T09:04:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.369+0000 7f0d8670a640 1 -- 172.21.15.43:0/536431330 shutdown_connections 2024-04-25T09:04:00.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.369+0000 7f0d8670a640 1 --2- 172.21.15.43:0/536431330 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0d801f6e00 0x7f0d801f7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.369+0000 7f0d8670a640 1 --2- 172.21.15.43:0/536431330 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0d800b6a00 0x7f0d800b4590 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.369+0000 7f0d8670a640 1 --2- 172.21.15.43:0/536431330 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0d6006b4d0 0x7f0d6006d990 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.369+0000 7f0d8670a640 1 --2- 172.21.15.43:0/536431330 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0d6c004d40 0x7f0d8014e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.369+0000 7f0d8670a640 1 -- 172.21.15.43:0/536431330 >> 172.21.15.43:0/536431330 conn(0x7f0d800d17e0 msgr2=0x7f0d800d0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:00.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.370+0000 7f0d8670a640 1 -- 172.21.15.43:0/536431330 shutdown_connections 2024-04-25T09:04:00.359 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.370+0000 7f0d8670a640 1 -- 172.21.15.43:0/536431330 wait complete. 2024-04-25T09:04:00.369 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:04:00.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.484+0000 7f5fc7807640 1 Processor -- start 2024-04-25T09:04:00.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.485+0000 7f5fc7807640 1 -- start start 2024-04-25T09:04:00.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.485+0000 7f5fc7807640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f5fc015ddd0 0x7f5fc00b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:00.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.485+0000 7f5fc7807640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5fc006ef90 0x7f5fc006f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:00.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.485+0000 7f5fc7807640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fc00d2620 con 0x7f5fc00cc460 2024-04-25T09:04:00.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.485+0000 7f5fc7807640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fc00b0b80 con 0x7f5fc006ef90 2024-04-25T09:04:00.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.485+0000 7f5fc7807640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fc00b0e60 con 0x7f5fc015ddd0 2024-04-25T09:04:00.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.486+0000 7f5fc4da2640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5fc006ef90 0x7f5fc006f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:00.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.486+0000 7f5fc4da2640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f5fc006ef90 0x7f5fc006f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41334/0 (socket says 172.21.15.43:41334) 2024-04-25T09:04:00.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.486+0000 7f5fc4da2640 1 -- 172.21.15.43:0/2268478095 learned_addr learned my addr 172.21.15.43:0/2268478095 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:00.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.486+0000 7f5fc55a3640 1 --2- 172.21.15.43:0/2268478095 >> v2:172.21.15.193:3300/0 conn(0x7f5fc015ddd0 0x7f5fc00b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:00.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.486+0000 7f5fc4da2640 1 -- 172.21.15.43:0/2268478095 >> v2:172.21.15.193:3300/0 conn(0x7f5fc015ddd0 msgr2=0x7f5fc00b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:00.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.486+0000 7f5fc4da2640 1 --2- 172.21.15.43:0/2268478095 >> v2:172.21.15.193:3300/0 conn(0x7f5fc015ddd0 0x7f5fc00b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.486+0000 7f5fc4da2640 1 -- 172.21.15.43:0/2268478095 >> v1:172.21.15.193:6789/0 conn(0x7f5fc00cc460 legacy=0x7f5fc00cc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:00.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.486+0000 7f5fc4da2640 1 -- 172.21.15.43:0/2268478095 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc00cbf40 con 0x7f5fc006ef90 2024-04-25T09:04:00.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.487+0000 7f5fc4da2640 1 --2- 172.21.15.43:0/2268478095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fc006ef90 0x7f5fc006f420 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f5fc006e510 tx=0x7f5fb402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a9e7696087ecf61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:00.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.487+0000 7f5fbffff640 1 -- 172.21.15.43:0/2268478095 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb4037070 con 0x7f5fc006ef90 2024-04-25T09:04:00.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.487+0000 7f5fbffff640 1 -- 172.21.15.43:0/2268478095 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb4032440 con 0x7f5fc006ef90 2024-04-25T09:04:00.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.487+0000 7f5fbffff640 1 -- 172.21.15.43:0/2268478095 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb4032810 con 0x7f5fc006ef90 2024-04-25T09:04:00.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.487+0000 7f5fc7807640 1 -- 172.21.15.43:0/2268478095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fc006ef90 msgr2=0x7f5fc006f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:00.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.487+0000 7f5fc7807640 1 --2- 172.21.15.43:0/2268478095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fc006ef90 0x7f5fc006f420 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f5fc006e510 tx=0x7f5fb402a270 comp rx=0 tx=0).stop 2024-04-25T09:04:00.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.487+0000 7f5fc7807640 1 -- 172.21.15.43:0/2268478095 shutdown_connections 2024-04-25T09:04:00.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.487+0000 7f5fc7807640 1 --2- 172.21.15.43:0/2268478095 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fc006ef90 0x7f5fc006f420 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.487+0000 7f5fc7807640 1 --2- 172.21.15.43:0/2268478095 >> v2:172.21.15.193:3300/0 conn(0x7f5fc015ddd0 0x7f5fc00b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.487+0000 7f5fc7807640 1 -- 172.21.15.43:0/2268478095 >> 172.21.15.43:0/2268478095 conn(0x7f5fc00d17e0 msgr2=0x7f5fc00d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:00.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.487+0000 7f5fc7807640 1 -- 172.21.15.43:0/2268478095 shutdown_connections 2024-04-25T09:04:00.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.487+0000 7f5fc7807640 1 -- 172.21.15.43:0/2268478095 wait complete. 2024-04-25T09:04:00.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.489+0000 7f5fc7807640 1 Processor -- start 2024-04-25T09:04:00.480 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.489+0000 7f5fc7807640 1 -- start start 2024-04-25T09:04:00.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.489+0000 7f5fc7807640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5fc006ef90 0x7f5fc015a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:00.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.489+0000 7f5fc7807640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5fc00cc460 0x7f5fc00cf910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:00.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.489+0000 7f5fc7807640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fc015ddd0 0x7f5fc01e74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:00.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.489+0000 7f5fc7807640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc006d880 con 0x7f5fc015ddd0 2024-04-25T09:04:00.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.489+0000 7f5fc7807640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc006d700 con 0x7f5fc00cc460 2024-04-25T09:04:00.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.489+0000 7f5fc7807640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc00ccf80 con 0x7f5fc006ef90 2024-04-25T09:04:00.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.489+0000 7f5fc5da4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fc015ddd0 0x7f5fc01e74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:00.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.489+0000 7f5fc5da4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fc015ddd0 0x7f5fc01e74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41336/0 (socket says 172.21.15.43:41336) 2024-04-25T09:04:00.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.489+0000 7f5fc5da4640 1 -- 172.21.15.43:0/2858769448 learned_addr learned my addr 172.21.15.43:0/2858769448 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:00.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.490+0000 7f5fc55a3640 1 --2- 172.21.15.43:0/2858769448 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5fc006ef90 0x7f5fc015a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:00.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.490+0000 7f5fc4da2640 1 --2- 172.21.15.43:0/2858769448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5fc00cc460 0x7f5fc00cf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:00.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.490+0000 7f5fc5da4640 1 -- 172.21.15.43:0/2858769448 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5fc006ef90 msgr2=0x7f5fc015a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:00.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.490+0000 7f5fc5da4640 1 --2- 172.21.15.43:0/2858769448 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5fc006ef90 0x7f5fc015a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.490+0000 7f5fc5da4640 1 -- 172.21.15.43:0/2858769448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5fc00cc460 msgr2=0x7f5fc00cf910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:00.481 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.490+0000 7f5fc5da4640 1 --2- 172.21.15.43:0/2858769448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5fc00cc460 0x7f5fc00cf910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.490+0000 7f5fc5da4640 1 -- 172.21.15.43:0/2858769448 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc00d03d0 con 0x7f5fc015ddd0 2024-04-25T09:04:00.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.490+0000 7f5fc5da4640 1 --2- 172.21.15.43:0/2858769448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fc015ddd0 0x7f5fc01e74d0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f5fb000c710 tx=0x7f5fb000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:00.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.490+0000 7f5fbdffb640 1 -- 172.21.15.43:0/2858769448 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb0016020 con 0x7f5fc015ddd0 2024-04-25T09:04:00.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.490+0000 7f5fbdffb640 1 -- 172.21.15.43:0/2858769448 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb0006d50 con 0x7f5fc015ddd0 2024-04-25T09:04:00.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.490+0000 7f5fc7807640 1 -- 172.21.15.43:0/2858769448 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fc00d06b0 con 0x7f5fc015ddd0 2024-04-25T09:04:00.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.490+0000 7f5fbdffb640 1 -- 172.21.15.43:0/2858769448 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb0003bb0 con 0x7f5fc015ddd0 2024-04-25T09:04:00.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.490+0000 7f5fc7807640 1 -- 172.21.15.43:0/2858769448 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fc00d0c70 con 0x7f5fc015ddd0 2024-04-25T09:04:00.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.491+0000 7f5fc7807640 1 -- 172.21.15.43:0/2858769448 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fc0074f60 con 0x7f5fc015ddd0 2024-04-25T09:04:00.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.492+0000 7f5fbdffb640 1 -- 172.21.15.43:0/2858769448 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5fb000a850 con 0x7f5fc015ddd0 2024-04-25T09:04:00.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.492+0000 7f5fbdffb640 1 --2- 172.21.15.43:0/2858769448 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5fac06b4d0 0x7f5fac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:00.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.492+0000 7f5fbdffb640 1 -- 172.21.15.43:0/2858769448 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f5fb0089da0 con 0x7f5fc015ddd0 2024-04-25T09:04:00.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.492+0000 7f5fc55a3640 1 --2- 172.21.15.43:0/2858769448 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5fac06b4d0 0x7f5fac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:00.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.493+0000 7f5fc55a3640 1 --2- 172.21.15.43:0/2858769448 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5fac06b4d0 0x7f5fac06d990 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f5fb80079a0 tx=0x7f5fb8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:00.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.496+0000 7f5fbdffb640 1 -- 172.21.15.43:0/2858769448 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5fb0020020 con 0x7f5fc015ddd0 2024-04-25T09:04:00.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.680+0000 7f5fc7807640 1 -- 172.21.15.43:0/2858769448 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5fc01614f0 con 0x7f5fc015ddd0 2024-04-25T09:04:00.669 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.680+0000 7f5fbdffb640 1 -- 172.21.15.43:0/2858769448 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v129) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5fb000ce20 con 0x7f5fc015ddd0 2024-04-25T09:04:00.669 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T09:04:00.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.683+0000 7f5fc7807640 1 -- 172.21.15.43:0/2858769448 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5fac06b4d0 msgr2=0x7f5fac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:00.671 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.683+0000 7f5fc7807640 1 --2- 172.21.15.43:0/2858769448 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5fac06b4d0 0x7f5fac06d990 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f5fb80079a0 tx=0x7f5fb8008040 comp rx=0 tx=0).stop 2024-04-25T09:04:00.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.683+0000 7f5fc7807640 1 -- 172.21.15.43:0/2858769448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fc015ddd0 msgr2=0x7f5fc01e74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:00.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.683+0000 7f5fc7807640 1 --2- 172.21.15.43:0/2858769448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fc015ddd0 0x7f5fc01e74d0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f5fb000c710 tx=0x7f5fb000cbd0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.683+0000 7f5fc7807640 1 -- 172.21.15.43:0/2858769448 shutdown_connections 2024-04-25T09:04:00.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.683+0000 7f5fc7807640 1 --2- 172.21.15.43:0/2858769448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5fc015ddd0 0x7f5fc01e74d0 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.683+0000 7f5fc7807640 1 --2- 172.21.15.43:0/2858769448 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5fac06b4d0 0x7f5fac06d990 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.683+0000 7f5fc7807640 1 --2- 172.21.15.43:0/2858769448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5fc00cc460 0x7f5fc00cf910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.683+0000 7f5fc7807640 1 --2- 172.21.15.43:0/2858769448 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5fc006ef90 0x7f5fc015a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.683+0000 7f5fc7807640 1 -- 172.21.15.43:0/2858769448 >> 172.21.15.43:0/2858769448 conn(0x7f5fc00d17e0 msgr2=0x7f5fc00d1bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:00.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.683+0000 7f5fc7807640 1 -- 172.21.15.43:0/2858769448 shutdown_connections 2024-04-25T09:04:00.672 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.683+0000 7f5fc7807640 1 -- 172.21.15.43:0/2858769448 wait complete. 2024-04-25T09:04:00.683 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:04:00.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.800+0000 7f87fec38640 1 Processor -- start 2024-04-25T09:04:00.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.800+0000 7f87fec38640 1 -- start start 2024-04-25T09:04:00.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.801+0000 7f87fec38640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f87f806ed70 0x7f87f806f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:00.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.801+0000 7f87fec38640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87f80d1090 0x7f87f80c77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:00.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.801+0000 7f87fec38640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87f815bf00 con 0x7f87f80cbda0 2024-04-25T09:04:00.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.801+0000 7f87fec38640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87f80cfdb0 con 0x7f87f806ed70 2024-04-25T09:04:00.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.801+0000 7f87fec38640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87f806f690 con 0x7f87f80d1090 2024-04-25T09:04:00.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.801+0000 7f87fc9d4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f87f806ed70 0x7f87f806f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:00.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.801+0000 7f87fc9d4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f87f806ed70 0x7f87f806f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41350/0 (socket says 172.21.15.43:41350) 2024-04-25T09:04:00.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.801+0000 7f87fc9d4640 1 -- 172.21.15.43:0/2854385856 learned_addr learned my addr 172.21.15.43:0/2854385856 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:00.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.801+0000 7f87f7fff640 1 --1- 172.21.15.43:0/2854385856 >> v1:172.21.15.43:6789/0 conn(0x7f87f80cbda0 0x7f87f80d0980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56690/0 (socket says 172.21.15.43:56690) 2024-04-25T09:04:00.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.801+0000 7f87fd1d5640 1 --2- 172.21.15.43:0/2854385856 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87f80d1090 0x7f87f80c77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:00.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.801+0000 7f87fc9d4640 1 -- 172.21.15.43:0/2854385856 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87f80d1090 msgr2=0x7f87f80c77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:00.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.801+0000 7f87fc9d4640 1 --2- 172.21.15.43:0/2854385856 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87f80d1090 0x7f87f80c77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.801+0000 7f87fc9d4640 1 -- 172.21.15.43:0/2854385856 >> v1:172.21.15.43:6789/0 conn(0x7f87f80cbda0 legacy=0x7f87f80d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:00.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.801+0000 7f87fc9d4640 1 -- 172.21.15.43:0/2854385856 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f80c6740 con 0x7f87f806ed70 2024-04-25T09:04:00.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.801+0000 7f87f77fe640 1 -- 172.21.15.43:0/2854385856 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 597290296 0 0) 0x7f87f815bf00 con 0x7f87f80cbda0 2024-04-25T09:04:00.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.801+0000 7f87fd1d5640 1 --2- 172.21.15.43:0/2854385856 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87f80d1090 0x7f87f80c77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:00.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.802+0000 7f87fc9d4640 1 --2- 172.21.15.43:0/2854385856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87f806ed70 0x7f87f806f150 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f87ec009870 tx=0x7f87ec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a8d30bc2cc251f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:00.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.802+0000 7f87f77fe640 1 -- 172.21.15.43:0/2854385856 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ec037070 con 0x7f87f806ed70 2024-04-25T09:04:00.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.802+0000 7f87f77fe640 1 -- 172.21.15.43:0/2854385856 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87ec032440 con 0x7f87f806ed70 2024-04-25T09:04:00.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.802+0000 7f87f77fe640 1 -- 172.21.15.43:0/2854385856 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ec032810 con 0x7f87f806ed70 2024-04-25T09:04:00.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.803+0000 7f87fec38640 1 -- 172.21.15.43:0/2854385856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87f806ed70 msgr2=0x7f87f806f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:00.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.803+0000 7f87fec38640 1 --2- 172.21.15.43:0/2854385856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87f806ed70 0x7f87f806f150 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f87ec009870 tx=0x7f87ec02a270 comp rx=0 tx=0).stop 2024-04-25T09:04:00.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.803+0000 7f87fec38640 1 -- 172.21.15.43:0/2854385856 shutdown_connections 2024-04-25T09:04:00.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.803+0000 7f87fec38640 1 --2- 172.21.15.43:0/2854385856 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87f80d1090 0x7f87f80c77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.803+0000 7f87fec38640 1 --2- 172.21.15.43:0/2854385856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87f806ed70 0x7f87f806f150 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.803+0000 7f87fec38640 1 -- 172.21.15.43:0/2854385856 >> 172.21.15.43:0/2854385856 conn(0x7f87f80d17e0 msgr2=0x7f87f80d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:00.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.803+0000 7f87fec38640 1 -- 172.21.15.43:0/2854385856 shutdown_connections 2024-04-25T09:04:00.793 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.803+0000 7f87fec38640 1 -- 172.21.15.43:0/2854385856 wait complete. 2024-04-25T09:04:00.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.804+0000 7f87fec38640 1 Processor -- start 2024-04-25T09:04:00.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.804+0000 7f87fec38640 1 -- start start 2024-04-25T09:04:00.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.805+0000 7f87fec38640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87f806ed70 0x7f87f8150800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:00.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.805+0000 7f87fec38640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87f80cbda0 0x7f87f80ca5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:00.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.805+0000 7f87fec38640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87f80d1090 0x7f87f80cab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:00.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.805+0000 7f87fec38640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f80c6fe0 con 0x7f87f80cbda0 2024-04-25T09:04:00.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.805+0000 7f87fec38640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f80c6d00 con 0x7f87f80d1090 2024-04-25T09:04:00.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.805+0000 7f87fec38640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87f80c72c0 con 0x7f87f806ed70 2024-04-25T09:04:00.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.805+0000 7f87f7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87f80cbda0 0x7f87f80ca5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:00.796 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.805+0000 7f87f7fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87f80cbda0 0x7f87f80ca5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41360/0 (socket says 172.21.15.43:41360) 2024-04-25T09:04:00.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.805+0000 7f87f7fff640 1 -- 172.21.15.43:0/3942291424 learned_addr learned my addr 172.21.15.43:0/3942291424 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:00.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.805+0000 7f87fc9d4640 1 --2- 172.21.15.43:0/3942291424 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87f806ed70 0x7f87f8150800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:00.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.805+0000 7f87fd1d5640 1 --2- 172.21.15.43:0/3942291424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87f80d1090 0x7f87f80cab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:00.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.805+0000 7f87f7fff640 1 -- 172.21.15.43:0/3942291424 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87f806ed70 msgr2=0x7f87f8150800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:00.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.805+0000 7f87f7fff640 1 --2- 172.21.15.43:0/3942291424 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87f806ed70 0x7f87f8150800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.805+0000 7f87f7fff640 1 -- 172.21.15.43:0/3942291424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87f80d1090 msgr2=0x7f87f80cab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:00.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.805+0000 7f87f7fff640 1 --2- 172.21.15.43:0/3942291424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87f80d1090 0x7f87f80cab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.805+0000 7f87f7fff640 1 -- 172.21.15.43:0/3942291424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f81582b0 con 0x7f87f80cbda0 2024-04-25T09:04:00.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.805+0000 7f87f7fff640 1 --2- 172.21.15.43:0/3942291424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87f80cbda0 0x7f87f80ca5e0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f87e400c460 tx=0x7f87e400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:00.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.806+0000 7f87f57fa640 1 -- 172.21.15.43:0/3942291424 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e4018070 con 0x7f87f80cbda0 2024-04-25T09:04:00.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.806+0000 7f87f57fa640 1 -- 172.21.15.43:0/3942291424 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87e4007500 con 0x7f87f80cbda0 2024-04-25T09:04:00.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.806+0000 7f87fec38640 1 -- 172.21.15.43:0/3942291424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87f80cb2a0 con 0x7f87f80cbda0 2024-04-25T09:04:00.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.806+0000 7f87f57fa640 1 -- 172.21.15.43:0/3942291424 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e4004a20 con 0x7f87f80cbda0 2024-04-25T09:04:00.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.806+0000 7f87fec38640 1 -- 172.21.15.43:0/3942291424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87f806bce0 con 0x7f87f80cbda0 2024-04-25T09:04:00.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.807+0000 7f87fec38640 1 -- 172.21.15.43:0/3942291424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87f8074f60 con 0x7f87f80cbda0 2024-04-25T09:04:00.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.807+0000 7f87f57fa640 1 -- 172.21.15.43:0/3942291424 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f87e401b070 con 0x7f87f80cbda0 2024-04-25T09:04:00.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.807+0000 7f87f57fa640 1 --2- 172.21.15.43:0/3942291424 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f87dc06b4d0 0x7f87dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:00.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.808+0000 7f87f57fa640 1 -- 172.21.15.43:0/3942291424 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f87e4089530 con 0x7f87f80cbda0 2024-04-25T09:04:00.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.808+0000 7f87fc9d4640 1 --2- 172.21.15.43:0/3942291424 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f87dc06b4d0 0x7f87dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:00.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.808+0000 7f87fc9d4640 1 --2- 172.21.15.43:0/3942291424 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f87dc06b4d0 0x7f87dc06d990 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f87ec034000 tx=0x7f87ec02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:00.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.812+0000 7f87f57fa640 1 -- 172.21.15.43:0/3942291424 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f87e4058320 con 0x7f87f80cbda0 2024-04-25T09:04:00.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.991+0000 7f87fec38640 1 -- 172.21.15.43:0/3942291424 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f87f8074030 con 0x7f87f80cbda0 2024-04-25T09:04:00.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.992+0000 7f87f57fa640 1 -- 172.21.15.43:0/3942291424 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v129) v1 ==== 100+0+11 (secure 0 0 0) 0x7f87e4057c20 con 0x7f87f80cbda0 2024-04-25T09:04:00.980 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T09:04:00.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.993+0000 7f87fec38640 1 -- 172.21.15.43:0/3942291424 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f87dc06b4d0 msgr2=0x7f87dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:00.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.993+0000 7f87fec38640 1 --2- 172.21.15.43:0/3942291424 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f87dc06b4d0 0x7f87dc06d990 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f87ec034000 tx=0x7f87ec02e000 comp rx=0 tx=0).stop 2024-04-25T09:04:00.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.993+0000 7f87fec38640 1 -- 172.21.15.43:0/3942291424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87f80cbda0 msgr2=0x7f87f80ca5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:00.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.993+0000 7f87fec38640 1 --2- 172.21.15.43:0/3942291424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87f80cbda0 0x7f87f80ca5e0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f87e400c460 tx=0x7f87e400c920 comp rx=0 tx=0).stop 2024-04-25T09:04:00.982 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.994+0000 7f87fec38640 1 -- 172.21.15.43:0/3942291424 shutdown_connections 2024-04-25T09:04:00.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.994+0000 7f87fec38640 1 --2- 172.21.15.43:0/3942291424 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f87dc06b4d0 0x7f87dc06d990 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.994+0000 7f87fec38640 1 --2- 172.21.15.43:0/3942291424 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f87f80d1090 0x7f87f80cab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.994+0000 7f87fec38640 1 --2- 172.21.15.43:0/3942291424 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f87f80cbda0 0x7f87f80ca5e0 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.994+0000 7f87fec38640 1 --2- 172.21.15.43:0/3942291424 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f87f806ed70 0x7f87f8150800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:00.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.994+0000 7f87fec38640 1 -- 172.21.15.43:0/3942291424 >> 172.21.15.43:0/3942291424 conn(0x7f87f80d17e0 msgr2=0x7f87f80af910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:00.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.994+0000 7f87fec38640 1 -- 172.21.15.43:0/3942291424 shutdown_connections 2024-04-25T09:04:00.983 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:00.994+0000 7f87fec38640 1 -- 172.21.15.43:0/3942291424 wait complete. 2024-04-25T09:04:00.994 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:04:01.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.106+0000 7f2a83f42640 1 Processor -- start 2024-04-25T09:04:01.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.106+0000 7f2a83f42640 1 -- start start 2024-04-25T09:04:01.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.106+0000 7f2a83f42640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2a7c15dce0 0x7f2a7c15ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:01.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.107+0000 7f2a83f42640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a7c0ce550 0x7f2a7c0d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:01.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.107+0000 7f2a83f42640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f2a7c0ce990 0x7f2a7c0d0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:01.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.107+0000 7f2a83f42640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a7c0cee80 con 0x7f2a7c15dce0 2024-04-25T09:04:01.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.107+0000 7f2a83f42640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a7c0cf160 con 0x7f2a7c0ce990 2024-04-25T09:04:01.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.107+0000 7f2a83f42640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c0cf440 con 0x7f2a7c0ce550 2024-04-25T09:04:01.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.107+0000 7f2a81cde640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2a7c15dce0 0x7f2a7c15ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:01.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.107+0000 7f2a81cde640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2a7c15dce0 0x7f2a7c15ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41364/0 (socket says 172.21.15.43:41364) 2024-04-25T09:04:01.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.107+0000 7f2a81cde640 1 -- 172.21.15.43:0/2931641308 learned_addr learned my addr 172.21.15.43:0/2931641308 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:01.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.107+0000 7f2a814dd640 1 --2- 172.21.15.43:0/2931641308 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a7c0ce550 0x7f2a7c0d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:01.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.107+0000 7f2a824df640 1 --2- 172.21.15.43:0/2931641308 >> v2:172.21.15.193:3300/0 conn(0x7f2a7c0ce990 0x7f2a7c0d0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:01.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.107+0000 7f2a81cde640 1 -- 172.21.15.43:0/2931641308 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a7c0ce550 msgr2=0x7f2a7c0d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:01.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.107+0000 7f2a81cde640 1 --2- 172.21.15.43:0/2931641308 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a7c0ce550 0x7f2a7c0d03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.107+0000 7f2a81cde640 1 -- 172.21.15.43:0/2931641308 >> v2:172.21.15.193:3300/0 conn(0x7f2a7c0ce990 msgr2=0x7f2a7c0d0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:01.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.107+0000 7f2a81cde640 1 --2- 172.21.15.43:0/2931641308 >> v2:172.21.15.193:3300/0 conn(0x7f2a7c0ce990 0x7f2a7c0d0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.107+0000 7f2a81cde640 1 -- 172.21.15.43:0/2931641308 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a7c0cf720 con 0x7f2a7c15dce0 2024-04-25T09:04:01.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.108+0000 7f2a81cde640 1 --2- 172.21.15.43:0/2931641308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a7c15dce0 0x7f2a7c15ef50 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c0d1480 tx=0x7f2a7802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dba108b8e412f16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:01.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.108+0000 7f2a80cdc640 1 -- 172.21.15.43:0/2931641308 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a78037070 con 0x7f2a7c15dce0 2024-04-25T09:04:01.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.108+0000 7f2a80cdc640 1 -- 172.21.15.43:0/2931641308 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a78032440 con 0x7f2a7c15dce0 2024-04-25T09:04:01.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.108+0000 7f2a80cdc640 1 -- 172.21.15.43:0/2931641308 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a78032810 con 0x7f2a7c15dce0 2024-04-25T09:04:01.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.109+0000 7f2a83f42640 1 -- 172.21.15.43:0/2931641308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a7c15dce0 msgr2=0x7f2a7c15ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:01.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.109+0000 7f2a83f42640 1 --2- 172.21.15.43:0/2931641308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a7c15dce0 0x7f2a7c15ef50 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c0d1480 tx=0x7f2a7802a140 comp rx=0 tx=0).stop 2024-04-25T09:04:01.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.109+0000 7f2a83f42640 1 -- 172.21.15.43:0/2931641308 shutdown_connections 2024-04-25T09:04:01.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.109+0000 7f2a83f42640 1 --2- 172.21.15.43:0/2931641308 >> v2:172.21.15.193:3300/0 conn(0x7f2a7c0ce990 0x7f2a7c0d0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.109+0000 7f2a83f42640 1 --2- 172.21.15.43:0/2931641308 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a7c0ce550 0x7f2a7c0d03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.109+0000 7f2a83f42640 1 --2- 172.21.15.43:0/2931641308 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a7c15dce0 0x7f2a7c15ef50 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.109+0000 7f2a83f42640 1 -- 172.21.15.43:0/2931641308 >> 172.21.15.43:0/2931641308 conn(0x7f2a7c09cd00 msgr2=0x7f2a7c0d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:01.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.109+0000 7f2a83f42640 1 -- 172.21.15.43:0/2931641308 shutdown_connections 2024-04-25T09:04:01.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.109+0000 7f2a83f42640 1 -- 172.21.15.43:0/2931641308 wait complete. 2024-04-25T09:04:01.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.110+0000 7f2a83f42640 1 Processor -- start 2024-04-25T09:04:01.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.110+0000 7f2a83f42640 1 -- start start 2024-04-25T09:04:01.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.111+0000 7f2a83f42640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a7c0ce550 0x7f2a7c0b88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:01.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.111+0000 7f2a83f42640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a7c0ce990 0x7f2a7c15af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:01.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.111+0000 7f2a83f42640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a7c15dce0 0x7f2a7c06c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:01.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.111+0000 7f2a83f42640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c160980 con 0x7f2a7c0ce990 2024-04-25T09:04:01.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.111+0000 7f2a83f42640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c15e0c0 con 0x7f2a7c0ce550 2024-04-25T09:04:01.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.111+0000 7f2a83f42640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7c160b00 con 0x7f2a7c15dce0 2024-04-25T09:04:01.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.111+0000 7f2a814dd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a7c0ce990 0x7f2a7c15af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:01.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.111+0000 7f2a814dd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a7c0ce990 0x7f2a7c15af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41378/0 (socket says 172.21.15.43:41378) 2024-04-25T09:04:01.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.111+0000 7f2a814dd640 1 -- 172.21.15.43:0/3393878857 learned_addr learned my addr 172.21.15.43:0/3393878857 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:01.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.111+0000 7f2a824df640 1 --2- 172.21.15.43:0/3393878857 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a7c15dce0 0x7f2a7c06c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:01.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.111+0000 7f2a81cde640 1 --2- 172.21.15.43:0/3393878857 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a7c0ce550 0x7f2a7c0b88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:01.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.111+0000 7f2a814dd640 1 -- 172.21.15.43:0/3393878857 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a7c15dce0 msgr2=0x7f2a7c06c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:01.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.111+0000 7f2a814dd640 1 --2- 172.21.15.43:0/3393878857 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a7c15dce0 0x7f2a7c06c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.111+0000 7f2a814dd640 1 -- 172.21.15.43:0/3393878857 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a7c0ce550 msgr2=0x7f2a7c0b88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:01.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.111+0000 7f2a814dd640 1 --2- 172.21.15.43:0/3393878857 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a7c0ce550 0x7f2a7c0b88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.111+0000 7f2a814dd640 1 -- 172.21.15.43:0/3393878857 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a7c0b8ea0 con 0x7f2a7c0ce990 2024-04-25T09:04:01.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.111+0000 7f2a81cde640 1 --2- 172.21.15.43:0/3393878857 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a7c0ce550 0x7f2a7c0b88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:01.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.111+0000 7f2a814dd640 1 --2- 172.21.15.43:0/3393878857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a7c0ce990 0x7f2a7c15af80 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c00e9e0 tx=0x7f2a6c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:01.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.111+0000 7f2a727fc640 1 -- 172.21.15.43:0/3393878857 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c017070 con 0x7f2a7c0ce990 2024-04-25T09:04:01.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.111+0000 7f2a727fc640 1 -- 172.21.15.43:0/3393878857 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a6c00c9d0 con 0x7f2a7c0ce990 2024-04-25T09:04:01.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.111+0000 7f2a727fc640 1 -- 172.21.15.43:0/3393878857 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c00cdc0 con 0x7f2a7c0ce990 2024-04-25T09:04:01.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.111+0000 7f2a83f42640 1 -- 172.21.15.43:0/3393878857 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a7c1f4460 con 0x7f2a7c0ce990 2024-04-25T09:04:01.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.112+0000 7f2a83f42640 1 -- 172.21.15.43:0/3393878857 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a7c1f4150 con 0x7f2a7c0ce990 2024-04-25T09:04:01.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.113+0000 7f2a83f42640 1 -- 172.21.15.43:0/3393878857 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a7c0680f0 con 0x7f2a7c0ce990 2024-04-25T09:04:01.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.113+0000 7f2a727fc640 1 -- 172.21.15.43:0/3393878857 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2a6c01a070 con 0x7f2a7c0ce990 2024-04-25T09:04:01.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.113+0000 7f2a727fc640 1 --2- 172.21.15.43:0/3393878857 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2a6006b500 0x7f2a6006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:01.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.114+0000 7f2a727fc640 1 -- 172.21.15.43:0/3393878857 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f2a6c08a590 con 0x7f2a7c0ce990 2024-04-25T09:04:01.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.114+0000 7f2a81cde640 1 --2- 172.21.15.43:0/3393878857 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2a6006b500 0x7f2a6006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:01.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.114+0000 7f2a81cde640 1 --2- 172.21.15.43:0/3393878857 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2a6006b500 0x7f2a6006d9c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f2a78034000 tx=0x7f2a7802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:01.106 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.118+0000 7f2a727fc640 1 -- 172.21.15.43:0/3393878857 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2a6c058d30 con 0x7f2a7c0ce990 2024-04-25T09:04:01.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.333+0000 7f2a83f42640 1 -- 172.21.15.43:0/3393878857 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a7c09d1a0 con 0x7f2a7c0ce990 2024-04-25T09:04:01.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.334+0000 7f2a727fc640 1 -- 172.21.15.43:0/3393878857 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4155 (secure 0 0 0) 0x7f2a6c05cdd0 con 0x7f2a7c0ce990 2024-04-25T09:04:01.322 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:04:01.322 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":80,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/891225068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":891225068},{"type":"v1","addr":"172.21.15.193:6836","nonce":891225068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":7175,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3679059390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":3679059390},{"type":"v1","addr":"172.21.15.193:6837","nonce":3679059390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:11.919337+0000","modified":"2024-04-25T09:03:54.462418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7948},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7948":{"gid":7948,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":15,"addr":"172.21.15.43:6835/2127363183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2127363183},{"type":"v1","addr":"172.21.15.43:6835","nonce":2127363183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:04:01.323 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 80 2024-04-25T09:04:01.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.335+0000 7f2a83f42640 1 -- 172.21.15.43:0/3393878857 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2a6006b500 msgr2=0x7f2a6006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:01.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.335+0000 7f2a83f42640 1 --2- 172.21.15.43:0/3393878857 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2a6006b500 0x7f2a6006d9c0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f2a78034000 tx=0x7f2a7802e000 comp rx=0 tx=0).stop 2024-04-25T09:04:01.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.335+0000 7f2a83f42640 1 -- 172.21.15.43:0/3393878857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a7c0ce990 msgr2=0x7f2a7c15af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:01.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.335+0000 7f2a83f42640 1 --2- 172.21.15.43:0/3393878857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a7c0ce990 0x7f2a7c15af80 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c00e9e0 tx=0x7f2a6c00eea0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.335+0000 7f2a83f42640 1 -- 172.21.15.43:0/3393878857 shutdown_connections 2024-04-25T09:04:01.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.335+0000 7f2a83f42640 1 --2- 172.21.15.43:0/3393878857 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2a7c15dce0 0x7f2a7c06c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.335+0000 7f2a83f42640 1 --2- 172.21.15.43:0/3393878857 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2a7c0ce990 0x7f2a7c15af80 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.335+0000 7f2a83f42640 1 --2- 172.21.15.43:0/3393878857 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f2a6006b500 0x7f2a6006d9c0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.335+0000 7f2a83f42640 1 --2- 172.21.15.43:0/3393878857 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2a7c0ce550 0x7f2a7c0b88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.335+0000 7f2a83f42640 1 -- 172.21.15.43:0/3393878857 >> 172.21.15.43:0/3393878857 conn(0x7f2a7c09cd00 msgr2=0x7f2a7c0cfa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:01.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.336+0000 7f2a83f42640 1 -- 172.21.15.43:0/3393878857 shutdown_connections 2024-04-25T09:04:01.325 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.336+0000 7f2a83f42640 1 -- 172.21.15.43:0/3393878857 wait complete. 2024-04-25T09:04:01.336 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:04:01.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.448+0000 7f7352d09640 1 Processor -- start 2024-04-25T09:04:01.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.448+0000 7f7352d09640 1 -- start start 2024-04-25T09:04:01.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.449+0000 7f7352d09640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f734c15dce0 0x7f734c15ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:01.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.449+0000 7f7352d09640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f734c0d0af0 0x7f734c160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:01.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.449+0000 7f7352d09640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f734c0cee80 con 0x7f734c0ce550 2024-04-25T09:04:01.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.449+0000 7f7352d09640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f734c0d2920 con 0x7f734c15dce0 2024-04-25T09:04:01.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.449+0000 7f7352d09640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f734c0cf160 con 0x7f734c0d0af0 2024-04-25T09:04:01.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.449+0000 7f73512a6640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f734c0d0af0 0x7f734c160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:01.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.449+0000 7f7350aa5640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f734c15dce0 0x7f734c15ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:01.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.449+0000 7f73512a6640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f734c0d0af0 0x7f734c160980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:48592/0 (socket says 172.21.15.43:48592) 2024-04-25T09:04:01.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.449+0000 7f734bfff640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f734c0ce550 0x7f734c0d03e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:35232/0 (socket says 172.21.15.43:35232) 2024-04-25T09:04:01.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.449+0000 7f73512a6640 1 -- 172.21.15.43:0/1569455067 learned_addr learned my addr 172.21.15.43:0/1569455067 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:01.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.449+0000 7f73512a6640 1 -- 172.21.15.43:0/1569455067 >> v2:172.21.15.193:3300/0 conn(0x7f734c15dce0 msgr2=0x7f734c15ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:01.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.449+0000 7f73512a6640 1 --2- 172.21.15.43:0/1569455067 >> v2:172.21.15.193:3300/0 conn(0x7f734c15dce0 0x7f734c15ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.449+0000 7f73512a6640 1 -- 172.21.15.43:0/1569455067 >> v1:172.21.15.193:6789/0 conn(0x7f734c0ce550 legacy=0x7f734c0d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:01.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.449+0000 7f73512a6640 1 -- 172.21.15.43:0/1569455067 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f734c0cf440 con 0x7f734c0d0af0 2024-04-25T09:04:01.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.450+0000 7f73512a6640 1 --2- 172.21.15.43:0/1569455067 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f734c0d0af0 0x7f734c160980 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7344009e50 tx=0x7f734402a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a77d638f9458177 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:01.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.450+0000 7f734b7fe640 1 -- 172.21.15.43:0/1569455067 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7344036070 con 0x7f734c0d0af0 2024-04-25T09:04:01.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.450+0000 7f734b7fe640 1 -- 172.21.15.43:0/1569455067 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7344031600 con 0x7f734c0d0af0 2024-04-25T09:04:01.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.450+0000 7f734b7fe640 1 -- 172.21.15.43:0/1569455067 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73440319d0 con 0x7f734c0d0af0 2024-04-25T09:04:01.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.451+0000 7f7352d09640 1 -- 172.21.15.43:0/1569455067 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f734c0d0af0 msgr2=0x7f734c160980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:01.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.451+0000 7f7352d09640 1 --2- 172.21.15.43:0/1569455067 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f734c0d0af0 0x7f734c160980 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7344009e50 tx=0x7f734402a680 comp rx=0 tx=0).stop 2024-04-25T09:04:01.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.451+0000 7f7352d09640 1 -- 172.21.15.43:0/1569455067 shutdown_connections 2024-04-25T09:04:01.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.451+0000 7f7352d09640 1 --2- 172.21.15.43:0/1569455067 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f734c0d0af0 0x7f734c160980 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.451+0000 7f7352d09640 1 --2- 172.21.15.43:0/1569455067 >> v2:172.21.15.193:3300/0 conn(0x7f734c15dce0 0x7f734c15ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.451+0000 7f7352d09640 1 -- 172.21.15.43:0/1569455067 >> 172.21.15.43:0/1569455067 conn(0x7f734c09cd00 msgr2=0x7f734c0d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:01.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.451+0000 7f7352d09640 1 -- 172.21.15.43:0/1569455067 shutdown_connections 2024-04-25T09:04:01.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.451+0000 7f7352d09640 1 -- 172.21.15.43:0/1569455067 wait complete. 2024-04-25T09:04:01.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.452+0000 7f7352d09640 1 Processor -- start 2024-04-25T09:04:01.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.452+0000 7f7352d09640 1 -- start start 2024-04-25T09:04:01.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.453+0000 7f7352d09640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f734c0ce550 0x7f734c14f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:01.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.453+0000 7f7352d09640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f734c0d0af0 0x7f734c0b0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:01.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.453+0000 7f7352d09640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f734c15dce0 0x7f734c0ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:01.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.453+0000 7f7352d09640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734c15e0c0 con 0x7f734c0ce550 2024-04-25T09:04:01.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.453+0000 7f7352d09640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734c15f6b0 con 0x7f734c0d0af0 2024-04-25T09:04:01.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.453+0000 7f7352d09640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f734c0ce9c0 con 0x7f734c15dce0 2024-04-25T09:04:01.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.453+0000 7f7350aa5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f734c0ce550 0x7f734c14f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:01.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.453+0000 7f7350aa5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f734c0ce550 0x7f734c14f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41380/0 (socket says 172.21.15.43:41380) 2024-04-25T09:04:01.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.453+0000 7f7350aa5640 1 -- 172.21.15.43:0/1606947580 learned_addr learned my addr 172.21.15.43:0/1606947580 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:01.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.453+0000 7f73512a6640 1 --2- 172.21.15.43:0/1606947580 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f734c15dce0 0x7f734c0ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:01.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.453+0000 7f734bfff640 1 --2- 172.21.15.43:0/1606947580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f734c0d0af0 0x7f734c0b0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:01.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.454+0000 7f7350aa5640 1 -- 172.21.15.43:0/1606947580 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f734c15dce0 msgr2=0x7f734c0ccc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:01.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.454+0000 7f7350aa5640 1 --2- 172.21.15.43:0/1606947580 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f734c15dce0 0x7f734c0ccc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.454+0000 7f7350aa5640 1 -- 172.21.15.43:0/1606947580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f734c0d0af0 msgr2=0x7f734c0b0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:01.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.454+0000 7f7350aa5640 1 --2- 172.21.15.43:0/1606947580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f734c0d0af0 0x7f734c0b0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.454+0000 7f7350aa5640 1 -- 172.21.15.43:0/1606947580 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f734c1520a0 con 0x7f734c0ce550 2024-04-25T09:04:01.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.454+0000 7f73512a6640 1 --2- 172.21.15.43:0/1606947580 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f734c15dce0 0x7f734c0ccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:01.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.454+0000 7f7350aa5640 1 --2- 172.21.15.43:0/1606947580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f734c0ce550 0x7f734c14f9e0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f734000e9b0 tx=0x7f734000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:01.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.454+0000 7f73497fa640 1 -- 172.21.15.43:0/1606947580 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7340017070 con 0x7f734c0ce550 2024-04-25T09:04:01.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.454+0000 7f73497fa640 1 -- 172.21.15.43:0/1606947580 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f734000c9a0 con 0x7f734c0ce550 2024-04-25T09:04:01.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.454+0000 7f7352d09640 1 -- 172.21.15.43:0/1606947580 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f734c152350 con 0x7f734c0ce550 2024-04-25T09:04:01.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.454+0000 7f73497fa640 1 -- 172.21.15.43:0/1606947580 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734000cd90 con 0x7f734c0ce550 2024-04-25T09:04:01.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.454+0000 7f7352d09640 1 -- 172.21.15.43:0/1606947580 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f734c152910 con 0x7f734c0ce550 2024-04-25T09:04:01.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.455+0000 7f7352d09640 1 -- 172.21.15.43:0/1606947580 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f734c0680f0 con 0x7f734c0ce550 2024-04-25T09:04:01.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.455+0000 7f73497fa640 1 -- 172.21.15.43:0/1606947580 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f734001a070 con 0x7f734c0ce550 2024-04-25T09:04:01.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.456+0000 7f73497fa640 1 --2- 172.21.15.43:0/1606947580 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f733006b4d0 0x7f733006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:01.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.456+0000 7f73497fa640 1 -- 172.21.15.43:0/1606947580 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f734008a300 con 0x7f734c0ce550 2024-04-25T09:04:01.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.456+0000 7f734bfff640 1 --2- 172.21.15.43:0/1606947580 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f733006b4d0 0x7f733006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:01.446 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.457+0000 7f734bfff640 1 --2- 172.21.15.43:0/1606947580 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f733006b4d0 0x7f733006d990 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f7338007fd0 tx=0x7f7338007c40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:01.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.460+0000 7f73497fa640 1 -- 172.21.15.43:0/1606947580 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7340059070 con 0x7f734c0ce550 2024-04-25T09:04:01.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.643+0000 7f7352d09640 1 -- 172.21.15.43:0/1606947580 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f734c074ef0 con 0x7f734c0ce550 2024-04-25T09:04:01.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.644+0000 7f73497fa640 1 -- 172.21.15.43:0/1606947580 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) v1 ==== 74+0+16814 (secure 0 0 0) 0x7f7340058970 con 0x7f734c0ce550 2024-04-25T09:04:01.633 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:04:01.633 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":129,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:03:55.660750+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:03:10.199822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T09:03:10.892664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.31","mappings":[{"from":2,"to":0}]},{"pgid":"19.3c","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6834/2603892772":"2024-04-26T09:03:52.443908+0000","172.21.15.43:6832/2603892772":"2024-04-26T09:03:52.443908+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:04:01.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.647+0000 7f7352d09640 1 -- 172.21.15.43:0/1606947580 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f733006b4d0 msgr2=0x7f733006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:01.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.647+0000 7f7352d09640 1 --2- 172.21.15.43:0/1606947580 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f733006b4d0 0x7f733006d990 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f7338007fd0 tx=0x7f7338007c40 comp rx=0 tx=0).stop 2024-04-25T09:04:01.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.647+0000 7f7352d09640 1 -- 172.21.15.43:0/1606947580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f734c0ce550 msgr2=0x7f734c14f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:01.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.647+0000 7f7352d09640 1 --2- 172.21.15.43:0/1606947580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f734c0ce550 0x7f734c14f9e0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f734000e9b0 tx=0x7f734000ee70 comp rx=0 tx=0).stop 2024-04-25T09:04:01.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.647+0000 7f7352d09640 1 -- 172.21.15.43:0/1606947580 shutdown_connections 2024-04-25T09:04:01.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.647+0000 7f7352d09640 1 --2- 172.21.15.43:0/1606947580 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f734c15dce0 0x7f734c0ccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.647+0000 7f7352d09640 1 --2- 172.21.15.43:0/1606947580 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f733006b4d0 0x7f733006d990 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.647+0000 7f7352d09640 1 --2- 172.21.15.43:0/1606947580 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f734c0d0af0 0x7f734c0b0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.647+0000 7f7352d09640 1 --2- 172.21.15.43:0/1606947580 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f734c0ce550 0x7f734c14f9e0 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.647+0000 7f7352d09640 1 -- 172.21.15.43:0/1606947580 >> 172.21.15.43:0/1606947580 conn(0x7f734c09cd00 msgr2=0x7f734c0cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:01.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.647+0000 7f7352d09640 1 -- 172.21.15.43:0/1606947580 shutdown_connections 2024-04-25T09:04:01.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.647+0000 7f7352d09640 1 -- 172.21.15.43:0/1606947580 wait complete. 2024-04-25T09:04:01.647 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:04:01.647 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:04:01.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.759+0000 7ff68d509640 1 Processor -- start 2024-04-25T09:04:01.748 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.759+0000 7ff68d509640 1 -- start start 2024-04-25T09:04:01.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.760+0000 7ff68d509640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff6880d39d0 0x7ff6880d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:01.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.760+0000 7ff68d509640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff6880d2500 0x7ff6880d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:01.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.760+0000 7ff68d509640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6880b6c50 con 0x7ff6880ce2a0 2024-04-25T09:04:01.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.760+0000 7ff68d509640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff68815e010 con 0x7ff6880d2500 2024-04-25T09:04:01.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.760+0000 7ff68d509640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6880d2020 con 0x7ff6880d39d0 2024-04-25T09:04:01.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.761+0000 7ff686ffd640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff6880d39d0 0x7ff6880d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:01.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.761+0000 7ff6877fe640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff6880d2500 0x7ff6880d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:01.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.761+0000 7ff6867fc640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7ff6880ce2a0 0x7ff6880d3db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:35238/0 (socket says 172.21.15.43:35238) 2024-04-25T09:04:01.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.761+0000 7ff6867fc640 1 -- 172.21.15.43:0/2859734046 learned_addr learned my addr 172.21.15.43:0/2859734046 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:01.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.761+0000 7ff686ffd640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff6880d39d0 0x7ff6880d0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:48598/0 (socket says 172.21.15.43:48598) 2024-04-25T09:04:01.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.761+0000 7ff686ffd640 1 -- 172.21.15.43:0/2859734046 >> v2:172.21.15.193:3300/0 conn(0x7ff6880d2500 msgr2=0x7ff6880d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:01.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.761+0000 7ff686ffd640 1 --2- 172.21.15.43:0/2859734046 >> v2:172.21.15.193:3300/0 conn(0x7ff6880d2500 0x7ff6880d2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.761+0000 7ff686ffd640 1 -- 172.21.15.43:0/2859734046 >> v1:172.21.15.193:6789/0 conn(0x7ff6880ce2a0 legacy=0x7ff6880d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:01.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.761+0000 7ff686ffd640 1 -- 172.21.15.43:0/2859734046 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6880cec40 con 0x7ff6880d39d0 2024-04-25T09:04:01.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.761+0000 7ff6877fe640 1 --2- 172.21.15.43:0/2859734046 >> v2:172.21.15.193:3300/0 conn(0x7ff6880d2500 0x7ff6880d2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:04:01.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.762+0000 7ff686ffd640 1 --2- 172.21.15.43:0/2859734046 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff6880d39d0 0x7ff6880d0d90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff6800097a0 tx=0x7ff68002a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=728a76aa76c14a85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:01.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.762+0000 7ff685ffb640 1 -- 172.21.15.43:0/2859734046 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff680037070 con 0x7ff6880d39d0 2024-04-25T09:04:01.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.762+0000 7ff685ffb640 1 -- 172.21.15.43:0/2859734046 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff68002ae50 con 0x7ff6880d39d0 2024-04-25T09:04:01.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.762+0000 7ff685ffb640 1 -- 172.21.15.43:0/2859734046 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff680032660 con 0x7ff6880d39d0 2024-04-25T09:04:01.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.762+0000 7ff68d509640 1 -- 172.21.15.43:0/2859734046 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff6880d39d0 msgr2=0x7ff6880d0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:01.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.762+0000 7ff68d509640 1 --2- 172.21.15.43:0/2859734046 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff6880d39d0 0x7ff6880d0d90 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff6800097a0 tx=0x7ff68002a100 comp rx=0 tx=0).stop 2024-04-25T09:04:01.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.762+0000 7ff68d509640 1 -- 172.21.15.43:0/2859734046 shutdown_connections 2024-04-25T09:04:01.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.762+0000 7ff68d509640 1 --2- 172.21.15.43:0/2859734046 >> v2:172.21.15.193:3300/0 conn(0x7ff6880d2500 0x7ff6880d2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.762+0000 7ff68d509640 1 --2- 172.21.15.43:0/2859734046 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff6880d39d0 0x7ff6880d0d90 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.762+0000 7ff68d509640 1 -- 172.21.15.43:0/2859734046 >> 172.21.15.43:0/2859734046 conn(0x7ff68806b000 msgr2=0x7ff68806b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:01.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.763+0000 7ff68d509640 1 -- 172.21.15.43:0/2859734046 shutdown_connections 2024-04-25T09:04:01.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.763+0000 7ff68d509640 1 -- 172.21.15.43:0/2859734046 wait complete. 2024-04-25T09:04:01.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.764+0000 7ff68d509640 1 Processor -- start 2024-04-25T09:04:01.753 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.764+0000 7ff68d509640 1 -- start start 2024-04-25T09:04:01.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.765+0000 7ff68d509640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff6880ce2a0 0x7ff68814ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:01.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.765+0000 7ff68d509640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff6880d2500 0x7ff6880cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:01.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.765+0000 7ff68d509640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6880d39d0 0x7ff6880cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:01.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.765+0000 7ff68d509640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6880cf200 con 0x7ff6880d39d0 2024-04-25T09:04:01.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.765+0000 7ff68d509640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6880b2f00 con 0x7ff6880d2500 2024-04-25T09:04:01.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.765+0000 7ff68d509640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6880d2ed0 con 0x7ff6880ce2a0 2024-04-25T09:04:01.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.765+0000 7ff6877fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6880d39d0 0x7ff6880cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:01.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.765+0000 7ff6877fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6880d39d0 0x7ff6880cc0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41394/0 (socket says 172.21.15.43:41394) 2024-04-25T09:04:01.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.765+0000 7ff6877fe640 1 -- 172.21.15.43:0/4097994577 learned_addr learned my addr 172.21.15.43:0/4097994577 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:01.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.765+0000 7ff6867fc640 1 --2- 172.21.15.43:0/4097994577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff6880d2500 0x7ff6880cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:01.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.765+0000 7ff686ffd640 1 --2- 172.21.15.43:0/4097994577 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff6880ce2a0 0x7ff68814ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:01.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.765+0000 7ff6877fe640 1 -- 172.21.15.43:0/4097994577 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff6880ce2a0 msgr2=0x7ff68814ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:01.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.765+0000 7ff6877fe640 1 --2- 172.21.15.43:0/4097994577 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff6880ce2a0 0x7ff68814ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.765+0000 7ff6877fe640 1 -- 172.21.15.43:0/4097994577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff6880d2500 msgr2=0x7ff6880cbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:01.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.765+0000 7ff6877fe640 1 --2- 172.21.15.43:0/4097994577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff6880d2500 0x7ff6880cbb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.765+0000 7ff6877fe640 1 -- 172.21.15.43:0/4097994577 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6880b3840 con 0x7ff6880d39d0 2024-04-25T09:04:01.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.765+0000 7ff6877fe640 1 --2- 172.21.15.43:0/4097994577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6880d39d0 0x7ff6880cc0a0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7ff678009f00 tx=0x7ff67800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:01.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.765+0000 7ff66f7fe640 1 -- 172.21.15.43:0/4097994577 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff678018030 con 0x7ff6880d39d0 2024-04-25T09:04:01.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.765+0000 7ff66f7fe640 1 -- 172.21.15.43:0/4097994577 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff67800ce50 con 0x7ff6880d39d0 2024-04-25T09:04:01.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.765+0000 7ff66f7fe640 1 -- 172.21.15.43:0/4097994577 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff678003bb0 con 0x7ff6880d39d0 2024-04-25T09:04:01.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.765+0000 7ff68d509640 1 -- 172.21.15.43:0/4097994577 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6880b3b20 con 0x7ff6880d39d0 2024-04-25T09:04:01.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.766+0000 7ff68d509640 1 -- 172.21.15.43:0/4097994577 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6880b40e0 con 0x7ff6880d39d0 2024-04-25T09:04:01.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.766+0000 7ff68d509640 1 -- 172.21.15.43:0/4097994577 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff688074ef0 con 0x7ff6880d39d0 2024-04-25T09:04:01.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.767+0000 7ff66f7fe640 1 -- 172.21.15.43:0/4097994577 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff67801b070 con 0x7ff6880d39d0 2024-04-25T09:04:01.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.767+0000 7ff66f7fe640 1 --2- 172.21.15.43:0/4097994577 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff67406b4d0 0x7ff67406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:01.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.767+0000 7ff66f7fe640 1 -- 172.21.15.43:0/4097994577 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5853+0+0 (secure 0 0 0) 0x7ff67808ac80 con 0x7ff6880d39d0 2024-04-25T09:04:01.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.767+0000 7ff686ffd640 1 --2- 172.21.15.43:0/4097994577 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff67406b4d0 0x7ff67406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:01.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.768+0000 7ff686ffd640 1 --2- 172.21.15.43:0/4097994577 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff67406b4d0 0x7ff67406d990 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7ff680034000 tx=0x7ff68002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:01.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.771+0000 7ff66f7fe640 1 -- 172.21.15.43:0/4097994577 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff678010040 con 0x7ff6880d39d0 2024-04-25T09:04:01.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.987+0000 7ff68d509640 1 -- 172.21.15.43:0/4097994577 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6880b42b0 con 0x7ff6880d39d0 2024-04-25T09:04:01.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.988+0000 7ff66f7fe640 1 -- 172.21.15.43:0/4097994577 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4155 (secure 0 0 0) 0x7ff678059380 con 0x7ff6880d39d0 2024-04-25T09:04:01.977 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:04:01.977 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":80,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/891225068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":891225068},{"type":"v1","addr":"172.21.15.193:6836","nonce":891225068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":7175,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3679059390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":3679059390},{"type":"v1","addr":"172.21.15.193:6837","nonce":3679059390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:11.919337+0000","modified":"2024-04-25T09:03:54.462418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7948},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7948":{"gid":7948,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":15,"addr":"172.21.15.43:6835/2127363183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2127363183},{"type":"v1","addr":"172.21.15.43:6835","nonce":2127363183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:04:01.977 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 80 2024-04-25T09:04:01.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.990+0000 7ff68d509640 1 -- 172.21.15.43:0/4097994577 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff67406b4d0 msgr2=0x7ff67406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:01.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.990+0000 7ff68d509640 1 --2- 172.21.15.43:0/4097994577 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff67406b4d0 0x7ff67406d990 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7ff680034000 tx=0x7ff68002e000 comp rx=0 tx=0).stop 2024-04-25T09:04:01.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.990+0000 7ff68d509640 1 -- 172.21.15.43:0/4097994577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6880d39d0 msgr2=0x7ff6880cc0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:01.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.990+0000 7ff68d509640 1 --2- 172.21.15.43:0/4097994577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6880d39d0 0x7ff6880cc0a0 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7ff678009f00 tx=0x7ff67800c490 comp rx=0 tx=0).stop 2024-04-25T09:04:01.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.990+0000 7ff68d509640 1 -- 172.21.15.43:0/4097994577 shutdown_connections 2024-04-25T09:04:01.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.990+0000 7ff68d509640 1 --2- 172.21.15.43:0/4097994577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff6880d39d0 0x7ff6880cc0a0 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.990+0000 7ff68d509640 1 --2- 172.21.15.43:0/4097994577 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff67406b4d0 0x7ff67406d990 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.990+0000 7ff68d509640 1 --2- 172.21.15.43:0/4097994577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff6880d2500 0x7ff6880cbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.990+0000 7ff68d509640 1 --2- 172.21.15.43:0/4097994577 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff6880ce2a0 0x7ff68814ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:01.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.990+0000 7ff68d509640 1 -- 172.21.15.43:0/4097994577 >> 172.21.15.43:0/4097994577 conn(0x7ff68806b000 msgr2=0x7ff6880d0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:01.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.990+0000 7ff68d509640 1 -- 172.21.15.43:0/4097994577 shutdown_connections 2024-04-25T09:04:01.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:01.990+0000 7ff68d509640 1 -- 172.21.15.43:0/4097994577 wait complete. 2024-04-25T09:04:01.990 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:04:02.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.102+0000 7f85b1d0a640 1 Processor -- start 2024-04-25T09:04:02.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.102+0000 7f85b1d0a640 1 -- start start 2024-04-25T09:04:02.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.103+0000 7f85b1d0a640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f85ac161390 0x7f85ac0b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:02.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.103+0000 7f85b1d0a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f85ac0d02a0 0x7f85ac0d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:02.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.103+0000 7f85b1d0a640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85ac0b4c30 con 0x7f85ac0ce5e0 2024-04-25T09:04:02.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.103+0000 7f85b1d0a640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85ac0d1910 con 0x7f85ac161390 2024-04-25T09:04:02.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.103+0000 7f85b1d0a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85ac0cf2c0 con 0x7f85ac0d02a0 2024-04-25T09:04:02.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.103+0000 7f85aaffd640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f85ac0d02a0 0x7f85ac0d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:02.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.103+0000 7f85ab7fe640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f85ac161390 0x7f85ac0b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:02.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.103+0000 7f85abfff640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f85ac0ce5e0 0x7f85ac0cea70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:35240/0 (socket says 172.21.15.43:35240) 2024-04-25T09:04:02.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.103+0000 7f85aaffd640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f85ac0d02a0 0x7f85ac0d0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:48628/0 (socket says 172.21.15.43:48628) 2024-04-25T09:04:02.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.103+0000 7f85abfff640 1 -- 172.21.15.43:0/1953526690 learned_addr learned my addr 172.21.15.43:0/1953526690 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:02.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.104+0000 7f85ab7fe640 1 -- 172.21.15.43:0/1953526690 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f85ac0d02a0 msgr2=0x7f85ac0d0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:02.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.104+0000 7f85ab7fe640 1 --2- 172.21.15.43:0/1953526690 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f85ac0d02a0 0x7f85ac0d0720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.104+0000 7f85ab7fe640 1 -- 172.21.15.43:0/1953526690 >> v1:172.21.15.193:6789/0 conn(0x7f85ac0ce5e0 legacy=0x7f85ac0cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:02.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.104+0000 7f85ab7fe640 1 -- 172.21.15.43:0/1953526690 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85ac0cf5a0 con 0x7f85ac161390 2024-04-25T09:04:02.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.104+0000 7f85aaffd640 1 --2- 172.21.15.43:0/1953526690 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f85ac0d02a0 0x7f85ac0d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:04:02.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.104+0000 7f85ab7fe640 1 --2- 172.21.15.43:0/1953526690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85ac161390 0x7f85ac0b6820 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f859c0097a0 tx=0x7f859c02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4daee6c6f4a7e0ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:02.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.105+0000 7f85aa7fc640 1 -- 172.21.15.43:0/1953526690 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859c037070 con 0x7f85ac161390 2024-04-25T09:04:02.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.105+0000 7f85aa7fc640 1 -- 172.21.15.43:0/1953526690 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f859c032440 con 0x7f85ac161390 2024-04-25T09:04:02.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.105+0000 7f85aa7fc640 1 -- 172.21.15.43:0/1953526690 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859c032810 con 0x7f85ac161390 2024-04-25T09:04:02.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.105+0000 7f85b1d0a640 1 -- 172.21.15.43:0/1953526690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85ac161390 msgr2=0x7f85ac0b6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:02.094 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.105+0000 7f85b1d0a640 1 --2- 172.21.15.43:0/1953526690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85ac161390 0x7f85ac0b6820 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f859c0097a0 tx=0x7f859c02a180 comp rx=0 tx=0).stop 2024-04-25T09:04:02.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.105+0000 7f85b1d0a640 1 -- 172.21.15.43:0/1953526690 shutdown_connections 2024-04-25T09:04:02.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.105+0000 7f85b1d0a640 1 --2- 172.21.15.43:0/1953526690 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f85ac0d02a0 0x7f85ac0d0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.105+0000 7f85b1d0a640 1 --2- 172.21.15.43:0/1953526690 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85ac161390 0x7f85ac0b6820 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.105+0000 7f85b1d0a640 1 -- 172.21.15.43:0/1953526690 >> 172.21.15.43:0/1953526690 conn(0x7f85ac09cd20 msgr2=0x7f85ac0d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:02.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.105+0000 7f85b1d0a640 1 -- 172.21.15.43:0/1953526690 shutdown_connections 2024-04-25T09:04:02.095 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.105+0000 7f85b1d0a640 1 -- 172.21.15.43:0/1953526690 wait complete. 2024-04-25T09:04:02.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.106+0000 7f85b1d0a640 1 Processor -- start 2024-04-25T09:04:02.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.107+0000 7f85b1d0a640 1 -- start start 2024-04-25T09:04:02.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.107+0000 7f85b1d0a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85ac0ce5e0 0x7f85ac0c4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:02.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.107+0000 7f85b1d0a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f85ac0d02a0 0x7f85ac1f6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:02.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.107+0000 7f85b1d0a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85ac161390 0x7f85ac13b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:02.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.107+0000 7f85b1d0a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ac161770 con 0x7f85ac161390 2024-04-25T09:04:02.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.107+0000 7f85b1d0a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85ac0d15a0 con 0x7f85ac0ce5e0 2024-04-25T09:04:02.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.107+0000 7f85b1d0a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85ac0d1da0 con 0x7f85ac0d02a0 2024-04-25T09:04:02.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.107+0000 7f85abfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85ac161390 0x7f85ac13b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:02.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.107+0000 7f85abfff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85ac161390 0x7f85ac13b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41408/0 (socket says 172.21.15.43:41408) 2024-04-25T09:04:02.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.107+0000 7f85abfff640 1 -- 172.21.15.43:0/3704000288 learned_addr learned my addr 172.21.15.43:0/3704000288 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:02.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.108+0000 7f85ab7fe640 1 --2- 172.21.15.43:0/3704000288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85ac0ce5e0 0x7f85ac0c4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:02.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.108+0000 7f85aaffd640 1 --2- 172.21.15.43:0/3704000288 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f85ac0d02a0 0x7f85ac1f6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:02.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.108+0000 7f85abfff640 1 -- 172.21.15.43:0/3704000288 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f85ac0d02a0 msgr2=0x7f85ac1f6e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:02.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.108+0000 7f85abfff640 1 --2- 172.21.15.43:0/3704000288 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f85ac0d02a0 0x7f85ac1f6e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.108+0000 7f85abfff640 1 -- 172.21.15.43:0/3704000288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85ac0ce5e0 msgr2=0x7f85ac0c4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:02.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.108+0000 7f85abfff640 1 --2- 172.21.15.43:0/3704000288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85ac0ce5e0 0x7f85ac0c4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.108+0000 7f85abfff640 1 -- 172.21.15.43:0/3704000288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85ac1f7360 con 0x7f85ac161390 2024-04-25T09:04:02.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.108+0000 7f85aaffd640 1 --2- 172.21.15.43:0/3704000288 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f85ac0d02a0 0x7f85ac1f6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:02.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.108+0000 7f85abfff640 1 --2- 172.21.15.43:0/3704000288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85ac161390 0x7f85ac13b400 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f85a400caa0 tx=0x7f85a400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:02.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.108+0000 7f8593fff640 1 -- 172.21.15.43:0/3704000288 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a4017070 con 0x7f85ac161390 2024-04-25T09:04:02.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.108+0000 7f8593fff640 1 -- 172.21.15.43:0/3704000288 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a400ac00 con 0x7f85ac161390 2024-04-25T09:04:02.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.108+0000 7f85b1d0a640 1 -- 172.21.15.43:0/3704000288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85ac15b940 con 0x7f85ac161390 2024-04-25T09:04:02.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.108+0000 7f8593fff640 1 -- 172.21.15.43:0/3704000288 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a4012620 con 0x7f85ac161390 2024-04-25T09:04:02.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.108+0000 7f85b1d0a640 1 -- 172.21.15.43:0/3704000288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85ac15bf00 con 0x7f85ac161390 2024-04-25T09:04:02.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.109+0000 7f85b1d0a640 1 -- 172.21.15.43:0/3704000288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85ac0680f0 con 0x7f85ac161390 2024-04-25T09:04:02.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.110+0000 7f8593fff640 1 -- 172.21.15.43:0/3704000288 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f85a401a070 con 0x7f85ac161390 2024-04-25T09:04:02.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.110+0000 7f8593fff640 1 --2- 172.21.15.43:0/3704000288 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f859806b4d0 0x7f859806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:02.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.110+0000 7f8593fff640 1 -- 172.21.15.43:0/3704000288 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f85a4089f10 con 0x7f85ac161390 2024-04-25T09:04:02.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.110+0000 7f85ab7fe640 1 --2- 172.21.15.43:0/3704000288 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f859806b4d0 0x7f859806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:02.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.111+0000 7f85ab7fe640 1 --2- 172.21.15.43:0/3704000288 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f859806b4d0 0x7f859806d990 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f859c034000 tx=0x7f859c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:02.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.114+0000 7f8593fff640 1 -- 172.21.15.43:0/3704000288 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f85ac0b4c30 con 0x7f85ac161390 2024-04-25T09:04:02.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.332+0000 7f85b1d0a640 1 -- 172.21.15.43:0/3704000288 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f85ac0cfd20 con 0x7f85ac161390 2024-04-25T09:04:02.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.333+0000 7f8593fff640 1 -- 172.21.15.43:0/3704000288 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) v1 ==== 94+0+4155 (secure 0 0 0) 0x7f85ac0cfd20 con 0x7f85ac161390 2024-04-25T09:04:02.321 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:04:02.321 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":80,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/891225068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":891225068},{"type":"v1","addr":"172.21.15.193:6836","nonce":891225068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":7175,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3679059390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":3679059390},{"type":"v1","addr":"172.21.15.193:6837","nonce":3679059390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:03:11.919337+0000","modified":"2024-04-25T09:03:54.462418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":127,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7948},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7948":{"gid":7948,"name":"c","rank":0,"incarnation":77,"state":"up:active","state_seq":15,"addr":"172.21.15.43:6835/2127363183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6833","nonce":2127363183},{"type":"v1","addr":"172.21.15.43:6835","nonce":2127363183}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T09:04:02.322 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 80 2024-04-25T09:04:02.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.334+0000 7f85b1d0a640 1 -- 172.21.15.43:0/3704000288 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f859806b4d0 msgr2=0x7f859806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:02.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.334+0000 7f85b1d0a640 1 --2- 172.21.15.43:0/3704000288 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f859806b4d0 0x7f859806d990 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f859c034000 tx=0x7f859c02e000 comp rx=0 tx=0).stop 2024-04-25T09:04:02.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.334+0000 7f85b1d0a640 1 -- 172.21.15.43:0/3704000288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85ac161390 msgr2=0x7f85ac13b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:02.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.334+0000 7f85b1d0a640 1 --2- 172.21.15.43:0/3704000288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85ac161390 0x7f85ac13b400 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f85a400caa0 tx=0x7f85a400cf60 comp rx=0 tx=0).stop 2024-04-25T09:04:02.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.334+0000 7f85b1d0a640 1 -- 172.21.15.43:0/3704000288 shutdown_connections 2024-04-25T09:04:02.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.335+0000 7f85b1d0a640 1 --2- 172.21.15.43:0/3704000288 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f85ac161390 0x7f85ac13b400 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.335+0000 7f85b1d0a640 1 --2- 172.21.15.43:0/3704000288 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f85ac0d02a0 0x7f85ac1f6e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.335+0000 7f85b1d0a640 1 --2- 172.21.15.43:0/3704000288 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f859806b4d0 0x7f859806d990 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.335+0000 7f85b1d0a640 1 --2- 172.21.15.43:0/3704000288 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f85ac0ce5e0 0x7f85ac0c4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.335+0000 7f85b1d0a640 1 -- 172.21.15.43:0/3704000288 >> 172.21.15.43:0/3704000288 conn(0x7f85ac09cd20 msgr2=0x7f85ac148960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:02.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.335+0000 7f85b1d0a640 1 -- 172.21.15.43:0/3704000288 shutdown_connections 2024-04-25T09:04:02.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.335+0000 7f85b1d0a640 1 -- 172.21.15.43:0/3704000288 wait complete. 2024-04-25T09:04:02.335 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:04:02.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.447+0000 7fc7ed05a640 1 Processor -- start 2024-04-25T09:04:02.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.447+0000 7fc7ed05a640 1 -- start start 2024-04-25T09:04:02.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.448+0000 7fc7ed05a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7e80c8d00 0x7fc7e8162100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:02.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.448+0000 7fc7ed05a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7e80d24a0 con 0x7fc7e815df50 2024-04-25T09:04:02.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.448+0000 7fc7ed05a640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7e80d2740 con 0x7fc7e80ce660 2024-04-25T09:04:02.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.448+0000 7fc7ed05a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e815e330 con 0x7fc7e80c8d00 2024-04-25T09:04:02.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.448+0000 7fc7e6d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc7e815df50 0x7fc7e81619f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56704/0 (socket says 172.21.15.43:56704) 2024-04-25T09:04:02.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.448+0000 7fc7e6d9d640 1 -- 172.21.15.43:0/462108540 learned_addr learned my addr 172.21.15.43:0/462108540 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:02.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.448+0000 7fc7e659c640 1 --2- 172.21.15.43:0/462108540 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7e80c8d00 0x7fc7e8162100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:02.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.448+0000 7fc7e5d9b640 1 -- 172.21.15.43:0/462108540 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 288005387 0 0) 0x7fc7e80d24a0 con 0x7fc7e815df50 2024-04-25T09:04:02.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.448+0000 7fc7e5d9b640 1 -- 172.21.15.43:0/462108540 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7d0003520 con 0x7fc7e815df50 2024-04-25T09:04:02.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.448+0000 7fc7e5d9b640 1 -- 172.21.15.43:0/462108540 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2107608110 0 0) 0x7fc7e80d2740 con 0x7fc7e80ce660 2024-04-25T09:04:02.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.448+0000 7fc7e5d9b640 1 -- 172.21.15.43:0/462108540 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7e80d24a0 con 0x7fc7e80ce660 2024-04-25T09:04:02.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.448+0000 7fc7e659c640 1 -- 172.21.15.43:0/462108540 >> v1:172.21.15.193:6789/0 conn(0x7fc7e80ce660 legacy=0x7fc7e80cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:02.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.448+0000 7fc7e659c640 1 -- 172.21.15.43:0/462108540 >> v1:172.21.15.43:6789/0 conn(0x7fc7e815df50 legacy=0x7fc7e81619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:02.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.448+0000 7fc7e5d9b640 1 -- 172.21.15.43:0/462108540 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 959654198 0 0) 0x7fc7d0003520 con 0x7fc7e815df50 2024-04-25T09:04:02.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.448+0000 7fc7e659c640 1 -- 172.21.15.43:0/462108540 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e80d2a20 con 0x7fc7e80c8d00 2024-04-25T09:04:02.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.449+0000 7fc7e659c640 1 --2- 172.21.15.43:0/462108540 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7e80c8d00 0x7fc7e8162100 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc0097d0 tx=0x7fc7dc02a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3815f97df5065540 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:02.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.449+0000 7fc7e5d9b640 1 -- 172.21.15.43:0/462108540 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7dc037070 con 0x7fc7e80c8d00 2024-04-25T09:04:02.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.449+0000 7fc7e5d9b640 1 -- 172.21.15.43:0/462108540 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7dc02ac30 con 0x7fc7e80c8d00 2024-04-25T09:04:02.439 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.449+0000 7fc7e5d9b640 1 -- 172.21.15.43:0/462108540 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d80019f0 con 0x7fc7e80c8d00 2024-04-25T09:04:02.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.450+0000 7fc7ed05a640 1 -- 172.21.15.43:0/462108540 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7e80c8d00 msgr2=0x7fc7e8162100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:02.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.450+0000 7fc7ed05a640 1 --2- 172.21.15.43:0/462108540 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7e80c8d00 0x7fc7e8162100 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc0097d0 tx=0x7fc7dc02a1a0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.450+0000 7fc7ed05a640 1 -- 172.21.15.43:0/462108540 shutdown_connections 2024-04-25T09:04:02.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.450+0000 7fc7ed05a640 1 --2- 172.21.15.43:0/462108540 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7e80c8d00 0x7fc7e8162100 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.450+0000 7fc7ed05a640 1 -- 172.21.15.43:0/462108540 >> 172.21.15.43:0/462108540 conn(0x7fc7e80d3a30 msgr2=0x7fc7e80d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:02.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.450+0000 7fc7ed05a640 1 -- 172.21.15.43:0/462108540 shutdown_connections 2024-04-25T09:04:02.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.450+0000 7fc7ed05a640 1 -- 172.21.15.43:0/462108540 wait complete. 2024-04-25T09:04:02.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.451+0000 7fc7ed05a640 1 Processor -- start 2024-04-25T09:04:02.440 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.451+0000 7fc7ed05a640 1 -- start start 2024-04-25T09:04:02.442 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.452+0000 7fc7ed05a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7e80c8d00 0x7fc7e814fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:02.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.452+0000 7fc7ed05a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7e80ce660 0x7fc7e81f7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:02.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.452+0000 7fc7ed05a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7e815df50 0x7fc7e81f7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:02.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.452+0000 7fc7ed05a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e80d2190 con 0x7fc7e80c8d00 2024-04-25T09:04:02.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.452+0000 7fc7ed05a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e80c77d0 con 0x7fc7e80ce660 2024-04-25T09:04:02.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.452+0000 7fc7ed05a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e8162640 con 0x7fc7e815df50 2024-04-25T09:04:02.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.452+0000 7fc7e6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7e80c8d00 0x7fc7e814fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:02.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.452+0000 7fc7e6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7e80c8d00 0x7fc7e814fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41418/0 (socket says 172.21.15.43:41418) 2024-04-25T09:04:02.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.452+0000 7fc7e6d9d640 1 -- 172.21.15.43:0/675162023 learned_addr learned my addr 172.21.15.43:0/675162023 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:02.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.452+0000 7fc7e659c640 1 --2- 172.21.15.43:0/675162023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7e80ce660 0x7fc7e81f7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:02.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.452+0000 7fc7e6d9d640 1 -- 172.21.15.43:0/675162023 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7e815df50 msgr2=0x7fc7e81f7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:02.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.452+0000 7fc7e759e640 1 --2- 172.21.15.43:0/675162023 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7e815df50 0x7fc7e81f7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:02.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.452+0000 7fc7e6d9d640 1 --2- 172.21.15.43:0/675162023 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7e815df50 0x7fc7e81f7d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.452+0000 7fc7e6d9d640 1 -- 172.21.15.43:0/675162023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7e80ce660 msgr2=0x7fc7e81f7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:02.443 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.452+0000 7fc7e6d9d640 1 --2- 172.21.15.43:0/675162023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7e80ce660 0x7fc7e81f7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.452+0000 7fc7e6d9d640 1 -- 172.21.15.43:0/675162023 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e815fcf0 con 0x7fc7e80c8d00 2024-04-25T09:04:02.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.452+0000 7fc7e659c640 1 --2- 172.21.15.43:0/675162023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7e80ce660 0x7fc7e81f7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:02.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.452+0000 7fc7e759e640 1 --2- 172.21.15.43:0/675162023 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7e815df50 0x7fc7e81f7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:02.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.453+0000 7fc7e6d9d640 1 --2- 172.21.15.43:0/675162023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7e80c8d00 0x7fc7e814fcd0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7fc7d800ccc0 tx=0x7fc7d800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:02.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.453+0000 7fc7cf7fe640 1 -- 172.21.15.43:0/675162023 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d8012400 con 0x7fc7e80c8d00 2024-04-25T09:04:02.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.453+0000 7fc7cf7fe640 1 -- 172.21.15.43:0/675162023 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7d80125a0 con 0x7fc7e80c8d00 2024-04-25T09:04:02.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.453+0000 7fc7ed05a640 1 -- 172.21.15.43:0/675162023 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7e81f8670 con 0x7fc7e80c8d00 2024-04-25T09:04:02.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.453+0000 7fc7cf7fe640 1 -- 172.21.15.43:0/675162023 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d8012990 con 0x7fc7e80c8d00 2024-04-25T09:04:02.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.453+0000 7fc7ed05a640 1 -- 172.21.15.43:0/675162023 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7e80cab10 con 0x7fc7e80c8d00 2024-04-25T09:04:02.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.454+0000 7fc7ed05a640 1 -- 172.21.15.43:0/675162023 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7e80680f0 con 0x7fc7e80c8d00 2024-04-25T09:04:02.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.454+0000 7fc7cf7fe640 1 -- 172.21.15.43:0/675162023 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc7d8012b30 con 0x7fc7e80c8d00 2024-04-25T09:04:02.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.455+0000 7fc7cf7fe640 1 --2- 172.21.15.43:0/675162023 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc7d406b4d0 0x7fc7d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:02.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.455+0000 7fc7cf7fe640 1 -- 172.21.15.43:0/675162023 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fc7d808aa90 con 0x7fc7e80c8d00 2024-04-25T09:04:02.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.455+0000 7fc7e659c640 1 --2- 172.21.15.43:0/675162023 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc7d406b4d0 0x7fc7d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:02.445 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.455+0000 7fc7e659c640 1 --2- 172.21.15.43:0/675162023 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc7d406b4d0 0x7fc7d406d990 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc034000 tx=0x7fc7dc02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:02.447 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.459+0000 7fc7cf7fe640 1 -- 172.21.15.43:0/675162023 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc7d80182b0 con 0x7fc7e80c8d00 2024-04-25T09:04:02.645 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.657+0000 7fc7ed05a640 1 -- 172.21.15.43:0/675162023 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7e80ca400 con 0x7fc7e80c8d00 2024-04-25T09:04:02.646 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.658+0000 7fc7cf7fe640 1 -- 172.21.15.43:0/675162023 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) v1 ==== 74+0+16814 (secure 0 0 0) 0x7fc7d8059940 con 0x7fc7e80c8d00 2024-04-25T09:04:02.647 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:04:02.647 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":129,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:03:55.660750+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:03:10.199822+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-04-25T09:03:10.892664+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"19.31","mappings":[{"from":2,"to":0}]},{"pgid":"19.3c","mappings":[{"from":2,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6834/2603892772":"2024-04-26T09:03:52.443908+0000","172.21.15.43:6832/2603892772":"2024-04-26T09:03:52.443908+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:04:02.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.660+0000 7fc7ed05a640 1 -- 172.21.15.43:0/675162023 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc7d406b4d0 msgr2=0x7fc7d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:02.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.660+0000 7fc7ed05a640 1 --2- 172.21.15.43:0/675162023 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc7d406b4d0 0x7fc7d406d990 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc034000 tx=0x7fc7dc02e000 comp rx=0 tx=0).stop 2024-04-25T09:04:02.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.660+0000 7fc7ed05a640 1 -- 172.21.15.43:0/675162023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7e80c8d00 msgr2=0x7fc7e814fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:02.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.660+0000 7fc7ed05a640 1 --2- 172.21.15.43:0/675162023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7e80c8d00 0x7fc7e814fcd0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7fc7d800ccc0 tx=0x7fc7d800a430 comp rx=0 tx=0).stop 2024-04-25T09:04:02.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.660+0000 7fc7ed05a640 1 -- 172.21.15.43:0/675162023 shutdown_connections 2024-04-25T09:04:02.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.660+0000 7fc7ed05a640 1 --2- 172.21.15.43:0/675162023 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7e815df50 0x7fc7e81f7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.660+0000 7fc7ed05a640 1 --2- 172.21.15.43:0/675162023 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc7d406b4d0 0x7fc7d406d990 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.660+0000 7fc7ed05a640 1 --2- 172.21.15.43:0/675162023 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7e80ce660 0x7fc7e81f7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.660+0000 7fc7ed05a640 1 --2- 172.21.15.43:0/675162023 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7e80c8d00 0x7fc7e814fcd0 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.660+0000 7fc7ed05a640 1 -- 172.21.15.43:0/675162023 >> 172.21.15.43:0/675162023 conn(0x7fc7e80d3a30 msgr2=0x7fc7e80cf2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:02.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.660+0000 7fc7ed05a640 1 -- 172.21.15.43:0/675162023 shutdown_connections 2024-04-25T09:04:02.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.660+0000 7fc7ed05a640 1 -- 172.21.15.43:0/675162023 wait complete. 2024-04-25T09:04:02.661 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:04:02.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.774+0000 7fae5d322640 1 Processor -- start 2024-04-25T09:04:02.763 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.774+0000 7fae5d322640 1 -- start start 2024-04-25T09:04:02.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.775+0000 7fae5d322640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fae5815df70 0x7fae580cc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:02.764 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.775+0000 7fae5d322640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fae580b0870 0x7fae580ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:02.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.775+0000 7fae5d322640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae580cbee0 con 0x7fae5806c510 2024-04-25T09:04:02.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.775+0000 7fae5d322640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae580d0400 con 0x7fae580b0870 2024-04-25T09:04:02.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.775+0000 7fae5d322640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae580d0650 con 0x7fae5815df70 2024-04-25T09:04:02.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.775+0000 7fae577fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fae580b0870 0x7fae580ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:02.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.775+0000 7fae577fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fae580b0870 0x7fae580ccbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41424/0 (socket says 172.21.15.43:41424) 2024-04-25T09:04:02.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.775+0000 7fae577fe640 1 -- 172.21.15.43:0/312203708 learned_addr learned my addr 172.21.15.43:0/312203708 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:02.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.776+0000 7fae567fc640 1 --2- 172.21.15.43:0/312203708 >> v2:172.21.15.193:3300/0 conn(0x7fae5815df70 0x7fae580cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:02.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.776+0000 7fae577fe640 1 -- 172.21.15.43:0/312203708 >> v2:172.21.15.193:3300/0 conn(0x7fae5815df70 msgr2=0x7fae580cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:02.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.776+0000 7fae577fe640 1 --2- 172.21.15.43:0/312203708 >> v2:172.21.15.193:3300/0 conn(0x7fae5815df70 0x7fae580cc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.776+0000 7fae577fe640 1 -- 172.21.15.43:0/312203708 >> v1:172.21.15.193:6789/0 conn(0x7fae5806c510 legacy=0x7fae5806c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:02.765 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.776+0000 7fae577fe640 1 -- 172.21.15.43:0/312203708 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae580d0930 con 0x7fae580b0870 2024-04-25T09:04:02.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.776+0000 7fae577fe640 1 --2- 172.21.15.43:0/312203708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae580b0870 0x7fae580ccbb0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7fae48009e50 tx=0x7fae4802a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd649c3fbb466ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:02.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.777+0000 7fae55ffb640 1 -- 172.21.15.43:0/312203708 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae48036070 con 0x7fae580b0870 2024-04-25T09:04:02.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.777+0000 7fae55ffb640 1 -- 172.21.15.43:0/312203708 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae48031600 con 0x7fae580b0870 2024-04-25T09:04:02.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.777+0000 7fae55ffb640 1 -- 172.21.15.43:0/312203708 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae480319d0 con 0x7fae580b0870 2024-04-25T09:04:02.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.777+0000 7fae5d322640 1 -- 172.21.15.43:0/312203708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae580b0870 msgr2=0x7fae580ccbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:02.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.777+0000 7fae5d322640 1 --2- 172.21.15.43:0/312203708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae580b0870 0x7fae580ccbb0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7fae48009e50 tx=0x7fae4802a6d0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.777+0000 7fae5d322640 1 -- 172.21.15.43:0/312203708 shutdown_connections 2024-04-25T09:04:02.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.777+0000 7fae5d322640 1 --2- 172.21.15.43:0/312203708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae580b0870 0x7fae580ccbb0 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.777+0000 7fae5d322640 1 --2- 172.21.15.43:0/312203708 >> v2:172.21.15.193:3300/0 conn(0x7fae5815df70 0x7fae580cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.767 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.777+0000 7fae5d322640 1 -- 172.21.15.43:0/312203708 >> 172.21.15.43:0/312203708 conn(0x7fae580d17f0 msgr2=0x7fae580d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:02.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.777+0000 7fae5d322640 1 -- 172.21.15.43:0/312203708 shutdown_connections 2024-04-25T09:04:02.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.777+0000 7fae5d322640 1 -- 172.21.15.43:0/312203708 wait complete. 2024-04-25T09:04:02.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.779+0000 7fae5d322640 1 Processor -- start 2024-04-25T09:04:02.768 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.779+0000 7fae5d322640 1 -- start start 2024-04-25T09:04:02.769 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.779+0000 7fae5d322640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae5806c510 0x7fae580cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:02.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.779+0000 7fae5d322640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae580b0870 0x7fae5806e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:02.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.779+0000 7fae5d322640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fae5815df70 0x7fae5815f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:02.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.779+0000 7fae5d322640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae5806bd10 con 0x7fae5806c510 2024-04-25T09:04:02.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.779+0000 7fae5d322640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae580d0c10 con 0x7fae580b0870 2024-04-25T09:04:02.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.779+0000 7fae5d322640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae5806be90 con 0x7fae5815df70 2024-04-25T09:04:02.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.779+0000 7fae56ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae5806c510 0x7fae580cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:02.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.779+0000 7fae56ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae5806c510 0x7fae580cb4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41440/0 (socket says 172.21.15.43:41440) 2024-04-25T09:04:02.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.780+0000 7fae56ffd640 1 -- 172.21.15.43:0/3330650262 learned_addr learned my addr 172.21.15.43:0/3330650262 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:02.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.780+0000 7fae567fc640 1 --2- 172.21.15.43:0/3330650262 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae580b0870 0x7fae5806e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:02.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.780+0000 7fae577fe640 1 --2- 172.21.15.43:0/3330650262 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fae5815df70 0x7fae5815f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:02.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.780+0000 7fae56ffd640 1 -- 172.21.15.43:0/3330650262 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fae5815df70 msgr2=0x7fae5815f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:02.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.780+0000 7fae56ffd640 1 --2- 172.21.15.43:0/3330650262 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fae5815df70 0x7fae5815f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.770 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.780+0000 7fae56ffd640 1 -- 172.21.15.43:0/3330650262 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae580b0870 msgr2=0x7fae5806e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:02.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.780+0000 7fae56ffd640 1 --2- 172.21.15.43:0/3330650262 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae580b0870 0x7fae5806e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.780+0000 7fae56ffd640 1 -- 172.21.15.43:0/3330650262 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae58155680 con 0x7fae5806c510 2024-04-25T09:04:02.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.780+0000 7fae56ffd640 1 --2- 172.21.15.43:0/3330650262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae5806c510 0x7fae580cb4f0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fae5000c400 tx=0x7fae5000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:02.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.780+0000 7fae3f7fe640 1 -- 172.21.15.43:0/3330650262 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae50017070 con 0x7fae5806c510 2024-04-25T09:04:02.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.780+0000 7fae3f7fe640 1 -- 172.21.15.43:0/3330650262 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae500052d0 con 0x7fae5806c510 2024-04-25T09:04:02.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.780+0000 7fae3f7fe640 1 -- 172.21.15.43:0/3330650262 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae50003830 con 0x7fae5806c510 2024-04-25T09:04:02.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.780+0000 7fae5d322640 1 -- 172.21.15.43:0/3330650262 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae58155960 con 0x7fae5806c510 2024-04-25T09:04:02.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.780+0000 7fae5d322640 1 -- 172.21.15.43:0/3330650262 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae581564f0 con 0x7fae5806c510 2024-04-25T09:04:02.771 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.781+0000 7fae5d322640 1 -- 172.21.15.43:0/3330650262 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae58074ef0 con 0x7fae5806c510 2024-04-25T09:04:02.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.782+0000 7fae3f7fe640 1 -- 172.21.15.43:0/3330650262 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae5001a070 con 0x7fae5806c510 2024-04-25T09:04:02.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.782+0000 7fae3f7fe640 1 --2- 172.21.15.43:0/3330650262 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fae4406b4d0 0x7fae4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:02.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.782+0000 7fae3f7fe640 1 -- 172.21.15.43:0/3330650262 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fae50089540 con 0x7fae5806c510 2024-04-25T09:04:02.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.782+0000 7fae567fc640 1 --2- 172.21.15.43:0/3330650262 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fae4406b4d0 0x7fae4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:02.773 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.783+0000 7fae567fc640 1 --2- 172.21.15.43:0/3330650262 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fae4406b4d0 0x7fae4406d990 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fae58156760 tx=0x7fae4c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:02.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:02.786+0000 7fae3f7fe640 1 -- 172.21.15.43:0/3330650262 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fae50057ce0 con 0x7fae5806c510 2024-04-25T09:04:02.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.002+0000 7fae5d322640 1 -- 172.21.15.43:0/3330650262 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fae5806fcd0 con 0x7fae5806c510 2024-04-25T09:04:02.990 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-04-25T09:04:03.002+0000 7fe4fb198640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:04:02.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.004+0000 7fae3f7fe640 1 -- 172.21.15.43:0/3330650262 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v81) v1 ==== 161+0+0 (secure 0 0 0) 0x7fae5005bd80 con 0x7fae5806c510 2024-04-25T09:04:02.992 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:04:02.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.006+0000 7fae5d322640 1 -- 172.21.15.43:0/3330650262 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fae4406b4d0 msgr2=0x7fae4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:02.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.006+0000 7fae5d322640 1 --2- 172.21.15.43:0/3330650262 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fae4406b4d0 0x7fae4406d990 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fae58156760 tx=0x7fae4c009190 comp rx=0 tx=0).stop 2024-04-25T09:04:02.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.006+0000 7fae5d322640 1 -- 172.21.15.43:0/3330650262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae5806c510 msgr2=0x7fae580cb4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:02.995 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.006+0000 7fae5d322640 1 --2- 172.21.15.43:0/3330650262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae5806c510 0x7fae580cb4f0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fae5000c400 tx=0x7fae5000c8c0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.006+0000 7fae5d322640 1 -- 172.21.15.43:0/3330650262 shutdown_connections 2024-04-25T09:04:02.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.006+0000 7fae5d322640 1 --2- 172.21.15.43:0/3330650262 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fae5815df70 0x7fae5815f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.006+0000 7fae5d322640 1 --2- 172.21.15.43:0/3330650262 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fae4406b4d0 0x7fae4406d990 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.006+0000 7fae5d322640 1 --2- 172.21.15.43:0/3330650262 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae580b0870 0x7fae5806e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.006+0000 7fae5d322640 1 --2- 172.21.15.43:0/3330650262 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae5806c510 0x7fae580cb4f0 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:02.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.006+0000 7fae5d322640 1 -- 172.21.15.43:0/3330650262 >> 172.21.15.43:0/3330650262 conn(0x7fae580d17f0 msgr2=0x7fae580cc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:02.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.006+0000 7fae5d322640 1 -- 172.21.15.43:0/3330650262 shutdown_connections 2024-04-25T09:04:02.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.006+0000 7fae5d322640 1 -- 172.21.15.43:0/3330650262 wait complete. 2024-04-25T09:04:03.007 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:04:03.032 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-04-25T09:04:03.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.124+0000 7feb6ba5d640 1 Processor -- start 2024-04-25T09:04:03.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.124+0000 7feb6ba5d640 1 -- start start 2024-04-25T09:04:03.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.125+0000 7feb6ba5d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7feb640d19f0 0x7feb640d1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:03.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.125+0000 7feb6ba5d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feb640d2370 0x7feb640c7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:03.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.125+0000 7feb6ba5d640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb640cf030 con 0x7feb64161310 2024-04-25T09:04:03.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.125+0000 7feb6ba5d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb640d4870 con 0x7feb640d19f0 2024-04-25T09:04:03.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.125+0000 7feb6ba5d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb640cf310 con 0x7feb640d2370 2024-04-25T09:04:03.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.125+0000 7feb68ff8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7feb640d19f0 0x7feb640d1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:03.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.125+0000 7feb68ff8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7feb640d19f0 0x7feb640d1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41472/0 (socket says 172.21.15.43:41472) 2024-04-25T09:04:03.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.125+0000 7feb68ff8640 1 -- 172.21.15.43:0/3738068817 learned_addr learned my addr 172.21.15.43:0/3738068817 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:03.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.125+0000 7feb69ffa640 1 --2- 172.21.15.43:0/3738068817 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feb640d2370 0x7feb640c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:03.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.125+0000 7feb68ff8640 1 -- 172.21.15.43:0/3738068817 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feb640d2370 msgr2=0x7feb640c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:03.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.125+0000 7feb68ff8640 1 --2- 172.21.15.43:0/3738068817 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feb640d2370 0x7feb640c7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:03.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.125+0000 7feb68ff8640 1 -- 172.21.15.43:0/3738068817 >> v1:172.21.15.193:6789/0 conn(0x7feb64161310 legacy=0x7feb640d4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:03.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.125+0000 7feb68ff8640 1 -- 172.21.15.43:0/3738068817 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb640cf5f0 con 0x7feb640d19f0 2024-04-25T09:04:03.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.125+0000 7feb5bfff640 1 -- 172.21.15.43:0/3738068817 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3854958409 0 0) 0x7feb640cf030 con 0x7feb64161310 2024-04-25T09:04:03.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.126+0000 7feb68ff8640 1 --2- 172.21.15.43:0/3738068817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb640d19f0 0x7feb640d1e30 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7feb54009870 tx=0x7feb5402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7310101ffae6c785 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:03.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.126+0000 7feb5bfff640 1 -- 172.21.15.43:0/3738068817 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb54037070 con 0x7feb640d19f0 2024-04-25T09:04:03.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.126+0000 7feb5bfff640 1 -- 172.21.15.43:0/3738068817 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb54032440 con 0x7feb640d19f0 2024-04-25T09:04:03.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.126+0000 7feb5bfff640 1 -- 172.21.15.43:0/3738068817 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb54032810 con 0x7feb640d19f0 2024-04-25T09:04:03.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.127+0000 7feb6ba5d640 1 -- 172.21.15.43:0/3738068817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb640d19f0 msgr2=0x7feb640d1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:03.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.127+0000 7feb6ba5d640 1 --2- 172.21.15.43:0/3738068817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb640d19f0 0x7feb640d1e30 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7feb54009870 tx=0x7feb5402a270 comp rx=0 tx=0).stop 2024-04-25T09:04:03.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.127+0000 7feb6ba5d640 1 -- 172.21.15.43:0/3738068817 shutdown_connections 2024-04-25T09:04:03.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.127+0000 7feb6ba5d640 1 --2- 172.21.15.43:0/3738068817 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feb640d2370 0x7feb640c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:03.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.127+0000 7feb6ba5d640 1 --2- 172.21.15.43:0/3738068817 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb640d19f0 0x7feb640d1e30 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:03.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.127+0000 7feb6ba5d640 1 -- 172.21.15.43:0/3738068817 >> 172.21.15.43:0/3738068817 conn(0x7feb6409cd20 msgr2=0x7feb640d3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:03.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.127+0000 7feb6ba5d640 1 -- 172.21.15.43:0/3738068817 shutdown_connections 2024-04-25T09:04:03.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.127+0000 7feb6ba5d640 1 -- 172.21.15.43:0/3738068817 wait complete. 2024-04-25T09:04:03.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.128+0000 7feb6ba5d640 1 Processor -- start 2024-04-25T09:04:03.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.128+0000 7feb6ba5d640 1 -- start start 2024-04-25T09:04:03.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.129+0000 7feb6ba5d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb640d19f0 0x7feb64159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:03.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.129+0000 7feb6ba5d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feb640d2370 0x7feb64158960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:03.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.129+0000 7feb6ba5d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb64161310 0x7feb640cc710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:03.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.129+0000 7feb6ba5d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb640c7da0 con 0x7feb64161310 2024-04-25T09:04:03.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.129+0000 7feb6ba5d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb640d2800 con 0x7feb640d19f0 2024-04-25T09:04:03.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.129+0000 7feb6ba5d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb640c7f20 con 0x7feb640d2370 2024-04-25T09:04:03.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.129+0000 7feb69ffa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb64161310 0x7feb640cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:03.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.129+0000 7feb69ffa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb64161310 0x7feb640cc710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41474/0 (socket says 172.21.15.43:41474) 2024-04-25T09:04:03.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.129+0000 7feb69ffa640 1 -- 172.21.15.43:0/2210245864 learned_addr learned my addr 172.21.15.43:0/2210245864 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:03.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.129+0000 7feb69ffa640 1 -- 172.21.15.43:0/2210245864 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feb640d2370 msgr2=0x7feb64158960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:03.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.129+0000 7feb69ffa640 1 --2- 172.21.15.43:0/2210245864 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feb640d2370 0x7feb64158960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:03.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.129+0000 7feb69ffa640 1 -- 172.21.15.43:0/2210245864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb640d19f0 msgr2=0x7feb64159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:03.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.129+0000 7feb69ffa640 1 --2- 172.21.15.43:0/2210245864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb640d19f0 0x7feb64159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:03.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.129+0000 7feb69ffa640 1 -- 172.21.15.43:0/2210245864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb640b6730 con 0x7feb64161310 2024-04-25T09:04:03.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.129+0000 7feb69ffa640 1 --2- 172.21.15.43:0/2210245864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb64161310 0x7feb640cc710 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7feb5c00efc0 tx=0x7feb5c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:03.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.130+0000 7feb59ffb640 1 -- 172.21.15.43:0/2210245864 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5c018020 con 0x7feb64161310 2024-04-25T09:04:03.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.130+0000 7feb59ffb640 1 -- 172.21.15.43:0/2210245864 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb5c014480 con 0x7feb64161310 2024-04-25T09:04:03.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.130+0000 7feb59ffb640 1 -- 172.21.15.43:0/2210245864 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb5c014870 con 0x7feb64161310 2024-04-25T09:04:03.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.130+0000 7feb6ba5d640 1 -- 172.21.15.43:0/2210245864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb640b6a10 con 0x7feb64161310 2024-04-25T09:04:03.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.130+0000 7feb6ba5d640 1 -- 172.21.15.43:0/2210245864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb64158ea0 con 0x7feb64161310 2024-04-25T09:04:03.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.131+0000 7feb6ba5d640 1 -- 172.21.15.43:0/2210245864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb640b64a0 con 0x7feb64161310 2024-04-25T09:04:03.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.131+0000 7feb59ffb640 1 -- 172.21.15.43:0/2210245864 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feb5c005d10 con 0x7feb64161310 2024-04-25T09:04:03.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.132+0000 7feb59ffb640 1 --2- 172.21.15.43:0/2210245864 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7feb4806b4d0 0x7feb4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:03.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.132+0000 7feb59ffb640 1 -- 172.21.15.43:0/2210245864 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5939+0+0 (secure 0 0 0) 0x7feb5c08b0e0 con 0x7feb64161310 2024-04-25T09:04:03.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.132+0000 7feb697f9640 1 --2- 172.21.15.43:0/2210245864 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7feb4806b4d0 0x7feb4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:03.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.133+0000 7feb697f9640 1 --2- 172.21.15.43:0/2210245864 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7feb4806b4d0 0x7feb4806d990 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7feb60001560 tx=0x7feb60007930 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:03.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.136+0000 7feb59ffb640 1 -- 172.21.15.43:0/2210245864 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feb5c022030 con 0x7feb64161310 2024-04-25T09:04:03.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:03.352+0000 7feb6ba5d640 1 -- 172.21.15.43:0/2210245864 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7feb640cee20 con 0x7feb64161310 2024-04-25T09:04:03.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.005+0000 7feb59ffb640 1 -- 172.21.15.43:0/2210245864 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v82) v1 ==== 104+0+0 (secure 0 0 0) 0x7feb5c010040 con 0x7feb64161310 2024-04-25T09:04:03.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.008+0000 7feb6ba5d640 1 -- 172.21.15.43:0/2210245864 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7feb4806b4d0 msgr2=0x7feb4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:03.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.008+0000 7feb6ba5d640 1 --2- 172.21.15.43:0/2210245864 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7feb4806b4d0 0x7feb4806d990 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7feb60001560 tx=0x7feb60007930 comp rx=0 tx=0).stop 2024-04-25T09:04:03.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.008+0000 7feb6ba5d640 1 -- 172.21.15.43:0/2210245864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb64161310 msgr2=0x7feb640cc710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:03.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.008+0000 7feb6ba5d640 1 --2- 172.21.15.43:0/2210245864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb64161310 0x7feb640cc710 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7feb5c00efc0 tx=0x7feb5c00c490 comp rx=0 tx=0).stop 2024-04-25T09:04:03.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.008+0000 7feb6ba5d640 1 -- 172.21.15.43:0/2210245864 shutdown_connections 2024-04-25T09:04:03.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.008+0000 7feb6ba5d640 1 --2- 172.21.15.43:0/2210245864 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb64161310 0x7feb640cc710 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:03.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.008+0000 7feb6ba5d640 1 --2- 172.21.15.43:0/2210245864 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feb640d2370 0x7feb64158960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:03.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.008+0000 7feb6ba5d640 1 --2- 172.21.15.43:0/2210245864 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7feb4806b4d0 0x7feb4806d990 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:03.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.008+0000 7feb6ba5d640 1 --2- 172.21.15.43:0/2210245864 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb640d19f0 0x7feb64159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:03.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.008+0000 7feb6ba5d640 1 -- 172.21.15.43:0/2210245864 >> 172.21.15.43:0/2210245864 conn(0x7feb6409cd20 msgr2=0x7feb640c82b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:03.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.008+0000 7feb6ba5d640 1 -- 172.21.15.43:0/2210245864 shutdown_connections 2024-04-25T09:04:03.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.008+0000 7feb6ba5d640 1 -- 172.21.15.43:0/2210245864 wait complete. 2024-04-25T09:04:04.011 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:04:04.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.176+0000 7f97f8c4a640 1 Processor -- start 2024-04-25T09:04:04.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.176+0000 7f97f8c4a640 1 -- start start 2024-04-25T09:04:04.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.177+0000 7f97f8c4a640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f97f415db50 0x7f97f409ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.177+0000 7f97f8c4a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f40b5ce0 con 0x7f97f406ccd0 2024-04-25T09:04:04.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.177+0000 7f97f8c4a640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f40b5f80 con 0x7f97f40b0cf0 2024-04-25T09:04:04.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.177+0000 7f97f8c4a640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97f40b1540 con 0x7f97f415db50 2024-04-25T09:04:04.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.177+0000 7f97f2d9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f97f406ccd0 0x7f97f40b19d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56718/0 (socket says 172.21.15.43:56718) 2024-04-25T09:04:04.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.177+0000 7f97f2d9d640 1 -- 172.21.15.43:0/2193705829 learned_addr learned my addr 172.21.15.43:0/2193705829 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:04.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.177+0000 7f97f259c640 1 --2- 172.21.15.43:0/2193705829 >> v2:172.21.15.193:3300/0 conn(0x7f97f415db50 0x7f97f409ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.177+0000 7f97f159a640 1 -- 172.21.15.43:0/2193705829 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1672071189 0 0) 0x7f97f40b5ce0 con 0x7f97f406ccd0 2024-04-25T09:04:04.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.177+0000 7f97f159a640 1 -- 172.21.15.43:0/2193705829 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97dc003520 con 0x7f97f406ccd0 2024-04-25T09:04:04.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.178+0000 7f97f159a640 1 -- 172.21.15.43:0/2193705829 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2938892664 0 0) 0x7f97f40b5f80 con 0x7f97f40b0cf0 2024-04-25T09:04:04.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.178+0000 7f97f159a640 1 -- 172.21.15.43:0/2193705829 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97f40b5ce0 con 0x7f97f40b0cf0 2024-04-25T09:04:04.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.178+0000 7f97f159a640 1 -- 172.21.15.43:0/2193705829 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 811998416 0 0) 0x7f97dc003520 con 0x7f97f406ccd0 2024-04-25T09:04:04.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.178+0000 7f97f159a640 1 -- 172.21.15.43:0/2193705829 >> v2:172.21.15.193:3300/0 conn(0x7f97f415db50 msgr2=0x7f97f409ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:04.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.178+0000 7f97f159a640 1 --2- 172.21.15.43:0/2193705829 >> v2:172.21.15.193:3300/0 conn(0x7f97f415db50 0x7f97f409ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.178+0000 7f97f159a640 1 -- 172.21.15.43:0/2193705829 >> v1:172.21.15.193:6789/0 conn(0x7f97f40b0cf0 legacy=0x7f97f406c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:04.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.178+0000 7f97f159a640 1 -- 172.21.15.43:0/2193705829 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f40b6260 con 0x7f97f406ccd0 2024-04-25T09:04:04.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.178+0000 7f97f159a640 1 -- 172.21.15.43:0/2193705829 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f97ec001a10 con 0x7f97f406ccd0 2024-04-25T09:04:04.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.178+0000 7f97f159a640 1 -- 172.21.15.43:0/2193705829 >> v1:172.21.15.43:6789/0 conn(0x7f97f406ccd0 legacy=0x7f97f40b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:04.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.178+0000 7f97f159a640 1 --2- 172.21.15.43:0/2193705829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97dc003b90 0x7f97dc003f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.178+0000 7f97f159a640 1 --2- 172.21.15.43:0/2193705829 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97dc004800 0x7f97dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.178+0000 7f97f159a640 1 --2- 172.21.15.43:0/2193705829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97dc0051c0 0x7f97dc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.178+0000 7f97f159a640 1 -- 172.21.15.43:0/2193705829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f40b1540 con 0x7f97dc0051c0 2024-04-25T09:04:04.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.178+0000 7f97f159a640 1 -- 172.21.15.43:0/2193705829 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc00c090 con 0x7f97dc003b90 2024-04-25T09:04:04.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.178+0000 7f97f159a640 1 -- 172.21.15.43:0/2193705829 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc00c290 con 0x7f97dc004800 2024-04-25T09:04:04.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.178+0000 7f97f2d9d640 1 --2- 172.21.15.43:0/2193705829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97dc0051c0 0x7f97dc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.178+0000 7f97f1d9b640 1 --2- 172.21.15.43:0/2193705829 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97dc004800 0x7f97dc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.178+0000 7f97f259c640 1 --2- 172.21.15.43:0/2193705829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97dc003b90 0x7f97dc003f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.178+0000 7f97f2d9d640 1 -- 172.21.15.43:0/2193705829 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97dc004800 msgr2=0x7f97dc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:04.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.178+0000 7f97f2d9d640 1 --2- 172.21.15.43:0/2193705829 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97dc004800 0x7f97dc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.178+0000 7f97f2d9d640 1 -- 172.21.15.43:0/2193705829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97dc003b90 msgr2=0x7f97dc003f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:04.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.178+0000 7f97f2d9d640 1 --2- 172.21.15.43:0/2193705829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97dc003b90 0x7f97dc003f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.178+0000 7f97f2d9d640 1 -- 172.21.15.43:0/2193705829 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97dc003520 con 0x7f97dc0051c0 2024-04-25T09:04:04.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.178+0000 7f97f1d9b640 1 --2- 172.21.15.43:0/2193705829 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97dc004800 0x7f97dc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:04.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.178+0000 7f97f259c640 1 --2- 172.21.15.43:0/2193705829 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97dc003b90 0x7f97dc003f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:04.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.179+0000 7f97f2d9d640 1 -- 172.21.15.43:0/2193705829 reap_dead start 2024-04-25T09:04:04.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.179+0000 7f97f2d9d640 1 --2- 172.21.15.43:0/2193705829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97dc0051c0 0x7f97dc00b9c0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f97ec00a740 tx=0x7f97dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0869713a63a0c85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:04.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.179+0000 7f97f159a640 1 -- 172.21.15.43:0/2193705829 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ec00a390 con 0x7f97dc0051c0 2024-04-25T09:04:04.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.179+0000 7f97f159a640 1 -- 172.21.15.43:0/2193705829 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97ec02f070 con 0x7f97dc0051c0 2024-04-25T09:04:04.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.179+0000 7f97f159a640 1 -- 172.21.15.43:0/2193705829 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ec02bb50 con 0x7f97dc0051c0 2024-04-25T09:04:04.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.180+0000 7f97f8c4a640 1 -- 172.21.15.43:0/2193705829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97dc0051c0 msgr2=0x7f97dc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:04.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.180+0000 7f97f8c4a640 1 --2- 172.21.15.43:0/2193705829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97dc0051c0 0x7f97dc00b9c0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f97ec00a740 tx=0x7f97dc005830 comp rx=0 tx=0).stop 2024-04-25T09:04:04.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.180+0000 7f97f8c4a640 1 -- 172.21.15.43:0/2193705829 shutdown_connections 2024-04-25T09:04:04.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.180+0000 7f97f8c4a640 1 --2- 172.21.15.43:0/2193705829 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97dc0051c0 0x7f97dc00b9c0 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.180+0000 7f97f8c4a640 1 -- 172.21.15.43:0/2193705829 >> 172.21.15.43:0/2193705829 conn(0x7f97f409aa70 msgr2=0x7f97f40b6be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:04.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.180+0000 7f97f8c4a640 1 -- 172.21.15.43:0/2193705829 shutdown_connections 2024-04-25T09:04:04.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.180+0000 7f97f8c4a640 1 -- 172.21.15.43:0/2193705829 wait complete. 2024-04-25T09:04:04.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.181+0000 7f97f8c4a640 1 Processor -- start 2024-04-25T09:04:04.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.181+0000 7f97f8c4a640 1 -- start start 2024-04-25T09:04:04.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.182+0000 7f97f8c4a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97dc0051c0 0x7f97f414b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.182+0000 7f97f8c4a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97f415bdb0 0x7f97f41f4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.182+0000 7f97f8c4a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97f4160d60 0x7f97f415b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.182+0000 7f97f8c4a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f415df30 con 0x7f97dc0051c0 2024-04-25T09:04:04.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.182+0000 7f97f8c4a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f40b11d0 con 0x7f97f4160d60 2024-04-25T09:04:04.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.182+0000 7f97f8c4a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f415e0b0 con 0x7f97f415bdb0 2024-04-25T09:04:04.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.182+0000 7f97f259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97dc0051c0 0x7f97f414b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.182+0000 7f97f259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97dc0051c0 0x7f97f414b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41494/0 (socket says 172.21.15.43:41494) 2024-04-25T09:04:04.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.182+0000 7f97f259c640 1 -- 172.21.15.43:0/3030118591 learned_addr learned my addr 172.21.15.43:0/3030118591 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:04.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.182+0000 7f97f1d9b640 1 --2- 172.21.15.43:0/3030118591 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97f415bdb0 0x7f97f41f4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.182+0000 7f97f2d9d640 1 --2- 172.21.15.43:0/3030118591 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97f4160d60 0x7f97f415b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.182+0000 7f97f259c640 1 -- 172.21.15.43:0/3030118591 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97f415bdb0 msgr2=0x7f97f41f4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:04.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.182+0000 7f97f259c640 1 --2- 172.21.15.43:0/3030118591 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97f415bdb0 0x7f97f41f4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.182+0000 7f97f259c640 1 -- 172.21.15.43:0/3030118591 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97f4160d60 msgr2=0x7f97f415b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:04.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.182+0000 7f97f259c640 1 --2- 172.21.15.43:0/3030118591 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97f4160d60 0x7f97f415b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:04.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.183+0000 7f97f259c640 1 -- 172.21.15.43:0/3030118591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f415e480 con 0x7f97dc0051c0 2024-04-25T09:04:04.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.183+0000 7f97f259c640 1 --2- 172.21.15.43:0/3030118591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97dc0051c0 0x7f97f414b450 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f97e4011540 tx=0x7f97e4011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:04.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.183+0000 7f97e2ffd640 1 -- 172.21.15.43:0/3030118591 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e4015870 con 0x7f97dc0051c0 2024-04-25T09:04:04.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.183+0000 7f97e2ffd640 1 -- 172.21.15.43:0/3030118591 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e4015a10 con 0x7f97dc0051c0 2024-04-25T09:04:04.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.183+0000 7f97e2ffd640 1 -- 172.21.15.43:0/3030118591 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e4015dc0 con 0x7f97dc0051c0 2024-04-25T09:04:04.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.183+0000 7f97f8c4a640 1 -- 172.21.15.43:0/3030118591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97f415e760 con 0x7f97dc0051c0 2024-04-25T09:04:04.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.183+0000 7f97f8c4a640 1 -- 172.21.15.43:0/3030118591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f415ed20 con 0x7f97dc0051c0 2024-04-25T09:04:04.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.184+0000 7f97f8c4a640 1 -- 172.21.15.43:0/3030118591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97f4068a40 con 0x7f97dc0051c0 2024-04-25T09:04:04.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.184+0000 7f97e2ffd640 1 -- 172.21.15.43:0/3030118591 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f97e400f460 con 0x7f97dc0051c0 2024-04-25T09:04:04.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.185+0000 7f97e2ffd640 1 --2- 172.21.15.43:0/3030118591 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f97d406b4d0 0x7f97d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:04.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.185+0000 7f97e2ffd640 1 -- 172.21.15.43:0/3030118591 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 5939+0+0 (secure 0 0 0) 0x7f97e408abf0 con 0x7f97dc0051c0 2024-04-25T09:04:04.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.185+0000 7f97f1d9b640 1 --2- 172.21.15.43:0/3030118591 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f97d406b4d0 0x7f97d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:04.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.186+0000 7f97f1d9b640 1 --2- 172.21.15.43:0/3030118591 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f97d406b4d0 0x7f97d406d990 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f97f415aeb0 tx=0x7f97e8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:04.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.189+0000 7f97e2ffd640 1 -- 172.21.15.43:0/3030118591 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f97e4019030 con 0x7f97dc0051c0 2024-04-25T09:04:04.357 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:04.368+0000 7f97f8c4a640 1 -- 172.21.15.43:0/3030118591 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f97f406dc30 con 0x7f97dc0051c0 2024-04-25T09:04:04.996 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.007+0000 7f97e2ffd640 1 -- 172.21.15.43:0/3030118591 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v131) v1 ==== 181+0+0 (secure 0 0 0) 0x7f97e4059970 con 0x7f97dc0051c0 2024-04-25T09:04:04.996 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:04:05.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.011+0000 7f97f8c4a640 1 -- 172.21.15.43:0/3030118591 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f97d406b4d0 msgr2=0x7f97d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:05.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.011+0000 7f97f8c4a640 1 --2- 172.21.15.43:0/3030118591 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f97d406b4d0 0x7f97d406d990 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f97f415aeb0 tx=0x7f97e8009040 comp rx=0 tx=0).stop 2024-04-25T09:04:05.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.011+0000 7f97f8c4a640 1 -- 172.21.15.43:0/3030118591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97dc0051c0 msgr2=0x7f97f414b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:05.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.011+0000 7f97f8c4a640 1 --2- 172.21.15.43:0/3030118591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97dc0051c0 0x7f97f414b450 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f97e4011540 tx=0x7f97e4011a00 comp rx=0 tx=0).stop 2024-04-25T09:04:05.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.011+0000 7f97f8c4a640 1 -- 172.21.15.43:0/3030118591 shutdown_connections 2024-04-25T09:04:05.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.011+0000 7f97f8c4a640 1 --2- 172.21.15.43:0/3030118591 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f97d406b4d0 0x7f97d406d990 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.011+0000 7f97f8c4a640 1 --2- 172.21.15.43:0/3030118591 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f97f4160d60 0x7f97f415b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.011+0000 7f97f8c4a640 1 --2- 172.21.15.43:0/3030118591 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f97f415bdb0 0x7f97f41f4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.011+0000 7f97f8c4a640 1 --2- 172.21.15.43:0/3030118591 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f97dc0051c0 0x7f97f414b450 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.011+0000 7f97f8c4a640 1 -- 172.21.15.43:0/3030118591 >> 172.21.15.43:0/3030118591 conn(0x7f97f409aa70 msgr2=0x7f97f40b3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:05.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.012+0000 7f97f8c4a640 1 -- 172.21.15.43:0/3030118591 shutdown_connections 2024-04-25T09:04:05.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.012+0000 7f97f8c4a640 1 -- 172.21.15.43:0/3030118591 wait complete. 2024-04-25T09:04:05.015 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:04:05.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.139+0000 7fd83716a640 1 Processor -- start 2024-04-25T09:04:05.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.139+0000 7fd83716a640 1 -- start start 2024-04-25T09:04:05.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.140+0000 7fd83716a640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fd83015bb40 0x7fd8300ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:05.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.140+0000 7fd83716a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd8300cca30 0x7fd830070b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:05.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.140+0000 7fd83716a640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd830160510 con 0x7fd8300aec70 2024-04-25T09:04:05.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.140+0000 7fd83716a640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd83015f750 con 0x7fd83015bb40 2024-04-25T09:04:05.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.140+0000 7fd83716a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd83015fa30 con 0x7fd8300cca30 2024-04-25T09:04:05.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.140+0000 7fd835707640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd8300cca30 0x7fd830070b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:05.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.140+0000 7fd834f06640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fd83015bb40 0x7fd8300ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:05.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.140+0000 7fd835707640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd8300cca30 0x7fd830070b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:48746/0 (socket says 172.21.15.43:48746) 2024-04-25T09:04:05.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.140+0000 7fd835707640 1 -- 172.21.15.43:0/3703780860 learned_addr learned my addr 172.21.15.43:0/3703780860 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:05.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.141+0000 7fd835707640 1 -- 172.21.15.43:0/3703780860 >> v2:172.21.15.193:3300/0 conn(0x7fd83015bb40 msgr2=0x7fd8300ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:05.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.141+0000 7fd835707640 1 --2- 172.21.15.43:0/3703780860 >> v2:172.21.15.193:3300/0 conn(0x7fd83015bb40 0x7fd8300ae730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.141+0000 7fd835707640 1 -- 172.21.15.43:0/3703780860 >> v1:172.21.15.193:6789/0 conn(0x7fd8300aec70 legacy=0x7fd8300cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:05.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.141+0000 7fd835707640 1 -- 172.21.15.43:0/3703780860 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd83015fd10 con 0x7fd8300cca30 2024-04-25T09:04:05.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.141+0000 7fd834f06640 1 --2- 172.21.15.43:0/3703780860 >> v2:172.21.15.193:3300/0 conn(0x7fd83015bb40 0x7fd8300ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:04:05.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.141+0000 7fd835707640 1 --2- 172.21.15.43:0/3703780860 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd8300cca30 0x7fd830070b80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd828009e50 tx=0x7fd82802a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=847ed91b35c5e641 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:05.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.142+0000 7fd82f7fe640 1 -- 172.21.15.43:0/3703780860 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd828036070 con 0x7fd8300cca30 2024-04-25T09:04:05.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.142+0000 7fd82f7fe640 1 -- 172.21.15.43:0/3703780860 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd828031600 con 0x7fd8300cca30 2024-04-25T09:04:05.144 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.142+0000 7fd82f7fe640 1 -- 172.21.15.43:0/3703780860 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8280319d0 con 0x7fd8300cca30 2024-04-25T09:04:05.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.142+0000 7fd83716a640 1 -- 172.21.15.43:0/3703780860 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd8300cca30 msgr2=0x7fd830070b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:05.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.142+0000 7fd83716a640 1 --2- 172.21.15.43:0/3703780860 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd8300cca30 0x7fd830070b80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd828009e50 tx=0x7fd82802a710 comp rx=0 tx=0).stop 2024-04-25T09:04:05.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.142+0000 7fd83716a640 1 -- 172.21.15.43:0/3703780860 shutdown_connections 2024-04-25T09:04:05.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.142+0000 7fd83716a640 1 --2- 172.21.15.43:0/3703780860 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd8300cca30 0x7fd830070b80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.142+0000 7fd83716a640 1 --2- 172.21.15.43:0/3703780860 >> v2:172.21.15.193:3300/0 conn(0x7fd83015bb40 0x7fd8300ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.142+0000 7fd83716a640 1 -- 172.21.15.43:0/3703780860 >> 172.21.15.43:0/3703780860 conn(0x7fd83006ab30 msgr2=0x7fd83006af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:05.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.142+0000 7fd83716a640 1 -- 172.21.15.43:0/3703780860 shutdown_connections 2024-04-25T09:04:05.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.142+0000 7fd83716a640 1 -- 172.21.15.43:0/3703780860 wait complete. 2024-04-25T09:04:05.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.143+0000 7fd83716a640 1 Processor -- start 2024-04-25T09:04:05.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.143+0000 7fd83716a640 1 -- start start 2024-04-25T09:04:05.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.144+0000 7fd83716a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd8300aec70 0x7fd83013ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:05.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.144+0000 7fd83716a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8300cca30 0x7fd830140ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:05.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.144+0000 7fd83716a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd83015bb40 0x7fd8300ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:05.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.144+0000 7fd83716a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd830071240 con 0x7fd8300cca30 2024-04-25T09:04:05.145 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.144+0000 7fd83716a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8300710c0 con 0x7fd83015bb40 2024-04-25T09:04:05.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.144+0000 7fd83716a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8300713c0 con 0x7fd8300aec70 2024-04-25T09:04:05.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.144+0000 7fd82ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8300cca30 0x7fd830140ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:05.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.144+0000 7fd82ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8300cca30 0x7fd830140ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41504/0 (socket says 172.21.15.43:41504) 2024-04-25T09:04:05.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.144+0000 7fd82ffff640 1 -- 172.21.15.43:0/4231427772 learned_addr learned my addr 172.21.15.43:0/4231427772 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:05.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.144+0000 7fd834f06640 1 --2- 172.21.15.43:0/4231427772 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd8300aec70 0x7fd83013ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:05.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.144+0000 7fd835707640 1 --2- 172.21.15.43:0/4231427772 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd83015bb40 0x7fd8300ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:05.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.145+0000 7fd82ffff640 1 -- 172.21.15.43:0/4231427772 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd8300aec70 msgr2=0x7fd83013ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:05.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.145+0000 7fd82ffff640 1 --2- 172.21.15.43:0/4231427772 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd8300aec70 0x7fd83013ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.145+0000 7fd82ffff640 1 -- 172.21.15.43:0/4231427772 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd83015bb40 msgr2=0x7fd8300ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:05.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.145+0000 7fd82ffff640 1 --2- 172.21.15.43:0/4231427772 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd83015bb40 0x7fd8300ad8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:05.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.145+0000 7fd82ffff640 1 -- 172.21.15.43:0/4231427772 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8301e55b0 con 0x7fd8300cca30 2024-04-25T09:04:05.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.145+0000 7fd82ffff640 1 --2- 172.21.15.43:0/4231427772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8300cca30 0x7fd830140ba0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fd81c00c400 tx=0x7fd81c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:05.146 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.145+0000 7fd82d7fa640 1 -- 172.21.15.43:0/4231427772 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81c0052d0 con 0x7fd8300cca30 2024-04-25T09:04:05.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.145+0000 7fd82d7fa640 1 -- 172.21.15.43:0/4231427772 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd81c005490 con 0x7fd8300cca30 2024-04-25T09:04:05.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.145+0000 7fd83716a640 1 -- 172.21.15.43:0/4231427772 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd83015d350 con 0x7fd8300cca30 2024-04-25T09:04:05.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.145+0000 7fd82d7fa640 1 -- 172.21.15.43:0/4231427772 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81c007260 con 0x7fd8300cca30 2024-04-25T09:04:05.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.145+0000 7fd83716a640 1 -- 172.21.15.43:0/4231427772 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8301410e0 con 0x7fd8300cca30 2024-04-25T09:04:05.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.146+0000 7fd83716a640 1 -- 172.21.15.43:0/4231427772 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd83006e590 con 0x7fd8300cca30 2024-04-25T09:04:05.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.147+0000 7fd82d7fa640 1 -- 172.21.15.43:0/4231427772 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd81c007400 con 0x7fd8300cca30 2024-04-25T09:04:05.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.147+0000 7fd82d7fa640 1 --2- 172.21.15.43:0/4231427772 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd81406b500 0x7fd81406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:05.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.147+0000 7fd82d7fa640 1 -- 172.21.15.43:0/4231427772 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5501+0+0 (secure 0 0 0) 0x7fd81c0894d0 con 0x7fd8300cca30 2024-04-25T09:04:05.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.147+0000 7fd834f06640 1 --2- 172.21.15.43:0/4231427772 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd81406b500 0x7fd81406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:05.147 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.148+0000 7fd834f06640 1 --2- 172.21.15.43:0/4231427772 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd81406b500 0x7fd81406d9c0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fd824009560 tx=0x7fd824009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:05.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.151+0000 7fd82d7fa640 1 -- 172.21.15.43:0/4231427772 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd83006e590 con 0x7fd8300cca30 2024-04-25T09:04:05.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:05.340+0000 7fd83716a640 1 -- 172.21.15.43:0/4231427772 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd8300cbd60 con 0x7fd8300cca30 2024-04-25T09:04:06.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.013+0000 7fd82d7fa640 1 -- 172.21.15.43:0/4231427772 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v132) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd81c05ce20 con 0x7fd8300cca30 2024-04-25T09:04:06.002 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' removed 2024-04-25T09:04:06.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.018+0000 7fd83716a640 1 -- 172.21.15.43:0/4231427772 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd81406b500 msgr2=0x7fd81406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:06.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.018+0000 7fd83716a640 1 --2- 172.21.15.43:0/4231427772 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd81406b500 0x7fd81406d9c0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fd824009560 tx=0x7fd824009190 comp rx=0 tx=0).stop 2024-04-25T09:04:06.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.018+0000 7fd83716a640 1 -- 172.21.15.43:0/4231427772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8300cca30 msgr2=0x7fd830140ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:06.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.018+0000 7fd83716a640 1 --2- 172.21.15.43:0/4231427772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8300cca30 0x7fd830140ba0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fd81c00c400 tx=0x7fd81c00c8c0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.018+0000 7fd83716a640 1 -- 172.21.15.43:0/4231427772 shutdown_connections 2024-04-25T09:04:06.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.018+0000 7fd83716a640 1 --2- 172.21.15.43:0/4231427772 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd81406b500 0x7fd81406d9c0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.018+0000 7fd83716a640 1 --2- 172.21.15.43:0/4231427772 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd83015bb40 0x7fd8300ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.018+0000 7fd83716a640 1 --2- 172.21.15.43:0/4231427772 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd8300cca30 0x7fd830140ba0 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.018+0000 7fd83716a640 1 --2- 172.21.15.43:0/4231427772 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd8300aec70 0x7fd83013ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.018+0000 7fd83716a640 1 -- 172.21.15.43:0/4231427772 >> 172.21.15.43:0/4231427772 conn(0x7fd83006ab30 msgr2=0x7fd830156630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:06.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.018+0000 7fd83716a640 1 -- 172.21.15.43:0/4231427772 shutdown_connections 2024-04-25T09:04:06.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.018+0000 7fd83716a640 1 -- 172.21.15.43:0/4231427772 wait complete. 2024-04-25T09:04:06.028 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T09:04:06.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.175+0000 7f4b1a85a640 1 Processor -- start 2024-04-25T09:04:06.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.175+0000 7f4b1a85a640 1 -- start start 2024-04-25T09:04:06.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.176+0000 7f4b1a85a640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f4b1415dd60 0x7f4b140d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:06.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.176+0000 7f4b1a85a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4b140d0aa0 0x7f4b1406bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:06.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.176+0000 7f4b1a85a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b1415e140 con 0x7f4b140d1fa0 2024-04-25T09:04:06.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.176+0000 7f4b1a85a640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b140d27d0 con 0x7f4b1415dd60 2024-04-25T09:04:06.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.176+0000 7f4b1a85a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1406b1e0 con 0x7f4b140d0aa0 2024-04-25T09:04:06.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.176+0000 7f4b137fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f4b140d1fa0 0x7f4b140d0390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56722/0 (socket says 172.21.15.43:56722) 2024-04-25T09:04:06.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.176+0000 7f4b137fe640 1 -- 172.21.15.43:0/2548045206 learned_addr learned my addr 172.21.15.43:0/2548045206 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:06.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.177+0000 7f4b13fff640 1 --2- 172.21.15.43:0/2548045206 >> v2:172.21.15.193:3300/0 conn(0x7f4b1415dd60 0x7f4b140d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:06.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.177+0000 7f4b18df7640 1 --2- 172.21.15.43:0/2548045206 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4b140d0aa0 0x7f4b1406bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:06.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.177+0000 7f4b12ffd640 1 -- 172.21.15.43:0/2548045206 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4208812966 0 0) 0x7f4b1415e140 con 0x7f4b140d1fa0 2024-04-25T09:04:06.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.177+0000 7f4b12ffd640 1 -- 172.21.15.43:0/2548045206 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4afc003520 con 0x7f4b140d1fa0 2024-04-25T09:04:06.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.177+0000 7f4b12ffd640 1 -- 172.21.15.43:0/2548045206 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4013865306 0 0) 0x7f4afc003520 con 0x7f4b140d1fa0 2024-04-25T09:04:06.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.177+0000 7f4b12ffd640 1 -- 172.21.15.43:0/2548045206 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4b140d0aa0 msgr2=0x7f4b1406bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:06.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.177+0000 7f4b12ffd640 1 --2- 172.21.15.43:0/2548045206 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4b140d0aa0 0x7f4b1406bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.177+0000 7f4b12ffd640 1 -- 172.21.15.43:0/2548045206 >> v2:172.21.15.193:3300/0 conn(0x7f4b1415dd60 msgr2=0x7f4b140d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:06.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.177+0000 7f4b12ffd640 1 --2- 172.21.15.43:0/2548045206 >> v2:172.21.15.193:3300/0 conn(0x7f4b1415dd60 0x7f4b140d1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.177+0000 7f4b12ffd640 1 -- 172.21.15.43:0/2548045206 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b1406b4c0 con 0x7f4b140d1fa0 2024-04-25T09:04:06.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.177+0000 7f4b12ffd640 1 -- 172.21.15.43:0/2548045206 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f4b04001670 con 0x7f4b140d1fa0 2024-04-25T09:04:06.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.177+0000 7f4b12ffd640 1 -- 172.21.15.43:0/2548045206 >> v1:172.21.15.43:6789/0 conn(0x7f4b140d1fa0 legacy=0x7f4b140d0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:06.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.177+0000 7f4b12ffd640 1 --2- 172.21.15.43:0/2548045206 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4b140d0aa0 0x7f4afc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:06.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.177+0000 7f4b12ffd640 1 --2- 172.21.15.43:0/2548045206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4afc003f30 0x7f4afc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:06.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.177+0000 7f4b12ffd640 1 --2- 172.21.15.43:0/2548045206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4afc004d40 0x7f4afc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:06.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.177+0000 7f4b12ffd640 1 -- 172.21.15.43:0/2548045206 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b140d27d0 con 0x7f4afc003f30 2024-04-25T09:04:06.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.177+0000 7f4b12ffd640 1 -- 172.21.15.43:0/2548045206 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1406b1e0 con 0x7f4afc004d40 2024-04-25T09:04:06.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.177+0000 7f4b12ffd640 1 -- 172.21.15.43:0/2548045206 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc00b9c0 con 0x7f4b140d0aa0 2024-04-25T09:04:06.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.177+0000 7f4b137fe640 1 --2- 172.21.15.43:0/2548045206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4afc003f30 0x7f4afc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:06.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.177+0000 7f4b13fff640 1 --2- 172.21.15.43:0/2548045206 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4b140d0aa0 0x7f4afc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:06.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.177+0000 7f4b18df7640 1 --2- 172.21.15.43:0/2548045206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4afc004d40 0x7f4afc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:06.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.177+0000 7f4b137fe640 1 -- 172.21.15.43:0/2548045206 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4b140d0aa0 msgr2=0x7f4afc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:06.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.177+0000 7f4b137fe640 1 --2- 172.21.15.43:0/2548045206 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4b140d0aa0 0x7f4afc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.177+0000 7f4b137fe640 1 -- 172.21.15.43:0/2548045206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4afc004d40 msgr2=0x7f4afc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:06.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.177+0000 7f4b137fe640 1 --2- 172.21.15.43:0/2548045206 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4afc004d40 0x7f4afc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.177+0000 7f4b137fe640 1 -- 172.21.15.43:0/2548045206 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4afc003520 con 0x7f4afc003f30 2024-04-25T09:04:06.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.178+0000 7f4b137fe640 1 --2- 172.21.15.43:0/2548045206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4afc003f30 0x7f4afc004800 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f4b04001050 tx=0x7f4afc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91ce08aa940021e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:06.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.178+0000 7f4b12ffd640 1 -- 172.21.15.43:0/2548045206 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b04007d70 con 0x7f4afc003f30 2024-04-25T09:04:06.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.178+0000 7f4b12ffd640 1 -- 172.21.15.43:0/2548045206 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b04005af0 con 0x7f4afc003f30 2024-04-25T09:04:06.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.178+0000 7f4b12ffd640 1 -- 172.21.15.43:0/2548045206 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b04007070 con 0x7f4afc003f30 2024-04-25T09:04:06.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.179+0000 7f4b1a85a640 1 -- 172.21.15.43:0/2548045206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4afc003f30 msgr2=0x7f4afc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:06.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.179+0000 7f4b1a85a640 1 --2- 172.21.15.43:0/2548045206 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4afc003f30 0x7f4afc004800 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f4b04001050 tx=0x7f4afc005830 comp rx=0 tx=0).stop 2024-04-25T09:04:06.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.179+0000 7f4b18df7640 1 -- 172.21.15.43:0/2548045206 reap_dead start 2024-04-25T09:04:06.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.179+0000 7f4b1a85a640 1 -- 172.21.15.43:0/2548045206 shutdown_connections 2024-04-25T09:04:06.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.179+0000 7f4b1a85a640 1 -- 172.21.15.43:0/2548045206 >> 172.21.15.43:0/2548045206 conn(0x7f4b140d39c0 msgr2=0x7f4b140d3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:06.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.179+0000 7f4b1a85a640 1 -- 172.21.15.43:0/2548045206 shutdown_connections 2024-04-25T09:04:06.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.179+0000 7f4b1a85a640 1 -- 172.21.15.43:0/2548045206 wait complete. 2024-04-25T09:04:06.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.180+0000 7f4b1a85a640 1 Processor -- start 2024-04-25T09:04:06.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.180+0000 7f4b1a85a640 1 -- start start 2024-04-25T09:04:06.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.181+0000 7f4b1a85a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4afc003f30 0x7f4b140b6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:06.174 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.181+0000 7f4b1a85a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4b1415bb00 0x7f4b140c9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:06.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.181+0000 7f4b1a85a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b140b7e90 0x7f4b14160500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:06.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.181+0000 7f4b1a85a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b140d25b0 con 0x7f4afc003f30 2024-04-25T09:04:06.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.181+0000 7f4b1a85a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b140d2430 con 0x7f4b140b7e90 2024-04-25T09:04:06.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.181+0000 7f4b1a85a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1406bcc0 con 0x7f4b1415bb00 2024-04-25T09:04:06.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.181+0000 7f4b13fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4afc003f30 0x7f4b140b6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:06.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.181+0000 7f4b13fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4afc003f30 0x7f4b140b6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41532/0 (socket says 172.21.15.43:41532) 2024-04-25T09:04:06.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.181+0000 7f4b13fff640 1 -- 172.21.15.43:0/1149937271 learned_addr learned my addr 172.21.15.43:0/1149937271 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:06.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.181+0000 7f4b18df7640 1 --2- 172.21.15.43:0/1149937271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b140b7e90 0x7f4b14160500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:06.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.181+0000 7f4b137fe640 1 --2- 172.21.15.43:0/1149937271 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4b1415bb00 0x7f4b140c9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:06.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.181+0000 7f4b13fff640 1 -- 172.21.15.43:0/1149937271 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4b1415bb00 msgr2=0x7f4b140c9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:06.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.181+0000 7f4b13fff640 1 --2- 172.21.15.43:0/1149937271 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4b1415bb00 0x7f4b140c9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.175 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.181+0000 7f4b13fff640 1 -- 172.21.15.43:0/1149937271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b140b7e90 msgr2=0x7f4b14160500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:06.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.181+0000 7f4b13fff640 1 --2- 172.21.15.43:0/1149937271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b140b7e90 0x7f4b14160500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.181+0000 7f4b13fff640 1 -- 172.21.15.43:0/1149937271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b140b6df0 con 0x7f4afc003f30 2024-04-25T09:04:06.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.181+0000 7f4b137fe640 1 --2- 172.21.15.43:0/1149937271 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4b1415bb00 0x7f4b140c9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:06.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.182+0000 7f4b13fff640 1 --2- 172.21.15.43:0/1149937271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4afc003f30 0x7f4b140b6810 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f4b080119c0 tx=0x7f4b08011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:06.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.182+0000 7f4b10ff9640 1 -- 172.21.15.43:0/1149937271 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0800fb50 con 0x7f4afc003f30 2024-04-25T09:04:06.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.182+0000 7f4b10ff9640 1 -- 172.21.15.43:0/1149937271 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b0800fcf0 con 0x7f4afc003f30 2024-04-25T09:04:06.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.182+0000 7f4b1a85a640 1 -- 172.21.15.43:0/1149937271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b140b70d0 con 0x7f4afc003f30 2024-04-25T09:04:06.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.182+0000 7f4b10ff9640 1 -- 172.21.15.43:0/1149937271 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0800fea0 con 0x7f4afc003f30 2024-04-25T09:04:06.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.182+0000 7f4b1a85a640 1 -- 172.21.15.43:0/1149937271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b140b7690 con 0x7f4afc003f30 2024-04-25T09:04:06.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.183+0000 7f4b1a85a640 1 -- 172.21.15.43:0/1149937271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b14074f60 con 0x7f4afc003f30 2024-04-25T09:04:06.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.183+0000 7f4b10ff9640 1 -- 172.21.15.43:0/1149937271 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4b08012040 con 0x7f4afc003f30 2024-04-25T09:04:06.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.183+0000 7f4b10ff9640 1 --2- 172.21.15.43:0/1149937271 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4b0006b4d0 0x7f4b0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:06.176 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.184+0000 7f4b10ff9640 1 -- 172.21.15.43:0/1149937271 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f4b0808a590 con 0x7f4afc003f30 2024-04-25T09:04:06.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.184+0000 7f4b137fe640 1 --2- 172.21.15.43:0/1149937271 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4b0006b4d0 0x7f4b0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:06.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.184+0000 7f4b137fe640 1 --2- 172.21.15.43:0/1149937271 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4b0006b4d0 0x7f4b0006d990 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f4b04033000 tx=0x7f4b04009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:06.177 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.188+0000 7f4b10ff9640 1 -- 172.21.15.43:0/1149937271 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4b08019030 con 0x7f4afc003f30 2024-04-25T09:04:06.358 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.369+0000 7f4b1a85a640 1 -- 172.21.15.43:0/1149937271 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f4b140b4fd0 con 0x7f4afc003f30 2024-04-25T09:04:06.360 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.372+0000 7f4b10ff9640 1 -- 172.21.15.43:0/1149937271 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4b08059740 con 0x7f4afc003f30 2024-04-25T09:04:06.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.374+0000 7f4b1a85a640 1 -- 172.21.15.43:0/1149937271 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4b0006b4d0 msgr2=0x7f4b0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:06.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.374+0000 7f4b1a85a640 1 --2- 172.21.15.43:0/1149937271 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4b0006b4d0 0x7f4b0006d990 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f4b04033000 tx=0x7f4b04009030 comp rx=0 tx=0).stop 2024-04-25T09:04:06.363 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.374+0000 7f4b1a85a640 1 -- 172.21.15.43:0/1149937271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4afc003f30 msgr2=0x7f4b140b6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:06.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.374+0000 7f4b1a85a640 1 --2- 172.21.15.43:0/1149937271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4afc003f30 0x7f4b140b6810 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f4b080119c0 tx=0x7f4b08011e80 comp rx=0 tx=0).stop 2024-04-25T09:04:06.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.374+0000 7f4b1a85a640 1 -- 172.21.15.43:0/1149937271 shutdown_connections 2024-04-25T09:04:06.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.374+0000 7f4b1a85a640 1 --2- 172.21.15.43:0/1149937271 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4b0006b4d0 0x7f4b0006d990 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.374+0000 7f4b1a85a640 1 --2- 172.21.15.43:0/1149937271 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4b140b7e90 0x7f4b14160500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.374+0000 7f4b1a85a640 1 --2- 172.21.15.43:0/1149937271 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4b1415bb00 0x7f4b140c9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.374+0000 7f4b1a85a640 1 --2- 172.21.15.43:0/1149937271 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4afc003f30 0x7f4b140b6810 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.374+0000 7f4b1a85a640 1 -- 172.21.15.43:0/1149937271 >> 172.21.15.43:0/1149937271 conn(0x7f4b140d39c0 msgr2=0x7f4b140b3eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:06.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.375+0000 7f4b1a85a640 1 -- 172.21.15.43:0/1149937271 shutdown_connections 2024-04-25T09:04:06.364 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.375+0000 7f4b1a85a640 1 -- 172.21.15.43:0/1149937271 wait complete. 2024-04-25T09:04:06.375 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover' 2024-04-25T09:04:06.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.487+0000 7f3ed7cc6640 1 Processor -- start 2024-04-25T09:04:06.476 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.487+0000 7f3ed7cc6640 1 -- start start 2024-04-25T09:04:06.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.488+0000 7f3ed7cc6640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f3ed015ded0 0x7f3ed00ce740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:06.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.488+0000 7f3ed7cc6640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3ed006b240 0x7f3ed00ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:06.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.488+0000 7f3ed7cc6640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ed00cf6e0 con 0x7f3ed00cf250 2024-04-25T09:04:06.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.488+0000 7f3ed7cc6640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ed015e2b0 con 0x7f3ed006b240 2024-04-25T09:04:06.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.488+0000 7f3ed7cc6640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ed00cf9c0 con 0x7f3ed015ded0 2024-04-25T09:04:06.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.488+0000 7f3ed6263640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3ed006b240 0x7f3ed00ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:06.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.488+0000 7f3ed6263640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f3ed006b240 0x7f3ed00ad8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41538/0 (socket says 172.21.15.43:41538) 2024-04-25T09:04:06.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.488+0000 7f3ed6263640 1 -- 172.21.15.43:0/1787315485 learned_addr learned my addr 172.21.15.43:0/1787315485 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:06.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.488+0000 7f3ed5a62640 1 --2- 172.21.15.43:0/1787315485 >> v2:172.21.15.193:3300/0 conn(0x7f3ed015ded0 0x7f3ed00ce740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:06.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.488+0000 7f3ed6263640 1 -- 172.21.15.43:0/1787315485 >> v2:172.21.15.193:3300/0 conn(0x7f3ed015ded0 msgr2=0x7f3ed00ce740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:06.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.488+0000 7f3ed6263640 1 --2- 172.21.15.43:0/1787315485 >> v2:172.21.15.193:3300/0 conn(0x7f3ed015ded0 0x7f3ed00ce740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.488+0000 7f3ed6263640 1 -- 172.21.15.43:0/1787315485 >> v1:172.21.15.193:6789/0 conn(0x7f3ed00cf250 legacy=0x7f3ed006ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:06.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.488+0000 7f3ed6263640 1 -- 172.21.15.43:0/1787315485 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed00cfca0 con 0x7f3ed006b240 2024-04-25T09:04:06.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.489+0000 7f3ed6263640 1 --2- 172.21.15.43:0/1787315485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ed006b240 0x7f3ed00ad8e0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f3ed00d17e0 tx=0x7f3ec802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897db4d511e19d25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:06.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.489+0000 7f3ed4a60640 1 -- 172.21.15.43:0/1787315485 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec8037070 con 0x7f3ed006b240 2024-04-25T09:04:06.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.489+0000 7f3ed4a60640 1 -- 172.21.15.43:0/1787315485 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec8032440 con 0x7f3ed006b240 2024-04-25T09:04:06.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.489+0000 7f3ed4a60640 1 -- 172.21.15.43:0/1787315485 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec8032810 con 0x7f3ed006b240 2024-04-25T09:04:06.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.490+0000 7f3ed7cc6640 1 -- 172.21.15.43:0/1787315485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ed006b240 msgr2=0x7f3ed00ad8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:06.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.490+0000 7f3ed7cc6640 1 --2- 172.21.15.43:0/1787315485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ed006b240 0x7f3ed00ad8e0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f3ed00d17e0 tx=0x7f3ec802a270 comp rx=0 tx=0).stop 2024-04-25T09:04:06.482 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.490+0000 7f3ed7cc6640 1 -- 172.21.15.43:0/1787315485 shutdown_connections 2024-04-25T09:04:06.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.490+0000 7f3ed7cc6640 1 --2- 172.21.15.43:0/1787315485 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ed006b240 0x7f3ed00ad8e0 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.490+0000 7f3ed7cc6640 1 --2- 172.21.15.43:0/1787315485 >> v2:172.21.15.193:3300/0 conn(0x7f3ed015ded0 0x7f3ed00ce740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.490+0000 7f3ed7cc6640 1 -- 172.21.15.43:0/1787315485 >> 172.21.15.43:0/1787315485 conn(0x7f3ed00b2490 msgr2=0x7f3ed00b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:06.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.490+0000 7f3ed7cc6640 1 -- 172.21.15.43:0/1787315485 shutdown_connections 2024-04-25T09:04:06.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.490+0000 7f3ed7cc6640 1 -- 172.21.15.43:0/1787315485 wait complete. 2024-04-25T09:04:06.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.491+0000 7f3ed7cc6640 1 Processor -- start 2024-04-25T09:04:06.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.491+0000 7f3ed7cc6640 1 -- start start 2024-04-25T09:04:06.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.492+0000 7f3ed7cc6640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ed006b240 0x7f3ed015e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:06.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.492+0000 7f3ed7cc6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ed015ded0 0x7f3ed00aea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:06.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.492+0000 7f3ed7cc6640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ed00c5820 0x7f3ed00aef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:06.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.492+0000 7f3ed7cc6640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed00cefd0 con 0x7f3ed015ded0 2024-04-25T09:04:06.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.492+0000 7f3ed7cc6640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed00cee50 con 0x7f3ed006b240 2024-04-25T09:04:06.483 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.492+0000 7f3ed7cc6640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed00b4db0 con 0x7f3ed00c5820 2024-04-25T09:04:06.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.492+0000 7f3ed5261640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ed015ded0 0x7f3ed00aea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:06.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.492+0000 7f3ed5261640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ed015ded0 0x7f3ed00aea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41540/0 (socket says 172.21.15.43:41540) 2024-04-25T09:04:06.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.492+0000 7f3ed5261640 1 -- 172.21.15.43:0/2950567693 learned_addr learned my addr 172.21.15.43:0/2950567693 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:06.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.492+0000 7f3ed5a62640 1 --2- 172.21.15.43:0/2950567693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ed006b240 0x7f3ed015e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:06.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.492+0000 7f3ed6263640 1 --2- 172.21.15.43:0/2950567693 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ed00c5820 0x7f3ed00aef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:06.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.492+0000 7f3ed5261640 1 -- 172.21.15.43:0/2950567693 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ed00c5820 msgr2=0x7f3ed00aef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:06.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.492+0000 7f3ed5261640 1 --2- 172.21.15.43:0/2950567693 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ed00c5820 0x7f3ed00aef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.492+0000 7f3ed5261640 1 -- 172.21.15.43:0/2950567693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ed006b240 msgr2=0x7f3ed015e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:06.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.492+0000 7f3ed5261640 1 --2- 172.21.15.43:0/2950567693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ed006b240 0x7f3ed015e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:06.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.492+0000 7f3ed5261640 1 -- 172.21.15.43:0/2950567693 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed0160ea0 con 0x7f3ed015ded0 2024-04-25T09:04:06.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.493+0000 7f3ed5261640 1 --2- 172.21.15.43:0/2950567693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ed015ded0 0x7f3ed00aea20 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f3ec400c430 tx=0x7f3ec400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:06.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.493+0000 7f3ece7fc640 1 -- 172.21.15.43:0/2950567693 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec40052d0 con 0x7f3ed015ded0 2024-04-25T09:04:06.484 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.493+0000 7f3ece7fc640 1 -- 172.21.15.43:0/2950567693 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec4005490 con 0x7f3ed015ded0 2024-04-25T09:04:06.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.493+0000 7f3ed7cc6640 1 -- 172.21.15.43:0/2950567693 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ed0161180 con 0x7f3ed015ded0 2024-04-25T09:04:06.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.493+0000 7f3ece7fc640 1 -- 172.21.15.43:0/2950567693 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4007260 con 0x7f3ed015ded0 2024-04-25T09:04:06.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.493+0000 7f3ed7cc6640 1 -- 172.21.15.43:0/2950567693 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ed015fa40 con 0x7f3ed015ded0 2024-04-25T09:04:06.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.494+0000 7f3ed7cc6640 1 -- 172.21.15.43:0/2950567693 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ed0074ef0 con 0x7f3ed015ded0 2024-04-25T09:04:06.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.494+0000 7f3ece7fc640 1 -- 172.21.15.43:0/2950567693 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ec4007400 con 0x7f3ed015ded0 2024-04-25T09:04:06.485 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.495+0000 7f3ece7fc640 1 --2- 172.21.15.43:0/2950567693 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3ebc06b500 0x7f3ebc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:06.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.495+0000 7f3ece7fc640 1 -- 172.21.15.43:0/2950567693 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f3ec4089440 con 0x7f3ed015ded0 2024-04-25T09:04:06.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.495+0000 7f3ed5a62640 1 --2- 172.21.15.43:0/2950567693 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3ebc06b500 0x7f3ebc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:06.486 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.495+0000 7f3ed5a62640 1 --2- 172.21.15.43:0/2950567693 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3ebc06b500 0x7f3ebc06d9c0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0003aa0 tx=0x7f3ec0009830 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:06.487 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.499+0000 7f3ece7fc640 1 -- 172.21.15.43:0/2950567693 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3ec4057e70 con 0x7f3ed015ded0 2024-04-25T09:04:06.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:06.747+0000 7f3ed7cc6640 1 -- 172.21.15.43:0/2950567693 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]} v 0) v1 -- 0x7f3ed00b16e0 con 0x7f3ed015ded0 2024-04-25T09:04:07.002 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.014+0000 7f3ece7fc640 1 -- 172.21.15.43:0/2950567693 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]}]=0 v479) v1 ==== 166+0+0 (secure 0 0 0) 0x7f3ec405bf10 con 0x7f3ed015ded0 2024-04-25T09:04:07.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.016+0000 7f3ed7cc6640 1 -- 172.21.15.43:0/2950567693 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3ebc06b500 msgr2=0x7f3ebc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:07.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.016+0000 7f3ed7cc6640 1 --2- 172.21.15.43:0/2950567693 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3ebc06b500 0x7f3ebc06d9c0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f3ec0003aa0 tx=0x7f3ec0009830 comp rx=0 tx=0).stop 2024-04-25T09:04:07.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.016+0000 7f3ed7cc6640 1 -- 172.21.15.43:0/2950567693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ed015ded0 msgr2=0x7f3ed00aea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:07.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.016+0000 7f3ed7cc6640 1 --2- 172.21.15.43:0/2950567693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ed015ded0 0x7f3ed00aea20 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f3ec400c430 tx=0x7f3ec400c8f0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.016+0000 7f3ed7cc6640 1 -- 172.21.15.43:0/2950567693 shutdown_connections 2024-04-25T09:04:07.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.016+0000 7f3ed7cc6640 1 --2- 172.21.15.43:0/2950567693 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3ed00c5820 0x7f3ed00aef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.016+0000 7f3ed7cc6640 1 --2- 172.21.15.43:0/2950567693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3ed015ded0 0x7f3ed00aea20 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.016+0000 7f3ed7cc6640 1 --2- 172.21.15.43:0/2950567693 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3ebc06b500 0x7f3ebc06d9c0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.016+0000 7f3ed7cc6640 1 --2- 172.21.15.43:0/2950567693 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3ed006b240 0x7f3ed015e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.016+0000 7f3ed7cc6640 1 -- 172.21.15.43:0/2950567693 >> 172.21.15.43:0/2950567693 conn(0x7f3ed00b2490 msgr2=0x7f3ed015d4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:07.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.016+0000 7f3ed7cc6640 1 -- 172.21.15.43:0/2950567693 shutdown_connections 2024-04-25T09:04:07.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.016+0000 7f3ed7cc6640 1 -- 172.21.15.43:0/2950567693 wait complete. 2024-04-25T09:04:07.017 INFO:tasks.cephfs_test_runner:test_scrub_task_status_on_mds_failover (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T09:04:07.017 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T09:04:07.017 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:04:07.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.131+0000 7f922c891640 1 Processor -- start 2024-04-25T09:04:07.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.131+0000 7f922c891640 1 -- start start 2024-04-25T09:04:07.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.132+0000 7f922c891640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f922815def0 0x7f922806be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:07.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.132+0000 7f922c891640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f9228162560 0x7f92280b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:07.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.132+0000 7f922c891640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f922815e2d0 con 0x7f92281619f0 2024-04-25T09:04:07.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.132+0000 7f922c891640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92280d0e90 con 0x7f922815def0 2024-04-25T09:04:07.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.132+0000 7f922c891640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f922806c390 con 0x7f9228162560 2024-04-25T09:04:07.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.132+0000 7f922659c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f922815def0 0x7f922806be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:07.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.132+0000 7f921ffff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f92281619f0 0x7f9228161e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56732/0 (socket says 172.21.15.43:56732) 2024-04-25T09:04:07.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.132+0000 7f921ffff640 1 -- 172.21.15.43:0/3414062983 learned_addr learned my addr 172.21.15.43:0/3414062983 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:07.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.132+0000 7f9226d9d640 1 --2- 172.21.15.43:0/3414062983 >> v2:172.21.15.193:3300/0 conn(0x7f9228162560 0x7f92280b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:07.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.132+0000 7f922659c640 1 -- 172.21.15.43:0/3414062983 >> v2:172.21.15.193:3300/0 conn(0x7f9228162560 msgr2=0x7f92280b20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:07.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.132+0000 7f922659c640 1 --2- 172.21.15.43:0/3414062983 >> v2:172.21.15.193:3300/0 conn(0x7f9228162560 0x7f92280b20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.132+0000 7f922659c640 1 -- 172.21.15.43:0/3414062983 >> v1:172.21.15.43:6789/0 conn(0x7f92281619f0 legacy=0x7f9228161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:07.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.132+0000 7f922659c640 1 -- 172.21.15.43:0/3414062983 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92280b1020 con 0x7f922815def0 2024-04-25T09:04:07.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.133+0000 7f922659c640 1 --2- 172.21.15.43:0/3414062983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f922815def0 0x7f922806be50 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f9228162a90 tx=0x7f921802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc72ddfd00d014bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:07.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.133+0000 7f9225d9b640 1 -- 172.21.15.43:0/3414062983 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9218037070 con 0x7f922815def0 2024-04-25T09:04:07.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.133+0000 7f9225d9b640 1 -- 172.21.15.43:0/3414062983 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9218032440 con 0x7f922815def0 2024-04-25T09:04:07.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.133+0000 7f9225d9b640 1 -- 172.21.15.43:0/3414062983 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9218032810 con 0x7f922815def0 2024-04-25T09:04:07.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.134+0000 7f922c891640 1 -- 172.21.15.43:0/3414062983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f922815def0 msgr2=0x7f922806be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:07.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.134+0000 7f922c891640 1 --2- 172.21.15.43:0/3414062983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f922815def0 0x7f922806be50 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f9228162a90 tx=0x7f921802a1a0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.134+0000 7f922c891640 1 -- 172.21.15.43:0/3414062983 shutdown_connections 2024-04-25T09:04:07.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.134+0000 7f922c891640 1 --2- 172.21.15.43:0/3414062983 >> v2:172.21.15.193:3300/0 conn(0x7f9228162560 0x7f92280b20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.134+0000 7f922c891640 1 --2- 172.21.15.43:0/3414062983 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f922815def0 0x7f922806be50 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.134+0000 7f922c891640 1 -- 172.21.15.43:0/3414062983 >> 172.21.15.43:0/3414062983 conn(0x7f92280b32e0 msgr2=0x7f92280b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:07.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.134+0000 7f922c891640 1 -- 172.21.15.43:0/3414062983 shutdown_connections 2024-04-25T09:04:07.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.134+0000 7f922c891640 1 -- 172.21.15.43:0/3414062983 wait complete. 2024-04-25T09:04:07.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.135+0000 7f922c891640 1 Processor -- start 2024-04-25T09:04:07.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.135+0000 7f922c891640 1 -- start start 2024-04-25T09:04:07.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.136+0000 7f922c891640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f922815def0 0x7f92280c5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:07.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.136+0000 7f922c891640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92281619f0 0x7f922814fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:07.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.136+0000 7f922c891640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9228162560 0x7f92280b63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:07.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.136+0000 7f922c891640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92280b18c0 con 0x7f92281619f0 2024-04-25T09:04:07.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.136+0000 7f922c891640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92280b15e0 con 0x7f922815def0 2024-04-25T09:04:07.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.136+0000 7f922c891640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92280b1ba0 con 0x7f9228162560 2024-04-25T09:04:07.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.136+0000 7f921ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92281619f0 0x7f922814fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:07.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.136+0000 7f921ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92281619f0 0x7f922814fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41558/0 (socket says 172.21.15.43:41558) 2024-04-25T09:04:07.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.136+0000 7f921ffff640 1 -- 172.21.15.43:0/2092967625 learned_addr learned my addr 172.21.15.43:0/2092967625 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:07.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.136+0000 7f922659c640 1 --2- 172.21.15.43:0/2092967625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f922815def0 0x7f92280c5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:07.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.136+0000 7f9226d9d640 1 --2- 172.21.15.43:0/2092967625 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9228162560 0x7f92280b63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:07.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.136+0000 7f921ffff640 1 -- 172.21.15.43:0/2092967625 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9228162560 msgr2=0x7f92280b63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:07.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.136+0000 7f921ffff640 1 --2- 172.21.15.43:0/2092967625 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9228162560 0x7f92280b63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.136+0000 7f921ffff640 1 -- 172.21.15.43:0/2092967625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f922815def0 msgr2=0x7f92280c5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:07.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.136+0000 7f921ffff640 1 --2- 172.21.15.43:0/2092967625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f922815def0 0x7f92280c5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.136+0000 7f921ffff640 1 -- 172.21.15.43:0/2092967625 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92281617d0 con 0x7f92281619f0 2024-04-25T09:04:07.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.136+0000 7f921ffff640 1 --2- 172.21.15.43:0/2092967625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92281619f0 0x7f922814fc80 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f921000c400 tx=0x7f921000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:07.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.136+0000 7f921effd640 1 -- 172.21.15.43:0/2092967625 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9210017070 con 0x7f92281619f0 2024-04-25T09:04:07.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.137+0000 7f921effd640 1 -- 172.21.15.43:0/2092967625 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92100074e0 con 0x7f92281619f0 2024-04-25T09:04:07.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.137+0000 7f922c891640 1 -- 172.21.15.43:0/2092967625 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92280c5960 con 0x7f92281619f0 2024-04-25T09:04:07.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.137+0000 7f921effd640 1 -- 172.21.15.43:0/2092967625 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92100059a0 con 0x7f92281619f0 2024-04-25T09:04:07.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.137+0000 7f922c891640 1 -- 172.21.15.43:0/2092967625 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92280c5f20 con 0x7f92281619f0 2024-04-25T09:04:07.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.137+0000 7f922c891640 1 -- 172.21.15.43:0/2092967625 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9228074ef0 con 0x7f92281619f0 2024-04-25T09:04:07.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.138+0000 7f921effd640 1 -- 172.21.15.43:0/2092967625 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f921001a070 con 0x7f92281619f0 2024-04-25T09:04:07.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.138+0000 7f921effd640 1 --2- 172.21.15.43:0/2092967625 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f920806b4d0 0x7f920806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:07.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.139+0000 7f921effd640 1 -- 172.21.15.43:0/2092967625 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f9210089180 con 0x7f92281619f0 2024-04-25T09:04:07.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.139+0000 7f922659c640 1 --2- 172.21.15.43:0/2092967625 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f920806b4d0 0x7f920806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:07.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.139+0000 7f922659c640 1 --2- 172.21.15.43:0/2092967625 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f920806b4d0 0x7f920806d990 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f9218034000 tx=0x7f921802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:07.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.143+0000 7f921effd640 1 -- 172.21.15.43:0/2092967625 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9210057ce0 con 0x7f92281619f0 2024-04-25T09:04:07.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.322+0000 7f922c891640 1 -- 172.21.15.43:0/2092967625 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f92280b7710 con 0x7f92281619f0 2024-04-25T09:04:07.311 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.323+0000 7f921effd640 1 -- 172.21.15.43:0/2092967625 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) v1 ==== 74+0+13124 (secure 0 0 0) 0x7f921005bd80 con 0x7f92281619f0 2024-04-25T09:04:07.312 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:04:07.312 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":132,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:04:06.008968+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.43:6835/2127363183":"2024-04-26T09:04:03.003471+0000","172.21.15.43:6833/2127363183":"2024-04-26T09:04:03.003471+0000","172.21.15.43:6834/2603892772":"2024-04-26T09:03:52.443908+0000","172.21.15.43:6832/2603892772":"2024-04-26T09:03:52.443908+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:04:07.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.325+0000 7f922c891640 1 -- 172.21.15.43:0/2092967625 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f920806b4d0 msgr2=0x7f920806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:07.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.325+0000 7f922c891640 1 --2- 172.21.15.43:0/2092967625 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f920806b4d0 0x7f920806d990 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f9218034000 tx=0x7f921802e000 comp rx=0 tx=0).stop 2024-04-25T09:04:07.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.325+0000 7f922c891640 1 -- 172.21.15.43:0/2092967625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92281619f0 msgr2=0x7f922814fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:07.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.325+0000 7f922c891640 1 --2- 172.21.15.43:0/2092967625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92281619f0 0x7f922814fc80 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f921000c400 tx=0x7f921000c8c0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.325+0000 7f922c891640 1 -- 172.21.15.43:0/2092967625 shutdown_connections 2024-04-25T09:04:07.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.325+0000 7f922c891640 1 --2- 172.21.15.43:0/2092967625 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f9228162560 0x7f92280b63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.325+0000 7f922c891640 1 --2- 172.21.15.43:0/2092967625 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f92281619f0 0x7f922814fc80 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.325+0000 7f922c891640 1 --2- 172.21.15.43:0/2092967625 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f920806b4d0 0x7f920806d990 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.325+0000 7f922c891640 1 --2- 172.21.15.43:0/2092967625 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f922815def0 0x7f92280c5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.325+0000 7f922c891640 1 -- 172.21.15.43:0/2092967625 >> 172.21.15.43:0/2092967625 conn(0x7f92280b32e0 msgr2=0x7f92280ae5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:07.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.325+0000 7f922c891640 1 -- 172.21.15.43:0/2092967625 shutdown_connections 2024-04-25T09:04:07.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.325+0000 7f922c891640 1 -- 172.21.15.43:0/2092967625 wait complete. 2024-04-25T09:04:07.325 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:04:07.326 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:04:07.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.441+0000 7f706eeee640 1 Processor -- start 2024-04-25T09:04:07.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.441+0000 7f706eeee640 1 -- start start 2024-04-25T09:04:07.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.442+0000 7f706eeee640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f70680c8870 0x7f70680d2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:07.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.442+0000 7f706eeee640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70680cf030 con 0x7f70680d19f0 2024-04-25T09:04:07.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.442+0000 7f706eeee640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70680cf310 con 0x7f7068161310 2024-04-25T09:04:07.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.442+0000 7f706eeee640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70680d4870 con 0x7f70680c8870 2024-04-25T09:04:07.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.442+0000 7f706d48b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f70680c8870 0x7f70680d2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:07.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.442+0000 7f7067fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f70680d19f0 0x7f70680d1e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56744/0 (socket says 172.21.15.43:56744) 2024-04-25T09:04:07.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.442+0000 7f7067fff640 1 -- 172.21.15.43:0/2067727858 learned_addr learned my addr 172.21.15.43:0/2067727858 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:07.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.442+0000 7f706d48b640 1 -- 172.21.15.43:0/2067727858 >> v1:172.21.15.193:6789/0 conn(0x7f7068161310 legacy=0x7f70680d4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:07.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.442+0000 7f706d48b640 1 -- 172.21.15.43:0/2067727858 >> v1:172.21.15.43:6789/0 conn(0x7f70680d19f0 legacy=0x7f70680d1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:07.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.442+0000 7f706d48b640 1 -- 172.21.15.43:0/2067727858 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70680cf5f0 con 0x7f70680c8870 2024-04-25T09:04:07.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.443+0000 7f706d48b640 1 --2- 172.21.15.43:0/2067727858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70680c8870 0x7f70680d2540 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f7060009d90 tx=0x7f706002a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=677c96212ad7faef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:07.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.443+0000 7f70677fe640 1 -- 172.21.15.43:0/2067727858 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7060007740 con 0x7f70680c8870 2024-04-25T09:04:07.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.443+0000 7f70677fe640 1 -- 172.21.15.43:0/2067727858 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70600078e0 con 0x7f70680c8870 2024-04-25T09:04:07.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.443+0000 7f70677fe640 1 -- 172.21.15.43:0/2067727858 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7060007cb0 con 0x7f70680c8870 2024-04-25T09:04:07.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.444+0000 7f706eeee640 1 -- 172.21.15.43:0/2067727858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70680c8870 msgr2=0x7f70680d2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:07.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.444+0000 7f706eeee640 1 --2- 172.21.15.43:0/2067727858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70680c8870 0x7f70680d2540 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f7060009d90 tx=0x7f706002a5b0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.444+0000 7f706eeee640 1 -- 172.21.15.43:0/2067727858 shutdown_connections 2024-04-25T09:04:07.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.444+0000 7f706eeee640 1 --2- 172.21.15.43:0/2067727858 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70680c8870 0x7f70680d2540 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.444+0000 7f706eeee640 1 -- 172.21.15.43:0/2067727858 >> 172.21.15.43:0/2067727858 conn(0x7f706809cd20 msgr2=0x7f70680d3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:07.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.444+0000 7f706eeee640 1 -- 172.21.15.43:0/2067727858 shutdown_connections 2024-04-25T09:04:07.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.444+0000 7f706eeee640 1 -- 172.21.15.43:0/2067727858 wait complete. 2024-04-25T09:04:07.433 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.445+0000 7f706eeee640 1 Processor -- start 2024-04-25T09:04:07.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.445+0000 7f706eeee640 1 -- start start 2024-04-25T09:04:07.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.446+0000 7f706eeee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70680c8870 0x7f70680b8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:07.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.446+0000 7f706eeee640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f70680d19f0 0x7f706814ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:07.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.446+0000 7f706eeee640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7068161310 0x7f70680b6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:07.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.446+0000 7f706eeee640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7068168bb0 con 0x7f70680c8870 2024-04-25T09:04:07.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.446+0000 7f706eeee640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7068168a30 con 0x7f7068161310 2024-04-25T09:04:07.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.446+0000 7f706eeee640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7068168d30 con 0x7f70680d19f0 2024-04-25T09:04:07.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.446+0000 7f706cc8a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70680c8870 0x7f70680b8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:07.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.446+0000 7f706cc8a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70680c8870 0x7f70680b8c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41574/0 (socket says 172.21.15.43:41574) 2024-04-25T09:04:07.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.446+0000 7f706cc8a640 1 -- 172.21.15.43:0/501101933 learned_addr learned my addr 172.21.15.43:0/501101933 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:07.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.446+0000 7f706d48b640 1 --2- 172.21.15.43:0/501101933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7068161310 0x7f70680b6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:07.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.446+0000 7f706cc8a640 1 -- 172.21.15.43:0/501101933 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f70680d19f0 msgr2=0x7f706814ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:07.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.446+0000 7f706cc8a640 1 --2- 172.21.15.43:0/501101933 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f70680d19f0 0x7f706814ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.446+0000 7f706cc8a640 1 -- 172.21.15.43:0/501101933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7068161310 msgr2=0x7f70680b6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:07.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.446+0000 7f706cc8a640 1 --2- 172.21.15.43:0/501101933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7068161310 0x7f70680b6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.446+0000 7f706cc8a640 1 -- 172.21.15.43:0/501101933 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7068153600 con 0x7f70680c8870 2024-04-25T09:04:07.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.446+0000 7f706cc8a640 1 --2- 172.21.15.43:0/501101933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70680c8870 0x7f70680b8c80 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f705c00c430 tx=0x7f705c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:07.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.446+0000 7f70657fa640 1 -- 172.21.15.43:0/501101933 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705c011790 con 0x7f70680c8870 2024-04-25T09:04:07.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.446+0000 7f70657fa640 1 -- 172.21.15.43:0/501101933 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705c017070 con 0x7f70680c8870 2024-04-25T09:04:07.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.446+0000 7f706eeee640 1 -- 172.21.15.43:0/501101933 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f706814f2d0 con 0x7f70680c8870 2024-04-25T09:04:07.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.446+0000 7f70657fa640 1 -- 172.21.15.43:0/501101933 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705c011bb0 con 0x7f70680c8870 2024-04-25T09:04:07.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.446+0000 7f706eeee640 1 -- 172.21.15.43:0/501101933 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7068153870 con 0x7f70680c8870 2024-04-25T09:04:07.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.447+0000 7f706eeee640 1 -- 172.21.15.43:0/501101933 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7068074f60 con 0x7f70680c8870 2024-04-25T09:04:07.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.448+0000 7f70657fa640 1 -- 172.21.15.43:0/501101933 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f705c01c460 con 0x7f70680c8870 2024-04-25T09:04:07.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.448+0000 7f70657fa640 1 --2- 172.21.15.43:0/501101933 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f704c06b500 0x7f704c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:07.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.449+0000 7f70657fa640 1 -- 172.21.15.43:0/501101933 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f705c089df0 con 0x7f70680c8870 2024-04-25T09:04:07.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.449+0000 7f7067fff640 1 --2- 172.21.15.43:0/501101933 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f704c06b500 0x7f704c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:07.438 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.449+0000 7f7067fff640 1 --2- 172.21.15.43:0/501101933 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f704c06b500 0x7f704c06d9c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f7054007fd0 tx=0x7f70540079a0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:07.441 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.453+0000 7f70657fa640 1 -- 172.21.15.43:0/501101933 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f705c058f10 con 0x7f70680c8870 2024-04-25T09:04:07.623 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.635+0000 7f706eeee640 1 -- 172.21.15.43:0/501101933 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f70681585e0 con 0x7f70680c8870 2024-04-25T09:04:07.624 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.635+0000 7f70657fa640 1 -- 172.21.15.43:0/501101933 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v132) v1 ==== 93+0+10 (secure 0 0 0) 0x7f705c058810 con 0x7f70680c8870 2024-04-25T09:04:07.624 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T09:04:07.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.637+0000 7f706eeee640 1 -- 172.21.15.43:0/501101933 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f704c06b500 msgr2=0x7f704c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:07.626 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.637+0000 7f706eeee640 1 --2- 172.21.15.43:0/501101933 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f704c06b500 0x7f704c06d9c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f7054007fd0 tx=0x7f70540079a0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.637+0000 7f706eeee640 1 -- 172.21.15.43:0/501101933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70680c8870 msgr2=0x7f70680b8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:07.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.637+0000 7f706eeee640 1 --2- 172.21.15.43:0/501101933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70680c8870 0x7f70680b8c80 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f705c00c430 tx=0x7f705c00c8f0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.638+0000 7f706eeee640 1 -- 172.21.15.43:0/501101933 shutdown_connections 2024-04-25T09:04:07.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.638+0000 7f706eeee640 1 --2- 172.21.15.43:0/501101933 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f704c06b500 0x7f704c06d9c0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.638+0000 7f706eeee640 1 --2- 172.21.15.43:0/501101933 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7068161310 0x7f70680b6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.638+0000 7f706eeee640 1 --2- 172.21.15.43:0/501101933 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f70680d19f0 0x7f706814ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.638+0000 7f706eeee640 1 --2- 172.21.15.43:0/501101933 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f70680c8870 0x7f70680b8c80 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.638+0000 7f706eeee640 1 -- 172.21.15.43:0/501101933 >> 172.21.15.43:0/501101933 conn(0x7f706809cd20 msgr2=0x7f70680cf9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:07.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.638+0000 7f706eeee640 1 -- 172.21.15.43:0/501101933 shutdown_connections 2024-04-25T09:04:07.627 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.638+0000 7f706eeee640 1 -- 172.21.15.43:0/501101933 wait complete. 2024-04-25T09:04:07.638 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive' 2024-04-25T09:04:07.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.751+0000 7faea89d5640 1 Processor -- start 2024-04-25T09:04:07.740 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.751+0000 7faea89d5640 1 -- start start 2024-04-25T09:04:07.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.752+0000 7faea89d5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faea415dcf0 0x7faea406e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:07.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.752+0000 7faea89d5640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7faea406d4f0 0x7faea406f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:07.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.752+0000 7faea89d5640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faea40d0d80 con 0x7faea406ebf0 2024-04-25T09:04:07.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.752+0000 7faea89d5640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faea406f590 con 0x7faea406d4f0 2024-04-25T09:04:07.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.752+0000 7faea89d5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faea406c050 con 0x7faea415dcf0 2024-04-25T09:04:07.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.752+0000 7faea1d9b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7faea406ebf0 0x7faea41616e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56754/0 (socket says 172.21.15.43:56754) 2024-04-25T09:04:07.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.752+0000 7faea1d9b640 1 -- 172.21.15.43:0/1401706327 learned_addr learned my addr 172.21.15.43:0/1401706327 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:07.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.753+0000 7faea259c640 1 --2- 172.21.15.43:0/1401706327 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faea415dcf0 0x7faea406e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:07.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.753+0000 7faea2d9d640 1 --2- 172.21.15.43:0/1401706327 >> v2:172.21.15.193:3300/0 conn(0x7faea406d4f0 0x7faea406f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:07.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.752+0000 7faea159a640 1 -- 172.21.15.43:0/1401706327 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2267528662 0 0) 0x7faea40d0d80 con 0x7faea406ebf0 2024-04-25T09:04:07.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.753+0000 7faea159a640 1 -- 172.21.15.43:0/1401706327 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae8c003520 con 0x7faea406ebf0 2024-04-25T09:04:07.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.753+0000 7faea159a640 1 -- 172.21.15.43:0/1401706327 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1034961015 0 0) 0x7fae8c003520 con 0x7faea406ebf0 2024-04-25T09:04:07.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.753+0000 7faea159a640 1 -- 172.21.15.43:0/1401706327 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faea415dcf0 msgr2=0x7faea406e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:07.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.753+0000 7faea159a640 1 --2- 172.21.15.43:0/1401706327 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faea415dcf0 0x7faea406e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.753+0000 7faea159a640 1 -- 172.21.15.43:0/1401706327 >> v2:172.21.15.193:3300/0 conn(0x7faea406d4f0 msgr2=0x7faea406f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:07.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.753+0000 7faea159a640 1 --2- 172.21.15.43:0/1401706327 >> v2:172.21.15.193:3300/0 conn(0x7faea406d4f0 0x7faea406f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.753+0000 7faea159a640 1 -- 172.21.15.43:0/1401706327 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea40d04e0 con 0x7faea406ebf0 2024-04-25T09:04:07.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.753+0000 7faea159a640 1 -- 172.21.15.43:0/1401706327 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7fae98001670 con 0x7faea406ebf0 2024-04-25T09:04:07.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.753+0000 7faea159a640 1 -- 172.21.15.43:0/1401706327 >> v1:172.21.15.43:6789/0 conn(0x7faea406ebf0 legacy=0x7faea41616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:07.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.753+0000 7faea159a640 1 --2- 172.21.15.43:0/1401706327 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faea415dcf0 0x7fae8c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:07.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.753+0000 7faea159a640 1 --2- 172.21.15.43:0/1401706327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae8c003f30 0x7fae8c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:07.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.753+0000 7faea159a640 1 --2- 172.21.15.43:0/1401706327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae8c004d40 0x7fae8c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:07.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.753+0000 7faea159a640 1 -- 172.21.15.43:0/1401706327 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea406c050 con 0x7fae8c003f30 2024-04-25T09:04:07.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.753+0000 7faea159a640 1 -- 172.21.15.43:0/1401706327 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8c00b9c0 con 0x7fae8c004d40 2024-04-25T09:04:07.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.753+0000 7faea159a640 1 -- 172.21.15.43:0/1401706327 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae8c00bb90 con 0x7faea415dcf0 2024-04-25T09:04:07.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.753+0000 7faea1d9b640 1 --2- 172.21.15.43:0/1401706327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae8c003f30 0x7fae8c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:07.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.753+0000 7faea259c640 1 --2- 172.21.15.43:0/1401706327 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faea415dcf0 0x7fae8c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:07.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.753+0000 7faea2d9d640 1 --2- 172.21.15.43:0/1401706327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae8c004d40 0x7fae8c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:07.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.753+0000 7faea1d9b640 1 -- 172.21.15.43:0/1401706327 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faea415dcf0 msgr2=0x7fae8c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:07.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.753+0000 7faea1d9b640 1 --2- 172.21.15.43:0/1401706327 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faea415dcf0 0x7fae8c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.753+0000 7faea1d9b640 1 -- 172.21.15.43:0/1401706327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae8c004d40 msgr2=0x7fae8c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:07.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.753+0000 7faea1d9b640 1 --2- 172.21.15.43:0/1401706327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae8c004d40 0x7fae8c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.753+0000 7faea1d9b640 1 -- 172.21.15.43:0/1401706327 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae8c003520 con 0x7fae8c003f30 2024-04-25T09:04:07.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.753+0000 7faea2d9d640 1 --2- 172.21.15.43:0/1401706327 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fae8c004d40 0x7fae8c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:07.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.754+0000 7faea1d9b640 1 --2- 172.21.15.43:0/1401706327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae8c003f30 0x7fae8c004800 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fae9800c270 tx=0x7fae8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d7f3a47c1cf781 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:07.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.754+0000 7faea159a640 1 -- 172.21.15.43:0/1401706327 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae98007d30 con 0x7fae8c003f30 2024-04-25T09:04:07.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.754+0000 7faea159a640 1 -- 172.21.15.43:0/1401706327 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae98005ad0 con 0x7fae8c003f30 2024-04-25T09:04:07.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.754+0000 7faea159a640 1 -- 172.21.15.43:0/1401706327 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae98007070 con 0x7fae8c003f30 2024-04-25T09:04:07.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.755+0000 7faea89d5640 1 -- 172.21.15.43:0/1401706327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae8c003f30 msgr2=0x7fae8c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:07.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.755+0000 7faea89d5640 1 --2- 172.21.15.43:0/1401706327 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae8c003f30 0x7fae8c004800 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fae9800c270 tx=0x7fae8c005830 comp rx=0 tx=0).stop 2024-04-25T09:04:07.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.755+0000 7faea2d9d640 1 -- 172.21.15.43:0/1401706327 reap_dead start 2024-04-25T09:04:07.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.755+0000 7faea89d5640 1 -- 172.21.15.43:0/1401706327 shutdown_connections 2024-04-25T09:04:07.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.755+0000 7faea89d5640 1 -- 172.21.15.43:0/1401706327 >> 172.21.15.43:0/1401706327 conn(0x7faea40d1700 msgr2=0x7faea40d1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:07.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.755+0000 7faea89d5640 1 -- 172.21.15.43:0/1401706327 shutdown_connections 2024-04-25T09:04:07.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.755+0000 7faea89d5640 1 -- 172.21.15.43:0/1401706327 wait complete. 2024-04-25T09:04:07.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.756+0000 7faea89d5640 1 Processor -- start 2024-04-25T09:04:07.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.756+0000 7faea89d5640 1 -- start start 2024-04-25T09:04:07.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.757+0000 7faea89d5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae8c003f30 0x7faea415b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:07.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.757+0000 7faea89d5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faea409b5a0 0x7faea40c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:07.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.757+0000 7faea89d5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faea409cad0 0x7faea409d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:07.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.757+0000 7faea89d5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea406bce0 con 0x7fae8c003f30 2024-04-25T09:04:07.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.757+0000 7faea89d5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea40d0aa0 con 0x7faea409cad0 2024-04-25T09:04:07.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.757+0000 7faea89d5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faea40b0d50 con 0x7faea409b5a0 2024-04-25T09:04:07.749 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.757+0000 7faea259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae8c003f30 0x7faea415b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:07.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.757+0000 7faea259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae8c003f30 0x7faea415b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41596/0 (socket says 172.21.15.43:41596) 2024-04-25T09:04:07.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.757+0000 7faea259c640 1 -- 172.21.15.43:0/1077117179 learned_addr learned my addr 172.21.15.43:0/1077117179 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:07.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.757+0000 7faea1d9b640 1 --2- 172.21.15.43:0/1077117179 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faea409b5a0 0x7faea40c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:07.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.757+0000 7faea2d9d640 1 --2- 172.21.15.43:0/1077117179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faea409cad0 0x7faea409d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:07.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.757+0000 7faea259c640 1 -- 172.21.15.43:0/1077117179 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faea409b5a0 msgr2=0x7faea40c7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:07.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.757+0000 7faea259c640 1 --2- 172.21.15.43:0/1077117179 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faea409b5a0 0x7faea40c7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.757+0000 7faea259c640 1 -- 172.21.15.43:0/1077117179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faea409cad0 msgr2=0x7faea409d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:07.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.757+0000 7faea259c640 1 --2- 172.21.15.43:0/1077117179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faea409cad0 0x7faea409d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:07.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.757+0000 7faea259c640 1 -- 172.21.15.43:0/1077117179 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea409ba30 con 0x7fae8c003f30 2024-04-25T09:04:07.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.758+0000 7faea259c640 1 --2- 172.21.15.43:0/1077117179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae8c003f30 0x7faea415b030 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fae940119a0 tx=0x7fae94011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:07.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.758+0000 7fae92ffd640 1 -- 172.21.15.43:0/1077117179 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9400faf0 con 0x7fae8c003f30 2024-04-25T09:04:07.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.758+0000 7fae92ffd640 1 -- 172.21.15.43:0/1077117179 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae9400fc90 con 0x7fae8c003f30 2024-04-25T09:04:07.750 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.758+0000 7fae92ffd640 1 -- 172.21.15.43:0/1077117179 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae94014440 con 0x7fae8c003f30 2024-04-25T09:04:07.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.758+0000 7faea89d5640 1 -- 172.21.15.43:0/1077117179 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faea409bd10 con 0x7fae8c003f30 2024-04-25T09:04:07.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.758+0000 7faea89d5640 1 -- 172.21.15.43:0/1077117179 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faea409c2d0 con 0x7fae8c003f30 2024-04-25T09:04:07.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.759+0000 7faea89d5640 1 -- 172.21.15.43:0/1077117179 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faea40d2610 con 0x7fae8c003f30 2024-04-25T09:04:07.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.759+0000 7fae92ffd640 1 -- 172.21.15.43:0/1077117179 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae94012040 con 0x7fae8c003f30 2024-04-25T09:04:07.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.760+0000 7fae92ffd640 1 --2- 172.21.15.43:0/1077117179 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fae8406b4d0 0x7fae8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:07.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.760+0000 7fae92ffd640 1 -- 172.21.15.43:0/1077117179 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fae9408a600 con 0x7fae8c003f30 2024-04-25T09:04:07.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.760+0000 7faea1d9b640 1 --2- 172.21.15.43:0/1077117179 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fae8406b4d0 0x7fae8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:07.751 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.760+0000 7faea1d9b640 1 --2- 172.21.15.43:0/1077117179 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fae8406b4d0 0x7fae8406d990 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fae98033000 tx=0x7fae98009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:07.752 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:07.764+0000 7fae92ffd640 1 -- 172.21.15.43:0/1077117179 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faea40d2610 con 0x7fae8c003f30 2024-04-25T09:04:08.001 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.012+0000 7faea89d5640 1 -- 172.21.15.43:0/1077117179 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]} v 0) v1 -- 0x7faea4074ef0 con 0x7fae8c003f30 2024-04-25T09:04:08.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.019+0000 7fae92ffd640 1 -- 172.21.15.43:0/1077117179 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]}]=0 v480) v1 ==== 162+0+0 (secure 0 0 0) 0x7fae94019030 con 0x7fae8c003f30 2024-04-25T09:04:08.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.021+0000 7faea89d5640 1 -- 172.21.15.43:0/1077117179 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fae8406b4d0 msgr2=0x7fae8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.021+0000 7faea89d5640 1 --2- 172.21.15.43:0/1077117179 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fae8406b4d0 0x7fae8406d990 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fae98033000 tx=0x7fae98009030 comp rx=0 tx=0).stop 2024-04-25T09:04:08.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.021+0000 7faea89d5640 1 -- 172.21.15.43:0/1077117179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae8c003f30 msgr2=0x7faea415b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.021+0000 7faea89d5640 1 --2- 172.21.15.43:0/1077117179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae8c003f30 0x7faea415b030 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fae940119a0 tx=0x7fae94011e60 comp rx=0 tx=0).stop 2024-04-25T09:04:08.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.021+0000 7faea89d5640 1 -- 172.21.15.43:0/1077117179 shutdown_connections 2024-04-25T09:04:08.010 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.021+0000 7faea89d5640 1 --2- 172.21.15.43:0/1077117179 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fae8406b4d0 0x7fae8406d990 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.021+0000 7faea89d5640 1 --2- 172.21.15.43:0/1077117179 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7faea409cad0 0x7faea409d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.021+0000 7faea89d5640 1 --2- 172.21.15.43:0/1077117179 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7faea409b5a0 0x7faea40c7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.021+0000 7faea89d5640 1 --2- 172.21.15.43:0/1077117179 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fae8c003f30 0x7faea415b030 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.021+0000 7faea89d5640 1 -- 172.21.15.43:0/1077117179 >> 172.21.15.43:0/1077117179 conn(0x7faea40d1700 msgr2=0x7faea415cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:08.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.021+0000 7faea89d5640 1 -- 172.21.15.43:0/1077117179 shutdown_connections 2024-04-25T09:04:08.011 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.021+0000 7faea89d5640 1 -- 172.21.15.43:0/1077117179 wait complete. 2024-04-25T09:04:08.021 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T09:04:08.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.137+0000 7f5907bb0640 1 Processor -- start 2024-04-25T09:04:08.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.137+0000 7f5907bb0640 1 -- start start 2024-04-25T09:04:08.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.138+0000 7f5907bb0640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f590006f6a0 0x7f59000b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.138+0000 7f5907bb0640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59000d2620 con 0x7f59000cd470 2024-04-25T09:04:08.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.138+0000 7f5907bb0640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59000b0b80 con 0x7f590015ddd0 2024-04-25T09:04:08.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.138+0000 7f5907bb0640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59000b0e60 con 0x7f590006f6a0 2024-04-25T09:04:08.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.138+0000 7f590614d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f59000cd470 0x7f590006c4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56764/0 (socket says 172.21.15.43:56764) 2024-04-25T09:04:08.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.138+0000 7f590614d640 1 -- 172.21.15.43:0/3733550317 learned_addr learned my addr 172.21.15.43:0/3733550317 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:08.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.138+0000 7f590514b640 1 --2- 172.21.15.43:0/3733550317 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f590006f6a0 0x7f59000b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.138+0000 7f590494a640 1 -- 172.21.15.43:0/3733550317 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 157948997 0 0) 0x7f59000d2620 con 0x7f59000cd470 2024-04-25T09:04:08.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.138+0000 7f590494a640 1 -- 172.21.15.43:0/3733550317 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58ec003520 con 0x7f59000cd470 2024-04-25T09:04:08.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.138+0000 7f590494a640 1 -- 172.21.15.43:0/3733550317 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1126827075 0 0) 0x7f59000b0b80 con 0x7f590015ddd0 2024-04-25T09:04:08.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.138+0000 7f590494a640 1 -- 172.21.15.43:0/3733550317 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59000d2620 con 0x7f590015ddd0 2024-04-25T09:04:08.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.139+0000 7f590494a640 1 -- 172.21.15.43:0/3733550317 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1401565423 0 0) 0x7f58ec003520 con 0x7f59000cd470 2024-04-25T09:04:08.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.139+0000 7f590494a640 1 -- 172.21.15.43:0/3733550317 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f590006f6a0 msgr2=0x7f59000b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:08.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.139+0000 7f590494a640 1 --2- 172.21.15.43:0/3733550317 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f590006f6a0 0x7f59000b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.139+0000 7f590494a640 1 -- 172.21.15.43:0/3733550317 >> v1:172.21.15.193:6789/0 conn(0x7f590015ddd0 legacy=0x7f590006ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.139+0000 7f590494a640 1 -- 172.21.15.43:0/3733550317 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59000cbf40 con 0x7f59000cd470 2024-04-25T09:04:08.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.139+0000 7f590494a640 1 -- 172.21.15.43:0/3733550317 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f58f80019f0 con 0x7f59000cd470 2024-04-25T09:04:08.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.139+0000 7f590494a640 1 -- 172.21.15.43:0/3733550317 >> v1:172.21.15.43:6789/0 conn(0x7f59000cd470 legacy=0x7f590006c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.139+0000 7f590494a640 1 --2- 172.21.15.43:0/3733550317 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f590006f6a0 0x7f58ec003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.139+0000 7f590494a640 1 --2- 172.21.15.43:0/3733550317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58ec0040d0 0x7f58ec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.139+0000 7f590494a640 1 --2- 172.21.15.43:0/3733550317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58ec004d40 0x7f58ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.139+0000 7f590494a640 1 -- 172.21.15.43:0/3733550317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59000b0e60 con 0x7f58ec004d40 2024-04-25T09:04:08.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.139+0000 7f590494a640 1 -- 172.21.15.43:0/3733550317 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec00bb50 con 0x7f58ec0040d0 2024-04-25T09:04:08.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.139+0000 7f590494a640 1 -- 172.21.15.43:0/3733550317 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58ec00bd20 con 0x7f590006f6a0 2024-04-25T09:04:08.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.139+0000 7f590614d640 1 --2- 172.21.15.43:0/3733550317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58ec004d40 0x7f58ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.139+0000 7f590594c640 1 --2- 172.21.15.43:0/3733550317 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f590006f6a0 0x7f58ec003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.139+0000 7f590514b640 1 --2- 172.21.15.43:0/3733550317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58ec0040d0 0x7f58ec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.139+0000 7f590614d640 1 -- 172.21.15.43:0/3733550317 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f590006f6a0 msgr2=0x7f58ec003b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:08.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.139+0000 7f590614d640 1 --2- 172.21.15.43:0/3733550317 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f590006f6a0 0x7f58ec003b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.139+0000 7f590614d640 1 -- 172.21.15.43:0/3733550317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58ec0040d0 msgr2=0x7f58ec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:08.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.139+0000 7f590614d640 1 --2- 172.21.15.43:0/3733550317 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f58ec0040d0 0x7f58ec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.130 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.139+0000 7f590614d640 1 -- 172.21.15.43:0/3733550317 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ec003520 con 0x7f58ec004d40 2024-04-25T09:04:08.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.140+0000 7f590614d640 1 --2- 172.21.15.43:0/3733550317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58ec004d40 0x7f58ec0051c0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f58f800a800 tx=0x7f58ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c6faccfddb8d022 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:08.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.140+0000 7f590494a640 1 -- 172.21.15.43:0/3733550317 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f800a430 con 0x7f58ec004d40 2024-04-25T09:04:08.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.140+0000 7f590494a640 1 -- 172.21.15.43:0/3733550317 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f802f070 con 0x7f58ec004d40 2024-04-25T09:04:08.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.140+0000 7f590494a640 1 -- 172.21.15.43:0/3733550317 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f802bc90 con 0x7f58ec004d40 2024-04-25T09:04:08.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.140+0000 7f5907bb0640 1 -- 172.21.15.43:0/3733550317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58ec004d40 msgr2=0x7f58ec0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.140+0000 7f5907bb0640 1 --2- 172.21.15.43:0/3733550317 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f58ec004d40 0x7f58ec0051c0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f58f800a800 tx=0x7f58ec005830 comp rx=0 tx=0).stop 2024-04-25T09:04:08.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.140+0000 7f590614d640 1 -- 172.21.15.43:0/3733550317 reap_dead start 2024-04-25T09:04:08.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.141+0000 7f5907bb0640 1 -- 172.21.15.43:0/3733550317 shutdown_connections 2024-04-25T09:04:08.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.141+0000 7f5907bb0640 1 -- 172.21.15.43:0/3733550317 >> 172.21.15.43:0/3733550317 conn(0x7f59000d17e0 msgr2=0x7f59000d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:08.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.141+0000 7f5907bb0640 1 -- 172.21.15.43:0/3733550317 shutdown_connections 2024-04-25T09:04:08.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.141+0000 7f5907bb0640 1 -- 172.21.15.43:0/3733550317 wait complete. 2024-04-25T09:04:08.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.142+0000 7f5907bb0640 1 Processor -- start 2024-04-25T09:04:08.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.142+0000 7f5907bb0640 1 -- start start 2024-04-25T09:04:08.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.142+0000 7f5907bb0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59000c97f0 0x7f59000cb300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.142+0000 7f5907bb0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59000c9bd0 0x7f59000c8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.134 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.143+0000 7f5907bb0640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f59000af0a0 0x7f59000af520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.143+0000 7f5907bb0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59000b41a0 con 0x7f59000c9bd0 2024-04-25T09:04:08.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.143+0000 7f5907bb0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59000b4020 con 0x7f59000c97f0 2024-04-25T09:04:08.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.143+0000 7f5907bb0640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f590006cbf0 con 0x7f59000af0a0 2024-04-25T09:04:08.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.143+0000 7f590514b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59000c9bd0 0x7f59000c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.143+0000 7f590514b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59000c9bd0 0x7f59000c8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41606/0 (socket says 172.21.15.43:41606) 2024-04-25T09:04:08.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.143+0000 7f590514b640 1 -- 172.21.15.43:0/1925208138 learned_addr learned my addr 172.21.15.43:0/1925208138 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:08.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.143+0000 7f590614d640 1 --2- 172.21.15.43:0/1925208138 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f59000af0a0 0x7f59000af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.143+0000 7f590594c640 1 --2- 172.21.15.43:0/1925208138 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59000c97f0 0x7f59000cb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.143+0000 7f590514b640 1 -- 172.21.15.43:0/1925208138 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f59000af0a0 msgr2=0x7f59000af520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.143+0000 7f590514b640 1 --2- 172.21.15.43:0/1925208138 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f59000af0a0 0x7f59000af520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.143+0000 7f590514b640 1 -- 172.21.15.43:0/1925208138 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59000c97f0 msgr2=0x7f59000cb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.143+0000 7f590514b640 1 --2- 172.21.15.43:0/1925208138 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59000c97f0 0x7f59000cb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.143+0000 7f590514b640 1 -- 172.21.15.43:0/1925208138 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59001f6f40 con 0x7f59000c9bd0 2024-04-25T09:04:08.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.143+0000 7f590614d640 1 --2- 172.21.15.43:0/1925208138 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f59000af0a0 0x7f59000af520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:08.135 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.143+0000 7f590514b640 1 --2- 172.21.15.43:0/1925208138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59000c9bd0 0x7f59000c8820 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f58f00119d0 tx=0x7f58f0011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:08.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.143+0000 7f58f67fc640 1 -- 172.21.15.43:0/1925208138 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f000faf0 con 0x7f59000c9bd0 2024-04-25T09:04:08.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.143+0000 7f58f67fc640 1 -- 172.21.15.43:0/1925208138 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f000fc90 con 0x7f59000c9bd0 2024-04-25T09:04:08.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.143+0000 7f5907bb0640 1 -- 172.21.15.43:0/1925208138 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59001f7220 con 0x7f59000c9bd0 2024-04-25T09:04:08.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.143+0000 7f58f67fc640 1 -- 172.21.15.43:0/1925208138 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f0014440 con 0x7f59000c9bd0 2024-04-25T09:04:08.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.143+0000 7f5907bb0640 1 -- 172.21.15.43:0/1925208138 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59001f77e0 con 0x7f59000c9bd0 2024-04-25T09:04:08.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.144+0000 7f5907bb0640 1 -- 172.21.15.43:0/1925208138 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5900074f60 con 0x7f59000c9bd0 2024-04-25T09:04:08.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.145+0000 7f58f67fc640 1 -- 172.21.15.43:0/1925208138 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f58f0012040 con 0x7f59000c9bd0 2024-04-25T09:04:08.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.145+0000 7f58f67fc640 1 --2- 172.21.15.43:0/1925208138 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f58e406b500 0x7f58e406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.145+0000 7f58f67fc640 1 -- 172.21.15.43:0/1925208138 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f58f008a550 con 0x7f59000c9bd0 2024-04-25T09:04:08.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.146+0000 7f590594c640 1 --2- 172.21.15.43:0/1925208138 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f58e406b500 0x7f58e406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.136 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.146+0000 7f590594c640 1 --2- 172.21.15.43:0/1925208138 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f58e406b500 0x7f58e406d9c0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f58fc005570 tx=0x7f58fc009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:08.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.150+0000 7f58f67fc640 1 -- 172.21.15.43:0/1925208138 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f58f0019030 con 0x7f59000c9bd0 2024-04-25T09:04:08.317 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.328+0000 7f5907bb0640 1 -- 172.21.15.43:0/1925208138 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f59000d1e40 con 0x7f59000c9bd0 2024-04-25T09:04:08.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.331+0000 7f58f67fc640 1 -- 172.21.15.43:0/1925208138 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7f58f0059670 con 0x7f59000c9bd0 2024-04-25T09:04:08.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.333+0000 7f5907bb0640 1 -- 172.21.15.43:0/1925208138 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f58e406b500 msgr2=0x7f58e406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.333+0000 7f5907bb0640 1 --2- 172.21.15.43:0/1925208138 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f58e406b500 0x7f58e406d9c0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f58fc005570 tx=0x7f58fc009040 comp rx=0 tx=0).stop 2024-04-25T09:04:08.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.333+0000 7f5907bb0640 1 -- 172.21.15.43:0/1925208138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59000c9bd0 msgr2=0x7f59000c8820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.333+0000 7f5907bb0640 1 --2- 172.21.15.43:0/1925208138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59000c9bd0 0x7f59000c8820 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f58f00119d0 tx=0x7f58f0011e90 comp rx=0 tx=0).stop 2024-04-25T09:04:08.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.333+0000 7f5907bb0640 1 -- 172.21.15.43:0/1925208138 shutdown_connections 2024-04-25T09:04:08.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.333+0000 7f5907bb0640 1 --2- 172.21.15.43:0/1925208138 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f59000af0a0 0x7f59000af520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.333+0000 7f5907bb0640 1 --2- 172.21.15.43:0/1925208138 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f59000c9bd0 0x7f59000c8820 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.333+0000 7f5907bb0640 1 --2- 172.21.15.43:0/1925208138 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f58e406b500 0x7f58e406d9c0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.333+0000 7f5907bb0640 1 --2- 172.21.15.43:0/1925208138 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f59000c97f0 0x7f59000cb300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.334+0000 7f5907bb0640 1 -- 172.21.15.43:0/1925208138 >> 172.21.15.43:0/1925208138 conn(0x7f59000d17e0 msgr2=0x7f59000afee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:08.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.334+0000 7f5907bb0640 1 -- 172.21.15.43:0/1925208138 shutdown_connections 2024-04-25T09:04:08.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.334+0000 7f5907bb0640 1 -- 172.21.15.43:0/1925208138 wait complete. 2024-04-25T09:04:08.333 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:04:08.334 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:04:08.378 DEBUG:teuthology.orchestra.run.smithi193:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:04:08.435 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:04:08.435 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T09:04:08.435 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:04:08.451 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2374 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:04:08.451 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T09:04:08.451 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:04:08.469 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19508 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:04:08.469 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T09:04:08.469 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:04:08.506 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2374 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:04:08.507 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T09:04:08.507 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:04:08.527 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19508 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:04:08.527 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T09:04:08.527 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:04:08.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.658+0000 7fb676770640 1 Processor -- start 2024-04-25T09:04:08.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.658+0000 7fb676770640 1 -- start start 2024-04-25T09:04:08.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.659+0000 7fb676770640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb67015ef50 0x7fb6700d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.659+0000 7fb676770640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6700d0d30 con 0x7fb6700d0920 2024-04-25T09:04:08.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.659+0000 7fb676770640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6700d2920 con 0x7fb67015dce0 2024-04-25T09:04:08.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.659+0000 7fb676770640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6700cec60 con 0x7fb67015ef50 2024-04-25T09:04:08.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.659+0000 7fb66f7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb67015ef50 0x7fb6700d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.659+0000 7fb674d0d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb6700d0920 0x7fb670160980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56770/0 (socket says 172.21.15.43:56770) 2024-04-25T09:04:08.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.659+0000 7fb674d0d640 1 -- 172.21.15.43:0/3892316598 learned_addr learned my addr 172.21.15.43:0/3892316598 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:08.648 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.659+0000 7fb66f7fe640 1 --2- 172.21.15.43:0/3892316598 >> v2:172.21.15.43:3300/0 conn(0x7fb67015ef50 0x7fb6700d03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41620/0 (socket says 172.21.15.43:41620) 2024-04-25T09:04:08.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.660+0000 7fb66f7fe640 1 -- 172.21.15.43:0/3892316598 >> v1:172.21.15.193:6789/0 conn(0x7fb67015dce0 legacy=0x7fb6700ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:08.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.660+0000 7fb66f7fe640 1 -- 172.21.15.43:0/3892316598 >> v1:172.21.15.43:6789/0 conn(0x7fb6700d0920 legacy=0x7fb670160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.660+0000 7fb66f7fe640 1 -- 172.21.15.43:0/3892316598 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6700cee80 con 0x7fb67015ef50 2024-04-25T09:04:08.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.660+0000 7fb66f7fe640 1 --2- 172.21.15.43:0/3892316598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb67015ef50 0x7fb6700d03e0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fb6600097a0 tx=0x7fb66002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aedcaeb479339e3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:08.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.660+0000 7fb66effd640 1 -- 172.21.15.43:0/3892316598 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb660037070 con 0x7fb67015ef50 2024-04-25T09:04:08.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.660+0000 7fb66effd640 1 -- 172.21.15.43:0/3892316598 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb66002ae50 con 0x7fb67015ef50 2024-04-25T09:04:08.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.660+0000 7fb66effd640 1 -- 172.21.15.43:0/3892316598 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb660032660 con 0x7fb67015ef50 2024-04-25T09:04:08.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.661+0000 7fb676770640 1 -- 172.21.15.43:0/3892316598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb67015ef50 msgr2=0x7fb6700d03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.661+0000 7fb676770640 1 --2- 172.21.15.43:0/3892316598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb67015ef50 0x7fb6700d03e0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fb6600097a0 tx=0x7fb66002a140 comp rx=0 tx=0).stop 2024-04-25T09:04:08.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.661+0000 7fb676770640 1 -- 172.21.15.43:0/3892316598 shutdown_connections 2024-04-25T09:04:08.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.661+0000 7fb676770640 1 --2- 172.21.15.43:0/3892316598 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb67015ef50 0x7fb6700d03e0 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.661+0000 7fb676770640 1 -- 172.21.15.43:0/3892316598 >> 172.21.15.43:0/3892316598 conn(0x7fb67009cd00 msgr2=0x7fb6700d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:08.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.661+0000 7fb676770640 1 -- 172.21.15.43:0/3892316598 shutdown_connections 2024-04-25T09:04:08.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.661+0000 7fb676770640 1 -- 172.21.15.43:0/3892316598 wait complete. 2024-04-25T09:04:08.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.662+0000 7fb676770640 1 Processor -- start 2024-04-25T09:04:08.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.662+0000 7fb676770640 1 -- start start 2024-04-25T09:04:08.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.663+0000 7fb676770640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6700d0920 0x7fb6700cb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.663+0000 7fb676770640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb67015dce0 0x7fb6701f6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.663+0000 7fb676770640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb67015ef50 0x7fb67009fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.663+0000 7fb676770640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6700cf720 con 0x7fb6700d0920 2024-04-25T09:04:08.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.663+0000 7fb676770640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6700cf440 con 0x7fb67015dce0 2024-04-25T09:04:08.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.663+0000 7fb676770640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb67015f6b0 con 0x7fb67015ef50 2024-04-25T09:04:08.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.663+0000 7fb66ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6700d0920 0x7fb6700cb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.663+0000 7fb66ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6700d0920 0x7fb6700cb8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41624/0 (socket says 172.21.15.43:41624) 2024-04-25T09:04:08.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.663+0000 7fb66ffff640 1 -- 172.21.15.43:0/1252484826 learned_addr learned my addr 172.21.15.43:0/1252484826 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:08.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.664+0000 7fb66f7fe640 1 --2- 172.21.15.43:0/1252484826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb67015dce0 0x7fb6701f6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.664+0000 7fb674d0d640 1 --2- 172.21.15.43:0/1252484826 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb67015ef50 0x7fb67009fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.664+0000 7fb66ffff640 1 -- 172.21.15.43:0/1252484826 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb67015ef50 msgr2=0x7fb67009fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.664+0000 7fb66ffff640 1 --2- 172.21.15.43:0/1252484826 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb67015ef50 0x7fb67009fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.664+0000 7fb66ffff640 1 -- 172.21.15.43:0/1252484826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb67015dce0 msgr2=0x7fb6701f6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.664+0000 7fb66ffff640 1 --2- 172.21.15.43:0/1252484826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb67015dce0 0x7fb6701f6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.664+0000 7fb66ffff640 1 -- 172.21.15.43:0/1252484826 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6700cc1f0 con 0x7fb6700d0920 2024-04-25T09:04:08.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.664+0000 7fb66f7fe640 1 --2- 172.21.15.43:0/1252484826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb67015dce0 0x7fb6701f6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:08.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.664+0000 7fb66ffff640 1 --2- 172.21.15.43:0/1252484826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6700d0920 0x7fb6700cb8f0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fb65800c440 tx=0x7fb65800c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:08.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.664+0000 7fb66cff9640 1 -- 172.21.15.43:0/1252484826 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb658017070 con 0x7fb6700d0920 2024-04-25T09:04:08.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.664+0000 7fb66cff9640 1 -- 172.21.15.43:0/1252484826 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6580035f0 con 0x7fb6700d0920 2024-04-25T09:04:08.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.664+0000 7fb66cff9640 1 -- 172.21.15.43:0/1252484826 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6580039e0 con 0x7fb6700d0920 2024-04-25T09:04:08.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.664+0000 7fb676770640 1 -- 172.21.15.43:0/1252484826 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6700cc4d0 con 0x7fb6700d0920 2024-04-25T09:04:08.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.664+0000 7fb676770640 1 -- 172.21.15.43:0/1252484826 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6700cbe30 con 0x7fb6700d0920 2024-04-25T09:04:08.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.665+0000 7fb676770640 1 -- 172.21.15.43:0/1252484826 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6700680f0 con 0x7fb6700d0920 2024-04-25T09:04:08.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.666+0000 7fb66cff9640 1 -- 172.21.15.43:0/1252484826 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb65801a070 con 0x7fb6700d0920 2024-04-25T09:04:08.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.666+0000 7fb66cff9640 1 --2- 172.21.15.43:0/1252484826 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb65006b500 0x7fb65006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:08.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.666+0000 7fb66cff9640 1 -- 172.21.15.43:0/1252484826 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5029+0+0 (secure 0 0 0) 0x7fb658089290 con 0x7fb6700d0920 2024-04-25T09:04:08.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.666+0000 7fb66f7fe640 1 --2- 172.21.15.43:0/1252484826 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb65006b500 0x7fb65006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:08.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.667+0000 7fb66f7fe640 1 --2- 172.21.15.43:0/1252484826 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb65006b500 0x7fb65006d9c0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fb660034000 tx=0x7fb66002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:08.658 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.670+0000 7fb66cff9640 1 -- 172.21.15.43:0/1252484826 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb6580583b0 con 0x7fb6700d0920 2024-04-25T09:04:08.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.889+0000 7fb676770640 1 -- 172.21.15.43:0/1252484826 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb670068e90 con 0x7fb6700d0920 2024-04-25T09:04:08.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.889+0000 7fb66cff9640 1 -- 172.21.15.43:0/1252484826 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+2911 (secure 0 0 0) 0x7fb6580010c0 con 0x7fb6700d0920 2024-04-25T09:04:08.878 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:04:08.879 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":82,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7172,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/891225068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":891225068},{"type":"v1","addr":"172.21.15.193:6836","nonce":891225068}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":7175,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/3679059390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":3679059390},{"type":"v1","addr":"172.21.15.193:6837","nonce":3679059390}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":73},{"gid":9642,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/2855045181","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":2855045181},{"type":"v1","addr":"172.21.15.43:6833","nonce":2855045181}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":82}],"filesystems":[]} 2024-04-25T09:04:08.879 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 82 2024-04-25T09:04:08.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.891+0000 7fb676770640 1 -- 172.21.15.43:0/1252484826 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb65006b500 msgr2=0x7fb65006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.891+0000 7fb676770640 1 --2- 172.21.15.43:0/1252484826 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb65006b500 0x7fb65006d9c0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fb660034000 tx=0x7fb66002e000 comp rx=0 tx=0).stop 2024-04-25T09:04:08.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.891+0000 7fb676770640 1 -- 172.21.15.43:0/1252484826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6700d0920 msgr2=0x7fb6700cb8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:08.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.891+0000 7fb676770640 1 --2- 172.21.15.43:0/1252484826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6700d0920 0x7fb6700cb8f0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fb65800c440 tx=0x7fb65800c900 comp rx=0 tx=0).stop 2024-04-25T09:04:08.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.891+0000 7fb676770640 1 -- 172.21.15.43:0/1252484826 shutdown_connections 2024-04-25T09:04:08.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.891+0000 7fb676770640 1 --2- 172.21.15.43:0/1252484826 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb67015ef50 0x7fb67009fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.891+0000 7fb676770640 1 --2- 172.21.15.43:0/1252484826 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb65006b500 0x7fb65006d9c0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.891+0000 7fb676770640 1 --2- 172.21.15.43:0/1252484826 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb67015dce0 0x7fb6701f6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.891+0000 7fb676770640 1 --2- 172.21.15.43:0/1252484826 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb6700d0920 0x7fb6700cb8f0 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:08.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.891+0000 7fb676770640 1 -- 172.21.15.43:0/1252484826 >> 172.21.15.43:0/1252484826 conn(0x7fb67009cd00 msgr2=0x7fb6700d37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:08.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.891+0000 7fb676770640 1 -- 172.21.15.43:0/1252484826 shutdown_connections 2024-04-25T09:04:08.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:08.891+0000 7fb676770640 1 -- 172.21.15.43:0/1252484826 wait complete. 2024-04-25T09:04:08.894 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T09:04:08.894 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T09:04:08.895 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T09:04:08.895 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T09:04:08.895 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:04:08.895 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:04:08.895 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T09:04:08.895 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T09:04:08.895 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:04:08.895 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:04:08.896 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T09:04:08.896 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T09:04:08.896 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:04:08.896 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:04:08.906 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:04:08.906 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T09:04:08.907 INFO:tasks.ceph.mds.a:Started 2024-04-25T09:04:08.907 DEBUG:teuthology.parallel:result is None 2024-04-25T09:04:08.910 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:04:08.910 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T09:04:08.911 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:04:08.911 DEBUG:teuthology.orchestra.run.smithi193:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T09:04:08.948 INFO:tasks.ceph.mds.c:Started 2024-04-25T09:04:08.949 DEBUG:teuthology.parallel:result is None 2024-04-25T09:04:08.952 INFO:tasks.ceph.mds.d:Started 2024-04-25T09:04:08.952 DEBUG:teuthology.parallel:result is None 2024-04-25T09:04:08.953 INFO:tasks.ceph.mds.b:Started 2024-04-25T09:04:08.953 DEBUG:teuthology.parallel:result is None 2024-04-25T09:04:08.953 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T09:04:08.981 INFO:tasks.ceph.mds.a.smithi043.stdout:starting mds.a at 2024-04-25T09:04:09.024 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-04-25T09:04:09.034 INFO:tasks.ceph.mds.d.smithi193.stdout:starting mds.d at 2024-04-25T09:04:09.035 INFO:tasks.ceph.mds.b.smithi193.stdout:starting mds.b at 2024-04-25T09:04:09.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.072+0000 7f25e535c640 1 Processor -- start 2024-04-25T09:04:09.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.072+0000 7f25e535c640 1 -- start start 2024-04-25T09:04:09.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.073+0000 7f25e535c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f25e006c510 0x7f25e006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.073+0000 7f25e535c640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f25e015df70 0x7f25e00ce070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.073+0000 7f25e535c640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25e00cbee0 con 0x7f25e00b0870 2024-04-25T09:04:09.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.073+0000 7f25e535c640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25e00d0400 con 0x7f25e015df70 2024-04-25T09:04:09.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.073+0000 7f25e535c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e006ce30 con 0x7f25e006c510 2024-04-25T09:04:09.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.073+0000 7f25df7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f25e00b0870 0x7f25e00ce5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56780/0 (socket says 172.21.15.43:56780) 2024-04-25T09:04:09.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.073+0000 7f25df7fe640 1 -- 172.21.15.43:0/825157170 learned_addr learned my addr 172.21.15.43:0/825157170 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:09.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.073+0000 7f25d7fff640 1 --2- 172.21.15.43:0/825157170 >> v2:172.21.15.193:3300/0 conn(0x7f25e015df70 0x7f25e00ce070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.073+0000 7f25deffd640 1 --2- 172.21.15.43:0/825157170 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f25e006c510 0x7f25e006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.073+0000 7f25de7fc640 1 -- 172.21.15.43:0/825157170 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 44903065 0 0) 0x7f25e00cbee0 con 0x7f25e00b0870 2024-04-25T09:04:09.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.073+0000 7f25de7fc640 1 -- 172.21.15.43:0/825157170 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25cc003520 con 0x7f25e00b0870 2024-04-25T09:04:09.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.073+0000 7f25deffd640 1 -- 172.21.15.43:0/825157170 >> v2:172.21.15.193:3300/0 conn(0x7f25e015df70 msgr2=0x7f25e00ce070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:09.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.073+0000 7f25deffd640 1 --2- 172.21.15.43:0/825157170 >> v2:172.21.15.193:3300/0 conn(0x7f25e015df70 0x7f25e00ce070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.073+0000 7f25deffd640 1 -- 172.21.15.43:0/825157170 >> v1:172.21.15.43:6789/0 conn(0x7f25e00b0870 legacy=0x7f25e00ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.073+0000 7f25deffd640 1 -- 172.21.15.43:0/825157170 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e00d0650 con 0x7f25e006c510 2024-04-25T09:04:09.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.074+0000 7f25deffd640 1 --2- 172.21.15.43:0/825157170 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f25e006c510 0x7f25e006c8f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f25d00097a0 tx=0x7f25d002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8d023dbee86bc5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:09.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.074+0000 7f25de7fc640 1 -- 172.21.15.43:0/825157170 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d0037070 con 0x7f25e006c510 2024-04-25T09:04:09.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.074+0000 7f25de7fc640 1 -- 172.21.15.43:0/825157170 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d0032440 con 0x7f25e006c510 2024-04-25T09:04:09.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.074+0000 7f25de7fc640 1 -- 172.21.15.43:0/825157170 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d0032810 con 0x7f25e006c510 2024-04-25T09:04:09.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.075+0000 7f25e535c640 1 -- 172.21.15.43:0/825157170 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f25e006c510 msgr2=0x7f25e006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.075+0000 7f25e535c640 1 --2- 172.21.15.43:0/825157170 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f25e006c510 0x7f25e006c8f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f25d00097a0 tx=0x7f25d002a180 comp rx=0 tx=0).stop 2024-04-25T09:04:09.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.075+0000 7f25e535c640 1 -- 172.21.15.43:0/825157170 shutdown_connections 2024-04-25T09:04:09.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.075+0000 7f25e535c640 1 --2- 172.21.15.43:0/825157170 >> v2:172.21.15.193:3300/0 conn(0x7f25e015df70 0x7f25e00ce070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.075+0000 7f25e535c640 1 --2- 172.21.15.43:0/825157170 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f25e006c510 0x7f25e006c8f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.075+0000 7f25e535c640 1 -- 172.21.15.43:0/825157170 >> 172.21.15.43:0/825157170 conn(0x7f25e00d17f0 msgr2=0x7f25e00d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:09.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.075+0000 7f25e535c640 1 -- 172.21.15.43:0/825157170 shutdown_connections 2024-04-25T09:04:09.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.075+0000 7f25e535c640 1 -- 172.21.15.43:0/825157170 wait complete. 2024-04-25T09:04:09.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.076+0000 7f25e535c640 1 Processor -- start 2024-04-25T09:04:09.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.076+0000 7f25e535c640 1 -- start start 2024-04-25T09:04:09.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.077+0000 7f25e535c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f25e006c510 0x7f25e0160e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.077+0000 7f25e535c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25e00b0870 0x7f25e00cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.077+0000 7f25e535c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25e015df70 0x7f25e015e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.077+0000 7f25e535c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e00d0c10 con 0x7f25e015df70 2024-04-25T09:04:09.068 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.077+0000 7f25e535c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e00cc150 con 0x7f25e00b0870 2024-04-25T09:04:09.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.077+0000 7f25e535c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e00cecc0 con 0x7f25e006c510 2024-04-25T09:04:09.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.077+0000 7f25df7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25e015df70 0x7f25e015e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.077+0000 7f25df7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25e015df70 0x7f25e015e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:41670/0 (socket says 172.21.15.43:41670) 2024-04-25T09:04:09.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.077+0000 7f25df7fe640 1 -- 172.21.15.43:0/3315256240 learned_addr learned my addr 172.21.15.43:0/3315256240 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:09.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.077+0000 7f25d7fff640 1 --2- 172.21.15.43:0/3315256240 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25e00b0870 0x7f25e00cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.077+0000 7f25deffd640 1 --2- 172.21.15.43:0/3315256240 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f25e006c510 0x7f25e0160e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.077+0000 7f25df7fe640 1 -- 172.21.15.43:0/3315256240 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f25e006c510 msgr2=0x7f25e0160e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.077+0000 7f25df7fe640 1 --2- 172.21.15.43:0/3315256240 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f25e006c510 0x7f25e0160e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.077+0000 7f25df7fe640 1 -- 172.21.15.43:0/3315256240 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25e00b0870 msgr2=0x7f25e00cb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:09.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.077+0000 7f25df7fe640 1 --2- 172.21.15.43:0/3315256240 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25e00b0870 0x7f25e00cb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:09.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.077+0000 7f25df7fe640 1 -- 172.21.15.43:0/3315256240 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e00c5790 con 0x7f25e015df70 2024-04-25T09:04:09.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.078+0000 7f25df7fe640 1 --2- 172.21.15.43:0/3315256240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25e015df70 0x7f25e015e3b0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f25d800cb10 tx=0x7f25d800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:09.069 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.078+0000 7f25d77fe640 1 -- 172.21.15.43:0/3315256240 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d8017070 con 0x7f25e015df70 2024-04-25T09:04:09.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.078+0000 7f25d77fe640 1 -- 172.21.15.43:0/3315256240 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25d800ac70 con 0x7f25e015df70 2024-04-25T09:04:09.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.078+0000 7f25e535c640 1 -- 172.21.15.43:0/3315256240 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25e00c5a70 con 0x7f25e015df70 2024-04-25T09:04:09.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.078+0000 7f25d77fe640 1 -- 172.21.15.43:0/3315256240 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d8012620 con 0x7f25e015df70 2024-04-25T09:04:09.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.078+0000 7f25e535c640 1 -- 172.21.15.43:0/3315256240 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e00c6030 con 0x7f25e015df70 2024-04-25T09:04:09.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.079+0000 7f25e535c640 1 -- 172.21.15.43:0/3315256240 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25e0074ef0 con 0x7f25e015df70 2024-04-25T09:04:09.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.079+0000 7f25d77fe640 1 -- 172.21.15.43:0/3315256240 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f25d801a070 con 0x7f25e015df70 2024-04-25T09:04:09.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.080+0000 7f25d77fe640 1 --2- 172.21.15.43:0/3315256240 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f25c006b4d0 0x7f25c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:09.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.080+0000 7f25d77fe640 1 -- 172.21.15.43:0/3315256240 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5029+0+0 (secure 0 0 0) 0x7f25d8089c70 con 0x7f25e015df70 2024-04-25T09:04:09.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.080+0000 7f25deffd640 1 --2- 172.21.15.43:0/3315256240 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f25c006b4d0 0x7f25c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:09.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.081+0000 7f25deffd640 1 --2- 172.21.15.43:0/3315256240 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f25c006b4d0 0x7f25c006d990 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f25d0034000 tx=0x7f25d002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:09.073 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.084+0000 7f25d77fe640 1 -- 172.21.15.43:0/3315256240 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f25d800ae20 con 0x7f25e015df70 2024-04-25T09:04:09.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:09.278+0000 7f25e535c640 1 -- 172.21.15.43:0/3315256240 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f25e00afe40 con 0x7f25e015df70 2024-04-25T09:04:10.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.028+0000 7f25d77fe640 1 -- 172.21.15.43:0/3315256240 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v134) v1 ==== 97+0+0 (secure 0 0 0) 0x7f25d8058730 con 0x7f25e015df70 2024-04-25T09:04:10.017 INFO:teuthology.orchestra.run.smithi043.stderr: removed all blocklist entries 2024-04-25T09:04:10.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.032+0000 7f25e535c640 1 -- 172.21.15.43:0/3315256240 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f25c006b4d0 msgr2=0x7f25c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.032+0000 7f25e535c640 1 --2- 172.21.15.43:0/3315256240 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f25c006b4d0 0x7f25c006d990 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f25d0034000 tx=0x7f25d002e000 comp rx=0 tx=0).stop 2024-04-25T09:04:10.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.032+0000 7f25e535c640 1 -- 172.21.15.43:0/3315256240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25e015df70 msgr2=0x7f25e015e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.032+0000 7f25e535c640 1 --2- 172.21.15.43:0/3315256240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25e015df70 0x7f25e015e3b0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f25d800cb10 tx=0x7f25d800a060 comp rx=0 tx=0).stop 2024-04-25T09:04:10.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.032+0000 7f25e535c640 1 -- 172.21.15.43:0/3315256240 shutdown_connections 2024-04-25T09:04:10.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.032+0000 7f25e535c640 1 --2- 172.21.15.43:0/3315256240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25e015df70 0x7f25e015e3b0 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.032+0000 7f25e535c640 1 --2- 172.21.15.43:0/3315256240 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f25c006b4d0 0x7f25c006d990 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.032+0000 7f25e535c640 1 --2- 172.21.15.43:0/3315256240 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25e00b0870 0x7f25e00cb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.032+0000 7f25e535c640 1 --2- 172.21.15.43:0/3315256240 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f25e006c510 0x7f25e0160e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.032+0000 7f25e535c640 1 -- 172.21.15.43:0/3315256240 >> 172.21.15.43:0/3315256240 conn(0x7f25e00d17f0 msgr2=0x7f25e00d1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:10.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.032+0000 7f25e535c640 1 -- 172.21.15.43:0/3315256240 shutdown_connections 2024-04-25T09:04:10.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.032+0000 7f25e535c640 1 -- 172.21.15.43:0/3315256240 wait complete. 2024-04-25T09:04:10.035 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T09:04:10.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.197+0000 7fe26891d640 1 Processor -- start 2024-04-25T09:04:10.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.197+0000 7fe26891d640 1 -- start start 2024-04-25T09:04:10.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.198+0000 7fe26891d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe26415def0 0x7fe26406be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.198+0000 7fe26891d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe264162560 0x7fe2640b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.198+0000 7fe26891d640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe26415e2d0 con 0x7fe2641619f0 2024-04-25T09:04:10.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.198+0000 7fe26891d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2640d0e90 con 0x7fe264162560 2024-04-25T09:04:10.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.198+0000 7fe26891d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe26406c390 con 0x7fe26415def0 2024-04-25T09:04:10.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.198+0000 7fe262d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe264162560 0x7fe2640b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.198+0000 7fe262d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fe264162560 0x7fe2640b20c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47374/0 (socket says 172.21.15.43:47374) 2024-04-25T09:04:10.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.198+0000 7fe262d9d640 1 -- 172.21.15.43:0/472973300 learned_addr learned my addr 172.21.15.43:0/472973300 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:10.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.198+0000 7fe26259c640 1 --2- 172.21.15.43:0/472973300 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe26415def0 0x7fe26406be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.198+0000 7fe262d9d640 1 -- 172.21.15.43:0/472973300 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe26415def0 msgr2=0x7fe26406be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:10.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.198+0000 7fe262d9d640 1 --2- 172.21.15.43:0/472973300 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe26415def0 0x7fe26406be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.198+0000 7fe262d9d640 1 -- 172.21.15.43:0/472973300 >> v1:172.21.15.193:6789/0 conn(0x7fe2641619f0 legacy=0x7fe264161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.198+0000 7fe262d9d640 1 -- 172.21.15.43:0/472973300 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2640b1020 con 0x7fe264162560 2024-04-25T09:04:10.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.199+0000 7fe262d9d640 1 --2- 172.21.15.43:0/472973300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe264162560 0x7fe2640b20c0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7fe25c009e80 tx=0x7fe25c02a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3e6fb06b33cf158 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.199+0000 7fe26159a640 1 -- 172.21.15.43:0/472973300 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25c036070 con 0x7fe264162560 2024-04-25T09:04:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.199+0000 7fe26159a640 1 -- 172.21.15.43:0/472973300 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe25c031600 con 0x7fe264162560 2024-04-25T09:04:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.199+0000 7fe26159a640 1 -- 172.21.15.43:0/472973300 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25c0319d0 con 0x7fe264162560 2024-04-25T09:04:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.200+0000 7fe26891d640 1 -- 172.21.15.43:0/472973300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe264162560 msgr2=0x7fe2640b20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.200+0000 7fe26891d640 1 --2- 172.21.15.43:0/472973300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe264162560 0x7fe2640b20c0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7fe25c009e80 tx=0x7fe25c02a750 comp rx=0 tx=0).stop 2024-04-25T09:04:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.200+0000 7fe26891d640 1 -- 172.21.15.43:0/472973300 shutdown_connections 2024-04-25T09:04:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.200+0000 7fe26891d640 1 --2- 172.21.15.43:0/472973300 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe264162560 0x7fe2640b20c0 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.200+0000 7fe26891d640 1 --2- 172.21.15.43:0/472973300 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe26415def0 0x7fe26406be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.200+0000 7fe26891d640 1 -- 172.21.15.43:0/472973300 >> 172.21.15.43:0/472973300 conn(0x7fe2640b32e0 msgr2=0x7fe2640b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.200+0000 7fe26891d640 1 -- 172.21.15.43:0/472973300 shutdown_connections 2024-04-25T09:04:10.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.200+0000 7fe26891d640 1 -- 172.21.15.43:0/472973300 wait complete. 2024-04-25T09:04:10.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.201+0000 7fe26891d640 1 Processor -- start 2024-04-25T09:04:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.201+0000 7fe26891d640 1 -- start start 2024-04-25T09:04:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.202+0000 7fe26891d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe26415def0 0x7fe2640c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.202+0000 7fe26891d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2641619f0 0x7fe2640c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.202+0000 7fe26891d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe264162560 0x7fe264159da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.202+0000 7fe26891d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2640b18c0 con 0x7fe2641619f0 2024-04-25T09:04:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.202+0000 7fe26891d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2640b15e0 con 0x7fe26415def0 2024-04-25T09:04:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.202+0000 7fe26891d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2640b1ba0 con 0x7fe264162560 2024-04-25T09:04:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.202+0000 7fe261d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2641619f0 0x7fe2640c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.202+0000 7fe261d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2641619f0 0x7fe2640c52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47378/0 (socket says 172.21.15.43:47378) 2024-04-25T09:04:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.202+0000 7fe261d9b640 1 -- 172.21.15.43:0/2630533184 learned_addr learned my addr 172.21.15.43:0/2630533184 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.202+0000 7fe26259c640 1 --2- 172.21.15.43:0/2630533184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe26415def0 0x7fe2640c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.202+0000 7fe262d9d640 1 --2- 172.21.15.43:0/2630533184 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe264162560 0x7fe264159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.202+0000 7fe261d9b640 1 -- 172.21.15.43:0/2630533184 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe264162560 msgr2=0x7fe264159da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.202+0000 7fe261d9b640 1 --2- 172.21.15.43:0/2630533184 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe264162560 0x7fe264159da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.202+0000 7fe261d9b640 1 -- 172.21.15.43:0/2630533184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe26415def0 msgr2=0x7fe2640c72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.202+0000 7fe261d9b640 1 --2- 172.21.15.43:0/2630533184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe26415def0 0x7fe2640c72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.202+0000 7fe261d9b640 1 -- 172.21.15.43:0/2630533184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2640b4400 con 0x7fe2641619f0 2024-04-25T09:04:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.202+0000 7fe26259c640 1 --2- 172.21.15.43:0/2630533184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe26415def0 0x7fe2640c72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.203+0000 7fe261d9b640 1 --2- 172.21.15.43:0/2630533184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2641619f0 0x7fe2640c52a0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7fe25800c430 tx=0x7fe25800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.203+0000 7fe252ffd640 1 -- 172.21.15.43:0/2630533184 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2580052d0 con 0x7fe2641619f0 2024-04-25T09:04:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.203+0000 7fe252ffd640 1 -- 172.21.15.43:0/2630533184 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2580054b0 con 0x7fe2641619f0 2024-04-25T09:04:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.203+0000 7fe26891d640 1 -- 172.21.15.43:0/2630533184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2640b46e0 con 0x7fe2641619f0 2024-04-25T09:04:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.203+0000 7fe252ffd640 1 -- 172.21.15.43:0/2630533184 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe258006a60 con 0x7fe2641619f0 2024-04-25T09:04:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.203+0000 7fe26891d640 1 -- 172.21.15.43:0/2630533184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2640b4ca0 con 0x7fe2641619f0 2024-04-25T09:04:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.204+0000 7fe26891d640 1 -- 172.21.15.43:0/2630533184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe264074ef0 con 0x7fe2641619f0 2024-04-25T09:04:10.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.204+0000 7fe252ffd640 1 -- 172.21.15.43:0/2630533184 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe258006c00 con 0x7fe2641619f0 2024-04-25T09:04:10.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.205+0000 7fe252ffd640 1 --2- 172.21.15.43:0/2630533184 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe24406b4d0 0x7fe24406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.205+0000 7fe252ffd640 1 -- 172.21.15.43:0/2630533184 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe258089320 con 0x7fe2641619f0 2024-04-25T09:04:10.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.205+0000 7fe26259c640 1 --2- 172.21.15.43:0/2630533184 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe24406b4d0 0x7fe24406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.205+0000 7fe26259c640 1 --2- 172.21.15.43:0/2630533184 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe24406b4d0 0x7fe24406d990 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe254009a20 tx=0x7fe254008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:10.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.209+0000 7fe252ffd640 1 -- 172.21.15.43:0/2630533184 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe2580584f0 con 0x7fe2641619f0 2024-04-25T09:04:10.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.455+0000 7fe26891d640 1 -- 172.21.15.43:0/2630533184 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe26406ef70 con 0x7fe2641619f0 2024-04-25T09:04:10.444 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.455+0000 7fe252ffd640 1 -- 172.21.15.43:0/2630533184 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fe258057df0 con 0x7fe2641619f0 2024-04-25T09:04:10.444 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:04:10.444 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:04:10.444 INFO:teuthology.orchestra.run.smithi043.stdout: "auth_dump": [ 2024-04-25T09:04:10.444 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:04:10.444 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.a", 2024-04-25T09:04:10.444 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpmIjzEBRAAs2L10XrFO3mThjZevna5wA==", 2024-04-25T09:04:10.444 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:04:10.445 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T09:04:10.445 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T09:04:10.445 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T09:04:10.445 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:04:10.445 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:04:10.445 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:04:10.445 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:04:10.445 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.b", 2024-04-25T09:04:10.445 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm51cmERAAij4vy8xyihh8u/4dkmcllw==", 2024-04-25T09:04:10.445 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:04:10.445 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T09:04:10.445 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T09:04:10.445 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T09:04:10.445 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:04:10.446 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:04:10.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:04:10.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:04:10.446 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.c", 2024-04-25T09:04:10.446 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm1OOtDBAAlUYbCh1mBEEka9CWRnUz9A==", 2024-04-25T09:04:10.446 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:04:10.446 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T09:04:10.446 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T09:04:10.446 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T09:04:10.446 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:04:10.446 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:04:10.446 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:04:10.446 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:04:10.447 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mds.d", 2024-04-25T09:04:10.447 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpmpEvsFxAAXX1h37UWdFkyJ3Iw3xd8kw==", 2024-04-25T09:04:10.447 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:04:10.447 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T09:04:10.447 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T09:04:10.447 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T09:04:10.447 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:04:10.447 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:04:10.447 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:04:10.447 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:04:10.447 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.0", 2024-04-25T09:04:10.447 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD9GSpmi7qkORAAwSNgwzcVMitrPPf3nDyqvQ==", 2024-04-25T09:04:10.447 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:04:10.447 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T09:04:10.448 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T09:04:10.448 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:04:10.448 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:04:10.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:04:10.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:04:10.448 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.1", 2024-04-25T09:04:10.448 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAAGipmJDT3NxAAcWBCd9nt7Au+Xftjqcuc5w==", 2024-04-25T09:04:10.448 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:04:10.448 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T09:04:10.448 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T09:04:10.448 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:04:10.448 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:04:10.448 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:04:10.448 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:04:10.448 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.2", 2024-04-25T09:04:10.449 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQADGipm68kzNhAA0ZOnlr4edWDjx6RfoEHYIw==", 2024-04-25T09:04:10.449 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:04:10.449 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T09:04:10.449 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T09:04:10.449 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:04:10.449 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:04:10.449 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:04:10.449 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:04:10.449 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.3", 2024-04-25T09:04:10.449 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAGGipmT1AeNBAAoNPiEtkznSxxhVJ1EmZ1ZA==", 2024-04-25T09:04:10.449 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:04:10.449 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T09:04:10.449 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T09:04:10.449 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:04:10.449 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:04:10.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:04:10.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:04:10.450 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.4", 2024-04-25T09:04:10.450 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAMGipmBjHHNxAAH/SCzKHus8fKxxI3UN4wWQ==", 2024-04-25T09:04:10.450 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:04:10.450 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T09:04:10.450 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T09:04:10.450 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:04:10.450 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:04:10.450 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:04:10.450 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:04:10.450 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.5", 2024-04-25T09:04:10.450 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAPGipm3FXKNRAAL97YYWONg4QHr4ajC5D7fA==", 2024-04-25T09:04:10.451 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:04:10.451 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T09:04:10.451 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T09:04:10.451 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:04:10.451 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:04:10.451 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:04:10.451 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:04:10.451 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.6", 2024-04-25T09:04:10.451 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQASGipmCwKaMxAAcM0p9Q2sbNQviGkwSLg8AA==", 2024-04-25T09:04:10.451 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:04:10.451 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T09:04:10.451 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T09:04:10.451 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:04:10.451 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:04:10.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:04:10.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:04:10.452 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "osd.7", 2024-04-25T09:04:10.452 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQAVGipmP7A2MRAAxnFJFosMcbZ6VDbgH8r6wA==", 2024-04-25T09:04:10.452 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:04:10.452 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow profile osd", 2024-04-25T09:04:10.452 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile osd", 2024-04-25T09:04:10.452 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:04:10.452 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:04:10.452 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:04:10.452 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:04:10.452 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.0", 2024-04-25T09:04:10.452 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpmVOjWGxAAFNmxeGi0h2D//pa6Ne3eQw==", 2024-04-25T09:04:10.452 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:04:10.453 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T09:04:10.453 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow r", 2024-04-25T09:04:10.453 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T09:04:10.453 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:04:10.453 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:04:10.453 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:04:10.453 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.1", 2024-04-25T09:04:10.453 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm24laJBAA9ijbDlFDTmF/ZZ4IhLfBVQ==", 2024-04-25T09:04:10.453 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:04:10.453 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T09:04:10.453 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-04-25T09:04:10.453 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-04-25T09:04:10.453 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-04-25T09:04:10.453 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:04:10.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:04:10.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:04:10.454 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.2", 2024-04-25T09:04:10.454 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm8b5UIRAACcOwDL630oJrXfp7s+fNTA==", 2024-04-25T09:04:10.454 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:04:10.454 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T09:04:10.454 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-04-25T09:04:10.454 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-04-25T09:04:10.454 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-04-25T09:04:10.454 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:04:10.454 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:04:10.454 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:04:10.454 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.3", 2024-04-25T09:04:10.454 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpmLsbBKRAAIKWtRdpuyCaW1LnKMWTMiw==", 2024-04-25T09:04:10.455 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:04:10.455 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow", 2024-04-25T09:04:10.455 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow r", 2024-04-25T09:04:10.455 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow rw", 2024-04-25T09:04:10.455 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow rwx" 2024-04-25T09:04:10.455 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:04:10.455 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:04:10.455 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:04:10.455 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "client.admin", 2024-04-25T09:04:10.455 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD7GSpmE9C8JRAAS263n6a4L2DXzMyS9o42NQ==", 2024-04-25T09:04:10.455 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:04:10.455 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-04-25T09:04:10.455 INFO:teuthology.orchestra.run.smithi043.stdout: "mgr": "allow *", 2024-04-25T09:04:10.455 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow *", 2024-04-25T09:04:10.455 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:04:10.456 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:04:10.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:04:10.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:04:10.456 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.x", 2024-04-25T09:04:10.456 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD8GSpm/uGhAhAAr0FaJ6SwZ8Su1MEqyZwNRA==", 2024-04-25T09:04:10.456 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:04:10.456 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-04-25T09:04:10.456 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2024-04-25T09:04:10.456 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:04:10.456 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:04:10.456 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:04:10.456 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:04:10.456 INFO:teuthology.orchestra.run.smithi043.stdout: "entity": "mgr.y", 2024-04-25T09:04:10.456 INFO:teuthology.orchestra.run.smithi043.stdout: "key": "AQD7GSpmPwwpOhAALeiQ/LKM2zgjE7yu16RQOg==", 2024-04-25T09:04:10.457 INFO:teuthology.orchestra.run.smithi043.stdout: "caps": { 2024-04-25T09:04:10.457 INFO:teuthology.orchestra.run.smithi043.stdout: "mds": "allow *", 2024-04-25T09:04:10.457 INFO:teuthology.orchestra.run.smithi043.stdout: "mon": "allow profile mgr", 2024-04-25T09:04:10.457 INFO:teuthology.orchestra.run.smithi043.stdout: "osd": "allow *" 2024-04-25T09:04:10.457 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:04:10.457 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:04:10.457 INFO:teuthology.orchestra.run.smithi043.stdout: ] 2024-04-25T09:04:10.457 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:04:10.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.457+0000 7fe26891d640 1 -- 172.21.15.43:0/2630533184 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe24406b4d0 msgr2=0x7fe24406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.458 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.457+0000 7fe26891d640 1 --2- 172.21.15.43:0/2630533184 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe24406b4d0 0x7fe24406d990 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fe254009a20 tx=0x7fe254008040 comp rx=0 tx=0).stop 2024-04-25T09:04:10.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.457+0000 7fe26891d640 1 -- 172.21.15.43:0/2630533184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2641619f0 msgr2=0x7fe2640c52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.457+0000 7fe26891d640 1 --2- 172.21.15.43:0/2630533184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2641619f0 0x7fe2640c52a0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7fe25800c430 tx=0x7fe25800c8f0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.457+0000 7fe26891d640 1 -- 172.21.15.43:0/2630533184 shutdown_connections 2024-04-25T09:04:10.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.457+0000 7fe26891d640 1 --2- 172.21.15.43:0/2630533184 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fe264162560 0x7fe264159da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.457+0000 7fe26891d640 1 --2- 172.21.15.43:0/2630533184 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fe2641619f0 0x7fe2640c52a0 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.457+0000 7fe26891d640 1 --2- 172.21.15.43:0/2630533184 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fe24406b4d0 0x7fe24406d990 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.457+0000 7fe26891d640 1 --2- 172.21.15.43:0/2630533184 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fe26415def0 0x7fe2640c72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.457+0000 7fe26891d640 1 -- 172.21.15.43:0/2630533184 >> 172.21.15.43:0/2630533184 conn(0x7fe2640b32e0 msgr2=0x7fe26406e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:10.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.457+0000 7fe26891d640 1 -- 172.21.15.43:0/2630533184 shutdown_connections 2024-04-25T09:04:10.459 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.457+0000 7fe26891d640 1 -- 172.21.15.43:0/2630533184 wait complete. 2024-04-25T09:04:10.460 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:04:10.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.572+0000 7fef99d89640 1 Processor -- start 2024-04-25T09:04:10.561 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.572+0000 7fef99d89640 1 -- start start 2024-04-25T09:04:10.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.573+0000 7fef99d89640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fef9415def0 0x7fef9406be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.573+0000 7fef99d89640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef94162560 0x7fef940b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.573+0000 7fef99d89640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef9415e2d0 con 0x7fef941619f0 2024-04-25T09:04:10.562 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.573+0000 7fef99d89640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef940d0e90 con 0x7fef9415def0 2024-04-25T09:04:10.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.573+0000 7fef99d89640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef9406c390 con 0x7fef94162560 2024-04-25T09:04:10.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.573+0000 7fef937fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fef9415def0 0x7fef9406be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.573+0000 7fef937fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fef9415def0 0x7fef9406be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47394/0 (socket says 172.21.15.43:47394) 2024-04-25T09:04:10.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.573+0000 7fef937fe640 1 -- 172.21.15.43:0/2955961240 learned_addr learned my addr 172.21.15.43:0/2955961240 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:10.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.574+0000 7fef937fe640 1 -- 172.21.15.43:0/2955961240 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef94162560 msgr2=0x7fef940b20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:10.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.574+0000 7fef937fe640 1 --2- 172.21.15.43:0/2955961240 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef94162560 0x7fef940b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.574+0000 7fef937fe640 1 -- 172.21.15.43:0/2955961240 >> v1:172.21.15.193:6789/0 conn(0x7fef941619f0 legacy=0x7fef94161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:10.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.574+0000 7fef937fe640 1 -- 172.21.15.43:0/2955961240 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef940b1020 con 0x7fef9415def0 2024-04-25T09:04:10.563 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.574+0000 7fef937fe640 1 --2- 172.21.15.43:0/2955961240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef9415def0 0x7fef9406be50 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fef84009870 tx=0x7fef8402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7f8b509128b4d36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:10.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.574+0000 7fef927fc640 1 -- 172.21.15.43:0/2955961240 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef84037070 con 0x7fef9415def0 2024-04-25T09:04:10.564 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.574+0000 7fef927fc640 1 -- 172.21.15.43:0/2955961240 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef84032440 con 0x7fef9415def0 2024-04-25T09:04:10.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.574+0000 7fef927fc640 1 -- 172.21.15.43:0/2955961240 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef84032810 con 0x7fef9415def0 2024-04-25T09:04:10.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.575+0000 7fef99d89640 1 -- 172.21.15.43:0/2955961240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef9415def0 msgr2=0x7fef9406be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.575+0000 7fef99d89640 1 --2- 172.21.15.43:0/2955961240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef9415def0 0x7fef9406be50 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7fef84009870 tx=0x7fef8402a210 comp rx=0 tx=0).stop 2024-04-25T09:04:10.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.575+0000 7fef99d89640 1 -- 172.21.15.43:0/2955961240 shutdown_connections 2024-04-25T09:04:10.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.575+0000 7fef99d89640 1 --2- 172.21.15.43:0/2955961240 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef94162560 0x7fef940b20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.575+0000 7fef99d89640 1 --2- 172.21.15.43:0/2955961240 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef9415def0 0x7fef9406be50 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.575+0000 7fef99d89640 1 -- 172.21.15.43:0/2955961240 >> 172.21.15.43:0/2955961240 conn(0x7fef940b32e0 msgr2=0x7fef940b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:10.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.575+0000 7fef99d89640 1 -- 172.21.15.43:0/2955961240 shutdown_connections 2024-04-25T09:04:10.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.575+0000 7fef99d89640 1 -- 172.21.15.43:0/2955961240 wait complete. 2024-04-25T09:04:10.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.576+0000 7fef99d89640 1 Processor -- start 2024-04-25T09:04:10.565 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.576+0000 7fef99d89640 1 -- start start 2024-04-25T09:04:10.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.577+0000 7fef99d89640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef9415def0 0x7fef940b6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.577+0000 7fef99d89640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef941619f0 0x7fef9415e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.577+0000 7fef99d89640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef94162560 0x7fef9415ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.577+0000 7fef99d89640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef940b18c0 con 0x7fef9415def0 2024-04-25T09:04:10.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.577+0000 7fef99d89640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef940b15e0 con 0x7fef94162560 2024-04-25T09:04:10.569 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.577+0000 7fef99d89640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef940b1ba0 con 0x7fef941619f0 2024-04-25T09:04:10.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.577+0000 7fef937fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef9415def0 0x7fef940b6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.577+0000 7fef937fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef9415def0 0x7fef940b6c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47400/0 (socket says 172.21.15.43:47400) 2024-04-25T09:04:10.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.577+0000 7fef937fe640 1 -- 172.21.15.43:0/3939712082 learned_addr learned my addr 172.21.15.43:0/3939712082 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:10.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.577+0000 7fef93fff640 1 --2- 172.21.15.43:0/3939712082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef94162560 0x7fef9415ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.577+0000 7fef92ffd640 1 --2- 172.21.15.43:0/3939712082 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef941619f0 0x7fef9415e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.577+0000 7fef937fe640 1 -- 172.21.15.43:0/3939712082 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef941619f0 msgr2=0x7fef9415e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.577+0000 7fef937fe640 1 --2- 172.21.15.43:0/3939712082 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef941619f0 0x7fef9415e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.577+0000 7fef937fe640 1 -- 172.21.15.43:0/3939712082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef94162560 msgr2=0x7fef9415ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.577+0000 7fef937fe640 1 --2- 172.21.15.43:0/3939712082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef94162560 0x7fef9415ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.577+0000 7fef937fe640 1 -- 172.21.15.43:0/3939712082 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef941e9920 con 0x7fef9415def0 2024-04-25T09:04:10.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.577+0000 7fef92ffd640 1 --2- 172.21.15.43:0/3939712082 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef941619f0 0x7fef9415e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:10.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.578+0000 7fef937fe640 1 --2- 172.21.15.43:0/3939712082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef9415def0 0x7fef940b6c90 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fef84000c00 tx=0x7fef840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:10.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.578+0000 7fef7bfff640 1 -- 172.21.15.43:0/3939712082 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef84037040 con 0x7fef9415def0 2024-04-25T09:04:10.570 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.578+0000 7fef7bfff640 1 -- 172.21.15.43:0/3939712082 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef84032e50 con 0x7fef9415def0 2024-04-25T09:04:10.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.578+0000 7fef7bfff640 1 -- 172.21.15.43:0/3939712082 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef840309b0 con 0x7fef9415def0 2024-04-25T09:04:10.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.578+0000 7fef99d89640 1 -- 172.21.15.43:0/3939712082 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef941e9640 con 0x7fef9415def0 2024-04-25T09:04:10.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.578+0000 7fef99d89640 1 -- 172.21.15.43:0/3939712082 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef940b4b20 con 0x7fef9415def0 2024-04-25T09:04:10.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.579+0000 7fef99d89640 1 -- 172.21.15.43:0/3939712082 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef94074ef0 con 0x7fef9415def0 2024-04-25T09:04:10.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.579+0000 7fef7bfff640 1 -- 172.21.15.43:0/3939712082 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fef84044020 con 0x7fef9415def0 2024-04-25T09:04:10.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.580+0000 7fef7bfff640 1 --2- 172.21.15.43:0/3939712082 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fef8006b500 0x7fef8006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.580+0000 7fef7bfff640 1 -- 172.21.15.43:0/3939712082 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fef840aa100 con 0x7fef9415def0 2024-04-25T09:04:10.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.580+0000 7fef92ffd640 1 --2- 172.21.15.43:0/3939712082 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fef8006b500 0x7fef8006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.571 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.580+0000 7fef92ffd640 1 --2- 172.21.15.43:0/3939712082 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fef8006b500 0x7fef8006d9c0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fef940b0a70 tx=0x7fef88009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:10.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.584+0000 7fef7bfff640 1 -- 172.21.15.43:0/3939712082 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fef8404b020 con 0x7fef9415def0 2024-04-25T09:04:10.755 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.767+0000 7fef99d89640 1 -- 172.21.15.43:0/3939712082 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef940d1f70 con 0x7fef9415def0 2024-04-25T09:04:10.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.768+0000 7fef7bfff640 1 -- 172.21.15.43:0/3939712082 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) v1 ==== 74+0+12865 (secure 0 0 0) 0x7fef84078be0 con 0x7fef9415def0 2024-04-25T09:04:10.756 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:04:10.756 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":135,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:04:10.032201+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":128,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:04:10.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.770+0000 7fef99d89640 1 -- 172.21.15.43:0/3939712082 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fef8006b500 msgr2=0x7fef8006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.770+0000 7fef99d89640 1 --2- 172.21.15.43:0/3939712082 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fef8006b500 0x7fef8006d9c0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fef940b0a70 tx=0x7fef88009040 comp rx=0 tx=0).stop 2024-04-25T09:04:10.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.770+0000 7fef99d89640 1 -- 172.21.15.43:0/3939712082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef9415def0 msgr2=0x7fef940b6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.770+0000 7fef99d89640 1 --2- 172.21.15.43:0/3939712082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef9415def0 0x7fef940b6c90 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7fef84000c00 tx=0x7fef840046c0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.771+0000 7fef99d89640 1 -- 172.21.15.43:0/3939712082 shutdown_connections 2024-04-25T09:04:10.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.771+0000 7fef99d89640 1 --2- 172.21.15.43:0/3939712082 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fef8006b500 0x7fef8006d9c0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.771+0000 7fef99d89640 1 --2- 172.21.15.43:0/3939712082 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fef94162560 0x7fef9415ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.771+0000 7fef99d89640 1 --2- 172.21.15.43:0/3939712082 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fef941619f0 0x7fef9415e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.771+0000 7fef99d89640 1 --2- 172.21.15.43:0/3939712082 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fef9415def0 0x7fef940b6c90 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.771+0000 7fef99d89640 1 -- 172.21.15.43:0/3939712082 >> 172.21.15.43:0/3939712082 conn(0x7fef940b32e0 msgr2=0x7fef940d1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:10.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.771+0000 7fef99d89640 1 -- 172.21.15.43:0/3939712082 shutdown_connections 2024-04-25T09:04:10.760 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.771+0000 7fef99d89640 1 -- 172.21.15.43:0/3939712082 wait complete. 2024-04-25T09:04:10.771 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:04:10.771 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:04:10.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.889+0000 7f11218bc640 1 Processor -- start 2024-04-25T09:04:10.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.889+0000 7f11218bc640 1 -- start start 2024-04-25T09:04:10.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.890+0000 7f11218bc640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f111c15a750 0x7f111c0cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.890+0000 7f11218bc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f111c071fa0 0x7f111c0d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.890+0000 7f11218bc640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f111c0af850 0x7f111c0d1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.890+0000 7f11218bc640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f111c0b4350 con 0x7f111c071fa0 2024-04-25T09:04:10.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.890+0000 7f11218bc640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f111c0afda0 con 0x7f111c15a750 2024-04-25T09:04:10.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.890+0000 7f11218bc640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f111c0cf0f0 con 0x7f111c0af850 2024-04-25T09:04:10.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.890+0000 7f111a7fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f111c071fa0 0x7f111c0d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.890+0000 7f111a7fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f111c071fa0 0x7f111c0d0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47414/0 (socket says 172.21.15.43:47414) 2024-04-25T09:04:10.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.890+0000 7f111a7fc640 1 -- 172.21.15.43:0/2770963704 learned_addr learned my addr 172.21.15.43:0/2770963704 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:10.880 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.891+0000 7f111affd640 1 --2- 172.21.15.43:0/2770963704 >> v2:172.21.15.193:3300/0 conn(0x7f111c15a750 0x7f111c0cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.891+0000 7f111b7fe640 1 --2- 172.21.15.43:0/2770963704 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f111c0af850 0x7f111c0d1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.891+0000 7f111a7fc640 1 -- 172.21.15.43:0/2770963704 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f111c0af850 msgr2=0x7f111c0d1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:10.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.891+0000 7f111a7fc640 1 --2- 172.21.15.43:0/2770963704 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f111c0af850 0x7f111c0d1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.891+0000 7f111a7fc640 1 -- 172.21.15.43:0/2770963704 >> v2:172.21.15.193:3300/0 conn(0x7f111c15a750 msgr2=0x7f111c0cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:10.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.891+0000 7f111a7fc640 1 --2- 172.21.15.43:0/2770963704 >> v2:172.21.15.193:3300/0 conn(0x7f111c15a750 0x7f111c0cf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.891+0000 7f111a7fc640 1 -- 172.21.15.43:0/2770963704 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f111c0ce7e0 con 0x7f111c071fa0 2024-04-25T09:04:10.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.891+0000 7f111a7fc640 1 --2- 172.21.15.43:0/2770963704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f111c071fa0 0x7f111c0d0be0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f111c0b5d70 tx=0x7f110402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b29f58c5ac7f7e00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:10.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.892+0000 7f1119ffb640 1 -- 172.21.15.43:0/2770963704 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1104037070 con 0x7f111c071fa0 2024-04-25T09:04:10.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.892+0000 7f1119ffb640 1 -- 172.21.15.43:0/2770963704 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1104032440 con 0x7f111c071fa0 2024-04-25T09:04:10.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.892+0000 7f1119ffb640 1 -- 172.21.15.43:0/2770963704 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1104032810 con 0x7f111c071fa0 2024-04-25T09:04:10.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.892+0000 7f11218bc640 1 -- 172.21.15.43:0/2770963704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f111c071fa0 msgr2=0x7f111c0d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.892+0000 7f11218bc640 1 --2- 172.21.15.43:0/2770963704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f111c071fa0 0x7f111c0d0be0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f111c0b5d70 tx=0x7f110402a1e0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.892+0000 7f11218bc640 1 -- 172.21.15.43:0/2770963704 shutdown_connections 2024-04-25T09:04:10.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.892+0000 7f11218bc640 1 --2- 172.21.15.43:0/2770963704 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f111c0af850 0x7f111c0d1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.892+0000 7f11218bc640 1 --2- 172.21.15.43:0/2770963704 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f111c071fa0 0x7f111c0d0be0 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.892+0000 7f11218bc640 1 --2- 172.21.15.43:0/2770963704 >> v2:172.21.15.193:3300/0 conn(0x7f111c15a750 0x7f111c0cf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.892+0000 7f11218bc640 1 -- 172.21.15.43:0/2770963704 >> 172.21.15.43:0/2770963704 conn(0x7f111c06c060 msgr2=0x7f111c06c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:10.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.892+0000 7f11218bc640 1 -- 172.21.15.43:0/2770963704 shutdown_connections 2024-04-25T09:04:10.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.892+0000 7f11218bc640 1 -- 172.21.15.43:0/2770963704 wait complete. 2024-04-25T09:04:10.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.893+0000 7f11218bc640 1 Processor -- start 2024-04-25T09:04:10.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.894+0000 7f11218bc640 1 -- start start 2024-04-25T09:04:10.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.894+0000 7f11218bc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f111c071fa0 0x7f111c14d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.894+0000 7f11218bc640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f111c0af850 0x7f111c15d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.894+0000 7f11218bc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f111c15a750 0x7f111c14c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.894+0000 7f11218bc640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111c0ced80 con 0x7f111c15a750 2024-04-25T09:04:10.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.894+0000 7f11218bc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f111c0ceaa0 con 0x7f111c071fa0 2024-04-25T09:04:10.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.894+0000 7f11218bc640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f111c15ab30 con 0x7f111c0af850 2024-04-25T09:04:10.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.894+0000 7f111b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f111c15a750 0x7f111c14c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.894+0000 7f111b7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f111c15a750 0x7f111c14c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47426/0 (socket says 172.21.15.43:47426) 2024-04-25T09:04:10.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.894+0000 7f111b7fe640 1 -- 172.21.15.43:0/85908237 learned_addr learned my addr 172.21.15.43:0/85908237 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:10.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.895+0000 7f111affd640 1 --2- 172.21.15.43:0/85908237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f111c071fa0 0x7f111c14d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.895+0000 7f111a7fc640 1 --2- 172.21.15.43:0/85908237 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f111c0af850 0x7f111c15d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.895+0000 7f111b7fe640 1 -- 172.21.15.43:0/85908237 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f111c0af850 msgr2=0x7f111c15d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.895+0000 7f111b7fe640 1 --2- 172.21.15.43:0/85908237 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f111c0af850 0x7f111c15d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.895+0000 7f111b7fe640 1 -- 172.21.15.43:0/85908237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f111c071fa0 msgr2=0x7f111c14d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:10.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.895+0000 7f111b7fe640 1 --2- 172.21.15.43:0/85908237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f111c071fa0 0x7f111c14d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:10.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.895+0000 7f111b7fe640 1 -- 172.21.15.43:0/85908237 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f111c14e550 con 0x7f111c15a750 2024-04-25T09:04:10.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.895+0000 7f111a7fc640 1 --2- 172.21.15.43:0/85908237 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f111c0af850 0x7f111c15d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:10.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.895+0000 7f111b7fe640 1 --2- 172.21.15.43:0/85908237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f111c15a750 0x7f111c14c020 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f110c007e60 tx=0x7f110c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:10.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.895+0000 7f1113fff640 1 -- 172.21.15.43:0/85908237 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110c014420 con 0x7f111c15a750 2024-04-25T09:04:10.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.895+0000 7f1113fff640 1 -- 172.21.15.43:0/85908237 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f110c019070 con 0x7f111c15a750 2024-04-25T09:04:10.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.895+0000 7f11218bc640 1 -- 172.21.15.43:0/85908237 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f111c1f2a20 con 0x7f111c15a750 2024-04-25T09:04:10.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.895+0000 7f1113fff640 1 -- 172.21.15.43:0/85908237 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f110c014820 con 0x7f111c15a750 2024-04-25T09:04:10.886 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.895+0000 7f11218bc640 1 -- 172.21.15.43:0/85908237 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f111c14cec0 con 0x7f111c15a750 2024-04-25T09:04:10.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.896+0000 7f11218bc640 1 -- 172.21.15.43:0/85908237 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f111c074f60 con 0x7f111c15a750 2024-04-25T09:04:10.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.897+0000 7f1113fff640 1 -- 172.21.15.43:0/85908237 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f110c0149c0 con 0x7f111c15a750 2024-04-25T09:04:10.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.897+0000 7f1113fff640 1 --2- 172.21.15.43:0/85908237 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f10fc06b4d0 0x7f10fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:10.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.897+0000 7f1113fff640 1 -- 172.21.15.43:0/85908237 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f110c08a710 con 0x7f111c15a750 2024-04-25T09:04:10.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.897+0000 7f111affd640 1 --2- 172.21.15.43:0/85908237 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f10fc06b4d0 0x7f10fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:10.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.898+0000 7f111affd640 1 --2- 172.21.15.43:0/85908237 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f10fc06b4d0 0x7f10fc06d990 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f1114009a50 tx=0x7f1114008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:10.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:10.901+0000 7f1113fff640 1 -- 172.21.15.43:0/85908237 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f110c0598e0 con 0x7f111c15a750 2024-04-25T09:04:11.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.082+0000 7f11218bc640 1 -- 172.21.15.43:0/85908237 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f111c15efb0 con 0x7f111c15a750 2024-04-25T09:04:11.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.083+0000 7f1113fff640 1 -- 172.21.15.43:0/85908237 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v135) v1 ==== 93+0+10 (secure 0 0 0) 0x7f110c0591e0 con 0x7f111c15a750 2024-04-25T09:04:11.071 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T09:04:11.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.085+0000 7f11218bc640 1 -- 172.21.15.43:0/85908237 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f10fc06b4d0 msgr2=0x7f10fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.085+0000 7f11218bc640 1 --2- 172.21.15.43:0/85908237 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f10fc06b4d0 0x7f10fc06d990 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f1114009a50 tx=0x7f1114008040 comp rx=0 tx=0).stop 2024-04-25T09:04:11.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.085+0000 7f11218bc640 1 -- 172.21.15.43:0/85908237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f111c15a750 msgr2=0x7f111c14c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.085+0000 7f11218bc640 1 --2- 172.21.15.43:0/85908237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f111c15a750 0x7f111c14c020 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f110c007e60 tx=0x7f110c00e4b0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.085+0000 7f11218bc640 1 -- 172.21.15.43:0/85908237 shutdown_connections 2024-04-25T09:04:11.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.085+0000 7f11218bc640 1 --2- 172.21.15.43:0/85908237 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f111c15a750 0x7f111c14c020 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.085+0000 7f11218bc640 1 --2- 172.21.15.43:0/85908237 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f111c0af850 0x7f111c15d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.085+0000 7f11218bc640 1 --2- 172.21.15.43:0/85908237 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f10fc06b4d0 0x7f10fc06d990 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.085+0000 7f11218bc640 1 --2- 172.21.15.43:0/85908237 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f111c071fa0 0x7f111c14d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.085+0000 7f11218bc640 1 -- 172.21.15.43:0/85908237 >> 172.21.15.43:0/85908237 conn(0x7f111c06c060 msgr2=0x7f111c0b1940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:11.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.085+0000 7f11218bc640 1 -- 172.21.15.43:0/85908237 shutdown_connections 2024-04-25T09:04:11.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.085+0000 7f11218bc640 1 -- 172.21.15.43:0/85908237 wait complete. 2024-04-25T09:04:11.085 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T09:04:11.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.198+0000 7f68b55ab640 1 Processor -- start 2024-04-25T09:04:11.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.198+0000 7f68b55ab640 1 -- start start 2024-04-25T09:04:11.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.198+0000 7f68b55ab640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f68b015ef50 0x7f68b00d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.198+0000 7f68b55ab640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f68b00d0920 0x7f68b0160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.198+0000 7f68b55ab640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68b00d0d30 con 0x7f68b015dce0 2024-04-25T09:04:11.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.198+0000 7f68b55ab640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68b00cec60 con 0x7f68b015ef50 2024-04-25T09:04:11.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.198+0000 7f68b55ab640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68b00d2920 con 0x7f68b00d0920 2024-04-25T09:04:11.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.199+0000 7f68ae7fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f68b015ef50 0x7f68b00d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.199+0000 7f68aeffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f68b015dce0 0x7f68b00ce550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45674/0 (socket says 172.21.15.43:45674) 2024-04-25T09:04:11.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.199+0000 7f68aeffd640 1 -- 172.21.15.43:0/3280765942 learned_addr learned my addr 172.21.15.43:0/3280765942 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:11.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.199+0000 7f68af7fe640 1 --2- 172.21.15.43:0/3280765942 >> v2:172.21.15.193:3300/0 conn(0x7f68b00d0920 0x7f68b0160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.199+0000 7f68ae7fc640 1 -- 172.21.15.43:0/3280765942 >> v2:172.21.15.193:3300/0 conn(0x7f68b00d0920 msgr2=0x7f68b0160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:11.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.199+0000 7f68ae7fc640 1 --2- 172.21.15.43:0/3280765942 >> v2:172.21.15.193:3300/0 conn(0x7f68b00d0920 0x7f68b0160980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.199+0000 7f68ae7fc640 1 -- 172.21.15.43:0/3280765942 >> v1:172.21.15.43:6789/0 conn(0x7f68b015dce0 legacy=0x7f68b00ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.199+0000 7f68adffb640 1 -- 172.21.15.43:0/3280765942 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3342617027 0 0) 0x7f68a0001110 con 0x7f68b015dce0 2024-04-25T09:04:11.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.199+0000 7f68ae7fc640 1 -- 172.21.15.43:0/3280765942 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b00cee80 con 0x7f68b015ef50 2024-04-25T09:04:11.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.200+0000 7f68ae7fc640 1 --2- 172.21.15.43:0/3280765942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68b015ef50 0x7f68b00d03e0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f68b00b5470 tx=0x7f68a402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd5a2fdc82f2b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:11.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.200+0000 7f68adffb640 1 -- 172.21.15.43:0/3280765942 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a4037070 con 0x7f68b015ef50 2024-04-25T09:04:11.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.200+0000 7f68adffb640 1 -- 172.21.15.43:0/3280765942 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a4032440 con 0x7f68b015ef50 2024-04-25T09:04:11.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.200+0000 7f68adffb640 1 -- 172.21.15.43:0/3280765942 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a402ae60 con 0x7f68b015ef50 2024-04-25T09:04:11.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.200+0000 7f68b55ab640 1 -- 172.21.15.43:0/3280765942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68b015ef50 msgr2=0x7f68b00d03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.200+0000 7f68b55ab640 1 --2- 172.21.15.43:0/3280765942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68b015ef50 0x7f68b00d03e0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f68b00b5470 tx=0x7f68a402a110 comp rx=0 tx=0).stop 2024-04-25T09:04:11.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.200+0000 7f68b55ab640 1 -- 172.21.15.43:0/3280765942 shutdown_connections 2024-04-25T09:04:11.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.200+0000 7f68b55ab640 1 --2- 172.21.15.43:0/3280765942 >> v2:172.21.15.193:3300/0 conn(0x7f68b00d0920 0x7f68b0160980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.200+0000 7f68b55ab640 1 --2- 172.21.15.43:0/3280765942 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68b015ef50 0x7f68b00d03e0 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.200+0000 7f68b55ab640 1 -- 172.21.15.43:0/3280765942 >> 172.21.15.43:0/3280765942 conn(0x7f68b009cd00 msgr2=0x7f68b00d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:11.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.200+0000 7f68b55ab640 1 -- 172.21.15.43:0/3280765942 shutdown_connections 2024-04-25T09:04:11.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.200+0000 7f68b55ab640 1 -- 172.21.15.43:0/3280765942 wait complete. 2024-04-25T09:04:11.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.202+0000 7f68b55ab640 1 Processor -- start 2024-04-25T09:04:11.191 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.202+0000 7f68b55ab640 1 -- start start 2024-04-25T09:04:11.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.202+0000 7f68b55ab640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f68b00d0920 0x7f68b0154450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.202+0000 7f68b55ab640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68b015dce0 0x7f68b0157e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.202+0000 7f68b55ab640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68b015ef50 0x7f68b0159b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.202+0000 7f68b55ab640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b00cf720 con 0x7f68b015ef50 2024-04-25T09:04:11.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.202+0000 7f68b55ab640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b00cf440 con 0x7f68b015dce0 2024-04-25T09:04:11.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.202+0000 7f68b55ab640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b015f6b0 con 0x7f68b00d0920 2024-04-25T09:04:11.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.203+0000 7f68af7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68b015ef50 0x7f68b0159b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.203+0000 7f68af7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68b015ef50 0x7f68b0159b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47434/0 (socket says 172.21.15.43:47434) 2024-04-25T09:04:11.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.203+0000 7f68af7fe640 1 -- 172.21.15.43:0/3272259214 learned_addr learned my addr 172.21.15.43:0/3272259214 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:11.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.203+0000 7f68aeffd640 1 --2- 172.21.15.43:0/3272259214 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f68b00d0920 0x7f68b0154450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.203+0000 7f68ae7fc640 1 --2- 172.21.15.43:0/3272259214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68b015dce0 0x7f68b0157e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.203+0000 7f68af7fe640 1 -- 172.21.15.43:0/3272259214 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f68b00d0920 msgr2=0x7f68b0154450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.203+0000 7f68af7fe640 1 --2- 172.21.15.43:0/3272259214 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f68b00d0920 0x7f68b0154450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.203+0000 7f68af7fe640 1 -- 172.21.15.43:0/3272259214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68b015dce0 msgr2=0x7f68b0157e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.203+0000 7f68af7fe640 1 --2- 172.21.15.43:0/3272259214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68b015dce0 0x7f68b0157e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.203+0000 7f68af7fe640 1 -- 172.21.15.43:0/3272259214 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b00b67b0 con 0x7f68b015ef50 2024-04-25T09:04:11.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.203+0000 7f68af7fe640 1 --2- 172.21.15.43:0/3272259214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68b015ef50 0x7f68b0159b30 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f68a8007e60 tx=0x7f68a800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:11.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.203+0000 7f68977fe640 1 -- 172.21.15.43:0/3272259214 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a8014420 con 0x7f68b015ef50 2024-04-25T09:04:11.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.203+0000 7f68977fe640 1 -- 172.21.15.43:0/3272259214 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a8019070 con 0x7f68b015ef50 2024-04-25T09:04:11.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.203+0000 7f68b55ab640 1 -- 172.21.15.43:0/3272259214 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68b00b6a90 con 0x7f68b015ef50 2024-04-25T09:04:11.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.203+0000 7f68977fe640 1 -- 172.21.15.43:0/3272259214 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a8014820 con 0x7f68b015ef50 2024-04-25T09:04:11.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.203+0000 7f68b55ab640 1 -- 172.21.15.43:0/3272259214 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68b00b7050 con 0x7f68b015ef50 2024-04-25T09:04:11.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.204+0000 7f68b55ab640 1 -- 172.21.15.43:0/3272259214 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68b00680f0 con 0x7f68b015ef50 2024-04-25T09:04:11.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.205+0000 7f68977fe640 1 -- 172.21.15.43:0/3272259214 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f68a80149c0 con 0x7f68b015ef50 2024-04-25T09:04:11.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.205+0000 7f68977fe640 1 --2- 172.21.15.43:0/3272259214 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f689c06b4d0 0x7f689c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.205+0000 7f68977fe640 1 -- 172.21.15.43:0/3272259214 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f68a808a7a0 con 0x7f68b015ef50 2024-04-25T09:04:11.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.205+0000 7f68aeffd640 1 --2- 172.21.15.43:0/3272259214 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f689c06b4d0 0x7f689c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.206+0000 7f68aeffd640 1 --2- 172.21.15.43:0/3272259214 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f689c06b4d0 0x7f689c06d990 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f68a0000f60 tx=0x7f68a0007970 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:11.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.210+0000 7f68977fe640 1 -- 172.21.15.43:0/3272259214 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f68a80598f0 con 0x7f68b015ef50 2024-04-25T09:04:11.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.394+0000 7f68b55ab640 1 -- 172.21.15.43:0/3272259214 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f68b0074ef0 con 0x7f68b015ef50 2024-04-25T09:04:11.383 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.394+0000 7f68977fe640 1 -- 172.21.15.43:0/3272259214 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v135) v1 ==== 84+0+65 (secure 0 0 0) 0x7f68a80591f0 con 0x7f68b015ef50 2024-04-25T09:04:11.383 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:04:11.383 INFO:teuthology.orchestra.run.smithi043.stdout:[ 2024-04-25T09:04:11.383 INFO:teuthology.orchestra.run.smithi043.stdout: { 2024-04-25T09:04:11.383 INFO:teuthology.orchestra.run.smithi043.stdout: "poolnum": 1, 2024-04-25T09:04:11.383 INFO:teuthology.orchestra.run.smithi043.stdout: "poolname": ".mgr" 2024-04-25T09:04:11.383 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:04:11.384 INFO:teuthology.orchestra.run.smithi043.stdout:] 2024-04-25T09:04:11.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.396+0000 7f68b55ab640 1 -- 172.21.15.43:0/3272259214 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f689c06b4d0 msgr2=0x7f689c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.396+0000 7f68b55ab640 1 --2- 172.21.15.43:0/3272259214 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f689c06b4d0 0x7f689c06d990 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f68a0000f60 tx=0x7f68a0007970 comp rx=0 tx=0).stop 2024-04-25T09:04:11.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.396+0000 7f68b55ab640 1 -- 172.21.15.43:0/3272259214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68b015ef50 msgr2=0x7f68b0159b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.396+0000 7f68b55ab640 1 --2- 172.21.15.43:0/3272259214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68b015ef50 0x7f68b0159b30 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f68a8007e60 tx=0x7f68a800e4b0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.385 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.396+0000 7f68b55ab640 1 -- 172.21.15.43:0/3272259214 shutdown_connections 2024-04-25T09:04:11.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.396+0000 7f68b55ab640 1 --2- 172.21.15.43:0/3272259214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f68b015ef50 0x7f68b0159b30 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.396+0000 7f68b55ab640 1 --2- 172.21.15.43:0/3272259214 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f689c06b4d0 0x7f689c06d990 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.396+0000 7f68b55ab640 1 --2- 172.21.15.43:0/3272259214 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f68b015dce0 0x7f68b0157e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.396+0000 7f68b55ab640 1 --2- 172.21.15.43:0/3272259214 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f68b00d0920 0x7f68b0154450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.396+0000 7f68b55ab640 1 -- 172.21.15.43:0/3272259214 >> 172.21.15.43:0/3272259214 conn(0x7f68b009cd00 msgr2=0x7f68b00ce510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:11.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.396+0000 7f68b55ab640 1 -- 172.21.15.43:0/3272259214 shutdown_connections 2024-04-25T09:04:11.386 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.396+0000 7f68b55ab640 1 -- 172.21.15.43:0/3272259214 wait complete. 2024-04-25T09:04:11.396 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T09:04:11.396 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T09:04:11.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.512+0000 7fc7d7734640 1 Processor -- start 2024-04-25T09:04:11.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.512+0000 7fc7d7734640 1 -- start start 2024-04-25T09:04:11.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.513+0000 7fc7d7734640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fc7d00d3e70 0x7fc7d015e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.513+0000 7fc7d7734640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7d00b37a0 con 0x7fc7d015baa0 2024-04-25T09:04:11.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.513+0000 7fc7d7734640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7d015eab0 con 0x7fc7d015b6c0 2024-04-25T09:04:11.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.513+0000 7fc7d7734640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7d015fce0 con 0x7fc7d00d3e70 2024-04-25T09:04:11.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.513+0000 7fc7d4ccf640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fc7d015baa0 0x7fc7d00d3760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45682/0 (socket says 172.21.15.43:45682) 2024-04-25T09:04:11.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.513+0000 7fc7d4ccf640 1 -- 172.21.15.43:0/3420249470 learned_addr learned my addr 172.21.15.43:0/3420249470 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:11.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.513+0000 7fc7d5cd1640 1 --2- 172.21.15.43:0/3420249470 >> v2:172.21.15.193:3300/0 conn(0x7fc7d00d3e70 0x7fc7d015e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.513+0000 7fc7c7fff640 1 -- 172.21.15.43:0/3420249470 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2532224510 0 0) 0x7fc7d00b37a0 con 0x7fc7d015baa0 2024-04-25T09:04:11.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.513+0000 7fc7c7fff640 1 -- 172.21.15.43:0/3420249470 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7bc003520 con 0x7fc7d015baa0 2024-04-25T09:04:11.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.514+0000 7fc7c7fff640 1 -- 172.21.15.43:0/3420249470 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1636050090 0 0) 0x7fc7d015eab0 con 0x7fc7d015b6c0 2024-04-25T09:04:11.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.514+0000 7fc7c7fff640 1 -- 172.21.15.43:0/3420249470 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7d00b37a0 con 0x7fc7d015b6c0 2024-04-25T09:04:11.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.514+0000 7fc7c7fff640 1 -- 172.21.15.43:0/3420249470 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2232862291 0 0) 0x7fc7bc003520 con 0x7fc7d015baa0 2024-04-25T09:04:11.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.514+0000 7fc7c7fff640 1 -- 172.21.15.43:0/3420249470 >> v2:172.21.15.193:3300/0 conn(0x7fc7d00d3e70 msgr2=0x7fc7d015e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:11.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.514+0000 7fc7c7fff640 1 --2- 172.21.15.43:0/3420249470 >> v2:172.21.15.193:3300/0 conn(0x7fc7d00d3e70 0x7fc7d015e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.514+0000 7fc7c7fff640 1 -- 172.21.15.43:0/3420249470 >> v1:172.21.15.193:6789/0 conn(0x7fc7d015b6c0 legacy=0x7fc7d015f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.514+0000 7fc7c7fff640 1 -- 172.21.15.43:0/3420249470 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d015ed90 con 0x7fc7d015baa0 2024-04-25T09:04:11.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.514+0000 7fc7c7fff640 1 -- 172.21.15.43:0/3420249470 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7fc7c0001690 con 0x7fc7d015baa0 2024-04-25T09:04:11.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.514+0000 7fc7c7fff640 1 -- 172.21.15.43:0/3420249470 >> v1:172.21.15.43:6789/0 conn(0x7fc7d015baa0 legacy=0x7fc7d00d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.514+0000 7fc7c7fff640 1 --2- 172.21.15.43:0/3420249470 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7bc003b40 0x7fc7bc003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.514+0000 7fc7c7fff640 1 --2- 172.21.15.43:0/3420249470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7bc004800 0x7fc7bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.514+0000 7fc7c7fff640 1 --2- 172.21.15.43:0/3420249470 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7bc0051c0 0x7fc7bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.514+0000 7fc7c7fff640 1 -- 172.21.15.43:0/3420249470 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d015fce0 con 0x7fc7bc004800 2024-04-25T09:04:11.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.514+0000 7fc7c7fff640 1 -- 172.21.15.43:0/3420249470 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7bc00c040 con 0x7fc7bc0051c0 2024-04-25T09:04:11.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.514+0000 7fc7c7fff640 1 -- 172.21.15.43:0/3420249470 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7bc00c240 con 0x7fc7bc003b40 2024-04-25T09:04:11.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.514+0000 7fc7d5cd1640 1 --2- 172.21.15.43:0/3420249470 >> v2:172.21.15.193:3300/0 conn(0x7fc7d00d3e70 0x7fc7d015e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:04:11.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.514+0000 7fc7d4ccf640 1 --2- 172.21.15.43:0/3420249470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7bc004800 0x7fc7bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.514+0000 7fc7d54d0640 1 --2- 172.21.15.43:0/3420249470 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7bc003b40 0x7fc7bc003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.514+0000 7fc7d5cd1640 1 --2- 172.21.15.43:0/3420249470 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7bc0051c0 0x7fc7bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.514+0000 7fc7d4ccf640 1 -- 172.21.15.43:0/3420249470 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7bc003b40 msgr2=0x7fc7bc003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:11.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.514+0000 7fc7d4ccf640 1 --2- 172.21.15.43:0/3420249470 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7bc003b40 0x7fc7bc003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.514+0000 7fc7d4ccf640 1 -- 172.21.15.43:0/3420249470 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7bc0051c0 msgr2=0x7fc7bc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:11.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.514+0000 7fc7d4ccf640 1 --2- 172.21.15.43:0/3420249470 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7bc0051c0 0x7fc7bc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.514+0000 7fc7d4ccf640 1 -- 172.21.15.43:0/3420249470 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7bc003520 con 0x7fc7bc004800 2024-04-25T09:04:11.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.514+0000 7fc7d5cd1640 1 --2- 172.21.15.43:0/3420249470 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7bc0051c0 0x7fc7bc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:11.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.514+0000 7fc7d5cd1640 1 -- 172.21.15.43:0/3420249470 reap_dead start 2024-04-25T09:04:11.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.515+0000 7fc7d4ccf640 1 --2- 172.21.15.43:0/3420249470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7bc004800 0x7fc7bc004c80 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fc7c000c160 tx=0x7fc7bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d99e28b5569014ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:11.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.515+0000 7fc7c7fff640 1 -- 172.21.15.43:0/3420249470 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c0007cc0 con 0x7fc7bc004800 2024-04-25T09:04:11.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.515+0000 7fc7c7fff640 1 -- 172.21.15.43:0/3420249470 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c0005af0 con 0x7fc7bc004800 2024-04-25T09:04:11.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.515+0000 7fc7c7fff640 1 -- 172.21.15.43:0/3420249470 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c0007070 con 0x7fc7bc004800 2024-04-25T09:04:11.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.516+0000 7fc7d7734640 1 -- 172.21.15.43:0/3420249470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7bc004800 msgr2=0x7fc7bc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.516+0000 7fc7d7734640 1 --2- 172.21.15.43:0/3420249470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7bc004800 0x7fc7bc004c80 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7fc7c000c160 tx=0x7fc7bc005830 comp rx=0 tx=0).stop 2024-04-25T09:04:11.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.516+0000 7fc7d7734640 1 -- 172.21.15.43:0/3420249470 shutdown_connections 2024-04-25T09:04:11.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.516+0000 7fc7d7734640 1 --2- 172.21.15.43:0/3420249470 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7bc004800 0x7fc7bc004c80 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.516+0000 7fc7d7734640 1 -- 172.21.15.43:0/3420249470 >> 172.21.15.43:0/3420249470 conn(0x7fc7d008d460 msgr2=0x7fc7d009cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:11.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.516+0000 7fc7d7734640 1 -- 172.21.15.43:0/3420249470 shutdown_connections 2024-04-25T09:04:11.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.516+0000 7fc7d7734640 1 -- 172.21.15.43:0/3420249470 wait complete. 2024-04-25T09:04:11.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.517+0000 7fc7d7734640 1 Processor -- start 2024-04-25T09:04:11.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.517+0000 7fc7d7734640 1 -- start start 2024-04-25T09:04:11.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.518+0000 7fc7d7734640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7bc004800 0x7fc7d00c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.518+0000 7fc7d7734640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7d014a5b0 0x7fc7d00cfa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.518+0000 7fc7d7734640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7d00cff50 0x7fc7d014ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.518+0000 7fc7d7734640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d015e410 con 0x7fc7bc004800 2024-04-25T09:04:11.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.518+0000 7fc7d7734640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d015f350 con 0x7fc7d014a5b0 2024-04-25T09:04:11.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.518+0000 7fc7d7734640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d015e590 con 0x7fc7d00cff50 2024-04-25T09:04:11.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.518+0000 7fc7d54d0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7bc004800 0x7fc7d00c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.518+0000 7fc7d54d0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7bc004800 0x7fc7d00c9ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47454/0 (socket says 172.21.15.43:47454) 2024-04-25T09:04:11.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.518+0000 7fc7d54d0640 1 -- 172.21.15.43:0/2145613409 learned_addr learned my addr 172.21.15.43:0/2145613409 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:11.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.518+0000 7fc7d5cd1640 1 --2- 172.21.15.43:0/2145613409 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7d00cff50 0x7fc7d014ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.518+0000 7fc7d4ccf640 1 --2- 172.21.15.43:0/2145613409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7d014a5b0 0x7fc7d00cfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.518+0000 7fc7d54d0640 1 -- 172.21.15.43:0/2145613409 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7d00cff50 msgr2=0x7fc7d014ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.518+0000 7fc7d54d0640 1 --2- 172.21.15.43:0/2145613409 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7d00cff50 0x7fc7d014ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.518+0000 7fc7d54d0640 1 -- 172.21.15.43:0/2145613409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7d014a5b0 msgr2=0x7fc7d00cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:11.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.518+0000 7fc7d54d0640 1 --2- 172.21.15.43:0/2145613409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7d014a5b0 0x7fc7d00cfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:11.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.518+0000 7fc7d54d0640 1 -- 172.21.15.43:0/2145613409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d00b90a0 con 0x7fc7bc004800 2024-04-25T09:04:11.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.518+0000 7fc7d5cd1640 1 --2- 172.21.15.43:0/2145613409 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7d00cff50 0x7fc7d014ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:11.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.518+0000 7fc7d4ccf640 1 --2- 172.21.15.43:0/2145613409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7d014a5b0 0x7fc7d00cfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:11.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.518+0000 7fc7d54d0640 1 --2- 172.21.15.43:0/2145613409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7bc004800 0x7fc7d00c9ec0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc00f850 tx=0x7fc7cc00fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:11.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.518+0000 7fc7c5ffb640 1 -- 172.21.15.43:0/2145613409 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc00fe40 con 0x7fc7bc004800 2024-04-25T09:04:11.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.518+0000 7fc7c5ffb640 1 -- 172.21.15.43:0/2145613409 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7cc018070 con 0x7fc7bc004800 2024-04-25T09:04:11.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.518+0000 7fc7c5ffb640 1 -- 172.21.15.43:0/2145613409 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc00d6d0 con 0x7fc7bc004800 2024-04-25T09:04:11.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.519+0000 7fc7d7734640 1 -- 172.21.15.43:0/2145613409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7d00b9380 con 0x7fc7bc004800 2024-04-25T09:04:11.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.519+0000 7fc7d7734640 1 -- 172.21.15.43:0/2145613409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7d014b390 con 0x7fc7bc004800 2024-04-25T09:04:11.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.520+0000 7fc7d7734640 1 -- 172.21.15.43:0/2145613409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7d0068a40 con 0x7fc7bc004800 2024-04-25T09:04:11.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.520+0000 7fc7c5ffb640 1 -- 172.21.15.43:0/2145613409 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc7cc00d870 con 0x7fc7bc004800 2024-04-25T09:04:11.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.521+0000 7fc7c5ffb640 1 --2- 172.21.15.43:0/2145613409 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc7b406b4d0 0x7fc7b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:11.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.521+0000 7fc7c5ffb640 1 -- 172.21.15.43:0/2145613409 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc7cc089e10 con 0x7fc7bc004800 2024-04-25T09:04:11.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.521+0000 7fc7d4ccf640 1 --2- 172.21.15.43:0/2145613409 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc7b406b4d0 0x7fc7b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:11.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.521+0000 7fc7d4ccf640 1 --2- 172.21.15.43:0/2145613409 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc7b406b4d0 0x7fc7b406d990 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0032000 tx=0x7fc7c0009030 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:11.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.525+0000 7fc7c5ffb640 1 -- 172.21.15.43:0/2145613409 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc7cc058fe0 con 0x7fc7bc004800 2024-04-25T09:04:11.692 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:11.704+0000 7fc7d7734640 1 -- 172.21.15.43:0/2145613409 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc7d015de20 con 0x7fc7bc004800 2024-04-25T09:04:12.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.033+0000 7fc7c5ffb640 1 -- 172.21.15.43:0/2145613409 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v136) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc7cc0588e0 con 0x7fc7bc004800 2024-04-25T09:04:12.022 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' created 2024-04-25T09:04:12.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.035+0000 7fc7d7734640 1 -- 172.21.15.43:0/2145613409 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc7b406b4d0 msgr2=0x7fc7b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.035+0000 7fc7d7734640 1 --2- 172.21.15.43:0/2145613409 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc7b406b4d0 0x7fc7b406d990 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0032000 tx=0x7fc7c0009030 comp rx=0 tx=0).stop 2024-04-25T09:04:12.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.035+0000 7fc7d7734640 1 -- 172.21.15.43:0/2145613409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7bc004800 msgr2=0x7fc7d00c9ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.035+0000 7fc7d7734640 1 --2- 172.21.15.43:0/2145613409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7bc004800 0x7fc7d00c9ec0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc00f850 tx=0x7fc7cc00fd10 comp rx=0 tx=0).stop 2024-04-25T09:04:12.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.035+0000 7fc7d7734640 1 -- 172.21.15.43:0/2145613409 shutdown_connections 2024-04-25T09:04:12.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.035+0000 7fc7d7734640 1 --2- 172.21.15.43:0/2145613409 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc7d00cff50 0x7fc7d014ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.035+0000 7fc7d7734640 1 --2- 172.21.15.43:0/2145613409 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc7b406b4d0 0x7fc7b406d990 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.024 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.035+0000 7fc7d7734640 1 --2- 172.21.15.43:0/2145613409 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7d014a5b0 0x7fc7d00cfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.035+0000 7fc7d7734640 1 --2- 172.21.15.43:0/2145613409 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7bc004800 0x7fc7d00c9ec0 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.035+0000 7fc7d7734640 1 -- 172.21.15.43:0/2145613409 >> 172.21.15.43:0/2145613409 conn(0x7fc7d008d460 msgr2=0x7fc7d015bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:12.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.035+0000 7fc7d7734640 1 -- 172.21.15.43:0/2145613409 shutdown_connections 2024-04-25T09:04:12.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.035+0000 7fc7d7734640 1 -- 172.21.15.43:0/2145613409 wait complete. 2024-04-25T09:04:12.039 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T09:04:12.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.159+0000 7fb5b9cbd640 1 Processor -- start 2024-04-25T09:04:12.148 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.159+0000 7fb5b9cbd640 1 -- start start 2024-04-25T09:04:12.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.160+0000 7fb5b9cbd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb5b40adea0 0x7fb5b406acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.160+0000 7fb5b9cbd640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5b415e480 con 0x7fb5b40afe20 2024-04-25T09:04:12.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.160+0000 7fb5b9cbd640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5b415e760 con 0x7fb5b4158110 2024-04-25T09:04:12.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.160+0000 7fb5b9cbd640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5b415ea40 con 0x7fb5b40adea0 2024-04-25T09:04:12.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.160+0000 7fb5b2ffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb5b40adea0 0x7fb5b406acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.160+0000 7fb5b3fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb5b40afe20 0x7fb5b406b230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45686/0 (socket says 172.21.15.43:45686) 2024-04-25T09:04:12.149 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.161+0000 7fb5b3fff640 1 -- 172.21.15.43:0/468610800 learned_addr learned my addr 172.21.15.43:0/468610800 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:12.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.161+0000 7fb5b2ffd640 1 -- 172.21.15.43:0/468610800 >> v1:172.21.15.193:6789/0 conn(0x7fb5b4158110 legacy=0x7fb5b40af710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.161+0000 7fb5b2ffd640 1 -- 172.21.15.43:0/468610800 >> v1:172.21.15.43:6789/0 conn(0x7fb5b40afe20 legacy=0x7fb5b406b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.161+0000 7fb5b2ffd640 1 -- 172.21.15.43:0/468610800 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b415ed20 con 0x7fb5b40adea0 2024-04-25T09:04:12.150 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.161+0000 7fb5b27fc640 1 -- 172.21.15.43:0/468610800 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2184321040 0 0) 0x7fb5b415e480 con 0x7fb5b40afe20 2024-04-25T09:04:12.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.162+0000 7fb5b2ffd640 1 --2- 172.21.15.43:0/468610800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5b40adea0 0x7fb5b406acf0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fb5a80097a0 tx=0x7fb5a802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e3f4a1a416b7af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.162+0000 7fb5b27fc640 1 -- 172.21.15.43:0/468610800 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a8037070 con 0x7fb5b40adea0 2024-04-25T09:04:12.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.162+0000 7fb5b27fc640 1 -- 172.21.15.43:0/468610800 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5a8032440 con 0x7fb5b40adea0 2024-04-25T09:04:12.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.162+0000 7fb5b27fc640 1 -- 172.21.15.43:0/468610800 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a802ae60 con 0x7fb5b40adea0 2024-04-25T09:04:12.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.162+0000 7fb5b9cbd640 1 -- 172.21.15.43:0/468610800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5b40adea0 msgr2=0x7fb5b406acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.162+0000 7fb5b9cbd640 1 --2- 172.21.15.43:0/468610800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5b40adea0 0x7fb5b406acf0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fb5a80097a0 tx=0x7fb5a802a110 comp rx=0 tx=0).stop 2024-04-25T09:04:12.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.162+0000 7fb5b9cbd640 1 -- 172.21.15.43:0/468610800 shutdown_connections 2024-04-25T09:04:12.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.162+0000 7fb5b9cbd640 1 --2- 172.21.15.43:0/468610800 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5b40adea0 0x7fb5b406acf0 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.162+0000 7fb5b9cbd640 1 -- 172.21.15.43:0/468610800 >> 172.21.15.43:0/468610800 conn(0x7fb5b406bc40 msgr2=0x7fb5b406c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:12.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.162+0000 7fb5b9cbd640 1 -- 172.21.15.43:0/468610800 shutdown_connections 2024-04-25T09:04:12.152 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.162+0000 7fb5b9cbd640 1 -- 172.21.15.43:0/468610800 wait complete. 2024-04-25T09:04:12.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.163+0000 7fb5b9cbd640 1 Processor -- start 2024-04-25T09:04:12.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.164+0000 7fb5b9cbd640 1 -- start start 2024-04-25T09:04:12.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.164+0000 7fb5b9cbd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5b40adea0 0x7fb5b40b0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.164+0000 7fb5b9cbd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5b40afe20 0x7fb5b40b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.164+0000 7fb5b9cbd640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5b4158110 0x7fb5b4156a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.164+0000 7fb5b9cbd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b415e260 con 0x7fb5b40adea0 2024-04-25T09:04:12.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.164+0000 7fb5b9cbd640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b415e0e0 con 0x7fb5b40afe20 2024-04-25T09:04:12.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.164+0000 7fb5b9cbd640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5b4168ab0 con 0x7fb5b4158110 2024-04-25T09:04:12.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.165+0000 7fb5b37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5b40adea0 0x7fb5b40b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.165+0000 7fb5b37fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5b40adea0 0x7fb5b40b0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47466/0 (socket says 172.21.15.43:47466) 2024-04-25T09:04:12.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.165+0000 7fb5b37fe640 1 -- 172.21.15.43:0/2065920199 learned_addr learned my addr 172.21.15.43:0/2065920199 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:12.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.165+0000 7fb5b2ffd640 1 --2- 172.21.15.43:0/2065920199 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5b40afe20 0x7fb5b40b1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.165+0000 7fb5b3fff640 1 --2- 172.21.15.43:0/2065920199 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5b4158110 0x7fb5b4156a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.165+0000 7fb5b37fe640 1 -- 172.21.15.43:0/2065920199 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5b4158110 msgr2=0x7fb5b4156a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.165+0000 7fb5b37fe640 1 --2- 172.21.15.43:0/2065920199 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5b4158110 0x7fb5b4156a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.165+0000 7fb5b37fe640 1 -- 172.21.15.43:0/2065920199 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5b40afe20 msgr2=0x7fb5b40b1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:12.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.165+0000 7fb5b37fe640 1 --2- 172.21.15.43:0/2065920199 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5b40afe20 0x7fb5b40b1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:12.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.165+0000 7fb5b37fe640 1 -- 172.21.15.43:0/2065920199 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5b4156f40 con 0x7fb5b40adea0 2024-04-25T09:04:12.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.165+0000 7fb5b37fe640 1 --2- 172.21.15.43:0/2065920199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5b40adea0 0x7fb5b40b0360 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400c760 tx=0x7fb5a400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.165+0000 7fb59bfff640 1 -- 172.21.15.43:0/2065920199 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a4003b70 con 0x7fb5b40adea0 2024-04-25T09:04:12.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.165+0000 7fb59bfff640 1 -- 172.21.15.43:0/2065920199 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5a4003d30 con 0x7fb5b40adea0 2024-04-25T09:04:12.155 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.165+0000 7fb59bfff640 1 -- 172.21.15.43:0/2065920199 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5a400aa90 con 0x7fb5b40adea0 2024-04-25T09:04:12.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.165+0000 7fb5b9cbd640 1 -- 172.21.15.43:0/2065920199 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5b40b0a00 con 0x7fb5b40adea0 2024-04-25T09:04:12.156 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.165+0000 7fb5b9cbd640 1 -- 172.21.15.43:0/2065920199 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5b40b0fc0 con 0x7fb5b40adea0 2024-04-25T09:04:12.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.166+0000 7fb5b9cbd640 1 -- 172.21.15.43:0/2065920199 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5b4074f60 con 0x7fb5b40adea0 2024-04-25T09:04:12.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.167+0000 7fb59bfff640 1 -- 172.21.15.43:0/2065920199 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb5a400ac30 con 0x7fb5b40adea0 2024-04-25T09:04:12.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.167+0000 7fb59bfff640 1 --2- 172.21.15.43:0/2065920199 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb5a006b500 0x7fb5a006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:12.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.167+0000 7fb59bfff640 1 -- 172.21.15.43:0/2065920199 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fb5a4089eb0 con 0x7fb5b40adea0 2024-04-25T09:04:12.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.168+0000 7fb5b2ffd640 1 --2- 172.21.15.43:0/2065920199 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb5a006b500 0x7fb5a006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:12.157 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.168+0000 7fb5b2ffd640 1 --2- 172.21.15.43:0/2065920199 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb5a006b500 0x7fb5a006d9c0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8034000 tx=0x7fb5a802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:12.160 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.172+0000 7fb59bfff640 1 -- 172.21.15.43:0/2065920199 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb5a4058fa0 con 0x7fb5b40adea0 2024-04-25T09:04:12.340 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:12.351+0000 7fb5b9cbd640 1 -- 172.21.15.43:0/2065920199 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb5b406d780 con 0x7fb5b40adea0 2024-04-25T09:04:13.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.038+0000 7fb59bfff640 1 -- 172.21.15.43:0/2065920199 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v137) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb5a40588a0 con 0x7fb5b40adea0 2024-04-25T09:04:13.027 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' created 2024-04-25T09:04:13.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.041+0000 7fb5b9cbd640 1 -- 172.21.15.43:0/2065920199 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb5a006b500 msgr2=0x7fb5a006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.041+0000 7fb5b9cbd640 1 --2- 172.21.15.43:0/2065920199 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb5a006b500 0x7fb5a006d9c0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fb5a8034000 tx=0x7fb5a802e000 comp rx=0 tx=0).stop 2024-04-25T09:04:13.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.041+0000 7fb5b9cbd640 1 -- 172.21.15.43:0/2065920199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5b40adea0 msgr2=0x7fb5b40b0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.041+0000 7fb5b9cbd640 1 --2- 172.21.15.43:0/2065920199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5b40adea0 0x7fb5b40b0360 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fb5a400c760 tx=0x7fb5a400cc20 comp rx=0 tx=0).stop 2024-04-25T09:04:13.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.041+0000 7fb5b9cbd640 1 -- 172.21.15.43:0/2065920199 shutdown_connections 2024-04-25T09:04:13.030 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.041+0000 7fb5b9cbd640 1 --2- 172.21.15.43:0/2065920199 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb5b4158110 0x7fb5b4156a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.041+0000 7fb5b9cbd640 1 --2- 172.21.15.43:0/2065920199 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb5a006b500 0x7fb5a006d9c0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.041+0000 7fb5b9cbd640 1 --2- 172.21.15.43:0/2065920199 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb5b40afe20 0x7fb5b40b1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.041+0000 7fb5b9cbd640 1 --2- 172.21.15.43:0/2065920199 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb5b40adea0 0x7fb5b40b0360 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.041+0000 7fb5b9cbd640 1 -- 172.21.15.43:0/2065920199 >> 172.21.15.43:0/2065920199 conn(0x7fb5b406bc40 msgr2=0x7fb5b4152ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:13.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.042+0000 7fb5b9cbd640 1 -- 172.21.15.43:0/2065920199 shutdown_connections 2024-04-25T09:04:13.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.042+0000 7fb5b9cbd640 1 -- 172.21.15.43:0/2065920199 wait complete. 2024-04-25T09:04:13.054 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T09:04:13.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.173+0000 7fbd5883f640 1 Processor -- start 2024-04-25T09:04:13.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.173+0000 7fbd5883f640 1 -- start start 2024-04-25T09:04:13.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.174+0000 7fbd5883f640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fbd5415ddd0 0x7fbd540b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.174+0000 7fbd5883f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbd5406d4f0 0x7fbd5406f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.174+0000 7fbd5883f640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd5406d900 con 0x7fbd5406ef90 2024-04-25T09:04:13.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.174+0000 7fbd5883f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd540d2620 con 0x7fbd5406d4f0 2024-04-25T09:04:13.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.174+0000 7fbd5883f640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd540b0b80 con 0x7fbd5415ddd0 2024-04-25T09:04:13.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.174+0000 7fbd52d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbd5406d4f0 0x7fbd5406f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.174+0000 7fbd52d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbd5406d4f0 0x7fbd5406f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47468/0 (socket says 172.21.15.43:47468) 2024-04-25T09:04:13.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.174+0000 7fbd52d9d640 1 -- 172.21.15.43:0/1046447856 learned_addr learned my addr 172.21.15.43:0/1046447856 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:13.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.174+0000 7fbd5259c640 1 --2- 172.21.15.43:0/1046447856 >> v2:172.21.15.193:3300/0 conn(0x7fbd5415ddd0 0x7fbd540b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.174+0000 7fbd52d9d640 1 -- 172.21.15.43:0/1046447856 >> v2:172.21.15.193:3300/0 conn(0x7fbd5415ddd0 msgr2=0x7fbd540b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:13.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.174+0000 7fbd52d9d640 1 --2- 172.21.15.43:0/1046447856 >> v2:172.21.15.193:3300/0 conn(0x7fbd5415ddd0 0x7fbd540b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.174+0000 7fbd52d9d640 1 -- 172.21.15.43:0/1046447856 >> v1:172.21.15.193:6789/0 conn(0x7fbd5406ef90 legacy=0x7fbd540cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:13.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.174+0000 7fbd52d9d640 1 -- 172.21.15.43:0/1046447856 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd540b0e60 con 0x7fbd5406d4f0 2024-04-25T09:04:13.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.174+0000 7fbd5259c640 1 --2- 172.21.15.43:0/1046447856 >> v2:172.21.15.193:3300/0 conn(0x7fbd5415ddd0 0x7fbd540b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:13.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.175+0000 7fbd52d9d640 1 --2- 172.21.15.43:0/1046447856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd5406d4f0 0x7fbd5406f420 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7fbd5406e510 tx=0x7fbd4c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b41a064560d1e58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.175+0000 7fbd51d9b640 1 -- 172.21.15.43:0/1046447856 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c036070 con 0x7fbd5406d4f0 2024-04-25T09:04:13.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.175+0000 7fbd51d9b640 1 -- 172.21.15.43:0/1046447856 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd4c031600 con 0x7fbd5406d4f0 2024-04-25T09:04:13.165 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.175+0000 7fbd51d9b640 1 -- 172.21.15.43:0/1046447856 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c0319d0 con 0x7fbd5406d4f0 2024-04-25T09:04:13.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.176+0000 7fbd5883f640 1 -- 172.21.15.43:0/1046447856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd5406d4f0 msgr2=0x7fbd5406f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.176+0000 7fbd5883f640 1 --2- 172.21.15.43:0/1046447856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd5406d4f0 0x7fbd5406f420 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7fbd5406e510 tx=0x7fbd4c02a6f0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.176+0000 7fbd5883f640 1 -- 172.21.15.43:0/1046447856 shutdown_connections 2024-04-25T09:04:13.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.176+0000 7fbd5883f640 1 --2- 172.21.15.43:0/1046447856 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd5406d4f0 0x7fbd5406f420 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.176+0000 7fbd5883f640 1 --2- 172.21.15.43:0/1046447856 >> v2:172.21.15.193:3300/0 conn(0x7fbd5415ddd0 0x7fbd540b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.176+0000 7fbd5883f640 1 -- 172.21.15.43:0/1046447856 >> 172.21.15.43:0/1046447856 conn(0x7fbd540d17e0 msgr2=0x7fbd540d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:13.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.176+0000 7fbd5883f640 1 -- 172.21.15.43:0/1046447856 shutdown_connections 2024-04-25T09:04:13.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.176+0000 7fbd5883f640 1 -- 172.21.15.43:0/1046447856 wait complete. 2024-04-25T09:04:13.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.177+0000 7fbd5883f640 1 Processor -- start 2024-04-25T09:04:13.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.177+0000 7fbd5883f640 1 -- start start 2024-04-25T09:04:13.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.178+0000 7fbd5883f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd5406d4f0 0x7fbd540b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.178+0000 7fbd5883f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd5406ef90 0x7fbd5415a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.178+0000 7fbd5883f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd5415ddd0 0x7fbd540c8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.178+0000 7fbd5883f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd540cccf0 con 0x7fbd5406d4f0 2024-04-25T09:04:13.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.178+0000 7fbd5883f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd540ccb70 con 0x7fbd5406ef90 2024-04-25T09:04:13.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.178+0000 7fbd5883f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd540cce70 con 0x7fbd5415ddd0 2024-04-25T09:04:13.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.178+0000 7fbd5259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd5406d4f0 0x7fbd540b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.178+0000 7fbd5259c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd5406d4f0 0x7fbd540b1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47478/0 (socket says 172.21.15.43:47478) 2024-04-25T09:04:13.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.178+0000 7fbd5259c640 1 -- 172.21.15.43:0/3468519219 learned_addr learned my addr 172.21.15.43:0/3468519219 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:13.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.178+0000 7fbd5259c640 1 -- 172.21.15.43:0/3468519219 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd5415ddd0 msgr2=0x7fbd540c8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.178+0000 7fbd5259c640 1 --2- 172.21.15.43:0/3468519219 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd5415ddd0 0x7fbd540c8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.178+0000 7fbd5259c640 1 -- 172.21.15.43:0/3468519219 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd5406ef90 msgr2=0x7fbd5415a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:13.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.178+0000 7fbd4bfff640 1 --2- 172.21.15.43:0/3468519219 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd5406ef90 0x7fbd5415a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.178+0000 7fbd5259c640 1 --2- 172.21.15.43:0/3468519219 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd5406ef90 0x7fbd5415a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:13.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.178+0000 7fbd5259c640 1 -- 172.21.15.43:0/3468519219 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd540c8d60 con 0x7fbd5406d4f0 2024-04-25T09:04:13.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.178+0000 7fbd5259c640 1 --2- 172.21.15.43:0/3468519219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd5406d4f0 0x7fbd540b1c20 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7fbd4400c7f0 tx=0x7fbd4400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.178+0000 7fbd4affd640 1 -- 172.21.15.43:0/3468519219 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd44015070 con 0x7fbd5406d4f0 2024-04-25T09:04:13.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.179+0000 7fbd4affd640 1 -- 172.21.15.43:0/3468519219 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd44007820 con 0x7fbd5406d4f0 2024-04-25T09:04:13.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.179+0000 7fbd5883f640 1 -- 172.21.15.43:0/3468519219 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd540c9000 con 0x7fbd5406d4f0 2024-04-25T09:04:13.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.179+0000 7fbd4affd640 1 -- 172.21.15.43:0/3468519219 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd44007bd0 con 0x7fbd5406d4f0 2024-04-25T09:04:13.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.179+0000 7fbd5883f640 1 -- 172.21.15.43:0/3468519219 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd540c95c0 con 0x7fbd5406d4f0 2024-04-25T09:04:13.170 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.179+0000 7fbd5883f640 1 -- 172.21.15.43:0/3468519219 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd54074f60 con 0x7fbd5406d4f0 2024-04-25T09:04:13.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.180+0000 7fbd4affd640 1 -- 172.21.15.43:0/3468519219 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbd44018070 con 0x7fbd5406d4f0 2024-04-25T09:04:13.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.180+0000 7fbd4affd640 1 --2- 172.21.15.43:0/3468519219 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbd3406b4d0 0x7fbd3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:13.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.181+0000 7fbd4affd640 1 -- 172.21.15.43:0/3468519219 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fbd4408a2a0 con 0x7fbd5406d4f0 2024-04-25T09:04:13.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.181+0000 7fbd4bfff640 1 --2- 172.21.15.43:0/3468519219 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbd3406b4d0 0x7fbd3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:13.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.181+0000 7fbd4bfff640 1 --2- 172.21.15.43:0/3468519219 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbd3406b4d0 0x7fbd3406d990 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fbd540b7900 tx=0x7fbd3c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:13.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.185+0000 7fbd4affd640 1 -- 172.21.15.43:0/3468519219 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbd44059190 con 0x7fbd5406d4f0 2024-04-25T09:04:13.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:13.408+0000 7fbd5883f640 1 -- 172.21.15.43:0/3468519219 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fbd5406f520 con 0x7fbd5406d4f0 2024-04-25T09:04:14.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.073+0000 7fbd4affd640 1 -- 172.21.15.43:0/3468519219 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v85) v1 ==== 174+0+0 (secure 0 0 0) 0x7fbd44058a90 con 0x7fbd5406d4f0 2024-04-25T09:04:14.063 INFO:teuthology.orchestra.run.smithi043.stderr:new fs with metadata pool 20 and data pool 21 2024-04-25T09:04:14.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.080+0000 7fbd5883f640 1 -- 172.21.15.43:0/3468519219 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbd3406b4d0 msgr2=0x7fbd3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:14.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.080+0000 7fbd5883f640 1 --2- 172.21.15.43:0/3468519219 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbd3406b4d0 0x7fbd3406d990 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fbd540b7900 tx=0x7fbd3c009040 comp rx=0 tx=0).stop 2024-04-25T09:04:14.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.080+0000 7fbd5883f640 1 -- 172.21.15.43:0/3468519219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd5406d4f0 msgr2=0x7fbd540b1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:14.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.080+0000 7fbd5883f640 1 --2- 172.21.15.43:0/3468519219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd5406d4f0 0x7fbd540b1c20 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7fbd4400c7f0 tx=0x7fbd4400ccb0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.081+0000 7fbd5883f640 1 -- 172.21.15.43:0/3468519219 shutdown_connections 2024-04-25T09:04:14.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.081+0000 7fbd5883f640 1 --2- 172.21.15.43:0/3468519219 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbd5415ddd0 0x7fbd540c8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.081+0000 7fbd5883f640 1 --2- 172.21.15.43:0/3468519219 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbd3406b4d0 0x7fbd3406d990 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.070 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.081+0000 7fbd5883f640 1 --2- 172.21.15.43:0/3468519219 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbd5406ef90 0x7fbd5415a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.081+0000 7fbd5883f640 1 --2- 172.21.15.43:0/3468519219 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbd5406d4f0 0x7fbd540b1c20 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.081+0000 7fbd5883f640 1 -- 172.21.15.43:0/3468519219 >> 172.21.15.43:0/3468519219 conn(0x7fbd540d17e0 msgr2=0x7fbd540cd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:14.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.082+0000 7fbd5883f640 1 -- 172.21.15.43:0/3468519219 shutdown_connections 2024-04-25T09:04:14.071 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.082+0000 7fbd5883f640 1 -- 172.21.15.43:0/3468519219 wait complete. 2024-04-25T09:04:14.091 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:04:14.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.209+0000 7f1ad5df9640 1 Processor -- start 2024-04-25T09:04:14.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.210+0000 7f1ad5df9640 1 -- start start 2024-04-25T09:04:14.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.210+0000 7f1ad5df9640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1ad015f180 0x7f1ad00cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:14.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.210+0000 7f1ad5df9640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f1ad00b23f0 0x7f1ad00cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:14.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.210+0000 7f1ad5df9640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1ad00b2cc0 0x7f1ad015f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:14.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.210+0000 7f1ad5df9640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ad00cccc0 con 0x7f1ad015f180 2024-04-25T09:04:14.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.210+0000 7f1ad5df9640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ad00cd840 con 0x7f1ad00b23f0 2024-04-25T09:04:14.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.210+0000 7f1ad5df9640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad00ccf10 con 0x7f1ad00b2cc0 2024-04-25T09:04:14.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.210+0000 7f1acf7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1ad015f180 0x7f1ad00cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:14.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.210+0000 7f1acf7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1ad015f180 0x7f1ad00cf860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47492/0 (socket says 172.21.15.43:47492) 2024-04-25T09:04:14.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.211+0000 7f1acf7fe640 1 -- 172.21.15.43:0/629528924 learned_addr learned my addr 172.21.15.43:0/629528924 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:14.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.211+0000 7f1aceffd640 1 --2- 172.21.15.43:0/629528924 >> v2:172.21.15.193:3300/0 conn(0x7f1ad00b23f0 0x7f1ad00cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:14.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.211+0000 7f1acffff640 1 --2- 172.21.15.43:0/629528924 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1ad00b2cc0 0x7f1ad015f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:14.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.211+0000 7f1acf7fe640 1 -- 172.21.15.43:0/629528924 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1ad00b2cc0 msgr2=0x7f1ad015f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:14.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.211+0000 7f1acf7fe640 1 --2- 172.21.15.43:0/629528924 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1ad00b2cc0 0x7f1ad015f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.211+0000 7f1acf7fe640 1 -- 172.21.15.43:0/629528924 >> v2:172.21.15.193:3300/0 conn(0x7f1ad00b23f0 msgr2=0x7f1ad00cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:14.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.211+0000 7f1acf7fe640 1 --2- 172.21.15.43:0/629528924 >> v2:172.21.15.193:3300/0 conn(0x7f1ad00b23f0 0x7f1ad00cfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.211+0000 7f1acf7fe640 1 -- 172.21.15.43:0/629528924 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad00cd1f0 con 0x7f1ad015f180 2024-04-25T09:04:14.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.211+0000 7f1aceffd640 1 --2- 172.21.15.43:0/629528924 >> v2:172.21.15.193:3300/0 conn(0x7f1ad00b23f0 0x7f1ad00cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:14.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.211+0000 7f1acffff640 1 --2- 172.21.15.43:0/629528924 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1ad00b2cc0 0x7f1ad015f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:14.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.212+0000 7f1acf7fe640 1 --2- 172.21.15.43:0/629528924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ad015f180 0x7f1ad00cf860 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0071a40 tx=0x7f1ac002a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fb23b063dba7a14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:14.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.212+0000 7f1ace7fc640 1 -- 172.21.15.43:0/629528924 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0037070 con 0x7f1ad015f180 2024-04-25T09:04:14.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.212+0000 7f1ace7fc640 1 -- 172.21.15.43:0/629528924 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac0032440 con 0x7f1ad015f180 2024-04-25T09:04:14.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.212+0000 7f1ace7fc640 1 -- 172.21.15.43:0/629528924 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0032810 con 0x7f1ad015f180 2024-04-25T09:04:14.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.212+0000 7f1ad5df9640 1 -- 172.21.15.43:0/629528924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ad015f180 msgr2=0x7f1ad00cf860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:14.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.212+0000 7f1ad5df9640 1 --2- 172.21.15.43:0/629528924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ad015f180 0x7f1ad00cf860 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7f1ad0071a40 tx=0x7f1ac002a140 comp rx=0 tx=0).stop 2024-04-25T09:04:14.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.212+0000 7f1ad5df9640 1 -- 172.21.15.43:0/629528924 shutdown_connections 2024-04-25T09:04:14.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.212+0000 7f1ad5df9640 1 --2- 172.21.15.43:0/629528924 >> v2:172.21.15.193:3300/0 conn(0x7f1ad00b23f0 0x7f1ad00cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.212+0000 7f1ad5df9640 1 --2- 172.21.15.43:0/629528924 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1ad00b2cc0 0x7f1ad015f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.212+0000 7f1ad5df9640 1 --2- 172.21.15.43:0/629528924 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ad015f180 0x7f1ad00cf860 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.212+0000 7f1ad5df9640 1 -- 172.21.15.43:0/629528924 >> 172.21.15.43:0/629528924 conn(0x7f1ad00d17f0 msgr2=0x7f1ad00d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:14.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.212+0000 7f1ad5df9640 1 -- 172.21.15.43:0/629528924 shutdown_connections 2024-04-25T09:04:14.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.212+0000 7f1ad5df9640 1 -- 172.21.15.43:0/629528924 wait complete. 2024-04-25T09:04:14.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.213+0000 7f1ad5df9640 1 Processor -- start 2024-04-25T09:04:14.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.214+0000 7f1ad5df9640 1 -- start start 2024-04-25T09:04:14.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.214+0000 7f1ad5df9640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ad00b23f0 0x7f1ad015b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:14.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.214+0000 7f1ad5df9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ad00b2cc0 0x7f1ad00b0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:14.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.214+0000 7f1ad5df9640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1ad015f180 0x7f1ad014ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:14.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.214+0000 7f1ad5df9640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad015f560 con 0x7f1ad00b2cc0 2024-04-25T09:04:14.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.214+0000 7f1ad5df9640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad00d23f0 con 0x7f1ad00b23f0 2024-04-25T09:04:14.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.214+0000 7f1ad5df9640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad00b1470 con 0x7f1ad015f180 2024-04-25T09:04:14.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.214+0000 7f1aceffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ad00b2cc0 0x7f1ad00b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:14.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.214+0000 7f1aceffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ad00b2cc0 0x7f1ad00b0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47502/0 (socket says 172.21.15.43:47502) 2024-04-25T09:04:14.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.214+0000 7f1aceffd640 1 -- 172.21.15.43:0/3494478601 learned_addr learned my addr 172.21.15.43:0/3494478601 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:14.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.215+0000 7f1acf7fe640 1 --2- 172.21.15.43:0/3494478601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ad00b23f0 0x7f1ad015b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:14.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.215+0000 7f1acffff640 1 --2- 172.21.15.43:0/3494478601 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1ad015f180 0x7f1ad014ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:14.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.215+0000 7f1aceffd640 1 -- 172.21.15.43:0/3494478601 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1ad015f180 msgr2=0x7f1ad014ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:14.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.215+0000 7f1aceffd640 1 --2- 172.21.15.43:0/3494478601 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1ad015f180 0x7f1ad014ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.215+0000 7f1aceffd640 1 -- 172.21.15.43:0/3494478601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ad00b23f0 msgr2=0x7f1ad015b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:14.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.215+0000 7f1aceffd640 1 --2- 172.21.15.43:0/3494478601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ad00b23f0 0x7f1ad015b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.215+0000 7f1aceffd640 1 -- 172.21.15.43:0/3494478601 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad00b0860 con 0x7f1ad00b2cc0 2024-04-25T09:04:14.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.215+0000 7f1acffff640 1 --2- 172.21.15.43:0/3494478601 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1ad015f180 0x7f1ad014ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:14.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.215+0000 7f1aceffd640 1 --2- 172.21.15.43:0/3494478601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ad00b2cc0 0x7f1ad00b0130 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f1ac400c7c0 tx=0x7f1ac400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:14.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.215+0000 7f1ab7fff640 1 -- 172.21.15.43:0/3494478601 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4015070 con 0x7f1ad00b2cc0 2024-04-25T09:04:14.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.215+0000 7f1ab7fff640 1 -- 172.21.15.43:0/3494478601 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac4007a30 con 0x7f1ad00b2cc0 2024-04-25T09:04:14.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.215+0000 7f1ad5df9640 1 -- 172.21.15.43:0/3494478601 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ad00b0b40 con 0x7f1ad00b2cc0 2024-04-25T09:04:14.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.215+0000 7f1ab7fff640 1 -- 172.21.15.43:0/3494478601 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac4007e00 con 0x7f1ad00b2cc0 2024-04-25T09:04:14.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.215+0000 7f1ad5df9640 1 -- 172.21.15.43:0/3494478601 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad00b1100 con 0x7f1ad00b2cc0 2024-04-25T09:04:14.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.216+0000 7f1ad5df9640 1 -- 172.21.15.43:0/3494478601 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ad0074ef0 con 0x7f1ad00b2cc0 2024-04-25T09:04:14.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.217+0000 7f1ab7fff640 1 -- 172.21.15.43:0/3494478601 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1ac4018070 con 0x7f1ad00b2cc0 2024-04-25T09:04:14.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.217+0000 7f1ab7fff640 1 --2- 172.21.15.43:0/3494478601 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1abc06b4d0 0x7f1abc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:14.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.217+0000 7f1ab7fff640 1 -- 172.21.15.43:0/3494478601 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ac408a0c0 con 0x7f1ad00b2cc0 2024-04-25T09:04:14.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.217+0000 7f1acf7fe640 1 --2- 172.21.15.43:0/3494478601 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1abc06b4d0 0x7f1abc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:14.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.218+0000 7f1acf7fe640 1 --2- 172.21.15.43:0/3494478601 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1abc06b4d0 0x7f1abc06d990 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0034000 tx=0x7f1ac002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:14.209 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.221+0000 7f1ab7fff640 1 -- 172.21.15.43:0/3494478601 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1ac40588f0 con 0x7f1ad00b2cc0 2024-04-25T09:04:14.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.407+0000 7f1ad5df9640 1 -- 172.21.15.43:0/3494478601 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ad00cf050 con 0x7f1ad00b2cc0 2024-04-25T09:04:14.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.408+0000 7f1ab7fff640 1 -- 172.21.15.43:0/3494478601 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+16656 (secure 0 0 0) 0x7f1ac405c990 con 0x7f1ad00b2cc0 2024-04-25T09:04:14.397 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:04:14.397 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":138,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:04:13.410419+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:04:11.735785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T09:04:12.383453+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:04:14.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.410+0000 7f1ad5df9640 1 -- 172.21.15.43:0/3494478601 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1abc06b4d0 msgr2=0x7f1abc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:14.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.410+0000 7f1ad5df9640 1 --2- 172.21.15.43:0/3494478601 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1abc06b4d0 0x7f1abc06d990 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0034000 tx=0x7f1ac002e000 comp rx=0 tx=0).stop 2024-04-25T09:04:14.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.410+0000 7f1ad5df9640 1 -- 172.21.15.43:0/3494478601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ad00b2cc0 msgr2=0x7f1ad00b0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:14.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.410+0000 7f1ad5df9640 1 --2- 172.21.15.43:0/3494478601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ad00b2cc0 0x7f1ad00b0130 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f1ac400c7c0 tx=0x7f1ac400cc80 comp rx=0 tx=0).stop 2024-04-25T09:04:14.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.410+0000 7f1ad5df9640 1 -- 172.21.15.43:0/3494478601 shutdown_connections 2024-04-25T09:04:14.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.410+0000 7f1ad5df9640 1 --2- 172.21.15.43:0/3494478601 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1ad015f180 0x7f1ad014ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.410+0000 7f1ad5df9640 1 --2- 172.21.15.43:0/3494478601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1ad00b2cc0 0x7f1ad00b0130 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.410+0000 7f1ad5df9640 1 --2- 172.21.15.43:0/3494478601 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1abc06b4d0 0x7f1abc06d990 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.410+0000 7f1ad5df9640 1 --2- 172.21.15.43:0/3494478601 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1ad00b23f0 0x7f1ad015b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.410+0000 7f1ad5df9640 1 -- 172.21.15.43:0/3494478601 >> 172.21.15.43:0/3494478601 conn(0x7f1ad00d17f0 msgr2=0x7f1ad0159850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:14.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.410+0000 7f1ad5df9640 1 -- 172.21.15.43:0/3494478601 shutdown_connections 2024-04-25T09:04:14.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.410+0000 7f1ad5df9640 1 -- 172.21.15.43:0/3494478601 wait complete. 2024-04-25T09:04:14.410 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:04:14.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.523+0000 7fa16e8bd640 1 Processor -- start 2024-04-25T09:04:14.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.523+0000 7fa16e8bd640 1 -- start start 2024-04-25T09:04:14.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.524+0000 7fa16e8bd640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa16815be60 0x7fa1680ce880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:14.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.524+0000 7fa16e8bd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa1680cedc0 0x7fa1680cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:14.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.524+0000 7fa16e8bd640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1680cc760 con 0x7fa1680d02b0 2024-04-25T09:04:14.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.524+0000 7fa16e8bd640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa16815c240 con 0x7fa1680cedc0 2024-04-25T09:04:14.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.524+0000 7fa16e8bd640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1680cca40 con 0x7fa16815be60 2024-04-25T09:04:14.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.524+0000 7fa1677fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa1680cedc0 0x7fa1680cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:14.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.524+0000 7fa16ce5a640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa1680d02b0 0x7fa16815f8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45700/0 (socket says 172.21.15.43:45700) 2024-04-25T09:04:14.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.524+0000 7fa16ce5a640 1 -- 172.21.15.43:0/3371737126 learned_addr learned my addr 172.21.15.43:0/3371737126 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:14.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.524+0000 7fa167fff640 1 --2- 172.21.15.43:0/3371737126 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa16815be60 0x7fa1680ce880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:14.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.524+0000 7fa1677fe640 1 -- 172.21.15.43:0/3371737126 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa16815be60 msgr2=0x7fa1680ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:14.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.524+0000 7fa1677fe640 1 --2- 172.21.15.43:0/3371737126 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa16815be60 0x7fa1680ce880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.524+0000 7fa1677fe640 1 -- 172.21.15.43:0/3371737126 >> v1:172.21.15.43:6789/0 conn(0x7fa1680d02b0 legacy=0x7fa16815f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:14.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.524+0000 7fa1677fe640 1 -- 172.21.15.43:0/3371737126 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1680ccd20 con 0x7fa1680cedc0 2024-04-25T09:04:14.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.524+0000 7fa166ffd640 1 -- 172.21.15.43:0/3371737126 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2068767548 0 0) 0x7fa1680cc760 con 0x7fa1680d02b0 2024-04-25T09:04:14.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.524+0000 7fa167fff640 1 --2- 172.21.15.43:0/3371737126 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa16815be60 0x7fa1680ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:14.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.525+0000 7fa1677fe640 1 --2- 172.21.15.43:0/3371737126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1680cedc0 0x7fa1680cfd70 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7fa15c009790 tx=0x7fa15c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7df4a4d31911a915 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:14.514 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.525+0000 7fa166ffd640 1 -- 172.21.15.43:0/3371737126 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c037070 con 0x7fa1680cedc0 2024-04-25T09:04:14.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.525+0000 7fa166ffd640 1 -- 172.21.15.43:0/3371737126 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa15c032440 con 0x7fa1680cedc0 2024-04-25T09:04:14.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.525+0000 7fa166ffd640 1 -- 172.21.15.43:0/3371737126 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c032810 con 0x7fa1680cedc0 2024-04-25T09:04:14.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.526+0000 7fa16e8bd640 1 -- 172.21.15.43:0/3371737126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1680cedc0 msgr2=0x7fa1680cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:14.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.526+0000 7fa16e8bd640 1 --2- 172.21.15.43:0/3371737126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1680cedc0 0x7fa1680cfd70 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7fa15c009790 tx=0x7fa15c02a180 comp rx=0 tx=0).stop 2024-04-25T09:04:14.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.526+0000 7fa16e8bd640 1 -- 172.21.15.43:0/3371737126 shutdown_connections 2024-04-25T09:04:14.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.526+0000 7fa16e8bd640 1 --2- 172.21.15.43:0/3371737126 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1680cedc0 0x7fa1680cfd70 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.526+0000 7fa16e8bd640 1 --2- 172.21.15.43:0/3371737126 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa16815be60 0x7fa1680ce880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.526+0000 7fa16e8bd640 1 -- 172.21.15.43:0/3371737126 >> 172.21.15.43:0/3371737126 conn(0x7fa1680b32e0 msgr2=0x7fa1680b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:14.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.526+0000 7fa16e8bd640 1 -- 172.21.15.43:0/3371737126 shutdown_connections 2024-04-25T09:04:14.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.526+0000 7fa16e8bd640 1 -- 172.21.15.43:0/3371737126 wait complete. 2024-04-25T09:04:14.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.527+0000 7fa16e8bd640 1 Processor -- start 2024-04-25T09:04:14.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.527+0000 7fa16e8bd640 1 -- start start 2024-04-25T09:04:14.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.528+0000 7fa16e8bd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1680cedc0 0x7fa1680c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:14.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.528+0000 7fa16e8bd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1680d02b0 0x7fa1680b57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:14.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.528+0000 7fa16e8bd640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa16815be60 0x7fa1681f37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:14.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.528+0000 7fa16e8bd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1680cc4b0 con 0x7fa1680cedc0 2024-04-25T09:04:14.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.528+0000 7fa16e8bd640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1680cc330 con 0x7fa1680d02b0 2024-04-25T09:04:14.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.528+0000 7fa16e8bd640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1680b1e60 con 0x7fa16815be60 2024-04-25T09:04:14.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.528+0000 7fa167fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1680cedc0 0x7fa1680c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:14.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.528+0000 7fa167fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1680cedc0 0x7fa1680c98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47524/0 (socket says 172.21.15.43:47524) 2024-04-25T09:04:14.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.528+0000 7fa167fff640 1 -- 172.21.15.43:0/4159621510 learned_addr learned my addr 172.21.15.43:0/4159621510 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:14.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.528+0000 7fa1677fe640 1 --2- 172.21.15.43:0/4159621510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1680d02b0 0x7fa1680b57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:14.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.528+0000 7fa167fff640 1 -- 172.21.15.43:0/4159621510 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa16815be60 msgr2=0x7fa1681f37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:14.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.528+0000 7fa16ce5a640 1 --2- 172.21.15.43:0/4159621510 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa16815be60 0x7fa1681f37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:14.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.528+0000 7fa167fff640 1 --2- 172.21.15.43:0/4159621510 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa16815be60 0x7fa1681f37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.528+0000 7fa167fff640 1 -- 172.21.15.43:0/4159621510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1680d02b0 msgr2=0x7fa1680b57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:14.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.528+0000 7fa167fff640 1 --2- 172.21.15.43:0/4159621510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1680d02b0 0x7fa1680b57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.528+0000 7fa167fff640 1 -- 172.21.15.43:0/4159621510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa16806c3c0 con 0x7fa1680cedc0 2024-04-25T09:04:14.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.528+0000 7fa1677fe640 1 --2- 172.21.15.43:0/4159621510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1680d02b0 0x7fa1680b57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:14.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.528+0000 7fa16ce5a640 1 --2- 172.21.15.43:0/4159621510 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa16815be60 0x7fa1681f37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:14.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.528+0000 7fa167fff640 1 --2- 172.21.15.43:0/4159621510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1680cedc0 0x7fa1680c98b0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fa15800c7f0 tx=0x7fa15800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:14.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.529+0000 7fa164ff9640 1 -- 172.21.15.43:0/4159621510 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa158015070 con 0x7fa1680cedc0 2024-04-25T09:04:14.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.529+0000 7fa164ff9640 1 -- 172.21.15.43:0/4159621510 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa158007820 con 0x7fa1680cedc0 2024-04-25T09:04:14.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.529+0000 7fa164ff9640 1 -- 172.21.15.43:0/4159621510 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa158007bd0 con 0x7fa1680cedc0 2024-04-25T09:04:14.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.529+0000 7fa16e8bd640 1 -- 172.21.15.43:0/4159621510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1681f4340 con 0x7fa1680cedc0 2024-04-25T09:04:14.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.529+0000 7fa16e8bd640 1 -- 172.21.15.43:0/4159621510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa16806c680 con 0x7fa1680cedc0 2024-04-25T09:04:14.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.530+0000 7fa16e8bd640 1 -- 172.21.15.43:0/4159621510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa168074ef0 con 0x7fa1680cedc0 2024-04-25T09:04:14.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.530+0000 7fa164ff9640 1 -- 172.21.15.43:0/4159621510 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa158018070 con 0x7fa1680cedc0 2024-04-25T09:04:14.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.530+0000 7fa164ff9640 1 --2- 172.21.15.43:0/4159621510 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa15406b4d0 0x7fa15406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:14.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.530+0000 7fa164ff9640 1 -- 172.21.15.43:0/4159621510 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa15808a330 con 0x7fa1680cedc0 2024-04-25T09:04:14.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.531+0000 7fa1677fe640 1 --2- 172.21.15.43:0/4159621510 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa15406b4d0 0x7fa15406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:14.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.531+0000 7fa1677fe640 1 --2- 172.21.15.43:0/4159621510 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa15406b4d0 0x7fa15406d990 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fa15c034000 tx=0x7fa15c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:14.523 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.535+0000 7fa164ff9640 1 -- 172.21.15.43:0/4159621510 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa1580591b0 con 0x7fa1680cedc0 2024-04-25T09:04:14.706 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.717+0000 7fa16e8bd640 1 -- 172.21.15.43:0/4159621510 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1680cfb50 con 0x7fa1680cedc0 2024-04-25T09:04:14.707 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.719+0000 7fa164ff9640 1 -- 172.21.15.43:0/4159621510 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+16656 (secure 0 0 0) 0x7fa158058ab0 con 0x7fa1680cedc0 2024-04-25T09:04:14.707 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:04:14.707 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":138,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:04:13.410419+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:04:11.735785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T09:04:12.383453+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:04:14.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.721+0000 7fa16e8bd640 1 -- 172.21.15.43:0/4159621510 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa15406b4d0 msgr2=0x7fa15406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:14.709 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.721+0000 7fa16e8bd640 1 --2- 172.21.15.43:0/4159621510 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa15406b4d0 0x7fa15406d990 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fa15c034000 tx=0x7fa15c02e000 comp rx=0 tx=0).stop 2024-04-25T09:04:14.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.721+0000 7fa16e8bd640 1 -- 172.21.15.43:0/4159621510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1680cedc0 msgr2=0x7fa1680c98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:14.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.721+0000 7fa16e8bd640 1 --2- 172.21.15.43:0/4159621510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1680cedc0 0x7fa1680c98b0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fa15800c7f0 tx=0x7fa15800ccb0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.721+0000 7fa16e8bd640 1 -- 172.21.15.43:0/4159621510 shutdown_connections 2024-04-25T09:04:14.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.721+0000 7fa16e8bd640 1 --2- 172.21.15.43:0/4159621510 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa16815be60 0x7fa1681f37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.721+0000 7fa16e8bd640 1 --2- 172.21.15.43:0/4159621510 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa15406b4d0 0x7fa15406d990 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.721+0000 7fa16e8bd640 1 --2- 172.21.15.43:0/4159621510 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa1680d02b0 0x7fa1680b57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.721+0000 7fa16e8bd640 1 --2- 172.21.15.43:0/4159621510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa1680cedc0 0x7fa1680c98b0 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.721+0000 7fa16e8bd640 1 -- 172.21.15.43:0/4159621510 >> 172.21.15.43:0/4159621510 conn(0x7fa1680b32e0 msgr2=0x7fa16815cb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:14.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.721+0000 7fa16e8bd640 1 -- 172.21.15.43:0/4159621510 shutdown_connections 2024-04-25T09:04:14.710 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.721+0000 7fa16e8bd640 1 -- 172.21.15.43:0/4159621510 wait complete. 2024-04-25T09:04:14.721 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T09:04:14.824 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.836+0000 7f31f685d640 1 Processor -- start 2024-04-25T09:04:14.825 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.836+0000 7f31f685d640 1 -- start start 2024-04-25T09:04:14.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.837+0000 7f31f685d640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f31f00d04a0 0x7f31f015ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:14.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.837+0000 7f31f685d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f31f00ce8c0 0x7f31f00d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:14.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.837+0000 7f31f685d640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31f00cf110 con 0x7f31f015dec0 2024-04-25T09:04:14.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.837+0000 7f31f685d640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31f00cf3f0 con 0x7f31f00d04a0 2024-04-25T09:04:14.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.837+0000 7f31f685d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f00cf6d0 con 0x7f31f00ce8c0 2024-04-25T09:04:14.828 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.837+0000 7f31ef7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f31f015dec0 0x7f31f00d0880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45704/0 (socket says 172.21.15.43:45704) 2024-04-25T09:04:14.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.837+0000 7f31ef7fe640 1 -- 172.21.15.43:0/1717374805 learned_addr learned my addr 172.21.15.43:0/1717374805 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:14.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.837+0000 7f31effff640 1 --2- 172.21.15.43:0/1717374805 >> v2:172.21.15.193:3300/0 conn(0x7f31f00d04a0 0x7f31f015ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:14.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.837+0000 7f31f4dfa640 1 --2- 172.21.15.43:0/1717374805 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f31f00ce8c0 0x7f31f00d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:14.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.837+0000 7f31eeffd640 1 -- 172.21.15.43:0/1717374805 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1274538570 0 0) 0x7f31f00cf110 con 0x7f31f015dec0 2024-04-25T09:04:14.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.837+0000 7f31eeffd640 1 -- 172.21.15.43:0/1717374805 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31dc003520 con 0x7f31f015dec0 2024-04-25T09:04:14.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.837+0000 7f31eeffd640 1 -- 172.21.15.43:0/1717374805 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1922947305 0 0) 0x7f31dc003520 con 0x7f31f015dec0 2024-04-25T09:04:14.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.837+0000 7f31eeffd640 1 -- 172.21.15.43:0/1717374805 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f31f00ce8c0 msgr2=0x7f31f00d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:14.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.837+0000 7f31eeffd640 1 --2- 172.21.15.43:0/1717374805 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f31f00ce8c0 0x7f31f00d17a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.837+0000 7f31eeffd640 1 -- 172.21.15.43:0/1717374805 >> v2:172.21.15.193:3300/0 conn(0x7f31f00d04a0 msgr2=0x7f31f015ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:14.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.837+0000 7f31eeffd640 1 --2- 172.21.15.43:0/1717374805 >> v2:172.21.15.193:3300/0 conn(0x7f31f00d04a0 0x7f31f015ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.837+0000 7f31eeffd640 1 -- 172.21.15.43:0/1717374805 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f00cf9b0 con 0x7f31f015dec0 2024-04-25T09:04:14.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.837+0000 7f31eeffd640 1 -- 172.21.15.43:0/1717374805 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f31d8001690 con 0x7f31f015dec0 2024-04-25T09:04:14.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.838+0000 7f31eeffd640 1 -- 172.21.15.43:0/1717374805 >> v1:172.21.15.43:6789/0 conn(0x7f31f015dec0 legacy=0x7f31f00d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:14.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.838+0000 7f31eeffd640 1 --2- 172.21.15.43:0/1717374805 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f31f00ce8c0 0x7f31dc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:14.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.838+0000 7f31eeffd640 1 --2- 172.21.15.43:0/1717374805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31dc003f30 0x7f31dc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:14.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.838+0000 7f31eeffd640 1 --2- 172.21.15.43:0/1717374805 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31dc004d40 0x7f31dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:14.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.838+0000 7f31eeffd640 1 -- 172.21.15.43:0/1717374805 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f00cf3f0 con 0x7f31dc003f30 2024-04-25T09:04:14.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.838+0000 7f31eeffd640 1 -- 172.21.15.43:0/1717374805 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f00cf6d0 con 0x7f31dc004d40 2024-04-25T09:04:14.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.838+0000 7f31eeffd640 1 -- 172.21.15.43:0/1717374805 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31dc00b9c0 con 0x7f31f00ce8c0 2024-04-25T09:04:14.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.838+0000 7f31effff640 1 --2- 172.21.15.43:0/1717374805 >> v2:172.21.15.193:3300/0 conn(0x7f31f00d04a0 0x7f31f015ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:04:14.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.838+0000 7f31ef7fe640 1 --2- 172.21.15.43:0/1717374805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31dc003f30 0x7f31dc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:14.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.838+0000 7f31f4dfa640 1 --2- 172.21.15.43:0/1717374805 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31dc004d40 0x7f31dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:14.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.838+0000 7f31effff640 1 --2- 172.21.15.43:0/1717374805 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f31f00ce8c0 0x7f31dc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:14.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.838+0000 7f31ef7fe640 1 -- 172.21.15.43:0/1717374805 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f31f00ce8c0 msgr2=0x7f31dc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:14.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.838+0000 7f31ef7fe640 1 --2- 172.21.15.43:0/1717374805 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f31f00ce8c0 0x7f31dc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.838+0000 7f31ef7fe640 1 -- 172.21.15.43:0/1717374805 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31dc004d40 msgr2=0x7f31dc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:14.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.838+0000 7f31ef7fe640 1 --2- 172.21.15.43:0/1717374805 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31dc004d40 0x7f31dc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.838+0000 7f31ef7fe640 1 -- 172.21.15.43:0/1717374805 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31dc003520 con 0x7f31dc003f30 2024-04-25T09:04:14.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.838+0000 7f31effff640 1 --2- 172.21.15.43:0/1717374805 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f31f00ce8c0 0x7f31dc0039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:14.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.839+0000 7f31ef7fe640 1 --2- 172.21.15.43:0/1717374805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31dc003f30 0x7f31dc004800 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f31f00d15d0 tx=0x7f31dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13ee372ea9191509 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:14.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.839+0000 7f31eeffd640 1 -- 172.21.15.43:0/1717374805 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d8007d60 con 0x7f31dc003f30 2024-04-25T09:04:14.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.839+0000 7f31eeffd640 1 -- 172.21.15.43:0/1717374805 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31d8005af0 con 0x7f31dc003f30 2024-04-25T09:04:14.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.839+0000 7f31eeffd640 1 -- 172.21.15.43:0/1717374805 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31d8007070 con 0x7f31dc003f30 2024-04-25T09:04:14.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.839+0000 7f31f685d640 1 -- 172.21.15.43:0/1717374805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31dc003f30 msgr2=0x7f31dc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:14.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.839+0000 7f31f685d640 1 --2- 172.21.15.43:0/1717374805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31dc003f30 0x7f31dc004800 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f31f00d15d0 tx=0x7f31dc005830 comp rx=0 tx=0).stop 2024-04-25T09:04:14.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.839+0000 7f31f4dfa640 1 -- 172.21.15.43:0/1717374805 reap_dead start 2024-04-25T09:04:14.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.840+0000 7f31f685d640 1 -- 172.21.15.43:0/1717374805 shutdown_connections 2024-04-25T09:04:14.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.840+0000 7f31f685d640 1 -- 172.21.15.43:0/1717374805 >> 172.21.15.43:0/1717374805 conn(0x7f31f009cd40 msgr2=0x7f31f00d3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:14.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.840+0000 7f31f685d640 1 -- 172.21.15.43:0/1717374805 shutdown_connections 2024-04-25T09:04:14.832 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.840+0000 7f31f685d640 1 -- 172.21.15.43:0/1717374805 wait complete. 2024-04-25T09:04:14.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.841+0000 7f31f685d640 1 Processor -- start 2024-04-25T09:04:14.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.841+0000 7f31f685d640 1 -- start start 2024-04-25T09:04:14.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.842+0000 7f31f685d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31dc003f30 0x7f31f01f7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:14.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.842+0000 7f31f685d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f31f01f7ab0 0x7f31f00cbd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:14.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.842+0000 7f31f685d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31f00cc250 0x7f31f00b8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:14.834 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.842+0000 7f31f685d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f00cee50 con 0x7f31f00cc250 2024-04-25T09:04:14.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.842+0000 7f31f685d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31f00cecd0 con 0x7f31dc003f30 2024-04-25T09:04:14.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.842+0000 7f31f685d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31f00d17a0 con 0x7f31f01f7ab0 2024-04-25T09:04:14.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.842+0000 7f31f4dfa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31f00cc250 0x7f31f00b8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:14.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.842+0000 7f31f4dfa640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31f00cc250 0x7f31f00b8420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47544/0 (socket says 172.21.15.43:47544) 2024-04-25T09:04:14.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.842+0000 7f31f4dfa640 1 -- 172.21.15.43:0/131446325 learned_addr learned my addr 172.21.15.43:0/131446325 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:14.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.842+0000 7f31effff640 1 --2- 172.21.15.43:0/131446325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31dc003f30 0x7f31f01f7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:14.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.842+0000 7f31ef7fe640 1 --2- 172.21.15.43:0/131446325 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f31f01f7ab0 0x7f31f00cbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:14.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.842+0000 7f31f4dfa640 1 -- 172.21.15.43:0/131446325 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f31f01f7ab0 msgr2=0x7f31f00cbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:14.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.842+0000 7f31f4dfa640 1 --2- 172.21.15.43:0/131446325 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f31f01f7ab0 0x7f31f00cbd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.842+0000 7f31f4dfa640 1 -- 172.21.15.43:0/131446325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31dc003f30 msgr2=0x7f31f01f7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:14.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.842+0000 7f31f4dfa640 1 --2- 172.21.15.43:0/131446325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31dc003f30 0x7f31f01f7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:14.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.842+0000 7f31f4dfa640 1 -- 172.21.15.43:0/131446325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f00cc900 con 0x7f31f00cc250 2024-04-25T09:04:14.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.842+0000 7f31ef7fe640 1 --2- 172.21.15.43:0/131446325 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f31f01f7ab0 0x7f31f00cbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:14.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.842+0000 7f31f4dfa640 1 --2- 172.21.15.43:0/131446325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31f00cc250 0x7f31f00b8420 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f31f00b6660 tx=0x7f31e800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:14.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.842+0000 7f31ecff9640 1 -- 172.21.15.43:0/131446325 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e800fe10 con 0x7f31f00cc250 2024-04-25T09:04:14.835 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.842+0000 7f31ecff9640 1 -- 172.21.15.43:0/131446325 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31e801d070 con 0x7f31f00cc250 2024-04-25T09:04:14.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.842+0000 7f31ecff9640 1 -- 172.21.15.43:0/131446325 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31e8015670 con 0x7f31f00cc250 2024-04-25T09:04:14.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.842+0000 7f31f685d640 1 -- 172.21.15.43:0/131446325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31f00ccbe0 con 0x7f31f00cc250 2024-04-25T09:04:14.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.842+0000 7f31f685d640 1 -- 172.21.15.43:0/131446325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31f00c8080 con 0x7f31f00cc250 2024-04-25T09:04:14.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.843+0000 7f31f685d640 1 -- 172.21.15.43:0/131446325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31f0074f60 con 0x7f31f00cc250 2024-04-25T09:04:14.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.844+0000 7f31ecff9640 1 -- 172.21.15.43:0/131446325 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f31e8015810 con 0x7f31f00cc250 2024-04-25T09:04:14.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.844+0000 7f31ecff9640 1 --2- 172.21.15.43:0/131446325 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f31d006b500 0x7f31d006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:14.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.844+0000 7f31ecff9640 1 -- 172.21.15.43:0/131446325 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f31e808a9f0 con 0x7f31f00cc250 2024-04-25T09:04:14.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.844+0000 7f31effff640 1 --2- 172.21.15.43:0/131446325 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f31d006b500 0x7f31d006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:14.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.845+0000 7f31effff640 1 --2- 172.21.15.43:0/131446325 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f31d006b500 0x7f31d006d9c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f31e0009ba0 tx=0x7f31e0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:14.836 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:14.848+0000 7f31ecff9640 1 -- 172.21.15.43:0/131446325 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f31e8010040 con 0x7f31f00cc250 2024-04-25T09:04:15.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.067+0000 7f31f685d640 1 -- 172.21.15.43:0/131446325 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f31f01f7e90 con 0x7f31f00cc250 2024-04-25T09:04:15.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.071+0000 7f31ecff9640 1 -- 172.21.15.43:0/131446325 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v86) v1 ==== 118+0+0 (secure 0 0 0) 0x7f31e8059870 con 0x7f31f00cc250 2024-04-25T09:04:15.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.073+0000 7f31f685d640 1 -- 172.21.15.43:0/131446325 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f31d006b500 msgr2=0x7f31d006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:15.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.073+0000 7f31f685d640 1 --2- 172.21.15.43:0/131446325 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f31d006b500 0x7f31d006d9c0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f31e0009ba0 tx=0x7f31e0008040 comp rx=0 tx=0).stop 2024-04-25T09:04:15.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.073+0000 7f31f685d640 1 -- 172.21.15.43:0/131446325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31f00cc250 msgr2=0x7f31f00b8420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:15.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.073+0000 7f31f685d640 1 --2- 172.21.15.43:0/131446325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31f00cc250 0x7f31f00b8420 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f31f00b6660 tx=0x7f31e800f580 comp rx=0 tx=0).stop 2024-04-25T09:04:15.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.073+0000 7f31f685d640 1 -- 172.21.15.43:0/131446325 shutdown_connections 2024-04-25T09:04:15.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.073+0000 7f31f685d640 1 --2- 172.21.15.43:0/131446325 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f31f00cc250 0x7f31f00b8420 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:15.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.073+0000 7f31f685d640 1 --2- 172.21.15.43:0/131446325 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f31f01f7ab0 0x7f31f00cbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:15.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.073+0000 7f31f685d640 1 --2- 172.21.15.43:0/131446325 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f31d006b500 0x7f31d006d9c0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:15.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.073+0000 7f31f685d640 1 --2- 172.21.15.43:0/131446325 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f31dc003f30 0x7f31f01f7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:15.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.073+0000 7f31f685d640 1 -- 172.21.15.43:0/131446325 >> 172.21.15.43:0/131446325 conn(0x7f31f009cd40 msgr2=0x7f31f0160cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:15.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.073+0000 7f31f685d640 1 -- 172.21.15.43:0/131446325 shutdown_connections 2024-04-25T09:04:15.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.073+0000 7f31f685d640 1 -- 172.21.15.43:0/131446325 wait complete. 2024-04-25T09:04:15.074 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T09:04:15.074 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T09:04:15.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.190+0000 7fc94d57e640 1 Processor -- start 2024-04-25T09:04:15.178 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.190+0000 7fc94d57e640 1 -- start start 2024-04-25T09:04:15.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.191+0000 7fc94d57e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc9480d04a0 0x7fc94815ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:15.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.191+0000 7fc94d57e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc9480ce8c0 0x7fc9480d17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:15.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.191+0000 7fc94d57e640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9480cf110 con 0x7fc94815dec0 2024-04-25T09:04:15.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.191+0000 7fc94d57e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9480cf3f0 con 0x7fc9480ce8c0 2024-04-25T09:04:15.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.191+0000 7fc94d57e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9480cf6d0 con 0x7fc9480d04a0 2024-04-25T09:04:15.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.191+0000 7fc9477fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc9480ce8c0 0x7fc9480d17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:15.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.191+0000 7fc9477fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc9480ce8c0 0x7fc9480d17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47552/0 (socket says 172.21.15.43:47552) 2024-04-25T09:04:15.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.191+0000 7fc9477fe640 1 -- 172.21.15.43:0/924183407 learned_addr learned my addr 172.21.15.43:0/924183407 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:15.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.191+0000 7fc946ffd640 1 --2- 172.21.15.43:0/924183407 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc9480d04a0 0x7fc94815ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:15.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.191+0000 7fc9477fe640 1 -- 172.21.15.43:0/924183407 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc9480d04a0 msgr2=0x7fc94815ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:15.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.191+0000 7fc9477fe640 1 --2- 172.21.15.43:0/924183407 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc9480d04a0 0x7fc94815ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:15.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.191+0000 7fc9477fe640 1 -- 172.21.15.43:0/924183407 >> v1:172.21.15.193:6789/0 conn(0x7fc94815dec0 legacy=0x7fc9480d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:15.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.191+0000 7fc9477fe640 1 -- 172.21.15.43:0/924183407 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9480cf9b0 con 0x7fc9480ce8c0 2024-04-25T09:04:15.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.192+0000 7fc9477fe640 1 --2- 172.21.15.43:0/924183407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9480ce8c0 0x7fc9480d17a0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7fc940009e50 tx=0x7fc94002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96fe8e70687b2472 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:15.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.192+0000 7fc9467fc640 1 -- 172.21.15.43:0/924183407 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc940036070 con 0x7fc9480ce8c0 2024-04-25T09:04:15.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.192+0000 7fc9467fc640 1 -- 172.21.15.43:0/924183407 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc940031600 con 0x7fc9480ce8c0 2024-04-25T09:04:15.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.192+0000 7fc9467fc640 1 -- 172.21.15.43:0/924183407 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9400319d0 con 0x7fc9480ce8c0 2024-04-25T09:04:15.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.193+0000 7fc94d57e640 1 -- 172.21.15.43:0/924183407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9480ce8c0 msgr2=0x7fc9480d17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:15.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.193+0000 7fc94d57e640 1 --2- 172.21.15.43:0/924183407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9480ce8c0 0x7fc9480d17a0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7fc940009e50 tx=0x7fc94002a6c0 comp rx=0 tx=0).stop 2024-04-25T09:04:15.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.193+0000 7fc94d57e640 1 -- 172.21.15.43:0/924183407 shutdown_connections 2024-04-25T09:04:15.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.193+0000 7fc94d57e640 1 --2- 172.21.15.43:0/924183407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc9480ce8c0 0x7fc9480d17a0 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:15.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.193+0000 7fc94d57e640 1 --2- 172.21.15.43:0/924183407 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc9480d04a0 0x7fc94815ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:15.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.193+0000 7fc94d57e640 1 -- 172.21.15.43:0/924183407 >> 172.21.15.43:0/924183407 conn(0x7fc94809cd40 msgr2=0x7fc9480d3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:15.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.193+0000 7fc94d57e640 1 -- 172.21.15.43:0/924183407 shutdown_connections 2024-04-25T09:04:15.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.193+0000 7fc94d57e640 1 -- 172.21.15.43:0/924183407 wait complete. 2024-04-25T09:04:15.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.194+0000 7fc94d57e640 1 Processor -- start 2024-04-25T09:04:15.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.194+0000 7fc94d57e640 1 -- start start 2024-04-25T09:04:15.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.194+0000 7fc94d57e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc9480ce8c0 0x7fc948158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:15.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.194+0000 7fc94d57e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9480d04a0 0x7fc9481e9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:15.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.195+0000 7fc94d57e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc94815dec0 0x7fc9481e9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:15.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.195+0000 7fc94d57e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9480cee90 con 0x7fc94815dec0 2024-04-25T09:04:15.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.195+0000 7fc94d57e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9480ced10 con 0x7fc9480d04a0 2024-04-25T09:04:15.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.195+0000 7fc94d57e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9480d1ce0 con 0x7fc9480ce8c0 2024-04-25T09:04:15.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.195+0000 7fc9477fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc94815dec0 0x7fc9481e9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:15.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.195+0000 7fc9477fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc94815dec0 0x7fc9481e9e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47556/0 (socket says 172.21.15.43:47556) 2024-04-25T09:04:15.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.195+0000 7fc9477fe640 1 -- 172.21.15.43:0/3436021084 learned_addr learned my addr 172.21.15.43:0/3436021084 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:15.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.195+0000 7fc93ffff640 1 --2- 172.21.15.43:0/3436021084 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9480d04a0 0x7fc9481e9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:15.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.195+0000 7fc9477fe640 1 -- 172.21.15.43:0/3436021084 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc9480ce8c0 msgr2=0x7fc948158b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:15.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.195+0000 7fc9477fe640 1 --2- 172.21.15.43:0/3436021084 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc9480ce8c0 0x7fc948158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:15.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.195+0000 7fc9477fe640 1 -- 172.21.15.43:0/3436021084 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9480d04a0 msgr2=0x7fc9481e9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:15.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.195+0000 7fc9477fe640 1 --2- 172.21.15.43:0/3436021084 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9480d04a0 0x7fc9481e9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:15.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.195+0000 7fc9477fe640 1 -- 172.21.15.43:0/3436021084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9480c8c80 con 0x7fc94815dec0 2024-04-25T09:04:15.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.195+0000 7fc9477fe640 1 --2- 172.21.15.43:0/3436021084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc94815dec0 0x7fc9481e9e40 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fc940034040 tx=0x7fc940004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:15.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.195+0000 7fc93f7fe640 1 -- 172.21.15.43:0/3436021084 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc940036050 con 0x7fc94815dec0 2024-04-25T09:04:15.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.195+0000 7fc93f7fe640 1 -- 172.21.15.43:0/3436021084 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc940042070 con 0x7fc94815dec0 2024-04-25T09:04:15.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.195+0000 7fc94d57e640 1 -- 172.21.15.43:0/3436021084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9480c8f60 con 0x7fc94815dec0 2024-04-25T09:04:15.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.195+0000 7fc93f7fe640 1 -- 172.21.15.43:0/3436021084 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc940009510 con 0x7fc94815dec0 2024-04-25T09:04:15.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.195+0000 7fc94d57e640 1 -- 172.21.15.43:0/3436021084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9480c9520 con 0x7fc94815dec0 2024-04-25T09:04:15.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.196+0000 7fc94d57e640 1 -- 172.21.15.43:0/3436021084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc948074f60 con 0x7fc94815dec0 2024-04-25T09:04:15.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.197+0000 7fc93f7fe640 1 -- 172.21.15.43:0/3436021084 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc940007bf0 con 0x7fc94815dec0 2024-04-25T09:04:15.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.197+0000 7fc93f7fe640 1 --2- 172.21.15.43:0/3436021084 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc92806b4d0 0x7fc92806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:15.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.197+0000 7fc93f7fe640 1 -- 172.21.15.43:0/3436021084 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc9400ab1a0 con 0x7fc94815dec0 2024-04-25T09:04:15.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.198+0000 7fc946ffd640 1 --2- 172.21.15.43:0/3436021084 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc92806b4d0 0x7fc92806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:15.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.198+0000 7fc946ffd640 1 --2- 172.21.15.43:0/3436021084 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc92806b4d0 0x7fc92806d990 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fc938006fd0 tx=0x7fc938008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:15.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.201+0000 7fc93f7fe640 1 -- 172.21.15.43:0/3436021084 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc94007a020 con 0x7fc94815dec0 2024-04-25T09:04:15.411 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:15.423+0000 7fc94d57e640 1 -- 172.21.15.43:0/3436021084 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fc94808daf0 con 0x7fc94815dec0 2024-04-25T09:04:16.017 INFO:tasks.ceph.mds.b.smithi193.stderr:2024-04-25T09:04:16.029+0000 7f8d08e29640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T09:04:16.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.075+0000 7fc93f7fe640 1 -- 172.21.15.43:0/3436021084 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v87) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc940079920 con 0x7fc94815dec0 2024-04-25T09:04:16.066 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.078+0000 7fc94d57e640 1 -- 172.21.15.43:0/3436021084 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc92806b4d0 msgr2=0x7fc92806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:16.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.078+0000 7fc94d57e640 1 --2- 172.21.15.43:0/3436021084 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc92806b4d0 0x7fc92806d990 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fc938006fd0 tx=0x7fc938008040 comp rx=0 tx=0).stop 2024-04-25T09:04:16.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.078+0000 7fc94d57e640 1 -- 172.21.15.43:0/3436021084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc94815dec0 msgr2=0x7fc9481e9e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:16.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.078+0000 7fc94d57e640 1 --2- 172.21.15.43:0/3436021084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc94815dec0 0x7fc9481e9e40 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7fc940034040 tx=0x7fc940004a40 comp rx=0 tx=0).stop 2024-04-25T09:04:16.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.078+0000 7fc94d57e640 1 -- 172.21.15.43:0/3436021084 shutdown_connections 2024-04-25T09:04:16.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.078+0000 7fc94d57e640 1 --2- 172.21.15.43:0/3436021084 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc94815dec0 0x7fc9481e9e40 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.078+0000 7fc94d57e640 1 --2- 172.21.15.43:0/3436021084 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc92806b4d0 0x7fc92806d990 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.078+0000 7fc94d57e640 1 --2- 172.21.15.43:0/3436021084 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc9480d04a0 0x7fc9481e9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.078+0000 7fc94d57e640 1 --2- 172.21.15.43:0/3436021084 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc9480ce8c0 0x7fc948158b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.078+0000 7fc94d57e640 1 -- 172.21.15.43:0/3436021084 >> 172.21.15.43:0/3436021084 conn(0x7fc94809cd40 msgr2=0x7fc94806b130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:16.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.078+0000 7fc94d57e640 1 -- 172.21.15.43:0/3436021084 shutdown_connections 2024-04-25T09:04:16.067 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.078+0000 7fc94d57e640 1 -- 172.21.15.43:0/3436021084 wait complete. 2024-04-25T09:04:16.080 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T09:04:16.081 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:04:16.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.200+0000 7f5b31209640 1 Processor -- start 2024-04-25T09:04:16.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.200+0000 7f5b31209640 1 -- start start 2024-04-25T09:04:16.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.201+0000 7f5b31209640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f5b2c15def0 0x7f5b2c06be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:16.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.201+0000 7f5b31209640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b2c15e2d0 con 0x7f5b2c1619f0 2024-04-25T09:04:16.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.201+0000 7f5b31209640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b2c0b1020 con 0x7f5b2c162560 2024-04-25T09:04:16.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.201+0000 7f5b31209640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b2c0d0e90 con 0x7f5b2c15def0 2024-04-25T09:04:16.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.201+0000 7f5b2a59c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f5b2c1619f0 0x7f5b2c161e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45716/0 (socket says 172.21.15.43:45716) 2024-04-25T09:04:16.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.201+0000 7f5b2a59c640 1 -- 172.21.15.43:0/1858130707 learned_addr learned my addr 172.21.15.43:0/1858130707 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:16.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.201+0000 7f5b29d9b640 1 -- 172.21.15.43:0/1858130707 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 565709432 0 0) 0x7f5b2c15e2d0 con 0x7f5b2c1619f0 2024-04-25T09:04:16.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.201+0000 7f5b29d9b640 1 -- 172.21.15.43:0/1858130707 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b14003520 con 0x7f5b2c1619f0 2024-04-25T09:04:16.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.201+0000 7f5b2ad9d640 1 --2- 172.21.15.43:0/1858130707 >> v2:172.21.15.193:3300/0 conn(0x7f5b2c15def0 0x7f5b2c06be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:16.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.202+0000 7f5b29d9b640 1 -- 172.21.15.43:0/1858130707 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3376452373 0 0) 0x7f5b14003520 con 0x7f5b2c1619f0 2024-04-25T09:04:16.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.202+0000 7f5b29d9b640 1 -- 172.21.15.43:0/1858130707 >> v2:172.21.15.193:3300/0 conn(0x7f5b2c15def0 msgr2=0x7f5b2c06be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:16.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.202+0000 7f5b29d9b640 1 --2- 172.21.15.43:0/1858130707 >> v2:172.21.15.193:3300/0 conn(0x7f5b2c15def0 0x7f5b2c06be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.202+0000 7f5b29d9b640 1 -- 172.21.15.43:0/1858130707 >> v1:172.21.15.193:6789/0 conn(0x7f5b2c162560 legacy=0x7f5b2c0b20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:16.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.202+0000 7f5b29d9b640 1 -- 172.21.15.43:0/1858130707 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b2c0b1300 con 0x7f5b2c1619f0 2024-04-25T09:04:16.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.202+0000 7f5b29d9b640 1 -- 172.21.15.43:0/1858130707 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f5b200017f0 con 0x7f5b2c1619f0 2024-04-25T09:04:16.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.202+0000 7f5b29d9b640 1 -- 172.21.15.43:0/1858130707 >> v1:172.21.15.43:6789/0 conn(0x7f5b2c1619f0 legacy=0x7f5b2c161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:16.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.202+0000 7f5b29d9b640 1 --2- 172.21.15.43:0/1858130707 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5b140039b0 0x7f5b14003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:16.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.202+0000 7f5b29d9b640 1 --2- 172.21.15.43:0/1858130707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b14004800 0x7f5b14004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:16.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.202+0000 7f5b29d9b640 1 --2- 172.21.15.43:0/1858130707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b140051c0 0x7f5b1400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:16.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.202+0000 7f5b29d9b640 1 -- 172.21.15.43:0/1858130707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2c0d0e90 con 0x7f5b140051c0 2024-04-25T09:04:16.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.202+0000 7f5b29d9b640 1 -- 172.21.15.43:0/1858130707 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1400c060 con 0x7f5b14004800 2024-04-25T09:04:16.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.202+0000 7f5b29d9b640 1 -- 172.21.15.43:0/1858130707 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1400c260 con 0x7f5b140039b0 2024-04-25T09:04:16.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.202+0000 7f5b2b59e640 1 --2- 172.21.15.43:0/1858130707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b140051c0 0x7f5b1400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:16.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.202+0000 7f5b2a59c640 1 --2- 172.21.15.43:0/1858130707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b14004800 0x7f5b14004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:16.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.202+0000 7f5b2ad9d640 1 --2- 172.21.15.43:0/1858130707 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5b140039b0 0x7f5b14003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:16.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.202+0000 7f5b2b59e640 1 -- 172.21.15.43:0/1858130707 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5b140039b0 msgr2=0x7f5b14003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:16.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.202+0000 7f5b2b59e640 1 --2- 172.21.15.43:0/1858130707 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5b140039b0 0x7f5b14003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.202+0000 7f5b2b59e640 1 -- 172.21.15.43:0/1858130707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b14004800 msgr2=0x7f5b14004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:16.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.202+0000 7f5b2b59e640 1 --2- 172.21.15.43:0/1858130707 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b14004800 0x7f5b14004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.202+0000 7f5b2b59e640 1 -- 172.21.15.43:0/1858130707 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b14003520 con 0x7f5b140051c0 2024-04-25T09:04:16.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.203+0000 7f5b2b59e640 1 -- 172.21.15.43:0/1858130707 reap_dead start 2024-04-25T09:04:16.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.203+0000 7f5b2b59e640 1 --2- 172.21.15.43:0/1858130707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b140051c0 0x7f5b1400b9c0 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f5b24001720 tx=0x7f5b14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b67d6be2ee7fcbe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:16.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.203+0000 7f5b29d9b640 1 -- 172.21.15.43:0/1858130707 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b24001370 con 0x7f5b140051c0 2024-04-25T09:04:16.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.203+0000 7f5b29d9b640 1 -- 172.21.15.43:0/1858130707 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b240052d0 con 0x7f5b140051c0 2024-04-25T09:04:16.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.203+0000 7f5b29d9b640 1 -- 172.21.15.43:0/1858130707 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2402aba0 con 0x7f5b140051c0 2024-04-25T09:04:16.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.204+0000 7f5b31209640 1 -- 172.21.15.43:0/1858130707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b140051c0 msgr2=0x7f5b1400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:16.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.204+0000 7f5b31209640 1 --2- 172.21.15.43:0/1858130707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b140051c0 0x7f5b1400b9c0 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f5b24001720 tx=0x7f5b14005830 comp rx=0 tx=0).stop 2024-04-25T09:04:16.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.204+0000 7f5b31209640 1 -- 172.21.15.43:0/1858130707 shutdown_connections 2024-04-25T09:04:16.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.204+0000 7f5b31209640 1 --2- 172.21.15.43:0/1858130707 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b140051c0 0x7f5b1400b9c0 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.204+0000 7f5b31209640 1 -- 172.21.15.43:0/1858130707 >> 172.21.15.43:0/1858130707 conn(0x7f5b2c0b32e0 msgr2=0x7f5b2c0b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:16.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.204+0000 7f5b31209640 1 -- 172.21.15.43:0/1858130707 shutdown_connections 2024-04-25T09:04:16.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.204+0000 7f5b31209640 1 -- 172.21.15.43:0/1858130707 wait complete. 2024-04-25T09:04:16.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.205+0000 7f5b31209640 1 Processor -- start 2024-04-25T09:04:16.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.205+0000 7f5b31209640 1 -- start start 2024-04-25T09:04:16.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.206+0000 7f5b31209640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5b140051c0 0x7f5b2c152980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:16.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.206+0000 7f5b31209640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b2c158b00 0x7f5b2c0c5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:16.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.206+0000 7f5b31209640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b2c0c7850 0x7f5b2c0af520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:16.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.206+0000 7f5b31209640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2c0b1ba0 con 0x7f5b2c0c7850 2024-04-25T09:04:16.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.206+0000 7f5b31209640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2c0b18c0 con 0x7f5b2c158b00 2024-04-25T09:04:16.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.206+0000 7f5b31209640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2c0ae5c0 con 0x7f5b140051c0 2024-04-25T09:04:16.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.206+0000 7f5b2b59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b2c0c7850 0x7f5b2c0af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:16.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.206+0000 7f5b2b59e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b2c0c7850 0x7f5b2c0af520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47578/0 (socket says 172.21.15.43:47578) 2024-04-25T09:04:16.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.206+0000 7f5b2b59e640 1 -- 172.21.15.43:0/20589974 learned_addr learned my addr 172.21.15.43:0/20589974 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:16.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.206+0000 7f5b2ad9d640 1 --2- 172.21.15.43:0/20589974 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5b140051c0 0x7f5b2c152980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:16.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.206+0000 7f5b2a59c640 1 --2- 172.21.15.43:0/20589974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b2c158b00 0x7f5b2c0c5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:16.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.206+0000 7f5b2b59e640 1 -- 172.21.15.43:0/20589974 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5b140051c0 msgr2=0x7f5b2c152980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:16.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.206+0000 7f5b2b59e640 1 --2- 172.21.15.43:0/20589974 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5b140051c0 0x7f5b2c152980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.206+0000 7f5b2b59e640 1 -- 172.21.15.43:0/20589974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b2c158b00 msgr2=0x7f5b2c0c5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:16.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.206+0000 7f5b2b59e640 1 --2- 172.21.15.43:0/20589974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b2c158b00 0x7f5b2c0c5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.206+0000 7f5b2b59e640 1 -- 172.21.15.43:0/20589974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b2c0c6a90 con 0x7f5b2c0c7850 2024-04-25T09:04:16.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.207+0000 7f5b2b59e640 1 --2- 172.21.15.43:0/20589974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b2c0c7850 0x7f5b2c0af520 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f5b24001810 tx=0x7f5b24001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:16.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.207+0000 7f5b137fe640 1 -- 172.21.15.43:0/20589974 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b24007a30 con 0x7f5b2c0c7850 2024-04-25T09:04:16.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.207+0000 7f5b137fe640 1 -- 172.21.15.43:0/20589974 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b24000ed0 con 0x7f5b2c0c7850 2024-04-25T09:04:16.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.207+0000 7f5b31209640 1 -- 172.21.15.43:0/20589974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b2c0c6d70 con 0x7f5b2c0c7850 2024-04-25T09:04:16.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.207+0000 7f5b137fe640 1 -- 172.21.15.43:0/20589974 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b24005480 con 0x7f5b2c0c7850 2024-04-25T09:04:16.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.207+0000 7f5b31209640 1 -- 172.21.15.43:0/20589974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b2c0c7330 con 0x7f5b2c0c7850 2024-04-25T09:04:16.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.208+0000 7f5b31209640 1 -- 172.21.15.43:0/20589974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b2c068a40 con 0x7f5b2c0c7850 2024-04-25T09:04:16.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.208+0000 7f5b137fe640 1 -- 172.21.15.43:0/20589974 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5b24032440 con 0x7f5b2c0c7850 2024-04-25T09:04:16.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.209+0000 7f5b137fe640 1 --2- 172.21.15.43:0/20589974 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5b1806b4d0 0x7f5b1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:16.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.209+0000 7f5b137fe640 1 -- 172.21.15.43:0/20589974 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5b2402ae00 con 0x7f5b2c0c7850 2024-04-25T09:04:16.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.209+0000 7f5b2ad9d640 1 --2- 172.21.15.43:0/20589974 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5b1806b4d0 0x7f5b1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:16.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.210+0000 7f5b2ad9d640 1 --2- 172.21.15.43:0/20589974 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5b1806b4d0 0x7f5b1806d990 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c014450 tx=0x7f5b1c00b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:16.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.213+0000 7f5b137fe640 1 -- 172.21.15.43:0/20589974 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5b2c068a40 con 0x7f5b2c0c7850 2024-04-25T09:04:16.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.429+0000 7f5b31209640 1 -- 172.21.15.43:0/20589974 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b2c074ef0 con 0x7f5b2c0c7850 2024-04-25T09:04:16.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.430+0000 7f5b137fe640 1 -- 172.21.15.43:0/20589974 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4945 (secure 0 0 0) 0x7f5b24077770 con 0x7f5b2c0c7850 2024-04-25T09:04:16.418 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:04:16.419 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":87,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/2225978762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2225978762},{"type":"v1","addr":"172.21.15.193:6836","nonce":2225978762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9711,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/454458092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":454458092},{"type":"v1","addr":"172.21.15.43:6833","nonce":454458092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9720,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2314901977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":2314901977},{"type":"v1","addr":"172.21.15.43:6835","nonce":2314901977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:04:13.410799+0000","modified":"2024-04-25T09:04:15.424504+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9732},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9732":{"gid":9732,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.15.193:6837/2585534546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2585534546},{"type":"v1","addr":"172.21.15.193:6837","nonce":2585534546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T09:04:16.419 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 87 2024-04-25T09:04:16.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.432+0000 7f5b31209640 1 -- 172.21.15.43:0/20589974 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5b1806b4d0 msgr2=0x7f5b1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:16.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.432+0000 7f5b31209640 1 --2- 172.21.15.43:0/20589974 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5b1806b4d0 0x7f5b1806d990 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c014450 tx=0x7f5b1c00b040 comp rx=0 tx=0).stop 2024-04-25T09:04:16.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.432+0000 7f5b31209640 1 -- 172.21.15.43:0/20589974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b2c0c7850 msgr2=0x7f5b2c0af520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:16.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.432+0000 7f5b31209640 1 --2- 172.21.15.43:0/20589974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b2c0c7850 0x7f5b2c0af520 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f5b24001810 tx=0x7f5b24001120 comp rx=0 tx=0).stop 2024-04-25T09:04:16.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.432+0000 7f5b31209640 1 -- 172.21.15.43:0/20589974 shutdown_connections 2024-04-25T09:04:16.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.432+0000 7f5b31209640 1 --2- 172.21.15.43:0/20589974 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5b2c0c7850 0x7f5b2c0af520 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.432+0000 7f5b31209640 1 --2- 172.21.15.43:0/20589974 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5b1806b4d0 0x7f5b1806d990 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.432+0000 7f5b31209640 1 --2- 172.21.15.43:0/20589974 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5b2c158b00 0x7f5b2c0c5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.432+0000 7f5b31209640 1 --2- 172.21.15.43:0/20589974 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5b140051c0 0x7f5b2c152980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.432+0000 7f5b31209640 1 -- 172.21.15.43:0/20589974 >> 172.21.15.43:0/20589974 conn(0x7f5b2c0b32e0 msgr2=0x7f5b2c0cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:16.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.432+0000 7f5b31209640 1 -- 172.21.15.43:0/20589974 shutdown_connections 2024-04-25T09:04:16.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.432+0000 7f5b31209640 1 -- 172.21.15.43:0/20589974 wait complete. 2024-04-25T09:04:16.431 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:04:16.503 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:04:16.506 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:04:16.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.544+0000 7f7c9ce9e640 1 Processor -- start 2024-04-25T09:04:16.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.544+0000 7f7c9ce9e640 1 -- start start 2024-04-25T09:04:16.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.545+0000 7f7c9ce9e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7c9815e0f0 0x7f7c98072080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:16.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.545+0000 7f7c9ce9e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7c981582b0 0x7f7c980aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:16.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.545+0000 7f7c9ce9e640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c9806f0e0 con 0x7f7c98070b20 2024-04-25T09:04:16.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.545+0000 7f7c9ce9e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c980725c0 con 0x7f7c981582b0 2024-04-25T09:04:16.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.545+0000 7f7c9ce9e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c980ad0f0 con 0x7f7c9815e0f0 2024-04-25T09:04:16.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.545+0000 7f7c96d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7c981582b0 0x7f7c980aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:16.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.546+0000 7f7c96d9d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7c981582b0 0x7f7c980aeac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47588/0 (socket says 172.21.15.43:47588) 2024-04-25T09:04:16.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.546+0000 7f7c96d9d640 1 -- 172.21.15.43:0/2643734258 learned_addr learned my addr 172.21.15.43:0/2643734258 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:16.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.546+0000 7f7c9659c640 1 --2- 172.21.15.43:0/2643734258 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7c9815e0f0 0x7f7c98072080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:16.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.546+0000 7f7c96d9d640 1 -- 172.21.15.43:0/2643734258 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7c9815e0f0 msgr2=0x7f7c98072080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:16.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.546+0000 7f7c96d9d640 1 --2- 172.21.15.43:0/2643734258 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7c9815e0f0 0x7f7c98072080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.546+0000 7f7c96d9d640 1 -- 172.21.15.43:0/2643734258 >> v1:172.21.15.193:6789/0 conn(0x7f7c98070b20 legacy=0x7f7c98157ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:16.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.546+0000 7f7c96d9d640 1 -- 172.21.15.43:0/2643734258 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c98070fe0 con 0x7f7c981582b0 2024-04-25T09:04:16.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.547+0000 7f7c96d9d640 1 --2- 172.21.15.43:0/2643734258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c981582b0 0x7f7c980aeac0 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f7c90009e50 tx=0x7f7c9002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=473648e6dd0055c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:16.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.547+0000 7f7c9559a640 1 -- 172.21.15.43:0/2643734258 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c90036070 con 0x7f7c981582b0 2024-04-25T09:04:16.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.547+0000 7f7c9559a640 1 -- 172.21.15.43:0/2643734258 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c90031600 con 0x7f7c981582b0 2024-04-25T09:04:16.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.547+0000 7f7c9559a640 1 -- 172.21.15.43:0/2643734258 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c900319d0 con 0x7f7c981582b0 2024-04-25T09:04:16.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.547+0000 7f7c9ce9e640 1 -- 172.21.15.43:0/2643734258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c981582b0 msgr2=0x7f7c980aeac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:16.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.547+0000 7f7c9ce9e640 1 --2- 172.21.15.43:0/2643734258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c981582b0 0x7f7c980aeac0 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f7c90009e50 tx=0x7f7c9002a710 comp rx=0 tx=0).stop 2024-04-25T09:04:16.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.547+0000 7f7c9ce9e640 1 -- 172.21.15.43:0/2643734258 shutdown_connections 2024-04-25T09:04:16.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.547+0000 7f7c9ce9e640 1 --2- 172.21.15.43:0/2643734258 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c981582b0 0x7f7c980aeac0 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.547+0000 7f7c9ce9e640 1 --2- 172.21.15.43:0/2643734258 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7c9815e0f0 0x7f7c98072080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.547+0000 7f7c9ce9e640 1 -- 172.21.15.43:0/2643734258 >> 172.21.15.43:0/2643734258 conn(0x7f7c9806ab30 msgr2=0x7f7c9806af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:16.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.547+0000 7f7c9ce9e640 1 -- 172.21.15.43:0/2643734258 shutdown_connections 2024-04-25T09:04:16.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.547+0000 7f7c9ce9e640 1 -- 172.21.15.43:0/2643734258 wait complete. 2024-04-25T09:04:16.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.549+0000 7f7c9ce9e640 1 Processor -- start 2024-04-25T09:04:16.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.549+0000 7f7c9ce9e640 1 -- start start 2024-04-25T09:04:16.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.549+0000 7f7c9ce9e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7c98070b20 0x7f7c981335f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:16.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.550+0000 7f7c9ce9e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c981582b0 0x7f7c980cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:16.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.550+0000 7f7c9ce9e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c9815e0f0 0x7f7c980cc900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:16.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.550+0000 7f7c9ce9e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c98071880 con 0x7f7c981582b0 2024-04-25T09:04:16.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.550+0000 7f7c9ce9e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c980715a0 con 0x7f7c9815e0f0 2024-04-25T09:04:16.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.550+0000 7f7c9ce9e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c98071b60 con 0x7f7c98070b20 2024-04-25T09:04:16.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.550+0000 7f7c95d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c981582b0 0x7f7c980cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:16.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.550+0000 7f7c95d9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c981582b0 0x7f7c980cf2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47590/0 (socket says 172.21.15.43:47590) 2024-04-25T09:04:16.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.550+0000 7f7c95d9b640 1 -- 172.21.15.43:0/1175363191 learned_addr learned my addr 172.21.15.43:0/1175363191 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:16.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.550+0000 7f7c9659c640 1 --2- 172.21.15.43:0/1175363191 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7c98070b20 0x7f7c981335f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:16.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.550+0000 7f7c96d9d640 1 --2- 172.21.15.43:0/1175363191 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c9815e0f0 0x7f7c980cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:16.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.550+0000 7f7c95d9b640 1 -- 172.21.15.43:0/1175363191 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7c98070b20 msgr2=0x7f7c981335f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:16.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.550+0000 7f7c95d9b640 1 --2- 172.21.15.43:0/1175363191 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7c98070b20 0x7f7c981335f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.550+0000 7f7c95d9b640 1 -- 172.21.15.43:0/1175363191 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c9815e0f0 msgr2=0x7f7c980cc900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:16.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.550+0000 7f7c95d9b640 1 --2- 172.21.15.43:0/1175363191 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c9815e0f0 0x7f7c980cc900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.550+0000 7f7c95d9b640 1 -- 172.21.15.43:0/1175363191 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c9809ce20 con 0x7f7c981582b0 2024-04-25T09:04:16.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.550+0000 7f7c95d9b640 1 --2- 172.21.15.43:0/1175363191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c981582b0 0x7f7c980cf2b0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f7c8800c400 tx=0x7f7c8800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:16.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.550+0000 7f7c86ffd640 1 -- 172.21.15.43:0/1175363191 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c880052d0 con 0x7f7c981582b0 2024-04-25T09:04:16.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.550+0000 7f7c86ffd640 1 -- 172.21.15.43:0/1175363191 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c88005490 con 0x7f7c981582b0 2024-04-25T09:04:16.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.550+0000 7f7c9ce9e640 1 -- 172.21.15.43:0/1175363191 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c9809d100 con 0x7f7c981582b0 2024-04-25T09:04:16.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.550+0000 7f7c86ffd640 1 -- 172.21.15.43:0/1175363191 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c88007260 con 0x7f7c981582b0 2024-04-25T09:04:16.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.550+0000 7f7c9ce9e640 1 -- 172.21.15.43:0/1175363191 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c9809d6c0 con 0x7f7c981582b0 2024-04-25T09:04:16.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.551+0000 7f7c9ce9e640 1 -- 172.21.15.43:0/1175363191 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c9809cbb0 con 0x7f7c981582b0 2024-04-25T09:04:16.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.552+0000 7f7c86ffd640 1 -- 172.21.15.43:0/1175363191 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7c88007400 con 0x7f7c981582b0 2024-04-25T09:04:16.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.552+0000 7f7c86ffd640 1 --2- 172.21.15.43:0/1175363191 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7c7806b4d0 0x7f7c7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:16.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.552+0000 7f7c86ffd640 1 -- 172.21.15.43:0/1175363191 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7c880895f0 con 0x7f7c981582b0 2024-04-25T09:04:16.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.553+0000 7f7c9659c640 1 --2- 172.21.15.43:0/1175363191 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7c7806b4d0 0x7f7c7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:16.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.553+0000 7f7c9659c640 1 --2- 172.21.15.43:0/1175363191 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7c7806b4d0 0x7f7c7806d990 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c009aa0 tx=0x7f7c8c009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:16.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.557+0000 7f7c86ffd640 1 -- 172.21.15.43:0/1175363191 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7c9806f0e0 con 0x7f7c981582b0 2024-04-25T09:04:16.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.739+0000 7f7c9ce9e640 1 -- 172.21.15.43:0/1175363191 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c98074ef0 con 0x7f7c981582b0 2024-04-25T09:04:16.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.740+0000 7f7c86ffd640 1 -- 172.21.15.43:0/1175363191 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+16643 (secure 0 0 0) 0x7f7c88057eb0 con 0x7f7c981582b0 2024-04-25T09:04:16.728 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:04:16.729 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":139,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:04:15.059128+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:04:11.735785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T09:04:12.383453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:04:16.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.742+0000 7f7c9ce9e640 1 -- 172.21.15.43:0/1175363191 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7c7806b4d0 msgr2=0x7f7c7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:16.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.742+0000 7f7c9ce9e640 1 --2- 172.21.15.43:0/1175363191 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7c7806b4d0 0x7f7c7806d990 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f7c8c009aa0 tx=0x7f7c8c009190 comp rx=0 tx=0).stop 2024-04-25T09:04:16.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.742+0000 7f7c9ce9e640 1 -- 172.21.15.43:0/1175363191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c981582b0 msgr2=0x7f7c980cf2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:16.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.742+0000 7f7c9ce9e640 1 --2- 172.21.15.43:0/1175363191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c981582b0 0x7f7c980cf2b0 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f7c8800c400 tx=0x7f7c8800c8c0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.742+0000 7f7c9ce9e640 1 -- 172.21.15.43:0/1175363191 shutdown_connections 2024-04-25T09:04:16.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.742+0000 7f7c9ce9e640 1 --2- 172.21.15.43:0/1175363191 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7c7806b4d0 0x7f7c7806d990 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.742+0000 7f7c9ce9e640 1 --2- 172.21.15.43:0/1175363191 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7c9815e0f0 0x7f7c980cc900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.742+0000 7f7c9ce9e640 1 --2- 172.21.15.43:0/1175363191 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7c981582b0 0x7f7c980cf2b0 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.742+0000 7f7c9ce9e640 1 --2- 172.21.15.43:0/1175363191 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7c98070b20 0x7f7c981335f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.742+0000 7f7c9ce9e640 1 -- 172.21.15.43:0/1175363191 >> 172.21.15.43:0/1175363191 conn(0x7f7c9806ab30 msgr2=0x7f7c980afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:16.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.742+0000 7f7c9ce9e640 1 -- 172.21.15.43:0/1175363191 shutdown_connections 2024-04-25T09:04:16.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.742+0000 7f7c9ce9e640 1 -- 172.21.15.43:0/1175363191 wait complete. 2024-04-25T09:04:16.742 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T09:04:16.742 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:04:16.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.855+0000 7f7a6f65d640 1 Processor -- start 2024-04-25T09:04:16.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.855+0000 7f7a6f65d640 1 -- start start 2024-04-25T09:04:16.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.856+0000 7f7a6f65d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7a6815def0 0x7f7a6806be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:16.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.856+0000 7f7a6f65d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7a681619f0 0x7f7a68161e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:16.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.856+0000 7f7a6f65d640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f7a68162390 0x7f7a680cfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:16.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.856+0000 7f7a6f65d640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a680d0e90 con 0x7f7a681619f0 2024-04-25T09:04:16.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.856+0000 7f7a6f65d640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a6806c390 con 0x7f7a68162390 2024-04-25T09:04:16.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.856+0000 7f7a6f65d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a68162820 con 0x7f7a6815def0 2024-04-25T09:04:16.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.856+0000 7f7a6cbf8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7a681619f0 0x7f7a68161e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:16.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.856+0000 7f7a6cbf8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7a681619f0 0x7f7a68161e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47596/0 (socket says 172.21.15.43:47596) 2024-04-25T09:04:16.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.856+0000 7f7a6cbf8640 1 -- 172.21.15.43:0/814408792 learned_addr learned my addr 172.21.15.43:0/814408792 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:16.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.856+0000 7f7a6dbfa640 1 --2- 172.21.15.43:0/814408792 >> v2:172.21.15.193:3300/0 conn(0x7f7a68162390 0x7f7a680cfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:16.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.856+0000 7f7a6d3f9640 1 --2- 172.21.15.43:0/814408792 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7a6815def0 0x7f7a6806be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:16.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.856+0000 7f7a6cbf8640 1 -- 172.21.15.43:0/814408792 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7a6815def0 msgr2=0x7f7a6806be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:16.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.856+0000 7f7a6cbf8640 1 --2- 172.21.15.43:0/814408792 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7a6815def0 0x7f7a6806be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.856+0000 7f7a6cbf8640 1 -- 172.21.15.43:0/814408792 >> v2:172.21.15.193:3300/0 conn(0x7f7a68162390 msgr2=0x7f7a680cfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:16.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.856+0000 7f7a6cbf8640 1 --2- 172.21.15.43:0/814408792 >> v2:172.21.15.193:3300/0 conn(0x7f7a68162390 0x7f7a680cfa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.856+0000 7f7a6cbf8640 1 -- 172.21.15.43:0/814408792 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a6815e2d0 con 0x7f7a681619f0 2024-04-25T09:04:16.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.857+0000 7f7a6cbf8640 1 --2- 172.21.15.43:0/814408792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a681619f0 0x7f7a68161e50 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f7a58009850 tx=0x7f7a5802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a983014509922d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:16.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.857+0000 7f7a5ffff640 1 -- 172.21.15.43:0/814408792 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a58004030 con 0x7f7a681619f0 2024-04-25T09:04:16.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.857+0000 7f7a5ffff640 1 -- 172.21.15.43:0/814408792 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a580041d0 con 0x7f7a681619f0 2024-04-25T09:04:16.847 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.857+0000 7f7a5ffff640 1 -- 172.21.15.43:0/814408792 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a58003610 con 0x7f7a681619f0 2024-04-25T09:04:16.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.858+0000 7f7a6f65d640 1 -- 172.21.15.43:0/814408792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a681619f0 msgr2=0x7f7a68161e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:16.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.858+0000 7f7a6f65d640 1 --2- 172.21.15.43:0/814408792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a681619f0 0x7f7a68161e50 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f7a58009850 tx=0x7f7a5802a1e0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.858+0000 7f7a6f65d640 1 -- 172.21.15.43:0/814408792 shutdown_connections 2024-04-25T09:04:16.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.858+0000 7f7a6f65d640 1 --2- 172.21.15.43:0/814408792 >> v2:172.21.15.193:3300/0 conn(0x7f7a68162390 0x7f7a680cfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.858+0000 7f7a6f65d640 1 --2- 172.21.15.43:0/814408792 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a681619f0 0x7f7a68161e50 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.858+0000 7f7a6f65d640 1 --2- 172.21.15.43:0/814408792 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7a6815def0 0x7f7a6806be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.858+0000 7f7a6f65d640 1 -- 172.21.15.43:0/814408792 >> 172.21.15.43:0/814408792 conn(0x7f7a680b32e0 msgr2=0x7f7a680b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:16.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.858+0000 7f7a6f65d640 1 -- 172.21.15.43:0/814408792 shutdown_connections 2024-04-25T09:04:16.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.858+0000 7f7a6f65d640 1 -- 172.21.15.43:0/814408792 wait complete. 2024-04-25T09:04:16.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.859+0000 7f7a6f65d640 1 Processor -- start 2024-04-25T09:04:16.848 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.859+0000 7f7a6f65d640 1 -- start start 2024-04-25T09:04:16.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.860+0000 7f7a6f65d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a6815def0 0x7f7a6815f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:16.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.860+0000 7f7a6f65d640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a681619f0 0x7f7a680c5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:16.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.860+0000 7f7a6f65d640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7a68162390 0x7f7a68149b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:16.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.860+0000 7f7a6f65d640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a680b15e0 con 0x7f7a6815def0 2024-04-25T09:04:16.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.860+0000 7f7a6f65d640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a680b1300 con 0x7f7a681619f0 2024-04-25T09:04:16.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.860+0000 7f7a6f65d640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a680b18c0 con 0x7f7a68162390 2024-04-25T09:04:16.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.860+0000 7f7a6d3f9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a6815def0 0x7f7a6815f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:16.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.860+0000 7f7a6d3f9640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a6815def0 0x7f7a6815f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47610/0 (socket says 172.21.15.43:47610) 2024-04-25T09:04:16.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.860+0000 7f7a6d3f9640 1 -- 172.21.15.43:0/212930079 learned_addr learned my addr 172.21.15.43:0/212930079 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:16.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.860+0000 7f7a6cbf8640 1 --2- 172.21.15.43:0/212930079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a681619f0 0x7f7a680c5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:16.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.860+0000 7f7a6dbfa640 1 --2- 172.21.15.43:0/212930079 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7a68162390 0x7f7a68149b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:16.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.860+0000 7f7a6d3f9640 1 -- 172.21.15.43:0/212930079 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7a68162390 msgr2=0x7f7a68149b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:16.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.860+0000 7f7a6d3f9640 1 --2- 172.21.15.43:0/212930079 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7a68162390 0x7f7a68149b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.860+0000 7f7a6d3f9640 1 -- 172.21.15.43:0/212930079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a681619f0 msgr2=0x7f7a680c5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:16.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.860+0000 7f7a6d3f9640 1 --2- 172.21.15.43:0/212930079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a681619f0 0x7f7a680c5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:16.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.860+0000 7f7a6d3f9640 1 -- 172.21.15.43:0/212930079 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a68160e90 con 0x7f7a6815def0 2024-04-25T09:04:16.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.860+0000 7f7a6cbf8640 1 --2- 172.21.15.43:0/212930079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a681619f0 0x7f7a680c5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:16.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.861+0000 7f7a6d3f9640 1 --2- 172.21.15.43:0/212930079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a6815def0 0x7f7a6815f4f0 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f7a6400d800 tx=0x7f7a6400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:16.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.861+0000 7f7a5dffb640 1 -- 172.21.15.43:0/212930079 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6400b6c0 con 0x7f7a6815def0 2024-04-25T09:04:16.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.861+0000 7f7a5dffb640 1 -- 172.21.15.43:0/212930079 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a64019070 con 0x7f7a6815def0 2024-04-25T09:04:16.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.861+0000 7f7a6f65d640 1 -- 172.21.15.43:0/212930079 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a6815fdf0 con 0x7f7a6815def0 2024-04-25T09:04:16.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.861+0000 7f7a5dffb640 1 -- 172.21.15.43:0/212930079 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6400ba80 con 0x7f7a6815def0 2024-04-25T09:04:16.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.861+0000 7f7a6f65d640 1 -- 172.21.15.43:0/212930079 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a681603b0 con 0x7f7a6815def0 2024-04-25T09:04:16.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.862+0000 7f7a6f65d640 1 -- 172.21.15.43:0/212930079 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a68074ef0 con 0x7f7a6815def0 2024-04-25T09:04:16.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.862+0000 7f7a5dffb640 1 -- 172.21.15.43:0/212930079 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7a6400bc20 con 0x7f7a6815def0 2024-04-25T09:04:16.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.863+0000 7f7a5dffb640 1 --2- 172.21.15.43:0/212930079 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7a4c06b4d0 0x7f7a4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:16.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.863+0000 7f7a5dffb640 1 -- 172.21.15.43:0/212930079 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7a6408a620 con 0x7f7a6815def0 2024-04-25T09:04:16.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.863+0000 7f7a6cbf8640 1 --2- 172.21.15.43:0/212930079 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7a4c06b4d0 0x7f7a4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:16.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.864+0000 7f7a6cbf8640 1 --2- 172.21.15.43:0/212930079 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7a4c06b4d0 0x7f7a4c06d990 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f7a58033000 tx=0x7f7a5802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:16.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:16.867+0000 7f7a5dffb640 1 -- 172.21.15.43:0/212930079 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7a64059530 con 0x7f7a6815def0 2024-04-25T09:04:17.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.042+0000 7f7a6f65d640 1 -- 172.21.15.43:0/212930079 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7a680b7710 con 0x7f7a4c06b4d0 2024-04-25T09:04:17.037 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.049+0000 7f7a5dffb640 1 -- 172.21.15.43:0/212930079 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247089 (secure 0 0 0) 0x7f7a680b7710 con 0x7f7a4c06b4d0 2024-04-25T09:04:17.040 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:04:17.041 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-25T09:04:17.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.052+0000 7f7a6f65d640 1 -- 172.21.15.43:0/212930079 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7a4c06b4d0 msgr2=0x7f7a4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:17.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.052+0000 7f7a6f65d640 1 --2- 172.21.15.43:0/212930079 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7a4c06b4d0 0x7f7a4c06d990 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f7a58033000 tx=0x7f7a5802e000 comp rx=0 tx=0).stop 2024-04-25T09:04:17.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.052+0000 7f7a6f65d640 1 -- 172.21.15.43:0/212930079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a6815def0 msgr2=0x7f7a6815f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:17.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.052+0000 7f7a6f65d640 1 --2- 172.21.15.43:0/212930079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a6815def0 0x7f7a6815f4f0 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f7a6400d800 tx=0x7f7a6400dcc0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.052+0000 7f7a6f65d640 1 -- 172.21.15.43:0/212930079 shutdown_connections 2024-04-25T09:04:17.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.052+0000 7f7a6f65d640 1 --2- 172.21.15.43:0/212930079 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7a68162390 0x7f7a68149b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.052+0000 7f7a6f65d640 1 --2- 172.21.15.43:0/212930079 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7a4c06b4d0 0x7f7a4c06d990 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.052+0000 7f7a6f65d640 1 --2- 172.21.15.43:0/212930079 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7a681619f0 0x7f7a680c5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.052+0000 7f7a6f65d640 1 --2- 172.21.15.43:0/212930079 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7a6815def0 0x7f7a6815f4f0 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.043 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.052+0000 7f7a6f65d640 1 -- 172.21.15.43:0/212930079 >> 172.21.15.43:0/212930079 conn(0x7f7a680b32e0 msgr2=0x7f7a6806c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:17.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.052+0000 7f7a6f65d640 1 -- 172.21.15.43:0/212930079 shutdown_connections 2024-04-25T09:04:17.044 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.052+0000 7f7a6f65d640 1 -- 172.21.15.43:0/212930079 wait complete. 2024-04-25T09:04:17.051 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":441,"stamp":"2024-04-25T09:04:15.835885+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":498656,"kb_used_data":14568,"kb_used_omap":12,"kb_used_meta":483955,"kb_avail":754476064,"statfs":{"total":773094113280,"available":772583489536,"internally_reserved":0,"allocated":14917632,"data_stored":7178872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":495570481},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000966"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.053615+0000","last_change":"2024-04-25T09:04:13.053615+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.053615+0000","last_undegraded":"2024-04-25T09:04:13.053615+0000","last_fullsized":"2024-04-25T09:04:13.053615+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.047818+0000","last_change":"2024-04-25T09:04:13.047818+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.047818+0000","last_undegraded":"2024-04-25T09:04:13.047818+0000","last_fullsized":"2024-04-25T09:04:13.047818+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.049520+0000","last_change":"2024-04-25T09:04:13.049520+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.049520+0000","last_undegraded":"2024-04-25T09:04:13.049520+0000","last_fullsized":"2024-04-25T09:04:13.049520+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046148+0000","last_change":"2024-04-25T09:04:13.046148+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046148+0000","last_undegraded":"2024-04-25T09:04:13.046148+0000","last_fullsized":"2024-04-25T09:04:13.046148+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.047544+0000","last_change":"2024-04-25T09:04:13.047544+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.047544+0000","last_undegraded":"2024-04-25T09:04:13.047544+0000","last_fullsized":"2024-04-25T09:04:13.047544+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.051007+0000","last_change":"2024-04-25T09:04:13.051007+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.051007+0000","last_undegraded":"2024-04-25T09:04:13.051007+0000","last_fullsized":"2024-04-25T09:04:13.051007+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.048146+0000","last_change":"2024-04-25T09:04:13.048146+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.048146+0000","last_undegraded":"2024-04-25T09:04:13.048146+0000","last_fullsized":"2024-04-25T09:04:13.048146+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.049952+0000","last_change":"2024-04-25T09:04:13.049952+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.049952+0000","last_undegraded":"2024-04-25T09:04:13.049952+0000","last_fullsized":"2024-04-25T09:04:13.049952+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.047013+0000","last_change":"2024-04-25T09:04:13.047013+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.047013+0000","last_undegraded":"2024-04-25T09:04:13.047013+0000","last_fullsized":"2024-04-25T09:04:13.047013+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.052404+0000","last_change":"2024-04-25T09:04:13.052404+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.052404+0000","last_undegraded":"2024-04-25T09:04:13.052404+0000","last_fullsized":"2024-04-25T09:04:13.052404+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.050451+0000","last_change":"2024-04-25T09:04:13.050451+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.050451+0000","last_undegraded":"2024-04-25T09:04:13.050451+0000","last_fullsized":"2024-04-25T09:04:13.050451+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.054302+0000","last_change":"2024-04-25T09:04:13.054302+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.054302+0000","last_undegraded":"2024-04-25T09:04:13.054302+0000","last_fullsized":"2024-04-25T09:04:13.054302+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.047149+0000","last_change":"2024-04-25T09:04:13.047149+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.047149+0000","last_undegraded":"2024-04-25T09:04:13.047149+0000","last_fullsized":"2024-04-25T09:04:13.047149+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.048543+0000","last_change":"2024-04-25T09:04:13.048543+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.048543+0000","last_undegraded":"2024-04-25T09:04:13.048543+0000","last_fullsized":"2024-04-25T09:04:13.048543+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.050737+0000","last_change":"2024-04-25T09:04:13.050737+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.050737+0000","last_undegraded":"2024-04-25T09:04:13.050737+0000","last_fullsized":"2024-04-25T09:04:13.050737+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.049349+0000","last_change":"2024-04-25T09:04:13.049349+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.049349+0000","last_undegraded":"2024-04-25T09:04:13.049349+0000","last_fullsized":"2024-04-25T09:04:13.049349+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.051581+0000","last_change":"2024-04-25T09:04:13.051581+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.051581+0000","last_undegraded":"2024-04-25T09:04:13.051581+0000","last_fullsized":"2024-04-25T09:04:13.051581+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.052944+0000","last_change":"2024-04-25T09:04:13.052944+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.052944+0000","last_undegraded":"2024-04-25T09:04:13.052944+0000","last_fullsized":"2024-04-25T09:04:13.052944+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.049189+0000","last_change":"2024-04-25T09:04:13.049189+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.049189+0000","last_undegraded":"2024-04-25T09:04:13.049189+0000","last_fullsized":"2024-04-25T09:04:13.049189+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046642+0000","last_change":"2024-04-25T09:04:13.046642+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046642+0000","last_undegraded":"2024-04-25T09:04:13.046642+0000","last_fullsized":"2024-04-25T09:04:13.046642+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.051136+0000","last_change":"2024-04-25T09:04:13.051136+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.051136+0000","last_undegraded":"2024-04-25T09:04:13.051136+0000","last_fullsized":"2024-04-25T09:04:13.051136+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.048012+0000","last_change":"2024-04-25T09:04:13.048012+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.048012+0000","last_undegraded":"2024-04-25T09:04:13.048012+0000","last_fullsized":"2024-04-25T09:04:13.048012+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046277+0000","last_change":"2024-04-25T09:04:13.046277+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046277+0000","last_undegraded":"2024-04-25T09:04:13.046277+0000","last_fullsized":"2024-04-25T09:04:13.046277+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.051677+0000","last_change":"2024-04-25T09:04:13.051677+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.051677+0000","last_undegraded":"2024-04-25T09:04:13.051677+0000","last_fullsized":"2024-04-25T09:04:13.051677+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.052436+0000","last_change":"2024-04-25T09:04:13.052436+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.052436+0000","last_undegraded":"2024-04-25T09:04:13.052436+0000","last_fullsized":"2024-04-25T09:04:13.052436+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.049685+0000","last_change":"2024-04-25T09:04:13.049685+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.049685+0000","last_undegraded":"2024-04-25T09:04:13.049685+0000","last_fullsized":"2024-04-25T09:04:13.049685+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.048115+0000","last_change":"2024-04-25T09:04:13.048115+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.048115+0000","last_undegraded":"2024-04-25T09:04:13.048115+0000","last_fullsized":"2024-04-25T09:04:13.048115+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046108+0000","last_change":"2024-04-25T09:04:13.046108+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046108+0000","last_undegraded":"2024-04-25T09:04:13.046108+0000","last_fullsized":"2024-04-25T09:04:13.046108+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.048479+0000","last_change":"2024-04-25T09:04:13.048479+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.048479+0000","last_undegraded":"2024-04-25T09:04:13.048479+0000","last_fullsized":"2024-04-25T09:04:13.048479+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.047436+0000","last_change":"2024-04-25T09:04:13.047436+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.047436+0000","last_undegraded":"2024-04-25T09:04:13.047436+0000","last_fullsized":"2024-04-25T09:04:13.047436+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046485+0000","last_change":"2024-04-25T09:04:13.046485+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046485+0000","last_undegraded":"2024-04-25T09:04:13.046485+0000","last_fullsized":"2024-04-25T09:04:13.046485+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.048912+0000","last_change":"2024-04-25T09:04:13.048912+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.048912+0000","last_undegraded":"2024-04-25T09:04:13.048912+0000","last_fullsized":"2024-04-25T09:04:13.048912+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.051989+0000","last_change":"2024-04-25T09:04:13.051989+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.051989+0000","last_undegraded":"2024-04-25T09:04:13.051989+0000","last_fullsized":"2024-04-25T09:04:13.051989+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.063563+0000","last_change":"2024-04-25T09:04:13.062156+0000","last_active":"2024-04-25T09:04:13.063563+0000","last_peered":"2024-04-25T09:04:13.063563+0000","last_clean":"2024-04-25T09:04:13.063563+0000","last_became_active":"2024-04-25T09:04:13.061852+0000","last_became_peered":"2024-04-25T09:04:13.061852+0000","last_unstale":"2024-04-25T09:04:13.063563+0000","last_undegraded":"2024-04-25T09:04:13.063563+0000","last_fullsized":"2024-04-25T09:04:13.063563+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:10.458178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046457+0000","last_change":"2024-04-25T09:04:13.046457+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046457+0000","last_undegraded":"2024-04-25T09:04:13.046457+0000","last_fullsized":"2024-04-25T09:04:13.046457+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.059517+0000","last_change":"2024-04-25T09:04:13.058762+0000","last_active":"2024-04-25T09:04:13.059517+0000","last_peered":"2024-04-25T09:04:13.059517+0000","last_clean":"2024-04-25T09:04:13.059517+0000","last_became_active":"2024-04-25T09:04:13.058335+0000","last_became_peered":"2024-04-25T09:04:13.058335+0000","last_unstale":"2024-04-25T09:04:13.059517+0000","last_undegraded":"2024-04-25T09:04:13.059517+0000","last_fullsized":"2024-04-25T09:04:13.059517+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:59.348802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.048044+0000","last_change":"2024-04-25T09:04:13.048044+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.048044+0000","last_undegraded":"2024-04-25T09:04:13.048044+0000","last_fullsized":"2024-04-25T09:04:13.048044+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.059899+0000","last_change":"2024-04-25T09:04:13.057742+0000","last_active":"2024-04-25T09:04:13.059899+0000","last_peered":"2024-04-25T09:04:13.059899+0000","last_clean":"2024-04-25T09:04:13.059899+0000","last_became_active":"2024-04-25T09:04:13.057448+0000","last_became_peered":"2024-04-25T09:04:13.057448+0000","last_unstale":"2024-04-25T09:04:13.059899+0000","last_undegraded":"2024-04-25T09:04:13.059899+0000","last_fullsized":"2024-04-25T09:04:13.059899+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:47.545889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.053963+0000","last_change":"2024-04-25T09:04:13.053963+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.053963+0000","last_undegraded":"2024-04-25T09:04:13.053963+0000","last_fullsized":"2024-04-25T09:04:13.053963+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.059924+0000","last_change":"2024-04-25T09:04:13.058406+0000","last_active":"2024-04-25T09:04:13.059924+0000","last_peered":"2024-04-25T09:04:13.059924+0000","last_clean":"2024-04-25T09:04:13.059924+0000","last_became_active":"2024-04-25T09:04:13.058144+0000","last_became_peered":"2024-04-25T09:04:13.058144+0000","last_unstale":"2024-04-25T09:04:13.059924+0000","last_undegraded":"2024-04-25T09:04:13.059924+0000","last_fullsized":"2024-04-25T09:04:13.059924+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:33.060943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046085+0000","last_change":"2024-04-25T09:04:13.046085+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046085+0000","last_undegraded":"2024-04-25T09:04:13.046085+0000","last_fullsized":"2024-04-25T09:04:13.046085+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060240+0000","last_change":"2024-04-25T09:04:13.059157+0000","last_active":"2024-04-25T09:04:13.060240+0000","last_peered":"2024-04-25T09:04:13.060240+0000","last_clean":"2024-04-25T09:04:13.060240+0000","last_became_active":"2024-04-25T09:04:13.058883+0000","last_became_peered":"2024-04-25T09:04:13.058883+0000","last_unstale":"2024-04-25T09:04:13.060240+0000","last_undegraded":"2024-04-25T09:04:13.060240+0000","last_fullsized":"2024-04-25T09:04:13.060240+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:38.159347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046155+0000","last_change":"2024-04-25T09:04:13.046155+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046155+0000","last_undegraded":"2024-04-25T09:04:13.046155+0000","last_fullsized":"2024-04-25T09:04:13.046155+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.059938+0000","last_change":"2024-04-25T09:04:13.057593+0000","last_active":"2024-04-25T09:04:13.059938+0000","last_peered":"2024-04-25T09:04:13.059938+0000","last_clean":"2024-04-25T09:04:13.059938+0000","last_became_active":"2024-04-25T09:04:13.057330+0000","last_became_peered":"2024-04-25T09:04:13.057330+0000","last_unstale":"2024-04-25T09:04:13.059938+0000","last_undegraded":"2024-04-25T09:04:13.059938+0000","last_fullsized":"2024-04-25T09:04:13.059938+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:32.141616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046021+0000","last_change":"2024-04-25T09:04:13.046021+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046021+0000","last_undegraded":"2024-04-25T09:04:13.046021+0000","last_fullsized":"2024-04-25T09:04:13.046021+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.056354+0000","last_change":"2024-04-25T09:04:13.054808+0000","last_active":"2024-04-25T09:04:13.056354+0000","last_peered":"2024-04-25T09:04:13.056354+0000","last_clean":"2024-04-25T09:04:13.056354+0000","last_became_active":"2024-04-25T09:04:13.054486+0000","last_became_peered":"2024-04-25T09:04:13.054486+0000","last_unstale":"2024-04-25T09:04:13.056354+0000","last_undegraded":"2024-04-25T09:04:13.056354+0000","last_fullsized":"2024-04-25T09:04:13.056354+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:24.933992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.045906+0000","last_change":"2024-04-25T09:04:13.045906+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.045906+0000","last_undegraded":"2024-04-25T09:04:13.045906+0000","last_fullsized":"2024-04-25T09:04:13.045906+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.057196+0000","last_change":"2024-04-25T09:04:13.056215+0000","last_active":"2024-04-25T09:04:13.057196+0000","last_peered":"2024-04-25T09:04:13.057196+0000","last_clean":"2024-04-25T09:04:13.057196+0000","last_became_active":"2024-04-25T09:04:13.055952+0000","last_became_peered":"2024-04-25T09:04:13.055952+0000","last_unstale":"2024-04-25T09:04:13.057196+0000","last_undegraded":"2024-04-25T09:04:13.057196+0000","last_fullsized":"2024-04-25T09:04:13.057196+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:39.762504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.045952+0000","last_change":"2024-04-25T09:04:13.045952+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.045952+0000","last_undegraded":"2024-04-25T09:04:13.045952+0000","last_fullsized":"2024-04-25T09:04:13.045952+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.057234+0000","last_change":"2024-04-25T09:04:13.051625+0000","last_active":"2024-04-25T09:04:13.057234+0000","last_peered":"2024-04-25T09:04:13.057234+0000","last_clean":"2024-04-25T09:04:13.057234+0000","last_became_active":"2024-04-25T09:04:13.051307+0000","last_became_peered":"2024-04-25T09:04:13.051307+0000","last_unstale":"2024-04-25T09:04:13.057234+0000","last_undegraded":"2024-04-25T09:04:13.057234+0000","last_fullsized":"2024-04-25T09:04:13.057234+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:06.808616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.052044+0000","last_change":"2024-04-25T09:04:13.052044+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.052044+0000","last_undegraded":"2024-04-25T09:04:13.052044+0000","last_fullsized":"2024-04-25T09:04:13.052044+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060131+0000","last_change":"2024-04-25T09:04:13.058585+0000","last_active":"2024-04-25T09:04:13.060131+0000","last_peered":"2024-04-25T09:04:13.060131+0000","last_clean":"2024-04-25T09:04:13.060131+0000","last_became_active":"2024-04-25T09:04:13.058298+0000","last_became_peered":"2024-04-25T09:04:13.058298+0000","last_unstale":"2024-04-25T09:04:13.060131+0000","last_undegraded":"2024-04-25T09:04:13.060131+0000","last_fullsized":"2024-04-25T09:04:13.060131+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:25.170530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.047591+0000","last_change":"2024-04-25T09:04:13.047591+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.047591+0000","last_undegraded":"2024-04-25T09:04:13.047591+0000","last_fullsized":"2024-04-25T09:04:13.047591+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.061150+0000","last_change":"2024-04-25T09:04:13.058981+0000","last_active":"2024-04-25T09:04:13.061150+0000","last_peered":"2024-04-25T09:04:13.061150+0000","last_clean":"2024-04-25T09:04:13.061150+0000","last_became_active":"2024-04-25T09:04:13.058662+0000","last_became_peered":"2024-04-25T09:04:13.058662+0000","last_unstale":"2024-04-25T09:04:13.061150+0000","last_undegraded":"2024-04-25T09:04:13.061150+0000","last_fullsized":"2024-04-25T09:04:13.061150+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:35.416425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.049288+0000","last_change":"2024-04-25T09:04:13.049288+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.049288+0000","last_undegraded":"2024-04-25T09:04:13.049288+0000","last_fullsized":"2024-04-25T09:04:13.049288+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060649+0000","last_change":"2024-04-25T09:04:13.058867+0000","last_active":"2024-04-25T09:04:13.060649+0000","last_peered":"2024-04-25T09:04:13.060649+0000","last_clean":"2024-04-25T09:04:13.060649+0000","last_became_active":"2024-04-25T09:04:13.057622+0000","last_became_peered":"2024-04-25T09:04:13.057622+0000","last_unstale":"2024-04-25T09:04:13.060649+0000","last_undegraded":"2024-04-25T09:04:13.060649+0000","last_fullsized":"2024-04-25T09:04:13.060649+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:47.545889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.045500+0000","last_change":"2024-04-25T09:04:13.045500+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.045500+0000","last_undegraded":"2024-04-25T09:04:13.045500+0000","last_fullsized":"2024-04-25T09:04:13.045500+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.055227+0000","last_change":"2024-04-25T09:04:13.054034+0000","last_active":"2024-04-25T09:04:13.055227+0000","last_peered":"2024-04-25T09:04:13.055227+0000","last_clean":"2024-04-25T09:04:13.055227+0000","last_became_active":"2024-04-25T09:04:13.053760+0000","last_became_peered":"2024-04-25T09:04:13.053760+0000","last_unstale":"2024-04-25T09:04:13.055227+0000","last_undegraded":"2024-04-25T09:04:13.055227+0000","last_fullsized":"2024-04-25T09:04:13.055227+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:39.762504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.045249+0000","last_change":"2024-04-25T09:04:13.045249+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.045249+0000","last_undegraded":"2024-04-25T09:04:13.045249+0000","last_fullsized":"2024-04-25T09:04:13.045249+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060397+0000","last_change":"2024-04-25T09:04:13.056393+0000","last_active":"2024-04-25T09:04:13.060397+0000","last_peered":"2024-04-25T09:04:13.060397+0000","last_clean":"2024-04-25T09:04:13.060397+0000","last_became_active":"2024-04-25T09:04:13.056111+0000","last_became_peered":"2024-04-25T09:04:13.056111+0000","last_unstale":"2024-04-25T09:04:13.060397+0000","last_undegraded":"2024-04-25T09:04:13.060397+0000","last_fullsized":"2024-04-25T09:04:13.060397+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:39.762504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.049874+0000","last_change":"2024-04-25T09:04:13.049874+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.049874+0000","last_undegraded":"2024-04-25T09:04:13.049874+0000","last_fullsized":"2024-04-25T09:04:13.049874+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.062256+0000","last_change":"2024-04-25T09:04:13.061556+0000","last_active":"2024-04-25T09:04:13.062256+0000","last_peered":"2024-04-25T09:04:13.062256+0000","last_clean":"2024-04-25T09:04:13.062256+0000","last_became_active":"2024-04-25T09:04:13.061241+0000","last_became_peered":"2024-04-25T09:04:13.061241+0000","last_unstale":"2024-04-25T09:04:13.062256+0000","last_undegraded":"2024-04-25T09:04:13.062256+0000","last_fullsized":"2024-04-25T09:04:13.062256+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:47.072066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.051480+0000","last_change":"2024-04-25T09:04:13.051480+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.051480+0000","last_undegraded":"2024-04-25T09:04:13.051480+0000","last_fullsized":"2024-04-25T09:04:13.051480+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.055764+0000","last_change":"2024-04-25T09:04:13.053984+0000","last_active":"2024-04-25T09:04:13.055764+0000","last_peered":"2024-04-25T09:04:13.055764+0000","last_clean":"2024-04-25T09:04:13.055764+0000","last_became_active":"2024-04-25T09:04:13.053588+0000","last_became_peered":"2024-04-25T09:04:13.053588+0000","last_unstale":"2024-04-25T09:04:13.055764+0000","last_undegraded":"2024-04-25T09:04:13.055764+0000","last_fullsized":"2024-04-25T09:04:13.055764+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:05.903006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.051515+0000","last_change":"2024-04-25T09:04:13.051515+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.051515+0000","last_undegraded":"2024-04-25T09:04:13.051515+0000","last_fullsized":"2024-04-25T09:04:13.051515+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.057493+0000","last_change":"2024-04-25T09:04:13.055593+0000","last_active":"2024-04-25T09:04:13.057493+0000","last_peered":"2024-04-25T09:04:13.057493+0000","last_clean":"2024-04-25T09:04:13.057493+0000","last_became_active":"2024-04-25T09:04:13.055293+0000","last_became_peered":"2024-04-25T09:04:13.055293+0000","last_unstale":"2024-04-25T09:04:13.057493+0000","last_undegraded":"2024-04-25T09:04:13.057493+0000","last_fullsized":"2024-04-25T09:04:13.057493+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:09.194558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.050011+0000","last_change":"2024-04-25T09:04:13.050011+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.050011+0000","last_undegraded":"2024-04-25T09:04:13.050011+0000","last_fullsized":"2024-04-25T09:04:13.050011+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060206+0000","last_change":"2024-04-25T09:04:13.057835+0000","last_active":"2024-04-25T09:04:13.060206+0000","last_peered":"2024-04-25T09:04:13.060206+0000","last_clean":"2024-04-25T09:04:13.060206+0000","last_became_active":"2024-04-25T09:04:13.057562+0000","last_became_peered":"2024-04-25T09:04:13.057562+0000","last_unstale":"2024-04-25T09:04:13.060206+0000","last_undegraded":"2024-04-25T09:04:13.060206+0000","last_fullsized":"2024-04-25T09:04:13.060206+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:35.248830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.048188+0000","last_change":"2024-04-25T09:04:13.048188+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.048188+0000","last_undegraded":"2024-04-25T09:04:13.048188+0000","last_fullsized":"2024-04-25T09:04:13.048188+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.062094+0000","last_change":"2024-04-25T09:04:13.060717+0000","last_active":"2024-04-25T09:04:13.062094+0000","last_peered":"2024-04-25T09:04:13.062094+0000","last_clean":"2024-04-25T09:04:13.062094+0000","last_became_active":"2024-04-25T09:04:13.060060+0000","last_became_peered":"2024-04-25T09:04:13.060060+0000","last_unstale":"2024-04-25T09:04:13.062094+0000","last_undegraded":"2024-04-25T09:04:13.062094+0000","last_fullsized":"2024-04-25T09:04:13.062094+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:56.805484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046887+0000","last_change":"2024-04-25T09:04:13.046887+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046887+0000","last_undegraded":"2024-04-25T09:04:13.046887+0000","last_fullsized":"2024-04-25T09:04:13.046887+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.056661+0000","last_change":"2024-04-25T09:04:13.054317+0000","last_active":"2024-04-25T09:04:13.056661+0000","last_peered":"2024-04-25T09:04:13.056661+0000","last_clean":"2024-04-25T09:04:13.056661+0000","last_became_active":"2024-04-25T09:04:13.053816+0000","last_became_peered":"2024-04-25T09:04:13.053816+0000","last_unstale":"2024-04-25T09:04:13.056661+0000","last_undegraded":"2024-04-25T09:04:13.056661+0000","last_fullsized":"2024-04-25T09:04:13.056661+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:12.230718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046289+0000","last_change":"2024-04-25T09:04:13.046289+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046289+0000","last_undegraded":"2024-04-25T09:04:13.046289+0000","last_fullsized":"2024-04-25T09:04:13.046289+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.063552+0000","last_change":"2024-04-25T09:04:13.061478+0000","last_active":"2024-04-25T09:04:13.063552+0000","last_peered":"2024-04-25T09:04:13.063552+0000","last_clean":"2024-04-25T09:04:13.063552+0000","last_became_active":"2024-04-25T09:04:13.061150+0000","last_became_peered":"2024-04-25T09:04:13.061150+0000","last_unstale":"2024-04-25T09:04:13.063552+0000","last_undegraded":"2024-04-25T09:04:13.063552+0000","last_fullsized":"2024-04-25T09:04:13.063552+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:52.903859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.045727+0000","last_change":"2024-04-25T09:04:13.045727+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.045727+0000","last_undegraded":"2024-04-25T09:04:13.045727+0000","last_fullsized":"2024-04-25T09:04:13.045727+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060503+0000","last_change":"2024-04-25T09:04:13.059762+0000","last_active":"2024-04-25T09:04:13.060503+0000","last_peered":"2024-04-25T09:04:13.060503+0000","last_clean":"2024-04-25T09:04:13.060503+0000","last_became_active":"2024-04-25T09:04:13.059482+0000","last_became_peered":"2024-04-25T09:04:13.059482+0000","last_unstale":"2024-04-25T09:04:13.060503+0000","last_undegraded":"2024-04-25T09:04:13.060503+0000","last_fullsized":"2024-04-25T09:04:13.060503+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:30:52.194928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.045307+0000","last_change":"2024-04-25T09:04:13.045307+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.045307+0000","last_undegraded":"2024-04-25T09:04:13.045307+0000","last_fullsized":"2024-04-25T09:04:13.045307+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060790+0000","last_change":"2024-04-25T09:04:13.059953+0000","last_active":"2024-04-25T09:04:13.060790+0000","last_peered":"2024-04-25T09:04:13.060790+0000","last_clean":"2024-04-25T09:04:13.060790+0000","last_became_active":"2024-04-25T09:04:13.059633+0000","last_became_peered":"2024-04-25T09:04:13.059633+0000","last_unstale":"2024-04-25T09:04:13.060790+0000","last_undegraded":"2024-04-25T09:04:13.060790+0000","last_fullsized":"2024-04-25T09:04:13.060790+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:04.737828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.045467+0000","last_change":"2024-04-25T09:04:13.045467+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.045467+0000","last_undegraded":"2024-04-25T09:04:13.045467+0000","last_fullsized":"2024-04-25T09:04:13.045467+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.063544+0000","last_change":"2024-04-25T09:04:13.062137+0000","last_active":"2024-04-25T09:04:13.063544+0000","last_peered":"2024-04-25T09:04:13.063544+0000","last_clean":"2024-04-25T09:04:13.063544+0000","last_became_active":"2024-04-25T09:04:13.061505+0000","last_became_peered":"2024-04-25T09:04:13.061505+0000","last_unstale":"2024-04-25T09:04:13.063544+0000","last_undegraded":"2024-04-25T09:04:13.063544+0000","last_fullsized":"2024-04-25T09:04:13.063544+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:27.711642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.050690+0000","last_change":"2024-04-25T09:04:13.050690+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.050690+0000","last_undegraded":"2024-04-25T09:04:13.050690+0000","last_fullsized":"2024-04-25T09:04:13.050690+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060823+0000","last_change":"2024-04-25T09:04:13.057849+0000","last_active":"2024-04-25T09:04:13.060823+0000","last_peered":"2024-04-25T09:04:13.060823+0000","last_clean":"2024-04-25T09:04:13.060823+0000","last_became_active":"2024-04-25T09:04:13.057586+0000","last_became_peered":"2024-04-25T09:04:13.057586+0000","last_unstale":"2024-04-25T09:04:13.060823+0000","last_undegraded":"2024-04-25T09:04:13.060823+0000","last_fullsized":"2024-04-25T09:04:13.060823+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:33.060943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.044936+0000","last_change":"2024-04-25T09:04:13.044936+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.044936+0000","last_undegraded":"2024-04-25T09:04:13.044936+0000","last_fullsized":"2024-04-25T09:04:13.044936+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060734+0000","last_change":"2024-04-25T09:04:13.056588+0000","last_active":"2024-04-25T09:04:13.060734+0000","last_peered":"2024-04-25T09:04:13.060734+0000","last_clean":"2024-04-25T09:04:13.060734+0000","last_became_active":"2024-04-25T09:04:13.055410+0000","last_became_peered":"2024-04-25T09:04:13.055410+0000","last_unstale":"2024-04-25T09:04:13.060734+0000","last_undegraded":"2024-04-25T09:04:13.060734+0000","last_fullsized":"2024-04-25T09:04:13.060734+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:47.545889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.047647+0000","last_change":"2024-04-25T09:04:13.047647+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.047647+0000","last_undegraded":"2024-04-25T09:04:13.047647+0000","last_fullsized":"2024-04-25T09:04:13.047647+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060898+0000","last_change":"2024-04-25T09:04:13.058491+0000","last_active":"2024-04-25T09:04:13.060898+0000","last_peered":"2024-04-25T09:04:13.060898+0000","last_clean":"2024-04-25T09:04:13.060898+0000","last_became_active":"2024-04-25T09:04:13.058215+0000","last_became_peered":"2024-04-25T09:04:13.058215+0000","last_unstale":"2024-04-25T09:04:13.060898+0000","last_undegraded":"2024-04-25T09:04:13.060898+0000","last_fullsized":"2024-04-25T09:04:13.060898+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:04.737828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.047830+0000","last_change":"2024-04-25T09:04:13.047830+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.047830+0000","last_undegraded":"2024-04-25T09:04:13.047830+0000","last_fullsized":"2024-04-25T09:04:13.047830+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.056639+0000","last_change":"2024-04-25T09:04:13.054092+0000","last_active":"2024-04-25T09:04:13.056639+0000","last_peered":"2024-04-25T09:04:13.056639+0000","last_clean":"2024-04-25T09:04:13.056639+0000","last_became_active":"2024-04-25T09:04:13.053427+0000","last_became_peered":"2024-04-25T09:04:13.053427+0000","last_unstale":"2024-04-25T09:04:13.056639+0000","last_undegraded":"2024-04-25T09:04:13.056639+0000","last_fullsized":"2024-04-25T09:04:13.056639+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:47.205245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046670+0000","last_change":"2024-04-25T09:04:13.046670+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046670+0000","last_undegraded":"2024-04-25T09:04:13.046670+0000","last_fullsized":"2024-04-25T09:04:13.046670+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.063064+0000","last_change":"2024-04-25T09:04:13.060816+0000","last_active":"2024-04-25T09:04:13.063064+0000","last_peered":"2024-04-25T09:04:13.063064+0000","last_clean":"2024-04-25T09:04:13.063064+0000","last_became_active":"2024-04-25T09:04:13.060535+0000","last_became_peered":"2024-04-25T09:04:13.060535+0000","last_unstale":"2024-04-25T09:04:13.063064+0000","last_undegraded":"2024-04-25T09:04:13.063064+0000","last_fullsized":"2024-04-25T09:04:13.063064+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:49.426430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.050769+0000","last_change":"2024-04-25T09:04:13.050769+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.050769+0000","last_undegraded":"2024-04-25T09:04:13.050769+0000","last_fullsized":"2024-04-25T09:04:13.050769+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060740+0000","last_change":"2024-04-25T09:04:13.059109+0000","last_active":"2024-04-25T09:04:13.060740+0000","last_peered":"2024-04-25T09:04:13.060740+0000","last_clean":"2024-04-25T09:04:13.060740+0000","last_became_active":"2024-04-25T09:04:13.058224+0000","last_became_peered":"2024-04-25T09:04:13.058224+0000","last_unstale":"2024-04-25T09:04:13.060740+0000","last_undegraded":"2024-04-25T09:04:13.060740+0000","last_fullsized":"2024-04-25T09:04:13.060740+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:33.060943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.045284+0000","last_change":"2024-04-25T09:04:13.045284+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.045284+0000","last_undegraded":"2024-04-25T09:04:13.045284+0000","last_fullsized":"2024-04-25T09:04:13.045284+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060585+0000","last_change":"2024-04-25T09:04:13.057904+0000","last_active":"2024-04-25T09:04:13.060585+0000","last_peered":"2024-04-25T09:04:13.060585+0000","last_clean":"2024-04-25T09:04:13.060585+0000","last_became_active":"2024-04-25T09:04:13.057602+0000","last_became_peered":"2024-04-25T09:04:13.057602+0000","last_unstale":"2024-04-25T09:04:13.060585+0000","last_undegraded":"2024-04-25T09:04:13.060585+0000","last_fullsized":"2024-04-25T09:04:13.060585+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:24.697586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.045651+0000","last_change":"2024-04-25T09:04:13.045651+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.045651+0000","last_undegraded":"2024-04-25T09:04:13.045651+0000","last_fullsized":"2024-04-25T09:04:13.045651+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.054181+0000","last_change":"2024-04-25T09:04:13.053009+0000","last_active":"2024-04-25T09:04:13.054181+0000","last_peered":"2024-04-25T09:04:13.054181+0000","last_clean":"2024-04-25T09:04:13.054181+0000","last_became_active":"2024-04-25T09:04:13.052708+0000","last_became_peered":"2024-04-25T09:04:13.052708+0000","last_unstale":"2024-04-25T09:04:13.054181+0000","last_undegraded":"2024-04-25T09:04:13.054181+0000","last_fullsized":"2024-04-25T09:04:13.054181+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:33.000888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":388,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.054625+0000","last_change":"2024-04-25T08:54:00.818044+0000","last_active":"2024-04-25T09:04:13.054625+0000","last_peered":"2024-04-25T09:04:13.054625+0000","last_clean":"2024-04-25T09:04:13.054625+0000","last_became_active":"2024-04-25T08:54:00.817098+0000","last_became_peered":"2024-04-25T08:54:00.817098+0000","last_unstale":"2024-04-25T09:04:13.054625+0000","last_undegraded":"2024-04-25T09:04:13.054625+0000","last_fullsized":"2024-04-25T09:04:13.054625+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_clean_scrub_stamp":"2024-04-25T08:53:59.739677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:48.129513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574973,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47328,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324512,"statfs":{"total":96636764160,"available":96588300288,"internally_reserved":0,"allocated":2310144,"data_stored":1340135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574974,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68956,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302884,"statfs":{"total":96636764160,"available":96566153216,"internally_reserved":0,"allocated":1716224,"data_stored":749767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574976,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55064,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316776,"statfs":{"total":96636764160,"available":96580378624,"internally_reserved":0,"allocated":1716224,"data_stored":749767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574974,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68960,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302880,"statfs":{"total":96636764160,"available":96566149120,"internally_reserved":0,"allocated":1716224,"data_stored":749767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574974,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77276,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294564,"statfs":{"total":96636764160,"available":96557633536,"internally_reserved":0,"allocated":1716224,"data_stored":749767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574974,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46744,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325096,"statfs":{"total":96636764160,"available":96588898304,"internally_reserved":0,"allocated":1716224,"data_stored":749767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574974,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82840,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94289000,"statfs":{"total":96636764160,"available":96551936000,"internally_reserved":0,"allocated":1716224,"data_stored":749767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574975,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51488,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320352,"statfs":{"total":96636764160,"available":96584040448,"internally_reserved":0,"allocated":2310144,"data_stored":1340135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:04:17.056 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:04:17.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.204+0000 7fbbaff6a640 1 Processor -- start 2024-04-25T09:04:17.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.204+0000 7fbbaff6a640 1 -- start start 2024-04-25T09:04:17.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.205+0000 7fbbaff6a640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fbba8161390 0x7fbba80b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:17.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.205+0000 7fbbaff6a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbba80d02a0 0x7fbba80d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:17.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.205+0000 7fbbaff6a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbba80b4c30 con 0x7fbba80ce5e0 2024-04-25T09:04:17.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.205+0000 7fbbaff6a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbba80d1910 con 0x7fbba80d02a0 2024-04-25T09:04:17.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.205+0000 7fbbaff6a640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbba80cf2c0 con 0x7fbba8161390 2024-04-25T09:04:17.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.205+0000 7fbbad505640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbba80d02a0 0x7fbba80d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:17.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.205+0000 7fbbae507640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fbba80ce5e0 0x7fbba80cea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45720/0 (socket says 172.21.15.43:45720) 2024-04-25T09:04:17.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.205+0000 7fbbae507640 1 -- 172.21.15.43:0/1090913940 learned_addr learned my addr 172.21.15.43:0/1090913940 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:17.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.205+0000 7fbbadd06640 1 --2- 172.21.15.43:0/1090913940 >> v2:172.21.15.193:3300/0 conn(0x7fbba8161390 0x7fbba80b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:17.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.206+0000 7fbbad505640 1 -- 172.21.15.43:0/1090913940 >> v2:172.21.15.193:3300/0 conn(0x7fbba8161390 msgr2=0x7fbba80b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:17.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.206+0000 7fbbad505640 1 --2- 172.21.15.43:0/1090913940 >> v2:172.21.15.193:3300/0 conn(0x7fbba8161390 0x7fbba80b6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.206+0000 7fbbad505640 1 -- 172.21.15.43:0/1090913940 >> v1:172.21.15.43:6789/0 conn(0x7fbba80ce5e0 legacy=0x7fbba80cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:17.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.206+0000 7fbbad505640 1 -- 172.21.15.43:0/1090913940 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba80cf5a0 con 0x7fbba80d02a0 2024-04-25T09:04:17.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.206+0000 7fbbacd04640 1 -- 172.21.15.43:0/1090913940 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4054131908 0 0) 0x7fbba0001490 con 0x7fbba80ce5e0 2024-04-25T09:04:17.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.206+0000 7fbbadd06640 1 --2- 172.21.15.43:0/1090913940 >> v2:172.21.15.193:3300/0 conn(0x7fbba8161390 0x7fbba80b6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:17.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.206+0000 7fbbad505640 1 --2- 172.21.15.43:0/1090913940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbba80d02a0 0x7fbba80d0720 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7fbba80c74d0 tx=0x7fbb9802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d6de564be2864a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:17.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.206+0000 7fbbacd04640 1 -- 172.21.15.43:0/1090913940 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb98037070 con 0x7fbba80d02a0 2024-04-25T09:04:17.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.206+0000 7fbbacd04640 1 -- 172.21.15.43:0/1090913940 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb98032440 con 0x7fbba80d02a0 2024-04-25T09:04:17.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.206+0000 7fbbacd04640 1 -- 172.21.15.43:0/1090913940 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb98032810 con 0x7fbba80d02a0 2024-04-25T09:04:17.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.207+0000 7fbbaff6a640 1 -- 172.21.15.43:0/1090913940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbba80d02a0 msgr2=0x7fbba80d0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:17.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.207+0000 7fbbaff6a640 1 --2- 172.21.15.43:0/1090913940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbba80d02a0 0x7fbba80d0720 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7fbba80c74d0 tx=0x7fbb9802a1a0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.207+0000 7fbbaff6a640 1 -- 172.21.15.43:0/1090913940 shutdown_connections 2024-04-25T09:04:17.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.207+0000 7fbbaff6a640 1 --2- 172.21.15.43:0/1090913940 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbba80d02a0 0x7fbba80d0720 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.207+0000 7fbbaff6a640 1 --2- 172.21.15.43:0/1090913940 >> v2:172.21.15.193:3300/0 conn(0x7fbba8161390 0x7fbba80b6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.207+0000 7fbbaff6a640 1 -- 172.21.15.43:0/1090913940 >> 172.21.15.43:0/1090913940 conn(0x7fbba809cd20 msgr2=0x7fbba80d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:17.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.207+0000 7fbbaff6a640 1 -- 172.21.15.43:0/1090913940 shutdown_connections 2024-04-25T09:04:17.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.207+0000 7fbbaff6a640 1 -- 172.21.15.43:0/1090913940 wait complete. 2024-04-25T09:04:17.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.208+0000 7fbbaff6a640 1 Processor -- start 2024-04-25T09:04:17.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.208+0000 7fbbaff6a640 1 -- start start 2024-04-25T09:04:17.199 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.209+0000 7fbbaff6a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbba80ce5e0 0x7fbba8154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:17.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.209+0000 7fbbaff6a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbba80d02a0 0x7fbba8158040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:17.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.209+0000 7fbbaff6a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbba8161390 0x7fbba80c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:17.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.209+0000 7fbbaff6a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba8161770 con 0x7fbba80d02a0 2024-04-25T09:04:17.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.209+0000 7fbbaff6a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba80d15a0 con 0x7fbba8161390 2024-04-25T09:04:17.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.209+0000 7fbbaff6a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbba80d1da0 con 0x7fbba80ce5e0 2024-04-25T09:04:17.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.209+0000 7fbbad505640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbba80d02a0 0x7fbba8158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:17.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.209+0000 7fbbad505640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbba80d02a0 0x7fbba8158040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47620/0 (socket says 172.21.15.43:47620) 2024-04-25T09:04:17.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.209+0000 7fbbad505640 1 -- 172.21.15.43:0/687850284 learned_addr learned my addr 172.21.15.43:0/687850284 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:17.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.209+0000 7fbbad505640 1 -- 172.21.15.43:0/687850284 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbba80ce5e0 msgr2=0x7fbba8154ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:17.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.209+0000 7fbbad505640 1 --2- 172.21.15.43:0/687850284 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbba80ce5e0 0x7fbba8154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.209+0000 7fbbad505640 1 -- 172.21.15.43:0/687850284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbba8161390 msgr2=0x7fbba80c7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:17.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.209+0000 7fbbae507640 1 --2- 172.21.15.43:0/687850284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbba8161390 0x7fbba80c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:17.200 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.209+0000 7fbbad505640 1 --2- 172.21.15.43:0/687850284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbba8161390 0x7fbba80c7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.209+0000 7fbbad505640 1 -- 172.21.15.43:0/687850284 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba80c8960 con 0x7fbba80d02a0 2024-04-25T09:04:17.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.209+0000 7fbbae507640 1 --2- 172.21.15.43:0/687850284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbba8161390 0x7fbba80c7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:17.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.210+0000 7fbbad505640 1 --2- 172.21.15.43:0/687850284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbba80d02a0 0x7fbba8158040 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7fbb98000c00 tx=0x7fbb980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:17.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.210+0000 7fbb9e7fc640 1 -- 172.21.15.43:0/687850284 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb98037050 con 0x7fbba80d02a0 2024-04-25T09:04:17.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.210+0000 7fbb9e7fc640 1 -- 172.21.15.43:0/687850284 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb98038040 con 0x7fbba80d02a0 2024-04-25T09:04:17.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.210+0000 7fbbaff6a640 1 -- 172.21.15.43:0/687850284 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbba80cad80 con 0x7fbba80d02a0 2024-04-25T09:04:17.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.210+0000 7fbb9e7fc640 1 -- 172.21.15.43:0/687850284 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb98004450 con 0x7fbba80d02a0 2024-04-25T09:04:17.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.210+0000 7fbbaff6a640 1 -- 172.21.15.43:0/687850284 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbba8155430 con 0x7fbba80d02a0 2024-04-25T09:04:17.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.211+0000 7fbbaff6a640 1 -- 172.21.15.43:0/687850284 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbba80680f0 con 0x7fbba80d02a0 2024-04-25T09:04:17.201 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.211+0000 7fbb9e7fc640 1 -- 172.21.15.43:0/687850284 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbb98030980 con 0x7fbba80d02a0 2024-04-25T09:04:17.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.212+0000 7fbb9e7fc640 1 --2- 172.21.15.43:0/687850284 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbb8c06b500 0x7fbb8c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:17.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.212+0000 7fbb9e7fc640 1 -- 172.21.15.43:0/687850284 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb980aae30 con 0x7fbba80d02a0 2024-04-25T09:04:17.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.212+0000 7fbbadd06640 1 --2- 172.21.15.43:0/687850284 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbb8c06b500 0x7fbb8c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:17.202 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.213+0000 7fbbadd06640 1 --2- 172.21.15.43:0/687850284 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbb8c06b500 0x7fbb8c06d9c0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fbba4006fd0 tx=0x7fbba4008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:17.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.216+0000 7fbb9e7fc640 1 -- 172.21.15.43:0/687850284 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbb98079e60 con 0x7fbba80d02a0 2024-04-25T09:04:17.382 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.393+0000 7fbbaff6a640 1 -- 172.21.15.43:0/687850284 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbba8074ef0 con 0x7fbb8c06b500 2024-04-25T09:04:17.388 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.400+0000 7fbb9e7fc640 1 -- 172.21.15.43:0/687850284 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247089 (secure 0 0 0) 0x7fbba8074ef0 con 0x7fbb8c06b500 2024-04-25T09:04:17.391 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:04:17.392 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-25T09:04:17.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.403+0000 7fbbaff6a640 1 -- 172.21.15.43:0/687850284 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbb8c06b500 msgr2=0x7fbb8c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:17.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.403+0000 7fbbaff6a640 1 --2- 172.21.15.43:0/687850284 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbb8c06b500 0x7fbb8c06d9c0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fbba4006fd0 tx=0x7fbba4008040 comp rx=0 tx=0).stop 2024-04-25T09:04:17.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.403+0000 7fbbaff6a640 1 -- 172.21.15.43:0/687850284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbba80d02a0 msgr2=0x7fbba8158040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:17.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.403+0000 7fbbaff6a640 1 --2- 172.21.15.43:0/687850284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbba80d02a0 0x7fbba8158040 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7fbb98000c00 tx=0x7fbb980046c0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.403+0000 7fbbaff6a640 1 -- 172.21.15.43:0/687850284 shutdown_connections 2024-04-25T09:04:17.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.403+0000 7fbbaff6a640 1 --2- 172.21.15.43:0/687850284 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbb8c06b500 0x7fbb8c06d9c0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.403+0000 7fbbaff6a640 1 --2- 172.21.15.43:0/687850284 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbba8161390 0x7fbba80c7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.403+0000 7fbbaff6a640 1 --2- 172.21.15.43:0/687850284 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbba80d02a0 0x7fbba8158040 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.403+0000 7fbbaff6a640 1 --2- 172.21.15.43:0/687850284 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbba80ce5e0 0x7fbba8154ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.403+0000 7fbbaff6a640 1 -- 172.21.15.43:0/687850284 >> 172.21.15.43:0/687850284 conn(0x7fbba809cd20 msgr2=0x7fbba80b6d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:17.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.403+0000 7fbbaff6a640 1 -- 172.21.15.43:0/687850284 shutdown_connections 2024-04-25T09:04:17.394 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.403+0000 7fbbaff6a640 1 -- 172.21.15.43:0/687850284 wait complete. 2024-04-25T09:04:17.402 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":441,"stamp":"2024-04-25T09:04:15.835885+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":498656,"kb_used_data":14568,"kb_used_omap":12,"kb_used_meta":483955,"kb_avail":754476064,"statfs":{"total":773094113280,"available":772583489536,"internally_reserved":0,"allocated":14917632,"data_stored":7178872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":495570481},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000966"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.053615+0000","last_change":"2024-04-25T09:04:13.053615+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.053615+0000","last_undegraded":"2024-04-25T09:04:13.053615+0000","last_fullsized":"2024-04-25T09:04:13.053615+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.047818+0000","last_change":"2024-04-25T09:04:13.047818+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.047818+0000","last_undegraded":"2024-04-25T09:04:13.047818+0000","last_fullsized":"2024-04-25T09:04:13.047818+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.049520+0000","last_change":"2024-04-25T09:04:13.049520+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.049520+0000","last_undegraded":"2024-04-25T09:04:13.049520+0000","last_fullsized":"2024-04-25T09:04:13.049520+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046148+0000","last_change":"2024-04-25T09:04:13.046148+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046148+0000","last_undegraded":"2024-04-25T09:04:13.046148+0000","last_fullsized":"2024-04-25T09:04:13.046148+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.047544+0000","last_change":"2024-04-25T09:04:13.047544+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.047544+0000","last_undegraded":"2024-04-25T09:04:13.047544+0000","last_fullsized":"2024-04-25T09:04:13.047544+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.051007+0000","last_change":"2024-04-25T09:04:13.051007+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.051007+0000","last_undegraded":"2024-04-25T09:04:13.051007+0000","last_fullsized":"2024-04-25T09:04:13.051007+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.048146+0000","last_change":"2024-04-25T09:04:13.048146+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.048146+0000","last_undegraded":"2024-04-25T09:04:13.048146+0000","last_fullsized":"2024-04-25T09:04:13.048146+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.049952+0000","last_change":"2024-04-25T09:04:13.049952+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.049952+0000","last_undegraded":"2024-04-25T09:04:13.049952+0000","last_fullsized":"2024-04-25T09:04:13.049952+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.047013+0000","last_change":"2024-04-25T09:04:13.047013+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.047013+0000","last_undegraded":"2024-04-25T09:04:13.047013+0000","last_fullsized":"2024-04-25T09:04:13.047013+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.052404+0000","last_change":"2024-04-25T09:04:13.052404+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.052404+0000","last_undegraded":"2024-04-25T09:04:13.052404+0000","last_fullsized":"2024-04-25T09:04:13.052404+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.050451+0000","last_change":"2024-04-25T09:04:13.050451+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.050451+0000","last_undegraded":"2024-04-25T09:04:13.050451+0000","last_fullsized":"2024-04-25T09:04:13.050451+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.054302+0000","last_change":"2024-04-25T09:04:13.054302+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.054302+0000","last_undegraded":"2024-04-25T09:04:13.054302+0000","last_fullsized":"2024-04-25T09:04:13.054302+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.047149+0000","last_change":"2024-04-25T09:04:13.047149+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.047149+0000","last_undegraded":"2024-04-25T09:04:13.047149+0000","last_fullsized":"2024-04-25T09:04:13.047149+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.048543+0000","last_change":"2024-04-25T09:04:13.048543+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.048543+0000","last_undegraded":"2024-04-25T09:04:13.048543+0000","last_fullsized":"2024-04-25T09:04:13.048543+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.050737+0000","last_change":"2024-04-25T09:04:13.050737+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.050737+0000","last_undegraded":"2024-04-25T09:04:13.050737+0000","last_fullsized":"2024-04-25T09:04:13.050737+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.049349+0000","last_change":"2024-04-25T09:04:13.049349+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.049349+0000","last_undegraded":"2024-04-25T09:04:13.049349+0000","last_fullsized":"2024-04-25T09:04:13.049349+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.051581+0000","last_change":"2024-04-25T09:04:13.051581+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.051581+0000","last_undegraded":"2024-04-25T09:04:13.051581+0000","last_fullsized":"2024-04-25T09:04:13.051581+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.052944+0000","last_change":"2024-04-25T09:04:13.052944+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.052944+0000","last_undegraded":"2024-04-25T09:04:13.052944+0000","last_fullsized":"2024-04-25T09:04:13.052944+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.049189+0000","last_change":"2024-04-25T09:04:13.049189+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.049189+0000","last_undegraded":"2024-04-25T09:04:13.049189+0000","last_fullsized":"2024-04-25T09:04:13.049189+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046642+0000","last_change":"2024-04-25T09:04:13.046642+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046642+0000","last_undegraded":"2024-04-25T09:04:13.046642+0000","last_fullsized":"2024-04-25T09:04:13.046642+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.051136+0000","last_change":"2024-04-25T09:04:13.051136+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.051136+0000","last_undegraded":"2024-04-25T09:04:13.051136+0000","last_fullsized":"2024-04-25T09:04:13.051136+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.048012+0000","last_change":"2024-04-25T09:04:13.048012+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.048012+0000","last_undegraded":"2024-04-25T09:04:13.048012+0000","last_fullsized":"2024-04-25T09:04:13.048012+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046277+0000","last_change":"2024-04-25T09:04:13.046277+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046277+0000","last_undegraded":"2024-04-25T09:04:13.046277+0000","last_fullsized":"2024-04-25T09:04:13.046277+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.051677+0000","last_change":"2024-04-25T09:04:13.051677+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.051677+0000","last_undegraded":"2024-04-25T09:04:13.051677+0000","last_fullsized":"2024-04-25T09:04:13.051677+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.052436+0000","last_change":"2024-04-25T09:04:13.052436+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.052436+0000","last_undegraded":"2024-04-25T09:04:13.052436+0000","last_fullsized":"2024-04-25T09:04:13.052436+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.049685+0000","last_change":"2024-04-25T09:04:13.049685+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.049685+0000","last_undegraded":"2024-04-25T09:04:13.049685+0000","last_fullsized":"2024-04-25T09:04:13.049685+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.048115+0000","last_change":"2024-04-25T09:04:13.048115+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.048115+0000","last_undegraded":"2024-04-25T09:04:13.048115+0000","last_fullsized":"2024-04-25T09:04:13.048115+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046108+0000","last_change":"2024-04-25T09:04:13.046108+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046108+0000","last_undegraded":"2024-04-25T09:04:13.046108+0000","last_fullsized":"2024-04-25T09:04:13.046108+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.048479+0000","last_change":"2024-04-25T09:04:13.048479+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.048479+0000","last_undegraded":"2024-04-25T09:04:13.048479+0000","last_fullsized":"2024-04-25T09:04:13.048479+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.047436+0000","last_change":"2024-04-25T09:04:13.047436+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.047436+0000","last_undegraded":"2024-04-25T09:04:13.047436+0000","last_fullsized":"2024-04-25T09:04:13.047436+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046485+0000","last_change":"2024-04-25T09:04:13.046485+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046485+0000","last_undegraded":"2024-04-25T09:04:13.046485+0000","last_fullsized":"2024-04-25T09:04:13.046485+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.048912+0000","last_change":"2024-04-25T09:04:13.048912+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.048912+0000","last_undegraded":"2024-04-25T09:04:13.048912+0000","last_fullsized":"2024-04-25T09:04:13.048912+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.051989+0000","last_change":"2024-04-25T09:04:13.051989+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.051989+0000","last_undegraded":"2024-04-25T09:04:13.051989+0000","last_fullsized":"2024-04-25T09:04:13.051989+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.063563+0000","last_change":"2024-04-25T09:04:13.062156+0000","last_active":"2024-04-25T09:04:13.063563+0000","last_peered":"2024-04-25T09:04:13.063563+0000","last_clean":"2024-04-25T09:04:13.063563+0000","last_became_active":"2024-04-25T09:04:13.061852+0000","last_became_peered":"2024-04-25T09:04:13.061852+0000","last_unstale":"2024-04-25T09:04:13.063563+0000","last_undegraded":"2024-04-25T09:04:13.063563+0000","last_fullsized":"2024-04-25T09:04:13.063563+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:10.458178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046457+0000","last_change":"2024-04-25T09:04:13.046457+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046457+0000","last_undegraded":"2024-04-25T09:04:13.046457+0000","last_fullsized":"2024-04-25T09:04:13.046457+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.059517+0000","last_change":"2024-04-25T09:04:13.058762+0000","last_active":"2024-04-25T09:04:13.059517+0000","last_peered":"2024-04-25T09:04:13.059517+0000","last_clean":"2024-04-25T09:04:13.059517+0000","last_became_active":"2024-04-25T09:04:13.058335+0000","last_became_peered":"2024-04-25T09:04:13.058335+0000","last_unstale":"2024-04-25T09:04:13.059517+0000","last_undegraded":"2024-04-25T09:04:13.059517+0000","last_fullsized":"2024-04-25T09:04:13.059517+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:59.348802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.048044+0000","last_change":"2024-04-25T09:04:13.048044+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.048044+0000","last_undegraded":"2024-04-25T09:04:13.048044+0000","last_fullsized":"2024-04-25T09:04:13.048044+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.059899+0000","last_change":"2024-04-25T09:04:13.057742+0000","last_active":"2024-04-25T09:04:13.059899+0000","last_peered":"2024-04-25T09:04:13.059899+0000","last_clean":"2024-04-25T09:04:13.059899+0000","last_became_active":"2024-04-25T09:04:13.057448+0000","last_became_peered":"2024-04-25T09:04:13.057448+0000","last_unstale":"2024-04-25T09:04:13.059899+0000","last_undegraded":"2024-04-25T09:04:13.059899+0000","last_fullsized":"2024-04-25T09:04:13.059899+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:47.545889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.053963+0000","last_change":"2024-04-25T09:04:13.053963+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.053963+0000","last_undegraded":"2024-04-25T09:04:13.053963+0000","last_fullsized":"2024-04-25T09:04:13.053963+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.059924+0000","last_change":"2024-04-25T09:04:13.058406+0000","last_active":"2024-04-25T09:04:13.059924+0000","last_peered":"2024-04-25T09:04:13.059924+0000","last_clean":"2024-04-25T09:04:13.059924+0000","last_became_active":"2024-04-25T09:04:13.058144+0000","last_became_peered":"2024-04-25T09:04:13.058144+0000","last_unstale":"2024-04-25T09:04:13.059924+0000","last_undegraded":"2024-04-25T09:04:13.059924+0000","last_fullsized":"2024-04-25T09:04:13.059924+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:33.060943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046085+0000","last_change":"2024-04-25T09:04:13.046085+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046085+0000","last_undegraded":"2024-04-25T09:04:13.046085+0000","last_fullsized":"2024-04-25T09:04:13.046085+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060240+0000","last_change":"2024-04-25T09:04:13.059157+0000","last_active":"2024-04-25T09:04:13.060240+0000","last_peered":"2024-04-25T09:04:13.060240+0000","last_clean":"2024-04-25T09:04:13.060240+0000","last_became_active":"2024-04-25T09:04:13.058883+0000","last_became_peered":"2024-04-25T09:04:13.058883+0000","last_unstale":"2024-04-25T09:04:13.060240+0000","last_undegraded":"2024-04-25T09:04:13.060240+0000","last_fullsized":"2024-04-25T09:04:13.060240+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:38.159347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046155+0000","last_change":"2024-04-25T09:04:13.046155+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046155+0000","last_undegraded":"2024-04-25T09:04:13.046155+0000","last_fullsized":"2024-04-25T09:04:13.046155+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.059938+0000","last_change":"2024-04-25T09:04:13.057593+0000","last_active":"2024-04-25T09:04:13.059938+0000","last_peered":"2024-04-25T09:04:13.059938+0000","last_clean":"2024-04-25T09:04:13.059938+0000","last_became_active":"2024-04-25T09:04:13.057330+0000","last_became_peered":"2024-04-25T09:04:13.057330+0000","last_unstale":"2024-04-25T09:04:13.059938+0000","last_undegraded":"2024-04-25T09:04:13.059938+0000","last_fullsized":"2024-04-25T09:04:13.059938+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:32.141616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046021+0000","last_change":"2024-04-25T09:04:13.046021+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046021+0000","last_undegraded":"2024-04-25T09:04:13.046021+0000","last_fullsized":"2024-04-25T09:04:13.046021+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.056354+0000","last_change":"2024-04-25T09:04:13.054808+0000","last_active":"2024-04-25T09:04:13.056354+0000","last_peered":"2024-04-25T09:04:13.056354+0000","last_clean":"2024-04-25T09:04:13.056354+0000","last_became_active":"2024-04-25T09:04:13.054486+0000","last_became_peered":"2024-04-25T09:04:13.054486+0000","last_unstale":"2024-04-25T09:04:13.056354+0000","last_undegraded":"2024-04-25T09:04:13.056354+0000","last_fullsized":"2024-04-25T09:04:13.056354+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:24.933992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.045906+0000","last_change":"2024-04-25T09:04:13.045906+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.045906+0000","last_undegraded":"2024-04-25T09:04:13.045906+0000","last_fullsized":"2024-04-25T09:04:13.045906+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.057196+0000","last_change":"2024-04-25T09:04:13.056215+0000","last_active":"2024-04-25T09:04:13.057196+0000","last_peered":"2024-04-25T09:04:13.057196+0000","last_clean":"2024-04-25T09:04:13.057196+0000","last_became_active":"2024-04-25T09:04:13.055952+0000","last_became_peered":"2024-04-25T09:04:13.055952+0000","last_unstale":"2024-04-25T09:04:13.057196+0000","last_undegraded":"2024-04-25T09:04:13.057196+0000","last_fullsized":"2024-04-25T09:04:13.057196+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:39.762504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.045952+0000","last_change":"2024-04-25T09:04:13.045952+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.045952+0000","last_undegraded":"2024-04-25T09:04:13.045952+0000","last_fullsized":"2024-04-25T09:04:13.045952+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.057234+0000","last_change":"2024-04-25T09:04:13.051625+0000","last_active":"2024-04-25T09:04:13.057234+0000","last_peered":"2024-04-25T09:04:13.057234+0000","last_clean":"2024-04-25T09:04:13.057234+0000","last_became_active":"2024-04-25T09:04:13.051307+0000","last_became_peered":"2024-04-25T09:04:13.051307+0000","last_unstale":"2024-04-25T09:04:13.057234+0000","last_undegraded":"2024-04-25T09:04:13.057234+0000","last_fullsized":"2024-04-25T09:04:13.057234+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:06.808616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.052044+0000","last_change":"2024-04-25T09:04:13.052044+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.052044+0000","last_undegraded":"2024-04-25T09:04:13.052044+0000","last_fullsized":"2024-04-25T09:04:13.052044+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060131+0000","last_change":"2024-04-25T09:04:13.058585+0000","last_active":"2024-04-25T09:04:13.060131+0000","last_peered":"2024-04-25T09:04:13.060131+0000","last_clean":"2024-04-25T09:04:13.060131+0000","last_became_active":"2024-04-25T09:04:13.058298+0000","last_became_peered":"2024-04-25T09:04:13.058298+0000","last_unstale":"2024-04-25T09:04:13.060131+0000","last_undegraded":"2024-04-25T09:04:13.060131+0000","last_fullsized":"2024-04-25T09:04:13.060131+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:25.170530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.047591+0000","last_change":"2024-04-25T09:04:13.047591+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.047591+0000","last_undegraded":"2024-04-25T09:04:13.047591+0000","last_fullsized":"2024-04-25T09:04:13.047591+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.061150+0000","last_change":"2024-04-25T09:04:13.058981+0000","last_active":"2024-04-25T09:04:13.061150+0000","last_peered":"2024-04-25T09:04:13.061150+0000","last_clean":"2024-04-25T09:04:13.061150+0000","last_became_active":"2024-04-25T09:04:13.058662+0000","last_became_peered":"2024-04-25T09:04:13.058662+0000","last_unstale":"2024-04-25T09:04:13.061150+0000","last_undegraded":"2024-04-25T09:04:13.061150+0000","last_fullsized":"2024-04-25T09:04:13.061150+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:35.416425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.049288+0000","last_change":"2024-04-25T09:04:13.049288+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.049288+0000","last_undegraded":"2024-04-25T09:04:13.049288+0000","last_fullsized":"2024-04-25T09:04:13.049288+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060649+0000","last_change":"2024-04-25T09:04:13.058867+0000","last_active":"2024-04-25T09:04:13.060649+0000","last_peered":"2024-04-25T09:04:13.060649+0000","last_clean":"2024-04-25T09:04:13.060649+0000","last_became_active":"2024-04-25T09:04:13.057622+0000","last_became_peered":"2024-04-25T09:04:13.057622+0000","last_unstale":"2024-04-25T09:04:13.060649+0000","last_undegraded":"2024-04-25T09:04:13.060649+0000","last_fullsized":"2024-04-25T09:04:13.060649+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:47.545889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.045500+0000","last_change":"2024-04-25T09:04:13.045500+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.045500+0000","last_undegraded":"2024-04-25T09:04:13.045500+0000","last_fullsized":"2024-04-25T09:04:13.045500+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.055227+0000","last_change":"2024-04-25T09:04:13.054034+0000","last_active":"2024-04-25T09:04:13.055227+0000","last_peered":"2024-04-25T09:04:13.055227+0000","last_clean":"2024-04-25T09:04:13.055227+0000","last_became_active":"2024-04-25T09:04:13.053760+0000","last_became_peered":"2024-04-25T09:04:13.053760+0000","last_unstale":"2024-04-25T09:04:13.055227+0000","last_undegraded":"2024-04-25T09:04:13.055227+0000","last_fullsized":"2024-04-25T09:04:13.055227+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:39.762504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.045249+0000","last_change":"2024-04-25T09:04:13.045249+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.045249+0000","last_undegraded":"2024-04-25T09:04:13.045249+0000","last_fullsized":"2024-04-25T09:04:13.045249+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060397+0000","last_change":"2024-04-25T09:04:13.056393+0000","last_active":"2024-04-25T09:04:13.060397+0000","last_peered":"2024-04-25T09:04:13.060397+0000","last_clean":"2024-04-25T09:04:13.060397+0000","last_became_active":"2024-04-25T09:04:13.056111+0000","last_became_peered":"2024-04-25T09:04:13.056111+0000","last_unstale":"2024-04-25T09:04:13.060397+0000","last_undegraded":"2024-04-25T09:04:13.060397+0000","last_fullsized":"2024-04-25T09:04:13.060397+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:39.762504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.049874+0000","last_change":"2024-04-25T09:04:13.049874+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.049874+0000","last_undegraded":"2024-04-25T09:04:13.049874+0000","last_fullsized":"2024-04-25T09:04:13.049874+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.062256+0000","last_change":"2024-04-25T09:04:13.061556+0000","last_active":"2024-04-25T09:04:13.062256+0000","last_peered":"2024-04-25T09:04:13.062256+0000","last_clean":"2024-04-25T09:04:13.062256+0000","last_became_active":"2024-04-25T09:04:13.061241+0000","last_became_peered":"2024-04-25T09:04:13.061241+0000","last_unstale":"2024-04-25T09:04:13.062256+0000","last_undegraded":"2024-04-25T09:04:13.062256+0000","last_fullsized":"2024-04-25T09:04:13.062256+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:47.072066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.051480+0000","last_change":"2024-04-25T09:04:13.051480+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.051480+0000","last_undegraded":"2024-04-25T09:04:13.051480+0000","last_fullsized":"2024-04-25T09:04:13.051480+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.055764+0000","last_change":"2024-04-25T09:04:13.053984+0000","last_active":"2024-04-25T09:04:13.055764+0000","last_peered":"2024-04-25T09:04:13.055764+0000","last_clean":"2024-04-25T09:04:13.055764+0000","last_became_active":"2024-04-25T09:04:13.053588+0000","last_became_peered":"2024-04-25T09:04:13.053588+0000","last_unstale":"2024-04-25T09:04:13.055764+0000","last_undegraded":"2024-04-25T09:04:13.055764+0000","last_fullsized":"2024-04-25T09:04:13.055764+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:05.903006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.051515+0000","last_change":"2024-04-25T09:04:13.051515+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.051515+0000","last_undegraded":"2024-04-25T09:04:13.051515+0000","last_fullsized":"2024-04-25T09:04:13.051515+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.057493+0000","last_change":"2024-04-25T09:04:13.055593+0000","last_active":"2024-04-25T09:04:13.057493+0000","last_peered":"2024-04-25T09:04:13.057493+0000","last_clean":"2024-04-25T09:04:13.057493+0000","last_became_active":"2024-04-25T09:04:13.055293+0000","last_became_peered":"2024-04-25T09:04:13.055293+0000","last_unstale":"2024-04-25T09:04:13.057493+0000","last_undegraded":"2024-04-25T09:04:13.057493+0000","last_fullsized":"2024-04-25T09:04:13.057493+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:09.194558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.050011+0000","last_change":"2024-04-25T09:04:13.050011+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.050011+0000","last_undegraded":"2024-04-25T09:04:13.050011+0000","last_fullsized":"2024-04-25T09:04:13.050011+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060206+0000","last_change":"2024-04-25T09:04:13.057835+0000","last_active":"2024-04-25T09:04:13.060206+0000","last_peered":"2024-04-25T09:04:13.060206+0000","last_clean":"2024-04-25T09:04:13.060206+0000","last_became_active":"2024-04-25T09:04:13.057562+0000","last_became_peered":"2024-04-25T09:04:13.057562+0000","last_unstale":"2024-04-25T09:04:13.060206+0000","last_undegraded":"2024-04-25T09:04:13.060206+0000","last_fullsized":"2024-04-25T09:04:13.060206+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:35.248830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.048188+0000","last_change":"2024-04-25T09:04:13.048188+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.048188+0000","last_undegraded":"2024-04-25T09:04:13.048188+0000","last_fullsized":"2024-04-25T09:04:13.048188+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.062094+0000","last_change":"2024-04-25T09:04:13.060717+0000","last_active":"2024-04-25T09:04:13.062094+0000","last_peered":"2024-04-25T09:04:13.062094+0000","last_clean":"2024-04-25T09:04:13.062094+0000","last_became_active":"2024-04-25T09:04:13.060060+0000","last_became_peered":"2024-04-25T09:04:13.060060+0000","last_unstale":"2024-04-25T09:04:13.062094+0000","last_undegraded":"2024-04-25T09:04:13.062094+0000","last_fullsized":"2024-04-25T09:04:13.062094+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:56.805484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046887+0000","last_change":"2024-04-25T09:04:13.046887+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046887+0000","last_undegraded":"2024-04-25T09:04:13.046887+0000","last_fullsized":"2024-04-25T09:04:13.046887+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.056661+0000","last_change":"2024-04-25T09:04:13.054317+0000","last_active":"2024-04-25T09:04:13.056661+0000","last_peered":"2024-04-25T09:04:13.056661+0000","last_clean":"2024-04-25T09:04:13.056661+0000","last_became_active":"2024-04-25T09:04:13.053816+0000","last_became_peered":"2024-04-25T09:04:13.053816+0000","last_unstale":"2024-04-25T09:04:13.056661+0000","last_undegraded":"2024-04-25T09:04:13.056661+0000","last_fullsized":"2024-04-25T09:04:13.056661+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:12.230718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046289+0000","last_change":"2024-04-25T09:04:13.046289+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046289+0000","last_undegraded":"2024-04-25T09:04:13.046289+0000","last_fullsized":"2024-04-25T09:04:13.046289+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.063552+0000","last_change":"2024-04-25T09:04:13.061478+0000","last_active":"2024-04-25T09:04:13.063552+0000","last_peered":"2024-04-25T09:04:13.063552+0000","last_clean":"2024-04-25T09:04:13.063552+0000","last_became_active":"2024-04-25T09:04:13.061150+0000","last_became_peered":"2024-04-25T09:04:13.061150+0000","last_unstale":"2024-04-25T09:04:13.063552+0000","last_undegraded":"2024-04-25T09:04:13.063552+0000","last_fullsized":"2024-04-25T09:04:13.063552+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:52.903859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.045727+0000","last_change":"2024-04-25T09:04:13.045727+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.045727+0000","last_undegraded":"2024-04-25T09:04:13.045727+0000","last_fullsized":"2024-04-25T09:04:13.045727+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060503+0000","last_change":"2024-04-25T09:04:13.059762+0000","last_active":"2024-04-25T09:04:13.060503+0000","last_peered":"2024-04-25T09:04:13.060503+0000","last_clean":"2024-04-25T09:04:13.060503+0000","last_became_active":"2024-04-25T09:04:13.059482+0000","last_became_peered":"2024-04-25T09:04:13.059482+0000","last_unstale":"2024-04-25T09:04:13.060503+0000","last_undegraded":"2024-04-25T09:04:13.060503+0000","last_fullsized":"2024-04-25T09:04:13.060503+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:30:52.194928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.045307+0000","last_change":"2024-04-25T09:04:13.045307+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.045307+0000","last_undegraded":"2024-04-25T09:04:13.045307+0000","last_fullsized":"2024-04-25T09:04:13.045307+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060790+0000","last_change":"2024-04-25T09:04:13.059953+0000","last_active":"2024-04-25T09:04:13.060790+0000","last_peered":"2024-04-25T09:04:13.060790+0000","last_clean":"2024-04-25T09:04:13.060790+0000","last_became_active":"2024-04-25T09:04:13.059633+0000","last_became_peered":"2024-04-25T09:04:13.059633+0000","last_unstale":"2024-04-25T09:04:13.060790+0000","last_undegraded":"2024-04-25T09:04:13.060790+0000","last_fullsized":"2024-04-25T09:04:13.060790+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:04.737828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.045467+0000","last_change":"2024-04-25T09:04:13.045467+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.045467+0000","last_undegraded":"2024-04-25T09:04:13.045467+0000","last_fullsized":"2024-04-25T09:04:13.045467+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.063544+0000","last_change":"2024-04-25T09:04:13.062137+0000","last_active":"2024-04-25T09:04:13.063544+0000","last_peered":"2024-04-25T09:04:13.063544+0000","last_clean":"2024-04-25T09:04:13.063544+0000","last_became_active":"2024-04-25T09:04:13.061505+0000","last_became_peered":"2024-04-25T09:04:13.061505+0000","last_unstale":"2024-04-25T09:04:13.063544+0000","last_undegraded":"2024-04-25T09:04:13.063544+0000","last_fullsized":"2024-04-25T09:04:13.063544+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:27.711642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.050690+0000","last_change":"2024-04-25T09:04:13.050690+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.050690+0000","last_undegraded":"2024-04-25T09:04:13.050690+0000","last_fullsized":"2024-04-25T09:04:13.050690+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060823+0000","last_change":"2024-04-25T09:04:13.057849+0000","last_active":"2024-04-25T09:04:13.060823+0000","last_peered":"2024-04-25T09:04:13.060823+0000","last_clean":"2024-04-25T09:04:13.060823+0000","last_became_active":"2024-04-25T09:04:13.057586+0000","last_became_peered":"2024-04-25T09:04:13.057586+0000","last_unstale":"2024-04-25T09:04:13.060823+0000","last_undegraded":"2024-04-25T09:04:13.060823+0000","last_fullsized":"2024-04-25T09:04:13.060823+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:33.060943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.044936+0000","last_change":"2024-04-25T09:04:13.044936+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.044936+0000","last_undegraded":"2024-04-25T09:04:13.044936+0000","last_fullsized":"2024-04-25T09:04:13.044936+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060734+0000","last_change":"2024-04-25T09:04:13.056588+0000","last_active":"2024-04-25T09:04:13.060734+0000","last_peered":"2024-04-25T09:04:13.060734+0000","last_clean":"2024-04-25T09:04:13.060734+0000","last_became_active":"2024-04-25T09:04:13.055410+0000","last_became_peered":"2024-04-25T09:04:13.055410+0000","last_unstale":"2024-04-25T09:04:13.060734+0000","last_undegraded":"2024-04-25T09:04:13.060734+0000","last_fullsized":"2024-04-25T09:04:13.060734+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:47.545889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.047647+0000","last_change":"2024-04-25T09:04:13.047647+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.047647+0000","last_undegraded":"2024-04-25T09:04:13.047647+0000","last_fullsized":"2024-04-25T09:04:13.047647+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060898+0000","last_change":"2024-04-25T09:04:13.058491+0000","last_active":"2024-04-25T09:04:13.060898+0000","last_peered":"2024-04-25T09:04:13.060898+0000","last_clean":"2024-04-25T09:04:13.060898+0000","last_became_active":"2024-04-25T09:04:13.058215+0000","last_became_peered":"2024-04-25T09:04:13.058215+0000","last_unstale":"2024-04-25T09:04:13.060898+0000","last_undegraded":"2024-04-25T09:04:13.060898+0000","last_fullsized":"2024-04-25T09:04:13.060898+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:04.737828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.047830+0000","last_change":"2024-04-25T09:04:13.047830+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.047830+0000","last_undegraded":"2024-04-25T09:04:13.047830+0000","last_fullsized":"2024-04-25T09:04:13.047830+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.056639+0000","last_change":"2024-04-25T09:04:13.054092+0000","last_active":"2024-04-25T09:04:13.056639+0000","last_peered":"2024-04-25T09:04:13.056639+0000","last_clean":"2024-04-25T09:04:13.056639+0000","last_became_active":"2024-04-25T09:04:13.053427+0000","last_became_peered":"2024-04-25T09:04:13.053427+0000","last_unstale":"2024-04-25T09:04:13.056639+0000","last_undegraded":"2024-04-25T09:04:13.056639+0000","last_fullsized":"2024-04-25T09:04:13.056639+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:47.205245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046670+0000","last_change":"2024-04-25T09:04:13.046670+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046670+0000","last_undegraded":"2024-04-25T09:04:13.046670+0000","last_fullsized":"2024-04-25T09:04:13.046670+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.063064+0000","last_change":"2024-04-25T09:04:13.060816+0000","last_active":"2024-04-25T09:04:13.063064+0000","last_peered":"2024-04-25T09:04:13.063064+0000","last_clean":"2024-04-25T09:04:13.063064+0000","last_became_active":"2024-04-25T09:04:13.060535+0000","last_became_peered":"2024-04-25T09:04:13.060535+0000","last_unstale":"2024-04-25T09:04:13.063064+0000","last_undegraded":"2024-04-25T09:04:13.063064+0000","last_fullsized":"2024-04-25T09:04:13.063064+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:49.426430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.050769+0000","last_change":"2024-04-25T09:04:13.050769+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.050769+0000","last_undegraded":"2024-04-25T09:04:13.050769+0000","last_fullsized":"2024-04-25T09:04:13.050769+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060740+0000","last_change":"2024-04-25T09:04:13.059109+0000","last_active":"2024-04-25T09:04:13.060740+0000","last_peered":"2024-04-25T09:04:13.060740+0000","last_clean":"2024-04-25T09:04:13.060740+0000","last_became_active":"2024-04-25T09:04:13.058224+0000","last_became_peered":"2024-04-25T09:04:13.058224+0000","last_unstale":"2024-04-25T09:04:13.060740+0000","last_undegraded":"2024-04-25T09:04:13.060740+0000","last_fullsized":"2024-04-25T09:04:13.060740+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:33.060943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.045284+0000","last_change":"2024-04-25T09:04:13.045284+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.045284+0000","last_undegraded":"2024-04-25T09:04:13.045284+0000","last_fullsized":"2024-04-25T09:04:13.045284+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060585+0000","last_change":"2024-04-25T09:04:13.057904+0000","last_active":"2024-04-25T09:04:13.060585+0000","last_peered":"2024-04-25T09:04:13.060585+0000","last_clean":"2024-04-25T09:04:13.060585+0000","last_became_active":"2024-04-25T09:04:13.057602+0000","last_became_peered":"2024-04-25T09:04:13.057602+0000","last_unstale":"2024-04-25T09:04:13.060585+0000","last_undegraded":"2024-04-25T09:04:13.060585+0000","last_fullsized":"2024-04-25T09:04:13.060585+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:24.697586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.045651+0000","last_change":"2024-04-25T09:04:13.045651+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.045651+0000","last_undegraded":"2024-04-25T09:04:13.045651+0000","last_fullsized":"2024-04-25T09:04:13.045651+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.054181+0000","last_change":"2024-04-25T09:04:13.053009+0000","last_active":"2024-04-25T09:04:13.054181+0000","last_peered":"2024-04-25T09:04:13.054181+0000","last_clean":"2024-04-25T09:04:13.054181+0000","last_became_active":"2024-04-25T09:04:13.052708+0000","last_became_peered":"2024-04-25T09:04:13.052708+0000","last_unstale":"2024-04-25T09:04:13.054181+0000","last_undegraded":"2024-04-25T09:04:13.054181+0000","last_fullsized":"2024-04-25T09:04:13.054181+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:33.000888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":388,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.054625+0000","last_change":"2024-04-25T08:54:00.818044+0000","last_active":"2024-04-25T09:04:13.054625+0000","last_peered":"2024-04-25T09:04:13.054625+0000","last_clean":"2024-04-25T09:04:13.054625+0000","last_became_active":"2024-04-25T08:54:00.817098+0000","last_became_peered":"2024-04-25T08:54:00.817098+0000","last_unstale":"2024-04-25T09:04:13.054625+0000","last_undegraded":"2024-04-25T09:04:13.054625+0000","last_fullsized":"2024-04-25T09:04:13.054625+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_clean_scrub_stamp":"2024-04-25T08:53:59.739677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:48.129513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574973,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47328,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324512,"statfs":{"total":96636764160,"available":96588300288,"internally_reserved":0,"allocated":2310144,"data_stored":1340135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574974,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68956,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302884,"statfs":{"total":96636764160,"available":96566153216,"internally_reserved":0,"allocated":1716224,"data_stored":749767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574976,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55064,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316776,"statfs":{"total":96636764160,"available":96580378624,"internally_reserved":0,"allocated":1716224,"data_stored":749767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574974,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68960,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302880,"statfs":{"total":96636764160,"available":96566149120,"internally_reserved":0,"allocated":1716224,"data_stored":749767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574974,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77276,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294564,"statfs":{"total":96636764160,"available":96557633536,"internally_reserved":0,"allocated":1716224,"data_stored":749767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574974,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46744,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325096,"statfs":{"total":96636764160,"available":96588898304,"internally_reserved":0,"allocated":1716224,"data_stored":749767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574974,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82840,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94289000,"statfs":{"total":96636764160,"available":96551936000,"internally_reserved":0,"allocated":1716224,"data_stored":749767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574975,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51488,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320352,"statfs":{"total":96636764160,"available":96584040448,"internally_reserved":0,"allocated":2310144,"data_stored":1340135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:04:17.406 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:04:17.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.555+0000 7f8e2421f640 1 Processor -- start 2024-04-25T09:04:17.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.555+0000 7f8e2421f640 1 -- start start 2024-04-25T09:04:17.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.556+0000 7f8e2421f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8e1c0d09b0 0x7f8e1c0b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:17.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.556+0000 7f8e2421f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e1c0b4c30 con 0x7f8e1c0ce5e0 2024-04-25T09:04:17.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.556+0000 7f8e2421f640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e1c0cf2c0 con 0x7f8e1c161390 2024-04-25T09:04:17.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.556+0000 7f8e2421f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e1c0d1910 con 0x7f8e1c0d09b0 2024-04-25T09:04:17.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.556+0000 7f8e217ba640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8e1c0d09b0 0x7f8e1c0b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:17.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.556+0000 7f8e227bc640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8e1c0ce5e0 0x7f8e1c0cea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45734/0 (socket says 172.21.15.43:45734) 2024-04-25T09:04:17.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.556+0000 7f8e227bc640 1 -- 172.21.15.43:0/2987395853 learned_addr learned my addr 172.21.15.43:0/2987395853 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:17.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.556+0000 7f8e217ba640 1 -- 172.21.15.43:0/2987395853 >> v1:172.21.15.193:6789/0 conn(0x7f8e1c161390 legacy=0x7f8e1c0d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:17.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.556+0000 7f8e217ba640 1 -- 172.21.15.43:0/2987395853 >> v1:172.21.15.43:6789/0 conn(0x7f8e1c0ce5e0 legacy=0x7f8e1c0cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:17.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.556+0000 7f8e217ba640 1 -- 172.21.15.43:0/2987395853 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e1c0cf5a0 con 0x7f8e1c0d09b0 2024-04-25T09:04:17.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.557+0000 7f8e217ba640 1 --2- 172.21.15.43:0/2987395853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e1c0d09b0 0x7f8e1c0b6820 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c0097a0 tx=0x7f8e0c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa7c6d9f941421ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:17.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.557+0000 7f8e20fb9640 1 -- 172.21.15.43:0/2987395853 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c037070 con 0x7f8e1c0d09b0 2024-04-25T09:04:17.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.557+0000 7f8e20fb9640 1 -- 172.21.15.43:0/2987395853 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e0c032440 con 0x7f8e1c0d09b0 2024-04-25T09:04:17.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.557+0000 7f8e20fb9640 1 -- 172.21.15.43:0/2987395853 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e0c032810 con 0x7f8e1c0d09b0 2024-04-25T09:04:17.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.558+0000 7f8e2421f640 1 -- 172.21.15.43:0/2987395853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e1c0d09b0 msgr2=0x7f8e1c0b6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:17.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.558+0000 7f8e2421f640 1 --2- 172.21.15.43:0/2987395853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e1c0d09b0 0x7f8e1c0b6820 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c0097a0 tx=0x7f8e0c02a230 comp rx=0 tx=0).stop 2024-04-25T09:04:17.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.558+0000 7f8e2421f640 1 -- 172.21.15.43:0/2987395853 shutdown_connections 2024-04-25T09:04:17.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.558+0000 7f8e2421f640 1 --2- 172.21.15.43:0/2987395853 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e1c0d09b0 0x7f8e1c0b6820 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.558+0000 7f8e2421f640 1 -- 172.21.15.43:0/2987395853 >> 172.21.15.43:0/2987395853 conn(0x7f8e1c09cd20 msgr2=0x7f8e1c0d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:17.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.558+0000 7f8e2421f640 1 -- 172.21.15.43:0/2987395853 shutdown_connections 2024-04-25T09:04:17.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.558+0000 7f8e2421f640 1 -- 172.21.15.43:0/2987395853 wait complete. 2024-04-25T09:04:17.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.559+0000 7f8e2421f640 1 Processor -- start 2024-04-25T09:04:17.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.559+0000 7f8e2421f640 1 -- start start 2024-04-25T09:04:17.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.560+0000 7f8e2421f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8e1c0ce5e0 0x7f8e1c1f36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:17.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.560+0000 7f8e2421f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e1c0d09b0 0x7f8e1c15b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:17.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.560+0000 7f8e2421f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e1c161390 0x7f8e1c15b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:17.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.560+0000 7f8e2421f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1c161770 con 0x7f8e1c161390 2024-04-25T09:04:17.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.560+0000 7f8e2421f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1c0d15a0 con 0x7f8e1c0d09b0 2024-04-25T09:04:17.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.560+0000 7f8e2421f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e1c0d1da0 con 0x7f8e1c0ce5e0 2024-04-25T09:04:17.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.560+0000 7f8e227bc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e1c161390 0x7f8e1c15b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:17.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.560+0000 7f8e227bc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e1c161390 0x7f8e1c15b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47640/0 (socket says 172.21.15.43:47640) 2024-04-25T09:04:17.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.560+0000 7f8e227bc640 1 -- 172.21.15.43:0/1018160448 learned_addr learned my addr 172.21.15.43:0/1018160448 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:17.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.560+0000 7f8e217ba640 1 --2- 172.21.15.43:0/1018160448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e1c0d09b0 0x7f8e1c15b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:17.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.560+0000 7f8e21fbb640 1 --2- 172.21.15.43:0/1018160448 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8e1c0ce5e0 0x7f8e1c1f36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:17.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.560+0000 7f8e227bc640 1 -- 172.21.15.43:0/1018160448 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8e1c0ce5e0 msgr2=0x7f8e1c1f36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:17.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.560+0000 7f8e227bc640 1 --2- 172.21.15.43:0/1018160448 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8e1c0ce5e0 0x7f8e1c1f36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.560+0000 7f8e227bc640 1 -- 172.21.15.43:0/1018160448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e1c0d09b0 msgr2=0x7f8e1c15b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:17.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.560+0000 7f8e227bc640 1 --2- 172.21.15.43:0/1018160448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e1c0d09b0 0x7f8e1c15b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.560+0000 7f8e227bc640 1 -- 172.21.15.43:0/1018160448 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e1c1f7530 con 0x7f8e1c161390 2024-04-25T09:04:17.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.560+0000 7f8e21fbb640 1 --2- 172.21.15.43:0/1018160448 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8e1c0ce5e0 0x7f8e1c1f36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:17.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.560+0000 7f8e227bc640 1 --2- 172.21.15.43:0/1018160448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e1c161390 0x7f8e1c15b8d0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f8e1400cbf0 tx=0x7f8e1400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:17.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.560+0000 7f8e127fc640 1 -- 172.21.15.43:0/1018160448 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e14017070 con 0x7f8e1c161390 2024-04-25T09:04:17.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.561+0000 7f8e127fc640 1 -- 172.21.15.43:0/1018160448 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e1400ad40 con 0x7f8e1c161390 2024-04-25T09:04:17.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.561+0000 7f8e2421f640 1 -- 172.21.15.43:0/1018160448 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e1c1f7810 con 0x7f8e1c161390 2024-04-25T09:04:17.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.561+0000 7f8e127fc640 1 -- 172.21.15.43:0/1018160448 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e14012600 con 0x7f8e1c161390 2024-04-25T09:04:17.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.561+0000 7f8e2421f640 1 -- 172.21.15.43:0/1018160448 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e1c1f7dd0 con 0x7f8e1c161390 2024-04-25T09:04:17.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.562+0000 7f8e2421f640 1 -- 172.21.15.43:0/1018160448 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e1c1f83b0 con 0x7f8e1c161390 2024-04-25T09:04:17.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.562+0000 7f8e127fc640 1 -- 172.21.15.43:0/1018160448 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8e1401a070 con 0x7f8e1c161390 2024-04-25T09:04:17.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.562+0000 7f8e127fc640 1 --2- 172.21.15.43:0/1018160448 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8e0006b4d0 0x7f8e0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:17.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.563+0000 7f8e127fc640 1 -- 172.21.15.43:0/1018160448 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8e14089ed0 con 0x7f8e1c161390 2024-04-25T09:04:17.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.563+0000 7f8e21fbb640 1 --2- 172.21.15.43:0/1018160448 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8e0006b4d0 0x7f8e0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:17.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.563+0000 7f8e21fbb640 1 --2- 172.21.15.43:0/1018160448 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8e0006b4d0 0x7f8e0006d990 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f8e18001660 tx=0x7f8e18009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:17.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.567+0000 7f8e127fc640 1 -- 172.21.15.43:0/1018160448 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8e1c0b4c30 con 0x7f8e1c161390 2024-04-25T09:04:17.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.742+0000 7f8e2421f640 1 -- 172.21.15.43:0/1018160448 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8e1c0680f0 con 0x7f8e0006b4d0 2024-04-25T09:04:17.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.749+0000 7f8e127fc640 1 -- 172.21.15.43:0/1018160448 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247089 (secure 0 0 0) 0x7f8e1c0680f0 con 0x7f8e0006b4d0 2024-04-25T09:04:17.740 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:04:17.741 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-25T09:04:17.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.752+0000 7f8e2421f640 1 -- 172.21.15.43:0/1018160448 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8e0006b4d0 msgr2=0x7f8e0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:17.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.752+0000 7f8e2421f640 1 --2- 172.21.15.43:0/1018160448 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8e0006b4d0 0x7f8e0006d990 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f8e18001660 tx=0x7f8e18009040 comp rx=0 tx=0).stop 2024-04-25T09:04:17.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.752+0000 7f8e2421f640 1 -- 172.21.15.43:0/1018160448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e1c161390 msgr2=0x7f8e1c15b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:17.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.752+0000 7f8e2421f640 1 --2- 172.21.15.43:0/1018160448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e1c161390 0x7f8e1c15b8d0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f8e1400cbf0 tx=0x7f8e1400a430 comp rx=0 tx=0).stop 2024-04-25T09:04:17.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.752+0000 7f8e2421f640 1 -- 172.21.15.43:0/1018160448 shutdown_connections 2024-04-25T09:04:17.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.752+0000 7f8e2421f640 1 --2- 172.21.15.43:0/1018160448 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e1c161390 0x7f8e1c15b8d0 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.752+0000 7f8e2421f640 1 --2- 172.21.15.43:0/1018160448 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8e0006b4d0 0x7f8e0006d990 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.752+0000 7f8e2421f640 1 --2- 172.21.15.43:0/1018160448 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e1c0d09b0 0x7f8e1c15b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.752+0000 7f8e2421f640 1 --2- 172.21.15.43:0/1018160448 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8e1c0ce5e0 0x7f8e1c1f36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.752+0000 7f8e2421f640 1 -- 172.21.15.43:0/1018160448 >> 172.21.15.43:0/1018160448 conn(0x7f8e1c09cd20 msgr2=0x7f8e1c0b7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:17.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.752+0000 7f8e2421f640 1 -- 172.21.15.43:0/1018160448 shutdown_connections 2024-04-25T09:04:17.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.752+0000 7f8e2421f640 1 -- 172.21.15.43:0/1018160448 wait complete. 2024-04-25T09:04:17.751 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":441,"stamp":"2024-04-25T09:04:15.835885+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":498656,"kb_used_data":14568,"kb_used_omap":12,"kb_used_meta":483955,"kb_avail":754476064,"statfs":{"total":773094113280,"available":772583489536,"internally_reserved":0,"allocated":14917632,"data_stored":7178872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":495570481},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000966"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.053615+0000","last_change":"2024-04-25T09:04:13.053615+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.053615+0000","last_undegraded":"2024-04-25T09:04:13.053615+0000","last_fullsized":"2024-04-25T09:04:13.053615+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.047818+0000","last_change":"2024-04-25T09:04:13.047818+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.047818+0000","last_undegraded":"2024-04-25T09:04:13.047818+0000","last_fullsized":"2024-04-25T09:04:13.047818+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.049520+0000","last_change":"2024-04-25T09:04:13.049520+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.049520+0000","last_undegraded":"2024-04-25T09:04:13.049520+0000","last_fullsized":"2024-04-25T09:04:13.049520+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046148+0000","last_change":"2024-04-25T09:04:13.046148+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046148+0000","last_undegraded":"2024-04-25T09:04:13.046148+0000","last_fullsized":"2024-04-25T09:04:13.046148+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.047544+0000","last_change":"2024-04-25T09:04:13.047544+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.047544+0000","last_undegraded":"2024-04-25T09:04:13.047544+0000","last_fullsized":"2024-04-25T09:04:13.047544+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.051007+0000","last_change":"2024-04-25T09:04:13.051007+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.051007+0000","last_undegraded":"2024-04-25T09:04:13.051007+0000","last_fullsized":"2024-04-25T09:04:13.051007+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.048146+0000","last_change":"2024-04-25T09:04:13.048146+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.048146+0000","last_undegraded":"2024-04-25T09:04:13.048146+0000","last_fullsized":"2024-04-25T09:04:13.048146+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.049952+0000","last_change":"2024-04-25T09:04:13.049952+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.049952+0000","last_undegraded":"2024-04-25T09:04:13.049952+0000","last_fullsized":"2024-04-25T09:04:13.049952+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.047013+0000","last_change":"2024-04-25T09:04:13.047013+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.047013+0000","last_undegraded":"2024-04-25T09:04:13.047013+0000","last_fullsized":"2024-04-25T09:04:13.047013+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.052404+0000","last_change":"2024-04-25T09:04:13.052404+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.052404+0000","last_undegraded":"2024-04-25T09:04:13.052404+0000","last_fullsized":"2024-04-25T09:04:13.052404+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.050451+0000","last_change":"2024-04-25T09:04:13.050451+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.050451+0000","last_undegraded":"2024-04-25T09:04:13.050451+0000","last_fullsized":"2024-04-25T09:04:13.050451+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.054302+0000","last_change":"2024-04-25T09:04:13.054302+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.054302+0000","last_undegraded":"2024-04-25T09:04:13.054302+0000","last_fullsized":"2024-04-25T09:04:13.054302+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.047149+0000","last_change":"2024-04-25T09:04:13.047149+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.047149+0000","last_undegraded":"2024-04-25T09:04:13.047149+0000","last_fullsized":"2024-04-25T09:04:13.047149+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.048543+0000","last_change":"2024-04-25T09:04:13.048543+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.048543+0000","last_undegraded":"2024-04-25T09:04:13.048543+0000","last_fullsized":"2024-04-25T09:04:13.048543+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.050737+0000","last_change":"2024-04-25T09:04:13.050737+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.050737+0000","last_undegraded":"2024-04-25T09:04:13.050737+0000","last_fullsized":"2024-04-25T09:04:13.050737+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.049349+0000","last_change":"2024-04-25T09:04:13.049349+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.049349+0000","last_undegraded":"2024-04-25T09:04:13.049349+0000","last_fullsized":"2024-04-25T09:04:13.049349+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.051581+0000","last_change":"2024-04-25T09:04:13.051581+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.051581+0000","last_undegraded":"2024-04-25T09:04:13.051581+0000","last_fullsized":"2024-04-25T09:04:13.051581+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.052944+0000","last_change":"2024-04-25T09:04:13.052944+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.052944+0000","last_undegraded":"2024-04-25T09:04:13.052944+0000","last_fullsized":"2024-04-25T09:04:13.052944+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.049189+0000","last_change":"2024-04-25T09:04:13.049189+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.049189+0000","last_undegraded":"2024-04-25T09:04:13.049189+0000","last_fullsized":"2024-04-25T09:04:13.049189+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046642+0000","last_change":"2024-04-25T09:04:13.046642+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046642+0000","last_undegraded":"2024-04-25T09:04:13.046642+0000","last_fullsized":"2024-04-25T09:04:13.046642+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.051136+0000","last_change":"2024-04-25T09:04:13.051136+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.051136+0000","last_undegraded":"2024-04-25T09:04:13.051136+0000","last_fullsized":"2024-04-25T09:04:13.051136+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.048012+0000","last_change":"2024-04-25T09:04:13.048012+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.048012+0000","last_undegraded":"2024-04-25T09:04:13.048012+0000","last_fullsized":"2024-04-25T09:04:13.048012+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046277+0000","last_change":"2024-04-25T09:04:13.046277+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046277+0000","last_undegraded":"2024-04-25T09:04:13.046277+0000","last_fullsized":"2024-04-25T09:04:13.046277+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.051677+0000","last_change":"2024-04-25T09:04:13.051677+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.051677+0000","last_undegraded":"2024-04-25T09:04:13.051677+0000","last_fullsized":"2024-04-25T09:04:13.051677+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.052436+0000","last_change":"2024-04-25T09:04:13.052436+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.052436+0000","last_undegraded":"2024-04-25T09:04:13.052436+0000","last_fullsized":"2024-04-25T09:04:13.052436+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.049685+0000","last_change":"2024-04-25T09:04:13.049685+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.049685+0000","last_undegraded":"2024-04-25T09:04:13.049685+0000","last_fullsized":"2024-04-25T09:04:13.049685+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.048115+0000","last_change":"2024-04-25T09:04:13.048115+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.048115+0000","last_undegraded":"2024-04-25T09:04:13.048115+0000","last_fullsized":"2024-04-25T09:04:13.048115+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046108+0000","last_change":"2024-04-25T09:04:13.046108+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046108+0000","last_undegraded":"2024-04-25T09:04:13.046108+0000","last_fullsized":"2024-04-25T09:04:13.046108+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.048479+0000","last_change":"2024-04-25T09:04:13.048479+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.048479+0000","last_undegraded":"2024-04-25T09:04:13.048479+0000","last_fullsized":"2024-04-25T09:04:13.048479+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.047436+0000","last_change":"2024-04-25T09:04:13.047436+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.047436+0000","last_undegraded":"2024-04-25T09:04:13.047436+0000","last_fullsized":"2024-04-25T09:04:13.047436+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046485+0000","last_change":"2024-04-25T09:04:13.046485+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046485+0000","last_undegraded":"2024-04-25T09:04:13.046485+0000","last_fullsized":"2024-04-25T09:04:13.046485+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.048912+0000","last_change":"2024-04-25T09:04:13.048912+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.048912+0000","last_undegraded":"2024-04-25T09:04:13.048912+0000","last_fullsized":"2024-04-25T09:04:13.048912+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.051989+0000","last_change":"2024-04-25T09:04:13.051989+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.051989+0000","last_undegraded":"2024-04-25T09:04:13.051989+0000","last_fullsized":"2024-04-25T09:04:13.051989+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.063563+0000","last_change":"2024-04-25T09:04:13.062156+0000","last_active":"2024-04-25T09:04:13.063563+0000","last_peered":"2024-04-25T09:04:13.063563+0000","last_clean":"2024-04-25T09:04:13.063563+0000","last_became_active":"2024-04-25T09:04:13.061852+0000","last_became_peered":"2024-04-25T09:04:13.061852+0000","last_unstale":"2024-04-25T09:04:13.063563+0000","last_undegraded":"2024-04-25T09:04:13.063563+0000","last_fullsized":"2024-04-25T09:04:13.063563+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:10.458178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046457+0000","last_change":"2024-04-25T09:04:13.046457+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046457+0000","last_undegraded":"2024-04-25T09:04:13.046457+0000","last_fullsized":"2024-04-25T09:04:13.046457+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.059517+0000","last_change":"2024-04-25T09:04:13.058762+0000","last_active":"2024-04-25T09:04:13.059517+0000","last_peered":"2024-04-25T09:04:13.059517+0000","last_clean":"2024-04-25T09:04:13.059517+0000","last_became_active":"2024-04-25T09:04:13.058335+0000","last_became_peered":"2024-04-25T09:04:13.058335+0000","last_unstale":"2024-04-25T09:04:13.059517+0000","last_undegraded":"2024-04-25T09:04:13.059517+0000","last_fullsized":"2024-04-25T09:04:13.059517+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:59.348802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.048044+0000","last_change":"2024-04-25T09:04:13.048044+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.048044+0000","last_undegraded":"2024-04-25T09:04:13.048044+0000","last_fullsized":"2024-04-25T09:04:13.048044+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.059899+0000","last_change":"2024-04-25T09:04:13.057742+0000","last_active":"2024-04-25T09:04:13.059899+0000","last_peered":"2024-04-25T09:04:13.059899+0000","last_clean":"2024-04-25T09:04:13.059899+0000","last_became_active":"2024-04-25T09:04:13.057448+0000","last_became_peered":"2024-04-25T09:04:13.057448+0000","last_unstale":"2024-04-25T09:04:13.059899+0000","last_undegraded":"2024-04-25T09:04:13.059899+0000","last_fullsized":"2024-04-25T09:04:13.059899+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:47.545889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.053963+0000","last_change":"2024-04-25T09:04:13.053963+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.053963+0000","last_undegraded":"2024-04-25T09:04:13.053963+0000","last_fullsized":"2024-04-25T09:04:13.053963+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.059924+0000","last_change":"2024-04-25T09:04:13.058406+0000","last_active":"2024-04-25T09:04:13.059924+0000","last_peered":"2024-04-25T09:04:13.059924+0000","last_clean":"2024-04-25T09:04:13.059924+0000","last_became_active":"2024-04-25T09:04:13.058144+0000","last_became_peered":"2024-04-25T09:04:13.058144+0000","last_unstale":"2024-04-25T09:04:13.059924+0000","last_undegraded":"2024-04-25T09:04:13.059924+0000","last_fullsized":"2024-04-25T09:04:13.059924+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:33.060943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046085+0000","last_change":"2024-04-25T09:04:13.046085+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046085+0000","last_undegraded":"2024-04-25T09:04:13.046085+0000","last_fullsized":"2024-04-25T09:04:13.046085+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060240+0000","last_change":"2024-04-25T09:04:13.059157+0000","last_active":"2024-04-25T09:04:13.060240+0000","last_peered":"2024-04-25T09:04:13.060240+0000","last_clean":"2024-04-25T09:04:13.060240+0000","last_became_active":"2024-04-25T09:04:13.058883+0000","last_became_peered":"2024-04-25T09:04:13.058883+0000","last_unstale":"2024-04-25T09:04:13.060240+0000","last_undegraded":"2024-04-25T09:04:13.060240+0000","last_fullsized":"2024-04-25T09:04:13.060240+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:38.159347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046155+0000","last_change":"2024-04-25T09:04:13.046155+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046155+0000","last_undegraded":"2024-04-25T09:04:13.046155+0000","last_fullsized":"2024-04-25T09:04:13.046155+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.059938+0000","last_change":"2024-04-25T09:04:13.057593+0000","last_active":"2024-04-25T09:04:13.059938+0000","last_peered":"2024-04-25T09:04:13.059938+0000","last_clean":"2024-04-25T09:04:13.059938+0000","last_became_active":"2024-04-25T09:04:13.057330+0000","last_became_peered":"2024-04-25T09:04:13.057330+0000","last_unstale":"2024-04-25T09:04:13.059938+0000","last_undegraded":"2024-04-25T09:04:13.059938+0000","last_fullsized":"2024-04-25T09:04:13.059938+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:32.141616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046021+0000","last_change":"2024-04-25T09:04:13.046021+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046021+0000","last_undegraded":"2024-04-25T09:04:13.046021+0000","last_fullsized":"2024-04-25T09:04:13.046021+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.056354+0000","last_change":"2024-04-25T09:04:13.054808+0000","last_active":"2024-04-25T09:04:13.056354+0000","last_peered":"2024-04-25T09:04:13.056354+0000","last_clean":"2024-04-25T09:04:13.056354+0000","last_became_active":"2024-04-25T09:04:13.054486+0000","last_became_peered":"2024-04-25T09:04:13.054486+0000","last_unstale":"2024-04-25T09:04:13.056354+0000","last_undegraded":"2024-04-25T09:04:13.056354+0000","last_fullsized":"2024-04-25T09:04:13.056354+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:24.933992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.045906+0000","last_change":"2024-04-25T09:04:13.045906+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.045906+0000","last_undegraded":"2024-04-25T09:04:13.045906+0000","last_fullsized":"2024-04-25T09:04:13.045906+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.057196+0000","last_change":"2024-04-25T09:04:13.056215+0000","last_active":"2024-04-25T09:04:13.057196+0000","last_peered":"2024-04-25T09:04:13.057196+0000","last_clean":"2024-04-25T09:04:13.057196+0000","last_became_active":"2024-04-25T09:04:13.055952+0000","last_became_peered":"2024-04-25T09:04:13.055952+0000","last_unstale":"2024-04-25T09:04:13.057196+0000","last_undegraded":"2024-04-25T09:04:13.057196+0000","last_fullsized":"2024-04-25T09:04:13.057196+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:39.762504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.045952+0000","last_change":"2024-04-25T09:04:13.045952+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.045952+0000","last_undegraded":"2024-04-25T09:04:13.045952+0000","last_fullsized":"2024-04-25T09:04:13.045952+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.057234+0000","last_change":"2024-04-25T09:04:13.051625+0000","last_active":"2024-04-25T09:04:13.057234+0000","last_peered":"2024-04-25T09:04:13.057234+0000","last_clean":"2024-04-25T09:04:13.057234+0000","last_became_active":"2024-04-25T09:04:13.051307+0000","last_became_peered":"2024-04-25T09:04:13.051307+0000","last_unstale":"2024-04-25T09:04:13.057234+0000","last_undegraded":"2024-04-25T09:04:13.057234+0000","last_fullsized":"2024-04-25T09:04:13.057234+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:06.808616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.052044+0000","last_change":"2024-04-25T09:04:13.052044+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.052044+0000","last_undegraded":"2024-04-25T09:04:13.052044+0000","last_fullsized":"2024-04-25T09:04:13.052044+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060131+0000","last_change":"2024-04-25T09:04:13.058585+0000","last_active":"2024-04-25T09:04:13.060131+0000","last_peered":"2024-04-25T09:04:13.060131+0000","last_clean":"2024-04-25T09:04:13.060131+0000","last_became_active":"2024-04-25T09:04:13.058298+0000","last_became_peered":"2024-04-25T09:04:13.058298+0000","last_unstale":"2024-04-25T09:04:13.060131+0000","last_undegraded":"2024-04-25T09:04:13.060131+0000","last_fullsized":"2024-04-25T09:04:13.060131+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:25.170530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.047591+0000","last_change":"2024-04-25T09:04:13.047591+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.047591+0000","last_undegraded":"2024-04-25T09:04:13.047591+0000","last_fullsized":"2024-04-25T09:04:13.047591+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.061150+0000","last_change":"2024-04-25T09:04:13.058981+0000","last_active":"2024-04-25T09:04:13.061150+0000","last_peered":"2024-04-25T09:04:13.061150+0000","last_clean":"2024-04-25T09:04:13.061150+0000","last_became_active":"2024-04-25T09:04:13.058662+0000","last_became_peered":"2024-04-25T09:04:13.058662+0000","last_unstale":"2024-04-25T09:04:13.061150+0000","last_undegraded":"2024-04-25T09:04:13.061150+0000","last_fullsized":"2024-04-25T09:04:13.061150+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:35.416425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.049288+0000","last_change":"2024-04-25T09:04:13.049288+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.049288+0000","last_undegraded":"2024-04-25T09:04:13.049288+0000","last_fullsized":"2024-04-25T09:04:13.049288+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060649+0000","last_change":"2024-04-25T09:04:13.058867+0000","last_active":"2024-04-25T09:04:13.060649+0000","last_peered":"2024-04-25T09:04:13.060649+0000","last_clean":"2024-04-25T09:04:13.060649+0000","last_became_active":"2024-04-25T09:04:13.057622+0000","last_became_peered":"2024-04-25T09:04:13.057622+0000","last_unstale":"2024-04-25T09:04:13.060649+0000","last_undegraded":"2024-04-25T09:04:13.060649+0000","last_fullsized":"2024-04-25T09:04:13.060649+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:47.545889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.045500+0000","last_change":"2024-04-25T09:04:13.045500+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.045500+0000","last_undegraded":"2024-04-25T09:04:13.045500+0000","last_fullsized":"2024-04-25T09:04:13.045500+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.055227+0000","last_change":"2024-04-25T09:04:13.054034+0000","last_active":"2024-04-25T09:04:13.055227+0000","last_peered":"2024-04-25T09:04:13.055227+0000","last_clean":"2024-04-25T09:04:13.055227+0000","last_became_active":"2024-04-25T09:04:13.053760+0000","last_became_peered":"2024-04-25T09:04:13.053760+0000","last_unstale":"2024-04-25T09:04:13.055227+0000","last_undegraded":"2024-04-25T09:04:13.055227+0000","last_fullsized":"2024-04-25T09:04:13.055227+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:39.762504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.045249+0000","last_change":"2024-04-25T09:04:13.045249+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.045249+0000","last_undegraded":"2024-04-25T09:04:13.045249+0000","last_fullsized":"2024-04-25T09:04:13.045249+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060397+0000","last_change":"2024-04-25T09:04:13.056393+0000","last_active":"2024-04-25T09:04:13.060397+0000","last_peered":"2024-04-25T09:04:13.060397+0000","last_clean":"2024-04-25T09:04:13.060397+0000","last_became_active":"2024-04-25T09:04:13.056111+0000","last_became_peered":"2024-04-25T09:04:13.056111+0000","last_unstale":"2024-04-25T09:04:13.060397+0000","last_undegraded":"2024-04-25T09:04:13.060397+0000","last_fullsized":"2024-04-25T09:04:13.060397+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:39.762504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.049874+0000","last_change":"2024-04-25T09:04:13.049874+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.049874+0000","last_undegraded":"2024-04-25T09:04:13.049874+0000","last_fullsized":"2024-04-25T09:04:13.049874+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.062256+0000","last_change":"2024-04-25T09:04:13.061556+0000","last_active":"2024-04-25T09:04:13.062256+0000","last_peered":"2024-04-25T09:04:13.062256+0000","last_clean":"2024-04-25T09:04:13.062256+0000","last_became_active":"2024-04-25T09:04:13.061241+0000","last_became_peered":"2024-04-25T09:04:13.061241+0000","last_unstale":"2024-04-25T09:04:13.062256+0000","last_undegraded":"2024-04-25T09:04:13.062256+0000","last_fullsized":"2024-04-25T09:04:13.062256+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:47.072066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.051480+0000","last_change":"2024-04-25T09:04:13.051480+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.051480+0000","last_undegraded":"2024-04-25T09:04:13.051480+0000","last_fullsized":"2024-04-25T09:04:13.051480+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.055764+0000","last_change":"2024-04-25T09:04:13.053984+0000","last_active":"2024-04-25T09:04:13.055764+0000","last_peered":"2024-04-25T09:04:13.055764+0000","last_clean":"2024-04-25T09:04:13.055764+0000","last_became_active":"2024-04-25T09:04:13.053588+0000","last_became_peered":"2024-04-25T09:04:13.053588+0000","last_unstale":"2024-04-25T09:04:13.055764+0000","last_undegraded":"2024-04-25T09:04:13.055764+0000","last_fullsized":"2024-04-25T09:04:13.055764+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:05.903006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.051515+0000","last_change":"2024-04-25T09:04:13.051515+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.051515+0000","last_undegraded":"2024-04-25T09:04:13.051515+0000","last_fullsized":"2024-04-25T09:04:13.051515+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.057493+0000","last_change":"2024-04-25T09:04:13.055593+0000","last_active":"2024-04-25T09:04:13.057493+0000","last_peered":"2024-04-25T09:04:13.057493+0000","last_clean":"2024-04-25T09:04:13.057493+0000","last_became_active":"2024-04-25T09:04:13.055293+0000","last_became_peered":"2024-04-25T09:04:13.055293+0000","last_unstale":"2024-04-25T09:04:13.057493+0000","last_undegraded":"2024-04-25T09:04:13.057493+0000","last_fullsized":"2024-04-25T09:04:13.057493+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:09.194558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.050011+0000","last_change":"2024-04-25T09:04:13.050011+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.050011+0000","last_undegraded":"2024-04-25T09:04:13.050011+0000","last_fullsized":"2024-04-25T09:04:13.050011+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060206+0000","last_change":"2024-04-25T09:04:13.057835+0000","last_active":"2024-04-25T09:04:13.060206+0000","last_peered":"2024-04-25T09:04:13.060206+0000","last_clean":"2024-04-25T09:04:13.060206+0000","last_became_active":"2024-04-25T09:04:13.057562+0000","last_became_peered":"2024-04-25T09:04:13.057562+0000","last_unstale":"2024-04-25T09:04:13.060206+0000","last_undegraded":"2024-04-25T09:04:13.060206+0000","last_fullsized":"2024-04-25T09:04:13.060206+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:35.248830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.048188+0000","last_change":"2024-04-25T09:04:13.048188+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.048188+0000","last_undegraded":"2024-04-25T09:04:13.048188+0000","last_fullsized":"2024-04-25T09:04:13.048188+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.062094+0000","last_change":"2024-04-25T09:04:13.060717+0000","last_active":"2024-04-25T09:04:13.062094+0000","last_peered":"2024-04-25T09:04:13.062094+0000","last_clean":"2024-04-25T09:04:13.062094+0000","last_became_active":"2024-04-25T09:04:13.060060+0000","last_became_peered":"2024-04-25T09:04:13.060060+0000","last_unstale":"2024-04-25T09:04:13.062094+0000","last_undegraded":"2024-04-25T09:04:13.062094+0000","last_fullsized":"2024-04-25T09:04:13.062094+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:56.805484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046887+0000","last_change":"2024-04-25T09:04:13.046887+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046887+0000","last_undegraded":"2024-04-25T09:04:13.046887+0000","last_fullsized":"2024-04-25T09:04:13.046887+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.056661+0000","last_change":"2024-04-25T09:04:13.054317+0000","last_active":"2024-04-25T09:04:13.056661+0000","last_peered":"2024-04-25T09:04:13.056661+0000","last_clean":"2024-04-25T09:04:13.056661+0000","last_became_active":"2024-04-25T09:04:13.053816+0000","last_became_peered":"2024-04-25T09:04:13.053816+0000","last_unstale":"2024-04-25T09:04:13.056661+0000","last_undegraded":"2024-04-25T09:04:13.056661+0000","last_fullsized":"2024-04-25T09:04:13.056661+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:12.230718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046289+0000","last_change":"2024-04-25T09:04:13.046289+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046289+0000","last_undegraded":"2024-04-25T09:04:13.046289+0000","last_fullsized":"2024-04-25T09:04:13.046289+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.063552+0000","last_change":"2024-04-25T09:04:13.061478+0000","last_active":"2024-04-25T09:04:13.063552+0000","last_peered":"2024-04-25T09:04:13.063552+0000","last_clean":"2024-04-25T09:04:13.063552+0000","last_became_active":"2024-04-25T09:04:13.061150+0000","last_became_peered":"2024-04-25T09:04:13.061150+0000","last_unstale":"2024-04-25T09:04:13.063552+0000","last_undegraded":"2024-04-25T09:04:13.063552+0000","last_fullsized":"2024-04-25T09:04:13.063552+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:52.903859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.045727+0000","last_change":"2024-04-25T09:04:13.045727+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.045727+0000","last_undegraded":"2024-04-25T09:04:13.045727+0000","last_fullsized":"2024-04-25T09:04:13.045727+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060503+0000","last_change":"2024-04-25T09:04:13.059762+0000","last_active":"2024-04-25T09:04:13.060503+0000","last_peered":"2024-04-25T09:04:13.060503+0000","last_clean":"2024-04-25T09:04:13.060503+0000","last_became_active":"2024-04-25T09:04:13.059482+0000","last_became_peered":"2024-04-25T09:04:13.059482+0000","last_unstale":"2024-04-25T09:04:13.060503+0000","last_undegraded":"2024-04-25T09:04:13.060503+0000","last_fullsized":"2024-04-25T09:04:13.060503+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:30:52.194928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.045307+0000","last_change":"2024-04-25T09:04:13.045307+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.045307+0000","last_undegraded":"2024-04-25T09:04:13.045307+0000","last_fullsized":"2024-04-25T09:04:13.045307+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060790+0000","last_change":"2024-04-25T09:04:13.059953+0000","last_active":"2024-04-25T09:04:13.060790+0000","last_peered":"2024-04-25T09:04:13.060790+0000","last_clean":"2024-04-25T09:04:13.060790+0000","last_became_active":"2024-04-25T09:04:13.059633+0000","last_became_peered":"2024-04-25T09:04:13.059633+0000","last_unstale":"2024-04-25T09:04:13.060790+0000","last_undegraded":"2024-04-25T09:04:13.060790+0000","last_fullsized":"2024-04-25T09:04:13.060790+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:04.737828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.045467+0000","last_change":"2024-04-25T09:04:13.045467+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.045467+0000","last_undegraded":"2024-04-25T09:04:13.045467+0000","last_fullsized":"2024-04-25T09:04:13.045467+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.063544+0000","last_change":"2024-04-25T09:04:13.062137+0000","last_active":"2024-04-25T09:04:13.063544+0000","last_peered":"2024-04-25T09:04:13.063544+0000","last_clean":"2024-04-25T09:04:13.063544+0000","last_became_active":"2024-04-25T09:04:13.061505+0000","last_became_peered":"2024-04-25T09:04:13.061505+0000","last_unstale":"2024-04-25T09:04:13.063544+0000","last_undegraded":"2024-04-25T09:04:13.063544+0000","last_fullsized":"2024-04-25T09:04:13.063544+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:27.711642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.050690+0000","last_change":"2024-04-25T09:04:13.050690+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.050690+0000","last_undegraded":"2024-04-25T09:04:13.050690+0000","last_fullsized":"2024-04-25T09:04:13.050690+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060823+0000","last_change":"2024-04-25T09:04:13.057849+0000","last_active":"2024-04-25T09:04:13.060823+0000","last_peered":"2024-04-25T09:04:13.060823+0000","last_clean":"2024-04-25T09:04:13.060823+0000","last_became_active":"2024-04-25T09:04:13.057586+0000","last_became_peered":"2024-04-25T09:04:13.057586+0000","last_unstale":"2024-04-25T09:04:13.060823+0000","last_undegraded":"2024-04-25T09:04:13.060823+0000","last_fullsized":"2024-04-25T09:04:13.060823+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:33.060943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.044936+0000","last_change":"2024-04-25T09:04:13.044936+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.044936+0000","last_undegraded":"2024-04-25T09:04:13.044936+0000","last_fullsized":"2024-04-25T09:04:13.044936+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060734+0000","last_change":"2024-04-25T09:04:13.056588+0000","last_active":"2024-04-25T09:04:13.060734+0000","last_peered":"2024-04-25T09:04:13.060734+0000","last_clean":"2024-04-25T09:04:13.060734+0000","last_became_active":"2024-04-25T09:04:13.055410+0000","last_became_peered":"2024-04-25T09:04:13.055410+0000","last_unstale":"2024-04-25T09:04:13.060734+0000","last_undegraded":"2024-04-25T09:04:13.060734+0000","last_fullsized":"2024-04-25T09:04:13.060734+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:47.545889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.047647+0000","last_change":"2024-04-25T09:04:13.047647+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.047647+0000","last_undegraded":"2024-04-25T09:04:13.047647+0000","last_fullsized":"2024-04-25T09:04:13.047647+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060898+0000","last_change":"2024-04-25T09:04:13.058491+0000","last_active":"2024-04-25T09:04:13.060898+0000","last_peered":"2024-04-25T09:04:13.060898+0000","last_clean":"2024-04-25T09:04:13.060898+0000","last_became_active":"2024-04-25T09:04:13.058215+0000","last_became_peered":"2024-04-25T09:04:13.058215+0000","last_unstale":"2024-04-25T09:04:13.060898+0000","last_undegraded":"2024-04-25T09:04:13.060898+0000","last_fullsized":"2024-04-25T09:04:13.060898+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:04.737828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.047830+0000","last_change":"2024-04-25T09:04:13.047830+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.047830+0000","last_undegraded":"2024-04-25T09:04:13.047830+0000","last_fullsized":"2024-04-25T09:04:13.047830+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.056639+0000","last_change":"2024-04-25T09:04:13.054092+0000","last_active":"2024-04-25T09:04:13.056639+0000","last_peered":"2024-04-25T09:04:13.056639+0000","last_clean":"2024-04-25T09:04:13.056639+0000","last_became_active":"2024-04-25T09:04:13.053427+0000","last_became_peered":"2024-04-25T09:04:13.053427+0000","last_unstale":"2024-04-25T09:04:13.056639+0000","last_undegraded":"2024-04-25T09:04:13.056639+0000","last_fullsized":"2024-04-25T09:04:13.056639+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:47.205245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046670+0000","last_change":"2024-04-25T09:04:13.046670+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046670+0000","last_undegraded":"2024-04-25T09:04:13.046670+0000","last_fullsized":"2024-04-25T09:04:13.046670+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.063064+0000","last_change":"2024-04-25T09:04:13.060816+0000","last_active":"2024-04-25T09:04:13.063064+0000","last_peered":"2024-04-25T09:04:13.063064+0000","last_clean":"2024-04-25T09:04:13.063064+0000","last_became_active":"2024-04-25T09:04:13.060535+0000","last_became_peered":"2024-04-25T09:04:13.060535+0000","last_unstale":"2024-04-25T09:04:13.063064+0000","last_undegraded":"2024-04-25T09:04:13.063064+0000","last_fullsized":"2024-04-25T09:04:13.063064+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:49.426430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.050769+0000","last_change":"2024-04-25T09:04:13.050769+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.050769+0000","last_undegraded":"2024-04-25T09:04:13.050769+0000","last_fullsized":"2024-04-25T09:04:13.050769+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060740+0000","last_change":"2024-04-25T09:04:13.059109+0000","last_active":"2024-04-25T09:04:13.060740+0000","last_peered":"2024-04-25T09:04:13.060740+0000","last_clean":"2024-04-25T09:04:13.060740+0000","last_became_active":"2024-04-25T09:04:13.058224+0000","last_became_peered":"2024-04-25T09:04:13.058224+0000","last_unstale":"2024-04-25T09:04:13.060740+0000","last_undegraded":"2024-04-25T09:04:13.060740+0000","last_fullsized":"2024-04-25T09:04:13.060740+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:33.060943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.045284+0000","last_change":"2024-04-25T09:04:13.045284+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.045284+0000","last_undegraded":"2024-04-25T09:04:13.045284+0000","last_fullsized":"2024-04-25T09:04:13.045284+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060585+0000","last_change":"2024-04-25T09:04:13.057904+0000","last_active":"2024-04-25T09:04:13.060585+0000","last_peered":"2024-04-25T09:04:13.060585+0000","last_clean":"2024-04-25T09:04:13.060585+0000","last_became_active":"2024-04-25T09:04:13.057602+0000","last_became_peered":"2024-04-25T09:04:13.057602+0000","last_unstale":"2024-04-25T09:04:13.060585+0000","last_undegraded":"2024-04-25T09:04:13.060585+0000","last_fullsized":"2024-04-25T09:04:13.060585+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:24.697586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.045651+0000","last_change":"2024-04-25T09:04:13.045651+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.045651+0000","last_undegraded":"2024-04-25T09:04:13.045651+0000","last_fullsized":"2024-04-25T09:04:13.045651+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.054181+0000","last_change":"2024-04-25T09:04:13.053009+0000","last_active":"2024-04-25T09:04:13.054181+0000","last_peered":"2024-04-25T09:04:13.054181+0000","last_clean":"2024-04-25T09:04:13.054181+0000","last_became_active":"2024-04-25T09:04:13.052708+0000","last_became_peered":"2024-04-25T09:04:13.052708+0000","last_unstale":"2024-04-25T09:04:13.054181+0000","last_undegraded":"2024-04-25T09:04:13.054181+0000","last_fullsized":"2024-04-25T09:04:13.054181+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:33.000888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":388,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.054625+0000","last_change":"2024-04-25T08:54:00.818044+0000","last_active":"2024-04-25T09:04:13.054625+0000","last_peered":"2024-04-25T09:04:13.054625+0000","last_clean":"2024-04-25T09:04:13.054625+0000","last_became_active":"2024-04-25T08:54:00.817098+0000","last_became_peered":"2024-04-25T08:54:00.817098+0000","last_unstale":"2024-04-25T09:04:13.054625+0000","last_undegraded":"2024-04-25T09:04:13.054625+0000","last_fullsized":"2024-04-25T09:04:13.054625+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_clean_scrub_stamp":"2024-04-25T08:53:59.739677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:48.129513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574973,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47328,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324512,"statfs":{"total":96636764160,"available":96588300288,"internally_reserved":0,"allocated":2310144,"data_stored":1340135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574974,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68956,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302884,"statfs":{"total":96636764160,"available":96566153216,"internally_reserved":0,"allocated":1716224,"data_stored":749767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574976,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55064,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316776,"statfs":{"total":96636764160,"available":96580378624,"internally_reserved":0,"allocated":1716224,"data_stored":749767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574974,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68960,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302880,"statfs":{"total":96636764160,"available":96566149120,"internally_reserved":0,"allocated":1716224,"data_stored":749767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574974,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77276,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294564,"statfs":{"total":96636764160,"available":96557633536,"internally_reserved":0,"allocated":1716224,"data_stored":749767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574974,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46744,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325096,"statfs":{"total":96636764160,"available":96588898304,"internally_reserved":0,"allocated":1716224,"data_stored":749767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574974,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82840,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94289000,"statfs":{"total":96636764160,"available":96551936000,"internally_reserved":0,"allocated":1716224,"data_stored":749767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574975,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51488,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320352,"statfs":{"total":96636764160,"available":96584040448,"internally_reserved":0,"allocated":2310144,"data_stored":1340135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:04:17.783 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3f is not active+clean 2024-04-25T09:04:17.783 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.053615+0000', 'last_change': '2024-04-25T09:04:13.053615+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.053615+0000', 'last_undegraded': '2024-04-25T09:04:13.053615+0000', 'last_fullsized': '2024-04-25T09:04:13.053615+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:04:17.783 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3e is not active+clean 2024-04-25T09:04:17.784 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.047818+0000', 'last_change': '2024-04-25T09:04:13.047818+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.047818+0000', 'last_undegraded': '2024-04-25T09:04:13.047818+0000', 'last_fullsized': '2024-04-25T09:04:13.047818+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:04:17.784 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3d is not active+clean 2024-04-25T09:04:17.784 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.049520+0000', 'last_change': '2024-04-25T09:04:13.049520+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.049520+0000', 'last_undegraded': '2024-04-25T09:04:13.049520+0000', 'last_fullsized': '2024-04-25T09:04:13.049520+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:04:17.784 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3c is not active+clean 2024-04-25T09:04:17.784 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.046148+0000', 'last_change': '2024-04-25T09:04:13.046148+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.046148+0000', 'last_undegraded': '2024-04-25T09:04:13.046148+0000', 'last_fullsized': '2024-04-25T09:04:13.046148+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:04:17.784 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3b is not active+clean 2024-04-25T09:04:17.784 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.047544+0000', 'last_change': '2024-04-25T09:04:13.047544+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.047544+0000', 'last_undegraded': '2024-04-25T09:04:13.047544+0000', 'last_fullsized': '2024-04-25T09:04:13.047544+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:04:17.784 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3a is not active+clean 2024-04-25T09:04:17.784 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.051007+0000', 'last_change': '2024-04-25T09:04:13.051007+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.051007+0000', 'last_undegraded': '2024-04-25T09:04:13.051007+0000', 'last_fullsized': '2024-04-25T09:04:13.051007+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:04:17.784 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.39 is not active+clean 2024-04-25T09:04:17.784 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.048146+0000', 'last_change': '2024-04-25T09:04:13.048146+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.048146+0000', 'last_undegraded': '2024-04-25T09:04:13.048146+0000', 'last_fullsized': '2024-04-25T09:04:13.048146+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:04:17.785 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.38 is not active+clean 2024-04-25T09:04:17.785 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.049952+0000', 'last_change': '2024-04-25T09:04:13.049952+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.049952+0000', 'last_undegraded': '2024-04-25T09:04:13.049952+0000', 'last_fullsized': '2024-04-25T09:04:13.049952+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:04:17.785 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.37 is not active+clean 2024-04-25T09:04:17.785 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.047013+0000', 'last_change': '2024-04-25T09:04:13.047013+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.047013+0000', 'last_undegraded': '2024-04-25T09:04:13.047013+0000', 'last_fullsized': '2024-04-25T09:04:13.047013+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:04:17.785 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.36 is not active+clean 2024-04-25T09:04:17.785 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.052404+0000', 'last_change': '2024-04-25T09:04:13.052404+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.052404+0000', 'last_undegraded': '2024-04-25T09:04:13.052404+0000', 'last_fullsized': '2024-04-25T09:04:13.052404+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:04:17.785 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.35 is not active+clean 2024-04-25T09:04:17.785 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.050451+0000', 'last_change': '2024-04-25T09:04:13.050451+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.050451+0000', 'last_undegraded': '2024-04-25T09:04:13.050451+0000', 'last_fullsized': '2024-04-25T09:04:13.050451+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:04:17.785 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.34 is not active+clean 2024-04-25T09:04:17.785 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.054302+0000', 'last_change': '2024-04-25T09:04:13.054302+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.054302+0000', 'last_undegraded': '2024-04-25T09:04:13.054302+0000', 'last_fullsized': '2024-04-25T09:04:13.054302+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:04:17.786 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.33 is not active+clean 2024-04-25T09:04:17.786 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.047149+0000', 'last_change': '2024-04-25T09:04:13.047149+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.047149+0000', 'last_undegraded': '2024-04-25T09:04:13.047149+0000', 'last_fullsized': '2024-04-25T09:04:13.047149+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:04:17.786 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.32 is not active+clean 2024-04-25T09:04:17.786 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.048543+0000', 'last_change': '2024-04-25T09:04:13.048543+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.048543+0000', 'last_undegraded': '2024-04-25T09:04:13.048543+0000', 'last_fullsized': '2024-04-25T09:04:13.048543+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:04:17.786 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.31 is not active+clean 2024-04-25T09:04:17.786 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.050737+0000', 'last_change': '2024-04-25T09:04:13.050737+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.050737+0000', 'last_undegraded': '2024-04-25T09:04:13.050737+0000', 'last_fullsized': '2024-04-25T09:04:13.050737+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:04:17.786 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.30 is not active+clean 2024-04-25T09:04:17.786 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.049349+0000', 'last_change': '2024-04-25T09:04:13.049349+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.049349+0000', 'last_undegraded': '2024-04-25T09:04:13.049349+0000', 'last_fullsized': '2024-04-25T09:04:13.049349+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:04:17.786 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2f is not active+clean 2024-04-25T09:04:17.786 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.051581+0000', 'last_change': '2024-04-25T09:04:13.051581+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.051581+0000', 'last_undegraded': '2024-04-25T09:04:13.051581+0000', 'last_fullsized': '2024-04-25T09:04:13.051581+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:04:17.786 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2e is not active+clean 2024-04-25T09:04:17.787 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.052944+0000', 'last_change': '2024-04-25T09:04:13.052944+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.052944+0000', 'last_undegraded': '2024-04-25T09:04:13.052944+0000', 'last_fullsized': '2024-04-25T09:04:13.052944+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:04:17.787 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2d is not active+clean 2024-04-25T09:04:17.787 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.049189+0000', 'last_change': '2024-04-25T09:04:13.049189+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.049189+0000', 'last_undegraded': '2024-04-25T09:04:13.049189+0000', 'last_fullsized': '2024-04-25T09:04:13.049189+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:04:17.787 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2c is not active+clean 2024-04-25T09:04:17.787 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.046642+0000', 'last_change': '2024-04-25T09:04:13.046642+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.046642+0000', 'last_undegraded': '2024-04-25T09:04:13.046642+0000', 'last_fullsized': '2024-04-25T09:04:13.046642+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:04:17.787 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2b is not active+clean 2024-04-25T09:04:17.787 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.051136+0000', 'last_change': '2024-04-25T09:04:13.051136+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.051136+0000', 'last_undegraded': '2024-04-25T09:04:13.051136+0000', 'last_fullsized': '2024-04-25T09:04:13.051136+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:04:17.787 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2a is not active+clean 2024-04-25T09:04:17.787 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.048012+0000', 'last_change': '2024-04-25T09:04:13.048012+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.048012+0000', 'last_undegraded': '2024-04-25T09:04:13.048012+0000', 'last_fullsized': '2024-04-25T09:04:13.048012+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:04:17.787 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.29 is not active+clean 2024-04-25T09:04:17.787 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.046277+0000', 'last_change': '2024-04-25T09:04:13.046277+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.046277+0000', 'last_undegraded': '2024-04-25T09:04:13.046277+0000', 'last_fullsized': '2024-04-25T09:04:13.046277+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:04:17.788 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.28 is not active+clean 2024-04-25T09:04:17.788 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.051677+0000', 'last_change': '2024-04-25T09:04:13.051677+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.051677+0000', 'last_undegraded': '2024-04-25T09:04:13.051677+0000', 'last_fullsized': '2024-04-25T09:04:13.051677+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:04:17.788 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.27 is not active+clean 2024-04-25T09:04:17.788 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.052436+0000', 'last_change': '2024-04-25T09:04:13.052436+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.052436+0000', 'last_undegraded': '2024-04-25T09:04:13.052436+0000', 'last_fullsized': '2024-04-25T09:04:13.052436+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:04:17.788 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.26 is not active+clean 2024-04-25T09:04:17.788 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.049685+0000', 'last_change': '2024-04-25T09:04:13.049685+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.049685+0000', 'last_undegraded': '2024-04-25T09:04:13.049685+0000', 'last_fullsized': '2024-04-25T09:04:13.049685+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:04:17.788 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.25 is not active+clean 2024-04-25T09:04:17.788 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.048115+0000', 'last_change': '2024-04-25T09:04:13.048115+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.048115+0000', 'last_undegraded': '2024-04-25T09:04:13.048115+0000', 'last_fullsized': '2024-04-25T09:04:13.048115+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:04:17.788 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.24 is not active+clean 2024-04-25T09:04:17.788 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.046108+0000', 'last_change': '2024-04-25T09:04:13.046108+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.046108+0000', 'last_undegraded': '2024-04-25T09:04:13.046108+0000', 'last_fullsized': '2024-04-25T09:04:13.046108+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:04:17.789 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.23 is not active+clean 2024-04-25T09:04:17.789 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.048479+0000', 'last_change': '2024-04-25T09:04:13.048479+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.048479+0000', 'last_undegraded': '2024-04-25T09:04:13.048479+0000', 'last_fullsized': '2024-04-25T09:04:13.048479+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:04:17.789 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.22 is not active+clean 2024-04-25T09:04:17.789 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.047436+0000', 'last_change': '2024-04-25T09:04:13.047436+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.047436+0000', 'last_undegraded': '2024-04-25T09:04:13.047436+0000', 'last_fullsized': '2024-04-25T09:04:13.047436+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:04:17.789 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.21 is not active+clean 2024-04-25T09:04:17.789 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.046485+0000', 'last_change': '2024-04-25T09:04:13.046485+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.046485+0000', 'last_undegraded': '2024-04-25T09:04:13.046485+0000', 'last_fullsized': '2024-04-25T09:04:13.046485+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:04:17.790 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.20 is not active+clean 2024-04-25T09:04:17.790 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.048912+0000', 'last_change': '2024-04-25T09:04:13.048912+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.048912+0000', 'last_undegraded': '2024-04-25T09:04:13.048912+0000', 'last_fullsized': '2024-04-25T09:04:13.048912+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:04:17.790 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1e is not active+clean 2024-04-25T09:04:17.790 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.051989+0000', 'last_change': '2024-04-25T09:04:13.051989+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.051989+0000', 'last_undegraded': '2024-04-25T09:04:13.051989+0000', 'last_fullsized': '2024-04-25T09:04:13.051989+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:04:17.790 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1f is not active+clean 2024-04-25T09:04:17.790 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.046457+0000', 'last_change': '2024-04-25T09:04:13.046457+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.046457+0000', 'last_undegraded': '2024-04-25T09:04:13.046457+0000', 'last_fullsized': '2024-04-25T09:04:13.046457+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:04:17.790 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1c is not active+clean 2024-04-25T09:04:17.790 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.048044+0000', 'last_change': '2024-04-25T09:04:13.048044+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.048044+0000', 'last_undegraded': '2024-04-25T09:04:13.048044+0000', 'last_fullsized': '2024-04-25T09:04:13.048044+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:04:17.790 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1d is not active+clean 2024-04-25T09:04:17.790 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.053963+0000', 'last_change': '2024-04-25T09:04:13.053963+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.053963+0000', 'last_undegraded': '2024-04-25T09:04:13.053963+0000', 'last_fullsized': '2024-04-25T09:04:13.053963+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:04:17.790 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1a is not active+clean 2024-04-25T09:04:17.791 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.046085+0000', 'last_change': '2024-04-25T09:04:13.046085+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.046085+0000', 'last_undegraded': '2024-04-25T09:04:13.046085+0000', 'last_fullsized': '2024-04-25T09:04:13.046085+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:04:17.791 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1b is not active+clean 2024-04-25T09:04:17.791 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.046155+0000', 'last_change': '2024-04-25T09:04:13.046155+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.046155+0000', 'last_undegraded': '2024-04-25T09:04:13.046155+0000', 'last_fullsized': '2024-04-25T09:04:13.046155+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:04:17.791 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.18 is not active+clean 2024-04-25T09:04:17.791 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.046021+0000', 'last_change': '2024-04-25T09:04:13.046021+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.046021+0000', 'last_undegraded': '2024-04-25T09:04:13.046021+0000', 'last_fullsized': '2024-04-25T09:04:13.046021+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:04:17.791 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.19 is not active+clean 2024-04-25T09:04:17.791 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.045906+0000', 'last_change': '2024-04-25T09:04:13.045906+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.045906+0000', 'last_undegraded': '2024-04-25T09:04:13.045906+0000', 'last_fullsized': '2024-04-25T09:04:13.045906+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:04:17.791 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.16 is not active+clean 2024-04-25T09:04:17.791 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.045952+0000', 'last_change': '2024-04-25T09:04:13.045952+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.045952+0000', 'last_undegraded': '2024-04-25T09:04:13.045952+0000', 'last_fullsized': '2024-04-25T09:04:13.045952+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:04:17.791 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.17 is not active+clean 2024-04-25T09:04:17.791 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.052044+0000', 'last_change': '2024-04-25T09:04:13.052044+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.052044+0000', 'last_undegraded': '2024-04-25T09:04:13.052044+0000', 'last_fullsized': '2024-04-25T09:04:13.052044+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:04:17.792 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.15 is not active+clean 2024-04-25T09:04:17.792 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.047591+0000', 'last_change': '2024-04-25T09:04:13.047591+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.047591+0000', 'last_undegraded': '2024-04-25T09:04:13.047591+0000', 'last_fullsized': '2024-04-25T09:04:13.047591+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:04:17.792 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.12 is not active+clean 2024-04-25T09:04:17.792 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.049288+0000', 'last_change': '2024-04-25T09:04:13.049288+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.049288+0000', 'last_undegraded': '2024-04-25T09:04:13.049288+0000', 'last_fullsized': '2024-04-25T09:04:13.049288+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:04:17.792 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.13 is not active+clean 2024-04-25T09:04:17.792 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.045500+0000', 'last_change': '2024-04-25T09:04:13.045500+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.045500+0000', 'last_undegraded': '2024-04-25T09:04:13.045500+0000', 'last_fullsized': '2024-04-25T09:04:13.045500+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:04:17.792 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.10 is not active+clean 2024-04-25T09:04:17.792 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.045249+0000', 'last_change': '2024-04-25T09:04:13.045249+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.045249+0000', 'last_undegraded': '2024-04-25T09:04:13.045249+0000', 'last_fullsized': '2024-04-25T09:04:13.045249+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:04:17.792 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.11 is not active+clean 2024-04-25T09:04:17.792 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.049874+0000', 'last_change': '2024-04-25T09:04:13.049874+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.049874+0000', 'last_undegraded': '2024-04-25T09:04:13.049874+0000', 'last_fullsized': '2024-04-25T09:04:13.049874+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:04:17.792 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.e is not active+clean 2024-04-25T09:04:17.793 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.051480+0000', 'last_change': '2024-04-25T09:04:13.051480+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.051480+0000', 'last_undegraded': '2024-04-25T09:04:13.051480+0000', 'last_fullsized': '2024-04-25T09:04:13.051480+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:04:17.793 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.f is not active+clean 2024-04-25T09:04:17.793 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.051515+0000', 'last_change': '2024-04-25T09:04:13.051515+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.051515+0000', 'last_undegraded': '2024-04-25T09:04:13.051515+0000', 'last_fullsized': '2024-04-25T09:04:13.051515+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:04:17.793 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.c is not active+clean 2024-04-25T09:04:17.793 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.050011+0000', 'last_change': '2024-04-25T09:04:13.050011+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.050011+0000', 'last_undegraded': '2024-04-25T09:04:13.050011+0000', 'last_fullsized': '2024-04-25T09:04:13.050011+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:04:17.793 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.d is not active+clean 2024-04-25T09:04:17.793 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.048188+0000', 'last_change': '2024-04-25T09:04:13.048188+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.048188+0000', 'last_undegraded': '2024-04-25T09:04:13.048188+0000', 'last_fullsized': '2024-04-25T09:04:13.048188+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:04:17.793 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.a is not active+clean 2024-04-25T09:04:17.793 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.046887+0000', 'last_change': '2024-04-25T09:04:13.046887+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.046887+0000', 'last_undegraded': '2024-04-25T09:04:13.046887+0000', 'last_fullsized': '2024-04-25T09:04:13.046887+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:04:17.793 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.b is not active+clean 2024-04-25T09:04:17.793 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.046289+0000', 'last_change': '2024-04-25T09:04:13.046289+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.046289+0000', 'last_undegraded': '2024-04-25T09:04:13.046289+0000', 'last_fullsized': '2024-04-25T09:04:13.046289+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:04:17.794 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.8 is not active+clean 2024-04-25T09:04:17.794 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.045727+0000', 'last_change': '2024-04-25T09:04:13.045727+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.045727+0000', 'last_undegraded': '2024-04-25T09:04:13.045727+0000', 'last_fullsized': '2024-04-25T09:04:13.045727+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T09:04:17.794 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.9 is not active+clean 2024-04-25T09:04:17.794 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.045307+0000', 'last_change': '2024-04-25T09:04:13.045307+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.045307+0000', 'last_undegraded': '2024-04-25T09:04:13.045307+0000', 'last_fullsized': '2024-04-25T09:04:13.045307+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:04:17.794 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.6 is not active+clean 2024-04-25T09:04:17.794 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.045467+0000', 'last_change': '2024-04-25T09:04:13.045467+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.045467+0000', 'last_undegraded': '2024-04-25T09:04:13.045467+0000', 'last_fullsized': '2024-04-25T09:04:13.045467+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:04:17.794 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.7 is not active+clean 2024-04-25T09:04:17.794 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.050690+0000', 'last_change': '2024-04-25T09:04:13.050690+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.050690+0000', 'last_undegraded': '2024-04-25T09:04:13.050690+0000', 'last_fullsized': '2024-04-25T09:04:13.050690+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T09:04:17.795 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.4 is not active+clean 2024-04-25T09:04:17.795 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.044936+0000', 'last_change': '2024-04-25T09:04:13.044936+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.044936+0000', 'last_undegraded': '2024-04-25T09:04:13.044936+0000', 'last_fullsized': '2024-04-25T09:04:13.044936+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T09:04:17.795 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.5 is not active+clean 2024-04-25T09:04:17.795 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.047647+0000', 'last_change': '2024-04-25T09:04:13.047647+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.047647+0000', 'last_undegraded': '2024-04-25T09:04:13.047647+0000', 'last_fullsized': '2024-04-25T09:04:13.047647+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T09:04:17.795 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2 is not active+clean 2024-04-25T09:04:17.795 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.047830+0000', 'last_change': '2024-04-25T09:04:13.047830+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.047830+0000', 'last_undegraded': '2024-04-25T09:04:13.047830+0000', 'last_fullsized': '2024-04-25T09:04:13.047830+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T09:04:17.795 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3 is not active+clean 2024-04-25T09:04:17.795 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.046670+0000', 'last_change': '2024-04-25T09:04:13.046670+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.046670+0000', 'last_undegraded': '2024-04-25T09:04:13.046670+0000', 'last_fullsized': '2024-04-25T09:04:13.046670+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:04:17.795 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.0 is not active+clean 2024-04-25T09:04:17.795 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.050769+0000', 'last_change': '2024-04-25T09:04:13.050769+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.050769+0000', 'last_undegraded': '2024-04-25T09:04:13.050769+0000', 'last_fullsized': '2024-04-25T09:04:13.050769+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T09:04:17.795 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1 is not active+clean 2024-04-25T09:04:17.796 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.045284+0000', 'last_change': '2024-04-25T09:04:13.045284+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.045284+0000', 'last_undegraded': '2024-04-25T09:04:13.045284+0000', 'last_fullsized': '2024-04-25T09:04:13.045284+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T09:04:17.796 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.14 is not active+clean 2024-04-25T09:04:17.796 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 137, 'state': 'creating+peering', 'last_fresh': '2024-04-25T09:04:13.045651+0000', 'last_change': '2024-04-25T09:04:13.045651+0000', 'last_active': '2024-04-25T09:04:13.034365+0000', 'last_peered': '2024-04-25T09:04:13.034365+0000', 'last_clean': '2024-04-25T09:04:13.034365+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T09:04:13.045651+0000', 'last_undegraded': '2024-04-25T09:04:13.045651+0000', 'last_fullsized': '2024-04-25T09:04:13.045651+0000', 'mapping_epoch': 137, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 137, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'last_clean_scrub_stamp': '2024-04-25T09:04:13.034365+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T09:04:17.796 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T09:04:17.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.909+0000 7f1a6f7ce640 1 Processor -- start 2024-04-25T09:04:17.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.909+0000 7f1a6f7ce640 1 -- start start 2024-04-25T09:04:17.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.910+0000 7f1a6f7ce640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f1a6806d9a0 0x7f1a68071a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:17.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.910+0000 7f1a6f7ce640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1a68071f90 0x7f1a680cd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:17.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.910+0000 7f1a6f7ce640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a680723e0 con 0x7f1a6815a9d0 2024-04-25T09:04:17.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.910+0000 7f1a6f7ce640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a680704f0 con 0x7f1a68071f90 2024-04-25T09:04:17.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.910+0000 7f1a6f7ce640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a680707d0 con 0x7f1a6806d9a0 2024-04-25T09:04:17.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.910+0000 7f1a6dd6b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f1a68071f90 0x7f1a680cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:17.899 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.910+0000 7f1a6d56a640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f1a6815a9d0 0x7f1a6806d290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:45736/0 (socket says 172.21.15.43:45736) 2024-04-25T09:04:17.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.910+0000 7f1a6d56a640 1 -- 172.21.15.43:0/2601182420 learned_addr learned my addr 172.21.15.43:0/2601182420 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:17.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.910+0000 7f1a6cd69640 1 --2- 172.21.15.43:0/2601182420 >> v2:172.21.15.193:3300/0 conn(0x7f1a6806d9a0 0x7f1a68071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:17.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.910+0000 7f1a6dd6b640 1 -- 172.21.15.43:0/2601182420 >> v2:172.21.15.193:3300/0 conn(0x7f1a6806d9a0 msgr2=0x7f1a68071a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:17.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.910+0000 7f1a6dd6b640 1 --2- 172.21.15.43:0/2601182420 >> v2:172.21.15.193:3300/0 conn(0x7f1a6806d9a0 0x7f1a68071a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.910+0000 7f1a6dd6b640 1 -- 172.21.15.43:0/2601182420 >> v1:172.21.15.43:6789/0 conn(0x7f1a6815a9d0 legacy=0x7f1a6806d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:17.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.910+0000 7f1a6dd6b640 1 -- 172.21.15.43:0/2601182420 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a68070ab0 con 0x7f1a68071f90 2024-04-25T09:04:17.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.910+0000 7f1a67fff640 1 -- 172.21.15.43:0/2601182420 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 892560366 0 0) 0x7f1a680723e0 con 0x7f1a6815a9d0 2024-04-25T09:04:17.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.911+0000 7f1a6dd6b640 1 --2- 172.21.15.43:0/2601182420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a68071f90 0x7f1a680cd940 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f1a6806baf0 tx=0x7f1a6002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=febc2082e708464c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:17.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.911+0000 7f1a67fff640 1 -- 172.21.15.43:0/2601182420 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a60037070 con 0x7f1a68071f90 2024-04-25T09:04:17.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.911+0000 7f1a67fff640 1 -- 172.21.15.43:0/2601182420 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a60032440 con 0x7f1a68071f90 2024-04-25T09:04:17.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.911+0000 7f1a67fff640 1 -- 172.21.15.43:0/2601182420 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a60032810 con 0x7f1a68071f90 2024-04-25T09:04:17.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.911+0000 7f1a6f7ce640 1 -- 172.21.15.43:0/2601182420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a68071f90 msgr2=0x7f1a680cd940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:17.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.911+0000 7f1a6f7ce640 1 --2- 172.21.15.43:0/2601182420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a68071f90 0x7f1a680cd940 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f1a6806baf0 tx=0x7f1a6002a1a0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.911+0000 7f1a6f7ce640 1 -- 172.21.15.43:0/2601182420 shutdown_connections 2024-04-25T09:04:17.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.911+0000 7f1a6f7ce640 1 --2- 172.21.15.43:0/2601182420 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a68071f90 0x7f1a680cd940 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.911+0000 7f1a6f7ce640 1 --2- 172.21.15.43:0/2601182420 >> v2:172.21.15.193:3300/0 conn(0x7f1a6806d9a0 0x7f1a68071a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.911+0000 7f1a6f7ce640 1 -- 172.21.15.43:0/2601182420 >> 172.21.15.43:0/2601182420 conn(0x7f1a680b1480 msgr2=0x7f1a680b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:17.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.911+0000 7f1a6f7ce640 1 -- 172.21.15.43:0/2601182420 shutdown_connections 2024-04-25T09:04:17.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.911+0000 7f1a6f7ce640 1 -- 172.21.15.43:0/2601182420 wait complete. 2024-04-25T09:04:17.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.913+0000 7f1a6f7ce640 1 Processor -- start 2024-04-25T09:04:17.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.913+0000 7f1a6f7ce640 1 -- start start 2024-04-25T09:04:17.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.913+0000 7f1a6f7ce640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a6806d9a0 0x7f1a681452a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:17.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.913+0000 7f1a6f7ce640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a68071f90 0x7f1a680b2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:17.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.913+0000 7f1a6f7ce640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1a680b2dc0 0x7f1a681e2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:17.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.913+0000 7f1a6f7ce640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a680cde80 con 0x7f1a6806d9a0 2024-04-25T09:04:17.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.913+0000 7f1a6f7ce640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a680712b0 con 0x7f1a68071f90 2024-04-25T09:04:17.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.913+0000 7f1a6f7ce640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a680ce000 con 0x7f1a680b2dc0 2024-04-25T09:04:17.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.913+0000 7f1a6d56a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a6806d9a0 0x7f1a681452a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:17.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.913+0000 7f1a6d56a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a6806d9a0 0x7f1a681452a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47668/0 (socket says 172.21.15.43:47668) 2024-04-25T09:04:17.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.914+0000 7f1a6d56a640 1 -- 172.21.15.43:0/993149098 learned_addr learned my addr 172.21.15.43:0/993149098 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:17.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.914+0000 7f1a6cd69640 1 --2- 172.21.15.43:0/993149098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a68071f90 0x7f1a680b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:17.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.914+0000 7f1a6dd6b640 1 --2- 172.21.15.43:0/993149098 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1a680b2dc0 0x7f1a681e2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:17.904 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.914+0000 7f1a6d56a640 1 -- 172.21.15.43:0/993149098 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1a680b2dc0 msgr2=0x7f1a681e2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:17.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.914+0000 7f1a6d56a640 1 --2- 172.21.15.43:0/993149098 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1a680b2dc0 0x7f1a681e2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.914+0000 7f1a6d56a640 1 -- 172.21.15.43:0/993149098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a68071f90 msgr2=0x7f1a680b2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:17.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.914+0000 7f1a6d56a640 1 --2- 172.21.15.43:0/993149098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a68071f90 0x7f1a680b2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:17.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.914+0000 7f1a6d56a640 1 -- 172.21.15.43:0/993149098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a68148c90 con 0x7f1a6806d9a0 2024-04-25T09:04:17.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.914+0000 7f1a6d56a640 1 --2- 172.21.15.43:0/993149098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a6806d9a0 0x7f1a681452a0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800c7f0 tx=0x7f1a5800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:17.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.914+0000 7f1a65ffb640 1 -- 172.21.15.43:0/993149098 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a58018070 con 0x7f1a6806d9a0 2024-04-25T09:04:17.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.914+0000 7f1a65ffb640 1 -- 172.21.15.43:0/993149098 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a5800a640 con 0x7f1a6806d9a0 2024-04-25T09:04:17.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.914+0000 7f1a6f7ce640 1 -- 172.21.15.43:0/993149098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a681e3b60 con 0x7f1a6806d9a0 2024-04-25T09:04:17.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.914+0000 7f1a65ffb640 1 -- 172.21.15.43:0/993149098 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5800aa30 con 0x7f1a6806d9a0 2024-04-25T09:04:17.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.914+0000 7f1a6f7ce640 1 -- 172.21.15.43:0/993149098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a68146000 con 0x7f1a6806d9a0 2024-04-25T09:04:17.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.915+0000 7f1a6f7ce640 1 -- 172.21.15.43:0/993149098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a6806dfd0 con 0x7f1a6806d9a0 2024-04-25T09:04:17.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.916+0000 7f1a65ffb640 1 -- 172.21.15.43:0/993149098 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1a5801b070 con 0x7f1a6806d9a0 2024-04-25T09:04:17.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.916+0000 7f1a65ffb640 1 --2- 172.21.15.43:0/993149098 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1a5406b500 0x7f1a5406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:17.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.916+0000 7f1a65ffb640 1 -- 172.21.15.43:0/993149098 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1a58089760 con 0x7f1a6806d9a0 2024-04-25T09:04:17.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.917+0000 7f1a6cd69640 1 --2- 172.21.15.43:0/993149098 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1a5406b500 0x7f1a5406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:17.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.917+0000 7f1a6cd69640 1 --2- 172.21.15.43:0/993149098 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1a5406b500 0x7f1a5406d9c0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f1a680cbbc0 tx=0x7f1a5c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:17.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:17.920+0000 7f1a65ffb640 1 -- 172.21.15.43:0/993149098 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1a6806dfd0 con 0x7f1a6806d9a0 2024-04-25T09:04:18.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.172+0000 7f1a6f7ce640 1 -- 172.21.15.43:0/993149098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f1a680b31d0 con 0x7f1a6806d9a0 2024-04-25T09:04:18.161 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.173+0000 7f1a65ffb640 1 -- 172.21.15.43:0/993149098 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1328 (secure 0 0 0) 0x7f1a680b31d0 con 0x7f1a6806d9a0 2024-04-25T09:04:18.161 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:04:18.161 INFO:teuthology.orchestra.run.smithi043.stdout:{"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":630,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":139,"num_osds":8,"num_up_osds":8,"osd_up_since":1714035238,"num_in_osds":8,"osd_in_since":1714035229,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":510623744,"bytes_avail":772583489536,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":87,"id":10,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":10,"rank":0,"name":"b","status":"up:active","gid":9732}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":26,"modified":"2024-04-25T09:04:09.834413+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T09:04:18.163 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.175+0000 7f1a6f7ce640 1 -- 172.21.15.43:0/993149098 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1a5406b500 msgr2=0x7f1a5406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:18.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.175+0000 7f1a6f7ce640 1 --2- 172.21.15.43:0/993149098 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1a5406b500 0x7f1a5406d9c0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f1a680cbbc0 tx=0x7f1a5c008040 comp rx=0 tx=0).stop 2024-04-25T09:04:18.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.175+0000 7f1a6f7ce640 1 -- 172.21.15.43:0/993149098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a6806d9a0 msgr2=0x7f1a681452a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:18.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.175+0000 7f1a6f7ce640 1 --2- 172.21.15.43:0/993149098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a6806d9a0 0x7f1a681452a0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f1a5800c7f0 tx=0x7f1a5800ccb0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.175+0000 7f1a6f7ce640 1 -- 172.21.15.43:0/993149098 shutdown_connections 2024-04-25T09:04:18.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.175+0000 7f1a6f7ce640 1 --2- 172.21.15.43:0/993149098 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1a680b2dc0 0x7f1a681e2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.175+0000 7f1a6f7ce640 1 --2- 172.21.15.43:0/993149098 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f1a5406b500 0x7f1a5406d9c0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.175+0000 7f1a6f7ce640 1 --2- 172.21.15.43:0/993149098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1a68071f90 0x7f1a680b2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.175+0000 7f1a6f7ce640 1 --2- 172.21.15.43:0/993149098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f1a6806d9a0 0x7f1a681452a0 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.175+0000 7f1a6f7ce640 1 -- 172.21.15.43:0/993149098 >> 172.21.15.43:0/993149098 conn(0x7f1a680b1480 msgr2=0x7f1a680b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:18.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.175+0000 7f1a6f7ce640 1 -- 172.21.15.43:0/993149098 shutdown_connections 2024-04-25T09:04:18.164 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.175+0000 7f1a6f7ce640 1 -- 172.21.15.43:0/993149098 wait complete. 2024-04-25T09:04:18.175 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T09:04:18.175 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:04:18.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.289+0000 7f96ec09a640 1 Processor -- start 2024-04-25T09:04:18.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.289+0000 7f96ec09a640 1 -- start start 2024-04-25T09:04:18.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.290+0000 7f96ec09a640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f96e4161390 0x7f96e40b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:18.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.290+0000 7f96ec09a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f96e40cecf0 0x7f96e40d0720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:18.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.290+0000 7f96ec09a640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96e40b4c30 con 0x7f96e40d02a0 2024-04-25T09:04:18.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.290+0000 7f96ec09a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96e40d1910 con 0x7f96e40cecf0 2024-04-25T09:04:18.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.290+0000 7f96ec09a640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96e40cf2c0 con 0x7f96e4161390 2024-04-25T09:04:18.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.290+0000 7f96ea637640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f96e40cecf0 0x7f96e40d0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:18.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.290+0000 7f96ea637640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f96e40cecf0 0x7f96e40d0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47680/0 (socket says 172.21.15.43:47680) 2024-04-25T09:04:18.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.290+0000 7f96ea637640 1 -- 172.21.15.43:0/2939383510 learned_addr learned my addr 172.21.15.43:0/2939383510 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:18.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.291+0000 7f96ea637640 1 -- 172.21.15.43:0/2939383510 >> v2:172.21.15.193:3300/0 conn(0x7f96e4161390 msgr2=0x7f96e40b6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:18.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.291+0000 7f96ea637640 1 --2- 172.21.15.43:0/2939383510 >> v2:172.21.15.193:3300/0 conn(0x7f96e4161390 0x7f96e40b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.291+0000 7f96ea637640 1 -- 172.21.15.43:0/2939383510 >> v1:172.21.15.193:6789/0 conn(0x7f96e40d02a0 legacy=0x7f96e40ce5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:18.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.291+0000 7f96ea637640 1 -- 172.21.15.43:0/2939383510 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e40cf5a0 con 0x7f96e40cecf0 2024-04-25T09:04:18.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.291+0000 7f96ea637640 1 --2- 172.21.15.43:0/2939383510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96e40cecf0 0x7f96e40d0720 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f96e40c74d0 tx=0x7f96dc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f04715d654bcdc1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:18.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.291+0000 7f96e8e34640 1 -- 172.21.15.43:0/2939383510 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96dc036070 con 0x7f96e40cecf0 2024-04-25T09:04:18.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.291+0000 7f96e8e34640 1 -- 172.21.15.43:0/2939383510 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96dc031600 con 0x7f96e40cecf0 2024-04-25T09:04:18.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.291+0000 7f96e8e34640 1 -- 172.21.15.43:0/2939383510 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96dc0319d0 con 0x7f96e40cecf0 2024-04-25T09:04:18.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.292+0000 7f96ec09a640 1 -- 172.21.15.43:0/2939383510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96e40cecf0 msgr2=0x7f96e40d0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:18.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.292+0000 7f96ec09a640 1 --2- 172.21.15.43:0/2939383510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96e40cecf0 0x7f96e40d0720 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f96e40c74d0 tx=0x7f96dc02a6f0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.292+0000 7f96ec09a640 1 -- 172.21.15.43:0/2939383510 shutdown_connections 2024-04-25T09:04:18.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.292+0000 7f96ec09a640 1 --2- 172.21.15.43:0/2939383510 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96e40cecf0 0x7f96e40d0720 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.292+0000 7f96ec09a640 1 --2- 172.21.15.43:0/2939383510 >> v2:172.21.15.193:3300/0 conn(0x7f96e4161390 0x7f96e40b6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.292+0000 7f96ec09a640 1 -- 172.21.15.43:0/2939383510 >> 172.21.15.43:0/2939383510 conn(0x7f96e409cd20 msgr2=0x7f96e40d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:18.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.292+0000 7f96ec09a640 1 -- 172.21.15.43:0/2939383510 shutdown_connections 2024-04-25T09:04:18.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.292+0000 7f96ec09a640 1 -- 172.21.15.43:0/2939383510 wait complete. 2024-04-25T09:04:18.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.293+0000 7f96ec09a640 1 Processor -- start 2024-04-25T09:04:18.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.293+0000 7f96ec09a640 1 -- start start 2024-04-25T09:04:18.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.294+0000 7f96ec09a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96e40cecf0 0x7f96e41f6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:18.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.294+0000 7f96ec09a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96e40d02a0 0x7f96e41f4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:18.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.294+0000 7f96ec09a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f96e4161390 0x7f96e4159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:18.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.294+0000 7f96ec09a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e4161770 con 0x7f96e40cecf0 2024-04-25T09:04:18.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.294+0000 7f96ec09a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e40d15a0 con 0x7f96e40d02a0 2024-04-25T09:04:18.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.294+0000 7f96ec09a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96e40d1da0 con 0x7f96e4161390 2024-04-25T09:04:18.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.294+0000 7f96e9e36640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96e40cecf0 0x7f96e41f6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:18.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.294+0000 7f96e9e36640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96e40cecf0 0x7f96e41f6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:47688/0 (socket says 172.21.15.43:47688) 2024-04-25T09:04:18.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.294+0000 7f96e9e36640 1 -- 172.21.15.43:0/82962712 learned_addr learned my addr 172.21.15.43:0/82962712 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:18.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.294+0000 7f96e9e36640 1 -- 172.21.15.43:0/82962712 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f96e4161390 msgr2=0x7f96e4159c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:18.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.294+0000 7f96e9e36640 1 --2- 172.21.15.43:0/82962712 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f96e4161390 0x7f96e4159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.294+0000 7f96e9e36640 1 -- 172.21.15.43:0/82962712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96e40d02a0 msgr2=0x7f96e41f4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:18.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.294+0000 7f96e9635640 1 --2- 172.21.15.43:0/82962712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96e40d02a0 0x7f96e41f4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:18.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.294+0000 7f96e9e36640 1 --2- 172.21.15.43:0/82962712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96e40d02a0 0x7f96e41f4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.294+0000 7f96e9e36640 1 -- 172.21.15.43:0/82962712 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e40c7220 con 0x7f96e40cecf0 2024-04-25T09:04:18.285 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.294+0000 7f96e9635640 1 --2- 172.21.15.43:0/82962712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96e40d02a0 0x7f96e41f4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:18.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.295+0000 7f96e9e36640 1 --2- 172.21.15.43:0/82962712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96e40cecf0 0x7f96e41f6830 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f96e000b410 tx=0x7f96e000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:18.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.295+0000 7f96da7fc640 1 -- 172.21.15.43:0/82962712 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e0015020 con 0x7f96e40cecf0 2024-04-25T09:04:18.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.295+0000 7f96da7fc640 1 -- 172.21.15.43:0/82962712 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96e0011850 con 0x7f96e40cecf0 2024-04-25T09:04:18.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.295+0000 7f96ec09a640 1 -- 172.21.15.43:0/82962712 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96e40c7500 con 0x7f96e40cecf0 2024-04-25T09:04:18.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.295+0000 7f96da7fc640 1 -- 172.21.15.43:0/82962712 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96e0011c40 con 0x7f96e40cecf0 2024-04-25T09:04:18.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.295+0000 7f96ec09a640 1 -- 172.21.15.43:0/82962712 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96e40c7ac0 con 0x7f96e40cecf0 2024-04-25T09:04:18.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.296+0000 7f96ec09a640 1 -- 172.21.15.43:0/82962712 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96e40680f0 con 0x7f96e40cecf0 2024-04-25T09:04:18.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.296+0000 7f96da7fc640 1 -- 172.21.15.43:0/82962712 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f96e0018020 con 0x7f96e40cecf0 2024-04-25T09:04:18.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.297+0000 7f96da7fc640 1 --2- 172.21.15.43:0/82962712 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f96c806b4d0 0x7f96c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:18.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.297+0000 7f96da7fc640 1 -- 172.21.15.43:0/82962712 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f96e008a160 con 0x7f96e40cecf0 2024-04-25T09:04:18.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.297+0000 7f96e9635640 1 --2- 172.21.15.43:0/82962712 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f96c806b4d0 0x7f96c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:18.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.298+0000 7f96e9635640 1 --2- 172.21.15.43:0/82962712 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f96c806b4d0 0x7f96c806d990 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f96d4007fd0 tx=0x7f96d4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:18.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.301+0000 7f96da7fc640 1 -- 172.21.15.43:0/82962712 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f96e001f020 con 0x7f96e40cecf0 2024-04-25T09:04:18.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.478+0000 7f96ec09a640 1 -- 172.21.15.43:0/82962712 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f96e4068e90 con 0x7f96c806b4d0 2024-04-25T09:04:18.473 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.484+0000 7f96da7fc640 1 -- 172.21.15.43:0/82962712 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247089 (secure 0 0 0) 0x7f96e4068e90 con 0x7f96c806b4d0 2024-04-25T09:04:18.476 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:04:18.477 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-25T09:04:18.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.488+0000 7f96ec09a640 1 -- 172.21.15.43:0/82962712 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f96c806b4d0 msgr2=0x7f96c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:18.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.488+0000 7f96ec09a640 1 --2- 172.21.15.43:0/82962712 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f96c806b4d0 0x7f96c806d990 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f96d4007fd0 tx=0x7f96d4009040 comp rx=0 tx=0).stop 2024-04-25T09:04:18.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.488+0000 7f96ec09a640 1 -- 172.21.15.43:0/82962712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96e40cecf0 msgr2=0x7f96e41f6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:18.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.488+0000 7f96ec09a640 1 --2- 172.21.15.43:0/82962712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96e40cecf0 0x7f96e41f6830 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f96e000b410 tx=0x7f96e000b8d0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.488+0000 7f96ec09a640 1 -- 172.21.15.43:0/82962712 shutdown_connections 2024-04-25T09:04:18.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.488+0000 7f96ec09a640 1 --2- 172.21.15.43:0/82962712 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f96e4161390 0x7f96e4159c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.488+0000 7f96ec09a640 1 --2- 172.21.15.43:0/82962712 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f96c806b4d0 0x7f96c806d990 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.488+0000 7f96ec09a640 1 --2- 172.21.15.43:0/82962712 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f96e40d02a0 0x7f96e41f4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.488+0000 7f96ec09a640 1 --2- 172.21.15.43:0/82962712 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f96e40cecf0 0x7f96e41f6830 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:18.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.488+0000 7f96ec09a640 1 -- 172.21.15.43:0/82962712 >> 172.21.15.43:0/82962712 conn(0x7f96e409cd20 msgr2=0x7f96e406c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:18.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.488+0000 7f96ec09a640 1 -- 172.21.15.43:0/82962712 shutdown_connections 2024-04-25T09:04:18.479 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:18.488+0000 7f96ec09a640 1 -- 172.21.15.43:0/82962712 wait complete. 2024-04-25T09:04:18.487 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":442,"stamp":"2024-04-25T09:04:17.836278+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":498656,"kb_used_data":14568,"kb_used_omap":12,"kb_used_meta":483955,"kb_avail":754476064,"statfs":{"total":773094113280,"available":772583489536,"internally_reserved":0,"allocated":14917632,"data_stored":7178872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":495570481},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.794255"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.053615+0000","last_change":"2024-04-25T09:04:13.053615+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.053615+0000","last_undegraded":"2024-04-25T09:04:13.053615+0000","last_fullsized":"2024-04-25T09:04:13.053615+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.047818+0000","last_change":"2024-04-25T09:04:13.047818+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.047818+0000","last_undegraded":"2024-04-25T09:04:13.047818+0000","last_fullsized":"2024-04-25T09:04:13.047818+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.049520+0000","last_change":"2024-04-25T09:04:13.049520+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.049520+0000","last_undegraded":"2024-04-25T09:04:13.049520+0000","last_fullsized":"2024-04-25T09:04:13.049520+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046148+0000","last_change":"2024-04-25T09:04:13.046148+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046148+0000","last_undegraded":"2024-04-25T09:04:13.046148+0000","last_fullsized":"2024-04-25T09:04:13.046148+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.047544+0000","last_change":"2024-04-25T09:04:13.047544+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.047544+0000","last_undegraded":"2024-04-25T09:04:13.047544+0000","last_fullsized":"2024-04-25T09:04:13.047544+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.051007+0000","last_change":"2024-04-25T09:04:13.051007+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.051007+0000","last_undegraded":"2024-04-25T09:04:13.051007+0000","last_fullsized":"2024-04-25T09:04:13.051007+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.048146+0000","last_change":"2024-04-25T09:04:13.048146+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.048146+0000","last_undegraded":"2024-04-25T09:04:13.048146+0000","last_fullsized":"2024-04-25T09:04:13.048146+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.049952+0000","last_change":"2024-04-25T09:04:13.049952+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.049952+0000","last_undegraded":"2024-04-25T09:04:13.049952+0000","last_fullsized":"2024-04-25T09:04:13.049952+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.047013+0000","last_change":"2024-04-25T09:04:13.047013+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.047013+0000","last_undegraded":"2024-04-25T09:04:13.047013+0000","last_fullsized":"2024-04-25T09:04:13.047013+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.052404+0000","last_change":"2024-04-25T09:04:13.052404+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.052404+0000","last_undegraded":"2024-04-25T09:04:13.052404+0000","last_fullsized":"2024-04-25T09:04:13.052404+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.050451+0000","last_change":"2024-04-25T09:04:13.050451+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.050451+0000","last_undegraded":"2024-04-25T09:04:13.050451+0000","last_fullsized":"2024-04-25T09:04:13.050451+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.054302+0000","last_change":"2024-04-25T09:04:13.054302+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.054302+0000","last_undegraded":"2024-04-25T09:04:13.054302+0000","last_fullsized":"2024-04-25T09:04:13.054302+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.047149+0000","last_change":"2024-04-25T09:04:13.047149+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.047149+0000","last_undegraded":"2024-04-25T09:04:13.047149+0000","last_fullsized":"2024-04-25T09:04:13.047149+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.048543+0000","last_change":"2024-04-25T09:04:13.048543+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.048543+0000","last_undegraded":"2024-04-25T09:04:13.048543+0000","last_fullsized":"2024-04-25T09:04:13.048543+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.050737+0000","last_change":"2024-04-25T09:04:13.050737+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.050737+0000","last_undegraded":"2024-04-25T09:04:13.050737+0000","last_fullsized":"2024-04-25T09:04:13.050737+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.049349+0000","last_change":"2024-04-25T09:04:13.049349+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.049349+0000","last_undegraded":"2024-04-25T09:04:13.049349+0000","last_fullsized":"2024-04-25T09:04:13.049349+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.051581+0000","last_change":"2024-04-25T09:04:13.051581+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.051581+0000","last_undegraded":"2024-04-25T09:04:13.051581+0000","last_fullsized":"2024-04-25T09:04:13.051581+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.052944+0000","last_change":"2024-04-25T09:04:13.052944+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.052944+0000","last_undegraded":"2024-04-25T09:04:13.052944+0000","last_fullsized":"2024-04-25T09:04:13.052944+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.049189+0000","last_change":"2024-04-25T09:04:13.049189+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.049189+0000","last_undegraded":"2024-04-25T09:04:13.049189+0000","last_fullsized":"2024-04-25T09:04:13.049189+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046642+0000","last_change":"2024-04-25T09:04:13.046642+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046642+0000","last_undegraded":"2024-04-25T09:04:13.046642+0000","last_fullsized":"2024-04-25T09:04:13.046642+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.051136+0000","last_change":"2024-04-25T09:04:13.051136+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.051136+0000","last_undegraded":"2024-04-25T09:04:13.051136+0000","last_fullsized":"2024-04-25T09:04:13.051136+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.048012+0000","last_change":"2024-04-25T09:04:13.048012+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.048012+0000","last_undegraded":"2024-04-25T09:04:13.048012+0000","last_fullsized":"2024-04-25T09:04:13.048012+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046277+0000","last_change":"2024-04-25T09:04:13.046277+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046277+0000","last_undegraded":"2024-04-25T09:04:13.046277+0000","last_fullsized":"2024-04-25T09:04:13.046277+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.051677+0000","last_change":"2024-04-25T09:04:13.051677+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.051677+0000","last_undegraded":"2024-04-25T09:04:13.051677+0000","last_fullsized":"2024-04-25T09:04:13.051677+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.052436+0000","last_change":"2024-04-25T09:04:13.052436+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.052436+0000","last_undegraded":"2024-04-25T09:04:13.052436+0000","last_fullsized":"2024-04-25T09:04:13.052436+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.049685+0000","last_change":"2024-04-25T09:04:13.049685+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.049685+0000","last_undegraded":"2024-04-25T09:04:13.049685+0000","last_fullsized":"2024-04-25T09:04:13.049685+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.048115+0000","last_change":"2024-04-25T09:04:13.048115+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.048115+0000","last_undegraded":"2024-04-25T09:04:13.048115+0000","last_fullsized":"2024-04-25T09:04:13.048115+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046108+0000","last_change":"2024-04-25T09:04:13.046108+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046108+0000","last_undegraded":"2024-04-25T09:04:13.046108+0000","last_fullsized":"2024-04-25T09:04:13.046108+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.048479+0000","last_change":"2024-04-25T09:04:13.048479+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.048479+0000","last_undegraded":"2024-04-25T09:04:13.048479+0000","last_fullsized":"2024-04-25T09:04:13.048479+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.047436+0000","last_change":"2024-04-25T09:04:13.047436+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.047436+0000","last_undegraded":"2024-04-25T09:04:13.047436+0000","last_fullsized":"2024-04-25T09:04:13.047436+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046485+0000","last_change":"2024-04-25T09:04:13.046485+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046485+0000","last_undegraded":"2024-04-25T09:04:13.046485+0000","last_fullsized":"2024-04-25T09:04:13.046485+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.048912+0000","last_change":"2024-04-25T09:04:13.048912+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.048912+0000","last_undegraded":"2024-04-25T09:04:13.048912+0000","last_fullsized":"2024-04-25T09:04:13.048912+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.051989+0000","last_change":"2024-04-25T09:04:13.051989+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.051989+0000","last_undegraded":"2024-04-25T09:04:13.051989+0000","last_fullsized":"2024-04-25T09:04:13.051989+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.063563+0000","last_change":"2024-04-25T09:04:13.062156+0000","last_active":"2024-04-25T09:04:13.063563+0000","last_peered":"2024-04-25T09:04:13.063563+0000","last_clean":"2024-04-25T09:04:13.063563+0000","last_became_active":"2024-04-25T09:04:13.061852+0000","last_became_peered":"2024-04-25T09:04:13.061852+0000","last_unstale":"2024-04-25T09:04:13.063563+0000","last_undegraded":"2024-04-25T09:04:13.063563+0000","last_fullsized":"2024-04-25T09:04:13.063563+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:50:10.458178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046457+0000","last_change":"2024-04-25T09:04:13.046457+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046457+0000","last_undegraded":"2024-04-25T09:04:13.046457+0000","last_fullsized":"2024-04-25T09:04:13.046457+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.059517+0000","last_change":"2024-04-25T09:04:13.058762+0000","last_active":"2024-04-25T09:04:13.059517+0000","last_peered":"2024-04-25T09:04:13.059517+0000","last_clean":"2024-04-25T09:04:13.059517+0000","last_became_active":"2024-04-25T09:04:13.058335+0000","last_became_peered":"2024-04-25T09:04:13.058335+0000","last_unstale":"2024-04-25T09:04:13.059517+0000","last_undegraded":"2024-04-25T09:04:13.059517+0000","last_fullsized":"2024-04-25T09:04:13.059517+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:59.348802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.048044+0000","last_change":"2024-04-25T09:04:13.048044+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.048044+0000","last_undegraded":"2024-04-25T09:04:13.048044+0000","last_fullsized":"2024-04-25T09:04:13.048044+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.059899+0000","last_change":"2024-04-25T09:04:13.057742+0000","last_active":"2024-04-25T09:04:13.059899+0000","last_peered":"2024-04-25T09:04:13.059899+0000","last_clean":"2024-04-25T09:04:13.059899+0000","last_became_active":"2024-04-25T09:04:13.057448+0000","last_became_peered":"2024-04-25T09:04:13.057448+0000","last_unstale":"2024-04-25T09:04:13.059899+0000","last_undegraded":"2024-04-25T09:04:13.059899+0000","last_fullsized":"2024-04-25T09:04:13.059899+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:47.545889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.053963+0000","last_change":"2024-04-25T09:04:13.053963+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.053963+0000","last_undegraded":"2024-04-25T09:04:13.053963+0000","last_fullsized":"2024-04-25T09:04:13.053963+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.059924+0000","last_change":"2024-04-25T09:04:13.058406+0000","last_active":"2024-04-25T09:04:13.059924+0000","last_peered":"2024-04-25T09:04:13.059924+0000","last_clean":"2024-04-25T09:04:13.059924+0000","last_became_active":"2024-04-25T09:04:13.058144+0000","last_became_peered":"2024-04-25T09:04:13.058144+0000","last_unstale":"2024-04-25T09:04:13.059924+0000","last_undegraded":"2024-04-25T09:04:13.059924+0000","last_fullsized":"2024-04-25T09:04:13.059924+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:33.060943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046085+0000","last_change":"2024-04-25T09:04:13.046085+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046085+0000","last_undegraded":"2024-04-25T09:04:13.046085+0000","last_fullsized":"2024-04-25T09:04:13.046085+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060240+0000","last_change":"2024-04-25T09:04:13.059157+0000","last_active":"2024-04-25T09:04:13.060240+0000","last_peered":"2024-04-25T09:04:13.060240+0000","last_clean":"2024-04-25T09:04:13.060240+0000","last_became_active":"2024-04-25T09:04:13.058883+0000","last_became_peered":"2024-04-25T09:04:13.058883+0000","last_unstale":"2024-04-25T09:04:13.060240+0000","last_undegraded":"2024-04-25T09:04:13.060240+0000","last_fullsized":"2024-04-25T09:04:13.060240+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:38.159347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046155+0000","last_change":"2024-04-25T09:04:13.046155+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046155+0000","last_undegraded":"2024-04-25T09:04:13.046155+0000","last_fullsized":"2024-04-25T09:04:13.046155+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.059938+0000","last_change":"2024-04-25T09:04:13.057593+0000","last_active":"2024-04-25T09:04:13.059938+0000","last_peered":"2024-04-25T09:04:13.059938+0000","last_clean":"2024-04-25T09:04:13.059938+0000","last_became_active":"2024-04-25T09:04:13.057330+0000","last_became_peered":"2024-04-25T09:04:13.057330+0000","last_unstale":"2024-04-25T09:04:13.059938+0000","last_undegraded":"2024-04-25T09:04:13.059938+0000","last_fullsized":"2024-04-25T09:04:13.059938+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:32.141616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046021+0000","last_change":"2024-04-25T09:04:13.046021+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046021+0000","last_undegraded":"2024-04-25T09:04:13.046021+0000","last_fullsized":"2024-04-25T09:04:13.046021+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.056354+0000","last_change":"2024-04-25T09:04:13.054808+0000","last_active":"2024-04-25T09:04:13.056354+0000","last_peered":"2024-04-25T09:04:13.056354+0000","last_clean":"2024-04-25T09:04:13.056354+0000","last_became_active":"2024-04-25T09:04:13.054486+0000","last_became_peered":"2024-04-25T09:04:13.054486+0000","last_unstale":"2024-04-25T09:04:13.056354+0000","last_undegraded":"2024-04-25T09:04:13.056354+0000","last_fullsized":"2024-04-25T09:04:13.056354+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:24.933992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.045906+0000","last_change":"2024-04-25T09:04:13.045906+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.045906+0000","last_undegraded":"2024-04-25T09:04:13.045906+0000","last_fullsized":"2024-04-25T09:04:13.045906+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.057196+0000","last_change":"2024-04-25T09:04:13.056215+0000","last_active":"2024-04-25T09:04:13.057196+0000","last_peered":"2024-04-25T09:04:13.057196+0000","last_clean":"2024-04-25T09:04:13.057196+0000","last_became_active":"2024-04-25T09:04:13.055952+0000","last_became_peered":"2024-04-25T09:04:13.055952+0000","last_unstale":"2024-04-25T09:04:13.057196+0000","last_undegraded":"2024-04-25T09:04:13.057196+0000","last_fullsized":"2024-04-25T09:04:13.057196+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:39.762504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.045952+0000","last_change":"2024-04-25T09:04:13.045952+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.045952+0000","last_undegraded":"2024-04-25T09:04:13.045952+0000","last_fullsized":"2024-04-25T09:04:13.045952+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.057234+0000","last_change":"2024-04-25T09:04:13.051625+0000","last_active":"2024-04-25T09:04:13.057234+0000","last_peered":"2024-04-25T09:04:13.057234+0000","last_clean":"2024-04-25T09:04:13.057234+0000","last_became_active":"2024-04-25T09:04:13.051307+0000","last_became_peered":"2024-04-25T09:04:13.051307+0000","last_unstale":"2024-04-25T09:04:13.057234+0000","last_undegraded":"2024-04-25T09:04:13.057234+0000","last_fullsized":"2024-04-25T09:04:13.057234+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:06.808616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.052044+0000","last_change":"2024-04-25T09:04:13.052044+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.052044+0000","last_undegraded":"2024-04-25T09:04:13.052044+0000","last_fullsized":"2024-04-25T09:04:13.052044+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060131+0000","last_change":"2024-04-25T09:04:13.058585+0000","last_active":"2024-04-25T09:04:13.060131+0000","last_peered":"2024-04-25T09:04:13.060131+0000","last_clean":"2024-04-25T09:04:13.060131+0000","last_became_active":"2024-04-25T09:04:13.058298+0000","last_became_peered":"2024-04-25T09:04:13.058298+0000","last_unstale":"2024-04-25T09:04:13.060131+0000","last_undegraded":"2024-04-25T09:04:13.060131+0000","last_fullsized":"2024-04-25T09:04:13.060131+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:25.170530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.047591+0000","last_change":"2024-04-25T09:04:13.047591+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.047591+0000","last_undegraded":"2024-04-25T09:04:13.047591+0000","last_fullsized":"2024-04-25T09:04:13.047591+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.061150+0000","last_change":"2024-04-25T09:04:13.058981+0000","last_active":"2024-04-25T09:04:13.061150+0000","last_peered":"2024-04-25T09:04:13.061150+0000","last_clean":"2024-04-25T09:04:13.061150+0000","last_became_active":"2024-04-25T09:04:13.058662+0000","last_became_peered":"2024-04-25T09:04:13.058662+0000","last_unstale":"2024-04-25T09:04:13.061150+0000","last_undegraded":"2024-04-25T09:04:13.061150+0000","last_fullsized":"2024-04-25T09:04:13.061150+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:35.416425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.049288+0000","last_change":"2024-04-25T09:04:13.049288+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.049288+0000","last_undegraded":"2024-04-25T09:04:13.049288+0000","last_fullsized":"2024-04-25T09:04:13.049288+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060649+0000","last_change":"2024-04-25T09:04:13.058867+0000","last_active":"2024-04-25T09:04:13.060649+0000","last_peered":"2024-04-25T09:04:13.060649+0000","last_clean":"2024-04-25T09:04:13.060649+0000","last_became_active":"2024-04-25T09:04:13.057622+0000","last_became_peered":"2024-04-25T09:04:13.057622+0000","last_unstale":"2024-04-25T09:04:13.060649+0000","last_undegraded":"2024-04-25T09:04:13.060649+0000","last_fullsized":"2024-04-25T09:04:13.060649+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:47.545889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.045500+0000","last_change":"2024-04-25T09:04:13.045500+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.045500+0000","last_undegraded":"2024-04-25T09:04:13.045500+0000","last_fullsized":"2024-04-25T09:04:13.045500+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.055227+0000","last_change":"2024-04-25T09:04:13.054034+0000","last_active":"2024-04-25T09:04:13.055227+0000","last_peered":"2024-04-25T09:04:13.055227+0000","last_clean":"2024-04-25T09:04:13.055227+0000","last_became_active":"2024-04-25T09:04:13.053760+0000","last_became_peered":"2024-04-25T09:04:13.053760+0000","last_unstale":"2024-04-25T09:04:13.055227+0000","last_undegraded":"2024-04-25T09:04:13.055227+0000","last_fullsized":"2024-04-25T09:04:13.055227+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:39.762504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.045249+0000","last_change":"2024-04-25T09:04:13.045249+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.045249+0000","last_undegraded":"2024-04-25T09:04:13.045249+0000","last_fullsized":"2024-04-25T09:04:13.045249+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060397+0000","last_change":"2024-04-25T09:04:13.056393+0000","last_active":"2024-04-25T09:04:13.060397+0000","last_peered":"2024-04-25T09:04:13.060397+0000","last_clean":"2024-04-25T09:04:13.060397+0000","last_became_active":"2024-04-25T09:04:13.056111+0000","last_became_peered":"2024-04-25T09:04:13.056111+0000","last_unstale":"2024-04-25T09:04:13.060397+0000","last_undegraded":"2024-04-25T09:04:13.060397+0000","last_fullsized":"2024-04-25T09:04:13.060397+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:39.762504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.049874+0000","last_change":"2024-04-25T09:04:13.049874+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.049874+0000","last_undegraded":"2024-04-25T09:04:13.049874+0000","last_fullsized":"2024-04-25T09:04:13.049874+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.062256+0000","last_change":"2024-04-25T09:04:13.061556+0000","last_active":"2024-04-25T09:04:13.062256+0000","last_peered":"2024-04-25T09:04:13.062256+0000","last_clean":"2024-04-25T09:04:13.062256+0000","last_became_active":"2024-04-25T09:04:13.061241+0000","last_became_peered":"2024-04-25T09:04:13.061241+0000","last_unstale":"2024-04-25T09:04:13.062256+0000","last_undegraded":"2024-04-25T09:04:13.062256+0000","last_fullsized":"2024-04-25T09:04:13.062256+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:47.072066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.051480+0000","last_change":"2024-04-25T09:04:13.051480+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.051480+0000","last_undegraded":"2024-04-25T09:04:13.051480+0000","last_fullsized":"2024-04-25T09:04:13.051480+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.055764+0000","last_change":"2024-04-25T09:04:13.053984+0000","last_active":"2024-04-25T09:04:13.055764+0000","last_peered":"2024-04-25T09:04:13.055764+0000","last_clean":"2024-04-25T09:04:13.055764+0000","last_became_active":"2024-04-25T09:04:13.053588+0000","last_became_peered":"2024-04-25T09:04:13.053588+0000","last_unstale":"2024-04-25T09:04:13.055764+0000","last_undegraded":"2024-04-25T09:04:13.055764+0000","last_fullsized":"2024-04-25T09:04:13.055764+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:12:05.903006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.051515+0000","last_change":"2024-04-25T09:04:13.051515+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.051515+0000","last_undegraded":"2024-04-25T09:04:13.051515+0000","last_fullsized":"2024-04-25T09:04:13.051515+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.057493+0000","last_change":"2024-04-25T09:04:13.055593+0000","last_active":"2024-04-25T09:04:13.057493+0000","last_peered":"2024-04-25T09:04:13.057493+0000","last_clean":"2024-04-25T09:04:13.057493+0000","last_became_active":"2024-04-25T09:04:13.055293+0000","last_became_peered":"2024-04-25T09:04:13.055293+0000","last_unstale":"2024-04-25T09:04:13.057493+0000","last_undegraded":"2024-04-25T09:04:13.057493+0000","last_fullsized":"2024-04-25T09:04:13.057493+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:09.194558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.050011+0000","last_change":"2024-04-25T09:04:13.050011+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.050011+0000","last_undegraded":"2024-04-25T09:04:13.050011+0000","last_fullsized":"2024-04-25T09:04:13.050011+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060206+0000","last_change":"2024-04-25T09:04:13.057835+0000","last_active":"2024-04-25T09:04:13.060206+0000","last_peered":"2024-04-25T09:04:13.060206+0000","last_clean":"2024-04-25T09:04:13.060206+0000","last_became_active":"2024-04-25T09:04:13.057562+0000","last_became_peered":"2024-04-25T09:04:13.057562+0000","last_unstale":"2024-04-25T09:04:13.060206+0000","last_undegraded":"2024-04-25T09:04:13.060206+0000","last_fullsized":"2024-04-25T09:04:13.060206+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:35.248830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.048188+0000","last_change":"2024-04-25T09:04:13.048188+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.048188+0000","last_undegraded":"2024-04-25T09:04:13.048188+0000","last_fullsized":"2024-04-25T09:04:13.048188+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.062094+0000","last_change":"2024-04-25T09:04:13.060717+0000","last_active":"2024-04-25T09:04:13.062094+0000","last_peered":"2024-04-25T09:04:13.062094+0000","last_clean":"2024-04-25T09:04:13.062094+0000","last_became_active":"2024-04-25T09:04:13.060060+0000","last_became_peered":"2024-04-25T09:04:13.060060+0000","last_unstale":"2024-04-25T09:04:13.062094+0000","last_undegraded":"2024-04-25T09:04:13.062094+0000","last_fullsized":"2024-04-25T09:04:13.062094+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:56.805484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046887+0000","last_change":"2024-04-25T09:04:13.046887+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046887+0000","last_undegraded":"2024-04-25T09:04:13.046887+0000","last_fullsized":"2024-04-25T09:04:13.046887+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.056661+0000","last_change":"2024-04-25T09:04:13.054317+0000","last_active":"2024-04-25T09:04:13.056661+0000","last_peered":"2024-04-25T09:04:13.056661+0000","last_clean":"2024-04-25T09:04:13.056661+0000","last_became_active":"2024-04-25T09:04:13.053816+0000","last_became_peered":"2024-04-25T09:04:13.053816+0000","last_unstale":"2024-04-25T09:04:13.056661+0000","last_undegraded":"2024-04-25T09:04:13.056661+0000","last_fullsized":"2024-04-25T09:04:13.056661+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:12.230718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046289+0000","last_change":"2024-04-25T09:04:13.046289+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046289+0000","last_undegraded":"2024-04-25T09:04:13.046289+0000","last_fullsized":"2024-04-25T09:04:13.046289+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.063552+0000","last_change":"2024-04-25T09:04:13.061478+0000","last_active":"2024-04-25T09:04:13.063552+0000","last_peered":"2024-04-25T09:04:13.063552+0000","last_clean":"2024-04-25T09:04:13.063552+0000","last_became_active":"2024-04-25T09:04:13.061150+0000","last_became_peered":"2024-04-25T09:04:13.061150+0000","last_unstale":"2024-04-25T09:04:13.063552+0000","last_undegraded":"2024-04-25T09:04:13.063552+0000","last_fullsized":"2024-04-25T09:04:13.063552+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:52.903859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.045727+0000","last_change":"2024-04-25T09:04:13.045727+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.045727+0000","last_undegraded":"2024-04-25T09:04:13.045727+0000","last_fullsized":"2024-04-25T09:04:13.045727+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060503+0000","last_change":"2024-04-25T09:04:13.059762+0000","last_active":"2024-04-25T09:04:13.060503+0000","last_peered":"2024-04-25T09:04:13.060503+0000","last_clean":"2024-04-25T09:04:13.060503+0000","last_became_active":"2024-04-25T09:04:13.059482+0000","last_became_peered":"2024-04-25T09:04:13.059482+0000","last_unstale":"2024-04-25T09:04:13.060503+0000","last_undegraded":"2024-04-25T09:04:13.060503+0000","last_fullsized":"2024-04-25T09:04:13.060503+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:30:52.194928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.045307+0000","last_change":"2024-04-25T09:04:13.045307+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.045307+0000","last_undegraded":"2024-04-25T09:04:13.045307+0000","last_fullsized":"2024-04-25T09:04:13.045307+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060790+0000","last_change":"2024-04-25T09:04:13.059953+0000","last_active":"2024-04-25T09:04:13.060790+0000","last_peered":"2024-04-25T09:04:13.060790+0000","last_clean":"2024-04-25T09:04:13.060790+0000","last_became_active":"2024-04-25T09:04:13.059633+0000","last_became_peered":"2024-04-25T09:04:13.059633+0000","last_unstale":"2024-04-25T09:04:13.060790+0000","last_undegraded":"2024-04-25T09:04:13.060790+0000","last_fullsized":"2024-04-25T09:04:13.060790+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:04.737828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.045467+0000","last_change":"2024-04-25T09:04:13.045467+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.045467+0000","last_undegraded":"2024-04-25T09:04:13.045467+0000","last_fullsized":"2024-04-25T09:04:13.045467+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.063544+0000","last_change":"2024-04-25T09:04:13.062137+0000","last_active":"2024-04-25T09:04:13.063544+0000","last_peered":"2024-04-25T09:04:13.063544+0000","last_clean":"2024-04-25T09:04:13.063544+0000","last_became_active":"2024-04-25T09:04:13.061505+0000","last_became_peered":"2024-04-25T09:04:13.061505+0000","last_unstale":"2024-04-25T09:04:13.063544+0000","last_undegraded":"2024-04-25T09:04:13.063544+0000","last_fullsized":"2024-04-25T09:04:13.063544+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:27.711642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.050690+0000","last_change":"2024-04-25T09:04:13.050690+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.050690+0000","last_undegraded":"2024-04-25T09:04:13.050690+0000","last_fullsized":"2024-04-25T09:04:13.050690+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060823+0000","last_change":"2024-04-25T09:04:13.057849+0000","last_active":"2024-04-25T09:04:13.060823+0000","last_peered":"2024-04-25T09:04:13.060823+0000","last_clean":"2024-04-25T09:04:13.060823+0000","last_became_active":"2024-04-25T09:04:13.057586+0000","last_became_peered":"2024-04-25T09:04:13.057586+0000","last_unstale":"2024-04-25T09:04:13.060823+0000","last_undegraded":"2024-04-25T09:04:13.060823+0000","last_fullsized":"2024-04-25T09:04:13.060823+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:33.060943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.044936+0000","last_change":"2024-04-25T09:04:13.044936+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.044936+0000","last_undegraded":"2024-04-25T09:04:13.044936+0000","last_fullsized":"2024-04-25T09:04:13.044936+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060734+0000","last_change":"2024-04-25T09:04:13.056588+0000","last_active":"2024-04-25T09:04:13.060734+0000","last_peered":"2024-04-25T09:04:13.060734+0000","last_clean":"2024-04-25T09:04:13.060734+0000","last_became_active":"2024-04-25T09:04:13.055410+0000","last_became_peered":"2024-04-25T09:04:13.055410+0000","last_unstale":"2024-04-25T09:04:13.060734+0000","last_undegraded":"2024-04-25T09:04:13.060734+0000","last_fullsized":"2024-04-25T09:04:13.060734+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:47.545889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.047647+0000","last_change":"2024-04-25T09:04:13.047647+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.047647+0000","last_undegraded":"2024-04-25T09:04:13.047647+0000","last_fullsized":"2024-04-25T09:04:13.047647+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060898+0000","last_change":"2024-04-25T09:04:13.058491+0000","last_active":"2024-04-25T09:04:13.060898+0000","last_peered":"2024-04-25T09:04:13.060898+0000","last_clean":"2024-04-25T09:04:13.060898+0000","last_became_active":"2024-04-25T09:04:13.058215+0000","last_became_peered":"2024-04-25T09:04:13.058215+0000","last_unstale":"2024-04-25T09:04:13.060898+0000","last_undegraded":"2024-04-25T09:04:13.060898+0000","last_fullsized":"2024-04-25T09:04:13.060898+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:04.737828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.047830+0000","last_change":"2024-04-25T09:04:13.047830+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.047830+0000","last_undegraded":"2024-04-25T09:04:13.047830+0000","last_fullsized":"2024-04-25T09:04:13.047830+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.056639+0000","last_change":"2024-04-25T09:04:13.054092+0000","last_active":"2024-04-25T09:04:13.056639+0000","last_peered":"2024-04-25T09:04:13.056639+0000","last_clean":"2024-04-25T09:04:13.056639+0000","last_became_active":"2024-04-25T09:04:13.053427+0000","last_became_peered":"2024-04-25T09:04:13.053427+0000","last_unstale":"2024-04-25T09:04:13.056639+0000","last_undegraded":"2024-04-25T09:04:13.056639+0000","last_fullsized":"2024-04-25T09:04:13.056639+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:47.205245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.046670+0000","last_change":"2024-04-25T09:04:13.046670+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.046670+0000","last_undegraded":"2024-04-25T09:04:13.046670+0000","last_fullsized":"2024-04-25T09:04:13.046670+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.063064+0000","last_change":"2024-04-25T09:04:13.060816+0000","last_active":"2024-04-25T09:04:13.063064+0000","last_peered":"2024-04-25T09:04:13.063064+0000","last_clean":"2024-04-25T09:04:13.063064+0000","last_became_active":"2024-04-25T09:04:13.060535+0000","last_became_peered":"2024-04-25T09:04:13.060535+0000","last_unstale":"2024-04-25T09:04:13.063064+0000","last_undegraded":"2024-04-25T09:04:13.063064+0000","last_fullsized":"2024-04-25T09:04:13.063064+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:49.426430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.050769+0000","last_change":"2024-04-25T09:04:13.050769+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.050769+0000","last_undegraded":"2024-04-25T09:04:13.050769+0000","last_fullsized":"2024-04-25T09:04:13.050769+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060740+0000","last_change":"2024-04-25T09:04:13.059109+0000","last_active":"2024-04-25T09:04:13.060740+0000","last_peered":"2024-04-25T09:04:13.060740+0000","last_clean":"2024-04-25T09:04:13.060740+0000","last_became_active":"2024-04-25T09:04:13.058224+0000","last_became_peered":"2024-04-25T09:04:13.058224+0000","last_unstale":"2024-04-25T09:04:13.060740+0000","last_undegraded":"2024-04-25T09:04:13.060740+0000","last_fullsized":"2024-04-25T09:04:13.060740+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:33.060943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.045284+0000","last_change":"2024-04-25T09:04:13.045284+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.045284+0000","last_undegraded":"2024-04-25T09:04:13.045284+0000","last_fullsized":"2024-04-25T09:04:13.045284+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.060585+0000","last_change":"2024-04-25T09:04:13.057904+0000","last_active":"2024-04-25T09:04:13.060585+0000","last_peered":"2024-04-25T09:04:13.060585+0000","last_clean":"2024-04-25T09:04:13.060585+0000","last_became_active":"2024-04-25T09:04:13.057602+0000","last_became_peered":"2024-04-25T09:04:13.057602+0000","last_unstale":"2024-04-25T09:04:13.060585+0000","last_undegraded":"2024-04-25T09:04:13.060585+0000","last_fullsized":"2024-04-25T09:04:13.060585+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:24.697586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":1,"reported_epoch":137,"state":"creating+peering","last_fresh":"2024-04-25T09:04:13.045651+0000","last_change":"2024-04-25T09:04:13.045651+0000","last_active":"2024-04-25T09:04:13.034365+0000","last_peered":"2024-04-25T09:04:13.034365+0000","last_clean":"2024-04-25T09:04:13.034365+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T09:04:13.045651+0000","last_undegraded":"2024-04-25T09:04:13.045651+0000","last_fullsized":"2024-04-25T09:04:13.045651+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.054181+0000","last_change":"2024-04-25T09:04:13.053009+0000","last_active":"2024-04-25T09:04:13.054181+0000","last_peered":"2024-04-25T09:04:13.054181+0000","last_clean":"2024-04-25T09:04:13.054181+0000","last_became_active":"2024-04-25T09:04:13.052708+0000","last_became_peered":"2024-04-25T09:04:13.052708+0000","last_unstale":"2024-04-25T09:04:13.054181+0000","last_undegraded":"2024-04-25T09:04:13.054181+0000","last_fullsized":"2024-04-25T09:04:13.054181+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:33.000888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":388,"reported_epoch":137,"state":"active+clean","last_fresh":"2024-04-25T09:04:13.054625+0000","last_change":"2024-04-25T08:54:00.818044+0000","last_active":"2024-04-25T09:04:13.054625+0000","last_peered":"2024-04-25T09:04:13.054625+0000","last_clean":"2024-04-25T09:04:13.054625+0000","last_became_active":"2024-04-25T08:54:00.817098+0000","last_became_peered":"2024-04-25T08:54:00.817098+0000","last_unstale":"2024-04-25T09:04:13.054625+0000","last_undegraded":"2024-04-25T09:04:13.054625+0000","last_fullsized":"2024-04-25T09:04:13.054625+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_clean_scrub_stamp":"2024-04-25T08:53:59.739677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:48.129513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574973,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47328,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324512,"statfs":{"total":96636764160,"available":96588300288,"internally_reserved":0,"allocated":2310144,"data_stored":1340135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574974,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68956,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302884,"statfs":{"total":96636764160,"available":96566153216,"internally_reserved":0,"allocated":1716224,"data_stored":749767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574976,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55064,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316776,"statfs":{"total":96636764160,"available":96580378624,"internally_reserved":0,"allocated":1716224,"data_stored":749767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574974,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68960,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302880,"statfs":{"total":96636764160,"available":96566149120,"internally_reserved":0,"allocated":1716224,"data_stored":749767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574974,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77276,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294564,"statfs":{"total":96636764160,"available":96557633536,"internally_reserved":0,"allocated":1716224,"data_stored":749767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574974,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46744,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325096,"statfs":{"total":96636764160,"available":96588898304,"internally_reserved":0,"allocated":1716224,"data_stored":749767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574974,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82840,"kb_used_data":1676,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94289000,"statfs":{"total":96636764160,"available":96551936000,"internally_reserved":0,"allocated":1716224,"data_stored":749767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574975,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51488,"kb_used_data":2256,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320352,"statfs":{"total":96636764160,"available":96584040448,"internally_reserved":0,"allocated":2310144,"data_stored":1340135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:04:21.491 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:04:21.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.617+0000 7fa562046640 1 Processor -- start 2024-04-25T09:04:21.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.617+0000 7fa562046640 1 -- start start 2024-04-25T09:04:21.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.618+0000 7fa562046640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa55c0cbfc0 0x7fa55c06dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:21.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.618+0000 7fa562046640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa55c0ccbb0 0x7fa55c0cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:21.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.618+0000 7fa562046640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa55c0b2780 con 0x7fa55c15f5e0 2024-04-25T09:04:21.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.618+0000 7fa562046640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa55c0cd550 con 0x7fa55c0cbfc0 2024-04-25T09:04:21.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.618+0000 7fa562046640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa55c0b2a60 con 0x7fa55c0ccbb0 2024-04-25T09:04:21.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.618+0000 7fa55b7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa55c0cbfc0 0x7fa55c06dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:21.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.618+0000 7fa55b7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa55c0cbfc0 0x7fa55c06dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38806/0 (socket says 172.21.15.43:38806) 2024-04-25T09:04:21.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.618+0000 7fa55b7fe640 1 -- 172.21.15.43:0/641608892 learned_addr learned my addr 172.21.15.43:0/641608892 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:21.607 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.618+0000 7fa55affd640 1 --2- 172.21.15.43:0/641608892 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa55c0ccbb0 0x7fa55c0cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:21.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.618+0000 7fa55b7fe640 1 -- 172.21.15.43:0/641608892 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa55c0ccbb0 msgr2=0x7fa55c0cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:21.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.618+0000 7fa55b7fe640 1 --2- 172.21.15.43:0/641608892 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa55c0ccbb0 0x7fa55c0cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:21.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.618+0000 7fa55b7fe640 1 -- 172.21.15.43:0/641608892 >> v1:172.21.15.193:6789/0 conn(0x7fa55c15f5e0 legacy=0x7fa55c15fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:21.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.619+0000 7fa55b7fe640 1 -- 172.21.15.43:0/641608892 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa55c0b2d40 con 0x7fa55c0cbfc0 2024-04-25T09:04:21.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.619+0000 7fa55b7fe640 1 --2- 172.21.15.43:0/641608892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa55c0cbfc0 0x7fa55c06dc60 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fa54c009870 tx=0x7fa54c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfba6fc013174520 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:21.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.619+0000 7fa55a7fc640 1 -- 172.21.15.43:0/641608892 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54c037070 con 0x7fa55c0cbfc0 2024-04-25T09:04:21.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.619+0000 7fa55a7fc640 1 -- 172.21.15.43:0/641608892 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa54c032440 con 0x7fa55c0cbfc0 2024-04-25T09:04:21.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.619+0000 7fa55a7fc640 1 -- 172.21.15.43:0/641608892 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa54c032810 con 0x7fa55c0cbfc0 2024-04-25T09:04:21.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.620+0000 7fa562046640 1 -- 172.21.15.43:0/641608892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa55c0cbfc0 msgr2=0x7fa55c06dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:21.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.620+0000 7fa562046640 1 --2- 172.21.15.43:0/641608892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa55c0cbfc0 0x7fa55c06dc60 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fa54c009870 tx=0x7fa54c02a270 comp rx=0 tx=0).stop 2024-04-25T09:04:21.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.620+0000 7fa562046640 1 -- 172.21.15.43:0/641608892 shutdown_connections 2024-04-25T09:04:21.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.620+0000 7fa562046640 1 --2- 172.21.15.43:0/641608892 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa55c0ccbb0 0x7fa55c0cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:21.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.620+0000 7fa562046640 1 --2- 172.21.15.43:0/641608892 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa55c0cbfc0 0x7fa55c06dc60 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:21.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.620+0000 7fa562046640 1 -- 172.21.15.43:0/641608892 >> 172.21.15.43:0/641608892 conn(0x7fa55c0b1480 msgr2=0x7fa55c0b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:21.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.620+0000 7fa562046640 1 -- 172.21.15.43:0/641608892 shutdown_connections 2024-04-25T09:04:21.610 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.620+0000 7fa562046640 1 -- 172.21.15.43:0/641608892 wait complete. 2024-04-25T09:04:21.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.621+0000 7fa562046640 1 Processor -- start 2024-04-25T09:04:21.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.621+0000 7fa562046640 1 -- start start 2024-04-25T09:04:21.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.622+0000 7fa562046640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa55c0cbfc0 0x7fa55c0a9a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:21.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.622+0000 7fa562046640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa55c0ccbb0 0x7fa55c0ab0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:21.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.622+0000 7fa562046640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa55c15f5e0 0x7fa55c0cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:21.612 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.622+0000 7fa562046640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55c0b35e0 con 0x7fa55c0ccbb0 2024-04-25T09:04:21.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.622+0000 7fa562046640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55c0b3300 con 0x7fa55c0cbfc0 2024-04-25T09:04:21.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.622+0000 7fa562046640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa55c06e1a0 con 0x7fa55c15f5e0 2024-04-25T09:04:21.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.622+0000 7fa55affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa55c0ccbb0 0x7fa55c0ab0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:21.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.622+0000 7fa55affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa55c0ccbb0 0x7fa55c0ab0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38812/0 (socket says 172.21.15.43:38812) 2024-04-25T09:04:21.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.622+0000 7fa55affd640 1 -- 172.21.15.43:0/110912455 learned_addr learned my addr 172.21.15.43:0/110912455 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:21.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.622+0000 7fa55b7fe640 1 --2- 172.21.15.43:0/110912455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa55c0cbfc0 0x7fa55c0a9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:21.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.623+0000 7fa55bfff640 1 --2- 172.21.15.43:0/110912455 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa55c15f5e0 0x7fa55c0cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:21.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.623+0000 7fa55affd640 1 -- 172.21.15.43:0/110912455 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa55c15f5e0 msgr2=0x7fa55c0cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:21.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.623+0000 7fa55affd640 1 --2- 172.21.15.43:0/110912455 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa55c15f5e0 0x7fa55c0cb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:21.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.623+0000 7fa55affd640 1 -- 172.21.15.43:0/110912455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa55c0cbfc0 msgr2=0x7fa55c0a9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:21.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.623+0000 7fa55affd640 1 --2- 172.21.15.43:0/110912455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa55c0cbfc0 0x7fa55c0a9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:21.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.623+0000 7fa55affd640 1 -- 172.21.15.43:0/110912455 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa55c0aa020 con 0x7fa55c0ccbb0 2024-04-25T09:04:21.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.623+0000 7fa55b7fe640 1 --2- 172.21.15.43:0/110912455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa55c0cbfc0 0x7fa55c0a9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:21.613 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.623+0000 7fa55bfff640 1 --2- 172.21.15.43:0/110912455 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa55c15f5e0 0x7fa55c0cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:21.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.623+0000 7fa55affd640 1 --2- 172.21.15.43:0/110912455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa55c0ccbb0 0x7fa55c0ab0c0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fa55000d830 tx=0x7fa55000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:21.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.623+0000 7fa543fff640 1 -- 172.21.15.43:0/110912455 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55000b6c0 con 0x7fa55c0ccbb0 2024-04-25T09:04:21.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.623+0000 7fa543fff640 1 -- 172.21.15.43:0/110912455 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa550019070 con 0x7fa55c0ccbb0 2024-04-25T09:04:21.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.623+0000 7fa562046640 1 -- 172.21.15.43:0/110912455 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa55c0aa300 con 0x7fa55c0ccbb0 2024-04-25T09:04:21.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.623+0000 7fa543fff640 1 -- 172.21.15.43:0/110912455 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55000bac0 con 0x7fa55c0ccbb0 2024-04-25T09:04:21.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.623+0000 7fa562046640 1 -- 172.21.15.43:0/110912455 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa55c0aa8c0 con 0x7fa55c0ccbb0 2024-04-25T09:04:21.614 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.624+0000 7fa562046640 1 -- 172.21.15.43:0/110912455 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa55c074ef0 con 0x7fa55c0ccbb0 2024-04-25T09:04:21.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.625+0000 7fa543fff640 1 -- 172.21.15.43:0/110912455 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa55000bc60 con 0x7fa55c0ccbb0 2024-04-25T09:04:21.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.625+0000 7fa543fff640 1 --2- 172.21.15.43:0/110912455 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa54806b4d0 0x7fa54806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:21.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.625+0000 7fa543fff640 1 -- 172.21.15.43:0/110912455 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa55008a650 con 0x7fa55c0ccbb0 2024-04-25T09:04:21.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.626+0000 7fa55b7fe640 1 --2- 172.21.15.43:0/110912455 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa54806b4d0 0x7fa54806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:21.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.626+0000 7fa55b7fe640 1 --2- 172.21.15.43:0/110912455 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa54806b4d0 0x7fa54806d990 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fa54c034000 tx=0x7fa54c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:21.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.630+0000 7fa543fff640 1 -- 172.21.15.43:0/110912455 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa550059560 con 0x7fa55c0ccbb0 2024-04-25T09:04:21.795 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.806+0000 7fa562046640 1 -- 172.21.15.43:0/110912455 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa55c0cb730 con 0x7fa54806b4d0 2024-04-25T09:04:21.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.813+0000 7fa543fff640 1 -- 172.21.15.43:0/110912455 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252504 (secure 0 0 0) 0x7fa54c03c460 con 0x7fa54806b4d0 2024-04-25T09:04:21.803 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:04:21.805 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-25T09:04:21.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.816+0000 7fa562046640 1 -- 172.21.15.43:0/110912455 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa54806b4d0 msgr2=0x7fa54806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:21.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.816+0000 7fa562046640 1 --2- 172.21.15.43:0/110912455 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa54806b4d0 0x7fa54806d990 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fa54c034000 tx=0x7fa54c02e000 comp rx=0 tx=0).stop 2024-04-25T09:04:21.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.816+0000 7fa562046640 1 -- 172.21.15.43:0/110912455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa55c0ccbb0 msgr2=0x7fa55c0ab0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:21.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.816+0000 7fa562046640 1 --2- 172.21.15.43:0/110912455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa55c0ccbb0 0x7fa55c0ab0c0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fa55000d830 tx=0x7fa55000dcf0 comp rx=0 tx=0).stop 2024-04-25T09:04:21.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.816+0000 7fa562046640 1 -- 172.21.15.43:0/110912455 shutdown_connections 2024-04-25T09:04:21.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.816+0000 7fa562046640 1 --2- 172.21.15.43:0/110912455 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa55c15f5e0 0x7fa55c0cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:21.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.816+0000 7fa562046640 1 --2- 172.21.15.43:0/110912455 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa55c0ccbb0 0x7fa55c0ab0c0 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:21.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.816+0000 7fa562046640 1 --2- 172.21.15.43:0/110912455 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa54806b4d0 0x7fa54806d990 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:21.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.816+0000 7fa562046640 1 --2- 172.21.15.43:0/110912455 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa55c0cbfc0 0x7fa55c0a9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:21.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.816+0000 7fa562046640 1 -- 172.21.15.43:0/110912455 >> 172.21.15.43:0/110912455 conn(0x7fa55c0b1480 msgr2=0x7fa55c15eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:21.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.816+0000 7fa562046640 1 -- 172.21.15.43:0/110912455 shutdown_connections 2024-04-25T09:04:21.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.816+0000 7fa562046640 1 -- 172.21.15.43:0/110912455 wait complete. 2024-04-25T09:04:21.815 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":443,"stamp":"2024-04-25T09:04:19.836722+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":498908,"kb_used_data":14824,"kb_used_omap":12,"kb_used_meta":483955,"kb_avail":754475812,"statfs":{"total":773094113280,"available":772583231488,"internally_reserved":0,"allocated":15179776,"data_stored":7288292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":495570481},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.793783"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.080186+0000","last_change":"2024-04-25T09:04:14.077513+0000","last_active":"2024-04-25T09:04:15.080186+0000","last_peered":"2024-04-25T09:04:15.080186+0000","last_clean":"2024-04-25T09:04:15.080186+0000","last_became_active":"2024-04-25T09:04:14.077226+0000","last_became_peered":"2024-04-25T09:04:14.077226+0000","last_unstale":"2024-04-25T09:04:15.080186+0000","last_undegraded":"2024-04-25T09:04:15.080186+0000","last_fullsized":"2024-04-25T09:04:15.080186+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:00.353992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.065749+0000","last_change":"2024-04-25T09:04:14.071748+0000","last_active":"2024-04-25T09:04:15.065749+0000","last_peered":"2024-04-25T09:04:15.065749+0000","last_clean":"2024-04-25T09:04:15.065749+0000","last_became_active":"2024-04-25T09:04:14.071067+0000","last_became_peered":"2024-04-25T09:04:14.071067+0000","last_unstale":"2024-04-25T09:04:15.065749+0000","last_undegraded":"2024-04-25T09:04:15.065749+0000","last_fullsized":"2024-04-25T09:04:15.065749+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:17:58.963521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.973029+0000","last_change":"2024-04-25T09:04:14.082732+0000","last_active":"2024-04-25T09:04:15.973029+0000","last_peered":"2024-04-25T09:04:15.973029+0000","last_clean":"2024-04-25T09:04:15.973029+0000","last_became_active":"2024-04-25T09:04:14.082450+0000","last_became_peered":"2024-04-25T09:04:14.082450+0000","last_unstale":"2024-04-25T09:04:15.973029+0000","last_undegraded":"2024-04-25T09:04:15.973029+0000","last_fullsized":"2024-04-25T09:04:15.973029+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:55.615005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.066666+0000","last_change":"2024-04-25T09:04:14.085677+0000","last_active":"2024-04-25T09:04:15.066666+0000","last_peered":"2024-04-25T09:04:15.066666+0000","last_clean":"2024-04-25T09:04:15.066666+0000","last_became_active":"2024-04-25T09:04:14.085418+0000","last_became_peered":"2024-04-25T09:04:14.085418+0000","last_unstale":"2024-04-25T09:04:15.066666+0000","last_undegraded":"2024-04-25T09:04:15.066666+0000","last_fullsized":"2024-04-25T09:04:15.066666+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:31.639139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.067278+0000","last_change":"2024-04-25T09:04:14.080888+0000","last_active":"2024-04-25T09:04:15.067278+0000","last_peered":"2024-04-25T09:04:15.067278+0000","last_clean":"2024-04-25T09:04:15.067278+0000","last_became_active":"2024-04-25T09:04:14.080403+0000","last_became_peered":"2024-04-25T09:04:14.080403+0000","last_unstale":"2024-04-25T09:04:15.067278+0000","last_undegraded":"2024-04-25T09:04:15.067278+0000","last_fullsized":"2024-04-25T09:04:15.067278+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:05.743018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.079893+0000","last_change":"2024-04-25T09:04:14.086527+0000","last_active":"2024-04-25T09:04:15.079893+0000","last_peered":"2024-04-25T09:04:15.079893+0000","last_clean":"2024-04-25T09:04:15.079893+0000","last_became_active":"2024-04-25T09:04:14.085943+0000","last_became_peered":"2024-04-25T09:04:14.085943+0000","last_unstale":"2024-04-25T09:04:15.079893+0000","last_undegraded":"2024-04-25T09:04:15.079893+0000","last_fullsized":"2024-04-25T09:04:15.079893+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:49.273567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.972700+0000","last_change":"2024-04-25T09:04:14.071147+0000","last_active":"2024-04-25T09:04:15.972700+0000","last_peered":"2024-04-25T09:04:15.972700+0000","last_clean":"2024-04-25T09:04:15.972700+0000","last_became_active":"2024-04-25T09:04:14.070721+0000","last_became_peered":"2024-04-25T09:04:14.070721+0000","last_unstale":"2024-04-25T09:04:15.972700+0000","last_undegraded":"2024-04-25T09:04:15.972700+0000","last_fullsized":"2024-04-25T09:04:15.972700+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:52.757931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.079697+0000","last_change":"2024-04-25T09:04:14.073993+0000","last_active":"2024-04-25T09:04:15.079697+0000","last_peered":"2024-04-25T09:04:15.079697+0000","last_clean":"2024-04-25T09:04:15.079697+0000","last_became_active":"2024-04-25T09:04:14.072313+0000","last_became_peered":"2024-04-25T09:04:14.072313+0000","last_unstale":"2024-04-25T09:04:15.079697+0000","last_undegraded":"2024-04-25T09:04:15.079697+0000","last_fullsized":"2024-04-25T09:04:15.079697+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:31.639139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.065785+0000","last_change":"2024-04-25T09:04:14.077742+0000","last_active":"2024-04-25T09:04:15.065785+0000","last_peered":"2024-04-25T09:04:15.065785+0000","last_clean":"2024-04-25T09:04:15.065785+0000","last_became_active":"2024-04-25T09:04:14.076201+0000","last_became_peered":"2024-04-25T09:04:14.076201+0000","last_unstale":"2024-04-25T09:04:15.065785+0000","last_undegraded":"2024-04-25T09:04:15.065785+0000","last_fullsized":"2024-04-25T09:04:15.065785+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:34.153158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.082574+0000","last_change":"2024-04-25T09:04:14.089825+0000","last_active":"2024-04-25T09:04:15.082574+0000","last_peered":"2024-04-25T09:04:15.082574+0000","last_clean":"2024-04-25T09:04:15.082574+0000","last_became_active":"2024-04-25T09:04:14.089544+0000","last_became_peered":"2024-04-25T09:04:14.089544+0000","last_unstale":"2024-04-25T09:04:15.082574+0000","last_undegraded":"2024-04-25T09:04:15.082574+0000","last_fullsized":"2024-04-25T09:04:15.082574+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:27.468657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.080218+0000","last_change":"2024-04-25T09:04:14.073256+0000","last_active":"2024-04-25T09:04:15.080218+0000","last_peered":"2024-04-25T09:04:15.080218+0000","last_clean":"2024-04-25T09:04:15.080218+0000","last_became_active":"2024-04-25T09:04:14.072856+0000","last_became_peered":"2024-04-25T09:04:14.072856+0000","last_unstale":"2024-04-25T09:04:15.080218+0000","last_undegraded":"2024-04-25T09:04:15.080218+0000","last_fullsized":"2024-04-25T09:04:15.080218+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:57.810674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.082652+0000","last_change":"2024-04-25T09:04:14.085634+0000","last_active":"2024-04-25T09:04:15.082652+0000","last_peered":"2024-04-25T09:04:15.082652+0000","last_clean":"2024-04-25T09:04:15.082652+0000","last_became_active":"2024-04-25T09:04:14.084986+0000","last_became_peered":"2024-04-25T09:04:14.084986+0000","last_unstale":"2024-04-25T09:04:15.082652+0000","last_undegraded":"2024-04-25T09:04:15.082652+0000","last_fullsized":"2024-04-25T09:04:15.082652+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:28:58.091087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.067373+0000","last_change":"2024-04-25T09:04:14.080670+0000","last_active":"2024-04-25T09:04:15.067373+0000","last_peered":"2024-04-25T09:04:15.067373+0000","last_clean":"2024-04-25T09:04:15.067373+0000","last_became_active":"2024-04-25T09:04:14.080044+0000","last_became_peered":"2024-04-25T09:04:14.080044+0000","last_unstale":"2024-04-25T09:04:15.067373+0000","last_undegraded":"2024-04-25T09:04:15.067373+0000","last_fullsized":"2024-04-25T09:04:15.067373+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:14.627098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.066547+0000","last_change":"2024-04-25T09:04:14.077968+0000","last_active":"2024-04-25T09:04:15.066547+0000","last_peered":"2024-04-25T09:04:15.066547+0000","last_clean":"2024-04-25T09:04:15.066547+0000","last_became_active":"2024-04-25T09:04:14.077520+0000","last_became_peered":"2024-04-25T09:04:14.077520+0000","last_unstale":"2024-04-25T09:04:15.066547+0000","last_undegraded":"2024-04-25T09:04:15.066547+0000","last_fullsized":"2024-04-25T09:04:15.066547+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:39.591985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.082099+0000","last_change":"2024-04-25T09:04:14.083599+0000","last_active":"2024-04-25T09:04:15.082099+0000","last_peered":"2024-04-25T09:04:15.082099+0000","last_clean":"2024-04-25T09:04:15.082099+0000","last_became_active":"2024-04-25T09:04:14.083274+0000","last_became_peered":"2024-04-25T09:04:14.083274+0000","last_unstale":"2024-04-25T09:04:15.082099+0000","last_undegraded":"2024-04-25T09:04:15.082099+0000","last_fullsized":"2024-04-25T09:04:15.082099+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:28.997133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.972662+0000","last_change":"2024-04-25T09:04:14.078531+0000","last_active":"2024-04-25T09:04:15.972662+0000","last_peered":"2024-04-25T09:04:15.972662+0000","last_clean":"2024-04-25T09:04:15.972662+0000","last_became_active":"2024-04-25T09:04:14.078229+0000","last_became_peered":"2024-04-25T09:04:14.078229+0000","last_unstale":"2024-04-25T09:04:15.972662+0000","last_undegraded":"2024-04-25T09:04:15.972662+0000","last_fullsized":"2024-04-25T09:04:15.972662+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:05.574571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.082753+0000","last_change":"2024-04-25T09:04:14.086272+0000","last_active":"2024-04-25T09:04:15.082753+0000","last_peered":"2024-04-25T09:04:15.082753+0000","last_clean":"2024-04-25T09:04:15.082753+0000","last_became_active":"2024-04-25T09:04:14.084017+0000","last_became_peered":"2024-04-25T09:04:14.084017+0000","last_unstale":"2024-04-25T09:04:15.082753+0000","last_undegraded":"2024-04-25T09:04:15.082753+0000","last_fullsized":"2024-04-25T09:04:15.082753+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:31:56.265040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.973015+0000","last_change":"2024-04-25T09:04:14.082153+0000","last_active":"2024-04-25T09:04:15.973015+0000","last_peered":"2024-04-25T09:04:15.973015+0000","last_clean":"2024-04-25T09:04:15.973015+0000","last_became_active":"2024-04-25T09:04:14.081868+0000","last_became_peered":"2024-04-25T09:04:14.081868+0000","last_unstale":"2024-04-25T09:04:15.973015+0000","last_undegraded":"2024-04-25T09:04:15.973015+0000","last_fullsized":"2024-04-25T09:04:15.973015+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:59.704426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.080258+0000","last_change":"2024-04-25T09:04:14.076117+0000","last_active":"2024-04-25T09:04:15.080258+0000","last_peered":"2024-04-25T09:04:15.080258+0000","last_clean":"2024-04-25T09:04:15.080258+0000","last_became_active":"2024-04-25T09:04:14.075764+0000","last_became_peered":"2024-04-25T09:04:14.075764+0000","last_unstale":"2024-04-25T09:04:15.080258+0000","last_undegraded":"2024-04-25T09:04:15.080258+0000","last_fullsized":"2024-04-25T09:04:15.080258+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:29.459927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.973103+0000","last_change":"2024-04-25T09:04:14.074929+0000","last_active":"2024-04-25T09:04:15.973103+0000","last_peered":"2024-04-25T09:04:15.973103+0000","last_clean":"2024-04-25T09:04:15.973103+0000","last_became_active":"2024-04-25T09:04:14.074622+0000","last_became_peered":"2024-04-25T09:04:14.074622+0000","last_unstale":"2024-04-25T09:04:15.973103+0000","last_undegraded":"2024-04-25T09:04:15.973103+0000","last_fullsized":"2024-04-25T09:04:15.973103+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:47:34.006078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.082369+0000","last_change":"2024-04-25T09:04:14.091520+0000","last_active":"2024-04-25T09:04:15.082369+0000","last_peered":"2024-04-25T09:04:15.082369+0000","last_clean":"2024-04-25T09:04:15.082369+0000","last_became_active":"2024-04-25T09:04:14.091253+0000","last_became_peered":"2024-04-25T09:04:14.091253+0000","last_unstale":"2024-04-25T09:04:15.082369+0000","last_undegraded":"2024-04-25T09:04:15.082369+0000","last_fullsized":"2024-04-25T09:04:15.082369+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:31.322998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.066736+0000","last_change":"2024-04-25T09:04:14.080224+0000","last_active":"2024-04-25T09:04:15.066736+0000","last_peered":"2024-04-25T09:04:15.066736+0000","last_clean":"2024-04-25T09:04:15.066736+0000","last_became_active":"2024-04-25T09:04:14.079674+0000","last_became_peered":"2024-04-25T09:04:14.079674+0000","last_unstale":"2024-04-25T09:04:15.066736+0000","last_undegraded":"2024-04-25T09:04:15.066736+0000","last_fullsized":"2024-04-25T09:04:15.066736+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:59.704426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.066361+0000","last_change":"2024-04-25T09:04:14.079430+0000","last_active":"2024-04-25T09:04:15.066361+0000","last_peered":"2024-04-25T09:04:15.066361+0000","last_clean":"2024-04-25T09:04:15.066361+0000","last_became_active":"2024-04-25T09:04:14.079074+0000","last_became_peered":"2024-04-25T09:04:14.079074+0000","last_unstale":"2024-04-25T09:04:15.066361+0000","last_undegraded":"2024-04-25T09:04:15.066361+0000","last_fullsized":"2024-04-25T09:04:15.066361+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:37.765328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.082048+0000","last_change":"2024-04-25T09:04:14.086408+0000","last_active":"2024-04-25T09:04:15.082048+0000","last_peered":"2024-04-25T09:04:15.082048+0000","last_clean":"2024-04-25T09:04:15.082048+0000","last_became_active":"2024-04-25T09:04:14.086086+0000","last_became_peered":"2024-04-25T09:04:14.086086+0000","last_unstale":"2024-04-25T09:04:15.082048+0000","last_undegraded":"2024-04-25T09:04:15.082048+0000","last_fullsized":"2024-04-25T09:04:15.082048+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:34.157594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.972913+0000","last_change":"2024-04-25T09:04:14.081905+0000","last_active":"2024-04-25T09:04:15.972913+0000","last_peered":"2024-04-25T09:04:15.972913+0000","last_clean":"2024-04-25T09:04:15.972913+0000","last_became_active":"2024-04-25T09:04:14.081603+0000","last_became_peered":"2024-04-25T09:04:14.081603+0000","last_unstale":"2024-04-25T09:04:15.972913+0000","last_undegraded":"2024-04-25T09:04:15.972913+0000","last_fullsized":"2024-04-25T09:04:15.972913+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:38.856113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.080065+0000","last_change":"2024-04-25T09:04:14.075544+0000","last_active":"2024-04-25T09:04:15.080065+0000","last_peered":"2024-04-25T09:04:15.080065+0000","last_clean":"2024-04-25T09:04:15.080065+0000","last_became_active":"2024-04-25T09:04:14.075227+0000","last_became_peered":"2024-04-25T09:04:14.075227+0000","last_unstale":"2024-04-25T09:04:15.080065+0000","last_undegraded":"2024-04-25T09:04:15.080065+0000","last_fullsized":"2024-04-25T09:04:15.080065+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:10.199748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.973273+0000","last_change":"2024-04-25T09:04:14.089180+0000","last_active":"2024-04-25T09:04:15.973273+0000","last_peered":"2024-04-25T09:04:15.973273+0000","last_clean":"2024-04-25T09:04:15.973273+0000","last_became_active":"2024-04-25T09:04:14.088853+0000","last_became_peered":"2024-04-25T09:04:14.088853+0000","last_unstale":"2024-04-25T09:04:15.973273+0000","last_undegraded":"2024-04-25T09:04:15.973273+0000","last_fullsized":"2024-04-25T09:04:15.973273+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:40.767694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.972766+0000","last_change":"2024-04-25T09:04:14.085084+0000","last_active":"2024-04-25T09:04:15.972766+0000","last_peered":"2024-04-25T09:04:15.972766+0000","last_clean":"2024-04-25T09:04:15.972766+0000","last_became_active":"2024-04-25T09:04:14.084503+0000","last_became_peered":"2024-04-25T09:04:14.084503+0000","last_unstale":"2024-04-25T09:04:15.972766+0000","last_undegraded":"2024-04-25T09:04:15.972766+0000","last_fullsized":"2024-04-25T09:04:15.972766+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:04:34.157594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.079881+0000","last_change":"2024-04-25T09:04:14.085720+0000","last_active":"2024-04-25T09:04:15.079881+0000","last_peered":"2024-04-25T09:04:15.079881+0000","last_clean":"2024-04-25T09:04:15.079881+0000","last_became_active":"2024-04-25T09:04:14.085435+0000","last_became_peered":"2024-04-25T09:04:14.085435+0000","last_unstale":"2024-04-25T09:04:15.079881+0000","last_undegraded":"2024-04-25T09:04:15.079881+0000","last_fullsized":"2024-04-25T09:04:15.079881+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:59.704426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.066828+0000","last_change":"2024-04-25T09:04:14.079941+0000","last_active":"2024-04-25T09:04:15.066828+0000","last_peered":"2024-04-25T09:04:15.066828+0000","last_clean":"2024-04-25T09:04:15.066828+0000","last_became_active":"2024-04-25T09:04:14.079284+0000","last_became_peered":"2024-04-25T09:04:14.079284+0000","last_unstale":"2024-04-25T09:04:15.066828+0000","last_undegraded":"2024-04-25T09:04:15.066828+0000","last_fullsized":"2024-04-25T09:04:15.066828+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:55.615005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.065443+0000","last_change":"2024-04-25T09:04:14.088240+0000","last_active":"2024-04-25T09:04:15.065443+0000","last_peered":"2024-04-25T09:04:15.065443+0000","last_clean":"2024-04-25T09:04:15.065443+0000","last_became_active":"2024-04-25T09:04:14.087852+0000","last_became_peered":"2024-04-25T09:04:14.087852+0000","last_unstale":"2024-04-25T09:04:15.065443+0000","last_undegraded":"2024-04-25T09:04:15.065443+0000","last_fullsized":"2024-04-25T09:04:15.065443+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:13.022823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.972802+0000","last_change":"2024-04-25T09:04:14.091219+0000","last_active":"2024-04-25T09:04:15.972802+0000","last_peered":"2024-04-25T09:04:15.972802+0000","last_clean":"2024-04-25T09:04:15.972802+0000","last_became_active":"2024-04-25T09:04:14.090885+0000","last_became_peered":"2024-04-25T09:04:14.090885+0000","last_unstale":"2024-04-25T09:04:15.972802+0000","last_undegraded":"2024-04-25T09:04:15.972802+0000","last_fullsized":"2024-04-25T09:04:15.972802+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:00.645271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.082598+0000","last_change":"2024-04-25T09:04:14.085195+0000","last_active":"2024-04-25T09:04:15.082598+0000","last_peered":"2024-04-25T09:04:15.082598+0000","last_clean":"2024-04-25T09:04:15.082598+0000","last_became_active":"2024-04-25T09:04:14.084887+0000","last_became_peered":"2024-04-25T09:04:14.084887+0000","last_unstale":"2024-04-25T09:04:15.082598+0000","last_undegraded":"2024-04-25T09:04:15.082598+0000","last_fullsized":"2024-04-25T09:04:15.082598+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:16.099287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"138'4","reported_seq":18,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.082649+0000","last_change":"2024-04-25T09:04:13.062156+0000","last_active":"2024-04-25T09:04:15.082649+0000","last_peered":"2024-04-25T09:04:15.082649+0000","last_clean":"2024-04-25T09:04:15.082649+0000","last_became_active":"2024-04-25T09:04:13.061852+0000","last_became_peered":"2024-04-25T09:04:13.061852+0000","last_unstale":"2024-04-25T09:04:15.082649+0000","last_undegraded":"2024-04-25T09:04:15.082649+0000","last_fullsized":"2024-04-25T09:04:15.082649+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:47.205245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.065810+0000","last_change":"2024-04-25T09:04:14.071565+0000","last_active":"2024-04-25T09:04:15.065810+0000","last_peered":"2024-04-25T09:04:15.065810+0000","last_clean":"2024-04-25T09:04:15.065810+0000","last_became_active":"2024-04-25T09:04:14.071136+0000","last_became_peered":"2024-04-25T09:04:14.071136+0000","last_unstale":"2024-04-25T09:04:15.065810+0000","last_undegraded":"2024-04-25T09:04:15.065810+0000","last_fullsized":"2024-04-25T09:04:15.065810+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:35:58.699528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"138'1","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.065769+0000","last_change":"2024-04-25T09:04:13.058762+0000","last_active":"2024-04-25T09:04:15.065769+0000","last_peered":"2024-04-25T09:04:15.065769+0000","last_clean":"2024-04-25T09:04:15.065769+0000","last_became_active":"2024-04-25T09:04:13.058335+0000","last_became_peered":"2024-04-25T09:04:13.058335+0000","last_unstale":"2024-04-25T09:04:15.065769+0000","last_undegraded":"2024-04-25T09:04:15.065769+0000","last_fullsized":"2024-04-25T09:04:15.065769+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:25.170530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.079887+0000","last_change":"2024-04-25T09:04:14.069764+0000","last_active":"2024-04-25T09:04:15.079887+0000","last_peered":"2024-04-25T09:04:15.079887+0000","last_clean":"2024-04-25T09:04:15.079887+0000","last_became_active":"2024-04-25T09:04:14.069400+0000","last_became_peered":"2024-04-25T09:04:14.069400+0000","last_unstale":"2024-04-25T09:04:15.079887+0000","last_undegraded":"2024-04-25T09:04:15.079887+0000","last_fullsized":"2024-04-25T09:04:15.079887+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:19:55.615005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"138'2","reported_seq":16,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.066303+0000","last_change":"2024-04-25T09:04:13.057742+0000","last_active":"2024-04-25T09:04:15.066303+0000","last_peered":"2024-04-25T09:04:15.066303+0000","last_clean":"2024-04-25T09:04:15.066303+0000","last_became_active":"2024-04-25T09:04:13.057448+0000","last_became_peered":"2024-04-25T09:04:13.057448+0000","last_unstale":"2024-04-25T09:04:15.066303+0000","last_undegraded":"2024-04-25T09:04:15.066303+0000","last_fullsized":"2024-04-25T09:04:15.066303+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:18.055900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.082422+0000","last_change":"2024-04-25T09:04:14.084882+0000","last_active":"2024-04-25T09:04:15.082422+0000","last_peered":"2024-04-25T09:04:15.082422+0000","last_clean":"2024-04-25T09:04:15.082422+0000","last_became_active":"2024-04-25T09:04:14.084554+0000","last_became_peered":"2024-04-25T09:04:14.084554+0000","last_unstale":"2024-04-25T09:04:15.082422+0000","last_undegraded":"2024-04-25T09:04:15.082422+0000","last_fullsized":"2024-04-25T09:04:15.082422+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:31:41.665545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"138'2","reported_seq":16,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.066613+0000","last_change":"2024-04-25T09:04:13.058406+0000","last_active":"2024-04-25T09:04:15.066613+0000","last_peered":"2024-04-25T09:04:15.066613+0000","last_clean":"2024-04-25T09:04:15.066613+0000","last_became_active":"2024-04-25T09:04:13.058144+0000","last_became_peered":"2024-04-25T09:04:13.058144+0000","last_unstale":"2024-04-25T09:04:15.066613+0000","last_undegraded":"2024-04-25T09:04:15.066613+0000","last_fullsized":"2024-04-25T09:04:15.066613+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:04.195913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.065549+0000","last_change":"2024-04-25T09:04:14.082247+0000","last_active":"2024-04-25T09:04:15.065549+0000","last_peered":"2024-04-25T09:04:15.065549+0000","last_clean":"2024-04-25T09:04:15.065549+0000","last_became_active":"2024-04-25T09:04:14.081924+0000","last_became_peered":"2024-04-25T09:04:14.081924+0000","last_unstale":"2024-04-25T09:04:15.065549+0000","last_undegraded":"2024-04-25T09:04:15.065549+0000","last_fullsized":"2024-04-25T09:04:15.065549+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:09.422195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"138'1","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.972772+0000","last_change":"2024-04-25T09:04:13.059157+0000","last_active":"2024-04-25T09:04:15.972772+0000","last_peered":"2024-04-25T09:04:15.972772+0000","last_clean":"2024-04-25T09:04:15.972772+0000","last_became_active":"2024-04-25T09:04:13.058883+0000","last_became_peered":"2024-04-25T09:04:13.058883+0000","last_unstale":"2024-04-25T09:04:15.972772+0000","last_undegraded":"2024-04-25T09:04:15.972772+0000","last_fullsized":"2024-04-25T09:04:15.972772+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:46.622314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.066602+0000","last_change":"2024-04-25T09:04:14.079706+0000","last_active":"2024-04-25T09:04:15.066602+0000","last_peered":"2024-04-25T09:04:15.066602+0000","last_clean":"2024-04-25T09:04:15.066602+0000","last_became_active":"2024-04-25T09:04:14.079432+0000","last_became_peered":"2024-04-25T09:04:14.079432+0000","last_unstale":"2024-04-25T09:04:15.066602+0000","last_undegraded":"2024-04-25T09:04:15.066602+0000","last_fullsized":"2024-04-25T09:04:15.066602+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:05.647799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"138'1","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.067243+0000","last_change":"2024-04-25T09:04:13.057593+0000","last_active":"2024-04-25T09:04:15.067243+0000","last_peered":"2024-04-25T09:04:15.067243+0000","last_clean":"2024-04-25T09:04:15.067243+0000","last_became_active":"2024-04-25T09:04:13.057330+0000","last_became_peered":"2024-04-25T09:04:13.057330+0000","last_unstale":"2024-04-25T09:04:15.067243+0000","last_undegraded":"2024-04-25T09:04:15.067243+0000","last_fullsized":"2024-04-25T09:04:15.067243+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:54:29.663603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.065498+0000","last_change":"2024-04-25T09:04:14.069887+0000","last_active":"2024-04-25T09:04:15.065498+0000","last_peered":"2024-04-25T09:04:15.065498+0000","last_clean":"2024-04-25T09:04:15.065498+0000","last_became_active":"2024-04-25T09:04:14.069490+0000","last_became_peered":"2024-04-25T09:04:14.069490+0000","last_unstale":"2024-04-25T09:04:15.065498+0000","last_undegraded":"2024-04-25T09:04:15.065498+0000","last_fullsized":"2024-04-25T09:04:15.065498+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:30.396008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":14,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.973196+0000","last_change":"2024-04-25T09:04:13.054808+0000","last_active":"2024-04-25T09:04:15.973196+0000","last_peered":"2024-04-25T09:04:15.973196+0000","last_clean":"2024-04-25T09:04:15.973196+0000","last_became_active":"2024-04-25T09:04:13.054486+0000","last_became_peered":"2024-04-25T09:04:13.054486+0000","last_unstale":"2024-04-25T09:04:15.973196+0000","last_undegraded":"2024-04-25T09:04:15.973196+0000","last_fullsized":"2024-04-25T09:04:15.973196+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:43.616200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.067007+0000","last_change":"2024-04-25T09:04:14.082255+0000","last_active":"2024-04-25T09:04:15.067007+0000","last_peered":"2024-04-25T09:04:15.067007+0000","last_clean":"2024-04-25T09:04:15.067007+0000","last_became_active":"2024-04-25T09:04:14.081727+0000","last_became_peered":"2024-04-25T09:04:14.081727+0000","last_unstale":"2024-04-25T09:04:15.067007+0000","last_undegraded":"2024-04-25T09:04:15.067007+0000","last_fullsized":"2024-04-25T09:04:15.067007+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:48.551079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":14,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.066627+0000","last_change":"2024-04-25T09:04:13.056215+0000","last_active":"2024-04-25T09:04:15.066627+0000","last_peered":"2024-04-25T09:04:15.066627+0000","last_clean":"2024-04-25T09:04:15.066627+0000","last_became_active":"2024-04-25T09:04:13.055952+0000","last_became_peered":"2024-04-25T09:04:13.055952+0000","last_unstale":"2024-04-25T09:04:15.066627+0000","last_undegraded":"2024-04-25T09:04:15.066627+0000","last_fullsized":"2024-04-25T09:04:15.066627+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:04.737828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.067297+0000","last_change":"2024-04-25T09:04:14.088019+0000","last_active":"2024-04-25T09:04:15.067297+0000","last_peered":"2024-04-25T09:04:15.067297+0000","last_clean":"2024-04-25T09:04:15.067297+0000","last_became_active":"2024-04-25T09:04:14.087497+0000","last_became_peered":"2024-04-25T09:04:14.087497+0000","last_unstale":"2024-04-25T09:04:15.067297+0000","last_undegraded":"2024-04-25T09:04:15.067297+0000","last_fullsized":"2024-04-25T09:04:15.067297+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:05.201104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":14,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.065844+0000","last_change":"2024-04-25T09:04:13.051625+0000","last_active":"2024-04-25T09:04:15.065844+0000","last_peered":"2024-04-25T09:04:15.065844+0000","last_clean":"2024-04-25T09:04:15.065844+0000","last_became_active":"2024-04-25T09:04:13.051307+0000","last_became_peered":"2024-04-25T09:04:13.051307+0000","last_unstale":"2024-04-25T09:04:15.065844+0000","last_undegraded":"2024-04-25T09:04:15.065844+0000","last_fullsized":"2024-04-25T09:04:15.065844+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:28.454737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.082669+0000","last_change":"2024-04-25T09:04:14.084265+0000","last_active":"2024-04-25T09:04:15.082669+0000","last_peered":"2024-04-25T09:04:15.082669+0000","last_clean":"2024-04-25T09:04:15.082669+0000","last_became_active":"2024-04-25T09:04:14.083865+0000","last_became_peered":"2024-04-25T09:04:14.083865+0000","last_unstale":"2024-04-25T09:04:15.082669+0000","last_undegraded":"2024-04-25T09:04:15.082669+0000","last_fullsized":"2024-04-25T09:04:15.082669+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:40.610032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":14,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.082797+0000","last_change":"2024-04-25T09:04:13.058585+0000","last_active":"2024-04-25T09:04:15.082797+0000","last_peered":"2024-04-25T09:04:15.082797+0000","last_clean":"2024-04-25T09:04:15.082797+0000","last_became_active":"2024-04-25T09:04:13.058298+0000","last_became_peered":"2024-04-25T09:04:13.058298+0000","last_unstale":"2024-04-25T09:04:15.082797+0000","last_undegraded":"2024-04-25T09:04:15.082797+0000","last_fullsized":"2024-04-25T09:04:15.082797+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:35.416425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.973329+0000","last_change":"2024-04-25T09:04:14.072638+0000","last_active":"2024-04-25T09:04:15.973329+0000","last_peered":"2024-04-25T09:04:15.973329+0000","last_clean":"2024-04-25T09:04:15.973329+0000","last_became_active":"2024-04-25T09:04:14.072322+0000","last_became_peered":"2024-04-25T09:04:14.072322+0000","last_unstale":"2024-04-25T09:04:15.973329+0000","last_undegraded":"2024-04-25T09:04:15.973329+0000","last_fullsized":"2024-04-25T09:04:15.973329+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:48.551079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"138'6","reported_seq":20,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.973219+0000","last_change":"2024-04-25T09:04:13.058981+0000","last_active":"2024-04-25T09:04:15.973219+0000","last_peered":"2024-04-25T09:04:15.973219+0000","last_clean":"2024-04-25T09:04:15.973219+0000","last_became_active":"2024-04-25T09:04:13.058662+0000","last_became_peered":"2024-04-25T09:04:13.058662+0000","last_unstale":"2024-04-25T09:04:15.973219+0000","last_undegraded":"2024-04-25T09:04:15.973219+0000","last_fullsized":"2024-04-25T09:04:15.973219+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:12.017633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.972840+0000","last_change":"2024-04-25T09:04:14.079259+0000","last_active":"2024-04-25T09:04:15.972840+0000","last_peered":"2024-04-25T09:04:15.972840+0000","last_clean":"2024-04-25T09:04:15.972840+0000","last_became_active":"2024-04-25T09:04:14.078896+0000","last_became_peered":"2024-04-25T09:04:14.078896+0000","last_unstale":"2024-04-25T09:04:15.972840+0000","last_undegraded":"2024-04-25T09:04:15.972840+0000","last_fullsized":"2024-04-25T09:04:15.972840+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:05.647799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"138'2","reported_seq":16,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.972893+0000","last_change":"2024-04-25T09:04:13.058867+0000","last_active":"2024-04-25T09:04:15.972893+0000","last_peered":"2024-04-25T09:04:15.972893+0000","last_clean":"2024-04-25T09:04:15.972893+0000","last_became_active":"2024-04-25T09:04:13.057622+0000","last_became_peered":"2024-04-25T09:04:13.057622+0000","last_unstale":"2024-04-25T09:04:15.972893+0000","last_undegraded":"2024-04-25T09:04:15.972893+0000","last_fullsized":"2024-04-25T09:04:15.972893+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:13.621908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.065488+0000","last_change":"2024-04-25T09:04:14.071041+0000","last_active":"2024-04-25T09:04:15.065488+0000","last_peered":"2024-04-25T09:04:15.065488+0000","last_clean":"2024-04-25T09:04:15.065488+0000","last_became_active":"2024-04-25T09:04:14.070706+0000","last_became_peered":"2024-04-25T09:04:14.070706+0000","last_unstale":"2024-04-25T09:04:15.065488+0000","last_undegraded":"2024-04-25T09:04:15.065488+0000","last_fullsized":"2024-04-25T09:04:15.065488+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:31.718398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":14,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.972827+0000","last_change":"2024-04-25T09:04:13.054034+0000","last_active":"2024-04-25T09:04:15.972827+0000","last_peered":"2024-04-25T09:04:15.972827+0000","last_clean":"2024-04-25T09:04:15.972827+0000","last_became_active":"2024-04-25T09:04:13.053760+0000","last_became_peered":"2024-04-25T09:04:13.053760+0000","last_unstale":"2024-04-25T09:04:15.972827+0000","last_undegraded":"2024-04-25T09:04:15.972827+0000","last_fullsized":"2024-04-25T09:04:15.972827+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:30.960717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.066982+0000","last_change":"2024-04-25T09:04:14.079824+0000","last_active":"2024-04-25T09:04:15.066982+0000","last_peered":"2024-04-25T09:04:15.066982+0000","last_clean":"2024-04-25T09:04:15.066982+0000","last_became_active":"2024-04-25T09:04:14.079547+0000","last_became_peered":"2024-04-25T09:04:14.079547+0000","last_unstale":"2024-04-25T09:04:15.066982+0000","last_undegraded":"2024-04-25T09:04:15.066982+0000","last_fullsized":"2024-04-25T09:04:15.066982+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:04:40.767694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":14,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.079645+0000","last_change":"2024-04-25T09:04:13.056393+0000","last_active":"2024-04-25T09:04:15.079645+0000","last_peered":"2024-04-25T09:04:15.079645+0000","last_clean":"2024-04-25T09:04:15.079645+0000","last_became_active":"2024-04-25T09:04:13.056111+0000","last_became_peered":"2024-04-25T09:04:13.056111+0000","last_unstale":"2024-04-25T09:04:15.079645+0000","last_undegraded":"2024-04-25T09:04:15.079645+0000","last_fullsized":"2024-04-25T09:04:15.079645+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:04.195913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.972624+0000","last_change":"2024-04-25T09:04:14.083554+0000","last_active":"2024-04-25T09:04:15.972624+0000","last_peered":"2024-04-25T09:04:15.972624+0000","last_clean":"2024-04-25T09:04:15.972624+0000","last_became_active":"2024-04-25T09:04:14.083168+0000","last_became_peered":"2024-04-25T09:04:14.083168+0000","last_unstale":"2024-04-25T09:04:15.972624+0000","last_undegraded":"2024-04-25T09:04:15.972624+0000","last_fullsized":"2024-04-25T09:04:15.972624+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:39.591985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"138'8","reported_seq":22,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.972838+0000","last_change":"2024-04-25T09:04:13.061556+0000","last_active":"2024-04-25T09:04:15.972838+0000","last_peered":"2024-04-25T09:04:15.972838+0000","last_clean":"2024-04-25T09:04:15.972838+0000","last_became_active":"2024-04-25T09:04:13.061241+0000","last_became_peered":"2024-04-25T09:04:13.061241+0000","last_unstale":"2024-04-25T09:04:15.972838+0000","last_undegraded":"2024-04-25T09:04:15.972838+0000","last_fullsized":"2024-04-25T09:04:15.972838+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:40.360745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.082766+0000","last_change":"2024-04-25T09:04:14.086915+0000","last_active":"2024-04-25T09:04:15.082766+0000","last_peered":"2024-04-25T09:04:15.082766+0000","last_clean":"2024-04-25T09:04:15.082766+0000","last_became_active":"2024-04-25T09:04:14.086596+0000","last_became_peered":"2024-04-25T09:04:14.086596+0000","last_unstale":"2024-04-25T09:04:15.082766+0000","last_undegraded":"2024-04-25T09:04:15.082766+0000","last_fullsized":"2024-04-25T09:04:15.082766+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:35.488341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"138'1","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.067096+0000","last_change":"2024-04-25T09:04:13.053984+0000","last_active":"2024-04-25T09:04:15.067096+0000","last_peered":"2024-04-25T09:04:15.067096+0000","last_clean":"2024-04-25T09:04:15.067096+0000","last_became_active":"2024-04-25T09:04:13.053588+0000","last_became_peered":"2024-04-25T09:04:13.053588+0000","last_unstale":"2024-04-25T09:04:15.067096+0000","last_undegraded":"2024-04-25T09:04:15.067096+0000","last_fullsized":"2024-04-25T09:04:15.067096+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:58:20.138535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.973004+0000","last_change":"2024-04-25T09:04:14.076507+0000","last_active":"2024-04-25T09:04:15.973004+0000","last_peered":"2024-04-25T09:04:15.973004+0000","last_clean":"2024-04-25T09:04:15.973004+0000","last_became_active":"2024-04-25T09:04:14.076191+0000","last_became_peered":"2024-04-25T09:04:14.076191+0000","last_unstale":"2024-04-25T09:04:15.973004+0000","last_undegraded":"2024-04-25T09:04:15.973004+0000","last_fullsized":"2024-04-25T09:04:15.973004+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:08.423829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"138'2","reported_seq":16,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.065843+0000","last_change":"2024-04-25T09:04:13.055593+0000","last_active":"2024-04-25T09:04:15.065843+0000","last_peered":"2024-04-25T09:04:15.065843+0000","last_clean":"2024-04-25T09:04:15.065843+0000","last_became_active":"2024-04-25T09:04:13.055293+0000","last_became_peered":"2024-04-25T09:04:13.055293+0000","last_unstale":"2024-04-25T09:04:15.065843+0000","last_undegraded":"2024-04-25T09:04:15.065843+0000","last_fullsized":"2024-04-25T09:04:15.065843+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:30:52.194928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.973005+0000","last_change":"2024-04-25T09:04:14.081779+0000","last_active":"2024-04-25T09:04:15.973005+0000","last_peered":"2024-04-25T09:04:15.973005+0000","last_clean":"2024-04-25T09:04:15.973005+0000","last_became_active":"2024-04-25T09:04:14.076387+0000","last_became_peered":"2024-04-25T09:04:14.076387+0000","last_unstale":"2024-04-25T09:04:15.973005+0000","last_undegraded":"2024-04-25T09:04:15.973005+0000","last_fullsized":"2024-04-25T09:04:15.973005+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:31.639139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"138'1","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.973439+0000","last_change":"2024-04-25T09:04:13.057835+0000","last_active":"2024-04-25T09:04:15.973439+0000","last_peered":"2024-04-25T09:04:15.973439+0000","last_clean":"2024-04-25T09:04:15.973439+0000","last_became_active":"2024-04-25T09:04:13.057562+0000","last_became_peered":"2024-04-25T09:04:13.057562+0000","last_unstale":"2024-04-25T09:04:15.973439+0000","last_undegraded":"2024-04-25T09:04:15.973439+0000","last_fullsized":"2024-04-25T09:04:15.973439+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:53.970216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.079918+0000","last_change":"2024-04-25T09:04:14.086797+0000","last_active":"2024-04-25T09:04:15.079918+0000","last_peered":"2024-04-25T09:04:15.079918+0000","last_clean":"2024-04-25T09:04:15.079918+0000","last_became_active":"2024-04-25T09:04:14.086502+0000","last_became_peered":"2024-04-25T09:04:14.086502+0000","last_unstale":"2024-04-25T09:04:15.079918+0000","last_undegraded":"2024-04-25T09:04:15.079918+0000","last_fullsized":"2024-04-25T09:04:15.079918+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:07.813806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":14,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.065572+0000","last_change":"2024-04-25T09:04:13.060717+0000","last_active":"2024-04-25T09:04:15.065572+0000","last_peered":"2024-04-25T09:04:15.065572+0000","last_clean":"2024-04-25T09:04:15.065572+0000","last_became_active":"2024-04-25T09:04:13.060060+0000","last_became_peered":"2024-04-25T09:04:13.060060+0000","last_unstale":"2024-04-25T09:04:15.065572+0000","last_undegraded":"2024-04-25T09:04:15.065572+0000","last_fullsized":"2024-04-25T09:04:15.065572+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:52.903859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.066952+0000","last_change":"2024-04-25T09:04:14.076966+0000","last_active":"2024-04-25T09:04:15.066952+0000","last_peered":"2024-04-25T09:04:15.066952+0000","last_clean":"2024-04-25T09:04:15.066952+0000","last_became_active":"2024-04-25T09:04:14.076670+0000","last_became_peered":"2024-04-25T09:04:14.076670+0000","last_unstale":"2024-04-25T09:04:15.066952+0000","last_undegraded":"2024-04-25T09:04:15.066952+0000","last_fullsized":"2024-04-25T09:04:15.066952+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:49.273567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"138'2","reported_seq":16,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.972833+0000","last_change":"2024-04-25T09:04:13.054317+0000","last_active":"2024-04-25T09:04:15.972833+0000","last_peered":"2024-04-25T09:04:15.972833+0000","last_clean":"2024-04-25T09:04:15.972833+0000","last_became_active":"2024-04-25T09:04:13.053816+0000","last_became_peered":"2024-04-25T09:04:13.053816+0000","last_unstale":"2024-04-25T09:04:15.972833+0000","last_undegraded":"2024-04-25T09:04:15.972833+0000","last_fullsized":"2024-04-25T09:04:15.972833+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:51:30.713207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.972827+0000","last_change":"2024-04-25T09:04:14.083330+0000","last_active":"2024-04-25T09:04:15.972827+0000","last_peered":"2024-04-25T09:04:15.972827+0000","last_clean":"2024-04-25T09:04:15.972827+0000","last_became_active":"2024-04-25T09:04:14.079880+0000","last_became_peered":"2024-04-25T09:04:14.079880+0000","last_unstale":"2024-04-25T09:04:15.972827+0000","last_undegraded":"2024-04-25T09:04:15.972827+0000","last_fullsized":"2024-04-25T09:04:15.972827+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:28.997133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":14,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.082156+0000","last_change":"2024-04-25T09:04:13.061478+0000","last_active":"2024-04-25T09:04:15.082156+0000","last_peered":"2024-04-25T09:04:15.082156+0000","last_clean":"2024-04-25T09:04:15.082156+0000","last_became_active":"2024-04-25T09:04:13.061150+0000","last_became_peered":"2024-04-25T09:04:13.061150+0000","last_unstale":"2024-04-25T09:04:15.082156+0000","last_undegraded":"2024-04-25T09:04:15.082156+0000","last_fullsized":"2024-04-25T09:04:15.082156+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:19.878381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.066272+0000","last_change":"2024-04-25T09:04:14.071712+0000","last_active":"2024-04-25T09:04:15.066272+0000","last_peered":"2024-04-25T09:04:15.066272+0000","last_clean":"2024-04-25T09:04:15.066272+0000","last_became_active":"2024-04-25T09:04:14.071106+0000","last_became_peered":"2024-04-25T09:04:14.071106+0000","last_unstale":"2024-04-25T09:04:15.066272+0000","last_undegraded":"2024-04-25T09:04:15.066272+0000","last_fullsized":"2024-04-25T09:04:15.066272+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:38:41.601898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":14,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.082041+0000","last_change":"2024-04-25T09:04:13.059762+0000","last_active":"2024-04-25T09:04:15.082041+0000","last_peered":"2024-04-25T09:04:15.082041+0000","last_clean":"2024-04-25T09:04:15.082041+0000","last_became_active":"2024-04-25T09:04:13.059482+0000","last_became_peered":"2024-04-25T09:04:13.059482+0000","last_unstale":"2024-04-25T09:04:15.082041+0000","last_undegraded":"2024-04-25T09:04:15.082041+0000","last_fullsized":"2024-04-25T09:04:15.082041+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:45.612511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.972721+0000","last_change":"2024-04-25T09:04:14.084984+0000","last_active":"2024-04-25T09:04:15.972721+0000","last_peered":"2024-04-25T09:04:15.972721+0000","last_clean":"2024-04-25T09:04:15.972721+0000","last_became_active":"2024-04-25T09:04:14.083645+0000","last_became_peered":"2024-04-25T09:04:14.083645+0000","last_unstale":"2024-04-25T09:04:15.972721+0000","last_undegraded":"2024-04-25T09:04:15.972721+0000","last_fullsized":"2024-04-25T09:04:15.972721+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:45.369609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":14,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.972530+0000","last_change":"2024-04-25T09:04:13.059953+0000","last_active":"2024-04-25T09:04:15.972530+0000","last_peered":"2024-04-25T09:04:15.972530+0000","last_clean":"2024-04-25T09:04:15.972530+0000","last_became_active":"2024-04-25T09:04:13.059633+0000","last_became_peered":"2024-04-25T09:04:13.059633+0000","last_unstale":"2024-04-25T09:04:15.972530+0000","last_undegraded":"2024-04-25T09:04:15.972530+0000","last_fullsized":"2024-04-25T09:04:15.972530+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:04.381512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.067318+0000","last_change":"2024-04-25T09:04:14.081394+0000","last_active":"2024-04-25T09:04:15.067318+0000","last_peered":"2024-04-25T09:04:15.067318+0000","last_clean":"2024-04-25T09:04:15.067318+0000","last_became_active":"2024-04-25T09:04:14.079795+0000","last_became_peered":"2024-04-25T09:04:14.079795+0000","last_unstale":"2024-04-25T09:04:15.067318+0000","last_undegraded":"2024-04-25T09:04:15.067318+0000","last_fullsized":"2024-04-25T09:04:15.067318+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:19.061090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"138'2","reported_seq":16,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.082773+0000","last_change":"2024-04-25T09:04:13.062137+0000","last_active":"2024-04-25T09:04:15.082773+0000","last_peered":"2024-04-25T09:04:15.082773+0000","last_clean":"2024-04-25T09:04:15.082773+0000","last_became_active":"2024-04-25T09:04:13.061505+0000","last_became_peered":"2024-04-25T09:04:13.061505+0000","last_unstale":"2024-04-25T09:04:15.082773+0000","last_undegraded":"2024-04-25T09:04:15.082773+0000","last_fullsized":"2024-04-25T09:04:15.082773+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:12.230718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.973151+0000","last_change":"2024-04-25T09:04:14.084452+0000","last_active":"2024-04-25T09:04:15.973151+0000","last_peered":"2024-04-25T09:04:15.973151+0000","last_clean":"2024-04-25T09:04:15.973151+0000","last_became_active":"2024-04-25T09:04:14.084148+0000","last_became_peered":"2024-04-25T09:04:14.084148+0000","last_unstale":"2024-04-25T09:04:15.973151+0000","last_undegraded":"2024-04-25T09:04:15.973151+0000","last_fullsized":"2024-04-25T09:04:15.973151+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:49.273567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"139'1","reported_seq":15,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.080066+0000","last_change":"2024-04-25T09:04:13.057849+0000","last_active":"2024-04-25T09:04:15.080066+0000","last_peered":"2024-04-25T09:04:15.080066+0000","last_clean":"2024-04-25T09:04:15.080066+0000","last_became_active":"2024-04-25T09:04:13.057586+0000","last_became_peered":"2024-04-25T09:04:13.057586+0000","last_unstale":"2024-04-25T09:04:15.080066+0000","last_undegraded":"2024-04-25T09:04:15.080066+0000","last_fullsized":"2024-04-25T09:04:15.080066+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:18.055900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.065592+0000","last_change":"2024-04-25T09:04:14.079825+0000","last_active":"2024-04-25T09:04:15.065592+0000","last_peered":"2024-04-25T09:04:15.065592+0000","last_clean":"2024-04-25T09:04:15.065592+0000","last_became_active":"2024-04-25T09:04:14.079450+0000","last_became_peered":"2024-04-25T09:04:14.079450+0000","last_unstale":"2024-04-25T09:04:15.065592+0000","last_undegraded":"2024-04-25T09:04:15.065592+0000","last_fullsized":"2024-04-25T09:04:15.065592+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:32.141928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":14,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.080283+0000","last_change":"2024-04-25T09:04:13.056588+0000","last_active":"2024-04-25T09:04:15.080283+0000","last_peered":"2024-04-25T09:04:15.080283+0000","last_clean":"2024-04-25T09:04:15.080283+0000","last_became_active":"2024-04-25T09:04:13.055410+0000","last_became_peered":"2024-04-25T09:04:13.055410+0000","last_unstale":"2024-04-25T09:04:15.080283+0000","last_undegraded":"2024-04-25T09:04:15.080283+0000","last_fullsized":"2024-04-25T09:04:15.080283+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:04.381512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.080103+0000","last_change":"2024-04-25T09:04:14.069939+0000","last_active":"2024-04-25T09:04:15.080103+0000","last_peered":"2024-04-25T09:04:15.080103+0000","last_clean":"2024-04-25T09:04:15.080103+0000","last_became_active":"2024-04-25T09:04:14.069594+0000","last_became_peered":"2024-04-25T09:04:14.069594+0000","last_unstale":"2024-04-25T09:04:15.080103+0000","last_undegraded":"2024-04-25T09:04:15.080103+0000","last_fullsized":"2024-04-25T09:04:15.080103+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:25.702777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":14,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.079899+0000","last_change":"2024-04-25T09:04:13.058491+0000","last_active":"2024-04-25T09:04:15.079899+0000","last_peered":"2024-04-25T09:04:15.079899+0000","last_clean":"2024-04-25T09:04:15.079899+0000","last_became_active":"2024-04-25T09:04:13.058215+0000","last_became_peered":"2024-04-25T09:04:13.058215+0000","last_unstale":"2024-04-25T09:04:15.079899+0000","last_undegraded":"2024-04-25T09:04:15.079899+0000","last_fullsized":"2024-04-25T09:04:15.079899+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:13.621908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.972901+0000","last_change":"2024-04-25T09:04:14.081167+0000","last_active":"2024-04-25T09:04:15.972901+0000","last_peered":"2024-04-25T09:04:15.972901+0000","last_clean":"2024-04-25T09:04:15.972901+0000","last_became_active":"2024-04-25T09:04:14.080388+0000","last_became_peered":"2024-04-25T09:04:14.080388+0000","last_unstale":"2024-04-25T09:04:15.972901+0000","last_undegraded":"2024-04-25T09:04:15.972901+0000","last_fullsized":"2024-04-25T09:04:15.972901+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:16.791495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":14,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.972933+0000","last_change":"2024-04-25T09:04:13.054092+0000","last_active":"2024-04-25T09:04:15.972933+0000","last_peered":"2024-04-25T09:04:15.972933+0000","last_clean":"2024-04-25T09:04:15.972933+0000","last_became_active":"2024-04-25T09:04:13.053427+0000","last_became_peered":"2024-04-25T09:04:13.053427+0000","last_unstale":"2024-04-25T09:04:15.972933+0000","last_undegraded":"2024-04-25T09:04:15.972933+0000","last_fullsized":"2024-04-25T09:04:15.972933+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:29.390818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.067423+0000","last_change":"2024-04-25T09:04:14.088120+0000","last_active":"2024-04-25T09:04:15.067423+0000","last_peered":"2024-04-25T09:04:15.067423+0000","last_clean":"2024-04-25T09:04:15.067423+0000","last_became_active":"2024-04-25T09:04:14.087802+0000","last_became_peered":"2024-04-25T09:04:14.087802+0000","last_unstale":"2024-04-25T09:04:15.067423+0000","last_undegraded":"2024-04-25T09:04:15.067423+0000","last_fullsized":"2024-04-25T09:04:15.067423+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:05.386703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":14,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.082196+0000","last_change":"2024-04-25T09:04:13.060816+0000","last_active":"2024-04-25T09:04:15.082196+0000","last_peered":"2024-04-25T09:04:15.082196+0000","last_clean":"2024-04-25T09:04:15.082196+0000","last_became_active":"2024-04-25T09:04:13.060535+0000","last_became_peered":"2024-04-25T09:04:13.060535+0000","last_unstale":"2024-04-25T09:04:15.082196+0000","last_undegraded":"2024-04-25T09:04:15.082196+0000","last_fullsized":"2024-04-25T09:04:15.082196+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:19:16.484903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.082139+0000","last_change":"2024-04-25T09:04:14.083652+0000","last_active":"2024-04-25T09:04:15.082139+0000","last_peered":"2024-04-25T09:04:15.082139+0000","last_clean":"2024-04-25T09:04:15.082139+0000","last_became_active":"2024-04-25T09:04:14.082815+0000","last_became_peered":"2024-04-25T09:04:14.082815+0000","last_unstale":"2024-04-25T09:04:15.082139+0000","last_undegraded":"2024-04-25T09:04:15.082139+0000","last_fullsized":"2024-04-25T09:04:15.082139+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:40.149030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":14,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.972630+0000","last_change":"2024-04-25T09:04:13.059109+0000","last_active":"2024-04-25T09:04:15.972630+0000","last_peered":"2024-04-25T09:04:15.972630+0000","last_clean":"2024-04-25T09:04:15.972630+0000","last_became_active":"2024-04-25T09:04:13.058224+0000","last_became_peered":"2024-04-25T09:04:13.058224+0000","last_unstale":"2024-04-25T09:04:15.972630+0000","last_undegraded":"2024-04-25T09:04:15.972630+0000","last_fullsized":"2024-04-25T09:04:15.972630+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:12.452725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.067151+0000","last_change":"2024-04-25T09:04:14.077194+0000","last_active":"2024-04-25T09:04:15.067151+0000","last_peered":"2024-04-25T09:04:15.067151+0000","last_clean":"2024-04-25T09:04:15.067151+0000","last_became_active":"2024-04-25T09:04:14.076875+0000","last_became_peered":"2024-04-25T09:04:14.076875+0000","last_unstale":"2024-04-25T09:04:15.067151+0000","last_undegraded":"2024-04-25T09:04:15.067151+0000","last_fullsized":"2024-04-25T09:04:15.067151+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:09:34.066133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":14,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.065405+0000","last_change":"2024-04-25T09:04:13.057904+0000","last_active":"2024-04-25T09:04:15.065405+0000","last_peered":"2024-04-25T09:04:15.065405+0000","last_clean":"2024-04-25T09:04:15.065405+0000","last_became_active":"2024-04-25T09:04:13.057602+0000","last_became_peered":"2024-04-25T09:04:13.057602+0000","last_unstale":"2024-04-25T09:04:15.065405+0000","last_undegraded":"2024-04-25T09:04:15.065405+0000","last_fullsized":"2024-04-25T09:04:15.065405+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:49.426430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.973113+0000","last_change":"2024-04-25T09:04:14.071303+0000","last_active":"2024-04-25T09:04:15.973113+0000","last_peered":"2024-04-25T09:04:15.973113+0000","last_clean":"2024-04-25T09:04:15.973113+0000","last_became_active":"2024-04-25T09:04:14.070969+0000","last_became_peered":"2024-04-25T09:04:14.070969+0000","last_unstale":"2024-04-25T09:04:15.973113+0000","last_undegraded":"2024-04-25T09:04:15.973113+0000","last_fullsized":"2024-04-25T09:04:15.973113+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:13.034365+0000","last_clean_scrub_stamp":"2024-04-25T09:04:13.034365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T09:59:40.149030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"138'3","reported_seq":17,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.973036+0000","last_change":"2024-04-25T09:04:13.053009+0000","last_active":"2024-04-25T09:04:15.973036+0000","last_peered":"2024-04-25T09:04:15.973036+0000","last_clean":"2024-04-25T09:04:15.973036+0000","last_became_active":"2024-04-25T09:04:13.052708+0000","last_became_peered":"2024-04-25T09:04:13.052708+0000","last_unstale":"2024-04-25T09:04:15.973036+0000","last_undegraded":"2024-04-25T09:04:15.973036+0000","last_fullsized":"2024-04-25T09:04:15.973036+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T09:04:12.029175+0000","last_clean_scrub_stamp":"2024-04-25T09:04:12.029175+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:36.841563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":392,"reported_epoch":139,"state":"active+clean","last_fresh":"2024-04-25T09:04:15.082323+0000","last_change":"2024-04-25T08:54:00.818044+0000","last_active":"2024-04-25T09:04:15.082323+0000","last_peered":"2024-04-25T09:04:15.082323+0000","last_clean":"2024-04-25T09:04:15.082323+0000","last_became_active":"2024-04-25T08:54:00.817098+0000","last_became_peered":"2024-04-25T08:54:00.817098+0000","last_unstale":"2024-04-25T09:04:15.082323+0000","last_undegraded":"2024-04-25T09:04:15.082323+0000","last_fullsized":"2024-04-25T09:04:15.082323+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_clean_scrub_stamp":"2024-04-25T08:53:59.739677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:48.129513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574974,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47356,"kb_used_data":2288,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324484,"statfs":{"total":96636764160,"available":96588271616,"internally_reserved":0,"allocated":2342912,"data_stored":1353862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574975,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68988,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302852,"statfs":{"total":96636764160,"available":96566120448,"internally_reserved":0,"allocated":1753088,"data_stored":762988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574977,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55100,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316740,"statfs":{"total":96636764160,"available":96580341760,"internally_reserved":0,"allocated":1744896,"data_stored":762864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574975,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68980,"kb_used_data":1700,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302860,"statfs":{"total":96636764160,"available":96566128640,"internally_reserved":0,"allocated":1740800,"data_stored":762842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574975,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77300,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294540,"statfs":{"total":96636764160,"available":96557608960,"internally_reserved":0,"allocated":1744896,"data_stored":762932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574975,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46784,"kb_used_data":1716,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325056,"statfs":{"total":96636764160,"available":96588857344,"internally_reserved":0,"allocated":1757184,"data_stored":764524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574975,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82876,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94288964,"statfs":{"total":96636764160,"available":96551899136,"internally_reserved":0,"allocated":1744896,"data_stored":763424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574976,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51524,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320316,"statfs":{"total":96636764160,"available":96584003584,"internally_reserved":0,"allocated":2351104,"data_stored":1354856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:04:21.820 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T09:04:21.820 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T09:04:21.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.975+0000 7f987ab3b640 1 Processor -- start 2024-04-25T09:04:21.964 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.975+0000 7f987ab3b640 1 -- start start 2024-04-25T09:04:21.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.976+0000 7f987ab3b640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f98740cc6b0 0x7f987406d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:21.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.976+0000 7f987ab3b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f987415e3c0 con 0x7f987415a6b0 2024-04-25T09:04:21.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.976+0000 7f987ab3b640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f987415e680 con 0x7f987406da30 2024-04-25T09:04:21.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.976+0000 7f987ab3b640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f987406de60 con 0x7f98740cc6b0 2024-04-25T09:04:21.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.976+0000 7f98788d7640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f987415a6b0 0x7f98740ae9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49468/0 (socket says 172.21.15.43:49468) 2024-04-25T09:04:21.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.976+0000 7f98788d7640 1 -- 172.21.15.43:0/4175535754 learned_addr learned my addr 172.21.15.43:0/4175535754 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:21.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.976+0000 7f98737fe640 1 -- 172.21.15.43:0/4175535754 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 174714831 0 0) 0x7f987415e3c0 con 0x7f987415a6b0 2024-04-25T09:04:21.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.976+0000 7f98737fe640 1 -- 172.21.15.43:0/4175535754 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9860003520 con 0x7f987415a6b0 2024-04-25T09:04:21.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.976+0000 7f9873fff640 1 --2- 172.21.15.43:0/4175535754 >> v2:172.21.15.193:3300/0 conn(0x7f98740cc6b0 0x7f987406d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:21.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.977+0000 7f98737fe640 1 -- 172.21.15.43:0/4175535754 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 109311093 0 0) 0x7f9860003520 con 0x7f987415a6b0 2024-04-25T09:04:21.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.977+0000 7f98737fe640 1 -- 172.21.15.43:0/4175535754 >> v2:172.21.15.193:3300/0 conn(0x7f98740cc6b0 msgr2=0x7f987406d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:21.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.977+0000 7f98737fe640 1 --2- 172.21.15.43:0/4175535754 >> v2:172.21.15.193:3300/0 conn(0x7f98740cc6b0 0x7f987406d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:21.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.977+0000 7f98737fe640 1 -- 172.21.15.43:0/4175535754 >> v1:172.21.15.193:6789/0 conn(0x7f987406da30 legacy=0x7f98740b35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:21.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.977+0000 7f98737fe640 1 -- 172.21.15.43:0/4175535754 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f987415e960 con 0x7f987415a6b0 2024-04-25T09:04:21.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.977+0000 7f98737fe640 1 -- 172.21.15.43:0/4175535754 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f986c0019f0 con 0x7f987415a6b0 2024-04-25T09:04:21.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.977+0000 7f98737fe640 1 -- 172.21.15.43:0/4175535754 >> v1:172.21.15.43:6789/0 conn(0x7f987415a6b0 legacy=0x7f98740ae9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:21.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.977+0000 7f98737fe640 1 --2- 172.21.15.43:0/4175535754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98600039b0 0x7f9860003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:21.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.977+0000 7f98737fe640 1 --2- 172.21.15.43:0/4175535754 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9860004800 0x7f9860004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:21.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.977+0000 7f98737fe640 1 --2- 172.21.15.43:0/4175535754 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f98600051c0 0x7f986000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:21.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.977+0000 7f98737fe640 1 -- 172.21.15.43:0/4175535754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987406de60 con 0x7f98600039b0 2024-04-25T09:04:21.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.977+0000 7f98737fe640 1 -- 172.21.15.43:0/4175535754 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986000c060 con 0x7f9860004800 2024-04-25T09:04:21.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.977+0000 7f98737fe640 1 -- 172.21.15.43:0/4175535754 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f986000c260 con 0x7f98600051c0 2024-04-25T09:04:21.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.977+0000 7f98788d7640 1 --2- 172.21.15.43:0/4175535754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98600039b0 0x7f9860003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:21.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.977+0000 7f9873fff640 1 --2- 172.21.15.43:0/4175535754 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9860004800 0x7f9860004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:21.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.977+0000 7f98790d8640 1 --2- 172.21.15.43:0/4175535754 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f98600051c0 0x7f986000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:21.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.977+0000 7f98788d7640 1 -- 172.21.15.43:0/4175535754 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f98600051c0 msgr2=0x7f986000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:21.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.977+0000 7f98788d7640 1 --2- 172.21.15.43:0/4175535754 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f98600051c0 0x7f986000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:21.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.977+0000 7f98788d7640 1 -- 172.21.15.43:0/4175535754 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9860004800 msgr2=0x7f9860004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:21.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.977+0000 7f98788d7640 1 --2- 172.21.15.43:0/4175535754 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f9860004800 0x7f9860004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:21.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.977+0000 7f98788d7640 1 -- 172.21.15.43:0/4175535754 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9860003520 con 0x7f98600039b0 2024-04-25T09:04:21.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.977+0000 7f98790d8640 1 -- 172.21.15.43:0/4175535754 reap_dead start 2024-04-25T09:04:21.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.978+0000 7f98788d7640 1 --2- 172.21.15.43:0/4175535754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98600039b0 0x7f9860003dd0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f986c00a840 tx=0x7f9860005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1302a70bda78f24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:21.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.978+0000 7f98737fe640 1 -- 172.21.15.43:0/4175535754 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c00a460 con 0x7f98600039b0 2024-04-25T09:04:21.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.978+0000 7f98737fe640 1 -- 172.21.15.43:0/4175535754 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f986c02f070 con 0x7f98600039b0 2024-04-25T09:04:21.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.978+0000 7f98737fe640 1 -- 172.21.15.43:0/4175535754 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c02bcd0 con 0x7f98600039b0 2024-04-25T09:04:21.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.978+0000 7f987ab3b640 1 -- 172.21.15.43:0/4175535754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98600039b0 msgr2=0x7f9860003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:21.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.978+0000 7f987ab3b640 1 --2- 172.21.15.43:0/4175535754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98600039b0 0x7f9860003dd0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f986c00a840 tx=0x7f9860005830 comp rx=0 tx=0).stop 2024-04-25T09:04:21.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.978+0000 7f987ab3b640 1 -- 172.21.15.43:0/4175535754 shutdown_connections 2024-04-25T09:04:21.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.979+0000 7f987ab3b640 1 --2- 172.21.15.43:0/4175535754 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98600039b0 0x7f9860003dd0 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:21.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.979+0000 7f987ab3b640 1 -- 172.21.15.43:0/4175535754 >> 172.21.15.43:0/4175535754 conn(0x7f987406ab30 msgr2=0x7f987406af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:21.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.979+0000 7f987ab3b640 1 -- 172.21.15.43:0/4175535754 shutdown_connections 2024-04-25T09:04:21.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.979+0000 7f987ab3b640 1 -- 172.21.15.43:0/4175535754 wait complete. 2024-04-25T09:04:21.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.980+0000 7f987ab3b640 1 Processor -- start 2024-04-25T09:04:21.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.980+0000 7f987ab3b640 1 -- start start 2024-04-25T09:04:21.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.981+0000 7f987ab3b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98600039b0 0x7f98741f1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:21.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.981+0000 7f987ab3b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f987414f3b0 0x7f987414bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:21.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.981+0000 7f987ab3b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f98740b0d60 0x7f98740b11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:21.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.981+0000 7f987ab3b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987415f200 con 0x7f98600039b0 2024-04-25T09:04:21.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.981+0000 7f987ab3b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f987415ef20 con 0x7f987414f3b0 2024-04-25T09:04:21.973 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.981+0000 7f987ab3b640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98740ccaf0 con 0x7f98740b0d60 2024-04-25T09:04:21.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.981+0000 7f98788d7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98600039b0 0x7f98741f1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:21.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.981+0000 7f98788d7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98600039b0 0x7f98741f1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38834/0 (socket says 172.21.15.43:38834) 2024-04-25T09:04:21.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.981+0000 7f98788d7640 1 -- 172.21.15.43:0/3015005136 learned_addr learned my addr 172.21.15.43:0/3015005136 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:21.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.981+0000 7f9873fff640 1 --2- 172.21.15.43:0/3015005136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f987414f3b0 0x7f987414bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:21.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.981+0000 7f98790d8640 1 --2- 172.21.15.43:0/3015005136 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f98740b0d60 0x7f98740b11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:21.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.981+0000 7f98788d7640 1 -- 172.21.15.43:0/3015005136 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f98740b0d60 msgr2=0x7f98740b11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:21.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.981+0000 7f98788d7640 1 --2- 172.21.15.43:0/3015005136 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f98740b0d60 0x7f98740b11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:21.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.981+0000 7f98788d7640 1 -- 172.21.15.43:0/3015005136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f987414f3b0 msgr2=0x7f987414bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:21.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.981+0000 7f98788d7640 1 --2- 172.21.15.43:0/3015005136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f987414f3b0 0x7f987414bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:21.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.981+0000 7f98788d7640 1 -- 172.21.15.43:0/3015005136 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9874150650 con 0x7f98600039b0 2024-04-25T09:04:21.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.981+0000 7f9873fff640 1 --2- 172.21.15.43:0/3015005136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f987414f3b0 0x7f987414bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:21.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.981+0000 7f98788d7640 1 --2- 172.21.15.43:0/3015005136 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98600039b0 0x7f98741f1c40 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f986c00a960 tx=0x7f98740cc460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:21.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.981+0000 7f98717fa640 1 -- 172.21.15.43:0/3015005136 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c001fb0 con 0x7f98600039b0 2024-04-25T09:04:21.974 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.981+0000 7f98717fa640 1 -- 172.21.15.43:0/3015005136 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f986c034670 con 0x7f98600039b0 2024-04-25T09:04:21.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.981+0000 7f987ab3b640 1 -- 172.21.15.43:0/3015005136 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9874150930 con 0x7f98600039b0 2024-04-25T09:04:21.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.982+0000 7f98717fa640 1 -- 172.21.15.43:0/3015005136 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c034a20 con 0x7f98600039b0 2024-04-25T09:04:21.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.982+0000 7f987ab3b640 1 -- 172.21.15.43:0/3015005136 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f987414c140 con 0x7f98600039b0 2024-04-25T09:04:21.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.982+0000 7f987ab3b640 1 -- 172.21.15.43:0/3015005136 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9874072100 con 0x7f98600039b0 2024-04-25T09:04:21.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.983+0000 7f98717fa640 1 -- 172.21.15.43:0/3015005136 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f986c034bc0 con 0x7f98600039b0 2024-04-25T09:04:21.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.983+0000 7f98717fa640 1 --2- 172.21.15.43:0/3015005136 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f985406b500 0x7f985406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:21.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.983+0000 7f98717fa640 1 -- 172.21.15.43:0/3015005136 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f986c0aa4c0 con 0x7f98600039b0 2024-04-25T09:04:21.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.984+0000 7f9873fff640 1 --2- 172.21.15.43:0/3015005136 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f985406b500 0x7f985406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:21.975 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.984+0000 7f9873fff640 1 --2- 172.21.15.43:0/3015005136 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f985406b500 0x7f985406d9c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f987406cc80 tx=0x7f9864008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:21.976 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:21.988+0000 7f98717fa640 1 -- 172.21.15.43:0/3015005136 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9874072100 con 0x7f98600039b0 2024-04-25T09:04:22.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.239+0000 7f987ab3b640 1 -- 172.21.15.43:0/3015005136 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f98740b1c00 con 0x7f98600039b0 2024-04-25T09:04:22.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.241+0000 7f98717fa640 1 -- 172.21.15.43:0/3015005136 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+147 (secure 0 0 0) 0x7f98740b1c00 con 0x7f98600039b0 2024-04-25T09:04:22.230 INFO:teuthology.orchestra.run.smithi043.stdout:[client.0] 2024-04-25T09:04:22.230 INFO:teuthology.orchestra.run.smithi043.stdout: key = AQD8GSpmVOjWGxAAFNmxeGi0h2D//pa6Ne3eQw== 2024-04-25T09:04:22.230 INFO:teuthology.orchestra.run.smithi043.stdout: caps mds = "allow" 2024-04-25T09:04:22.230 INFO:teuthology.orchestra.run.smithi043.stdout: caps mon = "allow r" 2024-04-25T09:04:22.230 INFO:teuthology.orchestra.run.smithi043.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T09:04:22.231 INFO:teuthology.orchestra.run.smithi043.stderr:updated caps for client.0 2024-04-25T09:04:22.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.244+0000 7f987ab3b640 1 -- 172.21.15.43:0/3015005136 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f985406b500 msgr2=0x7f985406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.244+0000 7f987ab3b640 1 --2- 172.21.15.43:0/3015005136 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f985406b500 0x7f985406d9c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f987406cc80 tx=0x7f9864008040 comp rx=0 tx=0).stop 2024-04-25T09:04:22.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.244+0000 7f987ab3b640 1 -- 172.21.15.43:0/3015005136 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98600039b0 msgr2=0x7f98741f1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.244+0000 7f987ab3b640 1 --2- 172.21.15.43:0/3015005136 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98600039b0 0x7f98741f1c40 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f986c00a960 tx=0x7f98740cc460 comp rx=0 tx=0).stop 2024-04-25T09:04:22.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.244+0000 7f987ab3b640 1 -- 172.21.15.43:0/3015005136 shutdown_connections 2024-04-25T09:04:22.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.244+0000 7f987ab3b640 1 --2- 172.21.15.43:0/3015005136 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f98740b0d60 0x7f98740b11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.244+0000 7f987ab3b640 1 --2- 172.21.15.43:0/3015005136 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f985406b500 0x7f985406d9c0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.244+0000 7f987ab3b640 1 --2- 172.21.15.43:0/3015005136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f987414f3b0 0x7f987414bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.244+0000 7f987ab3b640 1 --2- 172.21.15.43:0/3015005136 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f98600039b0 0x7f98741f1c40 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.244+0000 7f987ab3b640 1 -- 172.21.15.43:0/3015005136 >> 172.21.15.43:0/3015005136 conn(0x7f987406ab30 msgr2=0x7f9874158d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:22.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.244+0000 7f987ab3b640 1 -- 172.21.15.43:0/3015005136 shutdown_connections 2024-04-25T09:04:22.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.244+0000 7f987ab3b640 1 -- 172.21.15.43:0/3015005136 wait complete. 2024-04-25T09:04:22.243 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:04:22.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.355+0000 7f9074b3b640 1 Processor -- start 2024-04-25T09:04:22.344 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.355+0000 7f9074b3b640 1 -- start start 2024-04-25T09:04:22.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.356+0000 7f9074b3b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f907015df70 0x7f90700cc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.356+0000 7f9074b3b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90700cbee0 con 0x7f90700b0870 2024-04-25T09:04:22.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.356+0000 7f9074b3b640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90700d0650 con 0x7f907006c510 2024-04-25T09:04:22.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.356+0000 7f9074b3b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90700d0400 con 0x7f907015df70 2024-04-25T09:04:22.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.356+0000 7f906dd9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f907015df70 0x7f90700cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:22.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.356+0000 7f906ed9d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f90700b0870 0x7f90700ccbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:49478/0 (socket says 172.21.15.43:49478) 2024-04-25T09:04:22.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.356+0000 7f906ed9d640 1 -- 172.21.15.43:0/723102696 learned_addr learned my addr 172.21.15.43:0/723102696 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:22.345 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.357+0000 7f906dd9b640 1 -- 172.21.15.43:0/723102696 >> v1:172.21.15.193:6789/0 conn(0x7f907006c510 legacy=0x7f907006c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:22.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.357+0000 7f906dd9b640 1 -- 172.21.15.43:0/723102696 >> v1:172.21.15.43:6789/0 conn(0x7f90700b0870 legacy=0x7f90700ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.357+0000 7f906dd9b640 1 -- 172.21.15.43:0/723102696 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90700d0930 con 0x7f907015df70 2024-04-25T09:04:22.346 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.357+0000 7f906d59a640 1 -- 172.21.15.43:0/723102696 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2980434808 0 0) 0x7f90700cbee0 con 0x7f90700b0870 2024-04-25T09:04:22.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.357+0000 7f906dd9b640 1 --2- 172.21.15.43:0/723102696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f907015df70 0x7f90700cc670 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f90580097d0 tx=0x7f905802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7978e04c77b9c075 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:22.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.357+0000 7f906d59a640 1 -- 172.21.15.43:0/723102696 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9058037070 con 0x7f907015df70 2024-04-25T09:04:22.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.357+0000 7f906d59a640 1 -- 172.21.15.43:0/723102696 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9058032440 con 0x7f907015df70 2024-04-25T09:04:22.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.357+0000 7f906d59a640 1 -- 172.21.15.43:0/723102696 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9058032810 con 0x7f907015df70 2024-04-25T09:04:22.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.358+0000 7f9074b3b640 1 -- 172.21.15.43:0/723102696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f907015df70 msgr2=0x7f90700cc670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.358+0000 7f9074b3b640 1 --2- 172.21.15.43:0/723102696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f907015df70 0x7f90700cc670 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f90580097d0 tx=0x7f905802a1c0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.358+0000 7f9074b3b640 1 -- 172.21.15.43:0/723102696 shutdown_connections 2024-04-25T09:04:22.347 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.358+0000 7f9074b3b640 1 --2- 172.21.15.43:0/723102696 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f907015df70 0x7f90700cc670 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.358+0000 7f9074b3b640 1 -- 172.21.15.43:0/723102696 >> 172.21.15.43:0/723102696 conn(0x7f90700d17f0 msgr2=0x7f90700d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:22.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.358+0000 7f9074b3b640 1 -- 172.21.15.43:0/723102696 shutdown_connections 2024-04-25T09:04:22.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.358+0000 7f9074b3b640 1 -- 172.21.15.43:0/723102696 wait complete. 2024-04-25T09:04:22.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.359+0000 7f9074b3b640 1 Processor -- start 2024-04-25T09:04:22.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.359+0000 7f9074b3b640 1 -- start start 2024-04-25T09:04:22.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.360+0000 7f9074b3b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f907006c510 0x7f90701f44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.360+0000 7f9074b3b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90700b0870 0x7f90700cae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.360+0000 7f9074b3b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f907015df70 0x7f90700c6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.360+0000 7f9074b3b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907006bd10 con 0x7f90700b0870 2024-04-25T09:04:22.350 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.360+0000 7f9074b3b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90700d0c10 con 0x7f907015df70 2024-04-25T09:04:22.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.360+0000 7f9074b3b640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f907006be90 con 0x7f907006c510 2024-04-25T09:04:22.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.360+0000 7f906dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90700b0870 0x7f90700cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:22.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.360+0000 7f906dd9b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90700b0870 0x7f90700cae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38858/0 (socket says 172.21.15.43:38858) 2024-04-25T09:04:22.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.360+0000 7f906dd9b640 1 -- 172.21.15.43:0/123577445 learned_addr learned my addr 172.21.15.43:0/123577445 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:22.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.360+0000 7f906ed9d640 1 --2- 172.21.15.43:0/123577445 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f907015df70 0x7f90700c6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:22.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.360+0000 7f906e59c640 1 --2- 172.21.15.43:0/123577445 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f907006c510 0x7f90701f44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:22.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.361+0000 7f906dd9b640 1 -- 172.21.15.43:0/123577445 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f907006c510 msgr2=0x7f90701f44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.361+0000 7f906dd9b640 1 --2- 172.21.15.43:0/123577445 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f907006c510 0x7f90701f44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.361+0000 7f906dd9b640 1 -- 172.21.15.43:0/123577445 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f907015df70 msgr2=0x7f90700c6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.361+0000 7f906dd9b640 1 --2- 172.21.15.43:0/123577445 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f907015df70 0x7f90700c6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.361+0000 7f906dd9b640 1 -- 172.21.15.43:0/123577445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90700b3da0 con 0x7f90700b0870 2024-04-25T09:04:22.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.361+0000 7f906ed9d640 1 --2- 172.21.15.43:0/123577445 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f907015df70 0x7f90700c6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:22.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.361+0000 7f906dd9b640 1 --2- 172.21.15.43:0/123577445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90700b0870 0x7f90700cae70 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f9058035040 tx=0x7f90580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:22.351 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.361+0000 7f9066ffd640 1 -- 172.21.15.43:0/123577445 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9058040020 con 0x7f90700b0870 2024-04-25T09:04:22.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.361+0000 7f9066ffd640 1 -- 172.21.15.43:0/123577445 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9058038040 con 0x7f90700b0870 2024-04-25T09:04:22.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.361+0000 7f9074b3b640 1 -- 172.21.15.43:0/123577445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90700ca0b0 con 0x7f90700b0870 2024-04-25T09:04:22.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.361+0000 7f9066ffd640 1 -- 172.21.15.43:0/123577445 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9058037070 con 0x7f90700b0870 2024-04-25T09:04:22.352 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.361+0000 7f9074b3b640 1 -- 172.21.15.43:0/123577445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90700ca670 con 0x7f90700b0870 2024-04-25T09:04:22.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.362+0000 7f9074b3b640 1 -- 172.21.15.43:0/123577445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9070074ef0 con 0x7f90700b0870 2024-04-25T09:04:22.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.362+0000 7f9066ffd640 1 -- 172.21.15.43:0/123577445 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9058043020 con 0x7f90700b0870 2024-04-25T09:04:22.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.363+0000 7f9066ffd640 1 --2- 172.21.15.43:0/123577445 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f905006b4d0 0x7f905006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:22.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.363+0000 7f9066ffd640 1 -- 172.21.15.43:0/123577445 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f90580aa5e0 con 0x7f90700b0870 2024-04-25T09:04:22.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.363+0000 7f906e59c640 1 --2- 172.21.15.43:0/123577445 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f905006b4d0 0x7f905006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:22.353 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.364+0000 7f906e59c640 1 --2- 172.21.15.43:0/123577445 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f905006b4d0 0x7f905006d990 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f90700c7210 tx=0x7f9068009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:22.355 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.367+0000 7f9066ffd640 1 -- 172.21.15.43:0/123577445 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f905804a030 con 0x7f90700b0870 2024-04-25T09:04:22.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.586+0000 7f9074b3b640 1 -- 172.21.15.43:0/123577445 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f90700c6f50 con 0x7f90700b0870 2024-04-25T09:04:22.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.587+0000 7f9066ffd640 1 -- 172.21.15.43:0/123577445 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4945 (secure 0 0 0) 0x7f90580794f0 con 0x7f90700b0870 2024-04-25T09:04:22.575 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:04:22.575 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":87,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/2225978762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2225978762},{"type":"v1","addr":"172.21.15.193:6836","nonce":2225978762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9711,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/454458092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":454458092},{"type":"v1","addr":"172.21.15.43:6833","nonce":454458092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":9720,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2314901977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":2314901977},{"type":"v1","addr":"172.21.15.43:6835","nonce":2314901977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T09:04:13.410799+0000","modified":"2024-04-25T09:04:15.424504+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9732},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9732":{"gid":9732,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.15.193:6837/2585534546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2585534546},{"type":"v1","addr":"172.21.15.193:6837","nonce":2585534546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T09:04:22.576 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 87 2024-04-25T09:04:22.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.589+0000 7f9074b3b640 1 -- 172.21.15.43:0/123577445 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f905006b4d0 msgr2=0x7f905006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.589+0000 7f9074b3b640 1 --2- 172.21.15.43:0/123577445 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f905006b4d0 0x7f905006d990 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f90700c7210 tx=0x7f9068009040 comp rx=0 tx=0).stop 2024-04-25T09:04:22.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.589+0000 7f9074b3b640 1 -- 172.21.15.43:0/123577445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90700b0870 msgr2=0x7f90700cae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:22.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.589+0000 7f9074b3b640 1 --2- 172.21.15.43:0/123577445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90700b0870 0x7f90700cae70 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f9058035040 tx=0x7f90580046c0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.589+0000 7f9074b3b640 1 -- 172.21.15.43:0/123577445 shutdown_connections 2024-04-25T09:04:22.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.589+0000 7f9074b3b640 1 --2- 172.21.15.43:0/123577445 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f905006b4d0 0x7f905006d990 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.589+0000 7f9074b3b640 1 --2- 172.21.15.43:0/123577445 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f907015df70 0x7f90700c6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.589+0000 7f9074b3b640 1 --2- 172.21.15.43:0/123577445 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f90700b0870 0x7f90700cae70 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.589+0000 7f9074b3b640 1 --2- 172.21.15.43:0/123577445 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f907006c510 0x7f90701f44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:22.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.589+0000 7f9074b3b640 1 -- 172.21.15.43:0/123577445 >> 172.21.15.43:0/123577445 conn(0x7f90700d17f0 msgr2=0x7f90700d1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:22.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.589+0000 7f9074b3b640 1 -- 172.21.15.43:0/123577445 shutdown_connections 2024-04-25T09:04:22.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:22.589+0000 7f9074b3b640 1 -- 172.21.15.43:0/123577445 wait complete. 2024-04-25T09:04:22.991 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 87, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T09:04:13.410799+0000', 'modified': '2024-04-25T09:04:15.424504+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9732}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9732': {'gid': 9732, 'name': 'b', 'rank': 0, 'incarnation': 85, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.193:6837/2585534546', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6835', 'nonce': 2585534546}, {'type': 'v1', 'addr': '172.21.15.193:6837', 'nonce': 2585534546}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:04:22.991 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:04:22.991 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T09:04:23.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.108+0000 7f918eb53640 1 Processor -- start 2024-04-25T09:04:23.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.108+0000 7f918eb53640 1 -- start start 2024-04-25T09:04:23.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.109+0000 7f918eb53640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f918815dcf0 0x7f918806e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.109+0000 7f918eb53640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f918806ebf0 0x7f918806f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.098 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.109+0000 7f918eb53640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91880d0d80 con 0x7f91881616e0 2024-04-25T09:04:23.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.109+0000 7f918eb53640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f918806f590 con 0x7f918815dcf0 2024-04-25T09:04:23.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.109+0000 7f918eb53640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f918806c050 con 0x7f918806ebf0 2024-04-25T09:04:23.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.109+0000 7f9187fff640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f918806ebf0 0x7f918806f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.109+0000 7f918c8ef640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f918815dcf0 0x7f918806e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.109+0000 7f918d0f0640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f91881616e0 0x7f9188161b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:47230/0 (socket says 172.21.15.43:47230) 2024-04-25T09:04:23.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.109+0000 7f9187fff640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f918806ebf0 0x7f918806f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:45814/0 (socket says 172.21.15.43:45814) 2024-04-25T09:04:23.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.109+0000 7f918c8ef640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f918815dcf0 0x7f918806e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:33916/0 (socket says 172.21.15.43:33916) 2024-04-25T09:04:23.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.109+0000 7f918d0f0640 1 -- 172.21.15.43:0/1968144502 learned_addr learned my addr 172.21.15.43:0/1968144502 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:23.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.110+0000 7f918c8ef640 1 -- 172.21.15.43:0/1968144502 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f918806ebf0 msgr2=0x7f918806f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:23.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.110+0000 7f918c8ef640 1 --2- 172.21.15.43:0/1968144502 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f918806ebf0 0x7f918806f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.110+0000 7f918c8ef640 1 -- 172.21.15.43:0/1968144502 >> v1:172.21.15.193:6789/0 conn(0x7f91881616e0 legacy=0x7f9188161b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.110+0000 7f918c8ef640 1 -- 172.21.15.43:0/1968144502 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91880d04e0 con 0x7f918815dcf0 2024-04-25T09:04:23.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.110+0000 7f9187fff640 1 --2- 172.21.15.43:0/1968144502 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f918806ebf0 0x7f918806f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:04:23.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.111+0000 7f918c8ef640 1 --2- 172.21.15.43:0/1968144502 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f918815dcf0 0x7f918806e530 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f91780097a0 tx=0x7f917802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88cd5bb3a14bb441 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:23.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.111+0000 7f91877fe640 1 -- 172.21.15.43:0/1968144502 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9178037070 con 0x7f918815dcf0 2024-04-25T09:04:23.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.111+0000 7f91877fe640 1 -- 172.21.15.43:0/1968144502 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f917802ae50 con 0x7f918815dcf0 2024-04-25T09:04:23.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.111+0000 7f91877fe640 1 -- 172.21.15.43:0/1968144502 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9178032660 con 0x7f918815dcf0 2024-04-25T09:04:23.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.111+0000 7f918eb53640 1 -- 172.21.15.43:0/1968144502 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f918815dcf0 msgr2=0x7f918806e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.111+0000 7f918eb53640 1 --2- 172.21.15.43:0/1968144502 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f918815dcf0 0x7f918806e530 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f91780097a0 tx=0x7f917802a100 comp rx=0 tx=0).stop 2024-04-25T09:04:23.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.111+0000 7f918eb53640 1 -- 172.21.15.43:0/1968144502 shutdown_connections 2024-04-25T09:04:23.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.111+0000 7f918eb53640 1 --2- 172.21.15.43:0/1968144502 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f918806ebf0 0x7f918806f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.111+0000 7f918eb53640 1 --2- 172.21.15.43:0/1968144502 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f918815dcf0 0x7f918806e530 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.111+0000 7f918eb53640 1 -- 172.21.15.43:0/1968144502 >> 172.21.15.43:0/1968144502 conn(0x7f91880d1700 msgr2=0x7f91880d1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:23.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.111+0000 7f918eb53640 1 -- 172.21.15.43:0/1968144502 shutdown_connections 2024-04-25T09:04:23.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.111+0000 7f918eb53640 1 -- 172.21.15.43:0/1968144502 wait complete. 2024-04-25T09:04:23.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.112+0000 7f918eb53640 1 Processor -- start 2024-04-25T09:04:23.102 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.113+0000 7f918eb53640 1 -- start start 2024-04-25T09:04:23.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.113+0000 7f918eb53640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f918806ebf0 0x7f91881f0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.113+0000 7f918eb53640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f918815dcf0 0x7f91881f77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.113+0000 7f918eb53640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91881616e0 0x7f91880b4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.103 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.113+0000 7f918eb53640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918806bce0 con 0x7f91881616e0 2024-04-25T09:04:23.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.113+0000 7f918eb53640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91880d0aa0 con 0x7f918806ebf0 2024-04-25T09:04:23.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.113+0000 7f918eb53640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91880b0d50 con 0x7f918815dcf0 2024-04-25T09:04:23.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.113+0000 7f918d0f0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91881616e0 0x7f91880b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.114+0000 7f918d0f0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91881616e0 0x7f91880b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38860/0 (socket says 172.21.15.43:38860) 2024-04-25T09:04:23.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.114+0000 7f918d0f0640 1 -- 172.21.15.43:0/1631325903 learned_addr learned my addr 172.21.15.43:0/1631325903 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:23.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.114+0000 7f918c8ef640 1 --2- 172.21.15.43:0/1631325903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f918806ebf0 0x7f91881f0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.114+0000 7f9187fff640 1 --2- 172.21.15.43:0/1631325903 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f918815dcf0 0x7f91881f77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.114+0000 7f918d0f0640 1 -- 172.21.15.43:0/1631325903 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f918815dcf0 msgr2=0x7f91881f77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.114+0000 7f918d0f0640 1 --2- 172.21.15.43:0/1631325903 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f918815dcf0 0x7f91881f77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.114+0000 7f918d0f0640 1 -- 172.21.15.43:0/1631325903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f918806ebf0 msgr2=0x7f91881f0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.114+0000 7f918d0f0640 1 --2- 172.21.15.43:0/1631325903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f918806ebf0 0x7f91881f0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.114+0000 7f918d0f0640 1 -- 172.21.15.43:0/1631325903 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91880b73e0 con 0x7f91881616e0 2024-04-25T09:04:23.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.114+0000 7f918d0f0640 1 --2- 172.21.15.43:0/1631325903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91881616e0 0x7f91880b4320 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f918000ca80 tx=0x7f918000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:23.104 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.114+0000 7f91857fa640 1 -- 172.21.15.43:0/1631325903 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9180017070 con 0x7f91881616e0 2024-04-25T09:04:23.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.114+0000 7f91857fa640 1 -- 172.21.15.43:0/1631325903 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f918000ac30 con 0x7f91881616e0 2024-04-25T09:04:23.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.114+0000 7f918eb53640 1 -- 172.21.15.43:0/1631325903 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91880b6640 con 0x7f91881616e0 2024-04-25T09:04:23.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.114+0000 7f91857fa640 1 -- 172.21.15.43:0/1631325903 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9180012600 con 0x7f91881616e0 2024-04-25T09:04:23.105 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.114+0000 7f918eb53640 1 -- 172.21.15.43:0/1631325903 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91880b6ee0 con 0x7f91881616e0 2024-04-25T09:04:23.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.116+0000 7f91857fa640 1 -- 172.21.15.43:0/1631325903 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f918001a070 con 0x7f91881616e0 2024-04-25T09:04:23.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.116+0000 7f91857fa640 1 --2- 172.21.15.43:0/1631325903 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f917406b4d0 0x7f917406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.117+0000 7f91857fa640 1 -- 172.21.15.43:0/1631325903 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9180089e40 con 0x7f91881616e0 2024-04-25T09:04:23.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.117+0000 7f918c8ef640 1 --2- 172.21.15.43:0/1631325903 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f917406b4d0 0x7f917406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.117+0000 7f9191854740 1 Processor -- start 2024-04-25T09:04:23.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.117+0000 7f9191854740 1 -- start start 2024-04-25T09:04:23.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.117+0000 7f918c8ef640 1 --2- 172.21.15.43:0/1631325903 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f917406b4d0 0x7f917406d990 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f9178034000 tx=0x7f917802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:23.107 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.118+0000 7f9191854740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5602958c2230 0x5602958c2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.118+0000 7f9191854740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5602958dfad0 0x5602958e0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.118+0000 7f9191854740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560295897f10 0x5602958de190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.118+0000 7f9191854740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602957faa10 con 0x5602958dfad0 2024-04-25T09:04:23.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.118+0000 7f9191854740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602957a3b30 con 0x560295897f10 2024-04-25T09:04:23.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.118+0000 7f9191854740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x5602957f6c50 con 0x5602958c2230 2024-04-25T09:04:23.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.118+0000 7f9187fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5602958dfad0 0x5602958e0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.118+0000 7f9187fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5602958dfad0 0x5602958e0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38866/0 (socket says 172.21.15.43:38866) 2024-04-25T09:04:23.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.118+0000 7f9187fff640 1 -- 172.21.15.43:0/2078592805 learned_addr learned my addr 172.21.15.43:0/2078592805 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:23.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.118+0000 7f918c8ef640 1 --2- 172.21.15.43:0/2078592805 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5602958c2230 0x5602958c2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.118+0000 7f918d0f0640 1 --2- 172.21.15.43:0/2078592805 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560295897f10 0x5602958de190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.118+0000 7f9187fff640 1 -- 172.21.15.43:0/2078592805 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5602958c2230 msgr2=0x5602958c2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:23.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.118+0000 7f9187fff640 1 --2- 172.21.15.43:0/2078592805 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5602958c2230 0x5602958c2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.118+0000 7f9187fff640 1 -- 172.21.15.43:0/2078592805 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560295897f10 msgr2=0x5602958de190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:23.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.118+0000 7f9187fff640 1 --2- 172.21.15.43:0/2078592805 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560295897f10 0x5602958de190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.118+0000 7f9187fff640 1 -- 172.21.15.43:0/2078592805 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5602957b8e10 con 0x5602958dfad0 2024-04-25T09:04:23.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.118+0000 7f9187fff640 1 --2- 172.21.15.43:0/2078592805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5602958dfad0 0x5602958e0020 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f917c009bf0 tx=0x7f917c011690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17c8010eb85b5562 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:23.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.119+0000 7f916e7fc640 1 -- 172.21.15.43:0/2078592805 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c019070 con 0x5602958dfad0 2024-04-25T09:04:23.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.119+0000 7f916e7fc640 1 -- 172.21.15.43:0/2078592805 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f917c004030 con 0x5602958dfad0 2024-04-25T09:04:23.109 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.119+0000 7f916e7fc640 1 -- 172.21.15.43:0/2078592805 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917c004400 con 0x5602958dfad0 2024-04-25T09:04:23.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.119+0000 7f9191854740 1 -- 172.21.15.43:0/2078592805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5602958dfad0 msgr2=0x5602958e0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.119+0000 7f9191854740 1 --2- 172.21.15.43:0/2078592805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5602958dfad0 0x5602958e0020 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f917c009bf0 tx=0x7f917c011690 comp rx=0 tx=0).stop 2024-04-25T09:04:23.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.119+0000 7f9191854740 1 -- 172.21.15.43:0/2078592805 shutdown_connections 2024-04-25T09:04:23.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.119+0000 7f9191854740 1 --2- 172.21.15.43:0/2078592805 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x560295897f10 0x5602958de190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.119+0000 7f9191854740 1 --2- 172.21.15.43:0/2078592805 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5602958dfad0 0x5602958e0020 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.119+0000 7f9191854740 1 --2- 172.21.15.43:0/2078592805 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5602958c2230 0x5602958c2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.119+0000 7f9191854740 1 -- 172.21.15.43:0/2078592805 >> 172.21.15.43:0/2078592805 conn(0x560295894840 msgr2=0x5602958dd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:23.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.119+0000 7f9191854740 1 -- 172.21.15.43:0/2078592805 shutdown_connections 2024-04-25T09:04:23.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.119+0000 7f9191854740 1 -- 172.21.15.43:0/2078592805 wait complete. 2024-04-25T09:04:23.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.120+0000 7f9191854740 1 -- start start 2024-04-25T09:04:23.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.120+0000 7f9191854740 1 Processor -- start 2024-04-25T09:04:23.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.121+0000 7f9191854740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5602958c2230 0x5602958dca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.121+0000 7f9191854740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5602958dfad0 0x5602958c3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.121+0000 7f9191854740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602958ab3a0 0x5602958c42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.121+0000 7f9191854740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x560295898cb0 con 0x5602958c2230 2024-04-25T09:04:23.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.121+0000 7f9191854740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602957b70c0 con 0x5602958ab3a0 2024-04-25T09:04:23.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.121+0000 7f9191854740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x560295892770 con 0x5602958dfad0 2024-04-25T09:04:23.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.121+0000 7f9191854740 10 client.0 _ll_register_callbacks cb 0x5602957aaec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:04:23.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.121+0000 7f918c8ef640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5602958c2230 0x5602958dca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.121+0000 7f918c8ef640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5602958c2230 0x5602958dca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38878/0 (socket says 172.21.15.43:38878) 2024-04-25T09:04:23.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.121+0000 7f918c8ef640 1 -- 172.21.15.43:0/1766615248 learned_addr learned my addr 172.21.15.43:0/1766615248 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:23.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.121+0000 7f9187fff640 1 --2- 172.21.15.43:0/1766615248 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5602958dfad0 0x5602958c3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.121+0000 7f918d0f0640 1 --2- 172.21.15.43:0/1766615248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602958ab3a0 0x5602958c42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.121+0000 7f918c8ef640 1 -- 172.21.15.43:0/1766615248 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5602958dfad0 msgr2=0x5602958c3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:23.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.121+0000 7f918c8ef640 1 --2- 172.21.15.43:0/1766615248 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5602958dfad0 0x5602958c3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.121+0000 7f918c8ef640 1 -- 172.21.15.43:0/1766615248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602958ab3a0 msgr2=0x5602958c42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:23.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.121+0000 7f918c8ef640 1 --2- 172.21.15.43:0/1766615248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602958ab3a0 0x5602958c42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.121+0000 7f918c8ef640 1 -- 172.21.15.43:0/1766615248 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56029550f2d0 con 0x5602958c2230 2024-04-25T09:04:23.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.122+0000 7f918c8ef640 1 --2- 172.21.15.43:0/1766615248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5602958c2230 0x5602958dca60 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f91780098c0 tx=0x7f9178004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c593e02048728467 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:23.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.122+0000 7f91677fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:04:23.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.122+0000 7f91677fe640 1 -- 172.21.15.43:0/1766615248 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9178037040 con 0x5602958c2230 2024-04-25T09:04:23.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.122+0000 7f91677fe640 1 -- 172.21.15.43:0/1766615248 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f917803c650 con 0x5602958c2230 2024-04-25T09:04:23.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.122+0000 7f9191854740 1 -- 172.21.15.43:0/1766615248 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f917c004030 con 0x5602958c2230 2024-04-25T09:04:23.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.122+0000 7f91677fe640 1 -- 172.21.15.43:0/1766615248 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9178054050 con 0x5602958c2230 2024-04-25T09:04:23.117 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.122+0000 7f9191854740 10 client.9894 fetch_fsmap learned FSMap version 87 2024-04-25T09:04:23.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.122+0000 7f9191854740 1 -- 172.21.15.43:0/1766615248 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x5602955d96b0 con 0x5602958c2230 2024-04-25T09:04:23.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.122+0000 7f91677fe640 1 -- 172.21.15.43:0/1766615248 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f917c004030 con 0x5602958c2230 2024-04-25T09:04:23.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.122+0000 7f91677fe640 1 -- 172.21.15.43:0/1766615248 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 87) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9178057020 con 0x5602958c2230 2024-04-25T09:04:23.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.122+0000 7f9191854740 10 client.9894 fetch_fsmap finished waiting for FSMap version 87 2024-04-25T09:04:23.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.122+0000 7f9191854740 10 client.9894 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T09:04:23.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.122+0000 7f9191854740 20 client.9894 populate_metadata read hostname 'smithi043' 2024-04-25T09:04:23.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.122+0000 7f9191854740 1 --2- 172.21.15.43:0/1766615248 >> [v2:172.21.15.193:6835/2585534546,v1:172.21.15.193:6837/2585534546] conn(0x5602958f9630 0x5602958fba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.122+0000 7f9191854740 4 client.9894 mds_command: new command op to 9732 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:04:23.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.122+0000 7f9191854740 1 -- 172.21.15.43:0/1766615248 --> [v2:172.21.15.193:6835/2585534546,v1:172.21.15.193:6837/2585534546] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560295859920 con 0x5602958f9630 2024-04-25T09:04:23.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.122+0000 7f9187fff640 1 --2- 172.21.15.43:0/1766615248 >> [v2:172.21.15.193:6835/2585534546,v1:172.21.15.193:6837/2585534546] conn(0x5602958f9630 0x5602958fba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.123+0000 7f9187fff640 1 --2- 172.21.15.43:0/1766615248 >> [v2:172.21.15.193:6835/2585534546,v1:172.21.15.193:6837/2585534546] conn(0x5602958f9630 0x5602958fba20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2ba0d563e9e978f server_cookie=bf76cad92226c0f5 in_seq=0 out_seq=0 2024-04-25T09:04:23.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.123+0000 7f91677fe640 10 client.9894 ms_handle_connect on v2:172.21.15.193:6835/2585534546 2024-04-25T09:04:23.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.124+0000 7f91677fe640 1 -- 172.21.15.43:0/1766615248 <== mds.0 v2:172.21.15.193:6835/2585534546 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560295859920 con 0x5602958f9630 2024-04-25T09:04:23.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.124+0000 7f91677fe640 10 client.9894 handle_command_reply: tid=0 2024-04-25T09:04:23.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.124+0000 7f9191854740 1 client.9894 shutdown 2024-04-25T09:04:23.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.124+0000 7f9191854740 1 -- 172.21.15.43:0/1766615248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5602958c2230 msgr2=0x5602958dca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.124+0000 7f9191854740 1 --2- 172.21.15.43:0/1766615248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5602958c2230 0x5602958dca60 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f91780098c0 tx=0x7f9178004730 comp rx=0 tx=0).stop 2024-04-25T09:04:23.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.124+0000 7f9191854740 1 -- 172.21.15.43:0/1766615248 shutdown_connections 2024-04-25T09:04:23.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.124+0000 7f9191854740 1 --2- 172.21.15.43:0/1766615248 >> [v2:172.21.15.193:6835/2585534546,v1:172.21.15.193:6837/2585534546] conn(0x5602958f9630 0x5602958fba20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.124+0000 7f9191854740 1 --2- 172.21.15.43:0/1766615248 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602958ab3a0 0x5602958c42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.124+0000 7f9191854740 1 --2- 172.21.15.43:0/1766615248 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5602958dfad0 0x5602958c3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.124+0000 7f9191854740 1 --2- 172.21.15.43:0/1766615248 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5602958c2230 0x5602958dca60 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.124+0000 7f91677fe640 0 client.9894 ms_handle_reset on v2:172.21.15.193:6835/2585534546 2024-04-25T09:04:23.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.124+0000 7f9191854740 1 -- 172.21.15.43:0/1766615248 >> 172.21.15.43:0/1766615248 conn(0x560295897f10 msgr2=0x5602958e0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:23.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.124+0000 7f9191854740 1 -- 172.21.15.43:0/1766615248 shutdown_connections 2024-04-25T09:04:23.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.124+0000 7f9191854740 1 -- 172.21.15.43:0/1766615248 wait complete. 2024-04-25T09:04:23.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.124+0000 7f9191854740 20 client.9894 trim_cache size 0 max 16384 2024-04-25T09:04:23.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.140+0000 7f9191854740 1 Processor -- start 2024-04-25T09:04:23.128 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.140+0000 7f9191854740 1 -- start start 2024-04-25T09:04:23.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.140+0000 7f9191854740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5602958ab3a0 0x5602958984e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.140+0000 7f9191854740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602958c2230 0x5602958c2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.140+0000 7f9191854740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5602958dfad0 0x5602958c3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.140+0000 7f9191854740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91881fffa0 con 0x5602958dfad0 2024-04-25T09:04:23.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.140+0000 7f9191854740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9188160100 con 0x5602958c2230 2024-04-25T09:04:23.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.140+0000 7f9191854740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9188146460 con 0x5602958ab3a0 2024-04-25T09:04:23.131 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.141+0000 7f918d0f0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5602958dfad0 0x5602958c3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.141+0000 7f918d0f0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5602958dfad0 0x5602958c3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38886/0 (socket says 172.21.15.43:38886) 2024-04-25T09:04:23.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.141+0000 7f918d0f0640 1 -- 172.21.15.43:0/4040859677 learned_addr learned my addr 172.21.15.43:0/4040859677 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:23.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.141+0000 7f918c8ef640 1 --2- 172.21.15.43:0/4040859677 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5602958ab3a0 0x5602958984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.141+0000 7f9187fff640 1 --2- 172.21.15.43:0/4040859677 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602958c2230 0x5602958c2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.141+0000 7f918d0f0640 1 -- 172.21.15.43:0/4040859677 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5602958ab3a0 msgr2=0x5602958984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:23.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.141+0000 7f918d0f0640 1 --2- 172.21.15.43:0/4040859677 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5602958ab3a0 0x5602958984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.141+0000 7f918d0f0640 1 -- 172.21.15.43:0/4040859677 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602958c2230 msgr2=0x5602958c2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:23.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.141+0000 7f918d0f0640 1 --2- 172.21.15.43:0/4040859677 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602958c2230 0x5602958c2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.141+0000 7f918d0f0640 1 -- 172.21.15.43:0/4040859677 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560295891d30 con 0x5602958dfad0 2024-04-25T09:04:23.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.141+0000 7f918c8ef640 1 --2- 172.21.15.43:0/4040859677 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5602958ab3a0 0x5602958984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:23.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.141+0000 7f918d0f0640 1 --2- 172.21.15.43:0/4040859677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5602958dfad0 0x5602958c3430 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f91800607a0 tx=0x7f9180054910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e41af4703ae3b47d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:23.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.141+0000 7f916dffb640 1 -- 172.21.15.43:0/4040859677 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9180060cd0 con 0x5602958dfad0 2024-04-25T09:04:23.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.141+0000 7f916dffb640 1 -- 172.21.15.43:0/4040859677 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9180056970 con 0x5602958dfad0 2024-04-25T09:04:23.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.141+0000 7f916dffb640 1 -- 172.21.15.43:0/4040859677 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9180060840 con 0x5602958dfad0 2024-04-25T09:04:23.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.142+0000 7f9191854740 1 -- 172.21.15.43:0/4040859677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5602958dfad0 msgr2=0x5602958c3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.142+0000 7f9191854740 1 --2- 172.21.15.43:0/4040859677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5602958dfad0 0x5602958c3430 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f91800607a0 tx=0x7f9180054910 comp rx=0 tx=0).stop 2024-04-25T09:04:23.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.142+0000 7f9191854740 1 -- 172.21.15.43:0/4040859677 shutdown_connections 2024-04-25T09:04:23.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.142+0000 7f9191854740 1 --2- 172.21.15.43:0/4040859677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5602958dfad0 0x5602958c3430 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.142+0000 7f9191854740 1 --2- 172.21.15.43:0/4040859677 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602958c2230 0x5602958c2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.142+0000 7f9191854740 1 --2- 172.21.15.43:0/4040859677 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5602958ab3a0 0x5602958984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.142+0000 7f9191854740 1 -- 172.21.15.43:0/4040859677 >> 172.21.15.43:0/4040859677 conn(0x560295897f10 msgr2=0x5602958df2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:23.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.142+0000 7f9191854740 1 -- 172.21.15.43:0/4040859677 shutdown_connections 2024-04-25T09:04:23.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.142+0000 7f9191854740 1 -- 172.21.15.43:0/4040859677 wait complete. 2024-04-25T09:04:23.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.142+0000 7f9191854740 1 -- start start 2024-04-25T09:04:23.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.143+0000 7f9191854740 1 Processor -- start 2024-04-25T09:04:23.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.143+0000 7f9191854740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5602958ab3a0 0x5602958e0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.143+0000 7f9191854740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602958c2230 0x5602958efad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.143+0000 7f9191854740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5602958dfad0 0x5602958f0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.143+0000 7f9191854740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x5602958c50f0 con 0x5602958ab3a0 2024-04-25T09:04:23.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.143+0000 7f9191854740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918815fb00 con 0x5602958c2230 2024-04-25T09:04:23.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.143+0000 7f9191854740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x5602958ab940 con 0x5602958dfad0 2024-04-25T09:04:23.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.143+0000 7f9191854740 10 client.0 _ll_register_callbacks cb 0x560295892c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:04:23.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.143+0000 7f918c8ef640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5602958ab3a0 0x5602958e0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.143+0000 7f918c8ef640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5602958ab3a0 0x5602958e0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38896/0 (socket says 172.21.15.43:38896) 2024-04-25T09:04:23.137 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.143+0000 7f918c8ef640 1 -- 172.21.15.43:0/2495551259 learned_addr learned my addr 172.21.15.43:0/2495551259 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:23.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.143+0000 7f9187fff640 1 --2- 172.21.15.43:0/2495551259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602958c2230 0x5602958efad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.143+0000 7f918d0f0640 1 --2- 172.21.15.43:0/2495551259 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5602958dfad0 0x5602958f0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.144+0000 7f918c8ef640 1 -- 172.21.15.43:0/2495551259 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5602958dfad0 msgr2=0x5602958f0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:23.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.144+0000 7f918c8ef640 1 --2- 172.21.15.43:0/2495551259 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5602958dfad0 0x5602958f0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.144+0000 7f918c8ef640 1 -- 172.21.15.43:0/2495551259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602958c2230 msgr2=0x5602958efad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:23.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.144+0000 7f918c8ef640 1 --2- 172.21.15.43:0/2495551259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602958c2230 0x5602958efad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.144+0000 7f918c8ef640 1 -- 172.21.15.43:0/2495551259 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5602958df310 con 0x5602958ab3a0 2024-04-25T09:04:23.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.144+0000 7f918c8ef640 1 --2- 172.21.15.43:0/2495551259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5602958ab3a0 0x5602958e0500 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f9178004760 tx=0x7f9178009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0878c8d041e1dbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:23.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.144+0000 7f91677fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:04:23.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.144+0000 7f91677fe640 1 -- 172.21.15.43:0/2495551259 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917805e020 con 0x5602958ab3a0 2024-04-25T09:04:23.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.144+0000 7f91677fe640 1 -- 172.21.15.43:0/2495551259 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9178048460 con 0x5602958ab3a0 2024-04-25T09:04:23.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.144+0000 7f9191854740 1 -- 172.21.15.43:0/2495551259 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9180056970 con 0x5602958ab3a0 2024-04-25T09:04:23.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.144+0000 7f91677fe640 1 -- 172.21.15.43:0/2495551259 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9178048660 con 0x5602958ab3a0 2024-04-25T09:04:23.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.144+0000 7f9191854740 10 client.9900 fetch_fsmap learned FSMap version 87 2024-04-25T09:04:23.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.144+0000 7f9191854740 1 -- 172.21.15.43:0/2495551259 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x5602958df850 con 0x5602958ab3a0 2024-04-25T09:04:23.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.144+0000 7f91677fe640 1 -- 172.21.15.43:0/2495551259 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9180056970 con 0x5602958ab3a0 2024-04-25T09:04:23.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.144+0000 7f91677fe640 1 -- 172.21.15.43:0/2495551259 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 87) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9178057040 con 0x5602958ab3a0 2024-04-25T09:04:23.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.144+0000 7f9191854740 10 client.9900 fetch_fsmap finished waiting for FSMap version 87 2024-04-25T09:04:23.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.144+0000 7f9191854740 10 client.9900 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T09:04:23.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.144+0000 7f9191854740 20 client.9900 populate_metadata read hostname 'smithi043' 2024-04-25T09:04:23.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.144+0000 7f9191854740 1 --2- 172.21.15.43:0/2495551259 >> [v2:172.21.15.193:6835/2585534546,v1:172.21.15.193:6837/2585534546] conn(0x5602958aaea0 0x5602958d29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.145+0000 7f9191854740 4 client.9900 mds_command: new command op to 9732 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:04:23.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.145+0000 7f9191854740 1 -- 172.21.15.43:0/2495551259 --> [v2:172.21.15.193:6835/2585534546,v1:172.21.15.193:6837/2585534546] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560295807f50 con 0x5602958aaea0 2024-04-25T09:04:23.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.145+0000 7f9187fff640 1 --2- 172.21.15.43:0/2495551259 >> [v2:172.21.15.193:6835/2585534546,v1:172.21.15.193:6837/2585534546] conn(0x5602958aaea0 0x5602958d29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.145+0000 7f91677fe640 10 client.9900 ms_handle_connect on v2:172.21.15.193:6835/2585534546 2024-04-25T09:04:23.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.145+0000 7f9187fff640 1 --2- 172.21.15.43:0/2495551259 >> [v2:172.21.15.193:6835/2585534546,v1:172.21.15.193:6837/2585534546] conn(0x5602958aaea0 0x5602958d29d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8d5abcbe4b06b77 server_cookie=c1011e44dcd323c3 in_seq=0 out_seq=0 2024-04-25T09:04:23.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.145+0000 7f91677fe640 1 -- 172.21.15.43:0/2495551259 <== mds.0 v2:172.21.15.193:6835/2585534546 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x560295807f50 con 0x5602958aaea0 2024-04-25T09:04:23.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.145+0000 7f91677fe640 10 client.9900 handle_command_reply: tid=0 2024-04-25T09:04:23.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.145+0000 7f9191854740 1 client.9900 shutdown 2024-04-25T09:04:23.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.145+0000 7f9191854740 1 -- 172.21.15.43:0/2495551259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5602958ab3a0 msgr2=0x5602958e0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.145+0000 7f9191854740 1 --2- 172.21.15.43:0/2495551259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5602958ab3a0 0x5602958e0500 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f9178004760 tx=0x7f9178009190 comp rx=0 tx=0).stop 2024-04-25T09:04:23.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.145+0000 7f9191854740 1 -- 172.21.15.43:0/2495551259 shutdown_connections 2024-04-25T09:04:23.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.145+0000 7f9191854740 1 --2- 172.21.15.43:0/2495551259 >> [v2:172.21.15.193:6835/2585534546,v1:172.21.15.193:6837/2585534546] conn(0x5602958aaea0 0x5602958d29d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.145+0000 7f9191854740 1 --2- 172.21.15.43:0/2495551259 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5602958dfad0 0x5602958f0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.145+0000 7f9191854740 1 --2- 172.21.15.43:0/2495551259 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5602958c2230 0x5602958efad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.145+0000 7f91677fe640 0 client.9900 ms_handle_reset on v2:172.21.15.193:6835/2585534546 2024-04-25T09:04:23.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.145+0000 7f9191854740 1 --2- 172.21.15.43:0/2495551259 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5602958ab3a0 0x5602958e0500 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.146+0000 7f9191854740 1 -- 172.21.15.43:0/2495551259 >> 172.21.15.43:0/2495551259 conn(0x560295897f10 msgr2=0x5602958c5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:23.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.146+0000 7f9191854740 1 -- 172.21.15.43:0/2495551259 shutdown_connections 2024-04-25T09:04:23.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.146+0000 7f9191854740 1 -- 172.21.15.43:0/2495551259 wait complete. 2024-04-25T09:04:23.140 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.146+0000 7f9191854740 20 client.9900 trim_cache size 0 max 16384 2024-04-25T09:04:23.141 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:04:23.141 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "6c4bd50e-aa00-4a7f-8d09-352344ef5a4b", 2024-04-25T09:04:23.141 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-04-25T09:04:23.141 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 9732, 2024-04-25T09:04:23.141 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-04-25T09:04:23.141 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-04-25T09:04:23.141 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-04-25T09:04:23.141 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 9.0748534420000002, 2024-04-25T09:04:23.141 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 87, 2024-04-25T09:04:23.142 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 139, 2024-04-25T09:04:23.142 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 139, 2024-04-25T09:04:23.142 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 14.097953727 2024-04-25T09:04:23.142 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:04:23.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.146+0000 7f918eb53640 1 -- 172.21.15.43:0/1631325903 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f917406b4d0 msgr2=0x7f917406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.146+0000 7f918eb53640 1 --2- 172.21.15.43:0/1631325903 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f917406b4d0 0x7f917406d990 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f9178034000 tx=0x7f917802e000 comp rx=0 tx=0).stop 2024-04-25T09:04:23.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.146+0000 7f918eb53640 1 -- 172.21.15.43:0/1631325903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91881616e0 msgr2=0x7f91880b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.146+0000 7f918eb53640 1 --2- 172.21.15.43:0/1631325903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91881616e0 0x7f91880b4320 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f918000ca80 tx=0x7f918000cf40 comp rx=0 tx=0).stop 2024-04-25T09:04:23.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.146+0000 7f918eb53640 1 -- 172.21.15.43:0/1631325903 shutdown_connections 2024-04-25T09:04:23.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.146+0000 7f918eb53640 1 --2- 172.21.15.43:0/1631325903 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f91881616e0 0x7f91880b4320 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.146+0000 7f918eb53640 1 --2- 172.21.15.43:0/1631325903 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f918815dcf0 0x7f91881f77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.146+0000 7f918eb53640 1 --2- 172.21.15.43:0/1631325903 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f917406b4d0 0x7f917406d990 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.146+0000 7f918eb53640 1 --2- 172.21.15.43:0/1631325903 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f918806ebf0 0x7f91881f0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.146+0000 7f918eb53640 1 -- 172.21.15.43:0/1631325903 >> 172.21.15.43:0/1631325903 conn(0x7f91880d1700 msgr2=0x7f918806c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:23.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.146+0000 7f918eb53640 1 -- 172.21.15.43:0/1631325903 shutdown_connections 2024-04-25T09:04:23.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.146+0000 7f918eb53640 1 -- 172.21.15.43:0/1631325903 wait complete. 2024-04-25T09:04:23.146 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T09:04:23.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.258+0000 7ff7936f5640 1 Processor -- start 2024-04-25T09:04:23.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.258+0000 7ff7936f5640 1 -- start start 2024-04-25T09:04:23.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.259+0000 7ff7936f5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff78c15dce0 0x7ff78c15ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.259+0000 7ff7936f5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff78c0ce550 0x7ff78c0d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.259+0000 7ff7936f5640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff78c0ce990 0x7ff78c0d0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.259+0000 7ff7936f5640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff78c0cee80 con 0x7ff78c0ce550 2024-04-25T09:04:23.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.259+0000 7ff7936f5640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff78c0cf160 con 0x7ff78c0ce990 2024-04-25T09:04:23.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.259+0000 7ff7936f5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c0cf440 con 0x7ff78c15dce0 2024-04-25T09:04:23.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.259+0000 7ff790c90640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff78c0ce550 0x7ff78c0d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.259+0000 7ff790c90640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff78c0ce550 0x7ff78c0d03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38900/0 (socket says 172.21.15.43:38900) 2024-04-25T09:04:23.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.259+0000 7ff790c90640 1 -- 172.21.15.43:0/510862820 learned_addr learned my addr 172.21.15.43:0/510862820 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:23.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.259+0000 7ff791c92640 1 --2- 172.21.15.43:0/510862820 >> v2:172.21.15.193:3300/0 conn(0x7ff78c0ce990 0x7ff78c0d0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.259+0000 7ff790c90640 1 -- 172.21.15.43:0/510862820 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff78c15dce0 msgr2=0x7ff78c15ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:23.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.259+0000 7ff791491640 1 --2- 172.21.15.43:0/510862820 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff78c15dce0 0x7ff78c15ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.259+0000 7ff790c90640 1 --2- 172.21.15.43:0/510862820 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff78c15dce0 0x7ff78c15ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.259+0000 7ff790c90640 1 -- 172.21.15.43:0/510862820 >> v2:172.21.15.193:3300/0 conn(0x7ff78c0ce990 msgr2=0x7ff78c0d0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:23.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.259+0000 7ff790c90640 1 --2- 172.21.15.43:0/510862820 >> v2:172.21.15.193:3300/0 conn(0x7ff78c0ce990 0x7ff78c0d0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.259+0000 7ff790c90640 1 -- 172.21.15.43:0/510862820 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff78c0cf720 con 0x7ff78c0ce550 2024-04-25T09:04:23.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.259+0000 7ff791c92640 1 --2- 172.21.15.43:0/510862820 >> v2:172.21.15.193:3300/0 conn(0x7ff78c0ce990 0x7ff78c0d0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:23.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.259+0000 7ff791491640 1 --2- 172.21.15.43:0/510862820 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff78c15dce0 0x7ff78c15ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:23.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.260+0000 7ff790c90640 1 --2- 172.21.15.43:0/510862820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff78c0ce550 0x7ff78c0d03e0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7ff78c0d1480 tx=0x7ff77c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be31ec0a865c02d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:23.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.260+0000 7ff783fff640 1 -- 172.21.15.43:0/510862820 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c037070 con 0x7ff78c0ce550 2024-04-25T09:04:23.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.260+0000 7ff783fff640 1 -- 172.21.15.43:0/510862820 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff77c032440 con 0x7ff78c0ce550 2024-04-25T09:04:23.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.260+0000 7ff783fff640 1 -- 172.21.15.43:0/510862820 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77c032810 con 0x7ff78c0ce550 2024-04-25T09:04:23.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.260+0000 7ff7936f5640 1 -- 172.21.15.43:0/510862820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff78c0ce550 msgr2=0x7ff78c0d03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.260+0000 7ff7936f5640 1 --2- 172.21.15.43:0/510862820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff78c0ce550 0x7ff78c0d03e0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7ff78c0d1480 tx=0x7ff77c02a140 comp rx=0 tx=0).stop 2024-04-25T09:04:23.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.261+0000 7ff7936f5640 1 -- 172.21.15.43:0/510862820 shutdown_connections 2024-04-25T09:04:23.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.261+0000 7ff7936f5640 1 --2- 172.21.15.43:0/510862820 >> v2:172.21.15.193:3300/0 conn(0x7ff78c0ce990 0x7ff78c0d0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.261+0000 7ff7936f5640 1 --2- 172.21.15.43:0/510862820 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff78c0ce550 0x7ff78c0d03e0 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.259 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.261+0000 7ff7936f5640 1 --2- 172.21.15.43:0/510862820 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff78c15dce0 0x7ff78c15ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.261+0000 7ff7936f5640 1 -- 172.21.15.43:0/510862820 >> 172.21.15.43:0/510862820 conn(0x7ff78c09cd00 msgr2=0x7ff78c0d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:23.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.261+0000 7ff7936f5640 1 -- 172.21.15.43:0/510862820 shutdown_connections 2024-04-25T09:04:23.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.261+0000 7ff7936f5640 1 -- 172.21.15.43:0/510862820 wait complete. 2024-04-25T09:04:23.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.262+0000 7ff7936f5640 1 Processor -- start 2024-04-25T09:04:23.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.262+0000 7ff7936f5640 1 -- start start 2024-04-25T09:04:23.260 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.263+0000 7ff7936f5640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff78c0ce550 0x7ff78c0cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.263+0000 7ff7936f5640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff78c0ce990 0x7ff78c157d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.263+0000 7ff7936f5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff78c15dce0 0x7ff78c0d1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.263+0000 7ff7936f5640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c160980 con 0x7ff78c15dce0 2024-04-25T09:04:23.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.263+0000 7ff7936f5640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c15e0c0 con 0x7ff78c0ce550 2024-04-25T09:04:23.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.263+0000 7ff7936f5640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff78c160b00 con 0x7ff78c0ce990 2024-04-25T09:04:23.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.263+0000 7ff791c92640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff78c15dce0 0x7ff78c0d1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.263+0000 7ff791c92640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff78c15dce0 0x7ff78c0d1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:38908/0 (socket says 172.21.15.43:38908) 2024-04-25T09:04:23.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.263+0000 7ff791c92640 1 -- 172.21.15.43:0/3636914098 learned_addr learned my addr 172.21.15.43:0/3636914098 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:23.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.263+0000 7ff791491640 1 --2- 172.21.15.43:0/3636914098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff78c0ce550 0x7ff78c0cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.263+0000 7ff790c90640 1 --2- 172.21.15.43:0/3636914098 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff78c0ce990 0x7ff78c157d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.263+0000 7ff791c92640 1 -- 172.21.15.43:0/3636914098 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff78c0ce990 msgr2=0x7ff78c157d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.263+0000 7ff791c92640 1 --2- 172.21.15.43:0/3636914098 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff78c0ce990 0x7ff78c157d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.263+0000 7ff791c92640 1 -- 172.21.15.43:0/3636914098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff78c0ce550 msgr2=0x7ff78c0cb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.263+0000 7ff791c92640 1 --2- 172.21.15.43:0/3636914098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff78c0ce550 0x7ff78c0cb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.263+0000 7ff791c92640 1 -- 172.21.15.43:0/3636914098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff78c0cb9f0 con 0x7ff78c15dce0 2024-04-25T09:04:23.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.263+0000 7ff791c92640 1 --2- 172.21.15.43:0/3636914098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff78c15dce0 0x7ff78c0d1f60 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7ff78400cef0 tx=0x7ff78400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:23.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.263+0000 7ff781ffb640 1 -- 172.21.15.43:0/3636914098 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78401a070 con 0x7ff78c15dce0 2024-04-25T09:04:23.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.263+0000 7ff781ffb640 1 -- 172.21.15.43:0/3636914098 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78400d860 con 0x7ff78c15dce0 2024-04-25T09:04:23.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.263+0000 7ff7936f5640 1 -- 172.21.15.43:0/3636914098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff78c158ac0 con 0x7ff78c15dce0 2024-04-25T09:04:23.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.263+0000 7ff781ffb640 1 -- 172.21.15.43:0/3636914098 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78400dc50 con 0x7ff78c15dce0 2024-04-25T09:04:23.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.264+0000 7ff7936f5640 1 -- 172.21.15.43:0/3636914098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff78c159080 con 0x7ff78c15dce0 2024-04-25T09:04:23.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.265+0000 7ff7936f5640 1 -- 172.21.15.43:0/3636914098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff78c0680f0 con 0x7ff78c15dce0 2024-04-25T09:04:23.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.265+0000 7ff781ffb640 1 -- 172.21.15.43:0/3636914098 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff78401d070 con 0x7ff78c15dce0 2024-04-25T09:04:23.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.265+0000 7ff781ffb640 1 --2- 172.21.15.43:0/3636914098 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff77006b4d0 0x7ff77006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:23.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.265+0000 7ff781ffb640 1 -- 172.21.15.43:0/3636914098 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff78408aab0 con 0x7ff78c15dce0 2024-04-25T09:04:23.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.265+0000 7ff791491640 1 --2- 172.21.15.43:0/3636914098 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff77006b4d0 0x7ff77006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:23.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.266+0000 7ff791491640 1 --2- 172.21.15.43:0/3636914098 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff77006b4d0 0x7ff77006d990 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7ff78800aa40 tx=0x7ff788008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:23.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.270+0000 7ff781ffb640 1 -- 172.21.15.43:0/3636914098 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff784059310 con 0x7ff78c15dce0 2024-04-25T09:04:23.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.487+0000 7ff7936f5640 1 -- 172.21.15.43:0/3636914098 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff78c074ef0 con 0x7ff78c15dce0 2024-04-25T09:04:23.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.487+0000 7ff781ffb640 1 -- 172.21.15.43:0/3636914098 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v87) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff78405d3b0 con 0x7ff78c15dce0 2024-04-25T09:04:23.476 INFO:teuthology.orchestra.run.smithi043.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T09:04:23.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.489+0000 7ff7936f5640 1 -- 172.21.15.43:0/3636914098 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff77006b4d0 msgr2=0x7ff77006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.489+0000 7ff7936f5640 1 --2- 172.21.15.43:0/3636914098 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff77006b4d0 0x7ff77006d990 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7ff78800aa40 tx=0x7ff788008040 comp rx=0 tx=0).stop 2024-04-25T09:04:23.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.489+0000 7ff7936f5640 1 -- 172.21.15.43:0/3636914098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff78c15dce0 msgr2=0x7ff78c0d1f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:23.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.489+0000 7ff7936f5640 1 --2- 172.21.15.43:0/3636914098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff78c15dce0 0x7ff78c0d1f60 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7ff78400cef0 tx=0x7ff78400a710 comp rx=0 tx=0).stop 2024-04-25T09:04:23.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.489+0000 7ff7936f5640 1 -- 172.21.15.43:0/3636914098 shutdown_connections 2024-04-25T09:04:23.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.489+0000 7ff7936f5640 1 --2- 172.21.15.43:0/3636914098 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff78c15dce0 0x7ff78c0d1f60 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.489+0000 7ff7936f5640 1 --2- 172.21.15.43:0/3636914098 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff78c0ce990 0x7ff78c157d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.489+0000 7ff7936f5640 1 --2- 172.21.15.43:0/3636914098 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff77006b4d0 0x7ff77006d990 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.489+0000 7ff7936f5640 1 --2- 172.21.15.43:0/3636914098 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff78c0ce550 0x7ff78c0cb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:23.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.489+0000 7ff7936f5640 1 -- 172.21.15.43:0/3636914098 >> 172.21.15.43:0/3636914098 conn(0x7ff78c09cd00 msgr2=0x7ff78c15cf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:23.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.489+0000 7ff7936f5640 1 -- 172.21.15.43:0/3636914098 shutdown_connections 2024-04-25T09:04:23.478 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:23.489+0000 7ff7936f5640 1 -- 172.21.15.43:0/3636914098 wait complete. 2024-04-25T09:04:23.488 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T09:04:23.489 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T09:04:23.489 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi043.front.sepia.ceph.com 2024-04-25T09:04:23.489 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T09:04:23.489 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T09:04:23.489 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T09:04:23.489 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T09:04:23.489 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T09:04:23.489 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:04:23.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:23.489 DEBUG:teuthology.orchestra.run.smithi043:> ip addr 2024-04-25T09:04:23.505 INFO:teuthology.orchestra.run.smithi043.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T09:04:23.505 INFO:teuthology.orchestra.run.smithi043.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T09:04:23.505 INFO:teuthology.orchestra.run.smithi043.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T09:04:23.505 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:23.505 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 ::1/128 scope host 2024-04-25T09:04:23.505 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:23.505 INFO:teuthology.orchestra.run.smithi043.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:04:23.506 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9c brd ff:ff:ff:ff:ff:ff 2024-04-25T09:04:23.506 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f0 2024-04-25T09:04:23.506 INFO:teuthology.orchestra.run.smithi043.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T09:04:23.506 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:58 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:04:23.506 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f0 2024-04-25T09:04:23.506 INFO:teuthology.orchestra.run.smithi043.stdout: inet 172.21.15.43/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-04-25T09:04:23.506 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft 42155sec preferred_lft 42155sec 2024-04-25T09:04:23.506 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ec4:7aff:febd:1358/64 scope link noprefixroute 2024-04-25T09:04:23.506 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:23.506 INFO:teuthology.orchestra.run.smithi043.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:04:23.506 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:6b:fd:9d brd ff:ff:ff:ff:ff:ff 2024-04-25T09:04:23.506 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp7s0f1 2024-04-25T09:04:23.506 INFO:teuthology.orchestra.run.smithi043.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T09:04:23.507 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 0c:c4:7a:bd:13:59 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:04:23.507 INFO:teuthology.orchestra.run.smithi043.stdout: altname enp1s0f1 2024-04-25T09:04:23.507 INFO:teuthology.orchestra.run.smithi043.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T09:04:23.507 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 12:2c:7a:72:36:14 brd ff:ff:ff:ff:ff:ff 2024-04-25T09:04:23.507 INFO:teuthology.orchestra.run.smithi043.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T09:04:23.507 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:23.507 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::ccff:4eff:fe9d:c8bf/64 scope link 2024-04-25T09:04:23.507 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:23.507 INFO:teuthology.orchestra.run.smithi043.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:04:23.507 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 16:9a:63:37:46:24 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:04:23.507 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::149a:63ff:fe37:4624/64 scope link 2024-04-25T09:04:23.507 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:23.507 INFO:teuthology.orchestra.run.smithi043.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T09:04:23.507 INFO:teuthology.orchestra.run.smithi043.stdout: link/ether 12:2c:7a:72:36:14 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:04:23.508 INFO:teuthology.orchestra.run.smithi043.stdout: inet6 fe80::102c:7aff:fe72:3614/64 scope link 2024-04-25T09:04:23.508 INFO:teuthology.orchestra.run.smithi043.stdout: valid_lft forever preferred_lft forever 2024-04-25T09:04:23.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:23.508 DEBUG:teuthology.orchestra.run.smithi043:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T09:04:23.568 INFO:teuthology.orchestra.run.smithi043.stdout:1 2024-04-25T09:04:23.569 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:23.570 DEBUG:teuthology.orchestra.run.smithi043:> route 2024-04-25T09:04:23.626 INFO:teuthology.orchestra.run.smithi043.stdout:Kernel IP routing table 2024-04-25T09:04:23.626 INFO:teuthology.orchestra.run.smithi043.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T09:04:23.627 INFO:teuthology.orchestra.run.smithi043.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-25T09:04:23.627 INFO:teuthology.orchestra.run.smithi043.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-25T09:04:23.627 INFO:teuthology.orchestra.run.smithi043.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T09:04:23.627 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:23.627 DEBUG:teuthology.orchestra.run.smithi043:> (cd / && exec bash -c ' 2024-04-25T09:04:23.627 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:04:23.627 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-25T09:04:23.627 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-25T09:04:23.627 DEBUG:teuthology.orchestra.run.smithi043:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-25T09:04:23.627 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:04:23.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:23.713 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-04-25T09:04:23.728 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:04:23.728 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:04:23.728 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:04:23.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:23.729 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link set brx.0 up 2024-04-25T09:04:23.792 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-25T09:04:23.793 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:04:23.793 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T09:04:23.848 INFO:teuthology.orchestra.run.smithi043.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T09:04:23.849 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T09:04:23.849 DEBUG:teuthology.orchestra.run.smithi043:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:04:23.904 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-04-25T09:04:23.969 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:04:24.025 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-04-25T09:04:24.025 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-04-25T09:04:24.025 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-04-25T09:04:24.025 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-04-25T09:04:24.025 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-04-25T09:04:24.026 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-04-25T09:04:24.026 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-04-25T09:04:24.026 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-04-25T09:04:24.026 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-04-25T09:04:24.026 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-04-25T09:04:24.026 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-04-25T09:04:24.026 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/selinux 2024-04-25T09:04:24.026 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:04:24.026 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-04-25T09:04:24.026 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-04-25T09:04:24.026 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-04-25T09:04:24.026 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-04-25T09:04:24.027 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:04:24.027 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-04-25T09:04:24.027 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-04-25T09:04:24.027 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T09:04:24.027 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T09:04:24.027 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T09:04:24.027 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T09:04:24.027 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-04-25T09:04:24.027 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-04-25T09:04:24.027 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:04:24.027 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:04:24.027 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:04:24.027 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:04:24.027 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-04-25T09:04:24.027 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:04:24.028 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:04:24.028 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:04:24.028 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:04:24.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:24.028 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-04-25T09:04:24.080 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T09:04:24.081 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-25T09:04:24.123 DEBUG:teuthology.orchestra.run.smithi043:> sudo modprobe fuse 2024-04-25T09:04:24.147 DEBUG:teuthology.orchestra.run.smithi043:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T09:04:24.203 INFO:teuthology.orchestra.run.smithi043.stdout:/proc 2024-04-25T09:04:24.203 INFO:teuthology.orchestra.run.smithi043.stdout:/sys 2024-04-25T09:04:24.203 INFO:teuthology.orchestra.run.smithi043.stdout:/dev 2024-04-25T09:04:24.203 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/security 2024-04-25T09:04:24.203 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/shm 2024-04-25T09:04:24.203 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/pts 2024-04-25T09:04:24.203 INFO:teuthology.orchestra.run.smithi043.stdout:/run 2024-04-25T09:04:24.203 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/cgroup 2024-04-25T09:04:24.203 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/pstore 2024-04-25T09:04:24.204 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/bpf 2024-04-25T09:04:24.204 INFO:teuthology.orchestra.run.smithi043.stdout:/ 2024-04-25T09:04:24.204 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/selinux 2024-04-25T09:04:24.204 INFO:teuthology.orchestra.run.smithi043.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T09:04:24.204 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/hugepages 2024-04-25T09:04:24.204 INFO:teuthology.orchestra.run.smithi043.stdout:/dev/mqueue 2024-04-25T09:04:24.204 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/debug 2024-04-25T09:04:24.204 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/tracing 2024-04-25T09:04:24.204 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysusers.service 2024-04-25T09:04:24.204 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/kernel/config 2024-04-25T09:04:24.204 INFO:teuthology.orchestra.run.smithi043.stdout:/sys/fs/fuse/connections 2024-04-25T09:04:24.204 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T09:04:24.204 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T09:04:24.205 INFO:teuthology.orchestra.run.smithi043.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T09:04:24.205 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T09:04:24.205 INFO:teuthology.orchestra.run.smithi043.stdout:/run/user/1000 2024-04-25T09:04:24.205 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph 2024-04-25T09:04:24.205 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T09:04:24.205 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T09:04:24.205 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T09:04:24.205 INFO:teuthology.orchestra.run.smithi043.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T09:04:24.205 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns 2024-04-25T09:04:24.206 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:04:24.206 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:04:24.206 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:04:24.206 INFO:teuthology.orchestra.run.smithi043.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:04:24.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:24.208 DEBUG:teuthology.orchestra.run.smithi043:> ls /sys/fs/fuse/connections 2024-04-25T09:04:24.209 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:2024-04-25T09:04:24.215+0000 7f89a100c480 -1 init, newargv = 0x561978491770 newargc=15 2024-04-25T09:04:24.209 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[96951]: starting ceph client 2024-04-25T09:04:24.209 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[96951]: starting fuse 2024-04-25T09:04:24.223 INFO:teuthology.orchestra.run.smithi043.stdout:43 2024-04-25T09:04:24.224 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [43] 2024-04-25T09:04:24.224 DEBUG:teuthology.orchestra.run.smithi043:> sudo stdin-killer -- python3 -c ' 2024-04-25T09:04:24.224 DEBUG:teuthology.orchestra.run.smithi043:> import glob 2024-04-25T09:04:24.224 DEBUG:teuthology.orchestra.run.smithi043:> import re 2024-04-25T09:04:24.224 DEBUG:teuthology.orchestra.run.smithi043:> import os 2024-04-25T09:04:24.224 DEBUG:teuthology.orchestra.run.smithi043:> import subprocess 2024-04-25T09:04:24.224 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T09:04:24.224 DEBUG:teuthology.orchestra.run.smithi043:> def _find_admin_socket(client_name): 2024-04-25T09:04:24.225 DEBUG:teuthology.orchestra.run.smithi043:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-25T09:04:24.225 DEBUG:teuthology.orchestra.run.smithi043:> files = glob.glob(asok_path) 2024-04-25T09:04:24.225 DEBUG:teuthology.orchestra.run.smithi043:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-25T09:04:24.225 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T09:04:24.225 DEBUG:teuthology.orchestra.run.smithi043:> # Given a non-glob path, it better be there 2024-04-25T09:04:24.225 DEBUG:teuthology.orchestra.run.smithi043:> if "*" not in asok_path: 2024-04-25T09:04:24.225 DEBUG:teuthology.orchestra.run.smithi043:> assert(len(files) == 1) 2024-04-25T09:04:24.225 DEBUG:teuthology.orchestra.run.smithi043:> return files[0] 2024-04-25T09:04:24.225 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T09:04:24.225 DEBUG:teuthology.orchestra.run.smithi043:> for f in files: 2024-04-25T09:04:24.225 DEBUG:teuthology.orchestra.run.smithi043:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T09:04:24.225 DEBUG:teuthology.orchestra.run.smithi043:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T09:04:24.225 DEBUG:teuthology.orchestra.run.smithi043:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T09:04:24.225 DEBUG:teuthology.orchestra.run.smithi043:> contents = proc_f.read() 2024-04-25T09:04:24.225 DEBUG:teuthology.orchestra.run.smithi043:> if mountpoint in contents: 2024-04-25T09:04:24.225 DEBUG:teuthology.orchestra.run.smithi043:> return f 2024-04-25T09:04:24.226 DEBUG:teuthology.orchestra.run.smithi043:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T09:04:24.226 DEBUG:teuthology.orchestra.run.smithi043:> 2024-04-25T09:04:24.226 DEBUG:teuthology.orchestra.run.smithi043:> print(_find_admin_socket("client.0")) 2024-04-25T09:04:24.226 DEBUG:teuthology.orchestra.run.smithi043:> ' 2024-04-25T09:04:24.345 INFO:teuthology.orchestra.run.smithi043.stdout:/var/run/ceph/ceph-client.0.96951.asok 2024-04-25T09:04:24.348 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T09:04:24.355 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.96951.asok 2024-04-25T09:04:24.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:24.355 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.96951.asok status 2024-04-25T09:04:24.491 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:04:24.491 INFO:teuthology.orchestra.run.smithi043.stdout: "metadata": { 2024-04-25T09:04:24.491 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T09:04:24.491 INFO:teuthology.orchestra.run.smithi043.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T09:04:24.491 INFO:teuthology.orchestra.run.smithi043.stdout: "entity_id": "0", 2024-04-25T09:04:24.491 INFO:teuthology.orchestra.run.smithi043.stdout: "hostname": "smithi043", 2024-04-25T09:04:24.491 INFO:teuthology.orchestra.run.smithi043.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-25T09:04:24.492 INFO:teuthology.orchestra.run.smithi043.stdout: "pid": "96951", 2024-04-25T09:04:24.492 INFO:teuthology.orchestra.run.smithi043.stdout: "root": "/" 2024-04-25T09:04:24.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:04:24.492 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_count": 0, 2024-04-25T09:04:24.492 INFO:teuthology.orchestra.run.smithi043.stdout: "dentry_pinned_count": 0, 2024-04-25T09:04:24.492 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 9915, 2024-04-25T09:04:24.492 INFO:teuthology.orchestra.run.smithi043.stdout: "inst": { 2024-04-25T09:04:24.492 INFO:teuthology.orchestra.run.smithi043.stdout: "name": { 2024-04-25T09:04:24.492 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "client", 2024-04-25T09:04:24.492 INFO:teuthology.orchestra.run.smithi043.stdout: "num": 9915 2024-04-25T09:04:24.492 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:04:24.492 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-04-25T09:04:24.492 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-04-25T09:04:24.492 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:04:24.492 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1598109953 2024-04-25T09:04:24.493 INFO:teuthology.orchestra.run.smithi043.stdout: } 2024-04-25T09:04:24.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:04:24.493 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": { 2024-04-25T09:04:24.493 INFO:teuthology.orchestra.run.smithi043.stdout: "type": "v1", 2024-04-25T09:04:24.493 INFO:teuthology.orchestra.run.smithi043.stdout: "addr": "192.168.0.1:0", 2024-04-25T09:04:24.493 INFO:teuthology.orchestra.run.smithi043.stdout: "nonce": 1598109953 2024-04-25T09:04:24.493 INFO:teuthology.orchestra.run.smithi043.stdout: }, 2024-04-25T09:04:24.493 INFO:teuthology.orchestra.run.smithi043.stdout: "inst_str": "client.9915 192.168.0.1:0/1598109953", 2024-04-25T09:04:24.493 INFO:teuthology.orchestra.run.smithi043.stdout: "addr_str": "192.168.0.1:0/1598109953", 2024-04-25T09:04:24.493 INFO:teuthology.orchestra.run.smithi043.stdout: "inode_count": 1, 2024-04-25T09:04:24.493 INFO:teuthology.orchestra.run.smithi043.stdout: "mds_epoch": 87, 2024-04-25T09:04:24.493 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch": 139, 2024-04-25T09:04:24.493 INFO:teuthology.orchestra.run.smithi043.stdout: "osd_epoch_barrier": 0, 2024-04-25T09:04:24.493 INFO:teuthology.orchestra.run.smithi043.stdout: "blocklisted": false, 2024-04-25T09:04:24.493 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs" 2024-04-25T09:04:24.494 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:04:24.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:24.499 DEBUG:teuthology.orchestra.run.smithi043:> stat --file-system '--printf=%T 2024-04-25T09:04:24.500 DEBUG:teuthology.orchestra.run.smithi043:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-25T09:04:24.515 INFO:teuthology.orchestra.run.smithi043.stdout:fuseblk 2024-04-25T09:04:24.515 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-25T09:04:24.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:24.515 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T09:04:24.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:24.580 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:04:24.581 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:04:24.581 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T09:04:24.581 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:04:24.581 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:04:24.581 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:04:24.581 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:04:24.661 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:24.661 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:24.661 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0110437 s, 94.9 MB/s 2024-04-25T09:04:24.674 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:24.675 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:24.675 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00938022 s, 112 MB/s 2024-04-25T09:04:24.686 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:24.686 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:24.686 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0106081 s, 98.8 MB/s 2024-04-25T09:04:24.697 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:24.697 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:24.697 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0085887 s, 122 MB/s 2024-04-25T09:04:24.708 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:24.708 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:24.708 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00875132 s, 120 MB/s 2024-04-25T09:04:24.719 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:24.719 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:24.719 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845518 s, 124 MB/s 2024-04-25T09:04:24.730 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:24.730 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:24.730 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815256 s, 129 MB/s 2024-04-25T09:04:24.741 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:24.741 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:24.741 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00889869 s, 118 MB/s 2024-04-25T09:04:24.752 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:24.752 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:24.752 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819326 s, 128 MB/s 2024-04-25T09:04:24.762 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:24.762 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:24.762 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744137 s, 141 MB/s 2024-04-25T09:04:24.774 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:24.774 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:24.774 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00918998 s, 114 MB/s 2024-04-25T09:04:24.785 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:24.785 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:24.785 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817555 s, 128 MB/s 2024-04-25T09:04:24.795 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:24.795 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:24.795 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773493 s, 136 MB/s 2024-04-25T09:04:24.806 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:24.806 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:24.806 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00803212 s, 131 MB/s 2024-04-25T09:04:24.816 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:24.816 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:24.816 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0078725 s, 133 MB/s 2024-04-25T09:04:24.828 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:24.828 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:24.828 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00893282 s, 117 MB/s 2024-04-25T09:04:24.839 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:24.839 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:24.839 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.008669 s, 121 MB/s 2024-04-25T09:04:24.849 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:24.849 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:24.849 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776975 s, 135 MB/s 2024-04-25T09:04:24.859 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:24.859 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:24.859 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00817133 s, 128 MB/s 2024-04-25T09:04:24.869 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:24.870 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:24.870 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757541 s, 138 MB/s 2024-04-25T09:04:24.882 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:24.882 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:24.882 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0100636 s, 104 MB/s 2024-04-25T09:04:24.892 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:24.892 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:24.892 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804855 s, 130 MB/s 2024-04-25T09:04:24.902 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:24.902 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:24.902 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758787 s, 138 MB/s 2024-04-25T09:04:24.911 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:24.911 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:24.912 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732251 s, 143 MB/s 2024-04-25T09:04:24.922 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:24.922 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:24.922 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834149 s, 126 MB/s 2024-04-25T09:04:24.932 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:24.932 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:24.932 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00768893 s, 136 MB/s 2024-04-25T09:04:24.941 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:24.941 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:24.941 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711481 s, 147 MB/s 2024-04-25T09:04:24.951 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:24.951 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:24.951 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00775241 s, 135 MB/s 2024-04-25T09:04:24.961 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:24.961 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:24.961 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741057 s, 141 MB/s 2024-04-25T09:04:24.972 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:24.972 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:24.972 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00888897 s, 118 MB/s 2024-04-25T09:04:24.982 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:24.982 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:24.982 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739504 s, 142 MB/s 2024-04-25T09:04:24.993 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:24.993 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:24.993 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00869532 s, 121 MB/s 2024-04-25T09:04:24.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:24.994 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:04:24.994 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:04:24.994 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T09:04:24.995 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:04:24.995 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:04:24.995 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:04:24.995 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:04:25.062 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.063 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.063 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766003 s, 137 MB/s 2024-04-25T09:04:25.072 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.072 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.072 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732444 s, 143 MB/s 2024-04-25T09:04:25.083 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.084 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.084 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00948082 s, 111 MB/s 2024-04-25T09:04:25.094 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.094 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.094 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725094 s, 145 MB/s 2024-04-25T09:04:25.102 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.102 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.102 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706518 s, 148 MB/s 2024-04-25T09:04:25.113 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.113 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.114 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0087041 s, 120 MB/s 2024-04-25T09:04:25.123 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.123 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.123 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729259 s, 144 MB/s 2024-04-25T09:04:25.134 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.134 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.134 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00857316 s, 122 MB/s 2024-04-25T09:04:25.144 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.144 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.144 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757596 s, 138 MB/s 2024-04-25T09:04:25.153 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.154 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.154 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076803 s, 137 MB/s 2024-04-25T09:04:25.163 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.163 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.163 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748381 s, 140 MB/s 2024-04-25T09:04:25.173 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.173 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.173 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760264 s, 138 MB/s 2024-04-25T09:04:25.183 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.183 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.183 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736745 s, 142 MB/s 2024-04-25T09:04:25.194 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.194 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.194 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00924275 s, 113 MB/s 2024-04-25T09:04:25.204 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.205 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.205 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00807401 s, 130 MB/s 2024-04-25T09:04:25.215 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.215 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.215 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793291 s, 132 MB/s 2024-04-25T09:04:25.225 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.225 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.225 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738395 s, 142 MB/s 2024-04-25T09:04:25.235 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.235 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.235 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797698 s, 131 MB/s 2024-04-25T09:04:25.245 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.245 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.245 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729655 s, 144 MB/s 2024-04-25T09:04:25.256 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.256 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.256 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084017 s, 125 MB/s 2024-04-25T09:04:25.265 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.266 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.266 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757711 s, 138 MB/s 2024-04-25T09:04:25.276 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.276 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.276 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00805913 s, 130 MB/s 2024-04-25T09:04:25.286 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.286 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.286 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756393 s, 139 MB/s 2024-04-25T09:04:25.296 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.296 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.296 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793732 s, 132 MB/s 2024-04-25T09:04:25.307 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.307 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.307 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00891234 s, 118 MB/s 2024-04-25T09:04:25.317 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.317 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.317 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787716 s, 133 MB/s 2024-04-25T09:04:25.327 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.327 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.327 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00787793 s, 133 MB/s 2024-04-25T09:04:25.337 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.337 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.337 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792956 s, 132 MB/s 2024-04-25T09:04:25.347 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.347 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.348 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783855 s, 134 MB/s 2024-04-25T09:04:25.357 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.357 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.357 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757019 s, 139 MB/s 2024-04-25T09:04:25.369 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.369 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.369 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00904948 s, 116 MB/s 2024-04-25T09:04:25.379 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.379 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.379 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00790024 s, 133 MB/s 2024-04-25T09:04:25.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:25.380 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:04:25.380 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:04:25.380 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T09:04:25.380 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:04:25.380 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:04:25.381 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:04:25.381 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:04:25.448 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.448 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.448 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743027 s, 141 MB/s 2024-04-25T09:04:25.458 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.458 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.458 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725473 s, 145 MB/s 2024-04-25T09:04:25.468 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.468 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.468 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00789547 s, 133 MB/s 2024-04-25T09:04:25.479 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.479 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.479 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00880609 s, 119 MB/s 2024-04-25T09:04:25.489 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.489 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.489 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727383 s, 144 MB/s 2024-04-25T09:04:25.500 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.500 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.500 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00867851 s, 121 MB/s 2024-04-25T09:04:25.511 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.511 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.511 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.008873 s, 118 MB/s 2024-04-25T09:04:25.522 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.522 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.522 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00867901 s, 121 MB/s 2024-04-25T09:04:25.531 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.532 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.532 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750432 s, 140 MB/s 2024-04-25T09:04:25.542 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.542 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.542 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075954 s, 138 MB/s 2024-04-25T09:04:25.553 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.553 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.553 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00911923 s, 115 MB/s 2024-04-25T09:04:25.564 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.564 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.564 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842326 s, 124 MB/s 2024-04-25T09:04:25.573 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.573 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.573 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713937 s, 147 MB/s 2024-04-25T09:04:25.584 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.584 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.584 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00869416 s, 121 MB/s 2024-04-25T09:04:25.595 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.595 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.595 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00799121 s, 131 MB/s 2024-04-25T09:04:25.606 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.606 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.606 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00902437 s, 116 MB/s 2024-04-25T09:04:25.616 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.616 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.616 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00803027 s, 131 MB/s 2024-04-25T09:04:25.626 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.626 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.626 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797003 s, 132 MB/s 2024-04-25T09:04:25.637 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.637 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.637 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861906 s, 122 MB/s 2024-04-25T09:04:25.648 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.648 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.648 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802593 s, 131 MB/s 2024-04-25T09:04:25.659 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.659 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.659 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0087696 s, 120 MB/s 2024-04-25T09:04:25.669 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.669 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.669 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752291 s, 139 MB/s 2024-04-25T09:04:25.679 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.679 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.679 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792165 s, 132 MB/s 2024-04-25T09:04:25.688 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.688 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.688 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720631 s, 146 MB/s 2024-04-25T09:04:25.698 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.698 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.698 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709605 s, 148 MB/s 2024-04-25T09:04:25.709 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.709 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.709 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00873827 s, 120 MB/s 2024-04-25T09:04:25.719 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.719 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.720 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773937 s, 135 MB/s 2024-04-25T09:04:25.730 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.730 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.730 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00877779 s, 119 MB/s 2024-04-25T09:04:25.740 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.740 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.741 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793388 s, 132 MB/s 2024-04-25T09:04:25.751 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.751 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.751 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802464 s, 131 MB/s 2024-04-25T09:04:25.761 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.762 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.762 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00859822 s, 122 MB/s 2024-04-25T09:04:25.772 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.772 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.772 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0080198 s, 131 MB/s 2024-04-25T09:04:25.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:25.773 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:04:25.773 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:04:25.773 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T09:04:25.773 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:04:25.773 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:04:25.773 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:04:25.773 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:04:25.841 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.842 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.842 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769231 s, 136 MB/s 2024-04-25T09:04:25.851 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.851 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.851 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722566 s, 145 MB/s 2024-04-25T09:04:25.862 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.862 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.862 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00856865 s, 122 MB/s 2024-04-25T09:04:25.873 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.874 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.874 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00925603 s, 113 MB/s 2024-04-25T09:04:25.884 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.884 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.884 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00807999 s, 130 MB/s 2024-04-25T09:04:25.894 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.894 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.894 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774946 s, 135 MB/s 2024-04-25T09:04:25.904 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.905 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.905 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00781346 s, 134 MB/s 2024-04-25T09:04:25.915 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.915 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.915 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00800396 s, 131 MB/s 2024-04-25T09:04:25.926 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.926 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.926 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00871511 s, 120 MB/s 2024-04-25T09:04:25.937 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.937 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.937 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00865501 s, 121 MB/s 2024-04-25T09:04:25.947 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.947 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.947 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00786513 s, 133 MB/s 2024-04-25T09:04:25.957 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.957 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.958 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082304 s, 127 MB/s 2024-04-25T09:04:25.967 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.967 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.967 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0077745 s, 135 MB/s 2024-04-25T09:04:25.977 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.977 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.978 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0077472 s, 135 MB/s 2024-04-25T09:04:25.989 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.989 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.989 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00922098 s, 114 MB/s 2024-04-25T09:04:25.999 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:25.999 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:25.999 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770018 s, 136 MB/s 2024-04-25T09:04:26.010 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.010 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.010 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00864161 s, 121 MB/s 2024-04-25T09:04:26.020 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.020 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.020 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780688 s, 134 MB/s 2024-04-25T09:04:26.032 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.032 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.032 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0086024 s, 122 MB/s 2024-04-25T09:04:26.041 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.042 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.042 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00847602 s, 124 MB/s 2024-04-25T09:04:26.052 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.052 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.052 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771497 s, 136 MB/s 2024-04-25T09:04:26.062 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.062 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.062 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00773924 s, 135 MB/s 2024-04-25T09:04:26.073 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.073 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.073 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842914 s, 124 MB/s 2024-04-25T09:04:26.083 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.083 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.083 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0079225 s, 132 MB/s 2024-04-25T09:04:26.092 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.093 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.093 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742446 s, 141 MB/s 2024-04-25T09:04:26.103 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.103 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.103 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791777 s, 132 MB/s 2024-04-25T09:04:26.114 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.114 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.114 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00897916 s, 117 MB/s 2024-04-25T09:04:26.124 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.125 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.125 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0080223 s, 131 MB/s 2024-04-25T09:04:26.135 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.135 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.135 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00803896 s, 130 MB/s 2024-04-25T09:04:26.146 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.146 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.146 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00902608 s, 116 MB/s 2024-04-25T09:04:26.156 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.156 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.156 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00784874 s, 134 MB/s 2024-04-25T09:04:26.167 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.167 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.167 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.008551 s, 123 MB/s 2024-04-25T09:04:26.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:26.168 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:04:26.168 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:04:26.168 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T09:04:26.168 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:04:26.169 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:04:26.169 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:04:26.169 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:04:26.237 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.237 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.238 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00849039 s, 124 MB/s 2024-04-25T09:04:26.248 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.248 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.248 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792375 s, 132 MB/s 2024-04-25T09:04:26.258 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.258 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.258 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00794237 s, 132 MB/s 2024-04-25T09:04:26.267 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.268 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.268 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721301 s, 145 MB/s 2024-04-25T09:04:26.277 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.277 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.277 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750291 s, 140 MB/s 2024-04-25T09:04:26.288 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.288 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.288 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0079221 s, 132 MB/s 2024-04-25T09:04:26.298 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.298 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.298 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007872 s, 133 MB/s 2024-04-25T09:04:26.309 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.309 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.309 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00887157 s, 118 MB/s 2024-04-25T09:04:26.319 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.319 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.319 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792286 s, 132 MB/s 2024-04-25T09:04:26.329 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.329 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.329 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759123 s, 138 MB/s 2024-04-25T09:04:26.339 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.339 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.339 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766414 s, 137 MB/s 2024-04-25T09:04:26.349 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.349 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.349 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00781786 s, 134 MB/s 2024-04-25T09:04:26.360 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.360 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.360 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.008568 s, 122 MB/s 2024-04-25T09:04:26.371 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.371 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.371 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00853898 s, 123 MB/s 2024-04-25T09:04:26.381 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.382 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.382 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841164 s, 125 MB/s 2024-04-25T09:04:26.391 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.391 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.392 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754959 s, 139 MB/s 2024-04-25T09:04:26.402 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.402 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.403 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00857563 s, 122 MB/s 2024-04-25T09:04:26.412 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.412 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.412 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736788 s, 142 MB/s 2024-04-25T09:04:26.422 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.422 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.423 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788678 s, 133 MB/s 2024-04-25T09:04:26.434 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.434 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.434 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00898822 s, 117 MB/s 2024-04-25T09:04:26.445 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.445 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.445 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00876389 s, 120 MB/s 2024-04-25T09:04:26.456 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.456 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.456 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851755 s, 123 MB/s 2024-04-25T09:04:26.466 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.466 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.466 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00775525 s, 135 MB/s 2024-04-25T09:04:26.476 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.476 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.476 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00795505 s, 132 MB/s 2024-04-25T09:04:26.487 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.487 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.487 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0085453 s, 123 MB/s 2024-04-25T09:04:26.497 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.497 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.497 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776451 s, 135 MB/s 2024-04-25T09:04:26.507 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.507 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.507 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769974 s, 136 MB/s 2024-04-25T09:04:26.517 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.517 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.517 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758452 s, 138 MB/s 2024-04-25T09:04:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.527 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00785271 s, 134 MB/s 2024-04-25T09:04:26.536 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.537 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.537 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743227 s, 141 MB/s 2024-04-25T09:04:26.547 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.547 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.547 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00797997 s, 131 MB/s 2024-04-25T09:04:26.557 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.557 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.557 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00778937 s, 135 MB/s 2024-04-25T09:04:26.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:26.558 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:04:26.558 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:04:26.558 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T09:04:26.558 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:04:26.558 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:04:26.558 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:04:26.558 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:04:26.627 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.627 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.627 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00785006 s, 134 MB/s 2024-04-25T09:04:26.638 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.638 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.638 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00802362 s, 131 MB/s 2024-04-25T09:04:26.648 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.648 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.648 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00794677 s, 132 MB/s 2024-04-25T09:04:26.659 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.659 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.659 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00885798 s, 118 MB/s 2024-04-25T09:04:26.669 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.669 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.670 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00800601 s, 131 MB/s 2024-04-25T09:04:26.679 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.679 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.679 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722086 s, 145 MB/s 2024-04-25T09:04:26.690 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.690 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.690 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00869315 s, 121 MB/s 2024-04-25T09:04:26.700 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.700 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.701 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780273 s, 134 MB/s 2024-04-25T09:04:26.710 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.710 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.710 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767644 s, 137 MB/s 2024-04-25T09:04:26.720 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.720 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.720 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770566 s, 136 MB/s 2024-04-25T09:04:26.731 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.731 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.731 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0086786 s, 121 MB/s 2024-04-25T09:04:26.741 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.741 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.741 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00789079 s, 133 MB/s 2024-04-25T09:04:26.751 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.751 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.751 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756266 s, 139 MB/s 2024-04-25T09:04:26.762 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.762 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.762 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00900922 s, 116 MB/s 2024-04-25T09:04:26.772 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.772 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.772 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00805938 s, 130 MB/s 2024-04-25T09:04:26.782 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.782 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.782 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730138 s, 144 MB/s 2024-04-25T09:04:26.792 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.792 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.792 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00769101 s, 136 MB/s 2024-04-25T09:04:26.803 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.803 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.803 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0089791 s, 117 MB/s 2024-04-25T09:04:26.813 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.813 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.813 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774455 s, 135 MB/s 2024-04-25T09:04:26.824 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.824 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.824 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00866792 s, 121 MB/s 2024-04-25T09:04:26.835 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.835 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.835 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00878281 s, 119 MB/s 2024-04-25T09:04:26.845 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.845 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.845 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760053 s, 138 MB/s 2024-04-25T09:04:26.855 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.855 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.855 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723296 s, 145 MB/s 2024-04-25T09:04:26.866 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.866 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.866 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00874019 s, 120 MB/s 2024-04-25T09:04:26.876 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.876 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.876 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00789655 s, 133 MB/s 2024-04-25T09:04:26.886 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.887 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.887 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00870041 s, 121 MB/s 2024-04-25T09:04:26.896 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.897 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.897 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761561 s, 138 MB/s 2024-04-25T09:04:26.906 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.907 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.907 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750152 s, 140 MB/s 2024-04-25T09:04:26.916 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.917 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.917 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00796555 s, 132 MB/s 2024-04-25T09:04:26.927 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.927 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.927 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00796682 s, 132 MB/s 2024-04-25T09:04:26.938 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.938 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.938 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00877126 s, 120 MB/s 2024-04-25T09:04:26.948 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:26.948 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:26.948 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00780086 s, 134 MB/s 2024-04-25T09:04:26.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:26.949 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:04:26.949 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:04:26.949 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T09:04:26.949 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:04:26.949 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:04:26.950 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:04:26.950 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:04:27.018 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.018 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.018 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759245 s, 138 MB/s 2024-04-25T09:04:27.028 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.028 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.028 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00801215 s, 131 MB/s 2024-04-25T09:04:27.038 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.038 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.038 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804834 s, 130 MB/s 2024-04-25T09:04:27.048 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.048 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.048 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00786678 s, 133 MB/s 2024-04-25T09:04:27.058 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.058 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.058 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759076 s, 138 MB/s 2024-04-25T09:04:27.069 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.069 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.069 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00894545 s, 117 MB/s 2024-04-25T09:04:27.079 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.079 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.079 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755918 s, 139 MB/s 2024-04-25T09:04:27.089 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.089 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.089 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754161 s, 139 MB/s 2024-04-25T09:04:27.099 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.099 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.100 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084878 s, 124 MB/s 2024-04-25T09:04:27.110 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.110 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.110 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783018 s, 134 MB/s 2024-04-25T09:04:27.119 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.119 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.119 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699241 s, 150 MB/s 2024-04-25T09:04:27.130 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.130 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.130 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084383 s, 124 MB/s 2024-04-25T09:04:27.139 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.139 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.139 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705207 s, 149 MB/s 2024-04-25T09:04:27.149 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.149 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.150 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007513 s, 140 MB/s 2024-04-25T09:04:27.159 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.159 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.159 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726236 s, 144 MB/s 2024-04-25T09:04:27.168 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.169 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.169 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757123 s, 138 MB/s 2024-04-25T09:04:27.179 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.179 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.179 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792055 s, 132 MB/s 2024-04-25T09:04:27.189 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.190 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.190 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00873139 s, 120 MB/s 2024-04-25T09:04:27.199 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.199 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.200 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738165 s, 142 MB/s 2024-04-25T09:04:27.209 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.209 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.209 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729971 s, 144 MB/s 2024-04-25T09:04:27.219 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.219 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.219 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744009 s, 141 MB/s 2024-04-25T09:04:27.229 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.229 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.229 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774288 s, 135 MB/s 2024-04-25T09:04:27.238 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.238 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.239 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733316 s, 143 MB/s 2024-04-25T09:04:27.248 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.248 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.248 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721557 s, 145 MB/s 2024-04-25T09:04:27.257 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.258 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.258 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736774 s, 142 MB/s 2024-04-25T09:04:27.267 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.268 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.269 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751796 s, 139 MB/s 2024-04-25T09:04:27.277 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.277 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.277 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726744 s, 144 MB/s 2024-04-25T09:04:27.286 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.286 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.286 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073819 s, 142 MB/s 2024-04-25T09:04:27.297 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.297 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.297 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833566 s, 126 MB/s 2024-04-25T09:04:27.306 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.306 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.306 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725142 s, 145 MB/s 2024-04-25T09:04:27.317 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.317 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.317 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836922 s, 125 MB/s 2024-04-25T09:04:27.326 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.326 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.327 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703904 s, 149 MB/s 2024-04-25T09:04:27.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:27.328 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:04:27.328 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:04:27.328 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T09:04:27.328 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:04:27.328 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:04:27.328 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:04:27.328 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:04:27.397 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.397 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.397 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772714 s, 136 MB/s 2024-04-25T09:04:27.407 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.407 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.407 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714899 s, 147 MB/s 2024-04-25T09:04:27.416 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.416 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.417 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758873 s, 138 MB/s 2024-04-25T09:04:27.427 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.427 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.427 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826197 s, 127 MB/s 2024-04-25T09:04:27.438 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.438 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.438 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00849684 s, 123 MB/s 2024-04-25T09:04:27.449 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.449 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.449 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00856944 s, 122 MB/s 2024-04-25T09:04:27.459 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.459 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.459 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755207 s, 139 MB/s 2024-04-25T09:04:27.469 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.469 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.469 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756337 s, 139 MB/s 2024-04-25T09:04:27.478 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.478 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.478 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761338 s, 138 MB/s 2024-04-25T09:04:27.487 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.488 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.488 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068646 s, 153 MB/s 2024-04-25T09:04:27.498 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.498 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.498 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828522 s, 127 MB/s 2024-04-25T09:04:27.509 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.509 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.509 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0085661 s, 122 MB/s 2024-04-25T09:04:27.520 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.520 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.520 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846095 s, 124 MB/s 2024-04-25T09:04:27.530 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.530 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.531 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833389 s, 126 MB/s 2024-04-25T09:04:27.540 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.541 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.541 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762417 s, 138 MB/s 2024-04-25T09:04:27.550 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.550 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.551 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765692 s, 137 MB/s 2024-04-25T09:04:27.561 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.562 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.562 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00872198 s, 120 MB/s 2024-04-25T09:04:27.571 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.571 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.571 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716351 s, 146 MB/s 2024-04-25T09:04:27.581 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.581 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.581 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746954 s, 140 MB/s 2024-04-25T09:04:27.591 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.591 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.591 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738509 s, 142 MB/s 2024-04-25T09:04:27.600 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.601 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.601 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754108 s, 139 MB/s 2024-04-25T09:04:27.610 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.610 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.610 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706303 s, 148 MB/s 2024-04-25T09:04:27.619 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.619 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.619 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673351 s, 156 MB/s 2024-04-25T09:04:27.628 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.629 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.629 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743795 s, 141 MB/s 2024-04-25T09:04:27.639 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.639 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.640 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00867378 s, 121 MB/s 2024-04-25T09:04:27.649 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.649 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.649 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715191 s, 147 MB/s 2024-04-25T09:04:27.659 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.659 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.659 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00715327 s, 147 MB/s 2024-04-25T09:04:27.668 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.668 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.668 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724464 s, 145 MB/s 2024-04-25T09:04:27.678 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.678 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.679 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811552 s, 129 MB/s 2024-04-25T09:04:27.689 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.689 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.689 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00804029 s, 130 MB/s 2024-04-25T09:04:27.698 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.699 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.699 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734917 s, 143 MB/s 2024-04-25T09:04:27.708 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.708 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.708 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698754 s, 150 MB/s 2024-04-25T09:04:27.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:27.709 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:04:27.709 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:04:27.709 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T09:04:27.709 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:04:27.709 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:04:27.709 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:04:27.709 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:04:27.777 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.777 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.777 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750893 s, 140 MB/s 2024-04-25T09:04:27.787 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.787 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.787 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767705 s, 137 MB/s 2024-04-25T09:04:27.797 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.798 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.798 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00847227 s, 124 MB/s 2024-04-25T09:04:27.808 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.808 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.808 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838433 s, 125 MB/s 2024-04-25T09:04:27.818 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.818 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.818 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752502 s, 139 MB/s 2024-04-25T09:04:27.828 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.828 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.828 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00696476 s, 151 MB/s 2024-04-25T09:04:27.837 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.837 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.837 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740776 s, 142 MB/s 2024-04-25T09:04:27.847 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.847 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.847 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747904 s, 140 MB/s 2024-04-25T09:04:27.856 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.857 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.857 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732633 s, 143 MB/s 2024-04-25T09:04:27.866 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.866 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.866 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686317 s, 153 MB/s 2024-04-25T09:04:27.876 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.876 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.876 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832314 s, 126 MB/s 2024-04-25T09:04:27.886 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.886 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.886 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743482 s, 141 MB/s 2024-04-25T09:04:27.895 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.895 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.895 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695468 s, 151 MB/s 2024-04-25T09:04:27.905 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.905 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.905 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726233 s, 144 MB/s 2024-04-25T09:04:27.914 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.914 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.914 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673718 s, 156 MB/s 2024-04-25T09:04:27.924 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.924 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.924 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832242 s, 126 MB/s 2024-04-25T09:04:27.933 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.933 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.933 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676561 s, 155 MB/s 2024-04-25T09:04:27.943 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.943 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.943 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735525 s, 143 MB/s 2024-04-25T09:04:27.953 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.953 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.953 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690989 s, 152 MB/s 2024-04-25T09:04:27.963 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.963 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.963 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759794 s, 138 MB/s 2024-04-25T09:04:27.972 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.972 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.972 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732277 s, 143 MB/s 2024-04-25T09:04:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.982 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716322 s, 146 MB/s 2024-04-25T09:04:27.991 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:27.991 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:27.991 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732017 s, 143 MB/s 2024-04-25T09:04:28.000 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.000 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.000 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736319 s, 142 MB/s 2024-04-25T09:04:28.010 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.010 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.010 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070811 s, 148 MB/s 2024-04-25T09:04:28.019 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.019 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.019 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726674 s, 144 MB/s 2024-04-25T09:04:28.029 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.029 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.029 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755695 s, 139 MB/s 2024-04-25T09:04:28.039 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.039 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.039 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708249 s, 148 MB/s 2024-04-25T09:04:28.049 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.049 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.050 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841264 s, 125 MB/s 2024-04-25T09:04:28.059 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.059 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.059 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749351 s, 140 MB/s 2024-04-25T09:04:28.069 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.069 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.069 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728476 s, 144 MB/s 2024-04-25T09:04:28.079 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.079 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.079 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753122 s, 139 MB/s 2024-04-25T09:04:28.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:28.080 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:04:28.080 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:04:28.080 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T09:04:28.080 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:04:28.080 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:04:28.080 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:04:28.080 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:04:28.148 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.148 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.149 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777269 s, 135 MB/s 2024-04-25T09:04:28.159 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.159 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.159 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838512 s, 125 MB/s 2024-04-25T09:04:28.169 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.169 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.169 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758112 s, 138 MB/s 2024-04-25T09:04:28.178 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.178 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.178 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708832 s, 148 MB/s 2024-04-25T09:04:28.188 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.188 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.188 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075079 s, 140 MB/s 2024-04-25T09:04:28.198 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.198 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.199 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825089 s, 127 MB/s 2024-04-25T09:04:28.208 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.208 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.208 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073314 s, 143 MB/s 2024-04-25T09:04:28.218 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.218 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.218 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735272 s, 143 MB/s 2024-04-25T09:04:28.228 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.228 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.228 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753965 s, 139 MB/s 2024-04-25T09:04:28.237 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.237 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.237 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00698415 s, 150 MB/s 2024-04-25T09:04:28.246 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.247 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.247 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073175 s, 143 MB/s 2024-04-25T09:04:28.257 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.257 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.257 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822033 s, 128 MB/s 2024-04-25T09:04:28.268 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.268 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.268 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00849673 s, 123 MB/s 2024-04-25T09:04:28.278 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.278 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.278 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776537 s, 135 MB/s 2024-04-25T09:04:28.288 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.288 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.288 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749165 s, 140 MB/s 2024-04-25T09:04:28.297 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.297 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.297 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00701383 s, 150 MB/s 2024-04-25T09:04:28.307 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.307 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.307 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748176 s, 140 MB/s 2024-04-25T09:04:28.317 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.317 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.317 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708867 s, 148 MB/s 2024-04-25T09:04:28.328 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.328 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.328 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851443 s, 123 MB/s 2024-04-25T09:04:28.337 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.337 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.337 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702797 s, 149 MB/s 2024-04-25T09:04:28.346 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.346 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.346 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069073 s, 152 MB/s 2024-04-25T09:04:28.356 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.356 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.356 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735056 s, 143 MB/s 2024-04-25T09:04:28.365 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.366 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.366 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740137 s, 142 MB/s 2024-04-25T09:04:28.375 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.375 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.375 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751905 s, 139 MB/s 2024-04-25T09:04:28.385 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.385 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.385 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718418 s, 146 MB/s 2024-04-25T09:04:28.395 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.395 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.395 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752006 s, 139 MB/s 2024-04-25T09:04:28.406 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.406 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.406 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820543 s, 128 MB/s 2024-04-25T09:04:28.415 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.415 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.415 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749231 s, 140 MB/s 2024-04-25T09:04:28.424 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.424 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.424 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692927 s, 151 MB/s 2024-04-25T09:04:28.435 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.435 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.435 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851217 s, 123 MB/s 2024-04-25T09:04:28.445 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.445 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.445 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758836 s, 138 MB/s 2024-04-25T09:04:28.454 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.454 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.455 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728977 s, 144 MB/s 2024-04-25T09:04:28.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:28.456 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:04:28.456 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:04:28.456 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T09:04:28.456 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:04:28.456 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:04:28.456 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:04:28.456 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:04:28.524 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.524 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.524 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729558 s, 144 MB/s 2024-04-25T09:04:28.534 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.534 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.534 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734009 s, 143 MB/s 2024-04-25T09:04:28.545 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.545 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.545 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00852993 s, 123 MB/s 2024-04-25T09:04:28.555 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.555 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.556 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846616 s, 124 MB/s 2024-04-25T09:04:28.565 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.565 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.566 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0079388 s, 132 MB/s 2024-04-25T09:04:28.575 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.575 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.575 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731437 s, 143 MB/s 2024-04-25T09:04:28.586 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.586 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.586 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00878151 s, 119 MB/s 2024-04-25T09:04:28.597 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.597 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.597 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0085396 s, 123 MB/s 2024-04-25T09:04:28.607 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.607 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.607 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824906 s, 127 MB/s 2024-04-25T09:04:28.616 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.616 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.616 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689237 s, 152 MB/s 2024-04-25T09:04:28.626 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.626 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.626 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071907 s, 146 MB/s 2024-04-25T09:04:28.637 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.637 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.638 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00908878 s, 115 MB/s 2024-04-25T09:04:28.647 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.647 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.647 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732952 s, 143 MB/s 2024-04-25T09:04:28.657 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.657 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.657 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755103 s, 139 MB/s 2024-04-25T09:04:28.667 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.667 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.667 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075826 s, 138 MB/s 2024-04-25T09:04:28.677 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.677 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.677 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822352 s, 128 MB/s 2024-04-25T09:04:28.687 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.687 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.687 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741383 s, 141 MB/s 2024-04-25T09:04:28.698 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.698 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.698 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00866356 s, 121 MB/s 2024-04-25T09:04:28.708 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.709 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.709 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00764682 s, 137 MB/s 2024-04-25T09:04:28.718 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.718 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.719 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747845 s, 140 MB/s 2024-04-25T09:04:28.728 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.728 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.728 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748411 s, 140 MB/s 2024-04-25T09:04:28.739 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.740 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.740 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00912129 s, 115 MB/s 2024-04-25T09:04:28.749 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.750 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.750 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762258 s, 138 MB/s 2024-04-25T09:04:28.759 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.759 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.759 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757244 s, 138 MB/s 2024-04-25T09:04:28.769 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.769 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.769 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748782 s, 140 MB/s 2024-04-25T09:04:28.780 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.780 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.780 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816181 s, 128 MB/s 2024-04-25T09:04:28.791 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.791 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.791 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861901 s, 122 MB/s 2024-04-25T09:04:28.801 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.801 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.801 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826153 s, 127 MB/s 2024-04-25T09:04:28.810 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.810 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.811 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697872 s, 150 MB/s 2024-04-25T09:04:28.820 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.820 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.820 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726696 s, 144 MB/s 2024-04-25T09:04:28.831 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.831 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.831 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825738 s, 127 MB/s 2024-04-25T09:04:28.841 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.841 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.841 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744095 s, 141 MB/s 2024-04-25T09:04:28.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:28.842 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:04:28.842 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:04:28.842 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T09:04:28.842 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:04:28.842 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:04:28.842 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:04:28.842 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:04:28.910 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.910 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.910 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753259 s, 139 MB/s 2024-04-25T09:04:28.920 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.920 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.920 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076391 s, 137 MB/s 2024-04-25T09:04:28.929 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.930 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.930 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074824 s, 140 MB/s 2024-04-25T09:04:28.939 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.939 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.939 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733447 s, 143 MB/s 2024-04-25T09:04:28.948 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.948 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.948 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719351 s, 146 MB/s 2024-04-25T09:04:28.959 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.959 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.959 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00843963 s, 124 MB/s 2024-04-25T09:04:28.970 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.970 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.970 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00847184 s, 124 MB/s 2024-04-25T09:04:28.981 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.981 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.981 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00897259 s, 117 MB/s 2024-04-25T09:04:28.992 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:28.992 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:28.992 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00849869 s, 123 MB/s 2024-04-25T09:04:29.003 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.003 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.003 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0086174 s, 122 MB/s 2024-04-25T09:04:29.012 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.012 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.012 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729916 s, 144 MB/s 2024-04-25T09:04:29.022 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.022 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.022 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709602 s, 148 MB/s 2024-04-25T09:04:29.032 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.032 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.032 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737963 s, 142 MB/s 2024-04-25T09:04:29.041 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.042 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.042 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771002 s, 136 MB/s 2024-04-25T09:04:29.053 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.054 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.054 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00933685 s, 112 MB/s 2024-04-25T09:04:29.065 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.065 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.065 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00895423 s, 117 MB/s 2024-04-25T09:04:29.074 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.074 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.075 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766271 s, 137 MB/s 2024-04-25T09:04:29.086 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.086 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.086 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00934463 s, 112 MB/s 2024-04-25T09:04:29.097 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.097 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.097 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00794408 s, 132 MB/s 2024-04-25T09:04:29.106 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.106 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.106 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741237 s, 141 MB/s 2024-04-25T09:04:29.118 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.118 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.118 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00908389 s, 115 MB/s 2024-04-25T09:04:29.129 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.129 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.129 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00880191 s, 119 MB/s 2024-04-25T09:04:29.140 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.140 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.140 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861923 s, 122 MB/s 2024-04-25T09:04:29.151 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.151 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.151 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826609 s, 127 MB/s 2024-04-25T09:04:29.160 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.161 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.161 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716112 s, 146 MB/s 2024-04-25T09:04:29.170 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.170 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.170 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746188 s, 141 MB/s 2024-04-25T09:04:29.181 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.181 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.181 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0086438 s, 121 MB/s 2024-04-25T09:04:29.192 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.192 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.192 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838471 s, 125 MB/s 2024-04-25T09:04:29.202 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.202 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.202 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754071 s, 139 MB/s 2024-04-25T09:04:29.214 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.214 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.214 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00964761 s, 109 MB/s 2024-04-25T09:04:29.224 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.224 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.224 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752808 s, 139 MB/s 2024-04-25T09:04:29.234 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.234 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.234 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00781736 s, 134 MB/s 2024-04-25T09:04:29.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:29.235 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:04:29.235 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:04:29.236 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T09:04:29.236 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:04:29.236 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:04:29.236 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:04:29.236 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:04:29.304 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.304 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.304 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00847189 s, 124 MB/s 2024-04-25T09:04:29.314 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.314 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.314 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751512 s, 140 MB/s 2024-04-25T09:04:29.324 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.324 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.324 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00757028 s, 139 MB/s 2024-04-25T09:04:29.333 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.333 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.334 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744804 s, 141 MB/s 2024-04-25T09:04:29.343 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.343 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.343 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726427 s, 144 MB/s 2024-04-25T09:04:29.352 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.353 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.353 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753383 s, 139 MB/s 2024-04-25T09:04:29.363 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.363 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.363 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774094 s, 135 MB/s 2024-04-25T09:04:29.373 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.373 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.373 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00776099 s, 135 MB/s 2024-04-25T09:04:29.382 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.382 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.382 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711576 s, 147 MB/s 2024-04-25T09:04:29.392 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.392 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.392 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717714 s, 146 MB/s 2024-04-25T09:04:29.401 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.402 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.402 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754699 s, 139 MB/s 2024-04-25T09:04:29.411 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.411 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.411 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705144 s, 149 MB/s 2024-04-25T09:04:29.422 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.422 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.422 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00874242 s, 120 MB/s 2024-04-25T09:04:29.432 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.432 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.432 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734616 s, 143 MB/s 2024-04-25T09:04:29.442 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.442 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.442 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082252 s, 127 MB/s 2024-04-25T09:04:29.453 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.453 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.453 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.008501 s, 123 MB/s 2024-04-25T09:04:29.463 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.463 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.463 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732564 s, 143 MB/s 2024-04-25T09:04:29.472 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.472 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.473 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690386 s, 152 MB/s 2024-04-25T09:04:29.483 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.483 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.483 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00826018 s, 127 MB/s 2024-04-25T09:04:29.494 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.494 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.494 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00854497 s, 123 MB/s 2024-04-25T09:04:29.504 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.504 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.504 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749598 s, 140 MB/s 2024-04-25T09:04:29.514 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.514 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.515 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00815625 s, 129 MB/s 2024-04-25T09:04:29.524 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.524 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.524 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718925 s, 146 MB/s 2024-04-25T09:04:29.534 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.534 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.534 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762955 s, 137 MB/s 2024-04-25T09:04:29.544 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.544 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.544 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00792655 s, 132 MB/s 2024-04-25T09:04:29.554 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.554 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.554 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756196 s, 139 MB/s 2024-04-25T09:04:29.564 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.564 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.564 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821099 s, 128 MB/s 2024-04-25T09:04:29.574 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.574 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.574 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720245 s, 146 MB/s 2024-04-25T09:04:29.584 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.584 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.584 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00789176 s, 133 MB/s 2024-04-25T09:04:29.594 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.594 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.594 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825593 s, 127 MB/s 2024-04-25T09:04:29.605 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.605 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.605 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820894 s, 128 MB/s 2024-04-25T09:04:29.614 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.614 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.614 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700229 s, 150 MB/s 2024-04-25T09:04:29.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:29.615 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:04:29.615 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:04:29.616 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T09:04:29.616 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:04:29.616 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:04:29.616 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:04:29.616 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:04:29.682 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.683 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.683 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704857 s, 149 MB/s 2024-04-25T09:04:29.692 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.692 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.692 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732151 s, 143 MB/s 2024-04-25T09:04:29.702 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.702 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.702 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725311 s, 145 MB/s 2024-04-25T09:04:29.711 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.711 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.711 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726011 s, 144 MB/s 2024-04-25T09:04:29.721 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.721 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.721 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714841 s, 147 MB/s 2024-04-25T09:04:29.730 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.730 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.731 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727646 s, 144 MB/s 2024-04-25T09:04:29.740 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.740 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.740 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767478 s, 137 MB/s 2024-04-25T09:04:29.750 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.750 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.750 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752576 s, 139 MB/s 2024-04-25T09:04:29.760 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.760 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.760 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770173 s, 136 MB/s 2024-04-25T09:04:29.771 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.771 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.771 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831932 s, 126 MB/s 2024-04-25T09:04:29.780 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.780 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.780 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734253 s, 143 MB/s 2024-04-25T09:04:29.790 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.790 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.790 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735925 s, 142 MB/s 2024-04-25T09:04:29.799 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.799 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.799 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070976 s, 148 MB/s 2024-04-25T09:04:29.809 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.809 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.809 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741876 s, 141 MB/s 2024-04-25T09:04:29.819 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.820 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.820 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861536 s, 122 MB/s 2024-04-25T09:04:29.829 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.829 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.829 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723724 s, 145 MB/s 2024-04-25T09:04:29.838 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.838 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.839 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721417 s, 145 MB/s 2024-04-25T09:04:29.849 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.849 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.849 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836122 s, 125 MB/s 2024-04-25T09:04:29.859 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.859 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.859 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727927 s, 144 MB/s 2024-04-25T09:04:29.867 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.867 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.867 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660305 s, 159 MB/s 2024-04-25T09:04:29.878 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.878 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.878 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829487 s, 126 MB/s 2024-04-25T09:04:29.887 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.887 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.887 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673497 s, 156 MB/s 2024-04-25T09:04:29.898 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.898 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.898 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827959 s, 127 MB/s 2024-04-25T09:04:29.907 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.907 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.908 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729116 s, 144 MB/s 2024-04-25T09:04:29.917 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.917 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.917 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723753 s, 145 MB/s 2024-04-25T09:04:29.928 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.928 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.928 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00883994 s, 119 MB/s 2024-04-25T09:04:29.938 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.938 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.938 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726897 s, 144 MB/s 2024-04-25T09:04:29.947 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.947 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.947 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00658412 s, 159 MB/s 2024-04-25T09:04:29.957 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.957 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.957 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00810761 s, 129 MB/s 2024-04-25T09:04:29.967 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.967 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.968 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739993 s, 142 MB/s 2024-04-25T09:04:29.977 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.977 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.977 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745486 s, 141 MB/s 2024-04-25T09:04:29.987 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:29.987 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:29.988 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824528 s, 127 MB/s 2024-04-25T09:04:29.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:29.989 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:04:29.989 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:04:29.989 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T09:04:29.989 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:04:29.989 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:04:29.989 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:04:29.989 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:04:30.058 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.058 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.059 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00849895 s, 123 MB/s 2024-04-25T09:04:30.069 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.070 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.070 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00891319 s, 118 MB/s 2024-04-25T09:04:30.080 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.080 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.080 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763311 s, 137 MB/s 2024-04-25T09:04:30.090 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.090 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.090 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746445 s, 140 MB/s 2024-04-25T09:04:30.099 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.099 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.099 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727396 s, 144 MB/s 2024-04-25T09:04:30.109 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.109 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.109 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740152 s, 142 MB/s 2024-04-25T09:04:30.119 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.119 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.119 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762859 s, 137 MB/s 2024-04-25T09:04:30.128 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.129 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.129 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744264 s, 141 MB/s 2024-04-25T09:04:30.139 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.139 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.139 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075698 s, 139 MB/s 2024-04-25T09:04:30.147 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.148 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.148 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672836 s, 156 MB/s 2024-04-25T09:04:30.157 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.157 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.158 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754583 s, 139 MB/s 2024-04-25T09:04:30.168 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.168 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.169 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861968 s, 122 MB/s 2024-04-25T09:04:30.177 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.177 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.177 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681424 s, 154 MB/s 2024-04-25T09:04:30.188 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.188 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.188 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076593 s, 137 MB/s 2024-04-25T09:04:30.198 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.198 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.198 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770132 s, 136 MB/s 2024-04-25T09:04:30.209 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.209 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.209 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00905669 s, 116 MB/s 2024-04-25T09:04:30.234 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.234 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.235 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00768452 s, 136 MB/s 2024-04-25T09:04:30.244 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.295 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.296 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697757 s, 150 MB/s 2024-04-25T09:04:30.296 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.296 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.296 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814863 s, 129 MB/s 2024-04-25T09:04:30.297 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.297 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.297 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075849 s, 138 MB/s 2024-04-25T09:04:30.297 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.297 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.297 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073121 s, 143 MB/s 2024-04-25T09:04:30.297 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.297 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.297 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713743 s, 147 MB/s 2024-04-25T09:04:30.297 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.297 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.297 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730584 s, 144 MB/s 2024-04-25T09:04:30.303 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.303 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.303 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830521 s, 126 MB/s 2024-04-25T09:04:30.312 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.312 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.312 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726239 s, 144 MB/s 2024-04-25T09:04:30.322 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.323 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.323 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747116 s, 140 MB/s 2024-04-25T09:04:30.332 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.332 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.332 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755349 s, 139 MB/s 2024-04-25T09:04:30.341 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.341 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.341 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669518 s, 157 MB/s 2024-04-25T09:04:30.351 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.351 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.351 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740288 s, 142 MB/s 2024-04-25T09:04:30.360 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.360 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.360 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702155 s, 149 MB/s 2024-04-25T09:04:30.370 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.370 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.370 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729908 s, 144 MB/s 2024-04-25T09:04:30.379 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.379 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.379 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718035 s, 146 MB/s 2024-04-25T09:04:30.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:30.380 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:04:30.380 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:04:30.381 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T09:04:30.381 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:04:30.381 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:04:30.381 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:04:30.381 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:04:30.448 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.449 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.449 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772274 s, 136 MB/s 2024-04-25T09:04:30.459 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.459 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.459 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811959 s, 129 MB/s 2024-04-25T09:04:30.469 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.469 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.469 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00852554 s, 123 MB/s 2024-04-25T09:04:30.478 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.478 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.479 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680634 s, 154 MB/s 2024-04-25T09:04:30.488 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.488 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.488 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748824 s, 140 MB/s 2024-04-25T09:04:30.498 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.498 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.498 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00768091 s, 137 MB/s 2024-04-25T09:04:30.508 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.508 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.508 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737072 s, 142 MB/s 2024-04-25T09:04:30.517 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.517 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.518 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075535 s, 139 MB/s 2024-04-25T09:04:30.528 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.528 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.529 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00852653 s, 123 MB/s 2024-04-25T09:04:30.538 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.538 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.538 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072378 s, 145 MB/s 2024-04-25T09:04:30.548 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.548 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.548 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736899 s, 142 MB/s 2024-04-25T09:04:30.558 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.558 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.558 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00850175 s, 123 MB/s 2024-04-25T09:04:30.568 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.568 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.568 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699387 s, 150 MB/s 2024-04-25T09:04:30.578 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.578 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.578 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761761 s, 138 MB/s 2024-04-25T09:04:30.587 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.588 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.588 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735496 s, 143 MB/s 2024-04-25T09:04:30.597 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.597 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.597 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694676 s, 151 MB/s 2024-04-25T09:04:30.606 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.606 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.607 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721063 s, 145 MB/s 2024-04-25T09:04:30.616 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.616 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.616 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007036 s, 149 MB/s 2024-04-25T09:04:30.626 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.626 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.626 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717661 s, 146 MB/s 2024-04-25T09:04:30.635 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.635 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.635 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694797 s, 151 MB/s 2024-04-25T09:04:30.644 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.644 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.644 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743308 s, 141 MB/s 2024-04-25T09:04:30.654 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.654 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.654 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724877 s, 145 MB/s 2024-04-25T09:04:30.664 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.664 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.664 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747637 s, 140 MB/s 2024-04-25T09:04:30.673 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.673 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.673 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742163 s, 141 MB/s 2024-04-25T09:04:30.683 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.683 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.683 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738886 s, 142 MB/s 2024-04-25T09:04:30.694 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.694 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.694 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00847386 s, 124 MB/s 2024-04-25T09:04:30.703 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.704 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.704 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699556 s, 150 MB/s 2024-04-25T09:04:30.713 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.713 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.713 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686221 s, 153 MB/s 2024-04-25T09:04:30.724 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.724 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.724 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00875268 s, 120 MB/s 2024-04-25T09:04:30.734 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.734 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.734 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767041 s, 137 MB/s 2024-04-25T09:04:30.743 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.743 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.743 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728957 s, 144 MB/s 2024-04-25T09:04:30.754 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.754 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.755 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00852452 s, 123 MB/s 2024-04-25T09:04:30.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:30.756 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:04:30.756 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:04:30.756 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T09:04:30.756 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:04:30.756 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:04:30.756 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:04:30.756 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:04:30.825 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.825 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.825 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753978 s, 139 MB/s 2024-04-25T09:04:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.834 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735724 s, 143 MB/s 2024-04-25T09:04:30.844 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.844 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.844 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074348 s, 141 MB/s 2024-04-25T09:04:30.854 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.854 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.854 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816447 s, 128 MB/s 2024-04-25T09:04:30.864 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.864 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.864 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732374 s, 143 MB/s 2024-04-25T09:04:30.875 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.875 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.875 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00887537 s, 118 MB/s 2024-04-25T09:04:30.885 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.886 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.886 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00813859 s, 129 MB/s 2024-04-25T09:04:30.896 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.897 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.897 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835223 s, 126 MB/s 2024-04-25T09:04:30.905 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.906 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.906 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665978 s, 157 MB/s 2024-04-25T09:04:30.915 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.916 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.916 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742702 s, 141 MB/s 2024-04-25T09:04:30.925 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.925 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.925 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730263 s, 144 MB/s 2024-04-25T09:04:30.936 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.936 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.936 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00852133 s, 123 MB/s 2024-04-25T09:04:30.946 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.946 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.946 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733906 s, 143 MB/s 2024-04-25T09:04:30.955 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.955 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.955 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00675884 s, 155 MB/s 2024-04-25T09:04:30.965 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.965 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.965 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763463 s, 137 MB/s 2024-04-25T09:04:30.976 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.976 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.976 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084913 s, 123 MB/s 2024-04-25T09:04:30.985 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.985 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.985 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726439 s, 144 MB/s 2024-04-25T09:04:30.996 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:30.996 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:30.996 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00862569 s, 122 MB/s 2024-04-25T09:04:31.006 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.006 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.006 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747216 s, 140 MB/s 2024-04-25T09:04:31.016 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.016 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.016 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752462 s, 139 MB/s 2024-04-25T09:04:31.025 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.025 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.025 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691595 s, 152 MB/s 2024-04-25T09:04:31.035 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.035 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.035 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749593 s, 140 MB/s 2024-04-25T09:04:31.045 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.045 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.045 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756624 s, 139 MB/s 2024-04-25T09:04:31.056 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.056 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.056 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00877702 s, 119 MB/s 2024-04-25T09:04:31.065 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.065 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.065 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070652 s, 148 MB/s 2024-04-25T09:04:31.075 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.075 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.076 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00788369 s, 133 MB/s 2024-04-25T09:04:31.085 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.085 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.085 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763195 s, 137 MB/s 2024-04-25T09:04:31.094 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.094 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.094 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669682 s, 157 MB/s 2024-04-25T09:04:31.103 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.103 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.103 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693802 s, 151 MB/s 2024-04-25T09:04:31.113 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.113 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.113 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00768144 s, 137 MB/s 2024-04-25T09:04:31.123 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.123 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.123 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739484 s, 142 MB/s 2024-04-25T09:04:31.133 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.133 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.133 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007425 s, 141 MB/s 2024-04-25T09:04:31.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:31.134 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:04:31.134 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:04:31.134 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T09:04:31.134 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:04:31.134 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:04:31.134 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:04:31.134 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:04:31.203 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.203 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.203 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748154 s, 140 MB/s 2024-04-25T09:04:31.212 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.212 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.213 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752163 s, 139 MB/s 2024-04-25T09:04:31.249 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.249 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.249 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758991 s, 138 MB/s 2024-04-25T09:04:31.259 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.259 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.259 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007517 s, 139 MB/s 2024-04-25T09:04:31.269 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.269 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.269 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748871 s, 140 MB/s 2024-04-25T09:04:31.280 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.280 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.280 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00813622 s, 129 MB/s 2024-04-25T09:04:31.289 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.289 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.289 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748392 s, 140 MB/s 2024-04-25T09:04:31.299 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.299 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.299 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758512 s, 138 MB/s 2024-04-25T09:04:31.309 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.309 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.309 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732648 s, 143 MB/s 2024-04-25T09:04:31.320 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.320 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.320 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833446 s, 126 MB/s 2024-04-25T09:04:31.330 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.330 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.331 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00854138 s, 123 MB/s 2024-04-25T09:04:31.340 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.340 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.340 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073049 s, 144 MB/s 2024-04-25T09:04:31.351 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.351 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.351 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00828559 s, 127 MB/s 2024-04-25T09:04:31.362 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.362 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.362 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00845195 s, 124 MB/s 2024-04-25T09:04:31.371 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.372 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.372 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751965 s, 139 MB/s 2024-04-25T09:04:31.381 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.381 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.381 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743887 s, 141 MB/s 2024-04-25T09:04:31.391 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.391 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.391 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749058 s, 140 MB/s 2024-04-25T09:04:31.400 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.400 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.400 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692734 s, 151 MB/s 2024-04-25T09:04:31.410 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.410 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.410 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746185 s, 141 MB/s 2024-04-25T09:04:31.421 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.421 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.422 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00866848 s, 121 MB/s 2024-04-25T09:04:31.431 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.431 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.431 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730024 s, 144 MB/s 2024-04-25T09:04:31.440 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.440 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.440 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690834 s, 152 MB/s 2024-04-25T09:04:31.451 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.451 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.451 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00843333 s, 124 MB/s 2024-04-25T09:04:31.461 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.461 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.461 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075255 s, 139 MB/s 2024-04-25T09:04:31.471 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.471 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.471 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738133 s, 142 MB/s 2024-04-25T09:04:31.480 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.480 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.480 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729256 s, 144 MB/s 2024-04-25T09:04:31.490 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.490 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.490 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076246 s, 138 MB/s 2024-04-25T09:04:31.500 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.500 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.500 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720419 s, 146 MB/s 2024-04-25T09:04:31.511 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.511 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.511 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00837994 s, 125 MB/s 2024-04-25T09:04:31.520 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.520 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.521 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744004 s, 141 MB/s 2024-04-25T09:04:31.531 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.531 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.532 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00872814 s, 120 MB/s 2024-04-25T09:04:31.541 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.541 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.541 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749083 s, 140 MB/s 2024-04-25T09:04:31.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:31.542 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:04:31.543 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:04:31.543 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T09:04:31.543 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:04:31.543 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:04:31.543 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:04:31.543 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:04:31.611 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.611 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.612 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00847264 s, 124 MB/s 2024-04-25T09:04:31.622 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.622 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.622 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820714 s, 128 MB/s 2024-04-25T09:04:31.632 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.632 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.632 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00758036 s, 138 MB/s 2024-04-25T09:04:31.641 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.641 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.641 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679825 s, 154 MB/s 2024-04-25T09:04:31.650 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.650 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.650 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702689 s, 149 MB/s 2024-04-25T09:04:31.660 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.660 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.660 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699054 s, 150 MB/s 2024-04-25T09:04:31.669 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.669 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.669 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742889 s, 141 MB/s 2024-04-25T09:04:31.679 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.679 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.679 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728438 s, 144 MB/s 2024-04-25T09:04:31.688 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.688 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.688 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717009 s, 146 MB/s 2024-04-25T09:04:31.697 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.697 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.698 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713985 s, 147 MB/s 2024-04-25T09:04:31.707 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.707 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.707 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700496 s, 150 MB/s 2024-04-25T09:04:31.716 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.716 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.716 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070893 s, 148 MB/s 2024-04-25T09:04:31.725 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.726 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.726 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712072 s, 147 MB/s 2024-04-25T09:04:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.735 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723078 s, 145 MB/s 2024-04-25T09:04:31.744 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.745 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.745 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725674 s, 144 MB/s 2024-04-25T09:04:31.754 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.754 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.754 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748748 s, 140 MB/s 2024-04-25T09:04:31.764 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.764 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.764 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071012 s, 148 MB/s 2024-04-25T09:04:31.773 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.774 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.774 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745625 s, 141 MB/s 2024-04-25T09:04:31.783 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.783 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.783 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073007 s, 144 MB/s 2024-04-25T09:04:31.792 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.793 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.793 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736478 s, 142 MB/s 2024-04-25T09:04:31.802 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.802 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.802 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00768128 s, 137 MB/s 2024-04-25T09:04:31.812 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.812 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.813 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0078994 s, 133 MB/s 2024-04-25T09:04:31.823 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.823 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.823 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842786 s, 124 MB/s 2024-04-25T09:04:31.832 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.832 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.832 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.006865 s, 153 MB/s 2024-04-25T09:04:31.842 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.842 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.842 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725544 s, 145 MB/s 2024-04-25T09:04:31.851 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.851 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.851 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00686698 s, 153 MB/s 2024-04-25T09:04:31.862 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.862 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.862 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00886155 s, 118 MB/s 2024-04-25T09:04:31.872 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.872 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.872 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760106 s, 138 MB/s 2024-04-25T09:04:31.883 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.883 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.883 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851461 s, 123 MB/s 2024-04-25T09:04:31.892 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.893 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.893 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731931 s, 143 MB/s 2024-04-25T09:04:31.902 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.902 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.902 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690386 s, 152 MB/s 2024-04-25T09:04:31.911 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.911 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.911 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697515 s, 150 MB/s 2024-04-25T09:04:31.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:31.912 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:04:31.912 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:04:31.912 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T09:04:31.912 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:04:31.912 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:04:31.912 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:04:31.912 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:04:31.981 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.981 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.981 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00786898 s, 133 MB/s 2024-04-25T09:04:31.990 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.990 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.990 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733036 s, 143 MB/s 2024-04-25T09:04:31.999 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:31.999 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:31.999 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679909 s, 154 MB/s 2024-04-25T09:04:32.008 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.008 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.009 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679761 s, 154 MB/s 2024-04-25T09:04:32.018 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.018 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.018 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729332 s, 144 MB/s 2024-04-25T09:04:32.029 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.029 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.029 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836535 s, 125 MB/s 2024-04-25T09:04:32.038 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.038 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.038 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073908 s, 142 MB/s 2024-04-25T09:04:32.048 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.048 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.048 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694725 s, 151 MB/s 2024-04-25T09:04:32.057 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.058 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.058 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714925 s, 147 MB/s 2024-04-25T09:04:32.068 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.068 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.068 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0085339 s, 123 MB/s 2024-04-25T09:04:32.078 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.078 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.078 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717805 s, 146 MB/s 2024-04-25T09:04:32.087 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.087 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.088 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716204 s, 146 MB/s 2024-04-25T09:04:32.097 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.097 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.097 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00727574 s, 144 MB/s 2024-04-25T09:04:32.107 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.107 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.107 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729856 s, 144 MB/s 2024-04-25T09:04:32.117 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.118 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.118 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00860342 s, 122 MB/s 2024-04-25T09:04:32.127 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.128 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.128 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762872 s, 137 MB/s 2024-04-25T09:04:32.136 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.136 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.136 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00685772 s, 153 MB/s 2024-04-25T09:04:32.146 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.146 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.146 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724614 s, 145 MB/s 2024-04-25T09:04:32.156 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.156 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.156 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0072285 s, 145 MB/s 2024-04-25T09:04:32.166 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.166 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.166 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0083776 s, 125 MB/s 2024-04-25T09:04:32.176 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.176 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.176 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740302 s, 142 MB/s 2024-04-25T09:04:32.186 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.186 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.187 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750124 s, 140 MB/s 2024-04-25T09:04:32.197 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.197 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.197 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820434 s, 128 MB/s 2024-04-25T09:04:32.208 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.208 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.208 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831553 s, 126 MB/s 2024-04-25T09:04:32.217 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.218 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.218 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746026 s, 141 MB/s 2024-04-25T09:04:32.229 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.229 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.229 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729921 s, 144 MB/s 2024-04-25T09:04:32.256 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.256 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.256 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851668 s, 123 MB/s 2024-04-25T09:04:32.266 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.266 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.266 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736181 s, 142 MB/s 2024-04-25T09:04:32.277 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.277 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.277 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824474 s, 127 MB/s 2024-04-25T09:04:32.287 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.288 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.288 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861186 s, 122 MB/s 2024-04-25T09:04:32.297 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.297 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.297 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750703 s, 140 MB/s 2024-04-25T09:04:32.307 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.307 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.307 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755871 s, 139 MB/s 2024-04-25T09:04:32.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:32.308 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:04:32.308 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:04:32.308 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T09:04:32.309 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:04:32.309 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:04:32.309 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:04:32.309 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:04:32.376 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.376 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.376 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722749 s, 145 MB/s 2024-04-25T09:04:32.385 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.385 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.385 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679802 s, 154 MB/s 2024-04-25T09:04:32.394 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.394 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.395 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740938 s, 142 MB/s 2024-04-25T09:04:32.405 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.405 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.405 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819895 s, 128 MB/s 2024-04-25T09:04:32.415 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.415 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.415 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00781839 s, 134 MB/s 2024-04-25T09:04:32.426 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.426 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.426 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816223 s, 128 MB/s 2024-04-25T09:04:32.435 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.435 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.435 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720597 s, 146 MB/s 2024-04-25T09:04:32.445 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.446 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.446 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824145 s, 127 MB/s 2024-04-25T09:04:32.456 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.457 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.457 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834384 s, 126 MB/s 2024-04-25T09:04:32.466 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.466 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.466 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737563 s, 142 MB/s 2024-04-25T09:04:32.476 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.476 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.476 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732875 s, 143 MB/s 2024-04-25T09:04:32.485 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.486 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.486 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721619 s, 145 MB/s 2024-04-25T09:04:32.495 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.495 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.495 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708415 s, 148 MB/s 2024-04-25T09:04:32.504 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.504 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.504 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735084 s, 143 MB/s 2024-04-25T09:04:32.515 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.515 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.515 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00843896 s, 124 MB/s 2024-04-25T09:04:32.524 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.525 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.525 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717552 s, 146 MB/s 2024-04-25T09:04:32.534 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.534 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.534 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752168 s, 139 MB/s 2024-04-25T09:04:32.544 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.544 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.544 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684507 s, 153 MB/s 2024-04-25T09:04:32.553 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.553 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.553 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709015 s, 148 MB/s 2024-04-25T09:04:32.563 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.563 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.563 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730399 s, 144 MB/s 2024-04-25T09:04:32.572 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.572 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.572 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00669328 s, 157 MB/s 2024-04-25T09:04:32.583 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.583 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.583 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829956 s, 126 MB/s 2024-04-25T09:04:32.592 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.593 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.593 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714952 s, 147 MB/s 2024-04-25T09:04:32.602 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.602 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.602 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732315 s, 143 MB/s 2024-04-25T09:04:32.612 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.612 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.612 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761996 s, 138 MB/s 2024-04-25T09:04:32.623 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.623 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.623 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851885 s, 123 MB/s 2024-04-25T09:04:32.633 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.633 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.633 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737971 s, 142 MB/s 2024-04-25T09:04:32.643 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.643 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.643 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753509 s, 139 MB/s 2024-04-25T09:04:32.652 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.653 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.653 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075854 s, 138 MB/s 2024-04-25T09:04:32.662 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.662 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.662 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720219 s, 146 MB/s 2024-04-25T09:04:32.672 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.673 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.673 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00843044 s, 124 MB/s 2024-04-25T09:04:32.682 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.682 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.682 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074574 s, 141 MB/s 2024-04-25T09:04:32.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:32.684 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:04:32.684 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:04:32.684 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T09:04:32.684 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:04:32.684 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:04:32.684 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:04:32.684 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:04:32.752 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.753 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.753 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074413 s, 141 MB/s 2024-04-25T09:04:32.763 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.763 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.763 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842427 s, 124 MB/s 2024-04-25T09:04:32.773 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.773 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.773 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746879 s, 140 MB/s 2024-04-25T09:04:32.783 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.783 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.783 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753519 s, 139 MB/s 2024-04-25T09:04:32.794 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.794 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.794 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841507 s, 125 MB/s 2024-04-25T09:04:32.804 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.804 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.805 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820133 s, 128 MB/s 2024-04-25T09:04:32.814 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.814 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.814 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692276 s, 151 MB/s 2024-04-25T09:04:32.823 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.823 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.824 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007097 s, 148 MB/s 2024-04-25T09:04:32.833 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.833 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.833 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752777 s, 139 MB/s 2024-04-25T09:04:32.843 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.843 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.843 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733256 s, 143 MB/s 2024-04-25T09:04:32.852 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.853 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.853 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700908 s, 150 MB/s 2024-04-25T09:04:32.862 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.862 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.862 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00724428 s, 145 MB/s 2024-04-25T09:04:32.871 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.871 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.871 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00699931 s, 150 MB/s 2024-04-25T09:04:32.880 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.880 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.880 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684631 s, 153 MB/s 2024-04-25T09:04:32.891 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.891 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.891 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833861 s, 126 MB/s 2024-04-25T09:04:32.901 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.901 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.901 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751194 s, 140 MB/s 2024-04-25T09:04:32.911 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.911 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.911 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835048 s, 126 MB/s 2024-04-25T09:04:32.921 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.921 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.921 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0068118 s, 154 MB/s 2024-04-25T09:04:32.931 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.931 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.931 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076411 s, 137 MB/s 2024-04-25T09:04:32.940 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.940 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.940 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00704002 s, 149 MB/s 2024-04-25T09:04:32.951 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.951 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.951 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829109 s, 126 MB/s 2024-04-25T09:04:32.961 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.961 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.961 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00793916 s, 132 MB/s 2024-04-25T09:04:32.970 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.970 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.970 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00684336 s, 153 MB/s 2024-04-25T09:04:32.981 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.981 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.981 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00835229 s, 126 MB/s 2024-04-25T09:04:32.991 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:32.991 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:32.991 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075385 s, 139 MB/s 2024-04-25T09:04:33.002 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.002 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.002 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0082347 s, 127 MB/s 2024-04-25T09:04:33.012 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.012 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.012 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759986 s, 138 MB/s 2024-04-25T09:04:33.022 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.023 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.023 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00853043 s, 123 MB/s 2024-04-25T09:04:33.033 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.033 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.033 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00781354 s, 134 MB/s 2024-04-25T09:04:33.043 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.043 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.043 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00825332 s, 127 MB/s 2024-04-25T09:04:33.054 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.054 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.054 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824744 s, 127 MB/s 2024-04-25T09:04:33.065 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.065 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.065 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829187 s, 126 MB/s 2024-04-25T09:04:33.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:33.066 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:04:33.066 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:04:33.066 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T09:04:33.066 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:04:33.066 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:04:33.066 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:04:33.066 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:04:33.134 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.134 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.134 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734892 s, 143 MB/s 2024-04-25T09:04:33.144 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.145 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.145 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814639 s, 129 MB/s 2024-04-25T09:04:33.154 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.154 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.154 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737715 s, 142 MB/s 2024-04-25T09:04:33.163 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.164 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.164 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700362 s, 150 MB/s 2024-04-25T09:04:33.174 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.174 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.174 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774483 s, 135 MB/s 2024-04-25T09:04:33.183 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.183 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.183 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007266 s, 144 MB/s 2024-04-25T09:04:33.194 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.194 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.194 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0081857 s, 128 MB/s 2024-04-25T09:04:33.203 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.203 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.203 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00735302 s, 143 MB/s 2024-04-25T09:04:33.213 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.213 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.213 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074614 s, 141 MB/s 2024-04-25T09:04:33.225 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.225 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.225 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00966993 s, 108 MB/s 2024-04-25T09:04:33.261 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.261 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.261 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00844031 s, 124 MB/s 2024-04-25T09:04:33.270 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.270 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.271 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761514 s, 138 MB/s 2024-04-25T09:04:33.280 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.280 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.280 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007252 s, 145 MB/s 2024-04-25T09:04:33.289 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.289 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.289 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00692783 s, 151 MB/s 2024-04-25T09:04:33.298 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.299 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.299 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726706 s, 144 MB/s 2024-04-25T09:04:33.309 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.309 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.309 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00809712 s, 129 MB/s 2024-04-25T09:04:33.318 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.319 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.319 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752184 s, 139 MB/s 2024-04-25T09:04:33.328 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.328 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.328 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736671 s, 142 MB/s 2024-04-25T09:04:33.337 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.338 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.338 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740185 s, 142 MB/s 2024-04-25T09:04:33.348 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.348 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.348 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076743 s, 137 MB/s 2024-04-25T09:04:33.358 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.358 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.358 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830646 s, 126 MB/s 2024-04-25T09:04:33.368 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.368 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.368 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00688185 s, 152 MB/s 2024-04-25T09:04:33.377 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.377 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.377 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00672723 s, 156 MB/s 2024-04-25T09:04:33.387 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.387 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.387 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734902 s, 143 MB/s 2024-04-25T09:04:33.396 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.396 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.396 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691272 s, 152 MB/s 2024-04-25T09:04:33.407 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.407 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.407 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00846589 s, 124 MB/s 2024-04-25T09:04:33.417 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.417 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.417 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733755 s, 143 MB/s 2024-04-25T09:04:33.426 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.426 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.427 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746715 s, 140 MB/s 2024-04-25T09:04:33.437 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.437 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.437 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0083316 s, 126 MB/s 2024-04-25T09:04:33.447 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.447 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.447 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00783227 s, 134 MB/s 2024-04-25T09:04:33.458 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.458 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.458 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841581 s, 125 MB/s 2024-04-25T09:04:33.468 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.468 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.468 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716822 s, 146 MB/s 2024-04-25T09:04:33.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:33.469 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:04:33.469 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:04:33.469 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T09:04:33.469 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:04:33.469 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:04:33.469 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:04:33.469 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:04:33.537 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.538 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.538 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748008 s, 140 MB/s 2024-04-25T09:04:33.548 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.548 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.548 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00816179 s, 128 MB/s 2024-04-25T09:04:33.557 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.557 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.557 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711021 s, 147 MB/s 2024-04-25T09:04:33.567 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.567 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.567 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756091 s, 139 MB/s 2024-04-25T09:04:33.576 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.576 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.576 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00676769 s, 155 MB/s 2024-04-25T09:04:33.586 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.586 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.587 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771523 s, 136 MB/s 2024-04-25T09:04:33.597 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.597 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.597 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00851825 s, 123 MB/s 2024-04-25T09:04:33.608 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.608 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.608 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836506 s, 125 MB/s 2024-04-25T09:04:33.618 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.618 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.619 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00841999 s, 125 MB/s 2024-04-25T09:04:33.628 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.628 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.628 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728293 s, 144 MB/s 2024-04-25T09:04:33.639 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.639 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.639 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00837676 s, 125 MB/s 2024-04-25T09:04:33.648 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.649 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.649 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719444 s, 146 MB/s 2024-04-25T09:04:33.658 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.658 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.658 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744463 s, 141 MB/s 2024-04-25T09:04:33.667 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.668 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.668 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728463 s, 144 MB/s 2024-04-25T09:04:33.677 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.677 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.677 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750083 s, 140 MB/s 2024-04-25T09:04:33.687 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.687 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.687 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754841 s, 139 MB/s 2024-04-25T09:04:33.698 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.698 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.698 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840623 s, 125 MB/s 2024-04-25T09:04:33.708 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.709 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.709 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00784356 s, 134 MB/s 2024-04-25T09:04:33.719 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.719 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.719 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00843466 s, 124 MB/s 2024-04-25T09:04:33.730 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.730 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.730 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00874483 s, 120 MB/s 2024-04-25T09:04:33.740 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.740 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.740 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736574 s, 142 MB/s 2024-04-25T09:04:33.749 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.750 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.750 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700959 s, 150 MB/s 2024-04-25T09:04:33.760 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.760 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.760 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00768969 s, 136 MB/s 2024-04-25T09:04:33.769 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.770 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.770 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00770052 s, 136 MB/s 2024-04-25T09:04:33.779 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.780 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.780 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763807 s, 137 MB/s 2024-04-25T09:04:33.790 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.790 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.790 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772537 s, 136 MB/s 2024-04-25T09:04:33.799 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.799 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.799 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00712014 s, 147 MB/s 2024-04-25T09:04:33.809 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.809 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.809 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746294 s, 141 MB/s 2024-04-25T09:04:33.818 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.818 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.818 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00694703 s, 151 MB/s 2024-04-25T09:04:33.827 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.827 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.827 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00713788 s, 147 MB/s 2024-04-25T09:04:33.838 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.838 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.838 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824794 s, 127 MB/s 2024-04-25T09:04:33.847 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.847 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.847 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073554 s, 143 MB/s 2024-04-25T09:04:33.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:33.848 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:04:33.849 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:04:33.849 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T09:04:33.849 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:04:33.849 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:04:33.849 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:04:33.849 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:04:33.916 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.916 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.917 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771639 s, 136 MB/s 2024-04-25T09:04:33.926 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.926 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.926 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745252 s, 141 MB/s 2024-04-25T09:04:33.934 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.935 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.935 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00660827 s, 159 MB/s 2024-04-25T09:04:33.944 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.944 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.944 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716121 s, 146 MB/s 2024-04-25T09:04:33.954 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.954 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.954 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746658 s, 140 MB/s 2024-04-25T09:04:33.964 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.964 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.964 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00811892 s, 129 MB/s 2024-04-25T09:04:33.974 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.974 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.974 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076727 s, 137 MB/s 2024-04-25T09:04:33.984 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.984 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.984 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722692 s, 145 MB/s 2024-04-25T09:04:33.993 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:33.993 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:33.993 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714359 s, 147 MB/s 2024-04-25T09:04:34.002 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.003 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.003 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731197 s, 143 MB/s 2024-04-25T09:04:34.012 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.012 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.012 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729151 s, 144 MB/s 2024-04-25T09:04:34.022 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.022 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.022 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0076896 s, 136 MB/s 2024-04-25T09:04:34.032 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.032 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.032 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751573 s, 140 MB/s 2024-04-25T09:04:34.042 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.042 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.042 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00774264 s, 135 MB/s 2024-04-25T09:04:34.052 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.053 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.053 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763016 s, 137 MB/s 2024-04-25T09:04:34.062 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.062 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.062 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720844 s, 145 MB/s 2024-04-25T09:04:34.073 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.073 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.073 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820274 s, 128 MB/s 2024-04-25T09:04:34.084 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.084 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.084 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00885483 s, 118 MB/s 2024-04-25T09:04:34.095 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.095 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.095 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832311 s, 126 MB/s 2024-04-25T09:04:34.106 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.106 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.106 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00864038 s, 121 MB/s 2024-04-25T09:04:34.117 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.117 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.117 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00885549 s, 118 MB/s 2024-04-25T09:04:34.128 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.128 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.128 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827332 s, 127 MB/s 2024-04-25T09:04:34.138 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.138 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.138 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00759435 s, 138 MB/s 2024-04-25T09:04:34.148 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.148 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.148 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818426 s, 128 MB/s 2024-04-25T09:04:34.158 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.158 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.158 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00747637 s, 140 MB/s 2024-04-25T09:04:34.168 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.168 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.168 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00791523 s, 132 MB/s 2024-04-25T09:04:34.178 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.178 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.178 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075636 s, 139 MB/s 2024-04-25T09:04:34.189 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.189 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.189 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827709 s, 127 MB/s 2024-04-25T09:04:34.199 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.200 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.200 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838096 s, 125 MB/s 2024-04-25T09:04:34.210 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.210 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.210 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819614 s, 128 MB/s 2024-04-25T09:04:34.219 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.220 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.220 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702838 s, 149 MB/s 2024-04-25T09:04:34.236 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.236 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.236 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0142767 s, 73.4 MB/s 2024-04-25T09:04:34.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:34.237 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:04:34.238 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:04:34.238 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T09:04:34.238 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:04:34.238 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:04:34.238 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:04:34.238 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:04:34.308 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.308 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.308 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00879395 s, 119 MB/s 2024-04-25T09:04:34.317 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.317 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.317 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728927 s, 144 MB/s 2024-04-25T09:04:34.328 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.328 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.328 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00843212 s, 124 MB/s 2024-04-25T09:04:34.338 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.338 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.339 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00781844 s, 134 MB/s 2024-04-25T09:04:34.348 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.348 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.348 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708796 s, 148 MB/s 2024-04-25T09:04:34.358 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.358 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.358 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00763142 s, 137 MB/s 2024-04-25T09:04:34.369 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.369 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.369 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836273 s, 125 MB/s 2024-04-25T09:04:34.380 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.380 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.380 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00819021 s, 128 MB/s 2024-04-25T09:04:34.389 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.389 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.389 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739318 s, 142 MB/s 2024-04-25T09:04:34.400 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.400 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.400 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00818257 s, 128 MB/s 2024-04-25T09:04:34.410 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.410 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.410 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00831707 s, 126 MB/s 2024-04-25T09:04:34.420 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.420 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.420 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723361 s, 145 MB/s 2024-04-25T09:04:34.430 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.431 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.431 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822213 s, 128 MB/s 2024-04-25T09:04:34.441 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.441 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.442 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834422 s, 126 MB/s 2024-04-25T09:04:34.451 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.451 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.452 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756192 s, 139 MB/s 2024-04-25T09:04:34.463 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.463 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.463 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839048 s, 125 MB/s 2024-04-25T09:04:34.472 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.472 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.472 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721807 s, 145 MB/s 2024-04-25T09:04:34.481 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.481 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.481 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690712 s, 152 MB/s 2024-04-25T09:04:34.491 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.491 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.491 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00777727 s, 135 MB/s 2024-04-25T09:04:34.500 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.500 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.500 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00668351 s, 157 MB/s 2024-04-25T09:04:34.510 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.510 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.510 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744164 s, 141 MB/s 2024-04-25T09:04:34.520 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.520 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.520 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750238 s, 140 MB/s 2024-04-25T09:04:34.530 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.530 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.530 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00752328 s, 139 MB/s 2024-04-25T09:04:34.539 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.539 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.539 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723285 s, 145 MB/s 2024-04-25T09:04:34.549 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.549 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.549 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760819 s, 138 MB/s 2024-04-25T09:04:34.558 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.558 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.558 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678853 s, 154 MB/s 2024-04-25T09:04:34.568 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.568 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.568 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717889 s, 146 MB/s 2024-04-25T09:04:34.577 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.577 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.578 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741533 s, 141 MB/s 2024-04-25T09:04:34.588 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.588 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.588 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00837416 s, 125 MB/s 2024-04-25T09:04:34.598 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.598 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.598 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742337 s, 141 MB/s 2024-04-25T09:04:34.608 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.608 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.608 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746913 s, 140 MB/s 2024-04-25T09:04:34.618 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.618 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.618 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766849 s, 137 MB/s 2024-04-25T09:04:34.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:34.619 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:04:34.619 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:04:34.619 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T09:04:34.619 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:04:34.619 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:04:34.619 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:04:34.620 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:04:34.687 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.688 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.688 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00720999 s, 145 MB/s 2024-04-25T09:04:34.698 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.698 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.699 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839779 s, 125 MB/s 2024-04-25T09:04:34.708 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.708 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.708 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706303 s, 148 MB/s 2024-04-25T09:04:34.717 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.717 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.717 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00678653 s, 155 MB/s 2024-04-25T09:04:34.727 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.727 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.727 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00725976 s, 144 MB/s 2024-04-25T09:04:34.736 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.736 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.736 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708393 s, 148 MB/s 2024-04-25T09:04:34.746 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.746 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.746 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760982 s, 138 MB/s 2024-04-25T09:04:34.756 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.756 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.756 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749195 s, 140 MB/s 2024-04-25T09:04:34.765 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.765 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.765 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00731253 s, 143 MB/s 2024-04-25T09:04:34.775 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.775 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.775 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.007469 s, 140 MB/s 2024-04-25T09:04:34.786 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.786 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.786 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00796071 s, 132 MB/s 2024-04-25T09:04:34.795 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.795 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.795 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0071951 s, 146 MB/s 2024-04-25T09:04:34.806 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.806 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.806 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814564 s, 129 MB/s 2024-04-25T09:04:34.816 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.817 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.817 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0086265 s, 122 MB/s 2024-04-25T09:04:34.826 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.826 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.826 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708142 s, 148 MB/s 2024-04-25T09:04:34.837 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.837 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.837 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00839908 s, 125 MB/s 2024-04-25T09:04:34.846 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.846 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.846 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723374 s, 145 MB/s 2024-04-25T09:04:34.856 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.856 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.856 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00742895 s, 141 MB/s 2024-04-25T09:04:34.866 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.866 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.866 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732209 s, 143 MB/s 2024-04-25T09:04:34.875 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.875 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.876 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723079 s, 145 MB/s 2024-04-25T09:04:34.885 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.885 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.885 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732763 s, 143 MB/s 2024-04-25T09:04:34.894 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.895 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.895 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716209 s, 146 MB/s 2024-04-25T09:04:34.905 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.906 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.906 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00889869 s, 118 MB/s 2024-04-25T09:04:34.915 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.915 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.915 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00728509 s, 144 MB/s 2024-04-25T09:04:34.925 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.925 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.925 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751224 s, 140 MB/s 2024-04-25T09:04:34.934 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.935 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.935 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739066 s, 142 MB/s 2024-04-25T09:04:34.945 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.945 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.946 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814571 s, 129 MB/s 2024-04-25T09:04:34.956 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.956 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.956 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00833043 s, 126 MB/s 2024-04-25T09:04:34.966 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.966 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.966 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734172 s, 143 MB/s 2024-04-25T09:04:34.975 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.975 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.975 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708628 s, 148 MB/s 2024-04-25T09:04:34.985 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.985 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.985 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740231 s, 142 MB/s 2024-04-25T09:04:34.994 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:34.994 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:34.994 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718422 s, 146 MB/s 2024-04-25T09:04:34.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:34.996 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:04:34.996 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:04:34.996 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T09:04:34.996 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:04:34.996 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:04:34.996 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:04:34.996 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:04:35.064 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.064 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.064 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00697165 s, 150 MB/s 2024-04-25T09:04:35.074 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.074 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.075 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767737 s, 137 MB/s 2024-04-25T09:04:35.084 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.084 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.084 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0075875 s, 138 MB/s 2024-04-25T09:04:35.095 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.095 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.095 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00884866 s, 119 MB/s 2024-04-25T09:04:35.106 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.106 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.106 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0084381 s, 124 MB/s 2024-04-25T09:04:35.116 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.116 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.116 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754279 s, 139 MB/s 2024-04-25T09:04:35.125 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.125 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.125 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681184 s, 154 MB/s 2024-04-25T09:04:35.135 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.136 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.136 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00814265 s, 129 MB/s 2024-04-25T09:04:35.146 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.146 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.146 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00766525 s, 137 MB/s 2024-04-25T09:04:35.155 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.155 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.156 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749931 s, 140 MB/s 2024-04-25T09:04:35.165 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.165 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.166 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0073539 s, 143 MB/s 2024-04-25T09:04:35.176 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.176 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.176 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827963 s, 127 MB/s 2024-04-25T09:04:35.186 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.187 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.187 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00771536 s, 136 MB/s 2024-04-25T09:04:35.195 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.196 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.196 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00705979 s, 149 MB/s 2024-04-25T09:04:35.206 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.206 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.207 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00854406 s, 123 MB/s 2024-04-25T09:04:35.216 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.216 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.216 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746316 s, 141 MB/s 2024-04-25T09:04:35.226 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.226 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.226 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691766 s, 152 MB/s 2024-04-25T09:04:35.242 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.242 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.242 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0136355 s, 76.9 MB/s 2024-04-25T09:04:35.269 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.269 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.269 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751833 s, 139 MB/s 2024-04-25T09:04:35.280 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.280 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.280 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00838319 s, 125 MB/s 2024-04-25T09:04:35.290 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.290 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.290 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723902 s, 145 MB/s 2024-04-25T09:04:35.299 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.300 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.300 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00745663 s, 141 MB/s 2024-04-25T09:04:35.309 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.309 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.309 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706453 s, 148 MB/s 2024-04-25T09:04:35.319 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.319 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.319 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732982 s, 143 MB/s 2024-04-25T09:04:35.328 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.328 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.328 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070329 s, 149 MB/s 2024-04-25T09:04:35.337 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.338 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.338 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709046 s, 148 MB/s 2024-04-25T09:04:35.347 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.347 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.347 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709901 s, 148 MB/s 2024-04-25T09:04:35.357 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.357 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.357 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748401 s, 140 MB/s 2024-04-25T09:04:35.366 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.366 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.367 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00723414 s, 145 MB/s 2024-04-25T09:04:35.381 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.381 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.381 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0121221 s, 86.5 MB/s 2024-04-25T09:04:35.392 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.392 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.392 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861627 s, 122 MB/s 2024-04-25T09:04:35.402 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.403 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.403 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00840468 s, 125 MB/s 2024-04-25T09:04:35.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:35.404 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:04:35.404 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:04:35.404 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T09:04:35.404 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:04:35.404 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:04:35.404 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:04:35.404 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:04:35.473 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.473 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.473 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00827933 s, 127 MB/s 2024-04-25T09:04:35.483 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.483 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.483 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716773 s, 146 MB/s 2024-04-25T09:04:35.492 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.492 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.492 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00726923 s, 144 MB/s 2024-04-25T09:04:35.502 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.502 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.502 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744031 s, 141 MB/s 2024-04-25T09:04:35.512 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.513 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.513 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0085476 s, 123 MB/s 2024-04-25T09:04:35.524 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.524 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.524 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00883767 s, 119 MB/s 2024-04-25T09:04:35.533 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.533 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.533 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679575 s, 154 MB/s 2024-04-25T09:04:35.542 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.542 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.543 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722779 s, 145 MB/s 2024-04-25T09:04:35.551 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.551 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.551 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00665298 s, 158 MB/s 2024-04-25T09:04:35.562 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.562 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.562 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00832892 s, 126 MB/s 2024-04-25T09:04:35.571 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.572 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.572 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741053 s, 141 MB/s 2024-04-25T09:04:35.580 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.580 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.581 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00673377 s, 156 MB/s 2024-04-25T09:04:35.590 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.590 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.590 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679529 s, 154 MB/s 2024-04-25T09:04:35.600 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.600 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.600 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761473 s, 138 MB/s 2024-04-25T09:04:35.609 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.609 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.609 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00677019 s, 155 MB/s 2024-04-25T09:04:35.618 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.618 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.619 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732458 s, 143 MB/s 2024-04-25T09:04:35.627 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.627 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.627 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00679313 s, 154 MB/s 2024-04-25T09:04:35.637 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.637 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.637 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737973 s, 142 MB/s 2024-04-25T09:04:35.647 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.647 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.647 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822902 s, 127 MB/s 2024-04-25T09:04:35.656 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.657 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.657 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00690793 s, 152 MB/s 2024-04-25T09:04:35.667 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.667 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.667 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756986 s, 139 MB/s 2024-04-25T09:04:35.676 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.676 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.677 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717067 s, 146 MB/s 2024-04-25T09:04:35.687 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.687 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.687 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00848628 s, 124 MB/s 2024-04-25T09:04:35.698 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.698 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.698 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00821901 s, 128 MB/s 2024-04-25T09:04:35.707 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.707 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.707 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00702969 s, 149 MB/s 2024-04-25T09:04:35.718 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.718 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.718 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00824018 s, 127 MB/s 2024-04-25T09:04:35.728 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.728 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.728 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00734333 s, 143 MB/s 2024-04-25T09:04:35.738 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.738 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.738 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760421 s, 138 MB/s 2024-04-25T09:04:35.748 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.748 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.748 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743751 s, 141 MB/s 2024-04-25T09:04:35.756 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.756 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.757 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00682071 s, 154 MB/s 2024-04-25T09:04:35.766 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.767 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.767 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00765444 s, 137 MB/s 2024-04-25T09:04:35.776 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.776 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.776 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00748373 s, 140 MB/s 2024-04-25T09:04:35.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:35.778 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:04:35.778 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:04:35.778 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T09:04:35.778 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:04:35.778 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:04:35.778 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:04:35.778 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:04:35.846 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.846 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.846 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714739 s, 147 MB/s 2024-04-25T09:04:35.856 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.856 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.856 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743894 s, 141 MB/s 2024-04-25T09:04:35.866 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.866 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.866 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00750097 s, 140 MB/s 2024-04-25T09:04:35.876 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.876 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.876 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730561 s, 144 MB/s 2024-04-25T09:04:35.885 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.885 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.885 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00708693 s, 148 MB/s 2024-04-25T09:04:35.894 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.894 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.894 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00700322 s, 150 MB/s 2024-04-25T09:04:35.904 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.904 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.904 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00706615 s, 148 MB/s 2024-04-25T09:04:35.914 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.915 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.915 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00820824 s, 128 MB/s 2024-04-25T09:04:35.924 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.924 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.924 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717267 s, 146 MB/s 2024-04-25T09:04:35.934 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.935 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.935 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00836445 s, 125 MB/s 2024-04-25T09:04:35.945 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.945 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.945 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751684 s, 139 MB/s 2024-04-25T09:04:35.954 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.955 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.955 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00754103 s, 139 MB/s 2024-04-25T09:04:35.964 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.964 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.964 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00703912 s, 149 MB/s 2024-04-25T09:04:35.973 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.973 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.973 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00695253 s, 151 MB/s 2024-04-25T09:04:35.984 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.984 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.984 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00847193 s, 124 MB/s 2024-04-25T09:04:35.993 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:35.994 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:35.994 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074502 s, 141 MB/s 2024-04-25T09:04:36.003 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.003 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.003 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744442 s, 141 MB/s 2024-04-25T09:04:36.014 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.014 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.014 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00772764 s, 136 MB/s 2024-04-25T09:04:36.023 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.023 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.024 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753652 s, 139 MB/s 2024-04-25T09:04:36.034 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.034 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.034 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842729 s, 124 MB/s 2024-04-25T09:04:36.044 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.044 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.044 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737577 s, 142 MB/s 2024-04-25T09:04:36.054 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.054 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.054 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738959 s, 142 MB/s 2024-04-25T09:04:36.064 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.065 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.065 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830837 s, 126 MB/s 2024-04-25T09:04:36.074 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.074 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.074 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00755938 s, 139 MB/s 2024-04-25T09:04:36.084 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.084 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.084 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00711874 s, 147 MB/s 2024-04-25T09:04:36.093 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.093 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.093 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743557 s, 141 MB/s 2024-04-25T09:04:36.103 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.103 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.103 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00717031 s, 146 MB/s 2024-04-25T09:04:36.114 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.114 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.114 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00875365 s, 120 MB/s 2024-04-25T09:04:36.125 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.125 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.125 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00709341 s, 148 MB/s 2024-04-25T09:04:36.134 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.134 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.134 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00761091 s, 138 MB/s 2024-04-25T09:04:36.144 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.144 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.144 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00830284 s, 126 MB/s 2024-04-25T09:04:36.154 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.154 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.154 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0074194 s, 141 MB/s 2024-04-25T09:04:36.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:36.155 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:04:36.155 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:04:36.155 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T09:04:36.155 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:04:36.156 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:04:36.156 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:04:36.156 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:04:36.225 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.225 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.225 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00873793 s, 120 MB/s 2024-04-25T09:04:36.234 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.234 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.234 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732966 s, 143 MB/s 2024-04-25T09:04:36.245 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.245 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.245 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00864474 s, 121 MB/s 2024-04-25T09:04:36.274 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.274 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.274 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749414 s, 140 MB/s 2024-04-25T09:04:36.285 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.285 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.285 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00861973 s, 122 MB/s 2024-04-25T09:04:36.295 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.295 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.295 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743475 s, 141 MB/s 2024-04-25T09:04:36.306 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.306 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.306 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00855888 s, 123 MB/s 2024-04-25T09:04:36.316 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.317 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.317 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00822895 s, 127 MB/s 2024-04-25T09:04:36.327 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.327 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.327 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00856391 s, 122 MB/s 2024-04-25T09:04:36.337 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.337 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.337 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00744144 s, 141 MB/s 2024-04-25T09:04:36.347 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.347 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.347 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741948 s, 141 MB/s 2024-04-25T09:04:36.356 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.357 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.357 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00767459 s, 137 MB/s 2024-04-25T09:04:36.366 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.366 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.366 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00718814 s, 146 MB/s 2024-04-25T09:04:36.376 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.376 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.376 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00746169 s, 141 MB/s 2024-04-25T09:04:36.385 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.385 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.385 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00667486 s, 157 MB/s 2024-04-25T09:04:36.394 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.394 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.395 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0070436 s, 149 MB/s 2024-04-25T09:04:36.404 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.404 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.404 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741915 s, 141 MB/s 2024-04-25T09:04:36.413 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.413 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.414 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00693439 s, 151 MB/s 2024-04-25T09:04:36.423 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.423 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.423 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00707517 s, 148 MB/s 2024-04-25T09:04:36.433 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.433 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.433 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741768 s, 141 MB/s 2024-04-25T09:04:36.442 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.442 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.443 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00714082 s, 147 MB/s 2024-04-25T09:04:36.453 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.454 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.454 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00873784 s, 120 MB/s 2024-04-25T09:04:36.464 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.464 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.464 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00794209 s, 132 MB/s 2024-04-25T09:04:36.475 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.475 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.475 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00852496 s, 123 MB/s 2024-04-25T09:04:36.485 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.485 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.485 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00775606 s, 135 MB/s 2024-04-25T09:04:36.494 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.494 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.494 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00739798 s, 142 MB/s 2024-04-25T09:04:36.504 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.504 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.504 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00722343 s, 145 MB/s 2024-04-25T09:04:36.513 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.513 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.513 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00689568 s, 152 MB/s 2024-04-25T09:04:36.523 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.523 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.523 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00756724 s, 139 MB/s 2024-04-25T09:04:36.533 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.533 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.533 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00740238 s, 142 MB/s 2024-04-25T09:04:36.543 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.543 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.543 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00751665 s, 140 MB/s 2024-04-25T09:04:36.552 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.553 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.553 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749385 s, 140 MB/s 2024-04-25T09:04:36.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:36.554 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T09:04:36.554 DEBUG:teuthology.orchestra.run.smithi043:> set -e 2024-04-25T09:04:36.554 DEBUG:teuthology.orchestra.run.smithi043:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T09:04:36.554 DEBUG:teuthology.orchestra.run.smithi043:> for ((i = 0; i < 32; i++)); do 2024-04-25T09:04:36.554 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T09:04:36.554 DEBUG:teuthology.orchestra.run.smithi043:> done 2024-04-25T09:04:36.554 DEBUG:teuthology.orchestra.run.smithi043:> ') 2024-04-25T09:04:36.624 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.624 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.624 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00753765 s, 139 MB/s 2024-04-25T09:04:36.633 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.633 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.633 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743796 s, 141 MB/s 2024-04-25T09:04:36.644 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.644 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.644 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00877756 s, 119 MB/s 2024-04-25T09:04:36.653 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.654 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.654 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0069061 s, 152 MB/s 2024-04-25T09:04:36.663 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.663 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.663 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00737148 s, 142 MB/s 2024-04-25T09:04:36.673 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.673 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.673 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0077006 s, 136 MB/s 2024-04-25T09:04:36.682 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.682 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.683 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00691946 s, 152 MB/s 2024-04-25T09:04:36.693 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.693 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.693 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00842894 s, 124 MB/s 2024-04-25T09:04:36.703 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.703 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.703 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729711 s, 144 MB/s 2024-04-25T09:04:36.712 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.712 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.712 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00666982 s, 157 MB/s 2024-04-25T09:04:36.721 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.721 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.722 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00716275 s, 146 MB/s 2024-04-25T09:04:36.731 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.731 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.731 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00710665 s, 148 MB/s 2024-04-25T09:04:36.740 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.740 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.740 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00736457 s, 142 MB/s 2024-04-25T09:04:36.751 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.751 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.751 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00878661 s, 119 MB/s 2024-04-25T09:04:36.761 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.761 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.761 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00730089 s, 144 MB/s 2024-04-25T09:04:36.770 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.770 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.770 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00681593 s, 154 MB/s 2024-04-25T09:04:36.781 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.781 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.782 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00889503 s, 118 MB/s 2024-04-25T09:04:36.791 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.791 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.791 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00721752 s, 145 MB/s 2024-04-25T09:04:36.801 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.801 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.801 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00741088 s, 141 MB/s 2024-04-25T09:04:36.810 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.810 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.810 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00680758 s, 154 MB/s 2024-04-25T09:04:36.821 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.821 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.821 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00834101 s, 126 MB/s 2024-04-25T09:04:36.832 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.832 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.832 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00860391 s, 122 MB/s 2024-04-25T09:04:36.842 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.842 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.843 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00829422 s, 126 MB/s 2024-04-25T09:04:36.852 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.852 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.852 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00729054 s, 144 MB/s 2024-04-25T09:04:36.862 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.862 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.862 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00760318 s, 138 MB/s 2024-04-25T09:04:36.871 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.872 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.872 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00749308 s, 140 MB/s 2024-04-25T09:04:36.881 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.881 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.881 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00733082 s, 143 MB/s 2024-04-25T09:04:36.891 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.891 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.892 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00732978 s, 143 MB/s 2024-04-25T09:04:36.901 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.901 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.901 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00762508 s, 138 MB/s 2024-04-25T09:04:36.910 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.910 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.910 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00719848 s, 146 MB/s 2024-04-25T09:04:36.920 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.920 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.920 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00738259 s, 142 MB/s 2024-04-25T09:04:36.929 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records in 2024-04-25T09:04:36.929 INFO:teuthology.orchestra.run.smithi043.stderr:1+0 records out 2024-04-25T09:04:36.929 INFO:teuthology.orchestra.run.smithi043.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00743204 s, 141 MB/s 2024-04-25T09:04:36.930 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 1 2024-04-25T09:04:37.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.087+0000 7fdde2eab640 1 Processor -- start 2024-04-25T09:04:37.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.087+0000 7fdde2eab640 1 -- start start 2024-04-25T09:04:37.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.087+0000 7fdde2eab640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdddc0d1a20 0x7fdddc0b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.087+0000 7fdde2eab640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdddc15bf90 con 0x7fdddc0d1e00 2024-04-25T09:04:37.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.087+0000 7fdde2eab640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdddc0cee70 con 0x7fdddc0d2260 2024-04-25T09:04:37.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.087+0000 7fdde2eab640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdddc0cf430 con 0x7fdddc0d1a20 2024-04-25T09:04:37.076 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.088+0000 7fdde0c47640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fdddc0d1a20 0x7fdddc0b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.088+0000 7fdddbfff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fdddc0d1e00 0x7fdddc0b5100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:55774/0 (socket says 172.21.15.43:55774) 2024-04-25T09:04:37.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.088+0000 7fdddbfff640 1 -- 172.21.15.43:0/4282963214 learned_addr learned my addr 172.21.15.43:0/4282963214 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:37.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.088+0000 7fdde0c47640 1 -- 172.21.15.43:0/4282963214 >> v1:172.21.15.193:6789/0 conn(0x7fdddc0d2260 legacy=0x7fdddc0b5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.088+0000 7fdde0c47640 1 -- 172.21.15.43:0/4282963214 >> v1:172.21.15.43:6789/0 conn(0x7fdddc0d1e00 legacy=0x7fdddc0b5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.088+0000 7fdddb7fe640 1 -- 172.21.15.43:0/4282963214 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3488570676 0 0) 0x7fdddc15bf90 con 0x7fdddc0d1e00 2024-04-25T09:04:37.077 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.088+0000 7fdde0c47640 1 -- 172.21.15.43:0/4282963214 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdddc0cf150 con 0x7fdddc0d1a20 2024-04-25T09:04:37.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.089+0000 7fdde0c47640 1 --2- 172.21.15.43:0/4282963214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdddc0d1a20 0x7fdddc0b2b90 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7fddd00097a0 tx=0x7fddd002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37204c80767c9a7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:37.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.089+0000 7fdddb7fe640 1 -- 172.21.15.43:0/4282963214 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd0037070 con 0x7fdddc0d1a20 2024-04-25T09:04:37.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.089+0000 7fdddb7fe640 1 -- 172.21.15.43:0/4282963214 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd0032440 con 0x7fdddc0d1a20 2024-04-25T09:04:37.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.089+0000 7fdddb7fe640 1 -- 172.21.15.43:0/4282963214 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd002ae60 con 0x7fdddc0d1a20 2024-04-25T09:04:37.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.089+0000 7fdde2eab640 1 -- 172.21.15.43:0/4282963214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdddc0d1a20 msgr2=0x7fdddc0b2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.089+0000 7fdde2eab640 1 --2- 172.21.15.43:0/4282963214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdddc0d1a20 0x7fdddc0b2b90 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7fddd00097a0 tx=0x7fddd002a110 comp rx=0 tx=0).stop 2024-04-25T09:04:37.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.089+0000 7fdde2eab640 1 -- 172.21.15.43:0/4282963214 shutdown_connections 2024-04-25T09:04:37.078 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.089+0000 7fdde2eab640 1 --2- 172.21.15.43:0/4282963214 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdddc0d1a20 0x7fdddc0b2b90 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.089+0000 7fdde2eab640 1 -- 172.21.15.43:0/4282963214 >> 172.21.15.43:0/4282963214 conn(0x7fdddc072aa0 msgr2=0x7fdddc09aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:37.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.089+0000 7fdde2eab640 1 -- 172.21.15.43:0/4282963214 shutdown_connections 2024-04-25T09:04:37.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.089+0000 7fdde2eab640 1 -- 172.21.15.43:0/4282963214 wait complete. 2024-04-25T09:04:37.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.091+0000 7fdde2eab640 1 Processor -- start 2024-04-25T09:04:37.079 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.091+0000 7fdde2eab640 1 -- start start 2024-04-25T09:04:37.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.091+0000 7fdde2eab640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdddc0d1a20 0x7fdddc15d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.092+0000 7fdde2eab640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdddc0d1e00 0x7fdddc0af5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.092+0000 7fdde2eab640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdddc0d2260 0x7fdddc143e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.092+0000 7fdde2eab640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc0cea60 con 0x7fdddc0d1a20 2024-04-25T09:04:37.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.092+0000 7fdde2eab640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc0b4be0 con 0x7fdddc0d1e00 2024-04-25T09:04:37.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.092+0000 7fdde2eab640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc0cebe0 con 0x7fdddc0d2260 2024-04-25T09:04:37.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.092+0000 7fdde0c47640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdddc0d1a20 0x7fdddc15d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.092+0000 7fdde0c47640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdddc0d1a20 0x7fdddc15d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55680/0 (socket says 172.21.15.43:55680) 2024-04-25T09:04:37.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.092+0000 7fdde0c47640 1 -- 172.21.15.43:0/4240652699 learned_addr learned my addr 172.21.15.43:0/4240652699 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:37.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.092+0000 7fdddbfff640 1 --2- 172.21.15.43:0/4240652699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdddc0d1e00 0x7fdddc0af5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.092+0000 7fdde0c47640 1 -- 172.21.15.43:0/4240652699 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdddc0d2260 msgr2=0x7fdddc143e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.092+0000 7fdde0c47640 1 --2- 172.21.15.43:0/4240652699 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdddc0d2260 0x7fdddc143e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.092+0000 7fdde0c47640 1 -- 172.21.15.43:0/4240652699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdddc0d1e00 msgr2=0x7fdddc0af5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:37.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.092+0000 7fdde0c47640 1 --2- 172.21.15.43:0/4240652699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdddc0d1e00 0x7fdddc0af5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:37.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.092+0000 7fdde0c47640 1 -- 172.21.15.43:0/4240652699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdddc0b12e0 con 0x7fdddc0d1a20 2024-04-25T09:04:37.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.092+0000 7fdde0c47640 1 --2- 172.21.15.43:0/4240652699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdddc0d1a20 0x7fdddc15d670 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7fddd002a700 tx=0x7fddd00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:37.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.092+0000 7fddd97fa640 1 -- 172.21.15.43:0/4240652699 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd0037040 con 0x7fdddc0d1a20 2024-04-25T09:04:37.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.092+0000 7fddd97fa640 1 -- 172.21.15.43:0/4240652699 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddd0042070 con 0x7fdddc0d1a20 2024-04-25T09:04:37.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.092+0000 7fdde2eab640 1 -- 172.21.15.43:0/4240652699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdddc0b15c0 con 0x7fdddc0d1a20 2024-04-25T09:04:37.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.092+0000 7fddd97fa640 1 -- 172.21.15.43:0/4240652699 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd0030c70 con 0x7fdddc0d1a20 2024-04-25T09:04:37.082 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.092+0000 7fdde2eab640 1 -- 172.21.15.43:0/4240652699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdddc0b1b80 con 0x7fdddc0d1a20 2024-04-25T09:04:37.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.093+0000 7fdde2eab640 1 -- 172.21.15.43:0/4240652699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdddc074f60 con 0x7fdddc0d1a20 2024-04-25T09:04:37.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.094+0000 7fddd97fa640 1 -- 172.21.15.43:0/4240652699 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fddd0047460 con 0x7fdddc0d1a20 2024-04-25T09:04:37.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.094+0000 7fddd97fa640 1 --2- 172.21.15.43:0/4240652699 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fddc006b4d0 0x7fddc006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:37.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.094+0000 7fddd97fa640 1 -- 172.21.15.43:0/4240652699 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fddd00aaf60 con 0x7fdddc0d1a20 2024-04-25T09:04:37.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.095+0000 7fdddbfff640 1 --2- 172.21.15.43:0/4240652699 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fddc006b4d0 0x7fddc006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:37.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.095+0000 7fdddbfff640 1 --2- 172.21.15.43:0/4240652699 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fddc006b4d0 0x7fddc006d990 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fddc8007fd0 tx=0x7fddc8009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:37.087 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.099+0000 7fddd97fa640 1 -- 172.21.15.43:0/4240652699 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fddd0045360 con 0x7fdddc0d1a20 2024-04-25T09:04:37.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:37.315+0000 7fdde2eab640 1 -- 172.21.15.43:0/4240652699 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7fdddc15ed50 con 0x7fdddc0d1a20 2024-04-25T09:04:38.096 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.107+0000 7fddd97fa640 1 -- 172.21.15.43:0/4240652699 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v88) v1 ==== 105+0+0 (secure 0 0 0) 0x7fddd0079d40 con 0x7fdddc0d1a20 2024-04-25T09:04:38.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.110+0000 7fdde2eab640 1 -- 172.21.15.43:0/4240652699 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fddc006b4d0 msgr2=0x7fddc006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.110+0000 7fdde2eab640 1 --2- 172.21.15.43:0/4240652699 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fddc006b4d0 0x7fddc006d990 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fddc8007fd0 tx=0x7fddc8009040 comp rx=0 tx=0).stop 2024-04-25T09:04:38.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.110+0000 7fdde2eab640 1 -- 172.21.15.43:0/4240652699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdddc0d1a20 msgr2=0x7fdddc15d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.110+0000 7fdde2eab640 1 --2- 172.21.15.43:0/4240652699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdddc0d1a20 0x7fdddc15d670 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7fddd002a700 tx=0x7fddd00046c0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.110+0000 7fdde2eab640 1 -- 172.21.15.43:0/4240652699 shutdown_connections 2024-04-25T09:04:38.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.110+0000 7fdde2eab640 1 --2- 172.21.15.43:0/4240652699 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fdddc0d2260 0x7fdddc143e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.110+0000 7fdde2eab640 1 --2- 172.21.15.43:0/4240652699 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fddc006b4d0 0x7fddc006d990 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.110+0000 7fdde2eab640 1 --2- 172.21.15.43:0/4240652699 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fdddc0d1e00 0x7fdddc0af5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.110+0000 7fdde2eab640 1 --2- 172.21.15.43:0/4240652699 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fdddc0d1a20 0x7fdddc15d670 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.110+0000 7fdde2eab640 1 -- 172.21.15.43:0/4240652699 >> 172.21.15.43:0/4240652699 conn(0x7fdddc072aa0 msgr2=0x7fdddc15c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:38.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.111+0000 7fdde2eab640 1 -- 172.21.15.43:0/4240652699 shutdown_connections 2024-04-25T09:04:38.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.111+0000 7fdde2eab640 1 -- 172.21.15.43:0/4240652699 wait complete. 2024-04-25T09:04:38.112 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-25T09:04:38.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.233+0000 7f248cce1640 1 Processor -- start 2024-04-25T09:04:38.221 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.233+0000 7f248cce1640 1 -- start start 2024-04-25T09:04:38.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.234+0000 7f248cce1640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f24880cd470 0x7f24880b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.234+0000 7f248cce1640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24880d2620 con 0x7f248806f6a0 2024-04-25T09:04:38.225 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.234+0000 7f248cce1640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24880b0b80 con 0x7f248815ddd0 2024-04-25T09:04:38.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.234+0000 7f248cce1640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24880b0e60 con 0x7f24880cd470 2024-04-25T09:04:38.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.234+0000 7f2485d9b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f248806f6a0 0x7f248806c4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:55786/0 (socket says 172.21.15.43:55786) 2024-04-25T09:04:38.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.234+0000 7f2485d9b640 1 -- 172.21.15.43:0/662621389 learned_addr learned my addr 172.21.15.43:0/662621389 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:38.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.234+0000 7f2486d9d640 1 --2- 172.21.15.43:0/662621389 >> v2:172.21.15.193:3300/0 conn(0x7f24880cd470 0x7f24880b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.234+0000 7f248559a640 1 -- 172.21.15.43:0/662621389 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2936710327 0 0) 0x7f24880d2620 con 0x7f248806f6a0 2024-04-25T09:04:38.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.234+0000 7f248559a640 1 -- 172.21.15.43:0/662621389 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2470003520 con 0x7f248806f6a0 2024-04-25T09:04:38.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.235+0000 7f248559a640 1 -- 172.21.15.43:0/662621389 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3372435043 0 0) 0x7f24880b0b80 con 0x7f248815ddd0 2024-04-25T09:04:38.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.235+0000 7f248559a640 1 -- 172.21.15.43:0/662621389 --> v1:172.21.15.193:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24880d2620 con 0x7f248815ddd0 2024-04-25T09:04:38.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.235+0000 7f248559a640 1 -- 172.21.15.43:0/662621389 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3235311922 0 0) 0x7f2470003520 con 0x7f248806f6a0 2024-04-25T09:04:38.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.235+0000 7f248559a640 1 -- 172.21.15.43:0/662621389 >> v2:172.21.15.193:3300/0 conn(0x7f24880cd470 msgr2=0x7f24880b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:38.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.235+0000 7f248559a640 1 --2- 172.21.15.43:0/662621389 >> v2:172.21.15.193:3300/0 conn(0x7f24880cd470 0x7f24880b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.235+0000 7f248559a640 1 -- 172.21.15.43:0/662621389 >> v1:172.21.15.193:6789/0 conn(0x7f248815ddd0 legacy=0x7f248806ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.235+0000 7f248559a640 1 -- 172.21.15.43:0/662621389 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24880cbf40 con 0x7f248806f6a0 2024-04-25T09:04:38.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.235+0000 7f248559a640 1 -- 172.21.15.43:0/662621389 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f247c001670 con 0x7f248806f6a0 2024-04-25T09:04:38.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.235+0000 7f248559a640 1 -- 172.21.15.43:0/662621389 >> v1:172.21.15.43:6789/0 conn(0x7f248806f6a0 legacy=0x7f248806c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.235+0000 7f248559a640 1 --2- 172.21.15.43:0/662621389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2470003b40 0x7f2470003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.235+0000 7f248559a640 1 --2- 172.21.15.43:0/662621389 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2470004800 0x7f2470004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.235+0000 7f248559a640 1 --2- 172.21.15.43:0/662621389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24700051c0 0x7f247000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.235+0000 7f248559a640 1 -- 172.21.15.43:0/662621389 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24880b0e60 con 0x7f24700051c0 2024-04-25T09:04:38.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.235+0000 7f248559a640 1 -- 172.21.15.43:0/662621389 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f247000c040 con 0x7f2470003b40 2024-04-25T09:04:38.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.235+0000 7f248559a640 1 -- 172.21.15.43:0/662621389 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f247000c240 con 0x7f2470004800 2024-04-25T09:04:38.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.235+0000 7f2486d9d640 1 --2- 172.21.15.43:0/662621389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24700051c0 0x7f247000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.235+0000 7f248659c640 1 --2- 172.21.15.43:0/662621389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2470003b40 0x7f2470003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.235+0000 7f2485d9b640 1 --2- 172.21.15.43:0/662621389 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2470004800 0x7f2470004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.235+0000 7f2486d9d640 1 -- 172.21.15.43:0/662621389 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2470004800 msgr2=0x7f2470004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:38.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.235+0000 7f2486d9d640 1 --2- 172.21.15.43:0/662621389 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f2470004800 0x7f2470004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.235+0000 7f2486d9d640 1 -- 172.21.15.43:0/662621389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2470003b40 msgr2=0x7f2470003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:38.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.235+0000 7f2486d9d640 1 --2- 172.21.15.43:0/662621389 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2470003b40 0x7f2470003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.235+0000 7f2486d9d640 1 -- 172.21.15.43:0/662621389 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2470003520 con 0x7f24700051c0 2024-04-25T09:04:38.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.236+0000 7f2486d9d640 1 -- 172.21.15.43:0/662621389 reap_dead start 2024-04-25T09:04:38.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.236+0000 7f2486d9d640 1 --2- 172.21.15.43:0/662621389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24700051c0 0x7f247000b9c0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f2480009fc0 tx=0x7f2470005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ead79dfb382ede94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:38.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.236+0000 7f248559a640 1 -- 172.21.15.43:0/662621389 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2480035020 con 0x7f24700051c0 2024-04-25T09:04:38.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.236+0000 7f248559a640 1 -- 172.21.15.43:0/662621389 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2480003bd0 con 0x7f24700051c0 2024-04-25T09:04:38.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.236+0000 7f248559a640 1 -- 172.21.15.43:0/662621389 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2480009c10 con 0x7f24700051c0 2024-04-25T09:04:38.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.237+0000 7f248cce1640 1 -- 172.21.15.43:0/662621389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24700051c0 msgr2=0x7f247000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.237+0000 7f248cce1640 1 --2- 172.21.15.43:0/662621389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24700051c0 0x7f247000b9c0 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f2480009fc0 tx=0x7f2470005830 comp rx=0 tx=0).stop 2024-04-25T09:04:38.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.237+0000 7f248cce1640 1 -- 172.21.15.43:0/662621389 shutdown_connections 2024-04-25T09:04:38.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.237+0000 7f248cce1640 1 --2- 172.21.15.43:0/662621389 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24700051c0 0x7f247000b9c0 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.237+0000 7f248cce1640 1 -- 172.21.15.43:0/662621389 >> 172.21.15.43:0/662621389 conn(0x7f24880d17e0 msgr2=0x7f24880d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:38.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.237+0000 7f248cce1640 1 -- 172.21.15.43:0/662621389 shutdown_connections 2024-04-25T09:04:38.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.237+0000 7f248cce1640 1 -- 172.21.15.43:0/662621389 wait complete. 2024-04-25T09:04:38.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.238+0000 7f248cce1640 1 Processor -- start 2024-04-25T09:04:38.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.238+0000 7f248cce1640 1 -- start start 2024-04-25T09:04:38.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.239+0000 7f248cce1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24700051c0 0x7f248815e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.239+0000 7f248cce1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f248815a8d0 0x7f248815ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.239+0000 7f248cce1640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f248815f100 0x7f248815f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.239+0000 7f248cce1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24880b4160 con 0x7f24700051c0 2024-04-25T09:04:38.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.239+0000 7f248cce1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24880b3fe0 con 0x7f248815a8d0 2024-04-25T09:04:38.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.239+0000 7f248cce1640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f248806ef90 con 0x7f248815f100 2024-04-25T09:04:38.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.239+0000 7f248659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24700051c0 0x7f248815e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.239+0000 7f248659c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24700051c0 0x7f248815e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55688/0 (socket says 172.21.15.43:55688) 2024-04-25T09:04:38.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.239+0000 7f248659c640 1 -- 172.21.15.43:0/752991174 learned_addr learned my addr 172.21.15.43:0/752991174 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:38.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.239+0000 7f2485d9b640 1 --2- 172.21.15.43:0/752991174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f248815a8d0 0x7f248815ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.239+0000 7f2486d9d640 1 --2- 172.21.15.43:0/752991174 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f248815f100 0x7f248815f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.239+0000 7f248659c640 1 -- 172.21.15.43:0/752991174 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f248815f100 msgr2=0x7f248815f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.239+0000 7f248659c640 1 --2- 172.21.15.43:0/752991174 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f248815f100 0x7f248815f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.239+0000 7f248659c640 1 -- 172.21.15.43:0/752991174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f248815a8d0 msgr2=0x7f248815ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:38.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.239+0000 7f248659c640 1 --2- 172.21.15.43:0/752991174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f248815a8d0 0x7f248815ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:38.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.239+0000 7f248659c640 1 -- 172.21.15.43:0/752991174 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24880b1d90 con 0x7f24700051c0 2024-04-25T09:04:38.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.240+0000 7f248659c640 1 --2- 172.21.15.43:0/752991174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24700051c0 0x7f248815e680 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f247800f7a0 tx=0x7f247800fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:38.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.240+0000 7f2476ffd640 1 -- 172.21.15.43:0/752991174 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247801a070 con 0x7f24700051c0 2024-04-25T09:04:38.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.240+0000 7f2476ffd640 1 -- 172.21.15.43:0/752991174 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f247800fe50 con 0x7f24700051c0 2024-04-25T09:04:38.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.240+0000 7f248cce1640 1 -- 172.21.15.43:0/752991174 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24880b2070 con 0x7f24700051c0 2024-04-25T09:04:38.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.240+0000 7f2476ffd640 1 -- 172.21.15.43:0/752991174 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2478007050 con 0x7f24700051c0 2024-04-25T09:04:38.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.240+0000 7f248cce1640 1 -- 172.21.15.43:0/752991174 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24880b2910 con 0x7f24700051c0 2024-04-25T09:04:38.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.241+0000 7f248cce1640 1 -- 172.21.15.43:0/752991174 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2488074f60 con 0x7f24700051c0 2024-04-25T09:04:38.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.241+0000 7f2476ffd640 1 -- 172.21.15.43:0/752991174 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f247801d070 con 0x7f24700051c0 2024-04-25T09:04:38.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.242+0000 7f2476ffd640 1 --2- 172.21.15.43:0/752991174 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f246806b4d0 0x7f246806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:38.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.242+0000 7f2476ffd640 1 -- 172.21.15.43:0/752991174 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2478089940 con 0x7f24700051c0 2024-04-25T09:04:38.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.242+0000 7f2485d9b640 1 --2- 172.21.15.43:0/752991174 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f246806b4d0 0x7f246806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:38.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.242+0000 7f2485d9b640 1 --2- 172.21.15.43:0/752991174 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f246806b4d0 0x7f246806d990 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f247c001050 tx=0x7f247c001e40 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:38.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.246+0000 7f2476ffd640 1 -- 172.21.15.43:0/752991174 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2478058170 con 0x7f24700051c0 2024-04-25T09:04:38.456 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:38.468+0000 7f248cce1640 1 -- 172.21.15.43:0/752991174 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f24880b2240 con 0x7f24700051c0 2024-04-25T09:04:39.097 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.109+0000 7f2476ffd640 1 -- 172.21.15.43:0/752991174 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v89) v1 ==== 121+0+0 (secure 0 0 0) 0x7f247805c210 con 0x7f24700051c0 2024-04-25T09:04:39.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.112+0000 7f248cce1640 1 -- 172.21.15.43:0/752991174 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f246806b4d0 msgr2=0x7f246806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.112+0000 7f248cce1640 1 --2- 172.21.15.43:0/752991174 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f246806b4d0 0x7f246806d990 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f247c001050 tx=0x7f247c001e40 comp rx=0 tx=0).stop 2024-04-25T09:04:39.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.112+0000 7f248cce1640 1 -- 172.21.15.43:0/752991174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24700051c0 msgr2=0x7f248815e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.112+0000 7f248cce1640 1 --2- 172.21.15.43:0/752991174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24700051c0 0x7f248815e680 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f247800f7a0 tx=0x7f247800fc60 comp rx=0 tx=0).stop 2024-04-25T09:04:39.100 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.112+0000 7f248cce1640 1 -- 172.21.15.43:0/752991174 shutdown_connections 2024-04-25T09:04:39.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.112+0000 7f248cce1640 1 --2- 172.21.15.43:0/752991174 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f248815f100 0x7f248815f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.112+0000 7f248cce1640 1 --2- 172.21.15.43:0/752991174 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f246806b4d0 0x7f246806d990 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.112+0000 7f248cce1640 1 --2- 172.21.15.43:0/752991174 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f248815a8d0 0x7f248815ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.112+0000 7f248cce1640 1 --2- 172.21.15.43:0/752991174 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f24700051c0 0x7f248815e680 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.112+0000 7f248cce1640 1 -- 172.21.15.43:0/752991174 >> 172.21.15.43:0/752991174 conn(0x7f24880d17e0 msgr2=0x7f24880c66a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:39.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.112+0000 7f248cce1640 1 -- 172.21.15.43:0/752991174 shutdown_connections 2024-04-25T09:04:39.101 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.112+0000 7f248cce1640 1 -- 172.21.15.43:0/752991174 wait complete. 2024-04-25T09:04:39.116 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:04:39.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.238+0000 7f0fad990640 1 Processor -- start 2024-04-25T09:04:39.226 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.238+0000 7f0fad990640 1 -- start start 2024-04-25T09:04:39.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.239+0000 7f0fad990640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0fa80c8d00 0x7f0fa8162100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.239+0000 7f0fad990640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0fa80ce660 0x7f0fa80ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.239+0000 7f0fad990640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fa80d24a0 con 0x7f0fa815df50 2024-04-25T09:04:39.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.239+0000 7f0fad990640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fa815e330 con 0x7f0fa80c8d00 2024-04-25T09:04:39.227 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.239+0000 7f0fad990640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa80d2740 con 0x7f0fa80ce660 2024-04-25T09:04:39.228 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.239+0000 7f0fa77fe640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0fa80ce660 0x7f0fa80ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.239+0000 7f0fa67fc640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f0fa80c8d00 0x7f0fa8162100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.239+0000 7f0fa6ffd640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f0fa815df50 0x7f0fa81619f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:42294/0 (socket says 172.21.15.43:42294) 2024-04-25T09:04:39.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.239+0000 7f0fa6ffd640 1 -- 172.21.15.43:0/3178097548 learned_addr learned my addr 172.21.15.43:0/3178097548 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:39.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.239+0000 7f0fa77fe640 1 -- 172.21.15.43:0/3178097548 >> v2:172.21.15.193:3300/0 conn(0x7f0fa80c8d00 msgr2=0x7f0fa8162100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:39.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.239+0000 7f0fa77fe640 1 --2- 172.21.15.43:0/3178097548 >> v2:172.21.15.193:3300/0 conn(0x7f0fa80c8d00 0x7f0fa8162100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.239+0000 7f0fa77fe640 1 -- 172.21.15.43:0/3178097548 >> v1:172.21.15.193:6789/0 conn(0x7f0fa815df50 legacy=0x7f0fa81619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.239+0000 7f0fa5ffb640 1 -- 172.21.15.43:0/3178097548 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3883105014 0 0) 0x7f0fa80d24a0 con 0x7f0fa815df50 2024-04-25T09:04:39.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.239+0000 7f0fa77fe640 1 -- 172.21.15.43:0/3178097548 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa80d2a20 con 0x7f0fa80ce660 2024-04-25T09:04:39.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.239+0000 7f0fa67fc640 1 --2- 172.21.15.43:0/3178097548 >> v2:172.21.15.193:3300/0 conn(0x7f0fa80c8d00 0x7f0fa8162100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:04:39.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.240+0000 7f0fa77fe640 1 --2- 172.21.15.43:0/3178097548 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0fa80ce660 0x7f0fa80ceab0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0009d90 tx=0x7f0fa002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3439de2235669658 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:39.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.240+0000 7f0fa5ffb640 1 -- 172.21.15.43:0/3178097548 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa0007740 con 0x7f0fa80ce660 2024-04-25T09:04:39.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.240+0000 7f0fa5ffb640 1 -- 172.21.15.43:0/3178097548 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fa00078e0 con 0x7f0fa80ce660 2024-04-25T09:04:39.229 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.240+0000 7f0fa5ffb640 1 -- 172.21.15.43:0/3178097548 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa0007cb0 con 0x7f0fa80ce660 2024-04-25T09:04:39.230 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.241+0000 7f0fad990640 1 -- 172.21.15.43:0/3178097548 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0fa80ce660 msgr2=0x7f0fa80ceab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.241+0000 7f0fad990640 1 --2- 172.21.15.43:0/3178097548 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0fa80ce660 0x7f0fa80ceab0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0009d90 tx=0x7f0fa002a660 comp rx=0 tx=0).stop 2024-04-25T09:04:39.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.241+0000 7f0fad990640 1 -- 172.21.15.43:0/3178097548 shutdown_connections 2024-04-25T09:04:39.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.241+0000 7f0fad990640 1 --2- 172.21.15.43:0/3178097548 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0fa80ce660 0x7f0fa80ceab0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.241+0000 7f0fad990640 1 --2- 172.21.15.43:0/3178097548 >> v2:172.21.15.193:3300/0 conn(0x7f0fa80c8d00 0x7f0fa8162100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.241+0000 7f0fad990640 1 -- 172.21.15.43:0/3178097548 >> 172.21.15.43:0/3178097548 conn(0x7f0fa80d3a30 msgr2=0x7f0fa80d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:39.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.241+0000 7f0fad990640 1 -- 172.21.15.43:0/3178097548 shutdown_connections 2024-04-25T09:04:39.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.241+0000 7f0fad990640 1 -- 172.21.15.43:0/3178097548 wait complete. 2024-04-25T09:04:39.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.242+0000 7f0fad990640 1 Processor -- start 2024-04-25T09:04:39.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.242+0000 7f0fad990640 1 -- start start 2024-04-25T09:04:39.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.243+0000 7f0fad990640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fa80c8d00 0x7f0fa80c65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.243+0000 7f0fad990640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0fa80ce660 0x7f0fa81e76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.243+0000 7f0fad990640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0fa815df50 0x7f0fa81e7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.233 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.243+0000 7f0fad990640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa80d2190 con 0x7f0fa815df50 2024-04-25T09:04:39.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.243+0000 7f0fad990640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa80c77d0 con 0x7f0fa80c8d00 2024-04-25T09:04:39.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.243+0000 7f0fad990640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa8162640 con 0x7f0fa80ce660 2024-04-25T09:04:39.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.243+0000 7f0fa77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0fa815df50 0x7f0fa81e7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.243+0000 7f0fa77fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0fa815df50 0x7f0fa81e7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:55692/0 (socket says 172.21.15.43:55692) 2024-04-25T09:04:39.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.243+0000 7f0fa77fe640 1 -- 172.21.15.43:0/1736915688 learned_addr learned my addr 172.21.15.43:0/1736915688 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:39.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.243+0000 7f0fa67fc640 1 --2- 172.21.15.43:0/1736915688 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0fa80ce660 0x7f0fa81e76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.243+0000 7f0fa6ffd640 1 --2- 172.21.15.43:0/1736915688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fa80c8d00 0x7f0fa80c65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.244+0000 7f0fa77fe640 1 -- 172.21.15.43:0/1736915688 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0fa80ce660 msgr2=0x7f0fa81e76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.244+0000 7f0fa77fe640 1 --2- 172.21.15.43:0/1736915688 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0fa80ce660 0x7f0fa81e76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.244+0000 7f0fa77fe640 1 -- 172.21.15.43:0/1736915688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fa80c8d00 msgr2=0x7f0fa80c65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.244+0000 7f0fa77fe640 1 --2- 172.21.15.43:0/1736915688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fa80c8d00 0x7f0fa80c65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.244+0000 7f0fa77fe640 1 -- 172.21.15.43:0/1736915688 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa80c6b10 con 0x7f0fa815df50 2024-04-25T09:04:39.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.244+0000 7f0fa67fc640 1 --2- 172.21.15.43:0/1736915688 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0fa80ce660 0x7f0fa81e76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:04:39.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.244+0000 7f0fa77fe640 1 --2- 172.21.15.43:0/1736915688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0fa815df50 0x7f0fa81e7c10 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0009d90 tx=0x7f0fa0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:39.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.244+0000 7f0f8ffff640 1 -- 172.21.15.43:0/1736915688 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa0040070 con 0x7f0fa815df50 2024-04-25T09:04:39.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.244+0000 7f0f8ffff640 1 -- 172.21.15.43:0/1736915688 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fa0031c40 con 0x7f0fa815df50 2024-04-25T09:04:39.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.244+0000 7f0fad990640 1 -- 172.21.15.43:0/1736915688 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fa80cb530 con 0x7f0fa815df50 2024-04-25T09:04:39.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.244+0000 7f0f8ffff640 1 -- 172.21.15.43:0/1736915688 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fa0009450 con 0x7f0fa815df50 2024-04-25T09:04:39.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.244+0000 7f0fad990640 1 -- 172.21.15.43:0/1736915688 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fa80cbaf0 con 0x7f0fa815df50 2024-04-25T09:04:39.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.245+0000 7f0fad990640 1 -- 172.21.15.43:0/1736915688 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fa80680f0 con 0x7f0fa815df50 2024-04-25T09:04:39.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.245+0000 7f0f8ffff640 1 -- 172.21.15.43:0/1736915688 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0fa0039040 con 0x7f0fa815df50 2024-04-25T09:04:39.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.246+0000 7f0f8ffff640 1 --2- 172.21.15.43:0/1736915688 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0f9406b4d0 0x7f0f9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.246+0000 7f0f8ffff640 1 -- 172.21.15.43:0/1736915688 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0fa00ab020 con 0x7f0fa815df50 2024-04-25T09:04:39.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.246+0000 7f0fa6ffd640 1 --2- 172.21.15.43:0/1736915688 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0f9406b4d0 0x7f0f9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.246+0000 7f0fa6ffd640 1 --2- 172.21.15.43:0/1736915688 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0f9406b4d0 0x7f0f9406d990 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f0f98007fd0 tx=0x7f0f98007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:39.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.250+0000 7f0f8ffff640 1 -- 172.21.15.43:0/1736915688 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0fa80c6b10 con 0x7f0fa815df50 2024-04-25T09:04:39.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.473+0000 7f0fad990640 1 -- 172.21.15.43:0/1736915688 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0fa8068e90 con 0x7f0fa815df50 2024-04-25T09:04:39.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.474+0000 7f0f8ffff640 1 -- 172.21.15.43:0/1736915688 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f0fa8068e90 con 0x7f0fa815df50 2024-04-25T09:04:39.463 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:04:39.463 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/2225978762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2225978762},{"type":"v1","addr":"172.21.15.193:6836","nonce":2225978762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9711,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/454458092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":454458092},{"type":"v1","addr":"172.21.15.43:6833","nonce":454458092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":90,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T09:04:13.410799+0000","modified":"2024-04-25T09:04:39.112330+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9732},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9720":{"gid":9720,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/2314901977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":2314901977},{"type":"v1","addr":"172.21.15.43:6835","nonce":2314901977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9732":{"gid":9732,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.15.193:6837/2585534546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2585534546},{"type":"v1","addr":"172.21.15.193:6837","nonce":2585534546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T09:04:39.463 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 90 2024-04-25T09:04:39.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.476+0000 7f0fad990640 1 -- 172.21.15.43:0/1736915688 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0f9406b4d0 msgr2=0x7f0f9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.476+0000 7f0fad990640 1 --2- 172.21.15.43:0/1736915688 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0f9406b4d0 0x7f0f9406d990 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f0f98007fd0 tx=0x7f0f98007de0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.476+0000 7f0fad990640 1 -- 172.21.15.43:0/1736915688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0fa815df50 msgr2=0x7f0fa81e7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.476+0000 7f0fad990640 1 --2- 172.21.15.43:0/1736915688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0fa815df50 0x7f0fa81e7c10 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f0fa0009d90 tx=0x7f0fa0004a40 comp rx=0 tx=0).stop 2024-04-25T09:04:39.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.476+0000 7f0fad990640 1 -- 172.21.15.43:0/1736915688 shutdown_connections 2024-04-25T09:04:39.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.476+0000 7f0fad990640 1 --2- 172.21.15.43:0/1736915688 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f0fa815df50 0x7f0fa81e7c10 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.476+0000 7f0fad990640 1 --2- 172.21.15.43:0/1736915688 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f0fa80ce660 0x7f0fa81e76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.476+0000 7f0fad990640 1 --2- 172.21.15.43:0/1736915688 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f0f9406b4d0 0x7f0f9406d990 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.476+0000 7f0fad990640 1 --2- 172.21.15.43:0/1736915688 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f0fa80c8d00 0x7f0fa80c65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.476+0000 7f0fad990640 1 -- 172.21.15.43:0/1736915688 >> 172.21.15.43:0/1736915688 conn(0x7f0fa80d3a30 msgr2=0x7f0fa81616f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:39.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.477+0000 7f0fad990640 1 -- 172.21.15.43:0/1736915688 shutdown_connections 2024-04-25T09:04:39.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.477+0000 7f0fad990640 1 -- 172.21.15.43:0/1736915688 wait complete. 2024-04-25T09:04:39.879 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T09:04:13.410799+0000', 'modified': '2024-04-25T09:04:39.112330+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9732}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9720': {'gid': 9720, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.43:6835/2314901977', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.43:6834', 'nonce': 2314901977}, {'type': 'v1', 'addr': '172.21.15.43:6835', 'nonce': 2314901977}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_9732': {'gid': 9732, 'name': 'b', 'rank': 0, 'incarnation': 85, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.193:6837/2585534546', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.193:6835', 'nonce': 2585534546}, {'type': 'v1', 'addr': '172.21.15.193:6837', 'nonce': 2585534546}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T09:04:39.879 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T09:04:39.879 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T09:04:39.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.997+0000 7fc59ad09640 1 Processor -- start 2024-04-25T09:04:39.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.997+0000 7fc59ad09640 1 -- start start 2024-04-25T09:04:39.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.998+0000 7fc59ad09640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc59415df50 0x7fc5941619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.998+0000 7fc59ad09640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc594161f30 0x7fc5941623a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.998+0000 7fc59ad09640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fc5940c7cf0 0x7fc5940c8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.998+0000 7fc59ad09640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc59415e330 con 0x7fc59415df50 2024-04-25T09:04:39.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.998+0000 7fc59ad09640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5940d24a0 con 0x7fc5940c7cf0 2024-04-25T09:04:39.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.998+0000 7fc59ad09640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5940d2740 con 0x7fc594161f30 2024-04-25T09:04:39.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.998+0000 7fc598aa5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc59415df50 0x7fc5941619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.998+0000 7fc598aa5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc59415df50 0x7fc5941619f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59828/0 (socket says 172.21.15.43:59828) 2024-04-25T09:04:39.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.998+0000 7fc598aa5640 1 -- 172.21.15.43:0/2373109034 learned_addr learned my addr 172.21.15.43:0/2373109034 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:39.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.999+0000 7fc5992a6640 1 --2- 172.21.15.43:0/2373109034 >> v2:172.21.15.193:3300/0 conn(0x7fc5940c7cf0 0x7fc5940c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.999+0000 7fc598aa5640 1 -- 172.21.15.43:0/2373109034 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc594161f30 msgr2=0x7fc5941623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:39.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.999+0000 7fc593fff640 1 --2- 172.21.15.43:0/2373109034 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc594161f30 0x7fc5941623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.999+0000 7fc598aa5640 1 --2- 172.21.15.43:0/2373109034 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc594161f30 0x7fc5941623a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.999+0000 7fc598aa5640 1 -- 172.21.15.43:0/2373109034 >> v2:172.21.15.193:3300/0 conn(0x7fc5940c7cf0 msgr2=0x7fc5940c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:39.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.999+0000 7fc598aa5640 1 --2- 172.21.15.43:0/2373109034 >> v2:172.21.15.193:3300/0 conn(0x7fc5940c7cf0 0x7fc5940c8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.999+0000 7fc598aa5640 1 -- 172.21.15.43:0/2373109034 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5940d2a20 con 0x7fc59415df50 2024-04-25T09:04:39.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.999+0000 7fc5992a6640 1 --2- 172.21.15.43:0/2373109034 >> v2:172.21.15.193:3300/0 conn(0x7fc5940c7cf0 0x7fc5940c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:39.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:39.999+0000 7fc598aa5640 1 --2- 172.21.15.43:0/2373109034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc59415df50 0x7fc5941619f0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fc5940cdb50 tx=0x7fc58c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b01799bdfa816cbd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:39.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.000+0000 7fc5937fe640 1 -- 172.21.15.43:0/2373109034 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58c036070 con 0x7fc59415df50 2024-04-25T09:04:39.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.000+0000 7fc5937fe640 1 -- 172.21.15.43:0/2373109034 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc58c031600 con 0x7fc59415df50 2024-04-25T09:04:39.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.000+0000 7fc5937fe640 1 -- 172.21.15.43:0/2373109034 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58c0319d0 con 0x7fc59415df50 2024-04-25T09:04:39.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.000+0000 7fc59ad09640 1 -- 172.21.15.43:0/2373109034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc59415df50 msgr2=0x7fc5941619f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.000+0000 7fc59ad09640 1 --2- 172.21.15.43:0/2373109034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc59415df50 0x7fc5941619f0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fc5940cdb50 tx=0x7fc58c02a6a0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.000+0000 7fc59ad09640 1 -- 172.21.15.43:0/2373109034 shutdown_connections 2024-04-25T09:04:39.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.000+0000 7fc59ad09640 1 --2- 172.21.15.43:0/2373109034 >> v2:172.21.15.193:3300/0 conn(0x7fc5940c7cf0 0x7fc5940c8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.000+0000 7fc59ad09640 1 --2- 172.21.15.43:0/2373109034 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc594161f30 0x7fc5941623a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.000+0000 7fc59ad09640 1 --2- 172.21.15.43:0/2373109034 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc59415df50 0x7fc5941619f0 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.000+0000 7fc59ad09640 1 -- 172.21.15.43:0/2373109034 >> 172.21.15.43:0/2373109034 conn(0x7fc5940d3a30 msgr2=0x7fc5940d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:39.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.000+0000 7fc59ad09640 1 -- 172.21.15.43:0/2373109034 shutdown_connections 2024-04-25T09:04:39.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.000+0000 7fc59ad09640 1 -- 172.21.15.43:0/2373109034 wait complete. 2024-04-25T09:04:39.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.001+0000 7fc59ad09640 1 Processor -- start 2024-04-25T09:04:39.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.001+0000 7fc59ad09640 1 -- start start 2024-04-25T09:04:39.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.002+0000 7fc59ad09640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc5940c7cf0 0x7fc5941f38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.002+0000 7fc59ad09640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc59415df50 0x7fc594161350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.002+0000 7fc59ad09640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc594161f30 0x7fc594161890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.002+0000 7fc59ad09640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5940d2190 con 0x7fc59415df50 2024-04-25T09:04:39.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.002+0000 7fc59ad09640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5940c77d0 con 0x7fc594161f30 2024-04-25T09:04:39.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.002+0000 7fc59ad09640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5940c8640 con 0x7fc5940c7cf0 2024-04-25T09:04:39.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.002+0000 7fc593fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc59415df50 0x7fc594161350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.002+0000 7fc593fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc59415df50 0x7fc594161350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59838/0 (socket says 172.21.15.43:59838) 2024-04-25T09:04:39.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.002+0000 7fc593fff640 1 -- 172.21.15.43:0/4236336914 learned_addr learned my addr 172.21.15.43:0/4236336914 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:39.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.003+0000 7fc598aa5640 1 --2- 172.21.15.43:0/4236336914 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc5940c7cf0 0x7fc5941f38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.003+0000 7fc5992a6640 1 --2- 172.21.15.43:0/4236336914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc594161f30 0x7fc594161890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.003+0000 7fc593fff640 1 -- 172.21.15.43:0/4236336914 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc5940c7cf0 msgr2=0x7fc5941f38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.993 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.003+0000 7fc593fff640 1 --2- 172.21.15.43:0/4236336914 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc5940c7cf0 0x7fc5941f38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.003+0000 7fc593fff640 1 -- 172.21.15.43:0/4236336914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc594161f30 msgr2=0x7fc594161890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.003+0000 7fc593fff640 1 --2- 172.21.15.43:0/4236336914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc594161f30 0x7fc594161890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.003+0000 7fc593fff640 1 -- 172.21.15.43:0/4236336914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc594154410 con 0x7fc59415df50 2024-04-25T09:04:39.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.003+0000 7fc593fff640 1 --2- 172.21.15.43:0/4236336914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc59415df50 0x7fc594161350 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7fc58800c7f0 tx=0x7fc58800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:39.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.003+0000 7fc5917fa640 1 -- 172.21.15.43:0/4236336914 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc588015070 con 0x7fc59415df50 2024-04-25T09:04:39.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.003+0000 7fc5917fa640 1 -- 172.21.15.43:0/4236336914 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc588004220 con 0x7fc59415df50 2024-04-25T09:04:39.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.003+0000 7fc5917fa640 1 -- 172.21.15.43:0/4236336914 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc588007620 con 0x7fc59415df50 2024-04-25T09:04:39.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.003+0000 7fc59ad09640 1 -- 172.21.15.43:0/4236336914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc594154130 con 0x7fc59415df50 2024-04-25T09:04:39.994 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.003+0000 7fc59ad09640 1 -- 172.21.15.43:0/4236336914 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc59409daf0 con 0x7fc59415df50 2024-04-25T09:04:39.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.005+0000 7fc5917fa640 1 -- 172.21.15.43:0/4236336914 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc5880079d0 con 0x7fc59415df50 2024-04-25T09:04:39.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.005+0000 7fc5917fa640 1 --2- 172.21.15.43:0/4236336914 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc58006b4d0 0x7fc58006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.006+0000 7fc5917fa640 1 -- 172.21.15.43:0/4236336914 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc58808a070 con 0x7fc59415df50 2024-04-25T09:04:39.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.006+0000 7fc598aa5640 1 --2- 172.21.15.43:0/4236336914 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc58006b4d0 0x7fc58006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.006+0000 7fc59da11740 1 Processor -- start 2024-04-25T09:04:39.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.006+0000 7fc59da11740 1 -- start start 2024-04-25T09:04:39.997 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.006+0000 7fc598aa5640 1 --2- 172.21.15.43:0/4236336914 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc58006b4d0 0x7fc58006d990 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fc58c033000 tx=0x7fc58c009000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:39.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.007+0000 7fc59da11740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567689f3230 0x5567689f3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.007+0000 7fc59da11740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556768a10ad0 0x556768a11020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.007+0000 7fc59da11740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567689c8f10 0x556768a0f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:39.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.007+0000 7fc59da11740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x55676892ba10 con 0x5567689c8f10 2024-04-25T09:04:39.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.007+0000 7fc59da11740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567688d4b30 con 0x5567689f3230 2024-04-25T09:04:39.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.007+0000 7fc59da11740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x556768927c50 con 0x556768a10ad0 2024-04-25T09:04:39.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.007+0000 7fc5992a6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567689c8f10 0x556768a0f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.007+0000 7fc5992a6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567689c8f10 0x556768a0f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59842/0 (socket says 172.21.15.43:59842) 2024-04-25T09:04:39.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.007+0000 7fc5992a6640 1 -- 172.21.15.43:0/2262955032 learned_addr learned my addr 172.21.15.43:0/2262955032 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:39.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.007+0000 7fc598aa5640 1 --2- 172.21.15.43:0/2262955032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567689f3230 0x5567689f3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.007+0000 7fc593fff640 1 --2- 172.21.15.43:0/2262955032 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556768a10ad0 0x556768a11020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:39.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.007+0000 7fc5992a6640 1 -- 172.21.15.43:0/2262955032 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556768a10ad0 msgr2=0x556768a11020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:39.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.007+0000 7fc5992a6640 1 --2- 172.21.15.43:0/2262955032 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556768a10ad0 0x556768a11020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.998 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.007+0000 7fc5992a6640 1 -- 172.21.15.43:0/2262955032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567689f3230 msgr2=0x5567689f3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:39.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.007+0000 7fc5992a6640 1 --2- 172.21.15.43:0/2262955032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567689f3230 0x5567689f3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.007+0000 7fc5992a6640 1 -- 172.21.15.43:0/2262955032 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567688e9e10 con 0x5567689c8f10 2024-04-25T09:04:39.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.008+0000 7fc5992a6640 1 --2- 172.21.15.43:0/2262955032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567689c8f10 0x556768a0f190 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7fc58400ab10 tx=0x7fc58400f240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0bf6b000b19b715 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:39.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.008+0000 7fc57a7fc640 1 -- 172.21.15.43:0/2262955032 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58400fe50 con 0x5567689c8f10 2024-04-25T09:04:39.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.008+0000 7fc57a7fc640 1 -- 172.21.15.43:0/2262955032 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc584013440 con 0x5567689c8f10 2024-04-25T09:04:39.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.008+0000 7fc57a7fc640 1 -- 172.21.15.43:0/2262955032 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5840137f0 con 0x5567689c8f10 2024-04-25T09:04:39.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.008+0000 7fc59da11740 1 -- 172.21.15.43:0/2262955032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567689c8f10 msgr2=0x556768a0f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:39.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.008+0000 7fc59da11740 1 --2- 172.21.15.43:0/2262955032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567689c8f10 0x556768a0f190 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7fc58400ab10 tx=0x7fc58400f240 comp rx=0 tx=0).stop 2024-04-25T09:04:39.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.008+0000 7fc59da11740 1 -- 172.21.15.43:0/2262955032 shutdown_connections 2024-04-25T09:04:39.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.008+0000 7fc59da11740 1 --2- 172.21.15.43:0/2262955032 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567689c8f10 0x556768a0f190 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.008+0000 7fc59da11740 1 --2- 172.21.15.43:0/2262955032 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556768a10ad0 0x556768a11020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.008+0000 7fc59da11740 1 --2- 172.21.15.43:0/2262955032 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x5567689f3230 0x5567689f3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:39.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.008+0000 7fc59da11740 1 -- 172.21.15.43:0/2262955032 >> 172.21.15.43:0/2262955032 conn(0x5567689c5840 msgr2=0x556768a0e040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:39.999 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.008+0000 7fc59da11740 1 -- 172.21.15.43:0/2262955032 shutdown_connections 2024-04-25T09:04:40.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.008+0000 7fc59da11740 1 -- 172.21.15.43:0/2262955032 wait complete. 2024-04-25T09:04:40.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.009+0000 7fc59da11740 1 -- start start 2024-04-25T09:04:40.000 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.009+0000 7fc59da11740 1 Processor -- start 2024-04-25T09:04:40.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.010+0000 7fc59da11740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5567689f3230 0x556768a1ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:40.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.010+0000 7fc59da11740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556768a10ad0 0x5567689c28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:40.003 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.010+0000 7fc59da11740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556768a21a50 0x5567689ef580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:40.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.010+0000 7fc59da11740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567689c9cb0 con 0x556768a10ad0 2024-04-25T09:04:40.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.010+0000 7fc59da11740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567688e80c0 con 0x556768a21a50 2024-04-25T09:04:40.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.010+0000 7fc59da11740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x5567689c3770 con 0x5567689f3230 2024-04-25T09:04:40.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.010+0000 7fc59da11740 10 client.0 _ll_register_callbacks cb 0x5567688dbec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:04:40.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.010+0000 7fc593fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556768a10ad0 0x5567689c28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:40.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.010+0000 7fc593fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556768a10ad0 0x5567689c28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59846/0 (socket says 172.21.15.43:59846) 2024-04-25T09:04:40.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.010+0000 7fc593fff640 1 -- 172.21.15.43:0/985554609 learned_addr learned my addr 172.21.15.43:0/985554609 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:40.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.010+0000 7fc593fff640 1 -- 172.21.15.43:0/985554609 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5567689f3230 msgr2=0x556768a1ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:40.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.010+0000 7fc5992a6640 1 --2- 172.21.15.43:0/985554609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556768a21a50 0x5567689ef580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:40.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.010+0000 7fc598aa5640 1 --2- 172.21.15.43:0/985554609 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5567689f3230 0x556768a1ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:40.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.010+0000 7fc593fff640 1 --2- 172.21.15.43:0/985554609 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5567689f3230 0x556768a1ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.010+0000 7fc593fff640 1 -- 172.21.15.43:0/985554609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556768a21a50 msgr2=0x5567689ef580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:40.004 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.010+0000 7fc593fff640 1 --2- 172.21.15.43:0/985554609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556768a21a50 0x5567689ef580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.010+0000 7fc593fff640 1 -- 172.21.15.43:0/985554609 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5567686402d0 con 0x556768a10ad0 2024-04-25T09:04:40.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.010+0000 7fc598aa5640 1 --2- 172.21.15.43:0/985554609 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5567689f3230 0x556768a1ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:40.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.010+0000 7fc5992a6640 1 --2- 172.21.15.43:0/985554609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556768a21a50 0x5567689ef580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:40.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.010+0000 7fc593fff640 1 --2- 172.21.15.43:0/985554609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556768a10ad0 0x5567689c28a0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7fc588054d40 tx=0x7fc588058a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=613c1e49dee209ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:40.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.011+0000 7fc5737fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:04:40.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.011+0000 7fc5737fe640 1 -- 172.21.15.43:0/985554609 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58801e040 con 0x556768a10ad0 2024-04-25T09:04:40.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.011+0000 7fc5737fe640 1 -- 172.21.15.43:0/985554609 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc588007cc0 con 0x556768a10ad0 2024-04-25T09:04:40.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.011+0000 7fc59da11740 1 -- 172.21.15.43:0/985554609 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc584013440 con 0x556768a10ad0 2024-04-25T09:04:40.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.011+0000 7fc5737fe640 1 -- 172.21.15.43:0/985554609 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc588075050 con 0x556768a10ad0 2024-04-25T09:04:40.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.011+0000 7fc59da11740 10 client.9945 fetch_fsmap learned FSMap version 90 2024-04-25T09:04:40.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.011+0000 7fc59da11740 1 -- 172.21.15.43:0/985554609 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x55676870a6b0 con 0x556768a10ad0 2024-04-25T09:04:40.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.011+0000 7fc5737fe640 1 -- 172.21.15.43:0/985554609 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc584013440 con 0x556768a10ad0 2024-04-25T09:04:40.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.011+0000 7fc5737fe640 1 -- 172.21.15.43:0/985554609 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 90) v1 ==== 3282+0+0 (secure 0 0 0) 0x7fc5880772f0 con 0x556768a10ad0 2024-04-25T09:04:40.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.011+0000 7fc59da11740 10 client.9945 fetch_fsmap finished waiting for FSMap version 90 2024-04-25T09:04:40.005 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.011+0000 7fc59da11740 10 client.9945 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T09:04:40.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.011+0000 7fc59da11740 20 client.9945 populate_metadata read hostname 'smithi043' 2024-04-25T09:04:40.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.011+0000 7fc59da11740 1 --2- 172.21.15.43:0/985554609 >> [v2:172.21.15.193:6835/2585534546,v1:172.21.15.193:6837/2585534546] conn(0x556768a05560 0x556768a05940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:40.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.011+0000 7fc59da11740 4 client.9945 mds_command: new command op to 9732 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T09:04:40.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.011+0000 7fc59da11740 1 -- 172.21.15.43:0/985554609 --> [v2:172.21.15.193:6835/2585534546,v1:172.21.15.193:6837/2585534546] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55676898a920 con 0x556768a05560 2024-04-25T09:04:40.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.011+0000 7fc598aa5640 1 --2- 172.21.15.43:0/985554609 >> [v2:172.21.15.193:6835/2585534546,v1:172.21.15.193:6837/2585534546] conn(0x556768a05560 0x556768a05940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:40.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.012+0000 7fc5737fe640 10 client.9945 ms_handle_connect on v2:172.21.15.193:6835/2585534546 2024-04-25T09:04:40.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.012+0000 7fc598aa5640 1 --2- 172.21.15.43:0/985554609 >> [v2:172.21.15.193:6835/2585534546,v1:172.21.15.193:6837/2585534546] conn(0x556768a05560 0x556768a05940 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed829a00fc92cd0d server_cookie=27314b818de3c36f in_seq=0 out_seq=0 2024-04-25T09:04:40.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.012+0000 7fc5737fe640 1 -- 172.21.15.43:0/985554609 <== mds.0 v2:172.21.15.193:6835/2585534546 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55676898a920 con 0x556768a05560 2024-04-25T09:04:40.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.012+0000 7fc5737fe640 10 client.9945 handle_command_reply: tid=0 2024-04-25T09:04:40.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.013+0000 7fc59da11740 1 client.9945 shutdown 2024-04-25T09:04:40.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.013+0000 7fc59da11740 1 -- 172.21.15.43:0/985554609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556768a10ad0 msgr2=0x5567689c28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:40.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.013+0000 7fc59da11740 1 --2- 172.21.15.43:0/985554609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556768a10ad0 0x5567689c28a0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7fc588054d40 tx=0x7fc588058a60 comp rx=0 tx=0).stop 2024-04-25T09:04:40.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.013+0000 7fc59da11740 1 -- 172.21.15.43:0/985554609 shutdown_connections 2024-04-25T09:04:40.006 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.013+0000 7fc59da11740 1 --2- 172.21.15.43:0/985554609 >> [v2:172.21.15.193:6835/2585534546,v1:172.21.15.193:6837/2585534546] conn(0x556768a05560 0x556768a05940 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.013+0000 7fc59da11740 1 --2- 172.21.15.43:0/985554609 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556768a21a50 0x5567689ef580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.013+0000 7fc59da11740 1 --2- 172.21.15.43:0/985554609 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556768a10ad0 0x5567689c28a0 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.013+0000 7fc5737fe640 0 client.9945 ms_handle_reset on v2:172.21.15.193:6835/2585534546 2024-04-25T09:04:40.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.013+0000 7fc59da11740 1 --2- 172.21.15.43:0/985554609 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5567689f3230 0x556768a1ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.013+0000 7fc59da11740 1 -- 172.21.15.43:0/985554609 >> 172.21.15.43:0/985554609 conn(0x5567689c8f10 msgr2=0x556768a115c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:40.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.013+0000 7fc59da11740 1 -- 172.21.15.43:0/985554609 shutdown_connections 2024-04-25T09:04:40.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.013+0000 7fc59da11740 1 -- 172.21.15.43:0/985554609 wait complete. 2024-04-25T09:04:40.007 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.013+0000 7fc59da11740 20 client.9945 trim_cache size 0 max 16384 2024-04-25T09:04:40.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.029+0000 7fc59da11740 1 Processor -- start 2024-04-25T09:04:40.017 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.029+0000 7fc59da11740 1 -- start start 2024-04-25T09:04:40.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.030+0000 7fc59da11740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5567689f3230 0x556768a10570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:40.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.030+0000 7fc59da11740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556768a10ad0 0x556768a0e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:40.019 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.030+0000 7fc59da11740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556768a21a50 0x556768a0e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:40.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.030+0000 7fc59da11740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5941ff780 con 0x556768a10ad0 2024-04-25T09:04:40.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.030+0000 7fc59da11740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5941ffda0 con 0x556768a21a50 2024-04-25T09:04:40.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.030+0000 7fc59da11740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc594160230 con 0x5567689f3230 2024-04-25T09:04:40.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.030+0000 7fc593fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556768a10ad0 0x556768a0e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:40.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.030+0000 7fc593fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556768a10ad0 0x556768a0e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59860/0 (socket says 172.21.15.43:59860) 2024-04-25T09:04:40.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.030+0000 7fc593fff640 1 -- 172.21.15.43:0/4073817907 learned_addr learned my addr 172.21.15.43:0/4073817907 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:40.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.030+0000 7fc593fff640 1 -- 172.21.15.43:0/4073817907 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5567689f3230 msgr2=0x556768a10570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:40.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.030+0000 7fc593fff640 1 --2- 172.21.15.43:0/4073817907 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5567689f3230 0x556768a10570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.030+0000 7fc593fff640 1 -- 172.21.15.43:0/4073817907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556768a21a50 msgr2=0x556768a0e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:40.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.030+0000 7fc593fff640 1 --2- 172.21.15.43:0/4073817907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556768a21a50 0x556768a0e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.030+0000 7fc593fff640 1 -- 172.21.15.43:0/4073817907 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556768a088c0 con 0x556768a10ad0 2024-04-25T09:04:40.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.030+0000 7fc593fff640 1 --2- 172.21.15.43:0/4073817907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556768a10ad0 0x556768a0e040 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7fc588060a60 tx=0x7fc58805b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d6bf03ffd288bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:40.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.030+0000 7fc579ffb640 1 -- 172.21.15.43:0/4073817907 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58801e070 con 0x556768a10ad0 2024-04-25T09:04:40.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.030+0000 7fc579ffb640 1 -- 172.21.15.43:0/4073817907 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc588086070 con 0x556768a10ad0 2024-04-25T09:04:40.020 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.030+0000 7fc579ffb640 1 -- 172.21.15.43:0/4073817907 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc588057f90 con 0x556768a10ad0 2024-04-25T09:04:40.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.031+0000 7fc59da11740 1 -- 172.21.15.43:0/4073817907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556768a10ad0 msgr2=0x556768a0e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:40.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.031+0000 7fc59da11740 1 --2- 172.21.15.43:0/4073817907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556768a10ad0 0x556768a0e040 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7fc588060a60 tx=0x7fc58805b000 comp rx=0 tx=0).stop 2024-04-25T09:04:40.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.031+0000 7fc59da11740 1 -- 172.21.15.43:0/4073817907 shutdown_connections 2024-04-25T09:04:40.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.031+0000 7fc59da11740 1 --2- 172.21.15.43:0/4073817907 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556768a21a50 0x556768a0e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.031+0000 7fc59da11740 1 --2- 172.21.15.43:0/4073817907 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x556768a10ad0 0x556768a0e040 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.031+0000 7fc59da11740 1 --2- 172.21.15.43:0/4073817907 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x5567689f3230 0x556768a10570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.031+0000 7fc59da11740 1 -- 172.21.15.43:0/4073817907 >> 172.21.15.43:0/4073817907 conn(0x5567689c8f10 msgr2=0x556768a1ede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:40.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.031+0000 7fc59da11740 1 -- 172.21.15.43:0/4073817907 shutdown_connections 2024-04-25T09:04:40.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.031+0000 7fc59da11740 1 -- 172.21.15.43:0/4073817907 wait complete. 2024-04-25T09:04:40.021 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.031+0000 7fc59da11740 1 -- start start 2024-04-25T09:04:40.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.032+0000 7fc59da11740 1 Processor -- start 2024-04-25T09:04:40.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.032+0000 7fc59da11740 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567689f3230 0x5567689f4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:40.025 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.032+0000 7fc59da11740 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556768a10ad0 0x556768a0e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:40.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.032+0000 7fc59da11740 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556768a21a50 0x556768a0e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:40.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.032+0000 7fc59da11740 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x556768a22820 con 0x5567689f3230 2024-04-25T09:04:40.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.032+0000 7fc59da11740 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5941fe9f0 con 0x556768a21a50 2024-04-25T09:04:40.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.032+0000 7fc59da11740 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x556768a06d30 con 0x556768a10ad0 2024-04-25T09:04:40.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.032+0000 7fc59da11740 10 client.0 _ll_register_callbacks cb 0x5567689c3c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T09:04:40.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.032+0000 7fc598aa5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567689f3230 0x5567689f4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:40.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.032+0000 7fc598aa5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567689f3230 0x5567689f4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59866/0 (socket says 172.21.15.43:59866) 2024-04-25T09:04:40.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.032+0000 7fc598aa5640 1 -- 172.21.15.43:0/2347083136 learned_addr learned my addr 172.21.15.43:0/2347083136 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:40.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.033+0000 7fc593fff640 1 --2- 172.21.15.43:0/2347083136 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556768a10ad0 0x556768a0e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:40.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.033+0000 7fc5992a6640 1 --2- 172.21.15.43:0/2347083136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556768a21a50 0x556768a0e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:40.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.033+0000 7fc598aa5640 1 -- 172.21.15.43:0/2347083136 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556768a10ad0 msgr2=0x556768a0e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:40.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.033+0000 7fc598aa5640 1 --2- 172.21.15.43:0/2347083136 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556768a10ad0 0x556768a0e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.033+0000 7fc598aa5640 1 -- 172.21.15.43:0/2347083136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556768a21a50 msgr2=0x556768a0e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:40.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.033+0000 7fc598aa5640 1 --2- 172.21.15.43:0/2347083136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556768a21a50 0x556768a0e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.026 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.033+0000 7fc598aa5640 1 -- 172.21.15.43:0/2347083136 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556768a08d60 con 0x5567689f3230 2024-04-25T09:04:40.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.033+0000 7fc5992a6640 1 --2- 172.21.15.43:0/2347083136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556768a21a50 0x556768a0e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:40.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.033+0000 7fc598aa5640 1 --2- 172.21.15.43:0/2347083136 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567689f3230 0x5567689f4900 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7fc58c034040 tx=0x7fc58c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=170c4aa1f3bed21f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:40.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.033+0000 7fc5737fe640 10 client.0 ms_handle_connect on v2:172.21.15.43:3300/0 2024-04-25T09:04:40.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.033+0000 7fc5737fe640 1 -- 172.21.15.43:0/2347083136 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58c036040 con 0x5567689f3230 2024-04-25T09:04:40.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.033+0000 7fc5737fe640 1 -- 172.21.15.43:0/2347083136 <== mon.0 v2:172.21.15.43:3300/0 2 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc58c03cb90 con 0x5567689f3230 2024-04-25T09:04:40.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.033+0000 7fc59da11740 1 -- 172.21.15.43:0/2347083136 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc588086070 con 0x5567689f3230 2024-04-25T09:04:40.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.033+0000 7fc5737fe640 1 -- 172.21.15.43:0/2347083136 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc58c059050 con 0x5567689f3230 2024-04-25T09:04:40.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.033+0000 7fc59da11740 10 client.9951 fetch_fsmap learned FSMap version 90 2024-04-25T09:04:40.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.033+0000 7fc59da11740 1 -- 172.21.15.43:0/2347083136 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x556768a79700 con 0x5567689f3230 2024-04-25T09:04:40.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.033+0000 7fc5737fe640 1 -- 172.21.15.43:0/2347083136 <== mon.0 v2:172.21.15.43:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc588086070 con 0x5567689f3230 2024-04-25T09:04:40.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.033+0000 7fc5737fe640 1 -- 172.21.15.43:0/2347083136 <== mon.0 v2:172.21.15.43:3300/0 5 ==== fsmap(e 90) v1 ==== 3282+0+0 (secure 0 0 0) 0x7fc58c05c020 con 0x5567689f3230 2024-04-25T09:04:40.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.033+0000 7fc59da11740 10 client.9951 fetch_fsmap finished waiting for FSMap version 90 2024-04-25T09:04:40.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.034+0000 7fc59da11740 10 client.9951 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T09:04:40.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.034+0000 7fc59da11740 20 client.9951 populate_metadata read hostname 'smithi043' 2024-04-25T09:04:40.027 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.034+0000 7fc59da11740 1 --2- 172.21.15.43:0/2347083136 >> [v2:172.21.15.193:6835/2585534546,v1:172.21.15.193:6837/2585534546] conn(0x556768a785c0 0x556768a789a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:40.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.034+0000 7fc59da11740 4 client.9951 mds_command: new command op to 9732 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T09:04:40.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.034+0000 7fc59da11740 1 -- 172.21.15.43:0/2347083136 --> [v2:172.21.15.193:6835/2585534546,v1:172.21.15.193:6837/2585534546] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5567689f3b30 con 0x556768a785c0 2024-04-25T09:04:40.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.034+0000 7fc593fff640 1 --2- 172.21.15.43:0/2347083136 >> [v2:172.21.15.193:6835/2585534546,v1:172.21.15.193:6837/2585534546] conn(0x556768a785c0 0x556768a789a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:40.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.034+0000 7fc5737fe640 10 client.9951 ms_handle_connect on v2:172.21.15.193:6835/2585534546 2024-04-25T09:04:40.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.034+0000 7fc593fff640 1 --2- 172.21.15.43:0/2347083136 >> [v2:172.21.15.193:6835/2585534546,v1:172.21.15.193:6837/2585534546] conn(0x556768a785c0 0x556768a789a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39814af7c8bd51f9 server_cookie=9f6b65a20a0ff78b in_seq=0 out_seq=0 2024-04-25T09:04:40.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.034+0000 7fc5737fe640 1 -- 172.21.15.43:0/2347083136 <== mds.0 v2:172.21.15.193:6835/2585534546 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x5567689f3b30 con 0x556768a785c0 2024-04-25T09:04:40.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.034+0000 7fc5737fe640 10 client.9951 handle_command_reply: tid=0 2024-04-25T09:04:40.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.034+0000 7fc59da11740 1 client.9951 shutdown 2024-04-25T09:04:40.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.034+0000 7fc59da11740 1 -- 172.21.15.43:0/2347083136 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567689f3230 msgr2=0x5567689f4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:40.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.034+0000 7fc59da11740 1 --2- 172.21.15.43:0/2347083136 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567689f3230 0x5567689f4900 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7fc58c034040 tx=0x7fc58c004ab0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.035+0000 7fc59da11740 1 -- 172.21.15.43:0/2347083136 shutdown_connections 2024-04-25T09:04:40.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.035+0000 7fc59da11740 1 --2- 172.21.15.43:0/2347083136 >> [v2:172.21.15.193:6835/2585534546,v1:172.21.15.193:6837/2585534546] conn(0x556768a785c0 0x556768a789a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.035+0000 7fc59da11740 1 --2- 172.21.15.43:0/2347083136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x556768a21a50 0x556768a0e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.028 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.035+0000 7fc59da11740 1 --2- 172.21.15.43:0/2347083136 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x556768a10ad0 0x556768a0e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.035+0000 7fc5737fe640 0 client.9951 ms_handle_reset on v2:172.21.15.193:6835/2585534546 2024-04-25T09:04:40.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.035+0000 7fc59da11740 1 --2- 172.21.15.43:0/2347083136 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x5567689f3230 0x5567689f4900 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.035+0000 7fc59da11740 1 -- 172.21.15.43:0/2347083136 >> 172.21.15.43:0/2347083136 conn(0x5567689c8f10 msgr2=0x5567689ef280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:40.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.035+0000 7fc59da11740 1 -- 172.21.15.43:0/2347083136 shutdown_connections 2024-04-25T09:04:40.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.035+0000 7fc59da11740 1 -- 172.21.15.43:0/2347083136 wait complete. 2024-04-25T09:04:40.029 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.035+0000 7fc59da11740 20 client.9951 trim_cache size 0 max 16384 2024-04-25T09:04:40.029 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:04:40.029 INFO:teuthology.orchestra.run.smithi043.stdout: "cluster_fsid": "6c4bd50e-aa00-4a7f-8d09-352344ef5a4b", 2024-04-25T09:04:40.029 INFO:teuthology.orchestra.run.smithi043.stdout: "whoami": 0, 2024-04-25T09:04:40.029 INFO:teuthology.orchestra.run.smithi043.stdout: "id": 9732, 2024-04-25T09:04:40.029 INFO:teuthology.orchestra.run.smithi043.stdout: "want_state": "up:active", 2024-04-25T09:04:40.030 INFO:teuthology.orchestra.run.smithi043.stdout: "state": "up:active", 2024-04-25T09:04:40.030 INFO:teuthology.orchestra.run.smithi043.stdout: "fs_name": "cephfs", 2024-04-25T09:04:40.030 INFO:teuthology.orchestra.run.smithi043.stdout: "rank_uptime": 25.963869675000002, 2024-04-25T09:04:40.030 INFO:teuthology.orchestra.run.smithi043.stdout: "mdsmap_epoch": 90, 2024-04-25T09:04:40.030 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch": 139, 2024-04-25T09:04:40.030 INFO:teuthology.orchestra.run.smithi043.stdout: "osdmap_epoch_barrier": 139, 2024-04-25T09:04:40.030 INFO:teuthology.orchestra.run.smithi043.stdout: "uptime": 30.986970749000001 2024-04-25T09:04:40.030 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:04:40.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.035+0000 7fc59ad09640 1 -- 172.21.15.43:0/4236336914 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc58006b4d0 msgr2=0x7fc58006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:40.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.035+0000 7fc59ad09640 1 --2- 172.21.15.43:0/4236336914 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc58006b4d0 0x7fc58006d990 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fc58c033000 tx=0x7fc58c009000 comp rx=0 tx=0).stop 2024-04-25T09:04:40.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.035+0000 7fc59ad09640 1 -- 172.21.15.43:0/4236336914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc59415df50 msgr2=0x7fc594161350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:40.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.035+0000 7fc59ad09640 1 --2- 172.21.15.43:0/4236336914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc59415df50 0x7fc594161350 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7fc58800c7f0 tx=0x7fc58800ccb0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.035+0000 7fc59ad09640 1 -- 172.21.15.43:0/4236336914 shutdown_connections 2024-04-25T09:04:40.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.035+0000 7fc59ad09640 1 --2- 172.21.15.43:0/4236336914 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc58006b4d0 0x7fc58006d990 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.035+0000 7fc59ad09640 1 --2- 172.21.15.43:0/4236336914 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc594161f30 0x7fc594161890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.035+0000 7fc59ad09640 1 --2- 172.21.15.43:0/4236336914 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc59415df50 0x7fc594161350 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.035+0000 7fc59ad09640 1 --2- 172.21.15.43:0/4236336914 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc5940c7cf0 0x7fc5941f38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.035+0000 7fc59ad09640 1 -- 172.21.15.43:0/4236336914 >> 172.21.15.43:0/4236336914 conn(0x7fc5940d3a30 msgr2=0x7fc5940cd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:40.031 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.035+0000 7fc59ad09640 1 -- 172.21.15.43:0/4236336914 shutdown_connections 2024-04-25T09:04:40.032 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.035+0000 7fc59ad09640 1 -- 172.21.15.43:0/4236336914 wait complete. 2024-04-25T09:04:40.034 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T09:04:40.034 DEBUG:teuthology.orchestra.run.smithi043:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:04:40.076 DEBUG:teuthology.orchestra.run.smithi193:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T09:04:40.117 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T09:04:40.117 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T09:04:40.117 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:04:40.133 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:04:40.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.286+0000 7fb8aa076640 1 Processor -- start 2024-04-25T09:04:40.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.286+0000 7fb8aa076640 1 -- start start 2024-04-25T09:04:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.287+0000 7fb8aa076640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb8a415dce0 0x7fb8a415ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.287+0000 7fb8aa076640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fb8a40ce550 0x7fb8a40d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.287+0000 7fb8aa076640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8a40cee80 con 0x7fb8a40ce9c0 2024-04-25T09:04:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.287+0000 7fb8aa076640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8a40d2920 con 0x7fb8a415dce0 2024-04-25T09:04:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.287+0000 7fb8aa076640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8a40cf160 con 0x7fb8a40ce550 2024-04-25T09:04:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.287+0000 7fb8a37fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb8a415dce0 0x7fb8a415ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.287+0000 7fb8a3fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb8a40ce9c0 0x7fb8a4160980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44570/0 (socket says 172.21.15.43:44570) 2024-04-25T09:04:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.287+0000 7fb8a3fff640 1 -- 172.21.15.43:0/296826061 learned_addr learned my addr 172.21.15.43:0/296826061 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.287+0000 7fb8a2ffd640 1 --2- 172.21.15.43:0/296826061 >> v2:172.21.15.193:3300/0 conn(0x7fb8a40ce550 0x7fb8a40d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.287+0000 7fb8a37fe640 1 -- 172.21.15.43:0/296826061 >> v2:172.21.15.193:3300/0 conn(0x7fb8a40ce550 msgr2=0x7fb8a40d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.287+0000 7fb8a37fe640 1 --2- 172.21.15.43:0/296826061 >> v2:172.21.15.193:3300/0 conn(0x7fb8a40ce550 0x7fb8a40d03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.287+0000 7fb8a37fe640 1 -- 172.21.15.43:0/296826061 >> v1:172.21.15.43:6789/0 conn(0x7fb8a40ce9c0 legacy=0x7fb8a4160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:40.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.287+0000 7fb8a27fc640 1 -- 172.21.15.43:0/296826061 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1806622124 0 0) 0x7fb8a40cee80 con 0x7fb8a40ce9c0 2024-04-25T09:04:40.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.287+0000 7fb8a37fe640 1 -- 172.21.15.43:0/296826061 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a40cf440 con 0x7fb8a415dce0 2024-04-25T09:04:40.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.288+0000 7fb8a37fe640 1 --2- 172.21.15.43:0/296826061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a415dce0 0x7fb8a415ef50 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7fb8a406afa0 tx=0x7fb89402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b92ed0127c0a4df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:40.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.288+0000 7fb8a27fc640 1 -- 172.21.15.43:0/296826061 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb894037070 con 0x7fb8a415dce0 2024-04-25T09:04:40.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.288+0000 7fb8a27fc640 1 -- 172.21.15.43:0/296826061 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb894032440 con 0x7fb8a415dce0 2024-04-25T09:04:40.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.288+0000 7fb8a27fc640 1 -- 172.21.15.43:0/296826061 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb89402ae60 con 0x7fb8a415dce0 2024-04-25T09:04:40.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.289+0000 7fb8aa076640 1 -- 172.21.15.43:0/296826061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a415dce0 msgr2=0x7fb8a415ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:40.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.289+0000 7fb8aa076640 1 --2- 172.21.15.43:0/296826061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a415dce0 0x7fb8a415ef50 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7fb8a406afa0 tx=0x7fb89402a110 comp rx=0 tx=0).stop 2024-04-25T09:04:40.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.289+0000 7fb8aa076640 1 -- 172.21.15.43:0/296826061 shutdown_connections 2024-04-25T09:04:40.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.289+0000 7fb8aa076640 1 --2- 172.21.15.43:0/296826061 >> v2:172.21.15.193:3300/0 conn(0x7fb8a40ce550 0x7fb8a40d03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.289+0000 7fb8aa076640 1 --2- 172.21.15.43:0/296826061 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a415dce0 0x7fb8a415ef50 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.289+0000 7fb8aa076640 1 -- 172.21.15.43:0/296826061 >> 172.21.15.43:0/296826061 conn(0x7fb8a409cd00 msgr2=0x7fb8a40d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:40.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.289+0000 7fb8aa076640 1 -- 172.21.15.43:0/296826061 shutdown_connections 2024-04-25T09:04:40.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.289+0000 7fb8aa076640 1 -- 172.21.15.43:0/296826061 wait complete. 2024-04-25T09:04:40.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.290+0000 7fb8aa076640 1 Processor -- start 2024-04-25T09:04:40.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.290+0000 7fb8aa076640 1 -- start start 2024-04-25T09:04:40.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.291+0000 7fb8aa076640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8a40ce550 0x7fb8a409e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:40.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.291+0000 7fb8aa076640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a40ce9c0 0x7fb8a40b7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:40.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.291+0000 7fb8aa076640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb8a415dce0 0x7fb8a40b7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:40.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.291+0000 7fb8aa076640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a415e0c0 con 0x7fb8a40ce9c0 2024-04-25T09:04:40.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.291+0000 7fb8aa076640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a415f6b0 con 0x7fb8a40ce550 2024-04-25T09:04:40.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.291+0000 7fb8aa076640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a40d0920 con 0x7fb8a415dce0 2024-04-25T09:04:40.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.291+0000 7fb8a2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a40ce9c0 0x7fb8a40b7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:40.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.291+0000 7fb8a2ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a40ce9c0 0x7fb8a40b7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59878/0 (socket says 172.21.15.43:59878) 2024-04-25T09:04:40.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.291+0000 7fb8a2ffd640 1 -- 172.21.15.43:0/4132092812 learned_addr learned my addr 172.21.15.43:0/4132092812 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:40.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.291+0000 7fb8a37fe640 1 --2- 172.21.15.43:0/4132092812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8a40ce550 0x7fb8a409e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:40.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.291+0000 7fb8a3fff640 1 --2- 172.21.15.43:0/4132092812 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb8a415dce0 0x7fb8a40b7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:40.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.291+0000 7fb8a2ffd640 1 -- 172.21.15.43:0/4132092812 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb8a415dce0 msgr2=0x7fb8a40b7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:40.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.291+0000 7fb8a2ffd640 1 --2- 172.21.15.43:0/4132092812 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb8a415dce0 0x7fb8a40b7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.291+0000 7fb8a2ffd640 1 -- 172.21.15.43:0/4132092812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8a40ce550 msgr2=0x7fb8a409e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:40.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.291+0000 7fb8a2ffd640 1 --2- 172.21.15.43:0/4132092812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8a40ce550 0x7fb8a409e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.291+0000 7fb8a2ffd640 1 -- 172.21.15.43:0/4132092812 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a40b81a0 con 0x7fb8a40ce9c0 2024-04-25T09:04:40.282 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.291+0000 7fb8a2ffd640 1 --2- 172.21.15.43:0/4132092812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a40ce9c0 0x7fb8a40b7720 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fb88c00e9b0 tx=0x7fb88c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:40.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.292+0000 7fb89bfff640 1 -- 172.21.15.43:0/4132092812 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88c017070 con 0x7fb8a40ce9c0 2024-04-25T09:04:40.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.292+0000 7fb89bfff640 1 -- 172.21.15.43:0/4132092812 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb88c00c9a0 con 0x7fb8a40ce9c0 2024-04-25T09:04:40.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.292+0000 7fb8aa076640 1 -- 172.21.15.43:0/4132092812 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8a40b6960 con 0x7fb8a40ce9c0 2024-04-25T09:04:40.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.292+0000 7fb89bfff640 1 -- 172.21.15.43:0/4132092812 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88c00cd90 con 0x7fb8a40ce9c0 2024-04-25T09:04:40.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.292+0000 7fb8aa076640 1 -- 172.21.15.43:0/4132092812 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8a40b6f20 con 0x7fb8a40ce9c0 2024-04-25T09:04:40.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.293+0000 7fb8aa076640 1 -- 172.21.15.43:0/4132092812 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8a40680f0 con 0x7fb8a40ce9c0 2024-04-25T09:04:40.283 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.293+0000 7fb89bfff640 1 -- 172.21.15.43:0/4132092812 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb88c01a070 con 0x7fb8a40ce9c0 2024-04-25T09:04:40.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.294+0000 7fb89bfff640 1 --2- 172.21.15.43:0/4132092812 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb88406b4d0 0x7fb88406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:40.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.294+0000 7fb89bfff640 1 -- 172.21.15.43:0/4132092812 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb88c08a0e0 con 0x7fb8a40ce9c0 2024-04-25T09:04:40.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.294+0000 7fb8a37fe640 1 --2- 172.21.15.43:0/4132092812 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb88406b4d0 0x7fb88406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:40.284 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.295+0000 7fb8a37fe640 1 --2- 172.21.15.43:0/4132092812 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb88406b4d0 0x7fb88406d990 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fb894034000 tx=0x7fb89402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:40.286 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.298+0000 7fb89bfff640 1 -- 172.21.15.43:0/4132092812 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb88c058910 con 0x7fb8a40ce9c0 2024-04-25T09:04:40.470 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.482+0000 7fb8aa076640 1 -- 172.21.15.43:0/4132092812 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb8a4074ef0 con 0x7fb8a40ce9c0 2024-04-25T09:04:40.471 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.483+0000 7fb89bfff640 1 -- 172.21.15.43:0/4132092812 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+16643 (secure 0 0 0) 0x7fb88c05c9b0 con 0x7fb8a40ce9c0 2024-04-25T09:04:40.472 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:04:40.472 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":139,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:04:15.059128+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:04:11.735785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T09:04:12.383453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:04:40.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.486+0000 7fb8aa076640 1 -- 172.21.15.43:0/4132092812 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb88406b4d0 msgr2=0x7fb88406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:40.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.486+0000 7fb8aa076640 1 --2- 172.21.15.43:0/4132092812 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb88406b4d0 0x7fb88406d990 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fb894034000 tx=0x7fb89402e000 comp rx=0 tx=0).stop 2024-04-25T09:04:40.474 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.486+0000 7fb8aa076640 1 -- 172.21.15.43:0/4132092812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a40ce9c0 msgr2=0x7fb8a40b7720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:40.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.486+0000 7fb8aa076640 1 --2- 172.21.15.43:0/4132092812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a40ce9c0 0x7fb8a40b7720 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fb88c00e9b0 tx=0x7fb88c00ee70 comp rx=0 tx=0).stop 2024-04-25T09:04:40.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.486+0000 7fb8aa076640 1 -- 172.21.15.43:0/4132092812 shutdown_connections 2024-04-25T09:04:40.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.486+0000 7fb8aa076640 1 --2- 172.21.15.43:0/4132092812 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb8a415dce0 0x7fb8a40b7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.486+0000 7fb8aa076640 1 --2- 172.21.15.43:0/4132092812 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb8a40ce9c0 0x7fb8a40b7720 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.486+0000 7fb8aa076640 1 --2- 172.21.15.43:0/4132092812 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb88406b4d0 0x7fb88406d990 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.486+0000 7fb8aa076640 1 --2- 172.21.15.43:0/4132092812 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb8a40ce550 0x7fb8a409e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.486+0000 7fb8aa076640 1 -- 172.21.15.43:0/4132092812 >> 172.21.15.43:0/4132092812 conn(0x7fb8a409cd00 msgr2=0x7fb8a406c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:40.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.486+0000 7fb8aa076640 1 -- 172.21.15.43:0/4132092812 shutdown_connections 2024-04-25T09:04:40.475 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.486+0000 7fb8aa076640 1 -- 172.21.15.43:0/4132092812 wait complete. 2024-04-25T09:04:40.485 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:04:11.735785+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '138', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:04:12.383453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '139', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:04:40.485 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:04:40.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.600+0000 7f4fca041640 1 Processor -- start 2024-04-25T09:04:40.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.600+0000 7f4fca041640 1 -- start start 2024-04-25T09:04:40.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.601+0000 7f4fca041640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4fc406ed70 0x7f4fc406f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:40.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.601+0000 7f4fca041640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f4fc40d1090 0x7f4fc40c77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:40.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.601+0000 7f4fca041640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fc415bf00 con 0x7f4fc40cbda0 2024-04-25T09:04:40.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.601+0000 7f4fca041640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fc40cfdb0 con 0x7f4fc40d1090 2024-04-25T09:04:40.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.601+0000 7f4fca041640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc406f690 con 0x7f4fc406ed70 2024-04-25T09:04:40.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.601+0000 7f4fc37fe640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4fc406ed70 0x7f4fc406f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:40.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.601+0000 7f4fc3fff640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f4fc40d1090 0x7f4fc40c77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:40.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.601+0000 7f4fc2ffd640 1 --1- >> v1:172.21.15.193:6789/0 conn(0x7f4fc40cbda0 0x7f4fc40d0980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.193:6789/0 says I am v1:172.21.15.43:57490/0 (socket says 172.21.15.43:57490) 2024-04-25T09:04:40.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.601+0000 7f4fc37fe640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4fc406ed70 0x7f4fc406f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:48750/0 (socket says 172.21.15.43:48750) 2024-04-25T09:04:40.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.601+0000 7f4fc2ffd640 1 -- 172.21.15.43:0/3511214556 learned_addr learned my addr 172.21.15.43:0/3511214556 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:40.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.602+0000 7f4fc37fe640 1 -- 172.21.15.43:0/3511214556 >> v2:172.21.15.193:3300/0 conn(0x7f4fc40d1090 msgr2=0x7f4fc40c77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:40.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.602+0000 7f4fc37fe640 1 --2- 172.21.15.43:0/3511214556 >> v2:172.21.15.193:3300/0 conn(0x7f4fc40d1090 0x7f4fc40c77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.602+0000 7f4fc37fe640 1 -- 172.21.15.43:0/3511214556 >> v1:172.21.15.193:6789/0 conn(0x7f4fc40cbda0 legacy=0x7f4fc40d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:40.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.602+0000 7f4fc37fe640 1 -- 172.21.15.43:0/3511214556 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc40c6740 con 0x7f4fc406ed70 2024-04-25T09:04:40.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.603+0000 7f4fc37fe640 1 --2- 172.21.15.43:0/3511214556 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4fc406ed70 0x7f4fc406f150 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4fb40097a0 tx=0x7f4fb402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdb8be3c00c314da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:40.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.603+0000 7f4fc27fc640 1 -- 172.21.15.43:0/3511214556 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4037070 con 0x7f4fc406ed70 2024-04-25T09:04:40.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.603+0000 7f4fc27fc640 1 -- 172.21.15.43:0/3511214556 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fb4032440 con 0x7f4fc406ed70 2024-04-25T09:04:40.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.603+0000 7f4fc27fc640 1 -- 172.21.15.43:0/3511214556 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4032810 con 0x7f4fc406ed70 2024-04-25T09:04:40.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.603+0000 7f4fca041640 1 -- 172.21.15.43:0/3511214556 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4fc406ed70 msgr2=0x7f4fc406f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:40.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.603+0000 7f4fca041640 1 --2- 172.21.15.43:0/3511214556 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4fc406ed70 0x7f4fc406f150 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4fb40097a0 tx=0x7f4fb402a180 comp rx=0 tx=0).stop 2024-04-25T09:04:40.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.603+0000 7f4fca041640 1 -- 172.21.15.43:0/3511214556 shutdown_connections 2024-04-25T09:04:40.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.603+0000 7f4fca041640 1 --2- 172.21.15.43:0/3511214556 >> v2:172.21.15.193:3300/0 conn(0x7f4fc40d1090 0x7f4fc40c77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.603+0000 7f4fca041640 1 --2- 172.21.15.43:0/3511214556 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4fc406ed70 0x7f4fc406f150 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.603+0000 7f4fca041640 1 -- 172.21.15.43:0/3511214556 >> 172.21.15.43:0/3511214556 conn(0x7f4fc40d17e0 msgr2=0x7f4fc40d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:40.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.603+0000 7f4fca041640 1 -- 172.21.15.43:0/3511214556 shutdown_connections 2024-04-25T09:04:40.593 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.603+0000 7f4fca041640 1 -- 172.21.15.43:0/3511214556 wait complete. 2024-04-25T09:04:40.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.605+0000 7f4fca041640 1 Processor -- start 2024-04-25T09:04:40.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.605+0000 7f4fca041640 1 -- start start 2024-04-25T09:04:40.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.605+0000 7f4fca041640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4fc406ed70 0x7f4fc41482d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:40.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.605+0000 7f4fca041640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4fc40cbda0 0x7f4fc409f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:40.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.605+0000 7f4fca041640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fc40d1090 0x7f4fc409b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:40.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.605+0000 7f4fca041640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc40c6fe0 con 0x7f4fc40d1090 2024-04-25T09:04:40.595 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.605+0000 7f4fca041640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc40c6d00 con 0x7f4fc406ed70 2024-04-25T09:04:40.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.605+0000 7f4fca041640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc40c72c0 con 0x7f4fc40cbda0 2024-04-25T09:04:40.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.606+0000 7f4fc3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fc40d1090 0x7f4fc409b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:40.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.606+0000 7f4fc3fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fc40d1090 0x7f4fc409b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59894/0 (socket says 172.21.15.43:59894) 2024-04-25T09:04:40.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.606+0000 7f4fc3fff640 1 -- 172.21.15.43:0/3227184777 learned_addr learned my addr 172.21.15.43:0/3227184777 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:40.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.606+0000 7f4fc37fe640 1 --2- 172.21.15.43:0/3227184777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4fc406ed70 0x7f4fc41482d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:40.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.606+0000 7f4fc2ffd640 1 --2- 172.21.15.43:0/3227184777 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4fc40cbda0 0x7f4fc409f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:40.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.606+0000 7f4fc3fff640 1 -- 172.21.15.43:0/3227184777 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4fc40cbda0 msgr2=0x7f4fc409f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:40.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.606+0000 7f4fc3fff640 1 --2- 172.21.15.43:0/3227184777 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4fc40cbda0 0x7f4fc409f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.606+0000 7f4fc3fff640 1 -- 172.21.15.43:0/3227184777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4fc406ed70 msgr2=0x7f4fc41482d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:40.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.606+0000 7f4fc3fff640 1 --2- 172.21.15.43:0/3227184777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4fc406ed70 0x7f4fc41482d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.606+0000 7f4fc3fff640 1 -- 172.21.15.43:0/3227184777 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc4155e90 con 0x7f4fc40d1090 2024-04-25T09:04:40.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.606+0000 7f4fc3fff640 1 --2- 172.21.15.43:0/3227184777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fc40d1090 0x7f4fc409b820 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc009dd0 tx=0x7f4fbc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:40.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.606+0000 7f4fabfff640 1 -- 172.21.15.43:0/3227184777 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc018020 con 0x7f4fc40d1090 2024-04-25T09:04:40.596 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.606+0000 7f4fabfff640 1 -- 172.21.15.43:0/3227184777 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fbc014480 con 0x7f4fc40d1090 2024-04-25T09:04:40.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.606+0000 7f4fca041640 1 -- 172.21.15.43:0/3227184777 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc4156170 con 0x7f4fc40d1090 2024-04-25T09:04:40.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.606+0000 7f4fabfff640 1 -- 172.21.15.43:0/3227184777 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc014870 con 0x7f4fc40d1090 2024-04-25T09:04:40.597 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.606+0000 7f4fca041640 1 -- 172.21.15.43:0/3227184777 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc409ec60 con 0x7f4fc40d1090 2024-04-25T09:04:40.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.608+0000 7f4fca041640 1 -- 172.21.15.43:0/3227184777 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc406e260 con 0x7f4fc40d1090 2024-04-25T09:04:40.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.608+0000 7f4fabfff640 1 -- 172.21.15.43:0/3227184777 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4fbc005d10 con 0x7f4fc40d1090 2024-04-25T09:04:40.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.608+0000 7f4fabfff640 1 --2- 172.21.15.43:0/3227184777 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4fb006b4d0 0x7f4fb006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:40.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.608+0000 7f4fabfff640 1 -- 172.21.15.43:0/3227184777 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4fbc08abc0 con 0x7f4fc40d1090 2024-04-25T09:04:40.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.609+0000 7f4fc37fe640 1 --2- 172.21.15.43:0/3227184777 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4fb006b4d0 0x7f4fb006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:40.598 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.609+0000 7f4fc37fe640 1 --2- 172.21.15.43:0/3227184777 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4fb006b4d0 0x7f4fb006d990 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4034000 tx=0x7f4fb402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:40.601 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.613+0000 7f4fabfff640 1 -- 172.21.15.43:0/3227184777 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4fc406e260 con 0x7f4fc40d1090 2024-04-25T09:04:40.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.792+0000 7f4fca041640 1 -- 172.21.15.43:0/3227184777 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4fc409ee60 con 0x7f4fc40d1090 2024-04-25T09:04:40.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.793+0000 7f4fabfff640 1 -- 172.21.15.43:0/3227184777 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v139) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4fc409ee60 con 0x7f4fc40d1090 2024-04-25T09:04:40.781 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T09:04:40.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.794+0000 7f4fca041640 1 -- 172.21.15.43:0/3227184777 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4fb006b4d0 msgr2=0x7f4fb006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:40.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.794+0000 7f4fca041640 1 --2- 172.21.15.43:0/3227184777 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4fb006b4d0 0x7f4fb006d990 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4034000 tx=0x7f4fb402e000 comp rx=0 tx=0).stop 2024-04-25T09:04:40.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.794+0000 7f4fca041640 1 -- 172.21.15.43:0/3227184777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fc40d1090 msgr2=0x7f4fc409b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:40.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.794+0000 7f4fca041640 1 --2- 172.21.15.43:0/3227184777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fc40d1090 0x7f4fc409b820 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc009dd0 tx=0x7f4fbc00c490 comp rx=0 tx=0).stop 2024-04-25T09:04:40.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.795+0000 7f4fca041640 1 -- 172.21.15.43:0/3227184777 shutdown_connections 2024-04-25T09:04:40.783 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.795+0000 7f4fca041640 1 --2- 172.21.15.43:0/3227184777 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4fc40d1090 0x7f4fc409b820 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.795+0000 7f4fca041640 1 --2- 172.21.15.43:0/3227184777 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4fc40cbda0 0x7f4fc409f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.795+0000 7f4fca041640 1 --2- 172.21.15.43:0/3227184777 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4fb006b4d0 0x7f4fb006d990 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.795+0000 7f4fca041640 1 --2- 172.21.15.43:0/3227184777 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4fc406ed70 0x7f4fc41482d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.795+0000 7f4fca041640 1 -- 172.21.15.43:0/3227184777 >> 172.21.15.43:0/3227184777 conn(0x7f4fc40d17e0 msgr2=0x7f4fc40af160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:40.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.795+0000 7f4fca041640 1 -- 172.21.15.43:0/3227184777 shutdown_connections 2024-04-25T09:04:40.784 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.795+0000 7f4fca041640 1 -- 172.21.15.43:0/3227184777 wait complete. 2024-04-25T09:04:40.794 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:04:40.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.909+0000 7fc0d30f7640 1 Processor -- start 2024-04-25T09:04:40.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.909+0000 7fc0d30f7640 1 -- start start 2024-04-25T09:04:40.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.910+0000 7fc0d30f7640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc0cc0aee80 0x7fc0cc06d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:40.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.910+0000 7fc0d30f7640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc0cc0cd2f0 0x7fc0cc15d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:40.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.910+0000 7fc0d30f7640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0cc15c280 con 0x7fc0cc06d600 2024-04-25T09:04:40.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.910+0000 7fc0d30f7640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0cc0af260 con 0x7fc0cc0aee80 2024-04-25T09:04:40.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.910+0000 7fc0d30f7640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0cc15c560 con 0x7fc0cc0cd2f0 2024-04-25T09:04:40.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.910+0000 7fc0d0e93640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc0cc0aee80 0x7fc0cc06d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:40.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.910+0000 7fc0d0e93640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc0cc0aee80 0x7fc0cc06d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59906/0 (socket says 172.21.15.43:59906) 2024-04-25T09:04:40.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.910+0000 7fc0d0e93640 1 -- 172.21.15.43:0/2719415904 learned_addr learned my addr 172.21.15.43:0/2719415904 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:40.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.910+0000 7fc0d1694640 1 --2- 172.21.15.43:0/2719415904 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc0cc0cd2f0 0x7fc0cc15d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:40.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.910+0000 7fc0d0e93640 1 -- 172.21.15.43:0/2719415904 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc0cc0cd2f0 msgr2=0x7fc0cc15d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:40.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.910+0000 7fc0d0e93640 1 --2- 172.21.15.43:0/2719415904 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc0cc0cd2f0 0x7fc0cc15d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.910+0000 7fc0d0e93640 1 -- 172.21.15.43:0/2719415904 >> v1:172.21.15.193:6789/0 conn(0x7fc0cc06d600 legacy=0x7fc0cc0ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:40.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.910+0000 7fc0d0e93640 1 -- 172.21.15.43:0/2719415904 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0cc15c840 con 0x7fc0cc0aee80 2024-04-25T09:04:40.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.910+0000 7fc0d1694640 1 --2- 172.21.15.43:0/2719415904 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc0cc0cd2f0 0x7fc0cc15d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:40.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.911+0000 7fc0d0e93640 1 --2- 172.21.15.43:0/2719415904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0cc0aee80 0x7fc0cc06d0c0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc072cd0 tx=0x7fc0c002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b282a52af44b83ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:40.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.911+0000 7fc0cb7fe640 1 -- 172.21.15.43:0/2719415904 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0037070 con 0x7fc0cc0aee80 2024-04-25T09:04:40.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.911+0000 7fc0cb7fe640 1 -- 172.21.15.43:0/2719415904 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0c0032440 con 0x7fc0cc0aee80 2024-04-25T09:04:40.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.911+0000 7fc0cb7fe640 1 -- 172.21.15.43:0/2719415904 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0032810 con 0x7fc0cc0aee80 2024-04-25T09:04:40.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.911+0000 7fc0d30f7640 1 -- 172.21.15.43:0/2719415904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0cc0aee80 msgr2=0x7fc0cc06d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:40.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.911+0000 7fc0d30f7640 1 --2- 172.21.15.43:0/2719415904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0cc0aee80 0x7fc0cc06d0c0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc072cd0 tx=0x7fc0c002a1e0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.912+0000 7fc0d30f7640 1 -- 172.21.15.43:0/2719415904 shutdown_connections 2024-04-25T09:04:40.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.912+0000 7fc0d30f7640 1 --2- 172.21.15.43:0/2719415904 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc0cc0cd2f0 0x7fc0cc15d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.912+0000 7fc0d30f7640 1 --2- 172.21.15.43:0/2719415904 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0cc0aee80 0x7fc0cc06d0c0 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.912+0000 7fc0d30f7640 1 -- 172.21.15.43:0/2719415904 >> 172.21.15.43:0/2719415904 conn(0x7fc0cc06ab30 msgr2=0x7fc0cc06af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:40.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.912+0000 7fc0d30f7640 1 -- 172.21.15.43:0/2719415904 shutdown_connections 2024-04-25T09:04:40.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.912+0000 7fc0d30f7640 1 -- 172.21.15.43:0/2719415904 wait complete. 2024-04-25T09:04:40.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.913+0000 7fc0d30f7640 1 Processor -- start 2024-04-25T09:04:40.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.913+0000 7fc0d30f7640 1 -- start start 2024-04-25T09:04:40.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.914+0000 7fc0d30f7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0cc06d600 0x7fc0cc13eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:40.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.914+0000 7fc0d30f7640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc0cc0aee80 0x7fc0cc1f5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:40.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.914+0000 7fc0d30f7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0cc0cd2f0 0x7fc0cc157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:40.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.914+0000 7fc0d30f7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0cc15dc50 con 0x7fc0cc0cd2f0 2024-04-25T09:04:40.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.914+0000 7fc0d30f7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0cc15dad0 con 0x7fc0cc06d600 2024-04-25T09:04:40.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.914+0000 7fc0d30f7640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0cc15ddd0 con 0x7fc0cc0aee80 2024-04-25T09:04:40.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.914+0000 7fc0d1694640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0cc0cd2f0 0x7fc0cc157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:40.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.914+0000 7fc0d1694640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0cc0cd2f0 0x7fc0cc157720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59914/0 (socket says 172.21.15.43:59914) 2024-04-25T09:04:40.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.914+0000 7fc0d1694640 1 -- 172.21.15.43:0/2174453916 learned_addr learned my addr 172.21.15.43:0/2174453916 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:40.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.914+0000 7fc0d0e93640 1 --2- 172.21.15.43:0/2174453916 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0cc06d600 0x7fc0cc13eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:40.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.914+0000 7fc0cbfff640 1 --2- 172.21.15.43:0/2174453916 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc0cc0aee80 0x7fc0cc1f5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:40.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.914+0000 7fc0d1694640 1 -- 172.21.15.43:0/2174453916 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc0cc0aee80 msgr2=0x7fc0cc1f5ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:40.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.914+0000 7fc0d1694640 1 --2- 172.21.15.43:0/2174453916 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc0cc0aee80 0x7fc0cc1f5ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.914+0000 7fc0d1694640 1 -- 172.21.15.43:0/2174453916 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0cc06d600 msgr2=0x7fc0cc13eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:40.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.914+0000 7fc0d1694640 1 --2- 172.21.15.43:0/2174453916 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0cc06d600 0x7fc0cc13eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:40.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.914+0000 7fc0d1694640 1 -- 172.21.15.43:0/2174453916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0cc157c60 con 0x7fc0cc0cd2f0 2024-04-25T09:04:40.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.914+0000 7fc0cbfff640 1 --2- 172.21.15.43:0/2174453916 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc0cc0aee80 0x7fc0cc1f5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:40.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.914+0000 7fc0d1694640 1 --2- 172.21.15.43:0/2174453916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0cc0cd2f0 0x7fc0cc157720 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7fc0c400cd80 tx=0x7fc0c400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:40.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.915+0000 7fc0c97fa640 1 -- 172.21.15.43:0/2174453916 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c400f680 con 0x7fc0cc0cd2f0 2024-04-25T09:04:40.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.915+0000 7fc0c97fa640 1 -- 172.21.15.43:0/2174453916 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0c400f840 con 0x7fc0cc0cd2f0 2024-04-25T09:04:40.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.915+0000 7fc0d30f7640 1 -- 172.21.15.43:0/2174453916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0cc1f1da0 con 0x7fc0cc0cd2f0 2024-04-25T09:04:40.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.915+0000 7fc0c97fa640 1 -- 172.21.15.43:0/2174453916 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c400fc30 con 0x7fc0cc0cd2f0 2024-04-25T09:04:40.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.915+0000 7fc0d30f7640 1 -- 172.21.15.43:0/2174453916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0cc141980 con 0x7fc0cc0cd2f0 2024-04-25T09:04:40.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.916+0000 7fc0d30f7640 1 -- 172.21.15.43:0/2174453916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0cc074f60 con 0x7fc0cc0cd2f0 2024-04-25T09:04:40.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.916+0000 7fc0c97fa640 1 -- 172.21.15.43:0/2174453916 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc0c40213f0 con 0x7fc0cc0cd2f0 2024-04-25T09:04:40.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.917+0000 7fc0c97fa640 1 --2- 172.21.15.43:0/2174453916 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc0b006b4d0 0x7fc0b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:40.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.917+0000 7fc0c97fa640 1 -- 172.21.15.43:0/2174453916 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc0c408adf0 con 0x7fc0cc0cd2f0 2024-04-25T09:04:40.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.917+0000 7fc0d0e93640 1 --2- 172.21.15.43:0/2174453916 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc0b006b4d0 0x7fc0b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:40.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.917+0000 7fc0d0e93640 1 --2- 172.21.15.43:0/2174453916 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc0b006b4d0 0x7fc0b006d990 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0034000 tx=0x7fc0c002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:40.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:40.921+0000 7fc0c97fa640 1 -- 172.21.15.43:0/2174453916 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc0c4014030 con 0x7fc0cc0cd2f0 2024-04-25T09:04:41.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.099+0000 7fc0d30f7640 1 -- 172.21.15.43:0/2174453916 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc0cc0725f0 con 0x7fc0cc0cd2f0 2024-04-25T09:04:41.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.100+0000 7fc0c97fa640 1 -- 172.21.15.43:0/2174453916 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v139) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc0c4059d00 con 0x7fc0cc0cd2f0 2024-04-25T09:04:41.088 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T09:04:41.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.102+0000 7fc0d30f7640 1 -- 172.21.15.43:0/2174453916 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc0b006b4d0 msgr2=0x7fc0b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:41.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.102+0000 7fc0d30f7640 1 --2- 172.21.15.43:0/2174453916 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc0b006b4d0 0x7fc0b006d990 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0034000 tx=0x7fc0c002e000 comp rx=0 tx=0).stop 2024-04-25T09:04:41.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.102+0000 7fc0d30f7640 1 -- 172.21.15.43:0/2174453916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0cc0cd2f0 msgr2=0x7fc0cc157720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:41.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.102+0000 7fc0d30f7640 1 --2- 172.21.15.43:0/2174453916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0cc0cd2f0 0x7fc0cc157720 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7fc0c400cd80 tx=0x7fc0c400a650 comp rx=0 tx=0).stop 2024-04-25T09:04:41.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.102+0000 7fc0d30f7640 1 -- 172.21.15.43:0/2174453916 shutdown_connections 2024-04-25T09:04:41.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.102+0000 7fc0d30f7640 1 --2- 172.21.15.43:0/2174453916 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc0cc0cd2f0 0x7fc0cc157720 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:41.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.102+0000 7fc0d30f7640 1 --2- 172.21.15.43:0/2174453916 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc0cc0aee80 0x7fc0cc1f5ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:41.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.102+0000 7fc0d30f7640 1 --2- 172.21.15.43:0/2174453916 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc0b006b4d0 0x7fc0b006d990 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:41.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.102+0000 7fc0d30f7640 1 --2- 172.21.15.43:0/2174453916 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc0cc06d600 0x7fc0cc13eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:41.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.102+0000 7fc0d30f7640 1 -- 172.21.15.43:0/2174453916 >> 172.21.15.43:0/2174453916 conn(0x7fc0cc06ab30 msgr2=0x7fc0cc0cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:41.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.102+0000 7fc0d30f7640 1 -- 172.21.15.43:0/2174453916 shutdown_connections 2024-04-25T09:04:41.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.102+0000 7fc0d30f7640 1 -- 172.21.15.43:0/2174453916 wait complete. 2024-04-25T09:04:41.101 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:04:41.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.216+0000 7f258a95a640 1 Processor -- start 2024-04-25T09:04:41.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.216+0000 7f258a95a640 1 -- start start 2024-04-25T09:04:41.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.217+0000 7f258a95a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f25840d04a0 0x7f258415ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:41.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.217+0000 7f258a95a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25840cf110 con 0x7f25840ce8c0 2024-04-25T09:04:41.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.217+0000 7f258a95a640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25840cf3f0 con 0x7f258415dec0 2024-04-25T09:04:41.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.217+0000 7f258a95a640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25840cf6d0 con 0x7f25840d04a0 2024-04-25T09:04:41.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.217+0000 7f2583fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f25840d04a0 0x7f258415ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:41.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.217+0000 7f2588ef7640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f25840ce8c0 0x7f25840d17a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44580/0 (socket says 172.21.15.43:44580) 2024-04-25T09:04:41.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.217+0000 7f2588ef7640 1 -- 172.21.15.43:0/401907467 learned_addr learned my addr 172.21.15.43:0/401907467 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:41.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.217+0000 7f2583fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f25840d04a0 0x7f258415ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59930/0 (socket says 172.21.15.43:59930) 2024-04-25T09:04:41.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.217+0000 7f2583fff640 1 -- 172.21.15.43:0/401907467 >> v1:172.21.15.193:6789/0 conn(0x7f258415dec0 legacy=0x7f25840d0880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:41.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.217+0000 7f2583fff640 1 -- 172.21.15.43:0/401907467 >> v1:172.21.15.43:6789/0 conn(0x7f25840ce8c0 legacy=0x7f25840d17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:41.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.217+0000 7f2583fff640 1 -- 172.21.15.43:0/401907467 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25840cf9b0 con 0x7f25840d04a0 2024-04-25T09:04:41.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.218+0000 7f2583fff640 1 --2- 172.21.15.43:0/401907467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25840d04a0 0x7f258415ed90 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f25740097a0 tx=0x7f257402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e71ea1f5fbbf7251 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:41.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.218+0000 7f2582ffd640 1 -- 172.21.15.43:0/401907467 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2574037070 con 0x7f25840d04a0 2024-04-25T09:04:41.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.218+0000 7f2582ffd640 1 -- 172.21.15.43:0/401907467 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2574032440 con 0x7f25840d04a0 2024-04-25T09:04:41.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.218+0000 7f2582ffd640 1 -- 172.21.15.43:0/401907467 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2574032810 con 0x7f25840d04a0 2024-04-25T09:04:41.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.218+0000 7f258a95a640 1 -- 172.21.15.43:0/401907467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25840d04a0 msgr2=0x7f258415ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:41.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.218+0000 7f258a95a640 1 --2- 172.21.15.43:0/401907467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25840d04a0 0x7f258415ed90 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f25740097a0 tx=0x7f257402a1e0 comp rx=0 tx=0).stop 2024-04-25T09:04:41.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.218+0000 7f258a95a640 1 -- 172.21.15.43:0/401907467 shutdown_connections 2024-04-25T09:04:41.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.219+0000 7f258a95a640 1 --2- 172.21.15.43:0/401907467 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f25840d04a0 0x7f258415ed90 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:41.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.219+0000 7f258a95a640 1 -- 172.21.15.43:0/401907467 >> 172.21.15.43:0/401907467 conn(0x7f258409cd40 msgr2=0x7f25840d3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:41.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.219+0000 7f258a95a640 1 -- 172.21.15.43:0/401907467 shutdown_connections 2024-04-25T09:04:41.208 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.219+0000 7f258a95a640 1 -- 172.21.15.43:0/401907467 wait complete. 2024-04-25T09:04:41.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.220+0000 7f258a95a640 1 Processor -- start 2024-04-25T09:04:41.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.220+0000 7f258a95a640 1 -- start start 2024-04-25T09:04:41.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.221+0000 7f258a95a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25840ce8c0 0x7f25841f4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:41.210 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.221+0000 7f258a95a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f25840d04a0 0x7f25840cd240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:41.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.221+0000 7f258a95a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f258415dec0 0x7f25841f7730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:41.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.221+0000 7f258a95a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25840cee50 con 0x7f258415dec0 2024-04-25T09:04:41.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.221+0000 7f258a95a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25840cecd0 con 0x7f25840ce8c0 2024-04-25T09:04:41.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.221+0000 7f258a95a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25840d1eb0 con 0x7f25840d04a0 2024-04-25T09:04:41.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.221+0000 7f2588ef7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f258415dec0 0x7f25841f7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:41.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.221+0000 7f2588ef7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f258415dec0 0x7f25841f7730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59938/0 (socket says 172.21.15.43:59938) 2024-04-25T09:04:41.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.221+0000 7f2588ef7640 1 -- 172.21.15.43:0/764816279 learned_addr learned my addr 172.21.15.43:0/764816279 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:41.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.221+0000 7f2583fff640 1 --2- 172.21.15.43:0/764816279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25840ce8c0 0x7f25841f4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:41.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.221+0000 7f2588ef7640 1 -- 172.21.15.43:0/764816279 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f25840d04a0 msgr2=0x7f25840cd240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:41.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.221+0000 7f2588ef7640 1 --2- 172.21.15.43:0/764816279 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f25840d04a0 0x7f25840cd240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:41.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.221+0000 7f2588ef7640 1 -- 172.21.15.43:0/764816279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25840ce8c0 msgr2=0x7f25841f4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:41.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.221+0000 7f2588ef7640 1 --2- 172.21.15.43:0/764816279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25840ce8c0 0x7f25841f4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:41.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.221+0000 7f2588ef7640 1 -- 172.21.15.43:0/764816279 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25841f81d0 con 0x7f258415dec0 2024-04-25T09:04:41.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.221+0000 7f2583fff640 1 --2- 172.21.15.43:0/764816279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25840ce8c0 0x7f25841f4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:41.211 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.221+0000 7f2588ef7640 1 --2- 172.21.15.43:0/764816279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f258415dec0 0x7f25841f7730 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f257c00cbc0 tx=0x7f257c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:41.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.221+0000 7f2580ff9640 1 -- 172.21.15.43:0/764816279 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c017070 con 0x7f258415dec0 2024-04-25T09:04:41.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.221+0000 7f2580ff9640 1 -- 172.21.15.43:0/764816279 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f257c00ad40 con 0x7f258415dec0 2024-04-25T09:04:41.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.221+0000 7f258a95a640 1 -- 172.21.15.43:0/764816279 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25841f84b0 con 0x7f258415dec0 2024-04-25T09:04:41.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.221+0000 7f2580ff9640 1 -- 172.21.15.43:0/764816279 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c012600 con 0x7f258415dec0 2024-04-25T09:04:41.212 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.221+0000 7f258a95a640 1 -- 172.21.15.43:0/764816279 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25841f46a0 con 0x7f258415dec0 2024-04-25T09:04:41.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.222+0000 7f258a95a640 1 -- 172.21.15.43:0/764816279 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f258406c140 con 0x7f258415dec0 2024-04-25T09:04:41.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.223+0000 7f2580ff9640 1 -- 172.21.15.43:0/764816279 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f257c01a070 con 0x7f258415dec0 2024-04-25T09:04:41.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.223+0000 7f2580ff9640 1 --2- 172.21.15.43:0/764816279 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f257006b500 0x7f257006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:41.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.223+0000 7f2580ff9640 1 -- 172.21.15.43:0/764816279 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f257c089de0 con 0x7f258415dec0 2024-04-25T09:04:41.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.224+0000 7f2583fff640 1 --2- 172.21.15.43:0/764816279 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f257006b500 0x7f257006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:41.213 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.224+0000 7f2583fff640 1 --2- 172.21.15.43:0/764816279 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f257006b500 0x7f257006d9c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f2574034000 tx=0x7f257402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:41.216 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.228+0000 7f2580ff9640 1 -- 172.21.15.43:0/764816279 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f258406c140 con 0x7f258415dec0 2024-04-25T09:04:41.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.406+0000 7f258a95a640 1 -- 172.21.15.43:0/764816279 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2584074f60 con 0x7f258415dec0 2024-04-25T09:04:41.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.407+0000 7f2580ff9640 1 -- 172.21.15.43:0/764816279 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v139) v1 ==== 100+0+11 (secure 0 0 0) 0x7f257c0586a0 con 0x7f258415dec0 2024-04-25T09:04:41.395 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T09:04:41.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.409+0000 7f258a95a640 1 -- 172.21.15.43:0/764816279 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f257006b500 msgr2=0x7f257006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:41.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.409+0000 7f258a95a640 1 --2- 172.21.15.43:0/764816279 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f257006b500 0x7f257006d9c0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f2574034000 tx=0x7f257402e000 comp rx=0 tx=0).stop 2024-04-25T09:04:41.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.409+0000 7f258a95a640 1 -- 172.21.15.43:0/764816279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f258415dec0 msgr2=0x7f25841f7730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:41.397 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.409+0000 7f258a95a640 1 --2- 172.21.15.43:0/764816279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f258415dec0 0x7f25841f7730 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f257c00cbc0 tx=0x7f257c00a430 comp rx=0 tx=0).stop 2024-04-25T09:04:41.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.409+0000 7f258a95a640 1 -- 172.21.15.43:0/764816279 shutdown_connections 2024-04-25T09:04:41.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.409+0000 7f258a95a640 1 --2- 172.21.15.43:0/764816279 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f258415dec0 0x7f25841f7730 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:41.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.409+0000 7f258a95a640 1 --2- 172.21.15.43:0/764816279 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f25840d04a0 0x7f25840cd240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:41.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.409+0000 7f258a95a640 1 --2- 172.21.15.43:0/764816279 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f257006b500 0x7f257006d9c0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:41.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.409+0000 7f258a95a640 1 --2- 172.21.15.43:0/764816279 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f25840ce8c0 0x7f25841f4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:41.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.409+0000 7f258a95a640 1 -- 172.21.15.43:0/764816279 >> 172.21.15.43:0/764816279 conn(0x7f258409cd40 msgr2=0x7f2584160df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:41.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.409+0000 7f258a95a640 1 -- 172.21.15.43:0/764816279 shutdown_connections 2024-04-25T09:04:41.398 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.409+0000 7f258a95a640 1 -- 172.21.15.43:0/764816279 wait complete. 2024-04-25T09:04:41.408 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T09:04:41.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.521+0000 7f6cf7b29640 1 Processor -- start 2024-04-25T09:04:41.509 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.521+0000 7f6cf7b29640 1 -- start start 2024-04-25T09:04:41.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.522+0000 7f6cf7b29640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6cf015a9d0 0x7f6cf006d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:41.510 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.522+0000 7f6cf7b29640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6cf006d7d0 0x7f6cf0071a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:41.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.522+0000 7f6cf7b29640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cf0072400 con 0x7f6cf0071f90 2024-04-25T09:04:41.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.522+0000 7f6cf7b29640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cf006dc30 con 0x7f6cf006d7d0 2024-04-25T09:04:41.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.522+0000 7f6cf7b29640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cf00704f0 con 0x7f6cf015a9d0 2024-04-25T09:04:41.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.522+0000 7f6cf50c4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f6cf006d7d0 0x7f6cf0071a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:41.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.522+0000 7f6cf60c6640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f6cf0071f90 0x7f6cf00b3990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44596/0 (socket says 172.21.15.43:44596) 2024-04-25T09:04:41.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.522+0000 7f6cf60c6640 1 -- 172.21.15.43:0/1588250225 learned_addr learned my addr 172.21.15.43:0/1588250225 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:41.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.522+0000 7f6cf50c4640 1 -- 172.21.15.43:0/1588250225 >> v2:172.21.15.193:3300/0 conn(0x7f6cf015a9d0 msgr2=0x7f6cf006d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:41.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.522+0000 7f6cf58c5640 1 --2- 172.21.15.43:0/1588250225 >> v2:172.21.15.193:3300/0 conn(0x7f6cf015a9d0 0x7f6cf006d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:41.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.522+0000 7f6cf50c4640 1 --2- 172.21.15.43:0/1588250225 >> v2:172.21.15.193:3300/0 conn(0x7f6cf015a9d0 0x7f6cf006d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:41.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.522+0000 7f6cf48c3640 1 -- 172.21.15.43:0/1588250225 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1117132484 0 0) 0x7f6cf0072400 con 0x7f6cf0071f90 2024-04-25T09:04:41.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.522+0000 7f6cf50c4640 1 -- 172.21.15.43:0/1588250225 >> v1:172.21.15.43:6789/0 conn(0x7f6cf0071f90 legacy=0x7f6cf00b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:41.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.522+0000 7f6cf50c4640 1 -- 172.21.15.43:0/1588250225 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf00707d0 con 0x7f6cf006d7d0 2024-04-25T09:04:41.511 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.522+0000 7f6cf58c5640 1 --2- 172.21.15.43:0/1588250225 >> v2:172.21.15.193:3300/0 conn(0x7f6cf015a9d0 0x7f6cf006d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:41.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.523+0000 7f6cf50c4640 1 --2- 172.21.15.43:0/1588250225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cf006d7d0 0x7f6cf0071a50 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f6cf006c190 tx=0x7f6ce402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4137a794d4d8b7eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:41.512 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.523+0000 7f6cf48c3640 1 -- 172.21.15.43:0/1588250225 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4037070 con 0x7f6cf006d7d0 2024-04-25T09:04:41.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.523+0000 7f6cf48c3640 1 -- 172.21.15.43:0/1588250225 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce4032440 con 0x7f6cf006d7d0 2024-04-25T09:04:41.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.523+0000 7f6cf48c3640 1 -- 172.21.15.43:0/1588250225 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4032810 con 0x7f6cf006d7d0 2024-04-25T09:04:41.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.523+0000 7f6cf7b29640 1 -- 172.21.15.43:0/1588250225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cf006d7d0 msgr2=0x7f6cf0071a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:41.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.523+0000 7f6cf7b29640 1 --2- 172.21.15.43:0/1588250225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cf006d7d0 0x7f6cf0071a50 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f6cf006c190 tx=0x7f6ce402a1a0 comp rx=0 tx=0).stop 2024-04-25T09:04:41.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.523+0000 7f6cf7b29640 1 -- 172.21.15.43:0/1588250225 shutdown_connections 2024-04-25T09:04:41.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.523+0000 7f6cf7b29640 1 --2- 172.21.15.43:0/1588250225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cf006d7d0 0x7f6cf0071a50 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:41.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.523+0000 7f6cf7b29640 1 --2- 172.21.15.43:0/1588250225 >> v2:172.21.15.193:3300/0 conn(0x7f6cf015a9d0 0x7f6cf006d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:41.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.523+0000 7f6cf7b29640 1 -- 172.21.15.43:0/1588250225 >> 172.21.15.43:0/1588250225 conn(0x7f6cf00b1480 msgr2=0x7f6cf00b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:41.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.524+0000 7f6cf7b29640 1 -- 172.21.15.43:0/1588250225 shutdown_connections 2024-04-25T09:04:41.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.524+0000 7f6cf7b29640 1 -- 172.21.15.43:0/1588250225 wait complete. 2024-04-25T09:04:41.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.525+0000 7f6cf7b29640 1 Processor -- start 2024-04-25T09:04:41.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.525+0000 7f6cf7b29640 1 -- start start 2024-04-25T09:04:41.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.525+0000 7f6cf7b29640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6cf006d7d0 0x7f6cf015d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:41.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.525+0000 7f6cf7b29640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cf0071f90 0x7f6cf0151af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:41.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.525+0000 7f6cf7b29640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cf01580a0 0x7f6cf01f1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:41.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.525+0000 7f6cf7b29640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf00712b0 con 0x7f6cf0071f90 2024-04-25T09:04:41.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.525+0000 7f6cf7b29640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf0070d90 con 0x7f6cf01580a0 2024-04-25T09:04:41.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.525+0000 7f6cf7b29640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf00ce950 con 0x7f6cf006d7d0 2024-04-25T09:04:41.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.526+0000 7f6cf50c4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cf0071f90 0x7f6cf0151af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:41.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.526+0000 7f6cf50c4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cf0071f90 0x7f6cf0151af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59954/0 (socket says 172.21.15.43:59954) 2024-04-25T09:04:41.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.526+0000 7f6cf50c4640 1 -- 172.21.15.43:0/1547569173 learned_addr learned my addr 172.21.15.43:0/1547569173 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:41.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.526+0000 7f6cf60c6640 1 --2- 172.21.15.43:0/1547569173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cf01580a0 0x7f6cf01f1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:41.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.526+0000 7f6cf58c5640 1 --2- 172.21.15.43:0/1547569173 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6cf006d7d0 0x7f6cf015d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:41.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.526+0000 7f6cf50c4640 1 -- 172.21.15.43:0/1547569173 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6cf006d7d0 msgr2=0x7f6cf015d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:41.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.526+0000 7f6cf50c4640 1 --2- 172.21.15.43:0/1547569173 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6cf006d7d0 0x7f6cf015d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:41.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.526+0000 7f6cf50c4640 1 -- 172.21.15.43:0/1547569173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cf01580a0 msgr2=0x7f6cf01f1cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:41.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.526+0000 7f6cf50c4640 1 --2- 172.21.15.43:0/1547569173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cf01580a0 0x7f6cf01f1cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:41.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.526+0000 7f6cf50c4640 1 -- 172.21.15.43:0/1547569173 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf015c8b0 con 0x7f6cf0071f90 2024-04-25T09:04:41.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.526+0000 7f6cf58c5640 1 --2- 172.21.15.43:0/1547569173 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6cf006d7d0 0x7f6cf015d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:41.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.526+0000 7f6cf50c4640 1 --2- 172.21.15.43:0/1547569173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cf0071f90 0x7f6cf0151af0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4009890 tx=0x7f6ce40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:41.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.526+0000 7f6cee7fc640 1 -- 172.21.15.43:0/1547569173 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4037040 con 0x7f6cf0071f90 2024-04-25T09:04:41.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.526+0000 7f6cee7fc640 1 -- 172.21.15.43:0/1547569173 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce4042070 con 0x7f6cf0071f90 2024-04-25T09:04:41.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.526+0000 7f6cf7b29640 1 -- 172.21.15.43:0/1547569173 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cf015cb90 con 0x7f6cf0071f90 2024-04-25T09:04:41.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.526+0000 7f6cee7fc640 1 -- 172.21.15.43:0/1547569173 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce4030a50 con 0x7f6cf0071f90 2024-04-25T09:04:41.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.526+0000 7f6cf7b29640 1 -- 172.21.15.43:0/1547569173 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cf015d150 con 0x7f6cf0071f90 2024-04-25T09:04:41.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.527+0000 7f6cf7b29640 1 -- 172.21.15.43:0/1547569173 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cf00677f0 con 0x7f6cf0071f90 2024-04-25T09:04:41.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.528+0000 7f6cee7fc640 1 -- 172.21.15.43:0/1547569173 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6ce4030bf0 con 0x7f6cf0071f90 2024-04-25T09:04:41.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.528+0000 7f6cee7fc640 1 --2- 172.21.15.43:0/1547569173 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6cdc06b4d0 0x7f6cdc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:41.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.528+0000 7f6cee7fc640 1 -- 172.21.15.43:0/1547569173 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6ce40ab810 con 0x7f6cf0071f90 2024-04-25T09:04:41.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.528+0000 7f6cf58c5640 1 --2- 172.21.15.43:0/1547569173 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6cdc06b4d0 0x7f6cdc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:41.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.529+0000 7f6cf58c5640 1 --2- 172.21.15.43:0/1547569173 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6cdc06b4d0 0x7f6cdc06d990 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8006fd0 tx=0x7f6ce8008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:41.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.532+0000 7f6cee7fc640 1 -- 172.21.15.43:0/1547569173 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6cf00677f0 con 0x7f6cf0071f90 2024-04-25T09:04:41.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.726+0000 7f6cf7b29640 1 -- 172.21.15.43:0/1547569173 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6cf00cc190 con 0x7f6cf0071f90 2024-04-25T09:04:41.715 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.727+0000 7f6cee7fc640 1 -- 172.21.15.43:0/1547569173 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v139) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6cf00cc190 con 0x7f6cf0071f90 2024-04-25T09:04:41.715 INFO:teuthology.orchestra.run.smithi043.stderr:listed 0 entries 2024-04-25T09:04:41.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.729+0000 7f6cf7b29640 1 -- 172.21.15.43:0/1547569173 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6cdc06b4d0 msgr2=0x7f6cdc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:41.717 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.729+0000 7f6cf7b29640 1 --2- 172.21.15.43:0/1547569173 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6cdc06b4d0 0x7f6cdc06d990 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f6ce8006fd0 tx=0x7f6ce8008040 comp rx=0 tx=0).stop 2024-04-25T09:04:41.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.729+0000 7f6cf7b29640 1 -- 172.21.15.43:0/1547569173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cf0071f90 msgr2=0x7f6cf0151af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:41.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.729+0000 7f6cf7b29640 1 --2- 172.21.15.43:0/1547569173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cf0071f90 0x7f6cf0151af0 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4009890 tx=0x7f6ce40046c0 comp rx=0 tx=0).stop 2024-04-25T09:04:41.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.729+0000 7f6cf7b29640 1 -- 172.21.15.43:0/1547569173 shutdown_connections 2024-04-25T09:04:41.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.729+0000 7f6cf7b29640 1 --2- 172.21.15.43:0/1547569173 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6cdc06b4d0 0x7f6cdc06d990 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:41.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.729+0000 7f6cf7b29640 1 --2- 172.21.15.43:0/1547569173 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6cf01580a0 0x7f6cf01f1cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:41.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.729+0000 7f6cf7b29640 1 --2- 172.21.15.43:0/1547569173 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6cf0071f90 0x7f6cf0151af0 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:41.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.729+0000 7f6cf7b29640 1 --2- 172.21.15.43:0/1547569173 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6cf006d7d0 0x7f6cf015d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:41.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.729+0000 7f6cf7b29640 1 -- 172.21.15.43:0/1547569173 >> 172.21.15.43:0/1547569173 conn(0x7f6cf00b1480 msgr2=0x7f6cf00b0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:41.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.729+0000 7f6cf7b29640 1 -- 172.21.15.43:0/1547569173 shutdown_connections 2024-04-25T09:04:41.718 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:41.729+0000 7f6cf7b29640 1 -- 172.21.15.43:0/1547569173 wait complete. 2024-04-25T09:04:41.728 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi043.front.sepia.ceph.com... 2024-04-25T09:04:41.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:41.728 DEBUG:teuthology.orchestra.run.smithi043:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-25T09:04:41.751 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi043.stderr:ceph-fuse[96951]: fuse finished with error 0 and tester_r 0 2024-04-25T09:04:41.823 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-04-25T09:04:41.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:41.824 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T09:04:41.991 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T09:04:41.991 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:04:42.007 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19508 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:04:42.007 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi193.front.sepia.ceph.com 2024-04-25T09:04:42.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:42.007 DEBUG:teuthology.orchestra.run.smithi193:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T09:04:42.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:04:42.062 INFO:teuthology.orchestra.run.smithi193.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-25T09:04:42.062 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T09:04:42.062 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:04:42.078 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2374 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:04:42.078 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi043.front.sepia.ceph.com 2024-04-25T09:04:42.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:42.078 DEBUG:teuthology.orchestra.run.smithi043:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T09:04:42.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:04:42.133 INFO:teuthology.orchestra.run.smithi043.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-25T09:04:42.134 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T09:04:42.134 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:04:42.149 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19508 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:04:42.150 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi193.front.sepia.ceph.com 2024-04-25T09:04:42.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:42.150 DEBUG:teuthology.orchestra.run.smithi193:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T09:04:42.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:04:42.205 INFO:teuthology.orchestra.run.smithi193.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-25T09:04:42.205 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:04:42.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.318+0000 7feb25b63640 1 Processor -- start 2024-04-25T09:04:42.307 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.318+0000 7feb25b63640 1 -- start start 2024-04-25T09:04:42.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.319+0000 7feb25b63640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7feb2015df50 0x7feb201619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:42.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.319+0000 7feb25b63640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feb20161f30 0x7feb201623a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:42.308 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.319+0000 7feb25b63640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb200d24a0 con 0x7feb200c7cf0 2024-04-25T09:04:42.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.319+0000 7feb25b63640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb2015e330 con 0x7feb2015df50 2024-04-25T09:04:42.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.319+0000 7feb25b63640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb200d2740 con 0x7feb20161f30 2024-04-25T09:04:42.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.319+0000 7feb1f7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7feb2015df50 0x7feb201619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:42.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.319+0000 7feb1f7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7feb2015df50 0x7feb201619f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59966/0 (socket says 172.21.15.43:59966) 2024-04-25T09:04:42.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.319+0000 7feb1f7fe640 1 -- 172.21.15.43:0/1705649796 learned_addr learned my addr 172.21.15.43:0/1705649796 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:42.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.319+0000 7feb1effd640 1 --2- 172.21.15.43:0/1705649796 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feb20161f30 0x7feb201623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:42.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.319+0000 7feb1f7fe640 1 -- 172.21.15.43:0/1705649796 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feb20161f30 msgr2=0x7feb201623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:42.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.319+0000 7feb1f7fe640 1 --2- 172.21.15.43:0/1705649796 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feb20161f30 0x7feb201623a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.319+0000 7feb1f7fe640 1 -- 172.21.15.43:0/1705649796 >> v1:172.21.15.193:6789/0 conn(0x7feb200c7cf0 legacy=0x7feb200c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:42.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.319+0000 7feb1f7fe640 1 -- 172.21.15.43:0/1705649796 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb200d2a20 con 0x7feb2015df50 2024-04-25T09:04:42.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.320+0000 7feb1f7fe640 1 --2- 172.21.15.43:0/1705649796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb2015df50 0x7feb201619f0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7feb10009870 tx=0x7feb1002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8900667c4e916bea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:42.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.320+0000 7feb1e7fc640 1 -- 172.21.15.43:0/1705649796 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb10037070 con 0x7feb2015df50 2024-04-25T09:04:42.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.320+0000 7feb1e7fc640 1 -- 172.21.15.43:0/1705649796 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb10032440 con 0x7feb2015df50 2024-04-25T09:04:42.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.320+0000 7feb1e7fc640 1 -- 172.21.15.43:0/1705649796 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb10032810 con 0x7feb2015df50 2024-04-25T09:04:42.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.321+0000 7feb25b63640 1 -- 172.21.15.43:0/1705649796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb2015df50 msgr2=0x7feb201619f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:42.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.321+0000 7feb25b63640 1 --2- 172.21.15.43:0/1705649796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb2015df50 0x7feb201619f0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7feb10009870 tx=0x7feb1002a270 comp rx=0 tx=0).stop 2024-04-25T09:04:42.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.321+0000 7feb25b63640 1 -- 172.21.15.43:0/1705649796 shutdown_connections 2024-04-25T09:04:42.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.321+0000 7feb25b63640 1 --2- 172.21.15.43:0/1705649796 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feb20161f30 0x7feb201623a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.321+0000 7feb25b63640 1 --2- 172.21.15.43:0/1705649796 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb2015df50 0x7feb201619f0 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.321+0000 7feb25b63640 1 -- 172.21.15.43:0/1705649796 >> 172.21.15.43:0/1705649796 conn(0x7feb200d3a30 msgr2=0x7feb200d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:42.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.321+0000 7feb25b63640 1 -- 172.21.15.43:0/1705649796 shutdown_connections 2024-04-25T09:04:42.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.321+0000 7feb25b63640 1 -- 172.21.15.43:0/1705649796 wait complete. 2024-04-25T09:04:42.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.322+0000 7feb25b63640 1 Processor -- start 2024-04-25T09:04:42.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.322+0000 7feb25b63640 1 -- start start 2024-04-25T09:04:42.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.323+0000 7feb25b63640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb200c7cf0 0x7feb200c6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:42.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.323+0000 7feb25b63640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feb2015df50 0x7feb2006b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:42.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.323+0000 7feb25b63640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb20161f30 0x7feb2006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:42.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.323+0000 7feb25b63640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb200d2190 con 0x7feb20161f30 2024-04-25T09:04:42.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.323+0000 7feb25b63640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb200c77d0 con 0x7feb200c7cf0 2024-04-25T09:04:42.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.323+0000 7feb25b63640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb200cf670 con 0x7feb2015df50 2024-04-25T09:04:42.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.323+0000 7feb1ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb20161f30 0x7feb2006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:42.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.323+0000 7feb1ffff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb20161f30 0x7feb2006b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59982/0 (socket says 172.21.15.43:59982) 2024-04-25T09:04:42.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.323+0000 7feb1ffff640 1 -- 172.21.15.43:0/2844112027 learned_addr learned my addr 172.21.15.43:0/2844112027 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:42.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.323+0000 7feb1f7fe640 1 --2- 172.21.15.43:0/2844112027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb200c7cf0 0x7feb200c6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:42.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.323+0000 7feb1effd640 1 --2- 172.21.15.43:0/2844112027 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feb2015df50 0x7feb2006b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:42.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.323+0000 7feb1ffff640 1 -- 172.21.15.43:0/2844112027 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feb2015df50 msgr2=0x7feb2006b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:42.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.323+0000 7feb1ffff640 1 --2- 172.21.15.43:0/2844112027 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feb2015df50 0x7feb2006b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.323+0000 7feb1ffff640 1 -- 172.21.15.43:0/2844112027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb200c7cf0 msgr2=0x7feb200c6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:42.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.323+0000 7feb1ffff640 1 --2- 172.21.15.43:0/2844112027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb200c7cf0 0x7feb200c6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.323+0000 7feb1ffff640 1 -- 172.21.15.43:0/2844112027 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb2009f520 con 0x7feb20161f30 2024-04-25T09:04:42.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.323+0000 7feb1ffff640 1 --2- 172.21.15.43:0/2844112027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb20161f30 0x7feb2006b760 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7feb1800c7d0 tx=0x7feb1800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:42.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.324+0000 7feb07fff640 1 -- 172.21.15.43:0/2844112027 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb18003b70 con 0x7feb20161f30 2024-04-25T09:04:42.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.324+0000 7feb07fff640 1 -- 172.21.15.43:0/2844112027 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb18003d30 con 0x7feb20161f30 2024-04-25T09:04:42.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.324+0000 7feb25b63640 1 -- 172.21.15.43:0/2844112027 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb2009f800 con 0x7feb20161f30 2024-04-25T09:04:42.314 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.324+0000 7feb07fff640 1 -- 172.21.15.43:0/2844112027 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1800aa90 con 0x7feb20161f30 2024-04-25T09:04:42.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.324+0000 7feb25b63640 1 -- 172.21.15.43:0/2844112027 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb2009fdc0 con 0x7feb20161f30 2024-04-25T09:04:42.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.325+0000 7feb25b63640 1 -- 172.21.15.43:0/2844112027 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb200680f0 con 0x7feb20161f30 2024-04-25T09:04:42.315 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.325+0000 7feb07fff640 1 -- 172.21.15.43:0/2844112027 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feb1800ac30 con 0x7feb20161f30 2024-04-25T09:04:42.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.325+0000 7feb07fff640 1 --2- 172.21.15.43:0/2844112027 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7feb0c06b4d0 0x7feb0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:42.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.326+0000 7feb07fff640 1 -- 172.21.15.43:0/2844112027 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feb18089fd0 con 0x7feb20161f30 2024-04-25T09:04:42.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.326+0000 7feb1f7fe640 1 --2- 172.21.15.43:0/2844112027 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7feb0c06b4d0 0x7feb0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:42.316 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.326+0000 7feb1f7fe640 1 --2- 172.21.15.43:0/2844112027 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7feb0c06b4d0 0x7feb0c06d990 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7feb10034000 tx=0x7feb1002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:42.318 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.330+0000 7feb07fff640 1 -- 172.21.15.43:0/2844112027 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feb18058800 con 0x7feb20161f30 2024-04-25T09:04:42.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.545+0000 7feb25b63640 1 -- 172.21.15.43:0/2844112027 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb200ca400 con 0x7feb20161f30 2024-04-25T09:04:42.534 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.545+0000 7feb07fff640 1 -- 172.21.15.43:0/2844112027 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4952 (secure 0 0 0) 0x7feb1805c8a0 con 0x7feb20161f30 2024-04-25T09:04:42.534 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:04:42.534 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/2225978762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2225978762},{"type":"v1","addr":"172.21.15.193:6836","nonce":2225978762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9711,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/454458092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":454458092},{"type":"v1","addr":"172.21.15.43:6833","nonce":454458092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":90,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T09:04:13.410799+0000","modified":"2024-04-25T09:04:39.112330+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9732},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9720":{"gid":9720,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/2314901977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":2314901977},{"type":"v1","addr":"172.21.15.43:6835","nonce":2314901977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9732":{"gid":9732,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.15.193:6837/2585534546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2585534546},{"type":"v1","addr":"172.21.15.193:6837","nonce":2585534546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T09:04:42.534 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 90 2024-04-25T09:04:42.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.547+0000 7feb25b63640 1 -- 172.21.15.43:0/2844112027 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7feb0c06b4d0 msgr2=0x7feb0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:42.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.547+0000 7feb25b63640 1 --2- 172.21.15.43:0/2844112027 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7feb0c06b4d0 0x7feb0c06d990 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7feb10034000 tx=0x7feb1002e000 comp rx=0 tx=0).stop 2024-04-25T09:04:42.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.547+0000 7feb25b63640 1 -- 172.21.15.43:0/2844112027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb20161f30 msgr2=0x7feb2006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:42.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.547+0000 7feb25b63640 1 --2- 172.21.15.43:0/2844112027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb20161f30 0x7feb2006b760 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7feb1800c7d0 tx=0x7feb1800cc90 comp rx=0 tx=0).stop 2024-04-25T09:04:42.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.547+0000 7feb25b63640 1 -- 172.21.15.43:0/2844112027 shutdown_connections 2024-04-25T09:04:42.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.547+0000 7feb25b63640 1 --2- 172.21.15.43:0/2844112027 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7feb20161f30 0x7feb2006b760 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.547+0000 7feb25b63640 1 --2- 172.21.15.43:0/2844112027 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7feb2015df50 0x7feb2006b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.547+0000 7feb25b63640 1 --2- 172.21.15.43:0/2844112027 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7feb0c06b4d0 0x7feb0c06d990 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.547+0000 7feb25b63640 1 --2- 172.21.15.43:0/2844112027 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7feb200c7cf0 0x7feb200c6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.547+0000 7feb25b63640 1 -- 172.21.15.43:0/2844112027 >> 172.21.15.43:0/2844112027 conn(0x7feb200d3a30 msgr2=0x7feb20153e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:42.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.547+0000 7feb25b63640 1 -- 172.21.15.43:0/2844112027 shutdown_connections 2024-04-25T09:04:42.537 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.547+0000 7feb25b63640 1 -- 172.21.15.43:0/2844112027 wait complete. 2024-04-25T09:04:42.547 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:04:42.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.659+0000 7fcd7747c640 1 Processor -- start 2024-04-25T09:04:42.647 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.659+0000 7fcd7747c640 1 -- start start 2024-04-25T09:04:42.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.660+0000 7fcd7747c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcd70166d10 0x7fcd70169100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:42.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.660+0000 7fcd7747c640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd70162810 con 0x7fcd7015def0 2024-04-25T09:04:42.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.660+0000 7fcd7747c640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd7015e2d0 con 0x7fcd7006be50 2024-04-25T09:04:42.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.660+0000 7fcd7747c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd700b30d0 con 0x7fcd70166d10 2024-04-25T09:04:42.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.660+0000 7fcd75a19640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fcd70166d10 0x7fcd70169100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:42.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.660+0000 7fcd75218640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fcd7015def0 0x7fcd701619f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44606/0 (socket says 172.21.15.43:44606) 2024-04-25T09:04:42.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.660+0000 7fcd75218640 1 -- 172.21.15.43:0/3118975188 learned_addr learned my addr 172.21.15.43:0/3118975188 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:42.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.660+0000 7fcd75a19640 1 -- 172.21.15.43:0/3118975188 >> v1:172.21.15.193:6789/0 conn(0x7fcd7006be50 legacy=0x7fcd70162100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:42.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.660+0000 7fcd75a19640 1 -- 172.21.15.43:0/3118975188 >> v1:172.21.15.43:6789/0 conn(0x7fcd7015def0 legacy=0x7fcd701619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:42.649 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.660+0000 7fcd75a19640 1 -- 172.21.15.43:0/3118975188 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd700b1020 con 0x7fcd70166d10 2024-04-25T09:04:42.650 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.661+0000 7fcd75a19640 1 --2- 172.21.15.43:0/3118975188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd70166d10 0x7fcd70169100 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009d90 tx=0x7fcd6802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13f28258dccd8ed7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:42.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.661+0000 7fcd67fff640 1 -- 172.21.15.43:0/3118975188 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd68007740 con 0x7fcd70166d10 2024-04-25T09:04:42.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.661+0000 7fcd67fff640 1 -- 172.21.15.43:0/3118975188 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd680078e0 con 0x7fcd70166d10 2024-04-25T09:04:42.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.661+0000 7fcd67fff640 1 -- 172.21.15.43:0/3118975188 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd68007cb0 con 0x7fcd70166d10 2024-04-25T09:04:42.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.662+0000 7fcd7747c640 1 -- 172.21.15.43:0/3118975188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd70166d10 msgr2=0x7fcd70169100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:42.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.662+0000 7fcd7747c640 1 --2- 172.21.15.43:0/3118975188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd70166d10 0x7fcd70169100 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7fcd68009d90 tx=0x7fcd6802a6a0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.662+0000 7fcd7747c640 1 -- 172.21.15.43:0/3118975188 shutdown_connections 2024-04-25T09:04:42.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.662+0000 7fcd7747c640 1 --2- 172.21.15.43:0/3118975188 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd70166d10 0x7fcd70169100 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.662+0000 7fcd7747c640 1 -- 172.21.15.43:0/3118975188 >> 172.21.15.43:0/3118975188 conn(0x7fcd700b32e0 msgr2=0x7fcd700b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:42.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.662+0000 7fcd7747c640 1 -- 172.21.15.43:0/3118975188 shutdown_connections 2024-04-25T09:04:42.651 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.662+0000 7fcd7747c640 1 -- 172.21.15.43:0/3118975188 wait complete. 2024-04-25T09:04:42.653 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.663+0000 7fcd7747c640 1 Processor -- start 2024-04-25T09:04:42.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.663+0000 7fcd7747c640 1 -- start start 2024-04-25T09:04:42.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.664+0000 7fcd7747c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcd7006be50 0x7fcd700c79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:42.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.664+0000 7fcd7747c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd7015def0 0x7fcd70158260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:42.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.664+0000 7fcd7747c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd70166d10 0x7fcd7014b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:42.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.664+0000 7fcd7747c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd700b15e0 con 0x7fcd7015def0 2024-04-25T09:04:42.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.664+0000 7fcd7747c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd700d0e90 con 0x7fcd70166d10 2024-04-25T09:04:42.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.664+0000 7fcd7747c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd700b18c0 con 0x7fcd7006be50 2024-04-25T09:04:42.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.664+0000 7fcd74a17640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd7015def0 0x7fcd70158260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:42.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.664+0000 7fcd74a17640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd7015def0 0x7fcd70158260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59994/0 (socket says 172.21.15.43:59994) 2024-04-25T09:04:42.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.664+0000 7fcd74a17640 1 -- 172.21.15.43:0/2442610875 learned_addr learned my addr 172.21.15.43:0/2442610875 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:42.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.664+0000 7fcd75a19640 1 --2- 172.21.15.43:0/2442610875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd70166d10 0x7fcd7014b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:42.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.664+0000 7fcd75218640 1 --2- 172.21.15.43:0/2442610875 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcd7006be50 0x7fcd700c79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:42.654 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.664+0000 7fcd74a17640 1 -- 172.21.15.43:0/2442610875 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcd7006be50 msgr2=0x7fcd700c79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:42.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.664+0000 7fcd74a17640 1 --2- 172.21.15.43:0/2442610875 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcd7006be50 0x7fcd700c79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.664+0000 7fcd74a17640 1 -- 172.21.15.43:0/2442610875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd70166d10 msgr2=0x7fcd7014b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:42.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.664+0000 7fcd74a17640 1 --2- 172.21.15.43:0/2442610875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd70166d10 0x7fcd7014b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.664+0000 7fcd74a17640 1 -- 172.21.15.43:0/2442610875 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd70158d70 con 0x7fcd7015def0 2024-04-25T09:04:42.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.664+0000 7fcd75a19640 1 --2- 172.21.15.43:0/2442610875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd70166d10 0x7fcd7014b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:42.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.664+0000 7fcd75218640 1 --2- 172.21.15.43:0/2442610875 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcd7006be50 0x7fcd700c79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:42.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.665+0000 7fcd74a17640 1 --2- 172.21.15.43:0/2442610875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd7015def0 0x7fcd70158260 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7fcd60004640 tx=0x7fcd6000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:42.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.665+0000 7fcd65ffb640 1 -- 172.21.15.43:0/2442610875 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd60011790 con 0x7fcd7015def0 2024-04-25T09:04:42.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.665+0000 7fcd65ffb640 1 -- 172.21.15.43:0/2442610875 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd60017070 con 0x7fcd7015def0 2024-04-25T09:04:42.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.665+0000 7fcd7747c640 1 -- 172.21.15.43:0/2442610875 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd70159050 con 0x7fcd7015def0 2024-04-25T09:04:42.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.665+0000 7fcd65ffb640 1 -- 172.21.15.43:0/2442610875 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd60011bb0 con 0x7fcd7015def0 2024-04-25T09:04:42.655 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.665+0000 7fcd7747c640 1 -- 172.21.15.43:0/2442610875 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd70159610 con 0x7fcd7015def0 2024-04-25T09:04:42.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.666+0000 7fcd7747c640 1 -- 172.21.15.43:0/2442610875 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd70159810 con 0x7fcd7015def0 2024-04-25T09:04:42.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.666+0000 7fcd65ffb640 1 -- 172.21.15.43:0/2442610875 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcd60011d50 con 0x7fcd7015def0 2024-04-25T09:04:42.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.667+0000 7fcd65ffb640 1 --2- 172.21.15.43:0/2442610875 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcd5406b4d0 0x7fcd5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:42.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.667+0000 7fcd65ffb640 1 -- 172.21.15.43:0/2442610875 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd60089de0 con 0x7fcd7015def0 2024-04-25T09:04:42.656 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.667+0000 7fcd75218640 1 --2- 172.21.15.43:0/2442610875 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcd5406b4d0 0x7fcd5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:42.657 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.668+0000 7fcd75218640 1 --2- 172.21.15.43:0/2442610875 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcd5406b4d0 0x7fcd5406d990 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fcd7006fc90 tx=0x7fcd6c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:42.659 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.671+0000 7fcd65ffb640 1 -- 172.21.15.43:0/2442610875 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcd60058be0 con 0x7fcd7015def0 2024-04-25T09:04:42.839 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.851+0000 7fcd7747c640 1 -- 172.21.15.43:0/2442610875 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcd70074ef0 con 0x7fcd7015def0 2024-04-25T09:04:42.840 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.852+0000 7fcd65ffb640 1 -- 172.21.15.43:0/2442610875 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+16643 (secure 0 0 0) 0x7fcd600584e0 con 0x7fcd7015def0 2024-04-25T09:04:42.841 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:04:42.841 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":139,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:04:15.059128+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:04:11.735785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T09:04:12.383453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:04:42.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.854+0000 7fcd7747c640 1 -- 172.21.15.43:0/2442610875 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcd5406b4d0 msgr2=0x7fcd5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:42.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.854+0000 7fcd7747c640 1 --2- 172.21.15.43:0/2442610875 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcd5406b4d0 0x7fcd5406d990 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fcd7006fc90 tx=0x7fcd6c009040 comp rx=0 tx=0).stop 2024-04-25T09:04:42.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.854+0000 7fcd7747c640 1 -- 172.21.15.43:0/2442610875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd7015def0 msgr2=0x7fcd70158260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:42.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.854+0000 7fcd7747c640 1 --2- 172.21.15.43:0/2442610875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd7015def0 0x7fcd70158260 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7fcd60004640 tx=0x7fcd6000c8e0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.854+0000 7fcd7747c640 1 -- 172.21.15.43:0/2442610875 shutdown_connections 2024-04-25T09:04:42.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.854+0000 7fcd7747c640 1 --2- 172.21.15.43:0/2442610875 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcd5406b4d0 0x7fcd5406d990 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.854+0000 7fcd7747c640 1 --2- 172.21.15.43:0/2442610875 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcd70166d10 0x7fcd7014b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.854+0000 7fcd7747c640 1 --2- 172.21.15.43:0/2442610875 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcd7015def0 0x7fcd70158260 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.854+0000 7fcd7747c640 1 --2- 172.21.15.43:0/2442610875 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcd7006be50 0x7fcd700c79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.854+0000 7fcd7747c640 1 -- 172.21.15.43:0/2442610875 >> 172.21.15.43:0/2442610875 conn(0x7fcd700b32e0 msgr2=0x7fcd7015c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:42.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.854+0000 7fcd7747c640 1 -- 172.21.15.43:0/2442610875 shutdown_connections 2024-04-25T09:04:42.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.854+0000 7fcd7747c640 1 -- 172.21.15.43:0/2442610875 wait complete. 2024-04-25T09:04:42.854 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T08:53:59.739591+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T09:04:11.735785+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '138', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T09:04:12.383453+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '139', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T09:04:42.854 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T09:04:42.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.970+0000 7fc73595c640 1 Processor -- start 2024-04-25T09:04:42.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.970+0000 7fc73595c640 1 -- start start 2024-04-25T09:04:42.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.971+0000 7fc73595c640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fc730161310 0x7fc7300d4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:42.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.971+0000 7fc73595c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc7300d0c70 0x7fc7300c7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:42.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.971+0000 7fc73595c640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7300d45c0 con 0x7fc7300d0120 2024-04-25T09:04:42.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.971+0000 7fc73595c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7300d4870 con 0x7fc7300d0c70 2024-04-25T09:04:42.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.971+0000 7fc73595c640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7300cf030 con 0x7fc730161310 2024-04-25T09:04:42.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.971+0000 7fc72f7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc7300d0c70 0x7fc7300c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:42.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.971+0000 7fc72f7fe640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fc7300d0c70 0x7fc7300c7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:59998/0 (socket says 172.21.15.43:59998) 2024-04-25T09:04:42.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.971+0000 7fc72f7fe640 1 -- 172.21.15.43:0/375553296 learned_addr learned my addr 172.21.15.43:0/375553296 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:42.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.971+0000 7fc72effd640 1 --2- 172.21.15.43:0/375553296 >> v2:172.21.15.193:3300/0 conn(0x7fc730161310 0x7fc7300d4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:42.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.971+0000 7fc72f7fe640 1 -- 172.21.15.43:0/375553296 >> v2:172.21.15.193:3300/0 conn(0x7fc730161310 msgr2=0x7fc7300d4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:42.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.971+0000 7fc72f7fe640 1 --2- 172.21.15.43:0/375553296 >> v2:172.21.15.193:3300/0 conn(0x7fc730161310 0x7fc7300d4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.971+0000 7fc72f7fe640 1 -- 172.21.15.43:0/375553296 >> v1:172.21.15.193:6789/0 conn(0x7fc7300d0120 legacy=0x7fc7300d0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:42.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.971+0000 7fc72f7fe640 1 -- 172.21.15.43:0/375553296 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7300cf310 con 0x7fc7300d0c70 2024-04-25T09:04:42.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.972+0000 7fc72f7fe640 1 --2- 172.21.15.43:0/375553296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7300d0c70 0x7fc7300c7860 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7fc73008d2f0 tx=0x7fc72802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53e98be8441bc761 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:42.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.972+0000 7fc72dffb640 1 -- 172.21.15.43:0/375553296 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc728036070 con 0x7fc7300d0c70 2024-04-25T09:04:42.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.972+0000 7fc72dffb640 1 -- 172.21.15.43:0/375553296 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc728031600 con 0x7fc7300d0c70 2024-04-25T09:04:42.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.972+0000 7fc72dffb640 1 -- 172.21.15.43:0/375553296 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7280319d0 con 0x7fc7300d0c70 2024-04-25T09:04:42.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.973+0000 7fc73595c640 1 -- 172.21.15.43:0/375553296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7300d0c70 msgr2=0x7fc7300c7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:42.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.973+0000 7fc73595c640 1 --2- 172.21.15.43:0/375553296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7300d0c70 0x7fc7300c7860 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7fc73008d2f0 tx=0x7fc72802a730 comp rx=0 tx=0).stop 2024-04-25T09:04:42.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.973+0000 7fc73595c640 1 -- 172.21.15.43:0/375553296 shutdown_connections 2024-04-25T09:04:42.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.973+0000 7fc73595c640 1 --2- 172.21.15.43:0/375553296 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7300d0c70 0x7fc7300c7860 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.973+0000 7fc73595c640 1 --2- 172.21.15.43:0/375553296 >> v2:172.21.15.193:3300/0 conn(0x7fc730161310 0x7fc7300d4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.973+0000 7fc73595c640 1 -- 172.21.15.43:0/375553296 >> 172.21.15.43:0/375553296 conn(0x7fc73009cd20 msgr2=0x7fc7300d3970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:42.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.973+0000 7fc73595c640 1 -- 172.21.15.43:0/375553296 shutdown_connections 2024-04-25T09:04:42.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.973+0000 7fc73595c640 1 -- 172.21.15.43:0/375553296 wait complete. 2024-04-25T09:04:42.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.974+0000 7fc73595c640 1 Processor -- start 2024-04-25T09:04:42.963 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.974+0000 7fc73595c640 1 -- start start 2024-04-25T09:04:42.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.975+0000 7fc73595c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7300d0120 0x7fc7301f2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:42.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.975+0000 7fc73595c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7300d0c70 0x7fc7300b6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:42.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.975+0000 7fc73595c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc730161310 0x7fc7300b84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:42.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.975+0000 7fc73595c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7300c7da0 con 0x7fc7300d0c70 2024-04-25T09:04:42.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.975+0000 7fc73595c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7301616f0 con 0x7fc7300d0120 2024-04-25T09:04:42.966 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.975+0000 7fc73595c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7300c7f20 con 0x7fc730161310 2024-04-25T09:04:42.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.975+0000 7fc72e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7300d0c70 0x7fc7300b6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:42.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.975+0000 7fc72e7fc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7300d0c70 0x7fc7300b6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60000/0 (socket says 172.21.15.43:60000) 2024-04-25T09:04:42.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.975+0000 7fc72e7fc640 1 -- 172.21.15.43:0/3012098136 learned_addr learned my addr 172.21.15.43:0/3012098136 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:42.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.975+0000 7fc72effd640 1 --2- 172.21.15.43:0/3012098136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7300d0120 0x7fc7301f2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:42.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.975+0000 7fc72f7fe640 1 --2- 172.21.15.43:0/3012098136 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc730161310 0x7fc7300b84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:42.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.975+0000 7fc72e7fc640 1 -- 172.21.15.43:0/3012098136 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc730161310 msgr2=0x7fc7300b84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:42.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.975+0000 7fc72e7fc640 1 --2- 172.21.15.43:0/3012098136 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc730161310 0x7fc7300b84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.975+0000 7fc72e7fc640 1 -- 172.21.15.43:0/3012098136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7300d0120 msgr2=0x7fc7301f2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:42.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.975+0000 7fc72e7fc640 1 --2- 172.21.15.43:0/3012098136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7300d0120 0x7fc7301f2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:42.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.975+0000 7fc72e7fc640 1 -- 172.21.15.43:0/3012098136 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7301f3470 con 0x7fc7300d0c70 2024-04-25T09:04:42.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.975+0000 7fc72effd640 1 --2- 172.21.15.43:0/3012098136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7300d0120 0x7fc7301f2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:42.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.975+0000 7fc72f7fe640 1 --2- 172.21.15.43:0/3012098136 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc730161310 0x7fc7300b84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:42.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.975+0000 7fc72e7fc640 1 --2- 172.21.15.43:0/3012098136 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7300d0c70 0x7fc7300b6f70 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7fc72400c430 tx=0x7fc72400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:42.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.976+0000 7fc717fff640 1 -- 172.21.15.43:0/3012098136 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7240052d0 con 0x7fc7300d0c70 2024-04-25T09:04:42.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.976+0000 7fc717fff640 1 -- 172.21.15.43:0/3012098136 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc724005490 con 0x7fc7300d0c70 2024-04-25T09:04:42.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.976+0000 7fc717fff640 1 -- 172.21.15.43:0/3012098136 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc724007260 con 0x7fc7300d0c70 2024-04-25T09:04:42.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.976+0000 7fc73595c640 1 -- 172.21.15.43:0/3012098136 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7300b77d0 con 0x7fc7300d0c70 2024-04-25T09:04:42.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.976+0000 7fc73595c640 1 -- 172.21.15.43:0/3012098136 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7300cbe10 con 0x7fc7300d0c70 2024-04-25T09:04:42.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.977+0000 7fc73595c640 1 -- 172.21.15.43:0/3012098136 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc730074f60 con 0x7fc7300d0c70 2024-04-25T09:04:42.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.977+0000 7fc717fff640 1 -- 172.21.15.43:0/3012098136 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc724007400 con 0x7fc7300d0c70 2024-04-25T09:04:42.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.978+0000 7fc717fff640 1 --2- 172.21.15.43:0/3012098136 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc71c06b4d0 0x7fc71c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:42.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.978+0000 7fc717fff640 1 -- 172.21.15.43:0/3012098136 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc7240897e0 con 0x7fc7300d0c70 2024-04-25T09:04:42.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.978+0000 7fc72effd640 1 --2- 172.21.15.43:0/3012098136 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc71c06b4d0 0x7fc71c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:42.968 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.978+0000 7fc72effd640 1 --2- 172.21.15.43:0/3012098136 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc71c06b4d0 0x7fc71c06d990 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fc720006fd0 tx=0x7fc720008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:42.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:42.982+0000 7fc717fff640 1 -- 172.21.15.43:0/3012098136 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc7240585e0 con 0x7fc7300d0c70 2024-04-25T09:04:43.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.162+0000 7fc73595c640 1 -- 172.21.15.43:0/3012098136 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc7300c9da0 con 0x7fc7300d0c70 2024-04-25T09:04:43.151 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.163+0000 7fc717fff640 1 -- 172.21.15.43:0/3012098136 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v139) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc724057ee0 con 0x7fc7300d0c70 2024-04-25T09:04:43.151 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 1 2024-04-25T09:04:43.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.165+0000 7fc73595c640 1 -- 172.21.15.43:0/3012098136 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc71c06b4d0 msgr2=0x7fc71c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:43.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.165+0000 7fc73595c640 1 --2- 172.21.15.43:0/3012098136 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc71c06b4d0 0x7fc71c06d990 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fc720006fd0 tx=0x7fc720008040 comp rx=0 tx=0).stop 2024-04-25T09:04:43.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.165+0000 7fc73595c640 1 -- 172.21.15.43:0/3012098136 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7300d0c70 msgr2=0x7fc7300b6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:43.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.165+0000 7fc73595c640 1 --2- 172.21.15.43:0/3012098136 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7300d0c70 0x7fc7300b6f70 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7fc72400c430 tx=0x7fc72400c8f0 comp rx=0 tx=0).stop 2024-04-25T09:04:43.153 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.165+0000 7fc73595c640 1 -- 172.21.15.43:0/3012098136 shutdown_connections 2024-04-25T09:04:43.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.165+0000 7fc73595c640 1 --2- 172.21.15.43:0/3012098136 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fc730161310 0x7fc7300b84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:43.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.165+0000 7fc73595c640 1 --2- 172.21.15.43:0/3012098136 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fc7300d0c70 0x7fc7300b6f70 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:43.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.165+0000 7fc73595c640 1 --2- 172.21.15.43:0/3012098136 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fc71c06b4d0 0x7fc71c06d990 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:43.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.165+0000 7fc73595c640 1 --2- 172.21.15.43:0/3012098136 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fc7300d0120 0x7fc7301f2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:43.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.165+0000 7fc73595c640 1 -- 172.21.15.43:0/3012098136 >> 172.21.15.43:0/3012098136 conn(0x7fc73009cd20 msgr2=0x7fc7300c9260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:43.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.165+0000 7fc73595c640 1 -- 172.21.15.43:0/3012098136 shutdown_connections 2024-04-25T09:04:43.154 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.165+0000 7fc73595c640 1 -- 172.21.15.43:0/3012098136 wait complete. 2024-04-25T09:04:43.165 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T09:04:43.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.281+0000 7f7e733b0640 1 Processor -- start 2024-04-25T09:04:43.269 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.281+0000 7f7e733b0640 1 -- start start 2024-04-25T09:04:43.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.282+0000 7f7e733b0640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7e6c071650 0x7f7e6c071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:43.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.282+0000 7f7e733b0640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f7e6c0cd0e0 0x7f7e6c0cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:43.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.282+0000 7f7e733b0640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e6c0b3160 con 0x7f7e6c15a1d0 2024-04-25T09:04:43.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.282+0000 7f7e733b0640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e6c071ff0 con 0x7f7e6c071650 2024-04-25T09:04:43.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.282+0000 7f7e733b0640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e6c0b3ff0 con 0x7f7e6c0cd0e0 2024-04-25T09:04:43.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.282+0000 7f7e7094b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7e6c071650 0x7f7e6c071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:43.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.282+0000 7f7e7094b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f7e6c071650 0x7f7e6c071ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60012/0 (socket says 172.21.15.43:60012) 2024-04-25T09:04:43.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.282+0000 7f7e7094b640 1 -- 172.21.15.43:0/4292105295 learned_addr learned my addr 172.21.15.43:0/4292105295 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:43.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.282+0000 7f7e7094b640 1 -- 172.21.15.43:0/4292105295 >> v2:172.21.15.193:3300/0 conn(0x7f7e6c0cd0e0 msgr2=0x7f7e6c0cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:43.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.282+0000 7f7e7194d640 1 --2- 172.21.15.43:0/4292105295 >> v2:172.21.15.193:3300/0 conn(0x7f7e6c0cd0e0 0x7f7e6c0cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:43.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.282+0000 7f7e7094b640 1 --2- 172.21.15.43:0/4292105295 >> v2:172.21.15.193:3300/0 conn(0x7f7e6c0cd0e0 0x7f7e6c0cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:43.271 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.282+0000 7f7e7094b640 1 -- 172.21.15.43:0/4292105295 >> v1:172.21.15.193:6789/0 conn(0x7f7e6c15a1d0 legacy=0x7f7e6c06d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:43.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.282+0000 7f7e7094b640 1 -- 172.21.15.43:0/4292105295 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e6c15a5b0 con 0x7f7e6c071650 2024-04-25T09:04:43.272 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.282+0000 7f7e7194d640 1 --2- 172.21.15.43:0/4292105295 >> v2:172.21.15.193:3300/0 conn(0x7f7e6c0cd0e0 0x7f7e6c0cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:43.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.283+0000 7f7e7094b640 1 --2- 172.21.15.43:0/4292105295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e6c071650 0x7f7e6c071ab0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c0b4170 tx=0x7f7e5c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=479168811883884c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:43.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.283+0000 7f7e63fff640 1 -- 172.21.15.43:0/4292105295 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c037070 con 0x7f7e6c071650 2024-04-25T09:04:43.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.283+0000 7f7e63fff640 1 -- 172.21.15.43:0/4292105295 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5c032440 con 0x7f7e6c071650 2024-04-25T09:04:43.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.283+0000 7f7e63fff640 1 -- 172.21.15.43:0/4292105295 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c032810 con 0x7f7e6c071650 2024-04-25T09:04:43.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.284+0000 7f7e733b0640 1 -- 172.21.15.43:0/4292105295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e6c071650 msgr2=0x7f7e6c071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:43.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.284+0000 7f7e733b0640 1 --2- 172.21.15.43:0/4292105295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e6c071650 0x7f7e6c071ab0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7f7e6c0b4170 tx=0x7f7e5c02a210 comp rx=0 tx=0).stop 2024-04-25T09:04:43.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.284+0000 7f7e733b0640 1 -- 172.21.15.43:0/4292105295 shutdown_connections 2024-04-25T09:04:43.273 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.284+0000 7f7e733b0640 1 --2- 172.21.15.43:0/4292105295 >> v2:172.21.15.193:3300/0 conn(0x7f7e6c0cd0e0 0x7f7e6c0cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:43.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.284+0000 7f7e733b0640 1 --2- 172.21.15.43:0/4292105295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e6c071650 0x7f7e6c071ab0 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:43.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.284+0000 7f7e733b0640 1 -- 172.21.15.43:0/4292105295 >> 172.21.15.43:0/4292105295 conn(0x7f7e6c06ab30 msgr2=0x7f7e6c06af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:43.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.284+0000 7f7e733b0640 1 -- 172.21.15.43:0/4292105295 shutdown_connections 2024-04-25T09:04:43.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.284+0000 7f7e733b0640 1 -- 172.21.15.43:0/4292105295 wait complete. 2024-04-25T09:04:43.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.285+0000 7f7e733b0640 1 Processor -- start 2024-04-25T09:04:43.274 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.285+0000 7f7e733b0640 1 -- start start 2024-04-25T09:04:43.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.286+0000 7f7e733b0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e6c071650 0x7f7e6c14dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:43.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.286+0000 7f7e733b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e6c0cd0e0 0x7f7e6c150480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:43.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.286+0000 7f7e733b0640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7e6c15a1d0 0x7f7e6c14a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:43.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.286+0000 7f7e733b0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c0b39a0 con 0x7f7e6c0cd0e0 2024-04-25T09:04:43.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.286+0000 7f7e733b0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c0b36c0 con 0x7f7e6c071650 2024-04-25T09:04:43.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.286+0000 7f7e733b0640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c0b3c80 con 0x7f7e6c15a1d0 2024-04-25T09:04:43.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.286+0000 7f7e7094b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e6c0cd0e0 0x7f7e6c150480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:43.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.286+0000 7f7e7094b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e6c0cd0e0 0x7f7e6c150480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60024/0 (socket says 172.21.15.43:60024) 2024-04-25T09:04:43.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.286+0000 7f7e7094b640 1 -- 172.21.15.43:0/2172876773 learned_addr learned my addr 172.21.15.43:0/2172876773 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:43.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.286+0000 7f7e7114c640 1 --2- 172.21.15.43:0/2172876773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e6c071650 0x7f7e6c14dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:43.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.286+0000 7f7e7194d640 1 --2- 172.21.15.43:0/2172876773 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7e6c15a1d0 0x7f7e6c14a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:43.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.286+0000 7f7e7094b640 1 -- 172.21.15.43:0/2172876773 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7e6c15a1d0 msgr2=0x7f7e6c14a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:43.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.286+0000 7f7e7094b640 1 --2- 172.21.15.43:0/2172876773 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7e6c15a1d0 0x7f7e6c14a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:43.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.286+0000 7f7e7094b640 1 -- 172.21.15.43:0/2172876773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e6c071650 msgr2=0x7f7e6c14dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:43.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.286+0000 7f7e7094b640 1 --2- 172.21.15.43:0/2172876773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e6c071650 0x7f7e6c14dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:43.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.286+0000 7f7e7094b640 1 -- 172.21.15.43:0/2172876773 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e6c14af10 con 0x7f7e6c0cd0e0 2024-04-25T09:04:43.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.286+0000 7f7e7114c640 1 --2- 172.21.15.43:0/2172876773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e6c071650 0x7f7e6c14dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:43.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.286+0000 7f7e7094b640 1 --2- 172.21.15.43:0/2172876773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e6c0cd0e0 0x7f7e6c150480 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c000c00 tx=0x7f7e5c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:43.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.286+0000 7f7e61ffb640 1 -- 172.21.15.43:0/2172876773 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c037040 con 0x7f7e6c0cd0e0 2024-04-25T09:04:43.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.286+0000 7f7e61ffb640 1 -- 172.21.15.43:0/2172876773 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5c042070 con 0x7f7e6c0cd0e0 2024-04-25T09:04:43.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.286+0000 7f7e61ffb640 1 -- 172.21.15.43:0/2172876773 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c030bb0 con 0x7f7e6c0cd0e0 2024-04-25T09:04:43.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.286+0000 7f7e733b0640 1 -- 172.21.15.43:0/2172876773 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e6c154740 con 0x7f7e6c0cd0e0 2024-04-25T09:04:43.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.287+0000 7f7e733b0640 1 -- 172.21.15.43:0/2172876773 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e6c154d00 con 0x7f7e6c0cd0e0 2024-04-25T09:04:43.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.288+0000 7f7e733b0640 1 -- 172.21.15.43:0/2172876773 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e6c074f60 con 0x7f7e6c0cd0e0 2024-04-25T09:04:43.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.288+0000 7f7e61ffb640 1 -- 172.21.15.43:0/2172876773 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7e5c047460 con 0x7f7e6c0cd0e0 2024-04-25T09:04:43.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.289+0000 7f7e61ffb640 1 --2- 172.21.15.43:0/2172876773 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7e5006b500 0x7f7e5006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:43.278 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.289+0000 7f7e61ffb640 1 -- 172.21.15.43:0/2172876773 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e5c0aadf0 con 0x7f7e6c0cd0e0 2024-04-25T09:04:43.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.289+0000 7f7e7114c640 1 --2- 172.21.15.43:0/2172876773 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7e5006b500 0x7f7e5006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:43.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.290+0000 7f7e7114c640 1 --2- 172.21.15.43:0/2172876773 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7e5006b500 0x7f7e5006d9c0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f7e68006fd0 tx=0x7f7e68008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:43.281 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.293+0000 7f7e61ffb640 1 -- 172.21.15.43:0/2172876773 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7e6c0b3160 con 0x7f7e6c0cd0e0 2024-04-25T09:04:43.462 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.474+0000 7f7e733b0640 1 -- 172.21.15.43:0/2172876773 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7e6c0cdcd0 con 0x7f7e6c0cd0e0 2024-04-25T09:04:43.463 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.474+0000 7f7e61ffb640 1 -- 172.21.15.43:0/2172876773 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v139) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7e5c045360 con 0x7f7e6c0cd0e0 2024-04-25T09:04:43.463 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 32 2024-04-25T09:04:43.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.476+0000 7f7e733b0640 1 -- 172.21.15.43:0/2172876773 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7e5006b500 msgr2=0x7f7e5006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:43.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.476+0000 7f7e733b0640 1 --2- 172.21.15.43:0/2172876773 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7e5006b500 0x7f7e5006d9c0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f7e68006fd0 tx=0x7f7e68008040 comp rx=0 tx=0).stop 2024-04-25T09:04:43.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.476+0000 7f7e733b0640 1 -- 172.21.15.43:0/2172876773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e6c0cd0e0 msgr2=0x7f7e6c150480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:43.465 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.476+0000 7f7e733b0640 1 --2- 172.21.15.43:0/2172876773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e6c0cd0e0 0x7f7e6c150480 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c000c00 tx=0x7f7e5c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:04:43.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.477+0000 7f7e733b0640 1 -- 172.21.15.43:0/2172876773 shutdown_connections 2024-04-25T09:04:43.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.477+0000 7f7e733b0640 1 --2- 172.21.15.43:0/2172876773 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7e6c15a1d0 0x7f7e6c14a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:43.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.477+0000 7f7e733b0640 1 --2- 172.21.15.43:0/2172876773 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7e6c0cd0e0 0x7f7e6c150480 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:43.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.477+0000 7f7e733b0640 1 --2- 172.21.15.43:0/2172876773 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f7e5006b500 0x7f7e5006d9c0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:43.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.477+0000 7f7e733b0640 1 --2- 172.21.15.43:0/2172876773 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f7e6c071650 0x7f7e6c14dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:43.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.477+0000 7f7e733b0640 1 -- 172.21.15.43:0/2172876773 >> 172.21.15.43:0/2172876773 conn(0x7f7e6c06ab30 msgr2=0x7f7e6c0ce1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:43.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.477+0000 7f7e733b0640 1 -- 172.21.15.43:0/2172876773 shutdown_connections 2024-04-25T09:04:43.466 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.477+0000 7f7e733b0640 1 -- 172.21.15.43:0/2172876773 wait complete. 2024-04-25T09:04:43.476 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T09:04:43.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.593+0000 7f20f54bd640 1 Processor -- start 2024-04-25T09:04:43.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.593+0000 7f20f54bd640 1 -- start start 2024-04-25T09:04:43.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.594+0000 7f20f54bd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f20f015e090 0x7f20f0161aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:43.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.594+0000 7f20f54bd640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f20f00b5c70 0x7f20f00d2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:43.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.594+0000 7f20f54bd640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20f0162610 con 0x7f20f0161fe0 2024-04-25T09:04:43.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.594+0000 7f20f54bd640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20f00d1da0 con 0x7f20f015e090 2024-04-25T09:04:43.583 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.594+0000 7f20f54bd640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20f00d1750 con 0x7f20f00b5c70 2024-04-25T09:04:43.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.594+0000 7f20eeffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f20f015e090 0x7f20f0161aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:43.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.594+0000 7f20eeffd640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f20f015e090 0x7f20f0161aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60032/0 (socket says 172.21.15.43:60032) 2024-04-25T09:04:43.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.594+0000 7f20eeffd640 1 -- 172.21.15.43:0/236877693 learned_addr learned my addr 172.21.15.43:0/236877693 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:43.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.594+0000 7f20ef7fe640 1 --2- 172.21.15.43:0/236877693 >> v2:172.21.15.193:3300/0 conn(0x7f20f00b5c70 0x7f20f00d2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:43.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.594+0000 7f20eeffd640 1 -- 172.21.15.43:0/236877693 >> v2:172.21.15.193:3300/0 conn(0x7f20f00b5c70 msgr2=0x7f20f00d2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:43.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.594+0000 7f20eeffd640 1 --2- 172.21.15.43:0/236877693 >> v2:172.21.15.193:3300/0 conn(0x7f20f00b5c70 0x7f20f00d2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:43.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.594+0000 7f20eeffd640 1 -- 172.21.15.43:0/236877693 >> v1:172.21.15.193:6789/0 conn(0x7f20f0161fe0 legacy=0x7f20f00b5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:43.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.594+0000 7f20eeffd640 1 -- 172.21.15.43:0/236877693 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f00d1a30 con 0x7f20f015e090 2024-04-25T09:04:43.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.594+0000 7f20ef7fe640 1 --2- 172.21.15.43:0/236877693 >> v2:172.21.15.193:3300/0 conn(0x7f20f00b5c70 0x7f20f00d2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:43.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.595+0000 7f20eeffd640 1 --2- 172.21.15.43:0/236877693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20f015e090 0x7f20f0161aa0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f20f01619e0 tx=0x7f20e002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=666ca132b1e52167 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:43.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.595+0000 7f20edffb640 1 -- 172.21.15.43:0/236877693 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e0037070 con 0x7f20f015e090 2024-04-25T09:04:43.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.595+0000 7f20edffb640 1 -- 172.21.15.43:0/236877693 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e002ae50 con 0x7f20f015e090 2024-04-25T09:04:43.584 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.595+0000 7f20edffb640 1 -- 172.21.15.43:0/236877693 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e0032860 con 0x7f20f015e090 2024-04-25T09:04:43.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.595+0000 7f20f54bd640 1 -- 172.21.15.43:0/236877693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20f015e090 msgr2=0x7f20f0161aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:43.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.595+0000 7f20f54bd640 1 --2- 172.21.15.43:0/236877693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20f015e090 0x7f20f0161aa0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7f20f01619e0 tx=0x7f20e002a300 comp rx=0 tx=0).stop 2024-04-25T09:04:43.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.596+0000 7f20f54bd640 1 -- 172.21.15.43:0/236877693 shutdown_connections 2024-04-25T09:04:43.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.596+0000 7f20f54bd640 1 --2- 172.21.15.43:0/236877693 >> v2:172.21.15.193:3300/0 conn(0x7f20f00b5c70 0x7f20f00d2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:43.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.596+0000 7f20f54bd640 1 --2- 172.21.15.43:0/236877693 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20f015e090 0x7f20f0161aa0 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:43.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.596+0000 7f20f54bd640 1 -- 172.21.15.43:0/236877693 >> 172.21.15.43:0/236877693 conn(0x7f20f00d0560 msgr2=0x7f20f00d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:43.587 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.596+0000 7f20f54bd640 1 -- 172.21.15.43:0/236877693 shutdown_connections 2024-04-25T09:04:43.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.596+0000 7f20f54bd640 1 -- 172.21.15.43:0/236877693 wait complete. 2024-04-25T09:04:43.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.597+0000 7f20f54bd640 1 Processor -- start 2024-04-25T09:04:43.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.597+0000 7f20f54bd640 1 -- start start 2024-04-25T09:04:43.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.598+0000 7f20f54bd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20f00b5c70 0x7f20f0152d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:43.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.598+0000 7f20f54bd640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20f015e090 0x7f20f0158fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:43.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.598+0000 7f20f54bd640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20f0161fe0 0x7f20f0155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:43.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.598+0000 7f20f54bd640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f00d2770 con 0x7f20f00b5c70 2024-04-25T09:04:43.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.598+0000 7f20f54bd640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f00b3280 con 0x7f20f015e090 2024-04-25T09:04:43.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.598+0000 7f20f54bd640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20f00d28f0 con 0x7f20f0161fe0 2024-04-25T09:04:43.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.598+0000 7f20eeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20f00b5c70 0x7f20f0152d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:43.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.598+0000 7f20eeffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20f00b5c70 0x7f20f0152d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60040/0 (socket says 172.21.15.43:60040) 2024-04-25T09:04:43.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.598+0000 7f20eeffd640 1 -- 172.21.15.43:0/1684649058 learned_addr learned my addr 172.21.15.43:0/1684649058 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:43.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.598+0000 7f20ee7fc640 1 --2- 172.21.15.43:0/1684649058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20f015e090 0x7f20f0158fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:43.588 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.598+0000 7f20ef7fe640 1 --2- 172.21.15.43:0/1684649058 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20f0161fe0 0x7f20f0155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:43.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.598+0000 7f20eeffd640 1 -- 172.21.15.43:0/1684649058 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20f0161fe0 msgr2=0x7f20f0155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:43.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.598+0000 7f20eeffd640 1 --2- 172.21.15.43:0/1684649058 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20f0161fe0 0x7f20f0155240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:43.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.598+0000 7f20eeffd640 1 -- 172.21.15.43:0/1684649058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20f015e090 msgr2=0x7f20f0158fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:43.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.598+0000 7f20eeffd640 1 --2- 172.21.15.43:0/1684649058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20f015e090 0x7f20f0158fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:43.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.598+0000 7f20eeffd640 1 -- 172.21.15.43:0/1684649058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f0153680 con 0x7f20f00b5c70 2024-04-25T09:04:43.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.598+0000 7f20ee7fc640 1 --2- 172.21.15.43:0/1684649058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20f015e090 0x7f20f0158fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:43.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.598+0000 7f20ef7fe640 1 --2- 172.21.15.43:0/1684649058 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20f0161fe0 0x7f20f0155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:43.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.598+0000 7f20eeffd640 1 --2- 172.21.15.43:0/1684649058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20f00b5c70 0x7f20f0152d80 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f20e002a340 tx=0x7f20e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:43.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.598+0000 7f20d77fe640 1 -- 172.21.15.43:0/1684649058 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e0041070 con 0x7f20f00b5c70 2024-04-25T09:04:43.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.598+0000 7f20d77fe640 1 -- 172.21.15.43:0/1684649058 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e003c670 con 0x7f20f00b5c70 2024-04-25T09:04:43.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.598+0000 7f20d77fe640 1 -- 172.21.15.43:0/1684649058 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e0037040 con 0x7f20f00b5c70 2024-04-25T09:04:43.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.598+0000 7f20f54bd640 1 -- 172.21.15.43:0/1684649058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20f0153960 con 0x7f20f00b5c70 2024-04-25T09:04:43.589 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.598+0000 7f20f54bd640 1 -- 172.21.15.43:0/1684649058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20f0153f20 con 0x7f20f00b5c70 2024-04-25T09:04:43.590 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.599+0000 7f20f54bd640 1 -- 172.21.15.43:0/1684649058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20f006bf00 con 0x7f20f00b5c70 2024-04-25T09:04:43.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.600+0000 7f20d77fe640 1 -- 172.21.15.43:0/1684649058 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f20e0044070 con 0x7f20f00b5c70 2024-04-25T09:04:43.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.600+0000 7f20d77fe640 1 --2- 172.21.15.43:0/1684649058 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f20dc06b4d0 0x7f20dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:43.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.600+0000 7f20d77fe640 1 -- 172.21.15.43:0/1684649058 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20e00aa870 con 0x7f20f00b5c70 2024-04-25T09:04:43.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.600+0000 7f20ee7fc640 1 --2- 172.21.15.43:0/1684649058 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f20dc06b4d0 0x7f20dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:43.591 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.601+0000 7f20ee7fc640 1 --2- 172.21.15.43:0/1684649058 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f20dc06b4d0 0x7f20dc06d990 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f20f0161360 tx=0x7f20e4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:43.592 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.604+0000 7f20d77fe640 1 -- 172.21.15.43:0/1684649058 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f20f006bf00 con 0x7f20f00b5c70 2024-04-25T09:04:43.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.787+0000 7f20f54bd640 1 -- 172.21.15.43:0/1684649058 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f20f0074f60 con 0x7f20f00b5c70 2024-04-25T09:04:43.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.787+0000 7f20d77fe640 1 -- 172.21.15.43:0/1684649058 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v139) v1 ==== 100+0+11 (secure 0 0 0) 0x7f20e00796e0 con 0x7f20f00b5c70 2024-04-25T09:04:43.775 INFO:teuthology.orchestra.run.smithi043.stdout:pg_num: 64 2024-04-25T09:04:43.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.789+0000 7f20f54bd640 1 -- 172.21.15.43:0/1684649058 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f20dc06b4d0 msgr2=0x7f20dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:43.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.789+0000 7f20f54bd640 1 --2- 172.21.15.43:0/1684649058 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f20dc06b4d0 0x7f20dc06d990 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f20f0161360 tx=0x7f20e4009040 comp rx=0 tx=0).stop 2024-04-25T09:04:43.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.789+0000 7f20f54bd640 1 -- 172.21.15.43:0/1684649058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20f00b5c70 msgr2=0x7f20f0152d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:43.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.789+0000 7f20f54bd640 1 --2- 172.21.15.43:0/1684649058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20f00b5c70 0x7f20f0152d80 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7f20e002a340 tx=0x7f20e00046c0 comp rx=0 tx=0).stop 2024-04-25T09:04:43.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.789+0000 7f20f54bd640 1 -- 172.21.15.43:0/1684649058 shutdown_connections 2024-04-25T09:04:43.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.789+0000 7f20f54bd640 1 --2- 172.21.15.43:0/1684649058 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f20f0161fe0 0x7f20f0155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:43.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.789+0000 7f20f54bd640 1 --2- 172.21.15.43:0/1684649058 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f20dc06b4d0 0x7f20dc06d990 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:43.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.789+0000 7f20f54bd640 1 --2- 172.21.15.43:0/1684649058 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f20f015e090 0x7f20f0158fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:43.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.789+0000 7f20f54bd640 1 --2- 172.21.15.43:0/1684649058 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f20f00b5c70 0x7f20f0152d80 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:43.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.789+0000 7f20f54bd640 1 -- 172.21.15.43:0/1684649058 >> 172.21.15.43:0/1684649058 conn(0x7f20f00d0560 msgr2=0x7f20f00d2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:43.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.789+0000 7f20f54bd640 1 -- 172.21.15.43:0/1684649058 shutdown_connections 2024-04-25T09:04:43.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.789+0000 7f20f54bd640 1 -- 172.21.15.43:0/1684649058 wait complete. 2024-04-25T09:04:43.789 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:04:43.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.900+0000 7f1348763640 1 Processor -- start 2024-04-25T09:04:43.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.901+0000 7f1348763640 1 -- start start 2024-04-25T09:04:43.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.901+0000 7f1348763640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f134015dce0 0x7f134015ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:43.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.901+0000 7f1348763640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13400ce550 0x7f13400d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:43.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.901+0000 7f1348763640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13400cee80 con 0x7f13400ce990 2024-04-25T09:04:43.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.901+0000 7f1348763640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13400d2920 con 0x7f134015dce0 2024-04-25T09:04:43.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.901+0000 7f1348763640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13400cf160 con 0x7f13400ce550 2024-04-25T09:04:43.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.902+0000 7f13464ff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f134015dce0 0x7f134015ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:43.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.902+0000 7f13464ff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f134015dce0 0x7f134015ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60052/0 (socket says 172.21.15.43:60052) 2024-04-25T09:04:43.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.902+0000 7f13464ff640 1 -- 172.21.15.43:0/1706951850 learned_addr learned my addr 172.21.15.43:0/1706951850 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:43.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.902+0000 7f1345cfe640 1 --2- 172.21.15.43:0/1706951850 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13400ce550 0x7f13400d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:43.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.902+0000 7f13464ff640 1 -- 172.21.15.43:0/1706951850 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13400ce550 msgr2=0x7f13400d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:43.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.902+0000 7f13464ff640 1 --2- 172.21.15.43:0/1706951850 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13400ce550 0x7f13400d03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:43.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.902+0000 7f13464ff640 1 -- 172.21.15.43:0/1706951850 >> v1:172.21.15.43:6789/0 conn(0x7f13400ce990 legacy=0x7f1340160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:43.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.902+0000 7f13464ff640 1 -- 172.21.15.43:0/1706951850 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13400cf440 con 0x7f134015dce0 2024-04-25T09:04:43.891 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.902+0000 7f13454fd640 1 -- 172.21.15.43:0/1706951850 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2436013581 0 0) 0x7f13400cee80 con 0x7f13400ce990 2024-04-25T09:04:43.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.903+0000 7f13464ff640 1 --2- 172.21.15.43:0/1706951850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f134015dce0 0x7f134015ef50 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f133c009870 tx=0x7f133c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea9b40dec3165e99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:43.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.903+0000 7f13454fd640 1 -- 172.21.15.43:0/1706951850 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133c037070 con 0x7f134015dce0 2024-04-25T09:04:43.892 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.903+0000 7f13454fd640 1 -- 172.21.15.43:0/1706951850 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f133c032440 con 0x7f134015dce0 2024-04-25T09:04:43.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.903+0000 7f13454fd640 1 -- 172.21.15.43:0/1706951850 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133c032810 con 0x7f134015dce0 2024-04-25T09:04:43.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.903+0000 7f1348763640 1 -- 172.21.15.43:0/1706951850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f134015dce0 msgr2=0x7f134015ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:43.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.903+0000 7f1348763640 1 --2- 172.21.15.43:0/1706951850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f134015dce0 0x7f134015ef50 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f133c009870 tx=0x7f133c02a1c0 comp rx=0 tx=0).stop 2024-04-25T09:04:43.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.904+0000 7f1348763640 1 -- 172.21.15.43:0/1706951850 shutdown_connections 2024-04-25T09:04:43.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.904+0000 7f1348763640 1 --2- 172.21.15.43:0/1706951850 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13400ce550 0x7f13400d03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:43.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.904+0000 7f1348763640 1 --2- 172.21.15.43:0/1706951850 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f134015dce0 0x7f134015ef50 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:43.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.904+0000 7f1348763640 1 -- 172.21.15.43:0/1706951850 >> 172.21.15.43:0/1706951850 conn(0x7f134009cd00 msgr2=0x7f13400d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:43.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.904+0000 7f1348763640 1 -- 172.21.15.43:0/1706951850 shutdown_connections 2024-04-25T09:04:43.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.904+0000 7f1348763640 1 -- 172.21.15.43:0/1706951850 wait complete. 2024-04-25T09:04:43.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.905+0000 7f1348763640 1 Processor -- start 2024-04-25T09:04:43.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.905+0000 7f1348763640 1 -- start start 2024-04-25T09:04:43.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.905+0000 7f1348763640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13400ce550 0x7f13400a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:43.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.905+0000 7f1348763640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13400ce990 0x7f134015ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:43.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.905+0000 7f1348763640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f134015dce0 0x7f134015ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:43.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.905+0000 7f1348763640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134015e0c0 con 0x7f134015dce0 2024-04-25T09:04:43.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.905+0000 7f1348763640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134015f6b0 con 0x7f13400ce550 2024-04-25T09:04:43.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.905+0000 7f1348763640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13400d0920 con 0x7f13400ce990 2024-04-25T09:04:43.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.906+0000 7f1346d00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f134015dce0 0x7f134015ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:43.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.906+0000 7f1346d00640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f134015dce0 0x7f134015ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60054/0 (socket says 172.21.15.43:60054) 2024-04-25T09:04:43.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.906+0000 7f1346d00640 1 -- 172.21.15.43:0/2794779834 learned_addr learned my addr 172.21.15.43:0/2794779834 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:43.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.906+0000 7f13464ff640 1 --2- 172.21.15.43:0/2794779834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13400ce550 0x7f13400a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:43.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.906+0000 7f1345cfe640 1 --2- 172.21.15.43:0/2794779834 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13400ce990 0x7f134015ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:43.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.906+0000 7f1346d00640 1 -- 172.21.15.43:0/2794779834 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13400ce990 msgr2=0x7f134015ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:43.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.906+0000 7f1346d00640 1 --2- 172.21.15.43:0/2794779834 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13400ce990 0x7f134015ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:43.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.906+0000 7f1346d00640 1 -- 172.21.15.43:0/2794779834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13400ce550 msgr2=0x7f13400a01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:43.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.906+0000 7f1346d00640 1 --2- 172.21.15.43:0/2794779834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13400ce550 0x7f13400a01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:43.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.906+0000 7f1346d00640 1 -- 172.21.15.43:0/2794779834 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13400a06f0 con 0x7f134015dce0 2024-04-25T09:04:43.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.906+0000 7f1346d00640 1 --2- 172.21.15.43:0/2794779834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f134015dce0 0x7f134015ef40 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f133800c760 tx=0x7f133800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:43.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.906+0000 7f1336ffd640 1 -- 172.21.15.43:0/2794779834 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1338003da0 con 0x7f134015dce0 2024-04-25T09:04:43.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.906+0000 7f1336ffd640 1 -- 172.21.15.43:0/2794779834 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1338005070 con 0x7f134015dce0 2024-04-25T09:04:43.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.906+0000 7f1336ffd640 1 -- 172.21.15.43:0/2794779834 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1338005460 con 0x7f134015dce0 2024-04-25T09:04:43.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.906+0000 7f1348763640 1 -- 172.21.15.43:0/2794779834 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f134009e820 con 0x7f134015dce0 2024-04-25T09:04:43.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.906+0000 7f1348763640 1 -- 172.21.15.43:0/2794779834 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f134009ede0 con 0x7f134015dce0 2024-04-25T09:04:43.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.907+0000 7f1348763640 1 -- 172.21.15.43:0/2794779834 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13400680f0 con 0x7f134015dce0 2024-04-25T09:04:43.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.908+0000 7f1336ffd640 1 -- 172.21.15.43:0/2794779834 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1338005600 con 0x7f134015dce0 2024-04-25T09:04:43.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.908+0000 7f1336ffd640 1 --2- 172.21.15.43:0/2794779834 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f132406b500 0x7f132406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:43.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.908+0000 7f1336ffd640 1 -- 172.21.15.43:0/2794779834 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1338089f90 con 0x7f134015dce0 2024-04-25T09:04:43.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.908+0000 7f13464ff640 1 --2- 172.21.15.43:0/2794779834 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f132406b500 0x7f132406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:43.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.909+0000 7f13464ff640 1 --2- 172.21.15.43:0/2794779834 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f132406b500 0x7f132406d9c0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f133c034000 tx=0x7f133c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:43.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:43.912+0000 7f1336ffd640 1 -- 172.21.15.43:0/2794779834 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f13380587c0 con 0x7f134015dce0 2024-04-25T09:04:44.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.133+0000 7f1348763640 1 -- 172.21.15.43:0/2794779834 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1340074ef0 con 0x7f134015dce0 2024-04-25T09:04:44.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.133+0000 7f1336ffd640 1 -- 172.21.15.43:0/2794779834 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f133805c860 con 0x7f134015dce0 2024-04-25T09:04:44.122 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:04:44.122 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/2225978762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2225978762},{"type":"v1","addr":"172.21.15.193:6836","nonce":2225978762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9711,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/454458092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":454458092},{"type":"v1","addr":"172.21.15.43:6833","nonce":454458092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":90,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T09:04:13.410799+0000","modified":"2024-04-25T09:04:39.112330+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9732},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9720":{"gid":9720,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/2314901977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":2314901977},{"type":"v1","addr":"172.21.15.43:6835","nonce":2314901977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9732":{"gid":9732,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.15.193:6837/2585534546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2585534546},{"type":"v1","addr":"172.21.15.193:6837","nonce":2585534546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T09:04:44.122 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 90 2024-04-25T09:04:44.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.135+0000 7f1348763640 1 -- 172.21.15.43:0/2794779834 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f132406b500 msgr2=0x7f132406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:44.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.135+0000 7f1348763640 1 --2- 172.21.15.43:0/2794779834 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f132406b500 0x7f132406d9c0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f133c034000 tx=0x7f133c02e000 comp rx=0 tx=0).stop 2024-04-25T09:04:44.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.135+0000 7f1348763640 1 -- 172.21.15.43:0/2794779834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f134015dce0 msgr2=0x7f134015ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:44.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.135+0000 7f1348763640 1 --2- 172.21.15.43:0/2794779834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f134015dce0 0x7f134015ef40 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f133800c760 tx=0x7f133800cc20 comp rx=0 tx=0).stop 2024-04-25T09:04:44.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.135+0000 7f1348763640 1 -- 172.21.15.43:0/2794779834 shutdown_connections 2024-04-25T09:04:44.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.135+0000 7f1348763640 1 --2- 172.21.15.43:0/2794779834 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f134015dce0 0x7f134015ef40 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:44.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.135+0000 7f1348763640 1 --2- 172.21.15.43:0/2794779834 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13400ce990 0x7f134015ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:44.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.135+0000 7f1348763640 1 --2- 172.21.15.43:0/2794779834 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f132406b500 0x7f132406d9c0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:44.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.135+0000 7f1348763640 1 --2- 172.21.15.43:0/2794779834 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13400ce550 0x7f13400a01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:44.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.135+0000 7f1348763640 1 -- 172.21.15.43:0/2794779834 >> 172.21.15.43:0/2794779834 conn(0x7f134009cd00 msgr2=0x7f13400d1910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:44.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.135+0000 7f1348763640 1 -- 172.21.15.43:0/2794779834 shutdown_connections 2024-04-25T09:04:44.126 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.135+0000 7f1348763640 1 -- 172.21.15.43:0/2794779834 wait complete. 2024-04-25T09:04:44.135 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:04:44.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.247+0000 7f11432ac640 1 Processor -- start 2024-04-25T09:04:44.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.247+0000 7f11432ac640 1 -- start start 2024-04-25T09:04:44.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.248+0000 7f11432ac640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f113c15df50 0x7f113c1619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:44.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.248+0000 7f11432ac640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f113c161f30 0x7f113c1623a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:44.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.248+0000 7f11432ac640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f113c0d24a0 con 0x7f113c0c7cf0 2024-04-25T09:04:44.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.248+0000 7f11432ac640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f113c15e330 con 0x7f113c15df50 2024-04-25T09:04:44.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.248+0000 7f11432ac640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f113c0d2740 con 0x7f113c161f30 2024-04-25T09:04:44.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.248+0000 7f1141048640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f113c15df50 0x7f113c1619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:44.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.248+0000 7f1141048640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f113c15df50 0x7f113c1619f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60056/0 (socket says 172.21.15.43:60056) 2024-04-25T09:04:44.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.248+0000 7f1141048640 1 -- 172.21.15.43:0/2796492141 learned_addr learned my addr 172.21.15.43:0/2796492141 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:44.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.248+0000 7f1141849640 1 --1- 172.21.15.43:0/2796492141 >> v1:172.21.15.43:6789/0 conn(0x7f113c0c7cf0 0x7f113c0c8100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44624/0 (socket says 172.21.15.43:44624) 2024-04-25T09:04:44.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.249+0000 7f1140847640 1 --2- 172.21.15.43:0/2796492141 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f113c161f30 0x7f113c1623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:44.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.249+0000 7f1141048640 1 -- 172.21.15.43:0/2796492141 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f113c161f30 msgr2=0x7f113c1623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:44.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.249+0000 7f1141048640 1 --2- 172.21.15.43:0/2796492141 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f113c161f30 0x7f113c1623a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:44.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.249+0000 7f1141048640 1 -- 172.21.15.43:0/2796492141 >> v1:172.21.15.43:6789/0 conn(0x7f113c0c7cf0 legacy=0x7f113c0c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:44.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.249+0000 7f1141048640 1 -- 172.21.15.43:0/2796492141 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f113c0d2a20 con 0x7f113c15df50 2024-04-25T09:04:44.238 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.249+0000 7f1133fff640 1 -- 172.21.15.43:0/2796492141 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2584738366 0 0) 0x7f113c0d24a0 con 0x7f113c0c7cf0 2024-04-25T09:04:44.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.249+0000 7f1141048640 1 --2- 172.21.15.43:0/2796492141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f113c15df50 0x7f113c1619f0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f1134009870 tx=0x7f113402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9123445f0064cf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:44.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.250+0000 7f1133fff640 1 -- 172.21.15.43:0/2796492141 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1134037070 con 0x7f113c15df50 2024-04-25T09:04:44.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.250+0000 7f1133fff640 1 -- 172.21.15.43:0/2796492141 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1134032440 con 0x7f113c15df50 2024-04-25T09:04:44.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.250+0000 7f1133fff640 1 -- 172.21.15.43:0/2796492141 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1134032810 con 0x7f113c15df50 2024-04-25T09:04:44.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.250+0000 7f11432ac640 1 -- 172.21.15.43:0/2796492141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f113c15df50 msgr2=0x7f113c1619f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:44.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.250+0000 7f11432ac640 1 --2- 172.21.15.43:0/2796492141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f113c15df50 0x7f113c1619f0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f1134009870 tx=0x7f113402a270 comp rx=0 tx=0).stop 2024-04-25T09:04:44.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.250+0000 7f11432ac640 1 -- 172.21.15.43:0/2796492141 shutdown_connections 2024-04-25T09:04:44.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.250+0000 7f11432ac640 1 --2- 172.21.15.43:0/2796492141 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f113c161f30 0x7f113c1623a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:44.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.250+0000 7f11432ac640 1 --2- 172.21.15.43:0/2796492141 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f113c15df50 0x7f113c1619f0 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:44.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.250+0000 7f11432ac640 1 -- 172.21.15.43:0/2796492141 >> 172.21.15.43:0/2796492141 conn(0x7f113c0d3a30 msgr2=0x7f113c0d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:44.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.250+0000 7f11432ac640 1 -- 172.21.15.43:0/2796492141 shutdown_connections 2024-04-25T09:04:44.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.250+0000 7f11432ac640 1 -- 172.21.15.43:0/2796492141 wait complete. 2024-04-25T09:04:44.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.251+0000 7f11432ac640 1 Processor -- start 2024-04-25T09:04:44.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.252+0000 7f11432ac640 1 -- start start 2024-04-25T09:04:44.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.252+0000 7f11432ac640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f113c0c7cf0 0x7f113c152140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:44.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.252+0000 7f11432ac640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f113c15df50 0x7f113c06b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:44.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.252+0000 7f11432ac640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f113c161f30 0x7f113c06b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:44.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.252+0000 7f11432ac640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113c0d2190 con 0x7f113c15df50 2024-04-25T09:04:44.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.252+0000 7f11432ac640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113c0c77d0 con 0x7f113c0c7cf0 2024-04-25T09:04:44.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.252+0000 7f11432ac640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f113c0cf670 con 0x7f113c161f30 2024-04-25T09:04:44.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.252+0000 7f1140847640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f113c15df50 0x7f113c06b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:44.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.252+0000 7f1140847640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f113c15df50 0x7f113c06b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60070/0 (socket says 172.21.15.43:60070) 2024-04-25T09:04:44.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.252+0000 7f1140847640 1 -- 172.21.15.43:0/3516952835 learned_addr learned my addr 172.21.15.43:0/3516952835 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:44.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.253+0000 7f1141048640 1 --2- 172.21.15.43:0/3516952835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f113c0c7cf0 0x7f113c152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:44.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.253+0000 7f1141849640 1 --2- 172.21.15.43:0/3516952835 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f113c161f30 0x7f113c06b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:44.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.253+0000 7f1140847640 1 -- 172.21.15.43:0/3516952835 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f113c161f30 msgr2=0x7f113c06b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:44.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.253+0000 7f1140847640 1 --2- 172.21.15.43:0/3516952835 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f113c161f30 0x7f113c06b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:44.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.253+0000 7f1140847640 1 -- 172.21.15.43:0/3516952835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f113c0c7cf0 msgr2=0x7f113c152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:44.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.253+0000 7f1140847640 1 --2- 172.21.15.43:0/3516952835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f113c0c7cf0 0x7f113c152140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:44.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.253+0000 7f1140847640 1 -- 172.21.15.43:0/3516952835 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f113c1f7630 con 0x7f113c15df50 2024-04-25T09:04:44.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.253+0000 7f1140847640 1 --2- 172.21.15.43:0/3516952835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f113c15df50 0x7f113c06b040 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f112c00d600 tx=0x7f112c00dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:44.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.253+0000 7f1131ffb640 1 -- 172.21.15.43:0/3516952835 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c0125f0 con 0x7f113c15df50 2024-04-25T09:04:44.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.253+0000 7f1131ffb640 1 -- 172.21.15.43:0/3516952835 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112c019070 con 0x7f113c15df50 2024-04-25T09:04:44.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.253+0000 7f11432ac640 1 -- 172.21.15.43:0/3516952835 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f113c1f7910 con 0x7f113c15df50 2024-04-25T09:04:44.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.253+0000 7f1131ffb640 1 -- 172.21.15.43:0/3516952835 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c0129f0 con 0x7f113c15df50 2024-04-25T09:04:44.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.254+0000 7f11432ac640 1 -- 172.21.15.43:0/3516952835 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f113c09f3d0 con 0x7f113c15df50 2024-04-25T09:04:44.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.255+0000 7f11432ac640 1 -- 172.21.15.43:0/3516952835 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f113c0680f0 con 0x7f113c15df50 2024-04-25T09:04:44.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.255+0000 7f1131ffb640 1 -- 172.21.15.43:0/3516952835 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f112c012b90 con 0x7f113c15df50 2024-04-25T09:04:44.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.255+0000 7f1131ffb640 1 --2- 172.21.15.43:0/3516952835 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f112006b4d0 0x7f112006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:44.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.255+0000 7f1131ffb640 1 -- 172.21.15.43:0/3516952835 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f112c08a440 con 0x7f113c15df50 2024-04-25T09:04:44.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.256+0000 7f1141048640 1 --2- 172.21.15.43:0/3516952835 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f112006b4d0 0x7f112006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:44.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.256+0000 7f1141048640 1 --2- 172.21.15.43:0/3516952835 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f112006b4d0 0x7f112006d990 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f1134034000 tx=0x7f113402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:44.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.260+0000 7f1131ffb640 1 -- 172.21.15.43:0/3516952835 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f112c058d00 con 0x7f113c15df50 2024-04-25T09:04:44.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.439+0000 7f11432ac640 1 -- 172.21.15.43:0/3516952835 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f113c074ef0 con 0x7f113c15df50 2024-04-25T09:04:44.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.440+0000 7f1131ffb640 1 -- 172.21.15.43:0/3516952835 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+16643 (secure 0 0 0) 0x7f112c05cda0 con 0x7f113c15df50 2024-04-25T09:04:44.429 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:04:44.429 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":139,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:04:15.059128+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:04:11.735785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T09:04:12.383453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:04:44.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.442+0000 7f11432ac640 1 -- 172.21.15.43:0/3516952835 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f112006b4d0 msgr2=0x7f112006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:44.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.442+0000 7f11432ac640 1 --2- 172.21.15.43:0/3516952835 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f112006b4d0 0x7f112006d990 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f1134034000 tx=0x7f113402e000 comp rx=0 tx=0).stop 2024-04-25T09:04:44.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.442+0000 7f11432ac640 1 -- 172.21.15.43:0/3516952835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f113c15df50 msgr2=0x7f113c06b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:44.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.442+0000 7f11432ac640 1 --2- 172.21.15.43:0/3516952835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f113c15df50 0x7f113c06b040 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f112c00d600 tx=0x7f112c00dac0 comp rx=0 tx=0).stop 2024-04-25T09:04:44.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.442+0000 7f11432ac640 1 -- 172.21.15.43:0/3516952835 shutdown_connections 2024-04-25T09:04:44.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.442+0000 7f11432ac640 1 --2- 172.21.15.43:0/3516952835 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f113c161f30 0x7f113c06b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:44.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.442+0000 7f11432ac640 1 --2- 172.21.15.43:0/3516952835 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f113c15df50 0x7f113c06b040 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:44.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.442+0000 7f11432ac640 1 --2- 172.21.15.43:0/3516952835 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f112006b4d0 0x7f112006d990 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:44.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.442+0000 7f11432ac640 1 --2- 172.21.15.43:0/3516952835 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f113c0c7cf0 0x7f113c152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:44.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.442+0000 7f11432ac640 1 -- 172.21.15.43:0/3516952835 >> 172.21.15.43:0/3516952835 conn(0x7f113c0d3a30 msgr2=0x7f113c15ccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:44.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.442+0000 7f11432ac640 1 -- 172.21.15.43:0/3516952835 shutdown_connections 2024-04-25T09:04:44.432 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.442+0000 7f11432ac640 1 -- 172.21.15.43:0/3516952835 wait complete. 2024-04-25T09:04:44.443 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:04:44.443 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:04:44.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.555+0000 7fed72891640 1 Processor -- start 2024-04-25T09:04:44.544 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.555+0000 7fed72891640 1 -- start start 2024-04-25T09:04:44.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.556+0000 7fed72891640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fed6c0cbfc0 0x7fed6c06dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:44.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.556+0000 7fed72891640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fed6c0ccbb0 0x7fed6c0cd010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:44.545 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.556+0000 7fed72891640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fed6c15f5e0 0x7fed6c15fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.556+0000 7fed72891640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed6c0cd550 con 0x7fed6c0cbfc0 2024-04-25T09:04:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.556+0000 7fed72891640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed6c0b2780 con 0x7fed6c0ccbb0 2024-04-25T09:04:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.556+0000 7fed72891640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c0b2a60 con 0x7fed6c15f5e0 2024-04-25T09:04:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.556+0000 7fed6bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fed6c0cbfc0 0x7fed6c06dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.556+0000 7fed6bfff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fed6c0cbfc0 0x7fed6c06dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60076/0 (socket says 172.21.15.43:60076) 2024-04-25T09:04:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.556+0000 7fed6bfff640 1 -- 172.21.15.43:0/2025640744 learned_addr learned my addr 172.21.15.43:0/2025640744 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.557+0000 7fed70e2e640 1 --2- 172.21.15.43:0/2025640744 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fed6c15f5e0 0x7fed6c15fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.557+0000 7fed6b7fe640 1 --2- 172.21.15.43:0/2025640744 >> v2:172.21.15.193:3300/0 conn(0x7fed6c0ccbb0 0x7fed6c0cd010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.557+0000 7fed6bfff640 1 -- 172.21.15.43:0/2025640744 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fed6c15f5e0 msgr2=0x7fed6c15fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.557+0000 7fed6bfff640 1 --2- 172.21.15.43:0/2025640744 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fed6c15f5e0 0x7fed6c15fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.557+0000 7fed6bfff640 1 -- 172.21.15.43:0/2025640744 >> v2:172.21.15.193:3300/0 conn(0x7fed6c0ccbb0 msgr2=0x7fed6c0cd010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.557+0000 7fed6bfff640 1 --2- 172.21.15.43:0/2025640744 >> v2:172.21.15.193:3300/0 conn(0x7fed6c0ccbb0 0x7fed6c0cd010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:44.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.557+0000 7fed6bfff640 1 -- 172.21.15.43:0/2025640744 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed6c0b2d40 con 0x7fed6c0cbfc0 2024-04-25T09:04:44.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.557+0000 7fed6bfff640 1 --2- 172.21.15.43:0/2025640744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed6c0cbfc0 0x7fed6c06dc60 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7fed6c0747f0 tx=0x7fed6002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea07bebb1cda953f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:44.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.558+0000 7fed6affd640 1 -- 172.21.15.43:0/2025640744 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed60037070 con 0x7fed6c0cbfc0 2024-04-25T09:04:44.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.558+0000 7fed6affd640 1 -- 172.21.15.43:0/2025640744 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed60032440 con 0x7fed6c0cbfc0 2024-04-25T09:04:44.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.558+0000 7fed6affd640 1 -- 172.21.15.43:0/2025640744 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed60032810 con 0x7fed6c0cbfc0 2024-04-25T09:04:44.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.558+0000 7fed72891640 1 -- 172.21.15.43:0/2025640744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed6c0cbfc0 msgr2=0x7fed6c06dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:44.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.558+0000 7fed72891640 1 --2- 172.21.15.43:0/2025640744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed6c0cbfc0 0x7fed6c06dc60 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7fed6c0747f0 tx=0x7fed6002a1e0 comp rx=0 tx=0).stop 2024-04-25T09:04:44.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.558+0000 7fed72891640 1 -- 172.21.15.43:0/2025640744 shutdown_connections 2024-04-25T09:04:44.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.558+0000 7fed72891640 1 --2- 172.21.15.43:0/2025640744 >> v2:172.21.15.193:3300/0 conn(0x7fed6c0ccbb0 0x7fed6c0cd010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:44.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.558+0000 7fed72891640 1 --2- 172.21.15.43:0/2025640744 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fed6c15f5e0 0x7fed6c15fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:44.550 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.558+0000 7fed72891640 1 --2- 172.21.15.43:0/2025640744 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed6c0cbfc0 0x7fed6c06dc60 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:44.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.558+0000 7fed72891640 1 -- 172.21.15.43:0/2025640744 >> 172.21.15.43:0/2025640744 conn(0x7fed6c0b1480 msgr2=0x7fed6c0b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:44.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.558+0000 7fed72891640 1 -- 172.21.15.43:0/2025640744 shutdown_connections 2024-04-25T09:04:44.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.558+0000 7fed72891640 1 -- 172.21.15.43:0/2025640744 wait complete. 2024-04-25T09:04:44.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.559+0000 7fed72891640 1 Processor -- start 2024-04-25T09:04:44.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.560+0000 7fed72891640 1 -- start start 2024-04-25T09:04:44.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.560+0000 7fed72891640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed6c0cbfc0 0x7fed6c1410b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:44.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.560+0000 7fed72891640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fed6c0ccbb0 0x7fed6c0c46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:44.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.560+0000 7fed72891640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed6c15f5e0 0x7fed6c1f17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:44.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.560+0000 7fed72891640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c0b35e0 con 0x7fed6c15f5e0 2024-04-25T09:04:44.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.560+0000 7fed72891640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c0b3300 con 0x7fed6c0cbfc0 2024-04-25T09:04:44.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.560+0000 7fed72891640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed6c06e1a0 con 0x7fed6c0ccbb0 2024-04-25T09:04:44.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.560+0000 7fed70e2e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed6c15f5e0 0x7fed6c1f17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:44.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.560+0000 7fed70e2e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed6c15f5e0 0x7fed6c1f17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60080/0 (socket says 172.21.15.43:60080) 2024-04-25T09:04:44.551 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.560+0000 7fed70e2e640 1 -- 172.21.15.43:0/3098888295 learned_addr learned my addr 172.21.15.43:0/3098888295 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:44.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.561+0000 7fed6bfff640 1 --2- 172.21.15.43:0/3098888295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed6c0cbfc0 0x7fed6c1410b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:44.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.561+0000 7fed6b7fe640 1 --2- 172.21.15.43:0/3098888295 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fed6c0ccbb0 0x7fed6c0c46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:44.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.561+0000 7fed70e2e640 1 -- 172.21.15.43:0/3098888295 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fed6c0ccbb0 msgr2=0x7fed6c0c46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:44.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.561+0000 7fed70e2e640 1 --2- 172.21.15.43:0/3098888295 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fed6c0ccbb0 0x7fed6c0c46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:44.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.561+0000 7fed70e2e640 1 -- 172.21.15.43:0/3098888295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed6c0cbfc0 msgr2=0x7fed6c1410b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:44.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.561+0000 7fed70e2e640 1 --2- 172.21.15.43:0/3098888295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed6c0cbfc0 0x7fed6c1410b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:44.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.561+0000 7fed70e2e640 1 -- 172.21.15.43:0/3098888295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed6c1415f0 con 0x7fed6c15f5e0 2024-04-25T09:04:44.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.561+0000 7fed6b7fe640 1 --2- 172.21.15.43:0/3098888295 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fed6c0ccbb0 0x7fed6c0c46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:44.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.561+0000 7fed6bfff640 1 --2- 172.21.15.43:0/3098888295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed6c0cbfc0 0x7fed6c1410b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:44.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.561+0000 7fed70e2e640 1 --2- 172.21.15.43:0/3098888295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed6c15f5e0 0x7fed6c1f17b0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7fed64007dc0 tx=0x7fed6400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:44.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.561+0000 7fed68ff9640 1 -- 172.21.15.43:0/3098888295 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed64014420 con 0x7fed6c15f5e0 2024-04-25T09:04:44.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.561+0000 7fed68ff9640 1 -- 172.21.15.43:0/3098888295 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed64019070 con 0x7fed6c15f5e0 2024-04-25T09:04:44.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.561+0000 7fed72891640 1 -- 172.21.15.43:0/3098888295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed6c0a9f80 con 0x7fed6c15f5e0 2024-04-25T09:04:44.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.561+0000 7fed68ff9640 1 -- 172.21.15.43:0/3098888295 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed64014820 con 0x7fed6c15f5e0 2024-04-25T09:04:44.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.561+0000 7fed72891640 1 -- 172.21.15.43:0/3098888295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed6c0aa540 con 0x7fed6c15f5e0 2024-04-25T09:04:44.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.562+0000 7fed72891640 1 -- 172.21.15.43:0/3098888295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed6c074ef0 con 0x7fed6c15f5e0 2024-04-25T09:04:44.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.563+0000 7fed68ff9640 1 -- 172.21.15.43:0/3098888295 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fed640149c0 con 0x7fed6c15f5e0 2024-04-25T09:04:44.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.563+0000 7fed68ff9640 1 --2- 172.21.15.43:0/3098888295 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fed5806b4d0 0x7fed5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:44.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.563+0000 7fed68ff9640 1 -- 172.21.15.43:0/3098888295 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fed6408aa60 con 0x7fed6c15f5e0 2024-04-25T09:04:44.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.563+0000 7fed6bfff640 1 --2- 172.21.15.43:0/3098888295 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fed5806b4d0 0x7fed5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:44.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.564+0000 7fed6bfff640 1 --2- 172.21.15.43:0/3098888295 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fed5806b4d0 0x7fed5806d990 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fed60034000 tx=0x7fed6002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:44.555 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.567+0000 7fed68ff9640 1 -- 172.21.15.43:0/3098888295 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fed640598e0 con 0x7fed6c15f5e0 2024-04-25T09:04:44.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.783+0000 7fed72891640 1 -- 172.21.15.43:0/3098888295 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed6c144750 con 0x7fed6c15f5e0 2024-04-25T09:04:44.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.784+0000 7fed68ff9640 1 -- 172.21.15.43:0/3098888295 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4952 (secure 0 0 0) 0x7fed640591e0 con 0x7fed6c15f5e0 2024-04-25T09:04:44.773 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:04:44.773 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/2225978762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2225978762},{"type":"v1","addr":"172.21.15.193:6836","nonce":2225978762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9711,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/454458092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":454458092},{"type":"v1","addr":"172.21.15.43:6833","nonce":454458092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":90,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T09:04:13.410799+0000","modified":"2024-04-25T09:04:39.112330+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9732},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9720":{"gid":9720,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/2314901977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":2314901977},{"type":"v1","addr":"172.21.15.43:6835","nonce":2314901977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9732":{"gid":9732,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.15.193:6837/2585534546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2585534546},{"type":"v1","addr":"172.21.15.193:6837","nonce":2585534546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T09:04:44.773 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 90 2024-04-25T09:04:44.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.786+0000 7fed72891640 1 -- 172.21.15.43:0/3098888295 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fed5806b4d0 msgr2=0x7fed5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:44.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.786+0000 7fed72891640 1 --2- 172.21.15.43:0/3098888295 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fed5806b4d0 0x7fed5806d990 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fed60034000 tx=0x7fed6002e000 comp rx=0 tx=0).stop 2024-04-25T09:04:44.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.786+0000 7fed72891640 1 -- 172.21.15.43:0/3098888295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed6c15f5e0 msgr2=0x7fed6c1f17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:44.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.786+0000 7fed72891640 1 --2- 172.21.15.43:0/3098888295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed6c15f5e0 0x7fed6c1f17b0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7fed64007dc0 tx=0x7fed6400e4b0 comp rx=0 tx=0).stop 2024-04-25T09:04:44.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.786+0000 7fed72891640 1 -- 172.21.15.43:0/3098888295 shutdown_connections 2024-04-25T09:04:44.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.786+0000 7fed72891640 1 --2- 172.21.15.43:0/3098888295 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fed6c15f5e0 0x7fed6c1f17b0 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:44.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.786+0000 7fed72891640 1 --2- 172.21.15.43:0/3098888295 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fed6c0ccbb0 0x7fed6c0c46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:44.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.786+0000 7fed72891640 1 --2- 172.21.15.43:0/3098888295 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fed5806b4d0 0x7fed5806d990 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:44.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.786+0000 7fed72891640 1 --2- 172.21.15.43:0/3098888295 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fed6c0cbfc0 0x7fed6c1410b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:44.775 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.786+0000 7fed72891640 1 -- 172.21.15.43:0/3098888295 >> 172.21.15.43:0/3098888295 conn(0x7fed6c0b1480 msgr2=0x7fed6c0b0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:44.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.786+0000 7fed72891640 1 -- 172.21.15.43:0/3098888295 shutdown_connections 2024-04-25T09:04:44.776 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.786+0000 7fed72891640 1 -- 172.21.15.43:0/3098888295 wait complete. 2024-04-25T09:04:44.786 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:04:44.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.898+0000 7ff82881f640 1 Processor -- start 2024-04-25T09:04:44.887 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.898+0000 7ff82881f640 1 -- start start 2024-04-25T09:04:44.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.899+0000 7ff82881f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff82415dcf0 0x7ff82406e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:44.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.899+0000 7ff82881f640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff82406ebf0 0x7ff82406f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:44.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.899+0000 7ff82881f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff8241616e0 0x7ff824161b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:44.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.899+0000 7ff82881f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff82406f590 con 0x7ff82415dcf0 2024-04-25T09:04:44.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.899+0000 7ff82881f640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff82406c050 con 0x7ff82406ebf0 2024-04-25T09:04:44.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.899+0000 7ff82881f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8240d0d80 con 0x7ff8241616e0 2024-04-25T09:04:44.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.899+0000 7ff82259c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff82415dcf0 0x7ff82406e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:44.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.899+0000 7ff82259c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff82415dcf0 0x7ff82406e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60084/0 (socket says 172.21.15.43:60084) 2024-04-25T09:04:44.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.899+0000 7ff82259c640 1 -- 172.21.15.43:0/3947005677 learned_addr learned my addr 172.21.15.43:0/3947005677 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:44.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.899+0000 7ff822d9d640 1 --2- 172.21.15.43:0/3947005677 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff8241616e0 0x7ff824161b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:44.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.899+0000 7ff82259c640 1 -- 172.21.15.43:0/3947005677 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff8241616e0 msgr2=0x7ff824161b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:44.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.899+0000 7ff82259c640 1 --2- 172.21.15.43:0/3947005677 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff8241616e0 0x7ff824161b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:44.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.899+0000 7ff821d9b640 1 --2- 172.21.15.43:0/3947005677 >> v2:172.21.15.193:3300/0 conn(0x7ff82406ebf0 0x7ff82406f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:44.889 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.899+0000 7ff82259c640 1 -- 172.21.15.43:0/3947005677 >> v2:172.21.15.193:3300/0 conn(0x7ff82406ebf0 msgr2=0x7ff82406f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:44.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.899+0000 7ff82259c640 1 --2- 172.21.15.43:0/3947005677 >> v2:172.21.15.193:3300/0 conn(0x7ff82406ebf0 0x7ff82406f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:44.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.899+0000 7ff82259c640 1 -- 172.21.15.43:0/3947005677 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8240d04e0 con 0x7ff82415dcf0 2024-04-25T09:04:44.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.899+0000 7ff822d9d640 1 --2- 172.21.15.43:0/3947005677 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff8241616e0 0x7ff824161b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:44.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.899+0000 7ff821d9b640 1 --2- 172.21.15.43:0/3947005677 >> v2:172.21.15.193:3300/0 conn(0x7ff82406ebf0 0x7ff82406f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:44.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.900+0000 7ff82259c640 1 --2- 172.21.15.43:0/3947005677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff82415dcf0 0x7ff82406e530 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7ff824072960 tx=0x7ff81c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1275498f5a8c06a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:44.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.900+0000 7ff82159a640 1 -- 172.21.15.43:0/3947005677 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c037070 con 0x7ff82415dcf0 2024-04-25T09:04:44.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.900+0000 7ff82159a640 1 -- 172.21.15.43:0/3947005677 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81c032440 con 0x7ff82415dcf0 2024-04-25T09:04:44.890 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.900+0000 7ff82159a640 1 -- 172.21.15.43:0/3947005677 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c032810 con 0x7ff82415dcf0 2024-04-25T09:04:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.901+0000 7ff82881f640 1 -- 172.21.15.43:0/3947005677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff82415dcf0 msgr2=0x7ff82406e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.901+0000 7ff82881f640 1 --2- 172.21.15.43:0/3947005677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff82415dcf0 0x7ff82406e530 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7ff824072960 tx=0x7ff81c02a140 comp rx=0 tx=0).stop 2024-04-25T09:04:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.901+0000 7ff82881f640 1 -- 172.21.15.43:0/3947005677 shutdown_connections 2024-04-25T09:04:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.901+0000 7ff82881f640 1 --2- 172.21.15.43:0/3947005677 >> v2:172.21.15.193:3300/0 conn(0x7ff82406ebf0 0x7ff82406f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.901+0000 7ff82881f640 1 --2- 172.21.15.43:0/3947005677 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff8241616e0 0x7ff824161b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.901+0000 7ff82881f640 1 --2- 172.21.15.43:0/3947005677 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff82415dcf0 0x7ff82406e530 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.901+0000 7ff82881f640 1 -- 172.21.15.43:0/3947005677 >> 172.21.15.43:0/3947005677 conn(0x7ff8240d1700 msgr2=0x7ff8240d1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.901+0000 7ff82881f640 1 -- 172.21.15.43:0/3947005677 shutdown_connections 2024-04-25T09:04:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.901+0000 7ff82881f640 1 -- 172.21.15.43:0/3947005677 wait complete. 2024-04-25T09:04:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.902+0000 7ff82881f640 1 Processor -- start 2024-04-25T09:04:44.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.902+0000 7ff82881f640 1 -- start start 2024-04-25T09:04:44.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.903+0000 7ff82881f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff82406ebf0 0x7ff8240b42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:44.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.903+0000 7ff82881f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff82415dcf0 0x7ff82414cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:44.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.903+0000 7ff82881f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8241616e0 0x7ff8241f3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:44.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.903+0000 7ff82881f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82406bce0 con 0x7ff8241616e0 2024-04-25T09:04:44.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.903+0000 7ff82881f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8240d0aa0 con 0x7ff82415dcf0 2024-04-25T09:04:44.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.903+0000 7ff82881f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8240b0d50 con 0x7ff82406ebf0 2024-04-25T09:04:44.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.903+0000 7ff822d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8241616e0 0x7ff8241f3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:44.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.903+0000 7ff822d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8241616e0 0x7ff8241f3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60096/0 (socket says 172.21.15.43:60096) 2024-04-25T09:04:44.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.903+0000 7ff822d9d640 1 -- 172.21.15.43:0/3643712263 learned_addr learned my addr 172.21.15.43:0/3643712263 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:44.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.903+0000 7ff821d9b640 1 --2- 172.21.15.43:0/3643712263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff82415dcf0 0x7ff82414cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:44.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.903+0000 7ff82259c640 1 --2- 172.21.15.43:0/3643712263 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff82406ebf0 0x7ff8240b42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:44.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.903+0000 7ff822d9d640 1 -- 172.21.15.43:0/3643712263 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff82406ebf0 msgr2=0x7ff8240b42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:44.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.903+0000 7ff822d9d640 1 --2- 172.21.15.43:0/3643712263 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff82406ebf0 0x7ff8240b42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:44.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.903+0000 7ff822d9d640 1 -- 172.21.15.43:0/3643712263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff82415dcf0 msgr2=0x7ff82414cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:44.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.903+0000 7ff822d9d640 1 --2- 172.21.15.43:0/3643712263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff82415dcf0 0x7ff82414cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:44.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.903+0000 7ff822d9d640 1 -- 172.21.15.43:0/3643712263 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8240b6c20 con 0x7ff8241616e0 2024-04-25T09:04:44.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.903+0000 7ff82259c640 1 --2- 172.21.15.43:0/3643712263 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff82406ebf0 0x7ff8240b42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:44.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.903+0000 7ff821d9b640 1 --2- 172.21.15.43:0/3643712263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff82415dcf0 0x7ff82414cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:44.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.904+0000 7ff822d9d640 1 --2- 172.21.15.43:0/3643712263 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8241616e0 0x7ff8241f3950 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7ff81400cd50 tx=0x7ff81400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:44.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.904+0000 7ff81affd640 1 -- 172.21.15.43:0/3643712263 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81400f680 con 0x7ff8241616e0 2024-04-25T09:04:44.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.904+0000 7ff81affd640 1 -- 172.21.15.43:0/3643712263 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81400f840 con 0x7ff8241616e0 2024-04-25T09:04:44.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.904+0000 7ff82881f640 1 -- 172.21.15.43:0/3643712263 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8240b6f00 con 0x7ff8241616e0 2024-04-25T09:04:44.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.904+0000 7ff81affd640 1 -- 172.21.15.43:0/3643712263 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81400fc30 con 0x7ff8241616e0 2024-04-25T09:04:44.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.904+0000 7ff82881f640 1 -- 172.21.15.43:0/3643712263 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8240b4820 con 0x7ff8241616e0 2024-04-25T09:04:44.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.905+0000 7ff82881f640 1 -- 172.21.15.43:0/3643712263 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff824074ef0 con 0x7ff8241616e0 2024-04-25T09:04:44.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.905+0000 7ff81affd640 1 -- 172.21.15.43:0/3643712263 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff8140213f0 con 0x7ff8241616e0 2024-04-25T09:04:44.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.906+0000 7ff81affd640 1 --2- 172.21.15.43:0/3643712263 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff80406b4d0 0x7ff80406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:44.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.906+0000 7ff81affd640 1 -- 172.21.15.43:0/3643712263 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff81408aea0 con 0x7ff8241616e0 2024-04-25T09:04:44.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.906+0000 7ff82259c640 1 --2- 172.21.15.43:0/3643712263 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff80406b4d0 0x7ff80406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:44.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.906+0000 7ff82259c640 1 --2- 172.21.15.43:0/3643712263 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff80406b4d0 0x7ff80406d990 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7ff81c034000 tx=0x7ff81c02e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:44.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:44.910+0000 7ff81affd640 1 -- 172.21.15.43:0/3643712263 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff814014030 con 0x7ff8241616e0 2024-04-25T09:04:45.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.127+0000 7ff82881f640 1 -- 172.21.15.43:0/3643712263 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8240cfca0 con 0x7ff8241616e0 2024-04-25T09:04:45.116 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.128+0000 7ff81affd640 1 -- 172.21.15.43:0/3643712263 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4952 (secure 0 0 0) 0x7ff814059db0 con 0x7ff8241616e0 2024-04-25T09:04:45.116 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:04:45.116 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":90,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/2225978762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2225978762},{"type":"v1","addr":"172.21.15.193:6836","nonce":2225978762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":9711,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/454458092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":454458092},{"type":"v1","addr":"172.21.15.43:6833","nonce":454458092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83}],"filesystems":[{"mdsmap":{"epoch":90,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T09:04:13.410799+0000","modified":"2024-04-25T09:04:39.112330+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9732},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9720":{"gid":9720,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.43:6835/2314901977","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":2314901977},{"type":"v1","addr":"172.21.15.43:6835","nonce":2314901977}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9732":{"gid":9732,"name":"b","rank":0,"incarnation":85,"state":"up:active","state_seq":3,"addr":"172.21.15.193:6837/2585534546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2585534546},{"type":"v1","addr":"172.21.15.193:6837","nonce":2585534546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T09:04:45.117 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 90 2024-04-25T09:04:45.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.130+0000 7ff82881f640 1 -- 172.21.15.43:0/3643712263 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff80406b4d0 msgr2=0x7ff80406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:45.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.130+0000 7ff82881f640 1 --2- 172.21.15.43:0/3643712263 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff80406b4d0 0x7ff80406d990 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7ff81c034000 tx=0x7ff81c02e000 comp rx=0 tx=0).stop 2024-04-25T09:04:45.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.130+0000 7ff82881f640 1 -- 172.21.15.43:0/3643712263 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8241616e0 msgr2=0x7ff8241f3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:45.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.130+0000 7ff82881f640 1 --2- 172.21.15.43:0/3643712263 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8241616e0 0x7ff8241f3950 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7ff81400cd50 tx=0x7ff81400a650 comp rx=0 tx=0).stop 2024-04-25T09:04:45.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.130+0000 7ff82881f640 1 -- 172.21.15.43:0/3643712263 shutdown_connections 2024-04-25T09:04:45.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.130+0000 7ff82881f640 1 --2- 172.21.15.43:0/3643712263 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff8241616e0 0x7ff8241f3950 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:45.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.130+0000 7ff82881f640 1 --2- 172.21.15.43:0/3643712263 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff80406b4d0 0x7ff80406d990 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:45.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.130+0000 7ff82881f640 1 --2- 172.21.15.43:0/3643712263 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff82415dcf0 0x7ff82414cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:45.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.130+0000 7ff82881f640 1 --2- 172.21.15.43:0/3643712263 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff82406ebf0 0x7ff8240b42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:45.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.130+0000 7ff82881f640 1 -- 172.21.15.43:0/3643712263 >> 172.21.15.43:0/3643712263 conn(0x7ff8240d1700 msgr2=0x7ff82406f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:45.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.130+0000 7ff82881f640 1 -- 172.21.15.43:0/3643712263 shutdown_connections 2024-04-25T09:04:45.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.130+0000 7ff82881f640 1 -- 172.21.15.43:0/3643712263 wait complete. 2024-04-25T09:04:45.130 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:04:45.231 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.243+0000 7f6230729640 1 Processor -- start 2024-04-25T09:04:45.232 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.243+0000 7f6230729640 1 -- start start 2024-04-25T09:04:45.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.244+0000 7f6230729640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f622815ef50 0x7f62280d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:45.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.244+0000 7f6230729640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62280d0920 0x7f6228160980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:45.234 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.244+0000 7f6230729640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62280cee80 con 0x7f622815dce0 2024-04-25T09:04:45.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.244+0000 7f6230729640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62280cec60 con 0x7f622815ef50 2024-04-25T09:04:45.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.244+0000 7f6230729640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62280d0d70 con 0x7f62280d0920 2024-04-25T09:04:45.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.244+0000 7f622e4c5640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f622815dce0 0x7f62280ce550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44634/0 (socket says 172.21.15.43:44634) 2024-04-25T09:04:45.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.244+0000 7f622e4c5640 1 -- 172.21.15.43:0/3127850471 learned_addr learned my addr 172.21.15.43:0/3127850471 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:45.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.244+0000 7f622d4c3640 1 -- 172.21.15.43:0/3127850471 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4081578189 0 0) 0x7f62280cee80 con 0x7f622815dce0 2024-04-25T09:04:45.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.244+0000 7f622d4c3640 1 -- 172.21.15.43:0/3127850471 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6210003520 con 0x7f622815dce0 2024-04-25T09:04:45.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.244+0000 7f622ecc6640 1 --2- 172.21.15.43:0/3127850471 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62280d0920 0x7f6228160980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:45.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.244+0000 7f622dcc4640 1 --2- 172.21.15.43:0/3127850471 >> v2:172.21.15.193:3300/0 conn(0x7f622815ef50 0x7f62280d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:45.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.245+0000 7f622d4c3640 1 -- 172.21.15.43:0/3127850471 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2814830704 0 0) 0x7f6210003520 con 0x7f622815dce0 2024-04-25T09:04:45.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.245+0000 7f622d4c3640 1 -- 172.21.15.43:0/3127850471 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62280d0920 msgr2=0x7f6228160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:45.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.245+0000 7f622d4c3640 1 --2- 172.21.15.43:0/3127850471 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62280d0920 0x7f6228160980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:45.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.245+0000 7f622d4c3640 1 -- 172.21.15.43:0/3127850471 >> v2:172.21.15.193:3300/0 conn(0x7f622815ef50 msgr2=0x7f62280d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:45.235 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.245+0000 7f622d4c3640 1 --2- 172.21.15.43:0/3127850471 >> v2:172.21.15.193:3300/0 conn(0x7f622815ef50 0x7f62280d03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:45.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.245+0000 7f622d4c3640 1 -- 172.21.15.43:0/3127850471 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62280d2920 con 0x7f622815dce0 2024-04-25T09:04:45.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.245+0000 7f622d4c3640 1 -- 172.21.15.43:0/3127850471 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f6220001670 con 0x7f622815dce0 2024-04-25T09:04:45.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.245+0000 7f622d4c3640 1 -- 172.21.15.43:0/3127850471 >> v1:172.21.15.43:6789/0 conn(0x7f622815dce0 legacy=0x7f62280ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:45.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.245+0000 7f622d4c3640 1 --2- 172.21.15.43:0/3127850471 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62280d0920 0x7f62100039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:45.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.245+0000 7f622d4c3640 1 --2- 172.21.15.43:0/3127850471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6210003fb0 0x7f6210004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:45.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.245+0000 7f622d4c3640 1 --2- 172.21.15.43:0/3127850471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6210004d40 0x7f62100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:45.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.245+0000 7f622d4c3640 1 -- 172.21.15.43:0/3127850471 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62280cec60 con 0x7f6210003fb0 2024-04-25T09:04:45.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.245+0000 7f622d4c3640 1 -- 172.21.15.43:0/3127850471 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62280d0d70 con 0x7f6210004d40 2024-04-25T09:04:45.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.245+0000 7f622d4c3640 1 -- 172.21.15.43:0/3127850471 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f621000baa0 con 0x7f62280d0920 2024-04-25T09:04:45.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.245+0000 7f622dcc4640 1 --2- 172.21.15.43:0/3127850471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6210003fb0 0x7f6210004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:45.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.245+0000 7f622ecc6640 1 --2- 172.21.15.43:0/3127850471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6210004d40 0x7f62100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:45.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.245+0000 7f622e4c5640 1 --2- 172.21.15.43:0/3127850471 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62280d0920 0x7f62100039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:45.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.245+0000 7f622dcc4640 1 -- 172.21.15.43:0/3127850471 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62280d0920 msgr2=0x7f62100039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:45.236 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.245+0000 7f622dcc4640 1 --2- 172.21.15.43:0/3127850471 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f62280d0920 0x7f62100039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:45.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.245+0000 7f622dcc4640 1 -- 172.21.15.43:0/3127850471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6210004d40 msgr2=0x7f62100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:45.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.245+0000 7f622dcc4640 1 --2- 172.21.15.43:0/3127850471 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6210004d40 0x7f62100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:45.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.245+0000 7f622dcc4640 1 -- 172.21.15.43:0/3127850471 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6210003520 con 0x7f6210003fb0 2024-04-25T09:04:45.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.246+0000 7f622dcc4640 1 --2- 172.21.15.43:0/3127850471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6210003fb0 0x7f6210004800 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f621c009ac0 tx=0x7f6210005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1adcb2a6333734d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:45.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.246+0000 7f622d4c3640 1 -- 172.21.15.43:0/3127850471 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c036070 con 0x7f6210003fb0 2024-04-25T09:04:45.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.246+0000 7f622d4c3640 1 -- 172.21.15.43:0/3127850471 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f621c031400 con 0x7f6210003fb0 2024-04-25T09:04:45.237 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.246+0000 7f622d4c3640 1 -- 172.21.15.43:0/3127850471 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621c009690 con 0x7f6210003fb0 2024-04-25T09:04:45.240 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.247+0000 7f6230729640 1 -- 172.21.15.43:0/3127850471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6210003fb0 msgr2=0x7f6210004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:45.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.247+0000 7f6230729640 1 --2- 172.21.15.43:0/3127850471 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6210003fb0 0x7f6210004800 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f621c009ac0 tx=0x7f6210005830 comp rx=0 tx=0).stop 2024-04-25T09:04:45.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.247+0000 7f622ecc6640 1 -- 172.21.15.43:0/3127850471 reap_dead start 2024-04-25T09:04:45.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.247+0000 7f6230729640 1 -- 172.21.15.43:0/3127850471 shutdown_connections 2024-04-25T09:04:45.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.247+0000 7f6230729640 1 -- 172.21.15.43:0/3127850471 >> 172.21.15.43:0/3127850471 conn(0x7f622809cd00 msgr2=0x7f62280d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:45.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.247+0000 7f6230729640 1 -- 172.21.15.43:0/3127850471 shutdown_connections 2024-04-25T09:04:45.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.247+0000 7f6230729640 1 -- 172.21.15.43:0/3127850471 wait complete. 2024-04-25T09:04:45.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.248+0000 7f6230729640 1 Processor -- start 2024-04-25T09:04:45.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.248+0000 7f6230729640 1 -- start start 2024-04-25T09:04:45.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.249+0000 7f6230729640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6210003fb0 0x7f62280a1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:45.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.249+0000 7f6230729640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f622815c110 0x7f622809e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:45.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.249+0000 7f6230729640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62280b4e80 0x7f62280b5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:45.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.249+0000 7f6230729640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62280cf720 con 0x7f62280b4e80 2024-04-25T09:04:45.241 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.249+0000 7f6230729640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62280cf440 con 0x7f6210003fb0 2024-04-25T09:04:45.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.249+0000 7f6230729640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f622815f6b0 con 0x7f622815c110 2024-04-25T09:04:45.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.249+0000 7f622ecc6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62280b4e80 0x7f62280b5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:45.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.249+0000 7f622ecc6640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62280b4e80 0x7f62280b5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60120/0 (socket says 172.21.15.43:60120) 2024-04-25T09:04:45.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.249+0000 7f622ecc6640 1 -- 172.21.15.43:0/3449251499 learned_addr learned my addr 172.21.15.43:0/3449251499 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:45.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.249+0000 7f622e4c5640 1 --2- 172.21.15.43:0/3449251499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6210003fb0 0x7f62280a1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:45.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.249+0000 7f622dcc4640 1 --2- 172.21.15.43:0/3449251499 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f622815c110 0x7f622809e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:45.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.249+0000 7f622ecc6640 1 -- 172.21.15.43:0/3449251499 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f622815c110 msgr2=0x7f622809e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:45.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.249+0000 7f622ecc6640 1 --2- 172.21.15.43:0/3449251499 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f622815c110 0x7f622809e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:45.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.249+0000 7f622ecc6640 1 -- 172.21.15.43:0/3449251499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6210003fb0 msgr2=0x7f62280a1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:45.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.249+0000 7f622ecc6640 1 --2- 172.21.15.43:0/3449251499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6210003fb0 0x7f62280a1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:45.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.249+0000 7f622ecc6640 1 -- 172.21.15.43:0/3449251499 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f622809eea0 con 0x7f62280b4e80 2024-04-25T09:04:45.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.249+0000 7f622e4c5640 1 --2- 172.21.15.43:0/3449251499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6210003fb0 0x7f62280a1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:45.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.249+0000 7f622dcc4640 1 --2- 172.21.15.43:0/3449251499 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f622815c110 0x7f622809e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:45.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.249+0000 7f622ecc6640 1 --2- 172.21.15.43:0/3449251499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62280b4e80 0x7f62280b5300 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f62280d1480 tx=0x7f621800f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:45.242 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.250+0000 7f6226ffd640 1 -- 172.21.15.43:0/3449251499 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621800fde0 con 0x7f62280b4e80 2024-04-25T09:04:45.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.250+0000 7f6226ffd640 1 -- 172.21.15.43:0/3449251499 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6218015430 con 0x7f62280b4e80 2024-04-25T09:04:45.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.250+0000 7f6230729640 1 -- 172.21.15.43:0/3449251499 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f622809f180 con 0x7f62280b4e80 2024-04-25T09:04:45.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.250+0000 7f6226ffd640 1 -- 172.21.15.43:0/3449251499 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6218015820 con 0x7f62280b4e80 2024-04-25T09:04:45.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.250+0000 7f6230729640 1 -- 172.21.15.43:0/3449251499 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f622809f740 con 0x7f62280b4e80 2024-04-25T09:04:45.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.251+0000 7f6230729640 1 -- 172.21.15.43:0/3449251499 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62280680f0 con 0x7f62280b4e80 2024-04-25T09:04:45.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.251+0000 7f6226ffd640 1 -- 172.21.15.43:0/3449251499 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f62180159c0 con 0x7f62280b4e80 2024-04-25T09:04:45.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.252+0000 7f6226ffd640 1 --2- 172.21.15.43:0/3449251499 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f621406b500 0x7f621406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:45.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.252+0000 7f6226ffd640 1 -- 172.21.15.43:0/3449251499 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f621808af30 con 0x7f62280b4e80 2024-04-25T09:04:45.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.252+0000 7f622e4c5640 1 --2- 172.21.15.43:0/3449251499 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f621406b500 0x7f621406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:45.243 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.252+0000 7f622e4c5640 1 --2- 172.21.15.43:0/3449251499 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f621406b500 0x7f621406d9c0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f6220001050 tx=0x7f6220009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:45.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.256+0000 7f6226ffd640 1 -- 172.21.15.43:0/3449251499 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6218019030 con 0x7f62280b4e80 2024-04-25T09:04:45.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.438+0000 7f6230729640 1 -- 172.21.15.43:0/3449251499 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f622808da00 con 0x7f62280b4e80 2024-04-25T09:04:45.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.439+0000 7f6226ffd640 1 -- 172.21.15.43:0/3449251499 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+16643 (secure 0 0 0) 0x7f622808da00 con 0x7f62280b4e80 2024-04-25T09:04:45.428 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:04:45.428 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":139,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:04:15.059128+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-04-25T09:04:11.735785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-04-25T09:04:12.383453+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:04:45.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.441+0000 7f6230729640 1 -- 172.21.15.43:0/3449251499 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f621406b500 msgr2=0x7f621406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:45.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.441+0000 7f6230729640 1 --2- 172.21.15.43:0/3449251499 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f621406b500 0x7f621406d9c0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f6220001050 tx=0x7f6220009040 comp rx=0 tx=0).stop 2024-04-25T09:04:45.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.441+0000 7f6230729640 1 -- 172.21.15.43:0/3449251499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62280b4e80 msgr2=0x7f62280b5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:45.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.441+0000 7f6230729640 1 --2- 172.21.15.43:0/3449251499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62280b4e80 0x7f62280b5300 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f62280d1480 tx=0x7f621800f580 comp rx=0 tx=0).stop 2024-04-25T09:04:45.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.441+0000 7f6230729640 1 -- 172.21.15.43:0/3449251499 shutdown_connections 2024-04-25T09:04:45.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.441+0000 7f6230729640 1 --2- 172.21.15.43:0/3449251499 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f62280b4e80 0x7f62280b5300 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:45.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.441+0000 7f6230729640 1 --2- 172.21.15.43:0/3449251499 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f622815c110 0x7f622809e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:45.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.441+0000 7f6230729640 1 --2- 172.21.15.43:0/3449251499 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f621406b500 0x7f621406d9c0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:45.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.441+0000 7f6230729640 1 --2- 172.21.15.43:0/3449251499 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6210003fb0 0x7f62280a1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:45.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.441+0000 7f6230729640 1 -- 172.21.15.43:0/3449251499 >> 172.21.15.43:0/3449251499 conn(0x7f622809cd00 msgr2=0x7f62280d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:45.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.441+0000 7f6230729640 1 -- 172.21.15.43:0/3449251499 shutdown_connections 2024-04-25T09:04:45.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.441+0000 7f6230729640 1 -- 172.21.15.43:0/3449251499 wait complete. 2024-04-25T09:04:45.441 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T09:04:45.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.554+0000 7fd374e1f640 1 Processor -- start 2024-04-25T09:04:45.543 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.554+0000 7fd374e1f640 1 -- start start 2024-04-25T09:04:45.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.555+0000 7fd374e1f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd3700d0d90 0x7fd3700ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:45.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.555+0000 7fd374e1f640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fd3700d2500 0x7fd3700d2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:45.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.555+0000 7fd374e1f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3700b6c50 con 0x7fd3700d39d0 2024-04-25T09:04:45.546 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.555+0000 7fd374e1f640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd37015e010 con 0x7fd3700d2500 2024-04-25T09:04:45.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.555+0000 7fd374e1f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3700d2020 con 0x7fd3700d0d90 2024-04-25T09:04:45.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.555+0000 7fd36e59c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fd3700d39d0 0x7fd3700d3db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44640/0 (socket says 172.21.15.43:44640) 2024-04-25T09:04:45.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.555+0000 7fd36e59c640 1 -- 172.21.15.43:0/2352000550 learned_addr learned my addr 172.21.15.43:0/2352000550 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:45.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.556+0000 7fd36d59a640 1 -- 172.21.15.43:0/2352000550 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1893015019 0 0) 0x7fd368001490 con 0x7fd3700d39d0 2024-04-25T09:04:45.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.556+0000 7fd36d59a640 1 -- 172.21.15.43:0/2352000550 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd35c003520 con 0x7fd3700d39d0 2024-04-25T09:04:45.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.556+0000 7fd36ed9d640 1 --2- 172.21.15.43:0/2352000550 >> v2:172.21.15.193:3300/0 conn(0x7fd3700d2500 0x7fd3700d2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:45.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.556+0000 7fd36dd9b640 1 --2- 172.21.15.43:0/2352000550 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd3700d0d90 0x7fd3700ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:45.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.556+0000 7fd36d59a640 1 -- 172.21.15.43:0/2352000550 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4238783481 0 0) 0x7fd35c003520 con 0x7fd3700d39d0 2024-04-25T09:04:45.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.556+0000 7fd36d59a640 1 -- 172.21.15.43:0/2352000550 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd3700d0d90 msgr2=0x7fd3700ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:45.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.556+0000 7fd36d59a640 1 --2- 172.21.15.43:0/2352000550 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd3700d0d90 0x7fd3700ce2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:45.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.556+0000 7fd36d59a640 1 -- 172.21.15.43:0/2352000550 >> v2:172.21.15.193:3300/0 conn(0x7fd3700d2500 msgr2=0x7fd3700d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:45.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.556+0000 7fd36d59a640 1 --2- 172.21.15.43:0/2352000550 >> v2:172.21.15.193:3300/0 conn(0x7fd3700d2500 0x7fd3700d2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:45.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.556+0000 7fd36d59a640 1 -- 172.21.15.43:0/2352000550 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3700cec40 con 0x7fd3700d39d0 2024-04-25T09:04:45.547 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.556+0000 7fd36d59a640 1 -- 172.21.15.43:0/2352000550 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7fd368001bc0 con 0x7fd3700d39d0 2024-04-25T09:04:45.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.556+0000 7fd36dd9b640 1 --2- 172.21.15.43:0/2352000550 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd3700d0d90 0x7fd3700ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:45.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.556+0000 7fd36d59a640 1 -- 172.21.15.43:0/2352000550 >> v1:172.21.15.43:6789/0 conn(0x7fd3700d39d0 legacy=0x7fd3700d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:45.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.556+0000 7fd36d59a640 1 --2- 172.21.15.43:0/2352000550 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd3700d0d90 0x7fd35c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:45.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.556+0000 7fd36d59a640 1 --2- 172.21.15.43:0/2352000550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd35c003ef0 0x7fd35c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:45.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.556+0000 7fd36d59a640 1 --2- 172.21.15.43:0/2352000550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd35c004d40 0x7fd35c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:45.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.556+0000 7fd36d59a640 1 -- 172.21.15.43:0/2352000550 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37015e010 con 0x7fd35c004d40 2024-04-25T09:04:45.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.556+0000 7fd36d59a640 1 -- 172.21.15.43:0/2352000550 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3700d2020 con 0x7fd35c003ef0 2024-04-25T09:04:45.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.556+0000 7fd36d59a640 1 -- 172.21.15.43:0/2352000550 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd35c00ba70 con 0x7fd3700d0d90 2024-04-25T09:04:45.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.556+0000 7fd36ed9d640 1 --2- 172.21.15.43:0/2352000550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd35c004d40 0x7fd35c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:45.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.556+0000 7fd36dd9b640 1 --2- 172.21.15.43:0/2352000550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd35c003ef0 0x7fd35c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:45.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.556+0000 7fd36e59c640 1 --2- 172.21.15.43:0/2352000550 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd3700d0d90 0x7fd35c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:45.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.556+0000 7fd36ed9d640 1 -- 172.21.15.43:0/2352000550 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd3700d0d90 msgr2=0x7fd35c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:45.548 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.556+0000 7fd36ed9d640 1 --2- 172.21.15.43:0/2352000550 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd3700d0d90 0x7fd35c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:45.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.556+0000 7fd36ed9d640 1 -- 172.21.15.43:0/2352000550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd35c003ef0 msgr2=0x7fd35c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:45.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.556+0000 7fd36ed9d640 1 --2- 172.21.15.43:0/2352000550 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd35c003ef0 0x7fd35c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:45.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.557+0000 7fd36ed9d640 1 -- 172.21.15.43:0/2352000550 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd35c003520 con 0x7fd35c004d40 2024-04-25T09:04:45.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.557+0000 7fd36ed9d640 1 --2- 172.21.15.43:0/2352000550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd35c004d40 0x7fd35c0051c0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7fd360009ac0 tx=0x7fd35c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3119e3e6711bcb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:45.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.557+0000 7fd36d59a640 1 -- 172.21.15.43:0/2352000550 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd360036070 con 0x7fd35c004d40 2024-04-25T09:04:45.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.557+0000 7fd36d59a640 1 -- 172.21.15.43:0/2352000550 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd360031400 con 0x7fd35c004d40 2024-04-25T09:04:45.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.557+0000 7fd36d59a640 1 -- 172.21.15.43:0/2352000550 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd360009690 con 0x7fd35c004d40 2024-04-25T09:04:45.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.558+0000 7fd374e1f640 1 -- 172.21.15.43:0/2352000550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd35c004d40 msgr2=0x7fd35c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:45.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.558+0000 7fd374e1f640 1 --2- 172.21.15.43:0/2352000550 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd35c004d40 0x7fd35c0051c0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7fd360009ac0 tx=0x7fd35c005830 comp rx=0 tx=0).stop 2024-04-25T09:04:45.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.558+0000 7fd36ed9d640 1 -- 172.21.15.43:0/2352000550 reap_dead start 2024-04-25T09:04:45.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.558+0000 7fd374e1f640 1 -- 172.21.15.43:0/2352000550 shutdown_connections 2024-04-25T09:04:45.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.558+0000 7fd374e1f640 1 -- 172.21.15.43:0/2352000550 >> 172.21.15.43:0/2352000550 conn(0x7fd37006b000 msgr2=0x7fd37006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:45.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.558+0000 7fd374e1f640 1 -- 172.21.15.43:0/2352000550 shutdown_connections 2024-04-25T09:04:45.549 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.558+0000 7fd374e1f640 1 -- 172.21.15.43:0/2352000550 wait complete. 2024-04-25T09:04:45.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.559+0000 7fd374e1f640 1 Processor -- start 2024-04-25T09:04:45.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.559+0000 7fd374e1f640 1 -- start start 2024-04-25T09:04:45.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.560+0000 7fd374e1f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd35c004d40 0x7fd3700b3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:45.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.560+0000 7fd374e1f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd3700b5510 0x7fd3701f3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:45.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.560+0000 7fd374e1f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd3700cd1d0 0x7fd3700cd650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:45.552 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.560+0000 7fd374e1f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3700cf200 con 0x7fd35c004d40 2024-04-25T09:04:45.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.560+0000 7fd374e1f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3700b2f00 con 0x7fd3700cd1d0 2024-04-25T09:04:45.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.560+0000 7fd374e1f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3700ce2a0 con 0x7fd3700b5510 2024-04-25T09:04:45.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.560+0000 7fd36e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd35c004d40 0x7fd3700b3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:45.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.560+0000 7fd36e59c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd35c004d40 0x7fd3700b3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60126/0 (socket says 172.21.15.43:60126) 2024-04-25T09:04:45.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.560+0000 7fd36e59c640 1 -- 172.21.15.43:0/2306369122 learned_addr learned my addr 172.21.15.43:0/2306369122 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:45.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.561+0000 7fd36dd9b640 1 --2- 172.21.15.43:0/2306369122 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd3700b5510 0x7fd3701f3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:45.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.561+0000 7fd36ed9d640 1 --2- 172.21.15.43:0/2306369122 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd3700cd1d0 0x7fd3700cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:45.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.561+0000 7fd36e59c640 1 -- 172.21.15.43:0/2306369122 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd3700b5510 msgr2=0x7fd3701f3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:45.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.561+0000 7fd36e59c640 1 --2- 172.21.15.43:0/2306369122 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd3700b5510 0x7fd3701f3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:45.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.561+0000 7fd36e59c640 1 -- 172.21.15.43:0/2306369122 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd3700cd1d0 msgr2=0x7fd3700cd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:45.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.561+0000 7fd36e59c640 1 --2- 172.21.15.43:0/2306369122 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd3700cd1d0 0x7fd3700cd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:45.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.561+0000 7fd36e59c640 1 -- 172.21.15.43:0/2306369122 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3700b35c0 con 0x7fd35c004d40 2024-04-25T09:04:45.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.561+0000 7fd36e59c640 1 --2- 172.21.15.43:0/2306369122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd35c004d40 0x7fd3700b3080 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7fd36800ad10 tx=0x7fd368007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:45.553 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.561+0000 7fd366ffd640 1 -- 172.21.15.43:0/2306369122 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd36800ae00 con 0x7fd35c004d40 2024-04-25T09:04:45.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.561+0000 7fd366ffd640 1 -- 172.21.15.43:0/2306369122 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd368018070 con 0x7fd35c004d40 2024-04-25T09:04:45.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.561+0000 7fd374e1f640 1 -- 172.21.15.43:0/2306369122 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd370151e40 con 0x7fd35c004d40 2024-04-25T09:04:45.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.561+0000 7fd366ffd640 1 -- 172.21.15.43:0/2306369122 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd368012780 con 0x7fd35c004d40 2024-04-25T09:04:45.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.561+0000 7fd374e1f640 1 -- 172.21.15.43:0/2306369122 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3700b5d00 con 0x7fd35c004d40 2024-04-25T09:04:45.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.562+0000 7fd374e1f640 1 -- 172.21.15.43:0/2306369122 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd370074ef0 con 0x7fd35c004d40 2024-04-25T09:04:45.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.563+0000 7fd366ffd640 1 -- 172.21.15.43:0/2306369122 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd368012920 con 0x7fd35c004d40 2024-04-25T09:04:45.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.563+0000 7fd366ffd640 1 --2- 172.21.15.43:0/2306369122 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd35006b500 0x7fd35006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:45.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.563+0000 7fd366ffd640 1 -- 172.21.15.43:0/2306369122 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd36808ace0 con 0x7fd35c004d40 2024-04-25T09:04:45.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.563+0000 7fd36dd9b640 1 --2- 172.21.15.43:0/2306369122 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd35006b500 0x7fd35006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:45.554 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.564+0000 7fd36dd9b640 1 --2- 172.21.15.43:0/2306369122 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd35006b500 0x7fd35006d9c0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fd3700b6ea0 tx=0x7fd358008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:45.557 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.568+0000 7fd366ffd640 1 -- 172.21.15.43:0/2306369122 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd3700b6c50 con 0x7fd35c004d40 2024-04-25T09:04:45.772 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.784+0000 7fd374e1f640 1 -- 172.21.15.43:0/2306369122 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd37015f5b0 con 0x7fd35c004d40 2024-04-25T09:04:45.772 INFO:tasks.ceph.mon.a.smithi043.stderr:2024-04-25T09:04:45.784+0000 7fe4fb198640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T09:04:45.774 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.785+0000 7fd366ffd640 1 -- 172.21.15.43:0/2306369122 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v91) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd368059b60 con 0x7fd35c004d40 2024-04-25T09:04:45.774 INFO:teuthology.orchestra.run.smithi043.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T09:04:45.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.788+0000 7fd374e1f640 1 -- 172.21.15.43:0/2306369122 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd35006b500 msgr2=0x7fd35006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:45.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.788+0000 7fd374e1f640 1 --2- 172.21.15.43:0/2306369122 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd35006b500 0x7fd35006d9c0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fd3700b6ea0 tx=0x7fd358008040 comp rx=0 tx=0).stop 2024-04-25T09:04:45.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.788+0000 7fd374e1f640 1 -- 172.21.15.43:0/2306369122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd35c004d40 msgr2=0x7fd3700b3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:45.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.788+0000 7fd374e1f640 1 --2- 172.21.15.43:0/2306369122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd35c004d40 0x7fd3700b3080 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7fd36800ad10 tx=0x7fd368007430 comp rx=0 tx=0).stop 2024-04-25T09:04:45.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.788+0000 7fd374e1f640 1 -- 172.21.15.43:0/2306369122 shutdown_connections 2024-04-25T09:04:45.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.788+0000 7fd374e1f640 1 --2- 172.21.15.43:0/2306369122 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fd35006b500 0x7fd35006d9c0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:45.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.788+0000 7fd374e1f640 1 --2- 172.21.15.43:0/2306369122 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fd3700cd1d0 0x7fd3700cd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:45.777 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.788+0000 7fd374e1f640 1 --2- 172.21.15.43:0/2306369122 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fd3700b5510 0x7fd3701f3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:45.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.788+0000 7fd374e1f640 1 --2- 172.21.15.43:0/2306369122 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fd35c004d40 0x7fd3700b3080 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:45.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.788+0000 7fd374e1f640 1 -- 172.21.15.43:0/2306369122 >> 172.21.15.43:0/2306369122 conn(0x7fd37006b000 msgr2=0x7fd370153cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:45.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.789+0000 7fd374e1f640 1 -- 172.21.15.43:0/2306369122 shutdown_connections 2024-04-25T09:04:45.778 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.789+0000 7fd374e1f640 1 -- 172.21.15.43:0/2306369122 wait complete. 2024-04-25T09:04:45.789 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T09:04:45.813 INFO:tasks.ceph.mds.c.smithi043.stdout:starting mds.c at 2024-04-25T09:04:45.817 INFO:tasks.ceph.mds.b.smithi193.stdout:starting mds.b at 2024-04-25T09:04:45.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.904+0000 7fcc003ce640 1 Processor -- start 2024-04-25T09:04:45.893 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.904+0000 7fcc003ce640 1 -- start start 2024-04-25T09:04:45.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.905+0000 7fcc003ce640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcbf815dd80 0x7fcbf80cf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:45.894 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.905+0000 7fcc003ce640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fcbf80d17b0 0x7fcbf80d1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:45.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.905+0000 7fcc003ce640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbf80b2fe0 con 0x7fcbf815e160 2024-04-25T09:04:45.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.905+0000 7fcc003ce640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbf80d2150 con 0x7fcbf80d17b0 2024-04-25T09:04:45.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.905+0000 7fcc003ce640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf80cfc60 con 0x7fcbf815dd80 2024-04-25T09:04:45.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.906+0000 7fcbfe96b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fcbf815e160 0x7fcbf815e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44656/0 (socket says 172.21.15.43:44656) 2024-04-25T09:04:45.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.906+0000 7fcbfe96b640 1 -- 172.21.15.43:0/1681223890 learned_addr learned my addr 172.21.15.43:0/1681223890 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:45.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.906+0000 7fcbfe16a640 1 --2- 172.21.15.43:0/1681223890 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcbf815dd80 0x7fcbf80cf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:45.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.906+0000 7fcbfd969640 1 --2- 172.21.15.43:0/1681223890 >> v2:172.21.15.193:3300/0 conn(0x7fcbf80d17b0 0x7fcbf80d1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:45.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.906+0000 7fcbfd168640 1 -- 172.21.15.43:0/1681223890 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2462980782 0 0) 0x7fcbf80b2fe0 con 0x7fcbf815e160 2024-04-25T09:04:45.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.906+0000 7fcbfd168640 1 -- 172.21.15.43:0/1681223890 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbe4003520 con 0x7fcbf815e160 2024-04-25T09:04:45.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.906+0000 7fcbfe16a640 1 -- 172.21.15.43:0/1681223890 >> v2:172.21.15.193:3300/0 conn(0x7fcbf80d17b0 msgr2=0x7fcbf80d1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:45.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.906+0000 7fcbfe16a640 1 --2- 172.21.15.43:0/1681223890 >> v2:172.21.15.193:3300/0 conn(0x7fcbf80d17b0 0x7fcbf80d1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:45.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.906+0000 7fcbfe16a640 1 -- 172.21.15.43:0/1681223890 >> v1:172.21.15.43:6789/0 conn(0x7fcbf815e160 legacy=0x7fcbf815e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:45.895 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.906+0000 7fcbfe16a640 1 -- 172.21.15.43:0/1681223890 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf80ced70 con 0x7fcbf815dd80 2024-04-25T09:04:45.896 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.906+0000 7fcbfd969640 1 --2- 172.21.15.43:0/1681223890 >> v2:172.21.15.193:3300/0 conn(0x7fcbf80d17b0 0x7fcbf80d1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:04:45.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.907+0000 7fcbfe16a640 1 --2- 172.21.15.43:0/1681223890 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcbf815dd80 0x7fcbf80cf720 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcbf40097a0 tx=0x7fcbf402a100 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f81c0d8c1318e2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:45.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.907+0000 7fcbfd168640 1 -- 172.21.15.43:0/1681223890 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf4037070 con 0x7fcbf815dd80 2024-04-25T09:04:45.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.907+0000 7fcbfd168640 1 -- 172.21.15.43:0/1681223890 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbf402ae50 con 0x7fcbf815dd80 2024-04-25T09:04:45.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.907+0000 7fcbfd168640 1 -- 172.21.15.43:0/1681223890 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf4032660 con 0x7fcbf815dd80 2024-04-25T09:04:45.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.908+0000 7fcc003ce640 1 -- 172.21.15.43:0/1681223890 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcbf815dd80 msgr2=0x7fcbf80cf720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:45.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.908+0000 7fcc003ce640 1 --2- 172.21.15.43:0/1681223890 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcbf815dd80 0x7fcbf80cf720 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fcbf40097a0 tx=0x7fcbf402a100 comp rx=0 tx=0).stop 2024-04-25T09:04:45.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.908+0000 7fcc003ce640 1 -- 172.21.15.43:0/1681223890 shutdown_connections 2024-04-25T09:04:45.897 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.908+0000 7fcc003ce640 1 --2- 172.21.15.43:0/1681223890 >> v2:172.21.15.193:3300/0 conn(0x7fcbf80d17b0 0x7fcbf80d1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:45.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.908+0000 7fcc003ce640 1 --2- 172.21.15.43:0/1681223890 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcbf815dd80 0x7fcbf80cf720 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:45.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.908+0000 7fcc003ce640 1 -- 172.21.15.43:0/1681223890 >> 172.21.15.43:0/1681223890 conn(0x7fcbf8072aa0 msgr2=0x7fcbf809aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:45.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.908+0000 7fcc003ce640 1 -- 172.21.15.43:0/1681223890 shutdown_connections 2024-04-25T09:04:45.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.908+0000 7fcc003ce640 1 -- 172.21.15.43:0/1681223890 wait complete. 2024-04-25T09:04:45.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.909+0000 7fcc003ce640 1 Processor -- start 2024-04-25T09:04:45.898 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.909+0000 7fcc003ce640 1 -- start start 2024-04-25T09:04:45.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.910+0000 7fcc003ce640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbf80d17b0 0x7fcbf8157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:45.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.910+0000 7fcc003ce640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcbf815dd80 0x7fcbf8158be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:45.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.910+0000 7fcc003ce640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbf815e160 0x7fcbf815a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:45.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.910+0000 7fcc003ce640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf80cf330 con 0x7fcbf815e160 2024-04-25T09:04:45.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.910+0000 7fcc003ce640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf80b32c0 con 0x7fcbf80d17b0 2024-04-25T09:04:45.900 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.910+0000 7fcc003ce640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf80b2c40 con 0x7fcbf815dd80 2024-04-25T09:04:45.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.910+0000 7fcbfe96b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbf815e160 0x7fcbf815a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:45.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.910+0000 7fcbfe16a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbf80d17b0 0x7fcbf8157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:45.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.910+0000 7fcbfe96b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbf815e160 0x7fcbf815a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60158/0 (socket says 172.21.15.43:60158) 2024-04-25T09:04:45.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.910+0000 7fcbfe96b640 1 -- 172.21.15.43:0/4054232981 learned_addr learned my addr 172.21.15.43:0/4054232981 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:45.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.910+0000 7fcbfd969640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcbf815dd80 0x7fcbf8158be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:45.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.910+0000 7fcbfe96b640 1 -- 172.21.15.43:0/4054232981 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcbf815dd80 msgr2=0x7fcbf8158be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:45.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.910+0000 7fcbfe96b640 1 --2- 172.21.15.43:0/4054232981 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcbf815dd80 0x7fcbf8158be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:45.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.910+0000 7fcbfe96b640 1 -- 172.21.15.43:0/4054232981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbf80d17b0 msgr2=0x7fcbf8157e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:45.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.910+0000 7fcbfe96b640 1 --2- 172.21.15.43:0/4054232981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbf80d17b0 0x7fcbf8157e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:45.901 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.910+0000 7fcbfe96b640 1 -- 172.21.15.43:0/4054232981 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf8159680 con 0x7fcbf815e160 2024-04-25T09:04:45.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.910+0000 7fcbfe16a640 1 --2- 172.21.15.43:0/4054232981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbf80d17b0 0x7fcbf8157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:04:45.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.910+0000 7fcbfd969640 1 --2- 172.21.15.43:0/4054232981 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcbf815dd80 0x7fcbf8158be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:04:45.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.910+0000 7fcbfe96b640 1 --2- 172.21.15.43:0/4054232981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbf815e160 0x7fcbf815a810 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7fcbf000cad0 tx=0x7fcbf000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:45.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.910+0000 7fcbeeffd640 1 -- 172.21.15.43:0/4054232981 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf0017070 con 0x7fcbf815e160 2024-04-25T09:04:45.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.911+0000 7fcbeeffd640 1 -- 172.21.15.43:0/4054232981 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbf000ac30 con 0x7fcbf815e160 2024-04-25T09:04:45.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.911+0000 7fcc003ce640 1 -- 172.21.15.43:0/4054232981 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbf8159960 con 0x7fcbf815e160 2024-04-25T09:04:45.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.911+0000 7fcbeeffd640 1 -- 172.21.15.43:0/4054232981 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf0012400 con 0x7fcbf815e160 2024-04-25T09:04:45.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.911+0000 7fcc003ce640 1 -- 172.21.15.43:0/4054232981 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbf80b4960 con 0x7fcbf815e160 2024-04-25T09:04:45.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.912+0000 7fcc003ce640 1 -- 172.21.15.43:0/4054232981 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbf8074f60 con 0x7fcbf815e160 2024-04-25T09:04:45.902 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.912+0000 7fcbeeffd640 1 -- 172.21.15.43:0/4054232981 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcbf001a070 con 0x7fcbf815e160 2024-04-25T09:04:45.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.913+0000 7fcbeeffd640 1 --2- 172.21.15.43:0/4054232981 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcbdc06b4d0 0x7fcbdc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:45.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.913+0000 7fcbeeffd640 1 -- 172.21.15.43:0/4054232981 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fcbf0089f80 con 0x7fcbf815e160 2024-04-25T09:04:45.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.913+0000 7fcbfe16a640 1 --2- 172.21.15.43:0/4054232981 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcbdc06b4d0 0x7fcbdc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:45.903 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.913+0000 7fcbfe16a640 1 --2- 172.21.15.43:0/4054232981 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcbdc06b4d0 0x7fcbdc06d990 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4034000 tx=0x7fcbf402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:45.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:45.918+0000 7fcbeeffd640 1 -- 172.21.15.43:0/4054232981 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcbf0058740 con 0x7fcbf815e160 2024-04-25T09:04:46.125 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.137+0000 7fcc003ce640 1 -- 172.21.15.43:0/4054232981 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fcbf80d23f0 con 0x7fcbf815e160 2024-04-25T09:04:46.566 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:04:46.569 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:04:46.779 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.791+0000 7fcbeeffd640 1 -- 172.21.15.43:0/4054232981 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v92) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcbf005c7e0 con 0x7fcbf815e160 2024-04-25T09:04:46.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.794+0000 7fcc003ce640 1 -- 172.21.15.43:0/4054232981 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcbdc06b4d0 msgr2=0x7fcbdc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:46.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.794+0000 7fcc003ce640 1 --2- 172.21.15.43:0/4054232981 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcbdc06b4d0 0x7fcbdc06d990 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fcbf4034000 tx=0x7fcbf402e000 comp rx=0 tx=0).stop 2024-04-25T09:04:46.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.794+0000 7fcc003ce640 1 -- 172.21.15.43:0/4054232981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbf815e160 msgr2=0x7fcbf815a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:46.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.794+0000 7fcc003ce640 1 --2- 172.21.15.43:0/4054232981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbf815e160 0x7fcbf815a810 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7fcbf000cad0 tx=0x7fcbf000cf90 comp rx=0 tx=0).stop 2024-04-25T09:04:46.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.794+0000 7fcc003ce640 1 -- 172.21.15.43:0/4054232981 shutdown_connections 2024-04-25T09:04:46.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.794+0000 7fcc003ce640 1 --2- 172.21.15.43:0/4054232981 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fcbf815e160 0x7fcbf815a810 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:46.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.794+0000 7fcc003ce640 1 --2- 172.21.15.43:0/4054232981 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fcbf815dd80 0x7fcbf8158be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:46.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.794+0000 7fcc003ce640 1 --2- 172.21.15.43:0/4054232981 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fcbdc06b4d0 0x7fcbdc06d990 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:46.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.794+0000 7fcc003ce640 1 --2- 172.21.15.43:0/4054232981 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fcbf80d17b0 0x7fcbf8157e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:46.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.794+0000 7fcc003ce640 1 -- 172.21.15.43:0/4054232981 >> 172.21.15.43:0/4054232981 conn(0x7fcbf8072aa0 msgr2=0x7fcbf80b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:46.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.794+0000 7fcc003ce640 1 -- 172.21.15.43:0/4054232981 shutdown_connections 2024-04-25T09:04:46.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.794+0000 7fcc003ce640 1 -- 172.21.15.43:0/4054232981 wait complete. 2024-04-25T09:04:46.796 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T09:04:46.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.917+0000 7f210744c640 1 Processor -- start 2024-04-25T09:04:46.905 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.917+0000 7f210744c640 1 -- start start 2024-04-25T09:04:46.906 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.918+0000 7f210744c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2100147290 0x7f21000cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:46.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.918+0000 7f210744c640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f21000ce9b0 0x7f210015e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:46.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.918+0000 7f210744c640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21000cef60 con 0x7f21000cd8f0 2024-04-25T09:04:46.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.918+0000 7f210744c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2100071460 con 0x7f2100147290 2024-04-25T09:04:46.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.918+0000 7f210744c640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21000716b0 con 0x7f21000ce9b0 2024-04-25T09:04:46.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.918+0000 7f21051e8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2100147290 0x7f21000cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:46.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.918+0000 7f21051e8640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f2100147290 0x7f21000cd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60164/0 (socket says 172.21.15.43:60164) 2024-04-25T09:04:46.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.918+0000 7f21051e8640 1 -- 172.21.15.43:0/2252954323 learned_addr learned my addr 172.21.15.43:0/2252954323 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:46.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.918+0000 7f21059e9640 1 --2- 172.21.15.43:0/2252954323 >> v2:172.21.15.193:3300/0 conn(0x7f21000ce9b0 0x7f210015e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:46.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.918+0000 7f21051e8640 1 -- 172.21.15.43:0/2252954323 >> v2:172.21.15.193:3300/0 conn(0x7f21000ce9b0 msgr2=0x7f210015e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:46.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.918+0000 7f21051e8640 1 --2- 172.21.15.43:0/2252954323 >> v2:172.21.15.193:3300/0 conn(0x7f21000ce9b0 0x7f210015e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:46.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.918+0000 7f21051e8640 1 -- 172.21.15.43:0/2252954323 >> v1:172.21.15.193:6789/0 conn(0x7f21000cd8f0 legacy=0x7f21000ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:46.907 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.918+0000 7f21051e8640 1 -- 172.21.15.43:0/2252954323 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2100071990 con 0x7f2100147290 2024-04-25T09:04:46.908 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.919+0000 7f21051e8640 1 --2- 172.21.15.43:0/2252954323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2100147290 0x7f21000cd3b0 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f21000b3f10 tx=0x7f20fc02a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d372b4b0d1a2378c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:46.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.919+0000 7f20f7fff640 1 -- 172.21.15.43:0/2252954323 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc037070 con 0x7f2100147290 2024-04-25T09:04:46.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.919+0000 7f20f7fff640 1 -- 172.21.15.43:0/2252954323 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20fc032440 con 0x7f2100147290 2024-04-25T09:04:46.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.919+0000 7f20f7fff640 1 -- 172.21.15.43:0/2252954323 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc032810 con 0x7f2100147290 2024-04-25T09:04:46.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.920+0000 7f210744c640 1 -- 172.21.15.43:0/2252954323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2100147290 msgr2=0x7f21000cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:46.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.920+0000 7f210744c640 1 --2- 172.21.15.43:0/2252954323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2100147290 0x7f21000cd3b0 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f21000b3f10 tx=0x7f20fc02a150 comp rx=0 tx=0).stop 2024-04-25T09:04:46.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.920+0000 7f210744c640 1 -- 172.21.15.43:0/2252954323 shutdown_connections 2024-04-25T09:04:46.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.920+0000 7f210744c640 1 --2- 172.21.15.43:0/2252954323 >> v2:172.21.15.193:3300/0 conn(0x7f21000ce9b0 0x7f210015e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:46.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.920+0000 7f210744c640 1 --2- 172.21.15.43:0/2252954323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f2100147290 0x7f21000cd3b0 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:46.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.920+0000 7f210744c640 1 -- 172.21.15.43:0/2252954323 >> 172.21.15.43:0/2252954323 conn(0x7f210006be20 msgr2=0x7f210006c200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:46.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.920+0000 7f210744c640 1 -- 172.21.15.43:0/2252954323 shutdown_connections 2024-04-25T09:04:46.909 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.920+0000 7f210744c640 1 -- 172.21.15.43:0/2252954323 wait complete. 2024-04-25T09:04:46.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.921+0000 7f210744c640 1 Processor -- start 2024-04-25T09:04:46.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.921+0000 7f210744c640 1 -- start start 2024-04-25T09:04:46.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.922+0000 7f210744c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21000cd8f0 0x7f210015e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:46.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.922+0000 7f210744c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f21000ce9b0 0x7f21001f2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:46.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.922+0000 7f210744c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2100147290 0x7f21001454f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:46.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.922+0000 7f210744c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21001477f0 con 0x7f21000cd8f0 2024-04-25T09:04:46.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.922+0000 7f210744c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2100147670 con 0x7f2100147290 2024-04-25T09:04:46.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.922+0000 7f210744c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2100072170 con 0x7f21000ce9b0 2024-04-25T09:04:46.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.922+0000 7f21051e8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21000cd8f0 0x7f210015e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:46.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.922+0000 7f21051e8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21000cd8f0 0x7f210015e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60172/0 (socket says 172.21.15.43:60172) 2024-04-25T09:04:46.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.922+0000 7f21051e8640 1 -- 172.21.15.43:0/1751654708 learned_addr learned my addr 172.21.15.43:0/1751654708 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:46.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.922+0000 7f21059e9640 1 --2- 172.21.15.43:0/1751654708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2100147290 0x7f21001454f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:46.912 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.922+0000 7f21049e7640 1 --2- 172.21.15.43:0/1751654708 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f21000ce9b0 0x7f21001f2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:46.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.922+0000 7f21051e8640 1 -- 172.21.15.43:0/1751654708 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f21000ce9b0 msgr2=0x7f21001f2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:46.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.922+0000 7f21051e8640 1 --2- 172.21.15.43:0/1751654708 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f21000ce9b0 0x7f21001f2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:46.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.922+0000 7f21051e8640 1 -- 172.21.15.43:0/1751654708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2100147290 msgr2=0x7f21001454f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:46.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.922+0000 7f21051e8640 1 --2- 172.21.15.43:0/1751654708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2100147290 0x7f21001454f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:46.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.922+0000 7f21051e8640 1 -- 172.21.15.43:0/1751654708 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f210015eee0 con 0x7f21000cd8f0 2024-04-25T09:04:46.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.922+0000 7f21051e8640 1 --2- 172.21.15.43:0/1751654708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21000cd8f0 0x7f210015e9a0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f20fc02a190 tx=0x7f20fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:46.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.922+0000 7f20f5ffb640 1 -- 172.21.15.43:0/1751654708 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc041020 con 0x7f21000cd8f0 2024-04-25T09:04:46.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.922+0000 7f20f5ffb640 1 -- 172.21.15.43:0/1751654708 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20fc0306c0 con 0x7f21000cd8f0 2024-04-25T09:04:46.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.922+0000 7f210744c640 1 -- 172.21.15.43:0/1751654708 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21001f3270 con 0x7f21000cd8f0 2024-04-25T09:04:46.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.922+0000 7f20f5ffb640 1 -- 172.21.15.43:0/1751654708 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc037040 con 0x7f21000cd8f0 2024-04-25T09:04:46.913 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.923+0000 7f210744c640 1 -- 172.21.15.43:0/1751654708 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21001602a0 con 0x7f21000cd8f0 2024-04-25T09:04:46.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.924+0000 7f210744c640 1 -- 172.21.15.43:0/1751654708 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2100068a40 con 0x7f21000cd8f0 2024-04-25T09:04:46.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.924+0000 7f20f5ffb640 1 -- 172.21.15.43:0/1751654708 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f20fc044020 con 0x7f21000cd8f0 2024-04-25T09:04:46.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.925+0000 7f20f5ffb640 1 --2- 172.21.15.43:0/1751654708 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f20e406b4d0 0x7f20e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:46.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.925+0000 7f20f5ffb640 1 -- 172.21.15.43:0/1751654708 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f20fc0aaa20 con 0x7f21000cd8f0 2024-04-25T09:04:46.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.925+0000 7f21049e7640 1 --2- 172.21.15.43:0/1751654708 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f20e406b4d0 0x7f20e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:46.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.925+0000 7f21049e7640 1 --2- 172.21.15.43:0/1751654708 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f20e406b4d0 0x7f20e406d990 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f2100146270 tx=0x7f20f0009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:46.917 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:46.929+0000 7f20f5ffb640 1 -- 172.21.15.43:0/1751654708 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f20fc04b030 con 0x7f21000cd8f0 2024-04-25T09:04:47.099 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.110+0000 7f210744c640 1 -- 172.21.15.43:0/1751654708 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2100160bd0 con 0x7f21000cd8f0 2024-04-25T09:04:47.781 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.793+0000 7f20f5ffb640 1 -- 172.21.15.43:0/1751654708 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v141) v1 ==== 181+0+0 (secure 0 0 0) 0x7f20fc07f030 con 0x7f21000cd8f0 2024-04-25T09:04:47.781 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_metadata' removed 2024-04-25T09:04:47.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.796+0000 7f210744c640 1 -- 172.21.15.43:0/1751654708 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f20e406b4d0 msgr2=0x7f20e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:47.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.796+0000 7f210744c640 1 --2- 172.21.15.43:0/1751654708 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f20e406b4d0 0x7f20e406d990 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f2100146270 tx=0x7f20f0009040 comp rx=0 tx=0).stop 2024-04-25T09:04:47.785 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.796+0000 7f210744c640 1 -- 172.21.15.43:0/1751654708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21000cd8f0 msgr2=0x7f210015e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:47.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.796+0000 7f210744c640 1 --2- 172.21.15.43:0/1751654708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21000cd8f0 0x7f210015e9a0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f20fc02a190 tx=0x7f20fc0046c0 comp rx=0 tx=0).stop 2024-04-25T09:04:47.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.797+0000 7f210744c640 1 -- 172.21.15.43:0/1751654708 shutdown_connections 2024-04-25T09:04:47.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.797+0000 7f210744c640 1 --2- 172.21.15.43:0/1751654708 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f20e406b4d0 0x7f20e406d990 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:47.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.797+0000 7f210744c640 1 --2- 172.21.15.43:0/1751654708 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f2100147290 0x7f21001454f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:47.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.797+0000 7f210744c640 1 --2- 172.21.15.43:0/1751654708 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f21000ce9b0 0x7f21001f2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:47.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.797+0000 7f210744c640 1 --2- 172.21.15.43:0/1751654708 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f21000cd8f0 0x7f210015e9a0 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:47.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.797+0000 7f210744c640 1 -- 172.21.15.43:0/1751654708 >> 172.21.15.43:0/1751654708 conn(0x7f210006be20 msgr2=0x7f21000b3140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:47.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.797+0000 7f210744c640 1 -- 172.21.15.43:0/1751654708 shutdown_connections 2024-04-25T09:04:47.786 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.797+0000 7f210744c640 1 -- 172.21.15.43:0/1751654708 wait complete. 2024-04-25T09:04:47.800 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T09:04:47.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.922+0000 7f10a64ee640 1 Processor -- start 2024-04-25T09:04:47.911 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.922+0000 7f10a64ee640 1 -- start start 2024-04-25T09:04:47.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.923+0000 7f10a64ee640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f10a00b0cf0 0x7f10a009ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:47.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.923+0000 7f10a64ee640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10a00b5ce0 con 0x7f10a006ccd0 2024-04-25T09:04:47.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.923+0000 7f10a64ee640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10a00b5f80 con 0x7f10a015db50 2024-04-25T09:04:47.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.923+0000 7f10a64ee640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a00b1540 con 0x7f10a00b0cf0 2024-04-25T09:04:47.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.923+0000 7f109f7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f10a006ccd0 0x7f10a00b19d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44664/0 (socket says 172.21.15.43:44664) 2024-04-25T09:04:47.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.923+0000 7f109f7fe640 1 -- 172.21.15.43:0/3466405644 learned_addr learned my addr 172.21.15.43:0/3466405644 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:47.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.923+0000 7f109effd640 1 -- 172.21.15.43:0/3466405644 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 914937381 0 0) 0x7f10a00b5ce0 con 0x7f10a006ccd0 2024-04-25T09:04:47.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.923+0000 7f10a4a8b640 1 --2- 172.21.15.43:0/3466405644 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f10a00b0cf0 0x7f10a009ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:47.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.924+0000 7f109effd640 1 -- 172.21.15.43:0/3466405644 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1088003520 con 0x7f10a006ccd0 2024-04-25T09:04:47.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.924+0000 7f109effd640 1 -- 172.21.15.43:0/3466405644 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2208395574 0 0) 0x7f1088003520 con 0x7f10a006ccd0 2024-04-25T09:04:47.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.924+0000 7f109effd640 1 -- 172.21.15.43:0/3466405644 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f10a00b0cf0 msgr2=0x7f10a009ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:47.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.924+0000 7f109effd640 1 --2- 172.21.15.43:0/3466405644 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f10a00b0cf0 0x7f10a009ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:47.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.924+0000 7f109effd640 1 -- 172.21.15.43:0/3466405644 >> v1:172.21.15.193:6789/0 conn(0x7f10a015db50 legacy=0x7f10a006c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:47.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.924+0000 7f109effd640 1 -- 172.21.15.43:0/3466405644 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a00b6260 con 0x7f10a006ccd0 2024-04-25T09:04:47.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.924+0000 7f109effd640 1 -- 172.21.15.43:0/3466405644 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f1090001690 con 0x7f10a006ccd0 2024-04-25T09:04:47.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.924+0000 7f109effd640 1 -- 172.21.15.43:0/3466405644 >> v1:172.21.15.43:6789/0 conn(0x7f10a006ccd0 legacy=0x7f10a00b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:47.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.924+0000 7f109effd640 1 --2- 172.21.15.43:0/3466405644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10880039e0 0x7f1088003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:47.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.924+0000 7f109effd640 1 --2- 172.21.15.43:0/3466405644 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f10a00b0cf0 0x7f1088004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:47.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.924+0000 7f109effd640 1 --2- 172.21.15.43:0/3466405644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1088004d40 0x7f10880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:47.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.924+0000 7f109effd640 1 -- 172.21.15.43:0/3466405644 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a00b1540 con 0x7f10880039e0 2024-04-25T09:04:47.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.924+0000 7f109effd640 1 -- 172.21.15.43:0/3466405644 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f108800baa0 con 0x7f1088004d40 2024-04-25T09:04:47.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.924+0000 7f109effd640 1 -- 172.21.15.43:0/3466405644 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f108800bca0 con 0x7f10a00b0cf0 2024-04-25T09:04:47.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.924+0000 7f109ffff640 1 --2- 172.21.15.43:0/3466405644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10880039e0 0x7f1088003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:47.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.924+0000 7f109f7fe640 1 --2- 172.21.15.43:0/3466405644 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f10a00b0cf0 0x7f1088004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:47.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.924+0000 7f10a4a8b640 1 --2- 172.21.15.43:0/3466405644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1088004d40 0x7f10880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:47.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.924+0000 7f109ffff640 1 -- 172.21.15.43:0/3466405644 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f10a00b0cf0 msgr2=0x7f1088004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:47.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.924+0000 7f109ffff640 1 --2- 172.21.15.43:0/3466405644 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f10a00b0cf0 0x7f1088004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:47.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.924+0000 7f109ffff640 1 -- 172.21.15.43:0/3466405644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1088004d40 msgr2=0x7f10880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:47.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.924+0000 7f109ffff640 1 --2- 172.21.15.43:0/3466405644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1088004d40 0x7f10880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:47.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.924+0000 7f109ffff640 1 -- 172.21.15.43:0/3466405644 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1088003520 con 0x7f10880039e0 2024-04-25T09:04:47.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.924+0000 7f109f7fe640 1 --2- 172.21.15.43:0/3466405644 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f10a00b0cf0 0x7f1088004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:47.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.924+0000 7f10a4a8b640 1 --2- 172.21.15.43:0/3466405644 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f1088004d40 0x7f10880051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:47.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.925+0000 7f109ffff640 1 --2- 172.21.15.43:0/3466405644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10880039e0 0x7f1088003e00 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f1094001720 tx=0x7f1088005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc17d41490a8dcf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:47.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.925+0000 7f109effd640 1 -- 172.21.15.43:0/3466405644 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1094001370 con 0x7f10880039e0 2024-04-25T09:04:47.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.925+0000 7f109effd640 1 -- 172.21.15.43:0/3466405644 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10940052d0 con 0x7f10880039e0 2024-04-25T09:04:47.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.925+0000 7f109effd640 1 -- 172.21.15.43:0/3466405644 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109402abc0 con 0x7f10880039e0 2024-04-25T09:04:47.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.926+0000 7f10a64ee640 1 -- 172.21.15.43:0/3466405644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10880039e0 msgr2=0x7f1088003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:47.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.926+0000 7f10a64ee640 1 --2- 172.21.15.43:0/3466405644 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10880039e0 0x7f1088003e00 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f1094001720 tx=0x7f1088005830 comp rx=0 tx=0).stop 2024-04-25T09:04:47.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.926+0000 7f10a4a8b640 1 -- 172.21.15.43:0/3466405644 reap_dead start 2024-04-25T09:04:47.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.926+0000 7f10a64ee640 1 -- 172.21.15.43:0/3466405644 shutdown_connections 2024-04-25T09:04:47.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.926+0000 7f10a64ee640 1 -- 172.21.15.43:0/3466405644 >> 172.21.15.43:0/3466405644 conn(0x7f10a009aa70 msgr2=0x7f10a00b6be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:47.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.926+0000 7f10a64ee640 1 -- 172.21.15.43:0/3466405644 shutdown_connections 2024-04-25T09:04:47.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.926+0000 7f10a64ee640 1 -- 172.21.15.43:0/3466405644 wait complete. 2024-04-25T09:04:47.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.927+0000 7f10a64ee640 1 Processor -- start 2024-04-25T09:04:47.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.927+0000 7f10a64ee640 1 -- start start 2024-04-25T09:04:47.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.928+0000 7f10a64ee640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f10880039e0 0x7f10a0160a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:47.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.928+0000 7f10a64ee640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10a006f540 0x7f10a006f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:47.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.928+0000 7f10a64ee640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10a01f3b40 0x7f10a01f3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:47.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.928+0000 7f10a64ee640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a015df30 con 0x7f10a01f3b40 2024-04-25T09:04:47.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.928+0000 7f10a64ee640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a00b11d0 con 0x7f10a006f540 2024-04-25T09:04:47.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.928+0000 7f10a64ee640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a015e0b0 con 0x7f10880039e0 2024-04-25T09:04:47.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.928+0000 7f10a4a8b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10a01f3b40 0x7f10a01f3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:47.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.928+0000 7f10a4a8b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10a01f3b40 0x7f10a01f3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60194/0 (socket says 172.21.15.43:60194) 2024-04-25T09:04:47.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.928+0000 7f10a4a8b640 1 -- 172.21.15.43:0/2814544671 learned_addr learned my addr 172.21.15.43:0/2814544671 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:47.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.928+0000 7f109ffff640 1 --2- 172.21.15.43:0/2814544671 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f10880039e0 0x7f10a0160a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:47.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.928+0000 7f109f7fe640 1 --2- 172.21.15.43:0/2814544671 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10a006f540 0x7f10a006f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:47.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.928+0000 7f10a4a8b640 1 -- 172.21.15.43:0/2814544671 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f10880039e0 msgr2=0x7f10a0160a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:47.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.928+0000 7f10a4a8b640 1 --2- 172.21.15.43:0/2814544671 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f10880039e0 0x7f10a0160a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:47.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.928+0000 7f10a4a8b640 1 -- 172.21.15.43:0/2814544671 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10a006f540 msgr2=0x7f10a006f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:47.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.928+0000 7f10a4a8b640 1 --2- 172.21.15.43:0/2814544671 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10a006f540 0x7f10a006f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:47.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.928+0000 7f10a4a8b640 1 -- 172.21.15.43:0/2814544671 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a01f4b90 con 0x7f10a01f3b40 2024-04-25T09:04:47.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.928+0000 7f109f7fe640 1 --2- 172.21.15.43:0/2814544671 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10a006f540 0x7f10a006f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:47.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.929+0000 7f10a4a8b640 1 --2- 172.21.15.43:0/2814544671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10a01f3b40 0x7f10a01f3fc0 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f10a0070d90 tx=0x7f1098010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:47.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.929+0000 7f109cff9640 1 -- 172.21.15.43:0/2814544671 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1098010d20 con 0x7f10a01f3b40 2024-04-25T09:04:47.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.929+0000 7f109cff9640 1 -- 172.21.15.43:0/2814544671 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1098003990 con 0x7f10a01f3b40 2024-04-25T09:04:47.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.929+0000 7f10a64ee640 1 -- 172.21.15.43:0/2814544671 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10a01f48b0 con 0x7f10a01f3b40 2024-04-25T09:04:47.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.929+0000 7f109cff9640 1 -- 172.21.15.43:0/2814544671 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1098003dc0 con 0x7f10a01f3b40 2024-04-25T09:04:47.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.929+0000 7f10a64ee640 1 -- 172.21.15.43:0/2814544671 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10a01f5260 con 0x7f10a01f3b40 2024-04-25T09:04:47.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.930+0000 7f10a64ee640 1 -- 172.21.15.43:0/2814544671 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10a0068a40 con 0x7f10a01f3b40 2024-04-25T09:04:47.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.930+0000 7f109cff9640 1 -- 172.21.15.43:0/2814544671 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1098005d10 con 0x7f10a01f3b40 2024-04-25T09:04:47.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.931+0000 7f109cff9640 1 --2- 172.21.15.43:0/2814544671 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f108c06b4d0 0x7f108c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:47.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.931+0000 7f109cff9640 1 -- 172.21.15.43:0/2814544671 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f109808a650 con 0x7f10a01f3b40 2024-04-25T09:04:47.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.931+0000 7f109ffff640 1 --2- 172.21.15.43:0/2814544671 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f108c06b4d0 0x7f108c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:47.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.931+0000 7f109ffff640 1 --2- 172.21.15.43:0/2814544671 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f108c06b4d0 0x7f108c06d990 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f1094032000 tx=0x7f109402e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:47.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:47.935+0000 7f109cff9640 1 -- 172.21.15.43:0/2814544671 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1098025020 con 0x7f10a01f3b40 2024-04-25T09:04:48.108 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.120+0000 7f10a64ee640 1 -- 172.21.15.43:0/2814544671 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f10a006dc30 con 0x7f10a01f3b40 2024-04-25T09:04:48.787 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.798+0000 7f109cff9640 1 -- 172.21.15.43:0/2814544671 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v142) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1098059630 con 0x7f10a01f3b40 2024-04-25T09:04:48.787 INFO:teuthology.orchestra.run.smithi043.stderr:pool 'cephfs_data' removed 2024-04-25T09:04:48.790 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.802+0000 7f10a64ee640 1 -- 172.21.15.43:0/2814544671 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f108c06b4d0 msgr2=0x7f108c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:48.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.802+0000 7f10a64ee640 1 --2- 172.21.15.43:0/2814544671 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f108c06b4d0 0x7f108c06d990 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f1094032000 tx=0x7f109402e000 comp rx=0 tx=0).stop 2024-04-25T09:04:48.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.802+0000 7f10a64ee640 1 -- 172.21.15.43:0/2814544671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10a01f3b40 msgr2=0x7f10a01f3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:48.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.802+0000 7f10a64ee640 1 --2- 172.21.15.43:0/2814544671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10a01f3b40 0x7f10a01f3fc0 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f10a0070d90 tx=0x7f1098010590 comp rx=0 tx=0).stop 2024-04-25T09:04:48.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.802+0000 7f10a64ee640 1 -- 172.21.15.43:0/2814544671 shutdown_connections 2024-04-25T09:04:48.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.802+0000 7f10a64ee640 1 --2- 172.21.15.43:0/2814544671 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f10a01f3b40 0x7f10a01f3fc0 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:48.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.802+0000 7f10a64ee640 1 --2- 172.21.15.43:0/2814544671 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f108c06b4d0 0x7f108c06d990 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:48.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.802+0000 7f10a64ee640 1 --2- 172.21.15.43:0/2814544671 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f10a006f540 0x7f10a006f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:48.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.802+0000 7f10a64ee640 1 --2- 172.21.15.43:0/2814544671 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f10880039e0 0x7f10a0160a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:48.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.802+0000 7f10a64ee640 1 -- 172.21.15.43:0/2814544671 >> 172.21.15.43:0/2814544671 conn(0x7f10a009aa70 msgr2=0x7f10a006e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:48.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.802+0000 7f10a64ee640 1 -- 172.21.15.43:0/2814544671 shutdown_connections 2024-04-25T09:04:48.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.802+0000 7f10a64ee640 1 -- 172.21.15.43:0/2814544671 wait complete. 2024-04-25T09:04:48.812 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T09:04:48.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.932+0000 7eff9bbb0640 1 Processor -- start 2024-04-25T09:04:48.921 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.933+0000 7eff9bbb0640 1 -- start start 2024-04-25T09:04:48.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.933+0000 7eff9bbb0640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7eff940b59e0 0x7eff940d2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:48.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.933+0000 7eff9bbb0640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff940b5e70 con 0x7eff940b5560 2024-04-25T09:04:48.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.933+0000 7eff9bbb0640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff94162610 con 0x7eff9415e090 2024-04-25T09:04:48.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.933+0000 7eff9bbb0640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff940d1da0 con 0x7eff940b59e0 2024-04-25T09:04:48.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.934+0000 7eff9a14d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7eff940b59e0 0x7eff940d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:48.922 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.934+0000 7eff9914b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7eff940b5560 0x7eff940d2230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44676/0 (socket says 172.21.15.43:44676) 2024-04-25T09:04:48.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.934+0000 7eff9914b640 1 -- 172.21.15.43:0/3956086679 learned_addr learned my addr 172.21.15.43:0/3956086679 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:48.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.934+0000 7eff9a14d640 1 -- 172.21.15.43:0/3956086679 >> v1:172.21.15.193:6789/0 conn(0x7eff9415e090 legacy=0x7eff94161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:48.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.934+0000 7eff9a14d640 1 -- 172.21.15.43:0/3956086679 >> v1:172.21.15.43:6789/0 conn(0x7eff940b5560 legacy=0x7eff940d2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:48.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.934+0000 7eff9a14d640 1 -- 172.21.15.43:0/3956086679 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff940d1750 con 0x7eff940b59e0 2024-04-25T09:04:48.923 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.934+0000 7eff9894a640 1 -- 172.21.15.43:0/3956086679 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1517629131 0 0) 0x7eff940b5e70 con 0x7eff940b5560 2024-04-25T09:04:48.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.935+0000 7eff9a14d640 1 --2- 172.21.15.43:0/3956086679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff940b59e0 0x7eff940d2940 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7eff84009d90 tx=0x7eff8402a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=928a3a0b97ab0bae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:48.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.935+0000 7eff9894a640 1 -- 172.21.15.43:0/3956086679 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff84037070 con 0x7eff940b59e0 2024-04-25T09:04:48.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.935+0000 7eff9894a640 1 -- 172.21.15.43:0/3956086679 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff84007740 con 0x7eff940b59e0 2024-04-25T09:04:48.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.935+0000 7eff9894a640 1 -- 172.21.15.43:0/3956086679 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff84007b10 con 0x7eff940b59e0 2024-04-25T09:04:48.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.935+0000 7eff9bbb0640 1 -- 172.21.15.43:0/3956086679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff940b59e0 msgr2=0x7eff940d2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:48.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.935+0000 7eff9bbb0640 1 --2- 172.21.15.43:0/3956086679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff940b59e0 0x7eff940d2940 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7eff84009d90 tx=0x7eff8402a780 comp rx=0 tx=0).stop 2024-04-25T09:04:48.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.935+0000 7eff9bbb0640 1 -- 172.21.15.43:0/3956086679 shutdown_connections 2024-04-25T09:04:48.924 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.935+0000 7eff9bbb0640 1 --2- 172.21.15.43:0/3956086679 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff940b59e0 0x7eff940d2940 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:48.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.935+0000 7eff9bbb0640 1 -- 172.21.15.43:0/3956086679 >> 172.21.15.43:0/3956086679 conn(0x7eff940d0560 msgr2=0x7eff940d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:48.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.935+0000 7eff9bbb0640 1 -- 172.21.15.43:0/3956086679 shutdown_connections 2024-04-25T09:04:48.925 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.935+0000 7eff9bbb0640 1 -- 172.21.15.43:0/3956086679 wait complete. 2024-04-25T09:04:48.928 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.937+0000 7eff9bbb0640 1 Processor -- start 2024-04-25T09:04:48.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.937+0000 7eff9bbb0640 1 -- start start 2024-04-25T09:04:48.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.937+0000 7eff9bbb0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff940b5560 0x7eff9415ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:48.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.937+0000 7eff9bbb0640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff940b59e0 0x7eff940cae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:48.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.937+0000 7eff9bbb0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff9415e090 0x7eff940c8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:48.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.937+0000 7eff9bbb0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff940b3280 con 0x7eff940b5560 2024-04-25T09:04:48.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.937+0000 7eff9bbb0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff941628f0 con 0x7eff9415e090 2024-04-25T09:04:48.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.937+0000 7eff9bbb0640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff941621b0 con 0x7eff940b59e0 2024-04-25T09:04:48.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.937+0000 7eff9994c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff940b5560 0x7eff9415ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:48.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.937+0000 7eff9994c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff940b5560 0x7eff9415ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60204/0 (socket says 172.21.15.43:60204) 2024-04-25T09:04:48.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.937+0000 7eff9994c640 1 -- 172.21.15.43:0/3114834889 learned_addr learned my addr 172.21.15.43:0/3114834889 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:48.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.938+0000 7eff9a14d640 1 --2- 172.21.15.43:0/3114834889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff9415e090 0x7eff940c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:48.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.938+0000 7eff9914b640 1 --2- 172.21.15.43:0/3114834889 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff940b59e0 0x7eff940cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:48.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.938+0000 7eff9994c640 1 -- 172.21.15.43:0/3114834889 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff940b59e0 msgr2=0x7eff940cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:48.929 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.938+0000 7eff9994c640 1 --2- 172.21.15.43:0/3114834889 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff940b59e0 0x7eff940cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:48.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.938+0000 7eff9994c640 1 -- 172.21.15.43:0/3114834889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff9415e090 msgr2=0x7eff940c8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:48.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.938+0000 7eff9994c640 1 --2- 172.21.15.43:0/3114834889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff9415e090 0x7eff940c8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:48.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.938+0000 7eff9994c640 1 -- 172.21.15.43:0/3114834889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff94148d60 con 0x7eff940b5560 2024-04-25T09:04:48.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.938+0000 7eff9994c640 1 --2- 172.21.15.43:0/3114834889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff940b5560 0x7eff9415ebf0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7eff8c00c400 tx=0x7eff8c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:48.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.938+0000 7eff927fc640 1 -- 172.21.15.43:0/3114834889 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c017070 con 0x7eff940b5560 2024-04-25T09:04:48.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.938+0000 7eff927fc640 1 -- 172.21.15.43:0/3114834889 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff8c0052d0 con 0x7eff940b5560 2024-04-25T09:04:48.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.938+0000 7eff9bbb0640 1 -- 172.21.15.43:0/3114834889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff94149040 con 0x7eff940b5560 2024-04-25T09:04:48.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.938+0000 7eff927fc640 1 -- 172.21.15.43:0/3114834889 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c003830 con 0x7eff940b5560 2024-04-25T09:04:48.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.938+0000 7eff9bbb0640 1 -- 172.21.15.43:0/3114834889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff94149600 con 0x7eff940b5560 2024-04-25T09:04:48.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.939+0000 7eff9bbb0640 1 -- 172.21.15.43:0/3114834889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff9406bf00 con 0x7eff940b5560 2024-04-25T09:04:48.930 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.940+0000 7eff927fc640 1 -- 172.21.15.43:0/3114834889 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7eff8c01a070 con 0x7eff940b5560 2024-04-25T09:04:48.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.940+0000 7eff927fc640 1 --2- 172.21.15.43:0/3114834889 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7eff8006b4d0 0x7eff8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:48.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.940+0000 7eff927fc640 1 -- 172.21.15.43:0/3114834889 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 4943+0+0 (secure 0 0 0) 0x7eff8c0891b0 con 0x7eff940b5560 2024-04-25T09:04:48.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.940+0000 7eff9914b640 1 --2- 172.21.15.43:0/3114834889 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7eff8006b4d0 0x7eff8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:48.931 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.941+0000 7eff9914b640 1 --2- 172.21.15.43:0/3114834889 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7eff8006b4d0 0x7eff8006d990 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7eff88007f70 tx=0x7eff88007f00 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:48.932 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:48.944+0000 7eff927fc640 1 -- 172.21.15.43:0/3114834889 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7eff9406bf00 con 0x7eff940b5560 2024-04-25T09:04:49.127 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.138+0000 7eff9bbb0640 1 -- 172.21.15.43:0/3114834889 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7eff94074f60 con 0x7eff940b5560 2024-04-25T09:04:49.129 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.141+0000 7eff927fc640 1 -- 172.21.15.43:0/3114834889 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7eff8c057ce0 con 0x7eff940b5560 2024-04-25T09:04:49.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.144+0000 7eff9bbb0640 1 -- 172.21.15.43:0/3114834889 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7eff8006b4d0 msgr2=0x7eff8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:49.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.144+0000 7eff9bbb0640 1 --2- 172.21.15.43:0/3114834889 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7eff8006b4d0 0x7eff8006d990 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7eff88007f70 tx=0x7eff88007f00 comp rx=0 tx=0).stop 2024-04-25T09:04:49.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.144+0000 7eff9bbb0640 1 -- 172.21.15.43:0/3114834889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff940b5560 msgr2=0x7eff9415ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:49.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.144+0000 7eff9bbb0640 1 --2- 172.21.15.43:0/3114834889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff940b5560 0x7eff9415ebf0 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7eff8c00c400 tx=0x7eff8c00c8c0 comp rx=0 tx=0).stop 2024-04-25T09:04:49.132 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.144+0000 7eff9bbb0640 1 -- 172.21.15.43:0/3114834889 shutdown_connections 2024-04-25T09:04:49.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.144+0000 7eff9bbb0640 1 --2- 172.21.15.43:0/3114834889 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7eff8006b4d0 0x7eff8006d990 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:49.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.144+0000 7eff9bbb0640 1 --2- 172.21.15.43:0/3114834889 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7eff9415e090 0x7eff940c8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:49.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.144+0000 7eff9bbb0640 1 --2- 172.21.15.43:0/3114834889 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7eff940b59e0 0x7eff940cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:49.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.144+0000 7eff9bbb0640 1 --2- 172.21.15.43:0/3114834889 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7eff940b5560 0x7eff9415ebf0 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:49.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.144+0000 7eff9bbb0640 1 -- 172.21.15.43:0/3114834889 >> 172.21.15.43:0/3114834889 conn(0x7eff940d0560 msgr2=0x7eff940c98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:49.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.144+0000 7eff9bbb0640 1 -- 172.21.15.43:0/3114834889 shutdown_connections 2024-04-25T09:04:49.133 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.144+0000 7eff9bbb0640 1 -- 172.21.15.43:0/3114834889 wait complete. 2024-04-25T09:04:49.143 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive' 2024-04-25T09:04:49.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.255+0000 7efe4b66f640 1 Processor -- start 2024-04-25T09:04:49.244 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.255+0000 7efe4b66f640 1 -- start start 2024-04-25T09:04:49.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.256+0000 7efe4b66f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7efe440b0870 0x7efe440ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:49.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.256+0000 7efe4b66f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe440cbee0 con 0x7efe4406c510 2024-04-25T09:04:49.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.256+0000 7efe4b66f640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe440d0650 con 0x7efe4415df70 2024-04-25T09:04:49.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.256+0000 7efe4b66f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe440d0400 con 0x7efe440b0870 2024-04-25T09:04:49.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.256+0000 7efe49c0c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7efe440b0870 0x7efe440ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:49.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.256+0000 7efe4940b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7efe4406c510 0x7efe4406c8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:44690/0 (socket says 172.21.15.43:44690) 2024-04-25T09:04:49.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.256+0000 7efe4940b640 1 -- 172.21.15.43:0/3559274884 learned_addr learned my addr 172.21.15.43:0/3559274884 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:49.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.257+0000 7efe49c0c640 1 -- 172.21.15.43:0/3559274884 >> v1:172.21.15.193:6789/0 conn(0x7efe4415df70 legacy=0x7efe440cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:49.245 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.257+0000 7efe49c0c640 1 -- 172.21.15.43:0/3559274884 >> v1:172.21.15.43:6789/0 conn(0x7efe4406c510 legacy=0x7efe4406c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:49.246 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.257+0000 7efe49c0c640 1 -- 172.21.15.43:0/3559274884 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe440d0930 con 0x7efe440b0870 2024-04-25T09:04:49.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.257+0000 7efe49c0c640 1 --2- 172.21.15.43:0/3559274884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe440b0870 0x7efe440ccd80 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7efe34009dc0 tx=0x7efe3402a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4133f9cc71dae272 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:49.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.258+0000 7efe43fff640 1 -- 172.21.15.43:0/3559274884 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe34007740 con 0x7efe440b0870 2024-04-25T09:04:49.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.258+0000 7efe43fff640 1 -- 172.21.15.43:0/3559274884 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe340078e0 con 0x7efe440b0870 2024-04-25T09:04:49.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.258+0000 7efe43fff640 1 -- 172.21.15.43:0/3559274884 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe34007cb0 con 0x7efe440b0870 2024-04-25T09:04:49.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.258+0000 7efe4b66f640 1 -- 172.21.15.43:0/3559274884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe440b0870 msgr2=0x7efe440ccd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:49.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.258+0000 7efe4b66f640 1 --2- 172.21.15.43:0/3559274884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe440b0870 0x7efe440ccd80 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7efe34009dc0 tx=0x7efe3402a640 comp rx=0 tx=0).stop 2024-04-25T09:04:49.247 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.258+0000 7efe4b66f640 1 -- 172.21.15.43:0/3559274884 shutdown_connections 2024-04-25T09:04:49.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.258+0000 7efe4b66f640 1 --2- 172.21.15.43:0/3559274884 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe440b0870 0x7efe440ccd80 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:49.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.258+0000 7efe4b66f640 1 -- 172.21.15.43:0/3559274884 >> 172.21.15.43:0/3559274884 conn(0x7efe440d17f0 msgr2=0x7efe440d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:49.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.258+0000 7efe4b66f640 1 -- 172.21.15.43:0/3559274884 shutdown_connections 2024-04-25T09:04:49.248 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.258+0000 7efe4b66f640 1 -- 172.21.15.43:0/3559274884 wait complete. 2024-04-25T09:04:49.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.259+0000 7efe4b66f640 1 Processor -- start 2024-04-25T09:04:49.250 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.259+0000 7efe4b66f640 1 -- start start 2024-04-25T09:04:49.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.260+0000 7efe4b66f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efe4406c510 0x7efe441f6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:49.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.260+0000 7efe4b66f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe440b0870 0x7efe440caa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:49.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.260+0000 7efe4b66f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe4415df70 0x7efe440cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:49.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.260+0000 7efe4b66f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4406bd10 con 0x7efe440b0870 2024-04-25T09:04:49.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.260+0000 7efe4b66f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe440d0c10 con 0x7efe4415df70 2024-04-25T09:04:49.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.260+0000 7efe4b66f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe4406be90 con 0x7efe4406c510 2024-04-25T09:04:49.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.260+0000 7efe48c0a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe440b0870 0x7efe440caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:49.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.260+0000 7efe48c0a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe440b0870 0x7efe440caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:60228/0 (socket says 172.21.15.43:60228) 2024-04-25T09:04:49.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.260+0000 7efe48c0a640 1 -- 172.21.15.43:0/368142441 learned_addr learned my addr 172.21.15.43:0/368142441 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:49.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.261+0000 7efe49c0c640 1 --2- 172.21.15.43:0/368142441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe4415df70 0x7efe440cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:49.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.261+0000 7efe4940b640 1 --2- 172.21.15.43:0/368142441 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efe4406c510 0x7efe441f6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:49.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.261+0000 7efe48c0a640 1 -- 172.21.15.43:0/368142441 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efe4406c510 msgr2=0x7efe441f6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:49.251 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.261+0000 7efe48c0a640 1 --2- 172.21.15.43:0/368142441 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efe4406c510 0x7efe441f6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:49.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.261+0000 7efe48c0a640 1 -- 172.21.15.43:0/368142441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe4415df70 msgr2=0x7efe440cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:49.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.261+0000 7efe48c0a640 1 --2- 172.21.15.43:0/368142441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe4415df70 0x7efe440cafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:49.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.261+0000 7efe48c0a640 1 -- 172.21.15.43:0/368142441 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe441f41f0 con 0x7efe440b0870 2024-04-25T09:04:49.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.261+0000 7efe49c0c640 1 --2- 172.21.15.43:0/368142441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe4415df70 0x7efe440cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:49.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.261+0000 7efe4940b640 1 --2- 172.21.15.43:0/368142441 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efe4406c510 0x7efe441f6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:49.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.261+0000 7efe48c0a640 1 --2- 172.21.15.43:0/368142441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe440b0870 0x7efe440caa90 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7efe3800c430 tx=0x7efe3800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:49.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.261+0000 7efe41ffb640 1 -- 172.21.15.43:0/368142441 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe38011790 con 0x7efe440b0870 2024-04-25T09:04:49.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.261+0000 7efe41ffb640 1 -- 172.21.15.43:0/368142441 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe38017070 con 0x7efe440b0870 2024-04-25T09:04:49.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.261+0000 7efe4b66f640 1 -- 172.21.15.43:0/368142441 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe441f44b0 con 0x7efe440b0870 2024-04-25T09:04:49.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.261+0000 7efe41ffb640 1 -- 172.21.15.43:0/368142441 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe38011b70 con 0x7efe440b0870 2024-04-25T09:04:49.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.261+0000 7efe4b66f640 1 -- 172.21.15.43:0/368142441 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe441f4a70 con 0x7efe440b0870 2024-04-25T09:04:49.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.262+0000 7efe4b66f640 1 -- 172.21.15.43:0/368142441 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe44074ed0 con 0x7efe440b0870 2024-04-25T09:04:49.252 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.262+0000 7efe41ffb640 1 -- 172.21.15.43:0/368142441 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efe38011d10 con 0x7efe440b0870 2024-04-25T09:04:49.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.263+0000 7efe41ffb640 1 --2- 172.21.15.43:0/368142441 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7efe3006b4d0 0x7efe3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:49.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.263+0000 7efe41ffb640 1 -- 172.21.15.43:0/368142441 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 4943+0+0 (secure 0 0 0) 0x7efe38089d70 con 0x7efe440b0870 2024-04-25T09:04:49.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.263+0000 7efe4940b640 1 --2- 172.21.15.43:0/368142441 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7efe3006b4d0 0x7efe3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:49.253 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.264+0000 7efe4940b640 1 --2- 172.21.15.43:0/368142441 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7efe3006b4d0 0x7efe3006d990 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7efe3c000f60 tx=0x7efe3c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:49.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.267+0000 7efe41ffb640 1 -- 172.21.15.43:0/368142441 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efe38058ef0 con 0x7efe440b0870 2024-04-25T09:04:49.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.516+0000 7efe4b66f640 1 -- 172.21.15.43:0/368142441 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]} v 0) v1 -- 0x7efe4415ca60 con 0x7efe440b0870 2024-04-25T09:04:49.788 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.800+0000 7efe41ffb640 1 -- 172.21.15.43:0/368142441 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]}]=0 v513) v1 ==== 159+0+0 (secure 0 0 0) 0x7efe380587f0 con 0x7efe440b0870 2024-04-25T09:04:49.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.803+0000 7efe4b66f640 1 -- 172.21.15.43:0/368142441 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7efe3006b4d0 msgr2=0x7efe3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:49.791 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.803+0000 7efe4b66f640 1 --2- 172.21.15.43:0/368142441 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7efe3006b4d0 0x7efe3006d990 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7efe3c000f60 tx=0x7efe3c009040 comp rx=0 tx=0).stop 2024-04-25T09:04:49.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.803+0000 7efe4b66f640 1 -- 172.21.15.43:0/368142441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe440b0870 msgr2=0x7efe440caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:49.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.803+0000 7efe4b66f640 1 --2- 172.21.15.43:0/368142441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe440b0870 0x7efe440caa90 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7efe3800c430 tx=0x7efe3800c8f0 comp rx=0 tx=0).stop 2024-04-25T09:04:49.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.803+0000 7efe4b66f640 1 -- 172.21.15.43:0/368142441 shutdown_connections 2024-04-25T09:04:49.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.803+0000 7efe4b66f640 1 --2- 172.21.15.43:0/368142441 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7efe3006b4d0 0x7efe3006d990 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:49.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.803+0000 7efe4b66f640 1 --2- 172.21.15.43:0/368142441 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7efe4415df70 0x7efe440cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:49.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.803+0000 7efe4b66f640 1 --2- 172.21.15.43:0/368142441 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7efe440b0870 0x7efe440caa90 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:49.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.803+0000 7efe4b66f640 1 --2- 172.21.15.43:0/368142441 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7efe4406c510 0x7efe441f6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:49.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.803+0000 7efe4b66f640 1 -- 172.21.15.43:0/368142441 >> 172.21.15.43:0/368142441 conn(0x7efe440d17f0 msgr2=0x7efe440d1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:49.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.803+0000 7efe4b66f640 1 -- 172.21.15.43:0/368142441 shutdown_connections 2024-04-25T09:04:49.792 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:49.803+0000 7efe4b66f640 1 -- 172.21.15.43:0/368142441 wait complete. 2024-04-25T09:04:49.804 INFO:tasks.cephfs_test_runner:test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ERROR 2024-04-25T09:04:49.804 INFO:tasks.cephfs_test_runner: 2024-04-25T09:04:49.804 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-25T09:04:49.805 INFO:tasks.cephfs_test_runner:ERROR: test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T09:04:49.805 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T09:04:49.805 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-25T09:04:49.805 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa/tasks/cephfs/test_scrub_checks.py", line 190, in test_scrub_when_mds_is_inactive 2024-04-25T09:04:49.805 INFO:tasks.cephfs_test_runner: sr_mds_id = self.get_daemon_names('up:standby-replay', status=status)[0] 2024-04-25T09:04:49.805 INFO:tasks.cephfs_test_runner:AttributeError: 'TestScrubControls' object has no attribute 'get_daemon_names' 2024-04-25T09:04:49.805 INFO:tasks.cephfs_test_runner: 2024-04-25T09:04:49.805 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T09:04:49.805 INFO:tasks.cephfs_test_runner:Ran 9 tests in 611.960s 2024-04-25T09:04:49.805 INFO:tasks.cephfs_test_runner: 2024-04-25T09:04:49.805 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2024-04-25T09:04:49.805 INFO:tasks.cephfs_test_runner: 2024-04-25T09:04:49.805 INFO:tasks.cephfs_test_runner:====================================================================== 2024-04-25T09:04:49.805 INFO:tasks.cephfs_test_runner:ERROR: test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T09:04:49.805 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T09:04:49.806 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2024-04-25T09:04:49.806 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa/tasks/cephfs/test_scrub_checks.py", line 190, in test_scrub_when_mds_is_inactive 2024-04-25T09:04:49.806 INFO:tasks.cephfs_test_runner: sr_mds_id = self.get_daemon_names('up:standby-replay', status=status)[0] 2024-04-25T09:04:49.806 INFO:tasks.cephfs_test_runner:AttributeError: 'TestScrubControls' object has no attribute 'get_daemon_names' 2024-04-25T09:04:49.806 INFO:tasks.cephfs_test_runner: 2024-04-25T09:04:49.806 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T09:04:50.020 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=da31073afa5e462f9cff1c4d746c5807 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_neesingh-rh_ceph_5787d42d10024ba410e822a8aba474331cb9af02/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T09:04:50.022 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-25T09:04:50.030 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-25T09:04:50.038 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-25T09:04:50.039 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T09:04:50.039 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:04:50.059 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2374 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:04:50.059 DEBUG:teuthology.orchestra.run.smithi043:> set -ex 2024-04-25T09:04:50.059 DEBUG:teuthology.orchestra.run.smithi043:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:04:50.116 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056882,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=2374 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249464k,nr_inodes=812366,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498932k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T09:04:50.116 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T09:04:50.116 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:04:50.135 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19508 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:04:50.135 DEBUG:teuthology.orchestra.run.smithi193:> set -ex 2024-04-25T09:04:50.135 DEBUG:teuthology.orchestra.run.smithi193:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T09:04:50.192 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056111,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19508 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248848k,nr_inodes=812212,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497700k,nr_inodes=819200,mode=755,inode64 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T09:04:50.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:50.192 DEBUG:teuthology.orchestra.run.smithi043:> ip netns list 2024-04-25T09:04:50.208 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T09:04:50.208 INFO:teuthology.orchestra.run.smithi043.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T09:04:50.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:50.209 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T09:04:50.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:50.278 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T09:04:50.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:50.345 DEBUG:teuthology.orchestra.run.smithi043:> sudo ip link delete ceph-brx 2024-04-25T09:04:50.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:50.441 DEBUG:teuthology.orchestra.run.smithi193:> ip netns list 2024-04-25T09:04:50.459 INFO:teuthology.orchestra.run.smithi193.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T09:04:50.459 INFO:teuthology.orchestra.run.smithi193.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T09:04:50.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:50.460 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T09:04:50.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:50.528 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T09:04:50.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T09:04:50.595 DEBUG:teuthology.orchestra.run.smithi193:> sudo ip link delete ceph-brx 2024-04-25T09:04:50.686 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T09:04:50.695 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T09:04:50.696 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:04:50.799 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.811+0000 7f5eb15b0640 1 Processor -- start 2024-04-25T09:04:50.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.811+0000 7f5eb15b0640 1 -- start start 2024-04-25T09:04:50.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.812+0000 7f5eb15b0640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f5eac0c8d00 0x7f5eac162100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:50.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.812+0000 7f5eb15b0640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5eac0ce660 0x7f5eac0ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:50.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.812+0000 7f5eb15b0640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eac0d24a0 con 0x7f5eac15df50 2024-04-25T09:04:50.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.812+0000 7f5eb15b0640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eac15e330 con 0x7f5eac0c8d00 2024-04-25T09:04:50.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.812+0000 7f5eb15b0640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac0d2740 con 0x7f5eac0ce660 2024-04-25T09:04:50.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.812+0000 7f5eab7fe640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5eac0ce660 0x7f5eac0ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:50.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.812+0000 7f5eaa7fc640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f5eac0c8d00 0x7f5eac162100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:50.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.812+0000 7f5eab7fe640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5eac0ce660 0x7f5eac0ceab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:33528/0 (socket says 172.21.15.43:33528) 2024-04-25T09:04:50.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.812+0000 7f5eab7fe640 1 -- 172.21.15.43:0/4288496093 learned_addr learned my addr 172.21.15.43:0/4288496093 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:50.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.812+0000 7f5eab7fe640 1 -- 172.21.15.43:0/4288496093 >> v2:172.21.15.193:3300/0 conn(0x7f5eac0c8d00 msgr2=0x7f5eac162100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:50.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.812+0000 7f5eab7fe640 1 --2- 172.21.15.43:0/4288496093 >> v2:172.21.15.193:3300/0 conn(0x7f5eac0c8d00 0x7f5eac162100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:50.802 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.812+0000 7f5eab7fe640 1 -- 172.21.15.43:0/4288496093 >> v1:172.21.15.193:6789/0 conn(0x7f5eac15df50 legacy=0x7f5eac1619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:50.803 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.812+0000 7f5eab7fe640 1 -- 172.21.15.43:0/4288496093 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eac0d2a20 con 0x7f5eac0ce660 2024-04-25T09:04:50.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.813+0000 7f5eab7fe640 1 --2- 172.21.15.43:0/4288496093 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5eac0ce660 0x7f5eac0ceab0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4009e50 tx=0x7f5ea402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1322a31841add78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:50.806 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.813+0000 7f5ea9ffb640 1 -- 172.21.15.43:0/4288496093 <== mon.2 v2:172.21.15.193:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea4036070 con 0x7f5eac0ce660 2024-04-25T09:04:50.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.813+0000 7f5ea9ffb640 1 -- 172.21.15.43:0/4288496093 <== mon.2 v2:172.21.15.193:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea4031600 con 0x7f5eac0ce660 2024-04-25T09:04:50.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.813+0000 7f5ea9ffb640 1 -- 172.21.15.43:0/4288496093 <== mon.2 v2:172.21.15.193:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea40319d0 con 0x7f5eac0ce660 2024-04-25T09:04:50.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.814+0000 7f5eb15b0640 1 -- 172.21.15.43:0/4288496093 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5eac0ce660 msgr2=0x7f5eac0ceab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:50.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.814+0000 7f5eb15b0640 1 --2- 172.21.15.43:0/4288496093 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5eac0ce660 0x7f5eac0ceab0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5ea4009e50 tx=0x7f5ea402a710 comp rx=0 tx=0).stop 2024-04-25T09:04:50.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.814+0000 7f5eb15b0640 1 -- 172.21.15.43:0/4288496093 shutdown_connections 2024-04-25T09:04:50.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.814+0000 7f5eb15b0640 1 --2- 172.21.15.43:0/4288496093 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5eac0ce660 0x7f5eac0ceab0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:50.807 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.814+0000 7f5eb15b0640 1 --2- 172.21.15.43:0/4288496093 >> v2:172.21.15.193:3300/0 conn(0x7f5eac0c8d00 0x7f5eac162100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:50.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.814+0000 7f5eb15b0640 1 -- 172.21.15.43:0/4288496093 >> 172.21.15.43:0/4288496093 conn(0x7f5eac0d3a30 msgr2=0x7f5eac0d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:50.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.814+0000 7f5eb15b0640 1 -- 172.21.15.43:0/4288496093 shutdown_connections 2024-04-25T09:04:50.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.814+0000 7f5eb15b0640 1 -- 172.21.15.43:0/4288496093 wait complete. 2024-04-25T09:04:50.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.815+0000 7f5eb15b0640 1 Processor -- start 2024-04-25T09:04:50.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.815+0000 7f5eb15b0640 1 -- start start 2024-04-25T09:04:50.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.816+0000 7f5eb15b0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5eac0c8d00 0x7f5eac14d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:50.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.816+0000 7f5eb15b0640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5eac0ce660 0x7f5eac15e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:50.808 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.816+0000 7f5eb15b0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5eac15df50 0x7f5eac15bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:50.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.816+0000 7f5eb15b0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac0d2190 con 0x7f5eac0c8d00 2024-04-25T09:04:50.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.816+0000 7f5eb15b0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac0c77d0 con 0x7f5eac15df50 2024-04-25T09:04:50.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.816+0000 7f5eb15b0640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eac162640 con 0x7f5eac0ce660 2024-04-25T09:04:50.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.816+0000 7f5eaaffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5eac0c8d00 0x7f5eac14d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:50.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.816+0000 7f5eaaffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5eac0c8d00 0x7f5eac14d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43416/0 (socket says 172.21.15.43:43416) 2024-04-25T09:04:50.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.816+0000 7f5eaaffd640 1 -- 172.21.15.43:0/82814709 learned_addr learned my addr 172.21.15.43:0/82814709 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:50.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.816+0000 7f5eaa7fc640 1 --2- 172.21.15.43:0/82814709 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5eac0ce660 0x7f5eac15e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:50.809 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.816+0000 7f5eab7fe640 1 --2- 172.21.15.43:0/82814709 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5eac15df50 0x7f5eac15bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:50.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.816+0000 7f5eaaffd640 1 -- 172.21.15.43:0/82814709 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5eac0ce660 msgr2=0x7f5eac15e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:50.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.816+0000 7f5eaaffd640 1 --2- 172.21.15.43:0/82814709 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5eac0ce660 0x7f5eac15e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:50.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.816+0000 7f5eaaffd640 1 -- 172.21.15.43:0/82814709 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5eac15df50 msgr2=0x7f5eac15bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:50.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.816+0000 7f5eaaffd640 1 --2- 172.21.15.43:0/82814709 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5eac15df50 0x7f5eac15bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:50.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.816+0000 7f5eaaffd640 1 -- 172.21.15.43:0/82814709 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eac15b720 con 0x7f5eac0c8d00 2024-04-25T09:04:50.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.816+0000 7f5eab7fe640 1 --2- 172.21.15.43:0/82814709 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5eac15df50 0x7f5eac15bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:50.810 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.817+0000 7f5eaaffd640 1 --2- 172.21.15.43:0/82814709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5eac0c8d00 0x7f5eac14d540 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c00c3c0 tx=0x7f5e9c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:50.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.817+0000 7f5e937fe640 1 -- 172.21.15.43:0/82814709 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c0052d0 con 0x7f5eac0c8d00 2024-04-25T09:04:50.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.817+0000 7f5e937fe640 1 -- 172.21.15.43:0/82814709 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e9c005490 con 0x7f5eac0c8d00 2024-04-25T09:04:50.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.817+0000 7f5eb15b0640 1 -- 172.21.15.43:0/82814709 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eac0cc440 con 0x7f5eac0c8d00 2024-04-25T09:04:50.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.817+0000 7f5e937fe640 1 -- 172.21.15.43:0/82814709 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e9c007260 con 0x7f5eac0c8d00 2024-04-25T09:04:50.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.817+0000 7f5eb15b0640 1 -- 172.21.15.43:0/82814709 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eac0cca00 con 0x7f5eac0c8d00 2024-04-25T09:04:50.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.818+0000 7f5eb15b0640 1 -- 172.21.15.43:0/82814709 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eac0680f0 con 0x7f5eac0c8d00 2024-04-25T09:04:50.811 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.818+0000 7f5e937fe640 1 -- 172.21.15.43:0/82814709 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e9c007400 con 0x7f5eac0c8d00 2024-04-25T09:04:50.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.819+0000 7f5e937fe640 1 --2- 172.21.15.43:0/82814709 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5e9806b4d0 0x7f5e9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:50.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.819+0000 7f5e937fe640 1 -- 172.21.15.43:0/82814709 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5e9c0892f0 con 0x7f5eac0c8d00 2024-04-25T09:04:50.812 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.819+0000 7f5eaa7fc640 1 --2- 172.21.15.43:0/82814709 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5e9806b4d0 0x7f5e9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:50.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.819+0000 7f5eaa7fc640 1 --2- 172.21.15.43:0/82814709 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5e9806b4d0 0x7f5e9806d990 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f5eac160100 tx=0x7f5ea0008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:50.813 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.823+0000 7f5e937fe640 1 -- 172.21.15.43:0/82814709 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5e9c057e20 con 0x7f5eac0c8d00 2024-04-25T09:04:50.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.998+0000 7f5eb15b0640 1 -- 172.21.15.43:0/82814709 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5eac0ccc00 con 0x7f5e9806b4d0 2024-04-25T09:04:50.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:50.999+0000 7f5e937fe640 1 -- 172.21.15.43:0/82814709 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13526 (secure 0 0 0) 0x7f5eac0ccc00 con 0x7f5e9806b4d0 2024-04-25T09:04:50.988 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:04:50.988 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-25T09:04:50.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.001+0000 7f5eb15b0640 1 -- 172.21.15.43:0/82814709 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5e9806b4d0 msgr2=0x7f5e9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:50.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.001+0000 7f5eb15b0640 1 --2- 172.21.15.43:0/82814709 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5e9806b4d0 0x7f5e9806d990 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f5eac160100 tx=0x7f5ea0008040 comp rx=0 tx=0).stop 2024-04-25T09:04:50.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.002+0000 7f5eb15b0640 1 -- 172.21.15.43:0/82814709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5eac0c8d00 msgr2=0x7f5eac14d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:50.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.002+0000 7f5eb15b0640 1 --2- 172.21.15.43:0/82814709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5eac0c8d00 0x7f5eac14d540 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f5e9c00c3c0 tx=0x7f5e9c00c880 comp rx=0 tx=0).stop 2024-04-25T09:04:50.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.002+0000 7f5eb15b0640 1 -- 172.21.15.43:0/82814709 shutdown_connections 2024-04-25T09:04:50.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.002+0000 7f5eb15b0640 1 --2- 172.21.15.43:0/82814709 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f5e9806b4d0 0x7f5e9806d990 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:50.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.002+0000 7f5eb15b0640 1 --2- 172.21.15.43:0/82814709 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f5eac15df50 0x7f5eac15bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:50.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.002+0000 7f5eb15b0640 1 --2- 172.21.15.43:0/82814709 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f5eac0ce660 0x7f5eac15e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:50.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.002+0000 7f5eb15b0640 1 --2- 172.21.15.43:0/82814709 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f5eac0c8d00 0x7f5eac14d540 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:50.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.002+0000 7f5eb15b0640 1 -- 172.21.15.43:0/82814709 >> 172.21.15.43:0/82814709 conn(0x7f5eac0d3a30 msgr2=0x7f5eac0d0360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:50.991 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.002+0000 7f5eb15b0640 1 -- 172.21.15.43:0/82814709 shutdown_connections 2024-04-25T09:04:50.992 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.002+0000 7f5eb15b0640 1 -- 172.21.15.43:0/82814709 wait complete. 2024-04-25T09:04:51.001 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":461,"stamp":"2024-04-25T09:04:49.843897+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2604328,"kb_used_data":2116096,"kb_used_omap":12,"kb_used_meta":488115,"kb_avail":752370392,"statfs":{"total":773094113280,"available":770427281408,"internally_reserved":0,"allocated":2166882304,"data_stored":2158930549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":499830321},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1077828404,"num_objects":-1046,"num_object_clones":0,"num_object_copies":-2092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1046,"num_whiteouts":0,"num_read":-19,"num_read_kb":-4003,"num_write":-1127,"num_write_kb":-1052713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001408"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":396,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:04:47.798685+0000","last_change":"2024-04-25T08:54:00.818044+0000","last_active":"2024-04-25T09:04:47.798685+0000","last_peered":"2024-04-25T09:04:47.798685+0000","last_clean":"2024-04-25T09:04:47.798685+0000","last_became_active":"2024-04-25T08:54:00.817098+0000","last_became_peered":"2024-04-25T08:54:00.817098+0000","last_unstale":"2024-04-25T09:04:47.798685+0000","last_undegraded":"2024-04-25T09:04:47.798685+0000","last_fullsized":"2024-04-25T09:04:47.798685+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_clean_scrub_stamp":"2024-04-25T08:53:59.739677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:48.129513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574980,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":353552,"kb_used_data":308484,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94018288,"statfs":{"total":96636764160,"available":96274726912,"internally_reserved":0,"allocated":315887616,"data_stored":314889518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574981,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":327048,"kb_used_data":259772,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94044792,"statfs":{"total":96636764160,"available":96301867008,"internally_reserved":0,"allocated":266006528,"data_stored":265010221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574983,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307024,"kb_used_data":253632,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94064816,"statfs":{"total":96636764160,"available":96322371584,"internally_reserved":0,"allocated":259719168,"data_stored":258724622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574981,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411004,"kb_used_data":343728,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":93960836,"statfs":{"total":96636764160,"available":96215896064,"internally_reserved":0,"allocated":351977472,"data_stored":350993307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574981,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":291340,"kb_used_data":215744,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94080500,"statfs":{"total":96636764160,"available":96338432000,"internally_reserved":0,"allocated":220921856,"data_stored":219927310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574981,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300816,"kb_used_data":251584,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94071024,"statfs":{"total":96636764160,"available":96328728576,"internally_reserved":0,"allocated":257622016,"data_stored":256627470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574981,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":344000,"kb_used_data":262836,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94027840,"statfs":{"total":96636764160,"available":96284508160,"internally_reserved":0,"allocated":269144064,"data_stored":268156385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574982,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":269544,"kb_used_data":220316,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94102296,"statfs":{"total":96636764160,"available":96360751104,"internally_reserved":0,"allocated":225603584,"data_stored":224601716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:04:51.002 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:04:51.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.123+0000 7f6ed803a640 1 Processor -- start 2024-04-25T09:04:51.112 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.124+0000 7f6ed803a640 1 -- start start 2024-04-25T09:04:51.113 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.124+0000 7f6ed803a640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6ed015a250 0x7f6ed0070d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:51.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.124+0000 7f6ed803a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ed00715a0 0x7f6ed0071a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:51.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.124+0000 7f6ed803a640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ed006dd30 con 0x7f6ed015dc50 2024-04-25T09:04:51.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.124+0000 7f6ed803a640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ed0071f40 con 0x7f6ed015a250 2024-04-25T09:04:51.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.124+0000 7f6ed803a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed006e010 con 0x7f6ed00715a0 2024-04-25T09:04:51.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.125+0000 7f6ed65d7640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f6ed015dc50 0x7f6ed015e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56524/0 (socket says 172.21.15.43:56524) 2024-04-25T09:04:51.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.125+0000 7f6ed65d7640 1 -- 172.21.15.43:0/3848336467 learned_addr learned my addr 172.21.15.43:0/3848336467 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:51.114 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.125+0000 7f6ed5dd6640 1 --2- 172.21.15.43:0/3848336467 >> v2:172.21.15.193:3300/0 conn(0x7f6ed015a250 0x7f6ed0070d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:51.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.125+0000 7f6ed55d5640 1 --2- 172.21.15.43:0/3848336467 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ed00715a0 0x7f6ed0071a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:51.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.125+0000 7f6ed4dd4640 1 -- 172.21.15.43:0/3848336467 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 831127034 0 0) 0x7f6ed006dd30 con 0x7f6ed015dc50 2024-04-25T09:04:51.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.125+0000 7f6ed4dd4640 1 -- 172.21.15.43:0/3848336467 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ebc003520 con 0x7f6ed015dc50 2024-04-25T09:04:51.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.125+0000 7f6ed4dd4640 1 -- 172.21.15.43:0/3848336467 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3768894953 0 0) 0x7f6ebc003520 con 0x7f6ed015dc50 2024-04-25T09:04:51.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.125+0000 7f6ed5dd6640 1 -- 172.21.15.43:0/3848336467 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ed00715a0 msgr2=0x7f6ed0071a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:51.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.125+0000 7f6ed5dd6640 1 --2- 172.21.15.43:0/3848336467 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ed00715a0 0x7f6ed0071a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:51.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.125+0000 7f6ed5dd6640 1 -- 172.21.15.43:0/3848336467 >> v1:172.21.15.43:6789/0 conn(0x7f6ed015dc50 legacy=0x7f6ed015e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:51.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.125+0000 7f6ed5dd6640 1 -- 172.21.15.43:0/3848336467 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed00b2900 con 0x7f6ed015a250 2024-04-25T09:04:51.115 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.125+0000 7f6ed55d5640 1 --2- 172.21.15.43:0/3848336467 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ed00715a0 0x7f6ed0071a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:04:51.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.126+0000 7f6ed5dd6640 1 --2- 172.21.15.43:0/3848336467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ed015a250 0x7f6ed0070d60 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc00a940 tx=0x7f6ecc02b4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d3c110e30183cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:51.118 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.126+0000 7f6ed4dd4640 1 -- 172.21.15.43:0/3848336467 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec80019f0 con 0x7f6ed015a250 2024-04-25T09:04:51.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.126+0000 7f6ed4dd4640 1 -- 172.21.15.43:0/3848336467 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ecc030510 con 0x7f6ed015a250 2024-04-25T09:04:51.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.126+0000 7f6ed4dd4640 1 -- 172.21.15.43:0/3848336467 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc00b040 con 0x7f6ed015a250 2024-04-25T09:04:51.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.127+0000 7f6ed803a640 1 -- 172.21.15.43:0/3848336467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ed015a250 msgr2=0x7f6ed0070d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:51.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.127+0000 7f6ed803a640 1 --2- 172.21.15.43:0/3848336467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ed015a250 0x7f6ed0070d60 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc00a940 tx=0x7f6ecc02b4d0 comp rx=0 tx=0).stop 2024-04-25T09:04:51.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.127+0000 7f6ed803a640 1 -- 172.21.15.43:0/3848336467 shutdown_connections 2024-04-25T09:04:51.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.127+0000 7f6ed803a640 1 --2- 172.21.15.43:0/3848336467 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ed00715a0 0x7f6ed0071a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:51.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.127+0000 7f6ed803a640 1 --2- 172.21.15.43:0/3848336467 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ed015a250 0x7f6ed0070d60 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:51.119 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.127+0000 7f6ed803a640 1 -- 172.21.15.43:0/3848336467 >> 172.21.15.43:0/3848336467 conn(0x7f6ed00b1480 msgr2=0x7f6ed00b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:51.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.127+0000 7f6ed803a640 1 -- 172.21.15.43:0/3848336467 shutdown_connections 2024-04-25T09:04:51.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.127+0000 7f6ed803a640 1 -- 172.21.15.43:0/3848336467 wait complete. 2024-04-25T09:04:51.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.128+0000 7f6ed803a640 1 Processor -- start 2024-04-25T09:04:51.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.128+0000 7f6ed803a640 1 -- start start 2024-04-25T09:04:51.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.129+0000 7f6ed803a640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ed00715a0 0x7f6ed0151a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:51.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.129+0000 7f6ed803a640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ed015a250 0x7f6ed0155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:51.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.129+0000 7f6ed803a640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ed015dc50 0x7f6ed0143f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:51.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.129+0000 7f6ed803a640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed00aeea0 con 0x7f6ed015dc50 2024-04-25T09:04:51.120 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.129+0000 7f6ed803a640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed00712a0 con 0x7f6ed00715a0 2024-04-25T09:04:51.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.129+0000 7f6ed803a640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed00af020 con 0x7f6ed015a250 2024-04-25T09:04:51.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.129+0000 7f6ed65d7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ed015dc50 0x7f6ed0143f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:51.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.129+0000 7f6ed65d7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ed015dc50 0x7f6ed0143f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43432/0 (socket says 172.21.15.43:43432) 2024-04-25T09:04:51.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.129+0000 7f6ed65d7640 1 -- 172.21.15.43:0/37770542 learned_addr learned my addr 172.21.15.43:0/37770542 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:51.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.129+0000 7f6ed5dd6640 1 --2- 172.21.15.43:0/37770542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ed00715a0 0x7f6ed0151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:51.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.129+0000 7f6ed55d5640 1 --2- 172.21.15.43:0/37770542 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ed015a250 0x7f6ed0155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:51.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.129+0000 7f6ed65d7640 1 -- 172.21.15.43:0/37770542 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ed015a250 msgr2=0x7f6ed0155ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:51.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.129+0000 7f6ed65d7640 1 --2- 172.21.15.43:0/37770542 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ed015a250 0x7f6ed0155ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:51.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.129+0000 7f6ed65d7640 1 -- 172.21.15.43:0/37770542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ed00715a0 msgr2=0x7f6ed0151a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:51.121 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.129+0000 7f6ed65d7640 1 --2- 172.21.15.43:0/37770542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ed00715a0 0x7f6ed0151a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:51.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.129+0000 7f6ed65d7640 1 -- 172.21.15.43:0/37770542 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed0156310 con 0x7f6ed015dc50 2024-04-25T09:04:51.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.129+0000 7f6ed65d7640 1 --2- 172.21.15.43:0/37770542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ed015dc50 0x7f6ed0143f90 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f6ec800cc40 tx=0x7f6ec800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:51.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.129+0000 7f6ec67fc640 1 -- 172.21.15.43:0/37770542 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8017070 con 0x7f6ed015dc50 2024-04-25T09:04:51.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.130+0000 7f6ec67fc640 1 -- 172.21.15.43:0/37770542 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ec800ce50 con 0x7f6ed015dc50 2024-04-25T09:04:51.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.130+0000 7f6ec67fc640 1 -- 172.21.15.43:0/37770542 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ec8012620 con 0x7f6ed015dc50 2024-04-25T09:04:51.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.130+0000 7f6ed803a640 1 -- 172.21.15.43:0/37770542 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ed01565f0 con 0x7f6ed015dc50 2024-04-25T09:04:51.122 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.130+0000 7f6ed803a640 1 -- 172.21.15.43:0/37770542 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ed0156bb0 con 0x7f6ed015dc50 2024-04-25T09:04:51.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.131+0000 7f6ed803a640 1 -- 172.21.15.43:0/37770542 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ed0067d30 con 0x7f6ed015dc50 2024-04-25T09:04:51.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.131+0000 7f6ec67fc640 1 -- 172.21.15.43:0/37770542 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6ec801a070 con 0x7f6ed015dc50 2024-04-25T09:04:51.123 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.131+0000 7f6ec67fc640 1 --2- 172.21.15.43:0/37770542 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6eb406b500 0x7f6eb406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:51.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.131+0000 7f6ec67fc640 1 -- 172.21.15.43:0/37770542 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6ec8089e10 con 0x7f6ed015dc50 2024-04-25T09:04:51.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.132+0000 7f6ed5dd6640 1 --2- 172.21.15.43:0/37770542 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6eb406b500 0x7f6eb406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:51.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.132+0000 7f6ed5dd6640 1 --2- 172.21.15.43:0/37770542 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6eb406b500 0x7f6eb406d9c0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc035000 tx=0x7f6ecc00c040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:51.124 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.136+0000 7f6ec67fc640 1 -- 172.21.15.43:0/37770542 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6ed0067d30 con 0x7f6ed015dc50 2024-04-25T09:04:51.303 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.314+0000 7f6ed803a640 1 -- 172.21.15.43:0/37770542 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6ed00ae8e0 con 0x7f6eb406b500 2024-04-25T09:04:51.304 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.315+0000 7f6ec67fc640 1 -- 172.21.15.43:0/37770542 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13526 (secure 0 0 0) 0x7f6ed00ae8e0 con 0x7f6eb406b500 2024-04-25T09:04:51.304 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:04:51.305 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-25T09:04:51.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.317+0000 7f6ed803a640 1 -- 172.21.15.43:0/37770542 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6eb406b500 msgr2=0x7f6eb406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:51.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.317+0000 7f6ed803a640 1 --2- 172.21.15.43:0/37770542 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6eb406b500 0x7f6eb406d9c0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc035000 tx=0x7f6ecc00c040 comp rx=0 tx=0).stop 2024-04-25T09:04:51.306 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.317+0000 7f6ed803a640 1 -- 172.21.15.43:0/37770542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ed015dc50 msgr2=0x7f6ed0143f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:51.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.317+0000 7f6ed803a640 1 --2- 172.21.15.43:0/37770542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ed015dc50 0x7f6ed0143f90 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f6ec800cc40 tx=0x7f6ec800a430 comp rx=0 tx=0).stop 2024-04-25T09:04:51.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.317+0000 7f6ed803a640 1 -- 172.21.15.43:0/37770542 shutdown_connections 2024-04-25T09:04:51.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.317+0000 7f6ed803a640 1 --2- 172.21.15.43:0/37770542 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6ed015dc50 0x7f6ed0143f90 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:51.309 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.317+0000 7f6ed803a640 1 --2- 172.21.15.43:0/37770542 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6ed015a250 0x7f6ed0155ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:51.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.317+0000 7f6ed803a640 1 --2- 172.21.15.43:0/37770542 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6eb406b500 0x7f6eb406d9c0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:51.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.317+0000 7f6ed803a640 1 --2- 172.21.15.43:0/37770542 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6ed00715a0 0x7f6ed0151a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:51.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.317+0000 7f6ed803a640 1 -- 172.21.15.43:0/37770542 >> 172.21.15.43:0/37770542 conn(0x7f6ed00b1480 msgr2=0x7f6ed015a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:51.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.317+0000 7f6ed803a640 1 -- 172.21.15.43:0/37770542 shutdown_connections 2024-04-25T09:04:51.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.317+0000 7f6ed803a640 1 -- 172.21.15.43:0/37770542 wait complete. 2024-04-25T09:04:51.316 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":461,"stamp":"2024-04-25T09:04:49.843897+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2604328,"kb_used_data":2116096,"kb_used_omap":12,"kb_used_meta":488115,"kb_avail":752370392,"statfs":{"total":773094113280,"available":770427281408,"internally_reserved":0,"allocated":2166882304,"data_stored":2158930549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":499830321},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1077828404,"num_objects":-1046,"num_object_clones":0,"num_object_copies":-2092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1046,"num_whiteouts":0,"num_read":-19,"num_read_kb":-4003,"num_write":-1127,"num_write_kb":-1052713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001408"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":396,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:04:47.798685+0000","last_change":"2024-04-25T08:54:00.818044+0000","last_active":"2024-04-25T09:04:47.798685+0000","last_peered":"2024-04-25T09:04:47.798685+0000","last_clean":"2024-04-25T09:04:47.798685+0000","last_became_active":"2024-04-25T08:54:00.817098+0000","last_became_peered":"2024-04-25T08:54:00.817098+0000","last_unstale":"2024-04-25T09:04:47.798685+0000","last_undegraded":"2024-04-25T09:04:47.798685+0000","last_fullsized":"2024-04-25T09:04:47.798685+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_clean_scrub_stamp":"2024-04-25T08:53:59.739677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:48.129513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574980,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":353552,"kb_used_data":308484,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94018288,"statfs":{"total":96636764160,"available":96274726912,"internally_reserved":0,"allocated":315887616,"data_stored":314889518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574981,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":327048,"kb_used_data":259772,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94044792,"statfs":{"total":96636764160,"available":96301867008,"internally_reserved":0,"allocated":266006528,"data_stored":265010221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574983,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307024,"kb_used_data":253632,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94064816,"statfs":{"total":96636764160,"available":96322371584,"internally_reserved":0,"allocated":259719168,"data_stored":258724622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574981,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411004,"kb_used_data":343728,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":93960836,"statfs":{"total":96636764160,"available":96215896064,"internally_reserved":0,"allocated":351977472,"data_stored":350993307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574981,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":291340,"kb_used_data":215744,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94080500,"statfs":{"total":96636764160,"available":96338432000,"internally_reserved":0,"allocated":220921856,"data_stored":219927310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574981,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300816,"kb_used_data":251584,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94071024,"statfs":{"total":96636764160,"available":96328728576,"internally_reserved":0,"allocated":257622016,"data_stored":256627470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574981,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":344000,"kb_used_data":262836,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94027840,"statfs":{"total":96636764160,"available":96284508160,"internally_reserved":0,"allocated":269144064,"data_stored":268156385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574982,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":269544,"kb_used_data":220316,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94102296,"statfs":{"total":96636764160,"available":96360751104,"internally_reserved":0,"allocated":225603584,"data_stored":224601716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:04:51.317 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T09:04:51.317 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:04:51.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.429+0000 7fb23d4b3640 1 Processor -- start 2024-04-25T09:04:51.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.429+0000 7fb23d4b3640 1 -- start start 2024-04-25T09:04:51.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.430+0000 7fb23d4b3640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb2380d09b0 0x7fb2380b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:51.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.430+0000 7fb23d4b3640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2380b4c30 con 0x7fb2380ce5e0 2024-04-25T09:04:51.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.430+0000 7fb23d4b3640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2380cf2c0 con 0x7fb238161390 2024-04-25T09:04:51.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.430+0000 7fb23d4b3640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2380d1910 con 0x7fb2380d09b0 2024-04-25T09:04:51.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.430+0000 7fb2367fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb2380d09b0 0x7fb2380b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:51.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.430+0000 7fb2377fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fb2380ce5e0 0x7fb2380cea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56528/0 (socket says 172.21.15.43:56528) 2024-04-25T09:04:51.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.430+0000 7fb2367fc640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fb2380d09b0 0x7fb2380b6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43444/0 (socket says 172.21.15.43:43444) 2024-04-25T09:04:51.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.430+0000 7fb2377fe640 1 -- 172.21.15.43:0/7953969 learned_addr learned my addr 172.21.15.43:0/7953969 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:51.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.430+0000 7fb2367fc640 1 -- 172.21.15.43:0/7953969 >> v1:172.21.15.193:6789/0 conn(0x7fb238161390 legacy=0x7fb2380d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:51.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.430+0000 7fb2367fc640 1 -- 172.21.15.43:0/7953969 >> v1:172.21.15.43:6789/0 conn(0x7fb2380ce5e0 legacy=0x7fb2380cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:51.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.430+0000 7fb2367fc640 1 -- 172.21.15.43:0/7953969 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2380cf5a0 con 0x7fb2380d09b0 2024-04-25T09:04:51.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.430+0000 7fb235ffb640 1 -- 172.21.15.43:0/7953969 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2763745055 0 0) 0x7fb230001490 con 0x7fb2380ce5e0 2024-04-25T09:04:51.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.431+0000 7fb2367fc640 1 --2- 172.21.15.43:0/7953969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2380d09b0 0x7fb2380b6820 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fb22c0097a0 tx=0x7fb22c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=558c1a5debff1785 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:51.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.431+0000 7fb235ffb640 1 -- 172.21.15.43:0/7953969 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22c037070 con 0x7fb2380d09b0 2024-04-25T09:04:51.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.431+0000 7fb235ffb640 1 -- 172.21.15.43:0/7953969 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb22c032440 con 0x7fb2380d09b0 2024-04-25T09:04:51.421 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.431+0000 7fb235ffb640 1 -- 172.21.15.43:0/7953969 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22c032810 con 0x7fb2380d09b0 2024-04-25T09:04:51.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.432+0000 7fb23d4b3640 1 -- 172.21.15.43:0/7953969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2380d09b0 msgr2=0x7fb2380b6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:51.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.432+0000 7fb23d4b3640 1 --2- 172.21.15.43:0/7953969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2380d09b0 0x7fb2380b6820 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fb22c0097a0 tx=0x7fb22c02a230 comp rx=0 tx=0).stop 2024-04-25T09:04:51.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.432+0000 7fb23d4b3640 1 -- 172.21.15.43:0/7953969 shutdown_connections 2024-04-25T09:04:51.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.432+0000 7fb23d4b3640 1 --2- 172.21.15.43:0/7953969 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb2380d09b0 0x7fb2380b6820 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:51.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.432+0000 7fb23d4b3640 1 -- 172.21.15.43:0/7953969 >> 172.21.15.43:0/7953969 conn(0x7fb23809cd20 msgr2=0x7fb2380d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:51.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.432+0000 7fb23d4b3640 1 -- 172.21.15.43:0/7953969 shutdown_connections 2024-04-25T09:04:51.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.432+0000 7fb23d4b3640 1 -- 172.21.15.43:0/7953969 wait complete. 2024-04-25T09:04:51.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.433+0000 7fb23d4b3640 1 Processor -- start 2024-04-25T09:04:51.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.433+0000 7fb23d4b3640 1 -- start start 2024-04-25T09:04:51.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.434+0000 7fb23d4b3640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2380ce5e0 0x7fb2380cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:51.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.434+0000 7fb23d4b3640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb2380d09b0 0x7fb2381f4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:51.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.434+0000 7fb23d4b3640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb238161390 0x7fb2380cb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:51.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.434+0000 7fb23d4b3640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb238161770 con 0x7fb238161390 2024-04-25T09:04:51.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.434+0000 7fb23d4b3640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2380d15a0 con 0x7fb2380ce5e0 2024-04-25T09:04:51.425 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.434+0000 7fb23d4b3640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2380d1da0 con 0x7fb2380d09b0 2024-04-25T09:04:51.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.434+0000 7fb2377fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb238161390 0x7fb2380cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:51.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.434+0000 7fb2377fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb238161390 0x7fb2380cb560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43448/0 (socket says 172.21.15.43:43448) 2024-04-25T09:04:51.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.434+0000 7fb2377fe640 1 -- 172.21.15.43:0/4067309578 learned_addr learned my addr 172.21.15.43:0/4067309578 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:51.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.434+0000 7fb2367fc640 1 --2- 172.21.15.43:0/4067309578 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb2380d09b0 0x7fb2381f4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:51.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.434+0000 7fb236ffd640 1 --2- 172.21.15.43:0/4067309578 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2380ce5e0 0x7fb2380cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:51.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.434+0000 7fb2377fe640 1 -- 172.21.15.43:0/4067309578 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb2380d09b0 msgr2=0x7fb2381f4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:51.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.434+0000 7fb2377fe640 1 --2- 172.21.15.43:0/4067309578 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb2380d09b0 0x7fb2381f4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:51.426 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.434+0000 7fb2377fe640 1 -- 172.21.15.43:0/4067309578 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2380ce5e0 msgr2=0x7fb2380cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:51.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.434+0000 7fb2377fe640 1 --2- 172.21.15.43:0/4067309578 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2380ce5e0 0x7fb2380cc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:51.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.434+0000 7fb2377fe640 1 -- 172.21.15.43:0/4067309578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2380cbaa0 con 0x7fb238161390 2024-04-25T09:04:51.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.434+0000 7fb2367fc640 1 --2- 172.21.15.43:0/4067309578 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb2380d09b0 0x7fb2381f4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:51.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.434+0000 7fb2377fe640 1 --2- 172.21.15.43:0/4067309578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb238161390 0x7fb2380cb560 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7fb23000c820 tx=0x7fb23000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:51.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.435+0000 7fb21f7fe640 1 -- 172.21.15.43:0/4067309578 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb230018070 con 0x7fb238161390 2024-04-25T09:04:51.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.435+0000 7fb21f7fe640 1 -- 172.21.15.43:0/4067309578 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb23000a5e0 con 0x7fb238161390 2024-04-25T09:04:51.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.435+0000 7fb23d4b3640 1 -- 172.21.15.43:0/4067309578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2380cc970 con 0x7fb238161390 2024-04-25T09:04:51.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.435+0000 7fb21f7fe640 1 -- 172.21.15.43:0/4067309578 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb23000a9d0 con 0x7fb238161390 2024-04-25T09:04:51.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.435+0000 7fb23d4b3640 1 -- 172.21.15.43:0/4067309578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2380ccf30 con 0x7fb238161390 2024-04-25T09:04:51.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.436+0000 7fb23d4b3640 1 -- 172.21.15.43:0/4067309578 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2380680f0 con 0x7fb238161390 2024-04-25T09:04:51.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.436+0000 7fb21f7fe640 1 -- 172.21.15.43:0/4067309578 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb23001b070 con 0x7fb238161390 2024-04-25T09:04:51.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.437+0000 7fb21f7fe640 1 --2- 172.21.15.43:0/4067309578 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb22406b500 0x7fb22406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:51.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.437+0000 7fb21f7fe640 1 -- 172.21.15.43:0/4067309578 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb230089450 con 0x7fb238161390 2024-04-25T09:04:51.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.437+0000 7fb236ffd640 1 --2- 172.21.15.43:0/4067309578 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb22406b500 0x7fb22406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:51.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.437+0000 7fb236ffd640 1 --2- 172.21.15.43:0/4067309578 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb22406b500 0x7fb22406d9c0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fb228000f60 tx=0x7fb228009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:51.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.441+0000 7fb21f7fe640 1 -- 172.21.15.43:0/4067309578 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb2380b4c30 con 0x7fb238161390 2024-04-25T09:04:51.605 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.616+0000 7fb23d4b3640 1 -- 172.21.15.43:0/4067309578 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb2381f5030 con 0x7fb22406b500 2024-04-25T09:04:51.606 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.617+0000 7fb21f7fe640 1 -- 172.21.15.43:0/4067309578 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13526 (secure 0 0 0) 0x7fb2280050d0 con 0x7fb22406b500 2024-04-25T09:04:51.606 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:04:51.606 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-25T09:04:51.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.619+0000 7fb23d4b3640 1 -- 172.21.15.43:0/4067309578 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb22406b500 msgr2=0x7fb22406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:51.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.619+0000 7fb23d4b3640 1 --2- 172.21.15.43:0/4067309578 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb22406b500 0x7fb22406d9c0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fb228000f60 tx=0x7fb228009040 comp rx=0 tx=0).stop 2024-04-25T09:04:51.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.619+0000 7fb23d4b3640 1 -- 172.21.15.43:0/4067309578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb238161390 msgr2=0x7fb2380cb560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:51.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.619+0000 7fb23d4b3640 1 --2- 172.21.15.43:0/4067309578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb238161390 0x7fb2380cb560 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7fb23000c820 tx=0x7fb23000cce0 comp rx=0 tx=0).stop 2024-04-25T09:04:51.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.619+0000 7fb23d4b3640 1 -- 172.21.15.43:0/4067309578 shutdown_connections 2024-04-25T09:04:51.608 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.619+0000 7fb23d4b3640 1 --2- 172.21.15.43:0/4067309578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fb238161390 0x7fb2380cb560 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:51.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.619+0000 7fb23d4b3640 1 --2- 172.21.15.43:0/4067309578 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fb2380d09b0 0x7fb2381f4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:51.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.619+0000 7fb23d4b3640 1 --2- 172.21.15.43:0/4067309578 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fb22406b500 0x7fb22406d9c0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:51.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.619+0000 7fb23d4b3640 1 --2- 172.21.15.43:0/4067309578 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fb2380ce5e0 0x7fb2380cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:51.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.619+0000 7fb23d4b3640 1 -- 172.21.15.43:0/4067309578 >> 172.21.15.43:0/4067309578 conn(0x7fb23809cd20 msgr2=0x7fb238148a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:51.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.619+0000 7fb23d4b3640 1 -- 172.21.15.43:0/4067309578 shutdown_connections 2024-04-25T09:04:51.609 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.619+0000 7fb23d4b3640 1 -- 172.21.15.43:0/4067309578 wait complete. 2024-04-25T09:04:51.619 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":461,"stamp":"2024-04-25T09:04:49.843897+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2604328,"kb_used_data":2116096,"kb_used_omap":12,"kb_used_meta":488115,"kb_avail":752370392,"statfs":{"total":773094113280,"available":770427281408,"internally_reserved":0,"allocated":2166882304,"data_stored":2158930549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":499830321},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1077828404,"num_objects":-1046,"num_object_clones":0,"num_object_copies":-2092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1046,"num_whiteouts":0,"num_read":-19,"num_read_kb":-4003,"num_write":-1127,"num_write_kb":-1052713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001408"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":396,"reported_epoch":141,"state":"active+clean","last_fresh":"2024-04-25T09:04:47.798685+0000","last_change":"2024-04-25T08:54:00.818044+0000","last_active":"2024-04-25T09:04:47.798685+0000","last_peered":"2024-04-25T09:04:47.798685+0000","last_clean":"2024-04-25T09:04:47.798685+0000","last_became_active":"2024-04-25T08:54:00.817098+0000","last_became_peered":"2024-04-25T08:54:00.817098+0000","last_unstale":"2024-04-25T09:04:47.798685+0000","last_undegraded":"2024-04-25T09:04:47.798685+0000","last_fullsized":"2024-04-25T09:04:47.798685+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_clean_scrub_stamp":"2024-04-25T08:53:59.739677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:48.129513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574980,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":353552,"kb_used_data":308484,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94018288,"statfs":{"total":96636764160,"available":96274726912,"internally_reserved":0,"allocated":315887616,"data_stored":314889518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574981,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":327048,"kb_used_data":259772,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94044792,"statfs":{"total":96636764160,"available":96301867008,"internally_reserved":0,"allocated":266006528,"data_stored":265010221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574983,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307024,"kb_used_data":253632,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94064816,"statfs":{"total":96636764160,"available":96322371584,"internally_reserved":0,"allocated":259719168,"data_stored":258724622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574981,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":411004,"kb_used_data":343728,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":93960836,"statfs":{"total":96636764160,"available":96215896064,"internally_reserved":0,"allocated":351977472,"data_stored":350993307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574981,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":291340,"kb_used_data":215744,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94080500,"statfs":{"total":96636764160,"available":96338432000,"internally_reserved":0,"allocated":220921856,"data_stored":219927310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574981,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300816,"kb_used_data":251584,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94071024,"statfs":{"total":96636764160,"available":96328728576,"internally_reserved":0,"allocated":257622016,"data_stored":256627470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574981,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":344000,"kb_used_data":262836,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94027840,"statfs":{"total":96636764160,"available":96284508160,"internally_reserved":0,"allocated":269144064,"data_stored":268156385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574982,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":269544,"kb_used_data":220316,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94102296,"statfs":{"total":96636764160,"available":96360751104,"internally_reserved":0,"allocated":225603584,"data_stored":224601716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:04:51.619 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T09:04:51.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.732+0000 7f4c26f29640 1 Processor -- start 2024-04-25T09:04:51.721 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.732+0000 7f4c26f29640 1 -- start start 2024-04-25T09:04:51.723 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.733+0000 7f4c26f29640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4c2015f180 0x7f4c200cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:51.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.733+0000 7f4c26f29640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f4c200b2cc0 0x7f4c200cfda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:51.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.733+0000 7f4c26f29640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c200ccf10 con 0x7f4c200b23f0 2024-04-25T09:04:51.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.733+0000 7f4c26f29640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c200cccc0 con 0x7f4c2015f180 2024-04-25T09:04:51.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.733+0000 7f4c26f29640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c200cd840 con 0x7f4c200b2cc0 2024-04-25T09:04:51.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.733+0000 7f4c24cc5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4c2015f180 0x7f4c200cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:51.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.733+0000 7f4c24cc5640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f4c2015f180 0x7f4c200cf860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43450/0 (socket says 172.21.15.43:43450) 2024-04-25T09:04:51.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.733+0000 7f4c24cc5640 1 -- 172.21.15.43:0/1894244601 learned_addr learned my addr 172.21.15.43:0/1894244601 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:51.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.733+0000 7f4c24cc5640 1 -- 172.21.15.43:0/1894244601 >> v2:172.21.15.193:3300/0 conn(0x7f4c200b2cc0 msgr2=0x7f4c200cfda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:51.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.733+0000 7f4c254c6640 1 --2- 172.21.15.43:0/1894244601 >> v2:172.21.15.193:3300/0 conn(0x7f4c200b2cc0 0x7f4c200cfda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:51.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.734+0000 7f4c24cc5640 1 --2- 172.21.15.43:0/1894244601 >> v2:172.21.15.193:3300/0 conn(0x7f4c200b2cc0 0x7f4c200cfda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:51.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.734+0000 7f4c24cc5640 1 -- 172.21.15.43:0/1894244601 >> v1:172.21.15.193:6789/0 conn(0x7f4c200b23f0 legacy=0x7f4c2015f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:51.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.734+0000 7f4c24cc5640 1 -- 172.21.15.43:0/1894244601 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c200cd1f0 con 0x7f4c2015f180 2024-04-25T09:04:51.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.734+0000 7f4c254c6640 1 --2- 172.21.15.43:0/1894244601 >> v2:172.21.15.193:3300/0 conn(0x7f4c200b2cc0 0x7f4c200cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:51.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.734+0000 7f4c24cc5640 1 --2- 172.21.15.43:0/1894244601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c2015f180 0x7f4c200cf860 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f4c200b3150 tx=0x7f4c1802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce57ef932506dda3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:51.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.734+0000 7f4c1f7fe640 1 -- 172.21.15.43:0/1894244601 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c18037070 con 0x7f4c2015f180 2024-04-25T09:04:51.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.734+0000 7f4c1f7fe640 1 -- 172.21.15.43:0/1894244601 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c18032440 con 0x7f4c2015f180 2024-04-25T09:04:51.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.734+0000 7f4c1f7fe640 1 -- 172.21.15.43:0/1894244601 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c18032810 con 0x7f4c2015f180 2024-04-25T09:04:51.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.735+0000 7f4c26f29640 1 -- 172.21.15.43:0/1894244601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c2015f180 msgr2=0x7f4c200cf860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:51.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.735+0000 7f4c26f29640 1 --2- 172.21.15.43:0/1894244601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c2015f180 0x7f4c200cf860 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f4c200b3150 tx=0x7f4c1802a1a0 comp rx=0 tx=0).stop 2024-04-25T09:04:51.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.735+0000 7f4c26f29640 1 -- 172.21.15.43:0/1894244601 shutdown_connections 2024-04-25T09:04:51.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.735+0000 7f4c26f29640 1 --2- 172.21.15.43:0/1894244601 >> v2:172.21.15.193:3300/0 conn(0x7f4c200b2cc0 0x7f4c200cfda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:51.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.735+0000 7f4c26f29640 1 --2- 172.21.15.43:0/1894244601 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c2015f180 0x7f4c200cf860 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:51.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.735+0000 7f4c26f29640 1 -- 172.21.15.43:0/1894244601 >> 172.21.15.43:0/1894244601 conn(0x7f4c200d17f0 msgr2=0x7f4c200d1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:51.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.735+0000 7f4c26f29640 1 -- 172.21.15.43:0/1894244601 shutdown_connections 2024-04-25T09:04:51.726 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.735+0000 7f4c26f29640 1 -- 172.21.15.43:0/1894244601 wait complete. 2024-04-25T09:04:51.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.736+0000 7f4c26f29640 1 Processor -- start 2024-04-25T09:04:51.730 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.736+0000 7f4c26f29640 1 -- start start 2024-04-25T09:04:51.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.737+0000 7f4c26f29640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c200b23f0 0x7f4c2006c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:51.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.737+0000 7f4c26f29640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4c200b2cc0 0x7f4c2014caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:51.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.737+0000 7f4c26f29640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c2015f180 0x7f4c200c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:51.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.737+0000 7f4c26f29640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2015f560 con 0x7f4c200b23f0 2024-04-25T09:04:51.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.737+0000 7f4c26f29640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c200cece0 con 0x7f4c2015f180 2024-04-25T09:04:51.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.737+0000 7f4c26f29640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2014cfe0 con 0x7f4c200b2cc0 2024-04-25T09:04:51.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.737+0000 7f4c24cc5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c200b23f0 0x7f4c2006c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:51.731 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.737+0000 7f4c24cc5640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c200b23f0 0x7f4c2006c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43464/0 (socket says 172.21.15.43:43464) 2024-04-25T09:04:51.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.737+0000 7f4c24cc5640 1 -- 172.21.15.43:0/1547750158 learned_addr learned my addr 172.21.15.43:0/1547750158 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:51.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.737+0000 7f4c254c6640 1 --2- 172.21.15.43:0/1547750158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c2015f180 0x7f4c200c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:51.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.737+0000 7f4c1ffff640 1 --2- 172.21.15.43:0/1547750158 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4c200b2cc0 0x7f4c2014caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:51.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.738+0000 7f4c24cc5640 1 -- 172.21.15.43:0/1547750158 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4c200b2cc0 msgr2=0x7f4c2014caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:51.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.738+0000 7f4c24cc5640 1 --2- 172.21.15.43:0/1547750158 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4c200b2cc0 0x7f4c2014caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:51.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.738+0000 7f4c24cc5640 1 -- 172.21.15.43:0/1547750158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c2015f180 msgr2=0x7f4c200c88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:51.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.738+0000 7f4c24cc5640 1 --2- 172.21.15.43:0/1547750158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c2015f180 0x7f4c200c88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:51.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.738+0000 7f4c24cc5640 1 -- 172.21.15.43:0/1547750158 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c2006d560 con 0x7f4c200b23f0 2024-04-25T09:04:51.732 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.738+0000 7f4c24cc5640 1 --2- 172.21.15.43:0/1547750158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c200b23f0 0x7f4c2006c650 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f4c18000c00 tx=0x7f4c180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:51.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.738+0000 7f4c1d7fa640 1 -- 172.21.15.43:0/1547750158 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c18037040 con 0x7f4c200b23f0 2024-04-25T09:04:51.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.738+0000 7f4c1d7fa640 1 -- 172.21.15.43:0/1547750158 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c18042070 con 0x7f4c200b23f0 2024-04-25T09:04:51.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.738+0000 7f4c26f29640 1 -- 172.21.15.43:0/1547750158 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c2006ccc0 con 0x7f4c200b23f0 2024-04-25T09:04:51.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.738+0000 7f4c1d7fa640 1 -- 172.21.15.43:0/1547750158 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c18030bb0 con 0x7f4c200b23f0 2024-04-25T09:04:51.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.738+0000 7f4c26f29640 1 -- 172.21.15.43:0/1547750158 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c2006d280 con 0x7f4c200b23f0 2024-04-25T09:04:51.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.739+0000 7f4c26f29640 1 -- 172.21.15.43:0/1547750158 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c20074ef0 con 0x7f4c200b23f0 2024-04-25T09:04:51.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.739+0000 7f4c1d7fa640 1 -- 172.21.15.43:0/1547750158 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4c18047460 con 0x7f4c200b23f0 2024-04-25T09:04:51.733 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.740+0000 7f4c1d7fa640 1 --2- 172.21.15.43:0/1547750158 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4c0006b4d0 0x7f4c0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:51.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.740+0000 7f4c1d7fa640 1 -- 172.21.15.43:0/1547750158 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4c180aa8d0 con 0x7f4c200b23f0 2024-04-25T09:04:51.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.740+0000 7f4c1ffff640 1 --2- 172.21.15.43:0/1547750158 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4c0006b4d0 0x7f4c0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:51.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.740+0000 7f4c1ffff640 1 --2- 172.21.15.43:0/1547750158 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4c0006b4d0 0x7f4c0006d990 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f4c200b0cb0 tx=0x7f4c10003d90 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:51.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.744+0000 7f4c1d7fa640 1 -- 172.21.15.43:0/1547750158 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4c18079490 con 0x7f4c200b23f0 2024-04-25T09:04:51.915 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.927+0000 7f4c26f29640 1 -- 172.21.15.43:0/1547750158 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c200cf050 con 0x7f4c200b23f0 2024-04-25T09:04:51.916 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.927+0000 7f4c1d7fa640 1 -- 172.21.15.43:0/1547750158 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v142) v1 ==== 74+0+12996 (secure 0 0 0) 0x7f4c1807d530 con 0x7f4c200b23f0 2024-04-25T09:04:51.916 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:04:51.916 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":142,"fsid":"6c4bd50e-aa00-4a7f-8d09-352344ef5a4b","created":"2024-04-25T08:53:47.572559+0000","modified":"2024-04-25T09:04:48.794615+0000","last_up_change":"2024-04-25T08:53:58.162936+0000","last_in_change":"2024-04-25T08:53:49.652694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T08:53:59.739591+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a7d2992b-c326-4e19-9b74-72f017cc2613","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6817","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6819","nonce":252044399}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6821","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6823","nonce":252044399}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6829","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6831","nonce":252044399}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6825","nonce":252044399},{"type":"v1","addr":"172.21.15.43:6827","nonce":252044399}]},"public_addr":"172.21.15.43:6819/252044399","cluster_addr":"172.21.15.43:6823/252044399","heartbeat_back_addr":"172.21.15.43:6831/252044399","heartbeat_front_addr":"172.21.15.43:6827/252044399","state":["exists","up"]},{"osd":1,"uuid":"88734834-c723-40a7-914b-1090e7fe47ac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6816","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6818","nonce":179790819}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6820","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6822","nonce":179790819}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6828","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6830","nonce":179790819}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6824","nonce":179790819},{"type":"v1","addr":"172.21.15.43:6826","nonce":179790819}]},"public_addr":"172.21.15.43:6818/179790819","cluster_addr":"172.21.15.43:6822/179790819","heartbeat_back_addr":"172.21.15.43:6830/179790819","heartbeat_front_addr":"172.21.15.43:6826/179790819","state":["exists","up"]},{"osd":2,"uuid":"b938e87d-7038-4a58-acdb-ff9a43a1855b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6808","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6809","nonce":1175991178}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6810","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6811","nonce":1175991178}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6814","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6815","nonce":1175991178}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6812","nonce":1175991178},{"type":"v1","addr":"172.21.15.43:6813","nonce":1175991178}]},"public_addr":"172.21.15.43:6809/1175991178","cluster_addr":"172.21.15.43:6811/1175991178","heartbeat_back_addr":"172.21.15.43:6815/1175991178","heartbeat_front_addr":"172.21.15.43:6813/1175991178","state":["exists","up"]},{"osd":3,"uuid":"b34857e6-f4b8-4177-82ea-041f4bb1c42c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6800","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6801","nonce":1101549916}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6802","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6803","nonce":1101549916}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6806","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6807","nonce":1101549916}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6804","nonce":1101549916},{"type":"v1","addr":"172.21.15.43:6805","nonce":1101549916}]},"public_addr":"172.21.15.43:6801/1101549916","cluster_addr":"172.21.15.43:6803/1101549916","heartbeat_back_addr":"172.21.15.43:6807/1101549916","heartbeat_front_addr":"172.21.15.43:6805/1101549916","state":["exists","up"]},{"osd":4,"uuid":"cbbc90d8-107f-4d0d-8a3a-38ccca6ff605","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6816","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6817","nonce":77587867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6818","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6819","nonce":77587867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6824","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6825","nonce":77587867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6820","nonce":77587867},{"type":"v1","addr":"172.21.15.193:6822","nonce":77587867}]},"public_addr":"172.21.15.193:6817/77587867","cluster_addr":"172.21.15.193:6819/77587867","heartbeat_back_addr":"172.21.15.193:6825/77587867","heartbeat_front_addr":"172.21.15.193:6822/77587867","state":["exists","up"]},{"osd":5,"uuid":"69190f87-1f31-4ab9-b6cb-f1904c0ec57a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6800","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6801","nonce":3423021177}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6802","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6803","nonce":3423021177}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6806","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6807","nonce":3423021177}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6804","nonce":3423021177},{"type":"v1","addr":"172.21.15.193:6805","nonce":3423021177}]},"public_addr":"172.21.15.193:6801/3423021177","cluster_addr":"172.21.15.193:6803/3423021177","heartbeat_back_addr":"172.21.15.193:6807/3423021177","heartbeat_front_addr":"172.21.15.193:6805/3423021177","state":["exists","up"]},{"osd":6,"uuid":"74c61ad4-7316-45da-9aa6-2cdbb3135ae3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6821","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6823","nonce":1737328270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6826","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6827","nonce":1737328270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6830","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6831","nonce":1737328270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6828","nonce":1737328270},{"type":"v1","addr":"172.21.15.193:6829","nonce":1737328270}]},"public_addr":"172.21.15.193:6823/1737328270","cluster_addr":"172.21.15.193:6827/1737328270","heartbeat_back_addr":"172.21.15.193:6831/1737328270","heartbeat_front_addr":"172.21.15.193:6829/1737328270","state":["exists","up"]},{"osd":7,"uuid":"5e4881b4-eaf1-4ef8-a114-9d1a4167d8b7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":137,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6808","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6809","nonce":3086001447}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6810","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6811","nonce":3086001447}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6814","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6815","nonce":3086001447}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6812","nonce":3086001447},{"type":"v1","addr":"172.21.15.193:6813","nonce":3086001447}]},"public_addr":"172.21.15.193:6809/3086001447","cluster_addr":"172.21.15.193:6811/3086001447","heartbeat_back_addr":"172.21.15.193:6815/3086001447","heartbeat_front_addr":"172.21.15.193:6813/3086001447","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.831552+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.822117+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.752145+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.757486+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.799078+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.707400+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.820649+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T08:53:54.760427+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.193:6837/2585534546":"2024-04-26T09:04:45.785495+0000","172.21.15.193:6835/2585534546":"2024-04-26T09:04:45.785495+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T09:04:51.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.930+0000 7f4c26f29640 1 -- 172.21.15.43:0/1547750158 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4c0006b4d0 msgr2=0x7f4c0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:51.918 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.930+0000 7f4c26f29640 1 --2- 172.21.15.43:0/1547750158 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4c0006b4d0 0x7f4c0006d990 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f4c200b0cb0 tx=0x7f4c10003d90 comp rx=0 tx=0).stop 2024-04-25T09:04:51.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.930+0000 7f4c26f29640 1 -- 172.21.15.43:0/1547750158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c200b23f0 msgr2=0x7f4c2006c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:51.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.930+0000 7f4c26f29640 1 --2- 172.21.15.43:0/1547750158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c200b23f0 0x7f4c2006c650 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f4c18000c00 tx=0x7f4c180046c0 comp rx=0 tx=0).stop 2024-04-25T09:04:51.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.930+0000 7f4c26f29640 1 -- 172.21.15.43:0/1547750158 shutdown_connections 2024-04-25T09:04:51.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.930+0000 7f4c26f29640 1 --2- 172.21.15.43:0/1547750158 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f4c0006b4d0 0x7f4c0006d990 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:51.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.930+0000 7f4c26f29640 1 --2- 172.21.15.43:0/1547750158 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f4c2015f180 0x7f4c200c88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:51.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.930+0000 7f4c26f29640 1 --2- 172.21.15.43:0/1547750158 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f4c200b2cc0 0x7f4c2014caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:51.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.930+0000 7f4c26f29640 1 --2- 172.21.15.43:0/1547750158 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f4c200b23f0 0x7f4c2006c650 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:51.919 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.930+0000 7f4c26f29640 1 -- 172.21.15.43:0/1547750158 >> 172.21.15.43:0/1547750158 conn(0x7f4c200d17f0 msgr2=0x7f4c2006fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:51.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.930+0000 7f4c26f29640 1 -- 172.21.15.43:0/1547750158 shutdown_connections 2024-04-25T09:04:51.920 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:51.930+0000 7f4c26f29640 1 -- 172.21.15.43:0/1547750158 wait complete. 2024-04-25T09:04:52.931 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T09:04:52.931 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T09:04:53.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.061+0000 7f36a4134640 1 Processor -- start 2024-04-25T09:04:53.049 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.061+0000 7f36a4134640 1 -- start start 2024-04-25T09:04:53.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.062+0000 7f36a4134640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f369c0b0b90 0x7f369c0af9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.062+0000 7f36a4134640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f369c0b1920 0x7f369c0b1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.062+0000 7f36a4134640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f369c0b1420 con 0x7f369c0b54a0 2024-04-25T09:04:53.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.062+0000 7f36a4134640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f369c15bf20 con 0x7f369c0b0b90 2024-04-25T09:04:53.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.062+0000 7f36a4134640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f369c0affd0 con 0x7f369c0b1920 2024-04-25T09:04:53.051 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.062+0000 7f36a16cf640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f369c0b0b90 0x7f369c0af9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.062+0000 7f36a16cf640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f369c0b0b90 0x7f369c0af9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43478/0 (socket says 172.21.15.43:43478) 2024-04-25T09:04:53.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.062+0000 7f36a16cf640 1 -- 172.21.15.43:0/958307028 learned_addr learned my addr 172.21.15.43:0/958307028 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:53.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.062+0000 7f36a26d1640 1 --2- 172.21.15.43:0/958307028 >> v2:172.21.15.193:3300/0 conn(0x7f369c0b1920 0x7f369c0b1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.062+0000 7f36a16cf640 1 -- 172.21.15.43:0/958307028 >> v2:172.21.15.193:3300/0 conn(0x7f369c0b1920 msgr2=0x7f369c0b1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:53.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.062+0000 7f36a16cf640 1 --2- 172.21.15.43:0/958307028 >> v2:172.21.15.193:3300/0 conn(0x7f369c0b1920 0x7f369c0b1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.062+0000 7f36a16cf640 1 -- 172.21.15.43:0/958307028 >> v1:172.21.15.193:6789/0 conn(0x7f369c0b54a0 legacy=0x7f369c0b5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.062+0000 7f36a0ece640 1 -- 172.21.15.43:0/958307028 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1552743851 0 0) 0x7f369c0b1420 con 0x7f369c0b54a0 2024-04-25T09:04:53.052 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.062+0000 7f36a16cf640 1 -- 172.21.15.43:0/958307028 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f369c0b02b0 con 0x7f369c0b0b90 2024-04-25T09:04:53.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.063+0000 7f36a16cf640 1 --2- 172.21.15.43:0/958307028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f369c0b0b90 0x7f369c0af9e0 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f369c0aec00 tx=0x7f368c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dec907d9c5244807 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:53.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.063+0000 7f36a0ece640 1 -- 172.21.15.43:0/958307028 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368c037070 con 0x7f369c0b0b90 2024-04-25T09:04:53.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.063+0000 7f36a0ece640 1 -- 172.21.15.43:0/958307028 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f368c032440 con 0x7f369c0b0b90 2024-04-25T09:04:53.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.063+0000 7f36a0ece640 1 -- 172.21.15.43:0/958307028 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368c032810 con 0x7f369c0b0b90 2024-04-25T09:04:53.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.064+0000 7f36a4134640 1 -- 172.21.15.43:0/958307028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f369c0b0b90 msgr2=0x7f369c0af9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.054 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.064+0000 7f36a4134640 1 --2- 172.21.15.43:0/958307028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f369c0b0b90 0x7f369c0af9e0 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f369c0aec00 tx=0x7f368c02a270 comp rx=0 tx=0).stop 2024-04-25T09:04:53.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.064+0000 7f36a4134640 1 -- 172.21.15.43:0/958307028 shutdown_connections 2024-04-25T09:04:53.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.064+0000 7f36a4134640 1 --2- 172.21.15.43:0/958307028 >> v2:172.21.15.193:3300/0 conn(0x7f369c0b1920 0x7f369c0b1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.064+0000 7f36a4134640 1 --2- 172.21.15.43:0/958307028 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f369c0b0b90 0x7f369c0af9e0 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.064+0000 7f36a4134640 1 -- 172.21.15.43:0/958307028 >> 172.21.15.43:0/958307028 conn(0x7f369c0b6ff0 msgr2=0x7f369c09aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:53.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.064+0000 7f36a4134640 1 -- 172.21.15.43:0/958307028 shutdown_connections 2024-04-25T09:04:53.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.064+0000 7f36a4134640 1 -- 172.21.15.43:0/958307028 wait complete. 2024-04-25T09:04:53.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.065+0000 7f36a4134640 1 Processor -- start 2024-04-25T09:04:53.055 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.065+0000 7f36a4134640 1 -- start start 2024-04-25T09:04:53.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.066+0000 7f36a4134640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f369c0b0b90 0x7f369c15c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.066+0000 7f36a4134640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f369c0b1920 0x7f369c144250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.066+0000 7f36a4134640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f369c0b54a0 0x7f369c1f7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.058 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.066+0000 7f36a4134640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369c0b4a70 con 0x7f369c0b1920 2024-04-25T09:04:53.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.066+0000 7f36a4134640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369c0b0870 con 0x7f369c0b0b90 2024-04-25T09:04:53.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.066+0000 7f36a4134640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f369c0b22d0 con 0x7f369c0b54a0 2024-04-25T09:04:53.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.066+0000 7f36a16cf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f369c0b1920 0x7f369c144250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.066+0000 7f36a16cf640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f369c0b1920 0x7f369c144250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43484/0 (socket says 172.21.15.43:43484) 2024-04-25T09:04:53.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.066+0000 7f36a16cf640 1 -- 172.21.15.43:0/2309039323 learned_addr learned my addr 172.21.15.43:0/2309039323 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:53.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.066+0000 7f36a26d1640 1 --2- 172.21.15.43:0/2309039323 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f369c0b54a0 0x7f369c1f7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.066+0000 7f36a1ed0640 1 --2- 172.21.15.43:0/2309039323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f369c0b0b90 0x7f369c15c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.059 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.067+0000 7f36a16cf640 1 -- 172.21.15.43:0/2309039323 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f369c0b54a0 msgr2=0x7f369c1f7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.067+0000 7f36a16cf640 1 --2- 172.21.15.43:0/2309039323 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f369c0b54a0 0x7f369c1f7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.067+0000 7f36a16cf640 1 -- 172.21.15.43:0/2309039323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f369c0b0b90 msgr2=0x7f369c15c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.067+0000 7f36a16cf640 1 --2- 172.21.15.43:0/2309039323 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f369c0b0b90 0x7f369c15c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.067+0000 7f36a16cf640 1 -- 172.21.15.43:0/2309039323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f369c1f7510 con 0x7f369c0b1920 2024-04-25T09:04:53.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.067+0000 7f36a26d1640 1 --2- 172.21.15.43:0/2309039323 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f369c0b54a0 0x7f369c1f7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:53.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.067+0000 7f36a16cf640 1 --2- 172.21.15.43:0/2309039323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f369c0b1920 0x7f369c144250 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f368c02a2b0 tx=0x7f368c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:53.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.067+0000 7f36927fc640 1 -- 172.21.15.43:0/2309039323 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368c037040 con 0x7f369c0b1920 2024-04-25T09:04:53.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.067+0000 7f36927fc640 1 -- 172.21.15.43:0/2309039323 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f368c042070 con 0x7f369c0b1920 2024-04-25T09:04:53.060 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.067+0000 7f36a4134640 1 -- 172.21.15.43:0/2309039323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f369c15ce90 con 0x7f369c0b1920 2024-04-25T09:04:53.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.067+0000 7f36927fc640 1 -- 172.21.15.43:0/2309039323 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368c030880 con 0x7f369c0b1920 2024-04-25T09:04:53.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.067+0000 7f36a4134640 1 -- 172.21.15.43:0/2309039323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f369c15c5e0 con 0x7f369c0b1920 2024-04-25T09:04:53.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.068+0000 7f36a4134640 1 -- 172.21.15.43:0/2309039323 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f369c068a40 con 0x7f369c0b1920 2024-04-25T09:04:53.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.068+0000 7f36927fc640 1 -- 172.21.15.43:0/2309039323 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f368c030a20 con 0x7f369c0b1920 2024-04-25T09:04:53.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.069+0000 7f36927fc640 1 --2- 172.21.15.43:0/2309039323 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f368006b4d0 0x7f368006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.069+0000 7f36927fc640 1 -- 172.21.15.43:0/2309039323 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f368c0aa700 con 0x7f369c0b1920 2024-04-25T09:04:53.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.069+0000 7f36927fc640 1 --2- 172.21.15.43:0/2309039323 >> [v2:172.21.15.43:6817/252044399,v1:172.21.15.43:6819/252044399] conn(0x7f3680071390 0x7f36800737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.069+0000 7f36927fc640 1 -- 172.21.15.43:0/2309039323 --> [v2:172.21.15.43:6817/252044399,v1:172.21.15.43:6819/252044399] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3680073e60 con 0x7f3680071390 2024-04-25T09:04:53.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.069+0000 7f36a1ed0640 1 --2- 172.21.15.43:0/2309039323 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f368006b4d0 0x7f368006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.069+0000 7f36927fc640 1 -- 172.21.15.43:0/2309039323 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=142) v2 ==== 24+0+0 (secure 0 0 0) 0x7f368c0b1050 con 0x7f369c0b1920 2024-04-25T09:04:53.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.069+0000 7f36a26d1640 1 --2- 172.21.15.43:0/2309039323 >> [v2:172.21.15.43:6817/252044399,v1:172.21.15.43:6819/252044399] conn(0x7f3680071390 0x7f36800737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.070+0000 7f36a1ed0640 1 --2- 172.21.15.43:0/2309039323 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f368006b4d0 0x7f368006d990 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f3698000f60 tx=0x7f3698009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:53.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.070+0000 7f36a26d1640 1 --2- 172.21.15.43:0/2309039323 >> [v2:172.21.15.43:6817/252044399,v1:172.21.15.43:6819/252044399] conn(0x7f3680071390 0x7f36800737b0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:53.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.070+0000 7f36927fc640 1 -- 172.21.15.43:0/2309039323 <== osd.0 v2:172.21.15.43:6817/252044399 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f3680073e60 con 0x7f3680071390 2024-04-25T09:04:53.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.084+0000 7f36a4134640 1 -- 172.21.15.43:0/2309039323 --> [v2:172.21.15.43:6817/252044399,v1:172.21.15.43:6819/252044399] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f369c0b42f0 con 0x7f3680071390 2024-04-25T09:04:53.072 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.084+0000 7f36927fc640 1 -- 172.21.15.43:0/2309039323 <== osd.0 v2:172.21.15.43:6817/252044399 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f369c0b42f0 con 0x7f3680071390 2024-04-25T09:04:53.072 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:04:53.073 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T09:04:53.073 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:04:53.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.084+0000 7f36a4134640 1 -- 172.21.15.43:0/2309039323 >> [v2:172.21.15.43:6817/252044399,v1:172.21.15.43:6819/252044399] conn(0x7f3680071390 msgr2=0x7f36800737b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.084+0000 7f36a4134640 1 --2- 172.21.15.43:0/2309039323 >> [v2:172.21.15.43:6817/252044399,v1:172.21.15.43:6819/252044399] conn(0x7f3680071390 0x7f36800737b0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.084+0000 7f36a4134640 1 -- 172.21.15.43:0/2309039323 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f368006b4d0 msgr2=0x7f368006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.084+0000 7f36a4134640 1 --2- 172.21.15.43:0/2309039323 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f368006b4d0 0x7f368006d990 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f3698000f60 tx=0x7f3698009040 comp rx=0 tx=0).stop 2024-04-25T09:04:53.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.084+0000 7f36a4134640 1 -- 172.21.15.43:0/2309039323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f369c0b1920 msgr2=0x7f369c144250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.074 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.084+0000 7f36a4134640 1 --2- 172.21.15.43:0/2309039323 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f369c0b1920 0x7f369c144250 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f368c02a2b0 tx=0x7f368c0046c0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.084+0000 7f36a26d1640 1 -- 172.21.15.43:0/2309039323 reap_dead start 2024-04-25T09:04:53.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.084+0000 7f36a4134640 1 -- 172.21.15.43:0/2309039323 shutdown_connections 2024-04-25T09:04:53.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.084+0000 7f36a4134640 1 -- 172.21.15.43:0/2309039323 >> 172.21.15.43:0/2309039323 conn(0x7f369c0b6ff0 msgr2=0x7f369c155ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:53.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.084+0000 7f36a4134640 1 -- 172.21.15.43:0/2309039323 shutdown_connections 2024-04-25T09:04:53.075 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.085+0000 7f36a4134640 1 -- 172.21.15.43:0/2309039323 wait complete. 2024-04-25T09:04:53.084 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T09:04:53.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.197+0000 7f8c4bfb7640 1 Processor -- start 2024-04-25T09:04:53.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.198+0000 7f8c4bfb7640 1 -- start start 2024-04-25T09:04:53.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.198+0000 7f8c4bfb7640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f8c4415dce0 0x7f8c4415ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.198+0000 7f8c4bfb7640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8c440ce550 0x7f8c440d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.198+0000 7f8c4bfb7640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c440cee80 con 0x7f8c440ce9c0 2024-04-25T09:04:53.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.198+0000 7f8c4bfb7640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c440d2920 con 0x7f8c440ce550 2024-04-25T09:04:53.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.198+0000 7f8c4bfb7640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c440cf160 con 0x7f8c4415dce0 2024-04-25T09:04:53.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.199+0000 7f8c49552640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8c440ce550 0x7f8c440d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.199+0000 7f8c49552640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8c440ce550 0x7f8c440d03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43494/0 (socket says 172.21.15.43:43494) 2024-04-25T09:04:53.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.199+0000 7f8c49552640 1 -- 172.21.15.43:0/2092956457 learned_addr learned my addr 172.21.15.43:0/2092956457 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:53.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.199+0000 7f8c49d53640 1 --2- 172.21.15.43:0/2092956457 >> v2:172.21.15.193:3300/0 conn(0x7f8c4415dce0 0x7f8c4415ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.199+0000 7f8c49552640 1 -- 172.21.15.43:0/2092956457 >> v2:172.21.15.193:3300/0 conn(0x7f8c4415dce0 msgr2=0x7f8c4415ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:53.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.199+0000 7f8c49552640 1 --2- 172.21.15.43:0/2092956457 >> v2:172.21.15.193:3300/0 conn(0x7f8c4415dce0 0x7f8c4415ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.199+0000 7f8c49552640 1 -- 172.21.15.43:0/2092956457 >> v1:172.21.15.193:6789/0 conn(0x7f8c440ce9c0 legacy=0x7f8c44160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.199+0000 7f8c49552640 1 -- 172.21.15.43:0/2092956457 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c440cf440 con 0x7f8c440ce550 2024-04-25T09:04:53.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.199+0000 7f8c49d53640 1 --2- 172.21.15.43:0/2092956457 >> v2:172.21.15.193:3300/0 conn(0x7f8c4415dce0 0x7f8c4415ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:53.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.199+0000 7f8c49552640 1 --2- 172.21.15.43:0/2092956457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c440ce550 0x7f8c440d03e0 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f8c4409d3c0 tx=0x7f8c3402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d070142d9c4106 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:53.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.200+0000 7f8c48d51640 1 -- 172.21.15.43:0/2092956457 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c34037070 con 0x7f8c440ce550 2024-04-25T09:04:53.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.200+0000 7f8c48d51640 1 -- 172.21.15.43:0/2092956457 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c34032440 con 0x7f8c440ce550 2024-04-25T09:04:53.190 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.200+0000 7f8c48d51640 1 -- 172.21.15.43:0/2092956457 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c34032810 con 0x7f8c440ce550 2024-04-25T09:04:53.192 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.200+0000 7f8c4bfb7640 1 -- 172.21.15.43:0/2092956457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c440ce550 msgr2=0x7f8c440d03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.200+0000 7f8c4bfb7640 1 --2- 172.21.15.43:0/2092956457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c440ce550 0x7f8c440d03e0 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f8c4409d3c0 tx=0x7f8c3402a1e0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.200+0000 7f8c4bfb7640 1 -- 172.21.15.43:0/2092956457 shutdown_connections 2024-04-25T09:04:53.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.200+0000 7f8c4bfb7640 1 --2- 172.21.15.43:0/2092956457 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c440ce550 0x7f8c440d03e0 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.200+0000 7f8c4bfb7640 1 --2- 172.21.15.43:0/2092956457 >> v2:172.21.15.193:3300/0 conn(0x7f8c4415dce0 0x7f8c4415ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.200+0000 7f8c4bfb7640 1 -- 172.21.15.43:0/2092956457 >> 172.21.15.43:0/2092956457 conn(0x7f8c4409cd00 msgr2=0x7f8c440d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:53.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.200+0000 7f8c4bfb7640 1 -- 172.21.15.43:0/2092956457 shutdown_connections 2024-04-25T09:04:53.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.200+0000 7f8c4bfb7640 1 -- 172.21.15.43:0/2092956457 wait complete. 2024-04-25T09:04:53.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.202+0000 7f8c4bfb7640 1 Processor -- start 2024-04-25T09:04:53.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.202+0000 7f8c4bfb7640 1 -- start start 2024-04-25T09:04:53.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.202+0000 7f8c4bfb7640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c440ce550 0x7f8c4409db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.202+0000 7f8c4bfb7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c440ce9c0 0x7f8c4414fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.202+0000 7f8c4bfb7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c4415dce0 0x7f8c4409fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.202+0000 7f8c4bfb7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4415e0c0 con 0x7f8c440ce9c0 2024-04-25T09:04:53.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.202+0000 7f8c4bfb7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4415f6b0 con 0x7f8c4415dce0 2024-04-25T09:04:53.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.202+0000 7f8c4bfb7640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c440d0920 con 0x7f8c440ce550 2024-04-25T09:04:53.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.202+0000 7f8c49552640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c440ce9c0 0x7f8c4414fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.202+0000 7f8c49552640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c440ce9c0 0x7f8c4414fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43504/0 (socket says 172.21.15.43:43504) 2024-04-25T09:04:53.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.203+0000 7f8c49552640 1 -- 172.21.15.43:0/3741824801 learned_addr learned my addr 172.21.15.43:0/3741824801 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:53.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.203+0000 7f8c49d53640 1 --2- 172.21.15.43:0/3741824801 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c440ce550 0x7f8c4409db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.203+0000 7f8c4a554640 1 --2- 172.21.15.43:0/3741824801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c4415dce0 0x7f8c4409fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.203+0000 7f8c49552640 1 -- 172.21.15.43:0/3741824801 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c440ce550 msgr2=0x7f8c4409db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.203+0000 7f8c49552640 1 --2- 172.21.15.43:0/3741824801 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c440ce550 0x7f8c4409db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.203+0000 7f8c49552640 1 -- 172.21.15.43:0/3741824801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c4415dce0 msgr2=0x7f8c4409fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.203+0000 7f8c49552640 1 --2- 172.21.15.43:0/3741824801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c4415dce0 0x7f8c4409fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.203+0000 7f8c49552640 1 -- 172.21.15.43:0/3741824801 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c440a0690 con 0x7f8c440ce9c0 2024-04-25T09:04:53.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.203+0000 7f8c49552640 1 --2- 172.21.15.43:0/3741824801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c440ce9c0 0x7f8c4414fec0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f8c3402a220 tx=0x7f8c340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:53.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.203+0000 7f8c3a7fc640 1 -- 172.21.15.43:0/3741824801 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c34041020 con 0x7f8c440ce9c0 2024-04-25T09:04:53.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.203+0000 7f8c3a7fc640 1 -- 172.21.15.43:0/3741824801 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c340306c0 con 0x7f8c440ce9c0 2024-04-25T09:04:53.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.203+0000 7f8c3a7fc640 1 -- 172.21.15.43:0/3741824801 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c34037040 con 0x7f8c440ce9c0 2024-04-25T09:04:53.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.203+0000 7f8c4bfb7640 1 -- 172.21.15.43:0/3741824801 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c440a0970 con 0x7f8c440ce9c0 2024-04-25T09:04:53.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.203+0000 7f8c4bfb7640 1 -- 172.21.15.43:0/3741824801 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c440a0f30 con 0x7f8c440ce9c0 2024-04-25T09:04:53.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.204+0000 7f8c4bfb7640 1 -- 172.21.15.43:0/3741824801 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c440680f0 con 0x7f8c440ce9c0 2024-04-25T09:04:53.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.205+0000 7f8c3a7fc640 1 -- 172.21.15.43:0/3741824801 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8c34044020 con 0x7f8c440ce9c0 2024-04-25T09:04:53.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.205+0000 7f8c3a7fc640 1 --2- 172.21.15.43:0/3741824801 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8c2806b500 0x7f8c2806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.205+0000 7f8c3a7fc640 1 -- 172.21.15.43:0/3741824801 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8c340aa500 con 0x7f8c440ce9c0 2024-04-25T09:04:53.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.205+0000 7f8c49d53640 1 --2- 172.21.15.43:0/3741824801 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8c2806b500 0x7f8c2806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.206+0000 7f8c49d53640 1 --2- 172.21.15.43:0/3741824801 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8c2806b500 0x7f8c2806d9c0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f8c40006e80 tx=0x7f8c40008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:53.198 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.209+0000 7f8c3a7fc640 1 -- 172.21.15.43:0/3741824801 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8c3404b030 con 0x7f8c440ce9c0 2024-04-25T09:04:53.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.388+0000 7f8c4bfb7640 1 -- 172.21.15.43:0/3741824801 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c440a0430 con 0x7f8c2806b500 2024-04-25T09:04:53.377 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.390+0000 7f8c3a7fc640 1 -- 172.21.15.43:0/3741824801 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8c40004030 con 0x7f8c2806b500 2024-04-25T09:04:53.377 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T09:04:53.379 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.391+0000 7f8c4bfb7640 1 -- 172.21.15.43:0/3741824801 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8c2806b500 msgr2=0x7f8c2806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.391+0000 7f8c4bfb7640 1 --2- 172.21.15.43:0/3741824801 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8c2806b500 0x7f8c2806d9c0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f8c40006e80 tx=0x7f8c40008040 comp rx=0 tx=0).stop 2024-04-25T09:04:53.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.391+0000 7f8c4bfb7640 1 -- 172.21.15.43:0/3741824801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c440ce9c0 msgr2=0x7f8c4414fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.391+0000 7f8c4bfb7640 1 --2- 172.21.15.43:0/3741824801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c440ce9c0 0x7f8c4414fec0 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f8c3402a220 tx=0x7f8c340046c0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.391+0000 7f8c4bfb7640 1 -- 172.21.15.43:0/3741824801 shutdown_connections 2024-04-25T09:04:53.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.391+0000 7f8c4bfb7640 1 --2- 172.21.15.43:0/3741824801 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8c2806b500 0x7f8c2806d9c0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.391+0000 7f8c4bfb7640 1 --2- 172.21.15.43:0/3741824801 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8c4415dce0 0x7f8c4409fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.380 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.391+0000 7f8c4bfb7640 1 --2- 172.21.15.43:0/3741824801 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8c440ce9c0 0x7f8c4414fec0 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.391+0000 7f8c4bfb7640 1 --2- 172.21.15.43:0/3741824801 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8c440ce550 0x7f8c4409db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.391+0000 7f8c4bfb7640 1 -- 172.21.15.43:0/3741824801 >> 172.21.15.43:0/3741824801 conn(0x7f8c4409cd00 msgr2=0x7f8c4406c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:53.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.391+0000 7f8c4bfb7640 1 -- 172.21.15.43:0/3741824801 shutdown_connections 2024-04-25T09:04:53.381 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.391+0000 7f8c4bfb7640 1 -- 172.21.15.43:0/3741824801 wait complete. 2024-04-25T09:04:53.390 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T09:04:53.391 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T09:04:53.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.507+0000 7ff781d42640 1 Processor -- start 2024-04-25T09:04:53.495 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.507+0000 7ff781d42640 1 -- start start 2024-04-25T09:04:53.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.508+0000 7ff781d42640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff77c0d3e70 0x7ff77c15e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.508+0000 7ff781d42640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff77c0b37a0 con 0x7ff77c15baa0 2024-04-25T09:04:53.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.508+0000 7ff781d42640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff77c15eab0 con 0x7ff77c15b6c0 2024-04-25T09:04:53.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.508+0000 7ff781d42640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff77c15fce0 con 0x7ff77c0d3e70 2024-04-25T09:04:53.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.508+0000 7ff77affd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff77c15baa0 0x7ff77c0d3760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56542/0 (socket says 172.21.15.43:56542) 2024-04-25T09:04:53.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.508+0000 7ff77affd640 1 -- 172.21.15.43:0/2675560444 learned_addr learned my addr 172.21.15.43:0/2675560444 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:53.497 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.508+0000 7ff77bfff640 1 --2- 172.21.15.43:0/2675560444 >> v2:172.21.15.193:3300/0 conn(0x7ff77c0d3e70 0x7ff77c15e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.508+0000 7ff77a7fc640 1 -- 172.21.15.43:0/2675560444 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1939914891 0 0) 0x7ff77c0b37a0 con 0x7ff77c15baa0 2024-04-25T09:04:53.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.508+0000 7ff77a7fc640 1 -- 172.21.15.43:0/2675560444 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff764003520 con 0x7ff77c15baa0 2024-04-25T09:04:53.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.508+0000 7ff77bfff640 1 -- 172.21.15.43:0/2675560444 >> v1:172.21.15.193:6789/0 conn(0x7ff77c15b6c0 legacy=0x7ff77c15f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.508+0000 7ff77bfff640 1 -- 172.21.15.43:0/2675560444 >> v1:172.21.15.43:6789/0 conn(0x7ff77c15baa0 legacy=0x7ff77c0d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.508+0000 7ff77a7fc640 1 -- 172.21.15.43:0/2675560444 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3716869832 0 0) 0x7ff77c15eab0 con 0x7ff77c15b6c0 2024-04-25T09:04:53.498 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.508+0000 7ff77bfff640 1 -- 172.21.15.43:0/2675560444 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff77c15ed90 con 0x7ff77c0d3e70 2024-04-25T09:04:53.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.509+0000 7ff77bfff640 1 --2- 172.21.15.43:0/2675560444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff77c0d3e70 0x7ff77c15e410 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ff774009d90 tx=0x7ff77402a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6548d2938ad00946 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:53.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.509+0000 7ff77a7fc640 1 -- 172.21.15.43:0/2675560444 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff774007740 con 0x7ff77c0d3e70 2024-04-25T09:04:53.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.509+0000 7ff77a7fc640 1 -- 172.21.15.43:0/2675560444 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7740078e0 con 0x7ff77c0d3e70 2024-04-25T09:04:53.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.509+0000 7ff77a7fc640 1 -- 172.21.15.43:0/2675560444 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff774007cb0 con 0x7ff77c0d3e70 2024-04-25T09:04:53.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.510+0000 7ff781d42640 1 -- 172.21.15.43:0/2675560444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff77c0d3e70 msgr2=0x7ff77c15e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.510+0000 7ff781d42640 1 --2- 172.21.15.43:0/2675560444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff77c0d3e70 0x7ff77c15e410 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7ff774009d90 tx=0x7ff77402a6f0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.510+0000 7ff781d42640 1 -- 172.21.15.43:0/2675560444 shutdown_connections 2024-04-25T09:04:53.500 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.510+0000 7ff781d42640 1 --2- 172.21.15.43:0/2675560444 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff77c0d3e70 0x7ff77c15e410 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.510+0000 7ff781d42640 1 -- 172.21.15.43:0/2675560444 >> 172.21.15.43:0/2675560444 conn(0x7ff77c08d460 msgr2=0x7ff77c09cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:53.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.510+0000 7ff781d42640 1 -- 172.21.15.43:0/2675560444 shutdown_connections 2024-04-25T09:04:53.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.510+0000 7ff781d42640 1 -- 172.21.15.43:0/2675560444 wait complete. 2024-04-25T09:04:53.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.511+0000 7ff781d42640 1 Processor -- start 2024-04-25T09:04:53.501 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.511+0000 7ff781d42640 1 -- start start 2024-04-25T09:04:53.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.512+0000 7ff781d42640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff77c0d3e70 0x7ff77c153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.512+0000 7ff781d42640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff77c15b6c0 0x7ff77c0cf390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.512+0000 7ff781d42640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff77c15baa0 0x7ff77c0d21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.512+0000 7ff781d42640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c0b4ea0 con 0x7ff77c15b6c0 2024-04-25T09:04:53.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.512+0000 7ff781d42640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c15f350 con 0x7ff77c0d3e70 2024-04-25T09:04:53.504 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.512+0000 7ff781d42640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff77c0b5020 con 0x7ff77c15baa0 2024-04-25T09:04:53.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.512+0000 7ff77affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff77c15b6c0 0x7ff77c0cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.512+0000 7ff77affd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff77c15b6c0 0x7ff77c0cf390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43508/0 (socket says 172.21.15.43:43508) 2024-04-25T09:04:53.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.512+0000 7ff77affd640 1 -- 172.21.15.43:0/177282225 learned_addr learned my addr 172.21.15.43:0/177282225 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:53.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.512+0000 7ff77b7fe640 1 --2- 172.21.15.43:0/177282225 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff77c0d3e70 0x7ff77c153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.512+0000 7ff77bfff640 1 --2- 172.21.15.43:0/177282225 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff77c15baa0 0x7ff77c0d21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.513+0000 7ff77affd640 1 -- 172.21.15.43:0/177282225 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff77c15baa0 msgr2=0x7ff77c0d21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.513+0000 7ff77affd640 1 --2- 172.21.15.43:0/177282225 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff77c15baa0 0x7ff77c0d21c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.513+0000 7ff77affd640 1 -- 172.21.15.43:0/177282225 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff77c0d3e70 msgr2=0x7ff77c153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.505 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.513+0000 7ff77affd640 1 --2- 172.21.15.43:0/177282225 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff77c0d3e70 0x7ff77c153b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.513+0000 7ff77affd640 1 -- 172.21.15.43:0/177282225 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff77c0cff50 con 0x7ff77c15b6c0 2024-04-25T09:04:53.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.513+0000 7ff77affd640 1 --2- 172.21.15.43:0/177282225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff77c15b6c0 0x7ff77c0cf390 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7ff77000c400 tx=0x7ff77000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:53.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.513+0000 7ff763fff640 1 -- 172.21.15.43:0/177282225 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff770017070 con 0x7ff77c15b6c0 2024-04-25T09:04:53.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.513+0000 7ff763fff640 1 -- 172.21.15.43:0/177282225 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff770005d00 con 0x7ff77c15b6c0 2024-04-25T09:04:53.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.513+0000 7ff781d42640 1 -- 172.21.15.43:0/177282225 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff77c0cfc70 con 0x7ff77c15b6c0 2024-04-25T09:04:53.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.513+0000 7ff763fff640 1 -- 172.21.15.43:0/177282225 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff770007310 con 0x7ff77c15b6c0 2024-04-25T09:04:53.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.513+0000 7ff781d42640 1 -- 172.21.15.43:0/177282225 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff77c0caf70 con 0x7ff77c15b6c0 2024-04-25T09:04:53.506 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.514+0000 7ff781d42640 1 -- 172.21.15.43:0/177282225 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff77c068a40 con 0x7ff77c15b6c0 2024-04-25T09:04:53.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.515+0000 7ff763fff640 1 -- 172.21.15.43:0/177282225 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff77001a070 con 0x7ff77c15b6c0 2024-04-25T09:04:53.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.515+0000 7ff763fff640 1 --2- 172.21.15.43:0/177282225 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff76806b4d0 0x7ff76806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.515+0000 7ff763fff640 1 -- 172.21.15.43:0/177282225 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff770089210 con 0x7ff77c15b6c0 2024-04-25T09:04:53.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.515+0000 7ff77b7fe640 1 --2- 172.21.15.43:0/177282225 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff76806b4d0 0x7ff76806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.515+0000 7ff763fff640 1 --2- 172.21.15.43:0/177282225 >> [v2:172.21.15.43:6816/179790819,v1:172.21.15.43:6818/179790819] conn(0x7ff768071390 0x7ff7680737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.515+0000 7ff763fff640 1 -- 172.21.15.43:0/177282225 --> [v2:172.21.15.43:6816/179790819,v1:172.21.15.43:6818/179790819] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff768073e60 con 0x7ff768071390 2024-04-25T09:04:53.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.515+0000 7ff763fff640 1 -- 172.21.15.43:0/177282225 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=142) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff770090050 con 0x7ff77c15b6c0 2024-04-25T09:04:53.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.515+0000 7ff77bfff640 1 --2- 172.21.15.43:0/177282225 >> [v2:172.21.15.43:6816/179790819,v1:172.21.15.43:6818/179790819] conn(0x7ff768071390 0x7ff7680737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.507 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.516+0000 7ff77b7fe640 1 --2- 172.21.15.43:0/177282225 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff76806b4d0 0x7ff76806d990 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ff76c000f60 tx=0x7ff76c009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:53.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.516+0000 7ff77bfff640 1 --2- 172.21.15.43:0/177282225 >> [v2:172.21.15.43:6816/179790819,v1:172.21.15.43:6818/179790819] conn(0x7ff768071390 0x7ff7680737b0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:53.508 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.516+0000 7ff763fff640 1 -- 172.21.15.43:0/177282225 <== osd.1 v2:172.21.15.43:6816/179790819 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff768073e60 con 0x7ff768071390 2024-04-25T09:04:53.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.529+0000 7ff781d42640 1 -- 172.21.15.43:0/177282225 --> [v2:172.21.15.43:6816/179790819,v1:172.21.15.43:6818/179790819] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff77c15df40 con 0x7ff768071390 2024-04-25T09:04:53.518 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.530+0000 7ff763fff640 1 -- 172.21.15.43:0/177282225 <== osd.1 v2:172.21.15.43:6816/179790819 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7ff77c15df40 con 0x7ff768071390 2024-04-25T09:04:53.518 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:04:53.519 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T09:04:53.519 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:04:53.519 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.530+0000 7ff781d42640 1 -- 172.21.15.43:0/177282225 >> [v2:172.21.15.43:6816/179790819,v1:172.21.15.43:6818/179790819] conn(0x7ff768071390 msgr2=0x7ff7680737b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.530+0000 7ff781d42640 1 --2- 172.21.15.43:0/177282225 >> [v2:172.21.15.43:6816/179790819,v1:172.21.15.43:6818/179790819] conn(0x7ff768071390 0x7ff7680737b0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.530+0000 7ff781d42640 1 -- 172.21.15.43:0/177282225 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff76806b4d0 msgr2=0x7ff76806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.530+0000 7ff781d42640 1 --2- 172.21.15.43:0/177282225 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff76806b4d0 0x7ff76806d990 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7ff76c000f60 tx=0x7ff76c009040 comp rx=0 tx=0).stop 2024-04-25T09:04:53.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.530+0000 7ff781d42640 1 -- 172.21.15.43:0/177282225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff77c15b6c0 msgr2=0x7ff77c0cf390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.530+0000 7ff781d42640 1 --2- 172.21.15.43:0/177282225 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff77c15b6c0 0x7ff77c0cf390 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7ff77000c400 tx=0x7ff77000c8c0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.530+0000 7ff77bfff640 1 -- 172.21.15.43:0/177282225 reap_dead start 2024-04-25T09:04:53.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.530+0000 7ff781d42640 1 -- 172.21.15.43:0/177282225 shutdown_connections 2024-04-25T09:04:53.520 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.530+0000 7ff781d42640 1 -- 172.21.15.43:0/177282225 >> 172.21.15.43:0/177282225 conn(0x7ff77c08d460 msgr2=0x7ff77c155e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:53.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.530+0000 7ff781d42640 1 -- 172.21.15.43:0/177282225 shutdown_connections 2024-04-25T09:04:53.521 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.530+0000 7ff781d42640 1 -- 172.21.15.43:0/177282225 wait complete. 2024-04-25T09:04:53.529 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T09:04:53.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.642+0000 7f8e4a23f640 1 Processor -- start 2024-04-25T09:04:53.631 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.642+0000 7f8e4a23f640 1 -- start start 2024-04-25T09:04:53.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.643+0000 7f8e4a23f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8e4415def0 0x7f8e4406be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.643+0000 7f8e4a23f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8e44162560 0x7f8e440b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.643+0000 7f8e4a23f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e4415e2d0 con 0x7f8e441619f0 2024-04-25T09:04:53.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.643+0000 7f8e4a23f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e440d0e90 con 0x7f8e44162560 2024-04-25T09:04:53.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.643+0000 7f8e4a23f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4406c390 con 0x7f8e4415def0 2024-04-25T09:04:53.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.643+0000 7f8e43fff640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8e44162560 0x7f8e440b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.643+0000 7f8e42ffd640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8e441619f0 0x7f8e44161e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56558/0 (socket says 172.21.15.43:56558) 2024-04-25T09:04:53.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.643+0000 7f8e42ffd640 1 -- 172.21.15.43:0/4236127577 learned_addr learned my addr 172.21.15.43:0/4236127577 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:53.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.643+0000 7f8e437fe640 1 --2- 172.21.15.43:0/4236127577 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8e4415def0 0x7f8e4406be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.644+0000 7f8e427fc640 1 -- 172.21.15.43:0/4236127577 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3649216262 0 0) 0x7f8e4415e2d0 con 0x7f8e441619f0 2024-04-25T09:04:53.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.644+0000 7f8e427fc640 1 -- 172.21.15.43:0/4236127577 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e2c003520 con 0x7f8e441619f0 2024-04-25T09:04:53.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.644+0000 7f8e43fff640 1 -- 172.21.15.43:0/4236127577 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8e4415def0 msgr2=0x7f8e4406be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:53.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.644+0000 7f8e43fff640 1 --2- 172.21.15.43:0/4236127577 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8e4415def0 0x7f8e4406be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.644+0000 7f8e43fff640 1 -- 172.21.15.43:0/4236127577 >> v1:172.21.15.43:6789/0 conn(0x7f8e441619f0 legacy=0x7f8e44161e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.644+0000 7f8e43fff640 1 -- 172.21.15.43:0/4236127577 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e440b1020 con 0x7f8e44162560 2024-04-25T09:04:53.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.644+0000 7f8e437fe640 1 --2- 172.21.15.43:0/4236127577 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8e4415def0 0x7f8e4406be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T09:04:53.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.644+0000 7f8e43fff640 1 --2- 172.21.15.43:0/4236127577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e44162560 0x7f8e440b20c0 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c009dc0 tx=0x7f8e3c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e0dcfd9e15b558e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:53.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.645+0000 7f8e427fc640 1 -- 172.21.15.43:0/4236127577 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c02ae60 con 0x7f8e44162560 2024-04-25T09:04:53.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.645+0000 7f8e427fc640 1 -- 172.21.15.43:0/4236127577 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e3c007740 con 0x7f8e44162560 2024-04-25T09:04:53.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.645+0000 7f8e427fc640 1 -- 172.21.15.43:0/4236127577 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c007b10 con 0x7f8e44162560 2024-04-25T09:04:53.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.645+0000 7f8e4a23f640 1 -- 172.21.15.43:0/4236127577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e44162560 msgr2=0x7f8e440b20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.645+0000 7f8e4a23f640 1 --2- 172.21.15.43:0/4236127577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e44162560 0x7f8e440b20c0 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c009dc0 tx=0x7f8e3c02a680 comp rx=0 tx=0).stop 2024-04-25T09:04:53.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.645+0000 7f8e4a23f640 1 -- 172.21.15.43:0/4236127577 shutdown_connections 2024-04-25T09:04:53.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.645+0000 7f8e4a23f640 1 --2- 172.21.15.43:0/4236127577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e44162560 0x7f8e440b20c0 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.645+0000 7f8e4a23f640 1 --2- 172.21.15.43:0/4236127577 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8e4415def0 0x7f8e4406be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.645+0000 7f8e4a23f640 1 -- 172.21.15.43:0/4236127577 >> 172.21.15.43:0/4236127577 conn(0x7f8e440b32e0 msgr2=0x7f8e440b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:53.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.645+0000 7f8e4a23f640 1 -- 172.21.15.43:0/4236127577 shutdown_connections 2024-04-25T09:04:53.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.645+0000 7f8e4a23f640 1 -- 172.21.15.43:0/4236127577 wait complete. 2024-04-25T09:04:53.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.646+0000 7f8e4a23f640 1 Processor -- start 2024-04-25T09:04:53.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.647+0000 7f8e4a23f640 1 -- start start 2024-04-25T09:04:53.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.647+0000 7f8e4a23f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e4415def0 0x7f8e4415a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.647+0000 7f8e4a23f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8e441619f0 0x7f8e440c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.647+0000 7f8e4a23f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e44162560 0x7f8e440b6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.647+0000 7f8e4a23f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e440b18c0 con 0x7f8e4415def0 2024-04-25T09:04:53.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.647+0000 7f8e4a23f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e440b15e0 con 0x7f8e44162560 2024-04-25T09:04:53.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.647+0000 7f8e4a23f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e440b1ba0 con 0x7f8e441619f0 2024-04-25T09:04:53.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.647+0000 7f8e437fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e4415def0 0x7f8e4415a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.647+0000 7f8e437fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e4415def0 0x7f8e4415a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43520/0 (socket says 172.21.15.43:43520) 2024-04-25T09:04:53.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.647+0000 7f8e437fe640 1 -- 172.21.15.43:0/3507931407 learned_addr learned my addr 172.21.15.43:0/3507931407 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:53.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.647+0000 7f8e42ffd640 1 --2- 172.21.15.43:0/3507931407 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8e441619f0 0x7f8e440c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.647+0000 7f8e43fff640 1 --2- 172.21.15.43:0/3507931407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e44162560 0x7f8e440b6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.648+0000 7f8e437fe640 1 -- 172.21.15.43:0/3507931407 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8e441619f0 msgr2=0x7f8e440c74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.648+0000 7f8e437fe640 1 --2- 172.21.15.43:0/3507931407 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8e441619f0 0x7f8e440c74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.648+0000 7f8e437fe640 1 -- 172.21.15.43:0/3507931407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e44162560 msgr2=0x7f8e440b6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.648+0000 7f8e437fe640 1 --2- 172.21.15.43:0/3507931407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e44162560 0x7f8e440b6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.648+0000 7f8e437fe640 1 -- 172.21.15.43:0/3507931407 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e440b5670 con 0x7f8e4415def0 2024-04-25T09:04:53.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.648+0000 7f8e437fe640 1 --2- 172.21.15.43:0/3507931407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e4415def0 0x7f8e4415a6c0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f8e3400e990 tx=0x7f8e3400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:53.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.648+0000 7f8e2bfff640 1 -- 172.21.15.43:0/3507931407 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e34017070 con 0x7f8e4415def0 2024-04-25T09:04:53.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.648+0000 7f8e2bfff640 1 -- 172.21.15.43:0/3507931407 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e3400c9e0 con 0x7f8e4415def0 2024-04-25T09:04:53.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.648+0000 7f8e2bfff640 1 -- 172.21.15.43:0/3507931407 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3400cdb0 con 0x7f8e4415def0 2024-04-25T09:04:53.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.648+0000 7f8e4a23f640 1 -- 172.21.15.43:0/3507931407 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e440b5950 con 0x7f8e4415def0 2024-04-25T09:04:53.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.648+0000 7f8e4a23f640 1 -- 172.21.15.43:0/3507931407 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e440b5f10 con 0x7f8e4415def0 2024-04-25T09:04:53.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.649+0000 7f8e4a23f640 1 -- 172.21.15.43:0/3507931407 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e44074ef0 con 0x7f8e4415def0 2024-04-25T09:04:53.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.650+0000 7f8e2bfff640 1 -- 172.21.15.43:0/3507931407 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8e3401a070 con 0x7f8e4415def0 2024-04-25T09:04:53.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.650+0000 7f8e2bfff640 1 --2- 172.21.15.43:0/3507931407 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8e3006b4d0 0x7f8e3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.650+0000 7f8e2bfff640 1 -- 172.21.15.43:0/3507931407 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8e3408a110 con 0x7f8e4415def0 2024-04-25T09:04:53.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.650+0000 7f8e42ffd640 1 --2- 172.21.15.43:0/3507931407 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8e3006b4d0 0x7f8e3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.651+0000 7f8e42ffd640 1 --2- 172.21.15.43:0/3507931407 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8e3006b4d0 0x7f8e3006d990 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f8e440c7f80 tx=0x7f8e38009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:53.643 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.655+0000 7f8e2bfff640 1 -- 172.21.15.43:0/3507931407 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8e34058bb0 con 0x7f8e4415def0 2024-04-25T09:04:53.826 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.838+0000 7f8e4a23f640 1 -- 172.21.15.43:0/3507931407 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e44070c60 con 0x7f8e3006b4d0 2024-04-25T09:04:53.827 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.839+0000 7f8e2bfff640 1 -- 172.21.15.43:0/3507931407 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8e44070c60 con 0x7f8e3006b4d0 2024-04-25T09:04:53.827 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T09:04:53.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.841+0000 7f8e4a23f640 1 -- 172.21.15.43:0/3507931407 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8e3006b4d0 msgr2=0x7f8e3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.829 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.841+0000 7f8e4a23f640 1 --2- 172.21.15.43:0/3507931407 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8e3006b4d0 0x7f8e3006d990 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f8e440c7f80 tx=0x7f8e38009040 comp rx=0 tx=0).stop 2024-04-25T09:04:53.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.841+0000 7f8e4a23f640 1 -- 172.21.15.43:0/3507931407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e4415def0 msgr2=0x7f8e4415a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.841+0000 7f8e4a23f640 1 --2- 172.21.15.43:0/3507931407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e4415def0 0x7f8e4415a6c0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f8e3400e990 tx=0x7f8e3400ee50 comp rx=0 tx=0).stop 2024-04-25T09:04:53.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.841+0000 7f8e4a23f640 1 -- 172.21.15.43:0/3507931407 shutdown_connections 2024-04-25T09:04:53.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.841+0000 7f8e4a23f640 1 --2- 172.21.15.43:0/3507931407 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8e3006b4d0 0x7f8e3006d990 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.841+0000 7f8e4a23f640 1 --2- 172.21.15.43:0/3507931407 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8e44162560 0x7f8e440b6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.841+0000 7f8e4a23f640 1 --2- 172.21.15.43:0/3507931407 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8e441619f0 0x7f8e440c74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.841+0000 7f8e4a23f640 1 --2- 172.21.15.43:0/3507931407 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8e4415def0 0x7f8e4415a6c0 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.830 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.841+0000 7f8e4a23f640 1 -- 172.21.15.43:0/3507931407 >> 172.21.15.43:0/3507931407 conn(0x7f8e440b32e0 msgr2=0x7f8e4406f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:53.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.841+0000 7f8e4a23f640 1 -- 172.21.15.43:0/3507931407 shutdown_connections 2024-04-25T09:04:53.831 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.841+0000 7f8e4a23f640 1 -- 172.21.15.43:0/3507931407 wait complete. 2024-04-25T09:04:53.840 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T09:04:53.840 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T09:04:53.945 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.957+0000 7f13acc1f640 1 Processor -- start 2024-04-25T09:04:53.946 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.957+0000 7f13acc1f640 1 -- start start 2024-04-25T09:04:53.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.957+0000 7f13acc1f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f13a80b3ff0 0x7f13a80d0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.948 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.959+0000 7f13acc1f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13a80d0fb0 0x7f13a815f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.959+0000 7f13acc1f640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13a80cfcb0 con 0x7f13a815f000 2024-04-25T09:04:53.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.959+0000 7f13acc1f640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13a80cfa80 con 0x7f13a80b3ff0 2024-04-25T09:04:53.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.959+0000 7f13acc1f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a815f3e0 con 0x7f13a80d0fb0 2024-04-25T09:04:53.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.959+0000 7f13a5d9b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f13a80b3ff0 0x7f13a80d0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.959+0000 7f13a659c640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f13a815f000 0x7f13a80cbed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56564/0 (socket says 172.21.15.43:56564) 2024-04-25T09:04:53.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.959+0000 7f13a659c640 1 -- 172.21.15.43:0/1155680606 learned_addr learned my addr 172.21.15.43:0/1155680606 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:53.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.959+0000 7f13a6d9d640 1 --2- 172.21.15.43:0/1155680606 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13a80d0fb0 0x7f13a815f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.959+0000 7f13a5d9b640 1 -- 172.21.15.43:0/1155680606 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13a80d0fb0 msgr2=0x7f13a815f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:53.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.959+0000 7f13a5d9b640 1 --2- 172.21.15.43:0/1155680606 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13a80d0fb0 0x7f13a815f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.959+0000 7f13a5d9b640 1 -- 172.21.15.43:0/1155680606 >> v1:172.21.15.43:6789/0 conn(0x7f13a815f000 legacy=0x7f13a80cbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.959+0000 7f13a5d9b640 1 -- 172.21.15.43:0/1155680606 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a80cff90 con 0x7f13a80b3ff0 2024-04-25T09:04:53.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.959+0000 7f13a559a640 1 -- 172.21.15.43:0/1155680606 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3596093831 0 0) 0x7f13a80cfcb0 con 0x7f13a815f000 2024-04-25T09:04:53.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.959+0000 7f13a5d9b640 1 --2- 172.21.15.43:0/1155680606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a80b3ff0 0x7f13a80d0a70 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f139c0097a0 tx=0x7f139c02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ebda08b2bf99513 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:53.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.959+0000 7f13a559a640 1 -- 172.21.15.43:0/1155680606 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139c037070 con 0x7f13a80b3ff0 2024-04-25T09:04:53.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.959+0000 7f13a559a640 1 -- 172.21.15.43:0/1155680606 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f139c02ae50 con 0x7f13a80b3ff0 2024-04-25T09:04:53.950 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.959+0000 7f13a559a640 1 -- 172.21.15.43:0/1155680606 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f139c032660 con 0x7f13a80b3ff0 2024-04-25T09:04:53.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.959+0000 7f13acc1f640 1 -- 172.21.15.43:0/1155680606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a80b3ff0 msgr2=0x7f13a80d0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.959+0000 7f13acc1f640 1 --2- 172.21.15.43:0/1155680606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a80b3ff0 0x7f13a80d0a70 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f139c0097a0 tx=0x7f139c02a100 comp rx=0 tx=0).stop 2024-04-25T09:04:53.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.960+0000 7f13acc1f640 1 -- 172.21.15.43:0/1155680606 shutdown_connections 2024-04-25T09:04:53.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.960+0000 7f13acc1f640 1 --2- 172.21.15.43:0/1155680606 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13a80d0fb0 0x7f13a815f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.960+0000 7f13acc1f640 1 --2- 172.21.15.43:0/1155680606 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a80b3ff0 0x7f13a80d0a70 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.960+0000 7f13acc1f640 1 -- 172.21.15.43:0/1155680606 >> 172.21.15.43:0/1155680606 conn(0x7f13a80b34d0 msgr2=0x7f13a80b38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:53.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.960+0000 7f13acc1f640 1 -- 172.21.15.43:0/1155680606 shutdown_connections 2024-04-25T09:04:53.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.960+0000 7f13acc1f640 1 -- 172.21.15.43:0/1155680606 wait complete. 2024-04-25T09:04:53.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.961+0000 7f13acc1f640 1 Processor -- start 2024-04-25T09:04:53.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.961+0000 7f13acc1f640 1 -- start start 2024-04-25T09:04:53.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.962+0000 7f13acc1f640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13a80b3ff0 0x7f13a81562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.962+0000 7f13acc1f640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13a80d0fb0 0x7f13a8158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.962+0000 7f13acc1f640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a815f000 0x7f13a8150a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.955 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.962+0000 7f13acc1f640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a80cc5e0 con 0x7f13a815f000 2024-04-25T09:04:53.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.962+0000 7f13acc1f640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a80d0550 con 0x7f13a80d0fb0 2024-04-25T09:04:53.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.962+0000 7f13acc1f640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a806b770 con 0x7f13a80b3ff0 2024-04-25T09:04:53.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.962+0000 7f13a6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a815f000 0x7f13a8150a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.962+0000 7f13a6d9d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a815f000 0x7f13a8150a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43536/0 (socket says 172.21.15.43:43536) 2024-04-25T09:04:53.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.962+0000 7f13a6d9d640 1 -- 172.21.15.43:0/252435249 learned_addr learned my addr 172.21.15.43:0/252435249 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:53.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.962+0000 7f13a659c640 1 --2- 172.21.15.43:0/252435249 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13a80b3ff0 0x7f13a81562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.962+0000 7f13a5d9b640 1 --2- 172.21.15.43:0/252435249 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13a80d0fb0 0x7f13a8158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.962+0000 7f13a6d9d640 1 -- 172.21.15.43:0/252435249 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13a80b3ff0 msgr2=0x7f13a81562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.956 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.962+0000 7f13a6d9d640 1 --2- 172.21.15.43:0/252435249 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f13a80b3ff0 0x7f13a81562b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.962+0000 7f13a6d9d640 1 -- 172.21.15.43:0/252435249 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13a80d0fb0 msgr2=0x7f13a8158050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.962+0000 7f13a6d9d640 1 --2- 172.21.15.43:0/252435249 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f13a80d0fb0 0x7f13a8158050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.962+0000 7f13a6d9d640 1 -- 172.21.15.43:0/252435249 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a81567f0 con 0x7f13a815f000 2024-04-25T09:04:53.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.962+0000 7f13a6d9d640 1 --2- 172.21.15.43:0/252435249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a815f000 0x7f13a8150a70 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f13a000efc0 tx=0x7f13a000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:53.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.962+0000 7f1396ffd640 1 -- 172.21.15.43:0/252435249 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a0018020 con 0x7f13a815f000 2024-04-25T09:04:53.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.962+0000 7f1396ffd640 1 -- 172.21.15.43:0/252435249 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a0014480 con 0x7f13a815f000 2024-04-25T09:04:53.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.962+0000 7f13acc1f640 1 -- 172.21.15.43:0/252435249 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13a80c7bd0 con 0x7f13a815f000 2024-04-25T09:04:53.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.963+0000 7f1396ffd640 1 -- 172.21.15.43:0/252435249 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a0014870 con 0x7f13a815f000 2024-04-25T09:04:53.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.963+0000 7f13acc1f640 1 -- 172.21.15.43:0/252435249 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13a80c8190 con 0x7f13a815f000 2024-04-25T09:04:53.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.963+0000 7f13acc1f640 1 -- 172.21.15.43:0/252435249 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f13a8068a40 con 0x7f13a815f000 2024-04-25T09:04:53.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.964+0000 7f1396ffd640 1 -- 172.21.15.43:0/252435249 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f13a0005d10 con 0x7f13a815f000 2024-04-25T09:04:53.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.964+0000 7f1396ffd640 1 --2- 172.21.15.43:0/252435249 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f138806b4d0 0x7f138806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.964+0000 7f1396ffd640 1 -- 172.21.15.43:0/252435249 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f13a008abc0 con 0x7f13a815f000 2024-04-25T09:04:53.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.965+0000 7f13a659c640 1 --2- 172.21.15.43:0/252435249 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f138806b4d0 0x7f138806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.965+0000 7f1396ffd640 1 --2- 172.21.15.43:0/252435249 >> [v2:172.21.15.43:6808/1175991178,v1:172.21.15.43:6809/1175991178] conn(0x7f1388071390 0x7f13880737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:53.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.965+0000 7f1396ffd640 1 -- 172.21.15.43:0/252435249 --> [v2:172.21.15.43:6808/1175991178,v1:172.21.15.43:6809/1175991178] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f13a0012e50 con 0x7f1388071390 2024-04-25T09:04:53.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.965+0000 7f1396ffd640 1 -- 172.21.15.43:0/252435249 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=142) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13a0091020 con 0x7f13a815f000 2024-04-25T09:04:53.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.965+0000 7f13a5d9b640 1 --2- 172.21.15.43:0/252435249 >> [v2:172.21.15.43:6808/1175991178,v1:172.21.15.43:6809/1175991178] conn(0x7f1388071390 0x7f13880737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:53.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.965+0000 7f13a659c640 1 --2- 172.21.15.43:0/252435249 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f138806b4d0 0x7f138806d990 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f1398007fd0 tx=0x7f1398007de0 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:53.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.965+0000 7f13a5d9b640 1 --2- 172.21.15.43:0/252435249 >> [v2:172.21.15.43:6808/1175991178,v1:172.21.15.43:6809/1175991178] conn(0x7f1388071390 0x7f13880737b0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:53.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.966+0000 7f1396ffd640 1 -- 172.21.15.43:0/252435249 <== osd.2 v2:172.21.15.43:6808/1175991178 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f13a0012e50 con 0x7f1388071390 2024-04-25T09:04:53.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.979+0000 7f13acc1f640 1 -- 172.21.15.43:0/252435249 --> [v2:172.21.15.43:6808/1175991178,v1:172.21.15.43:6809/1175991178] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f13a80cd2f0 con 0x7f1388071390 2024-04-25T09:04:53.967 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.979+0000 7f1396ffd640 1 -- 172.21.15.43:0/252435249 <== osd.2 v2:172.21.15.43:6808/1175991178 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f13a80cd2f0 con 0x7f1388071390 2024-04-25T09:04:53.968 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:04:53.968 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T09:04:53.968 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:04:53.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.979+0000 7f13acc1f640 1 -- 172.21.15.43:0/252435249 >> [v2:172.21.15.43:6808/1175991178,v1:172.21.15.43:6809/1175991178] conn(0x7f1388071390 msgr2=0x7f13880737b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.979+0000 7f13acc1f640 1 --2- 172.21.15.43:0/252435249 >> [v2:172.21.15.43:6808/1175991178,v1:172.21.15.43:6809/1175991178] conn(0x7f1388071390 0x7f13880737b0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.979+0000 7f13acc1f640 1 -- 172.21.15.43:0/252435249 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f138806b4d0 msgr2=0x7f138806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.979+0000 7f13acc1f640 1 --2- 172.21.15.43:0/252435249 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f138806b4d0 0x7f138806d990 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f1398007fd0 tx=0x7f1398007de0 comp rx=0 tx=0).stop 2024-04-25T09:04:53.969 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.979+0000 7f13acc1f640 1 -- 172.21.15.43:0/252435249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a815f000 msgr2=0x7f13a8150a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:53.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.979+0000 7f13acc1f640 1 --2- 172.21.15.43:0/252435249 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f13a815f000 0x7f13a8150a70 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f13a000efc0 tx=0x7f13a000c490 comp rx=0 tx=0).stop 2024-04-25T09:04:53.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.979+0000 7f13a6d9d640 1 -- 172.21.15.43:0/252435249 reap_dead start 2024-04-25T09:04:53.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.980+0000 7f13acc1f640 1 -- 172.21.15.43:0/252435249 shutdown_connections 2024-04-25T09:04:53.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.980+0000 7f13acc1f640 1 -- 172.21.15.43:0/252435249 >> 172.21.15.43:0/252435249 conn(0x7f13a80b34d0 msgr2=0x7f13a80b38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:53.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.980+0000 7f13acc1f640 1 -- 172.21.15.43:0/252435249 shutdown_connections 2024-04-25T09:04:53.970 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:53.980+0000 7f13acc1f640 1 -- 172.21.15.43:0/252435249 wait complete. 2024-04-25T09:04:53.979 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T09:04:54.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.093+0000 7fa39ea6b640 1 Processor -- start 2024-04-25T09:04:54.081 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.093+0000 7fa39ea6b640 1 -- start start 2024-04-25T09:04:54.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.093+0000 7fa39ea6b640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fa3980c8d00 0x7fa398162100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.093+0000 7fa39ea6b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa3980ce660 0x7fa3980ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.093+0000 7fa39ea6b640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3980d24a0 con 0x7fa39815df50 2024-04-25T09:04:54.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.093+0000 7fa39ea6b640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa39815e330 con 0x7fa3980c8d00 2024-04-25T09:04:54.083 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.093+0000 7fa39ea6b640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3980d2740 con 0x7fa3980ce660 2024-04-25T09:04:54.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.094+0000 7fa39c807640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa39815df50 0x7fa3981619f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56572/0 (socket says 172.21.15.43:56572) 2024-04-25T09:04:54.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.094+0000 7fa39c807640 1 -- 172.21.15.43:0/4146262909 learned_addr learned my addr 172.21.15.43:0/4146262909 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:54.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.094+0000 7fa397fff640 1 --2- 172.21.15.43:0/4146262909 >> v2:172.21.15.43:3300/0 conn(0x7fa3980c8d00 0x7fa398162100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.094+0000 7fa39d008640 1 --2- 172.21.15.43:0/4146262909 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa3980ce660 0x7fa3980ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.094+0000 7fa3977fe640 1 -- 172.21.15.43:0/4146262909 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3946602432 0 0) 0x7fa3980d24a0 con 0x7fa39815df50 2024-04-25T09:04:54.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.094+0000 7fa3977fe640 1 -- 172.21.15.43:0/4146262909 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa384003520 con 0x7fa39815df50 2024-04-25T09:04:54.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.094+0000 7fa397fff640 1 -- 172.21.15.43:0/4146262909 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa3980ce660 msgr2=0x7fa3980ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:54.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.094+0000 7fa397fff640 1 --2- 172.21.15.43:0/4146262909 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa3980ce660 0x7fa3980ceab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.094+0000 7fa397fff640 1 -- 172.21.15.43:0/4146262909 >> v1:172.21.15.43:6789/0 conn(0x7fa39815df50 legacy=0x7fa3981619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.084 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.094+0000 7fa397fff640 1 -- 172.21.15.43:0/4146262909 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3980d2a20 con 0x7fa3980c8d00 2024-04-25T09:04:54.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.094+0000 7fa39d008640 1 --2- 172.21.15.43:0/4146262909 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa3980ce660 0x7fa3980ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:54.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.095+0000 7fa397fff640 1 --2- 172.21.15.43:0/4146262909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3980c8d00 0x7fa398162100 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7fa3880097a0 tx=0x7fa38802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2272f8ca4a8bef06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:54.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.095+0000 7fa3977fe640 1 -- 172.21.15.43:0/4146262909 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa388037070 con 0x7fa3980c8d00 2024-04-25T09:04:54.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.095+0000 7fa3977fe640 1 -- 172.21.15.43:0/4146262909 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa388032440 con 0x7fa3980c8d00 2024-04-25T09:04:54.085 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.095+0000 7fa3977fe640 1 -- 172.21.15.43:0/4146262909 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3880327f0 con 0x7fa3980c8d00 2024-04-25T09:04:54.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.095+0000 7fa39ea6b640 1 -- 172.21.15.43:0/4146262909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3980c8d00 msgr2=0x7fa398162100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.095+0000 7fa39ea6b640 1 --2- 172.21.15.43:0/4146262909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3980c8d00 0x7fa398162100 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7fa3880097a0 tx=0x7fa38802a180 comp rx=0 tx=0).stop 2024-04-25T09:04:54.088 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.095+0000 7fa39ea6b640 1 -- 172.21.15.43:0/4146262909 shutdown_connections 2024-04-25T09:04:54.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.095+0000 7fa39ea6b640 1 --2- 172.21.15.43:0/4146262909 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa3980ce660 0x7fa3980ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.095+0000 7fa39ea6b640 1 --2- 172.21.15.43:0/4146262909 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3980c8d00 0x7fa398162100 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.095+0000 7fa39ea6b640 1 -- 172.21.15.43:0/4146262909 >> 172.21.15.43:0/4146262909 conn(0x7fa3980d3a30 msgr2=0x7fa3980d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:54.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.095+0000 7fa39ea6b640 1 -- 172.21.15.43:0/4146262909 shutdown_connections 2024-04-25T09:04:54.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.095+0000 7fa39ea6b640 1 -- 172.21.15.43:0/4146262909 wait complete. 2024-04-25T09:04:54.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.097+0000 7fa39ea6b640 1 Processor -- start 2024-04-25T09:04:54.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.097+0000 7fa39ea6b640 1 -- start start 2024-04-25T09:04:54.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.097+0000 7fa39ea6b640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3980c8d00 0x7fa3981f6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.097+0000 7fa39ea6b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3980ce660 0x7fa3980cd300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.089 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.099+0000 7fa39ea6b640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa39815df50 0x7fa3980d0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.099+0000 7fa39ea6b640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3980d2190 con 0x7fa3980ce660 2024-04-25T09:04:54.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.099+0000 7fa39ea6b640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3980c77d0 con 0x7fa3980c8d00 2024-04-25T09:04:54.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.099+0000 7fa39ea6b640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa398162640 con 0x7fa39815df50 2024-04-25T09:04:54.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.099+0000 7fa397fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3980ce660 0x7fa3980cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.099+0000 7fa397fff640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3980ce660 0x7fa3980cd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43562/0 (socket says 172.21.15.43:43562) 2024-04-25T09:04:54.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.099+0000 7fa397fff640 1 -- 172.21.15.43:0/2561941270 learned_addr learned my addr 172.21.15.43:0/2561941270 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:54.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.099+0000 7fa39d008640 1 --2- 172.21.15.43:0/2561941270 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa39815df50 0x7fa3980d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.090 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.099+0000 7fa39c807640 1 --2- 172.21.15.43:0/2561941270 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3980c8d00 0x7fa3981f6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.099+0000 7fa397fff640 1 -- 172.21.15.43:0/2561941270 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa39815df50 msgr2=0x7fa3980d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.099+0000 7fa397fff640 1 --2- 172.21.15.43:0/2561941270 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa39815df50 0x7fa3980d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.099+0000 7fa397fff640 1 -- 172.21.15.43:0/2561941270 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3980c8d00 msgr2=0x7fa3981f6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.099+0000 7fa397fff640 1 --2- 172.21.15.43:0/2561941270 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3980c8d00 0x7fa3981f6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.099+0000 7fa397fff640 1 -- 172.21.15.43:0/2561941270 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa39815c860 con 0x7fa3980ce660 2024-04-25T09:04:54.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.099+0000 7fa39c807640 1 --2- 172.21.15.43:0/2561941270 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3980c8d00 0x7fa3981f6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:54.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.099+0000 7fa397fff640 1 --2- 172.21.15.43:0/2561941270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3980ce660 0x7fa3980cd300 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7fa388035040 tx=0x7fa3880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:54.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.099+0000 7fa3957fa640 1 -- 172.21.15.43:0/2561941270 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa388041020 con 0x7fa3980ce660 2024-04-25T09:04:54.091 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.099+0000 7fa3957fa640 1 -- 172.21.15.43:0/2561941270 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa388030700 con 0x7fa3980ce660 2024-04-25T09:04:54.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.099+0000 7fa39ea6b640 1 -- 172.21.15.43:0/2561941270 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa39815cb40 con 0x7fa3980ce660 2024-04-25T09:04:54.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.099+0000 7fa3957fa640 1 -- 172.21.15.43:0/2561941270 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa388037040 con 0x7fa3980ce660 2024-04-25T09:04:54.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.099+0000 7fa39ea6b640 1 -- 172.21.15.43:0/2561941270 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa39815d100 con 0x7fa3980ce660 2024-04-25T09:04:54.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.099+0000 7fa39ea6b640 1 -- 172.21.15.43:0/2561941270 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3980680f0 con 0x7fa3980ce660 2024-04-25T09:04:54.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.100+0000 7fa3957fa640 1 -- 172.21.15.43:0/2561941270 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa388044020 con 0x7fa3980ce660 2024-04-25T09:04:54.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.100+0000 7fa3957fa640 1 --2- 172.21.15.43:0/2561941270 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa37806b500 0x7fa37806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.100+0000 7fa3957fa640 1 -- 172.21.15.43:0/2561941270 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa3880aa5a0 con 0x7fa3980ce660 2024-04-25T09:04:54.092 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.100+0000 7fa39c807640 1 --2- 172.21.15.43:0/2561941270 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa37806b500 0x7fa37806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.101+0000 7fa39c807640 1 --2- 172.21.15.43:0/2561941270 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa37806b500 0x7fa37806d9c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fa390005c00 tx=0x7fa390009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:54.093 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.105+0000 7fa3957fa640 1 -- 172.21.15.43:0/2561941270 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa38804b030 con 0x7fa3980ce660 2024-04-25T09:04:54.276 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.288+0000 7fa39ea6b640 1 -- 172.21.15.43:0/2561941270 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fa398074ef0 con 0x7fa37806b500 2024-04-25T09:04:54.277 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.290+0000 7fa3957fa640 1 -- 172.21.15.43:0/2561941270 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa398074ef0 con 0x7fa37806b500 2024-04-25T09:04:54.277 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T09:04:54.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.290+0000 7fa39ea6b640 1 -- 172.21.15.43:0/2561941270 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa37806b500 msgr2=0x7fa37806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.290+0000 7fa39ea6b640 1 --2- 172.21.15.43:0/2561941270 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa37806b500 0x7fa37806d9c0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fa390005c00 tx=0x7fa390009040 comp rx=0 tx=0).stop 2024-04-25T09:04:54.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.291+0000 7fa39ea6b640 1 -- 172.21.15.43:0/2561941270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3980ce660 msgr2=0x7fa3980cd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.279 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.291+0000 7fa39ea6b640 1 --2- 172.21.15.43:0/2561941270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3980ce660 0x7fa3980cd300 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7fa388035040 tx=0x7fa3880046c0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.291+0000 7fa39ea6b640 1 -- 172.21.15.43:0/2561941270 shutdown_connections 2024-04-25T09:04:54.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.291+0000 7fa39ea6b640 1 --2- 172.21.15.43:0/2561941270 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa39815df50 0x7fa3980d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.291+0000 7fa39ea6b640 1 --2- 172.21.15.43:0/2561941270 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa3980ce660 0x7fa3980cd300 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.291+0000 7fa39ea6b640 1 --2- 172.21.15.43:0/2561941270 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa37806b500 0x7fa37806d9c0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.291+0000 7fa39ea6b640 1 --2- 172.21.15.43:0/2561941270 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa3980c8d00 0x7fa3981f6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.291+0000 7fa39ea6b640 1 -- 172.21.15.43:0/2561941270 >> 172.21.15.43:0/2561941270 conn(0x7fa3980d3a30 msgr2=0x7fa3980c7f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:54.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.291+0000 7fa39ea6b640 1 -- 172.21.15.43:0/2561941270 shutdown_connections 2024-04-25T09:04:54.280 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.291+0000 7fa39ea6b640 1 -- 172.21.15.43:0/2561941270 wait complete. 2024-04-25T09:04:54.290 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T09:04:54.290 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T09:04:54.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.406+0000 7fa577322640 1 Processor -- start 2024-04-25T09:04:54.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.406+0000 7fa577322640 1 -- start start 2024-04-25T09:04:54.395 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.407+0000 7fa577322640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa57015df20 0x7fa57006c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.407+0000 7fa577322640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa57015f220 con 0x7fa57006c530 2024-04-25T09:04:54.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.407+0000 7fa577322640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa57015e380 con 0x7fa5700b57d0 2024-04-25T09:04:54.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.407+0000 7fa577322640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa57006ce50 con 0x7fa57015df20 2024-04-25T09:04:54.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.407+0000 7fa5750be640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fa57006c530 0x7fa570162270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56580/0 (socket says 172.21.15.43:56580) 2024-04-25T09:04:54.396 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.407+0000 7fa5750be640 1 -- 172.21.15.43:0/1440976839 learned_addr learned my addr 172.21.15.43:0/1440976839 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:54.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.408+0000 7fa56ffff640 1 -- 172.21.15.43:0/1440976839 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1267604237 0 0) 0x7fa57015f220 con 0x7fa57006c530 2024-04-25T09:04:54.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.408+0000 7fa56ffff640 1 -- 172.21.15.43:0/1440976839 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa558003520 con 0x7fa57006c530 2024-04-25T09:04:54.399 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.408+0000 7fa5748bd640 1 --2- 172.21.15.43:0/1440976839 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa57015df20 0x7fa57006c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.408+0000 7fa56ffff640 1 -- 172.21.15.43:0/1440976839 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2798557431 0 0) 0x7fa558003520 con 0x7fa57006c530 2024-04-25T09:04:54.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.408+0000 7fa56ffff640 1 -- 172.21.15.43:0/1440976839 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa57015df20 msgr2=0x7fa57006c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:54.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.408+0000 7fa56ffff640 1 --2- 172.21.15.43:0/1440976839 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa57015df20 0x7fa57006c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.408+0000 7fa56ffff640 1 -- 172.21.15.43:0/1440976839 >> v1:172.21.15.193:6789/0 conn(0x7fa5700b57d0 legacy=0x7fa5700b5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.408+0000 7fa56ffff640 1 -- 172.21.15.43:0/1440976839 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa57015e610 con 0x7fa57006c530 2024-04-25T09:04:54.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.408+0000 7fa56ffff640 1 -- 172.21.15.43:0/1440976839 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7fa568001670 con 0x7fa57006c530 2024-04-25T09:04:54.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.408+0000 7fa56ffff640 1 -- 172.21.15.43:0/1440976839 >> v1:172.21.15.43:6789/0 conn(0x7fa57006c530 legacy=0x7fa570162270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.408+0000 7fa56ffff640 1 --2- 172.21.15.43:0/1440976839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5580039e0 0x7fa558003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.400 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.408+0000 7fa56ffff640 1 --2- 172.21.15.43:0/1440976839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa558004800 0x7fa558004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.408+0000 7fa56ffff640 1 --2- 172.21.15.43:0/1440976839 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa57015df20 0x7fa558005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.408+0000 7fa56ffff640 1 -- 172.21.15.43:0/1440976839 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57006ce50 con 0x7fa558004800 2024-04-25T09:04:54.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.408+0000 7fa56ffff640 1 -- 172.21.15.43:0/1440976839 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa55800baf0 con 0x7fa5580039e0 2024-04-25T09:04:54.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.408+0000 7fa56ffff640 1 -- 172.21.15.43:0/1440976839 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa55800bcf0 con 0x7fa57015df20 2024-04-25T09:04:54.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.408+0000 7fa5748bd640 1 --2- 172.21.15.43:0/1440976839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa558004800 0x7fa558004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.408+0000 7fa5750be640 1 --2- 172.21.15.43:0/1440976839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5580039e0 0x7fa558003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.408+0000 7fa5758bf640 1 --2- 172.21.15.43:0/1440976839 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa57015df20 0x7fa558005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.408+0000 7fa5748bd640 1 -- 172.21.15.43:0/1440976839 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa57015df20 msgr2=0x7fa558005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:54.401 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.409+0000 7fa5748bd640 1 --2- 172.21.15.43:0/1440976839 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa57015df20 0x7fa558005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.409+0000 7fa5748bd640 1 -- 172.21.15.43:0/1440976839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5580039e0 msgr2=0x7fa558003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:54.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.409+0000 7fa5748bd640 1 --2- 172.21.15.43:0/1440976839 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa5580039e0 0x7fa558003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.409+0000 7fa5748bd640 1 -- 172.21.15.43:0/1440976839 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa558003520 con 0x7fa558004800 2024-04-25T09:04:54.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.409+0000 7fa5748bd640 1 --2- 172.21.15.43:0/1440976839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa558004800 0x7fa558004c80 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7fa564009ac0 tx=0x7fa558005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50652207fda74760 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:54.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.409+0000 7fa56ffff640 1 -- 172.21.15.43:0/1440976839 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa564036070 con 0x7fa558004800 2024-04-25T09:04:54.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.409+0000 7fa56ffff640 1 -- 172.21.15.43:0/1440976839 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa564031400 con 0x7fa558004800 2024-04-25T09:04:54.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.409+0000 7fa56ffff640 1 -- 172.21.15.43:0/1440976839 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa564009690 con 0x7fa558004800 2024-04-25T09:04:54.402 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.410+0000 7fa577322640 1 -- 172.21.15.43:0/1440976839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa558004800 msgr2=0x7fa558004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.410+0000 7fa577322640 1 --2- 172.21.15.43:0/1440976839 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa558004800 0x7fa558004c80 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7fa564009ac0 tx=0x7fa558005830 comp rx=0 tx=0).stop 2024-04-25T09:04:54.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.410+0000 7fa5758bf640 1 -- 172.21.15.43:0/1440976839 reap_dead start 2024-04-25T09:04:54.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.410+0000 7fa577322640 1 -- 172.21.15.43:0/1440976839 shutdown_connections 2024-04-25T09:04:54.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.410+0000 7fa577322640 1 -- 172.21.15.43:0/1440976839 >> 172.21.15.43:0/1440976839 conn(0x7fa5700b6ff0 msgr2=0x7fa57009aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:54.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.410+0000 7fa577322640 1 -- 172.21.15.43:0/1440976839 shutdown_connections 2024-04-25T09:04:54.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.410+0000 7fa577322640 1 -- 172.21.15.43:0/1440976839 wait complete. 2024-04-25T09:04:54.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.411+0000 7fa577322640 1 Processor -- start 2024-04-25T09:04:54.403 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.411+0000 7fa577322640 1 -- start start 2024-04-25T09:04:54.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.412+0000 7fa577322640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa57014e2f0 0x7fa5700cb870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.412+0000 7fa577322640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5700b34a0 0x7fa5700b3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.412+0000 7fa577322640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5700cc6d0 0x7fa5700ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.412+0000 7fa577322640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57015eeb0 con 0x7fa5700b34a0 2024-04-25T09:04:54.406 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.412+0000 7fa577322640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57015ebd0 con 0x7fa57014e2f0 2024-04-25T09:04:54.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.412+0000 7fa577322640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa57006d240 con 0x7fa5700cc6d0 2024-04-25T09:04:54.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.412+0000 7fa5748bd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5700b34a0 0x7fa5700b3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.412+0000 7fa5748bd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5700b34a0 0x7fa5700b3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43590/0 (socket says 172.21.15.43:43590) 2024-04-25T09:04:54.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.412+0000 7fa5748bd640 1 -- 172.21.15.43:0/326680163 learned_addr learned my addr 172.21.15.43:0/326680163 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:54.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.412+0000 7fa5748bd640 1 -- 172.21.15.43:0/326680163 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5700cc6d0 msgr2=0x7fa5700ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.412+0000 7fa5748bd640 1 --2- 172.21.15.43:0/326680163 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fa5700cc6d0 0x7fa5700ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.412+0000 7fa5748bd640 1 -- 172.21.15.43:0/326680163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa57014e2f0 msgr2=0x7fa5700cb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.412+0000 7fa5748bd640 1 --2- 172.21.15.43:0/326680163 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fa57014e2f0 0x7fa5700cb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.407 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.412+0000 7fa5748bd640 1 -- 172.21.15.43:0/326680163 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5700cadd0 con 0x7fa5700b34a0 2024-04-25T09:04:54.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.412+0000 7fa5748bd640 1 --2- 172.21.15.43:0/326680163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5700b34a0 0x7fa5700b3900 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7fa564034040 tx=0x7fa564004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:54.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.413+0000 7fa56dffb640 1 -- 172.21.15.43:0/326680163 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa564036050 con 0x7fa5700b34a0 2024-04-25T09:04:54.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.413+0000 7fa56dffb640 1 -- 172.21.15.43:0/326680163 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa564042070 con 0x7fa5700b34a0 2024-04-25T09:04:54.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.413+0000 7fa577322640 1 -- 172.21.15.43:0/326680163 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5700cbdb0 con 0x7fa5700b34a0 2024-04-25T09:04:54.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.413+0000 7fa56dffb640 1 -- 172.21.15.43:0/326680163 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5640044e0 con 0x7fa5700b34a0 2024-04-25T09:04:54.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.413+0000 7fa577322640 1 -- 172.21.15.43:0/326680163 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5700cb070 con 0x7fa5700b34a0 2024-04-25T09:04:54.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.414+0000 7fa577322640 1 -- 172.21.15.43:0/326680163 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa570068a40 con 0x7fa5700b34a0 2024-04-25T09:04:54.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.414+0000 7fa56dffb640 1 -- 172.21.15.43:0/326680163 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa564047460 con 0x7fa5700b34a0 2024-04-25T09:04:54.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.415+0000 7fa56dffb640 1 --2- 172.21.15.43:0/326680163 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa55c06b4d0 0x7fa55c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.408 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.415+0000 7fa56dffb640 1 -- 172.21.15.43:0/326680163 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa5640aa9d0 con 0x7fa5700b34a0 2024-04-25T09:04:54.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.415+0000 7fa56dffb640 1 --2- 172.21.15.43:0/326680163 >> [v2:172.21.15.43:6800/1101549916,v1:172.21.15.43:6801/1101549916] conn(0x7fa55c071390 0x7fa55c0737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.415+0000 7fa56dffb640 1 -- 172.21.15.43:0/326680163 --> [v2:172.21.15.43:6800/1101549916,v1:172.21.15.43:6801/1101549916] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa564001d30 con 0x7fa55c071390 2024-04-25T09:04:54.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.415+0000 7fa56dffb640 1 -- 172.21.15.43:0/326680163 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=142) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa5640af2c0 con 0x7fa5700b34a0 2024-04-25T09:04:54.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.415+0000 7fa5758bf640 1 --2- 172.21.15.43:0/326680163 >> [v2:172.21.15.43:6800/1101549916,v1:172.21.15.43:6801/1101549916] conn(0x7fa55c071390 0x7fa55c0737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.415+0000 7fa5750be640 1 --2- 172.21.15.43:0/326680163 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa55c06b4d0 0x7fa55c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.416+0000 7fa5758bf640 1 --2- 172.21.15.43:0/326680163 >> [v2:172.21.15.43:6800/1101549916,v1:172.21.15.43:6801/1101549916] conn(0x7fa55c071390 0x7fa55c0737b0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:54.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.416+0000 7fa5750be640 1 --2- 172.21.15.43:0/326680163 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa55c06b4d0 0x7fa55c06d990 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fa5680011e0 tx=0x7fa568001170 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:54.409 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.416+0000 7fa56dffb640 1 -- 172.21.15.43:0/326680163 <== osd.3 v2:172.21.15.43:6800/1101549916 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa564001d30 con 0x7fa55c071390 2024-04-25T09:04:54.417 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.429+0000 7fa577322640 1 -- 172.21.15.43:0/326680163 --> [v2:172.21.15.43:6800/1101549916,v1:172.21.15.43:6801/1101549916] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa5700d16b0 con 0x7fa55c071390 2024-04-25T09:04:54.418 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.429+0000 7fa56dffb640 1 -- 172.21.15.43:0/326680163 <== osd.3 v2:172.21.15.43:6800/1101549916 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fa5700d16b0 con 0x7fa55c071390 2024-04-25T09:04:54.418 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:04:54.418 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T09:04:54.418 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:04:54.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.430+0000 7fa577322640 1 -- 172.21.15.43:0/326680163 >> [v2:172.21.15.43:6800/1101549916,v1:172.21.15.43:6801/1101549916] conn(0x7fa55c071390 msgr2=0x7fa55c0737b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.430+0000 7fa577322640 1 --2- 172.21.15.43:0/326680163 >> [v2:172.21.15.43:6800/1101549916,v1:172.21.15.43:6801/1101549916] conn(0x7fa55c071390 0x7fa55c0737b0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.430+0000 7fa577322640 1 -- 172.21.15.43:0/326680163 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa55c06b4d0 msgr2=0x7fa55c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.419 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.430+0000 7fa577322640 1 --2- 172.21.15.43:0/326680163 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fa55c06b4d0 0x7fa55c06d990 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fa5680011e0 tx=0x7fa568001170 comp rx=0 tx=0).stop 2024-04-25T09:04:54.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.430+0000 7fa577322640 1 -- 172.21.15.43:0/326680163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5700b34a0 msgr2=0x7fa5700b3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.430+0000 7fa577322640 1 --2- 172.21.15.43:0/326680163 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fa5700b34a0 0x7fa5700b3900 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7fa564034040 tx=0x7fa564004730 comp rx=0 tx=0).stop 2024-04-25T09:04:54.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.430+0000 7fa5758bf640 1 -- 172.21.15.43:0/326680163 reap_dead start 2024-04-25T09:04:54.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.430+0000 7fa577322640 1 -- 172.21.15.43:0/326680163 shutdown_connections 2024-04-25T09:04:54.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.430+0000 7fa577322640 1 -- 172.21.15.43:0/326680163 >> 172.21.15.43:0/326680163 conn(0x7fa5700b6ff0 msgr2=0x7fa5700ae890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:54.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.430+0000 7fa577322640 1 -- 172.21.15.43:0/326680163 shutdown_connections 2024-04-25T09:04:54.420 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.430+0000 7fa577322640 1 -- 172.21.15.43:0/326680163 wait complete. 2024-04-25T09:04:54.429 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T09:04:54.530 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.542+0000 7f8b8594c640 1 Processor -- start 2024-04-25T09:04:54.531 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.542+0000 7f8b8594c640 1 -- start start 2024-04-25T09:04:54.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.543+0000 7f8b8594c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f8b8006f0e0 0x7f8b800b3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.543+0000 7f8b8594c640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b8006f4c0 con 0x7f8b8006e460 2024-04-25T09:04:54.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.543+0000 7f8b8594c640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b8015f2f0 con 0x7f8b8006f930 2024-04-25T09:04:54.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.543+0000 7f8b8594c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b80072130 con 0x7f8b8006f0e0 2024-04-25T09:04:54.532 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.543+0000 7f8b7f7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f8b8006e460 0x7f8b8015ebe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56588/0 (socket says 172.21.15.43:56588) 2024-04-25T09:04:54.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.543+0000 7f8b7f7fe640 1 -- 172.21.15.43:0/2485794049 learned_addr learned my addr 172.21.15.43:0/2485794049 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:54.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.543+0000 7f8b7effd640 1 --2- 172.21.15.43:0/2485794049 >> v2:172.21.15.43:3300/0 conn(0x7f8b8006f0e0 0x7f8b800b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.544+0000 7f8b7dffb640 1 -- 172.21.15.43:0/2485794049 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3484592028 0 0) 0x7f8b8006f4c0 con 0x7f8b8006e460 2024-04-25T09:04:54.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.544+0000 7f8b7dffb640 1 -- 172.21.15.43:0/2485794049 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b6c003520 con 0x7f8b8006e460 2024-04-25T09:04:54.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.544+0000 7f8b7effd640 1 -- 172.21.15.43:0/2485794049 >> v1:172.21.15.193:6789/0 conn(0x7f8b8006f930 legacy=0x7f8b8015e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.544+0000 7f8b7effd640 1 -- 172.21.15.43:0/2485794049 >> v1:172.21.15.43:6789/0 conn(0x7f8b8006e460 legacy=0x7f8b8015ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.533 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.544+0000 7f8b7effd640 1 -- 172.21.15.43:0/2485794049 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b800b3360 con 0x7f8b8006f0e0 2024-04-25T09:04:54.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.544+0000 7f8b7effd640 1 --2- 172.21.15.43:0/2485794049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b8006f0e0 0x7f8b800b3790 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f8b780097a0 tx=0x7f8b7802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58cbec0904bc3748 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:54.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.545+0000 7f8b7dffb640 1 -- 172.21.15.43:0/2485794049 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b78037070 con 0x7f8b8006f0e0 2024-04-25T09:04:54.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.545+0000 7f8b7dffb640 1 -- 172.21.15.43:0/2485794049 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b78032440 con 0x7f8b8006f0e0 2024-04-25T09:04:54.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.545+0000 7f8b7dffb640 1 -- 172.21.15.43:0/2485794049 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b78032810 con 0x7f8b8006f0e0 2024-04-25T09:04:54.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.545+0000 7f8b8594c640 1 -- 172.21.15.43:0/2485794049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b8006f0e0 msgr2=0x7f8b800b3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.545+0000 7f8b8594c640 1 --2- 172.21.15.43:0/2485794049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b8006f0e0 0x7f8b800b3790 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f8b780097a0 tx=0x7f8b7802a230 comp rx=0 tx=0).stop 2024-04-25T09:04:54.535 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.545+0000 7f8b8594c640 1 -- 172.21.15.43:0/2485794049 shutdown_connections 2024-04-25T09:04:54.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.545+0000 7f8b8594c640 1 --2- 172.21.15.43:0/2485794049 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b8006f0e0 0x7f8b800b3790 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.545+0000 7f8b8594c640 1 -- 172.21.15.43:0/2485794049 >> 172.21.15.43:0/2485794049 conn(0x7f8b800b2490 msgr2=0x7f8b800b28a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:54.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.545+0000 7f8b8594c640 1 -- 172.21.15.43:0/2485794049 shutdown_connections 2024-04-25T09:04:54.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.545+0000 7f8b8594c640 1 -- 172.21.15.43:0/2485794049 wait complete. 2024-04-25T09:04:54.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.546+0000 7f8b8594c640 1 Processor -- start 2024-04-25T09:04:54.536 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.547+0000 7f8b8594c640 1 -- start start 2024-04-25T09:04:54.538 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.547+0000 7f8b8594c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b8006e460 0x7f8b800b0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.547+0000 7f8b8594c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8b8006f0e0 0x7f8b80152d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.547+0000 7f8b8594c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b8006f930 0x7f8b8014ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.547+0000 7f8b8594c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b80071ae0 con 0x7f8b8006f930 2024-04-25T09:04:54.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.547+0000 7f8b8594c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b80071800 con 0x7f8b8006e460 2024-04-25T09:04:54.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.547+0000 7f8b8594c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b80071dc0 con 0x7f8b8006f0e0 2024-04-25T09:04:54.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.547+0000 7f8b7f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b8006f930 0x7f8b8014ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.547+0000 7f8b7f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b8006f930 0x7f8b8014ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43606/0 (socket says 172.21.15.43:43606) 2024-04-25T09:04:54.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.547+0000 7f8b7f7fe640 1 -- 172.21.15.43:0/3756137866 learned_addr learned my addr 172.21.15.43:0/3756137866 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:54.539 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.548+0000 7f8b7effd640 1 --2- 172.21.15.43:0/3756137866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b8006e460 0x7f8b800b0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.548+0000 7f8b7e7fc640 1 --2- 172.21.15.43:0/3756137866 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8b8006f0e0 0x7f8b80152d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.548+0000 7f8b7f7fe640 1 -- 172.21.15.43:0/3756137866 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8b8006f0e0 msgr2=0x7f8b80152d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.548+0000 7f8b7f7fe640 1 --2- 172.21.15.43:0/3756137866 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8b8006f0e0 0x7f8b80152d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.548+0000 7f8b7f7fe640 1 -- 172.21.15.43:0/3756137866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b8006e460 msgr2=0x7f8b800b0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.548+0000 7f8b7f7fe640 1 --2- 172.21.15.43:0/3756137866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b8006e460 0x7f8b800b0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.548+0000 7f8b7f7fe640 1 -- 172.21.15.43:0/3756137866 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b801f3b40 con 0x7f8b8006f930 2024-04-25T09:04:54.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.548+0000 7f8b7f7fe640 1 --2- 172.21.15.43:0/3756137866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b8006f930 0x7f8b8014ac50 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f8b7000caa0 tx=0x7f8b7000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:54.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.548+0000 7f8b77fff640 1 -- 172.21.15.43:0/3756137866 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b70017070 con 0x7f8b8006f930 2024-04-25T09:04:54.540 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.548+0000 7f8b77fff640 1 -- 172.21.15.43:0/3756137866 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b7000ac00 con 0x7f8b8006f930 2024-04-25T09:04:54.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.548+0000 7f8b8594c640 1 -- 172.21.15.43:0/3756137866 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b801f3e20 con 0x7f8b8006f930 2024-04-25T09:04:54.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.548+0000 7f8b77fff640 1 -- 172.21.15.43:0/3756137866 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b70012400 con 0x7f8b8006f930 2024-04-25T09:04:54.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.548+0000 7f8b8594c640 1 -- 172.21.15.43:0/3756137866 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b801f43e0 con 0x7f8b8006f930 2024-04-25T09:04:54.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.549+0000 7f8b8594c640 1 -- 172.21.15.43:0/3756137866 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b80074ef0 con 0x7f8b8006f930 2024-04-25T09:04:54.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.550+0000 7f8b77fff640 1 -- 172.21.15.43:0/3756137866 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8b7001a070 con 0x7f8b8006f930 2024-04-25T09:04:54.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.550+0000 7f8b77fff640 1 --2- 172.21.15.43:0/3756137866 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8b6006b4d0 0x7f8b6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.550+0000 7f8b77fff640 1 -- 172.21.15.43:0/3756137866 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8b70089ba0 con 0x7f8b8006f930 2024-04-25T09:04:54.541 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.550+0000 7f8b7effd640 1 --2- 172.21.15.43:0/3756137866 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8b6006b4d0 0x7f8b6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.551+0000 7f8b7effd640 1 --2- 172.21.15.43:0/3756137866 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8b6006b4d0 0x7f8b6006d990 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f8b78034000 tx=0x7f8b7802e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:54.542 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.554+0000 7f8b77fff640 1 -- 172.21.15.43:0/3756137866 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8b70058760 con 0x7f8b8006f930 2024-04-25T09:04:54.724 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.737+0000 7f8b8594c640 1 -- 172.21.15.43:0/3756137866 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b80157840 con 0x7f8b6006b4d0 2024-04-25T09:04:54.725 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.737+0000 7f8b77fff640 1 -- 172.21.15.43:0/3756137866 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8b80157840 con 0x7f8b6006b4d0 2024-04-25T09:04:54.725 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T09:04:54.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.739+0000 7f8b8594c640 1 -- 172.21.15.43:0/3756137866 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8b6006b4d0 msgr2=0x7f8b6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.727 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.739+0000 7f8b8594c640 1 --2- 172.21.15.43:0/3756137866 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8b6006b4d0 0x7f8b6006d990 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f8b78034000 tx=0x7f8b7802e000 comp rx=0 tx=0).stop 2024-04-25T09:04:54.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.740+0000 7f8b8594c640 1 -- 172.21.15.43:0/3756137866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b8006f930 msgr2=0x7f8b8014ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.740+0000 7f8b8594c640 1 --2- 172.21.15.43:0/3756137866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b8006f930 0x7f8b8014ac50 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f8b7000caa0 tx=0x7f8b7000cf60 comp rx=0 tx=0).stop 2024-04-25T09:04:54.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.740+0000 7f8b8594c640 1 -- 172.21.15.43:0/3756137866 shutdown_connections 2024-04-25T09:04:54.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.740+0000 7f8b8594c640 1 --2- 172.21.15.43:0/3756137866 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f8b8006f930 0x7f8b8014ac50 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.740+0000 7f8b8594c640 1 --2- 172.21.15.43:0/3756137866 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f8b8006f0e0 0x7f8b80152d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.740+0000 7f8b8594c640 1 --2- 172.21.15.43:0/3756137866 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f8b6006b4d0 0x7f8b6006d990 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.740+0000 7f8b8594c640 1 --2- 172.21.15.43:0/3756137866 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f8b8006e460 0x7f8b800b0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.740+0000 7f8b8594c640 1 -- 172.21.15.43:0/3756137866 >> 172.21.15.43:0/3756137866 conn(0x7f8b800b2490 msgr2=0x7f8b8015a540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:54.728 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.740+0000 7f8b8594c640 1 -- 172.21.15.43:0/3756137866 shutdown_connections 2024-04-25T09:04:54.729 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.740+0000 7f8b8594c640 1 -- 172.21.15.43:0/3756137866 wait complete. 2024-04-25T09:04:54.738 INFO:tasks.ceph:Scrubbing osd.4 2024-04-25T09:04:54.738 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-25T09:04:54.843 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.856+0000 7f35cfb7e640 1 Processor -- start 2024-04-25T09:04:54.844 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.856+0000 7f35cfb7e640 1 -- start start 2024-04-25T09:04:54.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.857+0000 7f35cfb7e640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f35c815db50 0x7f35c809ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.857+0000 7f35cfb7e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35c80b0cf0 0x7f35c806c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.857+0000 7f35cfb7e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35c80b5ce0 con 0x7f35c806cb00 2024-04-25T09:04:54.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.857+0000 7f35cfb7e640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35c806cf70 con 0x7f35c815db50 2024-04-25T09:04:54.845 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.857+0000 7f35cfb7e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35c80b1540 con 0x7f35c80b0cf0 2024-04-25T09:04:54.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.857+0000 7f35ce11b640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f35c806cb00 0x7f35c81617e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56594/0 (socket says 172.21.15.43:56594) 2024-04-25T09:04:54.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.857+0000 7f35ce11b640 1 -- 172.21.15.43:0/2786962149 learned_addr learned my addr 172.21.15.43:0/2786962149 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:54.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.857+0000 7f35cd91a640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f35c815db50 0x7f35c809ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.857+0000 7f35cd119640 1 --2- 172.21.15.43:0/2786962149 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35c80b0cf0 0x7f35c806c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.858+0000 7f35cd91a640 1 -- 172.21.15.43:0/2786962149 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35c80b0cf0 msgr2=0x7f35c806c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:54.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.858+0000 7f35cd91a640 1 --2- 172.21.15.43:0/2786962149 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35c80b0cf0 0x7f35c806c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.858+0000 7f35cd91a640 1 -- 172.21.15.43:0/2786962149 >> v1:172.21.15.43:6789/0 conn(0x7f35c806cb00 legacy=0x7f35c81617e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.846 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.858+0000 7f35cd91a640 1 -- 172.21.15.43:0/2786962149 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c80b5f80 con 0x7f35c815db50 2024-04-25T09:04:54.849 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.858+0000 7f35cd91a640 1 --2- 172.21.15.43:0/2786962149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35c815db50 0x7f35c809ae80 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f35c0009d90 tx=0x7f35c002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b50bcd89625c8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:54.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.858+0000 7f35cc918640 1 -- 172.21.15.43:0/2786962149 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c0007740 con 0x7f35c815db50 2024-04-25T09:04:54.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.858+0000 7f35cc918640 1 -- 172.21.15.43:0/2786962149 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c00078e0 con 0x7f35c815db50 2024-04-25T09:04:54.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.858+0000 7f35cc918640 1 -- 172.21.15.43:0/2786962149 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c0007cb0 con 0x7f35c815db50 2024-04-25T09:04:54.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.859+0000 7f35cfb7e640 1 -- 172.21.15.43:0/2786962149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35c815db50 msgr2=0x7f35c809ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.859+0000 7f35cfb7e640 1 --2- 172.21.15.43:0/2786962149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35c815db50 0x7f35c809ae80 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f35c0009d90 tx=0x7f35c002a660 comp rx=0 tx=0).stop 2024-04-25T09:04:54.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.859+0000 7f35cfb7e640 1 -- 172.21.15.43:0/2786962149 shutdown_connections 2024-04-25T09:04:54.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.859+0000 7f35cfb7e640 1 --2- 172.21.15.43:0/2786962149 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35c80b0cf0 0x7f35c806c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.850 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.859+0000 7f35cfb7e640 1 --2- 172.21.15.43:0/2786962149 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35c815db50 0x7f35c809ae80 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.859+0000 7f35cfb7e640 1 -- 172.21.15.43:0/2786962149 >> 172.21.15.43:0/2786962149 conn(0x7f35c809aa70 msgr2=0x7f35c80b6be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:54.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.859+0000 7f35cfb7e640 1 -- 172.21.15.43:0/2786962149 shutdown_connections 2024-04-25T09:04:54.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.859+0000 7f35cfb7e640 1 -- 172.21.15.43:0/2786962149 wait complete. 2024-04-25T09:04:54.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.860+0000 7f35cfb7e640 1 Processor -- start 2024-04-25T09:04:54.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.860+0000 7f35cfb7e640 1 -- start start 2024-04-25T09:04:54.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.861+0000 7f35cfb7e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35c806cb00 0x7f35c815a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.861+0000 7f35cfb7e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35c80b0cf0 0x7f35c8152ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.851 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.861+0000 7f35cfb7e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35c815db50 0x7f35c815efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.861+0000 7f35cfb7e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c80b11d0 con 0x7f35c815db50 2024-04-25T09:04:54.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.861+0000 7f35cfb7e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c80b6540 con 0x7f35c806cb00 2024-04-25T09:04:54.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.861+0000 7f35cfb7e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35c815df30 con 0x7f35c80b0cf0 2024-04-25T09:04:54.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.861+0000 7f35ce11b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35c815db50 0x7f35c815efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.861+0000 7f35ce11b640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35c815db50 0x7f35c815efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43610/0 (socket says 172.21.15.43:43610) 2024-04-25T09:04:54.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.861+0000 7f35ce11b640 1 -- 172.21.15.43:0/4261582477 learned_addr learned my addr 172.21.15.43:0/4261582477 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:54.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.861+0000 7f35cd119640 1 --2- 172.21.15.43:0/4261582477 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35c80b0cf0 0x7f35c8152ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.861+0000 7f35cd91a640 1 --2- 172.21.15.43:0/4261582477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35c806cb00 0x7f35c815a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.852 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.861+0000 7f35ce11b640 1 -- 172.21.15.43:0/4261582477 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35c80b0cf0 msgr2=0x7f35c8152ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.861+0000 7f35ce11b640 1 --2- 172.21.15.43:0/4261582477 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f35c80b0cf0 0x7f35c8152ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.861+0000 7f35ce11b640 1 -- 172.21.15.43:0/4261582477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35c806cb00 msgr2=0x7f35c815a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.861+0000 7f35ce11b640 1 --2- 172.21.15.43:0/4261582477 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f35c806cb00 0x7f35c815a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.861+0000 7f35ce11b640 1 -- 172.21.15.43:0/4261582477 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35c815ace0 con 0x7f35c815db50 2024-04-25T09:04:54.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.861+0000 7f35ce11b640 1 --2- 172.21.15.43:0/4261582477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35c815db50 0x7f35c815efd0 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f35b800c390 tx=0x7f35b800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:54.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.862+0000 7f35c67fc640 1 -- 172.21.15.43:0/4261582477 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b8017070 con 0x7f35c815db50 2024-04-25T09:04:54.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.862+0000 7f35c67fc640 1 -- 172.21.15.43:0/4261582477 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35b8005ce0 con 0x7f35c815db50 2024-04-25T09:04:54.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.862+0000 7f35cfb7e640 1 -- 172.21.15.43:0/4261582477 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35c815c6d0 con 0x7f35c815db50 2024-04-25T09:04:54.853 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.862+0000 7f35c67fc640 1 -- 172.21.15.43:0/4261582477 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35b8007310 con 0x7f35c815db50 2024-04-25T09:04:54.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.862+0000 7f35cfb7e640 1 -- 172.21.15.43:0/4261582477 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35c815c2f0 con 0x7f35c815db50 2024-04-25T09:04:54.854 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.862+0000 7f35cfb7e640 1 -- 172.21.15.43:0/4261582477 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f35c8068a40 con 0x7f35c815db50 2024-04-25T09:04:54.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.863+0000 7f35c67fc640 1 -- 172.21.15.43:0/4261582477 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f35b801a070 con 0x7f35c815db50 2024-04-25T09:04:54.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.864+0000 7f35c67fc640 1 --2- 172.21.15.43:0/4261582477 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f35b406b4d0 0x7f35b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.864+0000 7f35c67fc640 1 -- 172.21.15.43:0/4261582477 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f35b8089180 con 0x7f35c815db50 2024-04-25T09:04:54.855 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.864+0000 7f35c67fc640 1 --2- 172.21.15.43:0/4261582477 >> [v2:172.21.15.193:6816/77587867,v1:172.21.15.193:6817/77587867] conn(0x7f35b4071390 0x7f35b40737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.864+0000 7f35c67fc640 1 -- 172.21.15.43:0/4261582477 --> [v2:172.21.15.193:6816/77587867,v1:172.21.15.193:6817/77587867] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f35b8010e50 con 0x7f35b4071390 2024-04-25T09:04:54.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.864+0000 7f35c67fc640 1 -- 172.21.15.43:0/4261582477 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=142) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35b8090050 con 0x7f35c815db50 2024-04-25T09:04:54.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.864+0000 7f35cd91a640 1 --2- 172.21.15.43:0/4261582477 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f35b406b4d0 0x7f35b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.864+0000 7f35cd119640 1 --2- 172.21.15.43:0/4261582477 >> [v2:172.21.15.193:6816/77587867,v1:172.21.15.193:6817/77587867] conn(0x7f35b4071390 0x7f35b40737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.864+0000 7f35cd119640 1 --2- 172.21.15.43:0/4261582477 >> [v2:172.21.15.193:6816/77587867,v1:172.21.15.193:6817/77587867] conn(0x7f35b4071390 0x7f35b40737b0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:54.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.864+0000 7f35cd91a640 1 --2- 172.21.15.43:0/4261582477 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f35b406b4d0 0x7f35b406d990 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f35c0034000 tx=0x7f35c002f000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:54.856 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.865+0000 7f35c67fc640 1 -- 172.21.15.43:0/4261582477 <== osd.4 v2:172.21.15.193:6816/77587867 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f35b8010e50 con 0x7f35b4071390 2024-04-25T09:04:54.865 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.878+0000 7f35cfb7e640 1 -- 172.21.15.43:0/4261582477 --> [v2:172.21.15.193:6816/77587867,v1:172.21.15.193:6817/77587867] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f35c815d370 con 0x7f35b4071390 2024-04-25T09:04:54.866 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.879+0000 7f35c67fc640 1 -- 172.21.15.43:0/4261582477 <== osd.4 v2:172.21.15.193:6816/77587867 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f35c815d370 con 0x7f35b4071390 2024-04-25T09:04:54.866 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:04:54.866 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T09:04:54.867 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:04:54.867 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.879+0000 7f35cfb7e640 1 -- 172.21.15.43:0/4261582477 >> [v2:172.21.15.193:6816/77587867,v1:172.21.15.193:6817/77587867] conn(0x7f35b4071390 msgr2=0x7f35b40737b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.879+0000 7f35cfb7e640 1 --2- 172.21.15.43:0/4261582477 >> [v2:172.21.15.193:6816/77587867,v1:172.21.15.193:6817/77587867] conn(0x7f35b4071390 0x7f35b40737b0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.879+0000 7f35cfb7e640 1 -- 172.21.15.43:0/4261582477 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f35b406b4d0 msgr2=0x7f35b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.879+0000 7f35cfb7e640 1 --2- 172.21.15.43:0/4261582477 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f35b406b4d0 0x7f35b406d990 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f35c0034000 tx=0x7f35c002f000 comp rx=0 tx=0).stop 2024-04-25T09:04:54.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.879+0000 7f35cfb7e640 1 -- 172.21.15.43:0/4261582477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35c815db50 msgr2=0x7f35c815efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.879+0000 7f35cfb7e640 1 --2- 172.21.15.43:0/4261582477 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f35c815db50 0x7f35c815efd0 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f35b800c390 tx=0x7f35b800c850 comp rx=0 tx=0).stop 2024-04-25T09:04:54.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.879+0000 7f35ce11b640 1 -- 172.21.15.43:0/4261582477 reap_dead start 2024-04-25T09:04:54.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.879+0000 7f35cfb7e640 1 -- 172.21.15.43:0/4261582477 shutdown_connections 2024-04-25T09:04:54.868 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.879+0000 7f35cfb7e640 1 -- 172.21.15.43:0/4261582477 >> 172.21.15.43:0/4261582477 conn(0x7f35c809aa70 msgr2=0x7f35c80afc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:54.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.879+0000 7f35cfb7e640 1 -- 172.21.15.43:0/4261582477 shutdown_connections 2024-04-25T09:04:54.869 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.879+0000 7f35cfb7e640 1 -- 172.21.15.43:0/4261582477 wait complete. 2024-04-25T09:04:54.877 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-25T09:04:54.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.991+0000 7f02ff2e1640 1 Processor -- start 2024-04-25T09:04:54.979 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.991+0000 7f02ff2e1640 1 -- start start 2024-04-25T09:04:54.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.992+0000 7f02ff2e1640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f02f80d09b0 0x7f02f80b6820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.992+0000 7f02ff2e1640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f02f80ce5e0 0x7f02f80cea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.992+0000 7f02ff2e1640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02f80b4c30 con 0x7f02f8161390 2024-04-25T09:04:54.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.992+0000 7f02ff2e1640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02f80d1910 con 0x7f02f80d09b0 2024-04-25T09:04:54.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.992+0000 7f02ff2e1640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02f80cf2c0 con 0x7f02f80ce5e0 2024-04-25T09:04:54.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.992+0000 7f02fc87c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f02f80d09b0 0x7f02f80b6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.980 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.992+0000 7f02fd07d640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f02f8161390 0x7f02f80d02a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56598/0 (socket says 172.21.15.43:56598) 2024-04-25T09:04:54.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.992+0000 7f02fd07d640 1 -- 172.21.15.43:0/1036967878 learned_addr learned my addr 172.21.15.43:0/1036967878 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:54.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.992+0000 7f02fd87e640 1 --2- 172.21.15.43:0/1036967878 >> v2:172.21.15.193:3300/0 conn(0x7f02f80ce5e0 0x7f02f80cea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.992+0000 7f02fc87c640 1 -- 172.21.15.43:0/1036967878 >> v2:172.21.15.193:3300/0 conn(0x7f02f80ce5e0 msgr2=0x7f02f80cea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:54.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.992+0000 7f02fc87c640 1 --2- 172.21.15.43:0/1036967878 >> v2:172.21.15.193:3300/0 conn(0x7f02f80ce5e0 0x7f02f80cea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.992+0000 7f02fc87c640 1 -- 172.21.15.43:0/1036967878 >> v1:172.21.15.43:6789/0 conn(0x7f02f8161390 legacy=0x7f02f80d02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.993+0000 7f02fc87c640 1 -- 172.21.15.43:0/1036967878 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f80cf5a0 con 0x7f02f80d09b0 2024-04-25T09:04:54.981 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.993+0000 7f02fd87e640 1 --2- 172.21.15.43:0/1036967878 >> v2:172.21.15.193:3300/0 conn(0x7f02f80ce5e0 0x7f02f80cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:54.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.993+0000 7f02fc87c640 1 --2- 172.21.15.43:0/1036967878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02f80d09b0 0x7f02f80b6820 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f02f808dd30 tx=0x7f02e802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99147c953baeec48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:54.984 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.993+0000 7f02effff640 1 -- 172.21.15.43:0/1036967878 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e8037070 con 0x7f02f80d09b0 2024-04-25T09:04:54.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.993+0000 7f02effff640 1 -- 172.21.15.43:0/1036967878 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02e8032440 con 0x7f02f80d09b0 2024-04-25T09:04:54.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.993+0000 7f02effff640 1 -- 172.21.15.43:0/1036967878 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e8032810 con 0x7f02f80d09b0 2024-04-25T09:04:54.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.994+0000 7f02ff2e1640 1 -- 172.21.15.43:0/1036967878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02f80d09b0 msgr2=0x7f02f80b6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.994+0000 7f02ff2e1640 1 --2- 172.21.15.43:0/1036967878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02f80d09b0 0x7f02f80b6820 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f02f808dd30 tx=0x7f02e802a1a0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.994+0000 7f02ff2e1640 1 -- 172.21.15.43:0/1036967878 shutdown_connections 2024-04-25T09:04:54.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.994+0000 7f02ff2e1640 1 --2- 172.21.15.43:0/1036967878 >> v2:172.21.15.193:3300/0 conn(0x7f02f80ce5e0 0x7f02f80cea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.994+0000 7f02ff2e1640 1 --2- 172.21.15.43:0/1036967878 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02f80d09b0 0x7f02f80b6820 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.994+0000 7f02ff2e1640 1 -- 172.21.15.43:0/1036967878 >> 172.21.15.43:0/1036967878 conn(0x7f02f809cd20 msgr2=0x7f02f80d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:54.985 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.994+0000 7f02ff2e1640 1 -- 172.21.15.43:0/1036967878 shutdown_connections 2024-04-25T09:04:54.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.994+0000 7f02ff2e1640 1 -- 172.21.15.43:0/1036967878 wait complete. 2024-04-25T09:04:54.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.995+0000 7f02ff2e1640 1 Processor -- start 2024-04-25T09:04:54.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.995+0000 7f02ff2e1640 1 -- start start 2024-04-25T09:04:54.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.996+0000 7f02ff2e1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02f80ce5e0 0x7f02f81562e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.996+0000 7f02ff2e1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02f80d09b0 0x7f02f815d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.996+0000 7f02ff2e1640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f02f8161390 0x7f02f815ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.996+0000 7f02ff2e1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f8161770 con 0x7f02f80ce5e0 2024-04-25T09:04:54.986 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.996+0000 7f02ff2e1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f80d15a0 con 0x7f02f80d09b0 2024-04-25T09:04:54.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.996+0000 7f02ff2e1640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02f80cef70 con 0x7f02f8161390 2024-04-25T09:04:54.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.996+0000 7f02fd07d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02f80ce5e0 0x7f02f81562e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.996+0000 7f02fd07d640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02f80ce5e0 0x7f02f81562e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43626/0 (socket says 172.21.15.43:43626) 2024-04-25T09:04:54.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.996+0000 7f02fd07d640 1 -- 172.21.15.43:0/3663878170 learned_addr learned my addr 172.21.15.43:0/3663878170 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:54.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.996+0000 7f02fd07d640 1 -- 172.21.15.43:0/3663878170 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f02f8161390 msgr2=0x7f02f815ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.996+0000 7f02fd07d640 1 --2- 172.21.15.43:0/3663878170 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f02f8161390 0x7f02f815ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.996+0000 7f02fd07d640 1 -- 172.21.15.43:0/3663878170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02f80d09b0 msgr2=0x7f02f815d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:54.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.996+0000 7f02fd07d640 1 --2- 172.21.15.43:0/3663878170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02f80d09b0 0x7f02f815d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:54.987 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.996+0000 7f02fd07d640 1 -- 172.21.15.43:0/3663878170 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f80cbd40 con 0x7f02f80ce5e0 2024-04-25T09:04:54.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.996+0000 7f02fd07d640 1 --2- 172.21.15.43:0/3663878170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02f80ce5e0 0x7f02f81562e0 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f02f400c3c0 tx=0x7f02f400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:54.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.997+0000 7f02edffb640 1 -- 172.21.15.43:0/3663878170 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f4017070 con 0x7f02f80ce5e0 2024-04-25T09:04:54.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.997+0000 7f02edffb640 1 -- 172.21.15.43:0/3663878170 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02f4005ce0 con 0x7f02f80ce5e0 2024-04-25T09:04:54.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.997+0000 7f02ff2e1640 1 -- 172.21.15.43:0/3663878170 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02f80c82a0 con 0x7f02f80ce5e0 2024-04-25T09:04:54.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.997+0000 7f02edffb640 1 -- 172.21.15.43:0/3663878170 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f40072f0 con 0x7f02f80ce5e0 2024-04-25T09:04:54.988 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.997+0000 7f02ff2e1640 1 -- 172.21.15.43:0/3663878170 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02f8156820 con 0x7f02f80ce5e0 2024-04-25T09:04:54.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.998+0000 7f02ff2e1640 1 -- 172.21.15.43:0/3663878170 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02f80680f0 con 0x7f02f80ce5e0 2024-04-25T09:04:54.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.998+0000 7f02edffb640 1 -- 172.21.15.43:0/3663878170 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f02f401a070 con 0x7f02f80ce5e0 2024-04-25T09:04:54.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.998+0000 7f02edffb640 1 --2- 172.21.15.43:0/3663878170 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f02dc06b4d0 0x7f02dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:54.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.999+0000 7f02edffb640 1 -- 172.21.15.43:0/3663878170 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f02f40892e0 con 0x7f02f80ce5e0 2024-04-25T09:04:54.989 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.999+0000 7f02fc87c640 1 --2- 172.21.15.43:0/3663878170 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f02dc06b4d0 0x7f02dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:54.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:54.999+0000 7f02fc87c640 1 --2- 172.21.15.43:0/3663878170 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f02dc06b4d0 0x7f02dc06d990 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f02e8034000 tx=0x7f02e802c000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:54.990 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.003+0000 7f02edffb640 1 -- 172.21.15.43:0/3663878170 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f02f80b4c30 con 0x7f02f80ce5e0 2024-04-25T09:04:55.168 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.181+0000 7f02ff2e1640 1 -- 172.21.15.43:0/3663878170 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f02f80d1050 con 0x7f02dc06b4d0 2024-04-25T09:04:55.169 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.181+0000 7f02edffb640 1 -- 172.21.15.43:0/3663878170 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f02f80d1050 con 0x7f02dc06b4d0 2024-04-25T09:04:55.169 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 4 to deep-scrub 2024-04-25T09:04:55.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.183+0000 7f02ff2e1640 1 -- 172.21.15.43:0/3663878170 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f02dc06b4d0 msgr2=0x7f02dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.183+0000 7f02ff2e1640 1 --2- 172.21.15.43:0/3663878170 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f02dc06b4d0 0x7f02dc06d990 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f02e8034000 tx=0x7f02e802c000 comp rx=0 tx=0).stop 2024-04-25T09:04:55.171 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.184+0000 7f02ff2e1640 1 -- 172.21.15.43:0/3663878170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02f80ce5e0 msgr2=0x7f02f81562e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.184+0000 7f02ff2e1640 1 --2- 172.21.15.43:0/3663878170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02f80ce5e0 0x7f02f81562e0 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f02f400c3c0 tx=0x7f02f400c880 comp rx=0 tx=0).stop 2024-04-25T09:04:55.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.184+0000 7f02ff2e1640 1 -- 172.21.15.43:0/3663878170 shutdown_connections 2024-04-25T09:04:55.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.184+0000 7f02ff2e1640 1 --2- 172.21.15.43:0/3663878170 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f02f8161390 0x7f02f815ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.184+0000 7f02ff2e1640 1 --2- 172.21.15.43:0/3663878170 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f02dc06b4d0 0x7f02dc06d990 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.184+0000 7f02ff2e1640 1 --2- 172.21.15.43:0/3663878170 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f02f80d09b0 0x7f02f815d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.184+0000 7f02ff2e1640 1 --2- 172.21.15.43:0/3663878170 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f02f80ce5e0 0x7f02f81562e0 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.184+0000 7f02ff2e1640 1 -- 172.21.15.43:0/3663878170 >> 172.21.15.43:0/3663878170 conn(0x7f02f809cd20 msgr2=0x7f02f80b8420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:55.172 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.184+0000 7f02ff2e1640 1 -- 172.21.15.43:0/3663878170 shutdown_connections 2024-04-25T09:04:55.173 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.184+0000 7f02ff2e1640 1 -- 172.21.15.43:0/3663878170 wait complete. 2024-04-25T09:04:55.182 INFO:tasks.ceph:Scrubbing osd.5 2024-04-25T09:04:55.182 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-25T09:04:55.287 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.300+0000 7ff1db35c640 1 Processor -- start 2024-04-25T09:04:55.288 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.300+0000 7ff1db35c640 1 -- start start 2024-04-25T09:04:55.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.301+0000 7ff1db35c640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7ff1d4158160 0x7ff1d40719f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.301+0000 7ff1db35c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff1d415e370 0x7ff1d415e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.301+0000 7ff1db35c640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1d4070c30 con 0x7ff1d40aeb10 2024-04-25T09:04:55.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.301+0000 7ff1db35c640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1d415f0e0 con 0x7ff1d415e370 2024-04-25T09:04:55.289 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.301+0000 7ff1db35c640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1d4070f10 con 0x7ff1d4158160 2024-04-25T09:04:55.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.301+0000 7ff1d98f9640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff1d415e370 0x7ff1d415e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.301+0000 7ff1d98f9640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff1d415e370 0x7ff1d415e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43628/0 (socket says 172.21.15.43:43628) 2024-04-25T09:04:55.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.301+0000 7ff1d98f9640 1 -- 172.21.15.43:0/488548482 learned_addr learned my addr 172.21.15.43:0/488548482 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:55.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.301+0000 7ff1d90f8640 1 --2- 172.21.15.43:0/488548482 >> v2:172.21.15.193:3300/0 conn(0x7ff1d4158160 0x7ff1d40719f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.302+0000 7ff1d98f9640 1 -- 172.21.15.43:0/488548482 >> v2:172.21.15.193:3300/0 conn(0x7ff1d4158160 msgr2=0x7ff1d40719f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:55.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.302+0000 7ff1d98f9640 1 --2- 172.21.15.43:0/488548482 >> v2:172.21.15.193:3300/0 conn(0x7ff1d4158160 0x7ff1d40719f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.302+0000 7ff1d98f9640 1 -- 172.21.15.43:0/488548482 >> v1:172.21.15.193:6789/0 conn(0x7ff1d40aeb10 legacy=0x7ff1d40aefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.290 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.302+0000 7ff1d98f9640 1 -- 172.21.15.43:0/488548482 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d40711f0 con 0x7ff1d415e370 2024-04-25T09:04:55.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.302+0000 7ff1d98f9640 1 --2- 172.21.15.43:0/488548482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1d415e370 0x7ff1d415e780 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7ff1d40d0dc0 tx=0x7ff1cc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3269952f19dce398 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:55.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.302+0000 7ff1cbfff640 1 -- 172.21.15.43:0/488548482 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc036070 con 0x7ff1d415e370 2024-04-25T09:04:55.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.303+0000 7ff1cbfff640 1 -- 172.21.15.43:0/488548482 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1cc031600 con 0x7ff1d415e370 2024-04-25T09:04:55.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.303+0000 7ff1cbfff640 1 -- 172.21.15.43:0/488548482 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc0319d0 con 0x7ff1d415e370 2024-04-25T09:04:55.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.303+0000 7ff1db35c640 1 -- 172.21.15.43:0/488548482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1d415e370 msgr2=0x7ff1d415e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.292 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.303+0000 7ff1db35c640 1 --2- 172.21.15.43:0/488548482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1d415e370 0x7ff1d415e780 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7ff1d40d0dc0 tx=0x7ff1cc02a6a0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.303+0000 7ff1db35c640 1 -- 172.21.15.43:0/488548482 shutdown_connections 2024-04-25T09:04:55.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.303+0000 7ff1db35c640 1 --2- 172.21.15.43:0/488548482 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1d415e370 0x7ff1d415e780 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.303+0000 7ff1db35c640 1 --2- 172.21.15.43:0/488548482 >> v2:172.21.15.193:3300/0 conn(0x7ff1d4158160 0x7ff1d40719f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.303+0000 7ff1db35c640 1 -- 172.21.15.43:0/488548482 >> 172.21.15.43:0/488548482 conn(0x7ff1d406ab30 msgr2=0x7ff1d406af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:55.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.303+0000 7ff1db35c640 1 -- 172.21.15.43:0/488548482 shutdown_connections 2024-04-25T09:04:55.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.303+0000 7ff1db35c640 1 -- 172.21.15.43:0/488548482 wait complete. 2024-04-25T09:04:55.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.304+0000 7ff1db35c640 1 Processor -- start 2024-04-25T09:04:55.293 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.304+0000 7ff1db35c640 1 -- start start 2024-04-25T09:04:55.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.305+0000 7ff1db35c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1d40aeb10 0x7ff1d4155a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.305+0000 7ff1db35c640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1d4158160 0x7ff1d4159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.296 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.305+0000 7ff1db35c640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff1d415e370 0x7ff1d4158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.305+0000 7ff1db35c640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d415ecc0 con 0x7ff1d40aeb10 2024-04-25T09:04:55.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.305+0000 7ff1db35c640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d4158540 con 0x7ff1d4158160 2024-04-25T09:04:55.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.305+0000 7ff1db35c640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d415ee40 con 0x7ff1d415e370 2024-04-25T09:04:55.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.305+0000 7ff1d90f8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1d40aeb10 0x7ff1d4155a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.305+0000 7ff1d90f8640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1d40aeb10 0x7ff1d4155a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43640/0 (socket says 172.21.15.43:43640) 2024-04-25T09:04:55.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.305+0000 7ff1d90f8640 1 -- 172.21.15.43:0/3340608577 learned_addr learned my addr 172.21.15.43:0/3340608577 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:55.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.305+0000 7ff1d88f7640 1 --2- 172.21.15.43:0/3340608577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1d4158160 0x7ff1d4159ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.297 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.305+0000 7ff1d98f9640 1 --2- 172.21.15.43:0/3340608577 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff1d415e370 0x7ff1d4158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.306+0000 7ff1d90f8640 1 -- 172.21.15.43:0/3340608577 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff1d415e370 msgr2=0x7ff1d4158750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.306+0000 7ff1d90f8640 1 --2- 172.21.15.43:0/3340608577 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff1d415e370 0x7ff1d4158750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.306+0000 7ff1d90f8640 1 -- 172.21.15.43:0/3340608577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1d4158160 msgr2=0x7ff1d4159ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.306+0000 7ff1d90f8640 1 --2- 172.21.15.43:0/3340608577 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff1d4158160 0x7ff1d4159ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.306+0000 7ff1d90f8640 1 -- 172.21.15.43:0/3340608577 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d4158e50 con 0x7ff1d40aeb10 2024-04-25T09:04:55.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.306+0000 7ff1d98f9640 1 --2- 172.21.15.43:0/3340608577 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff1d415e370 0x7ff1d4158750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:55.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.306+0000 7ff1d90f8640 1 --2- 172.21.15.43:0/3340608577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1d40aeb10 0x7ff1d4155a90 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7ff1d000e9b0 tx=0x7ff1d000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:55.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.306+0000 7ff1c9ffb640 1 -- 172.21.15.43:0/3340608577 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0017070 con 0x7ff1d40aeb10 2024-04-25T09:04:55.298 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.306+0000 7ff1c9ffb640 1 -- 172.21.15.43:0/3340608577 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d000c9a0 con 0x7ff1d40aeb10 2024-04-25T09:04:55.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.306+0000 7ff1db35c640 1 -- 172.21.15.43:0/3340608577 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1d4159130 con 0x7ff1d40aeb10 2024-04-25T09:04:55.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.306+0000 7ff1c9ffb640 1 -- 172.21.15.43:0/3340608577 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d000cd90 con 0x7ff1d40aeb10 2024-04-25T09:04:55.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.306+0000 7ff1db35c640 1 -- 172.21.15.43:0/3340608577 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1d41596f0 con 0x7ff1d40aeb10 2024-04-25T09:04:55.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.307+0000 7ff1db35c640 1 -- 172.21.15.43:0/3340608577 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff1d4068a40 con 0x7ff1d40aeb10 2024-04-25T09:04:55.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.307+0000 7ff1c9ffb640 1 -- 172.21.15.43:0/3340608577 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff1d001a070 con 0x7ff1d40aeb10 2024-04-25T09:04:55.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.308+0000 7ff1c9ffb640 1 --2- 172.21.15.43:0/3340608577 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff1b806b4d0 0x7ff1b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.308+0000 7ff1c9ffb640 1 -- 172.21.15.43:0/3340608577 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff1d0089f40 con 0x7ff1d40aeb10 2024-04-25T09:04:55.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.308+0000 7ff1c9ffb640 1 --2- 172.21.15.43:0/3340608577 >> [v2:172.21.15.193:6800/3423021177,v1:172.21.15.193:6801/3423021177] conn(0x7ff1b8071390 0x7ff1b80737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.299 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.308+0000 7ff1c9ffb640 1 -- 172.21.15.43:0/3340608577 --> [v2:172.21.15.193:6800/3423021177,v1:172.21.15.193:6801/3423021177] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff1b8073e60 con 0x7ff1b8071390 2024-04-25T09:04:55.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.308+0000 7ff1c9ffb640 1 -- 172.21.15.43:0/3340608577 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=142) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1d008c5e0 con 0x7ff1d40aeb10 2024-04-25T09:04:55.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.308+0000 7ff1d88f7640 1 --2- 172.21.15.43:0/3340608577 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff1b806b4d0 0x7ff1b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.308+0000 7ff1d98f9640 1 --2- 172.21.15.43:0/3340608577 >> [v2:172.21.15.193:6800/3423021177,v1:172.21.15.193:6801/3423021177] conn(0x7ff1b8071390 0x7ff1b80737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.309+0000 7ff1d88f7640 1 --2- 172.21.15.43:0/3340608577 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff1b806b4d0 0x7ff1b806d990 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4007fd0 tx=0x7ff1c4009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:55.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.309+0000 7ff1d98f9640 1 --2- 172.21.15.43:0/3340608577 >> [v2:172.21.15.193:6800/3423021177,v1:172.21.15.193:6801/3423021177] conn(0x7ff1b8071390 0x7ff1b80737b0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:55.300 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.309+0000 7ff1c9ffb640 1 -- 172.21.15.43:0/3340608577 <== osd.5 v2:172.21.15.193:6800/3423021177 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff1b8073e60 con 0x7ff1b8071390 2024-04-25T09:04:55.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.323+0000 7ff1db35c640 1 -- 172.21.15.43:0/3340608577 --> [v2:172.21.15.193:6800/3423021177,v1:172.21.15.193:6801/3423021177] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff1d4071a90 con 0x7ff1b8071390 2024-04-25T09:04:55.310 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.323+0000 7ff1c9ffb640 1 -- 172.21.15.43:0/3340608577 <== osd.5 v2:172.21.15.193:6800/3423021177 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7ff1d4071a90 con 0x7ff1b8071390 2024-04-25T09:04:55.311 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:04:55.311 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T09:04:55.311 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:04:55.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.323+0000 7ff1db35c640 1 -- 172.21.15.43:0/3340608577 >> [v2:172.21.15.193:6800/3423021177,v1:172.21.15.193:6801/3423021177] conn(0x7ff1b8071390 msgr2=0x7ff1b80737b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.323+0000 7ff1db35c640 1 --2- 172.21.15.43:0/3340608577 >> [v2:172.21.15.193:6800/3423021177,v1:172.21.15.193:6801/3423021177] conn(0x7ff1b8071390 0x7ff1b80737b0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.324+0000 7ff1db35c640 1 -- 172.21.15.43:0/3340608577 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff1b806b4d0 msgr2=0x7ff1b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.324+0000 7ff1db35c640 1 --2- 172.21.15.43:0/3340608577 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff1b806b4d0 0x7ff1b806d990 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4007fd0 tx=0x7ff1c4009040 comp rx=0 tx=0).stop 2024-04-25T09:04:55.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.324+0000 7ff1db35c640 1 -- 172.21.15.43:0/3340608577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1d40aeb10 msgr2=0x7ff1d4155a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.324+0000 7ff1db35c640 1 --2- 172.21.15.43:0/3340608577 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff1d40aeb10 0x7ff1d4155a90 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7ff1d000e9b0 tx=0x7ff1d000ee70 comp rx=0 tx=0).stop 2024-04-25T09:04:55.312 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.324+0000 7ff1d98f9640 1 -- 172.21.15.43:0/3340608577 reap_dead start 2024-04-25T09:04:55.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.324+0000 7ff1db35c640 1 -- 172.21.15.43:0/3340608577 shutdown_connections 2024-04-25T09:04:55.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.324+0000 7ff1db35c640 1 -- 172.21.15.43:0/3340608577 >> 172.21.15.43:0/3340608577 conn(0x7ff1d406ab30 msgr2=0x7ff1d415f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:55.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.324+0000 7ff1db35c640 1 -- 172.21.15.43:0/3340608577 shutdown_connections 2024-04-25T09:04:55.313 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.324+0000 7ff1db35c640 1 -- 172.21.15.43:0/3340608577 wait complete. 2024-04-25T09:04:55.322 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-25T09:04:55.423 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.436+0000 7fbf903b7640 1 Processor -- start 2024-04-25T09:04:55.424 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.436+0000 7fbf903b7640 1 -- start start 2024-04-25T09:04:55.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.437+0000 7fbf903b7640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fbf880d39d0 0x7fbf880d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.437+0000 7fbf903b7640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf880b6c50 con 0x7fbf880ce2a0 2024-04-25T09:04:55.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.437+0000 7fbf903b7640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf8815e010 con 0x7fbf880d2500 2024-04-25T09:04:55.427 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.437+0000 7fbf903b7640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf880d2020 con 0x7fbf880d39d0 2024-04-25T09:04:55.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.437+0000 7fbf8d952640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fbf880ce2a0 0x7fbf880d3db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56614/0 (socket says 172.21.15.43:56614) 2024-04-25T09:04:55.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.437+0000 7fbf8d952640 1 -- 172.21.15.43:0/1477623437 learned_addr learned my addr 172.21.15.43:0/1477623437 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:55.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.438+0000 7fbf8e153640 1 --2- 172.21.15.43:0/1477623437 >> v2:172.21.15.193:3300/0 conn(0x7fbf880d39d0 0x7fbf880d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.438+0000 7fbf8d151640 1 -- 172.21.15.43:0/1477623437 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2612578232 0 0) 0x7fbf7c001110 con 0x7fbf880ce2a0 2024-04-25T09:04:55.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.438+0000 7fbf8d151640 1 -- 172.21.15.43:0/1477623437 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf70003520 con 0x7fbf880ce2a0 2024-04-25T09:04:55.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.438+0000 7fbf8d151640 1 -- 172.21.15.43:0/1477623437 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1698205273 0 0) 0x7fbf70003520 con 0x7fbf880ce2a0 2024-04-25T09:04:55.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.438+0000 7fbf8d151640 1 -- 172.21.15.43:0/1477623437 >> v2:172.21.15.193:3300/0 conn(0x7fbf880d39d0 msgr2=0x7fbf880d0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:55.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.438+0000 7fbf8d151640 1 --2- 172.21.15.43:0/1477623437 >> v2:172.21.15.193:3300/0 conn(0x7fbf880d39d0 0x7fbf880d0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.428 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.438+0000 7fbf8d151640 1 -- 172.21.15.43:0/1477623437 >> v1:172.21.15.193:6789/0 conn(0x7fbf880d2500 legacy=0x7fbf880d2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.438+0000 7fbf8d151640 1 -- 172.21.15.43:0/1477623437 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf880cec40 con 0x7fbf880ce2a0 2024-04-25T09:04:55.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.438+0000 7fbf8d151640 1 -- 172.21.15.43:0/1477623437 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7fbf7c001840 con 0x7fbf880ce2a0 2024-04-25T09:04:55.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.438+0000 7fbf8d151640 1 -- 172.21.15.43:0/1477623437 >> v1:172.21.15.43:6789/0 conn(0x7fbf880ce2a0 legacy=0x7fbf880d3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.438+0000 7fbf8d151640 1 --2- 172.21.15.43:0/1477623437 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf700039b0 0x7fbf70003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.438+0000 7fbf8d151640 1 --2- 172.21.15.43:0/1477623437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf70004800 0x7fbf70004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.438+0000 7fbf8d151640 1 --2- 172.21.15.43:0/1477623437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf700051c0 0x7fbf7000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.438+0000 7fbf8d151640 1 -- 172.21.15.43:0/1477623437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf880d2020 con 0x7fbf700051c0 2024-04-25T09:04:55.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.438+0000 7fbf8d151640 1 -- 172.21.15.43:0/1477623437 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7000c160 con 0x7fbf70004800 2024-04-25T09:04:55.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.438+0000 7fbf8d151640 1 -- 172.21.15.43:0/1477623437 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf7000c360 con 0x7fbf700039b0 2024-04-25T09:04:55.429 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.438+0000 7fbf8e954640 1 --2- 172.21.15.43:0/1477623437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf700051c0 0x7fbf7000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.438+0000 7fbf8e153640 1 --2- 172.21.15.43:0/1477623437 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf700039b0 0x7fbf70003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.438+0000 7fbf8d952640 1 --2- 172.21.15.43:0/1477623437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf70004800 0x7fbf70004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.438+0000 7fbf8e954640 1 -- 172.21.15.43:0/1477623437 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf700039b0 msgr2=0x7fbf70003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:55.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.438+0000 7fbf8e954640 1 --2- 172.21.15.43:0/1477623437 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf700039b0 0x7fbf70003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.438+0000 7fbf8e954640 1 -- 172.21.15.43:0/1477623437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf70004800 msgr2=0x7fbf70004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:55.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.439+0000 7fbf8e954640 1 --2- 172.21.15.43:0/1477623437 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf70004800 0x7fbf70004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.439+0000 7fbf8e954640 1 -- 172.21.15.43:0/1477623437 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf70003520 con 0x7fbf700051c0 2024-04-25T09:04:55.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.439+0000 7fbf8e954640 1 -- 172.21.15.43:0/1477623437 reap_dead start 2024-04-25T09:04:55.430 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.439+0000 7fbf8e954640 1 --2- 172.21.15.43:0/1477623437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf700051c0 0x7fbf7000b9c0 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7fbf78001750 tx=0x7fbf70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bb97f26cfb1cf7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:55.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.439+0000 7fbf8d151640 1 -- 172.21.15.43:0/1477623437 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf78001370 con 0x7fbf700051c0 2024-04-25T09:04:55.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.439+0000 7fbf8d151640 1 -- 172.21.15.43:0/1477623437 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf78005310 con 0x7fbf700051c0 2024-04-25T09:04:55.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.439+0000 7fbf8d151640 1 -- 172.21.15.43:0/1477623437 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7802abe0 con 0x7fbf700051c0 2024-04-25T09:04:55.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.440+0000 7fbf903b7640 1 -- 172.21.15.43:0/1477623437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf700051c0 msgr2=0x7fbf7000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.440+0000 7fbf903b7640 1 --2- 172.21.15.43:0/1477623437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf700051c0 0x7fbf7000b9c0 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7fbf78001750 tx=0x7fbf70005830 comp rx=0 tx=0).stop 2024-04-25T09:04:55.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.440+0000 7fbf903b7640 1 -- 172.21.15.43:0/1477623437 shutdown_connections 2024-04-25T09:04:55.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.440+0000 7fbf903b7640 1 --2- 172.21.15.43:0/1477623437 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf700051c0 0x7fbf7000b9c0 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.440+0000 7fbf903b7640 1 -- 172.21.15.43:0/1477623437 >> 172.21.15.43:0/1477623437 conn(0x7fbf8806b000 msgr2=0x7fbf8806b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:55.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.440+0000 7fbf903b7640 1 -- 172.21.15.43:0/1477623437 shutdown_connections 2024-04-25T09:04:55.431 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.440+0000 7fbf903b7640 1 -- 172.21.15.43:0/1477623437 wait complete. 2024-04-25T09:04:55.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.441+0000 7fbf903b7640 1 Processor -- start 2024-04-25T09:04:55.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.441+0000 7fbf903b7640 1 -- start start 2024-04-25T09:04:55.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.442+0000 7fbf903b7640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf700051c0 0x7fbf881f5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.442+0000 7fbf903b7640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf880b35e0 0x7fbf880b58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.442+0000 7fbf903b7640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf880cbee0 0x7fbf880cc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.442+0000 7fbf903b7640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf880cf200 con 0x7fbf880cbee0 2024-04-25T09:04:55.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.442+0000 7fbf903b7640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf880b2f00 con 0x7fbf700051c0 2024-04-25T09:04:55.434 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.442+0000 7fbf903b7640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf880d0d90 con 0x7fbf880b35e0 2024-04-25T09:04:55.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.442+0000 7fbf8e954640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf880cbee0 0x7fbf880cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.442+0000 7fbf8e954640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf880cbee0 0x7fbf880cc360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43660/0 (socket says 172.21.15.43:43660) 2024-04-25T09:04:55.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.442+0000 7fbf8e954640 1 -- 172.21.15.43:0/4264267172 learned_addr learned my addr 172.21.15.43:0/4264267172 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:55.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.442+0000 7fbf8d952640 1 --2- 172.21.15.43:0/4264267172 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf880b35e0 0x7fbf880b58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.442+0000 7fbf8e153640 1 --2- 172.21.15.43:0/4264267172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf700051c0 0x7fbf881f5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.442+0000 7fbf8e954640 1 -- 172.21.15.43:0/4264267172 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf880b35e0 msgr2=0x7fbf880b58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.442+0000 7fbf8e954640 1 --2- 172.21.15.43:0/4264267172 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf880b35e0 0x7fbf880b58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.442+0000 7fbf8e954640 1 -- 172.21.15.43:0/4264267172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf700051c0 msgr2=0x7fbf881f5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.435 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.442+0000 7fbf8e954640 1 --2- 172.21.15.43:0/4264267172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf700051c0 0x7fbf881f5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.442+0000 7fbf8e954640 1 -- 172.21.15.43:0/4264267172 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf880b6110 con 0x7fbf880cbee0 2024-04-25T09:04:55.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.442+0000 7fbf8d952640 1 --2- 172.21.15.43:0/4264267172 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf880b35e0 0x7fbf880b58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:55.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.442+0000 7fbf8e153640 1 --2- 172.21.15.43:0/4264267172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf700051c0 0x7fbf881f5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:55.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.443+0000 7fbf8e954640 1 --2- 172.21.15.43:0/4264267172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf880cbee0 0x7fbf880cc360 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7fbf78001840 tx=0x7fbf78001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:55.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.443+0000 7fbf86ffd640 1 -- 172.21.15.43:0/4264267172 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf78001370 con 0x7fbf880cbee0 2024-04-25T09:04:55.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.443+0000 7fbf86ffd640 1 -- 172.21.15.43:0/4264267172 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf7802ae20 con 0x7fbf880cbee0 2024-04-25T09:04:55.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.443+0000 7fbf86ffd640 1 -- 172.21.15.43:0/4264267172 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf78007a30 con 0x7fbf880cbee0 2024-04-25T09:04:55.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.443+0000 7fbf903b7640 1 -- 172.21.15.43:0/4264267172 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf880cb400 con 0x7fbf880cbee0 2024-04-25T09:04:55.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.443+0000 7fbf903b7640 1 -- 172.21.15.43:0/4264267172 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf880b5e00 con 0x7fbf880cbee0 2024-04-25T09:04:55.436 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.444+0000 7fbf903b7640 1 -- 172.21.15.43:0/4264267172 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf880cb210 con 0x7fbf880cbee0 2024-04-25T09:04:55.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.444+0000 7fbf86ffd640 1 -- 172.21.15.43:0/4264267172 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbf7803b030 con 0x7fbf880cbee0 2024-04-25T09:04:55.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.445+0000 7fbf86ffd640 1 --2- 172.21.15.43:0/4264267172 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbf7406b4d0 0x7fbf7406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.445+0000 7fbf86ffd640 1 -- 172.21.15.43:0/4264267172 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbf780a8bd0 con 0x7fbf880cbee0 2024-04-25T09:04:55.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.445+0000 7fbf8e153640 1 --2- 172.21.15.43:0/4264267172 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbf7406b4d0 0x7fbf7406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.445+0000 7fbf8e153640 1 --2- 172.21.15.43:0/4264267172 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbf7406b4d0 0x7fbf7406d990 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fbf80007440 tx=0x7fbf8000b040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:55.437 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.449+0000 7fbf86ffd640 1 -- 172.21.15.43:0/4264267172 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbf78077d50 con 0x7fbf880cbee0 2024-04-25T09:04:55.615 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.628+0000 7fbf903b7640 1 -- 172.21.15.43:0/4264267172 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf88074ef0 con 0x7fbf7406b4d0 2024-04-25T09:04:55.616 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.629+0000 7fbf86ffd640 1 -- 172.21.15.43:0/4264267172 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fbf88074ef0 con 0x7fbf7406b4d0 2024-04-25T09:04:55.616 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 5 to deep-scrub 2024-04-25T09:04:55.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.630+0000 7fbf903b7640 1 -- 172.21.15.43:0/4264267172 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbf7406b4d0 msgr2=0x7fbf7406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.630+0000 7fbf903b7640 1 --2- 172.21.15.43:0/4264267172 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbf7406b4d0 0x7fbf7406d990 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fbf80007440 tx=0x7fbf8000b040 comp rx=0 tx=0).stop 2024-04-25T09:04:55.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.630+0000 7fbf903b7640 1 -- 172.21.15.43:0/4264267172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf880cbee0 msgr2=0x7fbf880cc360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.630+0000 7fbf903b7640 1 --2- 172.21.15.43:0/4264267172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf880cbee0 0x7fbf880cc360 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7fbf78001840 tx=0x7fbf78001120 comp rx=0 tx=0).stop 2024-04-25T09:04:55.618 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.631+0000 7fbf903b7640 1 -- 172.21.15.43:0/4264267172 shutdown_connections 2024-04-25T09:04:55.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.631+0000 7fbf903b7640 1 --2- 172.21.15.43:0/4264267172 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbf880cbee0 0x7fbf880cc360 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.631+0000 7fbf903b7640 1 --2- 172.21.15.43:0/4264267172 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbf880b35e0 0x7fbf880b58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.631+0000 7fbf903b7640 1 --2- 172.21.15.43:0/4264267172 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbf7406b4d0 0x7fbf7406d990 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.631+0000 7fbf903b7640 1 --2- 172.21.15.43:0/4264267172 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbf700051c0 0x7fbf881f5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.631+0000 7fbf903b7640 1 -- 172.21.15.43:0/4264267172 >> 172.21.15.43:0/4264267172 conn(0x7fbf8806b000 msgr2=0x7fbf880b8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:55.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.631+0000 7fbf903b7640 1 -- 172.21.15.43:0/4264267172 shutdown_connections 2024-04-25T09:04:55.619 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.631+0000 7fbf903b7640 1 -- 172.21.15.43:0/4264267172 wait complete. 2024-04-25T09:04:55.629 INFO:tasks.ceph:Scrubbing osd.6 2024-04-25T09:04:55.629 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-25T09:04:55.734 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.747+0000 7fbb46de0640 1 Processor -- start 2024-04-25T09:04:55.735 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.747+0000 7fbb46de0640 1 -- start start 2024-04-25T09:04:55.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.748+0000 7fbb46de0640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbb4015bb40 0x7fbb400ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.736 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.748+0000 7fbb46de0640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbb400cca30 0x7fbb40070b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.748+0000 7fbb46de0640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb40160510 con 0x7fbb400aec70 2024-04-25T09:04:55.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.748+0000 7fbb46de0640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb400af0d0 con 0x7fbb400cca30 2024-04-25T09:04:55.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.748+0000 7fbb46de0640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4015f750 con 0x7fbb4015bb40 2024-04-25T09:04:55.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.748+0000 7fbb4537d640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fbb400cca30 0x7fbb40070b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.748+0000 7fbb37fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7fbb400aec70 0x7fbb400cc320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56618/0 (socket says 172.21.15.43:56618) 2024-04-25T09:04:55.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.748+0000 7fbb37fff640 1 -- 172.21.15.43:0/1813821578 learned_addr learned my addr 172.21.15.43:0/1813821578 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:55.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.748+0000 7fbb44b7c640 1 --2- 172.21.15.43:0/1813821578 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbb4015bb40 0x7fbb400ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.748+0000 7fbb4537d640 1 -- 172.21.15.43:0/1813821578 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbb4015bb40 msgr2=0x7fbb400ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:55.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.748+0000 7fbb4537d640 1 --2- 172.21.15.43:0/1813821578 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbb4015bb40 0x7fbb400ae730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.737 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.748+0000 7fbb4537d640 1 -- 172.21.15.43:0/1813821578 >> v1:172.21.15.43:6789/0 conn(0x7fbb400aec70 legacy=0x7fbb400cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.748+0000 7fbb4537d640 1 -- 172.21.15.43:0/1813821578 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb4015fa30 con 0x7fbb400cca30 2024-04-25T09:04:55.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.748+0000 7fbb377fe640 1 -- 172.21.15.43:0/1813821578 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 192105667 0 0) 0x7fbb40160510 con 0x7fbb400aec70 2024-04-25T09:04:55.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.748+0000 7fbb44b7c640 1 --2- 172.21.15.43:0/1813821578 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbb4015bb40 0x7fbb400ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:55.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.749+0000 7fbb4537d640 1 --2- 172.21.15.43:0/1813821578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb400cca30 0x7fbb40070b80 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7fbb38009d90 tx=0x7fbb3802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=349752d623222d1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:55.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.749+0000 7fbb377fe640 1 -- 172.21.15.43:0/1813821578 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb38007740 con 0x7fbb400cca30 2024-04-25T09:04:55.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.749+0000 7fbb377fe640 1 -- 172.21.15.43:0/1813821578 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb380078e0 con 0x7fbb400cca30 2024-04-25T09:04:55.738 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.749+0000 7fbb377fe640 1 -- 172.21.15.43:0/1813821578 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb38007cb0 con 0x7fbb400cca30 2024-04-25T09:04:55.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.750+0000 7fbb46de0640 1 -- 172.21.15.43:0/1813821578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb400cca30 msgr2=0x7fbb40070b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.750+0000 7fbb46de0640 1 --2- 172.21.15.43:0/1813821578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb400cca30 0x7fbb40070b80 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7fbb38009d90 tx=0x7fbb3802a660 comp rx=0 tx=0).stop 2024-04-25T09:04:55.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.750+0000 7fbb46de0640 1 -- 172.21.15.43:0/1813821578 shutdown_connections 2024-04-25T09:04:55.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.750+0000 7fbb46de0640 1 --2- 172.21.15.43:0/1813821578 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb400cca30 0x7fbb40070b80 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.741 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.750+0000 7fbb46de0640 1 --2- 172.21.15.43:0/1813821578 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbb4015bb40 0x7fbb400ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.750+0000 7fbb46de0640 1 -- 172.21.15.43:0/1813821578 >> 172.21.15.43:0/1813821578 conn(0x7fbb4006ab30 msgr2=0x7fbb4006af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:55.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.750+0000 7fbb46de0640 1 -- 172.21.15.43:0/1813821578 shutdown_connections 2024-04-25T09:04:55.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.750+0000 7fbb46de0640 1 -- 172.21.15.43:0/1813821578 wait complete. 2024-04-25T09:04:55.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.751+0000 7fbb46de0640 1 Processor -- start 2024-04-25T09:04:55.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.751+0000 7fbb46de0640 1 -- start start 2024-04-25T09:04:55.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.752+0000 7fbb46de0640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb400aec70 0x7fbb4014f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.752+0000 7fbb46de0640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbb400cca30 0x7fbb400b1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.752+0000 7fbb46de0640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb4015bb40 0x7fbb400b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.742 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.752+0000 7fbb46de0640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb400710c0 con 0x7fbb400aec70 2024-04-25T09:04:55.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.752+0000 7fbb46de0640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4015fff0 con 0x7fbb4015bb40 2024-04-25T09:04:55.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.752+0000 7fbb46de0640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb40071240 con 0x7fbb400cca30 2024-04-25T09:04:55.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.752+0000 7fbb44b7c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb400aec70 0x7fbb4014f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.752+0000 7fbb44b7c640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb400aec70 0x7fbb4014f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43674/0 (socket says 172.21.15.43:43674) 2024-04-25T09:04:55.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.752+0000 7fbb44b7c640 1 -- 172.21.15.43:0/2304331541 learned_addr learned my addr 172.21.15.43:0/2304331541 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:55.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.752+0000 7fbb37fff640 1 --2- 172.21.15.43:0/2304331541 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbb400cca30 0x7fbb400b1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.752+0000 7fbb4537d640 1 --2- 172.21.15.43:0/2304331541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb4015bb40 0x7fbb400b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.752+0000 7fbb44b7c640 1 -- 172.21.15.43:0/2304331541 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbb400cca30 msgr2=0x7fbb400b1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.752+0000 7fbb44b7c640 1 --2- 172.21.15.43:0/2304331541 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fbb400cca30 0x7fbb400b1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.743 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.752+0000 7fbb44b7c640 1 -- 172.21.15.43:0/2304331541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb4015bb40 msgr2=0x7fbb400b1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.752+0000 7fbb44b7c640 1 --2- 172.21.15.43:0/2304331541 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fbb4015bb40 0x7fbb400b1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.752+0000 7fbb44b7c640 1 -- 172.21.15.43:0/2304331541 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb401f4a20 con 0x7fbb400aec70 2024-04-25T09:04:55.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.752+0000 7fbb44b7c640 1 --2- 172.21.15.43:0/2304331541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb400aec70 0x7fbb4014f1f0 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00d7d0 tx=0x7fbb3c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:55.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.752+0000 7fbb357fa640 1 -- 172.21.15.43:0/2304331541 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c004030 con 0x7fbb400aec70 2024-04-25T09:04:55.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.753+0000 7fbb357fa640 1 -- 172.21.15.43:0/2304331541 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb3c0041d0 con 0x7fbb400aec70 2024-04-25T09:04:55.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.753+0000 7fbb46de0640 1 -- 172.21.15.43:0/2304331541 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb401f4d00 con 0x7fbb400aec70 2024-04-25T09:04:55.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.753+0000 7fbb357fa640 1 -- 172.21.15.43:0/2304331541 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c0096a0 con 0x7fbb400aec70 2024-04-25T09:04:55.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.753+0000 7fbb46de0640 1 -- 172.21.15.43:0/2304331541 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb401f52c0 con 0x7fbb400aec70 2024-04-25T09:04:55.744 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.754+0000 7fbb46de0640 1 -- 172.21.15.43:0/2304331541 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbb40068a40 con 0x7fbb400aec70 2024-04-25T09:04:55.745 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.754+0000 7fbb357fa640 1 -- 172.21.15.43:0/2304331541 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbb3c009840 con 0x7fbb400aec70 2024-04-25T09:04:55.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.754+0000 7fbb357fa640 1 --2- 172.21.15.43:0/2304331541 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbb2c06b4d0 0x7fbb2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.754+0000 7fbb357fa640 1 -- 172.21.15.43:0/2304331541 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbb3c08a2d0 con 0x7fbb400aec70 2024-04-25T09:04:55.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.755+0000 7fbb357fa640 1 --2- 172.21.15.43:0/2304331541 >> [v2:172.21.15.193:6821/1737328270,v1:172.21.15.193:6823/1737328270] conn(0x7fbb2c071390 0x7fbb2c0737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.755+0000 7fbb37fff640 1 --2- 172.21.15.43:0/2304331541 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbb2c06b4d0 0x7fbb2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.755+0000 7fbb357fa640 1 -- 172.21.15.43:0/2304331541 --> [v2:172.21.15.193:6821/1737328270,v1:172.21.15.193:6823/1737328270] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbb3c01ee50 con 0x7fbb2c071390 2024-04-25T09:04:55.746 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.755+0000 7fbb357fa640 1 -- 172.21.15.43:0/2304331541 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=142) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb3c091050 con 0x7fbb400aec70 2024-04-25T09:04:55.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.755+0000 7fbb4537d640 1 --2- 172.21.15.43:0/2304331541 >> [v2:172.21.15.193:6821/1737328270,v1:172.21.15.193:6823/1737328270] conn(0x7fbb2c071390 0x7fbb2c0737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.755+0000 7fbb37fff640 1 --2- 172.21.15.43:0/2304331541 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbb2c06b4d0 0x7fbb2c06d990 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fbb30007fd0 tx=0x7fbb30009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:55.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.755+0000 7fbb4537d640 1 --2- 172.21.15.43:0/2304331541 >> [v2:172.21.15.193:6821/1737328270,v1:172.21.15.193:6823/1737328270] conn(0x7fbb2c071390 0x7fbb2c0737b0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:55.747 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.756+0000 7fbb357fa640 1 -- 172.21.15.43:0/2304331541 <== osd.6 v2:172.21.15.193:6821/1737328270 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fbb3c01ee50 con 0x7fbb2c071390 2024-04-25T09:04:55.756 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.769+0000 7fbb46de0640 1 -- 172.21.15.43:0/2304331541 --> [v2:172.21.15.193:6821/1737328270,v1:172.21.15.193:6823/1737328270] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fbb400cbd60 con 0x7fbb2c071390 2024-04-25T09:04:55.757 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.770+0000 7fbb357fa640 1 -- 172.21.15.43:0/2304331541 <== osd.6 v2:172.21.15.193:6821/1737328270 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fbb400cbd60 con 0x7fbb2c071390 2024-04-25T09:04:55.757 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:04:55.757 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T09:04:55.757 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:04:55.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.770+0000 7fbb46de0640 1 -- 172.21.15.43:0/2304331541 >> [v2:172.21.15.193:6821/1737328270,v1:172.21.15.193:6823/1737328270] conn(0x7fbb2c071390 msgr2=0x7fbb2c0737b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.758 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.770+0000 7fbb46de0640 1 --2- 172.21.15.43:0/2304331541 >> [v2:172.21.15.193:6821/1737328270,v1:172.21.15.193:6823/1737328270] conn(0x7fbb2c071390 0x7fbb2c0737b0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.770+0000 7fbb46de0640 1 -- 172.21.15.43:0/2304331541 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbb2c06b4d0 msgr2=0x7fbb2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.770+0000 7fbb46de0640 1 --2- 172.21.15.43:0/2304331541 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fbb2c06b4d0 0x7fbb2c06d990 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fbb30007fd0 tx=0x7fbb30009040 comp rx=0 tx=0).stop 2024-04-25T09:04:55.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.770+0000 7fbb46de0640 1 -- 172.21.15.43:0/2304331541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb400aec70 msgr2=0x7fbb4014f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.770+0000 7fbb46de0640 1 --2- 172.21.15.43:0/2304331541 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fbb400aec70 0x7fbb4014f1f0 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00d7d0 tx=0x7fbb3c00dc90 comp rx=0 tx=0).stop 2024-04-25T09:04:55.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.770+0000 7fbb4537d640 1 -- 172.21.15.43:0/2304331541 reap_dead start 2024-04-25T09:04:55.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.770+0000 7fbb46de0640 1 -- 172.21.15.43:0/2304331541 shutdown_connections 2024-04-25T09:04:55.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.770+0000 7fbb46de0640 1 -- 172.21.15.43:0/2304331541 >> 172.21.15.43:0/2304331541 conn(0x7fbb4006ab30 msgr2=0x7fbb40071690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:55.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.770+0000 7fbb46de0640 1 -- 172.21.15.43:0/2304331541 shutdown_connections 2024-04-25T09:04:55.759 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.770+0000 7fbb46de0640 1 -- 172.21.15.43:0/2304331541 wait complete. 2024-04-25T09:04:55.768 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-25T09:04:55.870 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.883+0000 7f748dd9e640 1 Processor -- start 2024-04-25T09:04:55.871 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.883+0000 7f748dd9e640 1 -- start start 2024-04-25T09:04:55.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.884+0000 7f748dd9e640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f74880c8d00 0x7f7488162100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.884+0000 7f748dd9e640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74880d24a0 con 0x7f74880ce660 2024-04-25T09:04:55.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.884+0000 7f748dd9e640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74880d2740 con 0x7f748815df50 2024-04-25T09:04:55.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.884+0000 7f748dd9e640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f748815e330 con 0x7f74880c8d00 2024-04-25T09:04:55.874 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.884+0000 7f7487fff640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f74880ce660 0x7f74880cea70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56628/0 (socket says 172.21.15.43:56628) 2024-04-25T09:04:55.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.884+0000 7f7487fff640 1 -- 172.21.15.43:0/1419634265 learned_addr learned my addr 172.21.15.43:0/1419634265 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:55.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.884+0000 7f7486ffd640 1 --2- 172.21.15.43:0/1419634265 >> v2:172.21.15.193:3300/0 conn(0x7f74880c8d00 0x7f7488162100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.884+0000 7f74867fc640 1 -- 172.21.15.43:0/1419634265 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2133012347 0 0) 0x7f74880d24a0 con 0x7f74880ce660 2024-04-25T09:04:55.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.884+0000 7f74867fc640 1 -- 172.21.15.43:0/1419634265 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7470003520 con 0x7f74880ce660 2024-04-25T09:04:55.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.884+0000 7f74867fc640 1 -- 172.21.15.43:0/1419634265 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2935675701 0 0) 0x7f7470003520 con 0x7f74880ce660 2024-04-25T09:04:55.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.884+0000 7f74867fc640 1 -- 172.21.15.43:0/1419634265 >> v2:172.21.15.193:3300/0 conn(0x7f74880c8d00 msgr2=0x7f7488162100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:55.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.884+0000 7f74867fc640 1 --2- 172.21.15.43:0/1419634265 >> v2:172.21.15.193:3300/0 conn(0x7f74880c8d00 0x7f7488162100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.884+0000 7f74867fc640 1 -- 172.21.15.43:0/1419634265 >> v1:172.21.15.193:6789/0 conn(0x7f748815df50 legacy=0x7f74881619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.875 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.884+0000 7f74867fc640 1 -- 172.21.15.43:0/1419634265 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74880d2a20 con 0x7f74880ce660 2024-04-25T09:04:55.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.884+0000 7f74867fc640 1 -- 172.21.15.43:0/1419634265 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f7480001a10 con 0x7f74880ce660 2024-04-25T09:04:55.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.884+0000 7f74867fc640 1 -- 172.21.15.43:0/1419634265 >> v1:172.21.15.43:6789/0 conn(0x7f74880ce660 legacy=0x7f74880cea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.885+0000 7f74867fc640 1 --2- 172.21.15.43:0/1419634265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74700039b0 0x7f7470003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.885+0000 7f74867fc640 1 --2- 172.21.15.43:0/1419634265 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7470004800 0x7f7470004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.885+0000 7f74867fc640 1 --2- 172.21.15.43:0/1419634265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74700051c0 0x7f747000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.885+0000 7f74867fc640 1 -- 172.21.15.43:0/1419634265 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f748815e330 con 0x7f74700051c0 2024-04-25T09:04:55.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.885+0000 7f74867fc640 1 -- 172.21.15.43:0/1419634265 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f747000c060 con 0x7f74700039b0 2024-04-25T09:04:55.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.885+0000 7f74867fc640 1 -- 172.21.15.43:0/1419634265 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f747000c260 con 0x7f7470004800 2024-04-25T09:04:55.876 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.885+0000 7f7487fff640 1 --2- 172.21.15.43:0/1419634265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74700051c0 0x7f747000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.885+0000 7f74877fe640 1 --2- 172.21.15.43:0/1419634265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74700039b0 0x7f7470003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.885+0000 7f7486ffd640 1 --2- 172.21.15.43:0/1419634265 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7470004800 0x7f7470004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.885+0000 7f7487fff640 1 -- 172.21.15.43:0/1419634265 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7470004800 msgr2=0x7f7470004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:55.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.885+0000 7f7487fff640 1 --2- 172.21.15.43:0/1419634265 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f7470004800 0x7f7470004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.885+0000 7f7487fff640 1 -- 172.21.15.43:0/1419634265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74700039b0 msgr2=0x7f7470003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:55.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.885+0000 7f7487fff640 1 --2- 172.21.15.43:0/1419634265 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74700039b0 0x7f7470003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.885+0000 7f7487fff640 1 -- 172.21.15.43:0/1419634265 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7470003520 con 0x7f74700051c0 2024-04-25T09:04:55.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.886+0000 7f7487fff640 1 -- 172.21.15.43:0/1419634265 reap_dead start 2024-04-25T09:04:55.877 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.886+0000 7f7487fff640 1 --2- 172.21.15.43:0/1419634265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74700051c0 0x7f747000b9c0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f748000a8f0 tx=0x7f7470005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=799b6e8f308328f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:55.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.886+0000 7f74867fc640 1 -- 172.21.15.43:0/1419634265 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748000a4c0 con 0x7f74700051c0 2024-04-25T09:04:55.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.886+0000 7f74867fc640 1 -- 172.21.15.43:0/1419634265 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f748002f070 con 0x7f74700051c0 2024-04-25T09:04:55.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.886+0000 7f74867fc640 1 -- 172.21.15.43:0/1419634265 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748002bcb0 con 0x7f74700051c0 2024-04-25T09:04:55.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.886+0000 7f748dd9e640 1 -- 172.21.15.43:0/1419634265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74700051c0 msgr2=0x7f747000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.886+0000 7f748dd9e640 1 --2- 172.21.15.43:0/1419634265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74700051c0 0x7f747000b9c0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f748000a8f0 tx=0x7f7470005830 comp rx=0 tx=0).stop 2024-04-25T09:04:55.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.886+0000 7f748dd9e640 1 -- 172.21.15.43:0/1419634265 shutdown_connections 2024-04-25T09:04:55.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.886+0000 7f748dd9e640 1 --2- 172.21.15.43:0/1419634265 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f74700051c0 0x7f747000b9c0 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.878 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.886+0000 7f748dd9e640 1 -- 172.21.15.43:0/1419634265 >> 172.21.15.43:0/1419634265 conn(0x7f74880d3a30 msgr2=0x7f74880d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:55.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.886+0000 7f748dd9e640 1 -- 172.21.15.43:0/1419634265 shutdown_connections 2024-04-25T09:04:55.879 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.886+0000 7f748dd9e640 1 -- 172.21.15.43:0/1419634265 wait complete. 2024-04-25T09:04:55.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.888+0000 7f748dd9e640 1 Processor -- start 2024-04-25T09:04:55.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.888+0000 7f748dd9e640 1 -- start start 2024-04-25T09:04:55.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.888+0000 7f748dd9e640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f74700051c0 0x7f74881497b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.881 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.888+0000 7f748dd9e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7488149f70 0x7f748814e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.889+0000 7f748dd9e640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74881525d0 0x7f7488152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.889+0000 7f748dd9e640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74880d2190 con 0x7f7488149f70 2024-04-25T09:04:55.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.889+0000 7f748dd9e640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74880c77d0 con 0x7f74881525d0 2024-04-25T09:04:55.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.889+0000 7f748dd9e640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74881619f0 con 0x7f74700051c0 2024-04-25T09:04:55.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.889+0000 7f7486ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7488149f70 0x7f748814e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.889+0000 7f7486ffd640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7488149f70 0x7f748814e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43686/0 (socket says 172.21.15.43:43686) 2024-04-25T09:04:55.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.889+0000 7f7486ffd640 1 -- 172.21.15.43:0/4064704745 learned_addr learned my addr 172.21.15.43:0/4064704745 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:55.882 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.889+0000 7f7487fff640 1 --2- 172.21.15.43:0/4064704745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74881525d0 0x7f7488152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.889+0000 7f74877fe640 1 --2- 172.21.15.43:0/4064704745 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f74700051c0 0x7f74881497b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.889+0000 7f7486ffd640 1 -- 172.21.15.43:0/4064704745 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f74700051c0 msgr2=0x7f74881497b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.889+0000 7f7486ffd640 1 --2- 172.21.15.43:0/4064704745 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f74700051c0 0x7f74881497b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.889+0000 7f7486ffd640 1 -- 172.21.15.43:0/4064704745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74881525d0 msgr2=0x7f7488152a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:55.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.889+0000 7f7486ffd640 1 --2- 172.21.15.43:0/4064704745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74881525d0 0x7f7488152a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:55.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.889+0000 7f7486ffd640 1 -- 172.21.15.43:0/4064704745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f748814e5b0 con 0x7f7488149f70 2024-04-25T09:04:55.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.889+0000 7f74877fe640 1 --2- 172.21.15.43:0/4064704745 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f74700051c0 0x7f74881497b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:55.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.889+0000 7f7487fff640 1 --2- 172.21.15.43:0/4064704745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74881525d0 0x7f7488152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:55.883 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.889+0000 7f7486ffd640 1 --2- 172.21.15.43:0/4064704745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7488149f70 0x7f748814e070 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f747c011910 tx=0x7f747c011dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:55.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.889+0000 7f746ffff640 1 -- 172.21.15.43:0/4064704745 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c00faf0 con 0x7f7488149f70 2024-04-25T09:04:55.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.889+0000 7f746ffff640 1 -- 172.21.15.43:0/4064704745 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f747c00fc90 con 0x7f7488149f70 2024-04-25T09:04:55.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.889+0000 7f748dd9e640 1 -- 172.21.15.43:0/4064704745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7488153430 con 0x7f7488149f70 2024-04-25T09:04:55.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.889+0000 7f746ffff640 1 -- 172.21.15.43:0/4064704745 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f747c014440 con 0x7f7488149f70 2024-04-25T09:04:55.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.889+0000 7f748dd9e640 1 -- 172.21.15.43:0/4064704745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7488151080 con 0x7f7488149f70 2024-04-25T09:04:55.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.891+0000 7f748dd9e640 1 -- 172.21.15.43:0/4064704745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74880680f0 con 0x7f7488149f70 2024-04-25T09:04:55.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.891+0000 7f746ffff640 1 -- 172.21.15.43:0/4064704745 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f747c012040 con 0x7f7488149f70 2024-04-25T09:04:55.884 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.891+0000 7f746ffff640 1 --2- 172.21.15.43:0/4064704745 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f747406b4d0 0x7f747406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:55.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.891+0000 7f746ffff640 1 -- 172.21.15.43:0/4064704745 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f747c08a4a0 con 0x7f7488149f70 2024-04-25T09:04:55.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.892+0000 7f74877fe640 1 --2- 172.21.15.43:0/4064704745 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f747406b4d0 0x7f747406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:55.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.892+0000 7f74877fe640 1 --2- 172.21.15.43:0/4064704745 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f747406b4d0 0x7f747406d990 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f74880d1450 tx=0x7f7478009040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:55.885 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:55.896+0000 7f746ffff640 1 -- 172.21.15.43:0/4064704745 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f747c019030 con 0x7f7488149f70 2024-04-25T09:04:56.061 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.073+0000 7f748dd9e640 1 -- 172.21.15.43:0/4064704745 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f7488074f20 con 0x7f747406b4d0 2024-04-25T09:04:56.062 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.074+0000 7f746ffff640 1 -- 172.21.15.43:0/4064704745 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7488074f20 con 0x7f747406b4d0 2024-04-25T09:04:56.062 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 6 to deep-scrub 2024-04-25T09:04:56.063 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.076+0000 7f748dd9e640 1 -- 172.21.15.43:0/4064704745 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f747406b4d0 msgr2=0x7f747406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:56.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.076+0000 7f748dd9e640 1 --2- 172.21.15.43:0/4064704745 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f747406b4d0 0x7f747406d990 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f74880d1450 tx=0x7f7478009040 comp rx=0 tx=0).stop 2024-04-25T09:04:56.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.076+0000 7f748dd9e640 1 -- 172.21.15.43:0/4064704745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7488149f70 msgr2=0x7f748814e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:56.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.076+0000 7f748dd9e640 1 --2- 172.21.15.43:0/4064704745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7488149f70 0x7f748814e070 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f747c011910 tx=0x7f747c011dd0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.076+0000 7f748dd9e640 1 -- 172.21.15.43:0/4064704745 shutdown_connections 2024-04-25T09:04:56.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.076+0000 7f748dd9e640 1 --2- 172.21.15.43:0/4064704745 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f747406b4d0 0x7f747406d990 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.076+0000 7f748dd9e640 1 --2- 172.21.15.43:0/4064704745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f74881525d0 0x7f7488152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.076+0000 7f748dd9e640 1 --2- 172.21.15.43:0/4064704745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f7488149f70 0x7f748814e070 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.064 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.076+0000 7f748dd9e640 1 --2- 172.21.15.43:0/4064704745 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f74700051c0 0x7f74881497b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.076+0000 7f748dd9e640 1 -- 172.21.15.43:0/4064704745 >> 172.21.15.43:0/4064704745 conn(0x7f74880d3a30 msgr2=0x7f7488153b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:56.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.076+0000 7f748dd9e640 1 -- 172.21.15.43:0/4064704745 shutdown_connections 2024-04-25T09:04:56.065 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.076+0000 7f748dd9e640 1 -- 172.21.15.43:0/4064704745 wait complete. 2024-04-25T09:04:56.075 INFO:tasks.ceph:Scrubbing osd.7 2024-04-25T09:04:56.075 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-25T09:04:56.180 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.193+0000 7f19879ce640 1 Processor -- start 2024-04-25T09:04:56.181 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.193+0000 7f19879ce640 1 -- start start 2024-04-25T09:04:56.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.194+0000 7f19879ce640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f19800d3e70 0x7f198015e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.184 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.194+0000 7f19879ce640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19800b37a0 con 0x7f198015baa0 2024-04-25T09:04:56.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.194+0000 7f19879ce640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f198015eab0 con 0x7f198015b6c0 2024-04-25T09:04:56.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.194+0000 7f19879ce640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f198015fce0 con 0x7f19800d3e70 2024-04-25T09:04:56.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.194+0000 7f1984f69640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f198015baa0 0x7f19800d3760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56644/0 (socket says 172.21.15.43:56644) 2024-04-25T09:04:56.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.194+0000 7f1984f69640 1 -- 172.21.15.43:0/3030908125 learned_addr learned my addr 172.21.15.43:0/3030908125 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:56.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.195+0000 7f1985f6b640 1 --2- 172.21.15.43:0/3030908125 >> v2:172.21.15.193:3300/0 conn(0x7f19800d3e70 0x7f198015e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:56.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.195+0000 7f197ffff640 1 -- 172.21.15.43:0/3030908125 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1500874488 0 0) 0x7f19800b37a0 con 0x7f198015baa0 2024-04-25T09:04:56.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.195+0000 7f197ffff640 1 -- 172.21.15.43:0/3030908125 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1968003520 con 0x7f198015baa0 2024-04-25T09:04:56.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.195+0000 7f197ffff640 1 -- 172.21.15.43:0/3030908125 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4022939069 0 0) 0x7f1968003520 con 0x7f198015baa0 2024-04-25T09:04:56.185 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.195+0000 7f197ffff640 1 -- 172.21.15.43:0/3030908125 >> v2:172.21.15.193:3300/0 conn(0x7f19800d3e70 msgr2=0x7f198015e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:56.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.195+0000 7f197ffff640 1 --2- 172.21.15.43:0/3030908125 >> v2:172.21.15.193:3300/0 conn(0x7f19800d3e70 0x7f198015e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.195+0000 7f197ffff640 1 -- 172.21.15.43:0/3030908125 >> v1:172.21.15.193:6789/0 conn(0x7f198015b6c0 legacy=0x7f198015f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:56.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.195+0000 7f197ffff640 1 -- 172.21.15.43:0/3030908125 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f198015ed90 con 0x7f198015baa0 2024-04-25T09:04:56.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.195+0000 7f197ffff640 1 -- 172.21.15.43:0/3030908125 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f1974001690 con 0x7f198015baa0 2024-04-25T09:04:56.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.195+0000 7f197ffff640 1 -- 172.21.15.43:0/3030908125 >> v1:172.21.15.43:6789/0 conn(0x7f198015baa0 legacy=0x7f19800d3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:56.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.195+0000 7f197ffff640 1 --2- 172.21.15.43:0/3030908125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19680039b0 0x7f1968003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.186 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.195+0000 7f197ffff640 1 --2- 172.21.15.43:0/3030908125 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1968004800 0x7f1968004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.195+0000 7f197ffff640 1 --2- 172.21.15.43:0/3030908125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19680051c0 0x7f196800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.195+0000 7f197ffff640 1 -- 172.21.15.43:0/3030908125 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198015fce0 con 0x7f19680039b0 2024-04-25T09:04:56.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.195+0000 7f197ffff640 1 -- 172.21.15.43:0/3030908125 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196800c060 con 0x7f19680051c0 2024-04-25T09:04:56.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.195+0000 7f197ffff640 1 -- 172.21.15.43:0/3030908125 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f196800c260 con 0x7f1968004800 2024-04-25T09:04:56.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.195+0000 7f198576a640 1 --2- 172.21.15.43:0/3030908125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19680039b0 0x7f1968003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:56.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.195+0000 7f1984f69640 1 --2- 172.21.15.43:0/3030908125 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1968004800 0x7f1968004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:56.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.195+0000 7f1985f6b640 1 --2- 172.21.15.43:0/3030908125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19680051c0 0x7f196800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:56.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.195+0000 7f198576a640 1 -- 172.21.15.43:0/3030908125 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1968004800 msgr2=0x7f1968004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:56.187 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.195+0000 7f198576a640 1 --2- 172.21.15.43:0/3030908125 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1968004800 0x7f1968004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.195+0000 7f198576a640 1 -- 172.21.15.43:0/3030908125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19680051c0 msgr2=0x7f196800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:56.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.195+0000 7f198576a640 1 --2- 172.21.15.43:0/3030908125 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19680051c0 0x7f196800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.195+0000 7f198576a640 1 -- 172.21.15.43:0/3030908125 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1968003520 con 0x7f19680039b0 2024-04-25T09:04:56.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.195+0000 7f1985f6b640 1 -- 172.21.15.43:0/3030908125 reap_dead start 2024-04-25T09:04:56.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.195+0000 7f1984f69640 1 --2- 172.21.15.43:0/3030908125 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f1968004800 0x7f1968004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:04:56.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.196+0000 7f198576a640 1 --2- 172.21.15.43:0/3030908125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19680039b0 0x7f1968003dd0 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f1970001e50 tx=0x7f1968005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=febcf8e7560240da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:56.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.196+0000 7f197ffff640 1 -- 172.21.15.43:0/3030908125 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1970001a70 con 0x7f19680039b0 2024-04-25T09:04:56.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.196+0000 7f197ffff640 1 -- 172.21.15.43:0/3030908125 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f197002ae40 con 0x7f19680039b0 2024-04-25T09:04:56.188 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.196+0000 7f197ffff640 1 -- 172.21.15.43:0/3030908125 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1970003bb0 con 0x7f19680039b0 2024-04-25T09:04:56.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.197+0000 7f19879ce640 1 -- 172.21.15.43:0/3030908125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19680039b0 msgr2=0x7f1968003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:56.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.197+0000 7f19879ce640 1 --2- 172.21.15.43:0/3030908125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19680039b0 0x7f1968003dd0 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f1970001e50 tx=0x7f1968005830 comp rx=0 tx=0).stop 2024-04-25T09:04:56.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.197+0000 7f19879ce640 1 -- 172.21.15.43:0/3030908125 shutdown_connections 2024-04-25T09:04:56.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.197+0000 7f19879ce640 1 --2- 172.21.15.43:0/3030908125 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19680039b0 0x7f1968003dd0 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.197+0000 7f19879ce640 1 -- 172.21.15.43:0/3030908125 >> 172.21.15.43:0/3030908125 conn(0x7f198008d460 msgr2=0x7f198009cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:56.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.197+0000 7f19879ce640 1 -- 172.21.15.43:0/3030908125 shutdown_connections 2024-04-25T09:04:56.189 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.197+0000 7f19879ce640 1 -- 172.21.15.43:0/3030908125 wait complete. 2024-04-25T09:04:56.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.198+0000 7f19879ce640 1 Processor -- start 2024-04-25T09:04:56.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.198+0000 7f19879ce640 1 -- start start 2024-04-25T09:04:56.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.199+0000 7f19879ce640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19680039b0 0x7f198015af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.199+0000 7f19879ce640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19800cbe10 0x7f19800c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.199+0000 7f19879ce640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f19800cc290 0x7f19800c9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.199+0000 7f19879ce640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198015e410 con 0x7f19800cbe10 2024-04-25T09:04:56.193 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.199+0000 7f19879ce640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f198015f350 con 0x7f19680039b0 2024-04-25T09:04:56.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.199+0000 7f19879ce640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f198015e590 con 0x7f19800cc290 2024-04-25T09:04:56.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.199+0000 7f1984f69640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19800cbe10 0x7f19800c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:56.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.199+0000 7f1984f69640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19800cbe10 0x7f19800c7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43692/0 (socket says 172.21.15.43:43692) 2024-04-25T09:04:56.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.199+0000 7f1984f69640 1 -- 172.21.15.43:0/2672508804 learned_addr learned my addr 172.21.15.43:0/2672508804 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:56.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.199+0000 7f1985f6b640 1 --2- 172.21.15.43:0/2672508804 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f19800cc290 0x7f19800c9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:56.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.199+0000 7f198576a640 1 --2- 172.21.15.43:0/2672508804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19680039b0 0x7f198015af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:56.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.199+0000 7f1984f69640 1 -- 172.21.15.43:0/2672508804 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f19800cc290 msgr2=0x7f19800c9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:56.194 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.199+0000 7f1984f69640 1 --2- 172.21.15.43:0/2672508804 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f19800cc290 0x7f19800c9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.199+0000 7f1984f69640 1 -- 172.21.15.43:0/2672508804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19680039b0 msgr2=0x7f198015af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:56.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.199+0000 7f1984f69640 1 --2- 172.21.15.43:0/2672508804 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f19680039b0 0x7f198015af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.199+0000 7f1984f69640 1 -- 172.21.15.43:0/2672508804 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19800cea20 con 0x7f19800cbe10 2024-04-25T09:04:56.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.199+0000 7f1985f6b640 1 --2- 172.21.15.43:0/2672508804 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f19800cc290 0x7f19800c9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:56.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.199+0000 7f1984f69640 1 --2- 172.21.15.43:0/2672508804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19800cbe10 0x7f19800c7fd0 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f1974001ce0 tx=0x7f197400eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:56.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.199+0000 7f197dffb640 1 -- 172.21.15.43:0/2672508804 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197400c6c0 con 0x7f19800cbe10 2024-04-25T09:04:56.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.199+0000 7f197dffb640 1 -- 172.21.15.43:0/2672508804 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1974018070 con 0x7f19800cbe10 2024-04-25T09:04:56.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.200+0000 7f19879ce640 1 -- 172.21.15.43:0/2672508804 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f198015b4c0 con 0x7f19800cbe10 2024-04-25T09:04:56.195 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.200+0000 7f197dffb640 1 -- 172.21.15.43:0/2672508804 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f197400ca60 con 0x7f19800cbe10 2024-04-25T09:04:56.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.200+0000 7f19879ce640 1 -- 172.21.15.43:0/2672508804 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19800cefa0 con 0x7f19800cbe10 2024-04-25T09:04:56.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.201+0000 7f19879ce640 1 -- 172.21.15.43:0/2672508804 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1980068a40 con 0x7f19800cbe10 2024-04-25T09:04:56.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.201+0000 7f197dffb640 1 -- 172.21.15.43:0/2672508804 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f197400cc00 con 0x7f19800cbe10 2024-04-25T09:04:56.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.201+0000 7f197dffb640 1 --2- 172.21.15.43:0/2672508804 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f196c06b4d0 0x7f196c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.201+0000 7f197dffb640 1 -- 172.21.15.43:0/2672508804 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1974089c60 con 0x7f19800cbe10 2024-04-25T09:04:56.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.202+0000 7f198576a640 1 --2- 172.21.15.43:0/2672508804 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f196c06b4d0 0x7f196c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:56.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.202+0000 7f197dffb640 1 --2- 172.21.15.43:0/2672508804 >> [v2:172.21.15.193:6808/3086001447,v1:172.21.15.193:6809/3086001447] conn(0x7f196c071390 0x7f196c0737b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.202+0000 7f197dffb640 1 -- 172.21.15.43:0/2672508804 --> [v2:172.21.15.193:6808/3086001447,v1:172.21.15.193:6809/3086001447] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f196c073e60 con 0x7f196c071390 2024-04-25T09:04:56.196 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.202+0000 7f197dffb640 1 -- 172.21.15.43:0/2672508804 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_get_version_reply(handle=1 version=142) v2 ==== 24+0+0 (secure 0 0 0) 0x7f197408f070 con 0x7f19800cbe10 2024-04-25T09:04:56.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.202+0000 7f1985f6b640 1 --2- 172.21.15.43:0/2672508804 >> [v2:172.21.15.193:6808/3086001447,v1:172.21.15.193:6809/3086001447] conn(0x7f196c071390 0x7f196c0737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:56.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.202+0000 7f198576a640 1 --2- 172.21.15.43:0/2672508804 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f196c06b4d0 0x7f196c06d990 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f1970032000 tx=0x7f197002e000 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:56.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.202+0000 7f1985f6b640 1 --2- 172.21.15.43:0/2672508804 >> [v2:172.21.15.193:6808/3086001447,v1:172.21.15.193:6809/3086001447] conn(0x7f196c071390 0x7f196c0737b0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:56.197 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.203+0000 7f197dffb640 1 -- 172.21.15.43:0/2672508804 <== osd.7 v2:172.21.15.193:6808/3086001447 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f196c073e60 con 0x7f196c071390 2024-04-25T09:04:56.203 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.216+0000 7f19879ce640 1 -- 172.21.15.43:0/2672508804 --> [v2:172.21.15.193:6808/3086001447,v1:172.21.15.193:6809/3086001447] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f198015ddf0 con 0x7f196c071390 2024-04-25T09:04:56.204 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.217+0000 7f197dffb640 1 -- 172.21.15.43:0/2672508804 <== osd.7 v2:172.21.15.193:6808/3086001447 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f198015ddf0 con 0x7f196c071390 2024-04-25T09:04:56.204 INFO:teuthology.orchestra.run.smithi043.stdout:{ 2024-04-25T09:04:56.204 INFO:teuthology.orchestra.run.smithi043.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T09:04:56.205 INFO:teuthology.orchestra.run.smithi043.stdout:} 2024-04-25T09:04:56.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.217+0000 7f19879ce640 1 -- 172.21.15.43:0/2672508804 >> [v2:172.21.15.193:6808/3086001447,v1:172.21.15.193:6809/3086001447] conn(0x7f196c071390 msgr2=0x7f196c0737b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:56.205 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.217+0000 7f19879ce640 1 --2- 172.21.15.43:0/2672508804 >> [v2:172.21.15.193:6808/3086001447,v1:172.21.15.193:6809/3086001447] conn(0x7f196c071390 0x7f196c0737b0 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.217+0000 7f19879ce640 1 -- 172.21.15.43:0/2672508804 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f196c06b4d0 msgr2=0x7f196c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:56.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.217+0000 7f19879ce640 1 --2- 172.21.15.43:0/2672508804 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f196c06b4d0 0x7f196c06d990 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f1970032000 tx=0x7f197002e000 comp rx=0 tx=0).stop 2024-04-25T09:04:56.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.217+0000 7f19879ce640 1 -- 172.21.15.43:0/2672508804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19800cbe10 msgr2=0x7f19800c7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:56.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.217+0000 7f19879ce640 1 --2- 172.21.15.43:0/2672508804 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f19800cbe10 0x7f19800c7fd0 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f1974001ce0 tx=0x7f197400eb60 comp rx=0 tx=0).stop 2024-04-25T09:04:56.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.217+0000 7f1985f6b640 1 -- 172.21.15.43:0/2672508804 reap_dead start 2024-04-25T09:04:56.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.217+0000 7f19879ce640 1 -- 172.21.15.43:0/2672508804 shutdown_connections 2024-04-25T09:04:56.206 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.217+0000 7f19879ce640 1 -- 172.21.15.43:0/2672508804 >> 172.21.15.43:0/2672508804 conn(0x7f198008d460 msgr2=0x7f198015cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:56.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.217+0000 7f19879ce640 1 -- 172.21.15.43:0/2672508804 shutdown_connections 2024-04-25T09:04:56.207 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.217+0000 7f19879ce640 1 -- 172.21.15.43:0/2672508804 wait complete. 2024-04-25T09:04:56.218 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-25T09:04:56.320 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.333+0000 7f871f693640 1 Processor -- start 2024-04-25T09:04:56.321 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.333+0000 7f871f693640 1 -- start start 2024-04-25T09:04:56.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.334+0000 7f871f693640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f871815dcf0 0x7f871806e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.334+0000 7f871f693640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f871806d4f0 0x7f871806f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.334+0000 7f871f693640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87180d0d80 con 0x7f871806ebf0 2024-04-25T09:04:56.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.334+0000 7f871f693640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f871806f590 con 0x7f871815dcf0 2024-04-25T09:04:56.322 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.334+0000 7f871f693640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f871806c050 con 0x7f871806d4f0 2024-04-25T09:04:56.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.334+0000 7f871d42f640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7f871815dcf0 0x7f871806e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:56.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.334+0000 7f871cc2e640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f871806ebf0 0x7f87181616e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56648/0 (socket says 172.21.15.43:56648) 2024-04-25T09:04:56.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.334+0000 7f871cc2e640 1 -- 172.21.15.43:0/1231549354 learned_addr learned my addr 172.21.15.43:0/1231549354 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:04:56.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.334+0000 7f871dc30640 1 --2- 172.21.15.43:0/1231549354 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f871806d4f0 0x7f871806f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:56.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.334+0000 7f871d42f640 1 -- 172.21.15.43:0/1231549354 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f871806d4f0 msgr2=0x7f871806f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:56.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.334+0000 7f871d42f640 1 --2- 172.21.15.43:0/1231549354 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f871806d4f0 0x7f871806f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.334+0000 7f871d42f640 1 -- 172.21.15.43:0/1231549354 >> v1:172.21.15.43:6789/0 conn(0x7f871806ebf0 legacy=0x7f87181616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:56.323 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.334+0000 7f870ffff640 1 -- 172.21.15.43:0/1231549354 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2667997611 0 0) 0x7f87180d0d80 con 0x7f871806ebf0 2024-04-25T09:04:56.324 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.334+0000 7f871d42f640 1 -- 172.21.15.43:0/1231549354 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87180d04e0 con 0x7f871815dcf0 2024-04-25T09:04:56.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.335+0000 7f871d42f640 1 --2- 172.21.15.43:0/1231549354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f871815dcf0 0x7f871806e530 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f8714009d90 tx=0x7f871402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=767822be38834855 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:56.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.335+0000 7f870ffff640 1 -- 172.21.15.43:0/1231549354 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8714031600 con 0x7f871815dcf0 2024-04-25T09:04:56.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.335+0000 7f870ffff640 1 -- 172.21.15.43:0/1231549354 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87140317a0 con 0x7f871815dcf0 2024-04-25T09:04:56.327 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.335+0000 7f870ffff640 1 -- 172.21.15.43:0/1231549354 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8714031b70 con 0x7f871815dcf0 2024-04-25T09:04:56.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.336+0000 7f871f693640 1 -- 172.21.15.43:0/1231549354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f871815dcf0 msgr2=0x7f871806e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:56.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.336+0000 7f871f693640 1 --2- 172.21.15.43:0/1231549354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f871815dcf0 0x7f871806e530 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f8714009d90 tx=0x7f871402a5d0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.336+0000 7f871f693640 1 -- 172.21.15.43:0/1231549354 shutdown_connections 2024-04-25T09:04:56.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.336+0000 7f871f693640 1 --2- 172.21.15.43:0/1231549354 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f871806d4f0 0x7f871806f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.336+0000 7f871f693640 1 --2- 172.21.15.43:0/1231549354 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f871815dcf0 0x7f871806e530 unknown :-1 s=CLOSED pgs=2085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.336+0000 7f871f693640 1 -- 172.21.15.43:0/1231549354 >> 172.21.15.43:0/1231549354 conn(0x7f87180d1700 msgr2=0x7f87180d1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:56.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.336+0000 7f871f693640 1 -- 172.21.15.43:0/1231549354 shutdown_connections 2024-04-25T09:04:56.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.336+0000 7f871f693640 1 -- 172.21.15.43:0/1231549354 wait complete. 2024-04-25T09:04:56.328 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.337+0000 7f871f693640 1 Processor -- start 2024-04-25T09:04:56.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.337+0000 7f871f693640 1 -- start start 2024-04-25T09:04:56.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.338+0000 7f871f693640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f871806d4f0 0x7f87181e67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.338+0000 7f871f693640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f871806ebf0 0x7f87180af4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.338+0000 7f871f693640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f871815dcf0 0x7f87180afa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.338+0000 7f871f693640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871806bce0 con 0x7f871806ebf0 2024-04-25T09:04:56.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.338+0000 7f871f693640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87180d0aa0 con 0x7f871815dcf0 2024-04-25T09:04:56.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.338+0000 7f871f693640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87180b0d50 con 0x7f871806d4f0 2024-04-25T09:04:56.329 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.338+0000 7f871cc2e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f871806ebf0 0x7f87180af4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:56.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.338+0000 7f871cc2e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f871806ebf0 0x7f87180af4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43710/0 (socket says 172.21.15.43:43710) 2024-04-25T09:04:56.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.338+0000 7f871cc2e640 1 -- 172.21.15.43:0/207335912 learned_addr learned my addr 172.21.15.43:0/207335912 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:56.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.338+0000 7f871cc2e640 1 -- 172.21.15.43:0/207335912 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f871806d4f0 msgr2=0x7f87181e67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:56.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.338+0000 7f871dc30640 1 --2- 172.21.15.43:0/207335912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f871815dcf0 0x7f87180afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:56.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.338+0000 7f871d42f640 1 --2- 172.21.15.43:0/207335912 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f871806d4f0 0x7f87181e67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:56.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.338+0000 7f871cc2e640 1 --2- 172.21.15.43:0/207335912 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f871806d4f0 0x7f87181e67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.338+0000 7f871cc2e640 1 -- 172.21.15.43:0/207335912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f871815dcf0 msgr2=0x7f87180afa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:56.330 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.338+0000 7f871cc2e640 1 --2- 172.21.15.43:0/207335912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f871815dcf0 0x7f87180afa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.338+0000 7f871cc2e640 1 -- 172.21.15.43:0/207335912 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87181f4ba0 con 0x7f871806ebf0 2024-04-25T09:04:56.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.338+0000 7f871dc30640 1 --2- 172.21.15.43:0/207335912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f871815dcf0 0x7f87180afa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:56.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.338+0000 7f871d42f640 1 --2- 172.21.15.43:0/207335912 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f871806d4f0 0x7f87181e67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T09:04:56.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.338+0000 7f871cc2e640 1 --2- 172.21.15.43:0/207335912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f871806ebf0 0x7f87180af4e0 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f870800c4d0 tx=0x7f870800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:56.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.339+0000 7f870dffb640 1 -- 172.21.15.43:0/207335912 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8708018070 con 0x7f871806ebf0 2024-04-25T09:04:56.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.339+0000 7f870dffb640 1 -- 172.21.15.43:0/207335912 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8708007500 con 0x7f871806ebf0 2024-04-25T09:04:56.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.339+0000 7f871f693640 1 -- 172.21.15.43:0/207335912 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87180c69c0 con 0x7f871806ebf0 2024-04-25T09:04:56.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.339+0000 7f870dffb640 1 -- 172.21.15.43:0/207335912 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8708004a20 con 0x7f871806ebf0 2024-04-25T09:04:56.331 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.339+0000 7f871f693640 1 -- 172.21.15.43:0/207335912 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87180c6ca0 con 0x7f871806ebf0 2024-04-25T09:04:56.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.340+0000 7f871f693640 1 -- 172.21.15.43:0/207335912 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8718074ef0 con 0x7f871806ebf0 2024-04-25T09:04:56.332 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.340+0000 7f870dffb640 1 -- 172.21.15.43:0/207335912 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f870801b070 con 0x7f871806ebf0 2024-04-25T09:04:56.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.340+0000 7f870dffb640 1 --2- 172.21.15.43:0/207335912 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f86fc06b4d0 0x7f86fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.341+0000 7f870dffb640 1 -- 172.21.15.43:0/207335912 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8708089200 con 0x7f871806ebf0 2024-04-25T09:04:56.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.341+0000 7f871d42f640 1 --2- 172.21.15.43:0/207335912 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f86fc06b4d0 0x7f86fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:56.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.341+0000 7f871d42f640 1 --2- 172.21.15.43:0/207335912 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f86fc06b4d0 0x7f86fc06d990 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f8714034000 tx=0x7f8714033040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:56.333 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.345+0000 7f870dffb640 1 -- 172.21.15.43:0/207335912 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8708058380 con 0x7f871806ebf0 2024-04-25T09:04:56.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.525+0000 7f871f693640 1 -- 172.21.15.43:0/207335912 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f87181593a0 con 0x7f86fc06b4d0 2024-04-25T09:04:56.513 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.526+0000 7f870dffb640 1 -- 172.21.15.43:0/207335912 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f87181593a0 con 0x7f86fc06b4d0 2024-04-25T09:04:56.513 INFO:teuthology.orchestra.run.smithi043.stderr:instructed osd(s) 7 to deep-scrub 2024-04-25T09:04:56.515 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.528+0000 7f871f693640 1 -- 172.21.15.43:0/207335912 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f86fc06b4d0 msgr2=0x7f86fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:56.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.528+0000 7f871f693640 1 --2- 172.21.15.43:0/207335912 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f86fc06b4d0 0x7f86fc06d990 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f8714034000 tx=0x7f8714033040 comp rx=0 tx=0).stop 2024-04-25T09:04:56.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.528+0000 7f871f693640 1 -- 172.21.15.43:0/207335912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f871806ebf0 msgr2=0x7f87180af4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:56.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.528+0000 7f871f693640 1 --2- 172.21.15.43:0/207335912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f871806ebf0 0x7f87180af4e0 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f870800c4d0 tx=0x7f870800c990 comp rx=0 tx=0).stop 2024-04-25T09:04:56.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.528+0000 7f871f693640 1 -- 172.21.15.43:0/207335912 shutdown_connections 2024-04-25T09:04:56.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.528+0000 7f871f693640 1 --2- 172.21.15.43:0/207335912 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f86fc06b4d0 0x7f86fc06d990 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.528+0000 7f871f693640 1 --2- 172.21.15.43:0/207335912 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f871815dcf0 0x7f87180afa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.516 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.528+0000 7f871f693640 1 --2- 172.21.15.43:0/207335912 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f871806ebf0 0x7f87180af4e0 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.528+0000 7f871f693640 1 --2- 172.21.15.43:0/207335912 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f871806d4f0 0x7f87181e67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.528+0000 7f871f693640 1 -- 172.21.15.43:0/207335912 >> 172.21.15.43:0/207335912 conn(0x7f87180d1700 msgr2=0x7f87180b1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:56.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.528+0000 7f871f693640 1 -- 172.21.15.43:0/207335912 shutdown_connections 2024-04-25T09:04:56.517 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.528+0000 7f871f693640 1 -- 172.21.15.43:0/207335912 wait complete. 2024-04-25T09:04:56.526 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:04:56.628 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.640+0000 7f3f25afc640 1 Processor -- start 2024-04-25T09:04:56.630 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.640+0000 7f3f25afc640 1 -- start start 2024-04-25T09:04:56.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.641+0000 7f3f25afc640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f3f2015df50 0x7f3f201619f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.632 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.641+0000 7f3f25afc640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3f20161f30 0x7f3f201623a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.641+0000 7f3f25afc640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f200d24a0 con 0x7f3f200c7cf0 2024-04-25T09:04:56.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.641+0000 7f3f25afc640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f2015e330 con 0x7f3f2015df50 2024-04-25T09:04:56.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.641+0000 7f3f25afc640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f200d2740 con 0x7f3f20161f30 2024-04-25T09:04:56.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.641+0000 7f3f1f7fe640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f3f2015df50 0x7f3f201619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:56.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.641+0000 7f3f1effd640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3f20161f30 0x7f3f201623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:56.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.641+0000 7f3f1f7fe640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f3f2015df50 0x7f3f201619f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3300/0 says I am v2:172.21.15.43:37720/0 (socket says 172.21.15.43:37720) 2024-04-25T09:04:56.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.641+0000 7f3f1f7fe640 1 -- 172.21.15.43:0/2234822991 learned_addr learned my addr 172.21.15.43:0/2234822991 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:56.633 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.641+0000 7f3f1effd640 1 --2- 172.21.15.43:0/2234822991 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3f20161f30 0x7f3f201623a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.193:3301/0 says I am v2:172.21.15.43:33834/0 (socket says 172.21.15.43:33834) 2024-04-25T09:04:56.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.642+0000 7f3f1f7fe640 1 -- 172.21.15.43:0/2234822991 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3f20161f30 msgr2=0x7f3f201623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:04:56.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.642+0000 7f3f1f7fe640 1 --2- 172.21.15.43:0/2234822991 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3f20161f30 0x7f3f201623a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.642+0000 7f3f1f7fe640 1 -- 172.21.15.43:0/2234822991 >> v1:172.21.15.193:6789/0 conn(0x7f3f200c7cf0 legacy=0x7f3f200c8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:56.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.642+0000 7f3f1e7fc640 1 -- 172.21.15.43:0/2234822991 <== mon.1 v1:172.21.15.193:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1328873537 0 0) 0x7f3f200d24a0 con 0x7f3f200c7cf0 2024-04-25T09:04:56.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.642+0000 7f3f1f7fe640 1 -- 172.21.15.43:0/2234822991 --> v2:172.21.15.193:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f200d2a20 con 0x7f3f2015df50 2024-04-25T09:04:56.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.643+0000 7f3f1f7fe640 1 --2- 172.21.15.43:0/2234822991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f2015df50 0x7f3f201619f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f3f10009870 tx=0x7f3f1002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47d7da63392719e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:56.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.643+0000 7f3f1e7fc640 1 -- 172.21.15.43:0/2234822991 <== mon.1 v2:172.21.15.193:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f10037070 con 0x7f3f2015df50 2024-04-25T09:04:56.634 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.643+0000 7f3f1e7fc640 1 -- 172.21.15.43:0/2234822991 <== mon.1 v2:172.21.15.193:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f10032440 con 0x7f3f2015df50 2024-04-25T09:04:56.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.643+0000 7f3f1e7fc640 1 -- 172.21.15.43:0/2234822991 <== mon.1 v2:172.21.15.193:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f10032810 con 0x7f3f2015df50 2024-04-25T09:04:56.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.643+0000 7f3f25afc640 1 -- 172.21.15.43:0/2234822991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f2015df50 msgr2=0x7f3f201619f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:56.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.643+0000 7f3f25afc640 1 --2- 172.21.15.43:0/2234822991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f2015df50 0x7f3f201619f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f3f10009870 tx=0x7f3f1002a270 comp rx=0 tx=0).stop 2024-04-25T09:04:56.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.643+0000 7f3f25afc640 1 -- 172.21.15.43:0/2234822991 shutdown_connections 2024-04-25T09:04:56.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.643+0000 7f3f25afc640 1 --2- 172.21.15.43:0/2234822991 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3f20161f30 0x7f3f201623a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.643+0000 7f3f25afc640 1 --2- 172.21.15.43:0/2234822991 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f2015df50 0x7f3f201619f0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.643+0000 7f3f25afc640 1 -- 172.21.15.43:0/2234822991 >> 172.21.15.43:0/2234822991 conn(0x7f3f200d3a30 msgr2=0x7f3f200d3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:56.635 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.643+0000 7f3f25afc640 1 -- 172.21.15.43:0/2234822991 shutdown_connections 2024-04-25T09:04:56.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.643+0000 7f3f25afc640 1 -- 172.21.15.43:0/2234822991 wait complete. 2024-04-25T09:04:56.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.644+0000 7f3f25afc640 1 Processor -- start 2024-04-25T09:04:56.636 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.645+0000 7f3f25afc640 1 -- start start 2024-04-25T09:04:56.638 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.645+0000 7f3f25afc640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f200c7cf0 0x7f3f20161750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.645+0000 7f3f25afc640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f2015df50 0x7f3f2009fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.645+0000 7f3f25afc640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3f20161f30 0x7f3f201f3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.645+0000 7f3f25afc640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f200d2190 con 0x7f3f200c7cf0 2024-04-25T09:04:56.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.645+0000 7f3f25afc640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f200c77d0 con 0x7f3f2015df50 2024-04-25T09:04:56.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.645+0000 7f3f25afc640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f200cf670 con 0x7f3f20161f30 2024-04-25T09:04:56.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.646+0000 7f3f1f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f200c7cf0 0x7f3f20161750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:56.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.646+0000 7f3f1f7fe640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f200c7cf0 0x7f3f20161750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43720/0 (socket says 172.21.15.43:43720) 2024-04-25T09:04:56.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.646+0000 7f3f1f7fe640 1 -- 172.21.15.43:0/1487344489 learned_addr learned my addr 172.21.15.43:0/1487344489 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:04:56.639 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.646+0000 7f3f1effd640 1 --2- 172.21.15.43:0/1487344489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f2015df50 0x7f3f2009fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:56.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.646+0000 7f3f1ffff640 1 --2- 172.21.15.43:0/1487344489 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3f20161f30 0x7f3f201f3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:56.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.646+0000 7f3f1f7fe640 1 -- 172.21.15.43:0/1487344489 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3f20161f30 msgr2=0x7f3f201f3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:56.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.646+0000 7f3f1f7fe640 1 --2- 172.21.15.43:0/1487344489 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3f20161f30 0x7f3f201f3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.646+0000 7f3f1f7fe640 1 -- 172.21.15.43:0/1487344489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f2015df50 msgr2=0x7f3f2009fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:56.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.646+0000 7f3f1f7fe640 1 --2- 172.21.15.43:0/1487344489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f2015df50 0x7f3f2009fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.646+0000 7f3f1f7fe640 1 -- 172.21.15.43:0/1487344489 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f2015f890 con 0x7f3f200c7cf0 2024-04-25T09:04:56.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.646+0000 7f3f1f7fe640 1 --2- 172.21.15.43:0/1487344489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f200c7cf0 0x7f3f20161750 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f3f10035040 tx=0x7f3f100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:56.640 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.646+0000 7f3f07fff640 1 -- 172.21.15.43:0/1487344489 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f10037040 con 0x7f3f200c7cf0 2024-04-25T09:04:56.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.646+0000 7f3f07fff640 1 -- 172.21.15.43:0/1487344489 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f10004030 con 0x7f3f200c7cf0 2024-04-25T09:04:56.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.646+0000 7f3f25afc640 1 -- 172.21.15.43:0/1487344489 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f2015fb70 con 0x7f3f200c7cf0 2024-04-25T09:04:56.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.646+0000 7f3f07fff640 1 -- 172.21.15.43:0/1487344489 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f10004420 con 0x7f3f200c7cf0 2024-04-25T09:04:56.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.646+0000 7f3f25afc640 1 -- 172.21.15.43:0/1487344489 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f20160130 con 0x7f3f200c7cf0 2024-04-25T09:04:56.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.647+0000 7f3f25afc640 1 -- 172.21.15.43:0/1487344489 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f200680f0 con 0x7f3f200c7cf0 2024-04-25T09:04:56.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.648+0000 7f3f07fff640 1 -- 172.21.15.43:0/1487344489 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3f1003a070 con 0x7f3f200c7cf0 2024-04-25T09:04:56.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.648+0000 7f3f07fff640 1 --2- 172.21.15.43:0/1487344489 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3f0c06b4d0 0x7f3f0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:04:56.641 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.648+0000 7f3f07fff640 1 -- 172.21.15.43:0/1487344489 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3f100aa2d0 con 0x7f3f200c7cf0 2024-04-25T09:04:56.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.648+0000 7f3f1effd640 1 --2- 172.21.15.43:0/1487344489 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3f0c06b4d0 0x7f3f0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:04:56.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.649+0000 7f3f1effd640 1 --2- 172.21.15.43:0/1487344489 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3f0c06b4d0 0x7f3f0c06d990 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f3f2015bfd0 tx=0x7f3f14009190 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:04:56.642 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.653+0000 7f3f07fff640 1 -- 172.21.15.43:0/1487344489 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3f10041020 con 0x7f3f200c7cf0 2024-04-25T09:04:56.817 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.830+0000 7f3f25afc640 1 -- 172.21.15.43:0/1487344489 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3f200ca400 con 0x7f3f0c06b4d0 2024-04-25T09:04:56.818 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.831+0000 7f3f07fff640 1 -- 172.21.15.43:0/1487344489 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13517 (secure 0 0 0) 0x7f3f200ca400 con 0x7f3f0c06b4d0 2024-04-25T09:04:56.818 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:04:56.819 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-25T09:04:56.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.833+0000 7f3f25afc640 1 -- 172.21.15.43:0/1487344489 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3f0c06b4d0 msgr2=0x7f3f0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:56.820 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.833+0000 7f3f25afc640 1 --2- 172.21.15.43:0/1487344489 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3f0c06b4d0 0x7f3f0c06d990 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f3f2015bfd0 tx=0x7f3f14009190 comp rx=0 tx=0).stop 2024-04-25T09:04:56.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.833+0000 7f3f25afc640 1 -- 172.21.15.43:0/1487344489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f200c7cf0 msgr2=0x7f3f20161750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:04:56.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.833+0000 7f3f25afc640 1 --2- 172.21.15.43:0/1487344489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f200c7cf0 0x7f3f20161750 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f3f10035040 tx=0x7f3f100046c0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.833+0000 7f3f25afc640 1 -- 172.21.15.43:0/1487344489 shutdown_connections 2024-04-25T09:04:56.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.833+0000 7f3f25afc640 1 --2- 172.21.15.43:0/1487344489 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f3f20161f30 0x7f3f201f3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.833+0000 7f3f25afc640 1 --2- 172.21.15.43:0/1487344489 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f3f0c06b4d0 0x7f3f0c06d990 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.833+0000 7f3f25afc640 1 --2- 172.21.15.43:0/1487344489 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f3f2015df50 0x7f3f2009fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.821 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.833+0000 7f3f25afc640 1 --2- 172.21.15.43:0/1487344489 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f3f200c7cf0 0x7f3f20161750 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:04:56.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.833+0000 7f3f25afc640 1 -- 172.21.15.43:0/1487344489 >> 172.21.15.43:0/1487344489 conn(0x7f3f200d3a30 msgr2=0x7f3f2015cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:04:56.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.833+0000 7f3f25afc640 1 -- 172.21.15.43:0/1487344489 shutdown_connections 2024-04-25T09:04:56.822 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:04:56.833+0000 7f3f25afc640 1 -- 172.21.15.43:0/1487344489 wait complete. 2024-04-25T09:04:56.831 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":464,"stamp":"2024-04-25T09:04:55.844832+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":53,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":507512,"kb_used_data":15112,"kb_used_omap":12,"kb_used_meta":492275,"kb_avail":754467208,"statfs":{"total":773094113280,"available":772574420992,"internally_reserved":0,"allocated":15474688,"data_stored":7424664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":504090161},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1073741824,"num_objects":-1024,"num_object_clones":0,"num_object_copies":-2048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-1024,"num_write_kb":-1048576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.048948"},"pg_stats":[{"pgid":"1.0","version":"15'117","reported_seq":398,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T09:04:48.803229+0000","last_change":"2024-04-25T08:54:00.818044+0000","last_active":"2024-04-25T09:04:48.803229+0000","last_peered":"2024-04-25T09:04:48.803229+0000","last_clean":"2024-04-25T09:04:48.803229+0000","last_became_active":"2024-04-25T08:54:00.817098+0000","last_became_peered":"2024-04-25T08:54:00.817098+0000","last_unstale":"2024-04-25T09:04:48.803229+0000","last_undegraded":"2024-04-25T09:04:48.803229+0000","last_fullsized":"2024-04-25T09:04:48.803229+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T08:53:59.739677+0000","last_clean_scrub_stamp":"2024-04-25T08:53:59.739677+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:48.129513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574981,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47396,"kb_used_data":2324,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324444,"statfs":{"total":96636764160,"available":96588230656,"internally_reserved":0,"allocated":2379776,"data_stored":1370859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574982,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":69020,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302820,"statfs":{"total":96636764160,"available":96566087680,"internally_reserved":0,"allocated":1785856,"data_stored":780491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574984,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55132,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316708,"statfs":{"total":96636764160,"available":96580308992,"internally_reserved":0,"allocated":1785856,"data_stored":780491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574982,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":73180,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298660,"statfs":{"total":96636764160,"available":96561827840,"internally_reserved":0,"allocated":1785856,"data_stored":780491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73136582},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574982,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":77344,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294496,"statfs":{"total":96636764160,"available":96557563904,"internally_reserved":0,"allocated":1785856,"data_stored":780491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574982,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50972,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320868,"statfs":{"total":96636764160,"available":96584568832,"internally_reserved":0,"allocated":1785856,"data_stored":780491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574982,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":82912,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94288928,"statfs":{"total":96636764160,"available":96551862272,"internally_reserved":0,"allocated":1785856,"data_stored":780491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574983,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51556,"kb_used_data":2324,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320284,"statfs":{"total":96636764160,"available":96583970816,"internally_reserved":0,"allocated":2379776,"data_stored":1370859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:04:56.834 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-25T08:53:59.739677+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=8, tm_min=53, tm_sec=59, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=9, tm_min=4, tm_sec=51, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T09:04:56.834 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-25T09:05:16.638 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:05:16.641 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:05:16.835 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T09:05:16.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.961+0000 7fad7b6e1640 1 Processor -- start 2024-04-25T09:05:16.949 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.961+0000 7fad7b6e1640 1 -- start start 2024-04-25T09:05:16.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.962+0000 7fad7b6e1640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7fad7415dce0 0x7fad7415ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:16.951 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.962+0000 7fad7b6e1640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fad740ce550 0x7fad740d03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:16.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.962+0000 7fad7b6e1640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fad740ce9c0 0x7fad740d0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:16.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.962+0000 7fad7b6e1640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad740cee80 con 0x7fad740ce550 2024-04-25T09:05:16.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.962+0000 7fad7b6e1640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad740cf160 con 0x7fad7415dce0 2024-04-25T09:05:16.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.962+0000 7fad7b6e1640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad740cf440 con 0x7fad740ce9c0 2024-04-25T09:05:16.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.962+0000 7fad78c7c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fad740ce550 0x7fad740d03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:16.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.962+0000 7fad78c7c640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7fad740ce550 0x7fad740d03e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43050/0 (socket says 172.21.15.43:43050) 2024-04-25T09:05:16.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.962+0000 7fad78c7c640 1 -- 172.21.15.43:0/2250385245 learned_addr learned my addr 172.21.15.43:0/2250385245 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:05:16.952 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.962+0000 7fad79c7e640 1 --2- 172.21.15.43:0/2250385245 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fad740ce9c0 0x7fad740d0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:16.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.962+0000 7fad7947d640 1 --2- 172.21.15.43:0/2250385245 >> v2:172.21.15.193:3300/0 conn(0x7fad7415dce0 0x7fad7415ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:16.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.962+0000 7fad78c7c640 1 -- 172.21.15.43:0/2250385245 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fad740ce9c0 msgr2=0x7fad740d0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:16.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.962+0000 7fad78c7c640 1 --2- 172.21.15.43:0/2250385245 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fad740ce9c0 0x7fad740d0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:16.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.962+0000 7fad78c7c640 1 -- 172.21.15.43:0/2250385245 >> v2:172.21.15.193:3300/0 conn(0x7fad7415dce0 msgr2=0x7fad7415ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:16.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.962+0000 7fad78c7c640 1 --2- 172.21.15.43:0/2250385245 >> v2:172.21.15.193:3300/0 conn(0x7fad7415dce0 0x7fad7415ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:16.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.962+0000 7fad78c7c640 1 -- 172.21.15.43:0/2250385245 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad740cf720 con 0x7fad740ce550 2024-04-25T09:05:16.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.962+0000 7fad7947d640 1 --2- 172.21.15.43:0/2250385245 >> v2:172.21.15.193:3300/0 conn(0x7fad7415dce0 0x7fad7415ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T09:05:16.953 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.963+0000 7fad78c7c640 1 --2- 172.21.15.43:0/2250385245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad740ce550 0x7fad740d03e0 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7fad7409d3c0 tx=0x7fad6402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eeecc2be3824c92 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:16.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.963+0000 7fad6bfff640 1 -- 172.21.15.43:0/2250385245 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad64037070 con 0x7fad740ce550 2024-04-25T09:05:16.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.963+0000 7fad6bfff640 1 -- 172.21.15.43:0/2250385245 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad64032440 con 0x7fad740ce550 2024-04-25T09:05:16.954 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.963+0000 7fad6bfff640 1 -- 172.21.15.43:0/2250385245 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad64032810 con 0x7fad740ce550 2024-04-25T09:05:16.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.964+0000 7fad7b6e1640 1 -- 172.21.15.43:0/2250385245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad740ce550 msgr2=0x7fad740d03e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:16.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.964+0000 7fad7b6e1640 1 --2- 172.21.15.43:0/2250385245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad740ce550 0x7fad740d03e0 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7fad7409d3c0 tx=0x7fad6402a140 comp rx=0 tx=0).stop 2024-04-25T09:05:16.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.964+0000 7fad7b6e1640 1 -- 172.21.15.43:0/2250385245 shutdown_connections 2024-04-25T09:05:16.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.964+0000 7fad7b6e1640 1 --2- 172.21.15.43:0/2250385245 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fad740ce9c0 0x7fad740d0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:16.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.964+0000 7fad7b6e1640 1 --2- 172.21.15.43:0/2250385245 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad740ce550 0x7fad740d03e0 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:16.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.964+0000 7fad7b6e1640 1 --2- 172.21.15.43:0/2250385245 >> v2:172.21.15.193:3300/0 conn(0x7fad7415dce0 0x7fad7415ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:16.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.964+0000 7fad7b6e1640 1 -- 172.21.15.43:0/2250385245 >> 172.21.15.43:0/2250385245 conn(0x7fad7409cd00 msgr2=0x7fad740d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:16.957 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.964+0000 7fad7b6e1640 1 -- 172.21.15.43:0/2250385245 shutdown_connections 2024-04-25T09:05:16.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.964+0000 7fad7b6e1640 1 -- 172.21.15.43:0/2250385245 wait complete. 2024-04-25T09:05:16.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.965+0000 7fad7b6e1640 1 Processor -- start 2024-04-25T09:05:16.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.965+0000 7fad7b6e1640 1 -- start start 2024-04-25T09:05:16.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.966+0000 7fad7b6e1640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fad740ce550 0x7fad7415bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:16.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.966+0000 7fad7b6e1640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad740ce9c0 0x7fad740cb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:16.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.966+0000 7fad7b6e1640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad7415dce0 0x7fad740b0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:16.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.966+0000 7fad7b6e1640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad74160980 con 0x7fad7415dce0 2024-04-25T09:05:16.958 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.966+0000 7fad7b6e1640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad7415e0c0 con 0x7fad740ce9c0 2024-04-25T09:05:16.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.966+0000 7fad7b6e1640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad74160b00 con 0x7fad740ce550 2024-04-25T09:05:16.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.966+0000 7fad79c7e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad7415dce0 0x7fad740b0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:16.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.966+0000 7fad79c7e640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad7415dce0 0x7fad740b0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43062/0 (socket says 172.21.15.43:43062) 2024-04-25T09:05:16.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.966+0000 7fad79c7e640 1 -- 172.21.15.43:0/1143962999 learned_addr learned my addr 172.21.15.43:0/1143962999 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:05:16.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.966+0000 7fad7947d640 1 --2- 172.21.15.43:0/1143962999 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fad740ce550 0x7fad7415bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:16.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.966+0000 7fad78c7c640 1 --2- 172.21.15.43:0/1143962999 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad740ce9c0 0x7fad740cb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:16.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.966+0000 7fad79c7e640 1 -- 172.21.15.43:0/1143962999 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fad740ce550 msgr2=0x7fad7415bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:16.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.966+0000 7fad79c7e640 1 --2- 172.21.15.43:0/1143962999 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fad740ce550 0x7fad7415bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:16.959 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.966+0000 7fad79c7e640 1 -- 172.21.15.43:0/1143962999 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad740ce9c0 msgr2=0x7fad740cb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:16.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.967+0000 7fad79c7e640 1 --2- 172.21.15.43:0/1143962999 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad740ce9c0 0x7fad740cb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:16.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.967+0000 7fad79c7e640 1 -- 172.21.15.43:0/1143962999 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad740cbfd0 con 0x7fad7415dce0 2024-04-25T09:05:16.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.967+0000 7fad79c7e640 1 --2- 172.21.15.43:0/1143962999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad7415dce0 0x7fad740b0660 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7fad6c007e60 tx=0x7fad6c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:16.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.967+0000 7fad69ffb640 1 -- 172.21.15.43:0/1143962999 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c014420 con 0x7fad7415dce0 2024-04-25T09:05:16.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.967+0000 7fad69ffb640 1 -- 172.21.15.43:0/1143962999 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad6c019070 con 0x7fad7415dce0 2024-04-25T09:05:16.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.967+0000 7fad69ffb640 1 -- 172.21.15.43:0/1143962999 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6c014800 con 0x7fad7415dce0 2024-04-25T09:05:16.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.967+0000 7fad7b6e1640 1 -- 172.21.15.43:0/1143962999 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad740cc2b0 con 0x7fad7415dce0 2024-04-25T09:05:16.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.967+0000 7fad7b6e1640 1 -- 172.21.15.43:0/1143962999 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad740cc870 con 0x7fad7415dce0 2024-04-25T09:05:16.960 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.968+0000 7fad7b6e1640 1 -- 172.21.15.43:0/1143962999 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad740680d0 con 0x7fad7415dce0 2024-04-25T09:05:16.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.969+0000 7fad69ffb640 1 -- 172.21.15.43:0/1143962999 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fad6c0149a0 con 0x7fad7415dce0 2024-04-25T09:05:16.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.969+0000 7fad69ffb640 1 --2- 172.21.15.43:0/1143962999 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fad5806b4d0 0x7fad5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:16.961 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.969+0000 7fad69ffb640 1 -- 172.21.15.43:0/1143962999 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fad6c08a6c0 con 0x7fad7415dce0 2024-04-25T09:05:16.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.969+0000 7fad7947d640 1 --2- 172.21.15.43:0/1143962999 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fad5806b4d0 0x7fad5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:16.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.970+0000 7fad7947d640 1 --2- 172.21.15.43:0/1143962999 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fad5806b4d0 0x7fad5806d990 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fad740b92a0 tx=0x7fad70008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:16.962 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:16.974+0000 7fad69ffb640 1 -- 172.21.15.43:0/1143962999 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fad6c0598d0 con 0x7fad7415dce0 2024-04-25T09:05:17.138 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.150+0000 7fad7b6e1640 1 -- 172.21.15.43:0/1143962999 --> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fad74074ef0 con 0x7fad5806b4d0 2024-04-25T09:05:17.139 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.151+0000 7fad69ffb640 1 -- 172.21.15.43:0/1143962999 <== mgr.4108 v2:172.21.15.193:6832/1244825175 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13379 (secure 0 0 0) 0x7fad74074ef0 con 0x7fad5806b4d0 2024-04-25T09:05:17.140 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:05:17.140 INFO:teuthology.orchestra.run.smithi043.stderr:dumped all 2024-04-25T09:05:17.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.153+0000 7fad7b6e1640 1 -- 172.21.15.43:0/1143962999 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fad5806b4d0 msgr2=0x7fad5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:17.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.153+0000 7fad7b6e1640 1 --2- 172.21.15.43:0/1143962999 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fad5806b4d0 0x7fad5806d990 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fad740b92a0 tx=0x7fad70008040 comp rx=0 tx=0).stop 2024-04-25T09:05:17.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.153+0000 7fad7b6e1640 1 -- 172.21.15.43:0/1143962999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad7415dce0 msgr2=0x7fad740b0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:17.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.153+0000 7fad7b6e1640 1 --2- 172.21.15.43:0/1143962999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad7415dce0 0x7fad740b0660 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7fad6c007e60 tx=0x7fad6c00e4b0 comp rx=0 tx=0).stop 2024-04-25T09:05:17.142 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.153+0000 7fad7b6e1640 1 -- 172.21.15.43:0/1143962999 shutdown_connections 2024-04-25T09:05:17.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.153+0000 7fad7b6e1640 1 --2- 172.21.15.43:0/1143962999 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7fad7415dce0 0x7fad740b0660 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:17.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.153+0000 7fad7b6e1640 1 --2- 172.21.15.43:0/1143962999 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7fad5806b4d0 0x7fad5806d990 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:17.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.153+0000 7fad7b6e1640 1 --2- 172.21.15.43:0/1143962999 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7fad740ce9c0 0x7fad740cb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:17.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.153+0000 7fad7b6e1640 1 --2- 172.21.15.43:0/1143962999 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7fad740ce550 0x7fad7415bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:17.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.153+0000 7fad7b6e1640 1 -- 172.21.15.43:0/1143962999 >> 172.21.15.43:0/1143962999 conn(0x7fad7409cd00 msgr2=0x7fad7406c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:17.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.154+0000 7fad7b6e1640 1 -- 172.21.15.43:0/1143962999 shutdown_connections 2024-04-25T09:05:17.143 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.154+0000 7fad7b6e1640 1 -- 172.21.15.43:0/1143962999 wait complete. 2024-04-25T09:05:17.153 INFO:teuthology.orchestra.run.smithi043.stdout:{"pg_ready":true,"pg_map":{"version":474,"stamp":"2024-04-25T09:05:15.848756+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":507496,"kb_used_data":15112,"kb_used_omap":12,"kb_used_meta":492275,"kb_avail":754467224,"statfs":{"total":773094113280,"available":772574437376,"internally_reserved":0,"allocated":15474688,"data_stored":7424664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":504090161},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002398"},"pg_stats":[{"pgid":"1.0","version":"142'119","reported_seq":409,"reported_epoch":142,"state":"active+clean","last_fresh":"2024-04-25T09:04:57.016940+0000","last_change":"2024-04-25T09:04:57.016975+0000","last_active":"2024-04-25T09:04:57.016940+0000","last_peered":"2024-04-25T09:04:57.016940+0000","last_clean":"2024-04-25T09:04:57.016940+0000","last_became_active":"2024-04-25T08:54:00.817098+0000","last_became_peered":"2024-04-25T08:54:00.817098+0000","last_unstale":"2024-04-25T09:04:57.016940+0000","last_undegraded":"2024-04-25T09:04:57.016940+0000","last_fullsized":"2024-04-25T09:04:57.016940+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"142'119","last_scrub_stamp":"2024-04-25T09:04:57.016842+0000","last_deep_scrub":"142'119","last_deep_scrub_stamp":"2024-04-25T09:04:57.016842+0000","last_clean_scrub_stamp":"2024-04-25T09:04:57.016842+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:31.721659+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574985,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47388,"kb_used_data":2324,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324452,"statfs":{"total":96636764160,"available":96588238848,"internally_reserved":0,"allocated":2379776,"data_stored":1370859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":46135754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574986,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":69020,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94302820,"statfs":{"total":96636764160,"available":96566087680,"internally_reserved":0,"allocated":1785856,"data_stored":780491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":68876742},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574988,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55132,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316708,"statfs":{"total":96636764160,"available":96580308992,"internally_reserved":0,"allocated":1785856,"data_stored":780491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574986,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":73180,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94298660,"statfs":{"total":96636764160,"available":96561827840,"internally_reserved":0,"allocated":1785856,"data_stored":780491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":73136582},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574986,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":77344,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294496,"statfs":{"total":96636764160,"available":96557563904,"internally_reserved":0,"allocated":1785856,"data_stored":780491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":77396423},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574986,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":50972,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320868,"statfs":{"total":96636764160,"available":96584568832,"internally_reserved":0,"allocated":1785856,"data_stored":780491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574986,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":82912,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":81150,"kb_avail":94288928,"statfs":{"total":96636764160,"available":96551862272,"internally_reserved":0,"allocated":1785856,"data_stored":780491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83098053},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574987,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51548,"kb_used_data":2324,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320292,"statfs":{"total":96636764160,"available":96583979008,"internally_reserved":0,"allocated":2379776,"data_stored":1370859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T09:05:17.154 DEBUG:teuthology.orchestra.run.smithi043:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T09:05:17.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.266+0000 7ff2b3ec4640 1 Processor -- start 2024-04-25T09:05:17.255 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.266+0000 7ff2b3ec4640 1 -- start start 2024-04-25T09:05:17.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.267+0000 7ff2b3ec4640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff2ac0b5560 0x7ff2ac0b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:17.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.267+0000 7ff2b3ec4640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff2ac0d2230 0x7ff2ac0d2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:17.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.267+0000 7ff2b3ec4640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2ac162610 con 0x7ff2ac15e090 2024-04-25T09:05:17.256 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.267+0000 7ff2b3ec4640 1 -- --> v2:172.21.15.43:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2ac0d1da0 con 0x7ff2ac0d2230 2024-04-25T09:05:17.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.267+0000 7ff2b3ec4640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac0d1750 con 0x7ff2ac0b5560 2024-04-25T09:05:17.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.267+0000 7ff2b2461640 1 --2- >> v2:172.21.15.43:3300/0 conn(0x7ff2ac0d2230 0x7ff2ac0d2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:17.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.267+0000 7ff2b1c60640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7ff2ac15e090 0x7ff2ac161aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56082/0 (socket says 172.21.15.43:56082) 2024-04-25T09:05:17.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.267+0000 7ff2b1c60640 1 -- 172.21.15.43:0/2639004351 learned_addr learned my addr 172.21.15.43:0/2639004351 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:05:17.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.268+0000 7ff2b145f640 1 --2- 172.21.15.43:0/2639004351 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff2ac0b5560 0x7ff2ac0b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:17.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.268+0000 7ff2b2461640 1 -- 172.21.15.43:0/2639004351 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff2ac0b5560 msgr2=0x7ff2ac0b59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:17.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.268+0000 7ff2b2461640 1 --2- 172.21.15.43:0/2639004351 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff2ac0b5560 0x7ff2ac0b59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:17.257 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.268+0000 7ff2b2461640 1 -- 172.21.15.43:0/2639004351 >> v1:172.21.15.43:6789/0 conn(0x7ff2ac15e090 legacy=0x7ff2ac161aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:17.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.268+0000 7ff2b0c5e640 1 -- 172.21.15.43:0/2639004351 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3539674816 0 0) 0x7ff2ac162610 con 0x7ff2ac15e090 2024-04-25T09:05:17.258 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.268+0000 7ff2b2461640 1 -- 172.21.15.43:0/2639004351 --> v2:172.21.15.43:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2ac0d1a30 con 0x7ff2ac0d2230 2024-04-25T09:05:17.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.268+0000 7ff2b2461640 1 --2- 172.21.15.43:0/2639004351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2ac0d2230 0x7ff2ac0d2690 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7ff29c009d90 tx=0x7ff29c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=248d65f5ec0cc241 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:17.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.269+0000 7ff2b0c5e640 1 -- 172.21.15.43:0/2639004351 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c007740 con 0x7ff2ac0d2230 2024-04-25T09:05:17.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.269+0000 7ff2b0c5e640 1 -- 172.21.15.43:0/2639004351 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29c0078e0 con 0x7ff2ac0d2230 2024-04-25T09:05:17.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.269+0000 7ff2b0c5e640 1 -- 172.21.15.43:0/2639004351 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c007cb0 con 0x7ff2ac0d2230 2024-04-25T09:05:17.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.269+0000 7ff2b3ec4640 1 -- 172.21.15.43:0/2639004351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2ac0d2230 msgr2=0x7ff2ac0d2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:17.261 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.269+0000 7ff2b3ec4640 1 --2- 172.21.15.43:0/2639004351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2ac0d2230 0x7ff2ac0d2690 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7ff29c009d90 tx=0x7ff29c02a6f0 comp rx=0 tx=0).stop 2024-04-25T09:05:17.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.269+0000 7ff2b3ec4640 1 -- 172.21.15.43:0/2639004351 shutdown_connections 2024-04-25T09:05:17.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.269+0000 7ff2b3ec4640 1 --2- 172.21.15.43:0/2639004351 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2ac0d2230 0x7ff2ac0d2690 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:17.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.269+0000 7ff2b3ec4640 1 --2- 172.21.15.43:0/2639004351 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff2ac0b5560 0x7ff2ac0b59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:17.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.269+0000 7ff2b3ec4640 1 -- 172.21.15.43:0/2639004351 >> 172.21.15.43:0/2639004351 conn(0x7ff2ac0d0560 msgr2=0x7ff2ac0d0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:17.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.269+0000 7ff2b3ec4640 1 -- 172.21.15.43:0/2639004351 shutdown_connections 2024-04-25T09:05:17.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.269+0000 7ff2b3ec4640 1 -- 172.21.15.43:0/2639004351 wait complete. 2024-04-25T09:05:17.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.270+0000 7ff2b3ec4640 1 Processor -- start 2024-04-25T09:05:17.262 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.270+0000 7ff2b3ec4640 1 -- start start 2024-04-25T09:05:17.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.271+0000 7ff2b3ec4640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2ac0b5560 0x7ff2ac0ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:17.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.271+0000 7ff2b3ec4640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff2ac0d2230 0x7ff2ac1f4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:17.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.271+0000 7ff2b3ec4640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2ac15e090 0x7ff2ac1f45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:17.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.271+0000 7ff2b3ec4640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac1621b0 con 0x7ff2ac15e090 2024-04-25T09:05:17.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.271+0000 7ff2b3ec4640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac0b3280 con 0x7ff2ac0b5560 2024-04-25T09:05:17.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.271+0000 7ff2b3ec4640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac162330 con 0x7ff2ac0d2230 2024-04-25T09:05:17.263 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.271+0000 7ff2b2461640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2ac15e090 0x7ff2ac1f45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:17.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.271+0000 7ff2b2461640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2ac15e090 0x7ff2ac1f45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43082/0 (socket says 172.21.15.43:43082) 2024-04-25T09:05:17.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.271+0000 7ff2b2461640 1 -- 172.21.15.43:0/1336190745 learned_addr learned my addr 172.21.15.43:0/1336190745 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:05:17.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.271+0000 7ff2b145f640 1 --2- 172.21.15.43:0/1336190745 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff2ac0d2230 0x7ff2ac1f4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:17.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.272+0000 7ff2b2461640 1 -- 172.21.15.43:0/1336190745 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff2ac0d2230 msgr2=0x7ff2ac1f4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:17.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.272+0000 7ff2b2461640 1 --2- 172.21.15.43:0/1336190745 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff2ac0d2230 0x7ff2ac1f4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:17.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.272+0000 7ff2b2461640 1 -- 172.21.15.43:0/1336190745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2ac0b5560 msgr2=0x7ff2ac0ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:17.264 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.272+0000 7ff2b2461640 1 --2- 172.21.15.43:0/1336190745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2ac0b5560 0x7ff2ac0ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:17.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.272+0000 7ff2b2461640 1 -- 172.21.15.43:0/1336190745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2ac15bf10 con 0x7ff2ac15e090 2024-04-25T09:05:17.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.272+0000 7ff2b2461640 1 --2- 172.21.15.43:0/1336190745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2ac15e090 0x7ff2ac1f45a0 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7ff29c009e80 tx=0x7ff29c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:17.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.272+0000 7ff2aa7fc640 1 -- 172.21.15.43:0/1336190745 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c040070 con 0x7ff2ac15e090 2024-04-25T09:05:17.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.272+0000 7ff2aa7fc640 1 -- 172.21.15.43:0/1336190745 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29c031bf0 con 0x7ff2ac15e090 2024-04-25T09:05:17.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.272+0000 7ff2b3ec4640 1 -- 172.21.15.43:0/1336190745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2ac15c1f0 con 0x7ff2ac15e090 2024-04-25T09:05:17.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.272+0000 7ff2aa7fc640 1 -- 172.21.15.43:0/1336190745 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c045460 con 0x7ff2ac15e090 2024-04-25T09:05:17.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.272+0000 7ff2b3ec4640 1 -- 172.21.15.43:0/1336190745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2ac15c7b0 con 0x7ff2ac15e090 2024-04-25T09:05:17.265 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.273+0000 7ff2b3ec4640 1 -- 172.21.15.43:0/1336190745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2ac06bf00 con 0x7ff2ac15e090 2024-04-25T09:05:17.266 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.273+0000 7ff2aa7fc640 1 -- 172.21.15.43:0/1336190745 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff29c039040 con 0x7ff2ac15e090 2024-04-25T09:05:17.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.274+0000 7ff2aa7fc640 1 --2- 172.21.15.43:0/1336190745 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff29806b4d0 0x7ff29806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:17.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.274+0000 7ff2aa7fc640 1 -- 172.21.15.43:0/1336190745 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff29c0ab710 con 0x7ff2ac15e090 2024-04-25T09:05:17.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.274+0000 7ff2b1c60640 1 --2- 172.21.15.43:0/1336190745 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff29806b4d0 0x7ff29806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:17.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.275+0000 7ff2b1c60640 1 --2- 172.21.15.43:0/1336190745 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff29806b4d0 0x7ff29806d990 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4007fd0 tx=0x7ff2a4007960 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:17.267 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.278+0000 7ff2aa7fc640 1 -- 172.21.15.43:0/1336190745 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff2ac06bf00 con 0x7ff2ac15e090 2024-04-25T09:05:17.448 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.460+0000 7ff2b3ec4640 1 -- 172.21.15.43:0/1336190745 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7ff2ac074f60 con 0x7ff2ac15e090 2024-04-25T09:05:17.451 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.463+0000 7ff2aa7fc640 1 -- 172.21.15.43:0/1336190745 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v28)=0 v28) v1 ==== 123+0+0 (secure 0 0 0) 0x7ff29c07a920 con 0x7ff2ac15e090 2024-04-25T09:05:17.452 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.464+0000 7ff2aa7fc640 1 -- 172.21.15.43:0/1336190745 <== mon.0 v2:172.21.15.43:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7ff29c07a220 con 0x7ff2ac15e090 2024-04-25T09:05:17.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.465+0000 7ff2b3ec4640 1 -- 172.21.15.43:0/1336190745 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff29806b4d0 msgr2=0x7ff29806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:17.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.465+0000 7ff2b3ec4640 1 --2- 172.21.15.43:0/1336190745 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff29806b4d0 0x7ff29806d990 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4007fd0 tx=0x7ff2a4007960 comp rx=0 tx=0).stop 2024-04-25T09:05:17.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.465+0000 7ff2b3ec4640 1 -- 172.21.15.43:0/1336190745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2ac15e090 msgr2=0x7ff2ac1f45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:17.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.465+0000 7ff2b3ec4640 1 --2- 172.21.15.43:0/1336190745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2ac15e090 0x7ff2ac1f45a0 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7ff29c009e80 tx=0x7ff29c004a40 comp rx=0 tx=0).stop 2024-04-25T09:05:17.454 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.465+0000 7ff2b3ec4640 1 -- 172.21.15.43:0/1336190745 shutdown_connections 2024-04-25T09:05:17.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.465+0000 7ff2b3ec4640 1 --2- 172.21.15.43:0/1336190745 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7ff2ac15e090 0x7ff2ac1f45a0 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:17.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.465+0000 7ff2b3ec4640 1 --2- 172.21.15.43:0/1336190745 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7ff2ac0d2230 0x7ff2ac1f4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:17.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.465+0000 7ff2b3ec4640 1 --2- 172.21.15.43:0/1336190745 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7ff29806b4d0 0x7ff29806d990 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:17.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.465+0000 7ff2b3ec4640 1 --2- 172.21.15.43:0/1336190745 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7ff2ac0b5560 0x7ff2ac0ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:17.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.465+0000 7ff2b3ec4640 1 -- 172.21.15.43:0/1336190745 >> 172.21.15.43:0/1336190745 conn(0x7ff2ac0d0560 msgr2=0x7ff2ac0d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:17.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.466+0000 7ff2b3ec4640 1 -- 172.21.15.43:0/1336190745 shutdown_connections 2024-04-25T09:05:17.455 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.466+0000 7ff2b3ec4640 1 -- 172.21.15.43:0/1336190745 wait complete. 2024-04-25T09:05:17.465 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T09:05:17.465 DEBUG:teuthology.orchestra.run.smithi043:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T09:05:17.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.578+0000 7f6e826cb640 1 Processor -- start 2024-04-25T09:05:17.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.578+0000 7f6e826cb640 1 -- start start 2024-04-25T09:05:17.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.578+0000 7f6e826cb640 1 --2- >> v2:172.21.15.193:3300/0 conn(0x7f6e7c15dce0 0x7f6e7c15ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:17.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.578+0000 7f6e826cb640 1 -- --> v1:172.21.15.43:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e7c0d2920 con 0x7f6e7c0ce550 2024-04-25T09:05:17.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.578+0000 7f6e826cb640 1 -- --> v1:172.21.15.193:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e7c0cee80 con 0x7f6e7c0d0af0 2024-04-25T09:05:17.567 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.578+0000 7f6e826cb640 1 -- --> v2:172.21.15.193:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e7c0cf160 con 0x7f6e7c15dce0 2024-04-25T09:05:17.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.579+0000 7f6e7b7fe640 1 --1- >> v1:172.21.15.43:6789/0 conn(0x7f6e7c0ce550 0x7f6e7c0d03e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.43:6789/0 says I am v1:172.21.15.43:56090/0 (socket says 172.21.15.43:56090) 2024-04-25T09:05:17.568 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.579+0000 7f6e7b7fe640 1 -- 172.21.15.43:0/1087842474 learned_addr learned my addr 172.21.15.43:0/1087842474 (peer_addr_for_me v1:172.21.15.43:0/0) 2024-04-25T09:05:17.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.579+0000 7f6e7bfff640 1 --2- 172.21.15.43:0/1087842474 >> v2:172.21.15.193:3300/0 conn(0x7f6e7c15dce0 0x7f6e7c15ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:17.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.579+0000 7f6e7affd640 1 -- 172.21.15.43:0/1087842474 <== mon.0 v1:172.21.15.43:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 226501697 0 0) 0x7f6e7c0d2920 con 0x7f6e7c0ce550 2024-04-25T09:05:17.572 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.579+0000 7f6e7affd640 1 -- 172.21.15.43:0/1087842474 --> v1:172.21.15.43:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e68003520 con 0x7f6e7c0ce550 2024-04-25T09:05:17.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.579+0000 7f6e7affd640 1 -- 172.21.15.43:0/1087842474 <== mon.0 v1:172.21.15.43:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 769538829 0 0) 0x7f6e68003520 con 0x7f6e7c0ce550 2024-04-25T09:05:17.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.579+0000 7f6e7affd640 1 -- 172.21.15.43:0/1087842474 >> v2:172.21.15.193:3300/0 conn(0x7f6e7c15dce0 msgr2=0x7f6e7c15ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:17.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.579+0000 7f6e7affd640 1 --2- 172.21.15.43:0/1087842474 >> v2:172.21.15.193:3300/0 conn(0x7f6e7c15dce0 0x7f6e7c15ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:17.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.579+0000 7f6e7affd640 1 -- 172.21.15.43:0/1087842474 >> v1:172.21.15.193:6789/0 conn(0x7f6e7c0d0af0 legacy=0x7f6e7c160980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:17.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.579+0000 7f6e7affd640 1 -- 172.21.15.43:0/1087842474 --> v1:172.21.15.43:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7c0cf440 con 0x7f6e7c0ce550 2024-04-25T09:05:17.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.579+0000 7f6e7affd640 1 -- 172.21.15.43:0/1087842474 <== mon.0 v1:172.21.15.43:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4291179963 0 0) 0x7f6e64001690 con 0x7f6e7c0ce550 2024-04-25T09:05:17.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.579+0000 7f6e7affd640 1 -- 172.21.15.43:0/1087842474 >> v1:172.21.15.43:6789/0 conn(0x7f6e7c0ce550 legacy=0x7f6e7c0d03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:17.573 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.579+0000 7f6e7affd640 1 --2- 172.21.15.43:0/1087842474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e680039b0 0x7f6e68003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:17.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.579+0000 7f6e7affd640 1 --2- 172.21.15.43:0/1087842474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e68004800 0x7f6e68004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:17.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.579+0000 7f6e7affd640 1 --2- 172.21.15.43:0/1087842474 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6e680051c0 0x7f6e6800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:17.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.579+0000 7f6e7affd640 1 -- 172.21.15.43:0/1087842474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c0cf160 con 0x7f6e68004800 2024-04-25T09:05:17.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.579+0000 7f6e7affd640 1 -- 172.21.15.43:0/1087842474 --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6800c160 con 0x7f6e680039b0 2024-04-25T09:05:17.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.579+0000 7f6e7affd640 1 -- 172.21.15.43:0/1087842474 --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6800c360 con 0x7f6e680051c0 2024-04-25T09:05:17.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.579+0000 7f6e7b7fe640 1 --2- 172.21.15.43:0/1087842474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e68004800 0x7f6e68004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:17.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.579+0000 7f6e7bfff640 1 --2- 172.21.15.43:0/1087842474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e680039b0 0x7f6e68003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:17.574 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.580+0000 7f6e80c68640 1 --2- 172.21.15.43:0/1087842474 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6e680051c0 0x7f6e6800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:17.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.580+0000 7f6e7b7fe640 1 -- 172.21.15.43:0/1087842474 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6e680051c0 msgr2=0x7f6e6800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:17.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.580+0000 7f6e7b7fe640 1 --2- 172.21.15.43:0/1087842474 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6e680051c0 0x7f6e6800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:17.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.580+0000 7f6e7b7fe640 1 -- 172.21.15.43:0/1087842474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e680039b0 msgr2=0x7f6e68003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T09:05:17.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.580+0000 7f6e7b7fe640 1 --2- 172.21.15.43:0/1087842474 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e680039b0 0x7f6e68003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:17.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.580+0000 7f6e7b7fe640 1 -- 172.21.15.43:0/1087842474 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e68003520 con 0x7f6e68004800 2024-04-25T09:05:17.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.580+0000 7f6e80c68640 1 -- 172.21.15.43:0/1087842474 reap_dead start 2024-04-25T09:05:17.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.580+0000 7f6e7b7fe640 1 --2- 172.21.15.43:0/1087842474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e68004800 0x7f6e68004c80 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f6e6400c210 tx=0x7f6e68005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=770720ae8dca8047 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:17.575 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.580+0000 7f6e7affd640 1 -- 172.21.15.43:0/1087842474 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e64007d10 con 0x7f6e68004800 2024-04-25T09:05:17.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.580+0000 7f6e7affd640 1 -- 172.21.15.43:0/1087842474 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f6e64005af0 con 0x7f6e68004800 2024-04-25T09:05:17.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.580+0000 7f6e7affd640 1 -- 172.21.15.43:0/1087842474 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e64007070 con 0x7f6e68004800 2024-04-25T09:05:17.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.581+0000 7f6e826cb640 1 -- 172.21.15.43:0/1087842474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e68004800 msgr2=0x7f6e68004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:17.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.581+0000 7f6e826cb640 1 --2- 172.21.15.43:0/1087842474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e68004800 0x7f6e68004c80 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f6e6400c210 tx=0x7f6e68005830 comp rx=0 tx=0).stop 2024-04-25T09:05:17.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.581+0000 7f6e826cb640 1 -- 172.21.15.43:0/1087842474 shutdown_connections 2024-04-25T09:05:17.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.581+0000 7f6e826cb640 1 --2- 172.21.15.43:0/1087842474 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e68004800 0x7f6e68004c80 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:17.576 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.581+0000 7f6e826cb640 1 -- 172.21.15.43:0/1087842474 >> 172.21.15.43:0/1087842474 conn(0x7f6e7c09cd00 msgr2=0x7f6e7c0d39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:17.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.581+0000 7f6e826cb640 1 -- 172.21.15.43:0/1087842474 shutdown_connections 2024-04-25T09:05:17.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.581+0000 7f6e826cb640 1 -- 172.21.15.43:0/1087842474 wait complete. 2024-04-25T09:05:17.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.582+0000 7f6e826cb640 1 Processor -- start 2024-04-25T09:05:17.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.582+0000 7f6e826cb640 1 -- start start 2024-04-25T09:05:17.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.583+0000 7f6e826cb640 1 --2- >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e68004800 0x7f6e7c1e8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:17.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.583+0000 7f6e826cb640 1 --2- >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6e7c0b7550 0x7f6e7c1f4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:17.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.583+0000 7f6e826cb640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e7c1f47c0 0x7f6e7c1f4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:17.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.583+0000 7f6e826cb640 1 -- --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c15e0c0 con 0x7f6e7c1f47c0 2024-04-25T09:05:17.577 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.583+0000 7f6e826cb640 1 -- --> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c15f6b0 con 0x7f6e68004800 2024-04-25T09:05:17.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.583+0000 7f6e826cb640 1 -- --> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c0ce9c0 con 0x7f6e7c0b7550 2024-04-25T09:05:17.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.583+0000 7f6e80c68640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e7c1f47c0 0x7f6e7c1f4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:17.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.583+0000 7f6e80c68640 1 --2- >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e7c1f47c0 0x7f6e7c1f4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.43:3300/0 says I am v2:172.21.15.43:43106/0 (socket says 172.21.15.43:43106) 2024-04-25T09:05:17.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.583+0000 7f6e80c68640 1 -- 172.21.15.43:0/3238746446 learned_addr learned my addr 172.21.15.43:0/3238746446 (peer_addr_for_me v2:172.21.15.43:0/0) 2024-04-25T09:05:17.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.583+0000 7f6e7b7fe640 1 --2- 172.21.15.43:0/3238746446 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6e7c0b7550 0x7f6e7c1f4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:17.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.583+0000 7f6e7bfff640 1 --2- 172.21.15.43:0/3238746446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e68004800 0x7f6e7c1e8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:17.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.584+0000 7f6e80c68640 1 -- 172.21.15.43:0/3238746446 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6e7c0b7550 msgr2=0x7f6e7c1f4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:17.578 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.584+0000 7f6e80c68640 1 --2- 172.21.15.43:0/3238746446 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6e7c0b7550 0x7f6e7c1f4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:17.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.584+0000 7f6e80c68640 1 -- 172.21.15.43:0/3238746446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e68004800 msgr2=0x7f6e7c1e8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:17.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.584+0000 7f6e80c68640 1 --2- 172.21.15.43:0/3238746446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e68004800 0x7f6e7c1e8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:17.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.584+0000 7f6e80c68640 1 -- 172.21.15.43:0/3238746446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7c1f5180 con 0x7f6e7c1f47c0 2024-04-25T09:05:17.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.584+0000 7f6e80c68640 1 --2- 172.21.15.43:0/3238746446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e7c1f47c0 0x7f6e7c1f4c40 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c0b64d0 tx=0x7f6e7400eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:17.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.584+0000 7f6e78ff9640 1 -- 172.21.15.43:0/3238746446 <== mon.0 v2:172.21.15.43:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e74003970 con 0x7f6e7c1f47c0 2024-04-25T09:05:17.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.584+0000 7f6e78ff9640 1 -- 172.21.15.43:0/3238746446 <== mon.0 v2:172.21.15.43:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f6e74018070 con 0x7f6e7c1f47c0 2024-04-25T09:05:17.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.584+0000 7f6e826cb640 1 -- 172.21.15.43:0/3238746446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e7c0b7da0 con 0x7f6e7c1f47c0 2024-04-25T09:05:17.579 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.584+0000 7f6e78ff9640 1 -- 172.21.15.43:0/3238746446 <== mon.0 v2:172.21.15.43:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e74003d90 con 0x7f6e7c1f47c0 2024-04-25T09:05:17.580 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.584+0000 7f6e826cb640 1 -- 172.21.15.43:0/3238746446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e7c0b8360 con 0x7f6e7c1f47c0 2024-04-25T09:05:17.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.585+0000 7f6e826cb640 1 -- 172.21.15.43:0/3238746446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e7c0680f0 con 0x7f6e7c1f47c0 2024-04-25T09:05:17.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.585+0000 7f6e78ff9640 1 -- 172.21.15.43:0/3238746446 <== mon.0 v2:172.21.15.43:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6e740050d0 con 0x7f6e7c1f47c0 2024-04-25T09:05:17.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.586+0000 7f6e78ff9640 1 --2- 172.21.15.43:0/3238746446 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6e5c06b500 0x7f6e5c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T09:05:17.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.586+0000 7f6e78ff9640 1 -- 172.21.15.43:0/3238746446 <== mon.0 v2:172.21.15.43:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6e74089bb0 con 0x7f6e7c1f47c0 2024-04-25T09:05:17.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.586+0000 7f6e7bfff640 1 --2- 172.21.15.43:0/3238746446 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6e5c06b500 0x7f6e5c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T09:05:17.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.586+0000 7f6e7bfff640 1 --2- 172.21.15.43:0/3238746446 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6e5c06b500 0x7f6e5c06d9c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c009a60 tx=0x7f6e6c008040 comp rx=0 tx=0).ready entity=mgr.4108 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T09:05:17.581 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.590+0000 7f6e78ff9640 1 -- 172.21.15.43:0/3238746446 <== mon.0 v2:172.21.15.43:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6e7c1f5180 con 0x7f6e7c1f47c0 2024-04-25T09:05:17.797 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.809+0000 7f6e826cb640 1 -- 172.21.15.43:0/3238746446 --> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e7c074ef0 con 0x7f6e7c1f47c0 2024-04-25T09:05:17.798 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.810+0000 7f6e78ff9640 1 -- 172.21.15.43:0/3238746446 <== mon.0 v2:172.21.15.43:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) v1 ==== 94+0+3703 (secure 0 0 0) 0x7f6e7408d090 con 0x7f6e7c1f47c0 2024-04-25T09:05:17.799 INFO:teuthology.orchestra.run.smithi043.stdout: 2024-04-25T09:05:17.799 INFO:teuthology.orchestra.run.smithi043.stdout:{"epoch":92,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7448,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6836/2225978762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6834","nonce":2225978762},{"type":"v1","addr":"172.21.15.193:6836","nonce":2225978762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":8608,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.193:6837/2061558720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.193:6835","nonce":2061558720},{"type":"v1","addr":"172.21.15.193:6837","nonce":2061558720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92},{"gid":9711,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6833/454458092","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6832","nonce":454458092},{"type":"v1","addr":"172.21.15.43:6833","nonce":454458092}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":83},{"gid":10074,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.43:6835/2313770418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.43:6834","nonce":2313770418},{"type":"v1","addr":"172.21.15.43:6835","nonce":2313770418}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":92}],"filesystems":[]} 2024-04-25T09:05:17.799 INFO:teuthology.orchestra.run.smithi043.stderr:dumped fsmap epoch 92 2024-04-25T09:05:17.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.812+0000 7f6e826cb640 1 -- 172.21.15.43:0/3238746446 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6e5c06b500 msgr2=0x7f6e5c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:17.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.812+0000 7f6e826cb640 1 --2- 172.21.15.43:0/3238746446 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6e5c06b500 0x7f6e5c06d9c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c009a60 tx=0x7f6e6c008040 comp rx=0 tx=0).stop 2024-04-25T09:05:17.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.812+0000 7f6e826cb640 1 -- 172.21.15.43:0/3238746446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e7c1f47c0 msgr2=0x7f6e7c1f4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T09:05:17.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.812+0000 7f6e826cb640 1 --2- 172.21.15.43:0/3238746446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e7c1f47c0 0x7f6e7c1f4c40 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c0b64d0 tx=0x7f6e7400eb90 comp rx=0 tx=0).stop 2024-04-25T09:05:17.800 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.812+0000 7f6e826cb640 1 -- 172.21.15.43:0/3238746446 shutdown_connections 2024-04-25T09:05:17.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.812+0000 7f6e826cb640 1 --2- 172.21.15.43:0/3238746446 >> [v2:172.21.15.43:3300/0,v1:172.21.15.43:6789/0] conn(0x7f6e7c1f47c0 0x7f6e7c1f4c40 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:17.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.812+0000 7f6e826cb640 1 --2- 172.21.15.43:0/3238746446 >> [v2:172.21.15.193:3301/0,v1:172.21.15.193:6790/0] conn(0x7f6e7c0b7550 0x7f6e7c1f4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:17.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.812+0000 7f6e826cb640 1 --2- 172.21.15.43:0/3238746446 >> [v2:172.21.15.193:6832/1244825175,v1:172.21.15.193:6833/1244825175] conn(0x7f6e5c06b500 0x7f6e5c06d9c0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:17.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.812+0000 7f6e826cb640 1 --2- 172.21.15.43:0/3238746446 >> [v2:172.21.15.193:3300/0,v1:172.21.15.193:6789/0] conn(0x7f6e68004800 0x7f6e7c1e8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T09:05:17.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.812+0000 7f6e826cb640 1 -- 172.21.15.43:0/3238746446 >> 172.21.15.43:0/3238746446 conn(0x7f6e7c09cd00 msgr2=0x7f6e7c0c7500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T09:05:17.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.812+0000 7f6e826cb640 1 -- 172.21.15.43:0/3238746446 shutdown_connections 2024-04-25T09:05:17.801 INFO:teuthology.orchestra.run.smithi043.stderr:2024-04-25T09:05:17.812+0000 7f6e826cb640 1 -- 172.21.15.43:0/3238746446 wait complete. 2024-04-25T09:05:17.811 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-25T09:05:17.812 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T09:05:17.812 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T09:05:17.812 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:05:17.821 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T09:05:17.821 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T09:05:17.821 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:05:17.871 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T09:05:17.871 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T09:05:17.871 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:05:17.883 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T09:05:17.883 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T09:05:17.883 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:05:17.934 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T09:05:17.934 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T09:05:17.935 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T09:05:17.935 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:05:17.968 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T09:05:17.969 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T09:05:17.969 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:05:18.038 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T09:05:18.038 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T09:05:18.038 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:05:18.105 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T09:05:18.105 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T09:05:18.105 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:05:18.175 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T09:05:18.175 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-25T09:05:18.176 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:05:18.209 INFO:tasks.ceph.osd.4:Stopped 2024-04-25T09:05:18.210 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T09:05:18.210 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:05:18.283 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T09:05:18.283 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-25T09:05:18.283 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:05:18.362 INFO:tasks.ceph.osd.6:Stopped 2024-04-25T09:05:18.362 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-25T09:05:18.362 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:05:18.443 INFO:tasks.ceph.osd.7:Stopped 2024-04-25T09:05:18.443 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T09:05:18.444 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T09:05:18.444 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:05:18.469 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T09:05:18.470 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T09:05:18.470 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:05:18.512 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T09:05:18.513 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T09:05:18.513 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T09:05:18.513 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:05:18.524 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T09:05:18.525 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T09:05:18.525 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:05:18.564 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T09:05:18.564 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T09:05:18.564 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T09:05:18.617 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T09:05:18.617 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T09:05:18.618 DEBUG:teuthology.orchestra.run.smithi043:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'Replacing daemon mds' | egrep -v 'Scrub error on inode' | egrep -v 'Behind on trimming' | egrep -v 'Metadata damage detected' | egrep -v 'bad backtrace on inode' | egrep -v 'overall HEALTH_' | egrep -v '\(MDS_TRIM\)' | head -n 1 2024-04-25T09:05:18.655 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi043.front.sepia.ceph.com 2024-04-25T09:05:18.655 DEBUG:teuthology.orchestra.run.smithi043:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T09:05:18.863 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi043.front.sepia.ceph.com 2024-04-25T09:05:18.863 DEBUG:teuthology.orchestra.run.smithi043:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T09:05:18.994 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi043.front.sepia.ceph.com 2024-04-25T09:05:18.995 DEBUG:teuthology.orchestra.run.smithi043:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T09:05:19.051 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi043.front.sepia.ceph.com 2024-04-25T09:05:19.051 DEBUG:teuthology.orchestra.run.smithi043:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T09:05:19.101 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi193.front.sepia.ceph.com 2024-04-25T09:05:19.101 DEBUG:teuthology.orchestra.run.smithi193:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-25T09:05:19.300 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi193.front.sepia.ceph.com 2024-04-25T09:05:19.300 DEBUG:teuthology.orchestra.run.smithi193:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-25T09:05:19.374 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi193.front.sepia.ceph.com 2024-04-25T09:05:19.375 DEBUG:teuthology.orchestra.run.smithi193:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-25T09:05:19.435 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi193.front.sepia.ceph.com 2024-04-25T09:05:19.435 DEBUG:teuthology.orchestra.run.smithi193:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-25T09:05:19.492 INFO:tasks.ceph:Archiving mon data... 2024-04-25T09:05:19.493 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673004/data/mon.a.tgz 2024-04-25T09:05:19.493 DEBUG:teuthology.orchestra.run.smithi043:> mktemp 2024-04-25T09:05:19.509 INFO:teuthology.orchestra.run.smithi043.stdout:/tmp/tmp.6Z68hDpeDR 2024-04-25T09:05:19.509 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.6Z68hDpeDR 2024-04-25T09:05:19.667 DEBUG:teuthology.orchestra.run.smithi043:> sudo chmod 0666 /tmp/tmp.6Z68hDpeDR 2024-04-25T09:05:19.749 DEBUG:teuthology.orchestra.remote:smithi043:/tmp/tmp.6Z68hDpeDR is 442KB 2024-04-25T09:05:19.819 DEBUG:teuthology.orchestra.run.smithi043:> rm -fr /tmp/tmp.6Z68hDpeDR 2024-04-25T09:05:19.834 DEBUG:teuthology.misc:Transferring archived files from smithi193:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673004/data/mon.b.tgz 2024-04-25T09:05:19.834 DEBUG:teuthology.orchestra.run.smithi193:> mktemp 2024-04-25T09:05:19.851 INFO:teuthology.orchestra.run.smithi193.stdout:/tmp/tmp.EZ4iWKNBC6 2024-04-25T09:05:19.852 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.EZ4iWKNBC6 2024-04-25T09:05:20.011 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod 0666 /tmp/tmp.EZ4iWKNBC6 2024-04-25T09:05:20.092 DEBUG:teuthology.orchestra.remote:smithi193:/tmp/tmp.EZ4iWKNBC6 is 449KB 2024-04-25T09:05:20.190 DEBUG:teuthology.orchestra.run.smithi193:> rm -fr /tmp/tmp.EZ4iWKNBC6 2024-04-25T09:05:20.204 DEBUG:teuthology.misc:Transferring archived files from smithi193:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673004/data/mon.c.tgz 2024-04-25T09:05:20.204 DEBUG:teuthology.orchestra.run.smithi193:> mktemp 2024-04-25T09:05:20.258 INFO:teuthology.orchestra.run.smithi193.stdout:/tmp/tmp.ISGnPuGgGe 2024-04-25T09:05:20.258 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.ISGnPuGgGe 2024-04-25T09:05:20.400 DEBUG:teuthology.orchestra.run.smithi193:> sudo chmod 0666 /tmp/tmp.ISGnPuGgGe 2024-04-25T09:05:20.483 DEBUG:teuthology.orchestra.remote:smithi193:/tmp/tmp.ISGnPuGgGe is 449KB 2024-04-25T09:05:20.580 DEBUG:teuthology.orchestra.run.smithi193:> rm -fr /tmp/tmp.ISGnPuGgGe 2024-04-25T09:05:20.595 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T09:05:20.596 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T09:05:20.599 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T09:05:20.681 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T09:05:20.712 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T09:05:20.713 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T09:05:20.714 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673004/remote/smithi043/crash 2024-04-25T09:05:20.715 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T09:05:20.748 DEBUG:teuthology.misc:Transferring archived files from smithi193:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673004/remote/smithi193/crash 2024-04-25T09:05:20.749 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T09:05:20.775 INFO:tasks.ceph:Shutting down logrotate 2024-04-25T09:05:20.776 DEBUG:teuthology.orchestra.run.smithi043:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:05:20.792 DEBUG:teuthology.orchestra.run.smithi193:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T09:05:20.861 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T09:05:20.888 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T09:05:20.925 INFO:tasks.ceph:Compressing logs... 2024-04-25T09:05:20.925 DEBUG:teuthology.orchestra.run.smithi043:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T09:05:20.930 DEBUG:teuthology.orchestra.run.smithi193:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T09:05:21.009 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T09:05:21.009 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T09:05:21.010 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34751.log 2024-04-25T09:05:21.010 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-25T09:05:21.010 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/ceph-client.admin.34751.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-25T09:05:21.010 INFO:teuthology.orchestra.run.smithi193.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34751.log.gz 2024-04-25T09:05:21.010 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-25T09:05:21.010 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-25T09:05:21.010 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34831.log 2024-04-25T09:05:21.010 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34871.log 2024-04-25T09:05:21.011 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/ceph-client.admin.34831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34704.log 2024-04-25T09:05:21.011 INFO:teuthology.orchestra.run.smithi193.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.34831.log.gz/var/log/ceph/ceph-client.admin.34871.log: 2024-04-25T09:05:21.011 INFO:teuthology.orchestra.run.smithi193.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34871.log.gz 2024-04-25T09:05:21.011 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- 92.5% /var/log/ceph/ceph-mgr.x.log 2024-04-25T09:05:21.011 INFO:teuthology.orchestra.run.smithi193.stderr: -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T09:05:21.011 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/ceph-client.admin.34704.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-25T09:05:21.011 INFO:teuthology.orchestra.run.smithi193.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.34704.log.gz 2024-04-25T09:05:21.011 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-25T09:05:21.011 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34620.log 2024-04-25T09:05:21.012 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/ceph-osd.6.log: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T09:05:21.015 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34791.log 2024-04-25T09:05:21.016 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/ceph-client.admin.34620.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.34620.log.gz 2024-04-25T09:05:21.040 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34660.log 2024-04-25T09:05:21.041 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/ceph-client.admin.34791.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34791.log.gz 2024-04-25T09:05:21.041 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.40994.log 2024-04-25T09:05:21.041 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/ceph-client.admin.34660.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34660.log.gz 2024-04-25T09:05:21.041 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-25T09:05:21.042 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/ceph-client.1.40994.log: 89.5% -- replaced with /var/log/ceph/ceph-client.1.40994.log.gz 2024-04-25T09:05:21.042 INFO:teuthology.orchestra.run.smithi193.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-25T09:05:21.047 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34580.log 2024-04-25T09:05:21.061 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.41456.log 2024-04-25T09:05:21.061 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/ceph-client.admin.34580.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34580.log.gz 2024-04-25T09:05:21.080 INFO:teuthology.orchestra.run.smithi193.stderr:/var/log/ceph/ceph-client.3.41456.log: 90.7% -- replaced with /var/log/ceph/ceph-client.3.41456.log.gz 2024-04-25T09:05:21.107 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85826.log 2024-04-25T09:05:21.107 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41502.log 2024-04-25T09:05:21.107 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.85826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44931.log 2024-04-25T09:05:21.107 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.85826.log.gz 2024-04-25T09:05:21.107 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45403.log 2024-04-25T09:05:21.107 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41502.log: /var/log/ceph/ceph-client.admin.44931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89673.log 2024-04-25T09:05:21.108 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41502.log.gz 2024-04-25T09:05:21.108 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44931.log.gz 2024-04-25T09:05:21.108 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75920.log 2024-04-25T09:05:21.108 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45403.log.gz 2024-04-25T09:05:21.108 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.89673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95806.log 2024-04-25T09:05:21.108 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.75920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57380.log 2024-04-25T09:05:21.108 INFO:teuthology.orchestra.run.smithi043.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.89673.log.gz 2024-04-25T09:05:21.108 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.95806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74411.log 2024-04-25T09:05:21.108 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.95806.log.gz 2024-04-25T09:05:21.109 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.57380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99271.log 2024-04-25T09:05:21.109 INFO:teuthology.orchestra.run.smithi043.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.75920.log.gz 2024-04-25T09:05:21.109 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.74411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96416.log 2024-04-25T09:05:21.109 INFO:teuthology.orchestra.run.smithi043.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57380.log.gz 2024-04-25T09:05:21.109 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.99271.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.74411.log.gz 2024-04-25T09:05:21.109 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65452.log 2024-04-25T09:05:21.109 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.99271.log.gz 2024-04-25T09:05:21.109 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85775.log 2024-04-25T09:05:21.109 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.96416.log: /var/log/ceph/ceph-client.admin.65452.log: gzip -5 --verbose 88.3% -- replaced with /var/log/ceph/ceph-client.admin.96416.log.gz 2024-04-25T09:05:21.109 INFO:teuthology.orchestra.run.smithi043.stderr: -- /var/log/ceph/ceph-client.admin.84488.log 2024-04-25T09:05:21.110 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.85775.log: 89.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.65452.log.gz -5 2024-04-25T09:05:21.110 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.62213.log 2024-04-25T09:05:21.110 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.84488.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.85775.log.gz 2024-04-25T09:05:21.110 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85954.log 2024-04-25T09:05:21.110 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.84488.log.gz 2024-04-25T09:05:21.110 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.62213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40166.log 2024-04-25T09:05:21.110 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.85954.log: gzip -5 87.7% --verbose -- replaced with /var/log/ceph/ceph-client.admin.62213.log.gz -- 2024-04-25T09:05:21.110 INFO:teuthology.orchestra.run.smithi043.stderr: /var/log/ceph/ceph-client.admin.47101.log 2024-04-25T09:05:21.111 INFO:teuthology.orchestra.run.smithi043.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.85954.log.gz 2024-04-25T09:05:21.111 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34478.log 2024-04-25T09:05:21.111 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66530.log 2024-04-25T09:05:21.111 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40166.log.gz 2024-04-25T09:05:21.111 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73512.log 2024-04-25T09:05:21.111 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47101.log.gz 2024-04-25T09:05:21.111 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.66530.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34478.log.gzgzip 2024-04-25T09:05:21.111 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.46099.log 2024-04-25T09:05:21.111 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.73512.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.66530.log.gz 2024-04-25T09:05:21.111 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57464.log 2024-04-25T09:05:21.112 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94690.log 2024-04-25T09:05:21.112 INFO:teuthology.orchestra.run.smithi043.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.46099.log.gz 2024-04-25T09:05:21.112 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.57464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95423.log 2024-04-25T09:05:21.112 INFO:teuthology.orchestra.run.smithi043.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.73512.log.gz 2024-04-25T09:05:21.112 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.94690.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57464.log.gz 2024-04-25T09:05:21.112 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41159.log 2024-04-25T09:05:21.112 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.94690.log.gz 2024-04-25T09:05:21.112 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.95423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56489.log 2024-04-25T09:05:21.112 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41159.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.95423.log.gz 2024-04-25T09:05:21.113 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45631.log 2024-04-25T09:05:21.113 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89967.log 2024-04-25T09:05:21.113 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41159.log.gz 2024-04-25T09:05:21.113 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56489.log.gz 2024-04-25T09:05:21.113 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45757.log 2024-04-25T09:05:21.113 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45631.log.gz/var/log/ceph/ceph-client.admin.89967.log: 2024-04-25T09:05:21.113 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83260.log 2024-04-25T09:05:21.113 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.89967.log.gzgzip 2024-04-25T09:05:21.113 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.74495.log 2024-04-25T09:05:21.113 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45757.log: /var/log/ceph/ceph-client.admin.83260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64861.log 2024-04-25T09:05:21.114 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45757.log.gz 2024-04-25T09:05:21.114 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100951.log/var/log/ceph/ceph-client.admin.74495.log: 2024-04-25T09:05:21.114 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.64861.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.83260.log.gz 2024-04-25T09:05:21.114 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95068.log 2024-04-25T09:05:21.114 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.74495.log.gz 2024-04-25T09:05:21.114 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.100951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47937.log 2024-04-25T09:05:21.114 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.64861.log.gz 2024-04-25T09:05:21.114 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.95068.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.100951.log.gz 2024-04-25T09:05:21.114 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88715.log 2024-04-25T09:05:21.115 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95068.log.gz 2024-04-25T09:05:21.115 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65663.log 2024-04-25T09:05:21.115 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.88715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66045.log 2024-04-25T09:05:21.115 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47937.log.gz 2024-04-25T09:05:21.115 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.88715.log.gz 2024-04-25T09:05:21.115 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.65663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41164.log 2024-04-25T09:05:21.115 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65663.log.gz 2024-04-25T09:05:21.115 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45325.log 2024-04-25T09:05:21.115 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.66045.log: /var/log/ceph/ceph-client.admin.41164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85912.log 2024-04-25T09:05:21.115 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.66045.log.gz 2024-04-25T09:05:21.116 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56573.log 2024-04-25T09:05:21.116 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41164.log.gz 2024-04-25T09:05:21.116 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45325.log.gz 2024-04-25T09:05:21.116 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.85912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88631.log 2024-04-25T09:05:21.116 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56573.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.85912.log.gz 2024-04-25T09:05:21.116 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86122.log 2024-04-25T09:05:21.116 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.88631.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56573.log.gzgzip 2024-04-25T09:05:21.116 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.44542.log 2024-04-25T09:05:21.116 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.86122.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.88631.log.gz 2024-04-25T09:05:21.117 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90746.log 2024-04-25T09:05:21.117 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.86122.log.gz 2024-04-25T09:05:21.117 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91224.log 2024-04-25T09:05:21.117 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44542.log.gz 2024-04-25T09:05:21.117 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.90746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55398.log 2024-04-25T09:05:21.117 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.91224.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.90746.log.gz 2024-04-25T09:05:21.117 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65224.log 2024-04-25T09:05:21.117 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.91224.log.gz 2024-04-25T09:05:21.118 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63225.log 2024-04-25T09:05:21.118 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.65224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83794.log 2024-04-25T09:05:21.118 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.65224.log.gz 2024-04-25T09:05:21.118 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.63225.log: gzip -5 90.8% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.55398.log.gz /var/log/ceph/ceph-client.admin.56092.log 2024-04-25T09:05:21.118 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-04-25T09:05:21.118 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.63225.log.gz 2024-04-25T09:05:21.118 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.83794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60831.log 2024-04-25T09:05:21.118 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56092.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.83794.log.gz 2024-04-25T09:05:21.118 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84004.log 2024-04-25T09:05:21.119 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.60831.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.56092.log.gz 2024-04-25T09:05:21.119 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46387.log 2024-04-25T09:05:21.119 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60831.log.gz 2024-04-25T09:05:21.119 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.84004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48936.log 2024-04-25T09:05:21.119 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46387.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.84004.log.gz 2024-04-25T09:05:21.119 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83326.log 2024-04-25T09:05:21.119 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46387.log.gz 2024-04-25T09:05:21.119 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47599.log 2024-04-25T09:05:21.119 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.83326.log: gzip -5 --verbose 87.4% -- -- replaced with /var/log/ceph/ceph-client.admin.48936.log.gz 2024-04-25T09:05:21.119 INFO:teuthology.orchestra.run.smithi043.stderr: /var/log/ceph/ceph-client.admin.75522.log 2024-04-25T09:05:21.120 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47599.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.83326.log.gz 2024-04-25T09:05:21.120 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63141.log 2024-04-25T09:05:21.120 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2%/var/log/ceph/ceph-client.admin.75522.log: -- replaced with /var/log/ceph/ceph-client.admin.47599.log.gz 2024-04-25T09:05:21.120 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80929.log 2024-04-25T09:05:21.120 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.75522.log.gz 2024-04-25T09:05:21.120 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.63141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.76245.log 2024-04-25T09:05:21.120 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.80929.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.63141.log.gz 2024-04-25T09:05:21.120 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79740.log 2024-04-25T09:05:21.120 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.76245.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.80929.log.gz 2024-04-25T09:05:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99397.log 2024-04-25T09:05:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.79740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45199.log 2024-04-25T09:05:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.99397.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.79740.log.gz 2024-04-25T09:05:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95339.log 2024-04-25T09:05:21.121 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.99397.log.gz 2024-04-25T09:05:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91012.log 2024-04-25T09:05:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.95339.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45199.log.gz 2024-04-25T09:05:21.121 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45957.log 2024-04-25T09:05:21.122 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95339.log.gz 2024-04-25T09:05:21.122 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.91012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59687.log 2024-04-25T09:05:21.122 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.91012.log.gz 2024-04-25T09:05:21.122 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89631.log 2024-04-25T09:05:21.122 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.59687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95120.log 2024-04-25T09:05:21.122 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45957.log.gz 2024-04-25T09:05:21.122 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.89631.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59687.log.gz 2024-04-25T09:05:21.122 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63728.log 2024-04-25T09:05:21.123 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.89631.log.gz 2024-04-25T09:05:21.123 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.95120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90196.log 2024-04-25T09:05:21.123 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.63728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58604.log 2024-04-25T09:05:21.123 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95120.log.gz 2024-04-25T09:05:21.123 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.90196.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.63728.log.gz 2024-04-25T09:05:21.123 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89273.log 2024-04-25T09:05:21.123 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.90196.log.gz 2024-04-25T09:05:21.123 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.58604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47269.log 2024-04-25T09:05:21.123 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.89273.log: 87.6%gzip -5 --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.58604.log.gz 2024-04-25T09:05:21.124 INFO:teuthology.orchestra.run.smithi043.stderr: /var/log/ceph/ceph-client.admin.99229.log 2024-04-25T09:05:21.124 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47269.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.89273.log.gz 2024-04-25T09:05:21.124 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80845.log 2024-04-25T09:05:21.124 INFO:teuthology.orchestra.run.smithi043.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47269.log.gz 2024-04-25T09:05:21.124 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.99229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46597.log 2024-04-25T09:05:21.124 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99229.log.gz 2024-04-25T09:05:21.124 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.80845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58521.log 2024-04-25T09:05:21.124 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91182.log 2024-04-25T09:05:21.124 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.80845.log.gz 2024-04-25T09:05:21.125 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.58521.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.46597.log.gz 2024-04-25T09:05:21.125 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84335.log 2024-04-25T09:05:21.125 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89436.log 2024-04-25T09:05:21.125 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.91182.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.58521.log.gz 2024-04-25T09:05:21.125 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.84335.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.91182.log.gz 2024-04-25T09:05:21.125 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74663.log 2024-04-25T09:05:21.125 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91054.log 2024-04-25T09:05:21.125 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.89436.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.84335.log.gz 2024-04-25T09:05:21.125 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.74663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88973.log 2024-04-25T09:05:21.126 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.89436.log.gz 2024-04-25T09:05:21.126 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.74663.log.gz 2024-04-25T09:05:21.126 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.91054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98804.log 2024-04-25T09:05:21.126 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.88973.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.91054.log.gz 2024-04-25T09:05:21.126 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T09:05:21.126 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.98804.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.88973.log.gz 2024-04-25T09:05:21.126 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89841.log 2024-04-25T09:05:21.126 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.98804.log.gz 2024-04-25T09:05:21.126 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60747.log 2024-04-25T09:05:21.127 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.89841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80293.log 2024-04-25T09:05:21.127 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.89841.log.gz 2024-04-25T09:05:21.127 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.60747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74621.log 2024-04-25T09:05:21.127 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47725.log 2024-04-25T09:05:21.127 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.60747.log.gz 2024-04-25T09:05:21.127 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.80293.log: /var/log/ceph/ceph-client.admin.74621.log: gzip 88.2% -5 -- replaced with /var/log/ceph/ceph-client.admin.80293.log.gz --verbose 2024-04-25T09:05:21.127 INFO:teuthology.orchestra.run.smithi043.stderr: -- /var/log/ceph/ceph-client.admin.100513.log 2024-04-25T09:05:21.127 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56447.log 2024-04-25T09:05:21.127 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47725.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.74621.log.gz 2024-04-25T09:05:21.128 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.100513.log: 88.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.47725.log.gz 2024-04-25T09:05:21.128 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.96458.log 2024-04-25T09:05:21.128 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.100513.log.gz 2024-04-25T09:05:21.128 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80803.log 2024-04-25T09:05:21.128 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56447.log: 86.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T09:05:21.128 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.96458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80676.log 2024-04-25T09:05:21.128 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56447.log.gz 2024-04-25T09:05:21.128 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.96458.log.gz 2024-04-25T09:05:21.128 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.80803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44668.log 2024-04-25T09:05:21.129 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.80803.log.gz 2024-04-25T09:05:21.129 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.80676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65964.log 2024-04-25T09:05:21.129 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.80676.log.gz 2024-04-25T09:05:21.129 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78325.log 2024-04-25T09:05:21.129 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.65964.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44668.log.gz 2024-04-25T09:05:21.129 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94290.log 2024-04-25T09:05:21.129 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65964.log.gz 2024-04-25T09:05:21.129 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.78325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44374.log 2024-04-25T09:05:21.130 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.94290.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91452.log 2024-04-25T09:05:21.130 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.78325.log.gz 2024-04-25T09:05:21.130 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44374.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.94290.log.gz 2024-04-25T09:05:21.130 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44332.log 2024-04-25T09:05:21.130 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44374.log.gz 2024-04-25T09:05:21.130 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.91452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62001.log 2024-04-25T09:05:21.130 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59463.log 2024-04-25T09:05:21.130 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.62001.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.91452.log.gz 2024-04-25T09:05:21.131 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45673.log 2024-04-25T09:05:21.131 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44332.log.gz 2024-04-25T09:05:21.131 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62001.log.gz 2024-04-25T09:05:21.131 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.59463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59105.log 2024-04-25T09:05:21.131 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44290.log 2024-04-25T09:05:21.131 INFO:teuthology.orchestra.run.smithi043.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.59463.log.gz 2024-04-25T09:05:21.131 INFO:teuthology.orchestra.run.smithi043.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.45673.log.gz 2024-04-25T09:05:21.131 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.59105.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-25T09:05:21.131 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85220.log 2024-04-25T09:05:21.132 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44290.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59105.log.gz 2024-04-25T09:05:21.132 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59915.log 2024-04-25T09:05:21.132 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44290.log.gz 2024-04-25T09:05:21.132 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.85220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100723.log 2024-04-25T09:05:21.132 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.85220.log.gz 2024-04-25T09:05:21.132 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.59915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48412.log 2024-04-25T09:05:21.132 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75091.log 2024-04-25T09:05:21.132 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.100723.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.59915.log.gz 2024-04-25T09:05:21.132 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96020.log 2024-04-25T09:05:21.133 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.100723.log.gz 2024-04-25T09:05:21.133 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79049.log 2024-04-25T09:05:21.133 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.75091.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48412.log.gz 2024-04-25T09:05:21.133 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.96020.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75091.log.gz 2024-04-25T09:05:21.133 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80251.log 2024-04-25T09:05:21.133 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42373.log 2024-04-25T09:05:21.133 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.79049.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.96020.log.gz 2024-04-25T09:05:21.133 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.80251.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.79049.log.gz 2024-04-25T09:05:21.133 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47471.log 2024-04-25T09:05:21.133 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58371.log 2024-04-25T09:05:21.134 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42373.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.80251.log.gz 2024-04-25T09:05:21.134 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47471.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42373.log.gz 2024-04-25T09:05:21.134 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40335.log 2024-04-25T09:05:21.134 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90873.log 88.3% 2024-04-25T09:05:21.134 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47471.log.gz 2024-04-25T09:05:21.134 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.58371.log: /var/log/ceph/ceph-client.admin.40335.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58371.log.gz 2024-04-25T09:05:21.134 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46555.log 2024-04-25T09:05:21.134 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73922.log 2024-04-25T09:05:21.134 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40335.log.gz 2024-04-25T09:05:21.135 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.90873.log: /var/log/ceph/ceph-client.admin.46555.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.90873.log.gz 2024-04-25T09:05:21.135 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79427.log 2024-04-25T09:05:21.135 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46555.log.gz 2024-04-25T09:05:21.135 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.73922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47893.log 2024-04-25T09:05:21.135 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.79427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43296.log 2024-04-25T09:05:21.135 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73922.log.gz 2024-04-25T09:05:21.135 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47893.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.79427.log.gz 2024-04-25T09:05:21.135 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95649.log 2024-04-25T09:05:21.136 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.64398.log 2024-04-25T09:05:21.136 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43296.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47893.log.gz 2024-04-25T09:05:21.136 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.95649.log: gzip -5 --verbose -- 89.6% /var/log/ceph/ceph-client.admin.58973.log 2024-04-25T09:05:21.136 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43296.log.gz 2024-04-25T09:05:21.136 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.95649.log.gz 2024-04-25T09:05:21.136 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91934.log 2024-04-25T09:05:21.136 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.64398.log: /var/log/ceph/ceph-client.admin.58973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84046.log 2024-04-25T09:05:21.136 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.91934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88415.log 2024-04-25T09:05:21.136 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.91934.log.gz 2024-04-25T09:05:21.137 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.84046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85687.log 2024-04-25T09:05:21.137 INFO:teuthology.orchestra.run.smithi043.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.58973.log.gz 2024-04-25T09:05:21.137 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.88415.log: gzip -5 89.0% -- replaced with /var/log/ceph/ceph-client.admin.84046.log.gz 2024-04-25T09:05:21.137 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.89478.log 2024-04-25T09:05:21.137 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.85687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79656.log 2024-04-25T09:05:21.137 INFO:teuthology.orchestra.run.smithi043.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.64398.log.gz 2024-04-25T09:05:21.137 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.85687.log.gz 2024-04-25T09:05:21.137 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.89478.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.88415.log.gz 2024-04-25T09:05:21.137 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40903.log 2024-04-25T09:05:21.138 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95026.log 2024-04-25T09:05:21.138 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.79656.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.89478.log.gz 2024-04-25T09:05:21.138 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96542.log 2024-04-25T09:05:21.138 INFO:teuthology.orchestra.run.smithi043.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.79656.log.gz 2024-04-25T09:05:21.138 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6%/var/log/ceph/ceph-client.admin.95026.log: -- replaced with /var/log/ceph/ceph-client.admin.40903.log.gz 2024-04-25T09:05:21.138 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89147.log 2024-04-25T09:05:21.138 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.96542.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.95026.log.gz 2024-04-25T09:05:21.138 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62297.log 2024-04-25T09:05:21.139 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.96542.log.gz 2024-04-25T09:05:21.139 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61185.log 2024-04-25T09:05:21.139 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.89147.log: /var/log/ceph/ceph-client.admin.62297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73668.log 2024-04-25T09:05:21.139 INFO:teuthology.orchestra.run.smithi043.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.89147.log.gz 2024-04-25T09:05:21.139 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.62297.log.gz 2024-04-25T09:05:21.139 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.61185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55332.log 2024-04-25T09:05:21.139 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 88.4% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.61185.log.gz /var/log/ceph/ceph-client.admin.84167.log 2024-04-25T09:05:21.139 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-04-25T09:05:21.139 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.73668.log: /var/log/ceph/ceph-client.admin.55332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41165.log 2024-04-25T09:05:21.140 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.84167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66320.log 2024-04-25T09:05:21.140 INFO:teuthology.orchestra.run.smithi043.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.73668.log.gz 2024-04-25T09:05:21.140 INFO:teuthology.orchestra.run.smithi043.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.55332.log.gz 2024-04-25T09:05:21.140 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94453.log 2024-04-25T09:05:21.140 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41165.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.84167.log.gz 2024-04-25T09:05:21.140 INFO:teuthology.orchestra.run.smithi043.stderr:gzip/var/log/ceph/ceph-client.admin.66320.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.75341.log 2024-04-25T09:05:21.140 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41165.log.gz 2024-04-25T09:05:21.140 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.94453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65266.log 2024-04-25T09:05:21.140 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.66320.log.gz 2024-04-25T09:05:21.141 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.94453.log.gz 2024-04-25T09:05:21.141 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.75341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46639.log 2024-04-25T09:05:21.141 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75341.log.gzgzip 2024-04-25T09:05:21.141 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T09:05:21.141 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.65266.log: /var/log/ceph/ceph-client.admin.46639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60426.log 2024-04-25T09:05:21.141 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65266.log.gz 2024-04-25T09:05:21.141 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph.audit.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46639.log.gz 2024-04-25T09:05:21.141 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96584.log 2024-04-25T09:05:21.141 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65056.log 2024-04-25T09:05:21.142 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.60426.log: /var/log/ceph/ceph-client.admin.96584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.91777.log 2024-04-25T09:05:21.142 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60426.log.gz 2024-04-25T09:05:21.142 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.96584.log.gz 2024-04-25T09:05:21.142 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.65056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86164.log 2024-04-25T09:05:21.142 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.91777.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.65056.log.gz 2024-04-25T09:05:21.142 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84209.log 2024-04-25T09:05:21.142 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.86164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60113.log 2024-04-25T09:05:21.142 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.84209.log: gzip 88.1% -5 --verbose -- /var/log/ceph/ceph-client.admin.61778.log 2024-04-25T09:05:21.143 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.86164.log.gz 2024-04-25T09:05:21.143 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.60113.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.84209.log.gz 2024-04-25T09:05:21.143 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99650.log 2024-04-25T09:05:21.143 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.60113.log.gz 2024-04-25T09:05:21.144 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose/var/log/ceph/ceph-client.admin.61778.log: -- /var/log/ceph/ceph-client.admin.42205.log 2024-04-25T09:05:21.144 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.99650.log: 89.5% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T09:05:21.144 INFO:teuthology.orchestra.run.smithi043.stderr:gzip 87.9% -5 --verbose -- /var/log/ceph/ceph-client.admin.44500.log 2024-04-25T09:05:21.144 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61778.log.gz 2024-04-25T09:05:21.145 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42205.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.99650.log.gz 2024-04-25T09:05:21.145 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66656.log 2024-04-25T09:05:21.145 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42205.log.gz 2024-04-25T09:05:21.146 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44500.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55182.log 2024-04-25T09:05:21.146 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44500.log.gz 2024-04-25T09:05:21.146 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.66656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89883.log 2024-04-25T09:05:21.146 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84866.log 2024-04-25T09:05:21.147 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55182.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66656.log.gz 2024-04-25T09:05:21.147 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.89883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79385.log 2024-04-25T09:05:21.147 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.84866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46141.log 2024-04-25T09:05:21.147 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.89883.log.gz 2024-04-25T09:05:21.148 INFO:teuthology.orchestra.run.smithi043.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.55182.log.gz 2024-04-25T09:05:21.148 INFO:teuthology.orchestra.run.smithi043.stderr:gzip/var/log/ceph/ceph-client.admin.79385.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.89231.log 2024-04-25T09:05:21.148 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.84866.log.gz 2024-04-25T09:05:21.148 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94495.log 2024-04-25T09:05:21.148 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.79385.log.gz 2024-04-25T09:05:21.149 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46141.log: /var/log/ceph/ceph-client.admin.89231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41500.log 2024-04-25T09:05:21.149 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46141.log.gz 2024-04-25T09:05:21.149 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.94495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40987.log 2024-04-25T09:05:21.150 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.89231.log.gz 2024-04-25T09:05:21.150 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41500.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.94495.log.gz 2024-04-25T09:05:21.150 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74831.log 2024-04-25T09:05:21.150 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40987.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41500.log.gz 2024-04-25T09:05:21.151 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41841.log 2024-04-25T09:05:21.151 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.74831.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40987.log.gz 2024-04-25T09:05:21.151 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60155.log 2024-04-25T09:05:21.151 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.74831.log.gz 2024-04-25T09:05:21.152 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61353.log 2024-04-25T09:05:21.152 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79217.log 2024-04-25T09:05:21.152 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.60155.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41841.log.gz 2024-04-25T09:05:21.152 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.61353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91096.log 2024-04-25T09:05:21.153 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.61353.log.gz 2024-04-25T09:05:21.153 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.79217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93685.log 2024-04-25T09:05:21.153 INFO:teuthology.orchestra.run.smithi043.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.60155.log.gz 2024-04-25T09:05:21.153 INFO:teuthology.orchestra.run.smithi043.stderr:gzip/var/log/ceph/ceph-client.admin.91096.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.79217.log.gz 2024-04-25T09:05:21.154 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.93859.log 2024-04-25T09:05:21.154 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.93685.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.91096.log.gz 2024-04-25T09:05:21.154 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90704.log 2024-04-25T09:05:21.155 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.93859.log: gzip -5 --verbose -- 91.0% -- replaced with /var/log/ceph/ceph-client.admin.93685.log.gz 2024-04-25T09:05:21.155 INFO:teuthology.orchestra.run.smithi043.stderr: /var/log/ceph/ceph-client.admin.74537.log 2024-04-25T09:05:21.155 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.90704.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.93859.log.gz 2024-04-25T09:05:21.155 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47311.log 2024-04-25T09:05:21.156 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.90704.log.gz 2024-04-25T09:05:21.156 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.74537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73838.log 2024-04-25T09:05:21.156 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.74537.log.gz 2024-04-25T09:05:21.156 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80971.log 2024-04-25T09:05:21.157 INFO:teuthology.orchestra.run.smithi043.stderr:gzip/var/log/ceph/ceph-client.admin.73838.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47311.log.gz 2024-04-25T09:05:21.157 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.57040.log 2024-04-25T09:05:21.157 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.80971.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.73838.log.gz 2024-04-25T09:05:21.157 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41494.log 2024-04-25T09:05:21.158 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.80971.log.gz 2024-04-25T09:05:21.158 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.57040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96626.log 2024-04-25T09:05:21.158 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88673.log 2024-04-25T09:05:21.158 INFO:teuthology.orchestra.run.smithi043.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57040.log.gz 2024-04-25T09:05:21.159 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41494.log.gz 2024-04-25T09:05:21.159 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.96626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59213.log 2024-04-25T09:05:21.159 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57656.log 2024-04-25T09:05:21.160 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.88673.log: /var/log/ceph/ceph-client.admin.59213.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.88673.log.gz 2024-04-25T09:05:21.160 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose 91.0% -- replaced with /var/log/ceph/ceph-client.admin.96626.log.gz 2024-04-25T09:05:21.160 INFO:teuthology.orchestra.run.smithi043.stderr: -- /var/log/ceph/ceph-client.admin.90961.log 2024-04-25T09:05:21.160 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.57656.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.59213.log.gz 2024-04-25T09:05:21.160 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78577.log 2024-04-25T09:05:21.161 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.57656.log.gz 2024-04-25T09:05:21.161 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.90961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89757.log 2024-04-25T09:05:21.161 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.78577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62465.log 2024-04-25T09:05:21.161 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.90961.log.gz 2024-04-25T09:05:21.162 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4%/var/log/ceph/ceph-client.admin.89757.log: -- replaced with /var/log/ceph/ceph-client.admin.78577.log.gz 2024-04-25T09:05:21.162 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83662.log 2024-04-25T09:05:21.162 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.62465.log: 89.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34742.log 2024-04-25T09:05:21.162 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.89757.log.gz 2024-04-25T09:05:21.163 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.62465.log.gz 2024-04-25T09:05:21.163 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.83662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44458.log 2024-04-25T09:05:21.163 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75215.log 2024-04-25T09:05:21.163 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34742.log.gz 2024-04-25T09:05:21.163 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44458.log: gzip -5 91.0% --verbose -- /var/log/ceph/ceph-client.admin.47809.log 2024-04-25T09:05:21.163 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.83662.log.gz 2024-04-25T09:05:21.163 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.75215.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44458.log.gz 2024-04-25T09:05:21.164 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79698.log 2024-04-25T09:05:21.164 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.75215.log.gz 2024-04-25T09:05:21.164 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41835.log 2024-04-25T09:05:21.164 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47809.log.gz 2024-04-25T09:05:21.164 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.79698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63099.log 2024-04-25T09:05:21.164 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-25T09:05:21.164 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41835.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.79698.log.gz 2024-04-25T09:05:21.164 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.63099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83410.log 2024-04-25T09:05:21.164 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41835.log.gz 2024-04-25T09:05:21.165 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.63099.log.gz 2024-04-25T09:05:21.165 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85136.log 2024-04-25T09:05:21.165 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.83410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74915.log 2024-04-25T09:05:21.165 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.83410.log.gz 2024-04-25T09:05:21.165 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.85136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88349.log 2024-04-25T09:05:21.165 INFO:teuthology.orchestra.run.smithi043.stderr:gzip/var/log/ceph/ceph-client.admin.74915.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.56176.log 2024-04-25T09:05:21.165 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.85136.log.gz 2024-04-25T09:05:21.165 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.88349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48244.log 2024-04-25T09:05:21.165 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.74915.log.gz 2024-04-25T09:05:21.166 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56176.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.88349.log.gz 2024-04-25T09:05:21.166 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.56176.log.gz 2024-04-25T09:05:21.166 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84656.log 2024-04-25T09:05:21.166 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60047.log 2024-04-25T09:05:21.166 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48244.log.gz 2024-04-25T09:05:21.166 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.84656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40377.log 2024-04-25T09:05:21.166 INFO:teuthology.orchestra.run.smithi043.stderr:gzip/var/log/ceph/ceph-client.admin.60047.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.84656.log.gz 2024-04-25T09:05:21.167 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.84572.log 2024-04-25T09:05:21.167 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66172.log 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40377.log.gz 2024-04-25T09:05:21.167 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-04-25T09:05:21.167 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.84572.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.60047.log.gz 2024-04-25T09:05:21.167 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45463.log 87.2% -- replaced with /var/log/ceph/ceph-client.admin.84572.log.gz 2024-04-25T09:05:21.167 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-04-25T09:05:21.167 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.66172.log: gzip -5 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66172.log.gz 2024-04-25T09:05:21.167 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.62339.log 2024-04-25T09:05:21.168 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90831.log 2024-04-25T09:05:21.168 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.62339.log: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78283.log 2024-04-25T09:05:21.168 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45463.log.gz 2024-04-25T09:05:21.168 INFO:teuthology.orchestra.run.smithi043.stderr: 89.6%/var/log/ceph/ceph-client.admin.90831.log: -- replaced with /var/log/ceph/ceph-client.admin.62339.log.gz 2024-04-25T09:05:21.168 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99439.log 2024-04-25T09:05:21.168 INFO:teuthology.orchestra.run.smithi043.stderr:gzip 88.7% -- replaced with /var/log/ceph/ceph-client.admin.90831.log.gz 2024-04-25T09:05:21.168 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.78283.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.42037.log 2024-04-25T09:05:21.168 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.99439.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.78283.log.gz 2024-04-25T09:05:21.168 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40735.log 2024-04-25T09:05:21.169 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99439.log.gz 2024-04-25T09:05:21.169 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5/var/log/ceph/ceph-client.admin.42037.log: --verbose -- /var/log/ceph/ceph-client.admin.66848.log 2024-04-25T09:05:21.169 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40735.log: gzip -5 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42037.log.gz 2024-04-25T09:05:21.169 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.44164.log 2024-04-25T09:05:21.169 INFO:teuthology.orchestra.run.smithi043.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.40735.log.gz 2024-04-25T09:05:21.169 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.66848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89394.log 2024-04-25T09:05:21.169 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.66848.log.gz 2024-04-25T09:05:21.169 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66362.log 2024-04-25T09:05:21.170 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44164.log.gz 2024-04-25T09:05:21.170 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.89394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80381.log 2024-04-25T09:05:21.170 INFO:teuthology.orchestra.run.smithi043.stderr:gzip/var/log/ceph/ceph-client.admin.66362.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.57338.log 2024-04-25T09:05:21.170 INFO:teuthology.orchestra.run.smithi043.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.89394.log.gz 2024-04-25T09:05:21.170 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.80381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94047.log 2024-04-25T09:05:21.170 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.66362.log.gz 2024-04-25T09:05:21.170 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65308.log 2024-04-25T09:05:21.170 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.80381.log.gz/var/log/ceph/ceph-client.admin.57338.log: 2024-04-25T09:05:21.171 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.94047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80633.log 2024-04-25T09:05:21.171 INFO:teuthology.orchestra.run.smithi043.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57338.log.gz 2024-04-25T09:05:21.171 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.94047.log.gz 2024-04-25T09:05:21.171 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.56268.log 2024-04-25T09:05:21.171 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.65308.log: /var/log/ceph/ceph-client.admin.80633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86314.log 2024-04-25T09:05:21.171 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90788.log 2024-04-25T09:05:21.171 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.80633.log.gz 87.7% -- replaced with /var/log/ceph/ceph-client.admin.65308.log.gz 2024-04-25T09:05:21.171 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-04-25T09:05:21.172 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.86314.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56268.log.gz 2024-04-25T09:05:21.172 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66698.log 2024-04-25T09:05:21.172 INFO:teuthology.orchestra.run.smithi043.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.86314.log.gz 2024-04-25T09:05:21.172 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56008.log 2024-04-25T09:05:21.172 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.90788.log: /var/log/ceph/ceph-client.admin.66698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46723.log 2024-04-25T09:05:21.172 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.90788.log.gz 2024-04-25T09:05:21.172 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56008.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.66698.log.gz 2024-04-25T09:05:21.173 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58155.log 2024-04-25T09:05:21.173 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56008.log.gz 2024-04-25T09:05:21.173 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84698.log 2024-04-25T09:05:21.173 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46723.log: /var/log/ceph/ceph-client.admin.58155.log: gzip -5 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46723.log.gz 2024-04-25T09:05:21.174 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.47981.log 2024-04-25T09:05:21.174 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.84698.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.58155.log.gz 2024-04-25T09:05:21.174 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45057.log 2024-04-25T09:05:21.174 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.84698.log.gz 2024-04-25T09:05:21.174 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56531.log 2024-04-25T09:05:21.174 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47981.log: /var/log/ceph/ceph-client.admin.45057.log: gzip -5 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47981.log.gz 2024-04-25T09:05:21.174 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.58841.log 2024-04-25T09:05:21.175 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56531.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45057.log.gz 2024-04-25T09:05:21.175 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44836.log 2024-04-25T09:05:21.175 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56531.log.gz 2024-04-25T09:05:21.175 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99355.log 2024-04-25T09:05:21.175 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.58841.log: /var/log/ceph/ceph-client.admin.44836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95934.log 2024-04-25T09:05:21.175 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.99355.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44836.log.gz 2024-04-25T09:05:21.175 INFO:teuthology.orchestra.run.smithi043.stderr: 91.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74747.log 2024-04-25T09:05:21.175 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58841.log.gz 2024-04-25T09:05:21.175 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99355.log.gz 2024-04-25T09:05:21.176 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.95934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83728.log 2024-04-25T09:05:21.176 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64819.log 2024-04-25T09:05:21.176 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.95934.log.gz 2024-04-25T09:05:21.176 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.74747.log: /var/log/ceph/ceph-client.admin.83728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48202.log 2024-04-25T09:05:21.176 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2%/var/log/ceph/ceph-client.admin.64819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96332.log 2024-04-25T09:05:21.176 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.74747.log.gz 2024-04-25T09:05:21.176 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0% 90.8% -- replaced with /var/log/ceph/ceph-client.admin.83728.log.gz 2024-04-25T09:05:21.176 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.64819.log.gz/var/log/ceph/ceph-client.admin.48202.log: 2024-04-25T09:05:21.177 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100849.log 2024-04-25T09:05:21.177 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.96332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75710.log 2024-04-25T09:05:21.177 INFO:teuthology.orchestra.run.smithi043.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.48202.log.gz/var/log/ceph/ceph-client.admin.100849.log: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57548.log 2024-04-25T09:05:21.177 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.96332.log.gz 2024-04-25T09:05:21.177 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-04-25T09:05:21.177 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.75710.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.100849.log.gz 2024-04-25T09:05:21.177 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61490.log 2024-04-25T09:05:21.177 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.75710.log.gz 2024-04-25T09:05:21.177 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.57548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62874.log 2024-04-25T09:05:21.178 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57548.log.gz 2024-04-25T09:05:21.178 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.61490.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-25T09:05:21.178 INFO:teuthology.orchestra.run.smithi043.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.61490.log.gz 2024-04-25T09:05:21.178 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66214.log 2024-04-25T09:05:21.178 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.62874.log: /var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55840.log 2024-04-25T09:05:21.178 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.66214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66258.log 2024-04-25T09:05:21.178 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.66214.log.gz/var/log/ceph/ceph-client.admin.55840.log: 2024-04-25T09:05:21.178 INFO:teuthology.orchestra.run.smithi043.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.55840.log.gz 2024-04-25T09:05:21.179 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96062.log 2024-04-25T09:05:21.179 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.66258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88931.log 2024-04-25T09:05:21.179 INFO:teuthology.orchestra.run.smithi043.stderr: 92.0% -- replaced with /var/log/ceph/ceph-client.0.62874.log.gz 2024-04-25T09:05:21.179 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.66258.log.gz 2024-04-25T09:05:21.179 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.96062.log: gzip -5 88.6% -- replaced with /var/log/ceph/ceph-client.admin.96062.log.gz 2024-04-25T09:05:21.179 INFO:teuthology.orchestra.run.smithi043.stderr: --verbose -- /var/log/ceph/ceph-client.admin.94858.log 2024-04-25T09:05:21.179 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.88931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42331.log 2024-04-25T09:05:21.179 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.88931.log.gz 2024-04-25T09:05:21.179 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.94858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58263.log 2024-04-25T09:05:21.180 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.94858.log.gz 2024-04-25T09:05:21.180 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48160.log 2024-04-25T09:05:21.180 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.42331.log.gz/var/log/ceph/ceph-client.admin.58263.log: 2024-04-25T09:05:21.180 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83494.log 2024-04-25T09:05:21.180 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.58263.log.gz 2024-04-25T09:05:21.180 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46345.log 2024-04-25T09:05:21.180 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48160.log: /var/log/ceph/ceph-client.admin.83494.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48160.log.gz 2024-04-25T09:05:21.180 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.83494.log.gz 2024-04-25T09:05:21.181 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55687.log 2024-04-25T09:05:21.181 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85868.log 2024-04-25T09:05:21.181 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55687.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46345.log.gz 2024-04-25T09:05:21.181 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47017.log 2024-04-25T09:05:21.181 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55687.log.gz 2024-04-25T09:05:21.181 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98709.log 2024-04-25T09:05:21.181 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.85868.log: /var/log/ceph/ceph-client.admin.47017.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.85868.log.gz 2024-04-25T09:05:21.181 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47017.log.gz 2024-04-25T09:05:21.181 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75257.log 2024-04-25T09:05:21.182 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.98709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88547.log 2024-04-25T09:05:21.182 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.75257.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.98709.log.gz 2024-04-25T09:05:21.182 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89715.log 2024-04-25T09:05:21.182 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.75257.log.gz 2024-04-25T09:05:21.182 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85518.log 2024-04-25T09:05:21.182 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.88547.log: /var/log/ceph/ceph-client.admin.89715.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.88547.log.gz 2024-04-25T09:05:21.182 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.89715.log.gz 2024-04-25T09:05:21.182 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46807.log 2024-04-25T09:05:21.183 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.85518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89081.log 2024-04-25T09:05:21.183 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46807.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.85518.log.gz 2024-04-25T09:05:21.183 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100991.log 2024-04-25T09:05:21.183 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46807.log.gz 2024-04-25T09:05:21.183 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44248.log 2024-04-25T09:05:21.183 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.89081.log: /var/log/ceph/ceph-client.admin.100991.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.100991.log.gz 2024-04-25T09:05:21.183 INFO:teuthology.orchestra.run.smithi043.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.89081.log.gz 2024-04-25T09:05:21.184 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75986.log 2024-04-25T09:05:21.184 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45915.log 2024-04-25T09:05:21.184 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.75986.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44248.log.gz 2024-04-25T09:05:21.184 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57254.log 2024-04-25T09:05:21.184 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.75986.log.gz 2024-04-25T09:05:21.184 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79301.log 2024-04-25T09:05:21.184 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45915.log: /var/log/ceph/ceph-client.admin.57254.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45915.log.gz 2024-04-25T09:05:21.184 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57254.log.gz 2024-04-25T09:05:21.185 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74579.log 2024-04-25T09:05:21.185 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.79301.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-25T09:05:21.185 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.74579.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.79301.log.gz 2024-04-25T09:05:21.185 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60384.log 2024-04-25T09:05:21.185 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-25T09:05:21.185 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.74579.log.gz 2024-04-25T09:05:21.185 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph-client.admin.60384.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60384.log.gz 2024-04-25T09:05:21.185 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84782.log 2024-04-25T09:05:21.185 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44710.log 2024-04-25T09:05:21.186 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.84782.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.84782.log.gz 2024-04-25T09:05:21.186 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99117.log 2024-04-25T09:05:21.186 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44710.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44710.log.gz 2024-04-25T09:05:21.186 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79783.log 2024-04-25T09:05:21.186 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.99117.log: 94.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-25T09:05:21.186 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45589.log 2024-04-25T09:05:21.190 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.99117.log.gz 2024-04-25T09:05:21.190 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.79783.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.79783.log.gz 2024-04-25T09:05:21.190 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88865.log 2024-04-25T09:05:21.190 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79867.log 2024-04-25T09:05:21.190 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45589.log: /var/log/ceph/ceph-client.admin.88865.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45589.log.gz 2024-04-25T09:05:21.190 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44973.log 2024-04-25T09:05:21.191 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.79867.log: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.88865.log.gz 2024-04-25T09:05:21.191 INFO:teuthology.orchestra.run.smithi043.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.79867.log.gz 2024-04-25T09:05:21.191 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63183.log 2024-04-25T09:05:21.191 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55116.log 2024-04-25T09:05:21.191 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44973.log: /var/log/ceph/ceph-client.admin.63183.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44973.log.gz 2024-04-25T09:05:21.191 INFO:teuthology.orchestra.run.smithi043.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.63183.log.gz 2024-04-25T09:05:21.191 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43832.log 2024-04-25T09:05:21.191 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55116.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.55116.log.gz 2024-04-25T09:05:21.191 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100135.log 2024-04-25T09:05:21.192 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85262.log 2024-04-25T09:05:21.192 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43832.log: /var/log/ceph/ceph-client.admin.100135.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43832.log.gz 2024-04-25T09:05:21.192 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75794.log 2024-04-25T09:05:21.192 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.100135.log.gz 2024-04-25T09:05:21.192 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.85262.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.85262.log.gz 2024-04-25T09:05:21.192 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55645.log 2024-04-25T09:05:21.192 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78680.log 2024-04-25T09:05:21.193 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.75794.log: /var/log/ceph/ceph-client.admin.55645.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75794.log.gz 2024-04-25T09:05:21.193 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55645.log.gz 2024-04-25T09:05:21.193 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48779.log 2024-04-25T09:05:21.193 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.78680.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.78680.log.gz 2024-04-25T09:05:21.193 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74789.log 2024-04-25T09:05:21.193 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-25T09:05:21.193 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.48779.log: /var/log/ceph/ceph-client.admin.74789.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.74789.log.gz 2024-04-25T09:05:21.193 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63727.log 2024-04-25T09:05:21.194 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100051.log 2024-04-25T09:05:21.194 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.63727.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.63727.log.gz 2024-04-25T09:05:21.195 INFO:teuthology.orchestra.run.smithi193.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-25T09:05:21.200 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43996.log 2024-04-25T09:05:21.200 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.100051.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.100051.log.gz 2024-04-25T09:05:21.200 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61864.log 2024-04-25T09:05:21.200 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43996.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43996.log.gz 2024-04-25T09:05:21.200 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46225.log 2024-04-25T09:05:21.200 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.61864.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61864.log.gz 2024-04-25T09:05:21.201 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42163.log 2024-04-25T09:05:21.201 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46225.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46225.log.gz 2024-04-25T09:05:21.201 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65789.log 2024-04-25T09:05:21.201 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42163.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42163.log.gz 2024-04-25T09:05:21.201 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62549.log 2024-04-25T09:05:21.201 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.65789.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.65789.log.gz 2024-04-25T09:05:21.201 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90623.log 2024-04-25T09:05:21.201 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.62549.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.62549.log.gz 2024-04-25T09:05:21.203 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100387.log 2024-04-25T09:05:21.213 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.90623.log: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95978.log 2024-04-25T09:05:21.213 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.100387.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.100387.log.gz 2024-04-25T09:05:21.214 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.90623.log.gz 2024-04-25T09:05:21.221 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78938.log 2024-04-25T09:05:21.230 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.95978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40945.log 2024-04-25T09:05:21.234 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.78938.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95978.log.gz 2024-04-25T09:05:21.239 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88481.log 2024-04-25T09:05:21.243 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40945.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.78938.log.gz 2024-04-25T09:05:21.250 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96104.log 2024-04-25T09:05:21.251 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.88481.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.88481.log.gz 2024-04-25T09:05:21.252 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40945.log.gz 2024-04-25T09:05:21.261 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73578.log 2024-04-25T09:05:21.262 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.96104.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.96104.log.gz 2024-04-25T09:05:21.281 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41505.log 2024-04-25T09:05:21.282 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.73578.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.73578.log.gz 2024-04-25T09:05:21.290 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74201.log 2024-04-25T09:05:21.300 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96146.log 2024-04-25T09:05:21.303 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.74201.log: 88.4% 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41505.log.gz 2024-04-25T09:05:21.308 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80124.log 2024-04-25T09:05:21.313 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.96146.log: -- replaced with /var/log/ceph/ceph-client.admin.74201.log.gz 2024-04-25T09:05:21.316 INFO:teuthology.orchestra.run.smithi193.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-25T09:05:21.321 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.96146.log.gz 2024-04-25T09:05:21.325 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46429.log 2024-04-25T09:05:21.326 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.80124.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.80124.log.gz 2024-04-25T09:05:21.334 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94816.log 2024-04-25T09:05:21.344 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56134.log 2024-04-25T09:05:21.347 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.94816.log: 88.3% 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46429.log.gz 2024-04-25T09:05:21.352 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59612.log 2024-04-25T09:05:21.357 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56134.log: -- replaced with /var/log/ceph/ceph-client.admin.94816.log.gz 2024-04-25T09:05:21.361 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40251.log 2024-04-25T09:05:21.365 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.59612.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56134.log.gz 2024-04-25T09:05:21.370 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41837.log 2024-04-25T09:05:21.374 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40251.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.59612.log.gz 2024-04-25T09:05:21.379 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89799.log 2024-04-25T09:05:21.383 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41837.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40251.log.gz 2024-04-25T09:05:21.391 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94648.log 2024-04-25T09:05:21.391 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.89799.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.89799.log.gz 2024-04-25T09:05:21.393 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41837.log.gz 2024-04-25T09:05:21.399 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59147.log 2024-04-25T09:05:21.405 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.94648.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94648.log.gz 2024-04-25T09:05:21.415 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94369.log 2024-04-25T09:05:21.416 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.59147.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.59147.log.gz 2024-04-25T09:05:21.424 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64073.log 2024-04-25T09:05:21.433 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.94369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45283.log 2024-04-25T09:05:21.437 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.64073.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.94369.log.gz 2024-04-25T09:05:21.446 INFO:teuthology.orchestra.run.smithi043.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.64073.log.gz 2024-04-25T09:05:21.450 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85357.log 2024-04-25T09:05:21.451 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45283.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45283.log.gz 2024-04-25T09:05:21.467 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56871.log 2024-04-25T09:05:21.468 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.85357.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.85357.log.gz 2024-04-25T09:05:21.476 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74705.log 2024-04-25T09:05:21.488 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85178.log 2024-04-25T09:05:21.488 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.74705.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.74705.log.gz 2024-04-25T09:05:21.489 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56871.log.gz 2024-04-25T09:05:21.503 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78151.log 2024-04-25T09:05:21.504 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.85178.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.85178.log.gz 2024-04-25T09:05:21.512 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47227.log 2024-04-25T09:05:21.522 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.78151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101033.log 2024-04-25T09:05:21.525 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47227.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.78151.log.gz 2024-04-25T09:05:21.531 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56710.log 2024-04-25T09:05:21.532 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.101033.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.101033.log.gz 2024-04-25T09:05:21.534 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47227.log.gz 2024-04-25T09:05:21.540 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58679.log 2024-04-25T09:05:21.546 INFO:teuthology.orchestra.run.smithi193.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-25T09:05:21.551 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80423.log 2024-04-25T09:05:21.551 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.58679.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58679.log.gz 2024-04-25T09:05:21.553 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.56710.log.gz 2024-04-25T09:05:21.559 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94005.log 2024-04-25T09:05:21.572 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.80423.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.80423.log.gz 2024-04-25T09:05:21.580 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45873.log 2024-04-25T09:05:21.581 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.94005.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.94005.log.gz 2024-04-25T09:05:21.589 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58305.log 2024-04-25T09:05:21.599 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45873.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45873.log.gz 2024-04-25T09:05:21.599 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61694.log 2024-04-25T09:05:21.610 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.58305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90406.log 2024-04-25T09:05:21.610 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.61694.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61694.log.gz 2024-04-25T09:05:21.612 INFO:teuthology.orchestra.run.smithi043.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.58305.log.gz 2024-04-25T09:05:21.626 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65747.log 2024-04-25T09:05:21.627 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.90406.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.90406.log.gz 2024-04-25T09:05:21.641 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100345.log 2024-04-25T09:05:21.641 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.65747.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.65747.log.gz 2024-04-25T09:05:21.653 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96692.log 2024-04-25T09:05:21.654 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.100345.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.100345.log.gz 2024-04-25T09:05:21.674 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56405.log 2024-04-25T09:05:21.675 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.96692.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.96692.log.gz 2024-04-25T09:05:21.683 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79091.log 2024-04-25T09:05:21.693 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95465.log 2024-04-25T09:05:21.696 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.79091.log: 88.7% 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56405.log.gz 2024-04-25T09:05:21.701 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55603.log 2024-04-25T09:05:21.706 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.95465.log: -- replaced with /var/log/ceph/ceph-client.admin.79091.log.gz 2024-04-25T09:05:21.710 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62615.log 2024-04-25T09:05:21.714 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55603.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.95465.log.gz 2024-04-25T09:05:21.716 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.55603.log.gz 2024-04-25T09:05:21.719 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85560.log 2024-04-25T09:05:21.733 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.62615.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62615.log.gz 2024-04-25T09:05:21.738 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64693.log 2024-04-25T09:05:21.739 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.85560.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.85560.log.gz 2024-04-25T09:05:21.747 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63409.log 2024-04-25T09:05:21.756 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.64693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60468.log 2024-04-25T09:05:21.760 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.63409.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64693.log.gz 2024-04-25T09:05:21.765 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80507.log 2024-04-25T09:05:21.769 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.60468.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63409.log.gz 2024-04-25T09:05:21.770 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.60468.log.gz 2024-04-25T09:05:21.781 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84968.log 2024-04-25T09:05:21.782 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.80507.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.80507.log.gz 2024-04-25T09:05:21.797 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85645.log 2024-04-25T09:05:21.798 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.84968.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.84968.log.gz 2024-04-25T09:05:21.813 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91140.log 2024-04-25T09:05:21.814 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.85645.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.85645.log.gz 2024-04-25T09:05:21.822 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98846.log 2024-04-25T09:05:21.831 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.91140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45547.log 2024-04-25T09:05:21.835 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.98846.log: 91.4% 87.6% -- replaced with /var/log/ceph/ceph-client.admin.91140.log.gz 2024-04-25T09:05:21.840 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60621.log 2024-04-25T09:05:21.845 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45547.log: -- replaced with /var/log/ceph/ceph-client.admin.98846.log.gz 2024-04-25T09:05:21.849 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79521.log 2024-04-25T09:05:21.853 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.60621.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45547.log.gz 2024-04-25T09:05:21.858 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42121.log 2024-04-25T09:05:21.862 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.79521.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.60621.log.gz 2024-04-25T09:05:21.872 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.96951.log 2024-04-25T09:05:21.872 INFO:teuthology.orchestra.run.smithi043.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.79521.log.gz 2024-04-25T09:05:21.872 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42121.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.42121.log.gz 2024-04-25T09:05:21.880 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44794.log 2024-04-25T09:05:21.900 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.96951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55248.log 2024-04-25T09:05:21.901 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44794.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.44794.log.gz 2024-04-25T09:05:21.907 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41168.log 2024-04-25T09:05:21.916 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65392.log 2024-04-25T09:05:21.921 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41168.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.55248.log.gz 2024-04-25T09:05:21.925 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42667.log 2024-04-25T09:05:21.929 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.65392.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41168.log.gz 2024-04-25T09:05:21.934 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98991.log 2024-04-25T09:05:21.938 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42667.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.65392.log.gz 2024-04-25T09:05:21.947 INFO:teuthology.orchestra.run.smithi043.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.admin.42667.log.gz 2024-04-25T09:05:21.953 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57084.log 2024-04-25T09:05:21.954 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.98991.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.98991.log.gz 2024-04-25T09:05:21.962 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57590.log 2024-04-25T09:05:21.971 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.57084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75608.log 2024-04-25T09:05:21.975 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.57590.log: 91.1% 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57084.log.gz 2024-04-25T09:05:21.986 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57590.log.gz 2024-04-25T09:05:21.986 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60705.log 2024-04-25T09:05:21.986 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.75608.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.75608.log.gz 2024-04-25T09:05:21.995 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42541.log 2024-04-25T09:05:21.996 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.60705.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60705.log.gz 2024-04-25T09:05:22.006 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56955.log 2024-04-25T09:05:22.017 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42541.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42541.log.gz 2024-04-25T09:05:22.024 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55924.log 2024-04-25T09:05:22.025 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56955.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56955.log.gz 2024-04-25T09:05:22.038 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41503.log 2024-04-25T09:05:22.039 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55924.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.55924.log.gz 2024-04-25T09:05:22.047 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63267.log 2024-04-25T09:05:22.057 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60300.log 2024-04-25T09:05:22.060 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.63267.log: 87.7% 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41503.log.gz 2024-04-25T09:05:22.065 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100219.log 2024-04-25T09:05:22.067 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.60300.log: 93.8% -- replaced with /var/log/ceph/ceph-client.admin.63267.log.gz 2024-04-25T09:05:22.068 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.0.76245.log.gz 2024-04-25T09:05:22.075 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93751.log 2024-04-25T09:05:22.078 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.100219.log: 88.2% 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60300.log.gz 2024-04-25T09:05:22.084 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66614.log 2024-04-25T09:05:22.085 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.93751.log: 93.7% -- replaced with /var/log/ceph/ceph-client.0.91777.log.gz 2024-04-25T09:05:22.086 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.100219.log.gz 2024-04-25T09:05:22.087 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84614.log 2024-04-25T09:05:22.088 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.66614.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.66614.log.gz 2024-04-25T09:05:22.088 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47185.log 2024-04-25T09:05:22.091 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.84614.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.84614.log.gz 2024-04-25T09:05:22.091 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39989.log 2024-04-25T09:05:22.091 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47185.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47185.log.gz 2024-04-25T09:05:22.091 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75752.log 2024-04-25T09:05:22.091 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39989.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39989.log.gz 2024-04-25T09:05:22.091 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-25T09:05:22.093 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.75752.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.75752.log.gz 2024-04-25T09:05:22.093 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45505.log 2024-04-25T09:05:22.097 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-mon.a.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.93751.log.gz 2024-04-25T09:05:22.103 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48286.log 2024-04-25T09:05:22.103 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45505.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45505.log.gz 2024-04-25T09:05:22.116 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96500.log 2024-04-25T09:05:22.118 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48286.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48286.log.gz 2024-04-25T09:05:22.118 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93943.log 2024-04-25T09:05:22.124 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.96500.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61311.log 2024-04-25T09:05:22.124 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.96500.log.gz 2024-04-25T09:05:22.138 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56998.log 2024-04-25T09:05:22.138 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.93943.log: /var/log/ceph/ceph-client.admin.61311.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.93943.log.gz 2024-04-25T09:05:22.138 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.61311.log.gz 2024-04-25T09:05:22.144 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74117.log 2024-04-25T09:05:22.147 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56998.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.56998.log.gz 2024-04-25T09:05:22.153 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73880.log 2024-04-25T09:05:22.162 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.74117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78493.log 2024-04-25T09:05:22.163 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.73880.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.73880.log.gz 2024-04-25T09:05:22.166 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.74117.log.gz 2024-04-25T09:05:22.171 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90101.log 2024-04-25T09:05:22.175 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.78493.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.78493.log.gz 2024-04-25T09:05:22.187 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75836.log 2024-04-25T09:05:22.188 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.90101.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.90101.log.gz 2024-04-25T09:05:22.199 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62043.log 2024-04-25T09:05:22.200 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.75836.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.75836.log.gz 2024-04-25T09:05:22.211 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89189.log 2024-04-25T09:05:22.221 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.62043.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.62043.log.gz 2024-04-25T09:05:22.222 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59746.log 2024-04-25T09:05:22.223 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.89189.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.89189.log.gz 2024-04-25T09:05:22.231 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90154.log 2024-04-25T09:05:22.244 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.59746.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.59746.log.gz 2024-04-25T09:05:22.250 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65495.log 2024-04-25T09:05:22.251 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.90154.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.90154.log.gz 2024-04-25T09:05:22.264 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88799.log 2024-04-25T09:05:22.265 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.65495.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65495.log.gz 2024-04-25T09:05:22.273 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78367.log 2024-04-25T09:05:22.286 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.88799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65140.log 2024-04-25T09:05:22.286 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.78367.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.78367.log.gz 2024-04-25T09:05:22.286 INFO:teuthology.orchestra.run.smithi043.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.88799.log.gz 2024-04-25T09:05:22.294 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79614.log 2024-04-25T09:05:22.297 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.65140.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.65140.log.gz 2024-04-25T09:05:22.303 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99524.log 2024-04-25T09:05:22.316 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.79614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48032.log 2024-04-25T09:05:22.316 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.99524.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.99524.log.gz 2024-04-25T09:05:22.316 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.79614.log.gz 2024-04-25T09:05:22.324 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89925.log 2024-04-25T09:05:22.338 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48032.log: gzip -5 --verbose -- 89.0% -- replaced with /var/log/ceph/ceph-client.admin.48032.log.gz 2024-04-25T09:05:22.338 INFO:teuthology.orchestra.run.smithi043.stderr: /var/log/ceph/ceph-client.admin.90322.log 2024-04-25T09:05:22.338 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.89925.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.89925.log.gz 2024-04-25T09:05:22.350 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43192.log 2024-04-25T09:05:22.351 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.90322.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.90322.log.gz 2024-04-25T09:05:22.366 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64735.log 2024-04-25T09:05:22.367 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.43192.log: 90.3% -- replaced with /var/log/ceph/ceph-client.0.43192.log.gz 2024-04-25T09:05:22.375 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75878.log 2024-04-25T09:05:22.384 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.64735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90364.log 2024-04-25T09:05:22.388 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.75878.log: 88.9% 89.0% -- replaced with /var/log/ceph/ceph-client.admin.64735.log.gz 2024-04-25T09:05:22.393 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99075.log 2024-04-25T09:05:22.398 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.90364.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.75878.log.gz 2024-04-25T09:05:22.402 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83578.log 2024-04-25T09:05:22.408 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.99075.log: -- replaced with /var/log/ceph/ceph-client.admin.90364.log.gz 2024-04-25T09:05:22.409 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78619.log 2024-04-25T09:05:22.415 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.83578.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.99075.log.gz 2024-04-25T09:05:22.416 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60579.log 2024-04-25T09:05:22.422 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.78619.log: 88.0% 89.0% -- replaced with /var/log/ceph/ceph-client.admin.83578.log.gz 2024-04-25T09:05:22.423 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90917.log 2024-04-25T09:05:22.430 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.60579.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.78619.log.gz 2024-04-25T09:05:22.430 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63367.log 2024-04-25T09:05:22.437 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.90917.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60579.log.gz 2024-04-25T09:05:22.437 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63907.log 2024-04-25T09:05:22.444 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.63367.log: -- replaced with /var/log/ceph/ceph-client.admin.90917.log.gz 2024-04-25T09:05:22.444 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66087.log 2024-04-25T09:05:22.451 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.63907.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.63367.log.gz 2024-04-25T09:05:22.457 INFO:teuthology.orchestra.run.smithi043.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.63907.log.gz 2024-04-25T09:05:22.460 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60221.log 2024-04-25T09:05:22.460 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.66087.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.66087.log.gz 2024-04-25T09:05:22.478 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75480.log 2024-04-25T09:05:22.479 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.60221.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60221.log.gz 2024-04-25T09:05:22.491 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91350.log 2024-04-25T09:05:22.492 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.75480.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.75480.log.gz 2024-04-25T09:05:22.500 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93901.log 2024-04-25T09:05:22.506 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.91350.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.91350.log.gz 2024-04-25T09:05:22.509 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78854.log 2024-04-25T09:05:22.522 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.93901.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.93901.log.gz 2024-04-25T09:05:22.528 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67264.log 2024-04-25T09:05:22.529 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.78854.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.78854.log.gz 2024-04-25T09:05:22.537 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90009.log 2024-04-25T09:05:22.550 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.67264.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.67264.log.gz 2024-04-25T09:05:22.551 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47143.log 2024-04-25T09:05:22.552 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.90009.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.90009.log.gz 2024-04-25T09:05:22.570 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99033.log 2024-04-25T09:05:22.571 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47143.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47143.log.gz 2024-04-25T09:05:22.579 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45799.log 2024-04-25T09:05:22.588 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.99033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91976.log 2024-04-25T09:05:22.592 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45799.log: 88.4% 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99033.log.gz 2024-04-25T09:05:22.603 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45799.log.gz 2024-04-25T09:05:22.603 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84293.log 2024-04-25T09:05:22.603 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.91976.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.91976.log.gz 2024-04-25T09:05:22.618 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66740.log 2024-04-25T09:05:22.619 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.84293.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.84293.log.gz 2024-04-25T09:05:22.633 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64972.log 2024-04-25T09:05:22.634 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.66740.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.66740.log.gz 2024-04-25T09:05:22.642 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75173.log 2024-04-25T09:05:22.655 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.64972.log: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90280.log 2024-04-25T09:05:22.656 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.75173.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.75173.log.gz 2024-04-25T09:05:22.662 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.64972.log.gz 2024-04-25T09:05:22.664 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55798.log 2024-04-25T09:05:22.673 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.90280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60915.log 2024-04-25T09:05:22.677 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55798.log: 88.3% 88.1% -- replaced with /var/log/ceph/ceph-client.admin.90280.log.gz 2024-04-25T09:05:22.682 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62423.log 2024-04-25T09:05:22.687 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.60915.log: -- replaced with /var/log/ceph/ceph-client.admin.55798.log.gz 2024-04-25T09:05:22.695 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60915.log.gz 2024-04-25T09:05:22.703 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42457.log 2024-04-25T09:05:22.704 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.62423.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62423.log.gz 2024-04-25T09:05:22.724 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41840.log 2024-04-25T09:05:22.725 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42457.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42457.log.gz 2024-04-25T09:05:22.737 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74327.log 2024-04-25T09:05:22.738 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41840.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41840.log.gz 2024-04-25T09:05:22.755 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80761.log 2024-04-25T09:05:22.756 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.74327.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.74327.log.gz 2024-04-25T09:05:22.764 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95171.log 2024-04-25T09:05:22.773 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.80761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63614.log 2024-04-25T09:05:22.777 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.95171.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.80761.log.gz 2024-04-25T09:05:22.782 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46471.log 2024-04-25T09:05:22.786 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.63614.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.95171.log.gz 2024-04-25T09:05:22.791 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91266.log 2024-04-25T09:05:22.795 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46471.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.63614.log.gz 2024-04-25T09:05:22.804 INFO:teuthology.orchestra.run.smithi043.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46471.log.gz 2024-04-25T09:05:22.809 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58738.log 2024-04-25T09:05:22.810 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.91266.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.91266.log.gz 2024-04-25T09:05:22.827 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40633.log 2024-04-25T09:05:22.828 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.58738.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.58738.log.gz 2024-04-25T09:05:22.836 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80591.log 2024-04-25T09:05:22.845 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63451.log 2024-04-25T09:05:22.849 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.80591.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.40633.log.gz 2024-04-25T09:05:22.854 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57128.log 2024-04-25T09:05:22.858 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.63451.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.80591.log.gz 2024-04-25T09:05:22.867 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.63451.log.gz 2024-04-25T09:05:22.871 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93619.log 2024-04-25T09:05:22.872 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.57128.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57128.log.gz 2024-04-25T09:05:22.882 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90238.log 2024-04-25T09:05:22.883 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.93619.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.93619.log.gz 2024-04-25T09:05:22.891 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42769.log 2024-04-25T09:05:22.901 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.90238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59039.log 2024-04-25T09:05:22.904 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42769.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.90238.log.gz 2024-04-25T09:05:22.909 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79572.log 2024-04-25T09:05:22.913 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.59039.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42769.log.gz 2024-04-25T09:05:22.918 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83962.log 2024-04-25T09:05:22.922 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.79572.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.59039.log.gz 2024-04-25T09:05:22.927 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80208.log 2024-04-25T09:05:22.931 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.83962.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.79572.log.gz 2024-04-25T09:05:22.936 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99897.log 2024-04-25T09:05:22.940 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.80208.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.83962.log.gz 2024-04-25T09:05:22.945 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59529.log 2024-04-25T09:05:22.949 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.99897.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.80208.log.gz 2024-04-25T09:05:22.954 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99313.log 2024-04-25T09:05:22.958 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.59529.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.99897.log.gz 2024-04-25T09:05:22.960 INFO:teuthology.orchestra.run.smithi193.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-25T09:05:22.967 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75385.log 2024-04-25T09:05:22.968 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.99313.log: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.59529.log.gz 2024-04-25T09:05:22.976 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40208.log 2024-04-25T09:05:22.981 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.75385.log: -- replaced with /var/log/ceph/ceph-client.admin.99313.log.gz 2024-04-25T09:05:22.985 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46849.log 2024-04-25T09:05:22.989 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40208.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.75385.log.gz 2024-04-25T09:05:22.994 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84251.log 2024-04-25T09:05:22.998 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46849.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40208.log.gz 2024-04-25T09:05:23.003 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45157.log 2024-04-25T09:05:23.007 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.84251.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46849.log.gz 2024-04-25T09:05:23.012 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96290.log 2024-04-25T09:05:23.015 INFO:teuthology.orchestra.run.smithi193.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-25T09:05:23.016 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45157.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.84251.log.gz 2024-04-25T09:05:23.019 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45157.log.gz 2024-04-25T09:05:23.031 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89015.log 2024-04-25T09:05:23.032 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.96290.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.96290.log.gz 2024-04-25T09:05:23.039 INFO:teuthology.orchestra.run.smithi043.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.0.96951.log.gz 2024-04-25T09:05:23.040 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96374.log 2024-04-25T09:05:23.042 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.89015.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.89015.log.gz 2024-04-25T09:05:23.042 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42583.log 2024-04-25T09:05:23.043 INFO:teuthology.orchestra.run.smithi193.stderr: 94.3% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-25T09:05:23.045 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.96374.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.96374.log.gz 2024-04-25T09:05:23.045 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75564.log 2024-04-25T09:05:23.045 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42583.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42583.log.gz 2024-04-25T09:05:23.045 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74369.log 2024-04-25T09:05:23.047 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.75564.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.75564.log.gz 2024-04-25T09:05:23.047 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61395.log 2024-04-25T09:05:23.047 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.74369.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74369.log.gz 2024-04-25T09:05:23.049 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40419.log 2024-04-25T09:05:23.049 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.61395.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61395.log.gz 2024-04-25T09:05:23.049 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63571.log 2024-04-25T09:05:23.068 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40419.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40419.log.gz 2024-04-25T09:05:23.068 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44416.log 2024-04-25T09:05:23.068 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.63571.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.63571.log.gz 2024-04-25T09:05:23.068 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83452.log 2024-04-25T09:05:23.068 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44416.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44416.log.gz 2024-04-25T09:05:23.068 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73734.log 2024-04-25T09:05:23.069 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.83452.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.83452.log.gz 2024-04-25T09:05:23.069 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100177.log 2024-04-25T09:05:23.069 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.73734.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.73734.log.gz 2024-04-25T09:05:23.069 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41160.log 2024-04-25T09:05:23.069 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.100177.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100177.log.gz 2024-04-25T09:05:23.069 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40675.log 2024-04-25T09:05:23.069 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41160.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41160.log.gz 2024-04-25T09:05:23.069 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47059.log 2024-04-25T09:05:23.069 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40675.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40675.log.gz 2024-04-25T09:05:23.070 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84446.log 2024-04-25T09:05:23.070 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47059.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47059.log.gz 2024-04-25T09:05:23.070 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84088.log 2024-04-25T09:05:23.070 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.84446.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.84446.log.gz 2024-04-25T09:05:23.070 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86038.log 2024-04-25T09:05:23.070 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.84088.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.84088.log.gz 2024-04-25T09:05:23.070 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100891.log 2024-04-25T09:05:23.071 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.86038.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.86038.log.gz 2024-04-25T09:05:23.071 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41169.log 2024-04-25T09:05:23.071 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.100891.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100891.log.gz 2024-04-25T09:05:23.071 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100681.log 2024-04-25T09:05:23.071 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41169.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41169.log.gz 2024-04-25T09:05:23.104 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100093.log 2024-04-25T09:05:23.104 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.100681.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100681.log.gz 2024-04-25T09:05:23.104 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61143.log 2024-04-25T09:05:23.104 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.100093.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.100093.log.gz 2024-04-25T09:05:23.104 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57296.log 2024-04-25T09:05:23.105 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.61143.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.61143.log.gz 2024-04-25T09:05:23.105 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95764.log 2024-04-25T09:05:23.105 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.57296.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57296.log.gz 2024-04-25T09:05:23.105 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100639.log 2024-04-25T09:05:23.105 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.95764.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.95764.log.gz 2024-04-25T09:05:23.105 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99813.log 2024-04-25T09:05:23.105 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.100639.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100639.log.gz 2024-04-25T09:05:23.105 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41166.log 2024-04-25T09:05:23.106 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.99813.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.99813.log.gz 2024-04-25T09:05:23.106 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66572.log 2024-04-25T09:05:23.106 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41166.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41166.log.gz 2024-04-25T09:05:23.106 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94984.log 2024-04-25T09:05:23.106 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.66572.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.66572.log.gz 2024-04-25T09:05:23.106 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74243.log 2024-04-25T09:05:23.106 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.94984.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.94984.log.gz 2024-04-25T09:05:23.107 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90543.log 2024-04-25T09:05:23.107 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.74243.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74243.log.gz 2024-04-25T09:05:23.107 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45241.log 2024-04-25T09:05:23.107 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.90543.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.90543.log.gz 2024-04-25T09:05:23.107 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78085.log 2024-04-25T09:05:23.107 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45241.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45241.log.gz 2024-04-25T09:05:23.107 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99566.log 2024-04-25T09:05:23.107 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.78085.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.78085.log.gz 2024-04-25T09:05:23.108 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86206.log 2024-04-25T09:05:23.108 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.99566.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.99566.log.gz 2024-04-25T09:05:23.108 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88757.log 2024-04-25T09:05:23.108 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.86206.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.86206.log.gz 2024-04-25T09:05:23.108 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41838.log 2024-04-25T09:05:23.108 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.88757.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.88757.log.gz 2024-04-25T09:05:23.108 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66488.log 2024-04-25T09:05:23.108 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41838.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41838.log.gz 2024-04-25T09:05:23.108 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83920.log 2024-04-25T09:05:23.109 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.66488.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.66488.log.gz 2024-04-25T09:05:23.109 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41721.log 2024-04-25T09:05:23.109 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.83920.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.83920.log.gz 2024-04-25T09:05:23.109 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100261.log 2024-04-25T09:05:23.109 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41721.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41721.log.gz 2024-04-25T09:05:23.109 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65705.log 2024-04-25T09:05:23.109 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.100261.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100261.log.gz 2024-04-25T09:05:23.109 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58413.log 2024-04-25T09:05:23.109 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.65705.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.65705.log.gz 2024-04-25T09:05:23.110 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98949.log 2024-04-25T09:05:23.110 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.58413.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.58413.log.gz 2024-04-25T09:05:23.110 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100765.log 2024-04-25T09:05:23.110 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.98949.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.98949.log.gz 2024-04-25T09:05:23.110 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43790.log 2024-04-25T09:05:23.141 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.100765.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.100765.log.gz 2024-04-25T09:05:23.142 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62507.log 2024-04-25T09:05:23.142 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43790.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43790.log.gz 2024-04-25T09:05:23.142 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74006.log 2024-04-25T09:05:23.142 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.62507.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.62507.log.gz 2024-04-25T09:05:23.142 INFO:teuthology.orchestra.run.smithi043.stderr:gzip 90.8% -5 --verbose -- /var/log/ceph/ceph-client.admin.62255.log 2024-04-25T09:05:23.142 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-25T09:05:23.142 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40549.log 2024-04-25T09:05:23.143 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.62255.log: /var/log/ceph/ceph-client.admin.74006.log: 87.4% 88.0% -- replaced with /var/log/ceph/ceph-client.admin.74006.log.gz 2024-04-25T09:05:23.143 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62255.log.gz 2024-04-25T09:05:23.143 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94248.log 2024-04-25T09:05:23.143 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45015.log 2024-04-25T09:05:23.143 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40549.log: /var/log/ceph/ceph-client.admin.94248.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40549.log.gz 2024-04-25T09:05:23.143 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61959.log 2024-04-25T09:05:23.143 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.94248.log.gz 2024-04-25T09:05:23.143 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73446.log 2024-04-25T09:05:23.144 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.61959.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45015.log.gz 2024-04-25T09:05:23.144 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61959.log.gz 2024-04-25T09:05:23.144 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80465.log 2024-04-25T09:05:23.144 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.73446.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.73446.log.gzgzip 2024-04-25T09:05:23.144 INFO:teuthology.orchestra.run.smithi043.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.73964.log 2024-04-25T09:05:23.144 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94206.log 2024-04-25T09:05:23.144 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.73964.log: /var/log/ceph/ceph-client.admin.80465.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.80465.log.gz 2024-04-25T09:05:23.144 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.73964.log.gz 2024-04-25T09:05:23.145 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40085.log 2024-04-25T09:05:23.145 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60957.log 2024-04-25T09:05:23.145 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.94206.log: /var/log/ceph/ceph-client.admin.40085.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.94206.log.gz 2024-04-25T09:05:23.145 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65884.log 2024-04-25T09:05:23.145 INFO:teuthology.orchestra.run.smithi043.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40085.log.gz 2024-04-25T09:05:23.145 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.60957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79175.log 2024-04-25T09:05:23.145 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.65884.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.60957.log.gz 2024-04-25T09:05:23.145 INFO:teuthology.orchestra.run.smithi043.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.65884.log.gz 2024-04-25T09:05:23.146 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83620.log 2024-04-25T09:05:23.146 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48328.log 2024-04-25T09:05:23.146 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.79175.log: /var/log/ceph/ceph-client.admin.83620.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.79175.log.gz 2024-04-25T09:05:23.146 INFO:teuthology.orchestra.run.smithi043.stderr: 89.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83368.log 2024-04-25T09:05:23.146 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.83620.log.gz 2024-04-25T09:05:23.146 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64573.log 2024-04-25T09:05:23.146 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48328.log: /var/log/ceph/ceph-client.admin.83368.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48328.log.gz 2024-04-25T09:05:23.147 INFO:teuthology.orchestra.run.smithi043.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.83368.log.gz 2024-04-25T09:05:23.147 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.28406.log 2024-04-25T09:05:23.147 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.64573.log: -- /var/log/ceph/ceph-client.admin.80719.log 2024-04-25T09:05:23.147 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph.tmp-client.admin.28406.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.28406.log.gz 2024-04-25T09:05:23.147 INFO:teuthology.orchestra.run.smithi043.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.64573.log.gz 2024-04-25T09:05:23.147 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85996.log 2024-04-25T09:05:23.147 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.80719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47555.log 2024-04-25T09:05:23.147 INFO:teuthology.orchestra.run.smithi043.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.80719.log.gz 2024-04-25T09:05:23.147 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.85996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94732.log 2024-04-25T09:05:23.148 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9%/var/log/ceph/ceph-client.admin.47555.log: -- replaced with /var/log/ceph/ceph-client.admin.85996.log.gz 2024-04-25T09:05:23.148 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47555.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83836.log 2024-04-25T09:05:23.148 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-04-25T09:05:23.148 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61652.log 2024-04-25T09:05:23.148 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.83836.log: /var/log/ceph/ceph-client.admin.94732.log: 88.0% 89.4% -- replaced with /var/log/ceph/ceph-client.admin.94732.log.gz 2024-04-25T09:05:23.148 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.83836.log.gz 2024-04-25T09:05:23.148 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74873.log 2024-04-25T09:05:23.148 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90448.log 2024-04-25T09:05:23.148 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.61652.log: /var/log/ceph/ceph-client.admin.74873.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.61652.log.gz 89.0% -- replaced with /var/log/ceph/ceph-client.admin.74873.log.gz 2024-04-25T09:05:23.149 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-04-25T09:05:23.149 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43954.log 2024-04-25T09:05:23.149 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.90448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46267.log 2024-04-25T09:05:23.149 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.90448.log.gz 2024-04-25T09:05:23.149 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.43954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94537.log 2024-04-25T09:05:23.149 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46267.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43954.log.gz 2024-04-25T09:05:23.149 INFO:teuthology.orchestra.run.smithi043.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46267.log.gz 2024-04-25T09:05:23.149 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42625.log 2024-04-25T09:05:23.150 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61227.log 2024-04-25T09:05:23.150 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.94537.log: /var/log/ceph/ceph-client.admin.42625.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.94537.log.gz 2024-04-25T09:05:23.150 INFO:teuthology.orchestra.run.smithi043.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42625.log.gz 2024-04-25T09:05:23.150 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60789.log 2024-04-25T09:05:23.150 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.61227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99708.log 2024-04-25T09:05:23.150 INFO:teuthology.orchestra.run.smithi043.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.61227.log.gz 2024-04-25T09:05:23.150 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.60789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89352.log 2024-04-25T09:05:23.150 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.99708.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60789.log.gz 2024-04-25T09:05:23.151 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.99708.log.gz 2024-04-25T09:05:23.151 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83536.log 2024-04-25T09:05:23.151 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40293.log 2024-04-25T09:05:23.151 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.83536.log: /var/log/ceph/ceph-client.admin.89352.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.89352.log.gz 2024-04-25T09:05:23.151 INFO:teuthology.orchestra.run.smithi043.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.83536.log.gz 2024-04-25T09:05:23.151 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86080.log 2024-04-25T09:05:23.152 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84908.log 2024-04-25T09:05:23.152 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40293.log: /var/log/ceph/ceph-client.admin.86080.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40293.log.gz 2024-04-25T09:05:23.152 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.86080.log.gz 2024-04-25T09:05:23.152 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59331.log 2024-04-25T09:05:23.152 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.84908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66446.log 2024-04-25T09:05:23.152 INFO:teuthology.orchestra.run.smithi043.stderr: 89.2%/var/log/ceph/ceph-client.admin.59331.log: -- replaced with /var/log/ceph/ceph-client.admin.84908.log.gz 2024-04-25T09:05:23.152 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94149.log 2024-04-25T09:05:23.152 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.66446.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.66446.log.gz 90.9% -- replaced with /var/log/ceph/ceph-client.admin.59331.log.gz 2024-04-25T09:05:23.152 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-04-25T09:05:23.153 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57212.log 2024-04-25T09:05:23.153 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.57915.log 2024-04-25T09:05:23.153 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.94149.log: /var/log/ceph/ceph-client.admin.57212.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.94149.log.gz 2024-04-25T09:05:23.153 INFO:teuthology.orchestra.run.smithi043.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57212.log.gz 2024-04-25T09:05:23.153 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44626.log 2024-04-25T09:05:23.153 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41839.log 2024-04-25T09:05:23.153 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44626.log: /var/log/ceph/ceph-client.0.57915.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44626.log.gz 2024-04-25T09:05:23.153 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99855.log 2024-04-25T09:05:23.153 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41839.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41839.log.gz 2024-04-25T09:05:23.154 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59849.log 2024-04-25T09:05:23.154 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.99855.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.99855.log.gz 2024-04-25T09:05:23.154 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80549.log 2024-04-25T09:05:23.154 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.59849.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.59849.log.gz 2024-04-25T09:05:23.161 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75650.log 2024-04-25T09:05:23.162 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.80549.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.80549.log.gz 2024-04-25T09:05:23.162 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61570.log 2024-04-25T09:05:23.162 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.75650.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.75650.log.gz 2024-04-25T09:05:23.162 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79469.log 2024-04-25T09:05:23.162 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.61570.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.61570.log.gz 2024-04-25T09:05:23.162 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66404.log 2024-04-25T09:05:23.162 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.79469.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.79469.log.gz 2024-04-25T09:05:23.163 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66129.log 2024-04-25T09:05:23.163 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.66404.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.66404.log.gz 2024-04-25T09:05:23.163 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99771.log 2024-04-25T09:05:23.163 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.66129.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.66129.log.gz 2024-04-25T09:05:23.163 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65182.log 2024-04-25T09:05:23.163 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.99771.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99771.log.gz 2024-04-25T09:05:23.164 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39908.log 2024-04-25T09:05:23.164 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.65182.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65182.log.gz 2024-04-25T09:05:23.164 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74285.log 2024-04-25T09:05:23.164 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.39908.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39908.log.gz 2024-04-25T09:05:23.164 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60663.log 2024-04-25T09:05:23.164 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.74285.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.74285.log.gz 2024-04-25T09:05:23.165 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59397.log 2024-04-25T09:05:23.165 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.60663.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.60663.log.gz 2024-04-25T09:05:23.165 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100807.log 2024-04-25T09:05:23.165 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.59397.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.59397.log.gz 2024-04-25T09:05:23.165 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46765.log 2024-04-25T09:05:23.165 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.100807.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100807.log.gz 2024-04-25T09:05:23.165 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85731.log 2024-04-25T09:05:23.165 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46765.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46765.log.gz 2024-04-25T09:05:23.165 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95381.log 2024-04-25T09:05:23.166 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.85731.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.85731.log.gz 2024-04-25T09:05:23.166 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63729.log 2024-04-25T09:05:23.166 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.95381.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.95381.log.gz 2024-04-25T09:05:23.166 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64139.log 2024-04-25T09:05:23.166 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.63729.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63729.log.gz 2024-04-25T09:05:23.166 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88589.log 2024-04-25T09:05:23.166 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.64139.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.64139.log.gz 2024-04-25T09:05:23.167 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65014.log 2024-04-25T09:05:23.170 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.88589.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.88589.log.gz 2024-04-25T09:05:23.170 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63493.log 2024-04-25T09:05:23.170 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.65014.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.65014.log.gz 2024-04-25T09:05:23.170 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96188.log 2024-04-25T09:05:23.171 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.63493.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.63493.log.gz 2024-04-25T09:05:23.171 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100471.log 2024-04-25T09:05:23.171 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.96188.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.96188.log.gz 2024-04-25T09:05:23.171 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40777.log 2024-04-25T09:05:23.171 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.100471.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100471.log.gz 2024-04-25T09:05:23.171 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99481.log 2024-04-25T09:05:23.171 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40777.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40777.log.gz 2024-04-25T09:05:23.171 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57422.log 2024-04-25T09:05:23.172 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.99481.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99481.log.gz 2024-04-25T09:05:23.172 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95297.log 2024-04-25T09:05:23.172 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.57422.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57422.log.gz 2024-04-25T09:05:23.172 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67107.log 2024-04-25T09:05:23.173 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.95297.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95297.log.gz 2024-04-25T09:05:23.173 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64777.log 2024-04-25T09:05:23.184 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.67107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60342.log 2024-04-25T09:05:23.185 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.64777.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.64777.log.gz 2024-04-25T09:05:23.198 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56615.log 2024-04-25T09:05:23.199 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.60342.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60342.log.gz 2024-04-25T09:05:23.199 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91308.log 2024-04-25T09:05:23.200 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56615.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56615.log.gz 2024-04-25T09:05:23.212 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48370.log 2024-04-25T09:05:23.213 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.91308.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.91308.log.gz 2024-04-25T09:05:23.220 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45715.log 2024-04-25T09:05:23.220 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48370.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.48370.log.gz 2024-04-25T09:05:23.229 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75429.log 2024-04-25T09:05:23.229 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45715.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45715.log.gz 2024-04-25T09:05:23.238 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79259.log 2024-04-25T09:05:23.239 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.75429.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.75429.log.gz 2024-04-25T09:05:23.259 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62129.log 2024-04-25T09:05:23.260 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.79259.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.79259.log.gz 2024-04-25T09:05:23.268 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44752.log 2024-04-25T09:05:23.268 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.62129.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.62129.log.gz 2024-04-25T09:05:23.277 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66782.log 2024-04-25T09:05:23.277 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44752.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44752.log.gz 2024-04-25T09:05:23.291 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95560.log 2024-04-25T09:05:23.292 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.66782.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.66782.log.gz 2024-04-25T09:05:23.305 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47391.log 2024-04-25T09:05:23.305 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.95560.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.95560.log.gz 2024-04-25T09:05:23.325 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84740.log 2024-04-25T09:05:23.326 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47391.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47391.log.gz 2024-04-25T09:05:23.334 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40591.log 2024-04-25T09:05:23.343 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.84740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56913.log 2024-04-25T09:05:23.347 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40591.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.84740.log.gz 2024-04-25T09:05:23.352 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99608.log 2024-04-25T09:05:23.356 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56913.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40591.log.gz 2024-04-25T09:05:23.361 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78409.log 2024-04-25T09:05:23.365 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.99608.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56913.log.gz 2024-04-25T09:05:23.374 INFO:teuthology.orchestra.run.smithi043.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.99608.log.gz 2024-04-25T09:05:23.382 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47641.log 2024-04-25T09:05:23.382 INFO:teuthology.orchestra.run.smithi043.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-25T09:05:23.385 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.78409.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.78409.log.gz 2024-04-25T09:05:23.385 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81013.log 2024-04-25T09:05:23.385 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47641.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47641.log.gz 2024-04-25T09:05:23.385 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93817.log 2024-04-25T09:05:23.387 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.81013.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.81013.log.gz 2024-04-25T09:05:23.388 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74159.log 2024-04-25T09:05:23.388 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.93817.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.93817.log.gz 2024-04-25T09:05:23.388 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65098.log 2024-04-25T09:05:23.390 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.74159.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.74159.log.gz 2024-04-25T09:05:23.390 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56218.log 2024-04-25T09:05:23.390 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.65098.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65098.log.gz 2024-04-25T09:05:23.391 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85010.log 2024-04-25T09:05:23.393 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56218.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56218.log.gz 2024-04-25T09:05:23.393 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79963.log 2024-04-25T09:05:23.393 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.85010.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.85010.log.gz 2024-04-25T09:05:23.393 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94089.log 2024-04-25T09:05:23.395 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.79963.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.79963.log.gz 2024-04-25T09:05:23.395 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46513.log 2024-04-25T09:05:23.396 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.94089.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94089.log.gz 2024-04-25T09:05:23.396 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91518.log 2024-04-25T09:05:23.397 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46513.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46513.log.gz 2024-04-25T09:05:23.397 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81079.log 2024-04-25T09:05:23.399 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.91518.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.91518.log.gz 2024-04-25T09:05:23.399 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59981.log 2024-04-25T09:05:23.399 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.81079.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.81079.log.gz 2024-04-25T09:05:23.400 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74453.log 2024-04-25T09:05:23.401 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.59981.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.59981.log.gz 2024-04-25T09:05:23.402 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75299.log 2024-04-25T09:05:23.404 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.74453.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.74453.log.gz 2024-04-25T09:05:23.404 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78451.log 2024-04-25T09:05:23.404 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.75299.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.75299.log.gz 2024-04-25T09:05:23.405 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83128.log 2024-04-25T09:05:23.406 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.78451.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.78451.log.gz 2024-04-25T09:05:23.406 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57506.log 2024-04-25T09:05:23.408 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.83128.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.83128.log.gz 2024-04-25T09:05:23.408 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46681.log 2024-04-25T09:05:23.408 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.57506.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.57506.log.gz 2024-04-25T09:05:23.410 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40463.log 2024-04-25T09:05:23.410 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46681.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.46681.log.gz 2024-04-25T09:05:23.410 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98751.log 2024-04-25T09:05:23.412 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40463.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40463.log.gz 2024-04-25T09:05:23.412 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65579.log 2024-04-25T09:05:23.413 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.98751.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.98751.log.gz 2024-04-25T09:05:23.413 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56790.log 2024-04-25T09:05:23.414 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.65579.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.65579.log.gz 2024-04-25T09:05:23.415 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95848.log 2024-04-25T09:05:23.416 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56790.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.56790.log.gz 2024-04-25T09:05:23.416 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61041.log 2024-04-25T09:05:23.416 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.95848.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.95848.log.gz 2024-04-25T09:05:23.418 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63730.log 2024-04-25T09:05:23.418 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.61041.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61041.log.gz 2024-04-25T09:05:23.420 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46041.log 2024-04-25T09:05:23.420 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.63730.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.63730.log.gz 2024-04-25T09:05:23.420 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59793.log 2024-04-25T09:05:23.421 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46041.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.46041.log.gz 2024-04-25T09:05:23.421 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100303.log 2024-04-25T09:05:23.423 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.59793.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.59793.log.gz 2024-04-25T09:05:23.423 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78811.log 2024-04-25T09:05:23.423 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.100303.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.100303.log.gz 2024-04-25T09:05:23.425 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94900.log 2024-04-25T09:05:23.425 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.78811.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.78811.log.gz 2024-04-25T09:05:23.426 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47767.log 2024-04-25T09:05:23.427 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.94900.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.94900.log.gz 2024-04-25T09:05:23.427 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-25T09:05:23.428 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47767.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47767.log.gz 2024-04-25T09:05:23.429 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61269.log 2024-04-25T09:05:23.442 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78896.log 2024-04-25T09:05:23.442 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.61269.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.61269.log.gz 2024-04-25T09:05:23.455 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34861.log 2024-04-25T09:05:23.457 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.78896.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.78896.log.gz 2024-04-25T09:05:23.457 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57170.log 2024-04-25T09:05:23.468 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.34861.log: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96230.log 2024-04-25T09:05:23.468 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34861.log.gz 2024-04-25T09:05:23.469 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.57170.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.57170.log.gz 2024-04-25T09:05:23.487 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41492.log 2024-04-25T09:05:23.488 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.96230.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.96230.log.gz 2024-04-25T09:05:23.497 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42247.log 2024-04-25T09:05:23.498 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41492.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41492.log.gz 2024-04-25T09:05:23.507 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42289.log 2024-04-25T09:05:23.507 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42247.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42247.log.gz 2024-04-25T09:05:23.520 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.86573.log 2024-04-25T09:05:23.521 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42289.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42289.log.gz 2024-04-25T09:05:23.533 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78769.log 2024-04-25T09:05:23.548 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.86573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65621.log 2024-04-25T09:05:23.548 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.78769.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.78769.log.gz 2024-04-25T09:05:23.556 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78535.log 2024-04-25T09:05:23.557 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.65621.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.65621.log.gz 2024-04-25T09:05:23.578 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83194.log 2024-04-25T09:05:23.578 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.78535.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.78535.log.gz 2024-04-25T09:05:23.586 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48074.log 2024-04-25T09:05:23.587 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.83194.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.83194.log.gz 2024-04-25T09:05:23.603 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41498.log 2024-04-25T09:05:23.603 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48074.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48074.log.gz 2024-04-25T09:05:23.611 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100597.log 2024-04-25T09:05:23.612 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41498.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41498.log.gz 2024-04-25T09:05:23.628 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61908.log 2024-04-25T09:05:23.629 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.100597.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.100597.log.gz 2024-04-25T09:05:23.630 INFO:teuthology.orchestra.run.smithi193.stderr: 94.3% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-25T09:05:23.643 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40819.log 2024-04-25T09:05:23.646 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.61908.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61908.log.gz 2024-04-25T09:05:23.646 INFO:teuthology.orchestra.run.smithi043.stderr: 94.2% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-25T09:05:23.658 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89520.log 2024-04-25T09:05:23.659 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40819.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40819.log.gz 2024-04-25T09:05:23.667 INFO:teuthology.orchestra.run.smithi193.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-25T09:05:23.683 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80043.log 2024-04-25T09:05:23.684 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.89520.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.89520.log.gz 2024-04-25T09:05:23.692 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65537.log 2024-04-25T09:05:23.701 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.80043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80337.log 2024-04-25T09:05:23.705 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.65537.log: 88.5% 88.2% -- replaced with /var/log/ceph/ceph-client.admin.80043.log.gz 2024-04-25T09:05:23.710 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63969.log 2024-04-25T09:05:23.714 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.80337.log: -- replaced with /var/log/ceph/ceph-client.admin.65537.log.gz 2024-04-25T09:05:23.721 INFO:teuthology.orchestra.run.smithi043.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.80337.log.gz 2024-04-25T09:05:23.722 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83878.log 2024-04-25T09:05:23.723 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.63969.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.63969.log.gz 2024-04-25T09:05:23.731 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40861.log 2024-04-25T09:05:23.741 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.83878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58907.log 2024-04-25T09:05:23.744 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.40861.log: 88.7% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.83878.log.gz 2024-04-25T09:05:23.749 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85602.log 2024-04-25T09:05:23.751 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.58907.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.58907.log.gz 2024-04-25T09:05:23.753 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40861.log.gz 2024-04-25T09:05:23.762 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46891.log 2024-04-25T09:05:23.763 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.85602.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.85602.log.gz 2024-04-25T09:05:23.771 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48116.log 2024-04-25T09:05:23.780 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46891.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55966.log 2024-04-25T09:05:23.785 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48116.log: -- replaced with /var/log/ceph/ceph-client.admin.46891.log.gz 2024-04-25T09:05:23.789 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84530.log 2024-04-25T09:05:23.793 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55966.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.48116.log.gz 2024-04-25T09:05:23.802 INFO:teuthology.orchestra.run.smithi043.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55966.log.gz 2024-04-25T09:05:23.807 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80166.log 2024-04-25T09:05:23.808 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.84530.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.84530.log.gz 2024-04-25T09:05:23.818 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65350.log 2024-04-25T09:05:23.819 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.80166.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.80166.log.gz 2024-04-25T09:05:23.837 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75011.log 2024-04-25T09:05:23.838 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.65350.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.65350.log.gz 2024-04-25T09:05:23.846 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42079.log 2024-04-25T09:05:23.855 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.75011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100555.log 2024-04-25T09:05:23.859 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42079.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.75011.log.gz 2024-04-25T09:05:23.864 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44584.log 2024-04-25T09:05:23.868 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.100555.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42079.log.gz 2024-04-25T09:05:23.877 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.100555.log.gz 2024-04-25T09:05:23.878 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59289.log 2024-04-25T09:05:23.879 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44584.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44584.log.gz 2024-04-25T09:05:23.887 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41167.log 2024-04-25T09:05:23.897 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.59289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.91410.log 2024-04-25T09:05:23.900 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41167.log: 87.8% 89.1% -- replaced with /var/log/ceph/ceph-client.admin.59289.log.gz 2024-04-25T09:05:23.908 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48454.log 2024-04-25T09:05:23.909 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.91410.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.91410.log.gz 2024-04-25T09:05:23.910 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41167.log.gz 2024-04-25T09:05:23.917 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78217.log 2024-04-25T09:05:23.926 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48454.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.48454.log.gz 2024-04-25T09:05:23.938 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44038.log 2024-04-25T09:05:23.939 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.78217.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.78217.log.gz 2024-04-25T09:05:23.947 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42499.log 2024-04-25T09:05:23.956 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80887.log 2024-04-25T09:05:23.960 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42499.log: 88.8% 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44038.log.gz 2024-04-25T09:05:23.965 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45099.log 2024-04-25T09:05:23.970 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.80887.log: -- replaced with /var/log/ceph/ceph-client.admin.42499.log.gz 2024-04-25T09:05:23.974 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56321.log 2024-04-25T09:05:23.977 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45099.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.45099.log.gz 2024-04-25T09:05:23.978 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.80887.log.gz 2024-04-25T09:05:23.987 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55882.log 2024-04-25T09:05:23.988 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56321.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56321.log.gz 2024-04-25T09:05:23.996 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64031.log 2024-04-25T09:05:24.005 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79133.log 2024-04-25T09:05:24.009 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.64031.log: 87.5% 89.4% -- replaced with /var/log/ceph/ceph-client.admin.55882.log.gz 2024-04-25T09:05:24.017 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79343.log 2024-04-25T09:05:24.018 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.79133.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.79133.log.gz 2024-04-25T09:05:24.019 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.64031.log.gz 2024-04-25T09:05:24.026 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85437.log 2024-04-25T09:05:24.028 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.79343.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.79343.log.gz 2024-04-25T09:05:24.035 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47513.log 2024-04-25T09:05:24.045 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.85437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90051.log 2024-04-25T09:05:24.046 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47513.log: 88.4% 88.5% -- replaced with /var/log/ceph/ceph-client.admin.85437.log.gz 2024-04-25T09:05:24.053 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61736.log 2024-04-25T09:05:24.058 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.90051.log: -- replaced with /var/log/ceph/ceph-client.admin.47513.log.gz 2024-04-25T09:05:24.062 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94774.log 2024-04-25T09:05:24.066 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.61736.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.90051.log.gz 2024-04-25T09:05:24.070 INFO:teuthology.orchestra.run.smithi043.stderr: 94.3% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-25T09:05:24.072 INFO:teuthology.orchestra.run.smithi043.stderr: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41720.log 2024-04-25T09:05:24.072 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.94774.log: -- replaced with /var/log/ceph/ceph-client.admin.61736.log.gz 2024-04-25T09:05:24.072 INFO:teuthology.orchestra.run.smithi043.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94774.log.gz 2024-04-25T09:05:24.076 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95890.log 2024-04-25T09:05:24.076 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41720.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41720.log.gz 2024-04-25T09:05:24.076 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56050.log 2024-04-25T09:05:24.076 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.95890.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.95890.log.gz 2024-04-25T09:05:24.076 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84824.log 2024-04-25T09:05:24.076 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56050.log: 94.3% 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56050.log.gz 2024-04-25T09:05:24.076 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-25T09:05:24.077 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56363.log 2024-04-25T09:05:24.078 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.84824.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.84824.log.gz 2024-04-25T09:05:24.078 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47851.log 2024-04-25T09:05:24.078 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58197.log 2024-04-25T09:05:24.079 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.56363.log: /var/log/ceph/ceph-client.admin.47851.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.56363.log.gz 2024-04-25T09:05:24.079 INFO:teuthology.orchestra.run.smithi043.stderr: 88.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61820.log 2024-04-25T09:05:24.079 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47851.log.gz 2024-04-25T09:05:24.082 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.58197.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.58197.log.gz 2024-04-25T09:05:24.082 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42415.log 2024-04-25T09:05:24.082 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.61820.log: -- /var/log/ceph/ceph-client.admin.60873.log 2024-04-25T09:05:24.083 INFO:teuthology.orchestra.run.smithi043.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.61820.log.gz 2024-04-25T09:05:24.083 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.42415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61083.log 2024-04-25T09:05:24.083 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.60873.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42415.log.gz 2024-04-25T09:05:24.083 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58785.log 2024-04-25T09:05:24.083 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60873.log.gz 2024-04-25T09:05:24.083 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.61083.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.61083.log.gz 2024-04-25T09:05:24.089 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40507.log 2024-04-25T09:05:24.089 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47683.log 2024-04-25T09:05:24.089 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.58785.log: /var/log/ceph/ceph-client.admin.40507.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.58785.log.gz 2024-04-25T09:05:24.089 INFO:teuthology.orchestra.run.smithi043.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40507.log.gz 2024-04-25T09:05:24.089 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78019.log 2024-04-25T09:05:24.089 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63325.log 2024-04-25T09:05:24.090 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.47683.log: /var/log/ceph/ceph-client.admin.78019.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47683.log.gz 2024-04-25T09:05:24.090 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100429.log 2024-04-25T09:05:24.090 INFO:teuthology.orchestra.run.smithi043.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.78019.log.gz 2024-04-25T09:05:24.090 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.63325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95255.log 2024-04-25T09:05:24.090 INFO:teuthology.orchestra.run.smithi043.stderr: 88.7%/var/log/ceph/ceph-client.admin.100429.log: -- replaced with /var/log/ceph/ceph-client.admin.63325.log.gz 2024-04-25T09:05:24.090 INFO:teuthology.orchestra.run.smithi043.stderr: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79825.log 2024-04-25T09:05:24.090 INFO:teuthology.orchestra.run.smithi043.stderr: -- replaced with /var/log/ceph/ceph-client.admin.100429.log.gz 2024-04-25T09:05:24.090 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44080.log 2024-04-25T09:05:24.091 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.95255.log: /var/log/ceph/ceph-client.admin.79825.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.79825.log.gz 2024-04-25T09:05:24.091 INFO:teuthology.orchestra.run.smithi043.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.95255.log.gz 2024-04-25T09:05:24.091 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58479.log 2024-04-25T09:05:24.091 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55561.log 2024-04-25T09:05:24.091 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44080.log: /var/log/ceph/ceph-client.admin.58479.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44080.log.gz 2024-04-25T09:05:24.094 INFO:teuthology.orchestra.run.smithi043.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58479.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60999.log 2024-04-25T09:05:24.094 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-04-25T09:05:24.094 INFO:teuthology.orchestra.run.smithi043.stderr:gzip/var/log/ceph/ceph-client.admin.55561.log: -5 --verbose -- /var/log/ceph/ceph-client.0.81338.log 2024-04-25T09:05:24.094 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.60999.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.55561.log.gz 2024-04-25T09:05:24.094 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55519.log 2024-04-25T09:05:24.095 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.0.81338.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60999.log.gz 2024-04-25T09:05:24.095 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44122.log 2024-04-25T09:05:24.095 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.55519.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.55519.log.gz 2024-04-25T09:05:24.095 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.86248.log 2024-04-25T09:05:24.095 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44122.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44122.log.gz 2024-04-25T09:05:24.095 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95213.log 2024-04-25T09:05:24.096 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.86248.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.86248.log.gz 2024-04-25T09:05:24.096 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62381.log 2024-04-25T09:05:24.097 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.95213.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95213.log.gz 2024-04-25T09:05:24.097 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45999.log 2024-04-25T09:05:24.097 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.62381.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62381.log.gz 2024-04-25T09:05:24.098 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46975.log 2024-04-25T09:05:24.098 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.45999.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45999.log.gz 2024-04-25T09:05:24.098 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62087.log 2024-04-25T09:05:24.099 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46975.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46975.log.gz 2024-04-25T09:05:24.099 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44206.log 2024-04-25T09:05:24.100 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.62087.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62087.log.gz 2024-04-25T09:05:24.100 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46933.log 2024-04-25T09:05:24.100 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.44206.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44206.log.gz 2024-04-25T09:05:24.101 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41504.log 2024-04-25T09:05:24.102 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46933.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46933.log.gz 2024-04-25T09:05:24.102 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85094.log 2024-04-25T09:05:24.103 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.41504.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41504.log.gz 2024-04-25T09:05:24.103 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.43668.log 2024-04-25T09:05:24.104 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.85094.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.85094.log.gz 2024-04-25T09:05:24.104 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94942.log 2024-04-25T09:05:24.105 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.2.43668.log: 91.0% -- replaced with /var/log/ceph/ceph-client.2.43668.log.gz 2024-04-25T09:05:24.106 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48520.log 2024-04-25T09:05:24.106 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.94942.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.94942.log.gz 2024-04-25T09:05:24.107 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94411.log 2024-04-25T09:05:24.107 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.48520.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48520.log.gz 2024-04-25T09:05:24.108 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62171.log 2024-04-25T09:05:24.108 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.94411.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.94411.log.gz 2024-04-25T09:05:24.109 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85052.log 2024-04-25T09:05:24.110 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.62171.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.62171.log.gz 2024-04-25T09:05:24.110 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46183.log 2024-04-25T09:05:24.111 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.85052.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.85052.log.gz 2024-04-25T09:05:24.111 INFO:teuthology.orchestra.run.smithi043.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95722.log 2024-04-25T09:05:24.112 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.46183.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46183.log.gz 2024-04-25T09:05:24.113 INFO:teuthology.orchestra.run.smithi043.stderr:/var/log/ceph/ceph-client.admin.95722.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95722.log.gz 2024-04-25T09:05:24.659 INFO:teuthology.orchestra.run.smithi043.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.0.86573.log.gz 2024-04-25T09:05:25.184 INFO:teuthology.orchestra.run.smithi043.stderr: 93.8% -- replaced with /var/log/ceph/ceph-client.0.81338.log.gz 2024-04-25T09:05:25.446 INFO:teuthology.orchestra.run.smithi043.stderr: 94.1% -- replaced with /var/log/ceph/ceph-client.0.57915.log.gz 2024-04-25T09:05:26.099 INFO:teuthology.orchestra.run.smithi043.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-25T09:05:27.329 INFO:teuthology.orchestra.run.smithi043.stderr: 93.6% -- replaced with /var/log/ceph/ceph-client.0.48779.log.gz 2024-04-25T09:05:28.919 INFO:teuthology.orchestra.run.smithi043.stderr: 93.5% -- replaced with /var/log/ceph/ceph-client.0.67107.log.gz 2024-04-25T09:05:42.927 INFO:teuthology.orchestra.run.smithi193.stderr: 93.7% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-25T09:05:42.929 INFO:teuthology.orchestra.run.smithi193.stderr: 2024-04-25T09:05:42.929 INFO:teuthology.orchestra.run.smithi193.stderr:real 0m21.946s 2024-04-25T09:05:42.929 INFO:teuthology.orchestra.run.smithi193.stderr:user 0m32.993s 2024-04-25T09:05:42.929 INFO:teuthology.orchestra.run.smithi193.stderr:sys 0m1.043s 2024-04-25T09:05:44.751 INFO:teuthology.orchestra.run.smithi043.stderr: 93.8% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-25T09:05:44.753 INFO:teuthology.orchestra.run.smithi043.stderr: 2024-04-25T09:05:44.754 INFO:teuthology.orchestra.run.smithi043.stderr:real 0m23.811s 2024-04-25T09:05:44.754 INFO:teuthology.orchestra.run.smithi043.stderr:user 0m56.311s 2024-04-25T09:05:44.754 INFO:teuthology.orchestra.run.smithi043.stderr:sys 0m2.035s 2024-04-25T09:05:44.754 INFO:tasks.ceph:Archiving logs... 2024-04-25T09:05:44.755 DEBUG:teuthology.misc:Transferring archived files from smithi043:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673004/remote/smithi043/log 2024-04-25T09:05:44.756 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T09:05:52.118 DEBUG:teuthology.misc:Transferring archived files from smithi193:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673004/remote/smithi193/log 2024-04-25T09:05:52.119 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T09:05:56.704 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T09:05:56.714 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T09:05:56.714 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T09:05:56.717 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T09:05:56.818 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T09:05:56.827 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T09:05:56.828 DEBUG:teuthology.orchestra.run.smithi043:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T09:05:56.831 DEBUG:teuthology.orchestra.run.smithi193:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T09:05:56.846 INFO:teuthology.orchestra.run.smithi043.stderr:bash: line 1: ntpq: command not found 2024-04-25T09:05:56.846 INFO:teuthology.orchestra.run.smithi193.stderr:bash: line 1: ntpq: command not found 2024-04-25T09:05:56.850 INFO:teuthology.orchestra.run.smithi193.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T09:05:56.851 INFO:teuthology.orchestra.run.smithi193.stdout:=============================================================================== 2024-04-25T09:05:56.851 INFO:teuthology.orchestra.run.smithi193.stdout:^- hv01.front.sepia.ceph.com 2 6 377 15 +1876us[+1876us] +/- 662ms 2024-04-25T09:05:56.851 INFO:teuthology.orchestra.run.smithi193.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 15 -716us[ -716us] +/- 81ms 2024-04-25T09:05:56.851 INFO:teuthology.orchestra.run.smithi193.stdout:^* hv03.front.sepia.ceph.com 3 6 377 16 +618us[ +628us] +/- 77ms 2024-04-25T09:05:56.851 INFO:teuthology.orchestra.run.smithi193.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:05:56.853 INFO:teuthology.orchestra.run.smithi043.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T09:05:56.853 INFO:teuthology.orchestra.run.smithi043.stdout:=============================================================================== 2024-04-25T09:05:56.853 INFO:teuthology.orchestra.run.smithi043.stdout:^- hv01.front.sepia.ceph.com 2 6 377 20 +1936us[+1936us] +/- 662ms 2024-04-25T09:05:56.853 INFO:teuthology.orchestra.run.smithi043.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 18 -646us[ -647us] +/- 81ms 2024-04-25T09:05:56.853 INFO:teuthology.orchestra.run.smithi043.stdout:^* hv03.front.sepia.ceph.com 3 6 377 18 +693us[ +692us] +/- 77ms 2024-04-25T09:05:56.853 INFO:teuthology.orchestra.run.smithi043.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:05:56.854 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T09:05:56.866 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T09:05:56.866 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T09:05:56.881 DEBUG:teuthology.orchestra.run.smithi043:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T09:05:56.986 DEBUG:teuthology.orchestra.run.smithi193:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T09:05:57.051 DEBUG:teuthology.orchestra.run.smithi043:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T09:05:57.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:05:57.082 DEBUG:teuthology.orchestra.run.smithi193:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T09:05:57.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:05:57.110 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T09:05:57.120 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T09:05:57.132 INFO:teuthology.task.internal:Duration was 1074.549201 seconds 2024-04-25T09:05:57.132 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T09:05:57.147 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T09:05:57.147 DEBUG:teuthology.orchestra.run.smithi043:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T09:05:57.150 DEBUG:teuthology.orchestra.run.smithi193:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T09:05:57.187 INFO:teuthology.orchestra.run.smithi043.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T09:05:57.189 INFO:teuthology.orchestra.run.smithi193.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T09:05:57.420 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T09:05:57.420 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi043.front.sepia.ceph.com 2024-04-25T09:05:57.421 DEBUG:teuthology.orchestra.run.smithi043:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T09:05:57.481 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi193.front.sepia.ceph.com 2024-04-25T09:05:57.481 DEBUG:teuthology.orchestra.run.smithi193:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T09:05:57.516 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T09:05:57.517 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T09:05:57.523 DEBUG:teuthology.orchestra.run.smithi193:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T09:05:57.583 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T09:05:57.583 DEBUG:teuthology.orchestra.run.smithi043:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T09:05:57.589 DEBUG:teuthology.orchestra.run.smithi193:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T09:05:57.728 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T09:05:57.738 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T09:05:57.738 DEBUG:teuthology.orchestra.run.smithi043:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T09:05:57.762 DEBUG:teuthology.orchestra.run.smithi193:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T09:05:57.786 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T09:05:57.796 DEBUG:teuthology.orchestra.run.smithi043:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T09:05:57.804 DEBUG:teuthology.orchestra.run.smithi193:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T09:05:57.825 INFO:teuthology.orchestra.run.smithi043.stdout:kernel.core_pattern = core 2024-04-25T09:05:57.848 INFO:teuthology.orchestra.run.smithi193.stdout:kernel.core_pattern = core 2024-04-25T09:05:57.860 DEBUG:teuthology.orchestra.run.smithi043:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T09:05:57.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:05:57.892 DEBUG:teuthology.orchestra.run.smithi193:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T09:05:57.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:05:57.915 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T09:05:57.925 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T09:05:57.926 DEBUG:teuthology.misc:Transferring archived files from smithi043:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673004/remote/smithi043 2024-04-25T09:05:57.926 DEBUG:teuthology.orchestra.run.smithi043:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T09:05:57.969 DEBUG:teuthology.misc:Transferring archived files from smithi193:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673004/remote/smithi193 2024-04-25T09:05:57.970 DEBUG:teuthology.orchestra.run.smithi193:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T09:05:58.002 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T09:05:58.002 DEBUG:teuthology.orchestra.run.smithi043:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T09:05:58.005 DEBUG:teuthology.orchestra.run.smithi193:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T09:05:58.050 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T09:05:58.064 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T09:05:58.064 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T09:05:58.080 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T09:05:58.080 DEBUG:teuthology.orchestra.run.smithi043:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T09:05:58.082 DEBUG:teuthology.orchestra.run.smithi193:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T09:05:58.095 INFO:teuthology.orchestra.run.smithi043.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 09:05 /home/ubuntu/cephtest 2024-04-25T09:05:58.105 INFO:teuthology.orchestra.run.smithi193.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 09:05 /home/ubuntu/cephtest 2024-04-25T09:05:58.106 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T09:05:58.118 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T09:05:58.149 INFO:teuthology.nuke:Checking targets against current locks 2024-04-25T09:05:58.177 DEBUG:teuthology.nuke:shortname: smithi043 2024-04-25T09:05:58.177 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T09:05:58.202 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi043.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673004', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 08:42:42.018161', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAlXJxRzvrFjsp6Th37vUptyCRLd/bRZfLL7KIsn9/jD51iOJAApGr4gLlNVDsheg40xYpAol3+nyQCi2TiVVpk='} 2024-04-25T09:05:58.206 DEBUG:teuthology.nuke:shortname: smithi193 2024-04-25T09:05:58.206 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T09:05:58.230 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi193.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_08:15:18-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673004', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 08:42:42.016287', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPGxWHh2OQKftzsvA8ZjW49N0cBd2B0i2ghi1EO18C1qyJQn5z1JrwJ7tqcpAbfHh2zLyZr2NhN8MBszz3zzeIc='} 2024-04-25T09:05:58.257 INFO:teuthology.orchestra.console.smithi043:Power off 2024-04-25T09:05:58.258 DEBUG:teuthology.orchestra.console.smithi043:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T09:05:58.282 DEBUG:teuthology.orchestra.console.smithi043:power off output: Chassis Power Control: Down/Off 2024-04-25T09:05:58.282 DEBUG:teuthology.orchestra.console.smithi043:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T09:05:58.296 INFO:teuthology.orchestra.console.smithi193:Power off 2024-04-25T09:05:58.297 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-25T09:05:58.307 DEBUG:teuthology.orchestra.console.smithi043:check power output: Chassis Power is on 2024-04-25T09:05:58.320 DEBUG:teuthology.orchestra.console.smithi193:power off output: Chassis Power Control: Down/Off 2024-04-25T09:05:58.320 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T09:05:58.344 DEBUG:teuthology.orchestra.console.smithi193:check power output: Chassis Power is on 2024-04-25T09:06:02.308 DEBUG:teuthology.orchestra.console.smithi043:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T09:06:02.345 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T09:06:02.419 DEBUG:teuthology.orchestra.console.smithi043:check power output: Chassis Power is on 2024-04-25T09:06:02.455 DEBUG:teuthology.orchestra.console.smithi193:check power output: Chassis Power is on 2024-04-25T09:06:06.421 DEBUG:teuthology.orchestra.console.smithi043:pexpect command: ipmitool -H smithi043.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T09:06:06.455 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T09:06:06.532 DEBUG:teuthology.orchestra.console.smithi043:check power output: Chassis Power is off 2024-04-25T09:06:06.564 DEBUG:teuthology.orchestra.console.smithi193:check power output: Chassis Power is on 2024-04-25T09:06:06.634 INFO:teuthology.orchestra.console.smithi043:Power off completed 2024-04-25T09:06:10.566 DEBUG:teuthology.orchestra.console.smithi193:pexpect command: ipmitool -H smithi193.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-25T09:06:10.676 DEBUG:teuthology.orchestra.console.smithi193:check power output: Chassis Power is off 2024-04-25T09:06:10.777 INFO:teuthology.orchestra.console.smithi193:Power off completed 2024-04-25T09:06:10.882 INFO:teuthology.run:Summary data: description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v2 tasks/scrub} duration: 1074.5492007732391 failure_reason: 'Test failure: test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls)' flavor: default owner: scheduled_neesingh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=da31073afa5e462f9cff1c4d746c5807 status: fail success: false 2024-04-25T09:06:10.882 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T09:06:10.975 INFO:teuthology.run:FAIL